2024-08-31T05:46:41.274 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-08-31T05:46:41.301 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-08-31T05:46:41.395 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/mchangir-2024-08-30_08:30:37-fs-wip-mchangir-testing-20240828.085847-main-debug-distro-default-smithi/7881377 branch: wip-mchangir-testing-20240828.085847-main-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/postgres}} email: mchangir@redhat.com first_in_suite: false job_id: '7881377' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: mchangir-2024-08-30_08:30:37-fs-wip-mchangir-testing-20240828.085847-main-debug-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-mchangir-testing-20240828.085847-main-debug ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug cephsqlite: 20 mgr/snap_schedule/allow_m_granularity: true mgr/snap_schedule/dump_on_update: true mgr/snap_schedule/log_level: debug mon: mon op complaint time: 120 mon warn on pool no app: false osd: osd op complaint time: 180 conf: global: ms die on skipped message: false mds: mds_bal_replicate_threshold: 1 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request mgr-modules: - snap_schedule sha1: d19e6118f7015f0047264006cb0c873528eb3ae6 subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: d19e6118f7015f0047264006cb0c873528eb3ae6 kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs mntopts: - ms_mode=legacy - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-mchangir-testing-20240828.085847-main-debug sha1: d19e6118f7015f0047264006cb0c873528eb3ae6 owner: scheduled_mchangir@teuthology priority: 150 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 5413 sha1: d19e6118f7015f0047264006cb0c873528eb3ae6 sleep_before_teardown: 0 subset: 347/512 suite: fs suite_branch: wip-mchangir-testing-20240828.085847-main-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_d19e6118f7015f0047264006cb0c873528eb3ae6/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: d19e6118f7015f0047264006cb0c873528eb3ae6 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi016.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE7PzcyabzZsim3YS72IKIsI1feVox+080QVSylG9r2Vjui/VQiplxGVqAQV1IApkuBRew6gbk6HOHAtr+gS0z0= smithi155.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPKwLbwxTvM/xybMKeqsfANW3ttIawlv6X5sCo62OV+1tZkKxI0sUMWRwBOGKW3p29bVEXWGzTzJmKmmKFyhlL8= smithi184.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFNA1f3J0RSyn4jZEQpHDs5zkXhFK3aqKYUOjlO0Wz42boZhE2ONxtZwes49smHPn/1jVdlwsOTT4E92p14DCU0= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-08-30_08:30:37 tube: smithi user: mchangir verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-08-31T05:46:41.395 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_d19e6118f7015f0047264006cb0c873528eb3ae6/qa; will attempt to use it 2024-08-31T05:46:41.396 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_d19e6118f7015f0047264006cb0c873528eb3ae6/qa/tasks 2024-08-31T05:46:41.396 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-08-31T05:46:41.397 INFO:teuthology.task.internal:Checking packages... 2024-08-31T05:46:41.419 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd19e6118f7015f0047264006cb0c873528eb3ae6' 2024-08-31T05:46:41.419 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-08-31T05:46:41.419 INFO:teuthology.packaging:ref: None 2024-08-31T05:46:41.419 INFO:teuthology.packaging:tag: None 2024-08-31T05:46:41.419 INFO:teuthology.packaging:branch: wip-mchangir-testing-20240828.085847-main-debug 2024-08-31T05:46:41.419 INFO:teuthology.packaging:sha1: d19e6118f7015f0047264006cb0c873528eb3ae6 2024-08-31T05:46:41.420 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-mchangir-testing-20240828.085847-main-debug 2024-08-31T05:46:41.561 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4560.gd19e6118 2024-08-31T05:46:41.563 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-08-31T05:46:41.569 INFO:teuthology.task.internal:no buildpackages task found 2024-08-31T05:46:41.569 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-08-31T05:46:41.626 INFO:teuthology.task.internal:Saving configuration 2024-08-31T05:46:41.649 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-08-31T05:46:41.656 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-08-31T05:46:41.680 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-08-30_08:30:37-fs-wip-mchangir-testing-20240828.085847-main-debug-distro-default-smithi/7881377', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-31 05:41:36.381824', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE7PzcyabzZsim3YS72IKIsI1feVox+080QVSylG9r2Vjui/VQiplxGVqAQV1IApkuBRew6gbk6HOHAtr+gS0z0='} 2024-08-31T05:46:41.700 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi155.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-08-30_08:30:37-fs-wip-mchangir-testing-20240828.085847-main-debug-distro-default-smithi/7881377', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-31 05:41:36.385617', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPKwLbwxTvM/xybMKeqsfANW3ttIawlv6X5sCo62OV+1tZkKxI0sUMWRwBOGKW3p29bVEXWGzTzJmKmmKFyhlL8='} 2024-08-31T05:46:41.723 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi184.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-08-30_08:30:37-fs-wip-mchangir-testing-20240828.085847-main-debug-distro-default-smithi/7881377', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-31 05:41:36.383839', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFNA1f3J0RSyn4jZEQpHDs5zkXhFK3aqKYUOjlO0Wz42boZhE2ONxtZwes49smHPn/1jVdlwsOTT4E92p14DCU0='} 2024-08-31T05:46:41.723 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-08-31T05:46:41.729 INFO:teuthology.task.internal:roles: ubuntu@smithi016.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-08-31T05:46:41.729 INFO:teuthology.task.internal:roles: ubuntu@smithi155.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-08-31T05:46:41.729 INFO:teuthology.task.internal:roles: ubuntu@smithi184.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-08-31T05:46:41.729 INFO:teuthology.run_tasks:Running task console_log... 2024-08-31T05:46:41.809 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fcbe06bfac0>, signals=[15]) 2024-08-31T05:46:41.809 INFO:teuthology.run_tasks:Running task internal.connect... 2024-08-31T05:46:41.815 INFO:teuthology.task.internal:Opening connections... 2024-08-31T05:46:41.815 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi016.front.sepia.ceph.com 2024-08-31T05:46:41.816 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-31T05:46:41.895 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi155.front.sepia.ceph.com 2024-08-31T05:46:41.896 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-31T05:46:41.971 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi184.front.sepia.ceph.com 2024-08-31T05:46:41.972 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-31T05:46:42.048 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-08-31T05:46:42.055 DEBUG:teuthology.orchestra.run.smithi016:> uname -m 2024-08-31T05:46:42.073 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64 2024-08-31T05:46:42.073 DEBUG:teuthology.orchestra.run.smithi016:> cat /etc/os-release 2024-08-31T05:46:42.131 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="CentOS Stream" 2024-08-31T05:46:42.131 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="9" 2024-08-31T05:46:42.131 INFO:teuthology.orchestra.run.smithi016.stdout:ID="centos" 2024-08-31T05:46:42.131 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE="rhel fedora" 2024-08-31T05:46:42.131 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="9" 2024-08-31T05:46:42.131 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM_ID="platform:el9" 2024-08-31T05:46:42.132 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="CentOS Stream 9" 2024-08-31T05:46:42.132 INFO:teuthology.orchestra.run.smithi016.stdout:ANSI_COLOR="0;31" 2024-08-31T05:46:42.132 INFO:teuthology.orchestra.run.smithi016.stdout:LOGO="fedora-logo-icon" 2024-08-31T05:46:42.132 INFO:teuthology.orchestra.run.smithi016.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-08-31T05:46:42.132 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://centos.org/" 2024-08-31T05:46:42.132 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-08-31T05:46:42.132 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-08-31T05:46:42.132 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-08-31T05:46:42.133 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2024-08-31T05:46:42.152 DEBUG:teuthology.orchestra.run.smithi155:> uname -m 2024-08-31T05:46:42.170 INFO:teuthology.orchestra.run.smithi155.stdout:x86_64 2024-08-31T05:46:42.171 DEBUG:teuthology.orchestra.run.smithi155:> cat /etc/os-release 2024-08-31T05:46:42.229 INFO:teuthology.orchestra.run.smithi155.stdout:NAME="CentOS Stream" 2024-08-31T05:46:42.229 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION="9" 2024-08-31T05:46:42.229 INFO:teuthology.orchestra.run.smithi155.stdout:ID="centos" 2024-08-31T05:46:42.229 INFO:teuthology.orchestra.run.smithi155.stdout:ID_LIKE="rhel fedora" 2024-08-31T05:46:42.229 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION_ID="9" 2024-08-31T05:46:42.229 INFO:teuthology.orchestra.run.smithi155.stdout:PLATFORM_ID="platform:el9" 2024-08-31T05:46:42.230 INFO:teuthology.orchestra.run.smithi155.stdout:PRETTY_NAME="CentOS Stream 9" 2024-08-31T05:46:42.230 INFO:teuthology.orchestra.run.smithi155.stdout:ANSI_COLOR="0;31" 2024-08-31T05:46:42.230 INFO:teuthology.orchestra.run.smithi155.stdout:LOGO="fedora-logo-icon" 2024-08-31T05:46:42.230 INFO:teuthology.orchestra.run.smithi155.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-08-31T05:46:42.230 INFO:teuthology.orchestra.run.smithi155.stdout:HOME_URL="https://centos.org/" 2024-08-31T05:46:42.230 INFO:teuthology.orchestra.run.smithi155.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-08-31T05:46:42.230 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-08-31T05:46:42.230 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-08-31T05:46:42.231 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server 2024-08-31T05:46:42.252 DEBUG:teuthology.orchestra.run.smithi184:> uname -m 2024-08-31T05:46:42.270 INFO:teuthology.orchestra.run.smithi184.stdout:x86_64 2024-08-31T05:46:42.270 DEBUG:teuthology.orchestra.run.smithi184:> cat /etc/os-release 2024-08-31T05:46:42.328 INFO:teuthology.orchestra.run.smithi184.stdout:NAME="CentOS Stream" 2024-08-31T05:46:42.329 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION="9" 2024-08-31T05:46:42.329 INFO:teuthology.orchestra.run.smithi184.stdout:ID="centos" 2024-08-31T05:46:42.329 INFO:teuthology.orchestra.run.smithi184.stdout:ID_LIKE="rhel fedora" 2024-08-31T05:46:42.329 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION_ID="9" 2024-08-31T05:46:42.329 INFO:teuthology.orchestra.run.smithi184.stdout:PLATFORM_ID="platform:el9" 2024-08-31T05:46:42.329 INFO:teuthology.orchestra.run.smithi184.stdout:PRETTY_NAME="CentOS Stream 9" 2024-08-31T05:46:42.329 INFO:teuthology.orchestra.run.smithi184.stdout:ANSI_COLOR="0;31" 2024-08-31T05:46:42.329 INFO:teuthology.orchestra.run.smithi184.stdout:LOGO="fedora-logo-icon" 2024-08-31T05:46:42.329 INFO:teuthology.orchestra.run.smithi184.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-08-31T05:46:42.329 INFO:teuthology.orchestra.run.smithi184.stdout:HOME_URL="https://centos.org/" 2024-08-31T05:46:42.329 INFO:teuthology.orchestra.run.smithi184.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-08-31T05:46:42.329 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-08-31T05:46:42.329 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-08-31T05:46:42.330 INFO:teuthology.lock.ops:Updating smithi184.front.sepia.ceph.com on lock server 2024-08-31T05:46:42.352 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-08-31T05:46:42.361 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-08-31T05:46:42.367 INFO:teuthology.task.internal:Checking for old test directory... 2024-08-31T05:46:42.367 DEBUG:teuthology.orchestra.run.smithi016:> test '!' -e /home/ubuntu/cephtest 2024-08-31T05:46:42.370 DEBUG:teuthology.orchestra.run.smithi155:> test '!' -e /home/ubuntu/cephtest 2024-08-31T05:46:42.372 DEBUG:teuthology.orchestra.run.smithi184:> test '!' -e /home/ubuntu/cephtest 2024-08-31T05:46:42.389 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-08-31T05:46:42.395 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-08-31T05:46:42.395 DEBUG:teuthology.orchestra.run.smithi016:> test -z $(ls -A /var/lib/ceph) 2024-08-31T05:46:42.427 DEBUG:teuthology.orchestra.run.smithi155:> test -z $(ls -A /var/lib/ceph) 2024-08-31T05:46:42.429 DEBUG:teuthology.orchestra.run.smithi184:> test -z $(ls -A /var/lib/ceph) 2024-08-31T05:46:42.524 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-08-31T05:46:42.597 INFO:teuthology.run_tasks:Running task kernel... 2024-08-31T05:46:42.611 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-08-31T05:46:42.612 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-08-31T05:46:42.612 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-08-31T05:46:42.612 DEBUG:teuthology.orchestra.run.smithi016:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-31T05:46:42.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-31T05:46:42.630 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-08-31T05:46:42.800 DEBUG:teuthology.task.kernel:sha1 for client.0 is 8ad8b8b160987a14eb199bb67cc00e23a08d43f6 2024-08-31T05:46:42.801 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "8ad8b8b160987a14eb199bb67cc00e23a08d43f6"... 2024-08-31T05:46:42.801 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2024-08-31T05:46:42.818 INFO:teuthology.orchestra.run.smithi016.stdout:5.14.0-503.el9.x86_64 2024-08-31T05:46:42.819 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 8ad8b8b160987a14eb199bb67cc00e23a08d43f6 2024-08-31T05:46:42.819 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-08-31T05:46:42.819 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum upgrade -y linux-firmware 2024-08-31T05:46:45.233 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.2 MB 00:00 2024-08-31T05:46:47.200 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - AppStream 25 MB/s | 20 MB 00:00 2024-08-31T05:46:51.751 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.6 MB 00:00 2024-08-31T05:46:53.968 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - Extras packages 19 kB/s | 18 kB 00:00 2024-08-31T05:46:55.131 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 22 MB/s | 22 MB 00:01 2024-08-31T05:47:00.695 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-08-31T05:47:02.894 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-08-31T05:47:02.895 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2024-08-31T05:47:02.895 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-08-31T05:47:03.000 INFO:teuthology.task.kernel:Downloading kernel 8ad8b8b160987a14eb199bb67cc00e23a08d43f6 on client.0... 2024-08-31T05:47:03.001 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8ad8b8b160987a14eb199bb67cc00e23a08d43f6 2024-08-31T05:47:03.150 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/8ad8b8b160987a14eb199bb67cc00e23a08d43f6/centos/9/flavors/default/x86_64/ 2024-08-31T05:47:03.151 DEBUG:teuthology.orchestra.run.smithi016:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g8ad8b8b16098-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://1.chacra.ceph.com/r/kernel/testing/8ad8b8b160987a14eb199bb67cc00e23a08d43f6/centos/9/flavors/default/x86_64/ --input-file=- 2024-08-31T05:47:03.153 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi016.front.sepia.ceph.com to complete... 2024-08-31T05:47:09.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31 05:47:09 URL:https://1.chacra.ceph.com/r/kernel/testing/8ad8b8b160987a14eb199bb67cc00e23a08d43f6/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g8ad8b8b16098-1.x86_64.rpm [359646365/359646365] -> "/tmp/kernel.x86_64.rpm" [1] 2024-08-31T05:47:09.446 INFO:teuthology.orchestra.run.smithi016.stderr:FINISHED --2024-08-31 05:47:09-- 2024-08-31T05:47:09.447 INFO:teuthology.orchestra.run.smithi016.stderr:Total wall clock time: 6.3s 2024-08-31T05:47:09.447 INFO:teuthology.orchestra.run.smithi016.stderr:Downloaded: 1 files, 343M in 6.0s (57.2 MB/s) 2024-08-31T05:47:09.453 INFO:teuthology.task.kernel:Installing kernel 8ad8b8b160987a14eb199bb67cc00e23a08d43f6 on client.0... 2024-08-31T05:47:09.453 DEBUG:teuthology.orchestra.run.smithi016:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-08-31T05:47:11.661 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying... ######################################## 2024-08-31T05:47:11.804 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing... ######################################## 2024-08-31T05:47:11.910 INFO:teuthology.orchestra.run.smithi016.stdout:Updating / installing... 2024-08-31T05:47:20.232 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-6.11.0_rc5_g8ad8b8b16098-1 ######################################## 2024-08-31T05:48:00.812 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi016.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-08-31T05:48:00.813 DEBUG:teuthology.orchestra.run.smithi016:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-08-31T05:48:00.917 INFO:teuthology.orchestra.run.smithi016.stdout:/boot/System.map-6.11.0-rc5-g8ad8b8b16098 2024-08-31T05:48:00.917 INFO:teuthology.orchestra.run.smithi016.stdout:/boot/config-6.11.0-rc5-g8ad8b8b16098 2024-08-31T05:48:00.917 INFO:teuthology.orchestra.run.smithi016.stdout:/boot/vmlinuz-6.11.0-rc5-g8ad8b8b16098 2024-08-31T05:48:00.917 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098 2024-08-31T05:48:00.917 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/System.map 2024-08-31T05:48:00.917 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/config 2024-08-31T05:48:00.917 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel 2024-08-31T05:48:00.917 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch 2024-08-31T05:48:00.918 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86 2024-08-31T05:48:00.918 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto 2024-08-31T05:48:00.918 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-08-31T05:48:00.918 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-08-31T05:48:00.918 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-08-31T05:48:00.918 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-08-31T05:48:00.918 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-08-31T05:48:00.918 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-08-31T05:48:00.918 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-08-31T05:48:00.918 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/crc32c-intel.ko 2024-08-31T05:48:00.918 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-08-31T05:48:00.918 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-08-31T05:48:00.918 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-08-31T05:48:00.918 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-08-31T05:48:00.919 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/serpent-avx2.ko 2024-08-31T05:48:00.919 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-08-31T05:48:00.919 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-08-31T05:48:00.919 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-08-31T05:48:00.919 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-08-31T05:48:00.919 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-08-31T05:48:00.919 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-08-31T05:48:00.919 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-08-31T05:48:00.919 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/kernel 2024-08-31T05:48:00.919 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/kernel/cpu 2024-08-31T05:48:00.919 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/kernel/cpu/mce 2024-08-31T05:48:00.920 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-08-31T05:48:00.920 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/kvm 2024-08-31T05:48:00.920 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/kvm/kvm-amd.ko 2024-08-31T05:48:00.920 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/kvm/kvm-intel.ko 2024-08-31T05:48:00.920 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/kvm/kvm.ko 2024-08-31T05:48:00.920 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto 2024-08-31T05:48:00.920 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/ansi_cprng.ko 2024-08-31T05:48:00.920 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/anubis.ko 2024-08-31T05:48:00.920 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/arc4.ko 2024-08-31T05:48:00.920 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/async_tx 2024-08-31T05:48:00.920 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/async_tx/async_memcpy.ko 2024-08-31T05:48:00.920 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/async_tx/async_pq.ko 2024-08-31T05:48:00.921 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/async_tx/async_raid6_recov.ko 2024-08-31T05:48:00.921 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/async_tx/async_tx.ko 2024-08-31T05:48:00.921 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/async_tx/async_xor.ko 2024-08-31T05:48:00.921 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/async_tx/raid6test.ko 2024-08-31T05:48:00.921 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/authenc.ko 2024-08-31T05:48:00.921 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/authencesn.ko 2024-08-31T05:48:00.921 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/blake2b_generic.ko 2024-08-31T05:48:00.921 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/blowfish_common.ko 2024-08-31T05:48:00.921 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/blowfish_generic.ko 2024-08-31T05:48:00.921 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/camellia_generic.ko 2024-08-31T05:48:00.921 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/cast5_generic.ko 2024-08-31T05:48:00.921 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/cast6_generic.ko 2024-08-31T05:48:00.921 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/cast_common.ko 2024-08-31T05:48:00.922 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/ccm.ko 2024-08-31T05:48:00.922 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/cmac.ko 2024-08-31T05:48:00.922 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/crc32_generic.ko 2024-08-31T05:48:00.922 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/crypto_engine.ko 2024-08-31T05:48:00.922 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/crypto_user.ko 2024-08-31T05:48:00.922 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/cts.ko 2024-08-31T05:48:00.922 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/des_generic.ko 2024-08-31T05:48:00.922 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/dh_generic.ko 2024-08-31T05:48:00.922 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/ecc.ko 2024-08-31T05:48:00.922 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/ecdh_generic.ko 2024-08-31T05:48:00.922 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/echainiv.ko 2024-08-31T05:48:00.922 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/essiv.ko 2024-08-31T05:48:00.922 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/fcrypt.ko 2024-08-31T05:48:00.922 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/khazad.ko 2024-08-31T05:48:00.923 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/lz4.ko 2024-08-31T05:48:00.923 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/lz4hc.ko 2024-08-31T05:48:00.923 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/md4.ko 2024-08-31T05:48:00.923 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/michael_mic.ko 2024-08-31T05:48:00.923 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/pcbc.ko 2024-08-31T05:48:00.923 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/pcrypt.ko 2024-08-31T05:48:00.923 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/rmd160.ko 2024-08-31T05:48:00.923 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/seed.ko 2024-08-31T05:48:00.923 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/serpent_generic.ko 2024-08-31T05:48:00.923 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/tcrypt.ko 2024-08-31T05:48:00.923 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/tea.ko 2024-08-31T05:48:00.923 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/twofish_common.ko 2024-08-31T05:48:00.923 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/twofish_generic.ko 2024-08-31T05:48:00.923 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/vmac.ko 2024-08-31T05:48:00.924 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/wp512.ko 2024-08-31T05:48:00.924 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/xcbc.ko 2024-08-31T05:48:00.924 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/xor.ko 2024-08-31T05:48:00.924 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/xxhash_generic.ko 2024-08-31T05:48:00.924 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers 2024-08-31T05:48:00.924 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/acpi 2024-08-31T05:48:00.924 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/acpi/acpi_ipmi.ko 2024-08-31T05:48:00.924 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/acpi/acpi_pad.ko 2024-08-31T05:48:00.924 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/acpi/ec_sys.ko 2024-08-31T05:48:00.924 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/acpi/platform_profile.ko 2024-08-31T05:48:00.924 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/acpi/sbs.ko 2024-08-31T05:48:00.924 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/acpi/sbshc.ko 2024-08-31T05:48:00.924 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/acpi/video.ko 2024-08-31T05:48:00.925 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata 2024-08-31T05:48:00.925 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/acard-ahci.ko 2024-08-31T05:48:00.925 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/ahci_platform.ko 2024-08-31T05:48:00.925 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/ata_generic.ko 2024-08-31T05:48:00.925 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/libahci_platform.ko 2024-08-31T05:48:00.925 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_acpi.ko 2024-08-31T05:48:00.925 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_ali.ko 2024-08-31T05:48:00.925 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_amd.ko 2024-08-31T05:48:00.925 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_artop.ko 2024-08-31T05:48:00.925 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_atiixp.ko 2024-08-31T05:48:00.925 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_atp867x.ko 2024-08-31T05:48:00.925 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_cmd640.ko 2024-08-31T05:48:00.925 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_cmd64x.ko 2024-08-31T05:48:00.925 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_cypress.ko 2024-08-31T05:48:00.926 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_efar.ko 2024-08-31T05:48:00.926 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_hpt366.ko 2024-08-31T05:48:00.926 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_hpt37x.ko 2024-08-31T05:48:00.926 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_hpt3x2n.ko 2024-08-31T05:48:00.926 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_hpt3x3.ko 2024-08-31T05:48:00.926 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_it8213.ko 2024-08-31T05:48:00.926 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_it821x.ko 2024-08-31T05:48:00.926 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_jmicron.ko 2024-08-31T05:48:00.926 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_marvell.ko 2024-08-31T05:48:00.926 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_mpiix.ko 2024-08-31T05:48:00.926 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_netcell.ko 2024-08-31T05:48:00.926 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_ninja32.ko 2024-08-31T05:48:00.926 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_ns87410.ko 2024-08-31T05:48:00.926 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_ns87415.ko 2024-08-31T05:48:00.927 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_oldpiix.ko 2024-08-31T05:48:00.927 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_opti.ko 2024-08-31T05:48:00.927 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_optidma.ko 2024-08-31T05:48:00.927 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_pcmcia.ko 2024-08-31T05:48:00.927 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_pdc2027x.ko 2024-08-31T05:48:00.927 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-08-31T05:48:00.927 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_piccolo.ko 2024-08-31T05:48:00.927 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_rdc.ko 2024-08-31T05:48:00.927 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_sch.ko 2024-08-31T05:48:00.927 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_serverworks.ko 2024-08-31T05:48:00.927 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_sil680.ko 2024-08-31T05:48:00.927 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_sis.ko 2024-08-31T05:48:00.927 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_sl82c105.ko 2024-08-31T05:48:00.927 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_triflex.ko 2024-08-31T05:48:00.928 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_via.ko 2024-08-31T05:48:00.928 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pdc_adma.ko 2024-08-31T05:48:00.928 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_inic162x.ko 2024-08-31T05:48:00.928 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_mv.ko 2024-08-31T05:48:00.928 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_nv.ko 2024-08-31T05:48:00.928 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_promise.ko 2024-08-31T05:48:00.928 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_qstor.ko 2024-08-31T05:48:00.928 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_sil.ko 2024-08-31T05:48:00.928 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_sil24.ko 2024-08-31T05:48:00.928 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_sis.ko 2024-08-31T05:48:00.928 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_svw.ko 2024-08-31T05:48:00.928 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_sx4.ko 2024-08-31T05:48:00.928 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_uli.ko 2024-08-31T05:48:00.928 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_via.ko 2024-08-31T05:48:00.929 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_vsc.ko 2024-08-31T05:48:00.929 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/auxdisplay 2024-08-31T05:48:00.929 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/auxdisplay/ks0108.ko 2024-08-31T05:48:00.929 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/base 2024-08-31T05:48:00.929 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/base/regmap 2024-08-31T05:48:00.929 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/base/regmap/regmap-i2c.ko 2024-08-31T05:48:00.929 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/base/regmap/regmap-mmio.ko 2024-08-31T05:48:00.929 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/bcma 2024-08-31T05:48:00.929 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/bcma/bcma.ko 2024-08-31T05:48:00.929 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block 2024-08-31T05:48:00.929 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/aoe 2024-08-31T05:48:00.929 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/aoe/aoe.ko 2024-08-31T05:48:00.929 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/brd.ko 2024-08-31T05:48:00.930 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/drbd 2024-08-31T05:48:00.930 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/drbd/drbd.ko 2024-08-31T05:48:00.930 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/floppy.ko 2024-08-31T05:48:00.930 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/loop.ko 2024-08-31T05:48:00.930 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/mtip32xx 2024-08-31T05:48:00.930 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-08-31T05:48:00.930 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/nbd.ko 2024-08-31T05:48:00.930 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/null_blk 2024-08-31T05:48:00.930 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/null_blk/null_blk.ko 2024-08-31T05:48:00.930 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/pktcdvd.ko 2024-08-31T05:48:00.930 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/rbd.ko 2024-08-31T05:48:00.930 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/virtio_blk.ko 2024-08-31T05:48:00.930 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/xen-blkback 2024-08-31T05:48:00.930 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-08-31T05:48:00.931 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/xen-blkfront.ko 2024-08-31T05:48:00.931 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/zram 2024-08-31T05:48:00.931 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/zram/zram.ko 2024-08-31T05:48:00.931 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char 2024-08-31T05:48:00.931 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/hangcheck-timer.ko 2024-08-31T05:48:00.931 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/hw_random 2024-08-31T05:48:00.931 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/hw_random/amd-rng.ko 2024-08-31T05:48:00.931 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/hw_random/intel-rng.ko 2024-08-31T05:48:00.931 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-08-31T05:48:00.931 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/hw_random/via-rng.ko 2024-08-31T05:48:00.931 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/hw_random/virtio-rng.ko 2024-08-31T05:48:00.931 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/ipmi 2024-08-31T05:48:00.931 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-08-31T05:48:00.931 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-08-31T05:48:00.932 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-08-31T05:48:00.932 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/ipmi/ipmi_si.ko 2024-08-31T05:48:00.932 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-08-31T05:48:00.932 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-08-31T05:48:00.932 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/lp.ko 2024-08-31T05:48:00.932 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/mwave 2024-08-31T05:48:00.932 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/mwave/mwave.ko 2024-08-31T05:48:00.932 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/ppdev.ko 2024-08-31T05:48:00.932 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/tlclk.ko 2024-08-31T05:48:00.932 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/tpm 2024-08-31T05:48:00.932 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/tpm/tpm.ko 2024-08-31T05:48:00.932 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/tpm/tpm_atmel.ko 2024-08-31T05:48:00.932 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/tpm/tpm_infineon.ko 2024-08-31T05:48:00.932 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/tpm/tpm_nsc.ko 2024-08-31T05:48:00.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/tpm/tpm_tis.ko 2024-08-31T05:48:00.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-08-31T05:48:00.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/uv_mmtimer.ko 2024-08-31T05:48:00.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/virtio_console.ko 2024-08-31T05:48:00.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/cpufreq 2024-08-31T05:48:00.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-08-31T05:48:00.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-08-31T05:48:00.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/cpufreq/p4-clockmod.ko 2024-08-31T05:48:00.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-08-31T05:48:00.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/cpufreq/powernow-k8.ko 2024-08-31T05:48:00.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/cpufreq/speedstep-lib.ko 2024-08-31T05:48:00.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto 2024-08-31T05:48:00.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/ccp 2024-08-31T05:48:00.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-08-31T05:48:00.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/ccp/ccp.ko 2024-08-31T05:48:00.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/intel 2024-08-31T05:48:00.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/intel/qat 2024-08-31T05:48:00.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/intel/qat/qat_common 2024-08-31T05:48:00.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-08-31T05:48:00.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-08-31T05:48:00.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-08-31T05:48:00.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/padlock-aes.ko 2024-08-31T05:48:00.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/padlock-sha.ko 2024-08-31T05:48:00.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/virtio 2024-08-31T05:48:00.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-08-31T05:48:00.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/dca 2024-08-31T05:48:00.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/dca/dca.ko 2024-08-31T05:48:00.935 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/devfreq 2024-08-31T05:48:00.935 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-08-31T05:48:00.935 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/dma 2024-08-31T05:48:00.935 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/dma/dw 2024-08-31T05:48:00.935 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/dma/dw/dw_dmac.ko 2024-08-31T05:48:00.935 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/dma/ioat 2024-08-31T05:48:00.935 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/dma/ioat/ioatdma.ko 2024-08-31T05:48:00.935 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac 2024-08-31T05:48:00.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/amd64_edac.ko 2024-08-31T05:48:00.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/e752x_edac.ko 2024-08-31T05:48:00.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/edac_mce_amd.ko 2024-08-31T05:48:00.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/i3000_edac.ko 2024-08-31T05:48:00.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/i3200_edac.ko 2024-08-31T05:48:00.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/i5100_edac.ko 2024-08-31T05:48:00.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/i5400_edac.ko 2024-08-31T05:48:00.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/i7300_edac.ko 2024-08-31T05:48:00.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/i7core_edac.ko 2024-08-31T05:48:00.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/i82975x_edac.ko 2024-08-31T05:48:00.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/ie31200_edac.ko 2024-08-31T05:48:00.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/sb_edac.ko 2024-08-31T05:48:00.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/x38_edac.ko 2024-08-31T05:48:00.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/firmware 2024-08-31T05:48:00.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/firmware/edd.ko 2024-08-31T05:48:00.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/firmware/iscsi_ibft.ko 2024-08-31T05:48:00.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/gpio 2024-08-31T05:48:00.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/gpio/gpio-ich.ko 2024-08-31T05:48:00.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/gpio/gpio-viperboard.ko 2024-08-31T05:48:00.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid 2024-08-31T05:48:00.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-appleir.ko 2024-08-31T05:48:00.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-aureal.ko 2024-08-31T05:48:00.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-axff.ko 2024-08-31T05:48:00.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-dr.ko 2024-08-31T05:48:00.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-elecom.ko 2024-08-31T05:48:00.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-elo.ko 2024-08-31T05:48:00.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-emsff.ko 2024-08-31T05:48:00.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-gaff.ko 2024-08-31T05:48:00.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-gt683r.ko 2024-08-31T05:48:00.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-gyration.ko 2024-08-31T05:48:00.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-holtek-kbd.ko 2024-08-31T05:48:00.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-holtek-mouse.ko 2024-08-31T05:48:00.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-holtekff.ko 2024-08-31T05:48:00.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-hyperv.ko 2024-08-31T05:48:00.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-icade.ko 2024-08-31T05:48:00.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-keytouch.ko 2024-08-31T05:48:00.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-kye.ko 2024-08-31T05:48:00.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-lcpower.ko 2024-08-31T05:48:00.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-led.ko 2024-08-31T05:48:00.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-lenovo.ko 2024-08-31T05:48:00.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-logitech-dj.ko 2024-08-31T05:48:00.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-08-31T05:48:00.939 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-multitouch.ko 2024-08-31T05:48:00.939 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-ortek.ko 2024-08-31T05:48:00.939 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-penmount.ko 2024-08-31T05:48:00.939 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-petalynx.ko 2024-08-31T05:48:00.939 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-picolcd.ko 2024-08-31T05:48:00.939 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-pl.ko 2024-08-31T05:48:00.939 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-plantronics.ko 2024-08-31T05:48:00.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-primax.ko 2024-08-31T05:48:00.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-rmi.ko 2024-08-31T05:48:00.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat-arvo.ko 2024-08-31T05:48:00.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat-common.ko 2024-08-31T05:48:00.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat-isku.ko 2024-08-31T05:48:00.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat-kone.ko 2024-08-31T05:48:00.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-08-31T05:48:00.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat-konepure.ko 2024-08-31T05:48:00.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-08-31T05:48:00.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat-lua.ko 2024-08-31T05:48:00.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat-pyra.ko 2024-08-31T05:48:00.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat-ryos.ko 2024-08-31T05:48:00.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat-savu.ko 2024-08-31T05:48:00.941 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat.ko 2024-08-31T05:48:00.941 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-saitek.ko 2024-08-31T05:48:00.941 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-samsung.ko 2024-08-31T05:48:00.941 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-sensor-hub.ko 2024-08-31T05:48:00.941 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-sjoy.ko 2024-08-31T05:48:00.941 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-sony.ko 2024-08-31T05:48:00.941 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-speedlink.ko 2024-08-31T05:48:00.941 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-steelseries.ko 2024-08-31T05:48:00.941 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-sunplus.ko 2024-08-31T05:48:00.941 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-thrustmaster.ko 2024-08-31T05:48:00.941 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-tivo.ko 2024-08-31T05:48:00.941 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-tmff.ko 2024-08-31T05:48:00.941 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-topseed.ko 2024-08-31T05:48:00.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-twinhan.ko 2024-08-31T05:48:00.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-uclogic.ko 2024-08-31T05:48:00.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-waltop.ko 2024-08-31T05:48:00.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-wiimote.ko 2024-08-31T05:48:00.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-xinmo.ko 2024-08-31T05:48:00.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-zpff.ko 2024-08-31T05:48:00.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-zydacron.ko 2024-08-31T05:48:00.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/uhid.ko 2024-08-31T05:48:00.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/wacom.ko 2024-08-31T05:48:00.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hv 2024-08-31T05:48:00.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hv/hv_balloon.ko 2024-08-31T05:48:00.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hv/hv_utils.ko 2024-08-31T05:48:00.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hv/hv_vmbus.ko 2024-08-31T05:48:00.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon 2024-08-31T05:48:00.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/abituguru.ko 2024-08-31T05:48:00.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/abituguru3.ko 2024-08-31T05:48:00.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/acpi_power_meter.ko 2024-08-31T05:48:00.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ad7414.ko 2024-08-31T05:48:00.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ad7418.ko 2024-08-31T05:48:00.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adc128d818.ko 2024-08-31T05:48:00.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adm1025.ko 2024-08-31T05:48:00.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adm1026.ko 2024-08-31T05:48:00.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adm1029.ko 2024-08-31T05:48:00.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adm1031.ko 2024-08-31T05:48:00.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adm9240.ko 2024-08-31T05:48:00.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ads7828.ko 2024-08-31T05:48:00.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adt7410.ko 2024-08-31T05:48:00.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adt7411.ko 2024-08-31T05:48:00.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adt7462.ko 2024-08-31T05:48:00.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adt7470.ko 2024-08-31T05:48:00.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adt7475.ko 2024-08-31T05:48:00.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adt7x10.ko 2024-08-31T05:48:00.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/amc6821.ko 2024-08-31T05:48:00.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/applesmc.ko 2024-08-31T05:48:00.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/asb100.ko 2024-08-31T05:48:00.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/asc7621.ko 2024-08-31T05:48:00.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/asus_atk0110.ko 2024-08-31T05:48:00.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/atxp1.ko 2024-08-31T05:48:00.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/coretemp.ko 2024-08-31T05:48:00.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-08-31T05:48:00.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/dme1737.ko 2024-08-31T05:48:00.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ds1621.ko 2024-08-31T05:48:00.945 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ds620.ko 2024-08-31T05:48:00.945 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/emc1403.ko 2024-08-31T05:48:00.945 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/emc6w201.ko 2024-08-31T05:48:00.945 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/f71805f.ko 2024-08-31T05:48:00.945 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/f71882fg.ko 2024-08-31T05:48:00.945 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/f75375s.ko 2024-08-31T05:48:00.945 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/fam15h_power.ko 2024-08-31T05:48:00.945 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/fschmd.ko 2024-08-31T05:48:00.945 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/g760a.ko 2024-08-31T05:48:00.945 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/g762.ko 2024-08-31T05:48:00.945 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/gl518sm.ko 2024-08-31T05:48:00.945 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/gl520sm.ko 2024-08-31T05:48:00.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/hwmon-vid.ko 2024-08-31T05:48:00.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/i5500_temp.ko 2024-08-31T05:48:00.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/i5k_amb.ko 2024-08-31T05:48:00.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ibmaem.ko 2024-08-31T05:48:00.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ibmpex.ko 2024-08-31T05:48:00.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ina209.ko 2024-08-31T05:48:00.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ina2xx.ko 2024-08-31T05:48:00.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/it87.ko 2024-08-31T05:48:00.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/k10temp.ko 2024-08-31T05:48:00.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/k8temp.ko 2024-08-31T05:48:00.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lineage-pem.ko 2024-08-31T05:48:00.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm63.ko 2024-08-31T05:48:00.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm73.ko 2024-08-31T05:48:00.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm75.ko 2024-08-31T05:48:00.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm77.ko 2024-08-31T05:48:00.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm78.ko 2024-08-31T05:48:00.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm80.ko 2024-08-31T05:48:00.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm83.ko 2024-08-31T05:48:00.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm85.ko 2024-08-31T05:48:00.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm87.ko 2024-08-31T05:48:00.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm90.ko 2024-08-31T05:48:00.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm92.ko 2024-08-31T05:48:00.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm93.ko 2024-08-31T05:48:00.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm95234.ko 2024-08-31T05:48:00.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm95241.ko 2024-08-31T05:48:00.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm95245.ko 2024-08-31T05:48:00.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ltc2945.ko 2024-08-31T05:48:00.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ltc4151.ko 2024-08-31T05:48:00.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ltc4215.ko 2024-08-31T05:48:00.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ltc4222.ko 2024-08-31T05:48:00.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ltc4245.ko 2024-08-31T05:48:00.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ltc4260.ko 2024-08-31T05:48:00.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ltc4261.ko 2024-08-31T05:48:00.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/max16065.ko 2024-08-31T05:48:00.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/max1619.ko 2024-08-31T05:48:00.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/max1668.ko 2024-08-31T05:48:00.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/max197.ko 2024-08-31T05:48:00.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/max6639.ko 2024-08-31T05:48:00.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/max6650.ko 2024-08-31T05:48:00.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/max6697.ko 2024-08-31T05:48:00.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/mcp3021.ko 2024-08-31T05:48:00.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/nct6683.ko 2024-08-31T05:48:00.949 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/nct6775-core.ko 2024-08-31T05:48:00.949 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/nct6775.ko 2024-08-31T05:48:00.949 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/nct7802.ko 2024-08-31T05:48:00.949 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pc87360.ko 2024-08-31T05:48:00.949 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pc87427.ko 2024-08-31T05:48:00.949 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pcf8591.ko 2024-08-31T05:48:00.949 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus 2024-08-31T05:48:00.949 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-08-31T05:48:00.949 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-08-31T05:48:00.949 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-08-31T05:48:00.950 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/max16064.ko 2024-08-31T05:48:00.950 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/max34440.ko 2024-08-31T05:48:00.950 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/max8688.ko 2024-08-31T05:48:00.950 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-08-31T05:48:00.950 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-08-31T05:48:00.950 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-08-31T05:48:00.950 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-08-31T05:48:00.950 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-08-31T05:48:00.950 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-08-31T05:48:00.950 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/powr1220.ko 2024-08-31T05:48:00.950 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/sch5627.ko 2024-08-31T05:48:00.950 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/sch5636.ko 2024-08-31T05:48:00.950 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/sch56xx-common.ko 2024-08-31T05:48:00.951 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/sht15.ko 2024-08-31T05:48:00.951 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/sht21.ko 2024-08-31T05:48:00.951 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/shtc1.ko 2024-08-31T05:48:00.951 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/sis5595.ko 2024-08-31T05:48:00.951 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/smsc47b397.ko 2024-08-31T05:48:00.951 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/smsc47m1.ko 2024-08-31T05:48:00.951 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/smsc47m192.ko 2024-08-31T05:48:00.951 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/thmc50.ko 2024-08-31T05:48:00.951 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/tmp102.ko 2024-08-31T05:48:00.951 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/tmp103.ko 2024-08-31T05:48:00.951 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/tmp401.ko 2024-08-31T05:48:00.951 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/tmp421.ko 2024-08-31T05:48:00.951 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/via-cputemp.ko 2024-08-31T05:48:00.952 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/via686a.ko 2024-08-31T05:48:00.952 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/vt1211.ko 2024-08-31T05:48:00.952 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/vt8231.ko 2024-08-31T05:48:00.952 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/w83627ehf.ko 2024-08-31T05:48:00.952 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/w83627hf.ko 2024-08-31T05:48:00.952 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/w83781d.ko 2024-08-31T05:48:00.952 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/w83791d.ko 2024-08-31T05:48:00.952 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/w83792d.ko 2024-08-31T05:48:00.952 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/w83793.ko 2024-08-31T05:48:00.952 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/w83795.ko 2024-08-31T05:48:00.952 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/w83l785ts.ko 2024-08-31T05:48:00.952 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/w83l786ng.ko 2024-08-31T05:48:00.952 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c 2024-08-31T05:48:00.952 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/algos 2024-08-31T05:48:00.953 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-08-31T05:48:00.953 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-08-31T05:48:00.953 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses 2024-08-31T05:48:00.953 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-08-31T05:48:00.953 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-08-31T05:48:00.953 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-08-31T05:48:00.953 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-08-31T05:48:00.953 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-08-31T05:48:00.953 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-08-31T05:48:00.953 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-08-31T05:48:00.953 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-08-31T05:48:00.953 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-i801.ko 2024-08-31T05:48:00.953 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-isch.ko 2024-08-31T05:48:00.954 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-08-31T05:48:00.954 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-08-31T05:48:00.954 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-08-31T05:48:00.954 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-parport.ko 2024-08-31T05:48:00.954 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-08-31T05:48:00.954 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-08-31T05:48:00.954 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-08-31T05:48:00.954 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-08-31T05:48:00.954 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-08-31T05:48:00.954 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-08-31T05:48:00.954 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-via.ko 2024-08-31T05:48:00.954 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-08-31T05:48:00.954 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-08-31T05:48:00.954 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/i2c-dev.ko 2024-08-31T05:48:00.954 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/i2c-mux.ko 2024-08-31T05:48:00.955 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-08-31T05:48:00.955 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/i2c-smbus.ko 2024-08-31T05:48:00.955 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/i2c-stub.ko 2024-08-31T05:48:00.955 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband 2024-08-31T05:48:00.955 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/core 2024-08-31T05:48:00.955 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/core/ib_cm.ko 2024-08-31T05:48:00.955 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/core/ib_core.ko 2024-08-31T05:48:00.955 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/core/ib_umad.ko 2024-08-31T05:48:00.955 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-08-31T05:48:00.955 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/core/iw_cm.ko 2024-08-31T05:48:00.955 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/core/rdma_cm.ko 2024-08-31T05:48:00.955 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-08-31T05:48:00.955 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/hw 2024-08-31T05:48:00.955 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/hw/cxgb4 2024-08-31T05:48:00.956 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-08-31T05:48:00.956 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/hw/mlx4 2024-08-31T05:48:00.956 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-08-31T05:48:00.956 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/hw/mlx5 2024-08-31T05:48:00.956 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-08-31T05:48:00.956 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/hw/mthca 2024-08-31T05:48:00.956 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-08-31T05:48:00.956 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/ulp 2024-08-31T05:48:00.956 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/ulp/ipoib 2024-08-31T05:48:00.956 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-08-31T05:48:00.956 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/ulp/iser 2024-08-31T05:48:00.956 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-08-31T05:48:00.956 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/ulp/isert 2024-08-31T05:48:00.956 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-08-31T05:48:00.957 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/ulp/srp 2024-08-31T05:48:00.957 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-08-31T05:48:00.957 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/ulp/srpt 2024-08-31T05:48:00.957 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-08-31T05:48:00.957 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input 2024-08-31T05:48:00.957 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/gameport 2024-08-31T05:48:00.957 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-08-31T05:48:00.957 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/gameport/fm801-gp.ko 2024-08-31T05:48:00.957 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/gameport/gameport.ko 2024-08-31T05:48:00.957 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/joydev.ko 2024-08-31T05:48:00.957 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/keyboard 2024-08-31T05:48:00.957 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/keyboard/gpio_keys.ko 2024-08-31T05:48:00.958 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/mouse 2024-08-31T05:48:00.958 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/mouse/appletouch.ko 2024-08-31T05:48:00.958 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/mouse/bcm5974.ko 2024-08-31T05:48:00.958 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/mouse/cyapatp.ko 2024-08-31T05:48:00.958 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/mouse/elan_i2c.ko 2024-08-31T05:48:00.958 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/mouse/sermouse.ko 2024-08-31T05:48:00.958 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-08-31T05:48:00.958 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/mouse/synaptics_usb.ko 2024-08-31T05:48:00.958 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/mouse/vsxxxaa.ko 2024-08-31T05:48:00.958 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/rmi4 2024-08-31T05:48:00.958 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/rmi4/rmi_core.ko 2024-08-31T05:48:00.958 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/serio 2024-08-31T05:48:00.958 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/serio/altera_ps2.ko 2024-08-31T05:48:00.958 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/serio/arc_ps2.ko 2024-08-31T05:48:00.959 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-08-31T05:48:00.959 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/serio/serio_raw.ko 2024-08-31T05:48:00.959 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/sparse-keymap.ko 2024-08-31T05:48:00.959 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds 2024-08-31T05:48:00.959 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/leds-blinkm.ko 2024-08-31T05:48:00.959 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/leds-lm3530.ko 2024-08-31T05:48:00.959 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/leds-lp3944.ko 2024-08-31T05:48:00.959 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/leds-ss4200.ko 2024-08-31T05:48:00.959 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/trigger 2024-08-31T05:48:00.959 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-08-31T05:48:00.959 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-08-31T05:48:00.959 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-08-31T05:48:00.959 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-08-31T05:48:00.959 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-08-31T05:48:00.959 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-08-31T05:48:00.960 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-08-31T05:48:00.960 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-08-31T05:48:00.960 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md 2024-08-31T05:48:00.960 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/bcache 2024-08-31T05:48:00.960 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/bcache/bcache.ko 2024-08-31T05:48:00.960 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-bio-prison.ko 2024-08-31T05:48:00.960 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-cache-smq.ko 2024-08-31T05:48:00.960 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-cache.ko 2024-08-31T05:48:00.960 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-crypt.ko 2024-08-31T05:48:00.960 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-delay.ko 2024-08-31T05:48:00.960 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-flakey.ko 2024-08-31T05:48:00.960 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-log-userspace.ko 2024-08-31T05:48:00.960 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-multipath.ko 2024-08-31T05:48:00.961 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-queue-length.ko 2024-08-31T05:48:00.961 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-raid.ko 2024-08-31T05:48:00.961 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-round-robin.ko 2024-08-31T05:48:00.961 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-service-time.ko 2024-08-31T05:48:00.961 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-switch.ko 2024-08-31T05:48:00.961 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-thin-pool.ko 2024-08-31T05:48:00.961 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-verity.ko 2024-08-31T05:48:00.961 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/persistent-data 2024-08-31T05:48:00.961 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-08-31T05:48:00.961 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/raid0.ko 2024-08-31T05:48:00.961 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/raid1.ko 2024-08-31T05:48:00.961 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/raid10.ko 2024-08-31T05:48:00.961 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/raid456.ko 2024-08-31T05:48:00.961 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/message 2024-08-31T05:48:00.962 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/message/fusion 2024-08-31T05:48:00.962 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/message/fusion/mptbase.ko 2024-08-31T05:48:00.962 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/message/fusion/mptctl.ko 2024-08-31T05:48:00.962 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/message/fusion/mptfc.ko 2024-08-31T05:48:00.962 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/message/fusion/mptlan.ko 2024-08-31T05:48:00.962 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/message/fusion/mptsas.ko 2024-08-31T05:48:00.962 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/message/fusion/mptscsih.ko 2024-08-31T05:48:00.962 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/message/fusion/mptspi.ko 2024-08-31T05:48:00.962 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/mfd 2024-08-31T05:48:00.962 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/mfd/lpc_ich.ko 2024-08-31T05:48:00.962 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/mfd/lpc_sch.ko 2024-08-31T05:48:00.962 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/mfd/mfd-core.ko 2024-08-31T05:48:00.962 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/mfd/sm501.ko 2024-08-31T05:48:00.962 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/mfd/viperboard.ko 2024-08-31T05:48:00.962 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/mfd/vx855.ko 2024-08-31T05:48:00.963 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/mfd/wl1273-core.ko 2024-08-31T05:48:00.963 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc 2024-08-31T05:48:00.963 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/altera-stapl 2024-08-31T05:48:00.963 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-08-31T05:48:00.963 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/apds9802als.ko 2024-08-31T05:48:00.963 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/apds990x.ko 2024-08-31T05:48:00.963 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/bh1770glc.ko 2024-08-31T05:48:00.963 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/cb710 2024-08-31T05:48:00.963 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/cb710/cb710.ko 2024-08-31T05:48:00.963 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/eeprom 2024-08-31T05:48:00.963 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/eeprom/at24.ko 2024-08-31T05:48:00.963 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-08-31T05:48:00.963 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/eeprom/max6875.ko 2024-08-31T05:48:00.963 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/enclosure.ko 2024-08-31T05:48:00.963 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/hpilo.ko 2024-08-31T05:48:00.963 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/isl29003.ko 2024-08-31T05:48:00.964 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/isl29020.ko 2024-08-31T05:48:00.964 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/lis3lv02d 2024-08-31T05:48:00.964 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-08-31T05:48:00.964 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-08-31T05:48:00.964 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/mei 2024-08-31T05:48:00.964 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/mei/mei-me.ko 2024-08-31T05:48:00.964 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/mei/mei-txe.ko 2024-08-31T05:48:00.964 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/mei/mei.ko 2024-08-31T05:48:00.964 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/sgi-gru 2024-08-31T05:48:00.964 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/sgi-gru/gru.ko 2024-08-31T05:48:00.964 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/sgi-xp 2024-08-31T05:48:00.964 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/sgi-xp/xp.ko 2024-08-31T05:48:00.964 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/sgi-xp/xpc.ko 2024-08-31T05:48:00.964 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-08-31T05:48:00.965 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/tifm_7xx1.ko 2024-08-31T05:48:00.965 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/tifm_core.ko 2024-08-31T05:48:00.965 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/tsl2550.ko 2024-08-31T05:48:00.965 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/vmw_balloon.ko 2024-08-31T05:48:00.965 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/vmw_vmci 2024-08-31T05:48:00.965 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-08-31T05:48:00.965 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net 2024-08-31T05:48:00.965 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/bonding 2024-08-31T05:48:00.965 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/bonding/bonding.ko 2024-08-31T05:48:00.965 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/dsa 2024-08-31T05:48:00.965 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/dsa/b53 2024-08-31T05:48:00.965 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/dsa/b53/b53_common.ko 2024-08-31T05:48:00.965 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/dsa/bcm-sf2.ko 2024-08-31T05:48:00.966 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/dsa/mv88e6060.ko 2024-08-31T05:48:00.966 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/dsa/mv88e6xxx 2024-08-31T05:48:00.966 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-08-31T05:48:00.966 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/dummy.ko 2024-08-31T05:48:00.966 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/eql.ko 2024-08-31T05:48:00.966 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet 2024-08-31T05:48:00.966 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/3com 2024-08-31T05:48:00.966 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-08-31T05:48:00.966 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-08-31T05:48:00.966 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-08-31T05:48:00.966 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-08-31T05:48:00.966 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/8390 2024-08-31T05:48:00.966 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/8390/8390.ko 2024-08-31T05:48:00.966 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-08-31T05:48:00.967 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-08-31T05:48:00.967 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-08-31T05:48:00.967 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/adaptec 2024-08-31T05:48:00.967 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-08-31T05:48:00.967 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/agere 2024-08-31T05:48:00.967 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/agere/et131x.ko 2024-08-31T05:48:00.967 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/alteon 2024-08-31T05:48:00.967 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-08-31T05:48:00.967 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/altera 2024-08-31T05:48:00.967 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-08-31T05:48:00.967 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/amd 2024-08-31T05:48:00.967 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-08-31T05:48:00.967 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-08-31T05:48:00.968 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-08-31T05:48:00.968 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/atheros 2024-08-31T05:48:00.968 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/atheros/alx 2024-08-31T05:48:00.968 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-08-31T05:48:00.968 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/atheros/atl1c 2024-08-31T05:48:00.968 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-08-31T05:48:00.968 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/atheros/atl1e 2024-08-31T05:48:00.968 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-08-31T05:48:00.968 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/atheros/atlx 2024-08-31T05:48:00.968 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-08-31T05:48:00.968 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-08-31T05:48:00.968 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/broadcom 2024-08-31T05:48:00.968 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-08-31T05:48:00.968 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-08-31T05:48:00.968 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-08-31T05:48:00.969 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-08-31T05:48:00.969 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-08-31T05:48:00.969 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/broadcom/genet 2024-08-31T05:48:00.969 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-08-31T05:48:00.969 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-08-31T05:48:00.969 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/brocade 2024-08-31T05:48:00.969 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/brocade/bna 2024-08-31T05:48:00.969 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-08-31T05:48:00.969 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/chelsio 2024-08-31T05:48:00.969 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/chelsio/cxgb 2024-08-31T05:48:00.969 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-08-31T05:48:00.969 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-08-31T05:48:00.969 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-08-31T05:48:00.969 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-08-31T05:48:00.970 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-08-31T05:48:00.970 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-08-31T05:48:00.970 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-08-31T05:48:00.970 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-08-31T05:48:00.970 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-08-31T05:48:00.970 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/cisco 2024-08-31T05:48:00.970 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/cisco/enic 2024-08-31T05:48:00.970 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-08-31T05:48:00.970 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dec 2024-08-31T05:48:00.970 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dec/tulip 2024-08-31T05:48:00.970 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-08-31T05:48:00.970 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-08-31T05:48:00.970 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-08-31T05:48:00.970 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-08-31T05:48:00.971 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-08-31T05:48:00.971 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-08-31T05:48:00.971 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dlink 2024-08-31T05:48:00.971 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-08-31T05:48:00.971 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-08-31T05:48:00.971 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dnet.ko 2024-08-31T05:48:00.971 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/emulex 2024-08-31T05:48:00.971 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/emulex/benet 2024-08-31T05:48:00.971 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-08-31T05:48:00.971 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/ethoc.ko 2024-08-31T05:48:00.971 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/fealnx.ko 2024-08-31T05:48:00.971 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel 2024-08-31T05:48:00.971 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/e100.ko 2024-08-31T05:48:00.971 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/e1000 2024-08-31T05:48:00.971 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-08-31T05:48:00.972 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/e1000e 2024-08-31T05:48:00.972 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-08-31T05:48:00.972 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/fm10k 2024-08-31T05:48:00.972 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-08-31T05:48:00.972 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/i40e 2024-08-31T05:48:00.972 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-08-31T05:48:00.972 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/iavf 2024-08-31T05:48:00.972 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-08-31T05:48:00.972 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/igb 2024-08-31T05:48:00.972 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-08-31T05:48:00.972 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/igbvf 2024-08-31T05:48:00.972 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-08-31T05:48:00.972 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/ixgbe 2024-08-31T05:48:00.972 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-08-31T05:48:00.972 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/ixgbevf 2024-08-31T05:48:00.973 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-08-31T05:48:00.973 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/libeth 2024-08-31T05:48:00.973 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-08-31T05:48:00.973 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/libie 2024-08-31T05:48:00.973 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-08-31T05:48:00.973 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/jme.ko 2024-08-31T05:48:00.973 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/marvell 2024-08-31T05:48:00.973 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-08-31T05:48:00.973 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/marvell/skge.ko 2024-08-31T05:48:00.973 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-08-31T05:48:00.973 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/mellanox 2024-08-31T05:48:00.973 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/mellanox/mlx4 2024-08-31T05:48:00.973 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-08-31T05:48:00.973 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-08-31T05:48:00.973 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/mellanox/mlx5 2024-08-31T05:48:00.974 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-08-31T05:48:00.974 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-08-31T05:48:00.974 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/micrel 2024-08-31T05:48:00.974 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-08-31T05:48:00.974 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/myricom 2024-08-31T05:48:00.974 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/myricom/myri10ge 2024-08-31T05:48:00.974 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-08-31T05:48:00.974 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/natsemi 2024-08-31T05:48:00.974 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-08-31T05:48:00.974 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-08-31T05:48:00.974 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/neterion 2024-08-31T05:48:00.974 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-08-31T05:48:00.974 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/nvidia 2024-08-31T05:48:00.974 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-08-31T05:48:00.974 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/packetengines 2024-08-31T05:48:00.975 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-08-31T05:48:00.975 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-08-31T05:48:00.975 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/qlogic 2024-08-31T05:48:00.975 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/qlogic/netxen 2024-08-31T05:48:00.975 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-08-31T05:48:00.975 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-08-31T05:48:00.975 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-08-31T05:48:00.975 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-08-31T05:48:00.975 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/rdc 2024-08-31T05:48:00.975 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-08-31T05:48:00.975 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/realtek 2024-08-31T05:48:00.975 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-08-31T05:48:00.975 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-08-31T05:48:00.975 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/realtek/atp.ko 2024-08-31T05:48:00.975 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-08-31T05:48:00.976 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/rocker 2024-08-31T05:48:00.976 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-08-31T05:48:00.976 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/sfc 2024-08-31T05:48:00.976 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-08-31T05:48:00.976 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/silan 2024-08-31T05:48:00.976 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-08-31T05:48:00.976 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/sis 2024-08-31T05:48:00.976 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/sis/sis190.ko 2024-08-31T05:48:00.976 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/sis/sis900.ko 2024-08-31T05:48:00.976 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/smsc 2024-08-31T05:48:00.976 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-08-31T05:48:00.976 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-08-31T05:48:00.976 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-08-31T05:48:00.976 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-08-31T05:48:00.977 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/stmicro 2024-08-31T05:48:00.977 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/stmicro/stmmac 2024-08-31T05:48:00.977 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-08-31T05:48:00.977 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-08-31T05:48:00.977 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/sun 2024-08-31T05:48:00.977 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/sun/cassini.ko 2024-08-31T05:48:00.977 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/sun/niu.ko 2024-08-31T05:48:00.977 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/sun/sungem.ko 2024-08-31T05:48:00.977 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-08-31T05:48:00.977 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/tehuti 2024-08-31T05:48:00.977 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-08-31T05:48:00.977 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/ti 2024-08-31T05:48:00.977 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/ti/tlan.ko 2024-08-31T05:48:00.978 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/via 2024-08-31T05:48:00.978 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-08-31T05:48:00.978 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-08-31T05:48:00.978 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/wiznet 2024-08-31T05:48:00.978 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-08-31T05:48:00.978 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-08-31T05:48:00.978 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/xircom 2024-08-31T05:48:00.978 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-08-31T05:48:00.978 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/geneve.ko 2024-08-31T05:48:00.978 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/hyperv 2024-08-31T05:48:00.978 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-08-31T05:48:00.978 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ieee802154 2024-08-31T05:48:00.978 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ieee802154/fakelb.ko 2024-08-31T05:48:00.978 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ifb.ko 2024-08-31T05:48:00.979 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ipvlan 2024-08-31T05:48:00.979 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ipvlan/ipvlan.ko 2024-08-31T05:48:00.979 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/macvlan.ko 2024-08-31T05:48:00.979 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/macvtap.ko 2024-08-31T05:48:00.979 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/mdio 2024-08-31T05:48:00.979 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/mdio.ko 2024-08-31T05:48:00.979 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-08-31T05:48:00.979 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-08-31T05:48:00.979 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/mdio/mdio-regmap.ko 2024-08-31T05:48:00.979 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/mii.ko 2024-08-31T05:48:00.979 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/net_failover.ko 2024-08-31T05:48:00.979 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/netconsole.ko 2024-08-31T05:48:00.980 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/nlmon.ko 2024-08-31T05:48:00.980 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/pcs 2024-08-31T05:48:00.980 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/pcs/pcs-lynx.ko 2024-08-31T05:48:00.980 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-08-31T05:48:00.980 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy 2024-08-31T05:48:00.980 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/amd.ko 2024-08-31T05:48:00.980 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-08-31T05:48:00.980 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-08-31T05:48:00.980 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/bcm7xxx.ko 2024-08-31T05:48:00.980 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/bcm87xx.ko 2024-08-31T05:48:00.980 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/broadcom.ko 2024-08-31T05:48:00.980 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/cicada.ko 2024-08-31T05:48:00.980 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/davicom.ko 2024-08-31T05:48:00.981 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/dp83640.ko 2024-08-31T05:48:00.981 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/et1011c.ko 2024-08-31T05:48:00.981 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/icplus.ko 2024-08-31T05:48:00.981 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/lxt.ko 2024-08-31T05:48:00.981 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/marvell.ko 2024-08-31T05:48:00.981 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/micrel.ko 2024-08-31T05:48:00.981 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/national.ko 2024-08-31T05:48:00.981 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/phylink.ko 2024-08-31T05:48:00.981 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/qsemi.ko 2024-08-31T05:48:00.981 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/realtek.ko 2024-08-31T05:48:00.981 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/smsc.ko 2024-08-31T05:48:00.981 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/ste10Xp.ko 2024-08-31T05:48:00.981 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/vitesse.ko 2024-08-31T05:48:00.982 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ppp 2024-08-31T05:48:00.982 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ppp/bsd_comp.ko 2024-08-31T05:48:00.982 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ppp/ppp_async.ko 2024-08-31T05:48:00.982 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ppp/ppp_deflate.ko 2024-08-31T05:48:00.982 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ppp/ppp_generic.ko 2024-08-31T05:48:00.982 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ppp/ppp_mppe.ko 2024-08-31T05:48:00.982 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ppp/ppp_synctty.ko 2024-08-31T05:48:00.982 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ppp/pppoe.ko 2024-08-31T05:48:00.982 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ppp/pppox.ko 2024-08-31T05:48:00.982 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ppp/pptp.ko 2024-08-31T05:48:00.982 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/slip 2024-08-31T05:48:00.982 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/slip/slhc.ko 2024-08-31T05:48:00.982 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/slip/slip.ko 2024-08-31T05:48:00.982 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/sungem_phy.ko 2024-08-31T05:48:00.982 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/tap.ko 2024-08-31T05:48:00.983 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/team 2024-08-31T05:48:00.983 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/team/team.ko 2024-08-31T05:48:00.983 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/team/team_mode_activebackup.ko 2024-08-31T05:48:00.983 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/team/team_mode_broadcast.ko 2024-08-31T05:48:00.983 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-08-31T05:48:00.983 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/team/team_mode_random.ko 2024-08-31T05:48:00.983 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-08-31T05:48:00.983 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/tun.ko 2024-08-31T05:48:00.983 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/veth.ko 2024-08-31T05:48:00.983 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/virtio_net.ko 2024-08-31T05:48:00.983 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/vmxnet3 2024-08-31T05:48:00.983 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-08-31T05:48:00.983 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/vxlan 2024-08-31T05:48:00.984 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/vxlan/vxlan.ko 2024-08-31T05:48:00.984 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/xen-netback 2024-08-31T05:48:00.984 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/xen-netback/xen-netback.ko 2024-08-31T05:48:00.984 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/xen-netfront.ko 2024-08-31T05:48:00.984 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ntb 2024-08-31T05:48:00.984 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ntb/ntb.ko 2024-08-31T05:48:00.984 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/nvme 2024-08-31T05:48:00.984 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/nvme/host 2024-08-31T05:48:00.984 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/nvme/host/nvme-core.ko 2024-08-31T05:48:00.984 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/nvme/host/nvme.ko 2024-08-31T05:48:00.984 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/parport 2024-08-31T05:48:00.984 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/parport/parport.ko 2024-08-31T05:48:00.984 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/parport/parport_cs.ko 2024-08-31T05:48:00.984 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/parport/parport_pc.ko 2024-08-31T05:48:00.985 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/parport/parport_serial.ko 2024-08-31T05:48:00.985 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pci 2024-08-31T05:48:00.985 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pci/hotplug 2024-08-31T05:48:00.985 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-08-31T05:48:00.985 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pci/pcie 2024-08-31T05:48:00.985 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pci/pcie/aer_inject.ko 2024-08-31T05:48:00.985 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pci/xen-pcifront.ko 2024-08-31T05:48:00.985 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pcmcia 2024-08-31T05:48:00.985 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pcmcia/i82092.ko 2024-08-31T05:48:00.985 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pcmcia/pd6729.ko 2024-08-31T05:48:00.985 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pcmcia/yenta_socket.ko 2024-08-31T05:48:00.985 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pinctrl 2024-08-31T05:48:00.985 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pinctrl/intel 2024-08-31T05:48:00.986 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-08-31T05:48:00.986 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform 2024-08-31T05:48:00.986 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86 2024-08-31T05:48:00.986 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/acer-wmi.ko 2024-08-31T05:48:00.986 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/acerhdf.ko 2024-08-31T05:48:00.986 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/apple-gmux.ko 2024-08-31T05:48:00.986 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/asus-laptop.ko 2024-08-31T05:48:00.986 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-08-31T05:48:00.986 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/asus-wmi.ko 2024-08-31T05:48:00.986 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/classmate-laptop.ko 2024-08-31T05:48:00.986 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-08-31T05:48:00.986 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-08-31T05:48:00.986 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-08-31T05:48:00.986 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-08-31T05:48:00.986 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/hdaps.ko 2024-08-31T05:48:00.987 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/intel 2024-08-31T05:48:00.987 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-08-31T05:48:00.987 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/intel_ips.ko 2024-08-31T05:48:00.987 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/msi-wmi.ko 2024-08-31T05:48:00.987 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/mxm-wmi.ko 2024-08-31T05:48:00.987 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-08-31T05:48:00.987 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/samsung-laptop.ko 2024-08-31T05:48:00.987 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/samsung-q10.ko 2024-08-31T05:48:00.987 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/topstar-laptop.ko 2024-08-31T05:48:00.987 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-08-31T05:48:00.987 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/toshiba_haps.ko 2024-08-31T05:48:00.987 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/wmi-bmof.ko 2024-08-31T05:48:00.987 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/wmi.ko 2024-08-31T05:48:00.988 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/powercap 2024-08-31T05:48:00.988 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/powercap/intel_rapl_common.ko 2024-08-31T05:48:00.988 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/powercap/intel_rapl_msr.ko 2024-08-31T05:48:00.988 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pps 2024-08-31T05:48:00.988 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pps/clients 2024-08-31T05:48:00.988 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pps/clients/pps-gpio.ko 2024-08-31T05:48:00.988 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pps/clients/pps-ldisc.ko 2024-08-31T05:48:00.988 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pps/clients/pps_parport.ko 2024-08-31T05:48:00.988 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pps/pps_core.ko 2024-08-31T05:48:00.988 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ptp 2024-08-31T05:48:00.988 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ptp/ptp.ko 2024-08-31T05:48:00.988 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ptp/ptp_kvm.ko 2024-08-31T05:48:00.988 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ras 2024-08-31T05:48:00.988 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ras/amd 2024-08-31T05:48:00.988 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ras/amd/atl 2024-08-31T05:48:00.989 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-08-31T05:48:00.989 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ras/amd/fmpm.ko 2024-08-31T05:48:00.989 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc 2024-08-31T05:48:00.989 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-bq32k.ko 2024-08-31T05:48:00.989 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-ds1286.ko 2024-08-31T05:48:00.989 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-ds1307.ko 2024-08-31T05:48:00.989 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-ds1374.ko 2024-08-31T05:48:00.989 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-ds1511.ko 2024-08-31T05:48:00.989 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-ds1553.ko 2024-08-31T05:48:00.989 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-ds1672.ko 2024-08-31T05:48:00.989 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-ds1742.ko 2024-08-31T05:48:00.989 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-ds2404.ko 2024-08-31T05:48:00.989 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-ds3232.ko 2024-08-31T05:48:00.989 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-em3027.ko 2024-08-31T05:48:00.989 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-fm3130.ko 2024-08-31T05:48:00.990 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-isl12022.ko 2024-08-31T05:48:00.990 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-isl1208.ko 2024-08-31T05:48:00.990 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-m41t80.ko 2024-08-31T05:48:00.990 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-m48t35.ko 2024-08-31T05:48:00.990 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-m48t59.ko 2024-08-31T05:48:00.990 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-max6900.ko 2024-08-31T05:48:00.990 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-msm6242.ko 2024-08-31T05:48:00.990 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-pcf2127.ko 2024-08-31T05:48:00.990 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-pcf85063.ko 2024-08-31T05:48:00.990 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-pcf8523.ko 2024-08-31T05:48:00.990 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-pcf8563.ko 2024-08-31T05:48:00.990 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-pcf8583.ko 2024-08-31T05:48:00.990 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-rp5c01.ko 2024-08-31T05:48:00.990 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-rs5c372.ko 2024-08-31T05:48:00.990 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-08-31T05:48:00.991 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-rx8025.ko 2024-08-31T05:48:00.991 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-rx8581.ko 2024-08-31T05:48:00.991 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-08-31T05:48:00.991 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-x1205.ko 2024-08-31T05:48:00.991 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi 2024-08-31T05:48:00.991 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/3w-9xxx.ko 2024-08-31T05:48:00.991 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/3w-sas.ko 2024-08-31T05:48:00.991 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/3w-xxxx.ko 2024-08-31T05:48:00.991 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/BusLogic.ko 2024-08-31T05:48:00.991 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/a100u2w.ko 2024-08-31T05:48:00.991 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/aacraid 2024-08-31T05:48:00.991 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/aacraid/aacraid.ko 2024-08-31T05:48:00.991 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/advansys.ko 2024-08-31T05:48:00.991 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/aic7xxx 2024-08-31T05:48:00.991 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-08-31T05:48:00.991 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-08-31T05:48:00.992 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/am53c974.ko 2024-08-31T05:48:00.992 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/arcmsr 2024-08-31T05:48:00.992 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-08-31T05:48:00.992 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/atp870u.ko 2024-08-31T05:48:00.992 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/be2iscsi 2024-08-31T05:48:00.992 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-08-31T05:48:00.992 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/bfa 2024-08-31T05:48:00.992 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/bfa/bfa.ko 2024-08-31T05:48:00.992 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/bnx2fc 2024-08-31T05:48:00.992 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-08-31T05:48:00.992 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/bnx2i 2024-08-31T05:48:00.992 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-08-31T05:48:00.992 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/ch.ko 2024-08-31T05:48:00.992 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/csiostor 2024-08-31T05:48:00.993 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/csiostor/csiostor.ko 2024-08-31T05:48:00.993 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/cxgbi 2024-08-31T05:48:00.993 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/cxgbi/cxgb3i 2024-08-31T05:48:00.993 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-08-31T05:48:00.993 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/cxgbi/cxgb4i 2024-08-31T05:48:00.993 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-08-31T05:48:00.993 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-08-31T05:48:00.993 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/dc395x.ko 2024-08-31T05:48:00.993 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/device_handler 2024-08-31T05:48:00.993 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-08-31T05:48:00.993 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-08-31T05:48:00.993 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-08-31T05:48:00.993 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-08-31T05:48:00.993 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/dmx3191d.ko 2024-08-31T05:48:00.994 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/esas2r 2024-08-31T05:48:00.994 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/esas2r/esas2r.ko 2024-08-31T05:48:00.994 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/esp_scsi.ko 2024-08-31T05:48:00.994 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/fcoe 2024-08-31T05:48:00.994 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/fcoe/fcoe.ko 2024-08-31T05:48:00.994 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-08-31T05:48:00.994 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/fnic 2024-08-31T05:48:00.994 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/fnic/fnic.ko 2024-08-31T05:48:00.994 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/hpsa.ko 2024-08-31T05:48:00.994 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/hptiop.ko 2024-08-31T05:48:00.994 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/hv_storvsc.ko 2024-08-31T05:48:00.994 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/initio.ko 2024-08-31T05:48:00.994 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/ipr.ko 2024-08-31T05:48:00.994 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/ips.ko 2024-08-31T05:48:00.995 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/isci 2024-08-31T05:48:00.995 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/isci/isci.ko 2024-08-31T05:48:00.995 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-08-31T05:48:00.995 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/iscsi_tcp.ko 2024-08-31T05:48:00.995 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/libfc 2024-08-31T05:48:00.995 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/libfc/libfc.ko 2024-08-31T05:48:00.995 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/libiscsi.ko 2024-08-31T05:48:00.995 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/libiscsi_tcp.ko 2024-08-31T05:48:00.995 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/libsas 2024-08-31T05:48:00.995 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/libsas/libsas.ko 2024-08-31T05:48:00.995 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/lpfc 2024-08-31T05:48:00.995 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/lpfc/lpfc.ko 2024-08-31T05:48:00.995 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/megaraid 2024-08-31T05:48:00.996 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/megaraid.ko 2024-08-31T05:48:00.996 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-08-31T05:48:00.996 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-08-31T05:48:00.996 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-08-31T05:48:00.996 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/mpt3sas 2024-08-31T05:48:00.996 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-08-31T05:48:00.996 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/mvsas 2024-08-31T05:48:00.996 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/mvsas/mvsas.ko 2024-08-31T05:48:00.996 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/mvumi.ko 2024-08-31T05:48:00.996 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/pm8001 2024-08-31T05:48:00.996 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-08-31T05:48:00.996 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/pmcraid.ko 2024-08-31T05:48:00.996 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/qla1280.ko 2024-08-31T05:48:00.996 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/qla2xxx 2024-08-31T05:48:00.997 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-08-31T05:48:00.997 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-08-31T05:48:00.997 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/qla4xxx 2024-08-31T05:48:00.997 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-08-31T05:48:00.997 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/raid_class.ko 2024-08-31T05:48:00.997 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/scsi_debug.ko 2024-08-31T05:48:00.997 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/scsi_transport_fc.ko 2024-08-31T05:48:00.997 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-08-31T05:48:00.997 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/scsi_transport_sas.ko 2024-08-31T05:48:00.997 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/scsi_transport_spi.ko 2024-08-31T05:48:00.997 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/scsi_transport_srp.ko 2024-08-31T05:48:00.997 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/ses.ko 2024-08-31T05:48:00.997 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/st.ko 2024-08-31T05:48:00.997 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/stex.ko 2024-08-31T05:48:00.997 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/sym53c8xx_2 2024-08-31T05:48:00.998 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-08-31T05:48:00.998 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/virtio_scsi.ko 2024-08-31T05:48:00.998 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/vmw_pvscsi.ko 2024-08-31T05:48:00.998 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/wd719x.ko 2024-08-31T05:48:00.998 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ssb 2024-08-31T05:48:00.998 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ssb/ssb.ko 2024-08-31T05:48:00.998 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target 2024-08-31T05:48:00.998 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target/iscsi 2024-08-31T05:48:00.998 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-08-31T05:48:00.998 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target/loopback 2024-08-31T05:48:00.998 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target/loopback/tcm_loop.ko 2024-08-31T05:48:00.998 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target/target_core_file.ko 2024-08-31T05:48:00.998 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target/target_core_iblock.ko 2024-08-31T05:48:00.998 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target/target_core_mod.ko 2024-08-31T05:48:00.999 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target/target_core_pscsi.ko 2024-08-31T05:48:00.999 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target/target_core_user.ko 2024-08-31T05:48:00.999 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target/tcm_fc 2024-08-31T05:48:00.999 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-08-31T05:48:00.999 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal 2024-08-31T05:48:00.999 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel 2024-08-31T05:48:00.999 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal 2024-08-31T05:48:00.999 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-08-31T05:48:00.999 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-08-31T05:48:00.999 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-08-31T05:48:00.999 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-08-31T05:48:00.999 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-08-31T05:48:00.999 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-08-31T05:48:00.999 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-08-31T05:48:00.999 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-08-31T05:48:01.000 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-08-31T05:48:01.000 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-08-31T05:48:01.000 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-08-31T05:48:01.000 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-08-31T05:48:01.000 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-08-31T05:48:01.000 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-08-31T05:48:01.000 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-08-31T05:48:01.000 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-08-31T05:48:01.000 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-08-31T05:48:01.000 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-08-31T05:48:01.000 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty 2024-08-31T05:48:01.000 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty/ipwireless 2024-08-31T05:48:01.000 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-08-31T05:48:01.001 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty/n_gsm.ko 2024-08-31T05:48:01.001 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty/n_hdlc.ko 2024-08-31T05:48:01.001 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty/nozomi.ko 2024-08-31T05:48:01.001 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty/serial 2024-08-31T05:48:01.001 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty/serial/8250 2024-08-31T05:48:01.001 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-08-31T05:48:01.001 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty/serial/arc_uart.ko 2024-08-31T05:48:01.001 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty/serial/jsm 2024-08-31T05:48:01.001 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty/serial/jsm/jsm.ko 2024-08-31T05:48:01.001 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ufs 2024-08-31T05:48:01.001 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ufs/core 2024-08-31T05:48:01.001 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ufs/core/ufshcd-core.ko 2024-08-31T05:48:01.001 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ufs/host 2024-08-31T05:48:01.001 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-08-31T05:48:01.002 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/uio 2024-08-31T05:48:01.002 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/uio/uio.ko 2024-08-31T05:48:01.002 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/uio/uio_aec.ko 2024-08-31T05:48:01.002 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/uio/uio_cif.ko 2024-08-31T05:48:01.002 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/uio/uio_pci_generic.ko 2024-08-31T05:48:01.002 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/uio/uio_sercos3.ko 2024-08-31T05:48:01.002 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb 2024-08-31T05:48:01.002 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/class 2024-08-31T05:48:01.002 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/class/cdc-acm.ko 2024-08-31T05:48:01.002 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/class/cdc-wdm.ko 2024-08-31T05:48:01.002 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/class/usblp.ko 2024-08-31T05:48:01.002 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/class/usbtmc.ko 2024-08-31T05:48:01.002 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/host 2024-08-31T05:48:01.003 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/host/sl811-hcd.ko 2024-08-31T05:48:01.003 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/image 2024-08-31T05:48:01.003 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/image/mdc800.ko 2024-08-31T05:48:01.003 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/image/microtek.ko 2024-08-31T05:48:01.003 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc 2024-08-31T05:48:01.003 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/adutux.ko 2024-08-31T05:48:01.003 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/appledisplay.ko 2024-08-31T05:48:01.003 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/emi26.ko 2024-08-31T05:48:01.003 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/emi62.ko 2024-08-31T05:48:01.003 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/ezusb.ko 2024-08-31T05:48:01.003 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/idmouse.ko 2024-08-31T05:48:01.003 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/iowarrior.ko 2024-08-31T05:48:01.003 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/isight_firmware.ko 2024-08-31T05:48:01.003 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/ldusb.ko 2024-08-31T05:48:01.003 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/legousbtower.ko 2024-08-31T05:48:01.003 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/sisusbvga 2024-08-31T05:48:01.004 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-08-31T05:48:01.004 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/trancevibrator.ko 2024-08-31T05:48:01.004 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/usb3503.ko 2024-08-31T05:48:01.004 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/usblcd.ko 2024-08-31T05:48:01.004 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/usbsevseg.ko 2024-08-31T05:48:01.004 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/uss720.ko 2024-08-31T05:48:01.004 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/yurex.ko 2024-08-31T05:48:01.004 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/phy 2024-08-31T05:48:01.004 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/phy/phy-generic.ko 2024-08-31T05:48:01.004 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial 2024-08-31T05:48:01.004 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/aircable.ko 2024-08-31T05:48:01.004 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/ark3116.ko 2024-08-31T05:48:01.004 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/belkin_sa.ko 2024-08-31T05:48:01.004 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/ch341.ko 2024-08-31T05:48:01.004 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/cp210x.ko 2024-08-31T05:48:01.005 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/cyberjack.ko 2024-08-31T05:48:01.005 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/cypress_m8.ko 2024-08-31T05:48:01.005 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/digi_acceleport.ko 2024-08-31T05:48:01.005 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/empeg.ko 2024-08-31T05:48:01.005 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/ftdi_sio.ko 2024-08-31T05:48:01.005 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/garmin_gps.ko 2024-08-31T05:48:01.005 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/io_edgeport.ko 2024-08-31T05:48:01.005 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/io_ti.ko 2024-08-31T05:48:01.005 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/ipaq.ko 2024-08-31T05:48:01.005 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/ipw.ko 2024-08-31T05:48:01.005 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/ir-usb.ko 2024-08-31T05:48:01.005 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-08-31T05:48:01.005 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/keyspan.ko 2024-08-31T05:48:01.006 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/keyspan_pda.ko 2024-08-31T05:48:01.006 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/kl5kusb105.ko 2024-08-31T05:48:01.006 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/kobil_sct.ko 2024-08-31T05:48:01.006 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/mct_u232.ko 2024-08-31T05:48:01.006 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/mos7720.ko 2024-08-31T05:48:01.006 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/mos7840.ko 2024-08-31T05:48:01.006 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/navman.ko 2024-08-31T05:48:01.006 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/omninet.ko 2024-08-31T05:48:01.006 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/opticon.ko 2024-08-31T05:48:01.006 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/option.ko 2024-08-31T05:48:01.006 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/oti6858.ko 2024-08-31T05:48:01.006 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/pl2303.ko 2024-08-31T05:48:01.006 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/qcaux.ko 2024-08-31T05:48:01.006 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/qcserial.ko 2024-08-31T05:48:01.006 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/quatech2.ko 2024-08-31T05:48:01.006 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/safe_serial.ko 2024-08-31T05:48:01.007 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/sierra.ko 2024-08-31T05:48:01.007 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/spcp8x5.ko 2024-08-31T05:48:01.007 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/ssu100.ko 2024-08-31T05:48:01.007 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/symbolserial.ko 2024-08-31T05:48:01.007 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-08-31T05:48:01.007 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-08-31T05:48:01.007 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/usb_debug.ko 2024-08-31T05:48:01.007 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/usb_wwan.ko 2024-08-31T05:48:01.007 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/visor.ko 2024-08-31T05:48:01.007 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/whiteheat.ko 2024-08-31T05:48:01.007 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/xsens_mt.ko 2024-08-31T05:48:01.007 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage 2024-08-31T05:48:01.007 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/uas.ko 2024-08-31T05:48:01.007 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-alauda.ko 2024-08-31T05:48:01.007 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-cypress.ko 2024-08-31T05:48:01.008 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-datafab.ko 2024-08-31T05:48:01.008 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-08-31T05:48:01.008 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-freecom.ko 2024-08-31T05:48:01.008 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-isd200.ko 2024-08-31T05:48:01.008 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-08-31T05:48:01.008 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-karma.ko 2024-08-31T05:48:01.008 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-onetouch.ko 2024-08-31T05:48:01.008 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-realtek.ko 2024-08-31T05:48:01.008 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-sddr09.ko 2024-08-31T05:48:01.008 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-sddr55.ko 2024-08-31T05:48:01.008 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-usbat.ko 2024-08-31T05:48:01.008 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/usb-storage.ko 2024-08-31T05:48:01.008 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/usbip 2024-08-31T05:48:01.008 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/usbip/usbip-core.ko 2024-08-31T05:48:01.009 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/usbip/usbip-host.ko 2024-08-31T05:48:01.009 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-08-31T05:48:01.009 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/vfio 2024-08-31T05:48:01.009 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/vfio/pci 2024-08-31T05:48:01.009 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-08-31T05:48:01.009 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/vfio/pci/vfio-pci.ko 2024-08-31T05:48:01.009 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/vfio/vfio.ko 2024-08-31T05:48:01.009 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-08-31T05:48:01.009 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/vhost 2024-08-31T05:48:01.009 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/vhost/vhost.ko 2024-08-31T05:48:01.009 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/vhost/vhost_iotlb.ko 2024-08-31T05:48:01.009 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/vhost/vhost_net.ko 2024-08-31T05:48:01.009 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/vhost/vhost_scsi.ko 2024-08-31T05:48:01.009 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/video 2024-08-31T05:48:01.009 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/video/backlight 2024-08-31T05:48:01.010 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/video/backlight/apple_bl.ko 2024-08-31T05:48:01.010 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/video/backlight/lcd.ko 2024-08-31T05:48:01.010 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/video/backlight/platform_lcd.ko 2024-08-31T05:48:01.010 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/virtio 2024-08-31T05:48:01.010 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/virtio/virtio.ko 2024-08-31T05:48:01.010 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/virtio/virtio_balloon.ko 2024-08-31T05:48:01.010 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/virtio/virtio_mmio.ko 2024-08-31T05:48:01.010 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/virtio/virtio_pci.ko 2024-08-31T05:48:01.010 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-08-31T05:48:01.010 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-08-31T05:48:01.010 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/virtio/virtio_ring.ko 2024-08-31T05:48:01.010 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1 2024-08-31T05:48:01.010 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/masters 2024-08-31T05:48:01.010 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/masters/ds2482.ko 2024-08-31T05:48:01.010 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/masters/ds2490.ko 2024-08-31T05:48:01.011 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves 2024-08-31T05:48:01.011 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-08-31T05:48:01.011 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-08-31T05:48:01.011 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-08-31T05:48:01.011 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-08-31T05:48:01.011 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-08-31T05:48:01.011 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-08-31T05:48:01.011 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-08-31T05:48:01.011 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-08-31T05:48:01.011 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-08-31T05:48:01.011 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves/w1_smem.ko 2024-08-31T05:48:01.011 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves/w1_therm.ko 2024-08-31T05:48:01.011 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/wire.ko 2024-08-31T05:48:01.012 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog 2024-08-31T05:48:01.012 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/alim1535_wdt.ko 2024-08-31T05:48:01.012 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/alim7101_wdt.ko 2024-08-31T05:48:01.012 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/f71808e_wdt.ko 2024-08-31T05:48:01.012 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/hpwdt.ko 2024-08-31T05:48:01.012 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/i6300esb.ko 2024-08-31T05:48:01.012 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-08-31T05:48:01.012 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/iTCO_wdt.ko 2024-08-31T05:48:01.012 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/ib700wdt.ko 2024-08-31T05:48:01.012 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/ibmasr.ko 2024-08-31T05:48:01.012 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-08-31T05:48:01.012 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/it8712f_wdt.ko 2024-08-31T05:48:01.012 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/it87_wdt.ko 2024-08-31T05:48:01.012 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/machzwd.ko 2024-08-31T05:48:01.012 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/nv_tco.ko 2024-08-31T05:48:01.013 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/pcwd_pci.ko 2024-08-31T05:48:01.013 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/pcwd_usb.ko 2024-08-31T05:48:01.013 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-08-31T05:48:01.013 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/sch311x_wdt.ko 2024-08-31T05:48:01.013 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/softdog.ko 2024-08-31T05:48:01.013 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/sp5100_tco.ko 2024-08-31T05:48:01.013 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/via_wdt.ko 2024-08-31T05:48:01.013 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-08-31T05:48:01.013 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/w83877f_wdt.ko 2024-08-31T05:48:01.013 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/w83977f_wdt.ko 2024-08-31T05:48:01.013 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/wdt_pci.ko 2024-08-31T05:48:01.013 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/xen_wdt.ko 2024-08-31T05:48:01.013 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/xen 2024-08-31T05:48:01.013 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/xen/xen-acpi-processor.ko 2024-08-31T05:48:01.013 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/xen/xen-evtchn.ko 2024-08-31T05:48:01.014 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/xen/xen-gntalloc.ko 2024-08-31T05:48:01.014 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/xen/xen-gntdev.ko 2024-08-31T05:48:01.014 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/xen/xen-pciback 2024-08-31T05:48:01.014 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-08-31T05:48:01.014 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/xen/xen-privcmd.ko 2024-08-31T05:48:01.014 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/xen/xenfs 2024-08-31T05:48:01.014 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/xen/xenfs/xenfs.ko 2024-08-31T05:48:01.014 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs 2024-08-31T05:48:01.014 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/9p 2024-08-31T05:48:01.014 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/9p/9p.ko 2024-08-31T05:48:01.014 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/affs 2024-08-31T05:48:01.014 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/affs/affs.ko 2024-08-31T05:48:01.014 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/befs 2024-08-31T05:48:01.014 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/befs/befs.ko 2024-08-31T05:48:01.015 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/binfmt_misc.ko 2024-08-31T05:48:01.015 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/btrfs 2024-08-31T05:48:01.015 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/btrfs/btrfs.ko 2024-08-31T05:48:01.015 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/cachefiles 2024-08-31T05:48:01.015 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/cachefiles/cachefiles.ko 2024-08-31T05:48:01.015 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ceph 2024-08-31T05:48:01.015 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ceph/ceph.ko 2024-08-31T05:48:01.015 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/coda 2024-08-31T05:48:01.015 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/coda/coda.ko 2024-08-31T05:48:01.015 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/cramfs 2024-08-31T05:48:01.015 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/cramfs/cramfs.ko 2024-08-31T05:48:01.015 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/dlm 2024-08-31T05:48:01.015 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/dlm/dlm.ko 2024-08-31T05:48:01.015 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ecryptfs 2024-08-31T05:48:01.015 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ecryptfs/ecryptfs.ko 2024-08-31T05:48:01.016 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/f2fs 2024-08-31T05:48:01.016 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/f2fs/f2fs.ko 2024-08-31T05:48:01.016 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/fat 2024-08-31T05:48:01.016 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/fat/fat.ko 2024-08-31T05:48:01.016 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/fat/msdos.ko 2024-08-31T05:48:01.016 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/fat/vfat.ko 2024-08-31T05:48:01.016 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/fuse 2024-08-31T05:48:01.016 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/fuse/cuse.ko 2024-08-31T05:48:01.016 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/fuse/fuse.ko 2024-08-31T05:48:01.016 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/gfs2 2024-08-31T05:48:01.016 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/gfs2/gfs2.ko 2024-08-31T05:48:01.016 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/hfs 2024-08-31T05:48:01.016 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/hfs/hfs.ko 2024-08-31T05:48:01.016 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/hfsplus 2024-08-31T05:48:01.017 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/hfsplus/hfsplus.ko 2024-08-31T05:48:01.017 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/isofs 2024-08-31T05:48:01.017 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/isofs/isofs.ko 2024-08-31T05:48:01.017 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/jfs 2024-08-31T05:48:01.017 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/jfs/jfs.ko 2024-08-31T05:48:01.017 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/lockd 2024-08-31T05:48:01.017 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/lockd/lockd.ko 2024-08-31T05:48:01.017 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/minix 2024-08-31T05:48:01.017 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/minix/minix.ko 2024-08-31T05:48:01.017 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/netfs 2024-08-31T05:48:01.017 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/netfs/netfs.ko 2024-08-31T05:48:01.017 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs 2024-08-31T05:48:01.017 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs/blocklayout 2024-08-31T05:48:01.017 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-08-31T05:48:01.017 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs/filelayout 2024-08-31T05:48:01.018 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-08-31T05:48:01.018 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs/flexfilelayout 2024-08-31T05:48:01.018 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-08-31T05:48:01.018 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs/nfs.ko 2024-08-31T05:48:01.018 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs/nfsv3.ko 2024-08-31T05:48:01.018 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs/nfsv4.ko 2024-08-31T05:48:01.018 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs_common 2024-08-31T05:48:01.018 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs_common/grace.ko 2024-08-31T05:48:01.018 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs_common/nfs_acl.ko 2024-08-31T05:48:01.018 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfsd 2024-08-31T05:48:01.018 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfsd/nfsd.ko 2024-08-31T05:48:01.018 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nilfs2 2024-08-31T05:48:01.018 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nilfs2/nilfs2.ko 2024-08-31T05:48:01.018 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls 2024-08-31T05:48:01.018 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/mac-celtic.ko 2024-08-31T05:48:01.019 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/mac-centeuro.ko 2024-08-31T05:48:01.019 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/mac-croatian.ko 2024-08-31T05:48:01.019 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/mac-cyrillic.ko 2024-08-31T05:48:01.019 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/mac-gaelic.ko 2024-08-31T05:48:01.019 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/mac-greek.ko 2024-08-31T05:48:01.019 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/mac-iceland.ko 2024-08-31T05:48:01.019 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/mac-inuit.ko 2024-08-31T05:48:01.019 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/mac-roman.ko 2024-08-31T05:48:01.019 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/mac-romanian.ko 2024-08-31T05:48:01.019 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/mac-turkish.ko 2024-08-31T05:48:01.019 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp1250.ko 2024-08-31T05:48:01.019 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp1251.ko 2024-08-31T05:48:01.019 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp1255.ko 2024-08-31T05:48:01.019 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp737.ko 2024-08-31T05:48:01.019 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp775.ko 2024-08-31T05:48:01.020 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp850.ko 2024-08-31T05:48:01.020 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp852.ko 2024-08-31T05:48:01.020 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp855.ko 2024-08-31T05:48:01.020 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp857.ko 2024-08-31T05:48:01.020 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp860.ko 2024-08-31T05:48:01.020 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp861.ko 2024-08-31T05:48:01.020 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp862.ko 2024-08-31T05:48:01.020 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp863.ko 2024-08-31T05:48:01.020 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp864.ko 2024-08-31T05:48:01.021 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp865.ko 2024-08-31T05:48:01.021 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp866.ko 2024-08-31T05:48:01.021 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp869.ko 2024-08-31T05:48:01.021 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp874.ko 2024-08-31T05:48:01.021 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp932.ko 2024-08-31T05:48:01.021 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp936.ko 2024-08-31T05:48:01.021 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp949.ko 2024-08-31T05:48:01.021 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp950.ko 2024-08-31T05:48:01.021 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_euc-jp.ko 2024-08-31T05:48:01.021 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_iso8859-1.ko 2024-08-31T05:48:01.021 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_iso8859-13.ko 2024-08-31T05:48:01.021 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_iso8859-14.ko 2024-08-31T05:48:01.021 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_iso8859-15.ko 2024-08-31T05:48:01.021 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_iso8859-2.ko 2024-08-31T05:48:01.022 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_iso8859-3.ko 2024-08-31T05:48:01.022 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_iso8859-4.ko 2024-08-31T05:48:01.022 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_iso8859-5.ko 2024-08-31T05:48:01.022 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_iso8859-6.ko 2024-08-31T05:48:01.022 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_iso8859-7.ko 2024-08-31T05:48:01.022 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_iso8859-9.ko 2024-08-31T05:48:01.022 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_koi8-r.ko 2024-08-31T05:48:01.022 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_koi8-ru.ko 2024-08-31T05:48:01.022 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_koi8-u.ko 2024-08-31T05:48:01.022 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_ucs2_utils.ko 2024-08-31T05:48:01.022 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_utf8.ko 2024-08-31T05:48:01.022 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ocfs2 2024-08-31T05:48:01.022 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ocfs2/cluster 2024-08-31T05:48:01.022 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-08-31T05:48:01.022 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ocfs2/dlm 2024-08-31T05:48:01.023 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-08-31T05:48:01.023 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ocfs2/dlmfs 2024-08-31T05:48:01.023 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-08-31T05:48:01.023 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ocfs2/ocfs2.ko 2024-08-31T05:48:01.023 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-08-31T05:48:01.023 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-08-31T05:48:01.023 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-08-31T05:48:01.023 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/overlayfs 2024-08-31T05:48:01.023 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/overlayfs/overlay.ko 2024-08-31T05:48:01.023 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/pstore 2024-08-31T05:48:01.023 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/pstore/ramoops.ko 2024-08-31T05:48:01.023 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/reiserfs 2024-08-31T05:48:01.024 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/reiserfs/reiserfs.ko 2024-08-31T05:48:01.024 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/romfs 2024-08-31T05:48:01.024 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/romfs/romfs.ko 2024-08-31T05:48:01.024 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/smb 2024-08-31T05:48:01.024 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/smb/client 2024-08-31T05:48:01.024 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/smb/client/cifs.ko 2024-08-31T05:48:01.024 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/smb/common 2024-08-31T05:48:01.024 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/smb/common/cifs_arc4.ko 2024-08-31T05:48:01.024 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/smb/common/cifs_md4.ko 2024-08-31T05:48:01.024 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/squashfs 2024-08-31T05:48:01.024 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/squashfs/squashfs.ko 2024-08-31T05:48:01.024 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/sysv 2024-08-31T05:48:01.024 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/sysv/sysv.ko 2024-08-31T05:48:01.024 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/udf 2024-08-31T05:48:01.025 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/udf/udf.ko 2024-08-31T05:48:01.025 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ufs 2024-08-31T05:48:01.025 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ufs/ufs.ko 2024-08-31T05:48:01.025 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/xfs 2024-08-31T05:48:01.025 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/xfs/xfs.ko 2024-08-31T05:48:01.025 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/kernel 2024-08-31T05:48:01.025 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/kernel/rcu 2024-08-31T05:48:01.025 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/kernel/rcu/rcutorture.ko 2024-08-31T05:48:01.025 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/kernel/torture.ko 2024-08-31T05:48:01.025 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/kernel/trace 2024-08-31T05:48:01.025 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-08-31T05:48:01.025 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib 2024-08-31T05:48:01.025 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/asn1_encoder.ko 2024-08-31T05:48:01.025 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/crc-itu-t.ko 2024-08-31T05:48:01.026 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/crc8.ko 2024-08-31T05:48:01.026 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/crypto 2024-08-31T05:48:01.026 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/crypto/libaescfb.ko 2024-08-31T05:48:01.026 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/crypto/libarc4.ko 2024-08-31T05:48:01.026 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/crypto/libdes.ko 2024-08-31T05:48:01.026 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/libcrc32c.ko 2024-08-31T05:48:01.026 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/lru_cache.ko 2024-08-31T05:48:01.026 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/lz4 2024-08-31T05:48:01.026 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/lz4/lz4_compress.ko 2024-08-31T05:48:01.026 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/lz4/lz4hc_compress.ko 2024-08-31T05:48:01.026 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/math 2024-08-31T05:48:01.026 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/math/cordic.ko 2024-08-31T05:48:01.026 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/raid6 2024-08-31T05:48:01.026 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/raid6/raid6_pq.ko 2024-08-31T05:48:01.026 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/reed_solomon 2024-08-31T05:48:01.027 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/reed_solomon/reed_solomon.ko 2024-08-31T05:48:01.027 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/ts_bm.ko 2024-08-31T05:48:01.027 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/ts_fsm.ko 2024-08-31T05:48:01.027 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/ts_kmp.ko 2024-08-31T05:48:01.027 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/zstd 2024-08-31T05:48:01.027 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/zstd/zstd_compress.ko 2024-08-31T05:48:01.027 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/mm 2024-08-31T05:48:01.027 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/mm/hwpoison-inject.ko 2024-08-31T05:48:01.027 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net 2024-08-31T05:48:01.027 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/6lowpan 2024-08-31T05:48:01.027 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/6lowpan/6lowpan.ko 2024-08-31T05:48:01.027 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/6lowpan/nhc_dest.ko 2024-08-31T05:48:01.027 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/6lowpan/nhc_fragment.ko 2024-08-31T05:48:01.027 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/6lowpan/nhc_hop.ko 2024-08-31T05:48:01.027 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/6lowpan/nhc_ipv6.ko 2024-08-31T05:48:01.027 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/6lowpan/nhc_mobility.ko 2024-08-31T05:48:01.028 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/6lowpan/nhc_routing.ko 2024-08-31T05:48:01.028 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/6lowpan/nhc_udp.ko 2024-08-31T05:48:01.028 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/802 2024-08-31T05:48:01.028 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/802/garp.ko 2024-08-31T05:48:01.028 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/802/mrp.ko 2024-08-31T05:48:01.028 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/802/p8022.ko 2024-08-31T05:48:01.028 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/802/psnap.ko 2024-08-31T05:48:01.028 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/802/stp.ko 2024-08-31T05:48:01.028 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/8021q 2024-08-31T05:48:01.028 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/8021q/8021q.ko 2024-08-31T05:48:01.028 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/9p 2024-08-31T05:48:01.028 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/9p/9pnet.ko 2024-08-31T05:48:01.028 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/9p/9pnet_fd.ko 2024-08-31T05:48:01.028 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/9p/9pnet_rdma.ko 2024-08-31T05:48:01.029 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/9p/9pnet_virtio.ko 2024-08-31T05:48:01.029 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/batman-adv 2024-08-31T05:48:01.029 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/batman-adv/batman-adv.ko 2024-08-31T05:48:01.029 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge 2024-08-31T05:48:01.029 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/br_netfilter.ko 2024-08-31T05:48:01.029 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/bridge.ko 2024-08-31T05:48:01.029 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter 2024-08-31T05:48:01.029 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-08-31T05:48:01.029 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_among.ko 2024-08-31T05:48:01.029 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_arp.ko 2024-08-31T05:48:01.029 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-08-31T05:48:01.029 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-08-31T05:48:01.029 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_ip.ko 2024-08-31T05:48:01.029 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-08-31T05:48:01.030 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_limit.ko 2024-08-31T05:48:01.030 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_log.ko 2024-08-31T05:48:01.030 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_mark.ko 2024-08-31T05:48:01.030 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-08-31T05:48:01.030 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-08-31T05:48:01.030 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-08-31T05:48:01.030 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-08-31T05:48:01.030 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_snat.ko 2024-08-31T05:48:01.030 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_stp.ko 2024-08-31T05:48:01.030 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-08-31T05:48:01.030 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-08-31T05:48:01.030 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-08-31T05:48:01.030 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-08-31T05:48:01.030 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebtables.ko 2024-08-31T05:48:01.030 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-08-31T05:48:01.030 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-08-31T05:48:01.031 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-08-31T05:48:01.031 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ceph 2024-08-31T05:48:01.031 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ceph/libceph.ko 2024-08-31T05:48:01.031 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/core 2024-08-31T05:48:01.031 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/core/failover.ko 2024-08-31T05:48:01.031 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/core/pktgen.ko 2024-08-31T05:48:01.031 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dccp 2024-08-31T05:48:01.031 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dccp/dccp.ko 2024-08-31T05:48:01.031 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dccp/dccp_diag.ko 2024-08-31T05:48:01.031 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dccp/dccp_ipv4.ko 2024-08-31T05:48:01.031 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dccp/dccp_ipv6.ko 2024-08-31T05:48:01.031 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dns_resolver 2024-08-31T05:48:01.031 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dns_resolver/dns_resolver.ko 2024-08-31T05:48:01.031 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dsa 2024-08-31T05:48:01.031 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dsa/dsa_core.ko 2024-08-31T05:48:01.032 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dsa/tag_brcm.ko 2024-08-31T05:48:01.032 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dsa/tag_dsa.ko 2024-08-31T05:48:01.032 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dsa/tag_none.ko 2024-08-31T05:48:01.032 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dsa/tag_trailer.ko 2024-08-31T05:48:01.032 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ieee802154 2024-08-31T05:48:01.032 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ieee802154/6lowpan 2024-08-31T05:48:01.032 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-08-31T05:48:01.032 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ieee802154/ieee802154.ko 2024-08-31T05:48:01.032 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ieee802154/ieee802154_socket.ko 2024-08-31T05:48:01.032 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4 2024-08-31T05:48:01.032 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/ah4.ko 2024-08-31T05:48:01.032 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/esp4.ko 2024-08-31T05:48:01.032 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/fou.ko 2024-08-31T05:48:01.032 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/gre.ko 2024-08-31T05:48:01.033 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/inet_diag.ko 2024-08-31T05:48:01.033 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/ip_gre.ko 2024-08-31T05:48:01.033 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/ip_tunnel.ko 2024-08-31T05:48:01.033 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/ip_vti.ko 2024-08-31T05:48:01.033 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/ipcomp.ko 2024-08-31T05:48:01.033 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/ipip.ko 2024-08-31T05:48:01.033 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter 2024-08-31T05:48:01.033 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/arp_tables.ko 2024-08-31T05:48:01.033 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-08-31T05:48:01.033 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-08-31T05:48:01.033 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-08-31T05:48:01.033 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-08-31T05:48:01.033 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-08-31T05:48:01.033 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-08-31T05:48:01.033 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-08-31T05:48:01.034 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-08-31T05:48:01.034 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-08-31T05:48:01.034 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/iptable_security.ko 2024-08-31T05:48:01.034 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-08-31T05:48:01.034 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-08-31T05:48:01.034 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-08-31T05:48:01.034 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-08-31T05:48:01.034 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-08-31T05:48:01.034 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-08-31T05:48:01.034 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-08-31T05:48:01.034 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-08-31T05:48:01.034 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-08-31T05:48:01.034 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-08-31T05:48:01.034 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_bic.ko 2024-08-31T05:48:01.034 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_dctcp.ko 2024-08-31T05:48:01.034 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_diag.ko 2024-08-31T05:48:01.035 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_highspeed.ko 2024-08-31T05:48:01.035 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_htcp.ko 2024-08-31T05:48:01.035 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_hybla.ko 2024-08-31T05:48:01.035 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_illinois.ko 2024-08-31T05:48:01.035 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_lp.ko 2024-08-31T05:48:01.035 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_scalable.ko 2024-08-31T05:48:01.035 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_vegas.ko 2024-08-31T05:48:01.035 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_veno.ko 2024-08-31T05:48:01.035 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_westwood.ko 2024-08-31T05:48:01.035 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_yeah.ko 2024-08-31T05:48:01.035 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tunnel4.ko 2024-08-31T05:48:01.035 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/udp_diag.ko 2024-08-31T05:48:01.035 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/udp_tunnel.ko 2024-08-31T05:48:01.036 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/xfrm4_tunnel.ko 2024-08-31T05:48:01.036 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6 2024-08-31T05:48:01.036 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/ah6.ko 2024-08-31T05:48:01.036 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/esp6.ko 2024-08-31T05:48:01.036 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/fou6.ko 2024-08-31T05:48:01.036 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/ip6_tunnel.ko 2024-08-31T05:48:01.036 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-08-31T05:48:01.036 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/ip6_vti.ko 2024-08-31T05:48:01.036 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/ipcomp6.ko 2024-08-31T05:48:01.036 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter 2024-08-31T05:48:01.036 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-08-31T05:48:01.036 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-08-31T05:48:01.036 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-08-31T05:48:01.036 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-08-31T05:48:01.037 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-08-31T05:48:01.037 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-08-31T05:48:01.037 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-08-31T05:48:01.037 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-08-31T05:48:01.037 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-08-31T05:48:01.037 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-08-31T05:48:01.037 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-08-31T05:48:01.037 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-08-31T05:48:01.037 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-08-31T05:48:01.037 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-08-31T05:48:01.037 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-08-31T05:48:01.037 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-08-31T05:48:01.037 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-08-31T05:48:01.037 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-08-31T05:48:01.037 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-08-31T05:48:01.038 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-08-31T05:48:01.038 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-08-31T05:48:01.038 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-08-31T05:48:01.038 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-08-31T05:48:01.038 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-08-31T05:48:01.038 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/sit.ko 2024-08-31T05:48:01.038 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/tunnel6.ko 2024-08-31T05:48:01.038 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/xfrm6_tunnel.ko 2024-08-31T05:48:01.038 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/key 2024-08-31T05:48:01.038 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/key/af_key.ko 2024-08-31T05:48:01.038 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/l2tp 2024-08-31T05:48:01.038 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/l2tp/l2tp_core.ko 2024-08-31T05:48:01.038 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/l2tp/l2tp_debugfs.ko 2024-08-31T05:48:01.038 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/l2tp/l2tp_eth.ko 2024-08-31T05:48:01.038 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/l2tp/l2tp_ip.ko 2024-08-31T05:48:01.039 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/l2tp/l2tp_ip6.ko 2024-08-31T05:48:01.039 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/l2tp/l2tp_netlink.ko 2024-08-31T05:48:01.039 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/l2tp/l2tp_ppp.ko 2024-08-31T05:48:01.039 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/llc 2024-08-31T05:48:01.039 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/llc/llc.ko 2024-08-31T05:48:01.039 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/mac802154 2024-08-31T05:48:01.039 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/mac802154/mac802154.ko 2024-08-31T05:48:01.039 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/mpls 2024-08-31T05:48:01.039 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/mpls/mpls_gso.ko 2024-08-31T05:48:01.039 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter 2024-08-31T05:48:01.039 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset 2024-08-31T05:48:01.039 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set.ko 2024-08-31T05:48:01.039 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-08-31T05:48:01.039 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-08-31T05:48:01.039 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-08-31T05:48:01.040 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-08-31T05:48:01.040 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-08-31T05:48:01.040 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-08-31T05:48:01.040 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-08-31T05:48:01.040 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-08-31T05:48:01.040 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-08-31T05:48:01.040 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-08-31T05:48:01.040 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-08-31T05:48:01.040 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-08-31T05:48:01.040 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-08-31T05:48:01.040 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-08-31T05:48:01.040 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-08-31T05:48:01.041 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs 2024-08-31T05:48:01.041 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs.ko 2024-08-31T05:48:01.041 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-08-31T05:48:01.041 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-08-31T05:48:01.041 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-08-31T05:48:01.041 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-08-31T05:48:01.041 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-08-31T05:48:01.041 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-08-31T05:48:01.041 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-08-31T05:48:01.041 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-08-31T05:48:01.041 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-08-31T05:48:01.041 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-08-31T05:48:01.042 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-08-31T05:48:01.042 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-08-31T05:48:01.042 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-08-31T05:48:01.042 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conncount.ko 2024-08-31T05:48:01.042 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack.ko 2024-08-31T05:48:01.042 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-08-31T05:48:01.042 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-08-31T05:48:01.042 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-08-31T05:48:01.042 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_h323.ko 2024-08-31T05:48:01.042 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_irc.ko 2024-08-31T05:48:01.042 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-08-31T05:48:01.042 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-08-31T05:48:01.042 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-08-31T05:48:01.042 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_sane.ko 2024-08-31T05:48:01.043 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_sip.ko 2024-08-31T05:48:01.043 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-08-31T05:48:01.043 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-08-31T05:48:01.043 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_dup_netdev.ko 2024-08-31T05:48:01.043 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_flow_table.ko 2024-08-31T05:48:01.043 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_flow_table_inet.ko 2024-08-31T05:48:01.043 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_log_syslog.ko 2024-08-31T05:48:01.043 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_nat.ko 2024-08-31T05:48:01.043 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_nat_amanda.ko 2024-08-31T05:48:01.043 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_nat_ftp.ko 2024-08-31T05:48:01.043 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_nat_irc.ko 2024-08-31T05:48:01.043 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_nat_sip.ko 2024-08-31T05:48:01.043 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_nat_tftp.ko 2024-08-31T05:48:01.044 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_synproxy_core.ko 2024-08-31T05:48:01.044 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_tables.ko 2024-08-31T05:48:01.044 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nfnetlink.ko 2024-08-31T05:48:01.044 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nfnetlink_acct.ko 2024-08-31T05:48:01.044 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nfnetlink_log.ko 2024-08-31T05:48:01.044 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nfnetlink_osf.ko 2024-08-31T05:48:01.044 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nfnetlink_queue.ko 2024-08-31T05:48:01.044 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_chain_nat.ko 2024-08-31T05:48:01.044 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_compat.ko 2024-08-31T05:48:01.044 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_ct.ko 2024-08-31T05:48:01.044 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_dup_netdev.ko 2024-08-31T05:48:01.044 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_fib.ko 2024-08-31T05:48:01.044 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_fwd_netdev.ko 2024-08-31T05:48:01.044 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_hash.ko 2024-08-31T05:48:01.044 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_limit.ko 2024-08-31T05:48:01.044 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_log.ko 2024-08-31T05:48:01.045 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_masq.ko 2024-08-31T05:48:01.045 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_nat.ko 2024-08-31T05:48:01.045 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_numgen.ko 2024-08-31T05:48:01.045 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_queue.ko 2024-08-31T05:48:01.045 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_quota.ko 2024-08-31T05:48:01.045 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_redir.ko 2024-08-31T05:48:01.045 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_reject.ko 2024-08-31T05:48:01.045 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_reject_inet.ko 2024-08-31T05:48:01.045 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_socket.ko 2024-08-31T05:48:01.045 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_tunnel.ko 2024-08-31T05:48:01.045 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_AUDIT.ko 2024-08-31T05:48:01.045 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_CHECKSUM.ko 2024-08-31T05:48:01.045 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_CLASSIFY.ko 2024-08-31T05:48:01.045 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-08-31T05:48:01.045 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_CT.ko 2024-08-31T05:48:01.046 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_DSCP.ko 2024-08-31T05:48:01.046 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_HL.ko 2024-08-31T05:48:01.046 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_HMARK.ko 2024-08-31T05:48:01.046 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_IDLETIMER.ko 2024-08-31T05:48:01.046 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_LED.ko 2024-08-31T05:48:01.046 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_LOG.ko 2024-08-31T05:48:01.046 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_MASQUERADE.ko 2024-08-31T05:48:01.046 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_NETMAP.ko 2024-08-31T05:48:01.046 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_NFLOG.ko 2024-08-31T05:48:01.046 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_NFQUEUE.ko 2024-08-31T05:48:01.046 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_RATEEST.ko 2024-08-31T05:48:01.046 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_REDIRECT.ko 2024-08-31T05:48:01.046 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_SECMARK.ko 2024-08-31T05:48:01.046 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_TCPMSS.ko 2024-08-31T05:48:01.046 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-08-31T05:48:01.047 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_TEE.ko 2024-08-31T05:48:01.047 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_TPROXY.ko 2024-08-31T05:48:01.047 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_TRACE.ko 2024-08-31T05:48:01.047 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_addrtype.ko 2024-08-31T05:48:01.047 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_bpf.ko 2024-08-31T05:48:01.047 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_cgroup.ko 2024-08-31T05:48:01.047 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_cluster.ko 2024-08-31T05:48:01.047 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_comment.ko 2024-08-31T05:48:01.047 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_connbytes.ko 2024-08-31T05:48:01.047 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_connlabel.ko 2024-08-31T05:48:01.047 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_connlimit.ko 2024-08-31T05:48:01.047 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_connmark.ko 2024-08-31T05:48:01.047 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_conntrack.ko 2024-08-31T05:48:01.047 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_cpu.ko 2024-08-31T05:48:01.048 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_dccp.ko 2024-08-31T05:48:01.048 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_devgroup.ko 2024-08-31T05:48:01.048 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_dscp.ko 2024-08-31T05:48:01.048 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_ecn.ko 2024-08-31T05:48:01.048 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_esp.ko 2024-08-31T05:48:01.048 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_hashlimit.ko 2024-08-31T05:48:01.048 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_helper.ko 2024-08-31T05:48:01.048 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_hl.ko 2024-08-31T05:48:01.048 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_ipcomp.ko 2024-08-31T05:48:01.048 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_iprange.ko 2024-08-31T05:48:01.048 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_ipvs.ko 2024-08-31T05:48:01.048 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_l2tp.ko 2024-08-31T05:48:01.048 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_length.ko 2024-08-31T05:48:01.048 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_limit.ko 2024-08-31T05:48:01.048 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_mac.ko 2024-08-31T05:48:01.049 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_mark.ko 2024-08-31T05:48:01.049 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_multiport.ko 2024-08-31T05:48:01.049 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_nat.ko 2024-08-31T05:48:01.049 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_nfacct.ko 2024-08-31T05:48:01.049 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_osf.ko 2024-08-31T05:48:01.049 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_owner.ko 2024-08-31T05:48:01.049 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_physdev.ko 2024-08-31T05:48:01.049 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_pkttype.ko 2024-08-31T05:48:01.049 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_policy.ko 2024-08-31T05:48:01.049 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_quota.ko 2024-08-31T05:48:01.049 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_rateest.ko 2024-08-31T05:48:01.049 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_realm.ko 2024-08-31T05:48:01.049 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_recent.ko 2024-08-31T05:48:01.049 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_sctp.ko 2024-08-31T05:48:01.050 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_set.ko 2024-08-31T05:48:01.050 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_socket.ko 2024-08-31T05:48:01.050 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_state.ko 2024-08-31T05:48:01.050 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_statistic.ko 2024-08-31T05:48:01.050 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_string.ko 2024-08-31T05:48:01.050 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_tcpmss.ko 2024-08-31T05:48:01.050 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_time.ko 2024-08-31T05:48:01.050 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_u32.ko 2024-08-31T05:48:01.050 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netlink 2024-08-31T05:48:01.050 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netlink/netlink_diag.ko 2024-08-31T05:48:01.050 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/nsh 2024-08-31T05:48:01.050 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/nsh/nsh.ko 2024-08-31T05:48:01.050 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/openvswitch 2024-08-31T05:48:01.050 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/openvswitch/openvswitch.ko 2024-08-31T05:48:01.050 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/openvswitch/vport-geneve.ko 2024-08-31T05:48:01.051 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/openvswitch/vport-gre.ko 2024-08-31T05:48:01.051 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/openvswitch/vport-vxlan.ko 2024-08-31T05:48:01.051 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/packet 2024-08-31T05:48:01.051 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/packet/af_packet_diag.ko 2024-08-31T05:48:01.051 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/rds 2024-08-31T05:48:01.051 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/rds/rds.ko 2024-08-31T05:48:01.051 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/rds/rds_rdma.ko 2024-08-31T05:48:01.051 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/rds/rds_tcp.ko 2024-08-31T05:48:01.051 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched 2024-08-31T05:48:01.051 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/act_csum.ko 2024-08-31T05:48:01.051 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/act_gact.ko 2024-08-31T05:48:01.051 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/act_mirred.ko 2024-08-31T05:48:01.051 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/act_nat.ko 2024-08-31T05:48:01.052 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/act_pedit.ko 2024-08-31T05:48:01.052 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/act_police.ko 2024-08-31T05:48:01.052 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/act_simple.ko 2024-08-31T05:48:01.052 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/act_skbedit.ko 2024-08-31T05:48:01.052 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/act_vlan.ko 2024-08-31T05:48:01.052 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/cls_basic.ko 2024-08-31T05:48:01.052 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/cls_bpf.ko 2024-08-31T05:48:01.052 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/cls_flow.ko 2024-08-31T05:48:01.052 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/cls_fw.ko 2024-08-31T05:48:01.052 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/cls_route.ko 2024-08-31T05:48:01.052 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/cls_u32.ko 2024-08-31T05:48:01.052 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/em_cmp.ko 2024-08-31T05:48:01.052 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/em_ipset.ko 2024-08-31T05:48:01.052 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/em_meta.ko 2024-08-31T05:48:01.053 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/em_nbyte.ko 2024-08-31T05:48:01.053 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/em_text.ko 2024-08-31T05:48:01.053 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/em_u32.ko 2024-08-31T05:48:01.053 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_choke.ko 2024-08-31T05:48:01.053 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_codel.ko 2024-08-31T05:48:01.053 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_drr.ko 2024-08-31T05:48:01.053 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_fq.ko 2024-08-31T05:48:01.053 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_gred.ko 2024-08-31T05:48:01.053 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_hfsc.ko 2024-08-31T05:48:01.053 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_hhf.ko 2024-08-31T05:48:01.053 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_htb.ko 2024-08-31T05:48:01.053 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_ingress.ko 2024-08-31T05:48:01.053 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_mqprio.ko 2024-08-31T05:48:01.054 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_mqprio_lib.ko 2024-08-31T05:48:01.054 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_multiq.ko 2024-08-31T05:48:01.054 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_netem.ko 2024-08-31T05:48:01.054 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_pie.ko 2024-08-31T05:48:01.054 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_plug.ko 2024-08-31T05:48:01.054 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_prio.ko 2024-08-31T05:48:01.054 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_qfq.ko 2024-08-31T05:48:01.054 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_red.ko 2024-08-31T05:48:01.054 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_sfb.ko 2024-08-31T05:48:01.054 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_sfq.ko 2024-08-31T05:48:01.054 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_tbf.ko 2024-08-31T05:48:01.054 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_teql.ko 2024-08-31T05:48:01.054 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sctp 2024-08-31T05:48:01.054 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sctp/sctp.ko 2024-08-31T05:48:01.054 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sctp/sctp_diag.ko 2024-08-31T05:48:01.055 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sunrpc 2024-08-31T05:48:01.055 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sunrpc/auth_gss 2024-08-31T05:48:01.055 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-08-31T05:48:01.055 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-08-31T05:48:01.055 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sunrpc/sunrpc.ko 2024-08-31T05:48:01.055 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sunrpc/xprtrdma 2024-08-31T05:48:01.055 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-08-31T05:48:01.055 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/tipc 2024-08-31T05:48:01.055 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/tipc/tipc.ko 2024-08-31T05:48:01.055 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/tipc/tipc_diag.ko 2024-08-31T05:48:01.055 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/unix 2024-08-31T05:48:01.055 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/unix/unix_diag.ko 2024-08-31T05:48:01.055 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/vmw_vsock 2024-08-31T05:48:01.055 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-08-31T05:48:01.056 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-08-31T05:48:01.056 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/vmw_vsock/vsock.ko 2024-08-31T05:48:01.056 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/vmw_vsock/vsock_diag.ko 2024-08-31T05:48:01.056 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/vmw_vsock/vsock_loopback.ko 2024-08-31T05:48:01.056 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/xfrm 2024-08-31T05:48:01.056 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/xfrm/xfrm_ipcomp.ko 2024-08-31T05:48:01.056 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/security 2024-08-31T05:48:01.056 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/security/keys 2024-08-31T05:48:01.056 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/security/keys/encrypted-keys 2024-08-31T05:48:01.056 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-08-31T05:48:01.056 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/security/keys/trusted-keys 2024-08-31T05:48:01.056 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/security/keys/trusted-keys/trusted.ko 2024-08-31T05:48:01.056 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.alias 2024-08-31T05:48:01.056 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.alias.bin 2024-08-31T05:48:01.057 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.builtin 2024-08-31T05:48:01.057 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.builtin.alias.bin 2024-08-31T05:48:01.057 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.builtin.bin 2024-08-31T05:48:01.057 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.builtin.modinfo 2024-08-31T05:48:01.057 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.dep 2024-08-31T05:48:01.057 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.dep.bin 2024-08-31T05:48:01.057 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.devname 2024-08-31T05:48:01.057 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.order 2024-08-31T05:48:01.057 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.softdep 2024-08-31T05:48:01.057 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.symbols 2024-08-31T05:48:01.057 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.symbols.bin 2024-08-31T05:48:01.057 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.weakdep 2024-08-31T05:48:01.057 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/vmlinuz 2024-08-31T05:48:01.061 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g8ad8b8b16098 2024-08-31T05:48:01.061 DEBUG:teuthology.orchestra.run.smithi016:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-08-31T05:48:01.085 INFO:teuthology.orchestra.run.smithi016.stdout:postinstall scriptlet (using /bin/sh): 2024-08-31T05:48:01.085 INFO:teuthology.orchestra.run.smithi016.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-08-31T05:48:01.085 INFO:teuthology.orchestra.run.smithi016.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g8ad8b8b16098 /lib/modules/6.11.0-rc5-g8ad8b8b16098/vmlinuz 2024-08-31T05:48:01.085 INFO:teuthology.orchestra.run.smithi016.stdout:fi 2024-08-31T05:48:01.085 INFO:teuthology.orchestra.run.smithi016.stdout:for file in vmlinuz System.map config; do 2024-08-31T05:48:01.085 INFO:teuthology.orchestra.run.smithi016.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g8ad8b8b16098/${file}" "/boot/${file}-6.11.0-rc5-g8ad8b8b16098"; then 2024-08-31T05:48:01.085 INFO:teuthology.orchestra.run.smithi016.stdout: cp "/lib/modules/6.11.0-rc5-g8ad8b8b16098/${file}" "/boot/${file}-6.11.0-rc5-g8ad8b8b16098" 2024-08-31T05:48:01.085 INFO:teuthology.orchestra.run.smithi016.stdout: fi 2024-08-31T05:48:01.085 INFO:teuthology.orchestra.run.smithi016.stdout:done 2024-08-31T05:48:01.086 INFO:teuthology.orchestra.run.smithi016.stdout:if [ -d "/lib/modules/6.11.0-rc5-g8ad8b8b16098/dtb" ] && \ 2024-08-31T05:48:01.086 INFO:teuthology.orchestra.run.smithi016.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g8ad8b8b16098/dtb" "/boot/dtb-6.11.0-rc5-g8ad8b8b16098" >/dev/null 2>&1; then 2024-08-31T05:48:01.086 INFO:teuthology.orchestra.run.smithi016.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g8ad8b8b16098" 2024-08-31T05:48:01.086 INFO:teuthology.orchestra.run.smithi016.stdout: cp -r "/lib/modules/6.11.0-rc5-g8ad8b8b16098/dtb" "/boot/dtb-6.11.0-rc5-g8ad8b8b16098" 2024-08-31T05:48:01.086 INFO:teuthology.orchestra.run.smithi016.stdout:fi 2024-08-31T05:48:01.086 INFO:teuthology.orchestra.run.smithi016.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.dep" ]; then 2024-08-31T05:48:01.086 INFO:teuthology.orchestra.run.smithi016.stdout: /usr/sbin/depmod 6.11.0-rc5-g8ad8b8b16098 2024-08-31T05:48:01.086 INFO:teuthology.orchestra.run.smithi016.stdout:fi 2024-08-31T05:48:01.086 INFO:teuthology.orchestra.run.smithi016.stdout:preuninstall scriptlet (using /bin/sh): 2024-08-31T05:48:01.086 INFO:teuthology.orchestra.run.smithi016.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-08-31T05:48:01.086 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-install remove 6.11.0-rc5-g8ad8b8b16098 2024-08-31T05:48:01.086 INFO:teuthology.orchestra.run.smithi016.stdout:fi 2024-08-31T05:48:01.087 DEBUG:teuthology.orchestra.run.smithi016:> sudo rpm -qi grub2-tools 2024-08-31T05:48:01.163 INFO:teuthology.orchestra.run.smithi016.stdout:Name : grub2-tools 2024-08-31T05:48:01.163 INFO:teuthology.orchestra.run.smithi016.stdout:Epoch : 1 2024-08-31T05:48:01.164 INFO:teuthology.orchestra.run.smithi016.stdout:Version : 2.06 2024-08-31T05:48:01.164 INFO:teuthology.orchestra.run.smithi016.stdout:Release : 82.el9 2024-08-31T05:48:01.164 INFO:teuthology.orchestra.run.smithi016.stdout:Architecture: x86_64 2024-08-31T05:48:01.164 INFO:teuthology.orchestra.run.smithi016.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-08-31T05:48:01.164 INFO:teuthology.orchestra.run.smithi016.stdout:Group : Unspecified 2024-08-31T05:48:01.164 INFO:teuthology.orchestra.run.smithi016.stdout:Size : 8274670 2024-08-31T05:48:01.164 INFO:teuthology.orchestra.run.smithi016.stdout:License : GPLv3+ 2024-08-31T05:48:01.164 INFO:teuthology.orchestra.run.smithi016.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-08-31T05:48:01.164 INFO:teuthology.orchestra.run.smithi016.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-08-31T05:48:01.164 INFO:teuthology.orchestra.run.smithi016.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-08-31T05:48:01.164 INFO:teuthology.orchestra.run.smithi016.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-08-31T05:48:01.164 INFO:teuthology.orchestra.run.smithi016.stdout:Packager : builder@centos.org 2024-08-31T05:48:01.164 INFO:teuthology.orchestra.run.smithi016.stdout:Vendor : CentOS 2024-08-31T05:48:01.164 INFO:teuthology.orchestra.run.smithi016.stdout:URL : http://www.gnu.org/software/grub/ 2024-08-31T05:48:01.165 INFO:teuthology.orchestra.run.smithi016.stdout:Summary : Support tools for GRUB. 2024-08-31T05:48:01.165 INFO:teuthology.orchestra.run.smithi016.stdout:Description : 2024-08-31T05:48:01.165 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T05:48:01.165 INFO:teuthology.orchestra.run.smithi016.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-08-31T05:48:01.165 INFO:teuthology.orchestra.run.smithi016.stdout:customizable bootloader with modular architecture. It supports a rich 2024-08-31T05:48:01.165 INFO:teuthology.orchestra.run.smithi016.stdout:variety of kernel formats, file systems, computer architectures and 2024-08-31T05:48:01.165 INFO:teuthology.orchestra.run.smithi016.stdout:hardware devices. 2024-08-31T05:48:01.165 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T05:48:01.165 INFO:teuthology.orchestra.run.smithi016.stdout:This subpackage provides tools for support of all platforms. 2024-08-31T05:48:01.166 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-08-31T05:48:01.166 INFO:teuthology.task.kernel:Updating grub on smithi016 to boot 6.11.0-rc5-g8ad8b8b16098 2024-08-31T05:48:01.166 DEBUG:teuthology.orchestra.run.smithi016:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-08-31T05:48:01.729 INFO:teuthology.orchestra.run.smithi016.stderr:Generating grub configuration file ... 2024-08-31T05:48:03.107 INFO:teuthology.orchestra.run.smithi016.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-08-31T05:48:03.140 INFO:teuthology.orchestra.run.smithi016.stderr:done 2024-08-31T05:48:03.143 DEBUG:teuthology.orchestra.run.smithi016:> mktemp 2024-08-31T05:48:03.158 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/tmp.u3oDwcwQex 2024-08-31T05:48:03.158 DEBUG:teuthology.orchestra.run.smithi016:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.u3oDwcwQex 2024-08-31T05:48:03.224 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0666 /tmp/tmp.u3oDwcwQex 2024-08-31T05:48:03.372 DEBUG:teuthology.orchestra.remote:smithi016:/tmp/tmp.u3oDwcwQex is 6KB 2024-08-31T05:48:03.421 DEBUG:teuthology.orchestra.run.smithi016:> rm -fr /tmp/tmp.u3oDwcwQex 2024-08-31T05:48:03.435 DEBUG:teuthology.orchestra.run.smithi016:> sudo /bin/ls /boot/loader/entries || true 2024-08-31T05:48:03.498 INFO:teuthology.orchestra.run.smithi016.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-08-31T05:48:03.499 INFO:teuthology.orchestra.run.smithi016.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-08-31T05:48:03.499 INFO:teuthology.orchestra.run.smithi016.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-08-31T05:48:03.499 INFO:teuthology.orchestra.run.smithi016.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g8ad8b8b16098.conf 2024-08-31T05:48:03.500 DEBUG:teuthology.orchestra.run.smithi016:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g8ad8b8b16098 2024-08-31T05:48:03.645 DEBUG:teuthology.orchestra.run.smithi016:> sudo shutdown -r now 2024-08-31T05:48:33.648 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-08-31T05:48:33.649 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-08-31T05:48:33.649 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-31T05:48:52.157 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.16 2024-08-31T05:49:01.158 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-08-31T05:49:01.159 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-31T05:49:04.225 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.16 2024-08-31T05:49:16.225 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-08-31T05:49:16.226 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-31T05:49:48.128 DEBUG:teuthology.orchestra.run.smithi016:> true 2024-08-31T05:49:48.610 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-08-31T05:49:48.610 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g8ad8b8b16098"... 2024-08-31T05:49:48.610 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2024-08-31T05:49:48.644 INFO:teuthology.orchestra.run.smithi016.stdout:6.11.0-rc5-g8ad8b8b16098 2024-08-31T05:49:48.644 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g8ad8b8b16098 vs 6.11.0-rc5-g8ad8b8b16098 2024-08-31T05:49:48.645 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-08-31T05:49:48.645 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-08-31T05:49:49.645 DEBUG:teuthology.parallel:result is None 2024-08-31T05:49:49.646 INFO:teuthology.run_tasks:Running task internal.base... 2024-08-31T05:49:49.656 INFO:teuthology.task.internal:Creating test directory... 2024-08-31T05:49:49.656 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-31T05:49:49.660 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-31T05:49:49.662 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-31T05:49:49.691 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-08-31T05:49:49.702 INFO:teuthology.run_tasks:Running task internal.archive... 2024-08-31T05:49:49.718 INFO:teuthology.task.internal:Creating archive directory... 2024-08-31T05:49:49.718 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-31T05:49:49.734 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-31T05:49:49.736 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-31T05:49:50.016 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-08-31T05:49:50.027 INFO:teuthology.task.internal:Enabling coredump saving... 2024-08-31T05:49:50.027 DEBUG:teuthology.orchestra.run.smithi155:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-31T05:49:50.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-31T05:49:50.045 DEBUG:teuthology.orchestra.run.smithi184:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-31T05:49:50.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-31T05:49:50.063 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-31T05:49:50.067 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-31T05:49:50.088 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-31T05:49:50.140 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-31T05:49:50.161 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-31T05:49:50.162 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-31T05:49:50.182 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-31T05:49:50.215 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-31T05:49:50.255 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-31T05:49:50.258 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-08-31T05:49:50.270 INFO:teuthology.task.internal:Configuring sudo... 2024-08-31T05:49:50.270 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-31T05:49:50.274 DEBUG:teuthology.orchestra.run.smithi155:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-31T05:49:50.277 DEBUG:teuthology.orchestra.run.smithi184:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-31T05:49:50.328 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-08-31T05:49:50.341 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-08-31T05:49:50.342 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-31T05:49:50.345 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-31T05:49:50.348 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-31T05:49:50.377 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-31T05:49:50.479 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-31T05:49:50.570 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-31T05:49:50.636 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-31T05:49:50.706 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-08-31T05:49:50.707 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-31T05:49:50.763 DEBUG:teuthology.orchestra.run.smithi155:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-31T05:49:50.810 DEBUG:teuthology.orchestra.run.smithi155:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-31T05:49:50.862 DEBUG:teuthology.orchestra.run.smithi155:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-31T05:49:50.947 DEBUG:teuthology.orchestra.run.smithi155:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-31T05:49:50.988 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-08-31T05:49:50.989 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-31T05:49:51.055 DEBUG:teuthology.orchestra.run.smithi184:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-31T05:49:51.101 DEBUG:teuthology.orchestra.run.smithi184:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-31T05:49:51.159 DEBUG:teuthology.orchestra.run.smithi184:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-31T05:49:51.244 DEBUG:teuthology.orchestra.run.smithi184:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-31T05:49:51.287 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-08-31T05:49:51.287 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-31T05:49:51.356 DEBUG:teuthology.orchestra.run.smithi016:> sudo service rsyslog restart 2024-08-31T05:49:51.361 DEBUG:teuthology.orchestra.run.smithi155:> sudo service rsyslog restart 2024-08-31T05:49:51.363 DEBUG:teuthology.orchestra.run.smithi184:> sudo service rsyslog restart 2024-08-31T05:49:51.398 INFO:teuthology.orchestra.run.smithi155.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-31T05:49:51.435 INFO:teuthology.orchestra.run.smithi184.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-31T05:49:51.456 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-31T05:49:51.914 INFO:teuthology.run_tasks:Running task internal.timer... 2024-08-31T05:49:51.925 INFO:teuthology.task.internal:Starting timer... 2024-08-31T05:49:51.925 INFO:teuthology.run_tasks:Running task pcp... 2024-08-31T05:49:51.949 INFO:teuthology.run_tasks:Running task selinux... 2024-08-31T05:49:51.966 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-08-31T05:49:51.966 DEBUG:teuthology.orchestra.run.smithi016:> sudo service auditd rotate 2024-08-31T05:49:52.061 INFO:teuthology.orchestra.run.smithi016.stdout:Rotating logs: 2024-08-31T05:49:52.066 DEBUG:teuthology.orchestra.run.smithi155:> sudo service auditd rotate 2024-08-31T05:49:52.146 INFO:teuthology.orchestra.run.smithi155.stdout:Rotating logs: 2024-08-31T05:49:52.148 DEBUG:teuthology.orchestra.run.smithi184:> sudo service auditd rotate 2024-08-31T05:49:52.208 INFO:teuthology.orchestra.run.smithi184.stdout:Rotating logs: 2024-08-31T05:49:52.211 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-08-31T05:49:52.211 DEBUG:teuthology.orchestra.run.smithi016:> /usr/sbin/getenforce 2024-08-31T05:49:52.255 INFO:teuthology.orchestra.run.smithi016.stdout:Permissive 2024-08-31T05:49:52.255 DEBUG:teuthology.orchestra.run.smithi155:> /usr/sbin/getenforce 2024-08-31T05:49:52.276 INFO:teuthology.orchestra.run.smithi155.stdout:Permissive 2024-08-31T05:49:52.277 DEBUG:teuthology.orchestra.run.smithi184:> /usr/sbin/getenforce 2024-08-31T05:49:52.307 INFO:teuthology.orchestra.run.smithi184.stdout:Permissive 2024-08-31T05:49:52.308 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi016.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi155.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi184.front.sepia.ceph.com': 'permissive'} 2024-08-31T05:49:52.308 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-08-31T05:49:52.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-31T05:49:52.366 DEBUG:teuthology.orchestra.run.smithi155:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-08-31T05:49:52.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-31T05:49:52.395 DEBUG:teuthology.orchestra.run.smithi184:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-08-31T05:49:52.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-31T05:49:52.427 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-08-31T05:49:52.427 DEBUG:teuthology.orchestra.run.smithi016:> sudo /usr/sbin/setenforce permissive 2024-08-31T05:49:52.494 DEBUG:teuthology.orchestra.run.smithi155:> sudo /usr/sbin/setenforce permissive 2024-08-31T05:49:52.531 DEBUG:teuthology.orchestra.run.smithi184:> sudo /usr/sbin/setenforce permissive 2024-08-31T05:49:52.569 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-08-31T05:49:52.578 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-08-31T05:49:52.590 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-08-31T05:49:52.591 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi016.front.sepia.ceph.com,smithi155.front.sepia.ceph.com,smithi184.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-08-31T05:51:59.238 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi016.front.sepia.ceph.com'), Remote(name='ubuntu@smithi155.front.sepia.ceph.com'), Remote(name='ubuntu@smithi184.front.sepia.ceph.com')] 2024-08-31T05:51:59.239 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-08-31T05:51:59.240 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-31T05:51:59.332 DEBUG:teuthology.orchestra.run.smithi016:> true 2024-08-31T05:51:59.445 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-08-31T05:51:59.445 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-08-31T05:51:59.445 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-31T05:51:59.520 DEBUG:teuthology.orchestra.run.smithi155:> true 2024-08-31T05:51:59.601 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-08-31T05:51:59.601 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-08-31T05:51:59.602 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-31T05:51:59.680 DEBUG:teuthology.orchestra.run.smithi184:> true 2024-08-31T05:51:59.766 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-08-31T05:51:59.766 INFO:teuthology.run_tasks:Running task clock... 2024-08-31T05:51:59.777 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-08-31T05:51:59.777 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-31T05:51:59.777 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-31T05:51:59.781 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-31T05:51:59.781 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-31T05:51:59.784 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-31T05:51:59.784 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-31T05:51:59.811 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-08-31T05:51:59.827 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-08-31T05:51:59.838 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-08-31T05:51:59.842 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-08-31T05:51:59.851 INFO:teuthology.orchestra.run.smithi155.stderr:sudo: ntpd: command not found 2024-08-31T05:51:59.854 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-08-31T05:51:59.862 INFO:teuthology.orchestra.run.smithi155.stdout:506 Cannot talk to daemon 2024-08-31T05:51:59.872 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-08-31T05:51:59.876 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-08-31T05:51:59.880 INFO:teuthology.orchestra.run.smithi184.stderr:sudo: ntpd: command not found 2024-08-31T05:51:59.891 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-08-31T05:51:59.892 INFO:teuthology.orchestra.run.smithi184.stdout:506 Cannot talk to daemon 2024-08-31T05:51:59.907 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-08-31T05:51:59.921 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-08-31T05:51:59.925 INFO:teuthology.orchestra.run.smithi016.stderr:sudo: ntpd: command not found 2024-08-31T05:51:59.949 INFO:teuthology.orchestra.run.smithi016.stdout:506 Cannot talk to daemon 2024-08-31T05:51:59.955 INFO:teuthology.orchestra.run.smithi155.stderr:bash: line 1: ntpq: command not found 2024-08-31T05:51:59.959 INFO:teuthology.orchestra.run.smithi155.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-31T05:51:59.959 INFO:teuthology.orchestra.run.smithi155.stdout:=============================================================================== 2024-08-31T05:51:59.959 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-31T05:51:59.959 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-31T05:51:59.959 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-31T05:51:59.960 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-31T05:51:59.975 INFO:teuthology.orchestra.run.smithi184.stderr:bash: line 1: ntpq: command not found 2024-08-31T05:51:59.978 INFO:teuthology.orchestra.run.smithi184.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-31T05:51:59.978 INFO:teuthology.orchestra.run.smithi184.stdout:=============================================================================== 2024-08-31T05:51:59.979 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-31T05:51:59.979 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-31T05:51:59.979 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-31T05:51:59.979 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-31T05:51:59.979 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-08-31T05:52:00.008 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-08-31T05:52:00.103 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: ntpq: command not found 2024-08-31T05:52:00.110 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-31T05:52:00.110 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2024-08-31T05:52:00.110 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-31T05:52:00.110 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-31T05:52:00.110 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-31T05:52:00.110 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-31T05:52:00.111 INFO:teuthology.run_tasks:Running task install... 2024-08-31T05:52:00.120 DEBUG:teuthology.task.install:project ceph 2024-08-31T05:52:00.120 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd19e6118f7015f0047264006cb0c873528eb3ae6'}} 2024-08-31T05:52:00.121 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd19e6118f7015f0047264006cb0c873528eb3ae6'} 2024-08-31T05:52:00.121 INFO:teuthology.task.install:Using flavor: default 2024-08-31T05:52:00.126 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-08-31T05:52:00.126 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-08-31T05:52:00.126 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd19e6118f7015f0047264006cb0c873528eb3ae6', 'tag': None, 'wait_for_package': False} 2024-08-31T05:52:00.127 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d19e6118f7015f0047264006cb0c873528eb3ae6 2024-08-31T05:52:00.129 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd19e6118f7015f0047264006cb0c873528eb3ae6', 'tag': None, 'wait_for_package': False} 2024-08-31T05:52:00.129 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d19e6118f7015f0047264006cb0c873528eb3ae6 2024-08-31T05:52:00.130 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd19e6118f7015f0047264006cb0c873528eb3ae6', 'tag': None, 'wait_for_package': False} 2024-08-31T05:52:00.130 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d19e6118f7015f0047264006cb0c873528eb3ae6 2024-08-31T05:52:00.276 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/flavors/default/ 2024-08-31T05:52:00.276 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4560.gd19e6118 2024-08-31T05:52:00.282 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/flavors/default/ 2024-08-31T05:52:00.282 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4560.gd19e6118 2024-08-31T05:52:00.285 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/flavors/default/ 2024-08-31T05:52:00.285 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4560.gd19e6118 2024-08-31T05:52:00.530 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-08-31T05:52:00.530 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-08-31T05:52:00.530 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-08-31T05:52:00.536 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-08-31T05:52:00.536 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-08-31T05:52:00.536 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-08-31T05:52:00.538 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-08-31T05:52:00.538 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-08-31T05:52:00.539 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-08-31T05:52:00.560 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-08-31T05:52:00.560 DEBUG:teuthology.orchestra.run.smithi184:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d19e6118f7015f0047264006cb0c873528eb3ae6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-08-31T05:52:00.565 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-08-31T05:52:00.565 DEBUG:teuthology.orchestra.run.smithi155:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d19e6118f7015f0047264006cb0c873528eb3ae6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-08-31T05:52:00.595 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-08-31T05:52:00.596 DEBUG:teuthology.orchestra.run.smithi016:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d19e6118f7015f0047264006cb0c873528eb3ae6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-08-31T05:52:00.629 DEBUG:teuthology.orchestra.run.smithi184:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-08-31T05:52:00.632 DEBUG:teuthology.orchestra.run.smithi155:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-08-31T05:52:00.692 DEBUG:teuthology.orchestra.run.smithi016:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-08-31T05:52:00.705 DEBUG:teuthology.orchestra.run.smithi184:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-08-31T05:52:00.707 DEBUG:teuthology.orchestra.run.smithi155:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-08-31T05:52:00.770 DEBUG:teuthology.orchestra.run.smithi016:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-08-31T05:52:00.771 INFO:teuthology.orchestra.run.smithi184.stdout:check_obsoletes = 1 2024-08-31T05:52:00.772 INFO:teuthology.orchestra.run.smithi155.stdout:check_obsoletes = 1 2024-08-31T05:52:00.772 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum clean all 2024-08-31T05:52:00.773 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum clean all 2024-08-31T05:52:00.826 INFO:teuthology.orchestra.run.smithi016.stdout:check_obsoletes = 1 2024-08-31T05:52:00.829 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum clean all 2024-08-31T05:52:00.998 INFO:teuthology.orchestra.run.smithi184.stdout:45 files removed 2024-08-31T05:52:00.999 INFO:teuthology.orchestra.run.smithi155.stdout:45 files removed 2024-08-31T05:52:01.026 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-08-31T05:52:01.027 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-08-31T05:52:01.099 INFO:teuthology.orchestra.run.smithi016.stdout:45 files removed 2024-08-31T05:52:01.133 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-08-31T05:52:01.581 INFO:teuthology.orchestra.run.smithi155.stdout:ceph packages for x86_64 308 kB/s | 84 kB 00:00 2024-08-31T05:52:01.604 INFO:teuthology.orchestra.run.smithi184.stdout:ceph packages for x86_64 281 kB/s | 84 kB 00:00 2024-08-31T05:52:01.826 INFO:teuthology.orchestra.run.smithi016.stdout:ceph packages for x86_64 293 kB/s | 84 kB 00:00 2024-08-31T05:52:01.890 INFO:teuthology.orchestra.run.smithi155.stdout:ceph noarch packages 91 kB/s | 24 kB 00:00 2024-08-31T05:52:01.913 INFO:teuthology.orchestra.run.smithi184.stdout:ceph noarch packages 91 kB/s | 24 kB 00:00 2024-08-31T05:52:02.202 INFO:teuthology.orchestra.run.smithi016.stdout:ceph noarch packages 73 kB/s | 24 kB 00:00 2024-08-31T05:52:02.249 INFO:teuthology.orchestra.run.smithi155.stdout:ceph source packages 6.0 kB/s | 1.9 kB 00:00 2024-08-31T05:52:02.313 INFO:teuthology.orchestra.run.smithi184.stdout:ceph source packages 5.3 kB/s | 1.9 kB 00:00 2024-08-31T05:52:02.569 INFO:teuthology.orchestra.run.smithi016.stdout:ceph source packages 5.9 kB/s | 1.9 kB 00:00 2024-08-31T05:52:03.303 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.2 MB 00:00 2024-08-31T05:52:04.038 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - BaseOS 4.9 MB/s | 8.2 MB 00:01 2024-08-31T05:52:04.408 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - BaseOS 3.9 MB/s | 8.2 MB 00:02 2024-08-31T05:52:05.296 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - AppStream 25 MB/s | 20 MB 00:00 2024-08-31T05:52:07.592 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - AppStream 10 MB/s | 20 MB 00:02 2024-08-31T05:52:10.331 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - AppStream 3.9 MB/s | 20 MB 00:05 2024-08-31T05:52:11.271 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.6 MB 00:01 2024-08-31T05:52:12.789 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - Extras packages 100 kB/s | 18 kB 00:00 2024-08-31T05:52:13.426 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - CRB 3.5 MB/s | 6.6 MB 00:01 2024-08-31T05:52:14.043 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 20 MB/s | 22 MB 00:01 2024-08-31T05:52:14.998 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - CRB 7.5 MB/s | 6.6 MB 00:00 2024-08-31T05:52:15.260 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - Extras packages 32 kB/s | 18 kB 00:00 2024-08-31T05:52:16.424 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - Extras packages 87 kB/s | 18 kB 00:00 2024-08-31T05:52:16.456 INFO:teuthology.orchestra.run.smithi155.stdout:Extra Packages for Enterprise Linux 21 MB/s | 22 MB 00:01 2024-08-31T05:52:17.453 INFO:teuthology.orchestra.run.smithi184.stdout:Extra Packages for Enterprise Linux 25 MB/s | 22 MB 00:00 2024-08-31T05:52:20.665 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 40 kB/s | 1.7 kB 00:00 2024-08-31T05:52:22.012 INFO:teuthology.orchestra.run.smithi155.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-08-31T05:52:22.842 INFO:teuthology.orchestra.run.smithi184.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-08-31T05:52:23.281 INFO:teuthology.orchestra.run.smithi016.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-31T05:52:23.282 INFO:teuthology.orchestra.run.smithi016.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-31T05:52:23.297 INFO:teuthology.orchestra.run.smithi016.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-08-31T05:52:23.297 INFO:teuthology.orchestra.run.smithi016.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-08-31T05:52:23.298 INFO:teuthology.orchestra.run.smithi016.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-08-31T05:52:23.299 INFO:teuthology.orchestra.run.smithi016.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-08-31T05:52:23.301 INFO:teuthology.orchestra.run.smithi016.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-08-31T05:52:23.304 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-08-31T05:52:23.305 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-08-31T05:52:23.372 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-08-31T05:52:23.394 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================= 2024-08-31T05:52:23.395 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2024-08-31T05:52:23.395 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================= 2024-08-31T05:52:23.395 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-08-31T05:52:23.395 INFO:teuthology.orchestra.run.smithi016.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-08-31T05:52:23.395 INFO:teuthology.orchestra.run.smithi016.stdout: ceph x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 6.6 k 2024-08-31T05:52:23.395 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 5.5 M 2024-08-31T05:52:23.395 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 874 k 2024-08-31T05:52:23.395 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 148 k 2024-08-31T05:52:23.395 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 1.5 M 2024-08-31T05:52:23.395 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 154 k 2024-08-31T05:52:23.395 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 14 M 2024-08-31T05:52:23.395 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 7.4 M 2024-08-31T05:52:23.395 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 49 k 2024-08-31T05:52:23.396 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 10 M 2024-08-31T05:52:23.396 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 44 M 2024-08-31T05:52:23.396 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 285 k 2024-08-31T05:52:23.396 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 781 k 2024-08-31T05:52:23.396 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 226 k 2024-08-31T05:52:23.396 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-shell x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 27 k 2024-08-31T05:52:23.396 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 18 k 2024-08-31T05:52:23.396 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-08-31T05:52:23.396 INFO:teuthology.orchestra.run.smithi016.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-08-31T05:52:23.397 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-08-31T05:52:23.397 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 32 k 2024-08-31T05:52:23.397 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 766 k 2024-08-31T05:52:23.397 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 127 k 2024-08-31T05:52:23.397 INFO:teuthology.orchestra.run.smithi016.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-08-31T05:52:23.397 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-08-31T05:52:23.397 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel x86_64 1:3.2.2-4.el9 appstream 4.4 M 2024-08-31T05:52:23.397 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-08-31T05:52:23.397 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-08-31T05:52:23.398 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-08-31T05:52:23.398 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 161 k 2024-08-31T05:52:23.398 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 321 k 2024-08-31T05:52:23.398 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 305 k 2024-08-31T05:52:23.398 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 99 k 2024-08-31T05:52:23.398 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 86 k 2024-08-31T05:52:23.398 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 3.1 M 2024-08-31T05:52:23.398 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 170 k 2024-08-31T05:52:23.398 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-08-31T05:52:23.398 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrading: 2024-08-31T05:52:23.398 INFO:teuthology.orchestra.run.smithi016.stdout: librados2 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 3.4 M 2024-08-31T05:52:23.398 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 3.1 M 2024-08-31T05:52:23.398 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-08-31T05:52:23.399 INFO:teuthology.orchestra.run.smithi016.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-08-31T05:52:23.399 INFO:teuthology.orchestra.run.smithi016.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-08-31T05:52:23.399 INFO:teuthology.orchestra.run.smithi016.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-08-31T05:52:23.399 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 20 M 2024-08-31T05:52:23.399 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 34 k 2024-08-31T05:52:23.399 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 2.3 M 2024-08-31T05:52:23.399 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 290 k 2024-08-31T05:52:23.399 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 4.9 M 2024-08-31T05:52:23.399 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 18 M 2024-08-31T05:52:23.399 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 16 k 2024-08-31T05:52:23.399 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 25 k 2024-08-31T05:52:23.399 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-08-31T05:52:23.399 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-08-31T05:52:23.400 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-08-31T05:52:23.400 INFO:teuthology.orchestra.run.smithi016.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-08-31T05:52:23.400 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-08-31T05:52:23.400 INFO:teuthology.orchestra.run.smithi016.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-08-31T05:52:23.400 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-08-31T05:52:23.400 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-08-31T05:52:23.400 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 159 k 2024-08-31T05:52:23.400 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-08-31T05:52:23.400 INFO:teuthology.orchestra.run.smithi016.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-08-31T05:52:23.400 INFO:teuthology.orchestra.run.smithi016.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-08-31T05:52:23.400 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-08-31T05:52:23.400 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-08-31T05:52:23.401 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 506 k 2024-08-31T05:52:23.401 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-08-31T05:52:23.401 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 5.3 M 2024-08-31T05:52:23.401 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-08-31T05:52:23.401 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-08-31T05:52:23.401 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-08-31T05:52:23.401 INFO:teuthology.orchestra.run.smithi016.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-08-31T05:52:23.401 INFO:teuthology.orchestra.run.smithi016.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-08-31T05:52:23.401 INFO:teuthology.orchestra.run.smithi016.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-08-31T05:52:23.401 INFO:teuthology.orchestra.run.smithi016.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-08-31T05:52:23.401 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-08-31T05:52:23.401 INFO:teuthology.orchestra.run.smithi016.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-08-31T05:52:23.402 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-08-31T05:52:23.402 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-08-31T05:52:23.402 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-08-31T05:52:23.402 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-08-31T05:52:23.402 INFO:teuthology.orchestra.run.smithi016.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-08-31T05:52:23.402 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-08-31T05:52:23.402 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-08-31T05:52:23.402 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-08-31T05:52:23.402 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 45 k 2024-08-31T05:52:23.402 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 141 k 2024-08-31T05:52:23.402 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-08-31T05:52:23.402 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-08-31T05:52:23.402 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-08-31T05:52:23.403 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-08-31T05:52:23.403 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-08-31T05:52:23.403 INFO:teuthology.orchestra.run.smithi016.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-08-31T05:52:23.403 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-08-31T05:52:23.403 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-08-31T05:52:23.403 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-08-31T05:52:23.403 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-08-31T05:52:23.403 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-08-31T05:52:23.403 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-08-31T05:52:23.403 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-08-31T05:52:23.404 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-08-31T05:52:23.404 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-08-31T05:52:23.404 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-08-31T05:52:23.404 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-08-31T05:52:23.404 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-08-31T05:52:23.404 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-08-31T05:52:23.404 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-08-31T05:52:23.404 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-08-31T05:52:23.405 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-08-31T05:52:23.405 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-08-31T05:52:23.405 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-08-31T05:52:23.405 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-08-31T05:52:23.405 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-08-31T05:52:23.405 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-08-31T05:52:23.405 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-08-31T05:52:23.405 INFO:teuthology.orchestra.run.smithi016.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-08-31T05:52:23.405 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-08-31T05:52:23.405 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-08-31T05:52:23.405 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-08-31T05:52:23.405 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-08-31T05:52:23.405 INFO:teuthology.orchestra.run.smithi016.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-08-31T05:52:23.406 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-08-31T05:52:23.406 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-08-31T05:52:23.406 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-08-31T05:52:23.406 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-08-31T05:52:23.406 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-08-31T05:52:23.406 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-08-31T05:52:23.406 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-08-31T05:52:23.406 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-08-31T05:52:23.406 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-08-31T05:52:23.406 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-08-31T05:52:23.406 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-08-31T05:52:23.406 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-08-31T05:52:23.406 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-08-31T05:52:23.406 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-08-31T05:52:23.407 INFO:teuthology.orchestra.run.smithi016.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-08-31T05:52:23.407 INFO:teuthology.orchestra.run.smithi016.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-08-31T05:52:23.407 INFO:teuthology.orchestra.run.smithi016.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-08-31T05:52:23.407 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-08-31T05:52:23.407 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-08-31T05:52:23.407 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-08-31T05:52:23.407 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-08-31T05:52:23.407 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-08-31T05:52:23.407 INFO:teuthology.orchestra.run.smithi016.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-08-31T05:52:23.407 INFO:teuthology.orchestra.run.smithi016.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-08-31T05:52:23.407 INFO:teuthology.orchestra.run.smithi016.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-08-31T05:52:23.407 INFO:teuthology.orchestra.run.smithi016.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-08-31T05:52:23.407 INFO:teuthology.orchestra.run.smithi016.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-08-31T05:52:23.408 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-08-31T05:52:23.408 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-08-31T05:52:23.408 INFO:teuthology.orchestra.run.smithi016.stdout:Installing weak dependencies: 2024-08-31T05:52:23.408 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-08-31T05:52:23.408 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T05:52:23.408 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-08-31T05:52:23.408 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================= 2024-08-31T05:52:23.408 INFO:teuthology.orchestra.run.smithi016.stdout:Install 148 Packages 2024-08-31T05:52:23.408 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrade 2 Packages 2024-08-31T05:52:23.408 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T05:52:23.409 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 224 M 2024-08-31T05:52:23.409 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-08-31T05:52:24.001 INFO:teuthology.orchestra.run.smithi016.stdout:(1/150): ceph-19.3.0-4560.gd19e6118.el9.x86_64. 48 kB/s | 6.6 kB 00:00 2024-08-31T05:52:24.200 INFO:teuthology.orchestra.run.smithi016.stdout:(2/150): ceph-fuse-19.3.0-4560.gd19e6118.el9.x8 4.3 MB/s | 874 kB 00:00 2024-08-31T05:52:24.268 INFO:teuthology.orchestra.run.smithi016.stdout:(3/150): ceph-immutable-object-cache-19.3.0-456 2.2 MB/s | 148 kB 00:00 2024-08-31T05:52:24.317 INFO:teuthology.orchestra.run.smithi155.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-31T05:52:24.317 INFO:teuthology.orchestra.run.smithi155.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-31T05:52:24.330 INFO:teuthology.orchestra.run.smithi155.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-08-31T05:52:24.331 INFO:teuthology.orchestra.run.smithi155.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-08-31T05:52:24.332 INFO:teuthology.orchestra.run.smithi155.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-08-31T05:52:24.333 INFO:teuthology.orchestra.run.smithi155.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-08-31T05:52:24.335 INFO:teuthology.orchestra.run.smithi155.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-08-31T05:52:24.338 INFO:teuthology.orchestra.run.smithi155.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-08-31T05:52:24.339 INFO:teuthology.orchestra.run.smithi155.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-08-31T05:52:24.368 INFO:teuthology.orchestra.run.smithi016.stdout:(4/150): ceph-base-19.3.0-4560.gd19e6118.el9.x8 11 MB/s | 5.5 MB 00:00 2024-08-31T05:52:24.402 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-08-31T05:52:24.434 INFO:teuthology.orchestra.run.smithi155.stdout:======================================================================================= 2024-08-31T05:52:24.434 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2024-08-31T05:52:24.434 INFO:teuthology.orchestra.run.smithi155.stdout:======================================================================================= 2024-08-31T05:52:24.434 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-08-31T05:52:24.434 INFO:teuthology.orchestra.run.smithi155.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-08-31T05:52:24.434 INFO:teuthology.orchestra.run.smithi155.stdout: ceph x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 6.6 k 2024-08-31T05:52:24.434 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-base x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 5.5 M 2024-08-31T05:52:24.435 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-fuse x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 874 k 2024-08-31T05:52:24.435 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 148 k 2024-08-31T05:52:24.435 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 1.5 M 2024-08-31T05:52:24.435 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 154 k 2024-08-31T05:52:24.435 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 14 M 2024-08-31T05:52:24.435 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 7.4 M 2024-08-31T05:52:24.435 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-rook noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 49 k 2024-08-31T05:52:24.435 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-radosgw x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 10 M 2024-08-31T05:52:24.435 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-test x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 44 M 2024-08-31T05:52:24.435 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-volume noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 285 k 2024-08-31T05:52:24.435 INFO:teuthology.orchestra.run.smithi155.stdout: cephadm noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 781 k 2024-08-31T05:52:24.435 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-mirror x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 226 k 2024-08-31T05:52:24.435 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-shell x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 27 k 2024-08-31T05:52:24.435 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-top noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 18 k 2024-08-31T05:52:24.436 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-08-31T05:52:24.436 INFO:teuthology.orchestra.run.smithi155.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-08-31T05:52:24.436 INFO:teuthology.orchestra.run.smithi155.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-08-31T05:52:24.436 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs-devel x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 32 k 2024-08-31T05:52:24.436 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs2 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 766 k 2024-08-31T05:52:24.436 INFO:teuthology.orchestra.run.smithi155.stdout: librados-devel x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 127 k 2024-08-31T05:52:24.436 INFO:teuthology.orchestra.run.smithi155.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-08-31T05:52:24.436 INFO:teuthology.orchestra.run.smithi155.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-08-31T05:52:24.436 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-devel x86_64 1:3.2.2-4.el9 appstream 4.4 M 2024-08-31T05:52:24.436 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-08-31T05:52:24.436 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-08-31T05:52:24.436 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-08-31T05:52:24.436 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cephfs x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 161 k 2024-08-31T05:52:24.437 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rados x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 321 k 2024-08-31T05:52:24.437 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rbd x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 305 k 2024-08-31T05:52:24.437 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rgw x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 99 k 2024-08-31T05:52:24.437 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-fuse x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 86 k 2024-08-31T05:52:24.437 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-mirror x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 3.1 M 2024-08-31T05:52:24.437 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-nbd x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 170 k 2024-08-31T05:52:24.437 INFO:teuthology.orchestra.run.smithi155.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-08-31T05:52:24.437 INFO:teuthology.orchestra.run.smithi155.stdout:Upgrading: 2024-08-31T05:52:24.437 INFO:teuthology.orchestra.run.smithi155.stdout: librados2 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 3.4 M 2024-08-31T05:52:24.437 INFO:teuthology.orchestra.run.smithi155.stdout: librbd1 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 3.1 M 2024-08-31T05:52:24.437 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-08-31T05:52:24.437 INFO:teuthology.orchestra.run.smithi155.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-08-31T05:52:24.437 INFO:teuthology.orchestra.run.smithi155.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-08-31T05:52:24.437 INFO:teuthology.orchestra.run.smithi155.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-08-31T05:52:24.438 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-common x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 20 M 2024-08-31T05:52:24.438 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 34 k 2024-08-31T05:52:24.438 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mds x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 2.3 M 2024-08-31T05:52:24.438 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 290 k 2024-08-31T05:52:24.438 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mon x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 4.9 M 2024-08-31T05:52:24.438 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-osd x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 18 M 2024-08-31T05:52:24.438 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 16 k 2024-08-31T05:52:24.438 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-selinux x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 25 k 2024-08-31T05:52:24.438 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-08-31T05:52:24.438 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-08-31T05:52:24.438 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-08-31T05:52:24.438 INFO:teuthology.orchestra.run.smithi155.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-08-31T05:52:24.438 INFO:teuthology.orchestra.run.smithi155.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-08-31T05:52:24.438 INFO:teuthology.orchestra.run.smithi155.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-08-31T05:52:24.439 INFO:teuthology.orchestra.run.smithi155.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-08-31T05:52:24.439 INFO:teuthology.orchestra.run.smithi155.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-08-31T05:52:24.439 INFO:teuthology.orchestra.run.smithi155.stdout: libcephsqlite x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 159 k 2024-08-31T05:52:24.439 INFO:teuthology.orchestra.run.smithi155.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-08-31T05:52:24.439 INFO:teuthology.orchestra.run.smithi155.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-08-31T05:52:24.439 INFO:teuthology.orchestra.run.smithi155.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-08-31T05:52:24.439 INFO:teuthology.orchestra.run.smithi155.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-08-31T05:52:24.439 INFO:teuthology.orchestra.run.smithi155.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-08-31T05:52:24.440 INFO:teuthology.orchestra.run.smithi155.stdout: libradosstriper1 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 506 k 2024-08-31T05:52:24.440 INFO:teuthology.orchestra.run.smithi155.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-08-31T05:52:24.440 INFO:teuthology.orchestra.run.smithi155.stdout: librgw2 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 5.3 M 2024-08-31T05:52:24.440 INFO:teuthology.orchestra.run.smithi155.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-08-31T05:52:24.440 INFO:teuthology.orchestra.run.smithi155.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-08-31T05:52:24.440 INFO:teuthology.orchestra.run.smithi155.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-08-31T05:52:24.440 INFO:teuthology.orchestra.run.smithi155.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-08-31T05:52:24.440 INFO:teuthology.orchestra.run.smithi155.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-08-31T05:52:24.440 INFO:teuthology.orchestra.run.smithi155.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-08-31T05:52:24.440 INFO:teuthology.orchestra.run.smithi155.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-08-31T05:52:24.440 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-08-31T05:52:24.440 INFO:teuthology.orchestra.run.smithi155.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-08-31T05:52:24.440 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-08-31T05:52:24.440 INFO:teuthology.orchestra.run.smithi155.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-08-31T05:52:24.441 INFO:teuthology.orchestra.run.smithi155.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-08-31T05:52:24.441 INFO:teuthology.orchestra.run.smithi155.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-08-31T05:52:24.441 INFO:teuthology.orchestra.run.smithi155.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-08-31T05:52:24.441 INFO:teuthology.orchestra.run.smithi155.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-08-31T05:52:24.441 INFO:teuthology.orchestra.run.smithi155.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-08-31T05:52:24.441 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-08-31T05:52:24.441 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-argparse x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 45 k 2024-08-31T05:52:24.441 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-common x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 141 k 2024-08-31T05:52:24.441 INFO:teuthology.orchestra.run.smithi155.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-08-31T05:52:24.441 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-08-31T05:52:24.441 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-08-31T05:52:24.441 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-08-31T05:52:24.442 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-08-31T05:52:24.442 INFO:teuthology.orchestra.run.smithi155.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-08-31T05:52:24.442 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-08-31T05:52:24.442 INFO:teuthology.orchestra.run.smithi155.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-08-31T05:52:24.442 INFO:teuthology.orchestra.run.smithi155.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-08-31T05:52:24.442 INFO:teuthology.orchestra.run.smithi155.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-08-31T05:52:24.442 INFO:teuthology.orchestra.run.smithi155.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-08-31T05:52:24.442 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-08-31T05:52:24.443 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-08-31T05:52:24.443 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-08-31T05:52:24.443 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-08-31T05:52:24.443 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-08-31T05:52:24.443 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-08-31T05:52:24.443 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-08-31T05:52:24.443 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-08-31T05:52:24.443 INFO:teuthology.orchestra.run.smithi155.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-08-31T05:52:24.443 INFO:teuthology.orchestra.run.smithi155.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-08-31T05:52:24.443 INFO:teuthology.orchestra.run.smithi155.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-08-31T05:52:24.443 INFO:teuthology.orchestra.run.smithi155.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-08-31T05:52:24.443 INFO:teuthology.orchestra.run.smithi155.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-08-31T05:52:24.443 INFO:teuthology.orchestra.run.smithi155.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-08-31T05:52:24.443 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-08-31T05:52:24.443 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-08-31T05:52:24.444 INFO:teuthology.orchestra.run.smithi155.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-08-31T05:52:24.444 INFO:teuthology.orchestra.run.smithi155.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-08-31T05:52:24.444 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-08-31T05:52:24.444 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-08-31T05:52:24.444 INFO:teuthology.orchestra.run.smithi155.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-08-31T05:52:24.444 INFO:teuthology.orchestra.run.smithi155.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-08-31T05:52:24.444 INFO:teuthology.orchestra.run.smithi155.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-08-31T05:52:24.444 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-08-31T05:52:24.444 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-08-31T05:52:24.444 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-08-31T05:52:24.444 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-08-31T05:52:24.444 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-08-31T05:52:24.444 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-08-31T05:52:24.445 INFO:teuthology.orchestra.run.smithi155.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-08-31T05:52:24.445 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-08-31T05:52:24.445 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-08-31T05:52:24.445 INFO:teuthology.orchestra.run.smithi155.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-08-31T05:52:24.445 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-08-31T05:52:24.445 INFO:teuthology.orchestra.run.smithi155.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-08-31T05:52:24.445 INFO:teuthology.orchestra.run.smithi155.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-08-31T05:52:24.445 INFO:teuthology.orchestra.run.smithi155.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-08-31T05:52:24.445 INFO:teuthology.orchestra.run.smithi155.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-08-31T05:52:24.445 INFO:teuthology.orchestra.run.smithi155.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-08-31T05:52:24.445 INFO:teuthology.orchestra.run.smithi155.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-08-31T05:52:24.445 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-08-31T05:52:24.445 INFO:teuthology.orchestra.run.smithi155.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-08-31T05:52:24.446 INFO:teuthology.orchestra.run.smithi155.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-08-31T05:52:24.446 INFO:teuthology.orchestra.run.smithi155.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-08-31T05:52:24.446 INFO:teuthology.orchestra.run.smithi155.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-08-31T05:52:24.446 INFO:teuthology.orchestra.run.smithi155.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-08-31T05:52:24.446 INFO:teuthology.orchestra.run.smithi155.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-08-31T05:52:24.446 INFO:teuthology.orchestra.run.smithi155.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-08-31T05:52:24.446 INFO:teuthology.orchestra.run.smithi155.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-08-31T05:52:24.446 INFO:teuthology.orchestra.run.smithi155.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-08-31T05:52:24.446 INFO:teuthology.orchestra.run.smithi155.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-08-31T05:52:24.446 INFO:teuthology.orchestra.run.smithi155.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-08-31T05:52:24.446 INFO:teuthology.orchestra.run.smithi155.stdout:Installing weak dependencies: 2024-08-31T05:52:24.446 INFO:teuthology.orchestra.run.smithi155.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-08-31T05:52:24.446 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-08-31T05:52:24.446 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-08-31T05:52:24.447 INFO:teuthology.orchestra.run.smithi155.stdout:======================================================================================= 2024-08-31T05:52:24.447 INFO:teuthology.orchestra.run.smithi155.stdout:Install 148 Packages 2024-08-31T05:52:24.447 INFO:teuthology.orchestra.run.smithi155.stdout:Upgrade 2 Packages 2024-08-31T05:52:24.447 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-08-31T05:52:24.447 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 224 M 2024-08-31T05:52:24.447 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-08-31T05:52:24.493 INFO:teuthology.orchestra.run.smithi016.stdout:(5/150): ceph-mgr-19.3.0-4560.gd19e6118.el9.x86 12 MB/s | 1.5 MB 00:00 2024-08-31T05:52:24.577 INFO:teuthology.orchestra.run.smithi016.stdout:(6/150): ceph-mds-19.3.0-4560.gd19e6118.el9.x86 7.4 MB/s | 2.3 MB 00:00 2024-08-31T05:52:24.819 INFO:teuthology.orchestra.run.smithi016.stdout:(7/150): ceph-mon-19.3.0-4560.gd19e6118.el9.x86 15 MB/s | 4.9 MB 00:00 2024-08-31T05:52:24.863 INFO:teuthology.orchestra.run.smithi155.stdout:(1/150): ceph-19.3.0-4560.gd19e6118.el9.x86_64. 41 kB/s | 6.6 kB 00:00 2024-08-31T05:52:25.086 INFO:teuthology.orchestra.run.smithi016.stdout:(8/150): ceph-common-19.3.0-4560.gd19e6118.el9. 16 MB/s | 20 MB 00:01 2024-08-31T05:52:25.088 INFO:teuthology.orchestra.run.smithi155.stdout:(2/150): ceph-fuse-19.3.0-4560.gd19e6118.el9.x8 3.8 MB/s | 874 kB 00:00 2024-08-31T05:52:25.136 INFO:teuthology.orchestra.run.smithi016.stdout:(9/150): ceph-selinux-19.3.0-4560.gd19e6118.el9 508 kB/s | 25 kB 00:00 2024-08-31T05:52:25.139 INFO:teuthology.orchestra.run.smithi155.stdout:(3/150): ceph-immutable-object-cache-19.3.0-456 2.9 MB/s | 148 kB 00:00 2024-08-31T05:52:25.197 INFO:teuthology.orchestra.run.smithi184.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-31T05:52:25.198 INFO:teuthology.orchestra.run.smithi184.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-31T05:52:25.211 INFO:teuthology.orchestra.run.smithi184.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-08-31T05:52:25.212 INFO:teuthology.orchestra.run.smithi184.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-08-31T05:52:25.213 INFO:teuthology.orchestra.run.smithi184.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-08-31T05:52:25.214 INFO:teuthology.orchestra.run.smithi184.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-08-31T05:52:25.216 INFO:teuthology.orchestra.run.smithi184.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-08-31T05:52:25.219 INFO:teuthology.orchestra.run.smithi184.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-08-31T05:52:25.220 INFO:teuthology.orchestra.run.smithi184.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-08-31T05:52:25.279 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-08-31T05:52:25.299 INFO:teuthology.orchestra.run.smithi184.stdout:======================================================================================= 2024-08-31T05:52:25.299 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-08-31T05:52:25.299 INFO:teuthology.orchestra.run.smithi184.stdout:======================================================================================= 2024-08-31T05:52:25.299 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-08-31T05:52:25.299 INFO:teuthology.orchestra.run.smithi184.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-08-31T05:52:25.300 INFO:teuthology.orchestra.run.smithi184.stdout: ceph x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 6.6 k 2024-08-31T05:52:25.300 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-base x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 5.5 M 2024-08-31T05:52:25.300 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-fuse x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 874 k 2024-08-31T05:52:25.300 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 148 k 2024-08-31T05:52:25.300 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 1.5 M 2024-08-31T05:52:25.300 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 154 k 2024-08-31T05:52:25.300 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 14 M 2024-08-31T05:52:25.300 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 7.4 M 2024-08-31T05:52:25.300 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-rook noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 49 k 2024-08-31T05:52:25.300 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-radosgw x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 10 M 2024-08-31T05:52:25.300 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-test x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 44 M 2024-08-31T05:52:25.300 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-volume noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 285 k 2024-08-31T05:52:25.300 INFO:teuthology.orchestra.run.smithi184.stdout: cephadm noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 781 k 2024-08-31T05:52:25.301 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-mirror x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 226 k 2024-08-31T05:52:25.301 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-shell x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 27 k 2024-08-31T05:52:25.301 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-top noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 18 k 2024-08-31T05:52:25.301 INFO:teuthology.orchestra.run.smithi184.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-08-31T05:52:25.301 INFO:teuthology.orchestra.run.smithi184.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-08-31T05:52:25.301 INFO:teuthology.orchestra.run.smithi184.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-08-31T05:52:25.301 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs-devel x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 32 k 2024-08-31T05:52:25.301 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs2 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 766 k 2024-08-31T05:52:25.301 INFO:teuthology.orchestra.run.smithi184.stdout: librados-devel x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 127 k 2024-08-31T05:52:25.301 INFO:teuthology.orchestra.run.smithi184.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-08-31T05:52:25.301 INFO:teuthology.orchestra.run.smithi184.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-08-31T05:52:25.301 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-devel x86_64 1:3.2.2-4.el9 appstream 4.4 M 2024-08-31T05:52:25.301 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-08-31T05:52:25.302 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-08-31T05:52:25.302 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-08-31T05:52:25.302 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cephfs x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 161 k 2024-08-31T05:52:25.302 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rados x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 321 k 2024-08-31T05:52:25.302 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rbd x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 305 k 2024-08-31T05:52:25.302 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rgw x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 99 k 2024-08-31T05:52:25.302 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-fuse x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 86 k 2024-08-31T05:52:25.302 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-mirror x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 3.1 M 2024-08-31T05:52:25.302 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-nbd x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 170 k 2024-08-31T05:52:25.302 INFO:teuthology.orchestra.run.smithi184.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-08-31T05:52:25.302 INFO:teuthology.orchestra.run.smithi184.stdout:Upgrading: 2024-08-31T05:52:25.303 INFO:teuthology.orchestra.run.smithi184.stdout: librados2 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 3.4 M 2024-08-31T05:52:25.303 INFO:teuthology.orchestra.run.smithi184.stdout: librbd1 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 3.1 M 2024-08-31T05:52:25.303 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-08-31T05:52:25.303 INFO:teuthology.orchestra.run.smithi184.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-08-31T05:52:25.303 INFO:teuthology.orchestra.run.smithi184.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-08-31T05:52:25.303 INFO:teuthology.orchestra.run.smithi184.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-08-31T05:52:25.303 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-common x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 20 M 2024-08-31T05:52:25.303 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 34 k 2024-08-31T05:52:25.303 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mds x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 2.3 M 2024-08-31T05:52:25.303 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 290 k 2024-08-31T05:52:25.303 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mon x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 4.9 M 2024-08-31T05:52:25.303 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-osd x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 18 M 2024-08-31T05:52:25.303 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4560.gd19e6118.el9 ceph-noarch 16 k 2024-08-31T05:52:25.303 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-selinux x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 25 k 2024-08-31T05:52:25.303 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-08-31T05:52:25.304 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-08-31T05:52:25.304 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-08-31T05:52:25.304 INFO:teuthology.orchestra.run.smithi184.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-08-31T05:52:25.304 INFO:teuthology.orchestra.run.smithi184.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-08-31T05:52:25.304 INFO:teuthology.orchestra.run.smithi184.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-08-31T05:52:25.304 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-08-31T05:52:25.304 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-08-31T05:52:25.304 INFO:teuthology.orchestra.run.smithi184.stdout: libcephsqlite x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 159 k 2024-08-31T05:52:25.304 INFO:teuthology.orchestra.run.smithi184.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-08-31T05:52:25.304 INFO:teuthology.orchestra.run.smithi184.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-08-31T05:52:25.304 INFO:teuthology.orchestra.run.smithi184.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-08-31T05:52:25.304 INFO:teuthology.orchestra.run.smithi184.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-08-31T05:52:25.304 INFO:teuthology.orchestra.run.smithi184.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-08-31T05:52:25.305 INFO:teuthology.orchestra.run.smithi184.stdout: libradosstriper1 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 506 k 2024-08-31T05:52:25.305 INFO:teuthology.orchestra.run.smithi184.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-08-31T05:52:25.305 INFO:teuthology.orchestra.run.smithi184.stdout: librgw2 x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 5.3 M 2024-08-31T05:52:25.305 INFO:teuthology.orchestra.run.smithi184.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-08-31T05:52:25.305 INFO:teuthology.orchestra.run.smithi184.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-08-31T05:52:25.305 INFO:teuthology.orchestra.run.smithi184.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-08-31T05:52:25.305 INFO:teuthology.orchestra.run.smithi184.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-08-31T05:52:25.305 INFO:teuthology.orchestra.run.smithi184.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-08-31T05:52:25.305 INFO:teuthology.orchestra.run.smithi184.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-08-31T05:52:25.305 INFO:teuthology.orchestra.run.smithi184.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-08-31T05:52:25.305 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-08-31T05:52:25.305 INFO:teuthology.orchestra.run.smithi184.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-08-31T05:52:25.305 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-08-31T05:52:25.306 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-08-31T05:52:25.306 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-08-31T05:52:25.306 INFO:teuthology.orchestra.run.smithi184.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-08-31T05:52:25.306 INFO:teuthology.orchestra.run.smithi184.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-08-31T05:52:25.306 INFO:teuthology.orchestra.run.smithi184.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-08-31T05:52:25.306 INFO:teuthology.orchestra.run.smithi184.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-08-31T05:52:25.306 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-08-31T05:52:25.306 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-argparse x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 45 k 2024-08-31T05:52:25.306 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-common x86_64 2:19.3.0-4560.gd19e6118.el9 ceph 141 k 2024-08-31T05:52:25.306 INFO:teuthology.orchestra.run.smithi184.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-08-31T05:52:25.306 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-08-31T05:52:25.306 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-08-31T05:52:25.306 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-08-31T05:52:25.307 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-08-31T05:52:25.307 INFO:teuthology.orchestra.run.smithi184.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-08-31T05:52:25.307 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-08-31T05:52:25.307 INFO:teuthology.orchestra.run.smithi184.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-08-31T05:52:25.307 INFO:teuthology.orchestra.run.smithi184.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-08-31T05:52:25.307 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-08-31T05:52:25.307 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-08-31T05:52:25.307 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-08-31T05:52:25.307 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-08-31T05:52:25.307 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-08-31T05:52:25.307 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-08-31T05:52:25.308 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-08-31T05:52:25.308 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-08-31T05:52:25.308 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-08-31T05:52:25.308 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-08-31T05:52:25.308 INFO:teuthology.orchestra.run.smithi184.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-08-31T05:52:25.308 INFO:teuthology.orchestra.run.smithi184.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-08-31T05:52:25.308 INFO:teuthology.orchestra.run.smithi184.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-08-31T05:52:25.308 INFO:teuthology.orchestra.run.smithi184.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-08-31T05:52:25.308 INFO:teuthology.orchestra.run.smithi184.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-08-31T05:52:25.309 INFO:teuthology.orchestra.run.smithi184.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-08-31T05:52:25.309 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-08-31T05:52:25.309 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-08-31T05:52:25.309 INFO:teuthology.orchestra.run.smithi184.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-08-31T05:52:25.309 INFO:teuthology.orchestra.run.smithi184.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-08-31T05:52:25.309 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-08-31T05:52:25.309 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-08-31T05:52:25.309 INFO:teuthology.orchestra.run.smithi184.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-08-31T05:52:25.309 INFO:teuthology.orchestra.run.smithi184.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-08-31T05:52:25.309 INFO:teuthology.orchestra.run.smithi184.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-08-31T05:52:25.309 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-08-31T05:52:25.309 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-08-31T05:52:25.309 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-08-31T05:52:25.309 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-08-31T05:52:25.310 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-08-31T05:52:25.310 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-08-31T05:52:25.310 INFO:teuthology.orchestra.run.smithi184.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-08-31T05:52:25.310 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-08-31T05:52:25.310 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-08-31T05:52:25.310 INFO:teuthology.orchestra.run.smithi184.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-08-31T05:52:25.310 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-08-31T05:52:25.310 INFO:teuthology.orchestra.run.smithi184.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-08-31T05:52:25.310 INFO:teuthology.orchestra.run.smithi184.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-08-31T05:52:25.310 INFO:teuthology.orchestra.run.smithi184.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-08-31T05:52:25.310 INFO:teuthology.orchestra.run.smithi184.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-08-31T05:52:25.310 INFO:teuthology.orchestra.run.smithi184.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-08-31T05:52:25.310 INFO:teuthology.orchestra.run.smithi184.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-08-31T05:52:25.310 INFO:teuthology.orchestra.run.smithi184.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-08-31T05:52:25.310 INFO:teuthology.orchestra.run.smithi184.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-08-31T05:52:25.311 INFO:teuthology.orchestra.run.smithi184.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-08-31T05:52:25.311 INFO:teuthology.orchestra.run.smithi184.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-08-31T05:52:25.311 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-08-31T05:52:25.311 INFO:teuthology.orchestra.run.smithi184.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-08-31T05:52:25.311 INFO:teuthology.orchestra.run.smithi184.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-08-31T05:52:25.311 INFO:teuthology.orchestra.run.smithi184.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-08-31T05:52:25.311 INFO:teuthology.orchestra.run.smithi184.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-08-31T05:52:25.311 INFO:teuthology.orchestra.run.smithi184.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-08-31T05:52:25.311 INFO:teuthology.orchestra.run.smithi184.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-08-31T05:52:25.311 INFO:teuthology.orchestra.run.smithi184.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-08-31T05:52:25.311 INFO:teuthology.orchestra.run.smithi184.stdout:Installing weak dependencies: 2024-08-31T05:52:25.311 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-08-31T05:52:25.311 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-08-31T05:52:25.312 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-08-31T05:52:25.312 INFO:teuthology.orchestra.run.smithi184.stdout:======================================================================================= 2024-08-31T05:52:25.312 INFO:teuthology.orchestra.run.smithi184.stdout:Install 148 Packages 2024-08-31T05:52:25.312 INFO:teuthology.orchestra.run.smithi184.stdout:Upgrade 2 Packages 2024-08-31T05:52:25.312 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-08-31T05:52:25.312 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 224 M 2024-08-31T05:52:25.313 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-08-31T05:52:25.536 INFO:teuthology.orchestra.run.smithi016.stdout:(10/150): ceph-radosgw-19.3.0-4560.gd19e6118.el 15 MB/s | 10 MB 00:00 2024-08-31T05:52:25.581 INFO:teuthology.orchestra.run.smithi155.stdout:(4/150): ceph-mds-19.3.0-4560.gd19e6118.el9.x86 5.2 MB/s | 2.3 MB 00:00 2024-08-31T05:52:25.595 INFO:teuthology.orchestra.run.smithi016.stdout:(11/150): cephfs-mirror-19.3.0-4560.gd19e6118.e 3.8 MB/s | 226 kB 00:00 2024-08-31T05:52:25.670 INFO:teuthology.orchestra.run.smithi016.stdout:(12/150): cephfs-shell-19.3.0-4560.gd19e6118.el 362 kB/s | 27 kB 00:00 2024-08-31T05:52:25.745 INFO:teuthology.orchestra.run.smithi016.stdout:(13/150): libcephfs-devel-19.3.0-4560.gd19e6118 423 kB/s | 32 kB 00:00 2024-08-31T05:52:25.881 INFO:teuthology.orchestra.run.smithi155.stdout:(5/150): ceph-mgr-19.3.0-4560.gd19e6118.el9.x86 5.1 MB/s | 1.5 MB 00:00 2024-08-31T05:52:25.887 INFO:teuthology.orchestra.run.smithi016.stdout:(14/150): libcephfs2-19.3.0-4560.gd19e6118.el9. 5.3 MB/s | 766 kB 00:00 2024-08-31T05:52:25.935 INFO:teuthology.orchestra.run.smithi184.stdout:(1/150): ceph-19.3.0-4560.gd19e6118.el9.x86_64. 30 kB/s | 6.6 kB 00:00 2024-08-31T05:52:25.971 INFO:teuthology.orchestra.run.smithi016.stdout:(15/150): libcephsqlite-19.3.0-4560.gd19e6118.e 1.9 MB/s | 159 kB 00:00 2024-08-31T05:52:26.037 INFO:teuthology.orchestra.run.smithi016.stdout:(16/150): librados-devel-19.3.0-4560.gd19e6118. 1.9 MB/s | 127 kB 00:00 2024-08-31T05:52:26.246 INFO:teuthology.orchestra.run.smithi016.stdout:(17/150): ceph-osd-19.3.0-4560.gd19e6118.el9.x8 11 MB/s | 18 MB 00:01 2024-08-31T05:52:26.280 INFO:teuthology.orchestra.run.smithi016.stdout:(18/150): libradosstriper1-19.3.0-4560.gd19e611 2.0 MB/s | 506 kB 00:00 2024-08-31T05:52:26.363 INFO:teuthology.orchestra.run.smithi016.stdout:(19/150): python3-ceph-argparse-19.3.0-4560.gd1 542 kB/s | 45 kB 00:00 2024-08-31T05:52:26.464 INFO:teuthology.orchestra.run.smithi016.stdout:(20/150): python3-ceph-common-19.3.0-4560.gd19e 1.4 MB/s | 141 kB 00:00 2024-08-31T05:52:26.564 INFO:teuthology.orchestra.run.smithi016.stdout:(21/150): python3-cephfs-19.3.0-4560.gd19e6118. 1.6 MB/s | 161 kB 00:00 2024-08-31T05:52:26.689 INFO:teuthology.orchestra.run.smithi016.stdout:(22/150): python3-rados-19.3.0-4560.gd19e6118.e 2.5 MB/s | 321 kB 00:00 2024-08-31T05:52:26.790 INFO:teuthology.orchestra.run.smithi016.stdout:(23/150): python3-rbd-19.3.0-4560.gd19e6118.el9 3.0 MB/s | 305 kB 00:00 2024-08-31T05:52:26.885 INFO:teuthology.orchestra.run.smithi184.stdout:(2/150): ceph-fuse-19.3.0-4560.gd19e6118.el9.x8 920 kB/s | 874 kB 00:00 2024-08-31T05:52:26.890 INFO:teuthology.orchestra.run.smithi155.stdout:(6/150): ceph-mon-19.3.0-4560.gd19e6118.el9.x86 4.9 MB/s | 4.9 MB 00:01 2024-08-31T05:52:26.890 INFO:teuthology.orchestra.run.smithi016.stdout:(24/150): python3-rgw-19.3.0-4560.gd19e6118.el9 996 kB/s | 99 kB 00:00 2024-08-31T05:52:26.999 INFO:teuthology.orchestra.run.smithi016.stdout:(25/150): librgw2-19.3.0-4560.gd19e6118.el9.x86 7.0 MB/s | 5.3 MB 00:00 2024-08-31T05:52:27.024 INFO:teuthology.orchestra.run.smithi016.stdout:(26/150): rbd-fuse-19.3.0-4560.gd19e6118.el9.x8 645 kB/s | 86 kB 00:00 2024-08-31T05:52:27.035 INFO:teuthology.orchestra.run.smithi184.stdout:(3/150): ceph-immutable-object-cache-19.3.0-456 988 kB/s | 148 kB 00:00 2024-08-31T05:52:27.416 INFO:teuthology.orchestra.run.smithi016.stdout:(27/150): rbd-nbd-19.3.0-4560.gd19e6118.el9.x86 434 kB/s | 170 kB 00:00 2024-08-31T05:52:27.500 INFO:teuthology.orchestra.run.smithi016.stdout:(28/150): ceph-grafana-dashboards-19.3.0-4560.g 411 kB/s | 34 kB 00:00 2024-08-31T05:52:27.592 INFO:teuthology.orchestra.run.smithi016.stdout:(29/150): rbd-mirror-19.3.0-4560.gd19e6118.el9. 5.2 MB/s | 3.1 MB 00:00 2024-08-31T05:52:27.617 INFO:teuthology.orchestra.run.smithi016.stdout:(30/150): ceph-mgr-cephadm-19.3.0-4560.gd19e611 1.3 MB/s | 154 kB 00:00 2024-08-31T05:52:27.931 INFO:teuthology.orchestra.run.smithi155.stdout:(7/150): ceph-base-19.3.0-4560.gd19e6118.el9.x8 1.7 MB/s | 5.5 MB 00:03 2024-08-31T05:52:28.361 INFO:teuthology.orchestra.run.smithi184.stdout:(4/150): ceph-base-19.3.0-4560.gd19e6118.el9.x8 2.1 MB/s | 5.5 MB 00:02 2024-08-31T05:52:28.834 INFO:teuthology.orchestra.run.smithi016.stdout:(31/150): ceph-test-19.3.0-4560.gd19e6118.el9.x 12 MB/s | 44 MB 00:03 2024-08-31T05:52:28.844 INFO:teuthology.orchestra.run.smithi184.stdout:(5/150): ceph-mgr-19.3.0-4560.gd19e6118.el9.x86 3.1 MB/s | 1.5 MB 00:00 2024-08-31T05:52:28.926 INFO:teuthology.orchestra.run.smithi016.stdout:(32/150): ceph-mgr-modules-core-19.3.0-4560.gd1 3.1 MB/s | 290 kB 00:00 2024-08-31T05:52:29.009 INFO:teuthology.orchestra.run.smithi016.stdout:(33/150): ceph-mgr-rook-19.3.0-4560.gd19e6118.e 595 kB/s | 49 kB 00:00 2024-08-31T05:52:29.101 INFO:teuthology.orchestra.run.smithi016.stdout:(34/150): ceph-prometheus-alerts-19.3.0-4560.gd 179 kB/s | 16 kB 00:00 2024-08-31T05:52:29.178 INFO:teuthology.orchestra.run.smithi184.stdout:(6/150): ceph-mds-19.3.0-4560.gd19e6118.el9.x86 1.1 MB/s | 2.3 MB 00:02 2024-08-31T05:52:29.218 INFO:teuthology.orchestra.run.smithi016.stdout:(35/150): ceph-mgr-diskprediction-local-19.3.0- 4.6 MB/s | 7.4 MB 00:01 2024-08-31T05:52:29.243 INFO:teuthology.orchestra.run.smithi016.stdout:(36/150): ceph-volume-19.3.0-4560.gd19e6118.el9 2.0 MB/s | 285 kB 00:00 2024-08-31T05:52:29.364 INFO:teuthology.orchestra.run.smithi016.stdout:(37/150): cephfs-top-19.3.0-4560.gd19e6118.el9. 151 kB/s | 18 kB 00:00 2024-08-31T05:52:29.402 INFO:teuthology.orchestra.run.smithi016.stdout:(38/150): cephadm-19.3.0-4560.gd19e6118.el9.noa 4.2 MB/s | 781 kB 00:00 2024-08-31T05:52:29.503 INFO:teuthology.orchestra.run.smithi016.stdout:(39/150): fuse-2.9.9-16.el9.x86_64.rpm 579 kB/s | 80 kB 00:00 2024-08-31T05:52:29.586 INFO:teuthology.orchestra.run.smithi016.stdout:(40/150): libquadmath-11.5.0-2.el9.x86_64.rpm 2.2 MB/s | 191 kB 00:00 2024-08-31T05:52:29.620 INFO:teuthology.orchestra.run.smithi016.stdout:(41/150): libgfortran-11.5.0-2.el9.x86_64.rpm 3.6 MB/s | 801 kB 00:00 2024-08-31T05:52:29.654 INFO:teuthology.orchestra.run.smithi016.stdout:(42/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-08-31T05:52:29.704 INFO:teuthology.orchestra.run.smithi016.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-08-31T05:52:29.746 INFO:teuthology.orchestra.run.smithi016.stdout:(44/150): python3-cryptography-36.0.1-4.el9.x86 9.8 MB/s | 1.2 MB 00:00 2024-08-31T05:52:29.771 INFO:teuthology.orchestra.run.smithi016.stdout:(45/150): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-08-31T05:52:29.805 INFO:teuthology.orchestra.run.smithi016.stdout:(46/150): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-08-31T05:52:29.830 INFO:teuthology.orchestra.run.smithi016.stdout:(47/150): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-08-31T05:52:29.903 INFO:teuthology.orchestra.run.smithi184.stdout:(7/150): ceph-mon-19.3.0-4560.gd19e6118.el9.x86 4.6 MB/s | 4.9 MB 00:01 2024-08-31T05:52:29.972 INFO:teuthology.orchestra.run.smithi016.stdout:(48/150): automake-1.16.2-8.el9.noarch.rpm 4.0 MB/s | 689 kB 00:00 2024-08-31T05:52:30.039 INFO:teuthology.orchestra.run.smithi016.stdout:(49/150): bison-3.7.4-5.el9.x86_64.rpm 4.4 MB/s | 943 kB 00:00 2024-08-31T05:52:30.064 INFO:teuthology.orchestra.run.smithi016.stdout:(50/150): boost-program-options-1.75.0-8.el9.x8 1.1 MB/s | 107 kB 00:00 2024-08-31T05:52:30.090 INFO:teuthology.orchestra.run.smithi016.stdout:(51/150): elfutils-libelf-devel-0.191-4.el9.x86 428 kB/s | 22 kB 00:00 2024-08-31T05:52:30.132 INFO:teuthology.orchestra.run.smithi016.stdout:(52/150): flexiblas-3.0.4-8.el9.x86_64.rpm 791 kB/s | 33 kB 00:00 2024-08-31T05:52:30.165 INFO:teuthology.orchestra.run.smithi016.stdout:(53/150): flex-2.6.4-9.el9.x86_64.rpm 3.0 MB/s | 313 kB 00:00 2024-08-31T05:52:30.200 INFO:teuthology.orchestra.run.smithi016.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 547 kB/s | 18 kB 00:00 2024-08-31T05:52:30.233 INFO:teuthology.orchestra.run.smithi016.stdout:(55/150): libaio-devel-0.3.111-13.el9.x86_64.rp 338 kB/s | 11 kB 00:00 2024-08-31T05:52:30.325 INFO:teuthology.orchestra.run.smithi016.stdout:(56/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2024-08-31T05:52:30.358 INFO:teuthology.orchestra.run.smithi016.stdout:(57/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-08-31T05:52:30.559 INFO:teuthology.orchestra.run.smithi016.stdout:(58/150): librdkafka-1.6.1-102.el9.x86_64.rpm 3.2 MB/s | 662 kB 00:00 2024-08-31T05:52:30.742 INFO:teuthology.orchestra.run.smithi016.stdout:(59/150): libtool-2.4.6-46.el9.x86_64.rpm 3.1 MB/s | 581 kB 00:00 2024-08-31T05:52:30.776 INFO:teuthology.orchestra.run.smithi016.stdout:(60/150): libuuid-devel-2.37.4-20.el9.x86_64.rp 738 kB/s | 24 kB 00:00 2024-08-31T05:52:30.809 INFO:teuthology.orchestra.run.smithi016.stdout:(61/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-08-31T05:52:30.852 INFO:teuthology.orchestra.run.smithi016.stdout:(62/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.9 MB/s | 292 kB 00:00 2024-08-31T05:52:31.010 INFO:teuthology.orchestra.run.smithi016.stdout:(63/150): ceph-mgr-dashboard-19.3.0-4560.gd19e6 4.1 MB/s | 14 MB 00:03 2024-08-31T05:52:31.035 INFO:teuthology.orchestra.run.smithi016.stdout:(64/150): lua-5.4.4-4.el9.x86_64.rpm 1.0 MB/s | 188 kB 00:00 2024-08-31T05:52:31.086 INFO:teuthology.orchestra.run.smithi016.stdout:(65/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.1 MB/s | 3.0 MB 00:00 2024-08-31T05:52:31.111 INFO:teuthology.orchestra.run.smithi016.stdout:(66/150): openblas-0.3.26-2.el9.x86_64.rpm 384 kB/s | 38 kB 00:00 2024-08-31T05:52:31.395 INFO:teuthology.orchestra.run.smithi016.stdout:(67/150): postgresql-13.16-1.el9.x86_64.rpm 5.6 MB/s | 1.6 MB 00:00 2024-08-31T05:52:31.662 INFO:teuthology.orchestra.run.smithi016.stdout:(68/150): postgresql-contrib-13.16-1.el9.x86_64 3.3 MB/s | 910 kB 00:00 2024-08-31T05:52:31.737 INFO:teuthology.orchestra.run.smithi016.stdout:(69/150): postgresql-private-libs-13.16-1.el9.x 1.8 MB/s | 137 kB 00:00 2024-08-31T05:52:31.803 INFO:teuthology.orchestra.run.smithi184.stdout:(8/150): ceph-radosgw-19.3.0-4560.gd19e6118.el9 5.5 MB/s | 10 MB 00:01 2024-08-31T05:52:31.886 INFO:teuthology.orchestra.run.smithi184.stdout:(9/150): ceph-selinux-19.3.0-4560.gd19e6118.el9 303 kB/s | 25 kB 00:00 2024-08-31T05:52:32.204 INFO:teuthology.orchestra.run.smithi016.stdout:(70/150): openssl-devel-3.2.2-4.el9.x86_64.rpm 4.0 MB/s | 4.4 MB 00:01 2024-08-31T05:52:32.413 INFO:teuthology.orchestra.run.smithi016.stdout:(71/150): openblas-openmp-0.3.26-2.el9.x86_64.r 3.6 MB/s | 4.9 MB 00:01 2024-08-31T05:52:32.471 INFO:teuthology.orchestra.run.smithi016.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-08-31T05:52:32.571 INFO:teuthology.orchestra.run.smithi016.stdout:(73/150): protobuf-3.14.0-13.el9.x86_64.rpm 2.7 MB/s | 1.0 MB 00:00 2024-08-31T05:52:32.613 INFO:teuthology.orchestra.run.smithi016.stdout:(74/150): python3-colorama-0.4.6-3.el9.noarch.r 1.5 MB/s | 61 kB 00:00 2024-08-31T05:52:32.705 INFO:teuthology.orchestra.run.smithi016.stdout:(75/150): python3-devel-3.9.19-8.el9.x86_64.rpm 2.6 MB/s | 245 kB 00:00 2024-08-31T05:52:32.797 INFO:teuthology.orchestra.run.smithi016.stdout:(76/150): python3-jinja2-2.11.3-6.el9.noarch.rp 2.7 MB/s | 248 kB 00:00 2024-08-31T05:52:32.839 INFO:teuthology.orchestra.run.smithi016.stdout:(77/150): python3-jsonpatch-1.21-16.el9.noarch. 618 kB/s | 26 kB 00:00 2024-08-31T05:52:32.881 INFO:teuthology.orchestra.run.smithi016.stdout:(78/150): python3-jsonpointer-2.0-4.el9.noarch. 461 kB/s | 19 kB 00:00 2024-08-31T05:52:32.957 INFO:teuthology.orchestra.run.smithi016.stdout:(79/150): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-08-31T05:52:32.998 INFO:teuthology.orchestra.run.smithi016.stdout:(80/150): python3-markupsafe-1.1.1-12.el9.x86_6 838 kB/s | 35 kB 00:00 2024-08-31T05:52:33.365 INFO:teuthology.orchestra.run.smithi155.stdout:(8/150): ceph-osd-19.3.0-4560.gd19e6118.el9.x86 2.8 MB/s | 18 MB 00:06 2024-08-31T05:52:33.541 INFO:teuthology.orchestra.run.smithi016.stdout:(81/150): postgresql-server-13.16-1.el9.x86_64. 3.2 MB/s | 5.8 MB 00:01 2024-08-31T05:52:33.565 INFO:teuthology.orchestra.run.smithi155.stdout:(9/150): ceph-common-19.3.0-4560.gd19e6118.el9. 2.2 MB/s | 20 MB 00:08 2024-08-31T05:52:33.590 INFO:teuthology.orchestra.run.smithi155.stdout:(10/150): ceph-selinux-19.3.0-4560.gd19e6118.el 112 kB/s | 25 kB 00:00 2024-08-31T05:52:33.632 INFO:teuthology.orchestra.run.smithi016.stdout:(82/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-08-31T05:52:33.815 INFO:teuthology.orchestra.run.smithi016.stdout:(83/150): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-08-31T05:52:33.815 INFO:teuthology.orchestra.run.smithi016.stdout:(84/150): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-08-31T05:52:33.817 INFO:teuthology.orchestra.run.smithi016.stdout:(85/150): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-08-31T05:52:33.848 INFO:teuthology.orchestra.run.smithi155.stdout:(11/150): cephfs-mirror-19.3.0-4560.gd19e6118.e 875 kB/s | 226 kB 00:00 2024-08-31T05:52:33.924 INFO:teuthology.orchestra.run.smithi155.stdout:(12/150): cephfs-shell-19.3.0-4560.gd19e6118.el 361 kB/s | 27 kB 00:00 2024-08-31T05:52:33.925 INFO:teuthology.orchestra.run.smithi016.stdout:(86/150): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-08-31T05:52:34.016 INFO:teuthology.orchestra.run.smithi155.stdout:(13/150): libcephfs-devel-19.3.0-4560.gd19e6118 346 kB/s | 32 kB 00:00 2024-08-31T05:52:34.100 INFO:teuthology.orchestra.run.smithi016.stdout:(87/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 910 kB/s | 159 kB 00:00 2024-08-31T05:52:34.209 INFO:teuthology.orchestra.run.smithi016.stdout:(88/150): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-08-31T05:52:34.251 INFO:teuthology.orchestra.run.smithi016.stdout:(89/150): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-08-31T05:52:34.293 INFO:teuthology.orchestra.run.smithi016.stdout:(90/150): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 54 kB 00:00 2024-08-31T05:52:34.452 INFO:teuthology.orchestra.run.smithi016.stdout:(91/150): python3-babel-2.9.1-2.el9.noarch.rpm 3.0 MB/s | 6.0 MB 00:01 2024-08-31T05:52:34.496 INFO:teuthology.orchestra.run.smithi016.stdout:(92/150): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-08-31T05:52:34.535 INFO:teuthology.orchestra.run.smithi016.stdout:(93/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-08-31T05:52:34.616 INFO:teuthology.orchestra.run.smithi155.stdout:(14/150): libcephfs2-19.3.0-4560.gd19e6118.el9. 1.2 MB/s | 766 kB 00:00 2024-08-31T05:52:34.627 INFO:teuthology.orchestra.run.smithi016.stdout:(94/150): qatlib-24.02.0-1.el9.x86_64.rpm 2.4 MB/s | 221 kB 00:00 2024-08-31T05:52:34.719 INFO:teuthology.orchestra.run.smithi016.stdout:(95/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.0 MB/s | 5.1 MB 00:01 2024-08-31T05:52:34.736 INFO:teuthology.orchestra.run.smithi016.stdout:(96/150): qatlib-service-24.02.0-1.el9.x86_64.r 331 kB/s | 36 kB 00:00 2024-08-31T05:52:34.753 INFO:teuthology.orchestra.run.smithi016.stdout:(97/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-08-31T05:52:34.783 INFO:teuthology.orchestra.run.smithi155.stdout:(15/150): libcephsqlite-19.3.0-4560.gd19e6118.e 952 kB/s | 159 kB 00:00 2024-08-31T05:52:34.787 INFO:teuthology.orchestra.run.smithi016.stdout:(98/150): uuid-1.6.2-55.el9.x86_64.rpm 1.7 MB/s | 58 kB 00:00 2024-08-31T05:52:34.845 INFO:teuthology.orchestra.run.smithi016.stdout:(99/150): socat-1.7.4.1-6.el9.x86_64.rpm 2.7 MB/s | 304 kB 00:00 2024-08-31T05:52:34.871 INFO:teuthology.orchestra.run.smithi016.stdout:(100/150): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.1 MB/s | 96 kB 00:00 2024-08-31T05:52:34.900 INFO:teuthology.orchestra.run.smithi155.stdout:(16/150): librados-devel-19.3.0-4560.gd19e6118. 1.1 MB/s | 127 kB 00:00 2024-08-31T05:52:34.929 INFO:teuthology.orchestra.run.smithi016.stdout:(101/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 764 kB/s | 64 kB 00:00 2024-08-31T05:52:34.954 INFO:teuthology.orchestra.run.smithi016.stdout:(102/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 539 kB/s | 45 kB 00:00 2024-08-31T05:52:35.013 INFO:teuthology.orchestra.run.smithi016.stdout:(103/150): lua-devel-5.4.4-4.el9.x86_64.rpm 268 kB/s | 22 kB 00:00 2024-08-31T05:52:35.017 INFO:teuthology.orchestra.run.smithi155.stdout:(17/150): ceph-radosgw-19.3.0-4560.gd19e6118.el 1.5 MB/s | 10 MB 00:07 2024-08-31T05:52:35.072 INFO:teuthology.orchestra.run.smithi016.stdout:(104/150): abseil-cpp-20211102.0-3.el9.x86_64.r 9.3 MB/s | 548 kB 00:00 2024-08-31T05:52:35.105 INFO:teuthology.orchestra.run.smithi016.stdout:(105/150): gperftools-libs-2.9.1-2.el9.x86_64.r 9.1 MB/s | 309 kB 00:00 2024-08-31T05:52:35.131 INFO:teuthology.orchestra.run.smithi016.stdout:(106/150): grpc-data-1.46.7-10.el9.noarch.rpm 781 kB/s | 19 kB 00:00 2024-08-31T05:52:35.173 INFO:teuthology.orchestra.run.smithi016.stdout:(107/150): protobuf-compiler-3.14.0-13.el9.x86_ 3.9 MB/s | 863 kB 00:00 2024-08-31T05:52:35.223 INFO:teuthology.orchestra.run.smithi016.stdout:(108/150): libarrow-doc-9.0.0-11.el9.noarch.rpm 523 kB/s | 26 kB 00:00 2024-08-31T05:52:35.249 INFO:teuthology.orchestra.run.smithi016.stdout:(109/150): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-08-31T05:52:35.250 INFO:teuthology.orchestra.run.smithi155.stdout:(18/150): libradosstriper1-19.3.0-4560.gd19e611 1.4 MB/s | 506 kB 00:00 2024-08-31T05:52:35.274 INFO:teuthology.orchestra.run.smithi016.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 2.7 MB/s | 67 kB 00:00 2024-08-31T05:52:35.307 INFO:teuthology.orchestra.run.smithi016.stdout:(111/150): luarocks-3.9.2-5.el9.noarch.rpm 4.5 MB/s | 151 kB 00:00 2024-08-31T05:52:35.334 INFO:teuthology.orchestra.run.smithi155.stdout:(19/150): python3-ceph-argparse-19.3.0-4560.gd1 541 kB/s | 45 kB 00:00 2024-08-31T05:52:35.349 INFO:teuthology.orchestra.run.smithi016.stdout:(112/150): parquet-libs-9.0.0-11.el9.x86_64.rpm 20 MB/s | 839 kB 00:00 2024-08-31T05:52:35.400 INFO:teuthology.orchestra.run.smithi016.stdout:(113/150): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-08-31T05:52:35.425 INFO:teuthology.orchestra.run.smithi016.stdout:(114/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-08-31T05:52:35.450 INFO:teuthology.orchestra.run.smithi016.stdout:(115/150): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-08-31T05:52:35.451 INFO:teuthology.orchestra.run.smithi155.stdout:(20/150): python3-ceph-common-19.3.0-4560.gd19e 1.2 MB/s | 141 kB 00:00 2024-08-31T05:52:35.475 INFO:teuthology.orchestra.run.smithi016.stdout:(116/150): python3-certifi-2023.05.07-4.el9.noa 574 kB/s | 14 kB 00:00 2024-08-31T05:52:35.501 INFO:teuthology.orchestra.run.smithi016.stdout:(117/150): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-08-31T05:52:35.534 INFO:teuthology.orchestra.run.smithi016.stdout:(118/150): python3-cherrypy-18.6.1-2.el9.noarch 11 MB/s | 358 kB 00:00 2024-08-31T05:52:35.551 INFO:teuthology.orchestra.run.smithi155.stdout:(21/150): python3-cephfs-19.3.0-4560.gd19e6118. 1.6 MB/s | 161 kB 00:00 2024-08-31T05:52:35.610 INFO:teuthology.orchestra.run.smithi016.stdout:(119/150): libarrow-9.0.0-11.el9.x86_64.rpm 9.2 MB/s | 4.4 MB 00:00 2024-08-31T05:52:35.635 INFO:teuthology.orchestra.run.smithi016.stdout:(120/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.9 MB/s | 303 kB 00:00 2024-08-31T05:52:35.660 INFO:teuthology.orchestra.run.smithi016.stdout:(121/150): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-08-31T05:52:35.694 INFO:teuthology.orchestra.run.smithi016.stdout:(122/150): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-08-31T05:52:35.719 INFO:teuthology.orchestra.run.smithi016.stdout:(123/150): python3-jaraco-8.2.1-3.el9.noarch.rp 432 kB/s | 11 kB 00:00 2024-08-31T05:52:35.745 INFO:teuthology.orchestra.run.smithi016.stdout:(124/150): python3-jaraco-classes-3.2.1-5.el9.n 706 kB/s | 18 kB 00:00 2024-08-31T05:52:35.751 INFO:teuthology.orchestra.run.smithi155.stdout:(22/150): python3-rados-19.3.0-4560.gd19e6118.e 1.6 MB/s | 321 kB 00:00 2024-08-31T05:52:35.770 INFO:teuthology.orchestra.run.smithi016.stdout:(125/150): python3-jaraco-collections-3.0.0-8.e 935 kB/s | 23 kB 00:00 2024-08-31T05:52:35.816 INFO:teuthology.orchestra.run.smithi016.stdout:(126/150): python3-jaraco-functools-3.5.0-2.el9 431 kB/s | 19 kB 00:00 2024-08-31T05:52:35.845 INFO:teuthology.orchestra.run.smithi016.stdout:(127/150): python3-jaraco-text-3.2.0-6.el9.noar 670 kB/s | 20 kB 00:00 2024-08-31T05:52:35.852 INFO:teuthology.orchestra.run.smithi155.stdout:(23/150): python3-rbd-19.3.0-4560.gd19e6118.el9 3.0 MB/s | 305 kB 00:00 2024-08-31T05:52:35.861 INFO:teuthology.orchestra.run.smithi184.stdout:(10/150): ceph-common-19.3.0-4560.gd19e6118.el9 2.0 MB/s | 20 MB 00:10 2024-08-31T05:52:35.902 INFO:teuthology.orchestra.run.smithi155.stdout:(24/150): python3-rgw-19.3.0-4560.gd19e6118.el9 1.9 MB/s | 99 kB 00:00 2024-08-31T05:52:35.904 INFO:teuthology.orchestra.run.smithi016.stdout:(128/150): python3-kubernetes-26.1.0-2.el9.noar 18 MB/s | 1.0 MB 00:00 2024-08-31T05:52:35.929 INFO:teuthology.orchestra.run.smithi016.stdout:(129/150): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-08-31T05:52:35.955 INFO:teuthology.orchestra.run.smithi016.stdout:(130/150): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-08-31T05:52:35.961 INFO:teuthology.orchestra.run.smithi155.stdout:(25/150): rbd-fuse-19.3.0-4560.gd19e6118.el9.x8 1.4 MB/s | 86 kB 00:00 2024-08-31T05:52:35.978 INFO:teuthology.orchestra.run.smithi184.stdout:(11/150): cephfs-mirror-19.3.0-4560.gd19e6118.e 1.9 MB/s | 226 kB 00:00 2024-08-31T05:52:36.005 INFO:teuthology.orchestra.run.smithi016.stdout:(131/150): python3-grpcio-1.46.7-10.el9.x86_64. 5.5 MB/s | 2.0 MB 00:00 2024-08-31T05:52:36.030 INFO:teuthology.orchestra.run.smithi016.stdout:(132/150): python3-natsort-7.1.1-5.el9.noarch.r 768 kB/s | 58 kB 00:00 2024-08-31T05:52:36.045 INFO:teuthology.orchestra.run.smithi184.stdout:(12/150): cephfs-shell-19.3.0-4560.gd19e6118.el 407 kB/s | 27 kB 00:00 2024-08-31T05:52:36.055 INFO:teuthology.orchestra.run.smithi016.stdout:(133/150): python3-portend-3.1.0-2.el9.noarch.r 663 kB/s | 16 kB 00:00 2024-08-31T05:52:36.081 INFO:teuthology.orchestra.run.smithi016.stdout:(134/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-08-31T05:52:36.098 INFO:teuthology.orchestra.run.smithi016.stdout:(135/150): python3-pyperclip-1.8.2-2.el9.noarch 1.7 MB/s | 28 kB 00:00 2024-08-31T05:52:36.122 INFO:teuthology.orchestra.run.smithi184.stdout:(13/150): libcephfs-devel-19.3.0-4560.gd19e6118 418 kB/s | 32 kB 00:00 2024-08-31T05:52:36.123 INFO:teuthology.orchestra.run.smithi016.stdout:(136/150): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-08-31T05:52:36.149 INFO:teuthology.orchestra.run.smithi016.stdout:(137/150): python3-repoze-lru-0.7-16.el9.noarch 610 kB/s | 31 kB 00:00 2024-08-31T05:52:36.174 INFO:teuthology.orchestra.run.smithi016.stdout:(138/150): python3-rsa-4.9-2.el9.noarch.rpm 2.4 MB/s | 59 kB 00:00 2024-08-31T05:52:36.199 INFO:teuthology.orchestra.run.smithi016.stdout:(139/150): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-08-31T05:52:36.225 INFO:teuthology.orchestra.run.smithi016.stdout:(140/150): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-08-31T05:52:36.250 INFO:teuthology.orchestra.run.smithi016.stdout:(141/150): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-08-31T05:52:36.275 INFO:teuthology.orchestra.run.smithi016.stdout:(142/150): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-08-31T05:52:36.300 INFO:teuthology.orchestra.run.smithi016.stdout:(143/150): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-08-31T05:52:36.326 INFO:teuthology.orchestra.run.smithi016.stdout:(144/150): python3-zc-lockfile-2.0-10.el9.noarc 803 kB/s | 20 kB 00:00 2024-08-31T05:52:36.351 INFO:teuthology.orchestra.run.smithi016.stdout:(145/150): re2-20211101-20.el9.x86_64.rpm 7.5 MB/s | 191 kB 00:00 2024-08-31T05:52:36.379 INFO:teuthology.orchestra.run.smithi184.stdout:(14/150): libcephfs2-19.3.0-4560.gd19e6118.el9. 2.9 MB/s | 766 kB 00:00 2024-08-31T05:52:36.385 INFO:teuthology.orchestra.run.smithi016.stdout:(146/150): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-08-31T05:52:36.435 INFO:teuthology.orchestra.run.smithi016.stdout:(147/150): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-08-31T05:52:36.471 INFO:teuthology.orchestra.run.smithi184.stdout:(15/150): libcephsqlite-19.3.0-4560.gd19e6118.e 1.7 MB/s | 159 kB 00:00 2024-08-31T05:52:36.555 INFO:teuthology.orchestra.run.smithi184.stdout:(16/150): librados-devel-19.3.0-4560.gd19e6118. 1.5 MB/s | 127 kB 00:00 2024-08-31T05:52:36.755 INFO:teuthology.orchestra.run.smithi184.stdout:(17/150): libradosstriper1-19.3.0-4560.gd19e611 2.5 MB/s | 506 kB 00:00 2024-08-31T05:52:36.828 INFO:teuthology.orchestra.run.smithi155.stdout:(26/150): librgw2-19.3.0-4560.gd19e6118.el9.x86 2.9 MB/s | 5.3 MB 00:01 2024-08-31T05:52:36.877 INFO:teuthology.orchestra.run.smithi016.stdout:(148/150): librbd1-19.3.0-4560.gd19e6118.el9.x8 7.1 MB/s | 3.1 MB 00:00 2024-08-31T05:52:36.920 INFO:teuthology.orchestra.run.smithi155.stdout:(27/150): rbd-nbd-19.3.0-4560.gd19e6118.el9.x86 1.8 MB/s | 170 kB 00:00 2024-08-31T05:52:36.944 INFO:teuthology.orchestra.run.smithi016.stdout:(149/150): librados2-19.3.0-4560.gd19e6118.el9. 6.1 MB/s | 3.4 MB 00:00 2024-08-31T05:52:36.970 INFO:teuthology.orchestra.run.smithi155.stdout:(28/150): rbd-mirror-19.3.0-4560.gd19e6118.el9. 3.1 MB/s | 3.1 MB 00:01 2024-08-31T05:52:36.995 INFO:teuthology.orchestra.run.smithi155.stdout:(29/150): ceph-grafana-dashboards-19.3.0-4560.g 456 kB/s | 34 kB 00:00 2024-08-31T05:52:37.037 INFO:teuthology.orchestra.run.smithi155.stdout:(30/150): ceph-mgr-cephadm-19.3.0-4560.gd19e611 2.3 MB/s | 154 kB 00:00 2024-08-31T05:52:37.222 INFO:teuthology.orchestra.run.smithi184.stdout:(18/150): ceph-osd-19.3.0-4560.gd19e6118.el9.x8 2.2 MB/s | 18 MB 00:08 2024-08-31T05:52:37.622 INFO:teuthology.orchestra.run.smithi184.stdout:(19/150): ceph-test-19.3.0-4560.gd19e6118.el9.x 7.7 MB/s | 44 MB 00:05 2024-08-31T05:52:37.647 INFO:teuthology.orchestra.run.smithi184.stdout:(20/150): python3-ceph-argparse-19.3.0-4560.gd1 106 kB/s | 45 kB 00:00 2024-08-31T05:52:37.673 INFO:teuthology.orchestra.run.smithi184.stdout:(21/150): python3-ceph-common-19.3.0-4560.gd19e 2.8 MB/s | 141 kB 00:00 2024-08-31T05:52:37.715 INFO:teuthology.orchestra.run.smithi184.stdout:(22/150): python3-cephfs-19.3.0-4560.gd19e6118. 2.4 MB/s | 161 kB 00:00 2024-08-31T05:52:37.765 INFO:teuthology.orchestra.run.smithi184.stdout:(23/150): python3-rados-19.3.0-4560.gd19e6118.e 3.4 MB/s | 321 kB 00:00 2024-08-31T05:52:37.823 INFO:teuthology.orchestra.run.smithi184.stdout:(24/150): python3-rgw-19.3.0-4560.gd19e6118.el9 1.7 MB/s | 99 kB 00:00 2024-08-31T05:52:37.849 INFO:teuthology.orchestra.run.smithi184.stdout:(25/150): python3-rbd-19.3.0-4560.gd19e6118.el9 2.2 MB/s | 305 kB 00:00 2024-08-31T05:52:37.874 INFO:teuthology.orchestra.run.smithi184.stdout:(26/150): rbd-fuse-19.3.0-4560.gd19e6118.el9.x8 1.7 MB/s | 86 kB 00:00 2024-08-31T05:52:37.924 INFO:teuthology.orchestra.run.smithi184.stdout:(27/150): rbd-nbd-19.3.0-4560.gd19e6118.el9.x86 3.3 MB/s | 170 kB 00:00 2024-08-31T05:52:37.966 INFO:teuthology.orchestra.run.smithi184.stdout:(28/150): ceph-grafana-dashboards-19.3.0-4560.g 828 kB/s | 34 kB 00:00 2024-08-31T05:52:38.017 INFO:teuthology.orchestra.run.smithi184.stdout:(29/150): ceph-mgr-cephadm-19.3.0-4560.gd19e611 3.0 MB/s | 154 kB 00:00 2024-08-31T05:52:38.167 INFO:teuthology.orchestra.run.smithi184.stdout:(30/150): librgw2-19.3.0-4560.gd19e6118.el9.x86 3.8 MB/s | 5.3 MB 00:01 2024-08-31T05:52:38.238 INFO:teuthology.orchestra.run.smithi155.stdout:(31/150): ceph-mgr-diskprediction-local-19.3.0- 6.2 MB/s | 7.4 MB 00:01 2024-08-31T05:52:38.313 INFO:teuthology.orchestra.run.smithi155.stdout:(32/150): ceph-mgr-modules-core-19.3.0-4560.gd1 3.8 MB/s | 290 kB 00:00 2024-08-31T05:52:38.379 INFO:teuthology.orchestra.run.smithi155.stdout:(33/150): ceph-mgr-rook-19.3.0-4560.gd19e6118.e 746 kB/s | 49 kB 00:00 2024-08-31T05:52:38.401 INFO:teuthology.orchestra.run.smithi184.stdout:(31/150): rbd-mirror-19.3.0-4560.gd19e6118.el9. 5.6 MB/s | 3.1 MB 00:00 2024-08-31T05:52:38.430 INFO:teuthology.orchestra.run.smithi155.stdout:(34/150): ceph-prometheus-alerts-19.3.0-4560.gd 326 kB/s | 16 kB 00:00 2024-08-31T05:52:38.476 INFO:teuthology.orchestra.run.smithi184.stdout:(32/150): ceph-mgr-modules-core-19.3.0-4560.gd1 3.8 MB/s | 290 kB 00:00 2024-08-31T05:52:38.497 INFO:teuthology.orchestra.run.smithi155.stdout:(35/150): ceph-volume-19.3.0-4560.gd19e6118.el9 4.2 MB/s | 285 kB 00:00 2024-08-31T05:52:38.528 INFO:teuthology.orchestra.run.smithi184.stdout:(33/150): ceph-mgr-rook-19.3.0-4560.gd19e6118.e 991 kB/s | 49 kB 00:00 2024-08-31T05:52:38.576 INFO:teuthology.orchestra.run.smithi184.stdout:(34/150): ceph-prometheus-alerts-19.3.0-4560.gd 330 kB/s | 16 kB 00:00 2024-08-31T05:52:38.639 INFO:teuthology.orchestra.run.smithi155.stdout:(36/150): cephadm-19.3.0-4560.gd19e6118.el9.noa 5.4 MB/s | 781 kB 00:00 2024-08-31T05:52:38.651 INFO:teuthology.orchestra.run.smithi184.stdout:(35/150): ceph-volume-19.3.0-4560.gd19e6118.el9 3.7 MB/s | 285 kB 00:00 2024-08-31T05:52:38.689 INFO:teuthology.orchestra.run.smithi155.stdout:(37/150): cephfs-top-19.3.0-4560.gd19e6118.el9. 366 kB/s | 18 kB 00:00 2024-08-31T05:52:38.795 INFO:teuthology.orchestra.run.smithi184.stdout:(36/150): cephadm-19.3.0-4560.gd19e6118.el9.noa 5.4 MB/s | 781 kB 00:00 2024-08-31T05:52:38.823 INFO:teuthology.orchestra.run.smithi155.stdout:(38/150): fuse-2.9.9-16.el9.x86_64.rpm 599 kB/s | 80 kB 00:00 2024-08-31T05:52:38.844 INFO:teuthology.orchestra.run.smithi184.stdout:(37/150): cephfs-top-19.3.0-4560.gd19e6118.el9. 365 kB/s | 18 kB 00:00 2024-08-31T05:52:39.011 INFO:teuthology.orchestra.run.smithi184.stdout:(38/150): ceph-mgr-dashboard-19.3.0-4560.gd19e6 14 MB/s | 14 MB 00:00 2024-08-31T05:52:39.173 INFO:teuthology.orchestra.run.smithi155.stdout:(39/150): ceph-mgr-dashboard-19.3.0-4560.gd19e6 6.5 MB/s | 14 MB 00:02 2024-08-31T05:52:39.207 INFO:teuthology.orchestra.run.smithi155.stdout:(40/150): libgfortran-11.5.0-2.el9.x86_64.rpm 2.0 MB/s | 801 kB 00:00 2024-08-31T05:52:39.278 INFO:teuthology.orchestra.run.smithi184.stdout:(39/150): fuse-2.9.9-16.el9.x86_64.rpm 184 kB/s | 80 kB 00:00 2024-08-31T05:52:39.315 INFO:teuthology.orchestra.run.smithi155.stdout:(41/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 253 kB 00:00 2024-08-31T05:52:39.341 INFO:teuthology.orchestra.run.smithi155.stdout:(42/150): libquadmath-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 191 kB 00:00 2024-08-31T05:52:39.391 INFO:teuthology.orchestra.run.smithi155.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-08-31T05:52:39.453 INFO:teuthology.orchestra.run.smithi184.stdout:(40/150): ceph-mgr-diskprediction-local-19.3.0- 5.7 MB/s | 7.4 MB 00:01 2024-08-31T05:52:39.478 INFO:teuthology.orchestra.run.smithi184.stdout:(41/150): libquadmath-11.5.0-2.el9.x86_64.rpm 954 kB/s | 191 kB 00:00 2024-08-31T05:52:39.483 INFO:teuthology.orchestra.run.smithi155.stdout:(44/150): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 135 kB 00:00 2024-08-31T05:52:39.567 INFO:teuthology.orchestra.run.smithi155.stdout:(45/150): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-08-31T05:52:39.594 INFO:teuthology.orchestra.run.smithi016.stdout:(150/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 3.1 MB/s | 16 MB 00:05 2024-08-31T05:52:39.599 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-08-31T05:52:39.600 INFO:teuthology.orchestra.run.smithi016.stdout:Total 14 MB/s | 224 MB 00:16 2024-08-31T05:52:39.675 INFO:teuthology.orchestra.run.smithi155.stdout:(46/150): python3-urllib3-1.26.5-6.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-08-31T05:52:39.704 INFO:teuthology.orchestra.run.smithi184.stdout:(42/150): libgfortran-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 801 kB 00:00 2024-08-31T05:52:39.792 INFO:teuthology.orchestra.run.smithi155.stdout:(47/150): python3-cryptography-36.0.1-4.el9.x86 2.6 MB/s | 1.2 MB 00:00 2024-08-31T05:52:39.820 INFO:teuthology.orchestra.run.smithi184.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 914 kB/s | 106 kB 00:00 2024-08-31T05:52:39.854 INFO:teuthology.orchestra.run.smithi184.stdout:(44/150): python3-cryptography-36.0.1-4.el9.x86 3.3 MB/s | 1.2 MB 00:00 2024-08-31T05:52:39.901 INFO:teuthology.orchestra.run.smithi155.stdout:(48/150): automake-1.16.2-8.el9.noarch.rpm 3.0 MB/s | 689 kB 00:00 2024-08-31T05:52:39.929 INFO:teuthology.orchestra.run.smithi184.stdout:(45/150): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-08-31T05:52:39.951 INFO:teuthology.orchestra.run.smithi155.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-08-31T05:52:39.954 INFO:teuthology.orchestra.run.smithi184.stdout:(46/150): python3-requests-2.25.1-8.el9.noarch. 1.2 MB/s | 125 kB 00:00 2024-08-31T05:52:40.001 INFO:teuthology.orchestra.run.smithi155.stdout:(50/150): elfutils-libelf-devel-0.191-4.el9.x86 434 kB/s | 22 kB 00:00 2024-08-31T05:52:40.038 INFO:teuthology.orchestra.run.smithi184.stdout:(47/150): python3-urllib3-1.26.5-6.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-08-31T05:52:40.118 INFO:teuthology.orchestra.run.smithi155.stdout:(51/150): bison-3.7.4-5.el9.x86_64.rpm 2.8 MB/s | 943 kB 00:00 2024-08-31T05:52:40.122 INFO:teuthology.orchestra.run.smithi184.stdout:(48/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 379 kB/s | 253 kB 00:00 2024-08-31T05:52:40.144 INFO:teuthology.orchestra.run.smithi155.stdout:(52/150): flex-2.6.4-9.el9.x86_64.rpm 2.2 MB/s | 313 kB 00:00 2024-08-31T05:52:40.169 INFO:teuthology.orchestra.run.smithi155.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 650 kB/s | 33 kB 00:00 2024-08-31T05:52:40.219 INFO:teuthology.orchestra.run.smithi155.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-08-31T05:52:40.270 INFO:teuthology.orchestra.run.smithi155.stdout:(55/150): libaio-devel-0.3.111-13.el9.x86_64.rp 224 kB/s | 11 kB 00:00 2024-08-31T05:52:40.353 INFO:teuthology.orchestra.run.smithi155.stdout:(56/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-08-31T05:52:40.380 INFO:teuthology.orchestra.run.smithi184.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 414 kB/s | 107 kB 00:00 2024-08-31T05:52:40.403 INFO:teuthology.orchestra.run.smithi155.stdout:(57/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 904 kB/s | 45 kB 00:00 2024-08-31T05:52:40.456 INFO:teuthology.orchestra.run.smithi184.stdout:(50/150): elfutils-libelf-devel-0.191-4.el9.x86 289 kB/s | 22 kB 00:00 2024-08-31T05:52:40.623 INFO:teuthology.orchestra.run.smithi184.stdout:(51/150): automake-1.16.2-8.el9.noarch.rpm 1.0 MB/s | 689 kB 00:00 2024-08-31T05:52:40.654 INFO:teuthology.orchestra.run.smithi155.stdout:(58/150): librdkafka-1.6.1-102.el9.x86_64.rpm 2.6 MB/s | 662 kB 00:00 2024-08-31T05:52:40.698 INFO:teuthology.orchestra.run.smithi184.stdout:(52/150): flexiblas-3.0.4-8.el9.x86_64.rpm 437 kB/s | 33 kB 00:00 2024-08-31T05:52:40.748 INFO:teuthology.orchestra.run.smithi184.stdout:(53/150): bison-3.7.4-5.el9.x86_64.rpm 1.3 MB/s | 943 kB 00:00 2024-08-31T05:52:40.773 INFO:teuthology.orchestra.run.smithi184.stdout:(54/150): flex-2.6.4-9.el9.x86_64.rpm 988 kB/s | 313 kB 00:00 2024-08-31T05:52:40.807 INFO:teuthology.orchestra.run.smithi184.stdout:(55/150): flexiblas-openblas-openmp-3.0.4-8.el9 311 kB/s | 18 kB 00:00 2024-08-31T05:52:40.832 INFO:teuthology.orchestra.run.smithi184.stdout:(56/150): libaio-devel-0.3.111-13.el9.x86_64.rp 190 kB/s | 11 kB 00:00 2024-08-31T05:52:40.837 INFO:teuthology.orchestra.run.smithi155.stdout:(59/150): libtool-2.4.6-46.el9.x86_64.rpm 3.1 MB/s | 581 kB 00:00 2024-08-31T05:52:40.888 INFO:teuthology.orchestra.run.smithi155.stdout:(60/150): libuuid-devel-2.37.4-20.el9.x86_64.rp 491 kB/s | 24 kB 00:00 2024-08-31T05:52:40.899 INFO:teuthology.orchestra.run.smithi184.stdout:(57/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 680 kB/s | 45 kB 00:00 2024-08-31T05:52:40.925 INFO:teuthology.orchestra.run.smithi184.stdout:(58/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-08-31T05:52:40.938 INFO:teuthology.orchestra.run.smithi155.stdout:(61/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.8 MB/s | 3.0 MB 00:00 2024-08-31T05:52:40.963 INFO:teuthology.orchestra.run.smithi155.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 631 kB/s | 47 kB 00:00 2024-08-31T05:52:41.022 INFO:teuthology.orchestra.run.smithi155.stdout:(63/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-08-31T05:52:41.047 INFO:teuthology.orchestra.run.smithi155.stdout:(64/150): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-08-31T05:52:41.072 INFO:teuthology.orchestra.run.smithi155.stdout:(65/150): openblas-0.3.26-2.el9.x86_64.rpm 768 kB/s | 38 kB 00:00 2024-08-31T05:52:41.133 INFO:teuthology.orchestra.run.smithi184.stdout:(59/150): libtool-2.4.6-46.el9.x86_64.rpm 2.7 MB/s | 581 kB 00:00 2024-08-31T05:52:41.167 INFO:teuthology.orchestra.run.smithi184.stdout:(60/150): librdkafka-1.6.1-102.el9.x86_64.rpm 2.4 MB/s | 662 kB 00:00 2024-08-31T05:52:41.200 INFO:teuthology.orchestra.run.smithi184.stdout:(61/150): libuuid-devel-2.37.4-20.el9.x86_64.rp 365 kB/s | 24 kB 00:00 2024-08-31T05:52:41.342 INFO:teuthology.orchestra.run.smithi184.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 270 kB/s | 47 kB 00:00 2024-08-31T05:52:41.368 INFO:teuthology.orchestra.run.smithi184.stdout:(63/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.7 MB/s | 292 kB 00:00 2024-08-31T05:52:41.435 INFO:teuthology.orchestra.run.smithi184.stdout:(64/150): openblas-0.3.26-2.el9.x86_64.rpm 577 kB/s | 38 kB 00:00 2024-08-31T05:52:41.468 INFO:teuthology.orchestra.run.smithi184.stdout:(65/150): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-08-31T05:52:41.487 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-08-31T05:52:41.619 INFO:teuthology.orchestra.run.smithi184.stdout:(66/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.2 MB/s | 3.0 MB 00:00 2024-08-31T05:52:41.653 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-08-31T05:52:41.653 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-08-31T05:52:41.806 INFO:teuthology.orchestra.run.smithi155.stdout:(66/150): openssl-devel-3.2.2-4.el9.x86_64.rpm 6.0 MB/s | 4.4 MB 00:00 2024-08-31T05:52:41.890 INFO:teuthology.orchestra.run.smithi155.stdout:(67/150): openblas-openmp-0.3.26-2.el9.x86_64.r 5.9 MB/s | 4.9 MB 00:00 2024-08-31T05:52:42.032 INFO:teuthology.orchestra.run.smithi155.stdout:(68/150): postgresql-contrib-13.16-1.el9.x86_64 6.3 MB/s | 910 kB 00:00 2024-08-31T05:52:42.044 INFO:teuthology.orchestra.run.smithi184.stdout:(67/150): postgresql-13.16-1.el9.x86_64.rpm 3.7 MB/s | 1.6 MB 00:00 2024-08-31T05:52:42.098 INFO:teuthology.orchestra.run.smithi155.stdout:(69/150): postgresql-13.16-1.el9.x86_64.rpm 5.4 MB/s | 1.6 MB 00:00 2024-08-31T05:52:42.124 INFO:teuthology.orchestra.run.smithi155.stdout:(70/150): postgresql-private-libs-13.16-1.el9.x 1.5 MB/s | 137 kB 00:00 2024-08-31T05:52:42.344 INFO:teuthology.orchestra.run.smithi184.stdout:(68/150): postgresql-contrib-13.16-1.el9.x86_64 3.0 MB/s | 910 kB 00:00 2024-08-31T05:52:42.411 INFO:teuthology.orchestra.run.smithi184.stdout:(69/150): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-08-31T05:52:42.541 INFO:teuthology.orchestra.run.smithi155.stdout:(71/150): ceph-test-19.3.0-4560.gd19e6118.el9.x 4.9 MB/s | 44 MB 00:08 2024-08-31T05:52:42.583 INFO:teuthology.orchestra.run.smithi155.stdout:(72/150): protobuf-3.14.0-13.el9.x86_64.rpm 2.2 MB/s | 1.0 MB 00:00 2024-08-31T05:52:42.686 INFO:teuthology.orchestra.run.smithi184.stdout:(70/150): openssl-devel-3.2.2-4.el9.x86_64.rpm 3.6 MB/s | 4.4 MB 00:01 2024-08-31T05:52:42.691 INFO:teuthology.orchestra.run.smithi155.stdout:(73/150): python3-attrs-20.3.0-7.el9.noarch.rpm 584 kB/s | 87 kB 00:00 2024-08-31T05:52:42.742 INFO:teuthology.orchestra.run.smithi155.stdout:(74/150): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-08-31T05:52:42.803 INFO:teuthology.orchestra.run.smithi184.stdout:(71/150): openblas-openmp-0.3.26-2.el9.x86_64.r 3.6 MB/s | 4.9 MB 00:01 2024-08-31T05:52:42.850 INFO:teuthology.orchestra.run.smithi155.stdout:(75/150): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-08-31T05:52:42.870 INFO:teuthology.orchestra.run.smithi184.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-08-31T05:52:42.959 INFO:teuthology.orchestra.run.smithi155.stdout:(76/150): python3-jinja2-2.11.3-6.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-08-31T05:52:43.001 INFO:teuthology.orchestra.run.smithi155.stdout:(77/150): python3-jsonpatch-1.21-16.el9.noarch. 615 kB/s | 26 kB 00:00 2024-08-31T05:52:43.029 INFO:teuthology.orchestra.run.smithi184.stdout:(73/150): protobuf-3.14.0-13.el9.x86_64.rpm 2.9 MB/s | 1.0 MB 00:00 2024-08-31T05:52:43.043 INFO:teuthology.orchestra.run.smithi155.stdout:(78/150): python3-jsonpointer-2.0-4.el9.noarch. 460 kB/s | 19 kB 00:00 2024-08-31T05:52:43.143 INFO:teuthology.orchestra.run.smithi155.stdout:(79/150): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-08-31T05:52:43.154 INFO:teuthology.orchestra.run.smithi184.stdout:(74/150): python3-colorama-0.4.6-3.el9.noarch.r 490 kB/s | 61 kB 00:00 2024-08-31T05:52:43.168 INFO:teuthology.orchestra.run.smithi155.stdout:(80/150): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-08-31T05:52:43.235 INFO:teuthology.orchestra.run.smithi155.stdout:(81/150): postgresql-server-13.16-1.el9.x86_64. 5.1 MB/s | 5.8 MB 00:01 2024-08-31T05:52:43.261 INFO:teuthology.orchestra.run.smithi155.stdout:(82/150): python3-markupsafe-1.1.1-12.el9.x86_6 297 kB/s | 35 kB 00:00 2024-08-31T05:52:43.263 INFO:teuthology.orchestra.run.smithi184.stdout:(75/150): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-08-31T05:52:43.311 INFO:teuthology.orchestra.run.smithi155.stdout:(83/150): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-08-31T05:52:43.345 INFO:teuthology.orchestra.run.smithi155.stdout:(84/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.2 MB/s | 250 kB 00:00 2024-08-31T05:52:43.370 INFO:teuthology.orchestra.run.smithi155.stdout:(85/150): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-08-31T05:52:43.379 INFO:teuthology.orchestra.run.smithi184.stdout:(76/150): python3-jinja2-2.11.3-6.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-08-31T05:52:43.395 INFO:teuthology.orchestra.run.smithi155.stdout:(86/150): python3-prettytable-0.7.2-27.el9.noar 843 kB/s | 42 kB 00:00 2024-08-31T05:52:43.429 INFO:teuthology.orchestra.run.smithi155.stdout:(87/150): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-08-31T05:52:43.447 INFO:teuthology.orchestra.run.smithi184.stdout:(77/150): python3-jsonpatch-1.21-16.el9.noarch. 384 kB/s | 26 kB 00:00 2024-08-31T05:52:43.454 INFO:teuthology.orchestra.run.smithi155.stdout:(88/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-08-31T05:52:43.514 INFO:teuthology.orchestra.run.smithi184.stdout:(78/150): python3-jsonpointer-2.0-4.el9.noarch. 288 kB/s | 19 kB 00:00 2024-08-31T05:52:43.555 INFO:teuthology.orchestra.run.smithi155.stdout:(89/150): python3-pyasn1-modules-0.4.8-6.el9.no 2.2 MB/s | 279 kB 00:00 2024-08-31T05:52:43.588 INFO:teuthology.orchestra.run.smithi155.stdout:(90/150): python3-pytz-2021.1-5.el9.noarch.rpm 382 kB/s | 51 kB 00:00 2024-08-31T05:52:43.589 INFO:teuthology.orchestra.run.smithi184.stdout:(79/150): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-08-31T05:52:43.612 INFO:teuthology.orchestra.run.smithi155.stdout:(91/150): python3-requests-oauthlib-1.3.0-12.el 953 kB/s | 54 kB 00:00 2024-08-31T05:52:43.662 INFO:teuthology.orchestra.run.smithi155.stdout:(92/150): python3-toml-0.10.2-6.el9.noarch.rpm 832 kB/s | 42 kB 00:00 2024-08-31T05:52:43.664 INFO:teuthology.orchestra.run.smithi184.stdout:(80/150): python3-markupsafe-1.1.1-12.el9.x86_6 464 kB/s | 35 kB 00:00 2024-08-31T05:52:43.712 INFO:teuthology.orchestra.run.smithi155.stdout:(93/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 886 kB/s | 44 kB 00:00 2024-08-31T05:52:43.771 INFO:teuthology.orchestra.run.smithi155.stdout:(94/150): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-08-31T05:52:43.789 INFO:teuthology.orchestra.run.smithi184.stdout:(81/150): postgresql-server-13.16-1.el9.x86_64. 4.2 MB/s | 5.8 MB 00:01 2024-08-31T05:52:43.821 INFO:teuthology.orchestra.run.smithi155.stdout:(95/150): qatlib-service-24.02.0-1.el9.x86_64.r 717 kB/s | 36 kB 00:00 2024-08-31T05:52:43.872 INFO:teuthology.orchestra.run.smithi155.stdout:(96/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-08-31T05:52:43.906 INFO:teuthology.orchestra.run.smithi184.stdout:(82/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-08-31T05:52:43.930 INFO:teuthology.orchestra.run.smithi155.stdout:(97/150): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-08-31T05:52:43.982 INFO:teuthology.orchestra.run.smithi184.stdout:(83/150): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 222 kB 00:00 2024-08-31T05:52:43.989 INFO:teuthology.orchestra.run.smithi155.stdout:(98/150): uuid-1.6.2-55.el9.x86_64.rpm 982 kB/s | 58 kB 00:00 2024-08-31T05:52:44.048 INFO:teuthology.orchestra.run.smithi155.stdout:(99/150): xfsprogs-devel-6.4.0-4.el9.x86_64.rpm 1.6 MB/s | 96 kB 00:00 2024-08-31T05:52:44.049 INFO:teuthology.orchestra.run.smithi184.stdout:(84/150): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-08-31T05:52:44.098 INFO:teuthology.orchestra.run.smithi155.stdout:(100/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-08-31T05:52:44.115 INFO:teuthology.orchestra.run.smithi184.stdout:(85/150): python3-prettytable-0.7.2-27.el9.noar 634 kB/s | 42 kB 00:00 2024-08-31T05:52:44.149 INFO:teuthology.orchestra.run.smithi155.stdout:(101/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 897 kB/s | 45 kB 00:00 2024-08-31T05:52:44.173 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-08-31T05:52:44.173 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-08-31T05:52:44.241 INFO:teuthology.orchestra.run.smithi155.stdout:(102/150): lua-devel-5.4.4-4.el9.x86_64.rpm 244 kB/s | 22 kB 00:00 2024-08-31T05:52:44.274 INFO:teuthology.orchestra.run.smithi184.stdout:(86/150): python3-babel-2.9.1-2.el9.noarch.rpm 4.2 MB/s | 6.0 MB 00:01 2024-08-31T05:52:44.300 INFO:teuthology.orchestra.run.smithi184.stdout:(87/150): python3-protobuf-3.14.0-13.el9.noarch 1.4 MB/s | 269 kB 00:00 2024-08-31T05:52:44.383 INFO:teuthology.orchestra.run.smithi184.stdout:(88/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-08-31T05:52:44.408 INFO:teuthology.orchestra.run.smithi155.stdout:(103/150): protobuf-compiler-3.14.0-13.el9.x86_ 5.1 MB/s | 863 kB 00:00 2024-08-31T05:52:44.417 INFO:teuthology.orchestra.run.smithi184.stdout:(89/150): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-08-31T05:52:44.458 INFO:teuthology.orchestra.run.smithi155.stdout:(104/150): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-08-31T05:52:44.484 INFO:teuthology.orchestra.run.smithi184.stdout:(90/150): python3-pytz-2021.1-5.el9.noarch.rpm 509 kB/s | 51 kB 00:00 2024-08-31T05:52:44.491 INFO:teuthology.orchestra.run.smithi155.stdout:(105/150): gperftools-libs-2.9.1-2.el9.x86_64.r 9.0 MB/s | 309 kB 00:00 2024-08-31T05:52:44.517 INFO:teuthology.orchestra.run.smithi155.stdout:(106/150): grpc-data-1.46.7-10.el9.noarch.rpm 783 kB/s | 19 kB 00:00 2024-08-31T05:52:44.518 INFO:teuthology.orchestra.run.smithi184.stdout:(91/150): python3-requests-oauthlib-1.3.0-12.el 535 kB/s | 54 kB 00:00 2024-08-31T05:52:44.577 INFO:teuthology.orchestra.run.smithi184.stdout:(92/150): python3-toml-0.10.2-6.el9.noarch.rpm 715 kB/s | 42 kB 00:00 2024-08-31T05:52:44.584 INFO:teuthology.orchestra.run.smithi155.stdout:(107/150): python3-numpy-1.20.1-5.el9.x86_64.rp 3.6 MB/s | 5.1 MB 00:01 2024-08-31T05:52:44.626 INFO:teuthology.orchestra.run.smithi155.stdout:(108/150): libarrow-doc-9.0.0-11.el9.noarch.rpm 625 kB/s | 26 kB 00:00 2024-08-31T05:52:44.643 INFO:teuthology.orchestra.run.smithi184.stdout:(93/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 662 kB/s | 44 kB 00:00 2024-08-31T05:52:44.651 INFO:teuthology.orchestra.run.smithi155.stdout:(109/150): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-08-31T05:52:44.685 INFO:teuthology.orchestra.run.smithi155.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-08-31T05:52:44.710 INFO:teuthology.orchestra.run.smithi184.stdout:(94/150): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-08-31T05:52:44.718 INFO:teuthology.orchestra.run.smithi155.stdout:(111/150): luarocks-3.9.2-5.el9.noarch.rpm 4.4 MB/s | 151 kB 00:00 2024-08-31T05:52:44.777 INFO:teuthology.orchestra.run.smithi184.stdout:(95/150): qatlib-service-24.02.0-1.el9.x86_64.r 538 kB/s | 36 kB 00:00 2024-08-31T05:52:44.785 INFO:teuthology.orchestra.run.smithi155.stdout:(112/150): libarrow-9.0.0-11.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-08-31T05:52:44.827 INFO:teuthology.orchestra.run.smithi155.stdout:(113/150): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-08-31T05:52:44.852 INFO:teuthology.orchestra.run.smithi155.stdout:(114/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-08-31T05:52:44.860 INFO:teuthology.orchestra.run.smithi184.stdout:(96/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.3 MB/s | 5.1 MB 00:01 2024-08-31T05:52:44.878 INFO:teuthology.orchestra.run.smithi155.stdout:(115/150): python3-cachetools-4.2.4-1.el9.noarc 1.2 MB/s | 32 kB 00:00 2024-08-31T05:52:44.886 INFO:teuthology.orchestra.run.smithi184.stdout:(97/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 435 kB/s | 47 kB 00:00 2024-08-31T05:52:44.912 INFO:teuthology.orchestra.run.smithi155.stdout:(116/150): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.2 MB/s | 839 kB 00:00 2024-08-31T05:52:44.937 INFO:teuthology.orchestra.run.smithi155.stdout:(117/150): python3-certifi-2023.05.07-4.el9.noa 240 kB/s | 14 kB 00:00 2024-08-31T05:52:44.962 INFO:teuthology.orchestra.run.smithi155.stdout:(118/150): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-08-31T05:52:44.970 INFO:teuthology.orchestra.run.smithi184.stdout:(98/150): socat-1.7.4.1-6.el9.x86_64.rpm 2.7 MB/s | 304 kB 00:00 2024-08-31T05:52:44.987 INFO:teuthology.orchestra.run.smithi155.stdout:(119/150): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-08-31T05:52:44.995 INFO:teuthology.orchestra.run.smithi184.stdout:(99/150): uuid-1.6.2-55.el9.x86_64.rpm 532 kB/s | 58 kB 00:00 2024-08-31T05:52:45.021 INFO:teuthology.orchestra.run.smithi155.stdout:(120/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-08-31T05:52:45.037 INFO:teuthology.orchestra.run.smithi184.stdout:(100/150): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.4 MB/s | 96 kB 00:00 2024-08-31T05:52:45.046 INFO:teuthology.orchestra.run.smithi155.stdout:(121/150): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-08-31T05:52:45.071 INFO:teuthology.orchestra.run.smithi155.stdout:(122/150): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-08-31T05:52:45.104 INFO:teuthology.orchestra.run.smithi184.stdout:(101/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 672 kB/s | 45 kB 00:00 2024-08-31T05:52:45.105 INFO:teuthology.orchestra.run.smithi155.stdout:(123/150): python3-jaraco-8.2.1-3.el9.noarch.rp 322 kB/s | 11 kB 00:00 2024-08-31T05:52:45.122 INFO:teuthology.orchestra.run.smithi155.stdout:(124/150): python3-jaraco-classes-3.2.1-5.el9.n 1.0 MB/s | 18 kB 00:00 2024-08-31T05:52:45.129 INFO:teuthology.orchestra.run.smithi184.stdout:(102/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 476 kB/s | 64 kB 00:00 2024-08-31T05:52:45.139 INFO:teuthology.orchestra.run.smithi155.stdout:(125/150): python3-jaraco-collections-3.0.0-8.e 1.4 MB/s | 23 kB 00:00 2024-08-31T05:52:45.164 INFO:teuthology.orchestra.run.smithi155.stdout:(126/150): python3-jaraco-functools-3.5.0-2.el9 777 kB/s | 19 kB 00:00 2024-08-31T05:52:45.190 INFO:teuthology.orchestra.run.smithi155.stdout:(127/150): python3-jaraco-text-3.2.0-6.el9.noar 794 kB/s | 20 kB 00:00 2024-08-31T05:52:45.204 INFO:teuthology.orchestra.run.smithi184.stdout:(103/150): lua-devel-5.4.4-4.el9.x86_64.rpm 222 kB/s | 22 kB 00:00 2024-08-31T05:52:45.240 INFO:teuthology.orchestra.run.smithi155.stdout:(128/150): python3-grpcio-1.46.7-10.el9.x86_64. 9.3 MB/s | 2.0 MB 00:00 2024-08-31T05:52:45.254 INFO:teuthology.orchestra.run.smithi184.stdout:(104/150): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-08-31T05:52:45.265 INFO:teuthology.orchestra.run.smithi155.stdout:(129/150): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-08-31T05:52:45.288 INFO:teuthology.orchestra.run.smithi184.stdout:(105/150): gperftools-libs-2.9.1-2.el9.x86_64.r 9.1 MB/s | 309 kB 00:00 2024-08-31T05:52:45.290 INFO:teuthology.orchestra.run.smithi155.stdout:(130/150): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-08-31T05:52:45.313 INFO:teuthology.orchestra.run.smithi184.stdout:(106/150): grpc-data-1.46.7-10.el9.noarch.rpm 782 kB/s | 19 kB 00:00 2024-08-31T05:52:45.316 INFO:teuthology.orchestra.run.smithi155.stdout:(131/150): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-08-31T05:52:45.349 INFO:teuthology.orchestra.run.smithi155.stdout:(132/150): python3-kubernetes-26.1.0-2.el9.noar 6.4 MB/s | 1.0 MB 00:00 2024-08-31T05:52:45.374 INFO:teuthology.orchestra.run.smithi155.stdout:(133/150): python3-portend-3.1.0-2.el9.noarch.r 663 kB/s | 16 kB 00:00 2024-08-31T05:52:45.408 INFO:teuthology.orchestra.run.smithi155.stdout:(134/150): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-08-31T05:52:45.422 INFO:teuthology.orchestra.run.smithi184.stdout:(107/150): libarrow-9.0.0-11.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-08-31T05:52:45.434 INFO:teuthology.orchestra.run.smithi155.stdout:(135/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-08-31T05:52:45.447 INFO:teuthology.orchestra.run.smithi184.stdout:(108/150): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-08-31T05:52:45.459 INFO:teuthology.orchestra.run.smithi155.stdout:(136/150): python3-pyperclip-1.8.2-2.el9.noarch 564 kB/s | 28 kB 00:00 2024-08-31T05:52:45.472 INFO:teuthology.orchestra.run.smithi184.stdout:(109/150): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-08-31T05:52:45.492 INFO:teuthology.orchestra.run.smithi155.stdout:(137/150): python3-repoze-lru-0.7-16.el9.noarch 929 kB/s | 31 kB 00:00 2024-08-31T05:52:45.498 INFO:teuthology.orchestra.run.smithi184.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-08-31T05:52:45.518 INFO:teuthology.orchestra.run.smithi155.stdout:(138/150): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-08-31T05:52:45.531 INFO:teuthology.orchestra.run.smithi184.stdout:(111/150): protobuf-compiler-3.14.0-13.el9.x86_ 2.1 MB/s | 863 kB 00:00 2024-08-31T05:52:45.543 INFO:teuthology.orchestra.run.smithi155.stdout:(139/150): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-08-31T05:52:45.565 INFO:teuthology.orchestra.run.smithi184.stdout:(112/150): luarocks-3.9.2-5.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-08-31T05:52:45.568 INFO:teuthology.orchestra.run.smithi155.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-08-31T05:52:45.594 INFO:teuthology.orchestra.run.smithi155.stdout:(141/150): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-08-31T05:52:45.607 INFO:teuthology.orchestra.run.smithi184.stdout:(113/150): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-08-31T05:52:45.627 INFO:teuthology.orchestra.run.smithi155.stdout:(142/150): python3-websocket-client-1.2.3-2.el9 2.6 MB/s | 90 kB 00:00 2024-08-31T05:52:45.632 INFO:teuthology.orchestra.run.smithi184.stdout:(114/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-08-31T05:52:45.657 INFO:teuthology.orchestra.run.smithi184.stdout:(115/150): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-08-31T05:52:45.661 INFO:teuthology.orchestra.run.smithi155.stdout:(143/150): python3-webob-1.8.8-2.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-08-31T05:52:45.678 INFO:teuthology.orchestra.run.smithi184.stdout:(116/150): python3-certifi-2023.05.07-4.el9.noa 705 kB/s | 14 kB 00:00 2024-08-31T05:52:45.678 INFO:teuthology.orchestra.run.smithi155.stdout:(144/150): python3-zc-lockfile-2.0-10.el9.noarc 1.2 MB/s | 20 kB 00:00 2024-08-31T05:52:45.711 INFO:teuthology.orchestra.run.smithi184.stdout:(117/150): python3-cheroot-8.6.0-4.el9.noarch.r 5.0 MB/s | 172 kB 00:00 2024-08-31T05:52:45.712 INFO:teuthology.orchestra.run.smithi155.stdout:(145/150): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-08-31T05:52:45.737 INFO:teuthology.orchestra.run.smithi155.stdout:(146/150): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-08-31T05:52:45.745 INFO:teuthology.orchestra.run.smithi184.stdout:(118/150): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.8 MB/s | 839 kB 00:00 2024-08-31T05:52:45.770 INFO:teuthology.orchestra.run.smithi184.stdout:(119/150): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-08-31T05:52:45.795 INFO:teuthology.orchestra.run.smithi155.stdout:(147/150): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-08-31T05:52:45.796 INFO:teuthology.orchestra.run.smithi184.stdout:(120/150): python3-google-auth-2.34.0-1.el9.noa 8.5 MB/s | 220 kB 00:00 2024-08-31T05:52:45.825 INFO:teuthology.orchestra.run.smithi184.stdout:(121/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-08-31T05:52:45.855 INFO:teuthology.orchestra.run.smithi184.stdout:(122/150): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-08-31T05:52:45.893 INFO:teuthology.orchestra.run.smithi184.stdout:(123/150): python3-jaraco-8.2.1-3.el9.noarch.rp 281 kB/s | 11 kB 00:00 2024-08-31T05:52:45.918 INFO:teuthology.orchestra.run.smithi184.stdout:(124/150): python3-jaraco-classes-3.2.1-5.el9.n 710 kB/s | 18 kB 00:00 2024-08-31T05:52:45.944 INFO:teuthology.orchestra.run.smithi184.stdout:(125/150): python3-jaraco-collections-3.0.0-8.e 923 kB/s | 23 kB 00:00 2024-08-31T05:52:45.969 INFO:teuthology.orchestra.run.smithi184.stdout:(126/150): python3-jaraco-functools-3.5.0-2.el9 780 kB/s | 19 kB 00:00 2024-08-31T05:52:45.994 INFO:teuthology.orchestra.run.smithi184.stdout:(127/150): python3-jaraco-text-3.2.0-6.el9.noar 788 kB/s | 20 kB 00:00 2024-08-31T05:52:46.045 INFO:teuthology.orchestra.run.smithi184.stdout:(128/150): python3-grpcio-1.46.7-10.el9.x86_64. 8.2 MB/s | 2.0 MB 00:00 2024-08-31T05:52:46.062 INFO:teuthology.orchestra.run.smithi184.stdout:(129/150): python3-logutils-0.3.5-21.el9.noarch 2.7 MB/s | 46 kB 00:00 2024-08-31T05:52:46.087 INFO:teuthology.orchestra.run.smithi184.stdout:(130/150): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-08-31T05:52:46.112 INFO:teuthology.orchestra.run.smithi184.stdout:(131/150): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-08-31T05:52:46.146 INFO:teuthology.orchestra.run.smithi184.stdout:(132/150): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-08-31T05:52:46.163 INFO:teuthology.orchestra.run.smithi184.stdout:(133/150): python3-portend-3.1.0-2.el9.noarch.r 995 kB/s | 16 kB 00:00 2024-08-31T05:52:46.196 INFO:teuthology.orchestra.run.smithi184.stdout:(134/150): python3-kubernetes-26.1.0-2.el9.noar 5.1 MB/s | 1.0 MB 00:00 2024-08-31T05:52:46.222 INFO:teuthology.orchestra.run.smithi184.stdout:(135/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-08-31T05:52:46.247 INFO:teuthology.orchestra.run.smithi184.stdout:(136/150): python3-pyperclip-1.8.2-2.el9.noarch 561 kB/s | 28 kB 00:00 2024-08-31T05:52:46.255 INFO:teuthology.orchestra.run.smithi155.stdout:(148/150): librbd1-19.3.0-4560.gd19e6118.el9.x8 6.9 MB/s | 3.1 MB 00:00 2024-08-31T05:52:46.272 INFO:teuthology.orchestra.run.smithi184.stdout:(137/150): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-08-31T05:52:46.298 INFO:teuthology.orchestra.run.smithi184.stdout:(138/150): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-08-31T05:52:46.323 INFO:teuthology.orchestra.run.smithi184.stdout:(139/150): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-08-31T05:52:46.346 INFO:teuthology.orchestra.run.smithi155.stdout:(149/150): librados2-19.3.0-4560.gd19e6118.el9. 5.6 MB/s | 3.4 MB 00:00 2024-08-31T05:52:46.429 INFO:teuthology.orchestra.run.smithi184.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 274 kB/s | 36 kB 00:00 2024-08-31T05:52:46.557 INFO:teuthology.orchestra.run.smithi184.stdout:(141/150): python3-typing-extensions-4.12.2-2.e 319 kB/s | 74 kB 00:00 2024-08-31T05:52:46.582 INFO:teuthology.orchestra.run.smithi184.stdout:(142/150): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-08-31T05:52:46.607 INFO:teuthology.orchestra.run.smithi184.stdout:(143/150): python3-webob-1.8.8-2.el9.noarch.rpm 1.3 MB/s | 230 kB 00:00 2024-08-31T05:52:46.633 INFO:teuthology.orchestra.run.smithi184.stdout:(144/150): python3-zc-lockfile-2.0-10.el9.noarc 802 kB/s | 20 kB 00:00 2024-08-31T05:52:46.658 INFO:teuthology.orchestra.run.smithi184.stdout:(145/150): re2-20211101-20.el9.x86_64.rpm 7.4 MB/s | 191 kB 00:00 2024-08-31T05:52:46.683 INFO:teuthology.orchestra.run.smithi184.stdout:(146/150): python3-werkzeug-2.0.3-3.el9.1.noarc 4.1 MB/s | 427 kB 00:00 2024-08-31T05:52:46.734 INFO:teuthology.orchestra.run.smithi184.stdout:(147/150): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-08-31T05:52:47.017 INFO:teuthology.orchestra.run.smithi184.stdout:(148/150): librados2-19.3.0-4560.gd19e6118.el9. 10 MB/s | 3.4 MB 00:00 2024-08-31T05:52:47.243 INFO:teuthology.orchestra.run.smithi184.stdout:(149/150): librbd1-19.3.0-4560.gd19e6118.el9.x8 6.2 MB/s | 3.1 MB 00:00 2024-08-31T05:52:47.601 INFO:teuthology.orchestra.run.smithi184.stdout:(150/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 5.2 MB/s | 16 MB 00:03 2024-08-31T05:52:47.606 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-08-31T05:52:47.606 INFO:teuthology.orchestra.run.smithi184.stdout:Total 10 MB/s | 224 MB 00:22 2024-08-31T05:52:47.655 INFO:teuthology.orchestra.run.smithi155.stdout:(150/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 4.0 MB/s | 16 MB 00:04 2024-08-31T05:52:47.659 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-08-31T05:52:47.659 INFO:teuthology.orchestra.run.smithi155.stdout:Total 9.6 MB/s | 224 MB 00:23 2024-08-31T05:52:48.663 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-08-31T05:52:48.717 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-08-31T05:52:48.791 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-08-31T05:52:48.791 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-08-31T05:52:48.844 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-08-31T05:52:48.844 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-08-31T05:52:50.499 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-08-31T05:52:50.549 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-08-31T05:52:50.550 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-08-31T05:52:50.584 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-08-31T05:52:50.585 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-08-31T05:52:50.585 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-08-31T05:52:50.614 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-08-31T05:52:50.698 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-08-31T05:52:51.131 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-08-31T05:52:51.257 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librados2-2:19.3.0-4560.gd19e6118.el9.x86_64 5/152 2024-08-31T05:52:51.460 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:19.3.0-4560.gd19e6118.el9.x86_64 5/152 2024-08-31T05:52:51.491 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs2-2:19.3.0-4560.gd19e6118.el9.x86_64 6/152 2024-08-31T05:52:51.606 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephfs2-2:19.3.0-4560.gd19e6118.el9.x86_64 6/152 2024-08-31T05:52:51.673 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rados-2:19.3.0-4560.gd19e6118.el9.x86_64 7/152 2024-08-31T05:52:51.705 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-08-31T05:52:51.740 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-08-31T05:52:51.990 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 10/152 2024-08-31T05:52:52.063 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-13.16-1.el9.x86_64 11/152 2024-08-31T05:52:52.096 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephsqlite-2:19.3.0-4560.gd19e6118.el9.x86_64 12/152 2024-08-31T05:52:52.261 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4560.gd19e6118.el9.x86_64 12/152 2024-08-31T05:52:52.299 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libradosstriper1-2:19.3.0-4560.gd19e6118.el9.x86 13/152 2024-08-31T05:52:52.411 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4560.gd19e6118.el9.x86 13/152 2024-08-31T05:52:52.442 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-31T05:52:52.485 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-31T05:52:52.858 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-31T05:52:52.902 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-31T05:52:53.042 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : re2-1:20211101-20.el9.x86_64 16/152 2024-08-31T05:52:53.074 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 17/152 2024-08-31T05:52:53.120 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/152 2024-08-31T05:52:53.207 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/152 2024-08-31T05:52:53.274 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 20/152 2024-08-31T05:52:53.320 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-packaging-20.9-5.el9.noarch 21/152 2024-08-31T05:52:53.375 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-08-31T05:52:53.438 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-08-31T05:52:53.471 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-08-31T05:52:53.574 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-08-31T05:52:53.643 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 26/152 2024-08-31T05:52:53.683 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-08-31T05:52:53.739 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 28/152 2024-08-31T05:52:53.833 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 29/152 2024-08-31T05:52:53.871 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-common-2:19.3.0-4560.gd19e6118.el9. 30/152 2024-08-31T05:52:53.921 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-argparse-2:19.3.0-4560.gd19e6118.el 31/152 2024-08-31T05:52:53.951 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cephfs-2:19.3.0-4560.gd19e6118.el9.x86_6 32/152 2024-08-31T05:52:54.048 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-08-31T05:52:54.104 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : luarocks-3.9.2-5.el9.noarch 34/152 2024-08-31T05:52:54.199 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-08-31T05:52:54.386 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-08-31T05:52:54.450 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-08-31T05:52:55.137 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-08-31T05:52:55.254 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-08-31T05:52:55.301 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 40/152 2024-08-31T05:52:55.346 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-08-31T05:52:55.384 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librados-devel-2:19.3.0-4560.gd19e6118.el9.x86_6 42/152 2024-08-31T05:52:55.425 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-08-31T05:52:55.549 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-08-31T05:52:55.623 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 45/152 2024-08-31T05:52:55.709 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 46/152 2024-08-31T05:52:55.752 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 47/152 2024-08-31T05:52:55.789 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 48/152 2024-08-31T05:52:55.849 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 49/152 2024-08-31T05:52:55.889 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 50/152 2024-08-31T05:52:55.943 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 51/152 2024-08-31T05:52:55.993 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 52/152 2024-08-31T05:52:56.177 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 53/152 2024-08-31T05:52:56.401 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 54/152 2024-08-31T05:52:56.445 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 55/152 2024-08-31T05:52:56.578 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-08-31T05:52:56.619 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 57/152 2024-08-31T05:52:56.663 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-08-31T05:52:56.700 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-08-31T05:52:56.708 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-08-31T05:52:56.732 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 60/152 2024-08-31T05:52:56.778 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-08-31T05:52:56.798 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-08-31T05:52:56.803 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-08-31T05:52:56.845 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-08-31T05:52:56.852 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-08-31T05:52:56.873 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-08-31T05:52:56.898 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-08-31T05:52:56.941 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-08-31T05:52:57.019 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-08-31T05:52:57.066 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-08-31T05:52:57.105 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-08-31T05:52:57.168 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-08-31T05:52:57.185 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-08-31T05:52:57.220 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-08-31T05:52:57.263 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 67/152 2024-08-31T05:52:57.283 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-08-31T05:52:57.302 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-08-31T05:52:57.329 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : librados2-2:19.3.0-4560.gd19e6118.el9.x86_64 5/152 2024-08-31T05:52:57.338 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-08-31T05:52:57.373 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-08-31T05:52:57.408 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : librados2-2:19.3.0-4560.gd19e6118.el9.x86_64 5/152 2024-08-31T05:52:57.415 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-08-31T05:52:57.454 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librados2-2:19.3.0-4560.gd19e6118.el9.x86_64 5/152 2024-08-31T05:52:57.460 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-08-31T05:52:57.480 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephfs2-2:19.3.0-4560.gd19e6118.el9.x86_64 6/152 2024-08-31T05:52:57.492 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-08-31T05:52:57.517 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librados2-2:19.3.0-4560.gd19e6118.el9.x86_64 5/152 2024-08-31T05:52:57.542 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephfs2-2:19.3.0-4560.gd19e6118.el9.x86_64 6/152 2024-08-31T05:52:57.554 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libcephfs2-2:19.3.0-4560.gd19e6118.el9.x86_64 6/152 2024-08-31T05:52:57.596 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rados-2:19.3.0-4560.gd19e6118.el9.x86_64 7/152 2024-08-31T05:52:57.613 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-08-31T05:52:57.634 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libcephfs2-2:19.3.0-4560.gd19e6118.el9.x86_64 6/152 2024-08-31T05:52:57.643 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-08-31T05:52:57.669 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-08-31T05:52:57.675 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rados-2:19.3.0-4560.gd19e6118.el9.x86_64 7/152 2024-08-31T05:52:57.676 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-08-31T05:52:57.711 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-08-31T05:52:57.713 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-08-31T05:52:57.739 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-08-31T05:52:57.818 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 10/152 2024-08-31T05:52:57.841 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-08-31T05:52:57.874 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : postgresql-13.16-1.el9.x86_64 11/152 2024-08-31T05:52:57.890 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 10/152 2024-08-31T05:52:57.901 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 78/152 2024-08-31T05:52:57.902 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephsqlite-2:19.3.0-4560.gd19e6118.el9.x86_64 12/152 2024-08-31T05:52:57.936 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : postgresql-13.16-1.el9.x86_64 11/152 2024-08-31T05:52:57.964 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephsqlite-2:19.3.0-4560.gd19e6118.el9.x86_64 12/152 2024-08-31T05:52:58.007 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4560.gd19e6118.el9.x86_64 12/152 2024-08-31T05:52:58.036 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libradosstriper1-2:19.3.0-4560.gd19e6118.el9.x86 13/152 2024-08-31T05:52:58.069 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4560.gd19e6118.el9.x86_64 12/152 2024-08-31T05:52:58.098 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libradosstriper1-2:19.3.0-4560.gd19e6118.el9.x86 13/152 2024-08-31T05:52:58.099 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4560.gd19e6118.el9.x86 13/152 2024-08-31T05:52:58.120 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-31T05:52:58.145 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-31T05:52:58.161 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4560.gd19e6118.el9.x86 13/152 2024-08-31T05:52:58.182 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-31T05:52:58.206 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-31T05:52:58.448 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-31T05:52:58.478 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-31T05:52:58.507 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-31T05:52:58.542 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-31T05:52:58.586 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : re2-1:20211101-20.el9.x86_64 16/152 2024-08-31T05:52:58.626 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 17/152 2024-08-31T05:52:58.651 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : re2-1:20211101-20.el9.x86_64 16/152 2024-08-31T05:52:58.661 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/152 2024-08-31T05:52:58.683 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 17/152 2024-08-31T05:52:58.718 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/152 2024-08-31T05:52:58.736 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/152 2024-08-31T05:52:58.788 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 20/152 2024-08-31T05:52:58.793 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/152 2024-08-31T05:52:58.836 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-packaging-20.9-5.el9.noarch 21/152 2024-08-31T05:52:58.845 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 20/152 2024-08-31T05:52:58.867 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 79/152 2024-08-31T05:52:58.880 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-08-31T05:52:58.884 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-packaging-20.9-5.el9.noarch 21/152 2024-08-31T05:52:58.912 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-08-31T05:52:58.919 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 80/152 2024-08-31T05:52:58.935 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-08-31T05:52:58.940 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-08-31T05:52:58.952 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 81/152 2024-08-31T05:52:58.977 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-08-31T05:52:59.005 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-08-31T05:52:59.010 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-08-31T05:52:59.071 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 26/152 2024-08-31T05:52:59.077 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-08-31T05:52:59.100 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 82/152 2024-08-31T05:52:59.101 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-08-31T05:52:59.137 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 26/152 2024-08-31T05:52:59.145 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 28/152 2024-08-31T05:52:59.162 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 83/152 2024-08-31T05:52:59.166 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-08-31T05:52:59.210 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 28/152 2024-08-31T05:52:59.216 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 84/152 2024-08-31T05:52:59.222 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 29/152 2024-08-31T05:52:59.256 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ceph-common-2:19.3.0-4560.gd19e6118.el9. 30/152 2024-08-31T05:52:59.290 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 29/152 2024-08-31T05:52:59.299 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ceph-argparse-2:19.3.0-4560.gd19e6118.el 31/152 2024-08-31T05:52:59.321 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-ceph-common-2:19.3.0-4560.gd19e6118.el9. 30/152 2024-08-31T05:52:59.325 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-08-31T05:52:59.328 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cephfs-2:19.3.0-4560.gd19e6118.el9.x86_6 32/152 2024-08-31T05:52:59.363 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-08-31T05:52:59.365 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-ceph-argparse-2:19.3.0-4560.gd19e6118.el 31/152 2024-08-31T05:52:59.400 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-08-31T05:52:59.401 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cephfs-2:19.3.0-4560.gd19e6118.el9.x86_6 32/152 2024-08-31T05:52:59.451 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : luarocks-3.9.2-5.el9.noarch 34/152 2024-08-31T05:52:59.491 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-08-31T05:52:59.541 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : luarocks-3.9.2-5.el9.noarch 34/152 2024-08-31T05:52:59.542 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-08-31T05:52:59.609 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-0.3.26-2.el9.x86_64 87/152 2024-08-31T05:52:59.618 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-08-31T05:52:59.643 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 88/152 2024-08-31T05:52:59.677 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-08-31T05:52:59.728 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-08-31T05:52:59.749 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-08-31T05:52:59.756 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-08-31T05:52:59.812 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-08-31T05:53:00.263 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-08-31T05:53:00.349 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-08-31T05:53:00.357 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-08-31T05:53:00.385 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 40/152 2024-08-31T05:53:00.428 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-08-31T05:53:00.441 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-08-31T05:53:00.468 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 40/152 2024-08-31T05:53:00.474 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librados-devel-2:19.3.0-4560.gd19e6118.el9.x86_6 42/152 2024-08-31T05:53:00.507 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-08-31T05:53:00.511 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-08-31T05:53:00.549 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librados-devel-2:19.3.0-4560.gd19e6118.el9.x86_6 42/152 2024-08-31T05:53:00.582 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-08-31T05:53:00.600 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-08-31T05:53:00.624 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-08-31T05:53:00.663 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 45/152 2024-08-31T05:53:00.677 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-08-31T05:53:00.737 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 46/152 2024-08-31T05:53:00.739 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 45/152 2024-08-31T05:53:00.765 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-08-31T05:53:00.776 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 47/152 2024-08-31T05:53:00.812 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 48/152 2024-08-31T05:53:00.813 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 46/152 2024-08-31T05:53:00.858 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 49/152 2024-08-31T05:53:00.859 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 47/152 2024-08-31T05:53:00.895 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 50/152 2024-08-31T05:53:00.895 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 48/152 2024-08-31T05:53:00.937 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 51/152 2024-08-31T05:53:00.943 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 49/152 2024-08-31T05:53:00.978 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 50/152 2024-08-31T05:53:00.980 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 52/152 2024-08-31T05:53:01.022 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 51/152 2024-08-31T05:53:01.064 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 52/152 2024-08-31T05:53:01.119 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 53/152 2024-08-31T05:53:01.157 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 54/152 2024-08-31T05:53:01.198 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 55/152 2024-08-31T05:53:01.204 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 53/152 2024-08-31T05:53:01.241 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 54/152 2024-08-31T05:53:01.290 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 55/152 2024-08-31T05:53:01.302 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-08-31T05:53:01.331 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 57/152 2024-08-31T05:53:01.372 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-08-31T05:53:01.396 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-08-31T05:53:01.397 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-08-31T05:53:01.421 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 60/152 2024-08-31T05:53:01.440 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 57/152 2024-08-31T05:53:01.480 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-08-31T05:53:01.495 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-08-31T05:53:01.506 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-08-31T05:53:01.531 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 60/152 2024-08-31T05:53:01.606 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-08-31T05:53:01.623 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-08-31T05:53:01.654 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-08-31T05:53:01.673 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-08-31T05:53:01.707 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-08-31T05:53:01.734 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-08-31T05:53:01.751 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-08-31T05:53:01.772 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-08-31T05:53:01.789 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 67/152 2024-08-31T05:53:01.799 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-08-31T05:53:01.821 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-08-31T05:53:01.842 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-08-31T05:53:01.850 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-08-31T05:53:01.885 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-08-31T05:53:01.895 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-08-31T05:53:01.918 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-08-31T05:53:01.932 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 67/152 2024-08-31T05:53:01.959 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-08-31T05:53:01.973 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-08-31T05:53:01.994 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-08-31T05:53:02.002 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-08-31T05:53:02.037 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-08-31T05:53:02.070 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-08-31T05:53:02.081 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-08-31T05:53:02.112 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-08-31T05:53:02.124 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-08-31T05:53:02.146 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-08-31T05:53:02.156 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-08-31T05:53:02.236 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-08-31T05:53:02.259 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-08-31T05:53:02.293 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-08-31T05:53:02.333 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-08-31T05:53:02.437 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-08-31T05:53:02.439 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-08-31T05:53:02.493 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 78/152 2024-08-31T05:53:02.507 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 78/152 2024-08-31T05:53:02.532 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-31T05:53:02.596 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-08-31T05:53:02.626 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 95/152 2024-08-31T05:53:03.002 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-08-31T05:53:03.054 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librbd1-2:19.3.0-4560.gd19e6118.el9.x86_64 97/152 2024-08-31T05:53:03.188 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:19.3.0-4560.gd19e6118.el9.x86_64 97/152 2024-08-31T05:53:03.222 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rbd-2:19.3.0-4560.gd19e6118.el9.x86_64 98/152 2024-08-31T05:53:03.240 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 79/152 2024-08-31T05:53:03.240 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 79/152 2024-08-31T05:53:03.271 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-08-31T05:53:03.286 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 80/152 2024-08-31T05:53:03.290 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 80/152 2024-08-31T05:53:03.320 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 81/152 2024-08-31T05:53:03.323 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 81/152 2024-08-31T05:53:03.421 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 82/152 2024-08-31T05:53:03.425 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 82/152 2024-08-31T05:53:03.484 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 83/152 2024-08-31T05:53:03.508 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 83/152 2024-08-31T05:53:03.537 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 84/152 2024-08-31T05:53:03.553 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 84/152 2024-08-31T05:53:03.634 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-08-31T05:53:03.650 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-08-31T05:53:03.673 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-08-31T05:53:03.688 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-08-31T05:53:03.846 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 100/152 2024-08-31T05:53:03.856 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openblas-0.3.26-2.el9.x86_64 87/152 2024-08-31T05:53:03.872 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : openblas-0.3.26-2.el9.x86_64 87/152 2024-08-31T05:53:03.877 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librgw2-2:19.3.0-4560.gd19e6118.el9.x86_64 101/152 2024-08-31T05:53:03.895 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 88/152 2024-08-31T05:53:03.911 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 88/152 2024-08-31T05:53:03.969 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librgw2-2:19.3.0-4560.gd19e6118.el9.x86_64 101/152 2024-08-31T05:53:03.972 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-08-31T05:53:03.989 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-08-31T05:53:03.995 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rgw-2:19.3.0-4560.gd19e6118.el9.x86_64 102/152 2024-08-31T05:53:04.626 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-08-31T05:53:04.648 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-08-31T05:53:04.737 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-08-31T05:53:04.763 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-08-31T05:53:05.970 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-08-31T05:53:06.018 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-08-31T05:53:06.066 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-31T05:53:06.085 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-08-31T05:53:06.110 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 95/152 2024-08-31T05:53:06.116 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-31T05:53:06.144 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-08-31T05:53:06.177 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 95/152 2024-08-31T05:53:06.438 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-08-31T05:53:06.481 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : librbd1-2:19.3.0-4560.gd19e6118.el9.x86_64 97/152 2024-08-31T05:53:06.504 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-08-31T05:53:06.548 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : librbd1-2:19.3.0-4560.gd19e6118.el9.x86_64 97/152 2024-08-31T05:53:06.572 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librbd1-2:19.3.0-4560.gd19e6118.el9.x86_64 97/152 2024-08-31T05:53:06.603 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rbd-2:19.3.0-4560.gd19e6118.el9.x86_64 98/152 2024-08-31T05:53:06.638 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-08-31T05:53:06.639 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librbd1-2:19.3.0-4560.gd19e6118.el9.x86_64 97/152 2024-08-31T05:53:06.670 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rbd-2:19.3.0-4560.gd19e6118.el9.x86_64 98/152 2024-08-31T05:53:06.699 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 103/152 2024-08-31T05:53:06.705 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-08-31T05:53:06.742 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 103/152 2024-08-31T05:53:07.134 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 100/152 2024-08-31T05:53:07.167 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librgw2-2:19.3.0-4560.gd19e6118.el9.x86_64 101/152 2024-08-31T05:53:07.213 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 100/152 2024-08-31T05:53:07.239 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librgw2-2:19.3.0-4560.gd19e6118.el9.x86_64 101/152 2024-08-31T05:53:07.243 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librgw2-2:19.3.0-4560.gd19e6118.el9.x86_64 101/152 2024-08-31T05:53:07.269 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rgw-2:19.3.0-4560.gd19e6118.el9.x86_64 102/152 2024-08-31T05:53:07.314 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librgw2-2:19.3.0-4560.gd19e6118.el9.x86_64 101/152 2024-08-31T05:53:07.345 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rgw-2:19.3.0-4560.gd19e6118.el9.x86_64 102/152 2024-08-31T05:53:07.431 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 103/152 2024-08-31T05:53:07.471 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-base-2:19.3.0-4560.gd19e6118.el9.x86_64 104/152 2024-08-31T05:53:07.595 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-base-2:19.3.0-4560.gd19e6118.el9.x86_64 104/152 2024-08-31T05:53:07.596 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-08-31T05:53:07.596 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-08-31T05:53:07.596 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T05:53:07.655 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-selinux-2:19.3.0-4560.gd19e6118.el9.x86_64 105/152 2024-08-31T05:53:09.594 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 103/152 2024-08-31T05:53:09.617 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 103/152 2024-08-31T05:53:09.631 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 103/152 2024-08-31T05:53:09.650 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 103/152 2024-08-31T05:53:10.209 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 103/152 2024-08-31T05:53:10.227 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 103/152 2024-08-31T05:53:10.243 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-base-2:19.3.0-4560.gd19e6118.el9.x86_64 104/152 2024-08-31T05:53:10.270 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-base-2:19.3.0-4560.gd19e6118.el9.x86_64 104/152 2024-08-31T05:53:10.335 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-base-2:19.3.0-4560.gd19e6118.el9.x86_64 104/152 2024-08-31T05:53:10.335 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-08-31T05:53:10.336 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-08-31T05:53:10.336 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-08-31T05:53:10.366 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-base-2:19.3.0-4560.gd19e6118.el9.x86_64 104/152 2024-08-31T05:53:10.366 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-08-31T05:53:10.366 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-08-31T05:53:10.366 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-08-31T05:53:10.369 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-selinux-2:19.3.0-4560.gd19e6118.el9.x86_64 105/152 2024-08-31T05:53:10.396 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-selinux-2:19.3.0-4560.gd19e6118.el9.x86_64 105/152 2024-08-31T05:53:20.958 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4560.gd19e6118.el9.x86_64 105/152 2024-08-31T05:53:20.958 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /sys 2024-08-31T05:53:20.958 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /proc 2024-08-31T05:53:20.958 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /mnt 2024-08-31T05:53:20.958 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /var/tmp 2024-08-31T05:53:20.958 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /home 2024-08-31T05:53:20.958 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /root 2024-08-31T05:53:20.958 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /tmp 2024-08-31T05:53:20.958 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T05:53:21.204 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-volume-2:19.3.0-4560.gd19e6118.el9.noarch 106/152 2024-08-31T05:53:21.247 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-volume-2:19.3.0-4560.gd19e6118.el9.noarch 106/152 2024-08-31T05:53:21.247 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T05:53:21.247 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-08-31T05:53:21.247 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T05:53:22.178 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4560.gd19e6118.el9.x86_64 105/152 2024-08-31T05:53:22.179 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /sys 2024-08-31T05:53:22.179 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /proc 2024-08-31T05:53:22.179 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /mnt 2024-08-31T05:53:22.179 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /var/tmp 2024-08-31T05:53:22.179 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /home 2024-08-31T05:53:22.179 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /root 2024-08-31T05:53:22.179 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /tmp 2024-08-31T05:53:22.179 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-08-31T05:53:22.371 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-volume-2:19.3.0-4560.gd19e6118.el9.noarch 106/152 2024-08-31T05:53:22.387 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-volume-2:19.3.0-4560.gd19e6118.el9.noarch 106/152 2024-08-31T05:53:22.388 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T05:53:22.388 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-08-31T05:53:22.388 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-08-31T05:53:22.537 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4560.gd19e6118.el9.x86_64 105/152 2024-08-31T05:53:22.537 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /sys 2024-08-31T05:53:22.537 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /proc 2024-08-31T05:53:22.537 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /mnt 2024-08-31T05:53:22.537 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /var/tmp 2024-08-31T05:53:22.537 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /home 2024-08-31T05:53:22.537 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /root 2024-08-31T05:53:22.538 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /tmp 2024-08-31T05:53:22.538 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-08-31T05:53:22.729 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-volume-2:19.3.0-4560.gd19e6118.el9.noarch 106/152 2024-08-31T05:53:22.745 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-volume-2:19.3.0-4560.gd19e6118.el9.noarch 106/152 2024-08-31T05:53:22.745 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T05:53:22.745 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-08-31T05:53:22.745 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-08-31T05:53:23.039 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-osd-2:19.3.0-4560.gd19e6118.el9.x86_64 107/152 2024-08-31T05:53:23.098 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-osd-2:19.3.0-4560.gd19e6118.el9.x86_64 107/152 2024-08-31T05:53:23.098 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T05:53:23.098 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-08-31T05:53:23.098 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-31T05:53:23.098 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-31T05:53:23.098 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T05:53:23.356 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mds-2:19.3.0-4560.gd19e6118.el9.x86_64 108/152 2024-08-31T05:53:23.408 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mds-2:19.3.0-4560.gd19e6118.el9.x86_64 108/152 2024-08-31T05:53:23.409 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T05:53:23.409 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-08-31T05:53:23.409 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-31T05:53:23.409 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-31T05:53:23.409 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T05:53:23.941 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mon-2:19.3.0-4560.gd19e6118.el9.x86_64 109/152 2024-08-31T05:53:23.983 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-osd-2:19.3.0-4560.gd19e6118.el9.x86_64 107/152 2024-08-31T05:53:23.993 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mon-2:19.3.0-4560.gd19e6118.el9.x86_64 109/152 2024-08-31T05:53:23.994 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T05:53:23.994 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-08-31T05:53:23.994 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-31T05:53:23.994 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-31T05:53:23.994 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T05:53:24.009 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-osd-2:19.3.0-4560.gd19e6118.el9.x86_64 107/152 2024-08-31T05:53:24.009 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T05:53:24.009 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-08-31T05:53:24.009 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-31T05:53:24.009 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-31T05:53:24.009 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-08-31T05:53:24.125 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-08-31T05:53:24.187 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-08-31T05:53:24.234 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mds-2:19.3.0-4560.gd19e6118.el9.x86_64 108/152 2024-08-31T05:53:24.257 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mds-2:19.3.0-4560.gd19e6118.el9.x86_64 108/152 2024-08-31T05:53:24.257 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T05:53:24.257 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-08-31T05:53:24.257 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-31T05:53:24.257 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-31T05:53:24.257 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-08-31T05:53:24.259 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-08-31T05:53:24.311 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-osd-2:19.3.0-4560.gd19e6118.el9.x86_64 107/152 2024-08-31T05:53:24.338 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-osd-2:19.3.0-4560.gd19e6118.el9.x86_64 107/152 2024-08-31T05:53:24.338 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T05:53:24.338 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-08-31T05:53:24.338 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-31T05:53:24.338 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-31T05:53:24.338 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-08-31T05:53:24.505 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-08-31T05:53:24.566 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-08-31T05:53:24.570 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mds-2:19.3.0-4560.gd19e6118.el9.x86_64 108/152 2024-08-31T05:53:24.593 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mds-2:19.3.0-4560.gd19e6118.el9.x86_64 108/152 2024-08-31T05:53:24.593 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T05:53:24.593 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-08-31T05:53:24.593 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-31T05:53:24.593 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-31T05:53:24.593 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-08-31T05:53:24.654 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-08-31T05:53:24.718 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mon-2:19.3.0-4560.gd19e6118.el9.x86_64 109/152 2024-08-31T05:53:24.741 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mon-2:19.3.0-4560.gd19e6118.el9.x86_64 109/152 2024-08-31T05:53:24.741 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T05:53:24.741 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-08-31T05:53:24.741 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-31T05:53:24.741 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-31T05:53:24.742 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-08-31T05:53:24.767 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-08-31T05:53:24.843 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-08-31T05:53:24.900 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-08-31T05:53:24.943 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-08-31T05:53:24.955 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-08-31T05:53:24.993 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-08-31T05:53:25.038 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-08-31T05:53:25.054 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mon-2:19.3.0-4560.gd19e6118.el9.x86_64 109/152 2024-08-31T05:53:25.063 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : fuse-2.9.9-16.el9.x86_64 120/152 2024-08-31T05:53:25.077 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mon-2:19.3.0-4560.gd19e6118.el9.x86_64 109/152 2024-08-31T05:53:25.077 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T05:53:25.077 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-08-31T05:53:25.077 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-31T05:53:25.077 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-31T05:53:25.077 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-08-31T05:53:25.142 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-08-31T05:53:25.184 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-08-31T05:53:25.189 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-08-31T05:53:25.241 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-08-31T05:53:25.266 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-08-31T05:53:25.296 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-08-31T05:53:25.357 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-08-31T05:53:25.479 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-08-31T05:53:25.494 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-08-31T05:53:25.529 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-08-31T05:53:25.548 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-08-31T05:53:25.589 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-08-31T05:53:25.590 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 121/152 2024-08-31T05:53:25.598 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-08-31T05:53:25.616 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : fuse-2.9.9-16.el9.x86_64 120/152 2024-08-31T05:53:25.622 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 121/152 2024-08-31T05:53:25.664 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4560.gd19e6118.e 122/152 2024-08-31T05:53:25.686 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-08-31T05:53:25.747 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4560.gd19e6118. 123/152 2024-08-31T05:53:25.786 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4560.gd19e6118.el9.noa 124/152 2024-08-31T05:53:25.824 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-08-31T05:53:25.863 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-08-31T05:53:25.903 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-08-31T05:53:25.940 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : fuse-2.9.9-16.el9.x86_64 120/152 2024-08-31T05:53:26.113 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 121/152 2024-08-31T05:53:26.140 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 121/152 2024-08-31T05:53:26.172 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4560.gd19e6118.e 122/152 2024-08-31T05:53:26.238 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4560.gd19e6118. 123/152 2024-08-31T05:53:26.263 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4560.gd19e6118.el9.noa 124/152 2024-08-31T05:53:26.378 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 121/152 2024-08-31T05:53:26.422 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 121/152 2024-08-31T05:53:26.453 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4560.gd19e6118.e 122/152 2024-08-31T05:53:26.519 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4560.gd19e6118. 123/152 2024-08-31T05:53:26.553 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4560.gd19e6118.el9.noa 124/152 2024-08-31T05:53:29.448 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4560.gd19e6118.el9.noa 124/152 2024-08-31T05:53:29.520 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4560.gd19e6118.el9.n 125/152 2024-08-31T05:53:29.658 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4560.gd19e6118.el9.noa 124/152 2024-08-31T05:53:29.700 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4560.gd19e6118.el9.noa 124/152 2024-08-31T05:53:29.742 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4560.gd19e6118.el9.n 125/152 2024-08-31T05:53:29.758 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4560.gd19e6118.el9.n 125/152 2024-08-31T05:53:30.487 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4560.gd19e6118.el9.n 125/152 2024-08-31T05:53:30.520 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4560.gd19 126/152 2024-08-31T05:53:30.635 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4560.gd19 126/152 2024-08-31T05:53:30.715 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4560.gd19e6118.el9.n 125/152 2024-08-31T05:53:30.743 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4560.gd19 126/152 2024-08-31T05:53:30.803 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4560.gd19e6118.el 127/152 2024-08-31T05:53:30.830 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-2:19.3.0-4560.gd19e6118.el9.x86_64 128/152 2024-08-31T05:53:30.854 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4560.gd19e6118.el9.x86_64 128/152 2024-08-31T05:53:30.854 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T05:53:30.854 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-08-31T05:53:30.854 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-31T05:53:30.854 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-31T05:53:30.854 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-08-31T05:53:30.855 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4560.gd19 126/152 2024-08-31T05:53:30.894 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4560.gd19e6118.el9.n 125/152 2024-08-31T05:53:30.906 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-rook-2:19.3.0-4560.gd19e6118.el9.noarch 129/152 2024-08-31T05:53:30.918 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4560.gd19e6118.el9.noarch 129/152 2024-08-31T05:53:30.942 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4560.gd19 126/152 2024-08-31T05:53:31.016 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-2:19.3.0-4560.gd19e6118.el9.x86_64 130/152 2024-08-31T05:53:31.027 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4560.gd19e6118.el 127/152 2024-08-31T05:53:31.054 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-2:19.3.0-4560.gd19e6118.el9.x86_64 128/152 2024-08-31T05:53:31.071 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 131/152 2024-08-31T05:53:31.077 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4560.gd19e6118.el9.x86_64 128/152 2024-08-31T05:53:31.077 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T05:53:31.077 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-08-31T05:53:31.077 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-31T05:53:31.077 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-31T05:53:31.078 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-08-31T05:53:31.102 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4560.gd19 126/152 2024-08-31T05:53:31.121 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-rook-2:19.3.0-4560.gd19e6118.el9.noarch 129/152 2024-08-31T05:53:31.133 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4560.gd19e6118.el9.noarch 129/152 2024-08-31T05:53:31.232 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-2:19.3.0-4560.gd19e6118.el9.x86_64 130/152 2024-08-31T05:53:31.277 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 131/152 2024-08-31T05:53:31.289 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4560.gd19e6118.el 127/152 2024-08-31T05:53:31.323 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-2:19.3.0-4560.gd19e6118.el9.x86_64 128/152 2024-08-31T05:53:31.375 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4560.gd19e6118.el9.x86_64 128/152 2024-08-31T05:53:31.376 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T05:53:31.376 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-08-31T05:53:31.376 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-31T05:53:31.376 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-31T05:53:31.376 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T05:53:31.432 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-rook-2:19.3.0-4560.gd19e6118.el9.noarch 129/152 2024-08-31T05:53:31.459 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4560.gd19e6118.el9.noarch 129/152 2024-08-31T05:53:31.573 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-2:19.3.0-4560.gd19e6118.el9.x86_64 130/152 2024-08-31T05:53:31.635 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 131/152 2024-08-31T05:53:32.031 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libtool-2.4.6-46.el9.x86_64 132/152 2024-08-31T05:53:32.068 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-radosgw-2:19.3.0-4560.gd19e6118.el9.x86_64 133/152 2024-08-31T05:53:32.091 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4560.gd19e6118.el9.x86_64 133/152 2024-08-31T05:53:32.091 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T05:53:32.091 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-08-31T05:53:32.092 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-31T05:53:32.092 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-31T05:53:32.092 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-08-31T05:53:32.179 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4560.gd19e6 134/152 2024-08-31T05:53:32.201 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4560.gd19e6 134/152 2024-08-31T05:53:32.202 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T05:53:32.202 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-08-31T05:53:32.202 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-08-31T05:53:32.234 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libtool-2.4.6-46.el9.x86_64 132/152 2024-08-31T05:53:32.252 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephfs-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 135/152 2024-08-31T05:53:32.274 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-radosgw-2:19.3.0-4560.gd19e6118.el9.x86_64 133/152 2024-08-31T05:53:32.275 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 135/152 2024-08-31T05:53:32.275 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T05:53:32.275 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-08-31T05:53:32.275 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-31T05:53:32.275 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-31T05:53:32.276 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-08-31T05:53:32.297 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4560.gd19e6118.el9.x86_64 133/152 2024-08-31T05:53:32.297 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T05:53:32.297 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-08-31T05:53:32.297 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-31T05:53:32.297 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-31T05:53:32.297 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-08-31T05:53:32.341 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4560.gd19e6 134/152 2024-08-31T05:53:32.363 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4560.gd19e6 134/152 2024-08-31T05:53:32.364 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T05:53:32.364 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-08-31T05:53:32.364 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-08-31T05:53:32.417 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephfs-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 135/152 2024-08-31T05:53:32.439 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 135/152 2024-08-31T05:53:32.439 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T05:53:32.439 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-08-31T05:53:32.440 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-31T05:53:32.440 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-31T05:53:32.440 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-08-31T05:53:32.603 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 136/152 2024-08-31T05:53:32.626 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 136/152 2024-08-31T05:53:32.627 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T05:53:32.627 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-08-31T05:53:32.627 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-31T05:53:32.627 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-31T05:53:32.627 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-08-31T05:53:32.759 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 136/152 2024-08-31T05:53:32.762 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libtool-2.4.6-46.el9.x86_64 132/152 2024-08-31T05:53:32.782 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 136/152 2024-08-31T05:53:32.782 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T05:53:32.782 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-08-31T05:53:32.782 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-31T05:53:32.782 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-31T05:53:32.782 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-08-31T05:53:32.810 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-radosgw-2:19.3.0-4560.gd19e6118.el9.x86_64 133/152 2024-08-31T05:53:32.862 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4560.gd19e6118.el9.x86_64 133/152 2024-08-31T05:53:32.862 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T05:53:32.862 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-08-31T05:53:32.862 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-31T05:53:32.862 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-31T05:53:32.863 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T05:53:32.910 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4560.gd19e6 134/152 2024-08-31T05:53:32.962 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4560.gd19e6 134/152 2024-08-31T05:53:32.962 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T05:53:32.962 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-08-31T05:53:32.962 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T05:53:33.019 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 135/152 2024-08-31T05:53:33.072 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 135/152 2024-08-31T05:53:33.072 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T05:53:33.072 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-08-31T05:53:33.072 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-31T05:53:33.072 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-31T05:53:33.072 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T05:53:33.445 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 136/152 2024-08-31T05:53:33.497 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 136/152 2024-08-31T05:53:33.497 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-31T05:53:33.497 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-08-31T05:53:33.497 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-31T05:53:33.497 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-31T05:53:33.497 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T05:53:36.967 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-test-2:19.3.0-4560.gd19e6118.el9.x86_64 137/152 2024-08-31T05:53:37.017 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 138/152 2024-08-31T05:53:37.100 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-nbd-2:19.3.0-4560.gd19e6118.el9.x86_64 139/152 2024-08-31T05:53:37.113 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-test-2:19.3.0-4560.gd19e6118.el9.x86_64 137/152 2024-08-31T05:53:37.140 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 140/152 2024-08-31T05:53:37.154 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 138/152 2024-08-31T05:53:37.175 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 141/152 2024-08-31T05:53:37.194 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-nbd-2:19.3.0-4560.gd19e6118.el9.x86_64 139/152 2024-08-31T05:53:37.227 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 140/152 2024-08-31T05:53:37.254 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 141/152 2024-08-31T05:53:37.359 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephfs-shell-2:19.3.0-4560.gd19e6118.el9.x86_64 142/152 2024-08-31T05:53:37.397 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 143/152 2024-08-31T05:53:37.424 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephfs-shell-2:19.3.0-4560.gd19e6118.el9.x86_64 142/152 2024-08-31T05:53:37.428 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephfs-devel-2:19.3.0-4560.gd19e6118.el9.x86_ 144/152 2024-08-31T05:53:37.467 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 143/152 2024-08-31T05:53:37.497 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephfs-devel-2:19.3.0-4560.gd19e6118.el9.x86_ 144/152 2024-08-31T05:53:37.993 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-31T05:53:38.034 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-31T05:53:38.054 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-31T05:53:38.108 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-31T05:53:38.153 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-31T05:53:38.173 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-31T05:53:38.513 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-test-2:19.3.0-4560.gd19e6118.el9.x86_64 137/152 2024-08-31T05:53:38.555 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 138/152 2024-08-31T05:53:38.628 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-nbd-2:19.3.0-4560.gd19e6118.el9.x86_64 139/152 2024-08-31T05:53:38.669 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 140/152 2024-08-31T05:53:38.705 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 141/152 2024-08-31T05:53:38.909 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephfs-top-2:19.3.0-4560.gd19e6118.el9.noarch 146/152 2024-08-31T05:53:38.969 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-shell-2:19.3.0-4560.gd19e6118.el9.x86_64 142/152 2024-08-31T05:53:38.993 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephfs-top-2:19.3.0-4560.gd19e6118.el9.noarch 146/152 2024-08-31T05:53:38.997 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : openssl-devel-1:3.2.2-4.el9.x86_64 147/152 2024-08-31T05:53:39.026 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 143/152 2024-08-31T05:53:39.038 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-08-31T05:53:39.054 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs-devel-2:19.3.0-4560.gd19e6118.el9.x86_ 144/152 2024-08-31T05:53:39.073 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openssl-devel-1:3.2.2-4.el9.x86_64 147/152 2024-08-31T05:53:39.114 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-08-31T05:53:39.121 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-08-31T05:53:39.151 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-08-31T05:53:39.152 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-31T05:53:39.198 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-08-31T05:53:39.219 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-31T05:53:39.219 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-31T05:53:39.244 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-08-31T05:53:39.244 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-31T05:53:39.320 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-31T05:53:39.320 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-31T05:53:39.866 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-31T05:53:39.911 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-31T05:53:39.954 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-31T05:53:41.318 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-top-2:19.3.0-4560.gd19e6118.el9.noarch 146/152 2024-08-31T05:53:41.436 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openssl-devel-1:3.2.2-4.el9.x86_64 147/152 2024-08-31T05:53:41.485 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-08-31T05:53:41.617 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-08-31T05:53:41.675 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-08-31T05:53:41.675 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-31T05:53:41.782 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-31T05:53:41.782 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-31T05:53:42.157 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-31T05:53:42.157 INFO:teuthology.orchestra.run.smithi016.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-08-31T05:53:42.157 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T05:53:45.814 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-31T05:53:45.814 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-2:19.3.0-4560.gd19e6118.el9.x86_64 1/152 2024-08-31T05:53:45.814 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-base-2:19.3.0-4560.gd19e6118.el9.x86_64 2/152 2024-08-31T05:53:45.815 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 3/152 2024-08-31T05:53:45.815 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 4/152 2024-08-31T05:53:45.815 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4560.gd19e6 5/152 2024-08-31T05:53:45.815 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mds-2:19.3.0-4560.gd19e6118.el9.x86_64 6/152 2024-08-31T05:53:45.815 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-2:19.3.0-4560.gd19e6118.el9.x86_64 7/152 2024-08-31T05:53:45.815 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mon-2:19.3.0-4560.gd19e6118.el9.x86_64 8/152 2024-08-31T05:53:45.815 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-osd-2:19.3.0-4560.gd19e6118.el9.x86_64 9/152 2024-08-31T05:53:45.815 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-radosgw-2:19.3.0-4560.gd19e6118.el9.x86_64 10/152 2024-08-31T05:53:45.815 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-selinux-2:19.3.0-4560.gd19e6118.el9.x86_64 11/152 2024-08-31T05:53:45.815 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-test-2:19.3.0-4560.gd19e6118.el9.x86_64 12/152 2024-08-31T05:53:45.815 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephfs-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 13/152 2024-08-31T05:53:45.815 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephfs-shell-2:19.3.0-4560.gd19e6118.el9.x86_64 14/152 2024-08-31T05:53:45.815 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephfs-devel-2:19.3.0-4560.gd19e6118.el9.x86_ 15/152 2024-08-31T05:53:45.816 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephfs2-2:19.3.0-4560.gd19e6118.el9.x86_64 16/152 2024-08-31T05:53:45.816 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephsqlite-2:19.3.0-4560.gd19e6118.el9.x86_64 17/152 2024-08-31T05:53:45.816 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados-devel-2:19.3.0-4560.gd19e6118.el9.x86_6 18/152 2024-08-31T05:53:45.816 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libradosstriper1-2:19.3.0-4560.gd19e6118.el9.x86 19/152 2024-08-31T05:53:45.816 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librgw2-2:19.3.0-4560.gd19e6118.el9.x86_64 20/152 2024-08-31T05:53:45.816 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4560.gd19e6118.el 21/152 2024-08-31T05:53:45.816 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ceph-common-2:19.3.0-4560.gd19e6118.el9. 22/152 2024-08-31T05:53:45.816 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cephfs-2:19.3.0-4560.gd19e6118.el9.x86_6 23/152 2024-08-31T05:53:45.816 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rados-2:19.3.0-4560.gd19e6118.el9.x86_64 24/152 2024-08-31T05:53:45.816 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rbd-2:19.3.0-4560.gd19e6118.el9.x86_64 25/152 2024-08-31T05:53:45.816 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rgw-2:19.3.0-4560.gd19e6118.el9.x86_64 26/152 2024-08-31T05:53:45.816 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 27/152 2024-08-31T05:53:45.827 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 28/152 2024-08-31T05:53:45.827 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-nbd-2:19.3.0-4560.gd19e6118.el9.x86_64 29/152 2024-08-31T05:53:45.827 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4560.gd19e6118. 30/152 2024-08-31T05:53:45.827 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4560.gd19e6118.el9.noa 31/152 2024-08-31T05:53:45.827 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4560.gd19e6118.el9.n 32/152 2024-08-31T05:53:45.827 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4560.gd19 33/152 2024-08-31T05:53:45.827 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4560.gd19e6118.el 34/152 2024-08-31T05:53:45.827 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4560.gd19e6118.el9.noarch 35/152 2024-08-31T05:53:45.827 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4560.gd19e6118.e 36/152 2024-08-31T05:53:45.827 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-volume-2:19.3.0-4560.gd19e6118.el9.noarch 37/152 2024-08-31T05:53:45.827 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 38/152 2024-08-31T05:53:45.828 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephfs-top-2:19.3.0-4560.gd19e6118.el9.noarch 39/152 2024-08-31T05:53:45.828 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/152 2024-08-31T05:53:45.828 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/152 2024-08-31T05:53:45.828 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/152 2024-08-31T05:53:45.828 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-08-31T05:53:45.828 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-08-31T05:53:45.828 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-08-31T05:53:45.828 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-08-31T05:53:45.828 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-08-31T05:53:45.828 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/152 2024-08-31T05:53:45.828 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-08-31T05:53:45.828 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-08-31T05:53:45.828 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-08-31T05:53:45.829 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/152 2024-08-31T05:53:45.829 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-08-31T05:53:45.829 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-08-31T05:53:45.829 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-08-31T05:53:45.829 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-08-31T05:53:45.829 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-08-31T05:53:45.829 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-08-31T05:53:45.829 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-08-31T05:53:45.829 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-08-31T05:53:45.829 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/152 2024-08-31T05:53:45.829 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/152 2024-08-31T05:53:45.829 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-08-31T05:53:45.830 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-08-31T05:53:45.830 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-08-31T05:53:45.830 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/152 2024-08-31T05:53:45.830 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/152 2024-08-31T05:53:45.830 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openssl-devel-1:3.2.2-4.el9.x86_64 68/152 2024-08-31T05:53:45.830 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/152 2024-08-31T05:53:45.830 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/152 2024-08-31T05:53:45.830 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/152 2024-08-31T05:53:45.830 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/152 2024-08-31T05:53:45.830 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-08-31T05:53:45.830 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-08-31T05:53:45.830 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-08-31T05:53:45.831 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/152 2024-08-31T05:53:45.831 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/152 2024-08-31T05:53:45.831 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/152 2024-08-31T05:53:45.831 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/152 2024-08-31T05:53:45.831 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/152 2024-08-31T05:53:45.831 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/152 2024-08-31T05:53:45.831 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/152 2024-08-31T05:53:45.831 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/152 2024-08-31T05:53:45.831 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/152 2024-08-31T05:53:45.831 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/152 2024-08-31T05:53:45.831 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/152 2024-08-31T05:53:45.831 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/152 2024-08-31T05:53:45.831 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/152 2024-08-31T05:53:45.831 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/152 2024-08-31T05:53:45.832 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/152 2024-08-31T05:53:45.832 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/152 2024-08-31T05:53:45.832 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/152 2024-08-31T05:53:45.832 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-31T05:53:45.832 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/152 2024-08-31T05:53:45.832 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/152 2024-08-31T05:53:45.832 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/152 2024-08-31T05:53:45.832 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/152 2024-08-31T05:53:45.832 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/152 2024-08-31T05:53:45.832 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/152 2024-08-31T05:53:45.832 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/152 2024-08-31T05:53:45.832 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/152 2024-08-31T05:53:45.832 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/152 2024-08-31T05:53:45.833 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/152 2024-08-31T05:53:45.833 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/152 2024-08-31T05:53:45.833 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/152 2024-08-31T05:53:45.833 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/152 2024-08-31T05:53:45.833 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/152 2024-08-31T05:53:45.833 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/152 2024-08-31T05:53:45.833 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/152 2024-08-31T05:53:45.833 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/152 2024-08-31T05:53:45.833 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/152 2024-08-31T05:53:45.833 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/152 2024-08-31T05:53:45.833 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/152 2024-08-31T05:53:45.833 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/152 2024-08-31T05:53:45.833 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/152 2024-08-31T05:53:45.834 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/152 2024-08-31T05:53:45.834 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/152 2024-08-31T05:53:45.834 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/152 2024-08-31T05:53:45.834 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/152 2024-08-31T05:53:45.834 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/152 2024-08-31T05:53:45.834 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/152 2024-08-31T05:53:45.834 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/152 2024-08-31T05:53:45.834 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-08-31T05:53:45.834 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-08-31T05:53:45.834 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-08-31T05:53:45.834 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-08-31T05:53:45.834 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-08-31T05:53:45.834 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-08-31T05:53:45.834 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-08-31T05:53:45.835 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-08-31T05:53:45.835 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-08-31T05:53:45.835 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-08-31T05:53:45.835 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-08-31T05:53:45.835 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-08-31T05:53:45.835 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-08-31T05:53:45.835 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-08-31T05:53:45.835 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-08-31T05:53:45.835 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-08-31T05:53:45.835 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-08-31T05:53:45.835 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-08-31T05:53:45.835 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-08-31T05:53:45.835 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/152 2024-08-31T05:53:45.835 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/152 2024-08-31T05:53:45.835 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-08-31T05:53:45.836 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-08-31T05:53:45.836 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-08-31T05:53:45.836 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/152 2024-08-31T05:53:45.836 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-08-31T05:53:45.836 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados2-2:19.3.0-4560.gd19e6118.el9.x86_64 149/152 2024-08-31T05:53:45.836 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-08-31T05:53:45.836 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librbd1-2:19.3.0-4560.gd19e6118.el9.x86_64 151/152 2024-08-31T05:53:45.991 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-31T05:53:45.991 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-2:19.3.0-4560.gd19e6118.el9.x86_64 1/152 2024-08-31T05:53:45.991 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-base-2:19.3.0-4560.gd19e6118.el9.x86_64 2/152 2024-08-31T05:53:45.991 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 3/152 2024-08-31T05:53:45.991 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 4/152 2024-08-31T05:53:45.991 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4560.gd19e6 5/152 2024-08-31T05:53:45.992 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mds-2:19.3.0-4560.gd19e6118.el9.x86_64 6/152 2024-08-31T05:53:45.992 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-2:19.3.0-4560.gd19e6118.el9.x86_64 7/152 2024-08-31T05:53:45.992 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mon-2:19.3.0-4560.gd19e6118.el9.x86_64 8/152 2024-08-31T05:53:45.992 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-osd-2:19.3.0-4560.gd19e6118.el9.x86_64 9/152 2024-08-31T05:53:45.992 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-radosgw-2:19.3.0-4560.gd19e6118.el9.x86_64 10/152 2024-08-31T05:53:45.992 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-selinux-2:19.3.0-4560.gd19e6118.el9.x86_64 11/152 2024-08-31T05:53:45.992 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-test-2:19.3.0-4560.gd19e6118.el9.x86_64 12/152 2024-08-31T05:53:45.992 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephfs-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 13/152 2024-08-31T05:53:45.992 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephfs-shell-2:19.3.0-4560.gd19e6118.el9.x86_64 14/152 2024-08-31T05:53:45.992 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephfs-devel-2:19.3.0-4560.gd19e6118.el9.x86_ 15/152 2024-08-31T05:53:45.992 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephfs2-2:19.3.0-4560.gd19e6118.el9.x86_64 16/152 2024-08-31T05:53:45.993 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephsqlite-2:19.3.0-4560.gd19e6118.el9.x86_64 17/152 2024-08-31T05:53:45.993 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados-devel-2:19.3.0-4560.gd19e6118.el9.x86_6 18/152 2024-08-31T05:53:45.993 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libradosstriper1-2:19.3.0-4560.gd19e6118.el9.x86 19/152 2024-08-31T05:53:45.993 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librgw2-2:19.3.0-4560.gd19e6118.el9.x86_64 20/152 2024-08-31T05:53:45.993 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4560.gd19e6118.el 21/152 2024-08-31T05:53:45.993 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ceph-common-2:19.3.0-4560.gd19e6118.el9. 22/152 2024-08-31T05:53:45.993 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cephfs-2:19.3.0-4560.gd19e6118.el9.x86_6 23/152 2024-08-31T05:53:45.993 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rados-2:19.3.0-4560.gd19e6118.el9.x86_64 24/152 2024-08-31T05:53:45.993 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rbd-2:19.3.0-4560.gd19e6118.el9.x86_64 25/152 2024-08-31T05:53:45.993 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rgw-2:19.3.0-4560.gd19e6118.el9.x86_64 26/152 2024-08-31T05:53:45.993 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 27/152 2024-08-31T05:53:45.993 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 28/152 2024-08-31T05:53:45.993 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-nbd-2:19.3.0-4560.gd19e6118.el9.x86_64 29/152 2024-08-31T05:53:46.004 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4560.gd19e6118. 30/152 2024-08-31T05:53:46.004 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4560.gd19e6118.el9.noa 31/152 2024-08-31T05:53:46.004 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4560.gd19e6118.el9.n 32/152 2024-08-31T05:53:46.004 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4560.gd19 33/152 2024-08-31T05:53:46.004 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4560.gd19e6118.el 34/152 2024-08-31T05:53:46.004 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4560.gd19e6118.el9.noarch 35/152 2024-08-31T05:53:46.004 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4560.gd19e6118.e 36/152 2024-08-31T05:53:46.004 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-volume-2:19.3.0-4560.gd19e6118.el9.noarch 37/152 2024-08-31T05:53:46.004 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 38/152 2024-08-31T05:53:46.004 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephfs-top-2:19.3.0-4560.gd19e6118.el9.noarch 39/152 2024-08-31T05:53:46.005 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/152 2024-08-31T05:53:46.005 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/152 2024-08-31T05:53:46.005 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/152 2024-08-31T05:53:46.005 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-08-31T05:53:46.005 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-08-31T05:53:46.005 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-08-31T05:53:46.005 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-08-31T05:53:46.005 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-08-31T05:53:46.005 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/152 2024-08-31T05:53:46.005 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-08-31T05:53:46.005 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-08-31T05:53:46.005 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-08-31T05:53:46.005 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/152 2024-08-31T05:53:46.006 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-08-31T05:53:46.006 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-08-31T05:53:46.006 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-08-31T05:53:46.006 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-08-31T05:53:46.006 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-08-31T05:53:46.006 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-08-31T05:53:46.006 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-08-31T05:53:46.006 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-08-31T05:53:46.006 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/152 2024-08-31T05:53:46.006 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/152 2024-08-31T05:53:46.006 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-08-31T05:53:46.006 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-08-31T05:53:46.006 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-08-31T05:53:46.007 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/152 2024-08-31T05:53:46.007 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/152 2024-08-31T05:53:46.007 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openssl-devel-1:3.2.2-4.el9.x86_64 68/152 2024-08-31T05:53:46.007 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/152 2024-08-31T05:53:46.007 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/152 2024-08-31T05:53:46.007 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/152 2024-08-31T05:53:46.007 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/152 2024-08-31T05:53:46.007 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-08-31T05:53:46.007 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-08-31T05:53:46.007 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-08-31T05:53:46.007 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/152 2024-08-31T05:53:46.007 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/152 2024-08-31T05:53:46.007 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/152 2024-08-31T05:53:46.007 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/152 2024-08-31T05:53:46.007 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/152 2024-08-31T05:53:46.008 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/152 2024-08-31T05:53:46.008 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/152 2024-08-31T05:53:46.008 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/152 2024-08-31T05:53:46.008 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/152 2024-08-31T05:53:46.008 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/152 2024-08-31T05:53:46.008 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/152 2024-08-31T05:53:46.008 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/152 2024-08-31T05:53:46.008 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/152 2024-08-31T05:53:46.008 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/152 2024-08-31T05:53:46.008 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/152 2024-08-31T05:53:46.008 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/152 2024-08-31T05:53:46.008 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/152 2024-08-31T05:53:46.008 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-31T05:53:46.009 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/152 2024-08-31T05:53:46.009 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/152 2024-08-31T05:53:46.009 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/152 2024-08-31T05:53:46.009 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/152 2024-08-31T05:53:46.009 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/152 2024-08-31T05:53:46.009 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/152 2024-08-31T05:53:46.009 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/152 2024-08-31T05:53:46.009 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/152 2024-08-31T05:53:46.009 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/152 2024-08-31T05:53:46.009 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/152 2024-08-31T05:53:46.009 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/152 2024-08-31T05:53:46.009 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/152 2024-08-31T05:53:46.010 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/152 2024-08-31T05:53:46.010 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/152 2024-08-31T05:53:46.010 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/152 2024-08-31T05:53:46.010 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/152 2024-08-31T05:53:46.010 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/152 2024-08-31T05:53:46.010 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/152 2024-08-31T05:53:46.010 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/152 2024-08-31T05:53:46.010 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/152 2024-08-31T05:53:46.010 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/152 2024-08-31T05:53:46.010 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/152 2024-08-31T05:53:46.010 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/152 2024-08-31T05:53:46.010 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/152 2024-08-31T05:53:46.011 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/152 2024-08-31T05:53:46.011 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/152 2024-08-31T05:53:46.011 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/152 2024-08-31T05:53:46.011 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/152 2024-08-31T05:53:46.011 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/152 2024-08-31T05:53:46.011 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-08-31T05:53:46.011 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-08-31T05:53:46.011 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-08-31T05:53:46.011 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-08-31T05:53:46.011 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-08-31T05:53:46.011 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-08-31T05:53:46.012 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-08-31T05:53:46.012 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-08-31T05:53:46.012 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-08-31T05:53:46.012 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-08-31T05:53:46.012 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-08-31T05:53:46.012 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-08-31T05:53:46.012 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-08-31T05:53:46.012 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-08-31T05:53:46.012 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-08-31T05:53:46.012 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-08-31T05:53:46.012 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-08-31T05:53:46.012 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-08-31T05:53:46.012 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-08-31T05:53:46.012 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/152 2024-08-31T05:53:46.013 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/152 2024-08-31T05:53:46.013 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-08-31T05:53:46.013 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-08-31T05:53:46.013 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-08-31T05:53:46.013 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/152 2024-08-31T05:53:46.013 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-08-31T05:53:46.013 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados2-2:19.3.0-4560.gd19e6118.el9.x86_64 149/152 2024-08-31T05:53:46.013 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-08-31T05:53:46.013 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librbd1-2:19.3.0-4560.gd19e6118.el9.x86_64 151/152 2024-08-31T05:53:48.102 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-08-31T05:53:48.103 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-08-31T05:53:48.103 INFO:teuthology.orchestra.run.smithi155.stdout:Upgraded: 2024-08-31T05:53:48.103 INFO:teuthology.orchestra.run.smithi155.stdout: librados2-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.103 INFO:teuthology.orchestra.run.smithi155.stdout: librbd1-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.103 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-08-31T05:53:48.103 INFO:teuthology.orchestra.run.smithi155.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-08-31T05:53:48.103 INFO:teuthology.orchestra.run.smithi155.stdout: automake-1.16.2-8.el9.noarch 2024-08-31T05:53:48.103 INFO:teuthology.orchestra.run.smithi155.stdout: bison-3.7.4-5.el9.x86_64 2024-08-31T05:53:48.103 INFO:teuthology.orchestra.run.smithi155.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-08-31T05:53:48.103 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.103 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-base-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.103 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.104 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.104 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-grafana-dashboards-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T05:53:48.104 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-immutable-object-cache-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.104 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mds-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.104 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.104 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T05:53:48.104 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-dashboard-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T05:53:48.104 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T05:53:48.104 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-modules-core-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T05:53:48.104 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-rook-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T05:53:48.104 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mon-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.104 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-osd-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.105 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-prometheus-alerts-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T05:53:48.105 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-radosgw-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.105 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-selinux-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.105 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-test-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.105 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-volume-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T05:53:48.105 INFO:teuthology.orchestra.run.smithi155.stdout: cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T05:53:48.105 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.105 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-shell-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.105 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-top-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T05:53:48.105 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-08-31T05:53:48.105 INFO:teuthology.orchestra.run.smithi155.stdout: flex-2.6.4-9.el9.x86_64 2024-08-31T05:53:48.105 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-08-31T05:53:48.106 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-08-31T05:53:48.106 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-08-31T05:53:48.106 INFO:teuthology.orchestra.run.smithi155.stdout: fuse-2.9.9-16.el9.x86_64 2024-08-31T05:53:48.106 INFO:teuthology.orchestra.run.smithi155.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-08-31T05:53:48.106 INFO:teuthology.orchestra.run.smithi155.stdout: grpc-data-1.46.7-10.el9.noarch 2024-08-31T05:53:48.106 INFO:teuthology.orchestra.run.smithi155.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-08-31T05:53:48.106 INFO:teuthology.orchestra.run.smithi155.stdout: libarrow-9.0.0-11.el9.x86_64 2024-08-31T05:53:48.106 INFO:teuthology.orchestra.run.smithi155.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-08-31T05:53:48.106 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs-devel-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.106 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs2-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.106 INFO:teuthology.orchestra.run.smithi155.stdout: libcephsqlite-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.106 INFO:teuthology.orchestra.run.smithi155.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-08-31T05:53:48.107 INFO:teuthology.orchestra.run.smithi155.stdout: liboath-2.6.7-2.el9.x86_64 2024-08-31T05:53:48.107 INFO:teuthology.orchestra.run.smithi155.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-08-31T05:53:48.107 INFO:teuthology.orchestra.run.smithi155.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-08-31T05:53:48.107 INFO:teuthology.orchestra.run.smithi155.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-08-31T05:53:48.107 INFO:teuthology.orchestra.run.smithi155.stdout: librados-devel-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.107 INFO:teuthology.orchestra.run.smithi155.stdout: libradosstriper1-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.107 INFO:teuthology.orchestra.run.smithi155.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-08-31T05:53:48.107 INFO:teuthology.orchestra.run.smithi155.stdout: librgw2-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.107 INFO:teuthology.orchestra.run.smithi155.stdout: libtool-2.4.6-46.el9.x86_64 2024-08-31T05:53:48.107 INFO:teuthology.orchestra.run.smithi155.stdout: libunwind-1.6.2-1.el9.x86_64 2024-08-31T05:53:48.107 INFO:teuthology.orchestra.run.smithi155.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-08-31T05:53:48.107 INFO:teuthology.orchestra.run.smithi155.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-08-31T05:53:48.107 INFO:teuthology.orchestra.run.smithi155.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-08-31T05:53:48.107 INFO:teuthology.orchestra.run.smithi155.stdout: lua-5.4.4-4.el9.x86_64 2024-08-31T05:53:48.108 INFO:teuthology.orchestra.run.smithi155.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-08-31T05:53:48.108 INFO:teuthology.orchestra.run.smithi155.stdout: luarocks-3.9.2-5.el9.noarch 2024-08-31T05:53:48.108 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-0.3.26-2.el9.x86_64 2024-08-31T05:53:48.108 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-08-31T05:53:48.108 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-devel-1:3.2.2-4.el9.x86_64 2024-08-31T05:53:48.108 INFO:teuthology.orchestra.run.smithi155.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-08-31T05:53:48.108 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-13.16-1.el9.x86_64 2024-08-31T05:53:48.108 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-08-31T05:53:48.108 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-08-31T05:53:48.108 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-server-13.16-1.el9.x86_64 2024-08-31T05:53:48.108 INFO:teuthology.orchestra.run.smithi155.stdout: protobuf-3.14.0-13.el9.x86_64 2024-08-31T05:53:48.108 INFO:teuthology.orchestra.run.smithi155.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-08-31T05:53:48.108 INFO:teuthology.orchestra.run.smithi155.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-08-31T05:53:48.109 INFO:teuthology.orchestra.run.smithi155.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-08-31T05:53:48.109 INFO:teuthology.orchestra.run.smithi155.stdout: python3-babel-2.9.1-2.el9.noarch 2024-08-31T05:53:48.109 INFO:teuthology.orchestra.run.smithi155.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-08-31T05:53:48.109 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-08-31T05:53:48.109 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-argparse-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.109 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.109 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cephfs-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.109 INFO:teuthology.orchestra.run.smithi155.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-08-31T05:53:48.109 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-08-31T05:53:48.109 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-08-31T05:53:48.109 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-08-31T05:53:48.109 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-08-31T05:53:48.109 INFO:teuthology.orchestra.run.smithi155.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-08-31T05:53:48.109 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-08-31T05:53:48.110 INFO:teuthology.orchestra.run.smithi155.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-08-31T05:53:48.110 INFO:teuthology.orchestra.run.smithi155.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-08-31T05:53:48.110 INFO:teuthology.orchestra.run.smithi155.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-08-31T05:53:48.110 INFO:teuthology.orchestra.run.smithi155.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-08-31T05:53:48.110 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-08-31T05:53:48.110 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-08-31T05:53:48.110 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-08-31T05:53:48.110 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-08-31T05:53:48.110 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-08-31T05:53:48.110 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-08-31T05:53:48.110 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-08-31T05:53:48.110 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-08-31T05:53:48.110 INFO:teuthology.orchestra.run.smithi155.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-08-31T05:53:48.111 INFO:teuthology.orchestra.run.smithi155.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-08-31T05:53:48.111 INFO:teuthology.orchestra.run.smithi155.stdout: python3-mako-1.1.4-6.el9.noarch 2024-08-31T05:53:48.111 INFO:teuthology.orchestra.run.smithi155.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-08-31T05:53:48.111 INFO:teuthology.orchestra.run.smithi155.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-08-31T05:53:48.111 INFO:teuthology.orchestra.run.smithi155.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-08-31T05:53:48.111 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-08-31T05:53:48.111 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-08-31T05:53:48.111 INFO:teuthology.orchestra.run.smithi155.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-08-31T05:53:48.111 INFO:teuthology.orchestra.run.smithi155.stdout: python3-packaging-20.9-5.el9.noarch 2024-08-31T05:53:48.111 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-08-31T05:53:48.111 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ply-3.11-14.el9.noarch 2024-08-31T05:53:48.111 INFO:teuthology.orchestra.run.smithi155.stdout: python3-portend-3.1.0-2.el9.noarch 2024-08-31T05:53:48.111 INFO:teuthology.orchestra.run.smithi155.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-08-31T05:53:48.112 INFO:teuthology.orchestra.run.smithi155.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-08-31T05:53:48.112 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-08-31T05:53:48.112 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-08-31T05:53:48.112 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-08-31T05:53:48.112 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pycparser-2.20-6.el9.noarch 2024-08-31T05:53:48.112 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-08-31T05:53:48.112 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pytz-2021.1-5.el9.noarch 2024-08-31T05:53:48.112 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rados-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.112 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rbd-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.112 INFO:teuthology.orchestra.run.smithi155.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-08-31T05:53:48.112 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests-2.25.1-8.el9.noarch 2024-08-31T05:53:48.112 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-08-31T05:53:48.113 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rgw-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.113 INFO:teuthology.orchestra.run.smithi155.stdout: python3-routes-2.5.1-5.el9.noarch 2024-08-31T05:53:48.113 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rsa-4.9-2.el9.noarch 2024-08-31T05:53:48.113 INFO:teuthology.orchestra.run.smithi155.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-08-31T05:53:48.113 INFO:teuthology.orchestra.run.smithi155.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-08-31T05:53:48.113 INFO:teuthology.orchestra.run.smithi155.stdout: python3-toml-0.10.2-6.el9.noarch 2024-08-31T05:53:48.113 INFO:teuthology.orchestra.run.smithi155.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-08-31T05:53:48.113 INFO:teuthology.orchestra.run.smithi155.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-08-31T05:53:48.113 INFO:teuthology.orchestra.run.smithi155.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-08-31T05:53:48.113 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webob-1.8.8-2.el9.noarch 2024-08-31T05:53:48.113 INFO:teuthology.orchestra.run.smithi155.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-08-31T05:53:48.113 INFO:teuthology.orchestra.run.smithi155.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-08-31T05:53:48.113 INFO:teuthology.orchestra.run.smithi155.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-08-31T05:53:48.114 INFO:teuthology.orchestra.run.smithi155.stdout: qatlib-24.02.0-1.el9.x86_64 2024-08-31T05:53:48.114 INFO:teuthology.orchestra.run.smithi155.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-08-31T05:53:48.114 INFO:teuthology.orchestra.run.smithi155.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-08-31T05:53:48.114 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.114 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.114 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-nbd-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.114 INFO:teuthology.orchestra.run.smithi155.stdout: re2-1:20211101-20.el9.x86_64 2024-08-31T05:53:48.114 INFO:teuthology.orchestra.run.smithi155.stdout: socat-1.7.4.1-6.el9.x86_64 2024-08-31T05:53:48.114 INFO:teuthology.orchestra.run.smithi155.stdout: thrift-0.15.0-2.el9.x86_64 2024-08-31T05:53:48.114 INFO:teuthology.orchestra.run.smithi155.stdout: uuid-1.6.2-55.el9.x86_64 2024-08-31T05:53:48.114 INFO:teuthology.orchestra.run.smithi155.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-08-31T05:53:48.114 INFO:teuthology.orchestra.run.smithi155.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-08-31T05:53:48.115 INFO:teuthology.orchestra.run.smithi155.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-08-31T05:53:48.115 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-08-31T05:53:48.115 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-08-31T05:53:48.263 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-08-31T05:53:48.263 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-08-31T05:53:48.263 INFO:teuthology.orchestra.run.smithi184.stdout:Upgraded: 2024-08-31T05:53:48.264 INFO:teuthology.orchestra.run.smithi184.stdout: librados2-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.264 INFO:teuthology.orchestra.run.smithi184.stdout: librbd1-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.264 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-08-31T05:53:48.264 INFO:teuthology.orchestra.run.smithi184.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-08-31T05:53:48.264 INFO:teuthology.orchestra.run.smithi184.stdout: automake-1.16.2-8.el9.noarch 2024-08-31T05:53:48.264 INFO:teuthology.orchestra.run.smithi184.stdout: bison-3.7.4-5.el9.x86_64 2024-08-31T05:53:48.264 INFO:teuthology.orchestra.run.smithi184.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-08-31T05:53:48.264 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.264 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-base-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.264 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.264 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.264 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-grafana-dashboards-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T05:53:48.264 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-immutable-object-cache-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.265 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mds-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.265 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.265 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T05:53:48.265 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-dashboard-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T05:53:48.265 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T05:53:48.265 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-modules-core-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T05:53:48.265 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-rook-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T05:53:48.265 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mon-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.265 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-osd-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.265 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-prometheus-alerts-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T05:53:48.265 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-radosgw-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.265 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-selinux-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.265 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-test-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.266 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-volume-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T05:53:48.266 INFO:teuthology.orchestra.run.smithi184.stdout: cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T05:53:48.266 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.266 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-shell-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.266 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-top-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T05:53:48.266 INFO:teuthology.orchestra.run.smithi184.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-08-31T05:53:48.266 INFO:teuthology.orchestra.run.smithi184.stdout: flex-2.6.4-9.el9.x86_64 2024-08-31T05:53:48.266 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-08-31T05:53:48.266 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-08-31T05:53:48.266 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-08-31T05:53:48.266 INFO:teuthology.orchestra.run.smithi184.stdout: fuse-2.9.9-16.el9.x86_64 2024-08-31T05:53:48.266 INFO:teuthology.orchestra.run.smithi184.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-08-31T05:53:48.266 INFO:teuthology.orchestra.run.smithi184.stdout: grpc-data-1.46.7-10.el9.noarch 2024-08-31T05:53:48.266 INFO:teuthology.orchestra.run.smithi184.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-08-31T05:53:48.267 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow-9.0.0-11.el9.x86_64 2024-08-31T05:53:48.267 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-08-31T05:53:48.267 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs-devel-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.267 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs2-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.267 INFO:teuthology.orchestra.run.smithi184.stdout: libcephsqlite-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.267 INFO:teuthology.orchestra.run.smithi184.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-08-31T05:53:48.267 INFO:teuthology.orchestra.run.smithi184.stdout: liboath-2.6.7-2.el9.x86_64 2024-08-31T05:53:48.267 INFO:teuthology.orchestra.run.smithi184.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-08-31T05:53:48.267 INFO:teuthology.orchestra.run.smithi184.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-08-31T05:53:48.267 INFO:teuthology.orchestra.run.smithi184.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-08-31T05:53:48.267 INFO:teuthology.orchestra.run.smithi184.stdout: librados-devel-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.267 INFO:teuthology.orchestra.run.smithi184.stdout: libradosstriper1-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.267 INFO:teuthology.orchestra.run.smithi184.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-08-31T05:53:48.267 INFO:teuthology.orchestra.run.smithi184.stdout: librgw2-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.268 INFO:teuthology.orchestra.run.smithi184.stdout: libtool-2.4.6-46.el9.x86_64 2024-08-31T05:53:48.268 INFO:teuthology.orchestra.run.smithi184.stdout: libunwind-1.6.2-1.el9.x86_64 2024-08-31T05:53:48.268 INFO:teuthology.orchestra.run.smithi184.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-08-31T05:53:48.268 INFO:teuthology.orchestra.run.smithi184.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-08-31T05:53:48.268 INFO:teuthology.orchestra.run.smithi184.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-08-31T05:53:48.268 INFO:teuthology.orchestra.run.smithi184.stdout: lua-5.4.4-4.el9.x86_64 2024-08-31T05:53:48.268 INFO:teuthology.orchestra.run.smithi184.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-08-31T05:53:48.268 INFO:teuthology.orchestra.run.smithi184.stdout: luarocks-3.9.2-5.el9.noarch 2024-08-31T05:53:48.268 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-0.3.26-2.el9.x86_64 2024-08-31T05:53:48.268 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-08-31T05:53:48.268 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-devel-1:3.2.2-4.el9.x86_64 2024-08-31T05:53:48.268 INFO:teuthology.orchestra.run.smithi184.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-08-31T05:53:48.268 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-13.16-1.el9.x86_64 2024-08-31T05:53:48.269 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-08-31T05:53:48.269 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-08-31T05:53:48.269 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-server-13.16-1.el9.x86_64 2024-08-31T05:53:48.269 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf-3.14.0-13.el9.x86_64 2024-08-31T05:53:48.269 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-08-31T05:53:48.269 INFO:teuthology.orchestra.run.smithi184.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-08-31T05:53:48.269 INFO:teuthology.orchestra.run.smithi184.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-08-31T05:53:48.269 INFO:teuthology.orchestra.run.smithi184.stdout: python3-babel-2.9.1-2.el9.noarch 2024-08-31T05:53:48.269 INFO:teuthology.orchestra.run.smithi184.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-08-31T05:53:48.269 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-08-31T05:53:48.269 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-argparse-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.269 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.269 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cephfs-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.269 INFO:teuthology.orchestra.run.smithi184.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-08-31T05:53:48.270 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-08-31T05:53:48.270 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-08-31T05:53:48.270 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-08-31T05:53:48.270 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-08-31T05:53:48.270 INFO:teuthology.orchestra.run.smithi184.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-08-31T05:53:48.270 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-08-31T05:53:48.270 INFO:teuthology.orchestra.run.smithi184.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-08-31T05:53:48.270 INFO:teuthology.orchestra.run.smithi184.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-08-31T05:53:48.270 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-08-31T05:53:48.270 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-08-31T05:53:48.270 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-08-31T05:53:48.270 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-08-31T05:53:48.270 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-08-31T05:53:48.271 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-08-31T05:53:48.271 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-08-31T05:53:48.271 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-08-31T05:53:48.271 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-08-31T05:53:48.271 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-08-31T05:53:48.271 INFO:teuthology.orchestra.run.smithi184.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-08-31T05:53:48.271 INFO:teuthology.orchestra.run.smithi184.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-08-31T05:53:48.271 INFO:teuthology.orchestra.run.smithi184.stdout: python3-mako-1.1.4-6.el9.noarch 2024-08-31T05:53:48.271 INFO:teuthology.orchestra.run.smithi184.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-08-31T05:53:48.271 INFO:teuthology.orchestra.run.smithi184.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-08-31T05:53:48.271 INFO:teuthology.orchestra.run.smithi184.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-08-31T05:53:48.271 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-08-31T05:53:48.271 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-08-31T05:53:48.271 INFO:teuthology.orchestra.run.smithi184.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-08-31T05:53:48.272 INFO:teuthology.orchestra.run.smithi184.stdout: python3-packaging-20.9-5.el9.noarch 2024-08-31T05:53:48.272 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-08-31T05:53:48.272 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ply-3.11-14.el9.noarch 2024-08-31T05:53:48.272 INFO:teuthology.orchestra.run.smithi184.stdout: python3-portend-3.1.0-2.el9.noarch 2024-08-31T05:53:48.272 INFO:teuthology.orchestra.run.smithi184.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-08-31T05:53:48.272 INFO:teuthology.orchestra.run.smithi184.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-08-31T05:53:48.272 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-08-31T05:53:48.272 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-08-31T05:53:48.272 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-08-31T05:53:48.272 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pycparser-2.20-6.el9.noarch 2024-08-31T05:53:48.272 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-08-31T05:53:48.272 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pytz-2021.1-5.el9.noarch 2024-08-31T05:53:48.272 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rados-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.272 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rbd-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.273 INFO:teuthology.orchestra.run.smithi184.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-08-31T05:53:48.273 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-2.25.1-8.el9.noarch 2024-08-31T05:53:48.273 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-08-31T05:53:48.273 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rgw-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.273 INFO:teuthology.orchestra.run.smithi184.stdout: python3-routes-2.5.1-5.el9.noarch 2024-08-31T05:53:48.273 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rsa-4.9-2.el9.noarch 2024-08-31T05:53:48.273 INFO:teuthology.orchestra.run.smithi184.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-08-31T05:53:48.273 INFO:teuthology.orchestra.run.smithi184.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-08-31T05:53:48.273 INFO:teuthology.orchestra.run.smithi184.stdout: python3-toml-0.10.2-6.el9.noarch 2024-08-31T05:53:48.273 INFO:teuthology.orchestra.run.smithi184.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-08-31T05:53:48.273 INFO:teuthology.orchestra.run.smithi184.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-08-31T05:53:48.273 INFO:teuthology.orchestra.run.smithi184.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-08-31T05:53:48.274 INFO:teuthology.orchestra.run.smithi184.stdout: python3-webob-1.8.8-2.el9.noarch 2024-08-31T05:53:48.274 INFO:teuthology.orchestra.run.smithi184.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-08-31T05:53:48.274 INFO:teuthology.orchestra.run.smithi184.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-08-31T05:53:48.274 INFO:teuthology.orchestra.run.smithi184.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-08-31T05:53:48.274 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib-24.02.0-1.el9.x86_64 2024-08-31T05:53:48.274 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-08-31T05:53:48.274 INFO:teuthology.orchestra.run.smithi184.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-08-31T05:53:48.274 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.274 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.274 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-nbd-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:48.274 INFO:teuthology.orchestra.run.smithi184.stdout: re2-1:20211101-20.el9.x86_64 2024-08-31T05:53:48.274 INFO:teuthology.orchestra.run.smithi184.stdout: socat-1.7.4.1-6.el9.x86_64 2024-08-31T05:53:48.275 INFO:teuthology.orchestra.run.smithi184.stdout: thrift-0.15.0-2.el9.x86_64 2024-08-31T05:53:48.275 INFO:teuthology.orchestra.run.smithi184.stdout: uuid-1.6.2-55.el9.x86_64 2024-08-31T05:53:48.276 INFO:teuthology.orchestra.run.smithi184.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-08-31T05:53:48.276 INFO:teuthology.orchestra.run.smithi184.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-08-31T05:53:48.276 INFO:teuthology.orchestra.run.smithi184.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-08-31T05:53:48.276 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-08-31T05:53:48.276 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-08-31T05:53:48.473 DEBUG:teuthology.parallel:result is None 2024-08-31T05:53:48.490 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-2:19.3.0-4560.gd19e6118.el9.x86_64 1/152 2024-08-31T05:53:48.491 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-base-2:19.3.0-4560.gd19e6118.el9.x86_64 2/152 2024-08-31T05:53:48.491 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 3/152 2024-08-31T05:53:48.491 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 4/152 2024-08-31T05:53:48.491 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4560.gd19e6 5/152 2024-08-31T05:53:48.491 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mds-2:19.3.0-4560.gd19e6118.el9.x86_64 6/152 2024-08-31T05:53:48.491 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-2:19.3.0-4560.gd19e6118.el9.x86_64 7/152 2024-08-31T05:53:48.491 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mon-2:19.3.0-4560.gd19e6118.el9.x86_64 8/152 2024-08-31T05:53:48.491 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-osd-2:19.3.0-4560.gd19e6118.el9.x86_64 9/152 2024-08-31T05:53:48.491 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-radosgw-2:19.3.0-4560.gd19e6118.el9.x86_64 10/152 2024-08-31T05:53:48.491 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-selinux-2:19.3.0-4560.gd19e6118.el9.x86_64 11/152 2024-08-31T05:53:48.491 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-test-2:19.3.0-4560.gd19e6118.el9.x86_64 12/152 2024-08-31T05:53:48.491 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 13/152 2024-08-31T05:53:48.492 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-shell-2:19.3.0-4560.gd19e6118.el9.x86_64 14/152 2024-08-31T05:53:48.492 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs-devel-2:19.3.0-4560.gd19e6118.el9.x86_ 15/152 2024-08-31T05:53:48.492 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs2-2:19.3.0-4560.gd19e6118.el9.x86_64 16/152 2024-08-31T05:53:48.492 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephsqlite-2:19.3.0-4560.gd19e6118.el9.x86_64 17/152 2024-08-31T05:53:48.492 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados-devel-2:19.3.0-4560.gd19e6118.el9.x86_6 18/152 2024-08-31T05:53:48.492 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libradosstriper1-2:19.3.0-4560.gd19e6118.el9.x86 19/152 2024-08-31T05:53:48.492 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librgw2-2:19.3.0-4560.gd19e6118.el9.x86_64 20/152 2024-08-31T05:53:48.492 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4560.gd19e6118.el 21/152 2024-08-31T05:53:48.492 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-common-2:19.3.0-4560.gd19e6118.el9. 22/152 2024-08-31T05:53:48.492 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cephfs-2:19.3.0-4560.gd19e6118.el9.x86_6 23/152 2024-08-31T05:53:48.493 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rados-2:19.3.0-4560.gd19e6118.el9.x86_64 24/152 2024-08-31T05:53:48.493 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rbd-2:19.3.0-4560.gd19e6118.el9.x86_64 25/152 2024-08-31T05:53:48.493 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rgw-2:19.3.0-4560.gd19e6118.el9.x86_64 26/152 2024-08-31T05:53:48.493 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 27/152 2024-08-31T05:53:48.505 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 28/152 2024-08-31T05:53:48.505 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-nbd-2:19.3.0-4560.gd19e6118.el9.x86_64 29/152 2024-08-31T05:53:48.505 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4560.gd19e6118. 30/152 2024-08-31T05:53:48.506 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4560.gd19e6118.el9.noa 31/152 2024-08-31T05:53:48.506 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4560.gd19e6118.el9.n 32/152 2024-08-31T05:53:48.506 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4560.gd19 33/152 2024-08-31T05:53:48.506 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4560.gd19e6118.el 34/152 2024-08-31T05:53:48.506 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4560.gd19e6118.el9.noarch 35/152 2024-08-31T05:53:48.506 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4560.gd19e6118.e 36/152 2024-08-31T05:53:48.506 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-volume-2:19.3.0-4560.gd19e6118.el9.noarch 37/152 2024-08-31T05:53:48.506 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 38/152 2024-08-31T05:53:48.506 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-top-2:19.3.0-4560.gd19e6118.el9.noarch 39/152 2024-08-31T05:53:48.506 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/152 2024-08-31T05:53:48.506 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/152 2024-08-31T05:53:48.506 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/152 2024-08-31T05:53:48.506 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-08-31T05:53:48.507 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-08-31T05:53:48.507 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-08-31T05:53:48.507 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-08-31T05:53:48.507 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-08-31T05:53:48.507 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/152 2024-08-31T05:53:48.507 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-08-31T05:53:48.507 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-08-31T05:53:48.507 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-08-31T05:53:48.507 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/152 2024-08-31T05:53:48.507 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-08-31T05:53:48.507 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-08-31T05:53:48.507 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-08-31T05:53:48.507 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-08-31T05:53:48.508 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-08-31T05:53:48.508 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-08-31T05:53:48.508 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-08-31T05:53:48.508 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-08-31T05:53:48.508 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/152 2024-08-31T05:53:48.508 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/152 2024-08-31T05:53:48.508 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-08-31T05:53:48.508 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-08-31T05:53:48.508 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-08-31T05:53:48.508 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/152 2024-08-31T05:53:48.508 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/152 2024-08-31T05:53:48.509 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-devel-1:3.2.2-4.el9.x86_64 68/152 2024-08-31T05:53:48.509 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/152 2024-08-31T05:53:48.509 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/152 2024-08-31T05:53:48.509 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/152 2024-08-31T05:53:48.509 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/152 2024-08-31T05:53:48.509 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-08-31T05:53:48.509 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-08-31T05:53:48.509 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-08-31T05:53:48.509 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/152 2024-08-31T05:53:48.509 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/152 2024-08-31T05:53:48.509 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/152 2024-08-31T05:53:48.509 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/152 2024-08-31T05:53:48.509 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/152 2024-08-31T05:53:48.510 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/152 2024-08-31T05:53:48.510 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/152 2024-08-31T05:53:48.510 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/152 2024-08-31T05:53:48.510 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/152 2024-08-31T05:53:48.510 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/152 2024-08-31T05:53:48.510 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/152 2024-08-31T05:53:48.510 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/152 2024-08-31T05:53:48.510 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/152 2024-08-31T05:53:48.510 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/152 2024-08-31T05:53:48.510 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/152 2024-08-31T05:53:48.510 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/152 2024-08-31T05:53:48.510 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/152 2024-08-31T05:53:48.510 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-31T05:53:48.511 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/152 2024-08-31T05:53:48.511 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/152 2024-08-31T05:53:48.511 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/152 2024-08-31T05:53:48.511 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/152 2024-08-31T05:53:48.511 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/152 2024-08-31T05:53:48.511 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/152 2024-08-31T05:53:48.511 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/152 2024-08-31T05:53:48.511 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/152 2024-08-31T05:53:48.511 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/152 2024-08-31T05:53:48.511 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/152 2024-08-31T05:53:48.511 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/152 2024-08-31T05:53:48.511 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/152 2024-08-31T05:53:48.511 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/152 2024-08-31T05:53:48.511 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/152 2024-08-31T05:53:48.512 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/152 2024-08-31T05:53:48.512 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/152 2024-08-31T05:53:48.512 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/152 2024-08-31T05:53:48.512 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/152 2024-08-31T05:53:48.512 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/152 2024-08-31T05:53:48.512 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/152 2024-08-31T05:53:48.512 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/152 2024-08-31T05:53:48.512 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/152 2024-08-31T05:53:48.512 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/152 2024-08-31T05:53:48.512 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/152 2024-08-31T05:53:48.512 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/152 2024-08-31T05:53:48.512 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/152 2024-08-31T05:53:48.512 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/152 2024-08-31T05:53:48.512 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/152 2024-08-31T05:53:48.513 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/152 2024-08-31T05:53:48.513 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-08-31T05:53:48.513 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-08-31T05:53:48.513 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-08-31T05:53:48.513 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-08-31T05:53:48.513 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-08-31T05:53:48.513 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-08-31T05:53:48.513 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-08-31T05:53:48.513 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-08-31T05:53:48.513 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-08-31T05:53:48.513 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-08-31T05:53:48.513 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-08-31T05:53:48.513 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-08-31T05:53:48.514 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-08-31T05:53:48.514 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-08-31T05:53:48.514 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-08-31T05:53:48.514 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-08-31T05:53:48.514 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-08-31T05:53:48.514 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-08-31T05:53:48.514 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-08-31T05:53:48.514 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/152 2024-08-31T05:53:48.514 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/152 2024-08-31T05:53:48.514 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-08-31T05:53:48.514 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-08-31T05:53:48.514 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-08-31T05:53:48.514 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/152 2024-08-31T05:53:48.514 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-08-31T05:53:48.514 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:19.3.0-4560.gd19e6118.el9.x86_64 149/152 2024-08-31T05:53:48.515 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-08-31T05:53:48.515 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:19.3.0-4560.gd19e6118.el9.x86_64 151/152 2024-08-31T05:53:48.605 DEBUG:teuthology.parallel:result is None 2024-08-31T05:53:50.957 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-08-31T05:53:50.957 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T05:53:50.957 INFO:teuthology.orchestra.run.smithi016.stdout:Upgraded: 2024-08-31T05:53:50.957 INFO:teuthology.orchestra.run.smithi016.stdout: librados2-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:50.957 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:50.958 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-08-31T05:53:50.958 INFO:teuthology.orchestra.run.smithi016.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-08-31T05:53:50.958 INFO:teuthology.orchestra.run.smithi016.stdout: automake-1.16.2-8.el9.noarch 2024-08-31T05:53:50.958 INFO:teuthology.orchestra.run.smithi016.stdout: bison-3.7.4-5.el9.x86_64 2024-08-31T05:53:50.958 INFO:teuthology.orchestra.run.smithi016.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-08-31T05:53:50.958 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:50.958 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:50.958 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:50.958 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:50.958 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T05:53:50.958 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:50.958 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:50.959 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:50.959 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T05:53:50.959 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T05:53:50.959 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T05:53:50.959 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T05:53:50.959 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T05:53:50.959 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:50.959 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:50.959 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T05:53:50.959 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:50.959 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:50.959 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:50.959 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T05:53:50.959 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T05:53:50.960 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:50.960 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-shell-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:50.960 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top-2:19.3.0-4560.gd19e6118.el9.noarch 2024-08-31T05:53:50.960 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-08-31T05:53:50.960 INFO:teuthology.orchestra.run.smithi016.stdout: flex-2.6.4-9.el9.x86_64 2024-08-31T05:53:50.960 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-08-31T05:53:50.960 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-08-31T05:53:50.960 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-08-31T05:53:50.960 INFO:teuthology.orchestra.run.smithi016.stdout: fuse-2.9.9-16.el9.x86_64 2024-08-31T05:53:50.960 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-08-31T05:53:50.961 INFO:teuthology.orchestra.run.smithi016.stdout: grpc-data-1.46.7-10.el9.noarch 2024-08-31T05:53:50.961 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-08-31T05:53:50.961 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-9.0.0-11.el9.x86_64 2024-08-31T05:53:50.961 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-08-31T05:53:50.961 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:50.961 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:50.961 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:50.961 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-08-31T05:53:50.961 INFO:teuthology.orchestra.run.smithi016.stdout: liboath-2.6.7-2.el9.x86_64 2024-08-31T05:53:50.961 INFO:teuthology.orchestra.run.smithi016.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-08-31T05:53:50.961 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-08-31T05:53:50.961 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-08-31T05:53:50.962 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:50.962 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:50.962 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-08-31T05:53:50.962 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:50.962 INFO:teuthology.orchestra.run.smithi016.stdout: libtool-2.4.6-46.el9.x86_64 2024-08-31T05:53:50.962 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind-1.6.2-1.el9.x86_64 2024-08-31T05:53:50.962 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-08-31T05:53:50.962 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-08-31T05:53:50.962 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-08-31T05:53:50.962 INFO:teuthology.orchestra.run.smithi016.stdout: lua-5.4.4-4.el9.x86_64 2024-08-31T05:53:50.962 INFO:teuthology.orchestra.run.smithi016.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-08-31T05:53:50.962 INFO:teuthology.orchestra.run.smithi016.stdout: luarocks-3.9.2-5.el9.noarch 2024-08-31T05:53:50.962 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-0.3.26-2.el9.x86_64 2024-08-31T05:53:50.963 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-08-31T05:53:50.963 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel-1:3.2.2-4.el9.x86_64 2024-08-31T05:53:50.963 INFO:teuthology.orchestra.run.smithi016.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-08-31T05:53:50.963 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-13.16-1.el9.x86_64 2024-08-31T05:53:50.963 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-08-31T05:53:50.963 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-08-31T05:53:50.963 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-server-13.16-1.el9.x86_64 2024-08-31T05:53:50.963 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-3.14.0-13.el9.x86_64 2024-08-31T05:53:50.963 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-08-31T05:53:50.963 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-08-31T05:53:50.963 INFO:teuthology.orchestra.run.smithi016.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-08-31T05:53:50.963 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel-2.9.1-2.el9.noarch 2024-08-31T05:53:50.963 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-08-31T05:53:50.963 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-08-31T05:53:50.964 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:50.964 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:50.964 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:50.964 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-08-31T05:53:50.964 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-08-31T05:53:50.964 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-08-31T05:53:50.964 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-08-31T05:53:50.964 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-08-31T05:53:50.964 INFO:teuthology.orchestra.run.smithi016.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-08-31T05:53:50.964 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-08-31T05:53:50.964 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-08-31T05:53:50.964 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-08-31T05:53:50.964 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-08-31T05:53:50.965 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-08-31T05:53:50.965 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-08-31T05:53:50.965 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-08-31T05:53:50.965 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-08-31T05:53:50.965 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-08-31T05:53:50.965 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-08-31T05:53:50.965 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-08-31T05:53:50.965 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-08-31T05:53:50.965 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-08-31T05:53:50.965 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-08-31T05:53:50.965 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-08-31T05:53:50.965 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako-1.1.4-6.el9.noarch 2024-08-31T05:53:50.965 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-08-31T05:53:50.965 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-08-31T05:53:50.966 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-08-31T05:53:50.966 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-08-31T05:53:50.966 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-08-31T05:53:50.966 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-08-31T05:53:50.966 INFO:teuthology.orchestra.run.smithi016.stdout: python3-packaging-20.9-5.el9.noarch 2024-08-31T05:53:50.966 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-08-31T05:53:50.966 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ply-3.11-14.el9.noarch 2024-08-31T05:53:50.966 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend-3.1.0-2.el9.noarch 2024-08-31T05:53:50.966 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-08-31T05:53:50.966 INFO:teuthology.orchestra.run.smithi016.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-08-31T05:53:50.966 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-08-31T05:53:50.966 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-08-31T05:53:50.966 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-08-31T05:53:50.967 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser-2.20-6.el9.noarch 2024-08-31T05:53:50.967 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-08-31T05:53:50.967 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz-2021.1-5.el9.noarch 2024-08-31T05:53:50.967 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:50.967 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:50.967 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-08-31T05:53:50.967 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-2.25.1-8.el9.noarch 2024-08-31T05:53:50.967 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-08-31T05:53:50.967 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:50.967 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes-2.5.1-5.el9.noarch 2024-08-31T05:53:50.967 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa-4.9-2.el9.noarch 2024-08-31T05:53:50.967 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-08-31T05:53:50.967 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-08-31T05:53:50.967 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml-0.10.2-6.el9.noarch 2024-08-31T05:53:50.968 INFO:teuthology.orchestra.run.smithi016.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-08-31T05:53:50.968 INFO:teuthology.orchestra.run.smithi016.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-08-31T05:53:50.968 INFO:teuthology.orchestra.run.smithi016.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-08-31T05:53:50.968 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob-1.8.8-2.el9.noarch 2024-08-31T05:53:50.968 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-08-31T05:53:50.968 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-08-31T05:53:50.968 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-08-31T05:53:50.968 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-24.02.0-1.el9.x86_64 2024-08-31T05:53:50.968 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-08-31T05:53:50.968 INFO:teuthology.orchestra.run.smithi016.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-08-31T05:53:50.968 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:50.968 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:50.968 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd-2:19.3.0-4560.gd19e6118.el9.x86_64 2024-08-31T05:53:50.969 INFO:teuthology.orchestra.run.smithi016.stdout: re2-1:20211101-20.el9.x86_64 2024-08-31T05:53:50.969 INFO:teuthology.orchestra.run.smithi016.stdout: socat-1.7.4.1-6.el9.x86_64 2024-08-31T05:53:50.969 INFO:teuthology.orchestra.run.smithi016.stdout: thrift-0.15.0-2.el9.x86_64 2024-08-31T05:53:50.969 INFO:teuthology.orchestra.run.smithi016.stdout: uuid-1.6.2-55.el9.x86_64 2024-08-31T05:53:50.969 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-08-31T05:53:50.969 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-08-31T05:53:50.969 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-08-31T05:53:50.969 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T05:53:50.969 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-08-31T05:53:51.515 DEBUG:teuthology.parallel:result is None 2024-08-31T05:53:51.516 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d19e6118f7015f0047264006cb0c873528eb3ae6 2024-08-31T05:53:51.661 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-08-31T05:53:51.705 INFO:teuthology.orchestra.run.smithi016.stdout:19.3.0-4560.gd19e6118.el9 2024-08-31T05:53:51.706 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4560.gd19e6118.el9 2024-08-31T05:53:51.706 INFO:teuthology.task.install:The correct ceph version 19.3.0-4560.gd19e6118 is installed. 2024-08-31T05:53:51.708 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d19e6118f7015f0047264006cb0c873528eb3ae6 2024-08-31T05:53:51.842 DEBUG:teuthology.orchestra.run.smithi155:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-08-31T05:53:51.869 INFO:teuthology.orchestra.run.smithi155.stdout:19.3.0-4560.gd19e6118.el9 2024-08-31T05:53:51.869 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4560.gd19e6118.el9 2024-08-31T05:53:51.869 INFO:teuthology.task.install:The correct ceph version 19.3.0-4560.gd19e6118 is installed. 2024-08-31T05:53:51.871 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d19e6118f7015f0047264006cb0c873528eb3ae6 2024-08-31T05:53:52.006 DEBUG:teuthology.orchestra.run.smithi184:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-08-31T05:53:52.033 INFO:teuthology.orchestra.run.smithi184.stdout:19.3.0-4560.gd19e6118.el9 2024-08-31T05:53:52.033 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4560.gd19e6118.el9 2024-08-31T05:53:52.034 INFO:teuthology.task.install:The correct ceph version 19.3.0-4560.gd19e6118 is installed. 2024-08-31T05:53:52.036 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-08-31T05:53:52.036 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-08-31T05:53:52.036 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-31T05:53:52.096 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-08-31T05:53:52.096 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-31T05:53:52.128 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-08-31T05:53:52.128 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-31T05:53:52.163 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-08-31T05:53:52.163 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-08-31T05:53:52.163 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/daemon-helper 2024-08-31T05:53:52.223 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-31T05:53:52.324 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-08-31T05:53:52.325 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/daemon-helper 2024-08-31T05:53:52.355 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-31T05:53:52.423 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-08-31T05:53:52.423 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/daemon-helper 2024-08-31T05:53:52.454 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-31T05:53:52.525 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-08-31T05:53:52.525 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-08-31T05:53:52.525 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-31T05:53:52.594 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-31T05:53:52.697 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-08-31T05:53:52.697 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-31T05:53:52.727 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-31T05:53:52.797 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-08-31T05:53:52.797 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-31T05:53:52.827 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-31T05:53:52.897 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-08-31T05:53:52.897 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-08-31T05:53:52.897 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/stdin-killer 2024-08-31T05:53:52.954 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-31T05:53:53.052 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-08-31T05:53:53.052 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/stdin-killer 2024-08-31T05:53:53.082 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-31T05:53:53.151 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-08-31T05:53:53.151 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/stdin-killer 2024-08-31T05:53:53.181 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-31T05:53:53.249 INFO:teuthology.run_tasks:Running task cephadm... 2024-08-31T05:53:53.352 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug cephsqlite': 20, 'mgr/snap_schedule/allow_m_granularity': True, 'mgr/snap_schedule/dump_on_update': True, 'mgr/snap_schedule/log_level': 'debug'}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'global': {'ms die on skipped message': False}, 'mds': {'mds_bal_replicate_threshold': 1}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'mgr-modules': ['snap_schedule'], 'sha1': 'd19e6118f7015f0047264006cb0c873528eb3ae6', 'subvols': {'create': 2}} 2024-08-31T05:53:53.352 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 2024-08-31T05:53:53.353 INFO:tasks.cephadm:Cluster fsid is 673e4aa0-675d-11ef-bcd4-c7b262605968 2024-08-31T05:53:53.353 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-08-31T05:53:53.353 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.16', 'mon.b': '172.21.15.155', 'mon.c': '172.21.15.184'} 2024-08-31T05:53:53.353 INFO:tasks.cephadm:First mon is mon.a on smithi016 2024-08-31T05:53:53.353 INFO:tasks.cephadm:First mgr is x 2024-08-31T05:53:53.353 INFO:tasks.cephadm:Normalizing hostnames... 2024-08-31T05:53:53.354 DEBUG:teuthology.orchestra.run.smithi016:> sudo hostname $(hostname -s) 2024-08-31T05:53:53.414 DEBUG:teuthology.orchestra.run.smithi155:> sudo hostname $(hostname -s) 2024-08-31T05:53:53.444 DEBUG:teuthology.orchestra.run.smithi184:> sudo hostname $(hostname -s) 2024-08-31T05:53:53.475 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-08-31T05:53:53.475 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d19e6118f7015f0047264006cb0c873528eb3ae6 2024-08-31T05:53:53.621 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd19e6118f7015f0047264006cb0c873528eb3ae6', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/82586/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-4560-gd19e6118', 'node_name': '172.21.5.31+adami01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-4560.gd19e6118'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-08-28 11:15:32.801765', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-mchangir-testing-20240828.085847-main-debug', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-08-31T05:53:53.761 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-mchangir-testing-20240828.085847-main-debug, sha1 d19e6118f7015f0047264006cb0c873528eb3ae6 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d19e6118f7015f0047264006cb0c873528eb3ae6 2024-08-31T05:53:53.763 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/x86_64/flavors/default/cephadm 2024-08-31T05:53:53.763 INFO:tasks.cephadm:Downloading cephadm from url: https://4.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/x86_64/flavors/default/cephadm 2024-08-31T05:53:53.763 DEBUG:teuthology.orchestra.run.smithi016:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-08-31T05:53:54.368 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r--. 1 ubuntu ubuntu 801010 Aug 31 05:53 /home/ubuntu/cephtest/cephadm 2024-08-31T05:53:54.369 DEBUG:teuthology.orchestra.run.smithi155:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-08-31T05:53:54.920 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r--. 1 ubuntu ubuntu 801010 Aug 31 05:53 /home/ubuntu/cephtest/cephadm 2024-08-31T05:53:54.921 DEBUG:teuthology.orchestra.run.smithi184:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-20240828.085847-main-debug/d19e6118f7015f0047264006cb0c873528eb3ae6/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-08-31T05:53:55.487 INFO:teuthology.orchestra.run.smithi184.stdout:-rw-r--r--. 1 ubuntu ubuntu 801010 Aug 31 05:53 /home/ubuntu/cephtest/cephadm 2024-08-31T05:53:55.488 DEBUG:teuthology.orchestra.run.smithi016:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-08-31T05:53:55.525 DEBUG:teuthology.orchestra.run.smithi155:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-08-31T05:53:55.545 DEBUG:teuthology.orchestra.run.smithi184:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-08-31T05:53:55.580 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 on all hosts... 2024-08-31T05:53:55.580 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 pull 2024-08-31T05:53:55.584 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 pull 2024-08-31T05:53:55.587 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 pull 2024-08-31T05:53:57.218 INFO:teuthology.orchestra.run.smithi184.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6... 2024-08-31T05:53:57.312 INFO:teuthology.orchestra.run.smithi155.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6... 2024-08-31T05:53:58.194 INFO:teuthology.orchestra.run.smithi016.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6... 2024-08-31T05:54:27.216 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-08-31T05:54:27.217 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_version": "ceph version 19.3.0-4560-gd19e6118 (d19e6118f7015f0047264006cb0c873528eb3ae6) squid (dev)", 2024-08-31T05:54:27.217 INFO:teuthology.orchestra.run.smithi155.stdout: "image_id": "79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0", 2024-08-31T05:54:27.217 INFO:teuthology.orchestra.run.smithi155.stdout: "repo_digests": [ 2024-08-31T05:54:27.217 INFO:teuthology.orchestra.run.smithi155.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61" 2024-08-31T05:54:27.217 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-08-31T05:54:27.217 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-08-31T05:54:51.643 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-08-31T05:54:51.643 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_version": "ceph version 19.3.0-4560-gd19e6118 (d19e6118f7015f0047264006cb0c873528eb3ae6) squid (dev)", 2024-08-31T05:54:51.643 INFO:teuthology.orchestra.run.smithi184.stdout: "image_id": "79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0", 2024-08-31T05:54:51.643 INFO:teuthology.orchestra.run.smithi184.stdout: "repo_digests": [ 2024-08-31T05:54:51.643 INFO:teuthology.orchestra.run.smithi184.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61" 2024-08-31T05:54:51.643 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-08-31T05:54:51.643 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-08-31T05:54:58.529 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T05:54:58.529 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.3.0-4560-gd19e6118 (d19e6118f7015f0047264006cb0c873528eb3ae6) squid (dev)", 2024-08-31T05:54:58.529 INFO:teuthology.orchestra.run.smithi016.stdout: "image_id": "79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0", 2024-08-31T05:54:58.529 INFO:teuthology.orchestra.run.smithi016.stdout: "repo_digests": [ 2024-08-31T05:54:58.529 INFO:teuthology.orchestra.run.smithi016.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61" 2024-08-31T05:54:58.529 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-08-31T05:54:58.529 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T05:54:58.562 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph 2024-08-31T05:54:58.622 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /etc/ceph 2024-08-31T05:54:58.657 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /etc/ceph 2024-08-31T05:54:58.691 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 777 /etc/ceph 2024-08-31T05:54:58.747 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 777 /etc/ceph 2024-08-31T05:54:58.776 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 777 /etc/ceph 2024-08-31T05:54:58.805 INFO:tasks.cephadm:Writing seed config... 2024-08-31T05:54:58.806 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-08-31T05:54:58.806 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-08-31T05:54:58.806 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-08-31T05:54:58.806 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-08-31T05:54:58.806 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-08-31T05:54:58.806 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-08-31T05:54:58.807 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-08-31T05:54:58.807 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-08-31T05:54:58.807 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-08-31T05:54:58.807 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-08-31T05:54:58.807 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-08-31T05:54:58.807 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-08-31T05:54:58.807 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-08-31T05:54:58.807 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-08-31T05:54:58.807 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-08-31T05:54:58.807 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-08-31T05:54:58.807 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-08-31T05:54:58.807 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-08-31T05:54:58.807 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-08-31T05:54:58.807 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-08-31T05:54:58.808 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-08-31T05:54:58.808 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-08-31T05:54:58.808 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-08-31T05:54:58.808 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-08-31T05:54:58.808 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-08-31T05:54:58.809 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-08-31T05:54:58.809 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-08-31T05:54:58.844 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 673e4aa0-675d-11ef-bcd4-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [mds] mds_bal_replicate_threshold = 1 2024-08-31T05:54:58.844 DEBUG:teuthology.orchestra.run.smithi016:mon.a> sudo journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mon.a.service 2024-08-31T05:54:58.888 DEBUG:teuthology.orchestra.run.smithi016:mgr.x> sudo journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mgr.x.service 2024-08-31T05:54:58.932 INFO:tasks.cephadm:Bootstrapping... 2024-08-31T05:54:58.933 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 -v bootstrap --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.16 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-08-31T05:54:59.199 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-08-31T05:54:59.200 INFO:teuthology.orchestra.run.smithi016.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', '-v', 'bootstrap', '--fsid', '673e4aa0-675d-11ef-bcd4-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.16', '--skip-admin-label'] 2024-08-31T05:54:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-08-31T05:54:59.200 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying podman|docker is present... 2024-08-31T05:54:59.247 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stdout 5.2.2 2024-08-31T05:54:59.247 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying lvm2 is present... 2024-08-31T05:54:59.247 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying time synchronization is in place... 2024-08-31T05:54:59.261 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-08-31T05:54:59.261 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-08-31T05:54:59.274 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-08-31T05:54:59.274 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-08-31T05:54:59.288 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout enabled 2024-08-31T05:54:59.300 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout active 2024-08-31T05:54:59.300 INFO:teuthology.orchestra.run.smithi016.stdout:Unit chronyd.service is enabled and running 2024-08-31T05:54:59.301 INFO:teuthology.orchestra.run.smithi016.stdout:Repeating the final host check... 2024-08-31T05:54:59.343 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stdout 5.2.2 2024-08-31T05:54:59.343 INFO:teuthology.orchestra.run.smithi016.stdout:podman (/bin/podman) version 5.2.2 is present 2024-08-31T05:54:59.343 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl is present 2024-08-31T05:54:59.343 INFO:teuthology.orchestra.run.smithi016.stdout:lvcreate is present 2024-08-31T05:54:59.356 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-08-31T05:54:59.356 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-08-31T05:54:59.368 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-08-31T05:54:59.368 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-08-31T05:54:59.381 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout enabled 2024-08-31T05:54:59.394 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout active 2024-08-31T05:54:59.394 INFO:teuthology.orchestra.run.smithi016.stdout:Unit chronyd.service is enabled and running 2024-08-31T05:54:59.394 INFO:teuthology.orchestra.run.smithi016.stdout:Host looks OK 2024-08-31T05:54:59.394 INFO:teuthology.orchestra.run.smithi016.stdout:Cluster fsid: 673e4aa0-675d-11ef-bcd4-c7b262605968 2024-08-31T05:54:59.394 INFO:teuthology.orchestra.run.smithi016.stdout:Acquiring lock 140111747339312 on /run/cephadm/673e4aa0-675d-11ef-bcd4-c7b262605968.lock 2024-08-31T05:54:59.394 INFO:teuthology.orchestra.run.smithi016.stdout:Lock 140111747339312 acquired on /run/cephadm/673e4aa0-675d-11ef-bcd4-c7b262605968.lock 2024-08-31T05:54:59.394 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying IP 172.21.15.16 port 3300 ... 2024-08-31T05:54:59.395 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying IP 172.21.15.16 port 6789 ... 2024-08-31T05:54:59.395 INFO:teuthology.orchestra.run.smithi016.stdout:Base mon IP(s) is [172.21.15.16:3300, 172.21.15.16:6789], mon addrv is [v2:172.21.15.16:3300,v1:172.21.15.16:6789] 2024-08-31T05:54:59.402 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.16 metric 100 2024-08-31T05:54:59.402 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.16 metric 100 2024-08-31T05:54:59.408 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-08-31T05:54:59.408 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-08-31T05:54:59.408 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1796sec hoplimit 64 pref medium 2024-08-31T05:54:59.414 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-08-31T05:54:59.414 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-08-31T05:54:59.414 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-08-31T05:54:59.414 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-08-31T05:54:59.414 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-08-31T05:54:59.414 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-08-31T05:54:59.415 INFO:teuthology.orchestra.run.smithi016.stdout:Mon IP `172.21.15.16` is in CIDR network `172.21.0.0/20` 2024-08-31T05:54:59.415 INFO:teuthology.orchestra.run.smithi016.stdout:Mon IP `172.21.15.16` is in CIDR network `172.21.0.0/20` 2024-08-31T05:54:59.415 INFO:teuthology.orchestra.run.smithi016.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-08-31T05:54:59.416 INFO:teuthology.orchestra.run.smithi016.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-08-31T05:54:59.416 INFO:teuthology.orchestra.run.smithi016.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6... 2024-08-31T05:54:59.897 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stdout 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 2024-08-31T05:54:59.897 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6... 2024-08-31T05:54:59.898 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Getting image source signatures 2024-08-31T05:54:59.898 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Copying blob sha256:1c7f48e91f1eb9e77df623f13bab8a42aaf12d95954655c07fb62a1a85ca19b2 2024-08-31T05:54:59.898 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-08-31T05:54:59.898 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Copying config sha256:79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 2024-08-31T05:54:59.898 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Writing manifest to image destination 2024-08-31T05:55:00.985 INFO:teuthology.orchestra.run.smithi016.stdout:ceph: stdout ceph version 19.3.0-4560-gd19e6118 (d19e6118f7015f0047264006cb0c873528eb3ae6) squid (dev) 2024-08-31T05:55:00.985 INFO:teuthology.orchestra.run.smithi016.stdout:Ceph version: ceph version 19.3.0-4560-gd19e6118 (d19e6118f7015f0047264006cb0c873528eb3ae6) squid (dev) 2024-08-31T05:55:00.985 INFO:teuthology.orchestra.run.smithi016.stdout:Extracting ceph user uid/gid from container image... 2024-08-31T05:55:02.023 INFO:teuthology.orchestra.run.smithi016.stdout:stat: stdout 167 167 2024-08-31T05:55:02.023 INFO:teuthology.orchestra.run.smithi016.stdout:Creating initial keys... 2024-08-31T05:55:03.065 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph-authtool: stdout AQA2sNJmZZh8HhAA+kvsPMnL0U+/Dip3ts4Atw== 2024-08-31T05:55:04.206 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph-authtool: stdout AQA3sNJmBfX/HhAAgPjce3TUPKOp6bVygZj34w== 2024-08-31T05:55:05.210 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph-authtool: stdout AQA4sNJmAfI7JhAA8lguoSWuaBU6n8CPCLnaow== 2024-08-31T05:55:05.211 INFO:teuthology.orchestra.run.smithi016.stdout:Creating initial monmap... 2024-08-31T05:55:06.240 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-08-31T05:55:06.240 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-08-31T05:55:06.240 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 673e4aa0-675d-11ef-bcd4-c7b262605968 2024-08-31T05:55:06.240 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-08-31T05:55:06.240 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool for a [v2:172.21.15.16:3300,v1:172.21.15.16:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-08-31T05:55:06.240 INFO:teuthology.orchestra.run.smithi016.stdout:setting min_mon_release = quincy 2024-08-31T05:55:06.240 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: set fsid to 673e4aa0-675d-11ef-bcd4-c7b262605968 2024-08-31T05:55:06.240 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-08-31T05:55:06.240 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T05:55:06.240 INFO:teuthology.orchestra.run.smithi016.stdout:Creating mon... 2024-08-31T05:55:07.489 INFO:teuthology.orchestra.run.smithi016.stdout:create mon.a on 2024-08-31T05:55:07.830 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-08-31T05:55:08.059 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-08-31T05:55:08.315 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-673e4aa0-675d-11ef-bcd4-c7b262605968.target → /etc/systemd/system/ceph-673e4aa0-675d-11ef-bcd4-c7b262605968.target. 2024-08-31T05:55:08.315 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-673e4aa0-675d-11ef-bcd4-c7b262605968.target → /etc/systemd/system/ceph-673e4aa0-675d-11ef-bcd4-c7b262605968.target. 2024-08-31T05:55:08.656 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mon.a 2024-08-31T05:55:08.656 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to reset failed state of unit ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mon.a.service: Unit ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mon.a.service not loaded. 2024-08-31T05:55:08.917 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-673e4aa0-675d-11ef-bcd4-c7b262605968.target.wants/ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mon.a.service → /etc/systemd/system/ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@.service. 2024-08-31T05:55:09.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:09 smithi016 podman[16814]: 2024-08-31 05:55:09.50050706 +0000 UTC m=+0.335885899 container start a088bba0ce12230837d0feacdd3d1abad579a04b9c91dd8160fc8d37faddf3af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mon-a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.1, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-08-31T05:55:09.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:09 smithi016 bash[16814]: a088bba0ce12230837d0feacdd3d1abad579a04b9c91dd8160fc8d37faddf3af 2024-08-31T05:55:09.643 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-31T05:55:09.644 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-08-31T05:55:09.657 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-31T05:55:09.658 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-08-31T05:55:09.658 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-08-31T05:55:09.658 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to enable service . firewalld.service is not available 2024-08-31T05:55:09.658 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mon to start... 2024-08-31T05:55:09.658 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mon... 2024-08-31T05:55:09.891 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:09 smithi016 systemd[1]: Started Ceph mon.a for 673e4aa0-675d-11ef-bcd4-c7b262605968. 2024-08-31T05:55:09.891 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:09 smithi016 ceph-mon[16832]: mkfs 673e4aa0-675d-11ef-bcd4-c7b262605968 2024-08-31T05:55:09.891 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:09 smithi016 ceph-mon[16832]: mon.a is new leader, mons a in quorum (ranks 0) 2024-08-31T05:55:10.902 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:10 smithi016 ceph-mon[16832]: mon.a is new leader, mons a in quorum (ranks 0) 2024-08-31T05:55:10.902 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:10 smithi016 ceph-mon[16832]: monmap epoch 1 2024-08-31T05:55:10.902 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:10 smithi016 ceph-mon[16832]: fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 2024-08-31T05:55:10.902 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:10 smithi016 ceph-mon[16832]: last_changed 2024-08-31T05:55:05.650830+0000 2024-08-31T05:55:10.902 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:10 smithi016 ceph-mon[16832]: created 2024-08-31T05:55:05.650830+0000 2024-08-31T05:55:10.902 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:10 smithi016 ceph-mon[16832]: min_mon_release 19 (squid) 2024-08-31T05:55:10.902 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:10 smithi016 ceph-mon[16832]: election_strategy: 1 2024-08-31T05:55:10.902 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:10 smithi016 ceph-mon[16832]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-08-31T05:55:10.902 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:10 smithi016 ceph-mon[16832]: fsmap 2024-08-31T05:55:10.902 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:10 smithi016 ceph-mon[16832]: osdmap e1: 0 total, 0 up, 0 in 2024-08-31T05:55:10.902 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:10 smithi016 ceph-mon[16832]: mgrmap e1: no daemons active 2024-08-31T05:55:10.902 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:10 smithi016 ceph-mon[16832]: from='client.? 172.21.15.16:0/1076366803' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-08-31T05:55:10.906 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout cluster: 2024-08-31T05:55:10.908 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout id: 673e4aa0-675d-11ef-bcd4-c7b262605968 2024-08-31T05:55:10.908 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-08-31T05:55:10.908 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-08-31T05:55:10.908 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout services: 2024-08-31T05:55:10.909 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.619308s) 2024-08-31T05:55:10.909 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-08-31T05:55:10.909 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-08-31T05:55:10.909 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-08-31T05:55:10.909 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout data: 2024-08-31T05:55:10.909 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-08-31T05:55:10.909 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-08-31T05:55:10.909 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-08-31T05:55:10.909 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout pgs: 2024-08-31T05:55:10.909 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-08-31T05:55:10.909 INFO:teuthology.orchestra.run.smithi016.stdout:mon is available 2024-08-31T05:55:10.909 INFO:teuthology.orchestra.run.smithi016.stdout:Assimilating anything we can from ceph.conf... 2024-08-31T05:55:11.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:11 smithi016 ceph-mon[16832]: from='client.? 172.21.15.16:0/1730170880' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-08-31T05:55:11.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:11 smithi016 ceph-mon[16832]: from='client.? 172.21.15.16:0/1730170880' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-08-31T05:55:12.064 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-08-31T05:55:12.064 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [global] 2024-08-31T05:55:12.064 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout fsid = 673e4aa0-675d-11ef-bcd4-c7b262605968 2024-08-31T05:55:12.065 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-08-31T05:55:12.065 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.16:3300,v1:172.21.15.16:6789] 2024-08-31T05:55:12.065 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-08-31T05:55:12.065 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-08-31T05:55:12.065 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-08-31T05:55:12.065 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-08-31T05:55:12.065 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-08-31T05:55:12.065 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [mgr] 2024-08-31T05:55:12.065 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-08-31T05:55:12.065 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-08-31T05:55:12.065 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [osd] 2024-08-31T05:55:12.065 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-08-31T05:55:12.065 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-08-31T05:55:12.065 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-08-31T05:55:12.066 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-08-31T05:55:12.066 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-08-31T05:55:12.066 INFO:teuthology.orchestra.run.smithi016.stdout:Generating new minimal ceph.conf... 2024-08-31T05:55:12.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:12 smithi016 ceph-mon[16832]: from='client.? 172.21.15.16:0/1807853799' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:55:13.310 INFO:teuthology.orchestra.run.smithi016.stdout:Restarting the monitor... 2024-08-31T05:55:13.654 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:13 smithi016 systemd[1]: Stopping Ceph mon.a for 673e4aa0-675d-11ef-bcd4-c7b262605968... 2024-08-31T05:55:13.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:13 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mon-a[16828]: 2024-08-31T05:55:13.651+0000 7fbda7a00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T05:55:13.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:13 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mon-a[16828]: 2024-08-31T05:55:13.651+0000 7fbda7a00640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-08-31T05:55:13.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:13 smithi016 podman[17071]: 2024-08-31 05:55:13.84637676 +0000 UTC m=+0.311837843 container died a088bba0ce12230837d0feacdd3d1abad579a04b9c91dd8160fc8d37faddf3af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mon-a, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-31T05:55:14.408 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:14 smithi016 podman[17071]: 2024-08-31 05:55:14.23144452 +0000 UTC m=+0.696905602 container cleanup a088bba0ce12230837d0feacdd3d1abad579a04b9c91dd8160fc8d37faddf3af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-08-31T05:55:14.409 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:14 smithi016 bash[17071]: ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mon-a 2024-08-31T05:55:14.660 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:14 smithi016 podman[17084]: 2024-08-31 05:55:14.40723638 +0000 UTC m=+0.547855753 container remove a088bba0ce12230837d0feacdd3d1abad579a04b9c91dd8160fc8d37faddf3af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mon-a, io.buildah.version=1.37.1, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-08-31T05:55:14.991 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:14 smithi016 systemd[1]: ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mon.a.service: Deactivated successfully. 2024-08-31T05:55:14.991 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:14 smithi016 systemd[1]: Stopped Ceph mon.a for 673e4aa0-675d-11ef-bcd4-c7b262605968. 2024-08-31T05:55:14.991 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:14 smithi016 systemd[1]: ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mon.a.service: Consumed 1.330s CPU time. 2024-08-31T05:55:14.992 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:14 smithi016 systemd[1]: Starting Ceph mon.a for 673e4aa0-675d-11ef-bcd4-c7b262605968... 2024-08-31T05:55:15.243 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:14 smithi016 podman[17176]: 2024-08-31 05:55:14.893898178 +0000 UTC m=+0.027663378 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 2024-08-31T05:55:15.244 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:14 smithi016 podman[17176]: 2024-08-31 05:55:14.991515193 +0000 UTC m=+0.125280369 container create b27650bd7483dbf9f61bd09190df2beabe54158928cd34ed3e97f34435dab432 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mon-a, io.buildah.version=1.37.1, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-08-31T05:55:15.244 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 podman[17176]: 2024-08-31 05:55:15.235778016 +0000 UTC m=+0.369543217 container init b27650bd7483dbf9f61bd09190df2beabe54158928cd34ed3e97f34435dab432 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mon-a, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , io.buildah.version=1.37.1, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-08-31T05:55:15.244 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 podman[17176]: 2024-08-31 05:55:15.24252736 +0000 UTC m=+0.376292535 container start b27650bd7483dbf9f61bd09190df2beabe54158928cd34ed3e97f34435dab432 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.1, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-08-31T05:55:15.346 INFO:teuthology.orchestra.run.smithi016.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-08-31T05:55:15.500 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 bash[17176]: b27650bd7483dbf9f61bd09190df2beabe54158928cd34ed3e97f34435dab432 2024-08-31T05:55:15.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 systemd[1]: Started Ceph mon.a for 673e4aa0-675d-11ef-bcd4-c7b262605968. 2024-08-31T05:55:15.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: set uid:gid to 167:167 (ceph:ceph) 2024-08-31T05:55:15.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: ceph version 19.3.0-4560-gd19e6118 (d19e6118f7015f0047264006cb0c873528eb3ae6) squid (dev), process ceph-mon, pid 2 2024-08-31T05:55:15.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: pidfile_write: ignore empty --pid-file 2024-08-31T05:55:15.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: load: jerasure load: lrc 2024-08-31T05:55:15.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: RocksDB version: 7.9.2 2024-08-31T05:55:15.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Git sha 0 2024-08-31T05:55:15.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Compile date 2024-08-28 10:19:06 2024-08-31T05:55:15.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: DB SUMMARY 2024-08-31T05:55:15.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: DB Session ID: 513A9OV8KH8MKE1Y2D8K 2024-08-31T05:55:15.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: CURRENT file: CURRENT 2024-08-31T05:55:15.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: IDENTITY file: IDENTITY 2024-08-31T05:55:15.502 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-08-31T05:55:15.502 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-08-31T05:55:15.502 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 94973 ; 2024-08-31T05:55:15.502 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.error_if_exists: 0 2024-08-31T05:55:15.502 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.create_if_missing: 0 2024-08-31T05:55:15.502 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.paranoid_checks: 1 2024-08-31T05:55:15.502 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.flush_verify_memtable_count: 1 2024-08-31T05:55:15.502 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-08-31T05:55:15.502 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-08-31T05:55:15.502 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.env: 0x556e752285a0 2024-08-31T05:55:15.502 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.fs: PosixFileSystem 2024-08-31T05:55:15.502 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.info_log: 0x556ea94db4a0 2024-08-31T05:55:15.502 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.max_file_opening_threads: 16 2024-08-31T05:55:15.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.statistics: (nil) 2024-08-31T05:55:15.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.use_fsync: 0 2024-08-31T05:55:15.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.max_log_file_size: 0 2024-08-31T05:55:15.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-08-31T05:55:15.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.log_file_time_to_roll: 0 2024-08-31T05:55:15.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.keep_log_file_num: 1000 2024-08-31T05:55:15.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.recycle_log_file_num: 0 2024-08-31T05:55:15.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.allow_fallocate: 1 2024-08-31T05:55:15.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.allow_mmap_reads: 0 2024-08-31T05:55:15.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.allow_mmap_writes: 0 2024-08-31T05:55:15.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.use_direct_reads: 0 2024-08-31T05:55:15.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-08-31T05:55:15.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.create_missing_column_families: 0 2024-08-31T05:55:15.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.db_log_dir: 2024-08-31T05:55:15.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.wal_dir: 2024-08-31T05:55:15.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.table_cache_numshardbits: 6 2024-08-31T05:55:15.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.WAL_ttl_seconds: 0 2024-08-31T05:55:15.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.WAL_size_limit_MB: 0 2024-08-31T05:55:15.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-08-31T05:55:15.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-08-31T05:55:15.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.is_fd_close_on_exec: 1 2024-08-31T05:55:15.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.advise_random_on_open: 1 2024-08-31T05:55:15.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.db_write_buffer_size: 0 2024-08-31T05:55:15.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.write_buffer_manager: 0x556ea94df5e0 2024-08-31T05:55:15.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-08-31T05:55:15.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-08-31T05:55:15.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.use_adaptive_mutex: 0 2024-08-31T05:55:15.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.rate_limiter: (nil) 2024-08-31T05:55:15.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-08-31T05:55:15.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.wal_recovery_mode: 2 2024-08-31T05:55:15.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.enable_thread_tracking: 0 2024-08-31T05:55:15.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.enable_pipelined_write: 0 2024-08-31T05:55:15.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.unordered_write: 0 2024-08-31T05:55:15.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-08-31T05:55:15.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-08-31T05:55:15.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-08-31T05:55:15.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-08-31T05:55:15.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.row_cache: None 2024-08-31T05:55:15.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.wal_filter: None 2024-08-31T05:55:15.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-08-31T05:55:15.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.allow_ingest_behind: 0 2024-08-31T05:55:15.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.two_write_queues: 0 2024-08-31T05:55:15.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.manual_wal_flush: 0 2024-08-31T05:55:15.506 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.wal_compression: 0 2024-08-31T05:55:15.506 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.atomic_flush: 0 2024-08-31T05:55:15.506 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-08-31T05:55:15.506 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.persist_stats_to_disk: 0 2024-08-31T05:55:15.506 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.write_dbid_to_manifest: 0 2024-08-31T05:55:15.506 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.log_readahead_size: 0 2024-08-31T05:55:15.506 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-08-31T05:55:15.506 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.best_efforts_recovery: 0 2024-08-31T05:55:15.506 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-08-31T05:55:15.506 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-08-31T05:55:15.506 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.allow_data_in_errors: 0 2024-08-31T05:55:15.507 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.db_host_id: __hostname__ 2024-08-31T05:55:15.508 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.enforce_single_del_contracts: true 2024-08-31T05:55:15.508 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.max_background_jobs: 2 2024-08-31T05:55:15.508 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.max_background_compactions: -1 2024-08-31T05:55:15.508 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.max_subcompactions: 1 2024-08-31T05:55:15.508 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-08-31T05:55:15.508 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-08-31T05:55:15.508 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.delayed_write_rate : 16777216 2024-08-31T05:55:15.508 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.max_total_wal_size: 0 2024-08-31T05:55:15.508 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-08-31T05:55:15.508 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.stats_dump_period_sec: 600 2024-08-31T05:55:15.508 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.stats_persist_period_sec: 600 2024-08-31T05:55:15.508 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-08-31T05:55:15.509 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.max_open_files: -1 2024-08-31T05:55:15.509 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.bytes_per_sync: 0 2024-08-31T05:55:15.509 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.wal_bytes_per_sync: 0 2024-08-31T05:55:15.509 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.strict_bytes_per_sync: 0 2024-08-31T05:55:15.509 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.compaction_readahead_size: 0 2024-08-31T05:55:15.509 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.max_background_flushes: -1 2024-08-31T05:55:15.509 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Compression algorithms supported: 2024-08-31T05:55:15.509 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: kZSTD supported: 0 2024-08-31T05:55:15.509 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: kXpressCompression supported: 0 2024-08-31T05:55:15.509 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: kBZip2Compression supported: 0 2024-08-31T05:55:15.509 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-08-31T05:55:15.509 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: kLZ4Compression supported: 1 2024-08-31T05:55:15.509 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: kZlibCompression supported: 1 2024-08-31T05:55:15.510 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: kLZ4HCCompression supported: 1 2024-08-31T05:55:15.510 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: kSnappyCompression supported: 1 2024-08-31T05:55:15.510 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Fast CRC32 supported: Supported on x86 2024-08-31T05:55:15.510 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: DMutex implementation: pthread_mutex_t 2024-08-31T05:55:15.510 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-08-31T05:55:15.510 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-08-31T05:55:15.510 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-08-31T05:55:15.510 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.merge_operator: 2024-08-31T05:55:15.510 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.compaction_filter: None 2024-08-31T05:55:15.510 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.compaction_filter_factory: None 2024-08-31T05:55:15.510 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.sst_partitioner_factory: None 2024-08-31T05:55:15.510 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.memtable_factory: SkipListFactory 2024-08-31T05:55:15.510 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.table_factory: BlockBasedTable 2024-08-31T05:55:15.511 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556ea94db0e0) 2024-08-31T05:55:15.511 INFO:journalctl@ceph.mon.a.smithi016.stdout: cache_index_and_filter_blocks: 1 2024-08-31T05:55:15.511 INFO:journalctl@ceph.mon.a.smithi016.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-08-31T05:55:15.511 INFO:journalctl@ceph.mon.a.smithi016.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-08-31T05:55:15.511 INFO:journalctl@ceph.mon.a.smithi016.stdout: pin_top_level_index_and_filter: 1 2024-08-31T05:55:15.511 INFO:journalctl@ceph.mon.a.smithi016.stdout: index_type: 0 2024-08-31T05:55:15.511 INFO:journalctl@ceph.mon.a.smithi016.stdout: data_block_index_type: 0 2024-08-31T05:55:15.511 INFO:journalctl@ceph.mon.a.smithi016.stdout: index_shortening: 1 2024-08-31T05:55:15.511 INFO:journalctl@ceph.mon.a.smithi016.stdout: data_block_hash_table_util_ratio: 0.750000 2024-08-31T05:55:15.511 INFO:journalctl@ceph.mon.a.smithi016.stdout: checksum: 4 2024-08-31T05:55:15.511 INFO:journalctl@ceph.mon.a.smithi016.stdout: no_block_cache: 0 2024-08-31T05:55:15.511 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_cache: 0x556ea549e850 2024-08-31T05:55:15.511 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_cache_name: BinnedLRUCache 2024-08-31T05:55:15.512 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_cache_options: 2024-08-31T05:55:15.512 INFO:journalctl@ceph.mon.a.smithi016.stdout: capacity : 536870912 2024-08-31T05:55:15.512 INFO:journalctl@ceph.mon.a.smithi016.stdout: num_shard_bits : 4 2024-08-31T05:55:15.512 INFO:journalctl@ceph.mon.a.smithi016.stdout: strict_capacity_limit : 0 2024-08-31T05:55:15.512 INFO:journalctl@ceph.mon.a.smithi016.stdout: high_pri_pool_ratio: 0.000 2024-08-31T05:55:15.512 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_cache_compressed: (nil) 2024-08-31T05:55:15.512 INFO:journalctl@ceph.mon.a.smithi016.stdout: persistent_cache: (nil) 2024-08-31T05:55:15.512 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_size: 4096 2024-08-31T05:55:15.512 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_size_deviation: 10 2024-08-31T05:55:15.512 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_restart_interval: 16 2024-08-31T05:55:15.512 INFO:journalctl@ceph.mon.a.smithi016.stdout: index_block_restart_interval: 1 2024-08-31T05:55:15.512 INFO:journalctl@ceph.mon.a.smithi016.stdout: metadata_block_size: 4096 2024-08-31T05:55:15.512 INFO:journalctl@ceph.mon.a.smithi016.stdout: partition_filters: 0 2024-08-31T05:55:15.512 INFO:journalctl@ceph.mon.a.smithi016.stdout: use_delta_encoding: 1 2024-08-31T05:55:15.513 INFO:journalctl@ceph.mon.a.smithi016.stdout: filter_policy: bloomfilter 2024-08-31T05:55:15.513 INFO:journalctl@ceph.mon.a.smithi016.stdout: whole_key_filtering: 1 2024-08-31T05:55:15.513 INFO:journalctl@ceph.mon.a.smithi016.stdout: verify_compression: 0 2024-08-31T05:55:15.513 INFO:journalctl@ceph.mon.a.smithi016.stdout: read_amp_bytes_per_bit: 0 2024-08-31T05:55:15.513 INFO:journalctl@ceph.mon.a.smithi016.stdout: format_version: 5 2024-08-31T05:55:15.513 INFO:journalctl@ceph.mon.a.smithi016.stdout: enable_index_compression: 1 2024-08-31T05:55:15.513 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_align: 0 2024-08-31T05:55:15.513 INFO:journalctl@ceph.mon.a.smithi016.stdout: max_auto_readahead_size: 262144 2024-08-31T05:55:15.513 INFO:journalctl@ceph.mon.a.smithi016.stdout: prepopulate_block_cache: 0 2024-08-31T05:55:15.513 INFO:journalctl@ceph.mon.a.smithi016.stdout: initial_auto_readahead_size: 8192 2024-08-31T05:55:15.513 INFO:journalctl@ceph.mon.a.smithi016.stdout: num_file_reads_for_auto_readahead: 2 2024-08-31T05:55:15.513 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.write_buffer_size: 33554432 2024-08-31T05:55:15.513 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.max_write_buffer_number: 2 2024-08-31T05:55:15.514 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.compression: NoCompression 2024-08-31T05:55:15.514 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.bottommost_compression: Disabled 2024-08-31T05:55:15.514 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.prefix_extractor: nullptr 2024-08-31T05:55:15.514 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-08-31T05:55:15.514 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.num_levels: 7 2024-08-31T05:55:15.514 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-08-31T05:55:15.514 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-08-31T05:55:15.514 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-08-31T05:55:15.514 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-08-31T05:55:15.514 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-08-31T05:55:15.514 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-08-31T05:55:15.514 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-08-31T05:55:15.514 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-08-31T05:55:15.514 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-08-31T05:55:15.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-08-31T05:55:15.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-08-31T05:55:15.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-08-31T05:55:15.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.compression_opts.window_bits: -14 2024-08-31T05:55:15.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.compression_opts.level: 32767 2024-08-31T05:55:15.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.compression_opts.strategy: 0 2024-08-31T05:55:15.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-08-31T05:55:15.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-08-31T05:55:15.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-08-31T05:55:15.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-08-31T05:55:15.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.compression_opts.enabled: false 2024-08-31T05:55:15.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-08-31T05:55:15.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-08-31T05:55:15.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-08-31T05:55:15.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-08-31T05:55:15.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.target_file_size_base: 67108864 2024-08-31T05:55:15.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.target_file_size_multiplier: 1 2024-08-31T05:55:15.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-08-31T05:55:15.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-08-31T05:55:15.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-08-31T05:55:15.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-08-31T05:55:15.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-08-31T05:55:15.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-08-31T05:55:15.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-08-31T05:55:15.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-08-31T05:55:15.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-08-31T05:55:15.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-08-31T05:55:15.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-08-31T05:55:15.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-08-31T05:55:15.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-08-31T05:55:15.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.arena_block_size: 1048576 2024-08-31T05:55:15.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-08-31T05:55:15.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-08-31T05:55:15.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.disable_auto_compactions: 0 2024-08-31T05:55:15.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-08-31T05:55:15.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-08-31T05:55:15.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-08-31T05:55:15.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-08-31T05:55:15.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-08-31T05:55:15.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-08-31T05:55:15.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-08-31T05:55:15.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-08-31T05:55:15.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-08-31T05:55:15.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-08-31T05:55:15.518 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-08-31T05:55:15.518 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.inplace_update_support: 0 2024-08-31T05:55:15.518 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.inplace_update_num_locks: 10000 2024-08-31T05:55:15.518 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-08-31T05:55:15.518 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-08-31T05:55:15.518 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.memtable_huge_page_size: 0 2024-08-31T05:55:15.518 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.bloom_locality: 0 2024-08-31T05:55:15.518 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.max_successive_merges: 0 2024-08-31T05:55:15.518 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.optimize_filters_for_hits: 0 2024-08-31T05:55:15.518 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.paranoid_file_checks: 0 2024-08-31T05:55:15.518 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.force_consistency_checks: 1 2024-08-31T05:55:15.518 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.report_bg_io_stats: 0 2024-08-31T05:55:15.518 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.ttl: 2592000 2024-08-31T05:55:15.518 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.periodic_compaction_seconds: 0 2024-08-31T05:55:15.519 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-08-31T05:55:15.519 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-08-31T05:55:15.519 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.enable_blob_files: false 2024-08-31T05:55:15.519 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.min_blob_size: 0 2024-08-31T05:55:15.519 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.blob_file_size: 268435456 2024-08-31T05:55:15.519 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.blob_compression_type: NoCompression 2024-08-31T05:55:15.519 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.enable_blob_garbage_collection: false 2024-08-31T05:55:15.519 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-08-31T05:55:15.519 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-08-31T05:55:15.519 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-08-31T05:55:15.519 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.blob_file_starting_level: 0 2024-08-31T05:55:15.519 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-08-31T05:55:15.519 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-08-31T05:55:15.519 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-08-31T05:55:15.519 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 6299af54-b1be-4b54-90b6-e81c8deeb7cd 2024-08-31T05:55:15.520 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725083715353415, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-08-31T05:55:15.520 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-08-31T05:55:15.520 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725083715356717, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 91124, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 282, "table_properties": {"data_size": 89172, "index_size": 272, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11619, "raw_average_key_size": 48, "raw_value_size": 82646, "raw_average_value_size": 342, "num_data_blocks": 12, "num_entries": 241, "num_filter_entries": 241, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1725083715, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "6299af54-b1be-4b54-90b6-e81c8deeb7cd", "db_session_id": "513A9OV8KH8MKE1Y2D8K", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-08-31T05:55:15.520 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725083715357048, "job": 1, "event": "recovery_finished"} 2024-08-31T05:55:15.520 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-08-31T05:55:15.520 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-08-31T05:55:15.520 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556ea9500a80 2024-08-31T05:55:15.520 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: DB pointer 0x556eaa2a2000 2024-08-31T05:55:15.520 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-08-31T05:55:15.520 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-08-31T05:55:15.520 INFO:journalctl@ceph.mon.a.smithi016.stdout: ** DB Stats ** 2024-08-31T05:55:15.520 INFO:journalctl@ceph.mon.a.smithi016.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-31T05:55:15.520 INFO:journalctl@ceph.mon.a.smithi016.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-08-31T05:55:15.520 INFO:journalctl@ceph.mon.a.smithi016.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-31T05:55:15.520 INFO:journalctl@ceph.mon.a.smithi016.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-31T05:55:15.521 INFO:journalctl@ceph.mon.a.smithi016.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-08-31T05:55:15.521 INFO:journalctl@ceph.mon.a.smithi016.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-31T05:55:15.521 INFO:journalctl@ceph.mon.a.smithi016.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-31T05:55:15.521 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-08-31T05:55:15.521 INFO:journalctl@ceph.mon.a.smithi016.stdout: ** Compaction Stats [default] ** 2024-08-31T05:55:15.521 INFO:journalctl@ceph.mon.a.smithi016.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-31T05:55:15.521 INFO:journalctl@ceph.mon.a.smithi016.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-08-31T05:55:15.521 INFO:journalctl@ceph.mon.a.smithi016.stdout: L0 2/0 90.85 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 40.6 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-08-31T05:55:15.521 INFO:journalctl@ceph.mon.a.smithi016.stdout: Sum 2/0 90.85 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 40.6 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-08-31T05:55:15.521 INFO:journalctl@ceph.mon.a.smithi016.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 40.6 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-08-31T05:55:15.521 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-08-31T05:55:15.521 INFO:journalctl@ceph.mon.a.smithi016.stdout: ** Compaction Stats [default] ** 2024-08-31T05:55:15.521 INFO:journalctl@ceph.mon.a.smithi016.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-31T05:55:15.521 INFO:journalctl@ceph.mon.a.smithi016.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-08-31T05:55:15.521 INFO:journalctl@ceph.mon.a.smithi016.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 40.6 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-08-31T05:55:15.522 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-08-31T05:55:15.522 INFO:journalctl@ceph.mon.a.smithi016.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-08-31T05:55:15.522 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-08-31T05:55:15.522 INFO:journalctl@ceph.mon.a.smithi016.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-31T05:55:15.522 INFO:journalctl@ceph.mon.a.smithi016.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-08-31T05:55:15.522 INFO:journalctl@ceph.mon.a.smithi016.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-08-31T05:55:15.522 INFO:journalctl@ceph.mon.a.smithi016.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-08-31T05:55:15.522 INFO:journalctl@ceph.mon.a.smithi016.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-08-31T05:55:15.522 INFO:journalctl@ceph.mon.a.smithi016.stdout: AddFile(Keys): cumulative 0, interval 0 2024-08-31T05:55:15.522 INFO:journalctl@ceph.mon.a.smithi016.stdout: Cumulative compaction: 0.00 GB write, 8.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-31T05:55:15.522 INFO:journalctl@ceph.mon.a.smithi016.stdout: Interval compaction: 0.00 GB write, 8.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-31T05:55:15.522 INFO:journalctl@ceph.mon.a.smithi016.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-08-31T05:55:15.522 INFO:journalctl@ceph.mon.a.smithi016.stdout: Block cache BinnedLRUCache@0x556ea549e850#2 capacity: 512.00 MB usage: 1.31 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-08-31T05:55:15.522 INFO:journalctl@ceph.mon.a.smithi016.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.84 KB,0.000160933%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-08-31T05:55:15.522 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-08-31T05:55:15.522 INFO:journalctl@ceph.mon.a.smithi016.stdout: ** File Read Latency Histogram By Level [default] ** 2024-08-31T05:55:15.523 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: starting mon.a rank 0 at public addrs [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] at bind addrs [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 2024-08-31T05:55:15.523 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: mon.a@-1(???) e1 preinit fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 2024-08-31T05:55:15.523 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: mon.a@-1(???).mds e1 new map 2024-08-31T05:55:15.523 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: mon.a@-1(???).mds e1 print_map 2024-08-31T05:55:15.523 INFO:journalctl@ceph.mon.a.smithi016.stdout: e1 2024-08-31T05:55:15.523 INFO:journalctl@ceph.mon.a.smithi016.stdout: btime 2024-08-31T05:55:09:632201+0000 2024-08-31T05:55:15.523 INFO:journalctl@ceph.mon.a.smithi016.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-08-31T05:55:15.523 INFO:journalctl@ceph.mon.a.smithi016.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-31T05:55:15.523 INFO:journalctl@ceph.mon.a.smithi016.stdout: legacy client fscid: -1 2024-08-31T05:55:15.523 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-08-31T05:55:15.523 INFO:journalctl@ceph.mon.a.smithi016.stdout: No filesystems configured 2024-08-31T05:55:15.523 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-08-31T05:55:15.523 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-08-31T05:55:15.523 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-08-31T05:55:15.523 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-08-31T05:55:15.524 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-08-31T05:55:15.524 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: mon.a is new leader, mons a in quorum (ranks 0) 2024-08-31T05:55:15.524 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: monmap epoch 1 2024-08-31T05:55:15.524 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 2024-08-31T05:55:15.524 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: last_changed 2024-08-31T05:55:05.650830+0000 2024-08-31T05:55:15.524 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: created 2024-08-31T05:55:05.650830+0000 2024-08-31T05:55:15.524 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: min_mon_release 19 (squid) 2024-08-31T05:55:15.524 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: election_strategy: 1 2024-08-31T05:55:15.524 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-08-31T05:55:15.524 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: fsmap 2024-08-31T05:55:15.524 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: osdmap e1: 0 total, 0 up, 0 in 2024-08-31T05:55:15.524 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:15 smithi016 ceph-mon[17194]: mgrmap e1: no daemons active 2024-08-31T05:55:17.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:17 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/37996464' entity='client.admin' 2024-08-31T05:55:26.786 INFO:teuthology.orchestra.run.smithi016.stdout:Wrote config to /etc/ceph/ceph.conf 2024-08-31T05:55:26.812 INFO:teuthology.orchestra.run.smithi016.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-08-31T05:55:26.812 INFO:teuthology.orchestra.run.smithi016.stdout:Creating mgr... 2024-08-31T05:55:26.813 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying port 0.0.0.0:9283 ... 2024-08-31T05:55:26.813 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying port 0.0.0.0:8765 ... 2024-08-31T05:55:27.163 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mgr.x 2024-08-31T05:55:27.163 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to reset failed state of unit ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mgr.x.service: Unit ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mgr.x.service not loaded. 2024-08-31T05:55:27.402 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-673e4aa0-675d-11ef-bcd4-c7b262605968.target.wants/ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@.service. 2024-08-31T05:55:27.749 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:27 smithi016 systemd[1]: Starting Ceph mgr.x for 673e4aa0-675d-11ef-bcd4-c7b262605968... 2024-08-31T05:55:28.059 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:27 smithi016 podman[17406]: 2024-08-31 05:55:27.74690107 +0000 UTC m=+0.123178284 container create a9111fa496cbf6132c6a860b0a1f89996258208355acec3b4548aa7e64134eb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.1, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True) 2024-08-31T05:55:28.059 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:27 smithi016 podman[17406]: 2024-08-31 05:55:27.651202717 +0000 UTC m=+0.027479895 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 2024-08-31T05:55:28.059 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:27 smithi016 podman[17406]: 2024-08-31 05:55:27.966045923 +0000 UTC m=+0.342323097 container init a9111fa496cbf6132c6a860b0a1f89996258208355acec3b4548aa7e64134eb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611) 2024-08-31T05:55:28.059 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:27 smithi016 podman[17406]: 2024-08-31 05:55:27.972273322 +0000 UTC m=+0.348550496 container start a9111fa496cbf6132c6a860b0a1f89996258208355acec3b4548aa7e64134eb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-08-31T05:55:28.059 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:28 smithi016 bash[17406]: a9111fa496cbf6132c6a860b0a1f89996258208355acec3b4548aa7e64134eb2 2024-08-31T05:55:28.087 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-31T05:55:28.088 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-08-31T05:55:28.101 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-31T05:55:28.101 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-08-31T05:55:28.101 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-08-31T05:55:28.101 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to enable service . firewalld.service is not available 2024-08-31T05:55:28.118 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-31T05:55:28.119 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-08-31T05:55:28.133 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-31T05:55:28.133 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-08-31T05:55:28.133 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-08-31T05:55:28.134 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-08-31T05:55:28.134 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr to start... 2024-08-31T05:55:28.134 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr... 2024-08-31T05:55:28.309 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:28 smithi016 systemd[1]: Started Ceph mgr.x for 673e4aa0-675d-11ef-bcd4-c7b262605968. 2024-08-31T05:55:29.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:28 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/712209804' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-08-31T05:55:29.466 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-08-31T05:55:29.466 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-08-31T05:55:29.466 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsid": "673e4aa0-675d-11ef-bcd4-c7b262605968", 2024-08-31T05:55:29.466 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "health": { 2024-08-31T05:55:29.467 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-08-31T05:55:29.467 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-08-31T05:55:29.467 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-08-31T05:55:29.467 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-08-31T05:55:29.467 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-08-31T05:55:29.467 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-08-31T05:55:29.467 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 0 2024-08-31T05:55:29.467 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-08-31T05:55:29.467 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-08-31T05:55:29.467 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "a" 2024-08-31T05:55:29.467 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-08-31T05:55:29.467 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_age": 13, 2024-08-31T05:55:29.467 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "monmap": { 2024-08-31T05:55:29.467 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-31T05:55:29.468 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-08-31T05:55:29.468 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-08-31T05:55:29.468 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-08-31T05:55:29.468 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-08-31T05:55:29.468 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-31T05:55:29.468 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-08-31T05:55:29.468 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-08-31T05:55:29.468 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-08-31T05:55:29.468 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-08-31T05:55:29.468 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-08-31T05:55:29.468 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-08-31T05:55:29.472 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-08-31T05:55:29.472 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-08-31T05:55:29.472 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-08-31T05:55:29.472 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-08-31T05:55:29.472 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-08-31T05:55:29.472 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-08-31T05:55:29.472 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-08-31T05:55:29.472 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-08-31T05:55:29.472 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-08-31T05:55:29.472 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-08-31T05:55:29.472 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-08-31T05:55:29.473 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-08-31T05:55:29.473 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-31T05:55:29.473 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "btime": "2024-08-31T05:55:09:632201+0000", 2024-08-31T05:55:29.473 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-08-31T05:55:29.473 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-08-31T05:55:29.473 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-08-31T05:55:29.473 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-08-31T05:55:29.473 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": false, 2024-08-31T05:55:29.473 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-08-31T05:55:29.473 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modules": [ 2024-08-31T05:55:29.473 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "iostat", 2024-08-31T05:55:29.473 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "nfs", 2024-08-31T05:55:29.473 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "restful" 2024-08-31T05:55:29.474 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-08-31T05:55:29.474 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-31T05:55:29.474 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-08-31T05:55:29.474 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-08-31T05:55:29.474 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-31T05:55:29.474 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modified": "2024-08-31T05:55:09.633647+0000", 2024-08-31T05:55:29.474 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-31T05:55:29.474 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-08-31T05:55:29.474 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-08-31T05:55:29.474 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-08-31T05:55:29.474 INFO:teuthology.orchestra.run.smithi016.stdout:mgr not available, waiting (1/15)... 2024-08-31T05:55:32.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:32 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/4239428730' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-08-31T05:55:32.861 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-08-31T05:55:32.864 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-08-31T05:55:32.864 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsid": "673e4aa0-675d-11ef-bcd4-c7b262605968", 2024-08-31T05:55:32.864 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "health": { 2024-08-31T05:55:32.864 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-08-31T05:55:32.864 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-08-31T05:55:32.864 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-08-31T05:55:32.864 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-08-31T05:55:32.864 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-08-31T05:55:32.864 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-08-31T05:55:32.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 0 2024-08-31T05:55:32.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-08-31T05:55:32.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-08-31T05:55:32.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "a" 2024-08-31T05:55:32.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-08-31T05:55:32.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_age": 16, 2024-08-31T05:55:32.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "monmap": { 2024-08-31T05:55:32.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-31T05:55:32.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-08-31T05:55:32.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-08-31T05:55:32.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-08-31T05:55:32.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-08-31T05:55:32.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-31T05:55:32.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-08-31T05:55:32.866 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-08-31T05:55:32.866 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-08-31T05:55:32.866 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-08-31T05:55:32.866 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-08-31T05:55:32.866 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-08-31T05:55:32.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-08-31T05:55:32.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-08-31T05:55:32.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-08-31T05:55:32.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-08-31T05:55:32.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-08-31T05:55:32.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-08-31T05:55:32.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-08-31T05:55:32.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-08-31T05:55:32.870 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-08-31T05:55:32.870 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-08-31T05:55:32.870 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-08-31T05:55:32.870 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-08-31T05:55:32.870 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-31T05:55:32.870 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "btime": "2024-08-31T05:55:09:632201+0000", 2024-08-31T05:55:32.870 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-08-31T05:55:32.870 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-08-31T05:55:32.870 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-08-31T05:55:32.870 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-08-31T05:55:32.870 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": false, 2024-08-31T05:55:32.870 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-08-31T05:55:32.870 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modules": [ 2024-08-31T05:55:32.870 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "iostat", 2024-08-31T05:55:32.870 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "nfs", 2024-08-31T05:55:32.871 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "restful" 2024-08-31T05:55:32.871 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-08-31T05:55:32.871 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-31T05:55:32.871 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-08-31T05:55:32.871 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-08-31T05:55:32.871 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-31T05:55:32.871 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modified": "2024-08-31T05:55:09.633647+0000", 2024-08-31T05:55:32.871 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-31T05:55:32.871 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-08-31T05:55:32.871 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-08-31T05:55:32.871 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-08-31T05:55:32.871 INFO:teuthology.orchestra.run.smithi016.stdout:mgr not available, waiting (2/15)... 2024-08-31T05:55:34.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:34 smithi016 ceph-mon[17194]: Activating manager daemon x 2024-08-31T05:55:34.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:34 smithi016 ceph-mon[17194]: mgrmap e2: x(active, starting, since 0.00459634s) 2024-08-31T05:55:34.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:34 smithi016 ceph-mon[17194]: from='mgr.14100 172.21.15.16:0/4012187189' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-08-31T05:55:34.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:34 smithi016 ceph-mon[17194]: from='mgr.14100 172.21.15.16:0/4012187189' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-08-31T05:55:34.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:34 smithi016 ceph-mon[17194]: from='mgr.14100 172.21.15.16:0/4012187189' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-08-31T05:55:34.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:34 smithi016 ceph-mon[17194]: from='mgr.14100 172.21.15.16:0/4012187189' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-31T05:55:34.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:34 smithi016 ceph-mon[17194]: from='mgr.14100 172.21.15.16:0/4012187189' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-08-31T05:55:34.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:34 smithi016 ceph-mon[17194]: Manager daemon x is now available 2024-08-31T05:55:34.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:34 smithi016 ceph-mon[17194]: from='mgr.14100 172.21.15.16:0/4012187189' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-08-31T05:55:34.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:34 smithi016 ceph-mon[17194]: from='mgr.14100 172.21.15.16:0/4012187189' entity='mgr.x' 2024-08-31T05:55:34.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:34 smithi016 ceph-mon[17194]: from='mgr.14100 172.21.15.16:0/4012187189' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-08-31T05:55:34.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:34 smithi016 ceph-mon[17194]: from='mgr.14100 172.21.15.16:0/4012187189' entity='mgr.x' 2024-08-31T05:55:34.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:34 smithi016 ceph-mon[17194]: from='mgr.14100 172.21.15.16:0/4012187189' entity='mgr.x' 2024-08-31T05:55:36.413 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-08-31T05:55:36.413 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-08-31T05:55:36.414 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsid": "673e4aa0-675d-11ef-bcd4-c7b262605968", 2024-08-31T05:55:36.414 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "health": { 2024-08-31T05:55:36.414 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-08-31T05:55:36.414 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-08-31T05:55:36.414 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-08-31T05:55:36.414 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-08-31T05:55:36.414 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-08-31T05:55:36.414 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-08-31T05:55:36.414 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 0 2024-08-31T05:55:36.414 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-08-31T05:55:36.414 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-08-31T05:55:36.414 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "a" 2024-08-31T05:55:36.414 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-08-31T05:55:36.414 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_age": 20, 2024-08-31T05:55:36.415 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "monmap": { 2024-08-31T05:55:36.415 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-31T05:55:36.415 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-08-31T05:55:36.415 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-08-31T05:55:36.415 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-08-31T05:55:36.415 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-08-31T05:55:36.415 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-31T05:55:36.415 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-08-31T05:55:36.415 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-08-31T05:55:36.415 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-08-31T05:55:36.415 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-08-31T05:55:36.415 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-08-31T05:55:36.415 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-08-31T05:55:36.415 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-08-31T05:55:36.416 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-08-31T05:55:36.416 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-08-31T05:55:36.416 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-08-31T05:55:36.416 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-08-31T05:55:36.416 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-08-31T05:55:36.416 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-08-31T05:55:36.416 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-08-31T05:55:36.416 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-08-31T05:55:36.416 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-08-31T05:55:36.416 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-08-31T05:55:36.416 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-08-31T05:55:36.416 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-31T05:55:36.416 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "btime": "2024-08-31T05:55:09:632201+0000", 2024-08-31T05:55:36.416 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-08-31T05:55:36.416 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-08-31T05:55:36.417 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-08-31T05:55:36.417 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-08-31T05:55:36.417 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": true, 2024-08-31T05:55:36.417 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-08-31T05:55:36.417 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modules": [ 2024-08-31T05:55:36.417 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "iostat", 2024-08-31T05:55:36.417 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "nfs", 2024-08-31T05:55:36.417 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "restful" 2024-08-31T05:55:36.417 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-08-31T05:55:36.417 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-31T05:55:36.417 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-08-31T05:55:36.417 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-08-31T05:55:36.417 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-31T05:55:36.417 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modified": "2024-08-31T05:55:09.633647+0000", 2024-08-31T05:55:36.418 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-31T05:55:36.418 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-08-31T05:55:36.418 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-08-31T05:55:36.418 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-08-31T05:55:36.418 INFO:teuthology.orchestra.run.smithi016.stdout:mgr is available 2024-08-31T05:55:36.657 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:36 smithi016 ceph-mon[17194]: mgrmap e3: x(active, since 1.00897s) 2024-08-31T05:55:36.657 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:36 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/3375624716' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-08-31T05:55:37.843 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:37 smithi016 ceph-mon[17194]: mgrmap e4: x(active, since 2s) 2024-08-31T05:55:37.843 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:37 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/3987495355' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-08-31T05:55:37.851 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-08-31T05:55:37.851 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [global] 2024-08-31T05:55:37.852 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout fsid = 673e4aa0-675d-11ef-bcd4-c7b262605968 2024-08-31T05:55:37.852 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-08-31T05:55:37.852 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.16:3300,v1:172.21.15.16:6789] 2024-08-31T05:55:37.852 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-08-31T05:55:37.852 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-08-31T05:55:37.852 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-08-31T05:55:37.852 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-08-31T05:55:37.852 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-08-31T05:55:37.852 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [mgr] 2024-08-31T05:55:37.852 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-08-31T05:55:37.852 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-08-31T05:55:37.852 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [osd] 2024-08-31T05:55:37.852 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-08-31T05:55:37.852 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-08-31T05:55:37.852 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-08-31T05:55:37.853 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-08-31T05:55:37.853 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-08-31T05:55:37.853 INFO:teuthology.orchestra.run.smithi016.stdout:Enabling cephadm module... 2024-08-31T05:55:39.013 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:38 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/1888787950' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2024-08-31T05:55:39.013 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:38 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: ignoring --setuser ceph since I am not root 2024-08-31T05:55:39.014 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:38 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: ignoring --setgroup ceph since I am not root 2024-08-31T05:55:39.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:39 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/1888787950' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-08-31T05:55:39.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:39 smithi016 ceph-mon[17194]: mgrmap e5: x(active, since 4s) 2024-08-31T05:55:40.847 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:40 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/867094692' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-08-31T05:55:40.853 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-08-31T05:55:40.853 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-08-31T05:55:40.853 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": true, 2024-08-31T05:55:40.854 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-08-31T05:55:40.854 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-08-31T05:55:40.854 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-08-31T05:55:40.854 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for the mgr to restart... 2024-08-31T05:55:40.854 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr epoch 5... 2024-08-31T05:55:45.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:45 smithi016 ceph-mon[17194]: Active manager daemon x restarted 2024-08-31T05:55:45.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:45 smithi016 ceph-mon[17194]: Activating manager daemon x 2024-08-31T05:55:45.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:45 smithi016 ceph-mon[17194]: osdmap e2: 0 total, 0 up, 0 in 2024-08-31T05:55:45.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:45 smithi016 ceph-mon[17194]: mgrmap e6: x(active, starting, since 0.00398703s) 2024-08-31T05:55:45.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:45 smithi016 ceph-mon[17194]: from='mgr.14118 172.21.15.16:0/2588448908' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-31T05:55:45.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:45 smithi016 ceph-mon[17194]: from='mgr.14118 172.21.15.16:0/2588448908' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-08-31T05:55:45.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:45 smithi016 ceph-mon[17194]: from='mgr.14118 172.21.15.16:0/2588448908' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-08-31T05:55:45.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:45 smithi016 ceph-mon[17194]: from='mgr.14118 172.21.15.16:0/2588448908' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-08-31T05:55:45.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:45 smithi016 ceph-mon[17194]: from='mgr.14118 172.21.15.16:0/2588448908' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-08-31T05:55:45.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:45 smithi016 ceph-mon[17194]: Manager daemon x is now available 2024-08-31T05:55:46.720 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-08-31T05:55:46.720 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-08-31T05:55:46.720 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "initialized": true 2024-08-31T05:55:46.720 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-08-31T05:55:46.720 INFO:teuthology.orchestra.run.smithi016.stdout:mgr epoch 5 is available 2024-08-31T05:55:46.720 INFO:teuthology.orchestra.run.smithi016.stdout:Setting orchestrator backend to cephadm... 2024-08-31T05:55:46.984 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:46 smithi016 ceph-mon[17194]: from='mgr.14118 172.21.15.16:0/2588448908' entity='mgr.x' 2024-08-31T05:55:46.984 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:46 smithi016 ceph-mon[17194]: from='mgr.14118 172.21.15.16:0/2588448908' entity='mgr.x' 2024-08-31T05:55:46.985 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:46 smithi016 ceph-mon[17194]: Found migration_current of "None". Setting to last migration. 2024-08-31T05:55:46.985 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:46 smithi016 ceph-mon[17194]: from='mgr.14118 172.21.15.16:0/2588448908' entity='mgr.x' 2024-08-31T05:55:46.985 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:46 smithi016 ceph-mon[17194]: from='mgr.14118 172.21.15.16:0/2588448908' entity='mgr.x' 2024-08-31T05:55:46.985 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:46 smithi016 ceph-mon[17194]: from='mgr.14118 172.21.15.16:0/2588448908' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:55:46.985 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:46 smithi016 ceph-mon[17194]: from='mgr.14118 172.21.15.16:0/2588448908' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:55:46.985 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:46 smithi016 ceph-mon[17194]: from='mgr.14118 172.21.15.16:0/2588448908' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-08-31T05:55:46.985 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:46 smithi016 ceph-mon[17194]: from='mgr.14118 172.21.15.16:0/2588448908' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-08-31T05:55:46.985 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:46 smithi016 ceph-mon[17194]: mgrmap e7: x(active, since 1.00771s) 2024-08-31T05:55:47.810 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:47 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: 2024-08-31T05:55:47.615+0000 7f34e1a00640 -1 log_channel(cephadm) log [ERR] : [31/Aug/2024:05:55:47] ENGINE Error in HTTPServer.serve 2024-08-31T05:55:47.810 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:47 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: Traceback (most recent call last): 2024-08-31T05:55:47.810 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:47 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-31T05:55:47.810 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:47 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: self._connections.run(self.expiration_interval) 2024-08-31T05:55:47.810 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:47 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-31T05:55:47.811 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:47 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: self._run(expiration_interval) 2024-08-31T05:55:47.811 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:47 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-31T05:55:47.811 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:47 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: new_conn = self._from_server_socket(self.server.socket) 2024-08-31T05:55:47.811 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:47 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-31T05:55:47.811 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:47 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-31T05:55:47.811 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:47 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-31T05:55:47.811 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:47 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: s = self.context.wrap_socket( 2024-08-31T05:55:47.811 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:47 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-31T05:55:47.811 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:47 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: return self.sslsocket_class._create( 2024-08-31T05:55:47.811 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:47 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-31T05:55:47.811 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:47 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: self.do_handshake() 2024-08-31T05:55:47.811 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:47 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-31T05:55:47.811 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:47 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: self._sslobj.do_handshake() 2024-08-31T05:55:47.811 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:47 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-31T05:55:47.812 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:47 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: 2024-08-31T05:55:48.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:47 smithi016 ceph-mon[17194]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-08-31T05:55:48.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:47 smithi016 ceph-mon[17194]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-08-31T05:55:48.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:47 smithi016 ceph-mon[17194]: from='mgr.14118 172.21.15.16:0/2588448908' entity='mgr.x' 2024-08-31T05:55:48.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:47 smithi016 ceph-mon[17194]: from='mgr.14118 172.21.15.16:0/2588448908' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:55:48.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:47 smithi016 ceph-mon[17194]: from='mgr.14118 172.21.15.16:0/2588448908' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:55:48.969 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:48 smithi016 ceph-mon[17194]: [31/Aug/2024:05:55:47] ENGINE Bus STARTING 2024-08-31T05:55:48.969 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:48 smithi016 ceph-mon[17194]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T05:55:48.969 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:48 smithi016 ceph-mon[17194]: [31/Aug/2024:05:55:47] ENGINE Serving on https://172.21.15.16:7150 2024-08-31T05:55:48.969 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:48 smithi016 ceph-mon[17194]: [31/Aug/2024:05:55:47] ENGINE Error in HTTPServer.serve 2024-08-31T05:55:48.969 INFO:journalctl@ceph.mon.a.smithi016.stdout: Traceback (most recent call last): 2024-08-31T05:55:48.969 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-31T05:55:48.969 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._connections.run(self.expiration_interval) 2024-08-31T05:55:48.969 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-31T05:55:48.970 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._run(expiration_interval) 2024-08-31T05:55:48.970 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-31T05:55:48.970 INFO:journalctl@ceph.mon.a.smithi016.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-31T05:55:48.970 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-31T05:55:48.970 INFO:journalctl@ceph.mon.a.smithi016.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-31T05:55:48.970 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-31T05:55:48.970 INFO:journalctl@ceph.mon.a.smithi016.stdout: s = self.context.wrap_socket( 2024-08-31T05:55:48.970 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-31T05:55:48.970 INFO:journalctl@ceph.mon.a.smithi016.stdout: return self.sslsocket_class._create( 2024-08-31T05:55:48.970 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-31T05:55:48.970 INFO:journalctl@ceph.mon.a.smithi016.stdout: self.do_handshake() 2024-08-31T05:55:48.970 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-31T05:55:48.970 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._sslobj.do_handshake() 2024-08-31T05:55:48.970 INFO:journalctl@ceph.mon.a.smithi016.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-31T05:55:48.970 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:48 smithi016 ceph-mon[17194]: [31/Aug/2024:05:55:47] ENGINE Serving on http://172.21.15.16:8765 2024-08-31T05:55:48.970 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:48 smithi016 ceph-mon[17194]: [31/Aug/2024:05:55:47] ENGINE Bus STARTED 2024-08-31T05:55:48.971 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:48 smithi016 ceph-mon[17194]: mgrmap e8: x(active, since 2s) 2024-08-31T05:55:49.815 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout value unchanged 2024-08-31T05:55:49.815 INFO:teuthology.orchestra.run.smithi016.stdout:Generating ssh key... 2024-08-31T05:55:50.081 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:49 smithi016 ceph-mon[17194]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T05:55:51.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:50 smithi016 ceph-mon[17194]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T05:55:51.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:50 smithi016 ceph-mon[17194]: Generating ssh key... 2024-08-31T05:55:51.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:50 smithi016 ceph-mon[17194]: from='mgr.14118 172.21.15.16:0/2588448908' entity='mgr.x' 2024-08-31T05:55:51.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:50 smithi016 ceph-mon[17194]: from='mgr.14118 172.21.15.16:0/2588448908' entity='mgr.x' 2024-08-31T05:55:51.163 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:50 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: Generating public/private rsa key pair. 2024-08-31T05:55:51.163 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:50 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: Your identification has been saved in /tmp/tmp9apftwm6/key 2024-08-31T05:55:51.163 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:50 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: Your public key has been saved in /tmp/tmp9apftwm6/key.pub 2024-08-31T05:55:51.163 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:50 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: The key fingerprint is: 2024-08-31T05:55:51.163 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:50 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: SHA256:NIe25bApjRlv0XV+DNExrZIHrxR9e51Fvektl40hT00 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968 2024-08-31T05:55:51.163 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:50 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: The key's randomart image is: 2024-08-31T05:55:51.163 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:50 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: +---[RSA 3072]----+ 2024-08-31T05:55:51.163 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:50 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: | ..+*=| 2024-08-31T05:55:51.164 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:50 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: | o .oo.oE| 2024-08-31T05:55:51.164 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:50 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: | . B + =.=X| 2024-08-31T05:55:51.164 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:50 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: | O @ = **+| 2024-08-31T05:55:51.164 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:50 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: | + S .. B.o=| 2024-08-31T05:55:51.164 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:50 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: | o . +o+| 2024-08-31T05:55:51.164 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:50 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: | o | 2024-08-31T05:55:51.164 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:50 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: | | 2024-08-31T05:55:51.164 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:50 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: | | 2024-08-31T05:55:51.164 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:55:50 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: +----[SHA256]-----+ 2024-08-31T05:55:52.909 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjH1D+jnpR8sqM7Cs9bUaWxaLvH4cf2OQqsggcZ8yc+qmmHSRiPy6MKjVf/n9NmU9IMdnCDXe8nfD7eyF4tfhL+Ci0k5KwbmTBKXKnmFz5xenRvnjGfG6sRQymAu+HMq7aEBqnkr7nJtpDrWEe5fN/akFiFgZE2HusrooX6J60+EtVca0S6wLR3sbnNG1G5X9vM2PkY4omo5CzG0/sVvaRcHwsoSrYDplADIKjoYKwe2Bm0dXRaj6JX2aJKWaTA3WZWDW7ruYO+QV3ote94wisAiH6FVS2CumA9vNUPoC7ziTRhunaSJODXxa65L5l2+5R1OeH0gR48q/oVa1jSa9xMkUEr0WDbGa47mNmOjrxYVUWSQJfz6KYVD6mKw3Wytqh4tLUGY15URTjLqHPaUhmAgPfhAnMg4BIPUT1w+p9+zM9Z8Io2ttK5EwpcUBLxTwZtNxGzArF+BsRxg2bK3aFEy7M901GgZmdAQbEfmlFbIZUjPQm7/UsN7jDV/zEhmc= ceph-673e4aa0-675d-11ef-bcd4-c7b262605968 2024-08-31T05:55:52.910 INFO:teuthology.orchestra.run.smithi016.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-08-31T05:55:52.910 INFO:teuthology.orchestra.run.smithi016.stdout:Adding key to root@localhost authorized_keys... 2024-08-31T05:55:52.932 INFO:teuthology.orchestra.run.smithi016.stdout:Adding host smithi016... 2024-08-31T05:55:53.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:52 smithi016 ceph-mon[17194]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T05:55:54.181 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:53 smithi016 ceph-mon[17194]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi016", "addr": "172.21.15.16", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T05:55:56.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:55 smithi016 ceph-mon[17194]: Deploying cephadm binary to smithi016 2024-08-31T05:55:59.025 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Added host 'smithi016' with addr '172.21.15.16' 2024-08-31T05:55:59.025 INFO:teuthology.orchestra.run.smithi016.stdout:Deploying unmanaged mon service... 2024-08-31T05:55:59.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:59 smithi016 ceph-mon[17194]: from='mgr.14118 172.21.15.16:0/2588448908' entity='mgr.x' 2024-08-31T05:55:59.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:59 smithi016 ceph-mon[17194]: Added host smithi016 2024-08-31T05:55:59.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:55:59 smithi016 ceph-mon[17194]: from='mgr.14118 172.21.15.16:0/2588448908' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:56:00.750 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-08-31T05:56:00.750 INFO:teuthology.orchestra.run.smithi016.stdout:Deploying unmanaged mgr service... 2024-08-31T05:56:00.890 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:00 smithi016 ceph-mon[17194]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T05:56:00.891 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:00 smithi016 ceph-mon[17194]: Saving service mon spec with placement count:5 2024-08-31T05:56:00.891 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:00 smithi016 ceph-mon[17194]: from='mgr.14118 172.21.15.16:0/2588448908' entity='mgr.x' 2024-08-31T05:56:02.999 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:02 smithi016 ceph-mon[17194]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T05:56:03.000 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:02 smithi016 ceph-mon[17194]: Saving service mgr spec with placement count:2 2024-08-31T05:56:03.000 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:02 smithi016 ceph-mon[17194]: from='mgr.14118 172.21.15.16:0/2588448908' entity='mgr.x' 2024-08-31T05:56:03.000 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:02 smithi016 ceph-mon[17194]: from='mgr.14118 172.21.15.16:0/2588448908' entity='mgr.x' 2024-08-31T05:56:03.007 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-08-31T05:56:04.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:04 smithi016 ceph-mon[17194]: from='mgr.14118 172.21.15.16:0/2588448908' entity='mgr.x' 2024-08-31T05:56:04.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:04 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/2232759704' entity='client.admin' 2024-08-31T05:56:05.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:05 smithi016 ceph-mon[17194]: from='mgr.14118 172.21.15.16:0/2588448908' entity='mgr.x' 2024-08-31T05:56:06.520 INFO:teuthology.orchestra.run.smithi016.stdout:Enabling the dashboard module... 2024-08-31T05:56:06.822 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:06 smithi016 ceph-mon[17194]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:56:06.822 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:06 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/4123813568' entity='client.admin' 2024-08-31T05:56:06.822 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:06 smithi016 ceph-mon[17194]: from='mgr.14118 172.21.15.16:0/2588448908' entity='mgr.x' 2024-08-31T05:56:07.814 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:07 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/3582114591' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2024-08-31T05:56:08.162 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:56:07 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: ignoring --setuser ceph since I am not root 2024-08-31T05:56:08.162 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:56:07 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: ignoring --setgroup ceph since I am not root 2024-08-31T05:56:09.090 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:08 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/3582114591' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-08-31T05:56:09.090 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:08 smithi016 ceph-mon[17194]: mgrmap e9: x(active, since 22s) 2024-08-31T05:56:10.068 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:09 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/3803472593' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-08-31T05:56:10.382 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-08-31T05:56:10.382 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-08-31T05:56:10.382 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": true, 2024-08-31T05:56:10.383 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-08-31T05:56:10.383 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-08-31T05:56:10.383 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-08-31T05:56:10.383 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for the mgr to restart... 2024-08-31T05:56:10.383 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr epoch 9... 2024-08-31T05:56:14.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:14 smithi016 ceph-mon[17194]: Active manager daemon x restarted 2024-08-31T05:56:14.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:14 smithi016 ceph-mon[17194]: Activating manager daemon x 2024-08-31T05:56:14.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:14 smithi016 ceph-mon[17194]: osdmap e3: 0 total, 0 up, 0 in 2024-08-31T05:56:14.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:14 smithi016 ceph-mon[17194]: mgrmap e10: x(active, starting, since 0.00500932s) 2024-08-31T05:56:14.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:14 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-31T05:56:14.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:14 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-08-31T05:56:14.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:14 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-08-31T05:56:14.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:14 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-08-31T05:56:14.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:14 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-08-31T05:56:14.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:14 smithi016 ceph-mon[17194]: Manager daemon x is now available 2024-08-31T05:56:15.566 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:15 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:56:15.566 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:15 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-08-31T05:56:15.566 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:15 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-08-31T05:56:15.566 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:15 smithi016 ceph-mon[17194]: mgrmap e11: x(active, since 1.00886s) 2024-08-31T05:56:15.862 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-08-31T05:56:15.863 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-08-31T05:56:15.863 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "initialized": true 2024-08-31T05:56:15.863 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-08-31T05:56:15.863 INFO:teuthology.orchestra.run.smithi016.stdout:mgr epoch 9 is available 2024-08-31T05:56:15.863 INFO:teuthology.orchestra.run.smithi016.stdout:Using certmgr to generate dashboard self-signed certificate... 2024-08-31T05:56:16.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:16 smithi016 ceph-mon[17194]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-08-31T05:56:16.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:16 smithi016 ceph-mon[17194]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-08-31T05:56:16.413 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:56:16 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: 2024-08-31T05:56:16.267+0000 7f786e200640 -1 log_channel(cephadm) log [ERR] : [31/Aug/2024:05:56:16] ENGINE Error in HTTPServer.serve 2024-08-31T05:56:16.413 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:56:16 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: Traceback (most recent call last): 2024-08-31T05:56:16.413 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:56:16 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-31T05:56:16.413 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:56:16 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: self._connections.run(self.expiration_interval) 2024-08-31T05:56:16.413 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:56:16 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-31T05:56:16.413 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:56:16 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: self._run(expiration_interval) 2024-08-31T05:56:16.414 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:56:16 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-31T05:56:16.414 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:56:16 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: new_conn = self._from_server_socket(self.server.socket) 2024-08-31T05:56:16.414 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:56:16 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-31T05:56:16.414 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:56:16 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-31T05:56:16.414 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:56:16 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-31T05:56:16.414 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:56:16 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: s = self.context.wrap_socket( 2024-08-31T05:56:16.414 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:56:16 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-31T05:56:16.414 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:56:16 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: return self.sslsocket_class._create( 2024-08-31T05:56:16.414 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:56:16 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-31T05:56:16.414 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:56:16 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: self.do_handshake() 2024-08-31T05:56:16.414 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:56:16 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-31T05:56:16.414 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:56:16 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: self._sslobj.do_handshake() 2024-08-31T05:56:16.414 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:56:16 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-31T05:56:16.414 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:56:16 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: 2024-08-31T05:56:17.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:17 smithi016 ceph-mon[17194]: [31/Aug/2024:05:56:16] ENGINE Bus STARTING 2024-08-31T05:56:17.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:17 smithi016 ceph-mon[17194]: mgrmap e12: x(active, since 2s) 2024-08-31T05:56:17.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:17 smithi016 ceph-mon[17194]: [31/Aug/2024:05:56:16] ENGINE Serving on https://172.21.15.16:7150 2024-08-31T05:56:17.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:17 smithi016 ceph-mon[17194]: [31/Aug/2024:05:56:16] ENGINE Error in HTTPServer.serve 2024-08-31T05:56:17.663 INFO:journalctl@ceph.mon.a.smithi016.stdout: Traceback (most recent call last): 2024-08-31T05:56:17.663 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-31T05:56:17.663 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._connections.run(self.expiration_interval) 2024-08-31T05:56:17.663 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-31T05:56:17.663 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._run(expiration_interval) 2024-08-31T05:56:17.663 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-31T05:56:17.663 INFO:journalctl@ceph.mon.a.smithi016.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-31T05:56:17.664 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-31T05:56:17.664 INFO:journalctl@ceph.mon.a.smithi016.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-31T05:56:17.664 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-31T05:56:17.664 INFO:journalctl@ceph.mon.a.smithi016.stdout: s = self.context.wrap_socket( 2024-08-31T05:56:17.664 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-31T05:56:17.664 INFO:journalctl@ceph.mon.a.smithi016.stdout: return self.sslsocket_class._create( 2024-08-31T05:56:17.664 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-31T05:56:17.664 INFO:journalctl@ceph.mon.a.smithi016.stdout: self.do_handshake() 2024-08-31T05:56:17.664 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-31T05:56:17.664 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._sslobj.do_handshake() 2024-08-31T05:56:17.664 INFO:journalctl@ceph.mon.a.smithi016.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-31T05:56:17.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:17 smithi016 ceph-mon[17194]: [31/Aug/2024:05:56:16] ENGINE Serving on http://172.21.15.16:8765 2024-08-31T05:56:17.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:17 smithi016 ceph-mon[17194]: [31/Aug/2024:05:56:16] ENGINE Bus STARTED 2024-08-31T05:56:17.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:17 smithi016 ceph-mon[17194]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T05:56:17.665 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:17 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:18.954 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIE+zCCAuOgAwIBAgIUcOoTj08erq7lri8MPlrXHO49CXowDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMY2VwaGFkbS1yb290MB4XDTI0MDgzMTA1NTYxOFoXDTM0\nMDkwMTA1NTYxOFowFzEVMBMGA1UEAwwMMTcyLjIxLjE1LjE2MIICIjANBgkqhkiG\n9w0BAQEFAAOCAg8AMIICCgKCAgEAgkleFNvzey6ZhqM0ercgOJB2UYcHmHoylvGZ\nEJ+Pucaxtdc6wo9la3icuVbYXPWyW/Fvijtmj1eD5VJQCKhWtK1RTbfglI8d+03G\n7Jx5qZjW14vm2Bcyk3m9AxSIc5fNyr1OGCqZwJtzj9iODChirR0iAIUGpOcvtQ7I\njvTrQl+HRMVilgE6TBgQFM65GoIw7UefixHGkb78cHXajgchITeojL6o0O8g+C9e\nd5Z8qItE+Ra6u93Vo9QQEx+p5w78M+zOR255M+zY11Tgfx3E2i8wJCCwQBAqMpTP\nAQhyibIdDRPmSmxB9KG9cvQ1i+hyGhBu66EMk/ep3YwEb6reDRSvID2ceKKXFuLi\nRE+WSJPEk7kqcfUFvidzjR568+ko38cW/Ivha567oQC6joPYW8GPFmhCWNuyyqSD\noLRVuhaZv4MNXmp2W3T38kfr/z10hlykqgJbdRgS9R9jcSYo7d7wYGVsS7jRgfDS\n9s1qVjSWF8TdylW3FyvWhbkX346c26FjNaYfxpSsTuosgQsyi6ebmmU+/ek/Iqd8\nYRJr4fRVjcSXcr8SUP0O0sVsq86qHZ920ZgZ2ltepKtMNnkvyPkIPVvVEBb3bz8u\nPIYxDKS38VWQnxx9od7uThk2re/LkvjTjy9KuOISx6Vuo9qmENMxS/x/tmA5qJrn\n6cdTNssCAwEAAaM/MD0wLQYDVR0RBCYwJIIJc21pdGhpMDE2ghFkYXNoYm9hcmRf\nc2VydmVyc4cErBUPEDAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3DQEBCwUAA4ICAQA3\nxop4WXPv+0Pt6/zwxZKwceQ8VBzdLTrWvrerm8qAgG9vFM/Z3dKcSAWS5Qv76OK/\n3Ka6mIo0OnsENSNDSunvQ3uTfMhRFeW+ba8S9o6AzoRRNrmlWgf9JMtOIO+RhTaG\nhmIwm1jLNPkitqeDnvPWj+C0R5i2EHEV8rnsjEM+pDdxK7TY/mfDEpfCBoYMJ6NB\nN4pjgRTC8z6vHk4ow/0ZxTrsPlP3qrjcflXOFsGS1501BuaD5Q0utz1tpnGl1QJA\nk48PIdtqc14XQZsPp10eETMHIE8n96czXTwON61W1Jjpjog90Rk2qJJP3ntabsQS\nChP5uV8h71Ts/GfmaQ4tDTjb+sBqjvtTOOecUZnqyilL0ZlBbs17nyjgI499QrbE\nF6kwvUc6HC4sju6ule04a2KpAh91UiZXvbeA3LU5hHwWkd3zFhWII6cqZTg0FBdy\njhRXyl/GNw+VCaYQK8CstLXCP0IIpsOiaRbxj8ycAeE8lwIF61IGk815wUAJR14F\n3tz7nqPNXo8DGfEC4qlqFPmLPNG2dau0Bh1nqtddzQt+hQjZtSZ0t6QoCBf3xn50\nFEDkSw3XmFlo/w+7q361ntPjhdtOpm9nbL0y0+w8/AyyExuqgxvBfsQ9EYdfFD8T\ncSuZtNhJLSsaR/yewy3bSGokRotb493Am8xt+UjSBQ==\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJKAIBAAKCAgEAgkleFNvzey6ZhqM0ercgOJB2UYcHmHoylvGZEJ+Pucaxtdc6\nwo9la3icuVbYXPWyW/Fvijtmj1eD5VJQCKhWtK1RTbfglI8d+03G7Jx5qZjW14vm\n2Bcyk3m9AxSIc5fNyr1OGCqZwJtzj9iODChirR0iAIUGpOcvtQ7IjvTrQl+HRMVi\nlgE6TBgQFM65GoIw7UefixHGkb78cHXajgchITeojL6o0O8g+C9ed5Z8qItE+Ra6\nu93Vo9QQEx+p5w78M+zOR255M+zY11Tgfx3E2i8wJCCwQBAqMpTPAQhyibIdDRPm\nSmxB9KG9cvQ1i+hyGhBu66EMk/ep3YwEb6reDRSvID2ceKKXFuLiRE+WSJPEk7kq\ncfUFvidzjR568+ko38cW/Ivha567oQC6joPYW8GPFmhCWNuyyqSDoLRVuhaZv4MN\nXmp2W3T38kfr/z10hlykqgJbdRgS9R9jcSYo7d7wYGVsS7jRgfDS9s1qVjSWF8Td\nylW3FyvWhbkX346c26FjNaYfxpSsTuosgQsyi6ebmmU+/ek/Iqd8YRJr4fRVjcSX\ncr8SUP0O0sVsq86qHZ920ZgZ2ltepKtMNnkvyPkIPVvVEBb3bz8uPIYxDKS38VWQ\nnxx9od7uThk2re/LkvjTjy9KuOISx6Vuo9qmENMxS/x/tmA5qJrn6cdTNssCAwEA\nAQKCAgAdZHHeINToKFYoYq9XM/MH78dRkzXqdz7sgKq4blbdYW78CAdQl0sZGMe6\npFYBesh3J5uzQr1bfzHZTV2S5m9RKXybHO2EdlkcNaq3/IER+tIVJJG6lFkrnh4/\nEKXL9iZOI7D3kLzJEAjdHczF/2JrjZP7LAeYsjM4FJK4dZaRjb2VjTY1gydkvVT7\nbLujuFVTYoIivKrgfIa01FGvEvqtZMIYpnatVBVLCFy2e/O5jlnMHA2ajmxBHnlV\n5yHMZ7wL9tN+l/St+zPcOGDBLLUZR+YDcIByo3LxoFKm+AHnAJC/P4sHpS6oAskb\nou5lltYO/UAY1mdhBlGDErB5hMG9PVMPoXehFCm7ABiZPxI7mn4zcBdvDkNmLOK8\n1YYmAMa7/ImQjywkrOW+avWDQtKkf+AxB79T0wM+tMpEG8qQLW93YlBxFRNZqT7V\n+MMUx0NEw/cvOCowkkkpRhAlGGv9K7xzlD8vNfHgsgu4u7Xk3g9foH2QfY5c4p6D\nspRU6VWH+W6Gkcrdx4BGMzwnAOvcQzUhPy9TWMBUvnMcu3ZPZ4oep87REyw80gHJ\nYeGDuALAErnOjfNr5MGiUkE8aD/cvTmUQLgjuCSXhvW+U+FS18RWrCX7oF10rjoT\nrnyS6aXyUnF4g+lmMWfUVyTgb4WhUapqTKufLw2WUdWGkU9weQKCAQEAt2+Fojr6\n2NC49PVDYNg3cy2LpKfLpHoj0olT+b/Re4Gm5kzzf8cu+MaQsXMK9tQF/u6xHDdd\n/rp5dA9F3agQbplwAMZfVHuiX/2XkKEqQMvLlPXxVc/8jFwSvZaJLyl7DCY+4V3I\nCbNPYPGfwBlnfjdtUz8haYY+RBBvNp++e8ybebaiB21MYENbRU6KpBWNIobulyym\nTQamJkW1O3K3fkEeKh5TfdycmM9OPj58dXxE0dz1H41xFPXSRxS5c1+COVZ8Hrds\nqHmtAFDS256MSr2DfQwo4zVmc4bHZ8vtK4jEI/R0vGqWXTVTSHeTEGIlnCD1F/qa\nA3mZ7lDXVGIMJQKCAQEAtdN0yD01EcX3QV97xRMbEh4+kJuqxRD9JIEOxP9c+JwB\nmrlKS06MNZOgqzAQ1YztHh+gdoKQTTz+IR0xk4l+F5WRU5CcvvKOD8HwVy07RvBg\nwY3NevuN8E0L2OJ6SGS66OyrKCLi0691PioMhXzVOckJTAI0Byt9RofwgH6u2m29\nuQ7oWm874doTJfFwBV+RDeDov25B7kOTm0ZfZu2eggkKFnThyYYvJk1qLjxfbC7D\nW+5ewIJn6+DY1aL8GvNJhIYl1TPc8laiB8gcSvLLRXCPiDMnusLa5RANmb5Bbe58\nMsYDy1aro36gIrJm9Y0g9It3RRqBfJQxPqBPrVhMLwKCAQAlFhUMgaQppblfPR8M\nO2qqjLDI72vKqy8G5+Dkg5WUxNe7hB9O3DRw86RZXAsBcEhuDFAehksorwbTIitb\n7xizY64zWef4Kq7ffWPhR9RFGW7lhPvv/4bt0cewMPk3YcmQ4UKhSdahr3EDBpbn\nvbsg14/w3rignxWDqVby0kUuvmVIrFYB7mUcqUGQt5psJjdMnb8rLUnKFOgOoNVN\nWZojmoLAvTTgI3Q9qNwG1ZdQIXnAfE3KvUv5kuGaoWQFIXAwq32YsDj1rvP9NloL\nMPjoO3yUEZtKpIc8ICm8oIwo2xPcuRXI6jvQSHG4GwTOsSktcbRbzrzIqxMmDY4b\nb19JAoIBAQCXs5gw5NhdPtU/ZsHmOD2SzUcX8bj1+GmsRj0jYYeZKNgx1jxFPoC+\nxq/7g/vXoF/JjUuc9OwedS/2Qtw1IzbfyHtaxEHzEn4yaXXz1hop4q5nbr8sx6qx\n/iZPSw4JTnmfkq5uq0SipzZjqkOOMAvazX+DOidcgA1ihlLDcm7CcWMkfmqf4USN\n+BuXwavb39MvSCIIbZ52g00o9Prt7n0cMxyGhfhH4jt+coKPlvFqWk3GasgFSAga\nhhuDixtaBPemqSHxkvbZ/ivr4Cqy4YV1Jyep1DxmluJ40P738gO3C4318r4BrgtA\nOoqyyDQc1SXUwnO9ZRL59M5tWOWBVDUjAoIBAHT4tXtuRfBad0g3WFwI4+/NZ7o1\nlqZyV03vUtT+5obqSiGxGKidsWeu4RAUwIE6KnPpWmtOPC5tM3TK9h3ma+ujXwD+\nkLufTbr3Bm/6S2KZcl9i0UKqoDWk/xEKCFCqnPVZ6GzgtQ6eopkRv2fiwtlqnQkn\ndum3jQvzTChEY9k2ecR85AhxnkVkB9URRXVn6N9irVXb+vbwc6UZHkqfymXtmonG\nFBvt2K/BiFtfo9L9BbibaWzWcossGo7t7kbmHEmfF2HzpA+hZ6Ll+nepMcy4fiWh\nPKWnYKNzCSzAZ1bDp9mE7u9Lpm1NFwGbh2F3ootdYHeFTIqirod0osmQjjk=\n-----END RSA PRIVATE KEY-----\n"} 2024-08-31T05:56:19.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:19 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:20.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:20 smithi016 ceph-mon[17194]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T05:56:20.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:20 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:21.287 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout SSL certificate updated 2024-08-31T05:56:22.819 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout SSL certificate key updated 2024-08-31T05:56:22.819 INFO:teuthology.orchestra.run.smithi016.stdout:Creating initial admin user... 2024-08-31T05:56:23.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:23 smithi016 ceph-mon[17194]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T05:56:23.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:23 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:24.549 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$rEVJk0ihjY6JWNz8Fj9xhOH9R2S5gfu9ydr.nXBvEBsFUKJrI1f1q", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1725083783, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-08-31T05:56:24.549 INFO:teuthology.orchestra.run.smithi016.stdout:Fetching dashboard port number... 2024-08-31T05:56:25.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:24 smithi016 ceph-mon[17194]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T05:56:25.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:24 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:25.991 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:25 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/1309819945' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2024-08-31T05:56:26.000 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 8443 2024-08-31T05:56:26.018 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-31T05:56:26.018 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-08-31T05:56:26.032 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-31T05:56:26.032 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-08-31T05:56:26.033 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-08-31T05:56:26.033 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-08-31T05:56:26.035 INFO:teuthology.orchestra.run.smithi016.stdout:Ceph Dashboard is now available at: 2024-08-31T05:56:26.035 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T05:56:26.035 INFO:teuthology.orchestra.run.smithi016.stdout: URL: https://smithi016.front.sepia.ceph.com:8443/ 2024-08-31T05:56:26.035 INFO:teuthology.orchestra.run.smithi016.stdout: User: admin 2024-08-31T05:56:26.035 INFO:teuthology.orchestra.run.smithi016.stdout: Password: iay17st7i1 2024-08-31T05:56:26.035 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T05:56:26.036 INFO:teuthology.orchestra.run.smithi016.stdout:Saving cluster configuration to /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/config directory 2024-08-31T05:56:27.548 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-08-31T05:56:27.548 INFO:teuthology.orchestra.run.smithi016.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-08-31T05:56:27.548 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T05:56:27.548 INFO:teuthology.orchestra.run.smithi016.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-08-31T05:56:27.548 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T05:56:27.548 INFO:teuthology.orchestra.run.smithi016.stdout:Or, if you are only running a single cluster on this host: 2024-08-31T05:56:27.548 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T05:56:27.548 INFO:teuthology.orchestra.run.smithi016.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-08-31T05:56:27.548 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T05:56:27.549 INFO:teuthology.orchestra.run.smithi016.stdout:Please consider enabling telemetry to help improve Ceph: 2024-08-31T05:56:27.549 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T05:56:27.549 INFO:teuthology.orchestra.run.smithi016.stdout: ceph telemetry on 2024-08-31T05:56:27.549 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T05:56:27.549 INFO:teuthology.orchestra.run.smithi016.stdout:For more information see: 2024-08-31T05:56:27.549 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T05:56:27.549 INFO:teuthology.orchestra.run.smithi016.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-08-31T05:56:27.549 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T05:56:27.549 INFO:teuthology.orchestra.run.smithi016.stdout:Bootstrap complete. 2024-08-31T05:56:27.611 INFO:tasks.cephadm:Fetching config... 2024-08-31T05:56:27.612 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-08-31T05:56:27.612 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-08-31T05:56:27.648 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-08-31T05:56:27.649 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-08-31T05:56:27.649 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-08-31T05:56:27.726 INFO:tasks.cephadm:Fetching mon keyring... 2024-08-31T05:56:27.726 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-08-31T05:56:27.727 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/keyring of=/dev/stdout 2024-08-31T05:56:27.824 INFO:tasks.cephadm:Fetching pub ssh key... 2024-08-31T05:56:27.824 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-08-31T05:56:27.824 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-08-31T05:56:27.899 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-08-31T05:56:27.900 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjH1D+jnpR8sqM7Cs9bUaWxaLvH4cf2OQqsggcZ8yc+qmmHSRiPy6MKjVf/n9NmU9IMdnCDXe8nfD7eyF4tfhL+Ci0k5KwbmTBKXKnmFz5xenRvnjGfG6sRQymAu+HMq7aEBqnkr7nJtpDrWEe5fN/akFiFgZE2HusrooX6J60+EtVca0S6wLR3sbnNG1G5X9vM2PkY4omo5CzG0/sVvaRcHwsoSrYDplADIKjoYKwe2Bm0dXRaj6JX2aJKWaTA3WZWDW7ruYO+QV3ote94wisAiH6FVS2CumA9vNUPoC7ziTRhunaSJODXxa65L5l2+5R1OeH0gR48q/oVa1jSa9xMkUEr0WDbGa47mNmOjrxYVUWSQJfz6KYVD6mKw3Wytqh4tLUGY15URTjLqHPaUhmAgPfhAnMg4BIPUT1w+p9+zM9Z8Io2ttK5EwpcUBLxTwZtNxGzArF+BsRxg2bK3aFEy7M901GgZmdAQbEfmlFbIZUjPQm7/UsN7jDV/zEhmc= ceph-673e4aa0-675d-11ef-bcd4-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-08-31T05:56:28.024 INFO:teuthology.orchestra.run.smithi016.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjH1D+jnpR8sqM7Cs9bUaWxaLvH4cf2OQqsggcZ8yc+qmmHSRiPy6MKjVf/n9NmU9IMdnCDXe8nfD7eyF4tfhL+Ci0k5KwbmTBKXKnmFz5xenRvnjGfG6sRQymAu+HMq7aEBqnkr7nJtpDrWEe5fN/akFiFgZE2HusrooX6J60+EtVca0S6wLR3sbnNG1G5X9vM2PkY4omo5CzG0/sVvaRcHwsoSrYDplADIKjoYKwe2Bm0dXRaj6JX2aJKWaTA3WZWDW7ruYO+QV3ote94wisAiH6FVS2CumA9vNUPoC7ziTRhunaSJODXxa65L5l2+5R1OeH0gR48q/oVa1jSa9xMkUEr0WDbGa47mNmOjrxYVUWSQJfz6KYVD6mKw3Wytqh4tLUGY15URTjLqHPaUhmAgPfhAnMg4BIPUT1w+p9+zM9Z8Io2ttK5EwpcUBLxTwZtNxGzArF+BsRxg2bK3aFEy7M901GgZmdAQbEfmlFbIZUjPQm7/UsN7jDV/zEhmc= ceph-673e4aa0-675d-11ef-bcd4-c7b262605968 2024-08-31T05:56:28.042 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:27 smithi016 ceph-mon[17194]: mgrmap e13: x(active, since 12s) 2024-08-31T05:56:28.042 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:27 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/3442525330' entity='client.admin' 2024-08-31T05:56:28.051 DEBUG:teuthology.orchestra.run.smithi155:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjH1D+jnpR8sqM7Cs9bUaWxaLvH4cf2OQqsggcZ8yc+qmmHSRiPy6MKjVf/n9NmU9IMdnCDXe8nfD7eyF4tfhL+Ci0k5KwbmTBKXKnmFz5xenRvnjGfG6sRQymAu+HMq7aEBqnkr7nJtpDrWEe5fN/akFiFgZE2HusrooX6J60+EtVca0S6wLR3sbnNG1G5X9vM2PkY4omo5CzG0/sVvaRcHwsoSrYDplADIKjoYKwe2Bm0dXRaj6JX2aJKWaTA3WZWDW7ruYO+QV3ote94wisAiH6FVS2CumA9vNUPoC7ziTRhunaSJODXxa65L5l2+5R1OeH0gR48q/oVa1jSa9xMkUEr0WDbGa47mNmOjrxYVUWSQJfz6KYVD6mKw3Wytqh4tLUGY15URTjLqHPaUhmAgPfhAnMg4BIPUT1w+p9+zM9Z8Io2ttK5EwpcUBLxTwZtNxGzArF+BsRxg2bK3aFEy7M901GgZmdAQbEfmlFbIZUjPQm7/UsN7jDV/zEhmc= ceph-673e4aa0-675d-11ef-bcd4-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-08-31T05:56:28.092 INFO:teuthology.orchestra.run.smithi155.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjH1D+jnpR8sqM7Cs9bUaWxaLvH4cf2OQqsggcZ8yc+qmmHSRiPy6MKjVf/n9NmU9IMdnCDXe8nfD7eyF4tfhL+Ci0k5KwbmTBKXKnmFz5xenRvnjGfG6sRQymAu+HMq7aEBqnkr7nJtpDrWEe5fN/akFiFgZE2HusrooX6J60+EtVca0S6wLR3sbnNG1G5X9vM2PkY4omo5CzG0/sVvaRcHwsoSrYDplADIKjoYKwe2Bm0dXRaj6JX2aJKWaTA3WZWDW7ruYO+QV3ote94wisAiH6FVS2CumA9vNUPoC7ziTRhunaSJODXxa65L5l2+5R1OeH0gR48q/oVa1jSa9xMkUEr0WDbGa47mNmOjrxYVUWSQJfz6KYVD6mKw3Wytqh4tLUGY15URTjLqHPaUhmAgPfhAnMg4BIPUT1w+p9+zM9Z8Io2ttK5EwpcUBLxTwZtNxGzArF+BsRxg2bK3aFEy7M901GgZmdAQbEfmlFbIZUjPQm7/UsN7jDV/zEhmc= ceph-673e4aa0-675d-11ef-bcd4-c7b262605968 2024-08-31T05:56:28.105 DEBUG:teuthology.orchestra.run.smithi184:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjH1D+jnpR8sqM7Cs9bUaWxaLvH4cf2OQqsggcZ8yc+qmmHSRiPy6MKjVf/n9NmU9IMdnCDXe8nfD7eyF4tfhL+Ci0k5KwbmTBKXKnmFz5xenRvnjGfG6sRQymAu+HMq7aEBqnkr7nJtpDrWEe5fN/akFiFgZE2HusrooX6J60+EtVca0S6wLR3sbnNG1G5X9vM2PkY4omo5CzG0/sVvaRcHwsoSrYDplADIKjoYKwe2Bm0dXRaj6JX2aJKWaTA3WZWDW7ruYO+QV3ote94wisAiH6FVS2CumA9vNUPoC7ziTRhunaSJODXxa65L5l2+5R1OeH0gR48q/oVa1jSa9xMkUEr0WDbGa47mNmOjrxYVUWSQJfz6KYVD6mKw3Wytqh4tLUGY15URTjLqHPaUhmAgPfhAnMg4BIPUT1w+p9+zM9Z8Io2ttK5EwpcUBLxTwZtNxGzArF+BsRxg2bK3aFEy7M901GgZmdAQbEfmlFbIZUjPQm7/UsN7jDV/zEhmc= ceph-673e4aa0-675d-11ef-bcd4-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-08-31T05:56:28.146 INFO:teuthology.orchestra.run.smithi184.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjH1D+jnpR8sqM7Cs9bUaWxaLvH4cf2OQqsggcZ8yc+qmmHSRiPy6MKjVf/n9NmU9IMdnCDXe8nfD7eyF4tfhL+Ci0k5KwbmTBKXKnmFz5xenRvnjGfG6sRQymAu+HMq7aEBqnkr7nJtpDrWEe5fN/akFiFgZE2HusrooX6J60+EtVca0S6wLR3sbnNG1G5X9vM2PkY4omo5CzG0/sVvaRcHwsoSrYDplADIKjoYKwe2Bm0dXRaj6JX2aJKWaTA3WZWDW7ruYO+QV3ote94wisAiH6FVS2CumA9vNUPoC7ziTRhunaSJODXxa65L5l2+5R1OeH0gR48q/oVa1jSa9xMkUEr0WDbGa47mNmOjrxYVUWSQJfz6KYVD6mKw3Wytqh4tLUGY15URTjLqHPaUhmAgPfhAnMg4BIPUT1w+p9+zM9Z8Io2ttK5EwpcUBLxTwZtNxGzArF+BsRxg2bK3aFEy7M901GgZmdAQbEfmlFbIZUjPQm7/UsN7jDV/zEhmc= ceph-673e4aa0-675d-11ef-bcd4-c7b262605968 2024-08-31T05:56:28.159 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-08-31T05:56:28.557 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T05:56:30.165 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-08-31T05:56:30.166 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-08-31T05:56:30.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:30 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/2730388939' entity='client.admin' 2024-08-31T05:56:30.721 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T05:56:32.630 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi155 2024-08-31T05:56:32.630 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-08-31T05:56:32.630 DEBUG:teuthology.orchestra.run.smithi155:> dd of=/etc/ceph/ceph.conf 2024-08-31T05:56:32.649 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-08-31T05:56:32.649 DEBUG:teuthology.orchestra.run.smithi155:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-31T05:56:32.707 INFO:tasks.cephadm:Adding host smithi155 to orchestrator... 2024-08-31T05:56:32.707 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph orch host add smithi155 2024-08-31T05:56:33.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:32 smithi016 ceph-mon[17194]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T05:56:33.310 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:32 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:33.471 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T05:56:34.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:34 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:34.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:34 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:34.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:34 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi016", "name": "osd_memory_target"} : dispatch 2024-08-31T05:56:34.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:34 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:56:34.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:34 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T05:56:34.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:34 smithi016 ceph-mon[17194]: Updating smithi016:/etc/ceph/ceph.conf 2024-08-31T05:56:34.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:34 smithi016 ceph-mon[17194]: Updating smithi016:/var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T05:56:35.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:35 smithi016 ceph-mon[17194]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:56:35.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:35 smithi016 ceph-mon[17194]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-08-31T05:56:35.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:35 smithi016 ceph-mon[17194]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi155", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T05:56:35.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:35 smithi016 ceph-mon[17194]: Updating smithi016:/var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/config/ceph.client.admin.keyring 2024-08-31T05:56:35.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:35 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:35.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:35 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:35.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:35 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:36.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:36 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:56:36.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:36 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:56:36.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:36 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T05:56:36.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:36 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:36.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:36 smithi016 ceph-mon[17194]: Deploying cephadm binary to smithi155 2024-08-31T05:56:37.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:37 smithi016 ceph-mon[17194]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:56:38.311 INFO:teuthology.orchestra.run.smithi016.stdout:Added host 'smithi155' with addr '172.21.15.155' 2024-08-31T05:56:38.995 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph orch host ls --format=json 2024-08-31T05:56:39.362 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T05:56:39.627 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:39 smithi016 ceph-mon[17194]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:56:39.639 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:39 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:39.639 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:39 smithi016 ceph-mon[17194]: Added host smithi155 2024-08-31T05:56:39.639 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:39 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:56:39.639 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:39 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:39.639 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:39 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:40.164 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T05:56:40.164 INFO:teuthology.orchestra.run.smithi016.stdout:[{"addr": "172.21.15.16", "hostname": "smithi016", "labels": [], "status": ""}, {"addr": "172.21.15.155", "hostname": "smithi155", "labels": [], "status": ""}] 2024-08-31T05:56:40.834 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:40 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:40.874 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi184 2024-08-31T05:56:40.874 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-08-31T05:56:40.874 DEBUG:teuthology.orchestra.run.smithi184:> dd of=/etc/ceph/ceph.conf 2024-08-31T05:56:40.895 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-08-31T05:56:40.895 DEBUG:teuthology.orchestra.run.smithi184:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-31T05:56:40.953 INFO:tasks.cephadm:Adding host smithi184 to orchestrator... 2024-08-31T05:56:40.953 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph orch host add smithi184 2024-08-31T05:56:41.331 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T05:56:41.770 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:41 smithi016 ceph-mon[17194]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T05:56:41.770 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:41 smithi016 ceph-mon[17194]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:56:43.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:43 smithi016 ceph-mon[17194]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi184", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T05:56:43.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:43 smithi016 ceph-mon[17194]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:56:43.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:43 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:43.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:43 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:43.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:43 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:43.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:43 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:43.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:43 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi155", "name": "osd_memory_target"} : dispatch 2024-08-31T05:56:43.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:43 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:56:43.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:43 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T05:56:43.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:43 smithi016 ceph-mon[17194]: Updating smithi155:/etc/ceph/ceph.conf 2024-08-31T05:56:43.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:43 smithi016 ceph-mon[17194]: Updating smithi155:/var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T05:56:43.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:43 smithi016 ceph-mon[17194]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2024-08-31T05:56:43.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:43 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:43.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:43 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:43.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:43 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:44.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:44 smithi016 ceph-mon[17194]: Updating smithi155:/var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/config/ceph.client.admin.keyring 2024-08-31T05:56:44.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:44 smithi016 ceph-mon[17194]: Deploying cephadm binary to smithi184 2024-08-31T05:56:45.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:45 smithi016 ceph-mon[17194]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:56:46.104 INFO:teuthology.orchestra.run.smithi016.stdout:Added host 'smithi184' with addr '172.21.15.184' 2024-08-31T05:56:47.081 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph orch host ls --format=json 2024-08-31T05:56:47.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:47 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:47.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:47 smithi016 ceph-mon[17194]: Added host smithi184 2024-08-31T05:56:47.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:47 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:56:47.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:47 smithi016 ceph-mon[17194]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:56:47.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:47 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:47.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:47 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:47.449 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T05:56:48.334 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T05:56:48.335 INFO:teuthology.orchestra.run.smithi016.stdout:[{"addr": "172.21.15.16", "hostname": "smithi016", "labels": [], "status": ""}, {"addr": "172.21.15.155", "hostname": "smithi155", "labels": [], "status": ""}, {"addr": "172.21.15.184", "hostname": "smithi184", "labels": [], "status": ""}] 2024-08-31T05:56:48.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:48 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:49.067 INFO:tasks.cephadm:Setting crush tunables to default 2024-08-31T05:56:49.068 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph osd crush tunables default 2024-08-31T05:56:49.432 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T05:56:49.739 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:49 smithi016 ceph-mon[17194]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:56:49.739 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:49 smithi016 ceph-mon[17194]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T05:56:51.235 INFO:teuthology.orchestra.run.smithi016.stderr:adjusted tunables profile to default 2024-08-31T05:56:51.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:51 smithi016 ceph-mon[17194]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:56:51.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:51 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:51.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:51 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:51.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:51 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:51.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:51 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:51.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:51 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi184", "name": "osd_memory_target"} : dispatch 2024-08-31T05:56:51.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:51 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:56:51.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:51 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T05:56:51.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:51 smithi016 ceph-mon[17194]: Updating smithi184:/etc/ceph/ceph.conf 2024-08-31T05:56:51.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:51 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/2273401221' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-08-31T05:56:51.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:51 smithi016 ceph-mon[17194]: Updating smithi184:/var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T05:56:51.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:51 smithi016 ceph-mon[17194]: Updating smithi184:/etc/ceph/ceph.client.admin.keyring 2024-08-31T05:56:51.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:51 smithi016 ceph-mon[17194]: Updating smithi184:/var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/config/ceph.client.admin.keyring 2024-08-31T05:56:51.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:51 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:51.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:51 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:51.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:51 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:52.075 INFO:tasks.cephadm:Adding mon.a on smithi016 2024-08-31T05:56:52.075 INFO:tasks.cephadm:Adding mon.b on smithi155 2024-08-31T05:56:52.076 INFO:tasks.cephadm:Adding mon.c on smithi184 2024-08-31T05:56:52.076 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph orch apply mon '3;smithi016:172.21.15.16=a;smithi155:172.21.15.155=b;smithi184:172.21.15.184=c' 2024-08-31T05:56:52.323 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T05:56:52.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:52 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/2273401221' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-08-31T05:56:52.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:52 smithi016 ceph-mon[17194]: osdmap e4: 0 total, 0 up, 0 in 2024-08-31T05:56:52.933 INFO:teuthology.orchestra.run.smithi184.stdout:Scheduled mon update... 2024-08-31T05:56:53.574 DEBUG:teuthology.orchestra.run.smithi155:mon.b> sudo journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mon.b.service 2024-08-31T05:56:53.577 DEBUG:teuthology.orchestra.run.smithi184:mon.c> sudo journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mon.c.service 2024-08-31T05:56:53.579 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-31T05:56:53.580 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph mon dump -f json 2024-08-31T05:56:54.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:53 smithi016 ceph-mon[17194]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:56:54.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:53 smithi016 ceph-mon[17194]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi016:172.21.15.16=a;smithi155:172.21.15.155=b;smithi184:172.21.15.184=c", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T05:56:54.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:53 smithi016 ceph-mon[17194]: Saving service mon spec with placement smithi016:172.21.15.16=a;smithi155:172.21.15.155=b;smithi184:172.21.15.184=c;count:3 2024-08-31T05:56:54.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:53 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:54.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:53 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:56:54.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:53 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:56:54.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:53 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T05:56:54.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:53 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:54.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:53 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-31T05:56:54.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:53 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:56:55.344 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.c/config 2024-08-31T05:56:55.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:54 smithi016 ceph-mon[17194]: Deploying daemon mon.c on smithi184 2024-08-31T05:56:56.228 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-08-31T05:56:56.228 INFO:teuthology.orchestra.run.smithi184.stdout:{"epoch":1,"fsid":"673e4aa0-675d-11ef-bcd4-c7b262605968","modified":"2024-08-31T05:55:05.650830Z","created":"2024-08-31T05:55:05.650830Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-08-31T05:56:56.228 INFO:teuthology.orchestra.run.smithi184.stderr:dumped monmap epoch 1 2024-08-31T05:56:56.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:55 smithi016 ceph-mon[17194]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:56:56.541 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:56 smithi184 systemd[1]: Starting Ceph mon.c for 673e4aa0-675d-11ef-bcd4-c7b262605968... 2024-08-31T05:56:57.086 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:56 smithi184 podman[22313]: 2024-08-31 05:56:56.965095658 +0000 UTC m=+0.089124319 container create cc3de2bf8812a01cf5f5e345061bd69ae8bb6b09f8528ac1bf2d71bc6dfb899a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-08-31T05:56:57.086 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:56 smithi184 podman[22313]: 2024-08-31 05:56:56.891710246 +0000 UTC m=+0.015738909 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T05:56:57.314 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:56 smithi016 ceph-mon[17194]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:56:57.314 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:56:56 smithi016 ceph-mon[17194]: from='client.? 172.21.15.184:0/226115605' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-31T05:56:57.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 podman[22313]: 2024-08-31 05:56:57.192650526 +0000 UTC m=+0.316679186 container init cc3de2bf8812a01cf5f5e345061bd69ae8bb6b09f8528ac1bf2d71bc6dfb899a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mon-c, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-08-31T05:56:57.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 podman[22313]: 2024-08-31 05:56:57.195652026 +0000 UTC m=+0.319680687 container start cc3de2bf8812a01cf5f5e345061bd69ae8bb6b09f8528ac1bf2d71bc6dfb899a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mon-c, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.1, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-31T05:56:57.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: set uid:gid to 167:167 (ceph:ceph) 2024-08-31T05:56:57.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: ceph version 19.3.0-4560-gd19e6118 (d19e6118f7015f0047264006cb0c873528eb3ae6) squid (dev), process ceph-mon, pid 2 2024-08-31T05:56:57.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: pidfile_write: ignore empty --pid-file 2024-08-31T05:56:57.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: load: jerasure load: lrc 2024-08-31T05:56:57.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: RocksDB version: 7.9.2 2024-08-31T05:56:57.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Git sha 0 2024-08-31T05:56:57.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Compile date 2024-08-28 10:19:06 2024-08-31T05:56:57.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: DB SUMMARY 2024-08-31T05:56:57.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: DB Session ID: X2FN8UVDL0LSKG0MIDR8 2024-08-31T05:56:57.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: CURRENT file: CURRENT 2024-08-31T05:56:57.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: IDENTITY file: IDENTITY 2024-08-31T05:56:57.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-08-31T05:56:57.475 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-08-31T05:56:57.475 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-08-31T05:56:57.475 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.error_if_exists: 0 2024-08-31T05:56:57.475 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.create_if_missing: 0 2024-08-31T05:56:57.475 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.paranoid_checks: 1 2024-08-31T05:56:57.475 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.flush_verify_memtable_count: 1 2024-08-31T05:56:57.475 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-08-31T05:56:57.475 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-08-31T05:56:57.475 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.env: 0x563158aa45a0 2024-08-31T05:56:57.475 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.fs: PosixFileSystem 2024-08-31T05:56:57.475 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.info_log: 0x56315d73b4e0 2024-08-31T05:56:57.475 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.max_file_opening_threads: 16 2024-08-31T05:56:57.476 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.statistics: (nil) 2024-08-31T05:56:57.476 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.use_fsync: 0 2024-08-31T05:56:57.476 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.max_log_file_size: 0 2024-08-31T05:56:57.476 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-08-31T05:56:57.476 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.log_file_time_to_roll: 0 2024-08-31T05:56:57.476 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.keep_log_file_num: 1000 2024-08-31T05:56:57.476 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.recycle_log_file_num: 0 2024-08-31T05:56:57.476 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.allow_fallocate: 1 2024-08-31T05:56:57.476 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.allow_mmap_reads: 0 2024-08-31T05:56:57.476 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.allow_mmap_writes: 0 2024-08-31T05:56:57.476 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.use_direct_reads: 0 2024-08-31T05:56:57.476 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-08-31T05:56:57.477 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.create_missing_column_families: 0 2024-08-31T05:56:57.477 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.db_log_dir: 2024-08-31T05:56:57.477 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.wal_dir: 2024-08-31T05:56:57.477 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.table_cache_numshardbits: 6 2024-08-31T05:56:57.477 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.WAL_ttl_seconds: 0 2024-08-31T05:56:57.477 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.WAL_size_limit_MB: 0 2024-08-31T05:56:57.477 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-08-31T05:56:57.477 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-08-31T05:56:57.477 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.is_fd_close_on_exec: 1 2024-08-31T05:56:57.477 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.advise_random_on_open: 1 2024-08-31T05:56:57.477 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.db_write_buffer_size: 0 2024-08-31T05:56:57.477 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.write_buffer_manager: 0x56315d73f5e0 2024-08-31T05:56:57.477 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-08-31T05:56:57.478 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-08-31T05:56:57.478 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.use_adaptive_mutex: 0 2024-08-31T05:56:57.478 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.rate_limiter: (nil) 2024-08-31T05:56:57.478 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-08-31T05:56:57.478 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.wal_recovery_mode: 2 2024-08-31T05:56:57.478 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.enable_thread_tracking: 0 2024-08-31T05:56:57.478 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.enable_pipelined_write: 0 2024-08-31T05:56:57.478 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.unordered_write: 0 2024-08-31T05:56:57.478 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-08-31T05:56:57.478 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-08-31T05:56:57.478 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-08-31T05:56:57.478 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-08-31T05:56:57.478 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.row_cache: None 2024-08-31T05:56:57.480 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.wal_filter: None 2024-08-31T05:56:57.480 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-08-31T05:56:57.480 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.allow_ingest_behind: 0 2024-08-31T05:56:57.480 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.two_write_queues: 0 2024-08-31T05:56:57.480 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.manual_wal_flush: 0 2024-08-31T05:56:57.480 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.wal_compression: 0 2024-08-31T05:56:57.481 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.atomic_flush: 0 2024-08-31T05:56:57.481 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-08-31T05:56:57.481 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.persist_stats_to_disk: 0 2024-08-31T05:56:57.481 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.write_dbid_to_manifest: 0 2024-08-31T05:56:57.481 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.log_readahead_size: 0 2024-08-31T05:56:57.481 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-08-31T05:56:57.481 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.best_efforts_recovery: 0 2024-08-31T05:56:57.481 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-08-31T05:56:57.481 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-08-31T05:56:57.481 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.allow_data_in_errors: 0 2024-08-31T05:56:57.481 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.db_host_id: __hostname__ 2024-08-31T05:56:57.482 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.enforce_single_del_contracts: true 2024-08-31T05:56:57.482 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.max_background_jobs: 2 2024-08-31T05:56:57.482 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.max_background_compactions: -1 2024-08-31T05:56:57.482 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.max_subcompactions: 1 2024-08-31T05:56:57.482 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-08-31T05:56:57.482 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-08-31T05:56:57.482 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.delayed_write_rate : 16777216 2024-08-31T05:56:57.482 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.max_total_wal_size: 0 2024-08-31T05:56:57.482 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-08-31T05:56:57.482 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.stats_dump_period_sec: 600 2024-08-31T05:56:57.482 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.stats_persist_period_sec: 600 2024-08-31T05:56:57.483 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-08-31T05:56:57.483 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.max_open_files: -1 2024-08-31T05:56:57.483 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.bytes_per_sync: 0 2024-08-31T05:56:57.483 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.wal_bytes_per_sync: 0 2024-08-31T05:56:57.483 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.strict_bytes_per_sync: 0 2024-08-31T05:56:57.483 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.compaction_readahead_size: 0 2024-08-31T05:56:57.483 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.max_background_flushes: -1 2024-08-31T05:56:57.483 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Compression algorithms supported: 2024-08-31T05:56:57.483 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: kZSTD supported: 0 2024-08-31T05:56:57.483 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: kXpressCompression supported: 0 2024-08-31T05:56:57.483 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: kBZip2Compression supported: 0 2024-08-31T05:56:57.484 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-08-31T05:56:57.484 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: kLZ4Compression supported: 1 2024-08-31T05:56:57.484 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: kZlibCompression supported: 1 2024-08-31T05:56:57.484 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: kLZ4HCCompression supported: 1 2024-08-31T05:56:57.484 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: kSnappyCompression supported: 1 2024-08-31T05:56:57.484 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Fast CRC32 supported: Supported on x86 2024-08-31T05:56:57.484 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: DMutex implementation: pthread_mutex_t 2024-08-31T05:56:57.484 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-08-31T05:56:57.484 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-08-31T05:56:57.484 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-08-31T05:56:57.484 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.merge_operator: 2024-08-31T05:56:57.484 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.compaction_filter: None 2024-08-31T05:56:57.484 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.compaction_filter_factory: None 2024-08-31T05:56:57.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.sst_partitioner_factory: None 2024-08-31T05:56:57.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.memtable_factory: SkipListFactory 2024-08-31T05:56:57.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.table_factory: BlockBasedTable 2024-08-31T05:56:57.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56315d73b100) 2024-08-31T05:56:57.485 INFO:journalctl@ceph.mon.c.smithi184.stdout: cache_index_and_filter_blocks: 1 2024-08-31T05:56:57.485 INFO:journalctl@ceph.mon.c.smithi184.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-08-31T05:56:57.485 INFO:journalctl@ceph.mon.c.smithi184.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-08-31T05:56:57.485 INFO:journalctl@ceph.mon.c.smithi184.stdout: pin_top_level_index_and_filter: 1 2024-08-31T05:56:57.485 INFO:journalctl@ceph.mon.c.smithi184.stdout: index_type: 0 2024-08-31T05:56:57.485 INFO:journalctl@ceph.mon.c.smithi184.stdout: data_block_index_type: 0 2024-08-31T05:56:57.485 INFO:journalctl@ceph.mon.c.smithi184.stdout: index_shortening: 1 2024-08-31T05:56:57.485 INFO:journalctl@ceph.mon.c.smithi184.stdout: data_block_hash_table_util_ratio: 0.750000 2024-08-31T05:56:57.486 INFO:journalctl@ceph.mon.c.smithi184.stdout: checksum: 4 2024-08-31T05:56:57.486 INFO:journalctl@ceph.mon.c.smithi184.stdout: no_block_cache: 0 2024-08-31T05:56:57.486 INFO:journalctl@ceph.mon.c.smithi184.stdout: block_cache: 0x5631596fe850 2024-08-31T05:56:57.486 INFO:journalctl@ceph.mon.c.smithi184.stdout: block_cache_name: BinnedLRUCache 2024-08-31T05:56:57.486 INFO:journalctl@ceph.mon.c.smithi184.stdout: block_cache_options: 2024-08-31T05:56:57.486 INFO:journalctl@ceph.mon.c.smithi184.stdout: capacity : 536870912 2024-08-31T05:56:57.486 INFO:journalctl@ceph.mon.c.smithi184.stdout: num_shard_bits : 4 2024-08-31T05:56:57.486 INFO:journalctl@ceph.mon.c.smithi184.stdout: strict_capacity_limit : 0 2024-08-31T05:56:57.486 INFO:journalctl@ceph.mon.c.smithi184.stdout: high_pri_pool_ratio: 0.000 2024-08-31T05:56:57.486 INFO:journalctl@ceph.mon.c.smithi184.stdout: block_cache_compressed: (nil) 2024-08-31T05:56:57.486 INFO:journalctl@ceph.mon.c.smithi184.stdout: persistent_cache: (nil) 2024-08-31T05:56:57.486 INFO:journalctl@ceph.mon.c.smithi184.stdout: block_size: 4096 2024-08-31T05:56:57.486 INFO:journalctl@ceph.mon.c.smithi184.stdout: block_size_deviation: 10 2024-08-31T05:56:57.486 INFO:journalctl@ceph.mon.c.smithi184.stdout: block_restart_interval: 16 2024-08-31T05:56:57.487 INFO:journalctl@ceph.mon.c.smithi184.stdout: index_block_restart_interval: 1 2024-08-31T05:56:57.487 INFO:journalctl@ceph.mon.c.smithi184.stdout: metadata_block_size: 4096 2024-08-31T05:56:57.487 INFO:journalctl@ceph.mon.c.smithi184.stdout: partition_filters: 0 2024-08-31T05:56:57.487 INFO:journalctl@ceph.mon.c.smithi184.stdout: use_delta_encoding: 1 2024-08-31T05:56:57.487 INFO:journalctl@ceph.mon.c.smithi184.stdout: filter_policy: bloomfilter 2024-08-31T05:56:57.487 INFO:journalctl@ceph.mon.c.smithi184.stdout: whole_key_filtering: 1 2024-08-31T05:56:57.487 INFO:journalctl@ceph.mon.c.smithi184.stdout: verify_compression: 0 2024-08-31T05:56:57.487 INFO:journalctl@ceph.mon.c.smithi184.stdout: read_amp_bytes_per_bit: 0 2024-08-31T05:56:57.487 INFO:journalctl@ceph.mon.c.smithi184.stdout: format_version: 5 2024-08-31T05:56:57.487 INFO:journalctl@ceph.mon.c.smithi184.stdout: enable_index_compression: 1 2024-08-31T05:56:57.487 INFO:journalctl@ceph.mon.c.smithi184.stdout: block_align: 0 2024-08-31T05:56:57.487 INFO:journalctl@ceph.mon.c.smithi184.stdout: max_auto_readahead_size: 262144 2024-08-31T05:56:57.488 INFO:journalctl@ceph.mon.c.smithi184.stdout: prepopulate_block_cache: 0 2024-08-31T05:56:57.488 INFO:journalctl@ceph.mon.c.smithi184.stdout: initial_auto_readahead_size: 8192 2024-08-31T05:56:57.488 INFO:journalctl@ceph.mon.c.smithi184.stdout: num_file_reads_for_auto_readahead: 2 2024-08-31T05:56:57.488 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.write_buffer_size: 33554432 2024-08-31T05:56:57.488 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.max_write_buffer_number: 2 2024-08-31T05:56:57.488 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.compression: NoCompression 2024-08-31T05:56:57.488 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.bottommost_compression: Disabled 2024-08-31T05:56:57.488 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.prefix_extractor: nullptr 2024-08-31T05:56:57.488 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-08-31T05:56:57.488 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.num_levels: 7 2024-08-31T05:56:57.488 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-08-31T05:56:57.488 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-08-31T05:56:57.489 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-08-31T05:56:57.489 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-08-31T05:56:57.489 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-08-31T05:56:57.489 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-08-31T05:56:57.489 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-08-31T05:56:57.489 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-08-31T05:56:57.489 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-08-31T05:56:57.489 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-08-31T05:56:57.489 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-08-31T05:56:57.489 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-08-31T05:56:57.489 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.compression_opts.window_bits: -14 2024-08-31T05:56:57.489 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.compression_opts.level: 32767 2024-08-31T05:56:57.489 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.compression_opts.strategy: 0 2024-08-31T05:56:57.490 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-08-31T05:56:57.490 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-08-31T05:56:57.490 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-08-31T05:56:57.490 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-08-31T05:56:57.490 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.compression_opts.enabled: false 2024-08-31T05:56:57.490 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-08-31T05:56:57.490 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-08-31T05:56:57.490 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-08-31T05:56:57.490 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-08-31T05:56:57.490 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.target_file_size_base: 67108864 2024-08-31T05:56:57.490 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.target_file_size_multiplier: 1 2024-08-31T05:56:57.490 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-08-31T05:56:57.491 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-08-31T05:56:57.491 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-08-31T05:56:57.491 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-08-31T05:56:57.491 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-08-31T05:56:57.491 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-08-31T05:56:57.491 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-08-31T05:56:57.491 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-08-31T05:56:57.491 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-08-31T05:56:57.491 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-08-31T05:56:57.491 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-08-31T05:56:57.491 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-08-31T05:56:57.491 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-08-31T05:56:57.492 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.arena_block_size: 1048576 2024-08-31T05:56:57.492 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-08-31T05:56:57.492 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-08-31T05:56:57.492 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.disable_auto_compactions: 0 2024-08-31T05:56:57.492 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-08-31T05:56:57.492 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-08-31T05:56:57.492 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-08-31T05:56:57.492 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-08-31T05:56:57.492 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-08-31T05:56:57.492 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-08-31T05:56:57.492 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-08-31T05:56:57.492 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-08-31T05:56:57.493 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-08-31T05:56:57.493 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-08-31T05:56:57.493 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-08-31T05:56:57.493 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.inplace_update_support: 0 2024-08-31T05:56:57.493 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.inplace_update_num_locks: 10000 2024-08-31T05:56:57.493 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-08-31T05:56:57.493 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-08-31T05:56:57.493 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.memtable_huge_page_size: 0 2024-08-31T05:56:57.493 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.bloom_locality: 0 2024-08-31T05:56:57.493 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.max_successive_merges: 0 2024-08-31T05:56:57.493 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.optimize_filters_for_hits: 0 2024-08-31T05:56:57.493 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.paranoid_file_checks: 0 2024-08-31T05:56:57.493 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.force_consistency_checks: 1 2024-08-31T05:56:57.494 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.report_bg_io_stats: 0 2024-08-31T05:56:57.494 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.ttl: 2592000 2024-08-31T05:56:57.494 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.periodic_compaction_seconds: 0 2024-08-31T05:56:57.494 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-08-31T05:56:57.494 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-08-31T05:56:57.494 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.enable_blob_files: false 2024-08-31T05:56:57.494 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.min_blob_size: 0 2024-08-31T05:56:57.494 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.blob_file_size: 268435456 2024-08-31T05:56:57.494 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.blob_compression_type: NoCompression 2024-08-31T05:56:57.494 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.enable_blob_garbage_collection: false 2024-08-31T05:56:57.494 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-08-31T05:56:57.494 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-08-31T05:56:57.494 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-08-31T05:56:57.495 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.blob_file_starting_level: 0 2024-08-31T05:56:57.495 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-08-31T05:56:57.495 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-08-31T05:56:57.495 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-08-31T05:56:57.495 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 6dfecdb0-5d61-463a-96b6-e4a9e26cd536 2024-08-31T05:56:57.495 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725083817232621, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-08-31T05:56:57.495 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-08-31T05:56:57.495 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725083817233072, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1725083817, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "6dfecdb0-5d61-463a-96b6-e4a9e26cd536", "db_session_id": "X2FN8UVDL0LSKG0MIDR8", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-08-31T05:56:57.495 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725083817233167, "job": 1, "event": "recovery_finished"} 2024-08-31T05:56:57.495 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-08-31T05:56:57.495 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-08-31T05:56:57.495 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56315d75ea80 2024-08-31T05:56:57.495 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: DB pointer 0x56315e500000 2024-08-31T05:56:57.496 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-08-31T05:56:57.496 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-08-31T05:56:57.496 INFO:journalctl@ceph.mon.c.smithi184.stdout: ** DB Stats ** 2024-08-31T05:56:57.496 INFO:journalctl@ceph.mon.c.smithi184.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-31T05:56:57.496 INFO:journalctl@ceph.mon.c.smithi184.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-08-31T05:56:57.496 INFO:journalctl@ceph.mon.c.smithi184.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-31T05:56:57.496 INFO:journalctl@ceph.mon.c.smithi184.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-31T05:56:57.496 INFO:journalctl@ceph.mon.c.smithi184.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-08-31T05:56:57.496 INFO:journalctl@ceph.mon.c.smithi184.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-31T05:56:57.496 INFO:journalctl@ceph.mon.c.smithi184.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-31T05:56:57.496 INFO:journalctl@ceph.mon.c.smithi184.stdout: 2024-08-31T05:56:57.496 INFO:journalctl@ceph.mon.c.smithi184.stdout: ** Compaction Stats [default] ** 2024-08-31T05:56:57.496 INFO:journalctl@ceph.mon.c.smithi184.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-31T05:56:57.497 INFO:journalctl@ceph.mon.c.smithi184.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-08-31T05:56:57.497 INFO:journalctl@ceph.mon.c.smithi184.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-08-31T05:56:57.497 INFO:journalctl@ceph.mon.c.smithi184.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-08-31T05:56:57.497 INFO:journalctl@ceph.mon.c.smithi184.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-08-31T05:56:57.497 INFO:journalctl@ceph.mon.c.smithi184.stdout: 2024-08-31T05:56:57.497 INFO:journalctl@ceph.mon.c.smithi184.stdout: ** Compaction Stats [default] ** 2024-08-31T05:56:57.497 INFO:journalctl@ceph.mon.c.smithi184.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-31T05:56:57.497 INFO:journalctl@ceph.mon.c.smithi184.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-08-31T05:56:57.497 INFO:journalctl@ceph.mon.c.smithi184.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-08-31T05:56:57.497 INFO:journalctl@ceph.mon.c.smithi184.stdout: 2024-08-31T05:56:57.497 INFO:journalctl@ceph.mon.c.smithi184.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-08-31T05:56:57.497 INFO:journalctl@ceph.mon.c.smithi184.stdout: 2024-08-31T05:56:57.498 INFO:journalctl@ceph.mon.c.smithi184.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-31T05:56:57.498 INFO:journalctl@ceph.mon.c.smithi184.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-08-31T05:56:57.498 INFO:journalctl@ceph.mon.c.smithi184.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-08-31T05:56:57.498 INFO:journalctl@ceph.mon.c.smithi184.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-08-31T05:56:57.498 INFO:journalctl@ceph.mon.c.smithi184.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-08-31T05:56:57.498 INFO:journalctl@ceph.mon.c.smithi184.stdout: AddFile(Keys): cumulative 0, interval 0 2024-08-31T05:56:57.498 INFO:journalctl@ceph.mon.c.smithi184.stdout: Cumulative compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-31T05:56:57.498 INFO:journalctl@ceph.mon.c.smithi184.stdout: Interval compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-31T05:56:57.498 INFO:journalctl@ceph.mon.c.smithi184.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-08-31T05:56:57.498 INFO:journalctl@ceph.mon.c.smithi184.stdout: Block cache BinnedLRUCache@0x5631596fe850#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-08-31T05:56:57.498 INFO:journalctl@ceph.mon.c.smithi184.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-08-31T05:56:57.498 INFO:journalctl@ceph.mon.c.smithi184.stdout: 2024-08-31T05:56:57.499 INFO:journalctl@ceph.mon.c.smithi184.stdout: ** File Read Latency Histogram By Level [default] ** 2024-08-31T05:56:57.499 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-08-31T05:56:57.499 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: using public_addr v2:172.21.15.184:0/0 -> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] 2024-08-31T05:56:57.499 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: starting mon.c rank -1 at public addrs [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] at bind addrs [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 2024-08-31T05:56:57.499 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: mon.c@-1(???) e0 preinit fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 2024-08-31T05:56:57.499 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: mon.c@-1(synchronizing).mds e1 new map 2024-08-31T05:56:57.499 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: mon.c@-1(synchronizing).mds e1 print_map 2024-08-31T05:56:57.499 INFO:journalctl@ceph.mon.c.smithi184.stdout: e1 2024-08-31T05:56:57.499 INFO:journalctl@ceph.mon.c.smithi184.stdout: btime 2024-08-31T05:55:09:632201+0000 2024-08-31T05:56:57.499 INFO:journalctl@ceph.mon.c.smithi184.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-08-31T05:56:57.499 INFO:journalctl@ceph.mon.c.smithi184.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-31T05:56:57.499 INFO:journalctl@ceph.mon.c.smithi184.stdout: legacy client fscid: -1 2024-08-31T05:56:57.499 INFO:journalctl@ceph.mon.c.smithi184.stdout: 2024-08-31T05:56:57.500 INFO:journalctl@ceph.mon.c.smithi184.stdout: No filesystems configured 2024-08-31T05:56:57.500 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-08-31T05:56:57.500 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-08-31T05:56:57.500 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-08-31T05:56:57.500 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-08-31T05:56:57.500 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-08-31T05:56:57.500 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-08-31T05:56:57.500 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-08-31T05:56:57.500 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-31T05:56:57.500 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-31T05:56:57.501 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-31T05:56:57.501 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T05:56:57.501 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:56:57.501 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi184", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T05:56:57.501 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:56:57.501 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:57.501 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:57.501 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:57.501 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:57.501 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi155", "name": "osd_memory_target"} : dispatch 2024-08-31T05:56:57.501 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:56:57.502 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T05:56:57.502 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: Updating smithi155:/etc/ceph/ceph.conf 2024-08-31T05:56:57.502 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: Updating smithi155:/var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T05:56:57.502 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2024-08-31T05:56:57.502 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:57.502 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:57.502 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:57.502 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: Updating smithi155:/var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/config/ceph.client.admin.keyring 2024-08-31T05:56:57.502 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: Deploying cephadm binary to smithi184 2024-08-31T05:56:57.502 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:56:57.502 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:57.502 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: Added host smithi184 2024-08-31T05:56:57.503 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:56:57.503 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:56:57.503 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:57.503 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:57.503 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:57.503 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:56:57.503 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T05:56:57.503 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:56:57.503 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:57.503 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:57.503 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:57.503 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:57.504 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi184", "name": "osd_memory_target"} : dispatch 2024-08-31T05:56:57.504 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:56:57.504 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T05:56:57.504 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: Updating smithi184:/etc/ceph/ceph.conf 2024-08-31T05:56:57.504 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/2273401221' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-08-31T05:56:57.504 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: Updating smithi184:/var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T05:56:57.504 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: Updating smithi184:/etc/ceph/ceph.client.admin.keyring 2024-08-31T05:56:57.504 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: Updating smithi184:/var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/config/ceph.client.admin.keyring 2024-08-31T05:56:57.504 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:57.504 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:57.504 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:57.505 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/2273401221' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-08-31T05:56:57.505 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: osdmap e4: 0 total, 0 up, 0 in 2024-08-31T05:56:57.505 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:56:57.505 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi016:172.21.15.16=a;smithi155:172.21.15.155=b;smithi184:172.21.15.184=c", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T05:56:57.505 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: Saving service mon spec with placement smithi016:172.21.15.16=a;smithi155:172.21.15.155=b;smithi184:172.21.15.184=c;count:3 2024-08-31T05:56:57.505 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:57.505 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:56:57.505 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:56:57.505 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T05:56:57.505 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:56:57.505 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-31T05:56:57.506 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:56:57.506 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: Deploying daemon mon.c on smithi184 2024-08-31T05:56:57.506 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:56:57.506 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:56:57.506 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: from='client.? 172.21.15.184:0/226115605' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-31T05:56:57.506 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 ceph-mon[22331]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-08-31T05:56:57.506 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 bash[22313]: cc3de2bf8812a01cf5f5e345061bd69ae8bb6b09f8528ac1bf2d71bc6dfb899a 2024-08-31T05:56:57.506 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:56:57 smithi184 systemd[1]: Started Ceph mon.c for 673e4aa0-675d-11ef-bcd4-c7b262605968. 2024-08-31T05:56:57.836 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-31T05:56:57.836 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph mon dump -f json 2024-08-31T05:56:58.086 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.c/config 2024-08-31T05:57:02.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:02 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-31T05:57:02.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:02 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T05:57:02.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:02 smithi016 ceph-mon[17194]: mon.a calling monitor election 2024-08-31T05:57:02.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:02 smithi016 ceph-mon[17194]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:57:02.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:02 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T05:57:02.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:02 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T05:57:02.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:02 smithi016 ceph-mon[17194]: mon.c calling monitor election 2024-08-31T05:57:02.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:02 smithi016 ceph-mon[17194]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:57:02.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:02 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T05:57:02.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:02 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T05:57:02.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:02 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T05:57:02.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:02 smithi016 ceph-mon[17194]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-31T05:57:02.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:02 smithi016 ceph-mon[17194]: monmap epoch 2 2024-08-31T05:57:02.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:02 smithi016 ceph-mon[17194]: fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 2024-08-31T05:57:02.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:02 smithi016 ceph-mon[17194]: last_changed 2024-08-31T05:56:57.263839+0000 2024-08-31T05:57:02.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:02 smithi016 ceph-mon[17194]: created 2024-08-31T05:55:05.650830+0000 2024-08-31T05:57:02.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:02 smithi016 ceph-mon[17194]: min_mon_release 19 (squid) 2024-08-31T05:57:02.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:02 smithi016 ceph-mon[17194]: election_strategy: 1 2024-08-31T05:57:02.665 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:02 smithi016 ceph-mon[17194]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-08-31T05:57:02.665 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:02 smithi016 ceph-mon[17194]: 1: [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] mon.c 2024-08-31T05:57:02.665 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:02 smithi016 ceph-mon[17194]: fsmap 2024-08-31T05:57:02.665 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:02 smithi016 ceph-mon[17194]: osdmap e4: 0 total, 0 up, 0 in 2024-08-31T05:57:02.665 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:02 smithi016 ceph-mon[17194]: mgrmap e13: x(active, since 48s) 2024-08-31T05:57:02.665 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:02 smithi016 ceph-mon[17194]: overall HEALTH_OK 2024-08-31T05:57:02.665 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:02 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:02.665 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:02 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:02.665 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:02 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:02.665 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:02 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-31T05:57:02.665 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:02 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:57:02.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:02 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-31T05:57:02.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:02 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T05:57:02.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:02 smithi184 ceph-mon[22331]: mon.a calling monitor election 2024-08-31T05:57:02.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:02 smithi184 ceph-mon[22331]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:57:02.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:02 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T05:57:02.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:02 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T05:57:02.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:02 smithi184 ceph-mon[22331]: mon.c calling monitor election 2024-08-31T05:57:02.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:02 smithi184 ceph-mon[22331]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:57:02.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:02 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T05:57:02.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:02 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T05:57:02.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:02 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T05:57:02.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:02 smithi184 ceph-mon[22331]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-31T05:57:02.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:02 smithi184 ceph-mon[22331]: monmap epoch 2 2024-08-31T05:57:02.724 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:02 smithi184 ceph-mon[22331]: fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 2024-08-31T05:57:02.724 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:02 smithi184 ceph-mon[22331]: last_changed 2024-08-31T05:56:57.263839+0000 2024-08-31T05:57:02.724 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:02 smithi184 ceph-mon[22331]: created 2024-08-31T05:55:05.650830+0000 2024-08-31T05:57:02.724 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:02 smithi184 ceph-mon[22331]: min_mon_release 19 (squid) 2024-08-31T05:57:02.724 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:02 smithi184 ceph-mon[22331]: election_strategy: 1 2024-08-31T05:57:02.724 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:02 smithi184 ceph-mon[22331]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-08-31T05:57:02.724 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:02 smithi184 ceph-mon[22331]: 1: [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] mon.c 2024-08-31T05:57:02.724 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:02 smithi184 ceph-mon[22331]: fsmap 2024-08-31T05:57:02.724 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:02 smithi184 ceph-mon[22331]: osdmap e4: 0 total, 0 up, 0 in 2024-08-31T05:57:02.724 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:02 smithi184 ceph-mon[22331]: mgrmap e13: x(active, since 48s) 2024-08-31T05:57:02.724 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:02 smithi184 ceph-mon[22331]: overall HEALTH_OK 2024-08-31T05:57:02.724 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:02 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:02.724 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:02 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:02.724 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:02 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:02.725 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:02 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-31T05:57:02.725 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:02 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:57:03.188 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-08-31T05:57:03.188 INFO:teuthology.orchestra.run.smithi184.stdout:{"epoch":2,"fsid":"673e4aa0-675d-11ef-bcd4-c7b262605968","modified":"2024-08-31T05:56:57.263839Z","created":"2024-08-31T05:55:05.650830Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:3300","nonce":0},{"type":"v1","addr":"172.21.15.184:6789","nonce":0}]},"addr":"172.21.15.184:6789/0","public_addr":"172.21.15.184:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-08-31T05:57:03.188 INFO:teuthology.orchestra.run.smithi184.stderr:dumped monmap epoch 2 2024-08-31T05:57:03.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:03 smithi184 ceph-mon[22331]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:57:03.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:03 smithi184 ceph-mon[22331]: Deploying daemon mon.b on smithi155 2024-08-31T05:57:03.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:03 smithi184 ceph-mon[22331]: from='client.? 172.21.15.184:0/821174338' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-31T05:57:03.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:03 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T05:57:03.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:03 smithi016 ceph-mon[17194]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:57:03.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:03 smithi016 ceph-mon[17194]: Deploying daemon mon.b on smithi155 2024-08-31T05:57:03.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:03 smithi016 ceph-mon[17194]: from='client.? 172.21.15.184:0/821174338' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-31T05:57:03.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:03 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T05:57:04.747 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-31T05:57:04.747 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph mon dump -f json 2024-08-31T05:57:05.006 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.c/config 2024-08-31T05:57:05.454 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:05 smithi184 ceph-mon[22331]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:57:05.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:05 smithi016 ceph-mon[17194]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:57:05.718 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-08-31T05:57:05.719 INFO:teuthology.orchestra.run.smithi184.stdout:{"epoch":2,"fsid":"673e4aa0-675d-11ef-bcd4-c7b262605968","modified":"2024-08-31T05:56:57.263839Z","created":"2024-08-31T05:55:05.650830Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:3300","nonce":0},{"type":"v1","addr":"172.21.15.184:6789","nonce":0}]},"addr":"172.21.15.184:6789/0","public_addr":"172.21.15.184:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-08-31T05:57:05.719 INFO:teuthology.orchestra.run.smithi184.stderr:dumped monmap epoch 2 2024-08-31T05:57:06.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:06 smithi155 systemd[1]: Started Ceph mon.b for 673e4aa0-675d-11ef-bcd4-c7b262605968. 2024-08-31T05:57:07.285 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-31T05:57:07.286 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph mon dump -f json 2024-08-31T05:57:07.538 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.c/config 2024-08-31T05:57:11.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:11 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-31T05:57:11.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:11 smithi016 ceph-mon[17194]: mon.a calling monitor election 2024-08-31T05:57:11.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:11 smithi016 ceph-mon[17194]: mon.c calling monitor election 2024-08-31T05:57:11.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:11 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T05:57:11.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:11 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T05:57:11.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:11 smithi016 ceph-mon[17194]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:57:11.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:11 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T05:57:11.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:11 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T05:57:11.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:11 smithi016 ceph-mon[17194]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:57:11.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:11 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T05:57:11.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:11 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T05:57:11.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:11 smithi016 ceph-mon[17194]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:57:11.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:11 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T05:57:11.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:11 smithi016 ceph-mon[17194]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-31T05:57:11.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:11 smithi016 ceph-mon[17194]: monmap epoch 3 2024-08-31T05:57:11.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:11 smithi016 ceph-mon[17194]: fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 2024-08-31T05:57:11.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:11 smithi016 ceph-mon[17194]: last_changed 2024-08-31T05:57:06.070015+0000 2024-08-31T05:57:11.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:11 smithi016 ceph-mon[17194]: created 2024-08-31T05:55:05.650830+0000 2024-08-31T05:57:11.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:11 smithi016 ceph-mon[17194]: min_mon_release 19 (squid) 2024-08-31T05:57:11.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:11 smithi016 ceph-mon[17194]: election_strategy: 1 2024-08-31T05:57:11.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:11 smithi016 ceph-mon[17194]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-08-31T05:57:11.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:11 smithi016 ceph-mon[17194]: 1: [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] mon.c 2024-08-31T05:57:11.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:11 smithi016 ceph-mon[17194]: 2: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.b 2024-08-31T05:57:11.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:11 smithi016 ceph-mon[17194]: fsmap 2024-08-31T05:57:11.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:11 smithi016 ceph-mon[17194]: osdmap e4: 0 total, 0 up, 0 in 2024-08-31T05:57:11.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:11 smithi016 ceph-mon[17194]: mgrmap e13: x(active, since 56s) 2024-08-31T05:57:11.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:11 smithi016 ceph-mon[17194]: overall HEALTH_OK 2024-08-31T05:57:11.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:11 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:11.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:11 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:11.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:11 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:11.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:11 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:11.424 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-08-31T05:57:11.424 INFO:teuthology.orchestra.run.smithi184.stdout:{"epoch":3,"fsid":"673e4aa0-675d-11ef-bcd4-c7b262605968","modified":"2024-08-31T05:57:06.070015Z","created":"2024-08-31T05:55:05.650830Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:3300","nonce":0},{"type":"v1","addr":"172.21.15.184:6789","nonce":0}]},"addr":"172.21.15.184:6789/0","public_addr":"172.21.15.184:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:3300","nonce":0},{"type":"v1","addr":"172.21.15.155:6789","nonce":0}]},"addr":"172.21.15.155:6789/0","public_addr":"172.21.15.155:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-08-31T05:57:11.425 INFO:teuthology.orchestra.run.smithi184.stderr:dumped monmap epoch 3 2024-08-31T05:57:11.425 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:11 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-31T05:57:11.425 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:11 smithi184 ceph-mon[22331]: mon.a calling monitor election 2024-08-31T05:57:11.425 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:11 smithi184 ceph-mon[22331]: mon.c calling monitor election 2024-08-31T05:57:11.425 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:11 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T05:57:11.425 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:11 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T05:57:11.425 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:11 smithi184 ceph-mon[22331]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:57:11.425 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:11 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T05:57:11.425 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:11 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T05:57:11.426 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:11 smithi184 ceph-mon[22331]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:57:11.426 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:11 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T05:57:11.426 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:11 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T05:57:11.426 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:11 smithi184 ceph-mon[22331]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:57:11.426 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:11 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T05:57:11.426 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:11 smithi184 ceph-mon[22331]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-31T05:57:11.426 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:11 smithi184 ceph-mon[22331]: monmap epoch 3 2024-08-31T05:57:11.426 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:11 smithi184 ceph-mon[22331]: fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 2024-08-31T05:57:11.426 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:11 smithi184 ceph-mon[22331]: last_changed 2024-08-31T05:57:06.070015+0000 2024-08-31T05:57:11.426 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:11 smithi184 ceph-mon[22331]: created 2024-08-31T05:55:05.650830+0000 2024-08-31T05:57:11.426 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:11 smithi184 ceph-mon[22331]: min_mon_release 19 (squid) 2024-08-31T05:57:11.426 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:11 smithi184 ceph-mon[22331]: election_strategy: 1 2024-08-31T05:57:11.426 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:11 smithi184 ceph-mon[22331]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-08-31T05:57:11.426 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:11 smithi184 ceph-mon[22331]: 1: [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] mon.c 2024-08-31T05:57:11.426 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:11 smithi184 ceph-mon[22331]: 2: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.b 2024-08-31T05:57:11.427 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:11 smithi184 ceph-mon[22331]: fsmap 2024-08-31T05:57:11.427 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:11 smithi184 ceph-mon[22331]: osdmap e4: 0 total, 0 up, 0 in 2024-08-31T05:57:11.427 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:11 smithi184 ceph-mon[22331]: mgrmap e13: x(active, since 56s) 2024-08-31T05:57:11.427 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:11 smithi184 ceph-mon[22331]: overall HEALTH_OK 2024-08-31T05:57:11.427 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:11 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:11.427 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:11 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:11.427 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:11 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:11.427 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:11 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:11.993 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-08-31T05:57:11.994 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph config generate-minimal-conf 2024-08-31T05:57:12.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:12 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:57:12.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:12 smithi184 ceph-mon[22331]: from='client.? 172.21.15.184:0/1662382420' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-31T05:57:12.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:12 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T05:57:12.374 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T05:57:12.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:12 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:57:12.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:12 smithi016 ceph-mon[17194]: from='client.? 172.21.15.184:0/1662382420' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-31T05:57:12.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:12 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T05:57:13.235 INFO:teuthology.orchestra.run.smithi016.stdout:# minimal ceph.conf for 673e4aa0-675d-11ef-bcd4-c7b262605968 2024-08-31T05:57:13.235 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2024-08-31T05:57:13.235 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = 673e4aa0-675d-11ef-bcd4-c7b262605968 2024-08-31T05:57:13.235 INFO:teuthology.orchestra.run.smithi016.stdout: mon_host = [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] 2024-08-31T05:57:13.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-31T05:57:13.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: mon.a calling monitor election 2024-08-31T05:57:13.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: mon.c calling monitor election 2024-08-31T05:57:13.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T05:57:13.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T05:57:13.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:57:13.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T05:57:13.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T05:57:13.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:57:13.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T05:57:13.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T05:57:13.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:57:13.374 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T05:57:13.374 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-31T05:57:13.374 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: monmap epoch 3 2024-08-31T05:57:13.374 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 2024-08-31T05:57:13.374 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: last_changed 2024-08-31T05:57:06.070015+0000 2024-08-31T05:57:13.374 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: created 2024-08-31T05:55:05.650830+0000 2024-08-31T05:57:13.374 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: min_mon_release 19 (squid) 2024-08-31T05:57:13.374 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: election_strategy: 1 2024-08-31T05:57:13.374 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-08-31T05:57:13.374 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: 1: [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] mon.c 2024-08-31T05:57:13.374 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: 2: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.b 2024-08-31T05:57:13.374 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: fsmap 2024-08-31T05:57:13.375 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: osdmap e4: 0 total, 0 up, 0 in 2024-08-31T05:57:13.375 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: mgrmap e13: x(active, since 56s) 2024-08-31T05:57:13.375 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: overall HEALTH_OK 2024-08-31T05:57:13.375 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:13.375 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:13.375 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:13.375 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:13.375 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:57:13.375 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: from='client.? 172.21.15.184:0/1662382420' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-31T05:57:13.375 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T05:57:13.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: mon.b calling monitor election 2024-08-31T05:57:13.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: mon.b calling monitor election 2024-08-31T05:57:13.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: mon.c calling monitor election 2024-08-31T05:57:13.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: mon.a calling monitor election 2024-08-31T05:57:13.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-08-31T05:57:13.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: monmap epoch 3 2024-08-31T05:57:13.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 2024-08-31T05:57:13.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: last_changed 2024-08-31T05:57:06.070015+0000 2024-08-31T05:57:13.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: created 2024-08-31T05:55:05.650830+0000 2024-08-31T05:57:13.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: min_mon_release 19 (squid) 2024-08-31T05:57:13.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: election_strategy: 1 2024-08-31T05:57:13.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-08-31T05:57:13.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: 1: [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] mon.c 2024-08-31T05:57:13.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: 2: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.b 2024-08-31T05:57:13.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: fsmap 2024-08-31T05:57:13.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: osdmap e4: 0 total, 0 up, 0 in 2024-08-31T05:57:13.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: mgrmap e13: x(active, since 58s) 2024-08-31T05:57:13.874 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: overall HEALTH_OK 2024-08-31T05:57:13.874 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/3919182614' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:57:13.874 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:13.874 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:13.874 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:13.874 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:13 smithi155 ceph-mon[21742]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:13.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:13 smithi016 ceph-mon[17194]: mon.b calling monitor election 2024-08-31T05:57:13.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:13 smithi016 ceph-mon[17194]: mon.b calling monitor election 2024-08-31T05:57:13.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:13 smithi016 ceph-mon[17194]: mon.c calling monitor election 2024-08-31T05:57:13.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:13 smithi016 ceph-mon[17194]: mon.a calling monitor election 2024-08-31T05:57:13.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:13 smithi016 ceph-mon[17194]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-08-31T05:57:13.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:13 smithi016 ceph-mon[17194]: monmap epoch 3 2024-08-31T05:57:13.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:13 smithi016 ceph-mon[17194]: fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 2024-08-31T05:57:13.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:13 smithi016 ceph-mon[17194]: last_changed 2024-08-31T05:57:06.070015+0000 2024-08-31T05:57:13.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:13 smithi016 ceph-mon[17194]: created 2024-08-31T05:55:05.650830+0000 2024-08-31T05:57:13.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:13 smithi016 ceph-mon[17194]: min_mon_release 19 (squid) 2024-08-31T05:57:13.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:13 smithi016 ceph-mon[17194]: election_strategy: 1 2024-08-31T05:57:13.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:13 smithi016 ceph-mon[17194]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-08-31T05:57:13.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:13 smithi016 ceph-mon[17194]: 1: [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] mon.c 2024-08-31T05:57:13.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:13 smithi016 ceph-mon[17194]: 2: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.b 2024-08-31T05:57:13.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:13 smithi016 ceph-mon[17194]: fsmap 2024-08-31T05:57:13.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:13 smithi016 ceph-mon[17194]: osdmap e4: 0 total, 0 up, 0 in 2024-08-31T05:57:13.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:13 smithi016 ceph-mon[17194]: mgrmap e13: x(active, since 58s) 2024-08-31T05:57:13.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:13 smithi016 ceph-mon[17194]: overall HEALTH_OK 2024-08-31T05:57:13.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:13 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/3919182614' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:57:13.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:13 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:13.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:13 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:13.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:13 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:13.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:13 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:14.027 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-08-31T05:57:14.027 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-08-31T05:57:14.027 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.conf 2024-08-31T05:57:14.087 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-08-31T05:57:14.087 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-31T05:57:14.183 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-08-31T05:57:14.183 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/ceph/ceph.conf 2024-08-31T05:57:14.211 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-08-31T05:57:14.211 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-31T05:57:14.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:13 smithi184 ceph-mon[22331]: mon.b calling monitor election 2024-08-31T05:57:14.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:13 smithi184 ceph-mon[22331]: mon.b calling monitor election 2024-08-31T05:57:14.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:13 smithi184 ceph-mon[22331]: mon.c calling monitor election 2024-08-31T05:57:14.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:13 smithi184 ceph-mon[22331]: mon.a calling monitor election 2024-08-31T05:57:14.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:13 smithi184 ceph-mon[22331]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-08-31T05:57:14.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:13 smithi184 ceph-mon[22331]: monmap epoch 3 2024-08-31T05:57:14.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:13 smithi184 ceph-mon[22331]: fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 2024-08-31T05:57:14.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:13 smithi184 ceph-mon[22331]: last_changed 2024-08-31T05:57:06.070015+0000 2024-08-31T05:57:14.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:13 smithi184 ceph-mon[22331]: created 2024-08-31T05:55:05.650830+0000 2024-08-31T05:57:14.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:13 smithi184 ceph-mon[22331]: min_mon_release 19 (squid) 2024-08-31T05:57:14.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:13 smithi184 ceph-mon[22331]: election_strategy: 1 2024-08-31T05:57:14.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:13 smithi184 ceph-mon[22331]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-08-31T05:57:14.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:13 smithi184 ceph-mon[22331]: 1: [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] mon.c 2024-08-31T05:57:14.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:13 smithi184 ceph-mon[22331]: 2: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.b 2024-08-31T05:57:14.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:13 smithi184 ceph-mon[22331]: fsmap 2024-08-31T05:57:14.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:13 smithi184 ceph-mon[22331]: osdmap e4: 0 total, 0 up, 0 in 2024-08-31T05:57:14.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:13 smithi184 ceph-mon[22331]: mgrmap e13: x(active, since 58s) 2024-08-31T05:57:14.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:13 smithi184 ceph-mon[22331]: overall HEALTH_OK 2024-08-31T05:57:14.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:13 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/3919182614' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:57:14.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:13 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:14.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:13 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:14.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:13 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:14.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:13 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:14.276 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-08-31T05:57:14.277 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/ceph/ceph.conf 2024-08-31T05:57:14.305 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-08-31T05:57:14.305 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-31T05:57:14.371 INFO:tasks.cephadm:enabling module snap_schedule 2024-08-31T05:57:14.372 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- sudo ceph --cluster ceph mgr module enable snap_schedule 2024-08-31T05:57:15.073 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:14 smithi016 ceph-mon[17194]: Updating smithi184:/var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T05:57:15.073 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:14 smithi016 ceph-mon[17194]: Updating smithi155:/var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T05:57:15.073 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:14 smithi016 ceph-mon[17194]: Updating smithi016:/var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T05:57:15.073 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:14 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:15.074 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:14 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:15.074 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:14 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:15.074 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:14 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:15.074 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:14 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:15.074 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:14 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:15.074 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:14 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:15.074 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:14 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-31T05:57:15.074 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:14 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-31T05:57:15.074 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:14 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:57:15.074 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:14 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T05:57:15.074 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:14 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:15.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:14 smithi155 ceph-mon[21742]: Updating smithi184:/var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T05:57:15.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:14 smithi155 ceph-mon[21742]: Updating smithi155:/var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T05:57:15.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:14 smithi155 ceph-mon[21742]: Updating smithi016:/var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T05:57:15.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:14 smithi155 ceph-mon[21742]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:15.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:14 smithi155 ceph-mon[21742]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:15.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:14 smithi155 ceph-mon[21742]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:15.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:14 smithi155 ceph-mon[21742]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:15.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:14 smithi155 ceph-mon[21742]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:15.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:14 smithi155 ceph-mon[21742]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:15.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:14 smithi155 ceph-mon[21742]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:15.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:14 smithi155 ceph-mon[21742]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-31T05:57:15.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:14 smithi155 ceph-mon[21742]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-31T05:57:15.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:14 smithi155 ceph-mon[21742]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:57:15.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:14 smithi155 ceph-mon[21742]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T05:57:15.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:14 smithi155 ceph-mon[21742]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:15.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:14 smithi184 ceph-mon[22331]: Updating smithi184:/var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T05:57:15.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:14 smithi184 ceph-mon[22331]: Updating smithi155:/var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T05:57:15.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:14 smithi184 ceph-mon[22331]: Updating smithi016:/var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T05:57:15.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:14 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:15.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:14 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:15.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:14 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:15.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:14 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:15.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:14 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:15.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:14 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:15.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:14 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:15.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:14 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-31T05:57:15.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:14 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-31T05:57:15.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:14 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:57:15.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:14 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T05:57:15.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:14 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:15.328 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:57:15 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: 2024-08-31T05:57:15.070+0000 7f78ae800640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-08-31T05:57:15.651 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T05:57:15.910 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:15 smithi016 ceph-mon[17194]: Reconfiguring mon.a (unknown last config time)... 2024-08-31T05:57:15.911 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:15 smithi016 ceph-mon[17194]: Reconfiguring daemon mon.a on smithi016 2024-08-31T05:57:15.911 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:15 smithi016 ceph-mon[17194]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:57:15.911 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:15 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:15.911 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:15 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:15.911 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:15 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-31T05:57:15.911 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:15 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-31T05:57:15.911 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:15 smithi016 ceph-mon[17194]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:57:16.047 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:15 smithi155 ceph-mon[21742]: Reconfiguring mon.a (unknown last config time)... 2024-08-31T05:57:16.048 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:15 smithi155 ceph-mon[21742]: Reconfiguring daemon mon.a on smithi016 2024-08-31T05:57:16.048 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:15 smithi155 ceph-mon[21742]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:57:16.048 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:15 smithi155 ceph-mon[21742]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:16.048 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:15 smithi155 ceph-mon[21742]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:16.048 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:15 smithi155 ceph-mon[21742]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-31T05:57:16.048 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:15 smithi155 ceph-mon[21742]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-31T05:57:16.048 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:15 smithi155 ceph-mon[21742]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:57:16.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:15 smithi184 ceph-mon[22331]: Reconfiguring mon.a (unknown last config time)... 2024-08-31T05:57:16.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:15 smithi184 ceph-mon[22331]: Reconfiguring daemon mon.a on smithi016 2024-08-31T05:57:16.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:15 smithi184 ceph-mon[22331]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:57:16.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:15 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:16.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:15 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' 2024-08-31T05:57:16.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:15 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-31T05:57:16.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:15 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-31T05:57:16.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:15 smithi184 ceph-mon[22331]: from='mgr.14150 172.21.15.16:0/1630375144' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:57:17.050 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:16 smithi155 ceph-mon[21742]: Reconfiguring mon.b (monmap changed)... 2024-08-31T05:57:17.050 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:16 smithi155 ceph-mon[21742]: Reconfiguring daemon mon.b on smithi155 2024-08-31T05:57:17.050 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:16 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/4064226353' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-08-31T05:57:17.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:16 smithi016 ceph-mon[17194]: Reconfiguring mon.b (monmap changed)... 2024-08-31T05:57:17.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:16 smithi016 ceph-mon[17194]: Reconfiguring daemon mon.b on smithi155 2024-08-31T05:57:17.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:16 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/4064226353' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-08-31T05:57:17.163 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:57:16 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: ignoring --setuser ceph since I am not root 2024-08-31T05:57:17.163 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:57:16 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: ignoring --setgroup ceph since I am not root 2024-08-31T05:57:17.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:16 smithi184 ceph-mon[22331]: Reconfiguring mon.b (monmap changed)... 2024-08-31T05:57:17.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:16 smithi184 ceph-mon[22331]: Reconfiguring daemon mon.b on smithi155 2024-08-31T05:57:17.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:16 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/4064226353' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-08-31T05:57:17.515 INFO:tasks.cephadm:Adding mgr.x on smithi016 2024-08-31T05:57:17.515 INFO:tasks.cephadm:Adding mgr.y on smithi155 2024-08-31T05:57:17.515 INFO:tasks.cephadm:Adding mgr.z on smithi184 2024-08-31T05:57:17.515 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph orch apply mgr '3;smithi016=x;smithi155=y;smithi184=z' 2024-08-31T05:57:17.756 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.c/config 2024-08-31T05:57:18.103 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:17 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/4064226353' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-08-31T05:57:18.103 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:17 smithi184 ceph-mon[22331]: mgrmap e14: x(active, since 62s) 2024-08-31T05:57:18.121 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:17 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/4064226353' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-08-31T05:57:18.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:17 smithi155 ceph-mon[21742]: mgrmap e14: x(active, since 62s) 2024-08-31T05:57:18.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:17 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/4064226353' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-08-31T05:57:18.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:17 smithi016 ceph-mon[17194]: mgrmap e14: x(active, since 62s) 2024-08-31T05:57:23.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:23 smithi184 ceph-mon[22331]: Active manager daemon x restarted 2024-08-31T05:57:23.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:23 smithi184 ceph-mon[22331]: Activating manager daemon x 2024-08-31T05:57:23.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:23 smithi184 ceph-mon[22331]: osdmap e5: 0 total, 0 up, 0 in 2024-08-31T05:57:23.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:23 smithi184 ceph-mon[22331]: mgrmap e15: x(active, starting, since 0.00646476s) 2024-08-31T05:57:23.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:23 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-31T05:57:23.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:23 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T05:57:23.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:23 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T05:57:23.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:23 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-08-31T05:57:23.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:23 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-08-31T05:57:23.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:23 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-08-31T05:57:23.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:23 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-08-31T05:57:23.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:23 smithi184 ceph-mon[22331]: Manager daemon x is now available 2024-08-31T05:57:23.621 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:23 smithi155 ceph-mon[21742]: Active manager daemon x restarted 2024-08-31T05:57:23.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:23 smithi155 ceph-mon[21742]: Activating manager daemon x 2024-08-31T05:57:23.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:23 smithi155 ceph-mon[21742]: osdmap e5: 0 total, 0 up, 0 in 2024-08-31T05:57:23.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:23 smithi155 ceph-mon[21742]: mgrmap e15: x(active, starting, since 0.00646476s) 2024-08-31T05:57:23.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:23 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-31T05:57:23.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:23 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T05:57:23.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:23 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T05:57:23.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:23 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-08-31T05:57:23.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:23 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-08-31T05:57:23.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:23 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-08-31T05:57:23.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:23 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-08-31T05:57:23.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:23 smithi155 ceph-mon[21742]: Manager daemon x is now available 2024-08-31T05:57:23.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:23 smithi016 ceph-mon[17194]: Active manager daemon x restarted 2024-08-31T05:57:23.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:23 smithi016 ceph-mon[17194]: Activating manager daemon x 2024-08-31T05:57:23.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:23 smithi016 ceph-mon[17194]: osdmap e5: 0 total, 0 up, 0 in 2024-08-31T05:57:23.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:23 smithi016 ceph-mon[17194]: mgrmap e15: x(active, starting, since 0.00646476s) 2024-08-31T05:57:23.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:23 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-31T05:57:23.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:23 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T05:57:23.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:23 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T05:57:23.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:23 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-08-31T05:57:23.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:23 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-08-31T05:57:23.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:23 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-08-31T05:57:23.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:23 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-08-31T05:57:23.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:23 smithi016 ceph-mon[17194]: Manager daemon x is now available 2024-08-31T05:57:24.179 INFO:teuthology.orchestra.run.smithi184.stdout:Scheduled mgr update... 2024-08-31T05:57:24.464 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:24 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:57:24.464 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:24 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-08-31T05:57:24.465 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:24 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-08-31T05:57:24.465 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:24 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-08-31T05:57:24.465 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:24 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-08-31T05:57:24.465 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:24 smithi184 ceph-mon[22331]: mgrmap e16: x(active, since 1.01213s) 2024-08-31T05:57:24.465 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:24 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:24.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:24 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:57:24.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:24 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-08-31T05:57:24.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:24 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-08-31T05:57:24.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:24 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-08-31T05:57:24.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:24 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-08-31T05:57:24.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:24 smithi155 ceph-mon[21742]: mgrmap e16: x(active, since 1.01213s) 2024-08-31T05:57:24.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:24 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:24.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:24 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:57:24.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:24 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-08-31T05:57:24.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:24 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-08-31T05:57:24.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:24 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-08-31T05:57:24.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:24 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-08-31T05:57:24.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:24 smithi016 ceph-mon[17194]: mgrmap e16: x(active, since 1.01213s) 2024-08-31T05:57:24.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:24 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:24.774 DEBUG:teuthology.orchestra.run.smithi155:mgr.y> sudo journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mgr.y.service 2024-08-31T05:57:24.777 DEBUG:teuthology.orchestra.run.smithi184:mgr.z> sudo journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mgr.z.service 2024-08-31T05:57:24.779 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2024-08-31T05:57:24.779 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph config set client client_mount_timeout 600 2024-08-31T05:57:24.784 INFO:tasks.cephadm:setting config [client] debug_client = 20 2024-08-31T05:57:24.784 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph config set client debug_client 20 2024-08-31T05:57:24.829 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2024-08-31T05:57:24.829 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph config set client debug_ms 1 2024-08-31T05:57:24.875 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2024-08-31T05:57:24.875 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph config set client rados_mon_op_timeout 900 2024-08-31T05:57:24.921 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2024-08-31T05:57:24.921 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph config set client rados_osd_op_timeout 900 2024-08-31T05:57:24.968 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2024-08-31T05:57:24.968 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph config set mds debug_mds 20 2024-08-31T05:57:25.016 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2024-08-31T05:57:25.016 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph config set mds debug_mds_balancer 20 2024-08-31T05:57:25.055 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:57:25 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: 2024-08-31T05:57:25.044+0000 7fde9b800640 -1 log_channel(cephadm) log [ERR] : [31/Aug/2024:05:57:25] ENGINE Error in HTTPServer.serve 2024-08-31T05:57:25.055 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:57:25 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: Traceback (most recent call last): 2024-08-31T05:57:25.056 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:57:25 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-31T05:57:25.056 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:57:25 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: self._connections.run(self.expiration_interval) 2024-08-31T05:57:25.056 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:57:25 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-31T05:57:25.056 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:57:25 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: self._run(expiration_interval) 2024-08-31T05:57:25.056 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:57:25 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-31T05:57:25.056 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:57:25 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: new_conn = self._from_server_socket(self.server.socket) 2024-08-31T05:57:25.056 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:57:25 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-31T05:57:25.056 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:57:25 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-31T05:57:25.056 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:57:25 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-31T05:57:25.056 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:57:25 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: s = self.context.wrap_socket( 2024-08-31T05:57:25.056 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:57:25 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-31T05:57:25.056 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:57:25 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: return self.sslsocket_class._create( 2024-08-31T05:57:25.057 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:57:25 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-31T05:57:25.057 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:57:25 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: self.do_handshake() 2024-08-31T05:57:25.057 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:57:25 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-31T05:57:25.057 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:57:25 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: self._sslobj.do_handshake() 2024-08-31T05:57:25.057 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:57:25 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-31T05:57:25.057 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 05:57:25 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: 2024-08-31T05:57:25.064 INFO:tasks.cephadm:setting config [mds] debug_ms = 2 2024-08-31T05:57:25.064 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph config set mds debug_ms 2 2024-08-31T05:57:25.112 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2024-08-31T05:57:25.112 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph config set mds mds_bal_fragment_size_max 10000 2024-08-31T05:57:25.157 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2024-08-31T05:57:25.157 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph config set mds mds_bal_merge_size 5 2024-08-31T05:57:25.208 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2024-08-31T05:57:25.208 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph config set mds mds_bal_split_bits 3 2024-08-31T05:57:25.259 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2024-08-31T05:57:25.259 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph config set mds mds_bal_split_size 100 2024-08-31T05:57:25.312 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2024-08-31T05:57:25.312 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph config set mds mds_debug_frag True 2024-08-31T05:57:25.332 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:25 smithi016 ceph-mon[17194]: Saving service mgr spec with placement smithi016=x;smithi155=y;smithi184=z;count:3 2024-08-31T05:57:25.332 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:25 smithi016 ceph-mon[17194]: [31/Aug/2024:05:57:24] ENGINE Bus STARTING 2024-08-31T05:57:25.333 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:25 smithi016 ceph-mon[17194]: [31/Aug/2024:05:57:24] ENGINE Serving on http://172.21.15.16:8765 2024-08-31T05:57:25.333 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:25 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:25.333 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:25 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:25.343 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2024-08-31T05:57:25.343 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph config set mds mds_debug_scatterstat True 2024-08-31T05:57:25.400 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2024-08-31T05:57:25.400 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph config set mds mds_op_complaint_time 180 2024-08-31T05:57:25.454 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2024-08-31T05:57:25.454 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph config set mds mds_verify_scatter True 2024-08-31T05:57:25.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:25 smithi184 ceph-mon[22331]: Saving service mgr spec with placement smithi016=x;smithi155=y;smithi184=z;count:3 2024-08-31T05:57:25.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:25 smithi184 ceph-mon[22331]: [31/Aug/2024:05:57:24] ENGINE Bus STARTING 2024-08-31T05:57:25.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:25 smithi184 ceph-mon[22331]: [31/Aug/2024:05:57:24] ENGINE Serving on http://172.21.15.16:8765 2024-08-31T05:57:25.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:25 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:25.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:25 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:25.518 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2024-08-31T05:57:25.518 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph config set mds osd_op_complaint_time 180 2024-08-31T05:57:25.595 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2024-08-31T05:57:25.596 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph config set mds rados_mon_op_timeout 900 2024-08-31T05:57:25.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:25 smithi155 ceph-mon[21742]: Saving service mgr spec with placement smithi016=x;smithi155=y;smithi184=z;count:3 2024-08-31T05:57:25.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:25 smithi155 ceph-mon[21742]: [31/Aug/2024:05:57:24] ENGINE Bus STARTING 2024-08-31T05:57:25.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:25 smithi155 ceph-mon[21742]: [31/Aug/2024:05:57:24] ENGINE Serving on http://172.21.15.16:8765 2024-08-31T05:57:25.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:25 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:25.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:25 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:25.663 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2024-08-31T05:57:25.663 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph config set mds rados_osd_op_timeout 900 2024-08-31T05:57:25.730 INFO:tasks.cephadm:setting config [mgr] debug_cephsqlite = 20 2024-08-31T05:57:25.730 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph config set mgr debug_cephsqlite 20 2024-08-31T05:57:25.796 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/allow_m_granularity = True 2024-08-31T05:57:25.796 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-08-31T05:57:25.854 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/dump_on_update = True 2024-08-31T05:57:25.854 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-08-31T05:57:25.887 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T05:57:25.903 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/log_level = debug 2024-08-31T05:57:25.903 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph config set mgr mgr/snap_schedule/log_level debug 2024-08-31T05:57:25.912 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T05:57:25.925 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2024-08-31T05:57:25.925 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph config set mon mon_op_complaint_time 120 2024-08-31T05:57:25.992 INFO:tasks.cephadm:setting config [mon] mon_warn_on_pool_no_app = False 2024-08-31T05:57:25.992 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph config set mon mon_warn_on_pool_no_app False 2024-08-31T05:57:25.993 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T05:57:26.007 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T05:57:26.020 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2024-08-31T05:57:26.020 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph config set osd osd_op_complaint_time 180 2024-08-31T05:57:26.093 DEBUG:tasks.cephadm:set 26 configs 2024-08-31T05:57:26.093 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '673e4aa0-675d-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi016') 2024-08-31T05:57:26.382 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T05:57:26.595 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:26 smithi016 ceph-mon[17194]: [31/Aug/2024:05:57:25] ENGINE Serving on https://172.21.15.16:7150 2024-08-31T05:57:26.595 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:26 smithi016 ceph-mon[17194]: [31/Aug/2024:05:57:25] ENGINE Bus STARTED 2024-08-31T05:57:26.595 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:26 smithi016 ceph-mon[17194]: [31/Aug/2024:05:57:25] ENGINE Error in HTTPServer.serve 2024-08-31T05:57:26.595 INFO:journalctl@ceph.mon.a.smithi016.stdout: Traceback (most recent call last): 2024-08-31T05:57:26.595 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-31T05:57:26.595 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._connections.run(self.expiration_interval) 2024-08-31T05:57:26.595 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-31T05:57:26.595 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._run(expiration_interval) 2024-08-31T05:57:26.595 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-31T05:57:26.596 INFO:journalctl@ceph.mon.a.smithi016.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-31T05:57:26.596 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-31T05:57:26.596 INFO:journalctl@ceph.mon.a.smithi016.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-31T05:57:26.596 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-31T05:57:26.596 INFO:journalctl@ceph.mon.a.smithi016.stdout: s = self.context.wrap_socket( 2024-08-31T05:57:26.596 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-31T05:57:26.596 INFO:journalctl@ceph.mon.a.smithi016.stdout: return self.sslsocket_class._create( 2024-08-31T05:57:26.596 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-31T05:57:26.596 INFO:journalctl@ceph.mon.a.smithi016.stdout: self.do_handshake() 2024-08-31T05:57:26.596 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-31T05:57:26.596 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._sslobj.do_handshake() 2024-08-31T05:57:26.596 INFO:journalctl@ceph.mon.a.smithi016.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-31T05:57:26.596 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:26 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:26.597 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:26 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:26.597 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:26 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:26.597 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:26 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:26.597 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:26 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:26.597 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:26 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:26.597 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:26 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi155", "name": "osd_memory_target"} : dispatch 2024-08-31T05:57:26.597 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:26 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi155", "name": "osd_memory_target"} : dispatch 2024-08-31T05:57:26.597 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:26 smithi016 ceph-mon[17194]: mgrmap e17: x(active, since 2s) 2024-08-31T05:57:26.597 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:26 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:26.597 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:26 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:26.597 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:26 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi184", "name": "osd_memory_target"} : dispatch 2024-08-31T05:57:26.597 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:26 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi184", "name": "osd_memory_target"} : dispatch 2024-08-31T05:57:26.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:26 smithi155 ceph-mon[21742]: [31/Aug/2024:05:57:25] ENGINE Serving on https://172.21.15.16:7150 2024-08-31T05:57:26.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:26 smithi155 ceph-mon[21742]: [31/Aug/2024:05:57:25] ENGINE Bus STARTED 2024-08-31T05:57:26.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:26 smithi155 ceph-mon[21742]: [31/Aug/2024:05:57:25] ENGINE Error in HTTPServer.serve 2024-08-31T05:57:26.872 INFO:journalctl@ceph.mon.b.smithi155.stdout: Traceback (most recent call last): 2024-08-31T05:57:26.872 INFO:journalctl@ceph.mon.b.smithi155.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-31T05:57:26.872 INFO:journalctl@ceph.mon.b.smithi155.stdout: self._connections.run(self.expiration_interval) 2024-08-31T05:57:26.873 INFO:journalctl@ceph.mon.b.smithi155.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-31T05:57:26.873 INFO:journalctl@ceph.mon.b.smithi155.stdout: self._run(expiration_interval) 2024-08-31T05:57:26.873 INFO:journalctl@ceph.mon.b.smithi155.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-31T05:57:26.873 INFO:journalctl@ceph.mon.b.smithi155.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-31T05:57:26.873 INFO:journalctl@ceph.mon.b.smithi155.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-31T05:57:26.873 INFO:journalctl@ceph.mon.b.smithi155.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-31T05:57:26.873 INFO:journalctl@ceph.mon.b.smithi155.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-31T05:57:26.873 INFO:journalctl@ceph.mon.b.smithi155.stdout: s = self.context.wrap_socket( 2024-08-31T05:57:26.873 INFO:journalctl@ceph.mon.b.smithi155.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-31T05:57:26.873 INFO:journalctl@ceph.mon.b.smithi155.stdout: return self.sslsocket_class._create( 2024-08-31T05:57:26.873 INFO:journalctl@ceph.mon.b.smithi155.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-31T05:57:26.873 INFO:journalctl@ceph.mon.b.smithi155.stdout: self.do_handshake() 2024-08-31T05:57:26.873 INFO:journalctl@ceph.mon.b.smithi155.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-31T05:57:26.873 INFO:journalctl@ceph.mon.b.smithi155.stdout: self._sslobj.do_handshake() 2024-08-31T05:57:26.874 INFO:journalctl@ceph.mon.b.smithi155.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-31T05:57:26.874 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:26 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:26.874 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:26 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:26.874 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:26 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:26.874 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:26 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:26.874 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:26 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:26.874 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:26 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:26.874 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:26 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi155", "name": "osd_memory_target"} : dispatch 2024-08-31T05:57:26.874 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:26 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi155", "name": "osd_memory_target"} : dispatch 2024-08-31T05:57:26.874 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:26 smithi155 ceph-mon[21742]: mgrmap e17: x(active, since 2s) 2024-08-31T05:57:26.874 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:26 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:26.874 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:26 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:26.874 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:26 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi184", "name": "osd_memory_target"} : dispatch 2024-08-31T05:57:26.874 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:26 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi184", "name": "osd_memory_target"} : dispatch 2024-08-31T05:57:26.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:26 smithi184 ceph-mon[22331]: [31/Aug/2024:05:57:25] ENGINE Serving on https://172.21.15.16:7150 2024-08-31T05:57:26.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:26 smithi184 ceph-mon[22331]: [31/Aug/2024:05:57:25] ENGINE Bus STARTED 2024-08-31T05:57:26.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:26 smithi184 ceph-mon[22331]: [31/Aug/2024:05:57:25] ENGINE Error in HTTPServer.serve 2024-08-31T05:57:26.973 INFO:journalctl@ceph.mon.c.smithi184.stdout: Traceback (most recent call last): 2024-08-31T05:57:26.973 INFO:journalctl@ceph.mon.c.smithi184.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-31T05:57:26.973 INFO:journalctl@ceph.mon.c.smithi184.stdout: self._connections.run(self.expiration_interval) 2024-08-31T05:57:26.973 INFO:journalctl@ceph.mon.c.smithi184.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-31T05:57:26.973 INFO:journalctl@ceph.mon.c.smithi184.stdout: self._run(expiration_interval) 2024-08-31T05:57:26.973 INFO:journalctl@ceph.mon.c.smithi184.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-31T05:57:26.973 INFO:journalctl@ceph.mon.c.smithi184.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-31T05:57:26.973 INFO:journalctl@ceph.mon.c.smithi184.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-31T05:57:26.973 INFO:journalctl@ceph.mon.c.smithi184.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-31T05:57:26.974 INFO:journalctl@ceph.mon.c.smithi184.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-31T05:57:26.974 INFO:journalctl@ceph.mon.c.smithi184.stdout: s = self.context.wrap_socket( 2024-08-31T05:57:26.974 INFO:journalctl@ceph.mon.c.smithi184.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-31T05:57:26.974 INFO:journalctl@ceph.mon.c.smithi184.stdout: return self.sslsocket_class._create( 2024-08-31T05:57:26.974 INFO:journalctl@ceph.mon.c.smithi184.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-31T05:57:26.974 INFO:journalctl@ceph.mon.c.smithi184.stdout: self.do_handshake() 2024-08-31T05:57:26.974 INFO:journalctl@ceph.mon.c.smithi184.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-31T05:57:26.974 INFO:journalctl@ceph.mon.c.smithi184.stdout: self._sslobj.do_handshake() 2024-08-31T05:57:26.974 INFO:journalctl@ceph.mon.c.smithi184.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-31T05:57:26.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:26 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:26.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:26 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:26.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:26 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:26.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:26 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:26.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:26 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:26.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:26 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:26.975 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:26 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi155", "name": "osd_memory_target"} : dispatch 2024-08-31T05:57:26.975 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:26 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi155", "name": "osd_memory_target"} : dispatch 2024-08-31T05:57:26.975 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:26 smithi184 ceph-mon[22331]: mgrmap e17: x(active, since 2s) 2024-08-31T05:57:26.975 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:26 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:26.975 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:26 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:26.975 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:26 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi184", "name": "osd_memory_target"} : dispatch 2024-08-31T05:57:26.975 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:26 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi184", "name": "osd_memory_target"} : dispatch 2024-08-31T05:57:28.789 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:28 smithi016 ceph-mon[17194]: from='client.? ' entity='client.admin' 2024-08-31T05:57:28.789 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:28 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:28.789 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:28 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:28.789 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:28 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi016", "name": "osd_memory_target"} : dispatch 2024-08-31T05:57:28.790 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:28 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi016", "name": "osd_memory_target"} : dispatch 2024-08-31T05:57:28.790 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:28 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:57:28.790 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:28 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T05:57:28.799 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:28 smithi184 ceph-mon[22331]: from='client.? ' entity='client.admin' 2024-08-31T05:57:28.799 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:28 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:28.799 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:28 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:28.799 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:28 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi016", "name": "osd_memory_target"} : dispatch 2024-08-31T05:57:28.799 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:28 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi016", "name": "osd_memory_target"} : dispatch 2024-08-31T05:57:28.799 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:28 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:57:28.799 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:28 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T05:57:28.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:28 smithi155 ceph-mon[21742]: from='client.? ' entity='client.admin' 2024-08-31T05:57:28.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:28 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:28.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:28 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:28.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:28 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi016", "name": "osd_memory_target"} : dispatch 2024-08-31T05:57:28.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:28 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi016", "name": "osd_memory_target"} : dispatch 2024-08-31T05:57:28.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:28 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:57:28.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:28 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T05:57:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:29.557+0000 7f35b3a00640 1 -- 172.21.15.16:0/1212742349 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7f35ac110760 con 0x7f35ac109b10 2024-08-31T05:57:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:29.562+0000 7f35a9600640 1 -- 172.21.15.16:0/1212742349 <== mon.1 v2:172.21.15.184:3300/0 8 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f35a000a250 con 0x7f35ac109b10 2024-08-31T05:57:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:29.562+0000 7f35a9600640 1 -- 172.21.15.16:0/1212742349 <== mon.1 v2:172.21.15.184:3300/0 9 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v13)=0 v13) ==== 114+0+0 (secure 0 0 0) 0x7f35a0013dc0 con 0x7f35ac109b10 2024-08-31T05:57:29.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:29.565+0000 7f35b3a00640 1 -- 172.21.15.16:0/1212742349 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f359403f2d0 msgr2=0x7f35940417e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:29.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:29.565+0000 7f35b3a00640 1 --2- 172.21.15.16:0/1212742349 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f359403f2d0 0x7f35940417e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3598009bd0 tx=0x7f3598005f90 comp rx=0 tx=0).stop 2024-08-31T05:57:29.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:29.565+0000 7f35b3a00640 1 -- 172.21.15.16:0/1212742349 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35ac109b10 msgr2=0x7f35ac1a0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:29.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:29.565+0000 7f35b3a00640 1 --2- 172.21.15.16:0/1212742349 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35ac109b10 0x7f35ac1a0070 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f35a000ea60 tx=0x7f35a000ef20 comp rx=0 tx=0).stop 2024-08-31T05:57:29.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:29.565+0000 7f35b3a00640 1 -- 172.21.15.16:0/1212742349 shutdown_connections 2024-08-31T05:57:29.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:29.565+0000 7f35b3a00640 1 --2- 172.21.15.16:0/1212742349 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f359403f2d0 0x7f35940417e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:29.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:29.565+0000 7f35b3a00640 1 --2- 172.21.15.16:0/1212742349 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35ac10b540 0x7f35ac1a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:29.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:29.565+0000 7f35b3a00640 1 --2- 172.21.15.16:0/1212742349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35ac10aba0 0x7f35ac1a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:29.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:29.565+0000 7f35b3a00640 1 --2- 172.21.15.16:0/1212742349 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35ac109b10 0x7f35ac1a0070 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:29.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:29.565+0000 7f35b3a00640 1 -- 172.21.15.16:0/1212742349 >> 172.21.15.16:0/1212742349 conn(0x7f35ac078ec0 msgr2=0x7f35ac10dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:29.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:29.566+0000 7f35b3a00640 1 -- 172.21.15.16:0/1212742349 shutdown_connections 2024-08-31T05:57:29.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:29.566+0000 7f35b3a00640 1 -- 172.21.15.16:0/1212742349 wait complete. 2024-08-31T05:57:29.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:29.515+0000 7ff1f3400640 1 -- 172.21.15.16:0/3954786934 <== mon.1 v2:172.21.15.184:3300/0 8 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v12)=0 v12) ==== 109+0+0 (secure 0 0 0) 0x7ff1e80317f0 con 0x7ff1fc1043a0 2024-08-31T05:57:29.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:29.518+0000 7ff1fbe00640 1 -- 172.21.15.16:0/3954786934 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff1e003f2d0 msgr2=0x7ff1e00417e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:29.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:29.518+0000 7ff1fbe00640 1 --2- 172.21.15.16:0/3954786934 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff1e003f2d0 0x7ff1e00417e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec005fd0 tx=0x7ff1ec005850 comp rx=0 tx=0).stop 2024-08-31T05:57:29.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:29.518+0000 7ff1fbe00640 1 -- 172.21.15.16:0/3954786934 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1fc1043a0 msgr2=0x7ff1fc19c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:29.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:29.518+0000 7ff1fbe00640 1 --2- 172.21.15.16:0/3954786934 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1fc1043a0 0x7ff1fc19c1d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8005ab0 tx=0x7ff1e8005000 comp rx=0 tx=0).stop 2024-08-31T05:57:29.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:29.518+0000 7ff1fbe00640 1 -- 172.21.15.16:0/3954786934 shutdown_connections 2024-08-31T05:57:29.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:29.518+0000 7ff1fbe00640 1 --2- 172.21.15.16:0/3954786934 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff1e003f2d0 0x7ff1e00417e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:29.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:29.518+0000 7ff1fbe00640 1 --2- 172.21.15.16:0/3954786934 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1fc1043a0 0x7ff1fc19c1d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:29.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:29.518+0000 7ff1fbe00640 1 --2- 172.21.15.16:0/3954786934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1fc104d40 0x7ff1fc19cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:29.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:29.518+0000 7ff1fbe00640 1 --2- 172.21.15.16:0/3954786934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1fc103310 0x7ff1fc19bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:29.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:29.518+0000 7ff1fbe00640 1 -- 172.21.15.16:0/3954786934 >> 172.21.15.16:0/3954786934 conn(0x7ff1fc0fec60 msgr2=0x7ff1fc075390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:29.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:29.519+0000 7ff1fbe00640 1 -- 172.21.15.16:0/3954786934 shutdown_connections 2024-08-31T05:57:29.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:29.519+0000 7ff1fbe00640 1 -- 172.21.15.16:0/3954786934 wait complete. 2024-08-31T05:57:29.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:29 smithi155 ceph-mon[21742]: Updating smithi016:/etc/ceph/ceph.conf 2024-08-31T05:57:29.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:29 smithi155 ceph-mon[21742]: Updating smithi155:/etc/ceph/ceph.conf 2024-08-31T05:57:29.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:29 smithi155 ceph-mon[21742]: Updating smithi184:/etc/ceph/ceph.conf 2024-08-31T05:57:29.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:29 smithi155 ceph-mon[21742]: Updating smithi155:/var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T05:57:29.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:29 smithi155 ceph-mon[21742]: Updating smithi184:/var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T05:57:29.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:29 smithi155 ceph-mon[21742]: Updating smithi016:/var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T05:57:29.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:29 smithi155 ceph-mon[21742]: Updating smithi184:/etc/ceph/ceph.client.admin.keyring 2024-08-31T05:57:29.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:29 smithi155 ceph-mon[21742]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2024-08-31T05:57:29.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:29 smithi155 ceph-mon[21742]: from='client.? ' entity='client.admin' 2024-08-31T05:57:29.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:29 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:29.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:29 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:29.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:29 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:29.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:29 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:29.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:29 smithi155 ceph-mon[21742]: from='client.? ' entity='client.admin' 2024-08-31T05:57:29.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:29 smithi016 ceph-mon[17194]: Updating smithi016:/etc/ceph/ceph.conf 2024-08-31T05:57:29.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:29 smithi016 ceph-mon[17194]: Updating smithi155:/etc/ceph/ceph.conf 2024-08-31T05:57:29.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:29 smithi016 ceph-mon[17194]: Updating smithi184:/etc/ceph/ceph.conf 2024-08-31T05:57:29.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:29 smithi016 ceph-mon[17194]: Updating smithi155:/var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T05:57:29.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:29 smithi016 ceph-mon[17194]: Updating smithi184:/var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T05:57:29.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:29 smithi016 ceph-mon[17194]: Updating smithi016:/var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T05:57:29.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:29 smithi016 ceph-mon[17194]: Updating smithi184:/etc/ceph/ceph.client.admin.keyring 2024-08-31T05:57:29.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:29 smithi016 ceph-mon[17194]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2024-08-31T05:57:29.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:29 smithi016 ceph-mon[17194]: from='client.? ' entity='client.admin' 2024-08-31T05:57:29.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:29 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:29.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:29 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:29.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:29 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:29.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:29 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:29.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:29 smithi016 ceph-mon[17194]: from='client.? ' entity='client.admin' 2024-08-31T05:57:29.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:29 smithi184 ceph-mon[22331]: Updating smithi016:/etc/ceph/ceph.conf 2024-08-31T05:57:29.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:29 smithi184 ceph-mon[22331]: Updating smithi155:/etc/ceph/ceph.conf 2024-08-31T05:57:29.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:29 smithi184 ceph-mon[22331]: Updating smithi184:/etc/ceph/ceph.conf 2024-08-31T05:57:29.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:29 smithi184 ceph-mon[22331]: Updating smithi155:/var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T05:57:29.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:29 smithi184 ceph-mon[22331]: Updating smithi184:/var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T05:57:29.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:29 smithi184 ceph-mon[22331]: Updating smithi016:/var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/config/ceph.conf 2024-08-31T05:57:29.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:29 smithi184 ceph-mon[22331]: Updating smithi184:/etc/ceph/ceph.client.admin.keyring 2024-08-31T05:57:29.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:29 smithi184 ceph-mon[22331]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2024-08-31T05:57:29.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:29 smithi184 ceph-mon[22331]: from='client.? ' entity='client.admin' 2024-08-31T05:57:29.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:29 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:29.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:29 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:29.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:29 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:29.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:29 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:29.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:29 smithi184 ceph-mon[22331]: from='client.? ' entity='client.admin' 2024-08-31T05:57:30.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.151+0000 7f9d9f400640 1 -- 172.21.15.16:0/3211008074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d98104580 msgr2=0x7f9d98104990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:30.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.151+0000 7f9d9f400640 1 --2- 172.21.15.16:0/3211008074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d98104580 0x7f9d98104990 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9d84009900 tx=0x7f9d8402a480 comp rx=0 tx=0).stop 2024-08-31T05:57:30.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.151+0000 7f9d9f400640 1 -- 172.21.15.16:0/3211008074 shutdown_connections 2024-08-31T05:57:30.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.151+0000 7f9d9f400640 1 --2- 172.21.15.16:0/3211008074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d98104f20 0x7f9d9810b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:30.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.151+0000 7f9d9f400640 1 --2- 172.21.15.16:0/3211008074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d98104580 0x7f9d98104990 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:30.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.151+0000 7f9d9f400640 1 --2- 172.21.15.16:0/3211008074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d981034f0 0x7f9d98103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:30.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.151+0000 7f9d9f400640 1 -- 172.21.15.16:0/3211008074 >> 172.21.15.16:0/3211008074 conn(0x7f9d980fee80 msgr2=0x7f9d98101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:30.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.151+0000 7f9d9f400640 1 -- 172.21.15.16:0/3211008074 shutdown_connections 2024-08-31T05:57:30.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.151+0000 7f9d9f400640 1 -- 172.21.15.16:0/3211008074 wait complete. 2024-08-31T05:57:30.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.152+0000 7f9d9f400640 1 Processor -- start 2024-08-31T05:57:30.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.152+0000 7f9d9f400640 1 -- start start 2024-08-31T05:57:30.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.153+0000 7f9d9f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d981034f0 0x7f9d98076be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:30.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.153+0000 7f9d9f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d98104580 0x7f9d980754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:30.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.153+0000 7f9d9f400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d98104f20 0x7f9d980759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:30.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.153+0000 7f9d9f400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9810f050 con 0x7f9d98104580 2024-08-31T05:57:30.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.153+0000 7f9d9f400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9810eed0 con 0x7f9d981034f0 2024-08-31T05:57:30.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.153+0000 7f9d9f400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9810f1d0 con 0x7f9d98104f20 2024-08-31T05:57:30.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.153+0000 7f9d9ca00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d981034f0 0x7f9d98076be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:30.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.153+0000 7f9d9ca00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d981034f0 0x7f9d98076be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:39484/0 (socket says 172.21.15.16:39484) 2024-08-31T05:57:30.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.153+0000 7f9d9ca00640 1 -- 172.21.15.16:0/602971571 learned_addr learned my addr 172.21.15.16:0/602971571 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T05:57:30.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.153+0000 7f9d97e00640 1 --2- 172.21.15.16:0/602971571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d98104580 0x7f9d980754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:30.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.153+0000 7f9d9d400640 1 --2- 172.21.15.16:0/602971571 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d98104f20 0x7f9d980759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:30.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.153+0000 7f9d9ca00640 1 -- 172.21.15.16:0/602971571 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d98104f20 msgr2=0x7f9d980759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:30.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.153+0000 7f9d9ca00640 1 --2- 172.21.15.16:0/602971571 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d98104f20 0x7f9d980759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:30.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.153+0000 7f9d9ca00640 1 -- 172.21.15.16:0/602971571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d98104580 msgr2=0x7f9d980754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:30.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.153+0000 7f9d9ca00640 1 --2- 172.21.15.16:0/602971571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d98104580 0x7f9d980754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:30.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.154+0000 7f9d9ca00640 1 -- 172.21.15.16:0/602971571 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d980760f0 con 0x7f9d981034f0 2024-08-31T05:57:30.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.154+0000 7f9d9d400640 1 --2- 172.21.15.16:0/602971571 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d98104f20 0x7f9d980759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T05:57:30.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.154+0000 7f9d9ca00640 1 --2- 172.21.15.16:0/602971571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d981034f0 0x7f9d98076be0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c00e9f0 tx=0x7f9d8c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:57:30.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.154+0000 7f9d95600640 1 -- 172.21.15.16:0/602971571 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c00cd80 con 0x7f9d981034f0 2024-08-31T05:57:30.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.154+0000 7f9d95600640 1 -- 172.21.15.16:0/602971571 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9d8c0054b0 con 0x7f9d981034f0 2024-08-31T05:57:30.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.154+0000 7f9d95600640 1 -- 172.21.15.16:0/602971571 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c010430 con 0x7f9d981034f0 2024-08-31T05:57:30.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.155+0000 7f9d9f400640 1 -- 172.21.15.16:0/602971571 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d981a3d40 con 0x7f9d981034f0 2024-08-31T05:57:30.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.155+0000 7f9d9f400640 1 -- 172.21.15.16:0/602971571 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d981a4140 con 0x7f9d981034f0 2024-08-31T05:57:30.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.156+0000 7f9d95600640 1 -- 172.21.15.16:0/602971571 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 17) ==== 51270+0+0 (secure 0 0 0) 0x7f9d8c004fc0 con 0x7f9d981034f0 2024-08-31T05:57:30.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.156+0000 7f9d9f400640 1 -- 172.21.15.16:0/602971571 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d9810a6d0 con 0x7f9d981034f0 2024-08-31T05:57:30.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.157+0000 7f9d95600640 1 --2- 172.21.15.16:0/602971571 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9d8003f2d0 0x7f9d800417e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:30.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.157+0000 7f9d95600640 1 -- 172.21.15.16:0/602971571 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f9d8c0540a0 con 0x7f9d981034f0 2024-08-31T05:57:30.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.157+0000 7f9d97e00640 1 --2- 172.21.15.16:0/602971571 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9d8003f2d0 0x7f9d800417e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:30.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.158+0000 7f9d97e00640 1 --2- 172.21.15.16:0/602971571 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9d8003f2d0 0x7f9d800417e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9d84005760 tx=0x7f9d84031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:57:30.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.162+0000 7f9d95600640 1 -- 172.21.15.16:0/602971571 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f9d8c01c200 con 0x7f9d981034f0 2024-08-31T05:57:30.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.387+0000 7f9d9f400640 1 -- 172.21.15.16:0/602971571 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7f9d9802e430 con 0x7f9d981034f0 2024-08-31T05:57:30.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.393+0000 7f9d95600640 1 -- 172.21.15.16:0/602971571 <== mon.2 v2:172.21.15.155:3300/0 7 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d8c01f330 con 0x7f9d981034f0 2024-08-31T05:57:30.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.393+0000 7f9d95600640 1 -- 172.21.15.16:0/602971571 <== mon.2 v2:172.21.15.155:3300/0 8 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v14)=0 v14) ==== 123+0+0 (secure 0 0 0) 0x7f9d8c01bd90 con 0x7f9d981034f0 2024-08-31T05:57:30.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.395+0000 7f9d9f400640 1 -- 172.21.15.16:0/602971571 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9d8003f2d0 msgr2=0x7f9d800417e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:30.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.395+0000 7f9d9f400640 1 --2- 172.21.15.16:0/602971571 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9d8003f2d0 0x7f9d800417e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9d84005760 tx=0x7f9d84031400 comp rx=0 tx=0).stop 2024-08-31T05:57:30.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.395+0000 7f9d9f400640 1 -- 172.21.15.16:0/602971571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d981034f0 msgr2=0x7f9d98076be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:30.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.395+0000 7f9d9f400640 1 --2- 172.21.15.16:0/602971571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d981034f0 0x7f9d98076be0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c00e9f0 tx=0x7f9d8c00edf0 comp rx=0 tx=0).stop 2024-08-31T05:57:30.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.395+0000 7f9d9f400640 1 -- 172.21.15.16:0/602971571 shutdown_connections 2024-08-31T05:57:30.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.395+0000 7f9d9f400640 1 --2- 172.21.15.16:0/602971571 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9d8003f2d0 0x7f9d800417e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:30.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.395+0000 7f9d9f400640 1 --2- 172.21.15.16:0/602971571 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d98104f20 0x7f9d980759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:30.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.395+0000 7f9d9f400640 1 --2- 172.21.15.16:0/602971571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d98104580 0x7f9d980754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:30.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.395+0000 7f9d9f400640 1 --2- 172.21.15.16:0/602971571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d981034f0 0x7f9d98076be0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:30.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.395+0000 7f9d9f400640 1 -- 172.21.15.16:0/602971571 >> 172.21.15.16:0/602971571 conn(0x7f9d980fee80 msgr2=0x7f9d981078a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:30.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.395+0000 7f9d9f400640 1 -- 172.21.15.16:0/602971571 shutdown_connections 2024-08-31T05:57:30.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:30.395+0000 7f9d9f400640 1 -- 172.21.15.16:0/602971571 wait complete. 2024-08-31T05:57:30.603 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:30 smithi016 ceph-mon[17194]: Updating smithi184:/var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/config/ceph.client.admin.keyring 2024-08-31T05:57:30.603 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:30 smithi016 ceph-mon[17194]: Updating smithi155:/var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/config/ceph.client.admin.keyring 2024-08-31T05:57:30.603 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:30 smithi016 ceph-mon[17194]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-08-31T05:57:30.603 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:30 smithi016 ceph-mon[17194]: from='client.? ' entity='client.admin' 2024-08-31T05:57:30.603 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:30 smithi016 ceph-mon[17194]: Updating smithi016:/var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/config/ceph.client.admin.keyring 2024-08-31T05:57:30.603 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:30 smithi016 ceph-mon[17194]: from='client.? ' entity='client.admin' 2024-08-31T05:57:30.603 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:30 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:30.603 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:30 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:30.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:30 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:30.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:30 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-31T05:57:30.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:30 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-31T05:57:30.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:30 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-31T05:57:30.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:30 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-31T05:57:30.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:30 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:57:30.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:30 smithi155 ceph-mon[21742]: Updating smithi184:/var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/config/ceph.client.admin.keyring 2024-08-31T05:57:30.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:30 smithi155 ceph-mon[21742]: Updating smithi155:/var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/config/ceph.client.admin.keyring 2024-08-31T05:57:30.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:30 smithi155 ceph-mon[21742]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-08-31T05:57:30.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:30 smithi155 ceph-mon[21742]: from='client.? ' entity='client.admin' 2024-08-31T05:57:30.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:30 smithi155 ceph-mon[21742]: Updating smithi016:/var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/config/ceph.client.admin.keyring 2024-08-31T05:57:30.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:30 smithi155 ceph-mon[21742]: from='client.? ' entity='client.admin' 2024-08-31T05:57:30.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:30 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:30.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:30 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:30.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:30 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:30.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:30 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-31T05:57:30.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:30 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-31T05:57:30.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:30 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-31T05:57:30.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:30 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-31T05:57:30.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:30 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:57:30.893 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:30 smithi184 ceph-mon[22331]: Updating smithi184:/var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/config/ceph.client.admin.keyring 2024-08-31T05:57:30.894 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:30 smithi184 ceph-mon[22331]: Updating smithi155:/var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/config/ceph.client.admin.keyring 2024-08-31T05:57:30.894 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:30 smithi184 ceph-mon[22331]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-08-31T05:57:30.894 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:30 smithi184 ceph-mon[22331]: from='client.? ' entity='client.admin' 2024-08-31T05:57:30.894 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:30 smithi184 ceph-mon[22331]: Updating smithi016:/var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/config/ceph.client.admin.keyring 2024-08-31T05:57:30.894 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:30 smithi184 ceph-mon[22331]: from='client.? ' entity='client.admin' 2024-08-31T05:57:30.894 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:30 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:30.894 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:30 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:30.894 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:30 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:30.894 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:30 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-31T05:57:30.894 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:30 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-31T05:57:30.894 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:30 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-31T05:57:30.894 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:30 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-31T05:57:30.894 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:30 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:57:30.964 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T05:57:31.605 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T05:57:31.842 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:31 smithi016 ceph-mon[17194]: Deploying daemon mgr.z on smithi184 2024-08-31T05:57:31.871 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:31 smithi155 ceph-mon[21742]: Deploying daemon mgr.z on smithi184 2024-08-31T05:57:31.876 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:31 smithi184 ceph-mon[22331]: Deploying daemon mgr.z on smithi184 2024-08-31T05:57:32.064 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T05:57:32.972 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Aug 31 05:57:32 smithi184 systemd[1]: Started Ceph mgr.z for 673e4aa0-675d-11ef-bcd4-c7b262605968. 2024-08-31T05:57:33.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.658+0000 7fa273600640 1 -- 172.21.15.16:0/668268468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa26c103920 msgr2=0x7fa26c103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:33.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.658+0000 7fa273600640 1 --2- 172.21.15.16:0/668268468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa26c103920 0x7fa26c103d30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa260009900 tx=0x7fa26002a410 comp rx=0 tx=0).stop 2024-08-31T05:57:33.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.658+0000 7fa273600640 1 -- 172.21.15.16:0/668268468 shutdown_connections 2024-08-31T05:57:33.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.658+0000 7fa273600640 1 --2- 172.21.15.16:0/668268468 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa26c109260 0x7fa26c109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:33.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.658+0000 7fa273600640 1 --2- 172.21.15.16:0/668268468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa26c109bb0 0x7fa26c1103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:33.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.658+0000 7fa273600640 1 --2- 172.21.15.16:0/668268468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa26c103920 0x7fa26c103d30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:33.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.658+0000 7fa273600640 1 -- 172.21.15.16:0/668268468 >> 172.21.15.16:0/668268468 conn(0x7fa26c0ff3b0 msgr2=0x7fa26c101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:33.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.658+0000 7fa273600640 1 -- 172.21.15.16:0/668268468 shutdown_connections 2024-08-31T05:57:33.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.658+0000 7fa273600640 1 -- 172.21.15.16:0/668268468 wait complete. 2024-08-31T05:57:33.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.659+0000 7fa273600640 1 Processor -- start 2024-08-31T05:57:33.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.659+0000 7fa273600640 1 -- start start 2024-08-31T05:57:33.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.659+0000 7fa273600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa26c103920 0x7fa26c19a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:33.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.659+0000 7fa273600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa26c109260 0x7fa26c19a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:33.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.660+0000 7fa273600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa26c109bb0 0x7fa26c19fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:33.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.660+0000 7fa273600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa26c1138c0 con 0x7fa26c109260 2024-08-31T05:57:33.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.660+0000 7fa273600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa26c113740 con 0x7fa26c103920 2024-08-31T05:57:33.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.660+0000 7fa273600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa26c113a40 con 0x7fa26c109bb0 2024-08-31T05:57:33.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.660+0000 7fa270c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa26c103920 0x7fa26c19a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:33.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.660+0000 7fa270c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa26c103920 0x7fa26c19a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:39526/0 (socket says 172.21.15.16:39526) 2024-08-31T05:57:33.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.660+0000 7fa270c00640 1 -- 172.21.15.16:0/1894480930 learned_addr learned my addr 172.21.15.16:0/1894480930 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T05:57:33.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.660+0000 7fa26be00640 1 --2- 172.21.15.16:0/1894480930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa26c109260 0x7fa26c19a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:33.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.660+0000 7fa271600640 1 --2- 172.21.15.16:0/1894480930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa26c109bb0 0x7fa26c19fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:33.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.660+0000 7fa270c00640 1 -- 172.21.15.16:0/1894480930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa26c109bb0 msgr2=0x7fa26c19fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:33.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.660+0000 7fa270c00640 1 --2- 172.21.15.16:0/1894480930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa26c109bb0 0x7fa26c19fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:33.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.660+0000 7fa270c00640 1 -- 172.21.15.16:0/1894480930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa26c109260 msgr2=0x7fa26c19a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:33.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.660+0000 7fa270c00640 1 --2- 172.21.15.16:0/1894480930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa26c109260 0x7fa26c19a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:33.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.660+0000 7fa270c00640 1 -- 172.21.15.16:0/1894480930 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa26c1a0240 con 0x7fa26c103920 2024-08-31T05:57:33.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.660+0000 7fa271600640 1 --2- 172.21.15.16:0/1894480930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa26c109bb0 0x7fa26c19fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T05:57:33.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.661+0000 7fa270c00640 1 --2- 172.21.15.16:0/1894480930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa26c103920 0x7fa26c19a2d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa260005ab0 tx=0x7fa260003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:57:33.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.661+0000 7fa269600640 1 -- 172.21.15.16:0/1894480930 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26003f070 con 0x7fa26c103920 2024-08-31T05:57:33.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.661+0000 7fa273600640 1 -- 172.21.15.16:0/1894480930 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa26c1a04d0 con 0x7fa26c103920 2024-08-31T05:57:33.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.661+0000 7fa269600640 1 -- 172.21.15.16:0/1894480930 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa260031b60 con 0x7fa26c103920 2024-08-31T05:57:33.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.661+0000 7fa269600640 1 -- 172.21.15.16:0/1894480930 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa260035030 con 0x7fa26c103920 2024-08-31T05:57:33.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.661+0000 7fa273600640 1 -- 172.21.15.16:0/1894480930 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa26c075390 con 0x7fa26c103920 2024-08-31T05:57:33.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.662+0000 7fa269600640 1 -- 172.21.15.16:0/1894480930 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 17) ==== 51270+0+0 (secure 0 0 0) 0x7fa260042070 con 0x7fa26c103920 2024-08-31T05:57:33.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.663+0000 7fa273600640 1 -- 172.21.15.16:0/1894480930 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa26c104ba0 con 0x7fa26c103920 2024-08-31T05:57:33.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.663+0000 7fa269600640 1 --2- 172.21.15.16:0/1894480930 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa25403f2d0 0x7fa2540417e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:33.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.663+0000 7fa269600640 1 -- 172.21.15.16:0/1894480930 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fa260074370 con 0x7fa26c103920 2024-08-31T05:57:33.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.663+0000 7fa26be00640 1 --2- 172.21.15.16:0/1894480930 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa25403f2d0 0x7fa2540417e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:33.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.664+0000 7fa26be00640 1 --2- 172.21.15.16:0/1894480930 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa25403f2d0 0x7fa2540417e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa26c102ab0 tx=0x7fa258012040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:57:33.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.668+0000 7fa269600640 1 -- 172.21.15.16:0/1894480930 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fa26004c350 con 0x7fa26c103920 2024-08-31T05:57:33.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.589+0000 7fe483e00640 1 -- 172.21.15.16:0/4049300105 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe48410fcb0 msgr2=0x7fe4841120f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:33.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.589+0000 7fe483e00640 1 --2- 172.21.15.16:0/4049300105 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe48410fcb0 0x7fe4841120f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe47c009ca0 tx=0x7fe47c02a8c0 comp rx=0 tx=0).stop 2024-08-31T05:57:33.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.589+0000 7fe483e00640 1 -- 172.21.15.16:0/4049300105 shutdown_connections 2024-08-31T05:57:33.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.589+0000 7fe483e00640 1 --2- 172.21.15.16:0/4049300105 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe48410fcb0 0x7fe4841120f0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:33.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.589+0000 7fe483e00640 1 --2- 172.21.15.16:0/4049300105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe484075ea0 0x7fe48410f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:33.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.589+0000 7fe483e00640 1 --2- 172.21.15.16:0/4049300105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe484075550 0x7fe484075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:33.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.589+0000 7fe483e00640 1 -- 172.21.15.16:0/4049300105 >> 172.21.15.16:0/4049300105 conn(0x7fe4840ff1e0 msgr2=0x7fe484101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:33.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.589+0000 7fe483e00640 1 -- 172.21.15.16:0/4049300105 shutdown_connections 2024-08-31T05:57:33.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.589+0000 7fe483e00640 1 -- 172.21.15.16:0/4049300105 wait complete. 2024-08-31T05:57:33.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.591+0000 7fe483e00640 1 Processor -- start 2024-08-31T05:57:33.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.591+0000 7fe483e00640 1 -- start start 2024-08-31T05:57:33.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.591+0000 7fe483e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe484075550 0x7fe48419c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:33.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.591+0000 7fe483e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe484075ea0 0x7fe48419c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:33.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.591+0000 7fe483e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe48410fcb0 0x7fe4841a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:33.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.591+0000 7fe483e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe484115870 con 0x7fe484075ea0 2024-08-31T05:57:33.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.591+0000 7fe483e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe4841156f0 con 0x7fe48410fcb0 2024-08-31T05:57:33.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.591+0000 7fe483e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe4841159f0 con 0x7fe484075550 2024-08-31T05:57:33.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.592+0000 7fe482a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe484075550 0x7fe48419c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:33.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.592+0000 7fe482a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe484075550 0x7fe48419c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:51712/0 (socket says 172.21.15.16:51712) 2024-08-31T05:57:33.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.592+0000 7fe482a00640 1 -- 172.21.15.16:0/2606335677 learned_addr learned my addr 172.21.15.16:0/2606335677 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T05:57:33.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.592+0000 7fe482000640 1 --2- 172.21.15.16:0/2606335677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe484075ea0 0x7fe48419c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:33.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.592+0000 7fe483400640 1 --2- 172.21.15.16:0/2606335677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe48410fcb0 0x7fe4841a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:33.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.592+0000 7fe482a00640 1 -- 172.21.15.16:0/2606335677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe48410fcb0 msgr2=0x7fe4841a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:33.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.592+0000 7fe482a00640 1 --2- 172.21.15.16:0/2606335677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe48410fcb0 0x7fe4841a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:33.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.592+0000 7fe482a00640 1 -- 172.21.15.16:0/2606335677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe484075ea0 msgr2=0x7fe48419c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:33.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.592+0000 7fe482a00640 1 --2- 172.21.15.16:0/2606335677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe484075ea0 0x7fe48419c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:33.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.592+0000 7fe482a00640 1 -- 172.21.15.16:0/2606335677 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4841a2190 con 0x7fe484075550 2024-08-31T05:57:33.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.592+0000 7fe482000640 1 --2- 172.21.15.16:0/2606335677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe484075ea0 0x7fe48419c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T05:57:33.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.592+0000 7fe482a00640 1 --2- 172.21.15.16:0/2606335677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe484075550 0x7fe48419c220 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe47400d950 tx=0x7fe47400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:57:33.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.593+0000 7fe47b400640 1 -- 172.21.15.16:0/2606335677 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe474017070 con 0x7fe484075550 2024-08-31T05:57:33.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.593+0000 7fe47b400640 1 -- 172.21.15.16:0/2606335677 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe474009c30 con 0x7fe484075550 2024-08-31T05:57:33.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.593+0000 7fe47b400640 1 -- 172.21.15.16:0/2606335677 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47400f620 con 0x7fe484075550 2024-08-31T05:57:33.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.593+0000 7fe483e00640 1 -- 172.21.15.16:0/2606335677 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4841a2480 con 0x7fe484075550 2024-08-31T05:57:33.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.593+0000 7fe483e00640 1 -- 172.21.15.16:0/2606335677 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4841a8be0 con 0x7fe484075550 2024-08-31T05:57:33.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.594+0000 7fe47b400640 1 -- 172.21.15.16:0/2606335677 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 17) ==== 51270+0+0 (secure 0 0 0) 0x7fe47400f7c0 con 0x7fe484075550 2024-08-31T05:57:33.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.594+0000 7fe483e00640 1 -- 172.21.15.16:0/2606335677 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe484075960 con 0x7fe484075550 2024-08-31T05:57:33.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.595+0000 7fe47b400640 1 --2- 172.21.15.16:0/2606335677 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe46803f2d0 0x7fe4680417e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:33.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.595+0000 7fe47b400640 1 -- 172.21.15.16:0/2606335677 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fe4740537b0 con 0x7fe484075550 2024-08-31T05:57:33.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.595+0000 7fe482000640 1 --2- 172.21.15.16:0/2606335677 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe46803f2d0 0x7fe4680417e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:33.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.596+0000 7fe482000640 1 --2- 172.21.15.16:0/2606335677 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe46803f2d0 0x7fe4680417e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe484111e90 tx=0x7fe470005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:57:33.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.600+0000 7fe47b400640 1 -- 172.21.15.16:0/2606335677 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fe474019310 con 0x7fe484075550 2024-08-31T05:57:33.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.802+0000 7fe483e00640 1 -- 172.21.15.16:0/2606335677 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7fe48419d350 con 0x7fe484075550 2024-08-31T05:57:33.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.806+0000 7fe47b400640 1 -- 172.21.15.16:0/2606335677 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v15)=0 v15) ==== 117+0+0 (secure 0 0 0) 0x7fe474009de0 con 0x7fe484075550 2024-08-31T05:57:33.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.810+0000 7fe483e00640 1 -- 172.21.15.16:0/2606335677 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe46803f2d0 msgr2=0x7fe4680417e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:33.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.810+0000 7fe483e00640 1 --2- 172.21.15.16:0/2606335677 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe46803f2d0 0x7fe4680417e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe484111e90 tx=0x7fe470005f90 comp rx=0 tx=0).stop 2024-08-31T05:57:33.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.810+0000 7fe483e00640 1 -- 172.21.15.16:0/2606335677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe484075550 msgr2=0x7fe48419c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:33.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.810+0000 7fe483e00640 1 --2- 172.21.15.16:0/2606335677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe484075550 0x7fe48419c220 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe47400d950 tx=0x7fe47400de10 comp rx=0 tx=0).stop 2024-08-31T05:57:33.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.810+0000 7fe483e00640 1 -- 172.21.15.16:0/2606335677 shutdown_connections 2024-08-31T05:57:33.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.810+0000 7fe483e00640 1 --2- 172.21.15.16:0/2606335677 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe46803f2d0 0x7fe4680417e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:33.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.810+0000 7fe483e00640 1 --2- 172.21.15.16:0/2606335677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe48410fcb0 0x7fe4841a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:33.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.810+0000 7fe483e00640 1 --2- 172.21.15.16:0/2606335677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe484075ea0 0x7fe48419c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:33.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.810+0000 7fe483e00640 1 --2- 172.21.15.16:0/2606335677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe484075550 0x7fe48419c220 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:33.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.810+0000 7fe483e00640 1 -- 172.21.15.16:0/2606335677 >> 172.21.15.16:0/2606335677 conn(0x7fe4840ff1e0 msgr2=0x7fe4841097c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:33.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.810+0000 7fe483e00640 1 -- 172.21.15.16:0/2606335677 shutdown_connections 2024-08-31T05:57:33.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.810+0000 7fe483e00640 1 -- 172.21.15.16:0/2606335677 wait complete. 2024-08-31T05:57:33.835 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:33 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:33.835 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:33 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:33.835 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:33 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:33.835 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:33 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-31T05:57:33.835 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:33 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-31T05:57:33.836 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:33 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-31T05:57:33.836 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:33 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-31T05:57:33.836 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:33 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:57:33.836 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:33 smithi016 ceph-mon[17194]: Deploying daemon mgr.y on smithi155 2024-08-31T05:57:33.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.873+0000 7fa273600640 1 -- 172.21.15.16:0/1894480930 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7fa26c10b9e0 con 0x7fa26c103920 2024-08-31T05:57:33.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.878+0000 7fa269600640 1 -- 172.21.15.16:0/1894480930 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v16)=0 v16) ==== 108+0+0 (secure 0 0 0) 0x7fa260031d10 con 0x7fa26c103920 2024-08-31T05:57:33.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.880+0000 7fa273600640 1 -- 172.21.15.16:0/1894480930 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa25403f2d0 msgr2=0x7fa2540417e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:33.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.880+0000 7fa273600640 1 --2- 172.21.15.16:0/1894480930 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa25403f2d0 0x7fa2540417e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa26c102ab0 tx=0x7fa258012040 comp rx=0 tx=0).stop 2024-08-31T05:57:33.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.880+0000 7fa273600640 1 -- 172.21.15.16:0/1894480930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa26c103920 msgr2=0x7fa26c19a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:33.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.880+0000 7fa273600640 1 --2- 172.21.15.16:0/1894480930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa26c103920 0x7fa26c19a2d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa260005ab0 tx=0x7fa260003c10 comp rx=0 tx=0).stop 2024-08-31T05:57:33.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.881+0000 7fa273600640 1 -- 172.21.15.16:0/1894480930 shutdown_connections 2024-08-31T05:57:33.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.881+0000 7fa273600640 1 --2- 172.21.15.16:0/1894480930 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa25403f2d0 0x7fa2540417e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:33.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.881+0000 7fa273600640 1 --2- 172.21.15.16:0/1894480930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa26c109bb0 0x7fa26c19fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:33.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.881+0000 7fa273600640 1 --2- 172.21.15.16:0/1894480930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa26c109260 0x7fa26c19a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:33.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.881+0000 7fa273600640 1 --2- 172.21.15.16:0/1894480930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa26c103920 0x7fa26c19a2d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:33.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.881+0000 7fa273600640 1 -- 172.21.15.16:0/1894480930 >> 172.21.15.16:0/1894480930 conn(0x7fa26c0ff3b0 msgr2=0x7fa26c1088d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:33.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.881+0000 7fa273600640 1 -- 172.21.15.16:0/1894480930 shutdown_connections 2024-08-31T05:57:33.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:33.881+0000 7fa273600640 1 -- 172.21.15.16:0/1894480930 wait complete. 2024-08-31T05:57:34.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:33 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:34.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:33 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:34.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:33 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:34.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:33 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-31T05:57:34.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:33 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-31T05:57:34.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:33 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-31T05:57:34.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:33 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-31T05:57:34.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:33 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:57:34.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:33 smithi155 ceph-mon[21742]: Deploying daemon mgr.y on smithi155 2024-08-31T05:57:34.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:33 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:34.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:33 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:34.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:33 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:34.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:33 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-31T05:57:34.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:33 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-31T05:57:34.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:33 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-31T05:57:34.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:33 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-31T05:57:34.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:33 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:57:34.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:33 smithi184 ceph-mon[22331]: Deploying daemon mgr.y on smithi155 2024-08-31T05:57:34.308 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '673e4aa0-675d-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi016') 2024-08-31T05:57:34.308 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '673e4aa0-675d-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi016') 2024-08-31T05:57:34.308 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '673e4aa0-675d-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi016') 2024-08-31T05:57:34.308 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '673e4aa0-675d-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi016') 2024-08-31T05:57:34.308 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '673e4aa0-675d-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi016') 2024-08-31T05:57:34.809 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 31 05:57:34 smithi155 systemd[1]: Starting Ceph mgr.y for 673e4aa0-675d-11ef-bcd4-c7b262605968... 2024-08-31T05:57:35.076 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:34 smithi155 ceph-mon[21742]: from='client.? ' entity='client.admin' 2024-08-31T05:57:35.076 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:34 smithi155 ceph-mon[21742]: from='client.? ' entity='client.admin' 2024-08-31T05:57:35.076 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 31 05:57:34 smithi155 podman[24042]: 2024-08-31 05:57:34.722044559 +0000 UTC m=+0.015558583 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T05:57:35.076 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 31 05:57:34 smithi155 podman[24042]: 2024-08-31 05:57:34.834871015 +0000 UTC m=+0.128385070 container create 2368cdbe66a423b1322f71968490f8f68a56cf38ebc11ccb11518a5d98678339 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-y, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611) 2024-08-31T05:57:35.076 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 31 05:57:35 smithi155 podman[24042]: 2024-08-31 05:57:35.037186388 +0000 UTC m=+0.330700410 container init 2368cdbe66a423b1322f71968490f8f68a56cf38ebc11ccb11518a5d98678339 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-y, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.1, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.vendor=CentOS) 2024-08-31T05:57:35.076 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 31 05:57:35 smithi155 podman[24042]: 2024-08-31 05:57:35.039777383 +0000 UTC m=+0.333291411 container start 2368cdbe66a423b1322f71968490f8f68a56cf38ebc11ccb11518a5d98678339 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-y, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.1) 2024-08-31T05:57:35.076 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 31 05:57:35 smithi155 ceph-mgr[24061]: -- 172.21.15.155:0/1357944776 <== mon.2 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55af2d8df0e0 con 0x55af3267e480 2024-08-31T05:57:35.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:34 smithi016 ceph-mon[17194]: from='client.? ' entity='client.admin' 2024-08-31T05:57:35.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:34 smithi016 ceph-mon[17194]: from='client.? ' entity='client.admin' 2024-08-31T05:57:35.208 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T05:57:35.211 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T05:57:35.213 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T05:57:35.220 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T05:57:35.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:34 smithi184 ceph-mon[22331]: from='client.? ' entity='client.admin' 2024-08-31T05:57:35.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:34 smithi184 ceph-mon[22331]: from='client.? ' entity='client.admin' 2024-08-31T05:57:35.239 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T05:57:35.251 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T05:57:35.251 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T05:57:35.259 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T05:57:35.353 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 31 05:57:35 smithi155 bash[24042]: 2368cdbe66a423b1322f71968490f8f68a56cf38ebc11ccb11518a5d98678339 2024-08-31T05:57:35.353 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 31 05:57:35 smithi155 systemd[1]: Started Ceph mgr.y for 673e4aa0-675d-11ef-bcd4-c7b262605968. 2024-08-31T05:57:35.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.577+0000 7f9334000640 1 -- 172.21.15.16:0/1857372246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f932c10b460 msgr2=0x7f932c10b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:35.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.577+0000 7f9334000640 1 --2- 172.21.15.16:0/1857372246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f932c10b460 0x7f932c10b870 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9318009900 tx=0x7f931802a410 comp rx=0 tx=0).stop 2024-08-31T05:57:35.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.577+0000 7f9334000640 1 -- 172.21.15.16:0/1857372246 shutdown_connections 2024-08-31T05:57:35.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.577+0000 7f9334000640 1 --2- 172.21.15.16:0/1857372246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f932c10bdb0 0x7f932c1125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:35.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.577+0000 7f9334000640 1 --2- 172.21.15.16:0/1857372246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f932c10b460 0x7f932c10b870 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:35.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.577+0000 7f9334000640 1 --2- 172.21.15.16:0/1857372246 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f932c105b20 0x7f932c105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:35.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.577+0000 7f9334000640 1 -- 172.21.15.16:0/1857372246 >> 172.21.15.16:0/1857372246 conn(0x7f932c1015b0 msgr2=0x7f932c103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:35.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.577+0000 7f9334000640 1 -- 172.21.15.16:0/1857372246 shutdown_connections 2024-08-31T05:57:35.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.577+0000 7f9334000640 1 -- 172.21.15.16:0/1857372246 wait complete. 2024-08-31T05:57:35.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.579+0000 7f9334000640 1 Processor -- start 2024-08-31T05:57:35.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.579+0000 7f9334000640 1 -- start start 2024-08-31T05:57:35.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.579+0000 7f9334000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f932c105b20 0x7f932c19c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:35.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.579+0000 7f9334000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f932c10b460 0x7f932c19ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:35.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.579+0000 7f9334000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f932c10bdb0 0x7f932c1a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:35.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.579+0000 7f9334000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f932c115a70 con 0x7f932c10b460 2024-08-31T05:57:35.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.580+0000 7f9334000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f932c1158f0 con 0x7f932c10bdb0 2024-08-31T05:57:35.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.580+0000 7f9334000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f932c115bf0 con 0x7f932c105b20 2024-08-31T05:57:35.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.580+0000 7f9331600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f932c105b20 0x7f932c19c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:35.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.580+0000 7f9330c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f932c10b460 0x7f932c19ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:35.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.580+0000 7f9331600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f932c105b20 0x7f932c19c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:51766/0 (socket says 172.21.15.16:51766) 2024-08-31T05:57:35.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.580+0000 7f9331600640 1 -- 172.21.15.16:0/1710402022 learned_addr learned my addr 172.21.15.16:0/1710402022 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T05:57:35.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.580+0000 7f9332000640 1 --2- 172.21.15.16:0/1710402022 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f932c10bdb0 0x7f932c1a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:35.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.580+0000 7f9331600640 1 -- 172.21.15.16:0/1710402022 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f932c10bdb0 msgr2=0x7f932c1a1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:35.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.580+0000 7f9331600640 1 --2- 172.21.15.16:0/1710402022 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f932c10bdb0 0x7f932c1a1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:35.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.580+0000 7f9331600640 1 -- 172.21.15.16:0/1710402022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f932c10b460 msgr2=0x7f932c19ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:35.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.580+0000 7f9331600640 1 --2- 172.21.15.16:0/1710402022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f932c10b460 0x7f932c19ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:35.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.580+0000 7f9331600640 1 -- 172.21.15.16:0/1710402022 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f932c1a23f0 con 0x7f932c105b20 2024-08-31T05:57:35.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.580+0000 7f9330c00640 1 --2- 172.21.15.16:0/1710402022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f932c10b460 0x7f932c19ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T05:57:35.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.581+0000 7f9331600640 1 --2- 172.21.15.16:0/1710402022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f932c105b20 0x7f932c19c480 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f932000e9f0 tx=0x7f932000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:57:35.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.581+0000 7f932a000640 1 -- 172.21.15.16:0/1710402022 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932000ccc0 con 0x7f932c105b20 2024-08-31T05:57:35.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.581+0000 7f932a000640 1 -- 172.21.15.16:0/1710402022 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f932000ce60 con 0x7f932c105b20 2024-08-31T05:57:35.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.581+0000 7f9334000640 1 -- 172.21.15.16:0/1710402022 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f932c1a26e0 con 0x7f932c105b20 2024-08-31T05:57:35.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.581+0000 7f932a000640 1 -- 172.21.15.16:0/1710402022 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9320010430 con 0x7f932c105b20 2024-08-31T05:57:35.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.581+0000 7f9334000640 1 -- 172.21.15.16:0/1710402022 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f932c1a8e40 con 0x7f932c105b20 2024-08-31T05:57:35.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.582+0000 7f9334000640 1 -- 172.21.15.16:0/1710402022 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f932c105f30 con 0x7f932c105b20 2024-08-31T05:57:35.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.583+0000 7f932a000640 1 -- 172.21.15.16:0/1710402022 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 17) ==== 51270+0+0 (secure 0 0 0) 0x7f9320005020 con 0x7f932c105b20 2024-08-31T05:57:35.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.583+0000 7f932a000640 1 --2- 172.21.15.16:0/1710402022 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f931403f2d0 0x7f93140417e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:35.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.583+0000 7f932a000640 1 -- 172.21.15.16:0/1710402022 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f9320053f80 con 0x7f932c105b20 2024-08-31T05:57:35.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.583+0000 7f9330c00640 1 --2- 172.21.15.16:0/1710402022 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f931403f2d0 0x7f93140417e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:35.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.584+0000 7f9330c00640 1 --2- 172.21.15.16:0/1710402022 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f931403f2d0 0x7f93140417e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9318009230 tx=0x7f9318031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:57:35.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.588+0000 7f932a000640 1 -- 172.21.15.16:0/1710402022 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f9320016330 con 0x7f932c105b20 2024-08-31T05:57:35.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.786+0000 7f9334000640 1 -- 172.21.15.16:0/1710402022 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f932c10dbe0 con 0x7f932c105b20 2024-08-31T05:57:35.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.792+0000 7f932a000640 1 -- 172.21.15.16:0/1710402022 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v17)=0 v17) ==== 106+0+0 (secure 0 0 0) 0x7f932000a250 con 0x7f932c105b20 2024-08-31T05:57:35.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.794+0000 7f9334000640 1 -- 172.21.15.16:0/1710402022 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f931403f2d0 msgr2=0x7f93140417e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:35.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.794+0000 7f9334000640 1 --2- 172.21.15.16:0/1710402022 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f931403f2d0 0x7f93140417e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9318009230 tx=0x7f9318031400 comp rx=0 tx=0).stop 2024-08-31T05:57:35.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.794+0000 7f9334000640 1 -- 172.21.15.16:0/1710402022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f932c105b20 msgr2=0x7f932c19c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:35.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.794+0000 7f9334000640 1 --2- 172.21.15.16:0/1710402022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f932c105b20 0x7f932c19c480 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f932000e9f0 tx=0x7f932000edf0 comp rx=0 tx=0).stop 2024-08-31T05:57:35.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.794+0000 7f9334000640 1 -- 172.21.15.16:0/1710402022 shutdown_connections 2024-08-31T05:57:35.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.794+0000 7f9334000640 1 --2- 172.21.15.16:0/1710402022 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f931403f2d0 0x7f93140417e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:35.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.794+0000 7f9334000640 1 --2- 172.21.15.16:0/1710402022 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f932c10bdb0 0x7f932c1a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:35.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.794+0000 7f9334000640 1 --2- 172.21.15.16:0/1710402022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f932c10b460 0x7f932c19ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:35.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.794+0000 7f9334000640 1 --2- 172.21.15.16:0/1710402022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f932c105b20 0x7f932c19c480 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:35.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.794+0000 7f9334000640 1 -- 172.21.15.16:0/1710402022 >> 172.21.15.16:0/1710402022 conn(0x7f932c1015b0 msgr2=0x7f932c1039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:35.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.795+0000 7f9334000640 1 -- 172.21.15.16:0/1710402022 shutdown_connections 2024-08-31T05:57:35.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:35.795+0000 7f9334000640 1 -- 172.21.15.16:0/1710402022 wait complete. 2024-08-31T05:57:36.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.292+0000 7f7002800640 1 -- 172.21.15.16:0/1074215217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ffc10b2a0 msgr2=0x7f6ffc10b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:36.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.292+0000 7f7002800640 1 --2- 172.21.15.16:0/1074215217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ffc10b2a0 0x7f6ffc10b6b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8009900 tx=0x7f6fe802a410 comp rx=0 tx=0).stop 2024-08-31T05:57:36.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.293+0000 7f7002800640 1 -- 172.21.15.16:0/1074215217 shutdown_connections 2024-08-31T05:57:36.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.293+0000 7f7002800640 1 --2- 172.21.15.16:0/1074215217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ffc10bbf0 0x7f6ffc112420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:36.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.293+0000 7f7002800640 1 --2- 172.21.15.16:0/1074215217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ffc10b2a0 0x7f6ffc10b6b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:36.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.293+0000 7f7002800640 1 --2- 172.21.15.16:0/1074215217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ffc105960 0x7f6ffc105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:36.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.293+0000 7f7002800640 1 -- 172.21.15.16:0/1074215217 >> 172.21.15.16:0/1074215217 conn(0x7f6ffc101410 msgr2=0x7f6ffc103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:36.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.293+0000 7f7002800640 1 -- 172.21.15.16:0/1074215217 shutdown_connections 2024-08-31T05:57:36.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.293+0000 7f7002800640 1 -- 172.21.15.16:0/1074215217 wait complete. 2024-08-31T05:57:36.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.294+0000 7f7002800640 1 Processor -- start 2024-08-31T05:57:36.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.294+0000 7f7002800640 1 -- start start 2024-08-31T05:57:36.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.294+0000 7f7002800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ffc105960 0x7f6ffc19c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:36.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.294+0000 7f7002800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ffc10b2a0 0x7f6ffc19c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:36.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.294+0000 7f7002800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ffc10bbf0 0x7f6ffc1a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:36.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.294+0000 7f7002800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc1158b0 con 0x7f6ffc10b2a0 2024-08-31T05:57:36.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.294+0000 7f7002800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc115730 con 0x7f6ffc10bbf0 2024-08-31T05:57:36.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.294+0000 7f7002800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc115a30 con 0x7f6ffc105960 2024-08-31T05:57:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.294+0000 7f7001400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ffc105960 0x7f6ffc19c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.295+0000 7f7001400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ffc105960 0x7f6ffc19c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:37012/0 (socket says 172.21.15.16:37012) 2024-08-31T05:57:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.295+0000 7f7001400640 1 -- 172.21.15.16:0/3515177102 learned_addr learned my addr 172.21.15.16:0/3515177102 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T05:57:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.295+0000 7f7001e00640 1 --2- 172.21.15.16:0/3515177102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ffc10bbf0 0x7f6ffc1a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.295+0000 7f7000a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ffc10b2a0 0x7f6ffc19c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.295+0000 7f7001e00640 1 -- 172.21.15.16:0/3515177102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ffc105960 msgr2=0x7f6ffc19c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.295+0000 7f7001e00640 1 --2- 172.21.15.16:0/3515177102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ffc105960 0x7f6ffc19c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.295+0000 7f7001e00640 1 -- 172.21.15.16:0/3515177102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ffc10b2a0 msgr2=0x7f6ffc19c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.295+0000 7f7001e00640 1 --2- 172.21.15.16:0/3515177102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ffc10b2a0 0x7f6ffc19c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.295+0000 7f7001e00640 1 -- 172.21.15.16:0/3515177102 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ffc1a2270 con 0x7f6ffc10bbf0 2024-08-31T05:57:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.295+0000 7f7001e00640 1 --2- 172.21.15.16:0/3515177102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ffc10bbf0 0x7f6ffc1a1b40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6ff400ec50 tx=0x7f6ff400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:57:36.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.296+0000 7f6ffa000640 1 -- 172.21.15.16:0/3515177102 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff4003b70 con 0x7f6ffc10bbf0 2024-08-31T05:57:36.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.296+0000 7f6ffa000640 1 -- 172.21.15.16:0/3515177102 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ff4003d10 con 0x7f6ffc10bbf0 2024-08-31T05:57:36.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.296+0000 7f7002800640 1 -- 172.21.15.16:0/3515177102 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ffc1a2560 con 0x7f6ffc10bbf0 2024-08-31T05:57:36.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.296+0000 7f6ffa000640 1 -- 172.21.15.16:0/3515177102 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff4010830 con 0x7f6ffc10bbf0 2024-08-31T05:57:36.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.296+0000 7f7002800640 1 -- 172.21.15.16:0/3515177102 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ffc1a8cc0 con 0x7f6ffc10bbf0 2024-08-31T05:57:36.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.297+0000 7f7002800640 1 -- 172.21.15.16:0/3515177102 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ffc105d70 con 0x7f6ffc10bbf0 2024-08-31T05:57:36.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.297+0000 7f6ffa000640 1 -- 172.21.15.16:0/3515177102 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 17) ==== 51270+0+0 (secure 0 0 0) 0x7f6ff40109d0 con 0x7f6ffc10bbf0 2024-08-31T05:57:36.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.298+0000 7f6ffa000640 1 --2- 172.21.15.16:0/3515177102 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6fe403f2d0 0x7f6fe40417e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:36.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.298+0000 7f6ffa000640 1 -- 172.21.15.16:0/3515177102 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f6ff40541b0 con 0x7f6ffc10bbf0 2024-08-31T05:57:36.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.298+0000 7f7001400640 1 --2- 172.21.15.16:0/3515177102 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6fe403f2d0 0x7f6fe40417e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:36.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.299+0000 7f7001400640 1 --2- 172.21.15.16:0/3515177102 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6fe403f2d0 0x7f6fe40417e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0009cd0 tx=0x7f6ff0005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:57:36.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.307+0000 7f6ffa000640 1 -- 172.21.15.16:0/3515177102 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f6ff4010cc0 con 0x7f6ffc10bbf0 2024-08-31T05:57:36.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:36 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:36.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:36 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:36.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:36 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:36.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:36 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:36.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:36 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:57:36.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:36 smithi155 ceph-mon[21742]: from='client.? ' entity='client.admin' 2024-08-31T05:57:36.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:36 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:36.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:36 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:36.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:36 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:36.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:36 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:36.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:36 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:57:36.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:36 smithi016 ceph-mon[17194]: from='client.? ' entity='client.admin' 2024-08-31T05:57:36.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:36 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:36.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:36 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:36.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:36 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:36.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:36 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:36.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:36 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:57:36.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:36 smithi184 ceph-mon[22331]: from='client.? ' entity='client.admin' 2024-08-31T05:57:36.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.514+0000 7f7002800640 1 -- 172.21.15.16:0/3515177102 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7f6ffc10d430 con 0x7f6ffc10bbf0 2024-08-31T05:57:36.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.526+0000 7f6ffa000640 1 -- 172.21.15.16:0/3515177102 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v18)=0 v18) ==== 119+0+0 (secure 0 0 0) 0x7f6ff400f3c0 con 0x7f6ffc10bbf0 2024-08-31T05:57:36.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.529+0000 7f7002800640 1 -- 172.21.15.16:0/3515177102 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6fe403f2d0 msgr2=0x7f6fe40417e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:36.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.529+0000 7f7002800640 1 --2- 172.21.15.16:0/3515177102 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6fe403f2d0 0x7f6fe40417e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0009cd0 tx=0x7f6ff0005f90 comp rx=0 tx=0).stop 2024-08-31T05:57:36.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.529+0000 7f7002800640 1 -- 172.21.15.16:0/3515177102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ffc10bbf0 msgr2=0x7f6ffc1a1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:36.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.529+0000 7f7002800640 1 --2- 172.21.15.16:0/3515177102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ffc10bbf0 0x7f6ffc1a1b40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6ff400ec50 tx=0x7f6ff400c6a0 comp rx=0 tx=0).stop 2024-08-31T05:57:36.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.529+0000 7f7002800640 1 -- 172.21.15.16:0/3515177102 shutdown_connections 2024-08-31T05:57:36.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.529+0000 7f7002800640 1 --2- 172.21.15.16:0/3515177102 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6fe403f2d0 0x7f6fe40417e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:36.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.529+0000 7f7002800640 1 --2- 172.21.15.16:0/3515177102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ffc10bbf0 0x7f6ffc1a1b40 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:36.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.529+0000 7f7002800640 1 --2- 172.21.15.16:0/3515177102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ffc10b2a0 0x7f6ffc19c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:36.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.529+0000 7f7002800640 1 --2- 172.21.15.16:0/3515177102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ffc105960 0x7f6ffc19c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:36.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.529+0000 7f7002800640 1 -- 172.21.15.16:0/3515177102 >> 172.21.15.16:0/3515177102 conn(0x7f6ffc101410 msgr2=0x7f6ffc103830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:36.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.529+0000 7f7002800640 1 -- 172.21.15.16:0/3515177102 shutdown_connections 2024-08-31T05:57:36.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:36.529+0000 7f7002800640 1 -- 172.21.15.16:0/3515177102 wait complete. 2024-08-31T05:57:37.820 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:37 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:37.820 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:37 smithi016 ceph-mon[17194]: from='client.? ' entity='client.admin' 2024-08-31T05:57:37.820 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:37 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:37.820 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:37 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:37.821 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:37 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:37.821 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:37 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:57:37.821 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:37 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T05:57:37.821 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:37 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:37.821 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:37 smithi016 ceph-mon[17194]: Reconfiguring mgr.x (unknown last config time)... 2024-08-31T05:57:37.821 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:37 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-31T05:57:37.821 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:37 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-31T05:57:37.821 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:37 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-31T05:57:37.821 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:37 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:57:37.821 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:37 smithi016 ceph-mon[17194]: Reconfiguring daemon mgr.x on smithi016 2024-08-31T05:57:37.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:37 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:37.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:37 smithi155 ceph-mon[21742]: from='client.? ' entity='client.admin' 2024-08-31T05:57:37.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:37 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:37.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:37 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:37.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:37 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:37.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:37 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:57:37.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:37 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T05:57:37.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:37 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:37.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:37 smithi155 ceph-mon[21742]: Reconfiguring mgr.x (unknown last config time)... 2024-08-31T05:57:37.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:37 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-31T05:57:37.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:37 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-31T05:57:37.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:37 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-31T05:57:37.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:37 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:57:37.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:37 smithi155 ceph-mon[21742]: Reconfiguring daemon mgr.x on smithi016 2024-08-31T05:57:37.967 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T05:57:37.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:37 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:37.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:37 smithi184 ceph-mon[22331]: from='client.? ' entity='client.admin' 2024-08-31T05:57:37.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:37 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:37.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:37 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:37.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:37 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:37.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:37 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:57:37.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:37 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T05:57:37.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:37 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:37.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:37 smithi184 ceph-mon[22331]: Reconfiguring mgr.x (unknown last config time)... 2024-08-31T05:57:37.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:37 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-31T05:57:37.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:37 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-31T05:57:37.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:37 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-31T05:57:37.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:37 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:57:37.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:37 smithi184 ceph-mon[22331]: Reconfiguring daemon mgr.x on smithi016 2024-08-31T05:57:38.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.005+0000 7fe9f6200640 1 -- 172.21.15.16:0/2484710557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9f0109bc0 msgr2=0x7fe9f01103f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:38.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.005+0000 7fe9f6200640 1 --2- 172.21.15.16:0/2484710557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9f0109bc0 0x7fe9f01103f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8009ca0 tx=0x7fe9e802a8c0 comp rx=0 tx=0).stop 2024-08-31T05:57:38.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.005+0000 7fe9f6200640 1 -- 172.21.15.16:0/2484710557 shutdown_connections 2024-08-31T05:57:38.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.005+0000 7fe9f6200640 1 --2- 172.21.15.16:0/2484710557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9f0109270 0x7fe9f0109680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:38.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.005+0000 7fe9f6200640 1 --2- 172.21.15.16:0/2484710557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9f0109bc0 0x7fe9f01103f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:38.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.005+0000 7fe9f6200640 1 --2- 172.21.15.16:0/2484710557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9f0103930 0x7fe9f0103d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:38.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.005+0000 7fe9f6200640 1 -- 172.21.15.16:0/2484710557 >> 172.21.15.16:0/2484710557 conn(0x7fe9f00ff380 msgr2=0x7fe9f01017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:38.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.006+0000 7fe9f6200640 1 -- 172.21.15.16:0/2484710557 shutdown_connections 2024-08-31T05:57:38.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.006+0000 7fe9f6200640 1 -- 172.21.15.16:0/2484710557 wait complete. 2024-08-31T05:57:38.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.007+0000 7fe9f6200640 1 Processor -- start 2024-08-31T05:57:38.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.007+0000 7fe9f6200640 1 -- start start 2024-08-31T05:57:38.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.007+0000 7fe9f6200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9f0103930 0x7fe9f019c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:38.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.007+0000 7fe9f6200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9f0109270 0x7fe9f019ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:38.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.007+0000 7fe9f6200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9f0109bc0 0x7fe9f01a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:38.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.007+0000 7fe9f6200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f0113880 con 0x7fe9f0109270 2024-08-31T05:57:38.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.007+0000 7fe9f6200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f0113700 con 0x7fe9f0103930 2024-08-31T05:57:38.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.007+0000 7fe9f6200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f0113a00 con 0x7fe9f0109bc0 2024-08-31T05:57:38.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.008+0000 7fe9ef400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9f0103930 0x7fe9f019c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:38.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.008+0000 7fe9ef400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9f0103930 0x7fe9f019c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51156/0 (socket says 172.21.15.16:51156) 2024-08-31T05:57:38.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.008+0000 7fe9eea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9f0109270 0x7fe9f019ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:38.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.008+0000 7fe9ef400640 1 -- 172.21.15.16:0/4066690476 learned_addr learned my addr 172.21.15.16:0/4066690476 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T05:57:38.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.008+0000 7fe9efe00640 1 --2- 172.21.15.16:0/4066690476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9f0109bc0 0x7fe9f01a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:38.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.008+0000 7fe9ef400640 1 -- 172.21.15.16:0/4066690476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9f0109bc0 msgr2=0x7fe9f01a1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:38.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.008+0000 7fe9ef400640 1 --2- 172.21.15.16:0/4066690476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9f0109bc0 0x7fe9f01a1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:38.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.008+0000 7fe9ef400640 1 -- 172.21.15.16:0/4066690476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9f0109270 msgr2=0x7fe9f019ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:38.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.008+0000 7fe9ef400640 1 --2- 172.21.15.16:0/4066690476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9f0109270 0x7fe9f019ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:38.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.008+0000 7fe9ef400640 1 -- 172.21.15.16:0/4066690476 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9f01a2400 con 0x7fe9f0103930 2024-08-31T05:57:38.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.008+0000 7fe9efe00640 1 --2- 172.21.15.16:0/4066690476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9f0109bc0 0x7fe9f01a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T05:57:38.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.008+0000 7fe9ef400640 1 --2- 172.21.15.16:0/4066690476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9f0103930 0x7fe9f019c490 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe9e000ea60 tx=0x7fe9e000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:57:38.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.008+0000 7fe9e7e00640 1 -- 172.21.15.16:0/4066690476 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e000cde0 con 0x7fe9f0103930 2024-08-31T05:57:38.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.009+0000 7fe9e7e00640 1 -- 172.21.15.16:0/4066690476 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9e00035f0 con 0x7fe9f0103930 2024-08-31T05:57:38.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.009+0000 7fe9f6200640 1 -- 172.21.15.16:0/4066690476 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9f01a26f0 con 0x7fe9f0103930 2024-08-31T05:57:38.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.009+0000 7fe9e7e00640 1 -- 172.21.15.16:0/4066690476 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e0010690 con 0x7fe9f0103930 2024-08-31T05:57:38.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.009+0000 7fe9f6200640 1 -- 172.21.15.16:0/4066690476 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9f01a8e50 con 0x7fe9f0103930 2024-08-31T05:57:38.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.010+0000 7fe9e7e00640 1 -- 172.21.15.16:0/4066690476 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 17) ==== 51270+0+0 (secure 0 0 0) 0x7fe9e0010830 con 0x7fe9f0103930 2024-08-31T05:57:38.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.010+0000 7fe9f6200640 1 -- 172.21.15.16:0/4066690476 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9f0103d40 con 0x7fe9f0103930 2024-08-31T05:57:38.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.010+0000 7fe9e7e00640 1 --2- 172.21.15.16:0/4066690476 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe9d403f2d0 0x7fe9d40417e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:38.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.010+0000 7fe9e7e00640 1 -- 172.21.15.16:0/4066690476 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fe9e0053e10 con 0x7fe9f0103930 2024-08-31T05:57:38.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.011+0000 7fe9eea00640 1 --2- 172.21.15.16:0/4066690476 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe9d403f2d0 0x7fe9d40417e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:38.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.011+0000 7fe9eea00640 1 --2- 172.21.15.16:0/4066690476 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe9d403f2d0 0x7fe9d40417e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0105970 tx=0x7fe9dc005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:57:38.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.015+0000 7fe9e7e00640 1 -- 172.21.15.16:0/4066690476 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fe9e000f3c0 con 0x7fe9f0103930 2024-08-31T05:57:38.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.218+0000 7fe9f6200640 1 -- 172.21.15.16:0/4066690476 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) -- 0x7fe9f019d5c0 con 0x7fe9f0103930 2024-08-31T05:57:38.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.223+0000 7fe9e7e00640 1 -- 172.21.15.16:0/4066690476 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v19)=0 v19) ==== 118+0+0 (secure 0 0 0) 0x7fe9e000a250 con 0x7fe9f0103930 2024-08-31T05:57:38.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.226+0000 7fe9f6200640 1 -- 172.21.15.16:0/4066690476 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe9d403f2d0 msgr2=0x7fe9d40417e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:38.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.226+0000 7fe9f6200640 1 --2- 172.21.15.16:0/4066690476 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe9d403f2d0 0x7fe9d40417e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0105970 tx=0x7fe9dc005f90 comp rx=0 tx=0).stop 2024-08-31T05:57:38.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.226+0000 7fe9f6200640 1 -- 172.21.15.16:0/4066690476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9f0103930 msgr2=0x7fe9f019c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:38.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.226+0000 7fe9f6200640 1 --2- 172.21.15.16:0/4066690476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9f0103930 0x7fe9f019c490 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe9e000ea60 tx=0x7fe9e000ef20 comp rx=0 tx=0).stop 2024-08-31T05:57:38.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.227+0000 7fe9f6200640 1 -- 172.21.15.16:0/4066690476 shutdown_connections 2024-08-31T05:57:38.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.227+0000 7fe9f6200640 1 --2- 172.21.15.16:0/4066690476 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe9d403f2d0 0x7fe9d40417e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:38.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.227+0000 7fe9f6200640 1 --2- 172.21.15.16:0/4066690476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9f0109bc0 0x7fe9f01a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:38.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.227+0000 7fe9f6200640 1 --2- 172.21.15.16:0/4066690476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9f0109270 0x7fe9f019ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:38.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.227+0000 7fe9f6200640 1 --2- 172.21.15.16:0/4066690476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9f0103930 0x7fe9f019c490 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:38.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.227+0000 7fe9f6200640 1 -- 172.21.15.16:0/4066690476 >> 172.21.15.16:0/4066690476 conn(0x7fe9f00ff380 msgr2=0x7fe9f01088d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:38.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.227+0000 7fe9f6200640 1 -- 172.21.15.16:0/4066690476 shutdown_connections 2024-08-31T05:57:38.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.227+0000 7fe9f6200640 1 -- 172.21.15.16:0/4066690476 wait complete. 2024-08-31T05:57:38.618 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:38 smithi016 ceph-mon[17194]: Standby manager daemon z started 2024-08-31T05:57:38.618 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:38 smithi016 ceph-mon[17194]: from='mgr.? 172.21.15.184:0/2896494040' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-08-31T05:57:38.618 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:38 smithi016 ceph-mon[17194]: from='mgr.? 172.21.15.184:0/2896494040' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-08-31T05:57:38.618 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:38 smithi016 ceph-mon[17194]: from='mgr.? 172.21.15.184:0/2896494040' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-08-31T05:57:38.618 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:38 smithi016 ceph-mon[17194]: from='mgr.? 172.21.15.184:0/2896494040' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-08-31T05:57:38.618 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:38 smithi016 ceph-mon[17194]: from='client.? ' entity='client.admin' 2024-08-31T05:57:38.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:38 smithi155 ceph-mon[21742]: Standby manager daemon z started 2024-08-31T05:57:38.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:38 smithi155 ceph-mon[21742]: from='mgr.? 172.21.15.184:0/2896494040' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-08-31T05:57:38.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:38 smithi155 ceph-mon[21742]: from='mgr.? 172.21.15.184:0/2896494040' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-08-31T05:57:38.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:38 smithi155 ceph-mon[21742]: from='mgr.? 172.21.15.184:0/2896494040' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-08-31T05:57:38.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:38 smithi155 ceph-mon[21742]: from='mgr.? 172.21.15.184:0/2896494040' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-08-31T05:57:38.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:38 smithi155 ceph-mon[21742]: from='client.? ' entity='client.admin' 2024-08-31T05:57:38.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.905+0000 7fb4fe000640 1 -- 172.21.15.16:0/166369358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4f8103930 msgr2=0x7fb4f8103d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:38.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.905+0000 7fb4fe000640 1 --2- 172.21.15.16:0/166369358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4f8103930 0x7fb4f8103d40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8009900 tx=0x7fb4e802a410 comp rx=0 tx=0).stop 2024-08-31T05:57:38.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.905+0000 7fb4fe000640 1 -- 172.21.15.16:0/166369358 shutdown_connections 2024-08-31T05:57:38.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.905+0000 7fb4fe000640 1 --2- 172.21.15.16:0/166369358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4f8109bc0 0x7fb4f81103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:38.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.905+0000 7fb4fe000640 1 --2- 172.21.15.16:0/166369358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4f8109270 0x7fb4f8109680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:38.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.905+0000 7fb4fe000640 1 --2- 172.21.15.16:0/166369358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4f8103930 0x7fb4f8103d40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:38.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.905+0000 7fb4fe000640 1 -- 172.21.15.16:0/166369358 >> 172.21.15.16:0/166369358 conn(0x7fb4f80ff380 msgr2=0x7fb4f81017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:38.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.906+0000 7fb4fe000640 1 -- 172.21.15.16:0/166369358 shutdown_connections 2024-08-31T05:57:38.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.906+0000 7fb4fe000640 1 -- 172.21.15.16:0/166369358 wait complete. 2024-08-31T05:57:38.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.907+0000 7fb4fe000640 1 Processor -- start 2024-08-31T05:57:38.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.907+0000 7fb4fe000640 1 -- start start 2024-08-31T05:57:38.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.907+0000 7fb4fe000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4f8103930 0x7fb4f819c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:38.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.907+0000 7fb4fe000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4f8109270 0x7fb4f819ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:38.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.907+0000 7fb4fe000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4f8109bc0 0x7fb4f81a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:38.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.907+0000 7fb4fe000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f8113880 con 0x7fb4f8109bc0 2024-08-31T05:57:38.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.907+0000 7fb4fe000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f8113700 con 0x7fb4f8109270 2024-08-31T05:57:38.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.907+0000 7fb4fe000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f8113a00 con 0x7fb4f8103930 2024-08-31T05:57:38.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.907+0000 7fb4f6a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4f8109270 0x7fb4f819ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:38.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.907+0000 7fb4f7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4f8109bc0 0x7fb4f81a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:38.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.907+0000 7fb4f6a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4f8109270 0x7fb4f819ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51176/0 (socket says 172.21.15.16:51176) 2024-08-31T05:57:38.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.907+0000 7fb4f6a00640 1 -- 172.21.15.16:0/3290163229 learned_addr learned my addr 172.21.15.16:0/3290163229 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T05:57:38.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.908+0000 7fb4f7400640 1 --2- 172.21.15.16:0/3290163229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4f8103930 0x7fb4f819c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:38.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.908+0000 7fb4f6a00640 1 -- 172.21.15.16:0/3290163229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4f8103930 msgr2=0x7fb4f819c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:38.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.908+0000 7fb4f6a00640 1 --2- 172.21.15.16:0/3290163229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4f8103930 0x7fb4f819c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:38.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.908+0000 7fb4f6a00640 1 -- 172.21.15.16:0/3290163229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4f8109bc0 msgr2=0x7fb4f81a1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:38.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.908+0000 7fb4f6a00640 1 --2- 172.21.15.16:0/3290163229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4f8109bc0 0x7fb4f81a1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:38.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.908+0000 7fb4f6a00640 1 -- 172.21.15.16:0/3290163229 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4f81a23e0 con 0x7fb4f8109270 2024-08-31T05:57:38.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.908+0000 7fb4f6a00640 1 --2- 172.21.15.16:0/3290163229 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4f8109270 0x7fb4f819ca10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb4e400e970 tx=0x7fb4e400ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:57:38.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.908+0000 7fb4efe00640 1 -- 172.21.15.16:0/3290163229 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e400cd80 con 0x7fb4f8109270 2024-08-31T05:57:38.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.908+0000 7fb4efe00640 1 -- 172.21.15.16:0/3290163229 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4e40035f0 con 0x7fb4f8109270 2024-08-31T05:57:38.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.908+0000 7fb4fe000640 1 -- 172.21.15.16:0/3290163229 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4f81a26d0 con 0x7fb4f8109270 2024-08-31T05:57:38.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.908+0000 7fb4efe00640 1 -- 172.21.15.16:0/3290163229 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e4010690 con 0x7fb4f8109270 2024-08-31T05:57:38.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.909+0000 7fb4fe000640 1 -- 172.21.15.16:0/3290163229 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4f81a8e30 con 0x7fb4f8109270 2024-08-31T05:57:38.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.909+0000 7fb4fe000640 1 -- 172.21.15.16:0/3290163229 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4f8104bb0 con 0x7fb4f8109270 2024-08-31T05:57:38.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.911+0000 7fb4efe00640 1 -- 172.21.15.16:0/3290163229 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 18) ==== 101904+0+0 (secure 0 0 0) 0x7fb4e4003aa0 con 0x7fb4f8109270 2024-08-31T05:57:38.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.912+0000 7fb4efe00640 1 --2- 172.21.15.16:0/3290163229 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb4dc079ee0 0x7fb4dc07c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:38.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.912+0000 7fb4efe00640 1 -- 172.21.15.16:0/3290163229 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fb4e409ba30 con 0x7fb4f8109270 2024-08-31T05:57:38.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.912+0000 7fb4f7400640 1 --2- 172.21.15.16:0/3290163229 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb4dc079ee0 0x7fb4dc07c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:38.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.912+0000 7fb4f7400640 1 --2- 172.21.15.16:0/3290163229 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb4dc079ee0 0x7fb4dc07c3f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8031440 tx=0x7fb4e8009560 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:57:38.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:38.916+0000 7fb4efe00640 1 -- 172.21.15.16:0/3290163229 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fb4e4064210 con 0x7fb4f8109270 2024-08-31T05:57:38.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:38 smithi184 ceph-mon[22331]: Standby manager daemon z started 2024-08-31T05:57:38.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:38 smithi184 ceph-mon[22331]: from='mgr.? 172.21.15.184:0/2896494040' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-08-31T05:57:38.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:38 smithi184 ceph-mon[22331]: from='mgr.? 172.21.15.184:0/2896494040' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-08-31T05:57:38.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:38 smithi184 ceph-mon[22331]: from='mgr.? 172.21.15.184:0/2896494040' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-08-31T05:57:38.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:38 smithi184 ceph-mon[22331]: from='mgr.? 172.21.15.184:0/2896494040' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-08-31T05:57:38.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:38 smithi184 ceph-mon[22331]: from='client.? ' entity='client.admin' 2024-08-31T05:57:39.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:39.115+0000 7fb4fe000640 1 -- 172.21.15.16:0/3290163229 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7fb4f81a9090 con 0x7fb4f8109270 2024-08-31T05:57:39.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:39.121+0000 7fb4efe00640 1 -- 172.21.15.16:0/3290163229 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v20)=0 v20) ==== 115+0+0 (secure 0 0 0) 0x7fb4e40689d0 con 0x7fb4f8109270 2024-08-31T05:57:39.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:39.123+0000 7fb4fe000640 1 -- 172.21.15.16:0/3290163229 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb4dc079ee0 msgr2=0x7fb4dc07c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:39.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:39.123+0000 7fb4fe000640 1 --2- 172.21.15.16:0/3290163229 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb4dc079ee0 0x7fb4dc07c3f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8031440 tx=0x7fb4e8009560 comp rx=0 tx=0).stop 2024-08-31T05:57:39.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:39.123+0000 7fb4fe000640 1 -- 172.21.15.16:0/3290163229 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4f8109270 msgr2=0x7fb4f819ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:39.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:39.123+0000 7fb4fe000640 1 --2- 172.21.15.16:0/3290163229 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4f8109270 0x7fb4f819ca10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb4e400e970 tx=0x7fb4e400ee30 comp rx=0 tx=0).stop 2024-08-31T05:57:39.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:39.123+0000 7fb4fe000640 1 -- 172.21.15.16:0/3290163229 shutdown_connections 2024-08-31T05:57:39.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:39.123+0000 7fb4fe000640 1 --2- 172.21.15.16:0/3290163229 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb4dc079ee0 0x7fb4dc07c3f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:39.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:39.123+0000 7fb4fe000640 1 --2- 172.21.15.16:0/3290163229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4f8109bc0 0x7fb4f81a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:39.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:39.123+0000 7fb4fe000640 1 --2- 172.21.15.16:0/3290163229 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4f8109270 0x7fb4f819ca10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:39.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:39.123+0000 7fb4fe000640 1 --2- 172.21.15.16:0/3290163229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4f8103930 0x7fb4f819c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:39.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:39.123+0000 7fb4fe000640 1 -- 172.21.15.16:0/3290163229 >> 172.21.15.16:0/3290163229 conn(0x7fb4f80ff380 msgr2=0x7fb4f8108910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:39.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:39.123+0000 7fb4fe000640 1 -- 172.21.15.16:0/3290163229 shutdown_connections 2024-08-31T05:57:39.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:39.123+0000 7fb4fe000640 1 -- 172.21.15.16:0/3290163229 wait complete. 2024-08-31T05:57:39.764 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:39 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-08-31T05:57:39.764 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:39 smithi016 ceph-mon[17194]: mgrmap e18: x(active, since 15s), standbys: z 2024-08-31T05:57:39.764 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:39 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:39.764 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:39 smithi016 ceph-mon[17194]: from='client.? ' entity='client.admin' 2024-08-31T05:57:39.871 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:39 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-08-31T05:57:39.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:39 smithi155 ceph-mon[21742]: mgrmap e18: x(active, since 15s), standbys: z 2024-08-31T05:57:39.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:39 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:39.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:39 smithi155 ceph-mon[21742]: from='client.? ' entity='client.admin' 2024-08-31T05:57:39.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:39 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-08-31T05:57:39.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:39 smithi184 ceph-mon[22331]: mgrmap e18: x(active, since 15s), standbys: z 2024-08-31T05:57:39.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:39 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:39.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:39 smithi184 ceph-mon[22331]: from='client.? ' entity='client.admin' 2024-08-31T05:57:40.104 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T05:57:40.104 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T05:57:40.107 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T05:57:40.557 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '673e4aa0-675d-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi016') 2024-08-31T05:57:40.557 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '673e4aa0-675d-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi016') 2024-08-31T05:57:40.557 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '673e4aa0-675d-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi016') 2024-08-31T05:57:40.565 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:40 smithi016 ceph-mon[17194]: Standby manager daemon y started 2024-08-31T05:57:40.565 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:40 smithi016 ceph-mon[17194]: from='mgr.? 172.21.15.155:0/985502355' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-08-31T05:57:40.565 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:40 smithi016 ceph-mon[17194]: from='mgr.? 172.21.15.155:0/985502355' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-08-31T05:57:40.565 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:40 smithi016 ceph-mon[17194]: from='mgr.? 172.21.15.155:0/985502355' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-08-31T05:57:40.566 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:40 smithi016 ceph-mon[17194]: from='mgr.? 172.21.15.155:0/985502355' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-08-31T05:57:40.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:40 smithi155 ceph-mon[21742]: Standby manager daemon y started 2024-08-31T05:57:40.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:40 smithi155 ceph-mon[21742]: from='mgr.? 172.21.15.155:0/985502355' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-08-31T05:57:40.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:40 smithi155 ceph-mon[21742]: from='mgr.? 172.21.15.155:0/985502355' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-08-31T05:57:40.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:40 smithi155 ceph-mon[21742]: from='mgr.? 172.21.15.155:0/985502355' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-08-31T05:57:40.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:40 smithi155 ceph-mon[21742]: from='mgr.? 172.21.15.155:0/985502355' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-08-31T05:57:40.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.590+0000 7fbd83400640 1 -- 172.21.15.16:0/2519365081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd7c103900 msgr2=0x7fbd7c103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:40.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.590+0000 7fbd83400640 1 --2- 172.21.15.16:0/2519365081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd7c103900 0x7fbd7c103d10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbd70009900 tx=0x7fbd7002a410 comp rx=0 tx=0).stop 2024-08-31T05:57:40.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.590+0000 7fbd83400640 1 -- 172.21.15.16:0/2519365081 shutdown_connections 2024-08-31T05:57:40.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.590+0000 7fbd83400640 1 --2- 172.21.15.16:0/2519365081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd7c109240 0x7fbd7c109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:40.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.590+0000 7fbd83400640 1 --2- 172.21.15.16:0/2519365081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd7c109b90 0x7fbd7c1103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:40.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.590+0000 7fbd83400640 1 --2- 172.21.15.16:0/2519365081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd7c103900 0x7fbd7c103d10 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:40.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.590+0000 7fbd83400640 1 -- 172.21.15.16:0/2519365081 >> 172.21.15.16:0/2519365081 conn(0x7fbd7c07b5f0 msgr2=0x7fbd7c07ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:40.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.590+0000 7fbd83400640 1 -- 172.21.15.16:0/2519365081 shutdown_connections 2024-08-31T05:57:40.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.590+0000 7fbd83400640 1 -- 172.21.15.16:0/2519365081 wait complete. 2024-08-31T05:57:40.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.591+0000 7fbd83400640 1 Processor -- start 2024-08-31T05:57:40.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.592+0000 7fbd83400640 1 -- start start 2024-08-31T05:57:40.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.592+0000 7fbd83400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd7c103900 0x7fbd7c19c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:40.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.592+0000 7fbd83400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd7c109240 0x7fbd7c19ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:40.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.592+0000 7fbd83400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd7c109b90 0x7fbd7c1a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:40.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.592+0000 7fbd83400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7c113890 con 0x7fbd7c109b90 2024-08-31T05:57:40.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.592+0000 7fbd83400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7c113710 con 0x7fbd7c103900 2024-08-31T05:57:40.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.592+0000 7fbd83400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7c113a10 con 0x7fbd7c109240 2024-08-31T05:57:40.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.592+0000 7fbd81400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd7c109b90 0x7fbd7c1a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:40.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.592+0000 7fbd81400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd7c109b90 0x7fbd7c1a1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50496/0 (socket says 172.21.15.16:50496) 2024-08-31T05:57:40.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.592+0000 7fbd80a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd7c103900 0x7fbd7c19c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:40.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.592+0000 7fbd81400640 1 -- 172.21.15.16:0/905896979 learned_addr learned my addr 172.21.15.16:0/905896979 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T05:57:40.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.593+0000 7fbd7be00640 1 --2- 172.21.15.16:0/905896979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd7c109240 0x7fbd7c19ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:40.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.593+0000 7fbd81400640 1 -- 172.21.15.16:0/905896979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd7c109240 msgr2=0x7fbd7c19ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:40.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.593+0000 7fbd81400640 1 --2- 172.21.15.16:0/905896979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd7c109240 0x7fbd7c19ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:40.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.593+0000 7fbd81400640 1 -- 172.21.15.16:0/905896979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd7c103900 msgr2=0x7fbd7c19c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:40.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.593+0000 7fbd81400640 1 --2- 172.21.15.16:0/905896979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd7c103900 0x7fbd7c19c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:40.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.593+0000 7fbd81400640 1 -- 172.21.15.16:0/905896979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd7c1a2410 con 0x7fbd7c109b90 2024-08-31T05:57:40.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.593+0000 7fbd80a00640 1 --2- 172.21.15.16:0/905896979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd7c103900 0x7fbd7c19c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T05:57:40.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.593+0000 7fbd7be00640 1 --2- 172.21.15.16:0/905896979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd7c109240 0x7fbd7c19ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T05:57:40.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.593+0000 7fbd81400640 1 --2- 172.21.15.16:0/905896979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd7c109b90 0x7fbd7c1a1ce0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbd7400eb10 tx=0x7fbd7400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:57:40.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.593+0000 7fbd79600640 1 -- 172.21.15.16:0/905896979 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd74004b20 con 0x7fbd7c109b90 2024-08-31T05:57:40.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.593+0000 7fbd79600640 1 -- 172.21.15.16:0/905896979 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd74004cc0 con 0x7fbd7c109b90 2024-08-31T05:57:40.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.594+0000 7fbd79600640 1 -- 172.21.15.16:0/905896979 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd74010670 con 0x7fbd7c109b90 2024-08-31T05:57:40.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.594+0000 7fbd83400640 1 -- 172.21.15.16:0/905896979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd7c1a2700 con 0x7fbd7c109b90 2024-08-31T05:57:40.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.594+0000 7fbd83400640 1 -- 172.21.15.16:0/905896979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd7c1a8e60 con 0x7fbd7c109b90 2024-08-31T05:57:40.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.595+0000 7fbd83400640 1 -- 172.21.15.16:0/905896979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd7c103d10 con 0x7fbd7c109b90 2024-08-31T05:57:40.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.597+0000 7fbd79600640 1 -- 172.21.15.16:0/905896979 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) ==== 152538+0+0 (secure 0 0 0) 0x7fbd74005000 con 0x7fbd7c109b90 2024-08-31T05:57:40.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.598+0000 7fbd79600640 1 --2- 172.21.15.16:0/905896979 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbd640b4bf0 0x7fbd640b7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:40.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.599+0000 7fbd79600640 1 -- 172.21.15.16:0/905896979 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fbd740e3de0 con 0x7fbd7c109b90 2024-08-31T05:57:40.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.599+0000 7fbd80a00640 1 --2- 172.21.15.16:0/905896979 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbd640b4bf0 0x7fbd640b7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:40.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.599+0000 7fbd80a00640 1 --2- 172.21.15.16:0/905896979 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbd640b4bf0 0x7fbd640b7100 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbd70031440 tx=0x7fbd70032ed0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:57:40.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.600+0000 7fbd79600640 1 -- 172.21.15.16:0/905896979 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fbd74056130 con 0x7fbd7c109b90 2024-08-31T05:57:40.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.809+0000 7fbd83400640 1 -- 172.21.15.16:0/905896979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7fbd7c19d5d0 con 0x7fbd7c109b90 2024-08-31T05:57:40.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.814+0000 7fbd79600640 1 -- 172.21.15.16:0/905896979 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v21)=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7fbd7c19d5d0 con 0x7fbd7c109b90 2024-08-31T05:57:40.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.816+0000 7fbd83400640 1 -- 172.21.15.16:0/905896979 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbd640b4bf0 msgr2=0x7fbd640b7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:40.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.816+0000 7fbd83400640 1 --2- 172.21.15.16:0/905896979 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbd640b4bf0 0x7fbd640b7100 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbd70031440 tx=0x7fbd70032ed0 comp rx=0 tx=0).stop 2024-08-31T05:57:40.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.816+0000 7fbd83400640 1 -- 172.21.15.16:0/905896979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd7c109b90 msgr2=0x7fbd7c1a1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:40.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.816+0000 7fbd83400640 1 --2- 172.21.15.16:0/905896979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd7c109b90 0x7fbd7c1a1ce0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbd7400eb10 tx=0x7fbd7400c2d0 comp rx=0 tx=0).stop 2024-08-31T05:57:40.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.816+0000 7fbd83400640 1 -- 172.21.15.16:0/905896979 shutdown_connections 2024-08-31T05:57:40.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.816+0000 7fbd83400640 1 --2- 172.21.15.16:0/905896979 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbd640b4bf0 0x7fbd640b7100 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:40.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.816+0000 7fbd83400640 1 --2- 172.21.15.16:0/905896979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd7c109240 0x7fbd7c19ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:40.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.816+0000 7fbd83400640 1 --2- 172.21.15.16:0/905896979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd7c109b90 0x7fbd7c1a1ce0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:40.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.816+0000 7fbd83400640 1 --2- 172.21.15.16:0/905896979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd7c103900 0x7fbd7c19c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:40.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.816+0000 7fbd83400640 1 -- 172.21.15.16:0/905896979 >> 172.21.15.16:0/905896979 conn(0x7fbd7c07b5f0 msgr2=0x7fbd7c107050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:40.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.816+0000 7fbd83400640 1 -- 172.21.15.16:0/905896979 shutdown_connections 2024-08-31T05:57:40.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:40.816+0000 7fbd83400640 1 -- 172.21.15.16:0/905896979 wait complete. 2024-08-31T05:57:40.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:40 smithi184 ceph-mon[22331]: Standby manager daemon y started 2024-08-31T05:57:40.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:40 smithi184 ceph-mon[22331]: from='mgr.? 172.21.15.155:0/985502355' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-08-31T05:57:40.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:40 smithi184 ceph-mon[22331]: from='mgr.? 172.21.15.155:0/985502355' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-08-31T05:57:40.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:40 smithi184 ceph-mon[22331]: from='mgr.? 172.21.15.155:0/985502355' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-08-31T05:57:40.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:40 smithi184 ceph-mon[22331]: from='mgr.? 172.21.15.155:0/985502355' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-08-31T05:57:41.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.771+0000 7f2ef0600640 1 -- 172.21.15.16:0/1239364122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ee810fee0 msgr2=0x7f2ee8112300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:41.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.771+0000 7f2ef0600640 1 --2- 172.21.15.16:0/1239364122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ee810fee0 0x7f2ee8112300 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0009ca0 tx=0x7f2ee002a8c0 comp rx=0 tx=0).stop 2024-08-31T05:57:41.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.771+0000 7f2ef0600640 1 -- 172.21.15.16:0/1239364122 shutdown_connections 2024-08-31T05:57:41.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.771+0000 7f2ef0600640 1 --2- 172.21.15.16:0/1239364122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ee810fee0 0x7f2ee8112300 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:41.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.771+0000 7f2ef0600640 1 --2- 172.21.15.16:0/1239364122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ee8075e20 0x7f2ee810f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:41.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.771+0000 7f2ef0600640 1 --2- 172.21.15.16:0/1239364122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ee80754d0 0x7f2ee80758e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:41.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.771+0000 7f2ef0600640 1 -- 172.21.15.16:0/1239364122 >> 172.21.15.16:0/1239364122 conn(0x7f2ee807b5f0 msgr2=0x7f2ee807ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:41.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.771+0000 7f2ef0600640 1 -- 172.21.15.16:0/1239364122 shutdown_connections 2024-08-31T05:57:41.775 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:41 smithi016 ceph-mon[17194]: mgrmap e19: x(active, since 17s), standbys: z, y 2024-08-31T05:57:41.775 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:41 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-08-31T05:57:41.775 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:41 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/905896979' entity='client.admin' 2024-08-31T05:57:41.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.771+0000 7f2ef0600640 1 -- 172.21.15.16:0/1239364122 wait complete. 2024-08-31T05:57:41.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.772+0000 7f2ef0600640 1 Processor -- start 2024-08-31T05:57:41.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.773+0000 7f2ef0600640 1 -- start start 2024-08-31T05:57:41.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.773+0000 7f2ef0600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ee80754d0 0x7f2ee819c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:41.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.773+0000 7f2ef0600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ee8075e20 0x7f2ee819c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:41.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.773+0000 7f2ef0600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ee810fee0 0x7f2ee81a1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:41.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.773+0000 7f2ef0600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee8115a90 con 0x7f2ee8075e20 2024-08-31T05:57:41.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.773+0000 7f2ef0600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee8115910 con 0x7f2ee810fee0 2024-08-31T05:57:41.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.773+0000 7f2ef0600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee8115c10 con 0x7f2ee80754d0 2024-08-31T05:57:41.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.773+0000 7f2eed200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ee8075e20 0x7f2ee819c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:41.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.773+0000 7f2eedc00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ee80754d0 0x7f2ee819c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:41.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.773+0000 7f2eedc00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ee80754d0 0x7f2ee819c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:37110/0 (socket says 172.21.15.16:37110) 2024-08-31T05:57:41.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.773+0000 7f2eedc00640 1 -- 172.21.15.16:0/491978282 learned_addr learned my addr 172.21.15.16:0/491978282 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T05:57:41.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.774+0000 7f2eee600640 1 --2- 172.21.15.16:0/491978282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ee810fee0 0x7f2ee81a1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:41.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.774+0000 7f2eedc00640 1 -- 172.21.15.16:0/491978282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ee810fee0 msgr2=0x7f2ee81a1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:41.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.774+0000 7f2eedc00640 1 --2- 172.21.15.16:0/491978282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ee810fee0 0x7f2ee81a1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:41.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.774+0000 7f2eedc00640 1 -- 172.21.15.16:0/491978282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ee8075e20 msgr2=0x7f2ee819c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:41.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.774+0000 7f2eedc00640 1 --2- 172.21.15.16:0/491978282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ee8075e20 0x7f2ee819c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:41.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.774+0000 7f2eedc00640 1 -- 172.21.15.16:0/491978282 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee81a2390 con 0x7f2ee80754d0 2024-08-31T05:57:41.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.774+0000 7f2eed200640 1 --2- 172.21.15.16:0/491978282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ee8075e20 0x7f2ee819c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T05:57:41.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.774+0000 7f2eedc00640 1 --2- 172.21.15.16:0/491978282 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ee80754d0 0x7f2ee819c420 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2edc00e9f0 tx=0x7f2edc00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:57:41.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.775+0000 7f2ee6000640 1 -- 172.21.15.16:0/491978282 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc00cd80 con 0x7f2ee80754d0 2024-08-31T05:57:41.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.775+0000 7f2ee6000640 1 -- 172.21.15.16:0/491978282 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2edc0054d0 con 0x7f2ee80754d0 2024-08-31T05:57:41.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.775+0000 7f2ee6000640 1 -- 172.21.15.16:0/491978282 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc010690 con 0x7f2ee80754d0 2024-08-31T05:57:41.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.775+0000 7f2ef0600640 1 -- 172.21.15.16:0/491978282 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ee81a2680 con 0x7f2ee80754d0 2024-08-31T05:57:41.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.775+0000 7f2ef0600640 1 -- 172.21.15.16:0/491978282 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ee81a8de0 con 0x7f2ee80754d0 2024-08-31T05:57:41.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.776+0000 7f2ef0600640 1 -- 172.21.15.16:0/491978282 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ee80758e0 con 0x7f2ee80754d0 2024-08-31T05:57:41.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.778+0000 7f2ee6000640 1 -- 172.21.15.16:0/491978282 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 19) ==== 152538+0+0 (secure 0 0 0) 0x7f2edc004e90 con 0x7f2ee80754d0 2024-08-31T05:57:41.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.779+0000 7f2ee6000640 1 --2- 172.21.15.16:0/491978282 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f2ed00b4b70 0x7f2ed00b7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:41.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.780+0000 7f2ee6000640 1 -- 172.21.15.16:0/491978282 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f2edc0e3a40 con 0x7f2ee80754d0 2024-08-31T05:57:41.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.780+0000 7f2eed200640 1 --2- 172.21.15.16:0/491978282 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f2ed00b4b70 0x7f2ed00b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:41.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.780+0000 7f2eed200640 1 --2- 172.21.15.16:0/491978282 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f2ed00b4b70 0x7f2ed00b7080 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2ed40095f0 tx=0x7f2ed4005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:57:41.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.784+0000 7f2ee6000640 1 -- 172.21.15.16:0/491978282 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f2edc0ac6d0 con 0x7f2ee80754d0 2024-08-31T05:57:41.871 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:41 smithi155 ceph-mon[21742]: mgrmap e19: x(active, since 17s), standbys: z, y 2024-08-31T05:57:41.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:41 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-08-31T05:57:41.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:41 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/905896979' entity='client.admin' 2024-08-31T05:57:41.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:41 smithi184 ceph-mon[22331]: mgrmap e19: x(active, since 17s), standbys: z, y 2024-08-31T05:57:41.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:41 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-08-31T05:57:41.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:41 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/905896979' entity='client.admin' 2024-08-31T05:57:41.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.989+0000 7f2ef0600640 1 -- 172.21.15.16:0/491978282 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f2ee81a9120 con 0x7f2ee80754d0 2024-08-31T05:57:41.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.996+0000 7f2ee6000640 1 -- 172.21.15.16:0/491978282 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v23)=0 v23) ==== 120+0+0 (secure 0 0 0) 0x7f2edc0abfd0 con 0x7f2ee80754d0 2024-08-31T05:57:42.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.999+0000 7f2ef0600640 1 -- 172.21.15.16:0/491978282 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f2ed00b4b70 msgr2=0x7f2ed00b7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:42.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.999+0000 7f2ef0600640 1 --2- 172.21.15.16:0/491978282 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f2ed00b4b70 0x7f2ed00b7080 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2ed40095f0 tx=0x7f2ed4005f90 comp rx=0 tx=0).stop 2024-08-31T05:57:42.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.999+0000 7f2ef0600640 1 -- 172.21.15.16:0/491978282 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ee80754d0 msgr2=0x7f2ee819c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:42.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.999+0000 7f2ef0600640 1 --2- 172.21.15.16:0/491978282 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ee80754d0 0x7f2ee819c420 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2edc00e9f0 tx=0x7f2edc00edf0 comp rx=0 tx=0).stop 2024-08-31T05:57:42.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.999+0000 7f2ef0600640 1 -- 172.21.15.16:0/491978282 shutdown_connections 2024-08-31T05:57:42.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.999+0000 7f2ef0600640 1 --2- 172.21.15.16:0/491978282 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f2ed00b4b70 0x7f2ed00b7080 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:42.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.999+0000 7f2ef0600640 1 --2- 172.21.15.16:0/491978282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ee810fee0 0x7f2ee81a1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:42.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.999+0000 7f2ef0600640 1 --2- 172.21.15.16:0/491978282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ee8075e20 0x7f2ee819c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:42.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.999+0000 7f2ef0600640 1 --2- 172.21.15.16:0/491978282 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ee80754d0 0x7f2ee819c420 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:42.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.999+0000 7f2ef0600640 1 -- 172.21.15.16:0/491978282 >> 172.21.15.16:0/491978282 conn(0x7f2ee807b5f0 msgr2=0x7f2ee81079e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:42.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.999+0000 7f2ef0600640 1 -- 172.21.15.16:0/491978282 shutdown_connections 2024-08-31T05:57:42.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.999+0000 7f2ef0600640 1 -- 172.21.15.16:0/491978282 wait complete. 2024-08-31T05:57:42.639 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:42 smithi016 ceph-mon[17194]: from='client.? ' entity='client.admin' 2024-08-31T05:57:42.639 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:42 smithi016 ceph-mon[17194]: from='client.? ' entity='client.admin' 2024-08-31T05:57:42.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:42.817+0000 7f7a29600640 1 -- 172.21.15.16:0/84365596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a240754f0 msgr2=0x7f7a24075900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:42.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:42.817+0000 7f7a29600640 1 --2- 172.21.15.16:0/84365596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a240754f0 0x7f7a24075900 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7a14009900 tx=0x7f7a1402a410 comp rx=0 tx=0).stop 2024-08-31T05:57:42.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:42.818+0000 7f7a29600640 1 -- 172.21.15.16:0/84365596 shutdown_connections 2024-08-31T05:57:42.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:42.818+0000 7f7a29600640 1 --2- 172.21.15.16:0/84365596 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a2410fcf0 0x7f7a24112110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:42.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:42.818+0000 7f7a29600640 1 --2- 172.21.15.16:0/84365596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a24075e40 0x7f7a2410f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:42.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:42.818+0000 7f7a29600640 1 --2- 172.21.15.16:0/84365596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a240754f0 0x7f7a24075900 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:42.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:42.818+0000 7f7a29600640 1 -- 172.21.15.16:0/84365596 >> 172.21.15.16:0/84365596 conn(0x7f7a240ff180 msgr2=0x7f7a241015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:42.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:42.818+0000 7f7a29600640 1 -- 172.21.15.16:0/84365596 shutdown_connections 2024-08-31T05:57:42.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:42.818+0000 7f7a29600640 1 -- 172.21.15.16:0/84365596 wait complete. 2024-08-31T05:57:42.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:42.819+0000 7f7a29600640 1 Processor -- start 2024-08-31T05:57:42.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:42.819+0000 7f7a29600640 1 -- start start 2024-08-31T05:57:42.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:42.820+0000 7f7a29600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a240754f0 0x7f7a2419c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:42.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:42.820+0000 7f7a29600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a24075e40 0x7f7a2419c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:42.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:42.820+0000 7f7a29600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a2410fcf0 0x7f7a241a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:42.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:42.820+0000 7f7a29600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7a24115860 con 0x7f7a2410fcf0 2024-08-31T05:57:42.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:42.820+0000 7f7a29600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7a241156e0 con 0x7f7a24075e40 2024-08-31T05:57:42.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:42.820+0000 7f7a29600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7a241159e0 con 0x7f7a240754f0 2024-08-31T05:57:42.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:42.820+0000 7f7a23e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a240754f0 0x7f7a2419c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:42.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:42.820+0000 7f7a28c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a2410fcf0 0x7f7a241a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:42.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:42.820+0000 7f7a23e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a240754f0 0x7f7a2419c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:37132/0 (socket says 172.21.15.16:37132) 2024-08-31T05:57:42.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:42.820+0000 7f7a23e00640 1 -- 172.21.15.16:0/1796467491 learned_addr learned my addr 172.21.15.16:0/1796467491 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T05:57:42.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:42.820+0000 7f7a23400640 1 --2- 172.21.15.16:0/1796467491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a24075e40 0x7f7a2419c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:42.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:42.820+0000 7f7a23e00640 1 -- 172.21.15.16:0/1796467491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a24075e40 msgr2=0x7f7a2419c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:42.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:42.820+0000 7f7a23e00640 1 --2- 172.21.15.16:0/1796467491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a24075e40 0x7f7a2419c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:42.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:42.820+0000 7f7a23e00640 1 -- 172.21.15.16:0/1796467491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a2410fcf0 msgr2=0x7f7a241a1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:42.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:42.820+0000 7f7a23e00640 1 --2- 172.21.15.16:0/1796467491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a2410fcf0 0x7f7a241a1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:42.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:42.820+0000 7f7a23e00640 1 -- 172.21.15.16:0/1796467491 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a241a2240 con 0x7f7a240754f0 2024-08-31T05:57:42.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:42.820+0000 7f7a28c00640 1 --2- 172.21.15.16:0/1796467491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a2410fcf0 0x7f7a241a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T05:57:42.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:42.821+0000 7f7a23e00640 1 --2- 172.21.15.16:0/1796467491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a240754f0 0x7f7a2419c2d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7a14005940 tx=0x7f7a14004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:57:42.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:42.821+0000 7f7a20c00640 1 -- 172.21.15.16:0/1796467491 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1403f030 con 0x7f7a240754f0 2024-08-31T05:57:42.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:42.821+0000 7f7a20c00640 1 -- 172.21.15.16:0/1796467491 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a14003aa0 con 0x7f7a240754f0 2024-08-31T05:57:42.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:42.821+0000 7f7a29600640 1 -- 172.21.15.16:0/1796467491 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a241a24d0 con 0x7f7a240754f0 2024-08-31T05:57:42.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:42.821+0000 7f7a20c00640 1 -- 172.21.15.16:0/1796467491 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a14035070 con 0x7f7a240754f0 2024-08-31T05:57:42.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:42.821+0000 7f7a29600640 1 -- 172.21.15.16:0/1796467491 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a241a8c90 con 0x7f7a240754f0 2024-08-31T05:57:42.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:42.822+0000 7f7a29600640 1 -- 172.21.15.16:0/1796467491 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a24075900 con 0x7f7a240754f0 2024-08-31T05:57:42.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:42.824+0000 7f7a20c00640 1 -- 172.21.15.16:0/1796467491 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 19) ==== 152538+0+0 (secure 0 0 0) 0x7f7a14042030 con 0x7f7a240754f0 2024-08-31T05:57:42.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:42.826+0000 7f7a20c00640 1 --2- 172.21.15.16:0/1796467491 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7a080b4b70 0x7f7a080b7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:42.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:42.826+0000 7f7a20c00640 1 -- 172.21.15.16:0/1796467491 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f7a141039b0 con 0x7f7a240754f0 2024-08-31T05:57:42.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:42.826+0000 7f7a23400640 1 --2- 172.21.15.16:0/1796467491 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7a080b4b70 0x7f7a080b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:42.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:42.827+0000 7f7a23400640 1 --2- 172.21.15.16:0/1796467491 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7a080b4b70 0x7f7a080b7080 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7a10005d00 tx=0x7f7a10005c50 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:57:42.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:42.831+0000 7f7a20c00640 1 -- 172.21.15.16:0/1796467491 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f7a140cc640 con 0x7f7a240754f0 2024-08-31T05:57:42.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:42 smithi155 ceph-mon[21742]: from='client.? ' entity='client.admin' 2024-08-31T05:57:42.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:42 smithi155 ceph-mon[21742]: from='client.? ' entity='client.admin' 2024-08-31T05:57:42.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:42 smithi184 ceph-mon[22331]: from='client.? ' entity='client.admin' 2024-08-31T05:57:42.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:42 smithi184 ceph-mon[22331]: from='client.? ' entity='client.admin' 2024-08-31T05:57:43.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.028+0000 7f7a29600640 1 -- 172.21.15.16:0/1796467491 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f7a241a8ef0 con 0x7f7a240754f0 2024-08-31T05:57:43.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.035+0000 7f7a20c00640 1 -- 172.21.15.16:0/1796467491 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v24)=0 v24) ==== 121+0+0 (secure 0 0 0) 0x7f7a140cbf40 con 0x7f7a240754f0 2024-08-31T05:57:43.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.037+0000 7f7a29600640 1 -- 172.21.15.16:0/1796467491 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7a080b4b70 msgr2=0x7f7a080b7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:43.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.037+0000 7f7a29600640 1 --2- 172.21.15.16:0/1796467491 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7a080b4b70 0x7f7a080b7080 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7a10005d00 tx=0x7f7a10005c50 comp rx=0 tx=0).stop 2024-08-31T05:57:43.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.037+0000 7f7a29600640 1 -- 172.21.15.16:0/1796467491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a240754f0 msgr2=0x7f7a2419c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:43.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.037+0000 7f7a29600640 1 --2- 172.21.15.16:0/1796467491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a240754f0 0x7f7a2419c2d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7a14005940 tx=0x7f7a14004270 comp rx=0 tx=0).stop 2024-08-31T05:57:43.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.037+0000 7f7a29600640 1 -- 172.21.15.16:0/1796467491 shutdown_connections 2024-08-31T05:57:43.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.037+0000 7f7a29600640 1 --2- 172.21.15.16:0/1796467491 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7a080b4b70 0x7f7a080b7080 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:43.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.037+0000 7f7a29600640 1 --2- 172.21.15.16:0/1796467491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a2410fcf0 0x7f7a241a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:43.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.037+0000 7f7a29600640 1 --2- 172.21.15.16:0/1796467491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a24075e40 0x7f7a2419c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:43.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.037+0000 7f7a29600640 1 --2- 172.21.15.16:0/1796467491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a240754f0 0x7f7a2419c2d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:43.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.037+0000 7f7a29600640 1 -- 172.21.15.16:0/1796467491 >> 172.21.15.16:0/1796467491 conn(0x7f7a240ff180 msgr2=0x7f7a24109890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:43.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.038+0000 7f7a29600640 1 -- 172.21.15.16:0/1796467491 shutdown_connections 2024-08-31T05:57:43.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.038+0000 7f7a29600640 1 -- 172.21.15.16:0/1796467491 wait complete. 2024-08-31T05:57:43.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.102+0000 7fad17c00640 1 -- 172.21.15.16:0/1277447091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad10109240 msgr2=0x7fad10109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:43.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.102+0000 7fad17c00640 1 --2- 172.21.15.16:0/1277447091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad10109240 0x7fad10109650 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7facfc009900 tx=0x7facfc02a410 comp rx=0 tx=0).stop 2024-08-31T05:57:43.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.102+0000 7fad17c00640 1 -- 172.21.15.16:0/1277447091 shutdown_connections 2024-08-31T05:57:43.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.102+0000 7fad17c00640 1 --2- 172.21.15.16:0/1277447091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad10109b90 0x7fad101103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:43.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.102+0000 7fad17c00640 1 --2- 172.21.15.16:0/1277447091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad10109240 0x7fad10109650 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:43.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.102+0000 7fad17c00640 1 --2- 172.21.15.16:0/1277447091 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad10103900 0x7fad10103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:43.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.102+0000 7fad17c00640 1 -- 172.21.15.16:0/1277447091 >> 172.21.15.16:0/1277447091 conn(0x7fad1007b5f0 msgr2=0x7fad1007ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:43.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.102+0000 7fad17c00640 1 -- 172.21.15.16:0/1277447091 shutdown_connections 2024-08-31T05:57:43.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.102+0000 7fad17c00640 1 -- 172.21.15.16:0/1277447091 wait complete. 2024-08-31T05:57:43.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.104+0000 7fad17c00640 1 Processor -- start 2024-08-31T05:57:43.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.104+0000 7fad17c00640 1 -- start start 2024-08-31T05:57:43.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.104+0000 7fad17c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad10103900 0x7fad1019a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:43.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.104+0000 7fad17c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad10109240 0x7fad1019a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:43.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.105+0000 7fad17c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad10109b90 0x7fad1019fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:43.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.105+0000 7fad17c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fad101138a0 con 0x7fad10109240 2024-08-31T05:57:43.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.105+0000 7fad17c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fad10113720 con 0x7fad10109b90 2024-08-31T05:57:43.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.105+0000 7fad17c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fad10113a20 con 0x7fad10103900 2024-08-31T05:57:43.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.105+0000 7fad15200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad10103900 0x7fad1019a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:43.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.105+0000 7fad0fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad10109240 0x7fad1019a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:43.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.105+0000 7fad15200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad10103900 0x7fad1019a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:37154/0 (socket says 172.21.15.16:37154) 2024-08-31T05:57:43.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.105+0000 7fad15c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad10109b90 0x7fad1019fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:43.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.105+0000 7fad15200640 1 -- 172.21.15.16:0/2060067634 learned_addr learned my addr 172.21.15.16:0/2060067634 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T05:57:43.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.105+0000 7fad15200640 1 -- 172.21.15.16:0/2060067634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad10109b90 msgr2=0x7fad1019fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:43.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.105+0000 7fad15200640 1 --2- 172.21.15.16:0/2060067634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad10109b90 0x7fad1019fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:43.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.105+0000 7fad15200640 1 -- 172.21.15.16:0/2060067634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad10109240 msgr2=0x7fad1019a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:43.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.105+0000 7fad15200640 1 --2- 172.21.15.16:0/2060067634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad10109240 0x7fad1019a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:43.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.105+0000 7fad15200640 1 -- 172.21.15.16:0/2060067634 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad101a01a0 con 0x7fad10103900 2024-08-31T05:57:43.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.106+0000 7fad15200640 1 --2- 172.21.15.16:0/2060067634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad10103900 0x7fad1019a230 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fad0400e9f0 tx=0x7fad0400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:57:43.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.106+0000 7fad0d600640 1 -- 172.21.15.16:0/2060067634 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0400cd80 con 0x7fad10103900 2024-08-31T05:57:43.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.106+0000 7fad0d600640 1 -- 172.21.15.16:0/2060067634 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad040054d0 con 0x7fad10103900 2024-08-31T05:57:43.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.106+0000 7fad17c00640 1 -- 172.21.15.16:0/2060067634 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad101a0490 con 0x7fad10103900 2024-08-31T05:57:43.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.106+0000 7fad0d600640 1 -- 172.21.15.16:0/2060067634 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad04010690 con 0x7fad10103900 2024-08-31T05:57:43.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.106+0000 7fad17c00640 1 -- 172.21.15.16:0/2060067634 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad101a6bf0 con 0x7fad10103900 2024-08-31T05:57:43.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.107+0000 7fad17c00640 1 -- 172.21.15.16:0/2060067634 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad10103d10 con 0x7fad10103900 2024-08-31T05:57:43.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.109+0000 7fad0d600640 1 -- 172.21.15.16:0/2060067634 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 19) ==== 152538+0+0 (secure 0 0 0) 0x7fad04004e90 con 0x7fad10103900 2024-08-31T05:57:43.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.111+0000 7fad0d600640 1 --2- 172.21.15.16:0/2060067634 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7facf80b4bf0 0x7facf80b7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:43.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.111+0000 7fad0d600640 1 -- 172.21.15.16:0/2060067634 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fad040e3930 con 0x7fad10103900 2024-08-31T05:57:43.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.111+0000 7fad0fe00640 1 --2- 172.21.15.16:0/2060067634 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7facf80b4bf0 0x7facf80b7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:43.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.112+0000 7fad0fe00640 1 --2- 172.21.15.16:0/2060067634 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7facf80b4bf0 0x7facf80b7100 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7facfc02aa00 tx=0x7facfc005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:57:43.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.112+0000 7fad0d600640 1 -- 172.21.15.16:0/2060067634 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fad040ac5c0 con 0x7fad10103900 2024-08-31T05:57:43.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.378+0000 7fc6d4400640 1 -- 172.21.15.16:0/2760010498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6cc104750 msgr2=0x7fc6cc104b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:43.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.378+0000 7fc6d4400640 1 --2- 172.21.15.16:0/2760010498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6cc104750 0x7fc6cc104b60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0009900 tx=0x7fc6c002a3f0 comp rx=0 tx=0).stop 2024-08-31T05:57:43.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.378+0000 7fc6d4400640 1 -- 172.21.15.16:0/2760010498 shutdown_connections 2024-08-31T05:57:43.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.378+0000 7fc6d4400640 1 --2- 172.21.15.16:0/2760010498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6cc10dbb0 0x7fc6cc1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:43.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.378+0000 7fc6d4400640 1 --2- 172.21.15.16:0/2760010498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6cc1050a0 0x7fc6cc10d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:43.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.378+0000 7fc6d4400640 1 --2- 172.21.15.16:0/2760010498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6cc104750 0x7fc6cc104b60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:43.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.378+0000 7fc6d4400640 1 -- 172.21.15.16:0/2760010498 >> 172.21.15.16:0/2760010498 conn(0x7fc6cc0794d0 msgr2=0x7fc6cc0798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:43.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.378+0000 7fc6d4400640 1 -- 172.21.15.16:0/2760010498 shutdown_connections 2024-08-31T05:57:43.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.378+0000 7fc6d4400640 1 -- 172.21.15.16:0/2760010498 wait complete. 2024-08-31T05:57:43.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.379+0000 7fc6d4400640 1 Processor -- start 2024-08-31T05:57:43.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.379+0000 7fc6d4400640 1 -- start start 2024-08-31T05:57:43.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.380+0000 7fc6d4400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6cc104750 0x7fc6cc19c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:43.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.380+0000 7fc6d4400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6cc1050a0 0x7fc6cc19ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:43.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.380+0000 7fc6d4400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6cc10dbb0 0x7fc6cc1a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:43.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.380+0000 7fc6d4400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc6cc115a30 con 0x7fc6cc1050a0 2024-08-31T05:57:43.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.380+0000 7fc6d4400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc6cc1158b0 con 0x7fc6cc10dbb0 2024-08-31T05:57:43.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.380+0000 7fc6d4400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc6cc115bb0 con 0x7fc6cc104750 2024-08-31T05:57:43.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.380+0000 7fc6d1000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6cc1050a0 0x7fc6cc19ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:43.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.380+0000 7fc6d1a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6cc104750 0x7fc6cc19c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:43.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.380+0000 7fc6d1000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6cc1050a0 0x7fc6cc19ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50518/0 (socket says 172.21.15.16:50518) 2024-08-31T05:57:43.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.380+0000 7fc6d1000640 1 -- 172.21.15.16:0/2773057603 learned_addr learned my addr 172.21.15.16:0/2773057603 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T05:57:43.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.380+0000 7fc6d2400640 1 --2- 172.21.15.16:0/2773057603 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6cc10dbb0 0x7fc6cc1a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:43.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.381+0000 7fc6d1a00640 1 -- 172.21.15.16:0/2773057603 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6cc10dbb0 msgr2=0x7fc6cc1a1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:43.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.381+0000 7fc6d1a00640 1 --2- 172.21.15.16:0/2773057603 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6cc10dbb0 0x7fc6cc1a1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:43.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.381+0000 7fc6d1a00640 1 -- 172.21.15.16:0/2773057603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6cc1050a0 msgr2=0x7fc6cc19ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:43.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.381+0000 7fc6d1a00640 1 --2- 172.21.15.16:0/2773057603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6cc1050a0 0x7fc6cc19ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:43.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.381+0000 7fc6d1a00640 1 -- 172.21.15.16:0/2773057603 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6cc1a2410 con 0x7fc6cc104750 2024-08-31T05:57:43.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.381+0000 7fc6d2400640 1 --2- 172.21.15.16:0/2773057603 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6cc10dbb0 0x7fc6cc1a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T05:57:43.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.381+0000 7fc6d1000640 1 --2- 172.21.15.16:0/2773057603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6cc1050a0 0x7fc6cc19ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T05:57:43.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.381+0000 7fc6d1a00640 1 --2- 172.21.15.16:0/2773057603 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6cc104750 0x7fc6cc19c4a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc6c002a910 tx=0x7fc6c0004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:57:43.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.382+0000 7fc6ca000640 1 -- 172.21.15.16:0/2773057603 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c003f070 con 0x7fc6cc104750 2024-08-31T05:57:43.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.382+0000 7fc6ca000640 1 -- 172.21.15.16:0/2773057603 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6c0003ca0 con 0x7fc6cc104750 2024-08-31T05:57:43.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.382+0000 7fc6ca000640 1 -- 172.21.15.16:0/2773057603 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c0035070 con 0x7fc6cc104750 2024-08-31T05:57:43.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.382+0000 7fc6d4400640 1 -- 172.21.15.16:0/2773057603 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6cc1a26a0 con 0x7fc6cc104750 2024-08-31T05:57:43.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.382+0000 7fc6d4400640 1 -- 172.21.15.16:0/2773057603 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6cc1a8e60 con 0x7fc6cc104750 2024-08-31T05:57:43.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.383+0000 7fc6d4400640 1 -- 172.21.15.16:0/2773057603 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6cc06b300 con 0x7fc6cc104750 2024-08-31T05:57:43.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.385+0000 7fc6ca000640 1 -- 172.21.15.16:0/2773057603 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 19) ==== 152538+0+0 (secure 0 0 0) 0x7fc6c003b4f0 con 0x7fc6cc104750 2024-08-31T05:57:43.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.386+0000 7fc6ca000640 1 --2- 172.21.15.16:0/2773057603 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc6b40b4bf0 0x7fc6b40b7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:43.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.387+0000 7fc6ca000640 1 -- 172.21.15.16:0/2773057603 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fc6c0103940 con 0x7fc6cc104750 2024-08-31T05:57:43.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.387+0000 7fc6d1000640 1 --2- 172.21.15.16:0/2773057603 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc6b40b4bf0 0x7fc6b40b7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:43.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.387+0000 7fc6d1000640 1 --2- 172.21.15.16:0/2773057603 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc6b40b4bf0 0x7fc6b40b7100 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc079e40 tx=0x7fc6b800b410 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:57:43.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.391+0000 7fc6ca000640 1 -- 172.21.15.16:0/2773057603 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fc6c00cbf80 con 0x7fc6cc104750 2024-08-31T05:57:43.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.591+0000 7fc6d4400640 1 -- 172.21.15.16:0/2773057603 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7fc6cc108cb0 con 0x7fc6cc104750 2024-08-31T05:57:43.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.597+0000 7fc6ca000640 1 -- 172.21.15.16:0/2773057603 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v22)=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7fc6c00d0740 con 0x7fc6cc104750 2024-08-31T05:57:43.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.599+0000 7fc6d4400640 1 -- 172.21.15.16:0/2773057603 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc6b40b4bf0 msgr2=0x7fc6b40b7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:43.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.599+0000 7fc6d4400640 1 --2- 172.21.15.16:0/2773057603 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc6b40b4bf0 0x7fc6b40b7100 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc079e40 tx=0x7fc6b800b410 comp rx=0 tx=0).stop 2024-08-31T05:57:43.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.599+0000 7fc6d4400640 1 -- 172.21.15.16:0/2773057603 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6cc104750 msgr2=0x7fc6cc19c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:43.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.599+0000 7fc6d4400640 1 --2- 172.21.15.16:0/2773057603 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6cc104750 0x7fc6cc19c4a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc6c002a910 tx=0x7fc6c0004270 comp rx=0 tx=0).stop 2024-08-31T05:57:43.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.600+0000 7fc6d4400640 1 -- 172.21.15.16:0/2773057603 shutdown_connections 2024-08-31T05:57:43.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.600+0000 7fc6d4400640 1 --2- 172.21.15.16:0/2773057603 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc6b40b4bf0 0x7fc6b40b7100 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:43.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.600+0000 7fc6d4400640 1 --2- 172.21.15.16:0/2773057603 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6cc10dbb0 0x7fc6cc1a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:43.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.600+0000 7fc6d4400640 1 --2- 172.21.15.16:0/2773057603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6cc1050a0 0x7fc6cc19ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:43.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.600+0000 7fc6d4400640 1 --2- 172.21.15.16:0/2773057603 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6cc104750 0x7fc6cc19c4a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:43.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.600+0000 7fc6d4400640 1 -- 172.21.15.16:0/2773057603 >> 172.21.15.16:0/2773057603 conn(0x7fc6cc0794d0 msgr2=0x7fc6cc0ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:43.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.600+0000 7fc6d4400640 1 -- 172.21.15.16:0/2773057603 shutdown_connections 2024-08-31T05:57:43.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:41.600+0000 7fc6d4400640 1 -- 172.21.15.16:0/2773057603 wait complete. 2024-08-31T05:57:43.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.311+0000 7fad17c00640 1 -- 172.21.15.16:0/2060067634 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) -- 0x7fad1019b360 con 0x7fad10103900 2024-08-31T05:57:43.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.318+0000 7fad0d600640 1 -- 172.21.15.16:0/2060067634 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v25)=0 v25) ==== 116+0+0 (secure 0 0 0) 0x7fad1019b360 con 0x7fad10103900 2024-08-31T05:57:43.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.321+0000 7fad17c00640 1 -- 172.21.15.16:0/2060067634 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7facf80b4bf0 msgr2=0x7facf80b7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:43.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.321+0000 7fad17c00640 1 --2- 172.21.15.16:0/2060067634 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7facf80b4bf0 0x7facf80b7100 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7facfc02aa00 tx=0x7facfc005a20 comp rx=0 tx=0).stop 2024-08-31T05:57:43.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.321+0000 7fad17c00640 1 -- 172.21.15.16:0/2060067634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad10103900 msgr2=0x7fad1019a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:43.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.321+0000 7fad17c00640 1 --2- 172.21.15.16:0/2060067634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad10103900 0x7fad1019a230 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fad0400e9f0 tx=0x7fad0400edf0 comp rx=0 tx=0).stop 2024-08-31T05:57:43.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.321+0000 7fad17c00640 1 -- 172.21.15.16:0/2060067634 shutdown_connections 2024-08-31T05:57:43.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.321+0000 7fad17c00640 1 --2- 172.21.15.16:0/2060067634 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7facf80b4bf0 0x7facf80b7100 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:43.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.321+0000 7fad17c00640 1 --2- 172.21.15.16:0/2060067634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad10109b90 0x7fad1019fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:43.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.321+0000 7fad17c00640 1 --2- 172.21.15.16:0/2060067634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad10109240 0x7fad1019a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:43.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.321+0000 7fad17c00640 1 --2- 172.21.15.16:0/2060067634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad10103900 0x7fad1019a230 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:43.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.321+0000 7fad17c00640 1 -- 172.21.15.16:0/2060067634 >> 172.21.15.16:0/2060067634 conn(0x7fad1007b5f0 msgr2=0x7fad101052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:43.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.321+0000 7fad17c00640 1 -- 172.21.15.16:0/2060067634 shutdown_connections 2024-08-31T05:57:43.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.321+0000 7fad17c00640 1 -- 172.21.15.16:0/2060067634 wait complete. 2024-08-31T05:57:43.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.630+0000 7f45d2800640 1 -- 172.21.15.16:0/1316741491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45cc1099e0 msgr2=0x7f45cc110210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:43.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.630+0000 7f45d2800640 1 --2- 172.21.15.16:0/1316741491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45cc1099e0 0x7f45cc110210 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f45c4009be0 tx=0x7f45c402a6c0 comp rx=0 tx=0).stop 2024-08-31T05:57:43.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.630+0000 7f45d2800640 1 -- 172.21.15.16:0/1316741491 shutdown_connections 2024-08-31T05:57:43.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.630+0000 7f45d2800640 1 --2- 172.21.15.16:0/1316741491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45cc109090 0x7f45cc1094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:43.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.630+0000 7f45d2800640 1 --2- 172.21.15.16:0/1316741491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45cc1099e0 0x7f45cc110210 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:43.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.630+0000 7f45d2800640 1 --2- 172.21.15.16:0/1316741491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45cc103750 0x7f45cc103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:43.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.630+0000 7f45d2800640 1 -- 172.21.15.16:0/1316741491 >> 172.21.15.16:0/1316741491 conn(0x7f45cc0ff1e0 msgr2=0x7f45cc101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:43.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.630+0000 7f45d2800640 1 -- 172.21.15.16:0/1316741491 shutdown_connections 2024-08-31T05:57:43.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.630+0000 7f45d2800640 1 -- 172.21.15.16:0/1316741491 wait complete. 2024-08-31T05:57:43.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.632+0000 7f45d2800640 1 Processor -- start 2024-08-31T05:57:43.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.632+0000 7f45d2800640 1 -- start start 2024-08-31T05:57:43.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.632+0000 7f45d2800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45cc103750 0x7f45cc19a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:43.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.633+0000 7f45d2800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45cc109090 0x7f45cc19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:43.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.633+0000 7f45d2800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45cc1099e0 0x7f45cc19f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:43.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.633+0000 7f45d2800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc113690 con 0x7f45cc109090 2024-08-31T05:57:43.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.633+0000 7f45d2800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc113510 con 0x7f45cc1099e0 2024-08-31T05:57:43.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.633+0000 7f45d2800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc113810 con 0x7f45cc103750 2024-08-31T05:57:43.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.633+0000 7f45d0a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45cc109090 0x7f45cc19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:43.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.633+0000 7f45d0a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45cc109090 0x7f45cc19a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50614/0 (socket says 172.21.15.16:50614) 2024-08-31T05:57:43.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.633+0000 7f45d0a00640 1 -- 172.21.15.16:0/468963849 learned_addr learned my addr 172.21.15.16:0/468963849 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T05:57:43.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.633+0000 7f45d1400640 1 --2- 172.21.15.16:0/468963849 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45cc103750 0x7f45cc19a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:43.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.633+0000 7f45d1e00640 1 --2- 172.21.15.16:0/468963849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45cc1099e0 0x7f45cc19f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:43.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.633+0000 7f45d0a00640 1 -- 172.21.15.16:0/468963849 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45cc103750 msgr2=0x7f45cc19a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:43.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.633+0000 7f45d0a00640 1 --2- 172.21.15.16:0/468963849 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45cc103750 0x7f45cc19a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:43.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.633+0000 7f45d0a00640 1 -- 172.21.15.16:0/468963849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45cc1099e0 msgr2=0x7f45cc19f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:43.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.633+0000 7f45d0a00640 1 --2- 172.21.15.16:0/468963849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45cc1099e0 0x7f45cc19f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:43.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.633+0000 7f45d0a00640 1 -- 172.21.15.16:0/468963849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45cc19ffb0 con 0x7f45cc109090 2024-08-31T05:57:43.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.633+0000 7f45d1e00640 1 --2- 172.21.15.16:0/468963849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45cc1099e0 0x7f45cc19f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T05:57:43.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.634+0000 7f45d0a00640 1 --2- 172.21.15.16:0/468963849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45cc109090 0x7f45cc19a5e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f45b800e9f0 tx=0x7f45b800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:57:43.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.634+0000 7f45ca000640 1 -- 172.21.15.16:0/468963849 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b800ccc0 con 0x7f45cc109090 2024-08-31T05:57:43.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.634+0000 7f45ca000640 1 -- 172.21.15.16:0/468963849 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45b80108e0 con 0x7f45cc109090 2024-08-31T05:57:43.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.634+0000 7f45ca000640 1 -- 172.21.15.16:0/468963849 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b8018880 con 0x7f45cc109090 2024-08-31T05:57:43.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.634+0000 7f45d2800640 1 -- 172.21.15.16:0/468963849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45cc1a02a0 con 0x7f45cc109090 2024-08-31T05:57:43.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.634+0000 7f45d2800640 1 -- 172.21.15.16:0/468963849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45cc1a6a00 con 0x7f45cc109090 2024-08-31T05:57:43.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.635+0000 7f45d2800640 1 -- 172.21.15.16:0/468963849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45cc103b60 con 0x7f45cc109090 2024-08-31T05:57:43.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.638+0000 7f45ca000640 1 -- 172.21.15.16:0/468963849 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) ==== 152538+0+0 (secure 0 0 0) 0x7f45b8010430 con 0x7f45cc109090 2024-08-31T05:57:43.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.640+0000 7f45ca000640 1 --2- 172.21.15.16:0/468963849 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f45b40b4b70 0x7f45b40b7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:43.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.640+0000 7f45ca000640 1 -- 172.21.15.16:0/468963849 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f45b80e3da0 con 0x7f45cc109090 2024-08-31T05:57:43.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.640+0000 7f45d1400640 1 --2- 172.21.15.16:0/468963849 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f45b40b4b70 0x7f45b40b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:43.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.641+0000 7f45d1400640 1 --2- 172.21.15.16:0/468963849 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f45b40b4b70 0x7f45b40b7080 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f45c0005fd0 tx=0x7f45c0009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:57:43.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.641+0000 7f45ca000640 1 -- 172.21.15.16:0/468963849 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f45b80aca30 con 0x7f45cc109090 2024-08-31T05:57:43.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.843+0000 7f45d2800640 1 -- 172.21.15.16:0/468963849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f45cc19b170 con 0x7f45cc109090 2024-08-31T05:57:43.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.848+0000 7f45ca000640 1 -- 172.21.15.16:0/468963849 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v26)=0 v26) ==== 138+0+0 (secure 0 0 0) 0x7f45cc19b170 con 0x7f45cc109090 2024-08-31T05:57:43.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.850+0000 7f45d2800640 1 -- 172.21.15.16:0/468963849 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f45b40b4b70 msgr2=0x7f45b40b7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:43.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.850+0000 7f45d2800640 1 --2- 172.21.15.16:0/468963849 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f45b40b4b70 0x7f45b40b7080 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f45c0005fd0 tx=0x7f45c0009260 comp rx=0 tx=0).stop 2024-08-31T05:57:43.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.850+0000 7f45d2800640 1 -- 172.21.15.16:0/468963849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45cc109090 msgr2=0x7f45cc19a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:43.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.850+0000 7f45d2800640 1 --2- 172.21.15.16:0/468963849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45cc109090 0x7f45cc19a5e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f45b800e9f0 tx=0x7f45b800edf0 comp rx=0 tx=0).stop 2024-08-31T05:57:43.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.850+0000 7f45d2800640 1 -- 172.21.15.16:0/468963849 shutdown_connections 2024-08-31T05:57:43.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.850+0000 7f45d2800640 1 --2- 172.21.15.16:0/468963849 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f45b40b4b70 0x7f45b40b7080 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:43.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.850+0000 7f45d2800640 1 --2- 172.21.15.16:0/468963849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45cc1099e0 0x7f45cc19f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:43.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.850+0000 7f45d2800640 1 --2- 172.21.15.16:0/468963849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45cc109090 0x7f45cc19a5e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:43.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.850+0000 7f45d2800640 1 --2- 172.21.15.16:0/468963849 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45cc103750 0x7f45cc19a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:43.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.850+0000 7f45d2800640 1 -- 172.21.15.16:0/468963849 >> 172.21.15.16:0/468963849 conn(0x7f45cc0ff1e0 msgr2=0x7f45cc101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:43.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.850+0000 7f45d2800640 1 -- 172.21.15.16:0/468963849 shutdown_connections 2024-08-31T05:57:43.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:43.850+0000 7f45d2800640 1 -- 172.21.15.16:0/468963849 wait complete. 2024-08-31T05:57:44.317 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:44 smithi016 ceph-mon[17194]: from='client.? ' entity='client.admin' 2024-08-31T05:57:44.317 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:44 smithi016 ceph-mon[17194]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:57:44.317 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:44 smithi016 ceph-mon[17194]: from='client.? ' entity='client.admin' 2024-08-31T05:57:44.317 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:44 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/468963849' entity='client.admin' 2024-08-31T05:57:44.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:44 smithi155 ceph-mon[21742]: from='client.? ' entity='client.admin' 2024-08-31T05:57:44.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:44 smithi155 ceph-mon[21742]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:57:44.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:44 smithi155 ceph-mon[21742]: from='client.? ' entity='client.admin' 2024-08-31T05:57:44.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:44 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/468963849' entity='client.admin' 2024-08-31T05:57:44.397 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T05:57:44.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:44 smithi184 ceph-mon[22331]: from='client.? ' entity='client.admin' 2024-08-31T05:57:44.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:44 smithi184 ceph-mon[22331]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:57:44.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:44 smithi184 ceph-mon[22331]: from='client.? ' entity='client.admin' 2024-08-31T05:57:44.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:44 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/468963849' entity='client.admin' 2024-08-31T05:57:45.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:44.986+0000 7f5016200640 1 -- 172.21.15.16:0/630833350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5010103900 msgr2=0x7f5010103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:45.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:44.986+0000 7f5016200640 1 --2- 172.21.15.16:0/630833350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5010103900 0x7f5010103d10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5000009860 tx=0x7f500002a340 comp rx=0 tx=0).stop 2024-08-31T05:57:45.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:44.986+0000 7f5016200640 1 -- 172.21.15.16:0/630833350 shutdown_connections 2024-08-31T05:57:45.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:44.986+0000 7f5016200640 1 --2- 172.21.15.16:0/630833350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5010109240 0x7f5010109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:45.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:44.986+0000 7f5016200640 1 --2- 172.21.15.16:0/630833350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5010109b90 0x7f50101103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:45.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:44.986+0000 7f5016200640 1 --2- 172.21.15.16:0/630833350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5010103900 0x7f5010103d10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:45.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:44.986+0000 7f5016200640 1 -- 172.21.15.16:0/630833350 >> 172.21.15.16:0/630833350 conn(0x7f501007b5f0 msgr2=0x7f501007ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:45.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:44.986+0000 7f5016200640 1 -- 172.21.15.16:0/630833350 shutdown_connections 2024-08-31T05:57:45.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:44.986+0000 7f5016200640 1 -- 172.21.15.16:0/630833350 wait complete. 2024-08-31T05:57:45.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:44.987+0000 7f5016200640 1 Processor -- start 2024-08-31T05:57:45.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:44.987+0000 7f5016200640 1 -- start start 2024-08-31T05:57:45.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:44.988+0000 7f5016200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5010103900 0x7f501019c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:45.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:44.988+0000 7f5016200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5010109240 0x7f501019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:45.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:44.988+0000 7f5016200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5010109b90 0x7f50101a1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:45.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:44.988+0000 7f5016200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5010113850 con 0x7f5010109b90 2024-08-31T05:57:45.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:44.988+0000 7f5016200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f50101136d0 con 0x7f5010103900 2024-08-31T05:57:45.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:44.988+0000 7f5016200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f50101139d0 con 0x7f5010109240 2024-08-31T05:57:45.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:44.988+0000 7f500fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5010109b90 0x7f50101a1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:45.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:44.988+0000 7f500f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5010103900 0x7f501019c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:45.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:44.988+0000 7f500f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5010103900 0x7f501019c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51300/0 (socket says 172.21.15.16:51300) 2024-08-31T05:57:45.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:44.988+0000 7f5007e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5010109240 0x7f501019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:45.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:44.988+0000 7f500f400640 1 -- 172.21.15.16:0/213991719 learned_addr learned my addr 172.21.15.16:0/213991719 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T05:57:45.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:44.989+0000 7f500f400640 1 -- 172.21.15.16:0/213991719 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5010109240 msgr2=0x7f501019ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:45.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:44.989+0000 7f500f400640 1 --2- 172.21.15.16:0/213991719 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5010109240 0x7f501019ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:45.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:44.989+0000 7f500f400640 1 -- 172.21.15.16:0/213991719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5010109b90 msgr2=0x7f50101a1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:45.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:44.989+0000 7f500f400640 1 --2- 172.21.15.16:0/213991719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5010109b90 0x7f50101a1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:45.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:44.989+0000 7f500f400640 1 -- 172.21.15.16:0/213991719 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50101a2450 con 0x7f5010103900 2024-08-31T05:57:45.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:44.989+0000 7f500f400640 1 --2- 172.21.15.16:0/213991719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5010103900 0x7f501019c4e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f50000303f0 tx=0x7f5000032da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:57:45.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:44.989+0000 7f500cc00640 1 -- 172.21.15.16:0/213991719 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5000040020 con 0x7f5010103900 2024-08-31T05:57:45.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:44.989+0000 7f500cc00640 1 -- 172.21.15.16:0/213991719 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5000030dd0 con 0x7f5010103900 2024-08-31T05:57:45.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:44.989+0000 7f5016200640 1 -- 172.21.15.16:0/213991719 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50101a26e0 con 0x7f5010103900 2024-08-31T05:57:45.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:44.989+0000 7f500cc00640 1 -- 172.21.15.16:0/213991719 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5000037040 con 0x7f5010103900 2024-08-31T05:57:45.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:44.989+0000 7f5016200640 1 -- 172.21.15.16:0/213991719 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50101a8ea0 con 0x7f5010103900 2024-08-31T05:57:45.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:44.990+0000 7f5016200640 1 -- 172.21.15.16:0/213991719 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5010104b80 con 0x7f5010103900 2024-08-31T05:57:45.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:44.993+0000 7f500cc00640 1 -- 172.21.15.16:0/213991719 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 19) ==== 152538+0+0 (secure 0 0 0) 0x7f5000043020 con 0x7f5010103900 2024-08-31T05:57:45.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:44.995+0000 7f500cc00640 1 --2- 172.21.15.16:0/213991719 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f4ff40b4b70 0x7f4ff40b7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:45.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:44.995+0000 7f500cc00640 1 -- 172.21.15.16:0/213991719 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f5000103880 con 0x7f5010103900 2024-08-31T05:57:45.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:44.995+0000 7f5007e00640 1 --2- 172.21.15.16:0/213991719 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f4ff40b4b70 0x7f4ff40b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:45.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:44.995+0000 7f5007e00640 1 --2- 172.21.15.16:0/213991719 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f4ff40b4b70 0x7f4ff40b7080 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5010106260 tx=0x7f4ff8005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:57:45.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:44.996+0000 7f500cc00640 1 -- 172.21.15.16:0/213991719 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f500004a020 con 0x7f5010103900 2024-08-31T05:57:45.115 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T05:57:45.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:45.197+0000 7f5016200640 1 -- 172.21.15.16:0/213991719 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7f5010103d10 con 0x7f5010103900 2024-08-31T05:57:45.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:45.204+0000 7f500cc00640 1 -- 172.21.15.16:0/213991719 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v27)=0 v27) ==== 122+0+0 (secure 0 0 0) 0x7f5010103d10 con 0x7f5010103900 2024-08-31T05:57:45.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:45.206+0000 7f5016200640 1 -- 172.21.15.16:0/213991719 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f4ff40b4b70 msgr2=0x7f4ff40b7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:45.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:45.206+0000 7f5016200640 1 --2- 172.21.15.16:0/213991719 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f4ff40b4b70 0x7f4ff40b7080 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5010106260 tx=0x7f4ff8005f90 comp rx=0 tx=0).stop 2024-08-31T05:57:45.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:45.206+0000 7f5016200640 1 -- 172.21.15.16:0/213991719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5010103900 msgr2=0x7f501019c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:45.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:45.206+0000 7f5016200640 1 --2- 172.21.15.16:0/213991719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5010103900 0x7f501019c4e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f50000303f0 tx=0x7f5000032da0 comp rx=0 tx=0).stop 2024-08-31T05:57:45.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:45.206+0000 7f5016200640 1 -- 172.21.15.16:0/213991719 shutdown_connections 2024-08-31T05:57:45.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:45.206+0000 7f5016200640 1 --2- 172.21.15.16:0/213991719 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f4ff40b4b70 0x7f4ff40b7080 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:45.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:45.206+0000 7f5016200640 1 --2- 172.21.15.16:0/213991719 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5010109240 0x7f501019ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:45.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:45.206+0000 7f5016200640 1 --2- 172.21.15.16:0/213991719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5010109b90 0x7f50101a1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:45.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:45.206+0000 7f5016200640 1 --2- 172.21.15.16:0/213991719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5010103900 0x7f501019c4e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:45.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:45.206+0000 7f5016200640 1 -- 172.21.15.16:0/213991719 >> 172.21.15.16:0/213991719 conn(0x7f501007b5f0 msgr2=0x7f5010107050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:45.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:45.206+0000 7f5016200640 1 -- 172.21.15.16:0/213991719 shutdown_connections 2024-08-31T05:57:45.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:45.206+0000 7f5016200640 1 -- 172.21.15.16:0/213991719 wait complete. 2024-08-31T05:57:46.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:46 smithi184 ceph-mon[22331]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:57:46.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:46 smithi184 ceph-mon[22331]: from='client.? ' entity='client.admin' 2024-08-31T05:57:46.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:46 smithi016 ceph-mon[17194]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:57:46.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:46 smithi016 ceph-mon[17194]: from='client.? ' entity='client.admin' 2024-08-31T05:57:46.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:46 smithi155 ceph-mon[21742]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:57:46.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:46 smithi155 ceph-mon[21742]: from='client.? ' entity='client.admin' 2024-08-31T05:57:46.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:46.942+0000 7f0228a00640 1 -- 172.21.15.16:0/1341473095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0224075ea0 msgr2=0x7f022410f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:46.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:46.942+0000 7f0228a00640 1 --2- 172.21.15.16:0/1341473095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0224075ea0 0x7f022410f770 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0210009900 tx=0x7f021002a3f0 comp rx=0 tx=0).stop 2024-08-31T05:57:46.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:46.942+0000 7f0228a00640 1 -- 172.21.15.16:0/1341473095 shutdown_connections 2024-08-31T05:57:46.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:46.942+0000 7f0228a00640 1 --2- 172.21.15.16:0/1341473095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f022410fcb0 0x7f02241120f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:46.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:46.942+0000 7f0228a00640 1 --2- 172.21.15.16:0/1341473095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0224075ea0 0x7f022410f770 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:46.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:46.942+0000 7f0228a00640 1 --2- 172.21.15.16:0/1341473095 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0224075550 0x7f0224075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:46.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:46.942+0000 7f0228a00640 1 -- 172.21.15.16:0/1341473095 >> 172.21.15.16:0/1341473095 conn(0x7f02240ff1e0 msgr2=0x7f0224101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:46.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:46.942+0000 7f0228a00640 1 -- 172.21.15.16:0/1341473095 shutdown_connections 2024-08-31T05:57:46.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:46.942+0000 7f0228a00640 1 -- 172.21.15.16:0/1341473095 wait complete. 2024-08-31T05:57:46.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:46.944+0000 7f0228a00640 1 Processor -- start 2024-08-31T05:57:46.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:46.944+0000 7f0228a00640 1 -- start start 2024-08-31T05:57:46.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:46.945+0000 7f0228a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0224075550 0x7f022419c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:46.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:46.945+0000 7f0228a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0224075ea0 0x7f022419c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:46.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:46.945+0000 7f0228a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f022410fcb0 0x7f02241a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:46.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:46.945+0000 7f0228a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f02241158b0 con 0x7f0224075ea0 2024-08-31T05:57:46.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:46.945+0000 7f0228a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0224115730 con 0x7f022410fcb0 2024-08-31T05:57:46.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:46.945+0000 7f0228a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0224115a30 con 0x7f0224075550 2024-08-31T05:57:46.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:46.945+0000 7f0221e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0224075ea0 0x7f022419c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:46.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:46.945+0000 7f0223200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f022410fcb0 0x7f02241a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:46.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:46.945+0000 7f0221e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0224075ea0 0x7f022419c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43446/0 (socket says 172.21.15.16:43446) 2024-08-31T05:57:46.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:46.945+0000 7f0221e00640 1 -- 172.21.15.16:0/1352509868 learned_addr learned my addr 172.21.15.16:0/1352509868 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T05:57:46.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:46.945+0000 7f0222800640 1 --2- 172.21.15.16:0/1352509868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0224075550 0x7f022419c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:46.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:46.945+0000 7f0223200640 1 -- 172.21.15.16:0/1352509868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0224075550 msgr2=0x7f022419c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:46.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:46.945+0000 7f0223200640 1 --2- 172.21.15.16:0/1352509868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0224075550 0x7f022419c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:46.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:46.945+0000 7f0223200640 1 -- 172.21.15.16:0/1352509868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0224075ea0 msgr2=0x7f022419c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:46.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:46.945+0000 7f0223200640 1 --2- 172.21.15.16:0/1352509868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0224075ea0 0x7f022419c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:46.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:46.945+0000 7f0223200640 1 -- 172.21.15.16:0/1352509868 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02241a21d0 con 0x7f022410fcb0 2024-08-31T05:57:46.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:46.946+0000 7f0223200640 1 --2- 172.21.15.16:0/1352509868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f022410fcb0 0x7f02241a1aa0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f021c009fd0 tx=0x7f021c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:57:46.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:46.946+0000 7f021b400640 1 -- 172.21.15.16:0/1352509868 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021c00ce20 con 0x7f022410fcb0 2024-08-31T05:57:46.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:46.946+0000 7f0228a00640 1 -- 172.21.15.16:0/1352509868 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02241a24c0 con 0x7f022410fcb0 2024-08-31T05:57:46.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:46.946+0000 7f0228a00640 1 -- 172.21.15.16:0/1352509868 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02241a8c20 con 0x7f022410fcb0 2024-08-31T05:57:46.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:46.946+0000 7f021b400640 1 -- 172.21.15.16:0/1352509868 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f021c010430 con 0x7f022410fcb0 2024-08-31T05:57:46.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:46.946+0000 7f021b400640 1 -- 172.21.15.16:0/1352509868 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021c00f620 con 0x7f022410fcb0 2024-08-31T05:57:46.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:46.948+0000 7f0228a00640 1 -- 172.21.15.16:0/1352509868 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0224075960 con 0x7f022410fcb0 2024-08-31T05:57:46.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:46.950+0000 7f021b400640 1 -- 172.21.15.16:0/1352509868 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 19) ==== 152538+0+0 (secure 0 0 0) 0x7f021c0108e0 con 0x7f022410fcb0 2024-08-31T05:57:46.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:46.951+0000 7f021b400640 1 --2- 172.21.15.16:0/1352509868 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f02080b4af0 0x7f02080b7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:46.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:46.952+0000 7f021b400640 1 -- 172.21.15.16:0/1352509868 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f021c0e3920 con 0x7f022410fcb0 2024-08-31T05:57:46.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:46.952+0000 7f0222800640 1 --2- 172.21.15.16:0/1352509868 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f02080b4af0 0x7f02080b7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:46.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:46.952+0000 7f0222800640 1 --2- 172.21.15.16:0/1352509868 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f02080b4af0 0x7f02080b7000 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0214009bd0 tx=0x7f0214005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:57:46.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:46.955+0000 7f021b400640 1 -- 172.21.15.16:0/1352509868 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f021c0ac5b0 con 0x7f022410fcb0 2024-08-31T05:57:47.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.049+0000 7f2d39600640 1 -- 172.21.15.16:0/2375453560 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d34109bc0 msgr2=0x7f2d341103f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:47.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.049+0000 7f2d39600640 1 --2- 172.21.15.16:0/2375453560 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d34109bc0 0x7f2d341103f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c009be0 tx=0x7f2d2c02a6c0 comp rx=0 tx=0).stop 2024-08-31T05:57:47.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.049+0000 7f2d39600640 1 -- 172.21.15.16:0/2375453560 shutdown_connections 2024-08-31T05:57:47.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.049+0000 7f2d39600640 1 --2- 172.21.15.16:0/2375453560 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d34109bc0 0x7f2d341103f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:47.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.049+0000 7f2d39600640 1 --2- 172.21.15.16:0/2375453560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d34109270 0x7f2d34109680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:47.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.049+0000 7f2d39600640 1 --2- 172.21.15.16:0/2375453560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d34103930 0x7f2d34103d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:47.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.049+0000 7f2d39600640 1 -- 172.21.15.16:0/2375453560 >> 172.21.15.16:0/2375453560 conn(0x7f2d340ff380 msgr2=0x7f2d341017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:47.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.049+0000 7f2d39600640 1 -- 172.21.15.16:0/2375453560 shutdown_connections 2024-08-31T05:57:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.049+0000 7f2d39600640 1 -- 172.21.15.16:0/2375453560 wait complete. 2024-08-31T05:57:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.051+0000 7f2d39600640 1 Processor -- start 2024-08-31T05:57:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.051+0000 7f2d39600640 1 -- start start 2024-08-31T05:57:47.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.052+0000 7f2d39600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d34103930 0x7f2d3419a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:47.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.052+0000 7f2d39600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d34109270 0x7f2d3419a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:47.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.052+0000 7f2d39600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d34109bc0 0x7f2d3419fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:47.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.052+0000 7f2d39600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2d34113880 con 0x7f2d34103930 2024-08-31T05:57:47.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.052+0000 7f2d39600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2d34113700 con 0x7f2d34109bc0 2024-08-31T05:57:47.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.052+0000 7f2d39600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2d34113a00 con 0x7f2d34109270 2024-08-31T05:57:47.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.052+0000 7f2d32a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d34103930 0x7f2d3419a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:47.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.052+0000 7f2d32a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d34103930 0x7f2d3419a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43462/0 (socket says 172.21.15.16:43462) 2024-08-31T05:57:47.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.052+0000 7f2d32a00640 1 -- 172.21.15.16:0/932931734 learned_addr learned my addr 172.21.15.16:0/932931734 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T05:57:47.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.052+0000 7f2d32000640 1 --2- 172.21.15.16:0/932931734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d34109270 0x7f2d3419a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:47.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.052+0000 7f2d33400640 1 --2- 172.21.15.16:0/932931734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d34109bc0 0x7f2d3419fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.052+0000 7f2d32000640 1 -- 172.21.15.16:0/932931734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d34109bc0 msgr2=0x7f2d3419fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.052+0000 7f2d32000640 1 --2- 172.21.15.16:0/932931734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d34109bc0 0x7f2d3419fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.052+0000 7f2d32000640 1 -- 172.21.15.16:0/932931734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d34103930 msgr2=0x7f2d3419a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.052+0000 7f2d32000640 1 --2- 172.21.15.16:0/932931734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d34103930 0x7f2d3419a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.052+0000 7f2d32000640 1 -- 172.21.15.16:0/932931734 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d341a0160 con 0x7f2d34109270 2024-08-31T05:57:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.053+0000 7f2d32000640 1 --2- 172.21.15.16:0/932931734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d34109270 0x7f2d3419a790 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2d3406b8d0 tx=0x7f2d2000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:57:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.053+0000 7f2d2b400640 1 -- 172.21.15.16:0/932931734 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2000cc40 con 0x7f2d34109270 2024-08-31T05:57:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.053+0000 7f2d2b400640 1 -- 172.21.15.16:0/932931734 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d2000cde0 con 0x7f2d34109270 2024-08-31T05:57:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.053+0000 7f2d2b400640 1 -- 172.21.15.16:0/932931734 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d20004a00 con 0x7f2d34109270 2024-08-31T05:57:47.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.053+0000 7f2d39600640 1 -- 172.21.15.16:0/932931734 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d341a0450 con 0x7f2d34109270 2024-08-31T05:57:47.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.053+0000 7f2d39600640 1 -- 172.21.15.16:0/932931734 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d341a6bb0 con 0x7f2d34109270 2024-08-31T05:57:47.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.054+0000 7f2d39600640 1 -- 172.21.15.16:0/932931734 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d34103d40 con 0x7f2d34109270 2024-08-31T05:57:47.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.057+0000 7f2d2b400640 1 -- 172.21.15.16:0/932931734 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 19) ==== 152538+0+0 (secure 0 0 0) 0x7f2d20004ba0 con 0x7f2d34109270 2024-08-31T05:57:47.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.058+0000 7f2d2b400640 1 --2- 172.21.15.16:0/932931734 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f2d180b4b70 0x7f2d180b7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:47.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.058+0000 7f2d2b400640 1 -- 172.21.15.16:0/932931734 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f2d200e3420 con 0x7f2d34109270 2024-08-31T05:57:47.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.058+0000 7f2d32a00640 1 --2- 172.21.15.16:0/932931734 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f2d180b4b70 0x7f2d180b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:47.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.059+0000 7f2d32a00640 1 --2- 172.21.15.16:0/932931734 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f2d180b4b70 0x7f2d180b7080 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2d240058c0 tx=0x7f2d24009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:57:47.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.060+0000 7f2d2b400640 1 -- 172.21.15.16:0/932931734 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f2d200ac0b0 con 0x7f2d34109270 2024-08-31T05:57:47.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.161+0000 7f0228a00640 1 -- 172.21.15.16:0/1352509868 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) -- 0x7f02241a8f70 con 0x7f022410fcb0 2024-08-31T05:57:47.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.168+0000 7f021b400640 1 -- 172.21.15.16:0/1352509868 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v28)=0 v28) ==== 116+0+0 (secure 0 0 0) 0x7f021c0abeb0 con 0x7f022410fcb0 2024-08-31T05:57:47.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.170+0000 7f0228a00640 1 -- 172.21.15.16:0/1352509868 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f02080b4af0 msgr2=0x7f02080b7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:47.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.170+0000 7f0228a00640 1 --2- 172.21.15.16:0/1352509868 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f02080b4af0 0x7f02080b7000 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0214009bd0 tx=0x7f0214005f90 comp rx=0 tx=0).stop 2024-08-31T05:57:47.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.170+0000 7f0228a00640 1 -- 172.21.15.16:0/1352509868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f022410fcb0 msgr2=0x7f02241a1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:47.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.170+0000 7f0228a00640 1 --2- 172.21.15.16:0/1352509868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f022410fcb0 0x7f02241a1aa0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f021c009fd0 tx=0x7f021c00ef10 comp rx=0 tx=0).stop 2024-08-31T05:57:47.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.170+0000 7f0228a00640 1 -- 172.21.15.16:0/1352509868 shutdown_connections 2024-08-31T05:57:47.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.170+0000 7f0228a00640 1 --2- 172.21.15.16:0/1352509868 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f02080b4af0 0x7f02080b7000 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:47.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.170+0000 7f0228a00640 1 --2- 172.21.15.16:0/1352509868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f022410fcb0 0x7f02241a1aa0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:47.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.170+0000 7f0228a00640 1 --2- 172.21.15.16:0/1352509868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0224075ea0 0x7f022419c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:47.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.170+0000 7f0228a00640 1 --2- 172.21.15.16:0/1352509868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0224075550 0x7f022419c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:47.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.170+0000 7f0228a00640 1 -- 172.21.15.16:0/1352509868 >> 172.21.15.16:0/1352509868 conn(0x7f02240ff1e0 msgr2=0x7f0224107710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:47.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.170+0000 7f0228a00640 1 -- 172.21.15.16:0/1352509868 shutdown_connections 2024-08-31T05:57:47.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.171+0000 7f0228a00640 1 -- 172.21.15.16:0/1352509868 wait complete. 2024-08-31T05:57:47.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.265+0000 7f2d39600640 1 -- 172.21.15.16:0/932931734 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f2d3419b320 con 0x7f2d34109270 2024-08-31T05:57:47.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.272+0000 7f2d2b400640 1 -- 172.21.15.16:0/932931734 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v29)=0 v29) ==== 120+0+0 (secure 0 0 0) 0x7f2d3419b320 con 0x7f2d34109270 2024-08-31T05:57:47.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.274+0000 7f2d39600640 1 -- 172.21.15.16:0/932931734 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f2d180b4b70 msgr2=0x7f2d180b7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:47.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.274+0000 7f2d39600640 1 --2- 172.21.15.16:0/932931734 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f2d180b4b70 0x7f2d180b7080 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2d240058c0 tx=0x7f2d24009260 comp rx=0 tx=0).stop 2024-08-31T05:57:47.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.274+0000 7f2d39600640 1 -- 172.21.15.16:0/932931734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d34109270 msgr2=0x7f2d3419a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:47.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.274+0000 7f2d39600640 1 --2- 172.21.15.16:0/932931734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d34109270 0x7f2d3419a790 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2d3406b8d0 tx=0x7f2d2000ee50 comp rx=0 tx=0).stop 2024-08-31T05:57:47.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.274+0000 7f2d39600640 1 -- 172.21.15.16:0/932931734 shutdown_connections 2024-08-31T05:57:47.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.274+0000 7f2d39600640 1 --2- 172.21.15.16:0/932931734 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f2d180b4b70 0x7f2d180b7080 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:47.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.274+0000 7f2d39600640 1 --2- 172.21.15.16:0/932931734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d34109270 0x7f2d3419a790 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:47.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.274+0000 7f2d39600640 1 --2- 172.21.15.16:0/932931734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d34109bc0 0x7f2d3419fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:47.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.274+0000 7f2d39600640 1 --2- 172.21.15.16:0/932931734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d34103930 0x7f2d3419a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:47.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.274+0000 7f2d39600640 1 -- 172.21.15.16:0/932931734 >> 172.21.15.16:0/932931734 conn(0x7f2d340ff380 msgr2=0x7f2d341088c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:47.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.274+0000 7f2d39600640 1 -- 172.21.15.16:0/932931734 shutdown_connections 2024-08-31T05:57:47.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:47.274+0000 7f2d39600640 1 -- 172.21.15.16:0/932931734 wait complete. 2024-08-31T05:57:48.273 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T05:57:48.273 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T05:57:48.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:48 smithi016 ceph-mon[17194]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:57:48.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:48 smithi016 ceph-mon[17194]: from='client.? ' entity='client.admin' 2024-08-31T05:57:48.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:48 smithi016 ceph-mon[17194]: from='client.? ' entity='client.admin' 2024-08-31T05:57:48.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:48 smithi184 ceph-mon[22331]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:57:48.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:48 smithi184 ceph-mon[22331]: from='client.? ' entity='client.admin' 2024-08-31T05:57:48.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:48 smithi184 ceph-mon[22331]: from='client.? ' entity='client.admin' 2024-08-31T05:57:48.621 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:48 smithi155 ceph-mon[21742]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:57:48.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:48 smithi155 ceph-mon[21742]: from='client.? ' entity='client.admin' 2024-08-31T05:57:48.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:48 smithi155 ceph-mon[21742]: from='client.? ' entity='client.admin' 2024-08-31T05:57:50.308 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T05:57:50.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.609+0000 7fe4d3200640 1 -- 172.21.15.16:0/2521945751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4d4075e40 msgr2=0x7fe4d410f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:50.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.609+0000 7fe4d3200640 1 --2- 172.21.15.16:0/2521945751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4d4075e40 0x7fe4d410f7b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009900 tx=0x7fe4c002a3f0 comp rx=0 tx=0).stop 2024-08-31T05:57:50.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.610+0000 7fe4d3200640 1 -- 172.21.15.16:0/2521945751 shutdown_connections 2024-08-31T05:57:50.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.610+0000 7fe4d3200640 1 --2- 172.21.15.16:0/2521945751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4d410fcf0 0x7fe4d4112110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:50.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.610+0000 7fe4d3200640 1 --2- 172.21.15.16:0/2521945751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4d4075e40 0x7fe4d410f7b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:50.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.610+0000 7fe4d3200640 1 --2- 172.21.15.16:0/2521945751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4d40754f0 0x7fe4d4075900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:50.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.610+0000 7fe4d3200640 1 -- 172.21.15.16:0/2521945751 >> 172.21.15.16:0/2521945751 conn(0x7fe4d40ff180 msgr2=0x7fe4d41015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:50.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.610+0000 7fe4d3200640 1 -- 172.21.15.16:0/2521945751 shutdown_connections 2024-08-31T05:57:50.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:50 smithi016 ceph-mon[17194]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:57:50.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:50 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:50.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:50 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:50.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:50 smithi016 ceph-mon[17194]: Reconfiguring mon.b (monmap changed)... 2024-08-31T05:57:50.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:50 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-31T05:57:50.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:50 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-31T05:57:50.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:50 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:57:50.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:50 smithi016 ceph-mon[17194]: Reconfiguring daemon mon.b on smithi155 2024-08-31T05:57:50.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.610+0000 7fe4d3200640 1 -- 172.21.15.16:0/2521945751 wait complete. 2024-08-31T05:57:50.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.611+0000 7fe4d3200640 1 Processor -- start 2024-08-31T05:57:50.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.612+0000 7fe4d3200640 1 -- start start 2024-08-31T05:57:50.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.612+0000 7fe4d3200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4d40754f0 0x7fe4d419c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:50.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.612+0000 7fe4d3200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4d4075e40 0x7fe4d419c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:50.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.612+0000 7fe4d3200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4d410fcf0 0x7fe4d41a1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:50.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.612+0000 7fe4d3200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d4115860 con 0x7fe4d4075e40 2024-08-31T05:57:50.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.612+0000 7fe4d3200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d41156e0 con 0x7fe4d40754f0 2024-08-31T05:57:50.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.612+0000 7fe4d3200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d41159e0 con 0x7fe4d410fcf0 2024-08-31T05:57:50.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.612+0000 7fe4d1e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4d40754f0 0x7fe4d419c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:50.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.612+0000 7fe4d1400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4d4075e40 0x7fe4d419c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:50.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.612+0000 7fe4d1e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4d40754f0 0x7fe4d419c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:52140/0 (socket says 172.21.15.16:52140) 2024-08-31T05:57:50.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.612+0000 7fe4d1e00640 1 -- 172.21.15.16:0/3169544294 learned_addr learned my addr 172.21.15.16:0/3169544294 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T05:57:50.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.612+0000 7fe4d2800640 1 --2- 172.21.15.16:0/3169544294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4d410fcf0 0x7fe4d41a1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:50.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.613+0000 7fe4d1e00640 1 -- 172.21.15.16:0/3169544294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4d410fcf0 msgr2=0x7fe4d41a1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:50.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.613+0000 7fe4d1e00640 1 --2- 172.21.15.16:0/3169544294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4d410fcf0 0x7fe4d41a1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:50.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.613+0000 7fe4d1e00640 1 -- 172.21.15.16:0/3169544294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4d4075e40 msgr2=0x7fe4d419c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:50.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.613+0000 7fe4d1e00640 1 --2- 172.21.15.16:0/3169544294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4d4075e40 0x7fe4d419c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:50.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.613+0000 7fe4d1e00640 1 -- 172.21.15.16:0/3169544294 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d41a22c0 con 0x7fe4d40754f0 2024-08-31T05:57:50.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.613+0000 7fe4d1e00640 1 --2- 172.21.15.16:0/3169544294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4d40754f0 0x7fe4d419c350 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe4c400e9f0 tx=0x7fe4c400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:57:50.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.613+0000 7fe4caa00640 1 -- 172.21.15.16:0/3169544294 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c400cd80 con 0x7fe4d40754f0 2024-08-31T05:57:50.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.613+0000 7fe4caa00640 1 -- 172.21.15.16:0/3169544294 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4c40054d0 con 0x7fe4d40754f0 2024-08-31T05:57:50.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.613+0000 7fe4caa00640 1 -- 172.21.15.16:0/3169544294 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4010690 con 0x7fe4d40754f0 2024-08-31T05:57:50.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.613+0000 7fe4d3200640 1 -- 172.21.15.16:0/3169544294 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4d41a25b0 con 0x7fe4d40754f0 2024-08-31T05:57:50.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.614+0000 7fe4d3200640 1 -- 172.21.15.16:0/3169544294 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4d41a8d10 con 0x7fe4d40754f0 2024-08-31T05:57:50.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.615+0000 7fe4d3200640 1 -- 172.21.15.16:0/3169544294 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4d4075900 con 0x7fe4d40754f0 2024-08-31T05:57:50.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.617+0000 7fe4caa00640 1 -- 172.21.15.16:0/3169544294 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 19) ==== 152538+0+0 (secure 0 0 0) 0x7fe4c4004e90 con 0x7fe4d40754f0 2024-08-31T05:57:50.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.619+0000 7fe4caa00640 1 --2- 172.21.15.16:0/3169544294 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe4b80b4b70 0x7fe4b80b7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:50.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.619+0000 7fe4caa00640 1 -- 172.21.15.16:0/3169544294 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fe4c40e3930 con 0x7fe4d40754f0 2024-08-31T05:57:50.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.619+0000 7fe4d1400640 1 --2- 172.21.15.16:0/3169544294 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe4b80b4b70 0x7fe4b80b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:50.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.620+0000 7fe4d1400640 1 --2- 172.21.15.16:0/3169544294 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe4b80b4b70 0x7fe4b80b7080 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009230 tx=0x7fe4c0031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:57:50.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.620+0000 7fe4caa00640 1 -- 172.21.15.16:0/3169544294 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fe4c40ac5c0 con 0x7fe4d40754f0 2024-08-31T05:57:50.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.826+0000 7fe4d3200640 1 -- 172.21.15.16:0/3169544294 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7fe4d419d480 con 0x7fe4d40754f0 2024-08-31T05:57:50.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:50 smithi155 ceph-mon[21742]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:57:50.831 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:50 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:50.831 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:50 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:50.831 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:50 smithi155 ceph-mon[21742]: Reconfiguring mon.b (monmap changed)... 2024-08-31T05:57:50.831 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:50 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-31T05:57:50.831 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:50 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-31T05:57:50.831 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:50 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:57:50.832 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:50 smithi155 ceph-mon[21742]: Reconfiguring daemon mon.b on smithi155 2024-08-31T05:57:50.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.833+0000 7fe4caa00640 1 -- 172.21.15.16:0/3169544294 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v30)=0 v30) ==== 125+0+0 (secure 0 0 0) 0x7fe4c40abec0 con 0x7fe4d40754f0 2024-08-31T05:57:50.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.836+0000 7fe4d3200640 1 -- 172.21.15.16:0/3169544294 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe4b80b4b70 msgr2=0x7fe4b80b7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:50.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.836+0000 7fe4d3200640 1 --2- 172.21.15.16:0/3169544294 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe4b80b4b70 0x7fe4b80b7080 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009230 tx=0x7fe4c0031400 comp rx=0 tx=0).stop 2024-08-31T05:57:50.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.836+0000 7fe4d3200640 1 -- 172.21.15.16:0/3169544294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4d40754f0 msgr2=0x7fe4d419c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:50.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.836+0000 7fe4d3200640 1 --2- 172.21.15.16:0/3169544294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4d40754f0 0x7fe4d419c350 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe4c400e9f0 tx=0x7fe4c400edf0 comp rx=0 tx=0).stop 2024-08-31T05:57:50.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.836+0000 7fe4d3200640 1 -- 172.21.15.16:0/3169544294 shutdown_connections 2024-08-31T05:57:50.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.836+0000 7fe4d3200640 1 --2- 172.21.15.16:0/3169544294 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe4b80b4b70 0x7fe4b80b7080 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:50.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.836+0000 7fe4d3200640 1 --2- 172.21.15.16:0/3169544294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4d410fcf0 0x7fe4d41a1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:50.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.836+0000 7fe4d3200640 1 --2- 172.21.15.16:0/3169544294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4d4075e40 0x7fe4d419c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:50.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.836+0000 7fe4d3200640 1 --2- 172.21.15.16:0/3169544294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4d40754f0 0x7fe4d419c350 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:50.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.836+0000 7fe4d3200640 1 -- 172.21.15.16:0/3169544294 >> 172.21.15.16:0/3169544294 conn(0x7fe4d40ff180 msgr2=0x7fe4d41098d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:50.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.837+0000 7fe4d3200640 1 -- 172.21.15.16:0/3169544294 shutdown_connections 2024-08-31T05:57:50.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.837+0000 7fe4d3200640 1 -- 172.21.15.16:0/3169544294 wait complete. 2024-08-31T05:57:50.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.835+0000 7fafda800640 1 -- 172.21.15.16:0/2637605232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd4109250 msgr2=0x7fafd4109660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:50.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.835+0000 7fafda800640 1 --2- 172.21.15.16:0/2637605232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd4109250 0x7fafd4109660 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fafc0009900 tx=0x7fafc002a410 comp rx=0 tx=0).stop 2024-08-31T05:57:50.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.835+0000 7fafda800640 1 -- 172.21.15.16:0/2637605232 shutdown_connections 2024-08-31T05:57:50.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.835+0000 7fafda800640 1 --2- 172.21.15.16:0/2637605232 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafd4109ba0 0x7fafd41103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:50.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.835+0000 7fafda800640 1 --2- 172.21.15.16:0/2637605232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd4109250 0x7fafd4109660 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:50.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.835+0000 7fafda800640 1 --2- 172.21.15.16:0/2637605232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafd4103910 0x7fafd4103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:50.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.835+0000 7fafda800640 1 -- 172.21.15.16:0/2637605232 >> 172.21.15.16:0/2637605232 conn(0x7fafd40ff380 msgr2=0x7fafd41017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:50.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.835+0000 7fafda800640 1 -- 172.21.15.16:0/2637605232 shutdown_connections 2024-08-31T05:57:50.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.835+0000 7fafda800640 1 -- 172.21.15.16:0/2637605232 wait complete. 2024-08-31T05:57:50.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.836+0000 7fafda800640 1 Processor -- start 2024-08-31T05:57:50.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.837+0000 7fafda800640 1 -- start start 2024-08-31T05:57:50.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.837+0000 7fafda800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd4103910 0x7fafd419c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:50.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.837+0000 7fafda800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafd4109250 0x7fafd419ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:50.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.837+0000 7fafda800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafd4109ba0 0x7fafd41a1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:50.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.837+0000 7fafda800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fafd41138b0 con 0x7fafd4103910 2024-08-31T05:57:50.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.837+0000 7fafda800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fafd4113730 con 0x7fafd4109ba0 2024-08-31T05:57:50.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.837+0000 7fafda800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fafd4113a30 con 0x7fafd4109250 2024-08-31T05:57:50.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.837+0000 7fafd3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd4103910 0x7fafd419c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:50.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.838+0000 7fafd3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd4103910 0x7fafd419c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43506/0 (socket says 172.21.15.16:43506) 2024-08-31T05:57:50.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.838+0000 7fafd3400640 1 -- 172.21.15.16:0/811281435 learned_addr learned my addr 172.21.15.16:0/811281435 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T05:57:50.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.838+0000 7fafd3e00640 1 --2- 172.21.15.16:0/811281435 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafd4109ba0 0x7fafd41a1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:50.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.838+0000 7fafd3400640 1 -- 172.21.15.16:0/811281435 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafd4109250 msgr2=0x7fafd419ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-31T05:57:50.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.838+0000 7fafd3400640 1 --2- 172.21.15.16:0/811281435 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafd4109250 0x7fafd419ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:50.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.838+0000 7fafd3400640 1 -- 172.21.15.16:0/811281435 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafd4109ba0 msgr2=0x7fafd41a1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:50.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.838+0000 7fafd3400640 1 --2- 172.21.15.16:0/811281435 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafd4109ba0 0x7fafd41a1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:50.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.838+0000 7fafd3400640 1 -- 172.21.15.16:0/811281435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd41a2430 con 0x7fafd4103910 2024-08-31T05:57:50.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.839+0000 7fafd3400640 1 --2- 172.21.15.16:0/811281435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd4103910 0x7fafd419c4c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fafc400e9f0 tx=0x7fafc400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:57:50.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.839+0000 7fafcbe00640 1 -- 172.21.15.16:0/811281435 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc400cd80 con 0x7fafd4103910 2024-08-31T05:57:50.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.839+0000 7fafda800640 1 -- 172.21.15.16:0/811281435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafd41a2720 con 0x7fafd4103910 2024-08-31T05:57:50.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.839+0000 7fafcbe00640 1 -- 172.21.15.16:0/811281435 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafc40054d0 con 0x7fafd4103910 2024-08-31T05:57:50.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.839+0000 7fafda800640 1 -- 172.21.15.16:0/811281435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafd41a8e80 con 0x7fafd4103910 2024-08-31T05:57:50.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.839+0000 7fafcbe00640 1 -- 172.21.15.16:0/811281435 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc4010690 con 0x7fafd4103910 2024-08-31T05:57:50.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.840+0000 7fafda800640 1 -- 172.21.15.16:0/811281435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafb40052c0 con 0x7fafd4103910 2024-08-31T05:57:50.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.843+0000 7fafcbe00640 1 -- 172.21.15.16:0/811281435 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) ==== 152538+0+0 (secure 0 0 0) 0x7fafc40046c0 con 0x7fafd4103910 2024-08-31T05:57:50.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.844+0000 7fafcbe00640 1 --2- 172.21.15.16:0/811281435 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fafb80b4b70 0x7fafb80b7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:50.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.845+0000 7fafcbe00640 1 -- 172.21.15.16:0/811281435 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fafc40e3960 con 0x7fafd4103910 2024-08-31T05:57:50.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.845+0000 7fafd2a00640 1 --2- 172.21.15.16:0/811281435 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fafb80b4b70 0x7fafb80b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:50.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.846+0000 7fafd2a00640 1 --2- 172.21.15.16:0/811281435 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fafb80b4b70 0x7fafb80b7080 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fafc0009230 tx=0x7fafc0005fd0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:57:50.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:50.851+0000 7fafcbe00640 1 -- 172.21.15.16:0/811281435 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fafc4056130 con 0x7fafd4103910 2024-08-31T05:57:50.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:50 smithi184 ceph-mon[22331]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:57:50.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:50 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:50.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:50 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:50.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:50 smithi184 ceph-mon[22331]: Reconfiguring mon.b (monmap changed)... 2024-08-31T05:57:50.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:50 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-31T05:57:50.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:50 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-31T05:57:50.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:50 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:57:50.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:50 smithi184 ceph-mon[22331]: Reconfiguring daemon mon.b on smithi155 2024-08-31T05:57:51.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:51.062+0000 7fafda800640 1 -- 172.21.15.16:0/811281435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fafb40055b0 con 0x7fafd4103910 2024-08-31T05:57:51.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:51.066+0000 7fafcbe00640 1 -- 172.21.15.16:0/811281435 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v31)=0 v31) ==== 133+0+0 (secure 0 0 0) 0x7fafc40ac690 con 0x7fafd4103910 2024-08-31T05:57:51.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:51.069+0000 7fafda800640 1 -- 172.21.15.16:0/811281435 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fafb80b4b70 msgr2=0x7fafb80b7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:51.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:51.069+0000 7fafda800640 1 --2- 172.21.15.16:0/811281435 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fafb80b4b70 0x7fafb80b7080 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fafc0009230 tx=0x7fafc0005fd0 comp rx=0 tx=0).stop 2024-08-31T05:57:51.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:51.069+0000 7fafda800640 1 -- 172.21.15.16:0/811281435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd4103910 msgr2=0x7fafd419c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:51.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:51.069+0000 7fafda800640 1 --2- 172.21.15.16:0/811281435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd4103910 0x7fafd419c4c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fafc400e9f0 tx=0x7fafc400edf0 comp rx=0 tx=0).stop 2024-08-31T05:57:51.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:51.069+0000 7fafda800640 1 -- 172.21.15.16:0/811281435 shutdown_connections 2024-08-31T05:57:51.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:51.069+0000 7fafda800640 1 --2- 172.21.15.16:0/811281435 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fafb80b4b70 0x7fafb80b7080 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:51.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:51.069+0000 7fafda800640 1 --2- 172.21.15.16:0/811281435 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafd4109250 0x7fafd419ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:51.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:51.069+0000 7fafda800640 1 --2- 172.21.15.16:0/811281435 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafd4109ba0 0x7fafd41a1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:51.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:51.069+0000 7fafda800640 1 --2- 172.21.15.16:0/811281435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd4103910 0x7fafd419c4c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:51.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:51.069+0000 7fafda800640 1 -- 172.21.15.16:0/811281435 >> 172.21.15.16:0/811281435 conn(0x7fafd40ff380 msgr2=0x7fafd4106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:51.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:51.069+0000 7fafda800640 1 -- 172.21.15.16:0/811281435 shutdown_connections 2024-08-31T05:57:51.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:51.069+0000 7fafda800640 1 -- 172.21.15.16:0/811281435 wait complete. 2024-08-31T05:57:51.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:51 smithi184 ceph-mon[22331]: from='client.? ' entity='client.admin' 2024-08-31T05:57:51.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:51 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:51.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:51 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:51.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:51 smithi184 ceph-mon[22331]: Reconfiguring mon.c (monmap changed)... 2024-08-31T05:57:51.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:51 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-31T05:57:51.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:51 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-31T05:57:51.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:51 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:57:51.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:51 smithi184 ceph-mon[22331]: Reconfiguring daemon mon.c on smithi184 2024-08-31T05:57:51.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:51 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/811281435' entity='client.admin' 2024-08-31T05:57:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.031+0000 7fc655c00640 1 -- 172.21.15.16:0/2018935793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc650103750 msgr2=0x7fc650103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.031+0000 7fc655c00640 1 --2- 172.21.15.16:0/2018935793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc650103750 0x7fc650103b60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc640009860 tx=0x7fc64002a340 comp rx=0 tx=0).stop 2024-08-31T05:57:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.031+0000 7fc655c00640 1 -- 172.21.15.16:0/2018935793 shutdown_connections 2024-08-31T05:57:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.031+0000 7fc655c00640 1 --2- 172.21.15.16:0/2018935793 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6501099e0 0x7fc650110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.031+0000 7fc655c00640 1 --2- 172.21.15.16:0/2018935793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc650109090 0x7fc6501094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.031+0000 7fc655c00640 1 --2- 172.21.15.16:0/2018935793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc650103750 0x7fc650103b60 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.031+0000 7fc655c00640 1 -- 172.21.15.16:0/2018935793 >> 172.21.15.16:0/2018935793 conn(0x7fc6500ff1e0 msgr2=0x7fc650101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.031+0000 7fc655c00640 1 -- 172.21.15.16:0/2018935793 shutdown_connections 2024-08-31T05:57:52.034 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:51 smithi016 ceph-mon[17194]: from='client.? ' entity='client.admin' 2024-08-31T05:57:52.035 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:51 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:52.035 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:51 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:52.035 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:51 smithi016 ceph-mon[17194]: Reconfiguring mon.c (monmap changed)... 2024-08-31T05:57:52.035 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:51 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-31T05:57:52.035 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:51 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-31T05:57:52.035 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:51 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:57:52.035 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:51 smithi016 ceph-mon[17194]: Reconfiguring daemon mon.c on smithi184 2024-08-31T05:57:52.035 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:51 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/811281435' entity='client.admin' 2024-08-31T05:57:52.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.031+0000 7fc655c00640 1 -- 172.21.15.16:0/2018935793 wait complete. 2024-08-31T05:57:52.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.033+0000 7fc655c00640 1 Processor -- start 2024-08-31T05:57:52.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.033+0000 7fc655c00640 1 -- start start 2024-08-31T05:57:52.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.033+0000 7fc655c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc650103750 0x7fc65019c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:52.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.033+0000 7fc655c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc650109090 0x7fc65019c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:52.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.033+0000 7fc655c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6501099e0 0x7fc6501a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:52.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.033+0000 7fc655c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc650113690 con 0x7fc650109090 2024-08-31T05:57:52.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.033+0000 7fc655c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc650113510 con 0x7fc650103750 2024-08-31T05:57:52.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.033+0000 7fc655c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc650113810 con 0x7fc6501099e0 2024-08-31T05:57:52.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.034+0000 7fc64f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc650109090 0x7fc65019c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:52.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.034+0000 7fc64f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc650109090 0x7fc65019c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43524/0 (socket says 172.21.15.16:43524) 2024-08-31T05:57:52.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.034+0000 7fc64f400640 1 -- 172.21.15.16:0/1776437087 learned_addr learned my addr 172.21.15.16:0/1776437087 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T05:57:52.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.034+0000 7fc64fe00640 1 --2- 172.21.15.16:0/1776437087 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc650103750 0x7fc65019c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:52.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.034+0000 7fc655200640 1 --2- 172.21.15.16:0/1776437087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6501099e0 0x7fc6501a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:52.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.034+0000 7fc64f400640 1 -- 172.21.15.16:0/1776437087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6501099e0 msgr2=0x7fc6501a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:52.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.034+0000 7fc64f400640 1 --2- 172.21.15.16:0/1776437087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6501099e0 0x7fc6501a1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:52.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.034+0000 7fc64f400640 1 -- 172.21.15.16:0/1776437087 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc650103750 msgr2=0x7fc65019c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:52.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.034+0000 7fc64f400640 1 --2- 172.21.15.16:0/1776437087 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc650103750 0x7fc65019c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:52.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.034+0000 7fc64f400640 1 -- 172.21.15.16:0/1776437087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6501a2250 con 0x7fc650109090 2024-08-31T05:57:52.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.034+0000 7fc64fe00640 1 --2- 172.21.15.16:0/1776437087 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc650103750 0x7fc65019c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T05:57:52.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.034+0000 7fc64f400640 1 --2- 172.21.15.16:0/1776437087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc650109090 0x7fc65019c880 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc63c00e9c0 tx=0x7fc63c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:57:52.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.035+0000 7fc64cc00640 1 -- 172.21.15.16:0/1776437087 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63c00ccc0 con 0x7fc650109090 2024-08-31T05:57:52.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.035+0000 7fc64cc00640 1 -- 172.21.15.16:0/1776437087 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc63c0108e0 con 0x7fc650109090 2024-08-31T05:57:52.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.035+0000 7fc64cc00640 1 -- 172.21.15.16:0/1776437087 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63c0187e0 con 0x7fc650109090 2024-08-31T05:57:52.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.035+0000 7fc655c00640 1 -- 172.21.15.16:0/1776437087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6501a2540 con 0x7fc650109090 2024-08-31T05:57:52.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.035+0000 7fc655c00640 1 -- 172.21.15.16:0/1776437087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6501a8ca0 con 0x7fc650109090 2024-08-31T05:57:52.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.036+0000 7fc655c00640 1 -- 172.21.15.16:0/1776437087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc650103b60 con 0x7fc650109090 2024-08-31T05:57:52.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.038+0000 7fc64cc00640 1 -- 172.21.15.16:0/1776437087 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) ==== 152538+0+0 (secure 0 0 0) 0x7fc63c010430 con 0x7fc650109090 2024-08-31T05:57:52.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.040+0000 7fc64cc00640 1 --2- 172.21.15.16:0/1776437087 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc6340b4bf0 0x7fc6340b7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:52.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.040+0000 7fc64cc00640 1 -- 172.21.15.16:0/1776437087 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fc63c0e37e0 con 0x7fc650109090 2024-08-31T05:57:52.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.040+0000 7fc64fe00640 1 --2- 172.21.15.16:0/1776437087 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc6340b4bf0 0x7fc6340b7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:52.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.041+0000 7fc64fe00640 1 --2- 172.21.15.16:0/1776437087 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc6340b4bf0 0x7fc6340b7100 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc640009830 tx=0x7fc640030340 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:57:52.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.045+0000 7fc64cc00640 1 -- 172.21.15.16:0/1776437087 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fc63c0abe20 con 0x7fc650109090 2024-08-31T05:57:52.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:51 smithi155 ceph-mon[21742]: from='client.? ' entity='client.admin' 2024-08-31T05:57:52.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:51 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:52.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:51 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:52.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:51 smithi155 ceph-mon[21742]: Reconfiguring mon.c (monmap changed)... 2024-08-31T05:57:52.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:51 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-31T05:57:52.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:51 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-31T05:57:52.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:51 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:57:52.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:51 smithi155 ceph-mon[21742]: Reconfiguring daemon mon.c on smithi184 2024-08-31T05:57:52.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:51 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/811281435' entity='client.admin' 2024-08-31T05:57:52.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.242+0000 7fc655c00640 1 -- 172.21.15.16:0/1776437087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7fc6501a8f30 con 0x7fc650109090 2024-08-31T05:57:52.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.247+0000 7fc64cc00640 1 -- 172.21.15.16:0/1776437087 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v32)=0 v32) ==== 129+0+0 (secure 0 0 0) 0x7fc63c0b05e0 con 0x7fc650109090 2024-08-31T05:57:52.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.250+0000 7fc655c00640 1 -- 172.21.15.16:0/1776437087 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc6340b4bf0 msgr2=0x7fc6340b7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:52.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.250+0000 7fc655c00640 1 --2- 172.21.15.16:0/1776437087 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc6340b4bf0 0x7fc6340b7100 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc640009830 tx=0x7fc640030340 comp rx=0 tx=0).stop 2024-08-31T05:57:52.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.250+0000 7fc655c00640 1 -- 172.21.15.16:0/1776437087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc650109090 msgr2=0x7fc65019c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:52.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.250+0000 7fc655c00640 1 --2- 172.21.15.16:0/1776437087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc650109090 0x7fc65019c880 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc63c00e9c0 tx=0x7fc63c00ee80 comp rx=0 tx=0).stop 2024-08-31T05:57:52.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.250+0000 7fc655c00640 1 -- 172.21.15.16:0/1776437087 shutdown_connections 2024-08-31T05:57:52.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.250+0000 7fc655c00640 1 --2- 172.21.15.16:0/1776437087 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc6340b4bf0 0x7fc6340b7100 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:52.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.250+0000 7fc655c00640 1 --2- 172.21.15.16:0/1776437087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6501099e0 0x7fc6501a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:52.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.250+0000 7fc655c00640 1 --2- 172.21.15.16:0/1776437087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc650109090 0x7fc65019c880 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:52.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.250+0000 7fc655c00640 1 --2- 172.21.15.16:0/1776437087 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc650103750 0x7fc65019c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:52.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.250+0000 7fc655c00640 1 -- 172.21.15.16:0/1776437087 >> 172.21.15.16:0/1776437087 conn(0x7fc6500ff1e0 msgr2=0x7fc65010e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:52.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.250+0000 7fc655c00640 1 -- 172.21.15.16:0/1776437087 shutdown_connections 2024-08-31T05:57:52.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.250+0000 7fc655c00640 1 -- 172.21.15.16:0/1776437087 wait complete. 2024-08-31T05:57:52.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.361+0000 7f871ba00640 1 -- 172.21.15.16:0/1607970020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f871410b460 msgr2=0x7f871410b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:52.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.361+0000 7f871ba00640 1 --2- 172.21.15.16:0/1607970020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f871410b460 0x7f871410b870 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8700009900 tx=0x7f870002a410 comp rx=0 tx=0).stop 2024-08-31T05:57:52.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.361+0000 7f871ba00640 1 -- 172.21.15.16:0/1607970020 shutdown_connections 2024-08-31T05:57:52.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.361+0000 7f871ba00640 1 --2- 172.21.15.16:0/1607970020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f871410bdb0 0x7f87141125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:52.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.361+0000 7f871ba00640 1 --2- 172.21.15.16:0/1607970020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f871410b460 0x7f871410b870 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:52.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.361+0000 7f871ba00640 1 --2- 172.21.15.16:0/1607970020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8714105b20 0x7f8714105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:52.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.361+0000 7f871ba00640 1 -- 172.21.15.16:0/1607970020 >> 172.21.15.16:0/1607970020 conn(0x7f87141015b0 msgr2=0x7f8714103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:52.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.361+0000 7f871ba00640 1 -- 172.21.15.16:0/1607970020 shutdown_connections 2024-08-31T05:57:52.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.361+0000 7f871ba00640 1 -- 172.21.15.16:0/1607970020 wait complete. 2024-08-31T05:57:52.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.362+0000 7f871ba00640 1 Processor -- start 2024-08-31T05:57:52.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.362+0000 7f871ba00640 1 -- start start 2024-08-31T05:57:52.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.363+0000 7f871ba00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8714105b20 0x7f871419c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:52.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.363+0000 7f871ba00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f871410b460 0x7f871419c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:52.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.363+0000 7f871ba00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f871410bdb0 0x7f87141a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:52.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.363+0000 7f871ba00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8714115ac0 con 0x7f871410b460 2024-08-31T05:57:52.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.363+0000 7f871ba00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8714115940 con 0x7f871410bdb0 2024-08-31T05:57:52.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.363+0000 7f871ba00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8714115c40 con 0x7f8714105b20 2024-08-31T05:57:52.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.363+0000 7f8719000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8714105b20 0x7f871419c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:52.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.363+0000 7f8719000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8714105b20 0x7f871419c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:49980/0 (socket says 172.21.15.16:49980) 2024-08-31T05:57:52.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.363+0000 7f8719000640 1 -- 172.21.15.16:0/995529422 learned_addr learned my addr 172.21.15.16:0/995529422 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T05:57:52.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.363+0000 7f8719a00640 1 --2- 172.21.15.16:0/995529422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f871410bdb0 0x7f87141a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:52.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.363+0000 7f8713e00640 1 --2- 172.21.15.16:0/995529422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f871410b460 0x7f871419c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:52.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.363+0000 7f8719000640 1 -- 172.21.15.16:0/995529422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f871410bdb0 msgr2=0x7f87141a1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:52.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.363+0000 7f8719000640 1 --2- 172.21.15.16:0/995529422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f871410bdb0 0x7f87141a1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:52.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.363+0000 7f8719000640 1 -- 172.21.15.16:0/995529422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f871410b460 msgr2=0x7f871419c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:52.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.363+0000 7f8719000640 1 --2- 172.21.15.16:0/995529422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f871410b460 0x7f871419c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:52.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.363+0000 7f8719000640 1 -- 172.21.15.16:0/995529422 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87141a23c0 con 0x7f8714105b20 2024-08-31T05:57:52.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.364+0000 7f8719000640 1 --2- 172.21.15.16:0/995529422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8714105b20 0x7f871419c450 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f870400e9f0 tx=0x7f870400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:57:52.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.364+0000 7f8711600640 1 -- 172.21.15.16:0/995529422 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870400cd80 con 0x7f8714105b20 2024-08-31T05:57:52.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.364+0000 7f8711600640 1 -- 172.21.15.16:0/995529422 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87040108e0 con 0x7f8714105b20 2024-08-31T05:57:52.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.364+0000 7f8711600640 1 -- 172.21.15.16:0/995529422 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8704018980 con 0x7f8714105b20 2024-08-31T05:57:52.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.364+0000 7f871ba00640 1 -- 172.21.15.16:0/995529422 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87141a26b0 con 0x7f8714105b20 2024-08-31T05:57:52.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.364+0000 7f871ba00640 1 -- 172.21.15.16:0/995529422 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87141a8e10 con 0x7f8714105b20 2024-08-31T05:57:52.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.365+0000 7f871ba00640 1 -- 172.21.15.16:0/995529422 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8714105f30 con 0x7f8714105b20 2024-08-31T05:57:52.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.368+0000 7f8711600640 1 -- 172.21.15.16:0/995529422 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 19) ==== 152538+0+0 (secure 0 0 0) 0x7f8704010430 con 0x7f8714105b20 2024-08-31T05:57:52.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.369+0000 7f8711600640 1 --2- 172.21.15.16:0/995529422 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f86f80b4bf0 0x7f86f80b7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:52.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.370+0000 7f8711600640 1 -- 172.21.15.16:0/995529422 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f87040e3d70 con 0x7f8714105b20 2024-08-31T05:57:52.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.370+0000 7f8713e00640 1 --2- 172.21.15.16:0/995529422 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f86f80b4bf0 0x7f86f80b7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:52.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.370+0000 7f8713e00640 1 --2- 172.21.15.16:0/995529422 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f86f80b4bf0 0x7f86f80b7100 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f870002aa00 tx=0x7f8700005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:57:52.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.371+0000 7f8711600640 1 -- 172.21.15.16:0/995529422 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f87040ac3b0 con 0x7f8714105b20 2024-08-31T05:57:52.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.575+0000 7f871ba00640 1 -- 172.21.15.16:0/995529422 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) -- 0x7f871402e430 con 0x7f8714105b20 2024-08-31T05:57:52.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.584+0000 7f8711600640 1 -- 172.21.15.16:0/995529422 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v33)=0 v33) ==== 127+0+0 (secure 0 0 0) 0x7f87040b0b70 con 0x7f8714105b20 2024-08-31T05:57:52.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.586+0000 7f871ba00640 1 -- 172.21.15.16:0/995529422 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f86f80b4bf0 msgr2=0x7f86f80b7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:52.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.586+0000 7f871ba00640 1 --2- 172.21.15.16:0/995529422 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f86f80b4bf0 0x7f86f80b7100 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f870002aa00 tx=0x7f8700005a20 comp rx=0 tx=0).stop 2024-08-31T05:57:52.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.586+0000 7f871ba00640 1 -- 172.21.15.16:0/995529422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8714105b20 msgr2=0x7f871419c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:52.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.586+0000 7f871ba00640 1 --2- 172.21.15.16:0/995529422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8714105b20 0x7f871419c450 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f870400e9f0 tx=0x7f870400eeb0 comp rx=0 tx=0).stop 2024-08-31T05:57:52.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.586+0000 7f871ba00640 1 -- 172.21.15.16:0/995529422 shutdown_connections 2024-08-31T05:57:52.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.586+0000 7f871ba00640 1 --2- 172.21.15.16:0/995529422 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f86f80b4bf0 0x7f86f80b7100 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:52.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.586+0000 7f871ba00640 1 --2- 172.21.15.16:0/995529422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f871410bdb0 0x7f87141a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:52.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.586+0000 7f871ba00640 1 --2- 172.21.15.16:0/995529422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f871410b460 0x7f871419c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:52.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.586+0000 7f871ba00640 1 --2- 172.21.15.16:0/995529422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8714105b20 0x7f871419c450 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:52.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.586+0000 7f871ba00640 1 -- 172.21.15.16:0/995529422 >> 172.21.15.16:0/995529422 conn(0x7f87141015b0 msgr2=0x7f8714108a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:52.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.586+0000 7f871ba00640 1 -- 172.21.15.16:0/995529422 shutdown_connections 2024-08-31T05:57:52.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:52.586+0000 7f871ba00640 1 -- 172.21.15.16:0/995529422 wait complete. 2024-08-31T05:57:53.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:52 smithi155 ceph-mon[21742]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:57:53.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:52 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/1776437087' entity='client.admin' 2024-08-31T05:57:53.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:52 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:53.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:52 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:53.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:52 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:57:53.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:52 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:57:53.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:52 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T05:57:53.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:52 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:53.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:52 smithi155 ceph-mon[21742]: from='client.? ' entity='client.admin' 2024-08-31T05:57:53.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:52 smithi016 ceph-mon[17194]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:57:53.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:52 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/1776437087' entity='client.admin' 2024-08-31T05:57:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:52 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:52 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:52 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:57:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:52 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:57:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:52 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T05:57:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:52 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:52 smithi016 ceph-mon[17194]: from='client.? ' entity='client.admin' 2024-08-31T05:57:53.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:52 smithi184 ceph-mon[22331]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:57:53.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:52 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/1776437087' entity='client.admin' 2024-08-31T05:57:53.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:52 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:53.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:52 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:53.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:52 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:57:53.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:52 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:57:53.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:52 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T05:57:53.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:52 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:53.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:52 smithi184 ceph-mon[22331]: from='client.? ' entity='client.admin' 2024-08-31T05:57:54.700 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T05:57:55.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.056+0000 7f3572400640 1 -- 172.21.15.16:0/4098754850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f356c1050c0 msgr2=0x7f356c10d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:55.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.056+0000 7f3572400640 1 --2- 172.21.15.16:0/4098754850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f356c1050c0 0x7f356c10d610 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3558009900 tx=0x7f355802a3f0 comp rx=0 tx=0).stop 2024-08-31T05:57:55.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.056+0000 7f3572400640 1 -- 172.21.15.16:0/4098754850 shutdown_connections 2024-08-31T05:57:55.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.056+0000 7f3572400640 1 --2- 172.21.15.16:0/4098754850 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f356c10dbd0 0x7f356c1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:55.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.056+0000 7f3572400640 1 --2- 172.21.15.16:0/4098754850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f356c1050c0 0x7f356c10d610 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:55.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.056+0000 7f3572400640 1 --2- 172.21.15.16:0/4098754850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f356c104770 0x7f356c104b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:55.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.056+0000 7f3572400640 1 -- 172.21.15.16:0/4098754850 >> 172.21.15.16:0/4098754850 conn(0x7f356c0794e0 msgr2=0x7f356c0798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:55.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.056+0000 7f3572400640 1 -- 172.21.15.16:0/4098754850 shutdown_connections 2024-08-31T05:57:55.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.056+0000 7f3572400640 1 -- 172.21.15.16:0/4098754850 wait complete. 2024-08-31T05:57:55.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.058+0000 7f3572400640 1 Processor -- start 2024-08-31T05:57:55.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.058+0000 7f3572400640 1 -- start start 2024-08-31T05:57:55.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.058+0000 7f3572400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f356c104770 0x7f356c19a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:55.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.058+0000 7f3572400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f356c1050c0 0x7f356c19a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:55.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.058+0000 7f3572400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f356c10dbd0 0x7f356c19fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:55.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.059+0000 7f3572400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f356c113840 con 0x7f356c10dbd0 2024-08-31T05:57:55.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.059+0000 7f3572400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f356c1136c0 con 0x7f356c1050c0 2024-08-31T05:57:55.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.059+0000 7f3572400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f356c1139c0 con 0x7f356c104770 2024-08-31T05:57:55.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.059+0000 7f356be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f356c10dbd0 0x7f356c19fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:55.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.059+0000 7f356aa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f356c1050c0 0x7f356c19a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:55.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.059+0000 7f356b400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f356c104770 0x7f356c19a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:55.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.059+0000 7f356aa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f356c1050c0 0x7f356c19a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:52216/0 (socket says 172.21.15.16:52216) 2024-08-31T05:57:55.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.059+0000 7f356aa00640 1 -- 172.21.15.16:0/1644734372 learned_addr learned my addr 172.21.15.16:0/1644734372 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T05:57:55.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.059+0000 7f356b400640 1 -- 172.21.15.16:0/1644734372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f356c1050c0 msgr2=0x7f356c19a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:55.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.059+0000 7f356b400640 1 --2- 172.21.15.16:0/1644734372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f356c1050c0 0x7f356c19a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:55.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.059+0000 7f356b400640 1 -- 172.21.15.16:0/1644734372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f356c10dbd0 msgr2=0x7f356c19fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:55.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.059+0000 7f356b400640 1 --2- 172.21.15.16:0/1644734372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f356c10dbd0 0x7f356c19fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:55.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.059+0000 7f356b400640 1 -- 172.21.15.16:0/1644734372 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f356c1a0200 con 0x7f356c104770 2024-08-31T05:57:55.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.059+0000 7f356aa00640 1 --2- 172.21.15.16:0/1644734372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f356c1050c0 0x7f356c19a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T05:57:55.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.060+0000 7f356b400640 1 --2- 172.21.15.16:0/1644734372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f356c104770 0x7f356c19a290 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f355c00e9c0 tx=0x7f355c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:57:55.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.060+0000 7f3563e00640 1 -- 172.21.15.16:0/1644734372 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355c00ccc0 con 0x7f356c104770 2024-08-31T05:57:55.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.060+0000 7f3572400640 1 -- 172.21.15.16:0/1644734372 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f356c1a04f0 con 0x7f356c104770 2024-08-31T05:57:55.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.060+0000 7f3563e00640 1 -- 172.21.15.16:0/1644734372 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f355c0035f0 con 0x7f356c104770 2024-08-31T05:57:55.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.060+0000 7f3563e00640 1 -- 172.21.15.16:0/1644734372 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355c00ce60 con 0x7f356c104770 2024-08-31T05:57:55.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.060+0000 7f3572400640 1 -- 172.21.15.16:0/1644734372 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f356c1a6c50 con 0x7f356c104770 2024-08-31T05:57:55.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.061+0000 7f3572400640 1 -- 172.21.15.16:0/1644734372 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f356c06bfe0 con 0x7f356c104770 2024-08-31T05:57:55.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.064+0000 7f3563e00640 1 -- 172.21.15.16:0/1644734372 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 19) ==== 152538+0+0 (secure 0 0 0) 0x7f355c004000 con 0x7f356c104770 2024-08-31T05:57:55.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.065+0000 7f3563e00640 1 --2- 172.21.15.16:0/1644734372 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f35500b4af0 0x7f35500b7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:55.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.065+0000 7f3563e00640 1 -- 172.21.15.16:0/1644734372 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f355c0e3a10 con 0x7f356c104770 2024-08-31T05:57:55.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.065+0000 7f356aa00640 1 --2- 172.21.15.16:0/1644734372 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f35500b4af0 0x7f35500b7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:55.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.066+0000 7f356aa00640 1 --2- 172.21.15.16:0/1644734372 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f35500b4af0 0x7f35500b7000 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3558006010 tx=0x7f3558005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:57:55.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.067+0000 7f3563e00640 1 -- 172.21.15.16:0/1644734372 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f355c0ac6a0 con 0x7f356c104770 2024-08-31T05:57:55.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:54 smithi155 ceph-mon[21742]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:57:55.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:54 smithi016 ceph-mon[17194]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:57:55.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:54 smithi184 ceph-mon[22331]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:57:55.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.263+0000 7f3572400640 1 -- 172.21.15.16:0/1644734372 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f356c1a6f00 con 0x7f356c104770 2024-08-31T05:57:55.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.271+0000 7f3563e00640 1 -- 172.21.15.16:0/1644734372 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v34)=0 v34) ==== 121+0+0 (secure 0 0 0) 0x7f355c0abfa0 con 0x7f356c104770 2024-08-31T05:57:55.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.273+0000 7f3572400640 1 -- 172.21.15.16:0/1644734372 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f35500b4af0 msgr2=0x7f35500b7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:55.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.273+0000 7f3572400640 1 --2- 172.21.15.16:0/1644734372 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f35500b4af0 0x7f35500b7000 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3558006010 tx=0x7f3558005a20 comp rx=0 tx=0).stop 2024-08-31T05:57:55.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.273+0000 7f3572400640 1 -- 172.21.15.16:0/1644734372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f356c104770 msgr2=0x7f356c19a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:55.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.273+0000 7f3572400640 1 --2- 172.21.15.16:0/1644734372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f356c104770 0x7f356c19a290 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f355c00e9c0 tx=0x7f355c00ee80 comp rx=0 tx=0).stop 2024-08-31T05:57:55.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.273+0000 7f3572400640 1 -- 172.21.15.16:0/1644734372 shutdown_connections 2024-08-31T05:57:55.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.273+0000 7f3572400640 1 --2- 172.21.15.16:0/1644734372 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f35500b4af0 0x7f35500b7000 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:55.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.273+0000 7f3572400640 1 --2- 172.21.15.16:0/1644734372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f356c10dbd0 0x7f356c19fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:55.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.273+0000 7f3572400640 1 --2- 172.21.15.16:0/1644734372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f356c1050c0 0x7f356c19a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:55.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.273+0000 7f3572400640 1 --2- 172.21.15.16:0/1644734372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f356c104770 0x7f356c19a290 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:55.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.273+0000 7f3572400640 1 -- 172.21.15.16:0/1644734372 >> 172.21.15.16:0/1644734372 conn(0x7f356c0794e0 msgr2=0x7f356c1065e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:55.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.273+0000 7f3572400640 1 -- 172.21.15.16:0/1644734372 shutdown_connections 2024-08-31T05:57:55.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:55.273+0000 7f3572400640 1 -- 172.21.15.16:0/1644734372 wait complete. 2024-08-31T05:57:55.604 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '673e4aa0-675d-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi016') 2024-08-31T05:57:55.604 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '673e4aa0-675d-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi016') 2024-08-31T05:57:55.604 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '673e4aa0-675d-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi016') 2024-08-31T05:57:55.604 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '673e4aa0-675d-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi016') 2024-08-31T05:57:55.604 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '673e4aa0-675d-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi016') 2024-08-31T05:57:55.604 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '673e4aa0-675d-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi016') 2024-08-31T05:57:55.604 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '673e4aa0-675d-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi016') 2024-08-31T05:57:55.605 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '673e4aa0-675d-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi016') 2024-08-31T05:57:56.572 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:56 smithi016 ceph-mon[17194]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:57:56.572 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:56 smithi016 ceph-mon[17194]: from='client.? ' entity='client.admin' 2024-08-31T05:57:56.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:56 smithi155 ceph-mon[21742]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:57:56.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:56 smithi155 ceph-mon[21742]: from='client.? ' entity='client.admin' 2024-08-31T05:57:56.709 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '673e4aa0-675d-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi016') 2024-08-31T05:57:56.710 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '673e4aa0-675d-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi016') 2024-08-31T05:57:56.710 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '673e4aa0-675d-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mgr', 'debug_cephsqlite', '20'], check_status=True, hostname='smithi016') 2024-08-31T05:57:56.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:56 smithi184 ceph-mon[22331]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:57:56.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:56 smithi184 ceph-mon[22331]: from='client.? ' entity='client.admin' 2024-08-31T05:57:57.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.028+0000 7fe95e600640 1 -- 172.21.15.16:0/3763218863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9581099e0 msgr2=0x7fe958110210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:57.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.028+0000 7fe95e600640 1 --2- 172.21.15.16:0/3763218863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9581099e0 0x7fe958110210 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe950009ca0 tx=0x7fe95002a8c0 comp rx=0 tx=0).stop 2024-08-31T05:57:57.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.029+0000 7fe95e600640 1 -- 172.21.15.16:0/3763218863 shutdown_connections 2024-08-31T05:57:57.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.029+0000 7fe95e600640 1 --2- 172.21.15.16:0/3763218863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe958109090 0x7fe9581094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:57.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.029+0000 7fe95e600640 1 --2- 172.21.15.16:0/3763218863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9581099e0 0x7fe958110210 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:57.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.029+0000 7fe95e600640 1 --2- 172.21.15.16:0/3763218863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe958103750 0x7fe958103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:57.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.029+0000 7fe95e600640 1 -- 172.21.15.16:0/3763218863 >> 172.21.15.16:0/3763218863 conn(0x7fe9580ff1a0 msgr2=0x7fe9581015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:57.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.029+0000 7fe95e600640 1 -- 172.21.15.16:0/3763218863 shutdown_connections 2024-08-31T05:57:57.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.029+0000 7fe95e600640 1 -- 172.21.15.16:0/3763218863 wait complete. 2024-08-31T05:57:57.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.030+0000 7fe95e600640 1 Processor -- start 2024-08-31T05:57:57.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.030+0000 7fe95e600640 1 -- start start 2024-08-31T05:57:57.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.030+0000 7fe95e600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe958103750 0x7fe95819c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:57.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.030+0000 7fe95e600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe958109090 0x7fe95819c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:57.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.030+0000 7fe95e600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9581099e0 0x7fe9581a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:57.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.030+0000 7fe95e600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe9581136b0 con 0x7fe9581099e0 2024-08-31T05:57:57.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.030+0000 7fe95e600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe958113530 con 0x7fe958109090 2024-08-31T05:57:57.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.030+0000 7fe95e600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe958113830 con 0x7fe958103750 2024-08-31T05:57:57.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.031+0000 7fe95d200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe958103750 0x7fe95819c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:57.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.031+0000 7fe95dc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9581099e0 0x7fe9581a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:57.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.031+0000 7fe95d200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe958103750 0x7fe95819c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:46908/0 (socket says 172.21.15.16:46908) 2024-08-31T05:57:57.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.031+0000 7fe95d200640 1 -- 172.21.15.16:0/334756331 learned_addr learned my addr 172.21.15.16:0/334756331 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T05:57:57.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.031+0000 7fe957e00640 1 --2- 172.21.15.16:0/334756331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe958109090 0x7fe95819c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:57.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.031+0000 7fe95d200640 1 -- 172.21.15.16:0/334756331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe958109090 msgr2=0x7fe95819c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:57.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.031+0000 7fe95d200640 1 --2- 172.21.15.16:0/334756331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe958109090 0x7fe95819c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:57.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.031+0000 7fe95d200640 1 -- 172.21.15.16:0/334756331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9581099e0 msgr2=0x7fe9581a1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:57.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.031+0000 7fe95d200640 1 --2- 172.21.15.16:0/334756331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9581099e0 0x7fe9581a1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:57.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.031+0000 7fe95d200640 1 -- 172.21.15.16:0/334756331 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9581a21d0 con 0x7fe958103750 2024-08-31T05:57:57.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.031+0000 7fe95d200640 1 --2- 172.21.15.16:0/334756331 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe958103750 0x7fe95819c260 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe94c00e9f0 tx=0x7fe94c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:57:57.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.032+0000 7fe955600640 1 -- 172.21.15.16:0/334756331 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94c00cd80 con 0x7fe958103750 2024-08-31T05:57:57.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.032+0000 7fe955600640 1 -- 172.21.15.16:0/334756331 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe94c0108e0 con 0x7fe958103750 2024-08-31T05:57:57.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.032+0000 7fe95e600640 1 -- 172.21.15.16:0/334756331 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9581a24c0 con 0x7fe958103750 2024-08-31T05:57:57.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.032+0000 7fe955600640 1 -- 172.21.15.16:0/334756331 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94c018980 con 0x7fe958103750 2024-08-31T05:57:57.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.032+0000 7fe95e600640 1 -- 172.21.15.16:0/334756331 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9581a8c20 con 0x7fe958103750 2024-08-31T05:57:57.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.033+0000 7fe95e600640 1 -- 172.21.15.16:0/334756331 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe958103b60 con 0x7fe958103750 2024-08-31T05:57:57.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.035+0000 7fe955600640 1 -- 172.21.15.16:0/334756331 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 19) ==== 152538+0+0 (secure 0 0 0) 0x7fe94c010430 con 0x7fe958103750 2024-08-31T05:57:57.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.037+0000 7fe955600640 1 --2- 172.21.15.16:0/334756331 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe9400b4af0 0x7fe9400b7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:57:57.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.037+0000 7fe955600640 1 -- 172.21.15.16:0/334756331 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fe94c0e3df0 con 0x7fe958103750 2024-08-31T05:57:57.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.037+0000 7fe957e00640 1 --2- 172.21.15.16:0/334756331 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe9400b4af0 0x7fe9400b7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:57:57.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.038+0000 7fe957e00640 1 --2- 172.21.15.16:0/334756331 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe9400b4af0 0x7fe9400b7000 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe958105850 tx=0x7fe944005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:57:57.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.039+0000 7fe955600640 1 -- 172.21.15.16:0/334756331 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fe94c0ac430 con 0x7fe958103750 2024-08-31T05:57:57.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.235+0000 7fe95e600640 1 -- 172.21.15.16:0/334756331 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command([{prefix=config set, name=debug_cephsqlite}] v 0) -- 0x7fe9581a8f00 con 0x7fe958103750 2024-08-31T05:57:57.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.242+0000 7fe955600640 1 -- 172.21.15.16:0/334756331 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_cephsqlite}]=0 v35)=0 v35) ==== 115+0+0 (secure 0 0 0) 0x7fe94c0b0bf0 con 0x7fe958103750 2024-08-31T05:57:57.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.245+0000 7fe95e600640 1 -- 172.21.15.16:0/334756331 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe9400b4af0 msgr2=0x7fe9400b7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:57.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.245+0000 7fe95e600640 1 --2- 172.21.15.16:0/334756331 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe9400b4af0 0x7fe9400b7000 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe958105850 tx=0x7fe944005f90 comp rx=0 tx=0).stop 2024-08-31T05:57:57.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.245+0000 7fe95e600640 1 -- 172.21.15.16:0/334756331 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe958103750 msgr2=0x7fe95819c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:57:57.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.245+0000 7fe95e600640 1 --2- 172.21.15.16:0/334756331 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe958103750 0x7fe95819c260 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe94c00e9f0 tx=0x7fe94c00eeb0 comp rx=0 tx=0).stop 2024-08-31T05:57:57.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.245+0000 7fe95e600640 1 -- 172.21.15.16:0/334756331 shutdown_connections 2024-08-31T05:57:57.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.245+0000 7fe95e600640 1 --2- 172.21.15.16:0/334756331 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe9400b4af0 0x7fe9400b7000 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:57.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.245+0000 7fe95e600640 1 --2- 172.21.15.16:0/334756331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9581099e0 0x7fe9581a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:57.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.245+0000 7fe95e600640 1 --2- 172.21.15.16:0/334756331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe958109090 0x7fe95819c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:57.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.245+0000 7fe95e600640 1 --2- 172.21.15.16:0/334756331 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe958103750 0x7fe95819c260 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:57:57.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.245+0000 7fe95e600640 1 -- 172.21.15.16:0/334756331 >> 172.21.15.16:0/334756331 conn(0x7fe9580ff1a0 msgr2=0x7fe9581086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:57:57.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.245+0000 7fe95e600640 1 -- 172.21.15.16:0/334756331 shutdown_connections 2024-08-31T05:57:57.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:57:57.246+0000 7fe95e600640 1 -- 172.21.15.16:0/334756331 wait complete. 2024-08-31T05:57:57.930 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '673e4aa0-675d-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/allow_m_granularity', 'True'], check_status=True, hostname='smithi016') 2024-08-31T05:57:57.931 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '673e4aa0-675d-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/dump_on_update', 'True'], check_status=True, hostname='smithi016') 2024-08-31T05:57:57.931 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '673e4aa0-675d-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/log_level', 'debug'], check_status=True, hostname='smithi016') 2024-08-31T05:57:57.931 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '673e4aa0-675d-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi016') 2024-08-31T05:57:57.931 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '673e4aa0-675d-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'mon', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi016') 2024-08-31T05:57:57.931 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '673e4aa0-675d-11ef-bcd4-c7b262605968', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi016') 2024-08-31T05:57:57.931 INFO:tasks.cephadm:Deploying OSDs... 2024-08-31T05:57:57.932 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-08-31T05:57:57.932 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/scratch_devs of=/dev/stdout 2024-08-31T05:57:57.969 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-31T05:57:57.969 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_1 2024-08-31T05:57:58.048 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-31T05:57:58.048 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-31T05:57:58.048 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-08-31T05:57:58.048 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-31T05:57:58.048 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-08-31T05:57:58.048 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-08-31 05:56:32.019753959 +0000 2024-08-31T05:57:58.048 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-08-31 05:51:36.294314584 +0000 2024-08-31T05:57:58.048 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-08-31 05:51:36.294314584 +0000 2024-08-31T05:57:58.048 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-08-31 05:51:36.294314584 +0000 2024-08-31T05:57:58.049 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-31T05:57:58.147 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-08-31T05:57:58.147 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-08-31T05:57:58.147 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000186542 s, 2.7 MB/s 2024-08-31T05:57:58.150 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-31T05:57:58.189 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_2 2024-08-31T05:57:58.267 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-31T05:57:58.267 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-31T05:57:58.267 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-08-31T05:57:58.267 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-31T05:57:58.267 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-08-31T05:57:58.267 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-08-31 05:56:32.060753340 +0000 2024-08-31T05:57:58.267 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-08-31 05:51:36.748307911 +0000 2024-08-31T05:57:58.268 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-08-31 05:51:36.748307911 +0000 2024-08-31T05:57:58.268 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-08-31 05:51:36.748307911 +0000 2024-08-31T05:57:58.268 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-31T05:57:58.362 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-08-31T05:57:58.363 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-08-31T05:57:58.363 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000180949 s, 2.8 MB/s 2024-08-31T05:57:58.365 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-31T05:57:58.402 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_3 2024-08-31T05:57:58.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:58 smithi016 ceph-mon[17194]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:57:58.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:58 smithi016 ceph-mon[17194]: from='client.? ' entity='client.admin' 2024-08-31T05:57:58.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:58 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:57:58.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:58 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:57:58.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:58 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T05:57:58.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:57:58 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:58.447 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-31T05:57:58.447 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-31T05:57:58.447 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-08-31T05:57:58.447 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-31T05:57:58.447 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-08-31T05:57:58.448 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-08-31 05:56:32.100752736 +0000 2024-08-31T05:57:58.448 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-08-31 05:51:37.181301546 +0000 2024-08-31T05:57:58.448 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-08-31 05:51:37.181301546 +0000 2024-08-31T05:57:58.448 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-08-31 05:51:37.181301546 +0000 2024-08-31T05:57:58.448 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-31T05:57:58.525 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:58 smithi184 ceph-mon[22331]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:57:58.525 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:58 smithi184 ceph-mon[22331]: from='client.? ' entity='client.admin' 2024-08-31T05:57:58.525 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:58 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:57:58.525 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:58 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:57:58.526 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:58 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T05:57:58.526 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:57:58 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:58.539 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-08-31T05:57:58.540 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-08-31T05:57:58.540 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000197068 s, 2.6 MB/s 2024-08-31T05:57:58.542 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-31T05:57:58.579 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_4 2024-08-31T05:57:58.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:58 smithi155 ceph-mon[21742]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:57:58.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:58 smithi155 ceph-mon[21742]: from='client.? ' entity='client.admin' 2024-08-31T05:57:58.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:58 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:57:58.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:58 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:57:58.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:58 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T05:57:58.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:57:58 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:57:58.654 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-31T05:57:58.654 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-31T05:57:58.654 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 689 Links: 1 2024-08-31T05:57:58.654 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-31T05:57:58.654 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-08-31T05:57:58.655 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-08-31 05:56:32.140752132 +0000 2024-08-31T05:57:58.655 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-08-31 05:51:37.682294183 +0000 2024-08-31T05:57:58.655 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-08-31 05:51:37.682294183 +0000 2024-08-31T05:57:58.655 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-08-31 05:51:37.682294183 +0000 2024-08-31T05:57:58.655 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-31T05:57:58.749 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-08-31T05:57:58.749 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-08-31T05:57:58.749 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.00023117 s, 2.2 MB/s 2024-08-31T05:57:58.751 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-31T05:57:58.788 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-08-31T05:57:58.788 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/scratch_devs of=/dev/stdout 2024-08-31T05:57:58.808 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-31T05:57:58.808 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_1 2024-08-31T05:57:58.867 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-31T05:57:58.867 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-31T05:57:58.867 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-08-31T05:57:58.868 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-31T05:57:58.868 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-08-31T05:57:58.868 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-08-31 05:57:34.550148476 +0000 2024-08-31T05:57:58.868 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-08-31 05:51:37.165230566 +0000 2024-08-31T05:57:58.868 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-08-31 05:51:37.165230566 +0000 2024-08-31T05:57:58.868 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: 2024-08-31 05:51:37.165230566 +0000 2024-08-31T05:57:58.868 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-31T05:57:58.937 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-08-31T05:57:58.937 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-08-31T05:57:58.937 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000266641 s, 1.9 MB/s 2024-08-31T05:57:58.938 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-31T05:57:58.997 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_2 2024-08-31T05:57:59.055 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-31T05:57:59.055 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-31T05:57:59.055 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-08-31T05:57:59.055 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-31T05:57:59.055 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-08-31T05:57:59.055 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-08-31 05:57:34.550148476 +0000 2024-08-31T05:57:59.056 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-08-31 05:51:37.500223181 +0000 2024-08-31T05:57:59.056 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-08-31 05:51:37.500223181 +0000 2024-08-31T05:57:59.056 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: 2024-08-31 05:51:37.500223181 +0000 2024-08-31T05:57:59.056 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-31T05:57:59.121 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-08-31T05:57:59.121 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-08-31T05:57:59.121 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000216463 s, 2.4 MB/s 2024-08-31T05:57:59.123 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-31T05:57:59.180 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_3 2024-08-31T05:57:59.238 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-31T05:57:59.238 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-31T05:57:59.238 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-08-31T05:57:59.238 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-31T05:57:59.238 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-08-31T05:57:59.238 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-08-31 05:57:34.550148476 +0000 2024-08-31T05:57:59.238 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-08-31 05:51:37.824216039 +0000 2024-08-31T05:57:59.238 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-08-31 05:51:37.824216039 +0000 2024-08-31T05:57:59.239 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: 2024-08-31 05:51:37.824216039 +0000 2024-08-31T05:57:59.239 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-31T05:57:59.302 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-08-31T05:57:59.303 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-08-31T05:57:59.303 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000200228 s, 2.6 MB/s 2024-08-31T05:57:59.304 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-31T05:57:59.361 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_4 2024-08-31T05:57:59.416 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-31T05:57:59.417 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-31T05:57:59.417 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-08-31T05:57:59.417 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-31T05:57:59.417 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-08-31T05:57:59.417 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-08-31 05:57:34.551148454 +0000 2024-08-31T05:57:59.417 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-08-31 05:51:38.160208632 +0000 2024-08-31T05:57:59.417 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-08-31 05:51:38.160208632 +0000 2024-08-31T05:57:59.417 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: 2024-08-31 05:51:38.160208632 +0000 2024-08-31T05:57:59.418 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-31T05:57:59.481 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-08-31T05:57:59.481 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-08-31T05:57:59.481 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000220999 s, 2.3 MB/s 2024-08-31T05:57:59.482 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-31T05:57:59.540 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-08-31T05:57:59.540 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/scratch_devs of=/dev/stdout 2024-08-31T05:57:59.557 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-31T05:57:59.557 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_1 2024-08-31T05:57:59.613 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-31T05:57:59.614 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-31T05:57:59.614 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-08-31T05:57:59.614 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-31T05:57:59.614 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-08-31T05:57:59.614 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-08-31 05:57:32.159081178 +0000 2024-08-31T05:57:59.615 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-08-31 05:51:36.061730557 +0000 2024-08-31T05:57:59.615 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-08-31 05:51:36.061730557 +0000 2024-08-31T05:57:59.615 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-08-31 05:51:36.061730557 +0000 2024-08-31T05:57:59.615 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-31T05:57:59.679 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-08-31T05:57:59.680 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-08-31T05:57:59.680 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.00019445 s, 2.6 MB/s 2024-08-31T05:57:59.681 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-31T05:57:59.737 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_2 2024-08-31T05:57:59.792 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-31T05:57:59.792 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-31T05:57:59.792 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-08-31T05:57:59.793 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-31T05:57:59.793 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-08-31T05:57:59.793 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-08-31 05:57:32.160081157 +0000 2024-08-31T05:57:59.793 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-08-31 05:51:36.400723307 +0000 2024-08-31T05:57:59.793 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-08-31 05:51:36.400723307 +0000 2024-08-31T05:57:59.793 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-08-31 05:51:36.400723307 +0000 2024-08-31T05:57:59.793 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-31T05:57:59.856 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-08-31T05:57:59.856 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-08-31T05:57:59.857 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000204094 s, 2.5 MB/s 2024-08-31T05:57:59.858 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-31T05:57:59.914 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_3 2024-08-31T05:57:59.969 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-31T05:57:59.969 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-31T05:57:59.970 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-08-31T05:57:59.970 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-31T05:57:59.970 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-08-31T05:57:59.970 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-08-31 05:57:32.160081157 +0000 2024-08-31T05:57:59.970 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-08-31 05:51:36.725716351 +0000 2024-08-31T05:57:59.970 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-08-31 05:51:36.725716351 +0000 2024-08-31T05:57:59.971 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-08-31 05:51:36.725716351 +0000 2024-08-31T05:57:59.971 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-31T05:58:00.033 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-08-31T05:58:00.033 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-08-31T05:58:00.033 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000234094 s, 2.2 MB/s 2024-08-31T05:58:00.034 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-31T05:58:00.091 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_4 2024-08-31T05:58:00.146 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-31T05:58:00.146 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-31T05:58:00.146 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-08-31T05:58:00.146 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-31T05:58:00.147 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-08-31T05:58:00.147 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-08-31 05:57:32.160081157 +0000 2024-08-31T05:58:00.147 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-08-31 05:51:37.057709246 +0000 2024-08-31T05:58:00.147 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-08-31 05:51:37.057709246 +0000 2024-08-31T05:58:00.147 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-08-31 05:51:37.057709246 +0000 2024-08-31T05:58:00.147 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-31T05:58:00.210 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-08-31T05:58:00.210 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-08-31T05:58:00.210 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.00023224 s, 2.2 MB/s 2024-08-31T05:58:00.211 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-31T05:58:00.268 INFO:tasks.cephadm:Deploying osd.0 on smithi016 with /dev/vg_nvme/lv_4... 2024-08-31T05:58:00.268 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-08-31T05:58:00.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:00 smithi184 ceph-mon[22331]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:00.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:00 smithi155 ceph-mon[21742]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:00.649 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T05:58:00.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:00 smithi016 ceph-mon[17194]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:02.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:02 smithi016 ceph-mon[17194]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:02.621 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:02 smithi155 ceph-mon[21742]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:02.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:02 smithi184 ceph-mon[22331]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:03.581 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T05:58:03.613 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph orch daemon add osd smithi016:vg_nvme/lv_4 2024-08-31T05:58:03.981 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T05:58:04.402 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:04 smithi016 ceph-mon[17194]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:04.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:04.581+0000 7f5539e00640 1 -- 172.21.15.16:0/447676918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f553410bdc0 msgr2=0x7f55341125f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:58:04.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:04.581+0000 7f5539e00640 1 --2- 172.21.15.16:0/447676918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f553410bdc0 0x7f55341125f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f552c009ca0 tx=0x7f552c02a8c0 comp rx=0 tx=0).stop 2024-08-31T05:58:04.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:04.581+0000 7f5539e00640 1 -- 172.21.15.16:0/447676918 shutdown_connections 2024-08-31T05:58:04.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:04.581+0000 7f5539e00640 1 --2- 172.21.15.16:0/447676918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f553410bdc0 0x7f55341125f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:58:04.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:04.581+0000 7f5539e00640 1 --2- 172.21.15.16:0/447676918 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f553410b470 0x7f553410b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:58:04.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:04.581+0000 7f5539e00640 1 --2- 172.21.15.16:0/447676918 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5534105b30 0x7f5534105f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:58:04.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:04.581+0000 7f5539e00640 1 -- 172.21.15.16:0/447676918 >> 172.21.15.16:0/447676918 conn(0x7f5534101600 msgr2=0x7f5534103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:58:04.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:04.582+0000 7f5539e00640 1 -- 172.21.15.16:0/447676918 shutdown_connections 2024-08-31T05:58:04.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:04.582+0000 7f5539e00640 1 -- 172.21.15.16:0/447676918 wait complete. 2024-08-31T05:58:04.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:04.583+0000 7f5539e00640 1 Processor -- start 2024-08-31T05:58:04.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:04.583+0000 7f5539e00640 1 -- start start 2024-08-31T05:58:04.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:04.583+0000 7f5539e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5534105b30 0x7f553419c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:58:04.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:04.583+0000 7f5539e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f553410b470 0x7f553419c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:58:04.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:04.583+0000 7f5539e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f553410bdc0 0x7f55341a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:58:04.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:04.583+0000 7f5539e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5534115ad0 con 0x7f553410bdc0 2024-08-31T05:58:04.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:04.583+0000 7f5539e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5534115950 con 0x7f5534105b30 2024-08-31T05:58:04.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:04.583+0000 7f5539e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5534115c50 con 0x7f553410b470 2024-08-31T05:58:04.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:04.584+0000 7f5533400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f553410bdc0 0x7f55341a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:58:04.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:04.584+0000 7f5532000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f553410b470 0x7f553419c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:58:04.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:04.584+0000 7f5532a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5534105b30 0x7f553419c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:58:04.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:04.584+0000 7f5532000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f553410b470 0x7f553419c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:46932/0 (socket says 172.21.15.16:46932) 2024-08-31T05:58:04.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:04.584+0000 7f5532000640 1 -- 172.21.15.16:0/4285549725 learned_addr learned my addr 172.21.15.16:0/4285549725 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T05:58:04.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:04.584+0000 7f5533400640 1 -- 172.21.15.16:0/4285549725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f553410b470 msgr2=0x7f553419c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:58:04.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:04.584+0000 7f5533400640 1 --2- 172.21.15.16:0/4285549725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f553410b470 0x7f553419c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:58:04.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:04.584+0000 7f5533400640 1 -- 172.21.15.16:0/4285549725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5534105b30 msgr2=0x7f553419c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:58:04.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:04.584+0000 7f5533400640 1 --2- 172.21.15.16:0/4285549725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5534105b30 0x7f553419c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:58:04.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:04.584+0000 7f5533400640 1 -- 172.21.15.16:0/4285549725 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55341a23b0 con 0x7f553410bdc0 2024-08-31T05:58:04.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:04.584+0000 7f5532a00640 1 --2- 172.21.15.16:0/4285549725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5534105b30 0x7f553419c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T05:58:04.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:04.584+0000 7f5533400640 1 --2- 172.21.15.16:0/4285549725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f553410bdc0 0x7f55341a1c80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f552c005e30 tx=0x7f552c0317c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:58:04.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:04.585+0000 7f5538a00640 1 -- 172.21.15.16:0/4285549725 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552c003b70 con 0x7f553410bdc0 2024-08-31T05:58:04.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:04.585+0000 7f5538a00640 1 -- 172.21.15.16:0/4285549725 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f552c003d10 con 0x7f553410bdc0 2024-08-31T05:58:04.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:04.585+0000 7f5538a00640 1 -- 172.21.15.16:0/4285549725 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552c02f970 con 0x7f553410bdc0 2024-08-31T05:58:04.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:04.585+0000 7f5539e00640 1 -- 172.21.15.16:0/4285549725 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55341a2640 con 0x7f553410bdc0 2024-08-31T05:58:04.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:04.585+0000 7f5539e00640 1 -- 172.21.15.16:0/4285549725 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55341a8e00 con 0x7f553410bdc0 2024-08-31T05:58:04.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:04.586+0000 7f5539e00640 1 -- 172.21.15.16:0/4285549725 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5534105f40 con 0x7f553410bdc0 2024-08-31T05:58:04.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:04.588+0000 7f5538a00640 1 -- 172.21.15.16:0/4285549725 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) ==== 152538+0+0 (secure 0 0 0) 0x7f552c004b40 con 0x7f553410bdc0 2024-08-31T05:58:04.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:04.590+0000 7f5538a00640 1 --2- 172.21.15.16:0/4285549725 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f55180b4b70 0x7f55180b7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:58:04.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:04.590+0000 7f5538a00640 1 -- 172.21.15.16:0/4285549725 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f552c104770 con 0x7f553410bdc0 2024-08-31T05:58:04.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:04.590+0000 7f5532a00640 1 --2- 172.21.15.16:0/4285549725 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f55180b4b70 0x7f55180b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:58:04.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:04.591+0000 7f5532a00640 1 --2- 172.21.15.16:0/4285549725 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f55180b4b70 0x7f55180b7080 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5524009fc0 tx=0x7f5524005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:58:04.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:04.592+0000 7f5538a00640 1 -- 172.21.15.16:0/4285549725 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f552c0cd490 con 0x7f553410bdc0 2024-08-31T05:58:04.621 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:04 smithi155 ceph-mon[21742]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:04.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:04 smithi184 ceph-mon[22331]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:04.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:04.802+0000 7f5539e00640 1 -- 172.21.15.16:0/4285549725 --> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f553410dc10 con 0x7f55180b4b70 2024-08-31T05:58:05.534 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:05 smithi016 ceph-mon[17194]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T05:58:05.534 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:05 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T05:58:05.534 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:05 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T05:58:05.534 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:05 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:58:05.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:05 smithi155 ceph-mon[21742]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T05:58:05.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:05 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T05:58:05.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:05 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T05:58:05.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:05 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:58:05.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:05 smithi184 ceph-mon[22331]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T05:58:05.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:05 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T05:58:05.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:05 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T05:58:05.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:05 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:58:06.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:06 smithi016 ceph-mon[17194]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:06.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:06 smithi155 ceph-mon[21742]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:06.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:06 smithi184 ceph-mon[22331]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:08.451 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:08 smithi016 ceph-mon[17194]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:08.451 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:08 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/1950352113' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0e349dae-6aa5-4b6d-9107-a945655e11bd"} : dispatch 2024-08-31T05:58:08.452 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:08 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/1950352113' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e349dae-6aa5-4b6d-9107-a945655e11bd"}]': finished 2024-08-31T05:58:08.452 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:08 smithi016 ceph-mon[17194]: osdmap e6: 1 total, 0 up, 1 in 2024-08-31T05:58:08.452 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:08 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-31T05:58:08.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:08 smithi155 ceph-mon[21742]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:08.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:08 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/1950352113' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0e349dae-6aa5-4b6d-9107-a945655e11bd"} : dispatch 2024-08-31T05:58:08.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:08 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/1950352113' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e349dae-6aa5-4b6d-9107-a945655e11bd"}]': finished 2024-08-31T05:58:08.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:08 smithi155 ceph-mon[21742]: osdmap e6: 1 total, 0 up, 1 in 2024-08-31T05:58:08.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:08 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-31T05:58:08.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:08 smithi184 ceph-mon[22331]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:08.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:08 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/1950352113' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0e349dae-6aa5-4b6d-9107-a945655e11bd"} : dispatch 2024-08-31T05:58:08.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:08 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/1950352113' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e349dae-6aa5-4b6d-9107-a945655e11bd"}]': finished 2024-08-31T05:58:08.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:08 smithi184 ceph-mon[22331]: osdmap e6: 1 total, 0 up, 1 in 2024-08-31T05:58:08.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:08 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-31T05:58:09.621 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:09 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/2543769195' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T05:58:09.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:09 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/2543769195' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T05:58:09.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:09 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/2543769195' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T05:58:10.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:10 smithi155 ceph-mon[21742]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:10.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:10 smithi016 ceph-mon[17194]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:10.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:10 smithi184 ceph-mon[22331]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:12.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:12 smithi155 ceph-mon[21742]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:12.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:12 smithi016 ceph-mon[17194]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:12.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:12 smithi184 ceph-mon[22331]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:14.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:14 smithi155 ceph-mon[21742]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:14.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:14 smithi016 ceph-mon[17194]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:14.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:14 smithi184 ceph-mon[22331]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:16.450 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:16 smithi016 ceph-mon[17194]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:16.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:16 smithi155 ceph-mon[21742]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:16.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:16 smithi184 ceph-mon[22331]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:18.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:18 smithi016 ceph-mon[17194]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:18.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:18 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-08-31T05:58:18.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:18 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:58:18.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:18 smithi016 ceph-mon[17194]: Deploying daemon osd.0 on smithi016 2024-08-31T05:58:18.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:18 smithi155 ceph-mon[21742]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:18.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:18 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-08-31T05:58:18.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:18 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:58:18.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:18 smithi155 ceph-mon[21742]: Deploying daemon osd.0 on smithi016 2024-08-31T05:58:18.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:18 smithi184 ceph-mon[22331]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:18.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:18 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-08-31T05:58:18.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:18 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:58:18.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:18 smithi184 ceph-mon[22331]: Deploying daemon osd.0 on smithi016 2024-08-31T05:58:20.382 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:20 smithi016 ceph-mon[17194]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:20.621 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:20 smithi155 ceph-mon[21742]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:20.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:20 smithi184 ceph-mon[22331]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:22.391 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:22 smithi016 ceph-mon[17194]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:22.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:22 smithi155 ceph-mon[21742]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:22.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:22 smithi184 ceph-mon[22331]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:24.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:24 smithi155 ceph-mon[21742]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:24.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:24 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:58:24.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:24 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:24.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:24 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:24.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:24 smithi016 ceph-mon[17194]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:24.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:24 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:58:24.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:24 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:24.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:24 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:24.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:24 smithi184 ceph-mon[22331]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:24.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:24 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:58:24.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:24 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:24.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:24 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:26.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:26 smithi155 ceph-mon[21742]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:26.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:26 smithi016 ceph-mon[17194]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:26.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:26 smithi184 ceph-mon[22331]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:28.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:27 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:28.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:27 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:28.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:27 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:58:28.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:27 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T05:58:28.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:27 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:28.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:27 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:28.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:27 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:28.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:27 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:58:28.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:27 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T05:58:28.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:27 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:28.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:27 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:28.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:27 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:28.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:27 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:58:28.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:27 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T05:58:28.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:27 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:29.129 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:28 smithi016 ceph-mon[17194]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:29.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:28 smithi184 ceph-mon[22331]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:29.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:28 smithi155 ceph-mon[21742]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:29.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:29.778+0000 7f5538a00640 1 -- 172.21.15.16:0/4285549725 <== mgr.14220 v2:172.21.15.16:6800/4279751337 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f553410dc10 con 0x7f55180b4b70 2024-08-31T05:58:29.781 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 0 on host 'smithi016' 2024-08-31T05:58:29.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:29.781+0000 7f5539e00640 1 -- 172.21.15.16:0/4285549725 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f55180b4b70 msgr2=0x7f55180b7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:58:29.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:29.782+0000 7f5539e00640 1 --2- 172.21.15.16:0/4285549725 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f55180b4b70 0x7f55180b7080 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5524009fc0 tx=0x7f5524005f90 comp rx=0 tx=0).stop 2024-08-31T05:58:29.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:29.782+0000 7f5539e00640 1 -- 172.21.15.16:0/4285549725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f553410bdc0 msgr2=0x7f55341a1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:58:29.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:29.782+0000 7f5539e00640 1 --2- 172.21.15.16:0/4285549725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f553410bdc0 0x7f55341a1c80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f552c005e30 tx=0x7f552c0317c0 comp rx=0 tx=0).stop 2024-08-31T05:58:29.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:29.782+0000 7f5539e00640 1 -- 172.21.15.16:0/4285549725 shutdown_connections 2024-08-31T05:58:29.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:29.782+0000 7f5539e00640 1 --2- 172.21.15.16:0/4285549725 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f55180b4b70 0x7f55180b7080 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:58:29.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:29.782+0000 7f5539e00640 1 --2- 172.21.15.16:0/4285549725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f553410b470 0x7f553419c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:58:29.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:29.782+0000 7f5539e00640 1 --2- 172.21.15.16:0/4285549725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f553410bdc0 0x7f55341a1c80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:58:29.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:29.782+0000 7f5539e00640 1 --2- 172.21.15.16:0/4285549725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5534105b30 0x7f553419c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:58:29.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:29.782+0000 7f5539e00640 1 -- 172.21.15.16:0/4285549725 >> 172.21.15.16:0/4285549725 conn(0x7f5534101600 msgr2=0x7f55341089c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:58:29.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:29.782+0000 7f5539e00640 1 -- 172.21.15.16:0/4285549725 shutdown_connections 2024-08-31T05:58:29.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:58:29.782+0000 7f5539e00640 1 -- 172.21.15.16:0/4285549725 wait complete. 2024-08-31T05:58:30.029 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:29 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:58:30.029 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:29 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:30.029 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:29 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:30.030 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:29 smithi016 ceph-mon[17194]: from='osd.0 [v2:172.21.15.16:6802/1725037400,v1:172.21.15.16:6803/1725037400]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-08-31T05:58:30.030 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:29 smithi016 ceph-mon[17194]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-08-31T05:58:30.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:29 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:58:30.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:29 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:30.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:29 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:30.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:29 smithi184 ceph-mon[22331]: from='osd.0 [v2:172.21.15.16:6802/1725037400,v1:172.21.15.16:6803/1725037400]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-08-31T05:58:30.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:29 smithi184 ceph-mon[22331]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-08-31T05:58:30.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:29 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:58:30.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:29 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:30.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:29 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:30.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:29 smithi155 ceph-mon[21742]: from='osd.0 [v2:172.21.15.16:6802/1725037400,v1:172.21.15.16:6803/1725037400]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-08-31T05:58:30.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:29 smithi155 ceph-mon[21742]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-08-31T05:58:30.539 DEBUG:teuthology.orchestra.run.smithi016:osd.0> sudo journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@osd.0.service 2024-08-31T05:58:30.543 INFO:tasks.cephadm:Deploying osd.1 on smithi155 with /dev/vg_nvme/lv_4... 2024-08-31T05:58:30.544 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-08-31T05:58:30.784 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.b/config 2024-08-31T05:58:31.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:30 smithi184 ceph-mon[22331]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:31.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:30 smithi184 ceph-mon[22331]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-08-31T05:58:31.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:30 smithi184 ceph-mon[22331]: osdmap e7: 1 total, 0 up, 1 in 2024-08-31T05:58:31.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:30 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-31T05:58:31.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:30 smithi184 ceph-mon[22331]: from='osd.0 [v2:172.21.15.16:6802/1725037400,v1:172.21.15.16:6803/1725037400]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-08-31T05:58:31.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:30 smithi184 ceph-mon[22331]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-08-31T05:58:31.237 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:30 smithi016 ceph-mon[17194]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:31.237 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:30 smithi016 ceph-mon[17194]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-08-31T05:58:31.238 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:30 smithi016 ceph-mon[17194]: osdmap e7: 1 total, 0 up, 1 in 2024-08-31T05:58:31.238 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:30 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-31T05:58:31.238 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:30 smithi016 ceph-mon[17194]: from='osd.0 [v2:172.21.15.16:6802/1725037400,v1:172.21.15.16:6803/1725037400]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-08-31T05:58:31.238 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:30 smithi016 ceph-mon[17194]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-08-31T05:58:31.238 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:30 smithi155 ceph-mon[21742]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:31.239 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:30 smithi155 ceph-mon[21742]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-08-31T05:58:31.239 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:30 smithi155 ceph-mon[21742]: osdmap e7: 1 total, 0 up, 1 in 2024-08-31T05:58:31.239 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:30 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-31T05:58:31.239 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:30 smithi155 ceph-mon[21742]: from='osd.0 [v2:172.21.15.16:6802/1725037400,v1:172.21.15.16:6803/1725037400]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-08-31T05:58:31.239 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:30 smithi155 ceph-mon[21742]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-08-31T05:58:32.502 INFO:journalctl@ceph.osd.0.smithi016.stdout:Aug 31 05:58:32 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-0[35211]: 2024-08-31T05:58:32.314+0000 7fbab7800640 -1 osd.0 0 waiting for initial osdmap 2024-08-31T05:58:32.502 INFO:journalctl@ceph.osd.0.smithi016.stdout:Aug 31 05:58:32 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-0[35211]: 2024-08-31T05:58:32.329+0000 7fbaace00640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-31T05:58:33.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:32 smithi155 ceph-mon[21742]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:33.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:32 smithi155 ceph-mon[21742]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-08-31T05:58:33.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:32 smithi155 ceph-mon[21742]: osdmap e8: 1 total, 0 up, 1 in 2024-08-31T05:58:33.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:32 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-31T05:58:33.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:32 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-31T05:58:33.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:32 smithi155 ceph-mon[21742]: from='osd.0 ' entity='osd.0' 2024-08-31T05:58:33.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:32 smithi016 ceph-mon[17194]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:33.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:32 smithi016 ceph-mon[17194]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-08-31T05:58:33.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:32 smithi016 ceph-mon[17194]: osdmap e8: 1 total, 0 up, 1 in 2024-08-31T05:58:33.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:32 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-31T05:58:33.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:32 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-31T05:58:33.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:32 smithi016 ceph-mon[17194]: from='osd.0 ' entity='osd.0' 2024-08-31T05:58:33.177 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-08-31T05:58:33.203 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph orch daemon add osd smithi155:vg_nvme/lv_4 2024-08-31T05:58:33.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:32 smithi184 ceph-mon[22331]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:33.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:32 smithi184 ceph-mon[22331]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-08-31T05:58:33.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:32 smithi184 ceph-mon[22331]: osdmap e8: 1 total, 0 up, 1 in 2024-08-31T05:58:33.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:32 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-31T05:58:33.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:32 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-31T05:58:33.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:32 smithi184 ceph-mon[22331]: from='osd.0 ' entity='osd.0' 2024-08-31T05:58:33.459 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.b/config 2024-08-31T05:58:33.969 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:33.965+0000 7f2c7f941640 1 -- 172.21.15.155:0/3027132767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c781045e0 msgr2=0x7f2c781049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:58:33.969 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:33.965+0000 7f2c7f941640 1 --2- 172.21.15.155:0/3027132767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c781045e0 0x7f2c781049f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2c74009900 tx=0x7f2c7402a410 comp rx=0 tx=0).stop 2024-08-31T05:58:33.969 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:33.965+0000 7f2c7f941640 1 -- 172.21.15.155:0/3027132767 shutdown_connections 2024-08-31T05:58:33.969 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:33.965+0000 7f2c7f941640 1 --2- 172.21.15.155:0/3027132767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c7810db90 0x7f2c781100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:58:33.969 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:33.965+0000 7f2c7f941640 1 --2- 172.21.15.155:0/3027132767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c78105010 0x7f2c7810d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:58:33.970 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:33.965+0000 7f2c7f941640 1 --2- 172.21.15.155:0/3027132767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c781045e0 0x7f2c781049f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:58:33.970 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:33.965+0000 7f2c7f941640 1 -- 172.21.15.155:0/3027132767 >> 172.21.15.155:0/3027132767 conn(0x7f2c780794d0 msgr2=0x7f2c780798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:58:33.970 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:33.965+0000 7f2c7f941640 1 -- 172.21.15.155:0/3027132767 shutdown_connections 2024-08-31T05:58:33.970 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:33.965+0000 7f2c7f941640 1 -- 172.21.15.155:0/3027132767 wait complete. 2024-08-31T05:58:33.970 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:33.965+0000 7f2c7f941640 1 Processor -- start 2024-08-31T05:58:33.970 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:33.965+0000 7f2c7f941640 1 -- start start 2024-08-31T05:58:33.970 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:33.966+0000 7f2c7f941640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c781045e0 0x7f2c781024f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:58:33.970 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:33.966+0000 7f2c7f941640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c78105010 0x7f2c78102a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:58:33.970 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:33.966+0000 7f2c7f941640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c7810db90 0x7f2c78102f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:58:33.970 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:33.966+0000 7f2c7f941640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2c78113830 con 0x7f2c7810db90 2024-08-31T05:58:33.970 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:33.966+0000 7f2c7f941640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2c781136b0 con 0x7f2c78105010 2024-08-31T05:58:33.970 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:33.966+0000 7f2c7f941640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2c781139b0 con 0x7f2c781045e0 2024-08-31T05:58:33.971 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:33.966+0000 7f2c7cedc640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c78105010 0x7f2c78102a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:58:33.971 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:33.966+0000 7f2c7cedc640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c78105010 0x7f2c78102a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:54364/0 (socket says 172.21.15.155:54364) 2024-08-31T05:58:33.971 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:33.966+0000 7f2c7cedc640 1 -- 172.21.15.155:0/64789449 learned_addr learned my addr 172.21.15.155:0/64789449 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-08-31T05:58:33.971 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:33.966+0000 7f2c7d6dd640 1 --2- 172.21.15.155:0/64789449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c781045e0 0x7f2c781024f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:58:33.971 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:33.966+0000 7f2c7dede640 1 --2- 172.21.15.155:0/64789449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c7810db90 0x7f2c78102f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:58:33.971 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:33.966+0000 7f2c7cedc640 1 -- 172.21.15.155:0/64789449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c781045e0 msgr2=0x7f2c781024f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:58:33.971 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:33.966+0000 7f2c7cedc640 1 --2- 172.21.15.155:0/64789449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c781045e0 0x7f2c781024f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:58:33.972 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:33.966+0000 7f2c7cedc640 1 -- 172.21.15.155:0/64789449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c7810db90 msgr2=0x7f2c78102f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:58:33.972 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:33.966+0000 7f2c7cedc640 1 --2- 172.21.15.155:0/64789449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c7810db90 0x7f2c78102f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:58:33.972 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:33.966+0000 7f2c7cedc640 1 -- 172.21.15.155:0/64789449 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c781a8580 con 0x7f2c78105010 2024-08-31T05:58:33.972 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:33.966+0000 7f2c7cedc640 1 --2- 172.21.15.155:0/64789449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c78105010 0x7f2c78102a30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2c6800d950 tx=0x7f2c6800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:58:33.972 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:33.967+0000 7f2c6e7fc640 1 -- 172.21.15.155:0/64789449 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c68017070 con 0x7f2c78105010 2024-08-31T05:58:33.972 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:33.967+0000 7f2c6e7fc640 1 -- 172.21.15.155:0/64789449 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c68009c30 con 0x7f2c78105010 2024-08-31T05:58:33.972 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:33.967+0000 7f2c6e7fc640 1 -- 172.21.15.155:0/64789449 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c680047a0 con 0x7f2c78105010 2024-08-31T05:58:33.972 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:33.967+0000 7f2c7f941640 1 -- 172.21.15.155:0/64789449 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c781a8870 con 0x7f2c78105010 2024-08-31T05:58:33.973 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:33.967+0000 7f2c7f941640 1 -- 172.21.15.155:0/64789449 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c781a8d60 con 0x7f2c78105010 2024-08-31T05:58:33.973 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:33.968+0000 7f2c7f941640 1 -- 172.21.15.155:0/64789449 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c781049f0 con 0x7f2c78105010 2024-08-31T05:58:33.973 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:33.970+0000 7f2c6e7fc640 1 -- 172.21.15.155:0/64789449 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 19) ==== 152538+0+0 (secure 0 0 0) 0x7f2c68004020 con 0x7f2c78105010 2024-08-31T05:58:33.974 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:33.971+0000 7f2c6e7fc640 1 --2- 172.21.15.155:0/64789449 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f2c640b4af0 0x7f2c640b7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:58:33.974 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:33.972+0000 7f2c6e7fc640 1 -- 172.21.15.155:0/64789449 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1982+0+0 (secure 0 0 0) 0x7f2c680e37e0 con 0x7f2c78105010 2024-08-31T05:58:33.974 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:33.972+0000 7f2c7d6dd640 1 --2- 172.21.15.155:0/64789449 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f2c640b4af0 0x7f2c640b7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:58:33.974 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:33.972+0000 7f2c7d6dd640 1 --2- 172.21.15.155:0/64789449 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f2c640b4af0 0x7f2c640b7000 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2c74009230 tx=0x7f2c74031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:58:33.976 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:33.974+0000 7f2c6e7fc640 1 -- 172.21.15.155:0/64789449 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f2c68020020 con 0x7f2c78105010 2024-08-31T05:58:34.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:33 smithi155 ceph-mon[21742]: purged_snaps scrub starts 2024-08-31T05:58:34.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:33 smithi155 ceph-mon[21742]: purged_snaps scrub ok 2024-08-31T05:58:34.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:33 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-31T05:58:34.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:33 smithi155 ceph-mon[21742]: osd.0 [v2:172.21.15.16:6802/1725037400,v1:172.21.15.16:6803/1725037400] boot 2024-08-31T05:58:34.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:33 smithi155 ceph-mon[21742]: osdmap e9: 1 total, 1 up, 1 in 2024-08-31T05:58:34.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:33 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-31T05:58:34.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:33 smithi016 ceph-mon[17194]: purged_snaps scrub starts 2024-08-31T05:58:34.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:33 smithi016 ceph-mon[17194]: purged_snaps scrub ok 2024-08-31T05:58:34.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:33 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-31T05:58:34.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:33 smithi016 ceph-mon[17194]: osd.0 [v2:172.21.15.16:6802/1725037400,v1:172.21.15.16:6803/1725037400] boot 2024-08-31T05:58:34.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:33 smithi016 ceph-mon[17194]: osdmap e9: 1 total, 1 up, 1 in 2024-08-31T05:58:34.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:33 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-31T05:58:34.165 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:34.163+0000 7f2c7f941640 1 -- 172.21.15.155:0/64789449 --> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f2c780635a0 con 0x7f2c640b4af0 2024-08-31T05:58:34.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:33 smithi184 ceph-mon[22331]: purged_snaps scrub starts 2024-08-31T05:58:34.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:33 smithi184 ceph-mon[22331]: purged_snaps scrub ok 2024-08-31T05:58:34.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:33 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-31T05:58:34.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:33 smithi184 ceph-mon[22331]: osd.0 [v2:172.21.15.16:6802/1725037400,v1:172.21.15.16:6803/1725037400] boot 2024-08-31T05:58:34.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:33 smithi184 ceph-mon[22331]: osdmap e9: 1 total, 1 up, 1 in 2024-08-31T05:58:34.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:33 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-31T05:58:34.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:34 smithi016 ceph-mon[17194]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:34.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:34 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T05:58:34.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:34 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T05:58:34.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:34 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:58:34.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:34 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:34.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:34 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:34.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:34 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T05:58:34.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:34 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T05:58:34.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:34 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:34.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:34 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:58:34.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:34 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T05:58:34.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:34 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:35.125 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:34 smithi155 ceph-mon[21742]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:35.125 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:34 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T05:58:35.125 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:34 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T05:58:35.125 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:34 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:58:35.125 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:34 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:35.125 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:34 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:35.125 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:34 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T05:58:35.126 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:34 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T05:58:35.126 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:34 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:35.126 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:34 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:58:35.126 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:34 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T05:58:35.126 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:34 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:35.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:34 smithi184 ceph-mon[22331]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-31T05:58:35.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:34 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T05:58:35.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:34 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T05:58:35.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:34 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:58:35.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:34 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:35.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:34 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:35.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:34 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T05:58:35.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:34 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T05:58:35.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:34 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:35.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:34 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:58:35.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:34 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T05:58:35.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:34 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:36.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:35 smithi155 ceph-mon[21742]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T05:58:36.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:35 smithi155 ceph-mon[21742]: Detected new or changed devices on smithi016 2024-08-31T05:58:36.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:35 smithi155 ceph-mon[21742]: Adjusting osd_memory_target on smithi016 to 17094M 2024-08-31T05:58:36.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:35 smithi155 ceph-mon[21742]: osdmap e10: 1 total, 1 up, 1 in 2024-08-31T05:58:36.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:35 smithi016 ceph-mon[17194]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T05:58:36.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:35 smithi016 ceph-mon[17194]: Detected new or changed devices on smithi016 2024-08-31T05:58:36.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:35 smithi016 ceph-mon[17194]: Adjusting osd_memory_target on smithi016 to 17094M 2024-08-31T05:58:36.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:35 smithi016 ceph-mon[17194]: osdmap e10: 1 total, 1 up, 1 in 2024-08-31T05:58:36.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:35 smithi184 ceph-mon[22331]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T05:58:36.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:35 smithi184 ceph-mon[22331]: Detected new or changed devices on smithi016 2024-08-31T05:58:36.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:35 smithi184 ceph-mon[22331]: Adjusting osd_memory_target on smithi016 to 17094M 2024-08-31T05:58:36.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:35 smithi184 ceph-mon[22331]: osdmap e10: 1 total, 1 up, 1 in 2024-08-31T05:58:37.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:36 smithi155 ceph-mon[21742]: pgmap v34: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T05:58:37.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:36 smithi155 ceph-mon[21742]: from='client.? 172.21.15.155:0/563196498' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bd645dc3-93b5-40af-89d8-08ccec22e50d"} : dispatch 2024-08-31T05:58:37.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:36 smithi155 ceph-mon[21742]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bd645dc3-93b5-40af-89d8-08ccec22e50d"} : dispatch 2024-08-31T05:58:37.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:36 smithi155 ceph-mon[21742]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd645dc3-93b5-40af-89d8-08ccec22e50d"}]': finished 2024-08-31T05:58:37.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:36 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-31T05:58:37.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:36 smithi155 ceph-mon[21742]: osdmap e11: 2 total, 1 up, 2 in 2024-08-31T05:58:37.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:36 smithi016 ceph-mon[17194]: pgmap v34: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T05:58:37.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:36 smithi016 ceph-mon[17194]: from='client.? 172.21.15.155:0/563196498' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bd645dc3-93b5-40af-89d8-08ccec22e50d"} : dispatch 2024-08-31T05:58:37.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:36 smithi016 ceph-mon[17194]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bd645dc3-93b5-40af-89d8-08ccec22e50d"} : dispatch 2024-08-31T05:58:37.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:36 smithi016 ceph-mon[17194]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd645dc3-93b5-40af-89d8-08ccec22e50d"}]': finished 2024-08-31T05:58:37.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:36 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-31T05:58:37.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:36 smithi016 ceph-mon[17194]: osdmap e11: 2 total, 1 up, 2 in 2024-08-31T05:58:37.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:36 smithi184 ceph-mon[22331]: pgmap v34: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T05:58:37.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:36 smithi184 ceph-mon[22331]: from='client.? 172.21.15.155:0/563196498' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bd645dc3-93b5-40af-89d8-08ccec22e50d"} : dispatch 2024-08-31T05:58:37.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:36 smithi184 ceph-mon[22331]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bd645dc3-93b5-40af-89d8-08ccec22e50d"} : dispatch 2024-08-31T05:58:37.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:36 smithi184 ceph-mon[22331]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd645dc3-93b5-40af-89d8-08ccec22e50d"}]': finished 2024-08-31T05:58:37.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:36 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-31T05:58:37.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:36 smithi184 ceph-mon[22331]: osdmap e11: 2 total, 1 up, 2 in 2024-08-31T05:58:38.121 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:37 smithi155 ceph-mon[21742]: from='client.? 172.21.15.155:0/4154927604' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T05:58:38.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:37 smithi016 ceph-mon[17194]: from='client.? 172.21.15.155:0/4154927604' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T05:58:38.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:37 smithi184 ceph-mon[22331]: from='client.? 172.21.15.155:0/4154927604' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T05:58:39.121 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:38 smithi155 ceph-mon[21742]: pgmap v36: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T05:58:39.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:38 smithi016 ceph-mon[17194]: pgmap v36: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T05:58:39.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:38 smithi184 ceph-mon[22331]: pgmap v36: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T05:58:41.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:40 smithi155 ceph-mon[21742]: pgmap v37: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T05:58:41.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:40 smithi016 ceph-mon[17194]: pgmap v37: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T05:58:41.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:40 smithi184 ceph-mon[22331]: pgmap v37: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T05:58:43.111 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:42 smithi155 ceph-mon[21742]: pgmap v38: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T05:58:43.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:42 smithi016 ceph-mon[17194]: pgmap v38: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T05:58:43.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:42 smithi184 ceph-mon[22331]: pgmap v38: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T05:58:45.004 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:44 smithi155 ceph-mon[21742]: pgmap v39: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T05:58:45.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:44 smithi016 ceph-mon[17194]: pgmap v39: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T05:58:45.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:44 smithi184 ceph-mon[22331]: pgmap v39: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T05:58:46.121 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:45 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-08-31T05:58:46.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:45 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:58:46.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:45 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-08-31T05:58:46.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:45 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:58:46.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:45 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-08-31T05:58:46.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:45 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:58:46.960 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:46 smithi155 ceph-mon[21742]: Deploying daemon osd.1 on smithi155 2024-08-31T05:58:46.960 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:46 smithi155 ceph-mon[21742]: pgmap v40: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T05:58:47.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:46 smithi016 ceph-mon[17194]: Deploying daemon osd.1 on smithi155 2024-08-31T05:58:47.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:46 smithi016 ceph-mon[17194]: pgmap v40: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T05:58:47.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:46 smithi184 ceph-mon[22331]: Deploying daemon osd.1 on smithi155 2024-08-31T05:58:47.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:46 smithi184 ceph-mon[22331]: pgmap v40: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T05:58:49.082 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:48 smithi155 ceph-mon[21742]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T05:58:49.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:48 smithi016 ceph-mon[17194]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T05:58:49.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:48 smithi184 ceph-mon[22331]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T05:58:51.072 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:50 smithi155 ceph-mon[21742]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T05:58:51.072 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:50 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:58:51.072 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:50 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:51.072 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:50 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:51.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:50 smithi016 ceph-mon[17194]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T05:58:51.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:50 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:58:51.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:50 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:51.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:50 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:51.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:50 smithi184 ceph-mon[22331]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T05:58:51.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:50 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:58:51.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:50 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:51.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:50 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:53.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:52 smithi155 ceph-mon[21742]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T05:58:53.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:52 smithi016 ceph-mon[17194]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T05:58:53.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:52 smithi184 ceph-mon[22331]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T05:58:53.933 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:53.931+0000 7f2c6e7fc640 1 -- 172.21.15.155:0/64789449 <== mgr.14220 v2:172.21.15.16:6800/4279751337 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2c780635a0 con 0x7f2c640b4af0 2024-08-31T05:58:53.933 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 1 on host 'smithi155' 2024-08-31T05:58:53.936 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:53.934+0000 7f2c7f941640 1 -- 172.21.15.155:0/64789449 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f2c640b4af0 msgr2=0x7f2c640b7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:58:53.936 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:53.934+0000 7f2c7f941640 1 --2- 172.21.15.155:0/64789449 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f2c640b4af0 0x7f2c640b7000 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2c74009230 tx=0x7f2c74031400 comp rx=0 tx=0).stop 2024-08-31T05:58:53.937 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:53.934+0000 7f2c7f941640 1 -- 172.21.15.155:0/64789449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c78105010 msgr2=0x7f2c78102a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:58:53.937 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:53.934+0000 7f2c7f941640 1 --2- 172.21.15.155:0/64789449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c78105010 0x7f2c78102a30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2c6800d950 tx=0x7f2c6800de10 comp rx=0 tx=0).stop 2024-08-31T05:58:53.937 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:53.934+0000 7f2c7f941640 1 -- 172.21.15.155:0/64789449 shutdown_connections 2024-08-31T05:58:53.937 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:53.934+0000 7f2c7f941640 1 --2- 172.21.15.155:0/64789449 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f2c640b4af0 0x7f2c640b7000 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:58:53.937 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:53.934+0000 7f2c7f941640 1 --2- 172.21.15.155:0/64789449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c7810db90 0x7f2c78102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:58:53.937 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:53.934+0000 7f2c7f941640 1 --2- 172.21.15.155:0/64789449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c78105010 0x7f2c78102a30 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:58:53.937 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:53.934+0000 7f2c7f941640 1 --2- 172.21.15.155:0/64789449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c781045e0 0x7f2c781024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:58:53.937 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:53.934+0000 7f2c7f941640 1 -- 172.21.15.155:0/64789449 >> 172.21.15.155:0/64789449 conn(0x7f2c780794d0 msgr2=0x7f2c780ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:58:53.937 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:53.934+0000 7f2c7f941640 1 -- 172.21.15.155:0/64789449 shutdown_connections 2024-08-31T05:58:53.937 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:58:53.934+0000 7f2c7f941640 1 -- 172.21.15.155:0/64789449 wait complete. 2024-08-31T05:58:54.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:53 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:54.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:53 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:54.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:53 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:58:54.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:53 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T05:58:54.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:53 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:54.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:53 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:58:54.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:53 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:54.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:53 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:54.254 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:53 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:54.254 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:53 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:54.254 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:53 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:58:54.254 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:53 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T05:58:54.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:53 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:54.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:53 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:58:54.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:53 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:54.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:53 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:54.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:53 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:54.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:53 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:54.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:53 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:58:54.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:53 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T05:58:54.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:53 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:54.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:53 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:58:54.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:53 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:54.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:53 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:54.563 DEBUG:teuthology.orchestra.run.smithi155:osd.1> sudo journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@osd.1.service 2024-08-31T05:58:54.566 INFO:tasks.cephadm:Deploying osd.2 on smithi184 with /dev/vg_nvme/lv_4... 2024-08-31T05:58:54.566 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-08-31T05:58:54.818 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.c/config 2024-08-31T05:58:55.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:54 smithi184 ceph-mon[22331]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T05:58:55.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:54 smithi155 ceph-mon[21742]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T05:58:55.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:54 smithi016 ceph-mon[17194]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T05:58:56.372 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 31 05:58:56 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-1[27505]: 2024-08-31T05:58:56.121+0000 7f372c213640 -1 osd.1 0 log_to_monitors true 2024-08-31T05:58:57.184 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-08-31T05:58:57.211 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:56 smithi184 ceph-mon[22331]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T05:58:57.211 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:56 smithi184 ceph-mon[22331]: from='osd.1 [v2:172.21.15.155:6800/4241618440,v1:172.21.15.155:6801/4241618440]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-08-31T05:58:57.211 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:56 smithi184 ceph-mon[22331]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-08-31T05:58:57.212 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph orch daemon add osd smithi184:vg_nvme/lv_4 2024-08-31T05:58:57.215 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:56 smithi016 ceph-mon[17194]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T05:58:57.215 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:56 smithi016 ceph-mon[17194]: from='osd.1 [v2:172.21.15.155:6800/4241618440,v1:172.21.15.155:6801/4241618440]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-08-31T05:58:57.215 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:56 smithi016 ceph-mon[17194]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-08-31T05:58:57.368 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:56 smithi155 ceph-mon[21742]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T05:58:57.368 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:56 smithi155 ceph-mon[21742]: from='osd.1 [v2:172.21.15.155:6800/4241618440,v1:172.21.15.155:6801/4241618440]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-08-31T05:58:57.368 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:56 smithi155 ceph-mon[21742]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-08-31T05:58:57.469 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.c/config 2024-08-31T05:58:57.923 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:58:57.919+0000 7f1b82d53640 1 -- 172.21.15.184:0/2029959366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b7c105860 msgr2=0x7f1b7c10f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:58:57.923 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:58:57.919+0000 7f1b82d53640 1 --2- 172.21.15.184:0/2029959366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b7c105860 0x7f1b7c10f2d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1b74009ca0 tx=0x7f1b7402a8c0 comp rx=0 tx=0).stop 2024-08-31T05:58:57.923 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:58:57.919+0000 7f1b82d53640 1 -- 172.21.15.184:0/2029959366 shutdown_connections 2024-08-31T05:58:57.923 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:58:57.919+0000 7f1b82d53640 1 --2- 172.21.15.184:0/2029959366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b7c105860 0x7f1b7c10f2d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:58:57.923 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:58:57.919+0000 7f1b82d53640 1 --2- 172.21.15.184:0/2029959366 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b7c104f10 0x7f1b7c105320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:58:57.924 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:58:57.919+0000 7f1b82d53640 1 --2- 172.21.15.184:0/2029959366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b7c102f00 0x7f1b7c103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:58:57.924 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:58:57.919+0000 7f1b82d53640 1 -- 172.21.15.184:0/2029959366 >> 172.21.15.184:0/2029959366 conn(0x7f1b7c077a00 msgr2=0x7f1b7c077e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:58:57.924 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:58:57.919+0000 7f1b82d53640 1 -- 172.21.15.184:0/2029959366 shutdown_connections 2024-08-31T05:58:57.924 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:58:57.919+0000 7f1b82d53640 1 -- 172.21.15.184:0/2029959366 wait complete. 2024-08-31T05:58:57.924 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:58:57.920+0000 7f1b82d53640 1 Processor -- start 2024-08-31T05:58:57.924 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:58:57.920+0000 7f1b82d53640 1 -- start start 2024-08-31T05:58:57.924 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:58:57.920+0000 7f1b82d53640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b7c102f00 0x7f1b7c19b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:58:57.924 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:58:57.920+0000 7f1b82d53640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b7c104f10 0x7f1b7c19b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:58:57.924 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:58:57.920+0000 7f1b82d53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b7c105860 0x7f1b7c19bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:58:57.924 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:58:57.920+0000 7f1b82d53640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c1127c0 con 0x7f1b7c105860 2024-08-31T05:58:57.924 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:58:57.920+0000 7f1b82d53640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c112640 con 0x7f1b7c104f10 2024-08-31T05:58:57.924 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:58:57.920+0000 7f1b82d53640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c112940 con 0x7f1b7c102f00 2024-08-31T05:58:57.925 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:58:57.920+0000 7f1b80aef640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b7c102f00 0x7f1b7c19b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:58:57.925 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:58:57.920+0000 7f1b80aef640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b7c102f00 0x7f1b7c19b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:36720/0 (socket says 172.21.15.184:36720) 2024-08-31T05:58:57.925 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:58:57.920+0000 7f1b80aef640 1 -- 172.21.15.184:0/1899399469 learned_addr learned my addr 172.21.15.184:0/1899399469 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-08-31T05:58:57.925 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:58:57.920+0000 7f1b7bfff640 1 --2- 172.21.15.184:0/1899399469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b7c104f10 0x7f1b7c19b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:58:57.925 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:58:57.920+0000 7f1b812f0640 1 --2- 172.21.15.184:0/1899399469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b7c105860 0x7f1b7c19bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:58:57.925 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:58:57.921+0000 7f1b80aef640 1 -- 172.21.15.184:0/1899399469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b7c104f10 msgr2=0x7f1b7c19b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:58:57.925 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:58:57.921+0000 7f1b80aef640 1 --2- 172.21.15.184:0/1899399469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b7c104f10 0x7f1b7c19b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:58:57.925 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:58:57.921+0000 7f1b80aef640 1 -- 172.21.15.184:0/1899399469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b7c105860 msgr2=0x7f1b7c19bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:58:57.925 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:58:57.921+0000 7f1b80aef640 1 --2- 172.21.15.184:0/1899399469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b7c105860 0x7f1b7c19bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:58:57.925 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:58:57.921+0000 7f1b80aef640 1 -- 172.21.15.184:0/1899399469 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b7c1a6380 con 0x7f1b7c102f00 2024-08-31T05:58:57.925 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:58:57.921+0000 7f1b812f0640 1 --2- 172.21.15.184:0/1899399469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b7c105860 0x7f1b7c19bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T05:58:57.925 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:58:57.921+0000 7f1b80aef640 1 --2- 172.21.15.184:0/1899399469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b7c102f00 0x7f1b7c19b0f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1b7000d950 tx=0x7f1b7000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:58:57.926 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:58:57.921+0000 7f1b79ffb640 1 -- 172.21.15.184:0/1899399469 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b70017030 con 0x7f1b7c102f00 2024-08-31T05:58:57.926 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:58:57.921+0000 7f1b79ffb640 1 -- 172.21.15.184:0/1899399469 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b70004020 con 0x7f1b7c102f00 2024-08-31T05:58:57.926 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:58:57.921+0000 7f1b79ffb640 1 -- 172.21.15.184:0/1899399469 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b700047a0 con 0x7f1b7c102f00 2024-08-31T05:58:57.926 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:58:57.921+0000 7f1b82d53640 1 -- 172.21.15.184:0/1899399469 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b7c1a6670 con 0x7f1b7c102f00 2024-08-31T05:58:57.926 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:58:57.921+0000 7f1b82d53640 1 -- 172.21.15.184:0/1899399469 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b7c1a6b60 con 0x7f1b7c102f00 2024-08-31T05:58:57.926 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:58:57.922+0000 7f1b82d53640 1 -- 172.21.15.184:0/1899399469 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b7c103310 con 0x7f1b7c102f00 2024-08-31T05:58:57.927 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:58:57.924+0000 7f1b79ffb640 1 -- 172.21.15.184:0/1899399469 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 19) ==== 152538+0+0 (secure 0 0 0) 0x7f1b70009a30 con 0x7f1b7c102f00 2024-08-31T05:58:57.927 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:58:57.925+0000 7f1b79ffb640 1 --2- 172.21.15.184:0/1899399469 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f1b6c0b4ba0 0x7f1b6c0b70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:58:57.927 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:58:57.925+0000 7f1b79ffb640 1 -- 172.21.15.184:0/1899399469 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2118+0+0 (secure 0 0 0) 0x7f1b700e3850 con 0x7f1b7c102f00 2024-08-31T05:58:57.927 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:58:57.925+0000 7f1b7bfff640 1 --2- 172.21.15.184:0/1899399469 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f1b6c0b4ba0 0x7f1b6c0b70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:58:57.928 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:58:57.926+0000 7f1b7bfff640 1 --2- 172.21.15.184:0/1899399469 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f1b6c0b4ba0 0x7f1b6c0b70b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c107e00 tx=0x7f1b6800b390 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:58:57.934 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:58:57.932+0000 7f1b79ffb640 1 -- 172.21.15.184:0/1899399469 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f1b700ac310 con 0x7f1b7c102f00 2024-08-31T05:58:58.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:57 smithi016 ceph-mon[17194]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-08-31T05:58:58.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:57 smithi016 ceph-mon[17194]: osdmap e12: 2 total, 1 up, 2 in 2024-08-31T05:58:58.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:57 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-31T05:58:58.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:57 smithi016 ceph-mon[17194]: from='osd.1 [v2:172.21.15.155:6800/4241618440,v1:172.21.15.155:6801/4241618440]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi155", "root=default"]} : dispatch 2024-08-31T05:58:58.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:57 smithi016 ceph-mon[17194]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi155", "root=default"]} : dispatch 2024-08-31T05:58:58.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:57 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:58.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:57 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:58.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:57 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T05:58:58.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:57 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T05:58:58.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:57 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:58.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:57 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:58:58.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:57 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T05:58:58.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:57 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:58.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:57 smithi016 ceph-mon[17194]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-08-31T05:58:58.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:57 smithi016 ceph-mon[17194]: osdmap e13: 2 total, 1 up, 2 in 2024-08-31T05:58:58.127 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:58:58.125+0000 7f1b82d53640 1 -- 172.21.15.184:0/1899399469 --> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f1b7c0635a0 con 0x7f1b6c0b4ba0 2024-08-31T05:58:58.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:57 smithi184 ceph-mon[22331]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-08-31T05:58:58.127 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:57 smithi184 ceph-mon[22331]: osdmap e12: 2 total, 1 up, 2 in 2024-08-31T05:58:58.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:57 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-31T05:58:58.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:57 smithi184 ceph-mon[22331]: from='osd.1 [v2:172.21.15.155:6800/4241618440,v1:172.21.15.155:6801/4241618440]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi155", "root=default"]} : dispatch 2024-08-31T05:58:58.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:57 smithi184 ceph-mon[22331]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi155", "root=default"]} : dispatch 2024-08-31T05:58:58.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:57 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:58.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:57 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:58.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:57 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T05:58:58.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:57 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T05:58:58.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:57 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:58.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:57 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:58:58.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:57 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T05:58:58.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:57 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:58.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:57 smithi184 ceph-mon[22331]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-08-31T05:58:58.128 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:57 smithi184 ceph-mon[22331]: osdmap e13: 2 total, 1 up, 2 in 2024-08-31T05:58:58.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:57 smithi155 ceph-mon[21742]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-08-31T05:58:58.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:57 smithi155 ceph-mon[21742]: osdmap e12: 2 total, 1 up, 2 in 2024-08-31T05:58:58.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:57 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-31T05:58:58.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:57 smithi155 ceph-mon[21742]: from='osd.1 [v2:172.21.15.155:6800/4241618440,v1:172.21.15.155:6801/4241618440]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi155", "root=default"]} : dispatch 2024-08-31T05:58:58.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:57 smithi155 ceph-mon[21742]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi155", "root=default"]} : dispatch 2024-08-31T05:58:58.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:57 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:58.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:57 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:58.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:57 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T05:58:58.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:57 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T05:58:58.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:57 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:58.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:57 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:58:58.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:57 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T05:58:58.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:57 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:58:58.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:57 smithi155 ceph-mon[21742]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-08-31T05:58:58.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:57 smithi155 ceph-mon[21742]: osdmap e13: 2 total, 1 up, 2 in 2024-08-31T05:58:58.872 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 31 05:58:58 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-1[27505]: 2024-08-31T05:58:58.372+0000 7f37289aa640 -1 osd.1 0 waiting for initial osdmap 2024-08-31T05:58:58.872 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 31 05:58:58 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-1[27505]: 2024-08-31T05:58:58.376+0000 7f371ff8b640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-31T05:58:59.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:58 smithi184 ceph-mon[22331]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T05:58:59.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:58 smithi184 ceph-mon[22331]: Detected new or changed devices on smithi155 2024-08-31T05:58:59.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:58 smithi184 ceph-mon[22331]: Adjusting osd_memory_target on smithi155 to 17088M 2024-08-31T05:58:59.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:58 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-31T05:58:59.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:58 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-31T05:58:59.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:58 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T05:58:59.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:58 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T05:58:59.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:58 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:58:59.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:58 smithi184 ceph-mon[22331]: from='osd.1 ' entity='osd.1' 2024-08-31T05:58:59.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:58 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-31T05:58:59.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:58 smithi155 ceph-mon[21742]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T05:58:59.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:58 smithi155 ceph-mon[21742]: Detected new or changed devices on smithi155 2024-08-31T05:58:59.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:58 smithi155 ceph-mon[21742]: Adjusting osd_memory_target on smithi155 to 17088M 2024-08-31T05:58:59.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:58 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-31T05:58:59.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:58 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-31T05:58:59.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:58 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T05:58:59.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:58 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T05:58:59.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:58 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:58:59.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:58 smithi155 ceph-mon[21742]: from='osd.1 ' entity='osd.1' 2024-08-31T05:58:59.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:58 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-31T05:58:59.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:58 smithi016 ceph-mon[17194]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T05:58:59.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:58 smithi016 ceph-mon[17194]: Detected new or changed devices on smithi155 2024-08-31T05:58:59.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:58 smithi016 ceph-mon[17194]: Adjusting osd_memory_target on smithi155 to 17088M 2024-08-31T05:58:59.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:58 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-31T05:58:59.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:58 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-31T05:58:59.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:58 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T05:58:59.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:58 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T05:58:59.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:58 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:58:59.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:58 smithi016 ceph-mon[17194]: from='osd.1 ' entity='osd.1' 2024-08-31T05:58:59.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:58 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-31T05:59:00.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:59 smithi155 ceph-mon[21742]: purged_snaps scrub starts 2024-08-31T05:59:00.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:59 smithi155 ceph-mon[21742]: purged_snaps scrub ok 2024-08-31T05:59:00.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:59 smithi155 ceph-mon[21742]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T05:59:00.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:59 smithi155 ceph-mon[21742]: osd.1 [v2:172.21.15.155:6800/4241618440,v1:172.21.15.155:6801/4241618440] boot 2024-08-31T05:59:00.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:59 smithi155 ceph-mon[21742]: osdmap e14: 2 total, 2 up, 2 in 2024-08-31T05:59:00.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:58:59 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-31T05:59:00.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:59 smithi016 ceph-mon[17194]: purged_snaps scrub starts 2024-08-31T05:59:00.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:59 smithi016 ceph-mon[17194]: purged_snaps scrub ok 2024-08-31T05:59:00.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:59 smithi016 ceph-mon[17194]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T05:59:00.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:59 smithi016 ceph-mon[17194]: osd.1 [v2:172.21.15.155:6800/4241618440,v1:172.21.15.155:6801/4241618440] boot 2024-08-31T05:59:00.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:59 smithi016 ceph-mon[17194]: osdmap e14: 2 total, 2 up, 2 in 2024-08-31T05:59:00.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:58:59 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-31T05:59:00.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:59 smithi184 ceph-mon[22331]: purged_snaps scrub starts 2024-08-31T05:59:00.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:59 smithi184 ceph-mon[22331]: purged_snaps scrub ok 2024-08-31T05:59:00.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:59 smithi184 ceph-mon[22331]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T05:59:00.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:59 smithi184 ceph-mon[22331]: osd.1 [v2:172.21.15.155:6800/4241618440,v1:172.21.15.155:6801/4241618440] boot 2024-08-31T05:59:00.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:59 smithi184 ceph-mon[22331]: osdmap e14: 2 total, 2 up, 2 in 2024-08-31T05:59:00.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:58:59 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-31T05:59:01.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:00 smithi184 ceph-mon[22331]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T05:59:01.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:00 smithi184 ceph-mon[22331]: from='client.? 172.21.15.184:0/1875859695' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c69c8c19-32f8-400a-86d4-0f4ce240a7de"} : dispatch 2024-08-31T05:59:01.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:00 smithi184 ceph-mon[22331]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c69c8c19-32f8-400a-86d4-0f4ce240a7de"} : dispatch 2024-08-31T05:59:01.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:00 smithi184 ceph-mon[22331]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c69c8c19-32f8-400a-86d4-0f4ce240a7de"}]': finished 2024-08-31T05:59:01.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:00 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-31T05:59:01.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:00 smithi184 ceph-mon[22331]: osdmap e15: 3 total, 2 up, 3 in 2024-08-31T05:59:01.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:00 smithi184 ceph-mon[22331]: from='client.? 172.21.15.184:0/794860000' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T05:59:01.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:00 smithi155 ceph-mon[21742]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T05:59:01.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:00 smithi155 ceph-mon[21742]: from='client.? 172.21.15.184:0/1875859695' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c69c8c19-32f8-400a-86d4-0f4ce240a7de"} : dispatch 2024-08-31T05:59:01.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:00 smithi155 ceph-mon[21742]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c69c8c19-32f8-400a-86d4-0f4ce240a7de"} : dispatch 2024-08-31T05:59:01.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:00 smithi155 ceph-mon[21742]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c69c8c19-32f8-400a-86d4-0f4ce240a7de"}]': finished 2024-08-31T05:59:01.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:00 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-31T05:59:01.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:00 smithi155 ceph-mon[21742]: osdmap e15: 3 total, 2 up, 3 in 2024-08-31T05:59:01.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:00 smithi155 ceph-mon[21742]: from='client.? 172.21.15.184:0/794860000' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T05:59:01.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:00 smithi016 ceph-mon[17194]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-31T05:59:01.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:00 smithi016 ceph-mon[17194]: from='client.? 172.21.15.184:0/1875859695' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c69c8c19-32f8-400a-86d4-0f4ce240a7de"} : dispatch 2024-08-31T05:59:01.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:00 smithi016 ceph-mon[17194]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c69c8c19-32f8-400a-86d4-0f4ce240a7de"} : dispatch 2024-08-31T05:59:01.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:00 smithi016 ceph-mon[17194]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c69c8c19-32f8-400a-86d4-0f4ce240a7de"}]': finished 2024-08-31T05:59:01.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:00 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-31T05:59:01.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:00 smithi016 ceph-mon[17194]: osdmap e15: 3 total, 2 up, 3 in 2024-08-31T05:59:01.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:00 smithi016 ceph-mon[17194]: from='client.? 172.21.15.184:0/794860000' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T05:59:03.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:02 smithi155 ceph-mon[21742]: pgmap v52: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-31T05:59:03.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:02 smithi016 ceph-mon[17194]: pgmap v52: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-31T05:59:03.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:02 smithi184 ceph-mon[22331]: pgmap v52: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-31T05:59:05.238 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:04 smithi184 ceph-mon[22331]: pgmap v53: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-31T05:59:05.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:04 smithi155 ceph-mon[21742]: pgmap v53: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-31T05:59:05.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:04 smithi016 ceph-mon[17194]: pgmap v53: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-31T05:59:07.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:06 smithi184 ceph-mon[22331]: pgmap v54: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-31T05:59:07.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:06 smithi155 ceph-mon[21742]: pgmap v54: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-31T05:59:07.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:06 smithi016 ceph-mon[17194]: pgmap v54: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-31T05:59:09.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:08 smithi184 ceph-mon[22331]: pgmap v55: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-31T05:59:09.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:08 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-08-31T05:59:09.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:08 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:59:09.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:08 smithi155 ceph-mon[21742]: pgmap v55: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-31T05:59:09.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:08 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-08-31T05:59:09.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:08 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:59:09.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:08 smithi016 ceph-mon[17194]: pgmap v55: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-31T05:59:09.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:08 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-08-31T05:59:09.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:08 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:59:10.346 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:09 smithi184 ceph-mon[22331]: Deploying daemon osd.2 on smithi184 2024-08-31T05:59:10.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:09 smithi155 ceph-mon[21742]: Deploying daemon osd.2 on smithi184 2024-08-31T05:59:10.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:09 smithi016 ceph-mon[17194]: Deploying daemon osd.2 on smithi184 2024-08-31T05:59:11.331 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:10 smithi184 ceph-mon[22331]: pgmap v56: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-31T05:59:11.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:10 smithi155 ceph-mon[21742]: pgmap v56: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-31T05:59:11.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:10 smithi016 ceph-mon[17194]: pgmap v56: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-31T05:59:13.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:13 smithi155 ceph-mon[21742]: pgmap v57: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-31T05:59:13.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:13 smithi016 ceph-mon[17194]: pgmap v57: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-31T05:59:13.436 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:13 smithi184 ceph-mon[22331]: pgmap v57: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-31T05:59:14.307 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:14 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:59:14.307 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:14 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:14.307 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:14 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:14.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:14 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:59:14.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:14 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:14.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:14 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:14.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:14 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:59:14.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:14 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:14.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:14 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:15.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:15 smithi155 ceph-mon[21742]: pgmap v58: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-31T05:59:15.394 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:15 smithi184 ceph-mon[22331]: pgmap v58: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-31T05:59:15.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:15 smithi016 ceph-mon[17194]: pgmap v58: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-31T05:59:17.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:17 smithi155 ceph-mon[21742]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-31T05:59:17.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:17 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:17.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:17 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:17.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:17 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:59:17.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:17 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T05:59:17.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:17 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:17.394 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:17 smithi184 ceph-mon[22331]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-31T05:59:17.394 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:17 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:17.394 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:17 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:17.394 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:17 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:59:17.394 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:17 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T05:59:17.394 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:17 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:17.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:17 smithi016 ceph-mon[17194]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-31T05:59:17.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:17 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:17.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:17 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:17.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:17 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:59:17.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:17 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T05:59:17.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:17 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:18.029 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:59:18.026+0000 7f1b79ffb640 1 -- 172.21.15.184:0/1899399469 <== mgr.14220 v2:172.21.15.16:6800/4279751337 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1b7c0635a0 con 0x7f1b6c0b4ba0 2024-08-31T05:59:18.029 INFO:teuthology.orchestra.run.smithi184.stdout:Created osd(s) 2 on host 'smithi184' 2024-08-31T05:59:18.031 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:59:18.029+0000 7f1b82d53640 1 -- 172.21.15.184:0/1899399469 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f1b6c0b4ba0 msgr2=0x7f1b6c0b70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:59:18.031 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:59:18.029+0000 7f1b82d53640 1 --2- 172.21.15.184:0/1899399469 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f1b6c0b4ba0 0x7f1b6c0b70b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c107e00 tx=0x7f1b6800b390 comp rx=0 tx=0).stop 2024-08-31T05:59:18.031 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:59:18.029+0000 7f1b82d53640 1 -- 172.21.15.184:0/1899399469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b7c102f00 msgr2=0x7f1b7c19b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:59:18.031 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:59:18.029+0000 7f1b82d53640 1 --2- 172.21.15.184:0/1899399469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b7c102f00 0x7f1b7c19b0f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1b7000d950 tx=0x7f1b7000de10 comp rx=0 tx=0).stop 2024-08-31T05:59:18.032 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:59:18.029+0000 7f1b82d53640 1 -- 172.21.15.184:0/1899399469 shutdown_connections 2024-08-31T05:59:18.032 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:59:18.029+0000 7f1b82d53640 1 --2- 172.21.15.184:0/1899399469 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f1b6c0b4ba0 0x7f1b6c0b70b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:59:18.032 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:59:18.029+0000 7f1b82d53640 1 --2- 172.21.15.184:0/1899399469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b7c105860 0x7f1b7c19bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:59:18.032 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:59:18.029+0000 7f1b82d53640 1 --2- 172.21.15.184:0/1899399469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b7c104f10 0x7f1b7c19b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:59:18.032 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:59:18.029+0000 7f1b82d53640 1 --2- 172.21.15.184:0/1899399469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b7c102f00 0x7f1b7c19b0f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:59:18.032 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:59:18.029+0000 7f1b82d53640 1 -- 172.21.15.184:0/1899399469 >> 172.21.15.184:0/1899399469 conn(0x7f1b7c077a00 msgr2=0x7f1b7c1061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:59:18.032 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:59:18.029+0000 7f1b82d53640 1 -- 172.21.15.184:0/1899399469 shutdown_connections 2024-08-31T05:59:18.032 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T05:59:18.029+0000 7f1b82d53640 1 -- 172.21.15.184:0/1899399469 wait complete. 2024-08-31T05:59:18.330 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:18 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:59:18.331 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:18 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:18.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:18 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:59:18.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:18 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:18.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:18 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:59:18.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:18 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:18.642 DEBUG:teuthology.orchestra.run.smithi184:osd.2> sudo journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@osd.2.service 2024-08-31T05:59:18.647 INFO:tasks.cephadm:Deploying osd.3 on smithi016 with /dev/vg_nvme/lv_3... 2024-08-31T05:59:18.647 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-08-31T05:59:19.031 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T05:59:19.184 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:19 smithi184 ceph-mon[22331]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-31T05:59:19.184 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:19 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:19.319 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:19 smithi016 ceph-mon[17194]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-31T05:59:19.319 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:19 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:19.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:19 smithi155 ceph-mon[21742]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-31T05:59:19.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:19 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:20.024 INFO:journalctl@ceph.osd.2.smithi184.stdout:Aug 31 05:59:19 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-2[28490]: 2024-08-31T05:59:19.815+0000 7fd7d46de640 -1 osd.2 0 log_to_monitors true 2024-08-31T05:59:20.241 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:20 smithi016 ceph-mon[17194]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-31T05:59:20.241 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:20 smithi016 ceph-mon[17194]: from='osd.2 [v2:172.21.15.184:6800/2193296422,v1:172.21.15.184:6801/2193296422]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-08-31T05:59:20.241 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:20 smithi016 ceph-mon[17194]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-08-31T05:59:20.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:20 smithi155 ceph-mon[21742]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-31T05:59:20.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:20 smithi155 ceph-mon[21742]: from='osd.2 [v2:172.21.15.184:6800/2193296422,v1:172.21.15.184:6801/2193296422]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-08-31T05:59:20.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:20 smithi155 ceph-mon[21742]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-08-31T05:59:20.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:20 smithi184 ceph-mon[22331]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-31T05:59:20.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:20 smithi184 ceph-mon[22331]: from='osd.2 [v2:172.21.15.184:6800/2193296422,v1:172.21.15.184:6801/2193296422]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-08-31T05:59:20.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:20 smithi184 ceph-mon[22331]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-08-31T05:59:21.304 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:21 smithi016 ceph-mon[17194]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-08-31T05:59:21.305 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:21 smithi016 ceph-mon[17194]: osdmap e16: 3 total, 2 up, 3 in 2024-08-31T05:59:21.305 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:21 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-31T05:59:21.305 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:21 smithi016 ceph-mon[17194]: from='osd.2 [v2:172.21.15.184:6800/2193296422,v1:172.21.15.184:6801/2193296422]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]} : dispatch 2024-08-31T05:59:21.305 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:21 smithi016 ceph-mon[17194]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]} : dispatch 2024-08-31T05:59:21.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:21 smithi155 ceph-mon[21742]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-08-31T05:59:21.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:21 smithi155 ceph-mon[21742]: osdmap e16: 3 total, 2 up, 3 in 2024-08-31T05:59:21.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:21 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-31T05:59:21.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:21 smithi155 ceph-mon[21742]: from='osd.2 [v2:172.21.15.184:6800/2193296422,v1:172.21.15.184:6801/2193296422]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]} : dispatch 2024-08-31T05:59:21.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:21 smithi155 ceph-mon[21742]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]} : dispatch 2024-08-31T05:59:21.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:21 smithi184 ceph-mon[22331]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-08-31T05:59:21.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:21 smithi184 ceph-mon[22331]: osdmap e16: 3 total, 2 up, 3 in 2024-08-31T05:59:21.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:21 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-31T05:59:21.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:21 smithi184 ceph-mon[22331]: from='osd.2 [v2:172.21.15.184:6800/2193296422,v1:172.21.15.184:6801/2193296422]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]} : dispatch 2024-08-31T05:59:21.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:21 smithi184 ceph-mon[22331]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]} : dispatch 2024-08-31T05:59:21.722 INFO:journalctl@ceph.osd.2.smithi184.stdout:Aug 31 05:59:21 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-2[28490]: 2024-08-31T05:59:21.465+0000 7fd7d0660640 -1 osd.2 0 waiting for initial osdmap 2024-08-31T05:59:21.722 INFO:journalctl@ceph.osd.2.smithi184.stdout:Aug 31 05:59:21 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-2[28490]: 2024-08-31T05:59:21.470+0000 7fd7c8456640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-31T05:59:22.012 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T05:59:22.048 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph orch daemon add osd smithi016:vg_nvme/lv_3 2024-08-31T05:59:22.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:22 smithi155 ceph-mon[21742]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-08-31T05:59:22.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:22 smithi155 ceph-mon[21742]: osdmap e17: 3 total, 2 up, 3 in 2024-08-31T05:59:22.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:22 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-31T05:59:22.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:22 smithi155 ceph-mon[21742]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-31T05:59:22.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:22 smithi155 ceph-mon[21742]: Detected new or changed devices on smithi184 2024-08-31T05:59:22.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:22 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:22.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:22 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:22.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:22 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T05:59:22.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:22 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T05:59:22.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:22 smithi155 ceph-mon[21742]: Adjusting osd_memory_target on smithi184 to 17088M 2024-08-31T05:59:22.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:22 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:22.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:22 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:59:22.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:22 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T05:59:22.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:22 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:22.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:22 smithi155 ceph-mon[21742]: from='osd.2 ' entity='osd.2' 2024-08-31T05:59:22.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:22 smithi016 ceph-mon[17194]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-08-31T05:59:22.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:22 smithi016 ceph-mon[17194]: osdmap e17: 3 total, 2 up, 3 in 2024-08-31T05:59:22.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:22 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-31T05:59:22.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:22 smithi016 ceph-mon[17194]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-31T05:59:22.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:22 smithi016 ceph-mon[17194]: Detected new or changed devices on smithi184 2024-08-31T05:59:22.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:22 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:22.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:22 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:22.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:22 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T05:59:22.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:22 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T05:59:22.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:22 smithi016 ceph-mon[17194]: Adjusting osd_memory_target on smithi184 to 17088M 2024-08-31T05:59:22.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:22 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:22.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:22 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:59:22.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:22 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T05:59:22.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:22 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:22.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:22 smithi016 ceph-mon[17194]: from='osd.2 ' entity='osd.2' 2024-08-31T05:59:22.428 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T05:59:22.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:22 smithi184 ceph-mon[22331]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-08-31T05:59:22.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:22 smithi184 ceph-mon[22331]: osdmap e17: 3 total, 2 up, 3 in 2024-08-31T05:59:22.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:22 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-31T05:59:22.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:22 smithi184 ceph-mon[22331]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-31T05:59:22.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:22 smithi184 ceph-mon[22331]: Detected new or changed devices on smithi184 2024-08-31T05:59:22.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:22 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:22.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:22 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:22.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:22 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T05:59:22.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:22 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T05:59:22.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:22 smithi184 ceph-mon[22331]: Adjusting osd_memory_target on smithi184 to 17088M 2024-08-31T05:59:22.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:22 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:22.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:22 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:59:22.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:22 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T05:59:22.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:22 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:22.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:22 smithi184 ceph-mon[22331]: from='osd.2 ' entity='osd.2' 2024-08-31T05:59:23.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:23.092+0000 7f238f200640 1 -- 172.21.15.16:0/3661150002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2390105950 msgr2=0x7f2390105d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:59:23.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:23.092+0000 7f238f200640 1 --2- 172.21.15.16:0/3661150002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2390105950 0x7f2390105d60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2380009900 tx=0x7f238002a410 comp rx=0 tx=0).stop 2024-08-31T05:59:23.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:23.092+0000 7f238f200640 1 -- 172.21.15.16:0/3661150002 shutdown_connections 2024-08-31T05:59:23.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:23.092+0000 7f238f200640 1 --2- 172.21.15.16:0/3661150002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f239010bbe0 0x7f2390112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:59:23.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:23.092+0000 7f238f200640 1 --2- 172.21.15.16:0/3661150002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f239010b290 0x7f239010b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:59:23.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:23.092+0000 7f238f200640 1 --2- 172.21.15.16:0/3661150002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2390105950 0x7f2390105d60 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:59:23.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:23.092+0000 7f238f200640 1 -- 172.21.15.16:0/3661150002 >> 172.21.15.16:0/3661150002 conn(0x7f2390101420 msgr2=0x7f2390103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:59:23.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:23.092+0000 7f238f200640 1 -- 172.21.15.16:0/3661150002 shutdown_connections 2024-08-31T05:59:23.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:23.092+0000 7f238f200640 1 -- 172.21.15.16:0/3661150002 wait complete. 2024-08-31T05:59:23.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:23.093+0000 7f238f200640 1 Processor -- start 2024-08-31T05:59:23.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:23.093+0000 7f238f200640 1 -- start start 2024-08-31T05:59:23.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:23.094+0000 7f238f200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2390105950 0x7f239019c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:59:23.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:23.094+0000 7f238f200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f239010b290 0x7f239019c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:59:23.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:23.094+0000 7f238f200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f239010bbe0 0x7f23901a1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:59:23.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:23.094+0000 7f238f200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f23901158a0 con 0x7f239010b290 2024-08-31T05:59:23.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:23.094+0000 7f238f200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2390115720 con 0x7f239010bbe0 2024-08-31T05:59:23.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:23.094+0000 7f238f200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2390115a20 con 0x7f2390105950 2024-08-31T05:59:23.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:23.094+0000 7f238d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f239010b290 0x7f239019c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:59:23.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:23.094+0000 7f238d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f239010b290 0x7f239019c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56788/0 (socket says 172.21.15.16:56788) 2024-08-31T05:59:23.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:23.094+0000 7f238d400640 1 -- 172.21.15.16:0/1189486911 learned_addr learned my addr 172.21.15.16:0/1189486911 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T05:59:23.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:23.094+0000 7f238de00640 1 --2- 172.21.15.16:0/1189486911 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2390105950 0x7f239019c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:59:23.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:23.094+0000 7f238e800640 1 --2- 172.21.15.16:0/1189486911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f239010bbe0 0x7f23901a1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:59:23.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:23.094+0000 7f238de00640 1 -- 172.21.15.16:0/1189486911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f239010bbe0 msgr2=0x7f23901a1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:59:23.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:23.094+0000 7f238de00640 1 --2- 172.21.15.16:0/1189486911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f239010bbe0 0x7f23901a1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:59:23.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:23.094+0000 7f238de00640 1 -- 172.21.15.16:0/1189486911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f239010b290 msgr2=0x7f239019c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:59:23.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:23.094+0000 7f238de00640 1 --2- 172.21.15.16:0/1189486911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f239010b290 0x7f239019c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:59:23.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:23.094+0000 7f238de00640 1 -- 172.21.15.16:0/1189486911 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23901a2280 con 0x7f2390105950 2024-08-31T05:59:23.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:23.094+0000 7f238e800640 1 --2- 172.21.15.16:0/1189486911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f239010bbe0 0x7f23901a1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T05:59:23.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:23.094+0000 7f238d400640 1 --2- 172.21.15.16:0/1189486911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f239010b290 0x7f239019c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T05:59:23.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:23.095+0000 7f238de00640 1 --2- 172.21.15.16:0/1189486911 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2390105950 0x7f239019c310 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f238002a930 tx=0x7f2380031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:59:23.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:23.095+0000 7f2386a00640 1 -- 172.21.15.16:0/1189486911 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238003f030 con 0x7f2390105950 2024-08-31T05:59:23.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:23.095+0000 7f238f200640 1 -- 172.21.15.16:0/1189486911 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23901a2510 con 0x7f2390105950 2024-08-31T05:59:23.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:23.095+0000 7f2386a00640 1 -- 172.21.15.16:0/1189486911 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2380004490 con 0x7f2390105950 2024-08-31T05:59:23.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:23.095+0000 7f2386a00640 1 -- 172.21.15.16:0/1189486911 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2380035070 con 0x7f2390105950 2024-08-31T05:59:23.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:23.095+0000 7f238f200640 1 -- 172.21.15.16:0/1189486911 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23901a8cd0 con 0x7f2390105950 2024-08-31T05:59:23.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:23.096+0000 7f238f200640 1 -- 172.21.15.16:0/1189486911 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2390105d60 con 0x7f2390105950 2024-08-31T05:59:23.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:23.099+0000 7f2386a00640 1 -- 172.21.15.16:0/1189486911 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 19) ==== 152538+0+0 (secure 0 0 0) 0x7f2380042030 con 0x7f2390105950 2024-08-31T05:59:23.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:23.100+0000 7f2386a00640 1 --2- 172.21.15.16:0/1189486911 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f23740b4af0 0x7f23740b7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:59:23.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:23.100+0000 7f2386a00640 1 -- 172.21.15.16:0/1189486911 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 3066+0+0 (secure 0 0 0) 0x7f23801041e0 con 0x7f2390105950 2024-08-31T05:59:23.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:23.100+0000 7f238d400640 1 --2- 172.21.15.16:0/1189486911 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f23740b4af0 0x7f23740b7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:59:23.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:23.101+0000 7f238d400640 1 --2- 172.21.15.16:0/1189486911 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f23740b4af0 0x7f23740b7000 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f239010ae40 tx=0x7f237c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:59:23.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:23.102+0000 7f2386a00640 1 -- 172.21.15.16:0/1189486911 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f23800cc7d0 con 0x7f2390105950 2024-08-31T05:59:23.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:23 smithi016 ceph-mon[17194]: purged_snaps scrub starts 2024-08-31T05:59:23.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:23 smithi016 ceph-mon[17194]: purged_snaps scrub ok 2024-08-31T05:59:23.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:23 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-31T05:59:23.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:23 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-31T05:59:23.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:23 smithi016 ceph-mon[17194]: osd.2 [v2:172.21.15.184:6800/2193296422,v1:172.21.15.184:6801/2193296422] boot 2024-08-31T05:59:23.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:23 smithi016 ceph-mon[17194]: osdmap e18: 3 total, 3 up, 3 in 2024-08-31T05:59:23.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:23.309+0000 7f238f200640 1 -- 172.21.15.16:0/1189486911 --> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f23900635a0 con 0x7f23740b4af0 2024-08-31T05:59:23.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:23 smithi155 ceph-mon[21742]: purged_snaps scrub starts 2024-08-31T05:59:23.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:23 smithi155 ceph-mon[21742]: purged_snaps scrub ok 2024-08-31T05:59:23.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:23 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-31T05:59:23.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:23 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-31T05:59:23.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:23 smithi155 ceph-mon[21742]: osd.2 [v2:172.21.15.184:6800/2193296422,v1:172.21.15.184:6801/2193296422] boot 2024-08-31T05:59:23.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:23 smithi155 ceph-mon[21742]: osdmap e18: 3 total, 3 up, 3 in 2024-08-31T05:59:23.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:23 smithi184 ceph-mon[22331]: purged_snaps scrub starts 2024-08-31T05:59:23.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:23 smithi184 ceph-mon[22331]: purged_snaps scrub ok 2024-08-31T05:59:23.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:23 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-31T05:59:23.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:23 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-31T05:59:23.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:23 smithi184 ceph-mon[22331]: osd.2 [v2:172.21.15.184:6800/2193296422,v1:172.21.15.184:6801/2193296422] boot 2024-08-31T05:59:23.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:23 smithi184 ceph-mon[22331]: osdmap e18: 3 total, 3 up, 3 in 2024-08-31T05:59:24.153 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:24 smithi016 ceph-mon[17194]: pgmap v66: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:24.153 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:24 smithi016 ceph-mon[17194]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T05:59:24.153 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:24 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T05:59:24.153 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:24 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T05:59:24.153 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:24 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:59:24.153 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:24 smithi016 ceph-mon[17194]: osdmap e19: 3 total, 3 up, 3 in 2024-08-31T05:59:24.154 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:24 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-08-31T05:59:24.154 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:24 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-08-31T05:59:24.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:24 smithi155 ceph-mon[21742]: pgmap v66: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:24.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:24 smithi155 ceph-mon[21742]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T05:59:24.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:24 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T05:59:24.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:24 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T05:59:24.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:24 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:59:24.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:24 smithi155 ceph-mon[21742]: osdmap e19: 3 total, 3 up, 3 in 2024-08-31T05:59:24.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:24 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-08-31T05:59:24.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:24 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-08-31T05:59:24.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:24 smithi184 ceph-mon[22331]: pgmap v66: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:24.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:24 smithi184 ceph-mon[22331]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T05:59:24.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:24 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T05:59:24.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:24 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T05:59:24.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:24 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:59:24.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:24 smithi184 ceph-mon[22331]: osdmap e19: 3 total, 3 up, 3 in 2024-08-31T05:59:24.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:24 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-08-31T05:59:24.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:24 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-08-31T05:59:25.783 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:25 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-08-31T05:59:25.783 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:25 smithi016 ceph-mon[17194]: osdmap e20: 3 total, 3 up, 3 in 2024-08-31T05:59:25.783 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:25 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-08-31T05:59:25.783 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:25 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-08-31T05:59:25.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:25 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-08-31T05:59:25.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:25 smithi155 ceph-mon[21742]: osdmap e20: 3 total, 3 up, 3 in 2024-08-31T05:59:25.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:25 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-08-31T05:59:25.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:25 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-08-31T05:59:25.965 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:25 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-08-31T05:59:25.965 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:25 smithi184 ceph-mon[22331]: osdmap e20: 3 total, 3 up, 3 in 2024-08-31T05:59:25.965 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:25 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-08-31T05:59:25.965 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:25 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-08-31T05:59:26.051 INFO:journalctl@ceph.osd.0.smithi016.stdout:Aug 31 05:59:25 smithi016 sudo[40039]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-31T05:59:26.052 INFO:journalctl@ceph.osd.0.smithi016.stdout:Aug 31 05:59:25 smithi016 sudo[40039]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-31T05:59:26.052 INFO:journalctl@ceph.osd.0.smithi016.stdout:Aug 31 05:59:25 smithi016 sudo[40039]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-31T05:59:26.052 INFO:journalctl@ceph.osd.0.smithi016.stdout:Aug 31 05:59:25 smithi016 sudo[40039]: pam_unix(sudo:session): session closed for user root 2024-08-31T05:59:26.052 INFO:journalctl@ceph.osd.0.smithi016.stdout:Aug 31 05:59:25 smithi016 sudo[40045]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-31T05:59:26.052 INFO:journalctl@ceph.osd.0.smithi016.stdout:Aug 31 05:59:25 smithi016 sudo[40045]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-31T05:59:26.052 INFO:journalctl@ceph.osd.0.smithi016.stdout:Aug 31 05:59:25 smithi016 sudo[40045]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-31T05:59:26.052 INFO:journalctl@ceph.osd.0.smithi016.stdout:Aug 31 05:59:25 smithi016 sudo[40045]: pam_unix(sudo:session): session closed for user root 2024-08-31T05:59:26.164 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 31 05:59:25 smithi155 sudo[31193]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-31T05:59:26.164 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 31 05:59:25 smithi155 sudo[31193]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-31T05:59:26.165 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 31 05:59:25 smithi155 sudo[31193]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-31T05:59:26.165 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 31 05:59:25 smithi155 sudo[31193]: pam_unix(sudo:session): session closed for user root 2024-08-31T05:59:26.165 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 31 05:59:25 smithi155 sudo[31196]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-31T05:59:26.165 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 31 05:59:25 smithi155 sudo[31196]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-31T05:59:26.165 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 31 05:59:25 smithi155 sudo[31196]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-31T05:59:26.165 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 31 05:59:25 smithi155 sudo[31196]: pam_unix(sudo:session): session closed for user root 2024-08-31T05:59:26.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:26 smithi184 sudo[32139]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-31T05:59:26.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:26 smithi184 sudo[32139]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-31T05:59:26.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:26 smithi184 sudo[32139]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-31T05:59:26.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:26 smithi184 sudo[32139]: pam_unix(sudo:session): session closed for user root 2024-08-31T05:59:26.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:26 smithi184 sudo[32142]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-31T05:59:26.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:26 smithi184 sudo[32142]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-31T05:59:26.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:26 smithi184 sudo[32142]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-31T05:59:26.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:26 smithi184 sudo[32142]: pam_unix(sudo:session): session closed for user root 2024-08-31T05:59:26.223 INFO:journalctl@ceph.osd.2.smithi184.stdout:Aug 31 05:59:25 smithi184 sudo[32133]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-31T05:59:26.223 INFO:journalctl@ceph.osd.2.smithi184.stdout:Aug 31 05:59:25 smithi184 sudo[32133]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-31T05:59:26.223 INFO:journalctl@ceph.osd.2.smithi184.stdout:Aug 31 05:59:25 smithi184 sudo[32133]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-31T05:59:26.223 INFO:journalctl@ceph.osd.2.smithi184.stdout:Aug 31 05:59:25 smithi184 sudo[32133]: pam_unix(sudo:session): session closed for user root 2024-08-31T05:59:26.224 INFO:journalctl@ceph.osd.2.smithi184.stdout:Aug 31 05:59:25 smithi184 sudo[32136]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-31T05:59:26.224 INFO:journalctl@ceph.osd.2.smithi184.stdout:Aug 31 05:59:25 smithi184 sudo[32136]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-31T05:59:26.224 INFO:journalctl@ceph.osd.2.smithi184.stdout:Aug 31 05:59:25 smithi184 sudo[32136]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-31T05:59:26.224 INFO:journalctl@ceph.osd.2.smithi184.stdout:Aug 31 05:59:25 smithi184 sudo[32136]: pam_unix(sudo:session): session closed for user root 2024-08-31T05:59:26.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:26 smithi016 sudo[40056]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-31T05:59:26.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:26 smithi016 sudo[40056]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-31T05:59:26.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:26 smithi016 sudo[40056]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-31T05:59:26.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:26 smithi016 sudo[40056]: pam_unix(sudo:session): session closed for user root 2024-08-31T05:59:26.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:26 smithi016 sudo[40059]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-31T05:59:26.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:26 smithi016 sudo[40059]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-31T05:59:26.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:26 smithi016 sudo[40059]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-31T05:59:26.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:26 smithi016 sudo[40059]: pam_unix(sudo:session): session closed for user root 2024-08-31T05:59:26.484 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:26 smithi155 sudo[31199]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-31T05:59:26.484 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:26 smithi155 sudo[31199]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-31T05:59:26.484 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:26 smithi155 sudo[31199]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-31T05:59:26.484 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:26 smithi155 sudo[31199]: pam_unix(sudo:session): session closed for user root 2024-08-31T05:59:26.484 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:26 smithi155 sudo[31202]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-31T05:59:26.484 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:26 smithi155 sudo[31202]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-31T05:59:26.484 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:26 smithi155 sudo[31202]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-31T05:59:26.485 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:26 smithi155 sudo[31202]: pam_unix(sudo:session): session closed for user root 2024-08-31T05:59:26.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:26 smithi155 ceph-mon[21742]: pgmap v69: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:26.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:26 smithi155 ceph-mon[21742]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-31T05:59:26.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:26 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-08-31T05:59:26.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:26 smithi155 ceph-mon[21742]: osdmap e21: 3 total, 3 up, 3 in 2024-08-31T05:59:26.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:26 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:26.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:26 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:26.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:26 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:26.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:26 smithi155 ceph-mon[21742]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-31T05:59:26.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:26 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-31T05:59:26.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:26 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T05:59:26.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:26 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T05:59:26.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:26 smithi155 ceph-mon[21742]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-31T05:59:26.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:26 smithi155 ceph-mon[21742]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-31T05:59:26.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:26 smithi155 ceph-mon[21742]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-31T05:59:26.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:26 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-31T05:59:26.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:26 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T05:59:26.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:26 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T05:59:26.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:26 smithi155 ceph-mon[21742]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-31T05:59:26.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:26 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-31T05:59:26.874 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:26 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T05:59:26.874 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:26 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T05:59:26.874 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:26 smithi155 ceph-mon[21742]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-31T05:59:26.876 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:26 smithi016 ceph-mon[17194]: pgmap v69: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:26.876 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:26 smithi016 ceph-mon[17194]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-31T05:59:26.876 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:26 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-08-31T05:59:26.876 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:26 smithi016 ceph-mon[17194]: osdmap e21: 3 total, 3 up, 3 in 2024-08-31T05:59:26.877 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:26 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:26.877 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:26 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:26.877 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:26 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:26.877 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:26 smithi016 ceph-mon[17194]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-31T05:59:26.877 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:26 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-31T05:59:26.877 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:26 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T05:59:26.877 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:26 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T05:59:26.877 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:26 smithi016 ceph-mon[17194]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-31T05:59:26.877 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:26 smithi016 ceph-mon[17194]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-31T05:59:26.877 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:26 smithi016 ceph-mon[17194]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-31T05:59:26.878 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:26 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-31T05:59:26.878 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:26 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T05:59:26.878 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:26 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T05:59:26.878 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:26 smithi016 ceph-mon[17194]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-31T05:59:26.878 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:26 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-31T05:59:26.878 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:26 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T05:59:26.878 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:26 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T05:59:26.878 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:26 smithi016 ceph-mon[17194]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-31T05:59:26.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:26 smithi184 ceph-mon[22331]: pgmap v69: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:26.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:26 smithi184 ceph-mon[22331]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-31T05:59:26.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:26 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-08-31T05:59:26.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:26 smithi184 ceph-mon[22331]: osdmap e21: 3 total, 3 up, 3 in 2024-08-31T05:59:26.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:26 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:26.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:26 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:26.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:26 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:26.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:26 smithi184 ceph-mon[22331]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-31T05:59:26.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:26 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-31T05:59:26.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:26 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T05:59:26.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:26 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T05:59:26.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:26 smithi184 ceph-mon[22331]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-31T05:59:26.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:26 smithi184 ceph-mon[22331]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-31T05:59:26.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:26 smithi184 ceph-mon[22331]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-31T05:59:26.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:26 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-31T05:59:26.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:26 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T05:59:26.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:26 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T05:59:26.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:26 smithi184 ceph-mon[22331]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-31T05:59:26.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:26 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-31T05:59:26.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:26 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T05:59:26.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:26 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T05:59:26.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:26 smithi184 ceph-mon[22331]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-31T05:59:27.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:27.582+0000 7f2386a00640 1 -- 172.21.15.16:0/1189486911 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7f23800c8360 con 0x7f2390105950 2024-08-31T05:59:27.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:27 smithi155 ceph-mon[21742]: osdmap e22: 3 total, 3 up, 3 in 2024-08-31T05:59:27.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:27 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/802567593' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8892e35f-6880-434c-a805-ab8ccbed2bf9"} : dispatch 2024-08-31T05:59:27.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:27 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/802567593' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8892e35f-6880-434c-a805-ab8ccbed2bf9"}]': finished 2024-08-31T05:59:27.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:27 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-31T05:59:27.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:27 smithi155 ceph-mon[21742]: osdmap e23: 4 total, 3 up, 4 in 2024-08-31T05:59:27.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:27 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/291028584' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T05:59:27.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:27 smithi016 ceph-mon[17194]: osdmap e22: 3 total, 3 up, 3 in 2024-08-31T05:59:27.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:27 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/802567593' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8892e35f-6880-434c-a805-ab8ccbed2bf9"} : dispatch 2024-08-31T05:59:27.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:27 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/802567593' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8892e35f-6880-434c-a805-ab8ccbed2bf9"}]': finished 2024-08-31T05:59:27.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:27 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-31T05:59:27.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:27 smithi016 ceph-mon[17194]: osdmap e23: 4 total, 3 up, 4 in 2024-08-31T05:59:27.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:27 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/291028584' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T05:59:27.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:27 smithi184 ceph-mon[22331]: osdmap e22: 3 total, 3 up, 3 in 2024-08-31T05:59:27.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:27 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/802567593' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8892e35f-6880-434c-a805-ab8ccbed2bf9"} : dispatch 2024-08-31T05:59:27.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:27 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/802567593' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8892e35f-6880-434c-a805-ab8ccbed2bf9"}]': finished 2024-08-31T05:59:27.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:27 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-31T05:59:27.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:27 smithi184 ceph-mon[22331]: osdmap e23: 4 total, 3 up, 4 in 2024-08-31T05:59:27.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:27 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/291028584' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T05:59:28.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:28 smithi155 ceph-mon[21742]: pgmap v73: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:28.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:28 smithi155 ceph-mon[21742]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-08-31T05:59:28.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:28 smithi155 ceph-mon[21742]: Cluster is now healthy 2024-08-31T05:59:28.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:28 smithi155 ceph-mon[21742]: mgrmap e20: x(active, since 2m), standbys: z, y 2024-08-31T05:59:28.873 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:28 smithi184 ceph-mon[22331]: pgmap v73: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:28.873 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:28 smithi184 ceph-mon[22331]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-08-31T05:59:28.874 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:28 smithi184 ceph-mon[22331]: Cluster is now healthy 2024-08-31T05:59:28.874 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:28 smithi184 ceph-mon[22331]: mgrmap e20: x(active, since 2m), standbys: z, y 2024-08-31T05:59:28.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:28 smithi016 ceph-mon[17194]: pgmap v73: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:28.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:28 smithi016 ceph-mon[17194]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-08-31T05:59:28.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:28 smithi016 ceph-mon[17194]: Cluster is now healthy 2024-08-31T05:59:28.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:28 smithi016 ceph-mon[17194]: mgrmap e20: x(active, since 2m), standbys: z, y 2024-08-31T05:59:30.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:30 smithi155 ceph-mon[21742]: pgmap v74: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:30.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:30 smithi016 ceph-mon[17194]: pgmap v74: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:30.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:30 smithi184 ceph-mon[22331]: pgmap v74: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:32.845 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:32 smithi016 ceph-mon[17194]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:32.871 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:32 smithi155 ceph-mon[21742]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:32.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:32 smithi184 ceph-mon[22331]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:34.797 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:34 smithi016 ceph-mon[17194]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:34.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:34 smithi155 ceph-mon[21742]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:34.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:34 smithi184 ceph-mon[22331]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:36.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:36 smithi016 ceph-mon[17194]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:36.809 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:36 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-08-31T05:59:36.809 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:36 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:59:36.871 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:36 smithi155 ceph-mon[21742]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:36.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:36 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-08-31T05:59:36.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:36 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:59:36.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:36 smithi184 ceph-mon[22331]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:36.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:36 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-08-31T05:59:36.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:36 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:59:37.691 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:37 smithi016 ceph-mon[17194]: Deploying daemon osd.3 on smithi016 2024-08-31T05:59:37.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:37 smithi155 ceph-mon[21742]: Deploying daemon osd.3 on smithi016 2024-08-31T05:59:37.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:37 smithi184 ceph-mon[22331]: Deploying daemon osd.3 on smithi016 2024-08-31T05:59:38.871 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:38 smithi155 ceph-mon[21742]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:38.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:38 smithi016 ceph-mon[17194]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:38.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:38 smithi184 ceph-mon[22331]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:40.788 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:40 smithi016 ceph-mon[17194]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:40.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:40 smithi155 ceph-mon[21742]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:40.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:40 smithi184 ceph-mon[22331]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:42.809 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:42 smithi016 ceph-mon[17194]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:42.871 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:42 smithi155 ceph-mon[21742]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:42.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:42 smithi184 ceph-mon[22331]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:43.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:43 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:59:43.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:43 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:43.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:43 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:43.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:43 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:59:43.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:43 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:43.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:43 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:43.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:43 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:59:43.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:43 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:43.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:43 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:44.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:44 smithi016 ceph-mon[17194]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:44.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:44 smithi155 ceph-mon[21742]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:44.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:44 smithi184 ceph-mon[22331]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:46.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:46 smithi016 ceph-mon[17194]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:46.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:46 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:46.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:46 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:46.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:46 smithi155 ceph-mon[21742]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:46.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:46 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:46.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:46 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:46.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:46 smithi184 ceph-mon[22331]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:46.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:46 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:46.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:46 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:47.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:47 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:59:47.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:47 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T05:59:47.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:47 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:47.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:47 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:59:47.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:47 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T05:59:47.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:47 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:47.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:47 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:59:47.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:47 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T05:59:47.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:47 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:48.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:48.972+0000 7f2386a00640 1 -- 172.21.15.16:0/1189486911 <== mgr.14220 v2:172.21.15.16:6800/4279751337 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f23900635a0 con 0x7f23740b4af0 2024-08-31T05:59:48.975 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 3 on host 'smithi016' 2024-08-31T05:59:48.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:48.974+0000 7f238f200640 1 -- 172.21.15.16:0/1189486911 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f23740b4af0 msgr2=0x7f23740b7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:59:48.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:48.974+0000 7f238f200640 1 --2- 172.21.15.16:0/1189486911 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f23740b4af0 0x7f23740b7000 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f239010ae40 tx=0x7f237c005f90 comp rx=0 tx=0).stop 2024-08-31T05:59:48.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:48.974+0000 7f238f200640 1 -- 172.21.15.16:0/1189486911 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2390105950 msgr2=0x7f239019c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:59:48.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:48.974+0000 7f238f200640 1 --2- 172.21.15.16:0/1189486911 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2390105950 0x7f239019c310 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f238002a930 tx=0x7f2380031a20 comp rx=0 tx=0).stop 2024-08-31T05:59:48.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:48.974+0000 7f238f200640 1 -- 172.21.15.16:0/1189486911 shutdown_connections 2024-08-31T05:59:48.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:48.974+0000 7f238f200640 1 --2- 172.21.15.16:0/1189486911 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f23740b4af0 0x7f23740b7000 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:59:48.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:48.974+0000 7f238f200640 1 --2- 172.21.15.16:0/1189486911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f239010bbe0 0x7f23901a1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:59:48.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:48.974+0000 7f238f200640 1 --2- 172.21.15.16:0/1189486911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f239010b290 0x7f239019c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:59:48.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:48.974+0000 7f238f200640 1 --2- 172.21.15.16:0/1189486911 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2390105950 0x7f239019c310 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:59:48.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:48.974+0000 7f238f200640 1 -- 172.21.15.16:0/1189486911 >> 172.21.15.16:0/1189486911 conn(0x7f2390101420 msgr2=0x7f2390103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:59:48.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:48.975+0000 7f238f200640 1 -- 172.21.15.16:0/1189486911 shutdown_connections 2024-08-31T05:59:48.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T05:59:48.975+0000 7f238f200640 1 -- 172.21.15.16:0/1189486911 wait complete. 2024-08-31T05:59:49.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:48 smithi016 ceph-mon[17194]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:49.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:48 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:59:49.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:48 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:49.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:48 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:49.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:48 smithi155 ceph-mon[21742]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:49.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:48 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:59:49.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:48 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:49.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:48 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:49.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:48 smithi184 ceph-mon[22331]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:49.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:48 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T05:59:49.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:48 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:49.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:48 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:49.708 DEBUG:teuthology.orchestra.run.smithi016:osd.3> sudo journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@osd.3.service 2024-08-31T05:59:49.712 INFO:tasks.cephadm:Deploying osd.4 on smithi155 with /dev/vg_nvme/lv_3... 2024-08-31T05:59:49.713 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-08-31T05:59:49.967 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.b/config 2024-08-31T05:59:50.088 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:49 smithi155 ceph-mon[21742]: from='osd.3 [v2:172.21.15.16:6810/81681815,v1:172.21.15.16:6811/81681815]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-08-31T05:59:50.088 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:49 smithi155 ceph-mon[21742]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-08-31T05:59:50.298 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:49 smithi016 ceph-mon[17194]: from='osd.3 [v2:172.21.15.16:6810/81681815,v1:172.21.15.16:6811/81681815]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-08-31T05:59:50.298 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:49 smithi016 ceph-mon[17194]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-08-31T05:59:50.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:49 smithi184 ceph-mon[22331]: from='osd.3 [v2:172.21.15.16:6810/81681815,v1:172.21.15.16:6811/81681815]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-08-31T05:59:50.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:49 smithi184 ceph-mon[22331]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-08-31T05:59:51.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:50 smithi016 ceph-mon[17194]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:51.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:50 smithi016 ceph-mon[17194]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-08-31T05:59:51.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:50 smithi016 ceph-mon[17194]: osdmap e24: 4 total, 3 up, 4 in 2024-08-31T05:59:51.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:50 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-31T05:59:51.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:50 smithi016 ceph-mon[17194]: from='osd.3 [v2:172.21.15.16:6810/81681815,v1:172.21.15.16:6811/81681815]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-08-31T05:59:51.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:50 smithi016 ceph-mon[17194]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-08-31T05:59:51.016 INFO:journalctl@ceph.osd.3.smithi016.stdout:Aug 31 05:59:51 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-3[42089]: 2024-08-31T05:59:51.000+0000 7fa93e800640 -1 osd.3 0 waiting for initial osdmap 2024-08-31T05:59:51.016 INFO:journalctl@ceph.osd.3.smithi016.stdout:Aug 31 05:59:51 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-3[42089]: 2024-08-31T05:59:51.013+0000 7fa939800640 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-31T05:59:51.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:50 smithi155 ceph-mon[21742]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:51.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:50 smithi155 ceph-mon[21742]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-08-31T05:59:51.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:50 smithi155 ceph-mon[21742]: osdmap e24: 4 total, 3 up, 4 in 2024-08-31T05:59:51.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:50 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-31T05:59:51.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:50 smithi155 ceph-mon[21742]: from='osd.3 [v2:172.21.15.16:6810/81681815,v1:172.21.15.16:6811/81681815]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-08-31T05:59:51.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:50 smithi155 ceph-mon[21742]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-08-31T05:59:51.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:50 smithi184 ceph-mon[22331]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:51.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:50 smithi184 ceph-mon[22331]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-08-31T05:59:51.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:50 smithi184 ceph-mon[22331]: osdmap e24: 4 total, 3 up, 4 in 2024-08-31T05:59:51.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:50 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-31T05:59:51.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:50 smithi184 ceph-mon[22331]: from='osd.3 [v2:172.21.15.16:6810/81681815,v1:172.21.15.16:6811/81681815]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-08-31T05:59:51.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:50 smithi184 ceph-mon[22331]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-08-31T05:59:52.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:52 smithi155 ceph-mon[21742]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-08-31T05:59:52.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:52 smithi155 ceph-mon[21742]: osdmap e25: 4 total, 3 up, 4 in 2024-08-31T05:59:52.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:52 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-31T05:59:52.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:52 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-31T05:59:52.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:52 smithi016 ceph-mon[17194]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-08-31T05:59:52.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:52 smithi016 ceph-mon[17194]: osdmap e25: 4 total, 3 up, 4 in 2024-08-31T05:59:52.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:52 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-31T05:59:52.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:52 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-31T05:59:52.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:52 smithi184 ceph-mon[22331]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-08-31T05:59:52.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:52 smithi184 ceph-mon[22331]: osdmap e25: 4 total, 3 up, 4 in 2024-08-31T05:59:52.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:52 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-31T05:59:52.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:52 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-31T05:59:52.628 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-08-31T05:59:52.654 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph orch daemon add osd smithi155:vg_nvme/lv_3 2024-08-31T05:59:52.911 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.b/config 2024-08-31T05:59:53.136 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:53 smithi016 ceph-mon[17194]: purged_snaps scrub starts 2024-08-31T05:59:53.136 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:53 smithi016 ceph-mon[17194]: purged_snaps scrub ok 2024-08-31T05:59:53.136 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:53 smithi016 ceph-mon[17194]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:53.136 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:53 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-31T05:59:53.136 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:53 smithi016 ceph-mon[17194]: osd.3 [v2:172.21.15.16:6810/81681815,v1:172.21.15.16:6811/81681815] boot 2024-08-31T05:59:53.136 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:53 smithi016 ceph-mon[17194]: osdmap e26: 4 total, 4 up, 4 in 2024-08-31T05:59:53.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:53 smithi155 ceph-mon[21742]: purged_snaps scrub starts 2024-08-31T05:59:53.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:53 smithi155 ceph-mon[21742]: purged_snaps scrub ok 2024-08-31T05:59:53.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:53 smithi155 ceph-mon[21742]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:53.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:53 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-31T05:59:53.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:53 smithi155 ceph-mon[21742]: osd.3 [v2:172.21.15.16:6810/81681815,v1:172.21.15.16:6811/81681815] boot 2024-08-31T05:59:53.276 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:53 smithi155 ceph-mon[21742]: osdmap e26: 4 total, 4 up, 4 in 2024-08-31T05:59:53.382 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:59:53.371+0000 7f7efbd69640 1 -- 172.21.15.155:0/3782013729 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ef4101d10 msgr2=0x7f7ef4102120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:59:53.383 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:59:53.371+0000 7f7efbd69640 1 --2- 172.21.15.155:0/3782013729 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ef4101d10 0x7f7ef4102120 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4009900 tx=0x7f7ee402a410 comp rx=0 tx=0).stop 2024-08-31T05:59:53.383 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:59:53.371+0000 7f7efbd69640 1 -- 172.21.15.155:0/3782013729 shutdown_connections 2024-08-31T05:59:53.383 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:59:53.371+0000 7f7efbd69640 1 --2- 172.21.15.155:0/3782013729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ef4109250 0x7f7ef4109660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:59:53.383 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:59:53.371+0000 7f7efbd69640 1 --2- 172.21.15.155:0/3782013729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ef4109ba0 0x7f7ef41103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:59:53.383 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:59:53.371+0000 7f7efbd69640 1 --2- 172.21.15.155:0/3782013729 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ef4101d10 0x7f7ef4102120 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:59:53.383 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:59:53.371+0000 7f7efbd69640 1 -- 172.21.15.155:0/3782013729 >> 172.21.15.155:0/3782013729 conn(0x7f7ef4077a00 msgr2=0x7f7ef4077e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T05:59:53.383 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:59:53.371+0000 7f7efbd69640 1 -- 172.21.15.155:0/3782013729 shutdown_connections 2024-08-31T05:59:53.384 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:59:53.371+0000 7f7efbd69640 1 -- 172.21.15.155:0/3782013729 wait complete. 2024-08-31T05:59:53.384 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:59:53.372+0000 7f7efbd69640 1 Processor -- start 2024-08-31T05:59:53.384 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:59:53.372+0000 7f7efbd69640 1 -- start start 2024-08-31T05:59:53.384 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:59:53.372+0000 7f7efbd69640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ef4101d10 0x7f7ef419a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:59:53.384 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:59:53.372+0000 7f7efbd69640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ef4109250 0x7f7ef419a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:59:53.384 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:59:53.372+0000 7f7efbd69640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ef4109ba0 0x7f7ef419faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:59:53.384 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:59:53.372+0000 7f7efbd69640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef4113860 con 0x7f7ef4109250 2024-08-31T05:59:53.384 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:59:53.372+0000 7f7efbd69640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef41136e0 con 0x7f7ef4109ba0 2024-08-31T05:59:53.384 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:59:53.372+0000 7f7efbd69640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef41139e0 con 0x7f7ef4101d10 2024-08-31T05:59:53.384 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:59:53.372+0000 7f7efa306640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ef4109ba0 0x7f7ef419faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:59:53.384 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:59:53.372+0000 7f7efa306640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ef4109ba0 0x7f7ef419faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:43436/0 (socket says 172.21.15.155:43436) 2024-08-31T05:59:53.385 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:59:53.372+0000 7f7efa306640 1 -- 172.21.15.155:0/2105109747 learned_addr learned my addr 172.21.15.155:0/2105109747 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-08-31T05:59:53.385 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:59:53.372+0000 7f7ef9304640 1 --2- 172.21.15.155:0/2105109747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ef4109250 0x7f7ef419a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:59:53.385 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:59:53.373+0000 7f7ef9b05640 1 --2- 172.21.15.155:0/2105109747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ef4101d10 0x7f7ef419a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:59:53.385 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:59:53.373+0000 7f7efa306640 1 -- 172.21.15.155:0/2105109747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ef4101d10 msgr2=0x7f7ef419a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:59:53.385 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:59:53.373+0000 7f7efa306640 1 --2- 172.21.15.155:0/2105109747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ef4101d10 0x7f7ef419a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:59:53.385 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:59:53.373+0000 7f7efa306640 1 -- 172.21.15.155:0/2105109747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ef4109250 msgr2=0x7f7ef419a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T05:59:53.385 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:59:53.373+0000 7f7efa306640 1 --2- 172.21.15.155:0/2105109747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ef4109250 0x7f7ef419a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T05:59:53.385 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:59:53.373+0000 7f7efa306640 1 -- 172.21.15.155:0/2105109747 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef41a0220 con 0x7f7ef4109ba0 2024-08-31T05:59:53.385 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:59:53.373+0000 7f7efa306640 1 --2- 172.21.15.155:0/2105109747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ef4109ba0 0x7f7ef419faf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7eec009fd0 tx=0x7f7eec00b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:59:53.385 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:59:53.373+0000 7f7eeaffd640 1 -- 172.21.15.155:0/2105109747 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec017070 con 0x7f7ef4109ba0 2024-08-31T05:59:53.385 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:59:53.373+0000 7f7efbd69640 1 -- 172.21.15.155:0/2105109747 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ef41a0510 con 0x7f7ef4109ba0 2024-08-31T05:59:53.385 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:59:53.373+0000 7f7eeaffd640 1 -- 172.21.15.155:0/2105109747 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7eec003e20 con 0x7f7ef4109ba0 2024-08-31T05:59:53.385 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:59:53.373+0000 7f7eeaffd640 1 -- 172.21.15.155:0/2105109747 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec010460 con 0x7f7ef4109ba0 2024-08-31T05:59:53.386 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:59:53.373+0000 7f7efbd69640 1 -- 172.21.15.155:0/2105109747 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ef41a6c70 con 0x7f7ef4109ba0 2024-08-31T05:59:53.386 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:59:53.374+0000 7f7efbd69640 1 -- 172.21.15.155:0/2105109747 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ef4102120 con 0x7f7ef4109ba0 2024-08-31T05:59:53.386 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:59:53.376+0000 7f7eeaffd640 1 -- 172.21.15.155:0/2105109747 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7f7eec003970 con 0x7f7ef4109ba0 2024-08-31T05:59:53.386 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:59:53.377+0000 7f7eeaffd640 1 --2- 172.21.15.155:0/2105109747 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7ee00b4b80 0x7f7ee00b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T05:59:53.386 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:59:53.377+0000 7f7eeaffd640 1 -- 172.21.15.155:0/2105109747 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3884+0+0 (secure 0 0 0) 0x7f7eec0e4180 con 0x7f7ef4109ba0 2024-08-31T05:59:53.386 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:59:53.378+0000 7f7ef9b05640 1 --2- 172.21.15.155:0/2105109747 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7ee00b4b80 0x7f7ee00b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T05:59:53.386 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:59:53.378+0000 7f7ef9b05640 1 --2- 172.21.15.155:0/2105109747 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7ee00b4b80 0x7f7ee00b7090 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4031440 tx=0x7f7ee4005d50 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T05:59:53.386 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:59:53.380+0000 7f7eeaffd640 1 -- 172.21.15.155:0/2105109747 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f7eec0ac400 con 0x7f7ef4109ba0 2024-08-31T05:59:53.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:53 smithi184 ceph-mon[22331]: purged_snaps scrub starts 2024-08-31T05:59:53.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:53 smithi184 ceph-mon[22331]: purged_snaps scrub ok 2024-08-31T05:59:53.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:53 smithi184 ceph-mon[22331]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-31T05:59:53.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:53 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-31T05:59:53.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:53 smithi184 ceph-mon[22331]: osd.3 [v2:172.21.15.16:6810/81681815,v1:172.21.15.16:6811/81681815] boot 2024-08-31T05:59:53.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:53 smithi184 ceph-mon[22331]: osdmap e26: 4 total, 4 up, 4 in 2024-08-31T05:59:53.586 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T05:59:53.583+0000 7f7efbd69640 1 -- 172.21.15.155:0/2105109747 --> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f7ef40635a0 con 0x7f7ee00b4b80 2024-08-31T05:59:54.278 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:54 smithi155 ceph-mon[21742]: osdmap e27: 4 total, 4 up, 4 in 2024-08-31T05:59:54.278 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:54 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:54.278 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:54 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:54.278 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:54 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T05:59:54.278 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:54 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T05:59:54.278 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:54 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T05:59:54.279 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:54 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T05:59:54.279 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:54 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:54.279 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:54 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:59:54.279 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:54 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T05:59:54.279 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:54 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:54.279 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:54 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T05:59:54.279 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:54 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T05:59:54.279 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:54 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:59:54.279 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:54 smithi155 ceph-mon[21742]: osdmap e28: 4 total, 4 up, 4 in 2024-08-31T05:59:54.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:54 smithi016 ceph-mon[17194]: osdmap e27: 4 total, 4 up, 4 in 2024-08-31T05:59:54.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:54 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:54.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:54 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:54.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:54 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T05:59:54.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:54 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T05:59:54.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:54 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T05:59:54.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:54 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T05:59:54.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:54 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:54.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:54 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:59:54.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:54 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T05:59:54.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:54 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:54.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:54 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T05:59:54.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:54 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T05:59:54.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:54 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:59:54.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:54 smithi016 ceph-mon[17194]: osdmap e28: 4 total, 4 up, 4 in 2024-08-31T05:59:54.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:54 smithi184 ceph-mon[22331]: osdmap e27: 4 total, 4 up, 4 in 2024-08-31T05:59:54.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:54 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:54.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:54 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:54.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:54 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T05:59:54.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:54 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T05:59:54.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:54 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T05:59:54.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:54 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T05:59:54.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:54 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:54.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:54 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:59:54.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:54 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T05:59:54.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:54 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T05:59:54.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:54 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T05:59:54.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:54 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T05:59:54.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:54 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T05:59:54.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:54 smithi184 ceph-mon[22331]: osdmap e28: 4 total, 4 up, 4 in 2024-08-31T05:59:55.335 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:55 smithi155 ceph-mon[21742]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-08-31T05:59:55.335 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:55 smithi155 ceph-mon[21742]: Detected new or changed devices on smithi016 2024-08-31T05:59:55.336 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:55 smithi155 ceph-mon[21742]: Adjusting osd_memory_target on smithi016 to 8547M 2024-08-31T05:59:55.336 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:55 smithi155 ceph-mon[21742]: from='client.24416 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T05:59:55.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:55 smithi016 ceph-mon[17194]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-08-31T05:59:55.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:55 smithi016 ceph-mon[17194]: Detected new or changed devices on smithi016 2024-08-31T05:59:55.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:55 smithi016 ceph-mon[17194]: Adjusting osd_memory_target on smithi016 to 8547M 2024-08-31T05:59:55.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:55 smithi016 ceph-mon[17194]: from='client.24416 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T05:59:55.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:55 smithi184 ceph-mon[22331]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-08-31T05:59:55.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:55 smithi184 ceph-mon[22331]: Detected new or changed devices on smithi016 2024-08-31T05:59:55.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:55 smithi184 ceph-mon[22331]: Adjusting osd_memory_target on smithi016 to 8547M 2024-08-31T05:59:55.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:55 smithi184 ceph-mon[22331]: from='client.24416 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T05:59:57.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:57 smithi155 ceph-mon[21742]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-08-31T05:59:57.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:57 smithi155 ceph-mon[21742]: from='client.? 172.21.15.155:0/1293869438' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d2515e73-88f7-4654-9762-adb8802777ed"} : dispatch 2024-08-31T05:59:57.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:57 smithi155 ceph-mon[21742]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d2515e73-88f7-4654-9762-adb8802777ed"} : dispatch 2024-08-31T05:59:57.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:57 smithi155 ceph-mon[21742]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d2515e73-88f7-4654-9762-adb8802777ed"}]': finished 2024-08-31T05:59:57.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:57 smithi155 ceph-mon[21742]: osdmap e29: 5 total, 4 up, 5 in 2024-08-31T05:59:57.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:57 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-31T05:59:57.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:57 smithi155 ceph-mon[21742]: from='client.? 172.21.15.155:0/2142300412' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T05:59:57.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:57 smithi016 ceph-mon[17194]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-08-31T05:59:57.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:57 smithi016 ceph-mon[17194]: from='client.? 172.21.15.155:0/1293869438' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d2515e73-88f7-4654-9762-adb8802777ed"} : dispatch 2024-08-31T05:59:57.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:57 smithi016 ceph-mon[17194]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d2515e73-88f7-4654-9762-adb8802777ed"} : dispatch 2024-08-31T05:59:57.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:57 smithi016 ceph-mon[17194]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d2515e73-88f7-4654-9762-adb8802777ed"}]': finished 2024-08-31T05:59:57.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:57 smithi016 ceph-mon[17194]: osdmap e29: 5 total, 4 up, 5 in 2024-08-31T05:59:57.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:57 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-31T05:59:57.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:57 smithi016 ceph-mon[17194]: from='client.? 172.21.15.155:0/2142300412' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T05:59:57.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:57 smithi184 ceph-mon[22331]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-08-31T05:59:57.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:57 smithi184 ceph-mon[22331]: from='client.? 172.21.15.155:0/1293869438' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d2515e73-88f7-4654-9762-adb8802777ed"} : dispatch 2024-08-31T05:59:57.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:57 smithi184 ceph-mon[22331]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d2515e73-88f7-4654-9762-adb8802777ed"} : dispatch 2024-08-31T05:59:57.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:57 smithi184 ceph-mon[22331]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d2515e73-88f7-4654-9762-adb8802777ed"}]': finished 2024-08-31T05:59:57.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:57 smithi184 ceph-mon[22331]: osdmap e29: 5 total, 4 up, 5 in 2024-08-31T05:59:57.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:57 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-31T05:59:57.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:57 smithi184 ceph-mon[22331]: from='client.? 172.21.15.155:0/2142300412' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T05:59:59.292 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 05:59:59 smithi184 ceph-mon[22331]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-08-31T05:59:59.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 05:59:59 smithi155 ceph-mon[21742]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-08-31T05:59:59.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 05:59:59 smithi016 ceph-mon[17194]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-08-31T06:00:00.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:00 smithi155 ceph-mon[21742]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-08-31T06:00:00.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:00 smithi155 ceph-mon[21742]: overall HEALTH_OK 2024-08-31T06:00:00.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:00 smithi016 ceph-mon[17194]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-08-31T06:00:00.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:00 smithi016 ceph-mon[17194]: overall HEALTH_OK 2024-08-31T06:00:00.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:00 smithi184 ceph-mon[22331]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-08-31T06:00:00.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:00 smithi184 ceph-mon[22331]: overall HEALTH_OK 2024-08-31T06:00:02.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:02 smithi155 ceph-mon[21742]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-31T06:00:02.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:02 smithi016 ceph-mon[17194]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-31T06:00:02.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:02 smithi184 ceph-mon[22331]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-31T06:00:04.513 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:04 smithi155 ceph-mon[21742]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-08-31T06:00:04.513 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:04 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-08-31T06:00:04.513 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:04 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:00:04.513 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:04 smithi155 ceph-mon[21742]: Deploying daemon osd.4 on smithi155 2024-08-31T06:00:04.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:04 smithi016 ceph-mon[17194]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-08-31T06:00:04.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:04 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-08-31T06:00:04.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:04 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:00:04.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:04 smithi016 ceph-mon[17194]: Deploying daemon osd.4 on smithi155 2024-08-31T06:00:04.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:04 smithi184 ceph-mon[22331]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-08-31T06:00:04.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:04 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-08-31T06:00:04.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:04 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:00:04.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:04 smithi184 ceph-mon[22331]: Deploying daemon osd.4 on smithi155 2024-08-31T06:00:06.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:06 smithi155 ceph-mon[21742]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-08-31T06:00:06.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:06 smithi016 ceph-mon[17194]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-08-31T06:00:06.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:06 smithi184 ceph-mon[22331]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-08-31T06:00:08.352 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:08 smithi155 ceph-mon[21742]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-08-31T06:00:08.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:08 smithi016 ceph-mon[17194]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-08-31T06:00:08.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:08 smithi184 ceph-mon[22331]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-08-31T06:00:10.450 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:10 smithi155 ceph-mon[21742]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-08-31T06:00:10.450 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:10 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:00:10.450 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:10 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:10.450 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:10 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:10.569 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:10 smithi184 ceph-mon[22331]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-08-31T06:00:10.569 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:10 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:00:10.569 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:10 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:10.570 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:10 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:10.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:10 smithi016 ceph-mon[17194]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-08-31T06:00:10.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:10 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:00:10.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:10 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:10.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:10 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:12.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:12 smithi155 ceph-mon[21742]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-08-31T06:00:12.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:12 smithi016 ceph-mon[17194]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-08-31T06:00:12.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:12 smithi184 ceph-mon[22331]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-08-31T06:00:13.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:13 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:13.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:13 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:13.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:13 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:00:13.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:13 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:00:13.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:13 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:13.731 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:13 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:13.731 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:13 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:13.731 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:13 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:00:13.731 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:13 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:00:13.731 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:13 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:13.780 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:00:13.777+0000 7f7eeaffd640 1 -- 172.21.15.155:0/2105109747 <== mgr.14220 v2:172.21.15.16:6800/4279751337 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7ef40635a0 con 0x7f7ee00b4b80 2024-08-31T06:00:13.780 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 4 on host 'smithi155' 2024-08-31T06:00:13.783 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:00:13.780+0000 7f7efbd69640 1 -- 172.21.15.155:0/2105109747 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7ee00b4b80 msgr2=0x7f7ee00b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:00:13.783 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:00:13.780+0000 7f7efbd69640 1 --2- 172.21.15.155:0/2105109747 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7ee00b4b80 0x7f7ee00b7090 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4031440 tx=0x7f7ee4005d50 comp rx=0 tx=0).stop 2024-08-31T06:00:13.783 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:00:13.780+0000 7f7efbd69640 1 -- 172.21.15.155:0/2105109747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ef4109ba0 msgr2=0x7f7ef419faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:00:13.783 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:00:13.780+0000 7f7efbd69640 1 --2- 172.21.15.155:0/2105109747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ef4109ba0 0x7f7ef419faf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7eec009fd0 tx=0x7f7eec00b260 comp rx=0 tx=0).stop 2024-08-31T06:00:13.783 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:00:13.780+0000 7f7efbd69640 1 -- 172.21.15.155:0/2105109747 shutdown_connections 2024-08-31T06:00:13.783 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:00:13.780+0000 7f7efbd69640 1 --2- 172.21.15.155:0/2105109747 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7ee00b4b80 0x7f7ee00b7090 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:00:13.783 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:00:13.780+0000 7f7efbd69640 1 --2- 172.21.15.155:0/2105109747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ef4109ba0 0x7f7ef419faf0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:00:13.784 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:00:13.780+0000 7f7efbd69640 1 --2- 172.21.15.155:0/2105109747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ef4109250 0x7f7ef419a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:00:13.784 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:00:13.780+0000 7f7efbd69640 1 --2- 172.21.15.155:0/2105109747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ef4101d10 0x7f7ef419a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:00:13.784 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:00:13.780+0000 7f7efbd69640 1 -- 172.21.15.155:0/2105109747 >> 172.21.15.155:0/2105109747 conn(0x7f7ef4077a00 msgr2=0x7f7ef4107350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:00:13.784 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:00:13.780+0000 7f7efbd69640 1 -- 172.21.15.155:0/2105109747 shutdown_connections 2024-08-31T06:00:13.784 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:00:13.780+0000 7f7efbd69640 1 -- 172.21.15.155:0/2105109747 wait complete. 2024-08-31T06:00:13.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:13 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:13.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:13 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:13.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:13 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:00:13.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:13 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:00:13.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:13 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:14.403 DEBUG:teuthology.orchestra.run.smithi155:osd.4> sudo journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@osd.4.service 2024-08-31T06:00:14.406 INFO:tasks.cephadm:Deploying osd.5 on smithi184 with /dev/vg_nvme/lv_3... 2024-08-31T06:00:14.407 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-08-31T06:00:14.665 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.c/config 2024-08-31T06:00:14.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:14 smithi184 ceph-mon[22331]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-08-31T06:00:14.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:14 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:00:14.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:14 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:14.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:14 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:14.741 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:14 smithi155 ceph-mon[21742]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-08-31T06:00:14.741 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:14 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:00:14.741 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:14 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:14.741 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:14 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:14.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:14 smithi016 ceph-mon[17194]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-08-31T06:00:14.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:14 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:00:14.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:14 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:14.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:14 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:15.872 INFO:journalctl@ceph.osd.4.smithi155.stdout:Aug 31 06:00:15 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-4[33944]: 2024-08-31T06:00:15.793+0000 7fc8b4ed7640 -1 osd.4 0 log_to_monitors true 2024-08-31T06:00:16.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:16 smithi155 ceph-mon[21742]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-08-31T06:00:16.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:16 smithi155 ceph-mon[21742]: from='osd.4 [v2:172.21.15.155:6808/3592422019,v1:172.21.15.155:6809/3592422019]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-08-31T06:00:16.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:16 smithi155 ceph-mon[21742]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-08-31T06:00:16.645 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:16 smithi184 ceph-mon[22331]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-08-31T06:00:16.645 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:16 smithi184 ceph-mon[22331]: from='osd.4 [v2:172.21.15.155:6808/3592422019,v1:172.21.15.155:6809/3592422019]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-08-31T06:00:16.645 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:16 smithi184 ceph-mon[22331]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-08-31T06:00:16.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:16 smithi016 ceph-mon[17194]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-08-31T06:00:16.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:16 smithi016 ceph-mon[17194]: from='osd.4 [v2:172.21.15.155:6808/3592422019,v1:172.21.15.155:6809/3592422019]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-08-31T06:00:16.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:16 smithi016 ceph-mon[17194]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-08-31T06:00:17.270 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-08-31T06:00:17.296 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph orch daemon add osd smithi184:vg_nvme/lv_3 2024-08-31T06:00:17.551 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.c/config 2024-08-31T06:00:17.585 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:17 smithi184 ceph-mon[22331]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-08-31T06:00:17.585 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:17 smithi184 ceph-mon[22331]: osdmap e30: 5 total, 4 up, 5 in 2024-08-31T06:00:17.585 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:17 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-31T06:00:17.585 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:17 smithi184 ceph-mon[22331]: from='osd.4 [v2:172.21.15.155:6808/3592422019,v1:172.21.15.155:6809/3592422019]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]} : dispatch 2024-08-31T06:00:17.585 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:17 smithi184 ceph-mon[22331]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]} : dispatch 2024-08-31T06:00:17.585 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:17 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:17.585 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:17 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:17.585 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:17 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T06:00:17.586 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:17 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T06:00:17.586 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:17 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-31T06:00:17.586 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:17 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-31T06:00:17.586 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:17 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:17.586 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:17 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:00:17.586 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:17 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:00:17.586 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:17 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:17.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:17 smithi155 ceph-mon[21742]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-08-31T06:00:17.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:17 smithi155 ceph-mon[21742]: osdmap e30: 5 total, 4 up, 5 in 2024-08-31T06:00:17.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:17 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-31T06:00:17.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:17 smithi155 ceph-mon[21742]: from='osd.4 [v2:172.21.15.155:6808/3592422019,v1:172.21.15.155:6809/3592422019]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]} : dispatch 2024-08-31T06:00:17.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:17 smithi155 ceph-mon[21742]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]} : dispatch 2024-08-31T06:00:17.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:17 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:17.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:17 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:17.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:17 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T06:00:17.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:17 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T06:00:17.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:17 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-31T06:00:17.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:17 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-31T06:00:17.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:17 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:17.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:17 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:00:17.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:17 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:00:17.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:17 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:17.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:17 smithi016 ceph-mon[17194]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-08-31T06:00:17.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:17 smithi016 ceph-mon[17194]: osdmap e30: 5 total, 4 up, 5 in 2024-08-31T06:00:17.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:17 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-31T06:00:17.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:17 smithi016 ceph-mon[17194]: from='osd.4 [v2:172.21.15.155:6808/3592422019,v1:172.21.15.155:6809/3592422019]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]} : dispatch 2024-08-31T06:00:17.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:17 smithi016 ceph-mon[17194]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]} : dispatch 2024-08-31T06:00:17.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:17 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:17.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:17 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:17.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:17 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T06:00:17.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:17 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T06:00:17.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:17 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-31T06:00:17.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:17 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-31T06:00:17.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:17 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:17.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:17 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:00:17.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:17 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:00:17.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:17 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:18.070 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:18.065+0000 7f37a80ef640 1 -- 172.21.15.184:0/1517399383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37a0108ac0 msgr2=0x7f37a010f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:00:18.071 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:18.065+0000 7f37a80ef640 1 --2- 172.21.15.184:0/1517399383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37a0108ac0 0x7f37a010f2f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3798009ca0 tx=0x7f379802a8c0 comp rx=0 tx=0).stop 2024-08-31T06:00:18.071 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:18.065+0000 7f37a80ef640 1 -- 172.21.15.184:0/1517399383 shutdown_connections 2024-08-31T06:00:18.071 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:18.065+0000 7f37a80ef640 1 --2- 172.21.15.184:0/1517399383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37a0108ac0 0x7f37a010f2f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:00:18.071 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:18.065+0000 7f37a80ef640 1 --2- 172.21.15.184:0/1517399383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37a0108170 0x7f37a0108580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:00:18.071 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:18.065+0000 7f37a80ef640 1 --2- 172.21.15.184:0/1517399383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a0102830 0x7f37a0102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:00:18.071 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:18.065+0000 7f37a80ef640 1 -- 172.21.15.184:0/1517399383 >> 172.21.15.184:0/1517399383 conn(0x7f37a00794d0 msgr2=0x7f37a00798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:00:18.071 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:18.065+0000 7f37a80ef640 1 -- 172.21.15.184:0/1517399383 shutdown_connections 2024-08-31T06:00:18.071 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:18.065+0000 7f37a80ef640 1 -- 172.21.15.184:0/1517399383 wait complete. 2024-08-31T06:00:18.071 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:18.065+0000 7f37a80ef640 1 Processor -- start 2024-08-31T06:00:18.071 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:18.065+0000 7f37a80ef640 1 -- start start 2024-08-31T06:00:18.071 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:18.066+0000 7f37a80ef640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37a0102830 0x7f37a010fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:00:18.072 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:18.066+0000 7f37a80ef640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37a0108170 0x7f37a0110310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:00:18.072 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:18.066+0000 7f37a80ef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a0108ac0 0x7f37a0110850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:00:18.072 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:18.066+0000 7f37a80ef640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f37a01138a0 con 0x7f37a0108ac0 2024-08-31T06:00:18.072 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:18.066+0000 7f37a80ef640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f37a0113720 con 0x7f37a0108170 2024-08-31T06:00:18.072 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:18.066+0000 7f37a80ef640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f37a0113a20 con 0x7f37a0102830 2024-08-31T06:00:18.072 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:18.066+0000 7f37a5e8b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37a0102830 0x7f37a010fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:00:18.072 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:18.066+0000 7f37a5e8b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37a0102830 0x7f37a010fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:55604/0 (socket says 172.21.15.184:55604) 2024-08-31T06:00:18.072 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:18.066+0000 7f37a5e8b640 1 -- 172.21.15.184:0/909338333 learned_addr learned my addr 172.21.15.184:0/909338333 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-08-31T06:00:18.072 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:18.066+0000 7f37a668c640 1 --2- 172.21.15.184:0/909338333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a0108ac0 0x7f37a0110850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:00:18.072 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:18.066+0000 7f37a568a640 1 --2- 172.21.15.184:0/909338333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37a0108170 0x7f37a0110310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:00:18.072 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:18.066+0000 7f37a5e8b640 1 -- 172.21.15.184:0/909338333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37a0108170 msgr2=0x7f37a0110310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:00:18.073 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:18.066+0000 7f37a5e8b640 1 --2- 172.21.15.184:0/909338333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37a0108170 0x7f37a0110310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:00:18.073 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:18.066+0000 7f37a5e8b640 1 -- 172.21.15.184:0/909338333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a0108ac0 msgr2=0x7f37a0110850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:00:18.073 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:18.066+0000 7f37a5e8b640 1 --2- 172.21.15.184:0/909338333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a0108ac0 0x7f37a0110850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:00:18.073 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:18.066+0000 7f37a5e8b640 1 -- 172.21.15.184:0/909338333 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37a01a8490 con 0x7f37a0102830 2024-08-31T06:00:18.073 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:18.066+0000 7f37a568a640 1 --2- 172.21.15.184:0/909338333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37a0108170 0x7f37a0110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:00:18.073 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:18.066+0000 7f37a5e8b640 1 --2- 172.21.15.184:0/909338333 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37a0102830 0x7f37a010fdd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f379000d950 tx=0x7f379000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:00:18.073 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:18.067+0000 7f3796ffd640 1 -- 172.21.15.184:0/909338333 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3790017070 con 0x7f37a0102830 2024-08-31T06:00:18.073 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:18.067+0000 7f3796ffd640 1 -- 172.21.15.184:0/909338333 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3790009c30 con 0x7f37a0102830 2024-08-31T06:00:18.073 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:18.067+0000 7f3796ffd640 1 -- 172.21.15.184:0/909338333 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379000f620 con 0x7f37a0102830 2024-08-31T06:00:18.073 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:18.067+0000 7f37a80ef640 1 -- 172.21.15.184:0/909338333 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37a01a8780 con 0x7f37a0102830 2024-08-31T06:00:18.073 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:18.067+0000 7f37a80ef640 1 -- 172.21.15.184:0/909338333 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37a01a8c70 con 0x7f37a0102830 2024-08-31T06:00:18.073 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:18.068+0000 7f37a80ef640 1 -- 172.21.15.184:0/909338333 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37a0103ab0 con 0x7f37a0102830 2024-08-31T06:00:18.074 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:18.070+0000 7f3796ffd640 1 -- 172.21.15.184:0/909338333 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7f3790010460 con 0x7f37a0102830 2024-08-31T06:00:18.074 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:18.071+0000 7f3796ffd640 1 --2- 172.21.15.184:0/909338333 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f378c0b4b80 0x7f378c0b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:00:18.075 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:18.071+0000 7f3796ffd640 1 -- 172.21.15.184:0/909338333 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4036+0+0 (secure 0 0 0) 0x7f37900e3e70 con 0x7f37a0102830 2024-08-31T06:00:18.075 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:18.071+0000 7f37a568a640 1 --2- 172.21.15.184:0/909338333 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f378c0b4b80 0x7f378c0b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:00:18.075 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:18.072+0000 7f37a568a640 1 --2- 172.21.15.184:0/909338333 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f378c0b4b80 0x7f378c0b7090 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f37a0105c10 tx=0x7f379c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:00:18.076 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:18.075+0000 7f3796ffd640 1 -- 172.21.15.184:0/909338333 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f37900abfb0 con 0x7f37a0102830 2024-08-31T06:00:18.270 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:18.268+0000 7f37a80ef640 1 -- 172.21.15.184:0/909338333 --> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f37a00635a0 con 0x7f378c0b4b80 2024-08-31T06:00:18.372 INFO:journalctl@ceph.osd.4.smithi155.stdout:Aug 31 06:00:17 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-4[33944]: 2024-08-31T06:00:17.939+0000 7fc8b0e59640 -1 osd.4 0 waiting for initial osdmap 2024-08-31T06:00:18.372 INFO:journalctl@ceph.osd.4.smithi155.stdout:Aug 31 06:00:17 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-4[33944]: 2024-08-31T06:00:17.947+0000 7fc8a8c4f640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-31T06:00:18.566 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:18 smithi184 ceph-mon[22331]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-08-31T06:00:18.566 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:18 smithi184 ceph-mon[22331]: Detected new or changed devices on smithi155 2024-08-31T06:00:18.566 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:18 smithi184 ceph-mon[22331]: Adjusting osd_memory_target on smithi155 to 8544M 2024-08-31T06:00:18.566 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:18 smithi184 ceph-mon[22331]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-08-31T06:00:18.566 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:18 smithi184 ceph-mon[22331]: osdmap e31: 5 total, 4 up, 5 in 2024-08-31T06:00:18.566 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:18 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-31T06:00:18.566 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:18 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-31T06:00:18.566 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:18 smithi184 ceph-mon[22331]: from='osd.4 ' entity='osd.4' 2024-08-31T06:00:18.567 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:18 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T06:00:18.567 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:18 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T06:00:18.567 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:18 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:00:18.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:18 smithi155 ceph-mon[21742]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-08-31T06:00:18.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:18 smithi155 ceph-mon[21742]: Detected new or changed devices on smithi155 2024-08-31T06:00:18.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:18 smithi155 ceph-mon[21742]: Adjusting osd_memory_target on smithi155 to 8544M 2024-08-31T06:00:18.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:18 smithi155 ceph-mon[21742]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-08-31T06:00:18.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:18 smithi155 ceph-mon[21742]: osdmap e31: 5 total, 4 up, 5 in 2024-08-31T06:00:18.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:18 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-31T06:00:18.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:18 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-31T06:00:18.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:18 smithi155 ceph-mon[21742]: from='osd.4 ' entity='osd.4' 2024-08-31T06:00:18.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:18 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T06:00:18.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:18 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T06:00:18.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:18 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:00:18.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:18 smithi016 ceph-mon[17194]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-08-31T06:00:18.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:18 smithi016 ceph-mon[17194]: Detected new or changed devices on smithi155 2024-08-31T06:00:18.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:18 smithi016 ceph-mon[17194]: Adjusting osd_memory_target on smithi155 to 8544M 2024-08-31T06:00:18.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:18 smithi016 ceph-mon[17194]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-08-31T06:00:18.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:18 smithi016 ceph-mon[17194]: osdmap e31: 5 total, 4 up, 5 in 2024-08-31T06:00:18.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:18 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-31T06:00:18.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:18 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-31T06:00:18.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:18 smithi016 ceph-mon[17194]: from='osd.4 ' entity='osd.4' 2024-08-31T06:00:18.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:18 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T06:00:18.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:18 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T06:00:18.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:18 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:00:19.693 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:19 smithi184 ceph-mon[22331]: purged_snaps scrub starts 2024-08-31T06:00:19.693 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:19 smithi184 ceph-mon[22331]: purged_snaps scrub ok 2024-08-31T06:00:19.693 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:19 smithi184 ceph-mon[22331]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:00:19.694 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:19 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-31T06:00:19.694 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:19 smithi184 ceph-mon[22331]: osd.4 [v2:172.21.15.155:6808/3592422019,v1:172.21.15.155:6809/3592422019] boot 2024-08-31T06:00:19.694 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:19 smithi184 ceph-mon[22331]: osdmap e32: 5 total, 5 up, 5 in 2024-08-31T06:00:19.694 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:19 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-31T06:00:19.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:19 smithi155 ceph-mon[21742]: purged_snaps scrub starts 2024-08-31T06:00:19.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:19 smithi155 ceph-mon[21742]: purged_snaps scrub ok 2024-08-31T06:00:19.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:19 smithi155 ceph-mon[21742]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:00:19.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:19 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-31T06:00:19.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:19 smithi155 ceph-mon[21742]: osd.4 [v2:172.21.15.155:6808/3592422019,v1:172.21.15.155:6809/3592422019] boot 2024-08-31T06:00:19.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:19 smithi155 ceph-mon[21742]: osdmap e32: 5 total, 5 up, 5 in 2024-08-31T06:00:19.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:19 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-31T06:00:19.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:19 smithi016 ceph-mon[17194]: purged_snaps scrub starts 2024-08-31T06:00:19.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:19 smithi016 ceph-mon[17194]: purged_snaps scrub ok 2024-08-31T06:00:19.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:19 smithi016 ceph-mon[17194]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:00:19.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:19 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-31T06:00:19.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:19 smithi016 ceph-mon[17194]: osd.4 [v2:172.21.15.155:6808/3592422019,v1:172.21.15.155:6809/3592422019] boot 2024-08-31T06:00:19.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:19 smithi016 ceph-mon[17194]: osdmap e32: 5 total, 5 up, 5 in 2024-08-31T06:00:19.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:19 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-31T06:00:21.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:20 smithi184 ceph-mon[22331]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-08-31T06:00:21.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:20 smithi184 ceph-mon[22331]: osdmap e33: 5 total, 5 up, 5 in 2024-08-31T06:00:21.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:20 smithi184 ceph-mon[22331]: from='client.? 172.21.15.184:0/3968612738' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5ce6e2fa-c17c-49b3-9402-f19bfc21076d"} : dispatch 2024-08-31T06:00:21.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:20 smithi184 ceph-mon[22331]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5ce6e2fa-c17c-49b3-9402-f19bfc21076d"} : dispatch 2024-08-31T06:00:21.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:20 smithi184 ceph-mon[22331]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5ce6e2fa-c17c-49b3-9402-f19bfc21076d"}]': finished 2024-08-31T06:00:21.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:20 smithi184 ceph-mon[22331]: osdmap e34: 6 total, 5 up, 6 in 2024-08-31T06:00:21.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:20 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-31T06:00:21.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:20 smithi155 ceph-mon[21742]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-08-31T06:00:21.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:20 smithi155 ceph-mon[21742]: osdmap e33: 5 total, 5 up, 5 in 2024-08-31T06:00:21.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:20 smithi155 ceph-mon[21742]: from='client.? 172.21.15.184:0/3968612738' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5ce6e2fa-c17c-49b3-9402-f19bfc21076d"} : dispatch 2024-08-31T06:00:21.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:20 smithi155 ceph-mon[21742]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5ce6e2fa-c17c-49b3-9402-f19bfc21076d"} : dispatch 2024-08-31T06:00:21.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:20 smithi155 ceph-mon[21742]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5ce6e2fa-c17c-49b3-9402-f19bfc21076d"}]': finished 2024-08-31T06:00:21.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:20 smithi155 ceph-mon[21742]: osdmap e34: 6 total, 5 up, 6 in 2024-08-31T06:00:21.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:20 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-31T06:00:21.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:20 smithi016 ceph-mon[17194]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-08-31T06:00:21.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:20 smithi016 ceph-mon[17194]: osdmap e33: 5 total, 5 up, 5 in 2024-08-31T06:00:21.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:20 smithi016 ceph-mon[17194]: from='client.? 172.21.15.184:0/3968612738' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5ce6e2fa-c17c-49b3-9402-f19bfc21076d"} : dispatch 2024-08-31T06:00:21.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:20 smithi016 ceph-mon[17194]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5ce6e2fa-c17c-49b3-9402-f19bfc21076d"} : dispatch 2024-08-31T06:00:21.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:20 smithi016 ceph-mon[17194]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5ce6e2fa-c17c-49b3-9402-f19bfc21076d"}]': finished 2024-08-31T06:00:21.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:20 smithi016 ceph-mon[17194]: osdmap e34: 6 total, 5 up, 6 in 2024-08-31T06:00:21.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:20 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-31T06:00:22.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:21 smithi184 ceph-mon[22331]: from='client.? 172.21.15.184:0/2588153782' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T06:00:22.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:21 smithi155 ceph-mon[21742]: from='client.? 172.21.15.184:0/2588153782' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T06:00:22.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:21 smithi016 ceph-mon[17194]: from='client.? 172.21.15.184:0/2588153782' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T06:00:23.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:22 smithi184 ceph-mon[22331]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-08-31T06:00:23.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:22 smithi155 ceph-mon[21742]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-08-31T06:00:23.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:22 smithi016 ceph-mon[17194]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-08-31T06:00:25.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:24 smithi184 ceph-mon[22331]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-08-31T06:00:25.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:24 smithi155 ceph-mon[21742]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-08-31T06:00:25.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:24 smithi016 ceph-mon[17194]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-08-31T06:00:27.215 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:26 smithi016 ceph-mon[17194]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-08-31T06:00:27.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:26 smithi184 ceph-mon[22331]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-08-31T06:00:27.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:26 smithi155 ceph-mon[21742]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-08-31T06:00:29.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:28 smithi016 ceph-mon[17194]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-08-31T06:00:29.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:28 smithi184 ceph-mon[22331]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-08-31T06:00:29.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:28 smithi155 ceph-mon[21742]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-08-31T06:00:30.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:29 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-08-31T06:00:30.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:29 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:00:30.263 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:29 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-08-31T06:00:30.263 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:29 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:00:30.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:29 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-08-31T06:00:30.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:29 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:00:31.070 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:30 smithi184 ceph-mon[22331]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-08-31T06:00:31.070 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:30 smithi184 ceph-mon[22331]: Deploying daemon osd.5 on smithi184 2024-08-31T06:00:31.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:30 smithi155 ceph-mon[21742]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-08-31T06:00:31.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:30 smithi155 ceph-mon[21742]: Deploying daemon osd.5 on smithi184 2024-08-31T06:00:31.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:30 smithi016 ceph-mon[17194]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-08-31T06:00:31.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:30 smithi016 ceph-mon[17194]: Deploying daemon osd.5 on smithi184 2024-08-31T06:00:33.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:32 smithi155 ceph-mon[21742]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-08-31T06:00:33.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:32 smithi016 ceph-mon[17194]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-08-31T06:00:33.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:32 smithi184 ceph-mon[22331]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-08-31T06:00:35.307 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:34 smithi184 ceph-mon[22331]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-08-31T06:00:35.307 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:34 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:00:35.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:34 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:35.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:34 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:35.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:34 smithi155 ceph-mon[21742]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-08-31T06:00:35.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:34 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:00:35.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:34 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:35.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:34 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:35.418 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:34 smithi016 ceph-mon[17194]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-08-31T06:00:35.418 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:34 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:00:35.418 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:34 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:35.418 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:34 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:37.231 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:36 smithi184 ceph-mon[22331]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-08-31T06:00:37.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:36 smithi155 ceph-mon[21742]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-08-31T06:00:37.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:36 smithi016 ceph-mon[17194]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-08-31T06:00:38.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:38 smithi184 ceph-mon[22331]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-08-31T06:00:38.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:38 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:38.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:38 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:38.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:38 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:00:38.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:38 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:00:38.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:38 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:38.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:38 smithi155 ceph-mon[21742]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-08-31T06:00:38.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:38 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:38.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:38 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:38.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:38 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:00:38.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:38 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:00:38.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:38 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:38.881 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:38.879+0000 7f3796ffd640 1 -- 172.21.15.184:0/909338333 <== mgr.14220 v2:172.21.15.16:6800/4279751337 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f37a00635a0 con 0x7f378c0b4b80 2024-08-31T06:00:38.882 INFO:teuthology.orchestra.run.smithi184.stdout:Created osd(s) 5 on host 'smithi184' 2024-08-31T06:00:38.884 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:38.882+0000 7f37a80ef640 1 -- 172.21.15.184:0/909338333 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f378c0b4b80 msgr2=0x7f378c0b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:00:38.884 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:38.882+0000 7f37a80ef640 1 --2- 172.21.15.184:0/909338333 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f378c0b4b80 0x7f378c0b7090 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f37a0105c10 tx=0x7f379c005f90 comp rx=0 tx=0).stop 2024-08-31T06:00:38.884 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:38.882+0000 7f37a80ef640 1 -- 172.21.15.184:0/909338333 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37a0102830 msgr2=0x7f37a010fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:00:38.884 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:38.882+0000 7f37a80ef640 1 --2- 172.21.15.184:0/909338333 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37a0102830 0x7f37a010fdd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f379000d950 tx=0x7f379000de10 comp rx=0 tx=0).stop 2024-08-31T06:00:38.885 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:38.882+0000 7f37a80ef640 1 -- 172.21.15.184:0/909338333 shutdown_connections 2024-08-31T06:00:38.885 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:38.882+0000 7f37a80ef640 1 --2- 172.21.15.184:0/909338333 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f378c0b4b80 0x7f378c0b7090 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:00:38.885 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:38.882+0000 7f37a80ef640 1 --2- 172.21.15.184:0/909338333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a0108ac0 0x7f37a0110850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:00:38.885 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:38.882+0000 7f37a80ef640 1 --2- 172.21.15.184:0/909338333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37a0108170 0x7f37a0110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:00:38.885 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:38.882+0000 7f37a80ef640 1 --2- 172.21.15.184:0/909338333 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37a0102830 0x7f37a010fdd0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:00:38.885 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:38.882+0000 7f37a80ef640 1 -- 172.21.15.184:0/909338333 >> 172.21.15.184:0/909338333 conn(0x7f37a00794d0 msgr2=0x7f37a0104140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:00:38.885 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:38.882+0000 7f37a80ef640 1 -- 172.21.15.184:0/909338333 shutdown_connections 2024-08-31T06:00:38.885 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:00:38.882+0000 7f37a80ef640 1 -- 172.21.15.184:0/909338333 wait complete. 2024-08-31T06:00:38.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:38 smithi016 ceph-mon[17194]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-08-31T06:00:38.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:38 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:38.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:38 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:38.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:38 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:00:38.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:38 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:00:38.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:38 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:39.507 DEBUG:teuthology.orchestra.run.smithi184:osd.5> sudo journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@osd.5.service 2024-08-31T06:00:39.509 INFO:tasks.cephadm:Deploying osd.6 on smithi016 with /dev/vg_nvme/lv_2... 2024-08-31T06:00:39.509 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-08-31T06:00:39.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:39 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:00:39.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:39 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:39.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:39 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:39.834 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:39 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:00:39.834 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:39 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:39.834 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:39 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:39.871 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:39 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:00:39.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:39 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:39.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:39 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:39.898 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:00:41.014 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:40 smithi184 ceph-mon[22331]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-08-31T06:00:41.014 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:40 smithi184 ceph-mon[22331]: from='osd.5 [v2:172.21.15.184:6808/2054894065,v1:172.21.15.184:6809/2054894065]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-08-31T06:00:41.014 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:40 smithi184 ceph-mon[22331]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-08-31T06:00:41.015 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:00:40 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5[34971]: 2024-08-31T06:00:40.769+0000 7f8af9fb6640 -1 osd.5 0 log_to_monitors true 2024-08-31T06:00:41.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:40 smithi016 ceph-mon[17194]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-08-31T06:00:41.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:40 smithi016 ceph-mon[17194]: from='osd.5 [v2:172.21.15.184:6808/2054894065,v1:172.21.15.184:6809/2054894065]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-08-31T06:00:41.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:40 smithi016 ceph-mon[17194]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-08-31T06:00:41.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:40 smithi155 ceph-mon[21742]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-08-31T06:00:41.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:40 smithi155 ceph-mon[21742]: from='osd.5 [v2:172.21.15.184:6808/2054894065,v1:172.21.15.184:6809/2054894065]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-08-31T06:00:41.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:40 smithi155 ceph-mon[21742]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-08-31T06:00:42.062 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:00:41 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5[34971]: 2024-08-31T06:00:41.895+0000 7f8af5f38640 -1 osd.5 0 waiting for initial osdmap 2024-08-31T06:00:42.062 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:00:41 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5[34971]: 2024-08-31T06:00:41.901+0000 7f8af1d36640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-31T06:00:42.062 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:41 smithi184 ceph-mon[22331]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-08-31T06:00:42.062 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:41 smithi184 ceph-mon[22331]: osdmap e35: 6 total, 5 up, 6 in 2024-08-31T06:00:42.062 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:41 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-31T06:00:42.063 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:41 smithi184 ceph-mon[22331]: from='osd.5 [v2:172.21.15.184:6808/2054894065,v1:172.21.15.184:6809/2054894065]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]} : dispatch 2024-08-31T06:00:42.063 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:41 smithi184 ceph-mon[22331]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]} : dispatch 2024-08-31T06:00:42.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:41 smithi016 ceph-mon[17194]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-08-31T06:00:42.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:41 smithi016 ceph-mon[17194]: osdmap e35: 6 total, 5 up, 6 in 2024-08-31T06:00:42.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:41 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-31T06:00:42.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:41 smithi016 ceph-mon[17194]: from='osd.5 [v2:172.21.15.184:6808/2054894065,v1:172.21.15.184:6809/2054894065]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]} : dispatch 2024-08-31T06:00:42.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:41 smithi016 ceph-mon[17194]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]} : dispatch 2024-08-31T06:00:42.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:41 smithi155 ceph-mon[21742]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-08-31T06:00:42.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:41 smithi155 ceph-mon[21742]: osdmap e35: 6 total, 5 up, 6 in 2024-08-31T06:00:42.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:41 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-31T06:00:42.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:41 smithi155 ceph-mon[21742]: from='osd.5 [v2:172.21.15.184:6808/2054894065,v1:172.21.15.184:6809/2054894065]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]} : dispatch 2024-08-31T06:00:42.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:41 smithi155 ceph-mon[21742]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]} : dispatch 2024-08-31T06:00:43.142 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:42 smithi016 ceph-mon[17194]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-08-31T06:00:43.142 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:42 smithi016 ceph-mon[17194]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-08-31T06:00:43.142 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:42 smithi016 ceph-mon[17194]: osdmap e36: 6 total, 5 up, 6 in 2024-08-31T06:00:43.142 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:42 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-31T06:00:43.142 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:42 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:43.142 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:42 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:43.142 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:42 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T06:00:43.142 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:42 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T06:00:43.142 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:42 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-31T06:00:43.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:42 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-31T06:00:43.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:42 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:43.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:42 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:00:43.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:42 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:00:43.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:42 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:43.148 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:00:43.179 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph orch daemon add osd smithi016:vg_nvme/lv_2 2024-08-31T06:00:43.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:42 smithi184 ceph-mon[22331]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-08-31T06:00:43.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:42 smithi184 ceph-mon[22331]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-08-31T06:00:43.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:42 smithi184 ceph-mon[22331]: osdmap e36: 6 total, 5 up, 6 in 2024-08-31T06:00:43.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:42 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-31T06:00:43.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:42 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:43.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:42 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:43.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:42 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T06:00:43.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:42 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T06:00:43.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:42 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-31T06:00:43.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:42 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-31T06:00:43.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:42 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:43.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:42 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:00:43.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:42 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:00:43.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:42 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:43.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:42 smithi155 ceph-mon[21742]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-08-31T06:00:43.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:42 smithi155 ceph-mon[21742]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-08-31T06:00:43.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:42 smithi155 ceph-mon[21742]: osdmap e36: 6 total, 5 up, 6 in 2024-08-31T06:00:43.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:42 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-31T06:00:43.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:42 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:43.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:42 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:43.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:42 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T06:00:43.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:42 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T06:00:43.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:42 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-31T06:00:43.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:42 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-31T06:00:43.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:42 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:43.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:42 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:00:43.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:42 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:00:43.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:42 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:00:43.550 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:00:44.120 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:43 smithi016 ceph-mon[17194]: purged_snaps scrub starts 2024-08-31T06:00:44.120 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:43 smithi016 ceph-mon[17194]: purged_snaps scrub ok 2024-08-31T06:00:44.120 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:43 smithi016 ceph-mon[17194]: Detected new or changed devices on smithi184 2024-08-31T06:00:44.120 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:43 smithi016 ceph-mon[17194]: Adjusting osd_memory_target on smithi184 to 8544M 2024-08-31T06:00:44.120 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:43 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-31T06:00:44.120 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:43 smithi016 ceph-mon[17194]: osd.5 [v2:172.21.15.184:6808/2054894065,v1:172.21.15.184:6809/2054894065] boot 2024-08-31T06:00:44.120 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:43 smithi016 ceph-mon[17194]: osdmap e37: 6 total, 6 up, 6 in 2024-08-31T06:00:44.120 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:43 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-31T06:00:44.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:43 smithi184 ceph-mon[22331]: purged_snaps scrub starts 2024-08-31T06:00:44.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:43 smithi184 ceph-mon[22331]: purged_snaps scrub ok 2024-08-31T06:00:44.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:43 smithi184 ceph-mon[22331]: Detected new or changed devices on smithi184 2024-08-31T06:00:44.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:43 smithi184 ceph-mon[22331]: Adjusting osd_memory_target on smithi184 to 8544M 2024-08-31T06:00:44.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:43 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-31T06:00:44.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:43 smithi184 ceph-mon[22331]: osd.5 [v2:172.21.15.184:6808/2054894065,v1:172.21.15.184:6809/2054894065] boot 2024-08-31T06:00:44.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:43 smithi184 ceph-mon[22331]: osdmap e37: 6 total, 6 up, 6 in 2024-08-31T06:00:44.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:43 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-31T06:00:44.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:00:44.299+0000 7f1a2a400640 1 -- 172.21.15.16:0/1024444570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a2410b290 msgr2=0x7f1a2410b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:00:44.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:00:44.299+0000 7f1a2a400640 1 --2- 172.21.15.16:0/1024444570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a2410b290 0x7f1a2410b6a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1a10009860 tx=0x7f1a1002a340 comp rx=0 tx=0).stop 2024-08-31T06:00:44.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:00:44.299+0000 7f1a2a400640 1 -- 172.21.15.16:0/1024444570 shutdown_connections 2024-08-31T06:00:44.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:00:44.299+0000 7f1a2a400640 1 --2- 172.21.15.16:0/1024444570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a2410bbe0 0x7f1a24112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:00:44.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:00:44.299+0000 7f1a2a400640 1 --2- 172.21.15.16:0/1024444570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a2410b290 0x7f1a2410b6a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:00:44.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:00:44.299+0000 7f1a2a400640 1 --2- 172.21.15.16:0/1024444570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a24105950 0x7f1a24105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:00:44.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:00:44.299+0000 7f1a2a400640 1 -- 172.21.15.16:0/1024444570 >> 172.21.15.16:0/1024444570 conn(0x7f1a24101420 msgr2=0x7f1a24103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:00:44.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:00:44.299+0000 7f1a2a400640 1 -- 172.21.15.16:0/1024444570 shutdown_connections 2024-08-31T06:00:44.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:00:44.299+0000 7f1a2a400640 1 -- 172.21.15.16:0/1024444570 wait complete. 2024-08-31T06:00:44.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:00:44.300+0000 7f1a2a400640 1 Processor -- start 2024-08-31T06:00:44.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:00:44.301+0000 7f1a2a400640 1 -- start start 2024-08-31T06:00:44.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:00:44.301+0000 7f1a2a400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a24105950 0x7f1a2419c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:00:44.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:00:44.301+0000 7f1a2a400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a2410b290 0x7f1a2419c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:00:44.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:00:44.301+0000 7f1a2a400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a2410bbe0 0x7f1a241a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:00:44.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:00:44.301+0000 7f1a2a400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1a241158a0 con 0x7f1a2410bbe0 2024-08-31T06:00:44.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:00:44.301+0000 7f1a2a400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1a24115720 con 0x7f1a2410b290 2024-08-31T06:00:44.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:00:44.301+0000 7f1a2a400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1a24115a20 con 0x7f1a24105950 2024-08-31T06:00:44.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:00:44.301+0000 7f1a29a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a2410bbe0 0x7f1a241a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:00:44.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:00:44.301+0000 7f1a29a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a2410bbe0 0x7f1a241a1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57924/0 (socket says 172.21.15.16:57924) 2024-08-31T06:00:44.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:00:44.301+0000 7f1a29a00640 1 -- 172.21.15.16:0/1733723173 learned_addr learned my addr 172.21.15.16:0/1733723173 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:00:44.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:00:44.302+0000 7f1a23e00640 1 --2- 172.21.15.16:0/1733723173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a2410b290 0x7f1a2419c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:00:44.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:00:44.302+0000 7f1a29000640 1 --2- 172.21.15.16:0/1733723173 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a24105950 0x7f1a2419c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:00:44.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:00:44.302+0000 7f1a29a00640 1 -- 172.21.15.16:0/1733723173 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a24105950 msgr2=0x7f1a2419c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:00:44.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:00:44.302+0000 7f1a29a00640 1 --2- 172.21.15.16:0/1733723173 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a24105950 0x7f1a2419c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:00:44.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:00:44.302+0000 7f1a29a00640 1 -- 172.21.15.16:0/1733723173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a2410b290 msgr2=0x7f1a2419c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:00:44.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:00:44.302+0000 7f1a29a00640 1 --2- 172.21.15.16:0/1733723173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a2410b290 0x7f1a2419c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:00:44.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:00:44.302+0000 7f1a29a00640 1 -- 172.21.15.16:0/1733723173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a241a2220 con 0x7f1a2410bbe0 2024-08-31T06:00:44.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:00:44.302+0000 7f1a29a00640 1 --2- 172.21.15.16:0/1733723173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a2410bbe0 0x7f1a241a1af0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c00eb10 tx=0x7f1a1c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:00:44.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:00:44.302+0000 7f1a21600640 1 -- 172.21.15.16:0/1733723173 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c004b20 con 0x7f1a2410bbe0 2024-08-31T06:00:44.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:00:44.303+0000 7f1a2a400640 1 -- 172.21.15.16:0/1733723173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a241a2510 con 0x7f1a2410bbe0 2024-08-31T06:00:44.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:00:44.303+0000 7f1a21600640 1 -- 172.21.15.16:0/1733723173 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a1c004cc0 con 0x7f1a2410bbe0 2024-08-31T06:00:44.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:00:44.303+0000 7f1a21600640 1 -- 172.21.15.16:0/1733723173 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c010670 con 0x7f1a2410bbe0 2024-08-31T06:00:44.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:00:44.303+0000 7f1a2a400640 1 -- 172.21.15.16:0/1733723173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a241a8c70 con 0x7f1a2410bbe0 2024-08-31T06:00:44.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:00:44.304+0000 7f1a2a400640 1 -- 172.21.15.16:0/1733723173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a24106bd0 con 0x7f1a2410bbe0 2024-08-31T06:00:44.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:00:44.306+0000 7f1a21600640 1 -- 172.21.15.16:0/1733723173 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7f1a1c005020 con 0x7f1a2410bbe0 2024-08-31T06:00:44.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:00:44.307+0000 7f1a21600640 1 --2- 172.21.15.16:0/1733723173 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f1a0c0b4b80 0x7f1a0c0b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:00:44.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:00:44.308+0000 7f1a21600640 1 -- 172.21.15.16:0/1733723173 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4748+0+0 (secure 0 0 0) 0x7f1a1c0e4a30 con 0x7f1a2410bbe0 2024-08-31T06:00:44.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:00:44.308+0000 7f1a29000640 1 --2- 172.21.15.16:0/1733723173 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f1a0c0b4b80 0x7f1a0c0b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:00:44.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:00:44.308+0000 7f1a29000640 1 --2- 172.21.15.16:0/1733723173 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f1a0c0b4b80 0x7f1a0c0b7090 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1a18009530 tx=0x7f1a18009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:00:44.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:00:44.312+0000 7f1a21600640 1 -- 172.21.15.16:0/1733723173 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f1a1c0ac8b0 con 0x7f1a2410bbe0 2024-08-31T06:00:44.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:43 smithi155 ceph-mon[21742]: purged_snaps scrub starts 2024-08-31T06:00:44.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:43 smithi155 ceph-mon[21742]: purged_snaps scrub ok 2024-08-31T06:00:44.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:43 smithi155 ceph-mon[21742]: Detected new or changed devices on smithi184 2024-08-31T06:00:44.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:43 smithi155 ceph-mon[21742]: Adjusting osd_memory_target on smithi184 to 8544M 2024-08-31T06:00:44.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:43 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-31T06:00:44.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:43 smithi155 ceph-mon[21742]: osd.5 [v2:172.21.15.184:6808/2054894065,v1:172.21.15.184:6809/2054894065] boot 2024-08-31T06:00:44.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:43 smithi155 ceph-mon[21742]: osdmap e37: 6 total, 6 up, 6 in 2024-08-31T06:00:44.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:43 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-31T06:00:44.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:00:44.517+0000 7f1a2a400640 1 -- 172.21.15.16:0/1733723173 --> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f1a240635a0 con 0x7f1a0c0b4b80 2024-08-31T06:00:45.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:44 smithi016 ceph-mon[17194]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-08-31T06:00:45.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:44 smithi016 ceph-mon[17194]: osdmap e38: 6 total, 6 up, 6 in 2024-08-31T06:00:45.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:44 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T06:00:45.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:44 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T06:00:45.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:44 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:00:45.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:44 smithi184 ceph-mon[22331]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-08-31T06:00:45.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:44 smithi184 ceph-mon[22331]: osdmap e38: 6 total, 6 up, 6 in 2024-08-31T06:00:45.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:44 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T06:00:45.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:44 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T06:00:45.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:44 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:00:45.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:44 smithi155 ceph-mon[21742]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-08-31T06:00:45.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:44 smithi155 ceph-mon[21742]: osdmap e38: 6 total, 6 up, 6 in 2024-08-31T06:00:45.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:44 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T06:00:45.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:44 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T06:00:45.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:44 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:00:46.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:45 smithi016 ceph-mon[17194]: from='client.14514 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:00:46.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:45 smithi016 ceph-mon[17194]: osdmap e39: 6 total, 6 up, 6 in 2024-08-31T06:00:46.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:45 smithi184 ceph-mon[22331]: from='client.14514 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:00:46.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:45 smithi184 ceph-mon[22331]: osdmap e39: 6 total, 6 up, 6 in 2024-08-31T06:00:46.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:45 smithi155 ceph-mon[21742]: from='client.14514 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:00:46.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:45 smithi155 ceph-mon[21742]: osdmap e39: 6 total, 6 up, 6 in 2024-08-31T06:00:47.149 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:46 smithi016 ceph-mon[17194]: pgmap v128: 1 pgs: 1 activating+degraded; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-08-31T06:00:47.149 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:46 smithi016 ceph-mon[17194]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-08-31T06:00:47.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:46 smithi184 ceph-mon[22331]: pgmap v128: 1 pgs: 1 activating+degraded; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-08-31T06:00:47.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:46 smithi184 ceph-mon[22331]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-08-31T06:00:47.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:46 smithi155 ceph-mon[21742]: pgmap v128: 1 pgs: 1 activating+degraded; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-08-31T06:00:47.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:46 smithi155 ceph-mon[21742]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-08-31T06:00:48.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:47 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/3181913672' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f257d8d1-ad59-4bd8-821d-bf29e50b547e"} : dispatch 2024-08-31T06:00:48.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:47 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/3181913672' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f257d8d1-ad59-4bd8-821d-bf29e50b547e"}]': finished 2024-08-31T06:00:48.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:47 smithi184 ceph-mon[22331]: osdmap e40: 7 total, 6 up, 7 in 2024-08-31T06:00:48.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:47 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-31T06:00:48.255 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:47 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/3181913672' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f257d8d1-ad59-4bd8-821d-bf29e50b547e"} : dispatch 2024-08-31T06:00:48.256 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:47 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/3181913672' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f257d8d1-ad59-4bd8-821d-bf29e50b547e"}]': finished 2024-08-31T06:00:48.256 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:47 smithi016 ceph-mon[17194]: osdmap e40: 7 total, 6 up, 7 in 2024-08-31T06:00:48.256 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:47 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-31T06:00:48.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:47 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/3181913672' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f257d8d1-ad59-4bd8-821d-bf29e50b547e"} : dispatch 2024-08-31T06:00:48.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:47 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/3181913672' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f257d8d1-ad59-4bd8-821d-bf29e50b547e"}]': finished 2024-08-31T06:00:48.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:47 smithi155 ceph-mon[21742]: osdmap e40: 7 total, 6 up, 7 in 2024-08-31T06:00:48.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:47 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-31T06:00:49.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:48 smithi184 ceph-mon[22331]: pgmap v129: 1 pgs: 1 activating+degraded; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-08-31T06:00:49.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:48 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/3938392766' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T06:00:49.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:48 smithi155 ceph-mon[21742]: pgmap v129: 1 pgs: 1 activating+degraded; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-08-31T06:00:49.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:48 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/3938392766' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T06:00:49.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:48 smithi016 ceph-mon[17194]: pgmap v129: 1 pgs: 1 activating+degraded; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-08-31T06:00:49.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:48 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/3938392766' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T06:00:51.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:50 smithi184 ceph-mon[22331]: pgmap v131: 1 pgs: 1 activating+degraded; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-08-31T06:00:51.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:50 smithi155 ceph-mon[21742]: pgmap v131: 1 pgs: 1 activating+degraded; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-08-31T06:00:51.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:50 smithi016 ceph-mon[17194]: pgmap v131: 1 pgs: 1 activating+degraded; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-08-31T06:00:52.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:51 smithi155 ceph-mon[21742]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-08-31T06:00:52.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:51 smithi155 ceph-mon[21742]: Cluster is now healthy 2024-08-31T06:00:52.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:51 smithi016 ceph-mon[17194]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-08-31T06:00:52.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:51 smithi016 ceph-mon[17194]: Cluster is now healthy 2024-08-31T06:00:52.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:51 smithi184 ceph-mon[22331]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-08-31T06:00:52.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:51 smithi184 ceph-mon[22331]: Cluster is now healthy 2024-08-31T06:00:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:52 smithi016 ceph-mon[17194]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-08-31T06:00:53.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:52 smithi155 ceph-mon[21742]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-08-31T06:00:53.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:52 smithi184 ceph-mon[22331]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-08-31T06:00:55.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:54 smithi016 ceph-mon[17194]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-08-31T06:00:55.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:54 smithi155 ceph-mon[21742]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-08-31T06:00:55.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:54 smithi184 ceph-mon[22331]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-08-31T06:00:57.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:56 smithi155 ceph-mon[21742]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-31T06:00:57.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:56 smithi016 ceph-mon[17194]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-31T06:00:57.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:56 smithi184 ceph-mon[22331]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-31T06:00:58.284 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:57 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-08-31T06:00:58.285 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:57 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:00:58.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:57 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-08-31T06:00:58.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:57 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:00:58.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:57 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-08-31T06:00:58.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:57 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:00:59.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:58 smithi155 ceph-mon[21742]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-31T06:00:59.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:00:58 smithi155 ceph-mon[21742]: Deploying daemon osd.6 on smithi016 2024-08-31T06:00:59.373 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:59 smithi184 ceph-mon[22331]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-31T06:00:59.373 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:00:59 smithi184 ceph-mon[22331]: Deploying daemon osd.6 on smithi016 2024-08-31T06:00:59.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:58 smithi016 ceph-mon[17194]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-31T06:00:59.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:00:58 smithi016 ceph-mon[17194]: Deploying daemon osd.6 on smithi016 2024-08-31T06:01:01.237 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:01 smithi016 ceph-mon[17194]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-08-31T06:01:01.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:01 smithi155 ceph-mon[21742]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-08-31T06:01:01.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:01 smithi184 ceph-mon[22331]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-08-31T06:01:03.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:03 smithi155 ceph-mon[21742]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-31T06:01:03.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:03 smithi016 ceph-mon[17194]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-31T06:01:03.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:03 smithi184 ceph-mon[22331]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-31T06:01:05.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:05 smithi155 ceph-mon[21742]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-08-31T06:01:05.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:05 smithi016 ceph-mon[17194]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-08-31T06:01:05.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:05 smithi184 ceph-mon[22331]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-08-31T06:01:06.083 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:06 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:01:06.084 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:06 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:06.084 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:06 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:06.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:06 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:01:06.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:06 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:06.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:06 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:06.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:06 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:01:06.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:06 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:06.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:06 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:07.273 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:07 smithi155 ceph-mon[21742]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-08-31T06:01:07.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:07 smithi016 ceph-mon[17194]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-08-31T06:01:07.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:07 smithi184 ceph-mon[22331]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-08-31T06:01:09.336 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:09 smithi016 ceph-mon[17194]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-08-31T06:01:09.337 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:09 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:09.337 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:09 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:09.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:09 smithi155 ceph-mon[21742]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-08-31T06:01:09.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:09 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:09.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:09 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:09.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:09 smithi184 ceph-mon[22331]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-08-31T06:01:09.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:09 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:09.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:09 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:10.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:10 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:01:10.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:10 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:01:10.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:10 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:10.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:10 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:01:10.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:10 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:01:10.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:10 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:10.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:10 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:01:10.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:10 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:01:10.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:10 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:10.473 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 6 on host 'smithi016' 2024-08-31T06:01:10.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:01:10.470+0000 7f1a21600640 1 -- 172.21.15.16:0/1733723173 <== mgr.14220 v2:172.21.15.16:6800/4279751337 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1a240635a0 con 0x7f1a0c0b4b80 2024-08-31T06:01:10.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:01:10.473+0000 7f1a2a400640 1 -- 172.21.15.16:0/1733723173 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f1a0c0b4b80 msgr2=0x7f1a0c0b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:01:10.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:01:10.473+0000 7f1a2a400640 1 --2- 172.21.15.16:0/1733723173 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f1a0c0b4b80 0x7f1a0c0b7090 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1a18009530 tx=0x7f1a18009260 comp rx=0 tx=0).stop 2024-08-31T06:01:10.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:01:10.473+0000 7f1a2a400640 1 -- 172.21.15.16:0/1733723173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a2410bbe0 msgr2=0x7f1a241a1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:01:10.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:01:10.473+0000 7f1a2a400640 1 --2- 172.21.15.16:0/1733723173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a2410bbe0 0x7f1a241a1af0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c00eb10 tx=0x7f1a1c00c2d0 comp rx=0 tx=0).stop 2024-08-31T06:01:10.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:01:10.473+0000 7f1a2a400640 1 -- 172.21.15.16:0/1733723173 shutdown_connections 2024-08-31T06:01:10.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:01:10.473+0000 7f1a2a400640 1 --2- 172.21.15.16:0/1733723173 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f1a0c0b4b80 0x7f1a0c0b7090 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:01:10.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:01:10.473+0000 7f1a2a400640 1 --2- 172.21.15.16:0/1733723173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a2410bbe0 0x7f1a241a1af0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:01:10.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:01:10.473+0000 7f1a2a400640 1 --2- 172.21.15.16:0/1733723173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a2410b290 0x7f1a2419c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:01:10.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:01:10.473+0000 7f1a2a400640 1 --2- 172.21.15.16:0/1733723173 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a24105950 0x7f1a2419c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:01:10.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:01:10.473+0000 7f1a2a400640 1 -- 172.21.15.16:0/1733723173 >> 172.21.15.16:0/1733723173 conn(0x7f1a24101420 msgr2=0x7f1a24103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:01:10.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:01:10.473+0000 7f1a2a400640 1 -- 172.21.15.16:0/1733723173 shutdown_connections 2024-08-31T06:01:10.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:01:10.473+0000 7f1a2a400640 1 -- 172.21.15.16:0/1733723173 wait complete. 2024-08-31T06:01:11.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:11 smithi016 ceph-mon[17194]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-08-31T06:01:11.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:11 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:01:11.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:11 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:11.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:11 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:11.293 DEBUG:teuthology.orchestra.run.smithi016:osd.6> sudo journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@osd.6.service 2024-08-31T06:01:11.297 INFO:tasks.cephadm:Deploying osd.7 on smithi155 with /dev/vg_nvme/lv_2... 2024-08-31T06:01:11.298 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-08-31T06:01:11.326 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:11 smithi155 ceph-mon[21742]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-08-31T06:01:11.326 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:11 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:01:11.326 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:11 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:11.326 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:11 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:11.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:11 smithi184 ceph-mon[22331]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-08-31T06:01:11.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:11 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:01:11.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:11 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:11.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:11 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:11.556 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.b/config 2024-08-31T06:01:12.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:12 smithi016 ceph-mon[17194]: from='osd.6 [v2:172.21.15.16:6818/1697421092,v1:172.21.15.16:6819/1697421092]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-08-31T06:01:12.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:12 smithi016 ceph-mon[17194]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-08-31T06:01:12.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:12 smithi155 ceph-mon[21742]: from='osd.6 [v2:172.21.15.16:6818/1697421092,v1:172.21.15.16:6819/1697421092]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-08-31T06:01:12.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:12 smithi155 ceph-mon[21742]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-08-31T06:01:12.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:12 smithi184 ceph-mon[22331]: from='osd.6 [v2:172.21.15.16:6818/1697421092,v1:172.21.15.16:6819/1697421092]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-08-31T06:01:12.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:12 smithi184 ceph-mon[22331]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-08-31T06:01:13.234 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:13 smithi016 ceph-mon[17194]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-08-31T06:01:13.234 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:13 smithi016 ceph-mon[17194]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-08-31T06:01:13.234 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:13 smithi016 ceph-mon[17194]: osdmap e41: 7 total, 6 up, 7 in 2024-08-31T06:01:13.234 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:13 smithi016 ceph-mon[17194]: from='osd.6 [v2:172.21.15.16:6818/1697421092,v1:172.21.15.16:6819/1697421092]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-08-31T06:01:13.234 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:13 smithi016 ceph-mon[17194]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-08-31T06:01:13.234 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:13 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-31T06:01:13.234 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:01:13 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:01:13.039+0000 7f41a0200640 -1 osd.6 0 waiting for initial osdmap 2024-08-31T06:01:13.235 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:01:13 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:01:13.056+0000 7f419b200640 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-31T06:01:13.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:13 smithi155 ceph-mon[21742]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-08-31T06:01:13.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:13 smithi155 ceph-mon[21742]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-08-31T06:01:13.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:13 smithi155 ceph-mon[21742]: osdmap e41: 7 total, 6 up, 7 in 2024-08-31T06:01:13.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:13 smithi155 ceph-mon[21742]: from='osd.6 [v2:172.21.15.16:6818/1697421092,v1:172.21.15.16:6819/1697421092]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-08-31T06:01:13.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:13 smithi155 ceph-mon[21742]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-08-31T06:01:13.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:13 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-31T06:01:13.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:13 smithi184 ceph-mon[22331]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-08-31T06:01:13.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:13 smithi184 ceph-mon[22331]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-08-31T06:01:13.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:13 smithi184 ceph-mon[22331]: osdmap e41: 7 total, 6 up, 7 in 2024-08-31T06:01:13.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:13 smithi184 ceph-mon[22331]: from='osd.6 [v2:172.21.15.16:6818/1697421092,v1:172.21.15.16:6819/1697421092]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-08-31T06:01:13.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:13 smithi184 ceph-mon[22331]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-08-31T06:01:13.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:13 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-31T06:01:14.137 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-08-31T06:01:14.165 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph orch daemon add osd smithi155:vg_nvme/lv_2 2024-08-31T06:01:14.299 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:14 smithi016 ceph-mon[17194]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-08-31T06:01:14.299 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:14 smithi016 ceph-mon[17194]: osdmap e42: 7 total, 6 up, 7 in 2024-08-31T06:01:14.299 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:14 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-31T06:01:14.299 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:14 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-31T06:01:14.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:14 smithi155 ceph-mon[21742]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-08-31T06:01:14.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:14 smithi155 ceph-mon[21742]: osdmap e42: 7 total, 6 up, 7 in 2024-08-31T06:01:14.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:14 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-31T06:01:14.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:14 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-31T06:01:14.454 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.b/config 2024-08-31T06:01:14.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:14 smithi184 ceph-mon[22331]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-08-31T06:01:14.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:14 smithi184 ceph-mon[22331]: osdmap e42: 7 total, 6 up, 7 in 2024-08-31T06:01:14.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:14 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-31T06:01:14.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:14 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-31T06:01:14.963 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:14.954+0000 7fb8ab007640 1 -- 172.21.15.155:0/3156520909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8a4108a90 msgr2=0x7fb8a410f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:01:14.964 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:14.954+0000 7fb8ab007640 1 --2- 172.21.15.155:0/3156520909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8a4108a90 0x7fb8a410f2c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb89c009ca0 tx=0x7fb89c02a8c0 comp rx=0 tx=0).stop 2024-08-31T06:01:14.964 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:14.954+0000 7fb8ab007640 1 -- 172.21.15.155:0/3156520909 shutdown_connections 2024-08-31T06:01:14.964 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:14.954+0000 7fb8ab007640 1 --2- 172.21.15.155:0/3156520909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8a4108140 0x7fb8a4108550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:01:14.964 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:14.954+0000 7fb8ab007640 1 --2- 172.21.15.155:0/3156520909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8a4108a90 0x7fb8a410f2c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:01:14.964 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:14.954+0000 7fb8ab007640 1 --2- 172.21.15.155:0/3156520909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8a4102800 0x7fb8a4102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:01:14.964 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:14.954+0000 7fb8ab007640 1 -- 172.21.15.155:0/3156520909 >> 172.21.15.155:0/3156520909 conn(0x7fb8a40794d0 msgr2=0x7fb8a40798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:01:14.964 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:14.954+0000 7fb8ab007640 1 -- 172.21.15.155:0/3156520909 shutdown_connections 2024-08-31T06:01:14.964 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:14.954+0000 7fb8ab007640 1 -- 172.21.15.155:0/3156520909 wait complete. 2024-08-31T06:01:14.964 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:14.955+0000 7fb8ab007640 1 Processor -- start 2024-08-31T06:01:14.964 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:14.955+0000 7fb8ab007640 1 -- start start 2024-08-31T06:01:14.964 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:14.955+0000 7fb8ab007640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8a4102800 0x7fb8a41990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:01:14.964 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:14.955+0000 7fb8ab007640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8a4108140 0x7fb8a4199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:01:14.965 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:14.955+0000 7fb8ab007640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8a4108a90 0x7fb8a419e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:01:14.965 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:14.955+0000 7fb8ab007640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a4112710 con 0x7fb8a4102800 2024-08-31T06:01:14.965 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:14.955+0000 7fb8ab007640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a4112590 con 0x7fb8a4108a90 2024-08-31T06:01:14.965 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:14.955+0000 7fb8ab007640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a4112890 con 0x7fb8a4108140 2024-08-31T06:01:14.965 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:14.955+0000 7fb8a95a4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8a4108a90 0x7fb8a419e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:01:14.965 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:14.955+0000 7fb8a95a4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8a4108a90 0x7fb8a419e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:38098/0 (socket says 172.21.15.155:38098) 2024-08-31T06:01:14.965 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:14.955+0000 7fb8a95a4640 1 -- 172.21.15.155:0/319214025 learned_addr learned my addr 172.21.15.155:0/319214025 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-08-31T06:01:14.965 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:14.955+0000 7fb8a3fff640 1 --2- 172.21.15.155:0/319214025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8a4108140 0x7fb8a4199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:01:14.965 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:14.956+0000 7fb8a95a4640 1 -- 172.21.15.155:0/319214025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8a4108140 msgr2=0x7fb8a4199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:01:14.965 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:14.956+0000 7fb8a95a4640 1 --2- 172.21.15.155:0/319214025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8a4108140 0x7fb8a4199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:01:14.965 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:14.956+0000 7fb8a95a4640 1 -- 172.21.15.155:0/319214025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8a4102800 msgr2=0x7fb8a41990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:01:14.965 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:14.956+0000 7fb8a95a4640 1 --2- 172.21.15.155:0/319214025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8a4102800 0x7fb8a41990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:01:14.966 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:14.956+0000 7fb8a95a4640 1 -- 172.21.15.155:0/319214025 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8a419f030 con 0x7fb8a4108a90 2024-08-31T06:01:14.966 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:14.956+0000 7fb8a95a4640 1 --2- 172.21.15.155:0/319214025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8a4108a90 0x7fb8a419e900 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb89c0098d0 tx=0x7fb89c0317b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:01:14.966 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:14.956+0000 7fb8a1ffb640 1 -- 172.21.15.155:0/319214025 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89c003b70 con 0x7fb8a4108a90 2024-08-31T06:01:14.966 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:14.956+0000 7fb8ab007640 1 -- 172.21.15.155:0/319214025 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8a419f2c0 con 0x7fb8a4108a90 2024-08-31T06:01:14.966 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:14.956+0000 7fb8a1ffb640 1 -- 172.21.15.155:0/319214025 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb89c003d10 con 0x7fb8a4108a90 2024-08-31T06:01:14.966 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:14.956+0000 7fb8a1ffb640 1 -- 172.21.15.155:0/319214025 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89c02fa20 con 0x7fb8a4108a90 2024-08-31T06:01:14.966 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:14.956+0000 7fb8ab007640 1 -- 172.21.15.155:0/319214025 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8a41a5a80 con 0x7fb8a4108a90 2024-08-31T06:01:14.966 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:14.957+0000 7fb8ab007640 1 -- 172.21.15.155:0/319214025 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8a4102c10 con 0x7fb8a4108a90 2024-08-31T06:01:14.966 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:14.959+0000 7fb8a1ffb640 1 -- 172.21.15.155:0/319214025 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7fb89c02fc20 con 0x7fb8a4108a90 2024-08-31T06:01:14.966 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:14.960+0000 7fb8a1ffb640 1 --2- 172.21.15.155:0/319214025 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb88c0b4b80 0x7fb88c0b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:01:14.966 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:14.960+0000 7fb8a1ffb640 1 -- 172.21.15.155:0/319214025 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5180+0+0 (secure 0 0 0) 0x7fb89c105ae0 con 0x7fb8a4108a90 2024-08-31T06:01:14.966 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:14.960+0000 7fb8a8da3640 1 --2- 172.21.15.155:0/319214025 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb88c0b4b80 0x7fb88c0b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:01:14.966 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:14.960+0000 7fb8a8da3640 1 --2- 172.21.15.155:0/319214025 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb88c0b4b80 0x7fb88c0b7090 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb89400ba80 tx=0x7fb894005ed0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:01:14.967 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:14.963+0000 7fb8a1ffb640 1 -- 172.21.15.155:0/319214025 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fb89c0393b0 con 0x7fb8a4108a90 2024-08-31T06:01:15.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:15 smithi155 ceph-mon[21742]: purged_snaps scrub starts 2024-08-31T06:01:15.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:15 smithi155 ceph-mon[21742]: purged_snaps scrub ok 2024-08-31T06:01:15.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:15 smithi155 ceph-mon[21742]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-08-31T06:01:15.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:15 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-31T06:01:15.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:15 smithi155 ceph-mon[21742]: osd.6 [v2:172.21.15.16:6818/1697421092,v1:172.21.15.16:6819/1697421092] boot 2024-08-31T06:01:15.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:15 smithi155 ceph-mon[21742]: osdmap e43: 7 total, 7 up, 7 in 2024-08-31T06:01:15.156 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:15.154+0000 7fb8ab007640 1 -- 172.21.15.155:0/319214025 --> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fb8a40635a0 con 0x7fb88c0b4b80 2024-08-31T06:01:15.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:15 smithi016 ceph-mon[17194]: purged_snaps scrub starts 2024-08-31T06:01:15.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:15 smithi016 ceph-mon[17194]: purged_snaps scrub ok 2024-08-31T06:01:15.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:15 smithi016 ceph-mon[17194]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-08-31T06:01:15.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:15 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-31T06:01:15.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:15 smithi016 ceph-mon[17194]: osd.6 [v2:172.21.15.16:6818/1697421092,v1:172.21.15.16:6819/1697421092] boot 2024-08-31T06:01:15.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:15 smithi016 ceph-mon[17194]: osdmap e43: 7 total, 7 up, 7 in 2024-08-31T06:01:15.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:15 smithi184 ceph-mon[22331]: purged_snaps scrub starts 2024-08-31T06:01:15.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:15 smithi184 ceph-mon[22331]: purged_snaps scrub ok 2024-08-31T06:01:15.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:15 smithi184 ceph-mon[22331]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-08-31T06:01:15.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:15 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-31T06:01:15.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:15 smithi184 ceph-mon[22331]: osd.6 [v2:172.21.15.16:6818/1697421092,v1:172.21.15.16:6819/1697421092] boot 2024-08-31T06:01:15.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:15 smithi184 ceph-mon[22331]: osdmap e43: 7 total, 7 up, 7 in 2024-08-31T06:01:16.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:16 smithi155 ceph-mon[21742]: osdmap e44: 7 total, 7 up, 7 in 2024-08-31T06:01:16.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:16 smithi155 ceph-mon[21742]: from='client.24485 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:01:16.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:16 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T06:01:16.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:16 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T06:01:16.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:16 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:01:16.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:16 smithi155 ceph-mon[21742]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T06:01:16.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:16 smithi155 ceph-mon[21742]: Detected new or changed devices on smithi016 2024-08-31T06:01:16.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:16 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:16.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:16 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:16.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:16 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T06:01:16.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:16 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T06:01:16.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:16 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T06:01:16.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:16 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T06:01:16.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:16 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-31T06:01:16.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:16 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-31T06:01:16.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:16 smithi155 ceph-mon[21742]: Adjusting osd_memory_target on smithi016 to 5698M 2024-08-31T06:01:16.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:16 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:16.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:16 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:01:16.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:16 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:01:16.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:16 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:16.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:16 smithi016 ceph-mon[17194]: osdmap e44: 7 total, 7 up, 7 in 2024-08-31T06:01:16.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:16 smithi016 ceph-mon[17194]: from='client.24485 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:01:16.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:16 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T06:01:16.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:16 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T06:01:16.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:16 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:01:16.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:16 smithi016 ceph-mon[17194]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T06:01:16.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:16 smithi016 ceph-mon[17194]: Detected new or changed devices on smithi016 2024-08-31T06:01:16.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:16 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:16.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:16 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:16.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:16 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T06:01:16.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:16 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T06:01:16.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:16 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T06:01:16.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:16 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T06:01:16.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:16 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-31T06:01:16.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:16 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-31T06:01:16.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:16 smithi016 ceph-mon[17194]: Adjusting osd_memory_target on smithi016 to 5698M 2024-08-31T06:01:16.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:16 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:16.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:16 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:01:16.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:16 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:01:16.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:16 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:16.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:16 smithi184 ceph-mon[22331]: osdmap e44: 7 total, 7 up, 7 in 2024-08-31T06:01:16.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:16 smithi184 ceph-mon[22331]: from='client.24485 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:01:16.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:16 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T06:01:16.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:16 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T06:01:16.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:16 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:01:16.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:16 smithi184 ceph-mon[22331]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T06:01:16.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:16 smithi184 ceph-mon[22331]: Detected new or changed devices on smithi016 2024-08-31T06:01:16.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:16 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:16.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:16 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:16.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:16 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T06:01:16.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:16 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T06:01:16.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:16 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T06:01:16.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:16 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T06:01:16.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:16 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-31T06:01:16.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:16 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-31T06:01:16.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:16 smithi184 ceph-mon[22331]: Adjusting osd_memory_target on smithi016 to 5698M 2024-08-31T06:01:16.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:16 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:16.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:16 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:01:16.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:16 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:01:16.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:16 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:18.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:18 smithi155 ceph-mon[21742]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T06:01:18.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:18 smithi155 ceph-mon[21742]: from='client.? 172.21.15.155:0/4031645097' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8e76c20e-8d25-423b-b7e8-de642d9c9a6c"} : dispatch 2024-08-31T06:01:18.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:18 smithi155 ceph-mon[21742]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8e76c20e-8d25-423b-b7e8-de642d9c9a6c"} : dispatch 2024-08-31T06:01:18.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:18 smithi155 ceph-mon[21742]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8e76c20e-8d25-423b-b7e8-de642d9c9a6c"}]': finished 2024-08-31T06:01:18.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:18 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-31T06:01:18.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:18 smithi155 ceph-mon[21742]: osdmap e45: 8 total, 7 up, 8 in 2024-08-31T06:01:18.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:18 smithi016 ceph-mon[17194]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T06:01:18.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:18 smithi016 ceph-mon[17194]: from='client.? 172.21.15.155:0/4031645097' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8e76c20e-8d25-423b-b7e8-de642d9c9a6c"} : dispatch 2024-08-31T06:01:18.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:18 smithi016 ceph-mon[17194]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8e76c20e-8d25-423b-b7e8-de642d9c9a6c"} : dispatch 2024-08-31T06:01:18.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:18 smithi016 ceph-mon[17194]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8e76c20e-8d25-423b-b7e8-de642d9c9a6c"}]': finished 2024-08-31T06:01:18.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:18 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-31T06:01:18.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:18 smithi016 ceph-mon[17194]: osdmap e45: 8 total, 7 up, 8 in 2024-08-31T06:01:18.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:18 smithi184 ceph-mon[22331]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T06:01:18.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:18 smithi184 ceph-mon[22331]: from='client.? 172.21.15.155:0/4031645097' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8e76c20e-8d25-423b-b7e8-de642d9c9a6c"} : dispatch 2024-08-31T06:01:18.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:18 smithi184 ceph-mon[22331]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8e76c20e-8d25-423b-b7e8-de642d9c9a6c"} : dispatch 2024-08-31T06:01:18.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:18 smithi184 ceph-mon[22331]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8e76c20e-8d25-423b-b7e8-de642d9c9a6c"}]': finished 2024-08-31T06:01:18.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:18 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-31T06:01:18.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:18 smithi184 ceph-mon[22331]: osdmap e45: 8 total, 7 up, 8 in 2024-08-31T06:01:19.621 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:19 smithi155 ceph-mon[21742]: from='client.? 172.21.15.155:0/1573197042' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T06:01:19.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:19 smithi016 ceph-mon[17194]: from='client.? 172.21.15.155:0/1573197042' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T06:01:19.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:19 smithi184 ceph-mon[22331]: from='client.? 172.21.15.155:0/1573197042' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T06:01:20.621 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:20 smithi155 ceph-mon[21742]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T06:01:20.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:20 smithi016 ceph-mon[17194]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T06:01:20.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:20 smithi184 ceph-mon[22331]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T06:01:22.574 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:22 smithi155 ceph-mon[21742]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T06:01:22.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:22 smithi016 ceph-mon[17194]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T06:01:22.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:22 smithi184 ceph-mon[22331]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T06:01:24.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:24 smithi155 ceph-mon[21742]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T06:01:24.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:24 smithi016 ceph-mon[17194]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T06:01:24.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:24 smithi184 ceph-mon[22331]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T06:01:26.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:26 smithi155 ceph-mon[21742]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T06:01:26.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:26 smithi016 ceph-mon[17194]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T06:01:26.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:26 smithi184 ceph-mon[22331]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T06:01:27.564 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:27 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-08-31T06:01:27.565 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:27 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:01:27.565 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:27 smithi016 ceph-mon[17194]: Deploying daemon osd.7 on smithi155 2024-08-31T06:01:27.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:27 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-08-31T06:01:27.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:27 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:01:27.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:27 smithi155 ceph-mon[21742]: Deploying daemon osd.7 on smithi155 2024-08-31T06:01:27.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:27 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-08-31T06:01:27.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:27 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:01:27.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:27 smithi184 ceph-mon[22331]: Deploying daemon osd.7 on smithi155 2024-08-31T06:01:28.479 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:28 smithi155 ceph-mon[21742]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T06:01:28.588 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:28 smithi016 ceph-mon[17194]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T06:01:28.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:28 smithi184 ceph-mon[22331]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T06:01:30.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:30 smithi016 ceph-mon[17194]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T06:01:30.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:30 smithi155 ceph-mon[21742]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T06:01:30.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:30 smithi184 ceph-mon[22331]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T06:01:32.601 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:32 smithi155 ceph-mon[21742]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T06:01:32.602 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:32 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:01:32.602 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:32 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:32.602 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:32 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:32.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:32 smithi016 ceph-mon[17194]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T06:01:32.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:32 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:01:32.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:32 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:32.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:32 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:32.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:32 smithi184 ceph-mon[22331]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T06:01:32.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:32 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:01:32.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:32 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:32.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:32 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:34.370 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:34 smithi155 ceph-mon[21742]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T06:01:34.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:34 smithi016 ceph-mon[17194]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T06:01:34.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:34 smithi184 ceph-mon[22331]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T06:01:36.201 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:36.198+0000 7fb8a1ffb640 1 -- 172.21.15.155:0/319214025 <== mgr.14220 v2:172.21.15.16:6800/4279751337 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb8a40635a0 con 0x7fb88c0b4b80 2024-08-31T06:01:36.201 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 7 on host 'smithi155' 2024-08-31T06:01:36.203 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:36.201+0000 7fb8ab007640 1 -- 172.21.15.155:0/319214025 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb88c0b4b80 msgr2=0x7fb88c0b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:01:36.203 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:36.201+0000 7fb8ab007640 1 --2- 172.21.15.155:0/319214025 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb88c0b4b80 0x7fb88c0b7090 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb89400ba80 tx=0x7fb894005ed0 comp rx=0 tx=0).stop 2024-08-31T06:01:36.203 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:36.201+0000 7fb8ab007640 1 -- 172.21.15.155:0/319214025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8a4108a90 msgr2=0x7fb8a419e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:01:36.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:36.201+0000 7fb8ab007640 1 --2- 172.21.15.155:0/319214025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8a4108a90 0x7fb8a419e900 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb89c0098d0 tx=0x7fb89c0317b0 comp rx=0 tx=0).stop 2024-08-31T06:01:36.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:36.201+0000 7fb8ab007640 1 -- 172.21.15.155:0/319214025 shutdown_connections 2024-08-31T06:01:36.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:36.201+0000 7fb8ab007640 1 --2- 172.21.15.155:0/319214025 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb88c0b4b80 0x7fb88c0b7090 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:01:36.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:36.201+0000 7fb8ab007640 1 --2- 172.21.15.155:0/319214025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8a4108140 0x7fb8a4199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:01:36.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:36.201+0000 7fb8ab007640 1 --2- 172.21.15.155:0/319214025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8a4108a90 0x7fb8a419e900 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:01:36.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:36.201+0000 7fb8ab007640 1 --2- 172.21.15.155:0/319214025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8a4102800 0x7fb8a41990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:01:36.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:36.201+0000 7fb8ab007640 1 -- 172.21.15.155:0/319214025 >> 172.21.15.155:0/319214025 conn(0x7fb8a40794d0 msgr2=0x7fb8a4105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:01:36.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:36.201+0000 7fb8ab007640 1 -- 172.21.15.155:0/319214025 shutdown_connections 2024-08-31T06:01:36.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:01:36.201+0000 7fb8ab007640 1 -- 172.21.15.155:0/319214025 wait complete. 2024-08-31T06:01:36.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:36 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:36.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:36 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:36.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:36 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:01:36.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:36 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:01:36.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:36 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:36.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:36 smithi155 ceph-mon[21742]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T06:01:36.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:36 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:36.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:36 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:36.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:36 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:01:36.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:36 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:01:36.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:36 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:36.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:36 smithi016 ceph-mon[17194]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T06:01:36.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:36 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:36.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:36 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:36.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:36 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:01:36.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:36 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:01:36.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:36 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:36.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:36 smithi184 ceph-mon[22331]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T06:01:36.896 DEBUG:teuthology.orchestra.run.smithi155:osd.7> sudo journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@osd.7.service 2024-08-31T06:01:36.899 INFO:tasks.cephadm:Deploying osd.8 on smithi184 with /dev/vg_nvme/lv_2... 2024-08-31T06:01:36.899 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-08-31T06:01:37.155 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.c/config 2024-08-31T06:01:37.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:37 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:01:37.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:37 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:37.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:37 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:37.404 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:37 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:01:37.404 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:37 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:37.405 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:37 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:37.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:37 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:01:37.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:37 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:37.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:37 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:38.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:38 smithi016 ceph-mon[17194]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T06:01:38.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:38 smithi184 ceph-mon[22331]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T06:01:38.492 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:38 smithi155 ceph-mon[21742]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T06:01:38.492 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:01:38 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:01:38.169+0000 7f4a1dbd5640 -1 osd.7 0 log_to_monitors true 2024-08-31T06:01:39.407 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:39 smithi184 ceph-mon[22331]: from='osd.7 [v2:172.21.15.155:6816/2671827901,v1:172.21.15.155:6817/2671827901]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-08-31T06:01:39.408 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:39 smithi184 ceph-mon[22331]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-08-31T06:01:39.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:39 smithi016 ceph-mon[17194]: from='osd.7 [v2:172.21.15.155:6816/2671827901,v1:172.21.15.155:6817/2671827901]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-08-31T06:01:39.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:39 smithi016 ceph-mon[17194]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-08-31T06:01:39.603 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:39 smithi155 ceph-mon[21742]: from='osd.7 [v2:172.21.15.155:6816/2671827901,v1:172.21.15.155:6817/2671827901]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-08-31T06:01:39.604 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:39 smithi155 ceph-mon[21742]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-08-31T06:01:39.760 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-08-31T06:01:39.788 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph orch daemon add osd smithi184:vg_nvme/lv_2 2024-08-31T06:01:40.045 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.c/config 2024-08-31T06:01:40.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:40 smithi155 ceph-mon[21742]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-08-31T06:01:40.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:40 smithi155 ceph-mon[21742]: osdmap e46: 8 total, 7 up, 8 in 2024-08-31T06:01:40.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:40 smithi155 ceph-mon[21742]: from='osd.7 [v2:172.21.15.155:6816/2671827901,v1:172.21.15.155:6817/2671827901]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]} : dispatch 2024-08-31T06:01:40.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:40 smithi155 ceph-mon[21742]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]} : dispatch 2024-08-31T06:01:40.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:40 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-31T06:01:40.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:40 smithi155 ceph-mon[21742]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T06:01:40.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:40 smithi155 ceph-mon[21742]: Detected new or changed devices on smithi155 2024-08-31T06:01:40.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:40 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:40.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:40 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:40.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:40 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T06:01:40.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:40 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T06:01:40.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:40 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-31T06:01:40.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:40 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-31T06:01:40.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:40 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-31T06:01:40.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:40 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-31T06:01:40.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:40 smithi155 ceph-mon[21742]: Adjusting osd_memory_target on smithi155 to 5696M 2024-08-31T06:01:40.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:40 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:40.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:40 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:01:40.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:40 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:01:40.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:40 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:40.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:40 smithi016 ceph-mon[17194]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-08-31T06:01:40.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:40 smithi016 ceph-mon[17194]: osdmap e46: 8 total, 7 up, 8 in 2024-08-31T06:01:40.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:40 smithi016 ceph-mon[17194]: from='osd.7 [v2:172.21.15.155:6816/2671827901,v1:172.21.15.155:6817/2671827901]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]} : dispatch 2024-08-31T06:01:40.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:40 smithi016 ceph-mon[17194]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]} : dispatch 2024-08-31T06:01:40.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:40 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-31T06:01:40.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:40 smithi016 ceph-mon[17194]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T06:01:40.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:40 smithi016 ceph-mon[17194]: Detected new or changed devices on smithi155 2024-08-31T06:01:40.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:40 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:40.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:40 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:40.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:40 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T06:01:40.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:40 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T06:01:40.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:40 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-31T06:01:40.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:40 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-31T06:01:40.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:40 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-31T06:01:40.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:40 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-31T06:01:40.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:40 smithi016 ceph-mon[17194]: Adjusting osd_memory_target on smithi155 to 5696M 2024-08-31T06:01:40.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:40 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:40.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:40 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:01:40.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:40 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:01:40.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:40 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:40.417 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:40 smithi184 ceph-mon[22331]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-08-31T06:01:40.417 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:40 smithi184 ceph-mon[22331]: osdmap e46: 8 total, 7 up, 8 in 2024-08-31T06:01:40.417 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:40 smithi184 ceph-mon[22331]: from='osd.7 [v2:172.21.15.155:6816/2671827901,v1:172.21.15.155:6817/2671827901]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]} : dispatch 2024-08-31T06:01:40.418 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:40 smithi184 ceph-mon[22331]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]} : dispatch 2024-08-31T06:01:40.418 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:40 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-31T06:01:40.418 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:40 smithi184 ceph-mon[22331]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T06:01:40.418 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:40 smithi184 ceph-mon[22331]: Detected new or changed devices on smithi155 2024-08-31T06:01:40.418 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:40 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:40.418 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:40 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:40.418 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:40 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T06:01:40.418 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:40 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T06:01:40.418 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:40 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-31T06:01:40.418 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:40 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-31T06:01:40.418 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:40 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-31T06:01:40.418 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:40 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-31T06:01:40.418 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:40 smithi184 ceph-mon[22331]: Adjusting osd_memory_target on smithi155 to 5696M 2024-08-31T06:01:40.418 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:40 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:40.419 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:40 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:01:40.419 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:40 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:01:40.419 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:40 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:40.553 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:01:40.539+0000 7fa0d0166640 1 -- 172.21.15.184:0/2067654236 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0c8109b90 msgr2=0x7fa0c81103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:01:40.553 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:01:40.539+0000 7fa0d0166640 1 --2- 172.21.15.184:0/2067654236 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0c8109b90 0x7fa0c81103c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0009ca0 tx=0x7fa0c002a8c0 comp rx=0 tx=0).stop 2024-08-31T06:01:40.553 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:01:40.539+0000 7fa0d0166640 1 -- 172.21.15.184:0/2067654236 shutdown_connections 2024-08-31T06:01:40.553 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:01:40.539+0000 7fa0d0166640 1 --2- 172.21.15.184:0/2067654236 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0c8109b90 0x7fa0c81103c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:01:40.554 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:01:40.539+0000 7fa0d0166640 1 --2- 172.21.15.184:0/2067654236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0c8109240 0x7fa0c8109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:01:40.554 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:01:40.539+0000 7fa0d0166640 1 --2- 172.21.15.184:0/2067654236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0c8103900 0x7fa0c8103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:01:40.554 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:01:40.539+0000 7fa0d0166640 1 -- 172.21.15.184:0/2067654236 >> 172.21.15.184:0/2067654236 conn(0x7fa0c807b5f0 msgr2=0x7fa0c807ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:01:40.554 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:01:40.540+0000 7fa0d0166640 1 -- 172.21.15.184:0/2067654236 shutdown_connections 2024-08-31T06:01:40.554 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:01:40.540+0000 7fa0d0166640 1 -- 172.21.15.184:0/2067654236 wait complete. 2024-08-31T06:01:40.554 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:01:40.540+0000 7fa0d0166640 1 Processor -- start 2024-08-31T06:01:40.554 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:01:40.540+0000 7fa0d0166640 1 -- start start 2024-08-31T06:01:40.554 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:01:40.540+0000 7fa0d0166640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0c8103900 0x7fa0c819a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:01:40.554 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:01:40.540+0000 7fa0d0166640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0c8109240 0x7fa0c819a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:01:40.554 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:01:40.540+0000 7fa0d0166640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0c8109b90 0x7fa0c819fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:01:40.554 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:01:40.540+0000 7fa0d0166640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c81138d0 con 0x7fa0c8109b90 2024-08-31T06:01:40.554 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:01:40.540+0000 7fa0d0166640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c8113750 con 0x7fa0c8109240 2024-08-31T06:01:40.554 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:01:40.540+0000 7fa0d0166640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c8113a50 con 0x7fa0c8103900 2024-08-31T06:01:40.555 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:01:40.541+0000 7fa0cdf02640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0c8103900 0x7fa0c819a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:01:40.555 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:01:40.541+0000 7fa0cdf02640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0c8103900 0x7fa0c819a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:46990/0 (socket says 172.21.15.184:46990) 2024-08-31T06:01:40.555 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:01:40.541+0000 7fa0cdf02640 1 -- 172.21.15.184:0/1818114285 learned_addr learned my addr 172.21.15.184:0/1818114285 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-08-31T06:01:40.555 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:01:40.541+0000 7fa0cd701640 1 --2- 172.21.15.184:0/1818114285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0c8109240 0x7fa0c819a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:01:40.555 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:01:40.541+0000 7fa0cdf02640 1 -- 172.21.15.184:0/1818114285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0c8109240 msgr2=0x7fa0c819a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:01:40.555 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:01:40.541+0000 7fa0cdf02640 1 --2- 172.21.15.184:0/1818114285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0c8109240 0x7fa0c819a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:01:40.555 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:01:40.541+0000 7fa0cdf02640 1 -- 172.21.15.184:0/1818114285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0c8109b90 msgr2=0x7fa0c819fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:01:40.555 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:01:40.541+0000 7fa0cdf02640 1 --2- 172.21.15.184:0/1818114285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0c8109b90 0x7fa0c819fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:01:40.555 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:01:40.541+0000 7fa0cdf02640 1 -- 172.21.15.184:0/1818114285 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c81a0270 con 0x7fa0c8103900 2024-08-31T06:01:40.555 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:01:40.541+0000 7fa0cdf02640 1 --2- 172.21.15.184:0/1818114285 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0c8103900 0x7fa0c819a300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc00b4a0 tx=0x7fa0bc00b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:01:40.555 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:01:40.541+0000 7fa0baffd640 1 -- 172.21.15.184:0/1818114285 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc014070 con 0x7fa0c8103900 2024-08-31T06:01:40.555 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:01:40.541+0000 7fa0baffd640 1 -- 172.21.15.184:0/1818114285 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0bc010ac0 con 0x7fa0c8103900 2024-08-31T06:01:40.556 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:01:40.541+0000 7fa0baffd640 1 -- 172.21.15.184:0/1818114285 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc017c30 con 0x7fa0c8103900 2024-08-31T06:01:40.556 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:01:40.541+0000 7fa0d0166640 1 -- 172.21.15.184:0/1818114285 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0c81a0560 con 0x7fa0c8103900 2024-08-31T06:01:40.556 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:01:40.541+0000 7fa0d0166640 1 -- 172.21.15.184:0/1818114285 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0c81a6cc0 con 0x7fa0c8103900 2024-08-31T06:01:40.556 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:01:40.542+0000 7fa0d0166640 1 -- 172.21.15.184:0/1818114285 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0c8103d10 con 0x7fa0c8103900 2024-08-31T06:01:40.556 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:01:40.544+0000 7fa0baffd640 1 -- 172.21.15.184:0/1818114285 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7fa0bc017070 con 0x7fa0c8103900 2024-08-31T06:01:40.556 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:01:40.545+0000 7fa0baffd640 1 --2- 172.21.15.184:0/1818114285 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa0b40b4b80 0x7fa0b40b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:01:40.556 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:01:40.545+0000 7fa0baffd640 1 -- 172.21.15.184:0/1818114285 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5365+0+0 (secure 0 0 0) 0x7fa0bc0e3bd0 con 0x7fa0c8103900 2024-08-31T06:01:40.556 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:01:40.545+0000 7fa0cd701640 1 --2- 172.21.15.184:0/1818114285 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa0b40b4b80 0x7fa0b40b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:01:40.556 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:01:40.545+0000 7fa0cd701640 1 --2- 172.21.15.184:0/1818114285 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa0b40b4b80 0x7fa0b40b7090 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8106200 tx=0x7fa0c4005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:01:40.556 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:01:40.549+0000 7fa0baffd640 1 -- 172.21.15.184:0/1818114285 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fa0bc0ab880 con 0x7fa0c8103900 2024-08-31T06:01:40.735 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:01:40.733+0000 7fa0d0166640 1 -- 172.21.15.184:0/1818114285 --> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fa0c80635a0 con 0x7fa0b40b4b80 2024-08-31T06:01:40.872 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:01:40 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:01:40.585+0000 7f4a19b57640 -1 osd.7 0 waiting for initial osdmap 2024-08-31T06:01:40.872 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:01:40 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:01:40.596+0000 7f4a1194d640 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-31T06:01:41.398 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:41 smithi184 ceph-mon[22331]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-08-31T06:01:41.398 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:41 smithi184 ceph-mon[22331]: osdmap e47: 8 total, 7 up, 8 in 2024-08-31T06:01:41.398 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:41 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-31T06:01:41.398 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:41 smithi184 ceph-mon[22331]: osdmap e48: 8 total, 7 up, 8 in 2024-08-31T06:01:41.398 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:41 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-31T06:01:41.398 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:41 smithi184 ceph-mon[22331]: from='osd.7 ' entity='osd.7' 2024-08-31T06:01:41.398 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:41 smithi184 ceph-mon[22331]: from='client.24499 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:01:41.398 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:41 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T06:01:41.398 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:41 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T06:01:41.398 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:41 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:01:41.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:41 smithi016 ceph-mon[17194]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-08-31T06:01:41.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:41 smithi016 ceph-mon[17194]: osdmap e47: 8 total, 7 up, 8 in 2024-08-31T06:01:41.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:41 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-31T06:01:41.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:41 smithi016 ceph-mon[17194]: osdmap e48: 8 total, 7 up, 8 in 2024-08-31T06:01:41.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:41 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-31T06:01:41.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:41 smithi016 ceph-mon[17194]: from='osd.7 ' entity='osd.7' 2024-08-31T06:01:41.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:41 smithi016 ceph-mon[17194]: from='client.24499 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:01:41.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:41 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T06:01:41.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:41 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T06:01:41.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:41 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:01:41.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:41 smithi155 ceph-mon[21742]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-08-31T06:01:41.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:41 smithi155 ceph-mon[21742]: osdmap e47: 8 total, 7 up, 8 in 2024-08-31T06:01:41.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:41 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-31T06:01:41.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:41 smithi155 ceph-mon[21742]: osdmap e48: 8 total, 7 up, 8 in 2024-08-31T06:01:41.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:41 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-31T06:01:41.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:41 smithi155 ceph-mon[21742]: from='osd.7 ' entity='osd.7' 2024-08-31T06:01:41.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:41 smithi155 ceph-mon[21742]: from='client.24499 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:01:41.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:41 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T06:01:41.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:41 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T06:01:41.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:41 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:01:42.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:42 smithi184 ceph-mon[22331]: purged_snaps scrub starts 2024-08-31T06:01:42.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:42 smithi184 ceph-mon[22331]: purged_snaps scrub ok 2024-08-31T06:01:42.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:42 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-31T06:01:42.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:42 smithi184 ceph-mon[22331]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T06:01:42.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:42 smithi184 ceph-mon[22331]: osd.7 [v2:172.21.15.155:6816/2671827901,v1:172.21.15.155:6817/2671827901] boot 2024-08-31T06:01:42.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:42 smithi184 ceph-mon[22331]: osdmap e49: 8 total, 8 up, 8 in 2024-08-31T06:01:42.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:42 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-31T06:01:42.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:42 smithi155 ceph-mon[21742]: purged_snaps scrub starts 2024-08-31T06:01:42.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:42 smithi155 ceph-mon[21742]: purged_snaps scrub ok 2024-08-31T06:01:42.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:42 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-31T06:01:42.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:42 smithi155 ceph-mon[21742]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T06:01:42.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:42 smithi155 ceph-mon[21742]: osd.7 [v2:172.21.15.155:6816/2671827901,v1:172.21.15.155:6817/2671827901] boot 2024-08-31T06:01:42.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:42 smithi155 ceph-mon[21742]: osdmap e49: 8 total, 8 up, 8 in 2024-08-31T06:01:42.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:42 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-31T06:01:42.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:42 smithi016 ceph-mon[17194]: purged_snaps scrub starts 2024-08-31T06:01:42.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:42 smithi016 ceph-mon[17194]: purged_snaps scrub ok 2024-08-31T06:01:42.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:42 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-31T06:01:42.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:42 smithi016 ceph-mon[17194]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-31T06:01:42.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:42 smithi016 ceph-mon[17194]: osd.7 [v2:172.21.15.155:6816/2671827901,v1:172.21.15.155:6817/2671827901] boot 2024-08-31T06:01:42.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:42 smithi016 ceph-mon[17194]: osdmap e49: 8 total, 8 up, 8 in 2024-08-31T06:01:42.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:42 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-31T06:01:43.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:43 smithi184 ceph-mon[22331]: osdmap e50: 8 total, 8 up, 8 in 2024-08-31T06:01:43.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:43 smithi184 ceph-mon[22331]: from='client.? 172.21.15.184:0/2776757773' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0f0bb23f-4bc3-4198-8381-5b5cd16ae394"} : dispatch 2024-08-31T06:01:43.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:43 smithi184 ceph-mon[22331]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0f0bb23f-4bc3-4198-8381-5b5cd16ae394"} : dispatch 2024-08-31T06:01:43.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:43 smithi184 ceph-mon[22331]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f0bb23f-4bc3-4198-8381-5b5cd16ae394"}]': finished 2024-08-31T06:01:43.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:43 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-31T06:01:43.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:43 smithi184 ceph-mon[22331]: osdmap e51: 9 total, 8 up, 9 in 2024-08-31T06:01:43.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:43 smithi155 ceph-mon[21742]: osdmap e50: 8 total, 8 up, 8 in 2024-08-31T06:01:43.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:43 smithi155 ceph-mon[21742]: from='client.? 172.21.15.184:0/2776757773' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0f0bb23f-4bc3-4198-8381-5b5cd16ae394"} : dispatch 2024-08-31T06:01:43.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:43 smithi155 ceph-mon[21742]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0f0bb23f-4bc3-4198-8381-5b5cd16ae394"} : dispatch 2024-08-31T06:01:43.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:43 smithi155 ceph-mon[21742]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f0bb23f-4bc3-4198-8381-5b5cd16ae394"}]': finished 2024-08-31T06:01:43.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:43 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-31T06:01:43.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:43 smithi155 ceph-mon[21742]: osdmap e51: 9 total, 8 up, 9 in 2024-08-31T06:01:43.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:43 smithi016 ceph-mon[17194]: osdmap e50: 8 total, 8 up, 8 in 2024-08-31T06:01:43.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:43 smithi016 ceph-mon[17194]: from='client.? 172.21.15.184:0/2776757773' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0f0bb23f-4bc3-4198-8381-5b5cd16ae394"} : dispatch 2024-08-31T06:01:43.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:43 smithi016 ceph-mon[17194]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0f0bb23f-4bc3-4198-8381-5b5cd16ae394"} : dispatch 2024-08-31T06:01:43.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:43 smithi016 ceph-mon[17194]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f0bb23f-4bc3-4198-8381-5b5cd16ae394"}]': finished 2024-08-31T06:01:43.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:43 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-31T06:01:43.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:43 smithi016 ceph-mon[17194]: osdmap e51: 9 total, 8 up, 9 in 2024-08-31T06:01:44.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:44 smithi184 ceph-mon[22331]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-08-31T06:01:44.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:44 smithi184 ceph-mon[22331]: from='client.? 172.21.15.184:0/1489504709' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T06:01:44.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:44 smithi155 ceph-mon[21742]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-08-31T06:01:44.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:44 smithi155 ceph-mon[21742]: from='client.? 172.21.15.184:0/1489504709' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T06:01:44.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:44 smithi016 ceph-mon[17194]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-08-31T06:01:44.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:44 smithi016 ceph-mon[17194]: from='client.? 172.21.15.184:0/1489504709' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T06:01:46.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:46 smithi184 ceph-mon[22331]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-08-31T06:01:46.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:46 smithi155 ceph-mon[21742]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-08-31T06:01:46.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:46 smithi016 ceph-mon[17194]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-08-31T06:01:48.724 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:48 smithi184 ceph-mon[22331]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-31T06:01:48.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:48 smithi155 ceph-mon[21742]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-31T06:01:48.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:48 smithi016 ceph-mon[17194]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-31T06:01:50.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:50 smithi184 ceph-mon[22331]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-08-31T06:01:50.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:50 smithi155 ceph-mon[21742]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-08-31T06:01:50.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:50 smithi016 ceph-mon[17194]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-08-31T06:01:52.739 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:52 smithi184 ceph-mon[22331]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-08-31T06:01:52.739 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:52 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-08-31T06:01:52.739 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:52 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:01:52.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:52 smithi155 ceph-mon[21742]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-08-31T06:01:52.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:52 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-08-31T06:01:52.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:52 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:01:52.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:52 smithi016 ceph-mon[17194]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-08-31T06:01:52.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:52 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-08-31T06:01:52.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:52 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:01:53.820 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:53 smithi184 ceph-mon[22331]: Deploying daemon osd.8 on smithi184 2024-08-31T06:01:53.871 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:53 smithi155 ceph-mon[21742]: Deploying daemon osd.8 on smithi184 2024-08-31T06:01:53.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:53 smithi016 ceph-mon[17194]: Deploying daemon osd.8 on smithi184 2024-08-31T06:01:54.871 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:54 smithi155 ceph-mon[21742]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-31T06:01:54.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:54 smithi016 ceph-mon[17194]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-31T06:01:54.919 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:54 smithi184 ceph-mon[22331]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-31T06:01:56.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:56 smithi184 ceph-mon[22331]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-31T06:01:56.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:56 smithi155 ceph-mon[21742]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-31T06:01:56.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:56 smithi016 ceph-mon[17194]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-31T06:01:58.552 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:58 smithi184 ceph-mon[22331]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-31T06:01:58.553 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:58 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:01:58.553 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:58 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:58.553 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:01:58 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:58.871 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:58 smithi155 ceph-mon[21742]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-31T06:01:58.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:58 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:01:58.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:58 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:58.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:01:58 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:58.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:58 smithi016 ceph-mon[17194]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-31T06:01:58.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:58 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:01:58.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:58 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:01:58.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:01:58 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:00.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:00 smithi155 ceph-mon[21742]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-08-31T06:02:00.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:00 smithi016 ceph-mon[17194]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-08-31T06:02:00.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:00 smithi184 ceph-mon[22331]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-08-31T06:02:01.849 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:02:01.847+0000 7fa0baffd640 1 -- 172.21.15.184:0/1818114285 <== mgr.14220 v2:172.21.15.16:6800/4279751337 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa0c80635a0 con 0x7fa0b40b4b80 2024-08-31T06:02:01.850 INFO:teuthology.orchestra.run.smithi184.stdout:Created osd(s) 8 on host 'smithi184' 2024-08-31T06:02:01.851 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:02:01.849+0000 7fa0d0166640 1 -- 172.21.15.184:0/1818114285 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa0b40b4b80 msgr2=0x7fa0b40b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:02:01.852 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:02:01.849+0000 7fa0d0166640 1 --2- 172.21.15.184:0/1818114285 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa0b40b4b80 0x7fa0b40b7090 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8106200 tx=0x7fa0c4005f90 comp rx=0 tx=0).stop 2024-08-31T06:02:01.852 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:02:01.849+0000 7fa0d0166640 1 -- 172.21.15.184:0/1818114285 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0c8103900 msgr2=0x7fa0c819a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:02:01.852 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:02:01.849+0000 7fa0d0166640 1 --2- 172.21.15.184:0/1818114285 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0c8103900 0x7fa0c819a300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc00b4a0 tx=0x7fa0bc00b960 comp rx=0 tx=0).stop 2024-08-31T06:02:01.852 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:02:01.849+0000 7fa0d0166640 1 -- 172.21.15.184:0/1818114285 shutdown_connections 2024-08-31T06:02:01.852 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:02:01.849+0000 7fa0d0166640 1 --2- 172.21.15.184:0/1818114285 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa0b40b4b80 0x7fa0b40b7090 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:02:01.852 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:02:01.849+0000 7fa0d0166640 1 --2- 172.21.15.184:0/1818114285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0c8109b90 0x7fa0c819fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:02:01.852 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:02:01.849+0000 7fa0d0166640 1 --2- 172.21.15.184:0/1818114285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0c8109240 0x7fa0c819a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:02:01.852 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:02:01.849+0000 7fa0d0166640 1 --2- 172.21.15.184:0/1818114285 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0c8103900 0x7fa0c819a300 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:02:01.852 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:02:01.849+0000 7fa0d0166640 1 -- 172.21.15.184:0/1818114285 >> 172.21.15.184:0/1818114285 conn(0x7fa0c807b5f0 msgr2=0x7fa0c8107250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:02:01.852 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:02:01.850+0000 7fa0d0166640 1 -- 172.21.15.184:0/1818114285 shutdown_connections 2024-08-31T06:02:01.852 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:02:01.850+0000 7fa0d0166640 1 -- 172.21.15.184:0/1818114285 wait complete. 2024-08-31T06:02:02.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:01 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:02.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:01 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:02.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:01 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:02:02.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:01 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:02:02.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:01 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:02.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:01 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:02:02.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:01 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:02.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:01 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:02.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:01 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:02:02.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:01 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:02:02.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:01 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:02.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:01 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:02:02.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:01 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:02.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:01 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:02.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:01 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:02:02.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:01 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:02:02.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:01 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:02.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:01 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:02:02.736 DEBUG:teuthology.orchestra.run.smithi184:osd.8> sudo journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@osd.8.service 2024-08-31T06:02:02.738 INFO:tasks.cephadm:Deploying osd.9 on smithi016 with /dev/vg_nvme/lv_1... 2024-08-31T06:02:02.739 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-08-31T06:02:02.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:02 smithi184 ceph-mon[22331]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-08-31T06:02:02.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:02 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:02.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:02 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:03.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:02 smithi155 ceph-mon[21742]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-08-31T06:02:03.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:02 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:03.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:02 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:03.125 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:02:03.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:02 smithi016 ceph-mon[17194]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-08-31T06:02:03.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:02 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:03.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:02 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:04.166 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:02:03 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:02:03.910+0000 7f68832ad640 -1 osd.8 0 log_to_monitors true 2024-08-31T06:02:04.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:04 smithi016 ceph-mon[17194]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-08-31T06:02:04.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:04 smithi016 ceph-mon[17194]: from='osd.8 [v2:172.21.15.184:6816/2479117823,v1:172.21.15.184:6817/2479117823]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-08-31T06:02:04.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:04 smithi016 ceph-mon[17194]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-08-31T06:02:05.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:04 smithi155 ceph-mon[21742]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-08-31T06:02:05.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:04 smithi155 ceph-mon[21742]: from='osd.8 [v2:172.21.15.184:6816/2479117823,v1:172.21.15.184:6817/2479117823]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-08-31T06:02:05.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:04 smithi155 ceph-mon[21742]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-08-31T06:02:05.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:04 smithi184 ceph-mon[22331]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-08-31T06:02:05.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:04 smithi184 ceph-mon[22331]: from='osd.8 [v2:172.21.15.184:6816/2479117823,v1:172.21.15.184:6817/2479117823]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-08-31T06:02:05.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:04 smithi184 ceph-mon[22331]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-08-31T06:02:05.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:05 smithi016 ceph-mon[17194]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-08-31T06:02:05.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:05 smithi016 ceph-mon[17194]: osdmap e52: 9 total, 8 up, 9 in 2024-08-31T06:02:05.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:05 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-31T06:02:05.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:05 smithi016 ceph-mon[17194]: from='osd.8 [v2:172.21.15.184:6816/2479117823,v1:172.21.15.184:6817/2479117823]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]} : dispatch 2024-08-31T06:02:05.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:05 smithi016 ceph-mon[17194]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]} : dispatch 2024-08-31T06:02:05.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:05 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:05.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:05 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:05.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:05 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T06:02:05.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:05 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T06:02:05.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:05 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-31T06:02:05.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:05 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-31T06:02:05.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:05 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-31T06:02:05.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:05 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-31T06:02:05.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:05 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:05.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:05 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:02:05.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:05 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:02:05.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:05 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:06.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:05 smithi155 ceph-mon[21742]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-08-31T06:02:06.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:05 smithi155 ceph-mon[21742]: osdmap e52: 9 total, 8 up, 9 in 2024-08-31T06:02:06.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:05 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-31T06:02:06.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:05 smithi155 ceph-mon[21742]: from='osd.8 [v2:172.21.15.184:6816/2479117823,v1:172.21.15.184:6817/2479117823]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]} : dispatch 2024-08-31T06:02:06.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:05 smithi155 ceph-mon[21742]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]} : dispatch 2024-08-31T06:02:06.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:05 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:06.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:05 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:06.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:05 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T06:02:06.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:05 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T06:02:06.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:05 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-31T06:02:06.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:05 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-31T06:02:06.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:05 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-31T06:02:06.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:05 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-31T06:02:06.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:05 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:06.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:05 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:02:06.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:05 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:02:06.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:05 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:06.223 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:02:05 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:02:05.851+0000 7f687f22f640 -1 osd.8 0 waiting for initial osdmap 2024-08-31T06:02:06.223 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:02:05 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:02:05.865+0000 7f687b02d640 -1 osd.8 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-31T06:02:06.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:05 smithi184 ceph-mon[22331]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-08-31T06:02:06.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:05 smithi184 ceph-mon[22331]: osdmap e52: 9 total, 8 up, 9 in 2024-08-31T06:02:06.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:05 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-31T06:02:06.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:05 smithi184 ceph-mon[22331]: from='osd.8 [v2:172.21.15.184:6816/2479117823,v1:172.21.15.184:6817/2479117823]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]} : dispatch 2024-08-31T06:02:06.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:05 smithi184 ceph-mon[22331]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]} : dispatch 2024-08-31T06:02:06.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:05 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:06.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:05 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:06.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:05 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T06:02:06.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:05 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T06:02:06.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:05 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-31T06:02:06.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:05 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-31T06:02:06.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:05 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-31T06:02:06.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:05 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-31T06:02:06.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:05 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:06.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:05 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:02:06.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:05 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:02:06.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:05 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:06.653 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:02:06.686 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph orch daemon add osd smithi016:vg_nvme/lv_1 2024-08-31T06:02:06.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:06 smithi016 ceph-mon[17194]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-08-31T06:02:06.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:06 smithi016 ceph-mon[17194]: Detected new or changed devices on smithi184 2024-08-31T06:02:06.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:06 smithi016 ceph-mon[17194]: Adjusting osd_memory_target on smithi184 to 5696M 2024-08-31T06:02:06.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:06 smithi016 ceph-mon[17194]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-08-31T06:02:06.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:06 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-31T06:02:06.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:06 smithi016 ceph-mon[17194]: osdmap e53: 9 total, 8 up, 9 in 2024-08-31T06:02:06.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:06 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-31T06:02:06.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:06 smithi016 ceph-mon[17194]: osd.8 [v2:172.21.15.184:6816/2479117823,v1:172.21.15.184:6817/2479117823] boot 2024-08-31T06:02:06.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:06 smithi016 ceph-mon[17194]: osdmap e54: 9 total, 9 up, 9 in 2024-08-31T06:02:07.063 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:02:07.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:06 smithi155 ceph-mon[21742]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-08-31T06:02:07.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:06 smithi155 ceph-mon[21742]: Detected new or changed devices on smithi184 2024-08-31T06:02:07.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:06 smithi155 ceph-mon[21742]: Adjusting osd_memory_target on smithi184 to 5696M 2024-08-31T06:02:07.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:06 smithi155 ceph-mon[21742]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-08-31T06:02:07.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:06 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-31T06:02:07.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:06 smithi155 ceph-mon[21742]: osdmap e53: 9 total, 8 up, 9 in 2024-08-31T06:02:07.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:06 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-31T06:02:07.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:06 smithi155 ceph-mon[21742]: osd.8 [v2:172.21.15.184:6816/2479117823,v1:172.21.15.184:6817/2479117823] boot 2024-08-31T06:02:07.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:06 smithi155 ceph-mon[21742]: osdmap e54: 9 total, 9 up, 9 in 2024-08-31T06:02:07.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:06 smithi184 ceph-mon[22331]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-08-31T06:02:07.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:06 smithi184 ceph-mon[22331]: Detected new or changed devices on smithi184 2024-08-31T06:02:07.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:06 smithi184 ceph-mon[22331]: Adjusting osd_memory_target on smithi184 to 5696M 2024-08-31T06:02:07.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:06 smithi184 ceph-mon[22331]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-08-31T06:02:07.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:06 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-31T06:02:07.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:06 smithi184 ceph-mon[22331]: osdmap e53: 9 total, 8 up, 9 in 2024-08-31T06:02:07.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:06 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-31T06:02:07.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:06 smithi184 ceph-mon[22331]: osd.8 [v2:172.21.15.184:6816/2479117823,v1:172.21.15.184:6817/2479117823] boot 2024-08-31T06:02:07.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:06 smithi184 ceph-mon[22331]: osdmap e54: 9 total, 9 up, 9 in 2024-08-31T06:02:07.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:07.679+0000 7f890f200640 1 -- 172.21.15.16:0/591730794 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8908109240 msgr2=0x7f8908109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:02:07.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:07.679+0000 7f890f200640 1 --2- 172.21.15.16:0/591730794 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8908109240 0x7f8908109650 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f88f0009900 tx=0x7f88f002a3f0 comp rx=0 tx=0).stop 2024-08-31T06:02:07.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:07.679+0000 7f890f200640 1 -- 172.21.15.16:0/591730794 shutdown_connections 2024-08-31T06:02:07.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:07.679+0000 7f890f200640 1 --2- 172.21.15.16:0/591730794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8908109b90 0x7f89081103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:02:07.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:07.679+0000 7f890f200640 1 --2- 172.21.15.16:0/591730794 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8908109240 0x7f8908109650 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:02:07.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:07.679+0000 7f890f200640 1 --2- 172.21.15.16:0/591730794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8908103900 0x7f8908103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:02:07.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:07.679+0000 7f890f200640 1 -- 172.21.15.16:0/591730794 >> 172.21.15.16:0/591730794 conn(0x7f890807b5f0 msgr2=0x7f890807ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:02:07.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:07.680+0000 7f890f200640 1 -- 172.21.15.16:0/591730794 shutdown_connections 2024-08-31T06:02:07.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:07.680+0000 7f890f200640 1 -- 172.21.15.16:0/591730794 wait complete. 2024-08-31T06:02:07.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:07.681+0000 7f890f200640 1 Processor -- start 2024-08-31T06:02:07.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:07.681+0000 7f890f200640 1 -- start start 2024-08-31T06:02:07.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:07.681+0000 7f890f200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8908103900 0x7f890819c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:02:07.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:07.681+0000 7f890f200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8908109240 0x7f890819c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:02:07.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:07.681+0000 7f890f200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8908109b90 0x7f89081a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:02:07.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:07.681+0000 7f890f200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f89081138a0 con 0x7f8908109b90 2024-08-31T06:02:07.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:07.681+0000 7f890f200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8908113720 con 0x7f8908109240 2024-08-31T06:02:07.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:07.681+0000 7f890f200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8908113a20 con 0x7f8908103900 2024-08-31T06:02:07.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:07.681+0000 7f8907e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8908103900 0x7f890819c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:02:07.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:07.681+0000 7f88ff400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8908109240 0x7f890819c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:02:07.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:07.682+0000 7f8907e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8908103900 0x7f890819c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:47302/0 (socket says 172.21.15.16:47302) 2024-08-31T06:02:07.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:07.682+0000 7f8907e00640 1 -- 172.21.15.16:0/179136701 learned_addr learned my addr 172.21.15.16:0/179136701 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:02:07.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:07.682+0000 7f890d200640 1 --2- 172.21.15.16:0/179136701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8908109b90 0x7f89081a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:02:07.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:07.682+0000 7f8907e00640 1 -- 172.21.15.16:0/179136701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8908109240 msgr2=0x7f890819c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:02:07.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:07.682+0000 7f8907e00640 1 --2- 172.21.15.16:0/179136701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8908109240 0x7f890819c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:02:07.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:07.682+0000 7f8907e00640 1 -- 172.21.15.16:0/179136701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8908109b90 msgr2=0x7f89081a1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:02:07.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:07.682+0000 7f8907e00640 1 --2- 172.21.15.16:0/179136701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8908109b90 0x7f89081a1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:02:07.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:07.682+0000 7f8907e00640 1 -- 172.21.15.16:0/179136701 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89081a23c0 con 0x7f8908103900 2024-08-31T06:02:07.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:07.682+0000 7f88ff400640 1 --2- 172.21.15.16:0/179136701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8908109240 0x7f890819c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:02:07.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:07.682+0000 7f8907e00640 1 --2- 172.21.15.16:0/179136701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8908103900 0x7f890819c450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f88f800e9f0 tx=0x7f88f800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:02:07.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:07.683+0000 7f8905600640 1 -- 172.21.15.16:0/179136701 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f800cd80 con 0x7f8908103900 2024-08-31T06:02:07.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:07.683+0000 7f890f200640 1 -- 172.21.15.16:0/179136701 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89081a26b0 con 0x7f8908103900 2024-08-31T06:02:07.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:07.683+0000 7f8905600640 1 -- 172.21.15.16:0/179136701 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88f80054d0 con 0x7f8908103900 2024-08-31T06:02:07.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:07.683+0000 7f8905600640 1 -- 172.21.15.16:0/179136701 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f8010690 con 0x7f8908103900 2024-08-31T06:02:07.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:07.683+0000 7f890f200640 1 -- 172.21.15.16:0/179136701 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89081a8e10 con 0x7f8908103900 2024-08-31T06:02:07.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:07.684+0000 7f890f200640 1 -- 172.21.15.16:0/179136701 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8908104b80 con 0x7f8908103900 2024-08-31T06:02:07.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:07.687+0000 7f8905600640 1 -- 172.21.15.16:0/179136701 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7f88f8004e90 con 0x7f8908103900 2024-08-31T06:02:07.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:07.688+0000 7f8905600640 1 --2- 172.21.15.16:0/179136701 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f88ec0b4b80 0x7f88ec0b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:02:07.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:07.688+0000 7f8905600640 1 -- 172.21.15.16:0/179136701 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6077+0+0 (secure 0 0 0) 0x7f88f80e4d00 con 0x7f8908103900 2024-08-31T06:02:07.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:07.688+0000 7f88ff400640 1 --2- 172.21.15.16:0/179136701 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f88ec0b4b80 0x7f88ec0b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:02:07.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:07.689+0000 7f88ff400640 1 --2- 172.21.15.16:0/179136701 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f88ec0b4b80 0x7f88ec0b7090 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f88f002a9e0 tx=0x7f88f0005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:02:07.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:07.690+0000 7f8905600640 1 -- 172.21.15.16:0/179136701 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f88f80ac650 con 0x7f8908103900 2024-08-31T06:02:07.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:07.898+0000 7f890f200640 1 -- 172.21.15.16:0/179136701 --> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f89081056e0 con 0x7f88ec0b4b80 2024-08-31T06:02:08.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:07 smithi155 ceph-mon[21742]: purged_snaps scrub starts 2024-08-31T06:02:08.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:07 smithi155 ceph-mon[21742]: purged_snaps scrub ok 2024-08-31T06:02:08.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:07 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-31T06:02:08.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:07 smithi016 ceph-mon[17194]: purged_snaps scrub starts 2024-08-31T06:02:08.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:07 smithi016 ceph-mon[17194]: purged_snaps scrub ok 2024-08-31T06:02:08.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:07 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-31T06:02:08.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:07 smithi184 ceph-mon[22331]: purged_snaps scrub starts 2024-08-31T06:02:08.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:07 smithi184 ceph-mon[22331]: purged_snaps scrub ok 2024-08-31T06:02:08.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:07 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-31T06:02:09.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:08 smithi155 ceph-mon[21742]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail 2024-08-31T06:02:09.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:08 smithi155 ceph-mon[21742]: osdmap e55: 9 total, 9 up, 9 in 2024-08-31T06:02:09.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:08 smithi155 ceph-mon[21742]: from='client.24526 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:02:09.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:08 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T06:02:09.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:08 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T06:02:09.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:08 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:02:09.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:08 smithi016 ceph-mon[17194]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail 2024-08-31T06:02:09.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:08 smithi016 ceph-mon[17194]: osdmap e55: 9 total, 9 up, 9 in 2024-08-31T06:02:09.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:08 smithi016 ceph-mon[17194]: from='client.24526 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:02:09.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:08 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T06:02:09.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:08 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T06:02:09.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:08 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:02:09.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:08 smithi184 ceph-mon[22331]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail 2024-08-31T06:02:09.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:08 smithi184 ceph-mon[22331]: osdmap e55: 9 total, 9 up, 9 in 2024-08-31T06:02:09.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:08 smithi184 ceph-mon[22331]: from='client.24526 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:02:09.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:08 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T06:02:09.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:08 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T06:02:09.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:08 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:02:10.041 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:09 smithi016 ceph-mon[17194]: osdmap e56: 9 total, 9 up, 9 in 2024-08-31T06:02:10.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:09 smithi184 ceph-mon[22331]: osdmap e56: 9 total, 9 up, 9 in 2024-08-31T06:02:10.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:09 smithi155 ceph-mon[21742]: osdmap e56: 9 total, 9 up, 9 in 2024-08-31T06:02:11.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:10 smithi016 ceph-mon[17194]: pgmap v187: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-08-31T06:02:11.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:10 smithi184 ceph-mon[22331]: pgmap v187: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-08-31T06:02:11.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:10 smithi155 ceph-mon[21742]: pgmap v187: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-08-31T06:02:12.151 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:11 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/4170492189' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bc27afcb-7871-4f72-a043-044fd5fe7d86"} : dispatch 2024-08-31T06:02:12.151 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:11 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/4170492189' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bc27afcb-7871-4f72-a043-044fd5fe7d86"}]': finished 2024-08-31T06:02:12.151 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:11 smithi016 ceph-mon[17194]: osdmap e57: 10 total, 9 up, 10 in 2024-08-31T06:02:12.151 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:11 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-31T06:02:12.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:11 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/4170492189' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bc27afcb-7871-4f72-a043-044fd5fe7d86"} : dispatch 2024-08-31T06:02:12.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:11 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/4170492189' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bc27afcb-7871-4f72-a043-044fd5fe7d86"}]': finished 2024-08-31T06:02:12.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:11 smithi184 ceph-mon[22331]: osdmap e57: 10 total, 9 up, 10 in 2024-08-31T06:02:12.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:11 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-31T06:02:12.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:11 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/4170492189' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bc27afcb-7871-4f72-a043-044fd5fe7d86"} : dispatch 2024-08-31T06:02:12.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:11 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/4170492189' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bc27afcb-7871-4f72-a043-044fd5fe7d86"}]': finished 2024-08-31T06:02:12.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:11 smithi155 ceph-mon[21742]: osdmap e57: 10 total, 9 up, 10 in 2024-08-31T06:02:12.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:11 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-31T06:02:13.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:12 smithi016 ceph-mon[17194]: pgmap v188: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-08-31T06:02:13.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:12 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/1872286310' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T06:02:13.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:12 smithi184 ceph-mon[22331]: pgmap v188: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-08-31T06:02:13.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:12 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/1872286310' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T06:02:13.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:12 smithi155 ceph-mon[21742]: pgmap v188: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-08-31T06:02:13.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:12 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/1872286310' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T06:02:15.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:14 smithi016 ceph-mon[17194]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-31T06:02:15.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:14 smithi184 ceph-mon[22331]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-31T06:02:15.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:14 smithi155 ceph-mon[21742]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-31T06:02:17.160 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:16 smithi016 ceph-mon[17194]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-08-31T06:02:17.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:16 smithi184 ceph-mon[22331]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-08-31T06:02:17.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:16 smithi155 ceph-mon[21742]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-08-31T06:02:19.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:18 smithi016 ceph-mon[17194]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-08-31T06:02:19.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:18 smithi184 ceph-mon[22331]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-08-31T06:02:19.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:18 smithi155 ceph-mon[21742]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-08-31T06:02:21.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:20 smithi016 ceph-mon[17194]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-31T06:02:21.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:20 smithi184 ceph-mon[22331]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-31T06:02:21.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:20 smithi155 ceph-mon[21742]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-31T06:02:22.156 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:21 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-08-31T06:02:22.157 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:21 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:02:22.157 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:21 smithi016 ceph-mon[17194]: Deploying daemon osd.9 on smithi016 2024-08-31T06:02:22.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:21 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-08-31T06:02:22.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:21 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:02:22.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:21 smithi184 ceph-mon[22331]: Deploying daemon osd.9 on smithi016 2024-08-31T06:02:22.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:21 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-08-31T06:02:22.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:21 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:02:22.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:21 smithi155 ceph-mon[21742]: Deploying daemon osd.9 on smithi016 2024-08-31T06:02:23.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:22 smithi016 ceph-mon[17194]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-31T06:02:23.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:22 smithi184 ceph-mon[22331]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-31T06:02:23.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:22 smithi155 ceph-mon[21742]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-31T06:02:25.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:24 smithi016 ceph-mon[17194]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-31T06:02:25.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:24 smithi184 ceph-mon[22331]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-31T06:02:25.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:24 smithi155 ceph-mon[21742]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-31T06:02:27.065 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:26 smithi016 ceph-mon[17194]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-08-31T06:02:27.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:26 smithi184 ceph-mon[22331]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-08-31T06:02:27.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:26 smithi155 ceph-mon[21742]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-08-31T06:02:29.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:28 smithi184 ceph-mon[22331]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-08-31T06:02:29.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:28 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:02:29.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:28 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:29.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:28 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:29.302 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:28 smithi016 ceph-mon[17194]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-08-31T06:02:29.302 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:28 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:02:29.302 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:28 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:29.302 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:28 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:29.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:28 smithi155 ceph-mon[21742]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-08-31T06:02:29.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:28 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:02:29.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:28 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:29.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:28 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:31.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:30 smithi184 ceph-mon[22331]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-08-31T06:02:31.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:30 smithi155 ceph-mon[21742]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-08-31T06:02:31.381 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:30 smithi016 ceph-mon[17194]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-08-31T06:02:33.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:32 smithi016 ceph-mon[17194]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-08-31T06:02:33.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:32 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:33.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:32 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:33.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:32 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:02:33.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:32 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:02:33.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:32 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:33.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:32 smithi184 ceph-mon[22331]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-08-31T06:02:33.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:32 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:33.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:32 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:33.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:32 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:02:33.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:32 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:02:33.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:32 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:33.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:32 smithi155 ceph-mon[21742]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-08-31T06:02:33.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:32 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:33.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:32 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:33.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:32 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:02:33.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:32 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:02:33.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:32 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:33.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:33.616+0000 7f8905600640 1 -- 172.21.15.16:0/179136701 <== mgr.14220 v2:172.21.15.16:6800/4279751337 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f89081056e0 con 0x7f88ec0b4b80 2024-08-31T06:02:33.619 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 9 on host 'smithi016' 2024-08-31T06:02:33.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:33.619+0000 7f890f200640 1 -- 172.21.15.16:0/179136701 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f88ec0b4b80 msgr2=0x7f88ec0b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:02:33.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:33.619+0000 7f890f200640 1 --2- 172.21.15.16:0/179136701 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f88ec0b4b80 0x7f88ec0b7090 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f88f002a9e0 tx=0x7f88f0005a20 comp rx=0 tx=0).stop 2024-08-31T06:02:33.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:33.619+0000 7f890f200640 1 -- 172.21.15.16:0/179136701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8908103900 msgr2=0x7f890819c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:02:33.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:33.619+0000 7f890f200640 1 --2- 172.21.15.16:0/179136701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8908103900 0x7f890819c450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f88f800e9f0 tx=0x7f88f800edf0 comp rx=0 tx=0).stop 2024-08-31T06:02:33.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:33.619+0000 7f890f200640 1 -- 172.21.15.16:0/179136701 shutdown_connections 2024-08-31T06:02:33.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:33.619+0000 7f890f200640 1 --2- 172.21.15.16:0/179136701 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f88ec0b4b80 0x7f88ec0b7090 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:02:33.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:33.619+0000 7f890f200640 1 --2- 172.21.15.16:0/179136701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8908109b90 0x7f89081a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:02:33.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:33.619+0000 7f890f200640 1 --2- 172.21.15.16:0/179136701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8908109240 0x7f890819c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:02:33.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:33.619+0000 7f890f200640 1 --2- 172.21.15.16:0/179136701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8908103900 0x7f890819c450 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:02:33.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:33.619+0000 7f890f200640 1 -- 172.21.15.16:0/179136701 >> 172.21.15.16:0/179136701 conn(0x7f890807b5f0 msgr2=0x7f8908104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:02:33.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:33.620+0000 7f890f200640 1 -- 172.21.15.16:0/179136701 shutdown_connections 2024-08-31T06:02:33.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:02:33.620+0000 7f890f200640 1 -- 172.21.15.16:0/179136701 wait complete. 2024-08-31T06:02:34.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:33 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:02:34.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:33 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:34.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:33 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:34.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:33 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:02:34.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:33 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:34.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:33 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:34.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:33 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:02:34.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:33 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:34.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:33 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:34.491 DEBUG:teuthology.orchestra.run.smithi016:osd.9> sudo journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@osd.9.service 2024-08-31T06:02:34.495 INFO:tasks.cephadm:Deploying osd.10 on smithi155 with /dev/vg_nvme/lv_1... 2024-08-31T06:02:34.495 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-08-31T06:02:34.752 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.b/config 2024-08-31T06:02:34.935 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:02:34 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:02:34.663+0000 7f63fb223640 -1 osd.9 0 log_to_monitors true 2024-08-31T06:02:35.174 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:34 smithi155 ceph-mon[21742]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-08-31T06:02:35.174 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:34 smithi155 ceph-mon[21742]: from='osd.9 [v2:172.21.15.16:6826/3015700546,v1:172.21.15.16:6827/3015700546]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-08-31T06:02:35.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:34 smithi184 ceph-mon[22331]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-08-31T06:02:35.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:34 smithi184 ceph-mon[22331]: from='osd.9 [v2:172.21.15.16:6826/3015700546,v1:172.21.15.16:6827/3015700546]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-08-31T06:02:35.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:34 smithi016 ceph-mon[17194]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-08-31T06:02:35.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:34 smithi016 ceph-mon[17194]: from='osd.9 [v2:172.21.15.16:6826/3015700546,v1:172.21.15.16:6827/3015700546]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-08-31T06:02:36.184 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:35 smithi155 ceph-mon[21742]: from='osd.9 [v2:172.21.15.16:6826/3015700546,v1:172.21.15.16:6827/3015700546]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-08-31T06:02:36.184 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:35 smithi155 ceph-mon[21742]: osdmap e58: 10 total, 9 up, 10 in 2024-08-31T06:02:36.184 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:35 smithi155 ceph-mon[21742]: from='osd.9 [v2:172.21.15.16:6826/3015700546,v1:172.21.15.16:6827/3015700546]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-08-31T06:02:36.184 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:35 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-31T06:02:36.184 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:35 smithi155 ceph-mon[21742]: from='osd.9 [v2:172.21.15.16:6826/3015700546,v1:172.21.15.16:6827/3015700546]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-08-31T06:02:36.218 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:35 smithi016 ceph-mon[17194]: from='osd.9 [v2:172.21.15.16:6826/3015700546,v1:172.21.15.16:6827/3015700546]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-08-31T06:02:36.218 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:35 smithi016 ceph-mon[17194]: osdmap e58: 10 total, 9 up, 10 in 2024-08-31T06:02:36.218 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:35 smithi016 ceph-mon[17194]: from='osd.9 [v2:172.21.15.16:6826/3015700546,v1:172.21.15.16:6827/3015700546]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-08-31T06:02:36.218 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:35 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-31T06:02:36.218 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:35 smithi016 ceph-mon[17194]: from='osd.9 [v2:172.21.15.16:6826/3015700546,v1:172.21.15.16:6827/3015700546]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-08-31T06:02:36.218 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:02:35 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:02:35.954+0000 7f63f5e00640 -1 osd.9 0 waiting for initial osdmap 2024-08-31T06:02:36.218 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:02:35 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:02:35.968+0000 7f63f0e00640 -1 osd.9 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-31T06:02:36.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:35 smithi184 ceph-mon[22331]: from='osd.9 [v2:172.21.15.16:6826/3015700546,v1:172.21.15.16:6827/3015700546]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-08-31T06:02:36.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:35 smithi184 ceph-mon[22331]: osdmap e58: 10 total, 9 up, 10 in 2024-08-31T06:02:36.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:35 smithi184 ceph-mon[22331]: from='osd.9 [v2:172.21.15.16:6826/3015700546,v1:172.21.15.16:6827/3015700546]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-08-31T06:02:36.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:35 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-31T06:02:36.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:35 smithi184 ceph-mon[22331]: from='osd.9 [v2:172.21.15.16:6826/3015700546,v1:172.21.15.16:6827/3015700546]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-08-31T06:02:37.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:36 smithi184 ceph-mon[22331]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-08-31T06:02:37.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:36 smithi184 ceph-mon[22331]: osdmap e59: 10 total, 9 up, 10 in 2024-08-31T06:02:37.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:36 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-31T06:02:37.278 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:36 smithi155 ceph-mon[21742]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-08-31T06:02:37.278 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:36 smithi155 ceph-mon[21742]: osdmap e59: 10 total, 9 up, 10 in 2024-08-31T06:02:37.278 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:36 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-31T06:02:37.281 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-08-31T06:02:37.308 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph orch daemon add osd smithi155:vg_nvme/lv_1 2024-08-31T06:02:37.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:36 smithi016 ceph-mon[17194]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-08-31T06:02:37.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:36 smithi016 ceph-mon[17194]: osdmap e59: 10 total, 9 up, 10 in 2024-08-31T06:02:37.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:36 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-31T06:02:37.562 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.b/config 2024-08-31T06:02:38.032 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:38.027+0000 7feae178b640 1 -- 172.21.15.155:0/2810060356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feadc10dc10 msgr2=0x7feadc1100e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:02:38.032 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:38.027+0000 7feae178b640 1 --2- 172.21.15.155:0/2810060356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feadc10dc10 0x7feadc1100e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fead4009ca0 tx=0x7fead402a8c0 comp rx=0 tx=0).stop 2024-08-31T06:02:38.032 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:38.027+0000 7feae178b640 1 -- 172.21.15.155:0/2810060356 shutdown_connections 2024-08-31T06:02:38.032 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:38.027+0000 7feae178b640 1 --2- 172.21.15.155:0/2810060356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feadc105040 0x7feadc10d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:02:38.032 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:38.027+0000 7feae178b640 1 --2- 172.21.15.155:0/2810060356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feadc10dc10 0x7feadc1100e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:02:38.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:38.027+0000 7feae178b640 1 --2- 172.21.15.155:0/2810060356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feadc104660 0x7feadc104a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:02:38.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:38.027+0000 7feae178b640 1 -- 172.21.15.155:0/2810060356 >> 172.21.15.155:0/2810060356 conn(0x7feadc079530 msgr2=0x7feadc079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:02:38.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:38.027+0000 7feae178b640 1 -- 172.21.15.155:0/2810060356 shutdown_connections 2024-08-31T06:02:38.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:38.027+0000 7feae178b640 1 -- 172.21.15.155:0/2810060356 wait complete. 2024-08-31T06:02:38.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:38.028+0000 7feae178b640 1 Processor -- start 2024-08-31T06:02:38.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:38.028+0000 7feae178b640 1 -- start start 2024-08-31T06:02:38.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:38.028+0000 7feae178b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feadc104660 0x7feadc19a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:02:38.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:38.028+0000 7feae178b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feadc105040 0x7feadc19a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:02:38.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:38.028+0000 7feae178b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feadc10dc10 0x7feadc19fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:02:38.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:38.028+0000 7feae178b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feadc1138f0 con 0x7feadc10dc10 2024-08-31T06:02:38.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:38.028+0000 7feae178b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7feadc113770 con 0x7feadc104660 2024-08-31T06:02:38.033 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:38.028+0000 7feae178b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7feadc113a70 con 0x7feadc105040 2024-08-31T06:02:38.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:38.028+0000 7feadaffd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feadc104660 0x7feadc19a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:02:38.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:38.028+0000 7feadaffd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feadc104660 0x7feadc19a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:36846/0 (socket says 172.21.15.155:36846) 2024-08-31T06:02:38.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:38.028+0000 7feadaffd640 1 -- 172.21.15.155:0/224187125 learned_addr learned my addr 172.21.15.155:0/224187125 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-08-31T06:02:38.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:38.029+0000 7feada7fc640 1 --2- 172.21.15.155:0/224187125 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feadc105040 0x7feadc19a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:02:38.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:38.029+0000 7feadb7fe640 1 --2- 172.21.15.155:0/224187125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feadc10dc10 0x7feadc19fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:02:38.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:38.029+0000 7feadaffd640 1 -- 172.21.15.155:0/224187125 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feadc105040 msgr2=0x7feadc19a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:02:38.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:38.029+0000 7feadaffd640 1 --2- 172.21.15.155:0/224187125 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feadc105040 0x7feadc19a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:02:38.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:38.029+0000 7feadaffd640 1 -- 172.21.15.155:0/224187125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feadc10dc10 msgr2=0x7feadc19fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:02:38.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:38.029+0000 7feadaffd640 1 --2- 172.21.15.155:0/224187125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feadc10dc10 0x7feadc19fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:02:38.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:38.029+0000 7feadaffd640 1 -- 172.21.15.155:0/224187125 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feadc1a01f0 con 0x7feadc104660 2024-08-31T06:02:38.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:38.029+0000 7feadaffd640 1 --2- 172.21.15.155:0/224187125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feadc104660 0x7feadc19a280 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feacc00b4a0 tx=0x7feacc00b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:02:38.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:38.029+0000 7fead3fff640 1 -- 172.21.15.155:0/224187125 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc014070 con 0x7feadc104660 2024-08-31T06:02:38.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:38.029+0000 7fead3fff640 1 -- 172.21.15.155:0/224187125 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feacc010ac0 con 0x7feadc104660 2024-08-31T06:02:38.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:38.029+0000 7fead3fff640 1 -- 172.21.15.155:0/224187125 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc017c30 con 0x7feadc104660 2024-08-31T06:02:38.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:38.029+0000 7feae178b640 1 -- 172.21.15.155:0/224187125 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feadc1a04e0 con 0x7feadc104660 2024-08-31T06:02:38.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:38.029+0000 7feae178b640 1 -- 172.21.15.155:0/224187125 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feadc1a6c40 con 0x7feadc104660 2024-08-31T06:02:38.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:38.031+0000 7feae178b640 1 -- 172.21.15.155:0/224187125 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feadc104a70 con 0x7feadc104660 2024-08-31T06:02:38.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:38.032+0000 7fead3fff640 1 -- 172.21.15.155:0/224187125 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7feacc017070 con 0x7feadc104660 2024-08-31T06:02:38.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:38.033+0000 7fead3fff640 1 --2- 172.21.15.155:0/224187125 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7feac80b4b80 0x7feac80b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:02:38.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:38.033+0000 7feada7fc640 1 --2- 172.21.15.155:0/224187125 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7feac80b4b80 0x7feac80b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:02:38.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:38.033+0000 7fead3fff640 1 -- 172.21.15.155:0/224187125 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6476+0+0 (secure 0 0 0) 0x7feacc0e4030 con 0x7feadc104660 2024-08-31T06:02:38.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:38.034+0000 7feada7fc640 1 --2- 172.21.15.155:0/224187125 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7feac80b4b80 0x7feac80b7090 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7feadc10f950 tx=0x7feac400b390 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:02:38.040 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:38.038+0000 7fead3fff640 1 -- 172.21.15.155:0/224187125 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7feacc0ab890 con 0x7feadc104660 2024-08-31T06:02:38.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:37 smithi184 ceph-mon[22331]: purged_snaps scrub starts 2024-08-31T06:02:38.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:37 smithi184 ceph-mon[22331]: purged_snaps scrub ok 2024-08-31T06:02:38.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:37 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-31T06:02:38.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:37 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-31T06:02:38.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:37 smithi184 ceph-mon[22331]: osd.9 [v2:172.21.15.16:6826/3015700546,v1:172.21.15.16:6827/3015700546] boot 2024-08-31T06:02:38.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:37 smithi184 ceph-mon[22331]: osdmap e60: 10 total, 10 up, 10 in 2024-08-31T06:02:38.233 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:37 smithi155 ceph-mon[21742]: purged_snaps scrub starts 2024-08-31T06:02:38.233 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:37 smithi155 ceph-mon[21742]: purged_snaps scrub ok 2024-08-31T06:02:38.233 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:37 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-31T06:02:38.233 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:37 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-31T06:02:38.233 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:37 smithi155 ceph-mon[21742]: osd.9 [v2:172.21.15.16:6826/3015700546,v1:172.21.15.16:6827/3015700546] boot 2024-08-31T06:02:38.233 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:37 smithi155 ceph-mon[21742]: osdmap e60: 10 total, 10 up, 10 in 2024-08-31T06:02:38.233 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:38.230+0000 7feae178b640 1 -- 172.21.15.155:0/224187125 --> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7feadc107e70 con 0x7feac80b4b80 2024-08-31T06:02:38.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:37 smithi016 ceph-mon[17194]: purged_snaps scrub starts 2024-08-31T06:02:38.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:37 smithi016 ceph-mon[17194]: purged_snaps scrub ok 2024-08-31T06:02:38.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:37 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-31T06:02:38.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:37 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-31T06:02:38.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:37 smithi016 ceph-mon[17194]: osd.9 [v2:172.21.15.16:6826/3015700546,v1:172.21.15.16:6827/3015700546] boot 2024-08-31T06:02:38.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:37 smithi016 ceph-mon[17194]: osdmap e60: 10 total, 10 up, 10 in 2024-08-31T06:02:39.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:38 smithi184 ceph-mon[22331]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-08-31T06:02:39.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:38 smithi184 ceph-mon[22331]: osdmap e61: 10 total, 10 up, 10 in 2024-08-31T06:02:39.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:38 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T06:02:39.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:38 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T06:02:39.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:38 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:02:39.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:38 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:39.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:38 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:39.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:38 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T06:02:39.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:38 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T06:02:39.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:38 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T06:02:39.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:38 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T06:02:39.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:38 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-31T06:02:39.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:38 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-31T06:02:39.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:38 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-31T06:02:39.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:38 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-31T06:02:39.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:38 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:39.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:38 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:02:39.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:38 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:02:39.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:38 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:39.263 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:38 smithi155 ceph-mon[21742]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-08-31T06:02:39.263 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:38 smithi155 ceph-mon[21742]: osdmap e61: 10 total, 10 up, 10 in 2024-08-31T06:02:39.263 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:38 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T06:02:39.263 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:38 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T06:02:39.263 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:38 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:02:39.263 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:38 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:39.264 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:38 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:39.264 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:38 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T06:02:39.264 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:38 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T06:02:39.264 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:38 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T06:02:39.264 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:38 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T06:02:39.264 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:38 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-31T06:02:39.264 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:38 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-31T06:02:39.264 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:38 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-31T06:02:39.264 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:38 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-31T06:02:39.264 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:38 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:39.264 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:38 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:02:39.264 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:38 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:02:39.264 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:38 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:39.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:38 smithi016 ceph-mon[17194]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-08-31T06:02:39.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:38 smithi016 ceph-mon[17194]: osdmap e61: 10 total, 10 up, 10 in 2024-08-31T06:02:39.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:38 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T06:02:39.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:38 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T06:02:39.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:38 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:02:39.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:38 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:39.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:38 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:39.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:38 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T06:02:39.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:38 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T06:02:39.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:38 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T06:02:39.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:38 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T06:02:39.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:38 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-31T06:02:39.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:38 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-31T06:02:39.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:38 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-31T06:02:39.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:38 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-31T06:02:39.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:38 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:39.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:38 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:02:39.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:38 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:02:39.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:38 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:40.118 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:39 smithi155 ceph-mon[21742]: from='client.24557 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:02:40.118 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:39 smithi155 ceph-mon[21742]: Detected new or changed devices on smithi016 2024-08-31T06:02:40.118 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:39 smithi155 ceph-mon[21742]: Adjusting osd_memory_target on smithi016 to 4273M 2024-08-31T06:02:40.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:39 smithi184 ceph-mon[22331]: from='client.24557 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:02:40.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:39 smithi184 ceph-mon[22331]: Detected new or changed devices on smithi016 2024-08-31T06:02:40.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:39 smithi184 ceph-mon[22331]: Adjusting osd_memory_target on smithi016 to 4273M 2024-08-31T06:02:40.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:39 smithi016 ceph-mon[17194]: from='client.24557 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:02:40.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:39 smithi016 ceph-mon[17194]: Detected new or changed devices on smithi016 2024-08-31T06:02:40.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:39 smithi016 ceph-mon[17194]: Adjusting osd_memory_target on smithi016 to 4273M 2024-08-31T06:02:41.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:40 smithi184 ceph-mon[22331]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-08-31T06:02:41.225 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:40 smithi184 ceph-mon[22331]: from='client.? 172.21.15.155:0/1231464689' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f2548997-72ee-400e-ab8d-5966d85b2cf5"} : dispatch 2024-08-31T06:02:41.225 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:40 smithi184 ceph-mon[22331]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f2548997-72ee-400e-ab8d-5966d85b2cf5"} : dispatch 2024-08-31T06:02:41.225 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:40 smithi184 ceph-mon[22331]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f2548997-72ee-400e-ab8d-5966d85b2cf5"}]': finished 2024-08-31T06:02:41.225 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:40 smithi184 ceph-mon[22331]: osdmap e62: 11 total, 10 up, 11 in 2024-08-31T06:02:41.225 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:40 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-31T06:02:41.225 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:40 smithi155 ceph-mon[21742]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-08-31T06:02:41.225 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:40 smithi155 ceph-mon[21742]: from='client.? 172.21.15.155:0/1231464689' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f2548997-72ee-400e-ab8d-5966d85b2cf5"} : dispatch 2024-08-31T06:02:41.225 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:40 smithi155 ceph-mon[21742]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f2548997-72ee-400e-ab8d-5966d85b2cf5"} : dispatch 2024-08-31T06:02:41.225 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:40 smithi155 ceph-mon[21742]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f2548997-72ee-400e-ab8d-5966d85b2cf5"}]': finished 2024-08-31T06:02:41.226 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:40 smithi155 ceph-mon[21742]: osdmap e62: 11 total, 10 up, 11 in 2024-08-31T06:02:41.226 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:40 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-31T06:02:41.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:40 smithi016 ceph-mon[17194]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-08-31T06:02:41.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:40 smithi016 ceph-mon[17194]: from='client.? 172.21.15.155:0/1231464689' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f2548997-72ee-400e-ab8d-5966d85b2cf5"} : dispatch 2024-08-31T06:02:41.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:40 smithi016 ceph-mon[17194]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f2548997-72ee-400e-ab8d-5966d85b2cf5"} : dispatch 2024-08-31T06:02:41.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:40 smithi016 ceph-mon[17194]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f2548997-72ee-400e-ab8d-5966d85b2cf5"}]': finished 2024-08-31T06:02:41.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:40 smithi016 ceph-mon[17194]: osdmap e62: 11 total, 10 up, 11 in 2024-08-31T06:02:41.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:40 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-31T06:02:42.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:41 smithi184 ceph-mon[22331]: from='client.? 172.21.15.155:0/3088505258' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T06:02:42.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:41 smithi155 ceph-mon[21742]: from='client.? 172.21.15.155:0/3088505258' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T06:02:42.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:41 smithi016 ceph-mon[17194]: from='client.? 172.21.15.155:0/3088505258' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T06:02:43.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:42 smithi184 ceph-mon[22331]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-08-31T06:02:43.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:42 smithi155 ceph-mon[21742]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-08-31T06:02:43.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:42 smithi016 ceph-mon[17194]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-08-31T06:02:45.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:44 smithi184 ceph-mon[22331]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-08-31T06:02:45.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:44 smithi155 ceph-mon[21742]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-08-31T06:02:45.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:44 smithi016 ceph-mon[17194]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-08-31T06:02:47.276 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:46 smithi155 ceph-mon[21742]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T06:02:47.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:46 smithi016 ceph-mon[17194]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T06:02:47.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:46 smithi184 ceph-mon[22331]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T06:02:49.272 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:48 smithi155 ceph-mon[21742]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T06:02:49.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:48 smithi016 ceph-mon[17194]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T06:02:49.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:48 smithi184 ceph-mon[22331]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T06:02:50.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:49 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-08-31T06:02:50.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:49 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:02:50.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:49 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-08-31T06:02:50.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:49 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:02:50.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:49 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-08-31T06:02:50.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:49 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:02:51.310 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:50 smithi155 ceph-mon[21742]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T06:02:51.311 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:50 smithi155 ceph-mon[21742]: Deploying daemon osd.10 on smithi155 2024-08-31T06:02:51.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:50 smithi016 ceph-mon[17194]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T06:02:51.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:50 smithi016 ceph-mon[17194]: Deploying daemon osd.10 on smithi155 2024-08-31T06:02:51.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:50 smithi184 ceph-mon[22331]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T06:02:51.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:50 smithi184 ceph-mon[22331]: Deploying daemon osd.10 on smithi155 2024-08-31T06:02:53.252 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:52 smithi155 ceph-mon[21742]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T06:02:53.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:52 smithi016 ceph-mon[17194]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T06:02:53.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:52 smithi184 ceph-mon[22331]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T06:02:55.308 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:54 smithi155 ceph-mon[21742]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T06:02:55.309 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:54 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:02:55.309 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:54 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:55.309 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:54 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:55.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:54 smithi016 ceph-mon[17194]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T06:02:55.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:54 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:02:55.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:54 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:55.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:54 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:55.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:54 smithi184 ceph-mon[22331]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T06:02:55.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:54 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:02:55.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:54 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:55.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:54 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:57.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:56 smithi155 ceph-mon[21742]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T06:02:57.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:56 smithi016 ceph-mon[17194]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T06:02:57.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:56 smithi184 ceph-mon[22331]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T06:02:58.602 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:58.599+0000 7fead3fff640 1 -- 172.21.15.155:0/224187125 <== mgr.14220 v2:172.21.15.16:6800/4279751337 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7feadc107e70 con 0x7feac80b4b80 2024-08-31T06:02:58.602 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 10 on host 'smithi155' 2024-08-31T06:02:58.604 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:58.602+0000 7feae178b640 1 -- 172.21.15.155:0/224187125 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7feac80b4b80 msgr2=0x7feac80b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:02:58.604 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:58.602+0000 7feae178b640 1 --2- 172.21.15.155:0/224187125 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7feac80b4b80 0x7feac80b7090 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7feadc10f950 tx=0x7feac400b390 comp rx=0 tx=0).stop 2024-08-31T06:02:58.605 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:58.602+0000 7feae178b640 1 -- 172.21.15.155:0/224187125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feadc104660 msgr2=0x7feadc19a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:02:58.605 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:58.602+0000 7feae178b640 1 --2- 172.21.15.155:0/224187125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feadc104660 0x7feadc19a280 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feacc00b4a0 tx=0x7feacc00b960 comp rx=0 tx=0).stop 2024-08-31T06:02:58.605 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:58.602+0000 7feae178b640 1 -- 172.21.15.155:0/224187125 shutdown_connections 2024-08-31T06:02:58.605 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:58.602+0000 7feae178b640 1 --2- 172.21.15.155:0/224187125 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7feac80b4b80 0x7feac80b7090 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:02:58.605 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:58.602+0000 7feae178b640 1 --2- 172.21.15.155:0/224187125 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feadc105040 0x7feadc19a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:02:58.605 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:58.602+0000 7feae178b640 1 --2- 172.21.15.155:0/224187125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feadc10dc10 0x7feadc19fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:02:58.605 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:58.602+0000 7feae178b640 1 --2- 172.21.15.155:0/224187125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feadc104660 0x7feadc19a280 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:02:58.605 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:58.602+0000 7feae178b640 1 -- 172.21.15.155:0/224187125 >> 172.21.15.155:0/224187125 conn(0x7feadc079530 msgr2=0x7feadc107760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:02:58.605 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:58.602+0000 7feae178b640 1 -- 172.21.15.155:0/224187125 shutdown_connections 2024-08-31T06:02:58.605 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-31T06:02:58.602+0000 7feae178b640 1 -- 172.21.15.155:0/224187125 wait complete. 2024-08-31T06:02:58.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:58 smithi155 ceph-mon[21742]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T06:02:58.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:58 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:58.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:58 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:58.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:58 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:02:58.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:58 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:02:58.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:58 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:58.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:58 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:02:58.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:58 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:58.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:02:58 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:59.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:58 smithi016 ceph-mon[17194]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T06:02:59.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:58 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:59.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:58 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:59.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:58 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:02:59.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:58 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:02:59.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:58 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:59.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:58 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:02:59.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:58 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:59.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:02:58 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:59.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:58 smithi184 ceph-mon[22331]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T06:02:59.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:58 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:59.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:58 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:59.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:58 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:02:59.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:58 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:02:59.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:58 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:59.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:58 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:02:59.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:58 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:59.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:02:58 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:02:59.269 DEBUG:teuthology.orchestra.run.smithi155:osd.10> sudo journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@osd.10.service 2024-08-31T06:02:59.271 INFO:tasks.cephadm:Deploying osd.11 on smithi184 with /dev/vg_nvme/lv_1... 2024-08-31T06:02:59.271 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-08-31T06:02:59.529 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.c/config 2024-08-31T06:03:00.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:00 smithi184 ceph-mon[22331]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T06:03:01.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:00 smithi155 ceph-mon[21742]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T06:03:01.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:03:00 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:03:00.915+0000 7fc673939640 -1 osd.10 0 log_to_monitors true 2024-08-31T06:03:01.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:00 smithi016 ceph-mon[17194]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T06:03:01.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:01 smithi184 ceph-mon[22331]: from='osd.10 [v2:172.21.15.155:6824/2427122475,v1:172.21.15.155:6825/2427122475]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-08-31T06:03:01.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:01 smithi184 ceph-mon[22331]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-08-31T06:03:02.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:01 smithi155 ceph-mon[21742]: from='osd.10 [v2:172.21.15.155:6824/2427122475,v1:172.21.15.155:6825/2427122475]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-08-31T06:03:02.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:01 smithi155 ceph-mon[21742]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-08-31T06:03:02.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:01 smithi016 ceph-mon[17194]: from='osd.10 [v2:172.21.15.155:6824/2427122475,v1:172.21.15.155:6825/2427122475]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-08-31T06:03:02.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:01 smithi016 ceph-mon[17194]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-08-31T06:03:02.164 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-08-31T06:03:02.191 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph orch daemon add osd smithi184:vg_nvme/lv_1 2024-08-31T06:03:02.445 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.c/config 2024-08-31T06:03:02.866 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:02 smithi184 ceph-mon[22331]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T06:03:02.866 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:02 smithi184 ceph-mon[22331]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-08-31T06:03:02.866 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:02 smithi184 ceph-mon[22331]: from='osd.10 [v2:172.21.15.155:6824/2427122475,v1:172.21.15.155:6825/2427122475]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]} : dispatch 2024-08-31T06:03:02.867 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:02 smithi184 ceph-mon[22331]: osdmap e63: 11 total, 10 up, 11 in 2024-08-31T06:03:02.867 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:02 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-31T06:03:02.867 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:02 smithi184 ceph-mon[22331]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]} : dispatch 2024-08-31T06:03:02.867 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:02 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:02.867 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:02 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:02.867 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:02 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:02.867 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:02 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:02.867 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:02 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:02.867 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:02 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:02.867 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:02 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:02.868 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:02 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:02.868 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:02 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:02.868 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:02 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:02.868 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:02 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:02.868 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:02 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:03:02.868 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:02 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:03:02.868 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:02 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:02.985 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:02.957+0000 7ff140185640 1 -- 172.21.15.184:0/3655515067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff138108ac0 msgr2=0x7ff13810f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:02.985 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:02.957+0000 7ff140185640 1 --2- 172.21.15.184:0/3655515067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff138108ac0 0x7ff13810f2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff130009ca0 tx=0x7ff13002a8c0 comp rx=0 tx=0).stop 2024-08-31T06:03:02.985 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:02.957+0000 7ff140185640 1 -- 172.21.15.184:0/3655515067 shutdown_connections 2024-08-31T06:03:02.985 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:02.957+0000 7ff140185640 1 --2- 172.21.15.184:0/3655515067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff138108ac0 0x7ff13810f2f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:02.985 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:02.957+0000 7ff140185640 1 --2- 172.21.15.184:0/3655515067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff138108170 0x7ff138108580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:02.985 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:02.957+0000 7ff140185640 1 --2- 172.21.15.184:0/3655515067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff138102830 0x7ff138102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:02.985 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:02.957+0000 7ff140185640 1 -- 172.21.15.184:0/3655515067 >> 172.21.15.184:0/3655515067 conn(0x7ff1380794f0 msgr2=0x7ff138079900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:03:02.985 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:02.957+0000 7ff140185640 1 -- 172.21.15.184:0/3655515067 shutdown_connections 2024-08-31T06:03:02.985 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:02.957+0000 7ff140185640 1 -- 172.21.15.184:0/3655515067 wait complete. 2024-08-31T06:03:02.986 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:02.958+0000 7ff140185640 1 Processor -- start 2024-08-31T06:03:02.986 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:02.958+0000 7ff140185640 1 -- start start 2024-08-31T06:03:02.986 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:02.958+0000 7ff140185640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff138102830 0x7ff13819aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:02.986 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:02.958+0000 7ff140185640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff138108170 0x7ff13819b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:02.986 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:02.958+0000 7ff140185640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff138108ac0 0x7ff13819ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:02.986 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:02.958+0000 7ff140185640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff138112740 con 0x7ff138108ac0 2024-08-31T06:03:02.986 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:02.958+0000 7ff140185640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff1381125c0 con 0x7ff138108170 2024-08-31T06:03:02.986 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:02.958+0000 7ff140185640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff1381128c0 con 0x7ff138102830 2024-08-31T06:03:02.986 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:02.958+0000 7ff13df21640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff138102830 0x7ff13819aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:02.986 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:02.958+0000 7ff13df21640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff138102830 0x7ff13819aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:55230/0 (socket says 172.21.15.184:55230) 2024-08-31T06:03:02.986 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:02.958+0000 7ff13df21640 1 -- 172.21.15.184:0/4079463717 learned_addr learned my addr 172.21.15.184:0/4079463717 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-08-31T06:03:02.986 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:02.958+0000 7ff13d720640 1 --2- 172.21.15.184:0/4079463717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff138108170 0x7ff13819b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:02.987 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:02.958+0000 7ff13e722640 1 --2- 172.21.15.184:0/4079463717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff138108ac0 0x7ff13819ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:02.987 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:02.958+0000 7ff13df21640 1 -- 172.21.15.184:0/4079463717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff138108170 msgr2=0x7ff13819b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:02.987 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:02.958+0000 7ff13df21640 1 --2- 172.21.15.184:0/4079463717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff138108170 0x7ff13819b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:02.987 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:02.958+0000 7ff13df21640 1 -- 172.21.15.184:0/4079463717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff138108ac0 msgr2=0x7ff13819ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:02.987 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:02.958+0000 7ff13df21640 1 --2- 172.21.15.184:0/4079463717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff138108ac0 0x7ff13819ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:02.987 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:02.958+0000 7ff13df21640 1 -- 172.21.15.184:0/4079463717 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1381a6280 con 0x7ff138102830 2024-08-31T06:03:02.987 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:02.959+0000 7ff13df21640 1 --2- 172.21.15.184:0/4079463717 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff138102830 0x7ff13819aff0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff13400e9f0 tx=0x7ff13400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:03:02.987 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:02.959+0000 7ff12effd640 1 -- 172.21.15.184:0/4079463717 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13400cd80 con 0x7ff138102830 2024-08-31T06:03:02.987 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:02.959+0000 7ff12effd640 1 -- 172.21.15.184:0/4079463717 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1340054d0 con 0x7ff138102830 2024-08-31T06:03:02.987 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:02.959+0000 7ff12effd640 1 -- 172.21.15.184:0/4079463717 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff134010690 con 0x7ff138102830 2024-08-31T06:03:02.987 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:02.959+0000 7ff140185640 1 -- 172.21.15.184:0/4079463717 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1381a6570 con 0x7ff138102830 2024-08-31T06:03:02.987 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:02.959+0000 7ff140185640 1 -- 172.21.15.184:0/4079463717 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1381a6a60 con 0x7ff138102830 2024-08-31T06:03:02.988 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:02.960+0000 7ff140185640 1 -- 172.21.15.184:0/4079463717 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff138103ab0 con 0x7ff138102830 2024-08-31T06:03:02.988 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:02.961+0000 7ff12effd640 1 -- 172.21.15.184:0/4079463717 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7ff134004e50 con 0x7ff138102830 2024-08-31T06:03:02.988 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:02.962+0000 7ff12effd640 1 --2- 172.21.15.184:0/4079463717 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff1240b4bb0 0x7ff1240b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:02.988 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:02.962+0000 7ff12effd640 1 -- 172.21.15.184:0/4079463717 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6629+0+0 (secure 0 0 0) 0x7ff1340e4f90 con 0x7ff138102830 2024-08-31T06:03:02.988 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:02.962+0000 7ff13d720640 1 --2- 172.21.15.184:0/4079463717 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff1240b4bb0 0x7ff1240b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:02.988 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:02.963+0000 7ff13d720640 1 --2- 172.21.15.184:0/4079463717 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff1240b4bb0 0x7ff1240b70c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff138105c30 tx=0x7ff12800a2d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:03:02.988 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:02.966+0000 7ff12effd640 1 -- 172.21.15.184:0/4079463717 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7ff1340ac750 con 0x7ff138102830 2024-08-31T06:03:03.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:03:02 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:03:02.782+0000 7fc66f8bb640 -1 osd.10 0 waiting for initial osdmap 2024-08-31T06:03:03.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:03:02 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:03:02.794+0000 7fc66b6b9640 -1 osd.10 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-31T06:03:03.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:02 smithi155 ceph-mon[21742]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T06:03:03.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:02 smithi155 ceph-mon[21742]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-08-31T06:03:03.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:02 smithi155 ceph-mon[21742]: from='osd.10 [v2:172.21.15.155:6824/2427122475,v1:172.21.15.155:6825/2427122475]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]} : dispatch 2024-08-31T06:03:03.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:02 smithi155 ceph-mon[21742]: osdmap e63: 11 total, 10 up, 11 in 2024-08-31T06:03:03.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:02 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-31T06:03:03.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:02 smithi155 ceph-mon[21742]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]} : dispatch 2024-08-31T06:03:03.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:02 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:03.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:02 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:03.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:02 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:03.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:02 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:03.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:02 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:03.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:02 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:03.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:02 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:03.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:02 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:03.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:02 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:03.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:02 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:03.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:02 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:03.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:02 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:03:03.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:02 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:03:03.124 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:02 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:03.153 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:03.151+0000 7ff140185640 1 -- 172.21.15.184:0/4079463717 --> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7ff1380635a0 con 0x7ff1240b4bb0 2024-08-31T06:03:03.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:02 smithi016 ceph-mon[17194]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T06:03:03.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:02 smithi016 ceph-mon[17194]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-08-31T06:03:03.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:02 smithi016 ceph-mon[17194]: from='osd.10 [v2:172.21.15.155:6824/2427122475,v1:172.21.15.155:6825/2427122475]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]} : dispatch 2024-08-31T06:03:03.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:02 smithi016 ceph-mon[17194]: osdmap e63: 11 total, 10 up, 11 in 2024-08-31T06:03:03.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:02 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-31T06:03:03.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:02 smithi016 ceph-mon[17194]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]} : dispatch 2024-08-31T06:03:03.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:02 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:03.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:02 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:03.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:02 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:03.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:02 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:03.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:02 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:03.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:02 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:03.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:02 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:03.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:02 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:03.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:02 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:03.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:02 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:03.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:02 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:03.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:02 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:03:03.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:02 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:03:03.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:02 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:03.833 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:03 smithi184 ceph-mon[22331]: Detected new or changed devices on smithi155 2024-08-31T06:03:03.833 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:03 smithi184 ceph-mon[22331]: Adjusting osd_memory_target on smithi155 to 4272M 2024-08-31T06:03:03.833 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:03 smithi184 ceph-mon[22331]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-08-31T06:03:03.833 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:03 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-31T06:03:03.833 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:03 smithi184 ceph-mon[22331]: osdmap e64: 11 total, 10 up, 11 in 2024-08-31T06:03:03.833 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:03 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-31T06:03:03.833 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:03 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T06:03:03.833 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:03 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T06:03:03.833 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:03 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:03:04.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:03 smithi155 ceph-mon[21742]: Detected new or changed devices on smithi155 2024-08-31T06:03:04.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:03 smithi155 ceph-mon[21742]: Adjusting osd_memory_target on smithi155 to 4272M 2024-08-31T06:03:04.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:03 smithi155 ceph-mon[21742]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-08-31T06:03:04.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:03 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-31T06:03:04.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:03 smithi155 ceph-mon[21742]: osdmap e64: 11 total, 10 up, 11 in 2024-08-31T06:03:04.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:03 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-31T06:03:04.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:03 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T06:03:04.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:03 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T06:03:04.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:03 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:03:04.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:03 smithi016 ceph-mon[17194]: Detected new or changed devices on smithi155 2024-08-31T06:03:04.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:03 smithi016 ceph-mon[17194]: Adjusting osd_memory_target on smithi155 to 4272M 2024-08-31T06:03:04.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:03 smithi016 ceph-mon[17194]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-08-31T06:03:04.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:03 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-31T06:03:04.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:03 smithi016 ceph-mon[17194]: osdmap e64: 11 total, 10 up, 11 in 2024-08-31T06:03:04.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:03 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-31T06:03:04.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:03 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-31T06:03:04.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:03 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-31T06:03:04.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:03 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:03:04.895 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:04 smithi184 ceph-mon[22331]: purged_snaps scrub starts 2024-08-31T06:03:04.895 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:04 smithi184 ceph-mon[22331]: purged_snaps scrub ok 2024-08-31T06:03:04.895 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:04 smithi184 ceph-mon[22331]: from='client.24574 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:03:04.895 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:04 smithi184 ceph-mon[22331]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T06:03:04.896 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:04 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-31T06:03:04.896 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:04 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-31T06:03:04.896 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:04 smithi184 ceph-mon[22331]: osd.10 [v2:172.21.15.155:6824/2427122475,v1:172.21.15.155:6825/2427122475] boot 2024-08-31T06:03:04.896 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:04 smithi184 ceph-mon[22331]: osdmap e65: 11 total, 11 up, 11 in 2024-08-31T06:03:05.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:04 smithi155 ceph-mon[21742]: purged_snaps scrub starts 2024-08-31T06:03:05.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:04 smithi155 ceph-mon[21742]: purged_snaps scrub ok 2024-08-31T06:03:05.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:04 smithi155 ceph-mon[21742]: from='client.24574 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:03:05.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:04 smithi155 ceph-mon[21742]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T06:03:05.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:04 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-31T06:03:05.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:04 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-31T06:03:05.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:04 smithi155 ceph-mon[21742]: osd.10 [v2:172.21.15.155:6824/2427122475,v1:172.21.15.155:6825/2427122475] boot 2024-08-31T06:03:05.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:04 smithi155 ceph-mon[21742]: osdmap e65: 11 total, 11 up, 11 in 2024-08-31T06:03:05.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:04 smithi016 ceph-mon[17194]: purged_snaps scrub starts 2024-08-31T06:03:05.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:04 smithi016 ceph-mon[17194]: purged_snaps scrub ok 2024-08-31T06:03:05.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:04 smithi016 ceph-mon[17194]: from='client.24574 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:03:05.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:04 smithi016 ceph-mon[17194]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-31T06:03:05.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:04 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-31T06:03:05.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:04 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-31T06:03:05.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:04 smithi016 ceph-mon[17194]: osd.10 [v2:172.21.15.155:6824/2427122475,v1:172.21.15.155:6825/2427122475] boot 2024-08-31T06:03:05.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:04 smithi016 ceph-mon[17194]: osdmap e65: 11 total, 11 up, 11 in 2024-08-31T06:03:06.121 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:05 smithi155 ceph-mon[21742]: osdmap e66: 11 total, 11 up, 11 in 2024-08-31T06:03:06.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:05 smithi155 ceph-mon[21742]: from='client.? 172.21.15.184:0/1360157121' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8297da1d-d3e2-4a4d-9396-426dfafa3e95"} : dispatch 2024-08-31T06:03:06.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:05 smithi155 ceph-mon[21742]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8297da1d-d3e2-4a4d-9396-426dfafa3e95"} : dispatch 2024-08-31T06:03:06.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:05 smithi155 ceph-mon[21742]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8297da1d-d3e2-4a4d-9396-426dfafa3e95"}]': finished 2024-08-31T06:03:06.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:05 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-31T06:03:06.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:05 smithi155 ceph-mon[21742]: osdmap e67: 12 total, 11 up, 12 in 2024-08-31T06:03:06.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:05 smithi016 ceph-mon[17194]: osdmap e66: 11 total, 11 up, 11 in 2024-08-31T06:03:06.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:05 smithi016 ceph-mon[17194]: from='client.? 172.21.15.184:0/1360157121' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8297da1d-d3e2-4a4d-9396-426dfafa3e95"} : dispatch 2024-08-31T06:03:06.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:05 smithi016 ceph-mon[17194]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8297da1d-d3e2-4a4d-9396-426dfafa3e95"} : dispatch 2024-08-31T06:03:06.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:05 smithi016 ceph-mon[17194]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8297da1d-d3e2-4a4d-9396-426dfafa3e95"}]': finished 2024-08-31T06:03:06.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:05 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-31T06:03:06.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:05 smithi016 ceph-mon[17194]: osdmap e67: 12 total, 11 up, 12 in 2024-08-31T06:03:06.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:05 smithi184 ceph-mon[22331]: osdmap e66: 11 total, 11 up, 11 in 2024-08-31T06:03:06.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:05 smithi184 ceph-mon[22331]: from='client.? 172.21.15.184:0/1360157121' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8297da1d-d3e2-4a4d-9396-426dfafa3e95"} : dispatch 2024-08-31T06:03:06.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:05 smithi184 ceph-mon[22331]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8297da1d-d3e2-4a4d-9396-426dfafa3e95"} : dispatch 2024-08-31T06:03:06.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:05 smithi184 ceph-mon[22331]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8297da1d-d3e2-4a4d-9396-426dfafa3e95"}]': finished 2024-08-31T06:03:06.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:05 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-31T06:03:06.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:05 smithi184 ceph-mon[22331]: osdmap e67: 12 total, 11 up, 12 in 2024-08-31T06:03:07.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:06 smithi155 ceph-mon[21742]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-08-31T06:03:07.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:06 smithi155 ceph-mon[21742]: from='client.? 172.21.15.184:0/3973271544' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T06:03:07.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:06 smithi016 ceph-mon[17194]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-08-31T06:03:07.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:06 smithi016 ceph-mon[17194]: from='client.? 172.21.15.184:0/3973271544' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T06:03:07.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:06 smithi184 ceph-mon[22331]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-08-31T06:03:07.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:06 smithi184 ceph-mon[22331]: from='client.? 172.21.15.184:0/3973271544' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-31T06:03:09.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:08 smithi155 ceph-mon[21742]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-08-31T06:03:09.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:08 smithi016 ceph-mon[17194]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-08-31T06:03:09.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:08 smithi184 ceph-mon[22331]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-08-31T06:03:10.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:10 smithi184 ceph-mon[22331]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-31T06:03:11.121 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:10 smithi155 ceph-mon[21742]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-31T06:03:11.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:10 smithi016 ceph-mon[17194]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-31T06:03:13.121 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:12 smithi155 ceph-mon[21742]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-08-31T06:03:13.162 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:12 smithi184 ceph-mon[22331]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-08-31T06:03:13.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:12 smithi016 ceph-mon[17194]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-08-31T06:03:14.948 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:14 smithi184 ceph-mon[22331]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-08-31T06:03:14.948 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:14 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-08-31T06:03:14.948 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:14 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:03:15.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:14 smithi155 ceph-mon[21742]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-08-31T06:03:15.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:14 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-08-31T06:03:15.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:14 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:03:15.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:14 smithi016 ceph-mon[17194]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-08-31T06:03:15.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:14 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-08-31T06:03:15.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:14 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:03:16.077 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:15 smithi184 ceph-mon[22331]: Deploying daemon osd.11 on smithi184 2024-08-31T06:03:16.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:15 smithi155 ceph-mon[21742]: Deploying daemon osd.11 on smithi184 2024-08-31T06:03:16.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:15 smithi016 ceph-mon[17194]: Deploying daemon osd.11 on smithi184 2024-08-31T06:03:16.960 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:16 smithi184 ceph-mon[22331]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-31T06:03:17.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:16 smithi155 ceph-mon[21742]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-31T06:03:17.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:16 smithi016 ceph-mon[17194]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-31T06:03:19.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:18 smithi155 ceph-mon[21742]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-08-31T06:03:19.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:18 smithi016 ceph-mon[17194]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-08-31T06:03:19.221 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:18 smithi184 ceph-mon[22331]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-08-31T06:03:21.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:20 smithi155 ceph-mon[21742]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-08-31T06:03:21.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:20 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:03:21.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:20 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:21.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:20 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:21.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:20 smithi016 ceph-mon[17194]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-08-31T06:03:21.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:20 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:03:21.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:20 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:21.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:20 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:21.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:20 smithi184 ceph-mon[22331]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-08-31T06:03:21.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:20 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:03:21.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:20 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:21.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:20 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:22.882 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:22.879+0000 7ff12effd640 1 -- 172.21.15.184:0/4079463717 <== mgr.14220 v2:172.21.15.16:6800/4279751337 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7ff1380635a0 con 0x7ff1240b4bb0 2024-08-31T06:03:22.882 INFO:teuthology.orchestra.run.smithi184.stdout:Created osd(s) 11 on host 'smithi184' 2024-08-31T06:03:22.884 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:22.882+0000 7ff140185640 1 -- 172.21.15.184:0/4079463717 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff1240b4bb0 msgr2=0x7ff1240b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:22.884 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:22.882+0000 7ff140185640 1 --2- 172.21.15.184:0/4079463717 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff1240b4bb0 0x7ff1240b70c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff138105c30 tx=0x7ff12800a2d0 comp rx=0 tx=0).stop 2024-08-31T06:03:22.885 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:22.882+0000 7ff140185640 1 -- 172.21.15.184:0/4079463717 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff138102830 msgr2=0x7ff13819aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:22.885 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:22.882+0000 7ff140185640 1 --2- 172.21.15.184:0/4079463717 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff138102830 0x7ff13819aff0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff13400e9f0 tx=0x7ff13400edf0 comp rx=0 tx=0).stop 2024-08-31T06:03:22.885 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:22.882+0000 7ff140185640 1 -- 172.21.15.184:0/4079463717 shutdown_connections 2024-08-31T06:03:22.885 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:22.882+0000 7ff140185640 1 --2- 172.21.15.184:0/4079463717 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff1240b4bb0 0x7ff1240b70c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:22.885 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:22.882+0000 7ff140185640 1 --2- 172.21.15.184:0/4079463717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff138108ac0 0x7ff13819ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:22.885 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:22.882+0000 7ff140185640 1 --2- 172.21.15.184:0/4079463717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff138108170 0x7ff13819b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:22.885 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:22.882+0000 7ff140185640 1 --2- 172.21.15.184:0/4079463717 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff138102830 0x7ff13819aff0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:22.885 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:22.882+0000 7ff140185640 1 -- 172.21.15.184:0/4079463717 >> 172.21.15.184:0/4079463717 conn(0x7ff1380794f0 msgr2=0x7ff138104180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:03:22.885 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:22.882+0000 7ff140185640 1 -- 172.21.15.184:0/4079463717 shutdown_connections 2024-08-31T06:03:22.885 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:22.882+0000 7ff140185640 1 -- 172.21.15.184:0/4079463717 wait complete. 2024-08-31T06:03:23.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:22 smithi155 ceph-mon[21742]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-08-31T06:03:23.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:22 smithi016 ceph-mon[17194]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-08-31T06:03:23.203 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:22 smithi184 ceph-mon[22331]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-08-31T06:03:23.871 DEBUG:teuthology.orchestra.run.smithi184:osd.11> sudo journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@osd.11.service 2024-08-31T06:03:23.873 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-08-31T06:03:23.874 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph osd stat -f json 2024-08-31T06:03:24.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:23 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:24.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:23 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:24.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:23 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:24.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:23 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:24.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:23 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:24.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:23 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:24.262 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:03:24.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:24.882+0000 7f7f28a00640 1 -- 172.21.15.16:0/588902833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f24103750 msgr2=0x7f7f24103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:24.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:24.882+0000 7f7f28a00640 1 --2- 172.21.15.16:0/588902833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f24103750 0x7f7f24103b60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7f14009860 tx=0x7f7f1402a340 comp rx=0 tx=0).stop 2024-08-31T06:03:24.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:24.882+0000 7f7f28a00640 1 -- 172.21.15.16:0/588902833 shutdown_connections 2024-08-31T06:03:24.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:24.882+0000 7f7f28a00640 1 --2- 172.21.15.16:0/588902833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f241099e0 0x7f7f24110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:24.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:24.882+0000 7f7f28a00640 1 --2- 172.21.15.16:0/588902833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f24109090 0x7f7f241094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:24.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:24.882+0000 7f7f28a00640 1 --2- 172.21.15.16:0/588902833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f24103750 0x7f7f24103b60 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:24.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:24.882+0000 7f7f28a00640 1 -- 172.21.15.16:0/588902833 >> 172.21.15.16:0/588902833 conn(0x7f7f240ff1a0 msgr2=0x7f7f241015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:03:24.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:24.882+0000 7f7f28a00640 1 -- 172.21.15.16:0/588902833 shutdown_connections 2024-08-31T06:03:24.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:24.882+0000 7f7f28a00640 1 -- 172.21.15.16:0/588902833 wait complete. 2024-08-31T06:03:24.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:24.883+0000 7f7f28a00640 1 Processor -- start 2024-08-31T06:03:24.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:24.884+0000 7f7f28a00640 1 -- start start 2024-08-31T06:03:24.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:24.884+0000 7f7f28a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f24103750 0x7f7f2419a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:24.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:24.884+0000 7f7f28a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f24109090 0x7f7f2419a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:24.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:24.884+0000 7f7f28a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f241099e0 0x7f7f2419f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:24.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:24.884+0000 7f7f28a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7f24113690 con 0x7f7f241099e0 2024-08-31T06:03:24.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:24.884+0000 7f7f28a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7f24113510 con 0x7f7f24103750 2024-08-31T06:03:24.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:24.884+0000 7f7f28a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7f24113810 con 0x7f7f24109090 2024-08-31T06:03:24.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:24.884+0000 7f7f22800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f24103750 0x7f7f2419a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:24.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:24.884+0000 7f7f22800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f24103750 0x7f7f2419a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:50838/0 (socket says 172.21.15.16:50838) 2024-08-31T06:03:24.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:24.884+0000 7f7f22800640 1 -- 172.21.15.16:0/723489210 learned_addr learned my addr 172.21.15.16:0/723489210 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:03:24.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:24.884+0000 7f7f23200640 1 --2- 172.21.15.16:0/723489210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f241099e0 0x7f7f2419f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:24.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:24.885+0000 7f7f21e00640 1 --2- 172.21.15.16:0/723489210 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f24109090 0x7f7f2419a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:24.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:24.885+0000 7f7f22800640 1 -- 172.21.15.16:0/723489210 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f24109090 msgr2=0x7f7f2419a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:24.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:24.885+0000 7f7f22800640 1 --2- 172.21.15.16:0/723489210 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f24109090 0x7f7f2419a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:24.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:24.885+0000 7f7f22800640 1 -- 172.21.15.16:0/723489210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f241099e0 msgr2=0x7f7f2419f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:24.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:24.885+0000 7f7f22800640 1 --2- 172.21.15.16:0/723489210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f241099e0 0x7f7f2419f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:24.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:24.885+0000 7f7f22800640 1 -- 172.21.15.16:0/723489210 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f2419ffd0 con 0x7f7f24103750 2024-08-31T06:03:24.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:24.885+0000 7f7f21e00640 1 --2- 172.21.15.16:0/723489210 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f24109090 0x7f7f2419a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:03:24.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:24.885+0000 7f7f22800640 1 --2- 172.21.15.16:0/723489210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f24103750 0x7f7f2419a060 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7f14035040 tx=0x7f7f14032d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:03:24.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:24.885+0000 7f7f1b400640 1 -- 172.21.15.16:0/723489210 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f14037040 con 0x7f7f24103750 2024-08-31T06:03:24.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:24.885+0000 7f7f28a00640 1 -- 172.21.15.16:0/723489210 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f241a0260 con 0x7f7f24103750 2024-08-31T06:03:24.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:24.885+0000 7f7f1b400640 1 -- 172.21.15.16:0/723489210 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f14041070 con 0x7f7f24103750 2024-08-31T06:03:24.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:24.885+0000 7f7f1b400640 1 -- 172.21.15.16:0/723489210 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f14003850 con 0x7f7f24103750 2024-08-31T06:03:24.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:24.886+0000 7f7f28a00640 1 -- 172.21.15.16:0/723489210 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f241a06e0 con 0x7f7f24103750 2024-08-31T06:03:24.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:24.887+0000 7f7f28a00640 1 -- 172.21.15.16:0/723489210 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f24103b60 con 0x7f7f24103750 2024-08-31T06:03:24.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:24.889+0000 7f7f1b400640 1 -- 172.21.15.16:0/723489210 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7f7f140039f0 con 0x7f7f24103750 2024-08-31T06:03:24.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:24.890+0000 7f7f1b400640 1 --2- 172.21.15.16:0/723489210 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7f080b4b80 0x7f7f080b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:24.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:24.890+0000 7f7f1b400640 1 -- 172.21.15.16:0/723489210 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 7024+0+0 (secure 0 0 0) 0x7f7f14104730 con 0x7f7f24103750 2024-08-31T06:03:24.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:24.890+0000 7f7f21e00640 1 --2- 172.21.15.16:0/723489210 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7f080b4b80 0x7f7f080b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:24.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:24.891+0000 7f7f21e00640 1 --2- 172.21.15.16:0/723489210 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7f080b4b80 0x7f7f080b7090 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7f24108c70 tx=0x7f7f10009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:03:24.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:24.895+0000 7f7f1b400640 1 -- 172.21.15.16:0/723489210 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f7f140cbcf0 con 0x7f7f24103750 2024-08-31T06:03:25.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:25.092+0000 7f7f28a00640 1 -- 172.21.15.16:0/723489210 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f7f2402e430 con 0x7f7f24103750 2024-08-31T06:03:25.095 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:24 smithi016 ceph-mon[17194]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-08-31T06:03:25.095 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:24 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:25.095 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:24 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:25.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:25.092+0000 7f7f1b400640 1 -- 172.21.15.16:0/723489210 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v67) ==== 74+0+133 (secure 0 0 0) 0x7f7f140cb5f0 con 0x7f7f24103750 2024-08-31T06:03:25.096 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:03:25.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:25.095+0000 7f7f28a00640 1 -- 172.21.15.16:0/723489210 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7f080b4b80 msgr2=0x7f7f080b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:25.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:25.095+0000 7f7f28a00640 1 --2- 172.21.15.16:0/723489210 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7f080b4b80 0x7f7f080b7090 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7f24108c70 tx=0x7f7f10009260 comp rx=0 tx=0).stop 2024-08-31T06:03:25.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:25.095+0000 7f7f28a00640 1 -- 172.21.15.16:0/723489210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f24103750 msgr2=0x7f7f2419a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:25.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:25.095+0000 7f7f28a00640 1 --2- 172.21.15.16:0/723489210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f24103750 0x7f7f2419a060 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7f14035040 tx=0x7f7f14032d50 comp rx=0 tx=0).stop 2024-08-31T06:03:25.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:25.095+0000 7f7f28a00640 1 -- 172.21.15.16:0/723489210 shutdown_connections 2024-08-31T06:03:25.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:25.095+0000 7f7f28a00640 1 --2- 172.21.15.16:0/723489210 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7f080b4b80 0x7f7f080b7090 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:25.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:25.095+0000 7f7f28a00640 1 --2- 172.21.15.16:0/723489210 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f24109090 0x7f7f2419a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:25.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:25.095+0000 7f7f28a00640 1 --2- 172.21.15.16:0/723489210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f241099e0 0x7f7f2419f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:25.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:25.095+0000 7f7f28a00640 1 --2- 172.21.15.16:0/723489210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f24103750 0x7f7f2419a060 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:25.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:25.095+0000 7f7f28a00640 1 -- 172.21.15.16:0/723489210 >> 172.21.15.16:0/723489210 conn(0x7f7f240ff1a0 msgr2=0x7f7f241015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:03:25.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:25.095+0000 7f7f28a00640 1 -- 172.21.15.16:0/723489210 shutdown_connections 2024-08-31T06:03:25.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:25.095+0000 7f7f28a00640 1 -- 172.21.15.16:0/723489210 wait complete. 2024-08-31T06:03:25.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:24 smithi184 ceph-mon[22331]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-08-31T06:03:25.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:24 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:25.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:24 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:25.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:24 smithi155 ceph-mon[21742]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-08-31T06:03:25.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:24 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:25.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:24 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:25.844 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:03:25 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:03:25.717+0000 7faa96aa1640 -1 osd.11 0 log_to_monitors true 2024-08-31T06:03:26.038 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":67,"num_osds":12,"num_up_osds":11,"osd_up_since":1725084183,"num_in_osds":12,"osd_in_since":1725084185,"num_remapped_pgs":0} 2024-08-31T06:03:26.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:25 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/723489210' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-08-31T06:03:26.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:25 smithi016 ceph-mon[17194]: from='osd.11 [v2:172.21.15.184:6824/399265701,v1:172.21.15.184:6825/399265701]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-08-31T06:03:26.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:25 smithi016 ceph-mon[17194]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-08-31T06:03:26.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:25 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/723489210' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-08-31T06:03:26.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:25 smithi184 ceph-mon[22331]: from='osd.11 [v2:172.21.15.184:6824/399265701,v1:172.21.15.184:6825/399265701]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-08-31T06:03:26.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:25 smithi184 ceph-mon[22331]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-08-31T06:03:26.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:25 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/723489210' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-08-31T06:03:26.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:25 smithi155 ceph-mon[21742]: from='osd.11 [v2:172.21.15.184:6824/399265701,v1:172.21.15.184:6825/399265701]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-08-31T06:03:26.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:25 smithi155 ceph-mon[21742]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-08-31T06:03:27.040 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph osd stat -f json 2024-08-31T06:03:27.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:26 smithi016 ceph-mon[17194]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-08-31T06:03:27.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:26 smithi016 ceph-mon[17194]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-08-31T06:03:27.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:26 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-31T06:03:27.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:26 smithi016 ceph-mon[17194]: from='osd.11 [v2:172.21.15.184:6824/399265701,v1:172.21.15.184:6825/399265701]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]} : dispatch 2024-08-31T06:03:27.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:26 smithi016 ceph-mon[17194]: osdmap e68: 12 total, 11 up, 12 in 2024-08-31T06:03:27.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:26 smithi016 ceph-mon[17194]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]} : dispatch 2024-08-31T06:03:27.222 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:03:26 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:03:26.895+0000 7faa92a23640 -1 osd.11 0 waiting for initial osdmap 2024-08-31T06:03:27.223 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:03:26 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:03:26.909+0000 7faa8e821640 -1 osd.11 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-31T06:03:27.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:26 smithi184 ceph-mon[22331]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-08-31T06:03:27.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:26 smithi184 ceph-mon[22331]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-08-31T06:03:27.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:26 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-31T06:03:27.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:26 smithi184 ceph-mon[22331]: from='osd.11 [v2:172.21.15.184:6824/399265701,v1:172.21.15.184:6825/399265701]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]} : dispatch 2024-08-31T06:03:27.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:26 smithi184 ceph-mon[22331]: osdmap e68: 12 total, 11 up, 12 in 2024-08-31T06:03:27.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:26 smithi184 ceph-mon[22331]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]} : dispatch 2024-08-31T06:03:27.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:26 smithi155 ceph-mon[21742]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-08-31T06:03:27.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:26 smithi155 ceph-mon[21742]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-08-31T06:03:27.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:26 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-31T06:03:27.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:26 smithi155 ceph-mon[21742]: from='osd.11 [v2:172.21.15.184:6824/399265701,v1:172.21.15.184:6825/399265701]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]} : dispatch 2024-08-31T06:03:27.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:26 smithi155 ceph-mon[21742]: osdmap e68: 12 total, 11 up, 12 in 2024-08-31T06:03:27.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:26 smithi155 ceph-mon[21742]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]} : dispatch 2024-08-31T06:03:27.460 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:03:28.032 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:27 smithi016 ceph-mon[17194]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-08-31T06:03:28.032 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:27 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-31T06:03:28.032 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:27 smithi016 ceph-mon[17194]: osdmap e69: 12 total, 11 up, 12 in 2024-08-31T06:03:28.032 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:27 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-31T06:03:28.032 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:27 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:28.032 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:27 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:28.033 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:27 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:28.033 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:27 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:28.033 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:27 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:28.033 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:27 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:28.033 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:27 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:28.034 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:27 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:28.034 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:27 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:28.034 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:27 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:28.034 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:27 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:28.034 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:27 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:03:28.034 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:27 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:03:28.034 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:27 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:28.034 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:27 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:03:28.034 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:27 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:03:28.034 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:27 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:03:28.034 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:27 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:28.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.103+0000 7f0369a00640 1 -- 172.21.15.16:0/154703817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0364109bb0 msgr2=0x7f03641103e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:28.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.103+0000 7f0369a00640 1 --2- 172.21.15.16:0/154703817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0364109bb0 0x7f03641103e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f035c009ca0 tx=0x7f035c02a8c0 comp rx=0 tx=0).stop 2024-08-31T06:03:28.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.103+0000 7f0369a00640 1 -- 172.21.15.16:0/154703817 shutdown_connections 2024-08-31T06:03:28.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.103+0000 7f0369a00640 1 --2- 172.21.15.16:0/154703817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0364109bb0 0x7f03641103e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:28.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.103+0000 7f0369a00640 1 --2- 172.21.15.16:0/154703817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0364109260 0x7f0364109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:28.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.103+0000 7f0369a00640 1 --2- 172.21.15.16:0/154703817 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0364103920 0x7f0364103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:28.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.103+0000 7f0369a00640 1 -- 172.21.15.16:0/154703817 >> 172.21.15.16:0/154703817 conn(0x7f03640ff3b0 msgr2=0x7f0364101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:03:28.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.103+0000 7f0369a00640 1 -- 172.21.15.16:0/154703817 shutdown_connections 2024-08-31T06:03:28.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.103+0000 7f0369a00640 1 -- 172.21.15.16:0/154703817 wait complete. 2024-08-31T06:03:28.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.105+0000 7f0369a00640 1 Processor -- start 2024-08-31T06:03:28.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.105+0000 7f0369a00640 1 -- start start 2024-08-31T06:03:28.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.105+0000 7f0369a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0364103920 0x7f036419a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:28.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.105+0000 7f0369a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0364109260 0x7f036419a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:28.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.105+0000 7f0369a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0364109bb0 0x7f036419fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:28.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.105+0000 7f0369a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f03641138c0 con 0x7f0364103920 2024-08-31T06:03:28.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.105+0000 7f0369a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0364113740 con 0x7f0364109260 2024-08-31T06:03:28.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.105+0000 7f0369a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0364113a40 con 0x7f0364109bb0 2024-08-31T06:03:28.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.106+0000 7f0362a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0364103920 0x7f036419a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:28.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.106+0000 7f0363400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0364109bb0 0x7f036419fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:28.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.106+0000 7f035be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0364109260 0x7f036419a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:28.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.106+0000 7f0363400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0364109bb0 0x7f036419fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:36926/0 (socket says 172.21.15.16:36926) 2024-08-31T06:03:28.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.106+0000 7f0363400640 1 -- 172.21.15.16:0/2099972334 learned_addr learned my addr 172.21.15.16:0/2099972334 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:03:28.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.106+0000 7f0363400640 1 -- 172.21.15.16:0/2099972334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0364109260 msgr2=0x7f036419a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:28.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.106+0000 7f0363400640 1 --2- 172.21.15.16:0/2099972334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0364109260 0x7f036419a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:28.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.106+0000 7f0363400640 1 -- 172.21.15.16:0/2099972334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0364103920 msgr2=0x7f036419a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:28.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.106+0000 7f0363400640 1 --2- 172.21.15.16:0/2099972334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0364103920 0x7f036419a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:28.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.106+0000 7f0363400640 1 -- 172.21.15.16:0/2099972334 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03641a0160 con 0x7f0364109bb0 2024-08-31T06:03:28.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.106+0000 7f0363400640 1 --2- 172.21.15.16:0/2099972334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0364109bb0 0x7f036419fa30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f035c009430 tx=0x7f035c0317c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:03:28.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.107+0000 7f035b400640 1 -- 172.21.15.16:0/2099972334 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f035c003b70 con 0x7f0364109bb0 2024-08-31T06:03:28.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.107+0000 7f035b400640 1 -- 172.21.15.16:0/2099972334 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f035c003d10 con 0x7f0364109bb0 2024-08-31T06:03:28.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.107+0000 7f0369a00640 1 -- 172.21.15.16:0/2099972334 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03641a03f0 con 0x7f0364109bb0 2024-08-31T06:03:28.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.107+0000 7f035b400640 1 -- 172.21.15.16:0/2099972334 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f035c02fa30 con 0x7f0364109bb0 2024-08-31T06:03:28.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.107+0000 7f0369a00640 1 -- 172.21.15.16:0/2099972334 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03641a6bb0 con 0x7f0364109bb0 2024-08-31T06:03:28.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.108+0000 7f0369a00640 1 -- 172.21.15.16:0/2099972334 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0364104ba0 con 0x7f0364109bb0 2024-08-31T06:03:28.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.110+0000 7f035b400640 1 -- 172.21.15.16:0/2099972334 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7f035c02fbd0 con 0x7f0364109bb0 2024-08-31T06:03:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.112+0000 7f035b400640 1 --2- 172.21.15.16:0/2099972334 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f03480b4b80 0x7f03480b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.112+0000 7f035b400640 1 -- 172.21.15.16:0/2099972334 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f035c106240 con 0x7f0364109bb0 2024-08-31T06:03:28.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.112+0000 7f0362a00640 1 --2- 172.21.15.16:0/2099972334 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f03480b4b80 0x7f03480b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:28.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.113+0000 7f0362a00640 1 --2- 172.21.15.16:0/2099972334 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f03480b4b80 0x7f03480b7090 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f035400bbe0 tx=0x7f035400b390 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:03:28.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.113+0000 7f035b400640 1 -- 172.21.15.16:0/2099972334 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f035c0cd730 con 0x7f0364109bb0 2024-08-31T06:03:28.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:27 smithi184 ceph-mon[22331]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-08-31T06:03:28.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:27 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-31T06:03:28.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:27 smithi184 ceph-mon[22331]: osdmap e69: 12 total, 11 up, 12 in 2024-08-31T06:03:28.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:27 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-31T06:03:28.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:27 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:28.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:27 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:28.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:27 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:28.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:27 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:28.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:27 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:28.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:27 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:28.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:27 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:28.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:27 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:28.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:27 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:28.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:27 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:28.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:27 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:28.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:27 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:03:28.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:27 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:03:28.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:27 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:28.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:27 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:03:28.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:27 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:03:28.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:27 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:03:28.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:27 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:28.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.309+0000 7f0369a00640 1 -- 172.21.15.16:0/2099972334 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f036419b320 con 0x7f0364109bb0 2024-08-31T06:03:28.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.310+0000 7f035b400640 1 -- 172.21.15.16:0/2099972334 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v70) ==== 74+0+133 (secure 0 0 0) 0x7f035c0cd030 con 0x7f0364109bb0 2024-08-31T06:03:28.312 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:03:28.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.312+0000 7f0369a00640 1 -- 172.21.15.16:0/2099972334 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f03480b4b80 msgr2=0x7f03480b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:28.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.312+0000 7f0369a00640 1 --2- 172.21.15.16:0/2099972334 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f03480b4b80 0x7f03480b7090 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f035400bbe0 tx=0x7f035400b390 comp rx=0 tx=0).stop 2024-08-31T06:03:28.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.312+0000 7f0369a00640 1 -- 172.21.15.16:0/2099972334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0364109bb0 msgr2=0x7f036419fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:28.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.312+0000 7f0369a00640 1 --2- 172.21.15.16:0/2099972334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0364109bb0 0x7f036419fa30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f035c009430 tx=0x7f035c0317c0 comp rx=0 tx=0).stop 2024-08-31T06:03:28.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.313+0000 7f0369a00640 1 -- 172.21.15.16:0/2099972334 shutdown_connections 2024-08-31T06:03:28.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.313+0000 7f0369a00640 1 --2- 172.21.15.16:0/2099972334 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f03480b4b80 0x7f03480b7090 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:28.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.313+0000 7f0369a00640 1 --2- 172.21.15.16:0/2099972334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0364109bb0 0x7f036419fa30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:28.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.313+0000 7f0369a00640 1 --2- 172.21.15.16:0/2099972334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0364109260 0x7f036419a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:28.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.313+0000 7f0369a00640 1 --2- 172.21.15.16:0/2099972334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0364103920 0x7f036419a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:28.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.313+0000 7f0369a00640 1 -- 172.21.15.16:0/2099972334 >> 172.21.15.16:0/2099972334 conn(0x7f03640ff3b0 msgr2=0x7f0364108850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:03:28.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.313+0000 7f0369a00640 1 -- 172.21.15.16:0/2099972334 shutdown_connections 2024-08-31T06:03:28.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:28.313+0000 7f0369a00640 1 -- 172.21.15.16:0/2099972334 wait complete. 2024-08-31T06:03:28.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:27 smithi155 ceph-mon[21742]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-08-31T06:03:28.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:27 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-31T06:03:28.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:27 smithi155 ceph-mon[21742]: osdmap e69: 12 total, 11 up, 12 in 2024-08-31T06:03:28.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:27 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-31T06:03:28.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:27 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:28.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:27 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:28.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:27 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:28.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:27 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:28.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:27 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:28.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:27 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:28.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:27 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:28.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:27 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:28.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:27 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:28.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:27 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-31T06:03:28.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:27 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:28.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:27 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:03:28.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:27 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:03:28.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:27 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:28.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:27 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:03:28.374 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:27 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:03:28.374 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:27 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:03:28.374 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:27 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:29.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:28 smithi016 ceph-mon[17194]: purged_snaps scrub starts 2024-08-31T06:03:29.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:28 smithi016 ceph-mon[17194]: purged_snaps scrub ok 2024-08-31T06:03:29.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:28 smithi016 ceph-mon[17194]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-08-31T06:03:29.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:28 smithi016 ceph-mon[17194]: Detected new or changed devices on smithi184 2024-08-31T06:03:29.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:28 smithi016 ceph-mon[17194]: Adjusting osd_memory_target on smithi184 to 4272M 2024-08-31T06:03:29.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:28 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-31T06:03:29.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:28 smithi016 ceph-mon[17194]: osd.11 [v2:172.21.15.184:6824/399265701,v1:172.21.15.184:6825/399265701] boot 2024-08-31T06:03:29.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:28 smithi016 ceph-mon[17194]: osdmap e70: 12 total, 12 up, 12 in 2024-08-31T06:03:29.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:28 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/2099972334' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-08-31T06:03:29.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:28 smithi184 ceph-mon[22331]: purged_snaps scrub starts 2024-08-31T06:03:29.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:28 smithi184 ceph-mon[22331]: purged_snaps scrub ok 2024-08-31T06:03:29.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:28 smithi184 ceph-mon[22331]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-08-31T06:03:29.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:28 smithi184 ceph-mon[22331]: Detected new or changed devices on smithi184 2024-08-31T06:03:29.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:28 smithi184 ceph-mon[22331]: Adjusting osd_memory_target on smithi184 to 4272M 2024-08-31T06:03:29.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:28 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-31T06:03:29.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:28 smithi184 ceph-mon[22331]: osd.11 [v2:172.21.15.184:6824/399265701,v1:172.21.15.184:6825/399265701] boot 2024-08-31T06:03:29.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:28 smithi184 ceph-mon[22331]: osdmap e70: 12 total, 12 up, 12 in 2024-08-31T06:03:29.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:28 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/2099972334' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-08-31T06:03:29.310 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":70,"num_osds":12,"num_up_osds":12,"osd_up_since":1725084207,"num_in_osds":12,"osd_in_since":1725084185,"num_remapped_pgs":0} 2024-08-31T06:03:29.310 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph osd dump --format=json 2024-08-31T06:03:29.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:28 smithi155 ceph-mon[21742]: purged_snaps scrub starts 2024-08-31T06:03:29.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:28 smithi155 ceph-mon[21742]: purged_snaps scrub ok 2024-08-31T06:03:29.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:28 smithi155 ceph-mon[21742]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-08-31T06:03:29.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:28 smithi155 ceph-mon[21742]: Detected new or changed devices on smithi184 2024-08-31T06:03:29.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:28 smithi155 ceph-mon[21742]: Adjusting osd_memory_target on smithi184 to 4272M 2024-08-31T06:03:29.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:28 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-31T06:03:29.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:28 smithi155 ceph-mon[21742]: osd.11 [v2:172.21.15.184:6824/399265701,v1:172.21.15.184:6825/399265701] boot 2024-08-31T06:03:29.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:28 smithi155 ceph-mon[21742]: osdmap e70: 12 total, 12 up, 12 in 2024-08-31T06:03:29.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:28 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/2099972334' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-08-31T06:03:29.684 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:03:30.133 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:29 smithi016 ceph-mon[17194]: osdmap e71: 12 total, 12 up, 12 in 2024-08-31T06:03:30.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:29 smithi184 ceph-mon[22331]: osdmap e71: 12 total, 12 up, 12 in 2024-08-31T06:03:30.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.310+0000 7feac6800640 1 -- 172.21.15.16:0/2478172138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feac0102ae0 msgr2=0x7feac010f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:30.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.310+0000 7feac6800640 1 --2- 172.21.15.16:0/2478172138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feac0102ae0 0x7feac010f800 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7feaac009860 tx=0x7feaac02a3d0 comp rx=0 tx=0).stop 2024-08-31T06:03:30.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.310+0000 7feac6800640 1 -- 172.21.15.16:0/2478172138 shutdown_connections 2024-08-31T06:03:30.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.310+0000 7feac6800640 1 --2- 172.21.15.16:0/2478172138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feac010fd40 0x7feac0112160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:30.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.310+0000 7feac6800640 1 --2- 172.21.15.16:0/2478172138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feac0102ae0 0x7feac010f800 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:30.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.310+0000 7feac6800640 1 --2- 172.21.15.16:0/2478172138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feac0102190 0x7feac01025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:30.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.310+0000 7feac6800640 1 -- 172.21.15.16:0/2478172138 >> 172.21.15.16:0/2478172138 conn(0x7feac00fe070 msgr2=0x7feac01004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:03:30.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.310+0000 7feac6800640 1 -- 172.21.15.16:0/2478172138 shutdown_connections 2024-08-31T06:03:30.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.310+0000 7feac6800640 1 -- 172.21.15.16:0/2478172138 wait complete. 2024-08-31T06:03:30.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.311+0000 7feac6800640 1 Processor -- start 2024-08-31T06:03:30.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.312+0000 7feac6800640 1 -- start start 2024-08-31T06:03:30.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.312+0000 7feac6800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feac0102190 0x7feac019c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:30.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.312+0000 7feac6800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feac0102ae0 0x7feac019c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:30.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.312+0000 7feac6800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feac010fd40 0x7feac01a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:30.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.312+0000 7feac6800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feac0115840 con 0x7feac0102ae0 2024-08-31T06:03:30.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.312+0000 7feac6800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7feac01156c0 con 0x7feac0102190 2024-08-31T06:03:30.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.312+0000 7feac6800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7feac01159c0 con 0x7feac010fd40 2024-08-31T06:03:30.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.312+0000 7feac4a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feac0102ae0 0x7feac019c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:30.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.312+0000 7feac4a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feac0102ae0 0x7feac019c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39366/0 (socket says 172.21.15.16:39366) 2024-08-31T06:03:30.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.312+0000 7feac4a00640 1 -- 172.21.15.16:0/450477293 learned_addr learned my addr 172.21.15.16:0/450477293 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:03:30.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.312+0000 7feac5400640 1 --2- 172.21.15.16:0/450477293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feac0102190 0x7feac019c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:30.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.312+0000 7feac5e00640 1 --2- 172.21.15.16:0/450477293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feac010fd40 0x7feac01a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:30.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.313+0000 7feac4a00640 1 -- 172.21.15.16:0/450477293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feac010fd40 msgr2=0x7feac01a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:30.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.313+0000 7feac4a00640 1 --2- 172.21.15.16:0/450477293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feac010fd40 0x7feac01a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:30.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.313+0000 7feac4a00640 1 -- 172.21.15.16:0/450477293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feac0102190 msgr2=0x7feac019c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:30.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.313+0000 7feac4a00640 1 --2- 172.21.15.16:0/450477293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feac0102190 0x7feac019c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:30.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.313+0000 7feac4a00640 1 -- 172.21.15.16:0/450477293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac01a2180 con 0x7feac0102ae0 2024-08-31T06:03:30.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.313+0000 7feac4a00640 1 --2- 172.21.15.16:0/450477293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feac0102ae0 0x7feac019c7b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7feaac009980 tx=0x7feaac030fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:03:30.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.313+0000 7feabe000640 1 -- 172.21.15.16:0/450477293 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaac040070 con 0x7feac0102ae0 2024-08-31T06:03:30.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.313+0000 7feabe000640 1 -- 172.21.15.16:0/450477293 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feaac005570 con 0x7feac0102ae0 2024-08-31T06:03:30.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.313+0000 7feac6800640 1 -- 172.21.15.16:0/450477293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feac01a2410 con 0x7feac0102ae0 2024-08-31T06:03:30.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.314+0000 7feabe000640 1 -- 172.21.15.16:0/450477293 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaac0046c0 con 0x7feac0102ae0 2024-08-31T06:03:30.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.314+0000 7feac6800640 1 -- 172.21.15.16:0/450477293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feac01a8bd0 con 0x7feac0102ae0 2024-08-31T06:03:30.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.315+0000 7feac6800640 1 -- 172.21.15.16:0/450477293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feac006be00 con 0x7feac0102ae0 2024-08-31T06:03:30.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.317+0000 7feabe000640 1 -- 172.21.15.16:0/450477293 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7feaac043070 con 0x7feac0102ae0 2024-08-31T06:03:30.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.319+0000 7feabe000640 1 --2- 172.21.15.16:0/450477293 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7feaa80b4b80 0x7feaa80b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:30.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.319+0000 7feabe000640 1 -- 172.21.15.16:0/450477293 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7feaac105cd0 con 0x7feac0102ae0 2024-08-31T06:03:30.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.319+0000 7feac5400640 1 --2- 172.21.15.16:0/450477293 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7feaa80b4b80 0x7feaa80b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:30.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.320+0000 7feac5400640 1 --2- 172.21.15.16:0/450477293 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7feaa80b4b80 0x7feaa80b7090 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7feab4005fd0 tx=0x7feab4009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:03:30.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.321+0000 7feabe000640 1 -- 172.21.15.16:0/450477293 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7feaac0cd1c0 con 0x7feac0102ae0 2024-08-31T06:03:30.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:29 smithi155 ceph-mon[21742]: osdmap e71: 12 total, 12 up, 12 in 2024-08-31T06:03:30.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.518+0000 7feac6800640 1 -- 172.21.15.16:0/450477293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feac010aed0 con 0x7feac0102ae0 2024-08-31T06:03:30.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.519+0000 7feabe000640 1 -- 172.21.15.16:0/450477293 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+18797 (secure 0 0 0) 0x7feaac0ccac0 con 0x7feac0102ae0 2024-08-31T06:03:30.522 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:03:30.522 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":71,"fsid":"673e4aa0-675d-11ef-bcd4-c7b262605968","created":"2024-08-31T05:55:09.632536+0000","modified":"2024-08-31T06:03:28.905145+0000","last_up_change":"2024-08-31T06:03:27.893697+0000","last_in_change":"2024-08-31T06:03:05.664945+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-31T05:59:23.642270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e349dae-6aa5-4b6d-9107-a945655e11bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6803","nonce":1725037400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6805","nonce":1725037400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6809","nonce":1725037400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6807","nonce":1725037400}]},"public_addr":"172.21.15.16:6803/1725037400","cluster_addr":"172.21.15.16:6805/1725037400","heartbeat_back_addr":"172.21.15.16:6809/1725037400","heartbeat_front_addr":"172.21.15.16:6807/1725037400","state":["exists","up"]},{"osd":1,"uuid":"bd645dc3-93b5-40af-89d8-08ccec22e50d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6801","nonce":4241618440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6803","nonce":4241618440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6807","nonce":4241618440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6805","nonce":4241618440}]},"public_addr":"172.21.15.155:6801/4241618440","cluster_addr":"172.21.15.155:6803/4241618440","heartbeat_back_addr":"172.21.15.155:6807/4241618440","heartbeat_front_addr":"172.21.15.155:6805/4241618440","state":["exists","up"]},{"osd":2,"uuid":"c69c8c19-32f8-400a-86d4-0f4ce240a7de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6801","nonce":2193296422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6803","nonce":2193296422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6807","nonce":2193296422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6805","nonce":2193296422}]},"public_addr":"172.21.15.184:6801/2193296422","cluster_addr":"172.21.15.184:6803/2193296422","heartbeat_back_addr":"172.21.15.184:6807/2193296422","heartbeat_front_addr":"172.21.15.184:6805/2193296422","state":["exists","up"]},{"osd":3,"uuid":"8892e35f-6880-434c-a805-ab8ccbed2bf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6811","nonce":81681815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6813","nonce":81681815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6817","nonce":81681815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6815","nonce":81681815}]},"public_addr":"172.21.15.16:6811/81681815","cluster_addr":"172.21.15.16:6813/81681815","heartbeat_back_addr":"172.21.15.16:6817/81681815","heartbeat_front_addr":"172.21.15.16:6815/81681815","state":["exists","up"]},{"osd":4,"uuid":"d2515e73-88f7-4654-9762-adb8802777ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6809","nonce":3592422019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6811","nonce":3592422019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6815","nonce":3592422019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6813","nonce":3592422019}]},"public_addr":"172.21.15.155:6809/3592422019","cluster_addr":"172.21.15.155:6811/3592422019","heartbeat_back_addr":"172.21.15.155:6815/3592422019","heartbeat_front_addr":"172.21.15.155:6813/3592422019","state":["exists","up"]},{"osd":5,"uuid":"5ce6e2fa-c17c-49b3-9402-f19bfc21076d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6809","nonce":2054894065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6811","nonce":2054894065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6815","nonce":2054894065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6813","nonce":2054894065}]},"public_addr":"172.21.15.184:6809/2054894065","cluster_addr":"172.21.15.184:6811/2054894065","heartbeat_back_addr":"172.21.15.184:6815/2054894065","heartbeat_front_addr":"172.21.15.184:6813/2054894065","state":["exists","up"]},{"osd":6,"uuid":"f257d8d1-ad59-4bd8-821d-bf29e50b547e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6819","nonce":1697421092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6821","nonce":1697421092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6825","nonce":1697421092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6823","nonce":1697421092}]},"public_addr":"172.21.15.16:6819/1697421092","cluster_addr":"172.21.15.16:6821/1697421092","heartbeat_back_addr":"172.21.15.16:6825/1697421092","heartbeat_front_addr":"172.21.15.16:6823/1697421092","state":["exists","up"]},{"osd":7,"uuid":"8e76c20e-8d25-423b-b7e8-de642d9c9a6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6817","nonce":2671827901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6819","nonce":2671827901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6823","nonce":2671827901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6821","nonce":2671827901}]},"public_addr":"172.21.15.155:6817/2671827901","cluster_addr":"172.21.15.155:6819/2671827901","heartbeat_back_addr":"172.21.15.155:6823/2671827901","heartbeat_front_addr":"172.21.15.155:6821/2671827901","state":["exists","up"]},{"osd":8,"uuid":"0f0bb23f-4bc3-4198-8381-5b5cd16ae394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6817","nonce":2479117823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6819","nonce":2479117823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6823","nonce":2479117823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6821","nonce":2479117823}]},"public_addr":"172.21.15.184:6817/2479117823","cluster_addr":"172.21.15.184:6819/2479117823","heartbeat_back_addr":"172.21.15.184:6823/2479117823","heartbeat_front_addr":"172.21.15.184:6821/2479117823","state":["exists","up"]},{"osd":9,"uuid":"bc27afcb-7871-4f72-a043-044fd5fe7d86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6827","nonce":3015700546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6829","nonce":3015700546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6833","nonce":3015700546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6831","nonce":3015700546}]},"public_addr":"172.21.15.16:6827/3015700546","cluster_addr":"172.21.15.16:6829/3015700546","heartbeat_back_addr":"172.21.15.16:6833/3015700546","heartbeat_front_addr":"172.21.15.16:6831/3015700546","state":["exists","up"]},{"osd":10,"uuid":"f2548997-72ee-400e-ab8d-5966d85b2cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6825","nonce":2427122475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6827","nonce":2427122475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6831","nonce":2427122475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6829","nonce":2427122475}]},"public_addr":"172.21.15.155:6825/2427122475","cluster_addr":"172.21.15.155:6827/2427122475","heartbeat_back_addr":"172.21.15.155:6831/2427122475","heartbeat_front_addr":"172.21.15.155:6829/2427122475","state":["exists","up"]},{"osd":11,"uuid":"8297da1d-d3e2-4a4d-9396-426dfafa3e95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6825","nonce":399265701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6827","nonce":399265701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6831","nonce":399265701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6829","nonce":399265701}]},"public_addr":"172.21.15.184:6825/399265701","cluster_addr":"172.21.15.184:6827/399265701","heartbeat_back_addr":"172.21.15.184:6831/399265701","heartbeat_front_addr":"172.21.15.184:6829/399265701","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:58:30.842360+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:58:57.164488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:59:20.778103+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:59:50.123991+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:00:16.789933+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:00:41.788905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:01:12.281376+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:01:39.211858+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:02:04.958771+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:02:35.645947+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:03:01.940433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:03:26.728673+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3923220960":"2024-09-01T05:57:23.153214+0000","172.21.15.16:0/3013533836":"2024-09-01T05:57:23.153214+0000","172.21.15.16:6800/3755683228":"2024-09-01T05:55:45.075902+0000","172.21.15.16:0/3064435470":"2024-09-01T05:55:45.075902+0000","172.21.15.16:0/911709962":"2024-09-01T05:55:45.075902+0000","172.21.15.16:0/2937368967":"2024-09-01T05:55:45.075902+0000","172.21.15.16:6800/1378972230":"2024-09-01T05:56:14.195704+0000","172.21.15.16:0/3047316995":"2024-09-01T05:56:14.195704+0000","172.21.15.16:6801/3755683228":"2024-09-01T05:55:45.075902+0000","172.21.15.16:6801/2571082634":"2024-09-01T05:57:23.153214+0000","172.21.15.16:0/2136704295":"2024-09-01T05:56:14.195704+0000","172.21.15.16:0/2259332778":"2024-09-01T05:56:14.195704+0000","172.21.15.16:6800/2571082634":"2024-09-01T05:57:23.153214+0000","172.21.15.16:6801/1378972230":"2024-09-01T05:56:14.195704+0000","172.21.15.16:0/2126457673":"2024-09-01T05:57:23.153214+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-31T06:03:30.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.521+0000 7feac6800640 1 -- 172.21.15.16:0/450477293 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7feaa80b4b80 msgr2=0x7feaa80b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:30.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.521+0000 7feac6800640 1 --2- 172.21.15.16:0/450477293 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7feaa80b4b80 0x7feaa80b7090 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7feab4005fd0 tx=0x7feab4009500 comp rx=0 tx=0).stop 2024-08-31T06:03:30.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.521+0000 7feac6800640 1 -- 172.21.15.16:0/450477293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feac0102ae0 msgr2=0x7feac019c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:30.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.521+0000 7feac6800640 1 --2- 172.21.15.16:0/450477293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feac0102ae0 0x7feac019c7b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7feaac009980 tx=0x7feaac030fa0 comp rx=0 tx=0).stop 2024-08-31T06:03:30.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.521+0000 7feac6800640 1 -- 172.21.15.16:0/450477293 shutdown_connections 2024-08-31T06:03:30.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.521+0000 7feac6800640 1 --2- 172.21.15.16:0/450477293 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7feaa80b4b80 0x7feaa80b7090 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:30.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.522+0000 7feac6800640 1 --2- 172.21.15.16:0/450477293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feac010fd40 0x7feac01a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:30.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.522+0000 7feac6800640 1 --2- 172.21.15.16:0/450477293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feac0102ae0 0x7feac019c7b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:30.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.522+0000 7feac6800640 1 --2- 172.21.15.16:0/450477293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feac0102190 0x7feac019c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.522+0000 7feac6800640 1 -- 172.21.15.16:0/450477293 >> 172.21.15.16:0/450477293 conn(0x7feac00fe070 msgr2=0x7feac0111eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:03:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.522+0000 7feac6800640 1 -- 172.21.15.16:0/450477293 shutdown_connections 2024-08-31T06:03:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:30.522+0000 7feac6800640 1 -- 172.21.15.16:0/450477293 wait complete. 2024-08-31T06:03:31.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:30 smithi184 ceph-mon[22331]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:31.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:30 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/450477293' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T06:03:31.347 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:30 smithi016 ceph-mon[17194]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:31.348 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:30 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/450477293' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T06:03:31.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:30 smithi155 ceph-mon[21742]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:31.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:30 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/450477293' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T06:03:31.387 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-31T05:59:23.642270+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-08-31T06:03:31.387 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-08-31T06:03:31.760 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:03:32.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.422+0000 7f2923200640 1 -- 172.21.15.16:0/2799982574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f291c10b460 msgr2=0x7f291c10b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:32.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.422+0000 7f2923200640 1 --2- 172.21.15.16:0/2799982574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f291c10b460 0x7f291c10b870 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2904009900 tx=0x7f290402a410 comp rx=0 tx=0).stop 2024-08-31T06:03:32.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.422+0000 7f2923200640 1 -- 172.21.15.16:0/2799982574 shutdown_connections 2024-08-31T06:03:32.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.422+0000 7f2923200640 1 --2- 172.21.15.16:0/2799982574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f291c10bdb0 0x7f291c1125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:32.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.422+0000 7f2923200640 1 --2- 172.21.15.16:0/2799982574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f291c10b460 0x7f291c10b870 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:32.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.422+0000 7f2923200640 1 --2- 172.21.15.16:0/2799982574 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f291c105b20 0x7f291c105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:32.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.422+0000 7f2923200640 1 -- 172.21.15.16:0/2799982574 >> 172.21.15.16:0/2799982574 conn(0x7f291c1015b0 msgr2=0x7f291c103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:03:32.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.423+0000 7f2923200640 1 -- 172.21.15.16:0/2799982574 shutdown_connections 2024-08-31T06:03:32.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.423+0000 7f2923200640 1 -- 172.21.15.16:0/2799982574 wait complete. 2024-08-31T06:03:32.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.424+0000 7f2923200640 1 Processor -- start 2024-08-31T06:03:32.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.424+0000 7f2923200640 1 -- start start 2024-08-31T06:03:32.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.424+0000 7f2923200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f291c105b20 0x7f291c19c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:32.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.424+0000 7f2923200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f291c10b460 0x7f291c19ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:32.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.424+0000 7f2923200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f291c10bdb0 0x7f291c1a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:32.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.424+0000 7f2923200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f291c115ac0 con 0x7f291c105b20 2024-08-31T06:03:32.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.424+0000 7f2923200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f291c115940 con 0x7f291c10bdb0 2024-08-31T06:03:32.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.424+0000 7f2923200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f291c115c40 con 0x7f291c10b460 2024-08-31T06:03:32.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.424+0000 7f291be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f291c105b20 0x7f291c19c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:32.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.424+0000 7f291be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f291c105b20 0x7f291c19c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39396/0 (socket says 172.21.15.16:39396) 2024-08-31T06:03:32.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.424+0000 7f291be00640 1 -- 172.21.15.16:0/980407507 learned_addr learned my addr 172.21.15.16:0/980407507 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:03:32.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.425+0000 7f2921200640 1 --2- 172.21.15.16:0/980407507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f291c10bdb0 0x7f291c1a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:32.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.425+0000 7f2913e00640 1 --2- 172.21.15.16:0/980407507 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f291c10b460 0x7f291c19ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:32.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.425+0000 7f291be00640 1 -- 172.21.15.16:0/980407507 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f291c10b460 msgr2=0x7f291c19ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:32.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.425+0000 7f291be00640 1 --2- 172.21.15.16:0/980407507 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f291c10b460 0x7f291c19ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:32.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.425+0000 7f291be00640 1 -- 172.21.15.16:0/980407507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f291c10bdb0 msgr2=0x7f291c1a1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:32.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.425+0000 7f291be00640 1 --2- 172.21.15.16:0/980407507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f291c10bdb0 0x7f291c1a1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:32.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.425+0000 7f291be00640 1 -- 172.21.15.16:0/980407507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f291c1a23e0 con 0x7f291c105b20 2024-08-31T06:03:32.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.425+0000 7f291be00640 1 --2- 172.21.15.16:0/980407507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f291c105b20 0x7f291c19c470 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f290c00e9f0 tx=0x7f290c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:03:32.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.426+0000 7f2919600640 1 -- 172.21.15.16:0/980407507 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c00cd80 con 0x7f291c105b20 2024-08-31T06:03:32.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.426+0000 7f2923200640 1 -- 172.21.15.16:0/980407507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f291c1a26d0 con 0x7f291c105b20 2024-08-31T06:03:32.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.426+0000 7f2919600640 1 -- 172.21.15.16:0/980407507 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f290c0054d0 con 0x7f291c105b20 2024-08-31T06:03:32.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.426+0000 7f2919600640 1 -- 172.21.15.16:0/980407507 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c010690 con 0x7f291c105b20 2024-08-31T06:03:32.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.426+0000 7f2923200640 1 -- 172.21.15.16:0/980407507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f291c1a8e30 con 0x7f291c105b20 2024-08-31T06:03:32.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.427+0000 7f2923200640 1 -- 172.21.15.16:0/980407507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f291c105f30 con 0x7f291c105b20 2024-08-31T06:03:32.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.429+0000 7f2919600640 1 -- 172.21.15.16:0/980407507 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7f290c004e90 con 0x7f291c105b20 2024-08-31T06:03:32.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.431+0000 7f2919600640 1 --2- 172.21.15.16:0/980407507 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f29000b4b80 0x7f29000b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:32.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.431+0000 7f2919600640 1 -- 172.21.15.16:0/980407507 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7f290c0e52b0 con 0x7f291c105b20 2024-08-31T06:03:32.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.431+0000 7f2913e00640 1 --2- 172.21.15.16:0/980407507 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f29000b4b80 0x7f29000b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:32.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.432+0000 7f2913e00640 1 --2- 172.21.15.16:0/980407507 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f29000b4b80 0x7f29000b7090 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f290402aa00 tx=0x7f2904005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:03:32.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.435+0000 7f2919600640 1 -- 172.21.15.16:0/980407507 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f290c0ac710 con 0x7f291c105b20 2024-08-31T06:03:32.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.628+0000 7f2923200640 1 -- 172.21.15.16:0/980407507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f291c10dbe0 con 0x7f291c105b20 2024-08-31T06:03:32.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.629+0000 7f2919600640 1 -- 172.21.15.16:0/980407507 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7f290c0ac010 con 0x7f291c105b20 2024-08-31T06:03:32.631 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-08-31T06:03:32.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.631+0000 7f2923200640 1 -- 172.21.15.16:0/980407507 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f29000b4b80 msgr2=0x7f29000b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:32.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.631+0000 7f2923200640 1 --2- 172.21.15.16:0/980407507 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f29000b4b80 0x7f29000b7090 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f290402aa00 tx=0x7f2904005a20 comp rx=0 tx=0).stop 2024-08-31T06:03:32.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.631+0000 7f2923200640 1 -- 172.21.15.16:0/980407507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f291c105b20 msgr2=0x7f291c19c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:32.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.631+0000 7f2923200640 1 --2- 172.21.15.16:0/980407507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f291c105b20 0x7f291c19c470 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f290c00e9f0 tx=0x7f290c00edf0 comp rx=0 tx=0).stop 2024-08-31T06:03:32.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.631+0000 7f2923200640 1 -- 172.21.15.16:0/980407507 shutdown_connections 2024-08-31T06:03:32.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.631+0000 7f2923200640 1 --2- 172.21.15.16:0/980407507 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f29000b4b80 0x7f29000b7090 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:32.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.631+0000 7f2923200640 1 --2- 172.21.15.16:0/980407507 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f291c10b460 0x7f291c19ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:32.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.631+0000 7f2923200640 1 --2- 172.21.15.16:0/980407507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f291c10bdb0 0x7f291c1a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:32.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.631+0000 7f2923200640 1 --2- 172.21.15.16:0/980407507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f291c105b20 0x7f291c19c470 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:32.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.631+0000 7f2923200640 1 -- 172.21.15.16:0/980407507 >> 172.21.15.16:0/980407507 conn(0x7f291c1015b0 msgr2=0x7f291c1039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:03:32.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.632+0000 7f2923200640 1 -- 172.21.15.16:0/980407507 shutdown_connections 2024-08-31T06:03:32.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:32.632+0000 7f2923200640 1 -- 172.21.15.16:0/980407507 wait complete. 2024-08-31T06:03:33.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:32 smithi184 ceph-mon[22331]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:33.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:32 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/980407507' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-31T06:03:33.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:32 smithi155 ceph-mon[21742]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:33.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:32 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/980407507' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-31T06:03:33.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:32 smithi016 ceph-mon[17194]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:33.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:32 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/980407507' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-31T06:03:33.478 INFO:tasks.cephadm:Adding mds.a on smithi016 2024-08-31T06:03:33.478 INFO:tasks.cephadm:Adding mds.d on smithi016 2024-08-31T06:03:33.478 INFO:tasks.cephadm:Adding mds.g on smithi016 2024-08-31T06:03:33.478 INFO:tasks.cephadm:Adding mds.j on smithi016 2024-08-31T06:03:33.478 INFO:tasks.cephadm:Adding mds.b on smithi155 2024-08-31T06:03:33.478 INFO:tasks.cephadm:Adding mds.e on smithi155 2024-08-31T06:03:33.478 INFO:tasks.cephadm:Adding mds.h on smithi155 2024-08-31T06:03:33.478 INFO:tasks.cephadm:Adding mds.k on smithi155 2024-08-31T06:03:33.478 INFO:tasks.cephadm:Adding mds.c on smithi184 2024-08-31T06:03:33.479 INFO:tasks.cephadm:Adding mds.f on smithi184 2024-08-31T06:03:33.479 INFO:tasks.cephadm:Adding mds.i on smithi184 2024-08-31T06:03:33.479 INFO:tasks.cephadm:Adding mds.l on smithi184 2024-08-31T06:03:33.479 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph orch apply mds all '12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l' 2024-08-31T06:03:33.739 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.c/config 2024-08-31T06:03:34.246 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.222+0000 7f9d30281640 1 -- 172.21.15.184:0/210020477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d28105120 msgr2=0x7f9d2810d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:34.246 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.222+0000 7f9d30281640 1 --2- 172.21.15.184:0/210020477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d28105120 0x7f9d2810d670 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9d18009900 tx=0x7f9d1802a410 comp rx=0 tx=0).stop 2024-08-31T06:03:34.246 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.222+0000 7f9d30281640 1 -- 172.21.15.184:0/210020477 shutdown_connections 2024-08-31T06:03:34.247 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.222+0000 7f9d30281640 1 --2- 172.21.15.184:0/210020477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d28105120 0x7f9d2810d670 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:34.247 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.222+0000 7f9d30281640 1 --2- 172.21.15.184:0/210020477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d2810dc00 0x7f9d281100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:34.247 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.222+0000 7f9d30281640 1 --2- 172.21.15.184:0/210020477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d281047d0 0x7f9d28104be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:34.247 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.222+0000 7f9d30281640 1 -- 172.21.15.184:0/210020477 >> 172.21.15.184:0/210020477 conn(0x7f9d28079550 msgr2=0x7f9d28079960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:03:34.247 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.222+0000 7f9d30281640 1 -- 172.21.15.184:0/210020477 shutdown_connections 2024-08-31T06:03:34.247 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.222+0000 7f9d30281640 1 -- 172.21.15.184:0/210020477 wait complete. 2024-08-31T06:03:34.247 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.223+0000 7f9d30281640 1 Processor -- start 2024-08-31T06:03:34.247 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.223+0000 7f9d30281640 1 -- start start 2024-08-31T06:03:34.247 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.223+0000 7f9d30281640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d281047d0 0x7f9d2819a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:34.247 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.223+0000 7f9d30281640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d28105120 0x7f9d2819a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:34.247 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.223+0000 7f9d30281640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d2810dc00 0x7f9d2819fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:34.247 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.223+0000 7f9d30281640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9d28113880 con 0x7f9d281047d0 2024-08-31T06:03:34.247 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.223+0000 7f9d30281640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9d28113700 con 0x7f9d2810dc00 2024-08-31T06:03:34.248 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.223+0000 7f9d30281640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9d28113a00 con 0x7f9d28105120 2024-08-31T06:03:34.248 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.223+0000 7f9d2d81c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d28105120 0x7f9d2819a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:34.248 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.223+0000 7f9d2d81c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d28105120 0x7f9d2819a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:53764/0 (socket says 172.21.15.184:53764) 2024-08-31T06:03:34.248 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.223+0000 7f9d2d81c640 1 -- 172.21.15.184:0/1873754017 learned_addr learned my addr 172.21.15.184:0/1873754017 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-08-31T06:03:34.248 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.223+0000 7f9d2e81e640 1 --2- 172.21.15.184:0/1873754017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d2810dc00 0x7f9d2819fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:34.248 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.224+0000 7f9d2e01d640 1 --2- 172.21.15.184:0/1873754017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d281047d0 0x7f9d2819a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:34.248 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.224+0000 7f9d2d81c640 1 -- 172.21.15.184:0/1873754017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d2810dc00 msgr2=0x7f9d2819fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:34.248 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.224+0000 7f9d2d81c640 1 --2- 172.21.15.184:0/1873754017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d2810dc00 0x7f9d2819fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:34.248 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.224+0000 7f9d2d81c640 1 -- 172.21.15.184:0/1873754017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d281047d0 msgr2=0x7f9d2819a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:34.248 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.224+0000 7f9d2d81c640 1 --2- 172.21.15.184:0/1873754017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d281047d0 0x7f9d2819a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:34.248 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.224+0000 7f9d2d81c640 1 -- 172.21.15.184:0/1873754017 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d281a01e0 con 0x7f9d28105120 2024-08-31T06:03:34.248 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.224+0000 7f9d2e81e640 1 --2- 172.21.15.184:0/1873754017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d2810dc00 0x7f9d2819fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:03:34.248 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.224+0000 7f9d2e01d640 1 --2- 172.21.15.184:0/1873754017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d281047d0 0x7f9d2819a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:03:34.248 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.224+0000 7f9d2d81c640 1 --2- 172.21.15.184:0/1873754017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d28105120 0x7f9d2819a810 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9d1802a930 tx=0x7f9d18004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:03:34.249 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.224+0000 7f9d1f7fe640 1 -- 172.21.15.184:0/1873754017 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1803f030 con 0x7f9d28105120 2024-08-31T06:03:34.249 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.224+0000 7f9d1f7fe640 1 -- 172.21.15.184:0/1873754017 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d18003aa0 con 0x7f9d28105120 2024-08-31T06:03:34.249 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.224+0000 7f9d30281640 1 -- 172.21.15.184:0/1873754017 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d281a0470 con 0x7f9d28105120 2024-08-31T06:03:34.249 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.224+0000 7f9d1f7fe640 1 -- 172.21.15.184:0/1873754017 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d18035070 con 0x7f9d28105120 2024-08-31T06:03:34.249 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.224+0000 7f9d30281640 1 -- 172.21.15.184:0/1873754017 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d281a6c30 con 0x7f9d28105120 2024-08-31T06:03:34.249 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.225+0000 7f9d30281640 1 -- 172.21.15.184:0/1873754017 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d2806bc70 con 0x7f9d28105120 2024-08-31T06:03:34.249 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.227+0000 7f9d1f7fe640 1 -- 172.21.15.184:0/1873754017 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7f9d18042030 con 0x7f9d28105120 2024-08-31T06:03:34.249 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.228+0000 7f9d1f7fe640 1 --2- 172.21.15.184:0/1873754017 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9d140b4b80 0x7f9d140b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:34.249 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.228+0000 7f9d1f7fe640 1 -- 172.21.15.184:0/1873754017 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7f9d18105950 con 0x7f9d28105120 2024-08-31T06:03:34.249 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.228+0000 7f9d2e01d640 1 --2- 172.21.15.184:0/1873754017 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9d140b4b80 0x7f9d140b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:34.249 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.229+0000 7f9d2e01d640 1 --2- 172.21.15.184:0/1873754017 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9d140b4b80 0x7f9d140b7090 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9d24005d00 tx=0x7f9d24005ed0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:03:34.249 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.231+0000 7f9d1f7fe640 1 -- 172.21.15.184:0/1873754017 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f9d180cce30 con 0x7f9d28105120 2024-08-31T06:03:34.421 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.419+0000 7f9d30281640 1 -- 172.21.15.184:0/1873754017 --> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l", "target": ["mon-mgr", ""]}) -- 0x7f9d280636a0 con 0x7f9d140b4b80 2024-08-31T06:03:34.432 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.430+0000 7f9d1f7fe640 1 -- 172.21.15.184:0/1873754017 <== mgr.14220 v2:172.21.15.16:6800/4279751337 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f9d280636a0 con 0x7f9d140b4b80 2024-08-31T06:03:34.432 INFO:teuthology.orchestra.run.smithi184.stdout:Scheduled mds.all update... 2024-08-31T06:03:34.435 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.433+0000 7f9d30281640 1 -- 172.21.15.184:0/1873754017 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9d140b4b80 msgr2=0x7f9d140b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:34.435 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.433+0000 7f9d30281640 1 --2- 172.21.15.184:0/1873754017 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9d140b4b80 0x7f9d140b7090 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9d24005d00 tx=0x7f9d24005ed0 comp rx=0 tx=0).stop 2024-08-31T06:03:34.435 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.433+0000 7f9d30281640 1 -- 172.21.15.184:0/1873754017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d28105120 msgr2=0x7f9d2819a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:34.435 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.433+0000 7f9d30281640 1 --2- 172.21.15.184:0/1873754017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d28105120 0x7f9d2819a810 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9d1802a930 tx=0x7f9d18004270 comp rx=0 tx=0).stop 2024-08-31T06:03:34.435 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.433+0000 7f9d30281640 1 -- 172.21.15.184:0/1873754017 shutdown_connections 2024-08-31T06:03:34.435 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.433+0000 7f9d30281640 1 --2- 172.21.15.184:0/1873754017 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9d140b4b80 0x7f9d140b7090 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:34.436 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.433+0000 7f9d30281640 1 --2- 172.21.15.184:0/1873754017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d28105120 0x7f9d2819a810 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:34.436 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.433+0000 7f9d30281640 1 --2- 172.21.15.184:0/1873754017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d2810dc00 0x7f9d2819fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:34.436 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.433+0000 7f9d30281640 1 --2- 172.21.15.184:0/1873754017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d281047d0 0x7f9d2819a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:34.436 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.433+0000 7f9d30281640 1 -- 172.21.15.184:0/1873754017 >> 172.21.15.184:0/1873754017 conn(0x7f9d28079550 msgr2=0x7f9d28100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:03:34.436 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.433+0000 7f9d30281640 1 -- 172.21.15.184:0/1873754017 shutdown_connections 2024-08-31T06:03:34.436 INFO:teuthology.orchestra.run.smithi184.stderr:2024-08-31T06:03:34.433+0000 7f9d30281640 1 -- 172.21.15.184:0/1873754017 wait complete. 2024-08-31T06:03:35.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:34 smithi184 ceph-mon[22331]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:35.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:34 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:35.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:34 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:03:35.165 DEBUG:teuthology.orchestra.run.smithi016:mds.a> sudo journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mds.a.service 2024-08-31T06:03:35.169 DEBUG:teuthology.orchestra.run.smithi016:mds.d> sudo journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mds.d.service 2024-08-31T06:03:35.215 DEBUG:teuthology.orchestra.run.smithi016:mds.g> sudo journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mds.g.service 2024-08-31T06:03:35.215 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:34 smithi016 ceph-mon[17194]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:35.216 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:34 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:35.216 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:34 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:03:35.260 DEBUG:teuthology.orchestra.run.smithi016:mds.j> sudo journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mds.j.service 2024-08-31T06:03:35.306 DEBUG:teuthology.orchestra.run.smithi155:mds.b> sudo journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mds.b.service 2024-08-31T06:03:35.309 DEBUG:teuthology.orchestra.run.smithi155:mds.e> sudo journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mds.e.service 2024-08-31T06:03:35.331 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:34 smithi155 ceph-mon[21742]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:35.331 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:34 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:35.331 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:34 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:03:35.332 DEBUG:teuthology.orchestra.run.smithi155:mds.h> sudo journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mds.h.service 2024-08-31T06:03:35.375 DEBUG:teuthology.orchestra.run.smithi155:mds.k> sudo journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mds.k.service 2024-08-31T06:03:35.418 DEBUG:teuthology.orchestra.run.smithi184:mds.c> sudo journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mds.c.service 2024-08-31T06:03:35.420 DEBUG:teuthology.orchestra.run.smithi184:mds.f> sudo journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mds.f.service 2024-08-31T06:03:35.463 DEBUG:teuthology.orchestra.run.smithi184:mds.i> sudo journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mds.i.service 2024-08-31T06:03:35.506 DEBUG:teuthology.orchestra.run.smithi184:mds.l> sudo journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mds.l.service 2024-08-31T06:03:35.549 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-08-31T06:03:35.549 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:03:35.922 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:03:35.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:35 smithi184 ceph-mon[22331]: from='client.24625 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:03:35.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:35 smithi184 ceph-mon[22331]: Saving service mds.all spec with placement smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l;count:12 2024-08-31T06:03:35.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:35 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:03:35.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:35 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:03:35.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:35 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:35.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:35 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:35.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:35 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:35.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:35 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T06:03:35.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:35 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:03:36.302 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:35 smithi016 ceph-mon[17194]: from='client.24625 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:03:36.302 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:35 smithi016 ceph-mon[17194]: Saving service mds.all spec with placement smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l;count:12 2024-08-31T06:03:36.302 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:35 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:03:36.303 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:35 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:03:36.303 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:35 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:36.303 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:35 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:36.303 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:35 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:36.303 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:35 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T06:03:36.303 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:35 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:03:36.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:35 smithi155 ceph-mon[21742]: from='client.24625 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:03:36.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:35 smithi155 ceph-mon[21742]: Saving service mds.all spec with placement smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l;count:12 2024-08-31T06:03:36.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:35 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:03:36.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:35 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:03:36.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:35 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:36.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:35 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:36.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:35 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:36.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:35 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T06:03:36.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:35 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:03:36.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.620+0000 7f6233e00640 1 -- 172.21.15.16:0/1949215518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f623410bbe0 msgr2=0x7f6234112410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:36.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.620+0000 7f6233e00640 1 --2- 172.21.15.16:0/1949215518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f623410bbe0 0x7f6234112410 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f622c009ca0 tx=0x7f622c02a8c0 comp rx=0 tx=0).stop 2024-08-31T06:03:36.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.620+0000 7f6233e00640 1 -- 172.21.15.16:0/1949215518 shutdown_connections 2024-08-31T06:03:36.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.620+0000 7f6233e00640 1 --2- 172.21.15.16:0/1949215518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f623410bbe0 0x7f6234112410 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:36.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.620+0000 7f6233e00640 1 --2- 172.21.15.16:0/1949215518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f623410b290 0x7f623410b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:36.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.620+0000 7f6233e00640 1 --2- 172.21.15.16:0/1949215518 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6234105950 0x7f6234105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:36.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.620+0000 7f6233e00640 1 -- 172.21.15.16:0/1949215518 >> 172.21.15.16:0/1949215518 conn(0x7f6234101420 msgr2=0x7f6234103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:03:36.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.620+0000 7f6233e00640 1 -- 172.21.15.16:0/1949215518 shutdown_connections 2024-08-31T06:03:36.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.620+0000 7f6233e00640 1 -- 172.21.15.16:0/1949215518 wait complete. 2024-08-31T06:03:36.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.621+0000 7f6233e00640 1 Processor -- start 2024-08-31T06:03:36.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.621+0000 7f6233e00640 1 -- start start 2024-08-31T06:03:36.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.621+0000 7f6233e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6234105950 0x7f623419c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:36.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.622+0000 7f6233e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f623410b290 0x7f623419c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:36.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.622+0000 7f6233e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f623410bbe0 0x7f62341a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:36.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.622+0000 7f6233e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f62341158f0 con 0x7f623410bbe0 2024-08-31T06:03:36.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.622+0000 7f6233e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6234115770 con 0x7f623410b290 2024-08-31T06:03:36.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.622+0000 7f6233e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6234115a70 con 0x7f6234105950 2024-08-31T06:03:36.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.622+0000 7f6232a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6234105950 0x7f623419c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:36.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.622+0000 7f6232a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6234105950 0x7f623419c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:39320/0 (socket says 172.21.15.16:39320) 2024-08-31T06:03:36.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.622+0000 7f6232a00640 1 -- 172.21.15.16:0/465384055 learned_addr learned my addr 172.21.15.16:0/465384055 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:03:36.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.622+0000 7f6233400640 1 --2- 172.21.15.16:0/465384055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f623410bbe0 0x7f62341a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:36.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.622+0000 7f622be00640 1 --2- 172.21.15.16:0/465384055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f623410b290 0x7f623419c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:36.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.622+0000 7f6232a00640 1 -- 172.21.15.16:0/465384055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f623410b290 msgr2=0x7f623419c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:36.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.622+0000 7f6232a00640 1 --2- 172.21.15.16:0/465384055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f623410b290 0x7f623419c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:36.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.622+0000 7f6232a00640 1 -- 172.21.15.16:0/465384055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f623410bbe0 msgr2=0x7f62341a1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:36.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.622+0000 7f6232a00640 1 --2- 172.21.15.16:0/465384055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f623410bbe0 0x7f62341a1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.622+0000 7f6232a00640 1 -- 172.21.15.16:0/465384055 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62341a21f0 con 0x7f6234105950 2024-08-31T06:03:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.623+0000 7f6232a00640 1 --2- 172.21.15.16:0/465384055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6234105950 0x7f623419c280 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f622400e9f0 tx=0x7f622400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:03:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.623+0000 7f622b400640 1 -- 172.21.15.16:0/465384055 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622400cd80 con 0x7f6234105950 2024-08-31T06:03:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.623+0000 7f6233e00640 1 -- 172.21.15.16:0/465384055 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62341a24e0 con 0x7f6234105950 2024-08-31T06:03:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.623+0000 7f6233e00640 1 -- 172.21.15.16:0/465384055 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62341a8c40 con 0x7f6234105950 2024-08-31T06:03:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.623+0000 7f622b400640 1 -- 172.21.15.16:0/465384055 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62240054d0 con 0x7f6234105950 2024-08-31T06:03:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.623+0000 7f622b400640 1 -- 172.21.15.16:0/465384055 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6224010430 con 0x7f6234105950 2024-08-31T06:03:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.624+0000 7f6233e00640 1 -- 172.21.15.16:0/465384055 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62100052c0 con 0x7f6234105950 2024-08-31T06:03:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.627+0000 7f622b400640 1 -- 172.21.15.16:0/465384055 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7f6224005000 con 0x7f6234105950 2024-08-31T06:03:36.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.628+0000 7f622b400640 1 --2- 172.21.15.16:0/465384055 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f62180b4b80 0x7f62180b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:36.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.628+0000 7f622b400640 1 -- 172.21.15.16:0/465384055 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7f62240e4fe0 con 0x7f6234105950 2024-08-31T06:03:36.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.628+0000 7f622be00640 1 --2- 172.21.15.16:0/465384055 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f62180b4b80 0x7f62180b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:36.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.629+0000 7f622be00640 1 --2- 172.21.15.16:0/465384055 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f62180b4b80 0x7f62180b7090 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f623410acf0 tx=0x7f621c00b390 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:03:36.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.632+0000 7f622b400640 1 -- 172.21.15.16:0/465384055 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f62240052f0 con 0x7f6234105950 2024-08-31T06:03:36.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.865+0000 7f6233e00640 1 -- 172.21.15.16:0/465384055 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62100055b0 con 0x7f6234105950 2024-08-31T06:03:36.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.866+0000 7f622b400640 1 -- 172.21.15.16:0/465384055 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f62240ac460 con 0x7f6234105950 2024-08-31T06:03:36.869 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:03:36.869 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":1,"btime":"2024-08-31T05:55:09:632201+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-08-31T06:03:36.869 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 1 2024-08-31T06:03:36.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.868+0000 7f6233e00640 1 -- 172.21.15.16:0/465384055 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f62180b4b80 msgr2=0x7f62180b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:36.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.868+0000 7f6233e00640 1 --2- 172.21.15.16:0/465384055 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f62180b4b80 0x7f62180b7090 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f623410acf0 tx=0x7f621c00b390 comp rx=0 tx=0).stop 2024-08-31T06:03:36.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.868+0000 7f6233e00640 1 -- 172.21.15.16:0/465384055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6234105950 msgr2=0x7f623419c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:36.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.868+0000 7f6233e00640 1 --2- 172.21.15.16:0/465384055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6234105950 0x7f623419c280 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f622400e9f0 tx=0x7f622400edf0 comp rx=0 tx=0).stop 2024-08-31T06:03:36.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.869+0000 7f6233e00640 1 -- 172.21.15.16:0/465384055 shutdown_connections 2024-08-31T06:03:36.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.869+0000 7f6233e00640 1 --2- 172.21.15.16:0/465384055 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f62180b4b80 0x7f62180b7090 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:36.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.869+0000 7f6233e00640 1 --2- 172.21.15.16:0/465384055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f623410bbe0 0x7f62341a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:36.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.869+0000 7f6233e00640 1 --2- 172.21.15.16:0/465384055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f623410b290 0x7f623419c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:36.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.869+0000 7f6233e00640 1 --2- 172.21.15.16:0/465384055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6234105950 0x7f623419c280 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:36.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.869+0000 7f6233e00640 1 -- 172.21.15.16:0/465384055 >> 172.21.15.16:0/465384055 conn(0x7f6234101420 msgr2=0x7f62341088b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:03:36.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.869+0000 7f6233e00640 1 -- 172.21.15.16:0/465384055 shutdown_connections 2024-08-31T06:03:36.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:36.869+0000 7f6233e00640 1 -- 172.21.15.16:0/465384055 wait complete. 2024-08-31T06:03:37.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:36 smithi016 ceph-mon[17194]: Deploying daemon mds.l on smithi184 2024-08-31T06:03:37.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:36 smithi016 ceph-mon[17194]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:37.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:36 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/465384055' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:03:37.223 INFO:journalctl@ceph.mds.l.smithi184.stdout:Aug 31 06:03:37 smithi184 systemd[1]: Starting Ceph mds.l for 673e4aa0-675d-11ef-bcd4-c7b262605968... 2024-08-31T06:03:37.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:36 smithi184 ceph-mon[22331]: Deploying daemon mds.l on smithi184 2024-08-31T06:03:37.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:36 smithi184 ceph-mon[22331]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:37.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:36 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/465384055' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:03:37.274 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:36 smithi155 ceph-mon[21742]: Deploying daemon mds.l on smithi184 2024-08-31T06:03:37.274 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:36 smithi155 ceph-mon[21742]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:37.274 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:36 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/465384055' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:03:37.605 INFO:journalctl@ceph.mds.l.smithi184.stdout:Aug 31 06:03:37 smithi184 podman[53472]: 2024-08-31 06:03:37.211019981 +0000 UTC m=+0.015735068 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T06:03:37.605 INFO:journalctl@ceph.mds.l.smithi184.stdout:Aug 31 06:03:37 smithi184 podman[53472]: 2024-08-31 06:03:37.328710574 +0000 UTC m=+0.133425661 container create 2eff53ca9a134cc00ac9f4c8a59ad4abe94631e4705031ccbb871707b9a76dfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-l, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.1, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-08-31T06:03:37.972 INFO:journalctl@ceph.mds.l.smithi184.stdout:Aug 31 06:03:37 smithi184 podman[53472]: 2024-08-31 06:03:37.603769476 +0000 UTC m=+0.408484564 container init 2eff53ca9a134cc00ac9f4c8a59ad4abe94631e4705031ccbb871707b9a76dfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-l, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.1, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-31T06:03:37.972 INFO:journalctl@ceph.mds.l.smithi184.stdout:Aug 31 06:03:37 smithi184 podman[53472]: 2024-08-31 06:03:37.606619713 +0000 UTC m=+0.411334801 container start 2eff53ca9a134cc00ac9f4c8a59ad4abe94631e4705031ccbb871707b9a76dfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-l, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.1) 2024-08-31T06:03:37.973 INFO:journalctl@ceph.mds.l.smithi184.stdout:Aug 31 06:03:37 smithi184 ceph-mds[53493]: -- 172.21.15.184:0/3364932993 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56113b7b3860 con 0x56113f959a80 2024-08-31T06:03:37.973 INFO:journalctl@ceph.mds.l.smithi184.stdout:Aug 31 06:03:37 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-l[53489]: starting mds.l at 2024-08-31T06:03:37.973 INFO:journalctl@ceph.mds.l.smithi184.stdout:Aug 31 06:03:37 smithi184 bash[53472]: 2eff53ca9a134cc00ac9f4c8a59ad4abe94631e4705031ccbb871707b9a76dfc 2024-08-31T06:03:37.973 INFO:journalctl@ceph.mds.l.smithi184.stdout:Aug 31 06:03:37 smithi184 systemd[1]: Started Ceph mds.l for 673e4aa0-675d-11ef-bcd4-c7b262605968. 2024-08-31T06:03:38.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:38 smithi155 ceph-mon[21742]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:38.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:38 smithi155 ceph-mon[21742]: mds.? [v2:172.21.15.184:6832/1204885928,v1:172.21.15.184:6833/1204885928] up:boot 2024-08-31T06:03:38.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:38 smithi155 ceph-mon[21742]: fsmap 1 up:standby 2024-08-31T06:03:38.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:38 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-08-31T06:03:38.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:38 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:38.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:38 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:38.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:38 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:38.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:38 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:38.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:38 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:38.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:38 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T06:03:38.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:38 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:03:38.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:38 smithi155 ceph-mon[21742]: Deploying daemon mds.k on smithi155 2024-08-31T06:03:38.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:38 smithi016 ceph-mon[17194]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:38.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:38 smithi016 ceph-mon[17194]: mds.? [v2:172.21.15.184:6832/1204885928,v1:172.21.15.184:6833/1204885928] up:boot 2024-08-31T06:03:38.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:38 smithi016 ceph-mon[17194]: fsmap 1 up:standby 2024-08-31T06:03:38.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:38 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-08-31T06:03:38.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:38 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:38.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:38 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:38.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:38 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:38.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:38 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:38.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:38 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:38.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:38 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T06:03:38.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:38 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:03:38.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:38 smithi016 ceph-mon[17194]: Deploying daemon mds.k on smithi155 2024-08-31T06:03:38.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:38 smithi184 ceph-mon[22331]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:38.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:38 smithi184 ceph-mon[22331]: mds.? [v2:172.21.15.184:6832/1204885928,v1:172.21.15.184:6833/1204885928] up:boot 2024-08-31T06:03:38.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:38 smithi184 ceph-mon[22331]: fsmap 1 up:standby 2024-08-31T06:03:38.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:38 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-08-31T06:03:38.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:38 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:38.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:38 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:38.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:38 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:38.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:38 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:38.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:38 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:38.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:38 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T06:03:38.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:38 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:03:38.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:38 smithi184 ceph-mon[22331]: Deploying daemon mds.k on smithi155 2024-08-31T06:03:39.655 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:03:40.039 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:03:40.122 INFO:journalctl@ceph.mds.k.smithi155.stdout:Aug 31 06:03:40 smithi155 systemd[1]: Starting Ceph mds.k for 673e4aa0-675d-11ef-bcd4-c7b262605968... 2024-08-31T06:03:40.622 INFO:journalctl@ceph.mds.k.smithi155.stdout:Aug 31 06:03:40 smithi155 podman[52420]: 2024-08-31 06:03:40.226143597 +0000 UTC m=+0.015802370 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T06:03:40.622 INFO:journalctl@ceph.mds.k.smithi155.stdout:Aug 31 06:03:40 smithi155 podman[52420]: 2024-08-31 06:03:40.357302279 +0000 UTC m=+0.146961042 container create e3c24c5dbb468e1b40ec87a9492dd7c429f41ae029d0e5cf4b3c28e6f89d8e87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-k, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-08-31T06:03:40.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.714+0000 7f7ea9e00640 1 -- 172.21.15.16:0/6820737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ea4105b20 msgr2=0x7f7ea4105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:40.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.714+0000 7f7ea9e00640 1 --2- 172.21.15.16:0/6820737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ea4105b20 0x7f7ea4105f30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7e94009900 tx=0x7f7e9402a3f0 comp rx=0 tx=0).stop 2024-08-31T06:03:40.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.714+0000 7f7ea9e00640 1 -- 172.21.15.16:0/6820737 shutdown_connections 2024-08-31T06:03:40.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.714+0000 7f7ea9e00640 1 --2- 172.21.15.16:0/6820737 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ea410bdb0 0x7f7ea41125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:40.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.714+0000 7f7ea9e00640 1 --2- 172.21.15.16:0/6820737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ea410b460 0x7f7ea410b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:40.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.714+0000 7f7ea9e00640 1 --2- 172.21.15.16:0/6820737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ea4105b20 0x7f7ea4105f30 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:40.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.714+0000 7f7ea9e00640 1 -- 172.21.15.16:0/6820737 >> 172.21.15.16:0/6820737 conn(0x7f7ea4101570 msgr2=0x7f7ea41039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:03:40.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.714+0000 7f7ea9e00640 1 -- 172.21.15.16:0/6820737 shutdown_connections 2024-08-31T06:03:40.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.714+0000 7f7ea9e00640 1 -- 172.21.15.16:0/6820737 wait complete. 2024-08-31T06:03:40.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.715+0000 7f7ea9e00640 1 Processor -- start 2024-08-31T06:03:40.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.716+0000 7f7ea9e00640 1 -- start start 2024-08-31T06:03:40.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.716+0000 7f7ea9e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ea4105b20 0x7f7ea419c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:40.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.716+0000 7f7ea9e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ea410b460 0x7f7ea419ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:40.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.716+0000 7f7ea9e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ea410bdb0 0x7f7ea41a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:40.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.716+0000 7f7ea9e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea4115a70 con 0x7f7ea410bdb0 2024-08-31T06:03:40.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.716+0000 7f7ea9e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea41158f0 con 0x7f7ea410b460 2024-08-31T06:03:40.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.716+0000 7f7ea9e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea4115bf0 con 0x7f7ea4105b20 2024-08-31T06:03:40.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.716+0000 7f7ea3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ea410bdb0 0x7f7ea41a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:40.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.716+0000 7f7e9a000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ea410b460 0x7f7ea419ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:40.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.716+0000 7f7e9a000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ea410b460 0x7f7ea419ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:55202/0 (socket says 172.21.15.16:55202) 2024-08-31T06:03:40.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.716+0000 7f7e9a000640 1 -- 172.21.15.16:0/3007477791 learned_addr learned my addr 172.21.15.16:0/3007477791 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:03:40.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.716+0000 7f7ea2a00640 1 --2- 172.21.15.16:0/3007477791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ea4105b20 0x7f7ea419c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:40.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.717+0000 7f7e9a000640 1 -- 172.21.15.16:0/3007477791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ea4105b20 msgr2=0x7f7ea419c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:40.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.717+0000 7f7e9a000640 1 --2- 172.21.15.16:0/3007477791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ea4105b20 0x7f7ea419c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:40.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.717+0000 7f7e9a000640 1 -- 172.21.15.16:0/3007477791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ea410bdb0 msgr2=0x7f7ea41a1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:40.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.717+0000 7f7e9a000640 1 --2- 172.21.15.16:0/3007477791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ea410bdb0 0x7f7ea41a1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:40.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.717+0000 7f7e9a000640 1 -- 172.21.15.16:0/3007477791 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea41a23f0 con 0x7f7ea410b460 2024-08-31T06:03:40.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.717+0000 7f7e9a000640 1 --2- 172.21.15.16:0/3007477791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ea410b460 0x7f7ea419ca20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c00e9f0 tx=0x7f7e8c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:03:40.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.717+0000 7f7ea8a00640 1 -- 172.21.15.16:0/3007477791 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c00ccc0 con 0x7f7ea410b460 2024-08-31T06:03:40.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.717+0000 7f7ea9e00640 1 -- 172.21.15.16:0/3007477791 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ea41a26e0 con 0x7f7ea410b460 2024-08-31T06:03:40.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.717+0000 7f7ea8a00640 1 -- 172.21.15.16:0/3007477791 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e8c0108e0 con 0x7f7ea410b460 2024-08-31T06:03:40.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.718+0000 7f7ea8a00640 1 -- 172.21.15.16:0/3007477791 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c018860 con 0x7f7ea410b460 2024-08-31T06:03:40.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.718+0000 7f7ea9e00640 1 -- 172.21.15.16:0/3007477791 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ea41a8e40 con 0x7f7ea410b460 2024-08-31T06:03:40.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.719+0000 7f7ea9e00640 1 -- 172.21.15.16:0/3007477791 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ea4105f30 con 0x7f7ea410b460 2024-08-31T06:03:40.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.721+0000 7f7ea8a00640 1 -- 172.21.15.16:0/3007477791 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7f7e8c010430 con 0x7f7ea410b460 2024-08-31T06:03:40.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.722+0000 7f7ea8a00640 1 --2- 172.21.15.16:0/3007477791 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7e880b4b80 0x7f7e880b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:40.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.723+0000 7f7ea8a00640 1 -- 172.21.15.16:0/3007477791 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7f7e8c0e5390 con 0x7f7ea410b460 2024-08-31T06:03:40.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.723+0000 7f7ea2a00640 1 --2- 172.21.15.16:0/3007477791 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7e880b4b80 0x7f7e880b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:40.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.723+0000 7f7ea2a00640 1 --2- 172.21.15.16:0/3007477791 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7e880b4b80 0x7f7e880b7090 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7e94031440 tx=0x7f7e94009560 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:03:40.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.724+0000 7f7ea8a00640 1 -- 172.21.15.16:0/3007477791 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f7e8c0ac1a0 con 0x7f7ea410b460 2024-08-31T06:03:40.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:40 smithi016 ceph-mon[17194]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:40.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.959+0000 7f7ea9e00640 1 -- 172.21.15.16:0/3007477791 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ea419d5b0 con 0x7f7ea410b460 2024-08-31T06:03:40.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.959+0000 7f7ea8a00640 1 -- 172.21.15.16:0/3007477791 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7f7ea419d5b0 con 0x7f7ea410b460 2024-08-31T06:03:40.962 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:03:40.962 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":2,"btime":"2024-08-31T06:03:37:644731+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-08-31T06:03:40.962 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 2 2024-08-31T06:03:40.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.962+0000 7f7ea9e00640 1 -- 172.21.15.16:0/3007477791 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7e880b4b80 msgr2=0x7f7e880b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:40.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.962+0000 7f7ea9e00640 1 --2- 172.21.15.16:0/3007477791 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7e880b4b80 0x7f7e880b7090 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7e94031440 tx=0x7f7e94009560 comp rx=0 tx=0).stop 2024-08-31T06:03:40.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.962+0000 7f7ea9e00640 1 -- 172.21.15.16:0/3007477791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ea410b460 msgr2=0x7f7ea419ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:40.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.962+0000 7f7ea9e00640 1 --2- 172.21.15.16:0/3007477791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ea410b460 0x7f7ea419ca20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c00e9f0 tx=0x7f7e8c00eeb0 comp rx=0 tx=0).stop 2024-08-31T06:03:40.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.962+0000 7f7ea9e00640 1 -- 172.21.15.16:0/3007477791 shutdown_connections 2024-08-31T06:03:40.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.962+0000 7f7ea9e00640 1 --2- 172.21.15.16:0/3007477791 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7e880b4b80 0x7f7e880b7090 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:40.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.962+0000 7f7ea9e00640 1 --2- 172.21.15.16:0/3007477791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ea410bdb0 0x7f7ea41a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:40.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.962+0000 7f7ea9e00640 1 --2- 172.21.15.16:0/3007477791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ea410b460 0x7f7ea419ca20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:40.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.962+0000 7f7ea9e00640 1 --2- 172.21.15.16:0/3007477791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ea4105b20 0x7f7ea419c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:40.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.962+0000 7f7ea9e00640 1 -- 172.21.15.16:0/3007477791 >> 172.21.15.16:0/3007477791 conn(0x7f7ea4101570 msgr2=0x7f7ea4103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:03:40.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.962+0000 7f7ea9e00640 1 -- 172.21.15.16:0/3007477791 shutdown_connections 2024-08-31T06:03:40.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:40.962+0000 7f7ea9e00640 1 -- 172.21.15.16:0/3007477791 wait complete. 2024-08-31T06:03:40.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:40 smithi184 ceph-mon[22331]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:41.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:40 smithi155 ceph-mon[21742]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:41.122 INFO:journalctl@ceph.mds.k.smithi155.stdout:Aug 31 06:03:40 smithi155 podman[52420]: 2024-08-31 06:03:40.635103532 +0000 UTC m=+0.424762298 container init e3c24c5dbb468e1b40ec87a9492dd7c429f41ae029d0e5cf4b3c28e6f89d8e87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-k, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-08-31T06:03:41.122 INFO:journalctl@ceph.mds.k.smithi155.stdout:Aug 31 06:03:40 smithi155 podman[52420]: 2024-08-31 06:03:40.638147518 +0000 UTC m=+0.427806285 container start e3c24c5dbb468e1b40ec87a9492dd7c429f41ae029d0e5cf4b3c28e6f89d8e87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-k, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-08-31T06:03:41.122 INFO:journalctl@ceph.mds.k.smithi155.stdout:Aug 31 06:03:40 smithi155 ceph-mds[52440]: -- 172.21.15.155:0/2075615312 <== mon.2 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f0fae21860 con 0x55f0fefc7600 2024-08-31T06:03:41.122 INFO:journalctl@ceph.mds.k.smithi155.stdout:Aug 31 06:03:40 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-k[52436]: starting mds.k at 2024-08-31T06:03:41.122 INFO:journalctl@ceph.mds.k.smithi155.stdout:Aug 31 06:03:40 smithi155 bash[52420]: e3c24c5dbb468e1b40ec87a9492dd7c429f41ae029d0e5cf4b3c28e6f89d8e87 2024-08-31T06:03:41.122 INFO:journalctl@ceph.mds.k.smithi155.stdout:Aug 31 06:03:40 smithi155 systemd[1]: Started Ceph mds.k for 673e4aa0-675d-11ef-bcd4-c7b262605968. 2024-08-31T06:03:42.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:41 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:42.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:41 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:42.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:41 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:42.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:41 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:42.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:41 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:42.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:41 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T06:03:42.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:41 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:03:42.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:41 smithi016 ceph-mon[17194]: Deploying daemon mds.j on smithi016 2024-08-31T06:03:42.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:41 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/3007477791' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:03:42.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:41 smithi016 ceph-mon[17194]: mds.? [v2:172.21.15.155:6832/2239981116,v1:172.21.15.155:6833/2239981116] up:boot 2024-08-31T06:03:42.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:41 smithi016 ceph-mon[17194]: fsmap 2 up:standby 2024-08-31T06:03:42.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:41 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-08-31T06:03:42.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:41 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:42.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:41 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:42.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:41 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:42.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:41 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:42.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:41 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:42.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:41 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T06:03:42.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:41 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:03:42.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:41 smithi155 ceph-mon[21742]: Deploying daemon mds.j on smithi016 2024-08-31T06:03:42.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:41 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/3007477791' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:03:42.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:41 smithi155 ceph-mon[21742]: mds.? [v2:172.21.15.155:6832/2239981116,v1:172.21.15.155:6833/2239981116] up:boot 2024-08-31T06:03:42.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:41 smithi155 ceph-mon[21742]: fsmap 2 up:standby 2024-08-31T06:03:42.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:41 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-08-31T06:03:42.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:41 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:42.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:41 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:42.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:41 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:42.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:41 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:42.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:41 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:42.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:41 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T06:03:42.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:41 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:03:42.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:41 smithi184 ceph-mon[22331]: Deploying daemon mds.j on smithi016 2024-08-31T06:03:42.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:41 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/3007477791' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:03:42.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:41 smithi184 ceph-mon[22331]: mds.? [v2:172.21.15.155:6832/2239981116,v1:172.21.15.155:6833/2239981116] up:boot 2024-08-31T06:03:42.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:41 smithi184 ceph-mon[22331]: fsmap 2 up:standby 2024-08-31T06:03:42.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:41 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-08-31T06:03:43.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:42 smithi155 ceph-mon[21742]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:43.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:42 smithi016 ceph-mon[17194]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:43.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:42 smithi184 ceph-mon[22331]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:43.873 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:03:44.294 INFO:journalctl@ceph.mds.j.smithi016.stdout:Aug 31 06:03:43 smithi016 systemd[1]: Starting Ceph mds.j for 673e4aa0-675d-11ef-bcd4-c7b262605968... 2024-08-31T06:03:44.568 INFO:journalctl@ceph.mds.j.smithi016.stdout:Aug 31 06:03:44 smithi016 podman[60790]: 2024-08-31 06:03:44.196202992 +0000 UTC m=+0.029579589 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T06:03:44.568 INFO:journalctl@ceph.mds.j.smithi016.stdout:Aug 31 06:03:44 smithi016 podman[60790]: 2024-08-31 06:03:44.304346701 +0000 UTC m=+0.137723303 container create 66f8fc11e80cc8d679f8855935b9d4f2044b5380b713cd6946b0d10a57f72b42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-j, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.1, org.label-schema.license=GPLv2) 2024-08-31T06:03:44.568 INFO:journalctl@ceph.mds.j.smithi016.stdout:Aug 31 06:03:44 smithi016 podman[60790]: 2024-08-31 06:03:44.566323426 +0000 UTC m=+0.399700042 container init 66f8fc11e80cc8d679f8855935b9d4f2044b5380b713cd6946b0d10a57f72b42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-j, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.1, ceph=True) 2024-08-31T06:03:44.759 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:03:44.819 INFO:journalctl@ceph.mds.j.smithi016.stdout:Aug 31 06:03:44 smithi016 podman[60790]: 2024-08-31 06:03:44.572750407 +0000 UTC m=+0.406127005 container start 66f8fc11e80cc8d679f8855935b9d4f2044b5380b713cd6946b0d10a57f72b42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-j, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-31T06:03:44.819 INFO:journalctl@ceph.mds.j.smithi016.stdout:Aug 31 06:03:44 smithi016 ceph-mds[60825]: -- 172.21.15.16:0/4159159911 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55da0226d860 con 0x55da06413a80 2024-08-31T06:03:44.819 INFO:journalctl@ceph.mds.j.smithi016.stdout:Aug 31 06:03:44 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-j[60821]: starting mds.j at 2024-08-31T06:03:44.819 INFO:journalctl@ceph.mds.j.smithi016.stdout:Aug 31 06:03:44 smithi016 bash[60790]: 66f8fc11e80cc8d679f8855935b9d4f2044b5380b713cd6946b0d10a57f72b42 2024-08-31T06:03:44.819 INFO:journalctl@ceph.mds.j.smithi016.stdout:Aug 31 06:03:44 smithi016 systemd[1]: Started Ceph mds.j for 673e4aa0-675d-11ef-bcd4-c7b262605968. 2024-08-31T06:03:44.820 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:44 smithi016 ceph-mon[17194]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:44.820 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:44 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:44.820 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:44 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:44.820 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:44 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:44.820 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:44 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:44.820 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:44 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:44.820 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:44 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T06:03:44.820 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:44 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:03:45.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:44 smithi155 ceph-mon[21742]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:45.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:44 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:45.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:44 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:45.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:44 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:45.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:44 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:45.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:44 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:45.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:44 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T06:03:45.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:44 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:03:45.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:44 smithi184 ceph-mon[22331]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:45.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:44 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:45.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:44 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:45.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:44 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:45.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:44 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:45.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:44 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:45.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:44 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T06:03:45.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:44 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:03:45.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.410+0000 7f43ed600640 1 -- 172.21.15.16:0/3270786260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43e810fe70 msgr2=0x7f43e81122b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:45.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.410+0000 7f43ed600640 1 --2- 172.21.15.16:0/3270786260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43e810fe70 0x7f43e81122b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f43e0009ca0 tx=0x7f43e002a8c0 comp rx=0 tx=0).stop 2024-08-31T06:03:45.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.410+0000 7f43ed600640 1 -- 172.21.15.16:0/3270786260 shutdown_connections 2024-08-31T06:03:45.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.410+0000 7f43ed600640 1 --2- 172.21.15.16:0/3270786260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43e810fe70 0x7f43e81122b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:45.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.410+0000 7f43ed600640 1 --2- 172.21.15.16:0/3270786260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43e8075ea0 0x7f43e810f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:45.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.410+0000 7f43ed600640 1 --2- 172.21.15.16:0/3270786260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43e8075550 0x7f43e8075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:45.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.410+0000 7f43ed600640 1 -- 172.21.15.16:0/3270786260 >> 172.21.15.16:0/3270786260 conn(0x7f43e80ff380 msgr2=0x7f43e81017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:03:45.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.410+0000 7f43ed600640 1 -- 172.21.15.16:0/3270786260 shutdown_connections 2024-08-31T06:03:45.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.410+0000 7f43ed600640 1 -- 172.21.15.16:0/3270786260 wait complete. 2024-08-31T06:03:45.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.411+0000 7f43ed600640 1 Processor -- start 2024-08-31T06:03:45.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.411+0000 7f43ed600640 1 -- start start 2024-08-31T06:03:45.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.411+0000 7f43ed600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43e8075550 0x7f43e819c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:45.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.412+0000 7f43ed600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43e8075ea0 0x7f43e819c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:45.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.412+0000 7f43ed600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43e810fe70 0x7f43e81a1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:45.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.412+0000 7f43ed600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f43e8115a30 con 0x7f43e8075ea0 2024-08-31T06:03:45.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.412+0000 7f43ed600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f43e81158b0 con 0x7f43e810fe70 2024-08-31T06:03:45.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.412+0000 7f43ed600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f43e8115bb0 con 0x7f43e8075550 2024-08-31T06:03:45.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.412+0000 7f43e6000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43e8075ea0 0x7f43e819c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:45.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.412+0000 7f43e6a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43e8075550 0x7f43e819c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:45.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.412+0000 7f43e6a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43e8075550 0x7f43e819c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:39384/0 (socket says 172.21.15.16:39384) 2024-08-31T06:03:45.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.412+0000 7f43e6a00640 1 -- 172.21.15.16:0/2710842705 learned_addr learned my addr 172.21.15.16:0/2710842705 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:03:45.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.412+0000 7f43e7400640 1 --2- 172.21.15.16:0/2710842705 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43e810fe70 0x7f43e81a1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:45.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.412+0000 7f43e6000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43e8075ea0 0x7f43e819c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33858/0 (socket says 172.21.15.16:33858) 2024-08-31T06:03:45.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.412+0000 7f43e6a00640 1 -- 172.21.15.16:0/2710842705 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43e810fe70 msgr2=0x7f43e81a1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:45.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.412+0000 7f43e6a00640 1 --2- 172.21.15.16:0/2710842705 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43e810fe70 0x7f43e81a1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:45.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.412+0000 7f43e6a00640 1 -- 172.21.15.16:0/2710842705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43e8075ea0 msgr2=0x7f43e819c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:45.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.412+0000 7f43e6a00640 1 --2- 172.21.15.16:0/2710842705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43e8075ea0 0x7f43e819c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:45.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.412+0000 7f43e6a00640 1 -- 172.21.15.16:0/2710842705 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43e81a22f0 con 0x7f43e8075550 2024-08-31T06:03:45.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.413+0000 7f43e6a00640 1 --2- 172.21.15.16:0/2710842705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43e8075550 0x7f43e819c380 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f43d800e9f0 tx=0x7f43d800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:03:45.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.413+0000 7f43df400640 1 -- 172.21.15.16:0/2710842705 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d800cd80 con 0x7f43e8075550 2024-08-31T06:03:45.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.413+0000 7f43ed600640 1 -- 172.21.15.16:0/2710842705 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43e81a25e0 con 0x7f43e8075550 2024-08-31T06:03:45.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.413+0000 7f43df400640 1 -- 172.21.15.16:0/2710842705 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43d80108e0 con 0x7f43e8075550 2024-08-31T06:03:45.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.413+0000 7f43df400640 1 -- 172.21.15.16:0/2710842705 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d8018980 con 0x7f43e8075550 2024-08-31T06:03:45.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.413+0000 7f43ed600640 1 -- 172.21.15.16:0/2710842705 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43e81a8d40 con 0x7f43e8075550 2024-08-31T06:03:45.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.414+0000 7f43ed600640 1 -- 172.21.15.16:0/2710842705 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43e8075960 con 0x7f43e8075550 2024-08-31T06:03:45.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.417+0000 7f43df400640 1 -- 172.21.15.16:0/2710842705 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7f43d8010430 con 0x7f43e8075550 2024-08-31T06:03:45.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.418+0000 7f43df400640 1 --2- 172.21.15.16:0/2710842705 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f43cc0b4b80 0x7f43cc0b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:45.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.419+0000 7f43df400640 1 -- 172.21.15.16:0/2710842705 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7f43d80e5660 con 0x7f43e8075550 2024-08-31T06:03:45.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.419+0000 7f43e6000640 1 --2- 172.21.15.16:0/2710842705 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f43cc0b4b80 0x7f43cc0b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:45.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.419+0000 7f43e6000640 1 --2- 172.21.15.16:0/2710842705 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f43cc0b4b80 0x7f43cc0b7090 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f43e8100420 tx=0x7f43d4005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:03:45.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.420+0000 7f43df400640 1 -- 172.21.15.16:0/2710842705 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f43d80ac470 con 0x7f43e8075550 2024-08-31T06:03:45.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.662+0000 7f43ed600640 1 -- 172.21.15.16:0/2710842705 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43e819d4b0 con 0x7f43e8075550 2024-08-31T06:03:45.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.663+0000 7f43df400640 1 -- 172.21.15.16:0/2710842705 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3091 (secure 0 0 0) 0x7f43d80b0c30 con 0x7f43e8075550 2024-08-31T06:03:45.665 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:03:45.665 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":4,"btime":"2024-08-31T06:03:44:809844+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-08-31T06:03:45.666 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 4 2024-08-31T06:03:45.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.665+0000 7f43ed600640 1 -- 172.21.15.16:0/2710842705 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f43cc0b4b80 msgr2=0x7f43cc0b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:45.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.665+0000 7f43ed600640 1 --2- 172.21.15.16:0/2710842705 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f43cc0b4b80 0x7f43cc0b7090 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f43e8100420 tx=0x7f43d4005f90 comp rx=0 tx=0).stop 2024-08-31T06:03:45.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.665+0000 7f43ed600640 1 -- 172.21.15.16:0/2710842705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43e8075550 msgr2=0x7f43e819c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:45.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.665+0000 7f43ed600640 1 --2- 172.21.15.16:0/2710842705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43e8075550 0x7f43e819c380 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f43d800e9f0 tx=0x7f43d800eeb0 comp rx=0 tx=0).stop 2024-08-31T06:03:45.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.665+0000 7f43ed600640 1 -- 172.21.15.16:0/2710842705 shutdown_connections 2024-08-31T06:03:45.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.665+0000 7f43ed600640 1 --2- 172.21.15.16:0/2710842705 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f43cc0b4b80 0x7f43cc0b7090 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:45.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.666+0000 7f43ed600640 1 --2- 172.21.15.16:0/2710842705 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43e810fe70 0x7f43e81a1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:45.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.666+0000 7f43ed600640 1 --2- 172.21.15.16:0/2710842705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43e8075ea0 0x7f43e819c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:45.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.666+0000 7f43ed600640 1 --2- 172.21.15.16:0/2710842705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43e8075550 0x7f43e819c380 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:45.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.666+0000 7f43ed600640 1 -- 172.21.15.16:0/2710842705 >> 172.21.15.16:0/2710842705 conn(0x7f43e80ff380 msgr2=0x7f43e8107930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:03:45.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.666+0000 7f43ed600640 1 -- 172.21.15.16:0/2710842705 shutdown_connections 2024-08-31T06:03:45.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:45.666+0000 7f43ed600640 1 -- 172.21.15.16:0/2710842705 wait complete. 2024-08-31T06:03:46.028 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:45 smithi016 ceph-mon[17194]: Deploying daemon mds.d on smithi016 2024-08-31T06:03:46.028 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:45 smithi016 ceph-mon[17194]: mds.? [v2:172.21.15.16:6834/3365059950,v1:172.21.15.16:6835/3365059950] up:boot 2024-08-31T06:03:46.028 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:45 smithi016 ceph-mon[17194]: fsmap 3 up:standby 2024-08-31T06:03:46.028 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:45 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-08-31T06:03:46.028 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:45 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/2710842705' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:03:46.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:45 smithi155 ceph-mon[21742]: Deploying daemon mds.d on smithi016 2024-08-31T06:03:46.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:45 smithi155 ceph-mon[21742]: mds.? [v2:172.21.15.16:6834/3365059950,v1:172.21.15.16:6835/3365059950] up:boot 2024-08-31T06:03:46.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:45 smithi155 ceph-mon[21742]: fsmap 3 up:standby 2024-08-31T06:03:46.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:45 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-08-31T06:03:46.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:45 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/2710842705' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:03:46.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:45 smithi184 ceph-mon[22331]: Deploying daemon mds.d on smithi016 2024-08-31T06:03:46.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:45 smithi184 ceph-mon[22331]: mds.? [v2:172.21.15.16:6834/3365059950,v1:172.21.15.16:6835/3365059950] up:boot 2024-08-31T06:03:46.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:45 smithi184 ceph-mon[22331]: fsmap 3 up:standby 2024-08-31T06:03:46.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:45 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-08-31T06:03:46.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:45 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/2710842705' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:03:47.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:46 smithi155 ceph-mon[21742]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:47.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:46 smithi016 ceph-mon[17194]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:47.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:46 smithi184 ceph-mon[22331]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:48.559 INFO:journalctl@ceph.mds.d.smithi016.stdout:Aug 31 06:03:48 smithi016 systemd[1]: Starting Ceph mds.d for 673e4aa0-675d-11ef-bcd4-c7b262605968... 2024-08-31T06:03:48.566 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:03:48.817 INFO:journalctl@ceph.mds.d.smithi016.stdout:Aug 31 06:03:48 smithi016 podman[61175]: 2024-08-31 06:03:48.461673069 +0000 UTC m=+0.027602253 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T06:03:48.817 INFO:journalctl@ceph.mds.d.smithi016.stdout:Aug 31 06:03:48 smithi016 podman[61175]: 2024-08-31 06:03:48.570947162 +0000 UTC m=+0.136876335 container create 08bfaf11110193eb744bb503c40d510ca1ee1967991958fa6e775e1ba947aef1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-d, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.1, org.label-schema.schema-version=1.0) 2024-08-31T06:03:48.818 INFO:journalctl@ceph.mds.d.smithi016.stdout:Aug 31 06:03:48 smithi016 podman[61175]: 2024-08-31 06:03:48.815732444 +0000 UTC m=+0.381661617 container init 08bfaf11110193eb744bb503c40d510ca1ee1967991958fa6e775e1ba947aef1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-d, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.1, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-08-31T06:03:49.069 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:03:49.121 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:48 smithi155 ceph-mon[21742]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:49.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:48 smithi016 ceph-mon[17194]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:49.163 INFO:journalctl@ceph.mds.d.smithi016.stdout:Aug 31 06:03:48 smithi016 podman[61175]: 2024-08-31 06:03:48.822667993 +0000 UTC m=+0.388597166 container start 08bfaf11110193eb744bb503c40d510ca1ee1967991958fa6e775e1ba947aef1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.1, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-08-31T06:03:49.163 INFO:journalctl@ceph.mds.d.smithi016.stdout:Aug 31 06:03:48 smithi016 ceph-mds[61220]: -- 172.21.15.16:0/3011266543 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5580a5d33c20 con 0x5580a9ed9a80 2024-08-31T06:03:49.163 INFO:journalctl@ceph.mds.d.smithi016.stdout:Aug 31 06:03:48 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-d[61209]: starting mds.d at 2024-08-31T06:03:49.163 INFO:journalctl@ceph.mds.d.smithi016.stdout:Aug 31 06:03:48 smithi016 bash[61175]: 08bfaf11110193eb744bb503c40d510ca1ee1967991958fa6e775e1ba947aef1 2024-08-31T06:03:49.163 INFO:journalctl@ceph.mds.d.smithi016.stdout:Aug 31 06:03:48 smithi016 systemd[1]: Started Ceph mds.d for 673e4aa0-675d-11ef-bcd4-c7b262605968. 2024-08-31T06:03:49.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:48 smithi184 ceph-mon[22331]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:49.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.726+0000 7f8886800640 1 -- 172.21.15.16:0/3419666370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88801099c0 msgr2=0x7f88801101f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:49.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.726+0000 7f8886800640 1 --2- 172.21.15.16:0/3419666370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88801099c0 0x7f88801101f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8878009ca0 tx=0x7f887802a8c0 comp rx=0 tx=0).stop 2024-08-31T06:03:49.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.726+0000 7f8886800640 1 -- 172.21.15.16:0/3419666370 shutdown_connections 2024-08-31T06:03:49.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.726+0000 7f8886800640 1 --2- 172.21.15.16:0/3419666370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88801099c0 0x7f88801101f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:49.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.726+0000 7f8886800640 1 --2- 172.21.15.16:0/3419666370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8880109070 0x7f8880109480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:49.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.726+0000 7f8886800640 1 --2- 172.21.15.16:0/3419666370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8880103730 0x7f8880103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:49.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.726+0000 7f8886800640 1 -- 172.21.15.16:0/3419666370 >> 172.21.15.16:0/3419666370 conn(0x7f88800ff180 msgr2=0x7f88801015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:03:49.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.726+0000 7f8886800640 1 -- 172.21.15.16:0/3419666370 shutdown_connections 2024-08-31T06:03:49.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.726+0000 7f8886800640 1 -- 172.21.15.16:0/3419666370 wait complete. 2024-08-31T06:03:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.727+0000 7f8886800640 1 Processor -- start 2024-08-31T06:03:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.728+0000 7f8886800640 1 -- start start 2024-08-31T06:03:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.728+0000 7f8886800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8880103730 0x7f888019c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.728+0000 7f8886800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8880109070 0x7f888019c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.728+0000 7f8886800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88801099c0 0x7f88801a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.728+0000 7f8886800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8880113670 con 0x7f8880109070 2024-08-31T06:03:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.728+0000 7f8886800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f88801134f0 con 0x7f88801099c0 2024-08-31T06:03:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.728+0000 7f8886800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f88801137f0 con 0x7f8880103730 2024-08-31T06:03:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.728+0000 7f8884a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8880109070 0x7f888019c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.728+0000 7f8885e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88801099c0 0x7f88801a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.728+0000 7f8885e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88801099c0 0x7f88801a1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:36328/0 (socket says 172.21.15.16:36328) 2024-08-31T06:03:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.728+0000 7f8885e00640 1 -- 172.21.15.16:0/2129683509 learned_addr learned my addr 172.21.15.16:0/2129683509 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:03:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.729+0000 7f8885400640 1 --2- 172.21.15.16:0/2129683509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8880103730 0x7f888019c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.729+0000 7f8885e00640 1 -- 172.21.15.16:0/2129683509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8880103730 msgr2=0x7f888019c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.729+0000 7f8885e00640 1 --2- 172.21.15.16:0/2129683509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8880103730 0x7f888019c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.729+0000 7f8885e00640 1 -- 172.21.15.16:0/2129683509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8880109070 msgr2=0x7f888019c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.729+0000 7f8885e00640 1 --2- 172.21.15.16:0/2129683509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8880109070 0x7f888019c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.729+0000 7f8885e00640 1 -- 172.21.15.16:0/2129683509 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88801a2170 con 0x7f88801099c0 2024-08-31T06:03:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.729+0000 7f8885400640 1 --2- 172.21.15.16:0/2129683509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8880103730 0x7f888019c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:03:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.729+0000 7f8885e00640 1 --2- 172.21.15.16:0/2129683509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88801099c0 0x7f88801a1a40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8878005cc0 tx=0x7f8878031700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:03:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.729+0000 7f887e000640 1 -- 172.21.15.16:0/2129683509 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8878003b70 con 0x7f88801099c0 2024-08-31T06:03:49.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.729+0000 7f887e000640 1 -- 172.21.15.16:0/2129683509 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88780318f0 con 0x7f88801099c0 2024-08-31T06:03:49.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.729+0000 7f8886800640 1 -- 172.21.15.16:0/2129683509 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88801a2400 con 0x7f88801099c0 2024-08-31T06:03:49.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.729+0000 7f887e000640 1 -- 172.21.15.16:0/2129683509 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887802f950 con 0x7f88801099c0 2024-08-31T06:03:49.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.730+0000 7f8886800640 1 -- 172.21.15.16:0/2129683509 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88801a8bc0 con 0x7f88801099c0 2024-08-31T06:03:49.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.731+0000 7f8886800640 1 -- 172.21.15.16:0/2129683509 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8880103b40 con 0x7f88801099c0 2024-08-31T06:03:49.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.733+0000 7f887e000640 1 -- 172.21.15.16:0/2129683509 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7f8878004b40 con 0x7f88801099c0 2024-08-31T06:03:49.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.734+0000 7f887e000640 1 --2- 172.21.15.16:0/2129683509 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f88680b4b80 0x7f88680b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:49.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.735+0000 7f887e000640 1 -- 172.21.15.16:0/2129683509 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7f8878106190 con 0x7f88801099c0 2024-08-31T06:03:49.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.735+0000 7f8885400640 1 --2- 172.21.15.16:0/2129683509 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f88680b4b80 0x7f88680b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:49.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.735+0000 7f8885400640 1 --2- 172.21.15.16:0/2129683509 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f88680b4b80 0x7f88680b7090 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8874009c20 tx=0x7f8874005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:03:49.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.736+0000 7f887e000640 1 -- 172.21.15.16:0/2129683509 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f88780cd5f0 con 0x7f88801099c0 2024-08-31T06:03:49.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.971+0000 7f8886800640 1 -- 172.21.15.16:0/2129683509 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88801a8e20 con 0x7f88801099c0 2024-08-31T06:03:49.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.972+0000 7f887e000640 1 -- 172.21.15.16:0/2129683509 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3913 (secure 0 0 0) 0x7f88780ccef0 con 0x7f88801099c0 2024-08-31T06:03:49.973 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:03:49.973 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":5,"btime":"2024-08-31T06:03:49:828032+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[]} 2024-08-31T06:03:49.974 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 5 2024-08-31T06:03:49.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.974+0000 7f8886800640 1 -- 172.21.15.16:0/2129683509 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f88680b4b80 msgr2=0x7f88680b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:49.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.974+0000 7f8886800640 1 --2- 172.21.15.16:0/2129683509 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f88680b4b80 0x7f88680b7090 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8874009c20 tx=0x7f8874005f90 comp rx=0 tx=0).stop 2024-08-31T06:03:49.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.974+0000 7f8886800640 1 -- 172.21.15.16:0/2129683509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88801099c0 msgr2=0x7f88801a1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:49.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.974+0000 7f8886800640 1 --2- 172.21.15.16:0/2129683509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88801099c0 0x7f88801a1a40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8878005cc0 tx=0x7f8878031700 comp rx=0 tx=0).stop 2024-08-31T06:03:49.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.974+0000 7f8886800640 1 -- 172.21.15.16:0/2129683509 shutdown_connections 2024-08-31T06:03:49.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.974+0000 7f8886800640 1 --2- 172.21.15.16:0/2129683509 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f88680b4b80 0x7f88680b7090 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:49.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.974+0000 7f8886800640 1 --2- 172.21.15.16:0/2129683509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88801099c0 0x7f88801a1a40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:49.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.974+0000 7f8886800640 1 --2- 172.21.15.16:0/2129683509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8880109070 0x7f888019c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:49.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.974+0000 7f8886800640 1 --2- 172.21.15.16:0/2129683509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8880103730 0x7f888019c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:49.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.974+0000 7f8886800640 1 -- 172.21.15.16:0/2129683509 >> 172.21.15.16:0/2129683509 conn(0x7f88800ff180 msgr2=0x7f888010e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:03:49.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.974+0000 7f8886800640 1 -- 172.21.15.16:0/2129683509 shutdown_connections 2024-08-31T06:03:49.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:49.974+0000 7f8886800640 1 -- 172.21.15.16:0/2129683509 wait complete. 2024-08-31T06:03:50.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:50 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:50.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:50 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:50.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:50 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:50.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:50 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:50.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:50 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:50.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:50 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T06:03:50.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:50 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:03:50.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:50 smithi155 ceph-mon[21742]: Deploying daemon mds.e on smithi155 2024-08-31T06:03:50.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:50 smithi155 ceph-mon[21742]: mds.? [v2:172.21.15.16:6836/368986894,v1:172.21.15.16:6837/368986894] up:boot 2024-08-31T06:03:50.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:50 smithi155 ceph-mon[21742]: fsmap 4 up:standby 2024-08-31T06:03:50.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:50 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-08-31T06:03:50.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:50 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/2129683509' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:03:50.326 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:50 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:50.326 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:50 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:50.326 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:50 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:50.327 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:50 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:50.327 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:50 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:50.327 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:50 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T06:03:50.327 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:50 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:03:50.327 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:50 smithi016 ceph-mon[17194]: Deploying daemon mds.e on smithi155 2024-08-31T06:03:50.327 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:50 smithi016 ceph-mon[17194]: mds.? [v2:172.21.15.16:6836/368986894,v1:172.21.15.16:6837/368986894] up:boot 2024-08-31T06:03:50.327 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:50 smithi016 ceph-mon[17194]: fsmap 4 up:standby 2024-08-31T06:03:50.327 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:50 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-08-31T06:03:50.327 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:50 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/2129683509' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:03:50.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:50 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:50.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:50 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:50.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:50 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:50.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:50 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:50.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:50 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:50.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:50 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T06:03:50.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:50 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:03:50.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:50 smithi184 ceph-mon[22331]: Deploying daemon mds.e on smithi155 2024-08-31T06:03:50.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:50 smithi184 ceph-mon[22331]: mds.? [v2:172.21.15.16:6836/368986894,v1:172.21.15.16:6837/368986894] up:boot 2024-08-31T06:03:50.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:50 smithi184 ceph-mon[22331]: fsmap 4 up:standby 2024-08-31T06:03:50.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:50 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-08-31T06:03:50.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:50 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/2129683509' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:03:51.324 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:51 smithi155 ceph-mon[21742]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:51.324 INFO:journalctl@ceph.mds.e.smithi155.stdout:Aug 31 06:03:51 smithi155 systemd[1]: Starting Ceph mds.e for 673e4aa0-675d-11ef-bcd4-c7b262605968... 2024-08-31T06:03:51.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:51 smithi016 ceph-mon[17194]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:51.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:51 smithi184 ceph-mon[22331]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:51.616 INFO:journalctl@ceph.mds.e.smithi155.stdout:Aug 31 06:03:51 smithi155 podman[52720]: 2024-08-31 06:03:51.224484714 +0000 UTC m=+0.015910152 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T06:03:51.616 INFO:journalctl@ceph.mds.e.smithi155.stdout:Aug 31 06:03:51 smithi155 podman[52720]: 2024-08-31 06:03:51.354392124 +0000 UTC m=+0.145817553 container create d7b37624caef48e6ec57b8a2c0ed5fdff2df36d6de8fe01c39e7d1a4a045e8dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.1, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-31T06:03:51.616 INFO:journalctl@ceph.mds.e.smithi155.stdout:Aug 31 06:03:51 smithi155 podman[52720]: 2024-08-31 06:03:51.614840268 +0000 UTC m=+0.406265696 container init d7b37624caef48e6ec57b8a2c0ed5fdff2df36d6de8fe01c39e7d1a4a045e8dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-e, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True, io.buildah.version=1.37.1, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-08-31T06:03:51.872 INFO:journalctl@ceph.mds.e.smithi155.stdout:Aug 31 06:03:51 smithi155 podman[52720]: 2024-08-31 06:03:51.617560623 +0000 UTC m=+0.408986064 container start d7b37624caef48e6ec57b8a2c0ed5fdff2df36d6de8fe01c39e7d1a4a045e8dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-e, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.1, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611) 2024-08-31T06:03:51.872 INFO:journalctl@ceph.mds.e.smithi155.stdout:Aug 31 06:03:51 smithi155 ceph-mds[52740]: -- 172.21.15.155:0/4125369136 <== mon.2 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563bee461860 con 0x563bf2607a80 2024-08-31T06:03:51.872 INFO:journalctl@ceph.mds.e.smithi155.stdout:Aug 31 06:03:51 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-e[52736]: starting mds.e at 2024-08-31T06:03:51.873 INFO:journalctl@ceph.mds.e.smithi155.stdout:Aug 31 06:03:51 smithi155 bash[52720]: d7b37624caef48e6ec57b8a2c0ed5fdff2df36d6de8fe01c39e7d1a4a045e8dd 2024-08-31T06:03:51.873 INFO:journalctl@ceph.mds.e.smithi155.stdout:Aug 31 06:03:51 smithi155 systemd[1]: Started Ceph mds.e for 673e4aa0-675d-11ef-bcd4-c7b262605968. 2024-08-31T06:03:52.776 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:03:53.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:52 smithi155 ceph-mon[21742]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:53.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:52 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:53.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:52 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:53.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:52 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:53.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:52 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:53.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:52 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:53.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:52 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T06:03:53.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:52 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:03:53.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:52 smithi155 ceph-mon[21742]: Deploying daemon mds.f on smithi184 2024-08-31T06:03:53.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:52 smithi155 ceph-mon[21742]: mds.? [v2:172.21.15.155:6834/1634403570,v1:172.21.15.155:6835/1634403570] up:boot 2024-08-31T06:03:53.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:52 smithi155 ceph-mon[21742]: fsmap 5 up:standby 2024-08-31T06:03:53.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:52 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-08-31T06:03:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:52 smithi016 ceph-mon[17194]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:52 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:52 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:52 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:52 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:52 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:52 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T06:03:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:52 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:03:53.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:52 smithi016 ceph-mon[17194]: Deploying daemon mds.f on smithi184 2024-08-31T06:03:53.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:52 smithi016 ceph-mon[17194]: mds.? [v2:172.21.15.155:6834/1634403570,v1:172.21.15.155:6835/1634403570] up:boot 2024-08-31T06:03:53.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:52 smithi016 ceph-mon[17194]: fsmap 5 up:standby 2024-08-31T06:03:53.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:52 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-08-31T06:03:53.164 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:03:53.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:52 smithi184 ceph-mon[22331]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:53.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:52 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:53.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:52 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:53.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:52 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:53.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:52 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:53.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:52 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:53.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:52 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T06:03:53.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:52 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:03:53.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:52 smithi184 ceph-mon[22331]: Deploying daemon mds.f on smithi184 2024-08-31T06:03:53.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:52 smithi184 ceph-mon[22331]: mds.? [v2:172.21.15.155:6834/1634403570,v1:172.21.15.155:6835/1634403570] up:boot 2024-08-31T06:03:53.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:52 smithi184 ceph-mon[22331]: fsmap 5 up:standby 2024-08-31T06:03:53.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:52 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-08-31T06:03:53.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:53.790+0000 7f9ddb600640 1 -- 172.21.15.16:0/456178252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dd410fe60 msgr2=0x7f9dd41122a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:53.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:53.790+0000 7f9ddb600640 1 --2- 172.21.15.16:0/456178252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dd410fe60 0x7f9dd41122a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc009ca0 tx=0x7f9dcc02a8c0 comp rx=0 tx=0).stop 2024-08-31T06:03:53.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:53.790+0000 7f9ddb600640 1 -- 172.21.15.16:0/456178252 shutdown_connections 2024-08-31T06:03:53.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:53.790+0000 7f9ddb600640 1 --2- 172.21.15.16:0/456178252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9dd4075ea0 0x7f9dd410f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:53.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:53.790+0000 7f9ddb600640 1 --2- 172.21.15.16:0/456178252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dd410fe60 0x7f9dd41122a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:53.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:53.790+0000 7f9ddb600640 1 --2- 172.21.15.16:0/456178252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9dd4075550 0x7f9dd4075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:53.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:53.790+0000 7f9ddb600640 1 -- 172.21.15.16:0/456178252 >> 172.21.15.16:0/456178252 conn(0x7f9dd407b5f0 msgr2=0x7f9dd407ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:03:53.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:53.790+0000 7f9ddb600640 1 -- 172.21.15.16:0/456178252 shutdown_connections 2024-08-31T06:03:53.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:53.790+0000 7f9ddb600640 1 -- 172.21.15.16:0/456178252 wait complete. 2024-08-31T06:03:53.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:53.791+0000 7f9ddb600640 1 Processor -- start 2024-08-31T06:03:53.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:53.791+0000 7f9ddb600640 1 -- start start 2024-08-31T06:03:53.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:53.792+0000 7f9ddb600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dd4075550 0x7f9dd419c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:53.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:53.792+0000 7f9ddb600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9dd4075ea0 0x7f9dd419c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:53.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:53.792+0000 7f9ddb600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9dd410fe60 0x7f9dd41a1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:53.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:53.792+0000 7f9ddb600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd4115a10 con 0x7f9dd4075550 2024-08-31T06:03:53.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:53.792+0000 7f9ddb600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd4115890 con 0x7f9dd4075ea0 2024-08-31T06:03:53.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:53.792+0000 7f9ddb600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd4115b90 con 0x7f9dd410fe60 2024-08-31T06:03:53.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:53.792+0000 7f9dd8c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dd4075550 0x7f9dd419c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:53.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:53.792+0000 7f9dd9600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9dd410fe60 0x7f9dd41a1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:53.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:53.792+0000 7f9dcbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9dd4075ea0 0x7f9dd419c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:53.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:53.792+0000 7f9dd9600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9dd410fe60 0x7f9dd41a1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:36704/0 (socket says 172.21.15.16:36704) 2024-08-31T06:03:53.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:53.792+0000 7f9dd8c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dd4075550 0x7f9dd419c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46414/0 (socket says 172.21.15.16:46414) 2024-08-31T06:03:53.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:53.792+0000 7f9dd9600640 1 -- 172.21.15.16:0/2601599740 learned_addr learned my addr 172.21.15.16:0/2601599740 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:03:53.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:53.793+0000 7f9dd9600640 1 -- 172.21.15.16:0/2601599740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9dd4075ea0 msgr2=0x7f9dd419c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:53.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:53.793+0000 7f9dd9600640 1 --2- 172.21.15.16:0/2601599740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9dd4075ea0 0x7f9dd419c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:53.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:53.793+0000 7f9dd9600640 1 -- 172.21.15.16:0/2601599740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dd4075550 msgr2=0x7f9dd419c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:53.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:53.793+0000 7f9dd9600640 1 --2- 172.21.15.16:0/2601599740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dd4075550 0x7f9dd419c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:53.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:53.793+0000 7f9dd9600640 1 -- 172.21.15.16:0/2601599740 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd41a2350 con 0x7f9dd410fe60 2024-08-31T06:03:53.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:53.793+0000 7f9dd8c00640 1 --2- 172.21.15.16:0/2601599740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dd4075550 0x7f9dd419c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:03:53.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:53.793+0000 7f9dd9600640 1 --2- 172.21.15.16:0/2601599740 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9dd410fe60 0x7f9dd41a1c20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc0098d0 tx=0x7f9dcc0317b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:03:53.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:53.793+0000 7f9dd2000640 1 -- 172.21.15.16:0/2601599740 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc003b70 con 0x7f9dd410fe60 2024-08-31T06:03:53.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:53.793+0000 7f9dd2000640 1 -- 172.21.15.16:0/2601599740 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dcc003d10 con 0x7f9dd410fe60 2024-08-31T06:03:53.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:53.793+0000 7f9dd2000640 1 -- 172.21.15.16:0/2601599740 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc02fa20 con 0x7f9dd410fe60 2024-08-31T06:03:53.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:53.793+0000 7f9ddb600640 1 -- 172.21.15.16:0/2601599740 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dd41a25e0 con 0x7f9dd410fe60 2024-08-31T06:03:53.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:53.794+0000 7f9ddb600640 1 -- 172.21.15.16:0/2601599740 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dd41a8da0 con 0x7f9dd410fe60 2024-08-31T06:03:53.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:53.795+0000 7f9ddb600640 1 -- 172.21.15.16:0/2601599740 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dd4076a60 con 0x7f9dd410fe60 2024-08-31T06:03:53.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:53.797+0000 7f9dd2000640 1 -- 172.21.15.16:0/2601599740 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7f9dcc02fc20 con 0x7f9dd410fe60 2024-08-31T06:03:53.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:53.798+0000 7f9dd2000640 1 --2- 172.21.15.16:0/2601599740 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9db80b4c00 0x7f9db80b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:53.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:53.798+0000 7f9dd2000640 1 -- 172.21.15.16:0/2601599740 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7f9dcc1061f0 con 0x7f9dd410fe60 2024-08-31T06:03:53.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:53.798+0000 7f9dd8c00640 1 --2- 172.21.15.16:0/2601599740 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9db80b4c00 0x7f9db80b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:53.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:53.799+0000 7f9dd8c00640 1 --2- 172.21.15.16:0/2601599740 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9db80b4c00 0x7f9db80b7110 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4005d00 tx=0x7f9dc400b410 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:03:53.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:53.802+0000 7f9dd2000640 1 -- 172.21.15.16:0/2601599740 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f9dcc0cd650 con 0x7f9dd410fe60 2024-08-31T06:03:54.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:54.039+0000 7f9ddb600640 1 -- 172.21.15.16:0/2601599740 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9dd41a8fa0 con 0x7f9dd410fe60 2024-08-31T06:03:54.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:54.040+0000 7f9dd2000640 1 -- 172.21.15.16:0/2601599740 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4741 (secure 0 0 0) 0x7f9dcc0ccf50 con 0x7f9dd410fe60 2024-08-31T06:03:54.042 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:03:54.042 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":6,"btime":"2024-08-31T06:03:52:037726+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-08-31T06:03:54.042 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 6 2024-08-31T06:03:54.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:54.042+0000 7f9ddb600640 1 -- 172.21.15.16:0/2601599740 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9db80b4c00 msgr2=0x7f9db80b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:54.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:54.042+0000 7f9ddb600640 1 --2- 172.21.15.16:0/2601599740 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9db80b4c00 0x7f9db80b7110 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4005d00 tx=0x7f9dc400b410 comp rx=0 tx=0).stop 2024-08-31T06:03:54.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:54.042+0000 7f9ddb600640 1 -- 172.21.15.16:0/2601599740 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9dd410fe60 msgr2=0x7f9dd41a1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:54.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:54.042+0000 7f9ddb600640 1 --2- 172.21.15.16:0/2601599740 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9dd410fe60 0x7f9dd41a1c20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc0098d0 tx=0x7f9dcc0317b0 comp rx=0 tx=0).stop 2024-08-31T06:03:54.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:54.042+0000 7f9ddb600640 1 -- 172.21.15.16:0/2601599740 shutdown_connections 2024-08-31T06:03:54.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:54.042+0000 7f9ddb600640 1 --2- 172.21.15.16:0/2601599740 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9db80b4c00 0x7f9db80b7110 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:54.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:54.042+0000 7f9ddb600640 1 --2- 172.21.15.16:0/2601599740 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9dd410fe60 0x7f9dd41a1c20 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:54.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:54.042+0000 7f9ddb600640 1 --2- 172.21.15.16:0/2601599740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9dd4075ea0 0x7f9dd419c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:54.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:54.042+0000 7f9ddb600640 1 --2- 172.21.15.16:0/2601599740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dd4075550 0x7f9dd419c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:54.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:54.043+0000 7f9ddb600640 1 -- 172.21.15.16:0/2601599740 >> 172.21.15.16:0/2601599740 conn(0x7f9dd407b5f0 msgr2=0x7f9dd41014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:03:54.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:54.043+0000 7f9ddb600640 1 -- 172.21.15.16:0/2601599740 shutdown_connections 2024-08-31T06:03:54.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:54.043+0000 7f9ddb600640 1 -- 172.21.15.16:0/2601599740 wait complete. 2024-08-31T06:03:54.223 INFO:journalctl@ceph.mds.f.smithi184.stdout:Aug 31 06:03:54 smithi184 podman[53775]: 2024-08-31 06:03:54.099881505 +0000 UTC m=+0.114695090 container create d162f2b072e8c11a0e1251a7719a190e6eb0b06913870801d0a46fc0cc196b18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-31T06:03:54.723 INFO:journalctl@ceph.mds.f.smithi184.stdout:Aug 31 06:03:54 smithi184 podman[53775]: 2024-08-31 06:03:54.344411691 +0000 UTC m=+0.359225276 container init d162f2b072e8c11a0e1251a7719a190e6eb0b06913870801d0a46fc0cc196b18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True) 2024-08-31T06:03:54.723 INFO:journalctl@ceph.mds.f.smithi184.stdout:Aug 31 06:03:54 smithi184 podman[53775]: 2024-08-31 06:03:54.347227316 +0000 UTC m=+0.362040902 container start d162f2b072e8c11a0e1251a7719a190e6eb0b06913870801d0a46fc0cc196b18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-f, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.1, GIT_CLEAN=True) 2024-08-31T06:03:54.723 INFO:journalctl@ceph.mds.f.smithi184.stdout:Aug 31 06:03:54 smithi184 ceph-mds[53795]: -- 172.21.15.184:0/3475573390 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a8a1721860 con 0x55a8a58c7600 2024-08-31T06:03:54.723 INFO:journalctl@ceph.mds.f.smithi184.stdout:Aug 31 06:03:54 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-f[53791]: starting mds.f at 2024-08-31T06:03:54.723 INFO:journalctl@ceph.mds.f.smithi184.stdout:Aug 31 06:03:54 smithi184 bash[53775]: d162f2b072e8c11a0e1251a7719a190e6eb0b06913870801d0a46fc0cc196b18 2024-08-31T06:03:54.723 INFO:journalctl@ceph.mds.f.smithi184.stdout:Aug 31 06:03:54 smithi184 systemd[1]: Started Ceph mds.f for 673e4aa0-675d-11ef-bcd4-c7b262605968. 2024-08-31T06:03:55.073 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:54 smithi184 ceph-mon[22331]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:55.074 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:54 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/2601599740' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:03:55.074 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:54 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:55.074 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:54 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:55.074 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:54 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:55.074 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:54 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:55.074 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:54 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:55.074 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:54 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T06:03:55.074 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:54 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:03:55.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:54 smithi155 ceph-mon[21742]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:55.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:54 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/2601599740' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:03:55.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:54 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:55.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:54 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:55.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:54 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:55.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:54 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:55.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:54 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:55.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:54 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T06:03:55.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:54 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:03:55.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:54 smithi016 ceph-mon[17194]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:55.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:54 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/2601599740' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:03:55.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:54 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:55.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:54 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:55.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:54 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:55.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:54 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:55.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:54 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:55.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:54 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T06:03:55.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:54 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:03:55.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:55 smithi184 ceph-mon[22331]: Deploying daemon mds.c on smithi184 2024-08-31T06:03:55.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:55 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-08-31T06:03:55.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:55 smithi184 ceph-mon[22331]: mds.? [v2:172.21.15.184:6834/1925472309,v1:172.21.15.184:6835/1925472309] up:boot 2024-08-31T06:03:55.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:55 smithi184 ceph-mon[22331]: fsmap 6 up:standby 2024-08-31T06:03:56.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:55 smithi155 ceph-mon[21742]: Deploying daemon mds.c on smithi184 2024-08-31T06:03:56.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:55 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-08-31T06:03:56.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:55 smithi155 ceph-mon[21742]: mds.? [v2:172.21.15.184:6834/1925472309,v1:172.21.15.184:6835/1925472309] up:boot 2024-08-31T06:03:56.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:55 smithi155 ceph-mon[21742]: fsmap 6 up:standby 2024-08-31T06:03:56.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:55 smithi016 ceph-mon[17194]: Deploying daemon mds.c on smithi184 2024-08-31T06:03:56.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:55 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-08-31T06:03:56.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:55 smithi016 ceph-mon[17194]: mds.? [v2:172.21.15.184:6834/1925472309,v1:172.21.15.184:6835/1925472309] up:boot 2024-08-31T06:03:56.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:55 smithi016 ceph-mon[17194]: fsmap 6 up:standby 2024-08-31T06:03:56.722 INFO:journalctl@ceph.mds.c.smithi184.stdout:Aug 31 06:03:56 smithi184 systemd[1]: Starting Ceph mds.c for 673e4aa0-675d-11ef-bcd4-c7b262605968... 2024-08-31T06:03:56.795 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:03:57.045 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:56 smithi184 ceph-mon[22331]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:57.045 INFO:journalctl@ceph.mds.c.smithi184.stdout:Aug 31 06:03:56 smithi184 podman[54077]: 2024-08-31 06:03:56.655021814 +0000 UTC m=+0.016324617 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T06:03:57.045 INFO:journalctl@ceph.mds.c.smithi184.stdout:Aug 31 06:03:56 smithi184 podman[54077]: 2024-08-31 06:03:56.781962583 +0000 UTC m=+0.143265384 container create 8f0a89efbce5693100751c23421504dec810209bbd1429a80031c3b55d8b2570 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.1, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-08-31T06:03:57.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:56 smithi155 ceph-mon[21742]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:57.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:56 smithi016 ceph-mon[17194]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:57.184 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:03:57.472 INFO:journalctl@ceph.mds.c.smithi184.stdout:Aug 31 06:03:57 smithi184 podman[54077]: 2024-08-31 06:03:57.043329598 +0000 UTC m=+0.404632385 container init 8f0a89efbce5693100751c23421504dec810209bbd1429a80031c3b55d8b2570 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-c, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.1, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611) 2024-08-31T06:03:57.473 INFO:journalctl@ceph.mds.c.smithi184.stdout:Aug 31 06:03:57 smithi184 podman[54077]: 2024-08-31 06:03:57.046252776 +0000 UTC m=+0.407555581 container start 8f0a89efbce5693100751c23421504dec810209bbd1429a80031c3b55d8b2570 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-c, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.1) 2024-08-31T06:03:57.473 INFO:journalctl@ceph.mds.c.smithi184.stdout:Aug 31 06:03:57 smithi184 ceph-mds[54096]: -- 172.21.15.184:0/1846023300 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555b4326b860 con 0x555b47411600 2024-08-31T06:03:57.473 INFO:journalctl@ceph.mds.c.smithi184.stdout:Aug 31 06:03:57 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-c[54092]: starting mds.c at 2024-08-31T06:03:57.473 INFO:journalctl@ceph.mds.c.smithi184.stdout:Aug 31 06:03:57 smithi184 bash[54077]: 8f0a89efbce5693100751c23421504dec810209bbd1429a80031c3b55d8b2570 2024-08-31T06:03:57.473 INFO:journalctl@ceph.mds.c.smithi184.stdout:Aug 31 06:03:57 smithi184 systemd[1]: Started Ceph mds.c for 673e4aa0-675d-11ef-bcd4-c7b262605968. 2024-08-31T06:03:57.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:57.835+0000 7f651b800640 1 -- 172.21.15.16:0/2802028767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6514109b90 msgr2=0x7f65141103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:57.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:57.835+0000 7f651b800640 1 --2- 172.21.15.16:0/2802028767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6514109b90 0x7f65141103c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f650c009ca0 tx=0x7f650c02a8c0 comp rx=0 tx=0).stop 2024-08-31T06:03:57.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:57.835+0000 7f651b800640 1 -- 172.21.15.16:0/2802028767 shutdown_connections 2024-08-31T06:03:57.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:57.835+0000 7f651b800640 1 --2- 172.21.15.16:0/2802028767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6514109240 0x7f6514109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:57.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:57.835+0000 7f651b800640 1 --2- 172.21.15.16:0/2802028767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6514109b90 0x7f65141103c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:57.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:57.835+0000 7f651b800640 1 --2- 172.21.15.16:0/2802028767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6514103900 0x7f6514103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:57.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:57.835+0000 7f651b800640 1 -- 172.21.15.16:0/2802028767 >> 172.21.15.16:0/2802028767 conn(0x7f651407b5f0 msgr2=0x7f651407ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:03:57.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:57.835+0000 7f651b800640 1 -- 172.21.15.16:0/2802028767 shutdown_connections 2024-08-31T06:03:57.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:57.835+0000 7f651b800640 1 -- 172.21.15.16:0/2802028767 wait complete. 2024-08-31T06:03:57.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:57.836+0000 7f651b800640 1 Processor -- start 2024-08-31T06:03:57.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:57.836+0000 7f651b800640 1 -- start start 2024-08-31T06:03:57.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:57.837+0000 7f651b800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6514103900 0x7f651419a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:57.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:57.837+0000 7f651b800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6514109240 0x7f651419a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:57.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:57.837+0000 7f651b800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6514109b90 0x7f651419fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:57.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:57.837+0000 7f651b800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6514113850 con 0x7f6514103900 2024-08-31T06:03:57.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:57.837+0000 7f651b800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f65141136d0 con 0x7f6514109240 2024-08-31T06:03:57.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:57.837+0000 7f651b800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f65141139d0 con 0x7f6514109b90 2024-08-31T06:03:57.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:57.837+0000 7f6518e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6514103900 0x7f651419a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:57.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:57.837+0000 7f650be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6514109240 0x7f651419a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:57.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:57.837+0000 7f6518e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6514103900 0x7f651419a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48584/0 (socket says 172.21.15.16:48584) 2024-08-31T06:03:57.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:57.837+0000 7f650be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6514109240 0x7f651419a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:48288/0 (socket says 172.21.15.16:48288) 2024-08-31T06:03:57.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:57.837+0000 7f6518e00640 1 -- 172.21.15.16:0/427567285 learned_addr learned my addr 172.21.15.16:0/427567285 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:03:57.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:57.837+0000 7f6519800640 1 --2- 172.21.15.16:0/427567285 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6514109b90 0x7f651419fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:57.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:57.838+0000 7f650be00640 1 -- 172.21.15.16:0/427567285 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6514109b90 msgr2=0x7f651419fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:57.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:57.838+0000 7f650be00640 1 --2- 172.21.15.16:0/427567285 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6514109b90 0x7f651419fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:57.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:57.838+0000 7f650be00640 1 -- 172.21.15.16:0/427567285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6514103900 msgr2=0x7f651419a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:57.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:57.838+0000 7f650be00640 1 --2- 172.21.15.16:0/427567285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6514103900 0x7f651419a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:57.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:57.838+0000 7f650be00640 1 -- 172.21.15.16:0/427567285 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65141a0150 con 0x7f6514109240 2024-08-31T06:03:57.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:57.838+0000 7f6518e00640 1 --2- 172.21.15.16:0/427567285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6514103900 0x7f651419a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:03:57.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:57.838+0000 7f650be00640 1 --2- 172.21.15.16:0/427567285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6514109240 0x7f651419a780 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f651406c790 tx=0x7f64fc00ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:03:57.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:57.838+0000 7f6512000640 1 -- 172.21.15.16:0/427567285 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64fc00cc40 con 0x7f6514109240 2024-08-31T06:03:57.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:57.838+0000 7f651b800640 1 -- 172.21.15.16:0/427567285 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65141a0440 con 0x7f6514109240 2024-08-31T06:03:57.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:57.838+0000 7f651b800640 1 -- 172.21.15.16:0/427567285 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65141a6ba0 con 0x7f6514109240 2024-08-31T06:03:57.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:57.838+0000 7f6512000640 1 -- 172.21.15.16:0/427567285 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64fc00cde0 con 0x7f6514109240 2024-08-31T06:03:57.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:57.838+0000 7f6512000640 1 -- 172.21.15.16:0/427567285 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64fc018640 con 0x7f6514109240 2024-08-31T06:03:57.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:57.839+0000 7f651b800640 1 -- 172.21.15.16:0/427567285 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64f0005290 con 0x7f6514109240 2024-08-31T06:03:57.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:57.842+0000 7f6512000640 1 -- 172.21.15.16:0/427567285 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7f64fc005030 con 0x7f6514109240 2024-08-31T06:03:57.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:57.844+0000 7f6512000640 1 --2- 172.21.15.16:0/427567285 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f64f80b4b80 0x7f64f80b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:03:57.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:57.844+0000 7f6512000640 1 -- 172.21.15.16:0/427567285 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7f64fc0e51e0 con 0x7f6514109240 2024-08-31T06:03:57.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:57.844+0000 7f6518e00640 1 --2- 172.21.15.16:0/427567285 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f64f80b4b80 0x7f64f80b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:03:57.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:57.845+0000 7f6512000640 1 -- 172.21.15.16:0/427567285 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f64fc0abff0 con 0x7f6514109240 2024-08-31T06:03:57.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:57.845+0000 7f6518e00640 1 --2- 172.21.15.16:0/427567285 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f64f80b4b80 0x7f64f80b7090 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f65040095f0 tx=0x7f6504005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:03:58.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:58.077+0000 7f651b800640 1 -- 172.21.15.16:0/427567285 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f64f0005580 con 0x7f6514109240 2024-08-31T06:03:58.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:58.078+0000 7f6512000640 1 -- 172.21.15.16:0/427567285 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6397 (secure 0 0 0) 0x7f64fc0b07b0 con 0x7f6514109240 2024-08-31T06:03:58.081 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:03:58.081 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"btime":"2024-08-31T06:03:57:785279+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24688,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24694,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-08-31T06:03:58.081 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2024-08-31T06:03:58.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:58.080+0000 7f651b800640 1 -- 172.21.15.16:0/427567285 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f64f80b4b80 msgr2=0x7f64f80b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:58.080+0000 7f651b800640 1 --2- 172.21.15.16:0/427567285 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f64f80b4b80 0x7f64f80b7090 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f65040095f0 tx=0x7f6504005f90 comp rx=0 tx=0).stop 2024-08-31T06:03:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:58.080+0000 7f651b800640 1 -- 172.21.15.16:0/427567285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6514109240 msgr2=0x7f651419a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:03:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:58.080+0000 7f651b800640 1 --2- 172.21.15.16:0/427567285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6514109240 0x7f651419a780 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f651406c790 tx=0x7f64fc00ee50 comp rx=0 tx=0).stop 2024-08-31T06:03:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:58.080+0000 7f651b800640 1 -- 172.21.15.16:0/427567285 shutdown_connections 2024-08-31T06:03:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:58.080+0000 7f651b800640 1 --2- 172.21.15.16:0/427567285 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f64f80b4b80 0x7f64f80b7090 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:58.080+0000 7f651b800640 1 --2- 172.21.15.16:0/427567285 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6514109b90 0x7f651419fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:58.080+0000 7f651b800640 1 --2- 172.21.15.16:0/427567285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6514109240 0x7f651419a780 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:58.080+0000 7f651b800640 1 --2- 172.21.15.16:0/427567285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6514103900 0x7f651419a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:03:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:58.080+0000 7f651b800640 1 -- 172.21.15.16:0/427567285 >> 172.21.15.16:0/427567285 conn(0x7f651407b5f0 msgr2=0x7f6514105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:03:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:58.080+0000 7f651b800640 1 -- 172.21.15.16:0/427567285 shutdown_connections 2024-08-31T06:03:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:03:58.080+0000 7f651b800640 1 -- 172.21.15.16:0/427567285 wait complete. 2024-08-31T06:03:58.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:58 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:58.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:58 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:58.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:58 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:58.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:58 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:58.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:58 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:58.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:58 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T06:03:58.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:58 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:03:58.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:58 smithi155 ceph-mon[21742]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:58.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:58 smithi155 ceph-mon[21742]: Deploying daemon mds.h on smithi155 2024-08-31T06:03:58.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:58 smithi155 ceph-mon[21742]: mds.? [v2:172.21.15.184:6836/1712755588,v1:172.21.15.184:6837/1712755588] up:boot 2024-08-31T06:03:58.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:58 smithi155 ceph-mon[21742]: fsmap 7 up:standby 2024-08-31T06:03:58.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:58 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-08-31T06:03:58.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:03:58 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/427567285' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:03:58.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:58 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:58.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:58 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:58.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:58 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:58.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:58 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:58.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:58 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:58.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:58 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T06:03:58.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:58 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:03:58.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:58 smithi016 ceph-mon[17194]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:58.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:58 smithi016 ceph-mon[17194]: Deploying daemon mds.h on smithi155 2024-08-31T06:03:58.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:58 smithi016 ceph-mon[17194]: mds.? [v2:172.21.15.184:6836/1712755588,v1:172.21.15.184:6837/1712755588] up:boot 2024-08-31T06:03:58.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:58 smithi016 ceph-mon[17194]: fsmap 7 up:standby 2024-08-31T06:03:58.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:58 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-08-31T06:03:58.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:03:58 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/427567285' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:03:58.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:58 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:58.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:58 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:58.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:58 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:03:58.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:58 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:58.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:58 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:03:58.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:58 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T06:03:58.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:58 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:03:58.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:58 smithi184 ceph-mon[22331]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:03:58.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:58 smithi184 ceph-mon[22331]: Deploying daemon mds.h on smithi155 2024-08-31T06:03:58.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:58 smithi184 ceph-mon[22331]: mds.? [v2:172.21.15.184:6836/1712755588,v1:172.21.15.184:6837/1712755588] up:boot 2024-08-31T06:03:58.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:58 smithi184 ceph-mon[22331]: fsmap 7 up:standby 2024-08-31T06:03:58.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:58 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-08-31T06:03:58.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:03:58 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/427567285' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:03:59.619 INFO:journalctl@ceph.mds.h.smithi155.stdout:Aug 31 06:03:59 smithi155 podman[53022]: 2024-08-31 06:03:59.479711292 +0000 UTC m=+0.016259676 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T06:03:59.619 INFO:journalctl@ceph.mds.h.smithi155.stdout:Aug 31 06:03:59 smithi155 podman[53022]: 2024-08-31 06:03:59.617707507 +0000 UTC m=+0.154255943 container create f74a94b68e1dfafe125401d06b7721b14505d3c3a6564eda2415ad977cd2ddc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-h, io.buildah.version=1.37.1, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-08-31T06:04:00.372 INFO:journalctl@ceph.mds.h.smithi155.stdout:Aug 31 06:03:59 smithi155 podman[53022]: 2024-08-31 06:03:59.903873518 +0000 UTC m=+0.440421892 container init f74a94b68e1dfafe125401d06b7721b14505d3c3a6564eda2415ad977cd2ddc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-h, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.1, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-08-31T06:04:00.372 INFO:journalctl@ceph.mds.h.smithi155.stdout:Aug 31 06:03:59 smithi155 podman[53022]: 2024-08-31 06:03:59.906852772 +0000 UTC m=+0.443401163 container start f74a94b68e1dfafe125401d06b7721b14505d3c3a6564eda2415ad977cd2ddc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-h, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.1, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-08-31T06:04:00.372 INFO:journalctl@ceph.mds.h.smithi155.stdout:Aug 31 06:03:59 smithi155 ceph-mds[53041]: -- 172.21.15.155:0/1318530236 <== mon.2 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55679b7dfc20 con 0x55679f985a80 2024-08-31T06:04:00.372 INFO:journalctl@ceph.mds.h.smithi155.stdout:Aug 31 06:03:59 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-h[53037]: starting mds.h at 2024-08-31T06:04:00.372 INFO:journalctl@ceph.mds.h.smithi155.stdout:Aug 31 06:03:59 smithi155 bash[53022]: f74a94b68e1dfafe125401d06b7721b14505d3c3a6564eda2415ad977cd2ddc0 2024-08-31T06:04:00.372 INFO:journalctl@ceph.mds.h.smithi155.stdout:Aug 31 06:03:59 smithi155 systemd[1]: Started Ceph mds.h for 673e4aa0-675d-11ef-bcd4-c7b262605968. 2024-08-31T06:04:00.925 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:04:01.281 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:01 smithi016 ceph-mon[17194]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:01.282 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:01 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:01.282 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:01 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:01.282 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:01 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:01.282 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:01 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:04:01.282 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:01 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:04:01.282 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:01 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T06:04:01.282 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:01 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:04:01.282 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:01 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-08-31T06:04:01.282 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:01 smithi016 ceph-mon[17194]: mds.? [v2:172.21.15.155:6836/1089710405,v1:172.21.15.155:6837/1089710405] up:boot 2024-08-31T06:04:01.282 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:01 smithi016 ceph-mon[17194]: fsmap 8 up:standby 2024-08-31T06:04:01.329 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:04:01.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:01 smithi155 ceph-mon[21742]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:01.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:01 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:01.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:01 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:01.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:01 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:01.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:01 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:04:01.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:01 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:04:01.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:01 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T06:04:01.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:01 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:04:01.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:01 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-08-31T06:04:01.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:01 smithi155 ceph-mon[21742]: mds.? [v2:172.21.15.155:6836/1089710405,v1:172.21.15.155:6837/1089710405] up:boot 2024-08-31T06:04:01.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:01 smithi155 ceph-mon[21742]: fsmap 8 up:standby 2024-08-31T06:04:01.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:01 smithi184 ceph-mon[22331]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:01.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:01 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:01.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:01 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:01.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:01 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:01.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:01 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:04:01.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:01 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:04:01.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:01 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T06:04:01.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:01 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:04:01.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:01 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-08-31T06:04:01.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:01 smithi184 ceph-mon[22331]: mds.? [v2:172.21.15.155:6836/1089710405,v1:172.21.15.155:6837/1089710405] up:boot 2024-08-31T06:04:01.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:01 smithi184 ceph-mon[22331]: fsmap 8 up:standby 2024-08-31T06:04:01.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:01.962+0000 7f84a7400640 1 -- 172.21.15.16:0/2066061162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84a0103900 msgr2=0x7f84a0103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:01.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:01.962+0000 7f84a7400640 1 --2- 172.21.15.16:0/2066061162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84a0103900 0x7f84a0103d10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8494009900 tx=0x7f849402a3f0 comp rx=0 tx=0).stop 2024-08-31T06:04:01.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:01.962+0000 7f84a7400640 1 -- 172.21.15.16:0/2066061162 shutdown_connections 2024-08-31T06:04:01.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:01.962+0000 7f84a7400640 1 --2- 172.21.15.16:0/2066061162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a0109b90 0x7f84a01103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:01.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:01.962+0000 7f84a7400640 1 --2- 172.21.15.16:0/2066061162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84a0109240 0x7f84a0109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:01.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:01.962+0000 7f84a7400640 1 --2- 172.21.15.16:0/2066061162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84a0103900 0x7f84a0103d10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:01.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:01.962+0000 7f84a7400640 1 -- 172.21.15.16:0/2066061162 >> 172.21.15.16:0/2066061162 conn(0x7f84a007b5f0 msgr2=0x7f84a007ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:04:01.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:01.963+0000 7f84a7400640 1 -- 172.21.15.16:0/2066061162 shutdown_connections 2024-08-31T06:04:01.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:01.963+0000 7f84a7400640 1 -- 172.21.15.16:0/2066061162 wait complete. 2024-08-31T06:04:01.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:01.964+0000 7f84a7400640 1 Processor -- start 2024-08-31T06:04:01.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:01.965+0000 7f84a7400640 1 -- start start 2024-08-31T06:04:01.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:01.965+0000 7f84a7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a0103900 0x7f84a00754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:01.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:01.965+0000 7f84a7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84a0109240 0x7f84a00759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:01.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:01.965+0000 7f84a7400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84a0109b90 0x7f84a0075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:01.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:01.965+0000 7f84a7400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f84a0113850 con 0x7f84a0103900 2024-08-31T06:04:01.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:01.965+0000 7f84a7400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f84a01136d0 con 0x7f84a0109240 2024-08-31T06:04:01.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:01.965+0000 7f84a7400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f84a01139d0 con 0x7f84a0109b90 2024-08-31T06:04:01.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:01.966+0000 7f84a5400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84a0109b90 0x7f84a0075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:01.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:01.966+0000 7f84a4a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a0103900 0x7f84a00754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:01.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:01.966+0000 7f84a5400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84a0109b90 0x7f84a0075f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:43308/0 (socket says 172.21.15.16:43308) 2024-08-31T06:04:01.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:01.966+0000 7f84a5400640 1 -- 172.21.15.16:0/3170639554 learned_addr learned my addr 172.21.15.16:0/3170639554 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:04:01.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:01.966+0000 7f84a4a00640 1 --2- 172.21.15.16:0/3170639554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a0103900 0x7f84a00754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48612/0 (socket says 172.21.15.16:48612) 2024-08-31T06:04:01.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:01.966+0000 7f849fe00640 1 --2- 172.21.15.16:0/3170639554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84a0109240 0x7f84a00759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:01.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:01.966+0000 7f84a5400640 1 -- 172.21.15.16:0/3170639554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84a0109240 msgr2=0x7f84a00759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:01.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:01.966+0000 7f84a5400640 1 --2- 172.21.15.16:0/3170639554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84a0109240 0x7f84a00759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:01.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:01.966+0000 7f84a5400640 1 -- 172.21.15.16:0/3170639554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a0103900 msgr2=0x7f84a00754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:01.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:01.966+0000 7f84a5400640 1 --2- 172.21.15.16:0/3170639554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a0103900 0x7f84a00754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:01.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:01.966+0000 7f84a5400640 1 -- 172.21.15.16:0/3170639554 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a01a8540 con 0x7f84a0109b90 2024-08-31T06:04:01.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:01.966+0000 7f84a5400640 1 --2- 172.21.15.16:0/3170639554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84a0109b90 0x7f84a0075f30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f849800efd0 tx=0x7f849800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:04:01.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:01.967+0000 7f849d600640 1 -- 172.21.15.16:0/3170639554 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8498003b70 con 0x7f84a0109b90 2024-08-31T06:04:01.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:01.967+0000 7f849d600640 1 -- 172.21.15.16:0/3170639554 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8498003d10 con 0x7f84a0109b90 2024-08-31T06:04:01.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:01.967+0000 7f84a7400640 1 -- 172.21.15.16:0/3170639554 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84a01a8830 con 0x7f84a0109b90 2024-08-31T06:04:01.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:01.967+0000 7f84a7400640 1 -- 172.21.15.16:0/3170639554 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84a01a8d70 con 0x7f84a0109b90 2024-08-31T06:04:01.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:01.967+0000 7f849d600640 1 -- 172.21.15.16:0/3170639554 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84980108a0 con 0x7f84a0109b90 2024-08-31T06:04:01.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:01.968+0000 7f84a7400640 1 -- 172.21.15.16:0/3170639554 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84a0103d10 con 0x7f84a0109b90 2024-08-31T06:04:01.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:01.970+0000 7f849d600640 1 -- 172.21.15.16:0/3170639554 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7f8498004020 con 0x7f84a0109b90 2024-08-31T06:04:01.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:01.973+0000 7f849d600640 1 --2- 172.21.15.16:0/3170639554 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f84880b4c00 0x7f84880b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:01.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:01.973+0000 7f849d600640 1 -- 172.21.15.16:0/3170639554 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7f84980e5220 con 0x7f84a0109b90 2024-08-31T06:04:01.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:01.973+0000 7f84a4a00640 1 --2- 172.21.15.16:0/3170639554 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f84880b4c00 0x7f84880b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:01.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:01.974+0000 7f84a4a00640 1 --2- 172.21.15.16:0/3170639554 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f84880b4c00 0x7f84880b7110 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8494000c00 tx=0x7f8494031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:04:01.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:01.975+0000 7f849d600640 1 -- 172.21.15.16:0/3170639554 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f84980ac680 con 0x7f84a0109b90 2024-08-31T06:04:02.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:02 smithi016 ceph-mon[17194]: Deploying daemon mds.i on smithi184 2024-08-31T06:04:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:02.210+0000 7f84a7400640 1 -- 172.21.15.16:0/3170639554 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84a002e430 con 0x7f84a0109b90 2024-08-31T06:04:02.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:02.210+0000 7f849d600640 1 -- 172.21.15.16:0/3170639554 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7225 (secure 0 0 0) 0x7f84980abf80 con 0x7f84a0109b90 2024-08-31T06:04:02.213 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:04:02.213 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":9,"btime":"2024-08-31T06:04:00:299637+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24688,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24694,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24698,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/1089710405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1089710405},{"type":"v1","addr":"172.21.15.155:6837","nonce":1089710405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-08-31T06:04:02.213 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 9 2024-08-31T06:04:02.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:02.213+0000 7f84a7400640 1 -- 172.21.15.16:0/3170639554 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f84880b4c00 msgr2=0x7f84880b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:02.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:02.213+0000 7f84a7400640 1 --2- 172.21.15.16:0/3170639554 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f84880b4c00 0x7f84880b7110 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8494000c00 tx=0x7f8494031400 comp rx=0 tx=0).stop 2024-08-31T06:04:02.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:02.213+0000 7f84a7400640 1 -- 172.21.15.16:0/3170639554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84a0109b90 msgr2=0x7f84a0075f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:02.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:02.213+0000 7f84a7400640 1 --2- 172.21.15.16:0/3170639554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84a0109b90 0x7f84a0075f30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f849800efd0 tx=0x7f849800c6a0 comp rx=0 tx=0).stop 2024-08-31T06:04:02.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:02.213+0000 7f84a7400640 1 -- 172.21.15.16:0/3170639554 shutdown_connections 2024-08-31T06:04:02.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:02.213+0000 7f84a7400640 1 --2- 172.21.15.16:0/3170639554 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f84880b4c00 0x7f84880b7110 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:02.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:02.213+0000 7f84a7400640 1 --2- 172.21.15.16:0/3170639554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84a0109b90 0x7f84a0075f30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:02.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:02.213+0000 7f84a7400640 1 --2- 172.21.15.16:0/3170639554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84a0109240 0x7f84a00759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:02.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:02.213+0000 7f84a7400640 1 --2- 172.21.15.16:0/3170639554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a0103900 0x7f84a00754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:02.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:02.213+0000 7f84a7400640 1 -- 172.21.15.16:0/3170639554 >> 172.21.15.16:0/3170639554 conn(0x7f84a007b5f0 msgr2=0x7f84a0107330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:04:02.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:02.213+0000 7f84a7400640 1 -- 172.21.15.16:0/3170639554 shutdown_connections 2024-08-31T06:04:02.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:02.213+0000 7f84a7400640 1 -- 172.21.15.16:0/3170639554 wait complete. 2024-08-31T06:04:02.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:02 smithi184 ceph-mon[22331]: Deploying daemon mds.i on smithi184 2024-08-31T06:04:02.223 INFO:journalctl@ceph.mds.i.smithi184.stdout:Aug 31 06:04:02 smithi184 systemd[1]: Starting Ceph mds.i for 673e4aa0-675d-11ef-bcd4-c7b262605968... 2024-08-31T06:04:02.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:02 smithi155 ceph-mon[21742]: Deploying daemon mds.i on smithi184 2024-08-31T06:04:02.600 INFO:journalctl@ceph.mds.i.smithi184.stdout:Aug 31 06:04:02 smithi184 podman[54380]: 2024-08-31 06:04:02.250317573 +0000 UTC m=+0.018167425 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T06:04:02.600 INFO:journalctl@ceph.mds.i.smithi184.stdout:Aug 31 06:04:02 smithi184 podman[54380]: 2024-08-31 06:04:02.38016025 +0000 UTC m=+0.148010083 container create bbb227bfc2fad76696427cb902cf9082995e793f404e274caa6ea4f89795c22d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-i, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.1) 2024-08-31T06:04:02.972 INFO:journalctl@ceph.mds.i.smithi184.stdout:Aug 31 06:04:02 smithi184 podman[54380]: 2024-08-31 06:04:02.66679976 +0000 UTC m=+0.434649607 container init bbb227bfc2fad76696427cb902cf9082995e793f404e274caa6ea4f89795c22d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.1, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-08-31T06:04:02.973 INFO:journalctl@ceph.mds.i.smithi184.stdout:Aug 31 06:04:02 smithi184 podman[54380]: 2024-08-31 06:04:02.669951966 +0000 UTC m=+0.437801799 container start bbb227bfc2fad76696427cb902cf9082995e793f404e274caa6ea4f89795c22d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-i, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.1, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-08-31T06:04:02.973 INFO:journalctl@ceph.mds.i.smithi184.stdout:Aug 31 06:04:02 smithi184 ceph-mds[54407]: -- 172.21.15.184:0/357323295 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ecb45ab860 con 0x55ecb8751180 2024-08-31T06:04:02.973 INFO:journalctl@ceph.mds.i.smithi184.stdout:Aug 31 06:04:02 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-i[54398]: starting mds.i at 2024-08-31T06:04:02.973 INFO:journalctl@ceph.mds.i.smithi184.stdout:Aug 31 06:04:02 smithi184 bash[54380]: bbb227bfc2fad76696427cb902cf9082995e793f404e274caa6ea4f89795c22d 2024-08-31T06:04:02.973 INFO:journalctl@ceph.mds.i.smithi184.stdout:Aug 31 06:04:02 smithi184 systemd[1]: Started Ceph mds.i for 673e4aa0-675d-11ef-bcd4-c7b262605968. 2024-08-31T06:04:03.369 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:03 smithi184 ceph-mon[22331]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:03.369 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:03 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/3170639554' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:04:03.369 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:03 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:03.369 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:03 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:03.369 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:03 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:03.369 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:03 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:04:03.369 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:03 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:04:03.369 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:03 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T06:04:03.369 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:03 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:04:03.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:03 smithi155 ceph-mon[21742]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:03.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:03 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/3170639554' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:04:03.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:03 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:03.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:03 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:03.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:03 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:03.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:03 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:04:03.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:03 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:04:03.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:03 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T06:04:03.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:03 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:04:03.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:03 smithi016 ceph-mon[17194]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:03.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:03 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/3170639554' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:04:03.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:03 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:03.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:03 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:03.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:03 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:03.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:03 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:04:03.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:03 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:04:03.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:03 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T06:04:03.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:03 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:04:04.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:04 smithi155 ceph-mon[21742]: Deploying daemon mds.g on smithi016 2024-08-31T06:04:04.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:04 smithi155 ceph-mon[21742]: mds.? [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] up:boot 2024-08-31T06:04:04.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:04 smithi155 ceph-mon[21742]: fsmap 9 up:standby 2024-08-31T06:04:04.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:04 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-08-31T06:04:04.410 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:04 smithi016 ceph-mon[17194]: Deploying daemon mds.g on smithi016 2024-08-31T06:04:04.410 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:04 smithi016 ceph-mon[17194]: mds.? [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] up:boot 2024-08-31T06:04:04.411 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:04 smithi016 ceph-mon[17194]: fsmap 9 up:standby 2024-08-31T06:04:04.411 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:04 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-08-31T06:04:04.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:04 smithi184 ceph-mon[22331]: Deploying daemon mds.g on smithi016 2024-08-31T06:04:04.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:04 smithi184 ceph-mon[22331]: mds.? [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] up:boot 2024-08-31T06:04:04.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:04 smithi184 ceph-mon[22331]: fsmap 9 up:standby 2024-08-31T06:04:04.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:04 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-08-31T06:04:05.015 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:04:05.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:05 smithi155 ceph-mon[21742]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:05.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:05 smithi016 ceph-mon[17194]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:05.465 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:04:05.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:05 smithi184 ceph-mon[22331]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:06.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.131+0000 7fb46f600640 1 -- 172.21.15.16:0/3649515049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4681101e0 msgr2=0x7fb468116a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:06.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.131+0000 7fb46f600640 1 --2- 172.21.15.16:0/3649515049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4681101e0 0x7fb468116a10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb460009ca0 tx=0x7fb46002a8c0 comp rx=0 tx=0).stop 2024-08-31T06:04:06.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.131+0000 7fb46f600640 1 -- 172.21.15.16:0/3649515049 shutdown_connections 2024-08-31T06:04:06.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.131+0000 7fb46f600640 1 --2- 172.21.15.16:0/3649515049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4681101e0 0x7fb468116a10 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:06.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.131+0000 7fb46f600640 1 --2- 172.21.15.16:0/3649515049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb46810f890 0x7fb46810fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:06.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.131+0000 7fb46f600640 1 --2- 172.21.15.16:0/3649515049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb468109f50 0x7fb46810a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:06.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.131+0000 7fb46f600640 1 -- 172.21.15.16:0/3649515049 >> 172.21.15.16:0/3649515049 conn(0x7fb468077e80 msgr2=0x7fb468107f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:04:06.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.131+0000 7fb46f600640 1 -- 172.21.15.16:0/3649515049 shutdown_connections 2024-08-31T06:04:06.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.131+0000 7fb46f600640 1 -- 172.21.15.16:0/3649515049 wait complete. 2024-08-31T06:04:06.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.132+0000 7fb46f600640 1 Processor -- start 2024-08-31T06:04:06.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.133+0000 7fb46f600640 1 -- start start 2024-08-31T06:04:06.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.133+0000 7fb46f600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb468109f50 0x7fb4681a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:06.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.133+0000 7fb46f600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb46810f890 0x7fb4681a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:06.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.133+0000 7fb46f600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4681101e0 0x7fb4681a6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:06.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.133+0000 7fb46f600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb468119ea0 con 0x7fb468109f50 2024-08-31T06:04:06.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.133+0000 7fb46f600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb468119d20 con 0x7fb4681101e0 2024-08-31T06:04:06.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.133+0000 7fb46f600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb46811a020 con 0x7fb46810f890 2024-08-31T06:04:06.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.133+0000 7fb46cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb468109f50 0x7fb4681a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:06.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.133+0000 7fb46cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb468109f50 0x7fb4681a0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39832/0 (socket says 172.21.15.16:39832) 2024-08-31T06:04:06.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.133+0000 7fb46cc00640 1 -- 172.21.15.16:0/308397841 learned_addr learned my addr 172.21.15.16:0/308397841 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:04:06.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.133+0000 7fb45fe00640 1 --2- 172.21.15.16:0/308397841 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb46810f890 0x7fb4681a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:06.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.133+0000 7fb46d600640 1 --2- 172.21.15.16:0/308397841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4681101e0 0x7fb4681a6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:06.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.134+0000 7fb46cc00640 1 -- 172.21.15.16:0/308397841 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb46810f890 msgr2=0x7fb4681a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:06.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.134+0000 7fb46cc00640 1 --2- 172.21.15.16:0/308397841 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb46810f890 0x7fb4681a0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:06.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.134+0000 7fb46cc00640 1 -- 172.21.15.16:0/308397841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4681101e0 msgr2=0x7fb4681a6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:06.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.134+0000 7fb46cc00640 1 --2- 172.21.15.16:0/308397841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4681101e0 0x7fb4681a6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:06.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.134+0000 7fb46cc00640 1 -- 172.21.15.16:0/308397841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4681a67c0 con 0x7fb468109f50 2024-08-31T06:04:06.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.134+0000 7fb45fe00640 1 --2- 172.21.15.16:0/308397841 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb46810f890 0x7fb4681a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:04:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.134+0000 7fb46cc00640 1 --2- 172.21.15.16:0/308397841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb468109f50 0x7fb4681a0850 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb45800e9f0 tx=0x7fb45800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:04:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.134+0000 7fb466000640 1 -- 172.21.15.16:0/308397841 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45800cd80 con 0x7fb468109f50 2024-08-31T06:04:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.134+0000 7fb466000640 1 -- 172.21.15.16:0/308397841 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4580108e0 con 0x7fb468109f50 2024-08-31T06:04:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.134+0000 7fb46f600640 1 -- 172.21.15.16:0/308397841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4681a6ab0 con 0x7fb468109f50 2024-08-31T06:04:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.134+0000 7fb466000640 1 -- 172.21.15.16:0/308397841 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb458018980 con 0x7fb468109f50 2024-08-31T06:04:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.135+0000 7fb46f600640 1 -- 172.21.15.16:0/308397841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4681ad210 con 0x7fb468109f50 2024-08-31T06:04:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.136+0000 7fb46f600640 1 -- 172.21.15.16:0/308397841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb46810a360 con 0x7fb468109f50 2024-08-31T06:04:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.138+0000 7fb466000640 1 -- 172.21.15.16:0/308397841 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7fb458010430 con 0x7fb468109f50 2024-08-31T06:04:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.139+0000 7fb466000640 1 --2- 172.21.15.16:0/308397841 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb44c0b4b80 0x7fb44c0b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.139+0000 7fb466000640 1 -- 172.21.15.16:0/308397841 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7fb4580e5660 con 0x7fb468109f50 2024-08-31T06:04:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.140+0000 7fb45fe00640 1 --2- 172.21.15.16:0/308397841 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb44c0b4b80 0x7fb44c0b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.140+0000 7fb45fe00640 1 --2- 172.21.15.16:0/308397841 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb44c0b4b80 0x7fb44c0b7090 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb46806adf0 tx=0x7fb450005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:04:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.142+0000 7fb466000640 1 -- 172.21.15.16:0/308397841 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fb4580ac470 con 0x7fb468109f50 2024-08-31T06:04:06.337 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:06 smithi016 ceph-mon[17194]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:06.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:06 smithi155 ceph-mon[21742]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:06.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.380+0000 7fb46f600640 1 -- 172.21.15.16:0/308397841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb46802e430 con 0x7fb468109f50 2024-08-31T06:04:06.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.380+0000 7fb466000640 1 -- 172.21.15.16:0/308397841 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8055 (secure 0 0 0) 0x7fb4580b0c30 con 0x7fb468109f50 2024-08-31T06:04:06.383 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:04:06.383 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":10,"btime":"2024-08-31T06:04:03:054986+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24688,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24694,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24698,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/1089710405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1089710405},{"type":"v1","addr":"172.21.15.155:6837","nonce":1089710405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24712,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-08-31T06:04:06.383 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 10 2024-08-31T06:04:06.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.383+0000 7fb46f600640 1 -- 172.21.15.16:0/308397841 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb44c0b4b80 msgr2=0x7fb44c0b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:06.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.383+0000 7fb46f600640 1 --2- 172.21.15.16:0/308397841 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb44c0b4b80 0x7fb44c0b7090 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb46806adf0 tx=0x7fb450005f90 comp rx=0 tx=0).stop 2024-08-31T06:04:06.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.383+0000 7fb46f600640 1 -- 172.21.15.16:0/308397841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb468109f50 msgr2=0x7fb4681a0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:06.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.383+0000 7fb46f600640 1 --2- 172.21.15.16:0/308397841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb468109f50 0x7fb4681a0850 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb45800e9f0 tx=0x7fb45800eeb0 comp rx=0 tx=0).stop 2024-08-31T06:04:06.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.383+0000 7fb46f600640 1 -- 172.21.15.16:0/308397841 shutdown_connections 2024-08-31T06:04:06.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.383+0000 7fb46f600640 1 --2- 172.21.15.16:0/308397841 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb44c0b4b80 0x7fb44c0b7090 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:06.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.383+0000 7fb46f600640 1 --2- 172.21.15.16:0/308397841 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb46810f890 0x7fb4681a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:06.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.383+0000 7fb46f600640 1 --2- 172.21.15.16:0/308397841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4681101e0 0x7fb4681a6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:06.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.383+0000 7fb46f600640 1 --2- 172.21.15.16:0/308397841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb468109f50 0x7fb4681a0850 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:06.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.383+0000 7fb46f600640 1 -- 172.21.15.16:0/308397841 >> 172.21.15.16:0/308397841 conn(0x7fb468077e80 msgr2=0x7fb468078580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:04:06.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.383+0000 7fb46f600640 1 -- 172.21.15.16:0/308397841 shutdown_connections 2024-08-31T06:04:06.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:06.383+0000 7fb46f600640 1 -- 172.21.15.16:0/308397841 wait complete. 2024-08-31T06:04:06.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:06 smithi184 ceph-mon[22331]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:07.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:07 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/308397841' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:04:07.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:07 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:07.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:07 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/308397841' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:04:07.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:07 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:07.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:07 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/308397841' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:04:07.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:07 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:08.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:08 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:08.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:08 smithi016 ceph-mon[17194]: mds.? [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] up:boot 2024-08-31T06:04:08.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:08 smithi016 ceph-mon[17194]: fsmap 10 up:standby 2024-08-31T06:04:08.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:08 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-08-31T06:04:08.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:08 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:08.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:08 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:04:08.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:08 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:04:08.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:08 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T06:04:08.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:08 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:04:08.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:08 smithi016 ceph-mon[17194]: Deploying daemon mds.a on smithi016 2024-08-31T06:04:08.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:08 smithi016 ceph-mon[17194]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:08.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:08 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:08.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:08 smithi155 ceph-mon[21742]: mds.? [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] up:boot 2024-08-31T06:04:08.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:08 smithi155 ceph-mon[21742]: fsmap 10 up:standby 2024-08-31T06:04:08.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:08 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-08-31T06:04:08.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:08 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:08.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:08 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:04:08.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:08 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:04:08.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:08 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T06:04:08.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:08 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:04:08.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:08 smithi155 ceph-mon[21742]: Deploying daemon mds.a on smithi016 2024-08-31T06:04:08.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:08 smithi155 ceph-mon[21742]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:08.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:08 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:08.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:08 smithi184 ceph-mon[22331]: mds.? [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] up:boot 2024-08-31T06:04:08.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:08 smithi184 ceph-mon[22331]: fsmap 10 up:standby 2024-08-31T06:04:08.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:08 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-08-31T06:04:08.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:08 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:08.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:08 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:04:08.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:08 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:04:08.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:08 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T06:04:08.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:08 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:04:08.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:08 smithi184 ceph-mon[22331]: Deploying daemon mds.a on smithi016 2024-08-31T06:04:08.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:08 smithi184 ceph-mon[22331]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:09.654 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:04:10.305 INFO:journalctl@ceph.mds.a.smithi016.stdout:Aug 31 06:04:09 smithi016 systemd[1]: Starting Ceph mds.a for 673e4aa0-675d-11ef-bcd4-c7b262605968... 2024-08-31T06:04:10.616 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:10 smithi016 ceph-mon[17194]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:10.617 INFO:journalctl@ceph.mds.a.smithi016.stdout:Aug 31 06:04:10 smithi016 podman[62366]: 2024-08-31 06:04:10.210260365 +0000 UTC m=+0.031514283 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T06:04:10.617 INFO:journalctl@ceph.mds.a.smithi016.stdout:Aug 31 06:04:10 smithi016 podman[62366]: 2024-08-31 06:04:10.370214835 +0000 UTC m=+0.191468733 container create d86cbad71303eb583dd1238165a42d1125b8a795203b96a1d7d40135ecf041dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-a, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.1, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-08-31T06:04:10.617 INFO:journalctl@ceph.mds.a.smithi016.stdout:Aug 31 06:04:10 smithi016 podman[62366]: 2024-08-31 06:04:10.614937247 +0000 UTC m=+0.436191170 container init d86cbad71303eb583dd1238165a42d1125b8a795203b96a1d7d40135ecf041dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.1, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-08-31T06:04:10.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:10 smithi155 ceph-mon[21742]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:10.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:10 smithi184 ceph-mon[22331]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:10.837 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:04:10.913 INFO:journalctl@ceph.mds.a.smithi016.stdout:Aug 31 06:04:10 smithi016 podman[62366]: 2024-08-31 06:04:10.623555635 +0000 UTC m=+0.444809543 container start d86cbad71303eb583dd1238165a42d1125b8a795203b96a1d7d40135ecf041dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.1, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-08-31T06:04:10.913 INFO:journalctl@ceph.mds.a.smithi016.stdout:Aug 31 06:04:10 smithi016 ceph-mds[62402]: -- 172.21.15.16:0/3032846887 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558f7ab9bc20 con 0x558f7ed41180 2024-08-31T06:04:10.913 INFO:journalctl@ceph.mds.a.smithi016.stdout:Aug 31 06:04:10 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-a[62398]: starting mds.a at 2024-08-31T06:04:10.913 INFO:journalctl@ceph.mds.a.smithi016.stdout:Aug 31 06:04:10 smithi016 bash[62366]: d86cbad71303eb583dd1238165a42d1125b8a795203b96a1d7d40135ecf041dc 2024-08-31T06:04:10.913 INFO:journalctl@ceph.mds.a.smithi016.stdout:Aug 31 06:04:10 smithi016 systemd[1]: Started Ceph mds.a for 673e4aa0-675d-11ef-bcd4-c7b262605968. 2024-08-31T06:04:11.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.658+0000 7ff5b7c00640 1 -- 172.21.15.16:0/1677827179 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5b0109240 msgr2=0x7ff5b0109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:11.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.658+0000 7ff5b7c00640 1 --2- 172.21.15.16:0/1677827179 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5b0109240 0x7ff5b0109650 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff598009860 tx=0x7ff59802a340 comp rx=0 tx=0).stop 2024-08-31T06:04:11.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.658+0000 7ff5b7c00640 1 -- 172.21.15.16:0/1677827179 shutdown_connections 2024-08-31T06:04:11.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.658+0000 7ff5b7c00640 1 --2- 172.21.15.16:0/1677827179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5b0109b90 0x7ff5b01103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:11.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.658+0000 7ff5b7c00640 1 --2- 172.21.15.16:0/1677827179 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5b0109240 0x7ff5b0109650 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:11.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.658+0000 7ff5b7c00640 1 --2- 172.21.15.16:0/1677827179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5b0103900 0x7ff5b0103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:11.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.658+0000 7ff5b7c00640 1 -- 172.21.15.16:0/1677827179 >> 172.21.15.16:0/1677827179 conn(0x7ff5b007b5f0 msgr2=0x7ff5b007ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:04:11.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.658+0000 7ff5b7c00640 1 -- 172.21.15.16:0/1677827179 shutdown_connections 2024-08-31T06:04:11.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.658+0000 7ff5b7c00640 1 -- 172.21.15.16:0/1677827179 wait complete. 2024-08-31T06:04:11.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.660+0000 7ff5b7c00640 1 Processor -- start 2024-08-31T06:04:11.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.660+0000 7ff5b7c00640 1 -- start start 2024-08-31T06:04:11.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.660+0000 7ff5b7c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5b0103900 0x7ff5b00754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:11.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.660+0000 7ff5b7c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5b0109240 0x7ff5b00759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:11.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.660+0000 7ff5b7c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5b0109b90 0x7ff5b0075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:11.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.660+0000 7ff5b7c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b0113850 con 0x7ff5b0109240 2024-08-31T06:04:11.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.660+0000 7ff5b7c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b01136d0 con 0x7ff5b0109b90 2024-08-31T06:04:11.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.660+0000 7ff5b7c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b01139d0 con 0x7ff5b0103900 2024-08-31T06:04:11.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.660+0000 7ff5a7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5b0109240 0x7ff5b00759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:11.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.661+0000 7ff5a7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5b0109240 0x7ff5b00759f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39900/0 (socket says 172.21.15.16:39900) 2024-08-31T06:04:11.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.661+0000 7ff5a7e00640 1 -- 172.21.15.16:0/1952197340 learned_addr learned my addr 172.21.15.16:0/1952197340 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:04:11.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.661+0000 7ff5b5c00640 1 --2- 172.21.15.16:0/1952197340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5b0109b90 0x7ff5b0075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:11.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.661+0000 7ff5b5200640 1 --2- 172.21.15.16:0/1952197340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5b0103900 0x7ff5b00754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:11.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.661+0000 7ff5a7e00640 1 -- 172.21.15.16:0/1952197340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5b0103900 msgr2=0x7ff5b00754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:11.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.661+0000 7ff5a7e00640 1 --2- 172.21.15.16:0/1952197340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5b0103900 0x7ff5b00754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:11.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.661+0000 7ff5a7e00640 1 -- 172.21.15.16:0/1952197340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5b0109b90 msgr2=0x7ff5b0075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:11.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.661+0000 7ff5a7e00640 1 --2- 172.21.15.16:0/1952197340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5b0109b90 0x7ff5b0075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:11.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.661+0000 7ff5a7e00640 1 -- 172.21.15.16:0/1952197340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5b01a84c0 con 0x7ff5b0109240 2024-08-31T06:04:11.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.661+0000 7ff5b5c00640 1 --2- 172.21.15.16:0/1952197340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5b0109b90 0x7ff5b0075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:04:11.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.661+0000 7ff5a7e00640 1 --2- 172.21.15.16:0/1952197340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5b0109240 0x7ff5b00759f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff598035040 tx=0x7ff598032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:04:11.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.662+0000 7ff5ae000640 1 -- 172.21.15.16:0/1952197340 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff598037040 con 0x7ff5b0109240 2024-08-31T06:04:11.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.662+0000 7ff5ae000640 1 -- 172.21.15.16:0/1952197340 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff598030c10 con 0x7ff5b0109240 2024-08-31T06:04:11.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.662+0000 7ff5b7c00640 1 -- 172.21.15.16:0/1952197340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5b01a8750 con 0x7ff5b0109240 2024-08-31T06:04:11.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.662+0000 7ff5ae000640 1 -- 172.21.15.16:0/1952197340 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5980035f0 con 0x7ff5b0109240 2024-08-31T06:04:11.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.662+0000 7ff5b7c00640 1 -- 172.21.15.16:0/1952197340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5b01a8c20 con 0x7ff5b0109240 2024-08-31T06:04:11.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.663+0000 7ff5b7c00640 1 -- 172.21.15.16:0/1952197340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5b0103d10 con 0x7ff5b0109240 2024-08-31T06:04:11.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.666+0000 7ff5ae000640 1 -- 172.21.15.16:0/1952197340 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7ff598043020 con 0x7ff5b0109240 2024-08-31T06:04:11.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.667+0000 7ff5ae000640 1 --2- 172.21.15.16:0/1952197340 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff5940b4c00 0x7ff5940b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:11.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.667+0000 7ff5ae000640 1 -- 172.21.15.16:0/1952197340 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7ff598104410 con 0x7ff5b0109240 2024-08-31T06:04:11.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.667+0000 7ff5b5200640 1 --2- 172.21.15.16:0/1952197340 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff5940b4c00 0x7ff5940b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:11.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.668+0000 7ff5b5200640 1 --2- 172.21.15.16:0/1952197340 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff5940b4c00 0x7ff5940b7110 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff5a00096c0 tx=0x7ff5a0009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:04:11.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.669+0000 7ff5ae000640 1 -- 172.21.15.16:0/1952197340 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7ff59804a060 con 0x7ff5b0109240 2024-08-31T06:04:11.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.903+0000 7ff5b7c00640 1 -- 172.21.15.16:0/1952197340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5b00767b0 con 0x7ff5b0109240 2024-08-31T06:04:11.906 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:11 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:11.906 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:11 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:11.906 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:11 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:11.906 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:11 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:04:11.906 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:11 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:04:11.906 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:11 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T06:04:11.907 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:11 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:04:11.907 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:11 smithi016 ceph-mon[17194]: Deploying daemon mds.b on smithi155 2024-08-31T06:04:11.907 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:11 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-08-31T06:04:11.907 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:11 smithi016 ceph-mon[17194]: mds.? [v2:172.21.15.16:6840/3621101104,v1:172.21.15.16:6841/3621101104] up:boot 2024-08-31T06:04:11.907 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:11 smithi016 ceph-mon[17194]: fsmap 11 up:standby 2024-08-31T06:04:11.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.904+0000 7ff5ae000640 1 -- 172.21.15.16:0/1952197340 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9704 (secure 0 0 0) 0x7ff5980cb8f0 con 0x7ff5b0109240 2024-08-31T06:04:11.908 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:04:11.908 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":12,"btime":"2024-08-31T06:04:11:303371+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24688,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24694,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24698,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/1089710405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1089710405},{"type":"v1","addr":"172.21.15.155:6837","nonce":1089710405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24712,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24724,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-08-31T06:04:11.908 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 12 2024-08-31T06:04:11.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.907+0000 7ff5b7c00640 1 -- 172.21.15.16:0/1952197340 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff5940b4c00 msgr2=0x7ff5940b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:11.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.907+0000 7ff5b7c00640 1 --2- 172.21.15.16:0/1952197340 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff5940b4c00 0x7ff5940b7110 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff5a00096c0 tx=0x7ff5a0009260 comp rx=0 tx=0).stop 2024-08-31T06:04:11.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.907+0000 7ff5b7c00640 1 -- 172.21.15.16:0/1952197340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5b0109240 msgr2=0x7ff5b00759f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:11.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.907+0000 7ff5b7c00640 1 --2- 172.21.15.16:0/1952197340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5b0109240 0x7ff5b00759f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff598035040 tx=0x7ff598032d50 comp rx=0 tx=0).stop 2024-08-31T06:04:11.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.907+0000 7ff5b7c00640 1 -- 172.21.15.16:0/1952197340 shutdown_connections 2024-08-31T06:04:11.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.907+0000 7ff5b7c00640 1 --2- 172.21.15.16:0/1952197340 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff5940b4c00 0x7ff5940b7110 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:11.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.907+0000 7ff5b7c00640 1 --2- 172.21.15.16:0/1952197340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5b0109b90 0x7ff5b0075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:11.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.907+0000 7ff5b7c00640 1 --2- 172.21.15.16:0/1952197340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5b0109240 0x7ff5b00759f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:11.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.907+0000 7ff5b7c00640 1 --2- 172.21.15.16:0/1952197340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5b0103900 0x7ff5b00754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:11.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.907+0000 7ff5b7c00640 1 -- 172.21.15.16:0/1952197340 >> 172.21.15.16:0/1952197340 conn(0x7ff5b007b5f0 msgr2=0x7ff5b0107330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:04:11.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.908+0000 7ff5b7c00640 1 -- 172.21.15.16:0/1952197340 shutdown_connections 2024-08-31T06:04:11.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:11.908+0000 7ff5b7c00640 1 -- 172.21.15.16:0/1952197340 wait complete. 2024-08-31T06:04:12.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:11 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:12.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:11 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:12.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:11 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:12.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:11 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:04:12.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:11 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:04:12.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:11 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T06:04:12.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:11 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:04:12.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:11 smithi155 ceph-mon[21742]: Deploying daemon mds.b on smithi155 2024-08-31T06:04:12.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:11 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-08-31T06:04:12.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:11 smithi155 ceph-mon[21742]: mds.? [v2:172.21.15.16:6840/3621101104,v1:172.21.15.16:6841/3621101104] up:boot 2024-08-31T06:04:12.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:11 smithi155 ceph-mon[21742]: fsmap 11 up:standby 2024-08-31T06:04:12.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:11 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:12.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:11 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:12.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:11 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:12.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:11 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:04:12.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:11 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-31T06:04:12.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:11 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-31T06:04:12.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:11 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:04:12.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:11 smithi184 ceph-mon[22331]: Deploying daemon mds.b on smithi155 2024-08-31T06:04:12.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:11 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-08-31T06:04:12.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:11 smithi184 ceph-mon[22331]: mds.? [v2:172.21.15.16:6840/3621101104,v1:172.21.15.16:6841/3621101104] up:boot 2024-08-31T06:04:12.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:11 smithi184 ceph-mon[22331]: fsmap 11 up:standby 2024-08-31T06:04:12.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:12 smithi155 ceph-mon[21742]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:12.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:12 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/1952197340' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:04:13.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:12 smithi016 ceph-mon[17194]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:13.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:12 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/1952197340' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:04:13.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:12 smithi184 ceph-mon[22331]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:13.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:12 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/1952197340' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:04:13.622 INFO:journalctl@ceph.mds.b.smithi155.stdout:Aug 31 06:04:13 smithi155 systemd[1]: Started Ceph mds.b for 673e4aa0-675d-11ef-bcd4-c7b262605968. 2024-08-31T06:04:14.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:14 smithi155 ceph-mon[21742]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:14.623 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:14 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:14.623 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:14 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:14.623 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:14 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:14.623 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:14 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:14.623 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:14 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:14.623 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:14 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:14.623 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:14 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:04:14.623 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:14 smithi155 ceph-mon[21742]: mds.? [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] up:boot 2024-08-31T06:04:14.623 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:14 smithi155 ceph-mon[21742]: fsmap 12 up:standby 2024-08-31T06:04:14.623 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:14 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-08-31T06:04:14.693 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:04:14.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:14 smithi184 ceph-mon[22331]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:14.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:14 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:14.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:14 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:14.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:14 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:14.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:14 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:14.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:14 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:14.724 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:14 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:14.724 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:14 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:04:14.724 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:14 smithi184 ceph-mon[22331]: mds.? [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] up:boot 2024-08-31T06:04:14.724 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:14 smithi184 ceph-mon[22331]: fsmap 12 up:standby 2024-08-31T06:04:14.724 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:14 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-08-31T06:04:14.751 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:14 smithi016 ceph-mon[17194]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:14.751 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:14 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:14.751 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:14 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:14.751 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:14 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:14.751 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:14 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:14.751 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:14 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:14.751 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:14 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:14.751 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:14 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:04:14.751 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:14 smithi016 ceph-mon[17194]: mds.? [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] up:boot 2024-08-31T06:04:14.752 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:14 smithi016 ceph-mon[17194]: fsmap 12 up:standby 2024-08-31T06:04:14.752 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:14 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-08-31T06:04:15.128 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:04:15.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:15.757+0000 7fc8dd800640 1 -- 172.21.15.16:0/3447463156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8d810b470 msgr2=0x7fc8d810b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:15.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:15.757+0000 7fc8dd800640 1 --2- 172.21.15.16:0/3447463156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8d810b470 0x7fc8d810b880 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4009900 tx=0x7fc8c402a3f0 comp rx=0 tx=0).stop 2024-08-31T06:04:15.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:15.757+0000 7fc8dd800640 1 -- 172.21.15.16:0/3447463156 shutdown_connections 2024-08-31T06:04:15.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:15.757+0000 7fc8dd800640 1 --2- 172.21.15.16:0/3447463156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8d810bdc0 0x7fc8d81125f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:15.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:15.757+0000 7fc8dd800640 1 --2- 172.21.15.16:0/3447463156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8d810b470 0x7fc8d810b880 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:15.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:15.757+0000 7fc8dd800640 1 --2- 172.21.15.16:0/3447463156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8d8105b30 0x7fc8d8105f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:15.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:15.757+0000 7fc8dd800640 1 -- 172.21.15.16:0/3447463156 >> 172.21.15.16:0/3447463156 conn(0x7fc8d8101600 msgr2=0x7fc8d8103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:04:15.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:15.757+0000 7fc8dd800640 1 -- 172.21.15.16:0/3447463156 shutdown_connections 2024-08-31T06:04:15.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:15.757+0000 7fc8dd800640 1 -- 172.21.15.16:0/3447463156 wait complete. 2024-08-31T06:04:15.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:15.758+0000 7fc8dd800640 1 Processor -- start 2024-08-31T06:04:15.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:15.758+0000 7fc8dd800640 1 -- start start 2024-08-31T06:04:15.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:15.759+0000 7fc8dd800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8d8105b30 0x7fc8d819c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:15.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:15.759+0000 7fc8dd800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8d810b470 0x7fc8d819cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:15.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:15.759+0000 7fc8dd800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8d810bdc0 0x7fc8d81a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:15.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:15.759+0000 7fc8dd800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d8115a80 con 0x7fc8d810b470 2024-08-31T06:04:15.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:15.759+0000 7fc8dd800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d8115900 con 0x7fc8d810bdc0 2024-08-31T06:04:15.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:15.759+0000 7fc8dd800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d8115c00 con 0x7fc8d8105b30 2024-08-31T06:04:15.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:15.759+0000 7fc8d6a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8d8105b30 0x7fc8d819c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:15.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:15.759+0000 7fc8d6a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8d8105b30 0x7fc8d819c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:57538/0 (socket says 172.21.15.16:57538) 2024-08-31T06:04:15.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:15.759+0000 7fc8d7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8d810bdc0 0x7fc8d81a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:15.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:15.759+0000 7fc8d6a00640 1 -- 172.21.15.16:0/2952115286 learned_addr learned my addr 172.21.15.16:0/2952115286 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:04:15.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:15.759+0000 7fc8d6000640 1 --2- 172.21.15.16:0/2952115286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8d810b470 0x7fc8d819cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:15.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:15.759+0000 7fc8d6a00640 1 -- 172.21.15.16:0/2952115286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8d810bdc0 msgr2=0x7fc8d81a1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:15.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:15.759+0000 7fc8d6a00640 1 --2- 172.21.15.16:0/2952115286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8d810bdc0 0x7fc8d81a1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:15.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:15.759+0000 7fc8d6a00640 1 -- 172.21.15.16:0/2952115286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8d810b470 msgr2=0x7fc8d819cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:15.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:15.759+0000 7fc8d6a00640 1 --2- 172.21.15.16:0/2952115286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8d810b470 0x7fc8d819cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:15.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:15.759+0000 7fc8d6a00640 1 -- 172.21.15.16:0/2952115286 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8d81a2480 con 0x7fc8d8105b30 2024-08-31T06:04:15.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:15.759+0000 7fc8d7400640 1 --2- 172.21.15.16:0/2952115286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8d810bdc0 0x7fc8d81a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:04:15.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:15.760+0000 7fc8d6a00640 1 --2- 172.21.15.16:0/2952115286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8d8105b30 0x7fc8d819c510 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc8c800ea60 tx=0x7fc8c800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:04:15.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:15.760+0000 7fc8cf400640 1 -- 172.21.15.16:0/2952115286 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c800cde0 con 0x7fc8d8105b30 2024-08-31T06:04:15.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:15.760+0000 7fc8dd800640 1 -- 172.21.15.16:0/2952115286 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8d81a2770 con 0x7fc8d8105b30 2024-08-31T06:04:15.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:15.760+0000 7fc8cf400640 1 -- 172.21.15.16:0/2952115286 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8c80035f0 con 0x7fc8d8105b30 2024-08-31T06:04:15.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:15.760+0000 7fc8cf400640 1 -- 172.21.15.16:0/2952115286 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c8010690 con 0x7fc8d8105b30 2024-08-31T06:04:15.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:15.760+0000 7fc8dd800640 1 -- 172.21.15.16:0/2952115286 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8d81a8ed0 con 0x7fc8d8105b30 2024-08-31T06:04:15.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:15.761+0000 7fc8dd800640 1 -- 172.21.15.16:0/2952115286 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8d8105f40 con 0x7fc8d8105b30 2024-08-31T06:04:15.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:15.764+0000 7fc8cf400640 1 -- 172.21.15.16:0/2952115286 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7fc8c80037a0 con 0x7fc8d8105b30 2024-08-31T06:04:15.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:15.765+0000 7fc8cf400640 1 --2- 172.21.15.16:0/2952115286 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc8bc0b4b80 0x7fc8bc0b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:15.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:15.765+0000 7fc8cf400640 1 -- 172.21.15.16:0/2952115286 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7fc8c80e5160 con 0x7fc8d8105b30 2024-08-31T06:04:15.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:15.765+0000 7fc8d6000640 1 --2- 172.21.15.16:0/2952115286 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc8bc0b4b80 0x7fc8bc0b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:15.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:15.766+0000 7fc8d6000640 1 --2- 172.21.15.16:0/2952115286 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc8bc0b4b80 0x7fc8bc0b7090 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4009230 tx=0x7fc8c4031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:04:15.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:15.769+0000 7fc8cf400640 1 -- 172.21.15.16:0/2952115286 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fc8c80ac5c0 con 0x7fc8d8105b30 2024-08-31T06:04:16.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:16.012+0000 7fc8dd800640 1 -- 172.21.15.16:0/2952115286 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc8d819d640 con 0x7fc8d8105b30 2024-08-31T06:04:16.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:16.013+0000 7fc8cf400640 1 -- 172.21.15.16:0/2952115286 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10530 (secure 0 0 0) 0x7fc8c80abec0 con 0x7fc8d8105b30 2024-08-31T06:04:16.015 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:04:16.016 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":13,"btime":"2024-08-31T06:04:13:840754+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24688,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24694,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24698,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/1089710405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1089710405},{"type":"v1","addr":"172.21.15.155:6837","nonce":1089710405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24712,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24724,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24737,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-08-31T06:04:16.016 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 13 2024-08-31T06:04:16.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:16.015+0000 7fc8dd800640 1 -- 172.21.15.16:0/2952115286 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc8bc0b4b80 msgr2=0x7fc8bc0b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:16.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:16.015+0000 7fc8dd800640 1 --2- 172.21.15.16:0/2952115286 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc8bc0b4b80 0x7fc8bc0b7090 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4009230 tx=0x7fc8c4031400 comp rx=0 tx=0).stop 2024-08-31T06:04:16.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:16.015+0000 7fc8dd800640 1 -- 172.21.15.16:0/2952115286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8d8105b30 msgr2=0x7fc8d819c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:16.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:16.015+0000 7fc8dd800640 1 --2- 172.21.15.16:0/2952115286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8d8105b30 0x7fc8d819c510 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc8c800ea60 tx=0x7fc8c800ef20 comp rx=0 tx=0).stop 2024-08-31T06:04:16.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:16.015+0000 7fc8dd800640 1 -- 172.21.15.16:0/2952115286 shutdown_connections 2024-08-31T06:04:16.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:16.015+0000 7fc8dd800640 1 --2- 172.21.15.16:0/2952115286 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc8bc0b4b80 0x7fc8bc0b7090 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:16.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:16.015+0000 7fc8dd800640 1 --2- 172.21.15.16:0/2952115286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8d810bdc0 0x7fc8d81a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:16.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:16.015+0000 7fc8dd800640 1 --2- 172.21.15.16:0/2952115286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8d810b470 0x7fc8d819cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:16.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:16.015+0000 7fc8dd800640 1 --2- 172.21.15.16:0/2952115286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8d8105b30 0x7fc8d819c510 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:16.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:16.015+0000 7fc8dd800640 1 -- 172.21.15.16:0/2952115286 >> 172.21.15.16:0/2952115286 conn(0x7fc8d8101600 msgr2=0x7fc8d8103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:04:16.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:16.016+0000 7fc8dd800640 1 -- 172.21.15.16:0/2952115286 shutdown_connections 2024-08-31T06:04:16.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:16.016+0000 7fc8dd800640 1 -- 172.21.15.16:0/2952115286 wait complete. 2024-08-31T06:04:16.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:16 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:16.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:16 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:16.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:16 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:16.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:16 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:16.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:16 smithi155 ceph-mon[21742]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:16.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:16 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/2952115286' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:04:16.375 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:16 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:16.375 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:16 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:16.375 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:16 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:16.375 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:16 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:16.376 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:16 smithi016 ceph-mon[17194]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:16.376 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:16 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/2952115286' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:04:16.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:16 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:16.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:16 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:16.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:16 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:16.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:16 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:16.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:16 smithi184 ceph-mon[22331]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:16.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:16 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/2952115286' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:04:16.749 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-08-31T06:04:17.152 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:04:17.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:17.822+0000 7f3540800640 1 -- 172.21.15.16:0/722485622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3538075b70 msgr2=0x7f3538075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:17.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:17.822+0000 7f3540800640 1 --2- 172.21.15.16:0/722485622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3538075b70 0x7f3538075f80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3528009860 tx=0x7f352802a340 comp rx=0 tx=0).stop 2024-08-31T06:04:17.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:17.822+0000 7f3540800640 1 -- 172.21.15.16:0/722485622 shutdown_connections 2024-08-31T06:04:17.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:17.822+0000 7f3540800640 1 --2- 172.21.15.16:0/722485622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3538075b70 0x7f3538075f80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:17.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:17.822+0000 7f3540800640 1 --2- 172.21.15.16:0/722485622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35381141f0 0x7f35381166c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:17.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:17.822+0000 7f3540800640 1 --2- 172.21.15.16:0/722485622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3538077120 0x7f3538075550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:17.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:17.822+0000 7f3540800640 1 -- 172.21.15.16:0/722485622 >> 172.21.15.16:0/722485622 conn(0x7f35380ff330 msgr2=0x7f3538101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:04:17.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:17.822+0000 7f3540800640 1 -- 172.21.15.16:0/722485622 shutdown_connections 2024-08-31T06:04:17.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:17.822+0000 7f3540800640 1 -- 172.21.15.16:0/722485622 wait complete. 2024-08-31T06:04:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:17.824+0000 7f3540800640 1 Processor -- start 2024-08-31T06:04:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:17.824+0000 7f3540800640 1 -- start start 2024-08-31T06:04:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:17.824+0000 7f3540800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3538075b70 0x7f35381a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:17.824+0000 7f3540800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3538077120 0x7f35381a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:17.824+0000 7f3540800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35381141f0 0x7f35381a60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:17.824+0000 7f3540800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3538119e30 con 0x7f3538075b70 2024-08-31T06:04:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:17.824+0000 7f3540800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3538119cb0 con 0x7f35381141f0 2024-08-31T06:04:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:17.824+0000 7f3540800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3538119fb0 con 0x7f3538077120 2024-08-31T06:04:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:17.824+0000 7f353de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3538075b70 0x7f35381a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:17.825+0000 7f353d400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3538077120 0x7f35381a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:17.825+0000 7f353de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3538075b70 0x7f35381a0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40136/0 (socket says 172.21.15.16:40136) 2024-08-31T06:04:17.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:17.825+0000 7f353de00640 1 -- 172.21.15.16:0/3085898926 learned_addr learned my addr 172.21.15.16:0/3085898926 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:04:17.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:17.825+0000 7f353e800640 1 --2- 172.21.15.16:0/3085898926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35381141f0 0x7f35381a60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:17.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:17.825+0000 7f353d400640 1 -- 172.21.15.16:0/3085898926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35381141f0 msgr2=0x7f35381a60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:17.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:17.825+0000 7f353d400640 1 --2- 172.21.15.16:0/3085898926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35381141f0 0x7f35381a60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:17.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:17.825+0000 7f353d400640 1 -- 172.21.15.16:0/3085898926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3538075b70 msgr2=0x7f35381a0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:17.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:17.825+0000 7f353d400640 1 --2- 172.21.15.16:0/3085898926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3538075b70 0x7f35381a0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:17.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:17.825+0000 7f353d400640 1 -- 172.21.15.16:0/3085898926 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35381a67d0 con 0x7f3538077120 2024-08-31T06:04:17.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:17.825+0000 7f353de00640 1 --2- 172.21.15.16:0/3085898926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3538075b70 0x7f35381a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:04:17.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:17.825+0000 7f353d400640 1 --2- 172.21.15.16:0/3085898926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3538077120 0x7f35381a0e00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f352802a860 tx=0x7f3528032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:04:17.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:17.826+0000 7f3532a00640 1 -- 172.21.15.16:0/3085898926 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3528037040 con 0x7f3538077120 2024-08-31T06:04:17.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:17.826+0000 7f3540800640 1 -- 172.21.15.16:0/3085898926 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35381a6a60 con 0x7f3538077120 2024-08-31T06:04:17.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:17.826+0000 7f3532a00640 1 -- 172.21.15.16:0/3085898926 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3528030c10 con 0x7f3538077120 2024-08-31T06:04:17.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:17.826+0000 7f3532a00640 1 -- 172.21.15.16:0/3085898926 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35280035f0 con 0x7f3538077120 2024-08-31T06:04:17.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:17.826+0000 7f3540800640 1 -- 172.21.15.16:0/3085898926 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35381ad220 con 0x7f3538077120 2024-08-31T06:04:17.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:17.827+0000 7f3540800640 1 -- 172.21.15.16:0/3085898926 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3538076a60 con 0x7f3538077120 2024-08-31T06:04:17.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:17.829+0000 7f3532a00640 1 -- 172.21.15.16:0/3085898926 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7f3528043020 con 0x7f3538077120 2024-08-31T06:04:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:17.831+0000 7f3532a00640 1 --2- 172.21.15.16:0/3085898926 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f35200b4c00 0x7f35200b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:17.831+0000 7f3532a00640 1 -- 172.21.15.16:0/3085898926 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7f3528103f40 con 0x7f3538077120 2024-08-31T06:04:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:17.831+0000 7f353de00640 1 --2- 172.21.15.16:0/3085898926 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f35200b4c00 0x7f35200b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:17.831+0000 7f353de00640 1 --2- 172.21.15.16:0/3085898926 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f35200b4c00 0x7f35200b7110 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f352c0058c0 tx=0x7f352c009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:04:17.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:17.835+0000 7f3532a00640 1 -- 172.21.15.16:0/3085898926 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f352804a060 con 0x7f3538077120 2024-08-31T06:04:17.851 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:17 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:17.851 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:17 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:17.851 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:17 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:04:17.852 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:17 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:04:17.852 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:17 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:17.852 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:17 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:04:17.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:17 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:17.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:17 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:17.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:17 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:04:17.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:17 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:04:17.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:17 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:17.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:17 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:04:17.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:17 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:17.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:17 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:17.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:17 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:04:17.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:17 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:04:17.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:17 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:17.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:17 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:04:18.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:18.044+0000 7f3540800640 1 -- 172.21.15.16:0/3085898926 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f3538112990 con 0x7f3538077120 2024-08-31T06:04:18.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:18.044+0000 7f3532a00640 1 -- 172.21.15.16:0/3085898926 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v71) ==== 84+0+65 (secure 0 0 0) 0x7f35280cb3a0 con 0x7f3538077120 2024-08-31T06:04:18.047 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:04:18.047 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-08-31T06:04:18.047 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:04:18.047 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-08-31T06:04:18.047 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-08-31T06:04:18.047 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:04:18.047 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-08-31T06:04:18.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:18.047+0000 7f3540800640 1 -- 172.21.15.16:0/3085898926 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f35200b4c00 msgr2=0x7f35200b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:18.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:18.047+0000 7f3540800640 1 --2- 172.21.15.16:0/3085898926 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f35200b4c00 0x7f35200b7110 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f352c0058c0 tx=0x7f352c009260 comp rx=0 tx=0).stop 2024-08-31T06:04:18.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:18.047+0000 7f3540800640 1 -- 172.21.15.16:0/3085898926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3538077120 msgr2=0x7f35381a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:18.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:18.047+0000 7f3540800640 1 --2- 172.21.15.16:0/3085898926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3538077120 0x7f35381a0e00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f352802a860 tx=0x7f3528032d50 comp rx=0 tx=0).stop 2024-08-31T06:04:18.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:18.047+0000 7f3540800640 1 -- 172.21.15.16:0/3085898926 shutdown_connections 2024-08-31T06:04:18.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:18.047+0000 7f3540800640 1 --2- 172.21.15.16:0/3085898926 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f35200b4c00 0x7f35200b7110 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:18.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:18.047+0000 7f3540800640 1 --2- 172.21.15.16:0/3085898926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3538077120 0x7f35381a0e00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:18.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:18.047+0000 7f3540800640 1 --2- 172.21.15.16:0/3085898926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35381141f0 0x7f35381a60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:18.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:18.047+0000 7f3540800640 1 --2- 172.21.15.16:0/3085898926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3538075b70 0x7f35381a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:18.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:18.047+0000 7f3540800640 1 -- 172.21.15.16:0/3085898926 >> 172.21.15.16:0/3085898926 conn(0x7f35380ff330 msgr2=0x7f35381100f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:04:18.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:18.047+0000 7f3540800640 1 -- 172.21.15.16:0/3085898926 shutdown_connections 2024-08-31T06:04:18.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:18.047+0000 7f3540800640 1 -- 172.21.15.16:0/3085898926 wait complete. 2024-08-31T06:04:19.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:18 smithi016 ceph-mon[17194]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:19.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:18 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/3085898926' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-08-31T06:04:19.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:18 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:19.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:18 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:19.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:18 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:19.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:18 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:19.121 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:18 smithi155 ceph-mon[21742]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:19.121 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:18 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/3085898926' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-08-31T06:04:19.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:18 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:19.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:18 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:19.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:18 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:19.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:18 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:19.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:18 smithi184 ceph-mon[22331]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:19.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:18 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/3085898926' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-08-31T06:04:19.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:18 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:19.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:18 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:19.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:18 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:19.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:18 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:19.595 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-08-31T06:04:19.595 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-08-31T06:04:19.996 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:04:20.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:20.630+0000 7fdba9000640 1 -- 172.21.15.16:0/930207713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdba4109250 msgr2=0x7fdba4109660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:20.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:20.630+0000 7fdba9000640 1 --2- 172.21.15.16:0/930207713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdba4109250 0x7fdba4109660 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdb90009900 tx=0x7fdb9002a410 comp rx=0 tx=0).stop 2024-08-31T06:04:20.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:20.630+0000 7fdba9000640 1 -- 172.21.15.16:0/930207713 shutdown_connections 2024-08-31T06:04:20.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:20.630+0000 7fdba9000640 1 --2- 172.21.15.16:0/930207713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdba4109ba0 0x7fdba41103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:20.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:20.630+0000 7fdba9000640 1 --2- 172.21.15.16:0/930207713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdba4109250 0x7fdba4109660 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:20.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:20.630+0000 7fdba9000640 1 --2- 172.21.15.16:0/930207713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdba4103910 0x7fdba4103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:20.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:20.630+0000 7fdba9000640 1 -- 172.21.15.16:0/930207713 >> 172.21.15.16:0/930207713 conn(0x7fdba40ff380 msgr2=0x7fdba41017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:04:20.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:20.630+0000 7fdba9000640 1 -- 172.21.15.16:0/930207713 shutdown_connections 2024-08-31T06:04:20.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:20.630+0000 7fdba9000640 1 -- 172.21.15.16:0/930207713 wait complete. 2024-08-31T06:04:20.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:20.631+0000 7fdba9000640 1 Processor -- start 2024-08-31T06:04:20.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:20.631+0000 7fdba9000640 1 -- start start 2024-08-31T06:04:20.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:20.631+0000 7fdba9000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdba4103910 0x7fdba419c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:20.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:20.631+0000 7fdba9000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdba4109250 0x7fdba419ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:20.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:20.632+0000 7fdba9000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdba4109ba0 0x7fdba41a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:20.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:20.632+0000 7fdba9000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdba4113860 con 0x7fdba4109ba0 2024-08-31T06:04:20.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:20.632+0000 7fdba9000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdba41136e0 con 0x7fdba4109250 2024-08-31T06:04:20.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:20.632+0000 7fdba9000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdba41139e0 con 0x7fdba4103910 2024-08-31T06:04:20.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:20.632+0000 7fdba1e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdba4103910 0x7fdba419c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:20.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:20.632+0000 7fdba2800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdba4109ba0 0x7fdba41a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:20.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:20.632+0000 7fdba1e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdba4103910 0x7fdba419c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:45008/0 (socket says 172.21.15.16:45008) 2024-08-31T06:04:20.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:20.632+0000 7fdba1e00640 1 -- 172.21.15.16:0/1037220455 learned_addr learned my addr 172.21.15.16:0/1037220455 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:04:20.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:20.632+0000 7fdba1400640 1 --2- 172.21.15.16:0/1037220455 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdba4109250 0x7fdba419ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:20.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:20.632+0000 7fdba1e00640 1 -- 172.21.15.16:0/1037220455 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdba4109250 msgr2=0x7fdba419ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:20.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:20.632+0000 7fdba1e00640 1 --2- 172.21.15.16:0/1037220455 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdba4109250 0x7fdba419ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:20.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:20.632+0000 7fdba1e00640 1 -- 172.21.15.16:0/1037220455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdba4109ba0 msgr2=0x7fdba41a1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:20.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:20.632+0000 7fdba1e00640 1 --2- 172.21.15.16:0/1037220455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdba4109ba0 0x7fdba41a1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:20.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:20.632+0000 7fdba1e00640 1 -- 172.21.15.16:0/1037220455 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdba41a23e0 con 0x7fdba4103910 2024-08-31T06:04:20.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:20.632+0000 7fdba2800640 1 --2- 172.21.15.16:0/1037220455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdba4109ba0 0x7fdba41a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:04:20.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:20.633+0000 7fdba1e00640 1 --2- 172.21.15.16:0/1037220455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdba4103910 0x7fdba419c470 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdb9400e9f0 tx=0x7fdb9400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:04:20.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:20.633+0000 7fdb9aa00640 1 -- 172.21.15.16:0/1037220455 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9400cd80 con 0x7fdba4103910 2024-08-31T06:04:20.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:20.633+0000 7fdb9aa00640 1 -- 172.21.15.16:0/1037220455 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb940108e0 con 0x7fdba4103910 2024-08-31T06:04:20.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:20.633+0000 7fdba9000640 1 -- 172.21.15.16:0/1037220455 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdba41a26d0 con 0x7fdba4103910 2024-08-31T06:04:20.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:20.633+0000 7fdb9aa00640 1 -- 172.21.15.16:0/1037220455 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb94018980 con 0x7fdba4103910 2024-08-31T06:04:20.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:20.633+0000 7fdba9000640 1 -- 172.21.15.16:0/1037220455 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdba41a8e30 con 0x7fdba4103910 2024-08-31T06:04:20.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:20.634+0000 7fdba9000640 1 -- 172.21.15.16:0/1037220455 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdba4103d20 con 0x7fdba4103910 2024-08-31T06:04:20.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:20.637+0000 7fdb9aa00640 1 -- 172.21.15.16:0/1037220455 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7fdb94010430 con 0x7fdba4103910 2024-08-31T06:04:20.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:20.638+0000 7fdb9aa00640 1 --2- 172.21.15.16:0/1037220455 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdb880b4b80 0x7fdb880b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:20.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:20.638+0000 7fdb9aa00640 1 -- 172.21.15.16:0/1037220455 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7fdb940e5660 con 0x7fdba4103910 2024-08-31T06:04:20.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:20.639+0000 7fdba1400640 1 --2- 172.21.15.16:0/1037220455 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdb880b4b80 0x7fdb880b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:20.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:20.639+0000 7fdba1400640 1 --2- 172.21.15.16:0/1037220455 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdb880b4b80 0x7fdb880b7090 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdb90009230 tx=0x7fdb90031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:04:20.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:20.640+0000 7fdb9aa00640 1 -- 172.21.15.16:0/1037220455 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fdb940ac470 con 0x7fdba4103910 2024-08-31T06:04:20.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:20.856+0000 7fdba9000640 1 -- 172.21.15.16:0/1037220455 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fdba41a9030 con 0x7fdba4103910 2024-08-31T06:04:21.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:20 smithi155 ceph-mon[21742]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:21.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:20 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:21.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:20 smithi016 ceph-mon[17194]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:21.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:20 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:21.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:20 smithi184 ceph-mon[22331]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:21.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:20 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:21.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:21.831+0000 7fdb9aa00640 1 -- 172.21.15.16:0/1037220455 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v72) ==== 138+0+0 (secure 0 0 0) 0x7fdb940b0c30 con 0x7fdba4103910 2024-08-31T06:04:21.835 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-08-31T06:04:21.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:21.837+0000 7fdba9000640 1 -- 172.21.15.16:0/1037220455 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdb880b4b80 msgr2=0x7fdb880b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:21.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:21.837+0000 7fdba9000640 1 --2- 172.21.15.16:0/1037220455 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdb880b4b80 0x7fdb880b7090 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdb90009230 tx=0x7fdb90031400 comp rx=0 tx=0).stop 2024-08-31T06:04:21.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:21.837+0000 7fdba9000640 1 -- 172.21.15.16:0/1037220455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdba4103910 msgr2=0x7fdba419c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:21.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:21.837+0000 7fdba9000640 1 --2- 172.21.15.16:0/1037220455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdba4103910 0x7fdba419c470 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdb9400e9f0 tx=0x7fdb9400eeb0 comp rx=0 tx=0).stop 2024-08-31T06:04:21.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:21.837+0000 7fdba9000640 1 -- 172.21.15.16:0/1037220455 shutdown_connections 2024-08-31T06:04:21.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:21.837+0000 7fdba9000640 1 --2- 172.21.15.16:0/1037220455 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdb880b4b80 0x7fdb880b7090 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:21.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:21.837+0000 7fdba9000640 1 --2- 172.21.15.16:0/1037220455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdba4109ba0 0x7fdba41a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:21.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:21.837+0000 7fdba9000640 1 --2- 172.21.15.16:0/1037220455 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdba4109250 0x7fdba419ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:21.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:21.837+0000 7fdba9000640 1 --2- 172.21.15.16:0/1037220455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdba4103910 0x7fdba419c470 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:21.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:21.837+0000 7fdba9000640 1 -- 172.21.15.16:0/1037220455 >> 172.21.15.16:0/1037220455 conn(0x7fdba40ff380 msgr2=0x7fdba410e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:04:21.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:21.838+0000 7fdba9000640 1 -- 172.21.15.16:0/1037220455 shutdown_connections 2024-08-31T06:04:21.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:21.838+0000 7fdba9000640 1 -- 172.21.15.16:0/1037220455 wait complete. 2024-08-31T06:04:22.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:21 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:22.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:21 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:04:22.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:21 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:04:22.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:21 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:22.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:21 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/1037220455' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-08-31T06:04:22.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:21 smithi155 ceph-mon[21742]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-08-31T06:04:22.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:21 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:22.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:21 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:04:22.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:21 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:04:22.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:21 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:22.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:21 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/1037220455' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-08-31T06:04:22.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:21 smithi016 ceph-mon[17194]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-08-31T06:04:22.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:21 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:22.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:21 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:04:22.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:21 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:04:22.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:21 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:04:22.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:21 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/1037220455' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-08-31T06:04:22.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:21 smithi184 ceph-mon[22331]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-08-31T06:04:22.796 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-08-31T06:04:23.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:22 smithi155 ceph-mon[21742]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:23.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:22 smithi155 ceph-mon[21742]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-08-31T06:04:23.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:22 smithi155 ceph-mon[21742]: osdmap e72: 12 total, 12 up, 12 in 2024-08-31T06:04:23.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:22 smithi016 ceph-mon[17194]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:23.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:22 smithi016 ceph-mon[17194]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-08-31T06:04:23.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:22 smithi016 ceph-mon[17194]: osdmap e72: 12 total, 12 up, 12 in 2024-08-31T06:04:23.194 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:04:23.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:22 smithi184 ceph-mon[22331]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:23.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:22 smithi184 ceph-mon[22331]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-08-31T06:04:23.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:22 smithi184 ceph-mon[22331]: osdmap e72: 12 total, 12 up, 12 in 2024-08-31T06:04:23.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:23.898+0000 7f828ca00640 1 -- 172.21.15.16:0/1405190985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f828810b270 msgr2=0x7f828810b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:23.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:23.898+0000 7f828ca00640 1 --2- 172.21.15.16:0/1405190985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f828810b270 0x7f828810b680 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8270009900 tx=0x7f827002a410 comp rx=0 tx=0).stop 2024-08-31T06:04:23.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:23.898+0000 7f828ca00640 1 -- 172.21.15.16:0/1405190985 shutdown_connections 2024-08-31T06:04:23.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:23.898+0000 7f828ca00640 1 --2- 172.21.15.16:0/1405190985 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f828810bbc0 0x7f82881123f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:23.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:23.898+0000 7f828ca00640 1 --2- 172.21.15.16:0/1405190985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f828810b270 0x7f828810b680 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:23.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:23.898+0000 7f828ca00640 1 --2- 172.21.15.16:0/1405190985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8288105930 0x7f8288105d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:23.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:23.898+0000 7f828ca00640 1 -- 172.21.15.16:0/1405190985 >> 172.21.15.16:0/1405190985 conn(0x7f8288101380 msgr2=0x7f82881037d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:04:23.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:23.898+0000 7f828ca00640 1 -- 172.21.15.16:0/1405190985 shutdown_connections 2024-08-31T06:04:23.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:23.898+0000 7f828ca00640 1 -- 172.21.15.16:0/1405190985 wait complete. 2024-08-31T06:04:23.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:23.899+0000 7f828ca00640 1 Processor -- start 2024-08-31T06:04:23.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:23.899+0000 7f828ca00640 1 -- start start 2024-08-31T06:04:23.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:23.900+0000 7f828ca00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8288105930 0x7f828819c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:23.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:23.900+0000 7f828ca00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f828810b270 0x7f828819c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:23.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:23.900+0000 7f828ca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f828810bbc0 0x7f82881a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:23.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:23.900+0000 7f828ca00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8288115880 con 0x7f828810bbc0 2024-08-31T06:04:23.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:23.900+0000 7f828ca00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8288115700 con 0x7f8288105930 2024-08-31T06:04:23.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:23.900+0000 7f828ca00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8288115a00 con 0x7f828810b270 2024-08-31T06:04:23.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:23.900+0000 7f8287200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f828810bbc0 0x7f82881a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:23.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:23.900+0000 7f8286800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8288105930 0x7f828819c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:23.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:23.900+0000 7f8287200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f828810bbc0 0x7f82881a1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40172/0 (socket says 172.21.15.16:40172) 2024-08-31T06:04:23.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:23.900+0000 7f8287200640 1 -- 172.21.15.16:0/3553704390 learned_addr learned my addr 172.21.15.16:0/3553704390 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:04:23.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:23.900+0000 7f827fe00640 1 --2- 172.21.15.16:0/3553704390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f828810b270 0x7f828819c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:23.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:23.901+0000 7f8287200640 1 -- 172.21.15.16:0/3553704390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f828810b270 msgr2=0x7f828819c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:23.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:23.901+0000 7f8287200640 1 --2- 172.21.15.16:0/3553704390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f828810b270 0x7f828819c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:23.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:23.901+0000 7f8287200640 1 -- 172.21.15.16:0/3553704390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8288105930 msgr2=0x7f828819c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:23.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:23.901+0000 7f8287200640 1 --2- 172.21.15.16:0/3553704390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8288105930 0x7f828819c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:23.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:23.901+0000 7f8287200640 1 -- 172.21.15.16:0/3553704390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82881a2200 con 0x7f828810bbc0 2024-08-31T06:04:23.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:23.901+0000 7f8287200640 1 --2- 172.21.15.16:0/3553704390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f828810bbc0 0x7f82881a1ad0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f828000eb10 tx=0x7f828000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:04:23.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:23.901+0000 7f827f400640 1 -- 172.21.15.16:0/3553704390 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8280004b20 con 0x7f828810bbc0 2024-08-31T06:04:23.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:23.902+0000 7f828ca00640 1 -- 172.21.15.16:0/3553704390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82881a24f0 con 0x7f828810bbc0 2024-08-31T06:04:23.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:23.902+0000 7f827f400640 1 -- 172.21.15.16:0/3553704390 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8280004cc0 con 0x7f828810bbc0 2024-08-31T06:04:23.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:23.902+0000 7f827f400640 1 -- 172.21.15.16:0/3553704390 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8280010690 con 0x7f828810bbc0 2024-08-31T06:04:23.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:23.902+0000 7f828ca00640 1 -- 172.21.15.16:0/3553704390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82881a8c50 con 0x7f828810bbc0 2024-08-31T06:04:23.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:23.903+0000 7f828ca00640 1 -- 172.21.15.16:0/3553704390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8288105d40 con 0x7f828810bbc0 2024-08-31T06:04:23.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:23.906+0000 7f827f400640 1 -- 172.21.15.16:0/3553704390 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7f8280005000 con 0x7f828810bbc0 2024-08-31T06:04:23.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:23.907+0000 7f827f400640 1 --2- 172.21.15.16:0/3553704390 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f826c0b4b80 0x7f826c0b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:23.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:23.907+0000 7f827f400640 1 -- 172.21.15.16:0/3553704390 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7712+0+0 (secure 0 0 0) 0x7f82800e5680 con 0x7f828810bbc0 2024-08-31T06:04:23.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:23.907+0000 7f8286800640 1 --2- 172.21.15.16:0/3553704390 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f826c0b4b80 0x7f826c0b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:23.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:23.908+0000 7f8286800640 1 --2- 172.21.15.16:0/3553704390 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f826c0b4b80 0x7f826c0b7090 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8278005d00 tx=0x7f827800b410 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:04:23.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:23.910+0000 7f827f400640 1 -- 172.21.15.16:0/3553704390 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f8280056130 con 0x7f828810bbc0 2024-08-31T06:04:24.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:24.104+0000 7f828ca00640 1 -- 172.21.15.16:0/3553704390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f828819d3c0 con 0x7f828810bbc0 2024-08-31T06:04:24.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:23 smithi016 ceph-mon[17194]: osdmap e73: 12 total, 12 up, 12 in 2024-08-31T06:04:24.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:23 smithi155 ceph-mon[21742]: osdmap e73: 12 total, 12 up, 12 in 2024-08-31T06:04:24.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:23 smithi184 ceph-mon[22331]: osdmap e73: 12 total, 12 up, 12 in 2024-08-31T06:04:24.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:24.851+0000 7f827f400640 1 -- 172.21.15.16:0/3553704390 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v75) ==== 170+0+0 (secure 0 0 0) 0x7f828819d3c0 con 0x7f828810bbc0 2024-08-31T06:04:24.854 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-08-31T06:04:24.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:24.855+0000 7f828ca00640 1 -- 172.21.15.16:0/3553704390 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f826c0b4b80 msgr2=0x7f826c0b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:24.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:24.855+0000 7f828ca00640 1 --2- 172.21.15.16:0/3553704390 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f826c0b4b80 0x7f826c0b7090 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8278005d00 tx=0x7f827800b410 comp rx=0 tx=0).stop 2024-08-31T06:04:24.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:24.855+0000 7f828ca00640 1 -- 172.21.15.16:0/3553704390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f828810bbc0 msgr2=0x7f82881a1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:24.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:24.855+0000 7f828ca00640 1 --2- 172.21.15.16:0/3553704390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f828810bbc0 0x7f82881a1ad0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f828000eb10 tx=0x7f828000c2d0 comp rx=0 tx=0).stop 2024-08-31T06:04:24.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:24.855+0000 7f828ca00640 1 -- 172.21.15.16:0/3553704390 shutdown_connections 2024-08-31T06:04:24.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:24.855+0000 7f828ca00640 1 --2- 172.21.15.16:0/3553704390 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f826c0b4b80 0x7f826c0b7090 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:24.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:24.855+0000 7f828ca00640 1 --2- 172.21.15.16:0/3553704390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f828810b270 0x7f828819c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:24.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:24.855+0000 7f828ca00640 1 --2- 172.21.15.16:0/3553704390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f828810bbc0 0x7f82881a1ad0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:24.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:24.855+0000 7f828ca00640 1 --2- 172.21.15.16:0/3553704390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8288105930 0x7f828819c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:24.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:24.855+0000 7f828ca00640 1 -- 172.21.15.16:0/3553704390 >> 172.21.15.16:0/3553704390 conn(0x7f8288101380 msgr2=0x7f82881037a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:04:24.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:24.856+0000 7f828ca00640 1 -- 172.21.15.16:0/3553704390 shutdown_connections 2024-08-31T06:04:24.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:24.856+0000 7f828ca00640 1 -- 172.21.15.16:0/3553704390 wait complete. 2024-08-31T06:04:25.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:24 smithi155 ceph-mon[21742]: pgmap v271: 33 pgs: 1 creating+activating, 9 creating+peering, 8 active+clean, 15 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:25.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:24 smithi155 ceph-mon[21742]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-31T06:04:25.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:24 smithi155 ceph-mon[21742]: osdmap e74: 12 total, 12 up, 12 in 2024-08-31T06:04:25.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:24 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/3553704390' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-08-31T06:04:25.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:24 smithi016 ceph-mon[17194]: pgmap v271: 33 pgs: 1 creating+activating, 9 creating+peering, 8 active+clean, 15 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:25.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:24 smithi016 ceph-mon[17194]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-31T06:04:25.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:24 smithi016 ceph-mon[17194]: osdmap e74: 12 total, 12 up, 12 in 2024-08-31T06:04:25.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:24 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/3553704390' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-08-31T06:04:25.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:24 smithi184 ceph-mon[22331]: pgmap v271: 33 pgs: 1 creating+activating, 9 creating+peering, 8 active+clean, 15 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:25.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:24 smithi184 ceph-mon[22331]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-31T06:04:25.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:24 smithi184 ceph-mon[22331]: osdmap e74: 12 total, 12 up, 12 in 2024-08-31T06:04:25.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:24 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/3553704390' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-08-31T06:04:25.774 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-08-31T06:04:26.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:25 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/3553704390' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-08-31T06:04:26.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:25 smithi155 ceph-mon[21742]: osdmap e75: 12 total, 12 up, 12 in 2024-08-31T06:04:26.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:25 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/3553704390' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-08-31T06:04:26.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:25 smithi016 ceph-mon[17194]: osdmap e75: 12 total, 12 up, 12 in 2024-08-31T06:04:26.179 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:04:26.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:25 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/3553704390' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-08-31T06:04:26.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:25 smithi184 ceph-mon[22331]: osdmap e75: 12 total, 12 up, 12 in 2024-08-31T06:04:26.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:26.838+0000 7efd3ec00640 1 -- 172.21.15.16:0/2187875989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd3810fcf0 msgr2=0x7efd38112110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:26.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:26.838+0000 7efd3ec00640 1 --2- 172.21.15.16:0/2187875989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd3810fcf0 0x7efd38112110 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7efd30009ca0 tx=0x7efd3002a8c0 comp rx=0 tx=0).stop 2024-08-31T06:04:26.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:26.838+0000 7efd3ec00640 1 -- 172.21.15.16:0/2187875989 shutdown_connections 2024-08-31T06:04:26.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:26.838+0000 7efd3ec00640 1 --2- 172.21.15.16:0/2187875989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd38075e40 0x7efd3810f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:26.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:26.838+0000 7efd3ec00640 1 --2- 172.21.15.16:0/2187875989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd3810fcf0 0x7efd38112110 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:26.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:26.838+0000 7efd3ec00640 1 --2- 172.21.15.16:0/2187875989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd380754f0 0x7efd38075900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:26.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:26.838+0000 7efd3ec00640 1 -- 172.21.15.16:0/2187875989 >> 172.21.15.16:0/2187875989 conn(0x7efd380ff180 msgr2=0x7efd381015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:04:26.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:26.839+0000 7efd3ec00640 1 -- 172.21.15.16:0/2187875989 shutdown_connections 2024-08-31T06:04:26.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:26.839+0000 7efd3ec00640 1 -- 172.21.15.16:0/2187875989 wait complete. 2024-08-31T06:04:26.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:26.840+0000 7efd3ec00640 1 Processor -- start 2024-08-31T06:04:26.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:26.840+0000 7efd3ec00640 1 -- start start 2024-08-31T06:04:26.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:26.840+0000 7efd3ec00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd380754f0 0x7efd3819c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:26.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:26.840+0000 7efd3ec00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd38075e40 0x7efd3819c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:26.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:26.840+0000 7efd3ec00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd3810fcf0 0x7efd381a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:26.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:26.840+0000 7efd3ec00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efd381158b0 con 0x7efd380754f0 2024-08-31T06:04:26.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:26.840+0000 7efd3ec00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efd38115730 con 0x7efd3810fcf0 2024-08-31T06:04:26.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:26.840+0000 7efd3ec00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efd38115a30 con 0x7efd38075e40 2024-08-31T06:04:26.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:26.840+0000 7efd3d800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd380754f0 0x7efd3819c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:26.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:26.840+0000 7efd3d800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd380754f0 0x7efd3819c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56542/0 (socket says 172.21.15.16:56542) 2024-08-31T06:04:26.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:26.840+0000 7efd3d800640 1 -- 172.21.15.16:0/75309579 learned_addr learned my addr 172.21.15.16:0/75309579 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:04:26.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:26.841+0000 7efd3e200640 1 --2- 172.21.15.16:0/75309579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd3810fcf0 0x7efd381a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:26.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:26.841+0000 7efd3ce00640 1 --2- 172.21.15.16:0/75309579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd38075e40 0x7efd3819c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:26.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:26.841+0000 7efd3d800640 1 -- 172.21.15.16:0/75309579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd38075e40 msgr2=0x7efd3819c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:26.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:26.841+0000 7efd3d800640 1 --2- 172.21.15.16:0/75309579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd38075e40 0x7efd3819c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:26.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:26.841+0000 7efd3d800640 1 -- 172.21.15.16:0/75309579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd3810fcf0 msgr2=0x7efd381a1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:26.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:26.841+0000 7efd3d800640 1 --2- 172.21.15.16:0/75309579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd3810fcf0 0x7efd381a1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:26.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:26.841+0000 7efd3d800640 1 -- 172.21.15.16:0/75309579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd381a21d0 con 0x7efd380754f0 2024-08-31T06:04:26.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:26.841+0000 7efd3d800640 1 --2- 172.21.15.16:0/75309579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd380754f0 0x7efd3819c260 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efd2c00e9f0 tx=0x7efd2c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:04:26.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:26.842+0000 7efd36000640 1 -- 172.21.15.16:0/75309579 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2c00cd80 con 0x7efd380754f0 2024-08-31T06:04:26.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:26.842+0000 7efd3ec00640 1 -- 172.21.15.16:0/75309579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd381a24c0 con 0x7efd380754f0 2024-08-31T06:04:26.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:26.842+0000 7efd36000640 1 -- 172.21.15.16:0/75309579 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd2c0108e0 con 0x7efd380754f0 2024-08-31T06:04:26.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:26.842+0000 7efd36000640 1 -- 172.21.15.16:0/75309579 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2c018980 con 0x7efd380754f0 2024-08-31T06:04:26.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:26.842+0000 7efd3ec00640 1 -- 172.21.15.16:0/75309579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd381a8c20 con 0x7efd380754f0 2024-08-31T06:04:26.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:26.843+0000 7efd3ec00640 1 -- 172.21.15.16:0/75309579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd38075900 con 0x7efd380754f0 2024-08-31T06:04:26.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:26.845+0000 7efd36000640 1 -- 172.21.15.16:0/75309579 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7efd2c010430 con 0x7efd380754f0 2024-08-31T06:04:26.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:26.847+0000 7efd36000640 1 --2- 172.21.15.16:0/75309579 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7efd200b4b80 0x7efd200b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:26.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:26.847+0000 7efd36000640 1 -- 172.21.15.16:0/75309579 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8094+0+0 (secure 0 0 0) 0x7efd2c0e5470 con 0x7efd380754f0 2024-08-31T06:04:26.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:26.847+0000 7efd3ce00640 1 --2- 172.21.15.16:0/75309579 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7efd200b4b80 0x7efd200b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:26.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:26.848+0000 7efd3ce00640 1 --2- 172.21.15.16:0/75309579 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7efd200b4b80 0x7efd200b7090 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efd381120e0 tx=0x7efd24005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:04:26.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:26.851+0000 7efd36000640 1 -- 172.21.15.16:0/75309579 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7efd2c0abf90 con 0x7efd380754f0 2024-08-31T06:04:26.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:26 smithi016 ceph-mon[17194]: pgmap v274: 97 pgs: 1 creating+activating, 21 creating+peering, 23 active+clean, 52 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:26.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:26 smithi016 ceph-mon[17194]: osdmap e76: 12 total, 12 up, 12 in 2024-08-31T06:04:27.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:27.097+0000 7efd3ec00640 1 -- 172.21.15.16:0/75309579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7efd381a8f10 con 0x7efd380754f0 2024-08-31T06:04:27.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:26 smithi155 ceph-mon[21742]: pgmap v274: 97 pgs: 1 creating+activating, 21 creating+peering, 23 active+clean, 52 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:27.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:26 smithi155 ceph-mon[21742]: osdmap e76: 12 total, 12 up, 12 in 2024-08-31T06:04:27.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:26 smithi184 ceph-mon[22331]: pgmap v274: 97 pgs: 1 creating+activating, 21 creating+peering, 23 active+clean, 52 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:27.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:26 smithi184 ceph-mon[22331]: osdmap e76: 12 total, 12 up, 12 in 2024-08-31T06:04:27.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:27.875+0000 7efd36000640 1 -- 172.21.15.16:0/75309579 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7efd2c0b0750 con 0x7efd380754f0 2024-08-31T06:04:27.878 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 2 and data pool 3 2024-08-31T06:04:27.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:27.878+0000 7efd3ec00640 1 -- 172.21.15.16:0/75309579 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7efd200b4b80 msgr2=0x7efd200b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:27.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:27.878+0000 7efd3ec00640 1 --2- 172.21.15.16:0/75309579 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7efd200b4b80 0x7efd200b7090 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efd381120e0 tx=0x7efd24005f90 comp rx=0 tx=0).stop 2024-08-31T06:04:27.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:27.879+0000 7efd3ec00640 1 -- 172.21.15.16:0/75309579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd380754f0 msgr2=0x7efd3819c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:27.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:27.879+0000 7efd3ec00640 1 --2- 172.21.15.16:0/75309579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd380754f0 0x7efd3819c260 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efd2c00e9f0 tx=0x7efd2c00eeb0 comp rx=0 tx=0).stop 2024-08-31T06:04:27.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:27.879+0000 7efd3ec00640 1 -- 172.21.15.16:0/75309579 shutdown_connections 2024-08-31T06:04:27.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:27.879+0000 7efd3ec00640 1 --2- 172.21.15.16:0/75309579 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7efd200b4b80 0x7efd200b7090 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:27.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:27.879+0000 7efd3ec00640 1 --2- 172.21.15.16:0/75309579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd38075e40 0x7efd3819c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:27.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:27.879+0000 7efd3ec00640 1 --2- 172.21.15.16:0/75309579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd3810fcf0 0x7efd381a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:27.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:27.879+0000 7efd3ec00640 1 --2- 172.21.15.16:0/75309579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd380754f0 0x7efd3819c260 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:27.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:27.879+0000 7efd3ec00640 1 -- 172.21.15.16:0/75309579 >> 172.21.15.16:0/75309579 conn(0x7efd380ff180 msgr2=0x7efd38109800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:04:27.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:27.879+0000 7efd3ec00640 1 -- 172.21.15.16:0/75309579 shutdown_connections 2024-08-31T06:04:27.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:27.879+0000 7efd3ec00640 1 -- 172.21.15.16:0/75309579 wait complete. 2024-08-31T06:04:28.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:27 smithi016 ceph-mon[17194]: osdmap e77: 12 total, 12 up, 12 in 2024-08-31T06:04:28.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:27 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/75309579' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-08-31T06:04:28.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:27 smithi016 ceph-mon[17194]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-08-31T06:04:28.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:27 smithi184 ceph-mon[22331]: osdmap e77: 12 total, 12 up, 12 in 2024-08-31T06:04:28.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:27 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/75309579' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-08-31T06:04:28.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:27 smithi184 ceph-mon[22331]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-08-31T06:04:28.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:27 smithi155 ceph-mon[21742]: osdmap e77: 12 total, 12 up, 12 in 2024-08-31T06:04:28.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:27 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/75309579' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-08-31T06:04:28.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:27 smithi155 ceph-mon[21742]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-08-31T06:04:28.740 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph osd dump --format=json 2024-08-31T06:04:29.126 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:04:29.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:28 smithi016 ceph-mon[17194]: pgmap v277: 97 pgs: 1 creating+activating, 12 creating+peering, 44 active+clean, 40 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:29.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:28 smithi016 ceph-mon[17194]: osdmap e78: 12 total, 12 up, 12 in 2024-08-31T06:04:29.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:28 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/75309579' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-08-31T06:04:29.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:28 smithi016 ceph-mon[17194]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-08-31T06:04:29.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:28 smithi016 ceph-mon[17194]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-08-31T06:04:29.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:28 smithi184 ceph-mon[22331]: pgmap v277: 97 pgs: 1 creating+activating, 12 creating+peering, 44 active+clean, 40 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:29.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:28 smithi184 ceph-mon[22331]: osdmap e78: 12 total, 12 up, 12 in 2024-08-31T06:04:29.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:28 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/75309579' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-08-31T06:04:29.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:28 smithi184 ceph-mon[22331]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-08-31T06:04:29.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:28 smithi184 ceph-mon[22331]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-08-31T06:04:29.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:28 smithi155 ceph-mon[21742]: pgmap v277: 97 pgs: 1 creating+activating, 12 creating+peering, 44 active+clean, 40 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:04:29.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:28 smithi155 ceph-mon[21742]: osdmap e78: 12 total, 12 up, 12 in 2024-08-31T06:04:29.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:28 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/75309579' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-08-31T06:04:29.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:28 smithi155 ceph-mon[21742]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-08-31T06:04:29.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:28 smithi155 ceph-mon[21742]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-08-31T06:04:29.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:29.788+0000 7f1197200640 1 -- 172.21.15.16:0/2580517150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f119010fe80 msgr2=0x7f11901122c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:29.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:29.788+0000 7f1197200640 1 --2- 172.21.15.16:0/2580517150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f119010fe80 0x7f11901122c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1188009ca0 tx=0x7f118802a8c0 comp rx=0 tx=0).stop 2024-08-31T06:04:29.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:29.788+0000 7f1197200640 1 -- 172.21.15.16:0/2580517150 shutdown_connections 2024-08-31T06:04:29.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:29.788+0000 7f1197200640 1 --2- 172.21.15.16:0/2580517150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f119010fe80 0x7f11901122c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:29.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:29.788+0000 7f1197200640 1 --2- 172.21.15.16:0/2580517150 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1190075ea0 0x7f119010f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:29.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:29.788+0000 7f1197200640 1 --2- 172.21.15.16:0/2580517150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1190075550 0x7f1190075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:29.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:29.788+0000 7f1197200640 1 -- 172.21.15.16:0/2580517150 >> 172.21.15.16:0/2580517150 conn(0x7f11900ff3b0 msgr2=0x7f1190101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:04:29.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:29.788+0000 7f1197200640 1 -- 172.21.15.16:0/2580517150 shutdown_connections 2024-08-31T06:04:29.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:29.788+0000 7f1197200640 1 -- 172.21.15.16:0/2580517150 wait complete. 2024-08-31T06:04:29.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:29.789+0000 7f1197200640 1 Processor -- start 2024-08-31T06:04:29.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:29.790+0000 7f1197200640 1 -- start start 2024-08-31T06:04:29.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:29.790+0000 7f1197200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1190075550 0x7f119019c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:29.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:29.790+0000 7f1197200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1190075ea0 0x7f119019c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:29.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:29.790+0000 7f1197200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f119010fe80 0x7f11901a1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:29.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:29.790+0000 7f1197200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1190115a80 con 0x7f1190075ea0 2024-08-31T06:04:29.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:29.790+0000 7f1197200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1190115900 con 0x7f119010fe80 2024-08-31T06:04:29.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:29.790+0000 7f1197200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1190115c00 con 0x7f1190075550 2024-08-31T06:04:29.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:29.791+0000 7f118f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1190075ea0 0x7f119019c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:29.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:29.791+0000 7f118fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1190075550 0x7f119019c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:29.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:29.791+0000 7f118f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1190075ea0 0x7f119019c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56556/0 (socket says 172.21.15.16:56556) 2024-08-31T06:04:29.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:29.791+0000 7f118f400640 1 -- 172.21.15.16:0/74495822 learned_addr learned my addr 172.21.15.16:0/74495822 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:04:29.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:29.791+0000 7f1195200640 1 --2- 172.21.15.16:0/74495822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f119010fe80 0x7f11901a1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:29.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:29.791+0000 7f118fe00640 1 -- 172.21.15.16:0/74495822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f119010fe80 msgr2=0x7f11901a1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:29.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:29.791+0000 7f118fe00640 1 --2- 172.21.15.16:0/74495822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f119010fe80 0x7f11901a1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:29.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:29.791+0000 7f118fe00640 1 -- 172.21.15.16:0/74495822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1190075ea0 msgr2=0x7f119019c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:29.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:29.791+0000 7f118fe00640 1 --2- 172.21.15.16:0/74495822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1190075ea0 0x7f119019c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:29.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:29.791+0000 7f118fe00640 1 -- 172.21.15.16:0/74495822 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11901a22e0 con 0x7f1190075550 2024-08-31T06:04:29.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:29.791+0000 7f118f400640 1 --2- 172.21.15.16:0/74495822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1190075ea0 0x7f119019c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:04:29.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:29.791+0000 7f118fe00640 1 --2- 172.21.15.16:0/74495822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1190075550 0x7f119019c370 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f118000e9f0 tx=0x7f118000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:04:29.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:29.792+0000 7f118cc00640 1 -- 172.21.15.16:0/74495822 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118000ccc0 con 0x7f1190075550 2024-08-31T06:04:29.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:29.792+0000 7f118cc00640 1 -- 172.21.15.16:0/74495822 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11800108e0 con 0x7f1190075550 2024-08-31T06:04:29.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:29.792+0000 7f1197200640 1 -- 172.21.15.16:0/74495822 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11901a25d0 con 0x7f1190075550 2024-08-31T06:04:29.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:29.792+0000 7f118cc00640 1 -- 172.21.15.16:0/74495822 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1180018880 con 0x7f1190075550 2024-08-31T06:04:29.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:29.792+0000 7f1197200640 1 -- 172.21.15.16:0/74495822 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11901a8d30 con 0x7f1190075550 2024-08-31T06:04:29.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:29.793+0000 7f1197200640 1 -- 172.21.15.16:0/74495822 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1190076a60 con 0x7f1190075550 2024-08-31T06:04:29.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:29.796+0000 7f118cc00640 1 -- 172.21.15.16:0/74495822 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7f1180010430 con 0x7f1190075550 2024-08-31T06:04:29.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:29.797+0000 7f118cc00640 1 --2- 172.21.15.16:0/74495822 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f11740b4b80 0x7f11740b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:29.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:29.797+0000 7f118cc00640 1 -- 172.21.15.16:0/74495822 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7f11800e59f0 con 0x7f1190075550 2024-08-31T06:04:29.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:29.797+0000 7f118f400640 1 --2- 172.21.15.16:0/74495822 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f11740b4b80 0x7f11740b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:29.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:29.798+0000 7f118f400640 1 --2- 172.21.15.16:0/74495822 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f11740b4b80 0x7f11740b7090 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f117c0095f0 tx=0x7f117c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:04:29.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:29.799+0000 7f118cc00640 1 -- 172.21.15.16:0/74495822 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f11800ac490 con 0x7f1190075550 2024-08-31T06:04:30.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:30.006+0000 7f1197200640 1 -- 172.21.15.16:0/74495822 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1190075960 con 0x7f1190075550 2024-08-31T06:04:30.009 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:29 smithi016 ceph-mon[17194]: mds.? [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] up:active 2024-08-31T06:04:30.009 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:29 smithi016 ceph-mon[17194]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-31T06:04:30.009 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:29 smithi016 ceph-mon[17194]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-31T06:04:30.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:30.007+0000 7f118cc00640 1 -- 172.21.15.16:0/74495822 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22608 (secure 0 0 0) 0x7f11800b0c50 con 0x7f1190075550 2024-08-31T06:04:30.011 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:04:30.011 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":78,"fsid":"673e4aa0-675d-11ef-bcd4-c7b262605968","created":"2024-08-31T05:55:09.632536+0000","modified":"2024-08-31T06:04:27.098694+0000","last_up_change":"2024-08-31T06:03:27.893697+0000","last_in_change":"2024-08-31T06:03:05.664945+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-31T05:59:23.642270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-31T06:04:20.871835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-31T06:04:24.119499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e349dae-6aa5-4b6d-9107-a945655e11bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6803","nonce":1725037400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6805","nonce":1725037400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6809","nonce":1725037400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6807","nonce":1725037400}]},"public_addr":"172.21.15.16:6803/1725037400","cluster_addr":"172.21.15.16:6805/1725037400","heartbeat_back_addr":"172.21.15.16:6809/1725037400","heartbeat_front_addr":"172.21.15.16:6807/1725037400","state":["exists","up"]},{"osd":1,"uuid":"bd645dc3-93b5-40af-89d8-08ccec22e50d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6801","nonce":4241618440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6803","nonce":4241618440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6807","nonce":4241618440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6805","nonce":4241618440}]},"public_addr":"172.21.15.155:6801/4241618440","cluster_addr":"172.21.15.155:6803/4241618440","heartbeat_back_addr":"172.21.15.155:6807/4241618440","heartbeat_front_addr":"172.21.15.155:6805/4241618440","state":["exists","up"]},{"osd":2,"uuid":"c69c8c19-32f8-400a-86d4-0f4ce240a7de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6801","nonce":2193296422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6803","nonce":2193296422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6807","nonce":2193296422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6805","nonce":2193296422}]},"public_addr":"172.21.15.184:6801/2193296422","cluster_addr":"172.21.15.184:6803/2193296422","heartbeat_back_addr":"172.21.15.184:6807/2193296422","heartbeat_front_addr":"172.21.15.184:6805/2193296422","state":["exists","up"]},{"osd":3,"uuid":"8892e35f-6880-434c-a805-ab8ccbed2bf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6811","nonce":81681815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6813","nonce":81681815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6817","nonce":81681815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6815","nonce":81681815}]},"public_addr":"172.21.15.16:6811/81681815","cluster_addr":"172.21.15.16:6813/81681815","heartbeat_back_addr":"172.21.15.16:6817/81681815","heartbeat_front_addr":"172.21.15.16:6815/81681815","state":["exists","up"]},{"osd":4,"uuid":"d2515e73-88f7-4654-9762-adb8802777ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6809","nonce":3592422019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6811","nonce":3592422019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6815","nonce":3592422019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6813","nonce":3592422019}]},"public_addr":"172.21.15.155:6809/3592422019","cluster_addr":"172.21.15.155:6811/3592422019","heartbeat_back_addr":"172.21.15.155:6815/3592422019","heartbeat_front_addr":"172.21.15.155:6813/3592422019","state":["exists","up"]},{"osd":5,"uuid":"5ce6e2fa-c17c-49b3-9402-f19bfc21076d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6809","nonce":2054894065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6811","nonce":2054894065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6815","nonce":2054894065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6813","nonce":2054894065}]},"public_addr":"172.21.15.184:6809/2054894065","cluster_addr":"172.21.15.184:6811/2054894065","heartbeat_back_addr":"172.21.15.184:6815/2054894065","heartbeat_front_addr":"172.21.15.184:6813/2054894065","state":["exists","up"]},{"osd":6,"uuid":"f257d8d1-ad59-4bd8-821d-bf29e50b547e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6819","nonce":1697421092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6821","nonce":1697421092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6825","nonce":1697421092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6823","nonce":1697421092}]},"public_addr":"172.21.15.16:6819/1697421092","cluster_addr":"172.21.15.16:6821/1697421092","heartbeat_back_addr":"172.21.15.16:6825/1697421092","heartbeat_front_addr":"172.21.15.16:6823/1697421092","state":["exists","up"]},{"osd":7,"uuid":"8e76c20e-8d25-423b-b7e8-de642d9c9a6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6817","nonce":2671827901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6819","nonce":2671827901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6823","nonce":2671827901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6821","nonce":2671827901}]},"public_addr":"172.21.15.155:6817/2671827901","cluster_addr":"172.21.15.155:6819/2671827901","heartbeat_back_addr":"172.21.15.155:6823/2671827901","heartbeat_front_addr":"172.21.15.155:6821/2671827901","state":["exists","up"]},{"osd":8,"uuid":"0f0bb23f-4bc3-4198-8381-5b5cd16ae394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6817","nonce":2479117823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6819","nonce":2479117823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6823","nonce":2479117823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6821","nonce":2479117823}]},"public_addr":"172.21.15.184:6817/2479117823","cluster_addr":"172.21.15.184:6819/2479117823","heartbeat_back_addr":"172.21.15.184:6823/2479117823","heartbeat_front_addr":"172.21.15.184:6821/2479117823","state":["exists","up"]},{"osd":9,"uuid":"bc27afcb-7871-4f72-a043-044fd5fe7d86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6827","nonce":3015700546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6829","nonce":3015700546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6833","nonce":3015700546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6831","nonce":3015700546}]},"public_addr":"172.21.15.16:6827/3015700546","cluster_addr":"172.21.15.16:6829/3015700546","heartbeat_back_addr":"172.21.15.16:6833/3015700546","heartbeat_front_addr":"172.21.15.16:6831/3015700546","state":["exists","up"]},{"osd":10,"uuid":"f2548997-72ee-400e-ab8d-5966d85b2cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6825","nonce":2427122475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6827","nonce":2427122475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6831","nonce":2427122475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6829","nonce":2427122475}]},"public_addr":"172.21.15.155:6825/2427122475","cluster_addr":"172.21.15.155:6827/2427122475","heartbeat_back_addr":"172.21.15.155:6831/2427122475","heartbeat_front_addr":"172.21.15.155:6829/2427122475","state":["exists","up"]},{"osd":11,"uuid":"8297da1d-d3e2-4a4d-9396-426dfafa3e95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6825","nonce":399265701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6827","nonce":399265701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6831","nonce":399265701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6829","nonce":399265701}]},"public_addr":"172.21.15.184:6825/399265701","cluster_addr":"172.21.15.184:6827/399265701","heartbeat_back_addr":"172.21.15.184:6831/399265701","heartbeat_front_addr":"172.21.15.184:6829/399265701","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:58:30.842360+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:58:57.164488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:59:20.778103+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:59:50.123991+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:00:16.789933+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:00:41.788905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:01:12.281376+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:01:39.211858+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:02:04.958771+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:02:35.645947+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:03:01.940433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:03:26.728673+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3923220960":"2024-09-01T05:57:23.153214+0000","172.21.15.16:0/3013533836":"2024-09-01T05:57:23.153214+0000","172.21.15.16:6800/3755683228":"2024-09-01T05:55:45.075902+0000","172.21.15.16:0/3064435470":"2024-09-01T05:55:45.075902+0000","172.21.15.16:0/911709962":"2024-09-01T05:55:45.075902+0000","172.21.15.16:0/2937368967":"2024-09-01T05:55:45.075902+0000","172.21.15.16:6800/1378972230":"2024-09-01T05:56:14.195704+0000","172.21.15.16:0/3047316995":"2024-09-01T05:56:14.195704+0000","172.21.15.16:6801/3755683228":"2024-09-01T05:55:45.075902+0000","172.21.15.16:6801/2571082634":"2024-09-01T05:57:23.153214+0000","172.21.15.16:0/2136704295":"2024-09-01T05:56:14.195704+0000","172.21.15.16:0/2259332778":"2024-09-01T05:56:14.195704+0000","172.21.15.16:6800/2571082634":"2024-09-01T05:57:23.153214+0000","172.21.15.16:6801/1378972230":"2024-09-01T05:56:14.195704+0000","172.21.15.16:0/2126457673":"2024-09-01T05:57:23.153214+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-31T06:04:30.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:30.010+0000 7f1197200640 1 -- 172.21.15.16:0/74495822 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f11740b4b80 msgr2=0x7f11740b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:30.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:30.010+0000 7f1197200640 1 --2- 172.21.15.16:0/74495822 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f11740b4b80 0x7f11740b7090 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f117c0095f0 tx=0x7f117c005f90 comp rx=0 tx=0).stop 2024-08-31T06:04:30.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:30.010+0000 7f1197200640 1 -- 172.21.15.16:0/74495822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1190075550 msgr2=0x7f119019c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:30.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:30.010+0000 7f1197200640 1 --2- 172.21.15.16:0/74495822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1190075550 0x7f119019c370 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f118000e9f0 tx=0x7f118000eeb0 comp rx=0 tx=0).stop 2024-08-31T06:04:30.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:30.010+0000 7f1197200640 1 -- 172.21.15.16:0/74495822 shutdown_connections 2024-08-31T06:04:30.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:30.010+0000 7f1197200640 1 --2- 172.21.15.16:0/74495822 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f11740b4b80 0x7f11740b7090 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:30.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:30.011+0000 7f1197200640 1 --2- 172.21.15.16:0/74495822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f119010fe80 0x7f11901a1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:30.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:30.011+0000 7f1197200640 1 --2- 172.21.15.16:0/74495822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1190075ea0 0x7f119019c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:30.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:30.011+0000 7f1197200640 1 --2- 172.21.15.16:0/74495822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1190075550 0x7f119019c370 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:30.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:30.011+0000 7f1197200640 1 -- 172.21.15.16:0/74495822 >> 172.21.15.16:0/74495822 conn(0x7f11900ff3b0 msgr2=0x7f1190109990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:04:30.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:30.011+0000 7f1197200640 1 -- 172.21.15.16:0/74495822 shutdown_connections 2024-08-31T06:04:30.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:30.011+0000 7f1197200640 1 -- 172.21.15.16:0/74495822 wait complete. 2024-08-31T06:04:30.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:29 smithi184 ceph-mon[22331]: mds.? [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] up:active 2024-08-31T06:04:30.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:29 smithi184 ceph-mon[22331]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-31T06:04:30.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:29 smithi184 ceph-mon[22331]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-31T06:04:30.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:29 smithi155 ceph-mon[21742]: mds.? [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] up:active 2024-08-31T06:04:30.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:29 smithi155 ceph-mon[21742]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-31T06:04:30.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:29 smithi155 ceph-mon[21742]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-31T06:04:30.748 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph osd dump --format=json 2024-08-31T06:04:31.132 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:04:31.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:30 smithi016 ceph-mon[17194]: pgmap v279: 97 pgs: 12 creating+peering, 79 active+clean, 6 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 2 op/s 2024-08-31T06:04:31.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:30 smithi016 ceph-mon[17194]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-08-31T06:04:31.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:30 smithi016 ceph-mon[17194]: Cluster is now healthy 2024-08-31T06:04:31.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:30 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/74495822' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T06:04:31.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:30 smithi184 ceph-mon[22331]: pgmap v279: 97 pgs: 12 creating+peering, 79 active+clean, 6 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 2 op/s 2024-08-31T06:04:31.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:30 smithi184 ceph-mon[22331]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-08-31T06:04:31.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:30 smithi184 ceph-mon[22331]: Cluster is now healthy 2024-08-31T06:04:31.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:30 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/74495822' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T06:04:31.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:30 smithi155 ceph-mon[21742]: pgmap v279: 97 pgs: 12 creating+peering, 79 active+clean, 6 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 2 op/s 2024-08-31T06:04:31.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:30 smithi155 ceph-mon[21742]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-08-31T06:04:31.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:30 smithi155 ceph-mon[21742]: Cluster is now healthy 2024-08-31T06:04:31.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:30 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/74495822' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T06:04:31.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.765+0000 7fdffc200640 1 -- 172.21.15.16:0/3705842133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdff4109b90 msgr2=0x7fdff41103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:31.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.765+0000 7fdffc200640 1 --2- 172.21.15.16:0/3705842133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdff4109b90 0x7fdff41103c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdfec009ca0 tx=0x7fdfec02a8c0 comp rx=0 tx=0).stop 2024-08-31T06:04:31.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.765+0000 7fdffc200640 1 -- 172.21.15.16:0/3705842133 shutdown_connections 2024-08-31T06:04:31.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.765+0000 7fdffc200640 1 --2- 172.21.15.16:0/3705842133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdff4109240 0x7fdff4109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:31.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.765+0000 7fdffc200640 1 --2- 172.21.15.16:0/3705842133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdff4109b90 0x7fdff41103c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:31.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.765+0000 7fdffc200640 1 --2- 172.21.15.16:0/3705842133 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdff4103900 0x7fdff4103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:31.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.765+0000 7fdffc200640 1 -- 172.21.15.16:0/3705842133 >> 172.21.15.16:0/3705842133 conn(0x7fdff407b5f0 msgr2=0x7fdff407ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:04:31.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.765+0000 7fdffc200640 1 -- 172.21.15.16:0/3705842133 shutdown_connections 2024-08-31T06:04:31.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.765+0000 7fdffc200640 1 -- 172.21.15.16:0/3705842133 wait complete. 2024-08-31T06:04:31.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.766+0000 7fdffc200640 1 Processor -- start 2024-08-31T06:04:31.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.766+0000 7fdffc200640 1 -- start start 2024-08-31T06:04:31.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.767+0000 7fdffc200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdff4103900 0x7fdff419c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:31.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.767+0000 7fdffc200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdff4109240 0x7fdff419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:31.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.767+0000 7fdffc200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdff4109b90 0x7fdff41a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:31.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.767+0000 7fdffc200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdff4113850 con 0x7fdff4103900 2024-08-31T06:04:31.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.767+0000 7fdffc200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdff41136d0 con 0x7fdff4109b90 2024-08-31T06:04:31.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.767+0000 7fdffc200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdff41139d0 con 0x7fdff4109240 2024-08-31T06:04:31.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.767+0000 7fdff9800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdff4103900 0x7fdff419c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:31.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.767+0000 7fdff9800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdff4103900 0x7fdff419c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56582/0 (socket says 172.21.15.16:56582) 2024-08-31T06:04:31.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.767+0000 7fdff9800640 1 -- 172.21.15.16:0/1760311724 learned_addr learned my addr 172.21.15.16:0/1760311724 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:04:31.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.767+0000 7fdff8e00640 1 --2- 172.21.15.16:0/1760311724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdff4109240 0x7fdff419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:31.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.767+0000 7fdffa200640 1 --2- 172.21.15.16:0/1760311724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdff4109b90 0x7fdff41a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:31.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.768+0000 7fdff8e00640 1 -- 172.21.15.16:0/1760311724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdff4109b90 msgr2=0x7fdff41a1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:31.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.768+0000 7fdff8e00640 1 --2- 172.21.15.16:0/1760311724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdff4109b90 0x7fdff41a1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:31.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.768+0000 7fdff8e00640 1 -- 172.21.15.16:0/1760311724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdff4103900 msgr2=0x7fdff419c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:31.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.768+0000 7fdff8e00640 1 --2- 172.21.15.16:0/1760311724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdff4103900 0x7fdff419c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:31.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.768+0000 7fdff8e00640 1 -- 172.21.15.16:0/1760311724 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdff41a2410 con 0x7fdff4109240 2024-08-31T06:04:31.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.768+0000 7fdff9800640 1 --2- 172.21.15.16:0/1760311724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdff4103900 0x7fdff419c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:04:31.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.768+0000 7fdff8e00640 1 --2- 172.21.15.16:0/1760311724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdff4109240 0x7fdff419ca40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdff406c790 tx=0x7fdfe000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:04:31.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.768+0000 7fdff2000640 1 -- 172.21.15.16:0/1760311724 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe000cc40 con 0x7fdff4109240 2024-08-31T06:04:31.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.768+0000 7fdffc200640 1 -- 172.21.15.16:0/1760311724 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdff41a2700 con 0x7fdff4109240 2024-08-31T06:04:31.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.768+0000 7fdffc200640 1 -- 172.21.15.16:0/1760311724 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdff41a8e60 con 0x7fdff4109240 2024-08-31T06:04:31.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.768+0000 7fdff2000640 1 -- 172.21.15.16:0/1760311724 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfe00044d0 con 0x7fdff4109240 2024-08-31T06:04:31.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.769+0000 7fdff2000640 1 -- 172.21.15.16:0/1760311724 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe001c070 con 0x7fdff4109240 2024-08-31T06:04:31.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.769+0000 7fdffc200640 1 -- 172.21.15.16:0/1760311724 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfd4005290 con 0x7fdff4109240 2024-08-31T06:04:31.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.773+0000 7fdff2000640 1 -- 172.21.15.16:0/1760311724 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7fdfe0005020 con 0x7fdff4109240 2024-08-31T06:04:31.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.775+0000 7fdff2000640 1 --2- 172.21.15.16:0/1760311724 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdfdc0b4b80 0x7fdfdc0b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:31.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.775+0000 7fdff2000640 1 -- 172.21.15.16:0/1760311724 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7fdfe00e4f40 con 0x7fdff4109240 2024-08-31T06:04:31.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.775+0000 7fdff9800640 1 --2- 172.21.15.16:0/1760311724 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdfdc0b4b80 0x7fdfdc0b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:31.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.776+0000 7fdff2000640 1 -- 172.21.15.16:0/1760311724 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fdfe00ac0c0 con 0x7fdff4109240 2024-08-31T06:04:31.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.776+0000 7fdff9800640 1 --2- 172.21.15.16:0/1760311724 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdfdc0b4b80 0x7fdfdc0b7090 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8011550 tx=0x7fdfe8005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:04:31.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.972+0000 7fdffc200640 1 -- 172.21.15.16:0/1760311724 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdfd4005850 con 0x7fdff4109240 2024-08-31T06:04:31.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.974+0000 7fdff2000640 1 -- 172.21.15.16:0/1760311724 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22608 (secure 0 0 0) 0x7fdfe00ab9c0 con 0x7fdff4109240 2024-08-31T06:04:31.976 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:04:31.977 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":78,"fsid":"673e4aa0-675d-11ef-bcd4-c7b262605968","created":"2024-08-31T05:55:09.632536+0000","modified":"2024-08-31T06:04:27.098694+0000","last_up_change":"2024-08-31T06:03:27.893697+0000","last_in_change":"2024-08-31T06:03:05.664945+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-31T05:59:23.642270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-31T06:04:20.871835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-31T06:04:24.119499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e349dae-6aa5-4b6d-9107-a945655e11bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6803","nonce":1725037400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6805","nonce":1725037400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6809","nonce":1725037400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6807","nonce":1725037400}]},"public_addr":"172.21.15.16:6803/1725037400","cluster_addr":"172.21.15.16:6805/1725037400","heartbeat_back_addr":"172.21.15.16:6809/1725037400","heartbeat_front_addr":"172.21.15.16:6807/1725037400","state":["exists","up"]},{"osd":1,"uuid":"bd645dc3-93b5-40af-89d8-08ccec22e50d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6801","nonce":4241618440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6803","nonce":4241618440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6807","nonce":4241618440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6805","nonce":4241618440}]},"public_addr":"172.21.15.155:6801/4241618440","cluster_addr":"172.21.15.155:6803/4241618440","heartbeat_back_addr":"172.21.15.155:6807/4241618440","heartbeat_front_addr":"172.21.15.155:6805/4241618440","state":["exists","up"]},{"osd":2,"uuid":"c69c8c19-32f8-400a-86d4-0f4ce240a7de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6801","nonce":2193296422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6803","nonce":2193296422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6807","nonce":2193296422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6805","nonce":2193296422}]},"public_addr":"172.21.15.184:6801/2193296422","cluster_addr":"172.21.15.184:6803/2193296422","heartbeat_back_addr":"172.21.15.184:6807/2193296422","heartbeat_front_addr":"172.21.15.184:6805/2193296422","state":["exists","up"]},{"osd":3,"uuid":"8892e35f-6880-434c-a805-ab8ccbed2bf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6811","nonce":81681815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6813","nonce":81681815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6817","nonce":81681815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6815","nonce":81681815}]},"public_addr":"172.21.15.16:6811/81681815","cluster_addr":"172.21.15.16:6813/81681815","heartbeat_back_addr":"172.21.15.16:6817/81681815","heartbeat_front_addr":"172.21.15.16:6815/81681815","state":["exists","up"]},{"osd":4,"uuid":"d2515e73-88f7-4654-9762-adb8802777ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6809","nonce":3592422019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6811","nonce":3592422019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6815","nonce":3592422019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6813","nonce":3592422019}]},"public_addr":"172.21.15.155:6809/3592422019","cluster_addr":"172.21.15.155:6811/3592422019","heartbeat_back_addr":"172.21.15.155:6815/3592422019","heartbeat_front_addr":"172.21.15.155:6813/3592422019","state":["exists","up"]},{"osd":5,"uuid":"5ce6e2fa-c17c-49b3-9402-f19bfc21076d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6809","nonce":2054894065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6811","nonce":2054894065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6815","nonce":2054894065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6813","nonce":2054894065}]},"public_addr":"172.21.15.184:6809/2054894065","cluster_addr":"172.21.15.184:6811/2054894065","heartbeat_back_addr":"172.21.15.184:6815/2054894065","heartbeat_front_addr":"172.21.15.184:6813/2054894065","state":["exists","up"]},{"osd":6,"uuid":"f257d8d1-ad59-4bd8-821d-bf29e50b547e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6819","nonce":1697421092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6821","nonce":1697421092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6825","nonce":1697421092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6823","nonce":1697421092}]},"public_addr":"172.21.15.16:6819/1697421092","cluster_addr":"172.21.15.16:6821/1697421092","heartbeat_back_addr":"172.21.15.16:6825/1697421092","heartbeat_front_addr":"172.21.15.16:6823/1697421092","state":["exists","up"]},{"osd":7,"uuid":"8e76c20e-8d25-423b-b7e8-de642d9c9a6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6817","nonce":2671827901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6819","nonce":2671827901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6823","nonce":2671827901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6821","nonce":2671827901}]},"public_addr":"172.21.15.155:6817/2671827901","cluster_addr":"172.21.15.155:6819/2671827901","heartbeat_back_addr":"172.21.15.155:6823/2671827901","heartbeat_front_addr":"172.21.15.155:6821/2671827901","state":["exists","up"]},{"osd":8,"uuid":"0f0bb23f-4bc3-4198-8381-5b5cd16ae394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6817","nonce":2479117823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6819","nonce":2479117823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6823","nonce":2479117823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6821","nonce":2479117823}]},"public_addr":"172.21.15.184:6817/2479117823","cluster_addr":"172.21.15.184:6819/2479117823","heartbeat_back_addr":"172.21.15.184:6823/2479117823","heartbeat_front_addr":"172.21.15.184:6821/2479117823","state":["exists","up"]},{"osd":9,"uuid":"bc27afcb-7871-4f72-a043-044fd5fe7d86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6827","nonce":3015700546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6829","nonce":3015700546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6833","nonce":3015700546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6831","nonce":3015700546}]},"public_addr":"172.21.15.16:6827/3015700546","cluster_addr":"172.21.15.16:6829/3015700546","heartbeat_back_addr":"172.21.15.16:6833/3015700546","heartbeat_front_addr":"172.21.15.16:6831/3015700546","state":["exists","up"]},{"osd":10,"uuid":"f2548997-72ee-400e-ab8d-5966d85b2cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6825","nonce":2427122475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6827","nonce":2427122475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6831","nonce":2427122475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6829","nonce":2427122475}]},"public_addr":"172.21.15.155:6825/2427122475","cluster_addr":"172.21.15.155:6827/2427122475","heartbeat_back_addr":"172.21.15.155:6831/2427122475","heartbeat_front_addr":"172.21.15.155:6829/2427122475","state":["exists","up"]},{"osd":11,"uuid":"8297da1d-d3e2-4a4d-9396-426dfafa3e95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6825","nonce":399265701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6827","nonce":399265701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6831","nonce":399265701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6829","nonce":399265701}]},"public_addr":"172.21.15.184:6825/399265701","cluster_addr":"172.21.15.184:6827/399265701","heartbeat_back_addr":"172.21.15.184:6831/399265701","heartbeat_front_addr":"172.21.15.184:6829/399265701","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:58:30.842360+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:58:57.164488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:59:20.778103+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:59:50.123991+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:00:16.789933+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:00:41.788905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:01:12.281376+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:01:39.211858+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:02:04.958771+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:02:35.645947+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:03:01.940433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:03:26.728673+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3923220960":"2024-09-01T05:57:23.153214+0000","172.21.15.16:0/3013533836":"2024-09-01T05:57:23.153214+0000","172.21.15.16:6800/3755683228":"2024-09-01T05:55:45.075902+0000","172.21.15.16:0/3064435470":"2024-09-01T05:55:45.075902+0000","172.21.15.16:0/911709962":"2024-09-01T05:55:45.075902+0000","172.21.15.16:0/2937368967":"2024-09-01T05:55:45.075902+0000","172.21.15.16:6800/1378972230":"2024-09-01T05:56:14.195704+0000","172.21.15.16:0/3047316995":"2024-09-01T05:56:14.195704+0000","172.21.15.16:6801/3755683228":"2024-09-01T05:55:45.075902+0000","172.21.15.16:6801/2571082634":"2024-09-01T05:57:23.153214+0000","172.21.15.16:0/2136704295":"2024-09-01T05:56:14.195704+0000","172.21.15.16:0/2259332778":"2024-09-01T05:56:14.195704+0000","172.21.15.16:6800/2571082634":"2024-09-01T05:57:23.153214+0000","172.21.15.16:6801/1378972230":"2024-09-01T05:56:14.195704+0000","172.21.15.16:0/2126457673":"2024-09-01T05:57:23.153214+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-31T06:04:31.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.976+0000 7fdffc200640 1 -- 172.21.15.16:0/1760311724 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdfdc0b4b80 msgr2=0x7fdfdc0b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:31.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.976+0000 7fdffc200640 1 --2- 172.21.15.16:0/1760311724 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdfdc0b4b80 0x7fdfdc0b7090 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8011550 tx=0x7fdfe8005f90 comp rx=0 tx=0).stop 2024-08-31T06:04:31.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.976+0000 7fdffc200640 1 -- 172.21.15.16:0/1760311724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdff4109240 msgr2=0x7fdff419ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:31.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.976+0000 7fdffc200640 1 --2- 172.21.15.16:0/1760311724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdff4109240 0x7fdff419ca40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdff406c790 tx=0x7fdfe000ee50 comp rx=0 tx=0).stop 2024-08-31T06:04:31.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.976+0000 7fdffc200640 1 -- 172.21.15.16:0/1760311724 shutdown_connections 2024-08-31T06:04:31.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.976+0000 7fdffc200640 1 --2- 172.21.15.16:0/1760311724 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdfdc0b4b80 0x7fdfdc0b7090 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:31.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.976+0000 7fdffc200640 1 --2- 172.21.15.16:0/1760311724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdff4109240 0x7fdff419ca40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:31.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.976+0000 7fdffc200640 1 --2- 172.21.15.16:0/1760311724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdff4109b90 0x7fdff41a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:31.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.976+0000 7fdffc200640 1 --2- 172.21.15.16:0/1760311724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdff4103900 0x7fdff419c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:31.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.976+0000 7fdffc200640 1 -- 172.21.15.16:0/1760311724 >> 172.21.15.16:0/1760311724 conn(0x7fdff407b5f0 msgr2=0x7fdff41070b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:04:31.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.976+0000 7fdffc200640 1 -- 172.21.15.16:0/1760311724 shutdown_connections 2024-08-31T06:04:31.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:31.977+0000 7fdffc200640 1 -- 172.21.15.16:0/1760311724 wait complete. 2024-08-31T06:04:32.884 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-08-31T06:04:33.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:32 smithi016 ceph-mon[17194]: pgmap v280: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 3 op/s 2024-08-31T06:04:33.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:32 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/1760311724' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T06:04:33.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:32 smithi184 ceph-mon[22331]: pgmap v280: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 3 op/s 2024-08-31T06:04:33.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:32 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/1760311724' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T06:04:33.263 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:04:33.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:32 smithi155 ceph-mon[21742]: pgmap v280: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 3 op/s 2024-08-31T06:04:33.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:32 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/1760311724' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T06:04:33.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:33.921+0000 7f10b7a00640 1 -- 172.21.15.16:0/136788718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10b0105b20 msgr2=0x7f10b0105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:33.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:33.921+0000 7f10b7a00640 1 --2- 172.21.15.16:0/136788718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10b0105b20 0x7f10b0105f30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f10a4009860 tx=0x7f10a402a340 comp rx=0 tx=0).stop 2024-08-31T06:04:33.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:33.921+0000 7f10b7a00640 1 -- 172.21.15.16:0/136788718 shutdown_connections 2024-08-31T06:04:33.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:33.921+0000 7f10b7a00640 1 --2- 172.21.15.16:0/136788718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10b010bdb0 0x7f10b01125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:33.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:33.921+0000 7f10b7a00640 1 --2- 172.21.15.16:0/136788718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10b010b460 0x7f10b010b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:33.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:33.921+0000 7f10b7a00640 1 --2- 172.21.15.16:0/136788718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10b0105b20 0x7f10b0105f30 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:33.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:33.921+0000 7f10b7a00640 1 -- 172.21.15.16:0/136788718 >> 172.21.15.16:0/136788718 conn(0x7f10b01015b0 msgr2=0x7f10b0103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:04:33.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:33.921+0000 7f10b7a00640 1 -- 172.21.15.16:0/136788718 shutdown_connections 2024-08-31T06:04:33.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:33.921+0000 7f10b7a00640 1 -- 172.21.15.16:0/136788718 wait complete. 2024-08-31T06:04:33.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:33.923+0000 7f10b7a00640 1 Processor -- start 2024-08-31T06:04:33.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:33.923+0000 7f10b7a00640 1 -- start start 2024-08-31T06:04:33.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:33.923+0000 7f10b7a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10b0105b20 0x7f10b019c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:33.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:33.923+0000 7f10b7a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10b010b460 0x7f10b019ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:33.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:33.923+0000 7f10b7a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10b010bdb0 0x7f10b01a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:33.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:33.923+0000 7f10b7a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f10b0115a70 con 0x7f10b010bdb0 2024-08-31T06:04:33.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:33.923+0000 7f10b7a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f10b01158f0 con 0x7f10b0105b20 2024-08-31T06:04:33.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:33.923+0000 7f10b7a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f10b0115bf0 con 0x7f10b010b460 2024-08-31T06:04:33.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:33.924+0000 7f10b5a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10b010bdb0 0x7f10b01a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:33.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:33.924+0000 7f10b5a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10b010bdb0 0x7f10b01a1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56602/0 (socket says 172.21.15.16:56602) 2024-08-31T06:04:33.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:33.924+0000 7f10b5a00640 1 -- 172.21.15.16:0/1677146765 learned_addr learned my addr 172.21.15.16:0/1677146765 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:04:33.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:33.924+0000 7f10b5a00640 1 -- 172.21.15.16:0/1677146765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10b010b460 msgr2=0x7f10b019ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:33.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:33.924+0000 7f10b5a00640 1 --2- 172.21.15.16:0/1677146765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10b010b460 0x7f10b019ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:33.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:33.924+0000 7f10b5a00640 1 -- 172.21.15.16:0/1677146765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10b0105b20 msgr2=0x7f10b019c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:33.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:33.924+0000 7f10b5a00640 1 --2- 172.21.15.16:0/1677146765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10b0105b20 0x7f10b019c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:33.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:33.924+0000 7f10b5a00640 1 -- 172.21.15.16:0/1677146765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10b01a23d0 con 0x7f10b010bdb0 2024-08-31T06:04:33.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:33.924+0000 7f10b5a00640 1 --2- 172.21.15.16:0/1677146765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10b010bdb0 0x7f10b01a1ca0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f10a800eb10 tx=0x7f10a800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:04:33.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:33.925+0000 7f10ad600640 1 -- 172.21.15.16:0/1677146765 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a8004b20 con 0x7f10b010bdb0 2024-08-31T06:04:33.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:33.925+0000 7f10b7a00640 1 -- 172.21.15.16:0/1677146765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10b01a26c0 con 0x7f10b010bdb0 2024-08-31T06:04:33.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:33.925+0000 7f10ad600640 1 -- 172.21.15.16:0/1677146765 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10a8004cc0 con 0x7f10b010bdb0 2024-08-31T06:04:33.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:33.925+0000 7f10ad600640 1 -- 172.21.15.16:0/1677146765 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a8010690 con 0x7f10b010bdb0 2024-08-31T06:04:33.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:33.925+0000 7f10b7a00640 1 -- 172.21.15.16:0/1677146765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10b01a8e20 con 0x7f10b010bdb0 2024-08-31T06:04:33.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:33.926+0000 7f10b7a00640 1 -- 172.21.15.16:0/1677146765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10b0105f30 con 0x7f10b010bdb0 2024-08-31T06:04:33.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:33.929+0000 7f10ad600640 1 -- 172.21.15.16:0/1677146765 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7f10a8005000 con 0x7f10b010bdb0 2024-08-31T06:04:33.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:33.930+0000 7f10ad600640 1 --2- 172.21.15.16:0/1677146765 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f10980b4b80 0x7f10980b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:33.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:33.931+0000 7f10ad600640 1 -- 172.21.15.16:0/1677146765 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7f10a80e5a90 con 0x7f10b010bdb0 2024-08-31T06:04:33.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:33.931+0000 7f10b5000640 1 --2- 172.21.15.16:0/1677146765 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f10980b4b80 0x7f10980b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:33.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:33.931+0000 7f10b5000640 1 --2- 172.21.15.16:0/1677146765 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f10980b4b80 0x7f10980b7090 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f10a40095f0 tx=0x7f10a4009580 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:04:33.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:33.932+0000 7f10ad600640 1 -- 172.21.15.16:0/1677146765 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f10a8056130 con 0x7f10b010bdb0 2024-08-31T06:04:34.122 INFO:journalctl@ceph.mds.b.smithi155.stdout:Aug 31 06:04:33 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-b[53353]: 2024-08-31T06:04:33.856+0000 7f6c202fe640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-08-31T06:04:34.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:34.163+0000 7f10b7a00640 1 -- 172.21.15.16:0/1677146765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f10b002e430 con 0x7f10b010bdb0 2024-08-31T06:04:34.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:34.894+0000 7f10ad600640 1 -- 172.21.15.16:0/1677146765 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f10a80acb80 con 0x7f10b010bdb0 2024-08-31T06:04:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:34.897+0000 7f10b7a00640 1 -- 172.21.15.16:0/1677146765 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f10980b4b80 msgr2=0x7f10980b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:34.897+0000 7f10b7a00640 1 --2- 172.21.15.16:0/1677146765 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f10980b4b80 0x7f10980b7090 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f10a40095f0 tx=0x7f10a4009580 comp rx=0 tx=0).stop 2024-08-31T06:04:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:34.897+0000 7f10b7a00640 1 -- 172.21.15.16:0/1677146765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10b010bdb0 msgr2=0x7f10b01a1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:34.897+0000 7f10b7a00640 1 --2- 172.21.15.16:0/1677146765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10b010bdb0 0x7f10b01a1ca0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f10a800eb10 tx=0x7f10a800c2d0 comp rx=0 tx=0).stop 2024-08-31T06:04:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:34.897+0000 7f10b7a00640 1 -- 172.21.15.16:0/1677146765 shutdown_connections 2024-08-31T06:04:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:34.897+0000 7f10b7a00640 1 --2- 172.21.15.16:0/1677146765 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f10980b4b80 0x7f10980b7090 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:34.897+0000 7f10b7a00640 1 --2- 172.21.15.16:0/1677146765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10b010b460 0x7f10b019ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:34.897+0000 7f10b7a00640 1 --2- 172.21.15.16:0/1677146765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10b010bdb0 0x7f10b01a1ca0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:34.897+0000 7f10b7a00640 1 --2- 172.21.15.16:0/1677146765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10b0105b20 0x7f10b019c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:34.897+0000 7f10b7a00640 1 -- 172.21.15.16:0/1677146765 >> 172.21.15.16:0/1677146765 conn(0x7f10b01015b0 msgr2=0x7f10b01039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:04:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:34.897+0000 7f10b7a00640 1 -- 172.21.15.16:0/1677146765 shutdown_connections 2024-08-31T06:04:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:34.897+0000 7f10b7a00640 1 -- 172.21.15.16:0/1677146765 wait complete. 2024-08-31T06:04:35.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:34 smithi016 ceph-mon[17194]: pgmap v281: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-08-31T06:04:35.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:34 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/1677146765' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-08-31T06:04:35.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:34 smithi184 ceph-mon[22331]: pgmap v281: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-08-31T06:04:35.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:34 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/1677146765' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-08-31T06:04:35.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:34 smithi155 ceph-mon[21742]: pgmap v281: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-08-31T06:04:35.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:34 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/1677146765' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-08-31T06:04:35.851 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-08-31T06:04:35.851 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-08-31T06:04:36.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:35 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/1677146765' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-08-31T06:04:36.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:35 smithi016 ceph-mon[17194]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-31T06:04:36.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:35 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/1677146765' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-08-31T06:04:36.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:35 smithi184 ceph-mon[22331]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-31T06:04:36.234 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:04:36.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:35 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/1677146765' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-08-31T06:04:36.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:35 smithi155 ceph-mon[21742]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-31T06:04:36.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:36.903+0000 7fc431400640 1 -- 172.21.15.16:0/3011187315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc42c10b460 msgr2=0x7fc42c10b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:36.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:36.903+0000 7fc431400640 1 --2- 172.21.15.16:0/3011187315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc42c10b460 0x7fc42c10b870 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc418009900 tx=0x7fc41802a3f0 comp rx=0 tx=0).stop 2024-08-31T06:04:36.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:36.903+0000 7fc431400640 1 -- 172.21.15.16:0/3011187315 shutdown_connections 2024-08-31T06:04:36.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:36.903+0000 7fc431400640 1 --2- 172.21.15.16:0/3011187315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc42c10bdb0 0x7fc42c1125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:36.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:36.903+0000 7fc431400640 1 --2- 172.21.15.16:0/3011187315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc42c10b460 0x7fc42c10b870 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:36.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:36.903+0000 7fc431400640 1 --2- 172.21.15.16:0/3011187315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc42c105b20 0x7fc42c105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:36.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:36.903+0000 7fc431400640 1 -- 172.21.15.16:0/3011187315 >> 172.21.15.16:0/3011187315 conn(0x7fc42c1015b0 msgr2=0x7fc42c103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:04:36.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:36.903+0000 7fc431400640 1 -- 172.21.15.16:0/3011187315 shutdown_connections 2024-08-31T06:04:36.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:36.903+0000 7fc431400640 1 -- 172.21.15.16:0/3011187315 wait complete. 2024-08-31T06:04:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:36.904+0000 7fc431400640 1 Processor -- start 2024-08-31T06:04:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:36.905+0000 7fc431400640 1 -- start start 2024-08-31T06:04:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:36.905+0000 7fc431400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc42c105b20 0x7fc42c19c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:36.905+0000 7fc431400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc42c10b460 0x7fc42c19ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:36.905+0000 7fc431400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc42c10bdb0 0x7fc42c1a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:36.905+0000 7fc431400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c115ac0 con 0x7fc42c10bdb0 2024-08-31T06:04:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:36.905+0000 7fc431400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c115940 con 0x7fc42c105b20 2024-08-31T06:04:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:36.905+0000 7fc431400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c115c40 con 0x7fc42c10b460 2024-08-31T06:04:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:36.905+0000 7fc42b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc42c10bdb0 0x7fc42c1a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:36.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:36.905+0000 7fc42a800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc42c105b20 0x7fc42c19c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:36.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:36.905+0000 7fc429e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc42c10b460 0x7fc42c19ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:36.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:36.905+0000 7fc42a800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc42c105b20 0x7fc42c19c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:58806/0 (socket says 172.21.15.16:58806) 2024-08-31T06:04:36.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:36.905+0000 7fc429e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc42c10b460 0x7fc42c19ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:46286/0 (socket says 172.21.15.16:46286) 2024-08-31T06:04:36.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:36.905+0000 7fc42a800640 1 -- 172.21.15.16:0/109381558 learned_addr learned my addr 172.21.15.16:0/109381558 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:04:36.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:36.906+0000 7fc42a800640 1 -- 172.21.15.16:0/109381558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc42c10b460 msgr2=0x7fc42c19ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:36.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:36.906+0000 7fc42a800640 1 --2- 172.21.15.16:0/109381558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc42c10b460 0x7fc42c19ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:36.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:36.906+0000 7fc42a800640 1 -- 172.21.15.16:0/109381558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc42c10bdb0 msgr2=0x7fc42c1a1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:36.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:36.906+0000 7fc42a800640 1 --2- 172.21.15.16:0/109381558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc42c10bdb0 0x7fc42c1a1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:36.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:36.906+0000 7fc42a800640 1 -- 172.21.15.16:0/109381558 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc42c1a2460 con 0x7fc42c105b20 2024-08-31T06:04:36.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:36.906+0000 7fc42b200640 1 --2- 172.21.15.16:0/109381558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc42c10bdb0 0x7fc42c1a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:04:36.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:36.906+0000 7fc42a800640 1 --2- 172.21.15.16:0/109381558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc42c105b20 0x7fc42c19c4f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc41c00e9f0 tx=0x7fc41c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:04:36.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:36.906+0000 7fc423400640 1 -- 172.21.15.16:0/109381558 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c00cd80 con 0x7fc42c105b20 2024-08-31T06:04:36.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:36.907+0000 7fc431400640 1 -- 172.21.15.16:0/109381558 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc42c1a2750 con 0x7fc42c105b20 2024-08-31T06:04:36.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:36.907+0000 7fc423400640 1 -- 172.21.15.16:0/109381558 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc41c0108e0 con 0x7fc42c105b20 2024-08-31T06:04:36.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:36.907+0000 7fc431400640 1 -- 172.21.15.16:0/109381558 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc42c1a8eb0 con 0x7fc42c105b20 2024-08-31T06:04:36.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:36.907+0000 7fc423400640 1 -- 172.21.15.16:0/109381558 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c0188e0 con 0x7fc42c105b20 2024-08-31T06:04:36.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:36.908+0000 7fc431400640 1 -- 172.21.15.16:0/109381558 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc42c105f30 con 0x7fc42c105b20 2024-08-31T06:04:36.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:36.910+0000 7fc423400640 1 -- 172.21.15.16:0/109381558 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7fc41c010430 con 0x7fc42c105b20 2024-08-31T06:04:36.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:36.912+0000 7fc423400640 1 --2- 172.21.15.16:0/109381558 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc4100b4c00 0x7fc4100b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:36.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:36.912+0000 7fc423400640 1 -- 172.21.15.16:0/109381558 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7fc41c0e5580 con 0x7fc42c105b20 2024-08-31T06:04:36.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:36.912+0000 7fc429e00640 1 --2- 172.21.15.16:0/109381558 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc4100b4c00 0x7fc4100b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:36.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:36.913+0000 7fc429e00640 1 --2- 172.21.15.16:0/109381558 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc4100b4c00 0x7fc4100b7110 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc41802a9e0 tx=0x7fc418005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:04:36.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:36.917+0000 7fc423400640 1 -- 172.21.15.16:0/109381558 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fc41c0ac080 con 0x7fc42c105b20 2024-08-31T06:04:37.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:37.150+0000 7fc431400640 1 -- 172.21.15.16:0/109381558 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7fc42c02e430 con 0x7fc42c105b20 2024-08-31T06:04:37.153 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:36 smithi016 ceph-mon[17194]: pgmap v282: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-08-31T06:04:37.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:36 smithi184 ceph-mon[22331]: pgmap v282: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-08-31T06:04:37.273 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:36 smithi155 ceph-mon[21742]: pgmap v282: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-08-31T06:04:37.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:37.919+0000 7fc423400640 1 -- 172.21.15.16:0/109381558 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7fc41c0b0840 con 0x7fc42c105b20 2024-08-31T06:04:37.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:37.923+0000 7fc431400640 1 -- 172.21.15.16:0/109381558 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc4100b4c00 msgr2=0x7fc4100b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:37.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:37.923+0000 7fc431400640 1 --2- 172.21.15.16:0/109381558 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc4100b4c00 0x7fc4100b7110 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc41802a9e0 tx=0x7fc418005a20 comp rx=0 tx=0).stop 2024-08-31T06:04:37.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:37.923+0000 7fc431400640 1 -- 172.21.15.16:0/109381558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc42c105b20 msgr2=0x7fc42c19c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:37.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:37.923+0000 7fc431400640 1 --2- 172.21.15.16:0/109381558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc42c105b20 0x7fc42c19c4f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc41c00e9f0 tx=0x7fc41c00eeb0 comp rx=0 tx=0).stop 2024-08-31T06:04:37.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:37.923+0000 7fc431400640 1 -- 172.21.15.16:0/109381558 shutdown_connections 2024-08-31T06:04:37.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:37.923+0000 7fc431400640 1 --2- 172.21.15.16:0/109381558 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc4100b4c00 0x7fc4100b7110 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:37.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:37.923+0000 7fc431400640 1 --2- 172.21.15.16:0/109381558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc42c10b460 0x7fc42c19ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:37.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:37.923+0000 7fc431400640 1 --2- 172.21.15.16:0/109381558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc42c10bdb0 0x7fc42c1a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:37.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:37.923+0000 7fc431400640 1 --2- 172.21.15.16:0/109381558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc42c105b20 0x7fc42c19c4f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:37.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:37.923+0000 7fc431400640 1 -- 172.21.15.16:0/109381558 >> 172.21.15.16:0/109381558 conn(0x7fc42c1015b0 msgr2=0x7fc42c1039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:04:37.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:37.923+0000 7fc431400640 1 -- 172.21.15.16:0/109381558 shutdown_connections 2024-08-31T06:04:37.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:37.923+0000 7fc431400640 1 -- 172.21.15.16:0/109381558 wait complete. 2024-08-31T06:04:38.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:37 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/109381558' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-08-31T06:04:38.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:37 smithi016 ceph-mon[17194]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-08-31T06:04:38.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:37 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/109381558' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-08-31T06:04:38.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:37 smithi184 ceph-mon[22331]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-08-31T06:04:38.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:37 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/109381558' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-08-31T06:04:38.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:37 smithi155 ceph-mon[21742]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-08-31T06:04:39.010 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-08-31T06:04:39.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:38 smithi184 ceph-mon[22331]: pgmap v283: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-08-31T06:04:39.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:38 smithi184 ceph-mon[22331]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-08-31T06:04:39.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:38 smithi184 ceph-mon[22331]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-08-31T06:04:39.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:38 smithi184 ceph-mon[22331]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-08-31T06:04:39.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:38 smithi184 ceph-mon[22331]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-31T06:04:39.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:38 smithi184 ceph-mon[22331]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-08-31T06:04:39.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:38 smithi184 ceph-mon[22331]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-08-31T06:04:39.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:38 smithi155 ceph-mon[21742]: pgmap v283: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-08-31T06:04:39.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:38 smithi155 ceph-mon[21742]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-08-31T06:04:39.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:38 smithi155 ceph-mon[21742]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-08-31T06:04:39.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:38 smithi155 ceph-mon[21742]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-08-31T06:04:39.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:38 smithi155 ceph-mon[21742]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-31T06:04:39.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:38 smithi155 ceph-mon[21742]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-08-31T06:04:39.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:38 smithi155 ceph-mon[21742]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-08-31T06:04:39.392 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:04:39.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:38 smithi016 ceph-mon[17194]: pgmap v283: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-08-31T06:04:39.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:38 smithi016 ceph-mon[17194]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-08-31T06:04:39.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:38 smithi016 ceph-mon[17194]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-08-31T06:04:39.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:38 smithi016 ceph-mon[17194]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-08-31T06:04:39.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:38 smithi016 ceph-mon[17194]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-31T06:04:39.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:38 smithi016 ceph-mon[17194]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-08-31T06:04:39.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:38 smithi016 ceph-mon[17194]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-08-31T06:04:40.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.025+0000 7f2d39a00640 1 -- 172.21.15.16:0/3649033267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d34103920 msgr2=0x7f2d34103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:40.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.025+0000 7f2d39a00640 1 --2- 172.21.15.16:0/3649033267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d34103920 0x7f2d34103d30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2d24009900 tx=0x7f2d2402a3f0 comp rx=0 tx=0).stop 2024-08-31T06:04:40.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.026+0000 7f2d39a00640 1 -- 172.21.15.16:0/3649033267 shutdown_connections 2024-08-31T06:04:40.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.026+0000 7f2d39a00640 1 --2- 172.21.15.16:0/3649033267 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d34109bb0 0x7f2d341103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:40.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.026+0000 7f2d39a00640 1 --2- 172.21.15.16:0/3649033267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d34109260 0x7f2d34109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:40.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.026+0000 7f2d39a00640 1 --2- 172.21.15.16:0/3649033267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d34103920 0x7f2d34103d30 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:40.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.026+0000 7f2d39a00640 1 -- 172.21.15.16:0/3649033267 >> 172.21.15.16:0/3649033267 conn(0x7f2d340ff370 msgr2=0x7f2d341017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:04:40.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.026+0000 7f2d39a00640 1 -- 172.21.15.16:0/3649033267 shutdown_connections 2024-08-31T06:04:40.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.026+0000 7f2d39a00640 1 -- 172.21.15.16:0/3649033267 wait complete. 2024-08-31T06:04:40.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.027+0000 7f2d39a00640 1 Processor -- start 2024-08-31T06:04:40.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.027+0000 7f2d39a00640 1 -- start start 2024-08-31T06:04:40.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.027+0000 7f2d39a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d34103920 0x7f2d3419a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:40.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.027+0000 7f2d39a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d34109260 0x7f2d3419a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:40.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.027+0000 7f2d39a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d34109bb0 0x7f2d3419fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:40.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.027+0000 7f2d39a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2d34113870 con 0x7f2d34109260 2024-08-31T06:04:40.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.027+0000 7f2d39a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2d341136f0 con 0x7f2d34109bb0 2024-08-31T06:04:40.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.027+0000 7f2d39a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2d341139f0 con 0x7f2d34103920 2024-08-31T06:04:40.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.028+0000 7f2d32000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d34109260 0x7f2d3419a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:40.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.028+0000 7f2d32000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d34109260 0x7f2d3419a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34350/0 (socket says 172.21.15.16:34350) 2024-08-31T06:04:40.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.028+0000 7f2d32000640 1 -- 172.21.15.16:0/2935797261 learned_addr learned my addr 172.21.15.16:0/2935797261 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:04:40.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.028+0000 7f2d32a00640 1 --2- 172.21.15.16:0/2935797261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d34103920 0x7f2d3419a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:40.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.028+0000 7f2d33400640 1 --2- 172.21.15.16:0/2935797261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d34109bb0 0x7f2d3419fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:40.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.028+0000 7f2d32000640 1 -- 172.21.15.16:0/2935797261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d34103920 msgr2=0x7f2d3419a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:40.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.028+0000 7f2d32000640 1 --2- 172.21.15.16:0/2935797261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d34103920 0x7f2d3419a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:40.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.028+0000 7f2d32000640 1 -- 172.21.15.16:0/2935797261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d34109bb0 msgr2=0x7f2d3419fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:40.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.028+0000 7f2d32000640 1 --2- 172.21.15.16:0/2935797261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d34109bb0 0x7f2d3419fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:40.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.028+0000 7f2d32000640 1 -- 172.21.15.16:0/2935797261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d341a0190 con 0x7f2d34109260 2024-08-31T06:04:40.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.029+0000 7f2d32000640 1 --2- 172.21.15.16:0/2935797261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d34109260 0x7f2d3419a7c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2d2000d950 tx=0x7f2d2000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:04:40.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.029+0000 7f2d2b400640 1 -- 172.21.15.16:0/2935797261 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d20017070 con 0x7f2d34109260 2024-08-31T06:04:40.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.029+0000 7f2d2b400640 1 -- 172.21.15.16:0/2935797261 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d20009c30 con 0x7f2d34109260 2024-08-31T06:04:40.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.029+0000 7f2d39a00640 1 -- 172.21.15.16:0/2935797261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d341a0480 con 0x7f2d34109260 2024-08-31T06:04:40.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.029+0000 7f2d2b400640 1 -- 172.21.15.16:0/2935797261 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d200047a0 con 0x7f2d34109260 2024-08-31T06:04:40.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.029+0000 7f2d39a00640 1 -- 172.21.15.16:0/2935797261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d341a6be0 con 0x7f2d34109260 2024-08-31T06:04:40.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.030+0000 7f2d39a00640 1 -- 172.21.15.16:0/2935797261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d34103d30 con 0x7f2d34109260 2024-08-31T06:04:40.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.033+0000 7f2d2b400640 1 -- 172.21.15.16:0/2935797261 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7f2d20004020 con 0x7f2d34109260 2024-08-31T06:04:40.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.035+0000 7f2d2b400640 1 --2- 172.21.15.16:0/2935797261 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f2d180b4c00 0x7f2d180b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:40.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.035+0000 7f2d2b400640 1 -- 172.21.15.16:0/2935797261 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7f2d200e51d0 con 0x7f2d34109260 2024-08-31T06:04:40.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.035+0000 7f2d32a00640 1 --2- 172.21.15.16:0/2935797261 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f2d180b4c00 0x7f2d180b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:40.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.036+0000 7f2d32a00640 1 --2- 172.21.15.16:0/2935797261 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f2d180b4c00 0x7f2d180b7110 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2d24031440 tx=0x7f2d24009560 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:04:40.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.036+0000 7f2d2b400640 1 -- 172.21.15.16:0/2935797261 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f2d20020020 con 0x7f2d34109260 2024-08-31T06:04:40.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:39 smithi016 ceph-mon[17194]: mds.? [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] up:active 2024-08-31T06:04:40.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:39 smithi016 ceph-mon[17194]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-08-31T06:04:40.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:39 smithi016 ceph-mon[17194]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-08-31T06:04:40.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:39 smithi016 ceph-mon[17194]: Cluster is now healthy 2024-08-31T06:04:40.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:39 smithi016 ceph-mon[17194]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-08-31T06:04:40.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:39 smithi016 ceph-mon[17194]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:creating} 9 up:standby 2024-08-31T06:04:40.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:39 smithi016 ceph-mon[17194]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-08-31T06:04:40.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:39 smithi184 ceph-mon[22331]: mds.? [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] up:active 2024-08-31T06:04:40.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:39 smithi184 ceph-mon[22331]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-08-31T06:04:40.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:39 smithi184 ceph-mon[22331]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-08-31T06:04:40.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:39 smithi184 ceph-mon[22331]: Cluster is now healthy 2024-08-31T06:04:40.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:39 smithi184 ceph-mon[22331]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-08-31T06:04:40.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:39 smithi184 ceph-mon[22331]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:creating} 9 up:standby 2024-08-31T06:04:40.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:39 smithi184 ceph-mon[22331]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-08-31T06:04:40.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.271+0000 7f2d39a00640 1 -- 172.21.15.16:0/2935797261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f2d3419b350 con 0x7f2d34109260 2024-08-31T06:04:40.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:39 smithi155 ceph-mon[21742]: mds.? [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] up:active 2024-08-31T06:04:40.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:39 smithi155 ceph-mon[21742]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-08-31T06:04:40.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:39 smithi155 ceph-mon[21742]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-08-31T06:04:40.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:39 smithi155 ceph-mon[21742]: Cluster is now healthy 2024-08-31T06:04:40.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:39 smithi155 ceph-mon[21742]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-08-31T06:04:40.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:39 smithi155 ceph-mon[21742]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:creating} 9 up:standby 2024-08-31T06:04:40.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:39 smithi155 ceph-mon[21742]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-08-31T06:04:40.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.492+0000 7f2d2b400640 1 -- 172.21.15.16:0/2935797261 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7f2d3419b350 con 0x7f2d34109260 2024-08-31T06:04:40.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.495+0000 7f2d39a00640 1 -- 172.21.15.16:0/2935797261 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f2d180b4c00 msgr2=0x7f2d180b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:40.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.495+0000 7f2d39a00640 1 --2- 172.21.15.16:0/2935797261 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f2d180b4c00 0x7f2d180b7110 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2d24031440 tx=0x7f2d24009560 comp rx=0 tx=0).stop 2024-08-31T06:04:40.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.495+0000 7f2d39a00640 1 -- 172.21.15.16:0/2935797261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d34109260 msgr2=0x7f2d3419a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:40.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.495+0000 7f2d39a00640 1 --2- 172.21.15.16:0/2935797261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d34109260 0x7f2d3419a7c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2d2000d950 tx=0x7f2d2000de10 comp rx=0 tx=0).stop 2024-08-31T06:04:40.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.495+0000 7f2d39a00640 1 -- 172.21.15.16:0/2935797261 shutdown_connections 2024-08-31T06:04:40.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.495+0000 7f2d39a00640 1 --2- 172.21.15.16:0/2935797261 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f2d180b4c00 0x7f2d180b7110 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:40.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.495+0000 7f2d39a00640 1 --2- 172.21.15.16:0/2935797261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d34109bb0 0x7f2d3419fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:40.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.495+0000 7f2d39a00640 1 --2- 172.21.15.16:0/2935797261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d34109260 0x7f2d3419a7c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:40.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.495+0000 7f2d39a00640 1 --2- 172.21.15.16:0/2935797261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d34103920 0x7f2d3419a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:40.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.495+0000 7f2d39a00640 1 -- 172.21.15.16:0/2935797261 >> 172.21.15.16:0/2935797261 conn(0x7f2d340ff370 msgr2=0x7f2d34101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:04:40.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.496+0000 7f2d39a00640 1 -- 172.21.15.16:0/2935797261 shutdown_connections 2024-08-31T06:04:40.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:40.496+0000 7f2d39a00640 1 -- 172.21.15.16:0/2935797261 wait complete. 2024-08-31T06:04:41.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:40 smithi016 ceph-mon[17194]: pgmap v284: 97 pgs: 97 active+clean; 580 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-08-31T06:04:41.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:40 smithi016 ceph-mon[17194]: mds.? [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] up:active 2024-08-31T06:04:41.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:40 smithi016 ceph-mon[17194]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 9 up:standby 2024-08-31T06:04:41.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:40 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/2935797261' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-08-31T06:04:41.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:40 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/2935797261' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-08-31T06:04:41.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:40 smithi016 ceph-mon[17194]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:04:41.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:40 smithi184 ceph-mon[22331]: pgmap v284: 97 pgs: 97 active+clean; 580 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-08-31T06:04:41.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:40 smithi184 ceph-mon[22331]: mds.? [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] up:active 2024-08-31T06:04:41.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:40 smithi184 ceph-mon[22331]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 9 up:standby 2024-08-31T06:04:41.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:40 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/2935797261' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-08-31T06:04:41.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:40 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/2935797261' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-08-31T06:04:41.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:40 smithi184 ceph-mon[22331]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:04:41.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:40 smithi155 ceph-mon[21742]: pgmap v284: 97 pgs: 97 active+clean; 580 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-08-31T06:04:41.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:40 smithi155 ceph-mon[21742]: mds.? [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] up:active 2024-08-31T06:04:41.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:40 smithi155 ceph-mon[21742]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 9 up:standby 2024-08-31T06:04:41.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:40 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/2935797261' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-08-31T06:04:41.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:40 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/2935797261' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-08-31T06:04:41.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:40 smithi155 ceph-mon[21742]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:04:41.441 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-08-31T06:04:41.819 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:04:42.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.494+0000 7fc959000640 1 -- 172.21.15.16:0/800321880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc95410b460 msgr2=0x7fc95410b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:42.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.494+0000 7fc959000640 1 --2- 172.21.15.16:0/800321880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc95410b460 0x7fc95410b870 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc93c009900 tx=0x7fc93c02a410 comp rx=0 tx=0).stop 2024-08-31T06:04:42.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.494+0000 7fc959000640 1 -- 172.21.15.16:0/800321880 shutdown_connections 2024-08-31T06:04:42.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.494+0000 7fc959000640 1 --2- 172.21.15.16:0/800321880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc95410bdb0 0x7fc9541125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:42.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.494+0000 7fc959000640 1 --2- 172.21.15.16:0/800321880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc95410b460 0x7fc95410b870 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:42.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.494+0000 7fc959000640 1 --2- 172.21.15.16:0/800321880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc954105b20 0x7fc954105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:42.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.494+0000 7fc959000640 1 -- 172.21.15.16:0/800321880 >> 172.21.15.16:0/800321880 conn(0x7fc954101570 msgr2=0x7fc9541039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:04:42.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.494+0000 7fc959000640 1 -- 172.21.15.16:0/800321880 shutdown_connections 2024-08-31T06:04:42.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.495+0000 7fc959000640 1 -- 172.21.15.16:0/800321880 wait complete. 2024-08-31T06:04:42.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.496+0000 7fc959000640 1 Processor -- start 2024-08-31T06:04:42.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.496+0000 7fc959000640 1 -- start start 2024-08-31T06:04:42.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.496+0000 7fc959000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc954105b20 0x7fc95419c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:42.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.496+0000 7fc959000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc95410b460 0x7fc95419ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:42.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.496+0000 7fc959000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc95410bdb0 0x7fc9541a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:42.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.496+0000 7fc959000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc954115b00 con 0x7fc95410bdb0 2024-08-31T06:04:42.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.496+0000 7fc959000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc954115980 con 0x7fc954105b20 2024-08-31T06:04:42.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.496+0000 7fc959000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc954115c80 con 0x7fc95410b460 2024-08-31T06:04:42.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.497+0000 7fc952800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc95410bdb0 0x7fc9541a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:42.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.497+0000 7fc94be00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc95410b460 0x7fc95419ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:42.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.497+0000 7fc952800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc95410bdb0 0x7fc9541a1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34370/0 (socket says 172.21.15.16:34370) 2024-08-31T06:04:42.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.497+0000 7fc951e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc954105b20 0x7fc95419c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:42.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.497+0000 7fc94be00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc95410b460 0x7fc95419ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:46318/0 (socket says 172.21.15.16:46318) 2024-08-31T06:04:42.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.497+0000 7fc952800640 1 -- 172.21.15.16:0/3094698960 learned_addr learned my addr 172.21.15.16:0/3094698960 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:04:42.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.497+0000 7fc952800640 1 -- 172.21.15.16:0/3094698960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc95410b460 msgr2=0x7fc95419ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:42.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.497+0000 7fc952800640 1 --2- 172.21.15.16:0/3094698960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc95410b460 0x7fc95419ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:42.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.497+0000 7fc952800640 1 -- 172.21.15.16:0/3094698960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc954105b20 msgr2=0x7fc95419c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:42.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.497+0000 7fc952800640 1 --2- 172.21.15.16:0/3094698960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc954105b20 0x7fc95419c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:42.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.497+0000 7fc952800640 1 -- 172.21.15.16:0/3094698960 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9541a2460 con 0x7fc95410bdb0 2024-08-31T06:04:42.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.498+0000 7fc952800640 1 --2- 172.21.15.16:0/3094698960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc95410bdb0 0x7fc9541a1d30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc94c00eb10 tx=0x7fc94c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:04:42.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.498+0000 7fc94aa00640 1 -- 172.21.15.16:0/3094698960 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c00ce20 con 0x7fc95410bdb0 2024-08-31T06:04:42.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.498+0000 7fc94aa00640 1 -- 172.21.15.16:0/3094698960 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc94c005020 con 0x7fc95410bdb0 2024-08-31T06:04:42.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.498+0000 7fc959000640 1 -- 172.21.15.16:0/3094698960 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9541a2750 con 0x7fc95410bdb0 2024-08-31T06:04:42.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.498+0000 7fc94aa00640 1 -- 172.21.15.16:0/3094698960 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c010690 con 0x7fc95410bdb0 2024-08-31T06:04:42.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.498+0000 7fc959000640 1 -- 172.21.15.16:0/3094698960 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9541a8eb0 con 0x7fc95410bdb0 2024-08-31T06:04:42.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.499+0000 7fc959000640 1 -- 172.21.15.16:0/3094698960 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc954105f30 con 0x7fc95410bdb0 2024-08-31T06:04:42.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.501+0000 7fc94aa00640 1 -- 172.21.15.16:0/3094698960 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7fc94c005510 con 0x7fc95410bdb0 2024-08-31T06:04:42.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.503+0000 7fc94aa00640 1 --2- 172.21.15.16:0/3094698960 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc9380b4b80 0x7fc9380b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:42.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.503+0000 7fc94aa00640 1 -- 172.21.15.16:0/3094698960 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7fc94c0e58e0 con 0x7fc95410bdb0 2024-08-31T06:04:42.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.503+0000 7fc951e00640 1 --2- 172.21.15.16:0/3094698960 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc9380b4b80 0x7fc9380b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:42.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.504+0000 7fc951e00640 1 --2- 172.21.15.16:0/3094698960 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc9380b4b80 0x7fc9380b7090 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc944009990 tx=0x7fc944009110 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:04:42.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.507+0000 7fc94aa00640 1 -- 172.21.15.16:0/3094698960 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fc94c0ac9d0 con 0x7fc95410bdb0 2024-08-31T06:04:42.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.743+0000 7fc959000640 1 -- 172.21.15.16:0/3094698960 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fc95402e430 con 0x7fc95410bdb0 2024-08-31T06:04:42.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.938+0000 7fc94aa00640 1 -- 172.21.15.16:0/3094698960 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v24) ==== 115+0+0 (secure 0 0 0) 0x7fc94c0ac2d0 con 0x7fc95410bdb0 2024-08-31T06:04:42.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.941+0000 7fc959000640 1 -- 172.21.15.16:0/3094698960 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc9380b4b80 msgr2=0x7fc9380b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:42.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.941+0000 7fc959000640 1 --2- 172.21.15.16:0/3094698960 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc9380b4b80 0x7fc9380b7090 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc944009990 tx=0x7fc944009110 comp rx=0 tx=0).stop 2024-08-31T06:04:42.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.941+0000 7fc959000640 1 -- 172.21.15.16:0/3094698960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc95410bdb0 msgr2=0x7fc9541a1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:42.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.941+0000 7fc959000640 1 --2- 172.21.15.16:0/3094698960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc95410bdb0 0x7fc9541a1d30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc94c00eb10 tx=0x7fc94c00ef10 comp rx=0 tx=0).stop 2024-08-31T06:04:42.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.941+0000 7fc959000640 1 -- 172.21.15.16:0/3094698960 shutdown_connections 2024-08-31T06:04:42.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.941+0000 7fc959000640 1 --2- 172.21.15.16:0/3094698960 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc9380b4b80 0x7fc9380b7090 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:42.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.941+0000 7fc959000640 1 --2- 172.21.15.16:0/3094698960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc95410b460 0x7fc95419ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:42.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.941+0000 7fc959000640 1 --2- 172.21.15.16:0/3094698960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc95410bdb0 0x7fc9541a1d30 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:42.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.941+0000 7fc959000640 1 --2- 172.21.15.16:0/3094698960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc954105b20 0x7fc95419c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:42.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.941+0000 7fc959000640 1 -- 172.21.15.16:0/3094698960 >> 172.21.15.16:0/3094698960 conn(0x7fc954101570 msgr2=0x7fc954103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:04:42.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.941+0000 7fc959000640 1 -- 172.21.15.16:0/3094698960 shutdown_connections 2024-08-31T06:04:42.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:42.941+0000 7fc959000640 1 -- 172.21.15.16:0/3094698960 wait complete. 2024-08-31T06:04:43.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:42 smithi016 ceph-mon[17194]: pgmap v285: 97 pgs: 97 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 6 op/s 2024-08-31T06:04:43.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:42 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/3094698960' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-08-31T06:04:43.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:42 smithi184 ceph-mon[22331]: pgmap v285: 97 pgs: 97 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 6 op/s 2024-08-31T06:04:43.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:42 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/3094698960' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-08-31T06:04:43.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:42 smithi155 ceph-mon[21742]: pgmap v285: 97 pgs: 97 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 6 op/s 2024-08-31T06:04:43.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:42 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/3094698960' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-08-31T06:04:43.758 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-08-31T06:04:43.759 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-08-31T06:04:44.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:43 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/3094698960' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-08-31T06:04:44.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:43 smithi155 ceph-mon[21742]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:04:44.122 INFO:journalctl@ceph.mds.b.smithi155.stdout:Aug 31 06:04:43 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-b[53353]: 2024-08-31T06:04:43.855+0000 7f6c202fe640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-08-31T06:04:44.122 INFO:journalctl@ceph.mds.b.smithi155.stdout:Aug 31 06:04:43 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-b[53353]: 2024-08-31T06:04:43.855+0000 7f6c202fe640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-08-31T06:04:44.141 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:04:44.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:43 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/3094698960' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-08-31T06:04:44.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:43 smithi016 ceph-mon[17194]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:04:44.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:43 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/3094698960' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-08-31T06:04:44.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:43 smithi184 ceph-mon[22331]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:04:44.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:44.777+0000 7fd1c7600640 1 -- 172.21.15.16:0/3162458431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1c0105b20 msgr2=0x7fd1c0105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:44.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:44.777+0000 7fd1c7600640 1 --2- 172.21.15.16:0/3162458431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1c0105b20 0x7fd1c0105f30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0009900 tx=0x7fd1b002a3f0 comp rx=0 tx=0).stop 2024-08-31T06:04:44.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:44.778+0000 7fd1c7600640 1 -- 172.21.15.16:0/3162458431 shutdown_connections 2024-08-31T06:04:44.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:44.778+0000 7fd1c7600640 1 --2- 172.21.15.16:0/3162458431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1c010bdb0 0x7fd1c01125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:44.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:44.778+0000 7fd1c7600640 1 --2- 172.21.15.16:0/3162458431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1c010b460 0x7fd1c010b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:44.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:44.778+0000 7fd1c7600640 1 --2- 172.21.15.16:0/3162458431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1c0105b20 0x7fd1c0105f30 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:44.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:44.778+0000 7fd1c7600640 1 -- 172.21.15.16:0/3162458431 >> 172.21.15.16:0/3162458431 conn(0x7fd1c01015b0 msgr2=0x7fd1c0103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:04:44.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:44.778+0000 7fd1c7600640 1 -- 172.21.15.16:0/3162458431 shutdown_connections 2024-08-31T06:04:44.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:44.778+0000 7fd1c7600640 1 -- 172.21.15.16:0/3162458431 wait complete. 2024-08-31T06:04:44.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:44.779+0000 7fd1c7600640 1 Processor -- start 2024-08-31T06:04:44.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:44.779+0000 7fd1c7600640 1 -- start start 2024-08-31T06:04:44.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:44.779+0000 7fd1c7600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1c0105b20 0x7fd1c019c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:44.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:44.779+0000 7fd1c7600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1c010b460 0x7fd1c019c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:44.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:44.779+0000 7fd1c7600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1c010bdb0 0x7fd1c01a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:44.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:44.779+0000 7fd1c7600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c0115a70 con 0x7fd1c0105b20 2024-08-31T06:04:44.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:44.779+0000 7fd1c7600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c01158f0 con 0x7fd1c010b460 2024-08-31T06:04:44.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:44.779+0000 7fd1c7600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c0115bf0 con 0x7fd1c010bdb0 2024-08-31T06:04:44.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:44.780+0000 7fd1c4c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1c0105b20 0x7fd1c019c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:44.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:44.780+0000 7fd1c4c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1c0105b20 0x7fd1c019c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34388/0 (socket says 172.21.15.16:34388) 2024-08-31T06:04:44.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:44.780+0000 7fd1c4c00640 1 -- 172.21.15.16:0/2115963987 learned_addr learned my addr 172.21.15.16:0/2115963987 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:04:44.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:44.780+0000 7fd1c5600640 1 --2- 172.21.15.16:0/2115963987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1c010bdb0 0x7fd1c01a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:44.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:44.780+0000 7fd1b7e00640 1 --2- 172.21.15.16:0/2115963987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1c010b460 0x7fd1c019c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:44.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:44.780+0000 7fd1c4c00640 1 -- 172.21.15.16:0/2115963987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1c010bdb0 msgr2=0x7fd1c01a1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:44.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:44.780+0000 7fd1c4c00640 1 --2- 172.21.15.16:0/2115963987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1c010bdb0 0x7fd1c01a1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:44.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:44.780+0000 7fd1c4c00640 1 -- 172.21.15.16:0/2115963987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1c010b460 msgr2=0x7fd1c019c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:44.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:44.780+0000 7fd1c4c00640 1 --2- 172.21.15.16:0/2115963987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1c010b460 0x7fd1c019c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:44.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:44.780+0000 7fd1c4c00640 1 -- 172.21.15.16:0/2115963987 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1c01a23b0 con 0x7fd1c0105b20 2024-08-31T06:04:44.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:44.781+0000 7fd1c4c00640 1 --2- 172.21.15.16:0/2115963987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1c0105b20 0x7fd1c019c440 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0005940 tx=0x7fd1b00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:04:44.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:44.781+0000 7fd1be000640 1 -- 172.21.15.16:0/2115963987 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b003f030 con 0x7fd1c0105b20 2024-08-31T06:04:44.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:44.781+0000 7fd1be000640 1 -- 172.21.15.16:0/2115963987 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1b0003cb0 con 0x7fd1c0105b20 2024-08-31T06:04:44.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:44.781+0000 7fd1be000640 1 -- 172.21.15.16:0/2115963987 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b0035070 con 0x7fd1c0105b20 2024-08-31T06:04:44.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:44.781+0000 7fd1c7600640 1 -- 172.21.15.16:0/2115963987 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1c01a2640 con 0x7fd1c0105b20 2024-08-31T06:04:44.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:44.781+0000 7fd1c7600640 1 -- 172.21.15.16:0/2115963987 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1c01a8e00 con 0x7fd1c0105b20 2024-08-31T06:04:44.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:44.782+0000 7fd1c7600640 1 -- 172.21.15.16:0/2115963987 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1c0105f30 con 0x7fd1c0105b20 2024-08-31T06:04:44.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:44.785+0000 7fd1be000640 1 -- 172.21.15.16:0/2115963987 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) ==== 152594+0+0 (secure 0 0 0) 0x7fd1b0042030 con 0x7fd1c0105b20 2024-08-31T06:04:44.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:44.787+0000 7fd1be000640 1 --2- 172.21.15.16:0/2115963987 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd1a40b4b80 0x7fd1a40b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:44.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:44.787+0000 7fd1be000640 1 -- 172.21.15.16:0/2115963987 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7fd1b0105bc0 con 0x7fd1c0105b20 2024-08-31T06:04:44.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:44.787+0000 7fd1b7e00640 1 --2- 172.21.15.16:0/2115963987 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd1a40b4b80 0x7fd1a40b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:44.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:44.788+0000 7fd1b7e00640 1 --2- 172.21.15.16:0/2115963987 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd1a40b4b80 0x7fd1a40b7090 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd1c010af80 tx=0x7fd1a800b410 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:04:44.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:44.790+0000 7fd1be000640 1 -- 172.21.15.16:0/2115963987 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fd1b00cc6c0 con 0x7fd1c0105b20 2024-08-31T06:04:45.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:45.015+0000 7fd1c7600640 1 -- 172.21.15.16:0/2115963987 --> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fd1c00635a0 con 0x7fd1a40b4b80 2024-08-31T06:04:45.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:45.050+0000 7fd1be000640 1 -- 172.21.15.16:0/2115963987 <== mgr.14220 v2:172.21.15.16:6800/4279751337 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd1c00635a0 con 0x7fd1a40b4b80 2024-08-31T06:04:45.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:45.052+0000 7fd1c7600640 1 -- 172.21.15.16:0/2115963987 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd1a40b4b80 msgr2=0x7fd1a40b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:45.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:45.052+0000 7fd1c7600640 1 --2- 172.21.15.16:0/2115963987 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd1a40b4b80 0x7fd1a40b7090 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd1c010af80 tx=0x7fd1a800b410 comp rx=0 tx=0).stop 2024-08-31T06:04:45.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:45.052+0000 7fd1c7600640 1 -- 172.21.15.16:0/2115963987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1c0105b20 msgr2=0x7fd1c019c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:45.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:45.052+0000 7fd1c7600640 1 --2- 172.21.15.16:0/2115963987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1c0105b20 0x7fd1c019c440 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0005940 tx=0x7fd1b00042f0 comp rx=0 tx=0).stop 2024-08-31T06:04:45.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:45.052+0000 7fd1c7600640 1 -- 172.21.15.16:0/2115963987 shutdown_connections 2024-08-31T06:04:45.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:45.052+0000 7fd1c7600640 1 --2- 172.21.15.16:0/2115963987 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd1a40b4b80 0x7fd1a40b7090 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:45.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:45.052+0000 7fd1c7600640 1 --2- 172.21.15.16:0/2115963987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1c010bdb0 0x7fd1c01a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:45.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:45.052+0000 7fd1c7600640 1 --2- 172.21.15.16:0/2115963987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1c010b460 0x7fd1c019c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:45.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:45.052+0000 7fd1c7600640 1 --2- 172.21.15.16:0/2115963987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1c0105b20 0x7fd1c019c440 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:45.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:45.052+0000 7fd1c7600640 1 -- 172.21.15.16:0/2115963987 >> 172.21.15.16:0/2115963987 conn(0x7fd1c01015b0 msgr2=0x7fd1c01039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:04:45.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:45.052+0000 7fd1c7600640 1 -- 172.21.15.16:0/2115963987 shutdown_connections 2024-08-31T06:04:45.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:45.052+0000 7fd1c7600640 1 -- 172.21.15.16:0/2115963987 wait complete. 2024-08-31T06:04:45.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:44 smithi184 ceph-mon[22331]: pgmap v286: 97 pgs: 97 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 2.1 KiB/s wr, 9 op/s 2024-08-31T06:04:45.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:44 smithi155 ceph-mon[21742]: pgmap v286: 97 pgs: 97 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 2.1 KiB/s wr, 9 op/s 2024-08-31T06:04:45.406 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:44 smithi016 ceph-mon[17194]: pgmap v286: 97 pgs: 97 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 2.1 KiB/s wr, 9 op/s 2024-08-31T06:04:45.406 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 06:04:45 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: 2024-08-31T06:04:45.022+0000 7fdeaae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-31T06:04:45.407 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 06:04:45 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: 2024-08-31T06:04:45.022+0000 7fdeaae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-31T06:04:45.407 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 06:04:45 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: 2024-08-31T06:04:45.022+0000 7fdeaae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-31T06:04:45.407 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 06:04:45 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: 2024-08-31T06:04:45.022+0000 7fdeaae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-31T06:04:45.407 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 06:04:45 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: 2024-08-31T06:04:45.022+0000 7fdeaae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-31T06:04:45.957 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-08-31T06:04:46.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:45 smithi184 ceph-mon[22331]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:04:46.342 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:04:46.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:45 smithi155 ceph-mon[21742]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:04:46.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:45 smithi016 ceph-mon[17194]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:04:47.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.020+0000 7f7125400640 1 -- 172.21.15.16:0/3069923137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7120105b20 msgr2=0x7f7120105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:47.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.020+0000 7f7125400640 1 --2- 172.21.15.16:0/3069923137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7120105b20 0x7f7120105f30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7110009900 tx=0x7f711002a410 comp rx=0 tx=0).stop 2024-08-31T06:04:47.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.020+0000 7f7125400640 1 -- 172.21.15.16:0/3069923137 shutdown_connections 2024-08-31T06:04:47.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.020+0000 7f7125400640 1 --2- 172.21.15.16:0/3069923137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f712010b460 0x7f712010b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:47.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.020+0000 7f7125400640 1 --2- 172.21.15.16:0/3069923137 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f712010bdb0 0x7f71201125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:47.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.020+0000 7f7125400640 1 --2- 172.21.15.16:0/3069923137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7120105b20 0x7f7120105f30 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:47.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.020+0000 7f7125400640 1 -- 172.21.15.16:0/3069923137 >> 172.21.15.16:0/3069923137 conn(0x7f71201015b0 msgr2=0x7f7120103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:04:47.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.020+0000 7f7125400640 1 -- 172.21.15.16:0/3069923137 shutdown_connections 2024-08-31T06:04:47.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.020+0000 7f7125400640 1 -- 172.21.15.16:0/3069923137 wait complete. 2024-08-31T06:04:47.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.021+0000 7f7125400640 1 Processor -- start 2024-08-31T06:04:47.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.021+0000 7f7125400640 1 -- start start 2024-08-31T06:04:47.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.022+0000 7f7125400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7120105b20 0x7f712019c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:47.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.022+0000 7f7125400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f712010b460 0x7f712019ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:47.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.022+0000 7f7125400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f712010bdb0 0x7f71201a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:47.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.022+0000 7f7125400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7120115ac0 con 0x7f712010b460 2024-08-31T06:04:47.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.022+0000 7f7125400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7120115940 con 0x7f712010bdb0 2024-08-31T06:04:47.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.022+0000 7f7125400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7120115c40 con 0x7f7120105b20 2024-08-31T06:04:47.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.022+0000 7f7117e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f712010b460 0x7f712019ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:47.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.022+0000 7f711e800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7120105b20 0x7f712019c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:47.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.022+0000 7f7117e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f712010b460 0x7f712019ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37448/0 (socket says 172.21.15.16:37448) 2024-08-31T06:04:47.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.022+0000 7f7117e00640 1 -- 172.21.15.16:0/3822641510 learned_addr learned my addr 172.21.15.16:0/3822641510 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:04:47.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.022+0000 7f711f200640 1 --2- 172.21.15.16:0/3822641510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f712010bdb0 0x7f71201a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:47.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.023+0000 7f711e800640 1 -- 172.21.15.16:0/3822641510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f712010bdb0 msgr2=0x7f71201a1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:47.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.023+0000 7f711e800640 1 --2- 172.21.15.16:0/3822641510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f712010bdb0 0x7f71201a1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:47.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.023+0000 7f711e800640 1 -- 172.21.15.16:0/3822641510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f712010b460 msgr2=0x7f712019ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:47.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.023+0000 7f711e800640 1 --2- 172.21.15.16:0/3822641510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f712010b460 0x7f712019ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:47.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.023+0000 7f711e800640 1 -- 172.21.15.16:0/3822641510 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71201a2460 con 0x7f7120105b20 2024-08-31T06:04:47.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.023+0000 7f7117e00640 1 --2- 172.21.15.16:0/3822641510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f712010b460 0x7f712019ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:04:47.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.023+0000 7f711e800640 1 --2- 172.21.15.16:0/3822641510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7120105b20 0x7f712019c4f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f711002aa00 tx=0x7f7110003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:04:47.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.023+0000 7f7117400640 1 -- 172.21.15.16:0/3822641510 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711003f070 con 0x7f7120105b20 2024-08-31T06:04:47.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.023+0000 7f7125400640 1 -- 172.21.15.16:0/3822641510 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71201a26f0 con 0x7f7120105b20 2024-08-31T06:04:47.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.023+0000 7f7117400640 1 -- 172.21.15.16:0/3822641510 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7110031b50 con 0x7f7120105b20 2024-08-31T06:04:47.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.023+0000 7f7117400640 1 -- 172.21.15.16:0/3822641510 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7110035030 con 0x7f7120105b20 2024-08-31T06:04:47.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.023+0000 7f7125400640 1 -- 172.21.15.16:0/3822641510 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71201a8eb0 con 0x7f7120105b20 2024-08-31T06:04:47.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.024+0000 7f7125400640 1 -- 172.21.15.16:0/3822641510 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7120106da0 con 0x7f7120105b20 2024-08-31T06:04:47.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.027+0000 7f7117400640 1 -- 172.21.15.16:0/3822641510 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f7110042070 con 0x7f7120105b20 2024-08-31T06:04:47.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.028+0000 7f7117400640 1 --2- 172.21.15.16:0/3822641510 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f71040b4c10 0x7f71040b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:47.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.028+0000 7f7117400640 1 -- 172.21.15.16:0/3822641510 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7f7110105ad0 con 0x7f7120105b20 2024-08-31T06:04:47.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.028+0000 7f7117e00640 1 --2- 172.21.15.16:0/3822641510 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f71040b4c10 0x7f71040b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:47.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.029+0000 7f7117e00640 1 --2- 172.21.15.16:0/3822641510 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f71040b4c10 0x7f71040b7120 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f712010a070 tx=0x7f7108005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:04:47.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.030+0000 7f7117400640 1 -- 172.21.15.16:0/3822641510 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f7110031d00 con 0x7f7120105b20 2024-08-31T06:04:47.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:46 smithi016 ceph-mon[17194]: pgmap v287: 97 pgs: 97 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-08-31T06:04:47.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:46 smithi016 ceph-mon[17194]: mgrmap e21: x(active, since 7m), standbys: z, y 2024-08-31T06:04:47.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:46 smithi184 ceph-mon[22331]: pgmap v287: 97 pgs: 97 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-08-31T06:04:47.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:46 smithi184 ceph-mon[22331]: mgrmap e21: x(active, since 7m), standbys: z, y 2024-08-31T06:04:47.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.251+0000 7f7125400640 1 -- 172.21.15.16:0/3822641510 --> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f712010dbe0 con 0x7f71040b4c10 2024-08-31T06:04:47.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.280+0000 7f7117400640 1 -- 172.21.15.16:0/3822641510 <== mgr.14220 v2:172.21.15.16:6800/4279751337 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f712010dbe0 con 0x7f71040b4c10 2024-08-31T06:04:47.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.282+0000 7f7125400640 1 -- 172.21.15.16:0/3822641510 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f71040b4c10 msgr2=0x7f71040b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:47.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.282+0000 7f7125400640 1 --2- 172.21.15.16:0/3822641510 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f71040b4c10 0x7f71040b7120 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f712010a070 tx=0x7f7108005f90 comp rx=0 tx=0).stop 2024-08-31T06:04:47.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.282+0000 7f7125400640 1 -- 172.21.15.16:0/3822641510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7120105b20 msgr2=0x7f712019c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:47.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.282+0000 7f7125400640 1 --2- 172.21.15.16:0/3822641510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7120105b20 0x7f712019c4f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f711002aa00 tx=0x7f7110003b90 comp rx=0 tx=0).stop 2024-08-31T06:04:47.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.282+0000 7f7125400640 1 -- 172.21.15.16:0/3822641510 shutdown_connections 2024-08-31T06:04:47.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.282+0000 7f7125400640 1 --2- 172.21.15.16:0/3822641510 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f71040b4c10 0x7f71040b7120 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:47.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.282+0000 7f7125400640 1 --2- 172.21.15.16:0/3822641510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f712010bdb0 0x7f71201a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:47.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.282+0000 7f7125400640 1 --2- 172.21.15.16:0/3822641510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f712010b460 0x7f712019ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:47.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.282+0000 7f7125400640 1 --2- 172.21.15.16:0/3822641510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7120105b20 0x7f712019c4f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:47.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.282+0000 7f7125400640 1 -- 172.21.15.16:0/3822641510 >> 172.21.15.16:0/3822641510 conn(0x7f71201015b0 msgr2=0x7f71201039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:04:47.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.283+0000 7f7125400640 1 -- 172.21.15.16:0/3822641510 shutdown_connections 2024-08-31T06:04:47.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:47.283+0000 7f7125400640 1 -- 172.21.15.16:0/3822641510 wait complete. 2024-08-31T06:04:47.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:46 smithi155 ceph-mon[21742]: pgmap v287: 97 pgs: 97 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-08-31T06:04:47.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:46 smithi155 ceph-mon[21742]: mgrmap e21: x(active, since 7m), standbys: z, y 2024-08-31T06:04:48.044 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-08-31T06:04:48.424 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:04:49.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.072+0000 7f1cdbe00640 1 -- 172.21.15.16:0/1993509516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cd410b460 msgr2=0x7f1cd410b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:49.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.072+0000 7f1cdbe00640 1 --2- 172.21.15.16:0/1993509516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cd410b460 0x7f1cd410b870 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0009900 tx=0x7f1cc002a410 comp rx=0 tx=0).stop 2024-08-31T06:04:49.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.072+0000 7f1cdbe00640 1 -- 172.21.15.16:0/1993509516 shutdown_connections 2024-08-31T06:04:49.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.072+0000 7f1cdbe00640 1 --2- 172.21.15.16:0/1993509516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1cd410bdb0 0x7f1cd41125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:49.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.072+0000 7f1cdbe00640 1 --2- 172.21.15.16:0/1993509516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cd410b460 0x7f1cd410b870 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:49.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.072+0000 7f1cdbe00640 1 --2- 172.21.15.16:0/1993509516 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1cd4105b20 0x7f1cd4105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:49.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.072+0000 7f1cdbe00640 1 -- 172.21.15.16:0/1993509516 >> 172.21.15.16:0/1993509516 conn(0x7f1cd41015b0 msgr2=0x7f1cd4103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:04:49.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.072+0000 7f1cdbe00640 1 -- 172.21.15.16:0/1993509516 shutdown_connections 2024-08-31T06:04:49.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.072+0000 7f1cdbe00640 1 -- 172.21.15.16:0/1993509516 wait complete. 2024-08-31T06:04:49.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.073+0000 7f1cdbe00640 1 Processor -- start 2024-08-31T06:04:49.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.074+0000 7f1cdbe00640 1 -- start start 2024-08-31T06:04:49.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.074+0000 7f1cdbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cd4105b20 0x7f1cd419c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:49.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.074+0000 7f1cdbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1cd410b460 0x7f1cd419caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:49.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.074+0000 7f1cdbe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1cd410bdb0 0x7f1cd41a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:49.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.074+0000 7f1cdbe00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd4115ae0 con 0x7f1cd4105b20 2024-08-31T06:04:49.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.074+0000 7f1cdbe00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd4115960 con 0x7f1cd410b460 2024-08-31T06:04:49.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.074+0000 7f1cdbe00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd4115c60 con 0x7f1cd410bdb0 2024-08-31T06:04:49.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.074+0000 7f1cd9400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cd4105b20 0x7f1cd419c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:49.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.074+0000 7f1cd9400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cd4105b20 0x7f1cd419c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37460/0 (socket says 172.21.15.16:37460) 2024-08-31T06:04:49.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.074+0000 7f1cd9400640 1 -- 172.21.15.16:0/1770355890 learned_addr learned my addr 172.21.15.16:0/1770355890 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:04:49.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.075+0000 7f1cd9e00640 1 --2- 172.21.15.16:0/1770355890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1cd410bdb0 0x7f1cd41a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:49.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.075+0000 7f1cd9400640 1 -- 172.21.15.16:0/1770355890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1cd410bdb0 msgr2=0x7f1cd41a1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:49.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.075+0000 7f1cd9400640 1 --2- 172.21.15.16:0/1770355890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1cd410bdb0 0x7f1cd41a1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:49.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.075+0000 7f1cd9400640 1 -- 172.21.15.16:0/1770355890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1cd410b460 msgr2=0x7f1cd419caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:49.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.075+0000 7f1cd9400640 1 --2- 172.21.15.16:0/1770355890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1cd410b460 0x7f1cd419caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:49.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.075+0000 7f1cd9400640 1 -- 172.21.15.16:0/1770355890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd41a24c0 con 0x7f1cd4105b20 2024-08-31T06:04:49.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.075+0000 7f1cd9400640 1 --2- 172.21.15.16:0/1770355890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cd4105b20 0x7f1cd419c550 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1cc800d950 tx=0x7f1cc800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:04:49.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.075+0000 7f1cd2000640 1 -- 172.21.15.16:0/1770355890 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8017070 con 0x7f1cd4105b20 2024-08-31T06:04:49.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.076+0000 7f1cd2000640 1 -- 172.21.15.16:0/1770355890 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cc8009c30 con 0x7f1cd4105b20 2024-08-31T06:04:49.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.076+0000 7f1cdbe00640 1 -- 172.21.15.16:0/1770355890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cd41a27b0 con 0x7f1cd4105b20 2024-08-31T06:04:49.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.076+0000 7f1cd2000640 1 -- 172.21.15.16:0/1770355890 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc800f620 con 0x7f1cd4105b20 2024-08-31T06:04:49.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.076+0000 7f1cdbe00640 1 -- 172.21.15.16:0/1770355890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cd41a8f10 con 0x7f1cd4105b20 2024-08-31T06:04:49.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.077+0000 7f1cdbe00640 1 -- 172.21.15.16:0/1770355890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cd4105f30 con 0x7f1cd4105b20 2024-08-31T06:04:49.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.079+0000 7f1cd2000640 1 -- 172.21.15.16:0/1770355890 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f1cc8010460 con 0x7f1cd4105b20 2024-08-31T06:04:49.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.081+0000 7f1cd2000640 1 --2- 172.21.15.16:0/1770355890 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f1cbc0b4c90 0x7f1cbc0b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:49.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.081+0000 7f1cd2000640 1 -- 172.21.15.16:0/1770355890 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7f1cc80e4f30 con 0x7f1cd4105b20 2024-08-31T06:04:49.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.081+0000 7f1cd8a00640 1 --2- 172.21.15.16:0/1770355890 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f1cbc0b4c90 0x7f1cbc0b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:49.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.082+0000 7f1cd8a00640 1 --2- 172.21.15.16:0/1770355890 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f1cbc0b4c90 0x7f1cbc0b71a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0009230 tx=0x7f1cc0031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:04:49.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.085+0000 7f1cd2000640 1 -- 172.21.15.16:0/1770355890 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f1cc80abec0 con 0x7f1cd4105b20 2024-08-31T06:04:49.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:48 smithi016 ceph-mon[17194]: from='client.24820 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:04:49.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:48 smithi016 ceph-mon[17194]: pgmap v288: 97 pgs: 97 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-08-31T06:04:49.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:48 smithi184 ceph-mon[22331]: from='client.24820 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:04:49.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:48 smithi184 ceph-mon[22331]: pgmap v288: 97 pgs: 97 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-08-31T06:04:49.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.310+0000 7f1cdbe00640 1 -- 172.21.15.16:0/1770355890 --> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f1cd4063630 con 0x7f1cbc0b4c90 2024-08-31T06:04:49.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.316+0000 7f1cd2000640 1 -- 172.21.15.16:0/1770355890 <== mgr.14220 v2:172.21.15.16:6800/4279751337 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f1cd4063630 con 0x7f1cbc0b4c90 2024-08-31T06:04:49.318 INFO:teuthology.orchestra.run.smithi016.stdout:/volumes/qa/sv_0/dd3ac3cc-4fe5-4fda-829a-5fd35e562726 2024-08-31T06:04:49.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.318+0000 7f1cdbe00640 1 -- 172.21.15.16:0/1770355890 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f1cbc0b4c90 msgr2=0x7f1cbc0b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:49.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.318+0000 7f1cdbe00640 1 --2- 172.21.15.16:0/1770355890 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f1cbc0b4c90 0x7f1cbc0b71a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0009230 tx=0x7f1cc0031400 comp rx=0 tx=0).stop 2024-08-31T06:04:49.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.318+0000 7f1cdbe00640 1 -- 172.21.15.16:0/1770355890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cd4105b20 msgr2=0x7f1cd419c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:49.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.318+0000 7f1cdbe00640 1 --2- 172.21.15.16:0/1770355890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cd4105b20 0x7f1cd419c550 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1cc800d950 tx=0x7f1cc800de10 comp rx=0 tx=0).stop 2024-08-31T06:04:49.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.318+0000 7f1cdbe00640 1 -- 172.21.15.16:0/1770355890 shutdown_connections 2024-08-31T06:04:49.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.318+0000 7f1cdbe00640 1 --2- 172.21.15.16:0/1770355890 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f1cbc0b4c90 0x7f1cbc0b71a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:49.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.318+0000 7f1cdbe00640 1 --2- 172.21.15.16:0/1770355890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1cd410bdb0 0x7f1cd41a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:49.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.318+0000 7f1cdbe00640 1 --2- 172.21.15.16:0/1770355890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1cd410b460 0x7f1cd419caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:49.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.318+0000 7f1cdbe00640 1 --2- 172.21.15.16:0/1770355890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cd4105b20 0x7f1cd419c550 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:49.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.318+0000 7f1cdbe00640 1 -- 172.21.15.16:0/1770355890 >> 172.21.15.16:0/1770355890 conn(0x7f1cd41015b0 msgr2=0x7f1cd410aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:04:49.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.318+0000 7f1cdbe00640 1 -- 172.21.15.16:0/1770355890 shutdown_connections 2024-08-31T06:04:49.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:49.318+0000 7f1cdbe00640 1 -- 172.21.15.16:0/1770355890 wait complete. 2024-08-31T06:04:49.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:48 smithi155 ceph-mon[21742]: from='client.24820 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:04:49.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:48 smithi155 ceph-mon[21742]: pgmap v288: 97 pgs: 97 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-08-31T06:04:50.236 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-08-31T06:04:50.612 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:04:51.067 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:50 smithi016 ceph-mon[17194]: pgmap v289: 97 pgs: 97 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-08-31T06:04:51.068 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:50 smithi016 ceph-mon[17194]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:04:51.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.248+0000 7fc431200640 1 -- 172.21.15.16:0/3265642336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc42c1141d0 msgr2=0x7fc42c1166a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:51.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.248+0000 7fc431200640 1 --2- 172.21.15.16:0/3265642336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc42c1141d0 0x7fc42c1166a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc424009ca0 tx=0x7fc42402a8c0 comp rx=0 tx=0).stop 2024-08-31T06:04:51.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.248+0000 7fc431200640 1 -- 172.21.15.16:0/3265642336 shutdown_connections 2024-08-31T06:04:51.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.248+0000 7fc431200640 1 --2- 172.21.15.16:0/3265642336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc42c1141d0 0x7fc42c1166a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:51.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.248+0000 7fc431200640 1 --2- 172.21.15.16:0/3265642336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc42c075b70 0x7fc42c075f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:51.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.248+0000 7fc431200640 1 --2- 172.21.15.16:0/3265642336 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc42c077120 0x7fc42c075550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:51.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.248+0000 7fc431200640 1 -- 172.21.15.16:0/3265642336 >> 172.21.15.16:0/3265642336 conn(0x7fc42c07b5f0 msgr2=0x7fc42c07ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:04:51.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.248+0000 7fc431200640 1 -- 172.21.15.16:0/3265642336 shutdown_connections 2024-08-31T06:04:51.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.248+0000 7fc431200640 1 -- 172.21.15.16:0/3265642336 wait complete. 2024-08-31T06:04:51.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.249+0000 7fc431200640 1 Processor -- start 2024-08-31T06:04:51.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.249+0000 7fc431200640 1 -- start start 2024-08-31T06:04:51.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.250+0000 7fc431200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc42c075b70 0x7fc42c1a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:51.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.250+0000 7fc431200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc42c077120 0x7fc42c1a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:51.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.250+0000 7fc431200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc42c1141d0 0x7fc42c1a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:51.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.250+0000 7fc431200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c119e60 con 0x7fc42c1141d0 2024-08-31T06:04:51.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.250+0000 7fc431200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c119ce0 con 0x7fc42c077120 2024-08-31T06:04:51.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.250+0000 7fc431200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c119fe0 con 0x7fc42c075b70 2024-08-31T06:04:51.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.250+0000 7fc42a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc42c1141d0 0x7fc42c1a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:51.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.250+0000 7fc429e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc42c075b70 0x7fc42c1a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:51.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.250+0000 7fc423e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc42c077120 0x7fc42c1a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:51.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.250+0000 7fc429e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc42c075b70 0x7fc42c1a0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:49144/0 (socket says 172.21.15.16:49144) 2024-08-31T06:04:51.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.250+0000 7fc429e00640 1 -- 172.21.15.16:0/906621164 learned_addr learned my addr 172.21.15.16:0/906621164 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:04:51.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.250+0000 7fc42a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc42c1141d0 0x7fc42c1a6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37476/0 (socket says 172.21.15.16:37476) 2024-08-31T06:04:51.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.250+0000 7fc423e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc42c077120 0x7fc42c1a0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:39280/0 (socket says 172.21.15.16:39280) 2024-08-31T06:04:51.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.251+0000 7fc429e00640 1 -- 172.21.15.16:0/906621164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc42c077120 msgr2=0x7fc42c1a0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:51.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.251+0000 7fc429e00640 1 --2- 172.21.15.16:0/906621164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc42c077120 0x7fc42c1a0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:51.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.251+0000 7fc429e00640 1 -- 172.21.15.16:0/906621164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc42c1141d0 msgr2=0x7fc42c1a6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:51.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.251+0000 7fc429e00640 1 --2- 172.21.15.16:0/906621164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc42c1141d0 0x7fc42c1a6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:51.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.251+0000 7fc429e00640 1 -- 172.21.15.16:0/906621164 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc42c1a67a0 con 0x7fc42c075b70 2024-08-31T06:04:51.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.251+0000 7fc429e00640 1 --2- 172.21.15.16:0/906621164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc42c075b70 0x7fc42c1a0830 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc41c00e9f0 tx=0x7fc41c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:04:51.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.251+0000 7fc422a00640 1 -- 172.21.15.16:0/906621164 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c00cd80 con 0x7fc42c075b70 2024-08-31T06:04:51.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.251+0000 7fc431200640 1 -- 172.21.15.16:0/906621164 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc42c1a6a90 con 0x7fc42c075b70 2024-08-31T06:04:51.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.251+0000 7fc422a00640 1 -- 172.21.15.16:0/906621164 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc41c0054d0 con 0x7fc42c075b70 2024-08-31T06:04:51.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.251+0000 7fc422a00640 1 -- 172.21.15.16:0/906621164 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c010690 con 0x7fc42c075b70 2024-08-31T06:04:51.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.252+0000 7fc431200640 1 -- 172.21.15.16:0/906621164 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc42c1ad1f0 con 0x7fc42c075b70 2024-08-31T06:04:51.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.253+0000 7fc431200640 1 -- 172.21.15.16:0/906621164 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc42c075f80 con 0x7fc42c075b70 2024-08-31T06:04:51.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.255+0000 7fc422a00640 1 -- 172.21.15.16:0/906621164 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7fc41c004e90 con 0x7fc42c075b70 2024-08-31T06:04:51.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.256+0000 7fc422a00640 1 --2- 172.21.15.16:0/906621164 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc4100b4c10 0x7fc4100b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:51.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.257+0000 7fc422a00640 1 -- 172.21.15.16:0/906621164 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7fc41c0e56a0 con 0x7fc42c075b70 2024-08-31T06:04:51.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.257+0000 7fc423e00640 1 --2- 172.21.15.16:0/906621164 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc4100b4c10 0x7fc4100b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:51.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.257+0000 7fc423e00640 1 --2- 172.21.15.16:0/906621164 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc4100b4c10 0x7fc4100b7120 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc42c115660 tx=0x7fc414005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:04:51.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.258+0000 7fc422a00640 1 -- 172.21.15.16:0/906621164 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fc41c0ac630 con 0x7fc42c075b70 2024-08-31T06:04:51.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:50 smithi155 ceph-mon[21742]: pgmap v289: 97 pgs: 97 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-08-31T06:04:51.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:50 smithi155 ceph-mon[21742]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:04:51.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:50 smithi184 ceph-mon[22331]: pgmap v289: 97 pgs: 97 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-08-31T06:04:51.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:50 smithi184 ceph-mon[22331]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:04:51.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.480+0000 7fc431200640 1 -- 172.21.15.16:0/906621164 --> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fc42c1134f0 con 0x7fc4100b4c10 2024-08-31T06:04:51.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.508+0000 7fc422a00640 1 -- 172.21.15.16:0/906621164 <== mgr.14220 v2:172.21.15.16:6800/4279751337 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc42c1134f0 con 0x7fc4100b4c10 2024-08-31T06:04:51.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.510+0000 7fc431200640 1 -- 172.21.15.16:0/906621164 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc4100b4c10 msgr2=0x7fc4100b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:51.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.510+0000 7fc431200640 1 --2- 172.21.15.16:0/906621164 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc4100b4c10 0x7fc4100b7120 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc42c115660 tx=0x7fc414005f90 comp rx=0 tx=0).stop 2024-08-31T06:04:51.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.510+0000 7fc431200640 1 -- 172.21.15.16:0/906621164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc42c075b70 msgr2=0x7fc42c1a0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:51.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.510+0000 7fc431200640 1 --2- 172.21.15.16:0/906621164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc42c075b70 0x7fc42c1a0830 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc41c00e9f0 tx=0x7fc41c00edf0 comp rx=0 tx=0).stop 2024-08-31T06:04:51.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.510+0000 7fc431200640 1 -- 172.21.15.16:0/906621164 shutdown_connections 2024-08-31T06:04:51.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.510+0000 7fc431200640 1 --2- 172.21.15.16:0/906621164 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc4100b4c10 0x7fc4100b7120 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:51.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.510+0000 7fc431200640 1 --2- 172.21.15.16:0/906621164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc42c1141d0 0x7fc42c1a6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:51.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.510+0000 7fc431200640 1 --2- 172.21.15.16:0/906621164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc42c077120 0x7fc42c1a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:51.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.510+0000 7fc431200640 1 --2- 172.21.15.16:0/906621164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc42c075b70 0x7fc42c1a0830 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:51.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.511+0000 7fc431200640 1 -- 172.21.15.16:0/906621164 >> 172.21.15.16:0/906621164 conn(0x7fc42c07b5f0 msgr2=0x7fc42c112de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:04:51.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.511+0000 7fc431200640 1 -- 172.21.15.16:0/906621164 shutdown_connections 2024-08-31T06:04:51.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:51.511+0000 7fc431200640 1 -- 172.21.15.16:0/906621164 wait complete. 2024-08-31T06:04:52.347 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-08-31T06:04:52.726 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:04:53.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:52 smithi016 ceph-mon[17194]: pgmap v290: 97 pgs: 97 active+clean; 607 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.4 KiB/s wr, 13 op/s 2024-08-31T06:04:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:52 smithi016 ceph-mon[17194]: from='client.24832 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:04:53.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:52 smithi155 ceph-mon[21742]: pgmap v290: 97 pgs: 97 active+clean; 607 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.4 KiB/s wr, 13 op/s 2024-08-31T06:04:53.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:52 smithi155 ceph-mon[21742]: from='client.24832 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:04:53.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.395+0000 7f7e8ba00640 1 -- 172.21.15.16:0/2826833382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e84109b90 msgr2=0x7f7e841103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:53.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.395+0000 7f7e8ba00640 1 --2- 172.21.15.16:0/2826833382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e84109b90 0x7f7e841103c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c009ca0 tx=0x7f7e7c02a8c0 comp rx=0 tx=0).stop 2024-08-31T06:04:53.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.395+0000 7f7e8ba00640 1 -- 172.21.15.16:0/2826833382 shutdown_connections 2024-08-31T06:04:53.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.395+0000 7f7e8ba00640 1 --2- 172.21.15.16:0/2826833382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e84109b90 0x7f7e841103c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:53.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.395+0000 7f7e8ba00640 1 --2- 172.21.15.16:0/2826833382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e84109240 0x7f7e84109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:53.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.395+0000 7f7e8ba00640 1 --2- 172.21.15.16:0/2826833382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e84103900 0x7f7e84103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:53.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.395+0000 7f7e8ba00640 1 -- 172.21.15.16:0/2826833382 >> 172.21.15.16:0/2826833382 conn(0x7f7e8407b5f0 msgr2=0x7f7e8407ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:04:53.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.395+0000 7f7e8ba00640 1 -- 172.21.15.16:0/2826833382 shutdown_connections 2024-08-31T06:04:53.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.395+0000 7f7e8ba00640 1 -- 172.21.15.16:0/2826833382 wait complete. 2024-08-31T06:04:53.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.396+0000 7f7e8ba00640 1 Processor -- start 2024-08-31T06:04:53.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.397+0000 7f7e8ba00640 1 -- start start 2024-08-31T06:04:53.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.397+0000 7f7e8ba00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e84103900 0x7f7e840754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:53.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.397+0000 7f7e8ba00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e84109240 0x7f7e840759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:53.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.397+0000 7f7e8ba00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e84109b90 0x7f7e84075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:53.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.397+0000 7f7e8ba00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7e84113850 con 0x7f7e84103900 2024-08-31T06:04:53.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.397+0000 7f7e8ba00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7e841136d0 con 0x7f7e84109240 2024-08-31T06:04:53.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.397+0000 7f7e8ba00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7e841139d0 con 0x7f7e84109b90 2024-08-31T06:04:53.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.397+0000 7f7e89000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e84103900 0x7f7e840754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:53.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.397+0000 7f7e89000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e84103900 0x7f7e840754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37498/0 (socket says 172.21.15.16:37498) 2024-08-31T06:04:53.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.397+0000 7f7e89000640 1 -- 172.21.15.16:0/661640331 learned_addr learned my addr 172.21.15.16:0/661640331 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:04:53.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.397+0000 7f7e89a00640 1 --2- 172.21.15.16:0/661640331 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e84109b90 0x7f7e84075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:53.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.398+0000 7f7e7be00640 1 --2- 172.21.15.16:0/661640331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e84109240 0x7f7e840759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:53.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.398+0000 7f7e89000640 1 -- 172.21.15.16:0/661640331 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e84109b90 msgr2=0x7f7e84075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:53.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.398+0000 7f7e89000640 1 --2- 172.21.15.16:0/661640331 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e84109b90 0x7f7e84075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:53.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.398+0000 7f7e89000640 1 -- 172.21.15.16:0/661640331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e84109240 msgr2=0x7f7e840759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:53.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.398+0000 7f7e89000640 1 --2- 172.21.15.16:0/661640331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e84109240 0x7f7e840759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:53.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.398+0000 7f7e89000640 1 -- 172.21.15.16:0/661640331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e841a8480 con 0x7f7e84103900 2024-08-31T06:04:53.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.398+0000 7f7e89000640 1 --2- 172.21.15.16:0/661640331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e84103900 0x7f7e840754b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7e7400e9f0 tx=0x7f7e7400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:04:53.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.398+0000 7f7e82000640 1 -- 172.21.15.16:0/661640331 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7400cd80 con 0x7f7e84103900 2024-08-31T06:04:53.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.399+0000 7f7e82000640 1 -- 172.21.15.16:0/661640331 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e740054d0 con 0x7f7e84103900 2024-08-31T06:04:53.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.399+0000 7f7e8ba00640 1 -- 172.21.15.16:0/661640331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e841a8770 con 0x7f7e84103900 2024-08-31T06:04:53.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.399+0000 7f7e82000640 1 -- 172.21.15.16:0/661640331 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e74010690 con 0x7f7e84103900 2024-08-31T06:04:53.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.399+0000 7f7e8ba00640 1 -- 172.21.15.16:0/661640331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e841a8c60 con 0x7f7e84103900 2024-08-31T06:04:53.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.400+0000 7f7e8ba00640 1 -- 172.21.15.16:0/661640331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e84103d10 con 0x7f7e84103900 2024-08-31T06:04:53.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.403+0000 7f7e82000640 1 -- 172.21.15.16:0/661640331 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f7e74004e90 con 0x7f7e84103900 2024-08-31T06:04:53.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.405+0000 7f7e82000640 1 --2- 172.21.15.16:0/661640331 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7e680b4c10 0x7f7e680b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:53.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.405+0000 7f7e82000640 1 -- 172.21.15.16:0/661640331 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7f7e740e57b0 con 0x7f7e84103900 2024-08-31T06:04:53.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.405+0000 7f7e7be00640 1 --2- 172.21.15.16:0/661640331 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7e680b4c10 0x7f7e680b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:53.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.405+0000 7f7e7be00640 1 --2- 172.21.15.16:0/661640331 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7e680b4c10 0x7f7e680b7120 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7e84108f30 tx=0x7f7e6c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:04:53.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.410+0000 7f7e82000640 1 -- 172.21.15.16:0/661640331 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f7e740ac740 con 0x7f7e84103900 2024-08-31T06:04:53.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:52 smithi184 ceph-mon[22331]: pgmap v290: 97 pgs: 97 active+clean; 607 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.4 KiB/s wr, 13 op/s 2024-08-31T06:04:53.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:52 smithi184 ceph-mon[22331]: from='client.24832 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:04:53.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.635+0000 7f7e8ba00640 1 -- 172.21.15.16:0/661640331 --> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f7e840635a0 con 0x7f7e680b4c10 2024-08-31T06:04:53.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.643+0000 7f7e82000640 1 -- 172.21.15.16:0/661640331 <== mgr.14220 v2:172.21.15.16:6800/4279751337 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f7e840635a0 con 0x7f7e680b4c10 2024-08-31T06:04:53.645 INFO:teuthology.orchestra.run.smithi016.stdout:/volumes/qa/sv_1/ba5423bc-5c83-4d64-ae1c-ad8850110ce4 2024-08-31T06:04:53.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.645+0000 7f7e8ba00640 1 -- 172.21.15.16:0/661640331 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7e680b4c10 msgr2=0x7f7e680b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:53.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.645+0000 7f7e8ba00640 1 --2- 172.21.15.16:0/661640331 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7e680b4c10 0x7f7e680b7120 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7e84108f30 tx=0x7f7e6c005f90 comp rx=0 tx=0).stop 2024-08-31T06:04:53.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.645+0000 7f7e8ba00640 1 -- 172.21.15.16:0/661640331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e84103900 msgr2=0x7f7e840754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:53.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.645+0000 7f7e8ba00640 1 --2- 172.21.15.16:0/661640331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e84103900 0x7f7e840754b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7e7400e9f0 tx=0x7f7e7400edf0 comp rx=0 tx=0).stop 2024-08-31T06:04:53.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.645+0000 7f7e8ba00640 1 -- 172.21.15.16:0/661640331 shutdown_connections 2024-08-31T06:04:53.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.645+0000 7f7e8ba00640 1 --2- 172.21.15.16:0/661640331 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7e680b4c10 0x7f7e680b7120 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:53.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.645+0000 7f7e8ba00640 1 --2- 172.21.15.16:0/661640331 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e84109b90 0x7f7e84075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:53.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.645+0000 7f7e8ba00640 1 --2- 172.21.15.16:0/661640331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e84109240 0x7f7e840759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:53.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.645+0000 7f7e8ba00640 1 --2- 172.21.15.16:0/661640331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e84103900 0x7f7e840754b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:53.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.645+0000 7f7e8ba00640 1 -- 172.21.15.16:0/661640331 >> 172.21.15.16:0/661640331 conn(0x7f7e8407b5f0 msgr2=0x7f7e84107290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:04:53.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.645+0000 7f7e8ba00640 1 -- 172.21.15.16:0/661640331 shutdown_connections 2024-08-31T06:04:53.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:53.645+0000 7f7e8ba00640 1 -- 172.21.15.16:0/661640331 wait complete. 2024-08-31T06:04:54.517 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:04:54.898 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:04:55.242 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:54 smithi016 ceph-mon[17194]: pgmap v291: 97 pgs: 97 active+clean; 607 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.8 KiB/s wr, 11 op/s 2024-08-31T06:04:55.243 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:54 smithi016 ceph-mon[17194]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:04:55.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:54 smithi155 ceph-mon[21742]: pgmap v291: 97 pgs: 97 active+clean; 607 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.8 KiB/s wr, 11 op/s 2024-08-31T06:04:55.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:54 smithi155 ceph-mon[21742]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:04:55.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:54 smithi184 ceph-mon[22331]: pgmap v291: 97 pgs: 97 active+clean; 607 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.8 KiB/s wr, 11 op/s 2024-08-31T06:04:55.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:54 smithi184 ceph-mon[22331]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:04:55.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.544+0000 7faffde00640 1 -- 172.21.15.16:0/907985798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faff810b290 msgr2=0x7faff810b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:55.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.544+0000 7faffde00640 1 --2- 172.21.15.16:0/907985798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faff810b290 0x7faff810b6a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fafe4009900 tx=0x7fafe402a410 comp rx=0 tx=0).stop 2024-08-31T06:04:55.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.544+0000 7faffde00640 1 -- 172.21.15.16:0/907985798 shutdown_connections 2024-08-31T06:04:55.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.544+0000 7faffde00640 1 --2- 172.21.15.16:0/907985798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faff810bbe0 0x7faff8112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:55.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.544+0000 7faffde00640 1 --2- 172.21.15.16:0/907985798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faff810b290 0x7faff810b6a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:55.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.544+0000 7faffde00640 1 --2- 172.21.15.16:0/907985798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faff8105950 0x7faff8105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:55.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.544+0000 7faffde00640 1 -- 172.21.15.16:0/907985798 >> 172.21.15.16:0/907985798 conn(0x7faff8101420 msgr2=0x7faff8103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:04:55.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.544+0000 7faffde00640 1 -- 172.21.15.16:0/907985798 shutdown_connections 2024-08-31T06:04:55.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.544+0000 7faffde00640 1 -- 172.21.15.16:0/907985798 wait complete. 2024-08-31T06:04:55.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.546+0000 7faffde00640 1 Processor -- start 2024-08-31T06:04:55.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.546+0000 7faffde00640 1 -- start start 2024-08-31T06:04:55.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.546+0000 7faffde00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faff8105950 0x7faff819c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:55.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.546+0000 7faffde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faff810b290 0x7faff819c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:55.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.546+0000 7faffde00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faff810bbe0 0x7faff81a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:55.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.546+0000 7faffde00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faff81158f0 con 0x7faff810b290 2024-08-31T06:04:55.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.546+0000 7faffde00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faff8115770 con 0x7faff8105950 2024-08-31T06:04:55.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.546+0000 7faffde00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faff8115a70 con 0x7faff810bbe0 2024-08-31T06:04:55.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.546+0000 7faff7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faff810b290 0x7faff819c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:55.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.546+0000 7faff7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faff810b290 0x7faff819c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37518/0 (socket says 172.21.15.16:37518) 2024-08-31T06:04:55.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.546+0000 7faff7e00640 1 -- 172.21.15.16:0/1911799755 learned_addr learned my addr 172.21.15.16:0/1911799755 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:04:55.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.547+0000 7faffca00640 1 --2- 172.21.15.16:0/1911799755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faff8105950 0x7faff819c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:55.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.547+0000 7faffd400640 1 --2- 172.21.15.16:0/1911799755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faff810bbe0 0x7faff81a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:55.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.547+0000 7faffca00640 1 -- 172.21.15.16:0/1911799755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faff810bbe0 msgr2=0x7faff81a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.547+0000 7faffca00640 1 --2- 172.21.15.16:0/1911799755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faff810bbe0 0x7faff81a1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.547+0000 7faffca00640 1 -- 172.21.15.16:0/1911799755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faff810b290 msgr2=0x7faff819c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.547+0000 7faffca00640 1 --2- 172.21.15.16:0/1911799755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faff810b290 0x7faff819c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.547+0000 7faffca00640 1 -- 172.21.15.16:0/1911799755 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff81a2250 con 0x7faff8105950 2024-08-31T06:04:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.547+0000 7faffd400640 1 --2- 172.21.15.16:0/1911799755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faff810bbe0 0x7faff81a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:04:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.547+0000 7faffca00640 1 --2- 172.21.15.16:0/1911799755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faff8105950 0x7faff819c2e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fafec00e9f0 tx=0x7fafec00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:04:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.548+0000 7faff5600640 1 -- 172.21.15.16:0/1911799755 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec00ccc0 con 0x7faff8105950 2024-08-31T06:04:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.548+0000 7faffde00640 1 -- 172.21.15.16:0/1911799755 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faff81a2540 con 0x7faff8105950 2024-08-31T06:04:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.548+0000 7faffde00640 1 -- 172.21.15.16:0/1911799755 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faff81a8ca0 con 0x7faff8105950 2024-08-31T06:04:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.548+0000 7faff5600640 1 -- 172.21.15.16:0/1911799755 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafec0108e0 con 0x7faff8105950 2024-08-31T06:04:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.548+0000 7faff5600640 1 -- 172.21.15.16:0/1911799755 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec018750 con 0x7faff8105950 2024-08-31T06:04:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.549+0000 7faffde00640 1 -- 172.21.15.16:0/1911799755 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafd80052c0 con 0x7faff8105950 2024-08-31T06:04:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.551+0000 7faff5600640 1 -- 172.21.15.16:0/1911799755 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7fafec010430 con 0x7faff8105950 2024-08-31T06:04:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.553+0000 7faff5600640 1 --2- 172.21.15.16:0/1911799755 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fafe00b4c90 0x7fafe00b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.553+0000 7faff5600640 1 -- 172.21.15.16:0/1911799755 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7fafec0e5510 con 0x7faff8105950 2024-08-31T06:04:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.553+0000 7faff7e00640 1 --2- 172.21.15.16:0/1911799755 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fafe00b4c90 0x7fafe00b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.554+0000 7faff7e00640 1 --2- 172.21.15.16:0/1911799755 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fafe00b4c90 0x7fafe00b71a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fafe4006010 tx=0x7fafe4005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:04:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.555+0000 7faff5600640 1 -- 172.21.15.16:0/1911799755 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fafec0ac4a0 con 0x7faff8105950 2024-08-31T06:04:55.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.789+0000 7faffde00640 1 -- 172.21.15.16:0/1911799755 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fafd80055b0 con 0x7faff8105950 2024-08-31T06:04:55.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.790+0000 7faff5600640 1 -- 172.21.15.16:0/1911799755 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11917 (secure 0 0 0) 0x7fafec0abda0 con 0x7faff8105950 2024-08-31T06:04:55.792 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:04:55.792 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"btime":"2024-08-31T06:04:42:937777+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:04:42.745448+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24698":{"gid":24698,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1089710405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1089710405},{"type":"v1","addr":"172.21.15.155:6837","nonce":1089710405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24712,24724,24737]},"id":1}]} 2024-08-31T06:04:55.793 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 24 2024-08-31T06:04:55.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.793+0000 7faffde00640 1 -- 172.21.15.16:0/1911799755 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fafe00b4c90 msgr2=0x7fafe00b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:55.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.793+0000 7faffde00640 1 --2- 172.21.15.16:0/1911799755 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fafe00b4c90 0x7fafe00b71a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fafe4006010 tx=0x7fafe4005a20 comp rx=0 tx=0).stop 2024-08-31T06:04:55.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.793+0000 7faffde00640 1 -- 172.21.15.16:0/1911799755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faff8105950 msgr2=0x7faff819c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:55.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.793+0000 7faffde00640 1 --2- 172.21.15.16:0/1911799755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faff8105950 0x7faff819c2e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fafec00e9f0 tx=0x7fafec00eeb0 comp rx=0 tx=0).stop 2024-08-31T06:04:55.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.793+0000 7faffde00640 1 -- 172.21.15.16:0/1911799755 shutdown_connections 2024-08-31T06:04:55.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.793+0000 7faffde00640 1 --2- 172.21.15.16:0/1911799755 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fafe00b4c90 0x7fafe00b71a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:55.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.793+0000 7faffde00640 1 --2- 172.21.15.16:0/1911799755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faff810bbe0 0x7faff81a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:55.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.793+0000 7faffde00640 1 --2- 172.21.15.16:0/1911799755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faff810b290 0x7faff819c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:55.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.793+0000 7faffde00640 1 --2- 172.21.15.16:0/1911799755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faff8105950 0x7faff819c2e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:55.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.793+0000 7faffde00640 1 -- 172.21.15.16:0/1911799755 >> 172.21.15.16:0/1911799755 conn(0x7faff8101420 msgr2=0x7faff8103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:04:55.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.793+0000 7faffde00640 1 -- 172.21.15.16:0/1911799755 shutdown_connections 2024-08-31T06:04:55.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:55.794+0000 7faffde00640 1 -- 172.21.15.16:0/1911799755 wait complete. 2024-08-31T06:04:56.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:55 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/1911799755' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:04:56.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:55 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/1911799755' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:04:56.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:55 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/1911799755' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:04:56.695 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph osd dump --format=json 2024-08-31T06:04:57.073 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:04:57.370 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:56 smithi016 ceph-mon[17194]: pgmap v292: 97 pgs: 97 active+clean; 630 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.1 KiB/s wr, 11 op/s 2024-08-31T06:04:57.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:56 smithi155 ceph-mon[21742]: pgmap v292: 97 pgs: 97 active+clean; 630 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.1 KiB/s wr, 11 op/s 2024-08-31T06:04:57.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:56 smithi184 ceph-mon[22331]: pgmap v292: 97 pgs: 97 active+clean; 630 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.1 KiB/s wr, 11 op/s 2024-08-31T06:04:57.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.720+0000 7f1747e00640 1 -- 172.21.15.16:0/2893509318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17481099e0 msgr2=0x7f1748110210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:57.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.720+0000 7f1747e00640 1 --2- 172.21.15.16:0/2893509318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17481099e0 0x7f1748110210 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1740009ca0 tx=0x7f174002a8c0 comp rx=0 tx=0).stop 2024-08-31T06:04:57.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.720+0000 7f1747e00640 1 -- 172.21.15.16:0/2893509318 shutdown_connections 2024-08-31T06:04:57.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.720+0000 7f1747e00640 1 --2- 172.21.15.16:0/2893509318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1748109090 0x7f17481094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:57.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.720+0000 7f1747e00640 1 --2- 172.21.15.16:0/2893509318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17481099e0 0x7f1748110210 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:57.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.720+0000 7f1747e00640 1 --2- 172.21.15.16:0/2893509318 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1748103750 0x7f1748103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:57.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.720+0000 7f1747e00640 1 -- 172.21.15.16:0/2893509318 >> 172.21.15.16:0/2893509318 conn(0x7f17480ff1e0 msgr2=0x7f1748101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:04:57.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.720+0000 7f1747e00640 1 -- 172.21.15.16:0/2893509318 shutdown_connections 2024-08-31T06:04:57.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.720+0000 7f1747e00640 1 -- 172.21.15.16:0/2893509318 wait complete. 2024-08-31T06:04:57.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.721+0000 7f1747e00640 1 Processor -- start 2024-08-31T06:04:57.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.721+0000 7f1747e00640 1 -- start start 2024-08-31T06:04:57.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.722+0000 7f1747e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1748103750 0x7f174819c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:57.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.722+0000 7f1747e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1748109090 0x7f174819c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:57.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.722+0000 7f1747e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17481099e0 0x7f17481a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:57.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.722+0000 7f1747e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f17481136e0 con 0x7f1748109090 2024-08-31T06:04:57.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.722+0000 7f1747e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1748113560 con 0x7f1748103750 2024-08-31T06:04:57.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.722+0000 7f1747e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1748113860 con 0x7f17481099e0 2024-08-31T06:04:57.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.722+0000 7f1746000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1748109090 0x7f174819c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:57.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.722+0000 7f1746000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1748109090 0x7f174819c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43304/0 (socket says 172.21.15.16:43304) 2024-08-31T06:04:57.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.722+0000 7f1746000640 1 -- 172.21.15.16:0/2603788707 learned_addr learned my addr 172.21.15.16:0/2603788707 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:04:57.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.722+0000 7f1746a00640 1 --2- 172.21.15.16:0/2603788707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1748103750 0x7f174819c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:57.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.722+0000 7f1747400640 1 --2- 172.21.15.16:0/2603788707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17481099e0 0x7f17481a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:57.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.723+0000 7f1746000640 1 -- 172.21.15.16:0/2603788707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17481099e0 msgr2=0x7f17481a1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:57.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.723+0000 7f1746000640 1 --2- 172.21.15.16:0/2603788707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17481099e0 0x7f17481a1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:57.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.723+0000 7f1746000640 1 -- 172.21.15.16:0/2603788707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1748103750 msgr2=0x7f174819c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.723+0000 7f1746000640 1 --2- 172.21.15.16:0/2603788707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1748103750 0x7f174819c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.723+0000 7f1746000640 1 -- 172.21.15.16:0/2603788707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17481a2200 con 0x7f1748109090 2024-08-31T06:04:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.723+0000 7f1746000640 1 --2- 172.21.15.16:0/2603788707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1748109090 0x7f174819c830 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f173400d740 tx=0x7f173400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:04:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.723+0000 7f174ca00640 1 -- 172.21.15.16:0/2603788707 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1734017070 con 0x7f1748109090 2024-08-31T06:04:57.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.723+0000 7f1747e00640 1 -- 172.21.15.16:0/2603788707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17481a24f0 con 0x7f1748109090 2024-08-31T06:04:57.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.723+0000 7f1747e00640 1 -- 172.21.15.16:0/2603788707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17481a8c50 con 0x7f1748109090 2024-08-31T06:04:57.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.723+0000 7f174ca00640 1 -- 172.21.15.16:0/2603788707 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f173400de50 con 0x7f1748109090 2024-08-31T06:04:57.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.724+0000 7f174ca00640 1 -- 172.21.15.16:0/2603788707 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173400f620 con 0x7f1748109090 2024-08-31T06:04:57.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.725+0000 7f1747e00640 1 -- 172.21.15.16:0/2603788707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1748103b60 con 0x7f1748109090 2024-08-31T06:04:57.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.727+0000 7f174ca00640 1 -- 172.21.15.16:0/2603788707 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f173401a070 con 0x7f1748109090 2024-08-31T06:04:57.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.729+0000 7f174ca00640 1 --2- 172.21.15.16:0/2603788707 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f172c0b4c90 0x7f172c0b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:57.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.729+0000 7f174ca00640 1 -- 172.21.15.16:0/2603788707 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7f17340e4bf0 con 0x7f1748109090 2024-08-31T06:04:57.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.729+0000 7f1746a00640 1 --2- 172.21.15.16:0/2603788707 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f172c0b4c90 0x7f172c0b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:57.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.730+0000 7f1746a00640 1 --2- 172.21.15.16:0/2603788707 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f172c0b4c90 0x7f172c0b71a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f173800bbe0 tx=0x7f173800b390 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:04:57.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.731+0000 7f174ca00640 1 -- 172.21.15.16:0/2603788707 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f17340ab5b0 con 0x7f1748109090 2024-08-31T06:04:57.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.931+0000 7f1747e00640 1 -- 172.21.15.16:0/2603788707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f174819d3c0 con 0x7f1748109090 2024-08-31T06:04:57.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.932+0000 7f174ca00640 1 -- 172.21.15.16:0/2603788707 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22608 (secure 0 0 0) 0x7f17340b4140 con 0x7f1748109090 2024-08-31T06:04:57.934 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:04:57.935 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":78,"fsid":"673e4aa0-675d-11ef-bcd4-c7b262605968","created":"2024-08-31T05:55:09.632536+0000","modified":"2024-08-31T06:04:27.098694+0000","last_up_change":"2024-08-31T06:03:27.893697+0000","last_in_change":"2024-08-31T06:03:05.664945+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-31T05:59:23.642270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-31T06:04:20.871835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-31T06:04:24.119499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e349dae-6aa5-4b6d-9107-a945655e11bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6803","nonce":1725037400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6805","nonce":1725037400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6809","nonce":1725037400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6807","nonce":1725037400}]},"public_addr":"172.21.15.16:6803/1725037400","cluster_addr":"172.21.15.16:6805/1725037400","heartbeat_back_addr":"172.21.15.16:6809/1725037400","heartbeat_front_addr":"172.21.15.16:6807/1725037400","state":["exists","up"]},{"osd":1,"uuid":"bd645dc3-93b5-40af-89d8-08ccec22e50d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6801","nonce":4241618440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6803","nonce":4241618440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6807","nonce":4241618440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6805","nonce":4241618440}]},"public_addr":"172.21.15.155:6801/4241618440","cluster_addr":"172.21.15.155:6803/4241618440","heartbeat_back_addr":"172.21.15.155:6807/4241618440","heartbeat_front_addr":"172.21.15.155:6805/4241618440","state":["exists","up"]},{"osd":2,"uuid":"c69c8c19-32f8-400a-86d4-0f4ce240a7de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6801","nonce":2193296422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6803","nonce":2193296422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6807","nonce":2193296422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6805","nonce":2193296422}]},"public_addr":"172.21.15.184:6801/2193296422","cluster_addr":"172.21.15.184:6803/2193296422","heartbeat_back_addr":"172.21.15.184:6807/2193296422","heartbeat_front_addr":"172.21.15.184:6805/2193296422","state":["exists","up"]},{"osd":3,"uuid":"8892e35f-6880-434c-a805-ab8ccbed2bf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6811","nonce":81681815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6813","nonce":81681815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6817","nonce":81681815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6815","nonce":81681815}]},"public_addr":"172.21.15.16:6811/81681815","cluster_addr":"172.21.15.16:6813/81681815","heartbeat_back_addr":"172.21.15.16:6817/81681815","heartbeat_front_addr":"172.21.15.16:6815/81681815","state":["exists","up"]},{"osd":4,"uuid":"d2515e73-88f7-4654-9762-adb8802777ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6809","nonce":3592422019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6811","nonce":3592422019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6815","nonce":3592422019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6813","nonce":3592422019}]},"public_addr":"172.21.15.155:6809/3592422019","cluster_addr":"172.21.15.155:6811/3592422019","heartbeat_back_addr":"172.21.15.155:6815/3592422019","heartbeat_front_addr":"172.21.15.155:6813/3592422019","state":["exists","up"]},{"osd":5,"uuid":"5ce6e2fa-c17c-49b3-9402-f19bfc21076d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6809","nonce":2054894065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6811","nonce":2054894065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6815","nonce":2054894065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6813","nonce":2054894065}]},"public_addr":"172.21.15.184:6809/2054894065","cluster_addr":"172.21.15.184:6811/2054894065","heartbeat_back_addr":"172.21.15.184:6815/2054894065","heartbeat_front_addr":"172.21.15.184:6813/2054894065","state":["exists","up"]},{"osd":6,"uuid":"f257d8d1-ad59-4bd8-821d-bf29e50b547e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6819","nonce":1697421092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6821","nonce":1697421092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6825","nonce":1697421092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6823","nonce":1697421092}]},"public_addr":"172.21.15.16:6819/1697421092","cluster_addr":"172.21.15.16:6821/1697421092","heartbeat_back_addr":"172.21.15.16:6825/1697421092","heartbeat_front_addr":"172.21.15.16:6823/1697421092","state":["exists","up"]},{"osd":7,"uuid":"8e76c20e-8d25-423b-b7e8-de642d9c9a6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6817","nonce":2671827901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6819","nonce":2671827901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6823","nonce":2671827901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6821","nonce":2671827901}]},"public_addr":"172.21.15.155:6817/2671827901","cluster_addr":"172.21.15.155:6819/2671827901","heartbeat_back_addr":"172.21.15.155:6823/2671827901","heartbeat_front_addr":"172.21.15.155:6821/2671827901","state":["exists","up"]},{"osd":8,"uuid":"0f0bb23f-4bc3-4198-8381-5b5cd16ae394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6817","nonce":2479117823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6819","nonce":2479117823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6823","nonce":2479117823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6821","nonce":2479117823}]},"public_addr":"172.21.15.184:6817/2479117823","cluster_addr":"172.21.15.184:6819/2479117823","heartbeat_back_addr":"172.21.15.184:6823/2479117823","heartbeat_front_addr":"172.21.15.184:6821/2479117823","state":["exists","up"]},{"osd":9,"uuid":"bc27afcb-7871-4f72-a043-044fd5fe7d86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6827","nonce":3015700546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6829","nonce":3015700546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6833","nonce":3015700546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6831","nonce":3015700546}]},"public_addr":"172.21.15.16:6827/3015700546","cluster_addr":"172.21.15.16:6829/3015700546","heartbeat_back_addr":"172.21.15.16:6833/3015700546","heartbeat_front_addr":"172.21.15.16:6831/3015700546","state":["exists","up"]},{"osd":10,"uuid":"f2548997-72ee-400e-ab8d-5966d85b2cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6825","nonce":2427122475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6827","nonce":2427122475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6831","nonce":2427122475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6829","nonce":2427122475}]},"public_addr":"172.21.15.155:6825/2427122475","cluster_addr":"172.21.15.155:6827/2427122475","heartbeat_back_addr":"172.21.15.155:6831/2427122475","heartbeat_front_addr":"172.21.15.155:6829/2427122475","state":["exists","up"]},{"osd":11,"uuid":"8297da1d-d3e2-4a4d-9396-426dfafa3e95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6825","nonce":399265701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6827","nonce":399265701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6831","nonce":399265701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6829","nonce":399265701}]},"public_addr":"172.21.15.184:6825/399265701","cluster_addr":"172.21.15.184:6827/399265701","heartbeat_back_addr":"172.21.15.184:6831/399265701","heartbeat_front_addr":"172.21.15.184:6829/399265701","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:58:30.842360+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:58:57.164488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:59:20.778103+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:59:50.123991+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:00:16.789933+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:00:41.788905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:01:12.281376+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:01:39.211858+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:02:04.958771+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:02:35.645947+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:03:01.940433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:03:26.728673+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3923220960":"2024-09-01T05:57:23.153214+0000","172.21.15.16:0/3013533836":"2024-09-01T05:57:23.153214+0000","172.21.15.16:6800/3755683228":"2024-09-01T05:55:45.075902+0000","172.21.15.16:0/3064435470":"2024-09-01T05:55:45.075902+0000","172.21.15.16:0/911709962":"2024-09-01T05:55:45.075902+0000","172.21.15.16:0/2937368967":"2024-09-01T05:55:45.075902+0000","172.21.15.16:6800/1378972230":"2024-09-01T05:56:14.195704+0000","172.21.15.16:0/3047316995":"2024-09-01T05:56:14.195704+0000","172.21.15.16:6801/3755683228":"2024-09-01T05:55:45.075902+0000","172.21.15.16:6801/2571082634":"2024-09-01T05:57:23.153214+0000","172.21.15.16:0/2136704295":"2024-09-01T05:56:14.195704+0000","172.21.15.16:0/2259332778":"2024-09-01T05:56:14.195704+0000","172.21.15.16:6800/2571082634":"2024-09-01T05:57:23.153214+0000","172.21.15.16:6801/1378972230":"2024-09-01T05:56:14.195704+0000","172.21.15.16:0/2126457673":"2024-09-01T05:57:23.153214+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-31T06:04:57.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.935+0000 7f1747e00640 1 -- 172.21.15.16:0/2603788707 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f172c0b4c90 msgr2=0x7f172c0b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:57.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.935+0000 7f1747e00640 1 --2- 172.21.15.16:0/2603788707 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f172c0b4c90 0x7f172c0b71a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f173800bbe0 tx=0x7f173800b390 comp rx=0 tx=0).stop 2024-08-31T06:04:57.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.935+0000 7f1747e00640 1 -- 172.21.15.16:0/2603788707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1748109090 msgr2=0x7f174819c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:57.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.935+0000 7f1747e00640 1 --2- 172.21.15.16:0/2603788707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1748109090 0x7f174819c830 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f173400d740 tx=0x7f173400dc00 comp rx=0 tx=0).stop 2024-08-31T06:04:57.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.935+0000 7f1747e00640 1 -- 172.21.15.16:0/2603788707 shutdown_connections 2024-08-31T06:04:57.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.935+0000 7f1747e00640 1 --2- 172.21.15.16:0/2603788707 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f172c0b4c90 0x7f172c0b71a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:57.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.935+0000 7f1747e00640 1 --2- 172.21.15.16:0/2603788707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17481099e0 0x7f17481a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:57.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.935+0000 7f1747e00640 1 --2- 172.21.15.16:0/2603788707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1748109090 0x7f174819c830 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:57.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.935+0000 7f1747e00640 1 --2- 172.21.15.16:0/2603788707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1748103750 0x7f174819c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:57.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.935+0000 7f1747e00640 1 -- 172.21.15.16:0/2603788707 >> 172.21.15.16:0/2603788707 conn(0x7f17480ff1e0 msgr2=0x7f1748108670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:04:57.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.935+0000 7f1747e00640 1 -- 172.21.15.16:0/2603788707 shutdown_connections 2024-08-31T06:04:57.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:57.935+0000 7f1747e00640 1 -- 172.21.15.16:0/2603788707 wait complete. 2024-08-31T06:04:58.289 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:57 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/2603788707' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T06:04:58.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:57 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/2603788707' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T06:04:58.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:57 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/2603788707' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T06:04:58.703 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-08-31T06:04:58.704 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph pg dump --format=json 2024-08-31T06:04:59.083 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:04:59.249 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:04:58 smithi016 ceph-mon[17194]: pgmap v293: 97 pgs: 97 active+clean; 630 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.1 KiB/s wr, 7 op/s 2024-08-31T06:04:59.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:04:58 smithi155 ceph-mon[21742]: pgmap v293: 97 pgs: 97 active+clean; 630 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.1 KiB/s wr, 7 op/s 2024-08-31T06:04:59.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:04:58 smithi184 ceph-mon[22331]: pgmap v293: 97 pgs: 97 active+clean; 630 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.1 KiB/s wr, 7 op/s 2024-08-31T06:04:59.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:59.789+0000 7f8d0aa00640 1 -- 172.21.15.16:0/82892774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d04077120 msgr2=0x7f8d04075550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:59.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:59.789+0000 7f8d0aa00640 1 --2- 172.21.15.16:0/82892774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d04077120 0x7f8d04075550 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8009900 tx=0x7f8cf802a3f0 comp rx=0 tx=0).stop 2024-08-31T06:04:59.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:59.789+0000 7f8d0aa00640 1 -- 172.21.15.16:0/82892774 shutdown_connections 2024-08-31T06:04:59.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:59.789+0000 7f8d0aa00640 1 --2- 172.21.15.16:0/82892774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d04075b70 0x7f8d04075f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:59.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:59.789+0000 7f8d0aa00640 1 --2- 172.21.15.16:0/82892774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d04114020 0x7f8d041164f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:59.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:59.789+0000 7f8d0aa00640 1 --2- 172.21.15.16:0/82892774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d04077120 0x7f8d04075550 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:59.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:59.789+0000 7f8d0aa00640 1 -- 172.21.15.16:0/82892774 >> 172.21.15.16:0/82892774 conn(0x7f8d040ff1e0 msgr2=0x7f8d04101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:04:59.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:59.789+0000 7f8d0aa00640 1 -- 172.21.15.16:0/82892774 shutdown_connections 2024-08-31T06:04:59.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:59.789+0000 7f8d0aa00640 1 -- 172.21.15.16:0/82892774 wait complete. 2024-08-31T06:04:59.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:59.791+0000 7f8d0aa00640 1 Processor -- start 2024-08-31T06:04:59.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:59.791+0000 7f8d0aa00640 1 -- start start 2024-08-31T06:04:59.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:59.791+0000 7f8d0aa00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d04075b70 0x7f8d041a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:59.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:59.791+0000 7f8d0aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d04077120 0x7f8d041a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:59.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:59.791+0000 7f8d0aa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d04114020 0x7f8d041a5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:59.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:59.791+0000 7f8d0aa00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8d04119c60 con 0x7f8d04077120 2024-08-31T06:04:59.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:59.791+0000 7f8d0aa00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8d04119ae0 con 0x7f8d04114020 2024-08-31T06:04:59.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:59.791+0000 7f8d0aa00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8d04119de0 con 0x7f8d04075b70 2024-08-31T06:04:59.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:59.791+0000 7f8d08c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d04077120 0x7f8d041a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:59.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:59.792+0000 7f8d08c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d04077120 0x7f8d041a0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43322/0 (socket says 172.21.15.16:43322) 2024-08-31T06:04:59.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:59.792+0000 7f8d08c00640 1 -- 172.21.15.16:0/1491855669 learned_addr learned my addr 172.21.15.16:0/1491855669 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:04:59.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:59.792+0000 7f8d09600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d04075b70 0x7f8d041a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:59.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:59.792+0000 7f8d0a000640 1 --2- 172.21.15.16:0/1491855669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d04114020 0x7f8d041a5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:59.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:59.792+0000 7f8d08c00640 1 -- 172.21.15.16:0/1491855669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d04075b70 msgr2=0x7f8d041a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:59.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:59.792+0000 7f8d08c00640 1 --2- 172.21.15.16:0/1491855669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d04075b70 0x7f8d041a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:59.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:59.792+0000 7f8d08c00640 1 -- 172.21.15.16:0/1491855669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d04114020 msgr2=0x7f8d041a5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:04:59.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:59.792+0000 7f8d08c00640 1 --2- 172.21.15.16:0/1491855669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d04114020 0x7f8d041a5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:04:59.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:59.792+0000 7f8d08c00640 1 -- 172.21.15.16:0/1491855669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d041a65e0 con 0x7f8d04077120 2024-08-31T06:04:59.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:59.792+0000 7f8d0a000640 1 --2- 172.21.15.16:0/1491855669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d04114020 0x7f8d041a5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:04:59.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:59.792+0000 7f8d08c00640 1 --2- 172.21.15.16:0/1491855669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d04077120 0x7f8d041a0c10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8cf000d770 tx=0x7f8cf000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:04:59.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:59.793+0000 7f8d02000640 1 -- 172.21.15.16:0/1491855669 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf0017070 con 0x7f8d04077120 2024-08-31T06:04:59.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:59.793+0000 7f8d02000640 1 -- 172.21.15.16:0/1491855669 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cf0009c30 con 0x7f8d04077120 2024-08-31T06:04:59.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:59.793+0000 7f8d02000640 1 -- 172.21.15.16:0/1491855669 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf000f620 con 0x7f8d04077120 2024-08-31T06:04:59.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:59.793+0000 7f8d0aa00640 1 -- 172.21.15.16:0/1491855669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d041a68d0 con 0x7f8d04077120 2024-08-31T06:04:59.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:59.793+0000 7f8d0aa00640 1 -- 172.21.15.16:0/1491855669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d041ad030 con 0x7f8d04077120 2024-08-31T06:04:59.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:59.794+0000 7f8d0aa00640 1 -- 172.21.15.16:0/1491855669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d04075f80 con 0x7f8d04077120 2024-08-31T06:04:59.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:59.796+0000 7f8d02000640 1 -- 172.21.15.16:0/1491855669 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f8cf001a070 con 0x7f8d04077120 2024-08-31T06:04:59.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:59.798+0000 7f8d02000640 1 --2- 172.21.15.16:0/1491855669 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f8cec0b4c10 0x7f8cec0b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:04:59.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:59.798+0000 7f8d02000640 1 -- 172.21.15.16:0/1491855669 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7f8cf00e4e50 con 0x7f8d04077120 2024-08-31T06:04:59.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:59.798+0000 7f8d09600640 1 --2- 172.21.15.16:0/1491855669 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f8cec0b4c10 0x7f8cec0b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:04:59.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:59.799+0000 7f8d09600640 1 --2- 172.21.15.16:0/1491855669 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f8cec0b4c10 0x7f8cec0b7120 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8031440 tx=0x7f8cf8009560 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:04:59.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:59.800+0000 7f8d02000640 1 -- 172.21.15.16:0/1491855669 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f8cf0009de0 con 0x7f8d04077120 2024-08-31T06:04:59.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:04:59.997+0000 7f8d0aa00640 1 -- 172.21.15.16:0/1491855669 --> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8d040635a0 con 0x7f8cec0b4c10 2024-08-31T06:05:00.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:00.003+0000 7f8d02000640 1 -- 172.21.15.16:0/1491855669 <== mgr.14220 v2:172.21.15.16:6800/4279751337 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258835 (secure 0 0 0) 0x7f8d040635a0 con 0x7f8cec0b4c10 2024-08-31T06:05:00.008 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:05:00.009 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-08-31T06:05:00.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:00.007+0000 7f8d0aa00640 1 -- 172.21.15.16:0/1491855669 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f8cec0b4c10 msgr2=0x7f8cec0b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:00.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:00.007+0000 7f8d0aa00640 1 --2- 172.21.15.16:0/1491855669 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f8cec0b4c10 0x7f8cec0b7120 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8031440 tx=0x7f8cf8009560 comp rx=0 tx=0).stop 2024-08-31T06:05:00.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:00.007+0000 7f8d0aa00640 1 -- 172.21.15.16:0/1491855669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d04077120 msgr2=0x7f8d041a0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:00.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:00.007+0000 7f8d0aa00640 1 --2- 172.21.15.16:0/1491855669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d04077120 0x7f8d041a0c10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8cf000d770 tx=0x7f8cf000dc30 comp rx=0 tx=0).stop 2024-08-31T06:05:00.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:00.007+0000 7f8d0aa00640 1 -- 172.21.15.16:0/1491855669 shutdown_connections 2024-08-31T06:05:00.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:00.007+0000 7f8d0aa00640 1 --2- 172.21.15.16:0/1491855669 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f8cec0b4c10 0x7f8cec0b7120 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:00.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:00.007+0000 7f8d0aa00640 1 --2- 172.21.15.16:0/1491855669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d04114020 0x7f8d041a5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:00.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:00.007+0000 7f8d0aa00640 1 --2- 172.21.15.16:0/1491855669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d04077120 0x7f8d041a0c10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:00.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:00.007+0000 7f8d0aa00640 1 --2- 172.21.15.16:0/1491855669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d04075b70 0x7f8d041a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:00.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:00.007+0000 7f8d0aa00640 1 -- 172.21.15.16:0/1491855669 >> 172.21.15.16:0/1491855669 conn(0x7f8d040ff1e0 msgr2=0x7f8d0410ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:00.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:00.007+0000 7f8d0aa00640 1 -- 172.21.15.16:0/1491855669 shutdown_connections 2024-08-31T06:05:00.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:00.007+0000 7f8d0aa00640 1 -- 172.21.15.16:0/1491855669 wait complete. 2024-08-31T06:05:00.836 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-08-31T06:04:59.262567+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":645104,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":201,"num_read_kb":185,"num_write":284,"num_write_kb":2250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":229,"ondisk_log_size":229,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359640,"kb_used_data":10392,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124582184,"statfs":{"total":1151940427776,"available":1151572156416,"internally_reserved":0,"allocated":10641408,"data_stored":6955332,"data_compressed":108957,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":49514,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":81,"num_write":36,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003084"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.897809+0000","last_change":"2024-08-31T06:04:25.877318+0000","last_active":"2024-08-31T06:04:27.897809+0000","last_peered":"2024-08-31T06:04:27.897809+0000","last_clean":"2024-08-31T06:04:27.897809+0000","last_became_active":"2024-08-31T06:04:25.876894+0000","last_became_peered":"2024-08-31T06:04:25.876894+0000","last_unstale":"2024-08-31T06:04:27.897809+0000","last_undegraded":"2024-08-31T06:04:27.897809+0000","last_fullsized":"2024-08-31T06:04:27.897809+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:41:09.449343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.891705+0000","last_change":"2024-08-31T06:04:25.885670+0000","last_active":"2024-08-31T06:04:27.891705+0000","last_peered":"2024-08-31T06:04:27.891705+0000","last_clean":"2024-08-31T06:04:27.891705+0000","last_became_active":"2024-08-31T06:04:25.883818+0000","last_became_peered":"2024-08-31T06:04:25.883818+0000","last_unstale":"2024-08-31T06:04:27.891705+0000","last_undegraded":"2024-08-31T06:04:27.891705+0000","last_fullsized":"2024-08-31T06:04:27.891705+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:40:01.344432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.891292+0000","last_change":"2024-08-31T06:04:25.883375+0000","last_active":"2024-08-31T06:04:27.891292+0000","last_peered":"2024-08-31T06:04:27.891292+0000","last_clean":"2024-08-31T06:04:27.891292+0000","last_became_active":"2024-08-31T06:04:25.881750+0000","last_became_peered":"2024-08-31T06:04:25.881750+0000","last_unstale":"2024-08-31T06:04:27.891292+0000","last_undegraded":"2024-08-31T06:04:27.891292+0000","last_fullsized":"2024-08-31T06:04:27.891292+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:46:46.757363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.893661+0000","last_change":"2024-08-31T06:04:25.882436+0000","last_active":"2024-08-31T06:04:27.893661+0000","last_peered":"2024-08-31T06:04:27.893661+0000","last_clean":"2024-08-31T06:04:27.893661+0000","last_became_active":"2024-08-31T06:04:25.881805+0000","last_became_peered":"2024-08-31T06:04:25.881805+0000","last_unstale":"2024-08-31T06:04:27.893661+0000","last_undegraded":"2024-08-31T06:04:27.893661+0000","last_fullsized":"2024-08-31T06:04:27.893661+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:42:39.793625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.887346+0000","last_change":"2024-08-31T06:04:25.894534+0000","last_active":"2024-08-31T06:04:27.887346+0000","last_peered":"2024-08-31T06:04:27.887346+0000","last_clean":"2024-08-31T06:04:27.887346+0000","last_became_active":"2024-08-31T06:04:25.894116+0000","last_became_peered":"2024-08-31T06:04:25.894116+0000","last_unstale":"2024-08-31T06:04:27.887346+0000","last_undegraded":"2024-08-31T06:04:27.887346+0000","last_fullsized":"2024-08-31T06:04:27.887346+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:02:31.257454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.872608+0000","last_change":"2024-08-31T06:04:25.886182+0000","last_active":"2024-08-31T06:04:27.872608+0000","last_peered":"2024-08-31T06:04:27.872608+0000","last_clean":"2024-08-31T06:04:27.872608+0000","last_became_active":"2024-08-31T06:04:25.885825+0000","last_became_peered":"2024-08-31T06:04:25.885825+0000","last_unstale":"2024-08-31T06:04:27.872608+0000","last_undegraded":"2024-08-31T06:04:27.872608+0000","last_fullsized":"2024-08-31T06:04:27.872608+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:58:12.427931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.889404+0000","last_change":"2024-08-31T06:04:25.877521+0000","last_active":"2024-08-31T06:04:27.889404+0000","last_peered":"2024-08-31T06:04:27.889404+0000","last_clean":"2024-08-31T06:04:27.889404+0000","last_became_active":"2024-08-31T06:04:25.877122+0000","last_became_peered":"2024-08-31T06:04:25.877122+0000","last_unstale":"2024-08-31T06:04:27.889404+0000","last_undegraded":"2024-08-31T06:04:27.889404+0000","last_fullsized":"2024-08-31T06:04:27.889404+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:17:27.406828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.893783+0000","last_change":"2024-08-31T06:04:25.892547+0000","last_active":"2024-08-31T06:04:27.893783+0000","last_peered":"2024-08-31T06:04:27.893783+0000","last_clean":"2024-08-31T06:04:27.893783+0000","last_became_active":"2024-08-31T06:04:25.892090+0000","last_became_peered":"2024-08-31T06:04:25.892090+0000","last_unstale":"2024-08-31T06:04:27.893783+0000","last_undegraded":"2024-08-31T06:04:27.893783+0000","last_fullsized":"2024-08-31T06:04:27.893783+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:28:53.414252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.888975+0000","last_change":"2024-08-31T06:04:25.879952+0000","last_active":"2024-08-31T06:04:27.888975+0000","last_peered":"2024-08-31T06:04:27.888975+0000","last_clean":"2024-08-31T06:04:27.888975+0000","last_became_active":"2024-08-31T06:04:25.879568+0000","last_became_peered":"2024-08-31T06:04:25.879568+0000","last_unstale":"2024-08-31T06:04:27.888975+0000","last_undegraded":"2024-08-31T06:04:27.888975+0000","last_fullsized":"2024-08-31T06:04:27.888975+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:28:04.558054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.873181+0000","last_change":"2024-08-31T06:04:25.884515+0000","last_active":"2024-08-31T06:04:27.873181+0000","last_peered":"2024-08-31T06:04:27.873181+0000","last_clean":"2024-08-31T06:04:27.873181+0000","last_became_active":"2024-08-31T06:04:25.884016+0000","last_became_peered":"2024-08-31T06:04:25.884016+0000","last_unstale":"2024-08-31T06:04:27.873181+0000","last_undegraded":"2024-08-31T06:04:27.873181+0000","last_fullsized":"2024-08-31T06:04:27.873181+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:37:06.835127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.890800+0000","last_change":"2024-08-31T06:04:25.883984+0000","last_active":"2024-08-31T06:04:27.890800+0000","last_peered":"2024-08-31T06:04:27.890800+0000","last_clean":"2024-08-31T06:04:27.890800+0000","last_became_active":"2024-08-31T06:04:25.883353+0000","last_became_peered":"2024-08-31T06:04:25.883353+0000","last_unstale":"2024-08-31T06:04:27.890800+0000","last_undegraded":"2024-08-31T06:04:27.890800+0000","last_fullsized":"2024-08-31T06:04:27.890800+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:16:25.178952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.893944+0000","last_change":"2024-08-31T06:04:25.883335+0000","last_active":"2024-08-31T06:04:27.893944+0000","last_peered":"2024-08-31T06:04:27.893944+0000","last_clean":"2024-08-31T06:04:27.893944+0000","last_became_active":"2024-08-31T06:04:25.881488+0000","last_became_peered":"2024-08-31T06:04:25.881488+0000","last_unstale":"2024-08-31T06:04:27.893944+0000","last_undegraded":"2024-08-31T06:04:27.893944+0000","last_fullsized":"2024-08-31T06:04:27.893944+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:57:24.472482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.890379+0000","last_change":"2024-08-31T06:04:25.891579+0000","last_active":"2024-08-31T06:04:27.890379+0000","last_peered":"2024-08-31T06:04:27.890379+0000","last_clean":"2024-08-31T06:04:27.890379+0000","last_became_active":"2024-08-31T06:04:25.887950+0000","last_became_peered":"2024-08-31T06:04:25.887950+0000","last_unstale":"2024-08-31T06:04:27.890379+0000","last_undegraded":"2024-08-31T06:04:27.890379+0000","last_fullsized":"2024-08-31T06:04:27.890379+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:38:35.284442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.890715+0000","last_change":"2024-08-31T06:04:25.882185+0000","last_active":"2024-08-31T06:04:27.890715+0000","last_peered":"2024-08-31T06:04:27.890715+0000","last_clean":"2024-08-31T06:04:27.890715+0000","last_became_active":"2024-08-31T06:04:25.876499+0000","last_became_peered":"2024-08-31T06:04:25.876499+0000","last_unstale":"2024-08-31T06:04:27.890715+0000","last_undegraded":"2024-08-31T06:04:27.890715+0000","last_fullsized":"2024-08-31T06:04:27.890715+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:40:01.132353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.890218+0000","last_change":"2024-08-31T06:04:25.883624+0000","last_active":"2024-08-31T06:04:27.890218+0000","last_peered":"2024-08-31T06:04:27.890218+0000","last_clean":"2024-08-31T06:04:27.890218+0000","last_became_active":"2024-08-31T06:04:25.878041+0000","last_became_peered":"2024-08-31T06:04:25.878041+0000","last_unstale":"2024-08-31T06:04:27.890218+0000","last_undegraded":"2024-08-31T06:04:27.890218+0000","last_fullsized":"2024-08-31T06:04:27.890218+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:05:21.278321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.887735+0000","last_change":"2024-08-31T06:04:25.882538+0000","last_active":"2024-08-31T06:04:27.887735+0000","last_peered":"2024-08-31T06:04:27.887735+0000","last_clean":"2024-08-31T06:04:27.887735+0000","last_became_active":"2024-08-31T06:04:25.882201+0000","last_became_peered":"2024-08-31T06:04:25.882201+0000","last_unstale":"2024-08-31T06:04:27.887735+0000","last_undegraded":"2024-08-31T06:04:27.887735+0000","last_fullsized":"2024-08-31T06:04:27.887735+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:47:16.880893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.897587+0000","last_change":"2024-08-31T06:04:25.880376+0000","last_active":"2024-08-31T06:04:27.897587+0000","last_peered":"2024-08-31T06:04:27.897587+0000","last_clean":"2024-08-31T06:04:27.897587+0000","last_became_active":"2024-08-31T06:04:25.877668+0000","last_became_peered":"2024-08-31T06:04:25.877668+0000","last_unstale":"2024-08-31T06:04:27.897587+0000","last_undegraded":"2024-08-31T06:04:27.897587+0000","last_fullsized":"2024-08-31T06:04:27.897587+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:55:32.737406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.872360+0000","last_change":"2024-08-31T06:04:25.875431+0000","last_active":"2024-08-31T06:04:27.872360+0000","last_peered":"2024-08-31T06:04:27.872360+0000","last_clean":"2024-08-31T06:04:27.872360+0000","last_became_active":"2024-08-31T06:04:25.874751+0000","last_became_peered":"2024-08-31T06:04:25.874751+0000","last_unstale":"2024-08-31T06:04:27.872360+0000","last_undegraded":"2024-08-31T06:04:27.872360+0000","last_fullsized":"2024-08-31T06:04:27.872360+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:16:57.695415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.889419+0000","last_change":"2024-08-31T06:04:25.895656+0000","last_active":"2024-08-31T06:04:27.889419+0000","last_peered":"2024-08-31T06:04:27.889419+0000","last_clean":"2024-08-31T06:04:27.889419+0000","last_became_active":"2024-08-31T06:04:25.895272+0000","last_became_peered":"2024-08-31T06:04:25.895272+0000","last_unstale":"2024-08-31T06:04:27.889419+0000","last_undegraded":"2024-08-31T06:04:27.889419+0000","last_fullsized":"2024-08-31T06:04:27.889419+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:18:47.276195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.895664+0000","last_change":"2024-08-31T06:04:25.877283+0000","last_active":"2024-08-31T06:04:27.895664+0000","last_peered":"2024-08-31T06:04:27.895664+0000","last_clean":"2024-08-31T06:04:27.895664+0000","last_became_active":"2024-08-31T06:04:25.876846+0000","last_became_peered":"2024-08-31T06:04:25.876846+0000","last_unstale":"2024-08-31T06:04:27.895664+0000","last_undegraded":"2024-08-31T06:04:27.895664+0000","last_fullsized":"2024-08-31T06:04:27.895664+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:54:05.086844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.890398+0000","last_change":"2024-08-31T06:04:25.884891+0000","last_active":"2024-08-31T06:04:27.890398+0000","last_peered":"2024-08-31T06:04:27.890398+0000","last_clean":"2024-08-31T06:04:27.890398+0000","last_became_active":"2024-08-31T06:04:25.884501+0000","last_became_peered":"2024-08-31T06:04:25.884501+0000","last_unstale":"2024-08-31T06:04:27.890398+0000","last_undegraded":"2024-08-31T06:04:27.890398+0000","last_fullsized":"2024-08-31T06:04:27.890398+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:22:54.846102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.888302+0000","last_change":"2024-08-31T06:04:25.877150+0000","last_active":"2024-08-31T06:04:27.888302+0000","last_peered":"2024-08-31T06:04:27.888302+0000","last_clean":"2024-08-31T06:04:27.888302+0000","last_became_active":"2024-08-31T06:04:25.876766+0000","last_became_peered":"2024-08-31T06:04:25.876766+0000","last_unstale":"2024-08-31T06:04:27.888302+0000","last_undegraded":"2024-08-31T06:04:27.888302+0000","last_fullsized":"2024-08-31T06:04:27.888302+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:46:26.724970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.895942+0000","last_change":"2024-08-31T06:04:25.881275+0000","last_active":"2024-08-31T06:04:27.895942+0000","last_peered":"2024-08-31T06:04:27.895942+0000","last_clean":"2024-08-31T06:04:27.895942+0000","last_became_active":"2024-08-31T06:04:25.879877+0000","last_became_peered":"2024-08-31T06:04:25.879877+0000","last_unstale":"2024-08-31T06:04:27.895942+0000","last_undegraded":"2024-08-31T06:04:27.895942+0000","last_fullsized":"2024-08-31T06:04:27.895942+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:07:07.237041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.872617+0000","last_change":"2024-08-31T06:04:25.895750+0000","last_active":"2024-08-31T06:04:27.872617+0000","last_peered":"2024-08-31T06:04:27.872617+0000","last_clean":"2024-08-31T06:04:27.872617+0000","last_became_active":"2024-08-31T06:04:25.895348+0000","last_became_peered":"2024-08-31T06:04:25.895348+0000","last_unstale":"2024-08-31T06:04:27.872617+0000","last_undegraded":"2024-08-31T06:04:27.872617+0000","last_fullsized":"2024-08-31T06:04:27.872617+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:53:10.014808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.895222+0000","last_change":"2024-08-31T06:04:25.895537+0000","last_active":"2024-08-31T06:04:27.895222+0000","last_peered":"2024-08-31T06:04:27.895222+0000","last_clean":"2024-08-31T06:04:27.895222+0000","last_became_active":"2024-08-31T06:04:25.895136+0000","last_became_peered":"2024-08-31T06:04:25.895136+0000","last_unstale":"2024-08-31T06:04:27.895222+0000","last_undegraded":"2024-08-31T06:04:27.895222+0000","last_fullsized":"2024-08-31T06:04:27.895222+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:41:57.554507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.892624+0000","last_change":"2024-08-31T06:04:25.880490+0000","last_active":"2024-08-31T06:04:27.892624+0000","last_peered":"2024-08-31T06:04:27.892624+0000","last_clean":"2024-08-31T06:04:27.892624+0000","last_became_active":"2024-08-31T06:04:25.879269+0000","last_became_peered":"2024-08-31T06:04:25.879269+0000","last_unstale":"2024-08-31T06:04:27.892624+0000","last_undegraded":"2024-08-31T06:04:27.892624+0000","last_fullsized":"2024-08-31T06:04:27.892624+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:05:49.575915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.896665+0000","last_change":"2024-08-31T06:04:25.881676+0000","last_active":"2024-08-31T06:04:27.896665+0000","last_peered":"2024-08-31T06:04:27.896665+0000","last_clean":"2024-08-31T06:04:27.896665+0000","last_became_active":"2024-08-31T06:04:25.878222+0000","last_became_peered":"2024-08-31T06:04:25.878222+0000","last_unstale":"2024-08-31T06:04:27.896665+0000","last_undegraded":"2024-08-31T06:04:27.896665+0000","last_fullsized":"2024-08-31T06:04:27.896665+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:24:54.757553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.889542+0000","last_change":"2024-08-31T06:04:25.900183+0000","last_active":"2024-08-31T06:04:27.889542+0000","last_peered":"2024-08-31T06:04:27.889542+0000","last_clean":"2024-08-31T06:04:27.889542+0000","last_became_active":"2024-08-31T06:04:25.898903+0000","last_became_peered":"2024-08-31T06:04:25.898903+0000","last_unstale":"2024-08-31T06:04:27.889542+0000","last_undegraded":"2024-08-31T06:04:27.889542+0000","last_fullsized":"2024-08-31T06:04:27.889542+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:34:05.796458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.892123+0000","last_change":"2024-08-31T06:04:25.897473+0000","last_active":"2024-08-31T06:04:27.892123+0000","last_peered":"2024-08-31T06:04:27.892123+0000","last_clean":"2024-08-31T06:04:27.892123+0000","last_became_active":"2024-08-31T06:04:25.895194+0000","last_became_peered":"2024-08-31T06:04:25.895194+0000","last_unstale":"2024-08-31T06:04:27.892123+0000","last_undegraded":"2024-08-31T06:04:27.892123+0000","last_fullsized":"2024-08-31T06:04:27.892123+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:43:04.539970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.890583+0000","last_change":"2024-08-31T06:04:25.894402+0000","last_active":"2024-08-31T06:04:27.890583+0000","last_peered":"2024-08-31T06:04:27.890583+0000","last_clean":"2024-08-31T06:04:27.890583+0000","last_became_active":"2024-08-31T06:04:25.893883+0000","last_became_peered":"2024-08-31T06:04:25.893883+0000","last_unstale":"2024-08-31T06:04:27.890583+0000","last_undegraded":"2024-08-31T06:04:27.890583+0000","last_fullsized":"2024-08-31T06:04:27.890583+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:45:53.778473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.891364+0000","last_change":"2024-08-31T06:04:25.881543+0000","last_active":"2024-08-31T06:04:27.891364+0000","last_peered":"2024-08-31T06:04:27.891364+0000","last_clean":"2024-08-31T06:04:27.891364+0000","last_became_active":"2024-08-31T06:04:25.881144+0000","last_became_peered":"2024-08-31T06:04:25.881144+0000","last_unstale":"2024-08-31T06:04:27.891364+0000","last_undegraded":"2024-08-31T06:04:27.891364+0000","last_fullsized":"2024-08-31T06:04:27.891364+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:13:02.282879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.896143+0000","last_change":"2024-08-31T06:04:25.881143+0000","last_active":"2024-08-31T06:04:27.896143+0000","last_peered":"2024-08-31T06:04:27.896143+0000","last_clean":"2024-08-31T06:04:27.896143+0000","last_became_active":"2024-08-31T06:04:25.879718+0000","last_became_peered":"2024-08-31T06:04:25.879718+0000","last_unstale":"2024-08-31T06:04:27.896143+0000","last_undegraded":"2024-08-31T06:04:27.896143+0000","last_fullsized":"2024-08-31T06:04:27.896143+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:32:07.771488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.887567+0000","last_change":"2024-08-31T06:04:25.879493+0000","last_active":"2024-08-31T06:04:27.887567+0000","last_peered":"2024-08-31T06:04:27.887567+0000","last_clean":"2024-08-31T06:04:27.887567+0000","last_became_active":"2024-08-31T06:04:25.879065+0000","last_became_peered":"2024-08-31T06:04:25.879065+0000","last_unstale":"2024-08-31T06:04:27.887567+0000","last_undegraded":"2024-08-31T06:04:27.887567+0000","last_fullsized":"2024-08-31T06:04:27.887567+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:53:43.848269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"78'6","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:57.532123+0000","last_change":"2024-08-31T06:04:22.844727+0000","last_active":"2024-08-31T06:04:57.532123+0000","last_peered":"2024-08-31T06:04:57.532123+0000","last_clean":"2024-08-31T06:04:57.532123+0000","last_became_active":"2024-08-31T06:04:22.844269+0000","last_became_peered":"2024-08-31T06:04:22.844269+0000","last_unstale":"2024-08-31T06:04:57.532123+0000","last_undegraded":"2024-08-31T06:04:57.532123+0000","last_fullsized":"2024-08-31T06:04:57.532123+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:19:03.010029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.872785+0000","last_change":"2024-08-31T06:04:25.896354+0000","last_active":"2024-08-31T06:04:27.872785+0000","last_peered":"2024-08-31T06:04:27.872785+0000","last_clean":"2024-08-31T06:04:27.872785+0000","last_became_active":"2024-08-31T06:04:25.896021+0000","last_became_peered":"2024-08-31T06:04:25.896021+0000","last_unstale":"2024-08-31T06:04:27.872785+0000","last_undegraded":"2024-08-31T06:04:27.872785+0000","last_fullsized":"2024-08-31T06:04:27.872785+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:08:40.213238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:38.940702+0000","last_change":"2024-08-31T06:04:22.844248+0000","last_active":"2024-08-31T06:04:38.940702+0000","last_peered":"2024-08-31T06:04:38.940702+0000","last_clean":"2024-08-31T06:04:38.940702+0000","last_became_active":"2024-08-31T06:04:22.843784+0000","last_became_peered":"2024-08-31T06:04:22.843784+0000","last_unstale":"2024-08-31T06:04:38.940702+0000","last_undegraded":"2024-08-31T06:04:38.940702+0000","last_fullsized":"2024-08-31T06:04:38.940702+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:56:41.422965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:38.942677+0000","last_change":"2024-08-31T06:04:22.847049+0000","last_active":"2024-08-31T06:04:38.942677+0000","last_peered":"2024-08-31T06:04:38.942677+0000","last_clean":"2024-08-31T06:04:38.942677+0000","last_became_active":"2024-08-31T06:04:22.846282+0000","last_became_peered":"2024-08-31T06:04:22.846282+0000","last_unstale":"2024-08-31T06:04:38.942677+0000","last_undegraded":"2024-08-31T06:04:38.942677+0000","last_fullsized":"2024-08-31T06:04:38.942677+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:35:21.030425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.893479+0000","last_change":"2024-08-31T06:04:25.886698+0000","last_active":"2024-08-31T06:04:27.893479+0000","last_peered":"2024-08-31T06:04:27.893479+0000","last_clean":"2024-08-31T06:04:27.893479+0000","last_became_active":"2024-08-31T06:04:25.886334+0000","last_became_peered":"2024-08-31T06:04:25.886334+0000","last_unstale":"2024-08-31T06:04:27.893479+0000","last_undegraded":"2024-08-31T06:04:27.893479+0000","last_fullsized":"2024-08-31T06:04:27.893479+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:24:43.821446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.873058+0000","last_change":"2024-08-31T06:04:22.845263+0000","last_active":"2024-08-31T06:04:27.873058+0000","last_peered":"2024-08-31T06:04:27.873058+0000","last_clean":"2024-08-31T06:04:27.873058+0000","last_became_active":"2024-08-31T06:04:22.844853+0000","last_became_peered":"2024-08-31T06:04:22.844853+0000","last_unstale":"2024-08-31T06:04:27.873058+0000","last_undegraded":"2024-08-31T06:04:27.873058+0000","last_fullsized":"2024-08-31T06:04:27.873058+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:55:12.561165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.872038+0000","last_change":"2024-08-31T06:04:25.894475+0000","last_active":"2024-08-31T06:04:27.872038+0000","last_peered":"2024-08-31T06:04:27.872038+0000","last_clean":"2024-08-31T06:04:27.872038+0000","last_became_active":"2024-08-31T06:04:25.894121+0000","last_became_peered":"2024-08-31T06:04:25.894121+0000","last_unstale":"2024-08-31T06:04:27.872038+0000","last_undegraded":"2024-08-31T06:04:27.872038+0000","last_fullsized":"2024-08-31T06:04:27.872038+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:40:06.570616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'3","reported_seq":40,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:53.524875+0000","last_change":"2024-08-31T06:04:22.834430+0000","last_active":"2024-08-31T06:04:53.524875+0000","last_peered":"2024-08-31T06:04:53.524875+0000","last_clean":"2024-08-31T06:04:53.524875+0000","last_became_active":"2024-08-31T06:04:22.834011+0000","last_became_peered":"2024-08-31T06:04:22.834011+0000","last_unstale":"2024-08-31T06:04:53.524875+0000","last_undegraded":"2024-08-31T06:04:53.524875+0000","last_fullsized":"2024-08-31T06:04:53.524875+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:51:16.734234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.888195+0000","last_change":"2024-08-31T06:04:25.877727+0000","last_active":"2024-08-31T06:04:27.888195+0000","last_peered":"2024-08-31T06:04:27.888195+0000","last_clean":"2024-08-31T06:04:27.888195+0000","last_became_active":"2024-08-31T06:04:25.877121+0000","last_became_peered":"2024-08-31T06:04:25.877121+0000","last_unstale":"2024-08-31T06:04:27.888195+0000","last_undegraded":"2024-08-31T06:04:27.888195+0000","last_fullsized":"2024-08-31T06:04:27.888195+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:41:12.247216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:37.935281+0000","last_change":"2024-08-31T06:04:22.846482+0000","last_active":"2024-08-31T06:04:37.935281+0000","last_peered":"2024-08-31T06:04:37.935281+0000","last_clean":"2024-08-31T06:04:37.935281+0000","last_became_active":"2024-08-31T06:04:22.846057+0000","last_became_peered":"2024-08-31T06:04:22.846057+0000","last_unstale":"2024-08-31T06:04:37.935281+0000","last_undegraded":"2024-08-31T06:04:37.935281+0000","last_fullsized":"2024-08-31T06:04:37.935281+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:29:11.861665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.891600+0000","last_change":"2024-08-31T06:04:25.893777+0000","last_active":"2024-08-31T06:04:27.891600+0000","last_peered":"2024-08-31T06:04:27.891600+0000","last_clean":"2024-08-31T06:04:27.891600+0000","last_became_active":"2024-08-31T06:04:25.893014+0000","last_became_peered":"2024-08-31T06:04:25.893014+0000","last_unstale":"2024-08-31T06:04:27.891600+0000","last_undegraded":"2024-08-31T06:04:27.891600+0000","last_fullsized":"2024-08-31T06:04:27.891600+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:04:48.185842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:37.939192+0000","last_change":"2024-08-31T06:04:22.965938+0000","last_active":"2024-08-31T06:04:37.939192+0000","last_peered":"2024-08-31T06:04:37.939192+0000","last_clean":"2024-08-31T06:04:37.939192+0000","last_became_active":"2024-08-31T06:04:22.965356+0000","last_became_peered":"2024-08-31T06:04:22.965356+0000","last_unstale":"2024-08-31T06:04:37.939192+0000","last_undegraded":"2024-08-31T06:04:37.939192+0000","last_fullsized":"2024-08-31T06:04:37.939192+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:36:58.938462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.889350+0000","last_change":"2024-08-31T06:04:25.881307+0000","last_active":"2024-08-31T06:04:27.889350+0000","last_peered":"2024-08-31T06:04:27.889350+0000","last_clean":"2024-08-31T06:04:27.889350+0000","last_became_active":"2024-08-31T06:04:25.880418+0000","last_became_peered":"2024-08-31T06:04:25.880418+0000","last_unstale":"2024-08-31T06:04:27.889350+0000","last_undegraded":"2024-08-31T06:04:27.889350+0000","last_fullsized":"2024-08-31T06:04:27.889350+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:21:46.904371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"78'25","reported_seq":84,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:54.528078+0000","last_change":"2024-08-31T06:04:22.843073+0000","last_active":"2024-08-31T06:04:54.528078+0000","last_peered":"2024-08-31T06:04:54.528078+0000","last_clean":"2024-08-31T06:04:54.528078+0000","last_became_active":"2024-08-31T06:04:22.842645+0000","last_became_peered":"2024-08-31T06:04:22.842645+0000","last_unstale":"2024-08-31T06:04:54.528078+0000","last_undegraded":"2024-08-31T06:04:54.528078+0000","last_fullsized":"2024-08-31T06:04:54.528078+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:41:37.651562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":37,"num_read_kb":66,"num_write":26,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.887999+0000","last_change":"2024-08-31T06:04:25.879929+0000","last_active":"2024-08-31T06:04:27.887999+0000","last_peered":"2024-08-31T06:04:27.887999+0000","last_clean":"2024-08-31T06:04:27.887999+0000","last_became_active":"2024-08-31T06:04:25.878475+0000","last_became_peered":"2024-08-31T06:04:25.878475+0000","last_unstale":"2024-08-31T06:04:27.887999+0000","last_undegraded":"2024-08-31T06:04:27.887999+0000","last_fullsized":"2024-08-31T06:04:27.887999+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:10:32.801558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":30,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:38.942569+0000","last_change":"2024-08-31T06:04:22.842878+0000","last_active":"2024-08-31T06:04:38.942569+0000","last_peered":"2024-08-31T06:04:38.942569+0000","last_clean":"2024-08-31T06:04:38.942569+0000","last_became_active":"2024-08-31T06:04:22.842306+0000","last_became_peered":"2024-08-31T06:04:22.842306+0000","last_unstale":"2024-08-31T06:04:38.942569+0000","last_undegraded":"2024-08-31T06:04:38.942569+0000","last_fullsized":"2024-08-31T06:04:38.942569+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:53:55.259238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.888749+0000","last_change":"2024-08-31T06:04:25.899357+0000","last_active":"2024-08-31T06:04:27.888749+0000","last_peered":"2024-08-31T06:04:27.888749+0000","last_clean":"2024-08-31T06:04:27.888749+0000","last_became_active":"2024-08-31T06:04:25.899000+0000","last_became_peered":"2024-08-31T06:04:25.899000+0000","last_unstale":"2024-08-31T06:04:27.888749+0000","last_undegraded":"2024-08-31T06:04:27.888749+0000","last_fullsized":"2024-08-31T06:04:27.888749+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:20:39.991071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:37.938984+0000","last_change":"2024-08-31T06:04:22.843063+0000","last_active":"2024-08-31T06:04:37.938984+0000","last_peered":"2024-08-31T06:04:37.938984+0000","last_clean":"2024-08-31T06:04:37.938984+0000","last_became_active":"2024-08-31T06:04:22.842654+0000","last_became_peered":"2024-08-31T06:04:22.842654+0000","last_unstale":"2024-08-31T06:04:37.938984+0000","last_undegraded":"2024-08-31T06:04:37.938984+0000","last_fullsized":"2024-08-31T06:04:37.938984+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:09:54.563116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.890628+0000","last_change":"2024-08-31T06:04:25.885556+0000","last_active":"2024-08-31T06:04:27.890628+0000","last_peered":"2024-08-31T06:04:27.890628+0000","last_clean":"2024-08-31T06:04:27.890628+0000","last_became_active":"2024-08-31T06:04:25.885216+0000","last_became_peered":"2024-08-31T06:04:25.885216+0000","last_unstale":"2024-08-31T06:04:27.890628+0000","last_undegraded":"2024-08-31T06:04:27.890628+0000","last_fullsized":"2024-08-31T06:04:27.890628+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:48:49.816261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'4","reported_seq":43,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:52.524485+0000","last_change":"2024-08-31T06:04:22.844839+0000","last_active":"2024-08-31T06:04:52.524485+0000","last_peered":"2024-08-31T06:04:52.524485+0000","last_clean":"2024-08-31T06:04:52.524485+0000","last_became_active":"2024-08-31T06:04:22.844425+0000","last_became_peered":"2024-08-31T06:04:22.844425+0000","last_unstale":"2024-08-31T06:04:52.524485+0000","last_undegraded":"2024-08-31T06:04:52.524485+0000","last_fullsized":"2024-08-31T06:04:52.524485+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:10:57.292173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.890010+0000","last_change":"2024-08-31T06:04:25.878195+0000","last_active":"2024-08-31T06:04:27.890010+0000","last_peered":"2024-08-31T06:04:27.890010+0000","last_clean":"2024-08-31T06:04:27.890010+0000","last_became_active":"2024-08-31T06:04:25.877823+0000","last_became_peered":"2024-08-31T06:04:25.877823+0000","last_unstale":"2024-08-31T06:04:27.890010+0000","last_undegraded":"2024-08-31T06:04:27.890010+0000","last_fullsized":"2024-08-31T06:04:27.890010+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:54:34.989583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:38.944046+0000","last_change":"2024-08-31T06:04:22.965488+0000","last_active":"2024-08-31T06:04:38.944046+0000","last_peered":"2024-08-31T06:04:38.944046+0000","last_clean":"2024-08-31T06:04:38.944046+0000","last_became_active":"2024-08-31T06:04:22.965066+0000","last_became_peered":"2024-08-31T06:04:22.965066+0000","last_unstale":"2024-08-31T06:04:38.944046+0000","last_undegraded":"2024-08-31T06:04:38.944046+0000","last_fullsized":"2024-08-31T06:04:38.944046+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:26:48.079318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.872606+0000","last_change":"2024-08-31T06:04:25.887580+0000","last_active":"2024-08-31T06:04:27.872606+0000","last_peered":"2024-08-31T06:04:27.872606+0000","last_clean":"2024-08-31T06:04:27.872606+0000","last_became_active":"2024-08-31T06:04:25.887158+0000","last_became_peered":"2024-08-31T06:04:25.887158+0000","last_unstale":"2024-08-31T06:04:27.872606+0000","last_undegraded":"2024-08-31T06:04:27.872606+0000","last_fullsized":"2024-08-31T06:04:27.872606+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:27:38.115742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:40.500346+0000","last_change":"2024-08-31T06:04:22.843806+0000","last_active":"2024-08-31T06:04:40.500346+0000","last_peered":"2024-08-31T06:04:40.500346+0000","last_clean":"2024-08-31T06:04:40.500346+0000","last_became_active":"2024-08-31T06:04:22.843218+0000","last_became_peered":"2024-08-31T06:04:22.843218+0000","last_unstale":"2024-08-31T06:04:40.500346+0000","last_undegraded":"2024-08-31T06:04:40.500346+0000","last_fullsized":"2024-08-31T06:04:40.500346+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:56:12.019875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.896175+0000","last_change":"2024-08-31T06:04:25.880845+0000","last_active":"2024-08-31T06:04:27.896175+0000","last_peered":"2024-08-31T06:04:27.896175+0000","last_clean":"2024-08-31T06:04:27.896175+0000","last_became_active":"2024-08-31T06:04:25.880423+0000","last_became_peered":"2024-08-31T06:04:25.880423+0000","last_unstale":"2024-08-31T06:04:27.896175+0000","last_undegraded":"2024-08-31T06:04:27.896175+0000","last_fullsized":"2024-08-31T06:04:27.896175+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:16:56.725116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.898554+0000","last_change":"2024-08-31T06:04:22.843393+0000","last_active":"2024-08-31T06:04:27.898554+0000","last_peered":"2024-08-31T06:04:27.898554+0000","last_clean":"2024-08-31T06:04:27.898554+0000","last_became_active":"2024-08-31T06:04:22.842770+0000","last_became_peered":"2024-08-31T06:04:22.842770+0000","last_unstale":"2024-08-31T06:04:27.898554+0000","last_undegraded":"2024-08-31T06:04:27.898554+0000","last_fullsized":"2024-08-31T06:04:27.898554+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:49:16.159167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.891050+0000","last_change":"2024-08-31T06:04:25.887829+0000","last_active":"2024-08-31T06:04:27.891050+0000","last_peered":"2024-08-31T06:04:27.891050+0000","last_clean":"2024-08-31T06:04:27.891050+0000","last_became_active":"2024-08-31T06:04:25.886567+0000","last_became_peered":"2024-08-31T06:04:25.886567+0000","last_unstale":"2024-08-31T06:04:27.891050+0000","last_undegraded":"2024-08-31T06:04:27.891050+0000","last_fullsized":"2024-08-31T06:04:27.891050+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:11:39.054966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.895607+0000","last_change":"2024-08-31T06:04:22.844218+0000","last_active":"2024-08-31T06:04:27.895607+0000","last_peered":"2024-08-31T06:04:27.895607+0000","last_clean":"2024-08-31T06:04:27.895607+0000","last_became_active":"2024-08-31T06:04:22.843786+0000","last_became_peered":"2024-08-31T06:04:22.843786+0000","last_unstale":"2024-08-31T06:04:27.895607+0000","last_undegraded":"2024-08-31T06:04:27.895607+0000","last_fullsized":"2024-08-31T06:04:27.895607+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:46:02.534190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'1","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.890593+0000","last_change":"2024-08-31T06:04:25.888946+0000","last_active":"2024-08-31T06:04:27.890593+0000","last_peered":"2024-08-31T06:04:27.890593+0000","last_clean":"2024-08-31T06:04:27.890593+0000","last_became_active":"2024-08-31T06:04:25.888564+0000","last_became_peered":"2024-08-31T06:04:25.888564+0000","last_unstale":"2024-08-31T06:04:27.890593+0000","last_undegraded":"2024-08-31T06:04:27.890593+0000","last_fullsized":"2024-08-31T06:04:27.890593+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:26:19.051971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:38.944078+0000","last_change":"2024-08-31T06:04:22.964483+0000","last_active":"2024-08-31T06:04:38.944078+0000","last_peered":"2024-08-31T06:04:38.944078+0000","last_clean":"2024-08-31T06:04:38.944078+0000","last_became_active":"2024-08-31T06:04:22.963874+0000","last_became_peered":"2024-08-31T06:04:22.963874+0000","last_unstale":"2024-08-31T06:04:38.944078+0000","last_undegraded":"2024-08-31T06:04:38.944078+0000","last_fullsized":"2024-08-31T06:04:38.944078+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:30:00.159040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.890863+0000","last_change":"2024-08-31T06:04:25.880941+0000","last_active":"2024-08-31T06:04:27.890863+0000","last_peered":"2024-08-31T06:04:27.890863+0000","last_clean":"2024-08-31T06:04:27.890863+0000","last_became_active":"2024-08-31T06:04:25.880454+0000","last_became_peered":"2024-08-31T06:04:25.880454+0000","last_unstale":"2024-08-31T06:04:27.890863+0000","last_undegraded":"2024-08-31T06:04:27.890863+0000","last_fullsized":"2024-08-31T06:04:27.890863+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:49:57.023988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":93,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.890458+0000","last_change":"2024-08-31T06:03:05.921629+0000","last_active":"2024-08-31T06:04:27.890458+0000","last_peered":"2024-08-31T06:04:27.890458+0000","last_clean":"2024-08-31T06:04:27.890458+0000","last_became_active":"2024-08-31T06:03:05.915844+0000","last_became_peered":"2024-08-31T06:03:05.915844+0000","last_unstale":"2024-08-31T06:04:27.890458+0000","last_undegraded":"2024-08-31T06:04:27.890458+0000","last_fullsized":"2024-08-31T06:04:27.890458+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T05:59:24.472429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T05:59:24.472429+0000","last_clean_scrub_stamp":"2024-08-31T05:59:24.472429+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:56:41.073422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:40.501184+0000","last_change":"2024-08-31T06:04:22.846325+0000","last_active":"2024-08-31T06:04:40.501184+0000","last_peered":"2024-08-31T06:04:40.501184+0000","last_clean":"2024-08-31T06:04:40.501184+0000","last_became_active":"2024-08-31T06:04:22.845772+0000","last_became_peered":"2024-08-31T06:04:22.845772+0000","last_unstale":"2024-08-31T06:04:40.501184+0000","last_undegraded":"2024-08-31T06:04:40.501184+0000","last_fullsized":"2024-08-31T06:04:40.501184+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:54:10.336786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.896714+0000","last_change":"2024-08-31T06:04:25.891437+0000","last_active":"2024-08-31T06:04:27.896714+0000","last_peered":"2024-08-31T06:04:27.896714+0000","last_clean":"2024-08-31T06:04:27.896714+0000","last_became_active":"2024-08-31T06:04:25.891053+0000","last_became_peered":"2024-08-31T06:04:25.891053+0000","last_unstale":"2024-08-31T06:04:27.896714+0000","last_undegraded":"2024-08-31T06:04:27.896714+0000","last_fullsized":"2024-08-31T06:04:27.896714+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:22:37.119483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:38.940337+0000","last_change":"2024-08-31T06:04:22.842885+0000","last_active":"2024-08-31T06:04:38.940337+0000","last_peered":"2024-08-31T06:04:38.940337+0000","last_clean":"2024-08-31T06:04:38.940337+0000","last_became_active":"2024-08-31T06:04:22.842473+0000","last_became_peered":"2024-08-31T06:04:22.842473+0000","last_unstale":"2024-08-31T06:04:38.940337+0000","last_undegraded":"2024-08-31T06:04:38.940337+0000","last_fullsized":"2024-08-31T06:04:38.940337+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:33:36.516659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.887889+0000","last_change":"2024-08-31T06:04:25.889213+0000","last_active":"2024-08-31T06:04:27.887889+0000","last_peered":"2024-08-31T06:04:27.887889+0000","last_clean":"2024-08-31T06:04:27.887889+0000","last_became_active":"2024-08-31T06:04:25.888860+0000","last_became_peered":"2024-08-31T06:04:25.888860+0000","last_unstale":"2024-08-31T06:04:27.887889+0000","last_undegraded":"2024-08-31T06:04:27.887889+0000","last_fullsized":"2024-08-31T06:04:27.887889+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:44:18.831229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"78'4","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:53.525394+0000","last_change":"2024-08-31T06:04:22.966091+0000","last_active":"2024-08-31T06:04:53.525394+0000","last_peered":"2024-08-31T06:04:53.525394+0000","last_clean":"2024-08-31T06:04:53.525394+0000","last_became_active":"2024-08-31T06:04:22.965727+0000","last_became_peered":"2024-08-31T06:04:22.965727+0000","last_unstale":"2024-08-31T06:04:53.525394+0000","last_undegraded":"2024-08-31T06:04:53.525394+0000","last_fullsized":"2024-08-31T06:04:53.525394+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:42:43.643623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.872545+0000","last_change":"2024-08-31T06:04:25.880039+0000","last_active":"2024-08-31T06:04:27.872545+0000","last_peered":"2024-08-31T06:04:27.872545+0000","last_clean":"2024-08-31T06:04:27.872545+0000","last_became_active":"2024-08-31T06:04:25.879669+0000","last_became_peered":"2024-08-31T06:04:25.879669+0000","last_unstale":"2024-08-31T06:04:27.872545+0000","last_undegraded":"2024-08-31T06:04:27.872545+0000","last_fullsized":"2024-08-31T06:04:27.872545+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:02:09.244281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.890686+0000","last_change":"2024-08-31T06:04:22.844572+0000","last_active":"2024-08-31T06:04:27.890686+0000","last_peered":"2024-08-31T06:04:27.890686+0000","last_clean":"2024-08-31T06:04:27.890686+0000","last_became_active":"2024-08-31T06:04:22.844227+0000","last_became_peered":"2024-08-31T06:04:22.844227+0000","last_unstale":"2024-08-31T06:04:27.890686+0000","last_undegraded":"2024-08-31T06:04:27.890686+0000","last_fullsized":"2024-08-31T06:04:27.890686+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:04:51.966901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.872666+0000","last_change":"2024-08-31T06:04:25.884442+0000","last_active":"2024-08-31T06:04:27.872666+0000","last_peered":"2024-08-31T06:04:27.872666+0000","last_clean":"2024-08-31T06:04:27.872666+0000","last_became_active":"2024-08-31T06:04:25.884016+0000","last_became_peered":"2024-08-31T06:04:25.884016+0000","last_unstale":"2024-08-31T06:04:27.872666+0000","last_undegraded":"2024-08-31T06:04:27.872666+0000","last_fullsized":"2024-08-31T06:04:27.872666+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:36:10.640262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.872083+0000","last_change":"2024-08-31T06:04:22.967845+0000","last_active":"2024-08-31T06:04:27.872083+0000","last_peered":"2024-08-31T06:04:27.872083+0000","last_clean":"2024-08-31T06:04:27.872083+0000","last_became_active":"2024-08-31T06:04:22.967421+0000","last_became_peered":"2024-08-31T06:04:22.967421+0000","last_unstale":"2024-08-31T06:04:27.872083+0000","last_undegraded":"2024-08-31T06:04:27.872083+0000","last_fullsized":"2024-08-31T06:04:27.872083+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:17:46.942551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'1","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:29.790797+0000","last_change":"2024-08-31T06:04:25.897087+0000","last_active":"2024-08-31T06:04:29.790797+0000","last_peered":"2024-08-31T06:04:29.790797+0000","last_clean":"2024-08-31T06:04:29.790797+0000","last_became_active":"2024-08-31T06:04:25.896725+0000","last_became_peered":"2024-08-31T06:04:25.896725+0000","last_unstale":"2024-08-31T06:04:29.790797+0000","last_undegraded":"2024-08-31T06:04:29.790797+0000","last_fullsized":"2024-08-31T06:04:29.790797+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:31:28.279489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:37.938086+0000","last_change":"2024-08-31T06:04:22.965299+0000","last_active":"2024-08-31T06:04:37.938086+0000","last_peered":"2024-08-31T06:04:37.938086+0000","last_clean":"2024-08-31T06:04:37.938086+0000","last_became_active":"2024-08-31T06:04:22.964849+0000","last_became_peered":"2024-08-31T06:04:22.964849+0000","last_unstale":"2024-08-31T06:04:37.938086+0000","last_undegraded":"2024-08-31T06:04:37.938086+0000","last_fullsized":"2024-08-31T06:04:37.938086+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:36:56.431678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.889796+0000","last_change":"2024-08-31T06:04:25.885115+0000","last_active":"2024-08-31T06:04:27.889796+0000","last_peered":"2024-08-31T06:04:27.889796+0000","last_clean":"2024-08-31T06:04:27.889796+0000","last_became_active":"2024-08-31T06:04:25.884722+0000","last_became_peered":"2024-08-31T06:04:25.884722+0000","last_unstale":"2024-08-31T06:04:27.889796+0000","last_undegraded":"2024-08-31T06:04:27.889796+0000","last_fullsized":"2024-08-31T06:04:27.889796+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:20:43.913785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'10","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:55.888270+0000","last_change":"2024-08-31T06:04:22.845119+0000","last_active":"2024-08-31T06:04:55.888270+0000","last_peered":"2024-08-31T06:04:55.888270+0000","last_clean":"2024-08-31T06:04:55.888270+0000","last_became_active":"2024-08-31T06:04:22.843814+0000","last_became_peered":"2024-08-31T06:04:22.843814+0000","last_unstale":"2024-08-31T06:04:55.888270+0000","last_undegraded":"2024-08-31T06:04:55.888270+0000","last_fullsized":"2024-08-31T06:04:55.888270+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:06:40.684887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.895548+0000","last_change":"2024-08-31T06:04:25.892838+0000","last_active":"2024-08-31T06:04:27.895548+0000","last_peered":"2024-08-31T06:04:27.895548+0000","last_clean":"2024-08-31T06:04:27.895548+0000","last_became_active":"2024-08-31T06:04:25.891955+0000","last_became_peered":"2024-08-31T06:04:25.891955+0000","last_unstale":"2024-08-31T06:04:27.895548+0000","last_undegraded":"2024-08-31T06:04:27.895548+0000","last_fullsized":"2024-08-31T06:04:27.895548+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:41:35.085724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:37.937713+0000","last_change":"2024-08-31T06:04:22.964183+0000","last_active":"2024-08-31T06:04:37.937713+0000","last_peered":"2024-08-31T06:04:37.937713+0000","last_clean":"2024-08-31T06:04:37.937713+0000","last_became_active":"2024-08-31T06:04:22.963688+0000","last_became_peered":"2024-08-31T06:04:22.963688+0000","last_unstale":"2024-08-31T06:04:37.937713+0000","last_undegraded":"2024-08-31T06:04:37.937713+0000","last_fullsized":"2024-08-31T06:04:37.937713+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:04:47.833811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.891792+0000","last_change":"2024-08-31T06:04:25.892266+0000","last_active":"2024-08-31T06:04:27.891792+0000","last_peered":"2024-08-31T06:04:27.891792+0000","last_clean":"2024-08-31T06:04:27.891792+0000","last_became_active":"2024-08-31T06:04:25.889014+0000","last_became_peered":"2024-08-31T06:04:25.889014+0000","last_unstale":"2024-08-31T06:04:27.891792+0000","last_undegraded":"2024-08-31T06:04:27.891792+0000","last_fullsized":"2024-08-31T06:04:27.891792+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:27:47.608470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:37.935935+0000","last_change":"2024-08-31T06:04:22.964677+0000","last_active":"2024-08-31T06:04:37.935935+0000","last_peered":"2024-08-31T06:04:37.935935+0000","last_clean":"2024-08-31T06:04:37.935935+0000","last_became_active":"2024-08-31T06:04:22.964237+0000","last_became_peered":"2024-08-31T06:04:22.964237+0000","last_unstale":"2024-08-31T06:04:37.935935+0000","last_undegraded":"2024-08-31T06:04:37.935935+0000","last_fullsized":"2024-08-31T06:04:37.935935+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:00:54.099592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.887298+0000","last_change":"2024-08-31T06:04:25.881699+0000","last_active":"2024-08-31T06:04:27.887298+0000","last_peered":"2024-08-31T06:04:27.887298+0000","last_clean":"2024-08-31T06:04:27.887298+0000","last_became_active":"2024-08-31T06:04:25.879304+0000","last_became_peered":"2024-08-31T06:04:25.879304+0000","last_unstale":"2024-08-31T06:04:27.887298+0000","last_undegraded":"2024-08-31T06:04:27.887298+0000","last_fullsized":"2024-08-31T06:04:27.887298+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:56:42.992034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:37.939990+0000","last_change":"2024-08-31T06:04:22.847934+0000","last_active":"2024-08-31T06:04:37.939990+0000","last_peered":"2024-08-31T06:04:37.939990+0000","last_clean":"2024-08-31T06:04:37.939990+0000","last_became_active":"2024-08-31T06:04:22.847518+0000","last_became_peered":"2024-08-31T06:04:22.847518+0000","last_unstale":"2024-08-31T06:04:37.939990+0000","last_undegraded":"2024-08-31T06:04:37.939990+0000","last_fullsized":"2024-08-31T06:04:37.939990+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:26:21.476014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.896409+0000","last_change":"2024-08-31T06:04:25.884653+0000","last_active":"2024-08-31T06:04:27.896409+0000","last_peered":"2024-08-31T06:04:27.896409+0000","last_clean":"2024-08-31T06:04:27.896409+0000","last_became_active":"2024-08-31T06:04:25.884244+0000","last_became_peered":"2024-08-31T06:04:25.884244+0000","last_unstale":"2024-08-31T06:04:27.896409+0000","last_undegraded":"2024-08-31T06:04:27.896409+0000","last_fullsized":"2024-08-31T06:04:27.896409+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:13:00.814071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.892362+0000","last_change":"2024-08-31T06:04:22.845924+0000","last_active":"2024-08-31T06:04:27.892362+0000","last_peered":"2024-08-31T06:04:27.892362+0000","last_clean":"2024-08-31T06:04:27.892362+0000","last_became_active":"2024-08-31T06:04:22.845445+0000","last_became_peered":"2024-08-31T06:04:22.845445+0000","last_unstale":"2024-08-31T06:04:27.892362+0000","last_undegraded":"2024-08-31T06:04:27.892362+0000","last_fullsized":"2024-08-31T06:04:27.892362+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:25:21.052136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.894363+0000","last_change":"2024-08-31T06:04:25.890069+0000","last_active":"2024-08-31T06:04:27.894363+0000","last_peered":"2024-08-31T06:04:27.894363+0000","last_clean":"2024-08-31T06:04:27.894363+0000","last_became_active":"2024-08-31T06:04:25.889709+0000","last_became_peered":"2024-08-31T06:04:25.889709+0000","last_unstale":"2024-08-31T06:04:27.894363+0000","last_undegraded":"2024-08-31T06:04:27.894363+0000","last_fullsized":"2024-08-31T06:04:27.894363+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:13:51.360023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:37.936469+0000","last_change":"2024-08-31T06:04:22.847845+0000","last_active":"2024-08-31T06:04:37.936469+0000","last_peered":"2024-08-31T06:04:37.936469+0000","last_clean":"2024-08-31T06:04:37.936469+0000","last_became_active":"2024-08-31T06:04:22.847001+0000","last_became_peered":"2024-08-31T06:04:22.847001+0000","last_unstale":"2024-08-31T06:04:37.936469+0000","last_undegraded":"2024-08-31T06:04:37.936469+0000","last_fullsized":"2024-08-31T06:04:37.936469+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:24:34.209076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.890202+0000","last_change":"2024-08-31T06:04:25.886759+0000","last_active":"2024-08-31T06:04:27.890202+0000","last_peered":"2024-08-31T06:04:27.890202+0000","last_clean":"2024-08-31T06:04:27.890202+0000","last_became_active":"2024-08-31T06:04:25.886176+0000","last_became_peered":"2024-08-31T06:04:25.886176+0000","last_unstale":"2024-08-31T06:04:27.890202+0000","last_undegraded":"2024-08-31T06:04:27.890202+0000","last_fullsized":"2024-08-31T06:04:27.890202+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:11:12.637451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.873125+0000","last_change":"2024-08-31T06:04:25.887461+0000","last_active":"2024-08-31T06:04:27.873125+0000","last_peered":"2024-08-31T06:04:27.873125+0000","last_clean":"2024-08-31T06:04:27.873125+0000","last_became_active":"2024-08-31T06:04:25.887094+0000","last_became_peered":"2024-08-31T06:04:25.887094+0000","last_unstale":"2024-08-31T06:04:27.873125+0000","last_undegraded":"2024-08-31T06:04:27.873125+0000","last_fullsized":"2024-08-31T06:04:27.873125+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:07:14.935691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.894542+0000","last_change":"2024-08-31T06:04:22.842349+0000","last_active":"2024-08-31T06:04:27.894542+0000","last_peered":"2024-08-31T06:04:27.894542+0000","last_clean":"2024-08-31T06:04:27.894542+0000","last_became_active":"2024-08-31T06:04:22.841883+0000","last_became_peered":"2024-08-31T06:04:22.841883+0000","last_unstale":"2024-08-31T06:04:27.894542+0000","last_undegraded":"2024-08-31T06:04:27.894542+0000","last_fullsized":"2024-08-31T06:04:27.894542+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:41:19.021914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.888217+0000","last_change":"2024-08-31T06:04:25.887524+0000","last_active":"2024-08-31T06:04:27.888217+0000","last_peered":"2024-08-31T06:04:27.888217+0000","last_clean":"2024-08-31T06:04:27.888217+0000","last_became_active":"2024-08-31T06:04:25.887176+0000","last_became_peered":"2024-08-31T06:04:25.887176+0000","last_unstale":"2024-08-31T06:04:27.888217+0000","last_undegraded":"2024-08-31T06:04:27.888217+0000","last_fullsized":"2024-08-31T06:04:27.888217+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:08:05.476804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:38.940903+0000","last_change":"2024-08-31T06:04:22.847747+0000","last_active":"2024-08-31T06:04:38.940903+0000","last_peered":"2024-08-31T06:04:38.940903+0000","last_clean":"2024-08-31T06:04:38.940903+0000","last_became_active":"2024-08-31T06:04:22.847332+0000","last_became_peered":"2024-08-31T06:04:22.847332+0000","last_unstale":"2024-08-31T06:04:38.940903+0000","last_undegraded":"2024-08-31T06:04:38.940903+0000","last_fullsized":"2024-08-31T06:04:38.940903+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:41:35.910991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.891374+0000","last_change":"2024-08-31T06:04:25.883110+0000","last_active":"2024-08-31T06:04:27.891374+0000","last_peered":"2024-08-31T06:04:27.891374+0000","last_clean":"2024-08-31T06:04:27.891374+0000","last_became_active":"2024-08-31T06:04:25.880487+0000","last_became_peered":"2024-08-31T06:04:25.880487+0000","last_unstale":"2024-08-31T06:04:27.891374+0000","last_undegraded":"2024-08-31T06:04:27.891374+0000","last_fullsized":"2024-08-31T06:04:27.891374+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:22:13.656078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.898507+0000","last_change":"2024-08-31T06:04:22.964758+0000","last_active":"2024-08-31T06:04:27.898507+0000","last_peered":"2024-08-31T06:04:27.898507+0000","last_clean":"2024-08-31T06:04:27.898507+0000","last_became_active":"2024-08-31T06:04:22.964391+0000","last_became_peered":"2024-08-31T06:04:22.964391+0000","last_unstale":"2024-08-31T06:04:27.898507+0000","last_undegraded":"2024-08-31T06:04:27.898507+0000","last_fullsized":"2024-08-31T06:04:27.898507+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:28:59.190657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.895257+0000","last_change":"2024-08-31T06:04:25.893738+0000","last_active":"2024-08-31T06:04:27.895257+0000","last_peered":"2024-08-31T06:04:27.895257+0000","last_clean":"2024-08-31T06:04:27.895257+0000","last_became_active":"2024-08-31T06:04:25.893308+0000","last_became_peered":"2024-08-31T06:04:25.893308+0000","last_unstale":"2024-08-31T06:04:27.895257+0000","last_undegraded":"2024-08-31T06:04:27.895257+0000","last_fullsized":"2024-08-31T06:04:27.895257+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:26:48.451894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:38.942319+0000","last_change":"2024-08-31T06:04:22.843346+0000","last_active":"2024-08-31T06:04:38.942319+0000","last_peered":"2024-08-31T06:04:38.942319+0000","last_clean":"2024-08-31T06:04:38.942319+0000","last_became_active":"2024-08-31T06:04:22.842249+0000","last_became_peered":"2024-08-31T06:04:22.842249+0000","last_unstale":"2024-08-31T06:04:38.942319+0000","last_undegraded":"2024-08-31T06:04:38.942319+0000","last_fullsized":"2024-08-31T06:04:38.942319+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:40:57.618940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":54502,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":120,"num_read_kb":116,"num_write":159,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":163506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":249373,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710740,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":843776,"data_stored":418917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874263,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":1060899,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037791,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":860160,"data_stored":419153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234020,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":860160,"data_stored":419073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397546,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":1011394,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593775,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":847872,"data_stored":418542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790004,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":1010687,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953530,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":851968,"data_stored":419370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149758,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":864256,"data_stored":418853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411397,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":925696,"data_stored":470186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542217,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":901120,"data_stored":468445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705744,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":868352,"data_stored":419813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34699,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34699,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34699,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":50110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":51851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":52079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-31T06:05:00.841 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph pg dump --format=json 2024-08-31T06:05:01.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:00 smithi016 ceph-mon[17194]: pgmap v294: 97 pgs: 97 active+clean; 630 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-08-31T06:05:01.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:00 smithi016 ceph-mon[17194]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T06:05:01.226 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:05:01.275 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:00 smithi184 ceph-mon[22331]: pgmap v294: 97 pgs: 97 active+clean; 630 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-08-31T06:05:01.275 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:00 smithi184 ceph-mon[22331]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T06:05:01.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:00 smithi155 ceph-mon[21742]: pgmap v294: 97 pgs: 97 active+clean; 630 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-08-31T06:05:01.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:00 smithi155 ceph-mon[21742]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T06:05:01.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:01.856+0000 7fe689600640 1 -- 172.21.15.16:0/1322890470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe684105b20 msgr2=0x7fe684105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:01.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:01.856+0000 7fe689600640 1 --2- 172.21.15.16:0/1322890470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe684105b20 0x7fe684105f30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe674009900 tx=0x7fe67402a410 comp rx=0 tx=0).stop 2024-08-31T06:05:01.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:01.856+0000 7fe689600640 1 -- 172.21.15.16:0/1322890470 shutdown_connections 2024-08-31T06:05:01.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:01.856+0000 7fe689600640 1 --2- 172.21.15.16:0/1322890470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe68410b460 0x7fe68410b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:01.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:01.856+0000 7fe689600640 1 --2- 172.21.15.16:0/1322890470 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe68410bdb0 0x7fe6841125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:01.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:01.856+0000 7fe689600640 1 --2- 172.21.15.16:0/1322890470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe684105b20 0x7fe684105f30 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:01.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:01.856+0000 7fe689600640 1 -- 172.21.15.16:0/1322890470 >> 172.21.15.16:0/1322890470 conn(0x7fe6841015b0 msgr2=0x7fe684103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:01.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:01.856+0000 7fe689600640 1 -- 172.21.15.16:0/1322890470 shutdown_connections 2024-08-31T06:05:01.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:01.856+0000 7fe689600640 1 -- 172.21.15.16:0/1322890470 wait complete. 2024-08-31T06:05:01.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:01.857+0000 7fe689600640 1 Processor -- start 2024-08-31T06:05:01.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:01.858+0000 7fe689600640 1 -- start start 2024-08-31T06:05:01.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:01.858+0000 7fe689600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe684105b20 0x7fe68419c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:01.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:01.858+0000 7fe689600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe68410b460 0x7fe68419ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:01.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:01.858+0000 7fe689600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe68410bdb0 0x7fe6841a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:01.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:01.858+0000 7fe689600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe684115ac0 con 0x7fe684105b20 2024-08-31T06:05:01.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:01.858+0000 7fe689600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe684115940 con 0x7fe68410b460 2024-08-31T06:05:01.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:01.858+0000 7fe689600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe684115c40 con 0x7fe68410bdb0 2024-08-31T06:05:01.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:01.858+0000 7fe682a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe684105b20 0x7fe68419c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:01.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:01.858+0000 7fe682a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe684105b20 0x7fe68419c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43340/0 (socket says 172.21.15.16:43340) 2024-08-31T06:05:01.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:01.858+0000 7fe682a00640 1 -- 172.21.15.16:0/3224297219 learned_addr learned my addr 172.21.15.16:0/3224297219 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:05:01.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:01.859+0000 7fe67be00640 1 --2- 172.21.15.16:0/3224297219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe68410b460 0x7fe68419ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:01.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:01.859+0000 7fe683400640 1 --2- 172.21.15.16:0/3224297219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe68410bdb0 0x7fe6841a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:01.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:01.859+0000 7fe682a00640 1 -- 172.21.15.16:0/3224297219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe68410bdb0 msgr2=0x7fe6841a1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:01.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:01.859+0000 7fe682a00640 1 --2- 172.21.15.16:0/3224297219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe68410bdb0 0x7fe6841a1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:01.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:01.859+0000 7fe682a00640 1 -- 172.21.15.16:0/3224297219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe68410b460 msgr2=0x7fe68419ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:01.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:01.859+0000 7fe682a00640 1 --2- 172.21.15.16:0/3224297219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe68410b460 0x7fe68419ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:01.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:01.859+0000 7fe682a00640 1 -- 172.21.15.16:0/3224297219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6841a23e0 con 0x7fe684105b20 2024-08-31T06:05:01.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:01.859+0000 7fe682a00640 1 --2- 172.21.15.16:0/3224297219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe684105b20 0x7fe68419c470 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe674005ab0 tx=0x7fe674003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:01.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:01.859+0000 7fe67b400640 1 -- 172.21.15.16:0/3224297219 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67403f070 con 0x7fe684105b20 2024-08-31T06:05:01.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:01.860+0000 7fe67b400640 1 -- 172.21.15.16:0/3224297219 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe674031b60 con 0x7fe684105b20 2024-08-31T06:05:01.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:01.860+0000 7fe689600640 1 -- 172.21.15.16:0/3224297219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6841a2670 con 0x7fe684105b20 2024-08-31T06:05:01.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:01.860+0000 7fe67b400640 1 -- 172.21.15.16:0/3224297219 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe674035030 con 0x7fe684105b20 2024-08-31T06:05:01.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:01.860+0000 7fe689600640 1 -- 172.21.15.16:0/3224297219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6841a8e30 con 0x7fe684105b20 2024-08-31T06:05:01.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:01.861+0000 7fe689600640 1 -- 172.21.15.16:0/3224297219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe684106da0 con 0x7fe684105b20 2024-08-31T06:05:01.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:01.863+0000 7fe67b400640 1 -- 172.21.15.16:0/3224297219 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7fe674042070 con 0x7fe684105b20 2024-08-31T06:05:01.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:01.864+0000 7fe67b400640 1 --2- 172.21.15.16:0/3224297219 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe6680b4c10 0x7fe6680b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:01.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:01.865+0000 7fe67b400640 1 -- 172.21.15.16:0/3224297219 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7fe6741060b0 con 0x7fe684105b20 2024-08-31T06:05:01.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:01.865+0000 7fe67be00640 1 --2- 172.21.15.16:0/3224297219 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe6680b4c10 0x7fe6680b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:01.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:01.865+0000 7fe67be00640 1 --2- 172.21.15.16:0/3224297219 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe6680b4c10 0x7fe6680b7120 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe684104e00 tx=0x7fe66c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:01.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:01.867+0000 7fe67b400640 1 -- 172.21.15.16:0/3224297219 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fe6740cc9f0 con 0x7fe684105b20 2024-08-31T06:05:02.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:02.061+0000 7fe689600640 1 -- 172.21.15.16:0/3224297219 --> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe6840635a0 con 0x7fe6680b4c10 2024-08-31T06:05:02.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:02.067+0000 7fe67b400640 1 -- 172.21.15.16:0/3224297219 <== mgr.14220 v2:172.21.15.16:6800/4279751337 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258837 (secure 0 0 0) 0x7fe6840635a0 con 0x7fe6680b4c10 2024-08-31T06:05:02.072 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:05:02.074 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-08-31T06:05:02.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:02.072+0000 7fe689600640 1 -- 172.21.15.16:0/3224297219 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe6680b4c10 msgr2=0x7fe6680b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:02.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:02.072+0000 7fe689600640 1 --2- 172.21.15.16:0/3224297219 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe6680b4c10 0x7fe6680b7120 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe684104e00 tx=0x7fe66c005f90 comp rx=0 tx=0).stop 2024-08-31T06:05:02.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:02.072+0000 7fe689600640 1 -- 172.21.15.16:0/3224297219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe684105b20 msgr2=0x7fe68419c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:02.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:02.072+0000 7fe689600640 1 --2- 172.21.15.16:0/3224297219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe684105b20 0x7fe68419c470 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe674005ab0 tx=0x7fe674003c10 comp rx=0 tx=0).stop 2024-08-31T06:05:02.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:02.073+0000 7fe689600640 1 -- 172.21.15.16:0/3224297219 shutdown_connections 2024-08-31T06:05:02.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:02.073+0000 7fe689600640 1 --2- 172.21.15.16:0/3224297219 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe6680b4c10 0x7fe6680b7120 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:02.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:02.073+0000 7fe689600640 1 --2- 172.21.15.16:0/3224297219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe68410bdb0 0x7fe6841a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:02.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:02.073+0000 7fe689600640 1 --2- 172.21.15.16:0/3224297219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe68410b460 0x7fe68419ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:02.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:02.073+0000 7fe689600640 1 --2- 172.21.15.16:0/3224297219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe684105b20 0x7fe68419c470 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:02.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:02.073+0000 7fe689600640 1 -- 172.21.15.16:0/3224297219 >> 172.21.15.16:0/3224297219 conn(0x7fe6841015b0 msgr2=0x7fe68410aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:02.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:02.073+0000 7fe689600640 1 -- 172.21.15.16:0/3224297219 shutdown_connections 2024-08-31T06:05:02.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:02.073+0000 7fe689600640 1 -- 172.21.15.16:0/3224297219 wait complete. 2024-08-31T06:05:02.883 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2024-08-31T06:05:01.263046+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649361,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":227,"num_read_kb":205,"num_write":285,"num_write_kb":2255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359648,"kb_used_data":10400,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124582176,"statfs":{"total":1151940427776,"available":1151572148224,"internally_reserved":0,"allocated":10649600,"data_stored":6963846,"data_compressed":108957,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":53380,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":91,"num_read_kb":100,"num_write":32,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003128"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.897809+0000","last_change":"2024-08-31T06:04:25.877318+0000","last_active":"2024-08-31T06:04:27.897809+0000","last_peered":"2024-08-31T06:04:27.897809+0000","last_clean":"2024-08-31T06:04:27.897809+0000","last_became_active":"2024-08-31T06:04:25.876894+0000","last_became_peered":"2024-08-31T06:04:25.876894+0000","last_unstale":"2024-08-31T06:04:27.897809+0000","last_undegraded":"2024-08-31T06:04:27.897809+0000","last_fullsized":"2024-08-31T06:04:27.897809+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:41:09.449343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.891705+0000","last_change":"2024-08-31T06:04:25.885670+0000","last_active":"2024-08-31T06:04:27.891705+0000","last_peered":"2024-08-31T06:04:27.891705+0000","last_clean":"2024-08-31T06:04:27.891705+0000","last_became_active":"2024-08-31T06:04:25.883818+0000","last_became_peered":"2024-08-31T06:04:25.883818+0000","last_unstale":"2024-08-31T06:04:27.891705+0000","last_undegraded":"2024-08-31T06:04:27.891705+0000","last_fullsized":"2024-08-31T06:04:27.891705+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:40:01.344432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.891292+0000","last_change":"2024-08-31T06:04:25.883375+0000","last_active":"2024-08-31T06:04:27.891292+0000","last_peered":"2024-08-31T06:04:27.891292+0000","last_clean":"2024-08-31T06:04:27.891292+0000","last_became_active":"2024-08-31T06:04:25.881750+0000","last_became_peered":"2024-08-31T06:04:25.881750+0000","last_unstale":"2024-08-31T06:04:27.891292+0000","last_undegraded":"2024-08-31T06:04:27.891292+0000","last_fullsized":"2024-08-31T06:04:27.891292+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:46:46.757363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.893661+0000","last_change":"2024-08-31T06:04:25.882436+0000","last_active":"2024-08-31T06:04:27.893661+0000","last_peered":"2024-08-31T06:04:27.893661+0000","last_clean":"2024-08-31T06:04:27.893661+0000","last_became_active":"2024-08-31T06:04:25.881805+0000","last_became_peered":"2024-08-31T06:04:25.881805+0000","last_unstale":"2024-08-31T06:04:27.893661+0000","last_undegraded":"2024-08-31T06:04:27.893661+0000","last_fullsized":"2024-08-31T06:04:27.893661+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:42:39.793625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.887346+0000","last_change":"2024-08-31T06:04:25.894534+0000","last_active":"2024-08-31T06:04:27.887346+0000","last_peered":"2024-08-31T06:04:27.887346+0000","last_clean":"2024-08-31T06:04:27.887346+0000","last_became_active":"2024-08-31T06:04:25.894116+0000","last_became_peered":"2024-08-31T06:04:25.894116+0000","last_unstale":"2024-08-31T06:04:27.887346+0000","last_undegraded":"2024-08-31T06:04:27.887346+0000","last_fullsized":"2024-08-31T06:04:27.887346+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:02:31.257454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.872608+0000","last_change":"2024-08-31T06:04:25.886182+0000","last_active":"2024-08-31T06:04:27.872608+0000","last_peered":"2024-08-31T06:04:27.872608+0000","last_clean":"2024-08-31T06:04:27.872608+0000","last_became_active":"2024-08-31T06:04:25.885825+0000","last_became_peered":"2024-08-31T06:04:25.885825+0000","last_unstale":"2024-08-31T06:04:27.872608+0000","last_undegraded":"2024-08-31T06:04:27.872608+0000","last_fullsized":"2024-08-31T06:04:27.872608+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:58:12.427931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.889404+0000","last_change":"2024-08-31T06:04:25.877521+0000","last_active":"2024-08-31T06:04:27.889404+0000","last_peered":"2024-08-31T06:04:27.889404+0000","last_clean":"2024-08-31T06:04:27.889404+0000","last_became_active":"2024-08-31T06:04:25.877122+0000","last_became_peered":"2024-08-31T06:04:25.877122+0000","last_unstale":"2024-08-31T06:04:27.889404+0000","last_undegraded":"2024-08-31T06:04:27.889404+0000","last_fullsized":"2024-08-31T06:04:27.889404+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:17:27.406828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.893783+0000","last_change":"2024-08-31T06:04:25.892547+0000","last_active":"2024-08-31T06:04:27.893783+0000","last_peered":"2024-08-31T06:04:27.893783+0000","last_clean":"2024-08-31T06:04:27.893783+0000","last_became_active":"2024-08-31T06:04:25.892090+0000","last_became_peered":"2024-08-31T06:04:25.892090+0000","last_unstale":"2024-08-31T06:04:27.893783+0000","last_undegraded":"2024-08-31T06:04:27.893783+0000","last_fullsized":"2024-08-31T06:04:27.893783+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:28:53.414252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.888975+0000","last_change":"2024-08-31T06:04:25.879952+0000","last_active":"2024-08-31T06:04:27.888975+0000","last_peered":"2024-08-31T06:04:27.888975+0000","last_clean":"2024-08-31T06:04:27.888975+0000","last_became_active":"2024-08-31T06:04:25.879568+0000","last_became_peered":"2024-08-31T06:04:25.879568+0000","last_unstale":"2024-08-31T06:04:27.888975+0000","last_undegraded":"2024-08-31T06:04:27.888975+0000","last_fullsized":"2024-08-31T06:04:27.888975+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:28:04.558054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.873181+0000","last_change":"2024-08-31T06:04:25.884515+0000","last_active":"2024-08-31T06:04:27.873181+0000","last_peered":"2024-08-31T06:04:27.873181+0000","last_clean":"2024-08-31T06:04:27.873181+0000","last_became_active":"2024-08-31T06:04:25.884016+0000","last_became_peered":"2024-08-31T06:04:25.884016+0000","last_unstale":"2024-08-31T06:04:27.873181+0000","last_undegraded":"2024-08-31T06:04:27.873181+0000","last_fullsized":"2024-08-31T06:04:27.873181+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:37:06.835127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.890800+0000","last_change":"2024-08-31T06:04:25.883984+0000","last_active":"2024-08-31T06:04:27.890800+0000","last_peered":"2024-08-31T06:04:27.890800+0000","last_clean":"2024-08-31T06:04:27.890800+0000","last_became_active":"2024-08-31T06:04:25.883353+0000","last_became_peered":"2024-08-31T06:04:25.883353+0000","last_unstale":"2024-08-31T06:04:27.890800+0000","last_undegraded":"2024-08-31T06:04:27.890800+0000","last_fullsized":"2024-08-31T06:04:27.890800+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:16:25.178952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.893944+0000","last_change":"2024-08-31T06:04:25.883335+0000","last_active":"2024-08-31T06:04:27.893944+0000","last_peered":"2024-08-31T06:04:27.893944+0000","last_clean":"2024-08-31T06:04:27.893944+0000","last_became_active":"2024-08-31T06:04:25.881488+0000","last_became_peered":"2024-08-31T06:04:25.881488+0000","last_unstale":"2024-08-31T06:04:27.893944+0000","last_undegraded":"2024-08-31T06:04:27.893944+0000","last_fullsized":"2024-08-31T06:04:27.893944+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:57:24.472482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.890379+0000","last_change":"2024-08-31T06:04:25.891579+0000","last_active":"2024-08-31T06:04:27.890379+0000","last_peered":"2024-08-31T06:04:27.890379+0000","last_clean":"2024-08-31T06:04:27.890379+0000","last_became_active":"2024-08-31T06:04:25.887950+0000","last_became_peered":"2024-08-31T06:04:25.887950+0000","last_unstale":"2024-08-31T06:04:27.890379+0000","last_undegraded":"2024-08-31T06:04:27.890379+0000","last_fullsized":"2024-08-31T06:04:27.890379+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:38:35.284442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.890715+0000","last_change":"2024-08-31T06:04:25.882185+0000","last_active":"2024-08-31T06:04:27.890715+0000","last_peered":"2024-08-31T06:04:27.890715+0000","last_clean":"2024-08-31T06:04:27.890715+0000","last_became_active":"2024-08-31T06:04:25.876499+0000","last_became_peered":"2024-08-31T06:04:25.876499+0000","last_unstale":"2024-08-31T06:04:27.890715+0000","last_undegraded":"2024-08-31T06:04:27.890715+0000","last_fullsized":"2024-08-31T06:04:27.890715+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:40:01.132353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.890218+0000","last_change":"2024-08-31T06:04:25.883624+0000","last_active":"2024-08-31T06:04:27.890218+0000","last_peered":"2024-08-31T06:04:27.890218+0000","last_clean":"2024-08-31T06:04:27.890218+0000","last_became_active":"2024-08-31T06:04:25.878041+0000","last_became_peered":"2024-08-31T06:04:25.878041+0000","last_unstale":"2024-08-31T06:04:27.890218+0000","last_undegraded":"2024-08-31T06:04:27.890218+0000","last_fullsized":"2024-08-31T06:04:27.890218+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:05:21.278321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.887735+0000","last_change":"2024-08-31T06:04:25.882538+0000","last_active":"2024-08-31T06:04:27.887735+0000","last_peered":"2024-08-31T06:04:27.887735+0000","last_clean":"2024-08-31T06:04:27.887735+0000","last_became_active":"2024-08-31T06:04:25.882201+0000","last_became_peered":"2024-08-31T06:04:25.882201+0000","last_unstale":"2024-08-31T06:04:27.887735+0000","last_undegraded":"2024-08-31T06:04:27.887735+0000","last_fullsized":"2024-08-31T06:04:27.887735+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:47:16.880893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.897587+0000","last_change":"2024-08-31T06:04:25.880376+0000","last_active":"2024-08-31T06:04:27.897587+0000","last_peered":"2024-08-31T06:04:27.897587+0000","last_clean":"2024-08-31T06:04:27.897587+0000","last_became_active":"2024-08-31T06:04:25.877668+0000","last_became_peered":"2024-08-31T06:04:25.877668+0000","last_unstale":"2024-08-31T06:04:27.897587+0000","last_undegraded":"2024-08-31T06:04:27.897587+0000","last_fullsized":"2024-08-31T06:04:27.897587+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:55:32.737406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.872360+0000","last_change":"2024-08-31T06:04:25.875431+0000","last_active":"2024-08-31T06:04:27.872360+0000","last_peered":"2024-08-31T06:04:27.872360+0000","last_clean":"2024-08-31T06:04:27.872360+0000","last_became_active":"2024-08-31T06:04:25.874751+0000","last_became_peered":"2024-08-31T06:04:25.874751+0000","last_unstale":"2024-08-31T06:04:27.872360+0000","last_undegraded":"2024-08-31T06:04:27.872360+0000","last_fullsized":"2024-08-31T06:04:27.872360+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:16:57.695415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.889419+0000","last_change":"2024-08-31T06:04:25.895656+0000","last_active":"2024-08-31T06:04:27.889419+0000","last_peered":"2024-08-31T06:04:27.889419+0000","last_clean":"2024-08-31T06:04:27.889419+0000","last_became_active":"2024-08-31T06:04:25.895272+0000","last_became_peered":"2024-08-31T06:04:25.895272+0000","last_unstale":"2024-08-31T06:04:27.889419+0000","last_undegraded":"2024-08-31T06:04:27.889419+0000","last_fullsized":"2024-08-31T06:04:27.889419+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:18:47.276195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.895664+0000","last_change":"2024-08-31T06:04:25.877283+0000","last_active":"2024-08-31T06:04:27.895664+0000","last_peered":"2024-08-31T06:04:27.895664+0000","last_clean":"2024-08-31T06:04:27.895664+0000","last_became_active":"2024-08-31T06:04:25.876846+0000","last_became_peered":"2024-08-31T06:04:25.876846+0000","last_unstale":"2024-08-31T06:04:27.895664+0000","last_undegraded":"2024-08-31T06:04:27.895664+0000","last_fullsized":"2024-08-31T06:04:27.895664+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:54:05.086844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.890398+0000","last_change":"2024-08-31T06:04:25.884891+0000","last_active":"2024-08-31T06:04:27.890398+0000","last_peered":"2024-08-31T06:04:27.890398+0000","last_clean":"2024-08-31T06:04:27.890398+0000","last_became_active":"2024-08-31T06:04:25.884501+0000","last_became_peered":"2024-08-31T06:04:25.884501+0000","last_unstale":"2024-08-31T06:04:27.890398+0000","last_undegraded":"2024-08-31T06:04:27.890398+0000","last_fullsized":"2024-08-31T06:04:27.890398+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:22:54.846102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.888302+0000","last_change":"2024-08-31T06:04:25.877150+0000","last_active":"2024-08-31T06:04:27.888302+0000","last_peered":"2024-08-31T06:04:27.888302+0000","last_clean":"2024-08-31T06:04:27.888302+0000","last_became_active":"2024-08-31T06:04:25.876766+0000","last_became_peered":"2024-08-31T06:04:25.876766+0000","last_unstale":"2024-08-31T06:04:27.888302+0000","last_undegraded":"2024-08-31T06:04:27.888302+0000","last_fullsized":"2024-08-31T06:04:27.888302+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:46:26.724970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.895942+0000","last_change":"2024-08-31T06:04:25.881275+0000","last_active":"2024-08-31T06:04:27.895942+0000","last_peered":"2024-08-31T06:04:27.895942+0000","last_clean":"2024-08-31T06:04:27.895942+0000","last_became_active":"2024-08-31T06:04:25.879877+0000","last_became_peered":"2024-08-31T06:04:25.879877+0000","last_unstale":"2024-08-31T06:04:27.895942+0000","last_undegraded":"2024-08-31T06:04:27.895942+0000","last_fullsized":"2024-08-31T06:04:27.895942+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:07:07.237041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.872617+0000","last_change":"2024-08-31T06:04:25.895750+0000","last_active":"2024-08-31T06:04:27.872617+0000","last_peered":"2024-08-31T06:04:27.872617+0000","last_clean":"2024-08-31T06:04:27.872617+0000","last_became_active":"2024-08-31T06:04:25.895348+0000","last_became_peered":"2024-08-31T06:04:25.895348+0000","last_unstale":"2024-08-31T06:04:27.872617+0000","last_undegraded":"2024-08-31T06:04:27.872617+0000","last_fullsized":"2024-08-31T06:04:27.872617+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:53:10.014808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.895222+0000","last_change":"2024-08-31T06:04:25.895537+0000","last_active":"2024-08-31T06:04:27.895222+0000","last_peered":"2024-08-31T06:04:27.895222+0000","last_clean":"2024-08-31T06:04:27.895222+0000","last_became_active":"2024-08-31T06:04:25.895136+0000","last_became_peered":"2024-08-31T06:04:25.895136+0000","last_unstale":"2024-08-31T06:04:27.895222+0000","last_undegraded":"2024-08-31T06:04:27.895222+0000","last_fullsized":"2024-08-31T06:04:27.895222+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:41:57.554507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.892624+0000","last_change":"2024-08-31T06:04:25.880490+0000","last_active":"2024-08-31T06:04:27.892624+0000","last_peered":"2024-08-31T06:04:27.892624+0000","last_clean":"2024-08-31T06:04:27.892624+0000","last_became_active":"2024-08-31T06:04:25.879269+0000","last_became_peered":"2024-08-31T06:04:25.879269+0000","last_unstale":"2024-08-31T06:04:27.892624+0000","last_undegraded":"2024-08-31T06:04:27.892624+0000","last_fullsized":"2024-08-31T06:04:27.892624+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:05:49.575915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.896665+0000","last_change":"2024-08-31T06:04:25.881676+0000","last_active":"2024-08-31T06:04:27.896665+0000","last_peered":"2024-08-31T06:04:27.896665+0000","last_clean":"2024-08-31T06:04:27.896665+0000","last_became_active":"2024-08-31T06:04:25.878222+0000","last_became_peered":"2024-08-31T06:04:25.878222+0000","last_unstale":"2024-08-31T06:04:27.896665+0000","last_undegraded":"2024-08-31T06:04:27.896665+0000","last_fullsized":"2024-08-31T06:04:27.896665+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:24:54.757553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.889542+0000","last_change":"2024-08-31T06:04:25.900183+0000","last_active":"2024-08-31T06:04:27.889542+0000","last_peered":"2024-08-31T06:04:27.889542+0000","last_clean":"2024-08-31T06:04:27.889542+0000","last_became_active":"2024-08-31T06:04:25.898903+0000","last_became_peered":"2024-08-31T06:04:25.898903+0000","last_unstale":"2024-08-31T06:04:27.889542+0000","last_undegraded":"2024-08-31T06:04:27.889542+0000","last_fullsized":"2024-08-31T06:04:27.889542+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:34:05.796458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.892123+0000","last_change":"2024-08-31T06:04:25.897473+0000","last_active":"2024-08-31T06:04:27.892123+0000","last_peered":"2024-08-31T06:04:27.892123+0000","last_clean":"2024-08-31T06:04:27.892123+0000","last_became_active":"2024-08-31T06:04:25.895194+0000","last_became_peered":"2024-08-31T06:04:25.895194+0000","last_unstale":"2024-08-31T06:04:27.892123+0000","last_undegraded":"2024-08-31T06:04:27.892123+0000","last_fullsized":"2024-08-31T06:04:27.892123+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:43:04.539970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.890583+0000","last_change":"2024-08-31T06:04:25.894402+0000","last_active":"2024-08-31T06:04:27.890583+0000","last_peered":"2024-08-31T06:04:27.890583+0000","last_clean":"2024-08-31T06:04:27.890583+0000","last_became_active":"2024-08-31T06:04:25.893883+0000","last_became_peered":"2024-08-31T06:04:25.893883+0000","last_unstale":"2024-08-31T06:04:27.890583+0000","last_undegraded":"2024-08-31T06:04:27.890583+0000","last_fullsized":"2024-08-31T06:04:27.890583+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:45:53.778473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.891364+0000","last_change":"2024-08-31T06:04:25.881543+0000","last_active":"2024-08-31T06:04:27.891364+0000","last_peered":"2024-08-31T06:04:27.891364+0000","last_clean":"2024-08-31T06:04:27.891364+0000","last_became_active":"2024-08-31T06:04:25.881144+0000","last_became_peered":"2024-08-31T06:04:25.881144+0000","last_unstale":"2024-08-31T06:04:27.891364+0000","last_undegraded":"2024-08-31T06:04:27.891364+0000","last_fullsized":"2024-08-31T06:04:27.891364+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:13:02.282879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.896143+0000","last_change":"2024-08-31T06:04:25.881143+0000","last_active":"2024-08-31T06:04:27.896143+0000","last_peered":"2024-08-31T06:04:27.896143+0000","last_clean":"2024-08-31T06:04:27.896143+0000","last_became_active":"2024-08-31T06:04:25.879718+0000","last_became_peered":"2024-08-31T06:04:25.879718+0000","last_unstale":"2024-08-31T06:04:27.896143+0000","last_undegraded":"2024-08-31T06:04:27.896143+0000","last_fullsized":"2024-08-31T06:04:27.896143+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:32:07.771488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.887567+0000","last_change":"2024-08-31T06:04:25.879493+0000","last_active":"2024-08-31T06:04:27.887567+0000","last_peered":"2024-08-31T06:04:27.887567+0000","last_clean":"2024-08-31T06:04:27.887567+0000","last_became_active":"2024-08-31T06:04:25.879065+0000","last_became_peered":"2024-08-31T06:04:25.879065+0000","last_unstale":"2024-08-31T06:04:27.887567+0000","last_undegraded":"2024-08-31T06:04:27.887567+0000","last_fullsized":"2024-08-31T06:04:27.887567+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:53:43.848269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"78'6","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:57.532123+0000","last_change":"2024-08-31T06:04:22.844727+0000","last_active":"2024-08-31T06:04:57.532123+0000","last_peered":"2024-08-31T06:04:57.532123+0000","last_clean":"2024-08-31T06:04:57.532123+0000","last_became_active":"2024-08-31T06:04:22.844269+0000","last_became_peered":"2024-08-31T06:04:22.844269+0000","last_unstale":"2024-08-31T06:04:57.532123+0000","last_undegraded":"2024-08-31T06:04:57.532123+0000","last_fullsized":"2024-08-31T06:04:57.532123+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:19:03.010029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.872785+0000","last_change":"2024-08-31T06:04:25.896354+0000","last_active":"2024-08-31T06:04:27.872785+0000","last_peered":"2024-08-31T06:04:27.872785+0000","last_clean":"2024-08-31T06:04:27.872785+0000","last_became_active":"2024-08-31T06:04:25.896021+0000","last_became_peered":"2024-08-31T06:04:25.896021+0000","last_unstale":"2024-08-31T06:04:27.872785+0000","last_undegraded":"2024-08-31T06:04:27.872785+0000","last_fullsized":"2024-08-31T06:04:27.872785+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:08:40.213238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:38.940702+0000","last_change":"2024-08-31T06:04:22.844248+0000","last_active":"2024-08-31T06:04:38.940702+0000","last_peered":"2024-08-31T06:04:38.940702+0000","last_clean":"2024-08-31T06:04:38.940702+0000","last_became_active":"2024-08-31T06:04:22.843784+0000","last_became_peered":"2024-08-31T06:04:22.843784+0000","last_unstale":"2024-08-31T06:04:38.940702+0000","last_undegraded":"2024-08-31T06:04:38.940702+0000","last_fullsized":"2024-08-31T06:04:38.940702+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:56:41.422965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:38.942677+0000","last_change":"2024-08-31T06:04:22.847049+0000","last_active":"2024-08-31T06:04:38.942677+0000","last_peered":"2024-08-31T06:04:38.942677+0000","last_clean":"2024-08-31T06:04:38.942677+0000","last_became_active":"2024-08-31T06:04:22.846282+0000","last_became_peered":"2024-08-31T06:04:22.846282+0000","last_unstale":"2024-08-31T06:04:38.942677+0000","last_undegraded":"2024-08-31T06:04:38.942677+0000","last_fullsized":"2024-08-31T06:04:38.942677+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:35:21.030425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.893479+0000","last_change":"2024-08-31T06:04:25.886698+0000","last_active":"2024-08-31T06:04:27.893479+0000","last_peered":"2024-08-31T06:04:27.893479+0000","last_clean":"2024-08-31T06:04:27.893479+0000","last_became_active":"2024-08-31T06:04:25.886334+0000","last_became_peered":"2024-08-31T06:04:25.886334+0000","last_unstale":"2024-08-31T06:04:27.893479+0000","last_undegraded":"2024-08-31T06:04:27.893479+0000","last_fullsized":"2024-08-31T06:04:27.893479+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:24:43.821446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.873058+0000","last_change":"2024-08-31T06:04:22.845263+0000","last_active":"2024-08-31T06:04:27.873058+0000","last_peered":"2024-08-31T06:04:27.873058+0000","last_clean":"2024-08-31T06:04:27.873058+0000","last_became_active":"2024-08-31T06:04:22.844853+0000","last_became_peered":"2024-08-31T06:04:22.844853+0000","last_unstale":"2024-08-31T06:04:27.873058+0000","last_undegraded":"2024-08-31T06:04:27.873058+0000","last_fullsized":"2024-08-31T06:04:27.873058+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:55:12.561165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.872038+0000","last_change":"2024-08-31T06:04:25.894475+0000","last_active":"2024-08-31T06:04:27.872038+0000","last_peered":"2024-08-31T06:04:27.872038+0000","last_clean":"2024-08-31T06:04:27.872038+0000","last_became_active":"2024-08-31T06:04:25.894121+0000","last_became_peered":"2024-08-31T06:04:25.894121+0000","last_unstale":"2024-08-31T06:04:27.872038+0000","last_undegraded":"2024-08-31T06:04:27.872038+0000","last_fullsized":"2024-08-31T06:04:27.872038+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:40:06.570616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'3","reported_seq":45,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:58.533140+0000","last_change":"2024-08-31T06:04:22.834430+0000","last_active":"2024-08-31T06:04:58.533140+0000","last_peered":"2024-08-31T06:04:58.533140+0000","last_clean":"2024-08-31T06:04:58.533140+0000","last_became_active":"2024-08-31T06:04:22.834011+0000","last_became_peered":"2024-08-31T06:04:22.834011+0000","last_unstale":"2024-08-31T06:04:58.533140+0000","last_undegraded":"2024-08-31T06:04:58.533140+0000","last_fullsized":"2024-08-31T06:04:58.533140+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:51:16.734234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.888195+0000","last_change":"2024-08-31T06:04:25.877727+0000","last_active":"2024-08-31T06:04:27.888195+0000","last_peered":"2024-08-31T06:04:27.888195+0000","last_clean":"2024-08-31T06:04:27.888195+0000","last_became_active":"2024-08-31T06:04:25.877121+0000","last_became_peered":"2024-08-31T06:04:25.877121+0000","last_unstale":"2024-08-31T06:04:27.888195+0000","last_undegraded":"2024-08-31T06:04:27.888195+0000","last_fullsized":"2024-08-31T06:04:27.888195+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:41:12.247216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:37.935281+0000","last_change":"2024-08-31T06:04:22.846482+0000","last_active":"2024-08-31T06:04:37.935281+0000","last_peered":"2024-08-31T06:04:37.935281+0000","last_clean":"2024-08-31T06:04:37.935281+0000","last_became_active":"2024-08-31T06:04:22.846057+0000","last_became_peered":"2024-08-31T06:04:22.846057+0000","last_unstale":"2024-08-31T06:04:37.935281+0000","last_undegraded":"2024-08-31T06:04:37.935281+0000","last_fullsized":"2024-08-31T06:04:37.935281+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:29:11.861665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.891600+0000","last_change":"2024-08-31T06:04:25.893777+0000","last_active":"2024-08-31T06:04:27.891600+0000","last_peered":"2024-08-31T06:04:27.891600+0000","last_clean":"2024-08-31T06:04:27.891600+0000","last_became_active":"2024-08-31T06:04:25.893014+0000","last_became_peered":"2024-08-31T06:04:25.893014+0000","last_unstale":"2024-08-31T06:04:27.891600+0000","last_undegraded":"2024-08-31T06:04:27.891600+0000","last_fullsized":"2024-08-31T06:04:27.891600+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:04:48.185842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:37.939192+0000","last_change":"2024-08-31T06:04:22.965938+0000","last_active":"2024-08-31T06:04:37.939192+0000","last_peered":"2024-08-31T06:04:37.939192+0000","last_clean":"2024-08-31T06:04:37.939192+0000","last_became_active":"2024-08-31T06:04:22.965356+0000","last_became_peered":"2024-08-31T06:04:22.965356+0000","last_unstale":"2024-08-31T06:04:37.939192+0000","last_undegraded":"2024-08-31T06:04:37.939192+0000","last_fullsized":"2024-08-31T06:04:37.939192+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:36:58.938462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.889350+0000","last_change":"2024-08-31T06:04:25.881307+0000","last_active":"2024-08-31T06:04:27.889350+0000","last_peered":"2024-08-31T06:04:27.889350+0000","last_clean":"2024-08-31T06:04:27.889350+0000","last_became_active":"2024-08-31T06:04:25.880418+0000","last_became_peered":"2024-08-31T06:04:25.880418+0000","last_unstale":"2024-08-31T06:04:27.889350+0000","last_undegraded":"2024-08-31T06:04:27.889350+0000","last_fullsized":"2024-08-31T06:04:27.889350+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:21:46.904371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"78'26","reported_seq":96,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:59.535108+0000","last_change":"2024-08-31T06:04:22.843073+0000","last_active":"2024-08-31T06:04:59.535108+0000","last_peered":"2024-08-31T06:04:59.535108+0000","last_clean":"2024-08-31T06:04:59.535108+0000","last_became_active":"2024-08-31T06:04:22.842645+0000","last_became_peered":"2024-08-31T06:04:22.842645+0000","last_unstale":"2024-08-31T06:04:59.535108+0000","last_undegraded":"2024-08-31T06:04:59.535108+0000","last_fullsized":"2024-08-31T06:04:59.535108+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:41:37.651562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54367,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":48,"num_read_kb":76,"num_write":27,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.887999+0000","last_change":"2024-08-31T06:04:25.879929+0000","last_active":"2024-08-31T06:04:27.887999+0000","last_peered":"2024-08-31T06:04:27.887999+0000","last_clean":"2024-08-31T06:04:27.887999+0000","last_became_active":"2024-08-31T06:04:25.878475+0000","last_became_peered":"2024-08-31T06:04:25.878475+0000","last_unstale":"2024-08-31T06:04:27.887999+0000","last_undegraded":"2024-08-31T06:04:27.887999+0000","last_fullsized":"2024-08-31T06:04:27.887999+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:10:32.801558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":30,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:38.942569+0000","last_change":"2024-08-31T06:04:22.842878+0000","last_active":"2024-08-31T06:04:38.942569+0000","last_peered":"2024-08-31T06:04:38.942569+0000","last_clean":"2024-08-31T06:04:38.942569+0000","last_became_active":"2024-08-31T06:04:22.842306+0000","last_became_peered":"2024-08-31T06:04:22.842306+0000","last_unstale":"2024-08-31T06:04:38.942569+0000","last_undegraded":"2024-08-31T06:04:38.942569+0000","last_fullsized":"2024-08-31T06:04:38.942569+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:53:55.259238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.888749+0000","last_change":"2024-08-31T06:04:25.899357+0000","last_active":"2024-08-31T06:04:27.888749+0000","last_peered":"2024-08-31T06:04:27.888749+0000","last_clean":"2024-08-31T06:04:27.888749+0000","last_became_active":"2024-08-31T06:04:25.899000+0000","last_became_peered":"2024-08-31T06:04:25.899000+0000","last_unstale":"2024-08-31T06:04:27.888749+0000","last_undegraded":"2024-08-31T06:04:27.888749+0000","last_fullsized":"2024-08-31T06:04:27.888749+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:20:39.991071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:37.938984+0000","last_change":"2024-08-31T06:04:22.843063+0000","last_active":"2024-08-31T06:04:37.938984+0000","last_peered":"2024-08-31T06:04:37.938984+0000","last_clean":"2024-08-31T06:04:37.938984+0000","last_became_active":"2024-08-31T06:04:22.842654+0000","last_became_peered":"2024-08-31T06:04:22.842654+0000","last_unstale":"2024-08-31T06:04:37.938984+0000","last_undegraded":"2024-08-31T06:04:37.938984+0000","last_fullsized":"2024-08-31T06:04:37.938984+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:09:54.563116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.890628+0000","last_change":"2024-08-31T06:04:25.885556+0000","last_active":"2024-08-31T06:04:27.890628+0000","last_peered":"2024-08-31T06:04:27.890628+0000","last_clean":"2024-08-31T06:04:27.890628+0000","last_became_active":"2024-08-31T06:04:25.885216+0000","last_became_peered":"2024-08-31T06:04:25.885216+0000","last_unstale":"2024-08-31T06:04:27.890628+0000","last_undegraded":"2024-08-31T06:04:27.890628+0000","last_fullsized":"2024-08-31T06:04:27.890628+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:48:49.816261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'4","reported_seq":48,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:57.532529+0000","last_change":"2024-08-31T06:04:22.844839+0000","last_active":"2024-08-31T06:04:57.532529+0000","last_peered":"2024-08-31T06:04:57.532529+0000","last_clean":"2024-08-31T06:04:57.532529+0000","last_became_active":"2024-08-31T06:04:22.844425+0000","last_became_peered":"2024-08-31T06:04:22.844425+0000","last_unstale":"2024-08-31T06:04:57.532529+0000","last_undegraded":"2024-08-31T06:04:57.532529+0000","last_fullsized":"2024-08-31T06:04:57.532529+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:10:57.292173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.890010+0000","last_change":"2024-08-31T06:04:25.878195+0000","last_active":"2024-08-31T06:04:27.890010+0000","last_peered":"2024-08-31T06:04:27.890010+0000","last_clean":"2024-08-31T06:04:27.890010+0000","last_became_active":"2024-08-31T06:04:25.877823+0000","last_became_peered":"2024-08-31T06:04:25.877823+0000","last_unstale":"2024-08-31T06:04:27.890010+0000","last_undegraded":"2024-08-31T06:04:27.890010+0000","last_fullsized":"2024-08-31T06:04:27.890010+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:54:34.989583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:38.944046+0000","last_change":"2024-08-31T06:04:22.965488+0000","last_active":"2024-08-31T06:04:38.944046+0000","last_peered":"2024-08-31T06:04:38.944046+0000","last_clean":"2024-08-31T06:04:38.944046+0000","last_became_active":"2024-08-31T06:04:22.965066+0000","last_became_peered":"2024-08-31T06:04:22.965066+0000","last_unstale":"2024-08-31T06:04:38.944046+0000","last_undegraded":"2024-08-31T06:04:38.944046+0000","last_fullsized":"2024-08-31T06:04:38.944046+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:26:48.079318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.872606+0000","last_change":"2024-08-31T06:04:25.887580+0000","last_active":"2024-08-31T06:04:27.872606+0000","last_peered":"2024-08-31T06:04:27.872606+0000","last_clean":"2024-08-31T06:04:27.872606+0000","last_became_active":"2024-08-31T06:04:25.887158+0000","last_became_peered":"2024-08-31T06:04:25.887158+0000","last_unstale":"2024-08-31T06:04:27.872606+0000","last_undegraded":"2024-08-31T06:04:27.872606+0000","last_fullsized":"2024-08-31T06:04:27.872606+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:27:38.115742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:40.500346+0000","last_change":"2024-08-31T06:04:22.843806+0000","last_active":"2024-08-31T06:04:40.500346+0000","last_peered":"2024-08-31T06:04:40.500346+0000","last_clean":"2024-08-31T06:04:40.500346+0000","last_became_active":"2024-08-31T06:04:22.843218+0000","last_became_peered":"2024-08-31T06:04:22.843218+0000","last_unstale":"2024-08-31T06:04:40.500346+0000","last_undegraded":"2024-08-31T06:04:40.500346+0000","last_fullsized":"2024-08-31T06:04:40.500346+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:56:12.019875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.896175+0000","last_change":"2024-08-31T06:04:25.880845+0000","last_active":"2024-08-31T06:04:27.896175+0000","last_peered":"2024-08-31T06:04:27.896175+0000","last_clean":"2024-08-31T06:04:27.896175+0000","last_became_active":"2024-08-31T06:04:25.880423+0000","last_became_peered":"2024-08-31T06:04:25.880423+0000","last_unstale":"2024-08-31T06:04:27.896175+0000","last_undegraded":"2024-08-31T06:04:27.896175+0000","last_fullsized":"2024-08-31T06:04:27.896175+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:16:56.725116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.898554+0000","last_change":"2024-08-31T06:04:22.843393+0000","last_active":"2024-08-31T06:04:27.898554+0000","last_peered":"2024-08-31T06:04:27.898554+0000","last_clean":"2024-08-31T06:04:27.898554+0000","last_became_active":"2024-08-31T06:04:22.842770+0000","last_became_peered":"2024-08-31T06:04:22.842770+0000","last_unstale":"2024-08-31T06:04:27.898554+0000","last_undegraded":"2024-08-31T06:04:27.898554+0000","last_fullsized":"2024-08-31T06:04:27.898554+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:49:16.159167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.891050+0000","last_change":"2024-08-31T06:04:25.887829+0000","last_active":"2024-08-31T06:04:27.891050+0000","last_peered":"2024-08-31T06:04:27.891050+0000","last_clean":"2024-08-31T06:04:27.891050+0000","last_became_active":"2024-08-31T06:04:25.886567+0000","last_became_peered":"2024-08-31T06:04:25.886567+0000","last_unstale":"2024-08-31T06:04:27.891050+0000","last_undegraded":"2024-08-31T06:04:27.891050+0000","last_fullsized":"2024-08-31T06:04:27.891050+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:11:39.054966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.895607+0000","last_change":"2024-08-31T06:04:22.844218+0000","last_active":"2024-08-31T06:04:27.895607+0000","last_peered":"2024-08-31T06:04:27.895607+0000","last_clean":"2024-08-31T06:04:27.895607+0000","last_became_active":"2024-08-31T06:04:22.843786+0000","last_became_peered":"2024-08-31T06:04:22.843786+0000","last_unstale":"2024-08-31T06:04:27.895607+0000","last_undegraded":"2024-08-31T06:04:27.895607+0000","last_fullsized":"2024-08-31T06:04:27.895607+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:46:02.534190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'1","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.890593+0000","last_change":"2024-08-31T06:04:25.888946+0000","last_active":"2024-08-31T06:04:27.890593+0000","last_peered":"2024-08-31T06:04:27.890593+0000","last_clean":"2024-08-31T06:04:27.890593+0000","last_became_active":"2024-08-31T06:04:25.888564+0000","last_became_peered":"2024-08-31T06:04:25.888564+0000","last_unstale":"2024-08-31T06:04:27.890593+0000","last_undegraded":"2024-08-31T06:04:27.890593+0000","last_fullsized":"2024-08-31T06:04:27.890593+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:26:19.051971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:38.944078+0000","last_change":"2024-08-31T06:04:22.964483+0000","last_active":"2024-08-31T06:04:38.944078+0000","last_peered":"2024-08-31T06:04:38.944078+0000","last_clean":"2024-08-31T06:04:38.944078+0000","last_became_active":"2024-08-31T06:04:22.963874+0000","last_became_peered":"2024-08-31T06:04:22.963874+0000","last_unstale":"2024-08-31T06:04:38.944078+0000","last_undegraded":"2024-08-31T06:04:38.944078+0000","last_fullsized":"2024-08-31T06:04:38.944078+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:30:00.159040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.890863+0000","last_change":"2024-08-31T06:04:25.880941+0000","last_active":"2024-08-31T06:04:27.890863+0000","last_peered":"2024-08-31T06:04:27.890863+0000","last_clean":"2024-08-31T06:04:27.890863+0000","last_became_active":"2024-08-31T06:04:25.880454+0000","last_became_peered":"2024-08-31T06:04:25.880454+0000","last_unstale":"2024-08-31T06:04:27.890863+0000","last_undegraded":"2024-08-31T06:04:27.890863+0000","last_fullsized":"2024-08-31T06:04:27.890863+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:49:57.023988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":93,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.890458+0000","last_change":"2024-08-31T06:03:05.921629+0000","last_active":"2024-08-31T06:04:27.890458+0000","last_peered":"2024-08-31T06:04:27.890458+0000","last_clean":"2024-08-31T06:04:27.890458+0000","last_became_active":"2024-08-31T06:03:05.915844+0000","last_became_peered":"2024-08-31T06:03:05.915844+0000","last_unstale":"2024-08-31T06:04:27.890458+0000","last_undegraded":"2024-08-31T06:04:27.890458+0000","last_fullsized":"2024-08-31T06:04:27.890458+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T05:59:24.472429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T05:59:24.472429+0000","last_clean_scrub_stamp":"2024-08-31T05:59:24.472429+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:56:41.073422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:40.501184+0000","last_change":"2024-08-31T06:04:22.846325+0000","last_active":"2024-08-31T06:04:40.501184+0000","last_peered":"2024-08-31T06:04:40.501184+0000","last_clean":"2024-08-31T06:04:40.501184+0000","last_became_active":"2024-08-31T06:04:22.845772+0000","last_became_peered":"2024-08-31T06:04:22.845772+0000","last_unstale":"2024-08-31T06:04:40.501184+0000","last_undegraded":"2024-08-31T06:04:40.501184+0000","last_fullsized":"2024-08-31T06:04:40.501184+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:54:10.336786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.896714+0000","last_change":"2024-08-31T06:04:25.891437+0000","last_active":"2024-08-31T06:04:27.896714+0000","last_peered":"2024-08-31T06:04:27.896714+0000","last_clean":"2024-08-31T06:04:27.896714+0000","last_became_active":"2024-08-31T06:04:25.891053+0000","last_became_peered":"2024-08-31T06:04:25.891053+0000","last_unstale":"2024-08-31T06:04:27.896714+0000","last_undegraded":"2024-08-31T06:04:27.896714+0000","last_fullsized":"2024-08-31T06:04:27.896714+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:22:37.119483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:38.940337+0000","last_change":"2024-08-31T06:04:22.842885+0000","last_active":"2024-08-31T06:04:38.940337+0000","last_peered":"2024-08-31T06:04:38.940337+0000","last_clean":"2024-08-31T06:04:38.940337+0000","last_became_active":"2024-08-31T06:04:22.842473+0000","last_became_peered":"2024-08-31T06:04:22.842473+0000","last_unstale":"2024-08-31T06:04:38.940337+0000","last_undegraded":"2024-08-31T06:04:38.940337+0000","last_fullsized":"2024-08-31T06:04:38.940337+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:33:36.516659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.887889+0000","last_change":"2024-08-31T06:04:25.889213+0000","last_active":"2024-08-31T06:04:27.887889+0000","last_peered":"2024-08-31T06:04:27.887889+0000","last_clean":"2024-08-31T06:04:27.887889+0000","last_became_active":"2024-08-31T06:04:25.888860+0000","last_became_peered":"2024-08-31T06:04:25.888860+0000","last_unstale":"2024-08-31T06:04:27.887889+0000","last_undegraded":"2024-08-31T06:04:27.887889+0000","last_fullsized":"2024-08-31T06:04:27.887889+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:44:18.831229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"78'4","reported_seq":46,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:58.533379+0000","last_change":"2024-08-31T06:04:22.966091+0000","last_active":"2024-08-31T06:04:58.533379+0000","last_peered":"2024-08-31T06:04:58.533379+0000","last_clean":"2024-08-31T06:04:58.533379+0000","last_became_active":"2024-08-31T06:04:22.965727+0000","last_became_peered":"2024-08-31T06:04:22.965727+0000","last_unstale":"2024-08-31T06:04:58.533379+0000","last_undegraded":"2024-08-31T06:04:58.533379+0000","last_fullsized":"2024-08-31T06:04:58.533379+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:42:43.643623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.872545+0000","last_change":"2024-08-31T06:04:25.880039+0000","last_active":"2024-08-31T06:04:27.872545+0000","last_peered":"2024-08-31T06:04:27.872545+0000","last_clean":"2024-08-31T06:04:27.872545+0000","last_became_active":"2024-08-31T06:04:25.879669+0000","last_became_peered":"2024-08-31T06:04:25.879669+0000","last_unstale":"2024-08-31T06:04:27.872545+0000","last_undegraded":"2024-08-31T06:04:27.872545+0000","last_fullsized":"2024-08-31T06:04:27.872545+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:02:09.244281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.890686+0000","last_change":"2024-08-31T06:04:22.844572+0000","last_active":"2024-08-31T06:04:27.890686+0000","last_peered":"2024-08-31T06:04:27.890686+0000","last_clean":"2024-08-31T06:04:27.890686+0000","last_became_active":"2024-08-31T06:04:22.844227+0000","last_became_peered":"2024-08-31T06:04:22.844227+0000","last_unstale":"2024-08-31T06:04:27.890686+0000","last_undegraded":"2024-08-31T06:04:27.890686+0000","last_fullsized":"2024-08-31T06:04:27.890686+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:04:51.966901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.872666+0000","last_change":"2024-08-31T06:04:25.884442+0000","last_active":"2024-08-31T06:04:27.872666+0000","last_peered":"2024-08-31T06:04:27.872666+0000","last_clean":"2024-08-31T06:04:27.872666+0000","last_became_active":"2024-08-31T06:04:25.884016+0000","last_became_peered":"2024-08-31T06:04:25.884016+0000","last_unstale":"2024-08-31T06:04:27.872666+0000","last_undegraded":"2024-08-31T06:04:27.872666+0000","last_fullsized":"2024-08-31T06:04:27.872666+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:36:10.640262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.872083+0000","last_change":"2024-08-31T06:04:22.967845+0000","last_active":"2024-08-31T06:04:27.872083+0000","last_peered":"2024-08-31T06:04:27.872083+0000","last_clean":"2024-08-31T06:04:27.872083+0000","last_became_active":"2024-08-31T06:04:22.967421+0000","last_became_peered":"2024-08-31T06:04:22.967421+0000","last_unstale":"2024-08-31T06:04:27.872083+0000","last_undegraded":"2024-08-31T06:04:27.872083+0000","last_fullsized":"2024-08-31T06:04:27.872083+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:17:46.942551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'1","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:29.790797+0000","last_change":"2024-08-31T06:04:25.897087+0000","last_active":"2024-08-31T06:04:29.790797+0000","last_peered":"2024-08-31T06:04:29.790797+0000","last_clean":"2024-08-31T06:04:29.790797+0000","last_became_active":"2024-08-31T06:04:25.896725+0000","last_became_peered":"2024-08-31T06:04:25.896725+0000","last_unstale":"2024-08-31T06:04:29.790797+0000","last_undegraded":"2024-08-31T06:04:29.790797+0000","last_fullsized":"2024-08-31T06:04:29.790797+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:31:28.279489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:37.938086+0000","last_change":"2024-08-31T06:04:22.965299+0000","last_active":"2024-08-31T06:04:37.938086+0000","last_peered":"2024-08-31T06:04:37.938086+0000","last_clean":"2024-08-31T06:04:37.938086+0000","last_became_active":"2024-08-31T06:04:22.964849+0000","last_became_peered":"2024-08-31T06:04:22.964849+0000","last_unstale":"2024-08-31T06:04:37.938086+0000","last_undegraded":"2024-08-31T06:04:37.938086+0000","last_fullsized":"2024-08-31T06:04:37.938086+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:36:56.431678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.889796+0000","last_change":"2024-08-31T06:04:25.885115+0000","last_active":"2024-08-31T06:04:27.889796+0000","last_peered":"2024-08-31T06:04:27.889796+0000","last_clean":"2024-08-31T06:04:27.889796+0000","last_became_active":"2024-08-31T06:04:25.884722+0000","last_became_peered":"2024-08-31T06:04:25.884722+0000","last_unstale":"2024-08-31T06:04:27.889796+0000","last_undegraded":"2024-08-31T06:04:27.889796+0000","last_fullsized":"2024-08-31T06:04:27.889796+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:20:43.913785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'10","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:55.888270+0000","last_change":"2024-08-31T06:04:22.845119+0000","last_active":"2024-08-31T06:04:55.888270+0000","last_peered":"2024-08-31T06:04:55.888270+0000","last_clean":"2024-08-31T06:04:55.888270+0000","last_became_active":"2024-08-31T06:04:22.843814+0000","last_became_peered":"2024-08-31T06:04:22.843814+0000","last_unstale":"2024-08-31T06:04:55.888270+0000","last_undegraded":"2024-08-31T06:04:55.888270+0000","last_fullsized":"2024-08-31T06:04:55.888270+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:06:40.684887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.895548+0000","last_change":"2024-08-31T06:04:25.892838+0000","last_active":"2024-08-31T06:04:27.895548+0000","last_peered":"2024-08-31T06:04:27.895548+0000","last_clean":"2024-08-31T06:04:27.895548+0000","last_became_active":"2024-08-31T06:04:25.891955+0000","last_became_peered":"2024-08-31T06:04:25.891955+0000","last_unstale":"2024-08-31T06:04:27.895548+0000","last_undegraded":"2024-08-31T06:04:27.895548+0000","last_fullsized":"2024-08-31T06:04:27.895548+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:41:35.085724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:37.937713+0000","last_change":"2024-08-31T06:04:22.964183+0000","last_active":"2024-08-31T06:04:37.937713+0000","last_peered":"2024-08-31T06:04:37.937713+0000","last_clean":"2024-08-31T06:04:37.937713+0000","last_became_active":"2024-08-31T06:04:22.963688+0000","last_became_peered":"2024-08-31T06:04:22.963688+0000","last_unstale":"2024-08-31T06:04:37.937713+0000","last_undegraded":"2024-08-31T06:04:37.937713+0000","last_fullsized":"2024-08-31T06:04:37.937713+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:04:47.833811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.891792+0000","last_change":"2024-08-31T06:04:25.892266+0000","last_active":"2024-08-31T06:04:27.891792+0000","last_peered":"2024-08-31T06:04:27.891792+0000","last_clean":"2024-08-31T06:04:27.891792+0000","last_became_active":"2024-08-31T06:04:25.889014+0000","last_became_peered":"2024-08-31T06:04:25.889014+0000","last_unstale":"2024-08-31T06:04:27.891792+0000","last_undegraded":"2024-08-31T06:04:27.891792+0000","last_fullsized":"2024-08-31T06:04:27.891792+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:27:47.608470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:37.935935+0000","last_change":"2024-08-31T06:04:22.964677+0000","last_active":"2024-08-31T06:04:37.935935+0000","last_peered":"2024-08-31T06:04:37.935935+0000","last_clean":"2024-08-31T06:04:37.935935+0000","last_became_active":"2024-08-31T06:04:22.964237+0000","last_became_peered":"2024-08-31T06:04:22.964237+0000","last_unstale":"2024-08-31T06:04:37.935935+0000","last_undegraded":"2024-08-31T06:04:37.935935+0000","last_fullsized":"2024-08-31T06:04:37.935935+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:00:54.099592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.887298+0000","last_change":"2024-08-31T06:04:25.881699+0000","last_active":"2024-08-31T06:04:27.887298+0000","last_peered":"2024-08-31T06:04:27.887298+0000","last_clean":"2024-08-31T06:04:27.887298+0000","last_became_active":"2024-08-31T06:04:25.879304+0000","last_became_peered":"2024-08-31T06:04:25.879304+0000","last_unstale":"2024-08-31T06:04:27.887298+0000","last_undegraded":"2024-08-31T06:04:27.887298+0000","last_fullsized":"2024-08-31T06:04:27.887298+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:56:42.992034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:37.939990+0000","last_change":"2024-08-31T06:04:22.847934+0000","last_active":"2024-08-31T06:04:37.939990+0000","last_peered":"2024-08-31T06:04:37.939990+0000","last_clean":"2024-08-31T06:04:37.939990+0000","last_became_active":"2024-08-31T06:04:22.847518+0000","last_became_peered":"2024-08-31T06:04:22.847518+0000","last_unstale":"2024-08-31T06:04:37.939990+0000","last_undegraded":"2024-08-31T06:04:37.939990+0000","last_fullsized":"2024-08-31T06:04:37.939990+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:26:21.476014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.896409+0000","last_change":"2024-08-31T06:04:25.884653+0000","last_active":"2024-08-31T06:04:27.896409+0000","last_peered":"2024-08-31T06:04:27.896409+0000","last_clean":"2024-08-31T06:04:27.896409+0000","last_became_active":"2024-08-31T06:04:25.884244+0000","last_became_peered":"2024-08-31T06:04:25.884244+0000","last_unstale":"2024-08-31T06:04:27.896409+0000","last_undegraded":"2024-08-31T06:04:27.896409+0000","last_fullsized":"2024-08-31T06:04:27.896409+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:13:00.814071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.892362+0000","last_change":"2024-08-31T06:04:22.845924+0000","last_active":"2024-08-31T06:04:27.892362+0000","last_peered":"2024-08-31T06:04:27.892362+0000","last_clean":"2024-08-31T06:04:27.892362+0000","last_became_active":"2024-08-31T06:04:22.845445+0000","last_became_peered":"2024-08-31T06:04:22.845445+0000","last_unstale":"2024-08-31T06:04:27.892362+0000","last_undegraded":"2024-08-31T06:04:27.892362+0000","last_fullsized":"2024-08-31T06:04:27.892362+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:25:21.052136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.894363+0000","last_change":"2024-08-31T06:04:25.890069+0000","last_active":"2024-08-31T06:04:27.894363+0000","last_peered":"2024-08-31T06:04:27.894363+0000","last_clean":"2024-08-31T06:04:27.894363+0000","last_became_active":"2024-08-31T06:04:25.889709+0000","last_became_peered":"2024-08-31T06:04:25.889709+0000","last_unstale":"2024-08-31T06:04:27.894363+0000","last_undegraded":"2024-08-31T06:04:27.894363+0000","last_fullsized":"2024-08-31T06:04:27.894363+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:13:51.360023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:37.936469+0000","last_change":"2024-08-31T06:04:22.847845+0000","last_active":"2024-08-31T06:04:37.936469+0000","last_peered":"2024-08-31T06:04:37.936469+0000","last_clean":"2024-08-31T06:04:37.936469+0000","last_became_active":"2024-08-31T06:04:22.847001+0000","last_became_peered":"2024-08-31T06:04:22.847001+0000","last_unstale":"2024-08-31T06:04:37.936469+0000","last_undegraded":"2024-08-31T06:04:37.936469+0000","last_fullsized":"2024-08-31T06:04:37.936469+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:24:34.209076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.890202+0000","last_change":"2024-08-31T06:04:25.886759+0000","last_active":"2024-08-31T06:04:27.890202+0000","last_peered":"2024-08-31T06:04:27.890202+0000","last_clean":"2024-08-31T06:04:27.890202+0000","last_became_active":"2024-08-31T06:04:25.886176+0000","last_became_peered":"2024-08-31T06:04:25.886176+0000","last_unstale":"2024-08-31T06:04:27.890202+0000","last_undegraded":"2024-08-31T06:04:27.890202+0000","last_fullsized":"2024-08-31T06:04:27.890202+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:11:12.637451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.873125+0000","last_change":"2024-08-31T06:04:25.887461+0000","last_active":"2024-08-31T06:04:27.873125+0000","last_peered":"2024-08-31T06:04:27.873125+0000","last_clean":"2024-08-31T06:04:27.873125+0000","last_became_active":"2024-08-31T06:04:25.887094+0000","last_became_peered":"2024-08-31T06:04:25.887094+0000","last_unstale":"2024-08-31T06:04:27.873125+0000","last_undegraded":"2024-08-31T06:04:27.873125+0000","last_fullsized":"2024-08-31T06:04:27.873125+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:07:14.935691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.894542+0000","last_change":"2024-08-31T06:04:22.842349+0000","last_active":"2024-08-31T06:04:27.894542+0000","last_peered":"2024-08-31T06:04:27.894542+0000","last_clean":"2024-08-31T06:04:27.894542+0000","last_became_active":"2024-08-31T06:04:22.841883+0000","last_became_peered":"2024-08-31T06:04:22.841883+0000","last_unstale":"2024-08-31T06:04:27.894542+0000","last_undegraded":"2024-08-31T06:04:27.894542+0000","last_fullsized":"2024-08-31T06:04:27.894542+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:41:19.021914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.888217+0000","last_change":"2024-08-31T06:04:25.887524+0000","last_active":"2024-08-31T06:04:27.888217+0000","last_peered":"2024-08-31T06:04:27.888217+0000","last_clean":"2024-08-31T06:04:27.888217+0000","last_became_active":"2024-08-31T06:04:25.887176+0000","last_became_peered":"2024-08-31T06:04:25.887176+0000","last_unstale":"2024-08-31T06:04:27.888217+0000","last_undegraded":"2024-08-31T06:04:27.888217+0000","last_fullsized":"2024-08-31T06:04:27.888217+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:08:05.476804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:38.940903+0000","last_change":"2024-08-31T06:04:22.847747+0000","last_active":"2024-08-31T06:04:38.940903+0000","last_peered":"2024-08-31T06:04:38.940903+0000","last_clean":"2024-08-31T06:04:38.940903+0000","last_became_active":"2024-08-31T06:04:22.847332+0000","last_became_peered":"2024-08-31T06:04:22.847332+0000","last_unstale":"2024-08-31T06:04:38.940903+0000","last_undegraded":"2024-08-31T06:04:38.940903+0000","last_fullsized":"2024-08-31T06:04:38.940903+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:41:35.910991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.891374+0000","last_change":"2024-08-31T06:04:25.883110+0000","last_active":"2024-08-31T06:04:27.891374+0000","last_peered":"2024-08-31T06:04:27.891374+0000","last_clean":"2024-08-31T06:04:27.891374+0000","last_became_active":"2024-08-31T06:04:25.880487+0000","last_became_peered":"2024-08-31T06:04:25.880487+0000","last_unstale":"2024-08-31T06:04:27.891374+0000","last_undegraded":"2024-08-31T06:04:27.891374+0000","last_fullsized":"2024-08-31T06:04:27.891374+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:22:13.656078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.898507+0000","last_change":"2024-08-31T06:04:22.964758+0000","last_active":"2024-08-31T06:04:27.898507+0000","last_peered":"2024-08-31T06:04:27.898507+0000","last_clean":"2024-08-31T06:04:27.898507+0000","last_became_active":"2024-08-31T06:04:22.964391+0000","last_became_peered":"2024-08-31T06:04:22.964391+0000","last_unstale":"2024-08-31T06:04:27.898507+0000","last_undegraded":"2024-08-31T06:04:27.898507+0000","last_fullsized":"2024-08-31T06:04:27.898507+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:28:59.190657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:27.895257+0000","last_change":"2024-08-31T06:04:25.893738+0000","last_active":"2024-08-31T06:04:27.895257+0000","last_peered":"2024-08-31T06:04:27.895257+0000","last_clean":"2024-08-31T06:04:27.895257+0000","last_became_active":"2024-08-31T06:04:25.893308+0000","last_became_peered":"2024-08-31T06:04:25.893308+0000","last_unstale":"2024-08-31T06:04:27.895257+0000","last_undegraded":"2024-08-31T06:04:27.895257+0000","last_fullsized":"2024-08-31T06:04:27.895257+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:26:48.451894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-08-31T06:04:38.942319+0000","last_change":"2024-08-31T06:04:22.843346+0000","last_active":"2024-08-31T06:04:38.942319+0000","last_peered":"2024-08-31T06:04:38.942319+0000","last_clean":"2024-08-31T06:04:38.942319+0000","last_became_active":"2024-08-31T06:04:22.842249+0000","last_became_peered":"2024-08-31T06:04:22.842249+0000","last_unstale":"2024-08-31T06:04:38.942319+0000","last_undegraded":"2024-08-31T06:04:38.942319+0000","last_fullsized":"2024-08-31T06:04:38.942319+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:40:57.618940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":58759,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":146,"num_read_kb":136,"num_write":160,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":172020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":249373,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710740,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":843776,"data_stored":418917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874264,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":1065156,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037791,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":860160,"data_stored":419153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234020,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":860160,"data_stored":419073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397546,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":1011394,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593776,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":847872,"data_stored":418542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790004,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":1010687,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953531,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":851968,"data_stored":419370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149758,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":864256,"data_stored":418853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411397,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":925696,"data_stored":470186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542218,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36228,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708924,"statfs":{"total":95995035648,"available":95957938176,"internally_reserved":0,"allocated":905216,"data_stored":472702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705744,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":868352,"data_stored":419813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34699,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34699,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34699,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":54367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":51851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":56336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-31T06:05:02.887 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-08-31T06:05:02.887 INFO:tasks.cephadm:Setting up client nodes... 2024-08-31T06:05:02.888 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-08-31T06:05:03.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:03 smithi016 ceph-mon[17194]: pgmap v295: 97 pgs: 97 active+clean; 634 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-08-31T06:05:03.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:03 smithi016 ceph-mon[17194]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T06:05:03.305 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:05:03.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:03 smithi155 ceph-mon[21742]: pgmap v295: 97 pgs: 97 active+clean; 634 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-08-31T06:05:03.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:03 smithi155 ceph-mon[21742]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T06:05:03.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:03 smithi184 ceph-mon[22331]: pgmap v295: 97 pgs: 97 active+clean; 634 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-08-31T06:05:03.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:03 smithi184 ceph-mon[22331]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T06:05:03.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:03.950+0000 7f9ab5400640 1 -- 172.21.15.16:0/229075026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ab0075e40 msgr2=0x7f9ab010f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:03.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:03.950+0000 7f9ab5400640 1 --2- 172.21.15.16:0/229075026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ab0075e40 0x7f9ab010f7b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9a98009900 tx=0x7f9a9802a3f0 comp rx=0 tx=0).stop 2024-08-31T06:05:03.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:03.950+0000 7f9ab5400640 1 -- 172.21.15.16:0/229075026 shutdown_connections 2024-08-31T06:05:03.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:03.950+0000 7f9ab5400640 1 --2- 172.21.15.16:0/229075026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ab010fcf0 0x7f9ab0112110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:03.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:03.950+0000 7f9ab5400640 1 --2- 172.21.15.16:0/229075026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ab0075e40 0x7f9ab010f7b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:03.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:03.950+0000 7f9ab5400640 1 --2- 172.21.15.16:0/229075026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ab00754f0 0x7f9ab0075900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:03.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:03.950+0000 7f9ab5400640 1 -- 172.21.15.16:0/229075026 >> 172.21.15.16:0/229075026 conn(0x7f9ab00ff180 msgr2=0x7f9ab01015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:03.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:03.950+0000 7f9ab5400640 1 -- 172.21.15.16:0/229075026 shutdown_connections 2024-08-31T06:05:03.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:03.950+0000 7f9ab5400640 1 -- 172.21.15.16:0/229075026 wait complete. 2024-08-31T06:05:03.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:03.951+0000 7f9ab5400640 1 Processor -- start 2024-08-31T06:05:03.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:03.951+0000 7f9ab5400640 1 -- start start 2024-08-31T06:05:03.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:03.952+0000 7f9ab5400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ab00754f0 0x7f9ab019c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:03.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:03.952+0000 7f9ab5400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ab0075e40 0x7f9ab019c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:03.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:03.952+0000 7f9ab5400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ab010fcf0 0x7f9ab01a1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:03.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:03.952+0000 7f9ab5400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab0115860 con 0x7f9ab00754f0 2024-08-31T06:05:03.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:03.952+0000 7f9ab5400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab01156e0 con 0x7f9ab0075e40 2024-08-31T06:05:03.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:03.952+0000 7f9ab5400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab01159e0 con 0x7f9ab010fcf0 2024-08-31T06:05:03.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:03.952+0000 7f9aafe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ab00754f0 0x7f9ab019c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:03.952+0000 7f9aa7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ab0075e40 0x7f9ab019c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:03.952+0000 7f9aafe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ab00754f0 0x7f9ab019c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43362/0 (socket says 172.21.15.16:43362) 2024-08-31T06:05:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:03.952+0000 7f9aafe00640 1 -- 172.21.15.16:0/2157569149 learned_addr learned my addr 172.21.15.16:0/2157569149 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:05:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:03.952+0000 7f9ab4a00640 1 --2- 172.21.15.16:0/2157569149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ab010fcf0 0x7f9ab01a1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:03.953+0000 7f9aafe00640 1 -- 172.21.15.16:0/2157569149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ab010fcf0 msgr2=0x7f9ab01a1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:03.953+0000 7f9aafe00640 1 --2- 172.21.15.16:0/2157569149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ab010fcf0 0x7f9ab01a1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:03.953+0000 7f9aafe00640 1 -- 172.21.15.16:0/2157569149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ab0075e40 msgr2=0x7f9ab019c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:03.953+0000 7f9aafe00640 1 --2- 172.21.15.16:0/2157569149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ab0075e40 0x7f9ab019c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:03.953+0000 7f9aafe00640 1 -- 172.21.15.16:0/2157569149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab01a2260 con 0x7f9ab00754f0 2024-08-31T06:05:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:03.953+0000 7f9aa7e00640 1 --2- 172.21.15.16:0/2157569149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ab0075e40 0x7f9ab019c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:05:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:03.953+0000 7f9aafe00640 1 --2- 172.21.15.16:0/2157569149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ab00754f0 0x7f9ab019c2f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9aa000e9f0 tx=0x7f9aa000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:03.953+0000 7f9aad600640 1 -- 172.21.15.16:0/2157569149 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa000cd80 con 0x7f9ab00754f0 2024-08-31T06:05:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:03.953+0000 7f9aad600640 1 -- 172.21.15.16:0/2157569149 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9aa00108e0 con 0x7f9ab00754f0 2024-08-31T06:05:03.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:03.953+0000 7f9ab5400640 1 -- 172.21.15.16:0/2157569149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ab01a2550 con 0x7f9ab00754f0 2024-08-31T06:05:03.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:03.953+0000 7f9aad600640 1 -- 172.21.15.16:0/2157569149 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0018980 con 0x7f9ab00754f0 2024-08-31T06:05:03.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:03.953+0000 7f9ab5400640 1 -- 172.21.15.16:0/2157569149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ab01a8cb0 con 0x7f9ab00754f0 2024-08-31T06:05:03.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:03.954+0000 7f9ab5400640 1 -- 172.21.15.16:0/2157569149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ab0075900 con 0x7f9ab00754f0 2024-08-31T06:05:03.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:03.957+0000 7f9aad600640 1 -- 172.21.15.16:0/2157569149 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f9aa0010430 con 0x7f9ab00754f0 2024-08-31T06:05:03.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:03.958+0000 7f9aad600640 1 --2- 172.21.15.16:0/2157569149 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9a940b4c80 0x7f9a940b7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:03.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:03.958+0000 7f9aad600640 1 -- 172.21.15.16:0/2157569149 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7f9aa00e5ae0 con 0x7f9ab00754f0 2024-08-31T06:05:03.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:03.959+0000 7f9aa7e00640 1 --2- 172.21.15.16:0/2157569149 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9a940b4c80 0x7f9a940b7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:03.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:03.959+0000 7f9aa7e00640 1 --2- 172.21.15.16:0/2157569149 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9a940b4c80 0x7f9a940b7190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9a98009230 tx=0x7f9a98031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:03.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:03.960+0000 7f9aad600640 1 -- 172.21.15.16:0/2157569149 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f9aa00ac420 con 0x7f9ab00754f0 2024-08-31T06:05:04.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:04.236+0000 7f9ab5400640 1 -- 172.21.15.16:0/2157569149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f9ab01a8f90 con 0x7f9ab00754f0 2024-08-31T06:05:04.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:04.239+0000 7f9aad600640 1 -- 172.21.15.16:0/2157569149 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f9aa00b0be0 con 0x7f9ab00754f0 2024-08-31T06:05:04.242 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-08-31T06:05:04.242 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCQstJmkLswDhAAbnFWCcRtoTdzlYcPZIOjXw== 2024-08-31T06:05:04.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:04.242+0000 7f9ab5400640 1 -- 172.21.15.16:0/2157569149 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9a940b4c80 msgr2=0x7f9a940b7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:04.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:04.242+0000 7f9ab5400640 1 --2- 172.21.15.16:0/2157569149 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9a940b4c80 0x7f9a940b7190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9a98009230 tx=0x7f9a98031400 comp rx=0 tx=0).stop 2024-08-31T06:05:04.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:04.242+0000 7f9ab5400640 1 -- 172.21.15.16:0/2157569149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ab00754f0 msgr2=0x7f9ab019c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:04.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:04.242+0000 7f9ab5400640 1 --2- 172.21.15.16:0/2157569149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ab00754f0 0x7f9ab019c2f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9aa000e9f0 tx=0x7f9aa000eeb0 comp rx=0 tx=0).stop 2024-08-31T06:05:04.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:04.242+0000 7f9ab5400640 1 -- 172.21.15.16:0/2157569149 shutdown_connections 2024-08-31T06:05:04.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:04.242+0000 7f9ab5400640 1 --2- 172.21.15.16:0/2157569149 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9a940b4c80 0x7f9a940b7190 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:04.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:04.242+0000 7f9ab5400640 1 --2- 172.21.15.16:0/2157569149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ab010fcf0 0x7f9ab01a1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:04.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:04.242+0000 7f9ab5400640 1 --2- 172.21.15.16:0/2157569149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ab0075e40 0x7f9ab019c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:04.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:04.242+0000 7f9ab5400640 1 --2- 172.21.15.16:0/2157569149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ab00754f0 0x7f9ab019c2f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:04.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:04.242+0000 7f9ab5400640 1 -- 172.21.15.16:0/2157569149 >> 172.21.15.16:0/2157569149 conn(0x7f9ab00ff180 msgr2=0x7f9ab0108780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:04.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:04.242+0000 7f9ab5400640 1 -- 172.21.15.16:0/2157569149 shutdown_connections 2024-08-31T06:05:04.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:04.242+0000 7f9ab5400640 1 -- 172.21.15.16:0/2157569149 wait complete. 2024-08-31T06:05:04.974 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-08-31T06:05:04.974 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-08-31T06:05:04.974 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-08-31T06:05:05.056 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph config log 1 --format=json 2024-08-31T06:05:05.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:05 smithi155 ceph-mon[21742]: pgmap v296: 97 pgs: 97 active+clean; 634 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-08-31T06:05:05.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:05 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/2157569149' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-08-31T06:05:05.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:05 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/2157569149' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-08-31T06:05:05.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:05 smithi016 ceph-mon[17194]: pgmap v296: 97 pgs: 97 active+clean; 634 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-08-31T06:05:05.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:05 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/2157569149' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-08-31T06:05:05.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:05 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/2157569149' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-08-31T06:05:05.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:05 smithi184 ceph-mon[22331]: pgmap v296: 97 pgs: 97 active+clean; 634 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-08-31T06:05:05.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:05 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/2157569149' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-08-31T06:05:05.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:05 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/2157569149' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-08-31T06:05:05.477 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:05:06.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.132+0000 7fdff2c00640 1 -- 172.21.15.16:0/1603644865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfec10b460 msgr2=0x7fdfec10b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:06.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.132+0000 7fdff2c00640 1 --2- 172.21.15.16:0/1603644865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfec10b460 0x7fdfec10b870 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4009900 tx=0x7fdfd402a410 comp rx=0 tx=0).stop 2024-08-31T06:05:06.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.132+0000 7fdff2c00640 1 -- 172.21.15.16:0/1603644865 shutdown_connections 2024-08-31T06:05:06.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.132+0000 7fdff2c00640 1 --2- 172.21.15.16:0/1603644865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfec10bdb0 0x7fdfec1125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:06.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.132+0000 7fdff2c00640 1 --2- 172.21.15.16:0/1603644865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfec10b460 0x7fdfec10b870 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:06.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.132+0000 7fdff2c00640 1 --2- 172.21.15.16:0/1603644865 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfec105b20 0x7fdfec105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:06.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.132+0000 7fdff2c00640 1 -- 172.21.15.16:0/1603644865 >> 172.21.15.16:0/1603644865 conn(0x7fdfec1015b0 msgr2=0x7fdfec103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:06.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.132+0000 7fdff2c00640 1 -- 172.21.15.16:0/1603644865 shutdown_connections 2024-08-31T06:05:06.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.132+0000 7fdff2c00640 1 -- 172.21.15.16:0/1603644865 wait complete. 2024-08-31T06:05:06.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.133+0000 7fdff2c00640 1 Processor -- start 2024-08-31T06:05:06.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.133+0000 7fdff2c00640 1 -- start start 2024-08-31T06:05:06.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.134+0000 7fdff2c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfec105b20 0x7fdfec19c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:06.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.134+0000 7fdff2c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfec10b460 0x7fdfec19c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:06.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.134+0000 7fdff2c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfec10bdb0 0x7fdfec1a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:06.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.134+0000 7fdff2c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdfec115ac0 con 0x7fdfec10bdb0 2024-08-31T06:05:06.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.134+0000 7fdff2c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdfec115940 con 0x7fdfec10b460 2024-08-31T06:05:06.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.134+0000 7fdff2c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdfec115c40 con 0x7fdfec105b20 2024-08-31T06:05:06.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.134+0000 7fdfebe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfec105b20 0x7fdfec19c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:06.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.134+0000 7fdfe3400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfec10b460 0x7fdfec19c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:06.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.134+0000 7fdfe3400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfec10b460 0x7fdfec19c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:40004/0 (socket says 172.21.15.16:40004) 2024-08-31T06:05:06.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.134+0000 7fdff0c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfec10bdb0 0x7fdfec1a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:06.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.134+0000 7fdfebe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfec105b20 0x7fdfec19c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:58756/0 (socket says 172.21.15.16:58756) 2024-08-31T06:05:06.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.134+0000 7fdfe3400640 1 -- 172.21.15.16:0/3372957500 learned_addr learned my addr 172.21.15.16:0/3372957500 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:05:06.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.135+0000 7fdfebe00640 1 -- 172.21.15.16:0/3372957500 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfec10b460 msgr2=0x7fdfec19c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:06.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.135+0000 7fdfebe00640 1 --2- 172.21.15.16:0/3372957500 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfec10b460 0x7fdfec19c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:06.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.135+0000 7fdfebe00640 1 -- 172.21.15.16:0/3372957500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfec10bdb0 msgr2=0x7fdfec1a1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:06.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.135+0000 7fdfebe00640 1 --2- 172.21.15.16:0/3372957500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfec10bdb0 0x7fdfec1a1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:06.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.135+0000 7fdfebe00640 1 -- 172.21.15.16:0/3372957500 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfec1a23c0 con 0x7fdfec105b20 2024-08-31T06:05:06.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.135+0000 7fdfebe00640 1 --2- 172.21.15.16:0/3372957500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfec105b20 0x7fdfec19c450 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc00e9f0 tx=0x7fdfdc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:06.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.135+0000 7fdfe9600640 1 -- 172.21.15.16:0/3372957500 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc00ccc0 con 0x7fdfec105b20 2024-08-31T06:05:06.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.135+0000 7fdff2c00640 1 -- 172.21.15.16:0/3372957500 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfec1a26b0 con 0x7fdfec105b20 2024-08-31T06:05:06.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.135+0000 7fdfe9600640 1 -- 172.21.15.16:0/3372957500 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfdc005020 con 0x7fdfec105b20 2024-08-31T06:05:06.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.135+0000 7fdfe9600640 1 -- 172.21.15.16:0/3372957500 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc004a00 con 0x7fdfec105b20 2024-08-31T06:05:06.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.135+0000 7fdff2c00640 1 -- 172.21.15.16:0/3372957500 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfec1a8e10 con 0x7fdfec105b20 2024-08-31T06:05:06.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.136+0000 7fdff2c00640 1 -- 172.21.15.16:0/3372957500 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfec105f30 con 0x7fdfec105b20 2024-08-31T06:05:06.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.139+0000 7fdfe9600640 1 -- 172.21.15.16:0/3372957500 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7fdfdc004ba0 con 0x7fdfec105b20 2024-08-31T06:05:06.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.140+0000 7fdfe9600640 1 --2- 172.21.15.16:0/3372957500 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdfd00b4c10 0x7fdfd00b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:06.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.140+0000 7fdfe9600640 1 -- 172.21.15.16:0/3372957500 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7fdfdc0e5260 con 0x7fdfec105b20 2024-08-31T06:05:06.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.141+0000 7fdfe3400640 1 --2- 172.21.15.16:0/3372957500 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdfd00b4c10 0x7fdfd00b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:06.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.141+0000 7fdfe3400640 1 --2- 172.21.15.16:0/3372957500 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdfd00b4c10 0x7fdfd00b7120 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4006010 tx=0x7fdfd4005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:06.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.142+0000 7fdfe9600640 1 -- 172.21.15.16:0/3372957500 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fdfdc0ac1f0 con 0x7fdfec105b20 2024-08-31T06:05:06.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.340+0000 7fdff2c00640 1 -- 172.21.15.16:0/3372957500 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7fdfec19d580 con 0x7fdfec105b20 2024-08-31T06:05:06.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.341+0000 7fdfe9600640 1 -- 172.21.15.16:0/3372957500 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v53) ==== 86+0+133 (secure 0 0 0) 0x7fdfec19d580 con 0x7fdfec105b20 2024-08-31T06:05:06.344 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:05:06.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.344+0000 7fdff2c00640 1 -- 172.21.15.16:0/3372957500 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdfd00b4c10 msgr2=0x7fdfd00b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:06.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.344+0000 7fdff2c00640 1 --2- 172.21.15.16:0/3372957500 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdfd00b4c10 0x7fdfd00b7120 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4006010 tx=0x7fdfd4005a20 comp rx=0 tx=0).stop 2024-08-31T06:05:06.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.344+0000 7fdff2c00640 1 -- 172.21.15.16:0/3372957500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfec105b20 msgr2=0x7fdfec19c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:06.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.344+0000 7fdff2c00640 1 --2- 172.21.15.16:0/3372957500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfec105b20 0x7fdfec19c450 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc00e9f0 tx=0x7fdfdc00eeb0 comp rx=0 tx=0).stop 2024-08-31T06:05:06.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.344+0000 7fdff2c00640 1 -- 172.21.15.16:0/3372957500 shutdown_connections 2024-08-31T06:05:06.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.344+0000 7fdff2c00640 1 --2- 172.21.15.16:0/3372957500 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdfd00b4c10 0x7fdfd00b7120 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:06.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.344+0000 7fdff2c00640 1 --2- 172.21.15.16:0/3372957500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfec10bdb0 0x7fdfec1a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:06.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.344+0000 7fdff2c00640 1 --2- 172.21.15.16:0/3372957500 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfec10b460 0x7fdfec19c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:06.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.344+0000 7fdff2c00640 1 --2- 172.21.15.16:0/3372957500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfec105b20 0x7fdfec19c450 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:06.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.344+0000 7fdff2c00640 1 -- 172.21.15.16:0/3372957500 >> 172.21.15.16:0/3372957500 conn(0x7fdfec1015b0 msgr2=0x7fdfec1039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:06.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.344+0000 7fdff2c00640 1 -- 172.21.15.16:0/3372957500 shutdown_connections 2024-08-31T06:05:06.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:06.344+0000 7fdff2c00640 1 -- 172.21.15.16:0/3372957500 wait complete. 2024-08-31T06:05:07.101 INFO:teuthology.orchestra.run.smithi016.stdout:[{"version":53,"timestamp":"2024-08-31T06:04:13.472625+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2024-08-31T06:05:07.101 INFO:tasks.ceph_manager:config epoch is 53 2024-08-31T06:05:07.102 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-08-31T06:05:07.102 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-08-31T06:05:07.102 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph mgr dump --format=json 2024-08-31T06:05:07.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:07 smithi155 ceph-mon[21742]: pgmap v297: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-08-31T06:05:07.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:07 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/3372957500' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-08-31T06:05:07.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:07 smithi016 ceph-mon[17194]: pgmap v297: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-08-31T06:05:07.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:07 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/3372957500' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-08-31T06:05:07.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:07 smithi184 ceph-mon[22331]: pgmap v297: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-08-31T06:05:07.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:07 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/3372957500' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-08-31T06:05:07.479 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:05:08.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.084+0000 7f2415c00640 1 -- 172.21.15.16:0/3447451669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2410102cb0 msgr2=0x7f24101071e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:08.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.085+0000 7f2415c00640 1 --2- 172.21.15.16:0/3447451669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2410102cb0 0x7f24101071e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f23f8009930 tx=0x7f23f802a420 comp rx=0 tx=0).stop 2024-08-31T06:05:08.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.085+0000 7f2415c00640 1 -- 172.21.15.16:0/3447451669 shutdown_connections 2024-08-31T06:05:08.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.085+0000 7f2415c00640 1 --2- 172.21.15.16:0/3447451669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24101077a0 0x7f2410107c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:08.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.085+0000 7f2415c00640 1 --2- 172.21.15.16:0/3447451669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2410102cb0 0x7f24101071e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:08.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.085+0000 7f2415c00640 1 --2- 172.21.15.16:0/3447451669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2410102360 0x7f2410102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:08.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.085+0000 7f2415c00640 1 -- 172.21.15.16:0/3447451669 >> 172.21.15.16:0/3447451669 conn(0x7f24100fe070 msgr2=0x7f24101004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:08.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.085+0000 7f2415c00640 1 -- 172.21.15.16:0/3447451669 shutdown_connections 2024-08-31T06:05:08.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.085+0000 7f2415c00640 1 -- 172.21.15.16:0/3447451669 wait complete. 2024-08-31T06:05:08.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.086+0000 7f2415c00640 1 Processor -- start 2024-08-31T06:05:08.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.086+0000 7f2415c00640 1 -- start start 2024-08-31T06:05:08.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.086+0000 7f2415c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2410102360 0x7f24101a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:08.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.086+0000 7f2415c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2410102cb0 0x7f24101a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:08.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.086+0000 7f2415c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24101077a0 0x7f24101a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:08.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.087+0000 7f2415c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2410119c40 con 0x7f24101077a0 2024-08-31T06:05:08.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.087+0000 7f2415c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2410119ac0 con 0x7f2410102360 2024-08-31T06:05:08.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.087+0000 7f2415c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2410119dc0 con 0x7f2410102cb0 2024-08-31T06:05:08.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.087+0000 7f240fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2410102360 0x7f24101a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:08.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.087+0000 7f2415200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24101077a0 0x7f24101a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:08.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.087+0000 7f2407e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2410102cb0 0x7f24101a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:08.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.087+0000 7f240fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2410102360 0x7f24101a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:40026/0 (socket says 172.21.15.16:40026) 2024-08-31T06:05:08.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.087+0000 7f240fe00640 1 -- 172.21.15.16:0/3281517231 learned_addr learned my addr 172.21.15.16:0/3281517231 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:05:08.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.087+0000 7f2415200640 1 -- 172.21.15.16:0/3281517231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2410102cb0 msgr2=0x7f24101a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:08.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.087+0000 7f2415200640 1 --2- 172.21.15.16:0/3281517231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2410102cb0 0x7f24101a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:08.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.087+0000 7f2415200640 1 -- 172.21.15.16:0/3281517231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2410102360 msgr2=0x7f24101a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:08.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.087+0000 7f2415200640 1 --2- 172.21.15.16:0/3281517231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2410102360 0x7f24101a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:08.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.087+0000 7f2415200640 1 -- 172.21.15.16:0/3281517231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24101a6600 con 0x7f24101077a0 2024-08-31T06:05:08.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.087+0000 7f240fe00640 1 --2- 172.21.15.16:0/3281517231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2410102360 0x7f24101a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:05:08.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.088+0000 7f2415200640 1 --2- 172.21.15.16:0/3281517231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24101077a0 0x7f24101a5ed0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f240800e940 tx=0x7f240800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:08.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.088+0000 7f240d600640 1 -- 172.21.15.16:0/3281517231 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2408004d20 con 0x7f24101077a0 2024-08-31T06:05:08.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.088+0000 7f240d600640 1 -- 172.21.15.16:0/3281517231 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2408005020 con 0x7f24101077a0 2024-08-31T06:05:08.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.088+0000 7f2415c00640 1 -- 172.21.15.16:0/3281517231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24101a68f0 con 0x7f24101077a0 2024-08-31T06:05:08.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.088+0000 7f240d600640 1 -- 172.21.15.16:0/3281517231 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24080116d0 con 0x7f24101077a0 2024-08-31T06:05:08.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.088+0000 7f2415c00640 1 -- 172.21.15.16:0/3281517231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24101ad050 con 0x7f24101077a0 2024-08-31T06:05:08.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.089+0000 7f2415c00640 1 -- 172.21.15.16:0/3281517231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24101071e0 con 0x7f24101077a0 2024-08-31T06:05:08.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.091+0000 7f240d600640 1 -- 172.21.15.16:0/3281517231 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f2408005460 con 0x7f24101077a0 2024-08-31T06:05:08.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.093+0000 7f240d600640 1 --2- 172.21.15.16:0/3281517231 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f23f40b4c90 0x7f23f40b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:08.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.093+0000 7f240d600640 1 -- 172.21.15.16:0/3281517231 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7f24080e5740 con 0x7f24101077a0 2024-08-31T06:05:08.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.093+0000 7f240fe00640 1 --2- 172.21.15.16:0/3281517231 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f23f40b4c90 0x7f23f40b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:08.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.094+0000 7f240fe00640 1 --2- 172.21.15.16:0/3281517231 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f23f40b4c90 0x7f23f40b71a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f24000104e0 tx=0x7f240000b410 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:08.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.095+0000 7f240d600640 1 -- 172.21.15.16:0/3281517231 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f2408016030 con 0x7f24101077a0 2024-08-31T06:05:08.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.340+0000 7f2415c00640 1 -- 172.21.15.16:0/3281517231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f24101a17c0 con 0x7f24101077a0 2024-08-31T06:05:08.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.346+0000 7f240d600640 1 -- 172.21.15.16:0/3281517231 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v21) ==== 74+0+293208 (secure 0 0 0) 0x7f24080ac100 con 0x7f24101077a0 2024-08-31T06:05:08.352 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:05:08.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.351+0000 7f2415c00640 1 -- 172.21.15.16:0/3281517231 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f23f40b4c90 msgr2=0x7f23f40b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:08.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.351+0000 7f2415c00640 1 --2- 172.21.15.16:0/3281517231 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f23f40b4c90 0x7f23f40b71a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f24000104e0 tx=0x7f240000b410 comp rx=0 tx=0).stop 2024-08-31T06:05:08.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.351+0000 7f2415c00640 1 -- 172.21.15.16:0/3281517231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24101077a0 msgr2=0x7f24101a5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:08.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.351+0000 7f2415c00640 1 --2- 172.21.15.16:0/3281517231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24101077a0 0x7f24101a5ed0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f240800e940 tx=0x7f240800ee00 comp rx=0 tx=0).stop 2024-08-31T06:05:08.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.351+0000 7f2415c00640 1 -- 172.21.15.16:0/3281517231 shutdown_connections 2024-08-31T06:05:08.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.351+0000 7f2415c00640 1 --2- 172.21.15.16:0/3281517231 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f23f40b4c90 0x7f23f40b71a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:08.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.351+0000 7f2415c00640 1 --2- 172.21.15.16:0/3281517231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2410102cb0 0x7f24101a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:08.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.351+0000 7f2415c00640 1 --2- 172.21.15.16:0/3281517231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24101077a0 0x7f24101a5ed0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:08.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.351+0000 7f2415c00640 1 --2- 172.21.15.16:0/3281517231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2410102360 0x7f24101a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:08.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.351+0000 7f2415c00640 1 -- 172.21.15.16:0/3281517231 >> 172.21.15.16:0/3281517231 conn(0x7f24100fe070 msgr2=0x7f24101051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:08.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.351+0000 7f2415c00640 1 -- 172.21.15.16:0/3281517231 shutdown_connections 2024-08-31T06:05:08.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:08.352+0000 7f2415c00640 1 -- 172.21.15.16:0/3281517231 wait complete. 2024-08-31T06:05:09.202 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"flags":0,"active_gid":14220,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4279751337},{"type":"v1","addr":"172.21.15.16:6801","nonce":4279751337}]},"active_addr":"172.21.15.16:6801/4279751337","active_change":"2024-08-31T05:57:23.153395+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24166,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24182,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.16:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":1836579324}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":2203903249}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":2975339075}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":3165937677}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":2086568733}]}]} 2024-08-31T06:05:09.210 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-08-31T06:05:09.210 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-08-31T06:05:09.210 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph osd dump --format=json 2024-08-31T06:05:09.263 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:09 smithi016 ceph-mon[17194]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-08-31T06:05:09.263 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:09 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/3281517231' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-08-31T06:05:09.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:09 smithi155 ceph-mon[21742]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-08-31T06:05:09.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:09 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/3281517231' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-08-31T06:05:09.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:09 smithi184 ceph-mon[22331]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-08-31T06:05:09.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:09 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/3281517231' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-08-31T06:05:09.588 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:05:10.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.217+0000 7f70db200640 1 -- 172.21.15.16:0/130313412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70dc109090 msgr2=0x7f70dc1094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:10.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.217+0000 7f70db200640 1 --2- 172.21.15.16:0/130313412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70dc109090 0x7f70dc1094a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f70c4009860 tx=0x7f70c402a340 comp rx=0 tx=0).stop 2024-08-31T06:05:10.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.217+0000 7f70db200640 1 -- 172.21.15.16:0/130313412 shutdown_connections 2024-08-31T06:05:10.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.217+0000 7f70db200640 1 --2- 172.21.15.16:0/130313412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70dc1099e0 0x7f70dc110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:10.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.217+0000 7f70db200640 1 --2- 172.21.15.16:0/130313412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70dc109090 0x7f70dc1094a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:10.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.217+0000 7f70db200640 1 --2- 172.21.15.16:0/130313412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70dc103750 0x7f70dc103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:10.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.217+0000 7f70db200640 1 -- 172.21.15.16:0/130313412 >> 172.21.15.16:0/130313412 conn(0x7f70dc0ff1a0 msgr2=0x7f70dc1015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:10.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.218+0000 7f70db200640 1 -- 172.21.15.16:0/130313412 shutdown_connections 2024-08-31T06:05:10.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.218+0000 7f70db200640 1 -- 172.21.15.16:0/130313412 wait complete. 2024-08-31T06:05:10.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.219+0000 7f70db200640 1 Processor -- start 2024-08-31T06:05:10.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.219+0000 7f70db200640 1 -- start start 2024-08-31T06:05:10.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.220+0000 7f70db200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70dc103750 0x7f70dc0754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:10.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.220+0000 7f70db200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70dc109090 0x7f70dc0759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:10.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.220+0000 7f70db200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70dc1099e0 0x7f70dc075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:10.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.220+0000 7f70db200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc113670 con 0x7f70dc109090 2024-08-31T06:05:10.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.220+0000 7f70db200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc1134f0 con 0x7f70dc1099e0 2024-08-31T06:05:10.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.220+0000 7f70db200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc1137f0 con 0x7f70dc103750 2024-08-31T06:05:10.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.220+0000 7f70d1400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70dc109090 0x7f70dc0759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:10.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.220+0000 7f70d9e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70dc103750 0x7f70dc0754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:10.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.220+0000 7f70d1400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70dc109090 0x7f70dc0759f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41406/0 (socket says 172.21.15.16:41406) 2024-08-31T06:05:10.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.220+0000 7f70d1400640 1 -- 172.21.15.16:0/176350501 learned_addr learned my addr 172.21.15.16:0/176350501 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:05:10.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.220+0000 7f70da800640 1 --2- 172.21.15.16:0/176350501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70dc1099e0 0x7f70dc075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:10.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.220+0000 7f70d9e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70dc103750 0x7f70dc0754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:58794/0 (socket says 172.21.15.16:58794) 2024-08-31T06:05:10.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.221+0000 7f70d1400640 1 -- 172.21.15.16:0/176350501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70dc103750 msgr2=0x7f70dc0754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:10.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.221+0000 7f70d1400640 1 --2- 172.21.15.16:0/176350501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70dc103750 0x7f70dc0754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:10.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.221+0000 7f70d1400640 1 -- 172.21.15.16:0/176350501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70dc1099e0 msgr2=0x7f70dc075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:10.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.221+0000 7f70d1400640 1 --2- 172.21.15.16:0/176350501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70dc1099e0 0x7f70dc075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:10.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.221+0000 7f70d1400640 1 -- 172.21.15.16:0/176350501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70dc1a8320 con 0x7f70dc109090 2024-08-31T06:05:10.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.221+0000 7f70d1400640 1 --2- 172.21.15.16:0/176350501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70dc109090 0x7f70dc0759f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f70c4035040 tx=0x7f70c4032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:10.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.221+0000 7f70d3400640 1 -- 172.21.15.16:0/176350501 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c4040020 con 0x7f70dc109090 2024-08-31T06:05:10.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.221+0000 7f70d3400640 1 -- 172.21.15.16:0/176350501 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70c4030e10 con 0x7f70dc109090 2024-08-31T06:05:10.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.221+0000 7f70d3400640 1 -- 172.21.15.16:0/176350501 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c4037040 con 0x7f70dc109090 2024-08-31T06:05:10.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.221+0000 7f70db200640 1 -- 172.21.15.16:0/176350501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70dc1a85b0 con 0x7f70dc109090 2024-08-31T06:05:10.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.221+0000 7f70db200640 1 -- 172.21.15.16:0/176350501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70dc1a8a40 con 0x7f70dc109090 2024-08-31T06:05:10.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.223+0000 7f70db200640 1 -- 172.21.15.16:0/176350501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70dc103b60 con 0x7f70dc109090 2024-08-31T06:05:10.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.225+0000 7f70d3400640 1 -- 172.21.15.16:0/176350501 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f70c4043020 con 0x7f70dc109090 2024-08-31T06:05:10.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.226+0000 7f70d3400640 1 --2- 172.21.15.16:0/176350501 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f70c00b4c90 0x7f70c00b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:10.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.226+0000 7f70d3400640 1 -- 172.21.15.16:0/176350501 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7f70c4104e90 con 0x7f70dc109090 2024-08-31T06:05:10.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.226+0000 7f70d9e00640 1 --2- 172.21.15.16:0/176350501 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f70c00b4c90 0x7f70c00b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:10.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.227+0000 7f70d9e00640 1 --2- 172.21.15.16:0/176350501 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f70c00b4c90 0x7f70c00b71a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f70dc076f00 tx=0x7f70cc010040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:10.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.230+0000 7f70d3400640 1 -- 172.21.15.16:0/176350501 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f70c404a030 con 0x7f70dc109090 2024-08-31T06:05:10.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.427+0000 7f70db200640 1 -- 172.21.15.16:0/176350501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f70dc10b820 con 0x7f70dc109090 2024-08-31T06:05:10.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.429+0000 7f70d3400640 1 -- 172.21.15.16:0/176350501 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22608 (secure 0 0 0) 0x7f70dc10b820 con 0x7f70dc109090 2024-08-31T06:05:10.432 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:05:10.432 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":78,"fsid":"673e4aa0-675d-11ef-bcd4-c7b262605968","created":"2024-08-31T05:55:09.632536+0000","modified":"2024-08-31T06:04:27.098694+0000","last_up_change":"2024-08-31T06:03:27.893697+0000","last_in_change":"2024-08-31T06:03:05.664945+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-31T05:59:23.642270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-31T06:04:20.871835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-31T06:04:24.119499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e349dae-6aa5-4b6d-9107-a945655e11bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6803","nonce":1725037400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6805","nonce":1725037400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6809","nonce":1725037400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6807","nonce":1725037400}]},"public_addr":"172.21.15.16:6803/1725037400","cluster_addr":"172.21.15.16:6805/1725037400","heartbeat_back_addr":"172.21.15.16:6809/1725037400","heartbeat_front_addr":"172.21.15.16:6807/1725037400","state":["exists","up"]},{"osd":1,"uuid":"bd645dc3-93b5-40af-89d8-08ccec22e50d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6801","nonce":4241618440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6803","nonce":4241618440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6807","nonce":4241618440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6805","nonce":4241618440}]},"public_addr":"172.21.15.155:6801/4241618440","cluster_addr":"172.21.15.155:6803/4241618440","heartbeat_back_addr":"172.21.15.155:6807/4241618440","heartbeat_front_addr":"172.21.15.155:6805/4241618440","state":["exists","up"]},{"osd":2,"uuid":"c69c8c19-32f8-400a-86d4-0f4ce240a7de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6801","nonce":2193296422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6803","nonce":2193296422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6807","nonce":2193296422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6805","nonce":2193296422}]},"public_addr":"172.21.15.184:6801/2193296422","cluster_addr":"172.21.15.184:6803/2193296422","heartbeat_back_addr":"172.21.15.184:6807/2193296422","heartbeat_front_addr":"172.21.15.184:6805/2193296422","state":["exists","up"]},{"osd":3,"uuid":"8892e35f-6880-434c-a805-ab8ccbed2bf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6811","nonce":81681815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6813","nonce":81681815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6817","nonce":81681815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6815","nonce":81681815}]},"public_addr":"172.21.15.16:6811/81681815","cluster_addr":"172.21.15.16:6813/81681815","heartbeat_back_addr":"172.21.15.16:6817/81681815","heartbeat_front_addr":"172.21.15.16:6815/81681815","state":["exists","up"]},{"osd":4,"uuid":"d2515e73-88f7-4654-9762-adb8802777ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6809","nonce":3592422019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6811","nonce":3592422019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6815","nonce":3592422019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6813","nonce":3592422019}]},"public_addr":"172.21.15.155:6809/3592422019","cluster_addr":"172.21.15.155:6811/3592422019","heartbeat_back_addr":"172.21.15.155:6815/3592422019","heartbeat_front_addr":"172.21.15.155:6813/3592422019","state":["exists","up"]},{"osd":5,"uuid":"5ce6e2fa-c17c-49b3-9402-f19bfc21076d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6809","nonce":2054894065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6811","nonce":2054894065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6815","nonce":2054894065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6813","nonce":2054894065}]},"public_addr":"172.21.15.184:6809/2054894065","cluster_addr":"172.21.15.184:6811/2054894065","heartbeat_back_addr":"172.21.15.184:6815/2054894065","heartbeat_front_addr":"172.21.15.184:6813/2054894065","state":["exists","up"]},{"osd":6,"uuid":"f257d8d1-ad59-4bd8-821d-bf29e50b547e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6819","nonce":1697421092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6821","nonce":1697421092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6825","nonce":1697421092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6823","nonce":1697421092}]},"public_addr":"172.21.15.16:6819/1697421092","cluster_addr":"172.21.15.16:6821/1697421092","heartbeat_back_addr":"172.21.15.16:6825/1697421092","heartbeat_front_addr":"172.21.15.16:6823/1697421092","state":["exists","up"]},{"osd":7,"uuid":"8e76c20e-8d25-423b-b7e8-de642d9c9a6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6817","nonce":2671827901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6819","nonce":2671827901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6823","nonce":2671827901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6821","nonce":2671827901}]},"public_addr":"172.21.15.155:6817/2671827901","cluster_addr":"172.21.15.155:6819/2671827901","heartbeat_back_addr":"172.21.15.155:6823/2671827901","heartbeat_front_addr":"172.21.15.155:6821/2671827901","state":["exists","up"]},{"osd":8,"uuid":"0f0bb23f-4bc3-4198-8381-5b5cd16ae394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6817","nonce":2479117823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6819","nonce":2479117823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6823","nonce":2479117823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6821","nonce":2479117823}]},"public_addr":"172.21.15.184:6817/2479117823","cluster_addr":"172.21.15.184:6819/2479117823","heartbeat_back_addr":"172.21.15.184:6823/2479117823","heartbeat_front_addr":"172.21.15.184:6821/2479117823","state":["exists","up"]},{"osd":9,"uuid":"bc27afcb-7871-4f72-a043-044fd5fe7d86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6827","nonce":3015700546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6829","nonce":3015700546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6833","nonce":3015700546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6831","nonce":3015700546}]},"public_addr":"172.21.15.16:6827/3015700546","cluster_addr":"172.21.15.16:6829/3015700546","heartbeat_back_addr":"172.21.15.16:6833/3015700546","heartbeat_front_addr":"172.21.15.16:6831/3015700546","state":["exists","up"]},{"osd":10,"uuid":"f2548997-72ee-400e-ab8d-5966d85b2cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6825","nonce":2427122475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6827","nonce":2427122475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6831","nonce":2427122475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6829","nonce":2427122475}]},"public_addr":"172.21.15.155:6825/2427122475","cluster_addr":"172.21.15.155:6827/2427122475","heartbeat_back_addr":"172.21.15.155:6831/2427122475","heartbeat_front_addr":"172.21.15.155:6829/2427122475","state":["exists","up"]},{"osd":11,"uuid":"8297da1d-d3e2-4a4d-9396-426dfafa3e95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6825","nonce":399265701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6827","nonce":399265701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6831","nonce":399265701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6829","nonce":399265701}]},"public_addr":"172.21.15.184:6825/399265701","cluster_addr":"172.21.15.184:6827/399265701","heartbeat_back_addr":"172.21.15.184:6831/399265701","heartbeat_front_addr":"172.21.15.184:6829/399265701","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:58:30.842360+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:58:57.164488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:59:20.778103+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:59:50.123991+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:00:16.789933+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:00:41.788905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:01:12.281376+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:01:39.211858+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:02:04.958771+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:02:35.645947+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:03:01.940433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:03:26.728673+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3923220960":"2024-09-01T05:57:23.153214+0000","172.21.15.16:0/3013533836":"2024-09-01T05:57:23.153214+0000","172.21.15.16:6800/3755683228":"2024-09-01T05:55:45.075902+0000","172.21.15.16:0/3064435470":"2024-09-01T05:55:45.075902+0000","172.21.15.16:0/911709962":"2024-09-01T05:55:45.075902+0000","172.21.15.16:0/2937368967":"2024-09-01T05:55:45.075902+0000","172.21.15.16:6800/1378972230":"2024-09-01T05:56:14.195704+0000","172.21.15.16:0/3047316995":"2024-09-01T05:56:14.195704+0000","172.21.15.16:6801/3755683228":"2024-09-01T05:55:45.075902+0000","172.21.15.16:6801/2571082634":"2024-09-01T05:57:23.153214+0000","172.21.15.16:0/2136704295":"2024-09-01T05:56:14.195704+0000","172.21.15.16:0/2259332778":"2024-09-01T05:56:14.195704+0000","172.21.15.16:6800/2571082634":"2024-09-01T05:57:23.153214+0000","172.21.15.16:6801/1378972230":"2024-09-01T05:56:14.195704+0000","172.21.15.16:0/2126457673":"2024-09-01T05:57:23.153214+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-31T06:05:10.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.432+0000 7f70db200640 1 -- 172.21.15.16:0/176350501 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f70c00b4c90 msgr2=0x7f70c00b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:10.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.432+0000 7f70db200640 1 --2- 172.21.15.16:0/176350501 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f70c00b4c90 0x7f70c00b71a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f70dc076f00 tx=0x7f70cc010040 comp rx=0 tx=0).stop 2024-08-31T06:05:10.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.432+0000 7f70db200640 1 -- 172.21.15.16:0/176350501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70dc109090 msgr2=0x7f70dc0759f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:10.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.432+0000 7f70db200640 1 --2- 172.21.15.16:0/176350501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70dc109090 0x7f70dc0759f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f70c4035040 tx=0x7f70c4032d50 comp rx=0 tx=0).stop 2024-08-31T06:05:10.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.432+0000 7f70db200640 1 -- 172.21.15.16:0/176350501 shutdown_connections 2024-08-31T06:05:10.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.432+0000 7f70db200640 1 --2- 172.21.15.16:0/176350501 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f70c00b4c90 0x7f70c00b71a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:10.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.432+0000 7f70db200640 1 --2- 172.21.15.16:0/176350501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70dc1099e0 0x7f70dc075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:10.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.432+0000 7f70db200640 1 --2- 172.21.15.16:0/176350501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70dc109090 0x7f70dc0759f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:10.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.432+0000 7f70db200640 1 --2- 172.21.15.16:0/176350501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70dc103750 0x7f70dc0754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:10.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.432+0000 7f70db200640 1 -- 172.21.15.16:0/176350501 >> 172.21.15.16:0/176350501 conn(0x7f70dc0ff1a0 msgr2=0x7f70dc1015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:10.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.432+0000 7f70db200640 1 -- 172.21.15.16:0/176350501 shutdown_connections 2024-08-31T06:05:10.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:10.432+0000 7f70db200640 1 -- 172.21.15.16:0/176350501 wait complete. 2024-08-31T06:05:11.177 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-08-31T06:05:11.178 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph osd dump --format=json 2024-08-31T06:05:11.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:11 smithi155 ceph-mon[21742]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-08-31T06:05:11.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:11 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/176350501' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T06:05:11.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:11 smithi016 ceph-mon[17194]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-08-31T06:05:11.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:11 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/176350501' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T06:05:11.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:11 smithi184 ceph-mon[22331]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-08-31T06:05:11.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:11 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/176350501' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T06:05:11.554 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:05:12.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.178+0000 7ff361200640 1 -- 172.21.15.16:0/3921059430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff35c105b20 msgr2=0x7ff35c105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:12.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.178+0000 7ff361200640 1 --2- 172.21.15.16:0/3921059430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff35c105b20 0x7ff35c105f30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff34c009900 tx=0x7ff34c02a410 comp rx=0 tx=0).stop 2024-08-31T06:05:12.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.178+0000 7ff361200640 1 -- 172.21.15.16:0/3921059430 shutdown_connections 2024-08-31T06:05:12.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.178+0000 7ff361200640 1 --2- 172.21.15.16:0/3921059430 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff35c10b460 0x7ff35c10b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:12.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.178+0000 7ff361200640 1 --2- 172.21.15.16:0/3921059430 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff35c10bdb0 0x7ff35c1125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:12.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.178+0000 7ff361200640 1 --2- 172.21.15.16:0/3921059430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff35c105b20 0x7ff35c105f30 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:12.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.178+0000 7ff361200640 1 -- 172.21.15.16:0/3921059430 >> 172.21.15.16:0/3921059430 conn(0x7ff35c1015b0 msgr2=0x7ff35c103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:12.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.178+0000 7ff361200640 1 -- 172.21.15.16:0/3921059430 shutdown_connections 2024-08-31T06:05:12.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.178+0000 7ff361200640 1 -- 172.21.15.16:0/3921059430 wait complete. 2024-08-31T06:05:12.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.179+0000 7ff361200640 1 Processor -- start 2024-08-31T06:05:12.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.179+0000 7ff361200640 1 -- start start 2024-08-31T06:05:12.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.179+0000 7ff361200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff35c105b20 0x7ff35c19c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:12.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.179+0000 7ff361200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff35c10b460 0x7ff35c19ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:12.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.179+0000 7ff361200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff35c10bdb0 0x7ff35c1a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:12.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.179+0000 7ff361200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff35c115a70 con 0x7ff35c10b460 2024-08-31T06:05:12.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.180+0000 7ff361200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff35c1158f0 con 0x7ff35c105b20 2024-08-31T06:05:12.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.180+0000 7ff361200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff35c115bf0 con 0x7ff35c10bdb0 2024-08-31T06:05:12.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.180+0000 7ff353e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff35c10b460 0x7ff35c19ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:12.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.180+0000 7ff359e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff35c105b20 0x7ff35c19c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:12.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.180+0000 7ff359e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff35c105b20 0x7ff35c19c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:40046/0 (socket says 172.21.15.16:40046) 2024-08-31T06:05:12.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.180+0000 7ff359e00640 1 -- 172.21.15.16:0/3520509087 learned_addr learned my addr 172.21.15.16:0/3520509087 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:05:12.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.180+0000 7ff35a800640 1 --2- 172.21.15.16:0/3520509087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff35c10bdb0 0x7ff35c1a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:12.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.180+0000 7ff353e00640 1 -- 172.21.15.16:0/3520509087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff35c10bdb0 msgr2=0x7ff35c1a1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:12.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.180+0000 7ff353e00640 1 --2- 172.21.15.16:0/3520509087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff35c10bdb0 0x7ff35c1a1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:12.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.180+0000 7ff353e00640 1 -- 172.21.15.16:0/3520509087 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff35c105b20 msgr2=0x7ff35c19c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:12.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.180+0000 7ff353e00640 1 --2- 172.21.15.16:0/3520509087 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff35c105b20 0x7ff35c19c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:12.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.180+0000 7ff353e00640 1 -- 172.21.15.16:0/3520509087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff35c1a23d0 con 0x7ff35c10b460 2024-08-31T06:05:12.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.180+0000 7ff35a800640 1 --2- 172.21.15.16:0/3520509087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff35c10bdb0 0x7ff35c1a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:05:12.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.181+0000 7ff353e00640 1 --2- 172.21.15.16:0/3520509087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff35c10b460 0x7ff35c19ca00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff35c06c840 tx=0x7ff34400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:12.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.181+0000 7ff352a00640 1 -- 172.21.15.16:0/3520509087 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff34400cc40 con 0x7ff35c10b460 2024-08-31T06:05:12.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.181+0000 7ff352a00640 1 -- 172.21.15.16:0/3520509087 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff34400cde0 con 0x7ff35c10b460 2024-08-31T06:05:12.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.181+0000 7ff352a00640 1 -- 172.21.15.16:0/3520509087 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff344004a00 con 0x7ff35c10b460 2024-08-31T06:05:12.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.181+0000 7ff361200640 1 -- 172.21.15.16:0/3520509087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff35c1a26c0 con 0x7ff35c10b460 2024-08-31T06:05:12.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.181+0000 7ff361200640 1 -- 172.21.15.16:0/3520509087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff35c1a8e20 con 0x7ff35c10b460 2024-08-31T06:05:12.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.182+0000 7ff361200640 1 -- 172.21.15.16:0/3520509087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff35c105f30 con 0x7ff35c10b460 2024-08-31T06:05:12.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.184+0000 7ff352a00640 1 -- 172.21.15.16:0/3520509087 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7ff344004ba0 con 0x7ff35c10b460 2024-08-31T06:05:12.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.187+0000 7ff352a00640 1 --2- 172.21.15.16:0/3520509087 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff3400b4c90 0x7ff3400b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:12.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.187+0000 7ff352a00640 1 -- 172.21.15.16:0/3520509087 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7ff3440e5270 con 0x7ff35c10b460 2024-08-31T06:05:12.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.187+0000 7ff359e00640 1 --2- 172.21.15.16:0/3520509087 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff3400b4c90 0x7ff3400b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:12.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.188+0000 7ff359e00640 1 --2- 172.21.15.16:0/3520509087 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff3400b4c90 0x7ff3400b71a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff34c031440 tx=0x7ff34c0058d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:12.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.188+0000 7ff352a00640 1 -- 172.21.15.16:0/3520509087 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7ff3440ac2f0 con 0x7ff35c10b460 2024-08-31T06:05:12.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.385+0000 7ff361200640 1 -- 172.21.15.16:0/3520509087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff35c19d590 con 0x7ff35c10b460 2024-08-31T06:05:12.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.387+0000 7ff352a00640 1 -- 172.21.15.16:0/3520509087 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22608 (secure 0 0 0) 0x7ff3440abbf0 con 0x7ff35c10b460 2024-08-31T06:05:12.389 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:05:12.390 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":78,"fsid":"673e4aa0-675d-11ef-bcd4-c7b262605968","created":"2024-08-31T05:55:09.632536+0000","modified":"2024-08-31T06:04:27.098694+0000","last_up_change":"2024-08-31T06:03:27.893697+0000","last_in_change":"2024-08-31T06:03:05.664945+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-31T05:59:23.642270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-31T06:04:20.871835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-31T06:04:24.119499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e349dae-6aa5-4b6d-9107-a945655e11bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6803","nonce":1725037400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6805","nonce":1725037400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6809","nonce":1725037400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6807","nonce":1725037400}]},"public_addr":"172.21.15.16:6803/1725037400","cluster_addr":"172.21.15.16:6805/1725037400","heartbeat_back_addr":"172.21.15.16:6809/1725037400","heartbeat_front_addr":"172.21.15.16:6807/1725037400","state":["exists","up"]},{"osd":1,"uuid":"bd645dc3-93b5-40af-89d8-08ccec22e50d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6801","nonce":4241618440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6803","nonce":4241618440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6807","nonce":4241618440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6805","nonce":4241618440}]},"public_addr":"172.21.15.155:6801/4241618440","cluster_addr":"172.21.15.155:6803/4241618440","heartbeat_back_addr":"172.21.15.155:6807/4241618440","heartbeat_front_addr":"172.21.15.155:6805/4241618440","state":["exists","up"]},{"osd":2,"uuid":"c69c8c19-32f8-400a-86d4-0f4ce240a7de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6801","nonce":2193296422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6803","nonce":2193296422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6807","nonce":2193296422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6805","nonce":2193296422}]},"public_addr":"172.21.15.184:6801/2193296422","cluster_addr":"172.21.15.184:6803/2193296422","heartbeat_back_addr":"172.21.15.184:6807/2193296422","heartbeat_front_addr":"172.21.15.184:6805/2193296422","state":["exists","up"]},{"osd":3,"uuid":"8892e35f-6880-434c-a805-ab8ccbed2bf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6811","nonce":81681815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6813","nonce":81681815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6817","nonce":81681815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6815","nonce":81681815}]},"public_addr":"172.21.15.16:6811/81681815","cluster_addr":"172.21.15.16:6813/81681815","heartbeat_back_addr":"172.21.15.16:6817/81681815","heartbeat_front_addr":"172.21.15.16:6815/81681815","state":["exists","up"]},{"osd":4,"uuid":"d2515e73-88f7-4654-9762-adb8802777ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6809","nonce":3592422019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6811","nonce":3592422019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6815","nonce":3592422019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6813","nonce":3592422019}]},"public_addr":"172.21.15.155:6809/3592422019","cluster_addr":"172.21.15.155:6811/3592422019","heartbeat_back_addr":"172.21.15.155:6815/3592422019","heartbeat_front_addr":"172.21.15.155:6813/3592422019","state":["exists","up"]},{"osd":5,"uuid":"5ce6e2fa-c17c-49b3-9402-f19bfc21076d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6809","nonce":2054894065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6811","nonce":2054894065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6815","nonce":2054894065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6813","nonce":2054894065}]},"public_addr":"172.21.15.184:6809/2054894065","cluster_addr":"172.21.15.184:6811/2054894065","heartbeat_back_addr":"172.21.15.184:6815/2054894065","heartbeat_front_addr":"172.21.15.184:6813/2054894065","state":["exists","up"]},{"osd":6,"uuid":"f257d8d1-ad59-4bd8-821d-bf29e50b547e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6819","nonce":1697421092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6821","nonce":1697421092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6825","nonce":1697421092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6823","nonce":1697421092}]},"public_addr":"172.21.15.16:6819/1697421092","cluster_addr":"172.21.15.16:6821/1697421092","heartbeat_back_addr":"172.21.15.16:6825/1697421092","heartbeat_front_addr":"172.21.15.16:6823/1697421092","state":["exists","up"]},{"osd":7,"uuid":"8e76c20e-8d25-423b-b7e8-de642d9c9a6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6817","nonce":2671827901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6819","nonce":2671827901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6823","nonce":2671827901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6821","nonce":2671827901}]},"public_addr":"172.21.15.155:6817/2671827901","cluster_addr":"172.21.15.155:6819/2671827901","heartbeat_back_addr":"172.21.15.155:6823/2671827901","heartbeat_front_addr":"172.21.15.155:6821/2671827901","state":["exists","up"]},{"osd":8,"uuid":"0f0bb23f-4bc3-4198-8381-5b5cd16ae394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6817","nonce":2479117823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6819","nonce":2479117823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6823","nonce":2479117823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6821","nonce":2479117823}]},"public_addr":"172.21.15.184:6817/2479117823","cluster_addr":"172.21.15.184:6819/2479117823","heartbeat_back_addr":"172.21.15.184:6823/2479117823","heartbeat_front_addr":"172.21.15.184:6821/2479117823","state":["exists","up"]},{"osd":9,"uuid":"bc27afcb-7871-4f72-a043-044fd5fe7d86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6827","nonce":3015700546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6829","nonce":3015700546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6833","nonce":3015700546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6831","nonce":3015700546}]},"public_addr":"172.21.15.16:6827/3015700546","cluster_addr":"172.21.15.16:6829/3015700546","heartbeat_back_addr":"172.21.15.16:6833/3015700546","heartbeat_front_addr":"172.21.15.16:6831/3015700546","state":["exists","up"]},{"osd":10,"uuid":"f2548997-72ee-400e-ab8d-5966d85b2cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6825","nonce":2427122475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6827","nonce":2427122475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6831","nonce":2427122475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6829","nonce":2427122475}]},"public_addr":"172.21.15.155:6825/2427122475","cluster_addr":"172.21.15.155:6827/2427122475","heartbeat_back_addr":"172.21.15.155:6831/2427122475","heartbeat_front_addr":"172.21.15.155:6829/2427122475","state":["exists","up"]},{"osd":11,"uuid":"8297da1d-d3e2-4a4d-9396-426dfafa3e95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6825","nonce":399265701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6827","nonce":399265701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6831","nonce":399265701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6829","nonce":399265701}]},"public_addr":"172.21.15.184:6825/399265701","cluster_addr":"172.21.15.184:6827/399265701","heartbeat_back_addr":"172.21.15.184:6831/399265701","heartbeat_front_addr":"172.21.15.184:6829/399265701","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:58:30.842360+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:58:57.164488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:59:20.778103+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:59:50.123991+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:00:16.789933+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:00:41.788905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:01:12.281376+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:01:39.211858+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:02:04.958771+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:02:35.645947+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:03:01.940433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:03:26.728673+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3923220960":"2024-09-01T05:57:23.153214+0000","172.21.15.16:0/3013533836":"2024-09-01T05:57:23.153214+0000","172.21.15.16:6800/3755683228":"2024-09-01T05:55:45.075902+0000","172.21.15.16:0/3064435470":"2024-09-01T05:55:45.075902+0000","172.21.15.16:0/911709962":"2024-09-01T05:55:45.075902+0000","172.21.15.16:0/2937368967":"2024-09-01T05:55:45.075902+0000","172.21.15.16:6800/1378972230":"2024-09-01T05:56:14.195704+0000","172.21.15.16:0/3047316995":"2024-09-01T05:56:14.195704+0000","172.21.15.16:6801/3755683228":"2024-09-01T05:55:45.075902+0000","172.21.15.16:6801/2571082634":"2024-09-01T05:57:23.153214+0000","172.21.15.16:0/2136704295":"2024-09-01T05:56:14.195704+0000","172.21.15.16:0/2259332778":"2024-09-01T05:56:14.195704+0000","172.21.15.16:6800/2571082634":"2024-09-01T05:57:23.153214+0000","172.21.15.16:6801/1378972230":"2024-09-01T05:56:14.195704+0000","172.21.15.16:0/2126457673":"2024-09-01T05:57:23.153214+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-31T06:05:12.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.390+0000 7ff361200640 1 -- 172.21.15.16:0/3520509087 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff3400b4c90 msgr2=0x7ff3400b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:12.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.390+0000 7ff361200640 1 --2- 172.21.15.16:0/3520509087 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff3400b4c90 0x7ff3400b71a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff34c031440 tx=0x7ff34c0058d0 comp rx=0 tx=0).stop 2024-08-31T06:05:12.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.390+0000 7ff361200640 1 -- 172.21.15.16:0/3520509087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff35c10b460 msgr2=0x7ff35c19ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:12.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.390+0000 7ff361200640 1 --2- 172.21.15.16:0/3520509087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff35c10b460 0x7ff35c19ca00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff35c06c840 tx=0x7ff34400ee50 comp rx=0 tx=0).stop 2024-08-31T06:05:12.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.391+0000 7ff361200640 1 -- 172.21.15.16:0/3520509087 shutdown_connections 2024-08-31T06:05:12.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.391+0000 7ff361200640 1 --2- 172.21.15.16:0/3520509087 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff3400b4c90 0x7ff3400b71a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:12.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.391+0000 7ff361200640 1 --2- 172.21.15.16:0/3520509087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff35c10bdb0 0x7ff35c1a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:12.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.391+0000 7ff361200640 1 --2- 172.21.15.16:0/3520509087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff35c10b460 0x7ff35c19ca00 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:12.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.391+0000 7ff361200640 1 --2- 172.21.15.16:0/3520509087 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff35c105b20 0x7ff35c19c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:12.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.391+0000 7ff361200640 1 -- 172.21.15.16:0/3520509087 >> 172.21.15.16:0/3520509087 conn(0x7ff35c1015b0 msgr2=0x7ff35c1039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:12.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.391+0000 7ff361200640 1 -- 172.21.15.16:0/3520509087 shutdown_connections 2024-08-31T06:05:12.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:12.391+0000 7ff361200640 1 -- 172.21.15.16:0/3520509087 wait complete. 2024-08-31T06:05:13.127 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-08-31T06:05:13.128 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-08-31T06:05:13.128 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-08-31T06:05:13.128 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-08-31T06:05:13.129 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-08-31T06:05:13.129 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-08-31T06:05:13.130 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-08-31T06:05:13.130 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-08-31T06:05:13.131 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-08-31T06:05:13.131 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-08-31T06:05:13.131 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-08-31T06:05:13.132 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-08-31T06:05:13.280 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:13 smithi016 ceph-mon[17194]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-08-31T06:05:13.280 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:13 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/3520509087' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T06:05:13.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:13 smithi155 ceph-mon[21742]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-08-31T06:05:13.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:13 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/3520509087' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T06:05:13.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:13 smithi184 ceph-mon[22331]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-08-31T06:05:13.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:13 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/3520509087' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T06:05:14.204 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:05:14.240 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:05:14.242 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:05:14.269 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:05:14.271 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:05:14.273 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:05:14.274 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:05:14.277 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:05:14.279 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:05:14.299 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:05:14.338 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:05:15.250 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:15 smithi016 ceph-mon[17194]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-08-31T06:05:15.267 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:05:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.328+0000 7f6c77600640 1 -- 172.21.15.16:0/1642930205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c70109ba0 msgr2=0x7f6c701103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.328+0000 7f6c77600640 1 --2- 172.21.15.16:0/1642930205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c70109ba0 0x7f6c701103d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6c68009ca0 tx=0x7f6c6802a8c0 comp rx=0 tx=0).stop 2024-08-31T06:05:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.328+0000 7f6c77600640 1 -- 172.21.15.16:0/1642930205 shutdown_connections 2024-08-31T06:05:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.328+0000 7f6c77600640 1 --2- 172.21.15.16:0/1642930205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c70109ba0 0x7f6c701103d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.328+0000 7f6c77600640 1 --2- 172.21.15.16:0/1642930205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c70109250 0x7f6c70109660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.328+0000 7f6c77600640 1 --2- 172.21.15.16:0/1642930205 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c70103910 0x7f6c70103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.328+0000 7f6c77600640 1 -- 172.21.15.16:0/1642930205 >> 172.21.15.16:0/1642930205 conn(0x7f6c700ff380 msgr2=0x7f6c701017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.328+0000 7f6c77600640 1 -- 172.21.15.16:0/1642930205 shutdown_connections 2024-08-31T06:05:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.328+0000 7f6c77600640 1 -- 172.21.15.16:0/1642930205 wait complete. 2024-08-31T06:05:15.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.329+0000 7f6c77600640 1 Processor -- start 2024-08-31T06:05:15.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.329+0000 7f6c77600640 1 -- start start 2024-08-31T06:05:15.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.330+0000 7f6c77600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c70103910 0x7f6c7019a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:15.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.330+0000 7f6c77600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c70109250 0x7f6c7019a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:15.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.330+0000 7f6c77600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c70109ba0 0x7f6c7019fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:15.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.330+0000 7f6c77600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6c70113860 con 0x7f6c70109ba0 2024-08-31T06:05:15.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.330+0000 7f6c77600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6c701136e0 con 0x7f6c70109250 2024-08-31T06:05:15.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.330+0000 7f6c77600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6c701139e0 con 0x7f6c70103910 2024-08-31T06:05:15.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.330+0000 7f6c75600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c70109ba0 0x7f6c7019fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:15.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.330+0000 7f6c6fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c70109250 0x7f6c7019a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:15.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.330+0000 7f6c75600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c70109ba0 0x7f6c7019fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41438/0 (socket says 172.21.15.16:41438) 2024-08-31T06:05:15.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.330+0000 7f6c75600640 1 -- 172.21.15.16:0/3444004862 learned_addr learned my addr 172.21.15.16:0/3444004862 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:05:15.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.330+0000 7f6c6fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c70109250 0x7f6c7019a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:40066/0 (socket says 172.21.15.16:40066) 2024-08-31T06:05:15.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.330+0000 7f6c74c00640 1 --2- 172.21.15.16:0/3444004862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c70103910 0x7f6c7019a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:15.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.331+0000 7f6c6fe00640 1 -- 172.21.15.16:0/3444004862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c70103910 msgr2=0x7f6c7019a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:15.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.331+0000 7f6c6fe00640 1 --2- 172.21.15.16:0/3444004862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c70103910 0x7f6c7019a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:15.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.331+0000 7f6c6fe00640 1 -- 172.21.15.16:0/3444004862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c70109ba0 msgr2=0x7f6c7019fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:15.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.331+0000 7f6c6fe00640 1 --2- 172.21.15.16:0/3444004862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c70109ba0 0x7f6c7019fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:15.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.331+0000 7f6c6fe00640 1 -- 172.21.15.16:0/3444004862 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c701a0160 con 0x7f6c70109250 2024-08-31T06:05:15.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.331+0000 7f6c75600640 1 --2- 172.21.15.16:0/3444004862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c70109ba0 0x7f6c7019fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:05:15.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.331+0000 7f6c6fe00640 1 --2- 172.21.15.16:0/3444004862 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c70109250 0x7f6c7019a790 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6c7006bae0 tx=0x7f6c5c00ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:15.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.331+0000 7f6c6d600640 1 -- 172.21.15.16:0/3444004862 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c00cc40 con 0x7f6c70109250 2024-08-31T06:05:15.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.331+0000 7f6c77600640 1 -- 172.21.15.16:0/3444004862 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c701a0450 con 0x7f6c70109250 2024-08-31T06:05:15.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.331+0000 7f6c6d600640 1 -- 172.21.15.16:0/3444004862 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c5c00cde0 con 0x7f6c70109250 2024-08-31T06:05:15.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.331+0000 7f6c6d600640 1 -- 172.21.15.16:0/3444004862 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c018760 con 0x7f6c70109250 2024-08-31T06:05:15.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.331+0000 7f6c77600640 1 -- 172.21.15.16:0/3444004862 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c701a6bb0 con 0x7f6c70109250 2024-08-31T06:05:15.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.332+0000 7f6c77600640 1 -- 172.21.15.16:0/3444004862 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6c700638b0 con 0x7f6c70109250 2024-08-31T06:05:15.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.335+0000 7f6c6d600640 1 -- 172.21.15.16:0/3444004862 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f6c5c010a90 con 0x7f6c70109250 2024-08-31T06:05:15.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.336+0000 7f6c6d600640 1 --2- 172.21.15.16:0/3444004862 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6c580b4c10 0x7f6c580b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:15.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.336+0000 7f6c6d600640 1 -- 172.21.15.16:0/3444004862 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7f6c5c0e5e70 con 0x7f6c70109250 2024-08-31T06:05:15.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.337+0000 7f6c74c00640 1 --2- 172.21.15.16:0/3444004862 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6c580b4c10 0x7f6c580b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:15.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.337+0000 7f6c6d600640 1 --2- 172.21.15.16:0/3444004862 >> [v2:172.21.15.16:6826/3015700546,v1:172.21.15.16:6827/3015700546] conn(0x7f6c580be620 0x7f6c580c0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:15.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.337+0000 7f6c6d600640 1 -- 172.21.15.16:0/3444004862 --> [v2:172.21.15.16:6826/3015700546,v1:172.21.15.16:6827/3015700546] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6c5c00be50 con 0x7f6c580be620 2024-08-31T06:05:15.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.337+0000 7f6c6d600640 1 -- 172.21.15.16:0/3444004862 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f6c5c0e6220 con 0x7f6c70109250 2024-08-31T06:05:15.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.337+0000 7f6c75600640 1 --2- 172.21.15.16:0/3444004862 >> [v2:172.21.15.16:6826/3015700546,v1:172.21.15.16:6827/3015700546] conn(0x7f6c580be620 0x7f6c580c0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:15.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.337+0000 7f6c74c00640 1 --2- 172.21.15.16:0/3444004862 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6c580b4c10 0x7f6c580b7120 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6c64009bd0 tx=0x7f6c64005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:15.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.338+0000 7f6c75600640 1 --2- 172.21.15.16:0/3444004862 >> [v2:172.21.15.16:6826/3015700546,v1:172.21.15.16:6827/3015700546] conn(0x7f6c580be620 0x7f6c580c0a70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:15.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.339+0000 7f6c6d600640 1 -- 172.21.15.16:0/3444004862 <== osd.9 v2:172.21.15.16:6826/3015700546 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6c5c00be50 con 0x7f6c580be620 2024-08-31T06:05:15.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.353+0000 7f6c77600640 1 -- 172.21.15.16:0/3444004862 --> [v2:172.21.15.16:6826/3015700546,v1:172.21.15.16:6827/3015700546] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6c70063cb0 con 0x7f6c580be620 2024-08-31T06:05:15.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.354+0000 7f6c6d600640 1 -- 172.21.15.16:0/3444004862 <== osd.9 v2:172.21.15.16:6826/3015700546 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6c70063cb0 con 0x7f6c580be620 2024-08-31T06:05:15.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.354+0000 7f6c77600640 1 -- 172.21.15.16:0/3444004862 >> [v2:172.21.15.16:6826/3015700546,v1:172.21.15.16:6827/3015700546] conn(0x7f6c580be620 msgr2=0x7f6c580c0a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:15.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.354+0000 7f6c77600640 1 --2- 172.21.15.16:0/3444004862 >> [v2:172.21.15.16:6826/3015700546,v1:172.21.15.16:6827/3015700546] conn(0x7f6c580be620 0x7f6c580c0a70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:15.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.354+0000 7f6c77600640 1 -- 172.21.15.16:0/3444004862 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6c580b4c10 msgr2=0x7f6c580b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:15.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.354+0000 7f6c77600640 1 --2- 172.21.15.16:0/3444004862 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6c580b4c10 0x7f6c580b7120 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6c64009bd0 tx=0x7f6c64005f90 comp rx=0 tx=0).stop 2024-08-31T06:05:15.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.354+0000 7f6c77600640 1 -- 172.21.15.16:0/3444004862 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c70109250 msgr2=0x7f6c7019a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:15.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.354+0000 7f6c77600640 1 --2- 172.21.15.16:0/3444004862 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c70109250 0x7f6c7019a790 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6c7006bae0 tx=0x7f6c5c00ee50 comp rx=0 tx=0).stop 2024-08-31T06:05:15.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.354+0000 7f6c75600640 1 -- 172.21.15.16:0/3444004862 reap_dead start 2024-08-31T06:05:15.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.354+0000 7f6c77600640 1 -- 172.21.15.16:0/3444004862 shutdown_connections 2024-08-31T06:05:15.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.354+0000 7f6c77600640 1 -- 172.21.15.16:0/3444004862 >> 172.21.15.16:0/3444004862 conn(0x7f6c700ff380 msgr2=0x7f6c70106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:15.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.354+0000 7f6c77600640 1 -- 172.21.15.16:0/3444004862 shutdown_connections 2024-08-31T06:05:15.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:15.354+0000 7f6c77600640 1 -- 172.21.15.16:0/3444004862 wait complete. 2024-08-31T06:05:15.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:15 smithi155 ceph-mon[21742]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-08-31T06:05:15.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:15 smithi184 ceph-mon[22331]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-08-31T06:05:16.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.289+0000 7fb1c2200640 1 -- 172.21.15.16:0/2106763583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1bc105b20 msgr2=0x7fb1bc105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:16.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.289+0000 7fb1c2200640 1 --2- 172.21.15.16:0/2106763583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1bc105b20 0x7fb1bc105f30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac009900 tx=0x7fb1ac02a410 comp rx=0 tx=0).stop 2024-08-31T06:05:16.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.289+0000 7fb1c2200640 1 -- 172.21.15.16:0/2106763583 shutdown_connections 2024-08-31T06:05:16.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.289+0000 7fb1c2200640 1 --2- 172.21.15.16:0/2106763583 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1bc10bdb0 0x7fb1bc1125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:16.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.289+0000 7fb1c2200640 1 --2- 172.21.15.16:0/2106763583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1bc10b460 0x7fb1bc10b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:16.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.289+0000 7fb1c2200640 1 --2- 172.21.15.16:0/2106763583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1bc105b20 0x7fb1bc105f30 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:16.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.289+0000 7fb1c2200640 1 -- 172.21.15.16:0/2106763583 >> 172.21.15.16:0/2106763583 conn(0x7fb1bc1015b0 msgr2=0x7fb1bc103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:16.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.289+0000 7fb1c2200640 1 -- 172.21.15.16:0/2106763583 shutdown_connections 2024-08-31T06:05:16.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.289+0000 7fb1c2200640 1 -- 172.21.15.16:0/2106763583 wait complete. 2024-08-31T06:05:16.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.290+0000 7fb1c2200640 1 Processor -- start 2024-08-31T06:05:16.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.290+0000 7fb1c2200640 1 -- start start 2024-08-31T06:05:16.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.291+0000 7fb1c2200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1bc105b20 0x7fb1bc19c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:16.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.291+0000 7fb1c2200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1bc10b460 0x7fb1bc19ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:16.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.291+0000 7fb1c2200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1bc10bdb0 0x7fb1bc1a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:16.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.291+0000 7fb1c2200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb1bc115ac0 con 0x7fb1bc10bdb0 2024-08-31T06:05:16.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.291+0000 7fb1c2200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb1bc115940 con 0x7fb1bc10b460 2024-08-31T06:05:16.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.291+0000 7fb1c2200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb1bc115c40 con 0x7fb1bc105b20 2024-08-31T06:05:16.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.291+0000 7fb1bb400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1bc105b20 0x7fb1bc19c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:16.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.291+0000 7fb1b2a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1bc10b460 0x7fb1bc19ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:16.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.291+0000 7fb1bbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1bc10bdb0 0x7fb1bc1a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:16.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.291+0000 7fb1b2a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1bc10b460 0x7fb1bc19ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:52398/0 (socket says 172.21.15.16:52398) 2024-08-31T06:05:16.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.291+0000 7fb1bb400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1bc105b20 0x7fb1bc19c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:43376/0 (socket says 172.21.15.16:43376) 2024-08-31T06:05:16.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.291+0000 7fb1b2a00640 1 -- 172.21.15.16:0/3021730081 learned_addr learned my addr 172.21.15.16:0/3021730081 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:05:16.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.292+0000 7fb1bb400640 1 -- 172.21.15.16:0/3021730081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1bc10b460 msgr2=0x7fb1bc19ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:16.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.292+0000 7fb1bb400640 1 --2- 172.21.15.16:0/3021730081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1bc10b460 0x7fb1bc19ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:16.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.292+0000 7fb1bb400640 1 -- 172.21.15.16:0/3021730081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1bc10bdb0 msgr2=0x7fb1bc1a1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:16.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.292+0000 7fb1bb400640 1 --2- 172.21.15.16:0/3021730081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1bc10bdb0 0x7fb1bc1a1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:16.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.292+0000 7fb1bb400640 1 -- 172.21.15.16:0/3021730081 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1bc1a23e0 con 0x7fb1bc105b20 2024-08-31T06:05:16.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.292+0000 7fb1bbe00640 1 --2- 172.21.15.16:0/3021730081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1bc10bdb0 0x7fb1bc1a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:05:16.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.292+0000 7fb1b2a00640 1 --2- 172.21.15.16:0/3021730081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1bc10b460 0x7fb1bc19ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:05:16.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.292+0000 7fb1bb400640 1 --2- 172.21.15.16:0/3021730081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1bc105b20 0x7fb1bc19c470 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac02aa00 tx=0x7fb1ac003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:16.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.292+0000 7fb1b8c00640 1 -- 172.21.15.16:0/3021730081 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac03f070 con 0x7fb1bc105b20 2024-08-31T06:05:16.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.292+0000 7fb1b8c00640 1 -- 172.21.15.16:0/3021730081 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1ac031b50 con 0x7fb1bc105b20 2024-08-31T06:05:16.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.292+0000 7fb1c2200640 1 -- 172.21.15.16:0/3021730081 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1bc1a2670 con 0x7fb1bc105b20 2024-08-31T06:05:16.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.292+0000 7fb1b8c00640 1 -- 172.21.15.16:0/3021730081 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac035030 con 0x7fb1bc105b20 2024-08-31T06:05:16.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.293+0000 7fb1c2200640 1 -- 172.21.15.16:0/3021730081 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1bc1a8e30 con 0x7fb1bc105b20 2024-08-31T06:05:16.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.293+0000 7fb1c2200640 1 -- 172.21.15.16:0/3021730081 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb1bc0638f0 con 0x7fb1bc105b20 2024-08-31T06:05:16.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.296+0000 7fb1b8c00640 1 -- 172.21.15.16:0/3021730081 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7fb1ac042070 con 0x7fb1bc105b20 2024-08-31T06:05:16.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.298+0000 7fb1b8c00640 1 --2- 172.21.15.16:0/3021730081 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb1a00b4c10 0x7fb1a00b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:16.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.298+0000 7fb1b8c00640 1 -- 172.21.15.16:0/3021730081 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7fb1ac106370 con 0x7fb1bc105b20 2024-08-31T06:05:16.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.298+0000 7fb1b2a00640 1 --2- 172.21.15.16:0/3021730081 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb1a00b4c10 0x7fb1a00b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:16.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.298+0000 7fb1b8c00640 1 --2- 172.21.15.16:0/3021730081 >> [v2:172.21.15.16:6818/1697421092,v1:172.21.15.16:6819/1697421092] conn(0x7fb1a00be620 0x7fb1a00c0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:16.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.298+0000 7fb1b8c00640 1 -- 172.21.15.16:0/3021730081 --> [v2:172.21.15.16:6818/1697421092,v1:172.21.15.16:6819/1697421092] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb1a00c1140 con 0x7fb1a00be620 2024-08-31T06:05:16.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.298+0000 7fb1b8c00640 1 -- 172.21.15.16:0/3021730081 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7fb1ac106720 con 0x7fb1bc105b20 2024-08-31T06:05:16.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.298+0000 7fb1bbe00640 1 --2- 172.21.15.16:0/3021730081 >> [v2:172.21.15.16:6818/1697421092,v1:172.21.15.16:6819/1697421092] conn(0x7fb1a00be620 0x7fb1a00c0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:16.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.299+0000 7fb1b2a00640 1 --2- 172.21.15.16:0/3021730081 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb1a00b4c10 0x7fb1a00b7120 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc104e00 tx=0x7fb1a400b440 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:16.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.299+0000 7fb1bbe00640 1 --2- 172.21.15.16:0/3021730081 >> [v2:172.21.15.16:6818/1697421092,v1:172.21.15.16:6819/1697421092] conn(0x7fb1a00be620 0x7fb1a00c0a70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:16.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.300+0000 7fb1b8c00640 1 -- 172.21.15.16:0/3021730081 <== osd.6 v2:172.21.15.16:6818/1697421092 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb1a00c1140 con 0x7fb1a00be620 2024-08-31T06:05:16.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.322+0000 7fb1c2200640 1 -- 172.21.15.16:0/3021730081 --> [v2:172.21.15.16:6818/1697421092,v1:172.21.15.16:6819/1697421092] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb1bc063cf0 con 0x7fb1a00be620 2024-08-31T06:05:16.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.323+0000 7fb1b8c00640 1 -- 172.21.15.16:0/3021730081 <== osd.6 v2:172.21.15.16:6818/1697421092 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb1bc063cf0 con 0x7fb1a00be620 2024-08-31T06:05:16.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.323+0000 7fb1c2200640 1 -- 172.21.15.16:0/3021730081 >> [v2:172.21.15.16:6818/1697421092,v1:172.21.15.16:6819/1697421092] conn(0x7fb1a00be620 msgr2=0x7fb1a00c0a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:16.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.323+0000 7fb1c2200640 1 --2- 172.21.15.16:0/3021730081 >> [v2:172.21.15.16:6818/1697421092,v1:172.21.15.16:6819/1697421092] conn(0x7fb1a00be620 0x7fb1a00c0a70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:16.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.324+0000 7fb1c2200640 1 -- 172.21.15.16:0/3021730081 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb1a00b4c10 msgr2=0x7fb1a00b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:16.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.324+0000 7fb1c2200640 1 --2- 172.21.15.16:0/3021730081 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb1a00b4c10 0x7fb1a00b7120 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc104e00 tx=0x7fb1a400b440 comp rx=0 tx=0).stop 2024-08-31T06:05:16.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.324+0000 7fb1c2200640 1 -- 172.21.15.16:0/3021730081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1bc105b20 msgr2=0x7fb1bc19c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:16.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.324+0000 7fb1c2200640 1 --2- 172.21.15.16:0/3021730081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1bc105b20 0x7fb1bc19c470 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac02aa00 tx=0x7fb1ac003b90 comp rx=0 tx=0).stop 2024-08-31T06:05:16.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.324+0000 7fb1bbe00640 1 -- 172.21.15.16:0/3021730081 reap_dead start 2024-08-31T06:05:16.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.324+0000 7fb1c2200640 1 -- 172.21.15.16:0/3021730081 shutdown_connections 2024-08-31T06:05:16.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.324+0000 7fb1c2200640 1 -- 172.21.15.16:0/3021730081 >> 172.21.15.16:0/3021730081 conn(0x7fb1bc1015b0 msgr2=0x7fb1bc10aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:16.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.324+0000 7fb1c2200640 1 -- 172.21.15.16:0/3021730081 shutdown_connections 2024-08-31T06:05:16.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.324+0000 7fb1c2200640 1 -- 172.21.15.16:0/3021730081 wait complete. 2024-08-31T06:05:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.155+0000 7fafde200640 1 -- 172.21.15.16:0/2578391687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd80754f0 msgr2=0x7fafd8075900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.155+0000 7fafde200640 1 --2- 172.21.15.16:0/2578391687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd80754f0 0x7fafd8075900 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fafc8009900 tx=0x7fafc802a3f0 comp rx=0 tx=0).stop 2024-08-31T06:05:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.155+0000 7fafde200640 1 -- 172.21.15.16:0/2578391687 shutdown_connections 2024-08-31T06:05:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.155+0000 7fafde200640 1 --2- 172.21.15.16:0/2578391687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafd810fec0 0x7fafd81122e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.155+0000 7fafde200640 1 --2- 172.21.15.16:0/2578391687 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafd8075e40 0x7fafd810f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.155+0000 7fafde200640 1 --2- 172.21.15.16:0/2578391687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd80754f0 0x7fafd8075900 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.155+0000 7fafde200640 1 -- 172.21.15.16:0/2578391687 >> 172.21.15.16:0/2578391687 conn(0x7fafd807b5f0 msgr2=0x7fafd807ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.155+0000 7fafde200640 1 -- 172.21.15.16:0/2578391687 shutdown_connections 2024-08-31T06:05:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.155+0000 7fafde200640 1 -- 172.21.15.16:0/2578391687 wait complete. 2024-08-31T06:05:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.157+0000 7fafde200640 1 Processor -- start 2024-08-31T06:05:16.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.157+0000 7fafde200640 1 -- start start 2024-08-31T06:05:16.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.157+0000 7fafde200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafd80754f0 0x7fafd819c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:16.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.157+0000 7fafde200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafd8075e40 0x7fafd819c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:16.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.158+0000 7fafde200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd810fec0 0x7fafd81a1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:16.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.158+0000 7fafde200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fafd8115a30 con 0x7fafd810fec0 2024-08-31T06:05:16.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.158+0000 7fafde200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fafd81158b0 con 0x7fafd80754f0 2024-08-31T06:05:16.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.158+0000 7fafde200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fafd8115bb0 con 0x7fafd8075e40 2024-08-31T06:05:16.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.158+0000 7fafd7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd810fec0 0x7fafd81a1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:16.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.158+0000 7fafd7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd810fec0 0x7fafd81a1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33944/0 (socket says 172.21.15.16:33944) 2024-08-31T06:05:16.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.158+0000 7fafd7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafd80754f0 0x7fafd819c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:16.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.158+0000 7fafd7e00640 1 -- 172.21.15.16:0/4057497747 learned_addr learned my addr 172.21.15.16:0/4057497747 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:05:16.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.158+0000 7fafd6a00640 1 --2- 172.21.15.16:0/4057497747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafd8075e40 0x7fafd819c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:16.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.158+0000 7fafd7400640 1 -- 172.21.15.16:0/4057497747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafd8075e40 msgr2=0x7fafd819c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:16.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.158+0000 7fafd7400640 1 --2- 172.21.15.16:0/4057497747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafd8075e40 0x7fafd819c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:16.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.158+0000 7fafd7400640 1 -- 172.21.15.16:0/4057497747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd810fec0 msgr2=0x7fafd81a1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:16.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.158+0000 7fafd7400640 1 --2- 172.21.15.16:0/4057497747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd810fec0 0x7fafd81a1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:16.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.158+0000 7fafd7400640 1 -- 172.21.15.16:0/4057497747 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd81a2350 con 0x7fafd80754f0 2024-08-31T06:05:16.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.158+0000 7fafd7e00640 1 --2- 172.21.15.16:0/4057497747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd810fec0 0x7fafd81a1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:05:16.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.159+0000 7fafd7400640 1 --2- 172.21.15.16:0/4057497747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafd80754f0 0x7fafd819c3e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fafc8006010 tx=0x7fafc8004060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:16.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.159+0000 7fafcfe00640 1 -- 172.21.15.16:0/4057497747 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc803f070 con 0x7fafd80754f0 2024-08-31T06:05:16.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.159+0000 7fafcfe00640 1 -- 172.21.15.16:0/4057497747 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafc8003ca0 con 0x7fafd80754f0 2024-08-31T06:05:16.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.159+0000 7fafde200640 1 -- 172.21.15.16:0/4057497747 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafd81a25e0 con 0x7fafd80754f0 2024-08-31T06:05:16.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.159+0000 7fafcfe00640 1 -- 172.21.15.16:0/4057497747 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc8035070 con 0x7fafd80754f0 2024-08-31T06:05:16.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.159+0000 7fafde200640 1 -- 172.21.15.16:0/4057497747 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafd81a8da0 con 0x7fafd80754f0 2024-08-31T06:05:16.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.160+0000 7fafde200640 1 -- 172.21.15.16:0/4057497747 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fafd81020d0 con 0x7fafd80754f0 2024-08-31T06:05:16.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.163+0000 7fafcfe00640 1 -- 172.21.15.16:0/4057497747 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7fafc803b460 con 0x7fafd80754f0 2024-08-31T06:05:16.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.164+0000 7fafcfe00640 1 --2- 172.21.15.16:0/4057497747 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fafbc0b4c10 0x7fafbc0b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:16.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.164+0000 7fafcfe00640 1 -- 172.21.15.16:0/4057497747 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7fafc8105e90 con 0x7fafd80754f0 2024-08-31T06:05:16.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.164+0000 7fafd6a00640 1 --2- 172.21.15.16:0/4057497747 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fafbc0b4c10 0x7fafbc0b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:16.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.165+0000 7fafcfe00640 1 --2- 172.21.15.16:0/4057497747 >> [v2:172.21.15.184:6808/2054894065,v1:172.21.15.184:6809/2054894065] conn(0x7fafbc0be620 0x7fafbc0c0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:16.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.165+0000 7fafcfe00640 1 -- 172.21.15.16:0/4057497747 --> [v2:172.21.15.184:6808/2054894065,v1:172.21.15.184:6809/2054894065] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fafc803aec0 con 0x7fafbc0be620 2024-08-31T06:05:16.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.165+0000 7fafcfe00640 1 -- 172.21.15.16:0/4057497747 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7fafc8106240 con 0x7fafd80754f0 2024-08-31T06:05:16.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.165+0000 7fafd7e00640 1 --2- 172.21.15.16:0/4057497747 >> [v2:172.21.15.184:6808/2054894065,v1:172.21.15.184:6809/2054894065] conn(0x7fafbc0be620 0x7fafbc0c0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:16.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.165+0000 7fafd6a00640 1 --2- 172.21.15.16:0/4057497747 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fafbc0b4c10 0x7fafbc0b7120 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fafd81117b0 tx=0x7fafc4005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:16.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.166+0000 7fafd7e00640 1 --2- 172.21.15.16:0/4057497747 >> [v2:172.21.15.184:6808/2054894065,v1:172.21.15.184:6809/2054894065] conn(0x7fafbc0be620 0x7fafbc0c0a70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:16.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.167+0000 7fafcfe00640 1 -- 172.21.15.16:0/4057497747 <== osd.5 v2:172.21.15.184:6808/2054894065 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fafc803aec0 con 0x7fafbc0be620 2024-08-31T06:05:16.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.181+0000 7fafde200640 1 -- 172.21.15.16:0/4057497747 --> [v2:172.21.15.184:6808/2054894065,v1:172.21.15.184:6809/2054894065] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fafd8076a00 con 0x7fafbc0be620 2024-08-31T06:05:16.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.182+0000 7fafcfe00640 1 -- 172.21.15.16:0/4057497747 <== osd.5 v2:172.21.15.184:6808/2054894065 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fafd8076a00 con 0x7fafbc0be620 2024-08-31T06:05:16.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.183+0000 7fafde200640 1 -- 172.21.15.16:0/4057497747 >> [v2:172.21.15.184:6808/2054894065,v1:172.21.15.184:6809/2054894065] conn(0x7fafbc0be620 msgr2=0x7fafbc0c0a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:16.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.183+0000 7fafde200640 1 --2- 172.21.15.16:0/4057497747 >> [v2:172.21.15.184:6808/2054894065,v1:172.21.15.184:6809/2054894065] conn(0x7fafbc0be620 0x7fafbc0c0a70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:16.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.183+0000 7fafde200640 1 -- 172.21.15.16:0/4057497747 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fafbc0b4c10 msgr2=0x7fafbc0b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:16.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.183+0000 7fafde200640 1 --2- 172.21.15.16:0/4057497747 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fafbc0b4c10 0x7fafbc0b7120 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fafd81117b0 tx=0x7fafc4005f90 comp rx=0 tx=0).stop 2024-08-31T06:05:16.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.183+0000 7fafde200640 1 -- 172.21.15.16:0/4057497747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafd80754f0 msgr2=0x7fafd819c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:16.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.183+0000 7fafde200640 1 --2- 172.21.15.16:0/4057497747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafd80754f0 0x7fafd819c3e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fafc8006010 tx=0x7fafc8004060 comp rx=0 tx=0).stop 2024-08-31T06:05:16.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.183+0000 7fafd7e00640 1 -- 172.21.15.16:0/4057497747 reap_dead start 2024-08-31T06:05:16.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.183+0000 7fafde200640 1 -- 172.21.15.16:0/4057497747 shutdown_connections 2024-08-31T06:05:16.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.183+0000 7fafde200640 1 -- 172.21.15.16:0/4057497747 >> 172.21.15.16:0/4057497747 conn(0x7fafd807b5f0 msgr2=0x7fafd8101670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:16.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.183+0000 7fafde200640 1 -- 172.21.15.16:0/4057497747 shutdown_connections 2024-08-31T06:05:16.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.183+0000 7fafde200640 1 -- 172.21.15.16:0/4057497747 wait complete. 2024-08-31T06:05:16.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.893+0000 7f18bd800640 1 -- 172.21.15.16:0/1091670494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18b810fd10 msgr2=0x7f18b8112130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:16.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.893+0000 7f18bd800640 1 --2- 172.21.15.16:0/1091670494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18b810fd10 0x7f18b8112130 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f18b0009ca0 tx=0x7f18b002a8c0 comp rx=0 tx=0).stop 2024-08-31T06:05:16.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.893+0000 7f18bd800640 1 -- 172.21.15.16:0/1091670494 shutdown_connections 2024-08-31T06:05:16.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.893+0000 7f18bd800640 1 --2- 172.21.15.16:0/1091670494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18b810fd10 0x7f18b8112130 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:16.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.893+0000 7f18bd800640 1 --2- 172.21.15.16:0/1091670494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18b8075e40 0x7f18b810f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:16.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.893+0000 7f18bd800640 1 --2- 172.21.15.16:0/1091670494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18b80754f0 0x7f18b8075900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:16.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.893+0000 7f18bd800640 1 -- 172.21.15.16:0/1091670494 >> 172.21.15.16:0/1091670494 conn(0x7f18b80ff1a0 msgr2=0x7f18b81015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:16.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.894+0000 7f18bd800640 1 -- 172.21.15.16:0/1091670494 shutdown_connections 2024-08-31T06:05:16.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.894+0000 7f18bd800640 1 -- 172.21.15.16:0/1091670494 wait complete. 2024-08-31T06:05:16.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.895+0000 7f18bd800640 1 Processor -- start 2024-08-31T06:05:16.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.895+0000 7f18bd800640 1 -- start start 2024-08-31T06:05:16.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.896+0000 7f18bd800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18b80754f0 0x7f18b819c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:16.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.896+0000 7f18bd800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18b8075e40 0x7f18b819c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:16.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.896+0000 7f18bd800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18b810fd10 0x7f18b81a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:16.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.896+0000 7f18bd800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f18b8115880 con 0x7f18b810fd10 2024-08-31T06:05:16.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.896+0000 7f18bd800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f18b8115700 con 0x7f18b80754f0 2024-08-31T06:05:16.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.896+0000 7f18bd800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f18b8115a00 con 0x7f18b8075e40 2024-08-31T06:05:16.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.896+0000 7f18b7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18b80754f0 0x7f18b819c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:16.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.896+0000 7f18b7400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18b8075e40 0x7f18b819c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:16.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.896+0000 7f18bce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18b810fd10 0x7f18b81a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:16.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.896+0000 7f18b7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18b80754f0 0x7f18b819c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:52420/0 (socket says 172.21.15.16:52420) 2024-08-31T06:05:16.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.896+0000 7f18b7400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18b8075e40 0x7f18b819c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:43398/0 (socket says 172.21.15.16:43398) 2024-08-31T06:05:16.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.896+0000 7f18b7e00640 1 -- 172.21.15.16:0/3271449517 learned_addr learned my addr 172.21.15.16:0/3271449517 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:05:16.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.896+0000 7f18bce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18b810fd10 0x7f18b81a1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33982/0 (socket says 172.21.15.16:33982) 2024-08-31T06:05:16.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.897+0000 7f18b7400640 1 -- 172.21.15.16:0/3271449517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18b80754f0 msgr2=0x7f18b819c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:16.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.897+0000 7f18b7400640 1 --2- 172.21.15.16:0/3271449517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18b80754f0 0x7f18b819c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:16.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.897+0000 7f18b7400640 1 -- 172.21.15.16:0/3271449517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18b810fd10 msgr2=0x7f18b81a1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:16.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.897+0000 7f18b7400640 1 --2- 172.21.15.16:0/3271449517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18b810fd10 0x7f18b81a1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:16.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.897+0000 7f18b7400640 1 -- 172.21.15.16:0/3271449517 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b81a2160 con 0x7f18b8075e40 2024-08-31T06:05:16.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.897+0000 7f18bce00640 1 --2- 172.21.15.16:0/3271449517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18b810fd10 0x7f18b81a1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:05:16.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.897+0000 7f18b7400640 1 --2- 172.21.15.16:0/3271449517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18b8075e40 0x7f18b819c790 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f18b8077570 tx=0x7f18a400ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:16.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.897+0000 7f18b4c00640 1 -- 172.21.15.16:0/3271449517 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a400cc40 con 0x7f18b8075e40 2024-08-31T06:05:16.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.897+0000 7f18bd800640 1 -- 172.21.15.16:0/3271449517 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18b81a2450 con 0x7f18b8075e40 2024-08-31T06:05:16.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.897+0000 7f18b4c00640 1 -- 172.21.15.16:0/3271449517 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18a400cde0 con 0x7f18b8075e40 2024-08-31T06:05:16.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.897+0000 7f18b4c00640 1 -- 172.21.15.16:0/3271449517 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a4018760 con 0x7f18b8075e40 2024-08-31T06:05:16.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.898+0000 7f18bd800640 1 -- 172.21.15.16:0/3271449517 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18b81a8bb0 con 0x7f18b8075e40 2024-08-31T06:05:16.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.898+0000 7f18bd800640 1 -- 172.21.15.16:0/3271449517 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f18b810a280 con 0x7f18b8075e40 2024-08-31T06:05:16.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.901+0000 7f18b4c00640 1 -- 172.21.15.16:0/3271449517 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f18a4010a90 con 0x7f18b8075e40 2024-08-31T06:05:16.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.902+0000 7f18b4c00640 1 --2- 172.21.15.16:0/3271449517 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f189c0b4c90 0x7f189c0b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:16.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.903+0000 7f18b4c00640 1 -- 172.21.15.16:0/3271449517 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7f18a40e5da0 con 0x7f18b8075e40 2024-08-31T06:05:16.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.903+0000 7f18b7e00640 1 --2- 172.21.15.16:0/3271449517 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f189c0b4c90 0x7f189c0b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:16.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.903+0000 7f18b4c00640 1 --2- 172.21.15.16:0/3271449517 >> [v2:172.21.15.155:6824/2427122475,v1:172.21.15.155:6825/2427122475] conn(0x7f189c0be6a0 0x7f189c0c0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:16.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.903+0000 7f18b4c00640 1 -- 172.21.15.16:0/3271449517 --> [v2:172.21.15.155:6824/2427122475,v1:172.21.15.155:6825/2427122475] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f18a4028ea0 con 0x7f189c0be6a0 2024-08-31T06:05:16.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.903+0000 7f18b4c00640 1 -- 172.21.15.16:0/3271449517 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f18a40e6150 con 0x7f18b8075e40 2024-08-31T06:05:16.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.903+0000 7f18bce00640 1 --2- 172.21.15.16:0/3271449517 >> [v2:172.21.15.155:6824/2427122475,v1:172.21.15.155:6825/2427122475] conn(0x7f189c0be6a0 0x7f189c0c0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:16.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.903+0000 7f18b7e00640 1 --2- 172.21.15.16:0/3271449517 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f189c0b4c90 0x7f189c0b71a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f18a80095f0 tx=0x7f18a8005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:16.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.904+0000 7f18bce00640 1 --2- 172.21.15.16:0/3271449517 >> [v2:172.21.15.155:6824/2427122475,v1:172.21.15.155:6825/2427122475] conn(0x7f189c0be6a0 0x7f189c0c0af0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:16.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.905+0000 7f18b4c00640 1 -- 172.21.15.16:0/3271449517 <== osd.10 v2:172.21.15.155:6824/2427122475 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f18a4028ea0 con 0x7f189c0be6a0 2024-08-31T06:05:16.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.919+0000 7f18bd800640 1 -- 172.21.15.16:0/3271449517 --> [v2:172.21.15.155:6824/2427122475,v1:172.21.15.155:6825/2427122475] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f18b8076a00 con 0x7f189c0be6a0 2024-08-31T06:05:16.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.920+0000 7f18b4c00640 1 -- 172.21.15.16:0/3271449517 <== osd.10 v2:172.21.15.155:6824/2427122475 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f18b8076a00 con 0x7f189c0be6a0 2024-08-31T06:05:16.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.920+0000 7f18bd800640 1 -- 172.21.15.16:0/3271449517 >> [v2:172.21.15.155:6824/2427122475,v1:172.21.15.155:6825/2427122475] conn(0x7f189c0be6a0 msgr2=0x7f189c0c0af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:16.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.920+0000 7f18bd800640 1 --2- 172.21.15.16:0/3271449517 >> [v2:172.21.15.155:6824/2427122475,v1:172.21.15.155:6825/2427122475] conn(0x7f189c0be6a0 0x7f189c0c0af0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:16.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.920+0000 7f18bd800640 1 -- 172.21.15.16:0/3271449517 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f189c0b4c90 msgr2=0x7f189c0b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:16.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.920+0000 7f18bd800640 1 --2- 172.21.15.16:0/3271449517 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f189c0b4c90 0x7f189c0b71a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f18a80095f0 tx=0x7f18a8005f90 comp rx=0 tx=0).stop 2024-08-31T06:05:16.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.920+0000 7f18bd800640 1 -- 172.21.15.16:0/3271449517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18b8075e40 msgr2=0x7f18b819c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:16.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.920+0000 7f18bd800640 1 --2- 172.21.15.16:0/3271449517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18b8075e40 0x7f18b819c790 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f18b8077570 tx=0x7f18a400ee50 comp rx=0 tx=0).stop 2024-08-31T06:05:16.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.921+0000 7f18bce00640 1 -- 172.21.15.16:0/3271449517 reap_dead start 2024-08-31T06:05:16.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.921+0000 7f18bd800640 1 -- 172.21.15.16:0/3271449517 shutdown_connections 2024-08-31T06:05:16.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.921+0000 7f18bd800640 1 -- 172.21.15.16:0/3271449517 >> 172.21.15.16:0/3271449517 conn(0x7f18b80ff1a0 msgr2=0x7f18b8109820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:16.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.921+0000 7f18bd800640 1 -- 172.21.15.16:0/3271449517 shutdown_connections 2024-08-31T06:05:16.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:16.921+0000 7f18bd800640 1 -- 172.21.15.16:0/3271449517 wait complete. 2024-08-31T06:05:17.295 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:17 smithi016 ceph-mon[17194]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-08-31T06:05:17.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:17 smithi155 ceph-mon[21742]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-08-31T06:05:17.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:17 smithi184 ceph-mon[22331]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-08-31T06:05:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.807+0000 7f58d2400640 1 -- 172.21.15.16:0/3964882228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58cc104eb0 msgr2=0x7f58cc10d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.807+0000 7f58d2400640 1 --2- 172.21.15.16:0/3964882228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58cc104eb0 0x7f58cc10d400 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f58b8009900 tx=0x7f58b802a410 comp rx=0 tx=0).stop 2024-08-31T06:05:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.807+0000 7f58d2400640 1 -- 172.21.15.16:0/3964882228 shutdown_connections 2024-08-31T06:05:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.807+0000 7f58d2400640 1 --2- 172.21.15.16:0/3964882228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58cc10d9c0 0x7f58cc10feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.807+0000 7f58d2400640 1 --2- 172.21.15.16:0/3964882228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58cc104eb0 0x7f58cc10d400 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.807+0000 7f58d2400640 1 --2- 172.21.15.16:0/3964882228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58cc104560 0x7f58cc104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.807+0000 7f58d2400640 1 -- 172.21.15.16:0/3964882228 >> 172.21.15.16:0/3964882228 conn(0x7f58cc0fe060 msgr2=0x7f58cc1004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.807+0000 7f58d2400640 1 -- 172.21.15.16:0/3964882228 shutdown_connections 2024-08-31T06:05:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.808+0000 7f58d2400640 1 -- 172.21.15.16:0/3964882228 wait complete. 2024-08-31T06:05:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.809+0000 7f58d2400640 1 Processor -- start 2024-08-31T06:05:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.809+0000 7f58d2400640 1 -- start start 2024-08-31T06:05:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.809+0000 7f58d2400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58cc104560 0x7f58cc19c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.809+0000 7f58d2400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58cc104eb0 0x7f58cc19c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.809+0000 7f58d2400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58cc10d9c0 0x7f58cc1a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.809+0000 7f58d2400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f58cc113660 con 0x7f58cc104eb0 2024-08-31T06:05:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.809+0000 7f58d2400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f58cc1134e0 con 0x7f58cc10d9c0 2024-08-31T06:05:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.809+0000 7f58d2400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f58cc1137e0 con 0x7f58cc104560 2024-08-31T06:05:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.809+0000 7f58cbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58cc104eb0 0x7f58cc19c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:17.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.809+0000 7f58d1000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58cc104560 0x7f58cc19c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:17.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.810+0000 7f58d1000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58cc104560 0x7f58cc19c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:43434/0 (socket says 172.21.15.16:43434) 2024-08-31T06:05:17.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.810+0000 7f58cbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58cc104eb0 0x7f58cc19c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34002/0 (socket says 172.21.15.16:34002) 2024-08-31T06:05:17.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.810+0000 7f58d1000640 1 -- 172.21.15.16:0/89426912 learned_addr learned my addr 172.21.15.16:0/89426912 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:05:17.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.810+0000 7f58d1a00640 1 --2- 172.21.15.16:0/89426912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58cc10d9c0 0x7f58cc1a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:17.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.810+0000 7f58d1000640 1 -- 172.21.15.16:0/89426912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58cc10d9c0 msgr2=0x7f58cc1a1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:17.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.810+0000 7f58d1000640 1 --2- 172.21.15.16:0/89426912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58cc10d9c0 0x7f58cc1a1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:17.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.810+0000 7f58d1000640 1 -- 172.21.15.16:0/89426912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58cc104eb0 msgr2=0x7f58cc19c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:17.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.810+0000 7f58d1000640 1 --2- 172.21.15.16:0/89426912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58cc104eb0 0x7f58cc19c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:17.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.810+0000 7f58d1000640 1 -- 172.21.15.16:0/89426912 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58cc1a21e0 con 0x7f58cc104560 2024-08-31T06:05:17.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.810+0000 7f58d1000640 1 --2- 172.21.15.16:0/89426912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58cc104560 0x7f58cc19c270 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f58c000d950 tx=0x7f58c000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:17.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.811+0000 7f58c9600640 1 -- 172.21.15.16:0/89426912 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c0017030 con 0x7f58cc104560 2024-08-31T06:05:17.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.811+0000 7f58c9600640 1 -- 172.21.15.16:0/89426912 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58c0004020 con 0x7f58cc104560 2024-08-31T06:05:17.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.811+0000 7f58d2400640 1 -- 172.21.15.16:0/89426912 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58cc1a24d0 con 0x7f58cc104560 2024-08-31T06:05:17.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.811+0000 7f58c9600640 1 -- 172.21.15.16:0/89426912 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c00047a0 con 0x7f58cc104560 2024-08-31T06:05:17.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.811+0000 7f58d2400640 1 -- 172.21.15.16:0/89426912 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58cc1a8c30 con 0x7f58cc104560 2024-08-31T06:05:17.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.811+0000 7f58d2400640 1 -- 172.21.15.16:0/89426912 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f58cc102bd0 con 0x7f58cc104560 2024-08-31T06:05:17.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.815+0000 7f58c9600640 1 -- 172.21.15.16:0/89426912 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f58c0009a30 con 0x7f58cc104560 2024-08-31T06:05:17.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.817+0000 7f58c9600640 1 --2- 172.21.15.16:0/89426912 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f58b40b4c10 0x7f58b40b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:17.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.817+0000 7f58c9600640 1 -- 172.21.15.16:0/89426912 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7f58c00e5440 con 0x7f58cc104560 2024-08-31T06:05:17.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.817+0000 7f58cbe00640 1 --2- 172.21.15.16:0/89426912 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f58b40b4c10 0x7f58b40b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:17.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.817+0000 7f58c9600640 1 --2- 172.21.15.16:0/89426912 >> [v2:172.21.15.16:6802/1725037400,v1:172.21.15.16:6803/1725037400] conn(0x7f58b40be620 0x7f58b40c0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:17.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.817+0000 7f58c9600640 1 -- 172.21.15.16:0/89426912 --> [v2:172.21.15.16:6802/1725037400,v1:172.21.15.16:6803/1725037400] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f58c0021e50 con 0x7f58b40be620 2024-08-31T06:05:17.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.817+0000 7f58c9600640 1 -- 172.21.15.16:0/89426912 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f58c00e57f0 con 0x7f58cc104560 2024-08-31T06:05:17.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.817+0000 7f58d1a00640 1 --2- 172.21.15.16:0/89426912 >> [v2:172.21.15.16:6802/1725037400,v1:172.21.15.16:6803/1725037400] conn(0x7f58b40be620 0x7f58b40c0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:17.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.818+0000 7f58cbe00640 1 --2- 172.21.15.16:0/89426912 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f58b40b4c10 0x7f58b40b7120 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f58b802aa00 tx=0x7f58b8005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:17.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.818+0000 7f58d1a00640 1 --2- 172.21.15.16:0/89426912 >> [v2:172.21.15.16:6802/1725037400,v1:172.21.15.16:6803/1725037400] conn(0x7f58b40be620 0x7f58b40c0a70 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:17.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.819+0000 7f58c9600640 1 -- 172.21.15.16:0/89426912 <== osd.0 v2:172.21.15.16:6802/1725037400 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f58c0021e50 con 0x7f58b40be620 2024-08-31T06:05:17.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.834+0000 7f58d2400640 1 -- 172.21.15.16:0/89426912 --> [v2:172.21.15.16:6802/1725037400,v1:172.21.15.16:6803/1725037400] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f58cc06b300 con 0x7f58b40be620 2024-08-31T06:05:17.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.834+0000 7f58c9600640 1 -- 172.21.15.16:0/89426912 <== osd.0 v2:172.21.15.16:6802/1725037400 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f58cc06b300 con 0x7f58b40be620 2024-08-31T06:05:17.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.834+0000 7f58d2400640 1 -- 172.21.15.16:0/89426912 >> [v2:172.21.15.16:6802/1725037400,v1:172.21.15.16:6803/1725037400] conn(0x7f58b40be620 msgr2=0x7f58b40c0a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:17.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.834+0000 7f58d2400640 1 --2- 172.21.15.16:0/89426912 >> [v2:172.21.15.16:6802/1725037400,v1:172.21.15.16:6803/1725037400] conn(0x7f58b40be620 0x7f58b40c0a70 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:17.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.835+0000 7f58d2400640 1 -- 172.21.15.16:0/89426912 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f58b40b4c10 msgr2=0x7f58b40b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:17.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.835+0000 7f58d2400640 1 --2- 172.21.15.16:0/89426912 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f58b40b4c10 0x7f58b40b7120 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f58b802aa00 tx=0x7f58b8005a20 comp rx=0 tx=0).stop 2024-08-31T06:05:17.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.835+0000 7f58d2400640 1 -- 172.21.15.16:0/89426912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58cc104560 msgr2=0x7f58cc19c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:17.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.835+0000 7f58d2400640 1 --2- 172.21.15.16:0/89426912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58cc104560 0x7f58cc19c270 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f58c000d950 tx=0x7f58c000de10 comp rx=0 tx=0).stop 2024-08-31T06:05:17.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.835+0000 7f58d1a00640 1 -- 172.21.15.16:0/89426912 reap_dead start 2024-08-31T06:05:17.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.835+0000 7f58d2400640 1 -- 172.21.15.16:0/89426912 shutdown_connections 2024-08-31T06:05:17.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.835+0000 7f58d2400640 1 -- 172.21.15.16:0/89426912 >> 172.21.15.16:0/89426912 conn(0x7f58cc0fe060 msgr2=0x7f58cc1021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:17.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.835+0000 7f58d2400640 1 -- 172.21.15.16:0/89426912 shutdown_connections 2024-08-31T06:05:17.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.835+0000 7f58d2400640 1 -- 172.21.15.16:0/89426912 wait complete. 2024-08-31T06:05:18.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.647+0000 7f8a5d800640 1 -- 172.21.15.16:0/1894613875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a58077120 msgr2=0x7f8a58075550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:18.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.647+0000 7f8a5d800640 1 --2- 172.21.15.16:0/1894613875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a58077120 0x7f8a58075550 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8a48009900 tx=0x7f8a4802a410 comp rx=0 tx=0).stop 2024-08-31T06:05:18.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.647+0000 7f8a5d800640 1 -- 172.21.15.16:0/1894613875 shutdown_connections 2024-08-31T06:05:18.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.647+0000 7f8a5d800640 1 --2- 172.21.15.16:0/1894613875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a58075b70 0x7f8a58075f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:18.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.647+0000 7f8a5d800640 1 --2- 172.21.15.16:0/1894613875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a581141f0 0x7f8a581166c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:18.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.647+0000 7f8a5d800640 1 --2- 172.21.15.16:0/1894613875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a58077120 0x7f8a58075550 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:18.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.647+0000 7f8a5d800640 1 -- 172.21.15.16:0/1894613875 >> 172.21.15.16:0/1894613875 conn(0x7f8a580ff330 msgr2=0x7f8a58101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:18.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.647+0000 7f8a5d800640 1 -- 172.21.15.16:0/1894613875 shutdown_connections 2024-08-31T06:05:18.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.647+0000 7f8a5d800640 1 -- 172.21.15.16:0/1894613875 wait complete. 2024-08-31T06:05:18.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.648+0000 7f8a5d800640 1 Processor -- start 2024-08-31T06:05:18.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.648+0000 7f8a5d800640 1 -- start start 2024-08-31T06:05:18.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.648+0000 7f8a5d800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a58075b70 0x7f8a581a0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:18.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.648+0000 7f8a5d800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a58077120 0x7f8a581a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:18.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.649+0000 7f8a5d800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a581141f0 0x7f8a581a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:18.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.649+0000 7f8a5d800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8a58119e80 con 0x7f8a58075b70 2024-08-31T06:05:18.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.649+0000 7f8a5d800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8a58119d00 con 0x7f8a58077120 2024-08-31T06:05:18.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.649+0000 7f8a5d800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5811a000 con 0x7f8a581141f0 2024-08-31T06:05:18.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.649+0000 7f8a56000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a58077120 0x7f8a581a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:18.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.649+0000 7f8a56000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a58077120 0x7f8a581a0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:52464/0 (socket says 172.21.15.16:52464) 2024-08-31T06:05:18.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.649+0000 7f8a56000640 1 -- 172.21.15.16:0/2025817461 learned_addr learned my addr 172.21.15.16:0/2025817461 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:05:18.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.649+0000 7f8a56a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a58075b70 0x7f8a581a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:18.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.649+0000 7f8a57400640 1 --2- 172.21.15.16:0/2025817461 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a581141f0 0x7f8a581a60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:18.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.649+0000 7f8a56000640 1 -- 172.21.15.16:0/2025817461 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a581141f0 msgr2=0x7f8a581a60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:18.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.649+0000 7f8a56000640 1 --2- 172.21.15.16:0/2025817461 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a581141f0 0x7f8a581a60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:18.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.649+0000 7f8a56000640 1 -- 172.21.15.16:0/2025817461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a58075b70 msgr2=0x7f8a581a0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:18.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.649+0000 7f8a56000640 1 --2- 172.21.15.16:0/2025817461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a58075b70 0x7f8a581a0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:18.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.649+0000 7f8a56000640 1 -- 172.21.15.16:0/2025817461 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a581a67e0 con 0x7f8a58077120 2024-08-31T06:05:18.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.650+0000 7f8a56000640 1 --2- 172.21.15.16:0/2025817461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a58077120 0x7f8a581a0e10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8a4400e9f0 tx=0x7f8a4400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:18.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.650+0000 7f8a4f400640 1 -- 172.21.15.16:0/2025817461 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4400ccc0 con 0x7f8a58077120 2024-08-31T06:05:18.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.650+0000 7f8a5d800640 1 -- 172.21.15.16:0/2025817461 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a581a6ad0 con 0x7f8a58077120 2024-08-31T06:05:18.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.650+0000 7f8a4f400640 1 -- 172.21.15.16:0/2025817461 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a440054d0 con 0x7f8a58077120 2024-08-31T06:05:18.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.650+0000 7f8a4f400640 1 -- 172.21.15.16:0/2025817461 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a44010430 con 0x7f8a58077120 2024-08-31T06:05:18.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.650+0000 7f8a5d800640 1 -- 172.21.15.16:0/2025817461 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a581ad230 con 0x7f8a58077120 2024-08-31T06:05:18.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.651+0000 7f8a5d800640 1 -- 172.21.15.16:0/2025817461 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8a580638b0 con 0x7f8a58077120 2024-08-31T06:05:18.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.655+0000 7f8a4f400640 1 -- 172.21.15.16:0/2025817461 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f8a440046c0 con 0x7f8a58077120 2024-08-31T06:05:18.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.656+0000 7f8a4f400640 1 --2- 172.21.15.16:0/2025817461 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f8a3c0b4c10 0x7f8a3c0b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:18.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.656+0000 7f8a4f400640 1 -- 172.21.15.16:0/2025817461 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7f8a440e5b00 con 0x7f8a58077120 2024-08-31T06:05:18.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.656+0000 7f8a56a00640 1 --2- 172.21.15.16:0/2025817461 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f8a3c0b4c10 0x7f8a3c0b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:18.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.656+0000 7f8a4f400640 1 --2- 172.21.15.16:0/2025817461 >> [v2:172.21.15.184:6816/2479117823,v1:172.21.15.184:6817/2479117823] conn(0x7f8a3c0be620 0x7f8a3c0c0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:18.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.656+0000 7f8a4f400640 1 -- 172.21.15.16:0/2025817461 --> [v2:172.21.15.184:6816/2479117823,v1:172.21.15.184:6817/2479117823] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8a4401de50 con 0x7f8a3c0be620 2024-08-31T06:05:18.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.656+0000 7f8a4f400640 1 -- 172.21.15.16:0/2025817461 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f8a440e5eb0 con 0x7f8a58077120 2024-08-31T06:05:18.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.657+0000 7f8a57400640 1 --2- 172.21.15.16:0/2025817461 >> [v2:172.21.15.184:6816/2479117823,v1:172.21.15.184:6817/2479117823] conn(0x7f8a3c0be620 0x7f8a3c0c0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:18.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.657+0000 7f8a56a00640 1 --2- 172.21.15.16:0/2025817461 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f8a3c0b4c10 0x7f8a3c0b7120 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8a48005ab0 tx=0x7f8a48005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:18.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.657+0000 7f8a57400640 1 --2- 172.21.15.16:0/2025817461 >> [v2:172.21.15.184:6816/2479117823,v1:172.21.15.184:6817/2479117823] conn(0x7f8a3c0be620 0x7f8a3c0c0a70 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:18.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.658+0000 7f8a4f400640 1 -- 172.21.15.16:0/2025817461 <== osd.8 v2:172.21.15.184:6816/2479117823 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8a4401de50 con 0x7f8a3c0be620 2024-08-31T06:05:18.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.673+0000 7f8a5d800640 1 -- 172.21.15.16:0/2025817461 --> [v2:172.21.15.184:6816/2479117823,v1:172.21.15.184:6817/2479117823] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8a58063d30 con 0x7f8a3c0be620 2024-08-31T06:05:18.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.673+0000 7f8a4f400640 1 -- 172.21.15.16:0/2025817461 <== osd.8 v2:172.21.15.184:6816/2479117823 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8a58063d30 con 0x7f8a3c0be620 2024-08-31T06:05:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.674+0000 7f8a5d800640 1 -- 172.21.15.16:0/2025817461 >> [v2:172.21.15.184:6816/2479117823,v1:172.21.15.184:6817/2479117823] conn(0x7f8a3c0be620 msgr2=0x7f8a3c0c0a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.674+0000 7f8a5d800640 1 --2- 172.21.15.16:0/2025817461 >> [v2:172.21.15.184:6816/2479117823,v1:172.21.15.184:6817/2479117823] conn(0x7f8a3c0be620 0x7f8a3c0c0a70 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.674+0000 7f8a5d800640 1 -- 172.21.15.16:0/2025817461 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f8a3c0b4c10 msgr2=0x7f8a3c0b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.674+0000 7f8a5d800640 1 --2- 172.21.15.16:0/2025817461 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f8a3c0b4c10 0x7f8a3c0b7120 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8a48005ab0 tx=0x7f8a48005a20 comp rx=0 tx=0).stop 2024-08-31T06:05:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.674+0000 7f8a5d800640 1 -- 172.21.15.16:0/2025817461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a58077120 msgr2=0x7f8a581a0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.674+0000 7f8a5d800640 1 --2- 172.21.15.16:0/2025817461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a58077120 0x7f8a581a0e10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8a4400e9f0 tx=0x7f8a4400eeb0 comp rx=0 tx=0).stop 2024-08-31T06:05:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.674+0000 7f8a57400640 1 -- 172.21.15.16:0/2025817461 reap_dead start 2024-08-31T06:05:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.674+0000 7f8a5d800640 1 -- 172.21.15.16:0/2025817461 shutdown_connections 2024-08-31T06:05:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.674+0000 7f8a5d800640 1 -- 172.21.15.16:0/2025817461 >> 172.21.15.16:0/2025817461 conn(0x7f8a580ff330 msgr2=0x7f8a581100f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.674+0000 7f8a5d800640 1 -- 172.21.15.16:0/2025817461 shutdown_connections 2024-08-31T06:05:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:18.674+0000 7f8a5d800640 1 -- 172.21.15.16:0/2025817461 wait complete. 2024-08-31T06:05:19.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:19 smithi016 ceph-mon[17194]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-31T06:05:19.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.183+0000 7f8d16e00640 1 -- 172.21.15.16:0/2019954105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d10109240 msgr2=0x7f8d10109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:19.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.183+0000 7f8d16e00640 1 --2- 172.21.15.16:0/2019954105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d10109240 0x7f8d10109650 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8009900 tx=0x7f8cf802a3f0 comp rx=0 tx=0).stop 2024-08-31T06:05:19.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.183+0000 7f8d16e00640 1 -- 172.21.15.16:0/2019954105 shutdown_connections 2024-08-31T06:05:19.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.183+0000 7f8d16e00640 1 --2- 172.21.15.16:0/2019954105 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d10109b90 0x7f8d101103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:19.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.183+0000 7f8d16e00640 1 --2- 172.21.15.16:0/2019954105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d10109240 0x7f8d10109650 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:19.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.183+0000 7f8d16e00640 1 --2- 172.21.15.16:0/2019954105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d10103900 0x7f8d10103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:19.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.183+0000 7f8d16e00640 1 -- 172.21.15.16:0/2019954105 >> 172.21.15.16:0/2019954105 conn(0x7f8d1007b5f0 msgr2=0x7f8d1007ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:19.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.183+0000 7f8d16e00640 1 -- 172.21.15.16:0/2019954105 shutdown_connections 2024-08-31T06:05:19.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.183+0000 7f8d16e00640 1 -- 172.21.15.16:0/2019954105 wait complete. 2024-08-31T06:05:19.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.184+0000 7f8d16e00640 1 Processor -- start 2024-08-31T06:05:19.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.184+0000 7f8d16e00640 1 -- start start 2024-08-31T06:05:19.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.184+0000 7f8d16e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d10103900 0x7f8d1019a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:19.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.185+0000 7f8d16e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d10109240 0x7f8d1019a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:19.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.185+0000 7f8d16e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d10109b90 0x7f8d1019fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:19.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.185+0000 7f8d16e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8d101138a0 con 0x7f8d10109b90 2024-08-31T06:05:19.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.185+0000 7f8d16e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8d10113720 con 0x7f8d10103900 2024-08-31T06:05:19.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.185+0000 7f8d16e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8d10113a20 con 0x7f8d10109240 2024-08-31T06:05:19.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.185+0000 7f8d0fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d10103900 0x7f8d1019a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:19.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.185+0000 7f8d0fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d10103900 0x7f8d1019a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:52488/0 (socket says 172.21.15.16:52488) 2024-08-31T06:05:19.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.185+0000 7f8d0fe00640 1 -- 172.21.15.16:0/3782748365 learned_addr learned my addr 172.21.15.16:0/3782748365 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:05:19.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.185+0000 7f8d07e00640 1 --2- 172.21.15.16:0/3782748365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d10109240 0x7f8d1019a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:19.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.185+0000 7f8d14e00640 1 --2- 172.21.15.16:0/3782748365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d10109b90 0x7f8d1019fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:19.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.185+0000 7f8d0fe00640 1 -- 172.21.15.16:0/3782748365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d10109240 msgr2=0x7f8d1019a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:19.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.185+0000 7f8d0fe00640 1 --2- 172.21.15.16:0/3782748365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d10109240 0x7f8d1019a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:19.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.185+0000 7f8d0fe00640 1 -- 172.21.15.16:0/3782748365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d10109b90 msgr2=0x7f8d1019fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:19.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.185+0000 7f8d0fe00640 1 --2- 172.21.15.16:0/3782748365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d10109b90 0x7f8d1019fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:19.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.185+0000 7f8d0fe00640 1 -- 172.21.15.16:0/3782748365 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d101a0280 con 0x7f8d10103900 2024-08-31T06:05:19.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.186+0000 7f8d0fe00640 1 --2- 172.21.15.16:0/3782748365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d10103900 0x7f8d1019a310 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8d0000e9f0 tx=0x7f8d0000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:19.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.186+0000 7f8d0d600640 1 -- 172.21.15.16:0/3782748365 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0000cd80 con 0x7f8d10103900 2024-08-31T06:05:19.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.186+0000 7f8d16e00640 1 -- 172.21.15.16:0/3782748365 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d101a0570 con 0x7f8d10103900 2024-08-31T06:05:19.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.186+0000 7f8d0d600640 1 -- 172.21.15.16:0/3782748365 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d000108e0 con 0x7f8d10103900 2024-08-31T06:05:19.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.186+0000 7f8d0d600640 1 -- 172.21.15.16:0/3782748365 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d00018980 con 0x7f8d10103900 2024-08-31T06:05:19.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.186+0000 7f8d16e00640 1 -- 172.21.15.16:0/3782748365 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d101a6cd0 con 0x7f8d10103900 2024-08-31T06:05:19.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.187+0000 7f8d16e00640 1 -- 172.21.15.16:0/3782748365 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8d100638f0 con 0x7f8d10103900 2024-08-31T06:05:19.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.190+0000 7f8d0d600640 1 -- 172.21.15.16:0/3782748365 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f8d00010430 con 0x7f8d10103900 2024-08-31T06:05:19.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.191+0000 7f8d0d600640 1 --2- 172.21.15.16:0/3782748365 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f8cf40b4c90 0x7f8cf40b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:19.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.191+0000 7f8d0d600640 1 -- 172.21.15.16:0/3782748365 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7f8d000e5d40 con 0x7f8d10103900 2024-08-31T06:05:19.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.191+0000 7f8d07e00640 1 --2- 172.21.15.16:0/3782748365 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f8cf40b4c90 0x7f8cf40b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:19.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.191+0000 7f8d0d600640 1 --2- 172.21.15.16:0/3782748365 >> [v2:172.21.15.155:6808/3592422019,v1:172.21.15.155:6809/3592422019] conn(0x7f8cf40be6a0 0x7f8cf40c0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:19.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.192+0000 7f8d0d600640 1 -- 172.21.15.16:0/3782748365 --> [v2:172.21.15.155:6808/3592422019,v1:172.21.15.155:6809/3592422019] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8cf40c11c0 con 0x7f8cf40be6a0 2024-08-31T06:05:19.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.192+0000 7f8d0d600640 1 -- 172.21.15.16:0/3782748365 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f8d000e60f0 con 0x7f8d10103900 2024-08-31T06:05:19.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.192+0000 7f8d14e00640 1 --2- 172.21.15.16:0/3782748365 >> [v2:172.21.15.155:6808/3592422019,v1:172.21.15.155:6809/3592422019] conn(0x7f8cf40be6a0 0x7f8cf40c0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:19.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.192+0000 7f8d07e00640 1 --2- 172.21.15.16:0/3782748365 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f8cf40b4c90 0x7f8cf40b71a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8cf802a9e0 tx=0x7f8cf8005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:19.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.193+0000 7f8d14e00640 1 --2- 172.21.15.16:0/3782748365 >> [v2:172.21.15.155:6808/3592422019,v1:172.21.15.155:6809/3592422019] conn(0x7f8cf40be6a0 0x7f8cf40c0af0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:19.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.193+0000 7f8d0d600640 1 -- 172.21.15.16:0/3782748365 <== osd.4 v2:172.21.15.155:6808/3592422019 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8cf40c11c0 con 0x7f8cf40be6a0 2024-08-31T06:05:19.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.208+0000 7f8d16e00640 1 -- 172.21.15.16:0/3782748365 --> [v2:172.21.15.155:6808/3592422019,v1:172.21.15.155:6809/3592422019] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8d10063d70 con 0x7f8cf40be6a0 2024-08-31T06:05:19.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.209+0000 7f8d0d600640 1 -- 172.21.15.16:0/3782748365 <== osd.4 v2:172.21.15.155:6808/3592422019 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8d10063d70 con 0x7f8cf40be6a0 2024-08-31T06:05:19.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.209+0000 7f8d16e00640 1 -- 172.21.15.16:0/3782748365 >> [v2:172.21.15.155:6808/3592422019,v1:172.21.15.155:6809/3592422019] conn(0x7f8cf40be6a0 msgr2=0x7f8cf40c0af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:19.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.209+0000 7f8d16e00640 1 --2- 172.21.15.16:0/3782748365 >> [v2:172.21.15.155:6808/3592422019,v1:172.21.15.155:6809/3592422019] conn(0x7f8cf40be6a0 0x7f8cf40c0af0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:19.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.209+0000 7f8d16e00640 1 -- 172.21.15.16:0/3782748365 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f8cf40b4c90 msgr2=0x7f8cf40b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:19.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.209+0000 7f8d16e00640 1 --2- 172.21.15.16:0/3782748365 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f8cf40b4c90 0x7f8cf40b71a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8cf802a9e0 tx=0x7f8cf8005a20 comp rx=0 tx=0).stop 2024-08-31T06:05:19.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.209+0000 7f8d16e00640 1 -- 172.21.15.16:0/3782748365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d10103900 msgr2=0x7f8d1019a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:19.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.209+0000 7f8d16e00640 1 --2- 172.21.15.16:0/3782748365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d10103900 0x7f8d1019a310 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8d0000e9f0 tx=0x7f8d0000eeb0 comp rx=0 tx=0).stop 2024-08-31T06:05:19.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.209+0000 7f8d14e00640 1 -- 172.21.15.16:0/3782748365 reap_dead start 2024-08-31T06:05:19.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.209+0000 7f8d16e00640 1 -- 172.21.15.16:0/3782748365 shutdown_connections 2024-08-31T06:05:19.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.209+0000 7f8d16e00640 1 -- 172.21.15.16:0/3782748365 >> 172.21.15.16:0/3782748365 conn(0x7f8d1007b5f0 msgr2=0x7f8d10105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:19.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.210+0000 7f8d16e00640 1 -- 172.21.15.16:0/3782748365 shutdown_connections 2024-08-31T06:05:19.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.210+0000 7f8d16e00640 1 -- 172.21.15.16:0/3782748365 wait complete. 2024-08-31T06:05:19.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:19 smithi155 ceph-mon[21742]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-31T06:05:19.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:19 smithi184 ceph-mon[22331]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-31T06:05:19.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.407+0000 7f29a7800640 1 -- 172.21.15.16:0/2594936229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29a0103900 msgr2=0x7f29a0103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:19.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.407+0000 7f29a7800640 1 --2- 172.21.15.16:0/2594936229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29a0103900 0x7f29a0103d10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2994009ca0 tx=0x7f299402a8c0 comp rx=0 tx=0).stop 2024-08-31T06:05:19.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.407+0000 7f29a7800640 1 -- 172.21.15.16:0/2594936229 shutdown_connections 2024-08-31T06:05:19.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.407+0000 7f29a7800640 1 --2- 172.21.15.16:0/2594936229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29a0109b90 0x7f29a01103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:19.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.407+0000 7f29a7800640 1 --2- 172.21.15.16:0/2594936229 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29a0109240 0x7f29a0109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:19.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.407+0000 7f29a7800640 1 --2- 172.21.15.16:0/2594936229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29a0103900 0x7f29a0103d10 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:19.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.407+0000 7f29a7800640 1 -- 172.21.15.16:0/2594936229 >> 172.21.15.16:0/2594936229 conn(0x7f29a007b5f0 msgr2=0x7f29a007ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:19.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.407+0000 7f29a7800640 1 -- 172.21.15.16:0/2594936229 shutdown_connections 2024-08-31T06:05:19.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.407+0000 7f29a7800640 1 -- 172.21.15.16:0/2594936229 wait complete. 2024-08-31T06:05:19.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.408+0000 7f29a7800640 1 Processor -- start 2024-08-31T06:05:19.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.409+0000 7f29a7800640 1 -- start start 2024-08-31T06:05:19.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.409+0000 7f29a7800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29a0103900 0x7f29a019c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:19.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.409+0000 7f29a7800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29a0109240 0x7f29a019c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:19.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.409+0000 7f29a7800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29a0109b90 0x7f29a01a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:19.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.409+0000 7f29a7800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f29a01138a0 con 0x7f29a0103900 2024-08-31T06:05:19.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.409+0000 7f29a7800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f29a0113720 con 0x7f29a0109b90 2024-08-31T06:05:19.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.409+0000 7f29a7800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f29a0113a20 con 0x7f29a0109240 2024-08-31T06:05:19.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.409+0000 7f29a4e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29a0103900 0x7f29a019c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:19.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.409+0000 7f29a4e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29a0103900 0x7f29a019c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34056/0 (socket says 172.21.15.16:34056) 2024-08-31T06:05:19.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.409+0000 7f29a4e00640 1 -- 172.21.15.16:0/166324720 learned_addr learned my addr 172.21.15.16:0/166324720 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:05:19.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.409+0000 7f299fe00640 1 --2- 172.21.15.16:0/166324720 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29a0109240 0x7f29a019c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:19.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.410+0000 7f29a5800640 1 --2- 172.21.15.16:0/166324720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29a0109b90 0x7f29a01a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:19.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.410+0000 7f29a4e00640 1 -- 172.21.15.16:0/166324720 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29a0109240 msgr2=0x7f29a019c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:19.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.410+0000 7f29a4e00640 1 --2- 172.21.15.16:0/166324720 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29a0109240 0x7f29a019c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:19.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.410+0000 7f29a4e00640 1 -- 172.21.15.16:0/166324720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29a0109b90 msgr2=0x7f29a01a1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:19.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.410+0000 7f29a4e00640 1 --2- 172.21.15.16:0/166324720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29a0109b90 0x7f29a01a1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:19.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.410+0000 7f29a4e00640 1 -- 172.21.15.16:0/166324720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a01a2380 con 0x7f29a0103900 2024-08-31T06:05:19.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.410+0000 7f29a4e00640 1 --2- 172.21.15.16:0/166324720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29a0103900 0x7f29a019c410 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2994005e30 tx=0x7f2994031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:19.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.410+0000 7f299d600640 1 -- 172.21.15.16:0/166324720 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2994003b70 con 0x7f29a0103900 2024-08-31T06:05:19.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.411+0000 7f299d600640 1 -- 172.21.15.16:0/166324720 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2994003d10 con 0x7f29a0103900 2024-08-31T06:05:19.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.411+0000 7f29a7800640 1 -- 172.21.15.16:0/166324720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29a01a2610 con 0x7f29a0103900 2024-08-31T06:05:19.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.411+0000 7f299d600640 1 -- 172.21.15.16:0/166324720 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299402fa30 con 0x7f29a0103900 2024-08-31T06:05:19.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.411+0000 7f29a7800640 1 -- 172.21.15.16:0/166324720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29a01a8dd0 con 0x7f29a0103900 2024-08-31T06:05:19.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.412+0000 7f29a7800640 1 -- 172.21.15.16:0/166324720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f29a0107ab0 con 0x7f29a0103900 2024-08-31T06:05:19.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.414+0000 7f299d600640 1 -- 172.21.15.16:0/166324720 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f299402fbd0 con 0x7f29a0103900 2024-08-31T06:05:19.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.415+0000 7f299d600640 1 --2- 172.21.15.16:0/166324720 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f29880b4c10 0x7f29880b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:19.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.416+0000 7f299d600640 1 -- 172.21.15.16:0/166324720 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7f2994106670 con 0x7f29a0103900 2024-08-31T06:05:19.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.416+0000 7f299fe00640 1 --2- 172.21.15.16:0/166324720 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f29880b4c10 0x7f29880b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:19.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.416+0000 7f299d600640 1 --2- 172.21.15.16:0/166324720 >> [v2:172.21.15.184:6824/399265701,v1:172.21.15.184:6825/399265701] conn(0x7f29880be620 0x7f29880c0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:19.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.416+0000 7f299d600640 1 -- 172.21.15.16:0/166324720 --> [v2:172.21.15.184:6824/399265701,v1:172.21.15.184:6825/399265701] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f299403fec0 con 0x7f29880be620 2024-08-31T06:05:19.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.416+0000 7f299d600640 1 -- 172.21.15.16:0/166324720 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f2994106a20 con 0x7f29a0103900 2024-08-31T06:05:19.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.416+0000 7f299fe00640 1 --2- 172.21.15.16:0/166324720 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f29880b4c10 0x7f29880b7120 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f29a0108b40 tx=0x7f298c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:19.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.417+0000 7f29a5800640 1 --2- 172.21.15.16:0/166324720 >> [v2:172.21.15.184:6824/399265701,v1:172.21.15.184:6825/399265701] conn(0x7f29880be620 0x7f29880c0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:19.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.417+0000 7f29a5800640 1 --2- 172.21.15.16:0/166324720 >> [v2:172.21.15.184:6824/399265701,v1:172.21.15.184:6825/399265701] conn(0x7f29880be620 0x7f29880c0a70 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:19.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.418+0000 7f299d600640 1 -- 172.21.15.16:0/166324720 <== osd.11 v2:172.21.15.184:6824/399265701 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f299403fec0 con 0x7f29880be620 2024-08-31T06:05:19.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.433+0000 7f29a7800640 1 -- 172.21.15.16:0/166324720 --> [v2:172.21.15.184:6824/399265701,v1:172.21.15.184:6825/399265701] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f29a0104b80 con 0x7f29880be620 2024-08-31T06:05:19.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.434+0000 7f299d600640 1 -- 172.21.15.16:0/166324720 <== osd.11 v2:172.21.15.184:6824/399265701 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f29a0104b80 con 0x7f29880be620 2024-08-31T06:05:19.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.434+0000 7f29a7800640 1 -- 172.21.15.16:0/166324720 >> [v2:172.21.15.184:6824/399265701,v1:172.21.15.184:6825/399265701] conn(0x7f29880be620 msgr2=0x7f29880c0a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:19.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.434+0000 7f29a7800640 1 --2- 172.21.15.16:0/166324720 >> [v2:172.21.15.184:6824/399265701,v1:172.21.15.184:6825/399265701] conn(0x7f29880be620 0x7f29880c0a70 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:19.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.434+0000 7f29a7800640 1 -- 172.21.15.16:0/166324720 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f29880b4c10 msgr2=0x7f29880b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:19.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.434+0000 7f29a7800640 1 --2- 172.21.15.16:0/166324720 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f29880b4c10 0x7f29880b7120 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f29a0108b40 tx=0x7f298c005f90 comp rx=0 tx=0).stop 2024-08-31T06:05:19.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.434+0000 7f29a7800640 1 -- 172.21.15.16:0/166324720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29a0103900 msgr2=0x7f29a019c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:19.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.434+0000 7f29a7800640 1 --2- 172.21.15.16:0/166324720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29a0103900 0x7f29a019c410 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2994005e30 tx=0x7f2994031840 comp rx=0 tx=0).stop 2024-08-31T06:05:19.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.434+0000 7f29a5800640 1 -- 172.21.15.16:0/166324720 reap_dead start 2024-08-31T06:05:19.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.435+0000 7f29a7800640 1 -- 172.21.15.16:0/166324720 shutdown_connections 2024-08-31T06:05:19.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.435+0000 7f29a7800640 1 -- 172.21.15.16:0/166324720 >> 172.21.15.16:0/166324720 conn(0x7f29a007b5f0 msgr2=0x7f29a0107050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:19.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.435+0000 7f29a7800640 1 -- 172.21.15.16:0/166324720 shutdown_connections 2024-08-31T06:05:19.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:19.435+0000 7f29a7800640 1 -- 172.21.15.16:0/166324720 wait complete. 2024-08-31T06:05:20.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.169+0000 7f8bd5600640 1 -- 172.21.15.16:0/1076995913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd0109b90 msgr2=0x7f8bd01103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:20.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.169+0000 7f8bd5600640 1 --2- 172.21.15.16:0/1076995913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd0109b90 0x7f8bd01103c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8009ca0 tx=0x7f8bc802a8c0 comp rx=0 tx=0).stop 2024-08-31T06:05:20.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.169+0000 7f8bd5600640 1 -- 172.21.15.16:0/1076995913 shutdown_connections 2024-08-31T06:05:20.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.169+0000 7f8bd5600640 1 --2- 172.21.15.16:0/1076995913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd0109b90 0x7f8bd01103c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:20.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.169+0000 7f8bd5600640 1 --2- 172.21.15.16:0/1076995913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bd0109240 0x7f8bd0109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:20.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.169+0000 7f8bd5600640 1 --2- 172.21.15.16:0/1076995913 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bd0103900 0x7f8bd0103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:20.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.169+0000 7f8bd5600640 1 -- 172.21.15.16:0/1076995913 >> 172.21.15.16:0/1076995913 conn(0x7f8bd007b5f0 msgr2=0x7f8bd007ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:20.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.169+0000 7f8bd5600640 1 -- 172.21.15.16:0/1076995913 shutdown_connections 2024-08-31T06:05:20.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.169+0000 7f8bd5600640 1 -- 172.21.15.16:0/1076995913 wait complete. 2024-08-31T06:05:20.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.170+0000 7f8bd5600640 1 Processor -- start 2024-08-31T06:05:20.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.171+0000 7f8bd5600640 1 -- start start 2024-08-31T06:05:20.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.171+0000 7f8bd5600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd0103900 0x7f8bd00754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:20.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.171+0000 7f8bd5600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bd0109240 0x7f8bd00759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:20.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.171+0000 7f8bd5600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bd0109b90 0x7f8bd0075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:20.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.171+0000 7f8bd5600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd01138a0 con 0x7f8bd0103900 2024-08-31T06:05:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.171+0000 7f8bd5600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd0113720 con 0x7f8bd0109b90 2024-08-31T06:05:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.171+0000 7f8bd5600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd0113a20 con 0x7f8bd0109240 2024-08-31T06:05:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.171+0000 7f8bcea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd0103900 0x7f8bd00754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.171+0000 7f8bcea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd0103900 0x7f8bd00754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34074/0 (socket says 172.21.15.16:34074) 2024-08-31T06:05:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.171+0000 7f8bcea00640 1 -- 172.21.15.16:0/668870393 learned_addr learned my addr 172.21.15.16:0/668870393 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:05:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.171+0000 7f8bc6000640 1 --2- 172.21.15.16:0/668870393 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bd0109240 0x7f8bd00759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.171+0000 7f8bcf400640 1 --2- 172.21.15.16:0/668870393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bd0109b90 0x7f8bd0075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.172+0000 7f8bcea00640 1 -- 172.21.15.16:0/668870393 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bd0109240 msgr2=0x7f8bd00759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.172+0000 7f8bcea00640 1 --2- 172.21.15.16:0/668870393 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bd0109240 0x7f8bd00759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.172+0000 7f8bcea00640 1 -- 172.21.15.16:0/668870393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bd0109b90 msgr2=0x7f8bd0075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.172+0000 7f8bcea00640 1 --2- 172.21.15.16:0/668870393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bd0109b90 0x7f8bd0075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.172+0000 7f8bcea00640 1 -- 172.21.15.16:0/668870393 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd01a8470 con 0x7f8bd0103900 2024-08-31T06:05:20.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.172+0000 7f8bcf400640 1 --2- 172.21.15.16:0/668870393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bd0109b90 0x7f8bd0075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:05:20.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.172+0000 7f8bcea00640 1 --2- 172.21.15.16:0/668870393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd0103900 0x7f8bd00754b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8bc000e9f0 tx=0x7f8bc000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:20.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.172+0000 7f8bc7e00640 1 -- 172.21.15.16:0/668870393 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc000cd80 con 0x7f8bd0103900 2024-08-31T06:05:20.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.172+0000 7f8bc7e00640 1 -- 172.21.15.16:0/668870393 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bc00108e0 con 0x7f8bd0103900 2024-08-31T06:05:20.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.172+0000 7f8bd5600640 1 -- 172.21.15.16:0/668870393 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bd01a8760 con 0x7f8bd0103900 2024-08-31T06:05:20.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.172+0000 7f8bc7e00640 1 -- 172.21.15.16:0/668870393 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0018980 con 0x7f8bd0103900 2024-08-31T06:05:20.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.172+0000 7f8bd5600640 1 -- 172.21.15.16:0/668870393 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bd01a8cf0 con 0x7f8bd0103900 2024-08-31T06:05:20.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.173+0000 7f8bd5600640 1 -- 172.21.15.16:0/668870393 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8bd00638f0 con 0x7f8bd0103900 2024-08-31T06:05:20.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.176+0000 7f8bc7e00640 1 -- 172.21.15.16:0/668870393 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f8bc0010430 con 0x7f8bd0103900 2024-08-31T06:05:20.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.177+0000 7f8bc7e00640 1 --2- 172.21.15.16:0/668870393 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f8bb40b4c90 0x7f8bb40b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:20.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.177+0000 7f8bc7e00640 1 -- 172.21.15.16:0/668870393 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7f8bc00e5d90 con 0x7f8bd0103900 2024-08-31T06:05:20.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.177+0000 7f8bc6000640 1 --2- 172.21.15.16:0/668870393 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f8bb40b4c90 0x7f8bb40b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:20.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.178+0000 7f8bc7e00640 1 --2- 172.21.15.16:0/668870393 >> [v2:172.21.15.184:6800/2193296422,v1:172.21.15.184:6801/2193296422] conn(0x7f8bb40be6a0 0x7f8bb40c0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:20.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.178+0000 7f8bc7e00640 1 -- 172.21.15.16:0/668870393 --> [v2:172.21.15.184:6800/2193296422,v1:172.21.15.184:6801/2193296422] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8bc0027ea0 con 0x7f8bb40be6a0 2024-08-31T06:05:20.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.178+0000 7f8bc7e00640 1 -- 172.21.15.16:0/668870393 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f8bc00e61e0 con 0x7f8bd0103900 2024-08-31T06:05:20.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.178+0000 7f8bcf400640 1 --2- 172.21.15.16:0/668870393 >> [v2:172.21.15.184:6800/2193296422,v1:172.21.15.184:6801/2193296422] conn(0x7f8bb40be6a0 0x7f8bb40c0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:20.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.178+0000 7f8bc6000640 1 --2- 172.21.15.16:0/668870393 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f8bb40b4c90 0x7f8bb40b71a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0106d00 tx=0x7f8bb8005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:20.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.179+0000 7f8bcf400640 1 --2- 172.21.15.16:0/668870393 >> [v2:172.21.15.184:6800/2193296422,v1:172.21.15.184:6801/2193296422] conn(0x7f8bb40be6a0 0x7f8bb40c0af0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:20.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.180+0000 7f8bc7e00640 1 -- 172.21.15.16:0/668870393 <== osd.2 v2:172.21.15.184:6800/2193296422 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8bc0027ea0 con 0x7f8bb40be6a0 2024-08-31T06:05:20.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.194+0000 7f8bd5600640 1 -- 172.21.15.16:0/668870393 --> [v2:172.21.15.184:6800/2193296422,v1:172.21.15.184:6801/2193296422] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8bd0063cf0 con 0x7f8bb40be6a0 2024-08-31T06:05:20.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.195+0000 7f8bc7e00640 1 -- 172.21.15.16:0/668870393 <== osd.2 v2:172.21.15.184:6800/2193296422 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8bd0063cf0 con 0x7f8bb40be6a0 2024-08-31T06:05:20.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.195+0000 7f8bd5600640 1 -- 172.21.15.16:0/668870393 >> [v2:172.21.15.184:6800/2193296422,v1:172.21.15.184:6801/2193296422] conn(0x7f8bb40be6a0 msgr2=0x7f8bb40c0af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:20.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.195+0000 7f8bd5600640 1 --2- 172.21.15.16:0/668870393 >> [v2:172.21.15.184:6800/2193296422,v1:172.21.15.184:6801/2193296422] conn(0x7f8bb40be6a0 0x7f8bb40c0af0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:20.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.195+0000 7f8bd5600640 1 -- 172.21.15.16:0/668870393 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f8bb40b4c90 msgr2=0x7f8bb40b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:20.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.195+0000 7f8bd5600640 1 --2- 172.21.15.16:0/668870393 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f8bb40b4c90 0x7f8bb40b71a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0106d00 tx=0x7f8bb8005f90 comp rx=0 tx=0).stop 2024-08-31T06:05:20.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.195+0000 7f8bd5600640 1 -- 172.21.15.16:0/668870393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd0103900 msgr2=0x7f8bd00754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:20.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.195+0000 7f8bd5600640 1 --2- 172.21.15.16:0/668870393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd0103900 0x7f8bd00754b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8bc000e9f0 tx=0x7f8bc000eeb0 comp rx=0 tx=0).stop 2024-08-31T06:05:20.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.195+0000 7f8bcf400640 1 -- 172.21.15.16:0/668870393 reap_dead start 2024-08-31T06:05:20.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.196+0000 7f8bd5600640 1 -- 172.21.15.16:0/668870393 shutdown_connections 2024-08-31T06:05:20.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.196+0000 7f8bd5600640 1 -- 172.21.15.16:0/668870393 >> 172.21.15.16:0/668870393 conn(0x7f8bd007b5f0 msgr2=0x7f8bd0105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:20.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.196+0000 7f8bd5600640 1 -- 172.21.15.16:0/668870393 shutdown_connections 2024-08-31T06:05:20.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:20.196+0000 7f8bd5600640 1 -- 172.21.15.16:0/668870393 wait complete. 2024-08-31T06:05:21.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.180+0000 7f13e2600640 1 -- 172.21.15.16:0/55170243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13dc109260 msgr2=0x7f13dc109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:21.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.180+0000 7f13e2600640 1 --2- 172.21.15.16:0/55170243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13dc109260 0x7f13dc109670 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f13c8009900 tx=0x7f13c802a3f0 comp rx=0 tx=0).stop 2024-08-31T06:05:21.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.180+0000 7f13e2600640 1 -- 172.21.15.16:0/55170243 shutdown_connections 2024-08-31T06:05:21.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.180+0000 7f13e2600640 1 --2- 172.21.15.16:0/55170243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13dc109bb0 0x7f13dc1103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:21.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.180+0000 7f13e2600640 1 --2- 172.21.15.16:0/55170243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13dc109260 0x7f13dc109670 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:21.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.180+0000 7f13e2600640 1 --2- 172.21.15.16:0/55170243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13dc103920 0x7f13dc103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:21.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.180+0000 7f13e2600640 1 -- 172.21.15.16:0/55170243 >> 172.21.15.16:0/55170243 conn(0x7f13dc0ff330 msgr2=0x7f13dc101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:21.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.180+0000 7f13e2600640 1 -- 172.21.15.16:0/55170243 shutdown_connections 2024-08-31T06:05:21.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.180+0000 7f13e2600640 1 -- 172.21.15.16:0/55170243 wait complete. 2024-08-31T06:05:21.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.181+0000 7f13e2600640 1 Processor -- start 2024-08-31T06:05:21.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.181+0000 7f13e2600640 1 -- start start 2024-08-31T06:05:21.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.182+0000 7f13e2600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13dc103920 0x7f13dc19a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:21.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.182+0000 7f13e2600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13dc109260 0x7f13dc19a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:21.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.182+0000 7f13e2600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13dc109bb0 0x7f13dc19fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:21.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.182+0000 7f13e2600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f13dc1138c0 con 0x7f13dc109bb0 2024-08-31T06:05:21.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.182+0000 7f13e2600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f13dc113740 con 0x7f13dc103920 2024-08-31T06:05:21.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.182+0000 7f13e2600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f13dc113a40 con 0x7f13dc109260 2024-08-31T06:05:21.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.182+0000 7f13db400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13dc103920 0x7f13dc19a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:21.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.182+0000 7f13db400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13dc103920 0x7f13dc19a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:52526/0 (socket says 172.21.15.16:52526) 2024-08-31T06:05:21.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.182+0000 7f13db400640 1 -- 172.21.15.16:0/3086246199 learned_addr learned my addr 172.21.15.16:0/3086246199 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:05:21.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.182+0000 7f13dbe00640 1 --2- 172.21.15.16:0/3086246199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13dc109bb0 0x7f13dc19fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:21.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.182+0000 7f13daa00640 1 --2- 172.21.15.16:0/3086246199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13dc109260 0x7f13dc19a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:21.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.182+0000 7f13db400640 1 -- 172.21.15.16:0/3086246199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13dc109260 msgr2=0x7f13dc19a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:21.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.182+0000 7f13db400640 1 --2- 172.21.15.16:0/3086246199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13dc109260 0x7f13dc19a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:21.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.182+0000 7f13db400640 1 -- 172.21.15.16:0/3086246199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13dc109bb0 msgr2=0x7f13dc19fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:21.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.182+0000 7f13db400640 1 --2- 172.21.15.16:0/3086246199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13dc109bb0 0x7f13dc19fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:21.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.182+0000 7f13db400640 1 -- 172.21.15.16:0/3086246199 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13dc1a0280 con 0x7f13dc103920 2024-08-31T06:05:21.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.183+0000 7f13db400640 1 --2- 172.21.15.16:0/3086246199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13dc103920 0x7f13dc19a310 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f13cc00e9f0 tx=0x7f13cc00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:21.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.183+0000 7f13d3e00640 1 -- 172.21.15.16:0/3086246199 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc00ccc0 con 0x7f13dc103920 2024-08-31T06:05:21.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.183+0000 7f13e2600640 1 -- 172.21.15.16:0/3086246199 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13dc1a0570 con 0x7f13dc103920 2024-08-31T06:05:21.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.183+0000 7f13e2600640 1 -- 172.21.15.16:0/3086246199 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13dc1a6cd0 con 0x7f13dc103920 2024-08-31T06:05:21.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.183+0000 7f13d3e00640 1 -- 172.21.15.16:0/3086246199 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13cc0108e0 con 0x7f13dc103920 2024-08-31T06:05:21.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.183+0000 7f13d3e00640 1 -- 172.21.15.16:0/3086246199 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc018880 con 0x7f13dc103920 2024-08-31T06:05:21.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.184+0000 7f13e2600640 1 -- 172.21.15.16:0/3086246199 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f13bc001060 con 0x7f13dc103920 2024-08-31T06:05:21.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.187+0000 7f13d3e00640 1 -- 172.21.15.16:0/3086246199 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f13cc010430 con 0x7f13dc103920 2024-08-31T06:05:21.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.188+0000 7f13d3e00640 1 --2- 172.21.15.16:0/3086246199 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f13c00b4c90 0x7f13c00b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:21.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.188+0000 7f13d3e00640 1 -- 172.21.15.16:0/3086246199 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7f13cc0e5f90 con 0x7f13dc103920 2024-08-31T06:05:21.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.189+0000 7f13daa00640 1 --2- 172.21.15.16:0/3086246199 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f13c00b4c90 0x7f13c00b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:21.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.189+0000 7f13d3e00640 1 --2- 172.21.15.16:0/3086246199 >> [v2:172.21.15.16:6810/81681815,v1:172.21.15.16:6811/81681815] conn(0x7f13c00be6a0 0x7f13c00c0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:21.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.189+0000 7f13d3e00640 1 -- 172.21.15.16:0/3086246199 --> [v2:172.21.15.16:6810/81681815,v1:172.21.15.16:6811/81681815] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f13cc015e80 con 0x7f13c00be6a0 2024-08-31T06:05:21.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.189+0000 7f13d3e00640 1 -- 172.21.15.16:0/3086246199 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f13cc0e6340 con 0x7f13dc103920 2024-08-31T06:05:21.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.189+0000 7f13dbe00640 1 --2- 172.21.15.16:0/3086246199 >> [v2:172.21.15.16:6810/81681815,v1:172.21.15.16:6811/81681815] conn(0x7f13c00be6a0 0x7f13c00c0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:21.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.189+0000 7f13daa00640 1 --2- 172.21.15.16:0/3086246199 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f13c00b4c90 0x7f13c00b71a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f13c802a9e0 tx=0x7f13c8005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:21.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.190+0000 7f13dbe00640 1 --2- 172.21.15.16:0/3086246199 >> [v2:172.21.15.16:6810/81681815,v1:172.21.15.16:6811/81681815] conn(0x7f13c00be6a0 0x7f13c00c0af0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:21.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.191+0000 7f13d3e00640 1 -- 172.21.15.16:0/3086246199 <== osd.3 v2:172.21.15.16:6810/81681815 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f13cc015e80 con 0x7f13c00be6a0 2024-08-31T06:05:21.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.206+0000 7f13e2600640 1 -- 172.21.15.16:0/3086246199 --> [v2:172.21.15.16:6810/81681815,v1:172.21.15.16:6811/81681815] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f13bc002e00 con 0x7f13c00be6a0 2024-08-31T06:05:21.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.207+0000 7f13d3e00640 1 -- 172.21.15.16:0/3086246199 <== osd.3 v2:172.21.15.16:6810/81681815 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f13bc002e00 con 0x7f13c00be6a0 2024-08-31T06:05:21.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.207+0000 7f13e2600640 1 -- 172.21.15.16:0/3086246199 >> [v2:172.21.15.16:6810/81681815,v1:172.21.15.16:6811/81681815] conn(0x7f13c00be6a0 msgr2=0x7f13c00c0af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:21.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.207+0000 7f13e2600640 1 --2- 172.21.15.16:0/3086246199 >> [v2:172.21.15.16:6810/81681815,v1:172.21.15.16:6811/81681815] conn(0x7f13c00be6a0 0x7f13c00c0af0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:21.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.207+0000 7f13e2600640 1 -- 172.21.15.16:0/3086246199 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f13c00b4c90 msgr2=0x7f13c00b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:21.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.207+0000 7f13e2600640 1 --2- 172.21.15.16:0/3086246199 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f13c00b4c90 0x7f13c00b71a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f13c802a9e0 tx=0x7f13c8005a20 comp rx=0 tx=0).stop 2024-08-31T06:05:21.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.207+0000 7f13e2600640 1 -- 172.21.15.16:0/3086246199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13dc103920 msgr2=0x7f13dc19a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:21.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.207+0000 7f13e2600640 1 --2- 172.21.15.16:0/3086246199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13dc103920 0x7f13dc19a310 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f13cc00e9f0 tx=0x7f13cc00edf0 comp rx=0 tx=0).stop 2024-08-31T06:05:21.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.207+0000 7f13dbe00640 1 -- 172.21.15.16:0/3086246199 reap_dead start 2024-08-31T06:05:21.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.208+0000 7f13e2600640 1 -- 172.21.15.16:0/3086246199 shutdown_connections 2024-08-31T06:05:21.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.208+0000 7f13e2600640 1 -- 172.21.15.16:0/3086246199 >> 172.21.15.16:0/3086246199 conn(0x7f13dc0ff330 msgr2=0x7f13dc1059b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:21.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.208+0000 7f13e2600640 1 -- 172.21.15.16:0/3086246199 shutdown_connections 2024-08-31T06:05:21.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:21.208+0000 7f13e2600640 1 -- 172.21.15.16:0/3086246199 wait complete. 2024-08-31T06:05:21.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:21 smithi155 ceph-mon[21742]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-08-31T06:05:21.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:21 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:05:21.400 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:21 smithi016 ceph-mon[17194]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-08-31T06:05:21.400 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:21 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:05:21.436 INFO:teuthology.orchestra.run.smithi016.stdout:38654705748 2024-08-31T06:05:21.437 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-08-31T06:05:21.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:21 smithi184 ceph-mon[22331]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-08-31T06:05:21.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:21 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:05:21.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.709+0000 7f84e5e00640 1 -- 172.21.15.16:0/64784649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e0103920 msgr2=0x7f84e0103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:21.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.709+0000 7f84e5e00640 1 --2- 172.21.15.16:0/64784649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e0103920 0x7f84e0103d30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f84d0009900 tx=0x7f84d002a410 comp rx=0 tx=0).stop 2024-08-31T06:05:21.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.709+0000 7f84e5e00640 1 -- 172.21.15.16:0/64784649 shutdown_connections 2024-08-31T06:05:21.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.709+0000 7f84e5e00640 1 --2- 172.21.15.16:0/64784649 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84e0109bb0 0x7f84e01103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:21.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.709+0000 7f84e5e00640 1 --2- 172.21.15.16:0/64784649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84e0109260 0x7f84e0109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:21.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.709+0000 7f84e5e00640 1 --2- 172.21.15.16:0/64784649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e0103920 0x7f84e0103d30 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:21.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.709+0000 7f84e5e00640 1 -- 172.21.15.16:0/64784649 >> 172.21.15.16:0/64784649 conn(0x7f84e00ff330 msgr2=0x7f84e0101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:21.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.709+0000 7f84e5e00640 1 -- 172.21.15.16:0/64784649 shutdown_connections 2024-08-31T06:05:21.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.709+0000 7f84e5e00640 1 -- 172.21.15.16:0/64784649 wait complete. 2024-08-31T06:05:21.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.710+0000 7f84e5e00640 1 Processor -- start 2024-08-31T06:05:21.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.710+0000 7f84e5e00640 1 -- start start 2024-08-31T06:05:21.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.711+0000 7f84e5e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e0103920 0x7f84e019a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:21.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.711+0000 7f84e5e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84e0109260 0x7f84e019a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:21.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.711+0000 7f84e5e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84e0109bb0 0x7f84e019fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:21.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.711+0000 7f84e5e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f84e0113870 con 0x7f84e0103920 2024-08-31T06:05:21.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.711+0000 7f84e5e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f84e01136f0 con 0x7f84e0109260 2024-08-31T06:05:21.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.711+0000 7f84e5e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f84e01139f0 con 0x7f84e0109bb0 2024-08-31T06:05:21.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.711+0000 7f84dea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e0103920 0x7f84e019a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:21.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.711+0000 7f84dea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e0103920 0x7f84e019a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33992/0 (socket says 172.21.15.16:33992) 2024-08-31T06:05:21.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.711+0000 7f84dea00640 1 -- 172.21.15.16:0/2297220165 learned_addr learned my addr 172.21.15.16:0/2297220165 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:05:21.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.711+0000 7f84d7e00640 1 --2- 172.21.15.16:0/2297220165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84e0109260 0x7f84e019a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:21.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.711+0000 7f84df400640 1 --2- 172.21.15.16:0/2297220165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84e0109bb0 0x7f84e019fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:21.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.712+0000 7f84dea00640 1 -- 172.21.15.16:0/2297220165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84e0109bb0 msgr2=0x7f84e019fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:21.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.712+0000 7f84dea00640 1 --2- 172.21.15.16:0/2297220165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84e0109bb0 0x7f84e019fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:21.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.712+0000 7f84dea00640 1 -- 172.21.15.16:0/2297220165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84e0109260 msgr2=0x7f84e019a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:21.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.712+0000 7f84dea00640 1 --2- 172.21.15.16:0/2297220165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84e0109260 0x7f84e019a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:21.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.712+0000 7f84dea00640 1 -- 172.21.15.16:0/2297220165 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e01a01f0 con 0x7f84e0103920 2024-08-31T06:05:21.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.712+0000 7f84d7e00640 1 --2- 172.21.15.16:0/2297220165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84e0109260 0x7f84e019a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:05:21.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.712+0000 7f84dea00640 1 --2- 172.21.15.16:0/2297220165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e0103920 0x7f84e019a280 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f84d0005940 tx=0x7f84d0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:21.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.712+0000 7f84e4a00640 1 -- 172.21.15.16:0/2297220165 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d003f030 con 0x7f84e0103920 2024-08-31T06:05:21.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.712+0000 7f84e5e00640 1 -- 172.21.15.16:0/2297220165 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84e01a0480 con 0x7f84e0103920 2024-08-31T06:05:21.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.712+0000 7f84e4a00640 1 -- 172.21.15.16:0/2297220165 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84d0003aa0 con 0x7f84e0103920 2024-08-31T06:05:21.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.713+0000 7f84e4a00640 1 -- 172.21.15.16:0/2297220165 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d0035070 con 0x7f84e0103920 2024-08-31T06:05:21.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.713+0000 7f84e5e00640 1 -- 172.21.15.16:0/2297220165 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84e01a6c40 con 0x7f84e0103920 2024-08-31T06:05:21.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.714+0000 7f84e5e00640 1 -- 172.21.15.16:0/2297220165 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f84e0108490 con 0x7f84e0103920 2024-08-31T06:05:21.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.716+0000 7f84e4a00640 1 -- 172.21.15.16:0/2297220165 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f84d0042030 con 0x7f84e0103920 2024-08-31T06:05:21.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.717+0000 7f84e4a00640 1 --2- 172.21.15.16:0/2297220165 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f84c40b4c10 0x7f84c40b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:21.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.718+0000 7f84e4a00640 1 -- 172.21.15.16:0/2297220165 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7f84d0106120 con 0x7f84e0103920 2024-08-31T06:05:21.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.718+0000 7f84d7e00640 1 --2- 172.21.15.16:0/2297220165 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f84c40b4c10 0x7f84c40b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:21.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.718+0000 7f84e4a00640 1 --2- 172.21.15.16:0/2297220165 >> [v2:172.21.15.155:6800/4241618440,v1:172.21.15.155:6801/4241618440] conn(0x7f84c40be620 0x7f84c40c0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:21.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.718+0000 7f84e4a00640 1 -- 172.21.15.16:0/2297220165 --> [v2:172.21.15.155:6800/4241618440,v1:172.21.15.155:6801/4241618440] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f84d003dec0 con 0x7f84c40be620 2024-08-31T06:05:21.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.718+0000 7f84e4a00640 1 -- 172.21.15.16:0/2297220165 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f84d01064d0 con 0x7f84e0103920 2024-08-31T06:05:21.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.719+0000 7f84d7e00640 1 --2- 172.21.15.16:0/2297220165 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f84c40b4c10 0x7f84c40b7120 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f84e0101b30 tx=0x7f84c8005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:21.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.719+0000 7f84df400640 1 --2- 172.21.15.16:0/2297220165 >> [v2:172.21.15.155:6800/4241618440,v1:172.21.15.155:6801/4241618440] conn(0x7f84c40be620 0x7f84c40c0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:21.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.719+0000 7f84df400640 1 --2- 172.21.15.16:0/2297220165 >> [v2:172.21.15.155:6800/4241618440,v1:172.21.15.155:6801/4241618440] conn(0x7f84c40be620 0x7f84c40c0a70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:21.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.720+0000 7f84e4a00640 1 -- 172.21.15.16:0/2297220165 <== osd.1 v2:172.21.15.155:6800/4241618440 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f84d003dec0 con 0x7f84c40be620 2024-08-31T06:05:21.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.736+0000 7f84e5e00640 1 -- 172.21.15.16:0/2297220165 --> [v2:172.21.15.155:6800/4241618440,v1:172.21.15.155:6801/4241618440] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f84e0104ba0 con 0x7f84c40be620 2024-08-31T06:05:21.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.737+0000 7f84e4a00640 1 -- 172.21.15.16:0/2297220165 <== osd.1 v2:172.21.15.155:6800/4241618440 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f84e0104ba0 con 0x7f84c40be620 2024-08-31T06:05:21.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.738+0000 7f84e5e00640 1 -- 172.21.15.16:0/2297220165 >> [v2:172.21.15.155:6800/4241618440,v1:172.21.15.155:6801/4241618440] conn(0x7f84c40be620 msgr2=0x7f84c40c0a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:21.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.738+0000 7f84e5e00640 1 --2- 172.21.15.16:0/2297220165 >> [v2:172.21.15.155:6800/4241618440,v1:172.21.15.155:6801/4241618440] conn(0x7f84c40be620 0x7f84c40c0a70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:21.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.738+0000 7f84e5e00640 1 -- 172.21.15.16:0/2297220165 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f84c40b4c10 msgr2=0x7f84c40b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:21.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.738+0000 7f84e5e00640 1 --2- 172.21.15.16:0/2297220165 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f84c40b4c10 0x7f84c40b7120 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f84e0101b30 tx=0x7f84c8005f90 comp rx=0 tx=0).stop 2024-08-31T06:05:21.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.738+0000 7f84e5e00640 1 -- 172.21.15.16:0/2297220165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e0103920 msgr2=0x7f84e019a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:21.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.738+0000 7f84e5e00640 1 --2- 172.21.15.16:0/2297220165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e0103920 0x7f84e019a280 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f84d0005940 tx=0x7f84d0004270 comp rx=0 tx=0).stop 2024-08-31T06:05:21.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.738+0000 7f84df400640 1 -- 172.21.15.16:0/2297220165 reap_dead start 2024-08-31T06:05:21.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.738+0000 7f84e5e00640 1 -- 172.21.15.16:0/2297220165 shutdown_connections 2024-08-31T06:05:21.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.738+0000 7f84e5e00640 1 -- 172.21.15.16:0/2297220165 >> 172.21.15.16:0/2297220165 conn(0x7f84e00ff330 msgr2=0x7f84e0107a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:21.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.738+0000 7f84e5e00640 1 -- 172.21.15.16:0/2297220165 shutdown_connections 2024-08-31T06:05:21.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:17.738+0000 7f84e5e00640 1 -- 172.21.15.16:0/2297220165 wait complete. 2024-08-31T06:05:22.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.558+0000 7f4764a00640 1 -- 172.21.15.16:0/2157854373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f476010b2a0 msgr2=0x7f476010b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:22.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.558+0000 7f4764a00640 1 --2- 172.21.15.16:0/2157854373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f476010b2a0 0x7f476010b6b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f474c009900 tx=0x7f474c02a3f0 comp rx=0 tx=0).stop 2024-08-31T06:05:22.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.559+0000 7f4764a00640 1 -- 172.21.15.16:0/2157854373 shutdown_connections 2024-08-31T06:05:22.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.559+0000 7f4764a00640 1 --2- 172.21.15.16:0/2157854373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f476010bbf0 0x7f4760112420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:22.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.559+0000 7f4764a00640 1 --2- 172.21.15.16:0/2157854373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f476010b2a0 0x7f476010b6b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:22.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.559+0000 7f4764a00640 1 --2- 172.21.15.16:0/2157854373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4760105960 0x7f4760105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:22.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.559+0000 7f4764a00640 1 -- 172.21.15.16:0/2157854373 >> 172.21.15.16:0/2157854373 conn(0x7f4760101410 msgr2=0x7f4760103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:22.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.559+0000 7f4764a00640 1 -- 172.21.15.16:0/2157854373 shutdown_connections 2024-08-31T06:05:22.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.559+0000 7f4764a00640 1 -- 172.21.15.16:0/2157854373 wait complete. 2024-08-31T06:05:22.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.560+0000 7f4764a00640 1 Processor -- start 2024-08-31T06:05:22.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.560+0000 7f4764a00640 1 -- start start 2024-08-31T06:05:22.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.560+0000 7f4764a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4760105960 0x7f476019c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:22.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.560+0000 7f4764a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f476010b2a0 0x7f476019c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:22.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.560+0000 7f4764a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f476010bbf0 0x7f47601a1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:22.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.560+0000 7f4764a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f47601158b0 con 0x7f4760105960 2024-08-31T06:05:22.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.560+0000 7f4764a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4760115730 con 0x7f476010bbf0 2024-08-31T06:05:22.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.560+0000 7f4764a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4760115a30 con 0x7f476010b2a0 2024-08-31T06:05:22.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.560+0000 7f475e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4760105960 0x7f476019c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:22.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.560+0000 7f475de00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f476010b2a0 0x7f476019c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:22.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.560+0000 7f475f200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f476010bbf0 0x7f47601a1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:22.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.560+0000 7f475e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4760105960 0x7f476019c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34106/0 (socket says 172.21.15.16:34106) 2024-08-31T06:05:22.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.560+0000 7f475e800640 1 -- 172.21.15.16:0/916001802 learned_addr learned my addr 172.21.15.16:0/916001802 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:05:22.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.561+0000 7f475f200640 1 -- 172.21.15.16:0/916001802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f476010b2a0 msgr2=0x7f476019c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:22.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.561+0000 7f475f200640 1 --2- 172.21.15.16:0/916001802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f476010b2a0 0x7f476019c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:22.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.561+0000 7f475f200640 1 -- 172.21.15.16:0/916001802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4760105960 msgr2=0x7f476019c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:22.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.561+0000 7f475f200640 1 --2- 172.21.15.16:0/916001802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4760105960 0x7f476019c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:22.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.561+0000 7f475f200640 1 -- 172.21.15.16:0/916001802 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47601a22b0 con 0x7f476010bbf0 2024-08-31T06:05:22.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.561+0000 7f475f200640 1 --2- 172.21.15.16:0/916001802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f476010bbf0 0x7f47601a1b80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f475800ec50 tx=0x7f475800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:22.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.561+0000 7f4757400640 1 -- 172.21.15.16:0/916001802 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4758003b70 con 0x7f476010bbf0 2024-08-31T06:05:22.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.561+0000 7f4757400640 1 -- 172.21.15.16:0/916001802 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4758003d10 con 0x7f476010bbf0 2024-08-31T06:05:22.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.561+0000 7f4764a00640 1 -- 172.21.15.16:0/916001802 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47601a25a0 con 0x7f476010bbf0 2024-08-31T06:05:22.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.562+0000 7f4757400640 1 -- 172.21.15.16:0/916001802 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4758010830 con 0x7f476010bbf0 2024-08-31T06:05:22.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.562+0000 7f4764a00640 1 -- 172.21.15.16:0/916001802 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47601a8d00 con 0x7f476010bbf0 2024-08-31T06:05:22.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.562+0000 7f4764a00640 1 -- 172.21.15.16:0/916001802 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f47600638f0 con 0x7f476010bbf0 2024-08-31T06:05:22.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.565+0000 7f4757400640 1 -- 172.21.15.16:0/916001802 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f47580109d0 con 0x7f476010bbf0 2024-08-31T06:05:22.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.567+0000 7f4757400640 1 --2- 172.21.15.16:0/916001802 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f47440b4c90 0x7f47440b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:22.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.567+0000 7f4757400640 1 -- 172.21.15.16:0/916001802 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7f47580e5a00 con 0x7f476010bbf0 2024-08-31T06:05:22.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.567+0000 7f475e800640 1 --2- 172.21.15.16:0/916001802 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f47440b4c90 0x7f47440b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:22.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.567+0000 7f4757400640 1 --2- 172.21.15.16:0/916001802 >> [v2:172.21.15.155:6816/2671827901,v1:172.21.15.155:6817/2671827901] conn(0x7f47440be6a0 0x7f47440c0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:22.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.567+0000 7f4757400640 1 -- 172.21.15.16:0/916001802 --> [v2:172.21.15.155:6816/2671827901,v1:172.21.15.155:6817/2671827901] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4758013ec0 con 0x7f47440be6a0 2024-08-31T06:05:22.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.567+0000 7f4757400640 1 -- 172.21.15.16:0/916001802 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f47580e5db0 con 0x7f476010bbf0 2024-08-31T06:05:22.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.567+0000 7f475de00640 1 --2- 172.21.15.16:0/916001802 >> [v2:172.21.15.155:6816/2671827901,v1:172.21.15.155:6817/2671827901] conn(0x7f47440be6a0 0x7f47440c0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:22.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.568+0000 7f475e800640 1 --2- 172.21.15.16:0/916001802 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f47440b4c90 0x7f47440b71a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f47500095f0 tx=0x7f4750005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:22.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.568+0000 7f475de00640 1 --2- 172.21.15.16:0/916001802 >> [v2:172.21.15.155:6816/2671827901,v1:172.21.15.155:6817/2671827901] conn(0x7f47440be6a0 0x7f47440c0af0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:22.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.569+0000 7f4757400640 1 -- 172.21.15.16:0/916001802 <== osd.7 v2:172.21.15.155:6816/2671827901 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4758013ec0 con 0x7f47440be6a0 2024-08-31T06:05:22.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.583+0000 7f4764a00640 1 -- 172.21.15.16:0/916001802 --> [v2:172.21.15.155:6816/2671827901,v1:172.21.15.155:6817/2671827901] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4760063d70 con 0x7f47440be6a0 2024-08-31T06:05:22.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.583+0000 7f4757400640 1 -- 172.21.15.16:0/916001802 <== osd.7 v2:172.21.15.155:6816/2671827901 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4760063d70 con 0x7f47440be6a0 2024-08-31T06:05:22.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.584+0000 7f4764a00640 1 -- 172.21.15.16:0/916001802 >> [v2:172.21.15.155:6816/2671827901,v1:172.21.15.155:6817/2671827901] conn(0x7f47440be6a0 msgr2=0x7f47440c0af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:22.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.584+0000 7f4764a00640 1 --2- 172.21.15.16:0/916001802 >> [v2:172.21.15.155:6816/2671827901,v1:172.21.15.155:6817/2671827901] conn(0x7f47440be6a0 0x7f47440c0af0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:22.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.584+0000 7f4764a00640 1 -- 172.21.15.16:0/916001802 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f47440b4c90 msgr2=0x7f47440b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:22.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.584+0000 7f4764a00640 1 --2- 172.21.15.16:0/916001802 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f47440b4c90 0x7f47440b71a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f47500095f0 tx=0x7f4750005f90 comp rx=0 tx=0).stop 2024-08-31T06:05:22.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.584+0000 7f4764a00640 1 -- 172.21.15.16:0/916001802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f476010bbf0 msgr2=0x7f47601a1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:22.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.584+0000 7f4764a00640 1 --2- 172.21.15.16:0/916001802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f476010bbf0 0x7f47601a1b80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f475800ec50 tx=0x7f475800c6a0 comp rx=0 tx=0).stop 2024-08-31T06:05:22.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.584+0000 7f475f200640 1 -- 172.21.15.16:0/916001802 reap_dead start 2024-08-31T06:05:22.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.584+0000 7f4764a00640 1 -- 172.21.15.16:0/916001802 shutdown_connections 2024-08-31T06:05:22.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.584+0000 7f4764a00640 1 -- 172.21.15.16:0/916001802 >> 172.21.15.16:0/916001802 conn(0x7f4760101410 msgr2=0x7f4760103830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:22.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.584+0000 7f4764a00640 1 -- 172.21.15.16:0/916001802 shutdown_connections 2024-08-31T06:05:22.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:22.584+0000 7f4764a00640 1 -- 172.21.15.16:0/916001802 wait complete. 2024-08-31T06:05:23.261 INFO:teuthology.orchestra.run.smithi016.stdout:231928234025 2024-08-31T06:05:23.262 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-08-31T06:05:23.312 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:23 smithi016 ceph-mon[17194]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-31T06:05:23.312 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:23 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:05:23.313 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:23 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:05:23.313 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:23 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:05:23.313 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:23 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:05:23.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:23 smithi155 ceph-mon[21742]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-31T06:05:23.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:23 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:05:23.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:23 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:05:23.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:23 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:05:23.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:23 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:05:23.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:23 smithi184 ceph-mon[22331]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-31T06:05:23.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:23 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:05:23.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:23 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:05:23.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:23 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:05:23.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:23 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:05:23.584 INFO:teuthology.orchestra.run.smithi016.stdout:111669149763 2024-08-31T06:05:23.585 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-08-31T06:05:23.920 INFO:teuthology.orchestra.run.smithi016.stdout:184683593780 2024-08-31T06:05:23.921 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-08-31T06:05:24.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:24 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]} : dispatch 2024-08-31T06:05:24.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:24 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]} : dispatch 2024-08-31T06:05:24.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:24 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]} : dispatch 2024-08-31T06:05:24.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:24 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]} : dispatch 2024-08-31T06:05:24.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:24 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]} : dispatch 2024-08-31T06:05:24.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:24 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]} : dispatch 2024-08-31T06:05:24.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:24 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]} : dispatch 2024-08-31T06:05:24.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:24 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]} : dispatch 2024-08-31T06:05:24.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:24 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]} : dispatch 2024-08-31T06:05:24.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:24 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]} : dispatch 2024-08-31T06:05:24.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:24 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]} : dispatch 2024-08-31T06:05:24.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:24 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]} : dispatch 2024-08-31T06:05:24.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:24 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]} : dispatch 2024-08-31T06:05:24.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:24 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]} : dispatch 2024-08-31T06:05:24.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:24 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]} : dispatch 2024-08-31T06:05:24.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:24 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]} : dispatch 2024-08-31T06:05:24.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:24 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]} : dispatch 2024-08-31T06:05:24.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:24 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]} : dispatch 2024-08-31T06:05:24.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:24 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]} : dispatch 2024-08-31T06:05:24.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:24 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]} : dispatch 2024-08-31T06:05:24.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:24 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]} : dispatch 2024-08-31T06:05:24.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:24 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]} : dispatch 2024-08-31T06:05:24.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:24 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]} : dispatch 2024-08-31T06:05:24.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:24 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]} : dispatch 2024-08-31T06:05:24.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:24 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]} : dispatch 2024-08-31T06:05:24.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:24 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]} : dispatch 2024-08-31T06:05:24.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:24 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]} : dispatch 2024-08-31T06:05:24.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:24 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]} : dispatch 2024-08-31T06:05:24.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:24 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]} : dispatch 2024-08-31T06:05:24.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:24 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]} : dispatch 2024-08-31T06:05:24.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:24 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]} : dispatch 2024-08-31T06:05:24.374 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:24 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]} : dispatch 2024-08-31T06:05:24.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:24 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]} : dispatch 2024-08-31T06:05:24.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:24 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]} : dispatch 2024-08-31T06:05:24.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:24 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]} : dispatch 2024-08-31T06:05:24.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:24 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]} : dispatch 2024-08-31T06:05:24.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:24 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]} : dispatch 2024-08-31T06:05:24.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:24 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]} : dispatch 2024-08-31T06:05:24.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:24 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]} : dispatch 2024-08-31T06:05:24.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:24 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]} : dispatch 2024-08-31T06:05:24.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:24 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]} : dispatch 2024-08-31T06:05:24.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:24 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]} : dispatch 2024-08-31T06:05:24.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:24 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]} : dispatch 2024-08-31T06:05:24.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:24 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]} : dispatch 2024-08-31T06:05:24.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:24 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]} : dispatch 2024-08-31T06:05:24.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:24 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]} : dispatch 2024-08-31T06:05:24.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:24 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]} : dispatch 2024-08-31T06:05:24.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:24 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]} : dispatch 2024-08-31T06:05:24.708 INFO:teuthology.orchestra.run.smithi016.stdout:257698037795 2024-08-31T06:05:24.708 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-08-31T06:05:25.051 INFO:teuthology.orchestra.run.smithi016.stdout:279172874268 2024-08-31T06:05:25.051 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-08-31T06:05:25.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:25 smithi016 ceph-mon[17194]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-31T06:05:25.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:25 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-08-31T06:05:25.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:25 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2024-08-31T06:05:25.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:25 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-08-31T06:05:25.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:25 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-08-31T06:05:25.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:25 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-08-31T06:05:25.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:25 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-08-31T06:05:25.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:25 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-08-31T06:05:25.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:25 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-08-31T06:05:25.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:25 smithi016 ceph-mon[17194]: osdmap e79: 12 total, 12 up, 12 in 2024-08-31T06:05:25.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:25 smithi155 ceph-mon[21742]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-31T06:05:25.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:25 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-08-31T06:05:25.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:25 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2024-08-31T06:05:25.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:25 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-08-31T06:05:25.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:25 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-08-31T06:05:25.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:25 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-08-31T06:05:25.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:25 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-08-31T06:05:25.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:25 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-08-31T06:05:25.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:25 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-08-31T06:05:25.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:25 smithi155 ceph-mon[21742]: osdmap e79: 12 total, 12 up, 12 in 2024-08-31T06:05:25.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:25 smithi184 ceph-mon[22331]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-31T06:05:25.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:25 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-08-31T06:05:25.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:25 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2024-08-31T06:05:25.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:25 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-08-31T06:05:25.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:25 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-08-31T06:05:25.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:25 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-08-31T06:05:25.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:25 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-08-31T06:05:25.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:25 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-08-31T06:05:25.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:25 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-08-31T06:05:25.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:25 smithi184 ceph-mon[22331]: osdmap e79: 12 total, 12 up, 12 in 2024-08-31T06:05:25.905 INFO:teuthology.orchestra.run.smithi016.stdout:60129542222 2024-08-31T06:05:25.906 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-08-31T06:05:26.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:26 smithi016 ceph-mon[17194]: osdmap e80: 12 total, 12 up, 12 in 2024-08-31T06:05:26.342 INFO:teuthology.orchestra.run.smithi016.stdout:137438953535 2024-08-31T06:05:26.342 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-08-31T06:05:26.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:26 smithi155 ceph-mon[21742]: osdmap e80: 12 total, 12 up, 12 in 2024-08-31T06:05:26.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:26 smithi184 ceph-mon[22331]: osdmap e80: 12 total, 12 up, 12 in 2024-08-31T06:05:26.879 INFO:teuthology.orchestra.run.smithi016.stdout:210453397552 2024-08-31T06:05:26.880 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-08-31T06:05:27.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:27 smithi016 ceph-mon[17194]: pgmap v309: 97 pgs: 1 peering, 96 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-08-31T06:05:27.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:27 smithi155 ceph-mon[21742]: pgmap v309: 97 pgs: 1 peering, 96 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-08-31T06:05:27.427 INFO:teuthology.orchestra.run.smithi016.stdout:158913790008 2024-08-31T06:05:27.428 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-08-31T06:05:27.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:27 smithi184 ceph-mon[22331]: pgmap v309: 97 pgs: 1 peering, 96 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-08-31T06:05:27.800 INFO:teuthology.orchestra.run.smithi016.stdout:77309411402 2024-08-31T06:05:27.800 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-08-31T06:05:28.223 INFO:teuthology.orchestra.run.smithi016.stdout:300647710745 2024-08-31T06:05:28.224 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-08-31T06:05:28.362 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:05:28.413 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:05:28.427 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:05:28.434 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:05:28.434 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:05:28.436 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:05:28.444 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:05:28.450 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:05:28.460 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:05:29.087 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:29 smithi016 ceph-mon[17194]: pgmap v310: 97 pgs: 1 peering, 96 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 13 B/s, 0 objects/s recovering 2024-08-31T06:05:29.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.268+0000 7fbc1f200640 1 -- 172.21.15.16:0/1902235509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc201099e0 msgr2=0x7fbc20110210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:29.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.268+0000 7fbc1f200640 1 --2- 172.21.15.16:0/1902235509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc201099e0 0x7fbc20110210 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbc18009ca0 tx=0x7fbc1802a8c0 comp rx=0 tx=0).stop 2024-08-31T06:05:29.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.268+0000 7fbc1f200640 1 -- 172.21.15.16:0/1902235509 shutdown_connections 2024-08-31T06:05:29.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.268+0000 7fbc1f200640 1 --2- 172.21.15.16:0/1902235509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc20109090 0x7fbc201094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:29.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.268+0000 7fbc1f200640 1 --2- 172.21.15.16:0/1902235509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc201099e0 0x7fbc20110210 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:29.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.268+0000 7fbc1f200640 1 --2- 172.21.15.16:0/1902235509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc20103750 0x7fbc20103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:29.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.268+0000 7fbc1f200640 1 -- 172.21.15.16:0/1902235509 >> 172.21.15.16:0/1902235509 conn(0x7fbc200ff1e0 msgr2=0x7fbc20101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:29.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.268+0000 7fbc1f200640 1 -- 172.21.15.16:0/1902235509 shutdown_connections 2024-08-31T06:05:29.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.268+0000 7fbc1f200640 1 -- 172.21.15.16:0/1902235509 wait complete. 2024-08-31T06:05:29.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.269+0000 7fbc1f200640 1 Processor -- start 2024-08-31T06:05:29.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.269+0000 7fbc1f200640 1 -- start start 2024-08-31T06:05:29.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.270+0000 7fbc1f200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc20103750 0x7fbc2019c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:29.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.270+0000 7fbc1f200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc20109090 0x7fbc2019c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:29.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.270+0000 7fbc1f200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc201099e0 0x7fbc201a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:29.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.270+0000 7fbc1f200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbc201136b0 con 0x7fbc20103750 2024-08-31T06:05:29.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.270+0000 7fbc1f200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbc20113530 con 0x7fbc20109090 2024-08-31T06:05:29.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.270+0000 7fbc1f200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbc20113830 con 0x7fbc201099e0 2024-08-31T06:05:29.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.270+0000 7fbc1de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc20103750 0x7fbc2019c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:29.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.270+0000 7fbc1de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc20103750 0x7fbc2019c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56618/0 (socket says 172.21.15.16:56618) 2024-08-31T06:05:29.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.270+0000 7fbc1de00640 1 -- 172.21.15.16:0/2437707679 learned_addr learned my addr 172.21.15.16:0/2437707679 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:05:29.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.270+0000 7fbc1e800640 1 --2- 172.21.15.16:0/2437707679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc201099e0 0x7fbc201a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:29.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.270+0000 7fbc1d400640 1 --2- 172.21.15.16:0/2437707679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc20109090 0x7fbc2019c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:29.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.271+0000 7fbc1de00640 1 -- 172.21.15.16:0/2437707679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc201099e0 msgr2=0x7fbc201a1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:29.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.271+0000 7fbc1de00640 1 --2- 172.21.15.16:0/2437707679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc201099e0 0x7fbc201a1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:29.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.271+0000 7fbc1de00640 1 -- 172.21.15.16:0/2437707679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc20109090 msgr2=0x7fbc2019c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:29.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.271+0000 7fbc1de00640 1 --2- 172.21.15.16:0/2437707679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc20109090 0x7fbc2019c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:29.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.271+0000 7fbc1de00640 1 -- 172.21.15.16:0/2437707679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc201a21d0 con 0x7fbc20103750 2024-08-31T06:05:29.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.271+0000 7fbc1de00640 1 --2- 172.21.15.16:0/2437707679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc20103750 0x7fbc2019c260 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbc1000e9f0 tx=0x7fbc1000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:29.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.271+0000 7fbc16a00640 1 -- 172.21.15.16:0/2437707679 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1000cd80 con 0x7fbc20103750 2024-08-31T06:05:29.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.271+0000 7fbc1f200640 1 -- 172.21.15.16:0/2437707679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc201a24c0 con 0x7fbc20103750 2024-08-31T06:05:29.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.271+0000 7fbc16a00640 1 -- 172.21.15.16:0/2437707679 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc100108e0 con 0x7fbc20103750 2024-08-31T06:05:29.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.271+0000 7fbc16a00640 1 -- 172.21.15.16:0/2437707679 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc10018980 con 0x7fbc20103750 2024-08-31T06:05:29.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.271+0000 7fbc1f200640 1 -- 172.21.15.16:0/2437707679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc201a8c20 con 0x7fbc20103750 2024-08-31T06:05:29.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.273+0000 7fbc1f200640 1 -- 172.21.15.16:0/2437707679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc20103b60 con 0x7fbc20103750 2024-08-31T06:05:29.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.275+0000 7fbc16a00640 1 -- 172.21.15.16:0/2437707679 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7fbc10010430 con 0x7fbc20103750 2024-08-31T06:05:29.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.277+0000 7fbc16a00640 1 --2- 172.21.15.16:0/2437707679 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbc040b4c90 0x7fbc040b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:29.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.277+0000 7fbc16a00640 1 -- 172.21.15.16:0/2437707679 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7fbc100e5bc0 con 0x7fbc20103750 2024-08-31T06:05:29.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.277+0000 7fbc1d400640 1 --2- 172.21.15.16:0/2437707679 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbc040b4c90 0x7fbc040b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:29.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.278+0000 7fbc1d400640 1 --2- 172.21.15.16:0/2437707679 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbc040b4c90 0x7fbc040b71a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbc20102ac0 tx=0x7fbc0c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:29.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.279+0000 7fbc16a00640 1 -- 172.21.15.16:0/2437707679 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fbc100ac420 con 0x7fbc20103750 2024-08-31T06:05:29.337 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:05:29.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:29 smithi155 ceph-mon[21742]: pgmap v310: 97 pgs: 1 peering, 96 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 13 B/s, 0 objects/s recovering 2024-08-31T06:05:29.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.383+0000 7f8515800640 1 -- 172.21.15.16:0/4139008326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8510103920 msgr2=0x7f8510103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:29.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.383+0000 7f8515800640 1 --2- 172.21.15.16:0/4139008326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8510103920 0x7f8510103d30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8500009900 tx=0x7f850002a3f0 comp rx=0 tx=0).stop 2024-08-31T06:05:29.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.383+0000 7f8515800640 1 -- 172.21.15.16:0/4139008326 shutdown_connections 2024-08-31T06:05:29.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.383+0000 7f8515800640 1 --2- 172.21.15.16:0/4139008326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8510109260 0x7f8510109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:29.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.383+0000 7f8515800640 1 --2- 172.21.15.16:0/4139008326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8510109bb0 0x7f85101103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:29.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.383+0000 7f8515800640 1 --2- 172.21.15.16:0/4139008326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8510103920 0x7f8510103d30 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:29.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.383+0000 7f8515800640 1 -- 172.21.15.16:0/4139008326 >> 172.21.15.16:0/4139008326 conn(0x7f85100ff3b0 msgr2=0x7f8510101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:29.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.383+0000 7f8515800640 1 -- 172.21.15.16:0/4139008326 shutdown_connections 2024-08-31T06:05:29.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.383+0000 7f8515800640 1 -- 172.21.15.16:0/4139008326 wait complete. 2024-08-31T06:05:29.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.384+0000 7f8515800640 1 Processor -- start 2024-08-31T06:05:29.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.384+0000 7f8515800640 1 -- start start 2024-08-31T06:05:29.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.385+0000 7f8515800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8510103920 0x7f8510076ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:29.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.385+0000 7f8515800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8510109260 0x7f85100754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:29.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.385+0000 7f8515800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8510109bb0 0x7f85100759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:29.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.385+0000 7f8515800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f85101138c0 con 0x7f8510109260 2024-08-31T06:05:29.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.385+0000 7f8515800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8510113740 con 0x7f8510103920 2024-08-31T06:05:29.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.385+0000 7f8515800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8510113a40 con 0x7f8510109bb0 2024-08-31T06:05:29.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.385+0000 7f850ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8510103920 0x7f8510076ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:29.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.385+0000 7f850e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8510109260 0x7f85100754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:29.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.385+0000 7f850ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8510103920 0x7f8510076ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:39452/0 (socket says 172.21.15.16:39452) 2024-08-31T06:05:29.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.385+0000 7f850ea00640 1 -- 172.21.15.16:0/162736846 learned_addr learned my addr 172.21.15.16:0/162736846 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:05:29.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.385+0000 7f850f400640 1 --2- 172.21.15.16:0/162736846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8510109bb0 0x7f85100759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:29.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.386+0000 7f850ea00640 1 -- 172.21.15.16:0/162736846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8510109bb0 msgr2=0x7f85100759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:29.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.386+0000 7f850ea00640 1 --2- 172.21.15.16:0/162736846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8510109bb0 0x7f85100759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:29.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.386+0000 7f850ea00640 1 -- 172.21.15.16:0/162736846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8510109260 msgr2=0x7f85100754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:29.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.386+0000 7f850ea00640 1 --2- 172.21.15.16:0/162736846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8510109260 0x7f85100754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:29.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.386+0000 7f850ea00640 1 -- 172.21.15.16:0/162736846 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8510076210 con 0x7f8510103920 2024-08-31T06:05:29.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.386+0000 7f850e000640 1 --2- 172.21.15.16:0/162736846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8510109260 0x7f85100754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:05:29.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.386+0000 7f850ea00640 1 --2- 172.21.15.16:0/162736846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8510103920 0x7f8510076ec0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8500005ab0 tx=0x7f8500003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:29.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.386+0000 7f8507400640 1 -- 172.21.15.16:0/162736846 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850003f070 con 0x7f8510103920 2024-08-31T06:05:29.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.386+0000 7f8515800640 1 -- 172.21.15.16:0/162736846 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85101acbb0 con 0x7f8510103920 2024-08-31T06:05:29.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.386+0000 7f8507400640 1 -- 172.21.15.16:0/162736846 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8500031b60 con 0x7f8510103920 2024-08-31T06:05:29.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.387+0000 7f8507400640 1 -- 172.21.15.16:0/162736846 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8500035030 con 0x7f8510103920 2024-08-31T06:05:29.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.387+0000 7f8515800640 1 -- 172.21.15.16:0/162736846 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85101ad090 con 0x7f8510103920 2024-08-31T06:05:29.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.388+0000 7f8515800640 1 -- 172.21.15.16:0/162736846 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8510104ba0 con 0x7f8510103920 2024-08-31T06:05:29.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.390+0000 7f8507400640 1 -- 172.21.15.16:0/162736846 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f8500042070 con 0x7f8510103920 2024-08-31T06:05:29.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.392+0000 7f8507400640 1 --2- 172.21.15.16:0/162736846 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f84f40b4c10 0x7f84f40b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:29.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.392+0000 7f8507400640 1 -- 172.21.15.16:0/162736846 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f8500105ea0 con 0x7f8510103920 2024-08-31T06:05:29.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.392+0000 7f850e000640 1 --2- 172.21.15.16:0/162736846 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f84f40b4c10 0x7f84f40b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:29.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.393+0000 7f850e000640 1 --2- 172.21.15.16:0/162736846 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f84f40b4c10 0x7f84f40b7120 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8510102c00 tx=0x7f84fc005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:29.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.394+0000 7f8507400640 1 -- 172.21.15.16:0/162736846 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f85000cc700 con 0x7f8510103920 2024-08-31T06:05:29.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:29 smithi184 ceph-mon[22331]: pgmap v310: 97 pgs: 1 peering, 96 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 13 B/s, 0 objects/s recovering 2024-08-31T06:05:29.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.497+0000 7fbc1f200640 1 -- 172.21.15.16:0/2437707679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fbc201a8fd0 con 0x7fbc20103750 2024-08-31T06:05:29.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.497+0000 7fbc16a00640 1 -- 172.21.15.16:0/2437707679 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbc100b0be0 con 0x7fbc20103750 2024-08-31T06:05:29.500 INFO:teuthology.orchestra.run.smithi016.stdout:111669149764 2024-08-31T06:05:29.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.500+0000 7fbc1f200640 1 -- 172.21.15.16:0/2437707679 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbc040b4c90 msgr2=0x7fbc040b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:29.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.500+0000 7fbc1f200640 1 --2- 172.21.15.16:0/2437707679 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbc040b4c90 0x7fbc040b71a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbc20102ac0 tx=0x7fbc0c005f90 comp rx=0 tx=0).stop 2024-08-31T06:05:29.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.500+0000 7fbc1f200640 1 -- 172.21.15.16:0/2437707679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc20103750 msgr2=0x7fbc2019c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:29.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.500+0000 7fbc1f200640 1 --2- 172.21.15.16:0/2437707679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc20103750 0x7fbc2019c260 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbc1000e9f0 tx=0x7fbc1000eeb0 comp rx=0 tx=0).stop 2024-08-31T06:05:29.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.500+0000 7fbc1f200640 1 -- 172.21.15.16:0/2437707679 shutdown_connections 2024-08-31T06:05:29.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.500+0000 7fbc1f200640 1 --2- 172.21.15.16:0/2437707679 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbc040b4c90 0x7fbc040b71a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:29.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.500+0000 7fbc1f200640 1 --2- 172.21.15.16:0/2437707679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc201099e0 0x7fbc201a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:29.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.500+0000 7fbc1f200640 1 --2- 172.21.15.16:0/2437707679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc20109090 0x7fbc2019c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:29.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.500+0000 7fbc1f200640 1 --2- 172.21.15.16:0/2437707679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc20103750 0x7fbc2019c260 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:29.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.500+0000 7fbc1f200640 1 -- 172.21.15.16:0/2437707679 >> 172.21.15.16:0/2437707679 conn(0x7fbc200ff1e0 msgr2=0x7fbc20108740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:29.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.500+0000 7fbc1f200640 1 -- 172.21.15.16:0/2437707679 shutdown_connections 2024-08-31T06:05:29.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.501+0000 7fbc1f200640 1 -- 172.21.15.16:0/2437707679 wait complete. 2024-08-31T06:05:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.607+0000 7f8515800640 1 -- 172.21.15.16:0/162736846 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f8510103d30 con 0x7f8510103920 2024-08-31T06:05:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.608+0000 7f8507400640 1 -- 172.21.15.16:0/162736846 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f85000c6e20 con 0x7f8510103920 2024-08-31T06:05:29.610 INFO:teuthology.orchestra.run.smithi016.stdout:38654705750 2024-08-31T06:05:29.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.610+0000 7f8515800640 1 -- 172.21.15.16:0/162736846 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f84f40b4c10 msgr2=0x7f84f40b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:29.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.610+0000 7f8515800640 1 --2- 172.21.15.16:0/162736846 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f84f40b4c10 0x7f84f40b7120 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8510102c00 tx=0x7f84fc005f90 comp rx=0 tx=0).stop 2024-08-31T06:05:29.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.610+0000 7f8515800640 1 -- 172.21.15.16:0/162736846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8510103920 msgr2=0x7f8510076ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:29.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.610+0000 7f8515800640 1 --2- 172.21.15.16:0/162736846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8510103920 0x7f8510076ec0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8500005ab0 tx=0x7f8500003c10 comp rx=0 tx=0).stop 2024-08-31T06:05:29.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.610+0000 7f8515800640 1 -- 172.21.15.16:0/162736846 shutdown_connections 2024-08-31T06:05:29.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.610+0000 7f8515800640 1 --2- 172.21.15.16:0/162736846 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f84f40b4c10 0x7f84f40b7120 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:29.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.610+0000 7f8515800640 1 --2- 172.21.15.16:0/162736846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8510109bb0 0x7f85100759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:29.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.610+0000 7f8515800640 1 --2- 172.21.15.16:0/162736846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8510109260 0x7f85100754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:29.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.610+0000 7f8515800640 1 --2- 172.21.15.16:0/162736846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8510103920 0x7f8510076ec0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:29.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.610+0000 7f8515800640 1 -- 172.21.15.16:0/162736846 >> 172.21.15.16:0/162736846 conn(0x7f85100ff3b0 msgr2=0x7f85101088b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:29.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.611+0000 7f8515800640 1 -- 172.21.15.16:0/162736846 shutdown_connections 2024-08-31T06:05:29.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:29.611+0000 7f8515800640 1 -- 172.21.15.16:0/162736846 wait complete. 2024-08-31T06:05:30.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:30 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/2437707679' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-08-31T06:05:30.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:30 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/162736846' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-08-31T06:05:30.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:30 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/2437707679' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-08-31T06:05:30.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:30 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/162736846' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-08-31T06:05:30.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:30 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/2437707679' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-08-31T06:05:30.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:30 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/162736846' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-08-31T06:05:30.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.663+0000 7ffbd3200640 1 -- 172.21.15.16:0/3016840084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbd41099e0 msgr2=0x7ffbd4110210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.663+0000 7ffbd3200640 1 --2- 172.21.15.16:0/3016840084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbd41099e0 0x7ffbd4110210 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc009ca0 tx=0x7ffbcc02a8c0 comp rx=0 tx=0).stop 2024-08-31T06:05:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.663+0000 7ffbd3200640 1 -- 172.21.15.16:0/3016840084 shutdown_connections 2024-08-31T06:05:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.663+0000 7ffbd3200640 1 --2- 172.21.15.16:0/3016840084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbd41099e0 0x7ffbd4110210 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.663+0000 7ffbd3200640 1 --2- 172.21.15.16:0/3016840084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbd4109090 0x7ffbd41094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.663+0000 7ffbd3200640 1 --2- 172.21.15.16:0/3016840084 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbd4103750 0x7ffbd4103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.663+0000 7ffbd3200640 1 -- 172.21.15.16:0/3016840084 >> 172.21.15.16:0/3016840084 conn(0x7ffbd40ff1e0 msgr2=0x7ffbd4101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.663+0000 7ffbd3200640 1 -- 172.21.15.16:0/3016840084 shutdown_connections 2024-08-31T06:05:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.664+0000 7ffbd3200640 1 -- 172.21.15.16:0/3016840084 wait complete. 2024-08-31T06:05:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.665+0000 7ffbd3200640 1 Processor -- start 2024-08-31T06:05:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.665+0000 7ffbd3200640 1 -- start start 2024-08-31T06:05:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.665+0000 7ffbd3200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbd4103750 0x7ffbd419c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.665+0000 7ffbd3200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbd4109090 0x7ffbd419c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.665+0000 7ffbd3200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbd41099e0 0x7ffbd41a1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.665+0000 7ffbd3200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd41136e0 con 0x7ffbd41099e0 2024-08-31T06:05:30.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.665+0000 7ffbd3200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd4113560 con 0x7ffbd4103750 2024-08-31T06:05:30.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.665+0000 7ffbd3200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd4113860 con 0x7ffbd4109090 2024-08-31T06:05:30.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.666+0000 7ffbd1e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbd4103750 0x7ffbd419c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:30.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.666+0000 7ffbd1e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbd4103750 0x7ffbd419c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:39460/0 (socket says 172.21.15.16:39460) 2024-08-31T06:05:30.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.666+0000 7ffbd1e00640 1 -- 172.21.15.16:0/890980528 learned_addr learned my addr 172.21.15.16:0/890980528 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:05:30.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.666+0000 7ffbd2800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbd41099e0 0x7ffbd41a1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:30.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.666+0000 7ffbd1400640 1 --2- 172.21.15.16:0/890980528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbd4109090 0x7ffbd419c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:30.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.666+0000 7ffbd2800640 1 -- 172.21.15.16:0/890980528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbd4109090 msgr2=0x7ffbd419c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:30.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.666+0000 7ffbd2800640 1 --2- 172.21.15.16:0/890980528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbd4109090 0x7ffbd419c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:30.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.666+0000 7ffbd2800640 1 -- 172.21.15.16:0/890980528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbd4103750 msgr2=0x7ffbd419c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:30.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.666+0000 7ffbd2800640 1 --2- 172.21.15.16:0/890980528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbd4103750 0x7ffbd419c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:30.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.666+0000 7ffbd2800640 1 -- 172.21.15.16:0/890980528 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbd41a21c0 con 0x7ffbd41099e0 2024-08-31T06:05:30.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.666+0000 7ffbd1e00640 1 --2- 172.21.15.16:0/890980528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbd4103750 0x7ffbd419c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:05:30.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.667+0000 7ffbd2800640 1 --2- 172.21.15.16:0/890980528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbd41099e0 0x7ffbd41a1a90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc005e30 tx=0x7ffbcc0317c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:30.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.667+0000 7ffbcaa00640 1 -- 172.21.15.16:0/890980528 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc003b70 con 0x7ffbd41099e0 2024-08-31T06:05:30.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.667+0000 7ffbd3200640 1 -- 172.21.15.16:0/890980528 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbd41a2450 con 0x7ffbd41099e0 2024-08-31T06:05:30.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.667+0000 7ffbcaa00640 1 -- 172.21.15.16:0/890980528 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbcc003d10 con 0x7ffbd41099e0 2024-08-31T06:05:30.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.667+0000 7ffbcaa00640 1 -- 172.21.15.16:0/890980528 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc02fa00 con 0x7ffbd41099e0 2024-08-31T06:05:30.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.667+0000 7ffbd3200640 1 -- 172.21.15.16:0/890980528 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbd41a8c10 con 0x7ffbd41099e0 2024-08-31T06:05:30.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.668+0000 7ffbd3200640 1 -- 172.21.15.16:0/890980528 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbd4103b60 con 0x7ffbd41099e0 2024-08-31T06:05:30.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.673+0000 7ffbcaa00640 1 -- 172.21.15.16:0/890980528 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7ffbcc004580 con 0x7ffbd41099e0 2024-08-31T06:05:30.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.674+0000 7ffbcaa00640 1 --2- 172.21.15.16:0/890980528 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ffbb80b4c10 0x7ffbb80b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:30.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.674+0000 7ffbcaa00640 1 -- 172.21.15.16:0/890980528 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7ffbcc106670 con 0x7ffbd41099e0 2024-08-31T06:05:30.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.674+0000 7ffbd1e00640 1 --2- 172.21.15.16:0/890980528 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ffbb80b4c10 0x7ffbb80b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:30.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.675+0000 7ffbcaa00640 1 -- 172.21.15.16:0/890980528 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7ffbcc0cd610 con 0x7ffbd41099e0 2024-08-31T06:05:30.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.675+0000 7ffbd1e00640 1 --2- 172.21.15.16:0/890980528 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ffbb80b4c10 0x7ffbb80b7120 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4005fd0 tx=0x7ffbc400b4a0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:30.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.884+0000 7ffbd3200640 1 -- 172.21.15.16:0/890980528 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7ffbd419d380 con 0x7ffbd41099e0 2024-08-31T06:05:30.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.884+0000 7ffbcaa00640 1 -- 172.21.15.16:0/890980528 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ffbd419d380 con 0x7ffbd41099e0 2024-08-31T06:05:30.887 INFO:teuthology.orchestra.run.smithi016.stdout:210453397553 2024-08-31T06:05:30.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.888+0000 7ffbd3200640 1 -- 172.21.15.16:0/890980528 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ffbb80b4c10 msgr2=0x7ffbb80b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:30.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.888+0000 7ffbd3200640 1 --2- 172.21.15.16:0/890980528 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ffbb80b4c10 0x7ffbb80b7120 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4005fd0 tx=0x7ffbc400b4a0 comp rx=0 tx=0).stop 2024-08-31T06:05:30.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.888+0000 7ffbd3200640 1 -- 172.21.15.16:0/890980528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbd41099e0 msgr2=0x7ffbd41a1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:30.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.888+0000 7ffbd3200640 1 --2- 172.21.15.16:0/890980528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbd41099e0 0x7ffbd41a1a90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc005e30 tx=0x7ffbcc0317c0 comp rx=0 tx=0).stop 2024-08-31T06:05:30.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.888+0000 7ffbd3200640 1 -- 172.21.15.16:0/890980528 shutdown_connections 2024-08-31T06:05:30.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.888+0000 7ffbd3200640 1 --2- 172.21.15.16:0/890980528 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ffbb80b4c10 0x7ffbb80b7120 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:30.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.888+0000 7ffbd3200640 1 --2- 172.21.15.16:0/890980528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbd4109090 0x7ffbd419c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:30.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.888+0000 7ffbd3200640 1 --2- 172.21.15.16:0/890980528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbd41099e0 0x7ffbd41a1a90 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:30.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.888+0000 7ffbd3200640 1 --2- 172.21.15.16:0/890980528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbd4103750 0x7ffbd419c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:30.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.888+0000 7ffbd3200640 1 -- 172.21.15.16:0/890980528 >> 172.21.15.16:0/890980528 conn(0x7ffbd40ff1e0 msgr2=0x7ffbd4108660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:30.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.888+0000 7ffbd3200640 1 -- 172.21.15.16:0/890980528 shutdown_connections 2024-08-31T06:05:30.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:30.888+0000 7ffbd3200640 1 -- 172.21.15.16:0/890980528 wait complete. 2024-08-31T06:05:31.318 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:31 smithi016 ceph-mon[17194]: pgmap v311: 97 pgs: 1 peering, 96 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s; 143 B/s, 1 objects/s recovering 2024-08-31T06:05:31.318 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:31 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/890980528' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-08-31T06:05:31.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:31 smithi155 ceph-mon[21742]: pgmap v311: 97 pgs: 1 peering, 96 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s; 143 B/s, 1 objects/s recovering 2024-08-31T06:05:31.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:31 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/890980528' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-08-31T06:05:31.373 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:31 smithi184 ceph-mon[22331]: pgmap v311: 97 pgs: 1 peering, 96 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s; 143 B/s, 1 objects/s recovering 2024-08-31T06:05:31.374 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:31 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/890980528' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-08-31T06:05:31.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:31.810+0000 7fe6fce00640 1 -- 172.21.15.16:0/3664866423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6f8114320 msgr2=0x7fe6f8116740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:31.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:31.810+0000 7fe6fce00640 1 --2- 172.21.15.16:0/3664866423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6f8114320 0x7fe6f8116740 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0009be0 tx=0x7fe6f002a6c0 comp rx=0 tx=0).stop 2024-08-31T06:05:31.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:31.811+0000 7fe6fce00640 1 -- 172.21.15.16:0/3664866423 shutdown_connections 2024-08-31T06:05:31.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:31.811+0000 7fe6fce00640 1 --2- 172.21.15.16:0/3664866423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6f8114320 0x7fe6f8116740 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:31.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:31.811+0000 7fe6fce00640 1 --2- 172.21.15.16:0/3664866423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6f8078030 0x7fe6f8113de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:31.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:31.811+0000 7fe6fce00640 1 --2- 172.21.15.16:0/3664866423 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6f80776e0 0x7fe6f8077af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:31.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:31.811+0000 7fe6fce00640 1 -- 172.21.15.16:0/3664866423 >> 172.21.15.16:0/3664866423 conn(0x7fe6f8101410 msgr2=0x7fe6f8103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:31.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:31.811+0000 7fe6fce00640 1 -- 172.21.15.16:0/3664866423 shutdown_connections 2024-08-31T06:05:31.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:31.811+0000 7fe6fce00640 1 -- 172.21.15.16:0/3664866423 wait complete. 2024-08-31T06:05:31.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:31.812+0000 7fe6fce00640 1 Processor -- start 2024-08-31T06:05:31.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:31.812+0000 7fe6fce00640 1 -- start start 2024-08-31T06:05:31.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:31.813+0000 7fe6fce00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6f80776e0 0x7fe6f81a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:31.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:31.813+0000 7fe6fce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6f8078030 0x7fe6f81a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:31.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:31.813+0000 7fe6fce00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6f8114320 0x7fe6f81a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:31.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:31.813+0000 7fe6fce00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f8119ca0 con 0x7fe6f8078030 2024-08-31T06:05:31.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:31.813+0000 7fe6fce00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f8119b20 con 0x7fe6f8114320 2024-08-31T06:05:31.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:31.813+0000 7fe6fce00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f8119e20 con 0x7fe6f80776e0 2024-08-31T06:05:31.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:31.813+0000 7fe6f6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6f8078030 0x7fe6f81a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:31.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:31.813+0000 7fe6f6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6f8078030 0x7fe6f81a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56668/0 (socket says 172.21.15.16:56668) 2024-08-31T06:05:31.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:31.813+0000 7fe6f6a00640 1 -- 172.21.15.16:0/2187414177 learned_addr learned my addr 172.21.15.16:0/2187414177 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:05:31.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:31.813+0000 7fe6f7e00640 1 --2- 172.21.15.16:0/2187414177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6f8114320 0x7fe6f81a5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:31.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:31.813+0000 7fe6f7400640 1 --2- 172.21.15.16:0/2187414177 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6f80776e0 0x7fe6f81a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:31.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:31.814+0000 7fe6f6a00640 1 -- 172.21.15.16:0/2187414177 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6f80776e0 msgr2=0x7fe6f81a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:31.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:31.814+0000 7fe6f6a00640 1 --2- 172.21.15.16:0/2187414177 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6f80776e0 0x7fe6f81a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:31.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:31.814+0000 7fe6f6a00640 1 -- 172.21.15.16:0/2187414177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6f8114320 msgr2=0x7fe6f81a5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:31.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:31.814+0000 7fe6f6a00640 1 --2- 172.21.15.16:0/2187414177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6f8114320 0x7fe6f81a5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:31.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:31.814+0000 7fe6f6a00640 1 -- 172.21.15.16:0/2187414177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6f81a6580 con 0x7fe6f8078030 2024-08-31T06:05:31.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:31.814+0000 7fe6f7e00640 1 --2- 172.21.15.16:0/2187414177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6f8114320 0x7fe6f81a5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:05:31.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:31.814+0000 7fe6f6a00640 1 --2- 172.21.15.16:0/2187414177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6f8078030 0x7fe6f81a0bb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe6e400e9c0 tx=0x7fe6e400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:31.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:31.814+0000 7fe6efe00640 1 -- 172.21.15.16:0/2187414177 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e400ccc0 con 0x7fe6f8078030 2024-08-31T06:05:31.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:31.814+0000 7fe6efe00640 1 -- 172.21.15.16:0/2187414177 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6e40108e0 con 0x7fe6f8078030 2024-08-31T06:05:31.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:31.814+0000 7fe6fce00640 1 -- 172.21.15.16:0/2187414177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6f81a6870 con 0x7fe6f8078030 2024-08-31T06:05:31.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:31.814+0000 7fe6efe00640 1 -- 172.21.15.16:0/2187414177 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e40187e0 con 0x7fe6f8078030 2024-08-31T06:05:31.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:31.814+0000 7fe6fce00640 1 -- 172.21.15.16:0/2187414177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6f81acfd0 con 0x7fe6f8078030 2024-08-31T06:05:31.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:31.815+0000 7fe6fce00640 1 -- 172.21.15.16:0/2187414177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6f8077af0 con 0x7fe6f8078030 2024-08-31T06:05:31.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:31.819+0000 7fe6efe00640 1 -- 172.21.15.16:0/2187414177 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7fe6e4010430 con 0x7fe6f8078030 2024-08-31T06:05:31.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:31.820+0000 7fe6efe00640 1 --2- 172.21.15.16:0/2187414177 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe6dc0b4c90 0x7fe6dc0b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:31.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:31.820+0000 7fe6efe00640 1 -- 172.21.15.16:0/2187414177 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7fe6e40e5ae0 con 0x7fe6f8078030 2024-08-31T06:05:31.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:31.821+0000 7fe6f7400640 1 --2- 172.21.15.16:0/2187414177 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe6dc0b4c90 0x7fe6dc0b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:31.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:31.821+0000 7fe6f7400640 1 --2- 172.21.15.16:0/2187414177 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe6dc0b4c90 0x7fe6dc0b71a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8009bd0 tx=0x7fe6e8005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:31.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:31.821+0000 7fe6efe00640 1 -- 172.21.15.16:0/2187414177 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fe6e40ac340 con 0x7fe6f8078030 2024-08-31T06:05:32.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.024+0000 7fe6fce00640 1 -- 172.21.15.16:0/2187414177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fe6f81a1740 con 0x7fe6f8078030 2024-08-31T06:05:32.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.024+0000 7fe6efe00640 1 -- 172.21.15.16:0/2187414177 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe6e40b0b00 con 0x7fe6f8078030 2024-08-31T06:05:32.027 INFO:teuthology.orchestra.run.smithi016.stdout:60129542224 2024-08-31T06:05:32.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.027+0000 7fe6fce00640 1 -- 172.21.15.16:0/2187414177 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe6dc0b4c90 msgr2=0x7fe6dc0b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:32.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.027+0000 7fe6fce00640 1 --2- 172.21.15.16:0/2187414177 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe6dc0b4c90 0x7fe6dc0b71a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8009bd0 tx=0x7fe6e8005f90 comp rx=0 tx=0).stop 2024-08-31T06:05:32.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.027+0000 7fe6fce00640 1 -- 172.21.15.16:0/2187414177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6f8078030 msgr2=0x7fe6f81a0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:32.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.027+0000 7fe6fce00640 1 --2- 172.21.15.16:0/2187414177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6f8078030 0x7fe6f81a0bb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe6e400e9c0 tx=0x7fe6e400ee80 comp rx=0 tx=0).stop 2024-08-31T06:05:32.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.027+0000 7fe6fce00640 1 -- 172.21.15.16:0/2187414177 shutdown_connections 2024-08-31T06:05:32.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.027+0000 7fe6fce00640 1 --2- 172.21.15.16:0/2187414177 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe6dc0b4c90 0x7fe6dc0b71a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:32.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.027+0000 7fe6fce00640 1 --2- 172.21.15.16:0/2187414177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6f8114320 0x7fe6f81a5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:32.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.027+0000 7fe6fce00640 1 --2- 172.21.15.16:0/2187414177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6f8078030 0x7fe6f81a0bb0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:32.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.027+0000 7fe6fce00640 1 --2- 172.21.15.16:0/2187414177 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6f80776e0 0x7fe6f81a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:32.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.027+0000 7fe6fce00640 1 -- 172.21.15.16:0/2187414177 >> 172.21.15.16:0/2187414177 conn(0x7fe6f8101410 msgr2=0x7fe6f8116360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:32.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.027+0000 7fe6fce00640 1 -- 172.21.15.16:0/2187414177 shutdown_connections 2024-08-31T06:05:32.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.028+0000 7fe6fce00640 1 -- 172.21.15.16:0/2187414177 wait complete. 2024-08-31T06:05:32.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.050+0000 7efe51400640 1 -- 172.21.15.16:0/2498164019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe4c109b90 msgr2=0x7efe4c1103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:32.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.050+0000 7efe51400640 1 --2- 172.21.15.16:0/2498164019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe4c109b90 0x7efe4c1103c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7efe44009ca0 tx=0x7efe4402a8c0 comp rx=0 tx=0).stop 2024-08-31T06:05:32.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.050+0000 7efe51400640 1 -- 172.21.15.16:0/2498164019 shutdown_connections 2024-08-31T06:05:32.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.050+0000 7efe51400640 1 --2- 172.21.15.16:0/2498164019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe4c109b90 0x7efe4c1103c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:32.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.050+0000 7efe51400640 1 --2- 172.21.15.16:0/2498164019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe4c109240 0x7efe4c109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:32.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.050+0000 7efe51400640 1 --2- 172.21.15.16:0/2498164019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe4c103900 0x7efe4c103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:32.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.051+0000 7efe51400640 1 -- 172.21.15.16:0/2498164019 >> 172.21.15.16:0/2498164019 conn(0x7efe4c07b5f0 msgr2=0x7efe4c07ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:32.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.051+0000 7efe51400640 1 -- 172.21.15.16:0/2498164019 shutdown_connections 2024-08-31T06:05:32.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.051+0000 7efe51400640 1 -- 172.21.15.16:0/2498164019 wait complete. 2024-08-31T06:05:32.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.052+0000 7efe51400640 1 Processor -- start 2024-08-31T06:05:32.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.052+0000 7efe51400640 1 -- start start 2024-08-31T06:05:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.052+0000 7efe51400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe4c103900 0x7efe4c19c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.052+0000 7efe51400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe4c109240 0x7efe4c19c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.053+0000 7efe51400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe4c109b90 0x7efe4c1a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.053+0000 7efe51400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efe4c1138a0 con 0x7efe4c103900 2024-08-31T06:05:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.053+0000 7efe51400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efe4c113720 con 0x7efe4c109240 2024-08-31T06:05:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.053+0000 7efe51400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efe4c113a20 con 0x7efe4c109b90 2024-08-31T06:05:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.053+0000 7efe4a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe4c103900 0x7efe4c19c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.053+0000 7efe4a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe4c103900 0x7efe4c19c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56682/0 (socket says 172.21.15.16:56682) 2024-08-31T06:05:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.053+0000 7efe4a800640 1 -- 172.21.15.16:0/1968127727 learned_addr learned my addr 172.21.15.16:0/1968127727 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:05:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.053+0000 7efe49e00640 1 --2- 172.21.15.16:0/1968127727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe4c109240 0x7efe4c19c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.053+0000 7efe4b200640 1 --2- 172.21.15.16:0/1968127727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe4c109b90 0x7efe4c1a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:32.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.053+0000 7efe4a800640 1 -- 172.21.15.16:0/1968127727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe4c109b90 msgr2=0x7efe4c1a1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:32.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.053+0000 7efe4a800640 1 --2- 172.21.15.16:0/1968127727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe4c109b90 0x7efe4c1a1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:32.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.053+0000 7efe4a800640 1 -- 172.21.15.16:0/1968127727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe4c109240 msgr2=0x7efe4c19c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:32.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.053+0000 7efe4a800640 1 --2- 172.21.15.16:0/1968127727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe4c109240 0x7efe4c19c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:32.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.053+0000 7efe4a800640 1 -- 172.21.15.16:0/1968127727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe4c1a23c0 con 0x7efe4c103900 2024-08-31T06:05:32.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.054+0000 7efe4a800640 1 --2- 172.21.15.16:0/1968127727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe4c103900 0x7efe4c19c450 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7efe3c00e9f0 tx=0x7efe3c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:32.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.054+0000 7efe43400640 1 -- 172.21.15.16:0/1968127727 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3c00cd80 con 0x7efe4c103900 2024-08-31T06:05:32.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.054+0000 7efe51400640 1 -- 172.21.15.16:0/1968127727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe4c1a26b0 con 0x7efe4c103900 2024-08-31T06:05:32.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.054+0000 7efe43400640 1 -- 172.21.15.16:0/1968127727 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe3c0108e0 con 0x7efe4c103900 2024-08-31T06:05:32.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.054+0000 7efe43400640 1 -- 172.21.15.16:0/1968127727 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3c018980 con 0x7efe4c103900 2024-08-31T06:05:32.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.054+0000 7efe51400640 1 -- 172.21.15.16:0/1968127727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe4c1a8e10 con 0x7efe4c103900 2024-08-31T06:05:32.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.056+0000 7efe51400640 1 -- 172.21.15.16:0/1968127727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe4c103d10 con 0x7efe4c103900 2024-08-31T06:05:32.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.060+0000 7efe43400640 1 -- 172.21.15.16:0/1968127727 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7efe3c010430 con 0x7efe4c103900 2024-08-31T06:05:32.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.061+0000 7efe43400640 1 --2- 172.21.15.16:0/1968127727 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7efe300b4c90 0x7efe300b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:32.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.062+0000 7efe43400640 1 -- 172.21.15.16:0/1968127727 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7efe3c0e5c00 con 0x7efe4c103900 2024-08-31T06:05:32.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.062+0000 7efe49e00640 1 --2- 172.21.15.16:0/1968127727 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7efe300b4c90 0x7efe300b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:32.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.063+0000 7efe49e00640 1 --2- 172.21.15.16:0/1968127727 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7efe300b4c90 0x7efe300b71a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efe380095f0 tx=0x7efe38005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:32.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.063+0000 7efe43400640 1 -- 172.21.15.16:0/1968127727 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7efe3c0acab0 con 0x7efe4c103900 2024-08-31T06:05:32.252 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:32 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/2187414177' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-08-31T06:05:32.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.265+0000 7efe51400640 1 -- 172.21.15.16:0/1968127727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7efe4c1a91e0 con 0x7efe4c103900 2024-08-31T06:05:32.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.266+0000 7efe43400640 1 -- 172.21.15.16:0/1968127727 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7efe3c0ac3b0 con 0x7efe4c103900 2024-08-31T06:05:32.269 INFO:teuthology.orchestra.run.smithi016.stdout:279172874271 2024-08-31T06:05:32.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.268+0000 7efe51400640 1 -- 172.21.15.16:0/1968127727 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7efe300b4c90 msgr2=0x7efe300b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:32.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.268+0000 7efe51400640 1 --2- 172.21.15.16:0/1968127727 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7efe300b4c90 0x7efe300b71a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efe380095f0 tx=0x7efe38005f90 comp rx=0 tx=0).stop 2024-08-31T06:05:32.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.268+0000 7efe51400640 1 -- 172.21.15.16:0/1968127727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe4c103900 msgr2=0x7efe4c19c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:32.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.268+0000 7efe51400640 1 --2- 172.21.15.16:0/1968127727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe4c103900 0x7efe4c19c450 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7efe3c00e9f0 tx=0x7efe3c00eeb0 comp rx=0 tx=0).stop 2024-08-31T06:05:32.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.269+0000 7efe51400640 1 -- 172.21.15.16:0/1968127727 shutdown_connections 2024-08-31T06:05:32.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.269+0000 7efe51400640 1 --2- 172.21.15.16:0/1968127727 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7efe300b4c90 0x7efe300b71a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:32.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.269+0000 7efe51400640 1 --2- 172.21.15.16:0/1968127727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe4c109b90 0x7efe4c1a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:32.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.269+0000 7efe51400640 1 --2- 172.21.15.16:0/1968127727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe4c109240 0x7efe4c19c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:32.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.269+0000 7efe51400640 1 --2- 172.21.15.16:0/1968127727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe4c103900 0x7efe4c19c450 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:32.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.269+0000 7efe51400640 1 -- 172.21.15.16:0/1968127727 >> 172.21.15.16:0/1968127727 conn(0x7efe4c07b5f0 msgr2=0x7efe4c1070b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:32.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.269+0000 7efe51400640 1 -- 172.21.15.16:0/1968127727 shutdown_connections 2024-08-31T06:05:32.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.269+0000 7efe51400640 1 -- 172.21.15.16:0/1968127727 wait complete. 2024-08-31T06:05:32.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:32 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/2187414177' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-08-31T06:05:32.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:32 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/2187414177' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-08-31T06:05:33.153 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:33 smithi016 ceph-mon[17194]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s; 143 B/s, 1 objects/s recovering 2024-08-31T06:05:33.153 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:33 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/1968127727' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-08-31T06:05:33.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:33 smithi155 ceph-mon[21742]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s; 143 B/s, 1 objects/s recovering 2024-08-31T06:05:33.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:33 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/1968127727' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-08-31T06:05:33.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:33 smithi184 ceph-mon[22331]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s; 143 B/s, 1 objects/s recovering 2024-08-31T06:05:33.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:33 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/1968127727' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-08-31T06:05:33.489 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:05:33.523 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705748 got 38654705750 for osd.0 2024-08-31T06:05:33.523 DEBUG:teuthology.parallel:result is None 2024-08-31T06:05:33.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.936+0000 7f9fb1e00640 1 -- 172.21.15.16:0/3555516407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fac078030 msgr2=0x7f9fac113fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:33.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.936+0000 7f9fb1e00640 1 --2- 172.21.15.16:0/3555516407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fac078030 0x7f9fac113fa0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9f94009900 tx=0x7f9f9402a410 comp rx=0 tx=0).stop 2024-08-31T06:05:33.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.936+0000 7f9fb1e00640 1 -- 172.21.15.16:0/3555516407 shutdown_connections 2024-08-31T06:05:33.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.936+0000 7f9fb1e00640 1 --2- 172.21.15.16:0/3555516407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fac1144e0 0x7f9fac116900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:33.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.936+0000 7f9fb1e00640 1 --2- 172.21.15.16:0/3555516407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fac078030 0x7f9fac113fa0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:33.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.936+0000 7f9fb1e00640 1 --2- 172.21.15.16:0/3555516407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9fac0776e0 0x7f9fac077af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:33.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.936+0000 7f9fb1e00640 1 -- 172.21.15.16:0/3555516407 >> 172.21.15.16:0/3555516407 conn(0x7f9fac1015b0 msgr2=0x7f9fac103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:33.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.936+0000 7f9fb1e00640 1 -- 172.21.15.16:0/3555516407 shutdown_connections 2024-08-31T06:05:33.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.936+0000 7f9fb1e00640 1 -- 172.21.15.16:0/3555516407 wait complete. 2024-08-31T06:05:33.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.937+0000 7f9fb1e00640 1 Processor -- start 2024-08-31T06:05:33.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.937+0000 7f9fb1e00640 1 -- start start 2024-08-31T06:05:33.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.938+0000 7f9fb1e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9fac0776e0 0x7f9fac1a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:33.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.938+0000 7f9fb1e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fac078030 0x7f9fac1a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:33.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.938+0000 7f9fb1e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fac1144e0 0x7f9fac1a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:33.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.938+0000 7f9fb1e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9fac119e90 con 0x7f9fac078030 2024-08-31T06:05:33.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.938+0000 7f9fb1e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9fac119d10 con 0x7f9fac0776e0 2024-08-31T06:05:33.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.938+0000 7f9fb1e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9fac11a010 con 0x7f9fac1144e0 2024-08-31T06:05:33.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.938+0000 7f9fa3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fac078030 0x7f9fac1a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:33.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.938+0000 7f9faaa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9fac0776e0 0x7f9fac1a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:33.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.938+0000 7f9fa3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fac078030 0x7f9fac1a0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56690/0 (socket says 172.21.15.16:56690) 2024-08-31T06:05:33.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.938+0000 7f9faaa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9fac0776e0 0x7f9fac1a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:39494/0 (socket says 172.21.15.16:39494) 2024-08-31T06:05:33.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.938+0000 7f9fa3e00640 1 -- 172.21.15.16:0/4229046749 learned_addr learned my addr 172.21.15.16:0/4229046749 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:05:33.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.938+0000 7f9fab400640 1 --2- 172.21.15.16:0/4229046749 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fac1144e0 0x7f9fac1a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:33.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.939+0000 7f9fa3e00640 1 -- 172.21.15.16:0/4229046749 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fac1144e0 msgr2=0x7f9fac1a6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:33.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.939+0000 7f9fa3e00640 1 --2- 172.21.15.16:0/4229046749 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fac1144e0 0x7f9fac1a6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:33.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.939+0000 7f9fa3e00640 1 -- 172.21.15.16:0/4229046749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9fac0776e0 msgr2=0x7f9fac1a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:33.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.939+0000 7f9fa3e00640 1 --2- 172.21.15.16:0/4229046749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9fac0776e0 0x7f9fac1a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:33.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.939+0000 7f9fa3e00640 1 -- 172.21.15.16:0/4229046749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fac1a6790 con 0x7f9fac078030 2024-08-31T06:05:33.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.939+0000 7f9fa3e00640 1 --2- 172.21.15.16:0/4229046749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fac078030 0x7f9fac1a0dc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9f94005ab0 tx=0x7f9f94003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:33.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.939+0000 7f9fb0a00640 1 -- 172.21.15.16:0/4229046749 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9403f070 con 0x7f9fac078030 2024-08-31T06:05:33.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.939+0000 7f9fb0a00640 1 -- 172.21.15.16:0/4229046749 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f94031b50 con 0x7f9fac078030 2024-08-31T06:05:33.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.939+0000 7f9fb0a00640 1 -- 172.21.15.16:0/4229046749 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f94035030 con 0x7f9fac078030 2024-08-31T06:05:33.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.939+0000 7f9fb1e00640 1 -- 172.21.15.16:0/4229046749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fac1a6a20 con 0x7f9fac078030 2024-08-31T06:05:33.803 INFO:teuthology.orchestra.run.smithi016.stdout:257698037798 2024-08-31T06:05:33.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.939+0000 7f9fb1e00640 1 -- 172.21.15.16:0/4229046749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fac1ad1e0 con 0x7f9fac078030 2024-08-31T06:05:33.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.940+0000 7f9fb1e00640 1 -- 172.21.15.16:0/4229046749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fac077af0 con 0x7f9fac078030 2024-08-31T06:05:33.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.943+0000 7f9fb0a00640 1 -- 172.21.15.16:0/4229046749 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f9f94042070 con 0x7f9fac078030 2024-08-31T06:05:33.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.944+0000 7f9fb0a00640 1 --2- 172.21.15.16:0/4229046749 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9f900b4c90 0x7f9f900b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:33.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.944+0000 7f9fb0a00640 1 -- 172.21.15.16:0/4229046749 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f9f941055e0 con 0x7f9fac078030 2024-08-31T06:05:33.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.944+0000 7f9faaa00640 1 --2- 172.21.15.16:0/4229046749 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9f900b4c90 0x7f9f900b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:33.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.945+0000 7f9faaa00640 1 --2- 172.21.15.16:0/4229046749 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9f900b4c90 0x7f9f900b71a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c009bd0 tx=0x7f9f9c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:33.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:32.948+0000 7f9fb0a00640 1 -- 172.21.15.16:0/4229046749 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f9f940cbe40 con 0x7f9fac078030 2024-08-31T06:05:33.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:33.150+0000 7f9fb1e00640 1 -- 172.21.15.16:0/4229046749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f9fac10f0e0 con 0x7f9fac078030 2024-08-31T06:05:33.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:33.151+0000 7f9fb0a00640 1 -- 172.21.15.16:0/4229046749 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9f940d0600 con 0x7f9fac078030 2024-08-31T06:05:33.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:33.153+0000 7f9fb1e00640 1 -- 172.21.15.16:0/4229046749 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9f900b4c90 msgr2=0x7f9f900b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:33.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:33.153+0000 7f9fb1e00640 1 --2- 172.21.15.16:0/4229046749 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9f900b4c90 0x7f9f900b71a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c009bd0 tx=0x7f9f9c005f90 comp rx=0 tx=0).stop 2024-08-31T06:05:33.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:33.153+0000 7f9fb1e00640 1 -- 172.21.15.16:0/4229046749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fac078030 msgr2=0x7f9fac1a0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:33.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:33.153+0000 7f9fb1e00640 1 --2- 172.21.15.16:0/4229046749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fac078030 0x7f9fac1a0dc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9f94005ab0 tx=0x7f9f94003b90 comp rx=0 tx=0).stop 2024-08-31T06:05:33.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:33.153+0000 7f9fb1e00640 1 -- 172.21.15.16:0/4229046749 shutdown_connections 2024-08-31T06:05:33.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:33.153+0000 7f9fb1e00640 1 --2- 172.21.15.16:0/4229046749 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9f900b4c90 0x7f9f900b71a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:33.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:33.153+0000 7f9fb1e00640 1 --2- 172.21.15.16:0/4229046749 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fac1144e0 0x7f9fac1a6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:33.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:33.153+0000 7f9fb1e00640 1 --2- 172.21.15.16:0/4229046749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fac078030 0x7f9fac1a0dc0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:33.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:33.153+0000 7f9fb1e00640 1 --2- 172.21.15.16:0/4229046749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9fac0776e0 0x7f9fac1a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:33.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:33.153+0000 7f9fb1e00640 1 -- 172.21.15.16:0/4229046749 >> 172.21.15.16:0/4229046749 conn(0x7f9fac1015b0 msgr2=0x7f9fac116540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:33.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:33.153+0000 7f9fb1e00640 1 -- 172.21.15.16:0/4229046749 shutdown_connections 2024-08-31T06:05:33.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:33.154+0000 7f9fb1e00640 1 -- 172.21.15.16:0/4229046749 wait complete. 2024-08-31T06:05:34.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:34 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/4229046749' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-08-31T06:05:34.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:34 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/4229046749' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-08-31T06:05:34.433 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:05:34.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:34 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/4229046749' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-08-31T06:05:34.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.477+0000 7fe679200640 1 -- 172.21.15.16:0/4087881423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe674109b90 msgr2=0x7fe6741103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:34.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.477+0000 7fe679200640 1 --2- 172.21.15.16:0/4087881423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe674109b90 0x7fe6741103c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe66c009ca0 tx=0x7fe66c02a8c0 comp rx=0 tx=0).stop 2024-08-31T06:05:34.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.477+0000 7fe679200640 1 -- 172.21.15.16:0/4087881423 shutdown_connections 2024-08-31T06:05:34.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.477+0000 7fe679200640 1 --2- 172.21.15.16:0/4087881423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe674109b90 0x7fe6741103c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:34.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.477+0000 7fe679200640 1 --2- 172.21.15.16:0/4087881423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe674109240 0x7fe674109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:34.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.477+0000 7fe679200640 1 --2- 172.21.15.16:0/4087881423 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe674103900 0x7fe674103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:34.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.477+0000 7fe679200640 1 -- 172.21.15.16:0/4087881423 >> 172.21.15.16:0/4087881423 conn(0x7fe67407b5f0 msgr2=0x7fe67407ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:34.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.477+0000 7fe679200640 1 -- 172.21.15.16:0/4087881423 shutdown_connections 2024-08-31T06:05:34.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.477+0000 7fe679200640 1 -- 172.21.15.16:0/4087881423 wait complete. 2024-08-31T06:05:34.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.478+0000 7fe679200640 1 Processor -- start 2024-08-31T06:05:34.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.478+0000 7fe679200640 1 -- start start 2024-08-31T06:05:34.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.479+0000 7fe679200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe674103900 0x7fe67419c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:34.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.479+0000 7fe679200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe674109240 0x7fe67419c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:34.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.479+0000 7fe679200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe674109b90 0x7fe6741a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:34.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.479+0000 7fe679200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe674113850 con 0x7fe674103900 2024-08-31T06:05:34.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.479+0000 7fe679200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe6741136d0 con 0x7fe674109240 2024-08-31T06:05:34.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.479+0000 7fe679200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe6741139d0 con 0x7fe674109b90 2024-08-31T06:05:34.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.479+0000 7fe671e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe674103900 0x7fe67419c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:34.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.479+0000 7fe671e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe674103900 0x7fe67419c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56718/0 (socket says 172.21.15.16:56718) 2024-08-31T06:05:34.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.479+0000 7fe671e00640 1 -- 172.21.15.16:0/2592077846 learned_addr learned my addr 172.21.15.16:0/2592077846 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:05:34.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.479+0000 7fe671400640 1 --2- 172.21.15.16:0/2592077846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe674109240 0x7fe67419c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:34.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.479+0000 7fe672800640 1 --2- 172.21.15.16:0/2592077846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe674109b90 0x7fe6741a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:34.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.480+0000 7fe671e00640 1 -- 172.21.15.16:0/2592077846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe674109b90 msgr2=0x7fe6741a1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:34.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.480+0000 7fe671e00640 1 --2- 172.21.15.16:0/2592077846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe674109b90 0x7fe6741a1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:34.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.480+0000 7fe671e00640 1 -- 172.21.15.16:0/2592077846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe674109240 msgr2=0x7fe67419c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:34.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.480+0000 7fe671e00640 1 --2- 172.21.15.16:0/2592077846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe674109240 0x7fe67419c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:34.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.480+0000 7fe671e00640 1 -- 172.21.15.16:0/2592077846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6741a2330 con 0x7fe674103900 2024-08-31T06:05:34.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.480+0000 7fe671400640 1 --2- 172.21.15.16:0/2592077846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe674109240 0x7fe67419c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:05:34.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.480+0000 7fe671e00640 1 --2- 172.21.15.16:0/2592077846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe674103900 0x7fe67419c3c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe66400e9f0 tx=0x7fe66400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:34.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.480+0000 7fe66aa00640 1 -- 172.21.15.16:0/2592077846 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66400cd80 con 0x7fe674103900 2024-08-31T06:05:34.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.480+0000 7fe66aa00640 1 -- 172.21.15.16:0/2592077846 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6640054d0 con 0x7fe674103900 2024-08-31T06:05:34.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.480+0000 7fe66aa00640 1 -- 172.21.15.16:0/2592077846 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe664010690 con 0x7fe674103900 2024-08-31T06:05:34.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.480+0000 7fe679200640 1 -- 172.21.15.16:0/2592077846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6741a2620 con 0x7fe674103900 2024-08-31T06:05:34.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.480+0000 7fe679200640 1 -- 172.21.15.16:0/2592077846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6741a8d80 con 0x7fe674103900 2024-08-31T06:05:34.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.481+0000 7fe679200640 1 -- 172.21.15.16:0/2592077846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe674103d10 con 0x7fe674103900 2024-08-31T06:05:34.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.484+0000 7fe66aa00640 1 -- 172.21.15.16:0/2592077846 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7fe664004e90 con 0x7fe674103900 2024-08-31T06:05:34.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.486+0000 7fe66aa00640 1 --2- 172.21.15.16:0/2592077846 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe6580b4c90 0x7fe6580b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:34.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.487+0000 7fe66aa00640 1 -- 172.21.15.16:0/2592077846 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7fe6640e57a0 con 0x7fe674103900 2024-08-31T06:05:34.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.487+0000 7fe671400640 1 --2- 172.21.15.16:0/2592077846 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe6580b4c90 0x7fe6580b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:34.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.487+0000 7fe671400640 1 --2- 172.21.15.16:0/2592077846 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe6580b4c90 0x7fe6580b71a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe674106e90 tx=0x7fe660005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:34.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.490+0000 7fe66aa00640 1 -- 172.21.15.16:0/2592077846 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fe6640ac6b0 con 0x7fe674103900 2024-08-31T06:05:34.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.690+0000 7fe679200640 1 -- 172.21.15.16:0/2592077846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fe67419d4f0 con 0x7fe674103900 2024-08-31T06:05:34.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.690+0000 7fe66aa00640 1 -- 172.21.15.16:0/2592077846 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe6640abfb0 con 0x7fe674103900 2024-08-31T06:05:34.693 INFO:teuthology.orchestra.run.smithi016.stdout:184683593783 2024-08-31T06:05:34.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.693+0000 7fe679200640 1 -- 172.21.15.16:0/2592077846 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe6580b4c90 msgr2=0x7fe6580b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:34.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.693+0000 7fe679200640 1 --2- 172.21.15.16:0/2592077846 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe6580b4c90 0x7fe6580b71a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe674106e90 tx=0x7fe660005f90 comp rx=0 tx=0).stop 2024-08-31T06:05:34.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.693+0000 7fe679200640 1 -- 172.21.15.16:0/2592077846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe674103900 msgr2=0x7fe67419c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:34.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.693+0000 7fe679200640 1 --2- 172.21.15.16:0/2592077846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe674103900 0x7fe67419c3c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe66400e9f0 tx=0x7fe66400edf0 comp rx=0 tx=0).stop 2024-08-31T06:05:34.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.693+0000 7fe679200640 1 -- 172.21.15.16:0/2592077846 shutdown_connections 2024-08-31T06:05:34.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.693+0000 7fe679200640 1 --2- 172.21.15.16:0/2592077846 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe6580b4c90 0x7fe6580b71a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:34.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.693+0000 7fe679200640 1 --2- 172.21.15.16:0/2592077846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe674109b90 0x7fe6741a1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:34.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.693+0000 7fe679200640 1 --2- 172.21.15.16:0/2592077846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe674109240 0x7fe67419c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:34.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.693+0000 7fe679200640 1 --2- 172.21.15.16:0/2592077846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe674103900 0x7fe67419c3c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:34.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.693+0000 7fe679200640 1 -- 172.21.15.16:0/2592077846 >> 172.21.15.16:0/2592077846 conn(0x7fe67407b5f0 msgr2=0x7fe674105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:34.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.693+0000 7fe679200640 1 -- 172.21.15.16:0/2592077846 shutdown_connections 2024-08-31T06:05:34.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.693+0000 7fe679200640 1 -- 172.21.15.16:0/2592077846 wait complete. 2024-08-31T06:05:35.015 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874268 got 279172874271 for osd.10 2024-08-31T06:05:35.015 DEBUG:teuthology.parallel:result is None 2024-08-31T06:05:35.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.289+0000 7f2777400640 1 -- 172.21.15.16:0/3269823293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27701141d0 msgr2=0x7f27701166a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:35.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.289+0000 7f2777400640 1 --2- 172.21.15.16:0/3269823293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27701141d0 0x7f27701166a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2768009ca0 tx=0x7f276802a8c0 comp rx=0 tx=0).stop 2024-08-31T06:05:35.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.289+0000 7f2777400640 1 -- 172.21.15.16:0/3269823293 shutdown_connections 2024-08-31T06:05:35.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.289+0000 7f2777400640 1 --2- 172.21.15.16:0/3269823293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2770075b70 0x7f2770075f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:35.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.289+0000 7f2777400640 1 --2- 172.21.15.16:0/3269823293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27701141d0 0x7f27701166a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:35.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.289+0000 7f2777400640 1 --2- 172.21.15.16:0/3269823293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2770077120 0x7f2770075550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:35.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.289+0000 7f2777400640 1 -- 172.21.15.16:0/3269823293 >> 172.21.15.16:0/3269823293 conn(0x7f277007b5f0 msgr2=0x7f277007ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:35.292 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:35 smithi016 ceph-mon[17194]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 4 op/s; 124 B/s, 1 objects/s recovering 2024-08-31T06:05:35.293 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:35 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/2592077846' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-08-31T06:05:35.293 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:35 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/2077359506' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-08-31T06:05:35.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.289+0000 7f2777400640 1 -- 172.21.15.16:0/3269823293 shutdown_connections 2024-08-31T06:05:35.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.289+0000 7f2777400640 1 -- 172.21.15.16:0/3269823293 wait complete. 2024-08-31T06:05:35.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.291+0000 7f2777400640 1 Processor -- start 2024-08-31T06:05:35.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.291+0000 7f2777400640 1 -- start start 2024-08-31T06:05:35.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.292+0000 7f2777400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2770075b70 0x7f27701a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:35.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.292+0000 7f2777400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2770077120 0x7f27701a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:35.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.292+0000 7f2777400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27701141d0 0x7f27701a6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:35.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.292+0000 7f2777400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2770119e60 con 0x7f2770077120 2024-08-31T06:05:35.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.292+0000 7f2777400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2770119ce0 con 0x7f27701141d0 2024-08-31T06:05:35.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.292+0000 7f2777400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2770119fe0 con 0x7f2770075b70 2024-08-31T06:05:35.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.292+0000 7f276fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2770077120 0x7f27701a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:35.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.292+0000 7f2774a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2770075b70 0x7f27701a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:35.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.292+0000 7f276fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2770077120 0x7f27701a0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56746/0 (socket says 172.21.15.16:56746) 2024-08-31T06:05:35.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.292+0000 7f276fe00640 1 -- 172.21.15.16:0/3733194985 learned_addr learned my addr 172.21.15.16:0/3733194985 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:05:35.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.292+0000 7f2775400640 1 --2- 172.21.15.16:0/3733194985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27701141d0 0x7f27701a6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:35.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.292+0000 7f2774a00640 1 -- 172.21.15.16:0/3733194985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27701141d0 msgr2=0x7f27701a6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:35.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.293+0000 7f2774a00640 1 --2- 172.21.15.16:0/3733194985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27701141d0 0x7f27701a6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:35.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.293+0000 7f2774a00640 1 -- 172.21.15.16:0/3733194985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2770077120 msgr2=0x7f27701a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:35.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.293+0000 7f2774a00640 1 --2- 172.21.15.16:0/3733194985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2770077120 0x7f27701a0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:35.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.293+0000 7f2774a00640 1 -- 172.21.15.16:0/3733194985 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27701a6740 con 0x7f2770075b70 2024-08-31T06:05:35.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.293+0000 7f276fe00640 1 --2- 172.21.15.16:0/3733194985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2770077120 0x7f27701a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:05:35.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.293+0000 7f2774a00640 1 --2- 172.21.15.16:0/3733194985 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2770075b70 0x7f27701a07d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f276400e9f0 tx=0x7f276400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:35.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.293+0000 7f276d600640 1 -- 172.21.15.16:0/3733194985 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276400ccc0 con 0x7f2770075b70 2024-08-31T06:05:35.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.293+0000 7f2777400640 1 -- 172.21.15.16:0/3733194985 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27701a6a30 con 0x7f2770075b70 2024-08-31T06:05:35.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.293+0000 7f276d600640 1 -- 172.21.15.16:0/3733194985 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27640108e0 con 0x7f2770075b70 2024-08-31T06:05:35.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.293+0000 7f276d600640 1 -- 172.21.15.16:0/3733194985 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2764018880 con 0x7f2770075b70 2024-08-31T06:05:35.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.293+0000 7f2777400640 1 -- 172.21.15.16:0/3733194985 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27701ad190 con 0x7f2770075b70 2024-08-31T06:05:35.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.294+0000 7f2777400640 1 -- 172.21.15.16:0/3733194985 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2770075f80 con 0x7f2770075b70 2024-08-31T06:05:35.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.297+0000 7f276d600640 1 -- 172.21.15.16:0/3733194985 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f2764010430 con 0x7f2770075b70 2024-08-31T06:05:35.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.298+0000 7f276d600640 1 --2- 172.21.15.16:0/3733194985 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f27580b4c10 0x7f27580b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:35.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.298+0000 7f276d600640 1 -- 172.21.15.16:0/3733194985 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f27640e5d10 con 0x7f2770075b70 2024-08-31T06:05:35.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.298+0000 7f276fe00640 1 --2- 172.21.15.16:0/3733194985 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f27580b4c10 0x7f27580b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:35.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.299+0000 7f276fe00640 1 --2- 172.21.15.16:0/3733194985 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f27580b4c10 0x7f27580b7120 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f27701169f0 tx=0x7f275c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:35.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.300+0000 7f276d600640 1 -- 172.21.15.16:0/3733194985 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f27640acbc0 con 0x7f2770075b70 2024-08-31T06:05:35.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:35 smithi155 ceph-mon[21742]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 4 op/s; 124 B/s, 1 objects/s recovering 2024-08-31T06:05:35.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:35 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/2592077846' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-08-31T06:05:35.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:35 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/2077359506' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-08-31T06:05:35.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:35 smithi184 ceph-mon[22331]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 4 op/s; 124 B/s, 1 objects/s recovering 2024-08-31T06:05:35.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:35 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/2592077846' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-08-31T06:05:35.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:35 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/2077359506' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-08-31T06:05:35.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.500+0000 7f2777400640 1 -- 172.21.15.16:0/3733194985 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f27701ad4c0 con 0x7f2770075b70 2024-08-31T06:05:35.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.501+0000 7f276d600640 1 -- 172.21.15.16:0/3733194985 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f27640b5100 con 0x7f2770075b70 2024-08-31T06:05:35.503 INFO:teuthology.orchestra.run.smithi016.stdout:231928234028 2024-08-31T06:05:35.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.504+0000 7f2777400640 1 -- 172.21.15.16:0/3733194985 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f27580b4c10 msgr2=0x7f27580b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:35.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.504+0000 7f2777400640 1 --2- 172.21.15.16:0/3733194985 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f27580b4c10 0x7f27580b7120 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f27701169f0 tx=0x7f275c005f90 comp rx=0 tx=0).stop 2024-08-31T06:05:35.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.504+0000 7f2777400640 1 -- 172.21.15.16:0/3733194985 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2770075b70 msgr2=0x7f27701a07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:35.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.504+0000 7f2777400640 1 --2- 172.21.15.16:0/3733194985 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2770075b70 0x7f27701a07d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f276400e9f0 tx=0x7f276400eeb0 comp rx=0 tx=0).stop 2024-08-31T06:05:35.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.504+0000 7f2777400640 1 -- 172.21.15.16:0/3733194985 shutdown_connections 2024-08-31T06:05:35.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.504+0000 7f2777400640 1 --2- 172.21.15.16:0/3733194985 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f27580b4c10 0x7f27580b7120 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:35.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.504+0000 7f2777400640 1 --2- 172.21.15.16:0/3733194985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27701141d0 0x7f27701a6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:35.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.504+0000 7f2777400640 1 --2- 172.21.15.16:0/3733194985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2770077120 0x7f27701a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:35.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.504+0000 7f2777400640 1 --2- 172.21.15.16:0/3733194985 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2770075b70 0x7f27701a07d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:35.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.504+0000 7f2777400640 1 -- 172.21.15.16:0/3733194985 >> 172.21.15.16:0/3733194985 conn(0x7f277007b5f0 msgr2=0x7f2770112aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:35.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.504+0000 7f2777400640 1 -- 172.21.15.16:0/3733194985 shutdown_connections 2024-08-31T06:05:35.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.504+0000 7f2777400640 1 -- 172.21.15.16:0/3733194985 wait complete. 2024-08-31T06:05:35.860 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149763 got 111669149764 for osd.3 2024-08-31T06:05:35.861 DEBUG:teuthology.parallel:result is None 2024-08-31T06:05:36.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:36 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/3733194985' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-08-31T06:05:36.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:36 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/3733194985' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-08-31T06:05:36.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:36 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/3733194985' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-08-31T06:05:36.603 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542222 got 60129542224 for osd.1 2024-08-31T06:05:36.603 DEBUG:teuthology.parallel:result is None 2024-08-31T06:05:37.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.811+0000 7f82b1c00640 1 -- 172.21.15.16:0/1194538073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82ac109090 msgr2=0x7f82ac1094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:37.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.811+0000 7f82b1c00640 1 --2- 172.21.15.16:0/1194538073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82ac109090 0x7f82ac1094a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8294009900 tx=0x7f829402a410 comp rx=0 tx=0).stop 2024-08-31T06:05:37.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.811+0000 7f82b1c00640 1 -- 172.21.15.16:0/1194538073 shutdown_connections 2024-08-31T06:05:37.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.811+0000 7f82b1c00640 1 --2- 172.21.15.16:0/1194538073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82ac1099e0 0x7f82ac110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:37.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.811+0000 7f82b1c00640 1 --2- 172.21.15.16:0/1194538073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82ac109090 0x7f82ac1094a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:37.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.811+0000 7f82b1c00640 1 --2- 172.21.15.16:0/1194538073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82ac103750 0x7f82ac103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:37.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.811+0000 7f82b1c00640 1 -- 172.21.15.16:0/1194538073 >> 172.21.15.16:0/1194538073 conn(0x7f82ac0ff1a0 msgr2=0x7f82ac1015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:37.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.811+0000 7f82b1c00640 1 -- 172.21.15.16:0/1194538073 shutdown_connections 2024-08-31T06:05:37.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.811+0000 7f82b1c00640 1 -- 172.21.15.16:0/1194538073 wait complete. 2024-08-31T06:05:37.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.812+0000 7f82b1c00640 1 Processor -- start 2024-08-31T06:05:37.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.813+0000 7f82b1c00640 1 -- start start 2024-08-31T06:05:37.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.813+0000 7f82b1c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82ac103750 0x7f82ac19a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:37.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.813+0000 7f82b1c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82ac109090 0x7f82ac19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:37.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.813+0000 7f82b1c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82ac1099e0 0x7f82ac19f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:37.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.813+0000 7f82b1c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac113650 con 0x7f82ac109090 2024-08-31T06:05:37.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.813+0000 7f82b1c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac1134d0 con 0x7f82ac1099e0 2024-08-31T06:05:37.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.813+0000 7f82b1c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac1137d0 con 0x7f82ac103750 2024-08-31T06:05:37.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.813+0000 7f82a3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82ac109090 0x7f82ac19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:37.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.813+0000 7f82a3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82ac109090 0x7f82ac19a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56732/0 (socket says 172.21.15.16:56732) 2024-08-31T06:05:37.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.813+0000 7f82a3e00640 1 -- 172.21.15.16:0/2077359506 learned_addr learned my addr 172.21.15.16:0/2077359506 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:05:37.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.813+0000 7f82abe00640 1 --2- 172.21.15.16:0/2077359506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82ac103750 0x7f82ac19a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:37.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.814+0000 7f82b1200640 1 --2- 172.21.15.16:0/2077359506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82ac1099e0 0x7f82ac19f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:37.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.814+0000 7f82a3e00640 1 -- 172.21.15.16:0/2077359506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82ac103750 msgr2=0x7f82ac19a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:37.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.814+0000 7f82a3e00640 1 --2- 172.21.15.16:0/2077359506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82ac103750 0x7f82ac19a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:37.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.814+0000 7f82a3e00640 1 -- 172.21.15.16:0/2077359506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82ac1099e0 msgr2=0x7f82ac19f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:37.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.814+0000 7f82a3e00640 1 --2- 172.21.15.16:0/2077359506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82ac1099e0 0x7f82ac19f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:37.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.814+0000 7f82a3e00640 1 -- 172.21.15.16:0/2077359506 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82ac19fff0 con 0x7f82ac109090 2024-08-31T06:05:37.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.814+0000 7f82b1200640 1 --2- 172.21.15.16:0/2077359506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82ac1099e0 0x7f82ac19f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:05:37.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.814+0000 7f82a3e00640 1 --2- 172.21.15.16:0/2077359506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82ac109090 0x7f82ac19a620 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f829402a930 tx=0x7f8294031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:37.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.814+0000 7f82a9600640 1 -- 172.21.15.16:0/2077359506 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829403f030 con 0x7f82ac109090 2024-08-31T06:05:37.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.814+0000 7f82a9600640 1 -- 172.21.15.16:0/2077359506 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8294004490 con 0x7f82ac109090 2024-08-31T06:05:37.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.814+0000 7f82b1c00640 1 -- 172.21.15.16:0/2077359506 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82ac1a0280 con 0x7f82ac109090 2024-08-31T06:05:37.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.815+0000 7f82a9600640 1 -- 172.21.15.16:0/2077359506 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8294035070 con 0x7f82ac109090 2024-08-31T06:05:37.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.815+0000 7f82b1c00640 1 -- 172.21.15.16:0/2077359506 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82ac1a6a40 con 0x7f82ac109090 2024-08-31T06:05:37.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.816+0000 7f82b1c00640 1 -- 172.21.15.16:0/2077359506 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82ac1049d0 con 0x7f82ac109090 2024-08-31T06:05:37.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.818+0000 7f82a9600640 1 -- 172.21.15.16:0/2077359506 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f8294042030 con 0x7f82ac109090 2024-08-31T06:05:37.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.819+0000 7f82a9600640 1 --2- 172.21.15.16:0/2077359506 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f82900b4c10 0x7f82900b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:37.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.819+0000 7f82a9600640 1 -- 172.21.15.16:0/2077359506 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f8294105480 con 0x7f82ac109090 2024-08-31T06:05:37.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.820+0000 7f82abe00640 1 --2- 172.21.15.16:0/2077359506 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f82900b4c10 0x7f82900b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:37.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.820+0000 7f82abe00640 1 --2- 172.21.15.16:0/2077359506 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f82900b4c10 0x7f82900b7120 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f829c009bd0 tx=0x7f829c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:37.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:34.823+0000 7f82a9600640 1 -- 172.21.15.16:0/2077359506 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f82940cc330 con 0x7f82ac109090 2024-08-31T06:05:37.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.033+0000 7f82b1c00640 1 -- 172.21.15.16:0/2077359506 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f82ac10b7f0 con 0x7f82ac109090 2024-08-31T06:05:37.098 INFO:teuthology.orchestra.run.smithi016.stdout:137438953538 2024-08-31T06:05:37.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.033+0000 7f82a9600640 1 -- 172.21.15.16:0/2077359506 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f82940cbc30 con 0x7f82ac109090 2024-08-31T06:05:37.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.036+0000 7f82b1c00640 1 -- 172.21.15.16:0/2077359506 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f82900b4c10 msgr2=0x7f82900b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:37.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.036+0000 7f82b1c00640 1 --2- 172.21.15.16:0/2077359506 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f82900b4c10 0x7f82900b7120 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f829c009bd0 tx=0x7f829c005f90 comp rx=0 tx=0).stop 2024-08-31T06:05:37.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.036+0000 7f82b1c00640 1 -- 172.21.15.16:0/2077359506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82ac109090 msgr2=0x7f82ac19a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:37.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.036+0000 7f82b1c00640 1 --2- 172.21.15.16:0/2077359506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82ac109090 0x7f82ac19a620 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f829402a930 tx=0x7f8294031a20 comp rx=0 tx=0).stop 2024-08-31T06:05:37.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.036+0000 7f82b1c00640 1 -- 172.21.15.16:0/2077359506 shutdown_connections 2024-08-31T06:05:37.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.036+0000 7f82b1c00640 1 --2- 172.21.15.16:0/2077359506 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f82900b4c10 0x7f82900b7120 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:37.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.036+0000 7f82b1c00640 1 --2- 172.21.15.16:0/2077359506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82ac1099e0 0x7f82ac19f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:37.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.036+0000 7f82b1c00640 1 --2- 172.21.15.16:0/2077359506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82ac109090 0x7f82ac19a620 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:37.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.036+0000 7f82b1c00640 1 --2- 172.21.15.16:0/2077359506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82ac103750 0x7f82ac19a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:37.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.036+0000 7f82b1c00640 1 -- 172.21.15.16:0/2077359506 >> 172.21.15.16:0/2077359506 conn(0x7f82ac0ff1a0 msgr2=0x7f82ac1015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:37.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.036+0000 7f82b1c00640 1 -- 172.21.15.16:0/2077359506 shutdown_connections 2024-08-31T06:05:37.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:35.036+0000 7f82b1c00640 1 -- 172.21.15.16:0/2077359506 wait complete. 2024-08-31T06:05:37.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:37 smithi016 ceph-mon[17194]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 112 B/s, 1 objects/s recovering 2024-08-31T06:05:37.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:37 smithi155 ceph-mon[21742]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 112 B/s, 1 objects/s recovering 2024-08-31T06:05:37.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:37 smithi184 ceph-mon[22331]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 112 B/s, 1 objects/s recovering 2024-08-31T06:05:38.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:37.856+0000 7f6361400640 1 -- 172.21.15.16:0/3200419865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f635c0754f0 msgr2=0x7f635c075900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:38.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:37.856+0000 7f6361400640 1 --2- 172.21.15.16:0/3200419865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f635c0754f0 0x7f635c075900 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f634c009900 tx=0x7f634c02a3f0 comp rx=0 tx=0).stop 2024-08-31T06:05:38.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:37.857+0000 7f6361400640 1 -- 172.21.15.16:0/3200419865 shutdown_connections 2024-08-31T06:05:38.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:37.857+0000 7f6361400640 1 --2- 172.21.15.16:0/3200419865 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f635c10fcf0 0x7f635c112110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:38.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:37.857+0000 7f6361400640 1 --2- 172.21.15.16:0/3200419865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f635c075e40 0x7f635c10f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:38.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:37.857+0000 7f6361400640 1 --2- 172.21.15.16:0/3200419865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f635c0754f0 0x7f635c075900 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:38.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:37.857+0000 7f6361400640 1 -- 172.21.15.16:0/3200419865 >> 172.21.15.16:0/3200419865 conn(0x7f635c0ff180 msgr2=0x7f635c1015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:38.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:37.857+0000 7f6361400640 1 -- 172.21.15.16:0/3200419865 shutdown_connections 2024-08-31T06:05:38.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:37.857+0000 7f6361400640 1 -- 172.21.15.16:0/3200419865 wait complete. 2024-08-31T06:05:38.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:37.858+0000 7f6361400640 1 Processor -- start 2024-08-31T06:05:38.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:37.858+0000 7f6361400640 1 -- start start 2024-08-31T06:05:38.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:37.858+0000 7f6361400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f635c0754f0 0x7f635c19c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:38.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:37.858+0000 7f6361400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f635c075e40 0x7f635c19c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:38.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:37.858+0000 7f6361400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f635c10fcf0 0x7f635c1a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:38.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:37.858+0000 7f6361400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f635c115860 con 0x7f635c075e40 2024-08-31T06:05:38.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:37.858+0000 7f6361400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f635c1156e0 con 0x7f635c0754f0 2024-08-31T06:05:38.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:37.858+0000 7f6361400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f635c1159e0 con 0x7f635c10fcf0 2024-08-31T06:05:38.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:37.859+0000 7f635be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f635c0754f0 0x7f635c19c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:38.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:37.859+0000 7f635b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f635c075e40 0x7f635c19c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:38.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:37.859+0000 7f635be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f635c0754f0 0x7f635c19c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51184/0 (socket says 172.21.15.16:51184) 2024-08-31T06:05:38.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:37.859+0000 7f635be00640 1 -- 172.21.15.16:0/2069041422 learned_addr learned my addr 172.21.15.16:0/2069041422 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:05:38.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:37.859+0000 7f6360a00640 1 --2- 172.21.15.16:0/2069041422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f635c10fcf0 0x7f635c1a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:38.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:37.859+0000 7f635be00640 1 -- 172.21.15.16:0/2069041422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f635c10fcf0 msgr2=0x7f635c1a1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:38.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:37.859+0000 7f635be00640 1 --2- 172.21.15.16:0/2069041422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f635c10fcf0 0x7f635c1a1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:38.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:37.859+0000 7f635be00640 1 -- 172.21.15.16:0/2069041422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f635c075e40 msgr2=0x7f635c19c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:38.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:37.859+0000 7f635be00640 1 --2- 172.21.15.16:0/2069041422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f635c075e40 0x7f635c19c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:38.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:37.859+0000 7f635be00640 1 -- 172.21.15.16:0/2069041422 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f635c1a2200 con 0x7f635c0754f0 2024-08-31T06:05:38.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:37.859+0000 7f635be00640 1 --2- 172.21.15.16:0/2069041422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f635c0754f0 0x7f635c19c290 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f634c005940 tx=0x7f634c004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:38.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:37.860+0000 7f6358c00640 1 -- 172.21.15.16:0/2069041422 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f634c03f030 con 0x7f635c0754f0 2024-08-31T06:05:38.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:37.860+0000 7f6358c00640 1 -- 172.21.15.16:0/2069041422 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f634c003aa0 con 0x7f635c0754f0 2024-08-31T06:05:38.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:37.860+0000 7f6361400640 1 -- 172.21.15.16:0/2069041422 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f635c1a2490 con 0x7f635c0754f0 2024-08-31T06:05:38.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:37.860+0000 7f6358c00640 1 -- 172.21.15.16:0/2069041422 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f634c035070 con 0x7f635c0754f0 2024-08-31T06:05:38.214 INFO:teuthology.orchestra.run.smithi016.stdout:77309411405 2024-08-31T06:05:38.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:37.860+0000 7f6361400640 1 -- 172.21.15.16:0/2069041422 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f635c1a8c50 con 0x7f635c0754f0 2024-08-31T06:05:38.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:37.861+0000 7f6361400640 1 -- 172.21.15.16:0/2069041422 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f635c075900 con 0x7f635c0754f0 2024-08-31T06:05:38.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:37.863+0000 7f6358c00640 1 -- 172.21.15.16:0/2069041422 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f634c042030 con 0x7f635c0754f0 2024-08-31T06:05:38.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:37.864+0000 7f6358c00640 1 --2- 172.21.15.16:0/2069041422 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f63400b4c10 0x7f63400b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:38.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:37.865+0000 7f6358c00640 1 -- 172.21.15.16:0/2069041422 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f634c105cf0 con 0x7f635c0754f0 2024-08-31T06:05:38.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:37.865+0000 7f635b400640 1 --2- 172.21.15.16:0/2069041422 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f63400b4c10 0x7f63400b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:38.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:37.865+0000 7f635b400640 1 --2- 172.21.15.16:0/2069041422 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f63400b4c10 0x7f63400b7120 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f63480095f0 tx=0x7f6348005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:38.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:37.866+0000 7f6358c00640 1 -- 172.21.15.16:0/2069041422 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f634c0cc550 con 0x7f635c0754f0 2024-08-31T06:05:38.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:38.067+0000 7f6361400640 1 -- 172.21.15.16:0/2069041422 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f635c1a8e50 con 0x7f635c0754f0 2024-08-31T06:05:38.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:38.068+0000 7f6358c00640 1 -- 172.21.15.16:0/2069041422 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f634c0d0d10 con 0x7f635c0754f0 2024-08-31T06:05:38.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:38.070+0000 7f6361400640 1 -- 172.21.15.16:0/2069041422 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f63400b4c10 msgr2=0x7f63400b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:38.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:38.070+0000 7f6361400640 1 --2- 172.21.15.16:0/2069041422 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f63400b4c10 0x7f63400b7120 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f63480095f0 tx=0x7f6348005f90 comp rx=0 tx=0).stop 2024-08-31T06:05:38.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:38.070+0000 7f6361400640 1 -- 172.21.15.16:0/2069041422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f635c0754f0 msgr2=0x7f635c19c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:38.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:38.070+0000 7f6361400640 1 --2- 172.21.15.16:0/2069041422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f635c0754f0 0x7f635c19c290 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f634c005940 tx=0x7f634c004270 comp rx=0 tx=0).stop 2024-08-31T06:05:38.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:38.070+0000 7f6361400640 1 -- 172.21.15.16:0/2069041422 shutdown_connections 2024-08-31T06:05:38.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:38.070+0000 7f6361400640 1 --2- 172.21.15.16:0/2069041422 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f63400b4c10 0x7f63400b7120 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:38.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:38.070+0000 7f6361400640 1 --2- 172.21.15.16:0/2069041422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f635c10fcf0 0x7f635c1a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:38.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:38.070+0000 7f6361400640 1 --2- 172.21.15.16:0/2069041422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f635c075e40 0x7f635c19c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:38.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:38.070+0000 7f6361400640 1 --2- 172.21.15.16:0/2069041422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f635c0754f0 0x7f635c19c290 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:38.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:38.070+0000 7f6361400640 1 -- 172.21.15.16:0/2069041422 >> 172.21.15.16:0/2069041422 conn(0x7f635c0ff180 msgr2=0x7f635c109890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:38.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:38.071+0000 7f6361400640 1 -- 172.21.15.16:0/2069041422 shutdown_connections 2024-08-31T06:05:38.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:38.071+0000 7f6361400640 1 -- 172.21.15.16:0/2069041422 wait complete. 2024-08-31T06:05:38.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:38 smithi155 ceph-mon[21742]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s; 95 B/s, 1 objects/s recovering 2024-08-31T06:05:38.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:38 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/2069041422' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-08-31T06:05:38.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:38 smithi016 ceph-mon[17194]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s; 95 B/s, 1 objects/s recovering 2024-08-31T06:05:38.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:38 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/2069041422' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-08-31T06:05:38.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:38 smithi184 ceph-mon[22331]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s; 95 B/s, 1 objects/s recovering 2024-08-31T06:05:38.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:38 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/2069041422' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-08-31T06:05:39.006 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953535 got 137438953538 for osd.4 2024-08-31T06:05:39.006 DEBUG:teuthology.parallel:result is None 2024-08-31T06:05:39.610 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037795 got 257698037798 for osd.9 2024-08-31T06:05:39.610 DEBUG:teuthology.parallel:result is None 2024-08-31T06:05:40.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:40 smithi155 ceph-mon[21742]: pgmap v316: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s; 86 B/s, 0 objects/s recovering 2024-08-31T06:05:40.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:40 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/3053765071' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-08-31T06:05:40.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:40 smithi016 ceph-mon[17194]: pgmap v316: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s; 86 B/s, 0 objects/s recovering 2024-08-31T06:05:40.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:40 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/3053765071' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-08-31T06:05:40.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:40 smithi184 ceph-mon[22331]: pgmap v316: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s; 86 B/s, 0 objects/s recovering 2024-08-31T06:05:40.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:40 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/3053765071' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-08-31T06:05:41.035 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397552 got 210453397553 for osd.7 2024-08-31T06:05:41.035 DEBUG:teuthology.parallel:result is None 2024-08-31T06:05:41.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.281+0000 7f6bb6800640 1 -- 172.21.15.16:0/4050611319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bb01099e0 msgr2=0x7f6bb0110210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:41.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.281+0000 7f6bb6800640 1 --2- 172.21.15.16:0/4050611319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bb01099e0 0x7f6bb0110210 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8009ca0 tx=0x7f6ba802a8c0 comp rx=0 tx=0).stop 2024-08-31T06:05:41.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.281+0000 7f6bb6800640 1 -- 172.21.15.16:0/4050611319 shutdown_connections 2024-08-31T06:05:41.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.281+0000 7f6bb6800640 1 --2- 172.21.15.16:0/4050611319 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6bb0109090 0x7f6bb01094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:41.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.281+0000 7f6bb6800640 1 --2- 172.21.15.16:0/4050611319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bb01099e0 0x7f6bb0110210 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:41.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.281+0000 7f6bb6800640 1 --2- 172.21.15.16:0/4050611319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bb0103750 0x7f6bb0103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:41.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.281+0000 7f6bb6800640 1 -- 172.21.15.16:0/4050611319 >> 172.21.15.16:0/4050611319 conn(0x7f6bb00ff1e0 msgr2=0x7f6bb0101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:41.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.281+0000 7f6bb6800640 1 -- 172.21.15.16:0/4050611319 shutdown_connections 2024-08-31T06:05:41.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.282+0000 7f6bb6800640 1 -- 172.21.15.16:0/4050611319 wait complete. 2024-08-31T06:05:41.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.283+0000 7f6bb6800640 1 Processor -- start 2024-08-31T06:05:41.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.283+0000 7f6bb6800640 1 -- start start 2024-08-31T06:05:41.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.284+0000 7f6bb6800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bb0103750 0x7f6bb00754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:41.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.284+0000 7f6bb6800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6bb0109090 0x7f6bb00759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:41.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.284+0000 7f6bb6800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bb01099e0 0x7f6bb0075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:41.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.284+0000 7f6bb6800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb0113690 con 0x7f6bb01099e0 2024-08-31T06:05:41.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.284+0000 7f6bb6800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb0113510 con 0x7f6bb0103750 2024-08-31T06:05:41.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.284+0000 7f6bb6800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb0113810 con 0x7f6bb0109090 2024-08-31T06:05:41.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.284+0000 7f6bb5400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bb0103750 0x7f6bb00754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:41.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.284+0000 7f6bb5e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bb01099e0 0x7f6bb0075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:41.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.284+0000 7f6bb5400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bb0103750 0x7f6bb00754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51230/0 (socket says 172.21.15.16:51230) 2024-08-31T06:05:41.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.284+0000 7f6bb5400640 1 -- 172.21.15.16:0/576884703 learned_addr learned my addr 172.21.15.16:0/576884703 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:05:41.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.284+0000 7f6bb4a00640 1 --2- 172.21.15.16:0/576884703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6bb0109090 0x7f6bb00759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:41.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.284+0000 7f6bb5400640 1 -- 172.21.15.16:0/576884703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6bb0109090 msgr2=0x7f6bb00759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:41.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.284+0000 7f6bb5400640 1 --2- 172.21.15.16:0/576884703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6bb0109090 0x7f6bb00759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:41.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.285+0000 7f6bb5400640 1 -- 172.21.15.16:0/576884703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bb01099e0 msgr2=0x7f6bb0075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:41.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.285+0000 7f6bb5400640 1 --2- 172.21.15.16:0/576884703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bb01099e0 0x7f6bb0075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:41.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.285+0000 7f6bb5400640 1 -- 172.21.15.16:0/576884703 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bb01a8320 con 0x7f6bb0103750 2024-08-31T06:05:41.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.285+0000 7f6bb5400640 1 --2- 172.21.15.16:0/576884703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bb0103750 0x7f6bb00754b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6ba400e9f0 tx=0x7f6ba400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:41.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.285+0000 7f6bae000640 1 -- 172.21.15.16:0/576884703 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba400cd80 con 0x7f6bb0103750 2024-08-31T06:05:41.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.285+0000 7f6bb6800640 1 -- 172.21.15.16:0/576884703 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bb01a8610 con 0x7f6bb0103750 2024-08-31T06:05:41.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.285+0000 7f6bae000640 1 -- 172.21.15.16:0/576884703 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ba40108e0 con 0x7f6bb0103750 2024-08-31T06:05:41.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.285+0000 7f6bae000640 1 -- 172.21.15.16:0/576884703 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4018980 con 0x7f6bb0103750 2024-08-31T06:05:41.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.285+0000 7f6bb6800640 1 -- 172.21.15.16:0/576884703 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bb01a8b50 con 0x7f6bb0103750 2024-08-31T06:05:41.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.286+0000 7f6bb6800640 1 -- 172.21.15.16:0/576884703 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bb0103b60 con 0x7f6bb0103750 2024-08-31T06:05:41.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.289+0000 7f6bae000640 1 -- 172.21.15.16:0/576884703 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f6ba4010430 con 0x7f6bb0103750 2024-08-31T06:05:41.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.292+0000 7f6bae000640 1 --2- 172.21.15.16:0/576884703 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6b980b4c10 0x7f6b980b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:41.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.292+0000 7f6bae000640 1 -- 172.21.15.16:0/576884703 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f6ba40e5c20 con 0x7f6bb0103750 2024-08-31T06:05:41.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.292+0000 7f6bb4a00640 1 --2- 172.21.15.16:0/576884703 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6b980b4c10 0x7f6b980b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:41.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.292+0000 7f6bae000640 1 -- 172.21.15.16:0/576884703 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f6ba40ac480 con 0x7f6bb0103750 2024-08-31T06:05:41.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.293+0000 7f6bb4a00640 1 --2- 172.21.15.16:0/576884703 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6b980b4c10 0x7f6b980b7120 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0107b70 tx=0x7f6b9c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:41.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.490+0000 7f6bb6800640 1 -- 172.21.15.16:0/576884703 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f6bb002e430 con 0x7f6bb0103750 2024-08-31T06:05:41.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.490+0000 7f6bae000640 1 -- 172.21.15.16:0/576884703 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f6ba40b0c40 con 0x7f6bb0103750 2024-08-31T06:05:41.493 INFO:teuthology.orchestra.run.smithi016.stdout:300647710749 2024-08-31T06:05:41.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.493+0000 7f6bb6800640 1 -- 172.21.15.16:0/576884703 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6b980b4c10 msgr2=0x7f6b980b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:41.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.493+0000 7f6bb6800640 1 --2- 172.21.15.16:0/576884703 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6b980b4c10 0x7f6b980b7120 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0107b70 tx=0x7f6b9c005f90 comp rx=0 tx=0).stop 2024-08-31T06:05:41.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.493+0000 7f6bb6800640 1 -- 172.21.15.16:0/576884703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bb0103750 msgr2=0x7f6bb00754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:41.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.493+0000 7f6bb6800640 1 --2- 172.21.15.16:0/576884703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bb0103750 0x7f6bb00754b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6ba400e9f0 tx=0x7f6ba400eeb0 comp rx=0 tx=0).stop 2024-08-31T06:05:41.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.493+0000 7f6bb6800640 1 -- 172.21.15.16:0/576884703 shutdown_connections 2024-08-31T06:05:41.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.493+0000 7f6bb6800640 1 --2- 172.21.15.16:0/576884703 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6b980b4c10 0x7f6b980b7120 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:41.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.493+0000 7f6bb6800640 1 --2- 172.21.15.16:0/576884703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6bb0109090 0x7f6bb00759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:41.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.493+0000 7f6bb6800640 1 --2- 172.21.15.16:0/576884703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bb01099e0 0x7f6bb0075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:41.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.493+0000 7f6bb6800640 1 --2- 172.21.15.16:0/576884703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bb0103750 0x7f6bb00754b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:41.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.493+0000 7f6bb6800640 1 -- 172.21.15.16:0/576884703 >> 172.21.15.16:0/576884703 conn(0x7f6bb00ff1e0 msgr2=0x7f6bb0101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:41.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.493+0000 7f6bb6800640 1 -- 172.21.15.16:0/576884703 shutdown_connections 2024-08-31T06:05:41.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:41.493+0000 7f6bb6800640 1 -- 172.21.15.16:0/576884703 wait complete. 2024-08-31T06:05:41.600 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411402 got 77309411405 for osd.2 2024-08-31T06:05:41.601 DEBUG:teuthology.parallel:result is None 2024-08-31T06:05:42.395 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710745 got 300647710749 for osd.11 2024-08-31T06:05:42.395 DEBUG:teuthology.parallel:result is None 2024-08-31T06:05:42.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:42 smithi155 ceph-mon[21742]: pgmap v317: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-08-31T06:05:42.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:42 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/576884703' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-08-31T06:05:42.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:42 smithi016 ceph-mon[17194]: pgmap v317: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-08-31T06:05:42.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:42 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/576884703' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-08-31T06:05:42.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:42 smithi184 ceph-mon[22331]: pgmap v317: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-08-31T06:05:42.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:42 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/576884703' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-08-31T06:05:42.785 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593780 got 184683593783 for osd.6 2024-08-31T06:05:42.785 DEBUG:teuthology.parallel:result is None 2024-08-31T06:05:42.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.393+0000 7f6645600640 1 -- 172.21.15.16:0/1275650796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6640078030 msgr2=0x7f6640113fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:42.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.393+0000 7f6645600640 1 --2- 172.21.15.16:0/1275650796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6640078030 0x7f6640113fa0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f662c009900 tx=0x7f662c02a410 comp rx=0 tx=0).stop 2024-08-31T06:05:42.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.393+0000 7f6645600640 1 -- 172.21.15.16:0/1275650796 shutdown_connections 2024-08-31T06:05:42.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.393+0000 7f6645600640 1 --2- 172.21.15.16:0/1275650796 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66401144e0 0x7f6640116900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.393+0000 7f6645600640 1 --2- 172.21.15.16:0/1275650796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6640078030 0x7f6640113fa0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.393+0000 7f6645600640 1 --2- 172.21.15.16:0/1275650796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66400776e0 0x7f6640077af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.393+0000 7f6645600640 1 -- 172.21.15.16:0/1275650796 >> 172.21.15.16:0/1275650796 conn(0x7f66401015b0 msgr2=0x7f6640103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.393+0000 7f6645600640 1 -- 172.21.15.16:0/1275650796 shutdown_connections 2024-08-31T06:05:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.393+0000 7f6645600640 1 -- 172.21.15.16:0/1275650796 wait complete. 2024-08-31T06:05:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.394+0000 7f6645600640 1 Processor -- start 2024-08-31T06:05:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.394+0000 7f6645600640 1 -- start start 2024-08-31T06:05:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.395+0000 7f6645600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66400776e0 0x7f66401a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.395+0000 7f6645600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6640078030 0x7f66401a0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.395+0000 7f6645600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66401144e0 0x7f66401a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.395+0000 7f6645600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6640119e40 con 0x7f6640078030 2024-08-31T06:05:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.395+0000 7f6645600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6640119cc0 con 0x7f66401144e0 2024-08-31T06:05:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.395+0000 7f6645600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6640119fc0 con 0x7f66400776e0 2024-08-31T06:05:42.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.395+0000 7f663f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66401144e0 0x7f66401a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:42.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.395+0000 7f663ea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66400776e0 0x7f66401a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:42.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.395+0000 7f663f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66401144e0 0x7f66401a6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51212/0 (socket says 172.21.15.16:51212) 2024-08-31T06:05:42.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.395+0000 7f663f400640 1 -- 172.21.15.16:0/3053765071 learned_addr learned my addr 172.21.15.16:0/3053765071 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:05:42.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.395+0000 7f663e000640 1 --2- 172.21.15.16:0/3053765071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6640078030 0x7f66401a0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:42.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.395+0000 7f663f400640 1 -- 172.21.15.16:0/3053765071 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66400776e0 msgr2=0x7f66401a08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:42.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.395+0000 7f663f400640 1 --2- 172.21.15.16:0/3053765071 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66400776e0 0x7f66401a08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:42.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.395+0000 7f663f400640 1 -- 172.21.15.16:0/3053765071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6640078030 msgr2=0x7f66401a0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:42.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.395+0000 7f663f400640 1 --2- 172.21.15.16:0/3053765071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6640078030 0x7f66401a0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:42.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.395+0000 7f663f400640 1 -- 172.21.15.16:0/3053765071 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66401a6840 con 0x7f66401144e0 2024-08-31T06:05:42.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.395+0000 7f663ea00640 1 --2- 172.21.15.16:0/3053765071 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66400776e0 0x7f66401a08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:05:42.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.396+0000 7f663f400640 1 --2- 172.21.15.16:0/3053765071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66401144e0 0x7f66401a6110 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f663800eb10 tx=0x7f663800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:42.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.396+0000 7f6637400640 1 -- 172.21.15.16:0/3053765071 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6638004b20 con 0x7f66401144e0 2024-08-31T06:05:42.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.396+0000 7f6637400640 1 -- 172.21.15.16:0/3053765071 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6638004cc0 con 0x7f66401144e0 2024-08-31T06:05:42.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.396+0000 7f6637400640 1 -- 172.21.15.16:0/3053765071 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6638010670 con 0x7f66401144e0 2024-08-31T06:05:42.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.396+0000 7f6645600640 1 -- 172.21.15.16:0/3053765071 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66401a6b30 con 0x7f66401144e0 2024-08-31T06:05:42.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.396+0000 7f6645600640 1 -- 172.21.15.16:0/3053765071 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66401ad290 con 0x7f66401144e0 2024-08-31T06:05:42.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.397+0000 7f6645600640 1 -- 172.21.15.16:0/3053765071 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6640077af0 con 0x7f66401144e0 2024-08-31T06:05:42.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.399+0000 7f6637400640 1 -- 172.21.15.16:0/3053765071 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f6638005020 con 0x7f66401144e0 2024-08-31T06:05:42.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.401+0000 7f6637400640 1 --2- 172.21.15.16:0/3053765071 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f66240b4c90 0x7f66240b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:42.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.401+0000 7f6637400640 1 -- 172.21.15.16:0/3053765071 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f66380e5890 con 0x7f66401144e0 2024-08-31T06:05:42.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.401+0000 7f663ea00640 1 --2- 172.21.15.16:0/3053765071 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f66240b4c90 0x7f66240b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:42.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.402+0000 7f663ea00640 1 --2- 172.21.15.16:0/3053765071 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f66240b4c90 0x7f66240b71a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f66300095f0 tx=0x7f6630005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:42.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.405+0000 7f6637400640 1 -- 172.21.15.16:0/3053765071 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f66380ac740 con 0x7f66401144e0 2024-08-31T06:05:42.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.627+0000 7f6645600640 1 -- 172.21.15.16:0/3053765071 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f664010f0e0 con 0x7f66401144e0 2024-08-31T06:05:42.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.627+0000 7f6637400640 1 -- 172.21.15.16:0/3053765071 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f66380ac040 con 0x7f66401144e0 2024-08-31T06:05:42.874 INFO:teuthology.orchestra.run.smithi016.stdout:158913790013 2024-08-31T06:05:42.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.630+0000 7f6645600640 1 -- 172.21.15.16:0/3053765071 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f66240b4c90 msgr2=0x7f66240b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:42.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.630+0000 7f6645600640 1 --2- 172.21.15.16:0/3053765071 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f66240b4c90 0x7f66240b71a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f66300095f0 tx=0x7f6630005f90 comp rx=0 tx=0).stop 2024-08-31T06:05:42.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.630+0000 7f6645600640 1 -- 172.21.15.16:0/3053765071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66401144e0 msgr2=0x7f66401a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:42.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.630+0000 7f6645600640 1 --2- 172.21.15.16:0/3053765071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66401144e0 0x7f66401a6110 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f663800eb10 tx=0x7f663800c2d0 comp rx=0 tx=0).stop 2024-08-31T06:05:42.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.630+0000 7f6645600640 1 -- 172.21.15.16:0/3053765071 shutdown_connections 2024-08-31T06:05:42.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.630+0000 7f6645600640 1 --2- 172.21.15.16:0/3053765071 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f66240b4c90 0x7f66240b71a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:42.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.630+0000 7f6645600640 1 --2- 172.21.15.16:0/3053765071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66401144e0 0x7f66401a6110 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:42.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.630+0000 7f6645600640 1 --2- 172.21.15.16:0/3053765071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6640078030 0x7f66401a0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:42.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.630+0000 7f6645600640 1 --2- 172.21.15.16:0/3053765071 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66400776e0 0x7f66401a08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:42.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.630+0000 7f6645600640 1 -- 172.21.15.16:0/3053765071 >> 172.21.15.16:0/3053765071 conn(0x7f66401015b0 msgr2=0x7f6640116580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:42.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.630+0000 7f6645600640 1 -- 172.21.15.16:0/3053765071 shutdown_connections 2024-08-31T06:05:42.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:39.630+0000 7f6645600640 1 -- 172.21.15.16:0/3053765071 wait complete. 2024-08-31T06:05:43.556 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790008 got 158913790013 for osd.5 2024-08-31T06:05:43.557 DEBUG:teuthology.parallel:result is None 2024-08-31T06:05:43.810 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234025 got 231928234028 for osd.8 2024-08-31T06:05:43.810 DEBUG:teuthology.parallel:result is None 2024-08-31T06:05:43.811 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-08-31T06:05:43.811 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph pg dump --format=json 2024-08-31T06:05:44.226 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:05:44.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:44 smithi155 ceph-mon[21742]: pgmap v318: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-31T06:05:44.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:44 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:05:44.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:44 smithi016 ceph-mon[17194]: pgmap v318: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-31T06:05:44.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:44 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:05:44.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:44 smithi184 ceph-mon[22331]: pgmap v318: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-31T06:05:44.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:44 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:05:44.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:44.906+0000 7f057d000640 1 -- 172.21.15.16:0/3360721511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0578109240 msgr2=0x7f0578109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:44.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:44.906+0000 7f057d000640 1 --2- 172.21.15.16:0/3360721511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0578109240 0x7f0578109650 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0560009900 tx=0x7f056002a410 comp rx=0 tx=0).stop 2024-08-31T06:05:44.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:44.906+0000 7f057d000640 1 -- 172.21.15.16:0/3360721511 shutdown_connections 2024-08-31T06:05:44.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:44.906+0000 7f057d000640 1 --2- 172.21.15.16:0/3360721511 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0578109b90 0x7f05781103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:44.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:44.906+0000 7f057d000640 1 --2- 172.21.15.16:0/3360721511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0578109240 0x7f0578109650 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:44.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:44.906+0000 7f057d000640 1 --2- 172.21.15.16:0/3360721511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0578103900 0x7f0578103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:44.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:44.906+0000 7f057d000640 1 -- 172.21.15.16:0/3360721511 >> 172.21.15.16:0/3360721511 conn(0x7f057807b5f0 msgr2=0x7f057807ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:44.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:44.906+0000 7f057d000640 1 -- 172.21.15.16:0/3360721511 shutdown_connections 2024-08-31T06:05:44.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:44.907+0000 7f057d000640 1 -- 172.21.15.16:0/3360721511 wait complete. 2024-08-31T06:05:44.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:44.908+0000 7f057d000640 1 Processor -- start 2024-08-31T06:05:44.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:44.908+0000 7f057d000640 1 -- start start 2024-08-31T06:05:44.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:44.908+0000 7f057d000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0578103900 0x7f057819c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:44.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:44.908+0000 7f057d000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0578109240 0x7f057819c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:44.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:44.908+0000 7f057d000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0578109b90 0x7f05781a1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:44.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:44.908+0000 7f057d000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f05781138a0 con 0x7f0578103900 2024-08-31T06:05:44.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:44.908+0000 7f057d000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0578113720 con 0x7f0578109240 2024-08-31T06:05:44.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:44.908+0000 7f057d000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0578113a20 con 0x7f0578109b90 2024-08-31T06:05:44.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:44.908+0000 7f0575e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0578103900 0x7f057819c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:44.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:44.908+0000 7f0575e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0578103900 0x7f057819c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49914/0 (socket says 172.21.15.16:49914) 2024-08-31T06:05:44.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:44.908+0000 7f0575e00640 1 -- 172.21.15.16:0/2227474733 learned_addr learned my addr 172.21.15.16:0/2227474733 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:05:44.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:44.909+0000 7f056d400640 1 --2- 172.21.15.16:0/2227474733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0578109240 0x7f057819c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:44.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:44.909+0000 7f0576800640 1 --2- 172.21.15.16:0/2227474733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0578109b90 0x7f05781a1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:44.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:44.909+0000 7f0575e00640 1 -- 172.21.15.16:0/2227474733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0578109b90 msgr2=0x7f05781a1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:44.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:44.909+0000 7f0575e00640 1 --2- 172.21.15.16:0/2227474733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0578109b90 0x7f05781a1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:44.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:44.909+0000 7f0575e00640 1 -- 172.21.15.16:0/2227474733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0578109240 msgr2=0x7f057819c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:44.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:44.909+0000 7f0575e00640 1 --2- 172.21.15.16:0/2227474733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0578109240 0x7f057819c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:44.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:44.909+0000 7f0575e00640 1 -- 172.21.15.16:0/2227474733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05781a23a0 con 0x7f0578103900 2024-08-31T06:05:44.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:44.909+0000 7f0575e00640 1 --2- 172.21.15.16:0/2227474733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0578103900 0x7f057819c430 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f056800e9f0 tx=0x7f056800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:44.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:44.910+0000 7f056f400640 1 -- 172.21.15.16:0/2227474733 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056800cd80 con 0x7f0578103900 2024-08-31T06:05:44.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:44.910+0000 7f056f400640 1 -- 172.21.15.16:0/2227474733 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05680108e0 con 0x7f0578103900 2024-08-31T06:05:44.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:44.910+0000 7f057d000640 1 -- 172.21.15.16:0/2227474733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05781a2690 con 0x7f0578103900 2024-08-31T06:05:44.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:44.910+0000 7f056f400640 1 -- 172.21.15.16:0/2227474733 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0568018980 con 0x7f0578103900 2024-08-31T06:05:44.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:44.910+0000 7f057d000640 1 -- 172.21.15.16:0/2227474733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05781a8df0 con 0x7f0578103900 2024-08-31T06:05:44.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:44.911+0000 7f057d000640 1 -- 172.21.15.16:0/2227474733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0578104b80 con 0x7f0578103900 2024-08-31T06:05:44.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:44.913+0000 7f056f400640 1 -- 172.21.15.16:0/2227474733 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f0568010430 con 0x7f0578103900 2024-08-31T06:05:44.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:44.915+0000 7f056f400640 1 --2- 172.21.15.16:0/2227474733 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f055c0b4c10 0x7f055c0b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:44.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:44.915+0000 7f056f400640 1 -- 172.21.15.16:0/2227474733 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f05680e5750 con 0x7f0578103900 2024-08-31T06:05:44.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:44.915+0000 7f056d400640 1 --2- 172.21.15.16:0/2227474733 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f055c0b4c10 0x7f055c0b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:44.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:44.916+0000 7f056d400640 1 --2- 172.21.15.16:0/2227474733 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f055c0b4c10 0x7f055c0b7120 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f056002aa00 tx=0x7f0560005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:44.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:44.919+0000 7f056f400640 1 -- 172.21.15.16:0/2227474733 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f05680abfb0 con 0x7f0578103900 2024-08-31T06:05:45.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:45.113+0000 7f057d000640 1 -- 172.21.15.16:0/2227474733 --> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0578107760 con 0x7f055c0b4c10 2024-08-31T06:05:45.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:45.122+0000 7f056f400640 1 -- 172.21.15.16:0/2227474733 <== mgr.14220 v2:172.21.15.16:6800/4279751337 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258944 (secure 0 0 0) 0x7f0578107760 con 0x7f055c0b4c10 2024-08-31T06:05:45.127 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:05:45.129 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-08-31T06:05:45.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:45.126+0000 7f057d000640 1 -- 172.21.15.16:0/2227474733 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f055c0b4c10 msgr2=0x7f055c0b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:45.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:45.126+0000 7f057d000640 1 --2- 172.21.15.16:0/2227474733 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f055c0b4c10 0x7f055c0b7120 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f056002aa00 tx=0x7f0560005a20 comp rx=0 tx=0).stop 2024-08-31T06:05:45.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:45.126+0000 7f057d000640 1 -- 172.21.15.16:0/2227474733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0578103900 msgr2=0x7f057819c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:45.126+0000 7f057d000640 1 --2- 172.21.15.16:0/2227474733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0578103900 0x7f057819c430 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f056800e9f0 tx=0x7f056800eeb0 comp rx=0 tx=0).stop 2024-08-31T06:05:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:45.126+0000 7f057d000640 1 -- 172.21.15.16:0/2227474733 shutdown_connections 2024-08-31T06:05:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:45.126+0000 7f057d000640 1 --2- 172.21.15.16:0/2227474733 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f055c0b4c10 0x7f055c0b7120 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:45.126+0000 7f057d000640 1 --2- 172.21.15.16:0/2227474733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0578109b90 0x7f05781a1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:45.126+0000 7f057d000640 1 --2- 172.21.15.16:0/2227474733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0578109240 0x7f057819c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:45.126+0000 7f057d000640 1 --2- 172.21.15.16:0/2227474733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0578103900 0x7f057819c430 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:45.126+0000 7f057d000640 1 -- 172.21.15.16:0/2227474733 >> 172.21.15.16:0/2227474733 conn(0x7f057807b5f0 msgr2=0x7f0578107050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:45.127+0000 7f057d000640 1 -- 172.21.15.16:0/2227474733 shutdown_connections 2024-08-31T06:05:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:45.127+0000 7f057d000640 1 -- 172.21.15.16:0/2227474733 wait complete. 2024-08-31T06:05:45.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:45 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:05:45.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:45 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:05:45.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:45 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:05:45.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:45 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:05:45.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:45 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:05:45.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:45 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:05:45.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:45 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:05:45.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:45 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:05:45.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:45 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:05:45.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:45 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:05:45.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:45 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:05:45.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:45 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:05:46.092 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2024-08-31T06:05:43.274849+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654756,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":458,"num_read_kb":328,"num_write":290,"num_write_kb":2263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":2952987,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":234,"ondisk_log_size":234,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360028,"kb_used_data":10704,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581796,"statfs":{"total":1151940427776,"available":1151571759104,"internally_reserved":0,"allocated":10960896,"data_stored":7194576,"data_compressed":178617,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003416"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048210+0000","last_change":"2024-08-31T06:04:25.877318+0000","last_active":"2024-08-31T06:05:25.048210+0000","last_peered":"2024-08-31T06:05:25.048210+0000","last_clean":"2024-08-31T06:05:25.048210+0000","last_became_active":"2024-08-31T06:04:25.876894+0000","last_became_peered":"2024-08-31T06:04:25.876894+0000","last_unstale":"2024-08-31T06:05:25.048210+0000","last_undegraded":"2024-08-31T06:05:25.048210+0000","last_fullsized":"2024-08-31T06:05:25.048210+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:41:09.449343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.056660+0000","last_change":"2024-08-31T06:04:25.885670+0000","last_active":"2024-08-31T06:05:25.056660+0000","last_peered":"2024-08-31T06:05:25.056660+0000","last_clean":"2024-08-31T06:05:25.056660+0000","last_became_active":"2024-08-31T06:04:25.883818+0000","last_became_peered":"2024-08-31T06:04:25.883818+0000","last_unstale":"2024-08-31T06:05:25.056660+0000","last_undegraded":"2024-08-31T06:05:25.056660+0000","last_fullsized":"2024-08-31T06:05:25.056660+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:40:01.344432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.056294+0000","last_change":"2024-08-31T06:04:25.883375+0000","last_active":"2024-08-31T06:05:25.056294+0000","last_peered":"2024-08-31T06:05:25.056294+0000","last_clean":"2024-08-31T06:05:25.056294+0000","last_became_active":"2024-08-31T06:04:25.881750+0000","last_became_peered":"2024-08-31T06:04:25.881750+0000","last_unstale":"2024-08-31T06:05:25.056294+0000","last_undegraded":"2024-08-31T06:05:25.056294+0000","last_fullsized":"2024-08-31T06:05:25.056294+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:46:46.757363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.044794+0000","last_change":"2024-08-31T06:04:25.882436+0000","last_active":"2024-08-31T06:05:25.044794+0000","last_peered":"2024-08-31T06:05:25.044794+0000","last_clean":"2024-08-31T06:05:25.044794+0000","last_became_active":"2024-08-31T06:04:25.881805+0000","last_became_peered":"2024-08-31T06:04:25.881805+0000","last_unstale":"2024-08-31T06:05:25.044794+0000","last_undegraded":"2024-08-31T06:05:25.044794+0000","last_fullsized":"2024-08-31T06:05:25.044794+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:42:39.793625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.051404+0000","last_change":"2024-08-31T06:04:25.894534+0000","last_active":"2024-08-31T06:05:25.051404+0000","last_peered":"2024-08-31T06:05:25.051404+0000","last_clean":"2024-08-31T06:05:25.051404+0000","last_became_active":"2024-08-31T06:04:25.894116+0000","last_became_peered":"2024-08-31T06:04:25.894116+0000","last_unstale":"2024-08-31T06:05:25.051404+0000","last_undegraded":"2024-08-31T06:05:25.051404+0000","last_fullsized":"2024-08-31T06:05:25.051404+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:02:31.257454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.046795+0000","last_change":"2024-08-31T06:04:25.886182+0000","last_active":"2024-08-31T06:05:25.046795+0000","last_peered":"2024-08-31T06:05:25.046795+0000","last_clean":"2024-08-31T06:05:25.046795+0000","last_became_active":"2024-08-31T06:04:25.885825+0000","last_became_peered":"2024-08-31T06:04:25.885825+0000","last_unstale":"2024-08-31T06:05:25.046795+0000","last_undegraded":"2024-08-31T06:05:25.046795+0000","last_fullsized":"2024-08-31T06:05:25.046795+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:58:12.427931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048313+0000","last_change":"2024-08-31T06:04:25.877521+0000","last_active":"2024-08-31T06:05:25.048313+0000","last_peered":"2024-08-31T06:05:25.048313+0000","last_clean":"2024-08-31T06:05:25.048313+0000","last_became_active":"2024-08-31T06:04:25.877122+0000","last_became_peered":"2024-08-31T06:04:25.877122+0000","last_unstale":"2024-08-31T06:05:25.048313+0000","last_undegraded":"2024-08-31T06:05:25.048313+0000","last_fullsized":"2024-08-31T06:05:25.048313+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:17:27.406828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.107691+0000","last_change":"2024-08-31T06:04:25.892547+0000","last_active":"2024-08-31T06:05:25.107691+0000","last_peered":"2024-08-31T06:05:25.107691+0000","last_clean":"2024-08-31T06:05:25.107691+0000","last_became_active":"2024-08-31T06:04:25.892090+0000","last_became_peered":"2024-08-31T06:04:25.892090+0000","last_unstale":"2024-08-31T06:05:25.107691+0000","last_undegraded":"2024-08-31T06:05:25.107691+0000","last_fullsized":"2024-08-31T06:05:25.107691+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:28:53.414252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.050038+0000","last_change":"2024-08-31T06:04:25.879952+0000","last_active":"2024-08-31T06:05:25.050038+0000","last_peered":"2024-08-31T06:05:25.050038+0000","last_clean":"2024-08-31T06:05:25.050038+0000","last_became_active":"2024-08-31T06:04:25.879568+0000","last_became_peered":"2024-08-31T06:04:25.879568+0000","last_unstale":"2024-08-31T06:05:25.050038+0000","last_undegraded":"2024-08-31T06:05:25.050038+0000","last_fullsized":"2024-08-31T06:05:25.050038+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:28:04.558054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.051270+0000","last_change":"2024-08-31T06:05:25.051173+0000","last_active":"2024-08-31T06:05:25.051270+0000","last_peered":"2024-08-31T06:05:25.051270+0000","last_clean":"2024-08-31T06:05:25.051270+0000","last_became_active":"2024-08-31T06:05:25.050867+0000","last_became_peered":"2024-08-31T06:05:25.050867+0000","last_unstale":"2024-08-31T06:05:25.051270+0000","last_undegraded":"2024-08-31T06:05:25.051270+0000","last_fullsized":"2024-08-31T06:05:25.051270+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:27:09.971497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048858+0000","last_change":"2024-08-31T06:04:25.883984+0000","last_active":"2024-08-31T06:05:25.048858+0000","last_peered":"2024-08-31T06:05:25.048858+0000","last_clean":"2024-08-31T06:05:25.048858+0000","last_became_active":"2024-08-31T06:04:25.883353+0000","last_became_peered":"2024-08-31T06:04:25.883353+0000","last_unstale":"2024-08-31T06:05:25.048858+0000","last_undegraded":"2024-08-31T06:05:25.048858+0000","last_fullsized":"2024-08-31T06:05:25.048858+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:16:25.178952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.044850+0000","last_change":"2024-08-31T06:04:25.883335+0000","last_active":"2024-08-31T06:05:25.044850+0000","last_peered":"2024-08-31T06:05:25.044850+0000","last_clean":"2024-08-31T06:05:25.044850+0000","last_became_active":"2024-08-31T06:04:25.881488+0000","last_became_peered":"2024-08-31T06:04:25.881488+0000","last_unstale":"2024-08-31T06:05:25.044850+0000","last_undegraded":"2024-08-31T06:05:25.044850+0000","last_fullsized":"2024-08-31T06:05:25.044850+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:57:24.472482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.046725+0000","last_change":"2024-08-31T06:04:25.891579+0000","last_active":"2024-08-31T06:05:25.046725+0000","last_peered":"2024-08-31T06:05:25.046725+0000","last_clean":"2024-08-31T06:05:25.046725+0000","last_became_active":"2024-08-31T06:04:25.887950+0000","last_became_peered":"2024-08-31T06:04:25.887950+0000","last_unstale":"2024-08-31T06:05:25.046725+0000","last_undegraded":"2024-08-31T06:05:25.046725+0000","last_fullsized":"2024-08-31T06:05:25.046725+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:38:35.284442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.056229+0000","last_change":"2024-08-31T06:04:25.882185+0000","last_active":"2024-08-31T06:05:25.056229+0000","last_peered":"2024-08-31T06:05:25.056229+0000","last_clean":"2024-08-31T06:05:25.056229+0000","last_became_active":"2024-08-31T06:04:25.876499+0000","last_became_peered":"2024-08-31T06:04:25.876499+0000","last_unstale":"2024-08-31T06:05:25.056229+0000","last_undegraded":"2024-08-31T06:05:25.056229+0000","last_fullsized":"2024-08-31T06:05:25.056229+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:40:01.132353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.056113+0000","last_change":"2024-08-31T06:04:25.883624+0000","last_active":"2024-08-31T06:05:25.056113+0000","last_peered":"2024-08-31T06:05:25.056113+0000","last_clean":"2024-08-31T06:05:25.056113+0000","last_became_active":"2024-08-31T06:04:25.878041+0000","last_became_peered":"2024-08-31T06:04:25.878041+0000","last_unstale":"2024-08-31T06:05:25.056113+0000","last_undegraded":"2024-08-31T06:05:25.056113+0000","last_fullsized":"2024-08-31T06:05:25.056113+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:05:21.278321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.049208+0000","last_change":"2024-08-31T06:04:25.882538+0000","last_active":"2024-08-31T06:05:25.049208+0000","last_peered":"2024-08-31T06:05:25.049208+0000","last_clean":"2024-08-31T06:05:25.049208+0000","last_became_active":"2024-08-31T06:04:25.882201+0000","last_became_peered":"2024-08-31T06:04:25.882201+0000","last_unstale":"2024-08-31T06:05:25.049208+0000","last_undegraded":"2024-08-31T06:05:25.049208+0000","last_fullsized":"2024-08-31T06:05:25.049208+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:47:16.880893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048602+0000","last_change":"2024-08-31T06:04:25.880376+0000","last_active":"2024-08-31T06:05:25.048602+0000","last_peered":"2024-08-31T06:05:25.048602+0000","last_clean":"2024-08-31T06:05:25.048602+0000","last_became_active":"2024-08-31T06:04:25.877668+0000","last_became_peered":"2024-08-31T06:04:25.877668+0000","last_unstale":"2024-08-31T06:05:25.048602+0000","last_undegraded":"2024-08-31T06:05:25.048602+0000","last_fullsized":"2024-08-31T06:05:25.048602+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:55:32.737406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.043803+0000","last_change":"2024-08-31T06:04:25.875431+0000","last_active":"2024-08-31T06:05:25.043803+0000","last_peered":"2024-08-31T06:05:25.043803+0000","last_clean":"2024-08-31T06:05:25.043803+0000","last_became_active":"2024-08-31T06:04:25.874751+0000","last_became_peered":"2024-08-31T06:04:25.874751+0000","last_unstale":"2024-08-31T06:05:25.043803+0000","last_undegraded":"2024-08-31T06:05:25.043803+0000","last_fullsized":"2024-08-31T06:05:25.043803+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:16:57.695415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.049406+0000","last_change":"2024-08-31T06:04:25.895656+0000","last_active":"2024-08-31T06:05:25.049406+0000","last_peered":"2024-08-31T06:05:25.049406+0000","last_clean":"2024-08-31T06:05:25.049406+0000","last_became_active":"2024-08-31T06:04:25.895272+0000","last_became_peered":"2024-08-31T06:04:25.895272+0000","last_unstale":"2024-08-31T06:05:25.049406+0000","last_undegraded":"2024-08-31T06:05:25.049406+0000","last_fullsized":"2024-08-31T06:05:25.049406+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:18:47.276195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.047722+0000","last_change":"2024-08-31T06:04:25.877283+0000","last_active":"2024-08-31T06:05:25.047722+0000","last_peered":"2024-08-31T06:05:25.047722+0000","last_clean":"2024-08-31T06:05:25.047722+0000","last_became_active":"2024-08-31T06:04:25.876846+0000","last_became_peered":"2024-08-31T06:04:25.876846+0000","last_unstale":"2024-08-31T06:05:25.047722+0000","last_undegraded":"2024-08-31T06:05:25.047722+0000","last_fullsized":"2024-08-31T06:05:25.047722+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:54:05.086844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.046754+0000","last_change":"2024-08-31T06:04:25.884891+0000","last_active":"2024-08-31T06:05:25.046754+0000","last_peered":"2024-08-31T06:05:25.046754+0000","last_clean":"2024-08-31T06:05:25.046754+0000","last_became_active":"2024-08-31T06:04:25.884501+0000","last_became_peered":"2024-08-31T06:04:25.884501+0000","last_unstale":"2024-08-31T06:05:25.046754+0000","last_undegraded":"2024-08-31T06:05:25.046754+0000","last_fullsized":"2024-08-31T06:05:25.046754+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:22:54.846102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.049559+0000","last_change":"2024-08-31T06:04:25.877150+0000","last_active":"2024-08-31T06:05:25.049559+0000","last_peered":"2024-08-31T06:05:25.049559+0000","last_clean":"2024-08-31T06:05:25.049559+0000","last_became_active":"2024-08-31T06:04:25.876766+0000","last_became_peered":"2024-08-31T06:04:25.876766+0000","last_unstale":"2024-08-31T06:05:25.049559+0000","last_undegraded":"2024-08-31T06:05:25.049559+0000","last_fullsized":"2024-08-31T06:05:25.049559+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:46:26.724970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.107727+0000","last_change":"2024-08-31T06:04:25.881275+0000","last_active":"2024-08-31T06:05:25.107727+0000","last_peered":"2024-08-31T06:05:25.107727+0000","last_clean":"2024-08-31T06:05:25.107727+0000","last_became_active":"2024-08-31T06:04:25.879877+0000","last_became_peered":"2024-08-31T06:04:25.879877+0000","last_unstale":"2024-08-31T06:05:25.107727+0000","last_undegraded":"2024-08-31T06:05:25.107727+0000","last_fullsized":"2024-08-31T06:05:25.107727+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:07:07.237041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.044574+0000","last_change":"2024-08-31T06:04:25.895750+0000","last_active":"2024-08-31T06:05:25.044574+0000","last_peered":"2024-08-31T06:05:25.044574+0000","last_clean":"2024-08-31T06:05:25.044574+0000","last_became_active":"2024-08-31T06:04:25.895348+0000","last_became_peered":"2024-08-31T06:04:25.895348+0000","last_unstale":"2024-08-31T06:05:25.044574+0000","last_undegraded":"2024-08-31T06:05:25.044574+0000","last_fullsized":"2024-08-31T06:05:25.044574+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:53:10.014808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.108056+0000","last_change":"2024-08-31T06:04:25.895537+0000","last_active":"2024-08-31T06:05:25.108056+0000","last_peered":"2024-08-31T06:05:25.108056+0000","last_clean":"2024-08-31T06:05:25.108056+0000","last_became_active":"2024-08-31T06:04:25.895136+0000","last_became_peered":"2024-08-31T06:04:25.895136+0000","last_unstale":"2024-08-31T06:05:25.108056+0000","last_undegraded":"2024-08-31T06:05:25.108056+0000","last_fullsized":"2024-08-31T06:05:25.108056+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:41:57.554507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048946+0000","last_change":"2024-08-31T06:04:25.880490+0000","last_active":"2024-08-31T06:05:25.048946+0000","last_peered":"2024-08-31T06:05:25.048946+0000","last_clean":"2024-08-31T06:05:25.048946+0000","last_became_active":"2024-08-31T06:04:25.879269+0000","last_became_peered":"2024-08-31T06:04:25.879269+0000","last_unstale":"2024-08-31T06:05:25.048946+0000","last_undegraded":"2024-08-31T06:05:25.048946+0000","last_fullsized":"2024-08-31T06:05:25.048946+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:05:49.575915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.053834+0000","last_change":"2024-08-31T06:05:25.053714+0000","last_active":"2024-08-31T06:05:25.053834+0000","last_peered":"2024-08-31T06:05:25.053834+0000","last_clean":"2024-08-31T06:05:25.053834+0000","last_became_active":"2024-08-31T06:05:25.052685+0000","last_became_peered":"2024-08-31T06:05:25.052685+0000","last_unstale":"2024-08-31T06:05:25.053834+0000","last_undegraded":"2024-08-31T06:05:25.053834+0000","last_fullsized":"2024-08-31T06:05:25.053834+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:17:31.474601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.058065+0000","last_change":"2024-08-31T06:05:25.057946+0000","last_active":"2024-08-31T06:05:25.058065+0000","last_peered":"2024-08-31T06:05:25.058065+0000","last_clean":"2024-08-31T06:05:25.058065+0000","last_became_active":"2024-08-31T06:05:25.057501+0000","last_became_peered":"2024-08-31T06:05:25.057501+0000","last_unstale":"2024-08-31T06:05:25.058065+0000","last_undegraded":"2024-08-31T06:05:25.058065+0000","last_fullsized":"2024-08-31T06:05:25.058065+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:16:05.721770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.056527+0000","last_change":"2024-08-31T06:04:25.897473+0000","last_active":"2024-08-31T06:05:25.056527+0000","last_peered":"2024-08-31T06:05:25.056527+0000","last_clean":"2024-08-31T06:05:25.056527+0000","last_became_active":"2024-08-31T06:04:25.895194+0000","last_became_peered":"2024-08-31T06:04:25.895194+0000","last_unstale":"2024-08-31T06:05:25.056527+0000","last_undegraded":"2024-08-31T06:05:25.056527+0000","last_fullsized":"2024-08-31T06:05:25.056527+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:43:04.539970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048442+0000","last_change":"2024-08-31T06:04:25.894402+0000","last_active":"2024-08-31T06:05:25.048442+0000","last_peered":"2024-08-31T06:05:25.048442+0000","last_clean":"2024-08-31T06:05:25.048442+0000","last_became_active":"2024-08-31T06:04:25.893883+0000","last_became_peered":"2024-08-31T06:04:25.893883+0000","last_unstale":"2024-08-31T06:05:25.048442+0000","last_undegraded":"2024-08-31T06:05:25.048442+0000","last_fullsized":"2024-08-31T06:05:25.048442+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:45:53.778473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048292+0000","last_change":"2024-08-31T06:04:25.881543+0000","last_active":"2024-08-31T06:05:25.048292+0000","last_peered":"2024-08-31T06:05:25.048292+0000","last_clean":"2024-08-31T06:05:25.048292+0000","last_became_active":"2024-08-31T06:04:25.881144+0000","last_became_peered":"2024-08-31T06:04:25.881144+0000","last_unstale":"2024-08-31T06:05:25.048292+0000","last_undegraded":"2024-08-31T06:05:25.048292+0000","last_fullsized":"2024-08-31T06:05:25.048292+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:13:02.282879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.047616+0000","last_change":"2024-08-31T06:04:25.881143+0000","last_active":"2024-08-31T06:05:25.047616+0000","last_peered":"2024-08-31T06:05:25.047616+0000","last_clean":"2024-08-31T06:05:25.047616+0000","last_became_active":"2024-08-31T06:04:25.879718+0000","last_became_peered":"2024-08-31T06:04:25.879718+0000","last_unstale":"2024-08-31T06:05:25.047616+0000","last_undegraded":"2024-08-31T06:05:25.047616+0000","last_fullsized":"2024-08-31T06:05:25.047616+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:32:07.771488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.051368+0000","last_change":"2024-08-31T06:04:25.879493+0000","last_active":"2024-08-31T06:05:25.051368+0000","last_peered":"2024-08-31T06:05:25.051368+0000","last_clean":"2024-08-31T06:05:25.051368+0000","last_became_active":"2024-08-31T06:04:25.879065+0000","last_became_peered":"2024-08-31T06:04:25.879065+0000","last_unstale":"2024-08-31T06:05:25.051368+0000","last_undegraded":"2024-08-31T06:05:25.051368+0000","last_fullsized":"2024-08-31T06:05:25.051368+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:53:43.848269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"78'6","reported_seq":106,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:37.072887+0000","last_change":"2024-08-31T06:05:25.058954+0000","last_active":"2024-08-31T06:05:37.072887+0000","last_peered":"2024-08-31T06:05:37.072887+0000","last_clean":"2024-08-31T06:05:37.072887+0000","last_became_active":"2024-08-31T06:05:25.051143+0000","last_became_peered":"2024-08-31T06:05:25.051143+0000","last_unstale":"2024-08-31T06:05:37.072887+0000","last_undegraded":"2024-08-31T06:05:37.072887+0000","last_fullsized":"2024-08-31T06:05:37.072887+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:38:23.533663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.044804+0000","last_change":"2024-08-31T06:04:25.896354+0000","last_active":"2024-08-31T06:05:25.044804+0000","last_peered":"2024-08-31T06:05:25.044804+0000","last_clean":"2024-08-31T06:05:25.044804+0000","last_became_active":"2024-08-31T06:04:25.896021+0000","last_became_peered":"2024-08-31T06:04:25.896021+0000","last_unstale":"2024-08-31T06:05:25.044804+0000","last_undegraded":"2024-08-31T06:05:25.044804+0000","last_fullsized":"2024-08-31T06:05:25.044804+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:08:40.213238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.047153+0000","last_change":"2024-08-31T06:04:22.844248+0000","last_active":"2024-08-31T06:05:25.047153+0000","last_peered":"2024-08-31T06:05:25.047153+0000","last_clean":"2024-08-31T06:05:25.047153+0000","last_became_active":"2024-08-31T06:04:22.843784+0000","last_became_peered":"2024-08-31T06:04:22.843784+0000","last_unstale":"2024-08-31T06:05:25.047153+0000","last_undegraded":"2024-08-31T06:05:25.047153+0000","last_fullsized":"2024-08-31T06:05:25.047153+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:56:41.422965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.046218+0000","last_change":"2024-08-31T06:04:22.847049+0000","last_active":"2024-08-31T06:05:25.046218+0000","last_peered":"2024-08-31T06:05:25.046218+0000","last_clean":"2024-08-31T06:05:25.046218+0000","last_became_active":"2024-08-31T06:04:22.846282+0000","last_became_peered":"2024-08-31T06:04:22.846282+0000","last_unstale":"2024-08-31T06:05:25.046218+0000","last_undegraded":"2024-08-31T06:05:25.046218+0000","last_fullsized":"2024-08-31T06:05:25.046218+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:35:21.030425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.107762+0000","last_change":"2024-08-31T06:04:25.886698+0000","last_active":"2024-08-31T06:05:25.107762+0000","last_peered":"2024-08-31T06:05:25.107762+0000","last_clean":"2024-08-31T06:05:25.107762+0000","last_became_active":"2024-08-31T06:04:25.886334+0000","last_became_peered":"2024-08-31T06:04:25.886334+0000","last_unstale":"2024-08-31T06:05:25.107762+0000","last_undegraded":"2024-08-31T06:05:25.107762+0000","last_fullsized":"2024-08-31T06:05:25.107762+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:24:43.821446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.044798+0000","last_change":"2024-08-31T06:04:22.845263+0000","last_active":"2024-08-31T06:05:25.044798+0000","last_peered":"2024-08-31T06:05:25.044798+0000","last_clean":"2024-08-31T06:05:25.044798+0000","last_became_active":"2024-08-31T06:04:22.844853+0000","last_became_peered":"2024-08-31T06:04:22.844853+0000","last_unstale":"2024-08-31T06:05:25.044798+0000","last_undegraded":"2024-08-31T06:05:25.044798+0000","last_fullsized":"2024-08-31T06:05:25.044798+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:55:12.561165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.043449+0000","last_change":"2024-08-31T06:04:25.894475+0000","last_active":"2024-08-31T06:05:25.043449+0000","last_peered":"2024-08-31T06:05:25.043449+0000","last_clean":"2024-08-31T06:05:25.043449+0000","last_became_active":"2024-08-31T06:04:25.894121+0000","last_became_peered":"2024-08-31T06:04:25.894121+0000","last_unstale":"2024-08-31T06:05:25.043449+0000","last_undegraded":"2024-08-31T06:05:25.043449+0000","last_fullsized":"2024-08-31T06:05:25.043449+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:40:06.570616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'3","reported_seq":88,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:38.073599+0000","last_change":"2024-08-31T06:04:22.834430+0000","last_active":"2024-08-31T06:05:38.073599+0000","last_peered":"2024-08-31T06:05:38.073599+0000","last_clean":"2024-08-31T06:05:38.073599+0000","last_became_active":"2024-08-31T06:04:22.834011+0000","last_became_peered":"2024-08-31T06:04:22.834011+0000","last_unstale":"2024-08-31T06:05:38.073599+0000","last_undegraded":"2024-08-31T06:05:38.073599+0000","last_fullsized":"2024-08-31T06:05:38.073599+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:51:16.734234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":59,"num_read_kb":59,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.049748+0000","last_change":"2024-08-31T06:04:25.877727+0000","last_active":"2024-08-31T06:05:25.049748+0000","last_peered":"2024-08-31T06:05:25.049748+0000","last_clean":"2024-08-31T06:05:25.049748+0000","last_became_active":"2024-08-31T06:04:25.877121+0000","last_became_peered":"2024-08-31T06:04:25.877121+0000","last_unstale":"2024-08-31T06:05:25.049748+0000","last_undegraded":"2024-08-31T06:05:25.049748+0000","last_fullsized":"2024-08-31T06:05:25.049748+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:41:12.247216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048823+0000","last_change":"2024-08-31T06:04:22.846482+0000","last_active":"2024-08-31T06:05:25.048823+0000","last_peered":"2024-08-31T06:05:25.048823+0000","last_clean":"2024-08-31T06:05:25.048823+0000","last_became_active":"2024-08-31T06:04:22.846057+0000","last_became_peered":"2024-08-31T06:04:22.846057+0000","last_unstale":"2024-08-31T06:05:25.048823+0000","last_undegraded":"2024-08-31T06:05:25.048823+0000","last_fullsized":"2024-08-31T06:05:25.048823+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:29:11.861665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.056422+0000","last_change":"2024-08-31T06:04:25.893777+0000","last_active":"2024-08-31T06:05:25.056422+0000","last_peered":"2024-08-31T06:05:25.056422+0000","last_clean":"2024-08-31T06:05:25.056422+0000","last_became_active":"2024-08-31T06:04:25.893014+0000","last_became_peered":"2024-08-31T06:04:25.893014+0000","last_unstale":"2024-08-31T06:05:25.056422+0000","last_undegraded":"2024-08-31T06:05:25.056422+0000","last_fullsized":"2024-08-31T06:05:25.056422+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:04:48.185842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.045472+0000","last_change":"2024-08-31T06:04:22.965938+0000","last_active":"2024-08-31T06:05:25.045472+0000","last_peered":"2024-08-31T06:05:25.045472+0000","last_clean":"2024-08-31T06:05:25.045472+0000","last_became_active":"2024-08-31T06:04:22.965356+0000","last_became_peered":"2024-08-31T06:04:22.965356+0000","last_unstale":"2024-08-31T06:05:25.045472+0000","last_undegraded":"2024-08-31T06:05:25.045472+0000","last_fullsized":"2024-08-31T06:05:25.045472+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:36:58.938462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.049791+0000","last_change":"2024-08-31T06:04:25.881307+0000","last_active":"2024-08-31T06:05:25.049791+0000","last_peered":"2024-08-31T06:05:25.049791+0000","last_clean":"2024-08-31T06:05:25.049791+0000","last_became_active":"2024-08-31T06:04:25.880418+0000","last_became_peered":"2024-08-31T06:04:25.880418+0000","last_unstale":"2024-08-31T06:05:25.049791+0000","last_undegraded":"2024-08-31T06:05:25.049791+0000","last_fullsized":"2024-08-31T06:05:25.049791+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:21:46.904371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"78'29","reported_seq":182,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:39.073142+0000","last_change":"2024-08-31T06:04:22.843073+0000","last_active":"2024-08-31T06:05:39.073142+0000","last_peered":"2024-08-31T06:05:39.073142+0000","last_clean":"2024-08-31T06:05:39.073142+0000","last_became_active":"2024-08-31T06:04:22.842645+0000","last_became_peered":"2024-08-31T06:04:22.842645+0000","last_unstale":"2024-08-31T06:05:39.073142+0000","last_undegraded":"2024-08-31T06:05:39.073142+0000","last_fullsized":"2024-08-31T06:05:39.073142+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:41:37.651562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59762,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":127,"num_read_kb":121,"num_write":30,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.053480+0000","last_change":"2024-08-31T06:04:25.879929+0000","last_active":"2024-08-31T06:05:25.053480+0000","last_peered":"2024-08-31T06:05:25.053480+0000","last_clean":"2024-08-31T06:05:25.053480+0000","last_became_active":"2024-08-31T06:04:25.878475+0000","last_became_peered":"2024-08-31T06:04:25.878475+0000","last_unstale":"2024-08-31T06:05:25.053480+0000","last_undegraded":"2024-08-31T06:05:25.053480+0000","last_fullsized":"2024-08-31T06:05:25.053480+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:10:32.801558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048188+0000","last_change":"2024-08-31T06:04:22.842878+0000","last_active":"2024-08-31T06:05:25.048188+0000","last_peered":"2024-08-31T06:05:25.048188+0000","last_clean":"2024-08-31T06:05:25.048188+0000","last_became_active":"2024-08-31T06:04:22.842306+0000","last_became_peered":"2024-08-31T06:04:22.842306+0000","last_unstale":"2024-08-31T06:05:25.048188+0000","last_undegraded":"2024-08-31T06:05:25.048188+0000","last_fullsized":"2024-08-31T06:05:25.048188+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:53:55.259238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.046274+0000","last_change":"2024-08-31T06:04:25.899357+0000","last_active":"2024-08-31T06:05:25.046274+0000","last_peered":"2024-08-31T06:05:25.046274+0000","last_clean":"2024-08-31T06:05:25.046274+0000","last_became_active":"2024-08-31T06:04:25.899000+0000","last_became_peered":"2024-08-31T06:04:25.899000+0000","last_unstale":"2024-08-31T06:05:25.046274+0000","last_undegraded":"2024-08-31T06:05:25.046274+0000","last_fullsized":"2024-08-31T06:05:25.046274+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:20:39.991071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.044693+0000","last_change":"2024-08-31T06:04:22.843063+0000","last_active":"2024-08-31T06:05:25.044693+0000","last_peered":"2024-08-31T06:05:25.044693+0000","last_clean":"2024-08-31T06:05:25.044693+0000","last_became_active":"2024-08-31T06:04:22.842654+0000","last_became_peered":"2024-08-31T06:04:22.842654+0000","last_unstale":"2024-08-31T06:05:25.044693+0000","last_undegraded":"2024-08-31T06:05:25.044693+0000","last_fullsized":"2024-08-31T06:05:25.044693+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:09:54.563116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048564+0000","last_change":"2024-08-31T06:04:25.885556+0000","last_active":"2024-08-31T06:05:25.048564+0000","last_peered":"2024-08-31T06:05:25.048564+0000","last_clean":"2024-08-31T06:05:25.048564+0000","last_became_active":"2024-08-31T06:04:25.885216+0000","last_became_peered":"2024-08-31T06:04:25.885216+0000","last_unstale":"2024-08-31T06:05:25.048564+0000","last_undegraded":"2024-08-31T06:05:25.048564+0000","last_fullsized":"2024-08-31T06:05:25.048564+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:48:49.816261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'4","reported_seq":92,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:37.599398+0000","last_change":"2024-08-31T06:04:22.844839+0000","last_active":"2024-08-31T06:05:37.599398+0000","last_peered":"2024-08-31T06:05:37.599398+0000","last_clean":"2024-08-31T06:05:37.599398+0000","last_became_active":"2024-08-31T06:04:22.844425+0000","last_became_peered":"2024-08-31T06:04:22.844425+0000","last_unstale":"2024-08-31T06:05:37.599398+0000","last_undegraded":"2024-08-31T06:05:37.599398+0000","last_fullsized":"2024-08-31T06:05:37.599398+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:10:57.292173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":63,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048224+0000","last_change":"2024-08-31T06:04:25.878195+0000","last_active":"2024-08-31T06:05:25.048224+0000","last_peered":"2024-08-31T06:05:25.048224+0000","last_clean":"2024-08-31T06:05:25.048224+0000","last_became_active":"2024-08-31T06:04:25.877823+0000","last_became_peered":"2024-08-31T06:04:25.877823+0000","last_unstale":"2024-08-31T06:05:25.048224+0000","last_undegraded":"2024-08-31T06:05:25.048224+0000","last_fullsized":"2024-08-31T06:05:25.048224+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:54:34.989583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":67,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.071462+0000","last_change":"2024-08-31T06:05:25.071391+0000","last_active":"2024-08-31T06:05:25.071462+0000","last_peered":"2024-08-31T06:05:25.071462+0000","last_clean":"2024-08-31T06:05:25.071462+0000","last_became_active":"2024-08-31T06:05:25.057546+0000","last_became_peered":"2024-08-31T06:05:25.057546+0000","last_unstale":"2024-08-31T06:05:25.071462+0000","last_undegraded":"2024-08-31T06:05:25.071462+0000","last_fullsized":"2024-08-31T06:05:25.071462+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:42:35.527395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.044680+0000","last_change":"2024-08-31T06:04:25.887580+0000","last_active":"2024-08-31T06:05:25.044680+0000","last_peered":"2024-08-31T06:05:25.044680+0000","last_clean":"2024-08-31T06:05:25.044680+0000","last_became_active":"2024-08-31T06:04:25.887158+0000","last_became_peered":"2024-08-31T06:04:25.887158+0000","last_unstale":"2024-08-31T06:05:25.044680+0000","last_undegraded":"2024-08-31T06:05:25.044680+0000","last_fullsized":"2024-08-31T06:05:25.044680+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:27:38.115742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.064443+0000","last_change":"2024-08-31T06:05:25.063859+0000","last_active":"2024-08-31T06:05:25.064443+0000","last_peered":"2024-08-31T06:05:25.064443+0000","last_clean":"2024-08-31T06:05:25.064443+0000","last_became_active":"2024-08-31T06:05:25.053596+0000","last_became_peered":"2024-08-31T06:05:25.053596+0000","last_unstale":"2024-08-31T06:05:25.064443+0000","last_undegraded":"2024-08-31T06:05:25.064443+0000","last_fullsized":"2024-08-31T06:05:25.064443+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:19:18.461630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048222+0000","last_change":"2024-08-31T06:04:25.880845+0000","last_active":"2024-08-31T06:05:25.048222+0000","last_peered":"2024-08-31T06:05:25.048222+0000","last_clean":"2024-08-31T06:05:25.048222+0000","last_became_active":"2024-08-31T06:04:25.880423+0000","last_became_peered":"2024-08-31T06:04:25.880423+0000","last_unstale":"2024-08-31T06:05:25.048222+0000","last_undegraded":"2024-08-31T06:05:25.048222+0000","last_fullsized":"2024-08-31T06:05:25.048222+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:16:56.725116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.046524+0000","last_change":"2024-08-31T06:04:22.843393+0000","last_active":"2024-08-31T06:05:25.046524+0000","last_peered":"2024-08-31T06:05:25.046524+0000","last_clean":"2024-08-31T06:05:25.046524+0000","last_became_active":"2024-08-31T06:04:22.842770+0000","last_became_peered":"2024-08-31T06:04:22.842770+0000","last_unstale":"2024-08-31T06:05:25.046524+0000","last_undegraded":"2024-08-31T06:05:25.046524+0000","last_fullsized":"2024-08-31T06:05:25.046524+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:49:16.159167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048937+0000","last_change":"2024-08-31T06:04:25.887829+0000","last_active":"2024-08-31T06:05:25.048937+0000","last_peered":"2024-08-31T06:05:25.048937+0000","last_clean":"2024-08-31T06:05:25.048937+0000","last_became_active":"2024-08-31T06:04:25.886567+0000","last_became_peered":"2024-08-31T06:04:25.886567+0000","last_unstale":"2024-08-31T06:05:25.048937+0000","last_undegraded":"2024-08-31T06:05:25.048937+0000","last_fullsized":"2024-08-31T06:05:25.048937+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:11:39.054966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.045559+0000","last_change":"2024-08-31T06:04:22.844218+0000","last_active":"2024-08-31T06:05:25.045559+0000","last_peered":"2024-08-31T06:05:25.045559+0000","last_clean":"2024-08-31T06:05:25.045559+0000","last_became_active":"2024-08-31T06:04:22.843786+0000","last_became_peered":"2024-08-31T06:04:22.843786+0000","last_unstale":"2024-08-31T06:05:25.045559+0000","last_undegraded":"2024-08-31T06:05:25.045559+0000","last_fullsized":"2024-08-31T06:05:25.045559+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:46:02.534190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'1","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048764+0000","last_change":"2024-08-31T06:04:25.888946+0000","last_active":"2024-08-31T06:05:25.048764+0000","last_peered":"2024-08-31T06:05:25.048764+0000","last_clean":"2024-08-31T06:05:25.048764+0000","last_became_active":"2024-08-31T06:04:25.888564+0000","last_became_peered":"2024-08-31T06:04:25.888564+0000","last_unstale":"2024-08-31T06:05:25.048764+0000","last_undegraded":"2024-08-31T06:05:25.048764+0000","last_fullsized":"2024-08-31T06:05:25.048764+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:26:19.051971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.049144+0000","last_change":"2024-08-31T06:04:22.964483+0000","last_active":"2024-08-31T06:05:25.049144+0000","last_peered":"2024-08-31T06:05:25.049144+0000","last_clean":"2024-08-31T06:05:25.049144+0000","last_became_active":"2024-08-31T06:04:22.963874+0000","last_became_peered":"2024-08-31T06:04:22.963874+0000","last_unstale":"2024-08-31T06:05:25.049144+0000","last_undegraded":"2024-08-31T06:05:25.049144+0000","last_fullsized":"2024-08-31T06:05:25.049144+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:30:00.159040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048856+0000","last_change":"2024-08-31T06:04:25.880941+0000","last_active":"2024-08-31T06:05:25.048856+0000","last_peered":"2024-08-31T06:05:25.048856+0000","last_clean":"2024-08-31T06:05:25.048856+0000","last_became_active":"2024-08-31T06:04:25.880454+0000","last_became_peered":"2024-08-31T06:04:25.880454+0000","last_unstale":"2024-08-31T06:05:25.048856+0000","last_undegraded":"2024-08-31T06:05:25.048856+0000","last_fullsized":"2024-08-31T06:05:25.048856+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:49:57.023988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":97,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048471+0000","last_change":"2024-08-31T06:03:05.921629+0000","last_active":"2024-08-31T06:05:25.048471+0000","last_peered":"2024-08-31T06:05:25.048471+0000","last_clean":"2024-08-31T06:05:25.048471+0000","last_became_active":"2024-08-31T06:03:05.915844+0000","last_became_peered":"2024-08-31T06:03:05.915844+0000","last_unstale":"2024-08-31T06:05:25.048471+0000","last_undegraded":"2024-08-31T06:05:25.048471+0000","last_fullsized":"2024-08-31T06:05:25.048471+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T05:59:24.472429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T05:59:24.472429+0000","last_clean_scrub_stamp":"2024-08-31T05:59:24.472429+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:56:41.073422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.044659+0000","last_change":"2024-08-31T06:04:22.846325+0000","last_active":"2024-08-31T06:05:25.044659+0000","last_peered":"2024-08-31T06:05:25.044659+0000","last_clean":"2024-08-31T06:05:25.044659+0000","last_became_active":"2024-08-31T06:04:22.845772+0000","last_became_peered":"2024-08-31T06:04:22.845772+0000","last_unstale":"2024-08-31T06:05:25.044659+0000","last_undegraded":"2024-08-31T06:05:25.044659+0000","last_fullsized":"2024-08-31T06:05:25.044659+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:54:10.336786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.107981+0000","last_change":"2024-08-31T06:04:25.891437+0000","last_active":"2024-08-31T06:05:25.107981+0000","last_peered":"2024-08-31T06:05:25.107981+0000","last_clean":"2024-08-31T06:05:25.107981+0000","last_became_active":"2024-08-31T06:04:25.891053+0000","last_became_peered":"2024-08-31T06:04:25.891053+0000","last_unstale":"2024-08-31T06:05:25.107981+0000","last_undegraded":"2024-08-31T06:05:25.107981+0000","last_fullsized":"2024-08-31T06:05:25.107981+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:22:37.119483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048351+0000","last_change":"2024-08-31T06:04:22.842885+0000","last_active":"2024-08-31T06:05:25.048351+0000","last_peered":"2024-08-31T06:05:25.048351+0000","last_clean":"2024-08-31T06:05:25.048351+0000","last_became_active":"2024-08-31T06:04:22.842473+0000","last_became_peered":"2024-08-31T06:04:22.842473+0000","last_unstale":"2024-08-31T06:05:25.048351+0000","last_undegraded":"2024-08-31T06:05:25.048351+0000","last_fullsized":"2024-08-31T06:05:25.048351+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:33:36.516659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.046184+0000","last_change":"2024-08-31T06:04:25.889213+0000","last_active":"2024-08-31T06:05:25.046184+0000","last_peered":"2024-08-31T06:05:25.046184+0000","last_clean":"2024-08-31T06:05:25.046184+0000","last_became_active":"2024-08-31T06:04:25.888860+0000","last_became_peered":"2024-08-31T06:04:25.888860+0000","last_unstale":"2024-08-31T06:05:25.046184+0000","last_undegraded":"2024-08-31T06:05:25.046184+0000","last_fullsized":"2024-08-31T06:05:25.046184+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:44:18.831229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"78'4","reported_seq":90,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:38.600719+0000","last_change":"2024-08-31T06:04:22.966091+0000","last_active":"2024-08-31T06:05:38.600719+0000","last_peered":"2024-08-31T06:05:38.600719+0000","last_clean":"2024-08-31T06:05:38.600719+0000","last_became_active":"2024-08-31T06:04:22.965727+0000","last_became_peered":"2024-08-31T06:04:22.965727+0000","last_unstale":"2024-08-31T06:05:38.600719+0000","last_undegraded":"2024-08-31T06:05:38.600719+0000","last_fullsized":"2024-08-31T06:05:38.600719+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:42:43.643623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":60,"num_read_kb":60,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.044657+0000","last_change":"2024-08-31T06:04:25.880039+0000","last_active":"2024-08-31T06:05:25.044657+0000","last_peered":"2024-08-31T06:05:25.044657+0000","last_clean":"2024-08-31T06:05:25.044657+0000","last_became_active":"2024-08-31T06:04:25.879669+0000","last_became_peered":"2024-08-31T06:04:25.879669+0000","last_unstale":"2024-08-31T06:05:25.044657+0000","last_undegraded":"2024-08-31T06:05:25.044657+0000","last_fullsized":"2024-08-31T06:05:25.044657+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:02:09.244281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048633+0000","last_change":"2024-08-31T06:04:22.844572+0000","last_active":"2024-08-31T06:05:25.048633+0000","last_peered":"2024-08-31T06:05:25.048633+0000","last_clean":"2024-08-31T06:05:25.048633+0000","last_became_active":"2024-08-31T06:04:22.844227+0000","last_became_peered":"2024-08-31T06:04:22.844227+0000","last_unstale":"2024-08-31T06:05:25.048633+0000","last_undegraded":"2024-08-31T06:05:25.048633+0000","last_fullsized":"2024-08-31T06:05:25.048633+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:04:51.966901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.047419+0000","last_change":"2024-08-31T06:04:25.884442+0000","last_active":"2024-08-31T06:05:25.047419+0000","last_peered":"2024-08-31T06:05:25.047419+0000","last_clean":"2024-08-31T06:05:25.047419+0000","last_became_active":"2024-08-31T06:04:25.884016+0000","last_became_peered":"2024-08-31T06:04:25.884016+0000","last_unstale":"2024-08-31T06:05:25.047419+0000","last_undegraded":"2024-08-31T06:05:25.047419+0000","last_fullsized":"2024-08-31T06:05:25.047419+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:36:10.640262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.043778+0000","last_change":"2024-08-31T06:04:22.967845+0000","last_active":"2024-08-31T06:05:25.043778+0000","last_peered":"2024-08-31T06:05:25.043778+0000","last_clean":"2024-08-31T06:05:25.043778+0000","last_became_active":"2024-08-31T06:04:22.967421+0000","last_became_peered":"2024-08-31T06:04:22.967421+0000","last_unstale":"2024-08-31T06:05:25.043778+0000","last_undegraded":"2024-08-31T06:05:25.043778+0000","last_fullsized":"2024-08-31T06:05:25.043778+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:17:46.942551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'1","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.047292+0000","last_change":"2024-08-31T06:04:25.897087+0000","last_active":"2024-08-31T06:05:25.047292+0000","last_peered":"2024-08-31T06:05:25.047292+0000","last_clean":"2024-08-31T06:05:25.047292+0000","last_became_active":"2024-08-31T06:04:25.896725+0000","last_became_peered":"2024-08-31T06:04:25.896725+0000","last_unstale":"2024-08-31T06:05:25.047292+0000","last_undegraded":"2024-08-31T06:05:25.047292+0000","last_fullsized":"2024-08-31T06:05:25.047292+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:31:28.279489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.053617+0000","last_change":"2024-08-31T06:04:22.965299+0000","last_active":"2024-08-31T06:05:25.053617+0000","last_peered":"2024-08-31T06:05:25.053617+0000","last_clean":"2024-08-31T06:05:25.053617+0000","last_became_active":"2024-08-31T06:04:22.964849+0000","last_became_peered":"2024-08-31T06:04:22.964849+0000","last_unstale":"2024-08-31T06:05:25.053617+0000","last_undegraded":"2024-08-31T06:05:25.053617+0000","last_fullsized":"2024-08-31T06:05:25.053617+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:36:56.431678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.046713+0000","last_change":"2024-08-31T06:04:25.885115+0000","last_active":"2024-08-31T06:05:25.046713+0000","last_peered":"2024-08-31T06:05:25.046713+0000","last_clean":"2024-08-31T06:05:25.046713+0000","last_became_active":"2024-08-31T06:04:25.884722+0000","last_became_peered":"2024-08-31T06:04:25.884722+0000","last_unstale":"2024-08-31T06:05:25.046713+0000","last_undegraded":"2024-08-31T06:05:25.046713+0000","last_fullsized":"2024-08-31T06:05:25.046713+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:20:43.913785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'11","reported_seq":38,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.043844+0000","last_change":"2024-08-31T06:04:22.845119+0000","last_active":"2024-08-31T06:05:25.043844+0000","last_peered":"2024-08-31T06:05:25.043844+0000","last_clean":"2024-08-31T06:05:25.043844+0000","last_became_active":"2024-08-31T06:04:22.843814+0000","last_became_peered":"2024-08-31T06:04:22.843814+0000","last_unstale":"2024-08-31T06:05:25.043844+0000","last_undegraded":"2024-08-31T06:05:25.043844+0000","last_fullsized":"2024-08-31T06:05:25.043844+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:06:40.684887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.108183+0000","last_change":"2024-08-31T06:04:25.892838+0000","last_active":"2024-08-31T06:05:25.108183+0000","last_peered":"2024-08-31T06:05:25.108183+0000","last_clean":"2024-08-31T06:05:25.108183+0000","last_became_active":"2024-08-31T06:04:25.891955+0000","last_became_peered":"2024-08-31T06:04:25.891955+0000","last_unstale":"2024-08-31T06:05:25.108183+0000","last_undegraded":"2024-08-31T06:05:25.108183+0000","last_fullsized":"2024-08-31T06:05:25.108183+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:41:35.085724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.044900+0000","last_change":"2024-08-31T06:04:22.964183+0000","last_active":"2024-08-31T06:05:25.044900+0000","last_peered":"2024-08-31T06:05:25.044900+0000","last_clean":"2024-08-31T06:05:25.044900+0000","last_became_active":"2024-08-31T06:04:22.963688+0000","last_became_peered":"2024-08-31T06:04:22.963688+0000","last_unstale":"2024-08-31T06:05:25.044900+0000","last_undegraded":"2024-08-31T06:05:25.044900+0000","last_fullsized":"2024-08-31T06:05:25.044900+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:04:47.833811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.056734+0000","last_change":"2024-08-31T06:04:25.892266+0000","last_active":"2024-08-31T06:05:25.056734+0000","last_peered":"2024-08-31T06:05:25.056734+0000","last_clean":"2024-08-31T06:05:25.056734+0000","last_became_active":"2024-08-31T06:04:25.889014+0000","last_became_peered":"2024-08-31T06:04:25.889014+0000","last_unstale":"2024-08-31T06:05:25.056734+0000","last_undegraded":"2024-08-31T06:05:25.056734+0000","last_fullsized":"2024-08-31T06:05:25.056734+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:27:47.608470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.049350+0000","last_change":"2024-08-31T06:04:22.964677+0000","last_active":"2024-08-31T06:05:25.049350+0000","last_peered":"2024-08-31T06:05:25.049350+0000","last_clean":"2024-08-31T06:05:25.049350+0000","last_became_active":"2024-08-31T06:04:22.964237+0000","last_became_peered":"2024-08-31T06:04:22.964237+0000","last_unstale":"2024-08-31T06:05:25.049350+0000","last_undegraded":"2024-08-31T06:05:25.049350+0000","last_fullsized":"2024-08-31T06:05:25.049350+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:00:54.099592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.051198+0000","last_change":"2024-08-31T06:04:25.881699+0000","last_active":"2024-08-31T06:05:25.051198+0000","last_peered":"2024-08-31T06:05:25.051198+0000","last_clean":"2024-08-31T06:05:25.051198+0000","last_became_active":"2024-08-31T06:04:25.879304+0000","last_became_peered":"2024-08-31T06:04:25.879304+0000","last_unstale":"2024-08-31T06:05:25.051198+0000","last_undegraded":"2024-08-31T06:05:25.051198+0000","last_fullsized":"2024-08-31T06:05:25.051198+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:56:42.992034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.108268+0000","last_change":"2024-08-31T06:04:22.847934+0000","last_active":"2024-08-31T06:05:25.108268+0000","last_peered":"2024-08-31T06:05:25.108268+0000","last_clean":"2024-08-31T06:05:25.108268+0000","last_became_active":"2024-08-31T06:04:22.847518+0000","last_became_peered":"2024-08-31T06:04:22.847518+0000","last_unstale":"2024-08-31T06:05:25.108268+0000","last_undegraded":"2024-08-31T06:05:25.108268+0000","last_fullsized":"2024-08-31T06:05:25.108268+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:26:21.476014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.047747+0000","last_change":"2024-08-31T06:04:25.884653+0000","last_active":"2024-08-31T06:05:25.047747+0000","last_peered":"2024-08-31T06:05:25.047747+0000","last_clean":"2024-08-31T06:05:25.047747+0000","last_became_active":"2024-08-31T06:04:25.884244+0000","last_became_peered":"2024-08-31T06:04:25.884244+0000","last_unstale":"2024-08-31T06:05:25.047747+0000","last_undegraded":"2024-08-31T06:05:25.047747+0000","last_fullsized":"2024-08-31T06:05:25.047747+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:13:00.814071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048578+0000","last_change":"2024-08-31T06:04:22.845924+0000","last_active":"2024-08-31T06:05:25.048578+0000","last_peered":"2024-08-31T06:05:25.048578+0000","last_clean":"2024-08-31T06:05:25.048578+0000","last_became_active":"2024-08-31T06:04:22.845445+0000","last_became_peered":"2024-08-31T06:04:22.845445+0000","last_unstale":"2024-08-31T06:05:25.048578+0000","last_undegraded":"2024-08-31T06:05:25.048578+0000","last_fullsized":"2024-08-31T06:05:25.048578+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:25:21.052136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.107948+0000","last_change":"2024-08-31T06:04:25.890069+0000","last_active":"2024-08-31T06:05:25.107948+0000","last_peered":"2024-08-31T06:05:25.107948+0000","last_clean":"2024-08-31T06:05:25.107948+0000","last_became_active":"2024-08-31T06:04:25.889709+0000","last_became_peered":"2024-08-31T06:04:25.889709+0000","last_unstale":"2024-08-31T06:05:25.107948+0000","last_undegraded":"2024-08-31T06:05:25.107948+0000","last_fullsized":"2024-08-31T06:05:25.107948+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:13:51.360023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.051547+0000","last_change":"2024-08-31T06:04:22.847845+0000","last_active":"2024-08-31T06:05:25.051547+0000","last_peered":"2024-08-31T06:05:25.051547+0000","last_clean":"2024-08-31T06:05:25.051547+0000","last_became_active":"2024-08-31T06:04:22.847001+0000","last_became_peered":"2024-08-31T06:04:22.847001+0000","last_unstale":"2024-08-31T06:05:25.051547+0000","last_undegraded":"2024-08-31T06:05:25.051547+0000","last_fullsized":"2024-08-31T06:05:25.051547+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:24:34.209076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048518+0000","last_change":"2024-08-31T06:04:25.886759+0000","last_active":"2024-08-31T06:05:25.048518+0000","last_peered":"2024-08-31T06:05:25.048518+0000","last_clean":"2024-08-31T06:05:25.048518+0000","last_became_active":"2024-08-31T06:04:25.886176+0000","last_became_peered":"2024-08-31T06:04:25.886176+0000","last_unstale":"2024-08-31T06:05:25.048518+0000","last_undegraded":"2024-08-31T06:05:25.048518+0000","last_fullsized":"2024-08-31T06:05:25.048518+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:11:12.637451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.054559+0000","last_change":"2024-08-31T06:05:25.054468+0000","last_active":"2024-08-31T06:05:25.054559+0000","last_peered":"2024-08-31T06:05:25.054559+0000","last_clean":"2024-08-31T06:05:25.054559+0000","last_became_active":"2024-08-31T06:05:25.053657+0000","last_became_peered":"2024-08-31T06:05:25.053657+0000","last_unstale":"2024-08-31T06:05:25.054559+0000","last_undegraded":"2024-08-31T06:05:25.054559+0000","last_fullsized":"2024-08-31T06:05:25.054559+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:43:44.887637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.059757+0000","last_change":"2024-08-31T06:05:25.059219+0000","last_active":"2024-08-31T06:05:25.059757+0000","last_peered":"2024-08-31T06:05:25.059757+0000","last_clean":"2024-08-31T06:05:25.059757+0000","last_became_active":"2024-08-31T06:05:25.053136+0000","last_became_peered":"2024-08-31T06:05:25.053136+0000","last_unstale":"2024-08-31T06:05:25.059757+0000","last_undegraded":"2024-08-31T06:05:25.059757+0000","last_fullsized":"2024-08-31T06:05:25.059757+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:29:59.137465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.049696+0000","last_change":"2024-08-31T06:04:25.887524+0000","last_active":"2024-08-31T06:05:25.049696+0000","last_peered":"2024-08-31T06:05:25.049696+0000","last_clean":"2024-08-31T06:05:25.049696+0000","last_became_active":"2024-08-31T06:04:25.887176+0000","last_became_peered":"2024-08-31T06:04:25.887176+0000","last_unstale":"2024-08-31T06:05:25.049696+0000","last_undegraded":"2024-08-31T06:05:25.049696+0000","last_fullsized":"2024-08-31T06:05:25.049696+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:08:05.476804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.053524+0000","last_change":"2024-08-31T06:04:22.847747+0000","last_active":"2024-08-31T06:05:25.053524+0000","last_peered":"2024-08-31T06:05:25.053524+0000","last_clean":"2024-08-31T06:05:25.053524+0000","last_became_active":"2024-08-31T06:04:22.847332+0000","last_became_peered":"2024-08-31T06:04:22.847332+0000","last_unstale":"2024-08-31T06:05:25.053524+0000","last_undegraded":"2024-08-31T06:05:25.053524+0000","last_fullsized":"2024-08-31T06:05:25.053524+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:41:35.910991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.056293+0000","last_change":"2024-08-31T06:04:25.883110+0000","last_active":"2024-08-31T06:05:25.056293+0000","last_peered":"2024-08-31T06:05:25.056293+0000","last_clean":"2024-08-31T06:05:25.056293+0000","last_became_active":"2024-08-31T06:04:25.880487+0000","last_became_peered":"2024-08-31T06:04:25.880487+0000","last_unstale":"2024-08-31T06:05:25.056293+0000","last_undegraded":"2024-08-31T06:05:25.056293+0000","last_fullsized":"2024-08-31T06:05:25.056293+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:22:13.656078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.108180+0000","last_change":"2024-08-31T06:04:22.964758+0000","last_active":"2024-08-31T06:05:25.108180+0000","last_peered":"2024-08-31T06:05:25.108180+0000","last_clean":"2024-08-31T06:05:25.108180+0000","last_became_active":"2024-08-31T06:04:22.964391+0000","last_became_peered":"2024-08-31T06:04:22.964391+0000","last_unstale":"2024-08-31T06:05:25.108180+0000","last_undegraded":"2024-08-31T06:05:25.108180+0000","last_fullsized":"2024-08-31T06:05:25.108180+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:28:59.190657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.108031+0000","last_change":"2024-08-31T06:04:25.893738+0000","last_active":"2024-08-31T06:05:25.108031+0000","last_peered":"2024-08-31T06:05:25.108031+0000","last_clean":"2024-08-31T06:05:25.108031+0000","last_became_active":"2024-08-31T06:04:25.893308+0000","last_became_peered":"2024-08-31T06:04:25.893308+0000","last_unstale":"2024-08-31T06:05:25.108031+0000","last_undegraded":"2024-08-31T06:05:25.108031+0000","last_fullsized":"2024-08-31T06:05:25.108031+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:26:48.451894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.043929+0000","last_change":"2024-08-31T06:04:22.843346+0000","last_active":"2024-08-31T06:05:25.043929+0000","last_peered":"2024-08-31T06:05:25.043929+0000","last_clean":"2024-08-31T06:05:25.043929+0000","last_became_active":"2024-08-31T06:04:22.842249+0000","last_became_peered":"2024-08-31T06:04:22.842249+0000","last_unstale":"2024-08-31T06:05:25.043929+0000","last_undegraded":"2024-08-31T06:05:25.043929+0000","last_fullsized":"2024-08-31T06:05:25.043929+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:40:57.618940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64154,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":377,"num_read_kb":259,"num_write":165,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1147,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":192462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":249373,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710749,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":872448,"data_stored":436463,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874273,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1019904,"data_stored":1088075,"data_compressed":42124,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037801,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":884736,"data_stored":436677,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234030,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":436575,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397556,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":933888,"data_stored":1028918,"data_compressed":42124,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593785,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":872448,"data_stored":436066,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790013,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":946176,"data_stored":1028211,"data_compressed":42124,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953540,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":884736,"data_stored":437907,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149768,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":888832,"data_stored":436377,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":942080,"data_stored":496237,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542227,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36268,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708884,"statfs":{"total":95995035648,"available":95957897216,"internally_reserved":0,"allocated":942080,"data_stored":495733,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705753,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":892928,"data_stored":437337,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34699,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34699,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34699,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":59874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":60378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-31T06:05:46.097 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph pg dump --format=json 2024-08-31T06:05:46.477 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:05:46.615 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:46 smithi016 ceph-mon[17194]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T06:05:46.616 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:46 smithi016 ceph-mon[17194]: pgmap v319: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-31T06:05:46.621 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:46 smithi155 ceph-mon[21742]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T06:05:46.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:46 smithi155 ceph-mon[21742]: pgmap v319: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-31T06:05:46.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:46 smithi184 ceph-mon[22331]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T06:05:46.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:46 smithi184 ceph-mon[22331]: pgmap v319: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-31T06:05:47.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.102+0000 7f21d5a00640 1 -- 172.21.15.16:0/2966245522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21d0103900 msgr2=0x7f21d0103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:47.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.102+0000 7f21d5a00640 1 --2- 172.21.15.16:0/2966245522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21d0103900 0x7f21d0103d10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f21c0009900 tx=0x7f21c002a410 comp rx=0 tx=0).stop 2024-08-31T06:05:47.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.102+0000 7f21d5a00640 1 -- 172.21.15.16:0/2966245522 shutdown_connections 2024-08-31T06:05:47.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.102+0000 7f21d5a00640 1 --2- 172.21.15.16:0/2966245522 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21d0109240 0x7f21d0109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:47.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.102+0000 7f21d5a00640 1 --2- 172.21.15.16:0/2966245522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21d0109b90 0x7f21d01103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:47.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.102+0000 7f21d5a00640 1 --2- 172.21.15.16:0/2966245522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21d0103900 0x7f21d0103d10 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:47.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.102+0000 7f21d5a00640 1 -- 172.21.15.16:0/2966245522 >> 172.21.15.16:0/2966245522 conn(0x7f21d007b5f0 msgr2=0x7f21d007ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:47.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.102+0000 7f21d5a00640 1 -- 172.21.15.16:0/2966245522 shutdown_connections 2024-08-31T06:05:47.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.102+0000 7f21d5a00640 1 -- 172.21.15.16:0/2966245522 wait complete. 2024-08-31T06:05:47.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.103+0000 7f21d5a00640 1 Processor -- start 2024-08-31T06:05:47.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.103+0000 7f21d5a00640 1 -- start start 2024-08-31T06:05:47.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.103+0000 7f21d5a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21d0103900 0x7f21d019a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:47.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.103+0000 7f21d5a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21d0109240 0x7f21d019a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:47.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.103+0000 7f21d5a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21d0109b90 0x7f21d019fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:47.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.103+0000 7f21d5a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f21d0113850 con 0x7f21d0103900 2024-08-31T06:05:47.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.103+0000 7f21d5a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f21d01136d0 con 0x7f21d0109240 2024-08-31T06:05:47.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.103+0000 7f21d5a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f21d01139d0 con 0x7f21d0109b90 2024-08-31T06:05:47.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.104+0000 7f21cea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21d0103900 0x7f21d019a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:47.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.104+0000 7f21c6000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21d0109240 0x7f21d019a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:47.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.104+0000 7f21c6000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21d0109240 0x7f21d019a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:53214/0 (socket says 172.21.15.16:53214) 2024-08-31T06:05:47.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.104+0000 7f21cea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21d0103900 0x7f21d019a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39038/0 (socket says 172.21.15.16:39038) 2024-08-31T06:05:47.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.104+0000 7f21c6000640 1 -- 172.21.15.16:0/1775437664 learned_addr learned my addr 172.21.15.16:0/1775437664 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:05:47.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.104+0000 7f21cf400640 1 --2- 172.21.15.16:0/1775437664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21d0109b90 0x7f21d019fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:47.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.104+0000 7f21c6000640 1 -- 172.21.15.16:0/1775437664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21d0109b90 msgr2=0x7f21d019fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:47.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.104+0000 7f21c6000640 1 --2- 172.21.15.16:0/1775437664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21d0109b90 0x7f21d019fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:47.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.104+0000 7f21c6000640 1 -- 172.21.15.16:0/1775437664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21d0103900 msgr2=0x7f21d019a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:47.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.104+0000 7f21c6000640 1 --2- 172.21.15.16:0/1775437664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21d0103900 0x7f21d019a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:47.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.104+0000 7f21c6000640 1 -- 172.21.15.16:0/1775437664 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21d01a01b0 con 0x7f21d0109240 2024-08-31T06:05:47.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.104+0000 7f21cea00640 1 --2- 172.21.15.16:0/1775437664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21d0103900 0x7f21d019a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:05:47.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.105+0000 7f21c6000640 1 --2- 172.21.15.16:0/1775437664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21d0109240 0x7f21d019a7e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f21b800e9c0 tx=0x7f21b800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:47.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.105+0000 7f21c7e00640 1 -- 172.21.15.16:0/1775437664 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b800ccc0 con 0x7f21d0109240 2024-08-31T06:05:47.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.105+0000 7f21c7e00640 1 -- 172.21.15.16:0/1775437664 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21b80054d0 con 0x7f21d0109240 2024-08-31T06:05:47.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.105+0000 7f21d5a00640 1 -- 172.21.15.16:0/1775437664 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21d01a04a0 con 0x7f21d0109240 2024-08-31T06:05:47.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.105+0000 7f21c7e00640 1 -- 172.21.15.16:0/1775437664 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b8010430 con 0x7f21d0109240 2024-08-31T06:05:47.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.105+0000 7f21d5a00640 1 -- 172.21.15.16:0/1775437664 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21d01a6c00 con 0x7f21d0109240 2024-08-31T06:05:47.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.106+0000 7f21d5a00640 1 -- 172.21.15.16:0/1775437664 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21d0103d10 con 0x7f21d0109240 2024-08-31T06:05:47.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.110+0000 7f21c7e00640 1 -- 172.21.15.16:0/1775437664 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f21b8005000 con 0x7f21d0109240 2024-08-31T06:05:47.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.111+0000 7f21c7e00640 1 --2- 172.21.15.16:0/1775437664 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f21b40b4c10 0x7f21b40b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:47.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.112+0000 7f21c7e00640 1 -- 172.21.15.16:0/1775437664 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f21b80e5920 con 0x7f21d0109240 2024-08-31T06:05:47.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.112+0000 7f21cea00640 1 --2- 172.21.15.16:0/1775437664 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f21b40b4c10 0x7f21b40b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:47.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.112+0000 7f21c7e00640 1 -- 172.21.15.16:0/1775437664 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f21b8056130 con 0x7f21d0109240 2024-08-31T06:05:47.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.112+0000 7f21cea00640 1 --2- 172.21.15.16:0/1775437664 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f21b40b4c10 0x7f21b40b7120 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f21c0031440 tx=0x7f21c0032ed0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:47.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.307+0000 7f21d5a00640 1 -- 172.21.15.16:0/1775437664 --> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f21d00635a0 con 0x7f21b40b4c10 2024-08-31T06:05:47.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.312+0000 7f21c7e00640 1 -- 172.21.15.16:0/1775437664 <== mgr.14220 v2:172.21.15.16:6800/4279751337 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258944 (secure 0 0 0) 0x7f21d00635a0 con 0x7f21b40b4c10 2024-08-31T06:05:47.317 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:05:47.318 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-08-31T06:05:47.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.317+0000 7f21d5a00640 1 -- 172.21.15.16:0/1775437664 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f21b40b4c10 msgr2=0x7f21b40b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:47.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.317+0000 7f21d5a00640 1 --2- 172.21.15.16:0/1775437664 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f21b40b4c10 0x7f21b40b7120 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f21c0031440 tx=0x7f21c0032ed0 comp rx=0 tx=0).stop 2024-08-31T06:05:47.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.317+0000 7f21d5a00640 1 -- 172.21.15.16:0/1775437664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21d0109240 msgr2=0x7f21d019a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:47.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.317+0000 7f21d5a00640 1 --2- 172.21.15.16:0/1775437664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21d0109240 0x7f21d019a7e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f21b800e9c0 tx=0x7f21b800ee80 comp rx=0 tx=0).stop 2024-08-31T06:05:47.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.317+0000 7f21d5a00640 1 -- 172.21.15.16:0/1775437664 shutdown_connections 2024-08-31T06:05:47.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.317+0000 7f21d5a00640 1 --2- 172.21.15.16:0/1775437664 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f21b40b4c10 0x7f21b40b7120 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:47.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.317+0000 7f21d5a00640 1 --2- 172.21.15.16:0/1775437664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21d0109b90 0x7f21d019fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:47.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.317+0000 7f21d5a00640 1 --2- 172.21.15.16:0/1775437664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21d0109240 0x7f21d019a7e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:47.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.317+0000 7f21d5a00640 1 --2- 172.21.15.16:0/1775437664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21d0103900 0x7f21d019a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:47.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.317+0000 7f21d5a00640 1 -- 172.21.15.16:0/1775437664 >> 172.21.15.16:0/1775437664 conn(0x7f21d007b5f0 msgr2=0x7f21d01072d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:47.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.317+0000 7f21d5a00640 1 -- 172.21.15.16:0/1775437664 shutdown_connections 2024-08-31T06:05:47.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:47.317+0000 7f21d5a00640 1 -- 172.21.15.16:0/1775437664 wait complete. 2024-08-31T06:05:48.075 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":320,"stamp":"2024-08-31T06:05:47.275904+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654756,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":488,"num_read_kb":343,"num_write":290,"num_write_kb":2263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":2952987,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":234,"ondisk_log_size":234,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360028,"kb_used_data":10704,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581796,"statfs":{"total":1151940427776,"available":1151571759104,"internally_reserved":0,"allocated":10960896,"data_stored":7194576,"data_compressed":178617,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003345"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048210+0000","last_change":"2024-08-31T06:04:25.877318+0000","last_active":"2024-08-31T06:05:25.048210+0000","last_peered":"2024-08-31T06:05:25.048210+0000","last_clean":"2024-08-31T06:05:25.048210+0000","last_became_active":"2024-08-31T06:04:25.876894+0000","last_became_peered":"2024-08-31T06:04:25.876894+0000","last_unstale":"2024-08-31T06:05:25.048210+0000","last_undegraded":"2024-08-31T06:05:25.048210+0000","last_fullsized":"2024-08-31T06:05:25.048210+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:41:09.449343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.056660+0000","last_change":"2024-08-31T06:04:25.885670+0000","last_active":"2024-08-31T06:05:25.056660+0000","last_peered":"2024-08-31T06:05:25.056660+0000","last_clean":"2024-08-31T06:05:25.056660+0000","last_became_active":"2024-08-31T06:04:25.883818+0000","last_became_peered":"2024-08-31T06:04:25.883818+0000","last_unstale":"2024-08-31T06:05:25.056660+0000","last_undegraded":"2024-08-31T06:05:25.056660+0000","last_fullsized":"2024-08-31T06:05:25.056660+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:40:01.344432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.056294+0000","last_change":"2024-08-31T06:04:25.883375+0000","last_active":"2024-08-31T06:05:25.056294+0000","last_peered":"2024-08-31T06:05:25.056294+0000","last_clean":"2024-08-31T06:05:25.056294+0000","last_became_active":"2024-08-31T06:04:25.881750+0000","last_became_peered":"2024-08-31T06:04:25.881750+0000","last_unstale":"2024-08-31T06:05:25.056294+0000","last_undegraded":"2024-08-31T06:05:25.056294+0000","last_fullsized":"2024-08-31T06:05:25.056294+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:46:46.757363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.044794+0000","last_change":"2024-08-31T06:04:25.882436+0000","last_active":"2024-08-31T06:05:25.044794+0000","last_peered":"2024-08-31T06:05:25.044794+0000","last_clean":"2024-08-31T06:05:25.044794+0000","last_became_active":"2024-08-31T06:04:25.881805+0000","last_became_peered":"2024-08-31T06:04:25.881805+0000","last_unstale":"2024-08-31T06:05:25.044794+0000","last_undegraded":"2024-08-31T06:05:25.044794+0000","last_fullsized":"2024-08-31T06:05:25.044794+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:42:39.793625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.051404+0000","last_change":"2024-08-31T06:04:25.894534+0000","last_active":"2024-08-31T06:05:25.051404+0000","last_peered":"2024-08-31T06:05:25.051404+0000","last_clean":"2024-08-31T06:05:25.051404+0000","last_became_active":"2024-08-31T06:04:25.894116+0000","last_became_peered":"2024-08-31T06:04:25.894116+0000","last_unstale":"2024-08-31T06:05:25.051404+0000","last_undegraded":"2024-08-31T06:05:25.051404+0000","last_fullsized":"2024-08-31T06:05:25.051404+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:02:31.257454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.046795+0000","last_change":"2024-08-31T06:04:25.886182+0000","last_active":"2024-08-31T06:05:25.046795+0000","last_peered":"2024-08-31T06:05:25.046795+0000","last_clean":"2024-08-31T06:05:25.046795+0000","last_became_active":"2024-08-31T06:04:25.885825+0000","last_became_peered":"2024-08-31T06:04:25.885825+0000","last_unstale":"2024-08-31T06:05:25.046795+0000","last_undegraded":"2024-08-31T06:05:25.046795+0000","last_fullsized":"2024-08-31T06:05:25.046795+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:58:12.427931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048313+0000","last_change":"2024-08-31T06:04:25.877521+0000","last_active":"2024-08-31T06:05:25.048313+0000","last_peered":"2024-08-31T06:05:25.048313+0000","last_clean":"2024-08-31T06:05:25.048313+0000","last_became_active":"2024-08-31T06:04:25.877122+0000","last_became_peered":"2024-08-31T06:04:25.877122+0000","last_unstale":"2024-08-31T06:05:25.048313+0000","last_undegraded":"2024-08-31T06:05:25.048313+0000","last_fullsized":"2024-08-31T06:05:25.048313+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:17:27.406828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.107691+0000","last_change":"2024-08-31T06:04:25.892547+0000","last_active":"2024-08-31T06:05:25.107691+0000","last_peered":"2024-08-31T06:05:25.107691+0000","last_clean":"2024-08-31T06:05:25.107691+0000","last_became_active":"2024-08-31T06:04:25.892090+0000","last_became_peered":"2024-08-31T06:04:25.892090+0000","last_unstale":"2024-08-31T06:05:25.107691+0000","last_undegraded":"2024-08-31T06:05:25.107691+0000","last_fullsized":"2024-08-31T06:05:25.107691+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:28:53.414252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.050038+0000","last_change":"2024-08-31T06:04:25.879952+0000","last_active":"2024-08-31T06:05:25.050038+0000","last_peered":"2024-08-31T06:05:25.050038+0000","last_clean":"2024-08-31T06:05:25.050038+0000","last_became_active":"2024-08-31T06:04:25.879568+0000","last_became_peered":"2024-08-31T06:04:25.879568+0000","last_unstale":"2024-08-31T06:05:25.050038+0000","last_undegraded":"2024-08-31T06:05:25.050038+0000","last_fullsized":"2024-08-31T06:05:25.050038+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:28:04.558054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.051270+0000","last_change":"2024-08-31T06:05:25.051173+0000","last_active":"2024-08-31T06:05:25.051270+0000","last_peered":"2024-08-31T06:05:25.051270+0000","last_clean":"2024-08-31T06:05:25.051270+0000","last_became_active":"2024-08-31T06:05:25.050867+0000","last_became_peered":"2024-08-31T06:05:25.050867+0000","last_unstale":"2024-08-31T06:05:25.051270+0000","last_undegraded":"2024-08-31T06:05:25.051270+0000","last_fullsized":"2024-08-31T06:05:25.051270+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:27:09.971497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048858+0000","last_change":"2024-08-31T06:04:25.883984+0000","last_active":"2024-08-31T06:05:25.048858+0000","last_peered":"2024-08-31T06:05:25.048858+0000","last_clean":"2024-08-31T06:05:25.048858+0000","last_became_active":"2024-08-31T06:04:25.883353+0000","last_became_peered":"2024-08-31T06:04:25.883353+0000","last_unstale":"2024-08-31T06:05:25.048858+0000","last_undegraded":"2024-08-31T06:05:25.048858+0000","last_fullsized":"2024-08-31T06:05:25.048858+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:16:25.178952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.044850+0000","last_change":"2024-08-31T06:04:25.883335+0000","last_active":"2024-08-31T06:05:25.044850+0000","last_peered":"2024-08-31T06:05:25.044850+0000","last_clean":"2024-08-31T06:05:25.044850+0000","last_became_active":"2024-08-31T06:04:25.881488+0000","last_became_peered":"2024-08-31T06:04:25.881488+0000","last_unstale":"2024-08-31T06:05:25.044850+0000","last_undegraded":"2024-08-31T06:05:25.044850+0000","last_fullsized":"2024-08-31T06:05:25.044850+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:57:24.472482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.046725+0000","last_change":"2024-08-31T06:04:25.891579+0000","last_active":"2024-08-31T06:05:25.046725+0000","last_peered":"2024-08-31T06:05:25.046725+0000","last_clean":"2024-08-31T06:05:25.046725+0000","last_became_active":"2024-08-31T06:04:25.887950+0000","last_became_peered":"2024-08-31T06:04:25.887950+0000","last_unstale":"2024-08-31T06:05:25.046725+0000","last_undegraded":"2024-08-31T06:05:25.046725+0000","last_fullsized":"2024-08-31T06:05:25.046725+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:38:35.284442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.056229+0000","last_change":"2024-08-31T06:04:25.882185+0000","last_active":"2024-08-31T06:05:25.056229+0000","last_peered":"2024-08-31T06:05:25.056229+0000","last_clean":"2024-08-31T06:05:25.056229+0000","last_became_active":"2024-08-31T06:04:25.876499+0000","last_became_peered":"2024-08-31T06:04:25.876499+0000","last_unstale":"2024-08-31T06:05:25.056229+0000","last_undegraded":"2024-08-31T06:05:25.056229+0000","last_fullsized":"2024-08-31T06:05:25.056229+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:40:01.132353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.056113+0000","last_change":"2024-08-31T06:04:25.883624+0000","last_active":"2024-08-31T06:05:25.056113+0000","last_peered":"2024-08-31T06:05:25.056113+0000","last_clean":"2024-08-31T06:05:25.056113+0000","last_became_active":"2024-08-31T06:04:25.878041+0000","last_became_peered":"2024-08-31T06:04:25.878041+0000","last_unstale":"2024-08-31T06:05:25.056113+0000","last_undegraded":"2024-08-31T06:05:25.056113+0000","last_fullsized":"2024-08-31T06:05:25.056113+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:05:21.278321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.049208+0000","last_change":"2024-08-31T06:04:25.882538+0000","last_active":"2024-08-31T06:05:25.049208+0000","last_peered":"2024-08-31T06:05:25.049208+0000","last_clean":"2024-08-31T06:05:25.049208+0000","last_became_active":"2024-08-31T06:04:25.882201+0000","last_became_peered":"2024-08-31T06:04:25.882201+0000","last_unstale":"2024-08-31T06:05:25.049208+0000","last_undegraded":"2024-08-31T06:05:25.049208+0000","last_fullsized":"2024-08-31T06:05:25.049208+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:47:16.880893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048602+0000","last_change":"2024-08-31T06:04:25.880376+0000","last_active":"2024-08-31T06:05:25.048602+0000","last_peered":"2024-08-31T06:05:25.048602+0000","last_clean":"2024-08-31T06:05:25.048602+0000","last_became_active":"2024-08-31T06:04:25.877668+0000","last_became_peered":"2024-08-31T06:04:25.877668+0000","last_unstale":"2024-08-31T06:05:25.048602+0000","last_undegraded":"2024-08-31T06:05:25.048602+0000","last_fullsized":"2024-08-31T06:05:25.048602+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:55:32.737406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.043803+0000","last_change":"2024-08-31T06:04:25.875431+0000","last_active":"2024-08-31T06:05:25.043803+0000","last_peered":"2024-08-31T06:05:25.043803+0000","last_clean":"2024-08-31T06:05:25.043803+0000","last_became_active":"2024-08-31T06:04:25.874751+0000","last_became_peered":"2024-08-31T06:04:25.874751+0000","last_unstale":"2024-08-31T06:05:25.043803+0000","last_undegraded":"2024-08-31T06:05:25.043803+0000","last_fullsized":"2024-08-31T06:05:25.043803+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:16:57.695415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.049406+0000","last_change":"2024-08-31T06:04:25.895656+0000","last_active":"2024-08-31T06:05:25.049406+0000","last_peered":"2024-08-31T06:05:25.049406+0000","last_clean":"2024-08-31T06:05:25.049406+0000","last_became_active":"2024-08-31T06:04:25.895272+0000","last_became_peered":"2024-08-31T06:04:25.895272+0000","last_unstale":"2024-08-31T06:05:25.049406+0000","last_undegraded":"2024-08-31T06:05:25.049406+0000","last_fullsized":"2024-08-31T06:05:25.049406+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:18:47.276195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.047722+0000","last_change":"2024-08-31T06:04:25.877283+0000","last_active":"2024-08-31T06:05:25.047722+0000","last_peered":"2024-08-31T06:05:25.047722+0000","last_clean":"2024-08-31T06:05:25.047722+0000","last_became_active":"2024-08-31T06:04:25.876846+0000","last_became_peered":"2024-08-31T06:04:25.876846+0000","last_unstale":"2024-08-31T06:05:25.047722+0000","last_undegraded":"2024-08-31T06:05:25.047722+0000","last_fullsized":"2024-08-31T06:05:25.047722+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:54:05.086844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.046754+0000","last_change":"2024-08-31T06:04:25.884891+0000","last_active":"2024-08-31T06:05:25.046754+0000","last_peered":"2024-08-31T06:05:25.046754+0000","last_clean":"2024-08-31T06:05:25.046754+0000","last_became_active":"2024-08-31T06:04:25.884501+0000","last_became_peered":"2024-08-31T06:04:25.884501+0000","last_unstale":"2024-08-31T06:05:25.046754+0000","last_undegraded":"2024-08-31T06:05:25.046754+0000","last_fullsized":"2024-08-31T06:05:25.046754+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:22:54.846102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.049559+0000","last_change":"2024-08-31T06:04:25.877150+0000","last_active":"2024-08-31T06:05:25.049559+0000","last_peered":"2024-08-31T06:05:25.049559+0000","last_clean":"2024-08-31T06:05:25.049559+0000","last_became_active":"2024-08-31T06:04:25.876766+0000","last_became_peered":"2024-08-31T06:04:25.876766+0000","last_unstale":"2024-08-31T06:05:25.049559+0000","last_undegraded":"2024-08-31T06:05:25.049559+0000","last_fullsized":"2024-08-31T06:05:25.049559+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:46:26.724970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.107727+0000","last_change":"2024-08-31T06:04:25.881275+0000","last_active":"2024-08-31T06:05:25.107727+0000","last_peered":"2024-08-31T06:05:25.107727+0000","last_clean":"2024-08-31T06:05:25.107727+0000","last_became_active":"2024-08-31T06:04:25.879877+0000","last_became_peered":"2024-08-31T06:04:25.879877+0000","last_unstale":"2024-08-31T06:05:25.107727+0000","last_undegraded":"2024-08-31T06:05:25.107727+0000","last_fullsized":"2024-08-31T06:05:25.107727+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:07:07.237041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.044574+0000","last_change":"2024-08-31T06:04:25.895750+0000","last_active":"2024-08-31T06:05:25.044574+0000","last_peered":"2024-08-31T06:05:25.044574+0000","last_clean":"2024-08-31T06:05:25.044574+0000","last_became_active":"2024-08-31T06:04:25.895348+0000","last_became_peered":"2024-08-31T06:04:25.895348+0000","last_unstale":"2024-08-31T06:05:25.044574+0000","last_undegraded":"2024-08-31T06:05:25.044574+0000","last_fullsized":"2024-08-31T06:05:25.044574+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:53:10.014808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.108056+0000","last_change":"2024-08-31T06:04:25.895537+0000","last_active":"2024-08-31T06:05:25.108056+0000","last_peered":"2024-08-31T06:05:25.108056+0000","last_clean":"2024-08-31T06:05:25.108056+0000","last_became_active":"2024-08-31T06:04:25.895136+0000","last_became_peered":"2024-08-31T06:04:25.895136+0000","last_unstale":"2024-08-31T06:05:25.108056+0000","last_undegraded":"2024-08-31T06:05:25.108056+0000","last_fullsized":"2024-08-31T06:05:25.108056+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:41:57.554507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048946+0000","last_change":"2024-08-31T06:04:25.880490+0000","last_active":"2024-08-31T06:05:25.048946+0000","last_peered":"2024-08-31T06:05:25.048946+0000","last_clean":"2024-08-31T06:05:25.048946+0000","last_became_active":"2024-08-31T06:04:25.879269+0000","last_became_peered":"2024-08-31T06:04:25.879269+0000","last_unstale":"2024-08-31T06:05:25.048946+0000","last_undegraded":"2024-08-31T06:05:25.048946+0000","last_fullsized":"2024-08-31T06:05:25.048946+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:05:49.575915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.053834+0000","last_change":"2024-08-31T06:05:25.053714+0000","last_active":"2024-08-31T06:05:25.053834+0000","last_peered":"2024-08-31T06:05:25.053834+0000","last_clean":"2024-08-31T06:05:25.053834+0000","last_became_active":"2024-08-31T06:05:25.052685+0000","last_became_peered":"2024-08-31T06:05:25.052685+0000","last_unstale":"2024-08-31T06:05:25.053834+0000","last_undegraded":"2024-08-31T06:05:25.053834+0000","last_fullsized":"2024-08-31T06:05:25.053834+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:17:31.474601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.058065+0000","last_change":"2024-08-31T06:05:25.057946+0000","last_active":"2024-08-31T06:05:25.058065+0000","last_peered":"2024-08-31T06:05:25.058065+0000","last_clean":"2024-08-31T06:05:25.058065+0000","last_became_active":"2024-08-31T06:05:25.057501+0000","last_became_peered":"2024-08-31T06:05:25.057501+0000","last_unstale":"2024-08-31T06:05:25.058065+0000","last_undegraded":"2024-08-31T06:05:25.058065+0000","last_fullsized":"2024-08-31T06:05:25.058065+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:16:05.721770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.056527+0000","last_change":"2024-08-31T06:04:25.897473+0000","last_active":"2024-08-31T06:05:25.056527+0000","last_peered":"2024-08-31T06:05:25.056527+0000","last_clean":"2024-08-31T06:05:25.056527+0000","last_became_active":"2024-08-31T06:04:25.895194+0000","last_became_peered":"2024-08-31T06:04:25.895194+0000","last_unstale":"2024-08-31T06:05:25.056527+0000","last_undegraded":"2024-08-31T06:05:25.056527+0000","last_fullsized":"2024-08-31T06:05:25.056527+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:43:04.539970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048442+0000","last_change":"2024-08-31T06:04:25.894402+0000","last_active":"2024-08-31T06:05:25.048442+0000","last_peered":"2024-08-31T06:05:25.048442+0000","last_clean":"2024-08-31T06:05:25.048442+0000","last_became_active":"2024-08-31T06:04:25.893883+0000","last_became_peered":"2024-08-31T06:04:25.893883+0000","last_unstale":"2024-08-31T06:05:25.048442+0000","last_undegraded":"2024-08-31T06:05:25.048442+0000","last_fullsized":"2024-08-31T06:05:25.048442+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:45:53.778473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048292+0000","last_change":"2024-08-31T06:04:25.881543+0000","last_active":"2024-08-31T06:05:25.048292+0000","last_peered":"2024-08-31T06:05:25.048292+0000","last_clean":"2024-08-31T06:05:25.048292+0000","last_became_active":"2024-08-31T06:04:25.881144+0000","last_became_peered":"2024-08-31T06:04:25.881144+0000","last_unstale":"2024-08-31T06:05:25.048292+0000","last_undegraded":"2024-08-31T06:05:25.048292+0000","last_fullsized":"2024-08-31T06:05:25.048292+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:13:02.282879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.047616+0000","last_change":"2024-08-31T06:04:25.881143+0000","last_active":"2024-08-31T06:05:25.047616+0000","last_peered":"2024-08-31T06:05:25.047616+0000","last_clean":"2024-08-31T06:05:25.047616+0000","last_became_active":"2024-08-31T06:04:25.879718+0000","last_became_peered":"2024-08-31T06:04:25.879718+0000","last_unstale":"2024-08-31T06:05:25.047616+0000","last_undegraded":"2024-08-31T06:05:25.047616+0000","last_fullsized":"2024-08-31T06:05:25.047616+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:32:07.771488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.051368+0000","last_change":"2024-08-31T06:04:25.879493+0000","last_active":"2024-08-31T06:05:25.051368+0000","last_peered":"2024-08-31T06:05:25.051368+0000","last_clean":"2024-08-31T06:05:25.051368+0000","last_became_active":"2024-08-31T06:04:25.879065+0000","last_became_peered":"2024-08-31T06:04:25.879065+0000","last_unstale":"2024-08-31T06:05:25.051368+0000","last_undegraded":"2024-08-31T06:05:25.051368+0000","last_fullsized":"2024-08-31T06:05:25.051368+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:53:43.848269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"78'6","reported_seq":111,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:42.078981+0000","last_change":"2024-08-31T06:05:25.058954+0000","last_active":"2024-08-31T06:05:42.078981+0000","last_peered":"2024-08-31T06:05:42.078981+0000","last_clean":"2024-08-31T06:05:42.078981+0000","last_became_active":"2024-08-31T06:05:25.051143+0000","last_became_peered":"2024-08-31T06:05:25.051143+0000","last_unstale":"2024-08-31T06:05:42.078981+0000","last_undegraded":"2024-08-31T06:05:42.078981+0000","last_fullsized":"2024-08-31T06:05:42.078981+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:38:23.533663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":61,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.044804+0000","last_change":"2024-08-31T06:04:25.896354+0000","last_active":"2024-08-31T06:05:25.044804+0000","last_peered":"2024-08-31T06:05:25.044804+0000","last_clean":"2024-08-31T06:05:25.044804+0000","last_became_active":"2024-08-31T06:04:25.896021+0000","last_became_peered":"2024-08-31T06:04:25.896021+0000","last_unstale":"2024-08-31T06:05:25.044804+0000","last_undegraded":"2024-08-31T06:05:25.044804+0000","last_fullsized":"2024-08-31T06:05:25.044804+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:08:40.213238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.047153+0000","last_change":"2024-08-31T06:04:22.844248+0000","last_active":"2024-08-31T06:05:25.047153+0000","last_peered":"2024-08-31T06:05:25.047153+0000","last_clean":"2024-08-31T06:05:25.047153+0000","last_became_active":"2024-08-31T06:04:22.843784+0000","last_became_peered":"2024-08-31T06:04:22.843784+0000","last_unstale":"2024-08-31T06:05:25.047153+0000","last_undegraded":"2024-08-31T06:05:25.047153+0000","last_fullsized":"2024-08-31T06:05:25.047153+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:56:41.422965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.046218+0000","last_change":"2024-08-31T06:04:22.847049+0000","last_active":"2024-08-31T06:05:25.046218+0000","last_peered":"2024-08-31T06:05:25.046218+0000","last_clean":"2024-08-31T06:05:25.046218+0000","last_became_active":"2024-08-31T06:04:22.846282+0000","last_became_peered":"2024-08-31T06:04:22.846282+0000","last_unstale":"2024-08-31T06:05:25.046218+0000","last_undegraded":"2024-08-31T06:05:25.046218+0000","last_fullsized":"2024-08-31T06:05:25.046218+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:35:21.030425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.107762+0000","last_change":"2024-08-31T06:04:25.886698+0000","last_active":"2024-08-31T06:05:25.107762+0000","last_peered":"2024-08-31T06:05:25.107762+0000","last_clean":"2024-08-31T06:05:25.107762+0000","last_became_active":"2024-08-31T06:04:25.886334+0000","last_became_peered":"2024-08-31T06:04:25.886334+0000","last_unstale":"2024-08-31T06:05:25.107762+0000","last_undegraded":"2024-08-31T06:05:25.107762+0000","last_fullsized":"2024-08-31T06:05:25.107762+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:24:43.821446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.044798+0000","last_change":"2024-08-31T06:04:22.845263+0000","last_active":"2024-08-31T06:05:25.044798+0000","last_peered":"2024-08-31T06:05:25.044798+0000","last_clean":"2024-08-31T06:05:25.044798+0000","last_became_active":"2024-08-31T06:04:22.844853+0000","last_became_peered":"2024-08-31T06:04:22.844853+0000","last_unstale":"2024-08-31T06:05:25.044798+0000","last_undegraded":"2024-08-31T06:05:25.044798+0000","last_fullsized":"2024-08-31T06:05:25.044798+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:55:12.561165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.043449+0000","last_change":"2024-08-31T06:04:25.894475+0000","last_active":"2024-08-31T06:05:25.043449+0000","last_peered":"2024-08-31T06:05:25.043449+0000","last_clean":"2024-08-31T06:05:25.043449+0000","last_became_active":"2024-08-31T06:04:25.894121+0000","last_became_peered":"2024-08-31T06:04:25.894121+0000","last_unstale":"2024-08-31T06:05:25.043449+0000","last_undegraded":"2024-08-31T06:05:25.043449+0000","last_fullsized":"2024-08-31T06:05:25.043449+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:40:06.570616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'3","reported_seq":93,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:43.079709+0000","last_change":"2024-08-31T06:04:22.834430+0000","last_active":"2024-08-31T06:05:43.079709+0000","last_peered":"2024-08-31T06:05:43.079709+0000","last_clean":"2024-08-31T06:05:43.079709+0000","last_became_active":"2024-08-31T06:04:22.834011+0000","last_became_peered":"2024-08-31T06:04:22.834011+0000","last_unstale":"2024-08-31T06:05:43.079709+0000","last_undegraded":"2024-08-31T06:05:43.079709+0000","last_fullsized":"2024-08-31T06:05:43.079709+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:51:16.734234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":64,"num_read_kb":64,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.049748+0000","last_change":"2024-08-31T06:04:25.877727+0000","last_active":"2024-08-31T06:05:25.049748+0000","last_peered":"2024-08-31T06:05:25.049748+0000","last_clean":"2024-08-31T06:05:25.049748+0000","last_became_active":"2024-08-31T06:04:25.877121+0000","last_became_peered":"2024-08-31T06:04:25.877121+0000","last_unstale":"2024-08-31T06:05:25.049748+0000","last_undegraded":"2024-08-31T06:05:25.049748+0000","last_fullsized":"2024-08-31T06:05:25.049748+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:41:12.247216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048823+0000","last_change":"2024-08-31T06:04:22.846482+0000","last_active":"2024-08-31T06:05:25.048823+0000","last_peered":"2024-08-31T06:05:25.048823+0000","last_clean":"2024-08-31T06:05:25.048823+0000","last_became_active":"2024-08-31T06:04:22.846057+0000","last_became_peered":"2024-08-31T06:04:22.846057+0000","last_unstale":"2024-08-31T06:05:25.048823+0000","last_undegraded":"2024-08-31T06:05:25.048823+0000","last_fullsized":"2024-08-31T06:05:25.048823+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:29:11.861665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.056422+0000","last_change":"2024-08-31T06:04:25.893777+0000","last_active":"2024-08-31T06:05:25.056422+0000","last_peered":"2024-08-31T06:05:25.056422+0000","last_clean":"2024-08-31T06:05:25.056422+0000","last_became_active":"2024-08-31T06:04:25.893014+0000","last_became_peered":"2024-08-31T06:04:25.893014+0000","last_unstale":"2024-08-31T06:05:25.056422+0000","last_undegraded":"2024-08-31T06:05:25.056422+0000","last_fullsized":"2024-08-31T06:05:25.056422+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:04:48.185842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.045472+0000","last_change":"2024-08-31T06:04:22.965938+0000","last_active":"2024-08-31T06:05:25.045472+0000","last_peered":"2024-08-31T06:05:25.045472+0000","last_clean":"2024-08-31T06:05:25.045472+0000","last_became_active":"2024-08-31T06:04:22.965356+0000","last_became_peered":"2024-08-31T06:04:22.965356+0000","last_unstale":"2024-08-31T06:05:25.045472+0000","last_undegraded":"2024-08-31T06:05:25.045472+0000","last_fullsized":"2024-08-31T06:05:25.045472+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:36:58.938462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.049791+0000","last_change":"2024-08-31T06:04:25.881307+0000","last_active":"2024-08-31T06:05:25.049791+0000","last_peered":"2024-08-31T06:05:25.049791+0000","last_clean":"2024-08-31T06:05:25.049791+0000","last_became_active":"2024-08-31T06:04:25.880418+0000","last_became_peered":"2024-08-31T06:04:25.880418+0000","last_unstale":"2024-08-31T06:05:25.049791+0000","last_undegraded":"2024-08-31T06:05:25.049791+0000","last_fullsized":"2024-08-31T06:05:25.049791+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:21:46.904371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"78'29","reported_seq":192,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:44.080561+0000","last_change":"2024-08-31T06:04:22.843073+0000","last_active":"2024-08-31T06:05:44.080561+0000","last_peered":"2024-08-31T06:05:44.080561+0000","last_clean":"2024-08-31T06:05:44.080561+0000","last_became_active":"2024-08-31T06:04:22.842645+0000","last_became_peered":"2024-08-31T06:04:22.842645+0000","last_unstale":"2024-08-31T06:05:44.080561+0000","last_undegraded":"2024-08-31T06:05:44.080561+0000","last_fullsized":"2024-08-31T06:05:44.080561+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:41:37.651562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59762,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":137,"num_read_kb":126,"num_write":30,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.053480+0000","last_change":"2024-08-31T06:04:25.879929+0000","last_active":"2024-08-31T06:05:25.053480+0000","last_peered":"2024-08-31T06:05:25.053480+0000","last_clean":"2024-08-31T06:05:25.053480+0000","last_became_active":"2024-08-31T06:04:25.878475+0000","last_became_peered":"2024-08-31T06:04:25.878475+0000","last_unstale":"2024-08-31T06:05:25.053480+0000","last_undegraded":"2024-08-31T06:05:25.053480+0000","last_fullsized":"2024-08-31T06:05:25.053480+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:10:32.801558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048188+0000","last_change":"2024-08-31T06:04:22.842878+0000","last_active":"2024-08-31T06:05:25.048188+0000","last_peered":"2024-08-31T06:05:25.048188+0000","last_clean":"2024-08-31T06:05:25.048188+0000","last_became_active":"2024-08-31T06:04:22.842306+0000","last_became_peered":"2024-08-31T06:04:22.842306+0000","last_unstale":"2024-08-31T06:05:25.048188+0000","last_undegraded":"2024-08-31T06:05:25.048188+0000","last_fullsized":"2024-08-31T06:05:25.048188+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:53:55.259238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.046274+0000","last_change":"2024-08-31T06:04:25.899357+0000","last_active":"2024-08-31T06:05:25.046274+0000","last_peered":"2024-08-31T06:05:25.046274+0000","last_clean":"2024-08-31T06:05:25.046274+0000","last_became_active":"2024-08-31T06:04:25.899000+0000","last_became_peered":"2024-08-31T06:04:25.899000+0000","last_unstale":"2024-08-31T06:05:25.046274+0000","last_undegraded":"2024-08-31T06:05:25.046274+0000","last_fullsized":"2024-08-31T06:05:25.046274+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:20:39.991071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.044693+0000","last_change":"2024-08-31T06:04:22.843063+0000","last_active":"2024-08-31T06:05:25.044693+0000","last_peered":"2024-08-31T06:05:25.044693+0000","last_clean":"2024-08-31T06:05:25.044693+0000","last_became_active":"2024-08-31T06:04:22.842654+0000","last_became_peered":"2024-08-31T06:04:22.842654+0000","last_unstale":"2024-08-31T06:05:25.044693+0000","last_undegraded":"2024-08-31T06:05:25.044693+0000","last_fullsized":"2024-08-31T06:05:25.044693+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:09:54.563116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048564+0000","last_change":"2024-08-31T06:04:25.885556+0000","last_active":"2024-08-31T06:05:25.048564+0000","last_peered":"2024-08-31T06:05:25.048564+0000","last_clean":"2024-08-31T06:05:25.048564+0000","last_became_active":"2024-08-31T06:04:25.885216+0000","last_became_peered":"2024-08-31T06:04:25.885216+0000","last_unstale":"2024-08-31T06:05:25.048564+0000","last_undegraded":"2024-08-31T06:05:25.048564+0000","last_fullsized":"2024-08-31T06:05:25.048564+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:48:49.816261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'4","reported_seq":97,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:42.609514+0000","last_change":"2024-08-31T06:04:22.844839+0000","last_active":"2024-08-31T06:05:42.609514+0000","last_peered":"2024-08-31T06:05:42.609514+0000","last_clean":"2024-08-31T06:05:42.609514+0000","last_became_active":"2024-08-31T06:04:22.844425+0000","last_became_peered":"2024-08-31T06:04:22.844425+0000","last_unstale":"2024-08-31T06:05:42.609514+0000","last_undegraded":"2024-08-31T06:05:42.609514+0000","last_fullsized":"2024-08-31T06:05:42.609514+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:10:57.292173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048224+0000","last_change":"2024-08-31T06:04:25.878195+0000","last_active":"2024-08-31T06:05:25.048224+0000","last_peered":"2024-08-31T06:05:25.048224+0000","last_clean":"2024-08-31T06:05:25.048224+0000","last_became_active":"2024-08-31T06:04:25.877823+0000","last_became_peered":"2024-08-31T06:04:25.877823+0000","last_unstale":"2024-08-31T06:05:25.048224+0000","last_undegraded":"2024-08-31T06:05:25.048224+0000","last_fullsized":"2024-08-31T06:05:25.048224+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:54:34.989583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":67,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.071462+0000","last_change":"2024-08-31T06:05:25.071391+0000","last_active":"2024-08-31T06:05:25.071462+0000","last_peered":"2024-08-31T06:05:25.071462+0000","last_clean":"2024-08-31T06:05:25.071462+0000","last_became_active":"2024-08-31T06:05:25.057546+0000","last_became_peered":"2024-08-31T06:05:25.057546+0000","last_unstale":"2024-08-31T06:05:25.071462+0000","last_undegraded":"2024-08-31T06:05:25.071462+0000","last_fullsized":"2024-08-31T06:05:25.071462+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:42:35.527395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.044680+0000","last_change":"2024-08-31T06:04:25.887580+0000","last_active":"2024-08-31T06:05:25.044680+0000","last_peered":"2024-08-31T06:05:25.044680+0000","last_clean":"2024-08-31T06:05:25.044680+0000","last_became_active":"2024-08-31T06:04:25.887158+0000","last_became_peered":"2024-08-31T06:04:25.887158+0000","last_unstale":"2024-08-31T06:05:25.044680+0000","last_undegraded":"2024-08-31T06:05:25.044680+0000","last_fullsized":"2024-08-31T06:05:25.044680+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:27:38.115742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.064443+0000","last_change":"2024-08-31T06:05:25.063859+0000","last_active":"2024-08-31T06:05:25.064443+0000","last_peered":"2024-08-31T06:05:25.064443+0000","last_clean":"2024-08-31T06:05:25.064443+0000","last_became_active":"2024-08-31T06:05:25.053596+0000","last_became_peered":"2024-08-31T06:05:25.053596+0000","last_unstale":"2024-08-31T06:05:25.064443+0000","last_undegraded":"2024-08-31T06:05:25.064443+0000","last_fullsized":"2024-08-31T06:05:25.064443+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:19:18.461630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048222+0000","last_change":"2024-08-31T06:04:25.880845+0000","last_active":"2024-08-31T06:05:25.048222+0000","last_peered":"2024-08-31T06:05:25.048222+0000","last_clean":"2024-08-31T06:05:25.048222+0000","last_became_active":"2024-08-31T06:04:25.880423+0000","last_became_peered":"2024-08-31T06:04:25.880423+0000","last_unstale":"2024-08-31T06:05:25.048222+0000","last_undegraded":"2024-08-31T06:05:25.048222+0000","last_fullsized":"2024-08-31T06:05:25.048222+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:16:56.725116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.046524+0000","last_change":"2024-08-31T06:04:22.843393+0000","last_active":"2024-08-31T06:05:25.046524+0000","last_peered":"2024-08-31T06:05:25.046524+0000","last_clean":"2024-08-31T06:05:25.046524+0000","last_became_active":"2024-08-31T06:04:22.842770+0000","last_became_peered":"2024-08-31T06:04:22.842770+0000","last_unstale":"2024-08-31T06:05:25.046524+0000","last_undegraded":"2024-08-31T06:05:25.046524+0000","last_fullsized":"2024-08-31T06:05:25.046524+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:49:16.159167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048937+0000","last_change":"2024-08-31T06:04:25.887829+0000","last_active":"2024-08-31T06:05:25.048937+0000","last_peered":"2024-08-31T06:05:25.048937+0000","last_clean":"2024-08-31T06:05:25.048937+0000","last_became_active":"2024-08-31T06:04:25.886567+0000","last_became_peered":"2024-08-31T06:04:25.886567+0000","last_unstale":"2024-08-31T06:05:25.048937+0000","last_undegraded":"2024-08-31T06:05:25.048937+0000","last_fullsized":"2024-08-31T06:05:25.048937+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:11:39.054966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.045559+0000","last_change":"2024-08-31T06:04:22.844218+0000","last_active":"2024-08-31T06:05:25.045559+0000","last_peered":"2024-08-31T06:05:25.045559+0000","last_clean":"2024-08-31T06:05:25.045559+0000","last_became_active":"2024-08-31T06:04:22.843786+0000","last_became_peered":"2024-08-31T06:04:22.843786+0000","last_unstale":"2024-08-31T06:05:25.045559+0000","last_undegraded":"2024-08-31T06:05:25.045559+0000","last_fullsized":"2024-08-31T06:05:25.045559+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:46:02.534190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'1","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048764+0000","last_change":"2024-08-31T06:04:25.888946+0000","last_active":"2024-08-31T06:05:25.048764+0000","last_peered":"2024-08-31T06:05:25.048764+0000","last_clean":"2024-08-31T06:05:25.048764+0000","last_became_active":"2024-08-31T06:04:25.888564+0000","last_became_peered":"2024-08-31T06:04:25.888564+0000","last_unstale":"2024-08-31T06:05:25.048764+0000","last_undegraded":"2024-08-31T06:05:25.048764+0000","last_fullsized":"2024-08-31T06:05:25.048764+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:26:19.051971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.049144+0000","last_change":"2024-08-31T06:04:22.964483+0000","last_active":"2024-08-31T06:05:25.049144+0000","last_peered":"2024-08-31T06:05:25.049144+0000","last_clean":"2024-08-31T06:05:25.049144+0000","last_became_active":"2024-08-31T06:04:22.963874+0000","last_became_peered":"2024-08-31T06:04:22.963874+0000","last_unstale":"2024-08-31T06:05:25.049144+0000","last_undegraded":"2024-08-31T06:05:25.049144+0000","last_fullsized":"2024-08-31T06:05:25.049144+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:30:00.159040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048856+0000","last_change":"2024-08-31T06:04:25.880941+0000","last_active":"2024-08-31T06:05:25.048856+0000","last_peered":"2024-08-31T06:05:25.048856+0000","last_clean":"2024-08-31T06:05:25.048856+0000","last_became_active":"2024-08-31T06:04:25.880454+0000","last_became_peered":"2024-08-31T06:04:25.880454+0000","last_unstale":"2024-08-31T06:05:25.048856+0000","last_undegraded":"2024-08-31T06:05:25.048856+0000","last_fullsized":"2024-08-31T06:05:25.048856+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:49:57.023988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":98,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048527+0000","last_change":"2024-08-31T06:03:05.921629+0000","last_active":"2024-08-31T06:05:25.048527+0000","last_peered":"2024-08-31T06:05:25.048527+0000","last_clean":"2024-08-31T06:05:25.048527+0000","last_became_active":"2024-08-31T06:03:05.915844+0000","last_became_peered":"2024-08-31T06:03:05.915844+0000","last_unstale":"2024-08-31T06:05:25.048527+0000","last_undegraded":"2024-08-31T06:05:25.048527+0000","last_fullsized":"2024-08-31T06:05:25.048527+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T05:59:24.472429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T05:59:24.472429+0000","last_clean_scrub_stamp":"2024-08-31T05:59:24.472429+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:56:41.073422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.044659+0000","last_change":"2024-08-31T06:04:22.846325+0000","last_active":"2024-08-31T06:05:25.044659+0000","last_peered":"2024-08-31T06:05:25.044659+0000","last_clean":"2024-08-31T06:05:25.044659+0000","last_became_active":"2024-08-31T06:04:22.845772+0000","last_became_peered":"2024-08-31T06:04:22.845772+0000","last_unstale":"2024-08-31T06:05:25.044659+0000","last_undegraded":"2024-08-31T06:05:25.044659+0000","last_fullsized":"2024-08-31T06:05:25.044659+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:54:10.336786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.107981+0000","last_change":"2024-08-31T06:04:25.891437+0000","last_active":"2024-08-31T06:05:25.107981+0000","last_peered":"2024-08-31T06:05:25.107981+0000","last_clean":"2024-08-31T06:05:25.107981+0000","last_became_active":"2024-08-31T06:04:25.891053+0000","last_became_peered":"2024-08-31T06:04:25.891053+0000","last_unstale":"2024-08-31T06:05:25.107981+0000","last_undegraded":"2024-08-31T06:05:25.107981+0000","last_fullsized":"2024-08-31T06:05:25.107981+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:22:37.119483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048351+0000","last_change":"2024-08-31T06:04:22.842885+0000","last_active":"2024-08-31T06:05:25.048351+0000","last_peered":"2024-08-31T06:05:25.048351+0000","last_clean":"2024-08-31T06:05:25.048351+0000","last_became_active":"2024-08-31T06:04:22.842473+0000","last_became_peered":"2024-08-31T06:04:22.842473+0000","last_unstale":"2024-08-31T06:05:25.048351+0000","last_undegraded":"2024-08-31T06:05:25.048351+0000","last_fullsized":"2024-08-31T06:05:25.048351+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:33:36.516659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.046184+0000","last_change":"2024-08-31T06:04:25.889213+0000","last_active":"2024-08-31T06:05:25.046184+0000","last_peered":"2024-08-31T06:05:25.046184+0000","last_clean":"2024-08-31T06:05:25.046184+0000","last_became_active":"2024-08-31T06:04:25.888860+0000","last_became_peered":"2024-08-31T06:04:25.888860+0000","last_unstale":"2024-08-31T06:05:25.046184+0000","last_undegraded":"2024-08-31T06:05:25.046184+0000","last_fullsized":"2024-08-31T06:05:25.046184+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:44:18.831229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"78'4","reported_seq":95,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:43.610918+0000","last_change":"2024-08-31T06:04:22.966091+0000","last_active":"2024-08-31T06:05:43.610918+0000","last_peered":"2024-08-31T06:05:43.610918+0000","last_clean":"2024-08-31T06:05:43.610918+0000","last_became_active":"2024-08-31T06:04:22.965727+0000","last_became_peered":"2024-08-31T06:04:22.965727+0000","last_unstale":"2024-08-31T06:05:43.610918+0000","last_undegraded":"2024-08-31T06:05:43.610918+0000","last_fullsized":"2024-08-31T06:05:43.610918+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:42:43.643623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":65,"num_read_kb":65,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.044657+0000","last_change":"2024-08-31T06:04:25.880039+0000","last_active":"2024-08-31T06:05:25.044657+0000","last_peered":"2024-08-31T06:05:25.044657+0000","last_clean":"2024-08-31T06:05:25.044657+0000","last_became_active":"2024-08-31T06:04:25.879669+0000","last_became_peered":"2024-08-31T06:04:25.879669+0000","last_unstale":"2024-08-31T06:05:25.044657+0000","last_undegraded":"2024-08-31T06:05:25.044657+0000","last_fullsized":"2024-08-31T06:05:25.044657+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:02:09.244281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048633+0000","last_change":"2024-08-31T06:04:22.844572+0000","last_active":"2024-08-31T06:05:25.048633+0000","last_peered":"2024-08-31T06:05:25.048633+0000","last_clean":"2024-08-31T06:05:25.048633+0000","last_became_active":"2024-08-31T06:04:22.844227+0000","last_became_peered":"2024-08-31T06:04:22.844227+0000","last_unstale":"2024-08-31T06:05:25.048633+0000","last_undegraded":"2024-08-31T06:05:25.048633+0000","last_fullsized":"2024-08-31T06:05:25.048633+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:04:51.966901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.047419+0000","last_change":"2024-08-31T06:04:25.884442+0000","last_active":"2024-08-31T06:05:25.047419+0000","last_peered":"2024-08-31T06:05:25.047419+0000","last_clean":"2024-08-31T06:05:25.047419+0000","last_became_active":"2024-08-31T06:04:25.884016+0000","last_became_peered":"2024-08-31T06:04:25.884016+0000","last_unstale":"2024-08-31T06:05:25.047419+0000","last_undegraded":"2024-08-31T06:05:25.047419+0000","last_fullsized":"2024-08-31T06:05:25.047419+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:36:10.640262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.043778+0000","last_change":"2024-08-31T06:04:22.967845+0000","last_active":"2024-08-31T06:05:25.043778+0000","last_peered":"2024-08-31T06:05:25.043778+0000","last_clean":"2024-08-31T06:05:25.043778+0000","last_became_active":"2024-08-31T06:04:22.967421+0000","last_became_peered":"2024-08-31T06:04:22.967421+0000","last_unstale":"2024-08-31T06:05:25.043778+0000","last_undegraded":"2024-08-31T06:05:25.043778+0000","last_fullsized":"2024-08-31T06:05:25.043778+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:17:46.942551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'1","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.047292+0000","last_change":"2024-08-31T06:04:25.897087+0000","last_active":"2024-08-31T06:05:25.047292+0000","last_peered":"2024-08-31T06:05:25.047292+0000","last_clean":"2024-08-31T06:05:25.047292+0000","last_became_active":"2024-08-31T06:04:25.896725+0000","last_became_peered":"2024-08-31T06:04:25.896725+0000","last_unstale":"2024-08-31T06:05:25.047292+0000","last_undegraded":"2024-08-31T06:05:25.047292+0000","last_fullsized":"2024-08-31T06:05:25.047292+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:31:28.279489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.053617+0000","last_change":"2024-08-31T06:04:22.965299+0000","last_active":"2024-08-31T06:05:25.053617+0000","last_peered":"2024-08-31T06:05:25.053617+0000","last_clean":"2024-08-31T06:05:25.053617+0000","last_became_active":"2024-08-31T06:04:22.964849+0000","last_became_peered":"2024-08-31T06:04:22.964849+0000","last_unstale":"2024-08-31T06:05:25.053617+0000","last_undegraded":"2024-08-31T06:05:25.053617+0000","last_fullsized":"2024-08-31T06:05:25.053617+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:36:56.431678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.046713+0000","last_change":"2024-08-31T06:04:25.885115+0000","last_active":"2024-08-31T06:05:25.046713+0000","last_peered":"2024-08-31T06:05:25.046713+0000","last_clean":"2024-08-31T06:05:25.046713+0000","last_became_active":"2024-08-31T06:04:25.884722+0000","last_became_peered":"2024-08-31T06:04:25.884722+0000","last_unstale":"2024-08-31T06:05:25.046713+0000","last_undegraded":"2024-08-31T06:05:25.046713+0000","last_fullsized":"2024-08-31T06:05:25.046713+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:20:43.913785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'11","reported_seq":38,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.043844+0000","last_change":"2024-08-31T06:04:22.845119+0000","last_active":"2024-08-31T06:05:25.043844+0000","last_peered":"2024-08-31T06:05:25.043844+0000","last_clean":"2024-08-31T06:05:25.043844+0000","last_became_active":"2024-08-31T06:04:22.843814+0000","last_became_peered":"2024-08-31T06:04:22.843814+0000","last_unstale":"2024-08-31T06:05:25.043844+0000","last_undegraded":"2024-08-31T06:05:25.043844+0000","last_fullsized":"2024-08-31T06:05:25.043844+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:06:40.684887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.108183+0000","last_change":"2024-08-31T06:04:25.892838+0000","last_active":"2024-08-31T06:05:25.108183+0000","last_peered":"2024-08-31T06:05:25.108183+0000","last_clean":"2024-08-31T06:05:25.108183+0000","last_became_active":"2024-08-31T06:04:25.891955+0000","last_became_peered":"2024-08-31T06:04:25.891955+0000","last_unstale":"2024-08-31T06:05:25.108183+0000","last_undegraded":"2024-08-31T06:05:25.108183+0000","last_fullsized":"2024-08-31T06:05:25.108183+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:41:35.085724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.044900+0000","last_change":"2024-08-31T06:04:22.964183+0000","last_active":"2024-08-31T06:05:25.044900+0000","last_peered":"2024-08-31T06:05:25.044900+0000","last_clean":"2024-08-31T06:05:25.044900+0000","last_became_active":"2024-08-31T06:04:22.963688+0000","last_became_peered":"2024-08-31T06:04:22.963688+0000","last_unstale":"2024-08-31T06:05:25.044900+0000","last_undegraded":"2024-08-31T06:05:25.044900+0000","last_fullsized":"2024-08-31T06:05:25.044900+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:04:47.833811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.056734+0000","last_change":"2024-08-31T06:04:25.892266+0000","last_active":"2024-08-31T06:05:25.056734+0000","last_peered":"2024-08-31T06:05:25.056734+0000","last_clean":"2024-08-31T06:05:25.056734+0000","last_became_active":"2024-08-31T06:04:25.889014+0000","last_became_peered":"2024-08-31T06:04:25.889014+0000","last_unstale":"2024-08-31T06:05:25.056734+0000","last_undegraded":"2024-08-31T06:05:25.056734+0000","last_fullsized":"2024-08-31T06:05:25.056734+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:27:47.608470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.049350+0000","last_change":"2024-08-31T06:04:22.964677+0000","last_active":"2024-08-31T06:05:25.049350+0000","last_peered":"2024-08-31T06:05:25.049350+0000","last_clean":"2024-08-31T06:05:25.049350+0000","last_became_active":"2024-08-31T06:04:22.964237+0000","last_became_peered":"2024-08-31T06:04:22.964237+0000","last_unstale":"2024-08-31T06:05:25.049350+0000","last_undegraded":"2024-08-31T06:05:25.049350+0000","last_fullsized":"2024-08-31T06:05:25.049350+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:00:54.099592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.051198+0000","last_change":"2024-08-31T06:04:25.881699+0000","last_active":"2024-08-31T06:05:25.051198+0000","last_peered":"2024-08-31T06:05:25.051198+0000","last_clean":"2024-08-31T06:05:25.051198+0000","last_became_active":"2024-08-31T06:04:25.879304+0000","last_became_peered":"2024-08-31T06:04:25.879304+0000","last_unstale":"2024-08-31T06:05:25.051198+0000","last_undegraded":"2024-08-31T06:05:25.051198+0000","last_fullsized":"2024-08-31T06:05:25.051198+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:56:42.992034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.108268+0000","last_change":"2024-08-31T06:04:22.847934+0000","last_active":"2024-08-31T06:05:25.108268+0000","last_peered":"2024-08-31T06:05:25.108268+0000","last_clean":"2024-08-31T06:05:25.108268+0000","last_became_active":"2024-08-31T06:04:22.847518+0000","last_became_peered":"2024-08-31T06:04:22.847518+0000","last_unstale":"2024-08-31T06:05:25.108268+0000","last_undegraded":"2024-08-31T06:05:25.108268+0000","last_fullsized":"2024-08-31T06:05:25.108268+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:26:21.476014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.047747+0000","last_change":"2024-08-31T06:04:25.884653+0000","last_active":"2024-08-31T06:05:25.047747+0000","last_peered":"2024-08-31T06:05:25.047747+0000","last_clean":"2024-08-31T06:05:25.047747+0000","last_became_active":"2024-08-31T06:04:25.884244+0000","last_became_peered":"2024-08-31T06:04:25.884244+0000","last_unstale":"2024-08-31T06:05:25.047747+0000","last_undegraded":"2024-08-31T06:05:25.047747+0000","last_fullsized":"2024-08-31T06:05:25.047747+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:13:00.814071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048578+0000","last_change":"2024-08-31T06:04:22.845924+0000","last_active":"2024-08-31T06:05:25.048578+0000","last_peered":"2024-08-31T06:05:25.048578+0000","last_clean":"2024-08-31T06:05:25.048578+0000","last_became_active":"2024-08-31T06:04:22.845445+0000","last_became_peered":"2024-08-31T06:04:22.845445+0000","last_unstale":"2024-08-31T06:05:25.048578+0000","last_undegraded":"2024-08-31T06:05:25.048578+0000","last_fullsized":"2024-08-31T06:05:25.048578+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:25:21.052136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.107948+0000","last_change":"2024-08-31T06:04:25.890069+0000","last_active":"2024-08-31T06:05:25.107948+0000","last_peered":"2024-08-31T06:05:25.107948+0000","last_clean":"2024-08-31T06:05:25.107948+0000","last_became_active":"2024-08-31T06:04:25.889709+0000","last_became_peered":"2024-08-31T06:04:25.889709+0000","last_unstale":"2024-08-31T06:05:25.107948+0000","last_undegraded":"2024-08-31T06:05:25.107948+0000","last_fullsized":"2024-08-31T06:05:25.107948+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:13:51.360023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.051547+0000","last_change":"2024-08-31T06:04:22.847845+0000","last_active":"2024-08-31T06:05:25.051547+0000","last_peered":"2024-08-31T06:05:25.051547+0000","last_clean":"2024-08-31T06:05:25.051547+0000","last_became_active":"2024-08-31T06:04:22.847001+0000","last_became_peered":"2024-08-31T06:04:22.847001+0000","last_unstale":"2024-08-31T06:05:25.051547+0000","last_undegraded":"2024-08-31T06:05:25.051547+0000","last_fullsized":"2024-08-31T06:05:25.051547+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:24:34.209076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048518+0000","last_change":"2024-08-31T06:04:25.886759+0000","last_active":"2024-08-31T06:05:25.048518+0000","last_peered":"2024-08-31T06:05:25.048518+0000","last_clean":"2024-08-31T06:05:25.048518+0000","last_became_active":"2024-08-31T06:04:25.886176+0000","last_became_peered":"2024-08-31T06:04:25.886176+0000","last_unstale":"2024-08-31T06:05:25.048518+0000","last_undegraded":"2024-08-31T06:05:25.048518+0000","last_fullsized":"2024-08-31T06:05:25.048518+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:11:12.637451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.054559+0000","last_change":"2024-08-31T06:05:25.054468+0000","last_active":"2024-08-31T06:05:25.054559+0000","last_peered":"2024-08-31T06:05:25.054559+0000","last_clean":"2024-08-31T06:05:25.054559+0000","last_became_active":"2024-08-31T06:05:25.053657+0000","last_became_peered":"2024-08-31T06:05:25.053657+0000","last_unstale":"2024-08-31T06:05:25.054559+0000","last_undegraded":"2024-08-31T06:05:25.054559+0000","last_fullsized":"2024-08-31T06:05:25.054559+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:43:44.887637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.059757+0000","last_change":"2024-08-31T06:05:25.059219+0000","last_active":"2024-08-31T06:05:25.059757+0000","last_peered":"2024-08-31T06:05:25.059757+0000","last_clean":"2024-08-31T06:05:25.059757+0000","last_became_active":"2024-08-31T06:05:25.053136+0000","last_became_peered":"2024-08-31T06:05:25.053136+0000","last_unstale":"2024-08-31T06:05:25.059757+0000","last_undegraded":"2024-08-31T06:05:25.059757+0000","last_fullsized":"2024-08-31T06:05:25.059757+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:29:59.137465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.049696+0000","last_change":"2024-08-31T06:04:25.887524+0000","last_active":"2024-08-31T06:05:25.049696+0000","last_peered":"2024-08-31T06:05:25.049696+0000","last_clean":"2024-08-31T06:05:25.049696+0000","last_became_active":"2024-08-31T06:04:25.887176+0000","last_became_peered":"2024-08-31T06:04:25.887176+0000","last_unstale":"2024-08-31T06:05:25.049696+0000","last_undegraded":"2024-08-31T06:05:25.049696+0000","last_fullsized":"2024-08-31T06:05:25.049696+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:08:05.476804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.053524+0000","last_change":"2024-08-31T06:04:22.847747+0000","last_active":"2024-08-31T06:05:25.053524+0000","last_peered":"2024-08-31T06:05:25.053524+0000","last_clean":"2024-08-31T06:05:25.053524+0000","last_became_active":"2024-08-31T06:04:22.847332+0000","last_became_peered":"2024-08-31T06:04:22.847332+0000","last_unstale":"2024-08-31T06:05:25.053524+0000","last_undegraded":"2024-08-31T06:05:25.053524+0000","last_fullsized":"2024-08-31T06:05:25.053524+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:41:35.910991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.056293+0000","last_change":"2024-08-31T06:04:25.883110+0000","last_active":"2024-08-31T06:05:25.056293+0000","last_peered":"2024-08-31T06:05:25.056293+0000","last_clean":"2024-08-31T06:05:25.056293+0000","last_became_active":"2024-08-31T06:04:25.880487+0000","last_became_peered":"2024-08-31T06:04:25.880487+0000","last_unstale":"2024-08-31T06:05:25.056293+0000","last_undegraded":"2024-08-31T06:05:25.056293+0000","last_fullsized":"2024-08-31T06:05:25.056293+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:22:13.656078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.108180+0000","last_change":"2024-08-31T06:04:22.964758+0000","last_active":"2024-08-31T06:05:25.108180+0000","last_peered":"2024-08-31T06:05:25.108180+0000","last_clean":"2024-08-31T06:05:25.108180+0000","last_became_active":"2024-08-31T06:04:22.964391+0000","last_became_peered":"2024-08-31T06:04:22.964391+0000","last_unstale":"2024-08-31T06:05:25.108180+0000","last_undegraded":"2024-08-31T06:05:25.108180+0000","last_fullsized":"2024-08-31T06:05:25.108180+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:28:59.190657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.108031+0000","last_change":"2024-08-31T06:04:25.893738+0000","last_active":"2024-08-31T06:05:25.108031+0000","last_peered":"2024-08-31T06:05:25.108031+0000","last_clean":"2024-08-31T06:05:25.108031+0000","last_became_active":"2024-08-31T06:04:25.893308+0000","last_became_peered":"2024-08-31T06:04:25.893308+0000","last_unstale":"2024-08-31T06:05:25.108031+0000","last_undegraded":"2024-08-31T06:05:25.108031+0000","last_fullsized":"2024-08-31T06:05:25.108031+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:26:48.451894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.043929+0000","last_change":"2024-08-31T06:04:22.843346+0000","last_active":"2024-08-31T06:05:25.043929+0000","last_peered":"2024-08-31T06:05:25.043929+0000","last_clean":"2024-08-31T06:05:25.043929+0000","last_became_active":"2024-08-31T06:04:22.842249+0000","last_became_peered":"2024-08-31T06:04:22.842249+0000","last_unstale":"2024-08-31T06:05:25.043929+0000","last_undegraded":"2024-08-31T06:05:25.043929+0000","last_fullsized":"2024-08-31T06:05:25.043929+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:40:57.618940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64154,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":407,"num_read_kb":274,"num_write":165,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1147,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":192462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":249373,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710750,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":872448,"data_stored":436463,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874274,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1019904,"data_stored":1088075,"data_compressed":42124,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037801,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":884736,"data_stored":436677,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234031,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":436575,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397557,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":933888,"data_stored":1028918,"data_compressed":42124,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593786,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":872448,"data_stored":436066,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790014,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":946176,"data_stored":1028211,"data_compressed":42124,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953541,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":884736,"data_stored":437907,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149768,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":888832,"data_stored":436377,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":942080,"data_stored":496237,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542228,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36268,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708884,"statfs":{"total":95995035648,"available":95957897216,"internally_reserved":0,"allocated":942080,"data_stored":495733,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705754,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":892928,"data_stored":437337,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34699,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34699,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34699,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":59874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":60378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-31T06:05:48.079 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-08-31T06:05:48.080 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-08-31T06:05:48.080 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-08-31T06:05:48.080 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph health --format=json 2024-08-31T06:05:48.497 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:05:48.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:48 smithi155 ceph-mon[21742]: pgmap v320: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-31T06:05:48.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:48 smithi155 ceph-mon[21742]: from='client.25037 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T06:05:48.636 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:48 smithi016 ceph-mon[17194]: pgmap v320: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-31T06:05:48.636 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:48 smithi016 ceph-mon[17194]: from='client.25037 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T06:05:48.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:48 smithi184 ceph-mon[22331]: pgmap v320: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-31T06:05:48.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:48 smithi184 ceph-mon[22331]: from='client.25037 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T06:05:49.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.148+0000 7fa2ad400640 1 -- 172.21.15.16:0/800323870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a81099e0 msgr2=0x7fa2a8110210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:49.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.148+0000 7fa2ad400640 1 --2- 172.21.15.16:0/800323870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a81099e0 0x7fa2a8110210 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009ca0 tx=0x7fa2a002a8c0 comp rx=0 tx=0).stop 2024-08-31T06:05:49.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.149+0000 7fa2ad400640 1 -- 172.21.15.16:0/800323870 shutdown_connections 2024-08-31T06:05:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.149+0000 7fa2ad400640 1 --2- 172.21.15.16:0/800323870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a81099e0 0x7fa2a8110210 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.149+0000 7fa2ad400640 1 --2- 172.21.15.16:0/800323870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2a8109090 0x7fa2a81094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.149+0000 7fa2ad400640 1 --2- 172.21.15.16:0/800323870 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2a8103750 0x7fa2a8103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.149+0000 7fa2ad400640 1 -- 172.21.15.16:0/800323870 >> 172.21.15.16:0/800323870 conn(0x7fa2a80ff1e0 msgr2=0x7fa2a8101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.149+0000 7fa2ad400640 1 -- 172.21.15.16:0/800323870 shutdown_connections 2024-08-31T06:05:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.149+0000 7fa2ad400640 1 -- 172.21.15.16:0/800323870 wait complete. 2024-08-31T06:05:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.150+0000 7fa2ad400640 1 Processor -- start 2024-08-31T06:05:49.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.150+0000 7fa2ad400640 1 -- start start 2024-08-31T06:05:49.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.151+0000 7fa2ad400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2a8103750 0x7fa2a80754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:49.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.151+0000 7fa2ad400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a8109090 0x7fa2a80759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:49.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.151+0000 7fa2ad400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2a81099e0 0x7fa2a8075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:49.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.151+0000 7fa2ad400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a8113700 con 0x7fa2a8109090 2024-08-31T06:05:49.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.151+0000 7fa2ad400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a8113580 con 0x7fa2a81099e0 2024-08-31T06:05:49.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.151+0000 7fa2ad400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a8113880 con 0x7fa2a8103750 2024-08-31T06:05:49.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.151+0000 7fa2a7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a8109090 0x7fa2a80759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:49.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.151+0000 7fa2a7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a8109090 0x7fa2a80759f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39052/0 (socket says 172.21.15.16:39052) 2024-08-31T06:05:49.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.151+0000 7fa2a7400640 1 -- 172.21.15.16:0/157306146 learned_addr learned my addr 172.21.15.16:0/157306146 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:05:49.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.151+0000 7fa2a7e00640 1 --2- 172.21.15.16:0/157306146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2a8103750 0x7fa2a80754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:49.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.151+0000 7fa2aca00640 1 --2- 172.21.15.16:0/157306146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2a81099e0 0x7fa2a8075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:49.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.152+0000 7fa2a7400640 1 -- 172.21.15.16:0/157306146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2a8103750 msgr2=0x7fa2a80754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:49.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.152+0000 7fa2a7400640 1 --2- 172.21.15.16:0/157306146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2a8103750 0x7fa2a80754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:49.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.152+0000 7fa2a7400640 1 -- 172.21.15.16:0/157306146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2a81099e0 msgr2=0x7fa2a8075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:49.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.152+0000 7fa2a7400640 1 --2- 172.21.15.16:0/157306146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2a81099e0 0x7fa2a8075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:49.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.152+0000 7fa2a7400640 1 -- 172.21.15.16:0/157306146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a81a83b0 con 0x7fa2a8109090 2024-08-31T06:05:49.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.152+0000 7fa2a7400640 1 --2- 172.21.15.16:0/157306146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a8109090 0x7fa2a80759f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa29400e9f0 tx=0x7fa29400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:49.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.152+0000 7fa2a4c00640 1 -- 172.21.15.16:0/157306146 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29400ccc0 con 0x7fa2a8109090 2024-08-31T06:05:49.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.152+0000 7fa2a4c00640 1 -- 172.21.15.16:0/157306146 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2940108e0 con 0x7fa2a8109090 2024-08-31T06:05:49.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.152+0000 7fa2a4c00640 1 -- 172.21.15.16:0/157306146 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa294018860 con 0x7fa2a8109090 2024-08-31T06:05:49.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.152+0000 7fa2ad400640 1 -- 172.21.15.16:0/157306146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a81a86a0 con 0x7fa2a8109090 2024-08-31T06:05:49.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.152+0000 7fa2ad400640 1 -- 172.21.15.16:0/157306146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a81a8b90 con 0x7fa2a8109090 2024-08-31T06:05:49.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.154+0000 7fa2ad400640 1 -- 172.21.15.16:0/157306146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2a8103b60 con 0x7fa2a8109090 2024-08-31T06:05:49.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.157+0000 7fa2a4c00640 1 -- 172.21.15.16:0/157306146 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7fa294010430 con 0x7fa2a8109090 2024-08-31T06:05:49.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.158+0000 7fa2a4c00640 1 --2- 172.21.15.16:0/157306146 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa28c0b4c10 0x7fa28c0b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:49.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.158+0000 7fa2a4c00640 1 -- 172.21.15.16:0/157306146 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7fa2940e5b40 con 0x7fa2a8109090 2024-08-31T06:05:49.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.158+0000 7fa2a7e00640 1 --2- 172.21.15.16:0/157306146 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa28c0b4c10 0x7fa28c0b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:49.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.159+0000 7fa2a7e00640 1 --2- 172.21.15.16:0/157306146 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa28c0b4c10 0x7fa28c0b7120 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8076f00 tx=0x7fa298005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:49.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.160+0000 7fa2a4c00640 1 -- 172.21.15.16:0/157306146 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fa2940ac3a0 con 0x7fa2a8109090 2024-08-31T06:05:49.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.395+0000 7fa2ad400640 1 -- 172.21.15.16:0/157306146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fa2a80767b0 con 0x7fa2a8109090 2024-08-31T06:05:49.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.396+0000 7fa2a4c00640 1 -- 172.21.15.16:0/157306146 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fa2a80767b0 con 0x7fa2a8109090 2024-08-31T06:05:49.398 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:05:49.398 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-08-31T06:05:49.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.399+0000 7fa2ad400640 1 -- 172.21.15.16:0/157306146 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa28c0b4c10 msgr2=0x7fa28c0b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:49.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.399+0000 7fa2ad400640 1 --2- 172.21.15.16:0/157306146 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa28c0b4c10 0x7fa28c0b7120 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8076f00 tx=0x7fa298005f90 comp rx=0 tx=0).stop 2024-08-31T06:05:49.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.399+0000 7fa2ad400640 1 -- 172.21.15.16:0/157306146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a8109090 msgr2=0x7fa2a80759f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:49.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.399+0000 7fa2ad400640 1 --2- 172.21.15.16:0/157306146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a8109090 0x7fa2a80759f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa29400e9f0 tx=0x7fa29400eeb0 comp rx=0 tx=0).stop 2024-08-31T06:05:49.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.399+0000 7fa2ad400640 1 -- 172.21.15.16:0/157306146 shutdown_connections 2024-08-31T06:05:49.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.399+0000 7fa2ad400640 1 --2- 172.21.15.16:0/157306146 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa28c0b4c10 0x7fa28c0b7120 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:49.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.399+0000 7fa2ad400640 1 --2- 172.21.15.16:0/157306146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2a81099e0 0x7fa2a8075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:49.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.399+0000 7fa2ad400640 1 --2- 172.21.15.16:0/157306146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a8109090 0x7fa2a80759f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:49.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.399+0000 7fa2ad400640 1 --2- 172.21.15.16:0/157306146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2a8103750 0x7fa2a80754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:49.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.399+0000 7fa2ad400640 1 -- 172.21.15.16:0/157306146 >> 172.21.15.16:0/157306146 conn(0x7fa2a80ff1e0 msgr2=0x7fa2a8106630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:49.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.399+0000 7fa2ad400640 1 -- 172.21.15.16:0/157306146 shutdown_connections 2024-08-31T06:05:49.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:49.399+0000 7fa2ad400640 1 -- 172.21.15.16:0/157306146 wait complete. 2024-08-31T06:05:50.193 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-08-31T06:05:50.194 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:05:50.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:50 smithi016 ceph-mon[17194]: pgmap v321: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-31T06:05:50.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:50 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/157306146' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-08-31T06:05:50.573 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:05:50.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:50 smithi155 ceph-mon[21742]: pgmap v321: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-31T06:05:50.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:50 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/157306146' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-08-31T06:05:50.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:50 smithi184 ceph-mon[22331]: pgmap v321: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-31T06:05:50.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:50 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/157306146' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-08-31T06:05:51.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.226+0000 7fc7bce00640 1 -- 172.21.15.16:0/4172295977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7b8105b50 msgr2=0x7fc7b8105f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:51.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.226+0000 7fc7bce00640 1 --2- 172.21.15.16:0/4172295977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7b8105b50 0x7fc7b8105f60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8009900 tx=0x7fc7a802a410 comp rx=0 tx=0).stop 2024-08-31T06:05:51.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.226+0000 7fc7bce00640 1 -- 172.21.15.16:0/4172295977 shutdown_connections 2024-08-31T06:05:51.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.226+0000 7fc7bce00640 1 --2- 172.21.15.16:0/4172295977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7b810b490 0x7fc7b810b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:51.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.226+0000 7fc7bce00640 1 --2- 172.21.15.16:0/4172295977 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7b810bde0 0x7fc7b8112610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:51.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.226+0000 7fc7bce00640 1 --2- 172.21.15.16:0/4172295977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7b8105b50 0x7fc7b8105f60 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:51.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.226+0000 7fc7bce00640 1 -- 172.21.15.16:0/4172295977 >> 172.21.15.16:0/4172295977 conn(0x7fc7b8101580 msgr2=0x7fc7b81039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:51.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.226+0000 7fc7bce00640 1 -- 172.21.15.16:0/4172295977 shutdown_connections 2024-08-31T06:05:51.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.226+0000 7fc7bce00640 1 -- 172.21.15.16:0/4172295977 wait complete. 2024-08-31T06:05:51.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.227+0000 7fc7bce00640 1 Processor -- start 2024-08-31T06:05:51.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.228+0000 7fc7bce00640 1 -- start start 2024-08-31T06:05:51.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.228+0000 7fc7bce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7b8105b50 0x7fc7b819c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:51.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.228+0000 7fc7bce00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7b810b490 0x7fc7b819ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:51.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.228+0000 7fc7bce00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7b810bde0 0x7fc7b81a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:51.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.228+0000 7fc7bce00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b8115af0 con 0x7fc7b8105b50 2024-08-31T06:05:51.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.228+0000 7fc7bce00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b8115970 con 0x7fc7b810b490 2024-08-31T06:05:51.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.228+0000 7fc7bce00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b8115c70 con 0x7fc7b810bde0 2024-08-31T06:05:51.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.228+0000 7fc7b5e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7b8105b50 0x7fc7b819c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:51.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.228+0000 7fc7b5e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7b8105b50 0x7fc7b819c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39074/0 (socket says 172.21.15.16:39074) 2024-08-31T06:05:51.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.228+0000 7fc7b5e00640 1 -- 172.21.15.16:0/2038570519 learned_addr learned my addr 172.21.15.16:0/2038570519 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:05:51.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.228+0000 7fc7b5400640 1 --2- 172.21.15.16:0/2038570519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7b810b490 0x7fc7b819ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:51.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.229+0000 7fc7b6800640 1 --2- 172.21.15.16:0/2038570519 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7b810bde0 0x7fc7b81a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:51.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.229+0000 7fc7b5e00640 1 -- 172.21.15.16:0/2038570519 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7b810bde0 msgr2=0x7fc7b81a1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:51.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.229+0000 7fc7b5e00640 1 --2- 172.21.15.16:0/2038570519 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7b810bde0 0x7fc7b81a1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:51.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.229+0000 7fc7b5e00640 1 -- 172.21.15.16:0/2038570519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7b810b490 msgr2=0x7fc7b819ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:51.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.229+0000 7fc7b5e00640 1 --2- 172.21.15.16:0/2038570519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7b810b490 0x7fc7b819ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:51.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.229+0000 7fc7b5e00640 1 -- 172.21.15.16:0/2038570519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b81a23d0 con 0x7fc7b8105b50 2024-08-31T06:05:51.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.229+0000 7fc7b6800640 1 --2- 172.21.15.16:0/2038570519 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7b810bde0 0x7fc7b81a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:05:51.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.229+0000 7fc7b5e00640 1 --2- 172.21.15.16:0/2038570519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7b8105b50 0x7fc7b819c460 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8005ab0 tx=0x7fc7a8003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:51.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.229+0000 7fc7aea00640 1 -- 172.21.15.16:0/2038570519 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a803f070 con 0x7fc7b8105b50 2024-08-31T06:05:51.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.229+0000 7fc7bce00640 1 -- 172.21.15.16:0/2038570519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7b81a2660 con 0x7fc7b8105b50 2024-08-31T06:05:51.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.230+0000 7fc7aea00640 1 -- 172.21.15.16:0/2038570519 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7a8031d60 con 0x7fc7b8105b50 2024-08-31T06:05:51.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.230+0000 7fc7aea00640 1 -- 172.21.15.16:0/2038570519 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a8035030 con 0x7fc7b8105b50 2024-08-31T06:05:51.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.230+0000 7fc7bce00640 1 -- 172.21.15.16:0/2038570519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7b81a8e20 con 0x7fc7b8105b50 2024-08-31T06:05:51.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.231+0000 7fc7bce00640 1 -- 172.21.15.16:0/2038570519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7b8105f60 con 0x7fc7b8105b50 2024-08-31T06:05:51.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.233+0000 7fc7aea00640 1 -- 172.21.15.16:0/2038570519 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7fc7a8042070 con 0x7fc7b8105b50 2024-08-31T06:05:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.235+0000 7fc7aea00640 1 --2- 172.21.15.16:0/2038570519 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc79c0b4c10 0x7fc79c0b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.235+0000 7fc7aea00640 1 -- 172.21.15.16:0/2038570519 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7fc7a8105f00 con 0x7fc7b8105b50 2024-08-31T06:05:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.235+0000 7fc7b5400640 1 --2- 172.21.15.16:0/2038570519 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc79c0b4c10 0x7fc79c0b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.236+0000 7fc7b5400640 1 --2- 172.21.15.16:0/2038570519 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc79c0b4c10 0x7fc79c0b7120 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8107dd0 tx=0x7fc7a400b410 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.237+0000 7fc7aea00640 1 -- 172.21.15.16:0/2038570519 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fc7a80ccdb0 con 0x7fc7b8105b50 2024-08-31T06:05:51.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.473+0000 7fc7bce00640 1 -- 172.21.15.16:0/2038570519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7b819d590 con 0x7fc7b8105b50 2024-08-31T06:05:51.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.474+0000 7fc7aea00640 1 -- 172.21.15.16:0/2038570519 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11917 (secure 0 0 0) 0x7fc7a80ede20 con 0x7fc7b8105b50 2024-08-31T06:05:51.477 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:05:51.477 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"btime":"2024-08-31T06:04:42:937777+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:04:42.745448+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24698":{"gid":24698,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1089710405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1089710405},{"type":"v1","addr":"172.21.15.155:6837","nonce":1089710405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24712,24724,24737]},"id":1}]} 2024-08-31T06:05:51.477 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 24 2024-08-31T06:05:51.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.477+0000 7fc7bce00640 1 -- 172.21.15.16:0/2038570519 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc79c0b4c10 msgr2=0x7fc79c0b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:51.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.477+0000 7fc7bce00640 1 --2- 172.21.15.16:0/2038570519 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc79c0b4c10 0x7fc79c0b7120 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8107dd0 tx=0x7fc7a400b410 comp rx=0 tx=0).stop 2024-08-31T06:05:51.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.477+0000 7fc7bce00640 1 -- 172.21.15.16:0/2038570519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7b8105b50 msgr2=0x7fc7b819c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:51.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.477+0000 7fc7bce00640 1 --2- 172.21.15.16:0/2038570519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7b8105b50 0x7fc7b819c460 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8005ab0 tx=0x7fc7a8003c80 comp rx=0 tx=0).stop 2024-08-31T06:05:51.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.477+0000 7fc7bce00640 1 -- 172.21.15.16:0/2038570519 shutdown_connections 2024-08-31T06:05:51.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.477+0000 7fc7bce00640 1 --2- 172.21.15.16:0/2038570519 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc79c0b4c10 0x7fc79c0b7120 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:51.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.477+0000 7fc7bce00640 1 --2- 172.21.15.16:0/2038570519 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7b810bde0 0x7fc7b81a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:51.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.477+0000 7fc7bce00640 1 --2- 172.21.15.16:0/2038570519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7b810b490 0x7fc7b819ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:51.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.477+0000 7fc7bce00640 1 --2- 172.21.15.16:0/2038570519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7b8105b50 0x7fc7b819c460 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:51.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.477+0000 7fc7bce00640 1 -- 172.21.15.16:0/2038570519 >> 172.21.15.16:0/2038570519 conn(0x7fc7b8101580 msgr2=0x7fc7b810ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:51.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.477+0000 7fc7bce00640 1 -- 172.21.15.16:0/2038570519 shutdown_connections 2024-08-31T06:05:51.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:51.477+0000 7fc7bce00640 1 -- 172.21.15.16:0/2038570519 wait complete. 2024-08-31T06:05:52.260 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:05:52.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:52 smithi155 ceph-mon[21742]: pgmap v322: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-31T06:05:52.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:52 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/2038570519' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:05:52.640 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:05:52.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:52 smithi016 ceph-mon[17194]: pgmap v322: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-31T06:05:52.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:52 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/2038570519' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:05:52.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:52 smithi184 ceph-mon[22331]: pgmap v322: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-31T06:05:52.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:52 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/2038570519' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:05:53.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.284+0000 7fecd7e00640 1 -- 172.21.15.16:0/2841362111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecd8109090 msgr2=0x7fecd81094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:53.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.284+0000 7fecd7e00640 1 --2- 172.21.15.16:0/2841362111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecd8109090 0x7fecd81094a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fecc0009900 tx=0x7fecc002a3f0 comp rx=0 tx=0).stop 2024-08-31T06:05:53.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.285+0000 7fecd7e00640 1 -- 172.21.15.16:0/2841362111 shutdown_connections 2024-08-31T06:05:53.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.285+0000 7fecd7e00640 1 --2- 172.21.15.16:0/2841362111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fecd81099e0 0x7fecd8110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:53.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.285+0000 7fecd7e00640 1 --2- 172.21.15.16:0/2841362111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecd8109090 0x7fecd81094a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:53.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.285+0000 7fecd7e00640 1 --2- 172.21.15.16:0/2841362111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fecd8103750 0x7fecd8103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:53.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.285+0000 7fecd7e00640 1 -- 172.21.15.16:0/2841362111 >> 172.21.15.16:0/2841362111 conn(0x7fecd80ff1a0 msgr2=0x7fecd81015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:53.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.285+0000 7fecd7e00640 1 -- 172.21.15.16:0/2841362111 shutdown_connections 2024-08-31T06:05:53.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.285+0000 7fecd7e00640 1 -- 172.21.15.16:0/2841362111 wait complete. 2024-08-31T06:05:53.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.286+0000 7fecd7e00640 1 Processor -- start 2024-08-31T06:05:53.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.286+0000 7fecd7e00640 1 -- start start 2024-08-31T06:05:53.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.286+0000 7fecd7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fecd8103750 0x7fecd819a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:53.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.286+0000 7fecd7e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fecd8109090 0x7fecd819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:53.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.286+0000 7fecd7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecd81099e0 0x7fecd819f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:53.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.286+0000 7fecd7e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fecd8113650 con 0x7fecd81099e0 2024-08-31T06:05:53.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.286+0000 7fecd7e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fecd81134d0 con 0x7fecd8103750 2024-08-31T06:05:53.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.286+0000 7fecd7e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fecd81137d0 con 0x7fecd8109090 2024-08-31T06:05:53.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.287+0000 7fecd7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecd81099e0 0x7fecd819f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:53.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.287+0000 7fecd6a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fecd8103750 0x7fecd819a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:53.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.287+0000 7feccfe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fecd8109090 0x7fecd819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:53.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.287+0000 7fecd6a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fecd8103750 0x7fecd819a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:53260/0 (socket says 172.21.15.16:53260) 2024-08-31T06:05:53.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.287+0000 7feccfe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fecd8109090 0x7fecd819a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:59990/0 (socket says 172.21.15.16:59990) 2024-08-31T06:05:53.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.287+0000 7fecd6a00640 1 -- 172.21.15.16:0/2949720744 learned_addr learned my addr 172.21.15.16:0/2949720744 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:05:53.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.287+0000 7fecd6a00640 1 -- 172.21.15.16:0/2949720744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fecd8109090 msgr2=0x7fecd819a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:53.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.287+0000 7fecd6a00640 1 --2- 172.21.15.16:0/2949720744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fecd8109090 0x7fecd819a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:53.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.287+0000 7fecd6a00640 1 -- 172.21.15.16:0/2949720744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecd81099e0 msgr2=0x7fecd819f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:53.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.287+0000 7fecd6a00640 1 --2- 172.21.15.16:0/2949720744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecd81099e0 0x7fecd819f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:53.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.287+0000 7fecd6a00640 1 -- 172.21.15.16:0/2949720744 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecd81a0010 con 0x7fecd8103750 2024-08-31T06:05:53.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.287+0000 7fecd7400640 1 --2- 172.21.15.16:0/2949720744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecd81099e0 0x7fecd819f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:05:53.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.288+0000 7fecd6a00640 1 --2- 172.21.15.16:0/2949720744 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fecd8103750 0x7fecd819a0a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fecc800d950 tx=0x7fecc800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:53.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.288+0000 7feccf400640 1 -- 172.21.15.16:0/2949720744 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc8017030 con 0x7fecd8103750 2024-08-31T06:05:53.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.288+0000 7feccf400640 1 -- 172.21.15.16:0/2949720744 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecc8004020 con 0x7fecd8103750 2024-08-31T06:05:53.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.288+0000 7feccf400640 1 -- 172.21.15.16:0/2949720744 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc80047a0 con 0x7fecd8103750 2024-08-31T06:05:53.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.288+0000 7fecd7e00640 1 -- 172.21.15.16:0/2949720744 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecd81a0300 con 0x7fecd8103750 2024-08-31T06:05:53.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.288+0000 7fecd7e00640 1 -- 172.21.15.16:0/2949720744 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecd81a6a60 con 0x7fecd8103750 2024-08-31T06:05:53.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.289+0000 7fecd7e00640 1 -- 172.21.15.16:0/2949720744 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecd8103b60 con 0x7fecd8103750 2024-08-31T06:05:53.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.292+0000 7feccf400640 1 -- 172.21.15.16:0/2949720744 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7fecc8009a30 con 0x7fecd8103750 2024-08-31T06:05:53.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.293+0000 7feccf400640 1 --2- 172.21.15.16:0/2949720744 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fecbc0b4c10 0x7fecbc0b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:53.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.293+0000 7feccf400640 1 -- 172.21.15.16:0/2949720744 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7fecc80e54f0 con 0x7fecd8103750 2024-08-31T06:05:53.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.294+0000 7feccfe00640 1 --2- 172.21.15.16:0/2949720744 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fecbc0b4c10 0x7fecbc0b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:53.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.294+0000 7feccfe00640 1 --2- 172.21.15.16:0/2949720744 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fecbc0b4c10 0x7fecbc0b7120 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fecc0009230 tx=0x7fecc0031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:53.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.295+0000 7feccf400640 1 -- 172.21.15.16:0/2949720744 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fecc80ac3a0 con 0x7fecd8103750 2024-08-31T06:05:53.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.533+0000 7fecd7e00640 1 -- 172.21.15.16:0/2949720744 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fecd819b1d0 con 0x7fecd8103750 2024-08-31T06:05:53.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.534+0000 7feccf400640 1 -- 172.21.15.16:0/2949720744 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11917 (secure 0 0 0) 0x7fecc80abca0 con 0x7fecd8103750 2024-08-31T06:05:53.537 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:05:53.537 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"btime":"2024-08-31T06:04:42:937777+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:04:42.745448+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24698":{"gid":24698,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1089710405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1089710405},{"type":"v1","addr":"172.21.15.155:6837","nonce":1089710405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24712,24724,24737]},"id":1}]} 2024-08-31T06:05:53.537 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 24 2024-08-31T06:05:53.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.537+0000 7fecd7e00640 1 -- 172.21.15.16:0/2949720744 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fecbc0b4c10 msgr2=0x7fecbc0b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:53.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.537+0000 7fecd7e00640 1 --2- 172.21.15.16:0/2949720744 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fecbc0b4c10 0x7fecbc0b7120 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fecc0009230 tx=0x7fecc0031400 comp rx=0 tx=0).stop 2024-08-31T06:05:53.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.537+0000 7fecd7e00640 1 -- 172.21.15.16:0/2949720744 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fecd8103750 msgr2=0x7fecd819a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:53.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.537+0000 7fecd7e00640 1 --2- 172.21.15.16:0/2949720744 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fecd8103750 0x7fecd819a0a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fecc800d950 tx=0x7fecc800de10 comp rx=0 tx=0).stop 2024-08-31T06:05:53.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.537+0000 7fecd7e00640 1 -- 172.21.15.16:0/2949720744 shutdown_connections 2024-08-31T06:05:53.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.537+0000 7fecd7e00640 1 --2- 172.21.15.16:0/2949720744 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fecbc0b4c10 0x7fecbc0b7120 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:53.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.537+0000 7fecd7e00640 1 --2- 172.21.15.16:0/2949720744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fecd8109090 0x7fecd819a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:53.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.537+0000 7fecd7e00640 1 --2- 172.21.15.16:0/2949720744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecd81099e0 0x7fecd819f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:53.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.537+0000 7fecd7e00640 1 --2- 172.21.15.16:0/2949720744 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fecd8103750 0x7fecd819a0a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:53.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.537+0000 7fecd7e00640 1 -- 172.21.15.16:0/2949720744 >> 172.21.15.16:0/2949720744 conn(0x7fecd80ff1a0 msgr2=0x7fecd81015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:53.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.537+0000 7fecd7e00640 1 -- 172.21.15.16:0/2949720744 shutdown_connections 2024-08-31T06:05:53.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:53.537+0000 7fecd7e00640 1 -- 172.21.15.16:0/2949720744 wait complete. 2024-08-31T06:05:54.470 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph osd dump --format=json 2024-08-31T06:05:54.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:54 smithi155 ceph-mon[21742]: pgmap v323: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-31T06:05:54.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:54 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/2949720744' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:05:54.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:54 smithi016 ceph-mon[17194]: pgmap v323: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-31T06:05:54.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:54 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/2949720744' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:05:54.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:54 smithi184 ceph-mon[22331]: pgmap v323: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-31T06:05:54.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:54 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/2949720744' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:05:54.847 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:05:55.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.470+0000 7fef17200640 1 -- 172.21.15.16:0/929519557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef18109070 msgr2=0x7fef18109480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:55.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.470+0000 7fef17200640 1 --2- 172.21.15.16:0/929519557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef18109070 0x7fef18109480 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fef04009900 tx=0x7fef0402a410 comp rx=0 tx=0).stop 2024-08-31T06:05:55.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.471+0000 7fef17200640 1 -- 172.21.15.16:0/929519557 shutdown_connections 2024-08-31T06:05:55.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.471+0000 7fef17200640 1 --2- 172.21.15.16:0/929519557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef181099c0 0x7fef181101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:55.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.471+0000 7fef17200640 1 --2- 172.21.15.16:0/929519557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef18109070 0x7fef18109480 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:55.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.471+0000 7fef17200640 1 --2- 172.21.15.16:0/929519557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef18103730 0x7fef18103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:55.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.471+0000 7fef17200640 1 -- 172.21.15.16:0/929519557 >> 172.21.15.16:0/929519557 conn(0x7fef180ff180 msgr2=0x7fef181015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:55.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.471+0000 7fef17200640 1 -- 172.21.15.16:0/929519557 shutdown_connections 2024-08-31T06:05:55.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.471+0000 7fef17200640 1 -- 172.21.15.16:0/929519557 wait complete. 2024-08-31T06:05:55.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.472+0000 7fef17200640 1 Processor -- start 2024-08-31T06:05:55.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.472+0000 7fef17200640 1 -- start start 2024-08-31T06:05:55.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.472+0000 7fef17200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef18103730 0x7fef1819c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:55.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.472+0000 7fef17200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef18109070 0x7fef1819c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:55.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.472+0000 7fef17200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef181099c0 0x7fef181a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:55.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.472+0000 7fef17200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fef18113650 con 0x7fef18109070 2024-08-31T06:05:55.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.472+0000 7fef17200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fef181134d0 con 0x7fef18103730 2024-08-31T06:05:55.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.472+0000 7fef17200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fef181137d0 con 0x7fef181099c0 2024-08-31T06:05:55.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.473+0000 7fef16800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef181099c0 0x7fef181a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:55.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.473+0000 7fef15400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef18109070 0x7fef1819c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:55.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.473+0000 7fef16800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef181099c0 0x7fef181a1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:60012/0 (socket says 172.21.15.16:60012) 2024-08-31T06:05:55.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.473+0000 7fef16800640 1 -- 172.21.15.16:0/2806481830 learned_addr learned my addr 172.21.15.16:0/2806481830 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:05:55.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.473+0000 7fef15e00640 1 --2- 172.21.15.16:0/2806481830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef18103730 0x7fef1819c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:55.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.473+0000 7fef15400640 1 -- 172.21.15.16:0/2806481830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef181099c0 msgr2=0x7fef181a1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:55.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.473+0000 7fef15400640 1 --2- 172.21.15.16:0/2806481830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef181099c0 0x7fef181a1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:55.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.473+0000 7fef15400640 1 -- 172.21.15.16:0/2806481830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef18103730 msgr2=0x7fef1819c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:55.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.473+0000 7fef15400640 1 --2- 172.21.15.16:0/2806481830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef18103730 0x7fef1819c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:55.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.473+0000 7fef15400640 1 -- 172.21.15.16:0/2806481830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef181a2210 con 0x7fef18109070 2024-08-31T06:05:55.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.473+0000 7fef16800640 1 --2- 172.21.15.16:0/2806481830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef181099c0 0x7fef181a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:05:55.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.474+0000 7fef15400640 1 --2- 172.21.15.16:0/2806481830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef18109070 0x7fef1819c840 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fef0402a930 tx=0x7fef04031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:55.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.474+0000 7fef0ea00640 1 -- 172.21.15.16:0/2806481830 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0403f070 con 0x7fef18109070 2024-08-31T06:05:55.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.474+0000 7fef0ea00640 1 -- 172.21.15.16:0/2806481830 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef04004490 con 0x7fef18109070 2024-08-31T06:05:55.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.474+0000 7fef0ea00640 1 -- 172.21.15.16:0/2806481830 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef04035070 con 0x7fef18109070 2024-08-31T06:05:55.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.474+0000 7fef17200640 1 -- 172.21.15.16:0/2806481830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef181a24a0 con 0x7fef18109070 2024-08-31T06:05:55.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.474+0000 7fef17200640 1 -- 172.21.15.16:0/2806481830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef181a8c60 con 0x7fef18109070 2024-08-31T06:05:55.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.475+0000 7fef17200640 1 -- 172.21.15.16:0/2806481830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef18103b40 con 0x7fef18109070 2024-08-31T06:05:55.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.478+0000 7fef0ea00640 1 -- 172.21.15.16:0/2806481830 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7fef0403b460 con 0x7fef18109070 2024-08-31T06:05:55.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.479+0000 7fef0ea00640 1 --2- 172.21.15.16:0/2806481830 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7feefc0b4c90 0x7feefc0b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:55.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.479+0000 7fef0ea00640 1 -- 172.21.15.16:0/2806481830 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7fef04105b00 con 0x7fef18109070 2024-08-31T06:05:55.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.479+0000 7fef15e00640 1 --2- 172.21.15.16:0/2806481830 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7feefc0b4c90 0x7feefc0b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:55.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.480+0000 7fef15e00640 1 --2- 172.21.15.16:0/2806481830 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7feefc0b4c90 0x7feefc0b71a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fef08009bd0 tx=0x7fef08005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:55.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.481+0000 7fef0ea00640 1 -- 172.21.15.16:0/2806481830 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fef040cc360 con 0x7fef18109070 2024-08-31T06:05:55.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.678+0000 7fef17200640 1 -- 172.21.15.16:0/2806481830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fef181a8e30 con 0x7fef18109070 2024-08-31T06:05:55.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.679+0000 7fef0ea00640 1 -- 172.21.15.16:0/2806481830 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+23043 (secure 0 0 0) 0x7fef040d0b20 con 0x7fef18109070 2024-08-31T06:05:55.682 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:05:55.682 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":80,"fsid":"673e4aa0-675d-11ef-bcd4-c7b262605968","created":"2024-08-31T05:55:09.632536+0000","modified":"2024-08-31T06:05:25.039584+0000","last_up_change":"2024-08-31T06:03:27.893697+0000","last_in_change":"2024-08-31T06:03:05.664945+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-31T05:59:23.642270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-31T06:04:20.871835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-31T06:04:24.119499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e349dae-6aa5-4b6d-9107-a945655e11bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6803","nonce":1725037400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6805","nonce":1725037400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6809","nonce":1725037400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6807","nonce":1725037400}]},"public_addr":"172.21.15.16:6803/1725037400","cluster_addr":"172.21.15.16:6805/1725037400","heartbeat_back_addr":"172.21.15.16:6809/1725037400","heartbeat_front_addr":"172.21.15.16:6807/1725037400","state":["exists","up"]},{"osd":1,"uuid":"bd645dc3-93b5-40af-89d8-08ccec22e50d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6801","nonce":4241618440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6803","nonce":4241618440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6807","nonce":4241618440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6805","nonce":4241618440}]},"public_addr":"172.21.15.155:6801/4241618440","cluster_addr":"172.21.15.155:6803/4241618440","heartbeat_back_addr":"172.21.15.155:6807/4241618440","heartbeat_front_addr":"172.21.15.155:6805/4241618440","state":["exists","up"]},{"osd":2,"uuid":"c69c8c19-32f8-400a-86d4-0f4ce240a7de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6801","nonce":2193296422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6803","nonce":2193296422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6807","nonce":2193296422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6805","nonce":2193296422}]},"public_addr":"172.21.15.184:6801/2193296422","cluster_addr":"172.21.15.184:6803/2193296422","heartbeat_back_addr":"172.21.15.184:6807/2193296422","heartbeat_front_addr":"172.21.15.184:6805/2193296422","state":["exists","up"]},{"osd":3,"uuid":"8892e35f-6880-434c-a805-ab8ccbed2bf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6811","nonce":81681815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6813","nonce":81681815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6817","nonce":81681815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6815","nonce":81681815}]},"public_addr":"172.21.15.16:6811/81681815","cluster_addr":"172.21.15.16:6813/81681815","heartbeat_back_addr":"172.21.15.16:6817/81681815","heartbeat_front_addr":"172.21.15.16:6815/81681815","state":["exists","up"]},{"osd":4,"uuid":"d2515e73-88f7-4654-9762-adb8802777ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6809","nonce":3592422019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6811","nonce":3592422019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6815","nonce":3592422019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6813","nonce":3592422019}]},"public_addr":"172.21.15.155:6809/3592422019","cluster_addr":"172.21.15.155:6811/3592422019","heartbeat_back_addr":"172.21.15.155:6815/3592422019","heartbeat_front_addr":"172.21.15.155:6813/3592422019","state":["exists","up"]},{"osd":5,"uuid":"5ce6e2fa-c17c-49b3-9402-f19bfc21076d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6809","nonce":2054894065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6811","nonce":2054894065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6815","nonce":2054894065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6813","nonce":2054894065}]},"public_addr":"172.21.15.184:6809/2054894065","cluster_addr":"172.21.15.184:6811/2054894065","heartbeat_back_addr":"172.21.15.184:6815/2054894065","heartbeat_front_addr":"172.21.15.184:6813/2054894065","state":["exists","up"]},{"osd":6,"uuid":"f257d8d1-ad59-4bd8-821d-bf29e50b547e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6819","nonce":1697421092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6821","nonce":1697421092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6825","nonce":1697421092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6823","nonce":1697421092}]},"public_addr":"172.21.15.16:6819/1697421092","cluster_addr":"172.21.15.16:6821/1697421092","heartbeat_back_addr":"172.21.15.16:6825/1697421092","heartbeat_front_addr":"172.21.15.16:6823/1697421092","state":["exists","up"]},{"osd":7,"uuid":"8e76c20e-8d25-423b-b7e8-de642d9c9a6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6817","nonce":2671827901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6819","nonce":2671827901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6823","nonce":2671827901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6821","nonce":2671827901}]},"public_addr":"172.21.15.155:6817/2671827901","cluster_addr":"172.21.15.155:6819/2671827901","heartbeat_back_addr":"172.21.15.155:6823/2671827901","heartbeat_front_addr":"172.21.15.155:6821/2671827901","state":["exists","up"]},{"osd":8,"uuid":"0f0bb23f-4bc3-4198-8381-5b5cd16ae394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6817","nonce":2479117823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6819","nonce":2479117823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6823","nonce":2479117823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6821","nonce":2479117823}]},"public_addr":"172.21.15.184:6817/2479117823","cluster_addr":"172.21.15.184:6819/2479117823","heartbeat_back_addr":"172.21.15.184:6823/2479117823","heartbeat_front_addr":"172.21.15.184:6821/2479117823","state":["exists","up"]},{"osd":9,"uuid":"bc27afcb-7871-4f72-a043-044fd5fe7d86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6827","nonce":3015700546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6829","nonce":3015700546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6833","nonce":3015700546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6831","nonce":3015700546}]},"public_addr":"172.21.15.16:6827/3015700546","cluster_addr":"172.21.15.16:6829/3015700546","heartbeat_back_addr":"172.21.15.16:6833/3015700546","heartbeat_front_addr":"172.21.15.16:6831/3015700546","state":["exists","up"]},{"osd":10,"uuid":"f2548997-72ee-400e-ab8d-5966d85b2cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6825","nonce":2427122475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6827","nonce":2427122475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6831","nonce":2427122475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6829","nonce":2427122475}]},"public_addr":"172.21.15.155:6825/2427122475","cluster_addr":"172.21.15.155:6827/2427122475","heartbeat_back_addr":"172.21.15.155:6831/2427122475","heartbeat_front_addr":"172.21.15.155:6829/2427122475","state":["exists","up"]},{"osd":11,"uuid":"8297da1d-d3e2-4a4d-9396-426dfafa3e95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6825","nonce":399265701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6827","nonce":399265701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6831","nonce":399265701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6829","nonce":399265701}]},"public_addr":"172.21.15.184:6825/399265701","cluster_addr":"172.21.15.184:6827/399265701","heartbeat_back_addr":"172.21.15.184:6831/399265701","heartbeat_front_addr":"172.21.15.184:6829/399265701","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:58:30.842360+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:58:57.164488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:59:20.778103+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:59:50.123991+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:00:16.789933+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:00:41.788905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:01:12.281376+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:01:39.211858+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:02:04.958771+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:02:35.645947+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:03:01.940433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:03:26.728673+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3923220960":"2024-09-01T05:57:23.153214+0000","172.21.15.16:0/3013533836":"2024-09-01T05:57:23.153214+0000","172.21.15.16:6800/3755683228":"2024-09-01T05:55:45.075902+0000","172.21.15.16:0/3064435470":"2024-09-01T05:55:45.075902+0000","172.21.15.16:0/911709962":"2024-09-01T05:55:45.075902+0000","172.21.15.16:0/2937368967":"2024-09-01T05:55:45.075902+0000","172.21.15.16:6800/1378972230":"2024-09-01T05:56:14.195704+0000","172.21.15.16:0/3047316995":"2024-09-01T05:56:14.195704+0000","172.21.15.16:6801/3755683228":"2024-09-01T05:55:45.075902+0000","172.21.15.16:6801/2571082634":"2024-09-01T05:57:23.153214+0000","172.21.15.16:0/2136704295":"2024-09-01T05:56:14.195704+0000","172.21.15.16:0/2259332778":"2024-09-01T05:56:14.195704+0000","172.21.15.16:6800/2571082634":"2024-09-01T05:57:23.153214+0000","172.21.15.16:6801/1378972230":"2024-09-01T05:56:14.195704+0000","172.21.15.16:0/2126457673":"2024-09-01T05:57:23.153214+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-31T06:05:55.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.682+0000 7fef17200640 1 -- 172.21.15.16:0/2806481830 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7feefc0b4c90 msgr2=0x7feefc0b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:55.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.682+0000 7fef17200640 1 --2- 172.21.15.16:0/2806481830 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7feefc0b4c90 0x7feefc0b71a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fef08009bd0 tx=0x7fef08005f90 comp rx=0 tx=0).stop 2024-08-31T06:05:55.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.682+0000 7fef17200640 1 -- 172.21.15.16:0/2806481830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef18109070 msgr2=0x7fef1819c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:55.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.682+0000 7fef17200640 1 --2- 172.21.15.16:0/2806481830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef18109070 0x7fef1819c840 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fef0402a930 tx=0x7fef04031a20 comp rx=0 tx=0).stop 2024-08-31T06:05:55.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.682+0000 7fef17200640 1 -- 172.21.15.16:0/2806481830 shutdown_connections 2024-08-31T06:05:55.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.682+0000 7fef17200640 1 --2- 172.21.15.16:0/2806481830 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7feefc0b4c90 0x7feefc0b71a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:55.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.682+0000 7fef17200640 1 --2- 172.21.15.16:0/2806481830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef181099c0 0x7fef181a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:55.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.682+0000 7fef17200640 1 --2- 172.21.15.16:0/2806481830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef18109070 0x7fef1819c840 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:55.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.682+0000 7fef17200640 1 --2- 172.21.15.16:0/2806481830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef18103730 0x7fef1819c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:55.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.682+0000 7fef17200640 1 -- 172.21.15.16:0/2806481830 >> 172.21.15.16:0/2806481830 conn(0x7fef180ff180 msgr2=0x7fef1810e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:55.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.682+0000 7fef17200640 1 -- 172.21.15.16:0/2806481830 shutdown_connections 2024-08-31T06:05:55.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:55.682+0000 7fef17200640 1 -- 172.21.15.16:0/2806481830 wait complete. 2024-08-31T06:05:56.589 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:05:56.621 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:56 smithi155 ceph-mon[21742]: pgmap v324: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-31T06:05:56.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:56 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/2806481830' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T06:05:56.637 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:56 smithi016 ceph-mon[17194]: pgmap v324: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-31T06:05:56.637 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:56 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/2806481830' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T06:05:56.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:56 smithi184 ceph-mon[22331]: pgmap v324: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-31T06:05:56.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:56 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/2806481830' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T06:05:56.966 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:05:57.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.614+0000 7fb253200640 1 -- 172.21.15.16:0/2439474060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb24c105b20 msgr2=0x7fb24c105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:57.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.614+0000 7fb253200640 1 --2- 172.21.15.16:0/2439474060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb24c105b20 0x7fb24c105f30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb23c009900 tx=0x7fb23c02a3f0 comp rx=0 tx=0).stop 2024-08-31T06:05:57.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.614+0000 7fb253200640 1 -- 172.21.15.16:0/2439474060 shutdown_connections 2024-08-31T06:05:57.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.614+0000 7fb253200640 1 --2- 172.21.15.16:0/2439474060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb24c10b460 0x7fb24c10b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:57.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.614+0000 7fb253200640 1 --2- 172.21.15.16:0/2439474060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb24c10bdb0 0x7fb24c1125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:57.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.614+0000 7fb253200640 1 --2- 172.21.15.16:0/2439474060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb24c105b20 0x7fb24c105f30 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:57.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.614+0000 7fb253200640 1 -- 172.21.15.16:0/2439474060 >> 172.21.15.16:0/2439474060 conn(0x7fb24c1015b0 msgr2=0x7fb24c103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:57.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.614+0000 7fb253200640 1 -- 172.21.15.16:0/2439474060 shutdown_connections 2024-08-31T06:05:57.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.614+0000 7fb253200640 1 -- 172.21.15.16:0/2439474060 wait complete. 2024-08-31T06:05:57.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.615+0000 7fb253200640 1 Processor -- start 2024-08-31T06:05:57.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.615+0000 7fb253200640 1 -- start start 2024-08-31T06:05:57.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.615+0000 7fb253200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb24c105b20 0x7fb24c19c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:57.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.615+0000 7fb253200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb24c10b460 0x7fb24c19ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:57.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.616+0000 7fb253200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb24c10bdb0 0x7fb24c1a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:57.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.616+0000 7fb253200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb24c115ac0 con 0x7fb24c10b460 2024-08-31T06:05:57.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.616+0000 7fb253200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb24c115940 con 0x7fb24c105b20 2024-08-31T06:05:57.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.616+0000 7fb253200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb24c115c40 con 0x7fb24c10bdb0 2024-08-31T06:05:57.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.616+0000 7fb24b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb24c10b460 0x7fb24c19ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:57.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.616+0000 7fb24be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb24c105b20 0x7fb24c19c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:57.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.616+0000 7fb24be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb24c105b20 0x7fb24c19c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:57310/0 (socket says 172.21.15.16:57310) 2024-08-31T06:05:57.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.616+0000 7fb24be00640 1 -- 172.21.15.16:0/2914513204 learned_addr learned my addr 172.21.15.16:0/2914513204 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:05:57.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.616+0000 7fb24b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb24c10b460 0x7fb24c19ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38042/0 (socket says 172.21.15.16:38042) 2024-08-31T06:05:57.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.616+0000 7fb251200640 1 --2- 172.21.15.16:0/2914513204 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb24c10bdb0 0x7fb24c1a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:57.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.616+0000 7fb24b400640 1 -- 172.21.15.16:0/2914513204 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb24c10bdb0 msgr2=0x7fb24c1a1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:57.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.616+0000 7fb24b400640 1 --2- 172.21.15.16:0/2914513204 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb24c10bdb0 0x7fb24c1a1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:57.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.616+0000 7fb24b400640 1 -- 172.21.15.16:0/2914513204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb24c105b20 msgr2=0x7fb24c19c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:57.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.616+0000 7fb24b400640 1 --2- 172.21.15.16:0/2914513204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb24c105b20 0x7fb24c19c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:57.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.616+0000 7fb24b400640 1 -- 172.21.15.16:0/2914513204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb24c1a2440 con 0x7fb24c10b460 2024-08-31T06:05:57.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.617+0000 7fb24b400640 1 --2- 172.21.15.16:0/2914513204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb24c10b460 0x7fb24c19ca70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb24c06c840 tx=0x7fb23800ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:57.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.617+0000 7fb248c00640 1 -- 172.21.15.16:0/2914513204 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23800ccc0 con 0x7fb24c10b460 2024-08-31T06:05:57.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.617+0000 7fb253200640 1 -- 172.21.15.16:0/2914513204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb24c1a2730 con 0x7fb24c10b460 2024-08-31T06:05:57.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.617+0000 7fb248c00640 1 -- 172.21.15.16:0/2914513204 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb238009e40 con 0x7fb24c10b460 2024-08-31T06:05:57.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.617+0000 7fb248c00640 1 -- 172.21.15.16:0/2914513204 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb238004a00 con 0x7fb24c10b460 2024-08-31T06:05:57.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.617+0000 7fb253200640 1 -- 172.21.15.16:0/2914513204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb24c1a8e90 con 0x7fb24c10b460 2024-08-31T06:05:57.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.618+0000 7fb253200640 1 -- 172.21.15.16:0/2914513204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb24c106da0 con 0x7fb24c10b460 2024-08-31T06:05:57.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.622+0000 7fb248c00640 1 -- 172.21.15.16:0/2914513204 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7fb238005020 con 0x7fb24c10b460 2024-08-31T06:05:57.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.623+0000 7fb248c00640 1 --2- 172.21.15.16:0/2914513204 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb2300b4c10 0x7fb2300b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:57.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.623+0000 7fb248c00640 1 -- 172.21.15.16:0/2914513204 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7fb2380e5440 con 0x7fb24c10b460 2024-08-31T06:05:57.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.623+0000 7fb24be00640 1 --2- 172.21.15.16:0/2914513204 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb2300b4c10 0x7fb2300b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:57.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.624+0000 7fb24be00640 1 --2- 172.21.15.16:0/2914513204 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb2300b4c10 0x7fb2300b7120 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb23c0096c0 tx=0x7fb23c005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:57.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.624+0000 7fb248c00640 1 -- 172.21.15.16:0/2914513204 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fb2380ac3e0 con 0x7fb24c10b460 2024-08-31T06:05:57.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.859+0000 7fb253200640 1 -- 172.21.15.16:0/2914513204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb24c10dbe0 con 0x7fb24c10b460 2024-08-31T06:05:57.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.860+0000 7fb248c00640 1 -- 172.21.15.16:0/2914513204 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11917 (secure 0 0 0) 0x7fb2380abce0 con 0x7fb24c10b460 2024-08-31T06:05:57.863 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:05:57.863 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"btime":"2024-08-31T06:04:42:937777+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:04:42.745448+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24698":{"gid":24698,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1089710405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1089710405},{"type":"v1","addr":"172.21.15.155:6837","nonce":1089710405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24712,24724,24737]},"id":1}]} 2024-08-31T06:05:57.863 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 24 2024-08-31T06:05:57.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.863+0000 7fb253200640 1 -- 172.21.15.16:0/2914513204 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb2300b4c10 msgr2=0x7fb2300b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:57.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.863+0000 7fb253200640 1 --2- 172.21.15.16:0/2914513204 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb2300b4c10 0x7fb2300b7120 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb23c0096c0 tx=0x7fb23c005a20 comp rx=0 tx=0).stop 2024-08-31T06:05:57.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.863+0000 7fb253200640 1 -- 172.21.15.16:0/2914513204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb24c10b460 msgr2=0x7fb24c19ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:57.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.863+0000 7fb253200640 1 --2- 172.21.15.16:0/2914513204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb24c10b460 0x7fb24c19ca70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb24c06c840 tx=0x7fb23800ed90 comp rx=0 tx=0).stop 2024-08-31T06:05:57.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.863+0000 7fb253200640 1 -- 172.21.15.16:0/2914513204 shutdown_connections 2024-08-31T06:05:57.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.863+0000 7fb253200640 1 --2- 172.21.15.16:0/2914513204 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb2300b4c10 0x7fb2300b7120 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:57.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.863+0000 7fb253200640 1 --2- 172.21.15.16:0/2914513204 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb24c10bdb0 0x7fb24c1a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:57.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.863+0000 7fb253200640 1 --2- 172.21.15.16:0/2914513204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb24c10b460 0x7fb24c19ca70 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:57.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.863+0000 7fb253200640 1 --2- 172.21.15.16:0/2914513204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb24c105b20 0x7fb24c19c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:57.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.863+0000 7fb253200640 1 -- 172.21.15.16:0/2914513204 >> 172.21.15.16:0/2914513204 conn(0x7fb24c1015b0 msgr2=0x7fb24c1039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:57.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.864+0000 7fb253200640 1 -- 172.21.15.16:0/2914513204 shutdown_connections 2024-08-31T06:05:57.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:57.864+0000 7fb253200640 1 -- 172.21.15.16:0/2914513204 wait complete. 2024-08-31T06:05:58.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:58 smithi155 ceph-mon[21742]: pgmap v325: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-31T06:05:58.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:05:58 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/2914513204' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:05:58.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:58 smithi016 ceph-mon[17194]: pgmap v325: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-31T06:05:58.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:05:58 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/2914513204' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:05:58.725 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:58 smithi184 ceph-mon[22331]: pgmap v325: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-31T06:05:58.725 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:05:58 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/2914513204' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:05:58.738 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-08-31T06:04:27.098649+0000', 'modified': '2024-08-31T06:04:42.745448+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24737, 'mds_1': 24724, 'mds_2': 24712}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24688': {'gid': 24688, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.184:6835/1925472309', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6834', 'nonce': 1925472309}, {'type': 'v1', 'addr': '172.21.15.184:6835', 'nonce': 1925472309}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24694': {'gid': 24694, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.184:6837/1712755588', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6836', 'nonce': 1712755588}, {'type': 'v1', 'addr': '172.21.15.184:6837', 'nonce': 1712755588}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24698': {'gid': 24698, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.155:6837/1089710405', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6836', 'nonce': 1089710405}, {'type': 'v1', 'addr': '172.21.15.155:6837', 'nonce': 1089710405}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24712': {'gid': 24712, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.184:6839/4011051479', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6838', 'nonce': 4011051479}, {'type': 'v1', 'addr': '172.21.15.184:6839', 'nonce': 4011051479}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24724': {'gid': 24724, 'name': 'g', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.16:6839/219664460', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6838', 'nonce': 219664460}, {'type': 'v1', 'addr': '172.21.15.16:6839', 'nonce': 219664460}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24737': {'gid': 24737, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.155:6839/390606290', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6838', 'nonce': 390606290}, {'type': 'v1', 'addr': '172.21.15.155:6839', 'nonce': 390606290}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24737, 'qdb_cluster': [24712, 24724, 24737]} 2024-08-31T06:05:58.739 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-08-31T06:05:58.739 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.i status 2024-08-31T06:05:59.115 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:05:59.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.735+0000 7f7a7b200640 1 -- 172.21.15.16:0/714798973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a74109260 msgr2=0x7f7a74109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:59.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.735+0000 7f7a7b200640 1 --2- 172.21.15.16:0/714798973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a74109260 0x7f7a74109670 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c009900 tx=0x7f7a5c02a3f0 comp rx=0 tx=0).stop 2024-08-31T06:05:59.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.735+0000 7f7a7b200640 1 -- 172.21.15.16:0/714798973 shutdown_connections 2024-08-31T06:05:59.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.735+0000 7f7a7b200640 1 --2- 172.21.15.16:0/714798973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a74109bb0 0x7f7a741103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:59.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.735+0000 7f7a7b200640 1 --2- 172.21.15.16:0/714798973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a74109260 0x7f7a74109670 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:59.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.735+0000 7f7a7b200640 1 --2- 172.21.15.16:0/714798973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a74103920 0x7f7a74103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:59.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.735+0000 7f7a7b200640 1 -- 172.21.15.16:0/714798973 >> 172.21.15.16:0/714798973 conn(0x7f7a740ff330 msgr2=0x7f7a74101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:59.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.735+0000 7f7a7b200640 1 -- 172.21.15.16:0/714798973 shutdown_connections 2024-08-31T06:05:59.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.735+0000 7f7a7b200640 1 -- 172.21.15.16:0/714798973 wait complete. 2024-08-31T06:05:59.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.736+0000 7f7a7b200640 1 Processor -- start 2024-08-31T06:05:59.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.737+0000 7f7a7b200640 1 -- start start 2024-08-31T06:05:59.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.737+0000 7f7a7b200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a74103920 0x7f7a7419c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:59.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.737+0000 7f7a7b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a74109260 0x7f7a7419ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:59.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.737+0000 7f7a7b200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a74109bb0 0x7f7a741a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:59.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.737+0000 7f7a7b200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7a741138c0 con 0x7f7a74109260 2024-08-31T06:05:59.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.737+0000 7f7a7b200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7a74113740 con 0x7f7a74103920 2024-08-31T06:05:59.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.737+0000 7f7a7b200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7a74113a40 con 0x7f7a74109bb0 2024-08-31T06:05:59.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.737+0000 7f7a6be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a74109260 0x7f7a7419ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:59.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.737+0000 7f7a6be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a74109260 0x7f7a7419ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38056/0 (socket says 172.21.15.16:38056) 2024-08-31T06:05:59.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.737+0000 7f7a6be00640 1 -- 172.21.15.16:0/3216707086 learned_addr learned my addr 172.21.15.16:0/3216707086 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:05:59.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.737+0000 7f7a73e00640 1 --2- 172.21.15.16:0/3216707086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a74103920 0x7f7a7419c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:59.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.738+0000 7f7a79200640 1 --2- 172.21.15.16:0/3216707086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a74109bb0 0x7f7a741a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.738+0000 7f7a6be00640 1 -- 172.21.15.16:0/3216707086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a74109bb0 msgr2=0x7f7a741a1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.738+0000 7f7a6be00640 1 --2- 172.21.15.16:0/3216707086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a74109bb0 0x7f7a741a1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.738+0000 7f7a6be00640 1 -- 172.21.15.16:0/3216707086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a74103920 msgr2=0x7f7a7419c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.738+0000 7f7a6be00640 1 --2- 172.21.15.16:0/3216707086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a74103920 0x7f7a7419c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.738+0000 7f7a6be00640 1 -- 172.21.15.16:0/3216707086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a741a2400 con 0x7f7a74109260 2024-08-31T06:05:59.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.738+0000 7f7a6be00640 1 --2- 172.21.15.16:0/3216707086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a74109260 0x7f7a7419ca30 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c005ab0 tx=0x7f7a5c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:59.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.739+0000 7f7a71600640 1 -- 172.21.15.16:0/3216707086 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c03f070 con 0x7f7a74109260 2024-08-31T06:05:59.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.739+0000 7f7a7b200640 1 -- 172.21.15.16:0/3216707086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a741a2690 con 0x7f7a74109260 2024-08-31T06:05:59.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.739+0000 7f7a7b200640 1 -- 172.21.15.16:0/3216707086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a741a8e50 con 0x7f7a74109260 2024-08-31T06:05:59.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.739+0000 7f7a71600640 1 -- 172.21.15.16:0/3216707086 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a5c031b50 con 0x7f7a74109260 2024-08-31T06:05:59.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.739+0000 7f7a71600640 1 -- 172.21.15.16:0/3216707086 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c035030 con 0x7f7a74109260 2024-08-31T06:05:59.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.742+0000 7f7a81e9e740 1 Processor -- start 2024-08-31T06:05:59.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.742+0000 7f7a81e9e740 1 -- start start 2024-08-31T06:05:59.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.743+0000 7f7a71600640 1 -- 172.21.15.16:0/3216707086 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f7a5c042070 con 0x7f7a74109260 2024-08-31T06:05:59.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.743+0000 7f7a81e9e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563aa8bd1f60 0x563aa8f778f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:59.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.743+0000 7f7a81e9e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563aa8ba90a0 0x563aa8f77e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:59.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.743+0000 7f7a81e9e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aa8bb1280 0x563aa8f78370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:59.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.743+0000 7f7a81e9e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563aa8eee550 con 0x563aa8bb1280 2024-08-31T06:05:59.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.743+0000 7f7a81e9e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563aa8eddae0 con 0x563aa8ba90a0 2024-08-31T06:05:59.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.743+0000 7f7a81e9e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563aa8ee1a90 con 0x563aa8bd1f60 2024-08-31T06:05:59.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.743+0000 7f7a6be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aa8bb1280 0x563aa8f78370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:59.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.743+0000 7f7a6be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aa8bb1280 0x563aa8f78370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38058/0 (socket says 172.21.15.16:38058) 2024-08-31T06:05:59.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.743+0000 7f7a79200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563aa8bd1f60 0x563aa8f778f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:59.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.743+0000 7f7a6be00640 1 -- 172.21.15.16:0/4232280203 learned_addr learned my addr 172.21.15.16:0/4232280203 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:05:59.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.744+0000 7f7a6be00640 1 -- 172.21.15.16:0/4232280203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563aa8bd1f60 msgr2=0x563aa8f778f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:05:59.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.744+0000 7f7a6be00640 1 --2- 172.21.15.16:0/4232280203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563aa8bd1f60 0x563aa8f778f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:59.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.744+0000 7f7a6be00640 1 -- 172.21.15.16:0/4232280203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563aa8ba90a0 msgr2=0x563aa8f77e30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T06:05:59.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.744+0000 7f7a6be00640 1 --2- 172.21.15.16:0/4232280203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563aa8ba90a0 0x563aa8f77e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:59.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.744+0000 7f7a6be00640 1 -- 172.21.15.16:0/4232280203 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563aa8c3d540 con 0x563aa8bb1280 2024-08-31T06:05:59.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.744+0000 7f7a6be00640 1 --2- 172.21.15.16:0/4232280203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aa8bb1280 0x563aa8f78370 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c102bb0 tx=0x7f7a5c10d640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2bed483d228856 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:59.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.744+0000 7f7a69600640 1 -- 172.21.15.16:0/4232280203 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c1074e0 con 0x563aa8bb1280 2024-08-31T06:05:59.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.744+0000 7f7a69600640 1 -- 172.21.15.16:0/4232280203 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a5c112030 con 0x563aa8bb1280 2024-08-31T06:05:59.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.744+0000 7f7a69600640 1 -- 172.21.15.16:0/4232280203 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c117920 con 0x563aa8bb1280 2024-08-31T06:05:59.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.745+0000 7f7a71600640 1 --2- 172.21.15.16:0/3216707086 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7a580b4c90 0x7f7a580b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:59.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.745+0000 7f7a71600640 1 -- 172.21.15.16:0/3216707086 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f7a5c105380 con 0x7f7a74109260 2024-08-31T06:05:59.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.745+0000 7f7a79200640 1 --2- 172.21.15.16:0/3216707086 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7a580b4c90 0x7f7a580b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:59.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.745+0000 7f7a81e9e740 1 -- 172.21.15.16:0/4232280203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aa8bb1280 msgr2=0x563aa8f78370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:59.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.745+0000 7f7a81e9e740 1 --2- 172.21.15.16:0/4232280203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aa8bb1280 0x563aa8f78370 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c102bb0 tx=0x7f7a5c10d640 comp rx=0 tx=0).stop 2024-08-31T06:05:59.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.746+0000 7f7a81e9e740 1 -- 172.21.15.16:0/4232280203 shutdown_connections 2024-08-31T06:05:59.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.746+0000 7f7a81e9e740 1 --2- 172.21.15.16:0/4232280203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aa8bb1280 0x563aa8f78370 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:59.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.746+0000 7f7a81e9e740 1 --2- 172.21.15.16:0/4232280203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563aa8ba90a0 0x563aa8f77e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:59.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.746+0000 7f7a81e9e740 1 --2- 172.21.15.16:0/4232280203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563aa8bd1f60 0x563aa8f778f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:59.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.746+0000 7f7a81e9e740 1 -- 172.21.15.16:0/4232280203 >> 172.21.15.16:0/4232280203 conn(0x563aa8bc6350 msgr2=0x563aa8f757e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:59.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.746+0000 7f7a79200640 1 --2- 172.21.15.16:0/3216707086 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7a580b4c90 0x7f7a580b71a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c005fd0 tx=0x7f7a6c009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:59.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.746+0000 7f7a81e9e740 1 -- 172.21.15.16:0/4232280203 shutdown_connections 2024-08-31T06:05:59.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.746+0000 7f7a81e9e740 1 -- 172.21.15.16:0/4232280203 wait complete. 2024-08-31T06:05:59.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.746+0000 7f7a81e9e740 1 -- start start 2024-08-31T06:05:59.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.747+0000 7f7a81e9e740 1 Processor -- start 2024-08-31T06:05:59.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.747+0000 7f7a81e9e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aa8ba90a0 0x563aa8f82f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:59.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.747+0000 7f7a81e9e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563aa8bb1280 0x563aa8f834b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:59.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.747+0000 7f7a81e9e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563aa8bd1f60 0x563aa8f8f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:59.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.747+0000 7f7a81e9e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563aa8d405f0 con 0x563aa8ba90a0 2024-08-31T06:05:59.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.747+0000 7f7a81e9e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563aa8e64bd0 con 0x563aa8bb1280 2024-08-31T06:05:59.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.747+0000 7f7a81e9e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563aa8f85e90 con 0x563aa8bd1f60 2024-08-31T06:05:59.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.747+0000 7f7a81e9e740 10 client.0 _ll_register_callbacks cb 0x563aa8e8a210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:05:59.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.748+0000 7f7a73e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aa8ba90a0 0x563aa8f82f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:59.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.748+0000 7f7a6be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563aa8bb1280 0x563aa8f834b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:59.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.748+0000 7f7a6be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563aa8bb1280 0x563aa8f834b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:57330/0 (socket says 172.21.15.16:57330) 2024-08-31T06:05:59.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.748+0000 7f7a6be00640 1 -- 172.21.15.16:0/2449547994 learned_addr learned my addr 172.21.15.16:0/2449547994 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:05:59.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.748+0000 7f7a73e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aa8ba90a0 0x563aa8f82f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38072/0 (socket says 172.21.15.16:38072) 2024-08-31T06:05:59.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.748+0000 7f7a79200640 1 --2- 172.21.15.16:0/2449547994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563aa8bd1f60 0x563aa8f8f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:59.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.748+0000 7f7a6be00640 1 -- 172.21.15.16:0/2449547994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563aa8bd1f60 msgr2=0x563aa8f8f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:05:59.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.748+0000 7f7a6be00640 1 --2- 172.21.15.16:0/2449547994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563aa8bd1f60 0x563aa8f8f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:59.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.748+0000 7f7a6be00640 1 -- 172.21.15.16:0/2449547994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aa8ba90a0 msgr2=0x563aa8f82f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:05:59.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.748+0000 7f7a6be00640 1 --2- 172.21.15.16:0/2449547994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aa8ba90a0 0x563aa8f82f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:59.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.748+0000 7f7a6be00640 1 -- 172.21.15.16:0/2449547994 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563aa8ea5650 con 0x563aa8bb1280 2024-08-31T06:05:59.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.749+0000 7f7a6be00640 1 --2- 172.21.15.16:0/2449547994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563aa8bb1280 0x563aa8f834b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c0c7cc0 tx=0x7f7a5c02a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2191707920ce6aa server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:59.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.749+0000 7f7a62000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:05:59.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.749+0000 7f7a62000640 1 -- 172.21.15.16:0/2449547994 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c049030 con 0x563aa8bb1280 2024-08-31T06:05:59.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.749+0000 7f7a81e9e740 1 -- 172.21.15.16:0/2449547994 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a5c112030 con 0x563aa8bb1280 2024-08-31T06:05:59.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.749+0000 7f7a62000640 1 -- 172.21.15.16:0/2449547994 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f7a5c102ec0 con 0x563aa8bb1280 2024-08-31T06:05:59.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.749+0000 7f7a62000640 1 -- 172.21.15.16:0/2449547994 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7a5c0c7b00 con 0x563aa8bb1280 2024-08-31T06:05:59.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.749+0000 7f7a81e9e740 10 client.25076 fetch_fsmap learned FSMap version 24 2024-08-31T06:05:59.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.749+0000 7f7a81e9e740 1 -- 172.21.15.16:0/2449547994 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563aa8f91b40 con 0x563aa8bb1280 2024-08-31T06:05:59.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.749+0000 7f7a62000640 1 -- 172.21.15.16:0/2449547994 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a5c112030 con 0x563aa8bb1280 2024-08-31T06:05:59.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.750+0000 7f7a62000640 1 -- 172.21.15.16:0/2449547994 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f7a5c04c060 con 0x563aa8bb1280 2024-08-31T06:05:59.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.750+0000 7f7a81e9e740 10 client.25076 fetch_fsmap finished waiting for FSMap version 24 2024-08-31T06:05:59.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.750+0000 7f7a81e9e740 10 client.25076 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-31T06:05:59.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.750+0000 7f7a81e9e740 20 client.25076 populate_metadata read hostname 'smithi016' 2024-08-31T06:05:59.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.750+0000 7f7a81e9e740 1 --2- 172.21.15.16:0/2449547994 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x563aa8b908e0 0x563aa8f953b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:59.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.750+0000 7f7a81e9e740 4 client.25076 mds_command: new command op to 24712 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:05:59.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.750+0000 7f7a81e9e740 1 -- 172.21.15.16:0/2449547994 --> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563aa8ef04a0 con 0x563aa8b908e0 2024-08-31T06:05:59.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.750+0000 7f7a73e00640 1 --2- 172.21.15.16:0/2449547994 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x563aa8b908e0 0x563aa8f953b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:59.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.751+0000 7f7a73e00640 1 --2- 172.21.15.16:0/2449547994 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x563aa8b908e0 0x563aa8f953b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f2585c3944c0633 server_cookie=5a9b2fee1a6449fb in_seq=0 out_seq=0 2024-08-31T06:05:59.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.751+0000 7f7a62000640 10 client.25076 ms_handle_connect on v2:172.21.15.184:6838/4011051479 2024-08-31T06:05:59.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.751+0000 7f7a62000640 1 -- 172.21.15.16:0/2449547994 <== mds.2 v2:172.21.15.184:6838/4011051479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563aa8ef04a0 con 0x563aa8b908e0 2024-08-31T06:05:59.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.751+0000 7f7a62000640 10 client.25076 handle_command_reply: tid=0 2024-08-31T06:05:59.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.752+0000 7f7a81e9e740 1 client.25076 shutdown 2024-08-31T06:05:59.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.752+0000 7f7a81e9e740 1 -- 172.21.15.16:0/2449547994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563aa8bb1280 msgr2=0x563aa8f834b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:59.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.752+0000 7f7a81e9e740 1 --2- 172.21.15.16:0/2449547994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563aa8bb1280 0x563aa8f834b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c0c7cc0 tx=0x7f7a5c02a3f0 comp rx=0 tx=0).stop 2024-08-31T06:05:59.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.752+0000 7f7a81e9e740 1 -- 172.21.15.16:0/2449547994 shutdown_connections 2024-08-31T06:05:59.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.752+0000 7f7a81e9e740 1 --2- 172.21.15.16:0/2449547994 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x563aa8b908e0 0x563aa8f953b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:59.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.752+0000 7f7a81e9e740 1 --2- 172.21.15.16:0/2449547994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563aa8bd1f60 0x563aa8f8f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:59.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.752+0000 7f7a81e9e740 1 --2- 172.21.15.16:0/2449547994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563aa8bb1280 0x563aa8f834b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:59.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.752+0000 7f7a62000640 0 client.25076 ms_handle_reset on v2:172.21.15.184:6838/4011051479 2024-08-31T06:05:59.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.752+0000 7f7a81e9e740 1 --2- 172.21.15.16:0/2449547994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aa8ba90a0 0x563aa8f82f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:59.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.752+0000 7f7a81e9e740 1 -- 172.21.15.16:0/2449547994 >> 172.21.15.16:0/2449547994 conn(0x563aa8bc6350 msgr2=0x563aa8f7aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:59.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.752+0000 7f7a81e9e740 1 -- 172.21.15.16:0/2449547994 shutdown_connections 2024-08-31T06:05:59.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.752+0000 7f7a81e9e740 1 -- 172.21.15.16:0/2449547994 wait complete. 2024-08-31T06:05:59.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.752+0000 7f7a81e9e740 20 client.25076 trim_cache size 0 max 16384 2024-08-31T06:05:59.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.771+0000 7f7a81e9e740 1 Processor -- start 2024-08-31T06:05:59.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.772+0000 7f7a81e9e740 1 -- start start 2024-08-31T06:05:59.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.772+0000 7f7a81e9e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563aa8ba90a0 0x563aa8f83430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:59.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.772+0000 7f7a81e9e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aa8bb1280 0x563aa8f8f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:59.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.772+0000 7f7a81e9e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563aa8bd1f60 0x563aa8f74db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:59.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.772+0000 7f7a81e9e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563aa8f936f0 con 0x563aa8bb1280 2024-08-31T06:05:59.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.772+0000 7f7a81e9e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563aa8f93dc0 con 0x563aa8bd1f60 2024-08-31T06:05:59.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.772+0000 7f7a81e9e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563aa8f94490 con 0x563aa8ba90a0 2024-08-31T06:05:59.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.772+0000 7f7a6be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aa8bb1280 0x563aa8f8f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:59.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.772+0000 7f7a6be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aa8bb1280 0x563aa8f8f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38076/0 (socket says 172.21.15.16:38076) 2024-08-31T06:05:59.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.772+0000 7f7a6be00640 1 -- 172.21.15.16:0/4089855746 learned_addr learned my addr 172.21.15.16:0/4089855746 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:05:59.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.773+0000 7f7a73e00640 1 --2- 172.21.15.16:0/4089855746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563aa8ba90a0 0x563aa8f83430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:59.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.773+0000 7f7a79200640 1 --2- 172.21.15.16:0/4089855746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563aa8bd1f60 0x563aa8f74db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:59.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.773+0000 7f7a6be00640 1 -- 172.21.15.16:0/4089855746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563aa8ba90a0 msgr2=0x563aa8f83430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:05:59.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.773+0000 7f7a6be00640 1 --2- 172.21.15.16:0/4089855746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563aa8ba90a0 0x563aa8f83430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:59.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.773+0000 7f7a6be00640 1 -- 172.21.15.16:0/4089855746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563aa8bd1f60 msgr2=0x563aa8f74db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:05:59.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.773+0000 7f7a6be00640 1 --2- 172.21.15.16:0/4089855746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563aa8bd1f60 0x563aa8f74db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:59.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.773+0000 7f7a6be00640 1 -- 172.21.15.16:0/4089855746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563aa8f84010 con 0x563aa8bb1280 2024-08-31T06:05:59.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.773+0000 7f7a73e00640 1 --2- 172.21.15.16:0/4089855746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563aa8ba90a0 0x563aa8f83430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:05:59.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.773+0000 7f7a6be00640 1 --2- 172.21.15.16:0/4089855746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aa8bb1280 0x563aa8f8f0f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c031440 tx=0x7f7a5c003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=178664169210cbc9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:59.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.773+0000 7f7a68c00640 1 -- 172.21.15.16:0/4089855746 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c049020 con 0x563aa8bb1280 2024-08-31T06:05:59.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.774+0000 7f7a68c00640 1 -- 172.21.15.16:0/4089855746 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a5c112070 con 0x563aa8bb1280 2024-08-31T06:05:59.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.774+0000 7f7a68c00640 1 -- 172.21.15.16:0/4089855746 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c0c73d0 con 0x563aa8bb1280 2024-08-31T06:05:59.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.774+0000 7f7a81e9e740 1 -- 172.21.15.16:0/4089855746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aa8bb1280 msgr2=0x563aa8f8f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:59.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.774+0000 7f7a81e9e740 1 --2- 172.21.15.16:0/4089855746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aa8bb1280 0x563aa8f8f0f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c031440 tx=0x7f7a5c003f90 comp rx=0 tx=0).stop 2024-08-31T06:05:59.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.775+0000 7f7a81e9e740 1 -- 172.21.15.16:0/4089855746 shutdown_connections 2024-08-31T06:05:59.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.775+0000 7f7a81e9e740 1 --2- 172.21.15.16:0/4089855746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563aa8bd1f60 0x563aa8f74db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:59.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.775+0000 7f7a81e9e740 1 --2- 172.21.15.16:0/4089855746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aa8bb1280 0x563aa8f8f0f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:59.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.775+0000 7f7a81e9e740 1 --2- 172.21.15.16:0/4089855746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563aa8ba90a0 0x563aa8f83430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:59.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.775+0000 7f7a81e9e740 1 -- 172.21.15.16:0/4089855746 >> 172.21.15.16:0/4089855746 conn(0x563aa8bc6350 msgr2=0x563aa8f82d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:59.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.775+0000 7f7a81e9e740 1 -- 172.21.15.16:0/4089855746 shutdown_connections 2024-08-31T06:05:59.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.775+0000 7f7a81e9e740 1 -- 172.21.15.16:0/4089855746 wait complete. 2024-08-31T06:05:59.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.775+0000 7f7a81e9e740 1 -- start start 2024-08-31T06:05:59.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.776+0000 7f7a81e9e740 1 Processor -- start 2024-08-31T06:05:59.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.776+0000 7f7a81e9e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563aa8ba90a0 0x563aa8f9dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:59.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.776+0000 7f7a81e9e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aa8bb1280 0x563aa8f97460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:59.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.777+0000 7f7a81e9e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563aa8bd1f60 0x563aa8f97b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:59.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.777+0000 7f7a81e9e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563aa8f95230 con 0x563aa8bb1280 2024-08-31T06:05:59.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.777+0000 7f7a81e9e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563aa8f94b60 con 0x563aa8bd1f60 2024-08-31T06:05:59.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.777+0000 7f7a81e9e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563aa8f95900 con 0x563aa8ba90a0 2024-08-31T06:05:59.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.777+0000 7f7a81e9e740 10 client.0 _ll_register_callbacks cb 0x7f7a5c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:05:59.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.777+0000 7f7a6be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aa8bb1280 0x563aa8f97460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:59.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.777+0000 7f7a6be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aa8bb1280 0x563aa8f97460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38078/0 (socket says 172.21.15.16:38078) 2024-08-31T06:05:59.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.777+0000 7f7a6be00640 1 -- 172.21.15.16:0/2906529233 learned_addr learned my addr 172.21.15.16:0/2906529233 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:05:59.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.777+0000 7f7a73e00640 1 --2- 172.21.15.16:0/2906529233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563aa8ba90a0 0x563aa8f9dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:59.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.777+0000 7f7a79200640 1 --2- 172.21.15.16:0/2906529233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563aa8bd1f60 0x563aa8f97b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:59.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.777+0000 7f7a6be00640 1 -- 172.21.15.16:0/2906529233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563aa8ba90a0 msgr2=0x563aa8f9dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:05:59.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.777+0000 7f7a6be00640 1 --2- 172.21.15.16:0/2906529233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563aa8ba90a0 0x563aa8f9dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:59.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.777+0000 7f7a6be00640 1 -- 172.21.15.16:0/2906529233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563aa8bd1f60 msgr2=0x563aa8f97b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:05:59.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.777+0000 7f7a6be00640 1 --2- 172.21.15.16:0/2906529233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563aa8bd1f60 0x563aa8f97b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:59.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.777+0000 7f7a6be00640 1 -- 172.21.15.16:0/2906529233 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563aa8f9cbf0 con 0x563aa8bb1280 2024-08-31T06:05:59.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.778+0000 7f7a6be00640 1 --2- 172.21.15.16:0/2906529233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aa8bb1280 0x563aa8f97460 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c1031f0 tx=0x7f7a5c005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3678151b31acadc server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:05:59.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.778+0000 7f7a62000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:05:59.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.778+0000 7f7a62000640 1 -- 172.21.15.16:0/2906529233 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c049030 con 0x563aa8bb1280 2024-08-31T06:05:59.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.778+0000 7f7a62000640 1 -- 172.21.15.16:0/2906529233 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f7a5c0c7ed0 con 0x563aa8bb1280 2024-08-31T06:05:59.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.778+0000 7f7a81e9e740 1 -- 172.21.15.16:0/2906529233 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a5c112070 con 0x563aa8bb1280 2024-08-31T06:05:59.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.778+0000 7f7a62000640 1 -- 172.21.15.16:0/2906529233 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7a5c0cb3b0 con 0x563aa8bb1280 2024-08-31T06:05:59.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.778+0000 7f7a81e9e740 10 client.15078 fetch_fsmap learned FSMap version 24 2024-08-31T06:05:59.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.778+0000 7f7a81e9e740 1 -- 172.21.15.16:0/2906529233 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563aa8f9cbf0 con 0x563aa8bb1280 2024-08-31T06:05:59.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.779+0000 7f7a62000640 1 -- 172.21.15.16:0/2906529233 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a5c112070 con 0x563aa8bb1280 2024-08-31T06:05:59.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.779+0000 7f7a62000640 1 -- 172.21.15.16:0/2906529233 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f7a5c03b460 con 0x563aa8bb1280 2024-08-31T06:05:59.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.779+0000 7f7a81e9e740 10 client.15078 fetch_fsmap finished waiting for FSMap version 24 2024-08-31T06:05:59.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.779+0000 7f7a81e9e740 10 client.15078 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-31T06:05:59.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.779+0000 7f7a81e9e740 20 client.15078 populate_metadata read hostname 'smithi016' 2024-08-31T06:05:59.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.779+0000 7f7a81e9e740 1 --2- 172.21.15.16:0/2906529233 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x563aa8b65c90 0x563aa9003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:05:59.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.779+0000 7f7a81e9e740 4 client.15078 mds_command: new command op to 24712 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-31T06:05:59.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.779+0000 7f7a81e9e740 1 -- 172.21.15.16:0/2906529233 --> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] -- command(tid 0: {"prefix": "status"}) -- 0x563aa8eebf40 con 0x563aa8b65c90 2024-08-31T06:05:59.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.779+0000 7f7a73e00640 1 --2- 172.21.15.16:0/2906529233 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x563aa8b65c90 0x563aa9003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:05:59.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.780+0000 7f7a73e00640 1 --2- 172.21.15.16:0/2906529233 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x563aa8b65c90 0x563aa9003ba0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e5ad61621cf74bc server_cookie=8b232105b20fb65a in_seq=0 out_seq=0 2024-08-31T06:05:59.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.780+0000 7f7a62000640 10 client.15078 ms_handle_connect on v2:172.21.15.184:6838/4011051479 2024-08-31T06:05:59.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.780+0000 7f7a62000640 1 -- 172.21.15.16:0/2906529233 <== mds.2 v2:172.21.15.184:6838/4011051479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x563aa8eebf40 con 0x563aa8b65c90 2024-08-31T06:05:59.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.780+0000 7f7a62000640 10 client.15078 handle_command_reply: tid=0 2024-08-31T06:05:59.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.780+0000 7f7a81e9e740 1 client.15078 shutdown 2024-08-31T06:05:59.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.780+0000 7f7a81e9e740 1 -- 172.21.15.16:0/2906529233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aa8bb1280 msgr2=0x563aa8f97460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:59.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.780+0000 7f7a81e9e740 1 --2- 172.21.15.16:0/2906529233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aa8bb1280 0x563aa8f97460 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c1031f0 tx=0x7f7a5c005dc0 comp rx=0 tx=0).stop 2024-08-31T06:05:59.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.781+0000 7f7a81e9e740 1 -- 172.21.15.16:0/2906529233 shutdown_connections 2024-08-31T06:05:59.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.781+0000 7f7a81e9e740 1 --2- 172.21.15.16:0/2906529233 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x563aa8b65c90 0x563aa9003ba0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:59.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.781+0000 7f7a81e9e740 1 --2- 172.21.15.16:0/2906529233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563aa8bd1f60 0x563aa8f97b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:59.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.781+0000 7f7a81e9e740 1 --2- 172.21.15.16:0/2906529233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aa8bb1280 0x563aa8f97460 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:59.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.781+0000 7f7a81e9e740 1 --2- 172.21.15.16:0/2906529233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563aa8ba90a0 0x563aa8f9dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:59.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.781+0000 7f7a81e9e740 1 -- 172.21.15.16:0/2906529233 >> 172.21.15.16:0/2906529233 conn(0x563aa8bc6350 msgr2=0x563aa8f7a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:59.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.781+0000 7f7a62000640 0 client.15078 ms_handle_reset on v2:172.21.15.184:6838/4011051479 2024-08-31T06:05:59.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.781+0000 7f7a81e9e740 1 -- 172.21.15.16:0/2906529233 shutdown_connections 2024-08-31T06:05:59.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.781+0000 7f7a81e9e740 1 -- 172.21.15.16:0/2906529233 wait complete. 2024-08-31T06:05:59.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.781+0000 7f7a81e9e740 20 client.15078 trim_cache size 0 max 16384 2024-08-31T06:05:59.785 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:05:59.785 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "673e4aa0-675d-11ef-bcd4-c7b262605968", 2024-08-31T06:05:59.785 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 2, 2024-08-31T06:05:59.785 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24712, 2024-08-31T06:05:59.785 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-08-31T06:05:59.785 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-08-31T06:05:59.785 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-08-31T06:05:59.785 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 80.849233029999994, 2024-08-31T06:05:59.785 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 24, 2024-08-31T06:05:59.785 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 80, 2024-08-31T06:05:59.786 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 78, 2024-08-31T06:05:59.786 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 117.07422258 2024-08-31T06:05:59.786 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:05:59.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.782+0000 7f7a7b200640 1 -- 172.21.15.16:0/3216707086 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7a580b4c90 msgr2=0x7f7a580b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:59.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.782+0000 7f7a7b200640 1 --2- 172.21.15.16:0/3216707086 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7a580b4c90 0x7f7a580b71a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c005fd0 tx=0x7f7a6c009500 comp rx=0 tx=0).stop 2024-08-31T06:05:59.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.782+0000 7f7a7b200640 1 -- 172.21.15.16:0/3216707086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a74109260 msgr2=0x7f7a7419ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:05:59.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.782+0000 7f7a7b200640 1 --2- 172.21.15.16:0/3216707086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a74109260 0x7f7a7419ca30 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c005ab0 tx=0x7f7a5c003b90 comp rx=0 tx=0).stop 2024-08-31T06:05:59.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.782+0000 7f7a7b200640 1 -- 172.21.15.16:0/3216707086 shutdown_connections 2024-08-31T06:05:59.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.782+0000 7f7a7b200640 1 --2- 172.21.15.16:0/3216707086 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7a580b4c90 0x7f7a580b71a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:59.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.782+0000 7f7a7b200640 1 --2- 172.21.15.16:0/3216707086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a74109bb0 0x7f7a741a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:59.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.782+0000 7f7a7b200640 1 --2- 172.21.15.16:0/3216707086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a74109260 0x7f7a7419ca30 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:59.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.782+0000 7f7a7b200640 1 --2- 172.21.15.16:0/3216707086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a74103920 0x7f7a7419c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:05:59.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.782+0000 7f7a7b200640 1 -- 172.21.15.16:0/3216707086 >> 172.21.15.16:0/3216707086 conn(0x7f7a740ff330 msgr2=0x7f7a74108890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:05:59.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.782+0000 7f7a7b200640 1 -- 172.21.15.16:0/3216707086 shutdown_connections 2024-08-31T06:05:59.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:05:59.783+0000 7f7a7b200640 1 -- 172.21.15.16:0/3216707086 wait complete. 2024-08-31T06:06:00.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:00 smithi155 ceph-mon[21742]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-31T06:06:00.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:00 smithi016 ceph-mon[17194]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-31T06:06:00.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:00 smithi184 ceph-mon[22331]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-31T06:06:00.788 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.g status 2024-08-31T06:06:01.183 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:06:01.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.813+0000 7fb5d8c00640 1 -- 172.21.15.16:0/1601008639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5d4104570 msgr2=0x7fb5d4104980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:01.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.813+0000 7fb5d8c00640 1 --2- 172.21.15.16:0/1601008639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5d4104570 0x7fb5d4104980 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4009900 tx=0x7fb5c402a3f0 comp rx=0 tx=0).stop 2024-08-31T06:06:01.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.813+0000 7fb5d8c00640 1 -- 172.21.15.16:0/1601008639 shutdown_connections 2024-08-31T06:06:01.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.813+0000 7fb5d8c00640 1 --2- 172.21.15.16:0/1601008639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5d4104ec0 0x7fb5d410d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:01.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.813+0000 7fb5d8c00640 1 --2- 172.21.15.16:0/1601008639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5d410d9d0 0x7fb5d410fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:01.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.813+0000 7fb5d8c00640 1 --2- 172.21.15.16:0/1601008639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5d4104570 0x7fb5d4104980 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:01.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.813+0000 7fb5d8c00640 1 -- 172.21.15.16:0/1601008639 >> 172.21.15.16:0/1601008639 conn(0x7fb5d40fe070 msgr2=0x7fb5d41004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:01.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.813+0000 7fb5d8c00640 1 -- 172.21.15.16:0/1601008639 shutdown_connections 2024-08-31T06:06:01.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.813+0000 7fb5d8c00640 1 -- 172.21.15.16:0/1601008639 wait complete. 2024-08-31T06:06:01.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.814+0000 7fb5d8c00640 1 Processor -- start 2024-08-31T06:06:01.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.814+0000 7fb5d8c00640 1 -- start start 2024-08-31T06:06:01.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.815+0000 7fb5d8c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5d4104570 0x7fb5d419c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:01.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.815+0000 7fb5d8c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5d4104ec0 0x7fb5d419c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:01.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.815+0000 7fb5d8c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5d410d9d0 0x7fb5d41a1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:01.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.815+0000 7fb5d8c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d4113690 con 0x7fb5d4104570 2024-08-31T06:06:01.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.815+0000 7fb5d8c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d4113510 con 0x7fb5d4104ec0 2024-08-31T06:06:01.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.815+0000 7fb5d8c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d4113810 con 0x7fb5d410d9d0 2024-08-31T06:06:01.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.815+0000 7fb5d3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5d4104570 0x7fb5d419c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:01.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.815+0000 7fb5d2a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5d4104ec0 0x7fb5d419c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:01.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.815+0000 7fb5d3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5d4104570 0x7fb5d419c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38088/0 (socket says 172.21.15.16:38088) 2024-08-31T06:06:01.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.815+0000 7fb5d3400640 1 -- 172.21.15.16:0/4144970141 learned_addr learned my addr 172.21.15.16:0/4144970141 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:01.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.815+0000 7fb5d3e00640 1 --2- 172.21.15.16:0/4144970141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5d410d9d0 0x7fb5d41a1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:01.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.815+0000 7fb5d2a00640 1 -- 172.21.15.16:0/4144970141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5d410d9d0 msgr2=0x7fb5d41a1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:01.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.815+0000 7fb5d2a00640 1 --2- 172.21.15.16:0/4144970141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5d410d9d0 0x7fb5d41a1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:01.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.815+0000 7fb5d2a00640 1 -- 172.21.15.16:0/4144970141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5d4104570 msgr2=0x7fb5d419c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:01.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.815+0000 7fb5d2a00640 1 --2- 172.21.15.16:0/4144970141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5d4104570 0x7fb5d419c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:01.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.815+0000 7fb5d2a00640 1 -- 172.21.15.16:0/4144970141 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5d41a2290 con 0x7fb5d4104ec0 2024-08-31T06:06:01.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.815+0000 7fb5d3400640 1 --2- 172.21.15.16:0/4144970141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5d4104570 0x7fb5d419c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:06:01.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.816+0000 7fb5d2a00640 1 --2- 172.21.15.16:0/4144970141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5d4104ec0 0x7fb5d419c8c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb5d406b240 tx=0x7fb5c000db30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:01.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.816+0000 7fb5cbe00640 1 -- 172.21.15.16:0/4144970141 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c0017070 con 0x7fb5d4104ec0 2024-08-31T06:06:01.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.816+0000 7fb5d8c00640 1 -- 172.21.15.16:0/4144970141 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5d41a2580 con 0x7fb5d4104ec0 2024-08-31T06:06:01.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.816+0000 7fb5cbe00640 1 -- 172.21.15.16:0/4144970141 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5c0009820 con 0x7fb5d4104ec0 2024-08-31T06:06:01.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.816+0000 7fb5cbe00640 1 -- 172.21.15.16:0/4144970141 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c0004ba0 con 0x7fb5d4104ec0 2024-08-31T06:06:01.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.816+0000 7fb5d8c00640 1 -- 172.21.15.16:0/4144970141 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5d41a8ce0 con 0x7fb5d4104ec0 2024-08-31T06:06:01.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.819+0000 7fb5e0eb5740 1 Processor -- start 2024-08-31T06:06:01.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.819+0000 7fb5e0eb5740 1 -- start start 2024-08-31T06:06:01.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.820+0000 7fb5e0eb5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c50af8af60 0x55c50b330aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:01.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.820+0000 7fb5e0eb5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c50af620a0 0x55c50b330fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:01.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.820+0000 7fb5e0eb5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c50af6a280 0x55c50b331520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:01.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.820+0000 7fb5e0eb5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c50b2a7550 con 0x55c50af620a0 2024-08-31T06:06:01.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.820+0000 7fb5e0eb5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c50b296ae0 con 0x55c50af8af60 2024-08-31T06:06:01.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.820+0000 7fb5e0eb5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c50b29aa90 con 0x55c50af6a280 2024-08-31T06:06:01.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.820+0000 7fb5d3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c50af620a0 0x55c50b330fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:01.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.820+0000 7fb5d3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c50af620a0 0x55c50b330fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38090/0 (socket says 172.21.15.16:38090) 2024-08-31T06:06:01.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.820+0000 7fb5d3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c50af8af60 0x55c50b330aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:01.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.820+0000 7fb5d3400640 1 -- 172.21.15.16:0/3103379570 learned_addr learned my addr 172.21.15.16:0/3103379570 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:01.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.820+0000 7fb5cbe00640 1 -- 172.21.15.16:0/4144970141 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7fb5c0009c60 con 0x7fb5d4104ec0 2024-08-31T06:06:01.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.821+0000 7fb5d3e00640 1 -- 172.21.15.16:0/3103379570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c50af6a280 msgr2=0x55c50b331520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:01.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.821+0000 7fb5d3e00640 1 --2- 172.21.15.16:0/3103379570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c50af6a280 0x55c50b331520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:01.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.821+0000 7fb5d3e00640 1 -- 172.21.15.16:0/3103379570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c50af620a0 msgr2=0x55c50b330fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:01.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.821+0000 7fb5d3e00640 1 --2- 172.21.15.16:0/3103379570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c50af620a0 0x55c50b330fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:01.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.821+0000 7fb5d3e00640 1 -- 172.21.15.16:0/3103379570 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c50aff6540 con 0x55c50af8af60 2024-08-31T06:06:01.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.821+0000 7fb5d3e00640 1 --2- 172.21.15.16:0/3103379570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c50af8af60 0x55c50b330aa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc009f00 tx=0x7fb5cc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c926b9657613e6a9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:01.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.821+0000 7fb5bfe00640 1 -- 172.21.15.16:0/3103379570 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc004d20 con 0x55c50af8af60 2024-08-31T06:06:01.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.821+0000 7fb5bfe00640 1 -- 172.21.15.16:0/3103379570 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5cc004050 con 0x55c50af8af60 2024-08-31T06:06:01.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.821+0000 7fb5bfe00640 1 -- 172.21.15.16:0/3103379570 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc005280 con 0x55c50af8af60 2024-08-31T06:06:01.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.822+0000 7fb5cbe00640 1 --2- 172.21.15.16:0/4144970141 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb5b80b4c10 0x7fb5b80b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:01.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.822+0000 7fb5e0eb5740 1 -- 172.21.15.16:0/3103379570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c50af8af60 msgr2=0x55c50b330aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:01.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.822+0000 7fb5e0eb5740 1 --2- 172.21.15.16:0/3103379570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c50af8af60 0x55c50b330aa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc009f00 tx=0x7fb5cc011810 comp rx=0 tx=0).stop 2024-08-31T06:06:01.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.822+0000 7fb5cbe00640 1 -- 172.21.15.16:0/4144970141 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7fb5c00e4cd0 con 0x7fb5d4104ec0 2024-08-31T06:06:01.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.822+0000 7fb5d3400640 1 --2- 172.21.15.16:0/4144970141 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb5b80b4c10 0x7fb5b80b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:01.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.822+0000 7fb5e0eb5740 1 -- 172.21.15.16:0/3103379570 shutdown_connections 2024-08-31T06:06:01.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.822+0000 7fb5e0eb5740 1 --2- 172.21.15.16:0/3103379570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c50af6a280 0x55c50b331520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:01.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.822+0000 7fb5e0eb5740 1 --2- 172.21.15.16:0/3103379570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c50af620a0 0x55c50b330fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:01.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.822+0000 7fb5e0eb5740 1 --2- 172.21.15.16:0/3103379570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c50af8af60 0x55c50b330aa0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:01.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.822+0000 7fb5e0eb5740 1 -- 172.21.15.16:0/3103379570 >> 172.21.15.16:0/3103379570 conn(0x55c50af7f350 msgr2=0x55c50b32e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:01.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.823+0000 7fb5e0eb5740 1 -- 172.21.15.16:0/3103379570 shutdown_connections 2024-08-31T06:06:01.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.823+0000 7fb5e0eb5740 1 -- 172.21.15.16:0/3103379570 wait complete. 2024-08-31T06:06:01.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.823+0000 7fb5d3400640 1 --2- 172.21.15.16:0/4144970141 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb5b80b4c10 0x7fb5b80b7120 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb5c40096c0 tx=0x7fb5c4005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:01.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.823+0000 7fb5e0eb5740 1 -- start start 2024-08-31T06:06:01.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.824+0000 7fb5e0eb5740 1 Processor -- start 2024-08-31T06:06:01.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.824+0000 7fb5e0eb5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c50af620a0 0x55c50b33c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:01.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.824+0000 7fb5e0eb5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c50af6a280 0x55c50b33c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:01.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.824+0000 7fb5e0eb5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c50af8af60 0x55c50b3485d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:01.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.824+0000 7fb5e0eb5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c50b0f95f0 con 0x55c50af8af60 2024-08-31T06:06:01.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.824+0000 7fb5e0eb5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c50b21dbd0 con 0x55c50af6a280 2024-08-31T06:06:01.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.824+0000 7fb5e0eb5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c50b33f280 con 0x55c50af620a0 2024-08-31T06:06:01.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.824+0000 7fb5e0eb5740 10 client.0 _ll_register_callbacks cb 0x55c50b243210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:06:01.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.824+0000 7fb5d2a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c50af6a280 0x55c50b33c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:01.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.825+0000 7fb5d3400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c50af620a0 0x55c50b33c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:01.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.825+0000 7fb5d3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c50af8af60 0x55c50b3485d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:01.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.825+0000 7fb5d2a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c50af6a280 0x55c50b33c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:57380/0 (socket says 172.21.15.16:57380) 2024-08-31T06:06:01.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.825+0000 7fb5d2a00640 1 -- 172.21.15.16:0/1323961693 learned_addr learned my addr 172.21.15.16:0/1323961693 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:01.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.825+0000 7fb5d3400640 1 --2- 172.21.15.16:0/1323961693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c50af620a0 0x55c50b33c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:52682/0 (socket says 172.21.15.16:52682) 2024-08-31T06:06:01.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.825+0000 7fb5d2a00640 1 -- 172.21.15.16:0/1323961693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c50af620a0 msgr2=0x55c50b33c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:01.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.825+0000 7fb5d2a00640 1 --2- 172.21.15.16:0/1323961693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c50af620a0 0x55c50b33c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:01.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.825+0000 7fb5d2a00640 1 -- 172.21.15.16:0/1323961693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c50af8af60 msgr2=0x55c50b3485d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:01.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.825+0000 7fb5d2a00640 1 --2- 172.21.15.16:0/1323961693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c50af8af60 0x55c50b3485d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:01.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.825+0000 7fb5d2a00640 1 -- 172.21.15.16:0/1323961693 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c50b25e650 con 0x55c50af6a280 2024-08-31T06:06:01.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.825+0000 7fb5d2a00640 1 --2- 172.21.15.16:0/1323961693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c50af6a280 0x55c50b33c8c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb5c00ab4c0 tx=0x7fb5c00aadd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d10a80924cd788c3 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:01.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.825+0000 7fb5bcc00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:06:01.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.825+0000 7fb5bcc00640 1 -- 172.21.15.16:0/1323961693 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c0020040 con 0x55c50af6a280 2024-08-31T06:06:01.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.826+0000 7fb5e0eb5740 1 -- 172.21.15.16:0/1323961693 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5cc004050 con 0x55c50af6a280 2024-08-31T06:06:01.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.826+0000 7fb5bcc00640 1 -- 172.21.15.16:0/1323961693 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7fb5c00ce070 con 0x55c50af6a280 2024-08-31T06:06:01.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.826+0000 7fb5bcc00640 1 -- 172.21.15.16:0/1323961693 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb5c00ce270 con 0x55c50af6a280 2024-08-31T06:06:01.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.826+0000 7fb5e0eb5740 10 client.25094 fetch_fsmap learned FSMap version 24 2024-08-31T06:06:01.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.826+0000 7fb5e0eb5740 1 -- 172.21.15.16:0/1323961693 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c50b34b170 con 0x55c50af6a280 2024-08-31T06:06:01.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.826+0000 7fb5bcc00640 1 -- 172.21.15.16:0/1323961693 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5cc004050 con 0x55c50af6a280 2024-08-31T06:06:01.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.826+0000 7fb5bcc00640 1 -- 172.21.15.16:0/1323961693 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fb5c00e2b70 con 0x55c50af6a280 2024-08-31T06:06:01.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.826+0000 7fb5e0eb5740 10 client.25094 fetch_fsmap finished waiting for FSMap version 24 2024-08-31T06:06:01.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.826+0000 7fb5e0eb5740 10 client.25094 resolve_mds: resolved name 'g' to daemon mds.g 2024-08-31T06:06:01.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.826+0000 7fb5e0eb5740 20 client.25094 populate_metadata read hostname 'smithi016' 2024-08-31T06:06:01.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.826+0000 7fb5e0eb5740 1 --2- 172.21.15.16:0/1323961693 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55c50af498e0 0x55c50b34ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:01.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.827+0000 7fb5e0eb5740 4 client.25094 mds_command: new command op to 24724 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:06:01.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.827+0000 7fb5e0eb5740 1 -- 172.21.15.16:0/1323961693 --> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c50b2a94a0 con 0x55c50af498e0 2024-08-31T06:06:01.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.827+0000 7fb5d3400640 1 --2- 172.21.15.16:0/1323961693 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55c50af498e0 0x55c50b34ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:01.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.827+0000 7fb5d3400640 1 --2- 172.21.15.16:0/1323961693 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55c50af498e0 0x55c50b34ec30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3ffdfada76c91799 server_cookie=8419017cfaffb82d in_seq=0 out_seq=0 2024-08-31T06:06:01.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.827+0000 7fb5bcc00640 10 client.25094 ms_handle_connect on v2:172.21.15.16:6838/219664460 2024-08-31T06:06:01.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.828+0000 7fb5bcc00640 1 -- 172.21.15.16:0/1323961693 <== mds.1 v2:172.21.15.16:6838/219664460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c50b2a94a0 con 0x55c50af498e0 2024-08-31T06:06:01.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.828+0000 7fb5bcc00640 10 client.25094 handle_command_reply: tid=0 2024-08-31T06:06:01.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.828+0000 7fb5e0eb5740 1 client.25094 shutdown 2024-08-31T06:06:01.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.829+0000 7fb5e0eb5740 1 -- 172.21.15.16:0/1323961693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c50af6a280 msgr2=0x55c50b33c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:01.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.829+0000 7fb5e0eb5740 1 --2- 172.21.15.16:0/1323961693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c50af6a280 0x55c50b33c8c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb5c00ab4c0 tx=0x7fb5c00aadd0 comp rx=0 tx=0).stop 2024-08-31T06:06:01.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.829+0000 7fb5e0eb5740 1 -- 172.21.15.16:0/1323961693 shutdown_connections 2024-08-31T06:06:01.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.829+0000 7fb5e0eb5740 1 --2- 172.21.15.16:0/1323961693 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55c50af498e0 0x55c50b34ec30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:01.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.829+0000 7fb5e0eb5740 1 --2- 172.21.15.16:0/1323961693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c50af8af60 0x55c50b3485d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:01.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.829+0000 7fb5e0eb5740 1 --2- 172.21.15.16:0/1323961693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c50af6a280 0x55c50b33c8c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:01.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.829+0000 7fb5bcc00640 0 client.25094 ms_handle_reset on v2:172.21.15.16:6838/219664460 2024-08-31T06:06:01.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.829+0000 7fb5e0eb5740 1 --2- 172.21.15.16:0/1323961693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c50af620a0 0x55c50b33c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:01.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.829+0000 7fb5e0eb5740 1 -- 172.21.15.16:0/1323961693 >> 172.21.15.16:0/1323961693 conn(0x55c50af7f350 msgr2=0x55c50b333c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:01.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.829+0000 7fb5e0eb5740 1 -- 172.21.15.16:0/1323961693 shutdown_connections 2024-08-31T06:06:01.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.829+0000 7fb5e0eb5740 1 -- 172.21.15.16:0/1323961693 wait complete. 2024-08-31T06:06:01.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.829+0000 7fb5e0eb5740 20 client.25094 trim_cache size 0 max 16384 2024-08-31T06:06:01.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.848+0000 7fb5e0eb5740 1 Processor -- start 2024-08-31T06:06:01.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.848+0000 7fb5e0eb5740 1 -- start start 2024-08-31T06:06:01.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.848+0000 7fb5e0eb5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c50af620a0 0x55c50b34c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:01.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.848+0000 7fb5e0eb5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c50af6a280 0x55c50b33aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:01.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.848+0000 7fb5e0eb5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c50af8af60 0x55c50b33b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:01.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.848+0000 7fb5e0eb5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c50b34cf70 con 0x55c50af6a280 2024-08-31T06:06:01.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.848+0000 7fb5e0eb5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c50b34d640 con 0x55c50af8af60 2024-08-31T06:06:01.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.848+0000 7fb5e0eb5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c50b34dd10 con 0x55c50af620a0 2024-08-31T06:06:01.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.849+0000 7fb5d2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c50af6a280 0x55c50b33aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:01.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.849+0000 7fb5d2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c50af6a280 0x55c50b33aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38118/0 (socket says 172.21.15.16:38118) 2024-08-31T06:06:01.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.849+0000 7fb5d2a00640 1 -- 172.21.15.16:0/3989039667 learned_addr learned my addr 172.21.15.16:0/3989039667 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:01.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.849+0000 7fb5d3e00640 1 --2- 172.21.15.16:0/3989039667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c50af8af60 0x55c50b33b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:01.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.849+0000 7fb5d3400640 1 --2- 172.21.15.16:0/3989039667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c50af620a0 0x55c50b34c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:01.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.849+0000 7fb5d2a00640 1 -- 172.21.15.16:0/3989039667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c50af620a0 msgr2=0x55c50b34c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:01.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.849+0000 7fb5d2a00640 1 --2- 172.21.15.16:0/3989039667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c50af620a0 0x55c50b34c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:01.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.849+0000 7fb5d2a00640 1 -- 172.21.15.16:0/3989039667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c50af8af60 msgr2=0x55c50b33b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:01.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.849+0000 7fb5d2a00640 1 --2- 172.21.15.16:0/3989039667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c50af8af60 0x55c50b33b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:01.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.849+0000 7fb5d2a00640 1 -- 172.21.15.16:0/3989039667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c50b336f50 con 0x55c50af6a280 2024-08-31T06:06:01.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.849+0000 7fb5d2a00640 1 --2- 172.21.15.16:0/3989039667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c50af6a280 0x55c50b33aba0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb5c00ce410 tx=0x7fb5c000df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=415f855146d64611 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:01.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.850+0000 7fb5bf400640 1 -- 172.21.15.16:0/3989039667 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c0020050 con 0x55c50af6a280 2024-08-31T06:06:01.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.850+0000 7fb5bf400640 1 -- 172.21.15.16:0/3989039667 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5c00ce0b0 con 0x55c50af6a280 2024-08-31T06:06:01.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.850+0000 7fb5bf400640 1 -- 172.21.15.16:0/3989039667 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c00ab530 con 0x55c50af6a280 2024-08-31T06:06:01.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.850+0000 7fb5e0eb5740 1 -- 172.21.15.16:0/3989039667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c50af6a280 msgr2=0x55c50b33aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:01.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.850+0000 7fb5e0eb5740 1 --2- 172.21.15.16:0/3989039667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c50af6a280 0x55c50b33aba0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb5c00ce410 tx=0x7fb5c000df90 comp rx=0 tx=0).stop 2024-08-31T06:06:01.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.850+0000 7fb5e0eb5740 1 -- 172.21.15.16:0/3989039667 shutdown_connections 2024-08-31T06:06:01.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.850+0000 7fb5e0eb5740 1 --2- 172.21.15.16:0/3989039667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c50af8af60 0x55c50b33b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:01.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.850+0000 7fb5e0eb5740 1 --2- 172.21.15.16:0/3989039667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c50af6a280 0x55c50b33aba0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:01.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.850+0000 7fb5e0eb5740 1 --2- 172.21.15.16:0/3989039667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c50af620a0 0x55c50b34c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:01.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.850+0000 7fb5e0eb5740 1 -- 172.21.15.16:0/3989039667 >> 172.21.15.16:0/3989039667 conn(0x55c50af7f350 msgr2=0x55c50b348750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:01.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.851+0000 7fb5e0eb5740 1 -- 172.21.15.16:0/3989039667 shutdown_connections 2024-08-31T06:06:01.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.851+0000 7fb5e0eb5740 1 -- 172.21.15.16:0/3989039667 wait complete. 2024-08-31T06:06:01.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.851+0000 7fb5e0eb5740 1 -- start start 2024-08-31T06:06:01.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.851+0000 7fb5e0eb5740 1 Processor -- start 2024-08-31T06:06:01.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.852+0000 7fb5e0eb5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c50af620a0 0x55c50b352e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:01.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.852+0000 7fb5e0eb5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c50af6a280 0x55c50b3533d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:01.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.852+0000 7fb5e0eb5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c50af8af60 0x55c50b353910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:01.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.852+0000 7fb5e0eb5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c50b34eab0 con 0x55c50af620a0 2024-08-31T06:06:01.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.852+0000 7fb5e0eb5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c50b34e3e0 con 0x55c50af8af60 2024-08-31T06:06:01.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.852+0000 7fb5e0eb5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c50b34f180 con 0x55c50af6a280 2024-08-31T06:06:01.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.852+0000 7fb5d3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c50af620a0 0x55c50b352e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:01.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.852+0000 7fb5d3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c50af620a0 0x55c50b352e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38120/0 (socket says 172.21.15.16:38120) 2024-08-31T06:06:01.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.852+0000 7fb5d3400640 1 -- 172.21.15.16:0/3886691503 learned_addr learned my addr 172.21.15.16:0/3886691503 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:01.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.852+0000 7fb5e0eb5740 10 client.0 _ll_register_callbacks cb 0x7fb5c0004f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:06:01.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.852+0000 7fb5d2a00640 1 --2- 172.21.15.16:0/3886691503 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c50af6a280 0x55c50b3533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:01.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.852+0000 7fb5d3e00640 1 --2- 172.21.15.16:0/3886691503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c50af8af60 0x55c50b353910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:01.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.853+0000 7fb5d3400640 1 -- 172.21.15.16:0/3886691503 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c50af6a280 msgr2=0x55c50b3533d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:01.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.853+0000 7fb5d3400640 1 --2- 172.21.15.16:0/3886691503 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c50af6a280 0x55c50b3533d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:01.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.853+0000 7fb5d3400640 1 -- 172.21.15.16:0/3886691503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c50af8af60 msgr2=0x55c50b353910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:01.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.853+0000 7fb5d3400640 1 --2- 172.21.15.16:0/3886691503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c50af8af60 0x55c50b353910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:01.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.853+0000 7fb5d3400640 1 -- 172.21.15.16:0/3886691503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c50b3b9100 con 0x55c50af620a0 2024-08-31T06:06:01.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.853+0000 7fb5d3400640 1 --2- 172.21.15.16:0/3886691503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c50af620a0 0x55c50b352e90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb5c402a9e0 tx=0x7fb5c4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a6cc4526c26bbb server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:01.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.853+0000 7fb5bcc00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:06:01.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.853+0000 7fb5bcc00640 1 -- 172.21.15.16:0/3886691503 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c405e060 con 0x55c50af620a0 2024-08-31T06:06:01.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.853+0000 7fb5bcc00640 1 -- 172.21.15.16:0/3886691503 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7fb5c4031a80 con 0x55c50af620a0 2024-08-31T06:06:01.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.853+0000 7fb5e0eb5740 1 -- 172.21.15.16:0/3886691503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c50b2a4f40 con 0x55c50af620a0 2024-08-31T06:06:01.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.854+0000 7fb5bcc00640 1 -- 172.21.15.16:0/3886691503 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb5c4035040 con 0x55c50af620a0 2024-08-31T06:06:01.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.854+0000 7fb5e0eb5740 10 client.15096 fetch_fsmap learned FSMap version 24 2024-08-31T06:06:01.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.854+0000 7fb5e0eb5740 1 -- 172.21.15.16:0/3886691503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c50b3ba380 con 0x55c50af620a0 2024-08-31T06:06:01.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.854+0000 7fb5bcc00640 1 -- 172.21.15.16:0/3886691503 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c50b2a4f40 con 0x55c50af620a0 2024-08-31T06:06:01.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.854+0000 7fb5bcc00640 1 -- 172.21.15.16:0/3886691503 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fb5c4044070 con 0x55c50af620a0 2024-08-31T06:06:01.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.854+0000 7fb5e0eb5740 10 client.15096 fetch_fsmap finished waiting for FSMap version 24 2024-08-31T06:06:01.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.854+0000 7fb5e0eb5740 10 client.15096 resolve_mds: resolved name 'g' to daemon mds.g 2024-08-31T06:06:01.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.854+0000 7fb5e0eb5740 20 client.15096 populate_metadata read hostname 'smithi016' 2024-08-31T06:06:01.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.854+0000 7fb5e0eb5740 1 --2- 172.21.15.16:0/3886691503 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55c50af1ec90 0x55c50b3bdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:01.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.854+0000 7fb5e0eb5740 4 client.15096 mds_command: new command op to 24724 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-31T06:06:01.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.854+0000 7fb5e0eb5740 1 -- 172.21.15.16:0/3886691503 --> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] -- command(tid 0: {"prefix": "status"}) -- 0x55c50b2a2680 con 0x55c50af1ec90 2024-08-31T06:06:01.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.855+0000 7fb5d2a00640 1 --2- 172.21.15.16:0/3886691503 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55c50af1ec90 0x55c50b3bdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:01.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.855+0000 7fb5bcc00640 10 client.15096 ms_handle_connect on v2:172.21.15.16:6838/219664460 2024-08-31T06:06:01.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.855+0000 7fb5d2a00640 1 --2- 172.21.15.16:0/3886691503 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55c50af1ec90 0x55c50b3bdc10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ec6b3c3236d0c118 server_cookie=ace06b29cfb836a9 in_seq=0 out_seq=0 2024-08-31T06:06:01.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.855+0000 7fb5bcc00640 1 -- 172.21.15.16:0/3886691503 <== mds.1 v2:172.21.15.16:6838/219664460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55c50b2a2680 con 0x55c50af1ec90 2024-08-31T06:06:01.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.855+0000 7fb5bcc00640 10 client.15096 handle_command_reply: tid=0 2024-08-31T06:06:01.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.856+0000 7fb5e0eb5740 1 client.15096 shutdown 2024-08-31T06:06:01.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.856+0000 7fb5e0eb5740 1 -- 172.21.15.16:0/3886691503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c50af620a0 msgr2=0x55c50b352e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:01.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.856+0000 7fb5e0eb5740 1 --2- 172.21.15.16:0/3886691503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c50af620a0 0x55c50b352e90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb5c402a9e0 tx=0x7fb5c4009260 comp rx=0 tx=0).stop 2024-08-31T06:06:01.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.856+0000 7fb5e0eb5740 1 -- 172.21.15.16:0/3886691503 shutdown_connections 2024-08-31T06:06:01.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.856+0000 7fb5e0eb5740 1 --2- 172.21.15.16:0/3886691503 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c50af6a280 0x55c50b3533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:01.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.856+0000 7fb5e0eb5740 1 --2- 172.21.15.16:0/3886691503 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55c50af1ec90 0x55c50b3bdc10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:01.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.856+0000 7fb5e0eb5740 1 --2- 172.21.15.16:0/3886691503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c50af8af60 0x55c50b353910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:01.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.856+0000 7fb5e0eb5740 1 --2- 172.21.15.16:0/3886691503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c50af620a0 0x55c50b352e90 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:01.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.856+0000 7fb5e0eb5740 1 -- 172.21.15.16:0/3886691503 >> 172.21.15.16:0/3886691503 conn(0x55c50af7f350 msgr2=0x55c50b348750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:01.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.856+0000 7fb5bcc00640 0 client.15096 ms_handle_reset on v2:172.21.15.16:6838/219664460 2024-08-31T06:06:01.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.856+0000 7fb5e0eb5740 1 -- 172.21.15.16:0/3886691503 shutdown_connections 2024-08-31T06:06:01.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.856+0000 7fb5e0eb5740 1 -- 172.21.15.16:0/3886691503 wait complete. 2024-08-31T06:06:01.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.856+0000 7fb5e0eb5740 20 client.15096 trim_cache size 0 max 16384 2024-08-31T06:06:01.861 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:06:01.861 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "673e4aa0-675d-11ef-bcd4-c7b262605968", 2024-08-31T06:06:01.861 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 1, 2024-08-31T06:06:01.862 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24724, 2024-08-31T06:06:01.862 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-08-31T06:06:01.862 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-08-31T06:06:01.862 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-08-31T06:06:01.862 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 83.931436470999998, 2024-08-31T06:06:01.862 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 24, 2024-08-31T06:06:01.862 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 80, 2024-08-31T06:06:01.862 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 78, 2024-08-31T06:06:01.862 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 114.86088429500001 2024-08-31T06:06:01.862 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:06:01.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.857+0000 7fb5d8c00640 1 -- 172.21.15.16:0/4144970141 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb5b80b4c10 msgr2=0x7fb5b80b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:01.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.857+0000 7fb5d8c00640 1 --2- 172.21.15.16:0/4144970141 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb5b80b4c10 0x7fb5b80b7120 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb5c40096c0 tx=0x7fb5c4005a20 comp rx=0 tx=0).stop 2024-08-31T06:06:01.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.857+0000 7fb5d8c00640 1 -- 172.21.15.16:0/4144970141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5d4104ec0 msgr2=0x7fb5d419c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:01.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.857+0000 7fb5d8c00640 1 --2- 172.21.15.16:0/4144970141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5d4104ec0 0x7fb5d419c8c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb5d406b240 tx=0x7fb5c000db30 comp rx=0 tx=0).stop 2024-08-31T06:06:01.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.857+0000 7fb5d8c00640 1 -- 172.21.15.16:0/4144970141 shutdown_connections 2024-08-31T06:06:01.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.857+0000 7fb5d8c00640 1 --2- 172.21.15.16:0/4144970141 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb5b80b4c10 0x7fb5b80b7120 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:01.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.857+0000 7fb5d8c00640 1 --2- 172.21.15.16:0/4144970141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5d410d9d0 0x7fb5d41a1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:01.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.857+0000 7fb5d8c00640 1 --2- 172.21.15.16:0/4144970141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5d4104ec0 0x7fb5d419c8c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:01.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.857+0000 7fb5d8c00640 1 --2- 172.21.15.16:0/4144970141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5d4104570 0x7fb5d419c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:01.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.857+0000 7fb5d8c00640 1 -- 172.21.15.16:0/4144970141 >> 172.21.15.16:0/4144970141 conn(0x7fb5d40fe070 msgr2=0x7fb5d4102220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:01.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.857+0000 7fb5d8c00640 1 -- 172.21.15.16:0/4144970141 shutdown_connections 2024-08-31T06:06:01.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:01.857+0000 7fb5d8c00640 1 -- 172.21.15.16:0/4144970141 wait complete. 2024-08-31T06:06:02.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:02 smithi016 ceph-mon[17194]: pgmap v327: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-31T06:06:02.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:02 smithi184 ceph-mon[22331]: pgmap v327: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-31T06:06:02.728 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.b status 2024-08-31T06:06:02.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:02 smithi155 ceph-mon[21742]: pgmap v327: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-31T06:06:03.108 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:06:03.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.765+0000 7f81b7000640 1 -- 172.21.15.16:0/1170305112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81b010fe80 msgr2=0x7f81b01122c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:03.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.765+0000 7f81b7000640 1 --2- 172.21.15.16:0/1170305112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81b010fe80 0x7f81b01122c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f81a8009ca0 tx=0x7f81a802a8c0 comp rx=0 tx=0).stop 2024-08-31T06:06:03.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.765+0000 7f81b7000640 1 -- 172.21.15.16:0/1170305112 shutdown_connections 2024-08-31T06:06:03.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.765+0000 7f81b7000640 1 --2- 172.21.15.16:0/1170305112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81b0075ea0 0x7f81b010f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:03.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.765+0000 7f81b7000640 1 --2- 172.21.15.16:0/1170305112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81b010fe80 0x7f81b01122c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:03.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.765+0000 7f81b7000640 1 --2- 172.21.15.16:0/1170305112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81b0075550 0x7f81b0075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:03.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.765+0000 7f81b7000640 1 -- 172.21.15.16:0/1170305112 >> 172.21.15.16:0/1170305112 conn(0x7f81b00ff370 msgr2=0x7f81b01017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:03.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.765+0000 7f81b7000640 1 -- 172.21.15.16:0/1170305112 shutdown_connections 2024-08-31T06:06:03.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.765+0000 7f81b7000640 1 -- 172.21.15.16:0/1170305112 wait complete. 2024-08-31T06:06:03.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.766+0000 7f81b7000640 1 Processor -- start 2024-08-31T06:06:03.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.766+0000 7f81b7000640 1 -- start start 2024-08-31T06:06:03.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.767+0000 7f81b7000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81b0075550 0x7f81b019c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:03.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.767+0000 7f81b7000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81b0075ea0 0x7f81b019c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:03.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.767+0000 7f81b7000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81b010fe80 0x7f81b01a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:03.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.767+0000 7f81b7000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f81b01159f0 con 0x7f81b010fe80 2024-08-31T06:06:03.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.767+0000 7f81b7000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f81b0115870 con 0x7f81b0075ea0 2024-08-31T06:06:03.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.767+0000 7f81b7000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f81b0115b70 con 0x7f81b0075550 2024-08-31T06:06:03.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.767+0000 7f81af400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81b0075ea0 0x7f81b019c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:03.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.767+0000 7f81b5000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81b010fe80 0x7f81b01a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:03.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.767+0000 7f81af400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81b0075ea0 0x7f81b019c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:57414/0 (socket says 172.21.15.16:57414) 2024-08-31T06:06:03.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.767+0000 7f81af400640 1 -- 172.21.15.16:0/4192829719 learned_addr learned my addr 172.21.15.16:0/4192829719 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:03.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.767+0000 7f81afe00640 1 --2- 172.21.15.16:0/4192829719 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81b0075550 0x7f81b019c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:03.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.767+0000 7f81af400640 1 -- 172.21.15.16:0/4192829719 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81b0075550 msgr2=0x7f81b019c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:03.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.767+0000 7f81af400640 1 --2- 172.21.15.16:0/4192829719 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81b0075550 0x7f81b019c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:03.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.767+0000 7f81af400640 1 -- 172.21.15.16:0/4192829719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81b010fe80 msgr2=0x7f81b01a1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:03.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.767+0000 7f81af400640 1 --2- 172.21.15.16:0/4192829719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81b010fe80 0x7f81b01a1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:03.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.767+0000 7f81af400640 1 -- 172.21.15.16:0/4192829719 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b01a2330 con 0x7f81b0075ea0 2024-08-31T06:06:03.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.768+0000 7f81af400640 1 --2- 172.21.15.16:0/4192829719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81b0075ea0 0x7f81b019c960 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f819c00d770 tx=0x7f819c00dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:03.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.768+0000 7f81acc00640 1 -- 172.21.15.16:0/4192829719 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819c017070 con 0x7f81b0075ea0 2024-08-31T06:06:03.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.768+0000 7f81b7000640 1 -- 172.21.15.16:0/4192829719 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81b01a2620 con 0x7f81b0075ea0 2024-08-31T06:06:03.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.768+0000 7f81acc00640 1 -- 172.21.15.16:0/4192829719 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f819c009c30 con 0x7f81b0075ea0 2024-08-31T06:06:03.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.768+0000 7f81acc00640 1 -- 172.21.15.16:0/4192829719 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819c003cc0 con 0x7f81b0075ea0 2024-08-31T06:06:03.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.768+0000 7f81b7000640 1 -- 172.21.15.16:0/4192829719 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81b01a8d80 con 0x7f81b0075ea0 2024-08-31T06:06:03.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.771+0000 7f81bdc7b740 1 Processor -- start 2024-08-31T06:06:03.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.772+0000 7f81bdc7b740 1 -- start start 2024-08-31T06:06:03.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.772+0000 7f81bdc7b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a5eec3f60 0x562a5f2698f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:03.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.772+0000 7f81bdc7b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a5ee9b0a0 0x562a5f269e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:03.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.772+0000 7f81bdc7b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a5eea3280 0x562a5f26a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:03.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.772+0000 7f81bdc7b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562a5f1e0550 con 0x562a5eea3280 2024-08-31T06:06:03.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.772+0000 7f81bdc7b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562a5f1cfae0 con 0x562a5eec3f60 2024-08-31T06:06:03.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.772+0000 7f81bdc7b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562a5f1d3a90 con 0x562a5ee9b0a0 2024-08-31T06:06:03.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.772+0000 7f81acc00640 1 -- 172.21.15.16:0/4192829719 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f819c003e60 con 0x7f81b0075ea0 2024-08-31T06:06:03.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.772+0000 7f81b5000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a5eec3f60 0x562a5f2698f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:03.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.773+0000 7f81afe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a5ee9b0a0 0x562a5f269e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:03.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.773+0000 7f81afe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a5ee9b0a0 0x562a5f269e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:52726/0 (socket says 172.21.15.16:52726) 2024-08-31T06:06:03.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.773+0000 7f81afe00640 1 -- 172.21.15.16:0/2725720928 learned_addr learned my addr 172.21.15.16:0/2725720928 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:03.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.773+0000 7f81af400640 1 --2- 172.21.15.16:0/2725720928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a5eea3280 0x562a5f26a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:03.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.773+0000 7f81b5000640 1 -- 172.21.15.16:0/2725720928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a5ee9b0a0 msgr2=0x562a5f269e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:03.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.773+0000 7f81b5000640 1 --2- 172.21.15.16:0/2725720928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a5ee9b0a0 0x562a5f269e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:03.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.773+0000 7f81b5000640 1 -- 172.21.15.16:0/2725720928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a5eea3280 msgr2=0x562a5f26a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:03.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.773+0000 7f81b5000640 1 --2- 172.21.15.16:0/2725720928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a5eea3280 0x562a5f26a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:03.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.773+0000 7f81b5000640 1 -- 172.21.15.16:0/2725720928 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a5ef2f540 con 0x562a5eec3f60 2024-08-31T06:06:03.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.773+0000 7f81afe00640 1 --2- 172.21.15.16:0/2725720928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a5ee9b0a0 0x562a5f269e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:06:03.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.773+0000 7f81b5000640 1 --2- 172.21.15.16:0/2725720928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a5eec3f60 0x562a5f2698f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f81a8005e30 tx=0x7f81a802f4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7569395ac8fe925 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:03.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.774+0000 7f81a5600640 1 -- 172.21.15.16:0/2725720928 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a8031cf0 con 0x562a5eec3f60 2024-08-31T06:06:03.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.774+0000 7f81a5600640 1 -- 172.21.15.16:0/2725720928 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81a8036030 con 0x562a5eec3f60 2024-08-31T06:06:03.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.774+0000 7f81a5600640 1 -- 172.21.15.16:0/2725720928 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a8003e60 con 0x562a5eec3f60 2024-08-31T06:06:03.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.774+0000 7f81acc00640 1 --2- 172.21.15.16:0/4192829719 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f81940b4c10 0x7f81940b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:03.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.775+0000 7f81acc00640 1 -- 172.21.15.16:0/4192829719 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f819c0e4cd0 con 0x7f81b0075ea0 2024-08-31T06:06:03.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.775+0000 7f81afe00640 1 --2- 172.21.15.16:0/4192829719 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f81940b4c10 0x7f81940b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:03.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.775+0000 7f81bdc7b740 1 -- 172.21.15.16:0/2725720928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a5eec3f60 msgr2=0x562a5f2698f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:03.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.775+0000 7f81bdc7b740 1 --2- 172.21.15.16:0/2725720928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a5eec3f60 0x562a5f2698f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f81a8005e30 tx=0x7f81a802f4c0 comp rx=0 tx=0).stop 2024-08-31T06:06:03.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.775+0000 7f81bdc7b740 1 -- 172.21.15.16:0/2725720928 shutdown_connections 2024-08-31T06:06:03.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.775+0000 7f81bdc7b740 1 --2- 172.21.15.16:0/2725720928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a5ee9b0a0 0x562a5f269e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:03.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.775+0000 7f81bdc7b740 1 --2- 172.21.15.16:0/2725720928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a5eea3280 0x562a5f26a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:03.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.775+0000 7f81bdc7b740 1 --2- 172.21.15.16:0/2725720928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a5eec3f60 0x562a5f2698f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:03.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.775+0000 7f81bdc7b740 1 -- 172.21.15.16:0/2725720928 >> 172.21.15.16:0/2725720928 conn(0x562a5eeb8350 msgr2=0x562a5f2677e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:03.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.775+0000 7f81bdc7b740 1 -- 172.21.15.16:0/2725720928 shutdown_connections 2024-08-31T06:06:03.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.775+0000 7f81bdc7b740 1 -- 172.21.15.16:0/2725720928 wait complete. 2024-08-31T06:06:03.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.775+0000 7f81afe00640 1 --2- 172.21.15.16:0/4192829719 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f81940b4c10 0x7f81940b7120 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x562a5f268570 tx=0x7f81a0005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:03.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.776+0000 7f81bdc7b740 1 -- start start 2024-08-31T06:06:03.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.776+0000 7f81bdc7b740 1 Processor -- start 2024-08-31T06:06:03.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.777+0000 7f81bdc7b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a5ee9b0a0 0x562a5f275000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:03.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.777+0000 7f81bdc7b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a5eea3280 0x562a5f275540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:03.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.777+0000 7f81bdc7b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a5eec3f60 0x562a5f2811e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:03.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.777+0000 7f81bdc7b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562a5f0325f0 con 0x562a5eea3280 2024-08-31T06:06:03.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.777+0000 7f81bdc7b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562a5f156bd0 con 0x562a5ee9b0a0 2024-08-31T06:06:03.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.777+0000 7f81bdc7b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562a5f277ef0 con 0x562a5eec3f60 2024-08-31T06:06:03.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.777+0000 7f81bdc7b740 10 client.0 _ll_register_callbacks cb 0x562a5f17c210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:06:03.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.777+0000 7f81af400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a5eea3280 0x562a5f275540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:03.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.777+0000 7f81af400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a5eea3280 0x562a5f275540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38154/0 (socket says 172.21.15.16:38154) 2024-08-31T06:06:03.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.777+0000 7f81af400640 1 -- 172.21.15.16:0/3229760261 learned_addr learned my addr 172.21.15.16:0/3229760261 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:03.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.777+0000 7f81afe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a5ee9b0a0 0x562a5f275000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:03.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.777+0000 7f81b5000640 1 --2- 172.21.15.16:0/3229760261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a5eec3f60 0x562a5f2811e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:03.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.777+0000 7f81af400640 1 -- 172.21.15.16:0/3229760261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a5eec3f60 msgr2=0x562a5f2811e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:03.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.777+0000 7f81af400640 1 --2- 172.21.15.16:0/3229760261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a5eec3f60 0x562a5f2811e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:03.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.777+0000 7f81af400640 1 -- 172.21.15.16:0/3229760261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a5ee9b0a0 msgr2=0x562a5f275000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:03.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.777+0000 7f81af400640 1 --2- 172.21.15.16:0/3229760261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a5ee9b0a0 0x562a5f275000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:03.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.777+0000 7f81af400640 1 -- 172.21.15.16:0/3229760261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a5f197650 con 0x562a5eea3280 2024-08-31T06:06:03.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.777+0000 7f81afe00640 1 --2- 172.21.15.16:0/3229760261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a5ee9b0a0 0x562a5f275000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:06:03.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.778+0000 7f81af400640 1 --2- 172.21.15.16:0/3229760261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a5eea3280 0x562a5f275540 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f819c0a7610 tx=0x7f819c0e2a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=103e89e832a5e760 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:03.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.778+0000 7f819a000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:06:03.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.778+0000 7f819a000640 1 -- 172.21.15.16:0/3229760261 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819c020050 con 0x562a5eea3280 2024-08-31T06:06:03.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.778+0000 7f819a000640 1 -- 172.21.15.16:0/3229760261 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f819c0ce0b0 con 0x562a5eea3280 2024-08-31T06:06:03.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.778+0000 7f81bdc7b740 1 -- 172.21.15.16:0/3229760261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81a8036030 con 0x562a5eea3280 2024-08-31T06:06:03.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.778+0000 7f819a000640 1 -- 172.21.15.16:0/3229760261 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f819c0aad00 con 0x562a5eea3280 2024-08-31T06:06:03.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.778+0000 7f81bdc7b740 10 client.15108 fetch_fsmap learned FSMap version 24 2024-08-31T06:06:03.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.778+0000 7f81bdc7b740 1 -- 172.21.15.16:0/3229760261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562a5f283cc0 con 0x562a5eea3280 2024-08-31T06:06:03.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.779+0000 7f819a000640 1 -- 172.21.15.16:0/3229760261 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81a8036030 con 0x562a5eea3280 2024-08-31T06:06:03.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.779+0000 7f819a000640 1 -- 172.21.15.16:0/3229760261 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f819c0e2bd0 con 0x562a5eea3280 2024-08-31T06:06:03.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.779+0000 7f81bdc7b740 10 client.15108 fetch_fsmap finished waiting for FSMap version 24 2024-08-31T06:06:03.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.779+0000 7f81bdc7b740 10 client.15108 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-31T06:06:03.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.779+0000 7f81bdc7b740 20 client.15108 populate_metadata read hostname 'smithi016' 2024-08-31T06:06:03.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.779+0000 7f81bdc7b740 1 --2- 172.21.15.16:0/3229760261 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x562a5ee828e0 0x562a5f287750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:03.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.779+0000 7f81bdc7b740 4 client.15108 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:06:03.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.779+0000 7f81bdc7b740 1 -- 172.21.15.16:0/3229760261 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a5f1e24a0 con 0x562a5ee828e0 2024-08-31T06:06:03.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.780+0000 7f81afe00640 1 --2- 172.21.15.16:0/3229760261 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x562a5ee828e0 0x562a5f287750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:03.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.780+0000 7f81afe00640 1 --2- 172.21.15.16:0/3229760261 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x562a5ee828e0 0x562a5f287750 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e9cedd1e710fed2 server_cookie=24a2d48be9d57bb3 in_seq=0 out_seq=0 2024-08-31T06:06:03.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.780+0000 7f819a000640 10 client.15108 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:06:03.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.781+0000 7f819a000640 1 -- 172.21.15.16:0/3229760261 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562a5f1e24a0 con 0x562a5ee828e0 2024-08-31T06:06:03.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.781+0000 7f819a000640 10 client.15108 handle_command_reply: tid=0 2024-08-31T06:06:03.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.781+0000 7f81bdc7b740 1 client.15108 shutdown 2024-08-31T06:06:03.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.781+0000 7f81bdc7b740 1 -- 172.21.15.16:0/3229760261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a5eea3280 msgr2=0x562a5f275540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:03.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.781+0000 7f81bdc7b740 1 --2- 172.21.15.16:0/3229760261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a5eea3280 0x562a5f275540 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f819c0a7610 tx=0x7f819c0e2a40 comp rx=0 tx=0).stop 2024-08-31T06:06:03.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.781+0000 7f81bdc7b740 1 -- 172.21.15.16:0/3229760261 shutdown_connections 2024-08-31T06:06:03.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.781+0000 7f81bdc7b740 1 --2- 172.21.15.16:0/3229760261 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x562a5ee828e0 0x562a5f287750 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:03.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.781+0000 7f81bdc7b740 1 --2- 172.21.15.16:0/3229760261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a5eec3f60 0x562a5f2811e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:03.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.781+0000 7f81bdc7b740 1 --2- 172.21.15.16:0/3229760261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a5eea3280 0x562a5f275540 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:03.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.781+0000 7f81bdc7b740 1 --2- 172.21.15.16:0/3229760261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a5ee9b0a0 0x562a5f275000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:03.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.781+0000 7f81bdc7b740 1 -- 172.21.15.16:0/3229760261 >> 172.21.15.16:0/3229760261 conn(0x562a5eeb8350 msgr2=0x562a5f26cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:03.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.782+0000 7f819a000640 0 client.15108 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:06:03.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.782+0000 7f81bdc7b740 1 -- 172.21.15.16:0/3229760261 shutdown_connections 2024-08-31T06:06:03.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.782+0000 7f81bdc7b740 1 -- 172.21.15.16:0/3229760261 wait complete. 2024-08-31T06:06:03.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.782+0000 7f81bdc7b740 20 client.15108 trim_cache size 0 max 16384 2024-08-31T06:06:03.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.801+0000 7f81bdc7b740 1 Processor -- start 2024-08-31T06:06:03.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.801+0000 7f81bdc7b740 1 -- start start 2024-08-31T06:06:03.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.801+0000 7f81bdc7b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a5ee9b0a0 0x562a5f285180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:03.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.801+0000 7f81bdc7b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a5eea3280 0x562a5f281150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:03.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.801+0000 7f81bdc7b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a5eec3f60 0x562a5f274db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:03.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.801+0000 7f81bdc7b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562a5f285a90 con 0x562a5ee9b0a0 2024-08-31T06:06:03.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.801+0000 7f81bdc7b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562a5f286160 con 0x562a5eec3f60 2024-08-31T06:06:03.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.801+0000 7f81bdc7b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562a5f286830 con 0x562a5eea3280 2024-08-31T06:06:03.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.802+0000 7f81afe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a5ee9b0a0 0x562a5f285180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:03.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.802+0000 7f81afe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a5ee9b0a0 0x562a5f285180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38164/0 (socket says 172.21.15.16:38164) 2024-08-31T06:06:03.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.802+0000 7f81afe00640 1 -- 172.21.15.16:0/2393175043 learned_addr learned my addr 172.21.15.16:0/2393175043 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:03.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.802+0000 7f81af400640 1 --2- 172.21.15.16:0/2393175043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a5eea3280 0x562a5f281150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:03.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.802+0000 7f81b5000640 1 --2- 172.21.15.16:0/2393175043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a5eec3f60 0x562a5f274db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:03.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.802+0000 7f81afe00640 1 -- 172.21.15.16:0/2393175043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a5eea3280 msgr2=0x562a5f281150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:03.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.802+0000 7f81afe00640 1 --2- 172.21.15.16:0/2393175043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a5eea3280 0x562a5f281150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:03.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.802+0000 7f81afe00640 1 -- 172.21.15.16:0/2393175043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a5eec3f60 msgr2=0x562a5f274db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:03.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.802+0000 7f81afe00640 1 --2- 172.21.15.16:0/2393175043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a5eec3f60 0x562a5f274db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:03.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.802+0000 7f81afe00640 1 -- 172.21.15.16:0/2393175043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a5f2685a0 con 0x562a5ee9b0a0 2024-08-31T06:06:03.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.802+0000 7f81afe00640 1 --2- 172.21.15.16:0/2393175043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a5ee9b0a0 0x562a5f285180 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f81a0009bb0 tx=0x7f81a001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c1aefc5687bbe12 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:03.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.803+0000 7f81a4c00640 1 -- 172.21.15.16:0/2393175043 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a0004020 con 0x562a5ee9b0a0 2024-08-31T06:06:03.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.803+0000 7f81a4c00640 1 -- 172.21.15.16:0/2393175043 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81a00041c0 con 0x562a5ee9b0a0 2024-08-31T06:06:03.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.803+0000 7f81a4c00640 1 -- 172.21.15.16:0/2393175043 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a001d860 con 0x562a5ee9b0a0 2024-08-31T06:06:03.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.803+0000 7f81bdc7b740 1 -- 172.21.15.16:0/2393175043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a5ee9b0a0 msgr2=0x562a5f285180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:03.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.803+0000 7f81bdc7b740 1 --2- 172.21.15.16:0/2393175043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a5ee9b0a0 0x562a5f285180 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f81a0009bb0 tx=0x7f81a001b000 comp rx=0 tx=0).stop 2024-08-31T06:06:03.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.803+0000 7f81bdc7b740 1 -- 172.21.15.16:0/2393175043 shutdown_connections 2024-08-31T06:06:03.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.803+0000 7f81bdc7b740 1 --2- 172.21.15.16:0/2393175043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a5eea3280 0x562a5f281150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:03.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.803+0000 7f81bdc7b740 1 --2- 172.21.15.16:0/2393175043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a5eec3f60 0x562a5f274db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:03.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.803+0000 7f81bdc7b740 1 --2- 172.21.15.16:0/2393175043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a5ee9b0a0 0x562a5f285180 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:03.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.803+0000 7f81bdc7b740 1 -- 172.21.15.16:0/2393175043 >> 172.21.15.16:0/2393175043 conn(0x562a5eeb8350 msgr2=0x562a5f273880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:03.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.804+0000 7f81bdc7b740 1 -- 172.21.15.16:0/2393175043 shutdown_connections 2024-08-31T06:06:03.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.804+0000 7f81bdc7b740 1 -- 172.21.15.16:0/2393175043 wait complete. 2024-08-31T06:06:03.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.804+0000 7f81bdc7b740 1 -- start start 2024-08-31T06:06:03.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.804+0000 7f81bdc7b740 1 Processor -- start 2024-08-31T06:06:03.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.805+0000 7f81bdc7b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a5ee9b0a0 0x562a5f28ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:03.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.805+0000 7f81bdc7b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a5eea3280 0x562a5f28f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:03.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.805+0000 7f81bdc7b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a5eec3f60 0x562a5f289020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:03.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.805+0000 7f81bdc7b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562a5f2875d0 con 0x562a5ee9b0a0 2024-08-31T06:06:03.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.805+0000 7f81bdc7b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562a5f286f00 con 0x562a5eec3f60 2024-08-31T06:06:03.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.805+0000 7f81bdc7b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562a5f287ca0 con 0x562a5eea3280 2024-08-31T06:06:03.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.805+0000 7f81bdc7b740 10 client.0 _ll_register_callbacks cb 0x7f819c00de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:06:03.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.805+0000 7f81afe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a5ee9b0a0 0x562a5f28ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:03.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.805+0000 7f81afe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a5ee9b0a0 0x562a5f28ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38178/0 (socket says 172.21.15.16:38178) 2024-08-31T06:06:03.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.805+0000 7f81afe00640 1 -- 172.21.15.16:0/2889315433 learned_addr learned my addr 172.21.15.16:0/2889315433 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:03.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.805+0000 7f81b5000640 1 --2- 172.21.15.16:0/2889315433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a5eec3f60 0x562a5f289020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:03.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.805+0000 7f81af400640 1 --2- 172.21.15.16:0/2889315433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a5eea3280 0x562a5f28f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:03.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.806+0000 7f81afe00640 1 -- 172.21.15.16:0/2889315433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a5eea3280 msgr2=0x562a5f28f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:03.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.806+0000 7f81afe00640 1 --2- 172.21.15.16:0/2889315433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a5eea3280 0x562a5f28f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:03.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.806+0000 7f81afe00640 1 -- 172.21.15.16:0/2889315433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a5eec3f60 msgr2=0x562a5f289020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:03.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.806+0000 7f81afe00640 1 --2- 172.21.15.16:0/2889315433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a5eec3f60 0x562a5f289020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:03.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.806+0000 7f81afe00640 1 -- 172.21.15.16:0/2889315433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a5f289690 con 0x562a5ee9b0a0 2024-08-31T06:06:03.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.806+0000 7f81afe00640 1 --2- 172.21.15.16:0/2889315433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a5ee9b0a0 0x562a5f28ea60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f81a0009e90 tx=0x7f81a0009990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7205546d2fbfebb5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:03.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.806+0000 7f819a000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:06:03.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.806+0000 7f819a000640 1 -- 172.21.15.16:0/2889315433 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a00035f0 con 0x562a5ee9b0a0 2024-08-31T06:06:03.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.806+0000 7f819a000640 1 -- 172.21.15.16:0/2889315433 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f81a0003790 con 0x562a5ee9b0a0 2024-08-31T06:06:03.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.806+0000 7f81bdc7b740 1 -- 172.21.15.16:0/2889315433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81a00041c0 con 0x562a5ee9b0a0 2024-08-31T06:06:03.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.806+0000 7f819a000640 1 -- 172.21.15.16:0/2889315433 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f81a0041050 con 0x562a5ee9b0a0 2024-08-31T06:06:03.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.807+0000 7f81bdc7b740 10 client.15114 fetch_fsmap learned FSMap version 24 2024-08-31T06:06:03.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.807+0000 7f81bdc7b740 1 -- 172.21.15.16:0/2889315433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562a5f28dfd0 con 0x562a5ee9b0a0 2024-08-31T06:06:03.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.807+0000 7f819a000640 1 -- 172.21.15.16:0/2889315433 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81a00041c0 con 0x562a5ee9b0a0 2024-08-31T06:06:03.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.807+0000 7f819a000640 1 -- 172.21.15.16:0/2889315433 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f81a002f030 con 0x562a5ee9b0a0 2024-08-31T06:06:03.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.807+0000 7f81bdc7b740 10 client.15114 fetch_fsmap finished waiting for FSMap version 24 2024-08-31T06:06:03.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.807+0000 7f81bdc7b740 10 client.15114 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-31T06:06:03.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.807+0000 7f81bdc7b740 20 client.15114 populate_metadata read hostname 'smithi016' 2024-08-31T06:06:03.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.807+0000 7f81bdc7b740 1 --2- 172.21.15.16:0/2889315433 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x562a5ee57c90 0x562a5f2f5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:03.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.807+0000 7f81bdc7b740 4 client.15114 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-31T06:06:03.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.807+0000 7f81bdc7b740 1 -- 172.21.15.16:0/2889315433 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "status"}) -- 0x562a5f1ddf40 con 0x562a5ee57c90 2024-08-31T06:06:03.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.808+0000 7f81af400640 1 --2- 172.21.15.16:0/2889315433 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x562a5ee57c90 0x562a5f2f5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:03.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.808+0000 7f81af400640 1 --2- 172.21.15.16:0/2889315433 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x562a5ee57c90 0x562a5f2f5c30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=496a28b88b8780bc server_cookie=b0f2c2c15fd81321 in_seq=0 out_seq=0 2024-08-31T06:06:03.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.808+0000 7f819a000640 10 client.15114 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:06:03.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.808+0000 7f819a000640 1 -- 172.21.15.16:0/2889315433 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x562a5f1ddf40 con 0x562a5ee57c90 2024-08-31T06:06:03.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.808+0000 7f819a000640 10 client.15114 handle_command_reply: tid=0 2024-08-31T06:06:03.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.808+0000 7f81bdc7b740 1 client.15114 shutdown 2024-08-31T06:06:03.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.809+0000 7f81bdc7b740 1 -- 172.21.15.16:0/2889315433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a5ee9b0a0 msgr2=0x562a5f28ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:03.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.809+0000 7f81bdc7b740 1 --2- 172.21.15.16:0/2889315433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a5ee9b0a0 0x562a5f28ea60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f81a0009e90 tx=0x7f81a0009990 comp rx=0 tx=0).stop 2024-08-31T06:06:03.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.809+0000 7f81bdc7b740 1 -- 172.21.15.16:0/2889315433 shutdown_connections 2024-08-31T06:06:03.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.809+0000 7f81bdc7b740 1 --2- 172.21.15.16:0/2889315433 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x562a5ee57c90 0x562a5f2f5c30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:03.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.809+0000 7f81bdc7b740 1 --2- 172.21.15.16:0/2889315433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a5eea3280 0x562a5f28f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:03.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.809+0000 7f81bdc7b740 1 --2- 172.21.15.16:0/2889315433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a5eec3f60 0x562a5f289020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:03.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.809+0000 7f81bdc7b740 1 --2- 172.21.15.16:0/2889315433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a5ee9b0a0 0x562a5f28ea60 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:03.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.809+0000 7f81bdc7b740 1 -- 172.21.15.16:0/2889315433 >> 172.21.15.16:0/2889315433 conn(0x562a5eeb8350 msgr2=0x562a5f274db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:03.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.809+0000 7f819a000640 0 client.15114 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:06:03.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.809+0000 7f81bdc7b740 1 -- 172.21.15.16:0/2889315433 shutdown_connections 2024-08-31T06:06:03.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.809+0000 7f81bdc7b740 1 -- 172.21.15.16:0/2889315433 wait complete. 2024-08-31T06:06:03.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.809+0000 7f81bdc7b740 20 client.15114 trim_cache size 0 max 16384 2024-08-31T06:06:03.815 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:06:03.815 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "673e4aa0-675d-11ef-bcd4-c7b262605968", 2024-08-31T06:06:03.815 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-08-31T06:06:03.815 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24737, 2024-08-31T06:06:03.815 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-08-31T06:06:03.815 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-08-31T06:06:03.815 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-08-31T06:06:03.815 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 95.930506319000003, 2024-08-31T06:06:03.816 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 24, 2024-08-31T06:06:03.816 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 80, 2024-08-31T06:06:03.816 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 78, 2024-08-31T06:06:03.816 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 110.48640233899999 2024-08-31T06:06:03.816 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:06:03.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.810+0000 7f81b7000640 1 -- 172.21.15.16:0/4192829719 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f81940b4c10 msgr2=0x7f81940b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:03.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.810+0000 7f81b7000640 1 --2- 172.21.15.16:0/4192829719 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f81940b4c10 0x7f81940b7120 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x562a5f268570 tx=0x7f81a0005f90 comp rx=0 tx=0).stop 2024-08-31T06:06:03.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.810+0000 7f81b7000640 1 -- 172.21.15.16:0/4192829719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81b0075ea0 msgr2=0x7f81b019c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:03.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.810+0000 7f81b7000640 1 --2- 172.21.15.16:0/4192829719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81b0075ea0 0x7f81b019c960 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f819c00d770 tx=0x7f819c00dc30 comp rx=0 tx=0).stop 2024-08-31T06:06:03.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.810+0000 7f81b7000640 1 -- 172.21.15.16:0/4192829719 shutdown_connections 2024-08-31T06:06:03.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.810+0000 7f81b7000640 1 --2- 172.21.15.16:0/4192829719 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f81940b4c10 0x7f81940b7120 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:03.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.810+0000 7f81b7000640 1 --2- 172.21.15.16:0/4192829719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81b010fe80 0x7f81b01a1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:03.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.810+0000 7f81b7000640 1 --2- 172.21.15.16:0/4192829719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81b0075ea0 0x7f81b019c960 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:03.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.810+0000 7f81b7000640 1 --2- 172.21.15.16:0/4192829719 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81b0075550 0x7f81b019c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:03.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.810+0000 7f81b7000640 1 -- 172.21.15.16:0/4192829719 >> 172.21.15.16:0/4192829719 conn(0x7f81b00ff370 msgr2=0x7f81b01099b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:03.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.810+0000 7f81b7000640 1 -- 172.21.15.16:0/4192829719 shutdown_connections 2024-08-31T06:06:03.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:03.810+0000 7f81b7000640 1 -- 172.21.15.16:0/4192829719 wait complete. 2024-08-31T06:06:04.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:04 smithi016 ceph-mon[17194]: pgmap v328: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-31T06:06:04.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:04 smithi184 ceph-mon[22331]: pgmap v328: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-31T06:06:04.833 INFO:tasks.cephadm:Setup complete, yielding 2024-08-31T06:06:04.834 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-08-31T06:06:04.844 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-08-31T06:06:04.844 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- bash -c 'ceph orch status' 2024-08-31T06:06:04.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:04 smithi155 ceph-mon[21742]: pgmap v328: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-31T06:06:05.225 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:06:05.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:05.874+0000 7f3e0b800640 1 -- 172.21.15.16:0/2407074722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e04109b90 msgr2=0x7f3e041103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:05.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:05.874+0000 7f3e0b800640 1 --2- 172.21.15.16:0/2407074722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e04109b90 0x7f3e041103c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc009ca0 tx=0x7f3dfc02a8c0 comp rx=0 tx=0).stop 2024-08-31T06:06:05.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:05.874+0000 7f3e0b800640 1 -- 172.21.15.16:0/2407074722 shutdown_connections 2024-08-31T06:06:05.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:05.874+0000 7f3e0b800640 1 --2- 172.21.15.16:0/2407074722 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e04109240 0x7f3e04109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:05.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:05.874+0000 7f3e0b800640 1 --2- 172.21.15.16:0/2407074722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e04109b90 0x7f3e041103c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:05.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:05.874+0000 7f3e0b800640 1 --2- 172.21.15.16:0/2407074722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e04103900 0x7f3e04103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:05.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:05.874+0000 7f3e0b800640 1 -- 172.21.15.16:0/2407074722 >> 172.21.15.16:0/2407074722 conn(0x7f3e0407b5f0 msgr2=0x7f3e0407ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:05.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:05.875+0000 7f3e0b800640 1 -- 172.21.15.16:0/2407074722 shutdown_connections 2024-08-31T06:06:05.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:05.875+0000 7f3e0b800640 1 -- 172.21.15.16:0/2407074722 wait complete. 2024-08-31T06:06:05.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:05.876+0000 7f3e0b800640 1 Processor -- start 2024-08-31T06:06:05.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:05.876+0000 7f3e0b800640 1 -- start start 2024-08-31T06:06:05.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:05.876+0000 7f3e0b800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e04103900 0x7f3e040754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:05.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:05.876+0000 7f3e0b800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e04109240 0x7f3e040759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:05.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:05.876+0000 7f3e0b800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e04109b90 0x7f3e04075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:05.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:05.876+0000 7f3e0b800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3e04113850 con 0x7f3e04103900 2024-08-31T06:06:05.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:05.876+0000 7f3e0b800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3e041136d0 con 0x7f3e04109b90 2024-08-31T06:06:05.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:05.876+0000 7f3e0b800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3e041139d0 con 0x7f3e04109240 2024-08-31T06:06:05.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:05.876+0000 7f3e08e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e04103900 0x7f3e040754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:05.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:05.876+0000 7f3e08e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e04103900 0x7f3e040754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35682/0 (socket says 172.21.15.16:35682) 2024-08-31T06:06:05.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:05.876+0000 7f3e08e00640 1 -- 172.21.15.16:0/758318122 learned_addr learned my addr 172.21.15.16:0/758318122 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:05.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:05.877+0000 7f3dfbe00640 1 --2- 172.21.15.16:0/758318122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e04109240 0x7f3e040759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:05.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:05.877+0000 7f3e09800640 1 --2- 172.21.15.16:0/758318122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e04109b90 0x7f3e04075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:05.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:05.877+0000 7f3e08e00640 1 -- 172.21.15.16:0/758318122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e04109240 msgr2=0x7f3e040759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:05.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:05.877+0000 7f3e08e00640 1 --2- 172.21.15.16:0/758318122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e04109240 0x7f3e040759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:05.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:05.877+0000 7f3e08e00640 1 -- 172.21.15.16:0/758318122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e04109b90 msgr2=0x7f3e04075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:05.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:05.877+0000 7f3e08e00640 1 --2- 172.21.15.16:0/758318122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e04109b90 0x7f3e04075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:05.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:05.877+0000 7f3e08e00640 1 -- 172.21.15.16:0/758318122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e041a84e0 con 0x7f3e04103900 2024-08-31T06:06:05.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:05.877+0000 7f3e08e00640 1 --2- 172.21.15.16:0/758318122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e04103900 0x7f3e040754b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3df400e9f0 tx=0x7f3df400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:05.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:05.878+0000 7f3e02000640 1 -- 172.21.15.16:0/758318122 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df400cd80 con 0x7f3e04103900 2024-08-31T06:06:05.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:05.878+0000 7f3e02000640 1 -- 172.21.15.16:0/758318122 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3df40108e0 con 0x7f3e04103900 2024-08-31T06:06:05.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:05.878+0000 7f3e0b800640 1 -- 172.21.15.16:0/758318122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e041a87d0 con 0x7f3e04103900 2024-08-31T06:06:05.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:05.878+0000 7f3e02000640 1 -- 172.21.15.16:0/758318122 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df4018980 con 0x7f3e04103900 2024-08-31T06:06:05.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:05.878+0000 7f3e0b800640 1 -- 172.21.15.16:0/758318122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e041a8d60 con 0x7f3e04103900 2024-08-31T06:06:05.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:05.879+0000 7f3e0b800640 1 -- 172.21.15.16:0/758318122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e04103d10 con 0x7f3e04103900 2024-08-31T06:06:05.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:05.881+0000 7f3e02000640 1 -- 172.21.15.16:0/758318122 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f3df4010430 con 0x7f3e04103900 2024-08-31T06:06:05.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:05.884+0000 7f3e02000640 1 --2- 172.21.15.16:0/758318122 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3de80b4c90 0x7f3de80b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:05.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:05.884+0000 7f3e02000640 1 -- 172.21.15.16:0/758318122 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f3df40e5b90 con 0x7f3e04103900 2024-08-31T06:06:05.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:05.884+0000 7f3dfbe00640 1 --2- 172.21.15.16:0/758318122 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3de80b4c90 0x7f3de80b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:05.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:05.885+0000 7f3dfbe00640 1 --2- 172.21.15.16:0/758318122 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3de80b4c90 0x7f3de80b71a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3e04106d80 tx=0x7f3dec005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:05.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:05.886+0000 7f3e02000640 1 -- 172.21.15.16:0/758318122 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f3df40ac3f0 con 0x7f3e04103900 2024-08-31T06:06:06.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:06.099+0000 7f3e0b800640 1 -- 172.21.15.16:0/758318122 --> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f3e040635a0 con 0x7f3de80b4c90 2024-08-31T06:06:06.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:06.101+0000 7f3e02000640 1 -- 172.21.15.16:0/758318122 <== mgr.14220 v2:172.21.15.16:6800/4279751337 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f3e040635a0 con 0x7f3de80b4c90 2024-08-31T06:06:06.102 INFO:teuthology.orchestra.run.smithi016.stdout:Backend: cephadm 2024-08-31T06:06:06.102 INFO:teuthology.orchestra.run.smithi016.stdout:Available: Yes 2024-08-31T06:06:06.102 INFO:teuthology.orchestra.run.smithi016.stdout:Paused: No 2024-08-31T06:06:06.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:06.103+0000 7f3e0b800640 1 -- 172.21.15.16:0/758318122 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3de80b4c90 msgr2=0x7f3de80b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:06.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:06.103+0000 7f3e0b800640 1 --2- 172.21.15.16:0/758318122 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3de80b4c90 0x7f3de80b71a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3e04106d80 tx=0x7f3dec005f90 comp rx=0 tx=0).stop 2024-08-31T06:06:06.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:06.103+0000 7f3e0b800640 1 -- 172.21.15.16:0/758318122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e04103900 msgr2=0x7f3e040754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:06.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:06.103+0000 7f3e0b800640 1 --2- 172.21.15.16:0/758318122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e04103900 0x7f3e040754b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3df400e9f0 tx=0x7f3df400eeb0 comp rx=0 tx=0).stop 2024-08-31T06:06:06.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:06.103+0000 7f3e0b800640 1 -- 172.21.15.16:0/758318122 shutdown_connections 2024-08-31T06:06:06.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:06.103+0000 7f3e0b800640 1 --2- 172.21.15.16:0/758318122 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3de80b4c90 0x7f3de80b71a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:06.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:06.103+0000 7f3e0b800640 1 --2- 172.21.15.16:0/758318122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e04109240 0x7f3e040759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:06.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:06.103+0000 7f3e0b800640 1 --2- 172.21.15.16:0/758318122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e04109b90 0x7f3e04075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:06.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:06.103+0000 7f3e0b800640 1 --2- 172.21.15.16:0/758318122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e04103900 0x7f3e040754b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:06.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:06.103+0000 7f3e0b800640 1 -- 172.21.15.16:0/758318122 >> 172.21.15.16:0/758318122 conn(0x7f3e0407b5f0 msgr2=0x7f3e041072d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:06.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:06.104+0000 7f3e0b800640 1 -- 172.21.15.16:0/758318122 shutdown_connections 2024-08-31T06:06:06.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:06.104+0000 7f3e0b800640 1 -- 172.21.15.16:0/758318122 wait complete. 2024-08-31T06:06:06.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:06 smithi016 ceph-mon[17194]: pgmap v329: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-31T06:06:06.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:06 smithi184 ceph-mon[22331]: pgmap v329: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-31T06:06:06.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:06 smithi155 ceph-mon[21742]: pgmap v329: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-31T06:06:06.983 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- bash -c 'ceph orch ps' 2024-08-31T06:06:07.362 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:06:07.654 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:07 smithi016 ceph-mon[17194]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:06:07.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:07 smithi184 ceph-mon[22331]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:06:07.773 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:07 smithi155 ceph-mon[21742]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:06:08.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.039+0000 7fb94a800640 1 -- 172.21.15.16:0/1100022467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb94410bdb0 msgr2=0x7fb9441125e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:08.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.039+0000 7fb94a800640 1 --2- 172.21.15.16:0/1100022467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb94410bdb0 0x7fb9441125e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb93c009ca0 tx=0x7fb93c02a8c0 comp rx=0 tx=0).stop 2024-08-31T06:06:08.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.039+0000 7fb94a800640 1 -- 172.21.15.16:0/1100022467 shutdown_connections 2024-08-31T06:06:08.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.039+0000 7fb94a800640 1 --2- 172.21.15.16:0/1100022467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb94410bdb0 0x7fb9441125e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:08.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.039+0000 7fb94a800640 1 --2- 172.21.15.16:0/1100022467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb94410b460 0x7fb94410b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:08.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.039+0000 7fb94a800640 1 --2- 172.21.15.16:0/1100022467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb944105b20 0x7fb944105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:08.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.039+0000 7fb94a800640 1 -- 172.21.15.16:0/1100022467 >> 172.21.15.16:0/1100022467 conn(0x7fb9441015b0 msgr2=0x7fb944103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:08.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.039+0000 7fb94a800640 1 -- 172.21.15.16:0/1100022467 shutdown_connections 2024-08-31T06:06:08.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.039+0000 7fb94a800640 1 -- 172.21.15.16:0/1100022467 wait complete. 2024-08-31T06:06:08.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.040+0000 7fb94a800640 1 Processor -- start 2024-08-31T06:06:08.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.041+0000 7fb94a800640 1 -- start start 2024-08-31T06:06:08.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.041+0000 7fb94a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb944105b20 0x7fb94419c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:08.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.041+0000 7fb94a800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb94410b460 0x7fb94419c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:08.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.041+0000 7fb94a800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb94410bdb0 0x7fb9441a1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:08.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.041+0000 7fb94a800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb944115a70 con 0x7fb944105b20 2024-08-31T06:06:08.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.041+0000 7fb94a800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb9441158f0 con 0x7fb94410b460 2024-08-31T06:06:08.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.041+0000 7fb94a800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb944115bf0 con 0x7fb94410bdb0 2024-08-31T06:06:08.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.041+0000 7fb943400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb944105b20 0x7fb94419c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:08.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.041+0000 7fb943400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb944105b20 0x7fb94419c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35712/0 (socket says 172.21.15.16:35712) 2024-08-31T06:06:08.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.041+0000 7fb943400640 1 -- 172.21.15.16:0/1763693314 learned_addr learned my addr 172.21.15.16:0/1763693314 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:08.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.041+0000 7fb93be00640 1 --2- 172.21.15.16:0/1763693314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb94410b460 0x7fb94419c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:08.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.042+0000 7fb943e00640 1 --2- 172.21.15.16:0/1763693314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb94410bdb0 0x7fb9441a1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:08.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.042+0000 7fb943400640 1 -- 172.21.15.16:0/1763693314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb94410bdb0 msgr2=0x7fb9441a1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:08.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.042+0000 7fb943400640 1 --2- 172.21.15.16:0/1763693314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb94410bdb0 0x7fb9441a1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:08.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.042+0000 7fb943400640 1 -- 172.21.15.16:0/1763693314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb94410b460 msgr2=0x7fb94419c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:08.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.042+0000 7fb943400640 1 --2- 172.21.15.16:0/1763693314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb94410b460 0x7fb94419c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:08.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.042+0000 7fb943400640 1 -- 172.21.15.16:0/1763693314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9441a2350 con 0x7fb944105b20 2024-08-31T06:06:08.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.042+0000 7fb943e00640 1 --2- 172.21.15.16:0/1763693314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb94410bdb0 0x7fb9441a1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:06:08.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.042+0000 7fb93be00640 1 --2- 172.21.15.16:0/1763693314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb94410b460 0x7fb94419c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:06:08.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.042+0000 7fb943400640 1 --2- 172.21.15.16:0/1763693314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb944105b20 0x7fb94419c3e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb93400d950 tx=0x7fb93400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:08.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.042+0000 7fb940c00640 1 -- 172.21.15.16:0/1763693314 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb934017030 con 0x7fb944105b20 2024-08-31T06:06:08.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.043+0000 7fb94a800640 1 -- 172.21.15.16:0/1763693314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9441a2640 con 0x7fb944105b20 2024-08-31T06:06:08.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.043+0000 7fb94a800640 1 -- 172.21.15.16:0/1763693314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9441a8da0 con 0x7fb944105b20 2024-08-31T06:06:08.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.043+0000 7fb940c00640 1 -- 172.21.15.16:0/1763693314 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb934004020 con 0x7fb944105b20 2024-08-31T06:06:08.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.043+0000 7fb940c00640 1 -- 172.21.15.16:0/1763693314 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb934005650 con 0x7fb944105b20 2024-08-31T06:06:08.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.044+0000 7fb94a800640 1 -- 172.21.15.16:0/1763693314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb944105f30 con 0x7fb944105b20 2024-08-31T06:06:08.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.046+0000 7fb940c00640 1 -- 172.21.15.16:0/1763693314 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7fb934009a30 con 0x7fb944105b20 2024-08-31T06:06:08.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.049+0000 7fb940c00640 1 --2- 172.21.15.16:0/1763693314 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb9280b4c10 0x7fb9280b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:08.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.049+0000 7fb940c00640 1 -- 172.21.15.16:0/1763693314 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7fb9340e4f00 con 0x7fb944105b20 2024-08-31T06:06:08.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.049+0000 7fb93be00640 1 --2- 172.21.15.16:0/1763693314 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb9280b4c10 0x7fb9280b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:08.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.050+0000 7fb93be00640 1 --2- 172.21.15.16:0/1763693314 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb9280b4c10 0x7fb9280b7120 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb944104e00 tx=0x7fb92c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:08.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.051+0000 7fb940c00640 1 -- 172.21.15.16:0/1763693314 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fb934022020 con 0x7fb944105b20 2024-08-31T06:06:08.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.257+0000 7fb94a800640 1 -- 172.21.15.16:0/1763693314 --> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fb9440635a0 con 0x7fb9280b4c10 2024-08-31T06:06:08.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.270+0000 7fb940c00640 1 -- 172.21.15.16:0/1763693314 <== mgr.14220 v2:172.21.15.16:6800/4279751337 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7fb9440635a0 con 0x7fb9280b4c10 2024-08-31T06:06:08.273 INFO:teuthology.orchestra.run.smithi016.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-08-31T06:06:08.273 INFO:teuthology.orchestra.run.smithi016.stdout:mds.a smithi016 running (118s) 23s ago 117s 80.3M - 19.3.0-4560-gd19e6118 79f4cde5e369 d86cbad71303 2024-08-31T06:06:08.273 INFO:teuthology.orchestra.run.smithi016.stdout:mds.b smithi155 running (115s) 45s ago 114s 96.2M - 19.3.0-4560-gd19e6118 79f4cde5e369 0830ee5e87b0 2024-08-31T06:06:08.273 INFO:teuthology.orchestra.run.smithi016.stdout:mds.c smithi184 running (2m) 45s ago 2m 84.2M - 19.3.0-4560-gd19e6118 79f4cde5e369 8f0a89efbce5 2024-08-31T06:06:08.273 INFO:teuthology.orchestra.run.smithi016.stdout:mds.d smithi016 running (2m) 23s ago 2m 80.4M - 19.3.0-4560-gd19e6118 79f4cde5e369 08bfaf111101 2024-08-31T06:06:08.273 INFO:teuthology.orchestra.run.smithi016.stdout:mds.e smithi155 running (2m) 45s ago 2m 81.8M - 19.3.0-4560-gd19e6118 79f4cde5e369 d7b37624caef 2024-08-31T06:06:08.273 INFO:teuthology.orchestra.run.smithi016.stdout:mds.f smithi184 running (2m) 45s ago 2m 83.4M - 19.3.0-4560-gd19e6118 79f4cde5e369 d162f2b072e8 2024-08-31T06:06:08.273 INFO:teuthology.orchestra.run.smithi016.stdout:mds.g smithi016 running (2m) 23s ago 2m 84.5M - 19.3.0-4560-gd19e6118 79f4cde5e369 52f052d8802b 2024-08-31T06:06:08.273 INFO:teuthology.orchestra.run.smithi016.stdout:mds.h smithi155 running (2m) 45s ago 2m 87.7M - 19.3.0-4560-gd19e6118 79f4cde5e369 f74a94b68e1d 2024-08-31T06:06:08.273 INFO:teuthology.orchestra.run.smithi016.stdout:mds.i smithi184 running (2m) 45s ago 2m 86.0M - 19.3.0-4560-gd19e6118 79f4cde5e369 bbb227bfc2fa 2024-08-31T06:06:08.273 INFO:teuthology.orchestra.run.smithi016.stdout:mds.j smithi016 running (2m) 23s ago 2m 80.5M - 19.3.0-4560-gd19e6118 79f4cde5e369 66f8fc11e80c 2024-08-31T06:06:08.273 INFO:teuthology.orchestra.run.smithi016.stdout:mds.k smithi155 running (2m) 45s ago 2m 84.5M - 19.3.0-4560-gd19e6118 79f4cde5e369 e3c24c5dbb46 2024-08-31T06:06:08.274 INFO:teuthology.orchestra.run.smithi016.stdout:mds.l smithi184 running (2m) 45s ago 2m 82.1M - 19.3.0-4560-gd19e6118 79f4cde5e369 2eff53ca9a13 2024-08-31T06:06:08.274 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.x smithi016 *:9283,8765 running (10m) 23s ago 10m 585M - 19.3.0-4560-gd19e6118 79f4cde5e369 a9111fa496cb 2024-08-31T06:06:08.274 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.y smithi155 *:8443,8765 running (8m) 45s ago 8m 522M - 19.3.0-4560-gd19e6118 79f4cde5e369 2368cdbe66a4 2024-08-31T06:06:08.274 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.z smithi184 *:8443,8765 running (8m) 45s ago 8m 523M - 19.3.0-4560-gd19e6118 79f4cde5e369 4154939d0c3b 2024-08-31T06:06:08.274 INFO:teuthology.orchestra.run.smithi016.stdout:mon.a smithi016 running (10m) 23s ago 10m 116M 2048M 19.3.0-4560-gd19e6118 79f4cde5e369 b27650bd7483 2024-08-31T06:06:08.274 INFO:teuthology.orchestra.run.smithi016.stdout:mon.b smithi155 running (9m) 45s ago 9m 113M 2048M 19.3.0-4560-gd19e6118 79f4cde5e369 d41e849cd3cb 2024-08-31T06:06:08.274 INFO:teuthology.orchestra.run.smithi016.stdout:mon.c smithi184 running (9m) 45s ago 9m 113M 2048M 19.3.0-4560-gd19e6118 79f4cde5e369 cc3de2bf8812 2024-08-31T06:06:08.274 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 smithi016 running (7m) 23s ago 7m 122M 4273M 19.3.0-4560-gd19e6118 79f4cde5e369 8f392c40d13c 2024-08-31T06:06:08.274 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 smithi155 running (7m) 45s ago 7m 124M 4272M 19.3.0-4560-gd19e6118 79f4cde5e369 850d12528ba8 2024-08-31T06:06:08.274 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 smithi184 running (6m) 45s ago 6m 124M 4272M 19.3.0-4560-gd19e6118 79f4cde5e369 90887a957a46 2024-08-31T06:06:08.274 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 smithi016 running (6m) 23s ago 6m 109M 4273M 19.3.0-4560-gd19e6118 79f4cde5e369 00de9756ea21 2024-08-31T06:06:08.274 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 smithi155 running (5m) 45s ago 5m 119M 4272M 19.3.0-4560-gd19e6118 79f4cde5e369 c9fbde35ffdf 2024-08-31T06:06:08.274 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 smithi184 running (5m) 45s ago 5m 110M 4272M 19.3.0-4560-gd19e6118 79f4cde5e369 e21ad848367e 2024-08-31T06:06:08.275 INFO:teuthology.orchestra.run.smithi016.stdout:osd.6 smithi016 running (5m) 23s ago 5m 108M 4273M 19.3.0-4560-gd19e6118 79f4cde5e369 db7f406efaa3 2024-08-31T06:06:08.275 INFO:teuthology.orchestra.run.smithi016.stdout:osd.7 smithi155 running (4m) 45s ago 4m 121M 4272M 19.3.0-4560-gd19e6118 79f4cde5e369 8582e224bf81 2024-08-31T06:06:08.275 INFO:teuthology.orchestra.run.smithi016.stdout:osd.8 smithi184 running (4m) 45s ago 4m 109M 4272M 19.3.0-4560-gd19e6118 79f4cde5e369 ddd6988e7500 2024-08-31T06:06:08.275 INFO:teuthology.orchestra.run.smithi016.stdout:osd.9 smithi016 running (3m) 23s ago 3m 107M 4273M 19.3.0-4560-gd19e6118 79f4cde5e369 2806ee4dcd91 2024-08-31T06:06:08.275 INFO:teuthology.orchestra.run.smithi016.stdout:osd.10 smithi155 running (3m) 45s ago 3m 110M 4272M 19.3.0-4560-gd19e6118 79f4cde5e369 ab393bcfbeca 2024-08-31T06:06:08.275 INFO:teuthology.orchestra.run.smithi016.stdout:osd.11 smithi184 running (2m) 45s ago 2m 107M 4272M 19.3.0-4560-gd19e6118 79f4cde5e369 be3c3a4cfb89 2024-08-31T06:06:08.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.273+0000 7fb94a800640 1 -- 172.21.15.16:0/1763693314 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb9280b4c10 msgr2=0x7fb9280b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:08.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.273+0000 7fb94a800640 1 --2- 172.21.15.16:0/1763693314 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb9280b4c10 0x7fb9280b7120 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb944104e00 tx=0x7fb92c005f90 comp rx=0 tx=0).stop 2024-08-31T06:06:08.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.273+0000 7fb94a800640 1 -- 172.21.15.16:0/1763693314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb944105b20 msgr2=0x7fb94419c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:08.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.273+0000 7fb94a800640 1 --2- 172.21.15.16:0/1763693314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb944105b20 0x7fb94419c3e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb93400d950 tx=0x7fb93400de10 comp rx=0 tx=0).stop 2024-08-31T06:06:08.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.273+0000 7fb94a800640 1 -- 172.21.15.16:0/1763693314 shutdown_connections 2024-08-31T06:06:08.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.273+0000 7fb94a800640 1 --2- 172.21.15.16:0/1763693314 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb9280b4c10 0x7fb9280b7120 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:08.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.273+0000 7fb94a800640 1 --2- 172.21.15.16:0/1763693314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb94410bdb0 0x7fb9441a1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:08.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.273+0000 7fb94a800640 1 --2- 172.21.15.16:0/1763693314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb94410b460 0x7fb94419c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:08.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.273+0000 7fb94a800640 1 --2- 172.21.15.16:0/1763693314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb944105b20 0x7fb94419c3e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:08.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.273+0000 7fb94a800640 1 -- 172.21.15.16:0/1763693314 >> 172.21.15.16:0/1763693314 conn(0x7fb9441015b0 msgr2=0x7fb944108a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:08.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.273+0000 7fb94a800640 1 -- 172.21.15.16:0/1763693314 shutdown_connections 2024-08-31T06:06:08.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:08.273+0000 7fb94a800640 1 -- 172.21.15.16:0/1763693314 wait complete. 2024-08-31T06:06:08.626 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:08 smithi016 ceph-mon[17194]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-31T06:06:08.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:08 smithi184 ceph-mon[22331]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-31T06:06:08.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:08 smithi155 ceph-mon[21742]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-31T06:06:09.031 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- bash -c 'ceph orch ls' 2024-08-31T06:06:09.411 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:06:09.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:09 smithi016 ceph-mon[17194]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:06:09.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:09 smithi184 ceph-mon[22331]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:06:09.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:09 smithi155 ceph-mon[21742]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:06:10.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.085+0000 7ff632600640 1 -- 172.21.15.16:0/2774925426 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff62c10dc70 msgr2=0x7ff62c110180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:10.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.085+0000 7ff632600640 1 --2- 172.21.15.16:0/2774925426 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff62c10dc70 0x7ff62c110180 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff624009ca0 tx=0x7ff62402a8c0 comp rx=0 tx=0).stop 2024-08-31T06:06:10.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.085+0000 7ff632600640 1 -- 172.21.15.16:0/2774925426 shutdown_connections 2024-08-31T06:06:10.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.085+0000 7ff632600640 1 --2- 172.21.15.16:0/2774925426 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff62c10dc70 0x7ff62c110180 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:10.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.085+0000 7ff632600640 1 --2- 172.21.15.16:0/2774925426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff62c1050f0 0x7ff62c10d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:10.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.085+0000 7ff632600640 1 --2- 172.21.15.16:0/2774925426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff62c1046c0 0x7ff62c104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:10.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.085+0000 7ff632600640 1 -- 172.21.15.16:0/2774925426 >> 172.21.15.16:0/2774925426 conn(0x7ff62c0794f0 msgr2=0x7ff62c079900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:10.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.085+0000 7ff632600640 1 -- 172.21.15.16:0/2774925426 shutdown_connections 2024-08-31T06:06:10.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.085+0000 7ff632600640 1 -- 172.21.15.16:0/2774925426 wait complete. 2024-08-31T06:06:10.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.086+0000 7ff632600640 1 Processor -- start 2024-08-31T06:06:10.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.086+0000 7ff632600640 1 -- start start 2024-08-31T06:06:10.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.086+0000 7ff632600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff62c1046c0 0x7ff62c19c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:10.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.087+0000 7ff632600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff62c1050f0 0x7ff62c19c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:10.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.087+0000 7ff632600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff62c10dc70 0x7ff62c1a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:10.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.087+0000 7ff632600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff62c113870 con 0x7ff62c1046c0 2024-08-31T06:06:10.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.087+0000 7ff632600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff62c1136f0 con 0x7ff62c1050f0 2024-08-31T06:06:10.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.087+0000 7ff632600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff62c1139f0 con 0x7ff62c10dc70 2024-08-31T06:06:10.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.087+0000 7ff62b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff62c1046c0 0x7ff62c19c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:10.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.087+0000 7ff623e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff62c1050f0 0x7ff62c19c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:10.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.087+0000 7ff623e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff62c1050f0 0x7ff62c19c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:54366/0 (socket says 172.21.15.16:54366) 2024-08-31T06:06:10.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.087+0000 7ff623e00640 1 -- 172.21.15.16:0/2246886157 learned_addr learned my addr 172.21.15.16:0/2246886157 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:10.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.087+0000 7ff62be00640 1 --2- 172.21.15.16:0/2246886157 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff62c10dc70 0x7ff62c1a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:10.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.087+0000 7ff623e00640 1 -- 172.21.15.16:0/2246886157 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff62c10dc70 msgr2=0x7ff62c1a1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:10.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.087+0000 7ff623e00640 1 --2- 172.21.15.16:0/2246886157 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff62c10dc70 0x7ff62c1a1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:10.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.087+0000 7ff623e00640 1 -- 172.21.15.16:0/2246886157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff62c1046c0 msgr2=0x7ff62c19c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:10.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.087+0000 7ff623e00640 1 --2- 172.21.15.16:0/2246886157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff62c1046c0 0x7ff62c19c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:10.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.087+0000 7ff623e00640 1 -- 172.21.15.16:0/2246886157 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff62c1a23b0 con 0x7ff62c1050f0 2024-08-31T06:06:10.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.088+0000 7ff623e00640 1 --2- 172.21.15.16:0/2246886157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff62c1050f0 0x7ff62c19c9e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff61400e9c0 tx=0x7ff61400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:10.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.088+0000 7ff628c00640 1 -- 172.21.15.16:0/2246886157 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61400ccc0 con 0x7ff62c1050f0 2024-08-31T06:06:10.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.088+0000 7ff632600640 1 -- 172.21.15.16:0/2246886157 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff62c1a26a0 con 0x7ff62c1050f0 2024-08-31T06:06:10.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.088+0000 7ff628c00640 1 -- 172.21.15.16:0/2246886157 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6140054d0 con 0x7ff62c1050f0 2024-08-31T06:06:10.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.088+0000 7ff628c00640 1 -- 172.21.15.16:0/2246886157 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff614010430 con 0x7ff62c1050f0 2024-08-31T06:06:10.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.088+0000 7ff632600640 1 -- 172.21.15.16:0/2246886157 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff62c1a8e00 con 0x7ff62c1050f0 2024-08-31T06:06:10.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.089+0000 7ff632600640 1 -- 172.21.15.16:0/2246886157 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff62c06bf80 con 0x7ff62c1050f0 2024-08-31T06:06:10.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.092+0000 7ff628c00640 1 -- 172.21.15.16:0/2246886157 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7ff614005000 con 0x7ff62c1050f0 2024-08-31T06:06:10.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.093+0000 7ff628c00640 1 --2- 172.21.15.16:0/2246886157 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff6100b4c10 0x7ff6100b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:10.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.093+0000 7ff628c00640 1 -- 172.21.15.16:0/2246886157 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7ff6140e5920 con 0x7ff62c1050f0 2024-08-31T06:06:10.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.093+0000 7ff62b400640 1 --2- 172.21.15.16:0/2246886157 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff6100b4c10 0x7ff6100b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:10.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.094+0000 7ff62b400640 1 --2- 172.21.15.16:0/2246886157 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff6100b4c10 0x7ff6100b7120 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff61c0095f0 tx=0x7ff61c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:10.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.095+0000 7ff628c00640 1 -- 172.21.15.16:0/2246886157 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7ff614056130 con 0x7ff62c1050f0 2024-08-31T06:06:10.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.298+0000 7ff632600640 1 -- 172.21.15.16:0/2246886157 --> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7ff62c0ffa30 con 0x7ff6100b4c10 2024-08-31T06:06:10.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.304+0000 7ff628c00640 1 -- 172.21.15.16:0/2246886157 <== mgr.14220 v2:172.21.15.16:6800/4279751337 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7ff62c0ffa30 con 0x7ff6100b4c10 2024-08-31T06:06:10.307 INFO:teuthology.orchestra.run.smithi016.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-08-31T06:06:10.307 INFO:teuthology.orchestra.run.smithi016.stdout:mds.all 12/12 47s ago 2m smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l;count:12 2024-08-31T06:06:10.307 INFO:teuthology.orchestra.run.smithi016.stdout:mgr 3/3 47s ago 8m smithi016=x;smithi155=y;smithi184=z;count:3 2024-08-31T06:06:10.307 INFO:teuthology.orchestra.run.smithi016.stdout:mon 3/3 47s ago 9m smithi016:172.21.15.16=a;smithi155:172.21.15.155=b;smithi184:172.21.15.184=c;count:3 2024-08-31T06:06:10.307 INFO:teuthology.orchestra.run.smithi016.stdout:osd 12 47s ago - 2024-08-31T06:06:10.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.307+0000 7ff632600640 1 -- 172.21.15.16:0/2246886157 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff6100b4c10 msgr2=0x7ff6100b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:10.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.307+0000 7ff632600640 1 --2- 172.21.15.16:0/2246886157 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff6100b4c10 0x7ff6100b7120 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff61c0095f0 tx=0x7ff61c005f90 comp rx=0 tx=0).stop 2024-08-31T06:06:10.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.307+0000 7ff632600640 1 -- 172.21.15.16:0/2246886157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff62c1050f0 msgr2=0x7ff62c19c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:10.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.307+0000 7ff632600640 1 --2- 172.21.15.16:0/2246886157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff62c1050f0 0x7ff62c19c9e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff61400e9c0 tx=0x7ff61400ee80 comp rx=0 tx=0).stop 2024-08-31T06:06:10.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.307+0000 7ff632600640 1 -- 172.21.15.16:0/2246886157 shutdown_connections 2024-08-31T06:06:10.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.307+0000 7ff632600640 1 --2- 172.21.15.16:0/2246886157 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff6100b4c10 0x7ff6100b7120 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:10.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.307+0000 7ff632600640 1 --2- 172.21.15.16:0/2246886157 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff62c10dc70 0x7ff62c1a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:10.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.307+0000 7ff632600640 1 --2- 172.21.15.16:0/2246886157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff62c1050f0 0x7ff62c19c9e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:10.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.307+0000 7ff632600640 1 --2- 172.21.15.16:0/2246886157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff62c1046c0 0x7ff62c19c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:10.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.307+0000 7ff632600640 1 -- 172.21.15.16:0/2246886157 >> 172.21.15.16:0/2246886157 conn(0x7ff62c0794f0 msgr2=0x7ff62c0ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:10.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.307+0000 7ff632600640 1 -- 172.21.15.16:0/2246886157 shutdown_connections 2024-08-31T06:06:10.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:10.307+0000 7ff632600640 1 -- 172.21.15.16:0/2246886157 wait complete. 2024-08-31T06:06:10.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:10 smithi016 ceph-mon[17194]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-31T06:06:10.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:10 smithi184 ceph-mon[22331]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-31T06:06:10.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:10 smithi155 ceph-mon[21742]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-31T06:06:11.277 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- bash -c 'ceph orch host ls' 2024-08-31T06:06:11.574 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:11 smithi016 ceph-mon[17194]: from='client.25136 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:06:11.667 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:06:11.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:11 smithi184 ceph-mon[22331]: from='client.25136 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:06:11.871 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:11 smithi155 ceph-mon[21742]: from='client.25136 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:06:12.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.303+0000 7f7849a00640 1 -- 172.21.15.16:0/397376338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f784410b460 msgr2=0x7f784410b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:12.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.303+0000 7f7849a00640 1 --2- 172.21.15.16:0/397376338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f784410b460 0x7f784410b870 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f782c009900 tx=0x7f782c02a3f0 comp rx=0 tx=0).stop 2024-08-31T06:06:12.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.303+0000 7f7849a00640 1 -- 172.21.15.16:0/397376338 shutdown_connections 2024-08-31T06:06:12.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.303+0000 7f7849a00640 1 --2- 172.21.15.16:0/397376338 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f784410bdb0 0x7f78441125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:12.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.303+0000 7f7849a00640 1 --2- 172.21.15.16:0/397376338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f784410b460 0x7f784410b870 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:12.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.303+0000 7f7849a00640 1 --2- 172.21.15.16:0/397376338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7844105b20 0x7f7844105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:12.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.303+0000 7f7849a00640 1 -- 172.21.15.16:0/397376338 >> 172.21.15.16:0/397376338 conn(0x7f78441015b0 msgr2=0x7f7844103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:12.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.303+0000 7f7849a00640 1 -- 172.21.15.16:0/397376338 shutdown_connections 2024-08-31T06:06:12.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.303+0000 7f7849a00640 1 -- 172.21.15.16:0/397376338 wait complete. 2024-08-31T06:06:12.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.305+0000 7f7849a00640 1 Processor -- start 2024-08-31T06:06:12.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.305+0000 7f7849a00640 1 -- start start 2024-08-31T06:06:12.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.305+0000 7f7849a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7844105b20 0x7f784419c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:12.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.305+0000 7f7849a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f784410b460 0x7f784419ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:12.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.306+0000 7f7849a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f784410bdb0 0x7f78441a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:12.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.306+0000 7f7849a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7844115ac0 con 0x7f784410b460 2024-08-31T06:06:12.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.306+0000 7f7849a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7844115940 con 0x7f7844105b20 2024-08-31T06:06:12.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.306+0000 7f7849a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7844115c40 con 0x7f784410bdb0 2024-08-31T06:06:12.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.306+0000 7f783be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f784410b460 0x7f784419ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:12.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.306+0000 7f7842a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7844105b20 0x7f784419c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:12.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.306+0000 7f783be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f784410b460 0x7f784419ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35750/0 (socket says 172.21.15.16:35750) 2024-08-31T06:06:12.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.306+0000 7f7842a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7844105b20 0x7f784419c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:54380/0 (socket says 172.21.15.16:54380) 2024-08-31T06:06:12.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.306+0000 7f783be00640 1 -- 172.21.15.16:0/325487886 learned_addr learned my addr 172.21.15.16:0/325487886 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:12.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.306+0000 7f7843400640 1 --2- 172.21.15.16:0/325487886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f784410bdb0 0x7f78441a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:12.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.306+0000 7f7842a00640 1 -- 172.21.15.16:0/325487886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f784410bdb0 msgr2=0x7f78441a1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:12.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.306+0000 7f7842a00640 1 --2- 172.21.15.16:0/325487886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f784410bdb0 0x7f78441a1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:12.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.306+0000 7f7842a00640 1 -- 172.21.15.16:0/325487886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f784410b460 msgr2=0x7f784419ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:12.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.306+0000 7f7842a00640 1 --2- 172.21.15.16:0/325487886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f784410b460 0x7f784419ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:12.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.306+0000 7f7842a00640 1 -- 172.21.15.16:0/325487886 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78441a23e0 con 0x7f7844105b20 2024-08-31T06:06:12.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.307+0000 7f7842a00640 1 --2- 172.21.15.16:0/325487886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7844105b20 0x7f784419c470 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f783400e9f0 tx=0x7f783400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:12.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.307+0000 7f783b400640 1 -- 172.21.15.16:0/325487886 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783400ccc0 con 0x7f7844105b20 2024-08-31T06:06:12.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.307+0000 7f7849a00640 1 -- 172.21.15.16:0/325487886 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78441a26d0 con 0x7f7844105b20 2024-08-31T06:06:12.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.307+0000 7f783b400640 1 -- 172.21.15.16:0/325487886 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78340108e0 con 0x7f7844105b20 2024-08-31T06:06:12.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.307+0000 7f783b400640 1 -- 172.21.15.16:0/325487886 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7834018880 con 0x7f7844105b20 2024-08-31T06:06:12.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.307+0000 7f7849a00640 1 -- 172.21.15.16:0/325487886 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78441a8e30 con 0x7f7844105b20 2024-08-31T06:06:12.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.308+0000 7f7849a00640 1 -- 172.21.15.16:0/325487886 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7844105f30 con 0x7f7844105b20 2024-08-31T06:06:12.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.311+0000 7f783b400640 1 -- 172.21.15.16:0/325487886 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f7834010430 con 0x7f7844105b20 2024-08-31T06:06:12.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.312+0000 7f783b400640 1 --2- 172.21.15.16:0/325487886 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f78280b4c10 0x7f78280b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:12.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.312+0000 7f783b400640 1 -- 172.21.15.16:0/325487886 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f78340e5c60 con 0x7f7844105b20 2024-08-31T06:06:12.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.312+0000 7f783be00640 1 --2- 172.21.15.16:0/325487886 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f78280b4c10 0x7f78280b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:12.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.313+0000 7f783be00640 1 --2- 172.21.15.16:0/325487886 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f78280b4c10 0x7f78280b7120 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f782c006010 tx=0x7f782c005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:12.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.314+0000 7f783b400640 1 -- 172.21.15.16:0/325487886 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f78340ac4c0 con 0x7f7844105b20 2024-08-31T06:06:12.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.516+0000 7f7849a00640 1 -- 172.21.15.16:0/325487886 --> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f78440635a0 con 0x7f78280b4c10 2024-08-31T06:06:12.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.518+0000 7f783b400640 1 -- 172.21.15.16:0/325487886 <== mgr.14220 v2:172.21.15.16:6800/4279751337 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f78440635a0 con 0x7f78280b4c10 2024-08-31T06:06:12.520 INFO:teuthology.orchestra.run.smithi016.stdout:HOST ADDR LABELS STATUS 2024-08-31T06:06:12.520 INFO:teuthology.orchestra.run.smithi016.stdout:smithi016 172.21.15.16 2024-08-31T06:06:12.521 INFO:teuthology.orchestra.run.smithi016.stdout:smithi155 172.21.15.155 2024-08-31T06:06:12.521 INFO:teuthology.orchestra.run.smithi016.stdout:smithi184 172.21.15.184 2024-08-31T06:06:12.521 INFO:teuthology.orchestra.run.smithi016.stdout:3 hosts in cluster 2024-08-31T06:06:12.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.520+0000 7f7849a00640 1 -- 172.21.15.16:0/325487886 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f78280b4c10 msgr2=0x7f78280b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:12.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.520+0000 7f7849a00640 1 --2- 172.21.15.16:0/325487886 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f78280b4c10 0x7f78280b7120 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f782c006010 tx=0x7f782c005a20 comp rx=0 tx=0).stop 2024-08-31T06:06:12.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.520+0000 7f7849a00640 1 -- 172.21.15.16:0/325487886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7844105b20 msgr2=0x7f784419c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:12.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.520+0000 7f7849a00640 1 --2- 172.21.15.16:0/325487886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7844105b20 0x7f784419c470 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f783400e9f0 tx=0x7f783400eeb0 comp rx=0 tx=0).stop 2024-08-31T06:06:12.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.520+0000 7f7849a00640 1 -- 172.21.15.16:0/325487886 shutdown_connections 2024-08-31T06:06:12.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.520+0000 7f7849a00640 1 --2- 172.21.15.16:0/325487886 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f78280b4c10 0x7f78280b7120 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:12.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.520+0000 7f7849a00640 1 --2- 172.21.15.16:0/325487886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f784410bdb0 0x7f78441a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:12.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.520+0000 7f7849a00640 1 --2- 172.21.15.16:0/325487886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f784410b460 0x7f784419ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:12.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.520+0000 7f7849a00640 1 --2- 172.21.15.16:0/325487886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7844105b20 0x7f784419c470 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:12.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.520+0000 7f7849a00640 1 -- 172.21.15.16:0/325487886 >> 172.21.15.16:0/325487886 conn(0x7f78441015b0 msgr2=0x7f78441039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:12.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.520+0000 7f7849a00640 1 -- 172.21.15.16:0/325487886 shutdown_connections 2024-08-31T06:06:12.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:12.520+0000 7f7849a00640 1 -- 172.21.15.16:0/325487886 wait complete. 2024-08-31T06:06:12.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:12 smithi016 ceph-mon[17194]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-31T06:06:12.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:12 smithi184 ceph-mon[22331]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-31T06:06:12.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:12 smithi155 ceph-mon[21742]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-31T06:06:13.429 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- bash -c 'ceph orch device ls' 2024-08-31T06:06:13.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:13 smithi016 ceph-mon[17194]: from='client.25142 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:06:13.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:13 smithi184 ceph-mon[22331]: from='client.25142 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:06:13.804 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:06:13.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:13 smithi155 ceph-mon[21742]: from='client.25142 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:06:14.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.427+0000 7f6b24200640 1 -- 172.21.15.16:0/1303787597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b1c10fec0 msgr2=0x7f6b1c1122e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:14.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.427+0000 7f6b24200640 1 --2- 172.21.15.16:0/1303787597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b1c10fec0 0x7f6b1c1122e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6b14009ca0 tx=0x7f6b1402a8c0 comp rx=0 tx=0).stop 2024-08-31T06:06:14.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.428+0000 7f6b24200640 1 -- 172.21.15.16:0/1303787597 shutdown_connections 2024-08-31T06:06:14.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.428+0000 7f6b24200640 1 --2- 172.21.15.16:0/1303787597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b1c075e40 0x7f6b1c10f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:14.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.428+0000 7f6b24200640 1 --2- 172.21.15.16:0/1303787597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b1c10fec0 0x7f6b1c1122e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:14.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.428+0000 7f6b24200640 1 --2- 172.21.15.16:0/1303787597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b1c0754f0 0x7f6b1c075900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:14.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.428+0000 7f6b24200640 1 -- 172.21.15.16:0/1303787597 >> 172.21.15.16:0/1303787597 conn(0x7f6b1c07b5f0 msgr2=0x7f6b1c07ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:14.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.428+0000 7f6b24200640 1 -- 172.21.15.16:0/1303787597 shutdown_connections 2024-08-31T06:06:14.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.428+0000 7f6b24200640 1 -- 172.21.15.16:0/1303787597 wait complete. 2024-08-31T06:06:14.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.429+0000 7f6b24200640 1 Processor -- start 2024-08-31T06:06:14.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.429+0000 7f6b24200640 1 -- start start 2024-08-31T06:06:14.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.429+0000 7f6b24200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b1c0754f0 0x7f6b1c19c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:14.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.429+0000 7f6b24200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b1c075e40 0x7f6b1c19c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:14.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.429+0000 7f6b24200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b1c10fec0 0x7f6b1c1a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:14.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.429+0000 7f6b24200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1c115a30 con 0x7f6b1c075e40 2024-08-31T06:06:14.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.429+0000 7f6b24200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1c1158b0 con 0x7f6b1c10fec0 2024-08-31T06:06:14.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.429+0000 7f6b24200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1c115bb0 con 0x7f6b1c0754f0 2024-08-31T06:06:14.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.430+0000 7f6b20e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b1c075e40 0x7f6b1c19c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:14.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.430+0000 7f6b20e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b1c075e40 0x7f6b1c19c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35768/0 (socket says 172.21.15.16:35768) 2024-08-31T06:06:14.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.430+0000 7f6b20e00640 1 -- 172.21.15.16:0/2133341602 learned_addr learned my addr 172.21.15.16:0/2133341602 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:14.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.430+0000 7f6b21800640 1 --2- 172.21.15.16:0/2133341602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b1c0754f0 0x7f6b1c19c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:14.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.430+0000 7f6b22200640 1 --2- 172.21.15.16:0/2133341602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b1c10fec0 0x7f6b1c1a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:14.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.430+0000 7f6b20e00640 1 -- 172.21.15.16:0/2133341602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b1c0754f0 msgr2=0x7f6b1c19c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:14.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.430+0000 7f6b20e00640 1 --2- 172.21.15.16:0/2133341602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b1c0754f0 0x7f6b1c19c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:14.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.430+0000 7f6b20e00640 1 -- 172.21.15.16:0/2133341602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b1c10fec0 msgr2=0x7f6b1c1a1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:14.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.430+0000 7f6b20e00640 1 --2- 172.21.15.16:0/2133341602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b1c10fec0 0x7f6b1c1a1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:14.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.430+0000 7f6b20e00640 1 -- 172.21.15.16:0/2133341602 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b1c1a2330 con 0x7f6b1c075e40 2024-08-31T06:06:14.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.430+0000 7f6b21800640 1 --2- 172.21.15.16:0/2133341602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b1c0754f0 0x7f6b1c19c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:06:14.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.430+0000 7f6b20e00640 1 --2- 172.21.15.16:0/2133341602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b1c075e40 0x7f6b1c19c960 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800e9c0 tx=0x7f6b0800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:14.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.431+0000 7f6b1a000640 1 -- 172.21.15.16:0/2133341602 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0800ccc0 con 0x7f6b1c075e40 2024-08-31T06:06:14.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.431+0000 7f6b1a000640 1 -- 172.21.15.16:0/2133341602 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b080054d0 con 0x7f6b1c075e40 2024-08-31T06:06:14.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.431+0000 7f6b1a000640 1 -- 172.21.15.16:0/2133341602 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b08010430 con 0x7f6b1c075e40 2024-08-31T06:06:14.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.431+0000 7f6b24200640 1 -- 172.21.15.16:0/2133341602 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b1c1a2620 con 0x7f6b1c075e40 2024-08-31T06:06:14.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.431+0000 7f6b24200640 1 -- 172.21.15.16:0/2133341602 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b1c1a8d80 con 0x7f6b1c075e40 2024-08-31T06:06:14.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.432+0000 7f6b24200640 1 -- 172.21.15.16:0/2133341602 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b1c075900 con 0x7f6b1c075e40 2024-08-31T06:06:14.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.434+0000 7f6b1a000640 1 -- 172.21.15.16:0/2133341602 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f6b08005020 con 0x7f6b1c075e40 2024-08-31T06:06:14.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.436+0000 7f6b1a000640 1 --2- 172.21.15.16:0/2133341602 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6b040b4c90 0x7f6b040b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:14.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.436+0000 7f6b1a000640 1 -- 172.21.15.16:0/2133341602 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f6b080e56f0 con 0x7f6b1c075e40 2024-08-31T06:06:14.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.436+0000 7f6b21800640 1 --2- 172.21.15.16:0/2133341602 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6b040b4c90 0x7f6b040b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:14.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.437+0000 7f6b21800640 1 --2- 172.21.15.16:0/2133341602 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6b040b4c90 0x7f6b040b71a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6b10009fc0 tx=0x7f6b10005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:14.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.440+0000 7f6b1a000640 1 -- 172.21.15.16:0/2133341602 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f6b080ac5a0 con 0x7f6b1c075e40 2024-08-31T06:06:14.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.641+0000 7f6b24200640 1 -- 172.21.15.16:0/2133341602 --> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f6b1c109040 con 0x7f6b040b4c90 2024-08-31T06:06:14.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:14 smithi016 ceph-mon[17194]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-31T06:06:14.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.643+0000 7f6b1a000640 1 -- 172.21.15.16:0/2133341602 <== mgr.14220 v2:172.21.15.16:6800/4279751337 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f6b1c109040 con 0x7f6b040b4c90 2024-08-31T06:06:14.645 INFO:teuthology.orchestra.run.smithi016.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-08-31T06:06:14.645 INFO:teuthology.orchestra.run.smithi016.stdout:smithi016 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513002C480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-08-31T06:06:14.645 INFO:teuthology.orchestra.run.smithi016.stdout:smithi155 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GW400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-08-31T06:06:14.645 INFO:teuthology.orchestra.run.smithi016.stdout:smithi184 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT626500CE400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-08-31T06:06:14.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.645+0000 7f6b24200640 1 -- 172.21.15.16:0/2133341602 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6b040b4c90 msgr2=0x7f6b040b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:14.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.645+0000 7f6b24200640 1 --2- 172.21.15.16:0/2133341602 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6b040b4c90 0x7f6b040b71a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6b10009fc0 tx=0x7f6b10005f90 comp rx=0 tx=0).stop 2024-08-31T06:06:14.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.645+0000 7f6b24200640 1 -- 172.21.15.16:0/2133341602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b1c075e40 msgr2=0x7f6b1c19c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:14.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.645+0000 7f6b24200640 1 --2- 172.21.15.16:0/2133341602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b1c075e40 0x7f6b1c19c960 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800e9c0 tx=0x7f6b0800ee80 comp rx=0 tx=0).stop 2024-08-31T06:06:14.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.645+0000 7f6b24200640 1 -- 172.21.15.16:0/2133341602 shutdown_connections 2024-08-31T06:06:14.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.645+0000 7f6b24200640 1 --2- 172.21.15.16:0/2133341602 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6b040b4c90 0x7f6b040b71a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:14.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.645+0000 7f6b24200640 1 --2- 172.21.15.16:0/2133341602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b1c10fec0 0x7f6b1c1a1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:14.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.645+0000 7f6b24200640 1 --2- 172.21.15.16:0/2133341602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b1c075e40 0x7f6b1c19c960 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:14.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.645+0000 7f6b24200640 1 --2- 172.21.15.16:0/2133341602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b1c0754f0 0x7f6b1c19c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:14.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.645+0000 7f6b24200640 1 -- 172.21.15.16:0/2133341602 >> 172.21.15.16:0/2133341602 conn(0x7f6b1c07b5f0 msgr2=0x7f6b1c107920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:14.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.645+0000 7f6b24200640 1 -- 172.21.15.16:0/2133341602 shutdown_connections 2024-08-31T06:06:14.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:14.645+0000 7f6b24200640 1 -- 172.21.15.16:0/2133341602 wait complete. 2024-08-31T06:06:14.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:14 smithi184 ceph-mon[22331]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-31T06:06:14.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:14 smithi155 ceph-mon[21742]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-31T06:06:15.538 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-08-31T06:06:15.547 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-08-31T06:06:15.547 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- bash -c 'ceph fs dump' 2024-08-31T06:06:15.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:15 smithi184 ceph-mon[22331]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:06:15.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:15 smithi155 ceph-mon[21742]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:06:15.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:15 smithi016 ceph-mon[17194]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:06:15.925 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:06:16.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.604+0000 7f5093e00640 1 -- 172.21.15.16:0/152335719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f508c10b490 msgr2=0x7f508c10b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:16.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.604+0000 7f5093e00640 1 --2- 172.21.15.16:0/152335719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f508c10b490 0x7f508c10b8a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5078009900 tx=0x7f507802a3f0 comp rx=0 tx=0).stop 2024-08-31T06:06:16.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.604+0000 7f5093e00640 1 -- 172.21.15.16:0/152335719 shutdown_connections 2024-08-31T06:06:16.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.604+0000 7f5093e00640 1 --2- 172.21.15.16:0/152335719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f508c10bde0 0x7f508c112610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:16.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.604+0000 7f5093e00640 1 --2- 172.21.15.16:0/152335719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f508c10b490 0x7f508c10b8a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:16.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.604+0000 7f5093e00640 1 --2- 172.21.15.16:0/152335719 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f508c105b50 0x7f508c105f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:16.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.604+0000 7f5093e00640 1 -- 172.21.15.16:0/152335719 >> 172.21.15.16:0/152335719 conn(0x7f508c101580 msgr2=0x7f508c1039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:16.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.605+0000 7f5093e00640 1 -- 172.21.15.16:0/152335719 shutdown_connections 2024-08-31T06:06:16.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.605+0000 7f5093e00640 1 -- 172.21.15.16:0/152335719 wait complete. 2024-08-31T06:06:16.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.606+0000 7f5093e00640 1 Processor -- start 2024-08-31T06:06:16.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.606+0000 7f5093e00640 1 -- start start 2024-08-31T06:06:16.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.606+0000 7f5093e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f508c105b50 0x7f508c19c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:16.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.606+0000 7f5093e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f508c10b490 0x7f508c19ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:16.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.606+0000 7f5093e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f508c10bde0 0x7f508c1a1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:16.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.606+0000 7f5093e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f508c115af0 con 0x7f508c10bde0 2024-08-31T06:06:16.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.606+0000 7f5093e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f508c115970 con 0x7f508c105b50 2024-08-31T06:06:16.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.606+0000 7f5093e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f508c115c70 con 0x7f508c10b490 2024-08-31T06:06:16.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.607+0000 7f5091400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f508c105b50 0x7f508c19c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:16.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.607+0000 7f5091400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f508c105b50 0x7f508c19c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:52898/0 (socket says 172.21.15.16:52898) 2024-08-31T06:06:16.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.607+0000 7f5091400640 1 -- 172.21.15.16:0/1032410980 learned_addr learned my addr 172.21.15.16:0/1032410980 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:16.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.607+0000 7f5090a00640 1 --2- 172.21.15.16:0/1032410980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f508c10b490 0x7f508c19ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:16.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.607+0000 7f5091e00640 1 --2- 172.21.15.16:0/1032410980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f508c10bde0 0x7f508c1a1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:16.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.607+0000 7f5091400640 1 -- 172.21.15.16:0/1032410980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f508c10b490 msgr2=0x7f508c19ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:16.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.607+0000 7f5091400640 1 --2- 172.21.15.16:0/1032410980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f508c10b490 0x7f508c19ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:16.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.607+0000 7f5091400640 1 -- 172.21.15.16:0/1032410980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f508c10bde0 msgr2=0x7f508c1a1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:16.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.607+0000 7f5091400640 1 --2- 172.21.15.16:0/1032410980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f508c10bde0 0x7f508c1a1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:16.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.607+0000 7f5091400640 1 -- 172.21.15.16:0/1032410980 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f508c1a2450 con 0x7f508c105b50 2024-08-31T06:06:16.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.607+0000 7f5090a00640 1 --2- 172.21.15.16:0/1032410980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f508c10b490 0x7f508c19ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:06:16.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.607+0000 7f5091e00640 1 --2- 172.21.15.16:0/1032410980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f508c10bde0 0x7f508c1a1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:06:16.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.607+0000 7f5091400640 1 --2- 172.21.15.16:0/1032410980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f508c105b50 0x7f508c19c4e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f508000e9f0 tx=0x7f508000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:16.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.608+0000 7f508a000640 1 -- 172.21.15.16:0/1032410980 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508000cd80 con 0x7f508c105b50 2024-08-31T06:06:16.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.608+0000 7f5093e00640 1 -- 172.21.15.16:0/1032410980 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f508c1a2740 con 0x7f508c105b50 2024-08-31T06:06:16.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.608+0000 7f508a000640 1 -- 172.21.15.16:0/1032410980 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50800054d0 con 0x7f508c105b50 2024-08-31T06:06:16.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.608+0000 7f508a000640 1 -- 172.21.15.16:0/1032410980 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5080010690 con 0x7f508c105b50 2024-08-31T06:06:16.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.608+0000 7f5093e00640 1 -- 172.21.15.16:0/1032410980 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f508c1a8ea0 con 0x7f508c105b50 2024-08-31T06:06:16.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.609+0000 7f5093e00640 1 -- 172.21.15.16:0/1032410980 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f508c105f60 con 0x7f508c105b50 2024-08-31T06:06:16.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.611+0000 7f508a000640 1 -- 172.21.15.16:0/1032410980 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f5080004e30 con 0x7f508c105b50 2024-08-31T06:06:16.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.613+0000 7f508a000640 1 --2- 172.21.15.16:0/1032410980 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f50740b4c10 0x7f50740b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:16.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.614+0000 7f508a000640 1 -- 172.21.15.16:0/1032410980 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f50800e5a10 con 0x7f508c105b50 2024-08-31T06:06:16.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.614+0000 7f5090a00640 1 --2- 172.21.15.16:0/1032410980 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f50740b4c10 0x7f50740b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:16.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.614+0000 7f5090a00640 1 --2- 172.21.15.16:0/1032410980 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f50740b4c10 0x7f50740b7120 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f507802a9e0 tx=0x7f5078005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:16.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.615+0000 7f508a000640 1 -- 172.21.15.16:0/1032410980 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f50800ac8c0 con 0x7f508c105b50 2024-08-31T06:06:16.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:16 smithi184 ceph-mon[22331]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-31T06:06:16.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.846+0000 7f5093e00640 1 -- 172.21.15.16:0/1032410980 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f508c10dc10 con 0x7f508c105b50 2024-08-31T06:06:16.849 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:16 smithi016 ceph-mon[17194]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-31T06:06:16.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.847+0000 7f508a000640 1 -- 172.21.15.16:0/1032410980 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) ==== 76+0+3091 (secure 0 0 0) 0x7f50800ac1c0 con 0x7f508c105b50 2024-08-31T06:06:16.849 INFO:teuthology.orchestra.run.smithi016.stdout:e24 2024-08-31T06:06:16.850 INFO:teuthology.orchestra.run.smithi016.stdout:btime 2024-08-31T06:04:42:937777+0000 2024-08-31T06:06:16.850 INFO:teuthology.orchestra.run.smithi016.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-08-31T06:06:16.850 INFO:teuthology.orchestra.run.smithi016.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-31T06:06:16.850 INFO:teuthology.orchestra.run.smithi016.stdout:legacy client fscid: 1 2024-08-31T06:06:16.850 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:06:16.850 INFO:teuthology.orchestra.run.smithi016.stdout:Filesystem 'cephfs' (1) 2024-08-31T06:06:16.850 INFO:teuthology.orchestra.run.smithi016.stdout:fs_name cephfs 2024-08-31T06:06:16.850 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 24 2024-08-31T06:06:16.850 INFO:teuthology.orchestra.run.smithi016.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-08-31T06:06:16.850 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-08-31T06:04:27.098649+0000 2024-08-31T06:06:16.850 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-08-31T06:04:42.745448+0000 2024-08-31T06:06:16.850 INFO:teuthology.orchestra.run.smithi016.stdout:tableserver 0 2024-08-31T06:06:16.850 INFO:teuthology.orchestra.run.smithi016.stdout:root 0 2024-08-31T06:06:16.850 INFO:teuthology.orchestra.run.smithi016.stdout:session_timeout 300 2024-08-31T06:06:16.850 INFO:teuthology.orchestra.run.smithi016.stdout:session_autoclose 300 2024-08-31T06:06:16.851 INFO:teuthology.orchestra.run.smithi016.stdout:max_file_size 1099511627776 2024-08-31T06:06:16.851 INFO:teuthology.orchestra.run.smithi016.stdout:max_xattr_size 65536 2024-08-31T06:06:16.851 INFO:teuthology.orchestra.run.smithi016.stdout:required_client_features {} 2024-08-31T06:06:16.851 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure 0 2024-08-31T06:06:16.851 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure_osd_epoch 0 2024-08-31T06:06:16.851 INFO:teuthology.orchestra.run.smithi016.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-31T06:06:16.851 INFO:teuthology.orchestra.run.smithi016.stdout:max_mds 3 2024-08-31T06:06:16.851 INFO:teuthology.orchestra.run.smithi016.stdout:in 0,1,2 2024-08-31T06:06:16.851 INFO:teuthology.orchestra.run.smithi016.stdout:up {0=24737,1=24724,2=24712} 2024-08-31T06:06:16.851 INFO:teuthology.orchestra.run.smithi016.stdout:failed 2024-08-31T06:06:16.851 INFO:teuthology.orchestra.run.smithi016.stdout:damaged 2024-08-31T06:06:16.851 INFO:teuthology.orchestra.run.smithi016.stdout:stopped 2024-08-31T06:06:16.851 INFO:teuthology.orchestra.run.smithi016.stdout:data_pools [3] 2024-08-31T06:06:16.852 INFO:teuthology.orchestra.run.smithi016.stdout:metadata_pool 2 2024-08-31T06:06:16.852 INFO:teuthology.orchestra.run.smithi016.stdout:inline_data disabled 2024-08-31T06:06:16.852 INFO:teuthology.orchestra.run.smithi016.stdout:balancer 2024-08-31T06:06:16.852 INFO:teuthology.orchestra.run.smithi016.stdout:bal_rank_mask -1 2024-08-31T06:06:16.852 INFO:teuthology.orchestra.run.smithi016.stdout:standby_count_wanted 0 2024-08-31T06:06:16.852 INFO:teuthology.orchestra.run.smithi016.stdout:qdb_cluster leader: 24737 members: 24712,24724,24737 2024-08-31T06:06:16.852 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b{0:24737} state up:active seq 5 addr [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] compat {c=[1],r=[1],i=[1fff]}] 2024-08-31T06:06:16.852 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.h{0:24698} state up:standby-replay seq 1 addr [v2:172.21.15.155:6836/1089710405,v1:172.21.15.155:6837/1089710405] compat {c=[1],r=[1],i=[1fff]}] 2024-08-31T06:06:16.852 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.g{1:24724} state up:active seq 9 addr [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] compat {c=[1],r=[1],i=[1fff]}] 2024-08-31T06:06:16.852 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c{1:24694} state up:standby-replay seq 1 addr [v2:172.21.15.184:6836/1712755588,v1:172.21.15.184:6837/1712755588] compat {c=[1],r=[1],i=[1fff]}] 2024-08-31T06:06:16.852 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.i{2:24712} state up:active seq 11 addr [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] compat {c=[1],r=[1],i=[1fff]}] 2024-08-31T06:06:16.852 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.f{2:24688} state up:standby-replay seq 1 addr [v2:172.21.15.184:6834/1925472309,v1:172.21.15.184:6835/1925472309] compat {c=[1],r=[1],i=[1fff]}] 2024-08-31T06:06:16.852 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:06:16.852 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:06:16.853 INFO:teuthology.orchestra.run.smithi016.stdout:Standby daemons: 2024-08-31T06:06:16.853 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:06:16.853 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.j{-1:14643} state up:standby seq 1 addr [v2:172.21.15.16:6834/3365059950,v1:172.21.15.16:6835/3365059950] compat {c=[1],r=[1],i=[1fff]}] 2024-08-31T06:06:16.853 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a{-1:14709} state up:standby seq 1 addr [v2:172.21.15.16:6840/3621101104,v1:172.21.15.16:6841/3621101104] compat {c=[1],r=[1],i=[1fff]}] 2024-08-31T06:06:16.853 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.l{-1:24637} state up:standby seq 1 addr [v2:172.21.15.184:6832/1204885928,v1:172.21.15.184:6833/1204885928] compat {c=[1],r=[1],i=[1fff]}] 2024-08-31T06:06:16.853 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.k{-1:24638} state up:standby seq 1 addr [v2:172.21.15.155:6832/2239981116,v1:172.21.15.155:6833/2239981116] compat {c=[1],r=[1],i=[1fff]}] 2024-08-31T06:06:16.853 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d{-1:24662} state up:standby seq 1 addr [v2:172.21.15.16:6836/368986894,v1:172.21.15.16:6837/368986894] compat {c=[1],r=[1],i=[1fff]}] 2024-08-31T06:06:16.853 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.e{-1:24674} state up:standby seq 1 addr [v2:172.21.15.155:6834/1634403570,v1:172.21.15.155:6835/1634403570] compat {c=[1],r=[1],i=[1fff]}] 2024-08-31T06:06:16.854 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 24 2024-08-31T06:06:16.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.849+0000 7f5093e00640 1 -- 172.21.15.16:0/1032410980 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f50740b4c10 msgr2=0x7f50740b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:16.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.849+0000 7f5093e00640 1 --2- 172.21.15.16:0/1032410980 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f50740b4c10 0x7f50740b7120 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f507802a9e0 tx=0x7f5078005a20 comp rx=0 tx=0).stop 2024-08-31T06:06:16.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.849+0000 7f5093e00640 1 -- 172.21.15.16:0/1032410980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f508c105b50 msgr2=0x7f508c19c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:16.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.849+0000 7f5093e00640 1 --2- 172.21.15.16:0/1032410980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f508c105b50 0x7f508c19c4e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f508000e9f0 tx=0x7f508000edf0 comp rx=0 tx=0).stop 2024-08-31T06:06:16.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.850+0000 7f5093e00640 1 -- 172.21.15.16:0/1032410980 shutdown_connections 2024-08-31T06:06:16.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.850+0000 7f5093e00640 1 --2- 172.21.15.16:0/1032410980 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f50740b4c10 0x7f50740b7120 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:16.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.850+0000 7f5093e00640 1 --2- 172.21.15.16:0/1032410980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f508c10b490 0x7f508c19ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:16.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.850+0000 7f5093e00640 1 --2- 172.21.15.16:0/1032410980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f508c10bde0 0x7f508c1a1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:16.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.850+0000 7f5093e00640 1 --2- 172.21.15.16:0/1032410980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f508c105b50 0x7f508c19c4e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:16.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.850+0000 7f5093e00640 1 -- 172.21.15.16:0/1032410980 >> 172.21.15.16:0/1032410980 conn(0x7f508c101580 msgr2=0x7f508c108af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:16.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.850+0000 7f5093e00640 1 -- 172.21.15.16:0/1032410980 shutdown_connections 2024-08-31T06:06:16.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:16.850+0000 7f5093e00640 1 -- 172.21.15.16:0/1032410980 wait complete. 2024-08-31T06:06:16.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:16 smithi155 ceph-mon[21742]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-31T06:06:17.632 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:17 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/1032410980' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-08-31T06:06:17.668 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- bash -c 'ceph osd dump' 2024-08-31T06:06:17.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:17 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/1032410980' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-08-31T06:06:17.871 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:17 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/1032410980' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-08-31T06:06:18.049 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:06:18.545 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:18 smithi016 ceph-mon[17194]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-31T06:06:18.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:18 smithi184 ceph-mon[22331]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-31T06:06:18.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.722+0000 7f80fc400640 1 -- 172.21.15.16:0/3258742133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80f410b460 msgr2=0x7f80f410b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:18.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.722+0000 7f80fc400640 1 --2- 172.21.15.16:0/3258742133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80f410b460 0x7f80f410b870 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f80e0009900 tx=0x7f80e002a3f0 comp rx=0 tx=0).stop 2024-08-31T06:06:18.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.722+0000 7f80fc400640 1 -- 172.21.15.16:0/3258742133 shutdown_connections 2024-08-31T06:06:18.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.722+0000 7f80fc400640 1 --2- 172.21.15.16:0/3258742133 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80f410bdb0 0x7f80f41125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:18.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.722+0000 7f80fc400640 1 --2- 172.21.15.16:0/3258742133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80f410b460 0x7f80f410b870 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:18.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.722+0000 7f80fc400640 1 --2- 172.21.15.16:0/3258742133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80f4105b20 0x7f80f4105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:18.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.722+0000 7f80fc400640 1 -- 172.21.15.16:0/3258742133 >> 172.21.15.16:0/3258742133 conn(0x7f80f41015b0 msgr2=0x7f80f4103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:18.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.722+0000 7f80fc400640 1 -- 172.21.15.16:0/3258742133 shutdown_connections 2024-08-31T06:06:18.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.722+0000 7f80fc400640 1 -- 172.21.15.16:0/3258742133 wait complete. 2024-08-31T06:06:18.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.723+0000 7f80fc400640 1 Processor -- start 2024-08-31T06:06:18.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.723+0000 7f80fc400640 1 -- start start 2024-08-31T06:06:18.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.723+0000 7f80fc400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80f4105b20 0x7f80f419c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:18.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.723+0000 7f80fc400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80f410b460 0x7f80f419ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:18.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.724+0000 7f80fc400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80f410bdb0 0x7f80f41a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:18.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.724+0000 7f80fc400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f80f4115ac0 con 0x7f80f410b460 2024-08-31T06:06:18.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.724+0000 7f80fc400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f80f4115940 con 0x7f80f4105b20 2024-08-31T06:06:18.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.724+0000 7f80fc400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f80f4115c40 con 0x7f80f410bdb0 2024-08-31T06:06:18.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.724+0000 7f80f9000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80f410b460 0x7f80f419ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:18.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.724+0000 7f80f9000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80f410b460 0x7f80f419ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59938/0 (socket says 172.21.15.16:59938) 2024-08-31T06:06:18.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.724+0000 7f80f9a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80f4105b20 0x7f80f419c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:18.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.724+0000 7f80f9000640 1 -- 172.21.15.16:0/47062087 learned_addr learned my addr 172.21.15.16:0/47062087 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:18.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.724+0000 7f80fa400640 1 --2- 172.21.15.16:0/47062087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80f410bdb0 0x7f80f41a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:18.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.724+0000 7f80f9a00640 1 -- 172.21.15.16:0/47062087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80f410bdb0 msgr2=0x7f80f41a1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:18.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.724+0000 7f80f9a00640 1 --2- 172.21.15.16:0/47062087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80f410bdb0 0x7f80f41a1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:18.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.724+0000 7f80f9a00640 1 -- 172.21.15.16:0/47062087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80f410b460 msgr2=0x7f80f419ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:18.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.724+0000 7f80f9a00640 1 --2- 172.21.15.16:0/47062087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80f410b460 0x7f80f419ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:18.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.724+0000 7f80f9a00640 1 -- 172.21.15.16:0/47062087 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80f41a2400 con 0x7f80f4105b20 2024-08-31T06:06:18.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.724+0000 7f80f9000640 1 --2- 172.21.15.16:0/47062087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80f410b460 0x7f80f419ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:06:18.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.725+0000 7f80f9a00640 1 --2- 172.21.15.16:0/47062087 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80f4105b20 0x7f80f419c490 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f80e800e9f0 tx=0x7f80e800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:18.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.725+0000 7f80f2000640 1 -- 172.21.15.16:0/47062087 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e800ccc0 con 0x7f80f4105b20 2024-08-31T06:06:18.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.725+0000 7f80fc400640 1 -- 172.21.15.16:0/47062087 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80f41a26f0 con 0x7f80f4105b20 2024-08-31T06:06:18.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.725+0000 7f80f2000640 1 -- 172.21.15.16:0/47062087 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80e80108e0 con 0x7f80f4105b20 2024-08-31T06:06:18.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.725+0000 7f80f2000640 1 -- 172.21.15.16:0/47062087 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e8018840 con 0x7f80f4105b20 2024-08-31T06:06:18.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.725+0000 7f80fc400640 1 -- 172.21.15.16:0/47062087 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80f41a8e50 con 0x7f80f4105b20 2024-08-31T06:06:18.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.726+0000 7f80fc400640 1 -- 172.21.15.16:0/47062087 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80f4105f30 con 0x7f80f4105b20 2024-08-31T06:06:18.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.729+0000 7f80f2000640 1 -- 172.21.15.16:0/47062087 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f80e8010430 con 0x7f80f4105b20 2024-08-31T06:06:18.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.730+0000 7f80f2000640 1 --2- 172.21.15.16:0/47062087 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f80dc0b4c90 0x7f80dc0b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:18.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.730+0000 7f80f2000640 1 -- 172.21.15.16:0/47062087 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f80e80e5af0 con 0x7f80f4105b20 2024-08-31T06:06:18.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.730+0000 7f80f9000640 1 --2- 172.21.15.16:0/47062087 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f80dc0b4c90 0x7f80dc0b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:18.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.731+0000 7f80f9000640 1 --2- 172.21.15.16:0/47062087 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f80dc0b4c90 0x7f80dc0b71a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f80e0006010 tx=0x7f80e0005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.732+0000 7f80f2000640 1 -- 172.21.15.16:0/47062087 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f80e80ac9a0 con 0x7f80f4105b20 2024-08-31T06:06:18.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:18 smithi155 ceph-mon[21742]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-31T06:06:18.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.929+0000 7f80fc400640 1 -- 172.21.15.16:0/47062087 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f80f419d5c0 con 0x7f80f4105b20 2024-08-31T06:06:18.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.930+0000 7f80f2000640 1 -- 172.21.15.16:0/47062087 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v80) ==== 56+0+5672 (secure 0 0 0) 0x7f80f419d5c0 con 0x7f80f4105b20 2024-08-31T06:06:18.933 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 80 2024-08-31T06:06:18.933 INFO:teuthology.orchestra.run.smithi016.stdout:fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 2024-08-31T06:06:18.933 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-08-31T05:55:09.632536+0000 2024-08-31T06:06:18.933 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-08-31T06:05:25.039584+0000 2024-08-31T06:06:18.933 INFO:teuthology.orchestra.run.smithi016.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-08-31T06:06:18.934 INFO:teuthology.orchestra.run.smithi016.stdout:crush_version 26 2024-08-31T06:06:18.934 INFO:teuthology.orchestra.run.smithi016.stdout:full_ratio 0.95 2024-08-31T06:06:18.934 INFO:teuthology.orchestra.run.smithi016.stdout:backfillfull_ratio 0.9 2024-08-31T06:06:18.934 INFO:teuthology.orchestra.run.smithi016.stdout:nearfull_ratio 0.85 2024-08-31T06:06:18.934 INFO:teuthology.orchestra.run.smithi016.stdout:require_min_compat_client luminous 2024-08-31T06:06:18.934 INFO:teuthology.orchestra.run.smithi016.stdout:min_compat_client luminous 2024-08-31T06:06:18.934 INFO:teuthology.orchestra.run.smithi016.stdout:require_osd_release squid 2024-08-31T06:06:18.934 INFO:teuthology.orchestra.run.smithi016.stdout:stretch_mode_enabled false 2024-08-31T06:06:18.934 INFO:teuthology.orchestra.run.smithi016.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-08-31T06:06:18.934 INFO:teuthology.orchestra.run.smithi016.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-08-31T06:06:18.934 INFO:teuthology.orchestra.run.smithi016.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-08-31T06:06:18.934 INFO:teuthology.orchestra.run.smithi016.stdout:max_osd 12 2024-08-31T06:06:18.934 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 up in weight 1 up_from 9 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6802/1725037400,v1:172.21.15.16:6803/1725037400] [v2:172.21.15.16:6804/1725037400,v1:172.21.15.16:6805/1725037400] exists,up 0e349dae-6aa5-4b6d-9107-a945655e11bd 2024-08-31T06:06:18.935 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 up in weight 1 up_from 14 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.155:6800/4241618440,v1:172.21.15.155:6801/4241618440] [v2:172.21.15.155:6802/4241618440,v1:172.21.15.155:6803/4241618440] exists,up bd645dc3-93b5-40af-89d8-08ccec22e50d 2024-08-31T06:06:18.935 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 up in weight 1 up_from 18 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.184:6800/2193296422,v1:172.21.15.184:6801/2193296422] [v2:172.21.15.184:6802/2193296422,v1:172.21.15.184:6803/2193296422] exists,up c69c8c19-32f8-400a-86d4-0f4ce240a7de 2024-08-31T06:06:18.935 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 up in weight 1 up_from 26 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6810/81681815,v1:172.21.15.16:6811/81681815] [v2:172.21.15.16:6812/81681815,v1:172.21.15.16:6813/81681815] exists,up 8892e35f-6880-434c-a805-ab8ccbed2bf9 2024-08-31T06:06:18.935 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 up in weight 1 up_from 32 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.155:6808/3592422019,v1:172.21.15.155:6809/3592422019] [v2:172.21.15.155:6810/3592422019,v1:172.21.15.155:6811/3592422019] exists,up d2515e73-88f7-4654-9762-adb8802777ed 2024-08-31T06:06:18.935 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 up in weight 1 up_from 37 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.184:6808/2054894065,v1:172.21.15.184:6809/2054894065] [v2:172.21.15.184:6810/2054894065,v1:172.21.15.184:6811/2054894065] exists,up 5ce6e2fa-c17c-49b3-9402-f19bfc21076d 2024-08-31T06:06:18.935 INFO:teuthology.orchestra.run.smithi016.stdout:osd.6 up in weight 1 up_from 43 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6818/1697421092,v1:172.21.15.16:6819/1697421092] [v2:172.21.15.16:6820/1697421092,v1:172.21.15.16:6821/1697421092] exists,up f257d8d1-ad59-4bd8-821d-bf29e50b547e 2024-08-31T06:06:18.935 INFO:teuthology.orchestra.run.smithi016.stdout:osd.7 up in weight 1 up_from 49 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.155:6816/2671827901,v1:172.21.15.155:6817/2671827901] [v2:172.21.15.155:6818/2671827901,v1:172.21.15.155:6819/2671827901] exists,up 8e76c20e-8d25-423b-b7e8-de642d9c9a6c 2024-08-31T06:06:18.935 INFO:teuthology.orchestra.run.smithi016.stdout:osd.8 up in weight 1 up_from 54 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.184:6816/2479117823,v1:172.21.15.184:6817/2479117823] [v2:172.21.15.184:6818/2479117823,v1:172.21.15.184:6819/2479117823] exists,up 0f0bb23f-4bc3-4198-8381-5b5cd16ae394 2024-08-31T06:06:18.935 INFO:teuthology.orchestra.run.smithi016.stdout:osd.9 up in weight 1 up_from 60 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6826/3015700546,v1:172.21.15.16:6827/3015700546] [v2:172.21.15.16:6828/3015700546,v1:172.21.15.16:6829/3015700546] exists,up bc27afcb-7871-4f72-a043-044fd5fe7d86 2024-08-31T06:06:18.935 INFO:teuthology.orchestra.run.smithi016.stdout:osd.10 up in weight 1 up_from 65 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.155:6824/2427122475,v1:172.21.15.155:6825/2427122475] [v2:172.21.15.155:6826/2427122475,v1:172.21.15.155:6827/2427122475] exists,up f2548997-72ee-400e-ab8d-5966d85b2cf5 2024-08-31T06:06:18.935 INFO:teuthology.orchestra.run.smithi016.stdout:osd.11 up in weight 1 up_from 70 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.184:6824/399265701,v1:172.21.15.184:6825/399265701] [v2:172.21.15.184:6826/399265701,v1:172.21.15.184:6827/399265701] exists,up 8297da1d-d3e2-4a4d-9396-426dfafa3e95 2024-08-31T06:06:18.935 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 2.f [8,11] 2024-08-31T06:06:18.935 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 2.10 [2,1] 2024-08-31T06:06:18.936 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 2.1b [2,4] 2024-08-31T06:06:18.936 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 2.1c [8,11] 2024-08-31T06:06:18.936 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.1d [4,8] 2024-08-31T06:06:18.936 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.24 [4,9] 2024-08-31T06:06:18.936 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.25 [5,9] 2024-08-31T06:06:18.936 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.36 [4,9] 2024-08-31T06:06:18.936 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/3923220960 expires 2024-09-01T05:57:23.153214+0000 2024-08-31T06:06:18.936 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/3013533836 expires 2024-09-01T05:57:23.153214+0000 2024-08-31T06:06:18.936 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6800/3755683228 expires 2024-09-01T05:55:45.075902+0000 2024-08-31T06:06:18.936 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/3064435470 expires 2024-09-01T05:55:45.075902+0000 2024-08-31T06:06:18.936 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/911709962 expires 2024-09-01T05:55:45.075902+0000 2024-08-31T06:06:18.936 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/2937368967 expires 2024-09-01T05:55:45.075902+0000 2024-08-31T06:06:18.936 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6800/1378972230 expires 2024-09-01T05:56:14.195704+0000 2024-08-31T06:06:18.937 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/3047316995 expires 2024-09-01T05:56:14.195704+0000 2024-08-31T06:06:18.937 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6801/3755683228 expires 2024-09-01T05:55:45.075902+0000 2024-08-31T06:06:18.937 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6801/2571082634 expires 2024-09-01T05:57:23.153214+0000 2024-08-31T06:06:18.937 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/2136704295 expires 2024-09-01T05:56:14.195704+0000 2024-08-31T06:06:18.937 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/2259332778 expires 2024-09-01T05:56:14.195704+0000 2024-08-31T06:06:18.937 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6800/2571082634 expires 2024-09-01T05:57:23.153214+0000 2024-08-31T06:06:18.937 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6801/1378972230 expires 2024-09-01T05:56:14.195704+0000 2024-08-31T06:06:18.937 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/2126457673 expires 2024-09-01T05:57:23.153214+0000 2024-08-31T06:06:18.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.933+0000 7f80fc400640 1 -- 172.21.15.16:0/47062087 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f80dc0b4c90 msgr2=0x7f80dc0b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:18.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.933+0000 7f80fc400640 1 --2- 172.21.15.16:0/47062087 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f80dc0b4c90 0x7f80dc0b71a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f80e0006010 tx=0x7f80e0005a20 comp rx=0 tx=0).stop 2024-08-31T06:06:18.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.933+0000 7f80fc400640 1 -- 172.21.15.16:0/47062087 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80f4105b20 msgr2=0x7f80f419c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:18.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.933+0000 7f80fc400640 1 --2- 172.21.15.16:0/47062087 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80f4105b20 0x7f80f419c490 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f80e800e9f0 tx=0x7f80e800edf0 comp rx=0 tx=0).stop 2024-08-31T06:06:18.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.933+0000 7f80fc400640 1 -- 172.21.15.16:0/47062087 shutdown_connections 2024-08-31T06:06:18.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.933+0000 7f80fc400640 1 --2- 172.21.15.16:0/47062087 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f80dc0b4c90 0x7f80dc0b71a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:18.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.933+0000 7f80fc400640 1 --2- 172.21.15.16:0/47062087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80f410bdb0 0x7f80f41a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:18.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.933+0000 7f80fc400640 1 --2- 172.21.15.16:0/47062087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80f410b460 0x7f80f419ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:18.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.933+0000 7f80fc400640 1 --2- 172.21.15.16:0/47062087 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80f4105b20 0x7f80f419c490 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:18.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.933+0000 7f80fc400640 1 -- 172.21.15.16:0/47062087 >> 172.21.15.16:0/47062087 conn(0x7f80f41015b0 msgr2=0x7f80f41039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:18.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.933+0000 7f80fc400640 1 -- 172.21.15.16:0/47062087 shutdown_connections 2024-08-31T06:06:18.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:18.933+0000 7f80fc400640 1 -- 172.21.15.16:0/47062087 wait complete. 2024-08-31T06:06:19.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:19 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/47062087' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-08-31T06:06:19.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:19 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/47062087' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-08-31T06:06:19.737 INFO:teuthology.run_tasks:Running task fs.ready... 2024-08-31T06:06:19.748 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:06:19.871 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:19 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/47062087' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-08-31T06:06:20.125 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:06:20.565 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:20 smithi016 ceph-mon[17194]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-31T06:06:20.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:20 smithi184 ceph-mon[22331]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-31T06:06:20.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.741+0000 7f97c0600640 1 -- 172.21.15.16:0/2665615393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97b8103900 msgr2=0x7f97b8103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:20.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.741+0000 7f97c0600640 1 --2- 172.21.15.16:0/2665615393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97b8103900 0x7f97b8103d10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f97ac009900 tx=0x7f97ac02a410 comp rx=0 tx=0).stop 2024-08-31T06:06:20.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.741+0000 7f97c0600640 1 -- 172.21.15.16:0/2665615393 shutdown_connections 2024-08-31T06:06:20.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.741+0000 7f97c0600640 1 --2- 172.21.15.16:0/2665615393 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97b8109240 0x7f97b8109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:20.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.741+0000 7f97c0600640 1 --2- 172.21.15.16:0/2665615393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97b8109b90 0x7f97b81103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:20.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.741+0000 7f97c0600640 1 --2- 172.21.15.16:0/2665615393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97b8103900 0x7f97b8103d10 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:20.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.741+0000 7f97c0600640 1 -- 172.21.15.16:0/2665615393 >> 172.21.15.16:0/2665615393 conn(0x7f97b807b5f0 msgr2=0x7f97b807ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:20.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.741+0000 7f97c0600640 1 -- 172.21.15.16:0/2665615393 shutdown_connections 2024-08-31T06:06:20.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.741+0000 7f97c0600640 1 -- 172.21.15.16:0/2665615393 wait complete. 2024-08-31T06:06:20.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.742+0000 7f97c0600640 1 Processor -- start 2024-08-31T06:06:20.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.742+0000 7f97c0600640 1 -- start start 2024-08-31T06:06:20.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.743+0000 7f97c0600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97b8103900 0x7f97b819a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:20.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.743+0000 7f97c0600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97b8109240 0x7f97b819a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:20.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.743+0000 7f97c0600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97b8109b90 0x7f97b819fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:20.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.743+0000 7f97c0600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f97b8113850 con 0x7f97b8109b90 2024-08-31T06:06:20.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.743+0000 7f97c0600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f97b81136d0 con 0x7f97b8103900 2024-08-31T06:06:20.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.743+0000 7f97c0600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f97b81139d0 con 0x7f97b8109240 2024-08-31T06:06:20.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.743+0000 7f97be600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97b8109b90 0x7f97b819fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:20.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.743+0000 7f97bdc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97b8103900 0x7f97b819a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:20.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.743+0000 7f97bdc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97b8103900 0x7f97b819a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:52934/0 (socket says 172.21.15.16:52934) 2024-08-31T06:06:20.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.743+0000 7f97bdc00640 1 -- 172.21.15.16:0/1163346203 learned_addr learned my addr 172.21.15.16:0/1163346203 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:20.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.743+0000 7f97bd200640 1 --2- 172.21.15.16:0/1163346203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97b8109240 0x7f97b819a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:20.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.744+0000 7f97bdc00640 1 -- 172.21.15.16:0/1163346203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97b8109240 msgr2=0x7f97b819a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:20.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.744+0000 7f97bdc00640 1 --2- 172.21.15.16:0/1163346203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97b8109240 0x7f97b819a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:20.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.744+0000 7f97bdc00640 1 -- 172.21.15.16:0/1163346203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97b8109b90 msgr2=0x7f97b819fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:20.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.744+0000 7f97bdc00640 1 --2- 172.21.15.16:0/1163346203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97b8109b90 0x7f97b819fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:20.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.744+0000 7f97bdc00640 1 -- 172.21.15.16:0/1163346203 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97b81a01b0 con 0x7f97b8103900 2024-08-31T06:06:20.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.744+0000 7f97bdc00640 1 --2- 172.21.15.16:0/1163346203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97b8103900 0x7f97b819a240 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f97ac005940 tx=0x7f97ac0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:20.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.744+0000 7f97b6000640 1 -- 172.21.15.16:0/1163346203 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ac03f030 con 0x7f97b8103900 2024-08-31T06:06:20.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.744+0000 7f97c0600640 1 -- 172.21.15.16:0/1163346203 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97b81a0440 con 0x7f97b8103900 2024-08-31T06:06:20.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.744+0000 7f97b6000640 1 -- 172.21.15.16:0/1163346203 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97ac0044c0 con 0x7f97b8103900 2024-08-31T06:06:20.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.745+0000 7f97c0600640 1 -- 172.21.15.16:0/1163346203 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97b81a6c00 con 0x7f97b8103900 2024-08-31T06:06:20.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.745+0000 7f97b6000640 1 -- 172.21.15.16:0/1163346203 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ac035070 con 0x7f97b8103900 2024-08-31T06:06:20.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.746+0000 7f97c0600640 1 -- 172.21.15.16:0/1163346203 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97b8103d10 con 0x7f97b8103900 2024-08-31T06:06:20.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.748+0000 7f97b6000640 1 -- 172.21.15.16:0/1163346203 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f97ac042030 con 0x7f97b8103900 2024-08-31T06:06:20.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.750+0000 7f97b6000640 1 --2- 172.21.15.16:0/1163346203 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f97a00b4c90 0x7f97a00b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:20.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.750+0000 7f97b6000640 1 -- 172.21.15.16:0/1163346203 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f97ac1059e0 con 0x7f97b8103900 2024-08-31T06:06:20.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.750+0000 7f97bd200640 1 --2- 172.21.15.16:0/1163346203 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f97a00b4c90 0x7f97a00b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:20.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.751+0000 7f97bd200640 1 --2- 172.21.15.16:0/1163346203 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f97a00b4c90 0x7f97a00b71a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f97b8108f40 tx=0x7f97a4005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:20.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.752+0000 7f97b6000640 1 -- 172.21.15.16:0/1163346203 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f97ac0cc240 con 0x7f97b8103900 2024-08-31T06:06:20.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:20 smithi155 ceph-mon[21742]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-31T06:06:20.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.985+0000 7f97c0600640 1 -- 172.21.15.16:0/1163346203 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97b819b370 con 0x7f97b8103900 2024-08-31T06:06:20.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.986+0000 7f97b6000640 1 -- 172.21.15.16:0/1163346203 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11917 (secure 0 0 0) 0x7f97b819b370 con 0x7f97b8103900 2024-08-31T06:06:20.989 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:06:20.989 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"btime":"2024-08-31T06:04:42:937777+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:04:42.745448+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24698":{"gid":24698,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1089710405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1089710405},{"type":"v1","addr":"172.21.15.155:6837","nonce":1089710405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24712,24724,24737]},"id":1}]} 2024-08-31T06:06:20.989 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 24 2024-08-31T06:06:20.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.989+0000 7f97c0600640 1 -- 172.21.15.16:0/1163346203 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f97a00b4c90 msgr2=0x7f97a00b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:20.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.989+0000 7f97c0600640 1 --2- 172.21.15.16:0/1163346203 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f97a00b4c90 0x7f97a00b71a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f97b8108f40 tx=0x7f97a4005f90 comp rx=0 tx=0).stop 2024-08-31T06:06:20.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.989+0000 7f97c0600640 1 -- 172.21.15.16:0/1163346203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97b8103900 msgr2=0x7f97b819a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:20.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.989+0000 7f97c0600640 1 --2- 172.21.15.16:0/1163346203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97b8103900 0x7f97b819a240 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f97ac005940 tx=0x7f97ac0040c0 comp rx=0 tx=0).stop 2024-08-31T06:06:20.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.989+0000 7f97c0600640 1 -- 172.21.15.16:0/1163346203 shutdown_connections 2024-08-31T06:06:20.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.989+0000 7f97c0600640 1 --2- 172.21.15.16:0/1163346203 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f97a00b4c90 0x7f97a00b71a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:20.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.989+0000 7f97c0600640 1 --2- 172.21.15.16:0/1163346203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97b8109240 0x7f97b819a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:20.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.989+0000 7f97c0600640 1 --2- 172.21.15.16:0/1163346203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97b8109b90 0x7f97b819fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:20.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.989+0000 7f97c0600640 1 --2- 172.21.15.16:0/1163346203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97b8103900 0x7f97b819a240 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:20.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.989+0000 7f97c0600640 1 -- 172.21.15.16:0/1163346203 >> 172.21.15.16:0/1163346203 conn(0x7f97b807b5f0 msgr2=0x7f97b81072d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:20.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.989+0000 7f97c0600640 1 -- 172.21.15.16:0/1163346203 shutdown_connections 2024-08-31T06:06:20.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:20.989+0000 7f97c0600640 1 -- 172.21.15.16:0/1163346203 wait complete. 2024-08-31T06:06:21.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:21 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/1163346203' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:06:21.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:21 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/1163346203' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:06:21.768 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:06:21.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:21 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/1163346203' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:06:22.144 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:06:22.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:22 smithi016 ceph-mon[17194]: pgmap v337: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-31T06:06:22.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:22 smithi184 ceph-mon[22331]: pgmap v337: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-31T06:06:22.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:22.860+0000 7ff53fe00640 1 -- 172.21.15.16:0/461287717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5401099c0 msgr2=0x7ff5401101f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:22.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:22.860+0000 7ff53fe00640 1 --2- 172.21.15.16:0/461287717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5401099c0 0x7ff5401101f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff538009ca0 tx=0x7ff53802a8c0 comp rx=0 tx=0).stop 2024-08-31T06:06:22.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:22.860+0000 7ff53fe00640 1 -- 172.21.15.16:0/461287717 shutdown_connections 2024-08-31T06:06:22.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:22.860+0000 7ff53fe00640 1 --2- 172.21.15.16:0/461287717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5401099c0 0x7ff5401101f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:22.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:22.860+0000 7ff53fe00640 1 --2- 172.21.15.16:0/461287717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff540109070 0x7ff540109480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:22.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:22.860+0000 7ff53fe00640 1 --2- 172.21.15.16:0/461287717 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff540103730 0x7ff540103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:22.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:22.860+0000 7ff53fe00640 1 -- 172.21.15.16:0/461287717 >> 172.21.15.16:0/461287717 conn(0x7ff5400ff180 msgr2=0x7ff5401015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:22.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:22.860+0000 7ff53fe00640 1 -- 172.21.15.16:0/461287717 shutdown_connections 2024-08-31T06:06:22.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:22.860+0000 7ff53fe00640 1 -- 172.21.15.16:0/461287717 wait complete. 2024-08-31T06:06:22.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:22.861+0000 7ff53fe00640 1 Processor -- start 2024-08-31T06:06:22.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:22.861+0000 7ff53fe00640 1 -- start start 2024-08-31T06:06:22.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:22.862+0000 7ff53fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff540103730 0x7ff54019c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:22.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:22.862+0000 7ff53fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff540109070 0x7ff54019c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:22.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:22.862+0000 7ff53fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5401099c0 0x7ff5401a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:22.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:22.862+0000 7ff53fe00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff5401136c0 con 0x7ff540109070 2024-08-31T06:06:22.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:22.862+0000 7ff53fe00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff540113540 con 0x7ff5401099c0 2024-08-31T06:06:22.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:22.862+0000 7ff53fe00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff540113840 con 0x7ff540103730 2024-08-31T06:06:22.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:22.862+0000 7ff537e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff540109070 0x7ff54019c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:22.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:22.862+0000 7ff537e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff540109070 0x7ff54019c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59960/0 (socket says 172.21.15.16:59960) 2024-08-31T06:06:22.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:22.862+0000 7ff537e00640 1 -- 172.21.15.16:0/3113683355 learned_addr learned my addr 172.21.15.16:0/3113683355 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:22.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:22.862+0000 7ff53ea00640 1 --2- 172.21.15.16:0/3113683355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff540103730 0x7ff54019c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:22.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:22.862+0000 7ff53f400640 1 --2- 172.21.15.16:0/3113683355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5401099c0 0x7ff5401a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:22.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:22.863+0000 7ff537e00640 1 -- 172.21.15.16:0/3113683355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff540103730 msgr2=0x7ff54019c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:22.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:22.863+0000 7ff537e00640 1 --2- 172.21.15.16:0/3113683355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff540103730 0x7ff54019c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:22.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:22.863+0000 7ff537e00640 1 -- 172.21.15.16:0/3113683355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5401099c0 msgr2=0x7ff5401a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:22.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:22.863+0000 7ff537e00640 1 --2- 172.21.15.16:0/3113683355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5401099c0 0x7ff5401a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:22.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:22.863+0000 7ff537e00640 1 -- 172.21.15.16:0/3113683355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5401a2180 con 0x7ff540109070 2024-08-31T06:06:22.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:22.863+0000 7ff537e00640 1 --2- 172.21.15.16:0/3113683355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff540109070 0x7ff54019c7b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff52800e9f0 tx=0x7ff52800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:22.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:22.863+0000 7ff544a00640 1 -- 172.21.15.16:0/3113683355 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52800ccc0 con 0x7ff540109070 2024-08-31T06:06:22.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:22.863+0000 7ff53fe00640 1 -- 172.21.15.16:0/3113683355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5401a2470 con 0x7ff540109070 2024-08-31T06:06:22.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:22.863+0000 7ff544a00640 1 -- 172.21.15.16:0/3113683355 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5280108e0 con 0x7ff540109070 2024-08-31T06:06:22.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:22.863+0000 7ff544a00640 1 -- 172.21.15.16:0/3113683355 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff528018860 con 0x7ff540109070 2024-08-31T06:06:22.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:22.863+0000 7ff53fe00640 1 -- 172.21.15.16:0/3113683355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5401a8bd0 con 0x7ff540109070 2024-08-31T06:06:22.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:22.865+0000 7ff53fe00640 1 -- 172.21.15.16:0/3113683355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff540103b40 con 0x7ff540109070 2024-08-31T06:06:22.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:22.867+0000 7ff544a00640 1 -- 172.21.15.16:0/3113683355 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7ff528010430 con 0x7ff540109070 2024-08-31T06:06:22.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:22.869+0000 7ff544a00640 1 --2- 172.21.15.16:0/3113683355 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff5240b4c90 0x7ff5240b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:22.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:22.869+0000 7ff544a00640 1 -- 172.21.15.16:0/3113683355 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7ff5280e5a80 con 0x7ff540109070 2024-08-31T06:06:22.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:22.869+0000 7ff53ea00640 1 --2- 172.21.15.16:0/3113683355 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff5240b4c90 0x7ff5240b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:22.871 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:22 smithi155 ceph-mon[21742]: pgmap v337: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-31T06:06:22.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:22.870+0000 7ff53ea00640 1 --2- 172.21.15.16:0/3113683355 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff5240b4c90 0x7ff5240b71a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff530005d00 tx=0x7ff53000b410 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:22.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:22.871+0000 7ff544a00640 1 -- 172.21.15.16:0/3113683355 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7ff5280ac2e0 con 0x7ff540109070 2024-08-31T06:06:23.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:23.107+0000 7ff53fe00640 1 -- 172.21.15.16:0/3113683355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff54019d340 con 0x7ff540109070 2024-08-31T06:06:23.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:23.108+0000 7ff544a00640 1 -- 172.21.15.16:0/3113683355 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11917 (secure 0 0 0) 0x7ff54019d340 con 0x7ff540109070 2024-08-31T06:06:23.110 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:06:23.110 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"btime":"2024-08-31T06:04:42:937777+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:04:42.745448+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24698":{"gid":24698,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1089710405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1089710405},{"type":"v1","addr":"172.21.15.155:6837","nonce":1089710405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24712,24724,24737]},"id":1}]} 2024-08-31T06:06:23.111 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 24 2024-08-31T06:06:23.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:23.110+0000 7ff53fe00640 1 -- 172.21.15.16:0/3113683355 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff5240b4c90 msgr2=0x7ff5240b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:23.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:23.110+0000 7ff53fe00640 1 --2- 172.21.15.16:0/3113683355 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff5240b4c90 0x7ff5240b71a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff530005d00 tx=0x7ff53000b410 comp rx=0 tx=0).stop 2024-08-31T06:06:23.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:23.110+0000 7ff53fe00640 1 -- 172.21.15.16:0/3113683355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff540109070 msgr2=0x7ff54019c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:23.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:23.110+0000 7ff53fe00640 1 --2- 172.21.15.16:0/3113683355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff540109070 0x7ff54019c7b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff52800e9f0 tx=0x7ff52800eeb0 comp rx=0 tx=0).stop 2024-08-31T06:06:23.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:23.110+0000 7ff53fe00640 1 -- 172.21.15.16:0/3113683355 shutdown_connections 2024-08-31T06:06:23.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:23.110+0000 7ff53fe00640 1 --2- 172.21.15.16:0/3113683355 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff5240b4c90 0x7ff5240b71a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:23.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:23.110+0000 7ff53fe00640 1 --2- 172.21.15.16:0/3113683355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5401099c0 0x7ff5401a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:23.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:23.110+0000 7ff53fe00640 1 --2- 172.21.15.16:0/3113683355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff540109070 0x7ff54019c7b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:23.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:23.110+0000 7ff53fe00640 1 --2- 172.21.15.16:0/3113683355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff540103730 0x7ff54019c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:23.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:23.110+0000 7ff53fe00640 1 -- 172.21.15.16:0/3113683355 >> 172.21.15.16:0/3113683355 conn(0x7ff5400ff180 msgr2=0x7ff540108680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:23.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:23.111+0000 7ff53fe00640 1 -- 172.21.15.16:0/3113683355 shutdown_connections 2024-08-31T06:06:23.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:23.111+0000 7ff53fe00640 1 -- 172.21.15.16:0/3113683355 wait complete. 2024-08-31T06:06:23.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:23 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/3113683355' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:06:23.843 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:23 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/3113683355' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:06:23.871 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:23 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/3113683355' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:06:23.883 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph osd dump --format=json 2024-08-31T06:06:24.261 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:06:24.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:24 smithi016 ceph-mon[17194]: pgmap v338: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-31T06:06:24.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:24 smithi184 ceph-mon[22331]: pgmap v338: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-31T06:06:24.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:24 smithi155 ceph-mon[21742]: pgmap v338: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-31T06:06:24.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:24.923+0000 7f5f2ea00640 1 -- 172.21.15.16:0/855469002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f281099e0 msgr2=0x7f5f28110210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:24.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:24.923+0000 7f5f2ea00640 1 --2- 172.21.15.16:0/855469002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f281099e0 0x7f5f28110210 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5f20009ca0 tx=0x7f5f2002a8c0 comp rx=0 tx=0).stop 2024-08-31T06:06:24.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:24.923+0000 7f5f2ea00640 1 -- 172.21.15.16:0/855469002 shutdown_connections 2024-08-31T06:06:24.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:24.923+0000 7f5f2ea00640 1 --2- 172.21.15.16:0/855469002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f281099e0 0x7f5f28110210 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:24.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:24.923+0000 7f5f2ea00640 1 --2- 172.21.15.16:0/855469002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f28109090 0x7f5f281094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:24.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:24.923+0000 7f5f2ea00640 1 --2- 172.21.15.16:0/855469002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f28103750 0x7f5f28103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:24.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:24.923+0000 7f5f2ea00640 1 -- 172.21.15.16:0/855469002 >> 172.21.15.16:0/855469002 conn(0x7f5f280ff1a0 msgr2=0x7f5f281015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:24.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:24.923+0000 7f5f2ea00640 1 -- 172.21.15.16:0/855469002 shutdown_connections 2024-08-31T06:06:24.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:24.923+0000 7f5f2ea00640 1 -- 172.21.15.16:0/855469002 wait complete. 2024-08-31T06:06:24.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:24.925+0000 7f5f2ea00640 1 Processor -- start 2024-08-31T06:06:24.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:24.925+0000 7f5f2ea00640 1 -- start start 2024-08-31T06:06:24.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:24.926+0000 7f5f2ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f28103750 0x7f5f2819c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:24.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:24.926+0000 7f5f2ea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f28109090 0x7f5f2819c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:24.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:24.926+0000 7f5f2ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f281099e0 0x7f5f281a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:24.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:24.926+0000 7f5f2ea00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5f281136e0 con 0x7f5f28103750 2024-08-31T06:06:24.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:24.926+0000 7f5f2ea00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5f28113560 con 0x7f5f281099e0 2024-08-31T06:06:24.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:24.926+0000 7f5f2ea00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5f28113860 con 0x7f5f28109090 2024-08-31T06:06:24.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:24.926+0000 7f5f2d600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f28103750 0x7f5f2819c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:24.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:24.926+0000 7f5f2d600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f28103750 0x7f5f2819c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59972/0 (socket says 172.21.15.16:59972) 2024-08-31T06:06:24.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:24.926+0000 7f5f2d600640 1 -- 172.21.15.16:0/704418246 learned_addr learned my addr 172.21.15.16:0/704418246 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:24.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:24.926+0000 7f5f2e000640 1 --2- 172.21.15.16:0/704418246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f281099e0 0x7f5f281a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:24.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:24.926+0000 7f5f2cc00640 1 --2- 172.21.15.16:0/704418246 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f28109090 0x7f5f2819c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:24.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:24.926+0000 7f5f2d600640 1 -- 172.21.15.16:0/704418246 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f28109090 msgr2=0x7f5f2819c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:24.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:24.926+0000 7f5f2d600640 1 --2- 172.21.15.16:0/704418246 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f28109090 0x7f5f2819c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:24.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:24.926+0000 7f5f2d600640 1 -- 172.21.15.16:0/704418246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f281099e0 msgr2=0x7f5f281a1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:24.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:24.926+0000 7f5f2d600640 1 --2- 172.21.15.16:0/704418246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f281099e0 0x7f5f281a1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:24.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:24.926+0000 7f5f2d600640 1 -- 172.21.15.16:0/704418246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f281a21a0 con 0x7f5f28103750 2024-08-31T06:06:24.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:24.926+0000 7f5f2cc00640 1 --2- 172.21.15.16:0/704418246 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f28109090 0x7f5f2819c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:06:24.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:24.927+0000 7f5f2d600640 1 --2- 172.21.15.16:0/704418246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f28103750 0x7f5f2819c230 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c00ea60 tx=0x7f5f1c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:24.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:24.927+0000 7f5f26000640 1 -- 172.21.15.16:0/704418246 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c00cde0 con 0x7f5f28103750 2024-08-31T06:06:24.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:24.927+0000 7f5f26000640 1 -- 172.21.15.16:0/704418246 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f1c0035f0 con 0x7f5f28103750 2024-08-31T06:06:24.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:24.927+0000 7f5f26000640 1 -- 172.21.15.16:0/704418246 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c010690 con 0x7f5f28103750 2024-08-31T06:06:24.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:24.927+0000 7f5f2ea00640 1 -- 172.21.15.16:0/704418246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f281a2490 con 0x7f5f28103750 2024-08-31T06:06:24.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:24.927+0000 7f5f2ea00640 1 -- 172.21.15.16:0/704418246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f281a8bf0 con 0x7f5f28103750 2024-08-31T06:06:24.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:24.928+0000 7f5f2ea00640 1 -- 172.21.15.16:0/704418246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f28103b60 con 0x7f5f28103750 2024-08-31T06:06:24.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:24.931+0000 7f5f26000640 1 -- 172.21.15.16:0/704418246 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 152645+0+0 (secure 0 0 0) 0x7f5f1c0037a0 con 0x7f5f28103750 2024-08-31T06:06:24.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:24.932+0000 7f5f26000640 1 --2- 172.21.15.16:0/704418246 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f5f100b4c90 0x7f5f100b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:24.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:24.932+0000 7f5f26000640 1 -- 172.21.15.16:0/704418246 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f5f1c0e5820 con 0x7f5f28103750 2024-08-31T06:06:24.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:24.932+0000 7f5f2cc00640 1 --2- 172.21.15.16:0/704418246 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f5f100b4c90 0x7f5f100b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:24.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:24.933+0000 7f5f2cc00640 1 --2- 172.21.15.16:0/704418246 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f5f100b4c90 0x7f5f100b71a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5f28109990 tx=0x7f5f14005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:24.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:24.934+0000 7f5f26000640 1 -- 172.21.15.16:0/704418246 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f5f1c056130 con 0x7f5f28103750 2024-08-31T06:06:25.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:25.132+0000 7f5f2ea00640 1 -- 172.21.15.16:0/704418246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5f2819d360 con 0x7f5f28103750 2024-08-31T06:06:25.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:25.134+0000 7f5f26000640 1 -- 172.21.15.16:0/704418246 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+23043 (secure 0 0 0) 0x7f5f1c0ac6d0 con 0x7f5f28103750 2024-08-31T06:06:25.137 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:06:25.137 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":80,"fsid":"673e4aa0-675d-11ef-bcd4-c7b262605968","created":"2024-08-31T05:55:09.632536+0000","modified":"2024-08-31T06:05:25.039584+0000","last_up_change":"2024-08-31T06:03:27.893697+0000","last_in_change":"2024-08-31T06:03:05.664945+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-31T05:59:23.642270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-31T06:04:20.871835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-31T06:04:24.119499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e349dae-6aa5-4b6d-9107-a945655e11bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6803","nonce":1725037400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6805","nonce":1725037400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6809","nonce":1725037400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6807","nonce":1725037400}]},"public_addr":"172.21.15.16:6803/1725037400","cluster_addr":"172.21.15.16:6805/1725037400","heartbeat_back_addr":"172.21.15.16:6809/1725037400","heartbeat_front_addr":"172.21.15.16:6807/1725037400","state":["exists","up"]},{"osd":1,"uuid":"bd645dc3-93b5-40af-89d8-08ccec22e50d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6801","nonce":4241618440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6803","nonce":4241618440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6807","nonce":4241618440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6805","nonce":4241618440}]},"public_addr":"172.21.15.155:6801/4241618440","cluster_addr":"172.21.15.155:6803/4241618440","heartbeat_back_addr":"172.21.15.155:6807/4241618440","heartbeat_front_addr":"172.21.15.155:6805/4241618440","state":["exists","up"]},{"osd":2,"uuid":"c69c8c19-32f8-400a-86d4-0f4ce240a7de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6801","nonce":2193296422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6803","nonce":2193296422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6807","nonce":2193296422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6805","nonce":2193296422}]},"public_addr":"172.21.15.184:6801/2193296422","cluster_addr":"172.21.15.184:6803/2193296422","heartbeat_back_addr":"172.21.15.184:6807/2193296422","heartbeat_front_addr":"172.21.15.184:6805/2193296422","state":["exists","up"]},{"osd":3,"uuid":"8892e35f-6880-434c-a805-ab8ccbed2bf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6811","nonce":81681815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6813","nonce":81681815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6817","nonce":81681815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6815","nonce":81681815}]},"public_addr":"172.21.15.16:6811/81681815","cluster_addr":"172.21.15.16:6813/81681815","heartbeat_back_addr":"172.21.15.16:6817/81681815","heartbeat_front_addr":"172.21.15.16:6815/81681815","state":["exists","up"]},{"osd":4,"uuid":"d2515e73-88f7-4654-9762-adb8802777ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6809","nonce":3592422019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6811","nonce":3592422019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6815","nonce":3592422019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6813","nonce":3592422019}]},"public_addr":"172.21.15.155:6809/3592422019","cluster_addr":"172.21.15.155:6811/3592422019","heartbeat_back_addr":"172.21.15.155:6815/3592422019","heartbeat_front_addr":"172.21.15.155:6813/3592422019","state":["exists","up"]},{"osd":5,"uuid":"5ce6e2fa-c17c-49b3-9402-f19bfc21076d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6809","nonce":2054894065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6811","nonce":2054894065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6815","nonce":2054894065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6813","nonce":2054894065}]},"public_addr":"172.21.15.184:6809/2054894065","cluster_addr":"172.21.15.184:6811/2054894065","heartbeat_back_addr":"172.21.15.184:6815/2054894065","heartbeat_front_addr":"172.21.15.184:6813/2054894065","state":["exists","up"]},{"osd":6,"uuid":"f257d8d1-ad59-4bd8-821d-bf29e50b547e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6819","nonce":1697421092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6821","nonce":1697421092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6825","nonce":1697421092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6823","nonce":1697421092}]},"public_addr":"172.21.15.16:6819/1697421092","cluster_addr":"172.21.15.16:6821/1697421092","heartbeat_back_addr":"172.21.15.16:6825/1697421092","heartbeat_front_addr":"172.21.15.16:6823/1697421092","state":["exists","up"]},{"osd":7,"uuid":"8e76c20e-8d25-423b-b7e8-de642d9c9a6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6817","nonce":2671827901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6819","nonce":2671827901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6823","nonce":2671827901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6821","nonce":2671827901}]},"public_addr":"172.21.15.155:6817/2671827901","cluster_addr":"172.21.15.155:6819/2671827901","heartbeat_back_addr":"172.21.15.155:6823/2671827901","heartbeat_front_addr":"172.21.15.155:6821/2671827901","state":["exists","up"]},{"osd":8,"uuid":"0f0bb23f-4bc3-4198-8381-5b5cd16ae394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6817","nonce":2479117823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6819","nonce":2479117823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6823","nonce":2479117823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6821","nonce":2479117823}]},"public_addr":"172.21.15.184:6817/2479117823","cluster_addr":"172.21.15.184:6819/2479117823","heartbeat_back_addr":"172.21.15.184:6823/2479117823","heartbeat_front_addr":"172.21.15.184:6821/2479117823","state":["exists","up"]},{"osd":9,"uuid":"bc27afcb-7871-4f72-a043-044fd5fe7d86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6827","nonce":3015700546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6829","nonce":3015700546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6833","nonce":3015700546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6831","nonce":3015700546}]},"public_addr":"172.21.15.16:6827/3015700546","cluster_addr":"172.21.15.16:6829/3015700546","heartbeat_back_addr":"172.21.15.16:6833/3015700546","heartbeat_front_addr":"172.21.15.16:6831/3015700546","state":["exists","up"]},{"osd":10,"uuid":"f2548997-72ee-400e-ab8d-5966d85b2cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6825","nonce":2427122475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6827","nonce":2427122475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6831","nonce":2427122475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6829","nonce":2427122475}]},"public_addr":"172.21.15.155:6825/2427122475","cluster_addr":"172.21.15.155:6827/2427122475","heartbeat_back_addr":"172.21.15.155:6831/2427122475","heartbeat_front_addr":"172.21.15.155:6829/2427122475","state":["exists","up"]},{"osd":11,"uuid":"8297da1d-d3e2-4a4d-9396-426dfafa3e95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6825","nonce":399265701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6827","nonce":399265701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6831","nonce":399265701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6829","nonce":399265701}]},"public_addr":"172.21.15.184:6825/399265701","cluster_addr":"172.21.15.184:6827/399265701","heartbeat_back_addr":"172.21.15.184:6831/399265701","heartbeat_front_addr":"172.21.15.184:6829/399265701","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:58:30.842360+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:58:57.164488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:59:20.778103+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:59:50.123991+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:00:16.789933+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:00:41.788905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:01:12.281376+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:01:39.211858+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:02:04.958771+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:02:35.645947+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:03:01.940433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:03:26.728673+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3923220960":"2024-09-01T05:57:23.153214+0000","172.21.15.16:0/3013533836":"2024-09-01T05:57:23.153214+0000","172.21.15.16:6800/3755683228":"2024-09-01T05:55:45.075902+0000","172.21.15.16:0/3064435470":"2024-09-01T05:55:45.075902+0000","172.21.15.16:0/911709962":"2024-09-01T05:55:45.075902+0000","172.21.15.16:0/2937368967":"2024-09-01T05:55:45.075902+0000","172.21.15.16:6800/1378972230":"2024-09-01T05:56:14.195704+0000","172.21.15.16:0/3047316995":"2024-09-01T05:56:14.195704+0000","172.21.15.16:6801/3755683228":"2024-09-01T05:55:45.075902+0000","172.21.15.16:6801/2571082634":"2024-09-01T05:57:23.153214+0000","172.21.15.16:0/2136704295":"2024-09-01T05:56:14.195704+0000","172.21.15.16:0/2259332778":"2024-09-01T05:56:14.195704+0000","172.21.15.16:6800/2571082634":"2024-09-01T05:57:23.153214+0000","172.21.15.16:6801/1378972230":"2024-09-01T05:56:14.195704+0000","172.21.15.16:0/2126457673":"2024-09-01T05:57:23.153214+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-31T06:06:25.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:25.136+0000 7f5f2ea00640 1 -- 172.21.15.16:0/704418246 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f5f100b4c90 msgr2=0x7f5f100b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:25.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:25.136+0000 7f5f2ea00640 1 --2- 172.21.15.16:0/704418246 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f5f100b4c90 0x7f5f100b71a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5f28109990 tx=0x7f5f14005f90 comp rx=0 tx=0).stop 2024-08-31T06:06:25.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:25.136+0000 7f5f2ea00640 1 -- 172.21.15.16:0/704418246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f28103750 msgr2=0x7f5f2819c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:25.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:25.136+0000 7f5f2ea00640 1 --2- 172.21.15.16:0/704418246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f28103750 0x7f5f2819c230 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c00ea60 tx=0x7f5f1c00ef20 comp rx=0 tx=0).stop 2024-08-31T06:06:25.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:25.137+0000 7f5f2ea00640 1 -- 172.21.15.16:0/704418246 shutdown_connections 2024-08-31T06:06:25.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:25.137+0000 7f5f2ea00640 1 --2- 172.21.15.16:0/704418246 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f5f100b4c90 0x7f5f100b71a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:25.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:25.137+0000 7f5f2ea00640 1 --2- 172.21.15.16:0/704418246 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f28109090 0x7f5f2819c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:25.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:25.137+0000 7f5f2ea00640 1 --2- 172.21.15.16:0/704418246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f281099e0 0x7f5f281a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:25.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:25.137+0000 7f5f2ea00640 1 --2- 172.21.15.16:0/704418246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f28103750 0x7f5f2819c230 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:25.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:25.137+0000 7f5f2ea00640 1 -- 172.21.15.16:0/704418246 >> 172.21.15.16:0/704418246 conn(0x7f5f280ff1a0 msgr2=0x7f5f28107650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:25.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:25.137+0000 7f5f2ea00640 1 -- 172.21.15.16:0/704418246 shutdown_connections 2024-08-31T06:06:25.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:25.137+0000 7f5f2ea00640 1 -- 172.21.15.16:0/704418246 wait complete. 2024-08-31T06:06:25.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:25 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/704418246' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T06:06:25.871 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:25 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/704418246' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T06:06:25.871 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:25 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/704418246' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T06:06:25.916 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-08-31T06:04:27.098649+0000', 'modified': '2024-08-31T06:04:42.745448+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24737, 'mds_1': 24724, 'mds_2': 24712}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24688': {'gid': 24688, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.184:6835/1925472309', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6834', 'nonce': 1925472309}, {'type': 'v1', 'addr': '172.21.15.184:6835', 'nonce': 1925472309}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24694': {'gid': 24694, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.184:6837/1712755588', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6836', 'nonce': 1712755588}, {'type': 'v1', 'addr': '172.21.15.184:6837', 'nonce': 1712755588}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24698': {'gid': 24698, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.155:6837/1089710405', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6836', 'nonce': 1089710405}, {'type': 'v1', 'addr': '172.21.15.155:6837', 'nonce': 1089710405}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24712': {'gid': 24712, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.184:6839/4011051479', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6838', 'nonce': 4011051479}, {'type': 'v1', 'addr': '172.21.15.184:6839', 'nonce': 4011051479}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24724': {'gid': 24724, 'name': 'g', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.16:6839/219664460', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6838', 'nonce': 219664460}, {'type': 'v1', 'addr': '172.21.15.16:6839', 'nonce': 219664460}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24737': {'gid': 24737, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.155:6839/390606290', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6838', 'nonce': 390606290}, {'type': 'v1', 'addr': '172.21.15.155:6839', 'nonce': 390606290}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24737, 'qdb_cluster': [24712, 24724, 24737]} 2024-08-31T06:06:25.916 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-08-31T06:06:25.916 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.i status 2024-08-31T06:06:26.292 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:06:26.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:26 smithi184 ceph-mon[22331]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 8 op/s 2024-08-31T06:06:26.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:26 smithi184 ceph-mon[22331]: mgrmap e22: x(active, since 9m), standbys: z, y 2024-08-31T06:06:26.750 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:26 smithi016 ceph-mon[17194]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 8 op/s 2024-08-31T06:06:26.750 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:26 smithi016 ceph-mon[17194]: mgrmap e22: x(active, since 9m), standbys: z, y 2024-08-31T06:06:26.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:26 smithi155 ceph-mon[21742]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 8 op/s 2024-08-31T06:06:26.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:26 smithi155 ceph-mon[21742]: mgrmap e22: x(active, since 9m), standbys: z, y 2024-08-31T06:06:26.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.926+0000 7f7ab8600640 1 -- 172.21.15.16:0/3192016905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab010b460 msgr2=0x7f7ab010b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:26.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.926+0000 7f7ab8600640 1 --2- 172.21.15.16:0/3192016905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab010b460 0x7f7ab010b870 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009900 tx=0x7f7a9c02a3f0 comp rx=0 tx=0).stop 2024-08-31T06:06:26.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.926+0000 7f7ab8600640 1 -- 172.21.15.16:0/3192016905 shutdown_connections 2024-08-31T06:06:26.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.926+0000 7f7ab8600640 1 --2- 172.21.15.16:0/3192016905 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ab010bdb0 0x7f7ab01125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:26.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.926+0000 7f7ab8600640 1 --2- 172.21.15.16:0/3192016905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab010b460 0x7f7ab010b870 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:26.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.926+0000 7f7ab8600640 1 --2- 172.21.15.16:0/3192016905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ab0105b20 0x7f7ab0105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:26.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.926+0000 7f7ab8600640 1 -- 172.21.15.16:0/3192016905 >> 172.21.15.16:0/3192016905 conn(0x7f7ab01015b0 msgr2=0x7f7ab0103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:26.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.926+0000 7f7ab8600640 1 -- 172.21.15.16:0/3192016905 shutdown_connections 2024-08-31T06:06:26.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.926+0000 7f7ab8600640 1 -- 172.21.15.16:0/3192016905 wait complete. 2024-08-31T06:06:26.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.928+0000 7f7ab8600640 1 Processor -- start 2024-08-31T06:06:26.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.928+0000 7f7ab8600640 1 -- start start 2024-08-31T06:06:26.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.928+0000 7f7ab8600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ab0105b20 0x7f7ab019c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:26.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.928+0000 7f7ab8600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ab010b460 0x7f7ab019c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:26.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.928+0000 7f7ab8600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab010bdb0 0x7f7ab01a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:26.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.928+0000 7f7ab8600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab0115ac0 con 0x7f7ab010bdb0 2024-08-31T06:06:26.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.928+0000 7f7ab8600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab0115940 con 0x7f7ab010b460 2024-08-31T06:06:26.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.928+0000 7f7ab8600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab0115c40 con 0x7f7ab0105b20 2024-08-31T06:06:26.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.928+0000 7f7ab5c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ab0105b20 0x7f7ab019c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:26.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.928+0000 7f7ab6600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab010bdb0 0x7f7ab01a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:26.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.928+0000 7f7ab5200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ab010b460 0x7f7ab019c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:26.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.929+0000 7f7ab6600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab010bdb0 0x7f7ab01a1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45244/0 (socket says 172.21.15.16:45244) 2024-08-31T06:06:26.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.929+0000 7f7ab6600640 1 -- 172.21.15.16:0/3099478457 learned_addr learned my addr 172.21.15.16:0/3099478457 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:26.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.929+0000 7f7ab5c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ab0105b20 0x7f7ab019c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:44930/0 (socket says 172.21.15.16:44930) 2024-08-31T06:06:26.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.929+0000 7f7ab5200640 1 -- 172.21.15.16:0/3099478457 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ab0105b20 msgr2=0x7f7ab019c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:26.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.929+0000 7f7ab5200640 1 --2- 172.21.15.16:0/3099478457 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ab0105b20 0x7f7ab019c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:26.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.929+0000 7f7ab5200640 1 -- 172.21.15.16:0/3099478457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab010bdb0 msgr2=0x7f7ab01a1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:26.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.929+0000 7f7ab5200640 1 --2- 172.21.15.16:0/3099478457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab010bdb0 0x7f7ab01a1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:26.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.929+0000 7f7ab5200640 1 -- 172.21.15.16:0/3099478457 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab01a23c0 con 0x7f7ab010b460 2024-08-31T06:06:26.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.929+0000 7f7ab6600640 1 --2- 172.21.15.16:0/3099478457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab010bdb0 0x7f7ab01a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:06:26.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.929+0000 7f7ab5200640 1 --2- 172.21.15.16:0/3099478457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ab010b460 0x7f7ab019c9f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009230 tx=0x7f7a9c003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:26.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.930+0000 7f7aae000640 1 -- 172.21.15.16:0/3099478457 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c03f070 con 0x7f7ab010b460 2024-08-31T06:06:26.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.930+0000 7f7ab8600640 1 -- 172.21.15.16:0/3099478457 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ab01a2650 con 0x7f7ab010b460 2024-08-31T06:06:26.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.930+0000 7f7aae000640 1 -- 172.21.15.16:0/3099478457 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a9c031b50 con 0x7f7ab010b460 2024-08-31T06:06:26.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.930+0000 7f7ab8600640 1 -- 172.21.15.16:0/3099478457 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ab01a8e10 con 0x7f7ab010b460 2024-08-31T06:06:26.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.930+0000 7f7aae000640 1 -- 172.21.15.16:0/3099478457 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c035030 con 0x7f7ab010b460 2024-08-31T06:06:26.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.933+0000 7f7abf28f740 1 Processor -- start 2024-08-31T06:06:26.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.933+0000 7f7abf28f740 1 -- start start 2024-08-31T06:06:26.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.933+0000 7f7abf28f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5e9ed8f60 0x55d5ea27e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:26.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.933+0000 7f7abf28f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5e9eb00a0 0x55d5ea27ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:26.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.933+0000 7f7abf28f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5e9eb8280 0x55d5ea27f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:26.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.933+0000 7f7abf28f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d5ea1f5550 con 0x55d5e9eb00a0 2024-08-31T06:06:26.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.933+0000 7f7abf28f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d5ea1e4ae0 con 0x55d5e9ed8f60 2024-08-31T06:06:26.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.933+0000 7f7abf28f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d5ea1e8a90 con 0x55d5e9eb8280 2024-08-31T06:06:26.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.934+0000 7f7ab6600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5e9ed8f60 0x55d5ea27e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:26.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.934+0000 7f7ab6600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5e9ed8f60 0x55d5ea27e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:50564/0 (socket says 172.21.15.16:50564) 2024-08-31T06:06:26.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.934+0000 7f7ab5c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5e9eb00a0 0x55d5ea27ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:26.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.934+0000 7f7ab6600640 1 -- 172.21.15.16:0/1243795141 learned_addr learned my addr 172.21.15.16:0/1243795141 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:26.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.934+0000 7f7aae000640 1 -- 172.21.15.16:0/3099478457 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7f7a9c042070 con 0x7f7ab010b460 2024-08-31T06:06:26.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.934+0000 7f7ab6600640 1 -- 172.21.15.16:0/1243795141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5e9eb8280 msgr2=0x55d5ea27f370 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T06:06:26.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.934+0000 7f7ab6600640 1 --2- 172.21.15.16:0/1243795141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5e9eb8280 0x55d5ea27f370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:26.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.934+0000 7f7ab6600640 1 -- 172.21.15.16:0/1243795141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5e9eb00a0 msgr2=0x55d5ea27ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:26.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.934+0000 7f7ab6600640 1 --2- 172.21.15.16:0/1243795141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5e9eb00a0 0x55d5ea27ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:26.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.934+0000 7f7ab6600640 1 -- 172.21.15.16:0/1243795141 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5e9f44540 con 0x55d5e9ed8f60 2024-08-31T06:06:26.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.935+0000 7f7ab6600640 1 --2- 172.21.15.16:0/1243795141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5e9ed8f60 0x55d5ea27e8f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7aa80118b0 tx=0x7f7aa8011d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=972fe91cb825a002 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:26.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.935+0000 7f7aa2a00640 1 -- 172.21.15.16:0/1243795141 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa801e070 con 0x55d5e9ed8f60 2024-08-31T06:06:26.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.935+0000 7f7aa2a00640 1 -- 172.21.15.16:0/1243795141 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7aa800fe70 con 0x55d5e9ed8f60 2024-08-31T06:06:26.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.935+0000 7f7aa2a00640 1 -- 172.21.15.16:0/1243795141 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa8019da0 con 0x55d5e9ed8f60 2024-08-31T06:06:26.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.936+0000 7f7abf28f740 1 -- 172.21.15.16:0/1243795141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5e9ed8f60 msgr2=0x55d5ea27e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:26.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.936+0000 7f7abf28f740 1 --2- 172.21.15.16:0/1243795141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5e9ed8f60 0x55d5ea27e8f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7aa80118b0 tx=0x7f7aa8011d70 comp rx=0 tx=0).stop 2024-08-31T06:06:26.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.936+0000 7f7aae000640 1 --2- 172.21.15.16:0/3099478457 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7a980b4b80 0x7f7a980b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:26.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.936+0000 7f7abf28f740 1 -- 172.21.15.16:0/1243795141 shutdown_connections 2024-08-31T06:06:26.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.936+0000 7f7abf28f740 1 --2- 172.21.15.16:0/1243795141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5e9eb8280 0x55d5ea27f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:26.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.936+0000 7f7abf28f740 1 --2- 172.21.15.16:0/1243795141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5e9eb00a0 0x55d5ea27ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:26.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.936+0000 7f7abf28f740 1 --2- 172.21.15.16:0/1243795141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5e9ed8f60 0x55d5ea27e8f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:26.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.936+0000 7f7abf28f740 1 -- 172.21.15.16:0/1243795141 >> 172.21.15.16:0/1243795141 conn(0x55d5e9ecd350 msgr2=0x55d5ea27c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:26.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.936+0000 7f7aae000640 1 -- 172.21.15.16:0/3099478457 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f7a9c1052f0 con 0x7f7ab010b460 2024-08-31T06:06:26.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.936+0000 7f7abf28f740 1 -- 172.21.15.16:0/1243795141 shutdown_connections 2024-08-31T06:06:26.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.936+0000 7f7ab5c00640 1 --2- 172.21.15.16:0/3099478457 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7a980b4b80 0x7f7a980b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:26.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.936+0000 7f7abf28f740 1 -- 172.21.15.16:0/1243795141 wait complete. 2024-08-31T06:06:26.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.937+0000 7f7abf28f740 1 -- start start 2024-08-31T06:06:26.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.937+0000 7f7ab5c00640 1 --2- 172.21.15.16:0/3099478457 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7a980b4b80 0x7f7a980b7090 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x55d5ea27d570 tx=0x7f7aa4005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:26.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.937+0000 7f7abf28f740 1 Processor -- start 2024-08-31T06:06:26.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.937+0000 7f7abf28f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5e9eb00a0 0x55d5ea289fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:26.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.937+0000 7f7abf28f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5e9eb8280 0x55d5ea28a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:26.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.937+0000 7f7abf28f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5e9ed8f60 0x55d5ea296450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:26.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.937+0000 7f7abf28f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d5ea0475f0 con 0x55d5e9eb00a0 2024-08-31T06:06:26.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.937+0000 7f7abf28f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d5ea16bbd0 con 0x55d5e9eb8280 2024-08-31T06:06:26.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.937+0000 7f7abf28f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d5ea28d0d0 con 0x55d5e9ed8f60 2024-08-31T06:06:26.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.937+0000 7f7abf28f740 10 client.0 _ll_register_callbacks cb 0x55d5ea191210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:06:26.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.938+0000 7f7ab5c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5e9eb00a0 0x55d5ea289fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:26.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.938+0000 7f7ab6600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5e9ed8f60 0x55d5ea296450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:26.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.938+0000 7f7ab5c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5e9eb00a0 0x55d5ea289fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45256/0 (socket says 172.21.15.16:45256) 2024-08-31T06:06:26.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.938+0000 7f7ab5c00640 1 -- 172.21.15.16:0/3307762361 learned_addr learned my addr 172.21.15.16:0/3307762361 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:26.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.938+0000 7f7ab6600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5e9ed8f60 0x55d5ea296450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:44944/0 (socket says 172.21.15.16:44944) 2024-08-31T06:06:26.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.938+0000 7f7ab5200640 1 --2- 172.21.15.16:0/3307762361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5e9eb8280 0x55d5ea28a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:26.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.938+0000 7f7ab6600640 1 -- 172.21.15.16:0/3307762361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5e9eb8280 msgr2=0x55d5ea28a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:26.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.938+0000 7f7ab6600640 1 --2- 172.21.15.16:0/3307762361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5e9eb8280 0x55d5ea28a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:26.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.938+0000 7f7ab6600640 1 -- 172.21.15.16:0/3307762361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5e9eb00a0 msgr2=0x55d5ea289fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:26.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.938+0000 7f7ab6600640 1 --2- 172.21.15.16:0/3307762361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5e9eb00a0 0x55d5ea289fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:26.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.938+0000 7f7ab6600640 1 -- 172.21.15.16:0/3307762361 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5ea1ac650 con 0x55d5e9ed8f60 2024-08-31T06:06:26.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.938+0000 7f7ab5c00640 1 --2- 172.21.15.16:0/3307762361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5e9eb00a0 0x55d5ea289fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:06:26.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.939+0000 7f7ab6600640 1 --2- 172.21.15.16:0/3307762361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5e9ed8f60 0x55d5ea296450 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7aa80049d0 tx=0x7f7aa8019660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fd16209b7b28bb7 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:26.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.939+0000 7f7a97400640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:06:26.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.939+0000 7f7a97400640 1 -- 172.21.15.16:0/3307762361 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa8027020 con 0x55d5e9ed8f60 2024-08-31T06:06:26.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.939+0000 7f7abf28f740 1 -- 172.21.15.16:0/3307762361 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7aa800fe70 con 0x55d5e9ed8f60 2024-08-31T06:06:26.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.939+0000 7f7a97400640 1 -- 172.21.15.16:0/3307762361 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f7aa8004a40 con 0x55d5e9ed8f60 2024-08-31T06:06:26.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.939+0000 7f7a97400640 1 -- 172.21.15.16:0/3307762361 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7aa8032050 con 0x55d5e9ed8f60 2024-08-31T06:06:26.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.939+0000 7f7abf28f740 10 client.25198 fetch_fsmap learned FSMap version 24 2024-08-31T06:06:26.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.939+0000 7f7abf28f740 1 -- 172.21.15.16:0/3307762361 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d5ea298e30 con 0x55d5e9ed8f60 2024-08-31T06:06:26.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.940+0000 7f7a97400640 1 -- 172.21.15.16:0/3307762361 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7aa800fe70 con 0x55d5e9ed8f60 2024-08-31T06:06:26.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.940+0000 7f7a97400640 1 -- 172.21.15.16:0/3307762361 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f7aa8005300 con 0x55d5e9ed8f60 2024-08-31T06:06:26.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.940+0000 7f7abf28f740 10 client.25198 fetch_fsmap finished waiting for FSMap version 24 2024-08-31T06:06:26.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.940+0000 7f7abf28f740 10 client.25198 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-31T06:06:26.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.940+0000 7f7abf28f740 20 client.25198 populate_metadata read hostname 'smithi016' 2024-08-31T06:06:26.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.940+0000 7f7abf28f740 1 --2- 172.21.15.16:0/3307762361 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55d5e9e978e0 0x55d5ea29c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:26.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.940+0000 7f7abf28f740 4 client.25198 mds_command: new command op to 24712 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:06:26.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.940+0000 7f7abf28f740 1 -- 172.21.15.16:0/3307762361 --> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5ea1f74a0 con 0x55d5e9e978e0 2024-08-31T06:06:26.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.941+0000 7f7ab5c00640 1 --2- 172.21.15.16:0/3307762361 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55d5e9e978e0 0x55d5ea29c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:26.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.941+0000 7f7ab5c00640 1 --2- 172.21.15.16:0/3307762361 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55d5e9e978e0 0x55d5ea29c6b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fbad88748d00e378 server_cookie=80bc69f3a0b08a7b in_seq=0 out_seq=0 2024-08-31T06:06:26.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.941+0000 7f7a97400640 10 client.25198 ms_handle_connect on v2:172.21.15.184:6838/4011051479 2024-08-31T06:06:26.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.942+0000 7f7a97400640 1 -- 172.21.15.16:0/3307762361 <== mds.2 v2:172.21.15.184:6838/4011051479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d5ea1f74a0 con 0x55d5e9e978e0 2024-08-31T06:06:26.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.942+0000 7f7a97400640 10 client.25198 handle_command_reply: tid=0 2024-08-31T06:06:26.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.942+0000 7f7abf28f740 1 client.25198 shutdown 2024-08-31T06:06:26.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.942+0000 7f7abf28f740 1 -- 172.21.15.16:0/3307762361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5e9ed8f60 msgr2=0x55d5ea296450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:26.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.942+0000 7f7abf28f740 1 --2- 172.21.15.16:0/3307762361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5e9ed8f60 0x55d5ea296450 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7aa80049d0 tx=0x7f7aa8019660 comp rx=0 tx=0).stop 2024-08-31T06:06:26.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.943+0000 7f7abf28f740 1 -- 172.21.15.16:0/3307762361 shutdown_connections 2024-08-31T06:06:26.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.943+0000 7f7abf28f740 1 --2- 172.21.15.16:0/3307762361 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55d5e9e978e0 0x55d5ea29c6b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:26.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.943+0000 7f7abf28f740 1 --2- 172.21.15.16:0/3307762361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5e9ed8f60 0x55d5ea296450 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:26.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.943+0000 7f7abf28f740 1 --2- 172.21.15.16:0/3307762361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5e9eb8280 0x55d5ea28a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:26.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.943+0000 7f7abf28f740 1 --2- 172.21.15.16:0/3307762361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5e9eb00a0 0x55d5ea289fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:26.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.943+0000 7f7a97400640 0 client.25198 ms_handle_reset on v2:172.21.15.184:6838/4011051479 2024-08-31T06:06:26.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.943+0000 7f7abf28f740 1 -- 172.21.15.16:0/3307762361 >> 172.21.15.16:0/3307762361 conn(0x55d5e9ecd350 msgr2=0x55d5ea281a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:26.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.943+0000 7f7abf28f740 1 -- 172.21.15.16:0/3307762361 shutdown_connections 2024-08-31T06:06:26.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.943+0000 7f7abf28f740 1 -- 172.21.15.16:0/3307762361 wait complete. 2024-08-31T06:06:26.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.943+0000 7f7abf28f740 20 client.25198 trim_cache size 0 max 16384 2024-08-31T06:06:26.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.962+0000 7f7abf28f740 1 Processor -- start 2024-08-31T06:06:26.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.962+0000 7f7abf28f740 1 -- start start 2024-08-31T06:06:26.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.963+0000 7f7abf28f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5e9eb00a0 0x55d5ea2963c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:26.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.963+0000 7f7abf28f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5e9eb8280 0x55d5ea285150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:26.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.963+0000 7f7abf28f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5e9ed8f60 0x55d5ea285690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:26.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.963+0000 7f7abf28f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d5ea29a9f0 con 0x55d5e9ed8f60 2024-08-31T06:06:26.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.963+0000 7f7abf28f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d5ea29b0c0 con 0x55d5e9eb8280 2024-08-31T06:06:26.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.963+0000 7f7abf28f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d5ea29b790 con 0x55d5e9eb00a0 2024-08-31T06:06:26.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.963+0000 7f7ab6600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5e9ed8f60 0x55d5ea285690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:26.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.963+0000 7f7ab6600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5e9ed8f60 0x55d5ea285690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45262/0 (socket says 172.21.15.16:45262) 2024-08-31T06:06:26.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.963+0000 7f7ab6600640 1 -- 172.21.15.16:0/43787021 learned_addr learned my addr 172.21.15.16:0/43787021 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:26.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.963+0000 7f7ab5c00640 1 --2- 172.21.15.16:0/43787021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5e9eb00a0 0x55d5ea2963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:26.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.963+0000 7f7ab5200640 1 --2- 172.21.15.16:0/43787021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5e9eb8280 0x55d5ea285150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:26.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.964+0000 7f7ab6600640 1 -- 172.21.15.16:0/43787021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5e9eb00a0 msgr2=0x55d5ea2963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:26.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.964+0000 7f7ab6600640 1 --2- 172.21.15.16:0/43787021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5e9eb00a0 0x55d5ea2963c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:26.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.964+0000 7f7ab6600640 1 -- 172.21.15.16:0/43787021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5e9eb8280 msgr2=0x55d5ea285150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:26.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.964+0000 7f7ab6600640 1 --2- 172.21.15.16:0/43787021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5e9eb8280 0x55d5ea285150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:26.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.964+0000 7f7ab6600640 1 -- 172.21.15.16:0/43787021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5ea280ed0 con 0x55d5e9ed8f60 2024-08-31T06:06:26.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.964+0000 7f7ab6600640 1 --2- 172.21.15.16:0/43787021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5e9ed8f60 0x55d5ea285690 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8005c40 tx=0x7f7aa801aa50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d177a63cc9e676 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:26.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.964+0000 7f7aa2000640 1 -- 172.21.15.16:0/43787021 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa8027040 con 0x55d5e9ed8f60 2024-08-31T06:06:26.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.964+0000 7f7aa2000640 1 -- 172.21.15.16:0/43787021 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7aa80114f0 con 0x55d5e9ed8f60 2024-08-31T06:06:26.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.964+0000 7f7aa2000640 1 -- 172.21.15.16:0/43787021 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa801e040 con 0x55d5e9ed8f60 2024-08-31T06:06:26.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.965+0000 7f7abf28f740 1 -- 172.21.15.16:0/43787021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5e9ed8f60 msgr2=0x55d5ea285690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:26.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.965+0000 7f7abf28f740 1 --2- 172.21.15.16:0/43787021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5e9ed8f60 0x55d5ea285690 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8005c40 tx=0x7f7aa801aa50 comp rx=0 tx=0).stop 2024-08-31T06:06:26.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.965+0000 7f7abf28f740 1 -- 172.21.15.16:0/43787021 shutdown_connections 2024-08-31T06:06:26.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.965+0000 7f7abf28f740 1 --2- 172.21.15.16:0/43787021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5e9ed8f60 0x55d5ea285690 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:26.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.965+0000 7f7abf28f740 1 --2- 172.21.15.16:0/43787021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5e9eb8280 0x55d5ea285150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:26.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.965+0000 7f7abf28f740 1 --2- 172.21.15.16:0/43787021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5e9eb00a0 0x55d5ea2963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:26.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.965+0000 7f7abf28f740 1 -- 172.21.15.16:0/43787021 >> 172.21.15.16:0/43787021 conn(0x55d5e9ecd350 msgr2=0x55d5ea2774c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:26.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.965+0000 7f7abf28f740 1 -- 172.21.15.16:0/43787021 shutdown_connections 2024-08-31T06:06:26.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.965+0000 7f7abf28f740 1 -- 172.21.15.16:0/43787021 wait complete. 2024-08-31T06:06:26.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.966+0000 7f7abf28f740 1 -- start start 2024-08-31T06:06:26.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.966+0000 7f7abf28f740 1 Processor -- start 2024-08-31T06:06:26.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.966+0000 7f7abf28f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5e9eb00a0 0x55d5ea29dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:26.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.967+0000 7f7abf28f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5e9eb8280 0x55d5ea29e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:26.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.967+0000 7f7abf28f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5e9ed8f60 0x55d5ea2a3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:26.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.967+0000 7f7abf28f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d5ea29c530 con 0x55d5e9eb8280 2024-08-31T06:06:26.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.967+0000 7f7abf28f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d5ea29be60 con 0x55d5e9ed8f60 2024-08-31T06:06:26.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.967+0000 7f7abf28f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d5ea29cc00 con 0x55d5e9eb00a0 2024-08-31T06:06:26.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.967+0000 7f7abf28f740 10 client.0 _ll_register_callbacks cb 0x7f7aa40097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:06:26.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.967+0000 7f7ab5200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5e9eb8280 0x55d5ea29e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:26.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.967+0000 7f7ab5c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5e9eb00a0 0x55d5ea29dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:26.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.967+0000 7f7ab6600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5e9ed8f60 0x55d5ea2a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:26.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.967+0000 7f7ab5200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5e9eb8280 0x55d5ea29e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45264/0 (socket says 172.21.15.16:45264) 2024-08-31T06:06:26.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.967+0000 7f7ab5200640 1 -- 172.21.15.16:0/2813671274 learned_addr learned my addr 172.21.15.16:0/2813671274 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:26.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.967+0000 7f7ab5200640 1 -- 172.21.15.16:0/2813671274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5e9eb00a0 msgr2=0x55d5ea29dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:26.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.967+0000 7f7ab5200640 1 --2- 172.21.15.16:0/2813671274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5e9eb00a0 0x55d5ea29dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:26.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.967+0000 7f7ab5200640 1 -- 172.21.15.16:0/2813671274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5e9ed8f60 msgr2=0x55d5ea2a3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:26.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.967+0000 7f7ab5200640 1 --2- 172.21.15.16:0/2813671274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5e9ed8f60 0x55d5ea2a3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:26.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.967+0000 7f7ab5200640 1 -- 172.21.15.16:0/2813671274 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5ea2a3b80 con 0x55d5e9eb8280 2024-08-31T06:06:26.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.967+0000 7f7ab6600640 1 --2- 172.21.15.16:0/2813671274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5e9ed8f60 0x55d5ea2a3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:06:26.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.968+0000 7f7ab5200640 1 --2- 172.21.15.16:0/2813671274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5e9eb8280 0x55d5ea29e570 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c0cb9b0 tx=0x7f7a9c0b3280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd799b23ff9d7d8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:26.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.968+0000 7f7a97400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:06:26.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.968+0000 7f7a97400640 1 -- 172.21.15.16:0/2813671274 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c049040 con 0x55d5e9eb8280 2024-08-31T06:06:26.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.968+0000 7f7abf28f740 1 -- 172.21.15.16:0/2813671274 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7aa80114f0 con 0x55d5e9eb8280 2024-08-31T06:06:26.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.968+0000 7f7a97400640 1 -- 172.21.15.16:0/2813671274 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f7a9c0d0100 con 0x55d5e9eb8280 2024-08-31T06:06:26.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.968+0000 7f7a97400640 1 -- 172.21.15.16:0/2813671274 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7a9c0cb7c0 con 0x55d5e9eb8280 2024-08-31T06:06:26.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.968+0000 7f7abf28f740 10 client.15192 fetch_fsmap learned FSMap version 24 2024-08-31T06:06:26.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.968+0000 7f7abf28f740 1 -- 172.21.15.16:0/2813671274 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d5ea2a4f60 con 0x55d5e9eb8280 2024-08-31T06:06:26.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.968+0000 7f7a97400640 1 -- 172.21.15.16:0/2813671274 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7aa80114f0 con 0x55d5e9eb8280 2024-08-31T06:06:26.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.969+0000 7f7a97400640 1 -- 172.21.15.16:0/2813671274 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f7a9c0eece0 con 0x55d5e9eb8280 2024-08-31T06:06:26.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.969+0000 7f7abf28f740 10 client.15192 fetch_fsmap finished waiting for FSMap version 24 2024-08-31T06:06:26.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.969+0000 7f7abf28f740 10 client.15192 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-31T06:06:26.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.969+0000 7f7abf28f740 20 client.15192 populate_metadata read hostname 'smithi016' 2024-08-31T06:06:26.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.969+0000 7f7abf28f740 1 --2- 172.21.15.16:0/2813671274 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55d5e9e6cc90 0x55d5ea30ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:26.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.969+0000 7f7abf28f740 4 client.15192 mds_command: new command op to 24712 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-31T06:06:26.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.969+0000 7f7abf28f740 1 -- 172.21.15.16:0/2813671274 --> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] -- command(tid 0: {"prefix": "status"}) -- 0x55d5ea1f2f40 con 0x55d5e9e6cc90 2024-08-31T06:06:26.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.969+0000 7f7ab5c00640 1 --2- 172.21.15.16:0/2813671274 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55d5e9e6cc90 0x55d5ea30ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:26.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.970+0000 7f7ab5c00640 1 --2- 172.21.15.16:0/2813671274 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55d5e9e6cc90 0x55d5ea30ab70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2d3dddfebc7c18c5 server_cookie=784a7966e1a89068 in_seq=0 out_seq=0 2024-08-31T06:06:26.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.970+0000 7f7a97400640 10 client.15192 ms_handle_connect on v2:172.21.15.184:6838/4011051479 2024-08-31T06:06:26.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.970+0000 7f7a97400640 1 -- 172.21.15.16:0/2813671274 <== mds.2 v2:172.21.15.184:6838/4011051479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55d5ea1f2f40 con 0x55d5e9e6cc90 2024-08-31T06:06:26.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.970+0000 7f7a97400640 10 client.15192 handle_command_reply: tid=0 2024-08-31T06:06:26.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.970+0000 7f7abf28f740 1 client.15192 shutdown 2024-08-31T06:06:26.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.970+0000 7f7abf28f740 1 -- 172.21.15.16:0/2813671274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5e9eb8280 msgr2=0x55d5ea29e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:26.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.970+0000 7f7abf28f740 1 --2- 172.21.15.16:0/2813671274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5e9eb8280 0x55d5ea29e570 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c0cb9b0 tx=0x7f7a9c0b3280 comp rx=0 tx=0).stop 2024-08-31T06:06:26.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.971+0000 7f7abf28f740 1 -- 172.21.15.16:0/2813671274 shutdown_connections 2024-08-31T06:06:26.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.971+0000 7f7abf28f740 1 --2- 172.21.15.16:0/2813671274 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55d5e9e6cc90 0x55d5ea30ab70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:26.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.971+0000 7f7abf28f740 1 --2- 172.21.15.16:0/2813671274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5e9ed8f60 0x55d5ea2a3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:26.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.971+0000 7f7abf28f740 1 --2- 172.21.15.16:0/2813671274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5e9eb8280 0x55d5ea29e570 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:26.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.971+0000 7f7abf28f740 1 --2- 172.21.15.16:0/2813671274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5e9eb00a0 0x55d5ea29dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:26.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.971+0000 7f7abf28f740 1 -- 172.21.15.16:0/2813671274 >> 172.21.15.16:0/2813671274 conn(0x55d5e9ecd350 msgr2=0x55d5ea285150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:26.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.971+0000 7f7a97400640 0 client.15192 ms_handle_reset on v2:172.21.15.184:6838/4011051479 2024-08-31T06:06:26.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.971+0000 7f7abf28f740 1 -- 172.21.15.16:0/2813671274 shutdown_connections 2024-08-31T06:06:26.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.971+0000 7f7abf28f740 1 -- 172.21.15.16:0/2813671274 wait complete. 2024-08-31T06:06:26.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.971+0000 7f7abf28f740 20 client.15192 trim_cache size 0 max 16384 2024-08-31T06:06:26.978 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:06:26.978 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "673e4aa0-675d-11ef-bcd4-c7b262605968", 2024-08-31T06:06:26.978 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 2, 2024-08-31T06:06:26.978 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24712, 2024-08-31T06:06:26.979 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-08-31T06:06:26.979 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-08-31T06:06:26.979 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-08-31T06:06:26.979 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 108.04026979, 2024-08-31T06:06:26.979 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 24, 2024-08-31T06:06:26.979 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 80, 2024-08-31T06:06:26.979 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 78, 2024-08-31T06:06:26.979 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 144.26525966599999 2024-08-31T06:06:26.979 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:06:26.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.972+0000 7f7ab8600640 1 -- 172.21.15.16:0/3099478457 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7a980b4b80 msgr2=0x7f7a980b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:26.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.972+0000 7f7ab8600640 1 --2- 172.21.15.16:0/3099478457 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7a980b4b80 0x7f7a980b7090 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x55d5ea27d570 tx=0x7f7aa4005f90 comp rx=0 tx=0).stop 2024-08-31T06:06:26.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.972+0000 7f7ab8600640 1 -- 172.21.15.16:0/3099478457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ab010b460 msgr2=0x7f7ab019c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:26.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.972+0000 7f7ab8600640 1 --2- 172.21.15.16:0/3099478457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ab010b460 0x7f7ab019c9f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009230 tx=0x7f7a9c003b90 comp rx=0 tx=0).stop 2024-08-31T06:06:26.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.972+0000 7f7ab8600640 1 -- 172.21.15.16:0/3099478457 shutdown_connections 2024-08-31T06:06:26.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.972+0000 7f7ab8600640 1 --2- 172.21.15.16:0/3099478457 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7a980b4b80 0x7f7a980b7090 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:26.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.972+0000 7f7ab8600640 1 --2- 172.21.15.16:0/3099478457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab010bdb0 0x7f7ab01a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:26.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.972+0000 7f7ab8600640 1 --2- 172.21.15.16:0/3099478457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ab010b460 0x7f7ab019c9f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:26.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.972+0000 7f7ab8600640 1 --2- 172.21.15.16:0/3099478457 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ab0105b20 0x7f7ab019c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:26.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.972+0000 7f7ab8600640 1 -- 172.21.15.16:0/3099478457 >> 172.21.15.16:0/3099478457 conn(0x7f7ab01015b0 msgr2=0x7f7ab010aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:26.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.972+0000 7f7ab8600640 1 -- 172.21.15.16:0/3099478457 shutdown_connections 2024-08-31T06:06:26.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:26.972+0000 7f7ab8600640 1 -- 172.21.15.16:0/3099478457 wait complete. 2024-08-31T06:06:27.759 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.g status 2024-08-31T06:06:28.135 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:06:28.576 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:28 smithi016 ceph-mon[17194]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-08-31T06:06:28.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:28 smithi184 ceph-mon[22331]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-08-31T06:06:28.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.752+0000 7fb0d7e00640 1 -- 172.21.15.16:0/3007361076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0d810b2a0 msgr2=0x7fb0d810b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:28.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.752+0000 7fb0d7e00640 1 --2- 172.21.15.16:0/3007361076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0d810b2a0 0x7fb0d810b6b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4009900 tx=0x7fb0c402a410 comp rx=0 tx=0).stop 2024-08-31T06:06:28.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.753+0000 7fb0d7e00640 1 -- 172.21.15.16:0/3007361076 shutdown_connections 2024-08-31T06:06:28.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.753+0000 7fb0d7e00640 1 --2- 172.21.15.16:0/3007361076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0d810bbf0 0x7fb0d8112420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:28.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.753+0000 7fb0d7e00640 1 --2- 172.21.15.16:0/3007361076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0d810b2a0 0x7fb0d810b6b0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:28.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.753+0000 7fb0d7e00640 1 --2- 172.21.15.16:0/3007361076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0d8105960 0x7fb0d8105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:28.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.753+0000 7fb0d7e00640 1 -- 172.21.15.16:0/3007361076 >> 172.21.15.16:0/3007361076 conn(0x7fb0d8101410 msgr2=0x7fb0d8103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:28.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.753+0000 7fb0d7e00640 1 -- 172.21.15.16:0/3007361076 shutdown_connections 2024-08-31T06:06:28.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.753+0000 7fb0d7e00640 1 -- 172.21.15.16:0/3007361076 wait complete. 2024-08-31T06:06:28.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.754+0000 7fb0d7e00640 1 Processor -- start 2024-08-31T06:06:28.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.754+0000 7fb0d7e00640 1 -- start start 2024-08-31T06:06:28.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.754+0000 7fb0d7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0d8105960 0x7fb0d819c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:28.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.754+0000 7fb0d7e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0d810b2a0 0x7fb0d819c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:28.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.754+0000 7fb0d7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0d810bbf0 0x7fb0d81a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:28.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.754+0000 7fb0d7e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d81158b0 con 0x7fb0d8105960 2024-08-31T06:06:28.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.754+0000 7fb0d7e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d8115730 con 0x7fb0d810bbf0 2024-08-31T06:06:28.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.754+0000 7fb0d7e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d8115a30 con 0x7fb0d810b2a0 2024-08-31T06:06:28.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.754+0000 7fb0d6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0d8105960 0x7fb0d819c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:28.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.755+0000 7fb0d6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0d8105960 0x7fb0d819c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45270/0 (socket says 172.21.15.16:45270) 2024-08-31T06:06:28.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.755+0000 7fb0d6a00640 1 -- 172.21.15.16:0/1118765627 learned_addr learned my addr 172.21.15.16:0/1118765627 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:28.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.755+0000 7fb0d6000640 1 --2- 172.21.15.16:0/1118765627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0d810b2a0 0x7fb0d819c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:28.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.755+0000 7fb0d7400640 1 --2- 172.21.15.16:0/1118765627 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0d810bbf0 0x7fb0d81a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:28.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.755+0000 7fb0d6a00640 1 -- 172.21.15.16:0/1118765627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0d810b2a0 msgr2=0x7fb0d819c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:28.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.755+0000 7fb0d6a00640 1 --2- 172.21.15.16:0/1118765627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0d810b2a0 0x7fb0d819c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:28.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.755+0000 7fb0d6a00640 1 -- 172.21.15.16:0/1118765627 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0d810bbf0 msgr2=0x7fb0d81a1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:28.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.755+0000 7fb0d6a00640 1 --2- 172.21.15.16:0/1118765627 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0d810bbf0 0x7fb0d81a1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:28.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.755+0000 7fb0d6a00640 1 -- 172.21.15.16:0/1118765627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d81a21f0 con 0x7fb0d8105960 2024-08-31T06:06:28.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.755+0000 7fb0d6a00640 1 --2- 172.21.15.16:0/1118765627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0d8105960 0x7fb0d819c280 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb0c800e9f0 tx=0x7fb0c800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:28.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.756+0000 7fb0cf400640 1 -- 172.21.15.16:0/1118765627 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c800cd80 con 0x7fb0d8105960 2024-08-31T06:06:28.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.756+0000 7fb0cf400640 1 -- 172.21.15.16:0/1118765627 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0c80054d0 con 0x7fb0d8105960 2024-08-31T06:06:28.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.756+0000 7fb0cf400640 1 -- 172.21.15.16:0/1118765627 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c8010690 con 0x7fb0d8105960 2024-08-31T06:06:28.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.756+0000 7fb0d7e00640 1 -- 172.21.15.16:0/1118765627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0d81a24e0 con 0x7fb0d8105960 2024-08-31T06:06:28.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.756+0000 7fb0d7e00640 1 -- 172.21.15.16:0/1118765627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0d81a8c40 con 0x7fb0d8105960 2024-08-31T06:06:28.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.759+0000 7fb0e4245740 1 Processor -- start 2024-08-31T06:06:28.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.759+0000 7fb0e4245740 1 -- start start 2024-08-31T06:06:28.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.759+0000 7fb0cf400640 1 -- 172.21.15.16:0/1118765627 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7fb0c8004e90 con 0x7fb0d8105960 2024-08-31T06:06:28.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.760+0000 7fb0e4245740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af9d8f7f60 0x55af9dc9daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:28.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.760+0000 7fb0e4245740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af9d8cf0a0 0x55af9dc9dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:28.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.760+0000 7fb0e4245740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af9d8d7280 0x55af9dc9e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:28.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.760+0000 7fb0e4245740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55af9dc14550 con 0x55af9d8f7f60 2024-08-31T06:06:28.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.760+0000 7fb0e4245740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55af9dc03ae0 con 0x55af9d8d7280 2024-08-31T06:06:28.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.760+0000 7fb0e4245740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55af9dc07a90 con 0x55af9d8cf0a0 2024-08-31T06:06:28.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.760+0000 7fb0d7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af9d8f7f60 0x55af9dc9daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:28.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.760+0000 7fb0d7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af9d8f7f60 0x55af9dc9daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45278/0 (socket says 172.21.15.16:45278) 2024-08-31T06:06:28.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.760+0000 7fb0d7400640 1 -- 172.21.15.16:0/2724601216 learned_addr learned my addr 172.21.15.16:0/2724601216 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:28.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.760+0000 7fb0d6a00640 1 --2- 172.21.15.16:0/2724601216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af9d8cf0a0 0x55af9dc9dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:28.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.760+0000 7fb0d6000640 1 --2- 172.21.15.16:0/2724601216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af9d8d7280 0x55af9dc9e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:28.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.761+0000 7fb0d7400640 1 -- 172.21.15.16:0/2724601216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af9d8cf0a0 msgr2=0x55af9dc9dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:28.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.761+0000 7fb0d7400640 1 --2- 172.21.15.16:0/2724601216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af9d8cf0a0 0x55af9dc9dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:28.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.761+0000 7fb0d7400640 1 -- 172.21.15.16:0/2724601216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af9d8d7280 msgr2=0x55af9dc9e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:28.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.761+0000 7fb0d7400640 1 --2- 172.21.15.16:0/2724601216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af9d8d7280 0x55af9dc9e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:28.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.761+0000 7fb0d7400640 1 -- 172.21.15.16:0/2724601216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af9d963540 con 0x55af9d8f7f60 2024-08-31T06:06:28.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.761+0000 7fb0d7400640 1 --2- 172.21.15.16:0/2724601216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af9d8f7f60 0x55af9dc9daa0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0009fd0 tx=0x7fb0d00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c9b7803d37dbac server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:28.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.761+0000 7fb0c3400640 1 -- 172.21.15.16:0/2724601216 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d0004d20 con 0x55af9d8f7f60 2024-08-31T06:06:28.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.761+0000 7fb0c3400640 1 -- 172.21.15.16:0/2724601216 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0d0004050 con 0x55af9d8f7f60 2024-08-31T06:06:28.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.761+0000 7fb0c3400640 1 -- 172.21.15.16:0/2724601216 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d0005280 con 0x55af9d8f7f60 2024-08-31T06:06:28.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.762+0000 7fb0cf400640 1 --2- 172.21.15.16:0/1118765627 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb0bc0b4c00 0x7fb0bc0b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:28.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.762+0000 7fb0cf400640 1 -- 172.21.15.16:0/1118765627 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7fb0c80e5670 con 0x7fb0d8105960 2024-08-31T06:06:28.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.762+0000 7fb0d6a00640 1 --2- 172.21.15.16:0/1118765627 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb0bc0b4c00 0x7fb0bc0b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:28.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.762+0000 7fb0e4245740 1 -- 172.21.15.16:0/2724601216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af9d8f7f60 msgr2=0x55af9dc9daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:28.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.762+0000 7fb0e4245740 1 --2- 172.21.15.16:0/2724601216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af9d8f7f60 0x55af9dc9daa0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0009fd0 tx=0x7fb0d00118d0 comp rx=0 tx=0).stop 2024-08-31T06:06:28.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.762+0000 7fb0e4245740 1 -- 172.21.15.16:0/2724601216 shutdown_connections 2024-08-31T06:06:28.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.762+0000 7fb0e4245740 1 --2- 172.21.15.16:0/2724601216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af9d8cf0a0 0x55af9dc9dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:28.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.762+0000 7fb0e4245740 1 --2- 172.21.15.16:0/2724601216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af9d8d7280 0x55af9dc9e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:28.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.762+0000 7fb0e4245740 1 --2- 172.21.15.16:0/2724601216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af9d8f7f60 0x55af9dc9daa0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:28.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.762+0000 7fb0e4245740 1 -- 172.21.15.16:0/2724601216 >> 172.21.15.16:0/2724601216 conn(0x55af9d8ec350 msgr2=0x55af9dc9b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:28.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.763+0000 7fb0e4245740 1 -- 172.21.15.16:0/2724601216 shutdown_connections 2024-08-31T06:06:28.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.763+0000 7fb0e4245740 1 -- 172.21.15.16:0/2724601216 wait complete. 2024-08-31T06:06:28.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.763+0000 7fb0d6a00640 1 --2- 172.21.15.16:0/1118765627 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb0bc0b4c00 0x7fb0bc0b7110 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb0c80abc80 tx=0x7fb0c80eb110 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:28.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.763+0000 7fb0e4245740 1 -- start start 2024-08-31T06:06:28.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.763+0000 7fb0e4245740 1 Processor -- start 2024-08-31T06:06:28.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.764+0000 7fb0e4245740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af9d8cf0a0 0x55af9dca9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:28.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.764+0000 7fb0e4245740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af9d8d7280 0x55af9dca9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:28.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.764+0000 7fb0e4245740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af9d8f7f60 0x55af9dcb55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:28.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.764+0000 7fb0e4245740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55af9da665f0 con 0x55af9d8d7280 2024-08-31T06:06:28.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.764+0000 7fb0e4245740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55af9db8abd0 con 0x55af9d8cf0a0 2024-08-31T06:06:28.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.764+0000 7fb0e4245740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55af9dcac2d0 con 0x55af9d8f7f60 2024-08-31T06:06:28.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.764+0000 7fb0e4245740 10 client.0 _ll_register_callbacks cb 0x55af9dbb0210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:06:28.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.764+0000 7fb0d6000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af9d8d7280 0x55af9dca9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:28.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.764+0000 7fb0d7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af9d8cf0a0 0x55af9dca9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:28.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.764+0000 7fb0d7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af9d8cf0a0 0x55af9dca9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:50640/0 (socket says 172.21.15.16:50640) 2024-08-31T06:06:28.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.764+0000 7fb0d6000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af9d8d7280 0x55af9dca9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45290/0 (socket says 172.21.15.16:45290) 2024-08-31T06:06:28.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.764+0000 7fb0d7400640 1 -- 172.21.15.16:0/3010435692 learned_addr learned my addr 172.21.15.16:0/3010435692 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:28.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.764+0000 7fb0d7400640 1 --2- 172.21.15.16:0/3010435692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af9d8f7f60 0x55af9dcb55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:28.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.765+0000 7fb0d6000640 1 -- 172.21.15.16:0/3010435692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af9d8f7f60 msgr2=0x55af9dcb55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:28.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.765+0000 7fb0d6000640 1 --2- 172.21.15.16:0/3010435692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af9d8f7f60 0x55af9dcb55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:28.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.765+0000 7fb0d6000640 1 -- 172.21.15.16:0/3010435692 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af9d8cf0a0 msgr2=0x55af9dca9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:28.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.765+0000 7fb0d6000640 1 --2- 172.21.15.16:0/3010435692 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af9d8cf0a0 0x55af9dca9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:28.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.765+0000 7fb0d6000640 1 -- 172.21.15.16:0/3010435692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af9dbcb650 con 0x55af9d8d7280 2024-08-31T06:06:28.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.765+0000 7fb0d6000640 1 --2- 172.21.15.16:0/3010435692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af9d8d7280 0x55af9dca9770 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb0c402a930 tx=0x7fb0c4032e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c67098377a73c295 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:28.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.765+0000 7fb0b7e00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:06:28.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.765+0000 7fb0b7e00640 1 -- 172.21.15.16:0/3010435692 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c4035040 con 0x55af9d8d7280 2024-08-31T06:06:28.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.765+0000 7fb0b7e00640 1 -- 172.21.15.16:0/3010435692 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7fb0c4031760 con 0x55af9d8d7280 2024-08-31T06:06:28.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.765+0000 7fb0e4245740 1 -- 172.21.15.16:0/3010435692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0d0004050 con 0x55af9d8d7280 2024-08-31T06:06:28.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.766+0000 7fb0b7e00640 1 -- 172.21.15.16:0/3010435692 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb0c4004040 con 0x55af9d8d7280 2024-08-31T06:06:28.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.766+0000 7fb0e4245740 10 client.15204 fetch_fsmap learned FSMap version 24 2024-08-31T06:06:28.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.766+0000 7fb0e4245740 1 -- 172.21.15.16:0/3010435692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55af9dcb8090 con 0x55af9d8d7280 2024-08-31T06:06:28.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.766+0000 7fb0b7e00640 1 -- 172.21.15.16:0/3010435692 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0d0004050 con 0x55af9d8d7280 2024-08-31T06:06:28.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.766+0000 7fb0b7e00640 1 -- 172.21.15.16:0/3010435692 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fb0c4004750 con 0x55af9d8d7280 2024-08-31T06:06:28.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.766+0000 7fb0e4245740 10 client.15204 fetch_fsmap finished waiting for FSMap version 24 2024-08-31T06:06:28.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.767+0000 7fb0e4245740 10 client.15204 resolve_mds: resolved name 'g' to daemon mds.g 2024-08-31T06:06:28.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.767+0000 7fb0e4245740 20 client.15204 populate_metadata read hostname 'smithi016' 2024-08-31T06:06:28.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.767+0000 7fb0e4245740 1 --2- 172.21.15.16:0/3010435692 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55af9d8b68e0 0x55af9dcbba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:28.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.767+0000 7fb0e4245740 4 client.15204 mds_command: new command op to 24724 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:06:28.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.767+0000 7fb0e4245740 1 -- 172.21.15.16:0/3010435692 --> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af9dc164a0 con 0x55af9d8b68e0 2024-08-31T06:06:28.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.767+0000 7fb0d7400640 1 --2- 172.21.15.16:0/3010435692 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55af9d8b68e0 0x55af9dcbba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:28.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.767+0000 7fb0d7400640 1 --2- 172.21.15.16:0/3010435692 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55af9d8b68e0 0x55af9dcbba80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3778c814574d0fa0 server_cookie=e3866f828fefbebe in_seq=0 out_seq=0 2024-08-31T06:06:28.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.767+0000 7fb0b7e00640 10 client.15204 ms_handle_connect on v2:172.21.15.16:6838/219664460 2024-08-31T06:06:28.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.768+0000 7fb0b7e00640 1 -- 172.21.15.16:0/3010435692 <== mds.1 v2:172.21.15.16:6838/219664460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55af9dc164a0 con 0x55af9d8b68e0 2024-08-31T06:06:28.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.768+0000 7fb0b7e00640 10 client.15204 handle_command_reply: tid=0 2024-08-31T06:06:28.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.768+0000 7fb0e4245740 1 client.15204 shutdown 2024-08-31T06:06:28.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.768+0000 7fb0e4245740 1 -- 172.21.15.16:0/3010435692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af9d8d7280 msgr2=0x55af9dca9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:28.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.768+0000 7fb0e4245740 1 --2- 172.21.15.16:0/3010435692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af9d8d7280 0x55af9dca9770 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb0c402a930 tx=0x7fb0c4032e90 comp rx=0 tx=0).stop 2024-08-31T06:06:28.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.769+0000 7fb0e4245740 1 -- 172.21.15.16:0/3010435692 shutdown_connections 2024-08-31T06:06:28.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.769+0000 7fb0e4245740 1 --2- 172.21.15.16:0/3010435692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af9d8f7f60 0x55af9dcb55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:28.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.769+0000 7fb0e4245740 1 --2- 172.21.15.16:0/3010435692 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55af9d8b68e0 0x55af9dcbba80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.769+0000 7fb0e4245740 1 --2- 172.21.15.16:0/3010435692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af9d8d7280 0x55af9dca9770 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.769+0000 7fb0b7e00640 0 client.15204 ms_handle_reset on v2:172.21.15.16:6838/219664460 2024-08-31T06:06:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.769+0000 7fb0e4245740 1 --2- 172.21.15.16:0/3010435692 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af9d8cf0a0 0x55af9dca9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.769+0000 7fb0e4245740 1 -- 172.21.15.16:0/3010435692 >> 172.21.15.16:0/3010435692 conn(0x55af9d8ec350 msgr2=0x55af9dca0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.769+0000 7fb0e4245740 1 -- 172.21.15.16:0/3010435692 shutdown_connections 2024-08-31T06:06:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.769+0000 7fb0e4245740 1 -- 172.21.15.16:0/3010435692 wait complete. 2024-08-31T06:06:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.769+0000 7fb0e4245740 20 client.15204 trim_cache size 0 max 16384 2024-08-31T06:06:28.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.788+0000 7fb0e4245740 1 Processor -- start 2024-08-31T06:06:28.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.788+0000 7fb0e4245740 1 -- start start 2024-08-31T06:06:28.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.788+0000 7fb0e4245740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af9d8cf0a0 0x55af9dcaae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:28.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.789+0000 7fb0e4245740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af9d8d7280 0x55af9dcb5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:28.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.789+0000 7fb0e4245740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af9d8f7f60 0x55af9dca6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:28.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.789+0000 7fb0e4245740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55af9dcb9dc0 con 0x55af9d8f7f60 2024-08-31T06:06:28.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.789+0000 7fb0e4245740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55af9dcba490 con 0x55af9d8d7280 2024-08-31T06:06:28.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.789+0000 7fb0e4245740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55af9dcbab60 con 0x55af9d8cf0a0 2024-08-31T06:06:28.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.789+0000 7fb0d7400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af9d8cf0a0 0x55af9dcaae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:28.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.789+0000 7fb0d7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af9d8f7f60 0x55af9dca6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:28.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.789+0000 7fb0d6000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af9d8d7280 0x55af9dcb5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:28.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.789+0000 7fb0d7400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af9d8cf0a0 0x55af9dcaae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:45018/0 (socket says 172.21.15.16:45018) 2024-08-31T06:06:28.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.789+0000 7fb0d7400640 1 -- 172.21.15.16:0/3579872391 learned_addr learned my addr 172.21.15.16:0/3579872391 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:28.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.789+0000 7fb0d6000640 1 -- 172.21.15.16:0/3579872391 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af9d8cf0a0 msgr2=0x55af9dcaae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:28.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.789+0000 7fb0d6000640 1 --2- 172.21.15.16:0/3579872391 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af9d8cf0a0 0x55af9dcaae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:28.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.789+0000 7fb0d6000640 1 -- 172.21.15.16:0/3579872391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af9d8f7f60 msgr2=0x55af9dca6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:28.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.789+0000 7fb0d6000640 1 --2- 172.21.15.16:0/3579872391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af9d8f7f60 0x55af9dca6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:28.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.789+0000 7fb0d6000640 1 -- 172.21.15.16:0/3579872391 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af9dc9ffc0 con 0x55af9d8d7280 2024-08-31T06:06:28.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.790+0000 7fb0d6000640 1 --2- 172.21.15.16:0/3579872391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af9d8d7280 0x55af9dcb5560 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4006010 tx=0x7fb0c40043f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60e667bbe9b09a56 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:28.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.790+0000 7fb0c2a00640 1 -- 172.21.15.16:0/3579872391 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c4035030 con 0x55af9d8d7280 2024-08-31T06:06:28.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.790+0000 7fb0c2a00640 1 -- 172.21.15.16:0/3579872391 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0c40048c0 con 0x55af9d8d7280 2024-08-31T06:06:28.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.790+0000 7fb0c2a00640 1 -- 172.21.15.16:0/3579872391 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c4003b10 con 0x55af9d8d7280 2024-08-31T06:06:28.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.791+0000 7fb0e4245740 1 -- 172.21.15.16:0/3579872391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af9d8d7280 msgr2=0x55af9dcb5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:28.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.791+0000 7fb0e4245740 1 --2- 172.21.15.16:0/3579872391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af9d8d7280 0x55af9dcb5560 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4006010 tx=0x7fb0c40043f0 comp rx=0 tx=0).stop 2024-08-31T06:06:28.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.791+0000 7fb0e4245740 1 -- 172.21.15.16:0/3579872391 shutdown_connections 2024-08-31T06:06:28.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.791+0000 7fb0e4245740 1 --2- 172.21.15.16:0/3579872391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af9d8f7f60 0x55af9dca6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:28.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.791+0000 7fb0e4245740 1 --2- 172.21.15.16:0/3579872391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af9d8d7280 0x55af9dcb5560 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:28.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.791+0000 7fb0e4245740 1 --2- 172.21.15.16:0/3579872391 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af9d8cf0a0 0x55af9dcaae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:28.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.791+0000 7fb0e4245740 1 -- 172.21.15.16:0/3579872391 >> 172.21.15.16:0/3579872391 conn(0x55af9d8ec350 msgr2=0x55af9dcaa750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:28.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.791+0000 7fb0e4245740 1 -- 172.21.15.16:0/3579872391 shutdown_connections 2024-08-31T06:06:28.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.791+0000 7fb0e4245740 1 -- 172.21.15.16:0/3579872391 wait complete. 2024-08-31T06:06:28.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.792+0000 7fb0e4245740 1 -- start start 2024-08-31T06:06:28.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.792+0000 7fb0e4245740 1 Processor -- start 2024-08-31T06:06:28.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.792+0000 7fb0e4245740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af9d8cf0a0 0x55af9dcbe460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.792+0000 7fb0e4245740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af9d8d7280 0x55af9dcbe9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.792+0000 7fb0e4245740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af9d8f7f60 0x55af9dcc39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.792+0000 7fb0e4245740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55af9dcbb900 con 0x55af9d8d7280 2024-08-31T06:06:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.792+0000 7fb0e4245740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55af9dcbb230 con 0x55af9d8f7f60 2024-08-31T06:06:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.792+0000 7fb0e4245740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55af9dcbbfd0 con 0x55af9d8cf0a0 2024-08-31T06:06:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.792+0000 7fb0e4245740 10 client.0 _ll_register_callbacks cb 0x7fb0c4032b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:06:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.793+0000 7fb0d7400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af9d8cf0a0 0x55af9dcbe460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.793+0000 7fb0d7400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af9d8cf0a0 0x55af9dcbe460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:45028/0 (socket says 172.21.15.16:45028) 2024-08-31T06:06:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.793+0000 7fb0d7400640 1 -- 172.21.15.16:0/2853949539 learned_addr learned my addr 172.21.15.16:0/2853949539 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.793+0000 7fb0d6000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af9d8d7280 0x55af9dcbe9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.793+0000 7fb0d7400640 1 --2- 172.21.15.16:0/2853949539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af9d8f7f60 0x55af9dcc39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:28.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.793+0000 7fb0d7400640 1 -- 172.21.15.16:0/2853949539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af9d8f7f60 msgr2=0x55af9dcc39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:28.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.793+0000 7fb0d7400640 1 --2- 172.21.15.16:0/2853949539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af9d8f7f60 0x55af9dcc39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:28.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.793+0000 7fb0d7400640 1 -- 172.21.15.16:0/2853949539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af9d8d7280 msgr2=0x55af9dcbe9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:28.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.793+0000 7fb0d7400640 1 --2- 172.21.15.16:0/2853949539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af9d8d7280 0x55af9dcbe9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:28.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.793+0000 7fb0d7400640 1 -- 172.21.15.16:0/2853949539 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af9dcc3f60 con 0x55af9d8cf0a0 2024-08-31T06:06:28.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.793+0000 7fb0d6000640 1 --2- 172.21.15.16:0/2853949539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af9d8d7280 0x55af9dcbe9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:06:28.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.794+0000 7fb0d7400640 1 --2- 172.21.15.16:0/2853949539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af9d8cf0a0 0x55af9dcbe460 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0013010 tx=0x7fb0d0013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7673efa511267671 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:28.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.794+0000 7fb0b7e00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:06:28.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.794+0000 7fb0b7e00640 1 -- 172.21.15.16:0/2853949539 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d0004230 con 0x55af9d8cf0a0 2024-08-31T06:06:28.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.794+0000 7fb0e4245740 1 -- 172.21.15.16:0/2853949539 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0c40048c0 con 0x55af9d8cf0a0 2024-08-31T06:06:28.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.794+0000 7fb0b7e00640 1 -- 172.21.15.16:0/2853949539 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7fb0d001f040 con 0x55af9d8cf0a0 2024-08-31T06:06:28.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.794+0000 7fb0b7e00640 1 -- 172.21.15.16:0/2853949539 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb0d00043d0 con 0x55af9d8cf0a0 2024-08-31T06:06:28.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.794+0000 7fb0e4245740 10 client.25222 fetch_fsmap learned FSMap version 24 2024-08-31T06:06:28.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.794+0000 7fb0e4245740 1 -- 172.21.15.16:0/2853949539 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55af9dd27730 con 0x55af9d8cf0a0 2024-08-31T06:06:28.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.794+0000 7fb0b7e00640 1 -- 172.21.15.16:0/2853949539 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0c40048c0 con 0x55af9d8cf0a0 2024-08-31T06:06:28.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.795+0000 7fb0b7e00640 1 -- 172.21.15.16:0/2853949539 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fb0d0028450 con 0x55af9d8cf0a0 2024-08-31T06:06:28.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.795+0000 7fb0e4245740 10 client.25222 fetch_fsmap finished waiting for FSMap version 24 2024-08-31T06:06:28.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.795+0000 7fb0e4245740 10 client.25222 resolve_mds: resolved name 'g' to daemon mds.g 2024-08-31T06:06:28.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.795+0000 7fb0e4245740 20 client.25222 populate_metadata read hostname 'smithi016' 2024-08-31T06:06:28.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.795+0000 7fb0e4245740 1 --2- 172.21.15.16:0/2853949539 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55af9d88bc90 0x55af9dd2ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:28.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.795+0000 7fb0e4245740 4 client.25222 mds_command: new command op to 24724 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-31T06:06:28.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.795+0000 7fb0e4245740 1 -- 172.21.15.16:0/2853949539 --> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] -- command(tid 0: {"prefix": "status"}) -- 0x55af9dc11f40 con 0x55af9d88bc90 2024-08-31T06:06:28.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.795+0000 7fb0d6000640 1 --2- 172.21.15.16:0/2853949539 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55af9d88bc90 0x55af9dd2ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:28.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.795+0000 7fb0d6000640 1 --2- 172.21.15.16:0/2853949539 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55af9d88bc90 0x55af9dd2ad10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f4046f86ce5e1e76 server_cookie=16073b609179e3a4 in_seq=0 out_seq=0 2024-08-31T06:06:28.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.795+0000 7fb0b7e00640 10 client.25222 ms_handle_connect on v2:172.21.15.16:6838/219664460 2024-08-31T06:06:28.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.796+0000 7fb0b7e00640 1 -- 172.21.15.16:0/2853949539 <== mds.1 v2:172.21.15.16:6838/219664460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55af9dc11f40 con 0x55af9d88bc90 2024-08-31T06:06:28.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.796+0000 7fb0b7e00640 10 client.25222 handle_command_reply: tid=0 2024-08-31T06:06:28.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.796+0000 7fb0e4245740 1 client.25222 shutdown 2024-08-31T06:06:28.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.796+0000 7fb0e4245740 1 -- 172.21.15.16:0/2853949539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af9d8cf0a0 msgr2=0x55af9dcbe460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:28.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.796+0000 7fb0e4245740 1 --2- 172.21.15.16:0/2853949539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af9d8cf0a0 0x55af9dcbe460 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0013010 tx=0x7fb0d0013040 comp rx=0 tx=0).stop 2024-08-31T06:06:28.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.796+0000 7fb0e4245740 1 -- 172.21.15.16:0/2853949539 shutdown_connections 2024-08-31T06:06:28.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.796+0000 7fb0e4245740 1 --2- 172.21.15.16:0/2853949539 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55af9d88bc90 0x55af9dd2ad10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:28.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.796+0000 7fb0e4245740 1 --2- 172.21.15.16:0/2853949539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af9d8f7f60 0x55af9dcc39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:28.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.796+0000 7fb0b7e00640 0 client.25222 ms_handle_reset on v2:172.21.15.16:6838/219664460 2024-08-31T06:06:28.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.796+0000 7fb0e4245740 1 --2- 172.21.15.16:0/2853949539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af9d8d7280 0x55af9dcbe9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:28.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.796+0000 7fb0e4245740 1 --2- 172.21.15.16:0/2853949539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af9d8cf0a0 0x55af9dcbe460 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:28.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.796+0000 7fb0e4245740 1 -- 172.21.15.16:0/2853949539 >> 172.21.15.16:0/2853949539 conn(0x55af9d8ec350 msgr2=0x55af9dcaa670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:28.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.796+0000 7fb0e4245740 1 -- 172.21.15.16:0/2853949539 shutdown_connections 2024-08-31T06:06:28.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.796+0000 7fb0e4245740 1 -- 172.21.15.16:0/2853949539 wait complete. 2024-08-31T06:06:28.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.796+0000 7fb0e4245740 20 client.25222 trim_cache size 0 max 16384 2024-08-31T06:06:28.803 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:06:28.803 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "673e4aa0-675d-11ef-bcd4-c7b262605968", 2024-08-31T06:06:28.803 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 1, 2024-08-31T06:06:28.803 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24724, 2024-08-31T06:06:28.803 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-08-31T06:06:28.803 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-08-31T06:06:28.803 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-08-31T06:06:28.804 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 110.87272224500001, 2024-08-31T06:06:28.804 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 24, 2024-08-31T06:06:28.804 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 80, 2024-08-31T06:06:28.804 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 78, 2024-08-31T06:06:28.804 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 141.80217006999999 2024-08-31T06:06:28.804 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:06:28.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.797+0000 7fb0d7e00640 1 -- 172.21.15.16:0/1118765627 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb0bc0b4c00 msgr2=0x7fb0bc0b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:28.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.797+0000 7fb0d7e00640 1 --2- 172.21.15.16:0/1118765627 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb0bc0b4c00 0x7fb0bc0b7110 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb0c80abc80 tx=0x7fb0c80eb110 comp rx=0 tx=0).stop 2024-08-31T06:06:28.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.797+0000 7fb0d7e00640 1 -- 172.21.15.16:0/1118765627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0d8105960 msgr2=0x7fb0d819c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:28.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.797+0000 7fb0d7e00640 1 --2- 172.21.15.16:0/1118765627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0d8105960 0x7fb0d819c280 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb0c800e9f0 tx=0x7fb0c800edf0 comp rx=0 tx=0).stop 2024-08-31T06:06:28.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.797+0000 7fb0d7e00640 1 -- 172.21.15.16:0/1118765627 shutdown_connections 2024-08-31T06:06:28.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.797+0000 7fb0d7e00640 1 --2- 172.21.15.16:0/1118765627 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb0bc0b4c00 0x7fb0bc0b7110 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:28.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.797+0000 7fb0d7e00640 1 --2- 172.21.15.16:0/1118765627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0d810b2a0 0x7fb0d819c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:28.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.797+0000 7fb0d7e00640 1 --2- 172.21.15.16:0/1118765627 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0d810bbf0 0x7fb0d81a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:28.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.797+0000 7fb0d7e00640 1 --2- 172.21.15.16:0/1118765627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0d8105960 0x7fb0d819c280 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:28.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.797+0000 7fb0d7e00640 1 -- 172.21.15.16:0/1118765627 >> 172.21.15.16:0/1118765627 conn(0x7fb0d8101410 msgr2=0x7fb0d8103830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:28.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.798+0000 7fb0d7e00640 1 -- 172.21.15.16:0/1118765627 shutdown_connections 2024-08-31T06:06:28.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:28.798+0000 7fb0d7e00640 1 -- 172.21.15.16:0/1118765627 wait complete. 2024-08-31T06:06:28.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:28 smithi155 ceph-mon[21742]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-08-31T06:06:29.665 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.b status 2024-08-31T06:06:30.039 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:06:30.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.689+0000 7ff49f200640 1 -- 172.21.15.16:0/4038885754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4a010fcb0 msgr2=0x7ff4a01120f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:30.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.689+0000 7ff49f200640 1 --2- 172.21.15.16:0/4038885754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4a010fcb0 0x7ff4a01120f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff498009ca0 tx=0x7ff49802a8c0 comp rx=0 tx=0).stop 2024-08-31T06:06:30.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.689+0000 7ff49f200640 1 -- 172.21.15.16:0/4038885754 shutdown_connections 2024-08-31T06:06:30.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.689+0000 7ff49f200640 1 --2- 172.21.15.16:0/4038885754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4a010fcb0 0x7ff4a01120f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:30.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.689+0000 7ff49f200640 1 --2- 172.21.15.16:0/4038885754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4a0075ea0 0x7ff4a010f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:30.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.689+0000 7ff49f200640 1 --2- 172.21.15.16:0/4038885754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4a0075550 0x7ff4a0075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:30.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.689+0000 7ff49f200640 1 -- 172.21.15.16:0/4038885754 >> 172.21.15.16:0/4038885754 conn(0x7ff4a00ff1e0 msgr2=0x7ff4a0101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:30.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.689+0000 7ff49f200640 1 -- 172.21.15.16:0/4038885754 shutdown_connections 2024-08-31T06:06:30.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.689+0000 7ff49f200640 1 -- 172.21.15.16:0/4038885754 wait complete. 2024-08-31T06:06:30.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.690+0000 7ff49f200640 1 Processor -- start 2024-08-31T06:06:30.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.690+0000 7ff49f200640 1 -- start start 2024-08-31T06:06:30.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.691+0000 7ff49f200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4a0075550 0x7ff4a019c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:30.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.691+0000 7ff49f200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4a0075ea0 0x7ff4a019c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:30.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.691+0000 7ff49f200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4a010fcb0 0x7ff4a01a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:30.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.691+0000 7ff49f200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a0115820 con 0x7ff4a010fcb0 2024-08-31T06:06:30.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.691+0000 7ff49f200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a01156a0 con 0x7ff4a0075550 2024-08-31T06:06:30.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.691+0000 7ff49f200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a01159a0 con 0x7ff4a0075ea0 2024-08-31T06:06:30.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.691+0000 7ff49de00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4a0075550 0x7ff4a019c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:30.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.691+0000 7ff49de00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4a0075550 0x7ff4a019c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:50668/0 (socket says 172.21.15.16:50668) 2024-08-31T06:06:30.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.691+0000 7ff49de00640 1 -- 172.21.15.16:0/1431405295 learned_addr learned my addr 172.21.15.16:0/1431405295 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:30.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.691+0000 7ff49e800640 1 --2- 172.21.15.16:0/1431405295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4a010fcb0 0x7ff4a01a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:30.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.691+0000 7ff49d400640 1 --2- 172.21.15.16:0/1431405295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4a0075ea0 0x7ff4a019c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:30.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.692+0000 7ff49de00640 1 -- 172.21.15.16:0/1431405295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4a0075ea0 msgr2=0x7ff4a019c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:30.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.692+0000 7ff49de00640 1 --2- 172.21.15.16:0/1431405295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4a0075ea0 0x7ff4a019c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:30.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.692+0000 7ff49de00640 1 -- 172.21.15.16:0/1431405295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4a010fcb0 msgr2=0x7ff4a01a1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:30.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.692+0000 7ff49de00640 1 --2- 172.21.15.16:0/1431405295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4a010fcb0 0x7ff4a01a1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:30.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.692+0000 7ff49de00640 1 -- 172.21.15.16:0/1431405295 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a01a21e0 con 0x7ff4a0075550 2024-08-31T06:06:30.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.692+0000 7ff49de00640 1 --2- 172.21.15.16:0/1431405295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4a0075550 0x7ff4a019c270 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff49000e9f0 tx=0x7ff49000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:30.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.692+0000 7ff496a00640 1 -- 172.21.15.16:0/1431405295 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49000cd80 con 0x7ff4a0075550 2024-08-31T06:06:30.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.692+0000 7ff49f200640 1 -- 172.21.15.16:0/1431405295 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4a01a24d0 con 0x7ff4a0075550 2024-08-31T06:06:30.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.692+0000 7ff496a00640 1 -- 172.21.15.16:0/1431405295 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4900108e0 con 0x7ff4a0075550 2024-08-31T06:06:30.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.692+0000 7ff496a00640 1 -- 172.21.15.16:0/1431405295 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff490018980 con 0x7ff4a0075550 2024-08-31T06:06:30.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.692+0000 7ff49f200640 1 -- 172.21.15.16:0/1431405295 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4a01a8c30 con 0x7ff4a0075550 2024-08-31T06:06:30.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.696+0000 7ff4abe41740 1 Processor -- start 2024-08-31T06:06:30.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.696+0000 7ff496a00640 1 -- 172.21.15.16:0/1431405295 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7ff490010430 con 0x7ff4a0075550 2024-08-31T06:06:30.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.696+0000 7ff4abe41740 1 -- start start 2024-08-31T06:06:30.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.697+0000 7ff4abe41740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a63e71f60 0x556a64217aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:30.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.697+0000 7ff4abe41740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a63e490a0 0x556a64217fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:30.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.697+0000 7ff4abe41740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a63e51280 0x556a64218520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:30.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.697+0000 7ff4abe41740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556a6418e550 con 0x556a63e51280 2024-08-31T06:06:30.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.697+0000 7ff4abe41740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556a6417dae0 con 0x556a63e490a0 2024-08-31T06:06:30.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.697+0000 7ff4abe41740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556a64181a90 con 0x556a63e71f60 2024-08-31T06:06:30.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.697+0000 7ff49d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a63e51280 0x556a64218520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:30.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.697+0000 7ff49de00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a63e490a0 0x556a64217fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:30.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.697+0000 7ff49d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a63e51280 0x556a64218520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45346/0 (socket says 172.21.15.16:45346) 2024-08-31T06:06:30.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.697+0000 7ff49d400640 1 -- 172.21.15.16:0/2405851645 learned_addr learned my addr 172.21.15.16:0/2405851645 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:30.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.697+0000 7ff49e800640 1 --2- 172.21.15.16:0/2405851645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a63e71f60 0x556a64217aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:30.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.698+0000 7ff49d400640 1 -- 172.21.15.16:0/2405851645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a63e71f60 msgr2=0x556a64217aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:30.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.698+0000 7ff49d400640 1 --2- 172.21.15.16:0/2405851645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a63e71f60 0x556a64217aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:30.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.698+0000 7ff49d400640 1 -- 172.21.15.16:0/2405851645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a63e490a0 msgr2=0x556a64217fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:30.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.698+0000 7ff49d400640 1 --2- 172.21.15.16:0/2405851645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a63e490a0 0x556a64217fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:30.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.698+0000 7ff49d400640 1 -- 172.21.15.16:0/2405851645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a63edd540 con 0x556a63e51280 2024-08-31T06:06:30.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.698+0000 7ff49d400640 1 --2- 172.21.15.16:0/2405851645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a63e51280 0x556a64218520 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff48c009c20 tx=0x7ff48c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b9069fc5dfb366a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:30.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.698+0000 7ff496a00640 1 --2- 172.21.15.16:0/1431405295 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff4840b4b80 0x7ff4840b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:30.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.698+0000 7ff48b400640 1 -- 172.21.15.16:0/2405851645 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48c009dd0 con 0x556a63e51280 2024-08-31T06:06:30.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.699+0000 7ff48b400640 1 -- 172.21.15.16:0/2405851645 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff48c005020 con 0x556a63e51280 2024-08-31T06:06:30.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.699+0000 7ff48b400640 1 -- 172.21.15.16:0/2405851645 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48c004a00 con 0x556a63e51280 2024-08-31T06:06:30.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.699+0000 7ff496a00640 1 -- 172.21.15.16:0/1431405295 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7ff4900e5560 con 0x7ff4a0075550 2024-08-31T06:06:30.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.699+0000 7ff49e800640 1 --2- 172.21.15.16:0/1431405295 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff4840b4b80 0x7ff4840b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:30.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.700+0000 7ff4abe41740 1 -- 172.21.15.16:0/2405851645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a63e51280 msgr2=0x556a64218520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:30.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.700+0000 7ff49e800640 1 --2- 172.21.15.16:0/1431405295 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff4840b4b80 0x7ff4840b7090 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff49802f500 tx=0x7ff49802f310 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:30.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.700+0000 7ff4abe41740 1 --2- 172.21.15.16:0/2405851645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a63e51280 0x556a64218520 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff48c009c20 tx=0x7ff48c011810 comp rx=0 tx=0).stop 2024-08-31T06:06:30.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.700+0000 7ff4abe41740 1 -- 172.21.15.16:0/2405851645 shutdown_connections 2024-08-31T06:06:30.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.700+0000 7ff4abe41740 1 --2- 172.21.15.16:0/2405851645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a63e51280 0x556a64218520 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:30.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.700+0000 7ff4abe41740 1 --2- 172.21.15.16:0/2405851645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a63e490a0 0x556a64217fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:30.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.700+0000 7ff4abe41740 1 --2- 172.21.15.16:0/2405851645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a63e71f60 0x556a64217aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:30.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.700+0000 7ff4abe41740 1 -- 172.21.15.16:0/2405851645 >> 172.21.15.16:0/2405851645 conn(0x556a63e66350 msgr2=0x556a64215990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:30.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.700+0000 7ff4abe41740 1 -- 172.21.15.16:0/2405851645 shutdown_connections 2024-08-31T06:06:30.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.700+0000 7ff4abe41740 1 -- 172.21.15.16:0/2405851645 wait complete. 2024-08-31T06:06:30.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.700+0000 7ff4abe41740 1 -- start start 2024-08-31T06:06:30.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.701+0000 7ff4abe41740 1 Processor -- start 2024-08-31T06:06:30.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.701+0000 7ff4abe41740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a63e490a0 0x556a642231e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:30.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.701+0000 7ff4abe41740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a63e51280 0x556a64223720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:30.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.701+0000 7ff4abe41740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a63e71f60 0x556a6422f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:30.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.701+0000 7ff4abe41740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556a63fe05f0 con 0x556a63e51280 2024-08-31T06:06:30.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.701+0000 7ff4abe41740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556a64104bd0 con 0x556a63e490a0 2024-08-31T06:06:30.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.701+0000 7ff4abe41740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556a64226280 con 0x556a63e71f60 2024-08-31T06:06:30.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.701+0000 7ff4abe41740 10 client.0 _ll_register_callbacks cb 0x556a6412a210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:06:30.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.702+0000 7ff49d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a63e51280 0x556a64223720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:30.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.702+0000 7ff49d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a63e51280 0x556a64223720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45354/0 (socket says 172.21.15.16:45354) 2024-08-31T06:06:30.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.702+0000 7ff49de00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a63e490a0 0x556a642231e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:30.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.702+0000 7ff49d400640 1 -- 172.21.15.16:0/3999613255 learned_addr learned my addr 172.21.15.16:0/3999613255 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:30.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.702+0000 7ff49e800640 1 --2- 172.21.15.16:0/3999613255 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a63e71f60 0x556a6422f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:30.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.702+0000 7ff49de00640 1 -- 172.21.15.16:0/3999613255 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a63e71f60 msgr2=0x556a6422f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:30.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.702+0000 7ff49de00640 1 --2- 172.21.15.16:0/3999613255 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a63e71f60 0x556a6422f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:30.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.702+0000 7ff49de00640 1 -- 172.21.15.16:0/3999613255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a63e51280 msgr2=0x556a64223720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:30.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.702+0000 7ff49de00640 1 --2- 172.21.15.16:0/3999613255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a63e51280 0x556a64223720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:30.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.702+0000 7ff49de00640 1 -- 172.21.15.16:0/3999613255 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a64145650 con 0x556a63e490a0 2024-08-31T06:06:30.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.702+0000 7ff49de00640 1 --2- 172.21.15.16:0/3999613255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a63e490a0 0x556a642231e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff4900e2ff0 tx=0x7ff4900e8c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8b8d5071a738767 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:30.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.702+0000 7ff47fe00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:06:30.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.703+0000 7ff47fe00640 1 -- 172.21.15.16:0/3999613255 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4900ab6f0 con 0x556a63e490a0 2024-08-31T06:06:30.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.703+0000 7ff4abe41740 1 -- 172.21.15.16:0/3999613255 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff48c005020 con 0x556a63e490a0 2024-08-31T06:06:30.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.703+0000 7ff47fe00640 1 -- 172.21.15.16:0/3999613255 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7ff4900ab3f0 con 0x556a63e490a0 2024-08-31T06:06:30.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.703+0000 7ff47fe00640 1 -- 172.21.15.16:0/3999613255 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff490010720 con 0x556a63e490a0 2024-08-31T06:06:30.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.703+0000 7ff4abe41740 10 client.25226 fetch_fsmap learned FSMap version 24 2024-08-31T06:06:30.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.703+0000 7ff4abe41740 1 -- 172.21.15.16:0/3999613255 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x556a642320f0 con 0x556a63e490a0 2024-08-31T06:06:30.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.703+0000 7ff47fe00640 1 -- 172.21.15.16:0/3999613255 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff48c005020 con 0x556a63e490a0 2024-08-31T06:06:30.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.703+0000 7ff47fe00640 1 -- 172.21.15.16:0/3999613255 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7ff4900b0380 con 0x556a63e490a0 2024-08-31T06:06:30.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.704+0000 7ff4abe41740 10 client.25226 fetch_fsmap finished waiting for FSMap version 24 2024-08-31T06:06:30.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.704+0000 7ff4abe41740 10 client.25226 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-31T06:06:30.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.704+0000 7ff4abe41740 20 client.25226 populate_metadata read hostname 'smithi016' 2024-08-31T06:06:30.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.704+0000 7ff4abe41740 1 --2- 172.21.15.16:0/3999613255 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556a63e308e0 0x556a64235aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:30.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.704+0000 7ff4abe41740 4 client.25226 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:06:30.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.704+0000 7ff4abe41740 1 -- 172.21.15.16:0/3999613255 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a641904a0 con 0x556a63e308e0 2024-08-31T06:06:30.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.704+0000 7ff49d400640 1 --2- 172.21.15.16:0/3999613255 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556a63e308e0 0x556a64235aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:30.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.705+0000 7ff49d400640 1 --2- 172.21.15.16:0/3999613255 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556a63e308e0 0x556a64235aa0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=310b50b7f79432cb server_cookie=2fe70bc693d0af50 in_seq=0 out_seq=0 2024-08-31T06:06:30.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.705+0000 7ff47fe00640 10 client.25226 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:06:30.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.706+0000 7ff47fe00640 1 -- 172.21.15.16:0/3999613255 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556a641904a0 con 0x556a63e308e0 2024-08-31T06:06:30.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.706+0000 7ff47fe00640 10 client.25226 handle_command_reply: tid=0 2024-08-31T06:06:30.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.706+0000 7ff4abe41740 1 client.25226 shutdown 2024-08-31T06:06:30.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.706+0000 7ff4abe41740 1 -- 172.21.15.16:0/3999613255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a63e490a0 msgr2=0x556a642231e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:30.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.706+0000 7ff4abe41740 1 --2- 172.21.15.16:0/3999613255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a63e490a0 0x556a642231e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff4900e2ff0 tx=0x7ff4900e8c30 comp rx=0 tx=0).stop 2024-08-31T06:06:30.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.706+0000 7ff4abe41740 1 -- 172.21.15.16:0/3999613255 shutdown_connections 2024-08-31T06:06:30.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.706+0000 7ff4abe41740 1 --2- 172.21.15.16:0/3999613255 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556a63e308e0 0x556a64235aa0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:30.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.706+0000 7ff4abe41740 1 --2- 172.21.15.16:0/3999613255 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a63e71f60 0x556a6422f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:30.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.706+0000 7ff4abe41740 1 --2- 172.21.15.16:0/3999613255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a63e51280 0x556a64223720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:30.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.706+0000 7ff4abe41740 1 --2- 172.21.15.16:0/3999613255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a63e490a0 0x556a642231e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:30.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.706+0000 7ff4abe41740 1 -- 172.21.15.16:0/3999613255 >> 172.21.15.16:0/3999613255 conn(0x556a63e66350 msgr2=0x556a6421ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:30.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.706+0000 7ff47fe00640 0 client.25226 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:06:30.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.706+0000 7ff4abe41740 1 -- 172.21.15.16:0/3999613255 shutdown_connections 2024-08-31T06:06:30.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.706+0000 7ff4abe41740 1 -- 172.21.15.16:0/3999613255 wait complete. 2024-08-31T06:06:30.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.706+0000 7ff4abe41740 20 client.25226 trim_cache size 0 max 16384 2024-08-31T06:06:30.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:30 smithi184 ceph-mon[22331]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-08-31T06:06:30.722 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:30 smithi016 ceph-mon[17194]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-08-31T06:06:30.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.728+0000 7ff4abe41740 1 Processor -- start 2024-08-31T06:06:30.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.728+0000 7ff4abe41740 1 -- start start 2024-08-31T06:06:30.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.729+0000 7ff4abe41740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a63e490a0 0x556a64234630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:30.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.729+0000 7ff4abe41740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a63e51280 0x556a64233560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:30.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.729+0000 7ff4abe41740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a63e71f60 0x556a6422f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:30.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.729+0000 7ff4abe41740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556a64233de0 con 0x556a63e71f60 2024-08-31T06:06:30.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.729+0000 7ff4abe41740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556a642344b0 con 0x556a63e490a0 2024-08-31T06:06:30.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.729+0000 7ff4abe41740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556a64234b80 con 0x556a63e51280 2024-08-31T06:06:30.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.729+0000 7ff49d400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a63e51280 0x556a64233560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:30.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.729+0000 7ff49d400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a63e51280 0x556a64233560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:45058/0 (socket says 172.21.15.16:45058) 2024-08-31T06:06:30.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.729+0000 7ff49d400640 1 -- 172.21.15.16:0/2528402800 learned_addr learned my addr 172.21.15.16:0/2528402800 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:30.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.729+0000 7ff49de00640 1 --2- 172.21.15.16:0/2528402800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a63e490a0 0x556a64234630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:30.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.729+0000 7ff49e800640 1 --2- 172.21.15.16:0/2528402800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a63e71f60 0x556a6422f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:30.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.730+0000 7ff49d400640 1 -- 172.21.15.16:0/2528402800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a63e490a0 msgr2=0x556a64234630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:30.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.730+0000 7ff49d400640 1 --2- 172.21.15.16:0/2528402800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a63e490a0 0x556a64234630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:30.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.730+0000 7ff49d400640 1 -- 172.21.15.16:0/2528402800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a63e71f60 msgr2=0x556a6422f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:30.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.730+0000 7ff49d400640 1 --2- 172.21.15.16:0/2528402800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a63e71f60 0x556a6422f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:30.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.730+0000 7ff49d400640 1 -- 172.21.15.16:0/2528402800 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a64216300 con 0x556a63e51280 2024-08-31T06:06:30.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.730+0000 7ff49de00640 1 --2- 172.21.15.16:0/2528402800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a63e490a0 0x556a64234630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:06:30.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.730+0000 7ff49e800640 1 --2- 172.21.15.16:0/2528402800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a63e71f60 0x556a6422f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:06:30.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.730+0000 7ff49d400640 1 --2- 172.21.15.16:0/2528402800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a63e51280 0x556a64233560 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff48c000c00 tx=0x7ff48c0045b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dea7b4852dddf3f8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:30.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.731+0000 7ff48aa00640 1 -- 172.21.15.16:0/2528402800 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48c032070 con 0x556a63e51280 2024-08-31T06:06:30.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.731+0000 7ff48aa00640 1 -- 172.21.15.16:0/2528402800 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff48c026440 con 0x556a63e51280 2024-08-31T06:06:30.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.731+0000 7ff48aa00640 1 -- 172.21.15.16:0/2528402800 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48c01d040 con 0x556a63e51280 2024-08-31T06:06:30.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.732+0000 7ff4abe41740 1 -- 172.21.15.16:0/2528402800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a63e51280 msgr2=0x556a64233560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:30.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.732+0000 7ff4abe41740 1 --2- 172.21.15.16:0/2528402800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a63e51280 0x556a64233560 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff48c000c00 tx=0x7ff48c0045b0 comp rx=0 tx=0).stop 2024-08-31T06:06:30.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.732+0000 7ff4abe41740 1 -- 172.21.15.16:0/2528402800 shutdown_connections 2024-08-31T06:06:30.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.732+0000 7ff4abe41740 1 --2- 172.21.15.16:0/2528402800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a63e51280 0x556a64233560 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:30.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.732+0000 7ff4abe41740 1 --2- 172.21.15.16:0/2528402800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a63e71f60 0x556a6422f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:30.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.732+0000 7ff4abe41740 1 --2- 172.21.15.16:0/2528402800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a63e490a0 0x556a64234630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:30.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.732+0000 7ff4abe41740 1 -- 172.21.15.16:0/2528402800 >> 172.21.15.16:0/2528402800 conn(0x556a63e66350 msgr2=0x556a64210710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:30.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.732+0000 7ff4abe41740 1 -- 172.21.15.16:0/2528402800 shutdown_connections 2024-08-31T06:06:30.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.732+0000 7ff4abe41740 1 -- 172.21.15.16:0/2528402800 wait complete. 2024-08-31T06:06:30.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.733+0000 7ff4abe41740 1 -- start start 2024-08-31T06:06:30.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.733+0000 7ff4abe41740 1 Processor -- start 2024-08-31T06:06:30.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.734+0000 7ff4abe41740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a63e490a0 0x556a642384a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:30.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.734+0000 7ff4abe41740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a63e51280 0x556a64238a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:30.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.734+0000 7ff4abe41740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a63e71f60 0x556a6423c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:30.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.734+0000 7ff4abe41740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556a64235920 con 0x556a63e51280 2024-08-31T06:06:30.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.734+0000 7ff4abe41740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556a64235250 con 0x556a63e71f60 2024-08-31T06:06:30.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.734+0000 7ff4abe41740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556a64235ff0 con 0x556a63e490a0 2024-08-31T06:06:30.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.734+0000 7ff4abe41740 10 client.0 _ll_register_callbacks cb 0x7ff4900e8d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:06:30.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.734+0000 7ff49d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a63e51280 0x556a64238a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:30.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.734+0000 7ff49d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a63e51280 0x556a64238a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45378/0 (socket says 172.21.15.16:45378) 2024-08-31T06:06:30.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.734+0000 7ff49d400640 1 -- 172.21.15.16:0/817485612 learned_addr learned my addr 172.21.15.16:0/817485612 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:30.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.734+0000 7ff49de00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a63e490a0 0x556a642384a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:30.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.734+0000 7ff49e800640 1 --2- 172.21.15.16:0/817485612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a63e71f60 0x556a6423c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:30.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.735+0000 7ff49de00640 1 -- 172.21.15.16:0/817485612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a63e71f60 msgr2=0x556a6423c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:30.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.735+0000 7ff49de00640 1 --2- 172.21.15.16:0/817485612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a63e71f60 0x556a6423c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:30.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.735+0000 7ff49de00640 1 -- 172.21.15.16:0/817485612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a63e51280 msgr2=0x556a64238a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:30.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.735+0000 7ff49de00640 1 --2- 172.21.15.16:0/817485612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a63e51280 0x556a64238a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:30.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.735+0000 7ff49de00640 1 -- 172.21.15.16:0/817485612 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a6423cb80 con 0x556a63e490a0 2024-08-31T06:06:30.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.735+0000 7ff49d400640 1 --2- 172.21.15.16:0/817485612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a63e51280 0x556a64238a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:06:30.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.735+0000 7ff49e800640 1 --2- 172.21.15.16:0/817485612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a63e71f60 0x556a6423c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:06:30.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.735+0000 7ff49de00640 1 --2- 172.21.15.16:0/817485612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a63e490a0 0x556a642384a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff4900a91f0 tx=0x7ff490000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23a7e4a1388a354 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:30.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.735+0000 7ff47fe00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:06:30.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.735+0000 7ff47fe00640 1 -- 172.21.15.16:0/817485612 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4900b51f0 con 0x556a63e490a0 2024-08-31T06:06:30.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.735+0000 7ff47fe00640 1 -- 172.21.15.16:0/817485612 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7ff4900e5060 con 0x556a63e490a0 2024-08-31T06:06:30.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.735+0000 7ff4abe41740 1 -- 172.21.15.16:0/817485612 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff48c026440 con 0x556a63e490a0 2024-08-31T06:06:30.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.736+0000 7ff47fe00640 1 -- 172.21.15.16:0/817485612 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff4900abf80 con 0x556a63e490a0 2024-08-31T06:06:30.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.736+0000 7ff4abe41740 10 client.25240 fetch_fsmap learned FSMap version 24 2024-08-31T06:06:30.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.736+0000 7ff4abe41740 1 -- 172.21.15.16:0/817485612 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x556a6423df40 con 0x556a63e490a0 2024-08-31T06:06:30.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.736+0000 7ff47fe00640 1 -- 172.21.15.16:0/817485612 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff48c026440 con 0x556a63e490a0 2024-08-31T06:06:30.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.736+0000 7ff47fe00640 1 -- 172.21.15.16:0/817485612 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7ff4900a7a80 con 0x556a63e490a0 2024-08-31T06:06:30.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.736+0000 7ff4abe41740 10 client.25240 fetch_fsmap finished waiting for FSMap version 24 2024-08-31T06:06:30.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.736+0000 7ff4abe41740 10 client.25240 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-31T06:06:30.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.736+0000 7ff4abe41740 20 client.25240 populate_metadata read hostname 'smithi016' 2024-08-31T06:06:30.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.736+0000 7ff4abe41740 1 --2- 172.21.15.16:0/817485612 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556a63e05c90 0x556a642a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:30.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.736+0000 7ff4abe41740 4 client.25240 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-31T06:06:30.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.736+0000 7ff4abe41740 1 -- 172.21.15.16:0/817485612 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "status"}) -- 0x556a6418bf40 con 0x556a63e05c90 2024-08-31T06:06:30.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.737+0000 7ff49d400640 1 --2- 172.21.15.16:0/817485612 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556a63e05c90 0x556a642a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:30.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.737+0000 7ff49d400640 1 --2- 172.21.15.16:0/817485612 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556a63e05c90 0x556a642a3a80 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14c29920149bfa32 server_cookie=36918bf240b046df in_seq=0 out_seq=0 2024-08-31T06:06:30.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.737+0000 7ff47fe00640 10 client.25240 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:06:30.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.737+0000 7ff47fe00640 1 -- 172.21.15.16:0/817485612 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x556a6418bf40 con 0x556a63e05c90 2024-08-31T06:06:30.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.737+0000 7ff47fe00640 10 client.25240 handle_command_reply: tid=0 2024-08-31T06:06:30.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.737+0000 7ff4abe41740 1 client.25240 shutdown 2024-08-31T06:06:30.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.738+0000 7ff4abe41740 1 -- 172.21.15.16:0/817485612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a63e490a0 msgr2=0x556a642384a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:30.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.738+0000 7ff4abe41740 1 --2- 172.21.15.16:0/817485612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a63e490a0 0x556a642384a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff4900a91f0 tx=0x7ff490000bc0 comp rx=0 tx=0).stop 2024-08-31T06:06:30.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.738+0000 7ff4abe41740 1 -- 172.21.15.16:0/817485612 shutdown_connections 2024-08-31T06:06:30.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.738+0000 7ff4abe41740 1 --2- 172.21.15.16:0/817485612 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556a63e05c90 0x556a642a3a80 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:30.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.738+0000 7ff4abe41740 1 --2- 172.21.15.16:0/817485612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a63e71f60 0x556a6423c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:30.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.738+0000 7ff4abe41740 1 --2- 172.21.15.16:0/817485612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a63e51280 0x556a64238a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:30.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.738+0000 7ff4abe41740 1 --2- 172.21.15.16:0/817485612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556a63e490a0 0x556a642384a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:30.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.738+0000 7ff4abe41740 1 -- 172.21.15.16:0/817485612 >> 172.21.15.16:0/817485612 conn(0x556a63e66350 msgr2=0x556a64233560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:30.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.738+0000 7ff47fe00640 0 client.25240 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:06:30.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.738+0000 7ff4abe41740 1 -- 172.21.15.16:0/817485612 shutdown_connections 2024-08-31T06:06:30.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.738+0000 7ff4abe41740 1 -- 172.21.15.16:0/817485612 wait complete. 2024-08-31T06:06:30.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.738+0000 7ff4abe41740 20 client.25240 trim_cache size 0 max 16384 2024-08-31T06:06:30.744 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:06:30.744 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "673e4aa0-675d-11ef-bcd4-c7b262605968", 2024-08-31T06:06:30.744 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-08-31T06:06:30.744 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24737, 2024-08-31T06:06:30.744 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-08-31T06:06:30.744 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-08-31T06:06:30.744 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-08-31T06:06:30.744 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 122.86048988100001, 2024-08-31T06:06:30.744 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 24, 2024-08-31T06:06:30.744 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 80, 2024-08-31T06:06:30.744 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 78, 2024-08-31T06:06:30.745 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 137.41638857999999 2024-08-31T06:06:30.745 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:06:30.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.739+0000 7ff49f200640 1 -- 172.21.15.16:0/1431405295 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff4840b4b80 msgr2=0x7ff4840b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:30.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.739+0000 7ff49f200640 1 --2- 172.21.15.16:0/1431405295 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff4840b4b80 0x7ff4840b7090 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff49802f500 tx=0x7ff49802f310 comp rx=0 tx=0).stop 2024-08-31T06:06:30.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.739+0000 7ff49f200640 1 -- 172.21.15.16:0/1431405295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4a0075550 msgr2=0x7ff4a019c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:30.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.739+0000 7ff49f200640 1 --2- 172.21.15.16:0/1431405295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4a0075550 0x7ff4a019c270 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff49000e9f0 tx=0x7ff49000eeb0 comp rx=0 tx=0).stop 2024-08-31T06:06:30.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.739+0000 7ff49f200640 1 -- 172.21.15.16:0/1431405295 shutdown_connections 2024-08-31T06:06:30.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.739+0000 7ff49f200640 1 --2- 172.21.15.16:0/1431405295 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff4840b4b80 0x7ff4840b7090 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:30.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.739+0000 7ff49f200640 1 --2- 172.21.15.16:0/1431405295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4a0075ea0 0x7ff4a019c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:30.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.739+0000 7ff49f200640 1 --2- 172.21.15.16:0/1431405295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4a010fcb0 0x7ff4a01a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:30.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.739+0000 7ff49f200640 1 --2- 172.21.15.16:0/1431405295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4a0075550 0x7ff4a019c270 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:30.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.739+0000 7ff49f200640 1 -- 172.21.15.16:0/1431405295 >> 172.21.15.16:0/1431405295 conn(0x7ff4a00ff1e0 msgr2=0x7ff4a01097e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:30.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.739+0000 7ff49f200640 1 -- 172.21.15.16:0/1431405295 shutdown_connections 2024-08-31T06:06:30.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:30.739+0000 7ff49f200640 1 -- 172.21.15.16:0/1431405295 wait complete. 2024-08-31T06:06:30.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:30 smithi155 ceph-mon[21742]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-08-31T06:06:31.575 INFO:teuthology.run_tasks:Running task kclient... 2024-08-31T06:06:31.600 INFO:tasks.kclient:Mounting kernel clients... 2024-08-31T06:06:31.600 INFO:tasks.kclient:config is {'client.0': {}} 2024-08-31T06:06:31.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-31T06:06:31.601 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-08-31T06:06:31.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-31T06:06:31.638 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2024-08-31T06:06:31.740 INFO:teuthology.orchestra.run.smithi016.stderr:Cannot find device "ceph-brx" 2024-08-31T06:06:31.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-31T06:06:31.743 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'wsync', 'context=system_u:object_r:postgresql_db_t:s0'], 'syntax': 'v1', 'mount_subvol_num': 1, 'mountpoint': '/tmp/cephfs'} 2024-08-31T06:06:31.743 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-08-31T06:06:31.743 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2024-08-31T06:06:31.744 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2024-08-31T06:06:31.744 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs ls 2024-08-31T06:06:32.123 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:06:32.579 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:32 smithi016 ceph-mon[17194]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-08-31T06:06:32.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:32 smithi184 ceph-mon[22331]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-08-31T06:06:32.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:32.757+0000 7f62e9400640 1 -- 172.21.15.16:0/1118157459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e4108b70 msgr2=0x7f62e410f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:32.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:32.757+0000 7f62e9400640 1 --2- 172.21.15.16:0/1118157459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e4108b70 0x7f62e410f3a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f62dc009ca0 tx=0x7f62dc02a8c0 comp rx=0 tx=0).stop 2024-08-31T06:06:32.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:32.757+0000 7f62e9400640 1 -- 172.21.15.16:0/1118157459 shutdown_connections 2024-08-31T06:06:32.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:32.757+0000 7f62e9400640 1 --2- 172.21.15.16:0/1118157459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e4108b70 0x7f62e410f3a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:32.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:32.757+0000 7f62e9400640 1 --2- 172.21.15.16:0/1118157459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62e4108220 0x7f62e4108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:32.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:32.757+0000 7f62e9400640 1 --2- 172.21.15.16:0/1118157459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62e41028e0 0x7f62e4102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:32.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:32.757+0000 7f62e9400640 1 -- 172.21.15.16:0/1118157459 >> 172.21.15.16:0/1118157459 conn(0x7f62e40794f0 msgr2=0x7f62e4079900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:32.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:32.758+0000 7f62e9400640 1 -- 172.21.15.16:0/1118157459 shutdown_connections 2024-08-31T06:06:32.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:32.758+0000 7f62e9400640 1 -- 172.21.15.16:0/1118157459 wait complete. 2024-08-31T06:06:32.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:32.759+0000 7f62e9400640 1 Processor -- start 2024-08-31T06:06:32.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:32.759+0000 7f62e9400640 1 -- start start 2024-08-31T06:06:32.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:32.759+0000 7f62e9400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62e41028e0 0x7f62e40ff440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:32.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:32.759+0000 7f62e9400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e4108220 0x7f62e40ff980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:32.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:32.759+0000 7f62e9400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62e4108b70 0x7f62e40ffec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:32.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:32.759+0000 7f62e9400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f62e4112790 con 0x7f62e4108220 2024-08-31T06:06:32.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:32.759+0000 7f62e9400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f62e4112610 con 0x7f62e4108b70 2024-08-31T06:06:32.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:32.759+0000 7f62e9400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f62e4112910 con 0x7f62e41028e0 2024-08-31T06:06:32.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:32.760+0000 7f62d9e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e4108220 0x7f62e40ff980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:32.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:32.760+0000 7f62d9e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e4108220 0x7f62e40ff980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45386/0 (socket says 172.21.15.16:45386) 2024-08-31T06:06:32.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:32.760+0000 7f62e2800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62e41028e0 0x7f62e40ff440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:32.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:32.760+0000 7f62d9e00640 1 -- 172.21.15.16:0/3877622518 learned_addr learned my addr 172.21.15.16:0/3877622518 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:32.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:32.760+0000 7f62e3200640 1 --2- 172.21.15.16:0/3877622518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62e4108b70 0x7f62e40ffec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:32.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:32.760+0000 7f62e2800640 1 -- 172.21.15.16:0/3877622518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62e4108b70 msgr2=0x7f62e40ffec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:32.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:32.760+0000 7f62e2800640 1 --2- 172.21.15.16:0/3877622518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62e4108b70 0x7f62e40ffec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:32.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:32.760+0000 7f62e2800640 1 -- 172.21.15.16:0/3877622518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e4108220 msgr2=0x7f62e40ff980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:32.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:32.760+0000 7f62e2800640 1 --2- 172.21.15.16:0/3877622518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e4108220 0x7f62e40ff980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:32.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:32.760+0000 7f62e2800640 1 -- 172.21.15.16:0/3877622518 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e41a8470 con 0x7f62e41028e0 2024-08-31T06:06:32.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:32.760+0000 7f62d9e00640 1 --2- 172.21.15.16:0/3877622518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e4108220 0x7f62e40ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:06:32.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:32.760+0000 7f62e2800640 1 --2- 172.21.15.16:0/3877622518 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62e41028e0 0x7f62e40ff440 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f62d400e9f0 tx=0x7f62d400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:32.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:32.761+0000 7f62dbe00640 1 -- 172.21.15.16:0/3877622518 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d400ccc0 con 0x7f62e41028e0 2024-08-31T06:06:32.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:32.761+0000 7f62e9400640 1 -- 172.21.15.16:0/3877622518 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62e41a8760 con 0x7f62e41028e0 2024-08-31T06:06:32.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:32.761+0000 7f62dbe00640 1 -- 172.21.15.16:0/3877622518 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62d4005020 con 0x7f62e41028e0 2024-08-31T06:06:32.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:32.761+0000 7f62dbe00640 1 -- 172.21.15.16:0/3877622518 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d4004a00 con 0x7f62e41028e0 2024-08-31T06:06:32.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:32.761+0000 7f62e9400640 1 -- 172.21.15.16:0/3877622518 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62e41a8c50 con 0x7f62e41028e0 2024-08-31T06:06:32.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:32.762+0000 7f62e9400640 1 -- 172.21.15.16:0/3877622518 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62e4103b60 con 0x7f62e41028e0 2024-08-31T06:06:32.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:32.764+0000 7f62dbe00640 1 -- 172.21.15.16:0/3877622518 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7f62d4004ba0 con 0x7f62e41028e0 2024-08-31T06:06:32.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:32.766+0000 7f62dbe00640 1 --2- 172.21.15.16:0/3877622518 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f62c80b4b80 0x7f62c80b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:32.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:32.766+0000 7f62dbe00640 1 -- 172.21.15.16:0/3877622518 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f62d40e5210 con 0x7f62e41028e0 2024-08-31T06:06:32.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:32.766+0000 7f62d9e00640 1 --2- 172.21.15.16:0/3877622518 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f62c80b4b80 0x7f62c80b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:32.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:32.767+0000 7f62d9e00640 1 --2- 172.21.15.16:0/3877622518 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f62c80b4b80 0x7f62c80b7090 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f62e406b060 tx=0x7f62cc005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:32.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:32.768+0000 7f62dbe00640 1 -- 172.21.15.16:0/3877622518 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f62d40ac220 con 0x7f62e41028e0 2024-08-31T06:06:32.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:32 smithi155 ceph-mon[21742]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-08-31T06:06:33.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:33.002+0000 7f62e9400640 1 -- 172.21.15.16:0/3877622518 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f62e4100740 con 0x7f62e41028e0 2024-08-31T06:06:33.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:33.003+0000 7f62dbe00640 1 -- 172.21.15.16:0/3877622518 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+73 (secure 0 0 0) 0x7f62e4100740 con 0x7f62e41028e0 2024-08-31T06:06:33.006 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-08-31T06:06:33.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:33.006+0000 7f62e9400640 1 -- 172.21.15.16:0/3877622518 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f62c80b4b80 msgr2=0x7f62c80b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:33.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:33.006+0000 7f62e9400640 1 --2- 172.21.15.16:0/3877622518 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f62c80b4b80 0x7f62c80b7090 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f62e406b060 tx=0x7f62cc005f90 comp rx=0 tx=0).stop 2024-08-31T06:06:33.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:33.006+0000 7f62e9400640 1 -- 172.21.15.16:0/3877622518 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62e41028e0 msgr2=0x7f62e40ff440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:33.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:33.006+0000 7f62e9400640 1 --2- 172.21.15.16:0/3877622518 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62e41028e0 0x7f62e40ff440 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f62d400e9f0 tx=0x7f62d400eeb0 comp rx=0 tx=0).stop 2024-08-31T06:06:33.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:33.006+0000 7f62e9400640 1 -- 172.21.15.16:0/3877622518 shutdown_connections 2024-08-31T06:06:33.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:33.006+0000 7f62e9400640 1 --2- 172.21.15.16:0/3877622518 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f62c80b4b80 0x7f62c80b7090 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:33.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:33.006+0000 7f62e9400640 1 --2- 172.21.15.16:0/3877622518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62e4108b70 0x7f62e40ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:33.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:33.006+0000 7f62e9400640 1 --2- 172.21.15.16:0/3877622518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e4108220 0x7f62e40ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:33.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:33.006+0000 7f62e9400640 1 --2- 172.21.15.16:0/3877622518 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62e41028e0 0x7f62e40ff440 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:33.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:33.006+0000 7f62e9400640 1 -- 172.21.15.16:0/3877622518 >> 172.21.15.16:0/3877622518 conn(0x7f62e40794f0 msgr2=0x7f62e4101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:33.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:33.006+0000 7f62e9400640 1 -- 172.21.15.16:0/3877622518 shutdown_connections 2024-08-31T06:06:33.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:33.006+0000 7f62e9400640 1 -- 172.21.15.16:0/3877622518 wait complete. 2024-08-31T06:06:33.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:33 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/3877622518' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-08-31T06:06:33.824 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-08-31T06:06:33.824 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-31T06:06:33.824 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-08-31T06:06:33.824 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-08-31T06:06:33.824 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2024-08-31T06:06:33.824 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-08-31T06:06:33.824 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-31T06:06:33.825 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-31T06:06:33.825 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2024-08-31T06:06:33.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-31T06:06:33.825 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-08-31T06:06:33.861 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-31T06:06:33.861 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-31T06:06:33.861 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-08-31T06:06:33.861 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-08-31T06:06:33.862 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-08-31T06:06:33.862 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-08-31T06:06:33.862 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-31T06:06:33.862 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-08-31T06:06:33.862 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-08-31T06:06:33.862 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-31T06:06:33.862 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-08-31T06:06:33.862 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-08-31T06:06:33.862 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-31T06:06:33.862 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-08-31T06:06:33.862 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-08-31T06:06:33.862 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-08-31T06:06:33.862 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42190sec preferred_lft 42190sec 2024-08-31T06:06:33.863 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-08-31T06:06:33.863 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-08-31T06:06:33.863 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-31T06:06:33.863 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-08-31T06:06:33.863 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-08-31T06:06:33.863 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-08-31T06:06:33.863 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-31T06:06:33.864 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-08-31T06:06:33.864 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add name ceph-brx type bridge 2024-08-31T06:06:33.864 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr flush dev ceph-brx 2024-08-31T06:06:33.864 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set ceph-brx up 2024-08-31T06:06:33.864 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-08-31T06:06:33.864 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-08-31T06:06:33.871 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:33 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/3877622518' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-08-31T06:06:33.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:33 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/3877622518' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-08-31T06:06:33.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-31T06:06:34.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-31T06:06:34.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-31T06:06:34.279 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-31T06:06:34.336 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-08-31T06:06:34.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-31T06:06:34.339 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-08-31T06:06:34.388 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-08-31T06:06:34.388 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-31T06:06:34.388 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-08-31T06:06:34.389 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-08-31T06:06:34.389 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-31T06:06:34.389 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-31T06:06:34.389 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-08-31T06:06:34.389 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-08-31T06:06:34.389 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-08-31T06:06:34.389 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-08-31T06:06:34.390 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-08-31T06:06:34.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-31T06:06:34.735 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:34 smithi016 ceph-mon[17194]: pgmap v343: 97 pgs: 97 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-08-31T06:06:34.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:34 smithi155 ceph-mon[21742]: pgmap v343: 97 pgs: 97 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-08-31T06:06:34.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:34 smithi184 ceph-mon[22331]: pgmap v343: 97 pgs: 97 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-08-31T06:06:35.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-31T06:06:35.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-31T06:06:35.016 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-08-31T06:06:35.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-31T06:06:35.053 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2024-08-31T06:06:35.129 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-31T06:06:35.129 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-08-31T06:06:35.129 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--tmp-cephfs 2024-08-31T06:06:35.129 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--tmp-cephfs 0 2024-08-31T06:06:35.129 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-08-31T06:06:35.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-31T06:06:35.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-31T06:06:35.310 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2024-08-31T06:06:35.310 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-31T06:06:35.310 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-08-31T06:06:35.310 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2024-08-31T06:06:35.311 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-08-31T06:06:35.311 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2024-08-31T06:06:35.311 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2024-08-31T06:06:35.311 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2024-08-31T06:06:35.311 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-08-31T06:06:35.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-31T06:06:35.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-31T06:06:35.550 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-31T06:06:35.551 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-08-31T06:06:35.551 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-08-31T06:06:35.551 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.0 master ceph-brx 2024-08-31T06:06:35.551 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-08-31T06:06:35.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-31T06:06:35.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-31T06:06:35.720 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-31T06:06:35.720 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /tmp/cephfs 2024-08-31T06:06:35.755 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/tmp/cephfs' 2024-08-31T06:06:35.755 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-31T06:06:35.756 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /tmp/cephfs 2024-08-31T06:06:35.830 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-08-31T06:06:35.830 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-08-31T06:06:35.830 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/ba5423bc-5c83-4d64-ae1c-ad8850110ce4 2024-08-31T06:06:35.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-31T06:06:35.830 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/ba5423bc-5c83-4d64-ae1c-ad8850110ce4 /tmp/cephfs -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync,context=system_u:object_r:postgresql_db_t:s0 2024-08-31T06:06:36.076 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync,context="system_u:object_r:postgresql_db_t:s0" 2024-08-31T06:06:36.077 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-08-31T06:06:36.077 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-08-31T06:06:36.746 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:36 smithi016 ceph-mon[17194]: pgmap v344: 97 pgs: 97 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-08-31T06:06:36.768 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync,context="system_u:object_r:postgresql_db_t:s0" 2024-08-31T06:06:36.769 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-08-31T06:06:36.769 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-08-31T06:06:36.769 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.155:6789,172.21.15.184:6789" 2024-08-31T06:06:36.769 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.155:6789,172.21.15.184:6789:/volumes/qa/sv_1/ba5423bc-5c83-4d64-ae1c-ad8850110ce4 2024-08-31T06:06:36.769 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync,context="system_u:object_r:postgresql_db_t:s0",key=0,fsid=673e4aa0-675d-11ef-bcd4-c7b262605968" will pass to kernel 2024-08-31T06:06:36.774 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-08-31T06:06:36.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-31T06:06:36.774 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /tmp/cephfs 2024-08-31T06:06:36.833 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-08-31T06:06:36.871 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:36 smithi155 ceph-mon[21742]: pgmap v344: 97 pgs: 97 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-08-31T06:06:36.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:36 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-08-31T06:06:36.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-31T06:06:36.926 INFO:teuthology.orchestra.run.smithi016.stdout:client15237 2024-08-31T06:06:36.927 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-08-31T06:06:36.927 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-31T06:06:36.964 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-08-31T06:06:36.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:36 smithi184 ceph-mon[22331]: pgmap v344: 97 pgs: 97 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-08-31T06:06:37.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-08-31T06:06:37.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-31T06:06:37.095 INFO:teuthology.orchestra.run.smithi016.stdout:client15237 2024-08-31T06:06:37.095 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-08-31T06:06:37.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-08-31T06:06:37.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-31T06:06:37.187 INFO:teuthology.orchestra.run.smithi016.stdout:client15237 2024-08-31T06:06:37.187 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/cephfs && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968.client15237/status') 2024-08-31T06:06:37.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-31T06:06:37.287 INFO:teuthology.orchestra.run.smithi016.stdout:instance: client.15237 (3)192.168.0.1:0/2554496825 2024-08-31T06:06:37.288 INFO:teuthology.orchestra.run.smithi016.stdout:blocklisted: false 2024-08-31T06:06:37.288 INFO:teuthology.orchestra.run.smithi016.stderr:0+1 records in 2024-08-31T06:06:37.288 INFO:teuthology.orchestra.run.smithi016.stderr:0+1 records out 2024-08-31T06:06:37.288 INFO:teuthology.orchestra.run.smithi016.stderr:70 bytes copied, 5.6728e-05 s, 1.2 MB/s 2024-08-31T06:06:37.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-31T06:06:37.298 INFO:teuthology.run_tasks:Running task exec... 2024-08-31T06:06:37.309 INFO:teuthology.task.exec:Executing custom commands... 2024-08-31T06:06:37.309 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-08-31T06:06:37.309 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-08-31T06:06:37.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37.529+0000 7f538ec00640 1 -- 172.21.15.16:0/1639935856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5388108fe0 msgr2=0x7f53881093f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:37.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37.529+0000 7f538ec00640 1 --2- 172.21.15.16:0/1639935856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5388108fe0 0x7f53881093f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5374009900 tx=0x7f537402a3f0 comp rx=0 tx=0).stop 2024-08-31T06:06:37.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37.530+0000 7f538ec00640 1 -- 172.21.15.16:0/1639935856 shutdown_connections 2024-08-31T06:06:37.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37.530+0000 7f538ec00640 1 --2- 172.21.15.16:0/1639935856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5388109930 0x7f5388110160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:37.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37.530+0000 7f538ec00640 1 --2- 172.21.15.16:0/1639935856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5388108fe0 0x7f53881093f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:37.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37.530+0000 7f538ec00640 1 --2- 172.21.15.16:0/1639935856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53881036b0 0x7f5388103ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:37.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37.530+0000 7f538ec00640 1 -- 172.21.15.16:0/1639935856 >> 172.21.15.16:0/1639935856 conn(0x7f53880ff0d0 msgr2=0x7f5388101520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:37.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37.530+0000 7f538ec00640 1 -- 172.21.15.16:0/1639935856 shutdown_connections 2024-08-31T06:06:37.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37.530+0000 7f538ec00640 1 -- 172.21.15.16:0/1639935856 wait complete. 2024-08-31T06:06:37.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37.531+0000 7f538ec00640 1 Processor -- start 2024-08-31T06:06:37.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37.531+0000 7f538ec00640 1 -- start start 2024-08-31T06:06:37.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37.531+0000 7f538ec00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53881036b0 0x7f538819c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:37.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37.531+0000 7f538ec00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5388108fe0 0x7f538819c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:37.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37.531+0000 7f538ec00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5388109930 0x7f53881a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:37.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37.531+0000 7f538ec00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f53881135c0 con 0x7f53881036b0 2024-08-31T06:06:37.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37.531+0000 7f538ec00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5388113440 con 0x7f5388108fe0 2024-08-31T06:06:37.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37.531+0000 7f538ec00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5388113740 con 0x7f5388109930 2024-08-31T06:06:37.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37.531+0000 7f538d800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53881036b0 0x7f538819c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:37.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37.531+0000 7f538d800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53881036b0 0x7f538819c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50714/0 (socket says 172.21.15.16:50714) 2024-08-31T06:06:37.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37.532+0000 7f538d800640 1 -- 172.21.15.16:0/2010000129 learned_addr learned my addr 172.21.15.16:0/2010000129 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:37.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37.532+0000 7f538e200640 1 --2- 172.21.15.16:0/2010000129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5388109930 0x7f53881a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:37.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37.532+0000 7f538ce00640 1 --2- 172.21.15.16:0/2010000129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5388108fe0 0x7f538819c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:37.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37.532+0000 7f538d800640 1 -- 172.21.15.16:0/2010000129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5388109930 msgr2=0x7f53881a1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:37.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37.532+0000 7f538d800640 1 --2- 172.21.15.16:0/2010000129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5388109930 0x7f53881a1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:37.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37.532+0000 7f538d800640 1 -- 172.21.15.16:0/2010000129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5388108fe0 msgr2=0x7f538819c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:37.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37.532+0000 7f538d800640 1 --2- 172.21.15.16:0/2010000129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5388108fe0 0x7f538819c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:37.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37.532+0000 7f538d800640 1 -- 172.21.15.16:0/2010000129 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53881a2210 con 0x7f53881036b0 2024-08-31T06:06:37.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37.532+0000 7f538e200640 1 --2- 172.21.15.16:0/2010000129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5388109930 0x7f53881a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:06:37.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37.532+0000 7f538d800640 1 --2- 172.21.15.16:0/2010000129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53881036b0 0x7f538819c2e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f537c00e9f0 tx=0x7f537c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:37.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37.532+0000 7f5386000640 1 -- 172.21.15.16:0/2010000129 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537c00cd80 con 0x7f53881036b0 2024-08-31T06:06:37.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37.533+0000 7f5386000640 1 -- 172.21.15.16:0/2010000129 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f537c0108e0 con 0x7f53881036b0 2024-08-31T06:06:37.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37.533+0000 7f538ec00640 1 -- 172.21.15.16:0/2010000129 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53881a2500 con 0x7f53881036b0 2024-08-31T06:06:37.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37.533+0000 7f5386000640 1 -- 172.21.15.16:0/2010000129 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537c018980 con 0x7f53881036b0 2024-08-31T06:06:37.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37.533+0000 7f538ec00640 1 -- 172.21.15.16:0/2010000129 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53881a8c90 con 0x7f53881036b0 2024-08-31T06:06:37.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37.534+0000 7f538ec00640 1 -- 172.21.15.16:0/2010000129 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5388103ac0 con 0x7f53881036b0 2024-08-31T06:06:37.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37.536+0000 7f5386000640 1 -- 172.21.15.16:0/2010000129 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7f537c010430 con 0x7f53881036b0 2024-08-31T06:06:37.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37.537+0000 7f5386000640 1 --2- 172.21.15.16:0/2010000129 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f53700b4b80 0x7f53700b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:37.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37.538+0000 7f5386000640 1 -- 172.21.15.16:0/2010000129 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f537c0e5aa0 con 0x7f53881036b0 2024-08-31T06:06:37.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37.538+0000 7f538ce00640 1 --2- 172.21.15.16:0/2010000129 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f53700b4b80 0x7f53700b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:37.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37.538+0000 7f538ce00640 1 --2- 172.21.15.16:0/2010000129 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f53700b4b80 0x7f53700b7090 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5374009230 tx=0x7f5374031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:37.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37.540+0000 7f5386000640 1 -- 172.21.15.16:0/2010000129 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f537c0ac460 con 0x7f53881036b0 2024-08-31T06:06:37.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:37.777+0000 7f538ec00640 1 -- 172.21.15.16:0/2010000129 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f538819d400 con 0x7f53881036b0 2024-08-31T06:06:38.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.491+0000 7f5386000640 1 -- 172.21.15.16:0/2010000129 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v25) ==== 117+0+0 (secure 0 0 0) 0x7f537c0b0c20 con 0x7f53881036b0 2024-08-31T06:06:38.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.494+0000 7f538ec00640 1 -- 172.21.15.16:0/2010000129 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f53700b4b80 msgr2=0x7f53700b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:38.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.494+0000 7f538ec00640 1 --2- 172.21.15.16:0/2010000129 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f53700b4b80 0x7f53700b7090 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5374009230 tx=0x7f5374031400 comp rx=0 tx=0).stop 2024-08-31T06:06:38.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.494+0000 7f538ec00640 1 -- 172.21.15.16:0/2010000129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53881036b0 msgr2=0x7f538819c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:38.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.494+0000 7f538ec00640 1 --2- 172.21.15.16:0/2010000129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53881036b0 0x7f538819c2e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f537c00e9f0 tx=0x7f537c00eeb0 comp rx=0 tx=0).stop 2024-08-31T06:06:38.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.494+0000 7f538ec00640 1 -- 172.21.15.16:0/2010000129 shutdown_connections 2024-08-31T06:06:38.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.494+0000 7f538ec00640 1 --2- 172.21.15.16:0/2010000129 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f53700b4b80 0x7f53700b7090 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:38.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.494+0000 7f538ec00640 1 --2- 172.21.15.16:0/2010000129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5388109930 0x7f53881a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:38.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.494+0000 7f538ec00640 1 --2- 172.21.15.16:0/2010000129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5388108fe0 0x7f538819c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:38.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.494+0000 7f538ec00640 1 --2- 172.21.15.16:0/2010000129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53881036b0 0x7f538819c2e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:38.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.494+0000 7f538ec00640 1 -- 172.21.15.16:0/2010000129 >> 172.21.15.16:0/2010000129 conn(0x7f53880ff0d0 msgr2=0x7f53881086e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:38.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.494+0000 7f538ec00640 1 -- 172.21.15.16:0/2010000129 shutdown_connections 2024-08-31T06:06:38.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.495+0000 7f538ec00640 1 -- 172.21.15.16:0/2010000129 wait complete. 2024-08-31T06:06:38.523 INFO:teuthology.run_tasks:Running task check-counter... 2024-08-31T06:06:38.534 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-08-31T06:06:38.545 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-31T06:06:38.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.773+0000 7f3dc7200640 1 -- 172.21.15.16:0/909741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc8105880 msgr2=0x7f3dc8105c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:38.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.773+0000 7f3dc7200640 1 --2- 172.21.15.16:0/909741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc8105880 0x7f3dc8105c90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3db8009900 tx=0x7f3db802a3f0 comp rx=0 tx=0).stop 2024-08-31T06:06:38.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.773+0000 7f3dc7200640 1 -- 172.21.15.16:0/909741 shutdown_connections 2024-08-31T06:06:38.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.773+0000 7f3dc7200640 1 --2- 172.21.15.16:0/909741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3dc810b1b0 0x7f3dc810b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:38.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.773+0000 7f3dc7200640 1 --2- 172.21.15.16:0/909741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3dc810bb00 0x7f3dc8112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:38.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.773+0000 7f3dc7200640 1 --2- 172.21.15.16:0/909741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc8105880 0x7f3dc8105c90 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:38.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.773+0000 7f3dc7200640 1 -- 172.21.15.16:0/909741 >> 172.21.15.16:0/909741 conn(0x7f3dc8101300 msgr2=0x7f3dc8103750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:38.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.773+0000 7f3dc7200640 1 -- 172.21.15.16:0/909741 shutdown_connections 2024-08-31T06:06:38.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.773+0000 7f3dc7200640 1 -- 172.21.15.16:0/909741 wait complete. 2024-08-31T06:06:38.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.774+0000 7f3dc7200640 1 Processor -- start 2024-08-31T06:06:38.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.774+0000 7f3dc7200640 1 -- start start 2024-08-31T06:06:38.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.775+0000 7f3dc7200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3dc8105880 0x7f3dc819c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:38.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.775+0000 7f3dc7200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3dc810b1b0 0x7f3dc819c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:38.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.775+0000 7f3dc7200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc810bb00 0x7f3dc81a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:38.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.775+0000 7f3dc7200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc8115820 con 0x7f3dc810bb00 2024-08-31T06:06:38.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.775+0000 7f3dc7200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc81156a0 con 0x7f3dc810b1b0 2024-08-31T06:06:38.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.775+0000 7f3dc7200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc81159a0 con 0x7f3dc8105880 2024-08-31T06:06:38.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.775+0000 7f3dc6800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc810bb00 0x7f3dc81a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:38.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.775+0000 7f3dc6800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc810bb00 0x7f3dc81a1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50728/0 (socket says 172.21.15.16:50728) 2024-08-31T06:06:38.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.775+0000 7f3dc6800640 1 -- 172.21.15.16:0/828528646 learned_addr learned my addr 172.21.15.16:0/828528646 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:38.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.775+0000 7f3dc5400640 1 --2- 172.21.15.16:0/828528646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3dc810b1b0 0x7f3dc819c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:38.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.775+0000 7f3dc5e00640 1 --2- 172.21.15.16:0/828528646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3dc8105880 0x7f3dc819c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:38.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.775+0000 7f3dc6800640 1 -- 172.21.15.16:0/828528646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3dc8105880 msgr2=0x7f3dc819c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:38.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.775+0000 7f3dc6800640 1 --2- 172.21.15.16:0/828528646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3dc8105880 0x7f3dc819c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:38.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.775+0000 7f3dc6800640 1 -- 172.21.15.16:0/828528646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3dc810b1b0 msgr2=0x7f3dc819c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:38.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.775+0000 7f3dc6800640 1 --2- 172.21.15.16:0/828528646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3dc810b1b0 0x7f3dc819c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:38.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.775+0000 7f3dc6800640 1 -- 172.21.15.16:0/828528646 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dc81a2110 con 0x7f3dc810bb00 2024-08-31T06:06:38.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.776+0000 7f3dc6800640 1 --2- 172.21.15.16:0/828528646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc810bb00 0x7f3dc81a1a10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3dc000eb10 tx=0x7f3dc000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:38.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.776+0000 7f3dbea00640 1 -- 172.21.15.16:0/828528646 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0004b20 con 0x7f3dc810bb00 2024-08-31T06:06:38.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.776+0000 7f3dbea00640 1 -- 172.21.15.16:0/828528646 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3dc0004cc0 con 0x7f3dc810bb00 2024-08-31T06:06:38.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.776+0000 7f3dc7200640 1 -- 172.21.15.16:0/828528646 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dc81a2400 con 0x7f3dc810bb00 2024-08-31T06:06:38.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.776+0000 7f3dbea00640 1 -- 172.21.15.16:0/828528646 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0010670 con 0x7f3dc810bb00 2024-08-31T06:06:38.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.776+0000 7f3dc7200640 1 -- 172.21.15.16:0/828528646 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dc81a8b90 con 0x7f3dc810bb00 2024-08-31T06:06:38.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.777+0000 7f3dc7200640 1 -- 172.21.15.16:0/828528646 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dc8105c90 con 0x7f3dc810bb00 2024-08-31T06:06:38.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.779+0000 7f3dbea00640 1 -- 172.21.15.16:0/828528646 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7f3dc0005000 con 0x7f3dc810bb00 2024-08-31T06:06:38.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.781+0000 7f3dbea00640 1 --2- 172.21.15.16:0/828528646 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3dac0b4c00 0x7f3dac0b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:38.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.781+0000 7f3dbea00640 1 -- 172.21.15.16:0/828528646 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f3dc00e59f0 con 0x7f3dc810bb00 2024-08-31T06:06:38.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.781+0000 7f3dc5e00640 1 --2- 172.21.15.16:0/828528646 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3dac0b4c00 0x7f3dac0b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:38.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.782+0000 7f3dc5e00640 1 --2- 172.21.15.16:0/828528646 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3dac0b4c00 0x7f3dac0b7110 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3db8005ab0 tx=0x7f3db8005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:38.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.783+0000 7f3dbea00640 1 -- 172.21.15.16:0/828528646 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f3dc0056130 con 0x7f3dc810bb00 2024-08-31T06:06:38.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:38 smithi155 ceph-mon[21742]: pgmap v345: 97 pgs: 97 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 5 op/s 2024-08-31T06:06:38.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:38 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/2010000129' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-08-31T06:06:38.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:38 smithi016 ceph-mon[17194]: pgmap v345: 97 pgs: 97 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 5 op/s 2024-08-31T06:06:38.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:38 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/2010000129' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-08-31T06:06:38.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:38 smithi184 ceph-mon[22331]: pgmap v345: 97 pgs: 97 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 5 op/s 2024-08-31T06:06:38.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:38 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/2010000129' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-08-31T06:06:38.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.984+0000 7f3dc7200640 1 -- 172.21.15.16:0/828528646 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3dc819d300 con 0x7f3dc810bb00 2024-08-31T06:06:38.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.986+0000 7f3dbea00640 1 -- 172.21.15.16:0/828528646 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+23043 (secure 0 0 0) 0x7f3dc819d300 con 0x7f3dc810bb00 2024-08-31T06:06:38.989 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:06:38.989 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":80,"fsid":"673e4aa0-675d-11ef-bcd4-c7b262605968","created":"2024-08-31T05:55:09.632536+0000","modified":"2024-08-31T06:05:25.039584+0000","last_up_change":"2024-08-31T06:03:27.893697+0000","last_in_change":"2024-08-31T06:03:05.664945+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-31T05:59:23.642270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-31T06:04:20.871835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-31T06:04:24.119499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e349dae-6aa5-4b6d-9107-a945655e11bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6803","nonce":1725037400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6805","nonce":1725037400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6809","nonce":1725037400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6807","nonce":1725037400}]},"public_addr":"172.21.15.16:6803/1725037400","cluster_addr":"172.21.15.16:6805/1725037400","heartbeat_back_addr":"172.21.15.16:6809/1725037400","heartbeat_front_addr":"172.21.15.16:6807/1725037400","state":["exists","up"]},{"osd":1,"uuid":"bd645dc3-93b5-40af-89d8-08ccec22e50d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6801","nonce":4241618440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6803","nonce":4241618440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6807","nonce":4241618440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6805","nonce":4241618440}]},"public_addr":"172.21.15.155:6801/4241618440","cluster_addr":"172.21.15.155:6803/4241618440","heartbeat_back_addr":"172.21.15.155:6807/4241618440","heartbeat_front_addr":"172.21.15.155:6805/4241618440","state":["exists","up"]},{"osd":2,"uuid":"c69c8c19-32f8-400a-86d4-0f4ce240a7de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6801","nonce":2193296422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6803","nonce":2193296422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6807","nonce":2193296422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6805","nonce":2193296422}]},"public_addr":"172.21.15.184:6801/2193296422","cluster_addr":"172.21.15.184:6803/2193296422","heartbeat_back_addr":"172.21.15.184:6807/2193296422","heartbeat_front_addr":"172.21.15.184:6805/2193296422","state":["exists","up"]},{"osd":3,"uuid":"8892e35f-6880-434c-a805-ab8ccbed2bf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6811","nonce":81681815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6813","nonce":81681815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6817","nonce":81681815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6815","nonce":81681815}]},"public_addr":"172.21.15.16:6811/81681815","cluster_addr":"172.21.15.16:6813/81681815","heartbeat_back_addr":"172.21.15.16:6817/81681815","heartbeat_front_addr":"172.21.15.16:6815/81681815","state":["exists","up"]},{"osd":4,"uuid":"d2515e73-88f7-4654-9762-adb8802777ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6809","nonce":3592422019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6811","nonce":3592422019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6815","nonce":3592422019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6813","nonce":3592422019}]},"public_addr":"172.21.15.155:6809/3592422019","cluster_addr":"172.21.15.155:6811/3592422019","heartbeat_back_addr":"172.21.15.155:6815/3592422019","heartbeat_front_addr":"172.21.15.155:6813/3592422019","state":["exists","up"]},{"osd":5,"uuid":"5ce6e2fa-c17c-49b3-9402-f19bfc21076d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6809","nonce":2054894065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6811","nonce":2054894065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6815","nonce":2054894065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6813","nonce":2054894065}]},"public_addr":"172.21.15.184:6809/2054894065","cluster_addr":"172.21.15.184:6811/2054894065","heartbeat_back_addr":"172.21.15.184:6815/2054894065","heartbeat_front_addr":"172.21.15.184:6813/2054894065","state":["exists","up"]},{"osd":6,"uuid":"f257d8d1-ad59-4bd8-821d-bf29e50b547e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6819","nonce":1697421092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6821","nonce":1697421092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6825","nonce":1697421092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6823","nonce":1697421092}]},"public_addr":"172.21.15.16:6819/1697421092","cluster_addr":"172.21.15.16:6821/1697421092","heartbeat_back_addr":"172.21.15.16:6825/1697421092","heartbeat_front_addr":"172.21.15.16:6823/1697421092","state":["exists","up"]},{"osd":7,"uuid":"8e76c20e-8d25-423b-b7e8-de642d9c9a6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6817","nonce":2671827901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6819","nonce":2671827901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6823","nonce":2671827901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6821","nonce":2671827901}]},"public_addr":"172.21.15.155:6817/2671827901","cluster_addr":"172.21.15.155:6819/2671827901","heartbeat_back_addr":"172.21.15.155:6823/2671827901","heartbeat_front_addr":"172.21.15.155:6821/2671827901","state":["exists","up"]},{"osd":8,"uuid":"0f0bb23f-4bc3-4198-8381-5b5cd16ae394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6817","nonce":2479117823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6819","nonce":2479117823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6823","nonce":2479117823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6821","nonce":2479117823}]},"public_addr":"172.21.15.184:6817/2479117823","cluster_addr":"172.21.15.184:6819/2479117823","heartbeat_back_addr":"172.21.15.184:6823/2479117823","heartbeat_front_addr":"172.21.15.184:6821/2479117823","state":["exists","up"]},{"osd":9,"uuid":"bc27afcb-7871-4f72-a043-044fd5fe7d86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6827","nonce":3015700546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6829","nonce":3015700546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6833","nonce":3015700546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6831","nonce":3015700546}]},"public_addr":"172.21.15.16:6827/3015700546","cluster_addr":"172.21.15.16:6829/3015700546","heartbeat_back_addr":"172.21.15.16:6833/3015700546","heartbeat_front_addr":"172.21.15.16:6831/3015700546","state":["exists","up"]},{"osd":10,"uuid":"f2548997-72ee-400e-ab8d-5966d85b2cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6825","nonce":2427122475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6827","nonce":2427122475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6831","nonce":2427122475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6829","nonce":2427122475}]},"public_addr":"172.21.15.155:6825/2427122475","cluster_addr":"172.21.15.155:6827/2427122475","heartbeat_back_addr":"172.21.15.155:6831/2427122475","heartbeat_front_addr":"172.21.15.155:6829/2427122475","state":["exists","up"]},{"osd":11,"uuid":"8297da1d-d3e2-4a4d-9396-426dfafa3e95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6825","nonce":399265701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6827","nonce":399265701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6831","nonce":399265701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6829","nonce":399265701}]},"public_addr":"172.21.15.184:6825/399265701","cluster_addr":"172.21.15.184:6827/399265701","heartbeat_back_addr":"172.21.15.184:6831/399265701","heartbeat_front_addr":"172.21.15.184:6829/399265701","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:58:30.842360+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:58:57.164488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:59:20.778103+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:59:50.123991+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:00:16.789933+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:00:41.788905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:01:12.281376+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:01:39.211858+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:02:04.958771+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:02:35.645947+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:03:01.940433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:03:26.728673+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3923220960":"2024-09-01T05:57:23.153214+0000","172.21.15.16:0/3013533836":"2024-09-01T05:57:23.153214+0000","172.21.15.16:6800/3755683228":"2024-09-01T05:55:45.075902+0000","172.21.15.16:0/3064435470":"2024-09-01T05:55:45.075902+0000","172.21.15.16:0/911709962":"2024-09-01T05:55:45.075902+0000","172.21.15.16:0/2937368967":"2024-09-01T05:55:45.075902+0000","172.21.15.16:6800/1378972230":"2024-09-01T05:56:14.195704+0000","172.21.15.16:0/3047316995":"2024-09-01T05:56:14.195704+0000","172.21.15.16:6801/3755683228":"2024-09-01T05:55:45.075902+0000","172.21.15.16:6801/2571082634":"2024-09-01T05:57:23.153214+0000","172.21.15.16:0/2136704295":"2024-09-01T05:56:14.195704+0000","172.21.15.16:0/2259332778":"2024-09-01T05:56:14.195704+0000","172.21.15.16:6800/2571082634":"2024-09-01T05:57:23.153214+0000","172.21.15.16:6801/1378972230":"2024-09-01T05:56:14.195704+0000","172.21.15.16:0/2126457673":"2024-09-01T05:57:23.153214+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-31T06:06:38.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.989+0000 7f3dc7200640 1 -- 172.21.15.16:0/828528646 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3dac0b4c00 msgr2=0x7f3dac0b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:38.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.989+0000 7f3dc7200640 1 --2- 172.21.15.16:0/828528646 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3dac0b4c00 0x7f3dac0b7110 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3db8005ab0 tx=0x7f3db8005a20 comp rx=0 tx=0).stop 2024-08-31T06:06:38.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.989+0000 7f3dc7200640 1 -- 172.21.15.16:0/828528646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc810bb00 msgr2=0x7f3dc81a1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:38.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.989+0000 7f3dc7200640 1 --2- 172.21.15.16:0/828528646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc810bb00 0x7f3dc81a1a10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3dc000eb10 tx=0x7f3dc000c2d0 comp rx=0 tx=0).stop 2024-08-31T06:06:38.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.989+0000 7f3dc7200640 1 -- 172.21.15.16:0/828528646 shutdown_connections 2024-08-31T06:06:38.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.989+0000 7f3dc7200640 1 --2- 172.21.15.16:0/828528646 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3dac0b4c00 0x7f3dac0b7110 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:38.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.989+0000 7f3dc7200640 1 --2- 172.21.15.16:0/828528646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc810bb00 0x7f3dc81a1a10 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:38.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.989+0000 7f3dc7200640 1 --2- 172.21.15.16:0/828528646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3dc810b1b0 0x7f3dc819c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:38.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.989+0000 7f3dc7200640 1 --2- 172.21.15.16:0/828528646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3dc8105880 0x7f3dc819c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:38.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.989+0000 7f3dc7200640 1 -- 172.21.15.16:0/828528646 >> 172.21.15.16:0/828528646 conn(0x7f3dc8101300 msgr2=0x7f3dc810a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:38.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.989+0000 7f3dc7200640 1 -- 172.21.15.16:0/828528646 shutdown_connections 2024-08-31T06:06:38.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:38.989+0000 7f3dc7200640 1 -- 172.21.15.16:0/828528646 wait complete. 2024-08-31T06:06:39.015 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-31T05:59:23.642270+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-31T06:04:20.871835+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-31T06:04:24.119499+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-08-31T06:06:39.015 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-31T06:06:39.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.239+0000 7f5569c00640 1 -- 172.21.15.16:0/4278705669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5564103690 msgr2=0x7f5564103aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:39.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.239+0000 7f5569c00640 1 --2- 172.21.15.16:0/4278705669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5564103690 0x7f5564103aa0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5554009900 tx=0x7f555402a3f0 comp rx=0 tx=0).stop 2024-08-31T06:06:39.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.239+0000 7f5569c00640 1 -- 172.21.15.16:0/4278705669 shutdown_connections 2024-08-31T06:06:39.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.239+0000 7f5569c00640 1 --2- 172.21.15.16:0/4278705669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5564109a10 0x7f5564110240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:39.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.239+0000 7f5569c00640 1 --2- 172.21.15.16:0/4278705669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55641090c0 0x7f55641094d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:39.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.239+0000 7f5569c00640 1 --2- 172.21.15.16:0/4278705669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5564103690 0x7f5564103aa0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:39.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.239+0000 7f5569c00640 1 -- 172.21.15.16:0/4278705669 >> 172.21.15.16:0/4278705669 conn(0x7f55640ff0d0 msgr2=0x7f5564101520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:39.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.239+0000 7f5569c00640 1 -- 172.21.15.16:0/4278705669 shutdown_connections 2024-08-31T06:06:39.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.240+0000 7f5569c00640 1 -- 172.21.15.16:0/4278705669 wait complete. 2024-08-31T06:06:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.240+0000 7f5569c00640 1 Processor -- start 2024-08-31T06:06:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.241+0000 7f5569c00640 1 -- start start 2024-08-31T06:06:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.241+0000 7f5569c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5564103690 0x7f556419a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.241+0000 7f5569c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55641090c0 0x7f556419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.241+0000 7f5569c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5564109a10 0x7f556419f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.241+0000 7f5569c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f55641136e0 con 0x7f5564109a10 2024-08-31T06:06:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.241+0000 7f5569c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5564113560 con 0x7f55641090c0 2024-08-31T06:06:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.241+0000 7f5569c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5564113860 con 0x7f5564103690 2024-08-31T06:06:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.241+0000 7f5569200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5564109a10 0x7f556419f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.241+0000 7f5569200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5564109a10 0x7f556419f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50752/0 (socket says 172.21.15.16:50752) 2024-08-31T06:06:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.241+0000 7f5569200640 1 -- 172.21.15.16:0/576342440 learned_addr learned my addr 172.21.15.16:0/576342440 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.241+0000 7f5563400640 1 --2- 172.21.15.16:0/576342440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55641090c0 0x7f556419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:39.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.241+0000 7f5563e00640 1 --2- 172.21.15.16:0/576342440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5564103690 0x7f556419a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:39.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.242+0000 7f5569200640 1 -- 172.21.15.16:0/576342440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5564103690 msgr2=0x7f556419a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:39.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.242+0000 7f5569200640 1 --2- 172.21.15.16:0/576342440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5564103690 0x7f556419a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:39.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.242+0000 7f5569200640 1 -- 172.21.15.16:0/576342440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55641090c0 msgr2=0x7f556419a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:39.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.242+0000 7f5569200640 1 --2- 172.21.15.16:0/576342440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55641090c0 0x7f556419a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:39.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.242+0000 7f5569200640 1 -- 172.21.15.16:0/576342440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55641a0030 con 0x7f5564109a10 2024-08-31T06:06:39.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.242+0000 7f5569200640 1 --2- 172.21.15.16:0/576342440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5564109a10 0x7f556419f930 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f555c00eb10 tx=0x7f555c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:39.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.242+0000 7f5560c00640 1 -- 172.21.15.16:0/576342440 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555c004b20 con 0x7f5564109a10 2024-08-31T06:06:39.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.242+0000 7f5569c00640 1 -- 172.21.15.16:0/576342440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55641a0320 con 0x7f5564109a10 2024-08-31T06:06:39.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.242+0000 7f5569c00640 1 -- 172.21.15.16:0/576342440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55641a6ab0 con 0x7f5564109a10 2024-08-31T06:06:39.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.243+0000 7f5560c00640 1 -- 172.21.15.16:0/576342440 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f555c004cc0 con 0x7f5564109a10 2024-08-31T06:06:39.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.243+0000 7f5560c00640 1 -- 172.21.15.16:0/576342440 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555c010690 con 0x7f5564109a10 2024-08-31T06:06:39.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.244+0000 7f5569c00640 1 -- 172.21.15.16:0/576342440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55440052c0 con 0x7f5564109a10 2024-08-31T06:06:39.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.246+0000 7f5560c00640 1 -- 172.21.15.16:0/576342440 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7f555c005000 con 0x7f5564109a10 2024-08-31T06:06:39.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.247+0000 7f5560c00640 1 --2- 172.21.15.16:0/576342440 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f55480b4b80 0x7f55480b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:39.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.247+0000 7f5560c00640 1 -- 172.21.15.16:0/576342440 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f555c0e5720 con 0x7f5564109a10 2024-08-31T06:06:39.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.247+0000 7f5563e00640 1 --2- 172.21.15.16:0/576342440 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f55480b4b80 0x7f55480b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:39.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.248+0000 7f5563e00640 1 --2- 172.21.15.16:0/576342440 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f55480b4b80 0x7f55480b7090 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5554031440 tx=0x7f5554009560 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:39.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.251+0000 7f5560c00640 1 -- 172.21.15.16:0/576342440 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f555c0ac730 con 0x7f5564109a10 2024-08-31T06:06:39.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.446+0000 7f5569c00640 1 -- 172.21.15.16:0/576342440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5544005dc0 con 0x7f5564109a10 2024-08-31T06:06:39.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.447+0000 7f5560c00640 1 -- 172.21.15.16:0/576342440 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) ==== 93+0+10 (secure 0 0 0) 0x7f555c0ac030 con 0x7f5564109a10 2024-08-31T06:06:39.449 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-08-31T06:06:39.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.449+0000 7f5569c00640 1 -- 172.21.15.16:0/576342440 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f55480b4b80 msgr2=0x7f55480b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:39.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.449+0000 7f5569c00640 1 --2- 172.21.15.16:0/576342440 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f55480b4b80 0x7f55480b7090 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5554031440 tx=0x7f5554009560 comp rx=0 tx=0).stop 2024-08-31T06:06:39.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.449+0000 7f5569c00640 1 -- 172.21.15.16:0/576342440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5564109a10 msgr2=0x7f556419f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:39.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.449+0000 7f5569c00640 1 --2- 172.21.15.16:0/576342440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5564109a10 0x7f556419f930 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f555c00eb10 tx=0x7f555c00c2d0 comp rx=0 tx=0).stop 2024-08-31T06:06:39.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.449+0000 7f5569c00640 1 -- 172.21.15.16:0/576342440 shutdown_connections 2024-08-31T06:06:39.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.449+0000 7f5569c00640 1 --2- 172.21.15.16:0/576342440 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f55480b4b80 0x7f55480b7090 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:39.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.449+0000 7f5569c00640 1 --2- 172.21.15.16:0/576342440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5564109a10 0x7f556419f930 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:39.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.449+0000 7f5569c00640 1 --2- 172.21.15.16:0/576342440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55641090c0 0x7f556419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:39.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.449+0000 7f5569c00640 1 --2- 172.21.15.16:0/576342440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5564103690 0x7f556419a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:39.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.449+0000 7f5569c00640 1 -- 172.21.15.16:0/576342440 >> 172.21.15.16:0/576342440 conn(0x7f55640ff0d0 msgr2=0x7f55641014f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:39.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.449+0000 7f5569c00640 1 -- 172.21.15.16:0/576342440 shutdown_connections 2024-08-31T06:06:39.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.449+0000 7f5569c00640 1 -- 172.21.15.16:0/576342440 wait complete. 2024-08-31T06:06:39.477 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-31T06:06:39.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.702+0000 7fa9dee00640 1 -- 172.21.15.16:0/3120030902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9d8103690 msgr2=0x7fa9d8103aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:39.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.702+0000 7fa9dee00640 1 --2- 172.21.15.16:0/3120030902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9d8103690 0x7fa9d8103aa0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc009900 tx=0x7fa9cc02a410 comp rx=0 tx=0).stop 2024-08-31T06:06:39.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.702+0000 7fa9dee00640 1 -- 172.21.15.16:0/3120030902 shutdown_connections 2024-08-31T06:06:39.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.702+0000 7fa9dee00640 1 --2- 172.21.15.16:0/3120030902 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9d8109910 0x7fa9d8110140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:39.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.702+0000 7fa9dee00640 1 --2- 172.21.15.16:0/3120030902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9d8108fc0 0x7fa9d81093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:39.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.702+0000 7fa9dee00640 1 --2- 172.21.15.16:0/3120030902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9d8103690 0x7fa9d8103aa0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:39.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.702+0000 7fa9dee00640 1 -- 172.21.15.16:0/3120030902 >> 172.21.15.16:0/3120030902 conn(0x7fa9d80ff0f0 msgr2=0x7fa9d8101540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:39.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.702+0000 7fa9dee00640 1 -- 172.21.15.16:0/3120030902 shutdown_connections 2024-08-31T06:06:39.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.702+0000 7fa9dee00640 1 -- 172.21.15.16:0/3120030902 wait complete. 2024-08-31T06:06:39.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.703+0000 7fa9dee00640 1 Processor -- start 2024-08-31T06:06:39.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.704+0000 7fa9dee00640 1 -- start start 2024-08-31T06:06:39.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.704+0000 7fa9dee00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9d8103690 0x7fa9d819c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:39.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.704+0000 7fa9dee00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9d8108fc0 0x7fa9d819c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:39.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.704+0000 7fa9dee00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9d8109910 0x7fa9d81a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:39.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.704+0000 7fa9dee00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d81135e0 con 0x7fa9d8109910 2024-08-31T06:06:39.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.704+0000 7fa9dee00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d8113460 con 0x7fa9d8103690 2024-08-31T06:06:39.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.704+0000 7fa9dee00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d8113760 con 0x7fa9d8108fc0 2024-08-31T06:06:39.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.704+0000 7fa9de400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9d8109910 0x7fa9d81a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:39.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.704+0000 7fa9de400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9d8109910 0x7fa9d81a1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50776/0 (socket says 172.21.15.16:50776) 2024-08-31T06:06:39.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.704+0000 7fa9de400640 1 -- 172.21.15.16:0/3120206772 learned_addr learned my addr 172.21.15.16:0/3120206772 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:39.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.704+0000 7fa9dd000640 1 --2- 172.21.15.16:0/3120206772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9d8108fc0 0x7fa9d819c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:39.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.704+0000 7fa9dda00640 1 --2- 172.21.15.16:0/3120206772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9d8103690 0x7fa9d819c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:39.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.705+0000 7fa9de400640 1 -- 172.21.15.16:0/3120206772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9d8108fc0 msgr2=0x7fa9d819c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:39.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.705+0000 7fa9de400640 1 --2- 172.21.15.16:0/3120206772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9d8108fc0 0x7fa9d819c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:39.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.705+0000 7fa9de400640 1 -- 172.21.15.16:0/3120206772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9d8103690 msgr2=0x7fa9d819c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:39.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.705+0000 7fa9de400640 1 --2- 172.21.15.16:0/3120206772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9d8103690 0x7fa9d819c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:39.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.705+0000 7fa9de400640 1 -- 172.21.15.16:0/3120206772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d81a2150 con 0x7fa9d8109910 2024-08-31T06:06:39.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.705+0000 7fa9de400640 1 --2- 172.21.15.16:0/3120206772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9d8109910 0x7fa9d81a1a50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa9d000eb10 tx=0x7fa9d000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:39.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.705+0000 7fa9d6000640 1 -- 172.21.15.16:0/3120206772 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d0004b20 con 0x7fa9d8109910 2024-08-31T06:06:39.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.705+0000 7fa9d6000640 1 -- 172.21.15.16:0/3120206772 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9d0004cc0 con 0x7fa9d8109910 2024-08-31T06:06:39.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.705+0000 7fa9dee00640 1 -- 172.21.15.16:0/3120206772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9d81a2440 con 0x7fa9d8109910 2024-08-31T06:06:39.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.705+0000 7fa9d6000640 1 -- 172.21.15.16:0/3120206772 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d0010670 con 0x7fa9d8109910 2024-08-31T06:06:39.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.705+0000 7fa9dee00640 1 -- 172.21.15.16:0/3120206772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9d81a8bd0 con 0x7fa9d8109910 2024-08-31T06:06:39.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.706+0000 7fa9dee00640 1 -- 172.21.15.16:0/3120206772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9d8103aa0 con 0x7fa9d8109910 2024-08-31T06:06:39.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.709+0000 7fa9d6000640 1 -- 172.21.15.16:0/3120206772 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7fa9d0005020 con 0x7fa9d8109910 2024-08-31T06:06:39.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.710+0000 7fa9d6000640 1 --2- 172.21.15.16:0/3120206772 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa9c00b4b80 0x7fa9c00b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:39.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.710+0000 7fa9d6000640 1 -- 172.21.15.16:0/3120206772 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7fa9d00e5840 con 0x7fa9d8109910 2024-08-31T06:06:39.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.710+0000 7fa9dda00640 1 --2- 172.21.15.16:0/3120206772 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa9c00b4b80 0x7fa9c00b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:39.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.711+0000 7fa9dda00640 1 --2- 172.21.15.16:0/3120206772 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa9c00b4b80 0x7fa9c00b7090 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc031440 tx=0x7fa9cc009560 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:39.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.714+0000 7fa9d6000640 1 -- 172.21.15.16:0/3120206772 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fa9d00ac850 con 0x7fa9d8109910 2024-08-31T06:06:39.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:39 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/2010000129' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-08-31T06:06:39.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:39 smithi155 ceph-mon[21742]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:06:39.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:39 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/828528646' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T06:06:39.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:39 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/576342440' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-31T06:06:39.915 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:39 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/2010000129' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-08-31T06:06:39.915 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:39 smithi016 ceph-mon[17194]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:06:39.915 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:39 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/828528646' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T06:06:39.915 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:39 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/576342440' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-31T06:06:39.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.913+0000 7fa9dee00640 1 -- 172.21.15.16:0/3120206772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa9d81a8ed0 con 0x7fa9d8109910 2024-08-31T06:06:39.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.914+0000 7fa9d6000640 1 -- 172.21.15.16:0/3120206772 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) ==== 104+0+11 (secure 0 0 0) 0x7fa9d00ac150 con 0x7fa9d8109910 2024-08-31T06:06:39.916 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-08-31T06:06:39.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.916+0000 7fa9dee00640 1 -- 172.21.15.16:0/3120206772 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa9c00b4b80 msgr2=0x7fa9c00b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:39.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.916+0000 7fa9dee00640 1 --2- 172.21.15.16:0/3120206772 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa9c00b4b80 0x7fa9c00b7090 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc031440 tx=0x7fa9cc009560 comp rx=0 tx=0).stop 2024-08-31T06:06:39.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.916+0000 7fa9dee00640 1 -- 172.21.15.16:0/3120206772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9d8109910 msgr2=0x7fa9d81a1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:39.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.916+0000 7fa9dee00640 1 --2- 172.21.15.16:0/3120206772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9d8109910 0x7fa9d81a1a50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa9d000eb10 tx=0x7fa9d000c2d0 comp rx=0 tx=0).stop 2024-08-31T06:06:39.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.917+0000 7fa9dee00640 1 -- 172.21.15.16:0/3120206772 shutdown_connections 2024-08-31T06:06:39.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.917+0000 7fa9dee00640 1 --2- 172.21.15.16:0/3120206772 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa9c00b4b80 0x7fa9c00b7090 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:39.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.917+0000 7fa9dee00640 1 --2- 172.21.15.16:0/3120206772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9d8108fc0 0x7fa9d819c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:39.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.917+0000 7fa9dee00640 1 --2- 172.21.15.16:0/3120206772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9d8109910 0x7fa9d81a1a50 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:39.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.917+0000 7fa9dee00640 1 --2- 172.21.15.16:0/3120206772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9d8103690 0x7fa9d819c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:39.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.917+0000 7fa9dee00640 1 -- 172.21.15.16:0/3120206772 >> 172.21.15.16:0/3120206772 conn(0x7fa9d80ff0f0 msgr2=0x7fa9d81086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:39.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.917+0000 7fa9dee00640 1 -- 172.21.15.16:0/3120206772 shutdown_connections 2024-08-31T06:06:39.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:39.917+0000 7fa9dee00640 1 -- 172.21.15.16:0/3120206772 wait complete. 2024-08-31T06:06:39.943 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-31T06:06:39.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:39 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/2010000129' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-08-31T06:06:39.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:39 smithi184 ceph-mon[22331]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:06:39.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:39 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/828528646' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T06:06:39.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:39 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/576342440' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-31T06:06:40.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.168+0000 7eff4ce00640 1 -- 172.21.15.16:0/2833394476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff48108fc0 msgr2=0x7eff481093d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:40.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.168+0000 7eff4ce00640 1 --2- 172.21.15.16:0/2833394476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff48108fc0 0x7eff481093d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7eff30009900 tx=0x7eff3002a3f0 comp rx=0 tx=0).stop 2024-08-31T06:06:40.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.168+0000 7eff4ce00640 1 -- 172.21.15.16:0/2833394476 shutdown_connections 2024-08-31T06:06:40.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.168+0000 7eff4ce00640 1 --2- 172.21.15.16:0/2833394476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff48109910 0x7eff48110140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:40.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.168+0000 7eff4ce00640 1 --2- 172.21.15.16:0/2833394476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff48108fc0 0x7eff481093d0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:40.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.168+0000 7eff4ce00640 1 --2- 172.21.15.16:0/2833394476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff48103690 0x7eff48103aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:40.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.168+0000 7eff4ce00640 1 -- 172.21.15.16:0/2833394476 >> 172.21.15.16:0/2833394476 conn(0x7eff480ff0f0 msgr2=0x7eff48101540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:40.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.168+0000 7eff4ce00640 1 -- 172.21.15.16:0/2833394476 shutdown_connections 2024-08-31T06:06:40.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.168+0000 7eff4ce00640 1 -- 172.21.15.16:0/2833394476 wait complete. 2024-08-31T06:06:40.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.169+0000 7eff4ce00640 1 Processor -- start 2024-08-31T06:06:40.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.170+0000 7eff4ce00640 1 -- start start 2024-08-31T06:06:40.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.170+0000 7eff4ce00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff48103690 0x7eff4819c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:40.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.170+0000 7eff4ce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff48108fc0 0x7eff4819c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:40.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.170+0000 7eff4ce00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff48109910 0x7eff481a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:40.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.170+0000 7eff4ce00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7eff481135f0 con 0x7eff48108fc0 2024-08-31T06:06:40.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.170+0000 7eff4ce00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7eff48113470 con 0x7eff48109910 2024-08-31T06:06:40.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.170+0000 7eff4ce00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7eff48113770 con 0x7eff48103690 2024-08-31T06:06:40.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.170+0000 7eff3fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff48108fc0 0x7eff4819c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:40.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.170+0000 7eff47e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff48109910 0x7eff481a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:40.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.170+0000 7eff3fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff48108fc0 0x7eff4819c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50802/0 (socket says 172.21.15.16:50802) 2024-08-31T06:06:40.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.170+0000 7eff3fe00640 1 -- 172.21.15.16:0/4035377529 learned_addr learned my addr 172.21.15.16:0/4035377529 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:40.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.170+0000 7eff47400640 1 --2- 172.21.15.16:0/4035377529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff48103690 0x7eff4819c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:40.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.171+0000 7eff47e00640 1 -- 172.21.15.16:0/4035377529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff48103690 msgr2=0x7eff4819c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:40.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.171+0000 7eff47e00640 1 --2- 172.21.15.16:0/4035377529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff48103690 0x7eff4819c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:40.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.171+0000 7eff47e00640 1 -- 172.21.15.16:0/4035377529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff48108fc0 msgr2=0x7eff4819c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:40.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.171+0000 7eff47e00640 1 --2- 172.21.15.16:0/4035377529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff48108fc0 0x7eff4819c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:40.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.171+0000 7eff47e00640 1 -- 172.21.15.16:0/4035377529 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff481a2140 con 0x7eff48109910 2024-08-31T06:06:40.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.171+0000 7eff47e00640 1 --2- 172.21.15.16:0/4035377529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff48109910 0x7eff481a1a40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7eff40009fd0 tx=0x7eff4000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.171+0000 7eff44c00640 1 -- 172.21.15.16:0/4035377529 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4000ce20 con 0x7eff48109910 2024-08-31T06:06:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.171+0000 7eff4ce00640 1 -- 172.21.15.16:0/4035377529 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff481a2430 con 0x7eff48109910 2024-08-31T06:06:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.171+0000 7eff44c00640 1 -- 172.21.15.16:0/4035377529 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff40005020 con 0x7eff48109910 2024-08-31T06:06:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.171+0000 7eff44c00640 1 -- 172.21.15.16:0/4035377529 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff40010690 con 0x7eff48109910 2024-08-31T06:06:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.172+0000 7eff4ce00640 1 -- 172.21.15.16:0/4035377529 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff481a8bc0 con 0x7eff48109910 2024-08-31T06:06:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.172+0000 7eff4ce00640 1 -- 172.21.15.16:0/4035377529 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff48103aa0 con 0x7eff48109910 2024-08-31T06:06:40.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.175+0000 7eff44c00640 1 -- 172.21.15.16:0/4035377529 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7eff400054d0 con 0x7eff48109910 2024-08-31T06:06:40.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.177+0000 7eff44c00640 1 --2- 172.21.15.16:0/4035377529 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7eff2c0b4b80 0x7eff2c0b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:40.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.177+0000 7eff44c00640 1 -- 172.21.15.16:0/4035377529 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7eff400e5940 con 0x7eff48109910 2024-08-31T06:06:40.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.177+0000 7eff47400640 1 --2- 172.21.15.16:0/4035377529 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7eff2c0b4b80 0x7eff2c0b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:40.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.177+0000 7eff47400640 1 --2- 172.21.15.16:0/4035377529 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7eff2c0b4b80 0x7eff2c0b7090 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7eff38009bd0 tx=0x7eff38005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:40.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.179+0000 7eff44c00640 1 -- 172.21.15.16:0/4035377529 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7eff400ac950 con 0x7eff48109910 2024-08-31T06:06:40.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.374+0000 7eff4ce00640 1 -- 172.21.15.16:0/4035377529 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7eff481a8ea0 con 0x7eff48109910 2024-08-31T06:06:40.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.375+0000 7eff44c00640 1 -- 172.21.15.16:0/4035377529 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) ==== 100+0+11 (secure 0 0 0) 0x7eff400ac250 con 0x7eff48109910 2024-08-31T06:06:40.377 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-08-31T06:06:40.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.377+0000 7eff4ce00640 1 -- 172.21.15.16:0/4035377529 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7eff2c0b4b80 msgr2=0x7eff2c0b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:40.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.377+0000 7eff4ce00640 1 --2- 172.21.15.16:0/4035377529 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7eff2c0b4b80 0x7eff2c0b7090 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7eff38009bd0 tx=0x7eff38005f90 comp rx=0 tx=0).stop 2024-08-31T06:06:40.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.377+0000 7eff4ce00640 1 -- 172.21.15.16:0/4035377529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff48109910 msgr2=0x7eff481a1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:40.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.377+0000 7eff4ce00640 1 --2- 172.21.15.16:0/4035377529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff48109910 0x7eff481a1a40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7eff40009fd0 tx=0x7eff4000ef10 comp rx=0 tx=0).stop 2024-08-31T06:06:40.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.377+0000 7eff4ce00640 1 -- 172.21.15.16:0/4035377529 shutdown_connections 2024-08-31T06:06:40.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.377+0000 7eff4ce00640 1 --2- 172.21.15.16:0/4035377529 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7eff2c0b4b80 0x7eff2c0b7090 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:40.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.377+0000 7eff4ce00640 1 --2- 172.21.15.16:0/4035377529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff48109910 0x7eff481a1a40 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:40.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.377+0000 7eff4ce00640 1 --2- 172.21.15.16:0/4035377529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff48108fc0 0x7eff4819c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:40.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.377+0000 7eff4ce00640 1 --2- 172.21.15.16:0/4035377529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff48103690 0x7eff4819c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:40.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.377+0000 7eff4ce00640 1 -- 172.21.15.16:0/4035377529 >> 172.21.15.16:0/4035377529 conn(0x7eff480ff0f0 msgr2=0x7eff48108630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:40.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.377+0000 7eff4ce00640 1 -- 172.21.15.16:0/4035377529 shutdown_connections 2024-08-31T06:06:40.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:40.377+0000 7eff4ce00640 1 -- 172.21.15.16:0/4035377529 wait complete. 2024-08-31T06:06:40.404 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-08-31T06:06:40.405 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:06:40.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:40 smithi016 ceph-mon[17194]: pgmap v346: 97 pgs: 97 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-08-31T06:06:40.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:40 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/3120206772' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-08-31T06:06:40.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:40 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/4035377529' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-08-31T06:06:40.787 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:06:40.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:40 smithi155 ceph-mon[21742]: pgmap v346: 97 pgs: 97 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-08-31T06:06:40.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:40 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/3120206772' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-08-31T06:06:40.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:40 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/4035377529' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-08-31T06:06:40.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:40 smithi184 ceph-mon[22331]: pgmap v346: 97 pgs: 97 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-08-31T06:06:40.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:40 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/3120206772' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-08-31T06:06:40.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:40 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/4035377529' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-08-31T06:06:41.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.445+0000 7f0adfa00640 1 -- 172.21.15.16:0/664692755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ad8109b90 msgr2=0x7f0ad81103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:41.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.445+0000 7f0adfa00640 1 --2- 172.21.15.16:0/664692755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ad8109b90 0x7f0ad81103c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0009ca0 tx=0x7f0ad002a8c0 comp rx=0 tx=0).stop 2024-08-31T06:06:41.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.445+0000 7f0adfa00640 1 -- 172.21.15.16:0/664692755 shutdown_connections 2024-08-31T06:06:41.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.445+0000 7f0adfa00640 1 --2- 172.21.15.16:0/664692755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ad8109b90 0x7f0ad81103c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:41.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.445+0000 7f0adfa00640 1 --2- 172.21.15.16:0/664692755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ad8109240 0x7f0ad8109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:41.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.445+0000 7f0adfa00640 1 --2- 172.21.15.16:0/664692755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ad8103900 0x7f0ad8103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:41.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.445+0000 7f0adfa00640 1 -- 172.21.15.16:0/664692755 >> 172.21.15.16:0/664692755 conn(0x7f0ad807b5f0 msgr2=0x7f0ad807ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:41.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.445+0000 7f0adfa00640 1 -- 172.21.15.16:0/664692755 shutdown_connections 2024-08-31T06:06:41.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.445+0000 7f0adfa00640 1 -- 172.21.15.16:0/664692755 wait complete. 2024-08-31T06:06:41.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.446+0000 7f0adfa00640 1 Processor -- start 2024-08-31T06:06:41.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.446+0000 7f0adfa00640 1 -- start start 2024-08-31T06:06:41.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.447+0000 7f0adfa00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ad8103900 0x7f0ad819a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:41.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.447+0000 7f0adfa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ad8109240 0x7f0ad819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:41.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.447+0000 7f0adfa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ad8109b90 0x7f0ad819fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:41.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.447+0000 7f0adfa00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad81138a0 con 0x7f0ad8109240 2024-08-31T06:06:41.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.447+0000 7f0adfa00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad8113720 con 0x7f0ad8109b90 2024-08-31T06:06:41.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.447+0000 7f0adfa00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad8113a20 con 0x7f0ad8103900 2024-08-31T06:06:41.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.447+0000 7f0ad7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ad8109240 0x7f0ad819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:41.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.447+0000 7f0add000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ad8103900 0x7f0ad819a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:41.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.447+0000 7f0ad7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ad8109240 0x7f0ad819a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50830/0 (socket says 172.21.15.16:50830) 2024-08-31T06:06:41.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.447+0000 7f0ad7e00640 1 -- 172.21.15.16:0/1651087121 learned_addr learned my addr 172.21.15.16:0/1651087121 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:41.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.447+0000 7f0adda00640 1 --2- 172.21.15.16:0/1651087121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ad8109b90 0x7f0ad819fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:41.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.448+0000 7f0add000640 1 -- 172.21.15.16:0/1651087121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ad8109b90 msgr2=0x7f0ad819fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:41.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.448+0000 7f0add000640 1 --2- 172.21.15.16:0/1651087121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ad8109b90 0x7f0ad819fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.448+0000 7f0add000640 1 -- 172.21.15.16:0/1651087121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ad8109240 msgr2=0x7f0ad819a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.448+0000 7f0add000640 1 --2- 172.21.15.16:0/1651087121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ad8109240 0x7f0ad819a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.448+0000 7f0add000640 1 -- 172.21.15.16:0/1651087121 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ad81a0140 con 0x7f0ad8103900 2024-08-31T06:06:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.448+0000 7f0ad7e00640 1 --2- 172.21.15.16:0/1651087121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ad8109240 0x7f0ad819a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:06:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.448+0000 7f0add000640 1 --2- 172.21.15.16:0/1651087121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ad8103900 0x7f0ad819a1d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0acc00d6d0 tx=0x7f0acc00db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.448+0000 7f0ad5600640 1 -- 172.21.15.16:0/1651087121 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0acc017070 con 0x7f0ad8103900 2024-08-31T06:06:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.448+0000 7f0adfa00640 1 -- 172.21.15.16:0/1651087121 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ad81a0430 con 0x7f0ad8103900 2024-08-31T06:06:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.449+0000 7f0ad5600640 1 -- 172.21.15.16:0/1651087121 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0acc009820 con 0x7f0ad8103900 2024-08-31T06:06:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.449+0000 7f0ad5600640 1 -- 172.21.15.16:0/1651087121 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0acc004c40 con 0x7f0ad8103900 2024-08-31T06:06:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.449+0000 7f0adfa00640 1 -- 172.21.15.16:0/1651087121 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ad81a6b90 con 0x7f0ad8103900 2024-08-31T06:06:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.450+0000 7f0adfa00640 1 -- 172.21.15.16:0/1651087121 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ad8103d10 con 0x7f0ad8103900 2024-08-31T06:06:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.453+0000 7f0ad5600640 1 -- 172.21.15.16:0/1651087121 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7f0acc005220 con 0x7f0ad8103900 2024-08-31T06:06:41.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.454+0000 7f0ad5600640 1 --2- 172.21.15.16:0/1651087121 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f0ac00b4b80 0x7f0ac00b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:41.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.455+0000 7f0ad5600640 1 -- 172.21.15.16:0/1651087121 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f0acc0e5100 con 0x7f0ad8103900 2024-08-31T06:06:41.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.455+0000 7f0ad7e00640 1 --2- 172.21.15.16:0/1651087121 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f0ac00b4b80 0x7f0ac00b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:41.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.456+0000 7f0ad7e00640 1 --2- 172.21.15.16:0/1651087121 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f0ac00b4b80 0x7f0ac00b7090 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8108e00 tx=0x7f0ac4005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:41.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.456+0000 7f0ad5600640 1 -- 172.21.15.16:0/1651087121 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f0acc0abac0 con 0x7f0ad8103900 2024-08-31T06:06:41.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.686+0000 7f0adfa00640 1 -- 172.21.15.16:0/1651087121 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ad819b300 con 0x7f0ad8103900 2024-08-31T06:06:41.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.688+0000 7f0ad5600640 1 -- 172.21.15.16:0/1651087121 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11917 (secure 0 0 0) 0x7f0acc0b0280 con 0x7f0ad8103900 2024-08-31T06:06:41.690 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:06:41.691 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-08-31T06:06:38:485237+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:06:37.778867+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24698":{"gid":24698,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1089710405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1089710405},{"type":"v1","addr":"172.21.15.155:6837","nonce":1089710405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24737,24724,24712]},"id":1}]} 2024-08-31T06:06:41.691 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-08-31T06:06:41.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.690+0000 7f0adfa00640 1 -- 172.21.15.16:0/1651087121 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f0ac00b4b80 msgr2=0x7f0ac00b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:41.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.690+0000 7f0adfa00640 1 --2- 172.21.15.16:0/1651087121 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f0ac00b4b80 0x7f0ac00b7090 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8108e00 tx=0x7f0ac4005f90 comp rx=0 tx=0).stop 2024-08-31T06:06:41.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.691+0000 7f0adfa00640 1 -- 172.21.15.16:0/1651087121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ad8103900 msgr2=0x7f0ad819a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:41.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.691+0000 7f0adfa00640 1 --2- 172.21.15.16:0/1651087121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ad8103900 0x7f0ad819a1d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0acc00d6d0 tx=0x7f0acc00db90 comp rx=0 tx=0).stop 2024-08-31T06:06:41.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.691+0000 7f0adfa00640 1 -- 172.21.15.16:0/1651087121 shutdown_connections 2024-08-31T06:06:41.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.691+0000 7f0adfa00640 1 --2- 172.21.15.16:0/1651087121 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f0ac00b4b80 0x7f0ac00b7090 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:41.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.691+0000 7f0adfa00640 1 --2- 172.21.15.16:0/1651087121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ad8109b90 0x7f0ad819fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:41.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.691+0000 7f0adfa00640 1 --2- 172.21.15.16:0/1651087121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ad8109240 0x7f0ad819a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:41.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.691+0000 7f0adfa00640 1 --2- 172.21.15.16:0/1651087121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ad8103900 0x7f0ad819a1d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:41.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.691+0000 7f0adfa00640 1 -- 172.21.15.16:0/1651087121 >> 172.21.15.16:0/1651087121 conn(0x7f0ad807b5f0 msgr2=0x7f0ad8107230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:41.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.691+0000 7f0adfa00640 1 -- 172.21.15.16:0/1651087121 shutdown_connections 2024-08-31T06:06:41.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:41.691+0000 7f0adfa00640 1 -- 172.21.15.16:0/1651087121 wait complete. 2024-08-31T06:06:42.427 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-08-31T06:06:42.427 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-08-31T06:06:42.427 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-31T06:06:42.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.650+0000 7f666f800640 1 -- 172.21.15.16:0/4089775544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6668105a70 msgr2=0x7f6668105e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:42.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.650+0000 7f666f800640 1 --2- 172.21.15.16:0/4089775544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6668105a70 0x7f6668105e80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f665c009900 tx=0x7f665c02a3f0 comp rx=0 tx=0).stop 2024-08-31T06:06:42.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.650+0000 7f666f800640 1 -- 172.21.15.16:0/4089775544 shutdown_connections 2024-08-31T06:06:42.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.650+0000 7f666f800640 1 --2- 172.21.15.16:0/4089775544 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f666810b3a0 0x7f666810b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:42.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.650+0000 7f666f800640 1 --2- 172.21.15.16:0/4089775544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f666810bcf0 0x7f6668112520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:42.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.650+0000 7f666f800640 1 --2- 172.21.15.16:0/4089775544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6668105a70 0x7f6668105e80 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:42.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.650+0000 7f666f800640 1 -- 172.21.15.16:0/4089775544 >> 172.21.15.16:0/4089775544 conn(0x7f6668101490 msgr2=0x7f66681038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:42.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.650+0000 7f666f800640 1 -- 172.21.15.16:0/4089775544 shutdown_connections 2024-08-31T06:06:42.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.650+0000 7f666f800640 1 -- 172.21.15.16:0/4089775544 wait complete. 2024-08-31T06:06:42.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.651+0000 7f666f800640 1 Processor -- start 2024-08-31T06:06:42.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.651+0000 7f666f800640 1 -- start start 2024-08-31T06:06:42.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.652+0000 7f666f800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6668105a70 0x7f666819c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:42.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.652+0000 7f666f800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f666810b3a0 0x7f666819c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:42.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.652+0000 7f666f800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f666810bcf0 0x7f66681a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:42.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.652+0000 7f666f800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f66681159b0 con 0x7f666810bcf0 2024-08-31T06:06:42.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.652+0000 7f666f800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6668115830 con 0x7f6668105a70 2024-08-31T06:06:42.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.652+0000 7f666f800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6668115b30 con 0x7f666810b3a0 2024-08-31T06:06:42.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.652+0000 7f666d800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f666810bcf0 0x7f66681a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:42.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.652+0000 7f666d800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f666810bcf0 0x7f66681a1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50844/0 (socket says 172.21.15.16:50844) 2024-08-31T06:06:42.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.652+0000 7f666d800640 1 -- 172.21.15.16:0/756541139 learned_addr learned my addr 172.21.15.16:0/756541139 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:42.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.652+0000 7f666ce00640 1 --2- 172.21.15.16:0/756541139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6668105a70 0x7f666819c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:42.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.652+0000 7f6667e00640 1 --2- 172.21.15.16:0/756541139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f666810b3a0 0x7f666819c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:42.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.652+0000 7f666d800640 1 -- 172.21.15.16:0/756541139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f666810b3a0 msgr2=0x7f666819c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:42.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.652+0000 7f666d800640 1 --2- 172.21.15.16:0/756541139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f666810b3a0 0x7f666819c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:42.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.652+0000 7f666d800640 1 -- 172.21.15.16:0/756541139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6668105a70 msgr2=0x7f666819c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:42.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.652+0000 7f666d800640 1 --2- 172.21.15.16:0/756541139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6668105a70 0x7f666819c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:42.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.652+0000 7f666d800640 1 -- 172.21.15.16:0/756541139 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66681a2300 con 0x7f666810bcf0 2024-08-31T06:06:42.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.653+0000 7f666d800640 1 --2- 172.21.15.16:0/756541139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f666810bcf0 0x7f66681a1c00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f666000eb10 tx=0x7f666000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:42.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.653+0000 7f6665600640 1 -- 172.21.15.16:0/756541139 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6660004b20 con 0x7f666810bcf0 2024-08-31T06:06:42.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.653+0000 7f6665600640 1 -- 172.21.15.16:0/756541139 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6660004cc0 con 0x7f666810bcf0 2024-08-31T06:06:42.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.653+0000 7f666f800640 1 -- 172.21.15.16:0/756541139 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66681a25f0 con 0x7f666810bcf0 2024-08-31T06:06:42.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.653+0000 7f6665600640 1 -- 172.21.15.16:0/756541139 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6660010670 con 0x7f666810bcf0 2024-08-31T06:06:42.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.653+0000 7f666f800640 1 -- 172.21.15.16:0/756541139 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66681a8d80 con 0x7f666810bcf0 2024-08-31T06:06:42.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.654+0000 7f666f800640 1 -- 172.21.15.16:0/756541139 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6668106ce0 con 0x7f666810bcf0 2024-08-31T06:06:42.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.656+0000 7f6665600640 1 -- 172.21.15.16:0/756541139 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7f6660005020 con 0x7f666810bcf0 2024-08-31T06:06:42.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.659+0000 7f6665600640 1 --2- 172.21.15.16:0/756541139 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f66500b4b80 0x7f66500b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:42.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.659+0000 7f6665600640 1 -- 172.21.15.16:0/756541139 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f66600e58a0 con 0x7f666810bcf0 2024-08-31T06:06:42.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.659+0000 7f666ce00640 1 --2- 172.21.15.16:0/756541139 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f66500b4b80 0x7f66500b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:42.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.660+0000 7f666ce00640 1 --2- 172.21.15.16:0/756541139 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f66500b4b80 0x7f66500b7090 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f665c031440 tx=0x7f665c009560 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:42.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:42 smithi016 ceph-mon[17194]: pgmap v347: 97 pgs: 97 active+clean; 645 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 7 op/s 2024-08-31T06:06:42.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:42 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/1651087121' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:06:42.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.664+0000 7f6665600640 1 -- 172.21.15.16:0/756541139 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f66600ac8b0 con 0x7f666810bcf0 2024-08-31T06:06:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.855+0000 7f666f800640 1 -- 172.21.15.16:0/756541139 --> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f66680635a0 con 0x7f66500b4b80 2024-08-31T06:06:42.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.860+0000 7f6665600640 1 -- 172.21.15.16:0/756541139 <== mgr.14220 v2:172.21.15.16:6800/4279751337 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259063 (secure 0 0 0) 0x7f66680635a0 con 0x7f66500b4b80 2024-08-31T06:06:42.866 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:06:42.867 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-08-31T06:06:42.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.864+0000 7f666f800640 1 -- 172.21.15.16:0/756541139 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f66500b4b80 msgr2=0x7f66500b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:42.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.864+0000 7f666f800640 1 --2- 172.21.15.16:0/756541139 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f66500b4b80 0x7f66500b7090 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f665c031440 tx=0x7f665c009560 comp rx=0 tx=0).stop 2024-08-31T06:06:42.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.864+0000 7f666f800640 1 -- 172.21.15.16:0/756541139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f666810bcf0 msgr2=0x7f66681a1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:42.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.864+0000 7f666f800640 1 --2- 172.21.15.16:0/756541139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f666810bcf0 0x7f66681a1c00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f666000eb10 tx=0x7f666000c2d0 comp rx=0 tx=0).stop 2024-08-31T06:06:42.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.864+0000 7f666f800640 1 -- 172.21.15.16:0/756541139 shutdown_connections 2024-08-31T06:06:42.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.864+0000 7f666f800640 1 --2- 172.21.15.16:0/756541139 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f66500b4b80 0x7f66500b7090 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:42.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.864+0000 7f666f800640 1 --2- 172.21.15.16:0/756541139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f666810b3a0 0x7f666819c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:42.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.864+0000 7f666f800640 1 --2- 172.21.15.16:0/756541139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f666810bcf0 0x7f66681a1c00 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:42.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.864+0000 7f666f800640 1 --2- 172.21.15.16:0/756541139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6668105a70 0x7f666819c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:42.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.864+0000 7f666f800640 1 -- 172.21.15.16:0/756541139 >> 172.21.15.16:0/756541139 conn(0x7f6668101490 msgr2=0x7f66681038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:42.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.864+0000 7f666f800640 1 -- 172.21.15.16:0/756541139 shutdown_connections 2024-08-31T06:06:42.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:42.864+0000 7f666f800640 1 -- 172.21.15.16:0/756541139 wait complete. 2024-08-31T06:06:42.871 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:42 smithi155 ceph-mon[21742]: pgmap v347: 97 pgs: 97 active+clean; 645 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 7 op/s 2024-08-31T06:06:42.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:42 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/1651087121' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:06:42.892 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2024-08-31T06:06:41.290709+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660971,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":823,"num_read_kb":518,"num_write":303,"num_write_kb":2278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":2952987,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":745,"ondisk_log_size":745,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372440,"kb_used_data":10712,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124569384,"statfs":{"total":1151940427776,"available":1151559049216,"internally_reserved":0,"allocated":10969088,"data_stored":7207848,"data_compressed":178617,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5684,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":87,"num_read_kb":52,"num_write":4,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003174"},"pg_stats":[{"pgid":"3.3f","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.686021+0000","last_change":"2024-08-31T06:04:25.877318+0000","last_active":"2024-08-31T06:06:23.686021+0000","last_peered":"2024-08-31T06:06:23.686021+0000","last_clean":"2024-08-31T06:06:23.686021+0000","last_became_active":"2024-08-31T06:04:25.876894+0000","last_became_peered":"2024-08-31T06:04:25.876894+0000","last_unstale":"2024-08-31T06:06:23.686021+0000","last_undegraded":"2024-08-31T06:06:23.686021+0000","last_fullsized":"2024-08-31T06:06:23.686021+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:41:09.449343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"80'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.685244+0000","last_change":"2024-08-31T06:04:25.885670+0000","last_active":"2024-08-31T06:06:23.685244+0000","last_peered":"2024-08-31T06:06:23.685244+0000","last_clean":"2024-08-31T06:06:23.685244+0000","last_became_active":"2024-08-31T06:04:25.883818+0000","last_became_peered":"2024-08-31T06:04:25.883818+0000","last_unstale":"2024-08-31T06:06:23.685244+0000","last_undegraded":"2024-08-31T06:06:23.685244+0000","last_fullsized":"2024-08-31T06:06:23.685244+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:40:01.344432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.687803+0000","last_change":"2024-08-31T06:04:25.883375+0000","last_active":"2024-08-31T06:06:23.687803+0000","last_peered":"2024-08-31T06:06:23.687803+0000","last_clean":"2024-08-31T06:06:23.687803+0000","last_became_active":"2024-08-31T06:04:25.881750+0000","last_became_peered":"2024-08-31T06:04:25.881750+0000","last_unstale":"2024-08-31T06:06:23.687803+0000","last_undegraded":"2024-08-31T06:06:23.687803+0000","last_fullsized":"2024-08-31T06:06:23.687803+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:46:46.757363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.676625+0000","last_change":"2024-08-31T06:04:25.882436+0000","last_active":"2024-08-31T06:06:23.676625+0000","last_peered":"2024-08-31T06:06:23.676625+0000","last_clean":"2024-08-31T06:06:23.676625+0000","last_became_active":"2024-08-31T06:04:25.881805+0000","last_became_peered":"2024-08-31T06:04:25.881805+0000","last_unstale":"2024-08-31T06:06:23.676625+0000","last_undegraded":"2024-08-31T06:06:23.676625+0000","last_fullsized":"2024-08-31T06:06:23.676625+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:42:39.793625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"80'4","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.661470+0000","last_change":"2024-08-31T06:04:25.894534+0000","last_active":"2024-08-31T06:06:23.661470+0000","last_peered":"2024-08-31T06:06:23.661470+0000","last_clean":"2024-08-31T06:06:23.661470+0000","last_became_active":"2024-08-31T06:04:25.894116+0000","last_became_peered":"2024-08-31T06:04:25.894116+0000","last_unstale":"2024-08-31T06:06:23.661470+0000","last_undegraded":"2024-08-31T06:06:23.661470+0000","last_fullsized":"2024-08-31T06:06:23.661470+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:02:31.257454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.669411+0000","last_change":"2024-08-31T06:04:25.886182+0000","last_active":"2024-08-31T06:06:23.669411+0000","last_peered":"2024-08-31T06:06:23.669411+0000","last_clean":"2024-08-31T06:06:23.669411+0000","last_became_active":"2024-08-31T06:04:25.885825+0000","last_became_peered":"2024-08-31T06:04:25.885825+0000","last_unstale":"2024-08-31T06:06:23.669411+0000","last_undegraded":"2024-08-31T06:06:23.669411+0000","last_fullsized":"2024-08-31T06:06:23.669411+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:58:12.427931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.682008+0000","last_change":"2024-08-31T06:04:25.877521+0000","last_active":"2024-08-31T06:06:23.682008+0000","last_peered":"2024-08-31T06:06:23.682008+0000","last_clean":"2024-08-31T06:06:23.682008+0000","last_became_active":"2024-08-31T06:04:25.877122+0000","last_became_peered":"2024-08-31T06:04:25.877122+0000","last_unstale":"2024-08-31T06:06:23.682008+0000","last_undegraded":"2024-08-31T06:06:23.682008+0000","last_fullsized":"2024-08-31T06:06:23.682008+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:17:27.406828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.680720+0000","last_change":"2024-08-31T06:04:25.892547+0000","last_active":"2024-08-31T06:06:23.680720+0000","last_peered":"2024-08-31T06:06:23.680720+0000","last_clean":"2024-08-31T06:06:23.680720+0000","last_became_active":"2024-08-31T06:04:25.892090+0000","last_became_peered":"2024-08-31T06:04:25.892090+0000","last_unstale":"2024-08-31T06:06:23.680720+0000","last_undegraded":"2024-08-31T06:06:23.680720+0000","last_fullsized":"2024-08-31T06:06:23.680720+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:28:53.414252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"80'11","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.688048+0000","last_change":"2024-08-31T06:04:25.879952+0000","last_active":"2024-08-31T06:06:23.688048+0000","last_peered":"2024-08-31T06:06:23.688048+0000","last_clean":"2024-08-31T06:06:23.688048+0000","last_became_active":"2024-08-31T06:04:25.879568+0000","last_became_peered":"2024-08-31T06:04:25.879568+0000","last_unstale":"2024-08-31T06:06:23.688048+0000","last_undegraded":"2024-08-31T06:06:23.688048+0000","last_fullsized":"2024-08-31T06:06:23.688048+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:28:04.558054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"80'16","reported_seq":44,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.676096+0000","last_change":"2024-08-31T06:05:25.051173+0000","last_active":"2024-08-31T06:06:23.676096+0000","last_peered":"2024-08-31T06:06:23.676096+0000","last_clean":"2024-08-31T06:06:23.676096+0000","last_became_active":"2024-08-31T06:05:25.050867+0000","last_became_peered":"2024-08-31T06:05:25.050867+0000","last_unstale":"2024-08-31T06:06:23.676096+0000","last_undegraded":"2024-08-31T06:06:23.676096+0000","last_fullsized":"2024-08-31T06:06:23.676096+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:27:09.971497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.686392+0000","last_change":"2024-08-31T06:04:25.883984+0000","last_active":"2024-08-31T06:06:23.686392+0000","last_peered":"2024-08-31T06:06:23.686392+0000","last_clean":"2024-08-31T06:06:23.686392+0000","last_became_active":"2024-08-31T06:04:25.883353+0000","last_became_peered":"2024-08-31T06:04:25.883353+0000","last_unstale":"2024-08-31T06:06:23.686392+0000","last_undegraded":"2024-08-31T06:06:23.686392+0000","last_fullsized":"2024-08-31T06:06:23.686392+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:16:25.178952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.672858+0000","last_change":"2024-08-31T06:04:25.883335+0000","last_active":"2024-08-31T06:06:23.672858+0000","last_peered":"2024-08-31T06:06:23.672858+0000","last_clean":"2024-08-31T06:06:23.672858+0000","last_became_active":"2024-08-31T06:04:25.881488+0000","last_became_peered":"2024-08-31T06:04:25.881488+0000","last_unstale":"2024-08-31T06:06:23.672858+0000","last_undegraded":"2024-08-31T06:06:23.672858+0000","last_fullsized":"2024-08-31T06:06:23.672858+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:57:24.472482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'10","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.678638+0000","last_change":"2024-08-31T06:04:25.891579+0000","last_active":"2024-08-31T06:06:23.678638+0000","last_peered":"2024-08-31T06:06:23.678638+0000","last_clean":"2024-08-31T06:06:23.678638+0000","last_became_active":"2024-08-31T06:04:25.887950+0000","last_became_peered":"2024-08-31T06:04:25.887950+0000","last_unstale":"2024-08-31T06:06:23.678638+0000","last_undegraded":"2024-08-31T06:06:23.678638+0000","last_fullsized":"2024-08-31T06:06:23.678638+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:38:35.284442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.682116+0000","last_change":"2024-08-31T06:04:25.882185+0000","last_active":"2024-08-31T06:06:23.682116+0000","last_peered":"2024-08-31T06:06:23.682116+0000","last_clean":"2024-08-31T06:06:23.682116+0000","last_became_active":"2024-08-31T06:04:25.876499+0000","last_became_peered":"2024-08-31T06:04:25.876499+0000","last_unstale":"2024-08-31T06:06:23.682116+0000","last_undegraded":"2024-08-31T06:06:23.682116+0000","last_fullsized":"2024-08-31T06:06:23.682116+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:40:01.132353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.683752+0000","last_change":"2024-08-31T06:04:25.883624+0000","last_active":"2024-08-31T06:06:23.683752+0000","last_peered":"2024-08-31T06:06:23.683752+0000","last_clean":"2024-08-31T06:06:23.683752+0000","last_became_active":"2024-08-31T06:04:25.878041+0000","last_became_peered":"2024-08-31T06:04:25.878041+0000","last_unstale":"2024-08-31T06:06:23.683752+0000","last_undegraded":"2024-08-31T06:06:23.683752+0000","last_fullsized":"2024-08-31T06:06:23.683752+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:05:21.278321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.687675+0000","last_change":"2024-08-31T06:04:25.882538+0000","last_active":"2024-08-31T06:06:23.687675+0000","last_peered":"2024-08-31T06:06:23.687675+0000","last_clean":"2024-08-31T06:06:23.687675+0000","last_became_active":"2024-08-31T06:04:25.882201+0000","last_became_peered":"2024-08-31T06:04:25.882201+0000","last_unstale":"2024-08-31T06:06:23.687675+0000","last_undegraded":"2024-08-31T06:06:23.687675+0000","last_fullsized":"2024-08-31T06:06:23.687675+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:47:16.880893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.686044+0000","last_change":"2024-08-31T06:04:25.880376+0000","last_active":"2024-08-31T06:06:23.686044+0000","last_peered":"2024-08-31T06:06:23.686044+0000","last_clean":"2024-08-31T06:06:23.686044+0000","last_became_active":"2024-08-31T06:04:25.877668+0000","last_became_peered":"2024-08-31T06:04:25.877668+0000","last_unstale":"2024-08-31T06:06:23.686044+0000","last_undegraded":"2024-08-31T06:06:23.686044+0000","last_fullsized":"2024-08-31T06:06:23.686044+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:55:32.737406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"80'10","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.673276+0000","last_change":"2024-08-31T06:04:25.875431+0000","last_active":"2024-08-31T06:06:23.673276+0000","last_peered":"2024-08-31T06:06:23.673276+0000","last_clean":"2024-08-31T06:06:23.673276+0000","last_became_active":"2024-08-31T06:04:25.874751+0000","last_became_peered":"2024-08-31T06:04:25.874751+0000","last_unstale":"2024-08-31T06:06:23.673276+0000","last_undegraded":"2024-08-31T06:06:23.673276+0000","last_fullsized":"2024-08-31T06:06:23.673276+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:16:57.695415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.675562+0000","last_change":"2024-08-31T06:04:25.895656+0000","last_active":"2024-08-31T06:06:23.675562+0000","last_peered":"2024-08-31T06:06:23.675562+0000","last_clean":"2024-08-31T06:06:23.675562+0000","last_became_active":"2024-08-31T06:04:25.895272+0000","last_became_peered":"2024-08-31T06:04:25.895272+0000","last_unstale":"2024-08-31T06:06:23.675562+0000","last_undegraded":"2024-08-31T06:06:23.675562+0000","last_fullsized":"2024-08-31T06:06:23.675562+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:18:47.276195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.679502+0000","last_change":"2024-08-31T06:04:25.877283+0000","last_active":"2024-08-31T06:06:23.679502+0000","last_peered":"2024-08-31T06:06:23.679502+0000","last_clean":"2024-08-31T06:06:23.679502+0000","last_became_active":"2024-08-31T06:04:25.876846+0000","last_became_peered":"2024-08-31T06:04:25.876846+0000","last_unstale":"2024-08-31T06:06:23.679502+0000","last_undegraded":"2024-08-31T06:06:23.679502+0000","last_fullsized":"2024-08-31T06:06:23.679502+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:54:05.086844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"80'11","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.685368+0000","last_change":"2024-08-31T06:04:25.884891+0000","last_active":"2024-08-31T06:06:23.685368+0000","last_peered":"2024-08-31T06:06:23.685368+0000","last_clean":"2024-08-31T06:06:23.685368+0000","last_became_active":"2024-08-31T06:04:25.884501+0000","last_became_peered":"2024-08-31T06:04:25.884501+0000","last_unstale":"2024-08-31T06:06:23.685368+0000","last_undegraded":"2024-08-31T06:06:23.685368+0000","last_fullsized":"2024-08-31T06:06:23.685368+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:22:54.846102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"80'10","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.691670+0000","last_change":"2024-08-31T06:04:25.877150+0000","last_active":"2024-08-31T06:06:23.691670+0000","last_peered":"2024-08-31T06:06:23.691670+0000","last_clean":"2024-08-31T06:06:23.691670+0000","last_became_active":"2024-08-31T06:04:25.876766+0000","last_became_peered":"2024-08-31T06:04:25.876766+0000","last_unstale":"2024-08-31T06:06:23.691670+0000","last_undegraded":"2024-08-31T06:06:23.691670+0000","last_fullsized":"2024-08-31T06:06:23.691670+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:46:26.724970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.676021+0000","last_change":"2024-08-31T06:04:25.881275+0000","last_active":"2024-08-31T06:06:23.676021+0000","last_peered":"2024-08-31T06:06:23.676021+0000","last_clean":"2024-08-31T06:06:23.676021+0000","last_became_active":"2024-08-31T06:04:25.879877+0000","last_became_peered":"2024-08-31T06:04:25.879877+0000","last_unstale":"2024-08-31T06:06:23.676021+0000","last_undegraded":"2024-08-31T06:06:23.676021+0000","last_fullsized":"2024-08-31T06:06:23.676021+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:07:07.237041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.676080+0000","last_change":"2024-08-31T06:04:25.895750+0000","last_active":"2024-08-31T06:06:23.676080+0000","last_peered":"2024-08-31T06:06:23.676080+0000","last_clean":"2024-08-31T06:06:23.676080+0000","last_became_active":"2024-08-31T06:04:25.895348+0000","last_became_peered":"2024-08-31T06:04:25.895348+0000","last_unstale":"2024-08-31T06:06:23.676080+0000","last_undegraded":"2024-08-31T06:06:23.676080+0000","last_fullsized":"2024-08-31T06:06:23.676080+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:53:10.014808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.670437+0000","last_change":"2024-08-31T06:04:25.895537+0000","last_active":"2024-08-31T06:06:23.670437+0000","last_peered":"2024-08-31T06:06:23.670437+0000","last_clean":"2024-08-31T06:06:23.670437+0000","last_became_active":"2024-08-31T06:04:25.895136+0000","last_became_peered":"2024-08-31T06:04:25.895136+0000","last_unstale":"2024-08-31T06:06:23.670437+0000","last_undegraded":"2024-08-31T06:06:23.670437+0000","last_fullsized":"2024-08-31T06:06:23.670437+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:41:57.554507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"80'15","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.676301+0000","last_change":"2024-08-31T06:04:25.880490+0000","last_active":"2024-08-31T06:06:23.676301+0000","last_peered":"2024-08-31T06:06:23.676301+0000","last_clean":"2024-08-31T06:06:23.676301+0000","last_became_active":"2024-08-31T06:04:25.879269+0000","last_became_peered":"2024-08-31T06:04:25.879269+0000","last_unstale":"2024-08-31T06:06:23.676301+0000","last_undegraded":"2024-08-31T06:06:23.676301+0000","last_fullsized":"2024-08-31T06:06:23.676301+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:05:49.575915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"80'6","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.676526+0000","last_change":"2024-08-31T06:05:25.053714+0000","last_active":"2024-08-31T06:06:23.676526+0000","last_peered":"2024-08-31T06:06:23.676526+0000","last_clean":"2024-08-31T06:06:23.676526+0000","last_became_active":"2024-08-31T06:05:25.052685+0000","last_became_peered":"2024-08-31T06:05:25.052685+0000","last_unstale":"2024-08-31T06:06:23.676526+0000","last_undegraded":"2024-08-31T06:06:23.676526+0000","last_fullsized":"2024-08-31T06:06:23.676526+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:17:31.474601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"80'4","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.684823+0000","last_change":"2024-08-31T06:05:25.057946+0000","last_active":"2024-08-31T06:06:23.684823+0000","last_peered":"2024-08-31T06:06:23.684823+0000","last_clean":"2024-08-31T06:06:23.684823+0000","last_became_active":"2024-08-31T06:05:25.057501+0000","last_became_peered":"2024-08-31T06:05:25.057501+0000","last_unstale":"2024-08-31T06:06:23.684823+0000","last_undegraded":"2024-08-31T06:06:23.684823+0000","last_fullsized":"2024-08-31T06:06:23.684823+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:16:05.721770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.685410+0000","last_change":"2024-08-31T06:04:25.897473+0000","last_active":"2024-08-31T06:06:23.685410+0000","last_peered":"2024-08-31T06:06:23.685410+0000","last_clean":"2024-08-31T06:06:23.685410+0000","last_became_active":"2024-08-31T06:04:25.895194+0000","last_became_peered":"2024-08-31T06:04:25.895194+0000","last_unstale":"2024-08-31T06:06:23.685410+0000","last_undegraded":"2024-08-31T06:06:23.685410+0000","last_fullsized":"2024-08-31T06:06:23.685410+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:43:04.539970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.684923+0000","last_change":"2024-08-31T06:04:25.894402+0000","last_active":"2024-08-31T06:06:23.684923+0000","last_peered":"2024-08-31T06:06:23.684923+0000","last_clean":"2024-08-31T06:06:23.684923+0000","last_became_active":"2024-08-31T06:04:25.893883+0000","last_became_peered":"2024-08-31T06:04:25.893883+0000","last_unstale":"2024-08-31T06:06:23.684923+0000","last_undegraded":"2024-08-31T06:06:23.684923+0000","last_fullsized":"2024-08-31T06:06:23.684923+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:45:53.778473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"80'12","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.681979+0000","last_change":"2024-08-31T06:04:25.881543+0000","last_active":"2024-08-31T06:06:23.681979+0000","last_peered":"2024-08-31T06:06:23.681979+0000","last_clean":"2024-08-31T06:06:23.681979+0000","last_became_active":"2024-08-31T06:04:25.881144+0000","last_became_peered":"2024-08-31T06:04:25.881144+0000","last_unstale":"2024-08-31T06:06:23.681979+0000","last_undegraded":"2024-08-31T06:06:23.681979+0000","last_fullsized":"2024-08-31T06:06:23.681979+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:13:02.282879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.669781+0000","last_change":"2024-08-31T06:04:25.881143+0000","last_active":"2024-08-31T06:06:23.669781+0000","last_peered":"2024-08-31T06:06:23.669781+0000","last_clean":"2024-08-31T06:06:23.669781+0000","last_became_active":"2024-08-31T06:04:25.879718+0000","last_became_peered":"2024-08-31T06:04:25.879718+0000","last_unstale":"2024-08-31T06:06:23.669781+0000","last_undegraded":"2024-08-31T06:06:23.669781+0000","last_fullsized":"2024-08-31T06:06:23.669781+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:32:07.771488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.680189+0000","last_change":"2024-08-31T06:04:25.879493+0000","last_active":"2024-08-31T06:06:23.680189+0000","last_peered":"2024-08-31T06:06:23.680189+0000","last_clean":"2024-08-31T06:06:23.680189+0000","last_became_active":"2024-08-31T06:04:25.879065+0000","last_became_peered":"2024-08-31T06:04:25.879065+0000","last_unstale":"2024-08-31T06:06:23.680189+0000","last_undegraded":"2024-08-31T06:06:23.680189+0000","last_fullsized":"2024-08-31T06:06:23.680189+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:53:43.848269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"80'7","reported_seq":168,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:37.163835+0000","last_change":"2024-08-31T06:05:25.058954+0000","last_active":"2024-08-31T06:06:37.163835+0000","last_peered":"2024-08-31T06:06:37.163835+0000","last_clean":"2024-08-31T06:06:37.163835+0000","last_became_active":"2024-08-31T06:05:25.051143+0000","last_became_peered":"2024-08-31T06:05:25.051143+0000","last_unstale":"2024-08-31T06:06:37.163835+0000","last_undegraded":"2024-08-31T06:06:37.163835+0000","last_fullsized":"2024-08-31T06:06:37.163835+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:38:23.533663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.682132+0000","last_change":"2024-08-31T06:04:25.896354+0000","last_active":"2024-08-31T06:06:23.682132+0000","last_peered":"2024-08-31T06:06:23.682132+0000","last_clean":"2024-08-31T06:06:23.682132+0000","last_became_active":"2024-08-31T06:04:25.896021+0000","last_became_peered":"2024-08-31T06:04:25.896021+0000","last_unstale":"2024-08-31T06:06:23.682132+0000","last_undegraded":"2024-08-31T06:06:23.682132+0000","last_fullsized":"2024-08-31T06:06:23.682132+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:08:40.213238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.047153+0000","last_change":"2024-08-31T06:04:22.844248+0000","last_active":"2024-08-31T06:05:25.047153+0000","last_peered":"2024-08-31T06:05:25.047153+0000","last_clean":"2024-08-31T06:05:25.047153+0000","last_became_active":"2024-08-31T06:04:22.843784+0000","last_became_peered":"2024-08-31T06:04:22.843784+0000","last_unstale":"2024-08-31T06:05:25.047153+0000","last_undegraded":"2024-08-31T06:05:25.047153+0000","last_fullsized":"2024-08-31T06:05:25.047153+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:56:41.422965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.046218+0000","last_change":"2024-08-31T06:04:22.847049+0000","last_active":"2024-08-31T06:05:25.046218+0000","last_peered":"2024-08-31T06:05:25.046218+0000","last_clean":"2024-08-31T06:05:25.046218+0000","last_became_active":"2024-08-31T06:04:22.846282+0000","last_became_peered":"2024-08-31T06:04:22.846282+0000","last_unstale":"2024-08-31T06:05:25.046218+0000","last_undegraded":"2024-08-31T06:05:25.046218+0000","last_fullsized":"2024-08-31T06:05:25.046218+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:35:21.030425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.679457+0000","last_change":"2024-08-31T06:04:25.886698+0000","last_active":"2024-08-31T06:06:23.679457+0000","last_peered":"2024-08-31T06:06:23.679457+0000","last_clean":"2024-08-31T06:06:23.679457+0000","last_became_active":"2024-08-31T06:04:25.886334+0000","last_became_peered":"2024-08-31T06:04:25.886334+0000","last_unstale":"2024-08-31T06:06:23.679457+0000","last_undegraded":"2024-08-31T06:06:23.679457+0000","last_fullsized":"2024-08-31T06:06:23.679457+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:24:43.821446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.044798+0000","last_change":"2024-08-31T06:04:22.845263+0000","last_active":"2024-08-31T06:05:25.044798+0000","last_peered":"2024-08-31T06:05:25.044798+0000","last_clean":"2024-08-31T06:05:25.044798+0000","last_became_active":"2024-08-31T06:04:22.844853+0000","last_became_peered":"2024-08-31T06:04:22.844853+0000","last_unstale":"2024-08-31T06:05:25.044798+0000","last_undegraded":"2024-08-31T06:05:25.044798+0000","last_fullsized":"2024-08-31T06:05:25.044798+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:55:12.561165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"80'11","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.673293+0000","last_change":"2024-08-31T06:04:25.894475+0000","last_active":"2024-08-31T06:06:23.673293+0000","last_peered":"2024-08-31T06:06:23.673293+0000","last_clean":"2024-08-31T06:06:23.673293+0000","last_became_active":"2024-08-31T06:04:25.894121+0000","last_became_peered":"2024-08-31T06:04:25.894121+0000","last_unstale":"2024-08-31T06:06:23.673293+0000","last_undegraded":"2024-08-31T06:06:23.673293+0000","last_fullsized":"2024-08-31T06:06:23.673293+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:40:06.570616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":149,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:38.164972+0000","last_change":"2024-08-31T06:04:22.834430+0000","last_active":"2024-08-31T06:06:38.164972+0000","last_peered":"2024-08-31T06:06:38.164972+0000","last_clean":"2024-08-31T06:06:38.164972+0000","last_became_active":"2024-08-31T06:04:22.834011+0000","last_became_peered":"2024-08-31T06:04:22.834011+0000","last_unstale":"2024-08-31T06:06:38.164972+0000","last_undegraded":"2024-08-31T06:06:38.164972+0000","last_fullsized":"2024-08-31T06:06:38.164972+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:51:16.734234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"80'10","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.689334+0000","last_change":"2024-08-31T06:04:25.877727+0000","last_active":"2024-08-31T06:06:23.689334+0000","last_peered":"2024-08-31T06:06:23.689334+0000","last_clean":"2024-08-31T06:06:23.689334+0000","last_became_active":"2024-08-31T06:04:25.877121+0000","last_became_peered":"2024-08-31T06:04:25.877121+0000","last_unstale":"2024-08-31T06:06:23.689334+0000","last_undegraded":"2024-08-31T06:06:23.689334+0000","last_fullsized":"2024-08-31T06:06:23.689334+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:41:12.247216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048823+0000","last_change":"2024-08-31T06:04:22.846482+0000","last_active":"2024-08-31T06:05:25.048823+0000","last_peered":"2024-08-31T06:05:25.048823+0000","last_clean":"2024-08-31T06:05:25.048823+0000","last_became_active":"2024-08-31T06:04:22.846057+0000","last_became_peered":"2024-08-31T06:04:22.846057+0000","last_unstale":"2024-08-31T06:05:25.048823+0000","last_undegraded":"2024-08-31T06:05:25.048823+0000","last_fullsized":"2024-08-31T06:05:25.048823+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:29:11.861665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.684452+0000","last_change":"2024-08-31T06:04:25.893777+0000","last_active":"2024-08-31T06:06:23.684452+0000","last_peered":"2024-08-31T06:06:23.684452+0000","last_clean":"2024-08-31T06:06:23.684452+0000","last_became_active":"2024-08-31T06:04:25.893014+0000","last_became_peered":"2024-08-31T06:04:25.893014+0000","last_unstale":"2024-08-31T06:06:23.684452+0000","last_undegraded":"2024-08-31T06:06:23.684452+0000","last_fullsized":"2024-08-31T06:06:23.684452+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:04:48.185842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.045472+0000","last_change":"2024-08-31T06:04:22.965938+0000","last_active":"2024-08-31T06:05:25.045472+0000","last_peered":"2024-08-31T06:05:25.045472+0000","last_clean":"2024-08-31T06:05:25.045472+0000","last_became_active":"2024-08-31T06:04:22.965356+0000","last_became_peered":"2024-08-31T06:04:22.965356+0000","last_unstale":"2024-08-31T06:05:25.045472+0000","last_undegraded":"2024-08-31T06:05:25.045472+0000","last_fullsized":"2024-08-31T06:05:25.045472+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:36:58.938462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"80'11","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.691767+0000","last_change":"2024-08-31T06:04:25.881307+0000","last_active":"2024-08-31T06:06:23.691767+0000","last_peered":"2024-08-31T06:06:23.691767+0000","last_clean":"2024-08-31T06:06:23.691767+0000","last_became_active":"2024-08-31T06:04:25.880418+0000","last_became_peered":"2024-08-31T06:04:25.880418+0000","last_unstale":"2024-08-31T06:06:23.691767+0000","last_undegraded":"2024-08-31T06:06:23.691767+0000","last_fullsized":"2024-08-31T06:06:23.691767+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:21:46.904371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'34","reported_seq":310,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:39.167165+0000","last_change":"2024-08-31T06:04:22.843073+0000","last_active":"2024-08-31T06:06:39.167165+0000","last_peered":"2024-08-31T06:06:39.167165+0000","last_clean":"2024-08-31T06:06:39.167165+0000","last_became_active":"2024-08-31T06:04:22.842645+0000","last_became_peered":"2024-08-31T06:04:22.842645+0000","last_unstale":"2024-08-31T06:06:39.167165+0000","last_undegraded":"2024-08-31T06:06:39.167165+0000","last_fullsized":"2024-08-31T06:06:39.167165+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:41:37.651562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65695,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":250,"num_read_kb":189,"num_write":35,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.680499+0000","last_change":"2024-08-31T06:04:25.879929+0000","last_active":"2024-08-31T06:06:23.680499+0000","last_peered":"2024-08-31T06:06:23.680499+0000","last_clean":"2024-08-31T06:06:23.680499+0000","last_became_active":"2024-08-31T06:04:25.878475+0000","last_became_peered":"2024-08-31T06:04:25.878475+0000","last_unstale":"2024-08-31T06:06:23.680499+0000","last_undegraded":"2024-08-31T06:06:23.680499+0000","last_fullsized":"2024-08-31T06:06:23.680499+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:10:32.801558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048188+0000","last_change":"2024-08-31T06:04:22.842878+0000","last_active":"2024-08-31T06:05:25.048188+0000","last_peered":"2024-08-31T06:05:25.048188+0000","last_clean":"2024-08-31T06:05:25.048188+0000","last_became_active":"2024-08-31T06:04:22.842306+0000","last_became_peered":"2024-08-31T06:04:22.842306+0000","last_unstale":"2024-08-31T06:05:25.048188+0000","last_undegraded":"2024-08-31T06:05:25.048188+0000","last_fullsized":"2024-08-31T06:05:25.048188+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:53:55.259238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.669813+0000","last_change":"2024-08-31T06:04:25.899357+0000","last_active":"2024-08-31T06:06:23.669813+0000","last_peered":"2024-08-31T06:06:23.669813+0000","last_clean":"2024-08-31T06:06:23.669813+0000","last_became_active":"2024-08-31T06:04:25.899000+0000","last_became_peered":"2024-08-31T06:04:25.899000+0000","last_unstale":"2024-08-31T06:06:23.669813+0000","last_undegraded":"2024-08-31T06:06:23.669813+0000","last_fullsized":"2024-08-31T06:06:23.669813+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:20:39.991071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.044693+0000","last_change":"2024-08-31T06:04:22.843063+0000","last_active":"2024-08-31T06:05:25.044693+0000","last_peered":"2024-08-31T06:05:25.044693+0000","last_clean":"2024-08-31T06:05:25.044693+0000","last_became_active":"2024-08-31T06:04:22.842654+0000","last_became_peered":"2024-08-31T06:04:22.842654+0000","last_unstale":"2024-08-31T06:05:25.044693+0000","last_undegraded":"2024-08-31T06:05:25.044693+0000","last_fullsized":"2024-08-31T06:05:25.044693+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:09:54.563116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"80'15","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.680868+0000","last_change":"2024-08-31T06:04:25.885556+0000","last_active":"2024-08-31T06:06:23.680868+0000","last_peered":"2024-08-31T06:06:23.680868+0000","last_clean":"2024-08-31T06:06:23.680868+0000","last_became_active":"2024-08-31T06:04:25.885216+0000","last_became_peered":"2024-08-31T06:04:25.885216+0000","last_unstale":"2024-08-31T06:06:23.680868+0000","last_undegraded":"2024-08-31T06:06:23.680868+0000","last_fullsized":"2024-08-31T06:06:23.680868+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:48:49.816261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'5","reported_seq":154,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:37.720103+0000","last_change":"2024-08-31T06:04:22.844839+0000","last_active":"2024-08-31T06:06:37.720103+0000","last_peered":"2024-08-31T06:06:37.720103+0000","last_clean":"2024-08-31T06:06:37.720103+0000","last_became_active":"2024-08-31T06:04:22.844425+0000","last_became_peered":"2024-08-31T06:04:22.844425+0000","last_unstale":"2024-08-31T06:06:37.720103+0000","last_undegraded":"2024-08-31T06:06:37.720103+0000","last_fullsized":"2024-08-31T06:06:37.720103+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:10:57.292173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.682264+0000","last_change":"2024-08-31T06:04:25.878195+0000","last_active":"2024-08-31T06:06:23.682264+0000","last_peered":"2024-08-31T06:06:23.682264+0000","last_clean":"2024-08-31T06:06:23.682264+0000","last_became_active":"2024-08-31T06:04:25.877823+0000","last_became_peered":"2024-08-31T06:04:25.877823+0000","last_unstale":"2024-08-31T06:06:23.682264+0000","last_undegraded":"2024-08-31T06:06:23.682264+0000","last_fullsized":"2024-08-31T06:06:23.682264+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:54:34.989583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":67,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.071462+0000","last_change":"2024-08-31T06:05:25.071391+0000","last_active":"2024-08-31T06:05:25.071462+0000","last_peered":"2024-08-31T06:05:25.071462+0000","last_clean":"2024-08-31T06:05:25.071462+0000","last_became_active":"2024-08-31T06:05:25.057546+0000","last_became_peered":"2024-08-31T06:05:25.057546+0000","last_unstale":"2024-08-31T06:05:25.071462+0000","last_undegraded":"2024-08-31T06:05:25.071462+0000","last_fullsized":"2024-08-31T06:05:25.071462+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:42:35.527395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.681869+0000","last_change":"2024-08-31T06:04:25.887580+0000","last_active":"2024-08-31T06:06:23.681869+0000","last_peered":"2024-08-31T06:06:23.681869+0000","last_clean":"2024-08-31T06:06:23.681869+0000","last_became_active":"2024-08-31T06:04:25.887158+0000","last_became_peered":"2024-08-31T06:04:25.887158+0000","last_unstale":"2024-08-31T06:06:23.681869+0000","last_undegraded":"2024-08-31T06:06:23.681869+0000","last_fullsized":"2024-08-31T06:06:23.681869+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:27:38.115742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.064443+0000","last_change":"2024-08-31T06:05:25.063859+0000","last_active":"2024-08-31T06:05:25.064443+0000","last_peered":"2024-08-31T06:05:25.064443+0000","last_clean":"2024-08-31T06:05:25.064443+0000","last_became_active":"2024-08-31T06:05:25.053596+0000","last_became_peered":"2024-08-31T06:05:25.053596+0000","last_unstale":"2024-08-31T06:05:25.064443+0000","last_undegraded":"2024-08-31T06:05:25.064443+0000","last_fullsized":"2024-08-31T06:05:25.064443+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:19:18.461630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.678741+0000","last_change":"2024-08-31T06:04:25.880845+0000","last_active":"2024-08-31T06:06:23.678741+0000","last_peered":"2024-08-31T06:06:23.678741+0000","last_clean":"2024-08-31T06:06:23.678741+0000","last_became_active":"2024-08-31T06:04:25.880423+0000","last_became_peered":"2024-08-31T06:04:25.880423+0000","last_unstale":"2024-08-31T06:06:23.678741+0000","last_undegraded":"2024-08-31T06:06:23.678741+0000","last_fullsized":"2024-08-31T06:06:23.678741+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:16:56.725116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.046584+0000","last_change":"2024-08-31T06:04:22.843393+0000","last_active":"2024-08-31T06:05:25.046584+0000","last_peered":"2024-08-31T06:05:25.046584+0000","last_clean":"2024-08-31T06:05:25.046584+0000","last_became_active":"2024-08-31T06:04:22.842770+0000","last_became_peered":"2024-08-31T06:04:22.842770+0000","last_unstale":"2024-08-31T06:05:25.046584+0000","last_undegraded":"2024-08-31T06:05:25.046584+0000","last_fullsized":"2024-08-31T06:05:25.046584+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:49:16.159167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.685920+0000","last_change":"2024-08-31T06:04:25.887829+0000","last_active":"2024-08-31T06:06:23.685920+0000","last_peered":"2024-08-31T06:06:23.685920+0000","last_clean":"2024-08-31T06:06:23.685920+0000","last_became_active":"2024-08-31T06:04:25.886567+0000","last_became_peered":"2024-08-31T06:04:25.886567+0000","last_unstale":"2024-08-31T06:06:23.685920+0000","last_undegraded":"2024-08-31T06:06:23.685920+0000","last_fullsized":"2024-08-31T06:06:23.685920+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:11:39.054966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.045627+0000","last_change":"2024-08-31T06:04:22.844218+0000","last_active":"2024-08-31T06:05:25.045627+0000","last_peered":"2024-08-31T06:05:25.045627+0000","last_clean":"2024-08-31T06:05:25.045627+0000","last_became_active":"2024-08-31T06:04:22.843786+0000","last_became_peered":"2024-08-31T06:04:22.843786+0000","last_unstale":"2024-08-31T06:05:25.045627+0000","last_undegraded":"2024-08-31T06:05:25.045627+0000","last_fullsized":"2024-08-31T06:05:25.045627+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:46:02.534190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.682258+0000","last_change":"2024-08-31T06:04:25.888946+0000","last_active":"2024-08-31T06:06:23.682258+0000","last_peered":"2024-08-31T06:06:23.682258+0000","last_clean":"2024-08-31T06:06:23.682258+0000","last_became_active":"2024-08-31T06:04:25.888564+0000","last_became_peered":"2024-08-31T06:04:25.888564+0000","last_unstale":"2024-08-31T06:06:23.682258+0000","last_undegraded":"2024-08-31T06:06:23.682258+0000","last_fullsized":"2024-08-31T06:06:23.682258+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:26:19.051971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.049211+0000","last_change":"2024-08-31T06:04:22.964483+0000","last_active":"2024-08-31T06:05:25.049211+0000","last_peered":"2024-08-31T06:05:25.049211+0000","last_clean":"2024-08-31T06:05:25.049211+0000","last_became_active":"2024-08-31T06:04:22.963874+0000","last_became_peered":"2024-08-31T06:04:22.963874+0000","last_unstale":"2024-08-31T06:05:25.049211+0000","last_undegraded":"2024-08-31T06:05:25.049211+0000","last_fullsized":"2024-08-31T06:05:25.049211+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:30:00.159040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.684363+0000","last_change":"2024-08-31T06:04:25.880941+0000","last_active":"2024-08-31T06:06:23.684363+0000","last_peered":"2024-08-31T06:06:23.684363+0000","last_clean":"2024-08-31T06:06:23.684363+0000","last_became_active":"2024-08-31T06:04:25.880454+0000","last_became_peered":"2024-08-31T06:04:25.880454+0000","last_unstale":"2024-08-31T06:06:23.684363+0000","last_undegraded":"2024-08-31T06:06:23.684363+0000","last_fullsized":"2024-08-31T06:06:23.684363+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:49:57.023988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":98,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048527+0000","last_change":"2024-08-31T06:03:05.921629+0000","last_active":"2024-08-31T06:05:25.048527+0000","last_peered":"2024-08-31T06:05:25.048527+0000","last_clean":"2024-08-31T06:05:25.048527+0000","last_became_active":"2024-08-31T06:03:05.915844+0000","last_became_peered":"2024-08-31T06:03:05.915844+0000","last_unstale":"2024-08-31T06:05:25.048527+0000","last_undegraded":"2024-08-31T06:05:25.048527+0000","last_fullsized":"2024-08-31T06:05:25.048527+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T05:59:24.472429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T05:59:24.472429+0000","last_clean_scrub_stamp":"2024-08-31T05:59:24.472429+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:56:41.073422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.044718+0000","last_change":"2024-08-31T06:04:22.846325+0000","last_active":"2024-08-31T06:05:25.044718+0000","last_peered":"2024-08-31T06:05:25.044718+0000","last_clean":"2024-08-31T06:05:25.044718+0000","last_became_active":"2024-08-31T06:04:22.845772+0000","last_became_peered":"2024-08-31T06:04:22.845772+0000","last_unstale":"2024-08-31T06:05:25.044718+0000","last_undegraded":"2024-08-31T06:05:25.044718+0000","last_fullsized":"2024-08-31T06:05:25.044718+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:54:10.336786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.691294+0000","last_change":"2024-08-31T06:04:25.891437+0000","last_active":"2024-08-31T06:06:23.691294+0000","last_peered":"2024-08-31T06:06:23.691294+0000","last_clean":"2024-08-31T06:06:23.691294+0000","last_became_active":"2024-08-31T06:04:25.891053+0000","last_became_peered":"2024-08-31T06:04:25.891053+0000","last_unstale":"2024-08-31T06:06:23.691294+0000","last_undegraded":"2024-08-31T06:06:23.691294+0000","last_fullsized":"2024-08-31T06:06:23.691294+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:22:37.119483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048411+0000","last_change":"2024-08-31T06:04:22.842885+0000","last_active":"2024-08-31T06:05:25.048411+0000","last_peered":"2024-08-31T06:05:25.048411+0000","last_clean":"2024-08-31T06:05:25.048411+0000","last_became_active":"2024-08-31T06:04:22.842473+0000","last_became_peered":"2024-08-31T06:04:22.842473+0000","last_unstale":"2024-08-31T06:05:25.048411+0000","last_undegraded":"2024-08-31T06:05:25.048411+0000","last_fullsized":"2024-08-31T06:05:25.048411+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:33:36.516659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.663699+0000","last_change":"2024-08-31T06:04:25.889213+0000","last_active":"2024-08-31T06:06:23.663699+0000","last_peered":"2024-08-31T06:06:23.663699+0000","last_clean":"2024-08-31T06:06:23.663699+0000","last_became_active":"2024-08-31T06:04:25.888860+0000","last_became_peered":"2024-08-31T06:04:25.888860+0000","last_unstale":"2024-08-31T06:06:23.663699+0000","last_undegraded":"2024-08-31T06:06:23.663699+0000","last_fullsized":"2024-08-31T06:06:23.663699+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:44:18.831229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":152,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:38.720896+0000","last_change":"2024-08-31T06:04:22.966091+0000","last_active":"2024-08-31T06:06:38.720896+0000","last_peered":"2024-08-31T06:06:38.720896+0000","last_clean":"2024-08-31T06:06:38.720896+0000","last_became_active":"2024-08-31T06:04:22.965727+0000","last_became_peered":"2024-08-31T06:04:22.965727+0000","last_unstale":"2024-08-31T06:06:38.720896+0000","last_undegraded":"2024-08-31T06:06:38.720896+0000","last_fullsized":"2024-08-31T06:06:38.720896+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:42:43.643623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.682138+0000","last_change":"2024-08-31T06:04:25.880039+0000","last_active":"2024-08-31T06:06:23.682138+0000","last_peered":"2024-08-31T06:06:23.682138+0000","last_clean":"2024-08-31T06:06:23.682138+0000","last_became_active":"2024-08-31T06:04:25.879669+0000","last_became_peered":"2024-08-31T06:04:25.879669+0000","last_unstale":"2024-08-31T06:06:23.682138+0000","last_undegraded":"2024-08-31T06:06:23.682138+0000","last_fullsized":"2024-08-31T06:06:23.682138+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:02:09.244281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048687+0000","last_change":"2024-08-31T06:04:22.844572+0000","last_active":"2024-08-31T06:05:25.048687+0000","last_peered":"2024-08-31T06:05:25.048687+0000","last_clean":"2024-08-31T06:05:25.048687+0000","last_became_active":"2024-08-31T06:04:22.844227+0000","last_became_peered":"2024-08-31T06:04:22.844227+0000","last_unstale":"2024-08-31T06:05:25.048687+0000","last_undegraded":"2024-08-31T06:05:25.048687+0000","last_fullsized":"2024-08-31T06:05:25.048687+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:04:51.966901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.677043+0000","last_change":"2024-08-31T06:04:25.884442+0000","last_active":"2024-08-31T06:06:23.677043+0000","last_peered":"2024-08-31T06:06:23.677043+0000","last_clean":"2024-08-31T06:06:23.677043+0000","last_became_active":"2024-08-31T06:04:25.884016+0000","last_became_peered":"2024-08-31T06:04:25.884016+0000","last_unstale":"2024-08-31T06:06:23.677043+0000","last_undegraded":"2024-08-31T06:06:23.677043+0000","last_fullsized":"2024-08-31T06:06:23.677043+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:36:10.640262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.043831+0000","last_change":"2024-08-31T06:04:22.967845+0000","last_active":"2024-08-31T06:05:25.043831+0000","last_peered":"2024-08-31T06:05:25.043831+0000","last_clean":"2024-08-31T06:05:25.043831+0000","last_became_active":"2024-08-31T06:04:22.967421+0000","last_became_peered":"2024-08-31T06:04:22.967421+0000","last_unstale":"2024-08-31T06:05:25.043831+0000","last_undegraded":"2024-08-31T06:05:25.043831+0000","last_fullsized":"2024-08-31T06:05:25.043831+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:17:46.942551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.673410+0000","last_change":"2024-08-31T06:04:25.897087+0000","last_active":"2024-08-31T06:06:23.673410+0000","last_peered":"2024-08-31T06:06:23.673410+0000","last_clean":"2024-08-31T06:06:23.673410+0000","last_became_active":"2024-08-31T06:04:25.896725+0000","last_became_peered":"2024-08-31T06:04:25.896725+0000","last_unstale":"2024-08-31T06:06:23.673410+0000","last_undegraded":"2024-08-31T06:06:23.673410+0000","last_fullsized":"2024-08-31T06:06:23.673410+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:31:28.279489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.053673+0000","last_change":"2024-08-31T06:04:22.965299+0000","last_active":"2024-08-31T06:05:25.053673+0000","last_peered":"2024-08-31T06:05:25.053673+0000","last_clean":"2024-08-31T06:05:25.053673+0000","last_became_active":"2024-08-31T06:04:22.964849+0000","last_became_peered":"2024-08-31T06:04:22.964849+0000","last_unstale":"2024-08-31T06:05:25.053673+0000","last_undegraded":"2024-08-31T06:05:25.053673+0000","last_fullsized":"2024-08-31T06:05:25.053673+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:36:56.431678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.679635+0000","last_change":"2024-08-31T06:04:25.885115+0000","last_active":"2024-08-31T06:06:23.679635+0000","last_peered":"2024-08-31T06:06:23.679635+0000","last_clean":"2024-08-31T06:06:23.679635+0000","last_became_active":"2024-08-31T06:04:25.884722+0000","last_became_peered":"2024-08-31T06:04:25.884722+0000","last_unstale":"2024-08-31T06:06:23.679635+0000","last_undegraded":"2024-08-31T06:06:23.679635+0000","last_fullsized":"2024-08-31T06:06:23.679635+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:20:43.913785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'13","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:32.962890+0000","last_change":"2024-08-31T06:04:22.845119+0000","last_active":"2024-08-31T06:06:32.962890+0000","last_peered":"2024-08-31T06:06:32.962890+0000","last_clean":"2024-08-31T06:06:32.962890+0000","last_became_active":"2024-08-31T06:04:22.843814+0000","last_became_peered":"2024-08-31T06:04:22.843814+0000","last_unstale":"2024-08-31T06:06:32.962890+0000","last_undegraded":"2024-08-31T06:06:32.962890+0000","last_fullsized":"2024-08-31T06:06:32.962890+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:06:40.684887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.680844+0000","last_change":"2024-08-31T06:04:25.892838+0000","last_active":"2024-08-31T06:06:23.680844+0000","last_peered":"2024-08-31T06:06:23.680844+0000","last_clean":"2024-08-31T06:06:23.680844+0000","last_became_active":"2024-08-31T06:04:25.891955+0000","last_became_peered":"2024-08-31T06:04:25.891955+0000","last_unstale":"2024-08-31T06:06:23.680844+0000","last_undegraded":"2024-08-31T06:06:23.680844+0000","last_fullsized":"2024-08-31T06:06:23.680844+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:41:35.085724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.044957+0000","last_change":"2024-08-31T06:04:22.964183+0000","last_active":"2024-08-31T06:05:25.044957+0000","last_peered":"2024-08-31T06:05:25.044957+0000","last_clean":"2024-08-31T06:05:25.044957+0000","last_became_active":"2024-08-31T06:04:22.963688+0000","last_became_peered":"2024-08-31T06:04:22.963688+0000","last_unstale":"2024-08-31T06:05:25.044957+0000","last_undegraded":"2024-08-31T06:05:25.044957+0000","last_fullsized":"2024-08-31T06:05:25.044957+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:04:47.833811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.683320+0000","last_change":"2024-08-31T06:04:25.892266+0000","last_active":"2024-08-31T06:06:23.683320+0000","last_peered":"2024-08-31T06:06:23.683320+0000","last_clean":"2024-08-31T06:06:23.683320+0000","last_became_active":"2024-08-31T06:04:25.889014+0000","last_became_peered":"2024-08-31T06:04:25.889014+0000","last_unstale":"2024-08-31T06:06:23.683320+0000","last_undegraded":"2024-08-31T06:06:23.683320+0000","last_fullsized":"2024-08-31T06:06:23.683320+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:27:47.608470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.049408+0000","last_change":"2024-08-31T06:04:22.964677+0000","last_active":"2024-08-31T06:05:25.049408+0000","last_peered":"2024-08-31T06:05:25.049408+0000","last_clean":"2024-08-31T06:05:25.049408+0000","last_became_active":"2024-08-31T06:04:22.964237+0000","last_became_peered":"2024-08-31T06:04:22.964237+0000","last_unstale":"2024-08-31T06:05:25.049408+0000","last_undegraded":"2024-08-31T06:05:25.049408+0000","last_fullsized":"2024-08-31T06:05:25.049408+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:00:54.099592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.648239+0000","last_change":"2024-08-31T06:04:25.881699+0000","last_active":"2024-08-31T06:06:23.648239+0000","last_peered":"2024-08-31T06:06:23.648239+0000","last_clean":"2024-08-31T06:06:23.648239+0000","last_became_active":"2024-08-31T06:04:25.879304+0000","last_became_peered":"2024-08-31T06:04:25.879304+0000","last_unstale":"2024-08-31T06:06:23.648239+0000","last_undegraded":"2024-08-31T06:06:23.648239+0000","last_fullsized":"2024-08-31T06:06:23.648239+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:56:42.992034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.108301+0000","last_change":"2024-08-31T06:04:22.847934+0000","last_active":"2024-08-31T06:05:25.108301+0000","last_peered":"2024-08-31T06:05:25.108301+0000","last_clean":"2024-08-31T06:05:25.108301+0000","last_became_active":"2024-08-31T06:04:22.847518+0000","last_became_peered":"2024-08-31T06:04:22.847518+0000","last_unstale":"2024-08-31T06:05:25.108301+0000","last_undegraded":"2024-08-31T06:05:25.108301+0000","last_fullsized":"2024-08-31T06:05:25.108301+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:26:21.476014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.685828+0000","last_change":"2024-08-31T06:04:25.884653+0000","last_active":"2024-08-31T06:06:23.685828+0000","last_peered":"2024-08-31T06:06:23.685828+0000","last_clean":"2024-08-31T06:06:23.685828+0000","last_became_active":"2024-08-31T06:04:25.884244+0000","last_became_peered":"2024-08-31T06:04:25.884244+0000","last_unstale":"2024-08-31T06:06:23.685828+0000","last_undegraded":"2024-08-31T06:06:23.685828+0000","last_fullsized":"2024-08-31T06:06:23.685828+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:13:00.814071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048648+0000","last_change":"2024-08-31T06:04:22.845924+0000","last_active":"2024-08-31T06:05:25.048648+0000","last_peered":"2024-08-31T06:05:25.048648+0000","last_clean":"2024-08-31T06:05:25.048648+0000","last_became_active":"2024-08-31T06:04:22.845445+0000","last_became_peered":"2024-08-31T06:04:22.845445+0000","last_unstale":"2024-08-31T06:05:25.048648+0000","last_undegraded":"2024-08-31T06:05:25.048648+0000","last_fullsized":"2024-08-31T06:05:25.048648+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:25:21.052136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.675011+0000","last_change":"2024-08-31T06:04:25.890069+0000","last_active":"2024-08-31T06:06:23.675011+0000","last_peered":"2024-08-31T06:06:23.675011+0000","last_clean":"2024-08-31T06:06:23.675011+0000","last_became_active":"2024-08-31T06:04:25.889709+0000","last_became_peered":"2024-08-31T06:04:25.889709+0000","last_unstale":"2024-08-31T06:06:23.675011+0000","last_undegraded":"2024-08-31T06:06:23.675011+0000","last_fullsized":"2024-08-31T06:06:23.675011+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:13:51.360023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.051611+0000","last_change":"2024-08-31T06:04:22.847845+0000","last_active":"2024-08-31T06:05:25.051611+0000","last_peered":"2024-08-31T06:05:25.051611+0000","last_clean":"2024-08-31T06:05:25.051611+0000","last_became_active":"2024-08-31T06:04:22.847001+0000","last_became_peered":"2024-08-31T06:04:22.847001+0000","last_unstale":"2024-08-31T06:05:25.051611+0000","last_undegraded":"2024-08-31T06:05:25.051611+0000","last_fullsized":"2024-08-31T06:05:25.051611+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:24:34.209076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.690813+0000","last_change":"2024-08-31T06:04:25.886759+0000","last_active":"2024-08-31T06:06:23.690813+0000","last_peered":"2024-08-31T06:06:23.690813+0000","last_clean":"2024-08-31T06:06:23.690813+0000","last_became_active":"2024-08-31T06:04:25.886176+0000","last_became_peered":"2024-08-31T06:04:25.886176+0000","last_unstale":"2024-08-31T06:06:23.690813+0000","last_undegraded":"2024-08-31T06:06:23.690813+0000","last_fullsized":"2024-08-31T06:06:23.690813+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:11:12.637451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"80'15","reported_seq":43,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.677867+0000","last_change":"2024-08-31T06:05:25.054468+0000","last_active":"2024-08-31T06:06:23.677867+0000","last_peered":"2024-08-31T06:06:23.677867+0000","last_clean":"2024-08-31T06:06:23.677867+0000","last_became_active":"2024-08-31T06:05:25.053657+0000","last_became_peered":"2024-08-31T06:05:25.053657+0000","last_unstale":"2024-08-31T06:06:23.677867+0000","last_undegraded":"2024-08-31T06:06:23.677867+0000","last_fullsized":"2024-08-31T06:06:23.677867+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:43:44.887637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.059757+0000","last_change":"2024-08-31T06:05:25.059219+0000","last_active":"2024-08-31T06:05:25.059757+0000","last_peered":"2024-08-31T06:05:25.059757+0000","last_clean":"2024-08-31T06:05:25.059757+0000","last_became_active":"2024-08-31T06:05:25.053136+0000","last_became_peered":"2024-08-31T06:05:25.053136+0000","last_unstale":"2024-08-31T06:05:25.059757+0000","last_undegraded":"2024-08-31T06:05:25.059757+0000","last_fullsized":"2024-08-31T06:05:25.059757+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:29:59.137465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.691662+0000","last_change":"2024-08-31T06:04:25.887524+0000","last_active":"2024-08-31T06:06:23.691662+0000","last_peered":"2024-08-31T06:06:23.691662+0000","last_clean":"2024-08-31T06:06:23.691662+0000","last_became_active":"2024-08-31T06:04:25.887176+0000","last_became_peered":"2024-08-31T06:04:25.887176+0000","last_unstale":"2024-08-31T06:06:23.691662+0000","last_undegraded":"2024-08-31T06:06:23.691662+0000","last_fullsized":"2024-08-31T06:06:23.691662+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:08:05.476804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.053524+0000","last_change":"2024-08-31T06:04:22.847747+0000","last_active":"2024-08-31T06:05:25.053524+0000","last_peered":"2024-08-31T06:05:25.053524+0000","last_clean":"2024-08-31T06:05:25.053524+0000","last_became_active":"2024-08-31T06:04:22.847332+0000","last_became_peered":"2024-08-31T06:04:22.847332+0000","last_unstale":"2024-08-31T06:05:25.053524+0000","last_undegraded":"2024-08-31T06:05:25.053524+0000","last_fullsized":"2024-08-31T06:05:25.053524+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:41:35.910991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.687796+0000","last_change":"2024-08-31T06:04:25.883110+0000","last_active":"2024-08-31T06:06:23.687796+0000","last_peered":"2024-08-31T06:06:23.687796+0000","last_clean":"2024-08-31T06:06:23.687796+0000","last_became_active":"2024-08-31T06:04:25.880487+0000","last_became_peered":"2024-08-31T06:04:25.880487+0000","last_unstale":"2024-08-31T06:06:23.687796+0000","last_undegraded":"2024-08-31T06:06:23.687796+0000","last_fullsized":"2024-08-31T06:06:23.687796+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:22:13.656078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.108210+0000","last_change":"2024-08-31T06:04:22.964758+0000","last_active":"2024-08-31T06:05:25.108210+0000","last_peered":"2024-08-31T06:05:25.108210+0000","last_clean":"2024-08-31T06:05:25.108210+0000","last_became_active":"2024-08-31T06:04:22.964391+0000","last_became_peered":"2024-08-31T06:04:22.964391+0000","last_unstale":"2024-08-31T06:05:25.108210+0000","last_undegraded":"2024-08-31T06:05:25.108210+0000","last_fullsized":"2024-08-31T06:05:25.108210+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:28:59.190657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.678294+0000","last_change":"2024-08-31T06:04:25.893738+0000","last_active":"2024-08-31T06:06:23.678294+0000","last_peered":"2024-08-31T06:06:23.678294+0000","last_clean":"2024-08-31T06:06:23.678294+0000","last_became_active":"2024-08-31T06:04:25.893308+0000","last_became_peered":"2024-08-31T06:04:25.893308+0000","last_unstale":"2024-08-31T06:06:23.678294+0000","last_undegraded":"2024-08-31T06:06:23.678294+0000","last_fullsized":"2024-08-31T06:06:23.678294+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:26:48.451894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":38,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.043990+0000","last_change":"2024-08-31T06:04:22.843346+0000","last_active":"2024-08-31T06:05:25.043990+0000","last_peered":"2024-08-31T06:05:25.043990+0000","last_clean":"2024-08-31T06:05:25.043990+0000","last_became_active":"2024-08-31T06:04:22.842249+0000","last_became_peered":"2024-08-31T06:04:22.842249+0000","last_unstale":"2024-08-31T06:05:25.043990+0000","last_undegraded":"2024-08-31T06:05:25.043990+0000","last_fullsized":"2024-08-31T06:05:25.043990+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:40:57.618940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70369,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":742,"num_read_kb":449,"num_write":178,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1147,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":205734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":249373,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710761,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":436463,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874285,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":1094149,"data_compressed":42124,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037812,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":436677,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234041,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":436575,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397567,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":933888,"data_stored":1029200,"data_compressed":42124,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593797,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":436066,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790025,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":1028352,"data_compressed":42124,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953552,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36208,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708944,"statfs":{"total":95995035648,"available":95957958656,"internally_reserved":0,"allocated":884736,"data_stored":438189,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149779,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":888832,"data_stored":436377,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411418,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":942080,"data_stored":496797,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542239,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36268,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708884,"statfs":{"total":95995035648,"available":95957897216,"internally_reserved":0,"allocated":946176,"data_stored":501666,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705765,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":892928,"data_stored":437337,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34699,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34699,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34699,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":65807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":60938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":67805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-31T06:06:42.897 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-31T06:06:42.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:42 smithi184 ceph-mon[22331]: pgmap v347: 97 pgs: 97 active+clean; 645 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 7 op/s 2024-08-31T06:06:42.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:42 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/1651087121' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:06:43.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.154+0000 7f7485600640 1 -- 172.21.15.16:0/3191747337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74801090c0 msgr2=0x7f74801094d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:43.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.154+0000 7f7485600640 1 --2- 172.21.15.16:0/3191747337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74801090c0 0x7f74801094d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f746c009900 tx=0x7f746c02a3f0 comp rx=0 tx=0).stop 2024-08-31T06:06:43.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.154+0000 7f7485600640 1 -- 172.21.15.16:0/3191747337 shutdown_connections 2024-08-31T06:06:43.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.154+0000 7f7485600640 1 --2- 172.21.15.16:0/3191747337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7480109a10 0x7f7480110240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:43.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.154+0000 7f7485600640 1 --2- 172.21.15.16:0/3191747337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74801090c0 0x7f74801094d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:43.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.154+0000 7f7485600640 1 --2- 172.21.15.16:0/3191747337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7480103690 0x7f7480103aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:43.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.154+0000 7f7485600640 1 -- 172.21.15.16:0/3191747337 >> 172.21.15.16:0/3191747337 conn(0x7f74800ff0d0 msgr2=0x7f7480101520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:43.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.155+0000 7f7485600640 1 -- 172.21.15.16:0/3191747337 shutdown_connections 2024-08-31T06:06:43.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.155+0000 7f7485600640 1 -- 172.21.15.16:0/3191747337 wait complete. 2024-08-31T06:06:43.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.156+0000 7f7485600640 1 Processor -- start 2024-08-31T06:06:43.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.156+0000 7f7485600640 1 -- start start 2024-08-31T06:06:43.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.156+0000 7f7485600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7480103690 0x7f748019a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:43.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.156+0000 7f7485600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74801090c0 0x7f748019a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:43.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.156+0000 7f7485600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7480109a10 0x7f748019f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:43.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.156+0000 7f7485600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f74801136d0 con 0x7f7480103690 2024-08-31T06:06:43.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.156+0000 7f7485600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7480113550 con 0x7f7480109a10 2024-08-31T06:06:43.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.156+0000 7f7485600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7480113850 con 0x7f74801090c0 2024-08-31T06:06:43.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.156+0000 7f747fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7480103690 0x7f748019a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:43.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.157+0000 7f747fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7480103690 0x7f748019a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50870/0 (socket says 172.21.15.16:50870) 2024-08-31T06:06:43.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.157+0000 7f747fe00640 1 -- 172.21.15.16:0/4067724275 learned_addr learned my addr 172.21.15.16:0/4067724275 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:43.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.157+0000 7f747f400640 1 --2- 172.21.15.16:0/4067724275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74801090c0 0x7f748019a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:43.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.157+0000 7f7484c00640 1 --2- 172.21.15.16:0/4067724275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7480109a10 0x7f748019f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:43.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.157+0000 7f747fe00640 1 -- 172.21.15.16:0/4067724275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74801090c0 msgr2=0x7f748019a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:43.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.157+0000 7f747fe00640 1 --2- 172.21.15.16:0/4067724275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74801090c0 0x7f748019a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:43.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.157+0000 7f747fe00640 1 -- 172.21.15.16:0/4067724275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7480109a10 msgr2=0x7f748019f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:43.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.157+0000 7f747fe00640 1 --2- 172.21.15.16:0/4067724275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7480109a10 0x7f748019f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:43.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.157+0000 7f747fe00640 1 -- 172.21.15.16:0/4067724275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74801a0060 con 0x7f7480103690 2024-08-31T06:06:43.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.157+0000 7f747fe00640 1 --2- 172.21.15.16:0/4067724275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7480103690 0x7f748019a130 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f747000ea60 tx=0x7f747000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:43.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.158+0000 7f747cc00640 1 -- 172.21.15.16:0/4067724275 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747000cde0 con 0x7f7480103690 2024-08-31T06:06:43.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.158+0000 7f747cc00640 1 -- 172.21.15.16:0/4067724275 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74700035f0 con 0x7f7480103690 2024-08-31T06:06:43.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.158+0000 7f7485600640 1 -- 172.21.15.16:0/4067724275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74801a0350 con 0x7f7480103690 2024-08-31T06:06:43.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.158+0000 7f747cc00640 1 -- 172.21.15.16:0/4067724275 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7470010690 con 0x7f7480103690 2024-08-31T06:06:43.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.158+0000 7f7485600640 1 -- 172.21.15.16:0/4067724275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74801a6ae0 con 0x7f7480103690 2024-08-31T06:06:43.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.159+0000 7f7485600640 1 -- 172.21.15.16:0/4067724275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7480103aa0 con 0x7f7480103690 2024-08-31T06:06:43.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.161+0000 7f747cc00640 1 -- 172.21.15.16:0/4067724275 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7f74700037a0 con 0x7f7480103690 2024-08-31T06:06:43.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.163+0000 7f747cc00640 1 --2- 172.21.15.16:0/4067724275 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f74640b4b80 0x7f74640b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:43.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.163+0000 7f747cc00640 1 -- 172.21.15.16:0/4067724275 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f74700e55b0 con 0x7f7480103690 2024-08-31T06:06:43.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.163+0000 7f747f400640 1 --2- 172.21.15.16:0/4067724275 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f74640b4b80 0x7f74640b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:43.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.163+0000 7f747f400640 1 --2- 172.21.15.16:0/4067724275 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f74640b4b80 0x7f74640b7090 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f746c02a9e0 tx=0x7f746c005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:43.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.166+0000 7f747cc00640 1 -- 172.21.15.16:0/4067724275 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f74700ac5c0 con 0x7f7480103690 2024-08-31T06:06:43.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.359+0000 7f7485600640 1 -- 172.21.15.16:0/4067724275 --> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f74800635a0 con 0x7f74640b4b80 2024-08-31T06:06:43.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.364+0000 7f747cc00640 1 -- 172.21.15.16:0/4067724275 <== mgr.14220 v2:172.21.15.16:6800/4279751337 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259063 (secure 0 0 0) 0x7f74800635a0 con 0x7f74640b4b80 2024-08-31T06:06:43.369 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:06:43.370 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-08-31T06:06:43.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.367+0000 7f7485600640 1 -- 172.21.15.16:0/4067724275 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f74640b4b80 msgr2=0x7f74640b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:43.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.367+0000 7f7485600640 1 --2- 172.21.15.16:0/4067724275 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f74640b4b80 0x7f74640b7090 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f746c02a9e0 tx=0x7f746c005a20 comp rx=0 tx=0).stop 2024-08-31T06:06:43.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.367+0000 7f7485600640 1 -- 172.21.15.16:0/4067724275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7480103690 msgr2=0x7f748019a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:43.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.367+0000 7f7485600640 1 --2- 172.21.15.16:0/4067724275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7480103690 0x7f748019a130 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f747000ea60 tx=0x7f747000ef20 comp rx=0 tx=0).stop 2024-08-31T06:06:43.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.368+0000 7f7485600640 1 -- 172.21.15.16:0/4067724275 shutdown_connections 2024-08-31T06:06:43.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.368+0000 7f7485600640 1 --2- 172.21.15.16:0/4067724275 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f74640b4b80 0x7f74640b7090 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:43.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.368+0000 7f7485600640 1 --2- 172.21.15.16:0/4067724275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74801090c0 0x7f748019a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:43.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.368+0000 7f7485600640 1 --2- 172.21.15.16:0/4067724275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7480109a10 0x7f748019f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:43.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.368+0000 7f7485600640 1 --2- 172.21.15.16:0/4067724275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7480103690 0x7f748019a130 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:43.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.368+0000 7f7485600640 1 -- 172.21.15.16:0/4067724275 >> 172.21.15.16:0/4067724275 conn(0x7f74800ff0d0 msgr2=0x7f74801014f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:43.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.368+0000 7f7485600640 1 -- 172.21.15.16:0/4067724275 shutdown_connections 2024-08-31T06:06:43.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.368+0000 7f7485600640 1 -- 172.21.15.16:0/4067724275 wait complete. 2024-08-31T06:06:43.396 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":348,"stamp":"2024-08-31T06:06:43.291186+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660971,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":823,"num_read_kb":518,"num_write":303,"num_write_kb":2278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":2952987,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":745,"ondisk_log_size":745,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372444,"kb_used_data":10716,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124569380,"statfs":{"total":1151940427776,"available":1151559045120,"internally_reserved":0,"allocated":10973184,"data_stored":7213221,"data_compressed":178617,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5684,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":37,"num_write":4,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003164"},"pg_stats":[{"pgid":"3.3f","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.686021+0000","last_change":"2024-08-31T06:04:25.877318+0000","last_active":"2024-08-31T06:06:23.686021+0000","last_peered":"2024-08-31T06:06:23.686021+0000","last_clean":"2024-08-31T06:06:23.686021+0000","last_became_active":"2024-08-31T06:04:25.876894+0000","last_became_peered":"2024-08-31T06:04:25.876894+0000","last_unstale":"2024-08-31T06:06:23.686021+0000","last_undegraded":"2024-08-31T06:06:23.686021+0000","last_fullsized":"2024-08-31T06:06:23.686021+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:41:09.449343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"80'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.685244+0000","last_change":"2024-08-31T06:04:25.885670+0000","last_active":"2024-08-31T06:06:23.685244+0000","last_peered":"2024-08-31T06:06:23.685244+0000","last_clean":"2024-08-31T06:06:23.685244+0000","last_became_active":"2024-08-31T06:04:25.883818+0000","last_became_peered":"2024-08-31T06:04:25.883818+0000","last_unstale":"2024-08-31T06:06:23.685244+0000","last_undegraded":"2024-08-31T06:06:23.685244+0000","last_fullsized":"2024-08-31T06:06:23.685244+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:40:01.344432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.687803+0000","last_change":"2024-08-31T06:04:25.883375+0000","last_active":"2024-08-31T06:06:23.687803+0000","last_peered":"2024-08-31T06:06:23.687803+0000","last_clean":"2024-08-31T06:06:23.687803+0000","last_became_active":"2024-08-31T06:04:25.881750+0000","last_became_peered":"2024-08-31T06:04:25.881750+0000","last_unstale":"2024-08-31T06:06:23.687803+0000","last_undegraded":"2024-08-31T06:06:23.687803+0000","last_fullsized":"2024-08-31T06:06:23.687803+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:46:46.757363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.676625+0000","last_change":"2024-08-31T06:04:25.882436+0000","last_active":"2024-08-31T06:06:23.676625+0000","last_peered":"2024-08-31T06:06:23.676625+0000","last_clean":"2024-08-31T06:06:23.676625+0000","last_became_active":"2024-08-31T06:04:25.881805+0000","last_became_peered":"2024-08-31T06:04:25.881805+0000","last_unstale":"2024-08-31T06:06:23.676625+0000","last_undegraded":"2024-08-31T06:06:23.676625+0000","last_fullsized":"2024-08-31T06:06:23.676625+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:42:39.793625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"80'4","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.661470+0000","last_change":"2024-08-31T06:04:25.894534+0000","last_active":"2024-08-31T06:06:23.661470+0000","last_peered":"2024-08-31T06:06:23.661470+0000","last_clean":"2024-08-31T06:06:23.661470+0000","last_became_active":"2024-08-31T06:04:25.894116+0000","last_became_peered":"2024-08-31T06:04:25.894116+0000","last_unstale":"2024-08-31T06:06:23.661470+0000","last_undegraded":"2024-08-31T06:06:23.661470+0000","last_fullsized":"2024-08-31T06:06:23.661470+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:02:31.257454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.669411+0000","last_change":"2024-08-31T06:04:25.886182+0000","last_active":"2024-08-31T06:06:23.669411+0000","last_peered":"2024-08-31T06:06:23.669411+0000","last_clean":"2024-08-31T06:06:23.669411+0000","last_became_active":"2024-08-31T06:04:25.885825+0000","last_became_peered":"2024-08-31T06:04:25.885825+0000","last_unstale":"2024-08-31T06:06:23.669411+0000","last_undegraded":"2024-08-31T06:06:23.669411+0000","last_fullsized":"2024-08-31T06:06:23.669411+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:58:12.427931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.682008+0000","last_change":"2024-08-31T06:04:25.877521+0000","last_active":"2024-08-31T06:06:23.682008+0000","last_peered":"2024-08-31T06:06:23.682008+0000","last_clean":"2024-08-31T06:06:23.682008+0000","last_became_active":"2024-08-31T06:04:25.877122+0000","last_became_peered":"2024-08-31T06:04:25.877122+0000","last_unstale":"2024-08-31T06:06:23.682008+0000","last_undegraded":"2024-08-31T06:06:23.682008+0000","last_fullsized":"2024-08-31T06:06:23.682008+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:17:27.406828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.680720+0000","last_change":"2024-08-31T06:04:25.892547+0000","last_active":"2024-08-31T06:06:23.680720+0000","last_peered":"2024-08-31T06:06:23.680720+0000","last_clean":"2024-08-31T06:06:23.680720+0000","last_became_active":"2024-08-31T06:04:25.892090+0000","last_became_peered":"2024-08-31T06:04:25.892090+0000","last_unstale":"2024-08-31T06:06:23.680720+0000","last_undegraded":"2024-08-31T06:06:23.680720+0000","last_fullsized":"2024-08-31T06:06:23.680720+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:28:53.414252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"80'11","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.688048+0000","last_change":"2024-08-31T06:04:25.879952+0000","last_active":"2024-08-31T06:06:23.688048+0000","last_peered":"2024-08-31T06:06:23.688048+0000","last_clean":"2024-08-31T06:06:23.688048+0000","last_became_active":"2024-08-31T06:04:25.879568+0000","last_became_peered":"2024-08-31T06:04:25.879568+0000","last_unstale":"2024-08-31T06:06:23.688048+0000","last_undegraded":"2024-08-31T06:06:23.688048+0000","last_fullsized":"2024-08-31T06:06:23.688048+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:28:04.558054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"80'16","reported_seq":44,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.676096+0000","last_change":"2024-08-31T06:05:25.051173+0000","last_active":"2024-08-31T06:06:23.676096+0000","last_peered":"2024-08-31T06:06:23.676096+0000","last_clean":"2024-08-31T06:06:23.676096+0000","last_became_active":"2024-08-31T06:05:25.050867+0000","last_became_peered":"2024-08-31T06:05:25.050867+0000","last_unstale":"2024-08-31T06:06:23.676096+0000","last_undegraded":"2024-08-31T06:06:23.676096+0000","last_fullsized":"2024-08-31T06:06:23.676096+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:27:09.971497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.686392+0000","last_change":"2024-08-31T06:04:25.883984+0000","last_active":"2024-08-31T06:06:23.686392+0000","last_peered":"2024-08-31T06:06:23.686392+0000","last_clean":"2024-08-31T06:06:23.686392+0000","last_became_active":"2024-08-31T06:04:25.883353+0000","last_became_peered":"2024-08-31T06:04:25.883353+0000","last_unstale":"2024-08-31T06:06:23.686392+0000","last_undegraded":"2024-08-31T06:06:23.686392+0000","last_fullsized":"2024-08-31T06:06:23.686392+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:16:25.178952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.672858+0000","last_change":"2024-08-31T06:04:25.883335+0000","last_active":"2024-08-31T06:06:23.672858+0000","last_peered":"2024-08-31T06:06:23.672858+0000","last_clean":"2024-08-31T06:06:23.672858+0000","last_became_active":"2024-08-31T06:04:25.881488+0000","last_became_peered":"2024-08-31T06:04:25.881488+0000","last_unstale":"2024-08-31T06:06:23.672858+0000","last_undegraded":"2024-08-31T06:06:23.672858+0000","last_fullsized":"2024-08-31T06:06:23.672858+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:57:24.472482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'10","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.678638+0000","last_change":"2024-08-31T06:04:25.891579+0000","last_active":"2024-08-31T06:06:23.678638+0000","last_peered":"2024-08-31T06:06:23.678638+0000","last_clean":"2024-08-31T06:06:23.678638+0000","last_became_active":"2024-08-31T06:04:25.887950+0000","last_became_peered":"2024-08-31T06:04:25.887950+0000","last_unstale":"2024-08-31T06:06:23.678638+0000","last_undegraded":"2024-08-31T06:06:23.678638+0000","last_fullsized":"2024-08-31T06:06:23.678638+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:38:35.284442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.682116+0000","last_change":"2024-08-31T06:04:25.882185+0000","last_active":"2024-08-31T06:06:23.682116+0000","last_peered":"2024-08-31T06:06:23.682116+0000","last_clean":"2024-08-31T06:06:23.682116+0000","last_became_active":"2024-08-31T06:04:25.876499+0000","last_became_peered":"2024-08-31T06:04:25.876499+0000","last_unstale":"2024-08-31T06:06:23.682116+0000","last_undegraded":"2024-08-31T06:06:23.682116+0000","last_fullsized":"2024-08-31T06:06:23.682116+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:40:01.132353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.683752+0000","last_change":"2024-08-31T06:04:25.883624+0000","last_active":"2024-08-31T06:06:23.683752+0000","last_peered":"2024-08-31T06:06:23.683752+0000","last_clean":"2024-08-31T06:06:23.683752+0000","last_became_active":"2024-08-31T06:04:25.878041+0000","last_became_peered":"2024-08-31T06:04:25.878041+0000","last_unstale":"2024-08-31T06:06:23.683752+0000","last_undegraded":"2024-08-31T06:06:23.683752+0000","last_fullsized":"2024-08-31T06:06:23.683752+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:05:21.278321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.687675+0000","last_change":"2024-08-31T06:04:25.882538+0000","last_active":"2024-08-31T06:06:23.687675+0000","last_peered":"2024-08-31T06:06:23.687675+0000","last_clean":"2024-08-31T06:06:23.687675+0000","last_became_active":"2024-08-31T06:04:25.882201+0000","last_became_peered":"2024-08-31T06:04:25.882201+0000","last_unstale":"2024-08-31T06:06:23.687675+0000","last_undegraded":"2024-08-31T06:06:23.687675+0000","last_fullsized":"2024-08-31T06:06:23.687675+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:47:16.880893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.686044+0000","last_change":"2024-08-31T06:04:25.880376+0000","last_active":"2024-08-31T06:06:23.686044+0000","last_peered":"2024-08-31T06:06:23.686044+0000","last_clean":"2024-08-31T06:06:23.686044+0000","last_became_active":"2024-08-31T06:04:25.877668+0000","last_became_peered":"2024-08-31T06:04:25.877668+0000","last_unstale":"2024-08-31T06:06:23.686044+0000","last_undegraded":"2024-08-31T06:06:23.686044+0000","last_fullsized":"2024-08-31T06:06:23.686044+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:55:32.737406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"80'10","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.673276+0000","last_change":"2024-08-31T06:04:25.875431+0000","last_active":"2024-08-31T06:06:23.673276+0000","last_peered":"2024-08-31T06:06:23.673276+0000","last_clean":"2024-08-31T06:06:23.673276+0000","last_became_active":"2024-08-31T06:04:25.874751+0000","last_became_peered":"2024-08-31T06:04:25.874751+0000","last_unstale":"2024-08-31T06:06:23.673276+0000","last_undegraded":"2024-08-31T06:06:23.673276+0000","last_fullsized":"2024-08-31T06:06:23.673276+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:16:57.695415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.675738+0000","last_change":"2024-08-31T06:04:25.895656+0000","last_active":"2024-08-31T06:06:23.675738+0000","last_peered":"2024-08-31T06:06:23.675738+0000","last_clean":"2024-08-31T06:06:23.675738+0000","last_became_active":"2024-08-31T06:04:25.895272+0000","last_became_peered":"2024-08-31T06:04:25.895272+0000","last_unstale":"2024-08-31T06:06:23.675738+0000","last_undegraded":"2024-08-31T06:06:23.675738+0000","last_fullsized":"2024-08-31T06:06:23.675738+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:18:47.276195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.679502+0000","last_change":"2024-08-31T06:04:25.877283+0000","last_active":"2024-08-31T06:06:23.679502+0000","last_peered":"2024-08-31T06:06:23.679502+0000","last_clean":"2024-08-31T06:06:23.679502+0000","last_became_active":"2024-08-31T06:04:25.876846+0000","last_became_peered":"2024-08-31T06:04:25.876846+0000","last_unstale":"2024-08-31T06:06:23.679502+0000","last_undegraded":"2024-08-31T06:06:23.679502+0000","last_fullsized":"2024-08-31T06:06:23.679502+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:54:05.086844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"80'11","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.685368+0000","last_change":"2024-08-31T06:04:25.884891+0000","last_active":"2024-08-31T06:06:23.685368+0000","last_peered":"2024-08-31T06:06:23.685368+0000","last_clean":"2024-08-31T06:06:23.685368+0000","last_became_active":"2024-08-31T06:04:25.884501+0000","last_became_peered":"2024-08-31T06:04:25.884501+0000","last_unstale":"2024-08-31T06:06:23.685368+0000","last_undegraded":"2024-08-31T06:06:23.685368+0000","last_fullsized":"2024-08-31T06:06:23.685368+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:22:54.846102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"80'10","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.691670+0000","last_change":"2024-08-31T06:04:25.877150+0000","last_active":"2024-08-31T06:06:23.691670+0000","last_peered":"2024-08-31T06:06:23.691670+0000","last_clean":"2024-08-31T06:06:23.691670+0000","last_became_active":"2024-08-31T06:04:25.876766+0000","last_became_peered":"2024-08-31T06:04:25.876766+0000","last_unstale":"2024-08-31T06:06:23.691670+0000","last_undegraded":"2024-08-31T06:06:23.691670+0000","last_fullsized":"2024-08-31T06:06:23.691670+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:46:26.724970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.676164+0000","last_change":"2024-08-31T06:04:25.881275+0000","last_active":"2024-08-31T06:06:23.676164+0000","last_peered":"2024-08-31T06:06:23.676164+0000","last_clean":"2024-08-31T06:06:23.676164+0000","last_became_active":"2024-08-31T06:04:25.879877+0000","last_became_peered":"2024-08-31T06:04:25.879877+0000","last_unstale":"2024-08-31T06:06:23.676164+0000","last_undegraded":"2024-08-31T06:06:23.676164+0000","last_fullsized":"2024-08-31T06:06:23.676164+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:07:07.237041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.676080+0000","last_change":"2024-08-31T06:04:25.895750+0000","last_active":"2024-08-31T06:06:23.676080+0000","last_peered":"2024-08-31T06:06:23.676080+0000","last_clean":"2024-08-31T06:06:23.676080+0000","last_became_active":"2024-08-31T06:04:25.895348+0000","last_became_peered":"2024-08-31T06:04:25.895348+0000","last_unstale":"2024-08-31T06:06:23.676080+0000","last_undegraded":"2024-08-31T06:06:23.676080+0000","last_fullsized":"2024-08-31T06:06:23.676080+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:53:10.014808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.670437+0000","last_change":"2024-08-31T06:04:25.895537+0000","last_active":"2024-08-31T06:06:23.670437+0000","last_peered":"2024-08-31T06:06:23.670437+0000","last_clean":"2024-08-31T06:06:23.670437+0000","last_became_active":"2024-08-31T06:04:25.895136+0000","last_became_peered":"2024-08-31T06:04:25.895136+0000","last_unstale":"2024-08-31T06:06:23.670437+0000","last_undegraded":"2024-08-31T06:06:23.670437+0000","last_fullsized":"2024-08-31T06:06:23.670437+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:41:57.554507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"80'15","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.676301+0000","last_change":"2024-08-31T06:04:25.880490+0000","last_active":"2024-08-31T06:06:23.676301+0000","last_peered":"2024-08-31T06:06:23.676301+0000","last_clean":"2024-08-31T06:06:23.676301+0000","last_became_active":"2024-08-31T06:04:25.879269+0000","last_became_peered":"2024-08-31T06:04:25.879269+0000","last_unstale":"2024-08-31T06:06:23.676301+0000","last_undegraded":"2024-08-31T06:06:23.676301+0000","last_fullsized":"2024-08-31T06:06:23.676301+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:05:49.575915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"80'6","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.676526+0000","last_change":"2024-08-31T06:05:25.053714+0000","last_active":"2024-08-31T06:06:23.676526+0000","last_peered":"2024-08-31T06:06:23.676526+0000","last_clean":"2024-08-31T06:06:23.676526+0000","last_became_active":"2024-08-31T06:05:25.052685+0000","last_became_peered":"2024-08-31T06:05:25.052685+0000","last_unstale":"2024-08-31T06:06:23.676526+0000","last_undegraded":"2024-08-31T06:06:23.676526+0000","last_fullsized":"2024-08-31T06:06:23.676526+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:17:31.474601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"80'4","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.684823+0000","last_change":"2024-08-31T06:05:25.057946+0000","last_active":"2024-08-31T06:06:23.684823+0000","last_peered":"2024-08-31T06:06:23.684823+0000","last_clean":"2024-08-31T06:06:23.684823+0000","last_became_active":"2024-08-31T06:05:25.057501+0000","last_became_peered":"2024-08-31T06:05:25.057501+0000","last_unstale":"2024-08-31T06:06:23.684823+0000","last_undegraded":"2024-08-31T06:06:23.684823+0000","last_fullsized":"2024-08-31T06:06:23.684823+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:16:05.721770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.685410+0000","last_change":"2024-08-31T06:04:25.897473+0000","last_active":"2024-08-31T06:06:23.685410+0000","last_peered":"2024-08-31T06:06:23.685410+0000","last_clean":"2024-08-31T06:06:23.685410+0000","last_became_active":"2024-08-31T06:04:25.895194+0000","last_became_peered":"2024-08-31T06:04:25.895194+0000","last_unstale":"2024-08-31T06:06:23.685410+0000","last_undegraded":"2024-08-31T06:06:23.685410+0000","last_fullsized":"2024-08-31T06:06:23.685410+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:43:04.539970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.684923+0000","last_change":"2024-08-31T06:04:25.894402+0000","last_active":"2024-08-31T06:06:23.684923+0000","last_peered":"2024-08-31T06:06:23.684923+0000","last_clean":"2024-08-31T06:06:23.684923+0000","last_became_active":"2024-08-31T06:04:25.893883+0000","last_became_peered":"2024-08-31T06:04:25.893883+0000","last_unstale":"2024-08-31T06:06:23.684923+0000","last_undegraded":"2024-08-31T06:06:23.684923+0000","last_fullsized":"2024-08-31T06:06:23.684923+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:45:53.778473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"80'12","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.681979+0000","last_change":"2024-08-31T06:04:25.881543+0000","last_active":"2024-08-31T06:06:23.681979+0000","last_peered":"2024-08-31T06:06:23.681979+0000","last_clean":"2024-08-31T06:06:23.681979+0000","last_became_active":"2024-08-31T06:04:25.881144+0000","last_became_peered":"2024-08-31T06:04:25.881144+0000","last_unstale":"2024-08-31T06:06:23.681979+0000","last_undegraded":"2024-08-31T06:06:23.681979+0000","last_fullsized":"2024-08-31T06:06:23.681979+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:13:02.282879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.669781+0000","last_change":"2024-08-31T06:04:25.881143+0000","last_active":"2024-08-31T06:06:23.669781+0000","last_peered":"2024-08-31T06:06:23.669781+0000","last_clean":"2024-08-31T06:06:23.669781+0000","last_became_active":"2024-08-31T06:04:25.879718+0000","last_became_peered":"2024-08-31T06:04:25.879718+0000","last_unstale":"2024-08-31T06:06:23.669781+0000","last_undegraded":"2024-08-31T06:06:23.669781+0000","last_fullsized":"2024-08-31T06:06:23.669781+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:32:07.771488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.680189+0000","last_change":"2024-08-31T06:04:25.879493+0000","last_active":"2024-08-31T06:06:23.680189+0000","last_peered":"2024-08-31T06:06:23.680189+0000","last_clean":"2024-08-31T06:06:23.680189+0000","last_became_active":"2024-08-31T06:04:25.879065+0000","last_became_peered":"2024-08-31T06:04:25.879065+0000","last_unstale":"2024-08-31T06:06:23.680189+0000","last_undegraded":"2024-08-31T06:06:23.680189+0000","last_fullsized":"2024-08-31T06:06:23.680189+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:53:43.848269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"80'7","reported_seq":168,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:37.163835+0000","last_change":"2024-08-31T06:05:25.058954+0000","last_active":"2024-08-31T06:06:37.163835+0000","last_peered":"2024-08-31T06:06:37.163835+0000","last_clean":"2024-08-31T06:06:37.163835+0000","last_became_active":"2024-08-31T06:05:25.051143+0000","last_became_peered":"2024-08-31T06:05:25.051143+0000","last_unstale":"2024-08-31T06:06:37.163835+0000","last_undegraded":"2024-08-31T06:06:37.163835+0000","last_fullsized":"2024-08-31T06:06:37.163835+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:38:23.533663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.682132+0000","last_change":"2024-08-31T06:04:25.896354+0000","last_active":"2024-08-31T06:06:23.682132+0000","last_peered":"2024-08-31T06:06:23.682132+0000","last_clean":"2024-08-31T06:06:23.682132+0000","last_became_active":"2024-08-31T06:04:25.896021+0000","last_became_peered":"2024-08-31T06:04:25.896021+0000","last_unstale":"2024-08-31T06:06:23.682132+0000","last_undegraded":"2024-08-31T06:06:23.682132+0000","last_fullsized":"2024-08-31T06:06:23.682132+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:08:40.213238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.047153+0000","last_change":"2024-08-31T06:04:22.844248+0000","last_active":"2024-08-31T06:05:25.047153+0000","last_peered":"2024-08-31T06:05:25.047153+0000","last_clean":"2024-08-31T06:05:25.047153+0000","last_became_active":"2024-08-31T06:04:22.843784+0000","last_became_peered":"2024-08-31T06:04:22.843784+0000","last_unstale":"2024-08-31T06:05:25.047153+0000","last_undegraded":"2024-08-31T06:05:25.047153+0000","last_fullsized":"2024-08-31T06:05:25.047153+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:56:41.422965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.046218+0000","last_change":"2024-08-31T06:04:22.847049+0000","last_active":"2024-08-31T06:05:25.046218+0000","last_peered":"2024-08-31T06:05:25.046218+0000","last_clean":"2024-08-31T06:05:25.046218+0000","last_became_active":"2024-08-31T06:04:22.846282+0000","last_became_peered":"2024-08-31T06:04:22.846282+0000","last_unstale":"2024-08-31T06:05:25.046218+0000","last_undegraded":"2024-08-31T06:05:25.046218+0000","last_fullsized":"2024-08-31T06:05:25.046218+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:35:21.030425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.679457+0000","last_change":"2024-08-31T06:04:25.886698+0000","last_active":"2024-08-31T06:06:23.679457+0000","last_peered":"2024-08-31T06:06:23.679457+0000","last_clean":"2024-08-31T06:06:23.679457+0000","last_became_active":"2024-08-31T06:04:25.886334+0000","last_became_peered":"2024-08-31T06:04:25.886334+0000","last_unstale":"2024-08-31T06:06:23.679457+0000","last_undegraded":"2024-08-31T06:06:23.679457+0000","last_fullsized":"2024-08-31T06:06:23.679457+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:24:43.821446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.044798+0000","last_change":"2024-08-31T06:04:22.845263+0000","last_active":"2024-08-31T06:05:25.044798+0000","last_peered":"2024-08-31T06:05:25.044798+0000","last_clean":"2024-08-31T06:05:25.044798+0000","last_became_active":"2024-08-31T06:04:22.844853+0000","last_became_peered":"2024-08-31T06:04:22.844853+0000","last_unstale":"2024-08-31T06:05:25.044798+0000","last_undegraded":"2024-08-31T06:05:25.044798+0000","last_fullsized":"2024-08-31T06:05:25.044798+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:55:12.561165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"80'11","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.673293+0000","last_change":"2024-08-31T06:04:25.894475+0000","last_active":"2024-08-31T06:06:23.673293+0000","last_peered":"2024-08-31T06:06:23.673293+0000","last_clean":"2024-08-31T06:06:23.673293+0000","last_became_active":"2024-08-31T06:04:25.894121+0000","last_became_peered":"2024-08-31T06:04:25.894121+0000","last_unstale":"2024-08-31T06:06:23.673293+0000","last_undegraded":"2024-08-31T06:06:23.673293+0000","last_fullsized":"2024-08-31T06:06:23.673293+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:40:06.570616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":149,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:38.164972+0000","last_change":"2024-08-31T06:04:22.834430+0000","last_active":"2024-08-31T06:06:38.164972+0000","last_peered":"2024-08-31T06:06:38.164972+0000","last_clean":"2024-08-31T06:06:38.164972+0000","last_became_active":"2024-08-31T06:04:22.834011+0000","last_became_peered":"2024-08-31T06:04:22.834011+0000","last_unstale":"2024-08-31T06:06:38.164972+0000","last_undegraded":"2024-08-31T06:06:38.164972+0000","last_fullsized":"2024-08-31T06:06:38.164972+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:51:16.734234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"80'10","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.689334+0000","last_change":"2024-08-31T06:04:25.877727+0000","last_active":"2024-08-31T06:06:23.689334+0000","last_peered":"2024-08-31T06:06:23.689334+0000","last_clean":"2024-08-31T06:06:23.689334+0000","last_became_active":"2024-08-31T06:04:25.877121+0000","last_became_peered":"2024-08-31T06:04:25.877121+0000","last_unstale":"2024-08-31T06:06:23.689334+0000","last_undegraded":"2024-08-31T06:06:23.689334+0000","last_fullsized":"2024-08-31T06:06:23.689334+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:41:12.247216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048823+0000","last_change":"2024-08-31T06:04:22.846482+0000","last_active":"2024-08-31T06:05:25.048823+0000","last_peered":"2024-08-31T06:05:25.048823+0000","last_clean":"2024-08-31T06:05:25.048823+0000","last_became_active":"2024-08-31T06:04:22.846057+0000","last_became_peered":"2024-08-31T06:04:22.846057+0000","last_unstale":"2024-08-31T06:05:25.048823+0000","last_undegraded":"2024-08-31T06:05:25.048823+0000","last_fullsized":"2024-08-31T06:05:25.048823+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:29:11.861665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.684452+0000","last_change":"2024-08-31T06:04:25.893777+0000","last_active":"2024-08-31T06:06:23.684452+0000","last_peered":"2024-08-31T06:06:23.684452+0000","last_clean":"2024-08-31T06:06:23.684452+0000","last_became_active":"2024-08-31T06:04:25.893014+0000","last_became_peered":"2024-08-31T06:04:25.893014+0000","last_unstale":"2024-08-31T06:06:23.684452+0000","last_undegraded":"2024-08-31T06:06:23.684452+0000","last_fullsized":"2024-08-31T06:06:23.684452+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:04:48.185842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.045472+0000","last_change":"2024-08-31T06:04:22.965938+0000","last_active":"2024-08-31T06:05:25.045472+0000","last_peered":"2024-08-31T06:05:25.045472+0000","last_clean":"2024-08-31T06:05:25.045472+0000","last_became_active":"2024-08-31T06:04:22.965356+0000","last_became_peered":"2024-08-31T06:04:22.965356+0000","last_unstale":"2024-08-31T06:05:25.045472+0000","last_undegraded":"2024-08-31T06:05:25.045472+0000","last_fullsized":"2024-08-31T06:05:25.045472+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:36:58.938462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"80'11","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.691767+0000","last_change":"2024-08-31T06:04:25.881307+0000","last_active":"2024-08-31T06:06:23.691767+0000","last_peered":"2024-08-31T06:06:23.691767+0000","last_clean":"2024-08-31T06:06:23.691767+0000","last_became_active":"2024-08-31T06:04:25.880418+0000","last_became_peered":"2024-08-31T06:04:25.880418+0000","last_unstale":"2024-08-31T06:06:23.691767+0000","last_undegraded":"2024-08-31T06:06:23.691767+0000","last_fullsized":"2024-08-31T06:06:23.691767+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:21:46.904371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'34","reported_seq":310,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:39.167165+0000","last_change":"2024-08-31T06:04:22.843073+0000","last_active":"2024-08-31T06:06:39.167165+0000","last_peered":"2024-08-31T06:06:39.167165+0000","last_clean":"2024-08-31T06:06:39.167165+0000","last_became_active":"2024-08-31T06:04:22.842645+0000","last_became_peered":"2024-08-31T06:04:22.842645+0000","last_unstale":"2024-08-31T06:06:39.167165+0000","last_undegraded":"2024-08-31T06:06:39.167165+0000","last_fullsized":"2024-08-31T06:06:39.167165+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:41:37.651562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65695,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":250,"num_read_kb":189,"num_write":35,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.680499+0000","last_change":"2024-08-31T06:04:25.879929+0000","last_active":"2024-08-31T06:06:23.680499+0000","last_peered":"2024-08-31T06:06:23.680499+0000","last_clean":"2024-08-31T06:06:23.680499+0000","last_became_active":"2024-08-31T06:04:25.878475+0000","last_became_peered":"2024-08-31T06:04:25.878475+0000","last_unstale":"2024-08-31T06:06:23.680499+0000","last_undegraded":"2024-08-31T06:06:23.680499+0000","last_fullsized":"2024-08-31T06:06:23.680499+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:10:32.801558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048264+0000","last_change":"2024-08-31T06:04:22.842878+0000","last_active":"2024-08-31T06:05:25.048264+0000","last_peered":"2024-08-31T06:05:25.048264+0000","last_clean":"2024-08-31T06:05:25.048264+0000","last_became_active":"2024-08-31T06:04:22.842306+0000","last_became_peered":"2024-08-31T06:04:22.842306+0000","last_unstale":"2024-08-31T06:05:25.048264+0000","last_undegraded":"2024-08-31T06:05:25.048264+0000","last_fullsized":"2024-08-31T06:05:25.048264+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:53:55.259238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.669813+0000","last_change":"2024-08-31T06:04:25.899357+0000","last_active":"2024-08-31T06:06:23.669813+0000","last_peered":"2024-08-31T06:06:23.669813+0000","last_clean":"2024-08-31T06:06:23.669813+0000","last_became_active":"2024-08-31T06:04:25.899000+0000","last_became_peered":"2024-08-31T06:04:25.899000+0000","last_unstale":"2024-08-31T06:06:23.669813+0000","last_undegraded":"2024-08-31T06:06:23.669813+0000","last_fullsized":"2024-08-31T06:06:23.669813+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:20:39.991071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.044693+0000","last_change":"2024-08-31T06:04:22.843063+0000","last_active":"2024-08-31T06:05:25.044693+0000","last_peered":"2024-08-31T06:05:25.044693+0000","last_clean":"2024-08-31T06:05:25.044693+0000","last_became_active":"2024-08-31T06:04:22.842654+0000","last_became_peered":"2024-08-31T06:04:22.842654+0000","last_unstale":"2024-08-31T06:05:25.044693+0000","last_undegraded":"2024-08-31T06:05:25.044693+0000","last_fullsized":"2024-08-31T06:05:25.044693+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:09:54.563116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"80'15","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.680868+0000","last_change":"2024-08-31T06:04:25.885556+0000","last_active":"2024-08-31T06:06:23.680868+0000","last_peered":"2024-08-31T06:06:23.680868+0000","last_clean":"2024-08-31T06:06:23.680868+0000","last_became_active":"2024-08-31T06:04:25.885216+0000","last_became_peered":"2024-08-31T06:04:25.885216+0000","last_unstale":"2024-08-31T06:06:23.680868+0000","last_undegraded":"2024-08-31T06:06:23.680868+0000","last_fullsized":"2024-08-31T06:06:23.680868+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:48:49.816261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'5","reported_seq":154,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:37.720103+0000","last_change":"2024-08-31T06:04:22.844839+0000","last_active":"2024-08-31T06:06:37.720103+0000","last_peered":"2024-08-31T06:06:37.720103+0000","last_clean":"2024-08-31T06:06:37.720103+0000","last_became_active":"2024-08-31T06:04:22.844425+0000","last_became_peered":"2024-08-31T06:04:22.844425+0000","last_unstale":"2024-08-31T06:06:37.720103+0000","last_undegraded":"2024-08-31T06:06:37.720103+0000","last_fullsized":"2024-08-31T06:06:37.720103+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:10:57.292173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.682264+0000","last_change":"2024-08-31T06:04:25.878195+0000","last_active":"2024-08-31T06:06:23.682264+0000","last_peered":"2024-08-31T06:06:23.682264+0000","last_clean":"2024-08-31T06:06:23.682264+0000","last_became_active":"2024-08-31T06:04:25.877823+0000","last_became_peered":"2024-08-31T06:04:25.877823+0000","last_unstale":"2024-08-31T06:06:23.682264+0000","last_undegraded":"2024-08-31T06:06:23.682264+0000","last_fullsized":"2024-08-31T06:06:23.682264+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:54:34.989583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":68,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.071826+0000","last_change":"2024-08-31T06:05:25.071391+0000","last_active":"2024-08-31T06:05:25.071826+0000","last_peered":"2024-08-31T06:05:25.071826+0000","last_clean":"2024-08-31T06:05:25.071826+0000","last_became_active":"2024-08-31T06:05:25.057546+0000","last_became_peered":"2024-08-31T06:05:25.057546+0000","last_unstale":"2024-08-31T06:05:25.071826+0000","last_undegraded":"2024-08-31T06:05:25.071826+0000","last_fullsized":"2024-08-31T06:05:25.071826+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:42:35.527395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.681869+0000","last_change":"2024-08-31T06:04:25.887580+0000","last_active":"2024-08-31T06:06:23.681869+0000","last_peered":"2024-08-31T06:06:23.681869+0000","last_clean":"2024-08-31T06:06:23.681869+0000","last_became_active":"2024-08-31T06:04:25.887158+0000","last_became_peered":"2024-08-31T06:04:25.887158+0000","last_unstale":"2024-08-31T06:06:23.681869+0000","last_undegraded":"2024-08-31T06:06:23.681869+0000","last_fullsized":"2024-08-31T06:06:23.681869+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:27:38.115742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.064443+0000","last_change":"2024-08-31T06:05:25.063859+0000","last_active":"2024-08-31T06:05:25.064443+0000","last_peered":"2024-08-31T06:05:25.064443+0000","last_clean":"2024-08-31T06:05:25.064443+0000","last_became_active":"2024-08-31T06:05:25.053596+0000","last_became_peered":"2024-08-31T06:05:25.053596+0000","last_unstale":"2024-08-31T06:05:25.064443+0000","last_undegraded":"2024-08-31T06:05:25.064443+0000","last_fullsized":"2024-08-31T06:05:25.064443+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:19:18.461630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.678741+0000","last_change":"2024-08-31T06:04:25.880845+0000","last_active":"2024-08-31T06:06:23.678741+0000","last_peered":"2024-08-31T06:06:23.678741+0000","last_clean":"2024-08-31T06:06:23.678741+0000","last_became_active":"2024-08-31T06:04:25.880423+0000","last_became_peered":"2024-08-31T06:04:25.880423+0000","last_unstale":"2024-08-31T06:06:23.678741+0000","last_undegraded":"2024-08-31T06:06:23.678741+0000","last_fullsized":"2024-08-31T06:06:23.678741+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:16:56.725116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.046584+0000","last_change":"2024-08-31T06:04:22.843393+0000","last_active":"2024-08-31T06:05:25.046584+0000","last_peered":"2024-08-31T06:05:25.046584+0000","last_clean":"2024-08-31T06:05:25.046584+0000","last_became_active":"2024-08-31T06:04:22.842770+0000","last_became_peered":"2024-08-31T06:04:22.842770+0000","last_unstale":"2024-08-31T06:05:25.046584+0000","last_undegraded":"2024-08-31T06:05:25.046584+0000","last_fullsized":"2024-08-31T06:05:25.046584+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:49:16.159167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.686211+0000","last_change":"2024-08-31T06:04:25.887829+0000","last_active":"2024-08-31T06:06:23.686211+0000","last_peered":"2024-08-31T06:06:23.686211+0000","last_clean":"2024-08-31T06:06:23.686211+0000","last_became_active":"2024-08-31T06:04:25.886567+0000","last_became_peered":"2024-08-31T06:04:25.886567+0000","last_unstale":"2024-08-31T06:06:23.686211+0000","last_undegraded":"2024-08-31T06:06:23.686211+0000","last_fullsized":"2024-08-31T06:06:23.686211+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:11:39.054966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.045627+0000","last_change":"2024-08-31T06:04:22.844218+0000","last_active":"2024-08-31T06:05:25.045627+0000","last_peered":"2024-08-31T06:05:25.045627+0000","last_clean":"2024-08-31T06:05:25.045627+0000","last_became_active":"2024-08-31T06:04:22.843786+0000","last_became_peered":"2024-08-31T06:04:22.843786+0000","last_unstale":"2024-08-31T06:05:25.045627+0000","last_undegraded":"2024-08-31T06:05:25.045627+0000","last_fullsized":"2024-08-31T06:05:25.045627+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:46:02.534190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.682380+0000","last_change":"2024-08-31T06:04:25.888946+0000","last_active":"2024-08-31T06:06:23.682380+0000","last_peered":"2024-08-31T06:06:23.682380+0000","last_clean":"2024-08-31T06:06:23.682380+0000","last_became_active":"2024-08-31T06:04:25.888564+0000","last_became_peered":"2024-08-31T06:04:25.888564+0000","last_unstale":"2024-08-31T06:06:23.682380+0000","last_undegraded":"2024-08-31T06:06:23.682380+0000","last_fullsized":"2024-08-31T06:06:23.682380+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:26:19.051971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.049211+0000","last_change":"2024-08-31T06:04:22.964483+0000","last_active":"2024-08-31T06:05:25.049211+0000","last_peered":"2024-08-31T06:05:25.049211+0000","last_clean":"2024-08-31T06:05:25.049211+0000","last_became_active":"2024-08-31T06:04:22.963874+0000","last_became_peered":"2024-08-31T06:04:22.963874+0000","last_unstale":"2024-08-31T06:05:25.049211+0000","last_undegraded":"2024-08-31T06:05:25.049211+0000","last_fullsized":"2024-08-31T06:05:25.049211+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:30:00.159040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.688990+0000","last_change":"2024-08-31T06:04:25.880941+0000","last_active":"2024-08-31T06:06:23.688990+0000","last_peered":"2024-08-31T06:06:23.688990+0000","last_clean":"2024-08-31T06:06:23.688990+0000","last_became_active":"2024-08-31T06:04:25.880454+0000","last_became_peered":"2024-08-31T06:04:25.880454+0000","last_unstale":"2024-08-31T06:06:23.688990+0000","last_undegraded":"2024-08-31T06:06:23.688990+0000","last_fullsized":"2024-08-31T06:06:23.688990+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:49:57.023988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":98,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048527+0000","last_change":"2024-08-31T06:03:05.921629+0000","last_active":"2024-08-31T06:05:25.048527+0000","last_peered":"2024-08-31T06:05:25.048527+0000","last_clean":"2024-08-31T06:05:25.048527+0000","last_became_active":"2024-08-31T06:03:05.915844+0000","last_became_peered":"2024-08-31T06:03:05.915844+0000","last_unstale":"2024-08-31T06:05:25.048527+0000","last_undegraded":"2024-08-31T06:05:25.048527+0000","last_fullsized":"2024-08-31T06:05:25.048527+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T05:59:24.472429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T05:59:24.472429+0000","last_clean_scrub_stamp":"2024-08-31T05:59:24.472429+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:56:41.073422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.044718+0000","last_change":"2024-08-31T06:04:22.846325+0000","last_active":"2024-08-31T06:05:25.044718+0000","last_peered":"2024-08-31T06:05:25.044718+0000","last_clean":"2024-08-31T06:05:25.044718+0000","last_became_active":"2024-08-31T06:04:22.845772+0000","last_became_peered":"2024-08-31T06:04:22.845772+0000","last_unstale":"2024-08-31T06:05:25.044718+0000","last_undegraded":"2024-08-31T06:05:25.044718+0000","last_fullsized":"2024-08-31T06:05:25.044718+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:54:10.336786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.691294+0000","last_change":"2024-08-31T06:04:25.891437+0000","last_active":"2024-08-31T06:06:23.691294+0000","last_peered":"2024-08-31T06:06:23.691294+0000","last_clean":"2024-08-31T06:06:23.691294+0000","last_became_active":"2024-08-31T06:04:25.891053+0000","last_became_peered":"2024-08-31T06:04:25.891053+0000","last_unstale":"2024-08-31T06:06:23.691294+0000","last_undegraded":"2024-08-31T06:06:23.691294+0000","last_fullsized":"2024-08-31T06:06:23.691294+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:22:37.119483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048411+0000","last_change":"2024-08-31T06:04:22.842885+0000","last_active":"2024-08-31T06:05:25.048411+0000","last_peered":"2024-08-31T06:05:25.048411+0000","last_clean":"2024-08-31T06:05:25.048411+0000","last_became_active":"2024-08-31T06:04:22.842473+0000","last_became_peered":"2024-08-31T06:04:22.842473+0000","last_unstale":"2024-08-31T06:05:25.048411+0000","last_undegraded":"2024-08-31T06:05:25.048411+0000","last_fullsized":"2024-08-31T06:05:25.048411+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:33:36.516659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.663699+0000","last_change":"2024-08-31T06:04:25.889213+0000","last_active":"2024-08-31T06:06:23.663699+0000","last_peered":"2024-08-31T06:06:23.663699+0000","last_clean":"2024-08-31T06:06:23.663699+0000","last_became_active":"2024-08-31T06:04:25.888860+0000","last_became_peered":"2024-08-31T06:04:25.888860+0000","last_unstale":"2024-08-31T06:06:23.663699+0000","last_undegraded":"2024-08-31T06:06:23.663699+0000","last_fullsized":"2024-08-31T06:06:23.663699+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:44:18.831229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":152,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:38.720896+0000","last_change":"2024-08-31T06:04:22.966091+0000","last_active":"2024-08-31T06:06:38.720896+0000","last_peered":"2024-08-31T06:06:38.720896+0000","last_clean":"2024-08-31T06:06:38.720896+0000","last_became_active":"2024-08-31T06:04:22.965727+0000","last_became_peered":"2024-08-31T06:04:22.965727+0000","last_unstale":"2024-08-31T06:06:38.720896+0000","last_undegraded":"2024-08-31T06:06:38.720896+0000","last_fullsized":"2024-08-31T06:06:38.720896+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:42:43.643623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.682138+0000","last_change":"2024-08-31T06:04:25.880039+0000","last_active":"2024-08-31T06:06:23.682138+0000","last_peered":"2024-08-31T06:06:23.682138+0000","last_clean":"2024-08-31T06:06:23.682138+0000","last_became_active":"2024-08-31T06:04:25.879669+0000","last_became_peered":"2024-08-31T06:04:25.879669+0000","last_unstale":"2024-08-31T06:06:23.682138+0000","last_undegraded":"2024-08-31T06:06:23.682138+0000","last_fullsized":"2024-08-31T06:06:23.682138+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:02:09.244281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048687+0000","last_change":"2024-08-31T06:04:22.844572+0000","last_active":"2024-08-31T06:05:25.048687+0000","last_peered":"2024-08-31T06:05:25.048687+0000","last_clean":"2024-08-31T06:05:25.048687+0000","last_became_active":"2024-08-31T06:04:22.844227+0000","last_became_peered":"2024-08-31T06:04:22.844227+0000","last_unstale":"2024-08-31T06:05:25.048687+0000","last_undegraded":"2024-08-31T06:05:25.048687+0000","last_fullsized":"2024-08-31T06:05:25.048687+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:04:51.966901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.677043+0000","last_change":"2024-08-31T06:04:25.884442+0000","last_active":"2024-08-31T06:06:23.677043+0000","last_peered":"2024-08-31T06:06:23.677043+0000","last_clean":"2024-08-31T06:06:23.677043+0000","last_became_active":"2024-08-31T06:04:25.884016+0000","last_became_peered":"2024-08-31T06:04:25.884016+0000","last_unstale":"2024-08-31T06:06:23.677043+0000","last_undegraded":"2024-08-31T06:06:23.677043+0000","last_fullsized":"2024-08-31T06:06:23.677043+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:36:10.640262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.043831+0000","last_change":"2024-08-31T06:04:22.967845+0000","last_active":"2024-08-31T06:05:25.043831+0000","last_peered":"2024-08-31T06:05:25.043831+0000","last_clean":"2024-08-31T06:05:25.043831+0000","last_became_active":"2024-08-31T06:04:22.967421+0000","last_became_peered":"2024-08-31T06:04:22.967421+0000","last_unstale":"2024-08-31T06:05:25.043831+0000","last_undegraded":"2024-08-31T06:05:25.043831+0000","last_fullsized":"2024-08-31T06:05:25.043831+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:17:46.942551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.673410+0000","last_change":"2024-08-31T06:04:25.897087+0000","last_active":"2024-08-31T06:06:23.673410+0000","last_peered":"2024-08-31T06:06:23.673410+0000","last_clean":"2024-08-31T06:06:23.673410+0000","last_became_active":"2024-08-31T06:04:25.896725+0000","last_became_peered":"2024-08-31T06:04:25.896725+0000","last_unstale":"2024-08-31T06:06:23.673410+0000","last_undegraded":"2024-08-31T06:06:23.673410+0000","last_fullsized":"2024-08-31T06:06:23.673410+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:31:28.279489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.053673+0000","last_change":"2024-08-31T06:04:22.965299+0000","last_active":"2024-08-31T06:05:25.053673+0000","last_peered":"2024-08-31T06:05:25.053673+0000","last_clean":"2024-08-31T06:05:25.053673+0000","last_became_active":"2024-08-31T06:04:22.964849+0000","last_became_peered":"2024-08-31T06:04:22.964849+0000","last_unstale":"2024-08-31T06:05:25.053673+0000","last_undegraded":"2024-08-31T06:05:25.053673+0000","last_fullsized":"2024-08-31T06:05:25.053673+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:36:56.431678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.679635+0000","last_change":"2024-08-31T06:04:25.885115+0000","last_active":"2024-08-31T06:06:23.679635+0000","last_peered":"2024-08-31T06:06:23.679635+0000","last_clean":"2024-08-31T06:06:23.679635+0000","last_became_active":"2024-08-31T06:04:25.884722+0000","last_became_peered":"2024-08-31T06:04:25.884722+0000","last_unstale":"2024-08-31T06:06:23.679635+0000","last_undegraded":"2024-08-31T06:06:23.679635+0000","last_fullsized":"2024-08-31T06:06:23.679635+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:20:43.913785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'13","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:32.962890+0000","last_change":"2024-08-31T06:04:22.845119+0000","last_active":"2024-08-31T06:06:32.962890+0000","last_peered":"2024-08-31T06:06:32.962890+0000","last_clean":"2024-08-31T06:06:32.962890+0000","last_became_active":"2024-08-31T06:04:22.843814+0000","last_became_peered":"2024-08-31T06:04:22.843814+0000","last_unstale":"2024-08-31T06:06:32.962890+0000","last_undegraded":"2024-08-31T06:06:32.962890+0000","last_fullsized":"2024-08-31T06:06:32.962890+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:06:40.684887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.680844+0000","last_change":"2024-08-31T06:04:25.892838+0000","last_active":"2024-08-31T06:06:23.680844+0000","last_peered":"2024-08-31T06:06:23.680844+0000","last_clean":"2024-08-31T06:06:23.680844+0000","last_became_active":"2024-08-31T06:04:25.891955+0000","last_became_peered":"2024-08-31T06:04:25.891955+0000","last_unstale":"2024-08-31T06:06:23.680844+0000","last_undegraded":"2024-08-31T06:06:23.680844+0000","last_fullsized":"2024-08-31T06:06:23.680844+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:41:35.085724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.044957+0000","last_change":"2024-08-31T06:04:22.964183+0000","last_active":"2024-08-31T06:05:25.044957+0000","last_peered":"2024-08-31T06:05:25.044957+0000","last_clean":"2024-08-31T06:05:25.044957+0000","last_became_active":"2024-08-31T06:04:22.963688+0000","last_became_peered":"2024-08-31T06:04:22.963688+0000","last_unstale":"2024-08-31T06:05:25.044957+0000","last_undegraded":"2024-08-31T06:05:25.044957+0000","last_fullsized":"2024-08-31T06:05:25.044957+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:04:47.833811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.683320+0000","last_change":"2024-08-31T06:04:25.892266+0000","last_active":"2024-08-31T06:06:23.683320+0000","last_peered":"2024-08-31T06:06:23.683320+0000","last_clean":"2024-08-31T06:06:23.683320+0000","last_became_active":"2024-08-31T06:04:25.889014+0000","last_became_peered":"2024-08-31T06:04:25.889014+0000","last_unstale":"2024-08-31T06:06:23.683320+0000","last_undegraded":"2024-08-31T06:06:23.683320+0000","last_fullsized":"2024-08-31T06:06:23.683320+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:27:47.608470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.049408+0000","last_change":"2024-08-31T06:04:22.964677+0000","last_active":"2024-08-31T06:05:25.049408+0000","last_peered":"2024-08-31T06:05:25.049408+0000","last_clean":"2024-08-31T06:05:25.049408+0000","last_became_active":"2024-08-31T06:04:22.964237+0000","last_became_peered":"2024-08-31T06:04:22.964237+0000","last_unstale":"2024-08-31T06:05:25.049408+0000","last_undegraded":"2024-08-31T06:05:25.049408+0000","last_fullsized":"2024-08-31T06:05:25.049408+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:00:54.099592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.648239+0000","last_change":"2024-08-31T06:04:25.881699+0000","last_active":"2024-08-31T06:06:23.648239+0000","last_peered":"2024-08-31T06:06:23.648239+0000","last_clean":"2024-08-31T06:06:23.648239+0000","last_became_active":"2024-08-31T06:04:25.879304+0000","last_became_peered":"2024-08-31T06:04:25.879304+0000","last_unstale":"2024-08-31T06:06:23.648239+0000","last_undegraded":"2024-08-31T06:06:23.648239+0000","last_fullsized":"2024-08-31T06:06:23.648239+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:56:42.992034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.108301+0000","last_change":"2024-08-31T06:04:22.847934+0000","last_active":"2024-08-31T06:05:25.108301+0000","last_peered":"2024-08-31T06:05:25.108301+0000","last_clean":"2024-08-31T06:05:25.108301+0000","last_became_active":"2024-08-31T06:04:22.847518+0000","last_became_peered":"2024-08-31T06:04:22.847518+0000","last_unstale":"2024-08-31T06:05:25.108301+0000","last_undegraded":"2024-08-31T06:05:25.108301+0000","last_fullsized":"2024-08-31T06:05:25.108301+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:26:21.476014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.685828+0000","last_change":"2024-08-31T06:04:25.884653+0000","last_active":"2024-08-31T06:06:23.685828+0000","last_peered":"2024-08-31T06:06:23.685828+0000","last_clean":"2024-08-31T06:06:23.685828+0000","last_became_active":"2024-08-31T06:04:25.884244+0000","last_became_peered":"2024-08-31T06:04:25.884244+0000","last_unstale":"2024-08-31T06:06:23.685828+0000","last_undegraded":"2024-08-31T06:06:23.685828+0000","last_fullsized":"2024-08-31T06:06:23.685828+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:13:00.814071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048648+0000","last_change":"2024-08-31T06:04:22.845924+0000","last_active":"2024-08-31T06:05:25.048648+0000","last_peered":"2024-08-31T06:05:25.048648+0000","last_clean":"2024-08-31T06:05:25.048648+0000","last_became_active":"2024-08-31T06:04:22.845445+0000","last_became_peered":"2024-08-31T06:04:22.845445+0000","last_unstale":"2024-08-31T06:05:25.048648+0000","last_undegraded":"2024-08-31T06:05:25.048648+0000","last_fullsized":"2024-08-31T06:05:25.048648+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:25:21.052136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.679162+0000","last_change":"2024-08-31T06:04:25.890069+0000","last_active":"2024-08-31T06:06:23.679162+0000","last_peered":"2024-08-31T06:06:23.679162+0000","last_clean":"2024-08-31T06:06:23.679162+0000","last_became_active":"2024-08-31T06:04:25.889709+0000","last_became_peered":"2024-08-31T06:04:25.889709+0000","last_unstale":"2024-08-31T06:06:23.679162+0000","last_undegraded":"2024-08-31T06:06:23.679162+0000","last_fullsized":"2024-08-31T06:06:23.679162+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:13:51.360023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.051611+0000","last_change":"2024-08-31T06:04:22.847845+0000","last_active":"2024-08-31T06:05:25.051611+0000","last_peered":"2024-08-31T06:05:25.051611+0000","last_clean":"2024-08-31T06:05:25.051611+0000","last_became_active":"2024-08-31T06:04:22.847001+0000","last_became_peered":"2024-08-31T06:04:22.847001+0000","last_unstale":"2024-08-31T06:05:25.051611+0000","last_undegraded":"2024-08-31T06:05:25.051611+0000","last_fullsized":"2024-08-31T06:05:25.051611+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:24:34.209076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.690813+0000","last_change":"2024-08-31T06:04:25.886759+0000","last_active":"2024-08-31T06:06:23.690813+0000","last_peered":"2024-08-31T06:06:23.690813+0000","last_clean":"2024-08-31T06:06:23.690813+0000","last_became_active":"2024-08-31T06:04:25.886176+0000","last_became_peered":"2024-08-31T06:04:25.886176+0000","last_unstale":"2024-08-31T06:06:23.690813+0000","last_undegraded":"2024-08-31T06:06:23.690813+0000","last_fullsized":"2024-08-31T06:06:23.690813+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:11:12.637451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"80'15","reported_seq":43,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.677867+0000","last_change":"2024-08-31T06:05:25.054468+0000","last_active":"2024-08-31T06:06:23.677867+0000","last_peered":"2024-08-31T06:06:23.677867+0000","last_clean":"2024-08-31T06:06:23.677867+0000","last_became_active":"2024-08-31T06:05:25.053657+0000","last_became_peered":"2024-08-31T06:05:25.053657+0000","last_unstale":"2024-08-31T06:06:23.677867+0000","last_undegraded":"2024-08-31T06:06:23.677867+0000","last_fullsized":"2024-08-31T06:06:23.677867+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:43:44.887637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.059757+0000","last_change":"2024-08-31T06:05:25.059219+0000","last_active":"2024-08-31T06:05:25.059757+0000","last_peered":"2024-08-31T06:05:25.059757+0000","last_clean":"2024-08-31T06:05:25.059757+0000","last_became_active":"2024-08-31T06:05:25.053136+0000","last_became_peered":"2024-08-31T06:05:25.053136+0000","last_unstale":"2024-08-31T06:05:25.059757+0000","last_undegraded":"2024-08-31T06:05:25.059757+0000","last_fullsized":"2024-08-31T06:05:25.059757+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:29:59.137465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.691662+0000","last_change":"2024-08-31T06:04:25.887524+0000","last_active":"2024-08-31T06:06:23.691662+0000","last_peered":"2024-08-31T06:06:23.691662+0000","last_clean":"2024-08-31T06:06:23.691662+0000","last_became_active":"2024-08-31T06:04:25.887176+0000","last_became_peered":"2024-08-31T06:04:25.887176+0000","last_unstale":"2024-08-31T06:06:23.691662+0000","last_undegraded":"2024-08-31T06:06:23.691662+0000","last_fullsized":"2024-08-31T06:06:23.691662+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:08:05.476804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.053524+0000","last_change":"2024-08-31T06:04:22.847747+0000","last_active":"2024-08-31T06:05:25.053524+0000","last_peered":"2024-08-31T06:05:25.053524+0000","last_clean":"2024-08-31T06:05:25.053524+0000","last_became_active":"2024-08-31T06:04:22.847332+0000","last_became_peered":"2024-08-31T06:04:22.847332+0000","last_unstale":"2024-08-31T06:05:25.053524+0000","last_undegraded":"2024-08-31T06:05:25.053524+0000","last_fullsized":"2024-08-31T06:05:25.053524+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:41:35.910991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.687796+0000","last_change":"2024-08-31T06:04:25.883110+0000","last_active":"2024-08-31T06:06:23.687796+0000","last_peered":"2024-08-31T06:06:23.687796+0000","last_clean":"2024-08-31T06:06:23.687796+0000","last_became_active":"2024-08-31T06:04:25.880487+0000","last_became_peered":"2024-08-31T06:04:25.880487+0000","last_unstale":"2024-08-31T06:06:23.687796+0000","last_undegraded":"2024-08-31T06:06:23.687796+0000","last_fullsized":"2024-08-31T06:06:23.687796+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:22:13.656078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.108210+0000","last_change":"2024-08-31T06:04:22.964758+0000","last_active":"2024-08-31T06:05:25.108210+0000","last_peered":"2024-08-31T06:05:25.108210+0000","last_clean":"2024-08-31T06:05:25.108210+0000","last_became_active":"2024-08-31T06:04:22.964391+0000","last_became_peered":"2024-08-31T06:04:22.964391+0000","last_unstale":"2024-08-31T06:05:25.108210+0000","last_undegraded":"2024-08-31T06:05:25.108210+0000","last_fullsized":"2024-08-31T06:05:25.108210+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:28:59.190657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.678294+0000","last_change":"2024-08-31T06:04:25.893738+0000","last_active":"2024-08-31T06:06:23.678294+0000","last_peered":"2024-08-31T06:06:23.678294+0000","last_clean":"2024-08-31T06:06:23.678294+0000","last_became_active":"2024-08-31T06:04:25.893308+0000","last_became_peered":"2024-08-31T06:04:25.893308+0000","last_unstale":"2024-08-31T06:06:23.678294+0000","last_undegraded":"2024-08-31T06:06:23.678294+0000","last_fullsized":"2024-08-31T06:06:23.678294+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:26:48.451894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":38,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.043990+0000","last_change":"2024-08-31T06:04:22.843346+0000","last_active":"2024-08-31T06:05:25.043990+0000","last_peered":"2024-08-31T06:05:25.043990+0000","last_clean":"2024-08-31T06:05:25.043990+0000","last_became_active":"2024-08-31T06:04:22.842249+0000","last_became_peered":"2024-08-31T06:04:22.842249+0000","last_unstale":"2024-08-31T06:05:25.043990+0000","last_undegraded":"2024-08-31T06:05:25.043990+0000","last_fullsized":"2024-08-31T06:05:25.043990+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:40:57.618940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70369,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":742,"num_read_kb":449,"num_write":178,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1147,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":211107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":249373,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710761,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":436463,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874285,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":1094149,"data_compressed":42124,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037813,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":436677,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234042,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":436575,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397568,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":933888,"data_stored":1029200,"data_compressed":42124,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593797,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":436066,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790025,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":1028352,"data_compressed":42124,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953552,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36208,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708944,"statfs":{"total":95995035648,"available":95957958656,"internally_reserved":0,"allocated":884736,"data_stored":438189,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149780,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":888832,"data_stored":436377,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411419,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36268,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708884,"statfs":{"total":95995035648,"available":95957897216,"internally_reserved":0,"allocated":946176,"data_stored":502170,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542239,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36268,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708884,"statfs":{"total":95995035648,"available":95957897216,"internally_reserved":0,"allocated":946176,"data_stored":501666,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705765,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":892928,"data_stored":437337,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34699,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34699,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34699,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":65807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":66311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":67805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-31T06:06:43.400 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-08-31T06:06:43.400 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-31T06:06:43.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.658+0000 7fdb1b600640 1 -- 172.21.15.16:0/1155635355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb141091a0 msgr2=0x7fdb141095b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:43.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.658+0000 7fdb1b600640 1 --2- 172.21.15.16:0/1155635355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb141091a0 0x7fdb141095b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fdb00009900 tx=0x7fdb0002a410 comp rx=0 tx=0).stop 2024-08-31T06:06:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.658+0000 7fdb1b600640 1 -- 172.21.15.16:0/1155635355 shutdown_connections 2024-08-31T06:06:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.658+0000 7fdb1b600640 1 --2- 172.21.15.16:0/1155635355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb14109af0 0x7fdb14110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.658+0000 7fdb1b600640 1 --2- 172.21.15.16:0/1155635355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb141091a0 0x7fdb141095b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.658+0000 7fdb1b600640 1 --2- 172.21.15.16:0/1155635355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb14103870 0x7fdb14103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.658+0000 7fdb1b600640 1 -- 172.21.15.16:0/1155635355 >> 172.21.15.16:0/1155635355 conn(0x7fdb140ff2b0 msgr2=0x7fdb14101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.658+0000 7fdb1b600640 1 -- 172.21.15.16:0/1155635355 shutdown_connections 2024-08-31T06:06:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.658+0000 7fdb1b600640 1 -- 172.21.15.16:0/1155635355 wait complete. 2024-08-31T06:06:43.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.659+0000 7fdb1b600640 1 Processor -- start 2024-08-31T06:06:43.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.659+0000 7fdb1b600640 1 -- start start 2024-08-31T06:06:43.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.659+0000 7fdb1b600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb14103870 0x7fdb1419c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:43.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.659+0000 7fdb1b600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb141091a0 0x7fdb1419c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:43.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.659+0000 7fdb1b600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb14109af0 0x7fdb141a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:43.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.659+0000 7fdb1b600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdb141137b0 con 0x7fdb14103870 2024-08-31T06:06:43.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.659+0000 7fdb1b600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdb14113630 con 0x7fdb141091a0 2024-08-31T06:06:43.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.659+0000 7fdb1b600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdb14113930 con 0x7fdb14109af0 2024-08-31T06:06:43.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.660+0000 7fdb18c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb14103870 0x7fdb1419c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:43.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.660+0000 7fdb13e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb141091a0 0x7fdb1419c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:43.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.660+0000 7fdb18c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb14103870 0x7fdb1419c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50894/0 (socket says 172.21.15.16:50894) 2024-08-31T06:06:43.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.660+0000 7fdb18c00640 1 -- 172.21.15.16:0/2622323798 learned_addr learned my addr 172.21.15.16:0/2622323798 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:43.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.660+0000 7fdb19600640 1 --2- 172.21.15.16:0/2622323798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb14109af0 0x7fdb141a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:43.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.660+0000 7fdb18c00640 1 -- 172.21.15.16:0/2622323798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb14109af0 msgr2=0x7fdb141a1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:43.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.660+0000 7fdb18c00640 1 --2- 172.21.15.16:0/2622323798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb14109af0 0x7fdb141a1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:43.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.660+0000 7fdb18c00640 1 -- 172.21.15.16:0/2622323798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb141091a0 msgr2=0x7fdb1419c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:43.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.660+0000 7fdb18c00640 1 --2- 172.21.15.16:0/2622323798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb141091a0 0x7fdb1419c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:43.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.660+0000 7fdb18c00640 1 -- 172.21.15.16:0/2622323798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb141a2300 con 0x7fdb14103870 2024-08-31T06:06:43.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.660+0000 7fdb13e00640 1 --2- 172.21.15.16:0/2622323798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb141091a0 0x7fdb1419c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:06:43.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:43 smithi016 ceph-mon[17194]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T06:06:43.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.661+0000 7fdb18c00640 1 --2- 172.21.15.16:0/2622323798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb14103870 0x7fdb1419c3d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdb0800e9f0 tx=0x7fdb0800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:43.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.661+0000 7fdb11600640 1 -- 172.21.15.16:0/2622323798 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0800cd80 con 0x7fdb14103870 2024-08-31T06:06:43.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.661+0000 7fdb11600640 1 -- 172.21.15.16:0/2622323798 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb080108e0 con 0x7fdb14103870 2024-08-31T06:06:43.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.661+0000 7fdb1b600640 1 -- 172.21.15.16:0/2622323798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb141a25f0 con 0x7fdb14103870 2024-08-31T06:06:43.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.661+0000 7fdb11600640 1 -- 172.21.15.16:0/2622323798 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb08018980 con 0x7fdb14103870 2024-08-31T06:06:43.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.661+0000 7fdb1b600640 1 -- 172.21.15.16:0/2622323798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb141a8d80 con 0x7fdb14103870 2024-08-31T06:06:43.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.662+0000 7fdb1b600640 1 -- 172.21.15.16:0/2622323798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb14104ae0 con 0x7fdb14103870 2024-08-31T06:06:43.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.664+0000 7fdb11600640 1 -- 172.21.15.16:0/2622323798 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7fdb08010430 con 0x7fdb14103870 2024-08-31T06:06:43.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.666+0000 7fdb11600640 1 --2- 172.21.15.16:0/2622323798 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdafc0b4b80 0x7fdafc0b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:43.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.666+0000 7fdb11600640 1 -- 172.21.15.16:0/2622323798 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7fdb080e55c0 con 0x7fdb14103870 2024-08-31T06:06:43.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.666+0000 7fdb13e00640 1 --2- 172.21.15.16:0/2622323798 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdafc0b4b80 0x7fdafc0b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:43.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.667+0000 7fdb13e00640 1 --2- 172.21.15.16:0/2622323798 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdafc0b4b80 0x7fdafc0b7090 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdb0002aa00 tx=0x7fdb00005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:43.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.669+0000 7fdb11600640 1 -- 172.21.15.16:0/2622323798 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fdb080abf80 con 0x7fdb14103870 2024-08-31T06:06:43.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.864+0000 7fdb1b600640 1 -- 172.21.15.16:0/2622323798 --> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdb1410f5f0 con 0x7fdafc0b4b80 2024-08-31T06:06:43.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.869+0000 7fdb11600640 1 -- 172.21.15.16:0/2622323798 <== mgr.14220 v2:172.21.15.16:6800/4279751337 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259063 (secure 0 0 0) 0x7fdb1410f5f0 con 0x7fdafc0b4b80 2024-08-31T06:06:43.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:43 smithi155 ceph-mon[21742]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T06:06:43.874 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:06:43.876 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-08-31T06:06:43.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.873+0000 7fdb1b600640 1 -- 172.21.15.16:0/2622323798 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdafc0b4b80 msgr2=0x7fdafc0b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:43.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.873+0000 7fdb1b600640 1 --2- 172.21.15.16:0/2622323798 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdafc0b4b80 0x7fdafc0b7090 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdb0002aa00 tx=0x7fdb00005a20 comp rx=0 tx=0).stop 2024-08-31T06:06:43.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.873+0000 7fdb1b600640 1 -- 172.21.15.16:0/2622323798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb14103870 msgr2=0x7fdb1419c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:43.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.873+0000 7fdb1b600640 1 --2- 172.21.15.16:0/2622323798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb14103870 0x7fdb1419c3d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdb0800e9f0 tx=0x7fdb0800eeb0 comp rx=0 tx=0).stop 2024-08-31T06:06:43.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.874+0000 7fdb1b600640 1 -- 172.21.15.16:0/2622323798 shutdown_connections 2024-08-31T06:06:43.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.874+0000 7fdb1b600640 1 --2- 172.21.15.16:0/2622323798 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdafc0b4b80 0x7fdafc0b7090 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:43.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.874+0000 7fdb1b600640 1 --2- 172.21.15.16:0/2622323798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb14109af0 0x7fdb141a1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:43.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.874+0000 7fdb1b600640 1 --2- 172.21.15.16:0/2622323798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb141091a0 0x7fdb1419c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:43.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.874+0000 7fdb1b600640 1 --2- 172.21.15.16:0/2622323798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb14103870 0x7fdb1419c3d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:43.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.874+0000 7fdb1b600640 1 -- 172.21.15.16:0/2622323798 >> 172.21.15.16:0/2622323798 conn(0x7fdb140ff2b0 msgr2=0x7fdb14108800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:43.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.874+0000 7fdb1b600640 1 -- 172.21.15.16:0/2622323798 shutdown_connections 2024-08-31T06:06:43.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:43.874+0000 7fdb1b600640 1 -- 172.21.15.16:0/2622323798 wait complete. 2024-08-31T06:06:43.900 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":348,"stamp":"2024-08-31T06:06:43.291186+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660971,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":823,"num_read_kb":518,"num_write":303,"num_write_kb":2278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":2952987,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":745,"ondisk_log_size":745,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372444,"kb_used_data":10716,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124569380,"statfs":{"total":1151940427776,"available":1151559045120,"internally_reserved":0,"allocated":10973184,"data_stored":7213221,"data_compressed":178617,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5684,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":37,"num_write":4,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003164"},"pg_stats":[{"pgid":"3.3f","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.686021+0000","last_change":"2024-08-31T06:04:25.877318+0000","last_active":"2024-08-31T06:06:23.686021+0000","last_peered":"2024-08-31T06:06:23.686021+0000","last_clean":"2024-08-31T06:06:23.686021+0000","last_became_active":"2024-08-31T06:04:25.876894+0000","last_became_peered":"2024-08-31T06:04:25.876894+0000","last_unstale":"2024-08-31T06:06:23.686021+0000","last_undegraded":"2024-08-31T06:06:23.686021+0000","last_fullsized":"2024-08-31T06:06:23.686021+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:41:09.449343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"80'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.685244+0000","last_change":"2024-08-31T06:04:25.885670+0000","last_active":"2024-08-31T06:06:23.685244+0000","last_peered":"2024-08-31T06:06:23.685244+0000","last_clean":"2024-08-31T06:06:23.685244+0000","last_became_active":"2024-08-31T06:04:25.883818+0000","last_became_peered":"2024-08-31T06:04:25.883818+0000","last_unstale":"2024-08-31T06:06:23.685244+0000","last_undegraded":"2024-08-31T06:06:23.685244+0000","last_fullsized":"2024-08-31T06:06:23.685244+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:40:01.344432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.687803+0000","last_change":"2024-08-31T06:04:25.883375+0000","last_active":"2024-08-31T06:06:23.687803+0000","last_peered":"2024-08-31T06:06:23.687803+0000","last_clean":"2024-08-31T06:06:23.687803+0000","last_became_active":"2024-08-31T06:04:25.881750+0000","last_became_peered":"2024-08-31T06:04:25.881750+0000","last_unstale":"2024-08-31T06:06:23.687803+0000","last_undegraded":"2024-08-31T06:06:23.687803+0000","last_fullsized":"2024-08-31T06:06:23.687803+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:46:46.757363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.676625+0000","last_change":"2024-08-31T06:04:25.882436+0000","last_active":"2024-08-31T06:06:23.676625+0000","last_peered":"2024-08-31T06:06:23.676625+0000","last_clean":"2024-08-31T06:06:23.676625+0000","last_became_active":"2024-08-31T06:04:25.881805+0000","last_became_peered":"2024-08-31T06:04:25.881805+0000","last_unstale":"2024-08-31T06:06:23.676625+0000","last_undegraded":"2024-08-31T06:06:23.676625+0000","last_fullsized":"2024-08-31T06:06:23.676625+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:42:39.793625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"80'4","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.661470+0000","last_change":"2024-08-31T06:04:25.894534+0000","last_active":"2024-08-31T06:06:23.661470+0000","last_peered":"2024-08-31T06:06:23.661470+0000","last_clean":"2024-08-31T06:06:23.661470+0000","last_became_active":"2024-08-31T06:04:25.894116+0000","last_became_peered":"2024-08-31T06:04:25.894116+0000","last_unstale":"2024-08-31T06:06:23.661470+0000","last_undegraded":"2024-08-31T06:06:23.661470+0000","last_fullsized":"2024-08-31T06:06:23.661470+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:02:31.257454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.669411+0000","last_change":"2024-08-31T06:04:25.886182+0000","last_active":"2024-08-31T06:06:23.669411+0000","last_peered":"2024-08-31T06:06:23.669411+0000","last_clean":"2024-08-31T06:06:23.669411+0000","last_became_active":"2024-08-31T06:04:25.885825+0000","last_became_peered":"2024-08-31T06:04:25.885825+0000","last_unstale":"2024-08-31T06:06:23.669411+0000","last_undegraded":"2024-08-31T06:06:23.669411+0000","last_fullsized":"2024-08-31T06:06:23.669411+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:58:12.427931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.682008+0000","last_change":"2024-08-31T06:04:25.877521+0000","last_active":"2024-08-31T06:06:23.682008+0000","last_peered":"2024-08-31T06:06:23.682008+0000","last_clean":"2024-08-31T06:06:23.682008+0000","last_became_active":"2024-08-31T06:04:25.877122+0000","last_became_peered":"2024-08-31T06:04:25.877122+0000","last_unstale":"2024-08-31T06:06:23.682008+0000","last_undegraded":"2024-08-31T06:06:23.682008+0000","last_fullsized":"2024-08-31T06:06:23.682008+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:17:27.406828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.680720+0000","last_change":"2024-08-31T06:04:25.892547+0000","last_active":"2024-08-31T06:06:23.680720+0000","last_peered":"2024-08-31T06:06:23.680720+0000","last_clean":"2024-08-31T06:06:23.680720+0000","last_became_active":"2024-08-31T06:04:25.892090+0000","last_became_peered":"2024-08-31T06:04:25.892090+0000","last_unstale":"2024-08-31T06:06:23.680720+0000","last_undegraded":"2024-08-31T06:06:23.680720+0000","last_fullsized":"2024-08-31T06:06:23.680720+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:28:53.414252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"80'11","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.688048+0000","last_change":"2024-08-31T06:04:25.879952+0000","last_active":"2024-08-31T06:06:23.688048+0000","last_peered":"2024-08-31T06:06:23.688048+0000","last_clean":"2024-08-31T06:06:23.688048+0000","last_became_active":"2024-08-31T06:04:25.879568+0000","last_became_peered":"2024-08-31T06:04:25.879568+0000","last_unstale":"2024-08-31T06:06:23.688048+0000","last_undegraded":"2024-08-31T06:06:23.688048+0000","last_fullsized":"2024-08-31T06:06:23.688048+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:28:04.558054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"80'16","reported_seq":44,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.676096+0000","last_change":"2024-08-31T06:05:25.051173+0000","last_active":"2024-08-31T06:06:23.676096+0000","last_peered":"2024-08-31T06:06:23.676096+0000","last_clean":"2024-08-31T06:06:23.676096+0000","last_became_active":"2024-08-31T06:05:25.050867+0000","last_became_peered":"2024-08-31T06:05:25.050867+0000","last_unstale":"2024-08-31T06:06:23.676096+0000","last_undegraded":"2024-08-31T06:06:23.676096+0000","last_fullsized":"2024-08-31T06:06:23.676096+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:27:09.971497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.686392+0000","last_change":"2024-08-31T06:04:25.883984+0000","last_active":"2024-08-31T06:06:23.686392+0000","last_peered":"2024-08-31T06:06:23.686392+0000","last_clean":"2024-08-31T06:06:23.686392+0000","last_became_active":"2024-08-31T06:04:25.883353+0000","last_became_peered":"2024-08-31T06:04:25.883353+0000","last_unstale":"2024-08-31T06:06:23.686392+0000","last_undegraded":"2024-08-31T06:06:23.686392+0000","last_fullsized":"2024-08-31T06:06:23.686392+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:16:25.178952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.672858+0000","last_change":"2024-08-31T06:04:25.883335+0000","last_active":"2024-08-31T06:06:23.672858+0000","last_peered":"2024-08-31T06:06:23.672858+0000","last_clean":"2024-08-31T06:06:23.672858+0000","last_became_active":"2024-08-31T06:04:25.881488+0000","last_became_peered":"2024-08-31T06:04:25.881488+0000","last_unstale":"2024-08-31T06:06:23.672858+0000","last_undegraded":"2024-08-31T06:06:23.672858+0000","last_fullsized":"2024-08-31T06:06:23.672858+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:57:24.472482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'10","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.678638+0000","last_change":"2024-08-31T06:04:25.891579+0000","last_active":"2024-08-31T06:06:23.678638+0000","last_peered":"2024-08-31T06:06:23.678638+0000","last_clean":"2024-08-31T06:06:23.678638+0000","last_became_active":"2024-08-31T06:04:25.887950+0000","last_became_peered":"2024-08-31T06:04:25.887950+0000","last_unstale":"2024-08-31T06:06:23.678638+0000","last_undegraded":"2024-08-31T06:06:23.678638+0000","last_fullsized":"2024-08-31T06:06:23.678638+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:38:35.284442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.682116+0000","last_change":"2024-08-31T06:04:25.882185+0000","last_active":"2024-08-31T06:06:23.682116+0000","last_peered":"2024-08-31T06:06:23.682116+0000","last_clean":"2024-08-31T06:06:23.682116+0000","last_became_active":"2024-08-31T06:04:25.876499+0000","last_became_peered":"2024-08-31T06:04:25.876499+0000","last_unstale":"2024-08-31T06:06:23.682116+0000","last_undegraded":"2024-08-31T06:06:23.682116+0000","last_fullsized":"2024-08-31T06:06:23.682116+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:40:01.132353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.683752+0000","last_change":"2024-08-31T06:04:25.883624+0000","last_active":"2024-08-31T06:06:23.683752+0000","last_peered":"2024-08-31T06:06:23.683752+0000","last_clean":"2024-08-31T06:06:23.683752+0000","last_became_active":"2024-08-31T06:04:25.878041+0000","last_became_peered":"2024-08-31T06:04:25.878041+0000","last_unstale":"2024-08-31T06:06:23.683752+0000","last_undegraded":"2024-08-31T06:06:23.683752+0000","last_fullsized":"2024-08-31T06:06:23.683752+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:05:21.278321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.687675+0000","last_change":"2024-08-31T06:04:25.882538+0000","last_active":"2024-08-31T06:06:23.687675+0000","last_peered":"2024-08-31T06:06:23.687675+0000","last_clean":"2024-08-31T06:06:23.687675+0000","last_became_active":"2024-08-31T06:04:25.882201+0000","last_became_peered":"2024-08-31T06:04:25.882201+0000","last_unstale":"2024-08-31T06:06:23.687675+0000","last_undegraded":"2024-08-31T06:06:23.687675+0000","last_fullsized":"2024-08-31T06:06:23.687675+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:47:16.880893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.686044+0000","last_change":"2024-08-31T06:04:25.880376+0000","last_active":"2024-08-31T06:06:23.686044+0000","last_peered":"2024-08-31T06:06:23.686044+0000","last_clean":"2024-08-31T06:06:23.686044+0000","last_became_active":"2024-08-31T06:04:25.877668+0000","last_became_peered":"2024-08-31T06:04:25.877668+0000","last_unstale":"2024-08-31T06:06:23.686044+0000","last_undegraded":"2024-08-31T06:06:23.686044+0000","last_fullsized":"2024-08-31T06:06:23.686044+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:55:32.737406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"80'10","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.673276+0000","last_change":"2024-08-31T06:04:25.875431+0000","last_active":"2024-08-31T06:06:23.673276+0000","last_peered":"2024-08-31T06:06:23.673276+0000","last_clean":"2024-08-31T06:06:23.673276+0000","last_became_active":"2024-08-31T06:04:25.874751+0000","last_became_peered":"2024-08-31T06:04:25.874751+0000","last_unstale":"2024-08-31T06:06:23.673276+0000","last_undegraded":"2024-08-31T06:06:23.673276+0000","last_fullsized":"2024-08-31T06:06:23.673276+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:16:57.695415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.675738+0000","last_change":"2024-08-31T06:04:25.895656+0000","last_active":"2024-08-31T06:06:23.675738+0000","last_peered":"2024-08-31T06:06:23.675738+0000","last_clean":"2024-08-31T06:06:23.675738+0000","last_became_active":"2024-08-31T06:04:25.895272+0000","last_became_peered":"2024-08-31T06:04:25.895272+0000","last_unstale":"2024-08-31T06:06:23.675738+0000","last_undegraded":"2024-08-31T06:06:23.675738+0000","last_fullsized":"2024-08-31T06:06:23.675738+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:18:47.276195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.679502+0000","last_change":"2024-08-31T06:04:25.877283+0000","last_active":"2024-08-31T06:06:23.679502+0000","last_peered":"2024-08-31T06:06:23.679502+0000","last_clean":"2024-08-31T06:06:23.679502+0000","last_became_active":"2024-08-31T06:04:25.876846+0000","last_became_peered":"2024-08-31T06:04:25.876846+0000","last_unstale":"2024-08-31T06:06:23.679502+0000","last_undegraded":"2024-08-31T06:06:23.679502+0000","last_fullsized":"2024-08-31T06:06:23.679502+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:54:05.086844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"80'11","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.685368+0000","last_change":"2024-08-31T06:04:25.884891+0000","last_active":"2024-08-31T06:06:23.685368+0000","last_peered":"2024-08-31T06:06:23.685368+0000","last_clean":"2024-08-31T06:06:23.685368+0000","last_became_active":"2024-08-31T06:04:25.884501+0000","last_became_peered":"2024-08-31T06:04:25.884501+0000","last_unstale":"2024-08-31T06:06:23.685368+0000","last_undegraded":"2024-08-31T06:06:23.685368+0000","last_fullsized":"2024-08-31T06:06:23.685368+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:22:54.846102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"80'10","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.691670+0000","last_change":"2024-08-31T06:04:25.877150+0000","last_active":"2024-08-31T06:06:23.691670+0000","last_peered":"2024-08-31T06:06:23.691670+0000","last_clean":"2024-08-31T06:06:23.691670+0000","last_became_active":"2024-08-31T06:04:25.876766+0000","last_became_peered":"2024-08-31T06:04:25.876766+0000","last_unstale":"2024-08-31T06:06:23.691670+0000","last_undegraded":"2024-08-31T06:06:23.691670+0000","last_fullsized":"2024-08-31T06:06:23.691670+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:46:26.724970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.676164+0000","last_change":"2024-08-31T06:04:25.881275+0000","last_active":"2024-08-31T06:06:23.676164+0000","last_peered":"2024-08-31T06:06:23.676164+0000","last_clean":"2024-08-31T06:06:23.676164+0000","last_became_active":"2024-08-31T06:04:25.879877+0000","last_became_peered":"2024-08-31T06:04:25.879877+0000","last_unstale":"2024-08-31T06:06:23.676164+0000","last_undegraded":"2024-08-31T06:06:23.676164+0000","last_fullsized":"2024-08-31T06:06:23.676164+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:07:07.237041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.676080+0000","last_change":"2024-08-31T06:04:25.895750+0000","last_active":"2024-08-31T06:06:23.676080+0000","last_peered":"2024-08-31T06:06:23.676080+0000","last_clean":"2024-08-31T06:06:23.676080+0000","last_became_active":"2024-08-31T06:04:25.895348+0000","last_became_peered":"2024-08-31T06:04:25.895348+0000","last_unstale":"2024-08-31T06:06:23.676080+0000","last_undegraded":"2024-08-31T06:06:23.676080+0000","last_fullsized":"2024-08-31T06:06:23.676080+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:53:10.014808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.670437+0000","last_change":"2024-08-31T06:04:25.895537+0000","last_active":"2024-08-31T06:06:23.670437+0000","last_peered":"2024-08-31T06:06:23.670437+0000","last_clean":"2024-08-31T06:06:23.670437+0000","last_became_active":"2024-08-31T06:04:25.895136+0000","last_became_peered":"2024-08-31T06:04:25.895136+0000","last_unstale":"2024-08-31T06:06:23.670437+0000","last_undegraded":"2024-08-31T06:06:23.670437+0000","last_fullsized":"2024-08-31T06:06:23.670437+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:41:57.554507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"80'15","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.676301+0000","last_change":"2024-08-31T06:04:25.880490+0000","last_active":"2024-08-31T06:06:23.676301+0000","last_peered":"2024-08-31T06:06:23.676301+0000","last_clean":"2024-08-31T06:06:23.676301+0000","last_became_active":"2024-08-31T06:04:25.879269+0000","last_became_peered":"2024-08-31T06:04:25.879269+0000","last_unstale":"2024-08-31T06:06:23.676301+0000","last_undegraded":"2024-08-31T06:06:23.676301+0000","last_fullsized":"2024-08-31T06:06:23.676301+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:05:49.575915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"80'6","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.676526+0000","last_change":"2024-08-31T06:05:25.053714+0000","last_active":"2024-08-31T06:06:23.676526+0000","last_peered":"2024-08-31T06:06:23.676526+0000","last_clean":"2024-08-31T06:06:23.676526+0000","last_became_active":"2024-08-31T06:05:25.052685+0000","last_became_peered":"2024-08-31T06:05:25.052685+0000","last_unstale":"2024-08-31T06:06:23.676526+0000","last_undegraded":"2024-08-31T06:06:23.676526+0000","last_fullsized":"2024-08-31T06:06:23.676526+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:17:31.474601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"80'4","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.684823+0000","last_change":"2024-08-31T06:05:25.057946+0000","last_active":"2024-08-31T06:06:23.684823+0000","last_peered":"2024-08-31T06:06:23.684823+0000","last_clean":"2024-08-31T06:06:23.684823+0000","last_became_active":"2024-08-31T06:05:25.057501+0000","last_became_peered":"2024-08-31T06:05:25.057501+0000","last_unstale":"2024-08-31T06:06:23.684823+0000","last_undegraded":"2024-08-31T06:06:23.684823+0000","last_fullsized":"2024-08-31T06:06:23.684823+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:16:05.721770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.685410+0000","last_change":"2024-08-31T06:04:25.897473+0000","last_active":"2024-08-31T06:06:23.685410+0000","last_peered":"2024-08-31T06:06:23.685410+0000","last_clean":"2024-08-31T06:06:23.685410+0000","last_became_active":"2024-08-31T06:04:25.895194+0000","last_became_peered":"2024-08-31T06:04:25.895194+0000","last_unstale":"2024-08-31T06:06:23.685410+0000","last_undegraded":"2024-08-31T06:06:23.685410+0000","last_fullsized":"2024-08-31T06:06:23.685410+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:43:04.539970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.684923+0000","last_change":"2024-08-31T06:04:25.894402+0000","last_active":"2024-08-31T06:06:23.684923+0000","last_peered":"2024-08-31T06:06:23.684923+0000","last_clean":"2024-08-31T06:06:23.684923+0000","last_became_active":"2024-08-31T06:04:25.893883+0000","last_became_peered":"2024-08-31T06:04:25.893883+0000","last_unstale":"2024-08-31T06:06:23.684923+0000","last_undegraded":"2024-08-31T06:06:23.684923+0000","last_fullsized":"2024-08-31T06:06:23.684923+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:45:53.778473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"80'12","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.681979+0000","last_change":"2024-08-31T06:04:25.881543+0000","last_active":"2024-08-31T06:06:23.681979+0000","last_peered":"2024-08-31T06:06:23.681979+0000","last_clean":"2024-08-31T06:06:23.681979+0000","last_became_active":"2024-08-31T06:04:25.881144+0000","last_became_peered":"2024-08-31T06:04:25.881144+0000","last_unstale":"2024-08-31T06:06:23.681979+0000","last_undegraded":"2024-08-31T06:06:23.681979+0000","last_fullsized":"2024-08-31T06:06:23.681979+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:13:02.282879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.669781+0000","last_change":"2024-08-31T06:04:25.881143+0000","last_active":"2024-08-31T06:06:23.669781+0000","last_peered":"2024-08-31T06:06:23.669781+0000","last_clean":"2024-08-31T06:06:23.669781+0000","last_became_active":"2024-08-31T06:04:25.879718+0000","last_became_peered":"2024-08-31T06:04:25.879718+0000","last_unstale":"2024-08-31T06:06:23.669781+0000","last_undegraded":"2024-08-31T06:06:23.669781+0000","last_fullsized":"2024-08-31T06:06:23.669781+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:32:07.771488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.680189+0000","last_change":"2024-08-31T06:04:25.879493+0000","last_active":"2024-08-31T06:06:23.680189+0000","last_peered":"2024-08-31T06:06:23.680189+0000","last_clean":"2024-08-31T06:06:23.680189+0000","last_became_active":"2024-08-31T06:04:25.879065+0000","last_became_peered":"2024-08-31T06:04:25.879065+0000","last_unstale":"2024-08-31T06:06:23.680189+0000","last_undegraded":"2024-08-31T06:06:23.680189+0000","last_fullsized":"2024-08-31T06:06:23.680189+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:53:43.848269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"80'7","reported_seq":168,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:37.163835+0000","last_change":"2024-08-31T06:05:25.058954+0000","last_active":"2024-08-31T06:06:37.163835+0000","last_peered":"2024-08-31T06:06:37.163835+0000","last_clean":"2024-08-31T06:06:37.163835+0000","last_became_active":"2024-08-31T06:05:25.051143+0000","last_became_peered":"2024-08-31T06:05:25.051143+0000","last_unstale":"2024-08-31T06:06:37.163835+0000","last_undegraded":"2024-08-31T06:06:37.163835+0000","last_fullsized":"2024-08-31T06:06:37.163835+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:38:23.533663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.682132+0000","last_change":"2024-08-31T06:04:25.896354+0000","last_active":"2024-08-31T06:06:23.682132+0000","last_peered":"2024-08-31T06:06:23.682132+0000","last_clean":"2024-08-31T06:06:23.682132+0000","last_became_active":"2024-08-31T06:04:25.896021+0000","last_became_peered":"2024-08-31T06:04:25.896021+0000","last_unstale":"2024-08-31T06:06:23.682132+0000","last_undegraded":"2024-08-31T06:06:23.682132+0000","last_fullsized":"2024-08-31T06:06:23.682132+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:08:40.213238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.047153+0000","last_change":"2024-08-31T06:04:22.844248+0000","last_active":"2024-08-31T06:05:25.047153+0000","last_peered":"2024-08-31T06:05:25.047153+0000","last_clean":"2024-08-31T06:05:25.047153+0000","last_became_active":"2024-08-31T06:04:22.843784+0000","last_became_peered":"2024-08-31T06:04:22.843784+0000","last_unstale":"2024-08-31T06:05:25.047153+0000","last_undegraded":"2024-08-31T06:05:25.047153+0000","last_fullsized":"2024-08-31T06:05:25.047153+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:56:41.422965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.046218+0000","last_change":"2024-08-31T06:04:22.847049+0000","last_active":"2024-08-31T06:05:25.046218+0000","last_peered":"2024-08-31T06:05:25.046218+0000","last_clean":"2024-08-31T06:05:25.046218+0000","last_became_active":"2024-08-31T06:04:22.846282+0000","last_became_peered":"2024-08-31T06:04:22.846282+0000","last_unstale":"2024-08-31T06:05:25.046218+0000","last_undegraded":"2024-08-31T06:05:25.046218+0000","last_fullsized":"2024-08-31T06:05:25.046218+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:35:21.030425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.679457+0000","last_change":"2024-08-31T06:04:25.886698+0000","last_active":"2024-08-31T06:06:23.679457+0000","last_peered":"2024-08-31T06:06:23.679457+0000","last_clean":"2024-08-31T06:06:23.679457+0000","last_became_active":"2024-08-31T06:04:25.886334+0000","last_became_peered":"2024-08-31T06:04:25.886334+0000","last_unstale":"2024-08-31T06:06:23.679457+0000","last_undegraded":"2024-08-31T06:06:23.679457+0000","last_fullsized":"2024-08-31T06:06:23.679457+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:24:43.821446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.044798+0000","last_change":"2024-08-31T06:04:22.845263+0000","last_active":"2024-08-31T06:05:25.044798+0000","last_peered":"2024-08-31T06:05:25.044798+0000","last_clean":"2024-08-31T06:05:25.044798+0000","last_became_active":"2024-08-31T06:04:22.844853+0000","last_became_peered":"2024-08-31T06:04:22.844853+0000","last_unstale":"2024-08-31T06:05:25.044798+0000","last_undegraded":"2024-08-31T06:05:25.044798+0000","last_fullsized":"2024-08-31T06:05:25.044798+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:55:12.561165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"80'11","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.673293+0000","last_change":"2024-08-31T06:04:25.894475+0000","last_active":"2024-08-31T06:06:23.673293+0000","last_peered":"2024-08-31T06:06:23.673293+0000","last_clean":"2024-08-31T06:06:23.673293+0000","last_became_active":"2024-08-31T06:04:25.894121+0000","last_became_peered":"2024-08-31T06:04:25.894121+0000","last_unstale":"2024-08-31T06:06:23.673293+0000","last_undegraded":"2024-08-31T06:06:23.673293+0000","last_fullsized":"2024-08-31T06:06:23.673293+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:40:06.570616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":149,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:38.164972+0000","last_change":"2024-08-31T06:04:22.834430+0000","last_active":"2024-08-31T06:06:38.164972+0000","last_peered":"2024-08-31T06:06:38.164972+0000","last_clean":"2024-08-31T06:06:38.164972+0000","last_became_active":"2024-08-31T06:04:22.834011+0000","last_became_peered":"2024-08-31T06:04:22.834011+0000","last_unstale":"2024-08-31T06:06:38.164972+0000","last_undegraded":"2024-08-31T06:06:38.164972+0000","last_fullsized":"2024-08-31T06:06:38.164972+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:51:16.734234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"80'10","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.689334+0000","last_change":"2024-08-31T06:04:25.877727+0000","last_active":"2024-08-31T06:06:23.689334+0000","last_peered":"2024-08-31T06:06:23.689334+0000","last_clean":"2024-08-31T06:06:23.689334+0000","last_became_active":"2024-08-31T06:04:25.877121+0000","last_became_peered":"2024-08-31T06:04:25.877121+0000","last_unstale":"2024-08-31T06:06:23.689334+0000","last_undegraded":"2024-08-31T06:06:23.689334+0000","last_fullsized":"2024-08-31T06:06:23.689334+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:41:12.247216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048823+0000","last_change":"2024-08-31T06:04:22.846482+0000","last_active":"2024-08-31T06:05:25.048823+0000","last_peered":"2024-08-31T06:05:25.048823+0000","last_clean":"2024-08-31T06:05:25.048823+0000","last_became_active":"2024-08-31T06:04:22.846057+0000","last_became_peered":"2024-08-31T06:04:22.846057+0000","last_unstale":"2024-08-31T06:05:25.048823+0000","last_undegraded":"2024-08-31T06:05:25.048823+0000","last_fullsized":"2024-08-31T06:05:25.048823+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:29:11.861665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.684452+0000","last_change":"2024-08-31T06:04:25.893777+0000","last_active":"2024-08-31T06:06:23.684452+0000","last_peered":"2024-08-31T06:06:23.684452+0000","last_clean":"2024-08-31T06:06:23.684452+0000","last_became_active":"2024-08-31T06:04:25.893014+0000","last_became_peered":"2024-08-31T06:04:25.893014+0000","last_unstale":"2024-08-31T06:06:23.684452+0000","last_undegraded":"2024-08-31T06:06:23.684452+0000","last_fullsized":"2024-08-31T06:06:23.684452+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:04:48.185842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.045472+0000","last_change":"2024-08-31T06:04:22.965938+0000","last_active":"2024-08-31T06:05:25.045472+0000","last_peered":"2024-08-31T06:05:25.045472+0000","last_clean":"2024-08-31T06:05:25.045472+0000","last_became_active":"2024-08-31T06:04:22.965356+0000","last_became_peered":"2024-08-31T06:04:22.965356+0000","last_unstale":"2024-08-31T06:05:25.045472+0000","last_undegraded":"2024-08-31T06:05:25.045472+0000","last_fullsized":"2024-08-31T06:05:25.045472+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:36:58.938462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"80'11","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.691767+0000","last_change":"2024-08-31T06:04:25.881307+0000","last_active":"2024-08-31T06:06:23.691767+0000","last_peered":"2024-08-31T06:06:23.691767+0000","last_clean":"2024-08-31T06:06:23.691767+0000","last_became_active":"2024-08-31T06:04:25.880418+0000","last_became_peered":"2024-08-31T06:04:25.880418+0000","last_unstale":"2024-08-31T06:06:23.691767+0000","last_undegraded":"2024-08-31T06:06:23.691767+0000","last_fullsized":"2024-08-31T06:06:23.691767+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:21:46.904371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'34","reported_seq":310,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:39.167165+0000","last_change":"2024-08-31T06:04:22.843073+0000","last_active":"2024-08-31T06:06:39.167165+0000","last_peered":"2024-08-31T06:06:39.167165+0000","last_clean":"2024-08-31T06:06:39.167165+0000","last_became_active":"2024-08-31T06:04:22.842645+0000","last_became_peered":"2024-08-31T06:04:22.842645+0000","last_unstale":"2024-08-31T06:06:39.167165+0000","last_undegraded":"2024-08-31T06:06:39.167165+0000","last_fullsized":"2024-08-31T06:06:39.167165+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:41:37.651562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65695,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":250,"num_read_kb":189,"num_write":35,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.680499+0000","last_change":"2024-08-31T06:04:25.879929+0000","last_active":"2024-08-31T06:06:23.680499+0000","last_peered":"2024-08-31T06:06:23.680499+0000","last_clean":"2024-08-31T06:06:23.680499+0000","last_became_active":"2024-08-31T06:04:25.878475+0000","last_became_peered":"2024-08-31T06:04:25.878475+0000","last_unstale":"2024-08-31T06:06:23.680499+0000","last_undegraded":"2024-08-31T06:06:23.680499+0000","last_fullsized":"2024-08-31T06:06:23.680499+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:10:32.801558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048264+0000","last_change":"2024-08-31T06:04:22.842878+0000","last_active":"2024-08-31T06:05:25.048264+0000","last_peered":"2024-08-31T06:05:25.048264+0000","last_clean":"2024-08-31T06:05:25.048264+0000","last_became_active":"2024-08-31T06:04:22.842306+0000","last_became_peered":"2024-08-31T06:04:22.842306+0000","last_unstale":"2024-08-31T06:05:25.048264+0000","last_undegraded":"2024-08-31T06:05:25.048264+0000","last_fullsized":"2024-08-31T06:05:25.048264+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:53:55.259238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.669813+0000","last_change":"2024-08-31T06:04:25.899357+0000","last_active":"2024-08-31T06:06:23.669813+0000","last_peered":"2024-08-31T06:06:23.669813+0000","last_clean":"2024-08-31T06:06:23.669813+0000","last_became_active":"2024-08-31T06:04:25.899000+0000","last_became_peered":"2024-08-31T06:04:25.899000+0000","last_unstale":"2024-08-31T06:06:23.669813+0000","last_undegraded":"2024-08-31T06:06:23.669813+0000","last_fullsized":"2024-08-31T06:06:23.669813+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:20:39.991071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.044693+0000","last_change":"2024-08-31T06:04:22.843063+0000","last_active":"2024-08-31T06:05:25.044693+0000","last_peered":"2024-08-31T06:05:25.044693+0000","last_clean":"2024-08-31T06:05:25.044693+0000","last_became_active":"2024-08-31T06:04:22.842654+0000","last_became_peered":"2024-08-31T06:04:22.842654+0000","last_unstale":"2024-08-31T06:05:25.044693+0000","last_undegraded":"2024-08-31T06:05:25.044693+0000","last_fullsized":"2024-08-31T06:05:25.044693+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:09:54.563116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"80'15","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.680868+0000","last_change":"2024-08-31T06:04:25.885556+0000","last_active":"2024-08-31T06:06:23.680868+0000","last_peered":"2024-08-31T06:06:23.680868+0000","last_clean":"2024-08-31T06:06:23.680868+0000","last_became_active":"2024-08-31T06:04:25.885216+0000","last_became_peered":"2024-08-31T06:04:25.885216+0000","last_unstale":"2024-08-31T06:06:23.680868+0000","last_undegraded":"2024-08-31T06:06:23.680868+0000","last_fullsized":"2024-08-31T06:06:23.680868+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:48:49.816261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'5","reported_seq":154,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:37.720103+0000","last_change":"2024-08-31T06:04:22.844839+0000","last_active":"2024-08-31T06:06:37.720103+0000","last_peered":"2024-08-31T06:06:37.720103+0000","last_clean":"2024-08-31T06:06:37.720103+0000","last_became_active":"2024-08-31T06:04:22.844425+0000","last_became_peered":"2024-08-31T06:04:22.844425+0000","last_unstale":"2024-08-31T06:06:37.720103+0000","last_undegraded":"2024-08-31T06:06:37.720103+0000","last_fullsized":"2024-08-31T06:06:37.720103+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:10:57.292173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.682264+0000","last_change":"2024-08-31T06:04:25.878195+0000","last_active":"2024-08-31T06:06:23.682264+0000","last_peered":"2024-08-31T06:06:23.682264+0000","last_clean":"2024-08-31T06:06:23.682264+0000","last_became_active":"2024-08-31T06:04:25.877823+0000","last_became_peered":"2024-08-31T06:04:25.877823+0000","last_unstale":"2024-08-31T06:06:23.682264+0000","last_undegraded":"2024-08-31T06:06:23.682264+0000","last_fullsized":"2024-08-31T06:06:23.682264+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:54:34.989583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":68,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.071826+0000","last_change":"2024-08-31T06:05:25.071391+0000","last_active":"2024-08-31T06:05:25.071826+0000","last_peered":"2024-08-31T06:05:25.071826+0000","last_clean":"2024-08-31T06:05:25.071826+0000","last_became_active":"2024-08-31T06:05:25.057546+0000","last_became_peered":"2024-08-31T06:05:25.057546+0000","last_unstale":"2024-08-31T06:05:25.071826+0000","last_undegraded":"2024-08-31T06:05:25.071826+0000","last_fullsized":"2024-08-31T06:05:25.071826+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:42:35.527395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.681869+0000","last_change":"2024-08-31T06:04:25.887580+0000","last_active":"2024-08-31T06:06:23.681869+0000","last_peered":"2024-08-31T06:06:23.681869+0000","last_clean":"2024-08-31T06:06:23.681869+0000","last_became_active":"2024-08-31T06:04:25.887158+0000","last_became_peered":"2024-08-31T06:04:25.887158+0000","last_unstale":"2024-08-31T06:06:23.681869+0000","last_undegraded":"2024-08-31T06:06:23.681869+0000","last_fullsized":"2024-08-31T06:06:23.681869+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:27:38.115742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.064443+0000","last_change":"2024-08-31T06:05:25.063859+0000","last_active":"2024-08-31T06:05:25.064443+0000","last_peered":"2024-08-31T06:05:25.064443+0000","last_clean":"2024-08-31T06:05:25.064443+0000","last_became_active":"2024-08-31T06:05:25.053596+0000","last_became_peered":"2024-08-31T06:05:25.053596+0000","last_unstale":"2024-08-31T06:05:25.064443+0000","last_undegraded":"2024-08-31T06:05:25.064443+0000","last_fullsized":"2024-08-31T06:05:25.064443+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:19:18.461630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.678741+0000","last_change":"2024-08-31T06:04:25.880845+0000","last_active":"2024-08-31T06:06:23.678741+0000","last_peered":"2024-08-31T06:06:23.678741+0000","last_clean":"2024-08-31T06:06:23.678741+0000","last_became_active":"2024-08-31T06:04:25.880423+0000","last_became_peered":"2024-08-31T06:04:25.880423+0000","last_unstale":"2024-08-31T06:06:23.678741+0000","last_undegraded":"2024-08-31T06:06:23.678741+0000","last_fullsized":"2024-08-31T06:06:23.678741+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:16:56.725116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.046584+0000","last_change":"2024-08-31T06:04:22.843393+0000","last_active":"2024-08-31T06:05:25.046584+0000","last_peered":"2024-08-31T06:05:25.046584+0000","last_clean":"2024-08-31T06:05:25.046584+0000","last_became_active":"2024-08-31T06:04:22.842770+0000","last_became_peered":"2024-08-31T06:04:22.842770+0000","last_unstale":"2024-08-31T06:05:25.046584+0000","last_undegraded":"2024-08-31T06:05:25.046584+0000","last_fullsized":"2024-08-31T06:05:25.046584+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:49:16.159167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.686211+0000","last_change":"2024-08-31T06:04:25.887829+0000","last_active":"2024-08-31T06:06:23.686211+0000","last_peered":"2024-08-31T06:06:23.686211+0000","last_clean":"2024-08-31T06:06:23.686211+0000","last_became_active":"2024-08-31T06:04:25.886567+0000","last_became_peered":"2024-08-31T06:04:25.886567+0000","last_unstale":"2024-08-31T06:06:23.686211+0000","last_undegraded":"2024-08-31T06:06:23.686211+0000","last_fullsized":"2024-08-31T06:06:23.686211+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:11:39.054966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.045627+0000","last_change":"2024-08-31T06:04:22.844218+0000","last_active":"2024-08-31T06:05:25.045627+0000","last_peered":"2024-08-31T06:05:25.045627+0000","last_clean":"2024-08-31T06:05:25.045627+0000","last_became_active":"2024-08-31T06:04:22.843786+0000","last_became_peered":"2024-08-31T06:04:22.843786+0000","last_unstale":"2024-08-31T06:05:25.045627+0000","last_undegraded":"2024-08-31T06:05:25.045627+0000","last_fullsized":"2024-08-31T06:05:25.045627+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:46:02.534190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.682380+0000","last_change":"2024-08-31T06:04:25.888946+0000","last_active":"2024-08-31T06:06:23.682380+0000","last_peered":"2024-08-31T06:06:23.682380+0000","last_clean":"2024-08-31T06:06:23.682380+0000","last_became_active":"2024-08-31T06:04:25.888564+0000","last_became_peered":"2024-08-31T06:04:25.888564+0000","last_unstale":"2024-08-31T06:06:23.682380+0000","last_undegraded":"2024-08-31T06:06:23.682380+0000","last_fullsized":"2024-08-31T06:06:23.682380+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:26:19.051971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.049211+0000","last_change":"2024-08-31T06:04:22.964483+0000","last_active":"2024-08-31T06:05:25.049211+0000","last_peered":"2024-08-31T06:05:25.049211+0000","last_clean":"2024-08-31T06:05:25.049211+0000","last_became_active":"2024-08-31T06:04:22.963874+0000","last_became_peered":"2024-08-31T06:04:22.963874+0000","last_unstale":"2024-08-31T06:05:25.049211+0000","last_undegraded":"2024-08-31T06:05:25.049211+0000","last_fullsized":"2024-08-31T06:05:25.049211+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:30:00.159040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.688990+0000","last_change":"2024-08-31T06:04:25.880941+0000","last_active":"2024-08-31T06:06:23.688990+0000","last_peered":"2024-08-31T06:06:23.688990+0000","last_clean":"2024-08-31T06:06:23.688990+0000","last_became_active":"2024-08-31T06:04:25.880454+0000","last_became_peered":"2024-08-31T06:04:25.880454+0000","last_unstale":"2024-08-31T06:06:23.688990+0000","last_undegraded":"2024-08-31T06:06:23.688990+0000","last_fullsized":"2024-08-31T06:06:23.688990+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:49:57.023988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":98,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048527+0000","last_change":"2024-08-31T06:03:05.921629+0000","last_active":"2024-08-31T06:05:25.048527+0000","last_peered":"2024-08-31T06:05:25.048527+0000","last_clean":"2024-08-31T06:05:25.048527+0000","last_became_active":"2024-08-31T06:03:05.915844+0000","last_became_peered":"2024-08-31T06:03:05.915844+0000","last_unstale":"2024-08-31T06:05:25.048527+0000","last_undegraded":"2024-08-31T06:05:25.048527+0000","last_fullsized":"2024-08-31T06:05:25.048527+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T05:59:24.472429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T05:59:24.472429+0000","last_clean_scrub_stamp":"2024-08-31T05:59:24.472429+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:56:41.073422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.044718+0000","last_change":"2024-08-31T06:04:22.846325+0000","last_active":"2024-08-31T06:05:25.044718+0000","last_peered":"2024-08-31T06:05:25.044718+0000","last_clean":"2024-08-31T06:05:25.044718+0000","last_became_active":"2024-08-31T06:04:22.845772+0000","last_became_peered":"2024-08-31T06:04:22.845772+0000","last_unstale":"2024-08-31T06:05:25.044718+0000","last_undegraded":"2024-08-31T06:05:25.044718+0000","last_fullsized":"2024-08-31T06:05:25.044718+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:54:10.336786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.691294+0000","last_change":"2024-08-31T06:04:25.891437+0000","last_active":"2024-08-31T06:06:23.691294+0000","last_peered":"2024-08-31T06:06:23.691294+0000","last_clean":"2024-08-31T06:06:23.691294+0000","last_became_active":"2024-08-31T06:04:25.891053+0000","last_became_peered":"2024-08-31T06:04:25.891053+0000","last_unstale":"2024-08-31T06:06:23.691294+0000","last_undegraded":"2024-08-31T06:06:23.691294+0000","last_fullsized":"2024-08-31T06:06:23.691294+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:22:37.119483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048411+0000","last_change":"2024-08-31T06:04:22.842885+0000","last_active":"2024-08-31T06:05:25.048411+0000","last_peered":"2024-08-31T06:05:25.048411+0000","last_clean":"2024-08-31T06:05:25.048411+0000","last_became_active":"2024-08-31T06:04:22.842473+0000","last_became_peered":"2024-08-31T06:04:22.842473+0000","last_unstale":"2024-08-31T06:05:25.048411+0000","last_undegraded":"2024-08-31T06:05:25.048411+0000","last_fullsized":"2024-08-31T06:05:25.048411+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:33:36.516659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.663699+0000","last_change":"2024-08-31T06:04:25.889213+0000","last_active":"2024-08-31T06:06:23.663699+0000","last_peered":"2024-08-31T06:06:23.663699+0000","last_clean":"2024-08-31T06:06:23.663699+0000","last_became_active":"2024-08-31T06:04:25.888860+0000","last_became_peered":"2024-08-31T06:04:25.888860+0000","last_unstale":"2024-08-31T06:06:23.663699+0000","last_undegraded":"2024-08-31T06:06:23.663699+0000","last_fullsized":"2024-08-31T06:06:23.663699+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:44:18.831229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":152,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:38.720896+0000","last_change":"2024-08-31T06:04:22.966091+0000","last_active":"2024-08-31T06:06:38.720896+0000","last_peered":"2024-08-31T06:06:38.720896+0000","last_clean":"2024-08-31T06:06:38.720896+0000","last_became_active":"2024-08-31T06:04:22.965727+0000","last_became_peered":"2024-08-31T06:04:22.965727+0000","last_unstale":"2024-08-31T06:06:38.720896+0000","last_undegraded":"2024-08-31T06:06:38.720896+0000","last_fullsized":"2024-08-31T06:06:38.720896+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:42:43.643623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.682138+0000","last_change":"2024-08-31T06:04:25.880039+0000","last_active":"2024-08-31T06:06:23.682138+0000","last_peered":"2024-08-31T06:06:23.682138+0000","last_clean":"2024-08-31T06:06:23.682138+0000","last_became_active":"2024-08-31T06:04:25.879669+0000","last_became_peered":"2024-08-31T06:04:25.879669+0000","last_unstale":"2024-08-31T06:06:23.682138+0000","last_undegraded":"2024-08-31T06:06:23.682138+0000","last_fullsized":"2024-08-31T06:06:23.682138+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:02:09.244281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048687+0000","last_change":"2024-08-31T06:04:22.844572+0000","last_active":"2024-08-31T06:05:25.048687+0000","last_peered":"2024-08-31T06:05:25.048687+0000","last_clean":"2024-08-31T06:05:25.048687+0000","last_became_active":"2024-08-31T06:04:22.844227+0000","last_became_peered":"2024-08-31T06:04:22.844227+0000","last_unstale":"2024-08-31T06:05:25.048687+0000","last_undegraded":"2024-08-31T06:05:25.048687+0000","last_fullsized":"2024-08-31T06:05:25.048687+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:04:51.966901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.677043+0000","last_change":"2024-08-31T06:04:25.884442+0000","last_active":"2024-08-31T06:06:23.677043+0000","last_peered":"2024-08-31T06:06:23.677043+0000","last_clean":"2024-08-31T06:06:23.677043+0000","last_became_active":"2024-08-31T06:04:25.884016+0000","last_became_peered":"2024-08-31T06:04:25.884016+0000","last_unstale":"2024-08-31T06:06:23.677043+0000","last_undegraded":"2024-08-31T06:06:23.677043+0000","last_fullsized":"2024-08-31T06:06:23.677043+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:36:10.640262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.043831+0000","last_change":"2024-08-31T06:04:22.967845+0000","last_active":"2024-08-31T06:05:25.043831+0000","last_peered":"2024-08-31T06:05:25.043831+0000","last_clean":"2024-08-31T06:05:25.043831+0000","last_became_active":"2024-08-31T06:04:22.967421+0000","last_became_peered":"2024-08-31T06:04:22.967421+0000","last_unstale":"2024-08-31T06:05:25.043831+0000","last_undegraded":"2024-08-31T06:05:25.043831+0000","last_fullsized":"2024-08-31T06:05:25.043831+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:17:46.942551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.673410+0000","last_change":"2024-08-31T06:04:25.897087+0000","last_active":"2024-08-31T06:06:23.673410+0000","last_peered":"2024-08-31T06:06:23.673410+0000","last_clean":"2024-08-31T06:06:23.673410+0000","last_became_active":"2024-08-31T06:04:25.896725+0000","last_became_peered":"2024-08-31T06:04:25.896725+0000","last_unstale":"2024-08-31T06:06:23.673410+0000","last_undegraded":"2024-08-31T06:06:23.673410+0000","last_fullsized":"2024-08-31T06:06:23.673410+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:31:28.279489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.053673+0000","last_change":"2024-08-31T06:04:22.965299+0000","last_active":"2024-08-31T06:05:25.053673+0000","last_peered":"2024-08-31T06:05:25.053673+0000","last_clean":"2024-08-31T06:05:25.053673+0000","last_became_active":"2024-08-31T06:04:22.964849+0000","last_became_peered":"2024-08-31T06:04:22.964849+0000","last_unstale":"2024-08-31T06:05:25.053673+0000","last_undegraded":"2024-08-31T06:05:25.053673+0000","last_fullsized":"2024-08-31T06:05:25.053673+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:36:56.431678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.679635+0000","last_change":"2024-08-31T06:04:25.885115+0000","last_active":"2024-08-31T06:06:23.679635+0000","last_peered":"2024-08-31T06:06:23.679635+0000","last_clean":"2024-08-31T06:06:23.679635+0000","last_became_active":"2024-08-31T06:04:25.884722+0000","last_became_peered":"2024-08-31T06:04:25.884722+0000","last_unstale":"2024-08-31T06:06:23.679635+0000","last_undegraded":"2024-08-31T06:06:23.679635+0000","last_fullsized":"2024-08-31T06:06:23.679635+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:20:43.913785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'13","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:32.962890+0000","last_change":"2024-08-31T06:04:22.845119+0000","last_active":"2024-08-31T06:06:32.962890+0000","last_peered":"2024-08-31T06:06:32.962890+0000","last_clean":"2024-08-31T06:06:32.962890+0000","last_became_active":"2024-08-31T06:04:22.843814+0000","last_became_peered":"2024-08-31T06:04:22.843814+0000","last_unstale":"2024-08-31T06:06:32.962890+0000","last_undegraded":"2024-08-31T06:06:32.962890+0000","last_fullsized":"2024-08-31T06:06:32.962890+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:06:40.684887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.680844+0000","last_change":"2024-08-31T06:04:25.892838+0000","last_active":"2024-08-31T06:06:23.680844+0000","last_peered":"2024-08-31T06:06:23.680844+0000","last_clean":"2024-08-31T06:06:23.680844+0000","last_became_active":"2024-08-31T06:04:25.891955+0000","last_became_peered":"2024-08-31T06:04:25.891955+0000","last_unstale":"2024-08-31T06:06:23.680844+0000","last_undegraded":"2024-08-31T06:06:23.680844+0000","last_fullsized":"2024-08-31T06:06:23.680844+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:41:35.085724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.044957+0000","last_change":"2024-08-31T06:04:22.964183+0000","last_active":"2024-08-31T06:05:25.044957+0000","last_peered":"2024-08-31T06:05:25.044957+0000","last_clean":"2024-08-31T06:05:25.044957+0000","last_became_active":"2024-08-31T06:04:22.963688+0000","last_became_peered":"2024-08-31T06:04:22.963688+0000","last_unstale":"2024-08-31T06:05:25.044957+0000","last_undegraded":"2024-08-31T06:05:25.044957+0000","last_fullsized":"2024-08-31T06:05:25.044957+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:04:47.833811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.683320+0000","last_change":"2024-08-31T06:04:25.892266+0000","last_active":"2024-08-31T06:06:23.683320+0000","last_peered":"2024-08-31T06:06:23.683320+0000","last_clean":"2024-08-31T06:06:23.683320+0000","last_became_active":"2024-08-31T06:04:25.889014+0000","last_became_peered":"2024-08-31T06:04:25.889014+0000","last_unstale":"2024-08-31T06:06:23.683320+0000","last_undegraded":"2024-08-31T06:06:23.683320+0000","last_fullsized":"2024-08-31T06:06:23.683320+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:27:47.608470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.049408+0000","last_change":"2024-08-31T06:04:22.964677+0000","last_active":"2024-08-31T06:05:25.049408+0000","last_peered":"2024-08-31T06:05:25.049408+0000","last_clean":"2024-08-31T06:05:25.049408+0000","last_became_active":"2024-08-31T06:04:22.964237+0000","last_became_peered":"2024-08-31T06:04:22.964237+0000","last_unstale":"2024-08-31T06:05:25.049408+0000","last_undegraded":"2024-08-31T06:05:25.049408+0000","last_fullsized":"2024-08-31T06:05:25.049408+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:00:54.099592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.648239+0000","last_change":"2024-08-31T06:04:25.881699+0000","last_active":"2024-08-31T06:06:23.648239+0000","last_peered":"2024-08-31T06:06:23.648239+0000","last_clean":"2024-08-31T06:06:23.648239+0000","last_became_active":"2024-08-31T06:04:25.879304+0000","last_became_peered":"2024-08-31T06:04:25.879304+0000","last_unstale":"2024-08-31T06:06:23.648239+0000","last_undegraded":"2024-08-31T06:06:23.648239+0000","last_fullsized":"2024-08-31T06:06:23.648239+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:56:42.992034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.108301+0000","last_change":"2024-08-31T06:04:22.847934+0000","last_active":"2024-08-31T06:05:25.108301+0000","last_peered":"2024-08-31T06:05:25.108301+0000","last_clean":"2024-08-31T06:05:25.108301+0000","last_became_active":"2024-08-31T06:04:22.847518+0000","last_became_peered":"2024-08-31T06:04:22.847518+0000","last_unstale":"2024-08-31T06:05:25.108301+0000","last_undegraded":"2024-08-31T06:05:25.108301+0000","last_fullsized":"2024-08-31T06:05:25.108301+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:26:21.476014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.685828+0000","last_change":"2024-08-31T06:04:25.884653+0000","last_active":"2024-08-31T06:06:23.685828+0000","last_peered":"2024-08-31T06:06:23.685828+0000","last_clean":"2024-08-31T06:06:23.685828+0000","last_became_active":"2024-08-31T06:04:25.884244+0000","last_became_peered":"2024-08-31T06:04:25.884244+0000","last_unstale":"2024-08-31T06:06:23.685828+0000","last_undegraded":"2024-08-31T06:06:23.685828+0000","last_fullsized":"2024-08-31T06:06:23.685828+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:13:00.814071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048648+0000","last_change":"2024-08-31T06:04:22.845924+0000","last_active":"2024-08-31T06:05:25.048648+0000","last_peered":"2024-08-31T06:05:25.048648+0000","last_clean":"2024-08-31T06:05:25.048648+0000","last_became_active":"2024-08-31T06:04:22.845445+0000","last_became_peered":"2024-08-31T06:04:22.845445+0000","last_unstale":"2024-08-31T06:05:25.048648+0000","last_undegraded":"2024-08-31T06:05:25.048648+0000","last_fullsized":"2024-08-31T06:05:25.048648+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:25:21.052136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.679162+0000","last_change":"2024-08-31T06:04:25.890069+0000","last_active":"2024-08-31T06:06:23.679162+0000","last_peered":"2024-08-31T06:06:23.679162+0000","last_clean":"2024-08-31T06:06:23.679162+0000","last_became_active":"2024-08-31T06:04:25.889709+0000","last_became_peered":"2024-08-31T06:04:25.889709+0000","last_unstale":"2024-08-31T06:06:23.679162+0000","last_undegraded":"2024-08-31T06:06:23.679162+0000","last_fullsized":"2024-08-31T06:06:23.679162+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:13:51.360023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.051611+0000","last_change":"2024-08-31T06:04:22.847845+0000","last_active":"2024-08-31T06:05:25.051611+0000","last_peered":"2024-08-31T06:05:25.051611+0000","last_clean":"2024-08-31T06:05:25.051611+0000","last_became_active":"2024-08-31T06:04:22.847001+0000","last_became_peered":"2024-08-31T06:04:22.847001+0000","last_unstale":"2024-08-31T06:05:25.051611+0000","last_undegraded":"2024-08-31T06:05:25.051611+0000","last_fullsized":"2024-08-31T06:05:25.051611+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:24:34.209076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.690813+0000","last_change":"2024-08-31T06:04:25.886759+0000","last_active":"2024-08-31T06:06:23.690813+0000","last_peered":"2024-08-31T06:06:23.690813+0000","last_clean":"2024-08-31T06:06:23.690813+0000","last_became_active":"2024-08-31T06:04:25.886176+0000","last_became_peered":"2024-08-31T06:04:25.886176+0000","last_unstale":"2024-08-31T06:06:23.690813+0000","last_undegraded":"2024-08-31T06:06:23.690813+0000","last_fullsized":"2024-08-31T06:06:23.690813+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:11:12.637451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"80'15","reported_seq":43,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.677867+0000","last_change":"2024-08-31T06:05:25.054468+0000","last_active":"2024-08-31T06:06:23.677867+0000","last_peered":"2024-08-31T06:06:23.677867+0000","last_clean":"2024-08-31T06:06:23.677867+0000","last_became_active":"2024-08-31T06:05:25.053657+0000","last_became_peered":"2024-08-31T06:05:25.053657+0000","last_unstale":"2024-08-31T06:06:23.677867+0000","last_undegraded":"2024-08-31T06:06:23.677867+0000","last_fullsized":"2024-08-31T06:06:23.677867+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:43:44.887637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.059757+0000","last_change":"2024-08-31T06:05:25.059219+0000","last_active":"2024-08-31T06:05:25.059757+0000","last_peered":"2024-08-31T06:05:25.059757+0000","last_clean":"2024-08-31T06:05:25.059757+0000","last_became_active":"2024-08-31T06:05:25.053136+0000","last_became_peered":"2024-08-31T06:05:25.053136+0000","last_unstale":"2024-08-31T06:05:25.059757+0000","last_undegraded":"2024-08-31T06:05:25.059757+0000","last_fullsized":"2024-08-31T06:05:25.059757+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:29:59.137465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.691662+0000","last_change":"2024-08-31T06:04:25.887524+0000","last_active":"2024-08-31T06:06:23.691662+0000","last_peered":"2024-08-31T06:06:23.691662+0000","last_clean":"2024-08-31T06:06:23.691662+0000","last_became_active":"2024-08-31T06:04:25.887176+0000","last_became_peered":"2024-08-31T06:04:25.887176+0000","last_unstale":"2024-08-31T06:06:23.691662+0000","last_undegraded":"2024-08-31T06:06:23.691662+0000","last_fullsized":"2024-08-31T06:06:23.691662+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:08:05.476804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.053524+0000","last_change":"2024-08-31T06:04:22.847747+0000","last_active":"2024-08-31T06:05:25.053524+0000","last_peered":"2024-08-31T06:05:25.053524+0000","last_clean":"2024-08-31T06:05:25.053524+0000","last_became_active":"2024-08-31T06:04:22.847332+0000","last_became_peered":"2024-08-31T06:04:22.847332+0000","last_unstale":"2024-08-31T06:05:25.053524+0000","last_undegraded":"2024-08-31T06:05:25.053524+0000","last_fullsized":"2024-08-31T06:05:25.053524+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:41:35.910991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.687796+0000","last_change":"2024-08-31T06:04:25.883110+0000","last_active":"2024-08-31T06:06:23.687796+0000","last_peered":"2024-08-31T06:06:23.687796+0000","last_clean":"2024-08-31T06:06:23.687796+0000","last_became_active":"2024-08-31T06:04:25.880487+0000","last_became_peered":"2024-08-31T06:04:25.880487+0000","last_unstale":"2024-08-31T06:06:23.687796+0000","last_undegraded":"2024-08-31T06:06:23.687796+0000","last_fullsized":"2024-08-31T06:06:23.687796+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:22:13.656078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.108210+0000","last_change":"2024-08-31T06:04:22.964758+0000","last_active":"2024-08-31T06:05:25.108210+0000","last_peered":"2024-08-31T06:05:25.108210+0000","last_clean":"2024-08-31T06:05:25.108210+0000","last_became_active":"2024-08-31T06:04:22.964391+0000","last_became_peered":"2024-08-31T06:04:22.964391+0000","last_unstale":"2024-08-31T06:05:25.108210+0000","last_undegraded":"2024-08-31T06:05:25.108210+0000","last_fullsized":"2024-08-31T06:05:25.108210+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:28:59.190657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.678294+0000","last_change":"2024-08-31T06:04:25.893738+0000","last_active":"2024-08-31T06:06:23.678294+0000","last_peered":"2024-08-31T06:06:23.678294+0000","last_clean":"2024-08-31T06:06:23.678294+0000","last_became_active":"2024-08-31T06:04:25.893308+0000","last_became_peered":"2024-08-31T06:04:25.893308+0000","last_unstale":"2024-08-31T06:06:23.678294+0000","last_undegraded":"2024-08-31T06:06:23.678294+0000","last_fullsized":"2024-08-31T06:06:23.678294+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:26:48.451894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":38,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.043990+0000","last_change":"2024-08-31T06:04:22.843346+0000","last_active":"2024-08-31T06:05:25.043990+0000","last_peered":"2024-08-31T06:05:25.043990+0000","last_clean":"2024-08-31T06:05:25.043990+0000","last_became_active":"2024-08-31T06:04:22.842249+0000","last_became_peered":"2024-08-31T06:04:22.842249+0000","last_unstale":"2024-08-31T06:05:25.043990+0000","last_undegraded":"2024-08-31T06:05:25.043990+0000","last_fullsized":"2024-08-31T06:05:25.043990+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:40:57.618940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70369,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":742,"num_read_kb":449,"num_write":178,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1147,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":211107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":249373,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710761,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":436463,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874285,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":1094149,"data_compressed":42124,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037813,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":436677,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234042,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":436575,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397568,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":933888,"data_stored":1029200,"data_compressed":42124,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593797,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":436066,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790025,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":1028352,"data_compressed":42124,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953552,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36208,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708944,"statfs":{"total":95995035648,"available":95957958656,"internally_reserved":0,"allocated":884736,"data_stored":438189,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149780,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":888832,"data_stored":436377,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411419,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36268,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708884,"statfs":{"total":95995035648,"available":95957897216,"internally_reserved":0,"allocated":946176,"data_stored":502170,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542239,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36268,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708884,"statfs":{"total":95995035648,"available":95957897216,"internally_reserved":0,"allocated":946176,"data_stored":501666,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705765,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":892928,"data_stored":437337,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34699,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34699,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34699,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":65807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":66311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":67805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-31T06:06:43.905 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:06:43.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:43 smithi184 ceph-mon[22331]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T06:06:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:06:44.646 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:44 smithi016 ceph-mon[17194]: pgmap v348: 97 pgs: 97 active+clean; 645 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2024-08-31T06:06:44.647 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:44 smithi016 ceph-mon[17194]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T06:06:44.647 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:44 smithi016 ceph-mon[17194]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T06:06:44.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:44 smithi155 ceph-mon[21742]: pgmap v348: 97 pgs: 97 active+clean; 645 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2024-08-31T06:06:44.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:44 smithi155 ceph-mon[21742]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T06:06:44.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:44 smithi155 ceph-mon[21742]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T06:06:44.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:44 smithi184 ceph-mon[22331]: pgmap v348: 97 pgs: 97 active+clean; 645 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2024-08-31T06:06:44.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:44 smithi184 ceph-mon[22331]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T06:06:44.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:44 smithi184 ceph-mon[22331]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T06:06:45.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.024+0000 7f33d3a00640 1 -- 172.21.15.16:0/86565970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc103900 msgr2=0x7f33cc103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:45.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.024+0000 7f33d3a00640 1 --2- 172.21.15.16:0/86565970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc103900 0x7f33cc103d10 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f33bc009900 tx=0x7f33bc02a3f0 comp rx=0 tx=0).stop 2024-08-31T06:06:45.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.024+0000 7f33d3a00640 1 -- 172.21.15.16:0/86565970 shutdown_connections 2024-08-31T06:06:45.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.024+0000 7f33d3a00640 1 --2- 172.21.15.16:0/86565970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33cc109240 0x7f33cc109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:45.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.024+0000 7f33d3a00640 1 --2- 172.21.15.16:0/86565970 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33cc109b90 0x7f33cc1103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:45.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.024+0000 7f33d3a00640 1 --2- 172.21.15.16:0/86565970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc103900 0x7f33cc103d10 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:45.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.024+0000 7f33d3a00640 1 -- 172.21.15.16:0/86565970 >> 172.21.15.16:0/86565970 conn(0x7f33cc07b5f0 msgr2=0x7f33cc07ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:45.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.024+0000 7f33d3a00640 1 -- 172.21.15.16:0/86565970 shutdown_connections 2024-08-31T06:06:45.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.024+0000 7f33d3a00640 1 -- 172.21.15.16:0/86565970 wait complete. 2024-08-31T06:06:45.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.026+0000 7f33d3a00640 1 Processor -- start 2024-08-31T06:06:45.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.026+0000 7f33d3a00640 1 -- start start 2024-08-31T06:06:45.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.027+0000 7f33d3a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33cc103900 0x7f33cc19c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:45.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.027+0000 7f33d3a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33cc109240 0x7f33cc19c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:45.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.027+0000 7f33d3a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc109b90 0x7f33cc1a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:45.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.027+0000 7f33d3a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc1138a0 con 0x7f33cc109b90 2024-08-31T06:06:45.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.027+0000 7f33d3a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc113720 con 0x7f33cc109240 2024-08-31T06:06:45.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.027+0000 7f33d3a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc113a20 con 0x7f33cc103900 2024-08-31T06:06:45.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.027+0000 7f33d1a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc109b90 0x7f33cc1a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:45.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.027+0000 7f33c3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33cc109240 0x7f33cc19c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:45.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.027+0000 7f33c3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33cc109240 0x7f33cc19c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:59798/0 (socket says 172.21.15.16:59798) 2024-08-31T06:06:45.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.027+0000 7f33d1a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc109b90 0x7f33cc1a1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50908/0 (socket says 172.21.15.16:50908) 2024-08-31T06:06:45.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.027+0000 7f33c3e00640 1 -- 172.21.15.16:0/1782639905 learned_addr learned my addr 172.21.15.16:0/1782639905 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:45.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.027+0000 7f33d1000640 1 --2- 172.21.15.16:0/1782639905 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33cc103900 0x7f33cc19c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:45.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.027+0000 7f33c3e00640 1 -- 172.21.15.16:0/1782639905 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33cc103900 msgr2=0x7f33cc19c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:45.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.027+0000 7f33c3e00640 1 --2- 172.21.15.16:0/1782639905 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33cc103900 0x7f33cc19c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:45.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.027+0000 7f33c3e00640 1 -- 172.21.15.16:0/1782639905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc109b90 msgr2=0x7f33cc1a1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:45.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.027+0000 7f33c3e00640 1 --2- 172.21.15.16:0/1782639905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc109b90 0x7f33cc1a1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:45.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.027+0000 7f33c3e00640 1 -- 172.21.15.16:0/1782639905 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33cc1a23c0 con 0x7f33cc109240 2024-08-31T06:06:45.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.028+0000 7f33c3e00640 1 --2- 172.21.15.16:0/1782639905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33cc109240 0x7f33cc19c9f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f33b400e9c0 tx=0x7f33b400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:45.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.028+0000 7f33ca000640 1 -- 172.21.15.16:0/1782639905 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b400ccc0 con 0x7f33cc109240 2024-08-31T06:06:45.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.028+0000 7f33ca000640 1 -- 172.21.15.16:0/1782639905 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33b40054d0 con 0x7f33cc109240 2024-08-31T06:06:45.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.028+0000 7f33d3a00640 1 -- 172.21.15.16:0/1782639905 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33cc1a26b0 con 0x7f33cc109240 2024-08-31T06:06:45.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.028+0000 7f33ca000640 1 -- 172.21.15.16:0/1782639905 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b4010430 con 0x7f33cc109240 2024-08-31T06:06:45.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.028+0000 7f33d3a00640 1 -- 172.21.15.16:0/1782639905 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33cc1a8e10 con 0x7f33cc109240 2024-08-31T06:06:45.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.030+0000 7f33d3a00640 1 -- 172.21.15.16:0/1782639905 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33cc103d10 con 0x7f33cc109240 2024-08-31T06:06:45.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.032+0000 7f33ca000640 1 -- 172.21.15.16:0/1782639905 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7f33b4005000 con 0x7f33cc109240 2024-08-31T06:06:45.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.033+0000 7f33ca000640 1 --2- 172.21.15.16:0/1782639905 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f33b00b4c00 0x7f33b00b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:45.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.034+0000 7f33ca000640 1 -- 172.21.15.16:0/1782639905 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f33b40e5820 con 0x7f33cc109240 2024-08-31T06:06:45.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.034+0000 7f33d1000640 1 --2- 172.21.15.16:0/1782639905 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f33b00b4c00 0x7f33b00b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:45.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.034+0000 7f33d1000640 1 --2- 172.21.15.16:0/1782639905 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f33b00b4c00 0x7f33b00b7110 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f33bc005ab0 tx=0x7f33bc005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:45.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.035+0000 7f33ca000640 1 -- 172.21.15.16:0/1782639905 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f33b4056130 con 0x7f33cc109240 2024-08-31T06:06:45.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.271+0000 7f33d3a00640 1 -- 172.21.15.16:0/1782639905 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33cc19d580 con 0x7f33cc109240 2024-08-31T06:06:45.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.272+0000 7f33ca000640 1 -- 172.21.15.16:0/1782639905 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11917 (secure 0 0 0) 0x7f33cc19d580 con 0x7f33cc109240 2024-08-31T06:06:45.275 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:06:45.275 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-08-31T06:06:38:485237+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:06:37.778867+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24698":{"gid":24698,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1089710405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1089710405},{"type":"v1","addr":"172.21.15.155:6837","nonce":1089710405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24737,24724,24712]},"id":1}]} 2024-08-31T06:06:45.275 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-08-31T06:06:45.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.275+0000 7f33d3a00640 1 -- 172.21.15.16:0/1782639905 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f33b00b4c00 msgr2=0x7f33b00b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:45.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.275+0000 7f33d3a00640 1 --2- 172.21.15.16:0/1782639905 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f33b00b4c00 0x7f33b00b7110 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f33bc005ab0 tx=0x7f33bc005a20 comp rx=0 tx=0).stop 2024-08-31T06:06:45.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.275+0000 7f33d3a00640 1 -- 172.21.15.16:0/1782639905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33cc109240 msgr2=0x7f33cc19c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:45.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.275+0000 7f33d3a00640 1 --2- 172.21.15.16:0/1782639905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33cc109240 0x7f33cc19c9f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f33b400e9c0 tx=0x7f33b400ee80 comp rx=0 tx=0).stop 2024-08-31T06:06:45.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.275+0000 7f33d3a00640 1 -- 172.21.15.16:0/1782639905 shutdown_connections 2024-08-31T06:06:45.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.275+0000 7f33d3a00640 1 --2- 172.21.15.16:0/1782639905 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f33b00b4c00 0x7f33b00b7110 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:45.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.275+0000 7f33d3a00640 1 --2- 172.21.15.16:0/1782639905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc109b90 0x7f33cc1a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:45.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.275+0000 7f33d3a00640 1 --2- 172.21.15.16:0/1782639905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33cc109240 0x7f33cc19c9f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:45.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.275+0000 7f33d3a00640 1 --2- 172.21.15.16:0/1782639905 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33cc103900 0x7f33cc19c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:45.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.275+0000 7f33d3a00640 1 -- 172.21.15.16:0/1782639905 >> 172.21.15.16:0/1782639905 conn(0x7f33cc07b5f0 msgr2=0x7f33cc107050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:45.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.275+0000 7f33d3a00640 1 -- 172.21.15.16:0/1782639905 shutdown_connections 2024-08-31T06:06:45.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:45.275+0000 7f33d3a00640 1 -- 172.21.15.16:0/1782639905 wait complete. 2024-08-31T06:06:45.615 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:45 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/1782639905' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:06:45.615 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:45 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:06:45.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:45 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/1782639905' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:06:45.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:45 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:06:45.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:45 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/1782639905' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:06:45.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:45 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:06:46.006 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph osd dump --format=json 2024-08-31T06:06:46.400 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:06:46.793 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:46 smithi016 ceph-mon[17194]: pgmap v349: 97 pgs: 97 active+clean; 645 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 7 op/s 2024-08-31T06:06:46.793 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:46 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:06:46.793 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:46 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:06:46.793 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:46 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:06:46.793 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:46 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:06:46.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:46 smithi184 ceph-mon[22331]: pgmap v349: 97 pgs: 97 active+clean; 645 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 7 op/s 2024-08-31T06:06:46.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:46 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:06:46.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:46 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:06:46.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:46 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:06:46.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:46 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:06:47.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.085+0000 7f0822600640 1 -- 172.21.15.16:0/66524059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f081c109090 msgr2=0x7f081c1094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:47.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.085+0000 7f0822600640 1 --2- 172.21.15.16:0/66524059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f081c109090 0x7f081c1094a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0808009900 tx=0x7f080802a410 comp rx=0 tx=0).stop 2024-08-31T06:06:47.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.085+0000 7f0822600640 1 -- 172.21.15.16:0/66524059 shutdown_connections 2024-08-31T06:06:47.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.085+0000 7f0822600640 1 --2- 172.21.15.16:0/66524059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f081c1099e0 0x7f081c110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:47.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.085+0000 7f0822600640 1 --2- 172.21.15.16:0/66524059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f081c109090 0x7f081c1094a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:47.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.085+0000 7f0822600640 1 --2- 172.21.15.16:0/66524059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f081c103750 0x7f081c103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:47.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.085+0000 7f0822600640 1 -- 172.21.15.16:0/66524059 >> 172.21.15.16:0/66524059 conn(0x7f081c0ff1e0 msgr2=0x7f081c101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:47.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.085+0000 7f0822600640 1 -- 172.21.15.16:0/66524059 shutdown_connections 2024-08-31T06:06:47.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.085+0000 7f0822600640 1 -- 172.21.15.16:0/66524059 wait complete. 2024-08-31T06:06:47.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.086+0000 7f0822600640 1 Processor -- start 2024-08-31T06:06:47.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.087+0000 7f0822600640 1 -- start start 2024-08-31T06:06:47.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.087+0000 7f0822600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f081c103750 0x7f081c19c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.087+0000 7f0822600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f081c109090 0x7f081c19c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.087+0000 7f0822600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f081c1099e0 0x7f081c1a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.087+0000 7f0822600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f081c1136a0 con 0x7f081c109090 2024-08-31T06:06:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.087+0000 7f0822600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f081c113520 con 0x7f081c103750 2024-08-31T06:06:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.087+0000 7f0822600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f081c113820 con 0x7f081c1099e0 2024-08-31T06:06:47.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.088+0000 7f0821200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f081c103750 0x7f081c19c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:47.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.088+0000 7f0821200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f081c103750 0x7f081c19c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:33648/0 (socket says 172.21.15.16:33648) 2024-08-31T06:06:47.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.088+0000 7f081be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f081c109090 0x7f081c19c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:47.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.088+0000 7f0821200640 1 -- 172.21.15.16:0/4162404704 learned_addr learned my addr 172.21.15.16:0/4162404704 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:47.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.088+0000 7f0821c00640 1 --2- 172.21.15.16:0/4162404704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f081c1099e0 0x7f081c1a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:47.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.088+0000 7f0821200640 1 -- 172.21.15.16:0/4162404704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f081c1099e0 msgr2=0x7f081c1a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:47.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.088+0000 7f0821200640 1 --2- 172.21.15.16:0/4162404704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f081c1099e0 0x7f081c1a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:47.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.088+0000 7f0821200640 1 -- 172.21.15.16:0/4162404704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f081c109090 msgr2=0x7f081c19c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:47.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.088+0000 7f0821200640 1 --2- 172.21.15.16:0/4162404704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f081c109090 0x7f081c19c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:47.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.088+0000 7f0821200640 1 -- 172.21.15.16:0/4162404704 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f081c1a2220 con 0x7f081c103750 2024-08-31T06:06:47.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.089+0000 7f0821200640 1 --2- 172.21.15.16:0/4162404704 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f081c103750 0x7f081c19c2b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f081000e9f0 tx=0x7f081000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:47.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.089+0000 7f0819600640 1 -- 172.21.15.16:0/4162404704 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081000cd80 con 0x7f081c103750 2024-08-31T06:06:47.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.089+0000 7f0819600640 1 -- 172.21.15.16:0/4162404704 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08100108e0 con 0x7f081c103750 2024-08-31T06:06:47.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.089+0000 7f0822600640 1 -- 172.21.15.16:0/4162404704 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f081c1a2510 con 0x7f081c103750 2024-08-31T06:06:47.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.089+0000 7f0819600640 1 -- 172.21.15.16:0/4162404704 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0810018980 con 0x7f081c103750 2024-08-31T06:06:47.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.089+0000 7f0822600640 1 -- 172.21.15.16:0/4162404704 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f081c1a8c70 con 0x7f081c103750 2024-08-31T06:06:47.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.091+0000 7f080ea00640 1 -- 172.21.15.16:0/4162404704 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f081c103b60 con 0x7f081c103750 2024-08-31T06:06:47.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.093+0000 7f0819600640 1 -- 172.21.15.16:0/4162404704 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7f0810010430 con 0x7f081c103750 2024-08-31T06:06:47.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.094+0000 7f0819600640 1 --2- 172.21.15.16:0/4162404704 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f08040b4c00 0x7f08040b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:47.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.094+0000 7f0819600640 1 -- 172.21.15.16:0/4162404704 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f08100e5540 con 0x7f081c103750 2024-08-31T06:06:47.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.095+0000 7f081be00640 1 --2- 172.21.15.16:0/4162404704 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f08040b4c00 0x7f08040b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:47.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.095+0000 7f081be00640 1 --2- 172.21.15.16:0/4162404704 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f08040b4c00 0x7f08040b7110 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f080802aa00 tx=0x7f0808005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:47.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.098+0000 7f0819600640 1 -- 172.21.15.16:0/4162404704 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f08100abf00 con 0x7f081c103750 2024-08-31T06:06:47.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:46 smithi155 ceph-mon[21742]: pgmap v349: 97 pgs: 97 active+clean; 645 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 7 op/s 2024-08-31T06:06:47.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:46 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:06:47.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:46 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:06:47.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:46 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:06:47.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:46 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:06:47.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.315+0000 7f080ea00640 1 -- 172.21.15.16:0/4162404704 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f081c063d60 con 0x7f081c103750 2024-08-31T06:06:47.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.317+0000 7f0819600640 1 -- 172.21.15.16:0/4162404704 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+23043 (secure 0 0 0) 0x7f08100b06c0 con 0x7f081c103750 2024-08-31T06:06:47.320 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:06:47.320 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":80,"fsid":"673e4aa0-675d-11ef-bcd4-c7b262605968","created":"2024-08-31T05:55:09.632536+0000","modified":"2024-08-31T06:05:25.039584+0000","last_up_change":"2024-08-31T06:03:27.893697+0000","last_in_change":"2024-08-31T06:03:05.664945+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-31T05:59:23.642270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-31T06:04:20.871835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-31T06:04:24.119499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e349dae-6aa5-4b6d-9107-a945655e11bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6803","nonce":1725037400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6805","nonce":1725037400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6809","nonce":1725037400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6807","nonce":1725037400}]},"public_addr":"172.21.15.16:6803/1725037400","cluster_addr":"172.21.15.16:6805/1725037400","heartbeat_back_addr":"172.21.15.16:6809/1725037400","heartbeat_front_addr":"172.21.15.16:6807/1725037400","state":["exists","up"]},{"osd":1,"uuid":"bd645dc3-93b5-40af-89d8-08ccec22e50d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6801","nonce":4241618440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6803","nonce":4241618440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6807","nonce":4241618440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6805","nonce":4241618440}]},"public_addr":"172.21.15.155:6801/4241618440","cluster_addr":"172.21.15.155:6803/4241618440","heartbeat_back_addr":"172.21.15.155:6807/4241618440","heartbeat_front_addr":"172.21.15.155:6805/4241618440","state":["exists","up"]},{"osd":2,"uuid":"c69c8c19-32f8-400a-86d4-0f4ce240a7de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6801","nonce":2193296422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6803","nonce":2193296422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6807","nonce":2193296422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6805","nonce":2193296422}]},"public_addr":"172.21.15.184:6801/2193296422","cluster_addr":"172.21.15.184:6803/2193296422","heartbeat_back_addr":"172.21.15.184:6807/2193296422","heartbeat_front_addr":"172.21.15.184:6805/2193296422","state":["exists","up"]},{"osd":3,"uuid":"8892e35f-6880-434c-a805-ab8ccbed2bf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6811","nonce":81681815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6813","nonce":81681815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6817","nonce":81681815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6815","nonce":81681815}]},"public_addr":"172.21.15.16:6811/81681815","cluster_addr":"172.21.15.16:6813/81681815","heartbeat_back_addr":"172.21.15.16:6817/81681815","heartbeat_front_addr":"172.21.15.16:6815/81681815","state":["exists","up"]},{"osd":4,"uuid":"d2515e73-88f7-4654-9762-adb8802777ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6809","nonce":3592422019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6811","nonce":3592422019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6815","nonce":3592422019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6813","nonce":3592422019}]},"public_addr":"172.21.15.155:6809/3592422019","cluster_addr":"172.21.15.155:6811/3592422019","heartbeat_back_addr":"172.21.15.155:6815/3592422019","heartbeat_front_addr":"172.21.15.155:6813/3592422019","state":["exists","up"]},{"osd":5,"uuid":"5ce6e2fa-c17c-49b3-9402-f19bfc21076d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6809","nonce":2054894065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6811","nonce":2054894065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6815","nonce":2054894065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6813","nonce":2054894065}]},"public_addr":"172.21.15.184:6809/2054894065","cluster_addr":"172.21.15.184:6811/2054894065","heartbeat_back_addr":"172.21.15.184:6815/2054894065","heartbeat_front_addr":"172.21.15.184:6813/2054894065","state":["exists","up"]},{"osd":6,"uuid":"f257d8d1-ad59-4bd8-821d-bf29e50b547e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6819","nonce":1697421092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6821","nonce":1697421092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6825","nonce":1697421092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6823","nonce":1697421092}]},"public_addr":"172.21.15.16:6819/1697421092","cluster_addr":"172.21.15.16:6821/1697421092","heartbeat_back_addr":"172.21.15.16:6825/1697421092","heartbeat_front_addr":"172.21.15.16:6823/1697421092","state":["exists","up"]},{"osd":7,"uuid":"8e76c20e-8d25-423b-b7e8-de642d9c9a6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6817","nonce":2671827901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6819","nonce":2671827901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6823","nonce":2671827901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6821","nonce":2671827901}]},"public_addr":"172.21.15.155:6817/2671827901","cluster_addr":"172.21.15.155:6819/2671827901","heartbeat_back_addr":"172.21.15.155:6823/2671827901","heartbeat_front_addr":"172.21.15.155:6821/2671827901","state":["exists","up"]},{"osd":8,"uuid":"0f0bb23f-4bc3-4198-8381-5b5cd16ae394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6817","nonce":2479117823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6819","nonce":2479117823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6823","nonce":2479117823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6821","nonce":2479117823}]},"public_addr":"172.21.15.184:6817/2479117823","cluster_addr":"172.21.15.184:6819/2479117823","heartbeat_back_addr":"172.21.15.184:6823/2479117823","heartbeat_front_addr":"172.21.15.184:6821/2479117823","state":["exists","up"]},{"osd":9,"uuid":"bc27afcb-7871-4f72-a043-044fd5fe7d86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6827","nonce":3015700546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6829","nonce":3015700546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6833","nonce":3015700546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6831","nonce":3015700546}]},"public_addr":"172.21.15.16:6827/3015700546","cluster_addr":"172.21.15.16:6829/3015700546","heartbeat_back_addr":"172.21.15.16:6833/3015700546","heartbeat_front_addr":"172.21.15.16:6831/3015700546","state":["exists","up"]},{"osd":10,"uuid":"f2548997-72ee-400e-ab8d-5966d85b2cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6825","nonce":2427122475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6827","nonce":2427122475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6831","nonce":2427122475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6829","nonce":2427122475}]},"public_addr":"172.21.15.155:6825/2427122475","cluster_addr":"172.21.15.155:6827/2427122475","heartbeat_back_addr":"172.21.15.155:6831/2427122475","heartbeat_front_addr":"172.21.15.155:6829/2427122475","state":["exists","up"]},{"osd":11,"uuid":"8297da1d-d3e2-4a4d-9396-426dfafa3e95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6825","nonce":399265701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6827","nonce":399265701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6831","nonce":399265701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6829","nonce":399265701}]},"public_addr":"172.21.15.184:6825/399265701","cluster_addr":"172.21.15.184:6827/399265701","heartbeat_back_addr":"172.21.15.184:6831/399265701","heartbeat_front_addr":"172.21.15.184:6829/399265701","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:58:30.842360+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:58:57.164488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:59:20.778103+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:59:50.123991+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:00:16.789933+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:00:41.788905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:01:12.281376+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:01:39.211858+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:02:04.958771+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:02:35.645947+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:03:01.940433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:03:26.728673+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3923220960":"2024-09-01T05:57:23.153214+0000","172.21.15.16:0/3013533836":"2024-09-01T05:57:23.153214+0000","172.21.15.16:6800/3755683228":"2024-09-01T05:55:45.075902+0000","172.21.15.16:0/3064435470":"2024-09-01T05:55:45.075902+0000","172.21.15.16:0/911709962":"2024-09-01T05:55:45.075902+0000","172.21.15.16:0/2937368967":"2024-09-01T05:55:45.075902+0000","172.21.15.16:6800/1378972230":"2024-09-01T05:56:14.195704+0000","172.21.15.16:0/3047316995":"2024-09-01T05:56:14.195704+0000","172.21.15.16:6801/3755683228":"2024-09-01T05:55:45.075902+0000","172.21.15.16:6801/2571082634":"2024-09-01T05:57:23.153214+0000","172.21.15.16:0/2136704295":"2024-09-01T05:56:14.195704+0000","172.21.15.16:0/2259332778":"2024-09-01T05:56:14.195704+0000","172.21.15.16:6800/2571082634":"2024-09-01T05:57:23.153214+0000","172.21.15.16:6801/1378972230":"2024-09-01T05:56:14.195704+0000","172.21.15.16:0/2126457673":"2024-09-01T05:57:23.153214+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-31T06:06:47.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.320+0000 7f080ea00640 1 -- 172.21.15.16:0/4162404704 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f08040b4c00 msgr2=0x7f08040b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:47.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.320+0000 7f080ea00640 1 --2- 172.21.15.16:0/4162404704 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f08040b4c00 0x7f08040b7110 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f080802aa00 tx=0x7f0808005a20 comp rx=0 tx=0).stop 2024-08-31T06:06:47.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.320+0000 7f080ea00640 1 -- 172.21.15.16:0/4162404704 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f081c103750 msgr2=0x7f081c19c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:47.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.320+0000 7f080ea00640 1 --2- 172.21.15.16:0/4162404704 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f081c103750 0x7f081c19c2b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f081000e9f0 tx=0x7f081000eeb0 comp rx=0 tx=0).stop 2024-08-31T06:06:47.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.320+0000 7f080ea00640 1 -- 172.21.15.16:0/4162404704 shutdown_connections 2024-08-31T06:06:47.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.320+0000 7f080ea00640 1 --2- 172.21.15.16:0/4162404704 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f08040b4c00 0x7f08040b7110 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:47.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.320+0000 7f080ea00640 1 --2- 172.21.15.16:0/4162404704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f081c1099e0 0x7f081c1a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:47.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.320+0000 7f080ea00640 1 --2- 172.21.15.16:0/4162404704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f081c109090 0x7f081c19c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:47.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.320+0000 7f080ea00640 1 --2- 172.21.15.16:0/4162404704 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f081c103750 0x7f081c19c2b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:47.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.320+0000 7f080ea00640 1 -- 172.21.15.16:0/4162404704 >> 172.21.15.16:0/4162404704 conn(0x7f081c0ff1e0 msgr2=0x7f081c1076d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:47.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.320+0000 7f080ea00640 1 -- 172.21.15.16:0/4162404704 shutdown_connections 2024-08-31T06:06:47.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:47.320+0000 7f080ea00640 1 -- 172.21.15.16:0/4162404704 wait complete. 2024-08-31T06:06:47.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:47 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:06:47.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:47 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:06:47.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:47 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:06:47.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:47 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:06:47.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:47 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/4162404704' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T06:06:48.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:47 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:06:48.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:47 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:06:48.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:47 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:06:48.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:47 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:06:48.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:47 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/4162404704' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T06:06:48.225 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:47 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:06:48.225 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:47 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:06:48.225 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:47 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:06:48.225 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:47 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:06:48.225 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:47 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/4162404704' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T06:06:48.701 DEBUG:tasks.fwd_scrub:Yielding 2024-08-31T06:06:48.701 INFO:teuthology.run_tasks:Running task exec... 2024-08-31T06:06:48.710 INFO:teuthology.task.exec:Executing custom commands... 2024-08-31T06:06:48.710 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-08-31T06:06:48.710 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-08-31T06:06:48.711 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-08-31T06:06:48.711 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T06:06:48.711 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T06:06:48.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:48.945+0000 7f4c1b600640 1 -- 172.21.15.16:0/4062664039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c141091a0 msgr2=0x7f4c141095b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:48.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:48.945+0000 7f4c1b600640 1 --2- 172.21.15.16:0/4062664039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c141091a0 0x7f4c141095b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4c00009900 tx=0x7f4c0002a410 comp rx=0 tx=0).stop 2024-08-31T06:06:48.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:48.945+0000 7f4c1b600640 1 -- 172.21.15.16:0/4062664039 shutdown_connections 2024-08-31T06:06:48.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:48.945+0000 7f4c1b600640 1 --2- 172.21.15.16:0/4062664039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c14109af0 0x7f4c14110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:48.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:48.945+0000 7f4c1b600640 1 --2- 172.21.15.16:0/4062664039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c141091a0 0x7f4c141095b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:48.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:48.945+0000 7f4c1b600640 1 --2- 172.21.15.16:0/4062664039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c14103870 0x7f4c14103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:48.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:48.945+0000 7f4c1b600640 1 -- 172.21.15.16:0/4062664039 >> 172.21.15.16:0/4062664039 conn(0x7f4c140ff2b0 msgr2=0x7f4c14101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:48.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:48.945+0000 7f4c1b600640 1 -- 172.21.15.16:0/4062664039 shutdown_connections 2024-08-31T06:06:48.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:48.945+0000 7f4c1b600640 1 -- 172.21.15.16:0/4062664039 wait complete. 2024-08-31T06:06:48.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:48.946+0000 7f4c1b600640 1 Processor -- start 2024-08-31T06:06:48.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:48.946+0000 7f4c1b600640 1 -- start start 2024-08-31T06:06:48.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:48.947+0000 7f4c1b600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c14103870 0x7f4c1419a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:48.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:48.947+0000 7f4c1b600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c141091a0 0x7f4c1419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:48.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:48.947+0000 7f4c1b600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c14109af0 0x7f4c1419f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:48.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:48.947+0000 7f4c1b600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4c141137b0 con 0x7f4c141091a0 2024-08-31T06:06:48.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:48.947+0000 7f4c1b600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4c14113630 con 0x7f4c14109af0 2024-08-31T06:06:48.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:48.947+0000 7f4c1b600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4c14113930 con 0x7f4c14103870 2024-08-31T06:06:48.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:48.947+0000 7f4c19600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c14109af0 0x7f4c1419f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:48.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:48.947+0000 7f4c18c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c14103870 0x7f4c1419a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:48.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:48.947+0000 7f4c19600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c14109af0 0x7f4c1419f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:33672/0 (socket says 172.21.15.16:33672) 2024-08-31T06:06:48.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:48.947+0000 7f4c13e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c141091a0 0x7f4c1419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:48.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:48.947+0000 7f4c19600640 1 -- 172.21.15.16:0/1915456789 learned_addr learned my addr 172.21.15.16:0/1915456789 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:48.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:48.947+0000 7f4c18c00640 1 --2- 172.21.15.16:0/1915456789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c14103870 0x7f4c1419a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:37542/0 (socket says 172.21.15.16:37542) 2024-08-31T06:06:48.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:48.947+0000 7f4c19600640 1 -- 172.21.15.16:0/1915456789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c14103870 msgr2=0x7f4c1419a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:48.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:48.947+0000 7f4c19600640 1 --2- 172.21.15.16:0/1915456789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c14103870 0x7f4c1419a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:48.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:48.947+0000 7f4c19600640 1 -- 172.21.15.16:0/1915456789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c141091a0 msgr2=0x7f4c1419a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:48.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:48.947+0000 7f4c19600640 1 --2- 172.21.15.16:0/1915456789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c141091a0 0x7f4c1419a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:48.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:48.947+0000 7f4c19600640 1 -- 172.21.15.16:0/1915456789 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c141a00e0 con 0x7f4c14109af0 2024-08-31T06:06:48.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:48.948+0000 7f4c19600640 1 --2- 172.21.15.16:0/1915456789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c14109af0 0x7f4c1419f9e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c00eb10 tx=0x7f4c0c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:48.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:48.948+0000 7f4c11600640 1 -- 172.21.15.16:0/1915456789 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c004b20 con 0x7f4c14109af0 2024-08-31T06:06:48.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:48.948+0000 7f4c11600640 1 -- 172.21.15.16:0/1915456789 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c0c004cc0 con 0x7f4c14109af0 2024-08-31T06:06:48.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:48.948+0000 7f4c1b600640 1 -- 172.21.15.16:0/1915456789 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c141a03d0 con 0x7f4c14109af0 2024-08-31T06:06:48.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:48.948+0000 7f4c11600640 1 -- 172.21.15.16:0/1915456789 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c010670 con 0x7f4c14109af0 2024-08-31T06:06:48.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:48.948+0000 7f4c1b600640 1 -- 172.21.15.16:0/1915456789 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c141a6b60 con 0x7f4c14109af0 2024-08-31T06:06:48.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:48.950+0000 7f4c1b600640 1 -- 172.21.15.16:0/1915456789 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c14104ae0 con 0x7f4c14109af0 2024-08-31T06:06:48.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:48.954+0000 7f4c11600640 1 -- 172.21.15.16:0/1915456789 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7f4c0c005020 con 0x7f4c14109af0 2024-08-31T06:06:48.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:48.956+0000 7f4c11600640 1 --2- 172.21.15.16:0/1915456789 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f4bfc0b4b80 0x7f4bfc0b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:48.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:48.957+0000 7f4c11600640 1 -- 172.21.15.16:0/1915456789 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f4c0c0e58a0 con 0x7f4c14109af0 2024-08-31T06:06:48.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:48.957+0000 7f4c18c00640 1 --2- 172.21.15.16:0/1915456789 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f4bfc0b4b80 0x7f4bfc0b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:48.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:48.958+0000 7f4c18c00640 1 --2- 172.21.15.16:0/1915456789 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f4bfc0b4b80 0x7f4bfc0b7090 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4c08009bd0 tx=0x7f4c08005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:48.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:48.959+0000 7f4c11600640 1 -- 172.21.15.16:0/1915456789 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f4c0c0ac8b0 con 0x7f4c14109af0 2024-08-31T06:06:49.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:48 smithi155 ceph-mon[21742]: pgmap v350: 97 pgs: 97 active+clean; 645 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2024-08-31T06:06:49.132 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:06:49.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:48 smithi016 ceph-mon[17194]: pgmap v350: 97 pgs: 97 active+clean; 645 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2024-08-31T06:06:49.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.212+0000 7f4c1b600640 1 -- 172.21.15.16:0/1915456789 --> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f4c1410b930 con 0x7f4bfc0b4b80 2024-08-31T06:06:49.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:48 smithi184 ceph-mon[22331]: pgmap v350: 97 pgs: 97 active+clean; 645 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2024-08-31T06:06:49.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.390+0000 7f4c11600640 1 -- 172.21.15.16:0/1915456789 <== mgr.14220 v2:172.21.15.16:6800/4279751337 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4c1410b930 con 0x7f4bfc0b4b80 2024-08-31T06:06:49.392 INFO:teuthology.orchestra.run.smithi016.stdout:Schedule set for path / 2024-08-31T06:06:49.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.392+0000 7f4c1b600640 1 -- 172.21.15.16:0/1915456789 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f4bfc0b4b80 msgr2=0x7f4bfc0b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:49.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.392+0000 7f4c1b600640 1 --2- 172.21.15.16:0/1915456789 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f4bfc0b4b80 0x7f4bfc0b7090 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4c08009bd0 tx=0x7f4c08005f90 comp rx=0 tx=0).stop 2024-08-31T06:06:49.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.392+0000 7f4c1b600640 1 -- 172.21.15.16:0/1915456789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c14109af0 msgr2=0x7f4c1419f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:49.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.392+0000 7f4c1b600640 1 --2- 172.21.15.16:0/1915456789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c14109af0 0x7f4c1419f9e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c00eb10 tx=0x7f4c0c00c2d0 comp rx=0 tx=0).stop 2024-08-31T06:06:49.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.392+0000 7f4c1b600640 1 -- 172.21.15.16:0/1915456789 shutdown_connections 2024-08-31T06:06:49.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.392+0000 7f4c1b600640 1 --2- 172.21.15.16:0/1915456789 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f4bfc0b4b80 0x7f4bfc0b7090 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:49.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.392+0000 7f4c1b600640 1 --2- 172.21.15.16:0/1915456789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c14109af0 0x7f4c1419f9e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:49.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.392+0000 7f4c1b600640 1 --2- 172.21.15.16:0/1915456789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c141091a0 0x7f4c1419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:49.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.392+0000 7f4c1b600640 1 --2- 172.21.15.16:0/1915456789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c14103870 0x7f4c1419a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:49.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.392+0000 7f4c1b600640 1 -- 172.21.15.16:0/1915456789 >> 172.21.15.16:0/1915456789 conn(0x7f4c140ff2b0 msgr2=0x7f4c141016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:49.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.393+0000 7f4c1b600640 1 -- 172.21.15.16:0/1915456789 shutdown_connections 2024-08-31T06:06:49.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.393+0000 7f4c1b600640 1 -- 172.21.15.16:0/1915456789 wait complete. 2024-08-31T06:06:49.424 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-08-31T06:06:49.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.655+0000 7f0c0b200640 1 -- 172.21.15.16:0/560148196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c04108fe0 msgr2=0x7f0c041093f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:49.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.655+0000 7f0c0b200640 1 --2- 172.21.15.16:0/560148196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c04108fe0 0x7f0c041093f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4009900 tx=0x7f0bf402a410 comp rx=0 tx=0).stop 2024-08-31T06:06:49.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.655+0000 7f0c0b200640 1 -- 172.21.15.16:0/560148196 shutdown_connections 2024-08-31T06:06:49.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.655+0000 7f0c0b200640 1 --2- 172.21.15.16:0/560148196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c04108fe0 0x7f0c041093f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:49.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.655+0000 7f0c0b200640 1 --2- 172.21.15.16:0/560148196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c04109930 0x7f0c04110160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:49.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.655+0000 7f0c0b200640 1 --2- 172.21.15.16:0/560148196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c041036b0 0x7f0c04103ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:49.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.655+0000 7f0c0b200640 1 -- 172.21.15.16:0/560148196 >> 172.21.15.16:0/560148196 conn(0x7f0c040ff0d0 msgr2=0x7f0c04101520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:49.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.655+0000 7f0c0b200640 1 -- 172.21.15.16:0/560148196 shutdown_connections 2024-08-31T06:06:49.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.655+0000 7f0c0b200640 1 -- 172.21.15.16:0/560148196 wait complete. 2024-08-31T06:06:49.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.656+0000 7f0c0b200640 1 Processor -- start 2024-08-31T06:06:49.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.657+0000 7f0c0b200640 1 -- start start 2024-08-31T06:06:49.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.657+0000 7f0c0b200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c041036b0 0x7f0c040754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:49.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.657+0000 7f0c0b200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c04108fe0 0x7f0c040759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:49.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.657+0000 7f0c0b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c04109930 0x7f0c04075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:49.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.657+0000 7f0c0b200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0c041135a0 con 0x7f0c04109930 2024-08-31T06:06:49.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.657+0000 7f0c0b200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0c04113420 con 0x7f0c04108fe0 2024-08-31T06:06:49.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.657+0000 7f0c0b200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0c04113720 con 0x7f0c041036b0 2024-08-31T06:06:49.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.657+0000 7f0c09e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c041036b0 0x7f0c040754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:49.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.657+0000 7f0c09400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c04108fe0 0x7f0c040759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:49.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.658+0000 7f0c09e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c041036b0 0x7f0c040754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:37592/0 (socket says 172.21.15.16:37592) 2024-08-31T06:06:49.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.658+0000 7f0c0a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c04109930 0x7f0c04075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:49.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.658+0000 7f0c09e00640 1 -- 172.21.15.16:0/3445285291 learned_addr learned my addr 172.21.15.16:0/3445285291 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:49.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.658+0000 7f0c09400640 1 -- 172.21.15.16:0/3445285291 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c041036b0 msgr2=0x7f0c040754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:49.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.658+0000 7f0c09400640 1 --2- 172.21.15.16:0/3445285291 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c041036b0 0x7f0c040754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:49.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.658+0000 7f0c09400640 1 -- 172.21.15.16:0/3445285291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c04109930 msgr2=0x7f0c04075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:49.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.658+0000 7f0c09400640 1 --2- 172.21.15.16:0/3445285291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c04109930 0x7f0c04075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:49.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.658+0000 7f0c09400640 1 -- 172.21.15.16:0/3445285291 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c041a8310 con 0x7f0c04108fe0 2024-08-31T06:06:49.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.658+0000 7f0c09e00640 1 --2- 172.21.15.16:0/3445285291 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c041036b0 0x7f0c040754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:06:49.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.658+0000 7f0c09400640 1 --2- 172.21.15.16:0/3445285291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c04108fe0 0x7f0c040759f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0bf402a930 tx=0x7f0bf4031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:49.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.659+0000 7f0bfea00640 1 -- 172.21.15.16:0/3445285291 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf403f070 con 0x7f0c04108fe0 2024-08-31T06:06:49.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.659+0000 7f0c0b200640 1 -- 172.21.15.16:0/3445285291 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c041a85a0 con 0x7f0c04108fe0 2024-08-31T06:06:49.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.659+0000 7f0bfea00640 1 -- 172.21.15.16:0/3445285291 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bf4004490 con 0x7f0c04108fe0 2024-08-31T06:06:49.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.659+0000 7f0bfea00640 1 -- 172.21.15.16:0/3445285291 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4035070 con 0x7f0c04108fe0 2024-08-31T06:06:49.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.659+0000 7f0c0b200640 1 -- 172.21.15.16:0/3445285291 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c041a8ae0 con 0x7f0c04108fe0 2024-08-31T06:06:49.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.660+0000 7f0c0b200640 1 -- 172.21.15.16:0/3445285291 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c04103ac0 con 0x7f0c04108fe0 2024-08-31T06:06:49.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.663+0000 7f0bfea00640 1 -- 172.21.15.16:0/3445285291 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7f0bf403b460 con 0x7f0c04108fe0 2024-08-31T06:06:49.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.665+0000 7f0bfea00640 1 --2- 172.21.15.16:0/3445285291 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f0bec0b4c30 0x7f0bec0b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:49.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.665+0000 7f0bfea00640 1 -- 172.21.15.16:0/3445285291 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f0bf41058f0 con 0x7f0c04108fe0 2024-08-31T06:06:49.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.665+0000 7f0c09e00640 1 --2- 172.21.15.16:0/3445285291 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f0bec0b4c30 0x7f0bec0b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:49.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.666+0000 7f0bfea00640 1 -- 172.21.15.16:0/3445285291 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f0bf40cc2b0 con 0x7f0c04108fe0 2024-08-31T06:06:49.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.666+0000 7f0c09e00640 1 --2- 172.21.15.16:0/3445285291 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f0bec0b4c30 0x7f0bec0b7140 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0c04076f00 tx=0x7f0bf8005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:49.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.793+0000 7f111e000640 1 -- 172.21.15.16:0/1093227960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1118077120 msgr2=0x7f1118075550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:49.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.793+0000 7f111e000640 1 --2- 172.21.15.16:0/1093227960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1118077120 0x7f1118075550 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f110c009900 tx=0x7f110c02a400 comp rx=0 tx=0).stop 2024-08-31T06:06:49.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.793+0000 7f111e000640 1 -- 172.21.15.16:0/1093227960 shutdown_connections 2024-08-31T06:06:49.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.793+0000 7f111e000640 1 --2- 172.21.15.16:0/1093227960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1118114020 0x7f11181164f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:49.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.793+0000 7f111e000640 1 --2- 172.21.15.16:0/1093227960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1118075b70 0x7f1118075f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:49.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.793+0000 7f111e000640 1 --2- 172.21.15.16:0/1093227960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1118077120 0x7f1118075550 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:49.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.793+0000 7f111e000640 1 -- 172.21.15.16:0/1093227960 >> 172.21.15.16:0/1093227960 conn(0x7f11180ff1e0 msgr2=0x7f1118101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:49.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.793+0000 7f111e000640 1 -- 172.21.15.16:0/1093227960 shutdown_connections 2024-08-31T06:06:49.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.793+0000 7f111e000640 1 -- 172.21.15.16:0/1093227960 wait complete. 2024-08-31T06:06:49.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.795+0000 7f111e000640 1 Processor -- start 2024-08-31T06:06:49.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.795+0000 7f111e000640 1 -- start start 2024-08-31T06:06:49.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.795+0000 7f111e000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1118075b70 0x7f1118074be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:49.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.795+0000 7f111e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1118077120 0x7f111806deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:49.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.795+0000 7f111e000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1118114020 0x7f111806e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:49.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.795+0000 7f111e000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1118119cb0 con 0x7f1118077120 2024-08-31T06:06:49.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.795+0000 7f111e000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1118119b30 con 0x7f1118114020 2024-08-31T06:06:49.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.795+0000 7f111e000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1118119e30 con 0x7f1118075b70 2024-08-31T06:06:49.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.796+0000 7f1117e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1118077120 0x7f111806deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:49.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.796+0000 7f111cc00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1118075b70 0x7f1118074be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:49.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.796+0000 7f111cc00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1118075b70 0x7f1118074be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:37606/0 (socket says 172.21.15.16:37606) 2024-08-31T06:06:49.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.796+0000 7f111cc00640 1 -- 172.21.15.16:0/2990082532 learned_addr learned my addr 172.21.15.16:0/2990082532 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:49.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.796+0000 7f111d600640 1 --2- 172.21.15.16:0/2990082532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1118114020 0x7f111806e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:49.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.796+0000 7f111cc00640 1 -- 172.21.15.16:0/2990082532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1118114020 msgr2=0x7f111806e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:49.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.796+0000 7f111cc00640 1 --2- 172.21.15.16:0/2990082532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1118114020 0x7f111806e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:49.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.796+0000 7f111cc00640 1 -- 172.21.15.16:0/2990082532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1118077120 msgr2=0x7f111806deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:49.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.796+0000 7f111cc00640 1 --2- 172.21.15.16:0/2990082532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1118077120 0x7f111806deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:49.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.796+0000 7f111cc00640 1 -- 172.21.15.16:0/2990082532 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f111806ec30 con 0x7f1118075b70 2024-08-31T06:06:49.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.797+0000 7f111cc00640 1 --2- 172.21.15.16:0/2990082532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1118075b70 0x7f1118074be0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f110c005ab0 tx=0x7f110c003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:49.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.797+0000 7f1117e00640 1 --2- 172.21.15.16:0/2990082532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1118077120 0x7f111806deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:06:49.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.797+0000 7f1115600640 1 -- 172.21.15.16:0/2990082532 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c03f070 con 0x7f1118075b70 2024-08-31T06:06:49.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.797+0000 7f1115600640 1 -- 172.21.15.16:0/2990082532 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f110c031b60 con 0x7f1118075b70 2024-08-31T06:06:49.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.797+0000 7f1115600640 1 -- 172.21.15.16:0/2990082532 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c035030 con 0x7f1118075b70 2024-08-31T06:06:49.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.797+0000 7f111e000640 1 -- 172.21.15.16:0/2990082532 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1118073850 con 0x7f1118075b70 2024-08-31T06:06:49.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.797+0000 7f111e000640 1 -- 172.21.15.16:0/2990082532 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1118073ce0 con 0x7f1118075b70 2024-08-31T06:06:49.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.801+0000 7f11261cd740 1 Processor -- start 2024-08-31T06:06:49.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.801+0000 7f11261cd740 1 -- start start 2024-08-31T06:06:49.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.801+0000 7f11261cd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ad25be350 0x555ad29647d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:49.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.801+0000 7f11261cd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ad2595490 0x555ad2964d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:49.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.801+0000 7f11261cd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ad259d670 0x555ad2965250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:49.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.801+0000 7f11261cd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555ad28da750 con 0x555ad2595490 2024-08-31T06:06:49.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.801+0000 7f11261cd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555ad28c9ce0 con 0x555ad259d670 2024-08-31T06:06:49.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.801+0000 7f11261cd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x555ad28cdc90 con 0x555ad25be350 2024-08-31T06:06:49.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.801+0000 7f1115600640 1 -- 172.21.15.16:0/2990082532 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 22) ==== 152594+0+0 (secure 0 0 0) 0x7f110c042070 con 0x7f1118075b70 2024-08-31T06:06:49.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.802+0000 7f111d600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ad25be350 0x555ad29647d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:49.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.802+0000 7f111d600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ad25be350 0x555ad29647d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:37616/0 (socket says 172.21.15.16:37616) 2024-08-31T06:06:49.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.802+0000 7f111d600640 1 -- 172.21.15.16:0/2351704538 learned_addr learned my addr 172.21.15.16:0/2351704538 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:49.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.802+0000 7f111cc00640 1 --2- 172.21.15.16:0/2351704538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ad2595490 0x555ad2964d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:49.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.802+0000 7f1117e00640 1 --2- 172.21.15.16:0/2351704538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ad259d670 0x555ad2965250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:49.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.802+0000 7f111d600640 1 -- 172.21.15.16:0/2351704538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ad259d670 msgr2=0x555ad2965250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:49.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.802+0000 7f111d600640 1 --2- 172.21.15.16:0/2351704538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ad259d670 0x555ad2965250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:49.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.802+0000 7f111d600640 1 -- 172.21.15.16:0/2351704538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ad2595490 msgr2=0x555ad2964d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:49.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.802+0000 7f111d600640 1 --2- 172.21.15.16:0/2351704538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ad2595490 0x555ad2964d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:49.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.802+0000 7f111d600640 1 -- 172.21.15.16:0/2351704538 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ad2629930 con 0x555ad25be350 2024-08-31T06:06:49.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.802+0000 7f111cc00640 1 --2- 172.21.15.16:0/2351704538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ad2595490 0x555ad2964d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:06:49.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.803+0000 7f111d600640 1 --2- 172.21.15.16:0/2351704538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ad25be350 0x555ad29647d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1110009fc0 tx=0x7f1110010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4a0740331c1c7ec server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:49.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.803+0000 7f1109c00640 1 -- 172.21.15.16:0/2351704538 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1110003b70 con 0x555ad25be350 2024-08-31T06:06:49.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.803+0000 7f1109c00640 1 -- 172.21.15.16:0/2351704538 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1110003d10 con 0x555ad25be350 2024-08-31T06:06:49.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.803+0000 7f1109c00640 1 -- 172.21.15.16:0/2351704538 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1110011a40 con 0x555ad25be350 2024-08-31T06:06:49.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.804+0000 7f1115600640 1 --2- 172.21.15.16:0/2990082532 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f11000b4c00 0x7f11000b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:49.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.804+0000 7f1115600640 1 -- 172.21.15.16:0/2990082532 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f110c105610 con 0x7f1118075b70 2024-08-31T06:06:49.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.804+0000 7f11261cd740 1 -- 172.21.15.16:0/2351704538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ad25be350 msgr2=0x555ad29647d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:49.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.804+0000 7f11261cd740 1 --2- 172.21.15.16:0/2351704538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ad25be350 0x555ad29647d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1110009fc0 tx=0x7f1110010860 comp rx=0 tx=0).stop 2024-08-31T06:06:49.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.804+0000 7f111cc00640 1 --2- 172.21.15.16:0/2990082532 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f11000b4c00 0x7f11000b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:49.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.805+0000 7f11261cd740 1 -- 172.21.15.16:0/2351704538 shutdown_connections 2024-08-31T06:06:49.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.805+0000 7f11261cd740 1 --2- 172.21.15.16:0/2351704538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ad259d670 0x555ad2965250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:49.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.805+0000 7f11261cd740 1 --2- 172.21.15.16:0/2351704538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ad2595490 0x555ad2964d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:49.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.805+0000 7f11261cd740 1 --2- 172.21.15.16:0/2351704538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ad25be350 0x555ad29647d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:49.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.805+0000 7f11261cd740 1 -- 172.21.15.16:0/2351704538 >> 172.21.15.16:0/2351704538 conn(0x555ad25b2740 msgr2=0x555ad29626c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:49.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.805+0000 7f11261cd740 1 -- 172.21.15.16:0/2351704538 shutdown_connections 2024-08-31T06:06:49.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.805+0000 7f11261cd740 1 -- 172.21.15.16:0/2351704538 wait complete. 2024-08-31T06:06:49.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.805+0000 7f111cc00640 1 --2- 172.21.15.16:0/2990082532 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f11000b4c00 0x7f11000b7110 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f110c0c9190 tx=0x7f110c000bc0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:49.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.806+0000 7f11261cd740 1 -- start start 2024-08-31T06:06:49.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.806+0000 7f11261cd740 1 Processor -- start 2024-08-31T06:06:49.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.807+0000 7f11261cd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ad2595490 0x555ad29703e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:49.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.807+0000 7f11261cd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ad259d670 0x555ad2970960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:49.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.807+0000 7f11261cd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ad25be350 0x555ad297c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:49.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.807+0000 7f11261cd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555ad272c890 con 0x555ad259d670 2024-08-31T06:06:49.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.807+0000 7f11261cd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555ad2850cd0 con 0x555ad25be350 2024-08-31T06:06:49.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.807+0000 7f11261cd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x555ad2972fb0 con 0x555ad2595490 2024-08-31T06:06:49.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.807+0000 7f11261cd740 10 client.0 _ll_register_callbacks cb 0x555ad28c4ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:06:49.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.807+0000 7f111d600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ad2595490 0x555ad29703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:49.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.807+0000 7f111d600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ad2595490 0x555ad29703e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:37628/0 (socket says 172.21.15.16:37628) 2024-08-31T06:06:49.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.807+0000 7f111d600640 1 -- 172.21.15.16:0/3407967527 learned_addr learned my addr 172.21.15.16:0/3407967527 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:49.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.807+0000 7f1117e00640 1 --2- 172.21.15.16:0/3407967527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ad259d670 0x555ad2970960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:49.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.807+0000 7f111d600640 1 --2- 172.21.15.16:0/3407967527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ad25be350 0x555ad297c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:49.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.808+0000 7f111d600640 1 -- 172.21.15.16:0/3407967527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ad25be350 msgr2=0x555ad297c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:49.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.808+0000 7f111d600640 1 --2- 172.21.15.16:0/3407967527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ad25be350 0x555ad297c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:49.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.808+0000 7f111d600640 1 -- 172.21.15.16:0/3407967527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ad259d670 msgr2=0x555ad2970960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:49.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.808+0000 7f111d600640 1 --2- 172.21.15.16:0/3407967527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ad259d670 0x555ad2970960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:49.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.808+0000 7f111d600640 1 -- 172.21.15.16:0/3407967527 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ad289a1b0 con 0x555ad2595490 2024-08-31T06:06:49.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.808+0000 7f111d600640 1 --2- 172.21.15.16:0/3407967527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ad2595490 0x555ad29703e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f11100116a0 tx=0x7f11100115a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a53f8654c61cc89e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:49.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.808+0000 7f10fe000640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:06:49.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.809+0000 7f10fe000640 1 -- 172.21.15.16:0/3407967527 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11100119f0 con 0x555ad2595490 2024-08-31T06:06:49.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.809+0000 7f11261cd740 1 -- 172.21.15.16:0/3407967527 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1110003d10 con 0x555ad2595490 2024-08-31T06:06:49.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.809+0000 7f10fe000640 1 -- 172.21.15.16:0/3407967527 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f1110011b90 con 0x555ad2595490 2024-08-31T06:06:49.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.809+0000 7f10fe000640 1 -- 172.21.15.16:0/3407967527 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1110032050 con 0x555ad2595490 2024-08-31T06:06:49.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.809+0000 7f11261cd740 10 client.25342 fetch_fsmap learned FSMap version 25 2024-08-31T06:06:49.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.809+0000 7f11261cd740 1 -- 172.21.15.16:0/3407967527 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555ad297f010 con 0x555ad2595490 2024-08-31T06:06:49.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.810+0000 7f10fe000640 1 -- 172.21.15.16:0/3407967527 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1110003d10 con 0x555ad2595490 2024-08-31T06:06:49.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.810+0000 7f10fe000640 1 -- 172.21.15.16:0/3407967527 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f111002a020 con 0x555ad2595490 2024-08-31T06:06:49.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.810+0000 7f11261cd740 10 client.25342 fetch_fsmap finished waiting for FSMap version 25 2024-08-31T06:06:49.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.810+0000 7f11261cd740 10 client.25342 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:06:49.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.810+0000 7f11261cd740 20 client.25342 populate_metadata read hostname 'smithi016' 2024-08-31T06:06:49.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.810+0000 7f11261cd740 1 --2- 172.21.15.16:0/3407967527 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x555ad257ccd0 0x555ad2982430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:49.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.810+0000 7f11261cd740 4 client.25342 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:06:49.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.810+0000 7f11261cd740 1 -- 172.21.15.16:0/3407967527 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ad28dc6a0 con 0x555ad257ccd0 2024-08-31T06:06:49.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.811+0000 7f1117e00640 1 --2- 172.21.15.16:0/3407967527 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x555ad257ccd0 0x555ad2982430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:49.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.811+0000 7f1117e00640 1 --2- 172.21.15.16:0/3407967527 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x555ad257ccd0 0x555ad2982430 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abe1150f5cd9c27a server_cookie=f18495c228f959f3 in_seq=0 out_seq=0 2024-08-31T06:06:49.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.811+0000 7f10fe000640 10 client.25342 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:06:49.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.812+0000 7f10fe000640 1 -- 172.21.15.16:0/3407967527 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555ad28dc6a0 con 0x555ad257ccd0 2024-08-31T06:06:49.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.812+0000 7f10fe000640 10 client.25342 handle_command_reply: tid=0 2024-08-31T06:06:49.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.813+0000 7f11261cd740 1 client.25342 shutdown 2024-08-31T06:06:49.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.813+0000 7f11261cd740 1 -- 172.21.15.16:0/3407967527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ad2595490 msgr2=0x555ad29703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:49.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.813+0000 7f11261cd740 1 --2- 172.21.15.16:0/3407967527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ad2595490 0x555ad29703e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f11100116a0 tx=0x7f11100115a0 comp rx=0 tx=0).stop 2024-08-31T06:06:49.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.813+0000 7f11261cd740 1 -- 172.21.15.16:0/3407967527 shutdown_connections 2024-08-31T06:06:49.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.813+0000 7f11261cd740 1 --2- 172.21.15.16:0/3407967527 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x555ad257ccd0 0x555ad2982430 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:49.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.813+0000 7f11261cd740 1 --2- 172.21.15.16:0/3407967527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ad25be350 0x555ad297c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:49.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.813+0000 7f11261cd740 1 --2- 172.21.15.16:0/3407967527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ad259d670 0x555ad2970960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:49.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.813+0000 7f11261cd740 1 --2- 172.21.15.16:0/3407967527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ad2595490 0x555ad29703e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:49.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.813+0000 7f11261cd740 1 -- 172.21.15.16:0/3407967527 >> 172.21.15.16:0/3407967527 conn(0x555ad25b2740 msgr2=0x555ad2967960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:49.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.813+0000 7f10fe000640 0 client.25342 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:06:49.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.813+0000 7f11261cd740 1 -- 172.21.15.16:0/3407967527 shutdown_connections 2024-08-31T06:06:49.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.813+0000 7f11261cd740 1 -- 172.21.15.16:0/3407967527 wait complete. 2024-08-31T06:06:49.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.814+0000 7f11261cd740 20 client.25342 trim_cache size 0 max 16384 2024-08-31T06:06:49.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.853+0000 7f11261cd740 1 Processor -- start 2024-08-31T06:06:49.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.853+0000 7f11261cd740 1 -- start start 2024-08-31T06:06:49.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.854+0000 7f11261cd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ad2595490 0x555ad2971e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:49.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.854+0000 7f11261cd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ad259d670 0x555ad297c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:49.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.854+0000 7f11261cd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ad25be350 0x555ad297c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:49.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.854+0000 7f11261cd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f11181c0920 con 0x555ad2595490 2024-08-31T06:06:49.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.854+0000 7f11261cd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f11181be1f0 con 0x555ad25be350 2024-08-31T06:06:49.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.854+0000 7f11261cd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f11181bc040 con 0x555ad259d670 2024-08-31T06:06:49.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.854+0000 7f111d600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ad2595490 0x555ad2971e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:49.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.854+0000 7f111d600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ad2595490 0x555ad2971e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35636/0 (socket says 172.21.15.16:35636) 2024-08-31T06:06:49.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.854+0000 7f111d600640 1 -- 172.21.15.16:0/2007823704 learned_addr learned my addr 172.21.15.16:0/2007823704 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:49.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.854+0000 7f1117e00640 1 --2- 172.21.15.16:0/2007823704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ad259d670 0x555ad297c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:49.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.855+0000 7f111d600640 1 --2- 172.21.15.16:0/2007823704 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ad25be350 0x555ad297c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:49.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.855+0000 7f1117e00640 1 -- 172.21.15.16:0/2007823704 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ad25be350 msgr2=0x555ad297c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:49.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.855+0000 7f1117e00640 1 --2- 172.21.15.16:0/2007823704 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ad25be350 0x555ad297c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:49.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.855+0000 7f1117e00640 1 -- 172.21.15.16:0/2007823704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ad2595490 msgr2=0x555ad2971e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:49.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.855+0000 7f1117e00640 1 --2- 172.21.15.16:0/2007823704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ad2595490 0x555ad2971e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:49.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.855+0000 7f1117e00640 1 -- 172.21.15.16:0/2007823704 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ad2966ac0 con 0x555ad259d670 2024-08-31T06:06:49.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.855+0000 7f1117e00640 1 --2- 172.21.15.16:0/2007823704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ad259d670 0x555ad297c1a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x555ad2973100 tx=0x7f1104005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f30e11ce7533a9e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:49.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.856+0000 7f1109200640 1 -- 172.21.15.16:0/2007823704 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1104012390 con 0x555ad259d670 2024-08-31T06:06:49.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.856+0000 7f1109200640 1 -- 172.21.15.16:0/2007823704 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11040054d0 con 0x555ad259d670 2024-08-31T06:06:49.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.856+0000 7f1109200640 1 -- 172.21.15.16:0/2007823704 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1104004a00 con 0x555ad259d670 2024-08-31T06:06:49.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.857+0000 7f11261cd740 1 -- 172.21.15.16:0/2007823704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ad259d670 msgr2=0x555ad297c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:49.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.857+0000 7f11261cd740 1 --2- 172.21.15.16:0/2007823704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ad259d670 0x555ad297c1a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x555ad2973100 tx=0x7f1104005f90 comp rx=0 tx=0).stop 2024-08-31T06:06:49.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.857+0000 7f11261cd740 1 -- 172.21.15.16:0/2007823704 shutdown_connections 2024-08-31T06:06:49.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.857+0000 7f11261cd740 1 --2- 172.21.15.16:0/2007823704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ad259d670 0x555ad297c1a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:49.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.857+0000 7f11261cd740 1 --2- 172.21.15.16:0/2007823704 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ad25be350 0x555ad297c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:49.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.857+0000 7f11261cd740 1 --2- 172.21.15.16:0/2007823704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ad2595490 0x555ad2971e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:49.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.857+0000 7f11261cd740 1 -- 172.21.15.16:0/2007823704 >> 172.21.15.16:0/2007823704 conn(0x555ad25b2740 msgr2=0x555ad2971700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:49.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.857+0000 7f11261cd740 1 -- 172.21.15.16:0/2007823704 shutdown_connections 2024-08-31T06:06:49.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.858+0000 7f11261cd740 1 -- 172.21.15.16:0/2007823704 wait complete. 2024-08-31T06:06:49.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.858+0000 7f11261cd740 1 -- start start 2024-08-31T06:06:49.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.858+0000 7f11261cd740 1 Processor -- start 2024-08-31T06:06:49.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.859+0000 7f11261cd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ad2595490 0x555ad2978400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:49.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.859+0000 7f11261cd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ad259d670 0x555ad2978940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:49.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.859+0000 7f11261cd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ad25be350 0x555ad296c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:49.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.859+0000 7f11261cd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f11181bc6b0 con 0x555ad2595490 2024-08-31T06:06:49.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.859+0000 7f11261cd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f11181bdb20 con 0x555ad25be350 2024-08-31T06:06:49.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.859+0000 7f11261cd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f11181bcd80 con 0x555ad259d670 2024-08-31T06:06:49.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.859+0000 7f11261cd740 10 client.0 _ll_register_callbacks cb 0x7f1110009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:06:49.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.860+0000 7f111d600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ad25be350 0x555ad296c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:49.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.860+0000 7f111d600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ad25be350 0x555ad296c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:33750/0 (socket says 172.21.15.16:33750) 2024-08-31T06:06:49.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.860+0000 7f111d600640 1 -- 172.21.15.16:0/3135511854 learned_addr learned my addr 172.21.15.16:0/3135511854 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:49.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.860+0000 7f111d600640 1 -- 172.21.15.16:0/3135511854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ad259d670 msgr2=0x555ad2978940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:49.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.860+0000 7f111d600640 1 --2- 172.21.15.16:0/3135511854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ad259d670 0x555ad2978940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:49.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.860+0000 7f111d600640 1 -- 172.21.15.16:0/3135511854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ad2595490 msgr2=0x555ad2978400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:49.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.860+0000 7f111d600640 1 --2- 172.21.15.16:0/3135511854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ad2595490 0x555ad2978400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:49.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.860+0000 7f111d600640 1 -- 172.21.15.16:0/3135511854 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ad296c900 con 0x555ad25be350 2024-08-31T06:06:49.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.860+0000 7f111d600640 1 --2- 172.21.15.16:0/3135511854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ad25be350 0x555ad296c3c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1110000f80 tx=0x7f11100096e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a8c8a89978506c7 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:49.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.860+0000 7f10fe000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:06:49.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.861+0000 7f10fe000640 1 -- 172.21.15.16:0/3135511854 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111003d060 con 0x555ad25be350 2024-08-31T06:06:49.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.861+0000 7f10fe000640 1 -- 172.21.15.16:0/3135511854 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f1110027040 con 0x555ad25be350 2024-08-31T06:06:49.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.861+0000 7f11261cd740 1 -- 172.21.15.16:0/3135511854 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11040054d0 con 0x555ad25be350 2024-08-31T06:06:49.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.862+0000 7f10fe000640 1 -- 172.21.15.16:0/3135511854 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f11100050a0 con 0x555ad25be350 2024-08-31T06:06:49.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.862+0000 7f11261cd740 10 client.25340 fetch_fsmap learned FSMap version 25 2024-08-31T06:06:49.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.863+0000 7f11261cd740 1 -- 172.21.15.16:0/3135511854 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555ad296db90 con 0x555ad25be350 2024-08-31T06:06:49.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.863+0000 7f10fe000640 1 -- 172.21.15.16:0/3135511854 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11040054d0 con 0x555ad25be350 2024-08-31T06:06:49.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.863+0000 7f10fe000640 1 -- 172.21.15.16:0/3135511854 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f111002a030 con 0x555ad25be350 2024-08-31T06:06:49.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.865+0000 7f11261cd740 10 client.25340 fetch_fsmap finished waiting for FSMap version 25 2024-08-31T06:06:49.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.865+0000 7f11261cd740 10 client.25340 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:06:49.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.865+0000 7f11261cd740 20 client.25340 populate_metadata read hostname 'smithi016' 2024-08-31T06:06:49.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.865+0000 7f11261cd740 1 --2- 172.21.15.16:0/3135511854 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x555ad2552080 0x555ad29efdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:49.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.865+0000 7f11261cd740 4 client.25340 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T06:06:49.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.865+0000 7f11261cd740 1 -- 172.21.15.16:0/3135511854 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555ad28d8140 con 0x555ad2552080 2024-08-31T06:06:49.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.865+0000 7f1117e00640 1 --2- 172.21.15.16:0/3135511854 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x555ad2552080 0x555ad29efdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:49.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.867+0000 7f1117e00640 1 --2- 172.21.15.16:0/3135511854 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x555ad2552080 0x555ad29efdf0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff3f993553cc81ec server_cookie=96416a5f685a03c6 in_seq=0 out_seq=0 2024-08-31T06:06:49.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.867+0000 7f10fe000640 10 client.25340 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:06:49.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.869+0000 7f10fe000640 1 -- 172.21.15.16:0/3135511854 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555ad28d8140 con 0x555ad2552080 2024-08-31T06:06:49.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.869+0000 7f10fe000640 10 client.25340 handle_command_reply: tid=0 2024-08-31T06:06:49.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.869+0000 7f11261cd740 1 client.25340 shutdown 2024-08-31T06:06:49.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.869+0000 7f11261cd740 1 -- 172.21.15.16:0/3135511854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ad25be350 msgr2=0x555ad296c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:49.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.869+0000 7f11261cd740 1 --2- 172.21.15.16:0/3135511854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ad25be350 0x555ad296c3c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1110000f80 tx=0x7f11100096e0 comp rx=0 tx=0).stop 2024-08-31T06:06:49.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.871+0000 7f11261cd740 1 -- 172.21.15.16:0/3135511854 shutdown_connections 2024-08-31T06:06:49.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.872+0000 7f11261cd740 1 --2- 172.21.15.16:0/3135511854 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x555ad2552080 0x555ad29efdf0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:49.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.872+0000 7f11261cd740 1 --2- 172.21.15.16:0/3135511854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555ad259d670 0x555ad2978940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:49.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.872+0000 7f11261cd740 1 --2- 172.21.15.16:0/3135511854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ad25be350 0x555ad296c3c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:49.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.872+0000 7f11261cd740 1 --2- 172.21.15.16:0/3135511854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ad2595490 0x555ad2978400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:49.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.872+0000 7f11261cd740 1 -- 172.21.15.16:0/3135511854 >> 172.21.15.16:0/3135511854 conn(0x555ad25b2740 msgr2=0x555ad2971fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:49.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.872+0000 7f10fe000640 0 client.25340 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:06:49.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.872+0000 7f11261cd740 1 -- 172.21.15.16:0/3135511854 shutdown_connections 2024-08-31T06:06:49.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.872+0000 7f11261cd740 1 -- 172.21.15.16:0/3135511854 wait complete. 2024-08-31T06:06:49.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.875+0000 7f11261cd740 20 client.25340 trim_cache size 0 max 16384 2024-08-31T06:06:49.880 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:06:49.880 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-08-31T06:06:49.880 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "c300b7e8-f3fd-482c-a909-9329bc61e570", 2024-08-31T06:06:49.880 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-08-31T06:06:49.881 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:06:49.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.877+0000 7f1115600640 1 -- 172.21.15.16:0/2990082532 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mgrmap(e 23) ==== 152651+0+0 (secure 0 0 0) 0x7f110c0b1d10 con 0x7f1118075b70 2024-08-31T06:06:49.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.881+0000 7f111e000640 1 -- 172.21.15.16:0/2990082532 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f11000b4c00 msgr2=0x7f11000b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:49.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.881+0000 7f111e000640 1 --2- 172.21.15.16:0/2990082532 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f11000b4c00 0x7f11000b7110 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f110c0c9190 tx=0x7f110c000bc0 comp rx=0 tx=0).stop 2024-08-31T06:06:49.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.881+0000 7f111e000640 1 -- 172.21.15.16:0/2990082532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1118075b70 msgr2=0x7f1118074be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:49.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.881+0000 7f111e000640 1 --2- 172.21.15.16:0/2990082532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1118075b70 0x7f1118074be0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f110c005ab0 tx=0x7f110c003c10 comp rx=0 tx=0).stop 2024-08-31T06:06:49.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.882+0000 7f111e000640 1 -- 172.21.15.16:0/2990082532 shutdown_connections 2024-08-31T06:06:49.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.882+0000 7f111e000640 1 --2- 172.21.15.16:0/2990082532 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f11000b4c00 0x7f11000b7110 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:49.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.882+0000 7f111e000640 1 --2- 172.21.15.16:0/2990082532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1118114020 0x7f111806e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:49.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.882+0000 7f111e000640 1 --2- 172.21.15.16:0/2990082532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1118077120 0x7f111806deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:49.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.882+0000 7f111e000640 1 --2- 172.21.15.16:0/2990082532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1118075b70 0x7f1118074be0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:49.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.882+0000 7f111e000640 1 -- 172.21.15.16:0/2990082532 >> 172.21.15.16:0/2990082532 conn(0x7f11180ff1e0 msgr2=0x7f111810ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:49.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.883+0000 7f0bfea00640 1 -- 172.21.15.16:0/3445285291 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mgrmap(e 23) ==== 152651+0+0 (secure 0 0 0) 0x7f0bf40c8540 con 0x7f0c04108fe0 2024-08-31T06:06:49.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.886+0000 7f111e000640 1 -- 172.21.15.16:0/2990082532 shutdown_connections 2024-08-31T06:06:49.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.887+0000 7f111e000640 1 -- 172.21.15.16:0/2990082532 wait complete. 2024-08-31T06:06:49.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:49.942+0000 7f0c0b200640 1 -- 172.21.15.16:0/3445285291 --> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f0c0410b740 con 0x7f0bec0b4c30 2024-08-31T06:06:50.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.027+0000 7f0bfea00640 1 -- 172.21.15.16:0/3445285291 <== mgr.14220 v2:172.21.15.16:6800/4279751337 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f0c0410b740 con 0x7f0bec0b4c30 2024-08-31T06:06:50.029 INFO:teuthology.orchestra.run.smithi016.stdout:Retention added to path / 2024-08-31T06:06:50.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.029+0000 7f0c0b200640 1 -- 172.21.15.16:0/3445285291 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f0bec0b4c30 msgr2=0x7f0bec0b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:50.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.029+0000 7f0c0b200640 1 --2- 172.21.15.16:0/3445285291 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f0bec0b4c30 0x7f0bec0b7140 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0c04076f00 tx=0x7f0bf8005f90 comp rx=0 tx=0).stop 2024-08-31T06:06:50.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.029+0000 7f0c0b200640 1 -- 172.21.15.16:0/3445285291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c04108fe0 msgr2=0x7f0c040759f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:50.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.029+0000 7f0c0b200640 1 --2- 172.21.15.16:0/3445285291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c04108fe0 0x7f0c040759f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0bf402a930 tx=0x7f0bf4031a20 comp rx=0 tx=0).stop 2024-08-31T06:06:50.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.030+0000 7f0c0b200640 1 -- 172.21.15.16:0/3445285291 shutdown_connections 2024-08-31T06:06:50.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.030+0000 7f0c0b200640 1 --2- 172.21.15.16:0/3445285291 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f0bec0b4c30 0x7f0bec0b7140 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:50.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.030+0000 7f0c0b200640 1 --2- 172.21.15.16:0/3445285291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c04109930 0x7f0c04075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:50.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.030+0000 7f0c0b200640 1 --2- 172.21.15.16:0/3445285291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c04108fe0 0x7f0c040759f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:50.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.030+0000 7f0c0b200640 1 --2- 172.21.15.16:0/3445285291 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c041036b0 0x7f0c040754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:50.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.030+0000 7f0c0b200640 1 -- 172.21.15.16:0/3445285291 >> 172.21.15.16:0/3445285291 conn(0x7f0c040ff0d0 msgr2=0x7f0c041014f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:50.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.030+0000 7f0c0b200640 1 -- 172.21.15.16:0/3445285291 shutdown_connections 2024-08-31T06:06:50.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.030+0000 7f0c0b200640 1 -- 172.21.15.16:0/3445285291 wait complete. 2024-08-31T06:06:50.060 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-08-31T06:06:50.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.281+0000 7f89f2e00640 1 -- 172.21.15.16:0/970819755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89ec1036b0 msgr2=0x7f89ec103ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:50.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.281+0000 7f89f2e00640 1 --2- 172.21.15.16:0/970819755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89ec1036b0 0x7f89ec103ac0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f89e0009860 tx=0x7f89e002a340 comp rx=0 tx=0).stop 2024-08-31T06:06:50.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.282+0000 7f89f2e00640 1 -- 172.21.15.16:0/970819755 shutdown_connections 2024-08-31T06:06:50.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.282+0000 7f89f2e00640 1 --2- 172.21.15.16:0/970819755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89ec109930 0x7f89ec110160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:50.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.282+0000 7f89f2e00640 1 --2- 172.21.15.16:0/970819755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89ec108fe0 0x7f89ec1093f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:50.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.282+0000 7f89f2e00640 1 --2- 172.21.15.16:0/970819755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89ec1036b0 0x7f89ec103ac0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:50.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.282+0000 7f89f2e00640 1 -- 172.21.15.16:0/970819755 >> 172.21.15.16:0/970819755 conn(0x7f89ec0ff0d0 msgr2=0x7f89ec101520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:50.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.282+0000 7f89f2e00640 1 -- 172.21.15.16:0/970819755 shutdown_connections 2024-08-31T06:06:50.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.282+0000 7f89f2e00640 1 -- 172.21.15.16:0/970819755 wait complete. 2024-08-31T06:06:50.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.283+0000 7f89f2e00640 1 Processor -- start 2024-08-31T06:06:50.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.283+0000 7f89f2e00640 1 -- start start 2024-08-31T06:06:50.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.283+0000 7f89f2e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89ec1036b0 0x7f89ec0754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:50.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.283+0000 7f89f2e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89ec108fe0 0x7f89ec0759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:50.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.283+0000 7f89f2e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89ec109930 0x7f89ec075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:50.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.283+0000 7f89f2e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f89ec1135e0 con 0x7f89ec108fe0 2024-08-31T06:06:50.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.283+0000 7f89f2e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f89ec113460 con 0x7f89ec1036b0 2024-08-31T06:06:50.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.283+0000 7f89f2e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f89ec113760 con 0x7f89ec109930 2024-08-31T06:06:50.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.284+0000 7f89f1a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89ec1036b0 0x7f89ec0754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:50.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.284+0000 7f89f1000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89ec108fe0 0x7f89ec0759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:50.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.284+0000 7f89f1a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89ec1036b0 0x7f89ec0754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:33772/0 (socket says 172.21.15.16:33772) 2024-08-31T06:06:50.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.284+0000 7f89f2400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89ec109930 0x7f89ec075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:50.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.284+0000 7f89f1a00640 1 -- 172.21.15.16:0/1503035687 learned_addr learned my addr 172.21.15.16:0/1503035687 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:50.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.284+0000 7f89f1a00640 1 -- 172.21.15.16:0/1503035687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89ec109930 msgr2=0x7f89ec075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:50.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.284+0000 7f89f1a00640 1 --2- 172.21.15.16:0/1503035687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89ec109930 0x7f89ec075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:50.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.284+0000 7f89f1a00640 1 -- 172.21.15.16:0/1503035687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89ec108fe0 msgr2=0x7f89ec0759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:50.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.284+0000 7f89f1a00640 1 --2- 172.21.15.16:0/1503035687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89ec108fe0 0x7f89ec0759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:50.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.284+0000 7f89f1a00640 1 -- 172.21.15.16:0/1503035687 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89ec1a8370 con 0x7f89ec1036b0 2024-08-31T06:06:50.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.284+0000 7f89f2400640 1 --2- 172.21.15.16:0/1503035687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89ec109930 0x7f89ec075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:06:50.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.284+0000 7f89f1000640 1 --2- 172.21.15.16:0/1503035687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89ec108fe0 0x7f89ec0759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:06:50.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.284+0000 7f89f1a00640 1 --2- 172.21.15.16:0/1503035687 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89ec1036b0 0x7f89ec0754b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f89e0035040 tx=0x7f89e0032d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:50.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.285+0000 7f89ea000640 1 -- 172.21.15.16:0/1503035687 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e0037040 con 0x7f89ec1036b0 2024-08-31T06:06:50.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.285+0000 7f89f2e00640 1 -- 172.21.15.16:0/1503035687 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89ec1a8600 con 0x7f89ec1036b0 2024-08-31T06:06:50.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.285+0000 7f89ea000640 1 -- 172.21.15.16:0/1503035687 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89e0041070 con 0x7f89ec1036b0 2024-08-31T06:06:50.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.285+0000 7f89ea000640 1 -- 172.21.15.16:0/1503035687 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e0003850 con 0x7f89ec1036b0 2024-08-31T06:06:50.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.285+0000 7f89f2e00640 1 -- 172.21.15.16:0/1503035687 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89ec1a8ae0 con 0x7f89ec1036b0 2024-08-31T06:06:50.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.286+0000 7f89f2e00640 1 -- 172.21.15.16:0/1503035687 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89ec103ac0 con 0x7f89ec1036b0 2024-08-31T06:06:50.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.288+0000 7f89ea000640 1 -- 172.21.15.16:0/1503035687 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 23) ==== 152651+0+0 (secure 0 0 0) 0x7f89e00039f0 con 0x7f89ec1036b0 2024-08-31T06:06:50.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.290+0000 7f89ea000640 1 --2- 172.21.15.16:0/1503035687 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f89d40b4c40 0x7f89d40b7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:50.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.290+0000 7f89ea000640 1 -- 172.21.15.16:0/1503035687 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f89e01057d0 con 0x7f89ec1036b0 2024-08-31T06:06:50.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.290+0000 7f89f1000640 1 --2- 172.21.15.16:0/1503035687 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f89d40b4c40 0x7f89d40b7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:50.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.291+0000 7f89f1000640 1 --2- 172.21.15.16:0/1503035687 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f89d40b4c40 0x7f89d40b7150 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f89ec108a60 tx=0x7f89d8005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:50.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.291+0000 7f89ea000640 1 -- 172.21.15.16:0/1503035687 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f89e004b020 con 0x7f89ec1036b0 2024-08-31T06:06:50.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.535+0000 7f89f2e00640 1 -- 172.21.15.16:0/1503035687 --> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f89ec0635a0 con 0x7f89d40b4c40 2024-08-31T06:06:50.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.543+0000 7f89ea000640 1 -- 172.21.15.16:0/1503035687 <== mgr.14220 v2:172.21.15.16:6800/4279751337 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f89ec0635a0 con 0x7f89d40b4c40 2024-08-31T06:06:50.546 INFO:teuthology.orchestra.run.smithi016.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-08-31T00:00:00", "created": "2024-08-31T06:06:49", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-08-31T06:06:50.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.545+0000 7f89f2e00640 1 -- 172.21.15.16:0/1503035687 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f89d40b4c40 msgr2=0x7f89d40b7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:50.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.545+0000 7f89f2e00640 1 --2- 172.21.15.16:0/1503035687 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f89d40b4c40 0x7f89d40b7150 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f89ec108a60 tx=0x7f89d8005f90 comp rx=0 tx=0).stop 2024-08-31T06:06:50.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.546+0000 7f89f2e00640 1 -- 172.21.15.16:0/1503035687 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89ec1036b0 msgr2=0x7f89ec0754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:50.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.546+0000 7f89f2e00640 1 --2- 172.21.15.16:0/1503035687 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89ec1036b0 0x7f89ec0754b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f89e0035040 tx=0x7f89e0032d50 comp rx=0 tx=0).stop 2024-08-31T06:06:50.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.546+0000 7f89f2e00640 1 -- 172.21.15.16:0/1503035687 shutdown_connections 2024-08-31T06:06:50.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.546+0000 7f89f2e00640 1 --2- 172.21.15.16:0/1503035687 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f89d40b4c40 0x7f89d40b7150 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:50.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.546+0000 7f89f2e00640 1 --2- 172.21.15.16:0/1503035687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89ec109930 0x7f89ec075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:50.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.546+0000 7f89f2e00640 1 --2- 172.21.15.16:0/1503035687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89ec108fe0 0x7f89ec0759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:50.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.546+0000 7f89f2e00640 1 --2- 172.21.15.16:0/1503035687 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89ec1036b0 0x7f89ec0754b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:50.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.546+0000 7f89f2e00640 1 -- 172.21.15.16:0/1503035687 >> 172.21.15.16:0/1503035687 conn(0x7f89ec0ff0d0 msgr2=0x7f89ec1014f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:50.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.546+0000 7f89f2e00640 1 -- 172.21.15.16:0/1503035687 shutdown_connections 2024-08-31T06:06:50.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.546+0000 7f89f2e00640 1 -- 172.21.15.16:0/1503035687 wait complete. 2024-08-31T06:06:50.573 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-08-31T06:06:50.726 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T06:06:50.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.794+0000 7f4828a00640 1 -- 172.21.15.16:0/3132348260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4824075ea0 msgr2=0x7f482410f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:50.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.794+0000 7f4828a00640 1 --2- 172.21.15.16:0/3132348260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4824075ea0 0x7f482410f6e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4810009900 tx=0x7f481002a400 comp rx=0 tx=0).stop 2024-08-31T06:06:50.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.794+0000 7f4828a00640 1 -- 172.21.15.16:0/3132348260 shutdown_connections 2024-08-31T06:06:50.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.794+0000 7f4828a00640 1 --2- 172.21.15.16:0/3132348260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f482410fc20 0x7f4824112080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:50.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.794+0000 7f4828a00640 1 --2- 172.21.15.16:0/3132348260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4824075ea0 0x7f482410f6e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:50.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.794+0000 7f4828a00640 1 --2- 172.21.15.16:0/3132348260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4824075550 0x7f4824075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:50.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.794+0000 7f4828a00640 1 -- 172.21.15.16:0/3132348260 >> 172.21.15.16:0/3132348260 conn(0x7f48240ff130 msgr2=0x7f4824101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:50.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.795+0000 7f4828a00640 1 -- 172.21.15.16:0/3132348260 shutdown_connections 2024-08-31T06:06:50.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.795+0000 7f4828a00640 1 -- 172.21.15.16:0/3132348260 wait complete. 2024-08-31T06:06:50.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.796+0000 7f4828a00640 1 Processor -- start 2024-08-31T06:06:50.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.796+0000 7f4828a00640 1 -- start start 2024-08-31T06:06:50.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.796+0000 7f4828a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4824075550 0x7f482419c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:50.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.796+0000 7f4828a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4824075ea0 0x7f482419c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:50.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.796+0000 7f4828a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f482410fc20 0x7f48241a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:50.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.796+0000 7f4828a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f48241157d0 con 0x7f482410fc20 2024-08-31T06:06:50.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.796+0000 7f4828a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4824115650 con 0x7f4824075550 2024-08-31T06:06:50.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.796+0000 7f4828a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4824115950 con 0x7f4824075ea0 2024-08-31T06:06:50.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.797+0000 7f4823e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f482410fc20 0x7f48241a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:50.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.797+0000 7f4822a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4824075ea0 0x7f482419c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:50.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.797+0000 7f4823400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4824075550 0x7f482419c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:50.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.797+0000 7f4822a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4824075ea0 0x7f482419c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:37672/0 (socket says 172.21.15.16:37672) 2024-08-31T06:06:50.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.797+0000 7f4822a00640 1 -- 172.21.15.16:0/1483309439 learned_addr learned my addr 172.21.15.16:0/1483309439 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:50.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.797+0000 7f4822a00640 1 -- 172.21.15.16:0/1483309439 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4824075550 msgr2=0x7f482419c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:50.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.797+0000 7f4822a00640 1 --2- 172.21.15.16:0/1483309439 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4824075550 0x7f482419c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:50.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.797+0000 7f4822a00640 1 -- 172.21.15.16:0/1483309439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f482410fc20 msgr2=0x7f48241a1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:50.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.797+0000 7f4822a00640 1 --2- 172.21.15.16:0/1483309439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f482410fc20 0x7f48241a1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:50.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.797+0000 7f4822a00640 1 -- 172.21.15.16:0/1483309439 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48241a21c0 con 0x7f4824075ea0 2024-08-31T06:06:50.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.797+0000 7f4822a00640 1 --2- 172.21.15.16:0/1483309439 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4824075ea0 0x7f482419c830 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f481002a920 tx=0x7f4810031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:50.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.798+0000 7f481be00640 1 -- 172.21.15.16:0/1483309439 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481003f030 con 0x7f4824075ea0 2024-08-31T06:06:50.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.798+0000 7f4828a00640 1 -- 172.21.15.16:0/1483309439 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48241a2450 con 0x7f4824075ea0 2024-08-31T06:06:50.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.798+0000 7f481be00640 1 -- 172.21.15.16:0/1483309439 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4810004490 con 0x7f4824075ea0 2024-08-31T06:06:50.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.798+0000 7f481be00640 1 -- 172.21.15.16:0/1483309439 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4810035070 con 0x7f4824075ea0 2024-08-31T06:06:50.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.798+0000 7f4828a00640 1 -- 172.21.15.16:0/1483309439 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48241a8c40 con 0x7f4824075ea0 2024-08-31T06:06:50.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.799+0000 7f4828a00640 1 -- 172.21.15.16:0/1483309439 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4824075960 con 0x7f4824075ea0 2024-08-31T06:06:50.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.801+0000 7f481be00640 1 -- 172.21.15.16:0/1483309439 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 23) ==== 152651+0+0 (secure 0 0 0) 0x7f4810042030 con 0x7f4824075ea0 2024-08-31T06:06:50.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.803+0000 7f481be00640 1 --2- 172.21.15.16:0/1483309439 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f48080b4c10 0x7f48080b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:50.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.803+0000 7f481be00640 1 -- 172.21.15.16:0/1483309439 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f4810105740 con 0x7f4824075ea0 2024-08-31T06:06:50.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.803+0000 7f4823400640 1 --2- 172.21.15.16:0/1483309439 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f48080b4c10 0x7f48080b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:50.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.804+0000 7f4823400640 1 --2- 172.21.15.16:0/1483309439 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f48080b4c10 0x7f48080b7120 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f48140095f0 tx=0x7f4814005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:50.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:50.807+0000 7f481be00640 1 -- 172.21.15.16:0/1483309439 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f48100cbfa0 con 0x7f4824075ea0 2024-08-31T06:06:51.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:51.063+0000 7f4828a00640 1 -- 172.21.15.16:0/1483309439 --> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f4824109f10 con 0x7f48080b4c10 2024-08-31T06:06:51.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:51.073+0000 7f481be00640 1 -- 172.21.15.16:0/1483309439 <== mgr.14220 v2:172.21.15.16:6800/4279751337 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f4824109f10 con 0x7f48080b4c10 2024-08-31T06:06:51.076 INFO:teuthology.orchestra.run.smithi016.stdout:/ 1m 6m3h 2024-08-31T06:06:51.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:51.076+0000 7f4828a00640 1 -- 172.21.15.16:0/1483309439 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f48080b4c10 msgr2=0x7f48080b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:51.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:51.076+0000 7f4828a00640 1 --2- 172.21.15.16:0/1483309439 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f48080b4c10 0x7f48080b7120 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f48140095f0 tx=0x7f4814005f90 comp rx=0 tx=0).stop 2024-08-31T06:06:51.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:51.076+0000 7f4828a00640 1 -- 172.21.15.16:0/1483309439 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4824075ea0 msgr2=0x7f482419c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:51.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:51.076+0000 7f4828a00640 1 --2- 172.21.15.16:0/1483309439 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4824075ea0 0x7f482419c830 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f481002a920 tx=0x7f4810031a20 comp rx=0 tx=0).stop 2024-08-31T06:06:51.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:51.076+0000 7f4828a00640 1 -- 172.21.15.16:0/1483309439 shutdown_connections 2024-08-31T06:06:51.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:51.076+0000 7f4828a00640 1 --2- 172.21.15.16:0/1483309439 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f48080b4c10 0x7f48080b7120 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:51.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:51.076+0000 7f4828a00640 1 --2- 172.21.15.16:0/1483309439 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4824075ea0 0x7f482419c830 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:51.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:51.076+0000 7f4828a00640 1 --2- 172.21.15.16:0/1483309439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f482410fc20 0x7f48241a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:51.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:51.076+0000 7f4828a00640 1 --2- 172.21.15.16:0/1483309439 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4824075550 0x7f482419c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:51.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:51.076+0000 7f4828a00640 1 -- 172.21.15.16:0/1483309439 >> 172.21.15.16:0/1483309439 conn(0x7f48240ff130 msgr2=0x7f4824109800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:51.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:51.076+0000 7f4828a00640 1 -- 172.21.15.16:0/1483309439 shutdown_connections 2024-08-31T06:06:51.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:51.076+0000 7f4828a00640 1 -- 172.21.15.16:0/1483309439 wait complete. 2024-08-31T06:06:51.102 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-08-31T06:06:51.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:50 smithi155 ceph-mon[21742]: from='client.25310 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:06:51.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:50 smithi155 ceph-mon[21742]: pgmap v351: 97 pgs: 97 active+clean; 645 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2024-08-31T06:06:51.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:50 smithi155 ceph-mon[21742]: scrub queued for path: / 2024-08-31T06:06:51.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:50 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:06:51.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:50 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:06:51.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:50 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:06:51.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:50 smithi155 ceph-mon[21742]: mgrmap e23: x(active, since 9m), standbys: z, y 2024-08-31T06:06:51.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:50 smithi155 ceph-mon[21742]: from='client.25322 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:06:51.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:50 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:06:51.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:50 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:06:51.156 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:06:51.156 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:50 smithi016 ceph-mon[17194]: from='client.25310 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:06:51.156 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:50 smithi016 ceph-mon[17194]: pgmap v351: 97 pgs: 97 active+clean; 645 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2024-08-31T06:06:51.156 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:50 smithi016 ceph-mon[17194]: scrub queued for path: / 2024-08-31T06:06:51.157 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:50 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:06:51.157 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:50 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:06:51.157 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:50 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:06:51.157 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:50 smithi016 ceph-mon[17194]: mgrmap e23: x(active, since 9m), standbys: z, y 2024-08-31T06:06:51.157 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:50 smithi016 ceph-mon[17194]: from='client.25322 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:06:51.157 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:50 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:06:51.157 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:50 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:06:51.171 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-08-31T06:06:51.181 INFO:teuthology.run_tasks:Running task background_exec... 2024-08-31T06:06:51.204 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-08-31T06:06:51.204 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-08-31T06:06:51.208 INFO:teuthology.run_tasks:Running task quiescer... 2024-08-31T06:06:51.221 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-08-31T06:06:51.222 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph pg dump --format=json 2024-08-31T06:06:51.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:50 smithi184 ceph-mon[22331]: from='client.25310 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:06:51.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:50 smithi184 ceph-mon[22331]: pgmap v351: 97 pgs: 97 active+clean; 645 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2024-08-31T06:06:51.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:50 smithi184 ceph-mon[22331]: scrub queued for path: / 2024-08-31T06:06:51.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:50 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:06:51.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:50 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:06:51.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:50 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:06:51.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:50 smithi184 ceph-mon[22331]: mgrmap e23: x(active, since 9m), standbys: z, y 2024-08-31T06:06:51.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:50 smithi184 ceph-mon[22331]: from='client.25322 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:06:51.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:50 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:06:51.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:50 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:06:51.998 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:06:52.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.007+0000 7fd082000640 1 -- 172.21.15.16:0/543185108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd07c1099e0 msgr2=0x7fd07c110210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:52.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.007+0000 7fd082000640 1 --2- 172.21.15.16:0/543185108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd07c1099e0 0x7fd07c110210 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd074009be0 tx=0x7fd07402a6d0 comp rx=0 tx=0).stop 2024-08-31T06:06:52.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.007+0000 7fd082000640 1 -- 172.21.15.16:0/543185108 shutdown_connections 2024-08-31T06:06:52.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.007+0000 7fd082000640 1 --2- 172.21.15.16:0/543185108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd07c109090 0x7fd07c1094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:52.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.007+0000 7fd082000640 1 --2- 172.21.15.16:0/543185108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd07c1099e0 0x7fd07c110210 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:52.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.007+0000 7fd082000640 1 --2- 172.21.15.16:0/543185108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd07c103750 0x7fd07c103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:52.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.007+0000 7fd082000640 1 -- 172.21.15.16:0/543185108 >> 172.21.15.16:0/543185108 conn(0x7fd07c0ff1e0 msgr2=0x7fd07c101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:52.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.007+0000 7fd082000640 1 -- 172.21.15.16:0/543185108 shutdown_connections 2024-08-31T06:06:52.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.007+0000 7fd082000640 1 -- 172.21.15.16:0/543185108 wait complete. 2024-08-31T06:06:52.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.009+0000 7fd082000640 1 Processor -- start 2024-08-31T06:06:52.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.009+0000 7fd082000640 1 -- start start 2024-08-31T06:06:52.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.009+0000 7fd082000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd07c103750 0x7fd07c19c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:52.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.009+0000 7fd082000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd07c109090 0x7fd07c19c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:52.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.009+0000 7fd082000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd07c1099e0 0x7fd07c1a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:52.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.009+0000 7fd082000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd07c113690 con 0x7fd07c1099e0 2024-08-31T06:06:52.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.009+0000 7fd082000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd07c113510 con 0x7fd07c109090 2024-08-31T06:06:52.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.009+0000 7fd082000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd07c113810 con 0x7fd07c103750 2024-08-31T06:06:52.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.010+0000 7fd07be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd07c109090 0x7fd07c19c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:52.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.010+0000 7fd080c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd07c103750 0x7fd07c19c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:52.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.010+0000 7fd081600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd07c1099e0 0x7fd07c1a1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:52.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.010+0000 7fd07be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd07c109090 0x7fd07c19c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:33816/0 (socket says 172.21.15.16:33816) 2024-08-31T06:06:52.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.010+0000 7fd07be00640 1 -- 172.21.15.16:0/3739705389 learned_addr learned my addr 172.21.15.16:0/3739705389 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:52.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.010+0000 7fd080c00640 1 -- 172.21.15.16:0/3739705389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd07c109090 msgr2=0x7fd07c19c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:52.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.010+0000 7fd080c00640 1 --2- 172.21.15.16:0/3739705389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd07c109090 0x7fd07c19c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:52.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.010+0000 7fd080c00640 1 -- 172.21.15.16:0/3739705389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd07c1099e0 msgr2=0x7fd07c1a1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:52.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.010+0000 7fd080c00640 1 --2- 172.21.15.16:0/3739705389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd07c1099e0 0x7fd07c1a1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:52.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.010+0000 7fd080c00640 1 -- 172.21.15.16:0/3739705389 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd07c1a21b0 con 0x7fd07c103750 2024-08-31T06:06:52.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.010+0000 7fd081600640 1 --2- 172.21.15.16:0/3739705389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd07c1099e0 0x7fd07c1a1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:06:52.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.010+0000 7fd07be00640 1 --2- 172.21.15.16:0/3739705389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd07c109090 0x7fd07c19c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:06:52.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.011+0000 7fd080c00640 1 --2- 172.21.15.16:0/3739705389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd07c103750 0x7fd07c19c240 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd070005fd0 tx=0x7fd07000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:52.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.011+0000 7fd079600640 1 -- 172.21.15.16:0/3739705389 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07000cd00 con 0x7fd07c103750 2024-08-31T06:06:52.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.011+0000 7fd079600640 1 -- 172.21.15.16:0/3739705389 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0700035f0 con 0x7fd07c103750 2024-08-31T06:06:52.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.011+0000 7fd079600640 1 -- 172.21.15.16:0/3739705389 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd070010670 con 0x7fd07c103750 2024-08-31T06:06:52.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.011+0000 7fd082000640 1 -- 172.21.15.16:0/3739705389 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd07c1a24a0 con 0x7fd07c103750 2024-08-31T06:06:52.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.011+0000 7fd082000640 1 -- 172.21.15.16:0/3739705389 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd07c1a8c00 con 0x7fd07c103750 2024-08-31T06:06:52.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.015+0000 7fd079600640 1 -- 172.21.15.16:0/3739705389 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 23) ==== 152651+0+0 (secure 0 0 0) 0x7fd0700037a0 con 0x7fd07c103750 2024-08-31T06:06:52.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.015+0000 7fd08a263740 1 Processor -- start 2024-08-31T06:06:52.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.016+0000 7fd08a263740 1 -- start start 2024-08-31T06:06:52.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.016+0000 7fd08a263740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5c7bbc070 0x55f5c7f61a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:52.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.016+0000 7fd08a263740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5c7b931b0 0x55f5c7f61f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:52.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.016+0000 7fd08a263740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c7b9b390 0x55f5c7f624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:52.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.016+0000 7fd08a263740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f5c7ed8770 con 0x55f5c7b9b390 2024-08-31T06:06:52.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.016+0000 7fd08a263740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f5c7ec7d00 con 0x55f5c7bbc070 2024-08-31T06:06:52.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.016+0000 7fd08a263740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f5c7ecbcb0 con 0x55f5c7b931b0 2024-08-31T06:06:52.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.017+0000 7fd081600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5c7bbc070 0x55f5c7f61a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:52.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.017+0000 7fd081600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5c7bbc070 0x55f5c7f61a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:33832/0 (socket says 172.21.15.16:33832) 2024-08-31T06:06:52.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.017+0000 7fd081600640 1 -- 172.21.15.16:0/17000703 learned_addr learned my addr 172.21.15.16:0/17000703 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:52.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.017+0000 7fd080c00640 1 --2- 172.21.15.16:0/17000703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5c7b931b0 0x55f5c7f61f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:52.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.017+0000 7fd07be00640 1 --2- 172.21.15.16:0/17000703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c7b9b390 0x55f5c7f624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:52.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.017+0000 7fd081600640 1 -- 172.21.15.16:0/17000703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5c7b931b0 msgr2=0x55f5c7f61f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:52.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.017+0000 7fd081600640 1 --2- 172.21.15.16:0/17000703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5c7b931b0 0x55f5c7f61f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:52.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.017+0000 7fd081600640 1 -- 172.21.15.16:0/17000703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c7b9b390 msgr2=0x55f5c7f624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:52.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.017+0000 7fd081600640 1 --2- 172.21.15.16:0/17000703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c7b9b390 0x55f5c7f624b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:52.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.017+0000 7fd081600640 1 -- 172.21.15.16:0/17000703 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5c7c27650 con 0x55f5c7bbc070 2024-08-31T06:06:52.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.017+0000 7fd07be00640 1 --2- 172.21.15.16:0/17000703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c7b9b390 0x55f5c7f624b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:06:52.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.018+0000 7fd081600640 1 --2- 172.21.15.16:0/17000703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5c7bbc070 0x55f5c7f61a30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd074009be0 tx=0x7fd0740307e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1ca78626024ac11 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:52.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.018+0000 7fd079600640 1 --2- 172.21.15.16:0/3739705389 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd0640b4c10 0x7fd0640b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:52.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.018+0000 7fd06dc00640 1 -- 172.21.15.16:0/17000703 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd074041030 con 0x55f5c7bbc070 2024-08-31T06:06:52.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.018+0000 7fd06dc00640 1 -- 172.21.15.16:0/17000703 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd074030970 con 0x55f5c7bbc070 2024-08-31T06:06:52.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.018+0000 7fd06dc00640 1 -- 172.21.15.16:0/17000703 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd074036040 con 0x55f5c7bbc070 2024-08-31T06:06:52.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.018+0000 7fd079600640 1 -- 172.21.15.16:0/3739705389 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7fd0700e5600 con 0x7fd07c103750 2024-08-31T06:06:52.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.018+0000 7fd080c00640 1 --2- 172.21.15.16:0/3739705389 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd0640b4c10 0x7fd0640b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:52.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.019+0000 7fd08a263740 1 -- 172.21.15.16:0/17000703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5c7bbc070 msgr2=0x55f5c7f61a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:52.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.019+0000 7fd080c00640 1 --2- 172.21.15.16:0/3739705389 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd0640b4c10 0x7fd0640b7120 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd0700e58a0 tx=0x7fd0700eb110 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:52.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.019+0000 7fd08a263740 1 --2- 172.21.15.16:0/17000703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5c7bbc070 0x55f5c7f61a30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd074009be0 tx=0x7fd0740307e0 comp rx=0 tx=0).stop 2024-08-31T06:06:52.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.020+0000 7fd08a263740 1 -- 172.21.15.16:0/17000703 shutdown_connections 2024-08-31T06:06:52.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.020+0000 7fd08a263740 1 --2- 172.21.15.16:0/17000703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5c7b931b0 0x55f5c7f61f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:52.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.020+0000 7fd08a263740 1 --2- 172.21.15.16:0/17000703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c7b9b390 0x55f5c7f624b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:52.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.020+0000 7fd08a263740 1 --2- 172.21.15.16:0/17000703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5c7bbc070 0x55f5c7f61a30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:52.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.020+0000 7fd08a263740 1 -- 172.21.15.16:0/17000703 >> 172.21.15.16:0/17000703 conn(0x55f5c7bb0460 msgr2=0x55f5c7f5f920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:52.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.020+0000 7fd08a263740 1 -- 172.21.15.16:0/17000703 shutdown_connections 2024-08-31T06:06:52.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.020+0000 7fd08a263740 1 -- 172.21.15.16:0/17000703 wait complete. 2024-08-31T06:06:52.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.020+0000 7fd08a263740 1 -- start start 2024-08-31T06:06:52.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.021+0000 7fd08a263740 1 Processor -- start 2024-08-31T06:06:52.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.021+0000 7fd08a263740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5c7b931b0 0x55f5c7f6d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:52.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.021+0000 7fd08a263740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c7b9b390 0x55f5c7f6dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:52.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.021+0000 7fd08a263740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5c7bbc070 0x55f5c7f794a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:52.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.021+0000 7fd08a263740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f5c7d2a700 con 0x55f5c7b9b390 2024-08-31T06:06:52.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.021+0000 7fd08a263740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f5c7e4ecb0 con 0x55f5c7b931b0 2024-08-31T06:06:52.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.021+0000 7fd08a263740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f5c7f701b0 con 0x55f5c7bbc070 2024-08-31T06:06:52.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.021+0000 7fd08a263740 10 client.0 _ll_register_callbacks cb 0x55f5c7e74430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:06:52.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.022+0000 7fd081600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5c7b931b0 0x55f5c7f6d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:52.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.022+0000 7fd081600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5c7b931b0 0x55f5c7f6d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:33844/0 (socket says 172.21.15.16:33844) 2024-08-31T06:06:52.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.022+0000 7fd081600640 1 -- 172.21.15.16:0/2670830575 learned_addr learned my addr 172.21.15.16:0/2670830575 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:52.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.022+0000 7fd081600640 1 --2- 172.21.15.16:0/2670830575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5c7bbc070 0x55f5c7f794a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:52.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.022+0000 7fd07be00640 1 --2- 172.21.15.16:0/2670830575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c7b9b390 0x55f5c7f6dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:52.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.022+0000 7fd081600640 1 -- 172.21.15.16:0/2670830575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5c7bbc070 msgr2=0x55f5c7f794a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:52.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.022+0000 7fd081600640 1 --2- 172.21.15.16:0/2670830575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5c7bbc070 0x55f5c7f794a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:52.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.022+0000 7fd081600640 1 -- 172.21.15.16:0/2670830575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c7b9b390 msgr2=0x55f5c7f6dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:52.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.022+0000 7fd081600640 1 --2- 172.21.15.16:0/2670830575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c7b9b390 0x55f5c7f6dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:52.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.022+0000 7fd081600640 1 -- 172.21.15.16:0/2670830575 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5c7e8f870 con 0x55f5c7b931b0 2024-08-31T06:06:52.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.023+0000 7fd081600640 1 --2- 172.21.15.16:0/2670830575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5c7b931b0 0x55f5c7f6d6a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd0740093f0 tx=0x7fd074004150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86b14f812af253fe server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:52.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.023+0000 7fd062000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:06:52.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.023+0000 7fd062000640 1 -- 172.21.15.16:0/2670830575 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd074036050 con 0x55f5c7b931b0 2024-08-31T06:06:52.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.023+0000 7fd08a263740 1 -- 172.21.15.16:0/2670830575 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd074030970 con 0x55f5c7b931b0 2024-08-31T06:06:52.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.023+0000 7fd062000640 1 -- 172.21.15.16:0/2670830575 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7fd074003b30 con 0x55f5c7b931b0 2024-08-31T06:06:52.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.023+0000 7fd062000640 1 -- 172.21.15.16:0/2670830575 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd074059050 con 0x55f5c7b931b0 2024-08-31T06:06:52.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.023+0000 7fd08a263740 10 client.25364 fetch_fsmap learned FSMap version 25 2024-08-31T06:06:52.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.023+0000 7fd08a263740 1 -- 172.21.15.16:0/2670830575 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f5c7f7c1e0 con 0x55f5c7b931b0 2024-08-31T06:06:52.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.024+0000 7fd062000640 1 -- 172.21.15.16:0/2670830575 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd074030970 con 0x55f5c7b931b0 2024-08-31T06:06:52.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.024+0000 7fd062000640 1 -- 172.21.15.16:0/2670830575 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd07403d760 con 0x55f5c7b931b0 2024-08-31T06:06:52.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.024+0000 7fd08a263740 10 client.25364 fetch_fsmap finished waiting for FSMap version 25 2024-08-31T06:06:52.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.024+0000 7fd08a263740 10 client.25364 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:06:52.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.024+0000 7fd08a263740 20 client.25364 populate_metadata read hostname 'smithi016' 2024-08-31T06:06:52.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.024+0000 7fd08a263740 1 --2- 172.21.15.16:0/2670830575 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f5c7b7a9f0 0x55f5c7f7f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:52.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.024+0000 7fd08a263740 4 client.25364 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:06:52.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.024+0000 7fd08a263740 1 -- 172.21.15.16:0/2670830575 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5c7eda6c0 con 0x55f5c7b7a9f0 2024-08-31T06:06:52.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.024+0000 7fd07be00640 1 --2- 172.21.15.16:0/2670830575 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f5c7b7a9f0 0x55f5c7f7f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:52.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.025+0000 7fd07be00640 1 --2- 172.21.15.16:0/2670830575 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f5c7b7a9f0 0x55f5c7f7f630 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26778de54c2bddf3 server_cookie=8944e83b70c3ec0c in_seq=0 out_seq=0 2024-08-31T06:06:52.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.025+0000 7fd062000640 10 client.25364 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:06:52.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.026+0000 7fd062000640 1 -- 172.21.15.16:0/2670830575 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f5c7eda6c0 con 0x55f5c7b7a9f0 2024-08-31T06:06:52.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.026+0000 7fd062000640 10 client.25364 handle_command_reply: tid=0 2024-08-31T06:06:52.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.026+0000 7fd08a263740 1 client.25364 shutdown 2024-08-31T06:06:52.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.026+0000 7fd08a263740 1 -- 172.21.15.16:0/2670830575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5c7b931b0 msgr2=0x55f5c7f6d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:52.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.026+0000 7fd08a263740 1 --2- 172.21.15.16:0/2670830575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5c7b931b0 0x55f5c7f6d6a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd0740093f0 tx=0x7fd074004150 comp rx=0 tx=0).stop 2024-08-31T06:06:52.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.027+0000 7fd08a263740 1 -- 172.21.15.16:0/2670830575 shutdown_connections 2024-08-31T06:06:52.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.027+0000 7fd08a263740 1 --2- 172.21.15.16:0/2670830575 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f5c7b7a9f0 0x55f5c7f7f630 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.027+0000 7fd08a263740 1 --2- 172.21.15.16:0/2670830575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5c7bbc070 0x55f5c7f794a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.027+0000 7fd08a263740 1 --2- 172.21.15.16:0/2670830575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c7b9b390 0x55f5c7f6dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.027+0000 7fd08a263740 1 --2- 172.21.15.16:0/2670830575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5c7b931b0 0x55f5c7f6d6a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.027+0000 7fd08a263740 1 -- 172.21.15.16:0/2670830575 >> 172.21.15.16:0/2670830575 conn(0x55f5c7bb0460 msgr2=0x55f5c7f64c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.027+0000 7fd062000640 0 client.25364 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:06:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.027+0000 7fd08a263740 1 -- 172.21.15.16:0/2670830575 shutdown_connections 2024-08-31T06:06:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.027+0000 7fd08a263740 1 -- 172.21.15.16:0/2670830575 wait complete. 2024-08-31T06:06:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.027+0000 7fd08a263740 20 client.25364 trim_cache size 0 max 16384 2024-08-31T06:06:52.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.047+0000 7fd08a263740 1 Processor -- start 2024-08-31T06:06:52.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.047+0000 7fd08a263740 1 -- start start 2024-08-31T06:06:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.047+0000 7fd08a263740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c7b931b0 0x55f5c7f67250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.047+0000 7fd08a263740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5c7b9b390 0x55f5c7f795b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.047+0000 7fd08a263740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5c7bbc070 0x55f5c7f5f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.047+0000 7fd08a263740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd07c1b3cd0 con 0x55f5c7b931b0 2024-08-31T06:06:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.047+0000 7fd08a263740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd07c1b15a0 con 0x55f5c7b9b390 2024-08-31T06:06:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.047+0000 7fd08a263740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd07c1af3f0 con 0x55f5c7bbc070 2024-08-31T06:06:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.048+0000 7fd07be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5c7b9b390 0x55f5c7f795b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.048+0000 7fd07be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5c7b9b390 0x55f5c7f795b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:33852/0 (socket says 172.21.15.16:33852) 2024-08-31T06:06:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.048+0000 7fd07be00640 1 -- 172.21.15.16:0/4237352369 learned_addr learned my addr 172.21.15.16:0/4237352369 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.048+0000 7fd081600640 1 --2- 172.21.15.16:0/4237352369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c7b931b0 0x55f5c7f67250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.048+0000 7fd081600640 1 --2- 172.21.15.16:0/4237352369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5c7bbc070 0x55f5c7f5f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.048+0000 7fd07be00640 1 -- 172.21.15.16:0/4237352369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5c7bbc070 msgr2=0x55f5c7f5f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.048+0000 7fd07be00640 1 --2- 172.21.15.16:0/4237352369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5c7bbc070 0x55f5c7f5f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.048+0000 7fd07be00640 1 -- 172.21.15.16:0/4237352369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c7b931b0 msgr2=0x55f5c7f67250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.048+0000 7fd07be00640 1 --2- 172.21.15.16:0/4237352369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c7b931b0 0x55f5c7f67250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.048+0000 7fd07be00640 1 -- 172.21.15.16:0/4237352369 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5c7f6c400 con 0x55f5c7b9b390 2024-08-31T06:06:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.049+0000 7fd07be00640 1 --2- 172.21.15.16:0/4237352369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5c7b9b390 0x55f5c7f795b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x55f5c7ec7d00 tx=0x7fd068005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=600cee11e1917053 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.049+0000 7fd06d200640 1 -- 172.21.15.16:0/4237352369 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd068004020 con 0x55f5c7b9b390 2024-08-31T06:06:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.049+0000 7fd06d200640 1 -- 172.21.15.16:0/4237352369 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0680041c0 con 0x55f5c7b9b390 2024-08-31T06:06:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.049+0000 7fd06d200640 1 -- 172.21.15.16:0/4237352369 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06801e720 con 0x55f5c7b9b390 2024-08-31T06:06:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.050+0000 7fd08a263740 1 -- 172.21.15.16:0/4237352369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5c7b9b390 msgr2=0x55f5c7f795b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.050+0000 7fd08a263740 1 --2- 172.21.15.16:0/4237352369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5c7b9b390 0x55f5c7f795b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x55f5c7ec7d00 tx=0x7fd068005dc0 comp rx=0 tx=0).stop 2024-08-31T06:06:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.050+0000 7fd08a263740 1 -- 172.21.15.16:0/4237352369 shutdown_connections 2024-08-31T06:06:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.050+0000 7fd08a263740 1 --2- 172.21.15.16:0/4237352369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5c7bbc070 0x55f5c7f5f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.050+0000 7fd08a263740 1 --2- 172.21.15.16:0/4237352369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5c7b9b390 0x55f5c7f795b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.050+0000 7fd08a263740 1 --2- 172.21.15.16:0/4237352369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c7b931b0 0x55f5c7f67250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.050+0000 7fd08a263740 1 -- 172.21.15.16:0/4237352369 >> 172.21.15.16:0/4237352369 conn(0x55f5c7bb0460 msgr2=0x55f5c7f66b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.050+0000 7fd08a263740 1 -- 172.21.15.16:0/4237352369 shutdown_connections 2024-08-31T06:06:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.050+0000 7fd08a263740 1 -- 172.21.15.16:0/4237352369 wait complete. 2024-08-31T06:06:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.051+0000 7fd08a263740 1 -- start start 2024-08-31T06:06:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.051+0000 7fd08a263740 1 Processor -- start 2024-08-31T06:06:52.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.051+0000 7fd08a263740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5c7b931b0 0x55f5c7f75410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:52.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.051+0000 7fd08a263740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5c7b9b390 0x55f5c7f75950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:52.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.051+0000 7fd08a263740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c7bbc070 0x55f5c7f703c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:52.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.052+0000 7fd08a263740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd07c1afa60 con 0x55f5c7bbc070 2024-08-31T06:06:52.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.052+0000 7fd08a263740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd07c1b0ed0 con 0x55f5c7b931b0 2024-08-31T06:06:52.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.052+0000 7fd08a263740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd07c1b0130 con 0x55f5c7b9b390 2024-08-31T06:06:52.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.052+0000 7fd08a263740 10 client.0 _ll_register_callbacks cb 0x7fd0740046f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:06:52.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.052+0000 7fd081600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5c7b931b0 0x55f5c7f75410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:52.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.052+0000 7fd07be00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5c7b9b390 0x55f5c7f75950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:52.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.052+0000 7fd07be00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5c7b9b390 0x55f5c7f75950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:37724/0 (socket says 172.21.15.16:37724) 2024-08-31T06:06:52.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.052+0000 7fd07be00640 1 -- 172.21.15.16:0/1595355024 learned_addr learned my addr 172.21.15.16:0/1595355024 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:52.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.052+0000 7fd081600640 1 --2- 172.21.15.16:0/1595355024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c7bbc070 0x55f5c7f703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:52.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.052+0000 7fd07be00640 1 -- 172.21.15.16:0/1595355024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5c7b931b0 msgr2=0x55f5c7f75410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:52.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.052+0000 7fd07be00640 1 --2- 172.21.15.16:0/1595355024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5c7b931b0 0x55f5c7f75410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:52.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.052+0000 7fd07be00640 1 -- 172.21.15.16:0/1595355024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c7bbc070 msgr2=0x55f5c7f703c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:06:52.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.052+0000 7fd07be00640 1 --2- 172.21.15.16:0/1595355024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c7bbc070 0x55f5c7f703c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:52.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.052+0000 7fd07be00640 1 -- 172.21.15.16:0/1595355024 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5c7f70900 con 0x55f5c7b9b390 2024-08-31T06:06:52.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.053+0000 7fd07be00640 1 --2- 172.21.15.16:0/1595355024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5c7b9b390 0x55f5c7f75950 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd068000c00 tx=0x7fd0680040a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a65700769a657d91 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:52.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.053+0000 7fd062000640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:06:52.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.053+0000 7fd062000640 1 -- 172.21.15.16:0/1595355024 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd068021040 con 0x55f5c7b9b390 2024-08-31T06:06:52.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.053+0000 7fd062000640 1 -- 172.21.15.16:0/1595355024 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7fd06801e430 con 0x55f5c7b9b390 2024-08-31T06:06:52.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.053+0000 7fd08a263740 1 -- 172.21.15.16:0/1595355024 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0680041c0 con 0x55f5c7b9b390 2024-08-31T06:06:52.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.053+0000 7fd062000640 1 -- 172.21.15.16:0/1595355024 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd06801e630 con 0x55f5c7b9b390 2024-08-31T06:06:52.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.053+0000 7fd08a263740 10 client.25375 fetch_fsmap learned FSMap version 25 2024-08-31T06:06:52.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.053+0000 7fd08a263740 1 -- 172.21.15.16:0/1595355024 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f5c7fea4f0 con 0x55f5c7b9b390 2024-08-31T06:06:52.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.054+0000 7fd062000640 1 -- 172.21.15.16:0/1595355024 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0680041c0 con 0x55f5c7b9b390 2024-08-31T06:06:52.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.054+0000 7fd062000640 1 -- 172.21.15.16:0/1595355024 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd06802d020 con 0x55f5c7b9b390 2024-08-31T06:06:52.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.054+0000 7fd08a263740 10 client.25375 fetch_fsmap finished waiting for FSMap version 25 2024-08-31T06:06:52.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.054+0000 7fd08a263740 10 client.25375 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:06:52.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.054+0000 7fd08a263740 20 client.25375 populate_metadata read hostname 'smithi016' 2024-08-31T06:06:52.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.054+0000 7fd08a263740 1 --2- 172.21.15.16:0/1595355024 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f5c7b4fda0 0x55f5c7fecb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:52.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.054+0000 7fd08a263740 4 client.25375 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T06:06:52.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.054+0000 7fd08a263740 1 -- 172.21.15.16:0/1595355024 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f5c7ed6160 con 0x55f5c7b4fda0 2024-08-31T06:06:52.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.054+0000 7fd081600640 1 --2- 172.21.15.16:0/1595355024 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f5c7b4fda0 0x55f5c7fecb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:52.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.055+0000 7fd081600640 1 --2- 172.21.15.16:0/1595355024 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f5c7b4fda0 0x55f5c7fecb50 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baa69d3b36a1b259 server_cookie=94a717e7798ac371 in_seq=0 out_seq=0 2024-08-31T06:06:52.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.055+0000 7fd062000640 10 client.25375 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:06:52.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.055+0000 7fd062000640 1 -- 172.21.15.16:0/1595355024 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+570 (crc 0 0 0) 0x55f5c7ed6160 con 0x55f5c7b4fda0 2024-08-31T06:06:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.055+0000 7fd062000640 10 client.25375 handle_command_reply: tid=0 2024-08-31T06:06:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.055+0000 7fd08a263740 1 client.25375 shutdown 2024-08-31T06:06:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.055+0000 7fd08a263740 1 -- 172.21.15.16:0/1595355024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5c7b9b390 msgr2=0x55f5c7f75950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.055+0000 7fd08a263740 1 --2- 172.21.15.16:0/1595355024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5c7b9b390 0x55f5c7f75950 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd068000c00 tx=0x7fd0680040a0 comp rx=0 tx=0).stop 2024-08-31T06:06:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.056+0000 7fd08a263740 1 -- 172.21.15.16:0/1595355024 shutdown_connections 2024-08-31T06:06:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.056+0000 7fd08a263740 1 --2- 172.21.15.16:0/1595355024 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f5c7b4fda0 0x55f5c7fecb50 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.056+0000 7fd08a263740 1 --2- 172.21.15.16:0/1595355024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f5c7b9b390 0x55f5c7f75950 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.056+0000 7fd08a263740 1 --2- 172.21.15.16:0/1595355024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c7bbc070 0x55f5c7f703c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.056+0000 7fd08a263740 1 --2- 172.21.15.16:0/1595355024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5c7b931b0 0x55f5c7f75410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.056+0000 7fd08a263740 1 -- 172.21.15.16:0/1595355024 >> 172.21.15.16:0/1595355024 conn(0x55f5c7bb0460 msgr2=0x55f5c7f5f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.056+0000 7fd062000640 0 client.25375 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:06:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.056+0000 7fd08a263740 1 -- 172.21.15.16:0/1595355024 shutdown_connections 2024-08-31T06:06:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.056+0000 7fd08a263740 1 -- 172.21.15.16:0/1595355024 wait complete. 2024-08-31T06:06:52.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.056+0000 7fd08a263740 20 client.25375 trim_cache size 0 max 16384 2024-08-31T06:06:52.062 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:06:52.062 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T06:06:52.062 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-08-31T06:06:52.062 INFO:teuthology.orchestra.run.smithi016.stdout: "c300b7e8-f3fd-482c-a909-9329bc61e570": { 2024-08-31T06:06:52.062 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:06:52.062 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "c300b7e8-f3fd-482c-a909-9329bc61e570", 2024-08-31T06:06:52.062 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-08-31T06:06:52.063 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:06:52.063 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:06:52.063 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-08-31T06:06:52.063 INFO:teuthology.orchestra.run.smithi016.stdout: "c300b7e8-f3fd-482c-a909-9329bc61e570": { 2024-08-31T06:06:52.063 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:06:49.868046+0000", 2024-08-31T06:06:52.063 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:06:52.063 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:06:52.063 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:06:52.063 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:06:52.063 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:06:52.063 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:06:52.064 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:06:52.064 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:06:52.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.057+0000 7fd082000640 1 -- 172.21.15.16:0/3739705389 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd0640b4c10 msgr2=0x7fd0640b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:52.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.057+0000 7fd082000640 1 --2- 172.21.15.16:0/3739705389 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd0640b4c10 0x7fd0640b7120 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd0700e58a0 tx=0x7fd0700eb110 comp rx=0 tx=0).stop 2024-08-31T06:06:52.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.057+0000 7fd082000640 1 -- 172.21.15.16:0/3739705389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd07c103750 msgr2=0x7fd07c19c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:52.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.057+0000 7fd082000640 1 --2- 172.21.15.16:0/3739705389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd07c103750 0x7fd07c19c240 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd070005fd0 tx=0x7fd07000ee50 comp rx=0 tx=0).stop 2024-08-31T06:06:52.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.057+0000 7fd082000640 1 -- 172.21.15.16:0/3739705389 shutdown_connections 2024-08-31T06:06:52.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.057+0000 7fd082000640 1 --2- 172.21.15.16:0/3739705389 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd0640b4c10 0x7fd0640b7120 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:52.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.057+0000 7fd082000640 1 --2- 172.21.15.16:0/3739705389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd07c1099e0 0x7fd07c1a1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:52.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.057+0000 7fd082000640 1 --2- 172.21.15.16:0/3739705389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd07c109090 0x7fd07c19c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:52.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.057+0000 7fd082000640 1 --2- 172.21.15.16:0/3739705389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd07c103750 0x7fd07c19c240 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:52.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.057+0000 7fd082000640 1 -- 172.21.15.16:0/3739705389 >> 172.21.15.16:0/3739705389 conn(0x7fd07c0ff1e0 msgr2=0x7fd07c10e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:52.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.057+0000 7fd082000640 1 -- 172.21.15.16:0/3739705389 shutdown_connections 2024-08-31T06:06:52.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.057+0000 7fd082000640 1 -- 172.21.15.16:0/3739705389 wait complete. 2024-08-31T06:06:52.081 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:51 smithi016 ceph-mon[17194]: from='client.25346 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:06:52.081 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:51 smithi016 ceph-mon[17194]: from='client.25357 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:06:52.082 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:51 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:06:52.082 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:51 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:06:52.082 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:51 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:06:52.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:51 smithi184 ceph-mon[22331]: from='client.25346 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:06:52.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:51 smithi184 ceph-mon[22331]: from='client.25357 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:06:52.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:51 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:06:52.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:51 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:06:52.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:51 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:06:52.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:51 smithi155 ceph-mon[21742]: from='client.25346 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:06:52.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:51 smithi155 ceph-mon[21742]: from='client.25357 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:06:52.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:51 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:06:52.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:51 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:06:52.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:51 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:06:52.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.905+0000 7f7a1f400640 1 -- 172.21.15.16:0/945424218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a18075b70 msgr2=0x7f7a18075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:52.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.905+0000 7f7a1f400640 1 --2- 172.21.15.16:0/945424218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a18075b70 0x7f7a18075f80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7a00009900 tx=0x7f7a0002a3f0 comp rx=0 tx=0).stop 2024-08-31T06:06:52.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.905+0000 7f7a1f400640 1 -- 172.21.15.16:0/945424218 shutdown_connections 2024-08-31T06:06:52.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.905+0000 7f7a1f400640 1 --2- 172.21.15.16:0/945424218 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a181141d0 0x7f7a181166a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:52.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.905+0000 7f7a1f400640 1 --2- 172.21.15.16:0/945424218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a18075b70 0x7f7a18075f80 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:52.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.905+0000 7f7a1f400640 1 --2- 172.21.15.16:0/945424218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a18077120 0x7f7a18075550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:52.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.905+0000 7f7a1f400640 1 -- 172.21.15.16:0/945424218 >> 172.21.15.16:0/945424218 conn(0x7f7a1807b5f0 msgr2=0x7f7a1807ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:52.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.905+0000 7f7a1f400640 1 -- 172.21.15.16:0/945424218 shutdown_connections 2024-08-31T06:06:52.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.905+0000 7f7a1f400640 1 -- 172.21.15.16:0/945424218 wait complete. 2024-08-31T06:06:52.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.906+0000 7f7a1f400640 1 Processor -- start 2024-08-31T06:06:52.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.907+0000 7f7a1f400640 1 -- start start 2024-08-31T06:06:52.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.907+0000 7f7a1f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a18075b70 0x7f7a181a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:52.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.907+0000 7f7a1f400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a18077120 0x7f7a181a0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:52.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.907+0000 7f7a1f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a181141d0 0x7f7a181a6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:52.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.907+0000 7f7a1f400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7a18119e20 con 0x7f7a181141d0 2024-08-31T06:06:52.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.907+0000 7f7a1f400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7a18119ca0 con 0x7f7a18075b70 2024-08-31T06:06:52.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.907+0000 7f7a1f400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7a18119fa0 con 0x7f7a18077120 2024-08-31T06:06:52.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.907+0000 7f7a1d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a181141d0 0x7f7a181a6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:52.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.907+0000 7f7a1d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a181141d0 0x7f7a181a6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35754/0 (socket says 172.21.15.16:35754) 2024-08-31T06:06:52.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.907+0000 7f7a1d400640 1 -- 172.21.15.16:0/733723631 learned_addr learned my addr 172.21.15.16:0/733723631 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:52.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.907+0000 7f7a1ca00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a18075b70 0x7f7a181a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:52.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.907+0000 7f7a0fe00640 1 --2- 172.21.15.16:0/733723631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a18077120 0x7f7a181a0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:52.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.908+0000 7f7a1d400640 1 -- 172.21.15.16:0/733723631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a18077120 msgr2=0x7f7a181a0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:52.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.908+0000 7f7a1d400640 1 --2- 172.21.15.16:0/733723631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a18077120 0x7f7a181a0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:52.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.908+0000 7f7a1d400640 1 -- 172.21.15.16:0/733723631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a18075b70 msgr2=0x7f7a181a0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:52.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.908+0000 7f7a1d400640 1 --2- 172.21.15.16:0/733723631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a18075b70 0x7f7a181a0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:52.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.908+0000 7f7a1d400640 1 -- 172.21.15.16:0/733723631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a181a6780 con 0x7f7a181141d0 2024-08-31T06:06:52.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.908+0000 7f7a1d400640 1 --2- 172.21.15.16:0/733723631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a181141d0 0x7f7a181a6050 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7a10009fd0 tx=0x7f7a1000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:52.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.909+0000 7f7a16000640 1 -- 172.21.15.16:0/733723631 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a10017070 con 0x7f7a181141d0 2024-08-31T06:06:52.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.909+0000 7f7a16000640 1 -- 172.21.15.16:0/733723631 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a10003e20 con 0x7f7a181141d0 2024-08-31T06:06:52.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.909+0000 7f7a16000640 1 -- 172.21.15.16:0/733723631 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a10010460 con 0x7f7a181141d0 2024-08-31T06:06:52.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.909+0000 7f7a1f400640 1 -- 172.21.15.16:0/733723631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a181a6a70 con 0x7f7a181141d0 2024-08-31T06:06:52.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.909+0000 7f7a1f400640 1 -- 172.21.15.16:0/733723631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a181ad1d0 con 0x7f7a181141d0 2024-08-31T06:06:52.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.910+0000 7f7a1f400640 1 -- 172.21.15.16:0/733723631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a18075f80 con 0x7f7a181141d0 2024-08-31T06:06:52.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.912+0000 7f7a16000640 1 -- 172.21.15.16:0/733723631 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 152651+0+0 (secure 0 0 0) 0x7f7a10003970 con 0x7f7a181141d0 2024-08-31T06:06:52.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.914+0000 7f7a16000640 1 --2- 172.21.15.16:0/733723631 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f79fc0b4c90 0x7f79fc0b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:52.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.914+0000 7f7a16000640 1 -- 172.21.15.16:0/733723631 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f7a100e50c0 con 0x7f7a181141d0 2024-08-31T06:06:52.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.914+0000 7f7a1ca00640 1 --2- 172.21.15.16:0/733723631 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f79fc0b4c90 0x7f79fc0b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:52.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.915+0000 7f7a1ca00640 1 --2- 172.21.15.16:0/733723631 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f79fc0b4c90 0x7f79fc0b71a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7a0800bbe0 tx=0x7f7a0800b390 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:52.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:52.918+0000 7f7a16000640 1 -- 172.21.15.16:0/733723631 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f7a100ab920 con 0x7f7a181141d0 2024-08-31T06:06:52.977 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:52 smithi016 ceph-mon[17194]: pgmap v352: 97 pgs: 97 active+clean; 645 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 7 op/s 2024-08-31T06:06:53.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:53.114+0000 7f7a1f400640 1 -- 172.21.15.16:0/733723631 --> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7a18113550 con 0x7f79fc0b4c90 2024-08-31T06:06:53.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:53.119+0000 7f7a16000640 1 -- 172.21.15.16:0/733723631 <== mgr.14220 v2:172.21.15.16:6800/4279751337 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259090 (secure 0 0 0) 0x7f7a18113550 con 0x7f79fc0b4c90 2024-08-31T06:06:53.123 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:06:53.125 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-08-31T06:06:53.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:53.123+0000 7f7a1f400640 1 -- 172.21.15.16:0/733723631 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f79fc0b4c90 msgr2=0x7f79fc0b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:53.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:53.123+0000 7f7a1f400640 1 --2- 172.21.15.16:0/733723631 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f79fc0b4c90 0x7f79fc0b71a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7a0800bbe0 tx=0x7f7a0800b390 comp rx=0 tx=0).stop 2024-08-31T06:06:53.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:53.123+0000 7f7a1f400640 1 -- 172.21.15.16:0/733723631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a181141d0 msgr2=0x7f7a181a6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:53.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:53.123+0000 7f7a1f400640 1 --2- 172.21.15.16:0/733723631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a181141d0 0x7f7a181a6050 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7a10009fd0 tx=0x7f7a1000b260 comp rx=0 tx=0).stop 2024-08-31T06:06:53.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:53.123+0000 7f7a1f400640 1 -- 172.21.15.16:0/733723631 shutdown_connections 2024-08-31T06:06:53.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:53.123+0000 7f7a1f400640 1 --2- 172.21.15.16:0/733723631 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f79fc0b4c90 0x7f79fc0b71a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:53.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:53.123+0000 7f7a1f400640 1 --2- 172.21.15.16:0/733723631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a18077120 0x7f7a181a0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:53.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:53.123+0000 7f7a1f400640 1 --2- 172.21.15.16:0/733723631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a181141d0 0x7f7a181a6050 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:53.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:53.123+0000 7f7a1f400640 1 --2- 172.21.15.16:0/733723631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a18075b70 0x7f7a181a0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:53.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:53.123+0000 7f7a1f400640 1 -- 172.21.15.16:0/733723631 >> 172.21.15.16:0/733723631 conn(0x7f7a1807b5f0 msgr2=0x7f7a18112e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:53.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:53.123+0000 7f7a1f400640 1 -- 172.21.15.16:0/733723631 shutdown_connections 2024-08-31T06:06:53.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:53.123+0000 7f7a1f400640 1 -- 172.21.15.16:0/733723631 wait complete. 2024-08-31T06:06:53.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:52 smithi184 ceph-mon[22331]: pgmap v352: 97 pgs: 97 active+clean; 645 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 7 op/s 2024-08-31T06:06:53.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:52 smithi155 ceph-mon[21742]: pgmap v352: 97 pgs: 97 active+clean; 645 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 7 op/s 2024-08-31T06:06:53.374 INFO:tasks.cephfs.filesystem:scrub status for tag:c300b7e8-f3fd-482c-a909-9329bc61e570 - {'path': '/', 'tag': 'c300b7e8-f3fd-482c-a909-9329bc61e570', 'options': 'recursive,force'} 2024-08-31T06:06:54.074 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2024-08-31T06:06:51.293188+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660971,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":883,"num_read_kb":548,"num_write":303,"num_write_kb":2278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":2952987,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":745,"ondisk_log_size":745,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372444,"kb_used_data":10716,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124569380,"statfs":{"total":1151940427776,"available":1151559045120,"internally_reserved":0,"allocated":10973184,"data_stored":7213221,"data_compressed":178617,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5684,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":87,"num_read_kb":52,"num_write":2,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002999"},"pg_stats":[{"pgid":"3.3f","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.686021+0000","last_change":"2024-08-31T06:04:25.877318+0000","last_active":"2024-08-31T06:06:23.686021+0000","last_peered":"2024-08-31T06:06:23.686021+0000","last_clean":"2024-08-31T06:06:23.686021+0000","last_became_active":"2024-08-31T06:04:25.876894+0000","last_became_peered":"2024-08-31T06:04:25.876894+0000","last_unstale":"2024-08-31T06:06:23.686021+0000","last_undegraded":"2024-08-31T06:06:23.686021+0000","last_fullsized":"2024-08-31T06:06:23.686021+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:41:09.449343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"80'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.685244+0000","last_change":"2024-08-31T06:04:25.885670+0000","last_active":"2024-08-31T06:06:23.685244+0000","last_peered":"2024-08-31T06:06:23.685244+0000","last_clean":"2024-08-31T06:06:23.685244+0000","last_became_active":"2024-08-31T06:04:25.883818+0000","last_became_peered":"2024-08-31T06:04:25.883818+0000","last_unstale":"2024-08-31T06:06:23.685244+0000","last_undegraded":"2024-08-31T06:06:23.685244+0000","last_fullsized":"2024-08-31T06:06:23.685244+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:40:01.344432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.687803+0000","last_change":"2024-08-31T06:04:25.883375+0000","last_active":"2024-08-31T06:06:23.687803+0000","last_peered":"2024-08-31T06:06:23.687803+0000","last_clean":"2024-08-31T06:06:23.687803+0000","last_became_active":"2024-08-31T06:04:25.881750+0000","last_became_peered":"2024-08-31T06:04:25.881750+0000","last_unstale":"2024-08-31T06:06:23.687803+0000","last_undegraded":"2024-08-31T06:06:23.687803+0000","last_fullsized":"2024-08-31T06:06:23.687803+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:46:46.757363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.676625+0000","last_change":"2024-08-31T06:04:25.882436+0000","last_active":"2024-08-31T06:06:23.676625+0000","last_peered":"2024-08-31T06:06:23.676625+0000","last_clean":"2024-08-31T06:06:23.676625+0000","last_became_active":"2024-08-31T06:04:25.881805+0000","last_became_peered":"2024-08-31T06:04:25.881805+0000","last_unstale":"2024-08-31T06:06:23.676625+0000","last_undegraded":"2024-08-31T06:06:23.676625+0000","last_fullsized":"2024-08-31T06:06:23.676625+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:42:39.793625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"80'4","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.661470+0000","last_change":"2024-08-31T06:04:25.894534+0000","last_active":"2024-08-31T06:06:23.661470+0000","last_peered":"2024-08-31T06:06:23.661470+0000","last_clean":"2024-08-31T06:06:23.661470+0000","last_became_active":"2024-08-31T06:04:25.894116+0000","last_became_peered":"2024-08-31T06:04:25.894116+0000","last_unstale":"2024-08-31T06:06:23.661470+0000","last_undegraded":"2024-08-31T06:06:23.661470+0000","last_fullsized":"2024-08-31T06:06:23.661470+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:02:31.257454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.669411+0000","last_change":"2024-08-31T06:04:25.886182+0000","last_active":"2024-08-31T06:06:23.669411+0000","last_peered":"2024-08-31T06:06:23.669411+0000","last_clean":"2024-08-31T06:06:23.669411+0000","last_became_active":"2024-08-31T06:04:25.885825+0000","last_became_peered":"2024-08-31T06:04:25.885825+0000","last_unstale":"2024-08-31T06:06:23.669411+0000","last_undegraded":"2024-08-31T06:06:23.669411+0000","last_fullsized":"2024-08-31T06:06:23.669411+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:58:12.427931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.682008+0000","last_change":"2024-08-31T06:04:25.877521+0000","last_active":"2024-08-31T06:06:23.682008+0000","last_peered":"2024-08-31T06:06:23.682008+0000","last_clean":"2024-08-31T06:06:23.682008+0000","last_became_active":"2024-08-31T06:04:25.877122+0000","last_became_peered":"2024-08-31T06:04:25.877122+0000","last_unstale":"2024-08-31T06:06:23.682008+0000","last_undegraded":"2024-08-31T06:06:23.682008+0000","last_fullsized":"2024-08-31T06:06:23.682008+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:17:27.406828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.680720+0000","last_change":"2024-08-31T06:04:25.892547+0000","last_active":"2024-08-31T06:06:23.680720+0000","last_peered":"2024-08-31T06:06:23.680720+0000","last_clean":"2024-08-31T06:06:23.680720+0000","last_became_active":"2024-08-31T06:04:25.892090+0000","last_became_peered":"2024-08-31T06:04:25.892090+0000","last_unstale":"2024-08-31T06:06:23.680720+0000","last_undegraded":"2024-08-31T06:06:23.680720+0000","last_fullsized":"2024-08-31T06:06:23.680720+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:28:53.414252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"80'11","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.689423+0000","last_change":"2024-08-31T06:04:25.879952+0000","last_active":"2024-08-31T06:06:23.689423+0000","last_peered":"2024-08-31T06:06:23.689423+0000","last_clean":"2024-08-31T06:06:23.689423+0000","last_became_active":"2024-08-31T06:04:25.879568+0000","last_became_peered":"2024-08-31T06:04:25.879568+0000","last_unstale":"2024-08-31T06:06:23.689423+0000","last_undegraded":"2024-08-31T06:06:23.689423+0000","last_fullsized":"2024-08-31T06:06:23.689423+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:28:04.558054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"80'16","reported_seq":44,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.676096+0000","last_change":"2024-08-31T06:05:25.051173+0000","last_active":"2024-08-31T06:06:23.676096+0000","last_peered":"2024-08-31T06:06:23.676096+0000","last_clean":"2024-08-31T06:06:23.676096+0000","last_became_active":"2024-08-31T06:05:25.050867+0000","last_became_peered":"2024-08-31T06:05:25.050867+0000","last_unstale":"2024-08-31T06:06:23.676096+0000","last_undegraded":"2024-08-31T06:06:23.676096+0000","last_fullsized":"2024-08-31T06:06:23.676096+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:27:09.971497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.686392+0000","last_change":"2024-08-31T06:04:25.883984+0000","last_active":"2024-08-31T06:06:23.686392+0000","last_peered":"2024-08-31T06:06:23.686392+0000","last_clean":"2024-08-31T06:06:23.686392+0000","last_became_active":"2024-08-31T06:04:25.883353+0000","last_became_peered":"2024-08-31T06:04:25.883353+0000","last_unstale":"2024-08-31T06:06:23.686392+0000","last_undegraded":"2024-08-31T06:06:23.686392+0000","last_fullsized":"2024-08-31T06:06:23.686392+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:16:25.178952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.672858+0000","last_change":"2024-08-31T06:04:25.883335+0000","last_active":"2024-08-31T06:06:23.672858+0000","last_peered":"2024-08-31T06:06:23.672858+0000","last_clean":"2024-08-31T06:06:23.672858+0000","last_became_active":"2024-08-31T06:04:25.881488+0000","last_became_peered":"2024-08-31T06:04:25.881488+0000","last_unstale":"2024-08-31T06:06:23.672858+0000","last_undegraded":"2024-08-31T06:06:23.672858+0000","last_fullsized":"2024-08-31T06:06:23.672858+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:57:24.472482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'10","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.678797+0000","last_change":"2024-08-31T06:04:25.891579+0000","last_active":"2024-08-31T06:06:23.678797+0000","last_peered":"2024-08-31T06:06:23.678797+0000","last_clean":"2024-08-31T06:06:23.678797+0000","last_became_active":"2024-08-31T06:04:25.887950+0000","last_became_peered":"2024-08-31T06:04:25.887950+0000","last_unstale":"2024-08-31T06:06:23.678797+0000","last_undegraded":"2024-08-31T06:06:23.678797+0000","last_fullsized":"2024-08-31T06:06:23.678797+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:38:35.284442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.682430+0000","last_change":"2024-08-31T06:04:25.882185+0000","last_active":"2024-08-31T06:06:23.682430+0000","last_peered":"2024-08-31T06:06:23.682430+0000","last_clean":"2024-08-31T06:06:23.682430+0000","last_became_active":"2024-08-31T06:04:25.876499+0000","last_became_peered":"2024-08-31T06:04:25.876499+0000","last_unstale":"2024-08-31T06:06:23.682430+0000","last_undegraded":"2024-08-31T06:06:23.682430+0000","last_fullsized":"2024-08-31T06:06:23.682430+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:40:01.132353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.683904+0000","last_change":"2024-08-31T06:04:25.883624+0000","last_active":"2024-08-31T06:06:23.683904+0000","last_peered":"2024-08-31T06:06:23.683904+0000","last_clean":"2024-08-31T06:06:23.683904+0000","last_became_active":"2024-08-31T06:04:25.878041+0000","last_became_peered":"2024-08-31T06:04:25.878041+0000","last_unstale":"2024-08-31T06:06:23.683904+0000","last_undegraded":"2024-08-31T06:06:23.683904+0000","last_fullsized":"2024-08-31T06:06:23.683904+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:05:21.278321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.690513+0000","last_change":"2024-08-31T06:04:25.882538+0000","last_active":"2024-08-31T06:06:23.690513+0000","last_peered":"2024-08-31T06:06:23.690513+0000","last_clean":"2024-08-31T06:06:23.690513+0000","last_became_active":"2024-08-31T06:04:25.882201+0000","last_became_peered":"2024-08-31T06:04:25.882201+0000","last_unstale":"2024-08-31T06:06:23.690513+0000","last_undegraded":"2024-08-31T06:06:23.690513+0000","last_fullsized":"2024-08-31T06:06:23.690513+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:47:16.880893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.686157+0000","last_change":"2024-08-31T06:04:25.880376+0000","last_active":"2024-08-31T06:06:23.686157+0000","last_peered":"2024-08-31T06:06:23.686157+0000","last_clean":"2024-08-31T06:06:23.686157+0000","last_became_active":"2024-08-31T06:04:25.877668+0000","last_became_peered":"2024-08-31T06:04:25.877668+0000","last_unstale":"2024-08-31T06:06:23.686157+0000","last_undegraded":"2024-08-31T06:06:23.686157+0000","last_fullsized":"2024-08-31T06:06:23.686157+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:55:32.737406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"80'10","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.673590+0000","last_change":"2024-08-31T06:04:25.875431+0000","last_active":"2024-08-31T06:06:23.673590+0000","last_peered":"2024-08-31T06:06:23.673590+0000","last_clean":"2024-08-31T06:06:23.673590+0000","last_became_active":"2024-08-31T06:04:25.874751+0000","last_became_peered":"2024-08-31T06:04:25.874751+0000","last_unstale":"2024-08-31T06:06:23.673590+0000","last_undegraded":"2024-08-31T06:06:23.673590+0000","last_fullsized":"2024-08-31T06:06:23.673590+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:16:57.695415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.675738+0000","last_change":"2024-08-31T06:04:25.895656+0000","last_active":"2024-08-31T06:06:23.675738+0000","last_peered":"2024-08-31T06:06:23.675738+0000","last_clean":"2024-08-31T06:06:23.675738+0000","last_became_active":"2024-08-31T06:04:25.895272+0000","last_became_peered":"2024-08-31T06:04:25.895272+0000","last_unstale":"2024-08-31T06:06:23.675738+0000","last_undegraded":"2024-08-31T06:06:23.675738+0000","last_fullsized":"2024-08-31T06:06:23.675738+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:18:47.276195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.679697+0000","last_change":"2024-08-31T06:04:25.877283+0000","last_active":"2024-08-31T06:06:23.679697+0000","last_peered":"2024-08-31T06:06:23.679697+0000","last_clean":"2024-08-31T06:06:23.679697+0000","last_became_active":"2024-08-31T06:04:25.876846+0000","last_became_peered":"2024-08-31T06:04:25.876846+0000","last_unstale":"2024-08-31T06:06:23.679697+0000","last_undegraded":"2024-08-31T06:06:23.679697+0000","last_fullsized":"2024-08-31T06:06:23.679697+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:54:05.086844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"80'11","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.685368+0000","last_change":"2024-08-31T06:04:25.884891+0000","last_active":"2024-08-31T06:06:23.685368+0000","last_peered":"2024-08-31T06:06:23.685368+0000","last_clean":"2024-08-31T06:06:23.685368+0000","last_became_active":"2024-08-31T06:04:25.884501+0000","last_became_peered":"2024-08-31T06:04:25.884501+0000","last_unstale":"2024-08-31T06:06:23.685368+0000","last_undegraded":"2024-08-31T06:06:23.685368+0000","last_fullsized":"2024-08-31T06:06:23.685368+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:22:54.846102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"80'10","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.691670+0000","last_change":"2024-08-31T06:04:25.877150+0000","last_active":"2024-08-31T06:06:23.691670+0000","last_peered":"2024-08-31T06:06:23.691670+0000","last_clean":"2024-08-31T06:06:23.691670+0000","last_became_active":"2024-08-31T06:04:25.876766+0000","last_became_peered":"2024-08-31T06:04:25.876766+0000","last_unstale":"2024-08-31T06:06:23.691670+0000","last_undegraded":"2024-08-31T06:06:23.691670+0000","last_fullsized":"2024-08-31T06:06:23.691670+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:46:26.724970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.676164+0000","last_change":"2024-08-31T06:04:25.881275+0000","last_active":"2024-08-31T06:06:23.676164+0000","last_peered":"2024-08-31T06:06:23.676164+0000","last_clean":"2024-08-31T06:06:23.676164+0000","last_became_active":"2024-08-31T06:04:25.879877+0000","last_became_peered":"2024-08-31T06:04:25.879877+0000","last_unstale":"2024-08-31T06:06:23.676164+0000","last_undegraded":"2024-08-31T06:06:23.676164+0000","last_fullsized":"2024-08-31T06:06:23.676164+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:07:07.237041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.676080+0000","last_change":"2024-08-31T06:04:25.895750+0000","last_active":"2024-08-31T06:06:23.676080+0000","last_peered":"2024-08-31T06:06:23.676080+0000","last_clean":"2024-08-31T06:06:23.676080+0000","last_became_active":"2024-08-31T06:04:25.895348+0000","last_became_peered":"2024-08-31T06:04:25.895348+0000","last_unstale":"2024-08-31T06:06:23.676080+0000","last_undegraded":"2024-08-31T06:06:23.676080+0000","last_fullsized":"2024-08-31T06:06:23.676080+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:53:10.014808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.670437+0000","last_change":"2024-08-31T06:04:25.895537+0000","last_active":"2024-08-31T06:06:23.670437+0000","last_peered":"2024-08-31T06:06:23.670437+0000","last_clean":"2024-08-31T06:06:23.670437+0000","last_became_active":"2024-08-31T06:04:25.895136+0000","last_became_peered":"2024-08-31T06:04:25.895136+0000","last_unstale":"2024-08-31T06:06:23.670437+0000","last_undegraded":"2024-08-31T06:06:23.670437+0000","last_fullsized":"2024-08-31T06:06:23.670437+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:41:57.554507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"80'15","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.676301+0000","last_change":"2024-08-31T06:04:25.880490+0000","last_active":"2024-08-31T06:06:23.676301+0000","last_peered":"2024-08-31T06:06:23.676301+0000","last_clean":"2024-08-31T06:06:23.676301+0000","last_became_active":"2024-08-31T06:04:25.879269+0000","last_became_peered":"2024-08-31T06:04:25.879269+0000","last_unstale":"2024-08-31T06:06:23.676301+0000","last_undegraded":"2024-08-31T06:06:23.676301+0000","last_fullsized":"2024-08-31T06:06:23.676301+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:05:49.575915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"80'6","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.676526+0000","last_change":"2024-08-31T06:05:25.053714+0000","last_active":"2024-08-31T06:06:23.676526+0000","last_peered":"2024-08-31T06:06:23.676526+0000","last_clean":"2024-08-31T06:06:23.676526+0000","last_became_active":"2024-08-31T06:05:25.052685+0000","last_became_peered":"2024-08-31T06:05:25.052685+0000","last_unstale":"2024-08-31T06:06:23.676526+0000","last_undegraded":"2024-08-31T06:06:23.676526+0000","last_fullsized":"2024-08-31T06:06:23.676526+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:17:31.474601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"80'4","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.684823+0000","last_change":"2024-08-31T06:05:25.057946+0000","last_active":"2024-08-31T06:06:23.684823+0000","last_peered":"2024-08-31T06:06:23.684823+0000","last_clean":"2024-08-31T06:06:23.684823+0000","last_became_active":"2024-08-31T06:05:25.057501+0000","last_became_peered":"2024-08-31T06:05:25.057501+0000","last_unstale":"2024-08-31T06:06:23.684823+0000","last_undegraded":"2024-08-31T06:06:23.684823+0000","last_fullsized":"2024-08-31T06:06:23.684823+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:16:05.721770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.685410+0000","last_change":"2024-08-31T06:04:25.897473+0000","last_active":"2024-08-31T06:06:23.685410+0000","last_peered":"2024-08-31T06:06:23.685410+0000","last_clean":"2024-08-31T06:06:23.685410+0000","last_became_active":"2024-08-31T06:04:25.895194+0000","last_became_peered":"2024-08-31T06:04:25.895194+0000","last_unstale":"2024-08-31T06:06:23.685410+0000","last_undegraded":"2024-08-31T06:06:23.685410+0000","last_fullsized":"2024-08-31T06:06:23.685410+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:43:04.539970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.684923+0000","last_change":"2024-08-31T06:04:25.894402+0000","last_active":"2024-08-31T06:06:23.684923+0000","last_peered":"2024-08-31T06:06:23.684923+0000","last_clean":"2024-08-31T06:06:23.684923+0000","last_became_active":"2024-08-31T06:04:25.893883+0000","last_became_peered":"2024-08-31T06:04:25.893883+0000","last_unstale":"2024-08-31T06:06:23.684923+0000","last_undegraded":"2024-08-31T06:06:23.684923+0000","last_fullsized":"2024-08-31T06:06:23.684923+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:45:53.778473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"80'12","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.681979+0000","last_change":"2024-08-31T06:04:25.881543+0000","last_active":"2024-08-31T06:06:23.681979+0000","last_peered":"2024-08-31T06:06:23.681979+0000","last_clean":"2024-08-31T06:06:23.681979+0000","last_became_active":"2024-08-31T06:04:25.881144+0000","last_became_peered":"2024-08-31T06:04:25.881144+0000","last_unstale":"2024-08-31T06:06:23.681979+0000","last_undegraded":"2024-08-31T06:06:23.681979+0000","last_fullsized":"2024-08-31T06:06:23.681979+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:13:02.282879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.669781+0000","last_change":"2024-08-31T06:04:25.881143+0000","last_active":"2024-08-31T06:06:23.669781+0000","last_peered":"2024-08-31T06:06:23.669781+0000","last_clean":"2024-08-31T06:06:23.669781+0000","last_became_active":"2024-08-31T06:04:25.879718+0000","last_became_peered":"2024-08-31T06:04:25.879718+0000","last_unstale":"2024-08-31T06:06:23.669781+0000","last_undegraded":"2024-08-31T06:06:23.669781+0000","last_fullsized":"2024-08-31T06:06:23.669781+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:32:07.771488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.680189+0000","last_change":"2024-08-31T06:04:25.879493+0000","last_active":"2024-08-31T06:06:23.680189+0000","last_peered":"2024-08-31T06:06:23.680189+0000","last_clean":"2024-08-31T06:06:23.680189+0000","last_became_active":"2024-08-31T06:04:25.879065+0000","last_became_peered":"2024-08-31T06:04:25.879065+0000","last_unstale":"2024-08-31T06:06:23.680189+0000","last_undegraded":"2024-08-31T06:06:23.680189+0000","last_fullsized":"2024-08-31T06:06:23.680189+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:53:43.848269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"80'7","reported_seq":178,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:47.179720+0000","last_change":"2024-08-31T06:05:25.058954+0000","last_active":"2024-08-31T06:06:47.179720+0000","last_peered":"2024-08-31T06:06:47.179720+0000","last_clean":"2024-08-31T06:06:47.179720+0000","last_became_active":"2024-08-31T06:05:25.051143+0000","last_became_peered":"2024-08-31T06:05:25.051143+0000","last_unstale":"2024-08-31T06:06:47.179720+0000","last_undegraded":"2024-08-31T06:06:47.179720+0000","last_fullsized":"2024-08-31T06:06:47.179720+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:38:23.533663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":127,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.682132+0000","last_change":"2024-08-31T06:04:25.896354+0000","last_active":"2024-08-31T06:06:23.682132+0000","last_peered":"2024-08-31T06:06:23.682132+0000","last_clean":"2024-08-31T06:06:23.682132+0000","last_became_active":"2024-08-31T06:04:25.896021+0000","last_became_peered":"2024-08-31T06:04:25.896021+0000","last_unstale":"2024-08-31T06:06:23.682132+0000","last_undegraded":"2024-08-31T06:06:23.682132+0000","last_fullsized":"2024-08-31T06:06:23.682132+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:08:40.213238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.047153+0000","last_change":"2024-08-31T06:04:22.844248+0000","last_active":"2024-08-31T06:05:25.047153+0000","last_peered":"2024-08-31T06:05:25.047153+0000","last_clean":"2024-08-31T06:05:25.047153+0000","last_became_active":"2024-08-31T06:04:22.843784+0000","last_became_peered":"2024-08-31T06:04:22.843784+0000","last_unstale":"2024-08-31T06:05:25.047153+0000","last_undegraded":"2024-08-31T06:05:25.047153+0000","last_fullsized":"2024-08-31T06:05:25.047153+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:56:41.422965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.046268+0000","last_change":"2024-08-31T06:04:22.847049+0000","last_active":"2024-08-31T06:05:25.046268+0000","last_peered":"2024-08-31T06:05:25.046268+0000","last_clean":"2024-08-31T06:05:25.046268+0000","last_became_active":"2024-08-31T06:04:22.846282+0000","last_became_peered":"2024-08-31T06:04:22.846282+0000","last_unstale":"2024-08-31T06:05:25.046268+0000","last_undegraded":"2024-08-31T06:05:25.046268+0000","last_fullsized":"2024-08-31T06:05:25.046268+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:35:21.030425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.679457+0000","last_change":"2024-08-31T06:04:25.886698+0000","last_active":"2024-08-31T06:06:23.679457+0000","last_peered":"2024-08-31T06:06:23.679457+0000","last_clean":"2024-08-31T06:06:23.679457+0000","last_became_active":"2024-08-31T06:04:25.886334+0000","last_became_peered":"2024-08-31T06:04:25.886334+0000","last_unstale":"2024-08-31T06:06:23.679457+0000","last_undegraded":"2024-08-31T06:06:23.679457+0000","last_fullsized":"2024-08-31T06:06:23.679457+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:24:43.821446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.044900+0000","last_change":"2024-08-31T06:04:22.845263+0000","last_active":"2024-08-31T06:05:25.044900+0000","last_peered":"2024-08-31T06:05:25.044900+0000","last_clean":"2024-08-31T06:05:25.044900+0000","last_became_active":"2024-08-31T06:04:22.844853+0000","last_became_peered":"2024-08-31T06:04:22.844853+0000","last_unstale":"2024-08-31T06:05:25.044900+0000","last_undegraded":"2024-08-31T06:05:25.044900+0000","last_fullsized":"2024-08-31T06:05:25.044900+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:55:12.561165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"80'11","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.673293+0000","last_change":"2024-08-31T06:04:25.894475+0000","last_active":"2024-08-31T06:06:23.673293+0000","last_peered":"2024-08-31T06:06:23.673293+0000","last_clean":"2024-08-31T06:06:23.673293+0000","last_became_active":"2024-08-31T06:04:25.894121+0000","last_became_peered":"2024-08-31T06:04:25.894121+0000","last_unstale":"2024-08-31T06:06:23.673293+0000","last_undegraded":"2024-08-31T06:06:23.673293+0000","last_fullsized":"2024-08-31T06:06:23.673293+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:40:06.570616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":160,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:49.182093+0000","last_change":"2024-08-31T06:04:22.834430+0000","last_active":"2024-08-31T06:06:49.182093+0000","last_peered":"2024-08-31T06:06:49.182093+0000","last_clean":"2024-08-31T06:06:49.182093+0000","last_became_active":"2024-08-31T06:04:22.834011+0000","last_became_peered":"2024-08-31T06:04:22.834011+0000","last_unstale":"2024-08-31T06:06:49.182093+0000","last_undegraded":"2024-08-31T06:06:49.182093+0000","last_fullsized":"2024-08-31T06:06:49.182093+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:51:16.734234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":129,"num_read_kb":129,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"80'10","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.689788+0000","last_change":"2024-08-31T06:04:25.877727+0000","last_active":"2024-08-31T06:06:23.689788+0000","last_peered":"2024-08-31T06:06:23.689788+0000","last_clean":"2024-08-31T06:06:23.689788+0000","last_became_active":"2024-08-31T06:04:25.877121+0000","last_became_peered":"2024-08-31T06:04:25.877121+0000","last_unstale":"2024-08-31T06:06:23.689788+0000","last_undegraded":"2024-08-31T06:06:23.689788+0000","last_fullsized":"2024-08-31T06:06:23.689788+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:41:12.247216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048879+0000","last_change":"2024-08-31T06:04:22.846482+0000","last_active":"2024-08-31T06:05:25.048879+0000","last_peered":"2024-08-31T06:05:25.048879+0000","last_clean":"2024-08-31T06:05:25.048879+0000","last_became_active":"2024-08-31T06:04:22.846057+0000","last_became_peered":"2024-08-31T06:04:22.846057+0000","last_unstale":"2024-08-31T06:05:25.048879+0000","last_undegraded":"2024-08-31T06:05:25.048879+0000","last_fullsized":"2024-08-31T06:05:25.048879+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:29:11.861665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.684452+0000","last_change":"2024-08-31T06:04:25.893777+0000","last_active":"2024-08-31T06:06:23.684452+0000","last_peered":"2024-08-31T06:06:23.684452+0000","last_clean":"2024-08-31T06:06:23.684452+0000","last_became_active":"2024-08-31T06:04:25.893014+0000","last_became_peered":"2024-08-31T06:04:25.893014+0000","last_unstale":"2024-08-31T06:06:23.684452+0000","last_undegraded":"2024-08-31T06:06:23.684452+0000","last_fullsized":"2024-08-31T06:06:23.684452+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:04:48.185842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.045551+0000","last_change":"2024-08-31T06:04:22.965938+0000","last_active":"2024-08-31T06:05:25.045551+0000","last_peered":"2024-08-31T06:05:25.045551+0000","last_clean":"2024-08-31T06:05:25.045551+0000","last_became_active":"2024-08-31T06:04:22.965356+0000","last_became_peered":"2024-08-31T06:04:22.965356+0000","last_unstale":"2024-08-31T06:05:25.045551+0000","last_undegraded":"2024-08-31T06:05:25.045551+0000","last_fullsized":"2024-08-31T06:05:25.045551+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:36:58.938462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"80'11","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.691850+0000","last_change":"2024-08-31T06:04:25.881307+0000","last_active":"2024-08-31T06:06:23.691850+0000","last_peered":"2024-08-31T06:06:23.691850+0000","last_clean":"2024-08-31T06:06:23.691850+0000","last_became_active":"2024-08-31T06:04:25.880418+0000","last_became_peered":"2024-08-31T06:04:25.880418+0000","last_unstale":"2024-08-31T06:06:23.691850+0000","last_undegraded":"2024-08-31T06:06:23.691850+0000","last_fullsized":"2024-08-31T06:06:23.691850+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:21:46.904371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'34","reported_seq":331,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:49.181432+0000","last_change":"2024-08-31T06:04:22.843073+0000","last_active":"2024-08-31T06:06:49.181432+0000","last_peered":"2024-08-31T06:06:49.181432+0000","last_clean":"2024-08-31T06:06:49.181432+0000","last_became_active":"2024-08-31T06:04:22.842645+0000","last_became_peered":"2024-08-31T06:04:22.842645+0000","last_unstale":"2024-08-31T06:06:49.181432+0000","last_undegraded":"2024-08-31T06:06:49.181432+0000","last_fullsized":"2024-08-31T06:06:49.181432+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:41:37.651562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65695,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":270,"num_read_kb":199,"num_write":35,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.680499+0000","last_change":"2024-08-31T06:04:25.879929+0000","last_active":"2024-08-31T06:06:23.680499+0000","last_peered":"2024-08-31T06:06:23.680499+0000","last_clean":"2024-08-31T06:06:23.680499+0000","last_became_active":"2024-08-31T06:04:25.878475+0000","last_became_peered":"2024-08-31T06:04:25.878475+0000","last_unstale":"2024-08-31T06:06:23.680499+0000","last_undegraded":"2024-08-31T06:06:23.680499+0000","last_fullsized":"2024-08-31T06:06:23.680499+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:10:32.801558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048264+0000","last_change":"2024-08-31T06:04:22.842878+0000","last_active":"2024-08-31T06:05:25.048264+0000","last_peered":"2024-08-31T06:05:25.048264+0000","last_clean":"2024-08-31T06:05:25.048264+0000","last_became_active":"2024-08-31T06:04:22.842306+0000","last_became_peered":"2024-08-31T06:04:22.842306+0000","last_unstale":"2024-08-31T06:05:25.048264+0000","last_undegraded":"2024-08-31T06:05:25.048264+0000","last_fullsized":"2024-08-31T06:05:25.048264+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:53:55.259238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.669929+0000","last_change":"2024-08-31T06:04:25.899357+0000","last_active":"2024-08-31T06:06:23.669929+0000","last_peered":"2024-08-31T06:06:23.669929+0000","last_clean":"2024-08-31T06:06:23.669929+0000","last_became_active":"2024-08-31T06:04:25.899000+0000","last_became_peered":"2024-08-31T06:04:25.899000+0000","last_unstale":"2024-08-31T06:06:23.669929+0000","last_undegraded":"2024-08-31T06:06:23.669929+0000","last_fullsized":"2024-08-31T06:06:23.669929+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:20:39.991071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.044759+0000","last_change":"2024-08-31T06:04:22.843063+0000","last_active":"2024-08-31T06:05:25.044759+0000","last_peered":"2024-08-31T06:05:25.044759+0000","last_clean":"2024-08-31T06:05:25.044759+0000","last_became_active":"2024-08-31T06:04:22.842654+0000","last_became_peered":"2024-08-31T06:04:22.842654+0000","last_unstale":"2024-08-31T06:05:25.044759+0000","last_undegraded":"2024-08-31T06:05:25.044759+0000","last_fullsized":"2024-08-31T06:05:25.044759+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:09:54.563116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"80'15","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.683856+0000","last_change":"2024-08-31T06:04:25.885556+0000","last_active":"2024-08-31T06:06:23.683856+0000","last_peered":"2024-08-31T06:06:23.683856+0000","last_clean":"2024-08-31T06:06:23.683856+0000","last_became_active":"2024-08-31T06:04:25.885216+0000","last_became_peered":"2024-08-31T06:04:25.885216+0000","last_unstale":"2024-08-31T06:06:23.683856+0000","last_undegraded":"2024-08-31T06:06:23.683856+0000","last_fullsized":"2024-08-31T06:06:23.683856+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:48:49.816261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'5","reported_seq":165,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:47.738312+0000","last_change":"2024-08-31T06:04:22.844839+0000","last_active":"2024-08-31T06:06:47.738312+0000","last_peered":"2024-08-31T06:06:47.738312+0000","last_clean":"2024-08-31T06:06:47.738312+0000","last_became_active":"2024-08-31T06:04:22.844425+0000","last_became_peered":"2024-08-31T06:04:22.844425+0000","last_unstale":"2024-08-31T06:06:47.738312+0000","last_undegraded":"2024-08-31T06:06:47.738312+0000","last_fullsized":"2024-08-31T06:06:47.738312+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:10:57.292173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":134,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.682394+0000","last_change":"2024-08-31T06:04:25.878195+0000","last_active":"2024-08-31T06:06:23.682394+0000","last_peered":"2024-08-31T06:06:23.682394+0000","last_clean":"2024-08-31T06:06:23.682394+0000","last_became_active":"2024-08-31T06:04:25.877823+0000","last_became_peered":"2024-08-31T06:04:25.877823+0000","last_unstale":"2024-08-31T06:06:23.682394+0000","last_undegraded":"2024-08-31T06:06:23.682394+0000","last_fullsized":"2024-08-31T06:06:23.682394+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:54:34.989583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":68,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.071826+0000","last_change":"2024-08-31T06:05:25.071391+0000","last_active":"2024-08-31T06:05:25.071826+0000","last_peered":"2024-08-31T06:05:25.071826+0000","last_clean":"2024-08-31T06:05:25.071826+0000","last_became_active":"2024-08-31T06:05:25.057546+0000","last_became_peered":"2024-08-31T06:05:25.057546+0000","last_unstale":"2024-08-31T06:05:25.071826+0000","last_undegraded":"2024-08-31T06:05:25.071826+0000","last_fullsized":"2024-08-31T06:05:25.071826+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:42:35.527395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.681995+0000","last_change":"2024-08-31T06:04:25.887580+0000","last_active":"2024-08-31T06:06:23.681995+0000","last_peered":"2024-08-31T06:06:23.681995+0000","last_clean":"2024-08-31T06:06:23.681995+0000","last_became_active":"2024-08-31T06:04:25.887158+0000","last_became_peered":"2024-08-31T06:04:25.887158+0000","last_unstale":"2024-08-31T06:06:23.681995+0000","last_undegraded":"2024-08-31T06:06:23.681995+0000","last_fullsized":"2024-08-31T06:06:23.681995+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:27:38.115742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.064443+0000","last_change":"2024-08-31T06:05:25.063859+0000","last_active":"2024-08-31T06:05:25.064443+0000","last_peered":"2024-08-31T06:05:25.064443+0000","last_clean":"2024-08-31T06:05:25.064443+0000","last_became_active":"2024-08-31T06:05:25.053596+0000","last_became_peered":"2024-08-31T06:05:25.053596+0000","last_unstale":"2024-08-31T06:05:25.064443+0000","last_undegraded":"2024-08-31T06:05:25.064443+0000","last_fullsized":"2024-08-31T06:05:25.064443+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:19:18.461630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.678982+0000","last_change":"2024-08-31T06:04:25.880845+0000","last_active":"2024-08-31T06:06:23.678982+0000","last_peered":"2024-08-31T06:06:23.678982+0000","last_clean":"2024-08-31T06:06:23.678982+0000","last_became_active":"2024-08-31T06:04:25.880423+0000","last_became_peered":"2024-08-31T06:04:25.880423+0000","last_unstale":"2024-08-31T06:06:23.678982+0000","last_undegraded":"2024-08-31T06:06:23.678982+0000","last_fullsized":"2024-08-31T06:06:23.678982+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:16:56.725116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.046584+0000","last_change":"2024-08-31T06:04:22.843393+0000","last_active":"2024-08-31T06:05:25.046584+0000","last_peered":"2024-08-31T06:05:25.046584+0000","last_clean":"2024-08-31T06:05:25.046584+0000","last_became_active":"2024-08-31T06:04:22.842770+0000","last_became_peered":"2024-08-31T06:04:22.842770+0000","last_unstale":"2024-08-31T06:05:25.046584+0000","last_undegraded":"2024-08-31T06:05:25.046584+0000","last_fullsized":"2024-08-31T06:05:25.046584+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:49:16.159167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.686211+0000","last_change":"2024-08-31T06:04:25.887829+0000","last_active":"2024-08-31T06:06:23.686211+0000","last_peered":"2024-08-31T06:06:23.686211+0000","last_clean":"2024-08-31T06:06:23.686211+0000","last_became_active":"2024-08-31T06:04:25.886567+0000","last_became_peered":"2024-08-31T06:04:25.886567+0000","last_unstale":"2024-08-31T06:06:23.686211+0000","last_undegraded":"2024-08-31T06:06:23.686211+0000","last_fullsized":"2024-08-31T06:06:23.686211+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:11:39.054966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.045627+0000","last_change":"2024-08-31T06:04:22.844218+0000","last_active":"2024-08-31T06:05:25.045627+0000","last_peered":"2024-08-31T06:05:25.045627+0000","last_clean":"2024-08-31T06:05:25.045627+0000","last_became_active":"2024-08-31T06:04:22.843786+0000","last_became_peered":"2024-08-31T06:04:22.843786+0000","last_unstale":"2024-08-31T06:05:25.045627+0000","last_undegraded":"2024-08-31T06:05:25.045627+0000","last_fullsized":"2024-08-31T06:05:25.045627+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:46:02.534190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.682380+0000","last_change":"2024-08-31T06:04:25.888946+0000","last_active":"2024-08-31T06:06:23.682380+0000","last_peered":"2024-08-31T06:06:23.682380+0000","last_clean":"2024-08-31T06:06:23.682380+0000","last_became_active":"2024-08-31T06:04:25.888564+0000","last_became_peered":"2024-08-31T06:04:25.888564+0000","last_unstale":"2024-08-31T06:06:23.682380+0000","last_undegraded":"2024-08-31T06:06:23.682380+0000","last_fullsized":"2024-08-31T06:06:23.682380+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:26:19.051971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.049211+0000","last_change":"2024-08-31T06:04:22.964483+0000","last_active":"2024-08-31T06:05:25.049211+0000","last_peered":"2024-08-31T06:05:25.049211+0000","last_clean":"2024-08-31T06:05:25.049211+0000","last_became_active":"2024-08-31T06:04:22.963874+0000","last_became_peered":"2024-08-31T06:04:22.963874+0000","last_unstale":"2024-08-31T06:05:25.049211+0000","last_undegraded":"2024-08-31T06:05:25.049211+0000","last_fullsized":"2024-08-31T06:05:25.049211+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:30:00.159040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.688990+0000","last_change":"2024-08-31T06:04:25.880941+0000","last_active":"2024-08-31T06:06:23.688990+0000","last_peered":"2024-08-31T06:06:23.688990+0000","last_clean":"2024-08-31T06:06:23.688990+0000","last_became_active":"2024-08-31T06:04:25.880454+0000","last_became_peered":"2024-08-31T06:04:25.880454+0000","last_unstale":"2024-08-31T06:06:23.688990+0000","last_undegraded":"2024-08-31T06:06:23.688990+0000","last_fullsized":"2024-08-31T06:06:23.688990+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:49:57.023988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":98,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048527+0000","last_change":"2024-08-31T06:03:05.921629+0000","last_active":"2024-08-31T06:05:25.048527+0000","last_peered":"2024-08-31T06:05:25.048527+0000","last_clean":"2024-08-31T06:05:25.048527+0000","last_became_active":"2024-08-31T06:03:05.915844+0000","last_became_peered":"2024-08-31T06:03:05.915844+0000","last_unstale":"2024-08-31T06:05:25.048527+0000","last_undegraded":"2024-08-31T06:05:25.048527+0000","last_fullsized":"2024-08-31T06:05:25.048527+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T05:59:24.472429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T05:59:24.472429+0000","last_clean_scrub_stamp":"2024-08-31T05:59:24.472429+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:56:41.073422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.044718+0000","last_change":"2024-08-31T06:04:22.846325+0000","last_active":"2024-08-31T06:05:25.044718+0000","last_peered":"2024-08-31T06:05:25.044718+0000","last_clean":"2024-08-31T06:05:25.044718+0000","last_became_active":"2024-08-31T06:04:22.845772+0000","last_became_peered":"2024-08-31T06:04:22.845772+0000","last_unstale":"2024-08-31T06:05:25.044718+0000","last_undegraded":"2024-08-31T06:05:25.044718+0000","last_fullsized":"2024-08-31T06:05:25.044718+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:54:10.336786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.691294+0000","last_change":"2024-08-31T06:04:25.891437+0000","last_active":"2024-08-31T06:06:23.691294+0000","last_peered":"2024-08-31T06:06:23.691294+0000","last_clean":"2024-08-31T06:06:23.691294+0000","last_became_active":"2024-08-31T06:04:25.891053+0000","last_became_peered":"2024-08-31T06:04:25.891053+0000","last_unstale":"2024-08-31T06:06:23.691294+0000","last_undegraded":"2024-08-31T06:06:23.691294+0000","last_fullsized":"2024-08-31T06:06:23.691294+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:22:37.119483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048411+0000","last_change":"2024-08-31T06:04:22.842885+0000","last_active":"2024-08-31T06:05:25.048411+0000","last_peered":"2024-08-31T06:05:25.048411+0000","last_clean":"2024-08-31T06:05:25.048411+0000","last_became_active":"2024-08-31T06:04:22.842473+0000","last_became_peered":"2024-08-31T06:04:22.842473+0000","last_unstale":"2024-08-31T06:05:25.048411+0000","last_undegraded":"2024-08-31T06:05:25.048411+0000","last_fullsized":"2024-08-31T06:05:25.048411+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:33:36.516659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.663699+0000","last_change":"2024-08-31T06:04:25.889213+0000","last_active":"2024-08-31T06:06:23.663699+0000","last_peered":"2024-08-31T06:06:23.663699+0000","last_clean":"2024-08-31T06:06:23.663699+0000","last_became_active":"2024-08-31T06:04:25.888860+0000","last_became_peered":"2024-08-31T06:04:25.888860+0000","last_unstale":"2024-08-31T06:06:23.663699+0000","last_undegraded":"2024-08-31T06:06:23.663699+0000","last_fullsized":"2024-08-31T06:06:23.663699+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:44:18.831229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":162,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:48.739363+0000","last_change":"2024-08-31T06:04:22.966091+0000","last_active":"2024-08-31T06:06:48.739363+0000","last_peered":"2024-08-31T06:06:48.739363+0000","last_clean":"2024-08-31T06:06:48.739363+0000","last_became_active":"2024-08-31T06:04:22.965727+0000","last_became_peered":"2024-08-31T06:04:22.965727+0000","last_unstale":"2024-08-31T06:06:48.739363+0000","last_undegraded":"2024-08-31T06:06:48.739363+0000","last_fullsized":"2024-08-31T06:06:48.739363+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:42:43.643623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":130,"num_read_kb":130,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.682138+0000","last_change":"2024-08-31T06:04:25.880039+0000","last_active":"2024-08-31T06:06:23.682138+0000","last_peered":"2024-08-31T06:06:23.682138+0000","last_clean":"2024-08-31T06:06:23.682138+0000","last_became_active":"2024-08-31T06:04:25.879669+0000","last_became_peered":"2024-08-31T06:04:25.879669+0000","last_unstale":"2024-08-31T06:06:23.682138+0000","last_undegraded":"2024-08-31T06:06:23.682138+0000","last_fullsized":"2024-08-31T06:06:23.682138+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:02:09.244281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048687+0000","last_change":"2024-08-31T06:04:22.844572+0000","last_active":"2024-08-31T06:05:25.048687+0000","last_peered":"2024-08-31T06:05:25.048687+0000","last_clean":"2024-08-31T06:05:25.048687+0000","last_became_active":"2024-08-31T06:04:22.844227+0000","last_became_peered":"2024-08-31T06:04:22.844227+0000","last_unstale":"2024-08-31T06:05:25.048687+0000","last_undegraded":"2024-08-31T06:05:25.048687+0000","last_fullsized":"2024-08-31T06:05:25.048687+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:04:51.966901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.677043+0000","last_change":"2024-08-31T06:04:25.884442+0000","last_active":"2024-08-31T06:06:23.677043+0000","last_peered":"2024-08-31T06:06:23.677043+0000","last_clean":"2024-08-31T06:06:23.677043+0000","last_became_active":"2024-08-31T06:04:25.884016+0000","last_became_peered":"2024-08-31T06:04:25.884016+0000","last_unstale":"2024-08-31T06:06:23.677043+0000","last_undegraded":"2024-08-31T06:06:23.677043+0000","last_fullsized":"2024-08-31T06:06:23.677043+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:36:10.640262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.043831+0000","last_change":"2024-08-31T06:04:22.967845+0000","last_active":"2024-08-31T06:05:25.043831+0000","last_peered":"2024-08-31T06:05:25.043831+0000","last_clean":"2024-08-31T06:05:25.043831+0000","last_became_active":"2024-08-31T06:04:22.967421+0000","last_became_peered":"2024-08-31T06:04:22.967421+0000","last_unstale":"2024-08-31T06:05:25.043831+0000","last_undegraded":"2024-08-31T06:05:25.043831+0000","last_fullsized":"2024-08-31T06:05:25.043831+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:17:46.942551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.673410+0000","last_change":"2024-08-31T06:04:25.897087+0000","last_active":"2024-08-31T06:06:23.673410+0000","last_peered":"2024-08-31T06:06:23.673410+0000","last_clean":"2024-08-31T06:06:23.673410+0000","last_became_active":"2024-08-31T06:04:25.896725+0000","last_became_peered":"2024-08-31T06:04:25.896725+0000","last_unstale":"2024-08-31T06:06:23.673410+0000","last_undegraded":"2024-08-31T06:06:23.673410+0000","last_fullsized":"2024-08-31T06:06:23.673410+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:31:28.279489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.053673+0000","last_change":"2024-08-31T06:04:22.965299+0000","last_active":"2024-08-31T06:05:25.053673+0000","last_peered":"2024-08-31T06:05:25.053673+0000","last_clean":"2024-08-31T06:05:25.053673+0000","last_became_active":"2024-08-31T06:04:22.964849+0000","last_became_peered":"2024-08-31T06:04:22.964849+0000","last_unstale":"2024-08-31T06:05:25.053673+0000","last_undegraded":"2024-08-31T06:05:25.053673+0000","last_fullsized":"2024-08-31T06:05:25.053673+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:36:56.431678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.679635+0000","last_change":"2024-08-31T06:04:25.885115+0000","last_active":"2024-08-31T06:06:23.679635+0000","last_peered":"2024-08-31T06:06:23.679635+0000","last_clean":"2024-08-31T06:06:23.679635+0000","last_became_active":"2024-08-31T06:04:25.884722+0000","last_became_peered":"2024-08-31T06:04:25.884722+0000","last_unstale":"2024-08-31T06:06:23.679635+0000","last_undegraded":"2024-08-31T06:06:23.679635+0000","last_fullsized":"2024-08-31T06:06:23.679635+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:20:43.913785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'13","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:32.962890+0000","last_change":"2024-08-31T06:04:22.845119+0000","last_active":"2024-08-31T06:06:32.962890+0000","last_peered":"2024-08-31T06:06:32.962890+0000","last_clean":"2024-08-31T06:06:32.962890+0000","last_became_active":"2024-08-31T06:04:22.843814+0000","last_became_peered":"2024-08-31T06:04:22.843814+0000","last_unstale":"2024-08-31T06:06:32.962890+0000","last_undegraded":"2024-08-31T06:06:32.962890+0000","last_fullsized":"2024-08-31T06:06:32.962890+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:06:40.684887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.680844+0000","last_change":"2024-08-31T06:04:25.892838+0000","last_active":"2024-08-31T06:06:23.680844+0000","last_peered":"2024-08-31T06:06:23.680844+0000","last_clean":"2024-08-31T06:06:23.680844+0000","last_became_active":"2024-08-31T06:04:25.891955+0000","last_became_peered":"2024-08-31T06:04:25.891955+0000","last_unstale":"2024-08-31T06:06:23.680844+0000","last_undegraded":"2024-08-31T06:06:23.680844+0000","last_fullsized":"2024-08-31T06:06:23.680844+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:41:35.085724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.044957+0000","last_change":"2024-08-31T06:04:22.964183+0000","last_active":"2024-08-31T06:05:25.044957+0000","last_peered":"2024-08-31T06:05:25.044957+0000","last_clean":"2024-08-31T06:05:25.044957+0000","last_became_active":"2024-08-31T06:04:22.963688+0000","last_became_peered":"2024-08-31T06:04:22.963688+0000","last_unstale":"2024-08-31T06:05:25.044957+0000","last_undegraded":"2024-08-31T06:05:25.044957+0000","last_fullsized":"2024-08-31T06:05:25.044957+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:04:47.833811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.683320+0000","last_change":"2024-08-31T06:04:25.892266+0000","last_active":"2024-08-31T06:06:23.683320+0000","last_peered":"2024-08-31T06:06:23.683320+0000","last_clean":"2024-08-31T06:06:23.683320+0000","last_became_active":"2024-08-31T06:04:25.889014+0000","last_became_peered":"2024-08-31T06:04:25.889014+0000","last_unstale":"2024-08-31T06:06:23.683320+0000","last_undegraded":"2024-08-31T06:06:23.683320+0000","last_fullsized":"2024-08-31T06:06:23.683320+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:27:47.608470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.049408+0000","last_change":"2024-08-31T06:04:22.964677+0000","last_active":"2024-08-31T06:05:25.049408+0000","last_peered":"2024-08-31T06:05:25.049408+0000","last_clean":"2024-08-31T06:05:25.049408+0000","last_became_active":"2024-08-31T06:04:22.964237+0000","last_became_peered":"2024-08-31T06:04:22.964237+0000","last_unstale":"2024-08-31T06:05:25.049408+0000","last_undegraded":"2024-08-31T06:05:25.049408+0000","last_fullsized":"2024-08-31T06:05:25.049408+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:00:54.099592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.648239+0000","last_change":"2024-08-31T06:04:25.881699+0000","last_active":"2024-08-31T06:06:23.648239+0000","last_peered":"2024-08-31T06:06:23.648239+0000","last_clean":"2024-08-31T06:06:23.648239+0000","last_became_active":"2024-08-31T06:04:25.879304+0000","last_became_peered":"2024-08-31T06:04:25.879304+0000","last_unstale":"2024-08-31T06:06:23.648239+0000","last_undegraded":"2024-08-31T06:06:23.648239+0000","last_fullsized":"2024-08-31T06:06:23.648239+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:56:42.992034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.108301+0000","last_change":"2024-08-31T06:04:22.847934+0000","last_active":"2024-08-31T06:05:25.108301+0000","last_peered":"2024-08-31T06:05:25.108301+0000","last_clean":"2024-08-31T06:05:25.108301+0000","last_became_active":"2024-08-31T06:04:22.847518+0000","last_became_peered":"2024-08-31T06:04:22.847518+0000","last_unstale":"2024-08-31T06:05:25.108301+0000","last_undegraded":"2024-08-31T06:05:25.108301+0000","last_fullsized":"2024-08-31T06:05:25.108301+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:26:21.476014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.685828+0000","last_change":"2024-08-31T06:04:25.884653+0000","last_active":"2024-08-31T06:06:23.685828+0000","last_peered":"2024-08-31T06:06:23.685828+0000","last_clean":"2024-08-31T06:06:23.685828+0000","last_became_active":"2024-08-31T06:04:25.884244+0000","last_became_peered":"2024-08-31T06:04:25.884244+0000","last_unstale":"2024-08-31T06:06:23.685828+0000","last_undegraded":"2024-08-31T06:06:23.685828+0000","last_fullsized":"2024-08-31T06:06:23.685828+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:13:00.814071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048648+0000","last_change":"2024-08-31T06:04:22.845924+0000","last_active":"2024-08-31T06:05:25.048648+0000","last_peered":"2024-08-31T06:05:25.048648+0000","last_clean":"2024-08-31T06:05:25.048648+0000","last_became_active":"2024-08-31T06:04:22.845445+0000","last_became_peered":"2024-08-31T06:04:22.845445+0000","last_unstale":"2024-08-31T06:05:25.048648+0000","last_undegraded":"2024-08-31T06:05:25.048648+0000","last_fullsized":"2024-08-31T06:05:25.048648+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:25:21.052136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.679162+0000","last_change":"2024-08-31T06:04:25.890069+0000","last_active":"2024-08-31T06:06:23.679162+0000","last_peered":"2024-08-31T06:06:23.679162+0000","last_clean":"2024-08-31T06:06:23.679162+0000","last_became_active":"2024-08-31T06:04:25.889709+0000","last_became_peered":"2024-08-31T06:04:25.889709+0000","last_unstale":"2024-08-31T06:06:23.679162+0000","last_undegraded":"2024-08-31T06:06:23.679162+0000","last_fullsized":"2024-08-31T06:06:23.679162+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:13:51.360023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.051611+0000","last_change":"2024-08-31T06:04:22.847845+0000","last_active":"2024-08-31T06:05:25.051611+0000","last_peered":"2024-08-31T06:05:25.051611+0000","last_clean":"2024-08-31T06:05:25.051611+0000","last_became_active":"2024-08-31T06:04:22.847001+0000","last_became_peered":"2024-08-31T06:04:22.847001+0000","last_unstale":"2024-08-31T06:05:25.051611+0000","last_undegraded":"2024-08-31T06:05:25.051611+0000","last_fullsized":"2024-08-31T06:05:25.051611+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:24:34.209076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.690813+0000","last_change":"2024-08-31T06:04:25.886759+0000","last_active":"2024-08-31T06:06:23.690813+0000","last_peered":"2024-08-31T06:06:23.690813+0000","last_clean":"2024-08-31T06:06:23.690813+0000","last_became_active":"2024-08-31T06:04:25.886176+0000","last_became_peered":"2024-08-31T06:04:25.886176+0000","last_unstale":"2024-08-31T06:06:23.690813+0000","last_undegraded":"2024-08-31T06:06:23.690813+0000","last_fullsized":"2024-08-31T06:06:23.690813+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:11:12.637451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"80'15","reported_seq":43,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.677867+0000","last_change":"2024-08-31T06:05:25.054468+0000","last_active":"2024-08-31T06:06:23.677867+0000","last_peered":"2024-08-31T06:06:23.677867+0000","last_clean":"2024-08-31T06:06:23.677867+0000","last_became_active":"2024-08-31T06:05:25.053657+0000","last_became_peered":"2024-08-31T06:05:25.053657+0000","last_unstale":"2024-08-31T06:06:23.677867+0000","last_undegraded":"2024-08-31T06:06:23.677867+0000","last_fullsized":"2024-08-31T06:06:23.677867+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:43:44.887637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.059757+0000","last_change":"2024-08-31T06:05:25.059219+0000","last_active":"2024-08-31T06:05:25.059757+0000","last_peered":"2024-08-31T06:05:25.059757+0000","last_clean":"2024-08-31T06:05:25.059757+0000","last_became_active":"2024-08-31T06:05:25.053136+0000","last_became_peered":"2024-08-31T06:05:25.053136+0000","last_unstale":"2024-08-31T06:05:25.059757+0000","last_undegraded":"2024-08-31T06:05:25.059757+0000","last_fullsized":"2024-08-31T06:05:25.059757+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:29:59.137465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.691662+0000","last_change":"2024-08-31T06:04:25.887524+0000","last_active":"2024-08-31T06:06:23.691662+0000","last_peered":"2024-08-31T06:06:23.691662+0000","last_clean":"2024-08-31T06:06:23.691662+0000","last_became_active":"2024-08-31T06:04:25.887176+0000","last_became_peered":"2024-08-31T06:04:25.887176+0000","last_unstale":"2024-08-31T06:06:23.691662+0000","last_undegraded":"2024-08-31T06:06:23.691662+0000","last_fullsized":"2024-08-31T06:06:23.691662+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:08:05.476804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.053524+0000","last_change":"2024-08-31T06:04:22.847747+0000","last_active":"2024-08-31T06:05:25.053524+0000","last_peered":"2024-08-31T06:05:25.053524+0000","last_clean":"2024-08-31T06:05:25.053524+0000","last_became_active":"2024-08-31T06:04:22.847332+0000","last_became_peered":"2024-08-31T06:04:22.847332+0000","last_unstale":"2024-08-31T06:05:25.053524+0000","last_undegraded":"2024-08-31T06:05:25.053524+0000","last_fullsized":"2024-08-31T06:05:25.053524+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:41:35.910991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.687796+0000","last_change":"2024-08-31T06:04:25.883110+0000","last_active":"2024-08-31T06:06:23.687796+0000","last_peered":"2024-08-31T06:06:23.687796+0000","last_clean":"2024-08-31T06:06:23.687796+0000","last_became_active":"2024-08-31T06:04:25.880487+0000","last_became_peered":"2024-08-31T06:04:25.880487+0000","last_unstale":"2024-08-31T06:06:23.687796+0000","last_undegraded":"2024-08-31T06:06:23.687796+0000","last_fullsized":"2024-08-31T06:06:23.687796+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:22:13.656078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.108210+0000","last_change":"2024-08-31T06:04:22.964758+0000","last_active":"2024-08-31T06:05:25.108210+0000","last_peered":"2024-08-31T06:05:25.108210+0000","last_clean":"2024-08-31T06:05:25.108210+0000","last_became_active":"2024-08-31T06:04:22.964391+0000","last_became_peered":"2024-08-31T06:04:22.964391+0000","last_unstale":"2024-08-31T06:05:25.108210+0000","last_undegraded":"2024-08-31T06:05:25.108210+0000","last_fullsized":"2024-08-31T06:05:25.108210+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:28:59.190657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.678294+0000","last_change":"2024-08-31T06:04:25.893738+0000","last_active":"2024-08-31T06:06:23.678294+0000","last_peered":"2024-08-31T06:06:23.678294+0000","last_clean":"2024-08-31T06:06:23.678294+0000","last_became_active":"2024-08-31T06:04:25.893308+0000","last_became_peered":"2024-08-31T06:04:25.893308+0000","last_unstale":"2024-08-31T06:06:23.678294+0000","last_undegraded":"2024-08-31T06:06:23.678294+0000","last_fullsized":"2024-08-31T06:06:23.678294+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:26:48.451894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":38,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.043990+0000","last_change":"2024-08-31T06:04:22.843346+0000","last_active":"2024-08-31T06:05:25.043990+0000","last_peered":"2024-08-31T06:05:25.043990+0000","last_clean":"2024-08-31T06:05:25.043990+0000","last_became_active":"2024-08-31T06:04:22.842249+0000","last_became_peered":"2024-08-31T06:04:22.842249+0000","last_unstale":"2024-08-31T06:05:25.043990+0000","last_undegraded":"2024-08-31T06:05:25.043990+0000","last_fullsized":"2024-08-31T06:05:25.043990+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:40:57.618940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70369,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":802,"num_read_kb":479,"num_write":178,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1147,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":483328,"data_stored":604883,"data_compressed":17729,"data_compressed_allocated":24576,"data_compressed_original":368640,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":249373,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710763,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":436463,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874287,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":1094149,"data_compressed":42124,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037814,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":436677,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234043,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":436575,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397569,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":933888,"data_stored":1029200,"data_compressed":42124,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593799,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":436066,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790027,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":1028352,"data_compressed":42124,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953554,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36208,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708944,"statfs":{"total":95995035648,"available":95957958656,"internally_reserved":0,"allocated":884736,"data_stored":438189,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149781,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":888832,"data_stored":436377,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411420,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36268,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708884,"statfs":{"total":95995035648,"available":95957897216,"internally_reserved":0,"allocated":946176,"data_stored":502170,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542241,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36268,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708884,"statfs":{"total":95995035648,"available":95957897216,"internally_reserved":0,"allocated":946176,"data_stored":501666,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705767,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":892928,"data_stored":437337,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34699,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34699,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34699,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":65807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":66311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":159744,"data_stored":461581,"data_compressed":17729,"data_compressed_allocated":24576,"data_compressed_original":368640,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-31T06:06:54.130 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph pg dump --format=json 2024-08-31T06:06:54.522 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:06:54.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:54 smithi016 ceph-mon[17194]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T06:06:54.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:54 smithi016 ceph-mon[17194]: pgmap v353: 97 pgs: 97 active+clean; 1.0 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 101 KiB/s wr, 43 op/s 2024-08-31T06:06:55.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.148+0000 7fb3af400640 1 -- 172.21.15.16:0/344309872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3a8103920 msgr2=0x7fb3a8103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:55.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.148+0000 7fb3af400640 1 --2- 172.21.15.16:0/344309872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3a8103920 0x7fb3a8103d30 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb39c009900 tx=0x7fb39c02a410 comp rx=0 tx=0).stop 2024-08-31T06:06:55.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.148+0000 7fb3af400640 1 -- 172.21.15.16:0/344309872 shutdown_connections 2024-08-31T06:06:55.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.148+0000 7fb3af400640 1 --2- 172.21.15.16:0/344309872 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3a8109bb0 0x7fb3a81103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:55.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.148+0000 7fb3af400640 1 --2- 172.21.15.16:0/344309872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3a8109260 0x7fb3a8109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:55.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.148+0000 7fb3af400640 1 --2- 172.21.15.16:0/344309872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3a8103920 0x7fb3a8103d30 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:55.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.148+0000 7fb3af400640 1 -- 172.21.15.16:0/344309872 >> 172.21.15.16:0/344309872 conn(0x7fb3a80ff370 msgr2=0x7fb3a81017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:55.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.148+0000 7fb3af400640 1 -- 172.21.15.16:0/344309872 shutdown_connections 2024-08-31T06:06:55.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.148+0000 7fb3af400640 1 -- 172.21.15.16:0/344309872 wait complete. 2024-08-31T06:06:55.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.149+0000 7fb3af400640 1 Processor -- start 2024-08-31T06:06:55.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.149+0000 7fb3af400640 1 -- start start 2024-08-31T06:06:55.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.150+0000 7fb3af400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3a8103920 0x7fb3a819c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:55.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.150+0000 7fb3af400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3a8109260 0x7fb3a819caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:55.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.150+0000 7fb3af400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3a8109bb0 0x7fb3a81a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:55.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.150+0000 7fb3af400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a8113870 con 0x7fb3a8103920 2024-08-31T06:06:55.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.150+0000 7fb3af400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a81136f0 con 0x7fb3a8109260 2024-08-31T06:06:55.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.150+0000 7fb3af400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a81139f0 con 0x7fb3a8109bb0 2024-08-31T06:06:55.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.150+0000 7fb3aca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3a8103920 0x7fb3a819c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:55.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.150+0000 7fb3aca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3a8103920 0x7fb3a819c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35778/0 (socket says 172.21.15.16:35778) 2024-08-31T06:06:55.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.150+0000 7fb3a7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3a8109260 0x7fb3a819caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:55.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.150+0000 7fb3aca00640 1 -- 172.21.15.16:0/405909068 learned_addr learned my addr 172.21.15.16:0/405909068 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:55.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.150+0000 7fb3ad400640 1 --2- 172.21.15.16:0/405909068 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3a8109bb0 0x7fb3a81a1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:55.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.151+0000 7fb3aca00640 1 -- 172.21.15.16:0/405909068 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3a8109bb0 msgr2=0x7fb3a81a1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:55.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.151+0000 7fb3aca00640 1 --2- 172.21.15.16:0/405909068 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3a8109bb0 0x7fb3a81a1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:55.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.151+0000 7fb3aca00640 1 -- 172.21.15.16:0/405909068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3a8109260 msgr2=0x7fb3a819caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:55.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.151+0000 7fb3aca00640 1 --2- 172.21.15.16:0/405909068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3a8109260 0x7fb3a819caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:55.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.151+0000 7fb3aca00640 1 -- 172.21.15.16:0/405909068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3a81a2470 con 0x7fb3a8103920 2024-08-31T06:06:55.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.151+0000 7fb3aca00640 1 --2- 172.21.15.16:0/405909068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3a8103920 0x7fb3a819c500 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb39c005940 tx=0x7fb39c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:55.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.151+0000 7fb3a5600640 1 -- 172.21.15.16:0/405909068 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39c03f030 con 0x7fb3a8103920 2024-08-31T06:06:55.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.151+0000 7fb3a5600640 1 -- 172.21.15.16:0/405909068 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb39c003aa0 con 0x7fb3a8103920 2024-08-31T06:06:55.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.151+0000 7fb3a5600640 1 -- 172.21.15.16:0/405909068 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39c035070 con 0x7fb3a8103920 2024-08-31T06:06:55.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.152+0000 7fb3af400640 1 -- 172.21.15.16:0/405909068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3a81a2700 con 0x7fb3a8103920 2024-08-31T06:06:55.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.152+0000 7fb3af400640 1 -- 172.21.15.16:0/405909068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3a81a8ec0 con 0x7fb3a8103920 2024-08-31T06:06:55.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.153+0000 7fb3af400640 1 -- 172.21.15.16:0/405909068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3a8103d30 con 0x7fb3a8103920 2024-08-31T06:06:55.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.156+0000 7fb3a5600640 1 -- 172.21.15.16:0/405909068 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 152651+0+0 (secure 0 0 0) 0x7fb39c042030 con 0x7fb3a8103920 2024-08-31T06:06:55.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.157+0000 7fb3a5600640 1 --2- 172.21.15.16:0/405909068 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb3900b4c10 0x7fb3900b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:55.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.157+0000 7fb3a5600640 1 -- 172.21.15.16:0/405909068 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7fb39c105d40 con 0x7fb3a8103920 2024-08-31T06:06:55.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.158+0000 7fb3a7e00640 1 --2- 172.21.15.16:0/405909068 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb3900b4c10 0x7fb3900b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:55.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.158+0000 7fb3a7e00640 1 --2- 172.21.15.16:0/405909068 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb3900b4c10 0x7fb3900b7120 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8105850 tx=0x7fb394005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:55.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.159+0000 7fb3a5600640 1 -- 172.21.15.16:0/405909068 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fb39c0cc5a0 con 0x7fb3a8103920 2024-08-31T06:06:55.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:54 smithi184 ceph-mon[22331]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T06:06:55.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:54 smithi184 ceph-mon[22331]: pgmap v353: 97 pgs: 97 active+clean; 1.0 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 101 KiB/s wr, 43 op/s 2024-08-31T06:06:55.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.353+0000 7fb3af400640 1 -- 172.21.15.16:0/405909068 --> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb3a810f690 con 0x7fb3900b4c10 2024-08-31T06:06:55.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.359+0000 7fb3a5600640 1 -- 172.21.15.16:0/405909068 <== mgr.14220 v2:172.21.15.16:6800/4279751337 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259208 (secure 0 0 0) 0x7fb3a810f690 con 0x7fb3900b4c10 2024-08-31T06:06:55.364 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:06:55.365 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-08-31T06:06:55.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.363+0000 7fb3af400640 1 -- 172.21.15.16:0/405909068 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb3900b4c10 msgr2=0x7fb3900b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:55.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.363+0000 7fb3af400640 1 --2- 172.21.15.16:0/405909068 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb3900b4c10 0x7fb3900b7120 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8105850 tx=0x7fb394005f90 comp rx=0 tx=0).stop 2024-08-31T06:06:55.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.363+0000 7fb3af400640 1 -- 172.21.15.16:0/405909068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3a8103920 msgr2=0x7fb3a819c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:55.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.363+0000 7fb3af400640 1 --2- 172.21.15.16:0/405909068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3a8103920 0x7fb3a819c500 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb39c005940 tx=0x7fb39c004270 comp rx=0 tx=0).stop 2024-08-31T06:06:55.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.363+0000 7fb3af400640 1 -- 172.21.15.16:0/405909068 shutdown_connections 2024-08-31T06:06:55.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.363+0000 7fb3af400640 1 --2- 172.21.15.16:0/405909068 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb3900b4c10 0x7fb3900b7120 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:55.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.363+0000 7fb3af400640 1 --2- 172.21.15.16:0/405909068 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3a8109bb0 0x7fb3a81a1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:55.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.363+0000 7fb3af400640 1 --2- 172.21.15.16:0/405909068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3a8109260 0x7fb3a819caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:55.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.363+0000 7fb3af400640 1 --2- 172.21.15.16:0/405909068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3a8103920 0x7fb3a819c500 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:55.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.363+0000 7fb3af400640 1 -- 172.21.15.16:0/405909068 >> 172.21.15.16:0/405909068 conn(0x7fb3a80ff370 msgr2=0x7fb3a8108950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:55.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.363+0000 7fb3af400640 1 -- 172.21.15.16:0/405909068 shutdown_connections 2024-08-31T06:06:55.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:55.363+0000 7fb3af400640 1 -- 172.21.15.16:0/405909068 wait complete. 2024-08-31T06:06:55.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:54 smithi155 ceph-mon[21742]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T06:06:55.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:54 smithi155 ceph-mon[21742]: pgmap v353: 97 pgs: 97 active+clean; 1.0 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 101 KiB/s wr, 43 op/s 2024-08-31T06:06:56.139 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":354,"stamp":"2024-08-31T06:06:55.294326+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1447963,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1249,"num_read_kb":864,"num_write":527,"num_write_kb":4366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":2952987,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":888,"ondisk_log_size":888,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372684,"kb_used_data":10944,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124569140,"statfs":{"total":1151940427776,"available":1151558799360,"internally_reserved":0,"allocated":11206656,"data_stored":9574197,"data_compressed":290757,"data_compressed_allocated":380928,"data_compressed_original":4251648,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":786992,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":426,"num_read_kb":346,"num_write":224,"num_write_kb":2088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003140"},"pg_stats":[{"pgid":"3.3f","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.686021+0000","last_change":"2024-08-31T06:04:25.877318+0000","last_active":"2024-08-31T06:06:23.686021+0000","last_peered":"2024-08-31T06:06:23.686021+0000","last_clean":"2024-08-31T06:06:23.686021+0000","last_became_active":"2024-08-31T06:04:25.876894+0000","last_became_peered":"2024-08-31T06:04:25.876894+0000","last_unstale":"2024-08-31T06:06:23.686021+0000","last_undegraded":"2024-08-31T06:06:23.686021+0000","last_fullsized":"2024-08-31T06:06:23.686021+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:41:09.449343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"80'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.685244+0000","last_change":"2024-08-31T06:04:25.885670+0000","last_active":"2024-08-31T06:06:23.685244+0000","last_peered":"2024-08-31T06:06:23.685244+0000","last_clean":"2024-08-31T06:06:23.685244+0000","last_became_active":"2024-08-31T06:04:25.883818+0000","last_became_peered":"2024-08-31T06:04:25.883818+0000","last_unstale":"2024-08-31T06:06:23.685244+0000","last_undegraded":"2024-08-31T06:06:23.685244+0000","last_fullsized":"2024-08-31T06:06:23.685244+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:40:01.344432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.687803+0000","last_change":"2024-08-31T06:04:25.883375+0000","last_active":"2024-08-31T06:06:23.687803+0000","last_peered":"2024-08-31T06:06:23.687803+0000","last_clean":"2024-08-31T06:06:23.687803+0000","last_became_active":"2024-08-31T06:04:25.881750+0000","last_became_peered":"2024-08-31T06:04:25.881750+0000","last_unstale":"2024-08-31T06:06:23.687803+0000","last_undegraded":"2024-08-31T06:06:23.687803+0000","last_fullsized":"2024-08-31T06:06:23.687803+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:46:46.757363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.676625+0000","last_change":"2024-08-31T06:04:25.882436+0000","last_active":"2024-08-31T06:06:23.676625+0000","last_peered":"2024-08-31T06:06:23.676625+0000","last_clean":"2024-08-31T06:06:23.676625+0000","last_became_active":"2024-08-31T06:04:25.881805+0000","last_became_peered":"2024-08-31T06:04:25.881805+0000","last_unstale":"2024-08-31T06:06:23.676625+0000","last_undegraded":"2024-08-31T06:06:23.676625+0000","last_fullsized":"2024-08-31T06:06:23.676625+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:42:39.793625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"80'4","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.661470+0000","last_change":"2024-08-31T06:04:25.894534+0000","last_active":"2024-08-31T06:06:23.661470+0000","last_peered":"2024-08-31T06:06:23.661470+0000","last_clean":"2024-08-31T06:06:23.661470+0000","last_became_active":"2024-08-31T06:04:25.894116+0000","last_became_peered":"2024-08-31T06:04:25.894116+0000","last_unstale":"2024-08-31T06:06:23.661470+0000","last_undegraded":"2024-08-31T06:06:23.661470+0000","last_fullsized":"2024-08-31T06:06:23.661470+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:02:31.257454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.669411+0000","last_change":"2024-08-31T06:04:25.886182+0000","last_active":"2024-08-31T06:06:23.669411+0000","last_peered":"2024-08-31T06:06:23.669411+0000","last_clean":"2024-08-31T06:06:23.669411+0000","last_became_active":"2024-08-31T06:04:25.885825+0000","last_became_peered":"2024-08-31T06:04:25.885825+0000","last_unstale":"2024-08-31T06:06:23.669411+0000","last_undegraded":"2024-08-31T06:06:23.669411+0000","last_fullsized":"2024-08-31T06:06:23.669411+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:58:12.427931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.682008+0000","last_change":"2024-08-31T06:04:25.877521+0000","last_active":"2024-08-31T06:06:23.682008+0000","last_peered":"2024-08-31T06:06:23.682008+0000","last_clean":"2024-08-31T06:06:23.682008+0000","last_became_active":"2024-08-31T06:04:25.877122+0000","last_became_peered":"2024-08-31T06:04:25.877122+0000","last_unstale":"2024-08-31T06:06:23.682008+0000","last_undegraded":"2024-08-31T06:06:23.682008+0000","last_fullsized":"2024-08-31T06:06:23.682008+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:17:27.406828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.680720+0000","last_change":"2024-08-31T06:04:25.892547+0000","last_active":"2024-08-31T06:06:23.680720+0000","last_peered":"2024-08-31T06:06:23.680720+0000","last_clean":"2024-08-31T06:06:23.680720+0000","last_became_active":"2024-08-31T06:04:25.892090+0000","last_became_peered":"2024-08-31T06:04:25.892090+0000","last_unstale":"2024-08-31T06:06:23.680720+0000","last_undegraded":"2024-08-31T06:06:23.680720+0000","last_fullsized":"2024-08-31T06:06:23.680720+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:28:53.414252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"80'11","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.689423+0000","last_change":"2024-08-31T06:04:25.879952+0000","last_active":"2024-08-31T06:06:23.689423+0000","last_peered":"2024-08-31T06:06:23.689423+0000","last_clean":"2024-08-31T06:06:23.689423+0000","last_became_active":"2024-08-31T06:04:25.879568+0000","last_became_peered":"2024-08-31T06:04:25.879568+0000","last_unstale":"2024-08-31T06:06:23.689423+0000","last_undegraded":"2024-08-31T06:06:23.689423+0000","last_fullsized":"2024-08-31T06:06:23.689423+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:28:04.558054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"80'16","reported_seq":45,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.677734+0000","last_change":"2024-08-31T06:05:25.051173+0000","last_active":"2024-08-31T06:06:23.677734+0000","last_peered":"2024-08-31T06:06:23.677734+0000","last_clean":"2024-08-31T06:06:23.677734+0000","last_became_active":"2024-08-31T06:05:25.050867+0000","last_became_peered":"2024-08-31T06:05:25.050867+0000","last_unstale":"2024-08-31T06:06:23.677734+0000","last_undegraded":"2024-08-31T06:06:23.677734+0000","last_fullsized":"2024-08-31T06:06:23.677734+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:27:09.971497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.686612+0000","last_change":"2024-08-31T06:04:25.883984+0000","last_active":"2024-08-31T06:06:23.686612+0000","last_peered":"2024-08-31T06:06:23.686612+0000","last_clean":"2024-08-31T06:06:23.686612+0000","last_became_active":"2024-08-31T06:04:25.883353+0000","last_became_peered":"2024-08-31T06:04:25.883353+0000","last_unstale":"2024-08-31T06:06:23.686612+0000","last_undegraded":"2024-08-31T06:06:23.686612+0000","last_fullsized":"2024-08-31T06:06:23.686612+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:16:25.178952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.673031+0000","last_change":"2024-08-31T06:04:25.883335+0000","last_active":"2024-08-31T06:06:23.673031+0000","last_peered":"2024-08-31T06:06:23.673031+0000","last_clean":"2024-08-31T06:06:23.673031+0000","last_became_active":"2024-08-31T06:04:25.881488+0000","last_became_peered":"2024-08-31T06:04:25.881488+0000","last_unstale":"2024-08-31T06:06:23.673031+0000","last_undegraded":"2024-08-31T06:06:23.673031+0000","last_fullsized":"2024-08-31T06:06:23.673031+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:57:24.472482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'10","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.678797+0000","last_change":"2024-08-31T06:04:25.891579+0000","last_active":"2024-08-31T06:06:23.678797+0000","last_peered":"2024-08-31T06:06:23.678797+0000","last_clean":"2024-08-31T06:06:23.678797+0000","last_became_active":"2024-08-31T06:04:25.887950+0000","last_became_peered":"2024-08-31T06:04:25.887950+0000","last_unstale":"2024-08-31T06:06:23.678797+0000","last_undegraded":"2024-08-31T06:06:23.678797+0000","last_fullsized":"2024-08-31T06:06:23.678797+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:38:35.284442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.682430+0000","last_change":"2024-08-31T06:04:25.882185+0000","last_active":"2024-08-31T06:06:23.682430+0000","last_peered":"2024-08-31T06:06:23.682430+0000","last_clean":"2024-08-31T06:06:23.682430+0000","last_became_active":"2024-08-31T06:04:25.876499+0000","last_became_peered":"2024-08-31T06:04:25.876499+0000","last_unstale":"2024-08-31T06:06:23.682430+0000","last_undegraded":"2024-08-31T06:06:23.682430+0000","last_fullsized":"2024-08-31T06:06:23.682430+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:40:01.132353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.683904+0000","last_change":"2024-08-31T06:04:25.883624+0000","last_active":"2024-08-31T06:06:23.683904+0000","last_peered":"2024-08-31T06:06:23.683904+0000","last_clean":"2024-08-31T06:06:23.683904+0000","last_became_active":"2024-08-31T06:04:25.878041+0000","last_became_peered":"2024-08-31T06:04:25.878041+0000","last_unstale":"2024-08-31T06:06:23.683904+0000","last_undegraded":"2024-08-31T06:06:23.683904+0000","last_fullsized":"2024-08-31T06:06:23.683904+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:05:21.278321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.690513+0000","last_change":"2024-08-31T06:04:25.882538+0000","last_active":"2024-08-31T06:06:23.690513+0000","last_peered":"2024-08-31T06:06:23.690513+0000","last_clean":"2024-08-31T06:06:23.690513+0000","last_became_active":"2024-08-31T06:04:25.882201+0000","last_became_peered":"2024-08-31T06:04:25.882201+0000","last_unstale":"2024-08-31T06:06:23.690513+0000","last_undegraded":"2024-08-31T06:06:23.690513+0000","last_fullsized":"2024-08-31T06:06:23.690513+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:47:16.880893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.686157+0000","last_change":"2024-08-31T06:04:25.880376+0000","last_active":"2024-08-31T06:06:23.686157+0000","last_peered":"2024-08-31T06:06:23.686157+0000","last_clean":"2024-08-31T06:06:23.686157+0000","last_became_active":"2024-08-31T06:04:25.877668+0000","last_became_peered":"2024-08-31T06:04:25.877668+0000","last_unstale":"2024-08-31T06:06:23.686157+0000","last_undegraded":"2024-08-31T06:06:23.686157+0000","last_fullsized":"2024-08-31T06:06:23.686157+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:55:32.737406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"80'10","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.673590+0000","last_change":"2024-08-31T06:04:25.875431+0000","last_active":"2024-08-31T06:06:23.673590+0000","last_peered":"2024-08-31T06:06:23.673590+0000","last_clean":"2024-08-31T06:06:23.673590+0000","last_became_active":"2024-08-31T06:04:25.874751+0000","last_became_peered":"2024-08-31T06:04:25.874751+0000","last_unstale":"2024-08-31T06:06:23.673590+0000","last_undegraded":"2024-08-31T06:06:23.673590+0000","last_fullsized":"2024-08-31T06:06:23.673590+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:16:57.695415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.675738+0000","last_change":"2024-08-31T06:04:25.895656+0000","last_active":"2024-08-31T06:06:23.675738+0000","last_peered":"2024-08-31T06:06:23.675738+0000","last_clean":"2024-08-31T06:06:23.675738+0000","last_became_active":"2024-08-31T06:04:25.895272+0000","last_became_peered":"2024-08-31T06:04:25.895272+0000","last_unstale":"2024-08-31T06:06:23.675738+0000","last_undegraded":"2024-08-31T06:06:23.675738+0000","last_fullsized":"2024-08-31T06:06:23.675738+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:18:47.276195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.679697+0000","last_change":"2024-08-31T06:04:25.877283+0000","last_active":"2024-08-31T06:06:23.679697+0000","last_peered":"2024-08-31T06:06:23.679697+0000","last_clean":"2024-08-31T06:06:23.679697+0000","last_became_active":"2024-08-31T06:04:25.876846+0000","last_became_peered":"2024-08-31T06:04:25.876846+0000","last_unstale":"2024-08-31T06:06:23.679697+0000","last_undegraded":"2024-08-31T06:06:23.679697+0000","last_fullsized":"2024-08-31T06:06:23.679697+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:54:05.086844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"80'11","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.685368+0000","last_change":"2024-08-31T06:04:25.884891+0000","last_active":"2024-08-31T06:06:23.685368+0000","last_peered":"2024-08-31T06:06:23.685368+0000","last_clean":"2024-08-31T06:06:23.685368+0000","last_became_active":"2024-08-31T06:04:25.884501+0000","last_became_peered":"2024-08-31T06:04:25.884501+0000","last_unstale":"2024-08-31T06:06:23.685368+0000","last_undegraded":"2024-08-31T06:06:23.685368+0000","last_fullsized":"2024-08-31T06:06:23.685368+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:22:54.846102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"80'10","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.691670+0000","last_change":"2024-08-31T06:04:25.877150+0000","last_active":"2024-08-31T06:06:23.691670+0000","last_peered":"2024-08-31T06:06:23.691670+0000","last_clean":"2024-08-31T06:06:23.691670+0000","last_became_active":"2024-08-31T06:04:25.876766+0000","last_became_peered":"2024-08-31T06:04:25.876766+0000","last_unstale":"2024-08-31T06:06:23.691670+0000","last_undegraded":"2024-08-31T06:06:23.691670+0000","last_fullsized":"2024-08-31T06:06:23.691670+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:46:26.724970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.676164+0000","last_change":"2024-08-31T06:04:25.881275+0000","last_active":"2024-08-31T06:06:23.676164+0000","last_peered":"2024-08-31T06:06:23.676164+0000","last_clean":"2024-08-31T06:06:23.676164+0000","last_became_active":"2024-08-31T06:04:25.879877+0000","last_became_peered":"2024-08-31T06:04:25.879877+0000","last_unstale":"2024-08-31T06:06:23.676164+0000","last_undegraded":"2024-08-31T06:06:23.676164+0000","last_fullsized":"2024-08-31T06:06:23.676164+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:07:07.237041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.676080+0000","last_change":"2024-08-31T06:04:25.895750+0000","last_active":"2024-08-31T06:06:23.676080+0000","last_peered":"2024-08-31T06:06:23.676080+0000","last_clean":"2024-08-31T06:06:23.676080+0000","last_became_active":"2024-08-31T06:04:25.895348+0000","last_became_peered":"2024-08-31T06:04:25.895348+0000","last_unstale":"2024-08-31T06:06:23.676080+0000","last_undegraded":"2024-08-31T06:06:23.676080+0000","last_fullsized":"2024-08-31T06:06:23.676080+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:53:10.014808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.670437+0000","last_change":"2024-08-31T06:04:25.895537+0000","last_active":"2024-08-31T06:06:23.670437+0000","last_peered":"2024-08-31T06:06:23.670437+0000","last_clean":"2024-08-31T06:06:23.670437+0000","last_became_active":"2024-08-31T06:04:25.895136+0000","last_became_peered":"2024-08-31T06:04:25.895136+0000","last_unstale":"2024-08-31T06:06:23.670437+0000","last_undegraded":"2024-08-31T06:06:23.670437+0000","last_fullsized":"2024-08-31T06:06:23.670437+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:41:57.554507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"80'15","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.676301+0000","last_change":"2024-08-31T06:04:25.880490+0000","last_active":"2024-08-31T06:06:23.676301+0000","last_peered":"2024-08-31T06:06:23.676301+0000","last_clean":"2024-08-31T06:06:23.676301+0000","last_became_active":"2024-08-31T06:04:25.879269+0000","last_became_peered":"2024-08-31T06:04:25.879269+0000","last_unstale":"2024-08-31T06:06:23.676301+0000","last_undegraded":"2024-08-31T06:06:23.676301+0000","last_fullsized":"2024-08-31T06:06:23.676301+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:05:49.575915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"80'6","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.676526+0000","last_change":"2024-08-31T06:05:25.053714+0000","last_active":"2024-08-31T06:06:23.676526+0000","last_peered":"2024-08-31T06:06:23.676526+0000","last_clean":"2024-08-31T06:06:23.676526+0000","last_became_active":"2024-08-31T06:05:25.052685+0000","last_became_peered":"2024-08-31T06:05:25.052685+0000","last_unstale":"2024-08-31T06:06:23.676526+0000","last_undegraded":"2024-08-31T06:06:23.676526+0000","last_fullsized":"2024-08-31T06:06:23.676526+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:17:31.474601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"80'4","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.684823+0000","last_change":"2024-08-31T06:05:25.057946+0000","last_active":"2024-08-31T06:06:23.684823+0000","last_peered":"2024-08-31T06:06:23.684823+0000","last_clean":"2024-08-31T06:06:23.684823+0000","last_became_active":"2024-08-31T06:05:25.057501+0000","last_became_peered":"2024-08-31T06:05:25.057501+0000","last_unstale":"2024-08-31T06:06:23.684823+0000","last_undegraded":"2024-08-31T06:06:23.684823+0000","last_fullsized":"2024-08-31T06:06:23.684823+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:16:05.721770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.685410+0000","last_change":"2024-08-31T06:04:25.897473+0000","last_active":"2024-08-31T06:06:23.685410+0000","last_peered":"2024-08-31T06:06:23.685410+0000","last_clean":"2024-08-31T06:06:23.685410+0000","last_became_active":"2024-08-31T06:04:25.895194+0000","last_became_peered":"2024-08-31T06:04:25.895194+0000","last_unstale":"2024-08-31T06:06:23.685410+0000","last_undegraded":"2024-08-31T06:06:23.685410+0000","last_fullsized":"2024-08-31T06:06:23.685410+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:43:04.539970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.684923+0000","last_change":"2024-08-31T06:04:25.894402+0000","last_active":"2024-08-31T06:06:23.684923+0000","last_peered":"2024-08-31T06:06:23.684923+0000","last_clean":"2024-08-31T06:06:23.684923+0000","last_became_active":"2024-08-31T06:04:25.893883+0000","last_became_peered":"2024-08-31T06:04:25.893883+0000","last_unstale":"2024-08-31T06:06:23.684923+0000","last_undegraded":"2024-08-31T06:06:23.684923+0000","last_fullsized":"2024-08-31T06:06:23.684923+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:45:53.778473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"80'12","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.681979+0000","last_change":"2024-08-31T06:04:25.881543+0000","last_active":"2024-08-31T06:06:23.681979+0000","last_peered":"2024-08-31T06:06:23.681979+0000","last_clean":"2024-08-31T06:06:23.681979+0000","last_became_active":"2024-08-31T06:04:25.881144+0000","last_became_peered":"2024-08-31T06:04:25.881144+0000","last_unstale":"2024-08-31T06:06:23.681979+0000","last_undegraded":"2024-08-31T06:06:23.681979+0000","last_fullsized":"2024-08-31T06:06:23.681979+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:13:02.282879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.669781+0000","last_change":"2024-08-31T06:04:25.881143+0000","last_active":"2024-08-31T06:06:23.669781+0000","last_peered":"2024-08-31T06:06:23.669781+0000","last_clean":"2024-08-31T06:06:23.669781+0000","last_became_active":"2024-08-31T06:04:25.879718+0000","last_became_peered":"2024-08-31T06:04:25.879718+0000","last_unstale":"2024-08-31T06:06:23.669781+0000","last_undegraded":"2024-08-31T06:06:23.669781+0000","last_fullsized":"2024-08-31T06:06:23.669781+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:32:07.771488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.680189+0000","last_change":"2024-08-31T06:04:25.879493+0000","last_active":"2024-08-31T06:06:23.680189+0000","last_peered":"2024-08-31T06:06:23.680189+0000","last_clean":"2024-08-31T06:06:23.680189+0000","last_became_active":"2024-08-31T06:04:25.879065+0000","last_became_peered":"2024-08-31T06:04:25.879065+0000","last_unstale":"2024-08-31T06:06:23.680189+0000","last_undegraded":"2024-08-31T06:06:23.680189+0000","last_fullsized":"2024-08-31T06:06:23.680189+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:53:43.848269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"80'7","reported_seq":184,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:53.188018+0000","last_change":"2024-08-31T06:05:25.058954+0000","last_active":"2024-08-31T06:06:53.188018+0000","last_peered":"2024-08-31T06:06:53.188018+0000","last_clean":"2024-08-31T06:06:53.188018+0000","last_became_active":"2024-08-31T06:05:25.051143+0000","last_became_peered":"2024-08-31T06:05:25.051143+0000","last_unstale":"2024-08-31T06:06:53.188018+0000","last_undegraded":"2024-08-31T06:06:53.188018+0000","last_fullsized":"2024-08-31T06:06:53.188018+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:38:23.533663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":132,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.682132+0000","last_change":"2024-08-31T06:04:25.896354+0000","last_active":"2024-08-31T06:06:23.682132+0000","last_peered":"2024-08-31T06:06:23.682132+0000","last_clean":"2024-08-31T06:06:23.682132+0000","last_became_active":"2024-08-31T06:04:25.896021+0000","last_became_peered":"2024-08-31T06:04:25.896021+0000","last_unstale":"2024-08-31T06:06:23.682132+0000","last_undegraded":"2024-08-31T06:06:23.682132+0000","last_fullsized":"2024-08-31T06:06:23.682132+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:08:40.213238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.047221+0000","last_change":"2024-08-31T06:04:22.844248+0000","last_active":"2024-08-31T06:05:25.047221+0000","last_peered":"2024-08-31T06:05:25.047221+0000","last_clean":"2024-08-31T06:05:25.047221+0000","last_became_active":"2024-08-31T06:04:22.843784+0000","last_became_peered":"2024-08-31T06:04:22.843784+0000","last_unstale":"2024-08-31T06:05:25.047221+0000","last_undegraded":"2024-08-31T06:05:25.047221+0000","last_fullsized":"2024-08-31T06:05:25.047221+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:56:41.422965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.046268+0000","last_change":"2024-08-31T06:04:22.847049+0000","last_active":"2024-08-31T06:05:25.046268+0000","last_peered":"2024-08-31T06:05:25.046268+0000","last_clean":"2024-08-31T06:05:25.046268+0000","last_became_active":"2024-08-31T06:04:22.846282+0000","last_became_peered":"2024-08-31T06:04:22.846282+0000","last_unstale":"2024-08-31T06:05:25.046268+0000","last_undegraded":"2024-08-31T06:05:25.046268+0000","last_fullsized":"2024-08-31T06:05:25.046268+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:35:21.030425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.679608+0000","last_change":"2024-08-31T06:04:25.886698+0000","last_active":"2024-08-31T06:06:23.679608+0000","last_peered":"2024-08-31T06:06:23.679608+0000","last_clean":"2024-08-31T06:06:23.679608+0000","last_became_active":"2024-08-31T06:04:25.886334+0000","last_became_peered":"2024-08-31T06:04:25.886334+0000","last_unstale":"2024-08-31T06:06:23.679608+0000","last_undegraded":"2024-08-31T06:06:23.679608+0000","last_fullsized":"2024-08-31T06:06:23.679608+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:24:43.821446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.044900+0000","last_change":"2024-08-31T06:04:22.845263+0000","last_active":"2024-08-31T06:05:25.044900+0000","last_peered":"2024-08-31T06:05:25.044900+0000","last_clean":"2024-08-31T06:05:25.044900+0000","last_became_active":"2024-08-31T06:04:22.844853+0000","last_became_peered":"2024-08-31T06:04:22.844853+0000","last_unstale":"2024-08-31T06:05:25.044900+0000","last_undegraded":"2024-08-31T06:05:25.044900+0000","last_fullsized":"2024-08-31T06:05:25.044900+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:55:12.561165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"80'11","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.673457+0000","last_change":"2024-08-31T06:04:25.894475+0000","last_active":"2024-08-31T06:06:23.673457+0000","last_peered":"2024-08-31T06:06:23.673457+0000","last_clean":"2024-08-31T06:06:23.673457+0000","last_became_active":"2024-08-31T06:04:25.894121+0000","last_became_peered":"2024-08-31T06:04:25.894121+0000","last_unstale":"2024-08-31T06:06:23.673457+0000","last_undegraded":"2024-08-31T06:06:23.673457+0000","last_fullsized":"2024-08-31T06:06:23.673457+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:40:06.570616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":165,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:53.187549+0000","last_change":"2024-08-31T06:04:22.834430+0000","last_active":"2024-08-31T06:06:53.187549+0000","last_peered":"2024-08-31T06:06:53.187549+0000","last_clean":"2024-08-31T06:06:53.187549+0000","last_became_active":"2024-08-31T06:04:22.834011+0000","last_became_peered":"2024-08-31T06:04:22.834011+0000","last_unstale":"2024-08-31T06:06:53.187549+0000","last_undegraded":"2024-08-31T06:06:53.187549+0000","last_fullsized":"2024-08-31T06:06:53.187549+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:51:16.734234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":134,"num_read_kb":134,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"80'10","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.689788+0000","last_change":"2024-08-31T06:04:25.877727+0000","last_active":"2024-08-31T06:06:23.689788+0000","last_peered":"2024-08-31T06:06:23.689788+0000","last_clean":"2024-08-31T06:06:23.689788+0000","last_became_active":"2024-08-31T06:04:25.877121+0000","last_became_peered":"2024-08-31T06:04:25.877121+0000","last_unstale":"2024-08-31T06:06:23.689788+0000","last_undegraded":"2024-08-31T06:06:23.689788+0000","last_fullsized":"2024-08-31T06:06:23.689788+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:41:12.247216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048879+0000","last_change":"2024-08-31T06:04:22.846482+0000","last_active":"2024-08-31T06:05:25.048879+0000","last_peered":"2024-08-31T06:05:25.048879+0000","last_clean":"2024-08-31T06:05:25.048879+0000","last_became_active":"2024-08-31T06:04:22.846057+0000","last_became_peered":"2024-08-31T06:04:22.846057+0000","last_unstale":"2024-08-31T06:05:25.048879+0000","last_undegraded":"2024-08-31T06:05:25.048879+0000","last_fullsized":"2024-08-31T06:05:25.048879+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:29:11.861665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.684568+0000","last_change":"2024-08-31T06:04:25.893777+0000","last_active":"2024-08-31T06:06:23.684568+0000","last_peered":"2024-08-31T06:06:23.684568+0000","last_clean":"2024-08-31T06:06:23.684568+0000","last_became_active":"2024-08-31T06:04:25.893014+0000","last_became_peered":"2024-08-31T06:04:25.893014+0000","last_unstale":"2024-08-31T06:06:23.684568+0000","last_undegraded":"2024-08-31T06:06:23.684568+0000","last_fullsized":"2024-08-31T06:06:23.684568+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:04:48.185842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.045551+0000","last_change":"2024-08-31T06:04:22.965938+0000","last_active":"2024-08-31T06:05:25.045551+0000","last_peered":"2024-08-31T06:05:25.045551+0000","last_clean":"2024-08-31T06:05:25.045551+0000","last_became_active":"2024-08-31T06:04:22.965356+0000","last_became_peered":"2024-08-31T06:04:22.965356+0000","last_unstale":"2024-08-31T06:05:25.045551+0000","last_undegraded":"2024-08-31T06:05:25.045551+0000","last_fullsized":"2024-08-31T06:05:25.045551+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:36:58.938462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"80'11","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.691850+0000","last_change":"2024-08-31T06:04:25.881307+0000","last_active":"2024-08-31T06:06:23.691850+0000","last_peered":"2024-08-31T06:06:23.691850+0000","last_clean":"2024-08-31T06:06:23.691850+0000","last_became_active":"2024-08-31T06:04:25.880418+0000","last_became_peered":"2024-08-31T06:04:25.880418+0000","last_unstale":"2024-08-31T06:06:23.691850+0000","last_undegraded":"2024-08-31T06:06:23.691850+0000","last_fullsized":"2024-08-31T06:06:23.691850+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:21:46.904371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'34","reported_seq":342,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:54.188643+0000","last_change":"2024-08-31T06:04:22.843073+0000","last_active":"2024-08-31T06:06:54.188643+0000","last_peered":"2024-08-31T06:06:54.188643+0000","last_clean":"2024-08-31T06:06:54.188643+0000","last_became_active":"2024-08-31T06:04:22.842645+0000","last_became_peered":"2024-08-31T06:04:22.842645+0000","last_unstale":"2024-08-31T06:06:54.188643+0000","last_undegraded":"2024-08-31T06:06:54.188643+0000","last_fullsized":"2024-08-31T06:06:54.188643+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:41:37.651562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65695,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":281,"num_read_kb":205,"num_write":35,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.680627+0000","last_change":"2024-08-31T06:04:25.879929+0000","last_active":"2024-08-31T06:06:23.680627+0000","last_peered":"2024-08-31T06:06:23.680627+0000","last_clean":"2024-08-31T06:06:23.680627+0000","last_became_active":"2024-08-31T06:04:25.878475+0000","last_became_peered":"2024-08-31T06:04:25.878475+0000","last_unstale":"2024-08-31T06:06:23.680627+0000","last_undegraded":"2024-08-31T06:06:23.680627+0000","last_fullsized":"2024-08-31T06:06:23.680627+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:10:32.801558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048264+0000","last_change":"2024-08-31T06:04:22.842878+0000","last_active":"2024-08-31T06:05:25.048264+0000","last_peered":"2024-08-31T06:05:25.048264+0000","last_clean":"2024-08-31T06:05:25.048264+0000","last_became_active":"2024-08-31T06:04:22.842306+0000","last_became_peered":"2024-08-31T06:04:22.842306+0000","last_unstale":"2024-08-31T06:05:25.048264+0000","last_undegraded":"2024-08-31T06:05:25.048264+0000","last_fullsized":"2024-08-31T06:05:25.048264+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:53:55.259238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.669929+0000","last_change":"2024-08-31T06:04:25.899357+0000","last_active":"2024-08-31T06:06:23.669929+0000","last_peered":"2024-08-31T06:06:23.669929+0000","last_clean":"2024-08-31T06:06:23.669929+0000","last_became_active":"2024-08-31T06:04:25.899000+0000","last_became_peered":"2024-08-31T06:04:25.899000+0000","last_unstale":"2024-08-31T06:06:23.669929+0000","last_undegraded":"2024-08-31T06:06:23.669929+0000","last_fullsized":"2024-08-31T06:06:23.669929+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:20:39.991071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.044759+0000","last_change":"2024-08-31T06:04:22.843063+0000","last_active":"2024-08-31T06:05:25.044759+0000","last_peered":"2024-08-31T06:05:25.044759+0000","last_clean":"2024-08-31T06:05:25.044759+0000","last_became_active":"2024-08-31T06:04:22.842654+0000","last_became_peered":"2024-08-31T06:04:22.842654+0000","last_unstale":"2024-08-31T06:05:25.044759+0000","last_undegraded":"2024-08-31T06:05:25.044759+0000","last_fullsized":"2024-08-31T06:05:25.044759+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:09:54.563116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"80'15","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.683856+0000","last_change":"2024-08-31T06:04:25.885556+0000","last_active":"2024-08-31T06:06:23.683856+0000","last_peered":"2024-08-31T06:06:23.683856+0000","last_clean":"2024-08-31T06:06:23.683856+0000","last_became_active":"2024-08-31T06:04:25.885216+0000","last_became_peered":"2024-08-31T06:04:25.885216+0000","last_unstale":"2024-08-31T06:06:23.683856+0000","last_undegraded":"2024-08-31T06:06:23.683856+0000","last_fullsized":"2024-08-31T06:06:23.683856+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:48:49.816261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'5","reported_seq":170,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:52.747883+0000","last_change":"2024-08-31T06:04:22.844839+0000","last_active":"2024-08-31T06:06:52.747883+0000","last_peered":"2024-08-31T06:06:52.747883+0000","last_clean":"2024-08-31T06:06:52.747883+0000","last_became_active":"2024-08-31T06:04:22.844425+0000","last_became_peered":"2024-08-31T06:04:22.844425+0000","last_unstale":"2024-08-31T06:06:52.747883+0000","last_undegraded":"2024-08-31T06:06:52.747883+0000","last_fullsized":"2024-08-31T06:06:52.747883+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:10:57.292173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":139,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.682394+0000","last_change":"2024-08-31T06:04:25.878195+0000","last_active":"2024-08-31T06:06:23.682394+0000","last_peered":"2024-08-31T06:06:23.682394+0000","last_clean":"2024-08-31T06:06:23.682394+0000","last_became_active":"2024-08-31T06:04:25.877823+0000","last_became_peered":"2024-08-31T06:04:25.877823+0000","last_unstale":"2024-08-31T06:06:23.682394+0000","last_undegraded":"2024-08-31T06:06:23.682394+0000","last_fullsized":"2024-08-31T06:06:23.682394+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:54:34.989583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":68,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.071826+0000","last_change":"2024-08-31T06:05:25.071391+0000","last_active":"2024-08-31T06:05:25.071826+0000","last_peered":"2024-08-31T06:05:25.071826+0000","last_clean":"2024-08-31T06:05:25.071826+0000","last_became_active":"2024-08-31T06:05:25.057546+0000","last_became_peered":"2024-08-31T06:05:25.057546+0000","last_unstale":"2024-08-31T06:05:25.071826+0000","last_undegraded":"2024-08-31T06:05:25.071826+0000","last_fullsized":"2024-08-31T06:05:25.071826+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:42:35.527395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.681995+0000","last_change":"2024-08-31T06:04:25.887580+0000","last_active":"2024-08-31T06:06:23.681995+0000","last_peered":"2024-08-31T06:06:23.681995+0000","last_clean":"2024-08-31T06:06:23.681995+0000","last_became_active":"2024-08-31T06:04:25.887158+0000","last_became_peered":"2024-08-31T06:04:25.887158+0000","last_unstale":"2024-08-31T06:06:23.681995+0000","last_undegraded":"2024-08-31T06:06:23.681995+0000","last_fullsized":"2024-08-31T06:06:23.681995+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:27:38.115742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.064443+0000","last_change":"2024-08-31T06:05:25.063859+0000","last_active":"2024-08-31T06:05:25.064443+0000","last_peered":"2024-08-31T06:05:25.064443+0000","last_clean":"2024-08-31T06:05:25.064443+0000","last_became_active":"2024-08-31T06:05:25.053596+0000","last_became_peered":"2024-08-31T06:05:25.053596+0000","last_unstale":"2024-08-31T06:05:25.064443+0000","last_undegraded":"2024-08-31T06:05:25.064443+0000","last_fullsized":"2024-08-31T06:05:25.064443+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:19:18.461630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.678982+0000","last_change":"2024-08-31T06:04:25.880845+0000","last_active":"2024-08-31T06:06:23.678982+0000","last_peered":"2024-08-31T06:06:23.678982+0000","last_clean":"2024-08-31T06:06:23.678982+0000","last_became_active":"2024-08-31T06:04:25.880423+0000","last_became_peered":"2024-08-31T06:04:25.880423+0000","last_unstale":"2024-08-31T06:06:23.678982+0000","last_undegraded":"2024-08-31T06:06:23.678982+0000","last_fullsized":"2024-08-31T06:06:23.678982+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:16:56.725116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.046584+0000","last_change":"2024-08-31T06:04:22.843393+0000","last_active":"2024-08-31T06:05:25.046584+0000","last_peered":"2024-08-31T06:05:25.046584+0000","last_clean":"2024-08-31T06:05:25.046584+0000","last_became_active":"2024-08-31T06:04:22.842770+0000","last_became_peered":"2024-08-31T06:04:22.842770+0000","last_unstale":"2024-08-31T06:05:25.046584+0000","last_undegraded":"2024-08-31T06:05:25.046584+0000","last_fullsized":"2024-08-31T06:05:25.046584+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:49:16.159167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.686211+0000","last_change":"2024-08-31T06:04:25.887829+0000","last_active":"2024-08-31T06:06:23.686211+0000","last_peered":"2024-08-31T06:06:23.686211+0000","last_clean":"2024-08-31T06:06:23.686211+0000","last_became_active":"2024-08-31T06:04:25.886567+0000","last_became_peered":"2024-08-31T06:04:25.886567+0000","last_unstale":"2024-08-31T06:06:23.686211+0000","last_undegraded":"2024-08-31T06:06:23.686211+0000","last_fullsized":"2024-08-31T06:06:23.686211+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:11:39.054966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'60","reported_seq":117,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:50.541212+0000","last_change":"2024-08-31T06:04:22.844218+0000","last_active":"2024-08-31T06:06:50.541212+0000","last_peered":"2024-08-31T06:06:50.541212+0000","last_clean":"2024-08-31T06:06:50.541212+0000","last_became_active":"2024-08-31T06:04:22.843786+0000","last_became_peered":"2024-08-31T06:04:22.843786+0000","last_unstale":"2024-08-31T06:06:50.541212+0000","last_undegraded":"2024-08-31T06:06:50.541212+0000","last_fullsized":"2024-08-31T06:06:50.541212+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:46:02.534190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393846,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":32,"num_write":71,"num_write_kb":873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.682380+0000","last_change":"2024-08-31T06:04:25.888946+0000","last_active":"2024-08-31T06:06:23.682380+0000","last_peered":"2024-08-31T06:06:23.682380+0000","last_clean":"2024-08-31T06:06:23.682380+0000","last_became_active":"2024-08-31T06:04:25.888564+0000","last_became_peered":"2024-08-31T06:04:25.888564+0000","last_unstale":"2024-08-31T06:06:23.682380+0000","last_undegraded":"2024-08-31T06:06:23.682380+0000","last_fullsized":"2024-08-31T06:06:23.682380+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:26:19.051971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.049211+0000","last_change":"2024-08-31T06:04:22.964483+0000","last_active":"2024-08-31T06:05:25.049211+0000","last_peered":"2024-08-31T06:05:25.049211+0000","last_clean":"2024-08-31T06:05:25.049211+0000","last_became_active":"2024-08-31T06:04:22.963874+0000","last_became_peered":"2024-08-31T06:04:22.963874+0000","last_unstale":"2024-08-31T06:05:25.049211+0000","last_undegraded":"2024-08-31T06:05:25.049211+0000","last_fullsized":"2024-08-31T06:05:25.049211+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:30:00.159040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.688990+0000","last_change":"2024-08-31T06:04:25.880941+0000","last_active":"2024-08-31T06:06:23.688990+0000","last_peered":"2024-08-31T06:06:23.688990+0000","last_clean":"2024-08-31T06:06:23.688990+0000","last_became_active":"2024-08-31T06:04:25.880454+0000","last_became_peered":"2024-08-31T06:04:25.880454+0000","last_unstale":"2024-08-31T06:06:23.688990+0000","last_undegraded":"2024-08-31T06:06:23.688990+0000","last_fullsized":"2024-08-31T06:06:23.688990+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:49:57.023988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":98,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048527+0000","last_change":"2024-08-31T06:03:05.921629+0000","last_active":"2024-08-31T06:05:25.048527+0000","last_peered":"2024-08-31T06:05:25.048527+0000","last_clean":"2024-08-31T06:05:25.048527+0000","last_became_active":"2024-08-31T06:03:05.915844+0000","last_became_peered":"2024-08-31T06:03:05.915844+0000","last_unstale":"2024-08-31T06:05:25.048527+0000","last_undegraded":"2024-08-31T06:05:25.048527+0000","last_fullsized":"2024-08-31T06:05:25.048527+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T05:59:24.472429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T05:59:24.472429+0000","last_clean_scrub_stamp":"2024-08-31T05:59:24.472429+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:56:41.073422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.044718+0000","last_change":"2024-08-31T06:04:22.846325+0000","last_active":"2024-08-31T06:05:25.044718+0000","last_peered":"2024-08-31T06:05:25.044718+0000","last_clean":"2024-08-31T06:05:25.044718+0000","last_became_active":"2024-08-31T06:04:22.845772+0000","last_became_peered":"2024-08-31T06:04:22.845772+0000","last_unstale":"2024-08-31T06:05:25.044718+0000","last_undegraded":"2024-08-31T06:05:25.044718+0000","last_fullsized":"2024-08-31T06:05:25.044718+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:54:10.336786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.691294+0000","last_change":"2024-08-31T06:04:25.891437+0000","last_active":"2024-08-31T06:06:23.691294+0000","last_peered":"2024-08-31T06:06:23.691294+0000","last_clean":"2024-08-31T06:06:23.691294+0000","last_became_active":"2024-08-31T06:04:25.891053+0000","last_became_peered":"2024-08-31T06:04:25.891053+0000","last_unstale":"2024-08-31T06:06:23.691294+0000","last_undegraded":"2024-08-31T06:06:23.691294+0000","last_fullsized":"2024-08-31T06:06:23.691294+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:22:37.119483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048411+0000","last_change":"2024-08-31T06:04:22.842885+0000","last_active":"2024-08-31T06:05:25.048411+0000","last_peered":"2024-08-31T06:05:25.048411+0000","last_clean":"2024-08-31T06:05:25.048411+0000","last_became_active":"2024-08-31T06:04:22.842473+0000","last_became_peered":"2024-08-31T06:04:22.842473+0000","last_unstale":"2024-08-31T06:05:25.048411+0000","last_undegraded":"2024-08-31T06:05:25.048411+0000","last_fullsized":"2024-08-31T06:05:25.048411+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:33:36.516659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.663699+0000","last_change":"2024-08-31T06:04:25.889213+0000","last_active":"2024-08-31T06:06:23.663699+0000","last_peered":"2024-08-31T06:06:23.663699+0000","last_clean":"2024-08-31T06:06:23.663699+0000","last_became_active":"2024-08-31T06:04:25.888860+0000","last_became_peered":"2024-08-31T06:04:25.888860+0000","last_unstale":"2024-08-31T06:06:23.663699+0000","last_undegraded":"2024-08-31T06:06:23.663699+0000","last_fullsized":"2024-08-31T06:06:23.663699+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:44:18.831229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":167,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:53.748738+0000","last_change":"2024-08-31T06:04:22.966091+0000","last_active":"2024-08-31T06:06:53.748738+0000","last_peered":"2024-08-31T06:06:53.748738+0000","last_clean":"2024-08-31T06:06:53.748738+0000","last_became_active":"2024-08-31T06:04:22.965727+0000","last_became_peered":"2024-08-31T06:04:22.965727+0000","last_unstale":"2024-08-31T06:06:53.748738+0000","last_undegraded":"2024-08-31T06:06:53.748738+0000","last_fullsized":"2024-08-31T06:06:53.748738+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:42:43.643623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":135,"num_read_kb":135,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.682138+0000","last_change":"2024-08-31T06:04:25.880039+0000","last_active":"2024-08-31T06:06:23.682138+0000","last_peered":"2024-08-31T06:06:23.682138+0000","last_clean":"2024-08-31T06:06:23.682138+0000","last_became_active":"2024-08-31T06:04:25.879669+0000","last_became_peered":"2024-08-31T06:04:25.879669+0000","last_unstale":"2024-08-31T06:06:23.682138+0000","last_undegraded":"2024-08-31T06:06:23.682138+0000","last_fullsized":"2024-08-31T06:06:23.682138+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:02:09.244281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048687+0000","last_change":"2024-08-31T06:04:22.844572+0000","last_active":"2024-08-31T06:05:25.048687+0000","last_peered":"2024-08-31T06:05:25.048687+0000","last_clean":"2024-08-31T06:05:25.048687+0000","last_became_active":"2024-08-31T06:04:22.844227+0000","last_became_peered":"2024-08-31T06:04:22.844227+0000","last_unstale":"2024-08-31T06:05:25.048687+0000","last_undegraded":"2024-08-31T06:05:25.048687+0000","last_fullsized":"2024-08-31T06:05:25.048687+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:04:51.966901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.677043+0000","last_change":"2024-08-31T06:04:25.884442+0000","last_active":"2024-08-31T06:06:23.677043+0000","last_peered":"2024-08-31T06:06:23.677043+0000","last_clean":"2024-08-31T06:06:23.677043+0000","last_became_active":"2024-08-31T06:04:25.884016+0000","last_became_peered":"2024-08-31T06:04:25.884016+0000","last_unstale":"2024-08-31T06:06:23.677043+0000","last_undegraded":"2024-08-31T06:06:23.677043+0000","last_fullsized":"2024-08-31T06:06:23.677043+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:36:10.640262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.043831+0000","last_change":"2024-08-31T06:04:22.967845+0000","last_active":"2024-08-31T06:05:25.043831+0000","last_peered":"2024-08-31T06:05:25.043831+0000","last_clean":"2024-08-31T06:05:25.043831+0000","last_became_active":"2024-08-31T06:04:22.967421+0000","last_became_peered":"2024-08-31T06:04:22.967421+0000","last_unstale":"2024-08-31T06:05:25.043831+0000","last_undegraded":"2024-08-31T06:05:25.043831+0000","last_fullsized":"2024-08-31T06:05:25.043831+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:17:46.942551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.673410+0000","last_change":"2024-08-31T06:04:25.897087+0000","last_active":"2024-08-31T06:06:23.673410+0000","last_peered":"2024-08-31T06:06:23.673410+0000","last_clean":"2024-08-31T06:06:23.673410+0000","last_became_active":"2024-08-31T06:04:25.896725+0000","last_became_peered":"2024-08-31T06:04:25.896725+0000","last_unstale":"2024-08-31T06:06:23.673410+0000","last_undegraded":"2024-08-31T06:06:23.673410+0000","last_fullsized":"2024-08-31T06:06:23.673410+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:31:28.279489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.053673+0000","last_change":"2024-08-31T06:04:22.965299+0000","last_active":"2024-08-31T06:05:25.053673+0000","last_peered":"2024-08-31T06:05:25.053673+0000","last_clean":"2024-08-31T06:05:25.053673+0000","last_became_active":"2024-08-31T06:04:22.964849+0000","last_became_peered":"2024-08-31T06:04:22.964849+0000","last_unstale":"2024-08-31T06:05:25.053673+0000","last_undegraded":"2024-08-31T06:05:25.053673+0000","last_fullsized":"2024-08-31T06:05:25.053673+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:36:56.431678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.679635+0000","last_change":"2024-08-31T06:04:25.885115+0000","last_active":"2024-08-31T06:06:23.679635+0000","last_peered":"2024-08-31T06:06:23.679635+0000","last_clean":"2024-08-31T06:06:23.679635+0000","last_became_active":"2024-08-31T06:04:25.884722+0000","last_became_peered":"2024-08-31T06:04:25.884722+0000","last_unstale":"2024-08-31T06:06:23.679635+0000","last_undegraded":"2024-08-31T06:06:23.679635+0000","last_fullsized":"2024-08-31T06:06:23.679635+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:20:43.913785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'97","reported_seq":214,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:51.071411+0000","last_change":"2024-08-31T06:04:22.845119+0000","last_active":"2024-08-31T06:06:51.071411+0000","last_peered":"2024-08-31T06:06:51.071411+0000","last_clean":"2024-08-31T06:06:51.071411+0000","last_became_active":"2024-08-31T06:04:22.843814+0000","last_became_peered":"2024-08-31T06:04:22.843814+0000","last_unstale":"2024-08-31T06:06:51.071411+0000","last_undegraded":"2024-08-31T06:06:51.071411+0000","last_fullsized":"2024-08-31T06:06:51.071411+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:06:40.684887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":302,"num_read_kb":268,"num_write":179,"num_write_kb":1222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.680844+0000","last_change":"2024-08-31T06:04:25.892838+0000","last_active":"2024-08-31T06:06:23.680844+0000","last_peered":"2024-08-31T06:06:23.680844+0000","last_clean":"2024-08-31T06:06:23.680844+0000","last_became_active":"2024-08-31T06:04:25.891955+0000","last_became_peered":"2024-08-31T06:04:25.891955+0000","last_unstale":"2024-08-31T06:06:23.680844+0000","last_undegraded":"2024-08-31T06:06:23.680844+0000","last_fullsized":"2024-08-31T06:06:23.680844+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:41:35.085724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.044957+0000","last_change":"2024-08-31T06:04:22.964183+0000","last_active":"2024-08-31T06:05:25.044957+0000","last_peered":"2024-08-31T06:05:25.044957+0000","last_clean":"2024-08-31T06:05:25.044957+0000","last_became_active":"2024-08-31T06:04:22.963688+0000","last_became_peered":"2024-08-31T06:04:22.963688+0000","last_unstale":"2024-08-31T06:05:25.044957+0000","last_undegraded":"2024-08-31T06:05:25.044957+0000","last_fullsized":"2024-08-31T06:05:25.044957+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:04:47.833811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.683320+0000","last_change":"2024-08-31T06:04:25.892266+0000","last_active":"2024-08-31T06:06:23.683320+0000","last_peered":"2024-08-31T06:06:23.683320+0000","last_clean":"2024-08-31T06:06:23.683320+0000","last_became_active":"2024-08-31T06:04:25.889014+0000","last_became_peered":"2024-08-31T06:04:25.889014+0000","last_unstale":"2024-08-31T06:06:23.683320+0000","last_undegraded":"2024-08-31T06:06:23.683320+0000","last_fullsized":"2024-08-31T06:06:23.683320+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:27:47.608470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.049408+0000","last_change":"2024-08-31T06:04:22.964677+0000","last_active":"2024-08-31T06:05:25.049408+0000","last_peered":"2024-08-31T06:05:25.049408+0000","last_clean":"2024-08-31T06:05:25.049408+0000","last_became_active":"2024-08-31T06:04:22.964237+0000","last_became_peered":"2024-08-31T06:04:22.964237+0000","last_unstale":"2024-08-31T06:05:25.049408+0000","last_undegraded":"2024-08-31T06:05:25.049408+0000","last_fullsized":"2024-08-31T06:05:25.049408+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:00:54.099592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.648239+0000","last_change":"2024-08-31T06:04:25.881699+0000","last_active":"2024-08-31T06:06:23.648239+0000","last_peered":"2024-08-31T06:06:23.648239+0000","last_clean":"2024-08-31T06:06:23.648239+0000","last_became_active":"2024-08-31T06:04:25.879304+0000","last_became_peered":"2024-08-31T06:04:25.879304+0000","last_unstale":"2024-08-31T06:06:23.648239+0000","last_undegraded":"2024-08-31T06:06:23.648239+0000","last_fullsized":"2024-08-31T06:06:23.648239+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:56:42.992034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.108301+0000","last_change":"2024-08-31T06:04:22.847934+0000","last_active":"2024-08-31T06:05:25.108301+0000","last_peered":"2024-08-31T06:05:25.108301+0000","last_clean":"2024-08-31T06:05:25.108301+0000","last_became_active":"2024-08-31T06:04:22.847518+0000","last_became_peered":"2024-08-31T06:04:22.847518+0000","last_unstale":"2024-08-31T06:05:25.108301+0000","last_undegraded":"2024-08-31T06:05:25.108301+0000","last_fullsized":"2024-08-31T06:05:25.108301+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:26:21.476014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.685828+0000","last_change":"2024-08-31T06:04:25.884653+0000","last_active":"2024-08-31T06:06:23.685828+0000","last_peered":"2024-08-31T06:06:23.685828+0000","last_clean":"2024-08-31T06:06:23.685828+0000","last_became_active":"2024-08-31T06:04:25.884244+0000","last_became_peered":"2024-08-31T06:04:25.884244+0000","last_unstale":"2024-08-31T06:06:23.685828+0000","last_undegraded":"2024-08-31T06:06:23.685828+0000","last_fullsized":"2024-08-31T06:06:23.685828+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:13:00.814071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048648+0000","last_change":"2024-08-31T06:04:22.845924+0000","last_active":"2024-08-31T06:05:25.048648+0000","last_peered":"2024-08-31T06:05:25.048648+0000","last_clean":"2024-08-31T06:05:25.048648+0000","last_became_active":"2024-08-31T06:04:22.845445+0000","last_became_peered":"2024-08-31T06:04:22.845445+0000","last_unstale":"2024-08-31T06:05:25.048648+0000","last_undegraded":"2024-08-31T06:05:25.048648+0000","last_fullsized":"2024-08-31T06:05:25.048648+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:25:21.052136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.679162+0000","last_change":"2024-08-31T06:04:25.890069+0000","last_active":"2024-08-31T06:06:23.679162+0000","last_peered":"2024-08-31T06:06:23.679162+0000","last_clean":"2024-08-31T06:06:23.679162+0000","last_became_active":"2024-08-31T06:04:25.889709+0000","last_became_peered":"2024-08-31T06:04:25.889709+0000","last_unstale":"2024-08-31T06:06:23.679162+0000","last_undegraded":"2024-08-31T06:06:23.679162+0000","last_fullsized":"2024-08-31T06:06:23.679162+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:13:51.360023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.051611+0000","last_change":"2024-08-31T06:04:22.847845+0000","last_active":"2024-08-31T06:05:25.051611+0000","last_peered":"2024-08-31T06:05:25.051611+0000","last_clean":"2024-08-31T06:05:25.051611+0000","last_became_active":"2024-08-31T06:04:22.847001+0000","last_became_peered":"2024-08-31T06:04:22.847001+0000","last_unstale":"2024-08-31T06:05:25.051611+0000","last_undegraded":"2024-08-31T06:05:25.051611+0000","last_fullsized":"2024-08-31T06:05:25.051611+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:24:34.209076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.690813+0000","last_change":"2024-08-31T06:04:25.886759+0000","last_active":"2024-08-31T06:06:23.690813+0000","last_peered":"2024-08-31T06:06:23.690813+0000","last_clean":"2024-08-31T06:06:23.690813+0000","last_became_active":"2024-08-31T06:04:25.886176+0000","last_became_peered":"2024-08-31T06:04:25.886176+0000","last_unstale":"2024-08-31T06:06:23.690813+0000","last_undegraded":"2024-08-31T06:06:23.690813+0000","last_fullsized":"2024-08-31T06:06:23.690813+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:11:12.637451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"80'15","reported_seq":43,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.677867+0000","last_change":"2024-08-31T06:05:25.054468+0000","last_active":"2024-08-31T06:06:23.677867+0000","last_peered":"2024-08-31T06:06:23.677867+0000","last_clean":"2024-08-31T06:06:23.677867+0000","last_became_active":"2024-08-31T06:05:25.053657+0000","last_became_peered":"2024-08-31T06:05:25.053657+0000","last_unstale":"2024-08-31T06:06:23.677867+0000","last_undegraded":"2024-08-31T06:06:23.677867+0000","last_fullsized":"2024-08-31T06:06:23.677867+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:43:44.887637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.059757+0000","last_change":"2024-08-31T06:05:25.059219+0000","last_active":"2024-08-31T06:05:25.059757+0000","last_peered":"2024-08-31T06:05:25.059757+0000","last_clean":"2024-08-31T06:05:25.059757+0000","last_became_active":"2024-08-31T06:05:25.053136+0000","last_became_peered":"2024-08-31T06:05:25.053136+0000","last_unstale":"2024-08-31T06:05:25.059757+0000","last_undegraded":"2024-08-31T06:05:25.059757+0000","last_fullsized":"2024-08-31T06:05:25.059757+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:29:59.137465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.691799+0000","last_change":"2024-08-31T06:04:25.887524+0000","last_active":"2024-08-31T06:06:23.691799+0000","last_peered":"2024-08-31T06:06:23.691799+0000","last_clean":"2024-08-31T06:06:23.691799+0000","last_became_active":"2024-08-31T06:04:25.887176+0000","last_became_peered":"2024-08-31T06:04:25.887176+0000","last_unstale":"2024-08-31T06:06:23.691799+0000","last_undegraded":"2024-08-31T06:06:23.691799+0000","last_fullsized":"2024-08-31T06:06:23.691799+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:08:05.476804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.053583+0000","last_change":"2024-08-31T06:04:22.847747+0000","last_active":"2024-08-31T06:05:25.053583+0000","last_peered":"2024-08-31T06:05:25.053583+0000","last_clean":"2024-08-31T06:05:25.053583+0000","last_became_active":"2024-08-31T06:04:22.847332+0000","last_became_peered":"2024-08-31T06:04:22.847332+0000","last_unstale":"2024-08-31T06:05:25.053583+0000","last_undegraded":"2024-08-31T06:05:25.053583+0000","last_fullsized":"2024-08-31T06:05:25.053583+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:41:35.910991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.687796+0000","last_change":"2024-08-31T06:04:25.883110+0000","last_active":"2024-08-31T06:06:23.687796+0000","last_peered":"2024-08-31T06:06:23.687796+0000","last_clean":"2024-08-31T06:06:23.687796+0000","last_became_active":"2024-08-31T06:04:25.880487+0000","last_became_peered":"2024-08-31T06:04:25.880487+0000","last_unstale":"2024-08-31T06:06:23.687796+0000","last_undegraded":"2024-08-31T06:06:23.687796+0000","last_fullsized":"2024-08-31T06:06:23.687796+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:22:13.656078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.108210+0000","last_change":"2024-08-31T06:04:22.964758+0000","last_active":"2024-08-31T06:05:25.108210+0000","last_peered":"2024-08-31T06:05:25.108210+0000","last_clean":"2024-08-31T06:05:25.108210+0000","last_became_active":"2024-08-31T06:04:22.964391+0000","last_became_peered":"2024-08-31T06:04:22.964391+0000","last_unstale":"2024-08-31T06:05:25.108210+0000","last_undegraded":"2024-08-31T06:05:25.108210+0000","last_fullsized":"2024-08-31T06:05:25.108210+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:28:59.190657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.678294+0000","last_change":"2024-08-31T06:04:25.893738+0000","last_active":"2024-08-31T06:06:23.678294+0000","last_peered":"2024-08-31T06:06:23.678294+0000","last_clean":"2024-08-31T06:06:23.678294+0000","last_became_active":"2024-08-31T06:04:25.893308+0000","last_became_peered":"2024-08-31T06:04:25.893308+0000","last_unstale":"2024-08-31T06:06:23.678294+0000","last_undegraded":"2024-08-31T06:06:23.678294+0000","last_fullsized":"2024-08-31T06:06:23.678294+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:26:48.451894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":39,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:27.962460+0000","last_change":"2024-08-31T06:04:22.843346+0000","last_active":"2024-08-31T06:06:27.962460+0000","last_peered":"2024-08-31T06:06:27.962460+0000","last_clean":"2024-08-31T06:06:27.962460+0000","last_became_active":"2024-08-31T06:04:22.842249+0000","last_became_peered":"2024-08-31T06:04:22.842249+0000","last_unstale":"2024-08-31T06:06:27.962460+0000","last_undegraded":"2024-08-31T06:06:27.962460+0000","last_fullsized":"2024-08-31T06:06:27.962460+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:40:57.618940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":857361,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1168,"num_read_kb":795,"num_write":402,"num_write_kb":2214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1147,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":663552,"data_stored":2572083,"data_compressed":112140,"data_compressed_allocated":147456,"data_compressed_original":2285568,"omap_allocated":0,"internal_metadata":0},"log_size":294,"ondisk_log_size":294,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":249373,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710764,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":436463,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874288,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1077248,"data_stored":1487925,"data_compressed":60215,"data_compressed_allocated":77824,"data_compressed_original":974848,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037815,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":829893,"data_compressed":25094,"data_compressed_allocated":32768,"data_compressed_original":409600,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234044,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":829791,"data_compressed":25094,"data_compressed_allocated":32768,"data_compressed_original":409600,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397570,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":933888,"data_stored":1029200,"data_compressed":42124,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593800,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":436066,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790028,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":999424,"data_stored":1422128,"data_compressed":60215,"data_compressed_allocated":77824,"data_compressed_original":974848,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953555,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36208,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708944,"statfs":{"total":95995035648,"available":95957958656,"internally_reserved":0,"allocated":884736,"data_stored":438189,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149782,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":966656,"data_stored":1223369,"data_compressed":43185,"data_compressed_allocated":57344,"data_compressed_original":778240,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411421,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36268,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708884,"statfs":{"total":95995035648,"available":95957897216,"internally_reserved":0,"allocated":946176,"data_stored":502170,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542242,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36268,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708884,"statfs":{"total":95995035648,"available":95957897216,"internally_reserved":0,"allocated":946176,"data_stored":501666,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705768,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":892928,"data_stored":437337,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34699,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34699,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34699,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":65807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":66311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":787276,"data_compressed":37380,"data_compressed_allocated":49152,"data_compressed_original":761856,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":395784,"data_compressed":18091,"data_compressed_allocated":24576,"data_compressed_original":368640,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393932,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":159744,"data_stored":461581,"data_compressed":18091,"data_compressed_allocated":24576,"data_compressed_original":368640,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-31T06:06:56.143 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-08-31T06:06:56.144 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph pg dump --format=json 2024-08-31T06:06:56.526 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:06:57.019 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:56 smithi016 ceph-mon[17194]: pgmap v354: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 174 KiB/s wr, 54 op/s 2024-08-31T06:06:57.019 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:56 smithi016 ceph-mon[17194]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T06:06:57.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.197+0000 7fdb01800640 1 -- 172.21.15.16:0/3966844410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdafc103920 msgr2=0x7fdafc103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:57.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.197+0000 7fdb01800640 1 --2- 172.21.15.16:0/3966844410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdafc103920 0x7fdafc103d30 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdaec009860 tx=0x7fdaec02a340 comp rx=0 tx=0).stop 2024-08-31T06:06:57.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.197+0000 7fdb01800640 1 -- 172.21.15.16:0/3966844410 shutdown_connections 2024-08-31T06:06:57.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.197+0000 7fdb01800640 1 --2- 172.21.15.16:0/3966844410 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdafc109260 0x7fdafc109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:57.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.197+0000 7fdb01800640 1 --2- 172.21.15.16:0/3966844410 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdafc109bb0 0x7fdafc1103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:57.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.197+0000 7fdb01800640 1 --2- 172.21.15.16:0/3966844410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdafc103920 0x7fdafc103d30 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:57.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.197+0000 7fdb01800640 1 -- 172.21.15.16:0/3966844410 >> 172.21.15.16:0/3966844410 conn(0x7fdafc0ff370 msgr2=0x7fdafc1017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:57.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.197+0000 7fdb01800640 1 -- 172.21.15.16:0/3966844410 shutdown_connections 2024-08-31T06:06:57.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.197+0000 7fdb01800640 1 -- 172.21.15.16:0/3966844410 wait complete. 2024-08-31T06:06:57.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.198+0000 7fdb01800640 1 Processor -- start 2024-08-31T06:06:57.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.198+0000 7fdb01800640 1 -- start start 2024-08-31T06:06:57.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.198+0000 7fdb01800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdafc103920 0x7fdafc19c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:57.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.198+0000 7fdb01800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdafc109260 0x7fdafc19ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:57.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.199+0000 7fdb01800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdafc109bb0 0x7fdafc1a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:57.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.199+0000 7fdb01800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc113870 con 0x7fdafc109260 2024-08-31T06:06:57.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.199+0000 7fdb01800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc1136f0 con 0x7fdafc109bb0 2024-08-31T06:06:57.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.199+0000 7fdb01800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc1139f0 con 0x7fdafc103920 2024-08-31T06:06:57.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.199+0000 7fdafaa00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdafc103920 0x7fdafc19c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:57.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.199+0000 7fdafaa00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdafc103920 0x7fdafc19c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:35518/0 (socket says 172.21.15.16:35518) 2024-08-31T06:06:57.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.199+0000 7fdaf3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdafc109260 0x7fdafc19ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:57.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.199+0000 7fdafaa00640 1 -- 172.21.15.16:0/534295058 learned_addr learned my addr 172.21.15.16:0/534295058 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:57.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.199+0000 7fdafb400640 1 --2- 172.21.15.16:0/534295058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdafc109bb0 0x7fdafc1a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:57.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.199+0000 7fdafaa00640 1 -- 172.21.15.16:0/534295058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdafc109bb0 msgr2=0x7fdafc1a1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:57.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.199+0000 7fdafaa00640 1 --2- 172.21.15.16:0/534295058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdafc109bb0 0x7fdafc1a1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:57.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.199+0000 7fdafaa00640 1 -- 172.21.15.16:0/534295058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdafc109260 msgr2=0x7fdafc19ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:57.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.199+0000 7fdafaa00640 1 --2- 172.21.15.16:0/534295058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdafc109260 0x7fdafc19ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:57.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.199+0000 7fdafaa00640 1 -- 172.21.15.16:0/534295058 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdafc1a23f0 con 0x7fdafc103920 2024-08-31T06:06:57.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.199+0000 7fdaf3e00640 1 --2- 172.21.15.16:0/534295058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdafc109260 0x7fdafc19ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:06:57.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.200+0000 7fdafaa00640 1 --2- 172.21.15.16:0/534295058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdafc103920 0x7fdafc19c480 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdaec035040 tx=0x7fdaec032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:57.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.200+0000 7fdaf3400640 1 -- 172.21.15.16:0/534295058 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec037040 con 0x7fdafc103920 2024-08-31T06:06:57.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.200+0000 7fdb01800640 1 -- 172.21.15.16:0/534295058 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdafc1a2680 con 0x7fdafc103920 2024-08-31T06:06:57.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.200+0000 7fdaf3400640 1 -- 172.21.15.16:0/534295058 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaec041070 con 0x7fdafc103920 2024-08-31T06:06:57.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.200+0000 7fdaf3400640 1 -- 172.21.15.16:0/534295058 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec003850 con 0x7fdafc103920 2024-08-31T06:06:57.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.200+0000 7fdb01800640 1 -- 172.21.15.16:0/534295058 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdafc1a8e40 con 0x7fdafc103920 2024-08-31T06:06:57.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.201+0000 7fdb01800640 1 -- 172.21.15.16:0/534295058 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdafc104ba0 con 0x7fdafc103920 2024-08-31T06:06:57.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.204+0000 7fdaf3400640 1 -- 172.21.15.16:0/534295058 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 23) ==== 152651+0+0 (secure 0 0 0) 0x7fdaec0039f0 con 0x7fdafc103920 2024-08-31T06:06:57.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.205+0000 7fdaf3400640 1 --2- 172.21.15.16:0/534295058 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdae00b4c90 0x7fdae00b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:57.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.205+0000 7fdaf3400640 1 -- 172.21.15.16:0/534295058 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7fdaec105740 con 0x7fdafc103920 2024-08-31T06:06:57.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.206+0000 7fdaf3e00640 1 --2- 172.21.15.16:0/534295058 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdae00b4c90 0x7fdae00b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:57.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.206+0000 7fdaf3e00640 1 --2- 172.21.15.16:0/534295058 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdae00b4c90 0x7fdae00b71a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdae40096c0 tx=0x7fdae4009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:57.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.207+0000 7fdaf3400640 1 -- 172.21.15.16:0/534295058 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fdaec0cc670 con 0x7fdafc103920 2024-08-31T06:06:57.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:56 smithi184 ceph-mon[22331]: pgmap v354: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 174 KiB/s wr, 54 op/s 2024-08-31T06:06:57.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:56 smithi184 ceph-mon[22331]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T06:06:57.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:56 smithi155 ceph-mon[21742]: pgmap v354: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 174 KiB/s wr, 54 op/s 2024-08-31T06:06:57.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:56 smithi155 ceph-mon[21742]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T06:06:57.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.403+0000 7fdb01800640 1 -- 172.21.15.16:0/534295058 --> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdafc10f690 con 0x7fdae00b4c90 2024-08-31T06:06:57.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.408+0000 7fdaf3400640 1 -- 172.21.15.16:0/534295058 <== mgr.14220 v2:172.21.15.16:6800/4279751337 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259208 (secure 0 0 0) 0x7fdafc10f690 con 0x7fdae00b4c90 2024-08-31T06:06:57.414 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:06:57.415 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-08-31T06:06:57.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.412+0000 7fdb01800640 1 -- 172.21.15.16:0/534295058 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdae00b4c90 msgr2=0x7fdae00b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:57.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.412+0000 7fdb01800640 1 --2- 172.21.15.16:0/534295058 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdae00b4c90 0x7fdae00b71a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdae40096c0 tx=0x7fdae4009260 comp rx=0 tx=0).stop 2024-08-31T06:06:57.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.412+0000 7fdb01800640 1 -- 172.21.15.16:0/534295058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdafc103920 msgr2=0x7fdafc19c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:57.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.412+0000 7fdb01800640 1 --2- 172.21.15.16:0/534295058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdafc103920 0x7fdafc19c480 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdaec035040 tx=0x7fdaec032d50 comp rx=0 tx=0).stop 2024-08-31T06:06:57.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.412+0000 7fdb01800640 1 -- 172.21.15.16:0/534295058 shutdown_connections 2024-08-31T06:06:57.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.412+0000 7fdb01800640 1 --2- 172.21.15.16:0/534295058 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdae00b4c90 0x7fdae00b71a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:57.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.412+0000 7fdb01800640 1 --2- 172.21.15.16:0/534295058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdafc109bb0 0x7fdafc1a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:57.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.412+0000 7fdb01800640 1 --2- 172.21.15.16:0/534295058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdafc109260 0x7fdafc19ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:57.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.412+0000 7fdb01800640 1 --2- 172.21.15.16:0/534295058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdafc103920 0x7fdafc19c480 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:57.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.412+0000 7fdb01800640 1 -- 172.21.15.16:0/534295058 >> 172.21.15.16:0/534295058 conn(0x7fdafc0ff370 msgr2=0x7fdafc108950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:57.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.413+0000 7fdb01800640 1 -- 172.21.15.16:0/534295058 shutdown_connections 2024-08-31T06:06:57.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:57.413+0000 7fdb01800640 1 -- 172.21.15.16:0/534295058 wait complete. 2024-08-31T06:06:58.141 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-08-31T06:06:57.295009+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1447963,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1249,"num_read_kb":864,"num_write":527,"num_write_kb":4366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":2952987,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":888,"ondisk_log_size":888,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372684,"kb_used_data":10944,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124569140,"statfs":{"total":1151940427776,"available":1151558799360,"internally_reserved":0,"allocated":11206656,"data_stored":9574197,"data_compressed":290757,"data_compressed_allocated":380928,"data_compressed_original":4251648,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":786992,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":396,"num_read_kb":331,"num_write":224,"num_write_kb":2088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003332"},"pg_stats":[{"pgid":"3.3f","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.686021+0000","last_change":"2024-08-31T06:04:25.877318+0000","last_active":"2024-08-31T06:06:23.686021+0000","last_peered":"2024-08-31T06:06:23.686021+0000","last_clean":"2024-08-31T06:06:23.686021+0000","last_became_active":"2024-08-31T06:04:25.876894+0000","last_became_peered":"2024-08-31T06:04:25.876894+0000","last_unstale":"2024-08-31T06:06:23.686021+0000","last_undegraded":"2024-08-31T06:06:23.686021+0000","last_fullsized":"2024-08-31T06:06:23.686021+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:41:09.449343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"80'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.685244+0000","last_change":"2024-08-31T06:04:25.885670+0000","last_active":"2024-08-31T06:06:23.685244+0000","last_peered":"2024-08-31T06:06:23.685244+0000","last_clean":"2024-08-31T06:06:23.685244+0000","last_became_active":"2024-08-31T06:04:25.883818+0000","last_became_peered":"2024-08-31T06:04:25.883818+0000","last_unstale":"2024-08-31T06:06:23.685244+0000","last_undegraded":"2024-08-31T06:06:23.685244+0000","last_fullsized":"2024-08-31T06:06:23.685244+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:40:01.344432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.687803+0000","last_change":"2024-08-31T06:04:25.883375+0000","last_active":"2024-08-31T06:06:23.687803+0000","last_peered":"2024-08-31T06:06:23.687803+0000","last_clean":"2024-08-31T06:06:23.687803+0000","last_became_active":"2024-08-31T06:04:25.881750+0000","last_became_peered":"2024-08-31T06:04:25.881750+0000","last_unstale":"2024-08-31T06:06:23.687803+0000","last_undegraded":"2024-08-31T06:06:23.687803+0000","last_fullsized":"2024-08-31T06:06:23.687803+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:46:46.757363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.676625+0000","last_change":"2024-08-31T06:04:25.882436+0000","last_active":"2024-08-31T06:06:23.676625+0000","last_peered":"2024-08-31T06:06:23.676625+0000","last_clean":"2024-08-31T06:06:23.676625+0000","last_became_active":"2024-08-31T06:04:25.881805+0000","last_became_peered":"2024-08-31T06:04:25.881805+0000","last_unstale":"2024-08-31T06:06:23.676625+0000","last_undegraded":"2024-08-31T06:06:23.676625+0000","last_fullsized":"2024-08-31T06:06:23.676625+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:42:39.793625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"80'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.671560+0000","last_change":"2024-08-31T06:04:25.894534+0000","last_active":"2024-08-31T06:06:23.671560+0000","last_peered":"2024-08-31T06:06:23.671560+0000","last_clean":"2024-08-31T06:06:23.671560+0000","last_became_active":"2024-08-31T06:04:25.894116+0000","last_became_peered":"2024-08-31T06:04:25.894116+0000","last_unstale":"2024-08-31T06:06:23.671560+0000","last_undegraded":"2024-08-31T06:06:23.671560+0000","last_fullsized":"2024-08-31T06:06:23.671560+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T18:02:31.257454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.669411+0000","last_change":"2024-08-31T06:04:25.886182+0000","last_active":"2024-08-31T06:06:23.669411+0000","last_peered":"2024-08-31T06:06:23.669411+0000","last_clean":"2024-08-31T06:06:23.669411+0000","last_became_active":"2024-08-31T06:04:25.885825+0000","last_became_peered":"2024-08-31T06:04:25.885825+0000","last_unstale":"2024-08-31T06:06:23.669411+0000","last_undegraded":"2024-08-31T06:06:23.669411+0000","last_fullsized":"2024-08-31T06:06:23.669411+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:58:12.427931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.682136+0000","last_change":"2024-08-31T06:04:25.877521+0000","last_active":"2024-08-31T06:06:23.682136+0000","last_peered":"2024-08-31T06:06:23.682136+0000","last_clean":"2024-08-31T06:06:23.682136+0000","last_became_active":"2024-08-31T06:04:25.877122+0000","last_became_peered":"2024-08-31T06:04:25.877122+0000","last_unstale":"2024-08-31T06:06:23.682136+0000","last_undegraded":"2024-08-31T06:06:23.682136+0000","last_fullsized":"2024-08-31T06:06:23.682136+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:17:27.406828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.680720+0000","last_change":"2024-08-31T06:04:25.892547+0000","last_active":"2024-08-31T06:06:23.680720+0000","last_peered":"2024-08-31T06:06:23.680720+0000","last_clean":"2024-08-31T06:06:23.680720+0000","last_became_active":"2024-08-31T06:04:25.892090+0000","last_became_peered":"2024-08-31T06:04:25.892090+0000","last_unstale":"2024-08-31T06:06:23.680720+0000","last_undegraded":"2024-08-31T06:06:23.680720+0000","last_fullsized":"2024-08-31T06:06:23.680720+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:28:53.414252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"80'11","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.689423+0000","last_change":"2024-08-31T06:04:25.879952+0000","last_active":"2024-08-31T06:06:23.689423+0000","last_peered":"2024-08-31T06:06:23.689423+0000","last_clean":"2024-08-31T06:06:23.689423+0000","last_became_active":"2024-08-31T06:04:25.879568+0000","last_became_peered":"2024-08-31T06:04:25.879568+0000","last_unstale":"2024-08-31T06:06:23.689423+0000","last_undegraded":"2024-08-31T06:06:23.689423+0000","last_fullsized":"2024-08-31T06:06:23.689423+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:28:04.558054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"80'16","reported_seq":45,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.677734+0000","last_change":"2024-08-31T06:05:25.051173+0000","last_active":"2024-08-31T06:06:23.677734+0000","last_peered":"2024-08-31T06:06:23.677734+0000","last_clean":"2024-08-31T06:06:23.677734+0000","last_became_active":"2024-08-31T06:05:25.050867+0000","last_became_peered":"2024-08-31T06:05:25.050867+0000","last_unstale":"2024-08-31T06:06:23.677734+0000","last_undegraded":"2024-08-31T06:06:23.677734+0000","last_fullsized":"2024-08-31T06:06:23.677734+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:27:09.971497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.686612+0000","last_change":"2024-08-31T06:04:25.883984+0000","last_active":"2024-08-31T06:06:23.686612+0000","last_peered":"2024-08-31T06:06:23.686612+0000","last_clean":"2024-08-31T06:06:23.686612+0000","last_became_active":"2024-08-31T06:04:25.883353+0000","last_became_peered":"2024-08-31T06:04:25.883353+0000","last_unstale":"2024-08-31T06:06:23.686612+0000","last_undegraded":"2024-08-31T06:06:23.686612+0000","last_fullsized":"2024-08-31T06:06:23.686612+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:16:25.178952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.673031+0000","last_change":"2024-08-31T06:04:25.883335+0000","last_active":"2024-08-31T06:06:23.673031+0000","last_peered":"2024-08-31T06:06:23.673031+0000","last_clean":"2024-08-31T06:06:23.673031+0000","last_became_active":"2024-08-31T06:04:25.881488+0000","last_became_peered":"2024-08-31T06:04:25.881488+0000","last_unstale":"2024-08-31T06:06:23.673031+0000","last_undegraded":"2024-08-31T06:06:23.673031+0000","last_fullsized":"2024-08-31T06:06:23.673031+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:57:24.472482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'10","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.678797+0000","last_change":"2024-08-31T06:04:25.891579+0000","last_active":"2024-08-31T06:06:23.678797+0000","last_peered":"2024-08-31T06:06:23.678797+0000","last_clean":"2024-08-31T06:06:23.678797+0000","last_became_active":"2024-08-31T06:04:25.887950+0000","last_became_peered":"2024-08-31T06:04:25.887950+0000","last_unstale":"2024-08-31T06:06:23.678797+0000","last_undegraded":"2024-08-31T06:06:23.678797+0000","last_fullsized":"2024-08-31T06:06:23.678797+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:38:35.284442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.682430+0000","last_change":"2024-08-31T06:04:25.882185+0000","last_active":"2024-08-31T06:06:23.682430+0000","last_peered":"2024-08-31T06:06:23.682430+0000","last_clean":"2024-08-31T06:06:23.682430+0000","last_became_active":"2024-08-31T06:04:25.876499+0000","last_became_peered":"2024-08-31T06:04:25.876499+0000","last_unstale":"2024-08-31T06:06:23.682430+0000","last_undegraded":"2024-08-31T06:06:23.682430+0000","last_fullsized":"2024-08-31T06:06:23.682430+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:40:01.132353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.683904+0000","last_change":"2024-08-31T06:04:25.883624+0000","last_active":"2024-08-31T06:06:23.683904+0000","last_peered":"2024-08-31T06:06:23.683904+0000","last_clean":"2024-08-31T06:06:23.683904+0000","last_became_active":"2024-08-31T06:04:25.878041+0000","last_became_peered":"2024-08-31T06:04:25.878041+0000","last_unstale":"2024-08-31T06:06:23.683904+0000","last_undegraded":"2024-08-31T06:06:23.683904+0000","last_fullsized":"2024-08-31T06:06:23.683904+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:05:21.278321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.690513+0000","last_change":"2024-08-31T06:04:25.882538+0000","last_active":"2024-08-31T06:06:23.690513+0000","last_peered":"2024-08-31T06:06:23.690513+0000","last_clean":"2024-08-31T06:06:23.690513+0000","last_became_active":"2024-08-31T06:04:25.882201+0000","last_became_peered":"2024-08-31T06:04:25.882201+0000","last_unstale":"2024-08-31T06:06:23.690513+0000","last_undegraded":"2024-08-31T06:06:23.690513+0000","last_fullsized":"2024-08-31T06:06:23.690513+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:47:16.880893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.686157+0000","last_change":"2024-08-31T06:04:25.880376+0000","last_active":"2024-08-31T06:06:23.686157+0000","last_peered":"2024-08-31T06:06:23.686157+0000","last_clean":"2024-08-31T06:06:23.686157+0000","last_became_active":"2024-08-31T06:04:25.877668+0000","last_became_peered":"2024-08-31T06:04:25.877668+0000","last_unstale":"2024-08-31T06:06:23.686157+0000","last_undegraded":"2024-08-31T06:06:23.686157+0000","last_fullsized":"2024-08-31T06:06:23.686157+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:55:32.737406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"80'10","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.673590+0000","last_change":"2024-08-31T06:04:25.875431+0000","last_active":"2024-08-31T06:06:23.673590+0000","last_peered":"2024-08-31T06:06:23.673590+0000","last_clean":"2024-08-31T06:06:23.673590+0000","last_became_active":"2024-08-31T06:04:25.874751+0000","last_became_peered":"2024-08-31T06:04:25.874751+0000","last_unstale":"2024-08-31T06:06:23.673590+0000","last_undegraded":"2024-08-31T06:06:23.673590+0000","last_fullsized":"2024-08-31T06:06:23.673590+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:16:57.695415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.675738+0000","last_change":"2024-08-31T06:04:25.895656+0000","last_active":"2024-08-31T06:06:23.675738+0000","last_peered":"2024-08-31T06:06:23.675738+0000","last_clean":"2024-08-31T06:06:23.675738+0000","last_became_active":"2024-08-31T06:04:25.895272+0000","last_became_peered":"2024-08-31T06:04:25.895272+0000","last_unstale":"2024-08-31T06:06:23.675738+0000","last_undegraded":"2024-08-31T06:06:23.675738+0000","last_fullsized":"2024-08-31T06:06:23.675738+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:18:47.276195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.679697+0000","last_change":"2024-08-31T06:04:25.877283+0000","last_active":"2024-08-31T06:06:23.679697+0000","last_peered":"2024-08-31T06:06:23.679697+0000","last_clean":"2024-08-31T06:06:23.679697+0000","last_became_active":"2024-08-31T06:04:25.876846+0000","last_became_peered":"2024-08-31T06:04:25.876846+0000","last_unstale":"2024-08-31T06:06:23.679697+0000","last_undegraded":"2024-08-31T06:06:23.679697+0000","last_fullsized":"2024-08-31T06:06:23.679697+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:54:05.086844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"80'11","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.685368+0000","last_change":"2024-08-31T06:04:25.884891+0000","last_active":"2024-08-31T06:06:23.685368+0000","last_peered":"2024-08-31T06:06:23.685368+0000","last_clean":"2024-08-31T06:06:23.685368+0000","last_became_active":"2024-08-31T06:04:25.884501+0000","last_became_peered":"2024-08-31T06:04:25.884501+0000","last_unstale":"2024-08-31T06:06:23.685368+0000","last_undegraded":"2024-08-31T06:06:23.685368+0000","last_fullsized":"2024-08-31T06:06:23.685368+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:22:54.846102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"80'10","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.691670+0000","last_change":"2024-08-31T06:04:25.877150+0000","last_active":"2024-08-31T06:06:23.691670+0000","last_peered":"2024-08-31T06:06:23.691670+0000","last_clean":"2024-08-31T06:06:23.691670+0000","last_became_active":"2024-08-31T06:04:25.876766+0000","last_became_peered":"2024-08-31T06:04:25.876766+0000","last_unstale":"2024-08-31T06:06:23.691670+0000","last_undegraded":"2024-08-31T06:06:23.691670+0000","last_fullsized":"2024-08-31T06:06:23.691670+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:46:26.724970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.676164+0000","last_change":"2024-08-31T06:04:25.881275+0000","last_active":"2024-08-31T06:06:23.676164+0000","last_peered":"2024-08-31T06:06:23.676164+0000","last_clean":"2024-08-31T06:06:23.676164+0000","last_became_active":"2024-08-31T06:04:25.879877+0000","last_became_peered":"2024-08-31T06:04:25.879877+0000","last_unstale":"2024-08-31T06:06:23.676164+0000","last_undegraded":"2024-08-31T06:06:23.676164+0000","last_fullsized":"2024-08-31T06:06:23.676164+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:07:07.237041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.676080+0000","last_change":"2024-08-31T06:04:25.895750+0000","last_active":"2024-08-31T06:06:23.676080+0000","last_peered":"2024-08-31T06:06:23.676080+0000","last_clean":"2024-08-31T06:06:23.676080+0000","last_became_active":"2024-08-31T06:04:25.895348+0000","last_became_peered":"2024-08-31T06:04:25.895348+0000","last_unstale":"2024-08-31T06:06:23.676080+0000","last_undegraded":"2024-08-31T06:06:23.676080+0000","last_fullsized":"2024-08-31T06:06:23.676080+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:53:10.014808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.670437+0000","last_change":"2024-08-31T06:04:25.895537+0000","last_active":"2024-08-31T06:06:23.670437+0000","last_peered":"2024-08-31T06:06:23.670437+0000","last_clean":"2024-08-31T06:06:23.670437+0000","last_became_active":"2024-08-31T06:04:25.895136+0000","last_became_peered":"2024-08-31T06:04:25.895136+0000","last_unstale":"2024-08-31T06:06:23.670437+0000","last_undegraded":"2024-08-31T06:06:23.670437+0000","last_fullsized":"2024-08-31T06:06:23.670437+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:41:57.554507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"80'15","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.676301+0000","last_change":"2024-08-31T06:04:25.880490+0000","last_active":"2024-08-31T06:06:23.676301+0000","last_peered":"2024-08-31T06:06:23.676301+0000","last_clean":"2024-08-31T06:06:23.676301+0000","last_became_active":"2024-08-31T06:04:25.879269+0000","last_became_peered":"2024-08-31T06:04:25.879269+0000","last_unstale":"2024-08-31T06:06:23.676301+0000","last_undegraded":"2024-08-31T06:06:23.676301+0000","last_fullsized":"2024-08-31T06:06:23.676301+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:05:49.575915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"80'6","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.676526+0000","last_change":"2024-08-31T06:05:25.053714+0000","last_active":"2024-08-31T06:06:23.676526+0000","last_peered":"2024-08-31T06:06:23.676526+0000","last_clean":"2024-08-31T06:06:23.676526+0000","last_became_active":"2024-08-31T06:05:25.052685+0000","last_became_peered":"2024-08-31T06:05:25.052685+0000","last_unstale":"2024-08-31T06:06:23.676526+0000","last_undegraded":"2024-08-31T06:06:23.676526+0000","last_fullsized":"2024-08-31T06:06:23.676526+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:17:31.474601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"80'4","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.684823+0000","last_change":"2024-08-31T06:05:25.057946+0000","last_active":"2024-08-31T06:06:23.684823+0000","last_peered":"2024-08-31T06:06:23.684823+0000","last_clean":"2024-08-31T06:06:23.684823+0000","last_became_active":"2024-08-31T06:05:25.057501+0000","last_became_peered":"2024-08-31T06:05:25.057501+0000","last_unstale":"2024-08-31T06:06:23.684823+0000","last_undegraded":"2024-08-31T06:06:23.684823+0000","last_fullsized":"2024-08-31T06:06:23.684823+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:16:05.721770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.685410+0000","last_change":"2024-08-31T06:04:25.897473+0000","last_active":"2024-08-31T06:06:23.685410+0000","last_peered":"2024-08-31T06:06:23.685410+0000","last_clean":"2024-08-31T06:06:23.685410+0000","last_became_active":"2024-08-31T06:04:25.895194+0000","last_became_peered":"2024-08-31T06:04:25.895194+0000","last_unstale":"2024-08-31T06:06:23.685410+0000","last_undegraded":"2024-08-31T06:06:23.685410+0000","last_fullsized":"2024-08-31T06:06:23.685410+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:43:04.539970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.684923+0000","last_change":"2024-08-31T06:04:25.894402+0000","last_active":"2024-08-31T06:06:23.684923+0000","last_peered":"2024-08-31T06:06:23.684923+0000","last_clean":"2024-08-31T06:06:23.684923+0000","last_became_active":"2024-08-31T06:04:25.893883+0000","last_became_peered":"2024-08-31T06:04:25.893883+0000","last_unstale":"2024-08-31T06:06:23.684923+0000","last_undegraded":"2024-08-31T06:06:23.684923+0000","last_fullsized":"2024-08-31T06:06:23.684923+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:45:53.778473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"80'12","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.681979+0000","last_change":"2024-08-31T06:04:25.881543+0000","last_active":"2024-08-31T06:06:23.681979+0000","last_peered":"2024-08-31T06:06:23.681979+0000","last_clean":"2024-08-31T06:06:23.681979+0000","last_became_active":"2024-08-31T06:04:25.881144+0000","last_became_peered":"2024-08-31T06:04:25.881144+0000","last_unstale":"2024-08-31T06:06:23.681979+0000","last_undegraded":"2024-08-31T06:06:23.681979+0000","last_fullsized":"2024-08-31T06:06:23.681979+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:13:02.282879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.669781+0000","last_change":"2024-08-31T06:04:25.881143+0000","last_active":"2024-08-31T06:06:23.669781+0000","last_peered":"2024-08-31T06:06:23.669781+0000","last_clean":"2024-08-31T06:06:23.669781+0000","last_became_active":"2024-08-31T06:04:25.879718+0000","last_became_peered":"2024-08-31T06:04:25.879718+0000","last_unstale":"2024-08-31T06:06:23.669781+0000","last_undegraded":"2024-08-31T06:06:23.669781+0000","last_fullsized":"2024-08-31T06:06:23.669781+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:32:07.771488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.680341+0000","last_change":"2024-08-31T06:04:25.879493+0000","last_active":"2024-08-31T06:06:23.680341+0000","last_peered":"2024-08-31T06:06:23.680341+0000","last_clean":"2024-08-31T06:06:23.680341+0000","last_became_active":"2024-08-31T06:04:25.879065+0000","last_became_peered":"2024-08-31T06:04:25.879065+0000","last_unstale":"2024-08-31T06:06:23.680341+0000","last_undegraded":"2024-08-31T06:06:23.680341+0000","last_fullsized":"2024-08-31T06:06:23.680341+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:53:43.848269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"80'7","reported_seq":184,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:53.188018+0000","last_change":"2024-08-31T06:05:25.058954+0000","last_active":"2024-08-31T06:06:53.188018+0000","last_peered":"2024-08-31T06:06:53.188018+0000","last_clean":"2024-08-31T06:06:53.188018+0000","last_became_active":"2024-08-31T06:05:25.051143+0000","last_became_peered":"2024-08-31T06:05:25.051143+0000","last_unstale":"2024-08-31T06:06:53.188018+0000","last_undegraded":"2024-08-31T06:06:53.188018+0000","last_fullsized":"2024-08-31T06:06:53.188018+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:38:23.533663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":132,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.682132+0000","last_change":"2024-08-31T06:04:25.896354+0000","last_active":"2024-08-31T06:06:23.682132+0000","last_peered":"2024-08-31T06:06:23.682132+0000","last_clean":"2024-08-31T06:06:23.682132+0000","last_became_active":"2024-08-31T06:04:25.896021+0000","last_became_peered":"2024-08-31T06:04:25.896021+0000","last_unstale":"2024-08-31T06:06:23.682132+0000","last_undegraded":"2024-08-31T06:06:23.682132+0000","last_fullsized":"2024-08-31T06:06:23.682132+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:08:40.213238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.047221+0000","last_change":"2024-08-31T06:04:22.844248+0000","last_active":"2024-08-31T06:05:25.047221+0000","last_peered":"2024-08-31T06:05:25.047221+0000","last_clean":"2024-08-31T06:05:25.047221+0000","last_became_active":"2024-08-31T06:04:22.843784+0000","last_became_peered":"2024-08-31T06:04:22.843784+0000","last_unstale":"2024-08-31T06:05:25.047221+0000","last_undegraded":"2024-08-31T06:05:25.047221+0000","last_fullsized":"2024-08-31T06:05:25.047221+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:56:41.422965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.046268+0000","last_change":"2024-08-31T06:04:22.847049+0000","last_active":"2024-08-31T06:05:25.046268+0000","last_peered":"2024-08-31T06:05:25.046268+0000","last_clean":"2024-08-31T06:05:25.046268+0000","last_became_active":"2024-08-31T06:04:22.846282+0000","last_became_peered":"2024-08-31T06:04:22.846282+0000","last_unstale":"2024-08-31T06:05:25.046268+0000","last_undegraded":"2024-08-31T06:05:25.046268+0000","last_fullsized":"2024-08-31T06:05:25.046268+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:35:21.030425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.679608+0000","last_change":"2024-08-31T06:04:25.886698+0000","last_active":"2024-08-31T06:06:23.679608+0000","last_peered":"2024-08-31T06:06:23.679608+0000","last_clean":"2024-08-31T06:06:23.679608+0000","last_became_active":"2024-08-31T06:04:25.886334+0000","last_became_peered":"2024-08-31T06:04:25.886334+0000","last_unstale":"2024-08-31T06:06:23.679608+0000","last_undegraded":"2024-08-31T06:06:23.679608+0000","last_fullsized":"2024-08-31T06:06:23.679608+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:24:43.821446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.044900+0000","last_change":"2024-08-31T06:04:22.845263+0000","last_active":"2024-08-31T06:05:25.044900+0000","last_peered":"2024-08-31T06:05:25.044900+0000","last_clean":"2024-08-31T06:05:25.044900+0000","last_became_active":"2024-08-31T06:04:22.844853+0000","last_became_peered":"2024-08-31T06:04:22.844853+0000","last_unstale":"2024-08-31T06:05:25.044900+0000","last_undegraded":"2024-08-31T06:05:25.044900+0000","last_fullsized":"2024-08-31T06:05:25.044900+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:55:12.561165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"80'11","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.673457+0000","last_change":"2024-08-31T06:04:25.894475+0000","last_active":"2024-08-31T06:06:23.673457+0000","last_peered":"2024-08-31T06:06:23.673457+0000","last_clean":"2024-08-31T06:06:23.673457+0000","last_became_active":"2024-08-31T06:04:25.894121+0000","last_became_peered":"2024-08-31T06:04:25.894121+0000","last_unstale":"2024-08-31T06:06:23.673457+0000","last_undegraded":"2024-08-31T06:06:23.673457+0000","last_fullsized":"2024-08-31T06:06:23.673457+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:40:06.570616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":165,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:53.187549+0000","last_change":"2024-08-31T06:04:22.834430+0000","last_active":"2024-08-31T06:06:53.187549+0000","last_peered":"2024-08-31T06:06:53.187549+0000","last_clean":"2024-08-31T06:06:53.187549+0000","last_became_active":"2024-08-31T06:04:22.834011+0000","last_became_peered":"2024-08-31T06:04:22.834011+0000","last_unstale":"2024-08-31T06:06:53.187549+0000","last_undegraded":"2024-08-31T06:06:53.187549+0000","last_fullsized":"2024-08-31T06:06:53.187549+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:51:16.734234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":134,"num_read_kb":134,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"80'10","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.689788+0000","last_change":"2024-08-31T06:04:25.877727+0000","last_active":"2024-08-31T06:06:23.689788+0000","last_peered":"2024-08-31T06:06:23.689788+0000","last_clean":"2024-08-31T06:06:23.689788+0000","last_became_active":"2024-08-31T06:04:25.877121+0000","last_became_peered":"2024-08-31T06:04:25.877121+0000","last_unstale":"2024-08-31T06:06:23.689788+0000","last_undegraded":"2024-08-31T06:06:23.689788+0000","last_fullsized":"2024-08-31T06:06:23.689788+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:41:12.247216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048879+0000","last_change":"2024-08-31T06:04:22.846482+0000","last_active":"2024-08-31T06:05:25.048879+0000","last_peered":"2024-08-31T06:05:25.048879+0000","last_clean":"2024-08-31T06:05:25.048879+0000","last_became_active":"2024-08-31T06:04:22.846057+0000","last_became_peered":"2024-08-31T06:04:22.846057+0000","last_unstale":"2024-08-31T06:05:25.048879+0000","last_undegraded":"2024-08-31T06:05:25.048879+0000","last_fullsized":"2024-08-31T06:05:25.048879+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:29:11.861665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.684568+0000","last_change":"2024-08-31T06:04:25.893777+0000","last_active":"2024-08-31T06:06:23.684568+0000","last_peered":"2024-08-31T06:06:23.684568+0000","last_clean":"2024-08-31T06:06:23.684568+0000","last_became_active":"2024-08-31T06:04:25.893014+0000","last_became_peered":"2024-08-31T06:04:25.893014+0000","last_unstale":"2024-08-31T06:06:23.684568+0000","last_undegraded":"2024-08-31T06:06:23.684568+0000","last_fullsized":"2024-08-31T06:06:23.684568+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:04:48.185842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.045551+0000","last_change":"2024-08-31T06:04:22.965938+0000","last_active":"2024-08-31T06:05:25.045551+0000","last_peered":"2024-08-31T06:05:25.045551+0000","last_clean":"2024-08-31T06:05:25.045551+0000","last_became_active":"2024-08-31T06:04:22.965356+0000","last_became_peered":"2024-08-31T06:04:22.965356+0000","last_unstale":"2024-08-31T06:05:25.045551+0000","last_undegraded":"2024-08-31T06:05:25.045551+0000","last_fullsized":"2024-08-31T06:05:25.045551+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:36:58.938462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"80'11","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.691850+0000","last_change":"2024-08-31T06:04:25.881307+0000","last_active":"2024-08-31T06:06:23.691850+0000","last_peered":"2024-08-31T06:06:23.691850+0000","last_clean":"2024-08-31T06:06:23.691850+0000","last_became_active":"2024-08-31T06:04:25.880418+0000","last_became_peered":"2024-08-31T06:04:25.880418+0000","last_unstale":"2024-08-31T06:06:23.691850+0000","last_undegraded":"2024-08-31T06:06:23.691850+0000","last_fullsized":"2024-08-31T06:06:23.691850+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:21:46.904371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'34","reported_seq":342,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:54.188643+0000","last_change":"2024-08-31T06:04:22.843073+0000","last_active":"2024-08-31T06:06:54.188643+0000","last_peered":"2024-08-31T06:06:54.188643+0000","last_clean":"2024-08-31T06:06:54.188643+0000","last_became_active":"2024-08-31T06:04:22.842645+0000","last_became_peered":"2024-08-31T06:04:22.842645+0000","last_unstale":"2024-08-31T06:06:54.188643+0000","last_undegraded":"2024-08-31T06:06:54.188643+0000","last_fullsized":"2024-08-31T06:06:54.188643+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:41:37.651562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65695,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":281,"num_read_kb":205,"num_write":35,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.680627+0000","last_change":"2024-08-31T06:04:25.879929+0000","last_active":"2024-08-31T06:06:23.680627+0000","last_peered":"2024-08-31T06:06:23.680627+0000","last_clean":"2024-08-31T06:06:23.680627+0000","last_became_active":"2024-08-31T06:04:25.878475+0000","last_became_peered":"2024-08-31T06:04:25.878475+0000","last_unstale":"2024-08-31T06:06:23.680627+0000","last_undegraded":"2024-08-31T06:06:23.680627+0000","last_fullsized":"2024-08-31T06:06:23.680627+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:10:32.801558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048264+0000","last_change":"2024-08-31T06:04:22.842878+0000","last_active":"2024-08-31T06:05:25.048264+0000","last_peered":"2024-08-31T06:05:25.048264+0000","last_clean":"2024-08-31T06:05:25.048264+0000","last_became_active":"2024-08-31T06:04:22.842306+0000","last_became_peered":"2024-08-31T06:04:22.842306+0000","last_unstale":"2024-08-31T06:05:25.048264+0000","last_undegraded":"2024-08-31T06:05:25.048264+0000","last_fullsized":"2024-08-31T06:05:25.048264+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:53:55.259238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.669929+0000","last_change":"2024-08-31T06:04:25.899357+0000","last_active":"2024-08-31T06:06:23.669929+0000","last_peered":"2024-08-31T06:06:23.669929+0000","last_clean":"2024-08-31T06:06:23.669929+0000","last_became_active":"2024-08-31T06:04:25.899000+0000","last_became_peered":"2024-08-31T06:04:25.899000+0000","last_unstale":"2024-08-31T06:06:23.669929+0000","last_undegraded":"2024-08-31T06:06:23.669929+0000","last_fullsized":"2024-08-31T06:06:23.669929+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:20:39.991071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.044759+0000","last_change":"2024-08-31T06:04:22.843063+0000","last_active":"2024-08-31T06:05:25.044759+0000","last_peered":"2024-08-31T06:05:25.044759+0000","last_clean":"2024-08-31T06:05:25.044759+0000","last_became_active":"2024-08-31T06:04:22.842654+0000","last_became_peered":"2024-08-31T06:04:22.842654+0000","last_unstale":"2024-08-31T06:05:25.044759+0000","last_undegraded":"2024-08-31T06:05:25.044759+0000","last_fullsized":"2024-08-31T06:05:25.044759+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:09:54.563116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"80'15","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.683856+0000","last_change":"2024-08-31T06:04:25.885556+0000","last_active":"2024-08-31T06:06:23.683856+0000","last_peered":"2024-08-31T06:06:23.683856+0000","last_clean":"2024-08-31T06:06:23.683856+0000","last_became_active":"2024-08-31T06:04:25.885216+0000","last_became_peered":"2024-08-31T06:04:25.885216+0000","last_unstale":"2024-08-31T06:06:23.683856+0000","last_undegraded":"2024-08-31T06:06:23.683856+0000","last_fullsized":"2024-08-31T06:06:23.683856+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:48:49.816261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'5","reported_seq":170,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:52.747883+0000","last_change":"2024-08-31T06:04:22.844839+0000","last_active":"2024-08-31T06:06:52.747883+0000","last_peered":"2024-08-31T06:06:52.747883+0000","last_clean":"2024-08-31T06:06:52.747883+0000","last_became_active":"2024-08-31T06:04:22.844425+0000","last_became_peered":"2024-08-31T06:04:22.844425+0000","last_unstale":"2024-08-31T06:06:52.747883+0000","last_undegraded":"2024-08-31T06:06:52.747883+0000","last_fullsized":"2024-08-31T06:06:52.747883+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:10:57.292173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":139,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.682394+0000","last_change":"2024-08-31T06:04:25.878195+0000","last_active":"2024-08-31T06:06:23.682394+0000","last_peered":"2024-08-31T06:06:23.682394+0000","last_clean":"2024-08-31T06:06:23.682394+0000","last_became_active":"2024-08-31T06:04:25.877823+0000","last_became_peered":"2024-08-31T06:04:25.877823+0000","last_unstale":"2024-08-31T06:06:23.682394+0000","last_undegraded":"2024-08-31T06:06:23.682394+0000","last_fullsized":"2024-08-31T06:06:23.682394+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:54:34.989583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":68,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.071826+0000","last_change":"2024-08-31T06:05:25.071391+0000","last_active":"2024-08-31T06:05:25.071826+0000","last_peered":"2024-08-31T06:05:25.071826+0000","last_clean":"2024-08-31T06:05:25.071826+0000","last_became_active":"2024-08-31T06:05:25.057546+0000","last_became_peered":"2024-08-31T06:05:25.057546+0000","last_unstale":"2024-08-31T06:05:25.071826+0000","last_undegraded":"2024-08-31T06:05:25.071826+0000","last_fullsized":"2024-08-31T06:05:25.071826+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:42:35.527395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.681995+0000","last_change":"2024-08-31T06:04:25.887580+0000","last_active":"2024-08-31T06:06:23.681995+0000","last_peered":"2024-08-31T06:06:23.681995+0000","last_clean":"2024-08-31T06:06:23.681995+0000","last_became_active":"2024-08-31T06:04:25.887158+0000","last_became_peered":"2024-08-31T06:04:25.887158+0000","last_unstale":"2024-08-31T06:06:23.681995+0000","last_undegraded":"2024-08-31T06:06:23.681995+0000","last_fullsized":"2024-08-31T06:06:23.681995+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:27:38.115742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.064443+0000","last_change":"2024-08-31T06:05:25.063859+0000","last_active":"2024-08-31T06:05:25.064443+0000","last_peered":"2024-08-31T06:05:25.064443+0000","last_clean":"2024-08-31T06:05:25.064443+0000","last_became_active":"2024-08-31T06:05:25.053596+0000","last_became_peered":"2024-08-31T06:05:25.053596+0000","last_unstale":"2024-08-31T06:05:25.064443+0000","last_undegraded":"2024-08-31T06:05:25.064443+0000","last_fullsized":"2024-08-31T06:05:25.064443+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:19:18.461630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.678982+0000","last_change":"2024-08-31T06:04:25.880845+0000","last_active":"2024-08-31T06:06:23.678982+0000","last_peered":"2024-08-31T06:06:23.678982+0000","last_clean":"2024-08-31T06:06:23.678982+0000","last_became_active":"2024-08-31T06:04:25.880423+0000","last_became_peered":"2024-08-31T06:04:25.880423+0000","last_unstale":"2024-08-31T06:06:23.678982+0000","last_undegraded":"2024-08-31T06:06:23.678982+0000","last_fullsized":"2024-08-31T06:06:23.678982+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:16:56.725116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.046584+0000","last_change":"2024-08-31T06:04:22.843393+0000","last_active":"2024-08-31T06:05:25.046584+0000","last_peered":"2024-08-31T06:05:25.046584+0000","last_clean":"2024-08-31T06:05:25.046584+0000","last_became_active":"2024-08-31T06:04:22.842770+0000","last_became_peered":"2024-08-31T06:04:22.842770+0000","last_unstale":"2024-08-31T06:05:25.046584+0000","last_undegraded":"2024-08-31T06:05:25.046584+0000","last_fullsized":"2024-08-31T06:05:25.046584+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:49:16.159167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.686211+0000","last_change":"2024-08-31T06:04:25.887829+0000","last_active":"2024-08-31T06:06:23.686211+0000","last_peered":"2024-08-31T06:06:23.686211+0000","last_clean":"2024-08-31T06:06:23.686211+0000","last_became_active":"2024-08-31T06:04:25.886567+0000","last_became_peered":"2024-08-31T06:04:25.886567+0000","last_unstale":"2024-08-31T06:06:23.686211+0000","last_undegraded":"2024-08-31T06:06:23.686211+0000","last_fullsized":"2024-08-31T06:06:23.686211+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:11:39.054966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'60","reported_seq":117,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:50.541212+0000","last_change":"2024-08-31T06:04:22.844218+0000","last_active":"2024-08-31T06:06:50.541212+0000","last_peered":"2024-08-31T06:06:50.541212+0000","last_clean":"2024-08-31T06:06:50.541212+0000","last_became_active":"2024-08-31T06:04:22.843786+0000","last_became_peered":"2024-08-31T06:04:22.843786+0000","last_unstale":"2024-08-31T06:06:50.541212+0000","last_undegraded":"2024-08-31T06:06:50.541212+0000","last_fullsized":"2024-08-31T06:06:50.541212+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:46:02.534190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393846,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":32,"num_write":71,"num_write_kb":873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.682380+0000","last_change":"2024-08-31T06:04:25.888946+0000","last_active":"2024-08-31T06:06:23.682380+0000","last_peered":"2024-08-31T06:06:23.682380+0000","last_clean":"2024-08-31T06:06:23.682380+0000","last_became_active":"2024-08-31T06:04:25.888564+0000","last_became_peered":"2024-08-31T06:04:25.888564+0000","last_unstale":"2024-08-31T06:06:23.682380+0000","last_undegraded":"2024-08-31T06:06:23.682380+0000","last_fullsized":"2024-08-31T06:06:23.682380+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:26:19.051971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.049211+0000","last_change":"2024-08-31T06:04:22.964483+0000","last_active":"2024-08-31T06:05:25.049211+0000","last_peered":"2024-08-31T06:05:25.049211+0000","last_clean":"2024-08-31T06:05:25.049211+0000","last_became_active":"2024-08-31T06:04:22.963874+0000","last_became_peered":"2024-08-31T06:04:22.963874+0000","last_unstale":"2024-08-31T06:05:25.049211+0000","last_undegraded":"2024-08-31T06:05:25.049211+0000","last_fullsized":"2024-08-31T06:05:25.049211+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:30:00.159040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.688990+0000","last_change":"2024-08-31T06:04:25.880941+0000","last_active":"2024-08-31T06:06:23.688990+0000","last_peered":"2024-08-31T06:06:23.688990+0000","last_clean":"2024-08-31T06:06:23.688990+0000","last_became_active":"2024-08-31T06:04:25.880454+0000","last_became_peered":"2024-08-31T06:04:25.880454+0000","last_unstale":"2024-08-31T06:06:23.688990+0000","last_undegraded":"2024-08-31T06:06:23.688990+0000","last_fullsized":"2024-08-31T06:06:23.688990+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:49:57.023988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":98,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048527+0000","last_change":"2024-08-31T06:03:05.921629+0000","last_active":"2024-08-31T06:05:25.048527+0000","last_peered":"2024-08-31T06:05:25.048527+0000","last_clean":"2024-08-31T06:05:25.048527+0000","last_became_active":"2024-08-31T06:03:05.915844+0000","last_became_peered":"2024-08-31T06:03:05.915844+0000","last_unstale":"2024-08-31T06:05:25.048527+0000","last_undegraded":"2024-08-31T06:05:25.048527+0000","last_fullsized":"2024-08-31T06:05:25.048527+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T05:59:24.472429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T05:59:24.472429+0000","last_clean_scrub_stamp":"2024-08-31T05:59:24.472429+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:56:41.073422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.044718+0000","last_change":"2024-08-31T06:04:22.846325+0000","last_active":"2024-08-31T06:05:25.044718+0000","last_peered":"2024-08-31T06:05:25.044718+0000","last_clean":"2024-08-31T06:05:25.044718+0000","last_became_active":"2024-08-31T06:04:22.845772+0000","last_became_peered":"2024-08-31T06:04:22.845772+0000","last_unstale":"2024-08-31T06:05:25.044718+0000","last_undegraded":"2024-08-31T06:05:25.044718+0000","last_fullsized":"2024-08-31T06:05:25.044718+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:54:10.336786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.691294+0000","last_change":"2024-08-31T06:04:25.891437+0000","last_active":"2024-08-31T06:06:23.691294+0000","last_peered":"2024-08-31T06:06:23.691294+0000","last_clean":"2024-08-31T06:06:23.691294+0000","last_became_active":"2024-08-31T06:04:25.891053+0000","last_became_peered":"2024-08-31T06:04:25.891053+0000","last_unstale":"2024-08-31T06:06:23.691294+0000","last_undegraded":"2024-08-31T06:06:23.691294+0000","last_fullsized":"2024-08-31T06:06:23.691294+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:22:37.119483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048411+0000","last_change":"2024-08-31T06:04:22.842885+0000","last_active":"2024-08-31T06:05:25.048411+0000","last_peered":"2024-08-31T06:05:25.048411+0000","last_clean":"2024-08-31T06:05:25.048411+0000","last_became_active":"2024-08-31T06:04:22.842473+0000","last_became_peered":"2024-08-31T06:04:22.842473+0000","last_unstale":"2024-08-31T06:05:25.048411+0000","last_undegraded":"2024-08-31T06:05:25.048411+0000","last_fullsized":"2024-08-31T06:05:25.048411+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:33:36.516659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.663699+0000","last_change":"2024-08-31T06:04:25.889213+0000","last_active":"2024-08-31T06:06:23.663699+0000","last_peered":"2024-08-31T06:06:23.663699+0000","last_clean":"2024-08-31T06:06:23.663699+0000","last_became_active":"2024-08-31T06:04:25.888860+0000","last_became_peered":"2024-08-31T06:04:25.888860+0000","last_unstale":"2024-08-31T06:06:23.663699+0000","last_undegraded":"2024-08-31T06:06:23.663699+0000","last_fullsized":"2024-08-31T06:06:23.663699+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:44:18.831229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":167,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:53.748738+0000","last_change":"2024-08-31T06:04:22.966091+0000","last_active":"2024-08-31T06:06:53.748738+0000","last_peered":"2024-08-31T06:06:53.748738+0000","last_clean":"2024-08-31T06:06:53.748738+0000","last_became_active":"2024-08-31T06:04:22.965727+0000","last_became_peered":"2024-08-31T06:04:22.965727+0000","last_unstale":"2024-08-31T06:06:53.748738+0000","last_undegraded":"2024-08-31T06:06:53.748738+0000","last_fullsized":"2024-08-31T06:06:53.748738+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:42:43.643623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":135,"num_read_kb":135,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.682138+0000","last_change":"2024-08-31T06:04:25.880039+0000","last_active":"2024-08-31T06:06:23.682138+0000","last_peered":"2024-08-31T06:06:23.682138+0000","last_clean":"2024-08-31T06:06:23.682138+0000","last_became_active":"2024-08-31T06:04:25.879669+0000","last_became_peered":"2024-08-31T06:04:25.879669+0000","last_unstale":"2024-08-31T06:06:23.682138+0000","last_undegraded":"2024-08-31T06:06:23.682138+0000","last_fullsized":"2024-08-31T06:06:23.682138+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:02:09.244281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048687+0000","last_change":"2024-08-31T06:04:22.844572+0000","last_active":"2024-08-31T06:05:25.048687+0000","last_peered":"2024-08-31T06:05:25.048687+0000","last_clean":"2024-08-31T06:05:25.048687+0000","last_became_active":"2024-08-31T06:04:22.844227+0000","last_became_peered":"2024-08-31T06:04:22.844227+0000","last_unstale":"2024-08-31T06:05:25.048687+0000","last_undegraded":"2024-08-31T06:05:25.048687+0000","last_fullsized":"2024-08-31T06:05:25.048687+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:04:51.966901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.677043+0000","last_change":"2024-08-31T06:04:25.884442+0000","last_active":"2024-08-31T06:06:23.677043+0000","last_peered":"2024-08-31T06:06:23.677043+0000","last_clean":"2024-08-31T06:06:23.677043+0000","last_became_active":"2024-08-31T06:04:25.884016+0000","last_became_peered":"2024-08-31T06:04:25.884016+0000","last_unstale":"2024-08-31T06:06:23.677043+0000","last_undegraded":"2024-08-31T06:06:23.677043+0000","last_fullsized":"2024-08-31T06:06:23.677043+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:36:10.640262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.043831+0000","last_change":"2024-08-31T06:04:22.967845+0000","last_active":"2024-08-31T06:05:25.043831+0000","last_peered":"2024-08-31T06:05:25.043831+0000","last_clean":"2024-08-31T06:05:25.043831+0000","last_became_active":"2024-08-31T06:04:22.967421+0000","last_became_peered":"2024-08-31T06:04:22.967421+0000","last_unstale":"2024-08-31T06:05:25.043831+0000","last_undegraded":"2024-08-31T06:05:25.043831+0000","last_fullsized":"2024-08-31T06:05:25.043831+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:17:46.942551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.673410+0000","last_change":"2024-08-31T06:04:25.897087+0000","last_active":"2024-08-31T06:06:23.673410+0000","last_peered":"2024-08-31T06:06:23.673410+0000","last_clean":"2024-08-31T06:06:23.673410+0000","last_became_active":"2024-08-31T06:04:25.896725+0000","last_became_peered":"2024-08-31T06:04:25.896725+0000","last_unstale":"2024-08-31T06:06:23.673410+0000","last_undegraded":"2024-08-31T06:06:23.673410+0000","last_fullsized":"2024-08-31T06:06:23.673410+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:31:28.279489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.053673+0000","last_change":"2024-08-31T06:04:22.965299+0000","last_active":"2024-08-31T06:05:25.053673+0000","last_peered":"2024-08-31T06:05:25.053673+0000","last_clean":"2024-08-31T06:05:25.053673+0000","last_became_active":"2024-08-31T06:04:22.964849+0000","last_became_peered":"2024-08-31T06:04:22.964849+0000","last_unstale":"2024-08-31T06:05:25.053673+0000","last_undegraded":"2024-08-31T06:05:25.053673+0000","last_fullsized":"2024-08-31T06:05:25.053673+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:36:56.431678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.679635+0000","last_change":"2024-08-31T06:04:25.885115+0000","last_active":"2024-08-31T06:06:23.679635+0000","last_peered":"2024-08-31T06:06:23.679635+0000","last_clean":"2024-08-31T06:06:23.679635+0000","last_became_active":"2024-08-31T06:04:25.884722+0000","last_became_peered":"2024-08-31T06:04:25.884722+0000","last_unstale":"2024-08-31T06:06:23.679635+0000","last_undegraded":"2024-08-31T06:06:23.679635+0000","last_fullsized":"2024-08-31T06:06:23.679635+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:20:43.913785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'97","reported_seq":214,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:51.071411+0000","last_change":"2024-08-31T06:04:22.845119+0000","last_active":"2024-08-31T06:06:51.071411+0000","last_peered":"2024-08-31T06:06:51.071411+0000","last_clean":"2024-08-31T06:06:51.071411+0000","last_became_active":"2024-08-31T06:04:22.843814+0000","last_became_peered":"2024-08-31T06:04:22.843814+0000","last_unstale":"2024-08-31T06:06:51.071411+0000","last_undegraded":"2024-08-31T06:06:51.071411+0000","last_fullsized":"2024-08-31T06:06:51.071411+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:06:40.684887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":302,"num_read_kb":268,"num_write":179,"num_write_kb":1222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.680844+0000","last_change":"2024-08-31T06:04:25.892838+0000","last_active":"2024-08-31T06:06:23.680844+0000","last_peered":"2024-08-31T06:06:23.680844+0000","last_clean":"2024-08-31T06:06:23.680844+0000","last_became_active":"2024-08-31T06:04:25.891955+0000","last_became_peered":"2024-08-31T06:04:25.891955+0000","last_unstale":"2024-08-31T06:06:23.680844+0000","last_undegraded":"2024-08-31T06:06:23.680844+0000","last_fullsized":"2024-08-31T06:06:23.680844+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:41:35.085724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.044957+0000","last_change":"2024-08-31T06:04:22.964183+0000","last_active":"2024-08-31T06:05:25.044957+0000","last_peered":"2024-08-31T06:05:25.044957+0000","last_clean":"2024-08-31T06:05:25.044957+0000","last_became_active":"2024-08-31T06:04:22.963688+0000","last_became_peered":"2024-08-31T06:04:22.963688+0000","last_unstale":"2024-08-31T06:05:25.044957+0000","last_undegraded":"2024-08-31T06:05:25.044957+0000","last_fullsized":"2024-08-31T06:05:25.044957+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:04:47.833811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.683320+0000","last_change":"2024-08-31T06:04:25.892266+0000","last_active":"2024-08-31T06:06:23.683320+0000","last_peered":"2024-08-31T06:06:23.683320+0000","last_clean":"2024-08-31T06:06:23.683320+0000","last_became_active":"2024-08-31T06:04:25.889014+0000","last_became_peered":"2024-08-31T06:04:25.889014+0000","last_unstale":"2024-08-31T06:06:23.683320+0000","last_undegraded":"2024-08-31T06:06:23.683320+0000","last_fullsized":"2024-08-31T06:06:23.683320+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T08:27:47.608470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.049408+0000","last_change":"2024-08-31T06:04:22.964677+0000","last_active":"2024-08-31T06:05:25.049408+0000","last_peered":"2024-08-31T06:05:25.049408+0000","last_clean":"2024-08-31T06:05:25.049408+0000","last_became_active":"2024-08-31T06:04:22.964237+0000","last_became_peered":"2024-08-31T06:04:22.964237+0000","last_unstale":"2024-08-31T06:05:25.049408+0000","last_undegraded":"2024-08-31T06:05:25.049408+0000","last_fullsized":"2024-08-31T06:05:25.049408+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:00:54.099592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.648239+0000","last_change":"2024-08-31T06:04:25.881699+0000","last_active":"2024-08-31T06:06:23.648239+0000","last_peered":"2024-08-31T06:06:23.648239+0000","last_clean":"2024-08-31T06:06:23.648239+0000","last_became_active":"2024-08-31T06:04:25.879304+0000","last_became_peered":"2024-08-31T06:04:25.879304+0000","last_unstale":"2024-08-31T06:06:23.648239+0000","last_undegraded":"2024-08-31T06:06:23.648239+0000","last_fullsized":"2024-08-31T06:06:23.648239+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T10:56:42.992034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.108301+0000","last_change":"2024-08-31T06:04:22.847934+0000","last_active":"2024-08-31T06:05:25.108301+0000","last_peered":"2024-08-31T06:05:25.108301+0000","last_clean":"2024-08-31T06:05:25.108301+0000","last_became_active":"2024-08-31T06:04:22.847518+0000","last_became_peered":"2024-08-31T06:04:22.847518+0000","last_unstale":"2024-08-31T06:05:25.108301+0000","last_undegraded":"2024-08-31T06:05:25.108301+0000","last_fullsized":"2024-08-31T06:05:25.108301+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:26:21.476014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.685828+0000","last_change":"2024-08-31T06:04:25.884653+0000","last_active":"2024-08-31T06:06:23.685828+0000","last_peered":"2024-08-31T06:06:23.685828+0000","last_clean":"2024-08-31T06:06:23.685828+0000","last_became_active":"2024-08-31T06:04:25.884244+0000","last_became_peered":"2024-08-31T06:04:25.884244+0000","last_unstale":"2024-08-31T06:06:23.685828+0000","last_undegraded":"2024-08-31T06:06:23.685828+0000","last_fullsized":"2024-08-31T06:06:23.685828+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T16:13:00.814071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.048648+0000","last_change":"2024-08-31T06:04:22.845924+0000","last_active":"2024-08-31T06:05:25.048648+0000","last_peered":"2024-08-31T06:05:25.048648+0000","last_clean":"2024-08-31T06:05:25.048648+0000","last_became_active":"2024-08-31T06:04:22.845445+0000","last_became_peered":"2024-08-31T06:04:22.845445+0000","last_unstale":"2024-08-31T06:05:25.048648+0000","last_undegraded":"2024-08-31T06:05:25.048648+0000","last_fullsized":"2024-08-31T06:05:25.048648+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:25:21.052136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"80'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.679162+0000","last_change":"2024-08-31T06:04:25.890069+0000","last_active":"2024-08-31T06:06:23.679162+0000","last_peered":"2024-08-31T06:06:23.679162+0000","last_clean":"2024-08-31T06:06:23.679162+0000","last_became_active":"2024-08-31T06:04:25.889709+0000","last_became_peered":"2024-08-31T06:04:25.889709+0000","last_unstale":"2024-08-31T06:06:23.679162+0000","last_undegraded":"2024-08-31T06:06:23.679162+0000","last_fullsized":"2024-08-31T06:06:23.679162+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:13:51.360023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.051611+0000","last_change":"2024-08-31T06:04:22.847845+0000","last_active":"2024-08-31T06:05:25.051611+0000","last_peered":"2024-08-31T06:05:25.051611+0000","last_clean":"2024-08-31T06:05:25.051611+0000","last_became_active":"2024-08-31T06:04:22.847001+0000","last_became_peered":"2024-08-31T06:04:22.847001+0000","last_unstale":"2024-08-31T06:05:25.051611+0000","last_undegraded":"2024-08-31T06:05:25.051611+0000","last_fullsized":"2024-08-31T06:05:25.051611+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T11:24:34.209076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.690813+0000","last_change":"2024-08-31T06:04:25.886759+0000","last_active":"2024-08-31T06:06:23.690813+0000","last_peered":"2024-08-31T06:06:23.690813+0000","last_clean":"2024-08-31T06:06:23.690813+0000","last_became_active":"2024-08-31T06:04:25.886176+0000","last_became_peered":"2024-08-31T06:04:25.886176+0000","last_unstale":"2024-08-31T06:06:23.690813+0000","last_undegraded":"2024-08-31T06:06:23.690813+0000","last_fullsized":"2024-08-31T06:06:23.690813+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T09:11:12.637451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"80'15","reported_seq":43,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.677867+0000","last_change":"2024-08-31T06:05:25.054468+0000","last_active":"2024-08-31T06:06:23.677867+0000","last_peered":"2024-08-31T06:06:23.677867+0000","last_clean":"2024-08-31T06:06:23.677867+0000","last_became_active":"2024-08-31T06:05:25.053657+0000","last_became_peered":"2024-08-31T06:05:25.053657+0000","last_unstale":"2024-08-31T06:06:23.677867+0000","last_undegraded":"2024-08-31T06:06:23.677867+0000","last_fullsized":"2024-08-31T06:06:23.677867+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T14:43:44.887637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.059757+0000","last_change":"2024-08-31T06:05:25.059219+0000","last_active":"2024-08-31T06:05:25.059757+0000","last_peered":"2024-08-31T06:05:25.059757+0000","last_clean":"2024-08-31T06:05:25.059757+0000","last_became_active":"2024-08-31T06:05:25.053136+0000","last_became_peered":"2024-08-31T06:05:25.053136+0000","last_unstale":"2024-08-31T06:05:25.059757+0000","last_undegraded":"2024-08-31T06:05:25.059757+0000","last_fullsized":"2024-08-31T06:05:25.059757+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:29:59.137465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"80'9","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.691799+0000","last_change":"2024-08-31T06:04:25.887524+0000","last_active":"2024-08-31T06:06:23.691799+0000","last_peered":"2024-08-31T06:06:23.691799+0000","last_clean":"2024-08-31T06:06:23.691799+0000","last_became_active":"2024-08-31T06:04:25.887176+0000","last_became_peered":"2024-08-31T06:04:25.887176+0000","last_unstale":"2024-08-31T06:06:23.691799+0000","last_undegraded":"2024-08-31T06:06:23.691799+0000","last_fullsized":"2024-08-31T06:06:23.691799+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T06:08:05.476804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.053583+0000","last_change":"2024-08-31T06:04:22.847747+0000","last_active":"2024-08-31T06:05:25.053583+0000","last_peered":"2024-08-31T06:05:25.053583+0000","last_clean":"2024-08-31T06:05:25.053583+0000","last_became_active":"2024-08-31T06:04:22.847332+0000","last_became_peered":"2024-08-31T06:04:22.847332+0000","last_unstale":"2024-08-31T06:05:25.053583+0000","last_undegraded":"2024-08-31T06:05:25.053583+0000","last_fullsized":"2024-08-31T06:05:25.053583+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T15:41:35.910991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.687796+0000","last_change":"2024-08-31T06:04:25.883110+0000","last_active":"2024-08-31T06:06:23.687796+0000","last_peered":"2024-08-31T06:06:23.687796+0000","last_clean":"2024-08-31T06:06:23.687796+0000","last_became_active":"2024-08-31T06:04:25.880487+0000","last_became_peered":"2024-08-31T06:04:25.880487+0000","last_unstale":"2024-08-31T06:06:23.687796+0000","last_undegraded":"2024-08-31T06:06:23.687796+0000","last_fullsized":"2024-08-31T06:06:23.687796+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T07:22:13.656078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:05:25.108210+0000","last_change":"2024-08-31T06:04:22.964758+0000","last_active":"2024-08-31T06:05:25.108210+0000","last_peered":"2024-08-31T06:05:25.108210+0000","last_clean":"2024-08-31T06:05:25.108210+0000","last_became_active":"2024-08-31T06:04:22.964391+0000","last_became_peered":"2024-08-31T06:04:22.964391+0000","last_unstale":"2024-08-31T06:05:25.108210+0000","last_undegraded":"2024-08-31T06:05:25.108210+0000","last_fullsized":"2024-08-31T06:05:25.108210+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T13:28:59.190657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:23.678294+0000","last_change":"2024-08-31T06:04:25.893738+0000","last_active":"2024-08-31T06:06:23.678294+0000","last_peered":"2024-08-31T06:06:23.678294+0000","last_clean":"2024-08-31T06:06:23.678294+0000","last_became_active":"2024-08-31T06:04:25.893308+0000","last_became_peered":"2024-08-31T06:04:25.893308+0000","last_unstale":"2024-08-31T06:06:23.678294+0000","last_undegraded":"2024-08-31T06:06:23.678294+0000","last_fullsized":"2024-08-31T06:06:23.678294+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:24.842836+0000","last_clean_scrub_stamp":"2024-08-31T06:04:24.842836+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T12:26:48.451894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":39,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-08-31T06:06:27.962460+0000","last_change":"2024-08-31T06:04:22.843346+0000","last_active":"2024-08-31T06:06:27.962460+0000","last_peered":"2024-08-31T06:06:27.962460+0000","last_clean":"2024-08-31T06:06:27.962460+0000","last_became_active":"2024-08-31T06:04:22.842249+0000","last_became_peered":"2024-08-31T06:04:22.842249+0000","last_unstale":"2024-08-31T06:06:27.962460+0000","last_undegraded":"2024-08-31T06:06:27.962460+0000","last_fullsized":"2024-08-31T06:06:27.962460+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-31T06:04:21.821443+0000","last_clean_scrub_stamp":"2024-08-31T06:04:21.821443+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-01T17:40:57.618940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":857361,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1168,"num_read_kb":795,"num_write":402,"num_write_kb":2214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1147,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":663552,"data_stored":2572083,"data_compressed":112140,"data_compressed_allocated":147456,"data_compressed_original":2285568,"omap_allocated":0,"internal_metadata":0},"log_size":294,"ondisk_log_size":294,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":249373,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710764,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":436463,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874288,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1077248,"data_stored":1487925,"data_compressed":60215,"data_compressed_allocated":77824,"data_compressed_original":974848,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037815,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":829893,"data_compressed":25094,"data_compressed_allocated":32768,"data_compressed_original":409600,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234045,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":829791,"data_compressed":25094,"data_compressed_allocated":32768,"data_compressed_original":409600,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397571,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":933888,"data_stored":1029200,"data_compressed":42124,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593800,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":436066,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790028,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":999424,"data_stored":1422128,"data_compressed":60215,"data_compressed_allocated":77824,"data_compressed_original":974848,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953555,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36208,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708944,"statfs":{"total":95995035648,"available":95957958656,"internally_reserved":0,"allocated":884736,"data_stored":438189,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149782,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":966656,"data_stored":1223369,"data_compressed":43185,"data_compressed_allocated":57344,"data_compressed_original":778240,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411421,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36268,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708884,"statfs":{"total":95995035648,"available":95957897216,"internally_reserved":0,"allocated":946176,"data_stored":502170,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542242,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36268,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708884,"statfs":{"total":95995035648,"available":95957897216,"internally_reserved":0,"allocated":946176,"data_stored":501666,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705768,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":892928,"data_stored":437337,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34699,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34699,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34699,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36319,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":65807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":66311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":787276,"data_compressed":37380,"data_compressed_allocated":49152,"data_compressed_original":761856,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":395784,"data_compressed":18091,"data_compressed_allocated":24576,"data_compressed_original":368640,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393932,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":159744,"data_stored":461581,"data_compressed":18091,"data_compressed_allocated":24576,"data_compressed_original":368640,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-31T06:06:58.145 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:06:58.526 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:06:58.999 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:58 smithi016 ceph-mon[17194]: pgmap v355: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 174 KiB/s wr, 51 op/s 2024-08-31T06:06:58.999 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:58 smithi016 ceph-mon[17194]: from='client.25393 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T06:06:59.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.175+0000 7f8f2be00640 1 -- 172.21.15.16:0/1391404738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f241141d0 msgr2=0x7f8f241166a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:59.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.175+0000 7f8f2be00640 1 --2- 172.21.15.16:0/1391404738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f241141d0 0x7f8f241166a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c009ca0 tx=0x7f8f1c02a8c0 comp rx=0 tx=0).stop 2024-08-31T06:06:59.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.175+0000 7f8f2be00640 1 -- 172.21.15.16:0/1391404738 shutdown_connections 2024-08-31T06:06:59.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.175+0000 7f8f2be00640 1 --2- 172.21.15.16:0/1391404738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f241141d0 0x7f8f241166a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:59.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.175+0000 7f8f2be00640 1 --2- 172.21.15.16:0/1391404738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f24075b70 0x7f8f24075f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:59.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.175+0000 7f8f2be00640 1 --2- 172.21.15.16:0/1391404738 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f24077120 0x7f8f24075550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:59.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.175+0000 7f8f2be00640 1 -- 172.21.15.16:0/1391404738 >> 172.21.15.16:0/1391404738 conn(0x7f8f2407b5f0 msgr2=0x7f8f2407ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:59.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.175+0000 7f8f2be00640 1 -- 172.21.15.16:0/1391404738 shutdown_connections 2024-08-31T06:06:59.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.175+0000 7f8f2be00640 1 -- 172.21.15.16:0/1391404738 wait complete. 2024-08-31T06:06:59.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.176+0000 7f8f2be00640 1 Processor -- start 2024-08-31T06:06:59.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.176+0000 7f8f2be00640 1 -- start start 2024-08-31T06:06:59.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.176+0000 7f8f2be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f24075b70 0x7f8f241a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:59.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.176+0000 7f8f2be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f24077120 0x7f8f241a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:59.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.176+0000 7f8f2be00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f241141d0 0x7f8f241a5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:59.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.177+0000 7f8f2be00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8f24119e60 con 0x7f8f24075b70 2024-08-31T06:06:59.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.177+0000 7f8f2be00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8f24119ce0 con 0x7f8f24077120 2024-08-31T06:06:59.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.177+0000 7f8f2be00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8f24119fe0 con 0x7f8f241141d0 2024-08-31T06:06:59.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.177+0000 7f8f29400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f24075b70 0x7f8f241a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:59.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.177+0000 7f8f28a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f24077120 0x7f8f241a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:59.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.177+0000 7f8f28a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f24077120 0x7f8f241a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:54500/0 (socket says 172.21.15.16:54500) 2024-08-31T06:06:59.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.177+0000 7f8f29400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f24075b70 0x7f8f241a0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60062/0 (socket says 172.21.15.16:60062) 2024-08-31T06:06:59.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.177+0000 7f8f28a00640 1 -- 172.21.15.16:0/3970643910 learned_addr learned my addr 172.21.15.16:0/3970643910 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:06:59.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.177+0000 7f8f29e00640 1 --2- 172.21.15.16:0/3970643910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f241141d0 0x7f8f241a5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:59.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.177+0000 7f8f28a00640 1 -- 172.21.15.16:0/3970643910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f241141d0 msgr2=0x7f8f241a5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:59.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.177+0000 7f8f28a00640 1 --2- 172.21.15.16:0/3970643910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f241141d0 0x7f8f241a5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:59.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.177+0000 7f8f28a00640 1 -- 172.21.15.16:0/3970643910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f24075b70 msgr2=0x7f8f241a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:59.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.177+0000 7f8f28a00640 1 --2- 172.21.15.16:0/3970643910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f24075b70 0x7f8f241a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:59.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.177+0000 7f8f28a00640 1 -- 172.21.15.16:0/3970643910 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f241a6700 con 0x7f8f24077120 2024-08-31T06:06:59.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.178+0000 7f8f28a00640 1 --2- 172.21.15.16:0/3970643910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f24077120 0x7f8f241a0d30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8f1000e9c0 tx=0x7f8f1000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:59.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.178+0000 7f8f22000640 1 -- 172.21.15.16:0/3970643910 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1000cd80 con 0x7f8f24077120 2024-08-31T06:06:59.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.178+0000 7f8f2be00640 1 -- 172.21.15.16:0/3970643910 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f241a69f0 con 0x7f8f24077120 2024-08-31T06:06:59.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.178+0000 7f8f22000640 1 -- 172.21.15.16:0/3970643910 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f100054d0 con 0x7f8f24077120 2024-08-31T06:06:59.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.178+0000 7f8f22000640 1 -- 172.21.15.16:0/3970643910 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f10010430 con 0x7f8f24077120 2024-08-31T06:06:59.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.178+0000 7f8f2be00640 1 -- 172.21.15.16:0/3970643910 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f241ad150 con 0x7f8f24077120 2024-08-31T06:06:59.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.179+0000 7f8f2be00640 1 -- 172.21.15.16:0/3970643910 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f24075f80 con 0x7f8f24077120 2024-08-31T06:06:59.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.182+0000 7f8f22000640 1 -- 172.21.15.16:0/3970643910 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 23) ==== 152651+0+0 (secure 0 0 0) 0x7f8f10004fb0 con 0x7f8f24077120 2024-08-31T06:06:59.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.183+0000 7f8f22000640 1 --2- 172.21.15.16:0/3970643910 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f8f0c0b4c10 0x7f8f0c0b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:06:59.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.183+0000 7f8f22000640 1 -- 172.21.15.16:0/3970643910 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f8f100e5a70 con 0x7f8f24077120 2024-08-31T06:06:59.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.184+0000 7f8f29400640 1 --2- 172.21.15.16:0/3970643910 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f8f0c0b4c10 0x7f8f0c0b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:06:59.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.184+0000 7f8f29400640 1 --2- 172.21.15.16:0/3970643910 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f8f0c0b4c10 0x7f8f0c0b7120 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8f18005fd0 tx=0x7f8f1800b390 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:06:59.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.185+0000 7f8f22000640 1 -- 172.21.15.16:0/3970643910 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f8f10056130 con 0x7f8f24077120 2024-08-31T06:06:59.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:58 smithi184 ceph-mon[22331]: pgmap v355: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 174 KiB/s wr, 51 op/s 2024-08-31T06:06:59.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:58 smithi184 ceph-mon[22331]: from='client.25393 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T06:06:59.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:58 smithi155 ceph-mon[21742]: pgmap v355: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 174 KiB/s wr, 51 op/s 2024-08-31T06:06:59.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:58 smithi155 ceph-mon[21742]: from='client.25393 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-31T06:06:59.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.420+0000 7f8f2be00640 1 -- 172.21.15.16:0/3970643910 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f241a18c0 con 0x7f8f24077120 2024-08-31T06:06:59.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.421+0000 7f8f22000640 1 -- 172.21.15.16:0/3970643910 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11917 (secure 0 0 0) 0x7f8f100ac920 con 0x7f8f24077120 2024-08-31T06:06:59.424 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:06:59.424 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-08-31T06:06:38:485237+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:06:37.778867+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24698":{"gid":24698,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1089710405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1089710405},{"type":"v1","addr":"172.21.15.155:6837","nonce":1089710405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24737,24724,24712]},"id":1}]} 2024-08-31T06:06:59.424 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-08-31T06:06:59.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.424+0000 7f8f2be00640 1 -- 172.21.15.16:0/3970643910 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f8f0c0b4c10 msgr2=0x7f8f0c0b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:59.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.424+0000 7f8f2be00640 1 --2- 172.21.15.16:0/3970643910 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f8f0c0b4c10 0x7f8f0c0b7120 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8f18005fd0 tx=0x7f8f1800b390 comp rx=0 tx=0).stop 2024-08-31T06:06:59.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.424+0000 7f8f2be00640 1 -- 172.21.15.16:0/3970643910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f24077120 msgr2=0x7f8f241a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:06:59.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.424+0000 7f8f2be00640 1 --2- 172.21.15.16:0/3970643910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f24077120 0x7f8f241a0d30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8f1000e9c0 tx=0x7f8f1000ee80 comp rx=0 tx=0).stop 2024-08-31T06:06:59.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.424+0000 7f8f2be00640 1 -- 172.21.15.16:0/3970643910 shutdown_connections 2024-08-31T06:06:59.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.424+0000 7f8f2be00640 1 --2- 172.21.15.16:0/3970643910 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f8f0c0b4c10 0x7f8f0c0b7120 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:59.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.424+0000 7f8f2be00640 1 --2- 172.21.15.16:0/3970643910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f241141d0 0x7f8f241a5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:59.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.424+0000 7f8f2be00640 1 --2- 172.21.15.16:0/3970643910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f24077120 0x7f8f241a0d30 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:59.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.424+0000 7f8f2be00640 1 --2- 172.21.15.16:0/3970643910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f24075b70 0x7f8f241a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:06:59.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.424+0000 7f8f2be00640 1 -- 172.21.15.16:0/3970643910 >> 172.21.15.16:0/3970643910 conn(0x7f8f2407b5f0 msgr2=0x7f8f24112aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:06:59.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.424+0000 7f8f2be00640 1 -- 172.21.15.16:0/3970643910 shutdown_connections 2024-08-31T06:06:59.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:06:59.424+0000 7f8f2be00640 1 -- 172.21.15.16:0/3970643910 wait complete. 2024-08-31T06:07:00.043 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:06:59 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/3970643910' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:07:00.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:06:59 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/3970643910' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:07:00.240 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:07:00.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:06:59 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/3970643910' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:07:00.413 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 06:07:00 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: 2024-08-31T06:07:00.039+0000 7fde8aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-31T06:07:00.413 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 06:07:00 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: 2024-08-31T06:07:00.039+0000 7fde8aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-31T06:07:00.413 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 06:07:00 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: 2024-08-31T06:07:00.039+0000 7fde8aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-31T06:07:00.413 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 06:07:00 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: 2024-08-31T06:07:00.039+0000 7fde8aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-31T06:07:00.413 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 06:07:00 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: 2024-08-31T06:07:00.039+0000 7fde8aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-31T06:07:00.628 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:07:01.126 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:00 smithi016 ceph-mon[17194]: pgmap v356: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 174 KiB/s wr, 52 op/s 2024-08-31T06:07:01.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:00 smithi184 ceph-mon[22331]: pgmap v356: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 174 KiB/s wr, 52 op/s 2024-08-31T06:07:01.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.303+0000 7fafaca00640 1 -- 172.21.15.16:0/3719847576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafa810bdb0 msgr2=0x7fafa81125e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:01.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.303+0000 7fafaca00640 1 --2- 172.21.15.16:0/3719847576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafa810bdb0 0x7fafa81125e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fafa0009900 tx=0x7fafa002a410 comp rx=0 tx=0).stop 2024-08-31T06:07:01.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.303+0000 7fafaca00640 1 -- 172.21.15.16:0/3719847576 shutdown_connections 2024-08-31T06:07:01.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.303+0000 7fafaca00640 1 --2- 172.21.15.16:0/3719847576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafa810bdb0 0x7fafa81125e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:01.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.303+0000 7fafaca00640 1 --2- 172.21.15.16:0/3719847576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafa810b460 0x7fafa810b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:01.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.303+0000 7fafaca00640 1 --2- 172.21.15.16:0/3719847576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafa8105b20 0x7fafa8105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:01.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.303+0000 7fafaca00640 1 -- 172.21.15.16:0/3719847576 >> 172.21.15.16:0/3719847576 conn(0x7fafa81015b0 msgr2=0x7fafa8103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:01.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.303+0000 7fafaca00640 1 -- 172.21.15.16:0/3719847576 shutdown_connections 2024-08-31T06:07:01.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.303+0000 7fafaca00640 1 -- 172.21.15.16:0/3719847576 wait complete. 2024-08-31T06:07:01.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.305+0000 7fafaca00640 1 Processor -- start 2024-08-31T06:07:01.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.305+0000 7fafaca00640 1 -- start start 2024-08-31T06:07:01.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.305+0000 7fafaca00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafa8105b20 0x7fafa819c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:01.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.305+0000 7fafaca00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafa810b460 0x7fafa819c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:01.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.306+0000 7fafaca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafa810bdb0 0x7fafa81a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:01.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.306+0000 7fafaca00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fafa8115ac0 con 0x7fafa810bdb0 2024-08-31T06:07:01.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.306+0000 7fafaca00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fafa8115940 con 0x7fafa8105b20 2024-08-31T06:07:01.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.306+0000 7fafaca00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fafa8115c40 con 0x7fafa810b460 2024-08-31T06:07:01.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.306+0000 7fafa5400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafa810b460 0x7fafa819c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:01.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.306+0000 7fafa5e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafa8105b20 0x7fafa819c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:01.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.306+0000 7fafa5400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafa810b460 0x7fafa819c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:35564/0 (socket says 172.21.15.16:35564) 2024-08-31T06:07:01.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.306+0000 7fafa5400640 1 -- 172.21.15.16:0/1960818759 learned_addr learned my addr 172.21.15.16:0/1960818759 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:01.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.306+0000 7fafa6800640 1 --2- 172.21.15.16:0/1960818759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafa810bdb0 0x7fafa81a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:01.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.306+0000 7fafa5e00640 1 -- 172.21.15.16:0/1960818759 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafa810b460 msgr2=0x7fafa819c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:01.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.306+0000 7fafa5e00640 1 --2- 172.21.15.16:0/1960818759 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafa810b460 0x7fafa819c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:01.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.306+0000 7fafa5e00640 1 -- 172.21.15.16:0/1960818759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafa810bdb0 msgr2=0x7fafa81a1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:01.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.306+0000 7fafa5e00640 1 --2- 172.21.15.16:0/1960818759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafa810bdb0 0x7fafa81a1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:01.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.306+0000 7fafa5e00640 1 -- 172.21.15.16:0/1960818759 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa81a2380 con 0x7fafa8105b20 2024-08-31T06:07:01.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.306+0000 7fafa5400640 1 --2- 172.21.15.16:0/1960818759 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafa810b460 0x7fafa819c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:07:01.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.307+0000 7fafa5e00640 1 --2- 172.21.15.16:0/1960818759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafa8105b20 0x7fafa819c410 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7faf98009fd0 tx=0x7faf9800ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:01.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.307+0000 7faf9ea00640 1 -- 172.21.15.16:0/1960818759 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf98004b20 con 0x7fafa8105b20 2024-08-31T06:07:01.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.307+0000 7fafaca00640 1 -- 172.21.15.16:0/1960818759 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafa81a2670 con 0x7fafa8105b20 2024-08-31T06:07:01.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.307+0000 7fafaca00640 1 -- 172.21.15.16:0/1960818759 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafa81a8dd0 con 0x7fafa8105b20 2024-08-31T06:07:01.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.307+0000 7faf9ea00640 1 -- 172.21.15.16:0/1960818759 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf98004cc0 con 0x7fafa8105b20 2024-08-31T06:07:01.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.307+0000 7faf9ea00640 1 -- 172.21.15.16:0/1960818759 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9800f600 con 0x7fafa8105b20 2024-08-31T06:07:01.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.308+0000 7fafaca00640 1 -- 172.21.15.16:0/1960818759 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafa8105f30 con 0x7fafa8105b20 2024-08-31T06:07:01.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.311+0000 7faf9ea00640 1 -- 172.21.15.16:0/1960818759 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 23) ==== 152651+0+0 (secure 0 0 0) 0x7faf98010970 con 0x7fafa8105b20 2024-08-31T06:07:01.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.312+0000 7faf9ea00640 1 --2- 172.21.15.16:0/1960818759 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7faf8c0b4c90 0x7faf8c0b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:01.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.312+0000 7faf9ea00640 1 -- 172.21.15.16:0/1960818759 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7faf980e5c50 con 0x7fafa8105b20 2024-08-31T06:07:01.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.313+0000 7fafa5400640 1 --2- 172.21.15.16:0/1960818759 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7faf8c0b4c90 0x7faf8c0b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:01.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.313+0000 7fafa5400640 1 --2- 172.21.15.16:0/1960818759 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7faf8c0b4c90 0x7faf8c0b71a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fafa8104e60 tx=0x7faf94005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:01.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.314+0000 7faf9ea00640 1 -- 172.21.15.16:0/1960818759 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7faf980acb60 con 0x7fafa8105b20 2024-08-31T06:07:01.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:00 smithi155 ceph-mon[21742]: pgmap v356: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 174 KiB/s wr, 52 op/s 2024-08-31T06:07:01.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.549+0000 7fafaca00640 1 -- 172.21.15.16:0/1960818759 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fafa819d540 con 0x7fafa8105b20 2024-08-31T06:07:01.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.550+0000 7faf9ea00640 1 -- 172.21.15.16:0/1960818759 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11917 (secure 0 0 0) 0x7fafa819d540 con 0x7fafa8105b20 2024-08-31T06:07:01.553 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:07:01.553 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-08-31T06:06:38:485237+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:06:37.778867+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24698":{"gid":24698,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1089710405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1089710405},{"type":"v1","addr":"172.21.15.155:6837","nonce":1089710405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24737,24724,24712]},"id":1}]} 2024-08-31T06:07:01.553 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-08-31T06:07:01.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.553+0000 7fafaca00640 1 -- 172.21.15.16:0/1960818759 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7faf8c0b4c90 msgr2=0x7faf8c0b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:01.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.553+0000 7fafaca00640 1 --2- 172.21.15.16:0/1960818759 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7faf8c0b4c90 0x7faf8c0b71a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fafa8104e60 tx=0x7faf94005f90 comp rx=0 tx=0).stop 2024-08-31T06:07:01.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.553+0000 7fafaca00640 1 -- 172.21.15.16:0/1960818759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafa8105b20 msgr2=0x7fafa819c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:01.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.553+0000 7fafaca00640 1 --2- 172.21.15.16:0/1960818759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafa8105b20 0x7fafa819c410 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7faf98009fd0 tx=0x7faf9800ef90 comp rx=0 tx=0).stop 2024-08-31T06:07:01.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.553+0000 7fafaca00640 1 -- 172.21.15.16:0/1960818759 shutdown_connections 2024-08-31T06:07:01.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.553+0000 7fafaca00640 1 --2- 172.21.15.16:0/1960818759 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7faf8c0b4c90 0x7faf8c0b71a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:01.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.553+0000 7fafaca00640 1 --2- 172.21.15.16:0/1960818759 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafa810b460 0x7fafa819c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:01.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.553+0000 7fafaca00640 1 --2- 172.21.15.16:0/1960818759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafa810bdb0 0x7fafa81a1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:01.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.553+0000 7fafaca00640 1 --2- 172.21.15.16:0/1960818759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafa8105b20 0x7fafa819c410 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:01.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.553+0000 7fafaca00640 1 -- 172.21.15.16:0/1960818759 >> 172.21.15.16:0/1960818759 conn(0x7fafa81015b0 msgr2=0x7fafa81089d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:01.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.553+0000 7fafaca00640 1 -- 172.21.15.16:0/1960818759 shutdown_connections 2024-08-31T06:07:01.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:01.553+0000 7fafaca00640 1 -- 172.21.15.16:0/1960818759 wait complete. 2024-08-31T06:07:02.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:01 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/1960818759' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:07:02.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:01 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/1960818759' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:07:02.330 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph osd dump --format=json 2024-08-31T06:07:02.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:01 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/1960818759' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:07:02.707 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:07:03.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:02 smithi016 ceph-mon[17194]: pgmap v357: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 174 KiB/s wr, 53 op/s 2024-08-31T06:07:03.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:02 smithi016 ceph-mon[17194]: mgrmap e24: x(active, since 9m), standbys: z, y 2024-08-31T06:07:03.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:02 smithi184 ceph-mon[22331]: pgmap v357: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 174 KiB/s wr, 53 op/s 2024-08-31T06:07:03.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:02 smithi184 ceph-mon[22331]: mgrmap e24: x(active, since 9m), standbys: z, y 2024-08-31T06:07:03.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.321+0000 7f8cf5800640 1 -- 172.21.15.16:0/3826993835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cf0109b90 msgr2=0x7f8cf01103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:03.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.321+0000 7f8cf5800640 1 --2- 172.21.15.16:0/3826993835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cf0109b90 0x7f8cf01103c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8009ca0 tx=0x7f8ce802a8c0 comp rx=0 tx=0).stop 2024-08-31T06:07:03.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.321+0000 7f8cf5800640 1 -- 172.21.15.16:0/3826993835 shutdown_connections 2024-08-31T06:07:03.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.321+0000 7f8cf5800640 1 --2- 172.21.15.16:0/3826993835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cf0109b90 0x7f8cf01103c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:03.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.321+0000 7f8cf5800640 1 --2- 172.21.15.16:0/3826993835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cf0109240 0x7f8cf0109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:03.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.321+0000 7f8cf5800640 1 --2- 172.21.15.16:0/3826993835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cf0103900 0x7f8cf0103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:03.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.321+0000 7f8cf5800640 1 -- 172.21.15.16:0/3826993835 >> 172.21.15.16:0/3826993835 conn(0x7f8cf007b5f0 msgr2=0x7f8cf007ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:03.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.321+0000 7f8cf5800640 1 -- 172.21.15.16:0/3826993835 shutdown_connections 2024-08-31T06:07:03.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.321+0000 7f8cf5800640 1 -- 172.21.15.16:0/3826993835 wait complete. 2024-08-31T06:07:03.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.323+0000 7f8cf5800640 1 Processor -- start 2024-08-31T06:07:03.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.323+0000 7f8cf5800640 1 -- start start 2024-08-31T06:07:03.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.323+0000 7f8cf5800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cf0103900 0x7f8cf00754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:03.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.323+0000 7f8cf5800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cf0109240 0x7f8cf00759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:03.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.323+0000 7f8cf5800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cf0109b90 0x7f8cf0075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:03.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.324+0000 7f8cf5800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf0113850 con 0x7f8cf0103900 2024-08-31T06:07:03.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.324+0000 7f8cf5800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf01136d0 con 0x7f8cf0109240 2024-08-31T06:07:03.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.324+0000 7f8cf5800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf01139d0 con 0x7f8cf0109b90 2024-08-31T06:07:03.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.324+0000 7f8ce7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cf0109240 0x7f8cf00759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:03.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.324+0000 7f8ceea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cf0103900 0x7f8cf00754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:03.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.324+0000 7f8ce7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cf0109240 0x7f8cf00759f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:54568/0 (socket says 172.21.15.16:54568) 2024-08-31T06:07:03.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.324+0000 7f8ce7e00640 1 -- 172.21.15.16:0/3626537788 learned_addr learned my addr 172.21.15.16:0/3626537788 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:03.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.324+0000 7f8cef400640 1 --2- 172.21.15.16:0/3626537788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cf0109b90 0x7f8cf0075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:03.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.324+0000 7f8ce7e00640 1 -- 172.21.15.16:0/3626537788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cf0109b90 msgr2=0x7f8cf0075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:03.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.324+0000 7f8ce7e00640 1 --2- 172.21.15.16:0/3626537788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cf0109b90 0x7f8cf0075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:03.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.324+0000 7f8ce7e00640 1 -- 172.21.15.16:0/3626537788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cf0103900 msgr2=0x7f8cf00754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:03.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.324+0000 7f8ce7e00640 1 --2- 172.21.15.16:0/3626537788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cf0103900 0x7f8cf00754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:03.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.324+0000 7f8ce7e00640 1 -- 172.21.15.16:0/3626537788 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf01a84e0 con 0x7f8cf0109240 2024-08-31T06:07:03.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.324+0000 7f8ceea00640 1 --2- 172.21.15.16:0/3626537788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cf0103900 0x7f8cf00754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:07:03.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.325+0000 7f8ce7e00640 1 --2- 172.21.15.16:0/3626537788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cf0109240 0x7f8cf00759f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8cd800e9f0 tx=0x7f8cd800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:03.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.325+0000 7f8ce7400640 1 -- 172.21.15.16:0/3626537788 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd800ccc0 con 0x7f8cf0109240 2024-08-31T06:07:03.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.325+0000 7f8cf5800640 1 -- 172.21.15.16:0/3626537788 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cf01a87d0 con 0x7f8cf0109240 2024-08-31T06:07:03.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.325+0000 7f8ce7400640 1 -- 172.21.15.16:0/3626537788 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cd80108e0 con 0x7f8cf0109240 2024-08-31T06:07:03.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.325+0000 7f8ce7400640 1 -- 172.21.15.16:0/3626537788 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd8018860 con 0x7f8cf0109240 2024-08-31T06:07:03.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.325+0000 7f8cf5800640 1 -- 172.21.15.16:0/3626537788 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cf01a8d60 con 0x7f8cf0109240 2024-08-31T06:07:03.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.326+0000 7f8cf5800640 1 -- 172.21.15.16:0/3626537788 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cf0103d10 con 0x7f8cf0109240 2024-08-31T06:07:03.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.329+0000 7f8ce7400640 1 -- 172.21.15.16:0/3626537788 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f8cd8010430 con 0x7f8cf0109240 2024-08-31T06:07:03.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.330+0000 7f8ce7400640 1 --2- 172.21.15.16:0/3626537788 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f8cd40b4ca0 0x7f8cd40b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:03.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.330+0000 7f8ce7400640 1 -- 172.21.15.16:0/3626537788 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f8cd80e5d20 con 0x7f8cf0109240 2024-08-31T06:07:03.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.330+0000 7f8ceea00640 1 --2- 172.21.15.16:0/3626537788 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f8cd40b4ca0 0x7f8cd40b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:03.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.331+0000 7f8ceea00640 1 --2- 172.21.15.16:0/3626537788 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f8cd40b4ca0 0x7f8cd40b71b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0076f00 tx=0x7f8ce0005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:03.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.332+0000 7f8ce7400640 1 -- 172.21.15.16:0/3626537788 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f8cd80acb10 con 0x7f8cf0109240 2024-08-31T06:07:03.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:02 smithi155 ceph-mon[21742]: pgmap v357: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 174 KiB/s wr, 53 op/s 2024-08-31T06:07:03.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:02 smithi155 ceph-mon[21742]: mgrmap e24: x(active, since 9m), standbys: z, y 2024-08-31T06:07:03.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.530+0000 7f8cf5800640 1 -- 172.21.15.16:0/3626537788 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8cf002e430 con 0x7f8cf0109240 2024-08-31T06:07:03.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.533+0000 7f8ce7400640 1 -- 172.21.15.16:0/3626537788 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+23043 (secure 0 0 0) 0x7f8cd80ac410 con 0x7f8cf0109240 2024-08-31T06:07:03.535 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:07:03.535 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":80,"fsid":"673e4aa0-675d-11ef-bcd4-c7b262605968","created":"2024-08-31T05:55:09.632536+0000","modified":"2024-08-31T06:05:25.039584+0000","last_up_change":"2024-08-31T06:03:27.893697+0000","last_in_change":"2024-08-31T06:03:05.664945+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-31T05:59:23.642270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-31T06:04:20.871835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-31T06:04:24.119499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e349dae-6aa5-4b6d-9107-a945655e11bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6803","nonce":1725037400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6805","nonce":1725037400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6809","nonce":1725037400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6807","nonce":1725037400}]},"public_addr":"172.21.15.16:6803/1725037400","cluster_addr":"172.21.15.16:6805/1725037400","heartbeat_back_addr":"172.21.15.16:6809/1725037400","heartbeat_front_addr":"172.21.15.16:6807/1725037400","state":["exists","up"]},{"osd":1,"uuid":"bd645dc3-93b5-40af-89d8-08ccec22e50d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6801","nonce":4241618440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6803","nonce":4241618440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6807","nonce":4241618440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6805","nonce":4241618440}]},"public_addr":"172.21.15.155:6801/4241618440","cluster_addr":"172.21.15.155:6803/4241618440","heartbeat_back_addr":"172.21.15.155:6807/4241618440","heartbeat_front_addr":"172.21.15.155:6805/4241618440","state":["exists","up"]},{"osd":2,"uuid":"c69c8c19-32f8-400a-86d4-0f4ce240a7de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6801","nonce":2193296422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6803","nonce":2193296422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6807","nonce":2193296422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6805","nonce":2193296422}]},"public_addr":"172.21.15.184:6801/2193296422","cluster_addr":"172.21.15.184:6803/2193296422","heartbeat_back_addr":"172.21.15.184:6807/2193296422","heartbeat_front_addr":"172.21.15.184:6805/2193296422","state":["exists","up"]},{"osd":3,"uuid":"8892e35f-6880-434c-a805-ab8ccbed2bf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6811","nonce":81681815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6813","nonce":81681815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6817","nonce":81681815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6815","nonce":81681815}]},"public_addr":"172.21.15.16:6811/81681815","cluster_addr":"172.21.15.16:6813/81681815","heartbeat_back_addr":"172.21.15.16:6817/81681815","heartbeat_front_addr":"172.21.15.16:6815/81681815","state":["exists","up"]},{"osd":4,"uuid":"d2515e73-88f7-4654-9762-adb8802777ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6809","nonce":3592422019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6811","nonce":3592422019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6815","nonce":3592422019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6813","nonce":3592422019}]},"public_addr":"172.21.15.155:6809/3592422019","cluster_addr":"172.21.15.155:6811/3592422019","heartbeat_back_addr":"172.21.15.155:6815/3592422019","heartbeat_front_addr":"172.21.15.155:6813/3592422019","state":["exists","up"]},{"osd":5,"uuid":"5ce6e2fa-c17c-49b3-9402-f19bfc21076d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6809","nonce":2054894065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6811","nonce":2054894065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6815","nonce":2054894065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6813","nonce":2054894065}]},"public_addr":"172.21.15.184:6809/2054894065","cluster_addr":"172.21.15.184:6811/2054894065","heartbeat_back_addr":"172.21.15.184:6815/2054894065","heartbeat_front_addr":"172.21.15.184:6813/2054894065","state":["exists","up"]},{"osd":6,"uuid":"f257d8d1-ad59-4bd8-821d-bf29e50b547e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6819","nonce":1697421092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6821","nonce":1697421092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6825","nonce":1697421092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6823","nonce":1697421092}]},"public_addr":"172.21.15.16:6819/1697421092","cluster_addr":"172.21.15.16:6821/1697421092","heartbeat_back_addr":"172.21.15.16:6825/1697421092","heartbeat_front_addr":"172.21.15.16:6823/1697421092","state":["exists","up"]},{"osd":7,"uuid":"8e76c20e-8d25-423b-b7e8-de642d9c9a6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6817","nonce":2671827901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6819","nonce":2671827901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6823","nonce":2671827901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6821","nonce":2671827901}]},"public_addr":"172.21.15.155:6817/2671827901","cluster_addr":"172.21.15.155:6819/2671827901","heartbeat_back_addr":"172.21.15.155:6823/2671827901","heartbeat_front_addr":"172.21.15.155:6821/2671827901","state":["exists","up"]},{"osd":8,"uuid":"0f0bb23f-4bc3-4198-8381-5b5cd16ae394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6817","nonce":2479117823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6819","nonce":2479117823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6823","nonce":2479117823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6821","nonce":2479117823}]},"public_addr":"172.21.15.184:6817/2479117823","cluster_addr":"172.21.15.184:6819/2479117823","heartbeat_back_addr":"172.21.15.184:6823/2479117823","heartbeat_front_addr":"172.21.15.184:6821/2479117823","state":["exists","up"]},{"osd":9,"uuid":"bc27afcb-7871-4f72-a043-044fd5fe7d86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6827","nonce":3015700546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6829","nonce":3015700546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6833","nonce":3015700546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6831","nonce":3015700546}]},"public_addr":"172.21.15.16:6827/3015700546","cluster_addr":"172.21.15.16:6829/3015700546","heartbeat_back_addr":"172.21.15.16:6833/3015700546","heartbeat_front_addr":"172.21.15.16:6831/3015700546","state":["exists","up"]},{"osd":10,"uuid":"f2548997-72ee-400e-ab8d-5966d85b2cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6825","nonce":2427122475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6827","nonce":2427122475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6831","nonce":2427122475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6829","nonce":2427122475}]},"public_addr":"172.21.15.155:6825/2427122475","cluster_addr":"172.21.15.155:6827/2427122475","heartbeat_back_addr":"172.21.15.155:6831/2427122475","heartbeat_front_addr":"172.21.15.155:6829/2427122475","state":["exists","up"]},{"osd":11,"uuid":"8297da1d-d3e2-4a4d-9396-426dfafa3e95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6825","nonce":399265701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6827","nonce":399265701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6831","nonce":399265701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6829","nonce":399265701}]},"public_addr":"172.21.15.184:6825/399265701","cluster_addr":"172.21.15.184:6827/399265701","heartbeat_back_addr":"172.21.15.184:6831/399265701","heartbeat_front_addr":"172.21.15.184:6829/399265701","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:58:30.842360+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:58:57.164488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:59:20.778103+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:59:50.123991+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:00:16.789933+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:00:41.788905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:01:12.281376+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:01:39.211858+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:02:04.958771+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:02:35.645947+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:03:01.940433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:03:26.728673+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3923220960":"2024-09-01T05:57:23.153214+0000","172.21.15.16:0/3013533836":"2024-09-01T05:57:23.153214+0000","172.21.15.16:6800/3755683228":"2024-09-01T05:55:45.075902+0000","172.21.15.16:0/3064435470":"2024-09-01T05:55:45.075902+0000","172.21.15.16:0/911709962":"2024-09-01T05:55:45.075902+0000","172.21.15.16:0/2937368967":"2024-09-01T05:55:45.075902+0000","172.21.15.16:6800/1378972230":"2024-09-01T05:56:14.195704+0000","172.21.15.16:0/3047316995":"2024-09-01T05:56:14.195704+0000","172.21.15.16:6801/3755683228":"2024-09-01T05:55:45.075902+0000","172.21.15.16:6801/2571082634":"2024-09-01T05:57:23.153214+0000","172.21.15.16:0/2136704295":"2024-09-01T05:56:14.195704+0000","172.21.15.16:0/2259332778":"2024-09-01T05:56:14.195704+0000","172.21.15.16:6800/2571082634":"2024-09-01T05:57:23.153214+0000","172.21.15.16:6801/1378972230":"2024-09-01T05:56:14.195704+0000","172.21.15.16:0/2126457673":"2024-09-01T05:57:23.153214+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-31T06:07:03.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.535+0000 7f8cf5800640 1 -- 172.21.15.16:0/3626537788 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f8cd40b4ca0 msgr2=0x7f8cd40b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:03.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.535+0000 7f8cf5800640 1 --2- 172.21.15.16:0/3626537788 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f8cd40b4ca0 0x7f8cd40b71b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0076f00 tx=0x7f8ce0005f90 comp rx=0 tx=0).stop 2024-08-31T06:07:03.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.535+0000 7f8cf5800640 1 -- 172.21.15.16:0/3626537788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cf0109240 msgr2=0x7f8cf00759f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:03.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.535+0000 7f8cf5800640 1 --2- 172.21.15.16:0/3626537788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cf0109240 0x7f8cf00759f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8cd800e9f0 tx=0x7f8cd800eeb0 comp rx=0 tx=0).stop 2024-08-31T06:07:03.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.535+0000 7f8cf5800640 1 -- 172.21.15.16:0/3626537788 shutdown_connections 2024-08-31T06:07:03.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.535+0000 7f8cf5800640 1 --2- 172.21.15.16:0/3626537788 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f8cd40b4ca0 0x7f8cd40b71b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:03.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.535+0000 7f8cf5800640 1 --2- 172.21.15.16:0/3626537788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cf0109b90 0x7f8cf0075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:03.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.535+0000 7f8cf5800640 1 --2- 172.21.15.16:0/3626537788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cf0109240 0x7f8cf00759f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:03.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.535+0000 7f8cf5800640 1 --2- 172.21.15.16:0/3626537788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cf0103900 0x7f8cf00754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:03.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.535+0000 7f8cf5800640 1 -- 172.21.15.16:0/3626537788 >> 172.21.15.16:0/3626537788 conn(0x7f8cf007b5f0 msgr2=0x7f8cf0105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:03.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.535+0000 7f8cf5800640 1 -- 172.21.15.16:0/3626537788 shutdown_connections 2024-08-31T06:07:03.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:03.535+0000 7f8cf5800640 1 -- 172.21.15.16:0/3626537788 wait complete. 2024-08-31T06:07:04.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:03 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:07:04.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:03 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/3626537788' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T06:07:04.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:03 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:07:04.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:03 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/3626537788' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T06:07:04.366 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-08-31T06:07:04.374 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:03 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:07:04.374 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:03 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/3626537788' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T06:07:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.477+0000 7f37fb400640 1 -- 172.21.15.16:0/3162402074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f41046d0 msgr2=0x7f37f4104ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.477+0000 7f37fb400640 1 --2- 172.21.15.16:0/3162402074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f41046d0 0x7f37f4104ae0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f37e8009900 tx=0x7f37e802a410 comp rx=0 tx=0).stop 2024-08-31T06:07:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.477+0000 7f37fb400640 1 -- 172.21.15.16:0/3162402074 shutdown_connections 2024-08-31T06:07:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.477+0000 7f37fb400640 1 --2- 172.21.15.16:0/3162402074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37f4105020 0x7f37f410d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.477+0000 7f37fb400640 1 --2- 172.21.15.16:0/3162402074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37f410db30 0x7f37f4110020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.477+0000 7f37fb400640 1 --2- 172.21.15.16:0/3162402074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f41046d0 0x7f37f4104ae0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.477+0000 7f37fb400640 1 -- 172.21.15.16:0/3162402074 >> 172.21.15.16:0/3162402074 conn(0x7f37f40fe160 msgr2=0x7f37f41005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.477+0000 7f37fb400640 1 -- 172.21.15.16:0/3162402074 shutdown_connections 2024-08-31T06:07:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.477+0000 7f37fb400640 1 -- 172.21.15.16:0/3162402074 wait complete. 2024-08-31T06:07:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.478+0000 7f37fb400640 1 Processor -- start 2024-08-31T06:07:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.478+0000 7f37fb400640 1 -- start start 2024-08-31T06:07:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.479+0000 7f37fb400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37f41046d0 0x7f37f419c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.479+0000 7f37fb400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37f4105020 0x7f37f419c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.479+0000 7f37fb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f410db30 0x7f37f41a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.479+0000 7f37fb400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f37f41137c0 con 0x7f37f410db30 2024-08-31T06:07:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.479+0000 7f37fb400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f37f4113640 con 0x7f37f41046d0 2024-08-31T06:07:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.479+0000 7f37fb400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f37f4113940 con 0x7f37f4105020 2024-08-31T06:07:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.479+0000 7f37f9400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f410db30 0x7f37f41a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.479+0000 7f37f8a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37f41046d0 0x7f37f419c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.479+0000 7f37f8a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37f41046d0 0x7f37f419c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:54582/0 (socket says 172.21.15.16:54582) 2024-08-31T06:07:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.479+0000 7f37f9400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f410db30 0x7f37f41a1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60142/0 (socket says 172.21.15.16:60142) 2024-08-31T06:07:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.479+0000 7f37f8a00640 1 -- 172.21.15.16:0/4273854815 learned_addr learned my addr 172.21.15.16:0/4273854815 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.479+0000 7f37f3e00640 1 --2- 172.21.15.16:0/4273854815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37f4105020 0x7f37f419c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.480+0000 7f37f8a00640 1 -- 172.21.15.16:0/4273854815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37f4105020 msgr2=0x7f37f419c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.480+0000 7f37f8a00640 1 --2- 172.21.15.16:0/4273854815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37f4105020 0x7f37f419c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.480+0000 7f37f8a00640 1 -- 172.21.15.16:0/4273854815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f410db30 msgr2=0x7f37f41a1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.480+0000 7f37f8a00640 1 --2- 172.21.15.16:0/4273854815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f410db30 0x7f37f41a1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:04.484 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.480+0000 7f37f8a00640 1 -- 172.21.15.16:0/4273854815 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37f41a2350 con 0x7f37f41046d0 2024-08-31T06:07:04.484 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.480+0000 7f37f9400640 1 --2- 172.21.15.16:0/4273854815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f410db30 0x7f37f41a1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:07:04.484 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.480+0000 7f37f8a00640 1 --2- 172.21.15.16:0/4273854815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37f41046d0 0x7f37f419c420 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f37e8005940 tx=0x7f37e8004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:04.484 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.480+0000 7f37f1600640 1 -- 172.21.15.16:0/4273854815 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e803f030 con 0x7f37f41046d0 2024-08-31T06:07:04.484 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.480+0000 7f37fb400640 1 -- 172.21.15.16:0/4273854815 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37f41a25e0 con 0x7f37f41046d0 2024-08-31T06:07:04.484 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.480+0000 7f37f1600640 1 -- 172.21.15.16:0/4273854815 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37e8003aa0 con 0x7f37f41046d0 2024-08-31T06:07:04.484 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.480+0000 7f37f1600640 1 -- 172.21.15.16:0/4273854815 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e8035070 con 0x7f37f41046d0 2024-08-31T06:07:04.484 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.481+0000 7f37fb400640 1 -- 172.21.15.16:0/4273854815 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37f41a8dd0 con 0x7f37f41046d0 2024-08-31T06:07:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.484+0000 7f38025d2740 1 Processor -- start 2024-08-31T06:07:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.484+0000 7f38025d2740 1 -- start start 2024-08-31T06:07:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.484+0000 7f38025d2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639b85a51b0 0x5639b8948690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.484+0000 7f38025d2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639b857c2f0 0x5639b8948bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.484+0000 7f38025d2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b85844d0 0x5639b8949110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:04.488 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.484+0000 7f38025d2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5639b88bdf70 con 0x5639b85844d0 2024-08-31T06:07:04.488 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.484+0000 7f38025d2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5639b88ad500 con 0x5639b85a51b0 2024-08-31T06:07:04.488 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.484+0000 7f38025d2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5639b88b14b0 con 0x5639b857c2f0 2024-08-31T06:07:04.488 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.484+0000 7f37f3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b85844d0 0x5639b8949110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:04.488 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.485+0000 7f37f3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b85844d0 0x5639b8949110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60148/0 (socket says 172.21.15.16:60148) 2024-08-31T06:07:04.488 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.485+0000 7f37f3e00640 1 -- 172.21.15.16:0/1735858893 learned_addr learned my addr 172.21.15.16:0/1735858893 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:04.488 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.485+0000 7f37f9400640 1 --2- 172.21.15.16:0/1735858893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639b85a51b0 0x5639b8948690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:04.488 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.485+0000 7f37f1600640 1 -- 172.21.15.16:0/4273854815 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f37e8042030 con 0x7f37f41046d0 2024-08-31T06:07:04.488 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.485+0000 7f37f3e00640 1 -- 172.21.15.16:0/1735858893 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639b857c2f0 msgr2=0x5639b8948bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:04.488 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.485+0000 7f37f8a00640 1 --2- 172.21.15.16:0/1735858893 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639b857c2f0 0x5639b8948bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:04.488 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.485+0000 7f37f3e00640 1 --2- 172.21.15.16:0/1735858893 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639b857c2f0 0x5639b8948bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:04.488 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.485+0000 7f37f3e00640 1 -- 172.21.15.16:0/1735858893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639b85a51b0 msgr2=0x5639b8948690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:04.488 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.485+0000 7f37f3e00640 1 --2- 172.21.15.16:0/1735858893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639b85a51b0 0x5639b8948690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:04.489 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.485+0000 7f37f3e00640 1 -- 172.21.15.16:0/1735858893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639b8867c10 con 0x5639b85844d0 2024-08-31T06:07:04.489 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.485+0000 7f37f9400640 1 --2- 172.21.15.16:0/1735858893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639b85a51b0 0x5639b8948690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:07:04.489 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.485+0000 7f37f8a00640 1 --2- 172.21.15.16:0/1735858893 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639b857c2f0 0x5639b8948bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:07:04.489 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.485+0000 7f37f3e00640 1 --2- 172.21.15.16:0/1735858893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b85844d0 0x5639b8949110 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f37e0009c20 tx=0x7f37e0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a866899ba6111360 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.486+0000 7f37e5c00640 1 -- 172.21.15.16:0/1735858893 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e0004230 con 0x5639b85844d0 2024-08-31T06:07:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.486+0000 7f37e5c00640 1 -- 172.21.15.16:0/1735858893 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37e00043d0 con 0x5639b85844d0 2024-08-31T06:07:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.486+0000 7f37e5c00640 1 -- 172.21.15.16:0/1735858893 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e0004d00 con 0x5639b85844d0 2024-08-31T06:07:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.486+0000 7f38025d2740 1 -- 172.21.15.16:0/1735858893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b85844d0 msgr2=0x5639b8949110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.486+0000 7f38025d2740 1 --2- 172.21.15.16:0/1735858893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b85844d0 0x5639b8949110 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f37e0009c20 tx=0x7f37e0011810 comp rx=0 tx=0).stop 2024-08-31T06:07:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.487+0000 7f38025d2740 1 -- 172.21.15.16:0/1735858893 shutdown_connections 2024-08-31T06:07:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.487+0000 7f38025d2740 1 --2- 172.21.15.16:0/1735858893 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639b857c2f0 0x5639b8948bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.487+0000 7f38025d2740 1 --2- 172.21.15.16:0/1735858893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b85844d0 0x5639b8949110 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.487+0000 7f38025d2740 1 --2- 172.21.15.16:0/1735858893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639b85a51b0 0x5639b8948690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.487+0000 7f38025d2740 1 -- 172.21.15.16:0/1735858893 >> 172.21.15.16:0/1735858893 conn(0x5639b8599300 msgr2=0x5639b8946580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.487+0000 7f38025d2740 1 -- 172.21.15.16:0/1735858893 shutdown_connections 2024-08-31T06:07:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.487+0000 7f38025d2740 1 -- 172.21.15.16:0/1735858893 wait complete. 2024-08-31T06:07:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.487+0000 7f38025d2740 1 -- start start 2024-08-31T06:07:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.487+0000 7f37f1600640 1 --2- 172.21.15.16:0/4273854815 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f37dc0b4c30 0x7f37dc0b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.488+0000 7f38025d2740 1 Processor -- start 2024-08-31T06:07:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.488+0000 7f37f9400640 1 --2- 172.21.15.16:0/4273854815 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f37dc0b4c30 0x7f37dc0b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.488+0000 7f37f1600640 1 -- 172.21.15.16:0/4273854815 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f37e8105b80 con 0x7f37f41046d0 2024-08-31T06:07:04.494 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.488+0000 7f37f9400640 1 --2- 172.21.15.16:0/4273854815 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f37dc0b4c30 0x7f37dc0b7140 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f37ec005fd0 tx=0x7f37ec0096a0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:04.494 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.488+0000 7f38025d2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b857c2f0 0x5639b8953c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:04.494 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.489+0000 7f38025d2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639b85844d0 0x5639b8954140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:04.494 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.489+0000 7f38025d2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639b85a51b0 0x5639b8954680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:04.494 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.489+0000 7f38025d2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5639b87e6ab0 con 0x5639b857c2f0 2024-08-31T06:07:04.494 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.489+0000 7f38025d2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5639b8854320 con 0x5639b85844d0 2024-08-31T06:07:04.494 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.489+0000 7f38025d2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5639b8956e10 con 0x5639b85a51b0 2024-08-31T06:07:04.494 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.489+0000 7f38025d2740 10 client.0 _ll_register_callbacks cb 0x5639b888de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:07:04.494 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.489+0000 7f37f8a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b857c2f0 0x5639b8953c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:04.495 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.489+0000 7f37f3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639b85844d0 0x5639b8954140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:04.495 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.489+0000 7f37f8a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b857c2f0 0x5639b8953c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60152/0 (socket says 172.21.15.16:60152) 2024-08-31T06:07:04.495 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.489+0000 7f37f8a00640 1 -- 172.21.15.16:0/3332688900 learned_addr learned my addr 172.21.15.16:0/3332688900 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:04.495 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.489+0000 7f37f9400640 1 --2- 172.21.15.16:0/3332688900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639b85a51b0 0x5639b8954680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:04.495 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.489+0000 7f37f8a00640 1 -- 172.21.15.16:0/3332688900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639b85a51b0 msgr2=0x5639b8954680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:04.495 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.489+0000 7f37f8a00640 1 --2- 172.21.15.16:0/3332688900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639b85a51b0 0x5639b8954680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:04.495 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.489+0000 7f37f8a00640 1 -- 172.21.15.16:0/3332688900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639b85844d0 msgr2=0x5639b8954140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:04.495 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.489+0000 7f37f8a00640 1 --2- 172.21.15.16:0/3332688900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639b85844d0 0x5639b8954140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:04.495 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.489+0000 7f37f8a00640 1 -- 172.21.15.16:0/3332688900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639b8610790 con 0x5639b857c2f0 2024-08-31T06:07:04.495 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.489+0000 7f37f3e00640 1 --2- 172.21.15.16:0/3332688900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639b85844d0 0x5639b8954140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:07:04.495 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.490+0000 7f37f8a00640 1 --2- 172.21.15.16:0/3332688900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b857c2f0 0x5639b8953c00 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f37e80c9590 tx=0x7f37e80c82a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a034010a6b931c68 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:04.495 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.490+0000 7f37da000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:07:04.495 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.490+0000 7f37da000640 1 -- 172.21.15.16:0/3332688900 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e8049050 con 0x5639b857c2f0 2024-08-31T06:07:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.490+0000 7f37da000640 1 -- 172.21.15.16:0/3332688900 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f37e80c85b0 con 0x5639b857c2f0 2024-08-31T06:07:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.490+0000 7f38025d2740 1 -- 172.21.15.16:0/3332688900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37e00043d0 con 0x5639b857c2f0 2024-08-31T06:07:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.490+0000 7f37da000640 1 -- 172.21.15.16:0/3332688900 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f37e80cba90 con 0x5639b857c2f0 2024-08-31T06:07:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.490+0000 7f38025d2740 10 client.15417 fetch_fsmap learned FSMap version 25 2024-08-31T06:07:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.490+0000 7f38025d2740 1 -- 172.21.15.16:0/3332688900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5639b8689620 con 0x5639b857c2f0 2024-08-31T06:07:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.490+0000 7f37da000640 1 -- 172.21.15.16:0/3332688900 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37e00043d0 con 0x5639b857c2f0 2024-08-31T06:07:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.491+0000 7f37da000640 1 -- 172.21.15.16:0/3332688900 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f37e8003c50 con 0x5639b857c2f0 2024-08-31T06:07:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.491+0000 7f38025d2740 10 client.15417 fetch_fsmap finished waiting for FSMap version 25 2024-08-31T06:07:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.491+0000 7f38025d2740 10 client.15417 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:07:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.491+0000 7f38025d2740 20 client.15417 populate_metadata read hostname 'smithi016' 2024-08-31T06:07:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.491+0000 7f38025d2740 1 --2- 172.21.15.16:0/3332688900 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5639b8563d80 0x5639b8965c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.491+0000 7f38025d2740 4 client.15417 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:07:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.491+0000 7f38025d2740 1 -- 172.21.15.16:0/3332688900 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639b88bfec0 con 0x5639b8563d80 2024-08-31T06:07:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.491+0000 7f37f3e00640 1 --2- 172.21.15.16:0/3332688900 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5639b8563d80 0x5639b8965c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:04.497 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.492+0000 7f37da000640 10 client.15417 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:04.497 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.492+0000 7f37f3e00640 1 --2- 172.21.15.16:0/3332688900 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5639b8563d80 0x5639b8965c60 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1e9ca561021428b server_cookie=4fb332223c753c2d in_seq=0 out_seq=0 2024-08-31T06:07:04.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.494+0000 7f37da000640 1 -- 172.21.15.16:0/3332688900 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5639b88bfec0 con 0x5639b8563d80 2024-08-31T06:07:04.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.494+0000 7f37da000640 10 client.15417 handle_command_reply: tid=0 2024-08-31T06:07:04.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.494+0000 7f38025d2740 1 client.15417 shutdown 2024-08-31T06:07:04.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.494+0000 7f38025d2740 1 -- 172.21.15.16:0/3332688900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b857c2f0 msgr2=0x5639b8953c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:04.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.494+0000 7f38025d2740 1 --2- 172.21.15.16:0/3332688900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b857c2f0 0x5639b8953c00 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f37e80c9590 tx=0x7f37e80c82a0 comp rx=0 tx=0).stop 2024-08-31T06:07:04.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.494+0000 7f38025d2740 1 -- 172.21.15.16:0/3332688900 shutdown_connections 2024-08-31T06:07:04.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.494+0000 7f38025d2740 1 --2- 172.21.15.16:0/3332688900 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5639b8563d80 0x5639b8965c60 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:04.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.494+0000 7f38025d2740 1 --2- 172.21.15.16:0/3332688900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639b85a51b0 0x5639b8954680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:04.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.494+0000 7f37da000640 0 client.15417 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:04.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.494+0000 7f38025d2740 1 --2- 172.21.15.16:0/3332688900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639b85844d0 0x5639b8954140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:04.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.494+0000 7f38025d2740 1 --2- 172.21.15.16:0/3332688900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b857c2f0 0x5639b8953c00 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:04.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.494+0000 7f38025d2740 1 -- 172.21.15.16:0/3332688900 >> 172.21.15.16:0/3332688900 conn(0x5639b8599300 msgr2=0x5639b89496c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:04.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.494+0000 7f38025d2740 1 -- 172.21.15.16:0/3332688900 shutdown_connections 2024-08-31T06:07:04.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.494+0000 7f38025d2740 1 -- 172.21.15.16:0/3332688900 wait complete. 2024-08-31T06:07:04.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.494+0000 7f38025d2740 20 client.15417 trim_cache size 0 max 16384 2024-08-31T06:07:04.511 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.509+0000 7f38025d2740 1 Processor -- start 2024-08-31T06:07:04.511 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.509+0000 7f38025d2740 1 -- start start 2024-08-31T06:07:04.511 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.509+0000 7f38025d2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b857c2f0 0x5639b8945230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:04.511 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.509+0000 7f38025d2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639b85844d0 0x5639b8950df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:04.511 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.509+0000 7f38025d2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639b85a51b0 0x5639b8951330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:04.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.509+0000 7f38025d2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f37f41b3c40 con 0x5639b857c2f0 2024-08-31T06:07:04.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.509+0000 7f38025d2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f37f41b1510 con 0x5639b85a51b0 2024-08-31T06:07:04.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.509+0000 7f38025d2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f37f41af360 con 0x5639b85844d0 2024-08-31T06:07:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.509+0000 7f37f8a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b857c2f0 0x5639b8945230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.509+0000 7f37f8a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b857c2f0 0x5639b8945230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60158/0 (socket says 172.21.15.16:60158) 2024-08-31T06:07:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.509+0000 7f37f8a00640 1 -- 172.21.15.16:0/3040176551 learned_addr learned my addr 172.21.15.16:0/3040176551 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.509+0000 7f37f3e00640 1 --2- 172.21.15.16:0/3040176551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639b85844d0 0x5639b8950df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.510+0000 7f37f9400640 1 --2- 172.21.15.16:0/3040176551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639b85a51b0 0x5639b8951330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.510+0000 7f37f3e00640 1 -- 172.21.15.16:0/3040176551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639b85a51b0 msgr2=0x5639b8951330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.510+0000 7f37f3e00640 1 --2- 172.21.15.16:0/3040176551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639b85a51b0 0x5639b8951330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:04.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.510+0000 7f37f3e00640 1 -- 172.21.15.16:0/3040176551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b857c2f0 msgr2=0x5639b8945230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:04.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.510+0000 7f37f3e00640 1 --2- 172.21.15.16:0/3040176551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b857c2f0 0x5639b8945230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:04.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.510+0000 7f37f3e00640 1 -- 172.21.15.16:0/3040176551 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639b894f760 con 0x5639b85844d0 2024-08-31T06:07:04.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.510+0000 7f37f8a00640 1 --2- 172.21.15.16:0/3040176551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b857c2f0 0x5639b8945230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:07:04.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.510+0000 7f37f3e00640 1 --2- 172.21.15.16:0/3040176551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639b85844d0 0x5639b8950df0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f37e0011d10 tx=0x7f37e00049a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b55d5d6333c9b07d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:04.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.510+0000 7f37e5200640 1 -- 172.21.15.16:0/3040176551 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e001e040 con 0x5639b85844d0 2024-08-31T06:07:04.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.511+0000 7f37e5200640 1 -- 172.21.15.16:0/3040176551 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37e001f040 con 0x5639b85844d0 2024-08-31T06:07:04.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.511+0000 7f37e5200640 1 -- 172.21.15.16:0/3040176551 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e0003850 con 0x5639b85844d0 2024-08-31T06:07:04.516 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.511+0000 7f38025d2740 1 -- 172.21.15.16:0/3040176551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639b85844d0 msgr2=0x5639b8950df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:04.516 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.511+0000 7f38025d2740 1 --2- 172.21.15.16:0/3040176551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639b85844d0 0x5639b8950df0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f37e0011d10 tx=0x7f37e00049a0 comp rx=0 tx=0).stop 2024-08-31T06:07:04.516 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.511+0000 7f38025d2740 1 -- 172.21.15.16:0/3040176551 shutdown_connections 2024-08-31T06:07:04.516 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.511+0000 7f38025d2740 1 --2- 172.21.15.16:0/3040176551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639b85844d0 0x5639b8950df0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:04.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.511+0000 7f38025d2740 1 --2- 172.21.15.16:0/3040176551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639b85a51b0 0x5639b8951330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:04.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.511+0000 7f38025d2740 1 --2- 172.21.15.16:0/3040176551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b857c2f0 0x5639b8945230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:04.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.511+0000 7f38025d2740 1 -- 172.21.15.16:0/3040176551 >> 172.21.15.16:0/3040176551 conn(0x5639b8599300 msgr2=0x5639b8944b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:04.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.512+0000 7f38025d2740 1 -- 172.21.15.16:0/3040176551 shutdown_connections 2024-08-31T06:07:04.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.512+0000 7f38025d2740 1 -- 172.21.15.16:0/3040176551 wait complete. 2024-08-31T06:07:04.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.512+0000 7f38025d2740 1 -- start start 2024-08-31T06:07:04.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.512+0000 7f38025d2740 1 Processor -- start 2024-08-31T06:07:04.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.513+0000 7f38025d2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639b857c2f0 0x5639b895c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:04.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.513+0000 7f38025d2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b85844d0 0x5639b8959bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:04.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.513+0000 7f38025d2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639b85a51b0 0x5639b895a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:04.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.513+0000 7f38025d2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f37f41af9d0 con 0x5639b85844d0 2024-08-31T06:07:04.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.513+0000 7f38025d2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f37f41b0e40 con 0x5639b857c2f0 2024-08-31T06:07:04.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.513+0000 7f38025d2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f37f41b00a0 con 0x5639b85a51b0 2024-08-31T06:07:04.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.513+0000 7f38025d2740 10 client.0 _ll_register_callbacks cb 0x7f37e8005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:07:04.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.513+0000 7f37f3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b85844d0 0x5639b8959bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:04.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.513+0000 7f37f3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b85844d0 0x5639b8959bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60170/0 (socket says 172.21.15.16:60170) 2024-08-31T06:07:04.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.513+0000 7f37f8a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639b857c2f0 0x5639b895c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:04.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.513+0000 7f37f3e00640 1 -- 172.21.15.16:0/409005096 learned_addr learned my addr 172.21.15.16:0/409005096 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:04.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.513+0000 7f37f9400640 1 --2- 172.21.15.16:0/409005096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639b85a51b0 0x5639b895a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:04.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.513+0000 7f37f8a00640 1 -- 172.21.15.16:0/409005096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639b85a51b0 msgr2=0x5639b895a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:04.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.513+0000 7f37f8a00640 1 --2- 172.21.15.16:0/409005096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639b85a51b0 0x5639b895a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:04.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.513+0000 7f37f8a00640 1 -- 172.21.15.16:0/409005096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b85844d0 msgr2=0x5639b8959bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:04.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.513+0000 7f37f8a00640 1 --2- 172.21.15.16:0/409005096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b85844d0 0x5639b8959bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:04.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.513+0000 7f37f8a00640 1 -- 172.21.15.16:0/409005096 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639b895a770 con 0x5639b857c2f0 2024-08-31T06:07:04.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.513+0000 7f37f3e00640 1 --2- 172.21.15.16:0/409005096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b85844d0 0x5639b8959bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:07:04.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.514+0000 7f37f8a00640 1 --2- 172.21.15.16:0/409005096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639b857c2f0 0x5639b895c730 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f37e80c83a0 tx=0x7f37e8003e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9aeb273c1d0fb771 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:04.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.514+0000 7f37da000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:07:04.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.514+0000 7f37da000640 1 -- 172.21.15.16:0/409005096 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e8049070 con 0x5639b857c2f0 2024-08-31T06:07:04.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.514+0000 7f38025d2740 1 -- 172.21.15.16:0/409005096 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37e001f040 con 0x5639b857c2f0 2024-08-31T06:07:04.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.514+0000 7f37da000640 1 -- 172.21.15.16:0/409005096 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f37e80d5310 con 0x5639b857c2f0 2024-08-31T06:07:04.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.514+0000 7f37da000640 1 -- 172.21.15.16:0/409005096 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f37e80cbc70 con 0x5639b857c2f0 2024-08-31T06:07:04.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.514+0000 7f38025d2740 10 client.25430 fetch_fsmap learned FSMap version 25 2024-08-31T06:07:04.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.514+0000 7f38025d2740 1 -- 172.21.15.16:0/409005096 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5639b89d0470 con 0x5639b857c2f0 2024-08-31T06:07:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.514+0000 7f37da000640 1 -- 172.21.15.16:0/409005096 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37e001f040 con 0x5639b857c2f0 2024-08-31T06:07:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.515+0000 7f37da000640 1 -- 172.21.15.16:0/409005096 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f37e80cb580 con 0x5639b857c2f0 2024-08-31T06:07:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.515+0000 7f38025d2740 10 client.25430 fetch_fsmap finished waiting for FSMap version 25 2024-08-31T06:07:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.515+0000 7f38025d2740 10 client.25430 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:07:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.515+0000 7f38025d2740 20 client.25430 populate_metadata read hostname 'smithi016' 2024-08-31T06:07:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.515+0000 7f38025d2740 1 --2- 172.21.15.16:0/409005096 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5639b85391c0 0x5639b89d2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.515+0000 7f38025d2740 4 client.25430 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:07:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.515+0000 7f38025d2740 1 -- 172.21.15.16:0/409005096 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5639b88bb960 con 0x5639b85391c0 2024-08-31T06:07:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.515+0000 7f37f3e00640 1 --2- 172.21.15.16:0/409005096 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5639b85391c0 0x5639b89d2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.516+0000 7f37f3e00640 1 --2- 172.21.15.16:0/409005096 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5639b85391c0 0x5639b89d2a80 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c9e3ea6bfcef6e0 server_cookie=488a73eed159df72 in_seq=0 out_seq=0 2024-08-31T06:07:04.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.516+0000 7f37da000640 10 client.25430 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:04.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.529+0000 7f37da000640 1 -- 172.21.15.16:0/409005096 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5639b88bb960 con 0x5639b85391c0 2024-08-31T06:07:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.529+0000 7f37da000640 10 client.25430 handle_command_reply: tid=0 2024-08-31T06:07:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.529+0000 7f38025d2740 1 client.25430 shutdown 2024-08-31T06:07:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.529+0000 7f38025d2740 1 -- 172.21.15.16:0/409005096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639b857c2f0 msgr2=0x5639b895c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.529+0000 7f38025d2740 1 --2- 172.21.15.16:0/409005096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639b857c2f0 0x5639b895c730 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f37e80c83a0 tx=0x7f37e8003e80 comp rx=0 tx=0).stop 2024-08-31T06:07:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.529+0000 7f38025d2740 1 -- 172.21.15.16:0/409005096 shutdown_connections 2024-08-31T06:07:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.529+0000 7f38025d2740 1 --2- 172.21.15.16:0/409005096 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5639b85391c0 0x5639b89d2a80 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.529+0000 7f38025d2740 1 --2- 172.21.15.16:0/409005096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639b85a51b0 0x5639b895a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.529+0000 7f38025d2740 1 --2- 172.21.15.16:0/409005096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b85844d0 0x5639b8959bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.529+0000 7f38025d2740 1 --2- 172.21.15.16:0/409005096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639b857c2f0 0x5639b895c730 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.529+0000 7f37da000640 0 client.25430 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.529+0000 7f38025d2740 1 -- 172.21.15.16:0/409005096 >> 172.21.15.16:0/409005096 conn(0x5639b8599300 msgr2=0x5639b8949e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.529+0000 7f38025d2740 1 -- 172.21.15.16:0/409005096 shutdown_connections 2024-08-31T06:07:04.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.529+0000 7f38025d2740 1 -- 172.21.15.16:0/409005096 wait complete. 2024-08-31T06:07:04.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.529+0000 7f38025d2740 20 client.25430 trim_cache size 0 max 16384 2024-08-31T06:07:04.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:07:04.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:07:04.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:07:04.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:07:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.530+0000 7f37fb400640 1 -- 172.21.15.16:0/4273854815 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f37dc0b4c30 msgr2=0x7f37dc0b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.530+0000 7f37fb400640 1 --2- 172.21.15.16:0/4273854815 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f37dc0b4c30 0x7f37dc0b7140 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f37ec005fd0 tx=0x7f37ec0096a0 comp rx=0 tx=0).stop 2024-08-31T06:07:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.530+0000 7f37fb400640 1 -- 172.21.15.16:0/4273854815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37f41046d0 msgr2=0x7f37f419c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.530+0000 7f37fb400640 1 --2- 172.21.15.16:0/4273854815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37f41046d0 0x7f37f419c420 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f37e8005940 tx=0x7f37e8004270 comp rx=0 tx=0).stop 2024-08-31T06:07:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.530+0000 7f37fb400640 1 -- 172.21.15.16:0/4273854815 shutdown_connections 2024-08-31T06:07:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.530+0000 7f37fb400640 1 --2- 172.21.15.16:0/4273854815 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f37dc0b4c30 0x7f37dc0b7140 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.530+0000 7f37fb400640 1 --2- 172.21.15.16:0/4273854815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37f4105020 0x7f37f419c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.530+0000 7f37fb400640 1 --2- 172.21.15.16:0/4273854815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f410db30 0x7f37f41a1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.530+0000 7f37fb400640 1 --2- 172.21.15.16:0/4273854815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37f41046d0 0x7f37f419c420 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.530+0000 7f37fb400640 1 -- 172.21.15.16:0/4273854815 >> 172.21.15.16:0/4273854815 conn(0x7f37f40fe160 msgr2=0x7f37f4102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:04.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.530+0000 7f37fb400640 1 -- 172.21.15.16:0/4273854815 shutdown_connections 2024-08-31T06:07:04.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:04.530+0000 7f37fb400640 1 -- 172.21.15.16:0/4273854815 wait complete. 2024-08-31T06:07:04.760 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:07:05.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:04 smithi016 ceph-mon[17194]: pgmap v358: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 185 KiB/s wr, 64 op/s 2024-08-31T06:07:05.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:04 smithi184 ceph-mon[22331]: pgmap v358: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 185 KiB/s wr, 64 op/s 2024-08-31T06:07:05.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:04 smithi155 ceph-mon[21742]: pgmap v358: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 185 KiB/s wr, 64 op/s 2024-08-31T06:07:06.392 INFO:teuthology.orchestra.run.smithi016.stdout:{"run_dir":"/var/run/ceph"} 2024-08-31T06:07:06.392 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-08-31T06:07:06.392 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 5413 2024-08-31T06:07:06.392 DEBUG:tasks.quiescer:Yielding 2024-08-31T06:07:06.393 INFO:teuthology.run_tasks:Running task exec... 2024-08-31T06:07:06.402 INFO:teuthology.task.exec:Executing custom commands... 2024-08-31T06:07:06.402 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi016.front.sepia.ceph.com 2024-08-31T06:07:06.403 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2024-08-31T06:07:06.403 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:07:06.487 INFO:teuthology.orchestra.run.smithi016.stdout:total 0 2024-08-31T06:07:06.518 INFO:teuthology.orchestra.run.smithi016.stdout:Filesystem Size Used Avail Use% Mounted on 2024-08-31T06:07:06.519 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6789,172.21.15.155:6789,172.21.15.184:6789:/volumes/qa/sv_1/ba5423bc-5c83-4d64-ae1c-ad8850110ce4 340G 0 340G 0% /tmp/cephfs 2024-08-31T06:07:06.524 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2024-08-31T06:07:06.675 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2024-08-31T06:07:06.761 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2024-08-31T06:07:06.792 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:07:07.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:06 smithi016 ceph-mon[17194]: pgmap v359: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 96 KiB/s wr, 32 op/s 2024-08-31T06:07:07.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:06 smithi184 ceph-mon[22331]: pgmap v359: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 96 KiB/s wr, 32 op/s 2024-08-31T06:07:07.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:06 smithi155 ceph-mon[21742]: pgmap v359: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 96 KiB/s wr, 32 op/s 2024-08-31T06:07:07.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.495+0000 7f302a400640 1 -- 172.21.15.16:0/1166048245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3024075b70 msgr2=0x7f3024075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:07.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.495+0000 7f302a400640 1 --2- 172.21.15.16:0/1166048245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3024075b70 0x7f3024075f80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3010009860 tx=0x7f301002a340 comp rx=0 tx=0).stop 2024-08-31T06:07:07.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.495+0000 7f302a400640 1 -- 172.21.15.16:0/1166048245 shutdown_connections 2024-08-31T06:07:07.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.495+0000 7f302a400640 1 --2- 172.21.15.16:0/1166048245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3024114020 0x7f30241164f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:07.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.495+0000 7f302a400640 1 --2- 172.21.15.16:0/1166048245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3024075b70 0x7f3024075f80 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:07.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.495+0000 7f302a400640 1 --2- 172.21.15.16:0/1166048245 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3024077120 0x7f3024075550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:07.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.495+0000 7f302a400640 1 -- 172.21.15.16:0/1166048245 >> 172.21.15.16:0/1166048245 conn(0x7f30240ff1e0 msgr2=0x7f3024101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:07.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.495+0000 7f302a400640 1 -- 172.21.15.16:0/1166048245 shutdown_connections 2024-08-31T06:07:07.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.495+0000 7f302a400640 1 -- 172.21.15.16:0/1166048245 wait complete. 2024-08-31T06:07:07.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.496+0000 7f302a400640 1 Processor -- start 2024-08-31T06:07:07.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.497+0000 7f302a400640 1 -- start start 2024-08-31T06:07:07.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.497+0000 7f302a400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3024075b70 0x7f30241a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:07.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.497+0000 7f302a400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3024077120 0x7f30241a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:07.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.497+0000 7f302a400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3024114020 0x7f30241a5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:07.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.497+0000 7f302a400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3024119c30 con 0x7f3024114020 2024-08-31T06:07:07.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.497+0000 7f302a400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3024119ab0 con 0x7f3024075b70 2024-08-31T06:07:07.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.497+0000 7f302a400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3024119db0 con 0x7f3024077120 2024-08-31T06:07:07.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.497+0000 7f3029a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3024114020 0x7f30241a5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:07.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.498+0000 7f3029000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3024075b70 0x7f30241a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:07.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.498+0000 7f3023e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3024077120 0x7f30241a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:07.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.498+0000 7f3029a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3024114020 0x7f30241a5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46364/0 (socket says 172.21.15.16:46364) 2024-08-31T06:07:07.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.498+0000 7f3029000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3024075b70 0x7f30241a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51786/0 (socket says 172.21.15.16:51786) 2024-08-31T06:07:07.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.498+0000 7f3029a00640 1 -- 172.21.15.16:0/1125492348 learned_addr learned my addr 172.21.15.16:0/1125492348 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:07.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.498+0000 7f3023e00640 1 -- 172.21.15.16:0/1125492348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3024075b70 msgr2=0x7f30241a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:07.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.498+0000 7f3023e00640 1 --2- 172.21.15.16:0/1125492348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3024075b70 0x7f30241a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:07.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.498+0000 7f3023e00640 1 -- 172.21.15.16:0/1125492348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3024114020 msgr2=0x7f30241a5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:07.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.498+0000 7f3023e00640 1 --2- 172.21.15.16:0/1125492348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3024114020 0x7f30241a5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:07.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.498+0000 7f3023e00640 1 -- 172.21.15.16:0/1125492348 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30241a65b0 con 0x7f3024077120 2024-08-31T06:07:07.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.498+0000 7f3029000640 1 --2- 172.21.15.16:0/1125492348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3024075b70 0x7f30241a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:07:07.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.498+0000 7f3023e00640 1 --2- 172.21.15.16:0/1125492348 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3024077120 0x7f30241a0be0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f301002a860 tx=0x7f3010032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:07.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.499+0000 7f3021600640 1 -- 172.21.15.16:0/1125492348 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3010037040 con 0x7f3024077120 2024-08-31T06:07:07.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.499+0000 7f302a400640 1 -- 172.21.15.16:0/1125492348 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30241a6840 con 0x7f3024077120 2024-08-31T06:07:07.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.499+0000 7f3021600640 1 -- 172.21.15.16:0/1125492348 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3010030c10 con 0x7f3024077120 2024-08-31T06:07:07.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.499+0000 7f302a400640 1 -- 172.21.15.16:0/1125492348 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30241ad000 con 0x7f3024077120 2024-08-31T06:07:07.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.499+0000 7f3021600640 1 -- 172.21.15.16:0/1125492348 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30100035f0 con 0x7f3024077120 2024-08-31T06:07:07.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.500+0000 7f302a400640 1 -- 172.21.15.16:0/1125492348 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3024076a60 con 0x7f3024077120 2024-08-31T06:07:07.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.503+0000 7f3021600640 1 -- 172.21.15.16:0/1125492348 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f3010043020 con 0x7f3024077120 2024-08-31T06:07:07.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.504+0000 7f3021600640 1 --2- 172.21.15.16:0/1125492348 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f300c0b4d20 0x7f300c0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:07.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.504+0000 7f3021600640 1 -- 172.21.15.16:0/1125492348 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f30101047c0 con 0x7f3024077120 2024-08-31T06:07:07.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.504+0000 7f3029000640 1 --2- 172.21.15.16:0/1125492348 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f300c0b4d20 0x7f300c0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:07.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.505+0000 7f3029000640 1 --2- 172.21.15.16:0/1125492348 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f300c0b4d20 0x7f300c0b7230 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f30180096c0 tx=0x7f3018009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:07.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.506+0000 7f3021600640 1 -- 172.21.15.16:0/1125492348 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f30100cb710 con 0x7f3024077120 2024-08-31T06:07:07.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.738+0000 7f302a400640 1 -- 172.21.15.16:0/1125492348 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3024075f80 con 0x7f3024077120 2024-08-31T06:07:07.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.739+0000 7f3021600640 1 -- 172.21.15.16:0/1125492348 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11917 (secure 0 0 0) 0x7f30100cb010 con 0x7f3024077120 2024-08-31T06:07:07.741 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:07:07.741 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-08-31T06:06:38:485237+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:06:37.778867+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24698":{"gid":24698,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1089710405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1089710405},{"type":"v1","addr":"172.21.15.155:6837","nonce":1089710405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24737,24724,24712]},"id":1}]} 2024-08-31T06:07:07.741 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-08-31T06:07:07.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.742+0000 7f302a400640 1 -- 172.21.15.16:0/1125492348 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f300c0b4d20 msgr2=0x7f300c0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:07.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.742+0000 7f302a400640 1 --2- 172.21.15.16:0/1125492348 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f300c0b4d20 0x7f300c0b7230 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f30180096c0 tx=0x7f3018009260 comp rx=0 tx=0).stop 2024-08-31T06:07:07.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.742+0000 7f302a400640 1 -- 172.21.15.16:0/1125492348 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3024077120 msgr2=0x7f30241a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:07.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.742+0000 7f302a400640 1 --2- 172.21.15.16:0/1125492348 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3024077120 0x7f30241a0be0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f301002a860 tx=0x7f3010032d50 comp rx=0 tx=0).stop 2024-08-31T06:07:07.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.742+0000 7f302a400640 1 -- 172.21.15.16:0/1125492348 shutdown_connections 2024-08-31T06:07:07.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.742+0000 7f302a400640 1 --2- 172.21.15.16:0/1125492348 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f300c0b4d20 0x7f300c0b7230 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:07.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.742+0000 7f302a400640 1 --2- 172.21.15.16:0/1125492348 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3024077120 0x7f30241a0be0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:07.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.742+0000 7f302a400640 1 --2- 172.21.15.16:0/1125492348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3024114020 0x7f30241a5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:07.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.742+0000 7f302a400640 1 --2- 172.21.15.16:0/1125492348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3024075b70 0x7f30241a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:07.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.742+0000 7f302a400640 1 -- 172.21.15.16:0/1125492348 >> 172.21.15.16:0/1125492348 conn(0x7f30240ff1e0 msgr2=0x7f302410ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:07.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.743+0000 7f302a400640 1 -- 172.21.15.16:0/1125492348 shutdown_connections 2024-08-31T06:07:07.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:07.743+0000 7f302a400640 1 -- 172.21.15.16:0/1125492348 wait complete. 2024-08-31T06:07:08.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:07 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/1125492348' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:07:08.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:07 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/1125492348' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:07:08.273 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:07 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/1125492348' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:07:09.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:08 smithi184 ceph-mon[22331]: pgmap v360: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-08-31T06:07:09.271 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:08 smithi016 ceph-mon[17194]: pgmap v360: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-08-31T06:07:09.310 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-08-31T06:04:27.098649+0000', 'modified': '2024-08-31T06:06:37.778867+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24737, 'mds_1': 24724, 'mds_2': 24712}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24688': {'gid': 24688, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.184:6835/1925472309', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6834', 'nonce': 1925472309}, {'type': 'v1', 'addr': '172.21.15.184:6835', 'nonce': 1925472309}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24694': {'gid': 24694, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.184:6837/1712755588', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6836', 'nonce': 1712755588}, {'type': 'v1', 'addr': '172.21.15.184:6837', 'nonce': 1712755588}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24698': {'gid': 24698, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.155:6837/1089710405', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6836', 'nonce': 1089710405}, {'type': 'v1', 'addr': '172.21.15.155:6837', 'nonce': 1089710405}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24712': {'gid': 24712, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.184:6839/4011051479', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6838', 'nonce': 4011051479}, {'type': 'v1', 'addr': '172.21.15.184:6839', 'nonce': 4011051479}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24724': {'gid': 24724, 'name': 'g', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.16:6839/219664460', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6838', 'nonce': 219664460}, {'type': 'v1', 'addr': '172.21.15.16:6839', 'nonce': 219664460}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24737': {'gid': 24737, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.155:6839/390606290', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6838', 'nonce': 390606290}, {'type': 'v1', 'addr': '172.21.15.155:6839', 'nonce': 390606290}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24737, 'qdb_cluster': [24737, 24724, 24712]} 2024-08-31T06:07:09.310 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-08-31T06:07:09.311 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.i status 2024-08-31T06:07:09.355 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2024-08-31T06:07:09.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:08 smithi155 ceph-mon[21742]: pgmap v360: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-08-31T06:07:09.473 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2024-08-31T06:07:09.477 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2024-08-31T06:07:09.562 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-08-31T06:07:09.641 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/: 2024-08-31T06:07:09.641 INFO:teuthology.orchestra.run.smithi016.stdout:total 0 2024-08-31T06:07:09.641 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 31 06:07 . 2024-08-31T06:07:09.642 INFO:teuthology.orchestra.run.smithi016.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Aug 31 06:07 .. 2024-08-31T06:07:09.642 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 data 2024-08-31T06:07:09.642 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:07:09.642 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data: 2024-08-31T06:07:09.642 INFO:teuthology.orchestra.run.smithi016.stdout:total 0 2024-08-31T06:07:09.642 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 . 2024-08-31T06:07:09.642 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 31 06:07 .. 2024-08-31T06:07:09.647 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2024-08-31T06:07:09.745 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:07:09.753 INFO:teuthology.orchestra.run.smithi016.stdout:[Service] 2024-08-31T06:07:09.753 INFO:teuthology.orchestra.run.smithi016.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2024-08-31T06:07:09.753 INFO:teuthology.orchestra.run.smithi016.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2024-08-31T06:07:09.758 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2024-08-31T06:07:09.967 INFO:teuthology.orchestra.run.smithi016.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2024-08-31T06:07:10.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.494+0000 7f6718200640 1 -- 172.21.15.16:0/645467097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f671010fe60 msgr2=0x7f67101122a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:10.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.494+0000 7f6718200640 1 --2- 172.21.15.16:0/645467097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f671010fe60 0x7f67101122a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6708009ca0 tx=0x7f670802a8c0 comp rx=0 tx=0).stop 2024-08-31T06:07:10.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.494+0000 7f6718200640 1 -- 172.21.15.16:0/645467097 shutdown_connections 2024-08-31T06:07:10.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.494+0000 7f6718200640 1 --2- 172.21.15.16:0/645467097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6710075ea0 0x7f671010f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:10.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.494+0000 7f6718200640 1 --2- 172.21.15.16:0/645467097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f671010fe60 0x7f67101122a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:10.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.494+0000 7f6718200640 1 --2- 172.21.15.16:0/645467097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6710075550 0x7f6710075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:10.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.494+0000 7f6718200640 1 -- 172.21.15.16:0/645467097 >> 172.21.15.16:0/645467097 conn(0x7f671007b5f0 msgr2=0x7f671007ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:10.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.494+0000 7f6718200640 1 -- 172.21.15.16:0/645467097 shutdown_connections 2024-08-31T06:07:10.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.495+0000 7f6718200640 1 -- 172.21.15.16:0/645467097 wait complete. 2024-08-31T06:07:10.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.496+0000 7f6718200640 1 Processor -- start 2024-08-31T06:07:10.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.496+0000 7f6718200640 1 -- start start 2024-08-31T06:07:10.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.497+0000 7f6718200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6710075550 0x7f671019c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:10.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.497+0000 7f6718200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6710075ea0 0x7f671019c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:10.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.497+0000 7f6718200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f671010fe60 0x7f67101a1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:10.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.497+0000 7f6718200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6710115a10 con 0x7f6710075550 2024-08-31T06:07:10.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.497+0000 7f6718200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6710115890 con 0x7f6710075ea0 2024-08-31T06:07:10.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.497+0000 7f6718200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6710115b90 con 0x7f671010fe60 2024-08-31T06:07:10.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.497+0000 7f6714e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6710075ea0 0x7f671019c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:10.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.497+0000 7f6714e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6710075ea0 0x7f671019c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51800/0 (socket says 172.21.15.16:51800) 2024-08-31T06:07:10.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.497+0000 7f6715800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6710075550 0x7f671019c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:10.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.497+0000 7f6714e00640 1 -- 172.21.15.16:0/2183885979 learned_addr learned my addr 172.21.15.16:0/2183885979 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:10.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.497+0000 7f6716200640 1 --2- 172.21.15.16:0/2183885979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f671010fe60 0x7f67101a1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:10.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.497+0000 7f6714e00640 1 -- 172.21.15.16:0/2183885979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f671010fe60 msgr2=0x7f67101a1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:10.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.497+0000 7f6714e00640 1 --2- 172.21.15.16:0/2183885979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f671010fe60 0x7f67101a1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:10.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.497+0000 7f6714e00640 1 -- 172.21.15.16:0/2183885979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6710075550 msgr2=0x7f671019c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:10.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.497+0000 7f6714e00640 1 --2- 172.21.15.16:0/2183885979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6710075550 0x7f671019c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:10.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.497+0000 7f6714e00640 1 -- 172.21.15.16:0/2183885979 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67101a22d0 con 0x7f6710075ea0 2024-08-31T06:07:10.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.498+0000 7f6714e00640 1 --2- 172.21.15.16:0/2183885979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6710075ea0 0x7f671019c900 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f66fc00e9c0 tx=0x7f66fc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:10.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.498+0000 7f670e000640 1 -- 172.21.15.16:0/2183885979 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66fc00cd80 con 0x7f6710075ea0 2024-08-31T06:07:10.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.498+0000 7f670e000640 1 -- 172.21.15.16:0/2183885979 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66fc0035f0 con 0x7f6710075ea0 2024-08-31T06:07:10.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.498+0000 7f670e000640 1 -- 172.21.15.16:0/2183885979 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66fc010690 con 0x7f6710075ea0 2024-08-31T06:07:10.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.498+0000 7f6718200640 1 -- 172.21.15.16:0/2183885979 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67101a25c0 con 0x7f6710075ea0 2024-08-31T06:07:10.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.498+0000 7f6718200640 1 -- 172.21.15.16:0/2183885979 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67101a8d20 con 0x7f6710075ea0 2024-08-31T06:07:10.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.501+0000 7f671ee67740 1 Processor -- start 2024-08-31T06:07:10.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.501+0000 7f671ee67740 1 -- start start 2024-08-31T06:07:10.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.502+0000 7f671ee67740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d4e444f60 0x560d4e7ea8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:10.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.502+0000 7f671ee67740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d4e41c0a0 0x560d4e7eae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:10.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.502+0000 7f671ee67740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d4e424280 0x560d4e7eb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:10.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.502+0000 7f671ee67740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560d4e761550 con 0x560d4e424280 2024-08-31T06:07:10.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.502+0000 7f671ee67740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560d4e750ae0 con 0x560d4e41c0a0 2024-08-31T06:07:10.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.502+0000 7f671ee67740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560d4e754a90 con 0x560d4e444f60 2024-08-31T06:07:10.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.502+0000 7f670e000640 1 -- 172.21.15.16:0/2183885979 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f66fc004000 con 0x7f6710075ea0 2024-08-31T06:07:10.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.502+0000 7f6715800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d4e41c0a0 0x560d4e7eae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:10.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.502+0000 7f6715800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d4e41c0a0 0x560d4e7eae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51810/0 (socket says 172.21.15.16:51810) 2024-08-31T06:07:10.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.502+0000 7f6715800640 1 -- 172.21.15.16:0/1156468575 learned_addr learned my addr 172.21.15.16:0/1156468575 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:10.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.502+0000 7f6716200640 1 --2- 172.21.15.16:0/1156468575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d4e444f60 0x560d4e7ea8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:10.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.502+0000 7f6714e00640 1 --2- 172.21.15.16:0/1156468575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d4e424280 0x560d4e7eb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:10.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.503+0000 7f6715800640 1 -- 172.21.15.16:0/1156468575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d4e444f60 msgr2=0x560d4e7ea8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:10.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.503+0000 7f6715800640 1 --2- 172.21.15.16:0/1156468575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d4e444f60 0x560d4e7ea8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:10.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.503+0000 7f6715800640 1 -- 172.21.15.16:0/1156468575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d4e424280 msgr2=0x560d4e7eb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:10.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.503+0000 7f6715800640 1 --2- 172.21.15.16:0/1156468575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d4e424280 0x560d4e7eb370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:10.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.503+0000 7f6715800640 1 -- 172.21.15.16:0/1156468575 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d4e4b0540 con 0x560d4e41c0a0 2024-08-31T06:07:10.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.503+0000 7f6716200640 1 --2- 172.21.15.16:0/1156468575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d4e444f60 0x560d4e7ea8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:07:10.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.503+0000 7f6715800640 1 --2- 172.21.15.16:0/1156468575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d4e41c0a0 0x560d4e7eae30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6704009cf0 tx=0x7f6704011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7408ab11fa0d6878 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:10.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.503+0000 7f6702a00640 1 -- 172.21.15.16:0/1156468575 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6704004230 con 0x560d4e41c0a0 2024-08-31T06:07:10.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.503+0000 7f6702a00640 1 -- 172.21.15.16:0/1156468575 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67040043d0 con 0x560d4e41c0a0 2024-08-31T06:07:10.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.504+0000 7f6702a00640 1 -- 172.21.15.16:0/1156468575 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6704019880 con 0x560d4e41c0a0 2024-08-31T06:07:10.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.504+0000 7f670e000640 1 --2- 172.21.15.16:0/2183885979 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f66f80b4d20 0x7f66f80b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:10.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.504+0000 7f670e000640 1 -- 172.21.15.16:0/2183885979 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f66fc0e5ac0 con 0x7f6710075ea0 2024-08-31T06:07:10.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.504+0000 7f6716200640 1 --2- 172.21.15.16:0/2183885979 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f66f80b4d20 0x7f66f80b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:10.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.505+0000 7f671ee67740 1 -- 172.21.15.16:0/1156468575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d4e41c0a0 msgr2=0x560d4e7eae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:10.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.505+0000 7f671ee67740 1 --2- 172.21.15.16:0/1156468575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d4e41c0a0 0x560d4e7eae30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6704009cf0 tx=0x7f6704011810 comp rx=0 tx=0).stop 2024-08-31T06:07:10.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.505+0000 7f671ee67740 1 -- 172.21.15.16:0/1156468575 shutdown_connections 2024-08-31T06:07:10.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.505+0000 7f671ee67740 1 --2- 172.21.15.16:0/1156468575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d4e424280 0x560d4e7eb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:10.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.505+0000 7f671ee67740 1 --2- 172.21.15.16:0/1156468575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d4e41c0a0 0x560d4e7eae30 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:10.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.505+0000 7f671ee67740 1 --2- 172.21.15.16:0/1156468575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d4e444f60 0x560d4e7ea8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:10.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.505+0000 7f671ee67740 1 -- 172.21.15.16:0/1156468575 >> 172.21.15.16:0/1156468575 conn(0x560d4e439350 msgr2=0x560d4e7e87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:10.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.505+0000 7f671ee67740 1 -- 172.21.15.16:0/1156468575 shutdown_connections 2024-08-31T06:07:10.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.505+0000 7f6716200640 1 --2- 172.21.15.16:0/2183885979 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f66f80b4d20 0x7f66f80b7230 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6708005e30 tx=0x7f6708005c30 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:10.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.505+0000 7f671ee67740 1 -- 172.21.15.16:0/1156468575 wait complete. 2024-08-31T06:07:10.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.506+0000 7f671ee67740 1 -- start start 2024-08-31T06:07:10.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.506+0000 7f671ee67740 1 Processor -- start 2024-08-31T06:07:10.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.506+0000 7f671ee67740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d4e41c0a0 0x560d4e7f5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:10.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.506+0000 7f671ee67740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d4e424280 0x560d4e7f6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:10.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.506+0000 7f671ee67740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d4e444f60 0x560d4e8023f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:10.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.507+0000 7f671ee67740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560d4e5b35f0 con 0x560d4e424280 2024-08-31T06:07:10.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.507+0000 7f671ee67740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560d4e6d7bd0 con 0x560d4e444f60 2024-08-31T06:07:10.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.507+0000 7f671ee67740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560d4e7f90d0 con 0x560d4e41c0a0 2024-08-31T06:07:10.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.507+0000 7f671ee67740 10 client.0 _ll_register_callbacks cb 0x560d4e6fd210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:07:10.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.507+0000 7f6716200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d4e444f60 0x560d4e8023f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:10.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.507+0000 7f6714e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d4e424280 0x560d4e7f6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:10.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.507+0000 7f6716200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d4e444f60 0x560d4e8023f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51824/0 (socket says 172.21.15.16:51824) 2024-08-31T06:07:10.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.507+0000 7f6716200640 1 -- 172.21.15.16:0/855636427 learned_addr learned my addr 172.21.15.16:0/855636427 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:10.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.507+0000 7f6715800640 1 --2- 172.21.15.16:0/855636427 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d4e41c0a0 0x560d4e7f5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:10.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.507+0000 7f6716200640 1 -- 172.21.15.16:0/855636427 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d4e41c0a0 msgr2=0x560d4e7f5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:10.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.507+0000 7f6716200640 1 --2- 172.21.15.16:0/855636427 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d4e41c0a0 0x560d4e7f5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:10.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.507+0000 7f6716200640 1 -- 172.21.15.16:0/855636427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d4e424280 msgr2=0x560d4e7f6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:10.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.507+0000 7f6716200640 1 --2- 172.21.15.16:0/855636427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d4e424280 0x560d4e7f6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:10.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.507+0000 7f6716200640 1 -- 172.21.15.16:0/855636427 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d4e718650 con 0x560d4e444f60 2024-08-31T06:07:10.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.508+0000 7f6716200640 1 --2- 172.21.15.16:0/855636427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d4e444f60 0x560d4e8023f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6708004a40 tx=0x7f670802a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=603b58b22f8c6468 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:10.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.508+0000 7f66f7400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:07:10.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.508+0000 7f66f7400640 1 -- 172.21.15.16:0/855636427 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6708003b80 con 0x560d4e444f60 2024-08-31T06:07:10.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.508+0000 7f671ee67740 1 -- 172.21.15.16:0/855636427 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67040043d0 con 0x560d4e444f60 2024-08-31T06:07:10.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.508+0000 7f66f7400640 1 -- 172.21.15.16:0/855636427 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f6708003d20 con 0x560d4e444f60 2024-08-31T06:07:10.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.508+0000 7f66f7400640 1 -- 172.21.15.16:0/855636427 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6708004040 con 0x560d4e444f60 2024-08-31T06:07:10.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.508+0000 7f671ee67740 10 client.25448 fetch_fsmap learned FSMap version 25 2024-08-31T06:07:10.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.508+0000 7f671ee67740 1 -- 172.21.15.16:0/855636427 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560d4e804e70 con 0x560d4e444f60 2024-08-31T06:07:10.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.509+0000 7f66f7400640 1 -- 172.21.15.16:0/855636427 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67040043d0 con 0x560d4e444f60 2024-08-31T06:07:10.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.509+0000 7f66f7400640 1 -- 172.21.15.16:0/855636427 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6708051070 con 0x560d4e444f60 2024-08-31T06:07:10.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.509+0000 7f671ee67740 10 client.25448 fetch_fsmap finished waiting for FSMap version 25 2024-08-31T06:07:10.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.509+0000 7f671ee67740 10 client.25448 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-31T06:07:10.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.509+0000 7f671ee67740 20 client.25448 populate_metadata read hostname 'smithi016' 2024-08-31T06:07:10.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.509+0000 7f671ee67740 1 --2- 172.21.15.16:0/855636427 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x560d4e4038e0 0x560d4e8089e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:10.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.509+0000 7f671ee67740 4 client.25448 mds_command: new command op to 24712 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:07:10.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.509+0000 7f671ee67740 1 -- 172.21.15.16:0/855636427 --> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d4e7634a0 con 0x560d4e4038e0 2024-08-31T06:07:10.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.509+0000 7f6715800640 1 --2- 172.21.15.16:0/855636427 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x560d4e4038e0 0x560d4e8089e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.510+0000 7f6715800640 1 --2- 172.21.15.16:0/855636427 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x560d4e4038e0 0x560d4e8089e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6258392992c26f9e server_cookie=62626d10ce44a56e in_seq=0 out_seq=0 2024-08-31T06:07:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.510+0000 7f66f7400640 10 client.25448 ms_handle_connect on v2:172.21.15.184:6838/4011051479 2024-08-31T06:07:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.511+0000 7f66f7400640 1 -- 172.21.15.16:0/855636427 <== mds.2 v2:172.21.15.184:6838/4011051479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560d4e7634a0 con 0x560d4e4038e0 2024-08-31T06:07:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.511+0000 7f66f7400640 10 client.25448 handle_command_reply: tid=0 2024-08-31T06:07:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.511+0000 7f671ee67740 1 client.25448 shutdown 2024-08-31T06:07:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.511+0000 7f671ee67740 1 -- 172.21.15.16:0/855636427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d4e444f60 msgr2=0x560d4e8023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.511+0000 7f671ee67740 1 --2- 172.21.15.16:0/855636427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d4e444f60 0x560d4e8023f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6708004a40 tx=0x7f670802a8c0 comp rx=0 tx=0).stop 2024-08-31T06:07:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.511+0000 7f671ee67740 1 -- 172.21.15.16:0/855636427 shutdown_connections 2024-08-31T06:07:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.511+0000 7f671ee67740 1 --2- 172.21.15.16:0/855636427 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x560d4e4038e0 0x560d4e8089e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.511+0000 7f671ee67740 1 --2- 172.21.15.16:0/855636427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d4e444f60 0x560d4e8023f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.511+0000 7f671ee67740 1 --2- 172.21.15.16:0/855636427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d4e424280 0x560d4e7f6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.511+0000 7f671ee67740 1 --2- 172.21.15.16:0/855636427 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d4e41c0a0 0x560d4e7f5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:10.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.511+0000 7f671ee67740 1 -- 172.21.15.16:0/855636427 >> 172.21.15.16:0/855636427 conn(0x560d4e439350 msgr2=0x560d4e7eda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:10.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.511+0000 7f66f7400640 0 client.25448 ms_handle_reset on v2:172.21.15.184:6838/4011051479 2024-08-31T06:07:10.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.511+0000 7f671ee67740 1 -- 172.21.15.16:0/855636427 shutdown_connections 2024-08-31T06:07:10.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.511+0000 7f671ee67740 1 -- 172.21.15.16:0/855636427 wait complete. 2024-08-31T06:07:10.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.512+0000 7f671ee67740 20 client.25448 trim_cache size 0 max 16384 2024-08-31T06:07:10.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.531+0000 7f671ee67740 1 Processor -- start 2024-08-31T06:07:10.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.531+0000 7f671ee67740 1 -- start start 2024-08-31T06:07:10.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.531+0000 7f671ee67740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d4e41c0a0 0x560d4e7f54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:10.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.531+0000 7f671ee67740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d4e424280 0x560d4e7f1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:10.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.531+0000 7f671ee67740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d4e444f60 0x560d4e7f1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:10.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.531+0000 7f671ee67740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560d4e806d20 con 0x560d4e41c0a0 2024-08-31T06:07:10.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.531+0000 7f671ee67740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560d4e8073f0 con 0x560d4e444f60 2024-08-31T06:07:10.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.531+0000 7f671ee67740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560d4e807ac0 con 0x560d4e424280 2024-08-31T06:07:10.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.532+0000 7f6715800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d4e41c0a0 0x560d4e7f54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:10.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.532+0000 7f6715800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d4e41c0a0 0x560d4e7f54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46412/0 (socket says 172.21.15.16:46412) 2024-08-31T06:07:10.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.532+0000 7f6715800640 1 -- 172.21.15.16:0/2898108016 learned_addr learned my addr 172.21.15.16:0/2898108016 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:10.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.532+0000 7f6714e00640 1 --2- 172.21.15.16:0/2898108016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d4e424280 0x560d4e7f1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:10.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.532+0000 7f6716200640 1 --2- 172.21.15.16:0/2898108016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d4e444f60 0x560d4e7f1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:10.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.532+0000 7f6715800640 1 -- 172.21.15.16:0/2898108016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d4e424280 msgr2=0x560d4e7f1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:10.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.532+0000 7f6715800640 1 --2- 172.21.15.16:0/2898108016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d4e424280 0x560d4e7f1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:10.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.532+0000 7f6715800640 1 -- 172.21.15.16:0/2898108016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d4e444f60 msgr2=0x560d4e7f1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:10.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.532+0000 7f6715800640 1 --2- 172.21.15.16:0/2898108016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d4e444f60 0x560d4e7f1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:10.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.532+0000 7f6715800640 1 -- 172.21.15.16:0/2898108016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d4e7f7090 con 0x560d4e41c0a0 2024-08-31T06:07:10.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.532+0000 7f6715800640 1 --2- 172.21.15.16:0/2898108016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d4e41c0a0 0x560d4e7f54e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6704009c40 tx=0x7f6704019420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=722cb318e0934d96 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:10.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.535+0000 7f6702000640 1 -- 172.21.15.16:0/2898108016 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67040035f0 con 0x560d4e41c0a0 2024-08-31T06:07:10.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.535+0000 7f6702000640 1 -- 172.21.15.16:0/2898108016 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6704005040 con 0x560d4e41c0a0 2024-08-31T06:07:10.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.535+0000 7f6702000640 1 -- 172.21.15.16:0/2898108016 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67040197b0 con 0x560d4e41c0a0 2024-08-31T06:07:10.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.536+0000 7f671ee67740 1 -- 172.21.15.16:0/2898108016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d4e41c0a0 msgr2=0x560d4e7f54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:10.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.536+0000 7f671ee67740 1 --2- 172.21.15.16:0/2898108016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d4e41c0a0 0x560d4e7f54e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6704009c40 tx=0x7f6704019420 comp rx=0 tx=0).stop 2024-08-31T06:07:10.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.536+0000 7f671ee67740 1 -- 172.21.15.16:0/2898108016 shutdown_connections 2024-08-31T06:07:10.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.536+0000 7f671ee67740 1 --2- 172.21.15.16:0/2898108016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d4e424280 0x560d4e7f1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:10.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.536+0000 7f671ee67740 1 --2- 172.21.15.16:0/2898108016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d4e444f60 0x560d4e7f1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:10.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.536+0000 7f671ee67740 1 --2- 172.21.15.16:0/2898108016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d4e41c0a0 0x560d4e7f54e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:10.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.536+0000 7f671ee67740 1 -- 172.21.15.16:0/2898108016 >> 172.21.15.16:0/2898108016 conn(0x560d4e439350 msgr2=0x560d4e7e34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:10.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.536+0000 7f671ee67740 1 -- 172.21.15.16:0/2898108016 shutdown_connections 2024-08-31T06:07:10.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.537+0000 7f671ee67740 1 -- 172.21.15.16:0/2898108016 wait complete. 2024-08-31T06:07:10.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.537+0000 7f671ee67740 1 -- start start 2024-08-31T06:07:10.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.537+0000 7f671ee67740 1 Processor -- start 2024-08-31T06:07:10.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.538+0000 7f671ee67740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d4e41c0a0 0x560d4e80bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:10.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.538+0000 7f671ee67740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d4e424280 0x560d4e80c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:10.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.538+0000 7f671ee67740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d4e444f60 0x560d4e871ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:10.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.538+0000 7f671ee67740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560d4e808860 con 0x560d4e424280 2024-08-31T06:07:10.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.538+0000 7f671ee67740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560d4e808190 con 0x560d4e41c0a0 2024-08-31T06:07:10.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.538+0000 7f671ee67740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560d4e808f30 con 0x560d4e444f60 2024-08-31T06:07:10.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.538+0000 7f671ee67740 10 client.0 _ll_register_callbacks cb 0x7f6708032fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:07:10.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.538+0000 7f6715800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d4e41c0a0 0x560d4e80bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:10.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.538+0000 7f6714e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d4e424280 0x560d4e80c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:10.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.538+0000 7f6715800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d4e41c0a0 0x560d4e80bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51840/0 (socket says 172.21.15.16:51840) 2024-08-31T06:07:10.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.538+0000 7f6715800640 1 -- 172.21.15.16:0/1352131498 learned_addr learned my addr 172.21.15.16:0/1352131498 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:10.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.538+0000 7f6716200640 1 --2- 172.21.15.16:0/1352131498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d4e444f60 0x560d4e871ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:10.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.539+0000 7f6715800640 1 -- 172.21.15.16:0/1352131498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d4e444f60 msgr2=0x560d4e871ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:10.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.539+0000 7f6715800640 1 --2- 172.21.15.16:0/1352131498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d4e444f60 0x560d4e871ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:10.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.539+0000 7f6715800640 1 -- 172.21.15.16:0/1352131498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d4e424280 msgr2=0x560d4e80c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:10.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.539+0000 7f6715800640 1 --2- 172.21.15.16:0/1352131498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d4e424280 0x560d4e80c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:10.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.539+0000 7f6715800640 1 -- 172.21.15.16:0/1352131498 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d4e80cc60 con 0x560d4e41c0a0 2024-08-31T06:07:10.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.539+0000 7f6714e00640 1 --2- 172.21.15.16:0/1352131498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d4e424280 0x560d4e80c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:07:10.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.539+0000 7f6716200640 1 --2- 172.21.15.16:0/1352131498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d4e444f60 0x560d4e871ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:07:10.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.539+0000 7f6715800640 1 --2- 172.21.15.16:0/1352131498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d4e41c0a0 0x560d4e80bf00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6704005e00 tx=0x7f6704005fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11163ce85198e7e3 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:10.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.539+0000 7f66f7400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:07:10.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.540+0000 7f66f7400640 1 -- 172.21.15.16:0/1352131498 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6704038070 con 0x560d4e41c0a0 2024-08-31T06:07:10.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.540+0000 7f66f7400640 1 -- 172.21.15.16:0/1352131498 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f6704005400 con 0x560d4e41c0a0 2024-08-31T06:07:10.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.540+0000 7f671ee67740 1 -- 172.21.15.16:0/1352131498 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6704005040 con 0x560d4e41c0a0 2024-08-31T06:07:10.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.540+0000 7f66f7400640 1 -- 172.21.15.16:0/1352131498 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6704044050 con 0x560d4e41c0a0 2024-08-31T06:07:10.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.540+0000 7f671ee67740 10 client.25454 fetch_fsmap learned FSMap version 25 2024-08-31T06:07:10.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.540+0000 7f671ee67740 1 -- 172.21.15.16:0/1352131498 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560d4e8733b0 con 0x560d4e41c0a0 2024-08-31T06:07:10.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.540+0000 7f66f7400640 1 -- 172.21.15.16:0/1352131498 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6704005040 con 0x560d4e41c0a0 2024-08-31T06:07:10.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.541+0000 7f66f7400640 1 -- 172.21.15.16:0/1352131498 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6704019520 con 0x560d4e41c0a0 2024-08-31T06:07:10.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.541+0000 7f671ee67740 10 client.25454 fetch_fsmap finished waiting for FSMap version 25 2024-08-31T06:07:10.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.541+0000 7f671ee67740 10 client.25454 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-31T06:07:10.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.541+0000 7f671ee67740 20 client.25454 populate_metadata read hostname 'smithi016' 2024-08-31T06:07:10.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.541+0000 7f671ee67740 1 --2- 172.21.15.16:0/1352131498 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x560d4e3d8c90 0x560d4e876570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:10.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.541+0000 7f671ee67740 4 client.25454 mds_command: new command op to 24712 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-31T06:07:10.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.541+0000 7f671ee67740 1 -- 172.21.15.16:0/1352131498 --> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] -- command(tid 0: {"prefix": "status"}) -- 0x560d4e802c50 con 0x560d4e3d8c90 2024-08-31T06:07:10.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.541+0000 7f6714e00640 1 --2- 172.21.15.16:0/1352131498 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x560d4e3d8c90 0x560d4e876570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:10.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.542+0000 7f6714e00640 1 --2- 172.21.15.16:0/1352131498 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x560d4e3d8c90 0x560d4e876570 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bdf8e211c5a35f42 server_cookie=69a338798f163439 in_seq=0 out_seq=0 2024-08-31T06:07:10.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.542+0000 7f66f7400640 10 client.25454 ms_handle_connect on v2:172.21.15.184:6838/4011051479 2024-08-31T06:07:10.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.542+0000 7f66f7400640 1 -- 172.21.15.16:0/1352131498 <== mds.2 v2:172.21.15.184:6838/4011051479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x560d4e802c50 con 0x560d4e3d8c90 2024-08-31T06:07:10.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.542+0000 7f66f7400640 10 client.25454 handle_command_reply: tid=0 2024-08-31T06:07:10.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.542+0000 7f671ee67740 1 client.25454 shutdown 2024-08-31T06:07:10.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.542+0000 7f671ee67740 1 -- 172.21.15.16:0/1352131498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d4e41c0a0 msgr2=0x560d4e80bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:10.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.542+0000 7f671ee67740 1 --2- 172.21.15.16:0/1352131498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d4e41c0a0 0x560d4e80bf00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6704005e00 tx=0x7f6704005fa0 comp rx=0 tx=0).stop 2024-08-31T06:07:10.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.542+0000 7f671ee67740 1 -- 172.21.15.16:0/1352131498 shutdown_connections 2024-08-31T06:07:10.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.542+0000 7f671ee67740 1 --2- 172.21.15.16:0/1352131498 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x560d4e3d8c90 0x560d4e876570 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:10.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.542+0000 7f671ee67740 1 --2- 172.21.15.16:0/1352131498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d4e444f60 0x560d4e871ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:10.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.542+0000 7f671ee67740 1 --2- 172.21.15.16:0/1352131498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d4e424280 0x560d4e80c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:10.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.542+0000 7f671ee67740 1 --2- 172.21.15.16:0/1352131498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d4e41c0a0 0x560d4e80bf00 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:10.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.542+0000 7f66f7400640 0 client.25454 ms_handle_reset on v2:172.21.15.184:6838/4011051479 2024-08-31T06:07:10.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.542+0000 7f671ee67740 1 -- 172.21.15.16:0/1352131498 >> 172.21.15.16:0/1352131498 conn(0x560d4e439350 msgr2=0x560d4e7f1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:10.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.542+0000 7f671ee67740 1 -- 172.21.15.16:0/1352131498 shutdown_connections 2024-08-31T06:07:10.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.543+0000 7f671ee67740 1 -- 172.21.15.16:0/1352131498 wait complete. 2024-08-31T06:07:10.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.543+0000 7f671ee67740 20 client.25454 trim_cache size 0 max 16384 2024-08-31T06:07:10.552 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:07:10.552 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "673e4aa0-675d-11ef-bcd4-c7b262605968", 2024-08-31T06:07:10.552 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 2, 2024-08-31T06:07:10.552 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24712, 2024-08-31T06:07:10.552 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-08-31T06:07:10.552 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-08-31T06:07:10.552 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-08-31T06:07:10.552 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 151.61090810600001, 2024-08-31T06:07:10.552 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 25, 2024-08-31T06:07:10.552 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 80, 2024-08-31T06:07:10.553 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 80, 2024-08-31T06:07:10.553 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 187.835900356 2024-08-31T06:07:10.553 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:07:10.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.544+0000 7f6718200640 1 -- 172.21.15.16:0/2183885979 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f66f80b4d20 msgr2=0x7f66f80b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:10.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.544+0000 7f6718200640 1 --2- 172.21.15.16:0/2183885979 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f66f80b4d20 0x7f66f80b7230 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6708005e30 tx=0x7f6708005c30 comp rx=0 tx=0).stop 2024-08-31T06:07:10.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.544+0000 7f6718200640 1 -- 172.21.15.16:0/2183885979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6710075ea0 msgr2=0x7f671019c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:10.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.544+0000 7f6718200640 1 --2- 172.21.15.16:0/2183885979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6710075ea0 0x7f671019c900 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f66fc00e9c0 tx=0x7f66fc00ee80 comp rx=0 tx=0).stop 2024-08-31T06:07:10.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.544+0000 7f6718200640 1 -- 172.21.15.16:0/2183885979 shutdown_connections 2024-08-31T06:07:10.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.544+0000 7f6718200640 1 --2- 172.21.15.16:0/2183885979 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f66f80b4d20 0x7f66f80b7230 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:10.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.544+0000 7f6718200640 1 --2- 172.21.15.16:0/2183885979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f671010fe60 0x7f67101a1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:10.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.544+0000 7f6718200640 1 --2- 172.21.15.16:0/2183885979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6710075ea0 0x7f671019c900 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:10.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.544+0000 7f6718200640 1 --2- 172.21.15.16:0/2183885979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6710075550 0x7f671019c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:10.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.544+0000 7f6718200640 1 -- 172.21.15.16:0/2183885979 >> 172.21.15.16:0/2183885979 conn(0x7f671007b5f0 msgr2=0x7f67101078c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:10.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.544+0000 7f6718200640 1 -- 172.21.15.16:0/2183885979 shutdown_connections 2024-08-31T06:07:10.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:10.544+0000 7f6718200640 1 -- 172.21.15.16:0/2183885979 wait complete. 2024-08-31T06:07:11.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:10 smithi016 ceph-mon[17194]: pgmap v361: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-08-31T06:07:11.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:10 smithi184 ceph-mon[22331]: pgmap v361: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-08-31T06:07:11.306 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.g status 2024-08-31T06:07:11.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:10 smithi155 ceph-mon[21742]: pgmap v361: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-08-31T06:07:11.696 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:07:12.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.417+0000 7fe997200640 1 -- 172.21.15.16:0/895886455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe998109090 msgr2=0x7fe9981094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:12.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.417+0000 7fe997200640 1 --2- 172.21.15.16:0/895886455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe998109090 0x7fe9981094a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe984009900 tx=0x7fe98402a410 comp rx=0 tx=0).stop 2024-08-31T06:07:12.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.417+0000 7fe997200640 1 -- 172.21.15.16:0/895886455 shutdown_connections 2024-08-31T06:07:12.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.417+0000 7fe997200640 1 --2- 172.21.15.16:0/895886455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9981099e0 0x7fe998110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:12.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.417+0000 7fe997200640 1 --2- 172.21.15.16:0/895886455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe998109090 0x7fe9981094a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:12.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.417+0000 7fe997200640 1 --2- 172.21.15.16:0/895886455 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe998103750 0x7fe998103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:12.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.417+0000 7fe997200640 1 -- 172.21.15.16:0/895886455 >> 172.21.15.16:0/895886455 conn(0x7fe9980ff1e0 msgr2=0x7fe998101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:12.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.417+0000 7fe997200640 1 -- 172.21.15.16:0/895886455 shutdown_connections 2024-08-31T06:07:12.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.417+0000 7fe997200640 1 -- 172.21.15.16:0/895886455 wait complete. 2024-08-31T06:07:12.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.418+0000 7fe997200640 1 Processor -- start 2024-08-31T06:07:12.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.419+0000 7fe997200640 1 -- start start 2024-08-31T06:07:12.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.419+0000 7fe997200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe998103750 0x7fe99819a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:12.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.419+0000 7fe997200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe998109090 0x7fe99819a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:12.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.419+0000 7fe997200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9981099e0 0x7fe99819f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:12.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.419+0000 7fe997200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe998113670 con 0x7fe9981099e0 2024-08-31T06:07:12.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.419+0000 7fe997200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe9981134f0 con 0x7fe998103750 2024-08-31T06:07:12.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.419+0000 7fe997200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe9981137f0 con 0x7fe998109090 2024-08-31T06:07:12.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.419+0000 7fe995e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe998103750 0x7fe99819a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:12.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.419+0000 7fe996800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9981099e0 0x7fe99819f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:12.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.419+0000 7fe995e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe998103750 0x7fe99819a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51858/0 (socket says 172.21.15.16:51858) 2024-08-31T06:07:12.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.419+0000 7fe995e00640 1 -- 172.21.15.16:0/1985107932 learned_addr learned my addr 172.21.15.16:0/1985107932 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:12.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.419+0000 7fe995400640 1 --2- 172.21.15.16:0/1985107932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe998109090 0x7fe99819a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:12.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.420+0000 7fe995e00640 1 -- 172.21.15.16:0/1985107932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe998109090 msgr2=0x7fe99819a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:12.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.420+0000 7fe995e00640 1 --2- 172.21.15.16:0/1985107932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe998109090 0x7fe99819a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:12.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.420+0000 7fe995e00640 1 -- 172.21.15.16:0/1985107932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9981099e0 msgr2=0x7fe99819f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:12.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.420+0000 7fe995e00640 1 --2- 172.21.15.16:0/1985107932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9981099e0 0x7fe99819f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:12.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.420+0000 7fe995e00640 1 -- 172.21.15.16:0/1985107932 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9981a0030 con 0x7fe998103750 2024-08-31T06:07:12.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.420+0000 7fe995e00640 1 --2- 172.21.15.16:0/1985107932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe998103750 0x7fe99819a0c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe98800e9f0 tx=0x7fe98800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:12.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.420+0000 7fe98ea00640 1 -- 172.21.15.16:0/1985107932 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98800cd80 con 0x7fe998103750 2024-08-31T06:07:12.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.420+0000 7fe98ea00640 1 -- 172.21.15.16:0/1985107932 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9880108e0 con 0x7fe998103750 2024-08-31T06:07:12.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.420+0000 7fe997200640 1 -- 172.21.15.16:0/1985107932 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9981a0320 con 0x7fe998103750 2024-08-31T06:07:12.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.420+0000 7fe98ea00640 1 -- 172.21.15.16:0/1985107932 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe988018980 con 0x7fe998103750 2024-08-31T06:07:12.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.421+0000 7fe997200640 1 -- 172.21.15.16:0/1985107932 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9981a6a80 con 0x7fe998103750 2024-08-31T06:07:12.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.424+0000 7fe9a3a63740 1 Processor -- start 2024-08-31T06:07:12.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.424+0000 7fe9a3a63740 1 -- start start 2024-08-31T06:07:12.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.424+0000 7fe98ea00640 1 -- 172.21.15.16:0/1985107932 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fe988010430 con 0x7fe998103750 2024-08-31T06:07:12.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.424+0000 7fe9a3a63740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9b93d8f60 0x55e9b977eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:12.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.424+0000 7fe9a3a63740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9b93b00a0 0x55e9b977efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:12.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.424+0000 7fe9a3a63740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9b93b8280 0x55e9b977f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:12.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.424+0000 7fe9a3a63740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e9b96f5550 con 0x55e9b93b8280 2024-08-31T06:07:12.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.424+0000 7fe9a3a63740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e9b96e4ae0 con 0x55e9b93d8f60 2024-08-31T06:07:12.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.424+0000 7fe9a3a63740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e9b96e8a90 con 0x55e9b93b00a0 2024-08-31T06:07:12.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.425+0000 7fe996800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9b93d8f60 0x55e9b977eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:12.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.425+0000 7fe996800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9b93d8f60 0x55e9b977eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51870/0 (socket says 172.21.15.16:51870) 2024-08-31T06:07:12.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.425+0000 7fe996800640 1 -- 172.21.15.16:0/2397873960 learned_addr learned my addr 172.21.15.16:0/2397873960 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:12.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.425+0000 7fe995e00640 1 --2- 172.21.15.16:0/2397873960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9b93b00a0 0x55e9b977efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:12.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.425+0000 7fe995400640 1 --2- 172.21.15.16:0/2397873960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9b93b8280 0x55e9b977f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:12.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.425+0000 7fe996800640 1 -- 172.21.15.16:0/2397873960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9b93b00a0 msgr2=0x55e9b977efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:12.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.425+0000 7fe996800640 1 --2- 172.21.15.16:0/2397873960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9b93b00a0 0x55e9b977efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:12.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.425+0000 7fe996800640 1 -- 172.21.15.16:0/2397873960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9b93b8280 msgr2=0x55e9b977f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:12.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.425+0000 7fe996800640 1 --2- 172.21.15.16:0/2397873960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9b93b8280 0x55e9b977f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:12.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.425+0000 7fe996800640 1 -- 172.21.15.16:0/2397873960 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9b9444540 con 0x55e9b93d8f60 2024-08-31T06:07:12.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.425+0000 7fe995e00640 1 --2- 172.21.15.16:0/2397873960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9b93b00a0 0x55e9b977efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:07:12.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.425+0000 7fe996800640 1 --2- 172.21.15.16:0/2397873960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9b93d8f60 0x55e9b977eaa0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe990009fd0 tx=0x7fe9900118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e374d7868a15e91 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:12.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.426+0000 7fe983400640 1 -- 172.21.15.16:0/2397873960 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe990004d20 con 0x55e9b93d8f60 2024-08-31T06:07:12.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.426+0000 7fe983400640 1 -- 172.21.15.16:0/2397873960 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe990004050 con 0x55e9b93d8f60 2024-08-31T06:07:12.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.426+0000 7fe983400640 1 -- 172.21.15.16:0/2397873960 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe990005280 con 0x55e9b93d8f60 2024-08-31T06:07:12.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.426+0000 7fe98ea00640 1 --2- 172.21.15.16:0/1985107932 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe97c0b4ca0 0x7fe97c0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:12.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.427+0000 7fe9a3a63740 1 -- 172.21.15.16:0/2397873960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9b93d8f60 msgr2=0x55e9b977eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:12.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.427+0000 7fe9a3a63740 1 --2- 172.21.15.16:0/2397873960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9b93d8f60 0x55e9b977eaa0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe990009fd0 tx=0x7fe9900118d0 comp rx=0 tx=0).stop 2024-08-31T06:07:12.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.427+0000 7fe995e00640 1 --2- 172.21.15.16:0/1985107932 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe97c0b4ca0 0x7fe97c0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:12.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.427+0000 7fe98ea00640 1 -- 172.21.15.16:0/1985107932 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7fe9880e5850 con 0x7fe998103750 2024-08-31T06:07:12.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.427+0000 7fe9a3a63740 1 -- 172.21.15.16:0/2397873960 shutdown_connections 2024-08-31T06:07:12.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.427+0000 7fe9a3a63740 1 --2- 172.21.15.16:0/2397873960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9b93b00a0 0x55e9b977efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:12.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.427+0000 7fe9a3a63740 1 --2- 172.21.15.16:0/2397873960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9b93b8280 0x55e9b977f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:12.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.427+0000 7fe9a3a63740 1 --2- 172.21.15.16:0/2397873960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9b93d8f60 0x55e9b977eaa0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:12.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.427+0000 7fe9a3a63740 1 -- 172.21.15.16:0/2397873960 >> 172.21.15.16:0/2397873960 conn(0x55e9b93cd350 msgr2=0x55e9b977c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:12.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.427+0000 7fe9a3a63740 1 -- 172.21.15.16:0/2397873960 shutdown_connections 2024-08-31T06:07:12.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.428+0000 7fe995e00640 1 --2- 172.21.15.16:0/1985107932 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe97c0b4ca0 0x7fe97c0b71b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x55e9b977d720 tx=0x7fe988000bc0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:12.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.428+0000 7fe9a3a63740 1 -- 172.21.15.16:0/2397873960 wait complete. 2024-08-31T06:07:12.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.428+0000 7fe9a3a63740 1 -- start start 2024-08-31T06:07:12.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.428+0000 7fe9a3a63740 1 Processor -- start 2024-08-31T06:07:12.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.429+0000 7fe9a3a63740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9b93b00a0 0x55e9b978a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:12.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.429+0000 7fe9a3a63740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9b93b8280 0x55e9b978a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:12.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.429+0000 7fe9a3a63740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9b93d8f60 0x55e9b979a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:12.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.429+0000 7fe9a3a63740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e9b95475f0 con 0x55e9b93d8f60 2024-08-31T06:07:12.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.429+0000 7fe9a3a63740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e9b966bbd0 con 0x55e9b93b8280 2024-08-31T06:07:12.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.429+0000 7fe9a3a63740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e9b978d280 con 0x55e9b93b00a0 2024-08-31T06:07:12.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.429+0000 7fe9a3a63740 10 client.0 _ll_register_callbacks cb 0x55e9b9691210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:07:12.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.429+0000 7fe995400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9b93b8280 0x55e9b978a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:12.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.429+0000 7fe996800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9b93b00a0 0x55e9b978a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:12.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.429+0000 7fe996800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9b93b00a0 0x55e9b978a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:46424/0 (socket says 172.21.15.16:46424) 2024-08-31T06:07:12.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.429+0000 7fe995400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9b93b8280 0x55e9b978a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51880/0 (socket says 172.21.15.16:51880) 2024-08-31T06:07:12.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.429+0000 7fe996800640 1 -- 172.21.15.16:0/431020188 learned_addr learned my addr 172.21.15.16:0/431020188 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:12.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.430+0000 7fe996800640 1 --2- 172.21.15.16:0/431020188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9b93d8f60 0x55e9b979a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:12.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.430+0000 7fe995400640 1 -- 172.21.15.16:0/431020188 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9b93b00a0 msgr2=0x55e9b978a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:12.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.430+0000 7fe995400640 1 --2- 172.21.15.16:0/431020188 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9b93b00a0 0x55e9b978a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:12.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.430+0000 7fe995400640 1 -- 172.21.15.16:0/431020188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9b93d8f60 msgr2=0x55e9b979a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:12.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.430+0000 7fe995400640 1 --2- 172.21.15.16:0/431020188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9b93d8f60 0x55e9b979a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:12.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.430+0000 7fe995400640 1 -- 172.21.15.16:0/431020188 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9b96ac650 con 0x55e9b93b8280 2024-08-31T06:07:12.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.430+0000 7fe996800640 1 --2- 172.21.15.16:0/431020188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9b93d8f60 0x55e9b979a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:07:12.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.430+0000 7fe995400640 1 --2- 172.21.15.16:0/431020188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9b93b8280 0x55e9b978a7a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe98402a930 tx=0x7fe984032dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1251a31e921216c2 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:12.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.430+0000 7fe977e00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:07:12.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.430+0000 7fe977e00640 1 -- 172.21.15.16:0/431020188 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe984035040 con 0x55e9b93b8280 2024-08-31T06:07:12.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.431+0000 7fe9a3a63740 1 -- 172.21.15.16:0/431020188 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe990004050 con 0x55e9b93b8280 2024-08-31T06:07:12.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.431+0000 7fe977e00640 1 -- 172.21.15.16:0/431020188 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7fe9840316a0 con 0x55e9b93b8280 2024-08-31T06:07:12.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.431+0000 7fe977e00640 1 -- 172.21.15.16:0/431020188 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe9840318a0 con 0x55e9b93b8280 2024-08-31T06:07:12.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.431+0000 7fe9a3a63740 10 client.25466 fetch_fsmap learned FSMap version 25 2024-08-31T06:07:12.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.431+0000 7fe9a3a63740 1 -- 172.21.15.16:0/431020188 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e9b979d540 con 0x55e9b93b8280 2024-08-31T06:07:12.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.431+0000 7fe977e00640 1 -- 172.21.15.16:0/431020188 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe990004050 con 0x55e9b93b8280 2024-08-31T06:07:12.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.431+0000 7fe977e00640 1 -- 172.21.15.16:0/431020188 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe984004750 con 0x55e9b93b8280 2024-08-31T06:07:12.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.431+0000 7fe9a3a63740 10 client.25466 fetch_fsmap finished waiting for FSMap version 25 2024-08-31T06:07:12.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.432+0000 7fe9a3a63740 10 client.25466 resolve_mds: resolved name 'g' to daemon mds.g 2024-08-31T06:07:12.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.432+0000 7fe9a3a63740 20 client.25466 populate_metadata read hostname 'smithi016' 2024-08-31T06:07:12.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.432+0000 7fe9a3a63740 1 --2- 172.21.15.16:0/431020188 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55e9b93978e0 0x55e9b97a0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:12.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.432+0000 7fe9a3a63740 4 client.25466 mds_command: new command op to 24724 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:07:12.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.432+0000 7fe9a3a63740 1 -- 172.21.15.16:0/431020188 --> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9b96f74a0 con 0x55e9b93978e0 2024-08-31T06:07:12.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.432+0000 7fe996800640 1 --2- 172.21.15.16:0/431020188 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55e9b93978e0 0x55e9b97a0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:12.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.432+0000 7fe996800640 1 --2- 172.21.15.16:0/431020188 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55e9b93978e0 0x55e9b97a0f10 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=97bb7278c7c734c server_cookie=51a521d01a0a9078 in_seq=0 out_seq=0 2024-08-31T06:07:12.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.432+0000 7fe977e00640 10 client.25466 ms_handle_connect on v2:172.21.15.16:6838/219664460 2024-08-31T06:07:12.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.433+0000 7fe977e00640 1 -- 172.21.15.16:0/431020188 <== mds.1 v2:172.21.15.16:6838/219664460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e9b96f74a0 con 0x55e9b93978e0 2024-08-31T06:07:12.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.433+0000 7fe977e00640 10 client.25466 handle_command_reply: tid=0 2024-08-31T06:07:12.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.433+0000 7fe9a3a63740 1 client.25466 shutdown 2024-08-31T06:07:12.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.434+0000 7fe9a3a63740 1 -- 172.21.15.16:0/431020188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9b93b8280 msgr2=0x55e9b978a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:12.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.434+0000 7fe9a3a63740 1 --2- 172.21.15.16:0/431020188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9b93b8280 0x55e9b978a7a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe98402a930 tx=0x7fe984032dc0 comp rx=0 tx=0).stop 2024-08-31T06:07:12.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.434+0000 7fe9a3a63740 1 -- 172.21.15.16:0/431020188 shutdown_connections 2024-08-31T06:07:12.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.434+0000 7fe9a3a63740 1 --2- 172.21.15.16:0/431020188 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55e9b93978e0 0x55e9b97a0f10 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:12.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.434+0000 7fe9a3a63740 1 --2- 172.21.15.16:0/431020188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9b93d8f60 0x55e9b979a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:12.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.434+0000 7fe9a3a63740 1 --2- 172.21.15.16:0/431020188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9b93b8280 0x55e9b978a7a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:12.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.434+0000 7fe9a3a63740 1 --2- 172.21.15.16:0/431020188 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9b93b00a0 0x55e9b978a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:12.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.434+0000 7fe9a3a63740 1 -- 172.21.15.16:0/431020188 >> 172.21.15.16:0/431020188 conn(0x55e9b93cd350 msgr2=0x55e9b9781c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:12.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.434+0000 7fe977e00640 0 client.25466 ms_handle_reset on v2:172.21.15.16:6838/219664460 2024-08-31T06:07:12.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.434+0000 7fe9a3a63740 1 -- 172.21.15.16:0/431020188 shutdown_connections 2024-08-31T06:07:12.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.434+0000 7fe9a3a63740 1 -- 172.21.15.16:0/431020188 wait complete. 2024-08-31T06:07:12.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.434+0000 7fe9a3a63740 20 client.25466 trim_cache size 0 max 16384 2024-08-31T06:07:12.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.453+0000 7fe9a3a63740 1 Processor -- start 2024-08-31T06:07:12.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.453+0000 7fe9a3a63740 1 -- start start 2024-08-31T06:07:12.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.454+0000 7fe9a3a63740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9b93b00a0 0x55e9b979abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:12.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.454+0000 7fe9a3a63740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9b93b8280 0x55e9b9788ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:12.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.454+0000 7fe9a3a63740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9b93d8f60 0x55e9b9789020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:12.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.454+0000 7fe9a3a63740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e9b979f250 con 0x55e9b93b8280 2024-08-31T06:07:12.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.454+0000 7fe9a3a63740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e9b979f920 con 0x55e9b93b00a0 2024-08-31T06:07:12.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.454+0000 7fe9a3a63740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e9b979fff0 con 0x55e9b93d8f60 2024-08-31T06:07:12.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.454+0000 7fe995400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9b93b8280 0x55e9b9788ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:12.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.454+0000 7fe995400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9b93b8280 0x55e9b9788ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46486/0 (socket says 172.21.15.16:46486) 2024-08-31T06:07:12.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.454+0000 7fe996800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9b93b00a0 0x55e9b979abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:12.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.454+0000 7fe995400640 1 -- 172.21.15.16:0/485021850 learned_addr learned my addr 172.21.15.16:0/485021850 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:12.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.454+0000 7fe996800640 1 --2- 172.21.15.16:0/485021850 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9b93d8f60 0x55e9b9789020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:12.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.455+0000 7fe996800640 1 -- 172.21.15.16:0/485021850 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9b93d8f60 msgr2=0x55e9b9789020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:12.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.455+0000 7fe996800640 1 --2- 172.21.15.16:0/485021850 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9b93d8f60 0x55e9b9789020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:12.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.455+0000 7fe996800640 1 -- 172.21.15.16:0/485021850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9b93b8280 msgr2=0x55e9b9788ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:12.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.455+0000 7fe996800640 1 --2- 172.21.15.16:0/485021850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9b93b8280 0x55e9b9788ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:12.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.455+0000 7fe996800640 1 -- 172.21.15.16:0/485021850 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9b9788760 con 0x55e9b93b00a0 2024-08-31T06:07:12.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.455+0000 7fe995400640 1 --2- 172.21.15.16:0/485021850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9b93b8280 0x55e9b9788ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:07:12.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.455+0000 7fe996800640 1 --2- 172.21.15.16:0/485021850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9b93b00a0 0x55e9b979abb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe990009fa0 tx=0x7fe990009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f6838cd4933f28c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:12.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.455+0000 7fe982a00640 1 -- 172.21.15.16:0/485021850 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe990004050 con 0x55e9b93b00a0 2024-08-31T06:07:12.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.455+0000 7fe982a00640 1 -- 172.21.15.16:0/485021850 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe99002b030 con 0x55e9b93b00a0 2024-08-31T06:07:12.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.455+0000 7fe982a00640 1 -- 172.21.15.16:0/485021850 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99000fae0 con 0x55e9b93b00a0 2024-08-31T06:07:12.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.456+0000 7fe9a3a63740 1 -- 172.21.15.16:0/485021850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9b93b00a0 msgr2=0x55e9b979abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:12.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.456+0000 7fe9a3a63740 1 --2- 172.21.15.16:0/485021850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9b93b00a0 0x55e9b979abb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe990009fa0 tx=0x7fe990009500 comp rx=0 tx=0).stop 2024-08-31T06:07:12.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.456+0000 7fe9a3a63740 1 -- 172.21.15.16:0/485021850 shutdown_connections 2024-08-31T06:07:12.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.456+0000 7fe9a3a63740 1 --2- 172.21.15.16:0/485021850 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9b93d8f60 0x55e9b9789020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:12.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.456+0000 7fe9a3a63740 1 --2- 172.21.15.16:0/485021850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9b93b8280 0x55e9b9788ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:12.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.456+0000 7fe9a3a63740 1 --2- 172.21.15.16:0/485021850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9b93b00a0 0x55e9b979abb0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:12.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.456+0000 7fe9a3a63740 1 -- 172.21.15.16:0/485021850 >> 172.21.15.16:0/485021850 conn(0x55e9b93cd350 msgr2=0x55e9b979e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:12.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.457+0000 7fe9a3a63740 1 -- 172.21.15.16:0/485021850 shutdown_connections 2024-08-31T06:07:12.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.457+0000 7fe9a3a63740 1 -- 172.21.15.16:0/485021850 wait complete. 2024-08-31T06:07:12.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.457+0000 7fe9a3a63740 1 -- start start 2024-08-31T06:07:12.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.457+0000 7fe9a3a63740 1 Processor -- start 2024-08-31T06:07:12.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.458+0000 7fe9a3a63740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9b93b00a0 0x55e9b97a9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:12.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.458+0000 7fe9a3a63740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9b93b8280 0x55e9b97a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:12.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.458+0000 7fe9a3a63740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9b93d8f60 0x55e9b97a5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:12.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.458+0000 7fe9a3a63740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e9b97a0d90 con 0x55e9b93d8f60 2024-08-31T06:07:12.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.458+0000 7fe9a3a63740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e9b97a06c0 con 0x55e9b93b00a0 2024-08-31T06:07:12.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.458+0000 7fe9a3a63740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e9b97a1460 con 0x55e9b93b8280 2024-08-31T06:07:12.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.458+0000 7fe9a3a63740 10 client.0 _ll_register_callbacks cb 0x7fe984005ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:07:12.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.458+0000 7fe996800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9b93b00a0 0x55e9b97a9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:12.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.458+0000 7fe995400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9b93b8280 0x55e9b97a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:12.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.458+0000 7fe996800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9b93b00a0 0x55e9b97a9b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51898/0 (socket says 172.21.15.16:51898) 2024-08-31T06:07:12.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.458+0000 7fe996800640 1 -- 172.21.15.16:0/1677973196 learned_addr learned my addr 172.21.15.16:0/1677973196 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:12.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.458+0000 7fe996800640 1 --2- 172.21.15.16:0/1677973196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9b93d8f60 0x55e9b97a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:12.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.459+0000 7fe996800640 1 -- 172.21.15.16:0/1677973196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9b93b8280 msgr2=0x55e9b97a4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:12.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.459+0000 7fe996800640 1 --2- 172.21.15.16:0/1677973196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9b93b8280 0x55e9b97a4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:12.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.459+0000 7fe996800640 1 -- 172.21.15.16:0/1677973196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9b93d8f60 msgr2=0x55e9b97a5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:12.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.459+0000 7fe996800640 1 --2- 172.21.15.16:0/1677973196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9b93d8f60 0x55e9b97a5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:12.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.459+0000 7fe996800640 1 -- 172.21.15.16:0/1677973196 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9b97a5e00 con 0x55e9b93b00a0 2024-08-31T06:07:12.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.459+0000 7fe996800640 1 --2- 172.21.15.16:0/1677973196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9b93b00a0 0x55e9b97a9b20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe990005c10 tx=0x7fe990009fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61930ab572fd63a6 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:12.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.459+0000 7fe977e00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:07:12.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.460+0000 7fe977e00640 1 -- 172.21.15.16:0/1677973196 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe990041060 con 0x55e9b93b00a0 2024-08-31T06:07:12.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.460+0000 7fe9a3a63740 1 -- 172.21.15.16:0/1677973196 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe99002b030 con 0x55e9b93b00a0 2024-08-31T06:07:12.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.460+0000 7fe977e00640 1 -- 172.21.15.16:0/1677973196 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7fe99000fa60 con 0x55e9b93b00a0 2024-08-31T06:07:12.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.460+0000 7fe977e00640 1 -- 172.21.15.16:0/1677973196 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe99000fc60 con 0x55e9b93b00a0 2024-08-31T06:07:12.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.460+0000 7fe9a3a63740 10 client.25472 fetch_fsmap learned FSMap version 25 2024-08-31T06:07:12.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.460+0000 7fe9a3a63740 1 -- 172.21.15.16:0/1677973196 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e9b980c490 con 0x55e9b93b00a0 2024-08-31T06:07:12.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.460+0000 7fe977e00640 1 -- 172.21.15.16:0/1677973196 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe99002b030 con 0x55e9b93b00a0 2024-08-31T06:07:12.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.461+0000 7fe977e00640 1 -- 172.21.15.16:0/1677973196 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe9900275e0 con 0x55e9b93b00a0 2024-08-31T06:07:12.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.461+0000 7fe9a3a63740 10 client.25472 fetch_fsmap finished waiting for FSMap version 25 2024-08-31T06:07:12.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.461+0000 7fe9a3a63740 10 client.25472 resolve_mds: resolved name 'g' to daemon mds.g 2024-08-31T06:07:12.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.461+0000 7fe9a3a63740 20 client.25472 populate_metadata read hostname 'smithi016' 2024-08-31T06:07:12.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.461+0000 7fe9a3a63740 1 --2- 172.21.15.16:0/1677973196 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55e9b936cc90 0x55e9b980f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:12.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.461+0000 7fe9a3a63740 4 client.25472 mds_command: new command op to 24724 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-31T06:07:12.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.461+0000 7fe9a3a63740 1 -- 172.21.15.16:0/1677973196 --> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] -- command(tid 0: {"prefix": "status"}) -- 0x55e9b96f2f40 con 0x55e9b936cc90 2024-08-31T06:07:12.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.461+0000 7fe995400640 1 --2- 172.21.15.16:0/1677973196 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55e9b936cc90 0x55e9b980f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:12.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.462+0000 7fe995400640 1 --2- 172.21.15.16:0/1677973196 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55e9b936cc90 0x55e9b980f600 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d52f096e316fb09b server_cookie=2db9badec2509204 in_seq=0 out_seq=0 2024-08-31T06:07:12.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.462+0000 7fe977e00640 10 client.25472 ms_handle_connect on v2:172.21.15.16:6838/219664460 2024-08-31T06:07:12.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.462+0000 7fe977e00640 1 -- 172.21.15.16:0/1677973196 <== mds.1 v2:172.21.15.16:6838/219664460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55e9b96f2f40 con 0x55e9b936cc90 2024-08-31T06:07:12.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.462+0000 7fe977e00640 10 client.25472 handle_command_reply: tid=0 2024-08-31T06:07:12.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.462+0000 7fe9a3a63740 1 client.25472 shutdown 2024-08-31T06:07:12.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.462+0000 7fe9a3a63740 1 -- 172.21.15.16:0/1677973196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9b93b00a0 msgr2=0x55e9b97a9b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:12.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.462+0000 7fe9a3a63740 1 --2- 172.21.15.16:0/1677973196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9b93b00a0 0x55e9b97a9b20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe990005c10 tx=0x7fe990009fd0 comp rx=0 tx=0).stop 2024-08-31T06:07:12.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.463+0000 7fe9a3a63740 1 -- 172.21.15.16:0/1677973196 shutdown_connections 2024-08-31T06:07:12.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.463+0000 7fe9a3a63740 1 --2- 172.21.15.16:0/1677973196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9b93b8280 0x55e9b97a4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:12.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.463+0000 7fe9a3a63740 1 --2- 172.21.15.16:0/1677973196 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55e9b936cc90 0x55e9b980f600 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:12.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.463+0000 7fe9a3a63740 1 --2- 172.21.15.16:0/1677973196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9b93d8f60 0x55e9b97a5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:12.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.463+0000 7fe9a3a63740 1 --2- 172.21.15.16:0/1677973196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9b93b00a0 0x55e9b97a9b20 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:12.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.463+0000 7fe977e00640 0 client.25472 ms_handle_reset on v2:172.21.15.16:6838/219664460 2024-08-31T06:07:12.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.463+0000 7fe9a3a63740 1 -- 172.21.15.16:0/1677973196 >> 172.21.15.16:0/1677973196 conn(0x55e9b93cd350 msgr2=0x55e9b979e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:12.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.463+0000 7fe9a3a63740 1 -- 172.21.15.16:0/1677973196 shutdown_connections 2024-08-31T06:07:12.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.463+0000 7fe9a3a63740 1 -- 172.21.15.16:0/1677973196 wait complete. 2024-08-31T06:07:12.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.463+0000 7fe9a3a63740 20 client.25472 trim_cache size 0 max 16384 2024-08-31T06:07:12.467 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:07:12.467 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "673e4aa0-675d-11ef-bcd4-c7b262605968", 2024-08-31T06:07:12.467 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 1, 2024-08-31T06:07:12.467 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24724, 2024-08-31T06:07:12.467 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-08-31T06:07:12.467 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-08-31T06:07:12.468 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-08-31T06:07:12.468 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 154.53903378199999, 2024-08-31T06:07:12.468 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 25, 2024-08-31T06:07:12.468 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 80, 2024-08-31T06:07:12.468 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 80, 2024-08-31T06:07:12.468 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 185.46848439999999 2024-08-31T06:07:12.468 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:07:12.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.464+0000 7fe997200640 1 -- 172.21.15.16:0/1985107932 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe97c0b4ca0 msgr2=0x7fe97c0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:12.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.464+0000 7fe997200640 1 --2- 172.21.15.16:0/1985107932 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe97c0b4ca0 0x7fe97c0b71b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x55e9b977d720 tx=0x7fe988000bc0 comp rx=0 tx=0).stop 2024-08-31T06:07:12.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.464+0000 7fe997200640 1 -- 172.21.15.16:0/1985107932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe998103750 msgr2=0x7fe99819a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:12.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.464+0000 7fe997200640 1 --2- 172.21.15.16:0/1985107932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe998103750 0x7fe99819a0c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe98800e9f0 tx=0x7fe98800eeb0 comp rx=0 tx=0).stop 2024-08-31T06:07:12.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.464+0000 7fe997200640 1 -- 172.21.15.16:0/1985107932 shutdown_connections 2024-08-31T06:07:12.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.464+0000 7fe997200640 1 --2- 172.21.15.16:0/1985107932 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe97c0b4ca0 0x7fe97c0b71b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:12.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.464+0000 7fe997200640 1 --2- 172.21.15.16:0/1985107932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe998109090 0x7fe99819a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:12.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.464+0000 7fe997200640 1 --2- 172.21.15.16:0/1985107932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9981099e0 0x7fe99819f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:12.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.464+0000 7fe997200640 1 --2- 172.21.15.16:0/1985107932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe998103750 0x7fe99819a0c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:12.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.464+0000 7fe997200640 1 -- 172.21.15.16:0/1985107932 >> 172.21.15.16:0/1985107932 conn(0x7fe9980ff1e0 msgr2=0x7fe998101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:12.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.464+0000 7fe997200640 1 -- 172.21.15.16:0/1985107932 shutdown_connections 2024-08-31T06:07:12.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:12.464+0000 7fe997200640 1 -- 172.21.15.16:0/1985107932 wait complete. 2024-08-31T06:07:13.065 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:12 smithi016 ceph-mon[17194]: pgmap v362: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 KiB/s wr, 27 op/s 2024-08-31T06:07:13.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:12 smithi184 ceph-mon[22331]: pgmap v362: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 KiB/s wr, 27 op/s 2024-08-31T06:07:13.247 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.b status 2024-08-31T06:07:13.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:12 smithi155 ceph-mon[21742]: pgmap v362: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 KiB/s wr, 27 op/s 2024-08-31T06:07:13.632 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:07:14.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.318+0000 7f3b76400640 1 -- 172.21.15.16:0/1130699573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b7010fec0 msgr2=0x7f3b701122e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:14.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.318+0000 7f3b76400640 1 --2- 172.21.15.16:0/1130699573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b7010fec0 0x7f3b701122e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3b68009ca0 tx=0x7f3b6802a8b0 comp rx=0 tx=0).stop 2024-08-31T06:07:14.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.319+0000 7f3b76400640 1 -- 172.21.15.16:0/1130699573 shutdown_connections 2024-08-31T06:07:14.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.319+0000 7f3b76400640 1 --2- 172.21.15.16:0/1130699573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b7010fec0 0x7f3b701122e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:14.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.319+0000 7f3b76400640 1 --2- 172.21.15.16:0/1130699573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b70075e40 0x7f3b7010f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:14.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.319+0000 7f3b76400640 1 --2- 172.21.15.16:0/1130699573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b700754f0 0x7f3b70075900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:14.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.319+0000 7f3b76400640 1 -- 172.21.15.16:0/1130699573 >> 172.21.15.16:0/1130699573 conn(0x7f3b7007b5f0 msgr2=0x7f3b7007ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:14.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.319+0000 7f3b76400640 1 -- 172.21.15.16:0/1130699573 shutdown_connections 2024-08-31T06:07:14.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.319+0000 7f3b76400640 1 -- 172.21.15.16:0/1130699573 wait complete. 2024-08-31T06:07:14.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.320+0000 7f3b76400640 1 Processor -- start 2024-08-31T06:07:14.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.320+0000 7f3b76400640 1 -- start start 2024-08-31T06:07:14.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.320+0000 7f3b76400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b700754f0 0x7f3b7019c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:14.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.320+0000 7f3b76400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b70075e40 0x7f3b7019ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:14.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.320+0000 7f3b76400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b7010fec0 0x7f3b701a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:14.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.320+0000 7f3b76400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3b70115a80 con 0x7f3b700754f0 2024-08-31T06:07:14.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.320+0000 7f3b76400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3b70115900 con 0x7f3b7010fec0 2024-08-31T06:07:14.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.320+0000 7f3b76400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3b70115c00 con 0x7f3b70075e40 2024-08-31T06:07:14.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.321+0000 7f3b67e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b70075e40 0x7f3b7019ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:14.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.321+0000 7f3b6f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b700754f0 0x7f3b7019c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:14.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.321+0000 7f3b67e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b70075e40 0x7f3b7019ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:46470/0 (socket says 172.21.15.16:46470) 2024-08-31T06:07:14.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.321+0000 7f3b67e00640 1 -- 172.21.15.16:0/1350379731 learned_addr learned my addr 172.21.15.16:0/1350379731 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:14.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.321+0000 7f3b6fe00640 1 --2- 172.21.15.16:0/1350379731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b7010fec0 0x7f3b701a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:14.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.321+0000 7f3b67e00640 1 -- 172.21.15.16:0/1350379731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b7010fec0 msgr2=0x7f3b701a1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:14.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.321+0000 7f3b67e00640 1 --2- 172.21.15.16:0/1350379731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b7010fec0 0x7f3b701a1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:14.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.321+0000 7f3b67e00640 1 -- 172.21.15.16:0/1350379731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b700754f0 msgr2=0x7f3b7019c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:14.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.321+0000 7f3b67e00640 1 --2- 172.21.15.16:0/1350379731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b700754f0 0x7f3b7019c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:14.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.321+0000 7f3b67e00640 1 -- 172.21.15.16:0/1350379731 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b701a2440 con 0x7f3b70075e40 2024-08-31T06:07:14.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.321+0000 7f3b6fe00640 1 --2- 172.21.15.16:0/1350379731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b7010fec0 0x7f3b701a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:07:14.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.321+0000 7f3b67e00640 1 --2- 172.21.15.16:0/1350379731 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b70075e40 0x7f3b7019ca70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3b5800e970 tx=0x7f3b5800ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:14.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.322+0000 7f3b6cc00640 1 -- 172.21.15.16:0/1350379731 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5800cd80 con 0x7f3b70075e40 2024-08-31T06:07:14.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.322+0000 7f3b76400640 1 -- 172.21.15.16:0/1350379731 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b701a2730 con 0x7f3b70075e40 2024-08-31T06:07:14.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.322+0000 7f3b6cc00640 1 -- 172.21.15.16:0/1350379731 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b580035f0 con 0x7f3b70075e40 2024-08-31T06:07:14.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.322+0000 7f3b6cc00640 1 -- 172.21.15.16:0/1350379731 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b58010690 con 0x7f3b70075e40 2024-08-31T06:07:14.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.322+0000 7f3b76400640 1 -- 172.21.15.16:0/1350379731 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b701a8e90 con 0x7f3b70075e40 2024-08-31T06:07:14.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.325+0000 7f3b7d0bb740 1 Processor -- start 2024-08-31T06:07:14.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.325+0000 7f3b7d0bb740 1 -- start start 2024-08-31T06:07:14.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.325+0000 7f3b6cc00640 1 -- 172.21.15.16:0/1350379731 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f3b58003aa0 con 0x7f3b70075e40 2024-08-31T06:07:14.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.326+0000 7f3b7d0bb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c73ab65f60 0x55c73af0b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:14.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.326+0000 7f3b7d0bb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73ab3d0a0 0x55c73af0be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:14.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.326+0000 7f3b7d0bb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c73ab45280 0x55c73af0c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:14.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.326+0000 7f3b7d0bb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c73ae82550 con 0x55c73ab3d0a0 2024-08-31T06:07:14.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.326+0000 7f3b7d0bb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c73ae71ae0 con 0x55c73ab65f60 2024-08-31T06:07:14.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.326+0000 7f3b7d0bb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c73ae75a90 con 0x55c73ab45280 2024-08-31T06:07:14.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.326+0000 7f3b67e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c73ab45280 0x55c73af0c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:14.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.326+0000 7f3b6fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c73ab65f60 0x55c73af0b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:14.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.326+0000 7f3b6f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73ab3d0a0 0x55c73af0be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:14.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.326+0000 7f3b67e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c73ab45280 0x55c73af0c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:46474/0 (socket says 172.21.15.16:46474) 2024-08-31T06:07:14.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.326+0000 7f3b67e00640 1 -- 172.21.15.16:0/3945176522 learned_addr learned my addr 172.21.15.16:0/3945176522 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:14.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.326+0000 7f3b6fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c73ab65f60 0x55c73af0b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51922/0 (socket says 172.21.15.16:51922) 2024-08-31T06:07:14.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.326+0000 7f3b67e00640 1 -- 172.21.15.16:0/3945176522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c73ab65f60 msgr2=0x55c73af0b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:14.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.326+0000 7f3b67e00640 1 --2- 172.21.15.16:0/3945176522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c73ab65f60 0x55c73af0b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:14.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.326+0000 7f3b67e00640 1 -- 172.21.15.16:0/3945176522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73ab3d0a0 msgr2=0x55c73af0be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.326+0000 7f3b67e00640 1 --2- 172.21.15.16:0/3945176522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73ab3d0a0 0x55c73af0be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.327+0000 7f3b67e00640 1 -- 172.21.15.16:0/3945176522 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c73abd1540 con 0x55c73ab45280 2024-08-31T06:07:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.327+0000 7f3b67e00640 1 --2- 172.21.15.16:0/3945176522 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c73ab45280 0x55c73af0c370 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3b580e2eb0 tx=0x7f3b580ebb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e94dc7823dadc5c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.327+0000 7f3b64c00640 1 -- 172.21.15.16:0/3945176522 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b580f4070 con 0x55c73ab45280 2024-08-31T06:07:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.327+0000 7f3b64c00640 1 -- 172.21.15.16:0/3945176522 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b580edd30 con 0x55c73ab45280 2024-08-31T06:07:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.327+0000 7f3b64c00640 1 -- 172.21.15.16:0/3945176522 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b580e7a50 con 0x55c73ab45280 2024-08-31T06:07:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.328+0000 7f3b6cc00640 1 --2- 172.21.15.16:0/1350379731 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3b540b4ca0 0x7f3b540b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.328+0000 7f3b6cc00640 1 -- 172.21.15.16:0/1350379731 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f3b580e5820 con 0x7f3b70075e40 2024-08-31T06:07:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.328+0000 7f3b6fe00640 1 --2- 172.21.15.16:0/1350379731 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3b540b4ca0 0x7f3b540b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.328+0000 7f3b7d0bb740 1 -- 172.21.15.16:0/3945176522 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c73ab45280 msgr2=0x55c73af0c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.328+0000 7f3b7d0bb740 1 --2- 172.21.15.16:0/3945176522 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c73ab45280 0x55c73af0c370 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3b580e2eb0 tx=0x7f3b580ebb20 comp rx=0 tx=0).stop 2024-08-31T06:07:14.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.328+0000 7f3b7d0bb740 1 -- 172.21.15.16:0/3945176522 shutdown_connections 2024-08-31T06:07:14.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.328+0000 7f3b7d0bb740 1 --2- 172.21.15.16:0/3945176522 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c73ab45280 0x55c73af0c370 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:14.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.328+0000 7f3b7d0bb740 1 --2- 172.21.15.16:0/3945176522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73ab3d0a0 0x55c73af0be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:14.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.328+0000 7f3b7d0bb740 1 --2- 172.21.15.16:0/3945176522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c73ab65f60 0x55c73af0b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:14.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.328+0000 7f3b7d0bb740 1 -- 172.21.15.16:0/3945176522 >> 172.21.15.16:0/3945176522 conn(0x55c73ab5a350 msgr2=0x55c73af097e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:14.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.329+0000 7f3b7d0bb740 1 -- 172.21.15.16:0/3945176522 shutdown_connections 2024-08-31T06:07:14.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.329+0000 7f3b6fe00640 1 --2- 172.21.15.16:0/1350379731 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3b540b4ca0 0x7f3b540b71b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3b68006010 tx=0x7f3b68009480 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:14.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.329+0000 7f3b7d0bb740 1 -- 172.21.15.16:0/3945176522 wait complete. 2024-08-31T06:07:14.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.329+0000 7f3b7d0bb740 1 -- start start 2024-08-31T06:07:14.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.330+0000 7f3b7d0bb740 1 Processor -- start 2024-08-31T06:07:14.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.330+0000 7f3b7d0bb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c73ab3d0a0 0x55c73af17070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:14.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.330+0000 7f3b7d0bb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73ab45280 0x55c73af175b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:14.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.330+0000 7f3b7d0bb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c73ab65f60 0x55c73af233c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:14.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.330+0000 7f3b7d0bb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c73acd45f0 con 0x55c73ab45280 2024-08-31T06:07:14.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.330+0000 7f3b7d0bb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c73adf8bd0 con 0x55c73ab3d0a0 2024-08-31T06:07:14.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.330+0000 7f3b7d0bb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c73af1a0d0 con 0x55c73ab65f60 2024-08-31T06:07:14.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.330+0000 7f3b7d0bb740 10 client.0 _ll_register_callbacks cb 0x55c73ae1e210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:07:14.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.330+0000 7f3b6f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c73ab3d0a0 0x55c73af17070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:14.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.330+0000 7f3b6f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c73ab3d0a0 0x55c73af17070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51936/0 (socket says 172.21.15.16:51936) 2024-08-31T06:07:14.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.330+0000 7f3b6f400640 1 -- 172.21.15.16:0/1477491867 learned_addr learned my addr 172.21.15.16:0/1477491867 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:14.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.330+0000 7f3b6fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c73ab65f60 0x55c73af233c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:14.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.330+0000 7f3b67e00640 1 --2- 172.21.15.16:0/1477491867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73ab45280 0x55c73af175b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:14.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.331+0000 7f3b6f400640 1 -- 172.21.15.16:0/1477491867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c73ab65f60 msgr2=0x55c73af233c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:14.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.331+0000 7f3b6f400640 1 --2- 172.21.15.16:0/1477491867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c73ab65f60 0x55c73af233c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:14.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.331+0000 7f3b6f400640 1 -- 172.21.15.16:0/1477491867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73ab45280 msgr2=0x55c73af175b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:14.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.331+0000 7f3b6f400640 1 --2- 172.21.15.16:0/1477491867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73ab45280 0x55c73af175b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:14.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.331+0000 7f3b6f400640 1 -- 172.21.15.16:0/1477491867 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c73ae39650 con 0x55c73ab3d0a0 2024-08-31T06:07:14.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.331+0000 7f3b6f400640 1 --2- 172.21.15.16:0/1477491867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c73ab3d0a0 0x55c73af17070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3b60012740 tx=0x7f3b60012c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83399f356b227677 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:14.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.331+0000 7f3b5d600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:07:14.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.332+0000 7f3b5d600640 1 -- 172.21.15.16:0/1477491867 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6000f920 con 0x55c73ab3d0a0 2024-08-31T06:07:14.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.332+0000 7f3b5d600640 1 -- 172.21.15.16:0/1477491867 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f3b6000fac0 con 0x55c73ab3d0a0 2024-08-31T06:07:14.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.332+0000 7f3b7d0bb740 1 -- 172.21.15.16:0/1477491867 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b580edd30 con 0x55c73ab3d0a0 2024-08-31T06:07:14.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.332+0000 7f3b5d600640 1 -- 172.21.15.16:0/1477491867 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3b6000fcc0 con 0x55c73ab3d0a0 2024-08-31T06:07:14.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.332+0000 7f3b7d0bb740 10 client.25484 fetch_fsmap learned FSMap version 25 2024-08-31T06:07:14.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.332+0000 7f3b7d0bb740 1 -- 172.21.15.16:0/1477491867 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c73af25ea0 con 0x55c73ab3d0a0 2024-08-31T06:07:14.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.332+0000 7f3b5d600640 1 -- 172.21.15.16:0/1477491867 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b580edd30 con 0x55c73ab3d0a0 2024-08-31T06:07:14.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.332+0000 7f3b5d600640 1 -- 172.21.15.16:0/1477491867 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3b6001c420 con 0x55c73ab3d0a0 2024-08-31T06:07:14.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.333+0000 7f3b7d0bb740 10 client.25484 fetch_fsmap finished waiting for FSMap version 25 2024-08-31T06:07:14.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.333+0000 7f3b7d0bb740 10 client.25484 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-31T06:07:14.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.333+0000 7f3b7d0bb740 20 client.25484 populate_metadata read hostname 'smithi016' 2024-08-31T06:07:14.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.333+0000 7f3b7d0bb740 1 --2- 172.21.15.16:0/1477491867 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55c73ab248e0 0x55c73af299b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:14.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.333+0000 7f3b7d0bb740 4 client.25484 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:07:14.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.333+0000 7f3b7d0bb740 1 -- 172.21.15.16:0/1477491867 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c73ae844a0 con 0x55c73ab248e0 2024-08-31T06:07:14.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.333+0000 7f3b67e00640 1 --2- 172.21.15.16:0/1477491867 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55c73ab248e0 0x55c73af299b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:14.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.334+0000 7f3b67e00640 1 --2- 172.21.15.16:0/1477491867 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55c73ab248e0 0x55c73af299b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11220ceae0745028 server_cookie=8afd4bb3caf7167f in_seq=0 out_seq=0 2024-08-31T06:07:14.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.334+0000 7f3b5d600640 10 client.25484 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:14.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.337+0000 7f3b5d600640 1 -- 172.21.15.16:0/1477491867 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c73ae844a0 con 0x55c73ab248e0 2024-08-31T06:07:14.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.337+0000 7f3b5d600640 10 client.25484 handle_command_reply: tid=0 2024-08-31T06:07:14.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.337+0000 7f3b7d0bb740 1 client.25484 shutdown 2024-08-31T06:07:14.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.337+0000 7f3b7d0bb740 1 -- 172.21.15.16:0/1477491867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c73ab3d0a0 msgr2=0x55c73af17070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:14.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.337+0000 7f3b7d0bb740 1 --2- 172.21.15.16:0/1477491867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c73ab3d0a0 0x55c73af17070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3b60012740 tx=0x7f3b60012c00 comp rx=0 tx=0).stop 2024-08-31T06:07:14.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.337+0000 7f3b7d0bb740 1 -- 172.21.15.16:0/1477491867 shutdown_connections 2024-08-31T06:07:14.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.337+0000 7f3b7d0bb740 1 --2- 172.21.15.16:0/1477491867 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55c73ab248e0 0x55c73af299b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:14.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.337+0000 7f3b7d0bb740 1 --2- 172.21.15.16:0/1477491867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c73ab65f60 0x55c73af233c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:14.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.337+0000 7f3b7d0bb740 1 --2- 172.21.15.16:0/1477491867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73ab45280 0x55c73af175b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:14.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.337+0000 7f3b5d600640 0 client.25484 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:14.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.337+0000 7f3b7d0bb740 1 --2- 172.21.15.16:0/1477491867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c73ab3d0a0 0x55c73af17070 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:14.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.337+0000 7f3b7d0bb740 1 -- 172.21.15.16:0/1477491867 >> 172.21.15.16:0/1477491867 conn(0x55c73ab5a350 msgr2=0x55c73af0ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:14.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.337+0000 7f3b7d0bb740 1 -- 172.21.15.16:0/1477491867 shutdown_connections 2024-08-31T06:07:14.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.337+0000 7f3b7d0bb740 1 -- 172.21.15.16:0/1477491867 wait complete. 2024-08-31T06:07:14.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.337+0000 7f3b7d0bb740 20 client.25484 trim_cache size 0 max 16384 2024-08-31T06:07:14.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.358+0000 7f3b7d0bb740 1 Processor -- start 2024-08-31T06:07:14.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.358+0000 7f3b7d0bb740 1 -- start start 2024-08-31T06:07:14.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.358+0000 7f3b7d0bb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c73ab3d0a0 0x55c73af27360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:14.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.358+0000 7f3b7d0bb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73ab45280 0x55c73af158f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:14.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.358+0000 7f3b7d0bb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c73ab65f60 0x55c73af15e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:14.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.358+0000 7f3b7d0bb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c73af27cf0 con 0x55c73ab45280 2024-08-31T06:07:14.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.358+0000 7f3b7d0bb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c73af283c0 con 0x55c73ab3d0a0 2024-08-31T06:07:14.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.358+0000 7f3b7d0bb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c73af28a90 con 0x55c73ab65f60 2024-08-31T06:07:14.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.359+0000 7f3b6fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c73ab65f60 0x55c73af15e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:14.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.359+0000 7f3b67e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73ab45280 0x55c73af158f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:14.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.359+0000 7f3b6f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c73ab3d0a0 0x55c73af27360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:14.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.359+0000 7f3b6fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c73ab65f60 0x55c73af15e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:46490/0 (socket says 172.21.15.16:46490) 2024-08-31T06:07:14.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.359+0000 7f3b6fe00640 1 -- 172.21.15.16:0/1918599779 learned_addr learned my addr 172.21.15.16:0/1918599779 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:14.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.359+0000 7f3b6fe00640 1 -- 172.21.15.16:0/1918599779 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c73ab3d0a0 msgr2=0x55c73af27360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:14.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.359+0000 7f3b6fe00640 1 --2- 172.21.15.16:0/1918599779 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c73ab3d0a0 0x55c73af27360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:14.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.359+0000 7f3b6fe00640 1 -- 172.21.15.16:0/1918599779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73ab45280 msgr2=0x55c73af158f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:14.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.359+0000 7f3b6fe00640 1 --2- 172.21.15.16:0/1918599779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73ab45280 0x55c73af158f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:14.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.359+0000 7f3b6fe00640 1 -- 172.21.15.16:0/1918599779 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c73af18330 con 0x55c73ab65f60 2024-08-31T06:07:14.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.359+0000 7f3b67e00640 1 --2- 172.21.15.16:0/1918599779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73ab45280 0x55c73af158f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:07:14.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.360+0000 7f3b6fe00640 1 --2- 172.21.15.16:0/1918599779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c73ab65f60 0x55c73af15e30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3b68009a30 tx=0x7f3b68041000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=481433762c2d92a4 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:14.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.360+0000 7f3b5fe00640 1 -- 172.21.15.16:0/1918599779 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b68030b70 con 0x55c73ab65f60 2024-08-31T06:07:14.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.360+0000 7f3b5fe00640 1 -- 172.21.15.16:0/1918599779 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b68036040 con 0x55c73ab65f60 2024-08-31T06:07:14.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.360+0000 7f3b5fe00640 1 -- 172.21.15.16:0/1918599779 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b68004670 con 0x55c73ab65f60 2024-08-31T06:07:14.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.361+0000 7f3b7d0bb740 1 -- 172.21.15.16:0/1918599779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c73ab65f60 msgr2=0x55c73af15e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:14.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.361+0000 7f3b7d0bb740 1 --2- 172.21.15.16:0/1918599779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c73ab65f60 0x55c73af15e30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3b68009a30 tx=0x7f3b68041000 comp rx=0 tx=0).stop 2024-08-31T06:07:14.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.361+0000 7f3b7d0bb740 1 -- 172.21.15.16:0/1918599779 shutdown_connections 2024-08-31T06:07:14.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.361+0000 7f3b7d0bb740 1 --2- 172.21.15.16:0/1918599779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c73ab65f60 0x55c73af15e30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:14.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.361+0000 7f3b7d0bb740 1 --2- 172.21.15.16:0/1918599779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73ab45280 0x55c73af158f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:14.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.361+0000 7f3b7d0bb740 1 --2- 172.21.15.16:0/1918599779 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c73ab3d0a0 0x55c73af27360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:14.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.361+0000 7f3b7d0bb740 1 -- 172.21.15.16:0/1918599779 >> 172.21.15.16:0/1918599779 conn(0x55c73ab5a350 msgr2=0x55c73af23540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:14.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.361+0000 7f3b7d0bb740 1 -- 172.21.15.16:0/1918599779 shutdown_connections 2024-08-31T06:07:14.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.361+0000 7f3b7d0bb740 1 -- 172.21.15.16:0/1918599779 wait complete. 2024-08-31T06:07:14.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.361+0000 7f3b7d0bb740 1 -- start start 2024-08-31T06:07:14.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.362+0000 7f3b7d0bb740 1 Processor -- start 2024-08-31T06:07:14.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.362+0000 7f3b7d0bb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c73ab3d0a0 0x55c73af32410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:14.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.362+0000 7f3b7d0bb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73ab45280 0x55c73af2d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:14.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.362+0000 7f3b7d0bb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c73ab65f60 0x55c73af2de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:14.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.362+0000 7f3b7d0bb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c73af29830 con 0x55c73ab45280 2024-08-31T06:07:14.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.362+0000 7f3b7d0bb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c73af29160 con 0x55c73ab65f60 2024-08-31T06:07:14.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.362+0000 7f3b7d0bb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c73af29f00 con 0x55c73ab3d0a0 2024-08-31T06:07:14.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.362+0000 7f3b7d0bb740 10 client.0 _ll_register_callbacks cb 0x7f3b60009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:07:14.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.362+0000 7f3b6f400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c73ab3d0a0 0x55c73af32410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:14.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.362+0000 7f3b6f400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c73ab3d0a0 0x55c73af32410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:46504/0 (socket says 172.21.15.16:46504) 2024-08-31T06:07:14.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.362+0000 7f3b6f400640 1 -- 172.21.15.16:0/3427684636 learned_addr learned my addr 172.21.15.16:0/3427684636 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:14.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.362+0000 7f3b6fe00640 1 --2- 172.21.15.16:0/3427684636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c73ab65f60 0x55c73af2de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:14.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.362+0000 7f3b67e00640 1 --2- 172.21.15.16:0/3427684636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73ab45280 0x55c73af2d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:14.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.363+0000 7f3b6f400640 1 -- 172.21.15.16:0/3427684636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c73ab65f60 msgr2=0x55c73af2de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:14.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.363+0000 7f3b6f400640 1 --2- 172.21.15.16:0/3427684636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c73ab65f60 0x55c73af2de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:14.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.363+0000 7f3b6f400640 1 -- 172.21.15.16:0/3427684636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73ab45280 msgr2=0x55c73af2d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:14.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.363+0000 7f3b6f400640 1 --2- 172.21.15.16:0/3427684636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73ab45280 0x55c73af2d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:14.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.363+0000 7f3b6f400640 1 -- 172.21.15.16:0/3427684636 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c73af2e540 con 0x55c73ab3d0a0 2024-08-31T06:07:14.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.363+0000 7f3b6f400640 1 --2- 172.21.15.16:0/3427684636 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c73ab3d0a0 0x55c73af32410 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3b6001c910 tx=0x7f3b6001c6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=926e4ed2c63b23bd server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:14.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.363+0000 7f3b5d600640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:07:14.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.363+0000 7f3b5d600640 1 -- 172.21.15.16:0/3427684636 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b60029070 con 0x55c73ab3d0a0 2024-08-31T06:07:14.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.363+0000 7f3b7d0bb740 1 -- 172.21.15.16:0/3427684636 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b68036040 con 0x55c73ab3d0a0 2024-08-31T06:07:14.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.363+0000 7f3b5d600640 1 -- 172.21.15.16:0/3427684636 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7f3b6001cd10 con 0x55c73ab3d0a0 2024-08-31T06:07:14.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.364+0000 7f3b5d600640 1 -- 172.21.15.16:0/3427684636 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3b6000fe60 con 0x55c73ab3d0a0 2024-08-31T06:07:14.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.364+0000 7f3b7d0bb740 10 client.25489 fetch_fsmap learned FSMap version 25 2024-08-31T06:07:14.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.364+0000 7f3b7d0bb740 1 -- 172.21.15.16:0/3427684636 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c73af94d70 con 0x55c73ab3d0a0 2024-08-31T06:07:14.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.364+0000 7f3b5d600640 1 -- 172.21.15.16:0/3427684636 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b68036040 con 0x55c73ab3d0a0 2024-08-31T06:07:14.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.364+0000 7f3b5d600640 1 -- 172.21.15.16:0/3427684636 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3b60003a00 con 0x55c73ab3d0a0 2024-08-31T06:07:14.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.364+0000 7f3b7d0bb740 10 client.25489 fetch_fsmap finished waiting for FSMap version 25 2024-08-31T06:07:14.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.364+0000 7f3b7d0bb740 10 client.25489 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-31T06:07:14.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.364+0000 7f3b7d0bb740 20 client.25489 populate_metadata read hostname 'smithi016' 2024-08-31T06:07:14.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.364+0000 7f3b7d0bb740 1 --2- 172.21.15.16:0/3427684636 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55c73aaf9c90 0x55c73af985e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:14.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.364+0000 7f3b7d0bb740 4 client.25489 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-31T06:07:14.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.364+0000 7f3b7d0bb740 1 -- 172.21.15.16:0/3427684636 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "status"}) -- 0x55c73ae7ff40 con 0x55c73aaf9c90 2024-08-31T06:07:14.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.365+0000 7f3b67e00640 1 --2- 172.21.15.16:0/3427684636 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55c73aaf9c90 0x55c73af985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:14.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.365+0000 7f3b67e00640 1 --2- 172.21.15.16:0/3427684636 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55c73aaf9c90 0x55c73af985e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8768bc1dbcc2b04 server_cookie=286a8cba34d08885 in_seq=0 out_seq=0 2024-08-31T06:07:14.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.365+0000 7f3b5d600640 10 client.25489 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:14.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.369+0000 7f3b5d600640 1 -- 172.21.15.16:0/3427684636 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55c73ae7ff40 con 0x55c73aaf9c90 2024-08-31T06:07:14.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.369+0000 7f3b5d600640 10 client.25489 handle_command_reply: tid=0 2024-08-31T06:07:14.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.369+0000 7f3b7d0bb740 1 client.25489 shutdown 2024-08-31T06:07:14.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.369+0000 7f3b7d0bb740 1 -- 172.21.15.16:0/3427684636 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c73ab3d0a0 msgr2=0x55c73af32410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:14.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.369+0000 7f3b7d0bb740 1 --2- 172.21.15.16:0/3427684636 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c73ab3d0a0 0x55c73af32410 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3b6001c910 tx=0x7f3b6001c6d0 comp rx=0 tx=0).stop 2024-08-31T06:07:14.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.369+0000 7f3b7d0bb740 1 -- 172.21.15.16:0/3427684636 shutdown_connections 2024-08-31T06:07:14.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.369+0000 7f3b7d0bb740 1 --2- 172.21.15.16:0/3427684636 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55c73aaf9c90 0x55c73af985e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:14.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.369+0000 7f3b7d0bb740 1 --2- 172.21.15.16:0/3427684636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c73ab65f60 0x55c73af2de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:14.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.369+0000 7f3b7d0bb740 1 --2- 172.21.15.16:0/3427684636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73ab45280 0x55c73af2d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:14.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.369+0000 7f3b5d600640 0 client.25489 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:14.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.369+0000 7f3b7d0bb740 1 --2- 172.21.15.16:0/3427684636 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c73ab3d0a0 0x55c73af32410 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:14.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.369+0000 7f3b7d0bb740 1 -- 172.21.15.16:0/3427684636 >> 172.21.15.16:0/3427684636 conn(0x55c73ab5a350 msgr2=0x55c73af27360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:14.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.370+0000 7f3b7d0bb740 1 -- 172.21.15.16:0/3427684636 shutdown_connections 2024-08-31T06:07:14.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.370+0000 7f3b7d0bb740 1 -- 172.21.15.16:0/3427684636 wait complete. 2024-08-31T06:07:14.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.370+0000 7f3b7d0bb740 20 client.25489 trim_cache size 0 max 16384 2024-08-31T06:07:14.377 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:07:14.377 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "673e4aa0-675d-11ef-bcd4-c7b262605968", 2024-08-31T06:07:14.377 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-08-31T06:07:14.377 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24737, 2024-08-31T06:07:14.377 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-08-31T06:07:14.377 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-08-31T06:07:14.377 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-08-31T06:07:14.378 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 166.491891627, 2024-08-31T06:07:14.378 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 25, 2024-08-31T06:07:14.378 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 80, 2024-08-31T06:07:14.378 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 80, 2024-08-31T06:07:14.378 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 181.04779305599999 2024-08-31T06:07:14.378 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:07:14.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.370+0000 7f3b76400640 1 -- 172.21.15.16:0/1350379731 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3b540b4ca0 msgr2=0x7f3b540b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.370+0000 7f3b76400640 1 --2- 172.21.15.16:0/1350379731 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3b540b4ca0 0x7f3b540b71b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3b68006010 tx=0x7f3b68009480 comp rx=0 tx=0).stop 2024-08-31T06:07:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.370+0000 7f3b76400640 1 -- 172.21.15.16:0/1350379731 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b70075e40 msgr2=0x7f3b7019ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.370+0000 7f3b76400640 1 --2- 172.21.15.16:0/1350379731 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b70075e40 0x7f3b7019ca70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3b5800e970 tx=0x7f3b5800ee30 comp rx=0 tx=0).stop 2024-08-31T06:07:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.371+0000 7f3b76400640 1 -- 172.21.15.16:0/1350379731 shutdown_connections 2024-08-31T06:07:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.371+0000 7f3b76400640 1 --2- 172.21.15.16:0/1350379731 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3b540b4ca0 0x7f3b540b71b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.371+0000 7f3b76400640 1 --2- 172.21.15.16:0/1350379731 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b70075e40 0x7f3b7019ca70 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.371+0000 7f3b76400640 1 --2- 172.21.15.16:0/1350379731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b7010fec0 0x7f3b701a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.371+0000 7f3b76400640 1 --2- 172.21.15.16:0/1350379731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b700754f0 0x7f3b7019c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.371+0000 7f3b76400640 1 -- 172.21.15.16:0/1350379731 >> 172.21.15.16:0/1350379731 conn(0x7f3b7007b5f0 msgr2=0x7f3b70108380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.371+0000 7f3b76400640 1 -- 172.21.15.16:0/1350379731 shutdown_connections 2024-08-31T06:07:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:14.371+0000 7f3b76400640 1 -- 172.21.15.16:0/1350379731 wait complete. 2024-08-31T06:07:14.469 INFO:teuthology.orchestra.run.smithi016.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2024-08-31T06:07:14.475 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-08-31T06:07:14.571 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/: 2024-08-31T06:07:14.571 INFO:teuthology.orchestra.run.smithi016.stdout:total 0 2024-08-31T06:07:14.571 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 31 06:07 . 2024-08-31T06:07:14.571 INFO:teuthology.orchestra.run.smithi016.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Aug 31 06:07 .. 2024-08-31T06:07:14.571 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 31 06:07 data 2024-08-31T06:07:14.571 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:07:14.571 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data: 2024-08-31T06:07:14.571 INFO:teuthology.orchestra.run.smithi016.stdout:total 35 2024-08-31T06:07:14.571 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 31 06:07 . 2024-08-31T06:07:14.571 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 31 06:07 .. 2024-08-31T06:07:14.572 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 31 06:07 base 2024-08-31T06:07:14.572 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 Aug 31 06:07 global 2024-08-31T06:07:14.572 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 log 2024-08-31T06:07:14.572 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 pg_commit_ts 2024-08-31T06:07:14.572 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 pg_dynshmem 2024-08-31T06:07:14.572 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Aug 31 06:07 pg_hba.conf 2024-08-31T06:07:14.572 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Aug 31 06:07 pg_ident.conf 2024-08-31T06:07:14.572 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 31 06:07 pg_logical 2024-08-31T06:07:14.572 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 31 06:07 pg_multixact 2024-08-31T06:07:14.572 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 pg_notify 2024-08-31T06:07:14.572 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 pg_replslot 2024-08-31T06:07:14.572 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 pg_serial 2024-08-31T06:07:14.572 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 pg_snapshots 2024-08-31T06:07:14.572 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 pg_stat 2024-08-31T06:07:14.573 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 pg_stat_tmp 2024-08-31T06:07:14.573 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 31 06:07 pg_subtrans 2024-08-31T06:07:14.573 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 pg_tblspc 2024-08-31T06:07:14.573 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 pg_twophase 2024-08-31T06:07:14.573 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 31 06:07 PG_VERSION 2024-08-31T06:07:14.573 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 31 06:07 pg_wal 2024-08-31T06:07:14.573 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 31 06:07 pg_xact 2024-08-31T06:07:14.573 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Aug 31 06:07 postgresql.auto.conf 2024-08-31T06:07:14.573 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28074 Aug 31 06:07 postgresql.conf 2024-08-31T06:07:14.573 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:07:14.573 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/base: 2024-08-31T06:07:14.573 INFO:teuthology.orchestra.run.smithi016.stdout:total 0 2024-08-31T06:07:14.573 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 31 06:07 . 2024-08-31T06:07:14.574 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 31 06:07 .. 2024-08-31T06:07:14.574 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Aug 31 06:07 1 2024-08-31T06:07:14.574 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Aug 31 06:07 13493 2024-08-31T06:07:14.574 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Aug 31 06:07 13494 2024-08-31T06:07:14.574 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:07:14.574 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/base/1: 2024-08-31T06:07:14.574 INFO:teuthology.orchestra.run.smithi016.stdout:total 7777 2024-08-31T06:07:14.574 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Aug 31 06:07 . 2024-08-31T06:07:14.574 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 31 06:07 .. 2024-08-31T06:07:14.574 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 112 2024-08-31T06:07:14.574 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 113 2024-08-31T06:07:14.574 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 31 06:07 1247 2024-08-31T06:07:14.574 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 1247_fsm 2024-08-31T06:07:14.574 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 1247_vm 2024-08-31T06:07:14.575 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Aug 31 06:07 1249 2024-08-31T06:07:14.575 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 1249_fsm 2024-08-31T06:07:14.575 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 1249_vm 2024-08-31T06:07:14.575 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Aug 31 06:07 1255 2024-08-31T06:07:14.575 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 1255_fsm 2024-08-31T06:07:14.575 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 1255_vm 2024-08-31T06:07:14.575 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Aug 31 06:07 1259 2024-08-31T06:07:14.575 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 1259_fsm 2024-08-31T06:07:14.575 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 1259_vm 2024-08-31T06:07:14.575 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Aug 31 06:07 13344 2024-08-31T06:07:14.575 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 13344_fsm 2024-08-31T06:07:14.575 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13344_vm 2024-08-31T06:07:14.575 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 13346 2024-08-31T06:07:14.576 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13348 2024-08-31T06:07:14.576 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13349 2024-08-31T06:07:14.576 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 13349_fsm 2024-08-31T06:07:14.576 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13349_vm 2024-08-31T06:07:14.576 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 13351 2024-08-31T06:07:14.576 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13353 2024-08-31T06:07:14.576 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13354 2024-08-31T06:07:14.576 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 13354_fsm 2024-08-31T06:07:14.576 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13354_vm 2024-08-31T06:07:14.576 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 13356 2024-08-31T06:07:14.576 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13358 2024-08-31T06:07:14.576 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13359 2024-08-31T06:07:14.576 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 13359_fsm 2024-08-31T06:07:14.576 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13359_vm 2024-08-31T06:07:14.577 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 13361 2024-08-31T06:07:14.577 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13363 2024-08-31T06:07:14.577 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 1417 2024-08-31T06:07:14.577 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 1418 2024-08-31T06:07:14.577 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 174 2024-08-31T06:07:14.577 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 175 2024-08-31T06:07:14.577 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2187 2024-08-31T06:07:14.577 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2224 2024-08-31T06:07:14.577 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2328 2024-08-31T06:07:14.577 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2336 2024-08-31T06:07:14.577 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2337 2024-08-31T06:07:14.578 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2579 2024-08-31T06:07:14.578 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2600 2024-08-31T06:07:14.578 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2600_fsm 2024-08-31T06:07:14.578 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2600_vm 2024-08-31T06:07:14.578 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2601 2024-08-31T06:07:14.578 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2601_fsm 2024-08-31T06:07:14.578 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2601_vm 2024-08-31T06:07:14.578 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 31 06:07 2602 2024-08-31T06:07:14.578 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2602_fsm 2024-08-31T06:07:14.578 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2602_vm 2024-08-31T06:07:14.578 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 2603 2024-08-31T06:07:14.579 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2603_fsm 2024-08-31T06:07:14.579 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2603_vm 2024-08-31T06:07:14.579 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2604 2024-08-31T06:07:14.579 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2605 2024-08-31T06:07:14.579 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2605_fsm 2024-08-31T06:07:14.579 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2605_vm 2024-08-31T06:07:14.579 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2606 2024-08-31T06:07:14.579 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2606_fsm 2024-08-31T06:07:14.579 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2606_vm 2024-08-31T06:07:14.579 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2607 2024-08-31T06:07:14.579 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2607_fsm 2024-08-31T06:07:14.579 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2607_vm 2024-08-31T06:07:14.580 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Aug 31 06:07 2608 2024-08-31T06:07:14.580 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2608_fsm 2024-08-31T06:07:14.580 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2608_vm 2024-08-31T06:07:14.580 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Aug 31 06:07 2609 2024-08-31T06:07:14.580 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2609_fsm 2024-08-31T06:07:14.580 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2609_vm 2024-08-31T06:07:14.580 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 2610 2024-08-31T06:07:14.580 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2610_fsm 2024-08-31T06:07:14.580 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2610_vm 2024-08-31T06:07:14.581 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2611 2024-08-31T06:07:14.581 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2612 2024-08-31T06:07:14.581 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2612_fsm 2024-08-31T06:07:14.581 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2612_vm 2024-08-31T06:07:14.581 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2613 2024-08-31T06:07:14.581 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2615 2024-08-31T06:07:14.581 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2615_fsm 2024-08-31T06:07:14.581 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2615_vm 2024-08-31T06:07:14.581 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2616 2024-08-31T06:07:14.581 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2616_fsm 2024-08-31T06:07:14.581 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2616_vm 2024-08-31T06:07:14.581 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Aug 31 06:07 2617 2024-08-31T06:07:14.581 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2617_fsm 2024-08-31T06:07:14.582 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2617_vm 2024-08-31T06:07:14.582 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Aug 31 06:07 2618 2024-08-31T06:07:14.582 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2618_fsm 2024-08-31T06:07:14.582 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2618_vm 2024-08-31T06:07:14.582 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Aug 31 06:07 2619 2024-08-31T06:07:14.582 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2619_fsm 2024-08-31T06:07:14.582 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2619_vm 2024-08-31T06:07:14.582 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2620 2024-08-31T06:07:14.582 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2650 2024-08-31T06:07:14.582 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2651 2024-08-31T06:07:14.582 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2652 2024-08-31T06:07:14.582 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 31 06:07 2653 2024-08-31T06:07:14.582 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 31 06:07 2654 2024-08-31T06:07:14.583 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 2655 2024-08-31T06:07:14.583 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2656 2024-08-31T06:07:14.583 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2657 2024-08-31T06:07:14.583 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Aug 31 06:07 2658 2024-08-31T06:07:14.583 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 31 06:07 2659 2024-08-31T06:07:14.583 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2660 2024-08-31T06:07:14.583 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2661 2024-08-31T06:07:14.583 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 2662 2024-08-31T06:07:14.583 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 31 06:07 2663 2024-08-31T06:07:14.583 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2664 2024-08-31T06:07:14.583 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2665 2024-08-31T06:07:14.584 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2666 2024-08-31T06:07:14.584 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2667 2024-08-31T06:07:14.584 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2668 2024-08-31T06:07:14.584 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2669 2024-08-31T06:07:14.584 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2670 2024-08-31T06:07:14.584 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Aug 31 06:07 2673 2024-08-31T06:07:14.584 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Aug 31 06:07 2674 2024-08-31T06:07:14.584 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Aug 31 06:07 2675 2024-08-31T06:07:14.585 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2678 2024-08-31T06:07:14.585 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2679 2024-08-31T06:07:14.585 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2680 2024-08-31T06:07:14.585 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2681 2024-08-31T06:07:14.585 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2682 2024-08-31T06:07:14.585 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2683 2024-08-31T06:07:14.585 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2684 2024-08-31T06:07:14.585 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2685 2024-08-31T06:07:14.585 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2686 2024-08-31T06:07:14.585 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2687 2024-08-31T06:07:14.585 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 31 06:07 2688 2024-08-31T06:07:14.585 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 31 06:07 2689 2024-08-31T06:07:14.586 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Aug 31 06:07 2690 2024-08-31T06:07:14.586 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Aug 31 06:07 2691 2024-08-31T06:07:14.586 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2692 2024-08-31T06:07:14.586 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2693 2024-08-31T06:07:14.586 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2696 2024-08-31T06:07:14.586 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2699 2024-08-31T06:07:14.586 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2701 2024-08-31T06:07:14.586 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2702 2024-08-31T06:07:14.586 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 2703 2024-08-31T06:07:14.586 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 31 06:07 2704 2024-08-31T06:07:14.586 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2753 2024-08-31T06:07:14.587 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2753_fsm 2024-08-31T06:07:14.587 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2753_vm 2024-08-31T06:07:14.587 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2754 2024-08-31T06:07:14.587 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2755 2024-08-31T06:07:14.587 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 2756 2024-08-31T06:07:14.587 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 2757 2024-08-31T06:07:14.587 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2830 2024-08-31T06:07:14.587 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2831 2024-08-31T06:07:14.587 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2832 2024-08-31T06:07:14.587 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2833 2024-08-31T06:07:14.587 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2834 2024-08-31T06:07:14.587 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2835 2024-08-31T06:07:14.588 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2836 2024-08-31T06:07:14.588 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2837 2024-08-31T06:07:14.588 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Aug 31 06:07 2838 2024-08-31T06:07:14.588 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2838_fsm 2024-08-31T06:07:14.588 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2838_vm 2024-08-31T06:07:14.588 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2839 2024-08-31T06:07:14.588 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2840 2024-08-31T06:07:14.588 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2840_fsm 2024-08-31T06:07:14.588 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2840_vm 2024-08-31T06:07:14.588 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2841 2024-08-31T06:07:14.588 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2995 2024-08-31T06:07:14.589 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2996 2024-08-31T06:07:14.589 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3079 2024-08-31T06:07:14.589 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3079_fsm 2024-08-31T06:07:14.589 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3079_vm 2024-08-31T06:07:14.589 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3080 2024-08-31T06:07:14.589 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3081 2024-08-31T06:07:14.589 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 31 06:07 3085 2024-08-31T06:07:14.589 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3118 2024-08-31T06:07:14.589 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3119 2024-08-31T06:07:14.589 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Aug 31 06:07 3164 2024-08-31T06:07:14.589 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3256 2024-08-31T06:07:14.589 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3257 2024-08-31T06:07:14.590 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3258 2024-08-31T06:07:14.590 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3350 2024-08-31T06:07:14.590 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3351 2024-08-31T06:07:14.590 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3379 2024-08-31T06:07:14.590 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3380 2024-08-31T06:07:14.590 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3381 2024-08-31T06:07:14.590 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3394 2024-08-31T06:07:14.590 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3394_fsm 2024-08-31T06:07:14.590 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3394_vm 2024-08-31T06:07:14.590 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3395 2024-08-31T06:07:14.590 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3429 2024-08-31T06:07:14.591 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3430 2024-08-31T06:07:14.591 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3431 2024-08-31T06:07:14.591 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3433 2024-08-31T06:07:14.591 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3439 2024-08-31T06:07:14.591 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3440 2024-08-31T06:07:14.591 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 31 06:07 3455 2024-08-31T06:07:14.591 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Aug 31 06:07 3456 2024-08-31T06:07:14.591 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3456_fsm 2024-08-31T06:07:14.591 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3456_vm 2024-08-31T06:07:14.591 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3466 2024-08-31T06:07:14.591 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3467 2024-08-31T06:07:14.591 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3468 2024-08-31T06:07:14.591 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3501 2024-08-31T06:07:14.592 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3502 2024-08-31T06:07:14.592 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3503 2024-08-31T06:07:14.592 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3534 2024-08-31T06:07:14.592 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3541 2024-08-31T06:07:14.592 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3541_fsm 2024-08-31T06:07:14.592 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3541_vm 2024-08-31T06:07:14.592 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3542 2024-08-31T06:07:14.592 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3574 2024-08-31T06:07:14.592 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3575 2024-08-31T06:07:14.592 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3576 2024-08-31T06:07:14.592 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3596 2024-08-31T06:07:14.592 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3597 2024-08-31T06:07:14.593 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3598 2024-08-31T06:07:14.593 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3599 2024-08-31T06:07:14.593 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3600 2024-08-31T06:07:14.593 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3600_fsm 2024-08-31T06:07:14.593 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3600_vm 2024-08-31T06:07:14.593 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3601 2024-08-31T06:07:14.593 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3601_fsm 2024-08-31T06:07:14.593 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3601_vm 2024-08-31T06:07:14.593 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3602 2024-08-31T06:07:14.593 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3602_fsm 2024-08-31T06:07:14.593 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3602_vm 2024-08-31T06:07:14.594 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3603 2024-08-31T06:07:14.594 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3603_fsm 2024-08-31T06:07:14.594 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3603_vm 2024-08-31T06:07:14.594 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3604 2024-08-31T06:07:14.594 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3605 2024-08-31T06:07:14.594 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3606 2024-08-31T06:07:14.594 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3607 2024-08-31T06:07:14.594 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3608 2024-08-31T06:07:14.594 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 3609 2024-08-31T06:07:14.594 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3712 2024-08-31T06:07:14.594 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3764 2024-08-31T06:07:14.594 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3764_fsm 2024-08-31T06:07:14.595 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3764_vm 2024-08-31T06:07:14.595 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3766 2024-08-31T06:07:14.595 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3767 2024-08-31T06:07:14.595 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3997 2024-08-31T06:07:14.595 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4143 2024-08-31T06:07:14.595 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4144 2024-08-31T06:07:14.595 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4145 2024-08-31T06:07:14.595 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4146 2024-08-31T06:07:14.595 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4147 2024-08-31T06:07:14.595 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4148 2024-08-31T06:07:14.595 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4149 2024-08-31T06:07:14.595 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4150 2024-08-31T06:07:14.595 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4151 2024-08-31T06:07:14.596 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4152 2024-08-31T06:07:14.596 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4153 2024-08-31T06:07:14.596 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4154 2024-08-31T06:07:14.596 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4155 2024-08-31T06:07:14.596 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4156 2024-08-31T06:07:14.596 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4157 2024-08-31T06:07:14.596 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4158 2024-08-31T06:07:14.596 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4159 2024-08-31T06:07:14.596 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4160 2024-08-31T06:07:14.596 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4161 2024-08-31T06:07:14.597 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4162 2024-08-31T06:07:14.597 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4163 2024-08-31T06:07:14.597 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4164 2024-08-31T06:07:14.597 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4165 2024-08-31T06:07:14.599 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4166 2024-08-31T06:07:14.599 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4167 2024-08-31T06:07:14.599 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4168 2024-08-31T06:07:14.599 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4169 2024-08-31T06:07:14.600 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4170 2024-08-31T06:07:14.600 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4171 2024-08-31T06:07:14.600 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4172 2024-08-31T06:07:14.600 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4173 2024-08-31T06:07:14.600 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4174 2024-08-31T06:07:14.600 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 5002 2024-08-31T06:07:14.600 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 548 2024-08-31T06:07:14.600 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 549 2024-08-31T06:07:14.600 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 6102 2024-08-31T06:07:14.600 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 6104 2024-08-31T06:07:14.600 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 6106 2024-08-31T06:07:14.600 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 6110 2024-08-31T06:07:14.601 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 6111 2024-08-31T06:07:14.601 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 6112 2024-08-31T06:07:14.601 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 6113 2024-08-31T06:07:14.601 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 6117 2024-08-31T06:07:14.601 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 826 2024-08-31T06:07:14.601 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 827 2024-08-31T06:07:14.601 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 828 2024-08-31T06:07:14.601 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Aug 31 06:07 pg_filenode.map 2024-08-31T06:07:14.601 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 31 06:07 PG_VERSION 2024-08-31T06:07:14.601 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:07:14.601 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-08-31T06:07:14.602 INFO:teuthology.orchestra.run.smithi016.stdout:total 7777 2024-08-31T06:07:14.602 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Aug 31 06:07 . 2024-08-31T06:07:14.602 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 31 06:07 .. 2024-08-31T06:07:14.602 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 112 2024-08-31T06:07:14.602 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 113 2024-08-31T06:07:14.602 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 31 06:07 1247 2024-08-31T06:07:14.602 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 1247_fsm 2024-08-31T06:07:14.602 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 1247_vm 2024-08-31T06:07:14.602 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Aug 31 06:07 1249 2024-08-31T06:07:14.602 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 1249_fsm 2024-08-31T06:07:14.602 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 1249_vm 2024-08-31T06:07:14.603 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Aug 31 06:07 1255 2024-08-31T06:07:14.603 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 1255_fsm 2024-08-31T06:07:14.603 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 1255_vm 2024-08-31T06:07:14.603 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Aug 31 06:07 1259 2024-08-31T06:07:14.603 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 1259_fsm 2024-08-31T06:07:14.603 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 1259_vm 2024-08-31T06:07:14.603 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Aug 31 06:07 13344 2024-08-31T06:07:14.603 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 13344_fsm 2024-08-31T06:07:14.603 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13344_vm 2024-08-31T06:07:14.603 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 13346 2024-08-31T06:07:14.603 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13348 2024-08-31T06:07:14.604 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13349 2024-08-31T06:07:14.604 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 13349_fsm 2024-08-31T06:07:14.604 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13349_vm 2024-08-31T06:07:14.604 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 13351 2024-08-31T06:07:14.604 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13353 2024-08-31T06:07:14.604 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13354 2024-08-31T06:07:14.604 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 13354_fsm 2024-08-31T06:07:14.604 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13354_vm 2024-08-31T06:07:14.604 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 13356 2024-08-31T06:07:14.604 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13358 2024-08-31T06:07:14.604 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13359 2024-08-31T06:07:14.604 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 13359_fsm 2024-08-31T06:07:14.605 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13359_vm 2024-08-31T06:07:14.605 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 13361 2024-08-31T06:07:14.605 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13363 2024-08-31T06:07:14.605 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 1417 2024-08-31T06:07:14.605 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 1418 2024-08-31T06:07:14.605 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 174 2024-08-31T06:07:14.605 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 175 2024-08-31T06:07:14.605 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2187 2024-08-31T06:07:14.605 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2224 2024-08-31T06:07:14.605 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2328 2024-08-31T06:07:14.605 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2336 2024-08-31T06:07:14.605 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2337 2024-08-31T06:07:14.606 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2579 2024-08-31T06:07:14.606 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2600 2024-08-31T06:07:14.606 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2600_fsm 2024-08-31T06:07:14.606 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2600_vm 2024-08-31T06:07:14.606 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2601 2024-08-31T06:07:14.606 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2601_fsm 2024-08-31T06:07:14.606 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2601_vm 2024-08-31T06:07:14.606 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 31 06:07 2602 2024-08-31T06:07:14.606 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2602_fsm 2024-08-31T06:07:14.606 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2602_vm 2024-08-31T06:07:14.606 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 2603 2024-08-31T06:07:14.606 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2603_fsm 2024-08-31T06:07:14.606 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2603_vm 2024-08-31T06:07:14.607 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2604 2024-08-31T06:07:14.607 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2605 2024-08-31T06:07:14.607 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2605_fsm 2024-08-31T06:07:14.607 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2605_vm 2024-08-31T06:07:14.607 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2606 2024-08-31T06:07:14.607 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2606_fsm 2024-08-31T06:07:14.607 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2606_vm 2024-08-31T06:07:14.607 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2607 2024-08-31T06:07:14.607 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2607_fsm 2024-08-31T06:07:14.607 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2607_vm 2024-08-31T06:07:14.607 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Aug 31 06:07 2608 2024-08-31T06:07:14.607 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2608_fsm 2024-08-31T06:07:14.608 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2608_vm 2024-08-31T06:07:14.608 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Aug 31 06:07 2609 2024-08-31T06:07:14.608 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2609_fsm 2024-08-31T06:07:14.608 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2609_vm 2024-08-31T06:07:14.608 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 2610 2024-08-31T06:07:14.608 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2610_fsm 2024-08-31T06:07:14.608 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2610_vm 2024-08-31T06:07:14.608 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2611 2024-08-31T06:07:14.608 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2612 2024-08-31T06:07:14.608 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2612_fsm 2024-08-31T06:07:14.608 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2612_vm 2024-08-31T06:07:14.608 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2613 2024-08-31T06:07:14.608 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2615 2024-08-31T06:07:14.609 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2615_fsm 2024-08-31T06:07:14.609 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2615_vm 2024-08-31T06:07:14.609 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2616 2024-08-31T06:07:14.609 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2616_fsm 2024-08-31T06:07:14.609 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2616_vm 2024-08-31T06:07:14.609 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Aug 31 06:07 2617 2024-08-31T06:07:14.609 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2617_fsm 2024-08-31T06:07:14.609 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2617_vm 2024-08-31T06:07:14.609 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Aug 31 06:07 2618 2024-08-31T06:07:14.609 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2618_fsm 2024-08-31T06:07:14.609 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2618_vm 2024-08-31T06:07:14.610 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Aug 31 06:07 2619 2024-08-31T06:07:14.610 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2619_fsm 2024-08-31T06:07:14.610 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2619_vm 2024-08-31T06:07:14.610 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2620 2024-08-31T06:07:14.610 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2650 2024-08-31T06:07:14.610 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2651 2024-08-31T06:07:14.610 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2652 2024-08-31T06:07:14.610 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 31 06:07 2653 2024-08-31T06:07:14.610 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 31 06:07 2654 2024-08-31T06:07:14.610 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 2655 2024-08-31T06:07:14.610 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2656 2024-08-31T06:07:14.611 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2657 2024-08-31T06:07:14.611 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Aug 31 06:07 2658 2024-08-31T06:07:14.611 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 31 06:07 2659 2024-08-31T06:07:14.611 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2660 2024-08-31T06:07:14.611 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2661 2024-08-31T06:07:14.611 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 2662 2024-08-31T06:07:14.611 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 31 06:07 2663 2024-08-31T06:07:14.611 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2664 2024-08-31T06:07:14.611 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2665 2024-08-31T06:07:14.611 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2666 2024-08-31T06:07:14.611 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2667 2024-08-31T06:07:14.612 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2668 2024-08-31T06:07:14.612 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2669 2024-08-31T06:07:14.612 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2670 2024-08-31T06:07:14.612 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Aug 31 06:07 2673 2024-08-31T06:07:14.612 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Aug 31 06:07 2674 2024-08-31T06:07:14.612 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Aug 31 06:07 2675 2024-08-31T06:07:14.612 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2678 2024-08-31T06:07:14.612 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2679 2024-08-31T06:07:14.612 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2680 2024-08-31T06:07:14.612 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2681 2024-08-31T06:07:14.612 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2682 2024-08-31T06:07:14.612 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2683 2024-08-31T06:07:14.613 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2684 2024-08-31T06:07:14.613 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2685 2024-08-31T06:07:14.613 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2686 2024-08-31T06:07:14.613 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2687 2024-08-31T06:07:14.613 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 31 06:07 2688 2024-08-31T06:07:14.613 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 31 06:07 2689 2024-08-31T06:07:14.613 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Aug 31 06:07 2690 2024-08-31T06:07:14.613 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Aug 31 06:07 2691 2024-08-31T06:07:14.613 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2692 2024-08-31T06:07:14.613 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2693 2024-08-31T06:07:14.613 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2696 2024-08-31T06:07:14.613 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2699 2024-08-31T06:07:14.614 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2701 2024-08-31T06:07:14.614 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2702 2024-08-31T06:07:14.614 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 2703 2024-08-31T06:07:14.614 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 31 06:07 2704 2024-08-31T06:07:14.614 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2753 2024-08-31T06:07:14.614 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2753_fsm 2024-08-31T06:07:14.614 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2753_vm 2024-08-31T06:07:14.614 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2754 2024-08-31T06:07:14.614 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2755 2024-08-31T06:07:14.614 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 2756 2024-08-31T06:07:14.614 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 2757 2024-08-31T06:07:14.615 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2830 2024-08-31T06:07:14.615 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2831 2024-08-31T06:07:14.615 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2832 2024-08-31T06:07:14.615 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2833 2024-08-31T06:07:14.615 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2834 2024-08-31T06:07:14.615 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2835 2024-08-31T06:07:14.615 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2836 2024-08-31T06:07:14.615 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2837 2024-08-31T06:07:14.615 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Aug 31 06:07 2838 2024-08-31T06:07:14.615 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2838_fsm 2024-08-31T06:07:14.615 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2838_vm 2024-08-31T06:07:14.616 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2839 2024-08-31T06:07:14.616 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2840 2024-08-31T06:07:14.616 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2840_fsm 2024-08-31T06:07:14.616 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2840_vm 2024-08-31T06:07:14.616 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2841 2024-08-31T06:07:14.616 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2995 2024-08-31T06:07:14.616 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2996 2024-08-31T06:07:14.616 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3079 2024-08-31T06:07:14.616 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3079_fsm 2024-08-31T06:07:14.616 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3079_vm 2024-08-31T06:07:14.616 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3080 2024-08-31T06:07:14.616 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3081 2024-08-31T06:07:14.617 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 31 06:07 3085 2024-08-31T06:07:14.617 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3118 2024-08-31T06:07:14.617 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3119 2024-08-31T06:07:14.617 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Aug 31 06:07 3164 2024-08-31T06:07:14.617 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3256 2024-08-31T06:07:14.617 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3257 2024-08-31T06:07:14.617 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3258 2024-08-31T06:07:14.617 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3350 2024-08-31T06:07:14.617 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3351 2024-08-31T06:07:14.617 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3379 2024-08-31T06:07:14.618 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3380 2024-08-31T06:07:14.618 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3381 2024-08-31T06:07:14.618 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3394 2024-08-31T06:07:14.618 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3394_fsm 2024-08-31T06:07:14.618 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3394_vm 2024-08-31T06:07:14.618 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3395 2024-08-31T06:07:14.618 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3429 2024-08-31T06:07:14.618 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3430 2024-08-31T06:07:14.618 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3431 2024-08-31T06:07:14.618 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3433 2024-08-31T06:07:14.618 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3439 2024-08-31T06:07:14.618 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3440 2024-08-31T06:07:14.619 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 31 06:07 3455 2024-08-31T06:07:14.619 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Aug 31 06:07 3456 2024-08-31T06:07:14.619 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3456_fsm 2024-08-31T06:07:14.619 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3456_vm 2024-08-31T06:07:14.619 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3466 2024-08-31T06:07:14.619 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3467 2024-08-31T06:07:14.619 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3468 2024-08-31T06:07:14.619 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3501 2024-08-31T06:07:14.619 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3502 2024-08-31T06:07:14.619 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3503 2024-08-31T06:07:14.619 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3534 2024-08-31T06:07:14.619 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3541 2024-08-31T06:07:14.619 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3541_fsm 2024-08-31T06:07:14.620 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3541_vm 2024-08-31T06:07:14.620 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3542 2024-08-31T06:07:14.620 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3574 2024-08-31T06:07:14.620 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3575 2024-08-31T06:07:14.620 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3576 2024-08-31T06:07:14.620 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3596 2024-08-31T06:07:14.620 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3597 2024-08-31T06:07:14.620 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3598 2024-08-31T06:07:14.620 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3599 2024-08-31T06:07:14.620 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3600 2024-08-31T06:07:14.620 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3600_fsm 2024-08-31T06:07:14.620 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3600_vm 2024-08-31T06:07:14.621 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3601 2024-08-31T06:07:14.621 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3601_fsm 2024-08-31T06:07:14.621 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3601_vm 2024-08-31T06:07:14.621 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3602 2024-08-31T06:07:14.621 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3602_fsm 2024-08-31T06:07:14.621 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3602_vm 2024-08-31T06:07:14.621 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3603 2024-08-31T06:07:14.621 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3603_fsm 2024-08-31T06:07:14.621 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3603_vm 2024-08-31T06:07:14.621 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3604 2024-08-31T06:07:14.621 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3605 2024-08-31T06:07:14.622 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3606 2024-08-31T06:07:14.622 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3607 2024-08-31T06:07:14.622 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3608 2024-08-31T06:07:14.622 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 3609 2024-08-31T06:07:14.622 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3712 2024-08-31T06:07:14.622 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3764 2024-08-31T06:07:14.622 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3764_fsm 2024-08-31T06:07:14.622 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3764_vm 2024-08-31T06:07:14.622 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3766 2024-08-31T06:07:14.622 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3767 2024-08-31T06:07:14.622 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3997 2024-08-31T06:07:14.623 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4143 2024-08-31T06:07:14.623 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4144 2024-08-31T06:07:14.623 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4145 2024-08-31T06:07:14.623 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4146 2024-08-31T06:07:14.623 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4147 2024-08-31T06:07:14.623 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4148 2024-08-31T06:07:14.623 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4149 2024-08-31T06:07:14.623 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4150 2024-08-31T06:07:14.623 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4151 2024-08-31T06:07:14.623 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4152 2024-08-31T06:07:14.623 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4153 2024-08-31T06:07:14.623 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4154 2024-08-31T06:07:14.624 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4155 2024-08-31T06:07:14.624 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4156 2024-08-31T06:07:14.624 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4157 2024-08-31T06:07:14.624 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4158 2024-08-31T06:07:14.624 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4159 2024-08-31T06:07:14.624 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4160 2024-08-31T06:07:14.624 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4161 2024-08-31T06:07:14.624 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4162 2024-08-31T06:07:14.624 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4163 2024-08-31T06:07:14.624 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4164 2024-08-31T06:07:14.624 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4165 2024-08-31T06:07:14.625 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4166 2024-08-31T06:07:14.625 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4167 2024-08-31T06:07:14.625 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4168 2024-08-31T06:07:14.625 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4169 2024-08-31T06:07:14.625 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4170 2024-08-31T06:07:14.625 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4171 2024-08-31T06:07:14.625 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4172 2024-08-31T06:07:14.625 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4173 2024-08-31T06:07:14.625 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4174 2024-08-31T06:07:14.625 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 5002 2024-08-31T06:07:14.625 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 548 2024-08-31T06:07:14.625 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 549 2024-08-31T06:07:14.626 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 6102 2024-08-31T06:07:14.626 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 6104 2024-08-31T06:07:14.626 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 6106 2024-08-31T06:07:14.626 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 6110 2024-08-31T06:07:14.626 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 6111 2024-08-31T06:07:14.626 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 6112 2024-08-31T06:07:14.626 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 6113 2024-08-31T06:07:14.626 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 6117 2024-08-31T06:07:14.626 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 826 2024-08-31T06:07:14.626 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 827 2024-08-31T06:07:14.626 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 828 2024-08-31T06:07:14.626 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Aug 31 06:07 pg_filenode.map 2024-08-31T06:07:14.626 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 31 06:07 PG_VERSION 2024-08-31T06:07:14.627 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:07:14.627 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-08-31T06:07:14.627 INFO:teuthology.orchestra.run.smithi016.stdout:total 7777 2024-08-31T06:07:14.627 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Aug 31 06:07 . 2024-08-31T06:07:14.627 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 31 06:07 .. 2024-08-31T06:07:14.627 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 112 2024-08-31T06:07:14.627 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 113 2024-08-31T06:07:14.627 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 31 06:07 1247 2024-08-31T06:07:14.627 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 1247_fsm 2024-08-31T06:07:14.627 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 1247_vm 2024-08-31T06:07:14.627 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Aug 31 06:07 1249 2024-08-31T06:07:14.627 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 1249_fsm 2024-08-31T06:07:14.627 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 1249_vm 2024-08-31T06:07:14.628 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Aug 31 06:07 1255 2024-08-31T06:07:14.628 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 1255_fsm 2024-08-31T06:07:14.628 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 1255_vm 2024-08-31T06:07:14.628 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Aug 31 06:07 1259 2024-08-31T06:07:14.628 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 1259_fsm 2024-08-31T06:07:14.628 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 1259_vm 2024-08-31T06:07:14.628 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Aug 31 06:07 13344 2024-08-31T06:07:14.628 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 13344_fsm 2024-08-31T06:07:14.628 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13344_vm 2024-08-31T06:07:14.628 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 13346 2024-08-31T06:07:14.628 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13348 2024-08-31T06:07:14.628 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13349 2024-08-31T06:07:14.629 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 13349_fsm 2024-08-31T06:07:14.629 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13349_vm 2024-08-31T06:07:14.629 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 13351 2024-08-31T06:07:14.629 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13353 2024-08-31T06:07:14.629 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13354 2024-08-31T06:07:14.629 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 13354_fsm 2024-08-31T06:07:14.629 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13354_vm 2024-08-31T06:07:14.629 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 13356 2024-08-31T06:07:14.629 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13358 2024-08-31T06:07:14.629 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13359 2024-08-31T06:07:14.629 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 13359_fsm 2024-08-31T06:07:14.629 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13359_vm 2024-08-31T06:07:14.629 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 13361 2024-08-31T06:07:14.630 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13363 2024-08-31T06:07:14.630 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 1417 2024-08-31T06:07:14.630 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 1418 2024-08-31T06:07:14.630 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 174 2024-08-31T06:07:14.630 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 175 2024-08-31T06:07:14.630 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2187 2024-08-31T06:07:14.630 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2224 2024-08-31T06:07:14.630 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2328 2024-08-31T06:07:14.630 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2336 2024-08-31T06:07:14.630 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2337 2024-08-31T06:07:14.630 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2579 2024-08-31T06:07:14.630 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2600 2024-08-31T06:07:14.630 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2600_fsm 2024-08-31T06:07:14.631 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2600_vm 2024-08-31T06:07:14.631 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2601 2024-08-31T06:07:14.631 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2601_fsm 2024-08-31T06:07:14.631 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2601_vm 2024-08-31T06:07:14.631 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 31 06:07 2602 2024-08-31T06:07:14.631 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2602_fsm 2024-08-31T06:07:14.631 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2602_vm 2024-08-31T06:07:14.631 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 2603 2024-08-31T06:07:14.631 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2603_fsm 2024-08-31T06:07:14.631 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2603_vm 2024-08-31T06:07:14.632 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2604 2024-08-31T06:07:14.632 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2605 2024-08-31T06:07:14.632 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2605_fsm 2024-08-31T06:07:14.632 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2605_vm 2024-08-31T06:07:14.632 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2606 2024-08-31T06:07:14.632 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2606_fsm 2024-08-31T06:07:14.632 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2606_vm 2024-08-31T06:07:14.632 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2607 2024-08-31T06:07:14.632 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2607_fsm 2024-08-31T06:07:14.632 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2607_vm 2024-08-31T06:07:14.632 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Aug 31 06:07 2608 2024-08-31T06:07:14.632 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2608_fsm 2024-08-31T06:07:14.633 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2608_vm 2024-08-31T06:07:14.633 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Aug 31 06:07 2609 2024-08-31T06:07:14.633 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2609_fsm 2024-08-31T06:07:14.633 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2609_vm 2024-08-31T06:07:14.633 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 2610 2024-08-31T06:07:14.633 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2610_fsm 2024-08-31T06:07:14.633 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2610_vm 2024-08-31T06:07:14.633 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2611 2024-08-31T06:07:14.633 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2612 2024-08-31T06:07:14.633 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2612_fsm 2024-08-31T06:07:14.633 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2612_vm 2024-08-31T06:07:14.633 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2613 2024-08-31T06:07:14.634 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2615 2024-08-31T06:07:14.634 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2615_fsm 2024-08-31T06:07:14.634 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2615_vm 2024-08-31T06:07:14.634 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2616 2024-08-31T06:07:14.634 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2616_fsm 2024-08-31T06:07:14.634 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2616_vm 2024-08-31T06:07:14.634 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Aug 31 06:07 2617 2024-08-31T06:07:14.634 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2617_fsm 2024-08-31T06:07:14.634 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2617_vm 2024-08-31T06:07:14.634 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Aug 31 06:07 2618 2024-08-31T06:07:14.634 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2618_fsm 2024-08-31T06:07:14.634 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2618_vm 2024-08-31T06:07:14.634 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Aug 31 06:07 2619 2024-08-31T06:07:14.635 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2619_fsm 2024-08-31T06:07:14.635 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2619_vm 2024-08-31T06:07:14.635 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2620 2024-08-31T06:07:14.635 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2650 2024-08-31T06:07:14.635 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2651 2024-08-31T06:07:14.635 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2652 2024-08-31T06:07:14.635 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 31 06:07 2653 2024-08-31T06:07:14.635 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 31 06:07 2654 2024-08-31T06:07:14.635 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 2655 2024-08-31T06:07:14.635 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2656 2024-08-31T06:07:14.635 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2657 2024-08-31T06:07:14.635 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Aug 31 06:07 2658 2024-08-31T06:07:14.635 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 31 06:07 2659 2024-08-31T06:07:14.636 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2660 2024-08-31T06:07:14.636 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2661 2024-08-31T06:07:14.636 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 2662 2024-08-31T06:07:14.636 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 31 06:07 2663 2024-08-31T06:07:14.636 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2664 2024-08-31T06:07:14.636 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2665 2024-08-31T06:07:14.636 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2666 2024-08-31T06:07:14.636 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2667 2024-08-31T06:07:14.636 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2668 2024-08-31T06:07:14.636 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2669 2024-08-31T06:07:14.636 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2670 2024-08-31T06:07:14.636 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Aug 31 06:07 2673 2024-08-31T06:07:14.637 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Aug 31 06:07 2674 2024-08-31T06:07:14.637 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Aug 31 06:07 2675 2024-08-31T06:07:14.637 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2678 2024-08-31T06:07:14.637 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2679 2024-08-31T06:07:14.637 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2680 2024-08-31T06:07:14.637 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2681 2024-08-31T06:07:14.637 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2682 2024-08-31T06:07:14.637 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2683 2024-08-31T06:07:14.637 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2684 2024-08-31T06:07:14.637 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2685 2024-08-31T06:07:14.637 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2686 2024-08-31T06:07:14.638 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2687 2024-08-31T06:07:14.638 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 31 06:07 2688 2024-08-31T06:07:14.638 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 31 06:07 2689 2024-08-31T06:07:14.638 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Aug 31 06:07 2690 2024-08-31T06:07:14.638 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Aug 31 06:07 2691 2024-08-31T06:07:14.638 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2692 2024-08-31T06:07:14.638 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2693 2024-08-31T06:07:14.638 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2696 2024-08-31T06:07:14.638 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2699 2024-08-31T06:07:14.638 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2701 2024-08-31T06:07:14.638 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2702 2024-08-31T06:07:14.639 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 2703 2024-08-31T06:07:14.639 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 31 06:07 2704 2024-08-31T06:07:14.639 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2753 2024-08-31T06:07:14.639 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2753_fsm 2024-08-31T06:07:14.639 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2753_vm 2024-08-31T06:07:14.639 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2754 2024-08-31T06:07:14.639 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2755 2024-08-31T06:07:14.639 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 2756 2024-08-31T06:07:14.639 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 2757 2024-08-31T06:07:14.639 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2830 2024-08-31T06:07:14.639 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2831 2024-08-31T06:07:14.639 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2832 2024-08-31T06:07:14.640 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2833 2024-08-31T06:07:14.640 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2834 2024-08-31T06:07:14.640 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2835 2024-08-31T06:07:14.640 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2836 2024-08-31T06:07:14.640 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2837 2024-08-31T06:07:14.640 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Aug 31 06:07 2838 2024-08-31T06:07:14.640 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2838_fsm 2024-08-31T06:07:14.640 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2838_vm 2024-08-31T06:07:14.640 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2839 2024-08-31T06:07:14.640 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2840 2024-08-31T06:07:14.640 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2840_fsm 2024-08-31T06:07:14.640 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2840_vm 2024-08-31T06:07:14.640 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2841 2024-08-31T06:07:14.641 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2995 2024-08-31T06:07:14.641 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2996 2024-08-31T06:07:14.641 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3079 2024-08-31T06:07:14.641 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3079_fsm 2024-08-31T06:07:14.641 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3079_vm 2024-08-31T06:07:14.641 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3080 2024-08-31T06:07:14.641 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3081 2024-08-31T06:07:14.641 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 31 06:07 3085 2024-08-31T06:07:14.641 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3118 2024-08-31T06:07:14.641 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3119 2024-08-31T06:07:14.641 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Aug 31 06:07 3164 2024-08-31T06:07:14.641 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3256 2024-08-31T06:07:14.642 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3257 2024-08-31T06:07:14.642 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3258 2024-08-31T06:07:14.642 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3350 2024-08-31T06:07:14.642 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3351 2024-08-31T06:07:14.642 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3379 2024-08-31T06:07:14.642 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3380 2024-08-31T06:07:14.642 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3381 2024-08-31T06:07:14.642 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3394 2024-08-31T06:07:14.642 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3394_fsm 2024-08-31T06:07:14.642 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3394_vm 2024-08-31T06:07:14.642 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3395 2024-08-31T06:07:14.642 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3429 2024-08-31T06:07:14.643 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3430 2024-08-31T06:07:14.643 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3431 2024-08-31T06:07:14.643 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3433 2024-08-31T06:07:14.643 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3439 2024-08-31T06:07:14.643 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3440 2024-08-31T06:07:14.643 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 31 06:07 3455 2024-08-31T06:07:14.643 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Aug 31 06:07 3456 2024-08-31T06:07:14.643 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3456_fsm 2024-08-31T06:07:14.643 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3456_vm 2024-08-31T06:07:14.643 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3466 2024-08-31T06:07:14.644 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3467 2024-08-31T06:07:14.644 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3468 2024-08-31T06:07:14.644 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3501 2024-08-31T06:07:14.644 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3502 2024-08-31T06:07:14.644 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3503 2024-08-31T06:07:14.644 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3534 2024-08-31T06:07:14.644 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3541 2024-08-31T06:07:14.644 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3541_fsm 2024-08-31T06:07:14.644 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3541_vm 2024-08-31T06:07:14.644 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3542 2024-08-31T06:07:14.644 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3574 2024-08-31T06:07:14.644 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3575 2024-08-31T06:07:14.645 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3576 2024-08-31T06:07:14.645 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3596 2024-08-31T06:07:14.645 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3597 2024-08-31T06:07:14.645 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3598 2024-08-31T06:07:14.645 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3599 2024-08-31T06:07:14.645 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3600 2024-08-31T06:07:14.645 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3600_fsm 2024-08-31T06:07:14.645 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3600_vm 2024-08-31T06:07:14.645 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3601 2024-08-31T06:07:14.645 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3601_fsm 2024-08-31T06:07:14.645 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3601_vm 2024-08-31T06:07:14.645 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3602 2024-08-31T06:07:14.646 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3602_fsm 2024-08-31T06:07:14.646 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3602_vm 2024-08-31T06:07:14.646 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3603 2024-08-31T06:07:14.646 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3603_fsm 2024-08-31T06:07:14.646 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3603_vm 2024-08-31T06:07:14.646 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3604 2024-08-31T06:07:14.646 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3605 2024-08-31T06:07:14.646 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3606 2024-08-31T06:07:14.646 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3607 2024-08-31T06:07:14.646 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3608 2024-08-31T06:07:14.646 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 3609 2024-08-31T06:07:14.646 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3712 2024-08-31T06:07:14.647 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3764 2024-08-31T06:07:14.647 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3764_fsm 2024-08-31T06:07:14.647 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3764_vm 2024-08-31T06:07:14.647 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3766 2024-08-31T06:07:14.647 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3767 2024-08-31T06:07:14.647 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3997 2024-08-31T06:07:14.647 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4143 2024-08-31T06:07:14.647 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4144 2024-08-31T06:07:14.647 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4145 2024-08-31T06:07:14.647 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4146 2024-08-31T06:07:14.647 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4147 2024-08-31T06:07:14.647 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4148 2024-08-31T06:07:14.648 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4149 2024-08-31T06:07:14.648 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4150 2024-08-31T06:07:14.648 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4151 2024-08-31T06:07:14.648 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4152 2024-08-31T06:07:14.648 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4153 2024-08-31T06:07:14.648 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4154 2024-08-31T06:07:14.648 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4155 2024-08-31T06:07:14.648 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4156 2024-08-31T06:07:14.648 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4157 2024-08-31T06:07:14.648 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4158 2024-08-31T06:07:14.648 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4159 2024-08-31T06:07:14.649 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4160 2024-08-31T06:07:14.649 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4161 2024-08-31T06:07:14.649 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4162 2024-08-31T06:07:14.649 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4163 2024-08-31T06:07:14.649 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4164 2024-08-31T06:07:14.649 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4165 2024-08-31T06:07:14.649 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4166 2024-08-31T06:07:14.649 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4167 2024-08-31T06:07:14.649 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4168 2024-08-31T06:07:14.649 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4169 2024-08-31T06:07:14.649 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4170 2024-08-31T06:07:14.650 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4171 2024-08-31T06:07:14.650 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4172 2024-08-31T06:07:14.650 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4173 2024-08-31T06:07:14.650 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4174 2024-08-31T06:07:14.650 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 5002 2024-08-31T06:07:14.650 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 548 2024-08-31T06:07:14.650 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 549 2024-08-31T06:07:14.650 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 6102 2024-08-31T06:07:14.650 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 6104 2024-08-31T06:07:14.650 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 6106 2024-08-31T06:07:14.650 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 6110 2024-08-31T06:07:14.650 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 6111 2024-08-31T06:07:14.651 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 6112 2024-08-31T06:07:14.651 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 6113 2024-08-31T06:07:14.651 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 6117 2024-08-31T06:07:14.651 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 826 2024-08-31T06:07:14.651 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 827 2024-08-31T06:07:14.651 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 828 2024-08-31T06:07:14.651 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Aug 31 06:07 pg_filenode.map 2024-08-31T06:07:14.651 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 31 06:07 PG_VERSION 2024-08-31T06:07:14.651 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:07:14.651 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/global: 2024-08-31T06:07:14.651 INFO:teuthology.orchestra.run.smithi016.stdout:total 537 2024-08-31T06:07:14.652 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 Aug 31 06:07 . 2024-08-31T06:07:14.652 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 31 06:07 .. 2024-08-31T06:07:14.652 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 1213 2024-08-31T06:07:14.652 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 1213_fsm 2024-08-31T06:07:14.652 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 1213_vm 2024-08-31T06:07:14.652 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 1214 2024-08-31T06:07:14.652 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 1214_fsm 2024-08-31T06:07:14.652 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 1214_vm 2024-08-31T06:07:14.652 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 1232 2024-08-31T06:07:14.652 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 1233 2024-08-31T06:07:14.652 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 1260 2024-08-31T06:07:14.652 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 1260_fsm 2024-08-31T06:07:14.653 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 1260_vm 2024-08-31T06:07:14.653 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 1261 2024-08-31T06:07:14.653 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 1261_fsm 2024-08-31T06:07:14.653 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 1261_vm 2024-08-31T06:07:14.653 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 1262 2024-08-31T06:07:14.653 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 1262_fsm 2024-08-31T06:07:14.653 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 1262_vm 2024-08-31T06:07:14.653 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2396 2024-08-31T06:07:14.653 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2396_fsm 2024-08-31T06:07:14.653 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2396_vm 2024-08-31T06:07:14.654 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2397 2024-08-31T06:07:14.654 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2671 2024-08-31T06:07:14.654 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2672 2024-08-31T06:07:14.654 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2676 2024-08-31T06:07:14.654 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2677 2024-08-31T06:07:14.654 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2694 2024-08-31T06:07:14.654 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2695 2024-08-31T06:07:14.654 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2697 2024-08-31T06:07:14.654 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2698 2024-08-31T06:07:14.654 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2846 2024-08-31T06:07:14.655 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2847 2024-08-31T06:07:14.655 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2964 2024-08-31T06:07:14.655 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2965 2024-08-31T06:07:14.655 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2966 2024-08-31T06:07:14.655 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2967 2024-08-31T06:07:14.655 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3592 2024-08-31T06:07:14.655 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3593 2024-08-31T06:07:14.655 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4060 2024-08-31T06:07:14.655 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4061 2024-08-31T06:07:14.655 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4175 2024-08-31T06:07:14.655 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4176 2024-08-31T06:07:14.656 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4177 2024-08-31T06:07:14.656 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4178 2024-08-31T06:07:14.656 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4181 2024-08-31T06:07:14.656 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4182 2024-08-31T06:07:14.656 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4183 2024-08-31T06:07:14.656 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4184 2024-08-31T06:07:14.656 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4185 2024-08-31T06:07:14.656 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4186 2024-08-31T06:07:14.656 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 6000 2024-08-31T06:07:14.656 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 6001 2024-08-31T06:07:14.656 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 6002 2024-08-31T06:07:14.657 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 6100 2024-08-31T06:07:14.657 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 6114 2024-08-31T06:07:14.657 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 6115 2024-08-31T06:07:14.657 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 pg_control 2024-08-31T06:07:14.657 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Aug 31 06:07 pg_filenode.map 2024-08-31T06:07:14.657 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:07:14.657 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/log: 2024-08-31T06:07:14.657 INFO:teuthology.orchestra.run.smithi016.stdout:total 0 2024-08-31T06:07:14.657 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 . 2024-08-31T06:07:14.657 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 31 06:07 .. 2024-08-31T06:07:14.657 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:07:14.657 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-08-31T06:07:14.658 INFO:teuthology.orchestra.run.smithi016.stdout:total 0 2024-08-31T06:07:14.658 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 . 2024-08-31T06:07:14.658 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 31 06:07 .. 2024-08-31T06:07:14.658 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:07:14.658 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-08-31T06:07:14.658 INFO:teuthology.orchestra.run.smithi016.stdout:total 0 2024-08-31T06:07:14.658 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 . 2024-08-31T06:07:14.658 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 31 06:07 .. 2024-08-31T06:07:14.658 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:07:14.658 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-08-31T06:07:14.658 INFO:teuthology.orchestra.run.smithi016.stdout:total 1 2024-08-31T06:07:14.659 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 31 06:07 . 2024-08-31T06:07:14.659 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 31 06:07 .. 2024-08-31T06:07:14.659 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 mappings 2024-08-31T06:07:14.659 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Aug 31 06:07 replorigin_checkpoint 2024-08-31T06:07:14.659 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 snapshots 2024-08-31T06:07:14.659 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:07:14.659 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-08-31T06:07:14.659 INFO:teuthology.orchestra.run.smithi016.stdout:total 0 2024-08-31T06:07:14.659 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 . 2024-08-31T06:07:14.659 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 31 06:07 .. 2024-08-31T06:07:14.659 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:07:14.659 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-08-31T06:07:14.659 INFO:teuthology.orchestra.run.smithi016.stdout:total 0 2024-08-31T06:07:14.660 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 . 2024-08-31T06:07:14.660 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 31 06:07 .. 2024-08-31T06:07:14.660 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:07:14.660 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-08-31T06:07:14.660 INFO:teuthology.orchestra.run.smithi016.stdout:total 0 2024-08-31T06:07:14.660 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 31 06:07 . 2024-08-31T06:07:14.660 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 31 06:07 .. 2024-08-31T06:07:14.660 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 31 06:07 members 2024-08-31T06:07:14.660 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 31 06:07 offsets 2024-08-31T06:07:14.660 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:07:14.660 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-08-31T06:07:14.661 INFO:teuthology.orchestra.run.smithi016.stdout:total 8 2024-08-31T06:07:14.661 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 31 06:07 . 2024-08-31T06:07:14.662 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 31 06:07 .. 2024-08-31T06:07:14.662 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 0000 2024-08-31T06:07:14.662 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:07:14.662 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-08-31T06:07:14.662 INFO:teuthology.orchestra.run.smithi016.stdout:total 8 2024-08-31T06:07:14.662 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 31 06:07 . 2024-08-31T06:07:14.663 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 31 06:07 .. 2024-08-31T06:07:14.663 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 0000 2024-08-31T06:07:14.663 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:07:14.663 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-08-31T06:07:14.663 INFO:teuthology.orchestra.run.smithi016.stdout:total 0 2024-08-31T06:07:14.663 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 . 2024-08-31T06:07:14.663 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 31 06:07 .. 2024-08-31T06:07:14.663 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:07:14.663 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-08-31T06:07:14.663 INFO:teuthology.orchestra.run.smithi016.stdout:total 0 2024-08-31T06:07:14.663 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 . 2024-08-31T06:07:14.663 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 31 06:07 .. 2024-08-31T06:07:14.664 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:07:14.664 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-08-31T06:07:14.664 INFO:teuthology.orchestra.run.smithi016.stdout:total 0 2024-08-31T06:07:14.664 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 . 2024-08-31T06:07:14.664 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 31 06:07 .. 2024-08-31T06:07:14.664 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:07:14.664 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-08-31T06:07:14.664 INFO:teuthology.orchestra.run.smithi016.stdout:total 0 2024-08-31T06:07:14.664 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 . 2024-08-31T06:07:14.664 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 31 06:07 .. 2024-08-31T06:07:14.664 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:07:14.664 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-08-31T06:07:14.664 INFO:teuthology.orchestra.run.smithi016.stdout:total 0 2024-08-31T06:07:14.665 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 . 2024-08-31T06:07:14.665 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 31 06:07 .. 2024-08-31T06:07:14.665 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:07:14.665 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-08-31T06:07:14.665 INFO:teuthology.orchestra.run.smithi016.stdout:total 0 2024-08-31T06:07:14.665 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 . 2024-08-31T06:07:14.665 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 31 06:07 .. 2024-08-31T06:07:14.665 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:07:14.665 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-08-31T06:07:14.665 INFO:teuthology.orchestra.run.smithi016.stdout:total 8 2024-08-31T06:07:14.665 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 31 06:07 . 2024-08-31T06:07:14.665 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 31 06:07 .. 2024-08-31T06:07:14.665 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 0000 2024-08-31T06:07:14.666 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:07:14.666 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-08-31T06:07:14.666 INFO:teuthology.orchestra.run.smithi016.stdout:total 0 2024-08-31T06:07:14.666 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 . 2024-08-31T06:07:14.666 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 31 06:07 .. 2024-08-31T06:07:14.666 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:07:14.666 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-08-31T06:07:14.666 INFO:teuthology.orchestra.run.smithi016.stdout:total 0 2024-08-31T06:07:14.666 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 . 2024-08-31T06:07:14.666 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 31 06:07 .. 2024-08-31T06:07:14.666 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:07:14.666 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-08-31T06:07:14.666 INFO:teuthology.orchestra.run.smithi016.stdout:total 16384 2024-08-31T06:07:14.666 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 31 06:07 . 2024-08-31T06:07:14.667 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 31 06:07 .. 2024-08-31T06:07:14.667 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:07 000000010000000000000001 2024-08-31T06:07:14.667 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 archive_status 2024-08-31T06:07:14.667 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:07:14.667 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-08-31T06:07:14.667 INFO:teuthology.orchestra.run.smithi016.stdout:total 0 2024-08-31T06:07:14.667 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 . 2024-08-31T06:07:14.667 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 31 06:07 .. 2024-08-31T06:07:14.667 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:07:14.667 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-08-31T06:07:14.667 INFO:teuthology.orchestra.run.smithi016.stdout:total 8 2024-08-31T06:07:14.667 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 31 06:07 . 2024-08-31T06:07:14.668 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 31 06:07 .. 2024-08-31T06:07:14.668 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 0000 2024-08-31T06:07:14.668 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2024-08-31T06:07:14.941 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2024-08-31T06:07:14.991 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:14 smithi016 ceph-mon[17194]: pgmap v363: 97 pgs: 97 active+clean; 15 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.2 MiB/s wr, 31 op/s 2024-08-31T06:07:15.034 INFO:teuthology.orchestra.run.smithi016.stderr:dropping old tables... 2024-08-31T06:07:15.035 INFO:teuthology.orchestra.run.smithi016.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2024-08-31T06:07:15.035 INFO:teuthology.orchestra.run.smithi016.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2024-08-31T06:07:15.035 INFO:teuthology.orchestra.run.smithi016.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2024-08-31T06:07:15.035 INFO:teuthology.orchestra.run.smithi016.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2024-08-31T06:07:15.035 INFO:teuthology.orchestra.run.smithi016.stderr:creating tables... 2024-08-31T06:07:15.051 INFO:teuthology.orchestra.run.smithi016.stderr:generating data (client-side)... 2024-08-31T06:07:15.185 INFO:teuthology.orchestra.run.smithi016.stderr:100000 of 3200000 tuples (3%) done (elapsed 0.11 s, remaining 3.32 s) 2024-08-31T06:07:15.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:14 smithi184 ceph-mon[22331]: pgmap v363: 97 pgs: 97 active+clean; 15 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.2 MiB/s wr, 31 op/s 2024-08-31T06:07:15.334 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-08-31T06:07:15.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:14 smithi155 ceph-mon[21742]: pgmap v363: 97 pgs: 97 active+clean; 15 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.2 MiB/s wr, 31 op/s 2024-08-31T06:07:15.506 INFO:teuthology.orchestra.run.smithi016.stderr:200000 of 3200000 tuples (6%) done (elapsed 0.43 s, remaining 6.43 s) 2024-08-31T06:07:15.791 INFO:teuthology.orchestra.run.smithi016.stderr:300000 of 3200000 tuples (9%) done (elapsed 0.71 s, remaining 6.90 s) 2024-08-31T06:07:15.956 INFO:teuthology.orchestra.run.smithi016.stderr:400000 of 3200000 tuples (12%) done (elapsed 0.88 s, remaining 6.15 s) 2024-08-31T06:07:16.164 INFO:teuthology.orchestra.run.smithi016.stderr:500000 of 3200000 tuples (15%) done (elapsed 1.09 s, remaining 5.87 s) 2024-08-31T06:07:16.297 INFO:teuthology.orchestra.run.smithi016.stderr:600000 of 3200000 tuples (18%) done (elapsed 1.22 s, remaining 5.28 s) 2024-08-31T06:07:16.615 INFO:teuthology.orchestra.run.smithi016.stderr:700000 of 3200000 tuples (21%) done (elapsed 1.54 s, remaining 5.49 s) 2024-08-31T06:07:16.956 INFO:teuthology.orchestra.run.smithi016.stderr:800000 of 3200000 tuples (25%) done (elapsed 1.88 s, remaining 5.63 s) 2024-08-31T06:07:17.157 INFO:teuthology.orchestra.run.smithi016.stderr:900000 of 3200000 tuples (28%) done (elapsed 2.08 s, remaining 5.31 s) 2024-08-31T06:07:17.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:16 smithi184 ceph-mon[22331]: pgmap v364: 97 pgs: 97 active+clean; 36 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 2.9 MiB/s wr, 116 op/s 2024-08-31T06:07:17.271 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:16 smithi016 ceph-mon[17194]: pgmap v364: 97 pgs: 97 active+clean; 36 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 2.9 MiB/s wr, 116 op/s 2024-08-31T06:07:17.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:16 smithi155 ceph-mon[21742]: pgmap v364: 97 pgs: 97 active+clean; 36 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 2.9 MiB/s wr, 116 op/s 2024-08-31T06:07:17.459 INFO:teuthology.orchestra.run.smithi016.stderr:1000000 of 3200000 tuples (31%) done (elapsed 2.38 s, remaining 5.24 s) 2024-08-31T06:07:17.730 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.728+0000 7fac58400640 1 -- 172.21.15.16:0/3555481311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac501046d0 msgr2=0x7fac50104ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:17.730 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.728+0000 7fac58400640 1 --2- 172.21.15.16:0/3555481311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac501046d0 0x7fac50104ae0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fac40009ca0 tx=0x7fac4002a8b0 comp rx=0 tx=0).stop 2024-08-31T06:07:17.730 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.728+0000 7fac58400640 1 -- 172.21.15.16:0/3555481311 shutdown_connections 2024-08-31T06:07:17.730 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.728+0000 7fac58400640 1 --2- 172.21.15.16:0/3555481311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac5010db30 0x7fac50110020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:17.731 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.728+0000 7fac58400640 1 --2- 172.21.15.16:0/3555481311 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac50105020 0x7fac5010d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:17.731 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.728+0000 7fac58400640 1 --2- 172.21.15.16:0/3555481311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac501046d0 0x7fac50104ae0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:17.731 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.728+0000 7fac58400640 1 -- 172.21.15.16:0/3555481311 >> 172.21.15.16:0/3555481311 conn(0x7fac500fe160 msgr2=0x7fac501005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:17.731 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.728+0000 7fac58400640 1 -- 172.21.15.16:0/3555481311 shutdown_connections 2024-08-31T06:07:17.731 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.728+0000 7fac58400640 1 -- 172.21.15.16:0/3555481311 wait complete. 2024-08-31T06:07:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.729+0000 7fac58400640 1 Processor -- start 2024-08-31T06:07:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.729+0000 7fac58400640 1 -- start start 2024-08-31T06:07:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.729+0000 7fac58400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac501046d0 0x7fac5019c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.729+0000 7fac58400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac50105020 0x7fac5019ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.729+0000 7fac58400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac5010db30 0x7fac501a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.729+0000 7fac58400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fac501137a0 con 0x7fac5010db30 2024-08-31T06:07:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.729+0000 7fac58400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fac50113620 con 0x7fac50105020 2024-08-31T06:07:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.729+0000 7fac58400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fac50113920 con 0x7fac501046d0 2024-08-31T06:07:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.730+0000 7fac55a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac501046d0 0x7fac5019c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.730+0000 7fac55a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac501046d0 0x7fac5019c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:32878/0 (socket says 172.21.15.16:32878) 2024-08-31T06:07:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.730+0000 7fac55a00640 1 -- 172.21.15.16:0/3150120110 learned_addr learned my addr 172.21.15.16:0/3150120110 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.730+0000 7fac56400640 1 --2- 172.21.15.16:0/3150120110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac5010db30 0x7fac501a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.730+0000 7fac55000640 1 --2- 172.21.15.16:0/3150120110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac50105020 0x7fac5019ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.730+0000 7fac56400640 1 -- 172.21.15.16:0/3150120110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac501046d0 msgr2=0x7fac5019c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.730+0000 7fac56400640 1 --2- 172.21.15.16:0/3150120110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac501046d0 0x7fac5019c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.730+0000 7fac56400640 1 -- 172.21.15.16:0/3150120110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac50105020 msgr2=0x7fac5019ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.730+0000 7fac56400640 1 --2- 172.21.15.16:0/3150120110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac50105020 0x7fac5019ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.730+0000 7fac56400640 1 -- 172.21.15.16:0/3150120110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac501a2300 con 0x7fac5010db30 2024-08-31T06:07:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.730+0000 7fac55a00640 1 --2- 172.21.15.16:0/3150120110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac501046d0 0x7fac5019c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:07:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.730+0000 7fac56400640 1 --2- 172.21.15.16:0/3150120110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac5010db30 0x7fac501a1c90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fac4800e970 tx=0x7fac4800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.731+0000 7fac4e000640 1 -- 172.21.15.16:0/3150120110 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4800ccc0 con 0x7fac5010db30 2024-08-31T06:07:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.731+0000 7fac58400640 1 -- 172.21.15.16:0/3150120110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac501a25f0 con 0x7fac5010db30 2024-08-31T06:07:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.731+0000 7fac4e000640 1 -- 172.21.15.16:0/3150120110 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac48005020 con 0x7fac5010db30 2024-08-31T06:07:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.731+0000 7fac4e000640 1 -- 172.21.15.16:0/3150120110 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac48004a00 con 0x7fac5010db30 2024-08-31T06:07:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.731+0000 7fac58400640 1 -- 172.21.15.16:0/3150120110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac501a8e10 con 0x7fac5010db30 2024-08-31T06:07:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.734+0000 7fac5f5da740 1 Processor -- start 2024-08-31T06:07:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.734+0000 7fac4e000640 1 -- 172.21.15.16:0/3150120110 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fac48004ba0 con 0x7fac5010db30 2024-08-31T06:07:17.737 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.734+0000 7fac5f5da740 1 -- start start 2024-08-31T06:07:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.735+0000 7fac5f5da740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d399ddc1b0 0x55d39a17f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.735+0000 7fac5f5da740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d399db32f0 0x55d39a17fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.735+0000 7fac5f5da740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d399dbb4d0 0x55d39a1803a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.735+0000 7fac5f5da740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d39a0f4f70 con 0x55d399dbb4d0 2024-08-31T06:07:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.735+0000 7fac5f5da740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d39a0e4500 con 0x55d399db32f0 2024-08-31T06:07:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.735+0000 7fac5f5da740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d39a0e84b0 con 0x55d399ddc1b0 2024-08-31T06:07:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.735+0000 7fac55a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d399db32f0 0x55d39a17fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.735+0000 7fac55000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d399ddc1b0 0x55d39a17f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.735+0000 7fac55000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d399ddc1b0 0x55d39a17f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:32882/0 (socket says 172.21.15.16:32882) 2024-08-31T06:07:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.735+0000 7fac55000640 1 -- 172.21.15.16:0/119097786 learned_addr learned my addr 172.21.15.16:0/119097786 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.735+0000 7fac55000640 1 --2- 172.21.15.16:0/119097786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d399dbb4d0 0x55d39a1803a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:17.739 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.736+0000 7fac55a00640 1 -- 172.21.15.16:0/119097786 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d399ddc1b0 msgr2=0x55d39a17f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:17.739 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.736+0000 7fac55a00640 1 --2- 172.21.15.16:0/119097786 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d399ddc1b0 0x55d39a17f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:17.739 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.736+0000 7fac55a00640 1 -- 172.21.15.16:0/119097786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d399dbb4d0 msgr2=0x55d39a1803a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:17.739 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.736+0000 7fac55a00640 1 --2- 172.21.15.16:0/119097786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d399dbb4d0 0x55d39a1803a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:17.739 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.736+0000 7fac55a00640 1 -- 172.21.15.16:0/119097786 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d39a09ec10 con 0x55d399db32f0 2024-08-31T06:07:17.740 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.736+0000 7fac55a00640 1 --2- 172.21.15.16:0/119097786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d399db32f0 0x55d39a17fe60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fac4002add0 tx=0x7fac400099d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68c6dbea8c064501 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:17.741 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.736+0000 7fac46a00640 1 -- 172.21.15.16:0/119097786 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac40030bc0 con 0x55d399db32f0 2024-08-31T06:07:17.741 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.736+0000 7fac46a00640 1 -- 172.21.15.16:0/119097786 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac40036070 con 0x55d399db32f0 2024-08-31T06:07:17.741 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.736+0000 7fac46a00640 1 -- 172.21.15.16:0/119097786 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac40004860 con 0x55d399db32f0 2024-08-31T06:07:17.741 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.737+0000 7fac4e000640 1 --2- 172.21.15.16:0/3150120110 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fac340b4ca0 0x7fac340b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:17.741 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.737+0000 7fac4e000640 1 -- 172.21.15.16:0/3150120110 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7fac480e5510 con 0x7fac5010db30 2024-08-31T06:07:17.741 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.737+0000 7fac55000640 1 --2- 172.21.15.16:0/3150120110 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fac340b4ca0 0x7fac340b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:17.741 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.737+0000 7fac5f5da740 1 -- 172.21.15.16:0/119097786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d399db32f0 msgr2=0x55d39a17fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:17.741 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.737+0000 7fac5f5da740 1 --2- 172.21.15.16:0/119097786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d399db32f0 0x55d39a17fe60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fac4002add0 tx=0x7fac400099d0 comp rx=0 tx=0).stop 2024-08-31T06:07:17.741 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.737+0000 7fac5f5da740 1 -- 172.21.15.16:0/119097786 shutdown_connections 2024-08-31T06:07:17.741 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.737+0000 7fac5f5da740 1 --2- 172.21.15.16:0/119097786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d399dbb4d0 0x55d39a1803a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:17.741 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.737+0000 7fac5f5da740 1 --2- 172.21.15.16:0/119097786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d399db32f0 0x55d39a17fe60 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:17.741 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.737+0000 7fac5f5da740 1 --2- 172.21.15.16:0/119097786 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d399ddc1b0 0x55d39a17f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:17.741 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.737+0000 7fac5f5da740 1 -- 172.21.15.16:0/119097786 >> 172.21.15.16:0/119097786 conn(0x55d399dd0300 msgr2=0x55d39a17d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:17.741 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.737+0000 7fac5f5da740 1 -- 172.21.15.16:0/119097786 shutdown_connections 2024-08-31T06:07:17.742 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.737+0000 7fac5f5da740 1 -- 172.21.15.16:0/119097786 wait complete. 2024-08-31T06:07:17.742 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.738+0000 7fac55000640 1 --2- 172.21.15.16:0/3150120110 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fac340b4ca0 0x7fac340b71b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fac3c00bfd0 tx=0x7fac3c015040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:17.742 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.738+0000 7fac5f5da740 1 -- start start 2024-08-31T06:07:17.742 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.738+0000 7fac5f5da740 1 Processor -- start 2024-08-31T06:07:17.744 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.739+0000 7fac5f5da740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d399db32f0 0x55d39a18ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:17.744 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.739+0000 7fac5f5da740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d399dbb4d0 0x55d39a18b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:17.744 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.739+0000 7fac5f5da740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d399ddc1b0 0x55d39a18b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:17.744 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.739+0000 7fac5f5da740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d39a01dab0 con 0x55d399db32f0 2024-08-31T06:07:17.744 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.739+0000 7fac5f5da740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d39a08b320 con 0x55d399dbb4d0 2024-08-31T06:07:17.744 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.739+0000 7fac5f5da740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d39a18e040 con 0x55d399ddc1b0 2024-08-31T06:07:17.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.739+0000 7fac5f5da740 10 client.0 _ll_register_callbacks cb 0x55d39a0916a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:07:17.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.739+0000 7fac55000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d399dbb4d0 0x55d39a18b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:17.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.739+0000 7fac56400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d399ddc1b0 0x55d39a18b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:17.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.739+0000 7fac55000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d399dbb4d0 0x55d39a18b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:56456/0 (socket says 172.21.15.16:56456) 2024-08-31T06:07:17.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.739+0000 7fac55000640 1 -- 172.21.15.16:0/217869411 learned_addr learned my addr 172.21.15.16:0/217869411 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:17.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.739+0000 7fac55a00640 1 --2- 172.21.15.16:0/217869411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d399db32f0 0x55d39a18ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:17.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.740+0000 7fac56400640 1 -- 172.21.15.16:0/217869411 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d399dbb4d0 msgr2=0x55d39a18b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:17.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.740+0000 7fac56400640 1 --2- 172.21.15.16:0/217869411 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d399dbb4d0 0x55d39a18b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:17.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.740+0000 7fac56400640 1 -- 172.21.15.16:0/217869411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d399db32f0 msgr2=0x55d39a18ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:17.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.740+0000 7fac56400640 1 --2- 172.21.15.16:0/217869411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d399db32f0 0x55d39a18ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:17.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.740+0000 7fac56400640 1 -- 172.21.15.16:0/217869411 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d399e47790 con 0x55d399ddc1b0 2024-08-31T06:07:17.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.740+0000 7fac56400640 1 --2- 172.21.15.16:0/217869411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d399ddc1b0 0x55d39a18b8d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fac480e2c60 tx=0x7fac480ab710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8edda41d370ea578 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:17.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.740+0000 7fac3b400640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:07:17.746 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.740+0000 7fac3b400640 1 -- 172.21.15.16:0/217869411 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac480b0070 con 0x55d399ddc1b0 2024-08-31T06:07:17.746 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.740+0000 7fac5f5da740 1 -- 172.21.15.16:0/217869411 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac40036070 con 0x55d399ddc1b0 2024-08-31T06:07:17.746 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.740+0000 7fac3b400640 1 -- 172.21.15.16:0/217869411 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7fac480b0210 con 0x55d399ddc1b0 2024-08-31T06:07:17.746 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.740+0000 7fac3b400640 1 -- 172.21.15.16:0/217869411 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fac4801c040 con 0x55d399ddc1b0 2024-08-31T06:07:17.746 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.740+0000 7fac5f5da740 10 client.25501 fetch_fsmap learned FSMap version 25 2024-08-31T06:07:17.746 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.740+0000 7fac5f5da740 1 -- 172.21.15.16:0/217869411 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d399ec0620 con 0x55d399ddc1b0 2024-08-31T06:07:17.746 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.740+0000 7fac3b400640 1 -- 172.21.15.16:0/217869411 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac40036070 con 0x55d399ddc1b0 2024-08-31T06:07:17.746 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.741+0000 7fac3b400640 1 -- 172.21.15.16:0/217869411 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fac480cece0 con 0x55d399ddc1b0 2024-08-31T06:07:17.746 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.741+0000 7fac5f5da740 10 client.25501 fetch_fsmap finished waiting for FSMap version 25 2024-08-31T06:07:17.746 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.741+0000 7fac5f5da740 10 client.25501 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:07:17.746 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.741+0000 7fac5f5da740 20 client.25501 populate_metadata read hostname 'smithi016' 2024-08-31T06:07:17.746 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.741+0000 7fac5f5da740 1 --2- 172.21.15.16:0/217869411 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d399d9ad80 0x55d39a19cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:17.746 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.741+0000 7fac5f5da740 4 client.25501 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:07:17.746 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.741+0000 7fac5f5da740 1 -- 172.21.15.16:0/217869411 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d39a0f6ec0 con 0x55d399d9ad80 2024-08-31T06:07:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.741+0000 7fac55a00640 1 --2- 172.21.15.16:0/217869411 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d399d9ad80 0x55d39a19cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.742+0000 7fac55a00640 1 --2- 172.21.15.16:0/217869411 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d399d9ad80 0x55d39a19cfb0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d22bfa7b420cb4a server_cookie=3115d6f6dd513e0c in_seq=0 out_seq=0 2024-08-31T06:07:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.742+0000 7fac3b400640 10 client.25501 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:17.748 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.743+0000 7fac3b400640 1 -- 172.21.15.16:0/217869411 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d39a0f6ec0 con 0x55d399d9ad80 2024-08-31T06:07:17.748 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.743+0000 7fac3b400640 10 client.25501 handle_command_reply: tid=0 2024-08-31T06:07:17.748 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.743+0000 7fac5f5da740 1 client.25501 shutdown 2024-08-31T06:07:17.748 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.743+0000 7fac5f5da740 1 -- 172.21.15.16:0/217869411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d399ddc1b0 msgr2=0x55d39a18b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:17.748 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.743+0000 7fac5f5da740 1 --2- 172.21.15.16:0/217869411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d399ddc1b0 0x55d39a18b8d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fac480e2c60 tx=0x7fac480ab710 comp rx=0 tx=0).stop 2024-08-31T06:07:17.748 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.744+0000 7fac5f5da740 1 -- 172.21.15.16:0/217869411 shutdown_connections 2024-08-31T06:07:17.748 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.744+0000 7fac5f5da740 1 --2- 172.21.15.16:0/217869411 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d399d9ad80 0x55d39a19cfb0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:17.748 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.744+0000 7fac5f5da740 1 --2- 172.21.15.16:0/217869411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d399ddc1b0 0x55d39a18b8d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:17.748 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.744+0000 7fac5f5da740 1 --2- 172.21.15.16:0/217869411 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d399dbb4d0 0x55d39a18b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:17.748 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.744+0000 7fac5f5da740 1 --2- 172.21.15.16:0/217869411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d399db32f0 0x55d39a18ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:17.748 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.744+0000 7fac3b400640 0 client.25501 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:17.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.744+0000 7fac5f5da740 1 -- 172.21.15.16:0/217869411 >> 172.21.15.16:0/217869411 conn(0x55d399dd0300 msgr2=0x55d39a180950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:17.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.744+0000 7fac5f5da740 1 -- 172.21.15.16:0/217869411 shutdown_connections 2024-08-31T06:07:17.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.744+0000 7fac5f5da740 1 -- 172.21.15.16:0/217869411 wait complete. 2024-08-31T06:07:17.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.744+0000 7fac5f5da740 20 client.25501 trim_cache size 0 max 16384 2024-08-31T06:07:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.759+0000 7fac5f5da740 1 Processor -- start 2024-08-31T06:07:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.759+0000 7fac5f5da740 1 -- start start 2024-08-31T06:07:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.760+0000 7fac5f5da740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d399db32f0 0x55d39a181310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.760+0000 7fac5f5da740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d399dbb4d0 0x55d39a1969c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.760+0000 7fac5f5da740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d399ddc1b0 0x55d39a188020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.760+0000 7fac5f5da740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fac501b3c80 con 0x55d399db32f0 2024-08-31T06:07:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.760+0000 7fac5f5da740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fac501b1550 con 0x55d399dbb4d0 2024-08-31T06:07:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.760+0000 7fac5f5da740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fac501af3a0 con 0x55d399ddc1b0 2024-08-31T06:07:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.760+0000 7fac55000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d399dbb4d0 0x55d39a1969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.760+0000 7fac56400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d399ddc1b0 0x55d39a188020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.760+0000 7fac55000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d399dbb4d0 0x55d39a1969c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:56468/0 (socket says 172.21.15.16:56468) 2024-08-31T06:07:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.760+0000 7fac55000640 1 -- 172.21.15.16:0/1570415944 learned_addr learned my addr 172.21.15.16:0/1570415944 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.760+0000 7fac55a00640 1 --2- 172.21.15.16:0/1570415944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d399db32f0 0x55d39a181310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.761+0000 7fac55000640 1 -- 172.21.15.16:0/1570415944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d399ddc1b0 msgr2=0x55d39a188020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.761+0000 7fac55000640 1 --2- 172.21.15.16:0/1570415944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d399ddc1b0 0x55d39a188020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.761+0000 7fac55000640 1 -- 172.21.15.16:0/1570415944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d399db32f0 msgr2=0x55d39a181310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.761+0000 7fac55000640 1 --2- 172.21.15.16:0/1570415944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d399db32f0 0x55d39a181310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.761+0000 7fac55000640 1 -- 172.21.15.16:0/1570415944 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d39a182730 con 0x55d399dbb4d0 2024-08-31T06:07:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.761+0000 7fac56400640 1 --2- 172.21.15.16:0/1570415944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d399ddc1b0 0x55d39a188020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:07:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.761+0000 7fac55000640 1 --2- 172.21.15.16:0/1570415944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d399dbb4d0 0x55d39a1969c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fac3c010d40 tx=0x7fac3c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f70b10f492e3d3b7 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.762+0000 7fac46000640 1 -- 172.21.15.16:0/1570415944 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c02d070 con 0x55d399dbb4d0 2024-08-31T06:07:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.762+0000 7fac46000640 1 -- 172.21.15.16:0/1570415944 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac3c00ed70 con 0x55d399dbb4d0 2024-08-31T06:07:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.762+0000 7fac46000640 1 -- 172.21.15.16:0/1570415944 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c028c70 con 0x55d399dbb4d0 2024-08-31T06:07:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.763+0000 7fac5f5da740 1 -- 172.21.15.16:0/1570415944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d399dbb4d0 msgr2=0x55d39a1969c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.763+0000 7fac5f5da740 1 --2- 172.21.15.16:0/1570415944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d399dbb4d0 0x55d39a1969c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fac3c010d40 tx=0x7fac3c0046c0 comp rx=0 tx=0).stop 2024-08-31T06:07:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.763+0000 7fac5f5da740 1 -- 172.21.15.16:0/1570415944 shutdown_connections 2024-08-31T06:07:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.763+0000 7fac5f5da740 1 --2- 172.21.15.16:0/1570415944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d399ddc1b0 0x55d39a188020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.763+0000 7fac5f5da740 1 --2- 172.21.15.16:0/1570415944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d399dbb4d0 0x55d39a1969c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.763+0000 7fac5f5da740 1 --2- 172.21.15.16:0/1570415944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d399db32f0 0x55d39a181310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.763+0000 7fac5f5da740 1 -- 172.21.15.16:0/1570415944 >> 172.21.15.16:0/1570415944 conn(0x55d399dd0300 msgr2=0x55d39a1962b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.763+0000 7fac5f5da740 1 -- 172.21.15.16:0/1570415944 shutdown_connections 2024-08-31T06:07:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.763+0000 7fac5f5da740 1 -- 172.21.15.16:0/1570415944 wait complete. 2024-08-31T06:07:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.763+0000 7fac5f5da740 1 -- start start 2024-08-31T06:07:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.763+0000 7fac5f5da740 1 Processor -- start 2024-08-31T06:07:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.764+0000 7fac5f5da740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d399db32f0 0x55d39a193a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.764+0000 7fac5f5da740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d399dbb4d0 0x55d39a190f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.764+0000 7fac5f5da740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d399ddc1b0 0x55d39a191450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.764+0000 7fac5f5da740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fac501afa10 con 0x55d399dbb4d0 2024-08-31T06:07:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.764+0000 7fac5f5da740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fac501b0e80 con 0x55d399ddc1b0 2024-08-31T06:07:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.764+0000 7fac5f5da740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fac501b00e0 con 0x55d399db32f0 2024-08-31T06:07:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.764+0000 7fac5f5da740 10 client.0 _ll_register_callbacks cb 0x7fac480ab550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:07:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.764+0000 7fac56400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d399ddc1b0 0x55d39a191450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.764+0000 7fac55000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d399dbb4d0 0x55d39a190f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.764+0000 7fac56400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d399ddc1b0 0x55d39a191450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:56472/0 (socket says 172.21.15.16:56472) 2024-08-31T06:07:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.764+0000 7fac56400640 1 -- 172.21.15.16:0/3940904404 learned_addr learned my addr 172.21.15.16:0/3940904404 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.764+0000 7fac55a00640 1 --2- 172.21.15.16:0/3940904404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d399db32f0 0x55d39a193a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.765+0000 7fac56400640 1 -- 172.21.15.16:0/3940904404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d399db32f0 msgr2=0x55d39a193a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.765+0000 7fac56400640 1 --2- 172.21.15.16:0/3940904404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d399db32f0 0x55d39a193a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.765+0000 7fac56400640 1 -- 172.21.15.16:0/3940904404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d399dbb4d0 msgr2=0x55d39a190f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.765+0000 7fac56400640 1 --2- 172.21.15.16:0/3940904404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d399dbb4d0 0x55d39a190f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.765+0000 7fac56400640 1 -- 172.21.15.16:0/3940904404 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d39a191b40 con 0x55d399ddc1b0 2024-08-31T06:07:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.765+0000 7fac55a00640 1 --2- 172.21.15.16:0/3940904404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d399db32f0 0x55d39a193a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:07:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.765+0000 7fac56400640 1 --2- 172.21.15.16:0/3940904404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d399ddc1b0 0x55d39a191450 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fac48009230 tx=0x7fac480ab600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec09c3c6ba04ef21 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.765+0000 7fac3b400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:07:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.765+0000 7fac3b400640 1 -- 172.21.15.16:0/3940904404 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac480c6060 con 0x55d399ddc1b0 2024-08-31T06:07:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.765+0000 7fac3b400640 1 -- 172.21.15.16:0/3940904404 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8426+0+0 (secure 0 0 0) 0x7fac480ab070 con 0x55d399ddc1b0 2024-08-31T06:07:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.765+0000 7fac5f5da740 1 -- 172.21.15.16:0/3940904404 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac3c00ed70 con 0x55d399ddc1b0 2024-08-31T06:07:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.765+0000 7fac3b400640 1 -- 172.21.15.16:0/3940904404 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fac480abc00 con 0x55d399ddc1b0 2024-08-31T06:07:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.765+0000 7fac5f5da740 10 client.25505 fetch_fsmap learned FSMap version 25 2024-08-31T06:07:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.765+0000 7fac5f5da740 1 -- 172.21.15.16:0/3940904404 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d39a207800 con 0x55d399ddc1b0 2024-08-31T06:07:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.766+0000 7fac3b400640 1 -- 172.21.15.16:0/3940904404 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac3c00ed70 con 0x55d399ddc1b0 2024-08-31T06:07:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.766+0000 7fac3b400640 1 -- 172.21.15.16:0/3940904404 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fac480cece0 con 0x55d399ddc1b0 2024-08-31T06:07:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.766+0000 7fac5f5da740 10 client.25505 fetch_fsmap finished waiting for FSMap version 25 2024-08-31T06:07:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.766+0000 7fac5f5da740 10 client.25505 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:07:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.766+0000 7fac5f5da740 20 client.25505 populate_metadata read hostname 'smithi016' 2024-08-31T06:07:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.766+0000 7fac5f5da740 1 --2- 172.21.15.16:0/3940904404 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d399d701c0 0x55d39a209ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.766+0000 7fac5f5da740 4 client.25505 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:07:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.766+0000 7fac5f5da740 1 -- 172.21.15.16:0/3940904404 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d39a0f2960 con 0x55d399d701c0 2024-08-31T06:07:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.767+0000 7fac55a00640 1 --2- 172.21.15.16:0/3940904404 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d399d701c0 0x55d39a209ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.767+0000 7fac55a00640 1 --2- 172.21.15.16:0/3940904404 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d399d701c0 0x55d39a209ff0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6efac7b868dd709 server_cookie=9850a8ef4dc5cb83 in_seq=0 out_seq=0 2024-08-31T06:07:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.767+0000 7fac3b400640 10 client.25505 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:17.799 INFO:teuthology.orchestra.run.smithi016.stderr:1100000 of 3200000 tuples (34%) done (elapsed 2.72 s, remaining 5.20 s) 2024-08-31T06:07:17.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.982+0000 7fac3b400640 1 -- 172.21.15.16:0/3940904404 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d39a0f2960 con 0x55d399d701c0 2024-08-31T06:07:17.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.982+0000 7fac3b400640 10 client.25505 handle_command_reply: tid=0 2024-08-31T06:07:17.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.982+0000 7fac5f5da740 1 client.25505 shutdown 2024-08-31T06:07:17.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.982+0000 7fac5f5da740 1 -- 172.21.15.16:0/3940904404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d399ddc1b0 msgr2=0x55d39a191450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:17.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.982+0000 7fac5f5da740 1 --2- 172.21.15.16:0/3940904404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d399ddc1b0 0x55d39a191450 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fac48009230 tx=0x7fac480ab600 comp rx=0 tx=0).stop 2024-08-31T06:07:17.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.982+0000 7fac5f5da740 1 -- 172.21.15.16:0/3940904404 shutdown_connections 2024-08-31T06:07:17.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.982+0000 7fac5f5da740 1 --2- 172.21.15.16:0/3940904404 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d399d701c0 0x55d39a209ff0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:17.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.982+0000 7fac5f5da740 1 --2- 172.21.15.16:0/3940904404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d399ddc1b0 0x55d39a191450 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:17.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.982+0000 7fac5f5da740 1 --2- 172.21.15.16:0/3940904404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d399dbb4d0 0x55d39a190f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:17.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.982+0000 7fac5f5da740 1 --2- 172.21.15.16:0/3940904404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d399db32f0 0x55d39a193a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:17.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.982+0000 7fac5f5da740 1 -- 172.21.15.16:0/3940904404 >> 172.21.15.16:0/3940904404 conn(0x55d399dd0300 msgr2=0x55d39a196a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:17.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.982+0000 7fac3b400640 0 client.25505 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:17.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.982+0000 7fac5f5da740 1 -- 172.21.15.16:0/3940904404 shutdown_connections 2024-08-31T06:07:17.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.982+0000 7fac5f5da740 1 -- 172.21.15.16:0/3940904404 wait complete. 2024-08-31T06:07:17.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.983+0000 7fac5f5da740 20 client.25505 trim_cache size 0 max 16384 2024-08-31T06:07:17.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:07:17.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:07:17.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:07:17.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:07:17.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.983+0000 7fac58400640 1 -- 172.21.15.16:0/3150120110 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fac340b4ca0 msgr2=0x7fac340b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:17.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.983+0000 7fac58400640 1 --2- 172.21.15.16:0/3150120110 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fac340b4ca0 0x7fac340b71b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fac3c00bfd0 tx=0x7fac3c015040 comp rx=0 tx=0).stop 2024-08-31T06:07:17.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.983+0000 7fac58400640 1 -- 172.21.15.16:0/3150120110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac5010db30 msgr2=0x7fac501a1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:17.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.983+0000 7fac58400640 1 --2- 172.21.15.16:0/3150120110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac5010db30 0x7fac501a1c90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fac4800e970 tx=0x7fac4800ee30 comp rx=0 tx=0).stop 2024-08-31T06:07:17.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.983+0000 7fac58400640 1 -- 172.21.15.16:0/3150120110 shutdown_connections 2024-08-31T06:07:17.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.983+0000 7fac58400640 1 --2- 172.21.15.16:0/3150120110 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fac340b4ca0 0x7fac340b71b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:17.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.983+0000 7fac58400640 1 --2- 172.21.15.16:0/3150120110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac5010db30 0x7fac501a1c90 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:17.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.983+0000 7fac58400640 1 --2- 172.21.15.16:0/3150120110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac50105020 0x7fac5019ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:17.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.983+0000 7fac58400640 1 --2- 172.21.15.16:0/3150120110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac501046d0 0x7fac5019c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:17.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.983+0000 7fac58400640 1 -- 172.21.15.16:0/3150120110 >> 172.21.15.16:0/3150120110 conn(0x7fac500fe160 msgr2=0x7fac5010f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:17.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.984+0000 7fac58400640 1 -- 172.21.15.16:0/3150120110 shutdown_connections 2024-08-31T06:07:17.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:17.984+0000 7fac58400640 1 -- 172.21.15.16:0/3150120110 wait complete. 2024-08-31T06:07:18.105 INFO:teuthology.orchestra.run.smithi016.stderr:1200000 of 3200000 tuples (37%) done (elapsed 3.03 s, remaining 5.05 s) 2024-08-31T06:07:18.378 INFO:teuthology.orchestra.run.smithi016.stderr:1300000 of 3200000 tuples (40%) done (elapsed 3.30 s, remaining 4.82 s) 2024-08-31T06:07:18.714 INFO:teuthology.orchestra.run.smithi016.stderr:1400000 of 3200000 tuples (43%) done (elapsed 3.64 s, remaining 4.68 s) 2024-08-31T06:07:18.871 INFO:teuthology.orchestra.run.smithi016.stderr:1500000 of 3200000 tuples (46%) done (elapsed 3.79 s, remaining 4.30 s) 2024-08-31T06:07:18.872 INFO:journalctl@ceph.mds.h.smithi155.stdout:Aug 31 06:07:18 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-h[53037]: ignoring --setuser ceph since I am not root 2024-08-31T06:07:18.872 INFO:journalctl@ceph.mds.h.smithi155.stdout:Aug 31 06:07:18 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-h[53037]: ignoring --setgroup ceph since I am not root 2024-08-31T06:07:18.872 INFO:journalctl@ceph.mds.h.smithi155.stdout:Aug 31 06:07:18 smithi155 ceph-mds[53041]: -- 172.21.15.155:0/3560541165 <== mon.2 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555c13783860 con 0x555c17929180 2024-08-31T06:07:18.872 INFO:journalctl@ceph.mds.h.smithi155.stdout:Aug 31 06:07:18 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-h[53037]: starting mds.h at 2024-08-31T06:07:19.103 INFO:teuthology.orchestra.run.smithi016.stderr:1600000 of 3200000 tuples (50%) done (elapsed 4.03 s, remaining 4.03 s) 2024-08-31T06:07:19.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:18 smithi184 ceph-mon[22331]: pgmap v365: 97 pgs: 97 active+clean; 48 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 4.6 MiB/s wr, 122 op/s 2024-08-31T06:07:19.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:18 smithi184 ceph-mon[22331]: daemon mds.h restarted 2024-08-31T06:07:19.269 INFO:teuthology.orchestra.run.smithi016.stderr:1700000 of 3200000 tuples (53%) done (elapsed 4.19 s, remaining 3.70 s) 2024-08-31T06:07:19.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:18 smithi155 ceph-mon[21742]: pgmap v365: 97 pgs: 97 active+clean; 48 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 4.6 MiB/s wr, 122 op/s 2024-08-31T06:07:19.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:18 smithi155 ceph-mon[21742]: daemon mds.h restarted 2024-08-31T06:07:19.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:18 smithi016 ceph-mon[17194]: pgmap v365: 97 pgs: 97 active+clean; 48 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 4.6 MiB/s wr, 122 op/s 2024-08-31T06:07:19.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:18 smithi016 ceph-mon[17194]: daemon mds.h restarted 2024-08-31T06:07:19.569 INFO:teuthology.orchestra.run.smithi016.stderr:1800000 of 3200000 tuples (56%) done (elapsed 4.49 s, remaining 3.49 s) 2024-08-31T06:07:19.871 INFO:teuthology.orchestra.run.smithi016.stderr:1900000 of 3200000 tuples (59%) done (elapsed 4.79 s, remaining 3.28 s) 2024-08-31T06:07:20.079 INFO:teuthology.orchestra.run.smithi016.stderr:2000000 of 3200000 tuples (62%) done (elapsed 5.00 s, remaining 3.00 s) 2024-08-31T06:07:20.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:19 smithi184 ceph-mon[22331]: osdmap e81: 12 total, 12 up, 12 in 2024-08-31T06:07:20.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:19 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-08-31T06:07:20.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:19 smithi184 ceph-mon[22331]: mds.? [v2:172.21.15.155:6836/1791912179,v1:172.21.15.155:6837/1791912179] up:boot 2024-08-31T06:07:20.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:19 smithi184 ceph-mon[22331]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-08-31T06:07:20.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:19 smithi184 ceph-mon[22331]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:07:20.345 INFO:teuthology.orchestra.run.smithi016.stderr:2100000 of 3200000 tuples (65%) done (elapsed 5.27 s, remaining 2.76 s) 2024-08-31T06:07:20.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:19 smithi155 ceph-mon[21742]: osdmap e81: 12 total, 12 up, 12 in 2024-08-31T06:07:20.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:19 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-08-31T06:07:20.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:19 smithi155 ceph-mon[21742]: mds.? [v2:172.21.15.155:6836/1791912179,v1:172.21.15.155:6837/1791912179] up:boot 2024-08-31T06:07:20.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:19 smithi155 ceph-mon[21742]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-08-31T06:07:20.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:19 smithi155 ceph-mon[21742]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:07:20.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:19 smithi016 ceph-mon[17194]: osdmap e81: 12 total, 12 up, 12 in 2024-08-31T06:07:20.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:19 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-08-31T06:07:20.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:19 smithi016 ceph-mon[17194]: mds.? [v2:172.21.15.155:6836/1791912179,v1:172.21.15.155:6837/1791912179] up:boot 2024-08-31T06:07:20.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:19 smithi016 ceph-mon[17194]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-08-31T06:07:20.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:19 smithi016 ceph-mon[17194]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:07:20.635 INFO:teuthology.orchestra.run.smithi016.stderr:2200000 of 3200000 tuples (68%) done (elapsed 5.56 s, remaining 2.53 s) 2024-08-31T06:07:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2300000 of 3200000 tuples (71%) done (elapsed 5.72 s, remaining 2.24 s) 2024-08-31T06:07:21.071 INFO:teuthology.orchestra.run.smithi016.stderr:2400000 of 3200000 tuples (75%) done (elapsed 5.99 s, remaining 2.00 s) 2024-08-31T06:07:21.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:20 smithi184 ceph-mon[22331]: pgmap v367: 97 pgs: 97 active+clean; 121 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 21 MiB/s wr, 283 op/s 2024-08-31T06:07:21.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:20 smithi155 ceph-mon[21742]: pgmap v367: 97 pgs: 97 active+clean; 121 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 21 MiB/s wr, 283 op/s 2024-08-31T06:07:21.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:20 smithi016 ceph-mon[17194]: pgmap v367: 97 pgs: 97 active+clean; 121 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 21 MiB/s wr, 283 op/s 2024-08-31T06:07:21.434 INFO:teuthology.orchestra.run.smithi016.stderr:2500000 of 3200000 tuples (78%) done (elapsed 6.36 s, remaining 1.78 s) 2024-08-31T06:07:21.598 INFO:teuthology.orchestra.run.smithi016.stderr:2600000 of 3200000 tuples (81%) done (elapsed 6.52 s, remaining 1.50 s) 2024-08-31T06:07:21.898 INFO:teuthology.orchestra.run.smithi016.stderr:2700000 of 3200000 tuples (84%) done (elapsed 6.82 s, remaining 1.26 s) 2024-08-31T06:07:22.198 INFO:teuthology.orchestra.run.smithi016.stderr:2800000 of 3200000 tuples (87%) done (elapsed 7.12 s, remaining 1.02 s) 2024-08-31T06:07:22.368 INFO:teuthology.orchestra.run.smithi016.stderr:2900000 of 3200000 tuples (90%) done (elapsed 7.29 s, remaining 0.75 s) 2024-08-31T06:07:22.654 INFO:teuthology.orchestra.run.smithi016.stderr:3000000 of 3200000 tuples (93%) done (elapsed 7.58 s, remaining 0.51 s) 2024-08-31T06:07:22.991 INFO:teuthology.orchestra.run.smithi016.stderr:3100000 of 3200000 tuples (96%) done (elapsed 7.91 s, remaining 0.26 s) 2024-08-31T06:07:23.082 INFO:teuthology.orchestra.run.smithi016.stderr:3200000 of 3200000 tuples (100%) done (elapsed 8.00 s, remaining 0.00 s) 2024-08-31T06:07:23.178 INFO:teuthology.orchestra.run.smithi016.stderr:vacuuming... 2024-08-31T06:07:23.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:22 smithi184 ceph-mon[22331]: pgmap v368: 97 pgs: 97 active+clean; 165 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 30 MiB/s wr, 349 op/s 2024-08-31T06:07:23.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:22 smithi155 ceph-mon[21742]: pgmap v368: 97 pgs: 97 active+clean; 165 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 30 MiB/s wr, 349 op/s 2024-08-31T06:07:23.375 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T06:07:23.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:22 smithi016 ceph-mon[17194]: pgmap v368: 97 pgs: 97 active+clean; 165 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 30 MiB/s wr, 349 op/s 2024-08-31T06:07:23.776 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:07:24.099 INFO:teuthology.orchestra.run.smithi016.stderr:creating primary keys... 2024-08-31T06:07:24.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.446+0000 7faa37400640 1 -- 172.21.15.16:0/2130254716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa301141f0 msgr2=0x7faa301166c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:24.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.446+0000 7faa37400640 1 --2- 172.21.15.16:0/2130254716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa301141f0 0x7faa301166c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7faa28009ca0 tx=0x7faa2802a8c0 comp rx=0 tx=0).stop 2024-08-31T06:07:24.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.446+0000 7faa37400640 1 -- 172.21.15.16:0/2130254716 shutdown_connections 2024-08-31T06:07:24.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.446+0000 7faa37400640 1 --2- 172.21.15.16:0/2130254716 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa30075b70 0x7faa30075f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:24.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.446+0000 7faa37400640 1 --2- 172.21.15.16:0/2130254716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa301141f0 0x7faa301166c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:24.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.446+0000 7faa37400640 1 --2- 172.21.15.16:0/2130254716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa30077120 0x7faa30075550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:24.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.446+0000 7faa37400640 1 -- 172.21.15.16:0/2130254716 >> 172.21.15.16:0/2130254716 conn(0x7faa300ff3b0 msgr2=0x7faa30101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:24.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.446+0000 7faa37400640 1 -- 172.21.15.16:0/2130254716 shutdown_connections 2024-08-31T06:07:24.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.446+0000 7faa37400640 1 -- 172.21.15.16:0/2130254716 wait complete. 2024-08-31T06:07:24.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.447+0000 7faa37400640 1 Processor -- start 2024-08-31T06:07:24.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.447+0000 7faa37400640 1 -- start start 2024-08-31T06:07:24.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.448+0000 7faa37400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa30075b70 0x7faa301a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:24.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.448+0000 7faa37400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa30077120 0x7faa301a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:24.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.448+0000 7faa37400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa301141f0 0x7faa301a5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:24.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.448+0000 7faa37400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faa30119e80 con 0x7faa30077120 2024-08-31T06:07:24.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.448+0000 7faa37400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faa30119d00 con 0x7faa301141f0 2024-08-31T06:07:24.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.448+0000 7faa37400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faa3011a000 con 0x7faa30075b70 2024-08-31T06:07:24.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.448+0000 7faa2fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa30077120 0x7faa301a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:24.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.448+0000 7faa35400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa301141f0 0x7faa301a5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:24.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.448+0000 7faa2fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa30077120 0x7faa301a0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37770/0 (socket says 172.21.15.16:37770) 2024-08-31T06:07:24.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.448+0000 7faa35400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa301141f0 0x7faa301a5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:56498/0 (socket says 172.21.15.16:56498) 2024-08-31T06:07:24.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.448+0000 7faa2fe00640 1 -- 172.21.15.16:0/3188188125 learned_addr learned my addr 172.21.15.16:0/3188188125 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:24.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.448+0000 7faa34a00640 1 --2- 172.21.15.16:0/3188188125 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa30075b70 0x7faa301a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:24.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.449+0000 7faa2fe00640 1 -- 172.21.15.16:0/3188188125 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa30075b70 msgr2=0x7faa301a07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:24.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.449+0000 7faa2fe00640 1 --2- 172.21.15.16:0/3188188125 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa30075b70 0x7faa301a07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:24.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.449+0000 7faa2fe00640 1 -- 172.21.15.16:0/3188188125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa301141f0 msgr2=0x7faa301a5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:24.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.449+0000 7faa2fe00640 1 --2- 172.21.15.16:0/3188188125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa301141f0 0x7faa301a5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:24.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.449+0000 7faa2fe00640 1 -- 172.21.15.16:0/3188188125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa301a6720 con 0x7faa30077120 2024-08-31T06:07:24.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.449+0000 7faa35400640 1 --2- 172.21.15.16:0/3188188125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa301141f0 0x7faa301a5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:07:24.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.449+0000 7faa2fe00640 1 --2- 172.21.15.16:0/3188188125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa30077120 0x7faa301a0d50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7faa1c00e9c0 tx=0x7faa1c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:24.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.450+0000 7faa2d600640 1 -- 172.21.15.16:0/3188188125 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1c00cd80 con 0x7faa30077120 2024-08-31T06:07:24.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.450+0000 7faa2d600640 1 -- 172.21.15.16:0/3188188125 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa1c0054d0 con 0x7faa30077120 2024-08-31T06:07:24.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.450+0000 7faa37400640 1 -- 172.21.15.16:0/3188188125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa301a6a10 con 0x7faa30077120 2024-08-31T06:07:24.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.450+0000 7faa37400640 1 -- 172.21.15.16:0/3188188125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa301ad170 con 0x7faa30077120 2024-08-31T06:07:24.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.450+0000 7faa2d600640 1 -- 172.21.15.16:0/3188188125 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1c010430 con 0x7faa30077120 2024-08-31T06:07:24.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.455+0000 7faa3e0d1740 1 Processor -- start 2024-08-31T06:07:24.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.455+0000 7faa3e0d1740 1 -- start start 2024-08-31T06:07:24.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.456+0000 7faa3e0d1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558154372070 0x558154717870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:24.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.456+0000 7faa3e0d1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581543491b0 0x558154717db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:24.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.456+0000 7faa3e0d1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558154351390 0x5581547182f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:24.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.456+0000 7faa3e0d1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55815468e770 con 0x558154372070 2024-08-31T06:07:24.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.456+0000 7faa3e0d1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55815467dd00 con 0x5581543491b0 2024-08-31T06:07:24.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.456+0000 7faa3e0d1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558154681cb0 con 0x558154351390 2024-08-31T06:07:24.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.456+0000 7faa2d600640 1 -- 172.21.15.16:0/3188188125 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7faa1c0046c0 con 0x7faa30077120 2024-08-31T06:07:24.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.456+0000 7faa34a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581543491b0 0x558154717db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:24.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.456+0000 7faa35400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558154372070 0x558154717870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:24.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.456+0000 7faa34a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581543491b0 0x558154717db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:56510/0 (socket says 172.21.15.16:56510) 2024-08-31T06:07:24.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.456+0000 7faa34a00640 1 -- 172.21.15.16:0/1688915103 learned_addr learned my addr 172.21.15.16:0/1688915103 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:24.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.457+0000 7faa2fe00640 1 --2- 172.21.15.16:0/1688915103 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558154351390 0x5581547182f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.457+0000 7faa34a00640 1 -- 172.21.15.16:0/1688915103 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558154351390 msgr2=0x5581547182f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.457+0000 7faa34a00640 1 --2- 172.21.15.16:0/1688915103 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558154351390 0x5581547182f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.457+0000 7faa34a00640 1 -- 172.21.15.16:0/1688915103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558154372070 msgr2=0x558154717870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.457+0000 7faa34a00640 1 --2- 172.21.15.16:0/1688915103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558154372070 0x558154717870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.457+0000 7faa34a00640 1 -- 172.21.15.16:0/1688915103 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581543dd650 con 0x5581543491b0 2024-08-31T06:07:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.457+0000 7faa35400640 1 --2- 172.21.15.16:0/1688915103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558154372070 0x558154717870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:07:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.458+0000 7faa34a00640 1 --2- 172.21.15.16:0/1688915103 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581543491b0 0x558154717db0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faa24009cf0 tx=0x7faa24011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9eddc473e7a49408 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.458+0000 7faa21c00640 1 -- 172.21.15.16:0/1688915103 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2401e070 con 0x5581543491b0 2024-08-31T06:07:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.458+0000 7faa21c00640 1 -- 172.21.15.16:0/1688915103 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa24019ae0 con 0x5581543491b0 2024-08-31T06:07:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.458+0000 7faa21c00640 1 -- 172.21.15.16:0/1688915103 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2400e9e0 con 0x5581543491b0 2024-08-31T06:07:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.458+0000 7faa2d600640 1 --2- 172.21.15.16:0/3188188125 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7faa180b4da0 0x7faa180b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:24.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.459+0000 7faa2d600640 1 -- 172.21.15.16:0/3188188125 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7faa1c0e5780 con 0x7faa30077120 2024-08-31T06:07:24.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.459+0000 7faa35400640 1 --2- 172.21.15.16:0/3188188125 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7faa180b4da0 0x7faa180b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:24.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.459+0000 7faa3e0d1740 1 -- 172.21.15.16:0/1688915103 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581543491b0 msgr2=0x558154717db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:24.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.459+0000 7faa3e0d1740 1 --2- 172.21.15.16:0/1688915103 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581543491b0 0x558154717db0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faa24009cf0 tx=0x7faa24011810 comp rx=0 tx=0).stop 2024-08-31T06:07:24.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.460+0000 7faa3e0d1740 1 -- 172.21.15.16:0/1688915103 shutdown_connections 2024-08-31T06:07:24.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.460+0000 7faa3e0d1740 1 --2- 172.21.15.16:0/1688915103 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558154351390 0x5581547182f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:24.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.460+0000 7faa3e0d1740 1 --2- 172.21.15.16:0/1688915103 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581543491b0 0x558154717db0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:24.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.460+0000 7faa3e0d1740 1 --2- 172.21.15.16:0/1688915103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558154372070 0x558154717870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:24.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.460+0000 7faa3e0d1740 1 -- 172.21.15.16:0/1688915103 >> 172.21.15.16:0/1688915103 conn(0x558154366460 msgr2=0x558154715760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:24.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.460+0000 7faa35400640 1 --2- 172.21.15.16:0/3188188125 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7faa180b4da0 0x7faa180b72b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7faa28006010 tx=0x7faa28005d80 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:24.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.460+0000 7faa3e0d1740 1 -- 172.21.15.16:0/1688915103 shutdown_connections 2024-08-31T06:07:24.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.460+0000 7faa3e0d1740 1 -- 172.21.15.16:0/1688915103 wait complete. 2024-08-31T06:07:24.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.460+0000 7faa3e0d1740 1 -- start start 2024-08-31T06:07:24.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.461+0000 7faa3e0d1740 1 Processor -- start 2024-08-31T06:07:24.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.461+0000 7faa3e0d1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581543491b0 0x558154723420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:24.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.461+0000 7faa3e0d1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558154351390 0x558154723960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:24.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.461+0000 7faa3e0d1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558154372070 0x55815472f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:24.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.461+0000 7faa3e0d1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5581544e0700 con 0x5581543491b0 2024-08-31T06:07:24.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.461+0000 7faa3e0d1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x558154604cb0 con 0x558154372070 2024-08-31T06:07:24.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.461+0000 7faa3e0d1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558154726050 con 0x558154351390 2024-08-31T06:07:24.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.461+0000 7faa3e0d1740 10 client.0 _ll_register_callbacks cb 0x55815462a430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:07:24.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.462+0000 7faa35400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558154372070 0x55815472f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:24.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.462+0000 7faa34a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581543491b0 0x558154723420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:24.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.462+0000 7faa35400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558154372070 0x55815472f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:56512/0 (socket says 172.21.15.16:56512) 2024-08-31T06:07:24.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.462+0000 7faa35400640 1 -- 172.21.15.16:0/1684370477 learned_addr learned my addr 172.21.15.16:0/1684370477 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:24.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.462+0000 7faa2fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558154351390 0x558154723960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:24.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.462+0000 7faa2fe00640 1 -- 172.21.15.16:0/1684370477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558154372070 msgr2=0x55815472f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:24.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.462+0000 7faa2fe00640 1 --2- 172.21.15.16:0/1684370477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558154372070 0x55815472f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:24.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.462+0000 7faa2fe00640 1 -- 172.21.15.16:0/1684370477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581543491b0 msgr2=0x558154723420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:24.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.462+0000 7faa2fe00640 1 --2- 172.21.15.16:0/1684370477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581543491b0 0x558154723420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:24.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.462+0000 7faa2fe00640 1 -- 172.21.15.16:0/1684370477 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558154645870 con 0x558154351390 2024-08-31T06:07:24.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.462+0000 7faa34a00640 1 --2- 172.21.15.16:0/1684370477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581543491b0 0x558154723420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:07:24.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.463+0000 7faa2fe00640 1 --2- 172.21.15.16:0/1684370477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558154351390 0x558154723960 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7faa1c0e2ed0 tx=0x7faa1c0abd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d687443d3b1bda8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:24.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.463+0000 7faa16000640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:07:24.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.463+0000 7faa16000640 1 -- 172.21.15.16:0/1684370477 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1c0ab980 con 0x558154351390 2024-08-31T06:07:24.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.463+0000 7faa16000640 1 -- 172.21.15.16:0/1684370477 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7faa1c0b0330 con 0x558154351390 2024-08-31T06:07:24.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.463+0000 7faa3e0d1740 1 -- 172.21.15.16:0/1684370477 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa24019ae0 con 0x558154351390 2024-08-31T06:07:24.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.463+0000 7faa16000640 1 -- 172.21.15.16:0/1684370477 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faa1c0ac570 con 0x558154351390 2024-08-31T06:07:24.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.463+0000 7faa3e0d1740 10 client.25522 fetch_fsmap learned FSMap version 27 2024-08-31T06:07:24.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.463+0000 7faa3e0d1740 1 -- 172.21.15.16:0/1684370477 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5581547320a0 con 0x558154351390 2024-08-31T06:07:24.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.464+0000 7faa16000640 1 -- 172.21.15.16:0/1684370477 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa24019ae0 con 0x558154351390 2024-08-31T06:07:24.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.464+0000 7faa16000640 1 -- 172.21.15.16:0/1684370477 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7faa1c0b4d00 con 0x558154351390 2024-08-31T06:07:24.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.464+0000 7faa3e0d1740 10 client.25522 fetch_fsmap finished waiting for FSMap version 27 2024-08-31T06:07:24.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.464+0000 7faa3e0d1740 10 client.25522 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:07:24.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.464+0000 7faa3e0d1740 20 client.25522 populate_metadata read hostname 'smithi016' 2024-08-31T06:07:24.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.464+0000 7faa3e0d1740 1 --2- 172.21.15.16:0/1684370477 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5581543309f0 0x558154735700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:24.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.464+0000 7faa3e0d1740 4 client.25522 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:07:24.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.464+0000 7faa3e0d1740 1 -- 172.21.15.16:0/1684370477 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581546906c0 con 0x5581543309f0 2024-08-31T06:07:24.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.464+0000 7faa34a00640 1 --2- 172.21.15.16:0/1684370477 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5581543309f0 0x558154735700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:24.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.465+0000 7faa34a00640 1 --2- 172.21.15.16:0/1684370477 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5581543309f0 0x558154735700 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8625454159389cee server_cookie=e6ae89aafeea0b30 in_seq=0 out_seq=0 2024-08-31T06:07:24.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.465+0000 7faa16000640 10 client.25522 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:24.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.466+0000 7faa16000640 1 -- 172.21.15.16:0/1684370477 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5581546906c0 con 0x5581543309f0 2024-08-31T06:07:24.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.466+0000 7faa16000640 10 client.25522 handle_command_reply: tid=0 2024-08-31T06:07:24.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.466+0000 7faa3e0d1740 1 client.25522 shutdown 2024-08-31T06:07:24.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.466+0000 7faa3e0d1740 1 -- 172.21.15.16:0/1684370477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558154351390 msgr2=0x558154723960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:24.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.466+0000 7faa3e0d1740 1 --2- 172.21.15.16:0/1684370477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558154351390 0x558154723960 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7faa1c0e2ed0 tx=0x7faa1c0abd70 comp rx=0 tx=0).stop 2024-08-31T06:07:24.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.466+0000 7faa3e0d1740 1 -- 172.21.15.16:0/1684370477 shutdown_connections 2024-08-31T06:07:24.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.466+0000 7faa3e0d1740 1 --2- 172.21.15.16:0/1684370477 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5581543309f0 0x558154735700 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:24.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.466+0000 7faa3e0d1740 1 --2- 172.21.15.16:0/1684370477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558154351390 0x558154723960 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:24.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.466+0000 7faa3e0d1740 1 --2- 172.21.15.16:0/1684370477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558154372070 0x55815472f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:24.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.466+0000 7faa3e0d1740 1 --2- 172.21.15.16:0/1684370477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581543491b0 0x558154723420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:24.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.466+0000 7faa16000640 0 client.25522 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:24.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.466+0000 7faa3e0d1740 1 -- 172.21.15.16:0/1684370477 >> 172.21.15.16:0/1684370477 conn(0x558154366460 msgr2=0x55815471aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:24.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.466+0000 7faa3e0d1740 1 -- 172.21.15.16:0/1684370477 shutdown_connections 2024-08-31T06:07:24.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.467+0000 7faa3e0d1740 1 -- 172.21.15.16:0/1684370477 wait complete. 2024-08-31T06:07:24.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.467+0000 7faa3e0d1740 20 client.25522 trim_cache size 0 max 16384 2024-08-31T06:07:24.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.485+0000 7faa3e0d1740 1 Processor -- start 2024-08-31T06:07:24.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.486+0000 7faa3e0d1740 1 -- start start 2024-08-31T06:07:24.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.486+0000 7faa3e0d1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581543491b0 0x55815471e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:24.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.486+0000 7faa3e0d1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558154351390 0x558154714ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:24.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.486+0000 7faa3e0d1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558154372070 0x558154715530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:24.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.486+0000 7faa3e0d1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faa10010f60 con 0x558154351390 2024-08-31T06:07:24.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.486+0000 7faa3e0d1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faa1000e830 con 0x558154372070 2024-08-31T06:07:24.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.486+0000 7faa3e0d1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faa1000c680 con 0x5581543491b0 2024-08-31T06:07:24.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.486+0000 7faa34a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581543491b0 0x55815471e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:24.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.486+0000 7faa34a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581543491b0 0x55815471e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:32978/0 (socket says 172.21.15.16:32978) 2024-08-31T06:07:24.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.486+0000 7faa34a00640 1 -- 172.21.15.16:0/1364311342 learned_addr learned my addr 172.21.15.16:0/1364311342 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:24.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.486+0000 7faa35400640 1 --2- 172.21.15.16:0/1364311342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558154372070 0x558154715530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:24.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.486+0000 7faa2fe00640 1 --2- 172.21.15.16:0/1364311342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558154351390 0x558154714ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:24.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.487+0000 7faa2fe00640 1 -- 172.21.15.16:0/1364311342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581543491b0 msgr2=0x55815471e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:24.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.487+0000 7faa2fe00640 1 --2- 172.21.15.16:0/1364311342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581543491b0 0x55815471e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:24.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.487+0000 7faa2fe00640 1 -- 172.21.15.16:0/1364311342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558154372070 msgr2=0x558154715530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:24.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.487+0000 7faa2fe00640 1 --2- 172.21.15.16:0/1364311342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558154372070 0x558154715530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:24.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.487+0000 7faa2fe00640 1 -- 172.21.15.16:0/1364311342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581547244e0 con 0x558154351390 2024-08-31T06:07:24.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.487+0000 7faa35400640 1 --2- 172.21.15.16:0/1364311342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558154372070 0x558154715530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:07:24.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.487+0000 7faa2fe00640 1 --2- 172.21.15.16:0/1364311342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558154351390 0x558154714ff0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7faa1c00c6a0 tx=0x7faa1c0e2f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e48476b9dafcbc server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:24.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.488+0000 7faa21200640 1 -- 172.21.15.16:0/1364311342 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1c0ab980 con 0x558154351390 2024-08-31T06:07:24.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.488+0000 7faa21200640 1 -- 172.21.15.16:0/1364311342 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa1c0932c0 con 0x558154351390 2024-08-31T06:07:24.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.488+0000 7faa21200640 1 -- 172.21.15.16:0/1364311342 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1c0b0330 con 0x558154351390 2024-08-31T06:07:24.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.489+0000 7faa3e0d1740 1 -- 172.21.15.16:0/1364311342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558154351390 msgr2=0x558154714ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:24.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.489+0000 7faa3e0d1740 1 --2- 172.21.15.16:0/1364311342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558154351390 0x558154714ff0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7faa1c00c6a0 tx=0x7faa1c0e2f00 comp rx=0 tx=0).stop 2024-08-31T06:07:24.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.489+0000 7faa3e0d1740 1 -- 172.21.15.16:0/1364311342 shutdown_connections 2024-08-31T06:07:24.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.489+0000 7faa3e0d1740 1 --2- 172.21.15.16:0/1364311342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558154372070 0x558154715530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:24.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.489+0000 7faa3e0d1740 1 --2- 172.21.15.16:0/1364311342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558154351390 0x558154714ff0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:24.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.489+0000 7faa3e0d1740 1 --2- 172.21.15.16:0/1364311342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581543491b0 0x55815471e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:24.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.489+0000 7faa3e0d1740 1 -- 172.21.15.16:0/1364311342 >> 172.21.15.16:0/1364311342 conn(0x558154366460 msgr2=0x558154721aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:24.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.489+0000 7faa3e0d1740 1 -- 172.21.15.16:0/1364311342 shutdown_connections 2024-08-31T06:07:24.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.489+0000 7faa3e0d1740 1 -- 172.21.15.16:0/1364311342 wait complete. 2024-08-31T06:07:24.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.490+0000 7faa3e0d1740 1 -- start start 2024-08-31T06:07:24.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.490+0000 7faa3e0d1740 1 Processor -- start 2024-08-31T06:07:24.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.490+0000 7faa3e0d1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581543491b0 0x55815472b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:24.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.490+0000 7faa3e0d1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558154351390 0x55815472a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:24.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.490+0000 7faa3e0d1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558154372070 0x55815472a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:24.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.490+0000 7faa3e0d1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faa1000ccf0 con 0x5581543491b0 2024-08-31T06:07:24.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.490+0000 7faa3e0d1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faa1000e160 con 0x558154372070 2024-08-31T06:07:24.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.490+0000 7faa3e0d1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faa1000d3c0 con 0x558154351390 2024-08-31T06:07:24.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.490+0000 7faa3e0d1740 10 client.0 _ll_register_callbacks cb 0x7faa1c0a7d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:07:24.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.491+0000 7faa34a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581543491b0 0x55815472b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:24.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.491+0000 7faa34a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581543491b0 0x55815472b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37810/0 (socket says 172.21.15.16:37810) 2024-08-31T06:07:24.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.491+0000 7faa34a00640 1 -- 172.21.15.16:0/1792267575 learned_addr learned my addr 172.21.15.16:0/1792267575 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:24.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.491+0000 7faa35400640 1 --2- 172.21.15.16:0/1792267575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558154372070 0x55815472a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:24.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.491+0000 7faa2fe00640 1 --2- 172.21.15.16:0/1792267575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558154351390 0x55815472a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:24.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.491+0000 7faa35400640 1 -- 172.21.15.16:0/1792267575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558154351390 msgr2=0x55815472a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:24.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.491+0000 7faa35400640 1 --2- 172.21.15.16:0/1792267575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558154351390 0x55815472a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:24.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.491+0000 7faa35400640 1 -- 172.21.15.16:0/1792267575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581543491b0 msgr2=0x55815472b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:24.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.491+0000 7faa35400640 1 --2- 172.21.15.16:0/1792267575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581543491b0 0x55815472b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:24.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.491+0000 7faa35400640 1 -- 172.21.15.16:0/1792267575 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55815472adf0 con 0x558154372070 2024-08-31T06:07:24.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.492+0000 7faa35400640 1 --2- 172.21.15.16:0/1792267575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558154372070 0x55815472a8b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7faa2802aeb0 tx=0x7faa2802a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fd58d311afb9787 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:24.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.492+0000 7faa16000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:07:24.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.492+0000 7faa16000640 1 -- 172.21.15.16:0/1792267575 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2802f610 con 0x558154372070 2024-08-31T06:07:24.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.492+0000 7faa16000640 1 -- 172.21.15.16:0/1792267575 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7faa28043030 con 0x558154372070 2024-08-31T06:07:24.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.492+0000 7faa3e0d1740 1 -- 172.21.15.16:0/1792267575 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa1c0932c0 con 0x558154372070 2024-08-31T06:07:24.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.492+0000 7faa16000640 1 -- 172.21.15.16:0/1792267575 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faa2802f7b0 con 0x558154372070 2024-08-31T06:07:24.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.492+0000 7faa3e0d1740 10 client.25529 fetch_fsmap learned FSMap version 27 2024-08-31T06:07:24.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.492+0000 7faa3e0d1740 1 -- 172.21.15.16:0/1792267575 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5581547a0d60 con 0x558154372070 2024-08-31T06:07:24.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.493+0000 7faa16000640 1 -- 172.21.15.16:0/1792267575 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa1c0932c0 con 0x558154372070 2024-08-31T06:07:24.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.493+0000 7faa16000640 1 -- 172.21.15.16:0/1792267575 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7faa2805d020 con 0x558154372070 2024-08-31T06:07:24.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.493+0000 7faa3e0d1740 10 client.25529 fetch_fsmap finished waiting for FSMap version 27 2024-08-31T06:07:24.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.493+0000 7faa3e0d1740 10 client.25529 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:07:24.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.493+0000 7faa3e0d1740 20 client.25529 populate_metadata read hostname 'smithi016' 2024-08-31T06:07:24.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.493+0000 7faa3e0d1740 1 --2- 172.21.15.16:0/1792267575 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x558154305da0 0x5581547a3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:24.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.493+0000 7faa3e0d1740 4 client.25529 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T06:07:24.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.493+0000 7faa3e0d1740 1 -- 172.21.15.16:0/1792267575 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55815468c160 con 0x558154305da0 2024-08-31T06:07:24.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.494+0000 7faa34a00640 1 --2- 172.21.15.16:0/1792267575 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x558154305da0 0x5581547a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:24.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.494+0000 7faa34a00640 1 --2- 172.21.15.16:0/1792267575 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x558154305da0 0x5581547a3570 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=319b9a5676f596b0 server_cookie=20bde9cf258b27b8 in_seq=0 out_seq=0 2024-08-31T06:07:24.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.494+0000 7faa16000640 10 client.25529 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:24.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.494+0000 7faa16000640 1 -- 172.21.15.16:0/1792267575 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+374 (crc 0 0 0) 0x55815468c160 con 0x558154305da0 2024-08-31T06:07:24.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.494+0000 7faa16000640 10 client.25529 handle_command_reply: tid=0 2024-08-31T06:07:24.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.494+0000 7faa3e0d1740 1 client.25529 shutdown 2024-08-31T06:07:24.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.495+0000 7faa3e0d1740 1 -- 172.21.15.16:0/1792267575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558154372070 msgr2=0x55815472a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:24.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.495+0000 7faa3e0d1740 1 --2- 172.21.15.16:0/1792267575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558154372070 0x55815472a8b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7faa2802aeb0 tx=0x7faa2802a8c0 comp rx=0 tx=0).stop 2024-08-31T06:07:24.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.495+0000 7faa3e0d1740 1 -- 172.21.15.16:0/1792267575 shutdown_connections 2024-08-31T06:07:24.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.495+0000 7faa3e0d1740 1 --2- 172.21.15.16:0/1792267575 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x558154305da0 0x5581547a3570 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:24.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.495+0000 7faa3e0d1740 1 --2- 172.21.15.16:0/1792267575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558154351390 0x55815472a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:24.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.495+0000 7faa3e0d1740 1 --2- 172.21.15.16:0/1792267575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558154372070 0x55815472a8b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:24.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.495+0000 7faa3e0d1740 1 --2- 172.21.15.16:0/1792267575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581543491b0 0x55815472b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:24.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.495+0000 7faa3e0d1740 1 -- 172.21.15.16:0/1792267575 >> 172.21.15.16:0/1792267575 conn(0x558154366460 msgr2=0x55815471e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:24.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.495+0000 7faa16000640 0 client.25529 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:24.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.495+0000 7faa3e0d1740 1 -- 172.21.15.16:0/1792267575 shutdown_connections 2024-08-31T06:07:24.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.495+0000 7faa3e0d1740 1 -- 172.21.15.16:0/1792267575 wait complete. 2024-08-31T06:07:24.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.495+0000 7faa3e0d1740 20 client.25529 trim_cache size 0 max 16384 2024-08-31T06:07:24.501 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:07:24.501 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-08-31T06:07:24.501 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {}, 2024-08-31T06:07:24.502 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-08-31T06:07:24.502 INFO:teuthology.orchestra.run.smithi016.stdout: "c300b7e8-f3fd-482c-a909-9329bc61e570": { 2024-08-31T06:07:24.502 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:06:49.868046+0000", 2024-08-31T06:07:24.502 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:07:24.502 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:07:24.502 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:07:24.502 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:07:24.502 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:07:24.502 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:07:24.502 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:07:24.502 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:07:24.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.496+0000 7faa37400640 1 -- 172.21.15.16:0/3188188125 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7faa180b4da0 msgr2=0x7faa180b72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:24.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.496+0000 7faa37400640 1 --2- 172.21.15.16:0/3188188125 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7faa180b4da0 0x7faa180b72b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7faa28006010 tx=0x7faa28005d80 comp rx=0 tx=0).stop 2024-08-31T06:07:24.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.496+0000 7faa37400640 1 -- 172.21.15.16:0/3188188125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa30077120 msgr2=0x7faa301a0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:24.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.496+0000 7faa37400640 1 --2- 172.21.15.16:0/3188188125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa30077120 0x7faa301a0d50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7faa1c00e9c0 tx=0x7faa1c00ee80 comp rx=0 tx=0).stop 2024-08-31T06:07:24.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.496+0000 7faa37400640 1 -- 172.21.15.16:0/3188188125 shutdown_connections 2024-08-31T06:07:24.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.496+0000 7faa37400640 1 --2- 172.21.15.16:0/3188188125 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7faa180b4da0 0x7faa180b72b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:24.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.496+0000 7faa37400640 1 --2- 172.21.15.16:0/3188188125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa301141f0 0x7faa301a5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:24.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.496+0000 7faa37400640 1 --2- 172.21.15.16:0/3188188125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa30077120 0x7faa301a0d50 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:24.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.496+0000 7faa37400640 1 --2- 172.21.15.16:0/3188188125 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa30075b70 0x7faa301a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:24.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.496+0000 7faa37400640 1 -- 172.21.15.16:0/3188188125 >> 172.21.15.16:0/3188188125 conn(0x7faa300ff3b0 msgr2=0x7faa301100f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:24.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.497+0000 7faa37400640 1 -- 172.21.15.16:0/3188188125 shutdown_connections 2024-08-31T06:07:24.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:24.497+0000 7faa37400640 1 -- 172.21.15.16:0/3188188125 wait complete. 2024-08-31T06:07:25.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:24 smithi184 ceph-mon[22331]: pgmap v369: 97 pgs: 97 active+clean; 241 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 43 MiB/s wr, 391 op/s 2024-08-31T06:07:25.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:24 smithi155 ceph-mon[21742]: pgmap v369: 97 pgs: 97 active+clean; 241 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 43 MiB/s wr, 391 op/s 2024-08-31T06:07:25.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:24 smithi016 ceph-mon[17194]: pgmap v369: 97 pgs: 97 active+clean; 241 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 43 MiB/s wr, 391 op/s 2024-08-31T06:07:25.437 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T06:07:25.438 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:07:25.894 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:07:26.509 INFO:teuthology.orchestra.run.smithi016.stderr:done in 11.47 s (drop tables 0.00 s, create tables 0.02 s, client-side generate 8.13 s, vacuum 0.92 s, primary keys 2.41 s). 2024-08-31T06:07:26.517 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2024-08-31T06:07:26.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.609+0000 7ffb21800640 1 -- 172.21.15.16:0/3284747130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb1c109260 msgr2=0x7ffb1c109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:26.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.609+0000 7ffb21800640 1 --2- 172.21.15.16:0/3284747130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb1c109260 0x7ffb1c109670 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ffb08009860 tx=0x7ffb0802a340 comp rx=0 tx=0).stop 2024-08-31T06:07:26.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.609+0000 7ffb21800640 1 -- 172.21.15.16:0/3284747130 shutdown_connections 2024-08-31T06:07:26.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.609+0000 7ffb21800640 1 --2- 172.21.15.16:0/3284747130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb1c109bb0 0x7ffb1c1103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:26.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.609+0000 7ffb21800640 1 --2- 172.21.15.16:0/3284747130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb1c109260 0x7ffb1c109670 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:26.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.609+0000 7ffb21800640 1 --2- 172.21.15.16:0/3284747130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb1c103920 0x7ffb1c103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:26.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.609+0000 7ffb21800640 1 -- 172.21.15.16:0/3284747130 >> 172.21.15.16:0/3284747130 conn(0x7ffb1c0ff370 msgr2=0x7ffb1c1017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:26.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.610+0000 7ffb21800640 1 -- 172.21.15.16:0/3284747130 shutdown_connections 2024-08-31T06:07:26.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.610+0000 7ffb21800640 1 -- 172.21.15.16:0/3284747130 wait complete. 2024-08-31T06:07:26.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.611+0000 7ffb21800640 1 Processor -- start 2024-08-31T06:07:26.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.611+0000 7ffb21800640 1 -- start start 2024-08-31T06:07:26.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.611+0000 7ffb21800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb1c103920 0x7ffb1c19c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:26.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.611+0000 7ffb21800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb1c109260 0x7ffb1c19ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:26.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.611+0000 7ffb21800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb1c109bb0 0x7ffb1c1a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:26.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.611+0000 7ffb21800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1c113870 con 0x7ffb1c109bb0 2024-08-31T06:07:26.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.611+0000 7ffb21800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1c1136f0 con 0x7ffb1c103920 2024-08-31T06:07:26.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.611+0000 7ffb21800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1c1139f0 con 0x7ffb1c109260 2024-08-31T06:07:26.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.612+0000 7ffb1b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb1c109bb0 0x7ffb1c1a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:26.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.612+0000 7ffb1b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb1c109bb0 0x7ffb1c1a1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45728/0 (socket says 172.21.15.16:45728) 2024-08-31T06:07:26.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.612+0000 7ffb1b400640 1 -- 172.21.15.16:0/2632911012 learned_addr learned my addr 172.21.15.16:0/2632911012 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:26.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.612+0000 7ffb1a000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb1c109260 0x7ffb1c19ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:26.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.612+0000 7ffb1aa00640 1 --2- 172.21.15.16:0/2632911012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb1c103920 0x7ffb1c19c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:26.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.612+0000 7ffb1a000640 1 -- 172.21.15.16:0/2632911012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb1c103920 msgr2=0x7ffb1c19c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:26.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.612+0000 7ffb1a000640 1 --2- 172.21.15.16:0/2632911012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb1c103920 0x7ffb1c19c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:26.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.612+0000 7ffb1a000640 1 -- 172.21.15.16:0/2632911012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb1c109bb0 msgr2=0x7ffb1c1a1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:26.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.612+0000 7ffb1a000640 1 --2- 172.21.15.16:0/2632911012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb1c109bb0 0x7ffb1c1a1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:26.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.612+0000 7ffb1a000640 1 -- 172.21.15.16:0/2632911012 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb1c1a23f0 con 0x7ffb1c109260 2024-08-31T06:07:26.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.612+0000 7ffb1b400640 1 --2- 172.21.15.16:0/2632911012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb1c109bb0 0x7ffb1c1a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:07:26.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.612+0000 7ffb1a000640 1 --2- 172.21.15.16:0/2632911012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb1c109260 0x7ffb1c19ca20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ffb0802a860 tx=0x7ffb08032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:26.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.613+0000 7ffb13400640 1 -- 172.21.15.16:0/2632911012 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb08040020 con 0x7ffb1c109260 2024-08-31T06:07:26.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.613+0000 7ffb21800640 1 -- 172.21.15.16:0/2632911012 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb1c1a2680 con 0x7ffb1c109260 2024-08-31T06:07:26.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.613+0000 7ffb21800640 1 -- 172.21.15.16:0/2632911012 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb1c1a8e40 con 0x7ffb1c109260 2024-08-31T06:07:26.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.613+0000 7ffb13400640 1 -- 172.21.15.16:0/2632911012 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb08030e10 con 0x7ffb1c109260 2024-08-31T06:07:26.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.613+0000 7ffb13400640 1 -- 172.21.15.16:0/2632911012 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb08037040 con 0x7ffb1c109260 2024-08-31T06:07:26.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.614+0000 7ffb21800640 1 -- 172.21.15.16:0/2632911012 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb1c103d30 con 0x7ffb1c109260 2024-08-31T06:07:26.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.617+0000 7ffb13400640 1 -- 172.21.15.16:0/2632911012 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7ffb08043020 con 0x7ffb1c109260 2024-08-31T06:07:26.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.619+0000 7ffb13400640 1 --2- 172.21.15.16:0/2632911012 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ffb000b4d20 0x7ffb000b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:26.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.619+0000 7ffb13400640 1 -- 172.21.15.16:0/2632911012 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7ffb08104c50 con 0x7ffb1c109260 2024-08-31T06:07:26.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.619+0000 7ffb1aa00640 1 --2- 172.21.15.16:0/2632911012 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ffb000b4d20 0x7ffb000b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:26.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.620+0000 7ffb13400640 1 -- 172.21.15.16:0/2632911012 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7ffb0804a030 con 0x7ffb1c109260 2024-08-31T06:07:26.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.621+0000 7ffb1aa00640 1 --2- 172.21.15.16:0/2632911012 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ffb000b4d20 0x7ffb000b7230 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c009530 tx=0x7ffb0c009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:26.623 INFO:teuthology.orchestra.run.smithi016.stderr:starting vacuum...end. 2024-08-31T06:07:26.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.941+0000 7ffb21800640 1 -- 172.21.15.16:0/2632911012 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb1c1a9010 con 0x7ffb1c109260 2024-08-31T06:07:26.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.942+0000 7ffb13400640 1 -- 172.21.15.16:0/2632911012 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11917 (secure 0 0 0) 0x7ffb080cbac0 con 0x7ffb1c109260 2024-08-31T06:07:26.945 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:07:26.945 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":27,"btime":"2024-08-31T06:07:18:945609+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:07:18.945583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24737,24724,24712]},"id":1}]} 2024-08-31T06:07:26.946 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 27 2024-08-31T06:07:26.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.946+0000 7ffb21800640 1 -- 172.21.15.16:0/2632911012 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ffb000b4d20 msgr2=0x7ffb000b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:26.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.946+0000 7ffb21800640 1 --2- 172.21.15.16:0/2632911012 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ffb000b4d20 0x7ffb000b7230 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c009530 tx=0x7ffb0c009260 comp rx=0 tx=0).stop 2024-08-31T06:07:26.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.946+0000 7ffb21800640 1 -- 172.21.15.16:0/2632911012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb1c109260 msgr2=0x7ffb1c19ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:26.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.946+0000 7ffb21800640 1 --2- 172.21.15.16:0/2632911012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb1c109260 0x7ffb1c19ca20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ffb0802a860 tx=0x7ffb08032d50 comp rx=0 tx=0).stop 2024-08-31T06:07:26.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.947+0000 7ffb21800640 1 -- 172.21.15.16:0/2632911012 shutdown_connections 2024-08-31T06:07:26.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.947+0000 7ffb21800640 1 --2- 172.21.15.16:0/2632911012 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ffb000b4d20 0x7ffb000b7230 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:26.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.947+0000 7ffb21800640 1 --2- 172.21.15.16:0/2632911012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb1c109260 0x7ffb1c19ca20 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:26.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.947+0000 7ffb21800640 1 --2- 172.21.15.16:0/2632911012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb1c109bb0 0x7ffb1c1a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:26.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.947+0000 7ffb21800640 1 --2- 172.21.15.16:0/2632911012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb1c103920 0x7ffb1c19c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:26.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.947+0000 7ffb21800640 1 -- 172.21.15.16:0/2632911012 >> 172.21.15.16:0/2632911012 conn(0x7ffb1c0ff370 msgr2=0x7ffb1c108950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:26.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.948+0000 7ffb21800640 1 -- 172.21.15.16:0/2632911012 shutdown_connections 2024-08-31T06:07:26.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:26.949+0000 7ffb21800640 1 -- 172.21.15.16:0/2632911012 wait complete. 2024-08-31T06:07:27.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:26 smithi184 ceph-mon[22331]: pgmap v370: 97 pgs: 97 active+clean; 361 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 66 MiB/s wr, 294 op/s 2024-08-31T06:07:27.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:26 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/2632911012' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:07:27.369 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:26 smithi016 ceph-mon[17194]: pgmap v370: 97 pgs: 97 active+clean; 361 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 66 MiB/s wr, 294 op/s 2024-08-31T06:07:27.369 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:26 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/2632911012' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:07:27.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:26 smithi155 ceph-mon[21742]: pgmap v370: 97 pgs: 97 active+clean; 361 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 66 MiB/s wr, 294 op/s 2024-08-31T06:07:27.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:26 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/2632911012' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:07:27.937 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T06:07:28.561 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:07:29.065 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:28 smithi016 ceph-mon[17194]: pgmap v371: 97 pgs: 97 active+clean; 393 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 69 MiB/s wr, 281 op/s 2024-08-31T06:07:29.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:28 smithi155 ceph-mon[21742]: pgmap v371: 97 pgs: 97 active+clean; 393 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 69 MiB/s wr, 281 op/s 2024-08-31T06:07:29.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.404+0000 7f08b0400640 1 -- 172.21.15.16:0/625301425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08a8108aa0 msgr2=0x7f08a810f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:29.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.404+0000 7f08b0400640 1 --2- 172.21.15.16:0/625301425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08a8108aa0 0x7f08a810f2d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f08a0009be0 tx=0x7f08a002a6e0 comp rx=0 tx=0).stop 2024-08-31T06:07:29.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.405+0000 7f08b0400640 1 -- 172.21.15.16:0/625301425 shutdown_connections 2024-08-31T06:07:29.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.405+0000 7f08b0400640 1 --2- 172.21.15.16:0/625301425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08a8108aa0 0x7f08a810f2d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:29.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.405+0000 7f08b0400640 1 --2- 172.21.15.16:0/625301425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08a8108150 0x7f08a8108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:29.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.405+0000 7f08b0400640 1 --2- 172.21.15.16:0/625301425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08a8102860 0x7f08a8102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:29.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.405+0000 7f08b0400640 1 -- 172.21.15.16:0/625301425 >> 172.21.15.16:0/625301425 conn(0x7f08a80794c0 msgr2=0x7f08a80798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:29.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.406+0000 7f08b0400640 1 -- 172.21.15.16:0/625301425 shutdown_connections 2024-08-31T06:07:29.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.407+0000 7f08b0400640 1 -- 172.21.15.16:0/625301425 wait complete. 2024-08-31T06:07:29.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.409+0000 7f08b0400640 1 Processor -- start 2024-08-31T06:07:29.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.409+0000 7f08b0400640 1 -- start start 2024-08-31T06:07:29.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.410+0000 7f08b0400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08a8102860 0x7f08a810e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:29.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.410+0000 7f08b0400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08a8108150 0x7f08a810e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:29.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.410+0000 7f08b0400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08a8108aa0 0x7f08a81a2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:29.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.410+0000 7f08b0400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f08a81110c0 con 0x7f08a8108aa0 2024-08-31T06:07:29.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.410+0000 7f08b0400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f08a8110f40 con 0x7f08a8108150 2024-08-31T06:07:29.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.410+0000 7f08b0400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f08a8111240 con 0x7f08a8102860 2024-08-31T06:07:29.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.410+0000 7f08ad000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08a8108150 0x7f08a810e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:29.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.410+0000 7f08ad000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08a8108150 0x7f08a810e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:56600/0 (socket says 172.21.15.16:56600) 2024-08-31T06:07:29.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.410+0000 7f08ad000640 1 -- 172.21.15.16:0/2566087832 learned_addr learned my addr 172.21.15.16:0/2566087832 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:29.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.410+0000 7f08ada00640 1 --2- 172.21.15.16:0/2566087832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08a8102860 0x7f08a810e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:29.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.411+0000 7f08ad000640 1 -- 172.21.15.16:0/2566087832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08a8102860 msgr2=0x7f08a810e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:29.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.411+0000 7f08ad000640 1 --2- 172.21.15.16:0/2566087832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08a8102860 0x7f08a810e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:29.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.411+0000 7f08ad000640 1 -- 172.21.15.16:0/2566087832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08a8108aa0 msgr2=0x7f08a81a2ee0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-31T06:07:29.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.411+0000 7f08ad000640 1 --2- 172.21.15.16:0/2566087832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08a8108aa0 0x7f08a81a2ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:29.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.411+0000 7f08ad000640 1 -- 172.21.15.16:0/2566087832 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08a81a3610 con 0x7f08a8108150 2024-08-31T06:07:29.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.411+0000 7f08ad000640 1 --2- 172.21.15.16:0/2566087832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08a8108150 0x7f08a810e880 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0894009a50 tx=0x7f0894009f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:29.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.412+0000 7f08a6000640 1 -- 172.21.15.16:0/2566087832 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089400ca40 con 0x7f08a8108150 2024-08-31T06:07:29.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.412+0000 7f08a6000640 1 -- 172.21.15.16:0/2566087832 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f089400cbe0 con 0x7f08a8108150 2024-08-31T06:07:29.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.412+0000 7f08b0400640 1 -- 172.21.15.16:0/2566087832 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08a81a3900 con 0x7f08a8108150 2024-08-31T06:07:29.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.412+0000 7f08a6000640 1 -- 172.21.15.16:0/2566087832 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0894003c50 con 0x7f08a8108150 2024-08-31T06:07:29.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.413+0000 7f08b0400640 1 -- 172.21.15.16:0/2566087832 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08a81ad280 con 0x7f08a8108150 2024-08-31T06:07:29.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.422+0000 7f08b7049740 1 Processor -- start 2024-08-31T06:07:29.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.424+0000 7f08a6000640 1 -- 172.21.15.16:0/2566087832 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f0894003df0 con 0x7f08a8108150 2024-08-31T06:07:29.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.425+0000 7f08b7049740 1 -- start start 2024-08-31T06:07:29.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.427+0000 7f08a6000640 1 --2- 172.21.15.16:0/2566087832 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f08900b4dd0 0x7f08900b72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:29.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.428+0000 7f08b7049740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fd0ff7070 0x557fd139c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:29.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.428+0000 7f08b7049740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fd0fce1b0 0x557fd139ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:29.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.428+0000 7f08b7049740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fd0fd6390 0x557fd139d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:29.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.428+0000 7f08b7049740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557fd13137b0 con 0x557fd0ff7070 2024-08-31T06:07:29.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.428+0000 7f08b7049740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557fd1302d40 con 0x557fd0fd6390 2024-08-31T06:07:29.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.428+0000 7f08b7049740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557fd1306cf0 con 0x557fd0fce1b0 2024-08-31T06:07:29.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.428+0000 7f08ada00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fd0ff7070 0x557fd139c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:29.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.428+0000 7f08ada00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fd0ff7070 0x557fd139c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45748/0 (socket says 172.21.15.16:45748) 2024-08-31T06:07:29.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.428+0000 7f08ada00640 1 -- 172.21.15.16:0/2776484396 learned_addr learned my addr 172.21.15.16:0/2776484396 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:29.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.429+0000 7f08ada00640 1 -- 172.21.15.16:0/2776484396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fd0fce1b0 msgr2=0x557fd139ced0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:07:29.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.429+0000 7f08ada00640 1 --2- 172.21.15.16:0/2776484396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fd0fce1b0 0x557fd139ced0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:29.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.429+0000 7f08ae400640 1 --2- 172.21.15.16:0/2566087832 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f08900b4dd0 0x7f08900b72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:29.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.430+0000 7f08ae400640 1 --2- 172.21.15.16:0/2776484396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fd0fd6390 0x557fd139d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:29.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.431+0000 7f08ada00640 1 -- 172.21.15.16:0/2776484396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fd0fd6390 msgr2=0x557fd139d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:29.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.431+0000 7f08ada00640 1 --2- 172.21.15.16:0/2776484396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fd0fd6390 0x557fd139d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:29.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.431+0000 7f08ada00640 1 -- 172.21.15.16:0/2776484396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fd1062650 con 0x557fd0ff7070 2024-08-31T06:07:29.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.431+0000 7f08ae400640 1 --2- 172.21.15.16:0/2776484396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fd0fd6390 0x557fd139d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:07:29.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.431+0000 7f08a6000640 1 -- 172.21.15.16:0/2566087832 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f08940e4b10 con 0x7f08a8108150 2024-08-31T06:07:29.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.431+0000 7f08ae400640 1 --2- 172.21.15.16:0/2566087832 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f08900b4dd0 0x7f08900b72e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f08a00099a0 tx=0x7f08a00304a0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:29.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.431+0000 7f08ada00640 1 --2- 172.21.15.16:0/2776484396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fd0ff7070 0x557fd139c990 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f089c00bcb0 tx=0x7f089c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7def79b2f281902a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:29.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.431+0000 7f089aa00640 1 -- 172.21.15.16:0/2776484396 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089c015070 con 0x557fd0ff7070 2024-08-31T06:07:29.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.432+0000 7f089aa00640 1 -- 172.21.15.16:0/2776484396 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f089c01a910 con 0x557fd0ff7070 2024-08-31T06:07:29.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.432+0000 7f089aa00640 1 -- 172.21.15.16:0/2776484396 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089c011980 con 0x557fd0ff7070 2024-08-31T06:07:29.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.433+0000 7f08b7049740 1 -- 172.21.15.16:0/2776484396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fd0ff7070 msgr2=0x557fd139c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:29.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.433+0000 7f08b7049740 1 --2- 172.21.15.16:0/2776484396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fd0ff7070 0x557fd139c990 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f089c00bcb0 tx=0x7f089c00f730 comp rx=0 tx=0).stop 2024-08-31T06:07:29.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.434+0000 7f08b7049740 1 -- 172.21.15.16:0/2776484396 shutdown_connections 2024-08-31T06:07:29.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.434+0000 7f08b7049740 1 --2- 172.21.15.16:0/2776484396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fd0fce1b0 0x557fd139ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:29.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.434+0000 7f08b7049740 1 --2- 172.21.15.16:0/2776484396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fd0fd6390 0x557fd139d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:29.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.434+0000 7f08b7049740 1 --2- 172.21.15.16:0/2776484396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fd0ff7070 0x557fd139c990 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:29.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.434+0000 7f08b7049740 1 -- 172.21.15.16:0/2776484396 >> 172.21.15.16:0/2776484396 conn(0x557fd0feb460 msgr2=0x557fd139a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:29.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.435+0000 7f08b7049740 1 -- 172.21.15.16:0/2776484396 shutdown_connections 2024-08-31T06:07:29.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.435+0000 7f08b7049740 1 -- 172.21.15.16:0/2776484396 wait complete. 2024-08-31T06:07:29.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.436+0000 7f08b7049740 1 -- start start 2024-08-31T06:07:29.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.436+0000 7f08b7049740 1 Processor -- start 2024-08-31T06:07:29.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.437+0000 7f08b7049740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fd0fce1b0 0x557fd13a8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:29.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.437+0000 7f08b7049740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fd0fd6390 0x557fd13a8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:29.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.437+0000 7f08b7049740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fd0ff7070 0x557fd13c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:29.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.437+0000 7f08b7049740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557fd1165700 con 0x557fd0ff7070 2024-08-31T06:07:29.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.437+0000 7f08b7049740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557fd1289cf0 con 0x557fd0fd6390 2024-08-31T06:07:29.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.437+0000 7f08b7049740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557fd13bc180 con 0x557fd0fce1b0 2024-08-31T06:07:29.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.437+0000 7f08b7049740 10 client.0 _ll_register_callbacks cb 0x557fd12af470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:07:29.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.437+0000 7f08ad000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fd0fd6390 0x557fd13a8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:29.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.437+0000 7f08ad000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fd0fd6390 0x557fd13a8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:56624/0 (socket says 172.21.15.16:56624) 2024-08-31T06:07:29.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.437+0000 7f08ad000640 1 -- 172.21.15.16:0/161985326 learned_addr learned my addr 172.21.15.16:0/161985326 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:29.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.438+0000 7f08ad000640 1 -- 172.21.15.16:0/161985326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fd0fce1b0 msgr2=0x557fd13a8550 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T06:07:29.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.438+0000 7f08ad000640 1 --2- 172.21.15.16:0/161985326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fd0fce1b0 0x557fd13a8550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:29.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.438+0000 7f08ad000640 1 -- 172.21.15.16:0/161985326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fd0ff7070 msgr2=0x557fd13c54a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:07:29.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.439+0000 7f08ae400640 1 --2- 172.21.15.16:0/161985326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fd0ff7070 0x557fd13c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:29.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.439+0000 7f08ad000640 1 --2- 172.21.15.16:0/161985326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fd0ff7070 0x557fd13c54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:29.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.439+0000 7f08ad000640 1 -- 172.21.15.16:0/161985326 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fd12ca8b0 con 0x557fd0fd6390 2024-08-31T06:07:29.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.440+0000 7f08ad000640 1 --2- 172.21.15.16:0/161985326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fd0fd6390 0x557fd13a8a90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f089400c640 tx=0x7f08940aaa30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb1b34c481e09b6a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:29.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.440+0000 7f088f400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:07:29.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.441+0000 7f088f400640 1 -- 172.21.15.16:0/161985326 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08940ab200 con 0x557fd0fd6390 2024-08-31T06:07:29.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.441+0000 7f088f400640 1 -- 172.21.15.16:0/161985326 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f08940e24e0 con 0x557fd0fd6390 2024-08-31T06:07:29.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.441+0000 7f08b7049740 1 -- 172.21.15.16:0/161985326 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f089c01a910 con 0x557fd0fd6390 2024-08-31T06:07:29.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.441+0000 7f088f400640 1 -- 172.21.15.16:0/161985326 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f08940afa70 con 0x557fd0fd6390 2024-08-31T06:07:29.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.441+0000 7f08b7049740 10 client.25544 fetch_fsmap learned FSMap version 28 2024-08-31T06:07:29.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.442+0000 7f08b7049740 1 -- 172.21.15.16:0/161985326 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557fd13c8140 con 0x557fd0fd6390 2024-08-31T06:07:29.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.442+0000 7f088f400640 1 -- 172.21.15.16:0/161985326 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f089c01a910 con 0x557fd0fd6390 2024-08-31T06:07:29.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.443+0000 7f088f400640 1 -- 172.21.15.16:0/161985326 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 28) ==== 8350+0+0 (secure 0 0 0) 0x7f08940040e0 con 0x557fd0fd6390 2024-08-31T06:07:29.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.443+0000 7f08b7049740 10 client.25544 fetch_fsmap finished waiting for FSMap version 28 2024-08-31T06:07:29.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.443+0000 7f08b7049740 10 client.25544 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T06:07:29.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.443+0000 7f08b7049740 20 client.25544 populate_metadata read hostname 'smithi016' 2024-08-31T06:07:29.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.443+0000 7f08b7049740 1 --2- 172.21.15.16:0/161985326 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x557fd0fb59f0 0x557fd13cb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:29.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.443+0000 7f08b7049740 4 client.25544 mds_command: new command op to 24712 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:07:29.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.443+0000 7f08b7049740 1 -- 172.21.15.16:0/161985326 --> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557fd1315700 con 0x557fd0fb59f0 2024-08-31T06:07:29.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.444+0000 7f08ada00640 1 --2- 172.21.15.16:0/161985326 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x557fd0fb59f0 0x557fd13cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:29.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.445+0000 7f08ada00640 1 --2- 172.21.15.16:0/161985326 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x557fd0fb59f0 0x557fd13cb690 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=28722f8b9f56cd9c server_cookie=397a7c8ccb24e604 in_seq=0 out_seq=0 2024-08-31T06:07:29.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.445+0000 7f088f400640 10 client.25544 ms_handle_connect on v2:172.21.15.184:6838/4011051479 2024-08-31T06:07:29.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.446+0000 7f088f400640 1 -- 172.21.15.16:0/161985326 <== mds.2 v2:172.21.15.184:6838/4011051479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557fd1315700 con 0x557fd0fb59f0 2024-08-31T06:07:29.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.446+0000 7f088f400640 10 client.25544 handle_command_reply: tid=0 2024-08-31T06:07:29.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.446+0000 7f08b7049740 1 client.25544 shutdown 2024-08-31T06:07:29.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.447+0000 7f08b7049740 1 -- 172.21.15.16:0/161985326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fd0fd6390 msgr2=0x557fd13a8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:29.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.447+0000 7f08b7049740 1 --2- 172.21.15.16:0/161985326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fd0fd6390 0x557fd13a8a90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f089400c640 tx=0x7f08940aaa30 comp rx=0 tx=0).stop 2024-08-31T06:07:29.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.448+0000 7f08b7049740 1 -- 172.21.15.16:0/161985326 shutdown_connections 2024-08-31T06:07:29.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.448+0000 7f08b7049740 1 --2- 172.21.15.16:0/161985326 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x557fd0fb59f0 0x557fd13cb690 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:29.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.448+0000 7f088f400640 0 client.25544 ms_handle_reset on v2:172.21.15.184:6838/4011051479 2024-08-31T06:07:29.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.448+0000 7f08b7049740 1 --2- 172.21.15.16:0/161985326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fd0ff7070 0x557fd13c54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:29.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.448+0000 7f08b7049740 1 --2- 172.21.15.16:0/161985326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fd0fd6390 0x557fd13a8a90 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:29.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.448+0000 7f08b7049740 1 --2- 172.21.15.16:0/161985326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fd0fce1b0 0x557fd13a8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:29.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.448+0000 7f08b7049740 1 -- 172.21.15.16:0/161985326 >> 172.21.15.16:0/161985326 conn(0x557fd0feb460 msgr2=0x557fd139fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:29.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.448+0000 7f08b7049740 1 -- 172.21.15.16:0/161985326 shutdown_connections 2024-08-31T06:07:29.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.448+0000 7f08b7049740 1 -- 172.21.15.16:0/161985326 wait complete. 2024-08-31T06:07:29.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.449+0000 7f08b7049740 20 client.25544 trim_cache size 0 max 16384 2024-08-31T06:07:29.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:28 smithi184 ceph-mon[22331]: pgmap v371: 97 pgs: 97 active+clean; 393 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 69 MiB/s wr, 281 op/s 2024-08-31T06:07:29.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.474+0000 7f08b7049740 1 Processor -- start 2024-08-31T06:07:29.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.475+0000 7f08b7049740 1 -- start start 2024-08-31T06:07:29.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.475+0000 7f08b7049740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fd0fb59f0 0x557fd13c53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:29.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.475+0000 7f08b7049740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fd0fce1b0 0x557fd13a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:29.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.475+0000 7f08b7049740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fd0fd6390 0x557fd13a5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:29.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.475+0000 7f08b7049740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f08a81afaa0 con 0x557fd0fce1b0 2024-08-31T06:07:29.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.475+0000 7f08b7049740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f08a81ad450 con 0x557fd0fd6390 2024-08-31T06:07:29.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.475+0000 7f08b7049740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f08a811ae20 con 0x557fd0fb59f0 2024-08-31T06:07:29.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.475+0000 7f08ae400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fd0fd6390 0x557fd13a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:29.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.475+0000 7f08ae400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fd0fd6390 0x557fd13a5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:56630/0 (socket says 172.21.15.16:56630) 2024-08-31T06:07:29.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.475+0000 7f08ae400640 1 -- 172.21.15.16:0/4052396721 learned_addr learned my addr 172.21.15.16:0/4052396721 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:29.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.476+0000 7f08ae400640 1 -- 172.21.15.16:0/4052396721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fd0fb59f0 msgr2=0x557fd13c53a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:07:29.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.476+0000 7f08ae400640 1 --2- 172.21.15.16:0/4052396721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fd0fb59f0 0x557fd13c53a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:29.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.476+0000 7f08ae400640 1 -- 172.21.15.16:0/4052396721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fd0fce1b0 msgr2=0x557fd13a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:29.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.476+0000 7f08ae400640 1 --2- 172.21.15.16:0/4052396721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fd0fce1b0 0x557fd13a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:29.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.477+0000 7f08ae400640 1 -- 172.21.15.16:0/4052396721 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fd139f7f0 con 0x557fd0fd6390 2024-08-31T06:07:29.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.477+0000 7f08ae400640 1 --2- 172.21.15.16:0/4052396721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fd0fd6390 0x557fd13a5540 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f08a0030800 tx=0x7f08a0005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c621379ac5117de1 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:29.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.477+0000 7f089a000640 1 -- 172.21.15.16:0/4052396721 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a0036040 con 0x557fd0fd6390 2024-08-31T06:07:29.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.478+0000 7f089a000640 1 -- 172.21.15.16:0/4052396721 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08a0043070 con 0x557fd0fd6390 2024-08-31T06:07:29.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.478+0000 7f089a000640 1 -- 172.21.15.16:0/4052396721 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a003f680 con 0x557fd0fd6390 2024-08-31T06:07:29.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.478+0000 7f08b7049740 1 -- 172.21.15.16:0/4052396721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fd0fd6390 msgr2=0x557fd13a5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:29.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.478+0000 7f08b7049740 1 --2- 172.21.15.16:0/4052396721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fd0fd6390 0x557fd13a5540 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f08a0030800 tx=0x7f08a0005fd0 comp rx=0 tx=0).stop 2024-08-31T06:07:29.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.479+0000 7f08b7049740 1 -- 172.21.15.16:0/4052396721 shutdown_connections 2024-08-31T06:07:29.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.479+0000 7f08b7049740 1 --2- 172.21.15.16:0/4052396721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fd0fd6390 0x557fd13a5540 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:29.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.479+0000 7f08b7049740 1 --2- 172.21.15.16:0/4052396721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fd0fce1b0 0x557fd13a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:29.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.479+0000 7f08b7049740 1 --2- 172.21.15.16:0/4052396721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fd0fb59f0 0x557fd13c53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:29.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.479+0000 7f08b7049740 1 -- 172.21.15.16:0/4052396721 >> 172.21.15.16:0/4052396721 conn(0x557fd0feb460 msgr2=0x557fd139fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:29.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.479+0000 7f08b7049740 1 -- 172.21.15.16:0/4052396721 shutdown_connections 2024-08-31T06:07:29.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.479+0000 7f08b7049740 1 -- 172.21.15.16:0/4052396721 wait complete. 2024-08-31T06:07:29.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.480+0000 7f08b7049740 1 -- start start 2024-08-31T06:07:29.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.480+0000 7f08b7049740 1 Processor -- start 2024-08-31T06:07:29.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.481+0000 7f08b7049740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fd0fb59f0 0x557fd13bdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:29.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.481+0000 7f08b7049740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fd0fce1b0 0x557fd13be320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:29.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.481+0000 7f08b7049740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fd0fd6390 0x557fd142f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:29.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.481+0000 7f08b7049740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f08a811b490 con 0x557fd0fb59f0 2024-08-31T06:07:29.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.481+0000 7f08b7049740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f08a811c900 con 0x557fd0fd6390 2024-08-31T06:07:29.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.481+0000 7f08b7049740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f08a811bb60 con 0x557fd0fce1b0 2024-08-31T06:07:29.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.481+0000 7f08b7049740 10 client.0 _ll_register_callbacks cb 0x7f08940aab50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:07:29.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.482+0000 7f08ad000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fd0fce1b0 0x557fd13be320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:29.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.482+0000 7f08ae400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fd0fd6390 0x557fd142f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:29.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.483+0000 7f08ad000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fd0fce1b0 0x557fd13be320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:36548/0 (socket says 172.21.15.16:36548) 2024-08-31T06:07:29.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.483+0000 7f08ad000640 1 -- 172.21.15.16:0/3614569693 learned_addr learned my addr 172.21.15.16:0/3614569693 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:29.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.483+0000 7f08ae400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fd0fd6390 0x557fd142f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:56642/0 (socket says 172.21.15.16:56642) 2024-08-31T06:07:29.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.483+0000 7f08ada00640 1 --2- 172.21.15.16:0/3614569693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fd0fb59f0 0x557fd13bdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:29.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.484+0000 7f08ae400640 1 -- 172.21.15.16:0/3614569693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fd0fce1b0 msgr2=0x557fd13be320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:29.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.484+0000 7f08ae400640 1 --2- 172.21.15.16:0/3614569693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fd0fce1b0 0x557fd13be320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:29.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.484+0000 7f08ae400640 1 -- 172.21.15.16:0/3614569693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fd0fb59f0 msgr2=0x557fd13bdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:29.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.484+0000 7f08ae400640 1 --2- 172.21.15.16:0/3614569693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fd0fb59f0 0x557fd13bdd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:29.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.484+0000 7f08ae400640 1 -- 172.21.15.16:0/3614569693 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fd142f690 con 0x557fd0fd6390 2024-08-31T06:07:29.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.484+0000 7f08ada00640 1 --2- 172.21.15.16:0/3614569693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fd0fb59f0 0x557fd13bdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:07:29.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.484+0000 7f08ae400640 1 --2- 172.21.15.16:0/3614569693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fd0fd6390 0x557fd142f020 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f08a002acd0 tx=0x7f08a00319b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e7027fa70e18141 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:29.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.484+0000 7f08ad000640 1 --2- 172.21.15.16:0/3614569693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fd0fce1b0 0x557fd13be320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:07:29.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.484+0000 7f088f400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:07:29.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.485+0000 7f088f400640 1 -- 172.21.15.16:0/3614569693 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a0036050 con 0x557fd0fd6390 2024-08-31T06:07:29.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.485+0000 7f088f400640 1 -- 172.21.15.16:0/3614569693 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f08a003f620 con 0x557fd0fd6390 2024-08-31T06:07:29.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.485+0000 7f08b7049740 1 -- 172.21.15.16:0/3614569693 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08a0043070 con 0x557fd0fd6390 2024-08-31T06:07:29.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.485+0000 7f088f400640 1 -- 172.21.15.16:0/3614569693 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f08a003f820 con 0x557fd0fd6390 2024-08-31T06:07:29.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.485+0000 7f08b7049740 10 client.25550 fetch_fsmap learned FSMap version 28 2024-08-31T06:07:29.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.485+0000 7f08b7049740 1 -- 172.21.15.16:0/3614569693 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557fd1436a60 con 0x557fd0fd6390 2024-08-31T06:07:29.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.486+0000 7f088f400640 1 -- 172.21.15.16:0/3614569693 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08a0043070 con 0x557fd0fd6390 2024-08-31T06:07:29.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.486+0000 7f088f400640 1 -- 172.21.15.16:0/3614569693 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 28) ==== 8350+0+0 (secure 0 0 0) 0x7f08a0048070 con 0x557fd0fd6390 2024-08-31T06:07:29.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.487+0000 7f08b7049740 10 client.25550 fetch_fsmap finished waiting for FSMap version 28 2024-08-31T06:07:29.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.487+0000 7f08b7049740 10 client.25550 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T06:07:29.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.487+0000 7f08b7049740 20 client.25550 populate_metadata read hostname 'smithi016' 2024-08-31T06:07:29.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.487+0000 7f08b7049740 1 --2- 172.21.15.16:0/3614569693 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x557fd0ff7070 0x557fd14390c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:29.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.487+0000 7f08b7049740 4 client.25550 mds_command: new command op to 24712 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:07:29.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.487+0000 7f08b7049740 1 -- 172.21.15.16:0/3614569693 --> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557fd13111a0 con 0x557fd0ff7070 2024-08-31T06:07:29.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.488+0000 7f08ada00640 1 --2- 172.21.15.16:0/3614569693 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x557fd0ff7070 0x557fd14390c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:29.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.490+0000 7f08ada00640 1 --2- 172.21.15.16:0/3614569693 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x557fd0ff7070 0x557fd14390c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2b6a73cd06081017 server_cookie=b973547f1f9676c6 in_seq=0 out_seq=0 2024-08-31T06:07:29.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.490+0000 7f088f400640 10 client.25550 ms_handle_connect on v2:172.21.15.184:6838/4011051479 2024-08-31T06:07:29.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.490+0000 7f088f400640 1 -- 172.21.15.16:0/3614569693 <== mds.2 v2:172.21.15.184:6838/4011051479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557fd13111a0 con 0x557fd0ff7070 2024-08-31T06:07:29.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.490+0000 7f088f400640 10 client.25550 handle_command_reply: tid=0 2024-08-31T06:07:29.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.491+0000 7f08b7049740 1 client.25550 shutdown 2024-08-31T06:07:29.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.491+0000 7f08b7049740 1 -- 172.21.15.16:0/3614569693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fd0fd6390 msgr2=0x557fd142f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:29.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.491+0000 7f08b7049740 1 --2- 172.21.15.16:0/3614569693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fd0fd6390 0x557fd142f020 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f08a002acd0 tx=0x7f08a00319b0 comp rx=0 tx=0).stop 2024-08-31T06:07:29.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.491+0000 7f08b7049740 1 -- 172.21.15.16:0/3614569693 shutdown_connections 2024-08-31T06:07:29.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.491+0000 7f08b7049740 1 --2- 172.21.15.16:0/3614569693 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x557fd0ff7070 0x557fd14390c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:29.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.491+0000 7f08b7049740 1 --2- 172.21.15.16:0/3614569693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fd0fce1b0 0x557fd13be320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:29.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.491+0000 7f08b7049740 1 --2- 172.21.15.16:0/3614569693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fd0fd6390 0x557fd142f020 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:29.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.491+0000 7f08b7049740 1 --2- 172.21.15.16:0/3614569693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fd0fb59f0 0x557fd13bdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:29.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.491+0000 7f08b7049740 1 -- 172.21.15.16:0/3614569693 >> 172.21.15.16:0/3614569693 conn(0x557fd0feb460 msgr2=0x557fd139fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:29.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.491+0000 7f088f400640 0 client.25550 ms_handle_reset on v2:172.21.15.184:6838/4011051479 2024-08-31T06:07:29.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.492+0000 7f08b7049740 1 -- 172.21.15.16:0/3614569693 shutdown_connections 2024-08-31T06:07:29.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.492+0000 7f08b7049740 1 -- 172.21.15.16:0/3614569693 wait complete. 2024-08-31T06:07:29.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.493+0000 7f08b7049740 20 client.25550 trim_cache size 0 max 16384 2024-08-31T06:07:29.497 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:07:29.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.496+0000 7f08b0400640 1 -- 172.21.15.16:0/2566087832 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f08900b4dd0 msgr2=0x7f08900b72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:29.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.496+0000 7f08b0400640 1 --2- 172.21.15.16:0/2566087832 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f08900b4dd0 0x7f08900b72e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f08a00099a0 tx=0x7f08a00304a0 comp rx=0 tx=0).stop 2024-08-31T06:07:29.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.496+0000 7f08b0400640 1 -- 172.21.15.16:0/2566087832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08a8108150 msgr2=0x7f08a810e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:29.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.496+0000 7f08b0400640 1 --2- 172.21.15.16:0/2566087832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08a8108150 0x7f08a810e880 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0894009a50 tx=0x7f0894009f10 comp rx=0 tx=0).stop 2024-08-31T06:07:29.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.497+0000 7f08b0400640 1 -- 172.21.15.16:0/2566087832 shutdown_connections 2024-08-31T06:07:29.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.497+0000 7f08b0400640 1 --2- 172.21.15.16:0/2566087832 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f08900b4dd0 0x7f08900b72e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:29.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.497+0000 7f08b0400640 1 --2- 172.21.15.16:0/2566087832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08a8108aa0 0x7f08a81a2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:29.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.497+0000 7f08b0400640 1 --2- 172.21.15.16:0/2566087832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08a8108150 0x7f08a810e880 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:29.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.497+0000 7f08b0400640 1 --2- 172.21.15.16:0/2566087832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08a8102860 0x7f08a810e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:29.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.497+0000 7f08b0400640 1 -- 172.21.15.16:0/2566087832 >> 172.21.15.16:0/2566087832 conn(0x7f08a80794c0 msgr2=0x7f08a8113ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:29.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.497+0000 7f08b0400640 1 -- 172.21.15.16:0/2566087832 shutdown_connections 2024-08-31T06:07:29.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:29.497+0000 7f08b0400640 1 -- 172.21.15.16:0/2566087832 wait complete. 2024-08-31T06:07:30.335 INFO:tasks.quiescer.fs.[cephfs]:Total duration (52.8) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-08-31T06:07:30.335 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.4 2024-08-31T06:07:30.335 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:07:30.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:29 smithi155 ceph-mon[21742]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:07:30.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:29 smithi016 ceph-mon[17194]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:07:30.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:29 smithi184 ceph-mon[22331]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:07:30.534 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T06:07:31.002 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:07:31.310 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:30 smithi016 ceph-mon[17194]: pgmap v372: 97 pgs: 97 active+clean; 457 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 62 MiB/s wr, 168 op/s 2024-08-31T06:07:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.317+0000 7fe1f4c00640 1 -- 172.21.15.16:0/1284231019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1f0101710 msgr2=0x7fe1f0101b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.317+0000 7fe1f4c00640 1 --2- 172.21.15.16:0/1284231019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1f0101710 0x7fe1f0101b20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0009ca0 tx=0x7fe1e002a8d0 comp rx=0 tx=0).stop 2024-08-31T06:07:31.321 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.318+0000 7fe1f4c00640 1 -- 172.21.15.16:0/1284231019 shutdown_connections 2024-08-31T06:07:31.321 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.318+0000 7fe1f4c00640 1 --2- 172.21.15.16:0/1284231019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1f0102060 0x7fe1f010f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:31.321 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.318+0000 7fe1f4c00640 1 --2- 172.21.15.16:0/1284231019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1f010fb50 0x7fe1f0112060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:31.321 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.319+0000 7fe1f4c00640 1 --2- 172.21.15.16:0/1284231019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1f0101710 0x7fe1f0101b20 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:31.321 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.319+0000 7fe1f4c00640 1 -- 172.21.15.16:0/1284231019 >> 172.21.15.16:0/1284231019 conn(0x7fe1f0078510 msgr2=0x7fe1f0078920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:31.321 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.319+0000 7fe1f4c00640 1 -- 172.21.15.16:0/1284231019 shutdown_connections 2024-08-31T06:07:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.319+0000 7fe1f4c00640 1 -- 172.21.15.16:0/1284231019 wait complete. 2024-08-31T06:07:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.321+0000 7fe1f4c00640 1 Processor -- start 2024-08-31T06:07:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.322+0000 7fe1f4c00640 1 -- start start 2024-08-31T06:07:31.325 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.323+0000 7fe1f4c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1f0101710 0x7fe1f019c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:31.325 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.323+0000 7fe1f4c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1f0102060 0x7fe1f019c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:31.326 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.323+0000 7fe1f4c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1f010fb50 0x7fe1f01a1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:31.326 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.324+0000 7fe1f4c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f01157e0 con 0x7fe1f0101710 2024-08-31T06:07:31.326 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.324+0000 7fe1efe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1f010fb50 0x7fe1f01a1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:31.326 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.324+0000 7fe1efe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1f010fb50 0x7fe1f01a1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:36570/0 (socket says 172.21.15.16:36570) 2024-08-31T06:07:31.326 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.324+0000 7fe1efe00640 1 -- 172.21.15.16:0/228329540 learned_addr learned my addr 172.21.15.16:0/228329540 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:31.326 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.324+0000 7fe1f4c00640 1 -- 172.21.15.16:0/228329540 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f0115660 con 0x7fe1f0102060 2024-08-31T06:07:31.327 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.324+0000 7fe1f4c00640 1 -- 172.21.15.16:0/228329540 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f0115960 con 0x7fe1f010fb50 2024-08-31T06:07:31.327 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.325+0000 7fe1ef400640 1 --2- 172.21.15.16:0/228329540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1f0101710 0x7fe1f019c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:31.328 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.325+0000 7fe1eea00640 1 --2- 172.21.15.16:0/228329540 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1f0102060 0x7fe1f019c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:31.328 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.325+0000 7fe1efe00640 1 -- 172.21.15.16:0/228329540 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1f0102060 msgr2=0x7fe1f019c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:31.328 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.325+0000 7fe1efe00640 1 --2- 172.21.15.16:0/228329540 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1f0102060 0x7fe1f019c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:31.328 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.325+0000 7fe1efe00640 1 -- 172.21.15.16:0/228329540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1f0101710 msgr2=0x7fe1f019c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:31.328 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.325+0000 7fe1efe00640 1 --2- 172.21.15.16:0/228329540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1f0101710 0x7fe1f019c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:31.328 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.325+0000 7fe1efe00640 1 -- 172.21.15.16:0/228329540 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f01a21a0 con 0x7fe1f010fb50 2024-08-31T06:07:31.328 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.325+0000 7fe1eea00640 1 --2- 172.21.15.16:0/228329540 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1f0102060 0x7fe1f019c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:07:31.328 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.325+0000 7fe1ef400640 1 --2- 172.21.15.16:0/228329540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1f0101710 0x7fe1f019c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:07:31.328 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.326+0000 7fe1efe00640 1 --2- 172.21.15.16:0/228329540 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1f010fb50 0x7fe1f01a1b30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe1e800c9b0 tx=0x7fe1e800ce70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:31.329 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.327+0000 7fe1e7e00640 1 -- 172.21.15.16:0/228329540 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e8007c70 con 0x7fe1f010fb50 2024-08-31T06:07:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.327+0000 7fe1e7e00640 1 -- 172.21.15.16:0/228329540 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1e8007e10 con 0x7fe1f010fb50 2024-08-31T06:07:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.327+0000 7fe1e7e00640 1 -- 172.21.15.16:0/228329540 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e800f470 con 0x7fe1f010fb50 2024-08-31T06:07:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.327+0000 7fe1f4c00640 1 -- 172.21.15.16:0/228329540 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1f01a2490 con 0x7fe1f010fb50 2024-08-31T06:07:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.327+0000 7fe1f4c00640 1 -- 172.21.15.16:0/228329540 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1f01a8cb0 con 0x7fe1f010fb50 2024-08-31T06:07:31.337 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.335+0000 7fe1fd3e5740 1 Processor -- start 2024-08-31T06:07:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.339+0000 7fe1e7e00640 1 -- 172.21.15.16:0/228329540 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fe1e8003710 con 0x7fe1f010fb50 2024-08-31T06:07:31.343 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.340+0000 7fe1fd3e5740 1 -- start start 2024-08-31T06:07:31.345 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.343+0000 7fe1e7e00640 1 --2- 172.21.15.16:0/228329540 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe1d40b4da0 0x7fe1d40b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.343+0000 7fe1fd3e5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f64153e1b0 0x55f6418ea0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.343+0000 7fe1fd3e5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6415152f0 0x55f6418ea5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.343+0000 7fe1fd3e5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f64151d4d0 0x55f6418eab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.343+0000 7fe1fd3e5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f641856f70 con 0x55f64151d4d0 2024-08-31T06:07:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.343+0000 7fe1eea00640 1 --2- 172.21.15.16:0/228329540 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe1d40b4da0 0x7fe1d40b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.343+0000 7fe1fd3e5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f641846500 con 0x55f6415152f0 2024-08-31T06:07:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.343+0000 7fe1fd3e5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f64184a4b0 con 0x55f64153e1b0 2024-08-31T06:07:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.344+0000 7fe1ef400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f64153e1b0 0x55f6418ea0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.344+0000 7fe1ef400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f64153e1b0 0x55f6418ea0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:36584/0 (socket says 172.21.15.16:36584) 2024-08-31T06:07:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.344+0000 7fe1eea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6415152f0 0x55f6418ea5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.344+0000 7fe1ef400640 1 -- 172.21.15.16:0/4246493604 learned_addr learned my addr 172.21.15.16:0/4246493604 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.344+0000 7fe1ef400640 1 -- 172.21.15.16:0/4246493604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6415152f0 msgr2=0x55f6418ea5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.344+0000 7fe1ef400640 1 --2- 172.21.15.16:0/4246493604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6415152f0 0x55f6418ea5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.344+0000 7fe1ef400640 1 -- 172.21.15.16:0/4246493604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f64151d4d0 msgr2=0x55f6418eab30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:07:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.345+0000 7fe1e7e00640 1 -- 172.21.15.16:0/228329540 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fe1e80e52d0 con 0x7fe1f010fb50 2024-08-31T06:07:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.345+0000 7fe1eea00640 1 --2- 172.21.15.16:0/228329540 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe1d40b4da0 0x7fe1d40b72b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0078dc0 tx=0x7fe1dc009ad0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.345+0000 7fe1ef400640 1 --2- 172.21.15.16:0/4246493604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f64151d4d0 0x55f6418eab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.345+0000 7fe1ef400640 1 -- 172.21.15.16:0/4246493604 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f641800c10 con 0x55f64153e1b0 2024-08-31T06:07:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.346+0000 7fe1ef400640 1 --2- 172.21.15.16:0/4246493604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f64153e1b0 0x55f6418ea0b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0006010 tx=0x7fe1e0009890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32c97997477663b0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.346+0000 7fe1dbe00640 1 -- 172.21.15.16:0/4246493604 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0030800 con 0x55f64153e1b0 2024-08-31T06:07:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.346+0000 7fe1dbe00640 1 -- 172.21.15.16:0/4246493604 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1e003f070 con 0x55f64153e1b0 2024-08-31T06:07:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.346+0000 7fe1dbe00640 1 -- 172.21.15.16:0/4246493604 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0004500 con 0x55f64153e1b0 2024-08-31T06:07:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.347+0000 7fe1fd3e5740 1 -- 172.21.15.16:0/4246493604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f64153e1b0 msgr2=0x55f6418ea0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.347+0000 7fe1fd3e5740 1 --2- 172.21.15.16:0/4246493604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f64153e1b0 0x55f6418ea0b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0006010 tx=0x7fe1e0009890 comp rx=0 tx=0).stop 2024-08-31T06:07:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.348+0000 7fe1fd3e5740 1 -- 172.21.15.16:0/4246493604 shutdown_connections 2024-08-31T06:07:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.348+0000 7fe1fd3e5740 1 --2- 172.21.15.16:0/4246493604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f64151d4d0 0x55f6418eab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.348+0000 7fe1fd3e5740 1 --2- 172.21.15.16:0/4246493604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6415152f0 0x55f6418ea5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.348+0000 7fe1fd3e5740 1 --2- 172.21.15.16:0/4246493604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f64153e1b0 0x55f6418ea0b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.348+0000 7fe1fd3e5740 1 -- 172.21.15.16:0/4246493604 >> 172.21.15.16:0/4246493604 conn(0x55f641532300 msgr2=0x55f6418e7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.348+0000 7fe1fd3e5740 1 -- 172.21.15.16:0/4246493604 shutdown_connections 2024-08-31T06:07:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.348+0000 7fe1fd3e5740 1 -- 172.21.15.16:0/4246493604 wait complete. 2024-08-31T06:07:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.349+0000 7fe1fd3e5740 1 -- start start 2024-08-31T06:07:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.349+0000 7fe1fd3e5740 1 Processor -- start 2024-08-31T06:07:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.350+0000 7fe1fd3e5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6415152f0 0x55f6418f57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.350+0000 7fe1fd3e5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f64151d4d0 0x55f6418f5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.350+0000 7fe1fd3e5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f64153e1b0 0x55f641901ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.350+0000 7fe1fd3e5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f64177fab0 con 0x55f64153e1b0 2024-08-31T06:07:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.350+0000 7fe1fd3e5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f6417ed320 con 0x55f6415152f0 2024-08-31T06:07:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.350+0000 7fe1fd3e5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f6418f87d0 con 0x55f64151d4d0 2024-08-31T06:07:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.350+0000 7fe1fd3e5740 10 client.0 _ll_register_callbacks cb 0x55f641826e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:07:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.350+0000 7fe1efe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f64153e1b0 0x55f641901ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.350+0000 7fe1ef400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6415152f0 0x55f6418f57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.350+0000 7fe1efe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f64153e1b0 0x55f641901ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45824/0 (socket says 172.21.15.16:45824) 2024-08-31T06:07:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.350+0000 7fe1efe00640 1 -- 172.21.15.16:0/2925206558 learned_addr learned my addr 172.21.15.16:0/2925206558 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.351+0000 7fe1ef400640 1 -- 172.21.15.16:0/2925206558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f64151d4d0 msgr2=0x55f6418f5d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:07:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.351+0000 7fe1eea00640 1 --2- 172.21.15.16:0/2925206558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f64151d4d0 0x55f6418f5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.351+0000 7fe1ef400640 1 --2- 172.21.15.16:0/2925206558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f64151d4d0 0x55f6418f5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.351+0000 7fe1ef400640 1 -- 172.21.15.16:0/2925206558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f64153e1b0 msgr2=0x55f641901ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.351+0000 7fe1ef400640 1 --2- 172.21.15.16:0/2925206558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f64153e1b0 0x55f641901ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.351+0000 7fe1ef400640 1 -- 172.21.15.16:0/2925206558 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6415a9790 con 0x55f6415152f0 2024-08-31T06:07:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.351+0000 7fe1efe00640 1 --2- 172.21.15.16:0/2925206558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f64153e1b0 0x55f641901ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:07:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.351+0000 7fe1eea00640 1 --2- 172.21.15.16:0/2925206558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f64151d4d0 0x55f6418f5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:07:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.352+0000 7fe1ef400640 1 --2- 172.21.15.16:0/2925206558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6415152f0 0x55f6418f57c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0000f80 tx=0x7fe1e0030710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c11bfe932575b09 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.352+0000 7fe1d8c00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:07:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.352+0000 7fe1d8c00640 1 -- 172.21.15.16:0/2925206558 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0003b80 con 0x55f6415152f0 2024-08-31T06:07:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.352+0000 7fe1fd3e5740 1 -- 172.21.15.16:0/2925206558 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1e003f070 con 0x55f6415152f0 2024-08-31T06:07:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.352+0000 7fe1d8c00640 1 -- 172.21.15.16:0/2925206558 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fe1e0003d20 con 0x55f6415152f0 2024-08-31T06:07:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.353+0000 7fe1d8c00640 1 -- 172.21.15.16:0/2925206558 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe1e0004280 con 0x55f6415152f0 2024-08-31T06:07:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.353+0000 7fe1fd3e5740 10 client.25559 fetch_fsmap learned FSMap version 28 2024-08-31T06:07:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.353+0000 7fe1fd3e5740 1 -- 172.21.15.16:0/2925206558 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f641622620 con 0x55f6415152f0 2024-08-31T06:07:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.353+0000 7fe1d8c00640 1 -- 172.21.15.16:0/2925206558 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1e003f070 con 0x55f6415152f0 2024-08-31T06:07:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.354+0000 7fe1d8c00640 1 -- 172.21.15.16:0/2925206558 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 28) ==== 8350+0+0 (secure 0 0 0) 0x7fe1e0058ac0 con 0x55f6415152f0 2024-08-31T06:07:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.354+0000 7fe1fd3e5740 10 client.25559 fetch_fsmap finished waiting for FSMap version 28 2024-08-31T06:07:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.354+0000 7fe1fd3e5740 10 client.25559 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:07:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.354+0000 7fe1fd3e5740 20 client.25559 populate_metadata read hostname 'smithi016' 2024-08-31T06:07:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.354+0000 7fe1fd3e5740 1 --2- 172.21.15.16:0/2925206558 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f6414fcd80 0x55f6419076d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.354+0000 7fe1fd3e5740 4 client.25559 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:07:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.354+0000 7fe1fd3e5740 1 -- 172.21.15.16:0/2925206558 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f641858ec0 con 0x55f6414fcd80 2024-08-31T06:07:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.355+0000 7fe1eea00640 1 --2- 172.21.15.16:0/2925206558 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f6414fcd80 0x55f6419076d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:31.359 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.356+0000 7fe1eea00640 1 --2- 172.21.15.16:0/2925206558 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f6414fcd80 0x55f6419076d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a3fa005a547b050 server_cookie=56805c64306381e1 in_seq=0 out_seq=0 2024-08-31T06:07:31.359 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.356+0000 7fe1d8c00640 10 client.25559 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:31.359 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.357+0000 7fe1d8c00640 1 -- 172.21.15.16:0/2925206558 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f641858ec0 con 0x55f6414fcd80 2024-08-31T06:07:31.359 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.357+0000 7fe1d8c00640 10 client.25559 handle_command_reply: tid=0 2024-08-31T06:07:31.359 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.357+0000 7fe1fd3e5740 1 client.25559 shutdown 2024-08-31T06:07:31.360 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.358+0000 7fe1fd3e5740 1 -- 172.21.15.16:0/2925206558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6415152f0 msgr2=0x55f6418f57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:31.360 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.358+0000 7fe1fd3e5740 1 --2- 172.21.15.16:0/2925206558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6415152f0 0x55f6418f57c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0000f80 tx=0x7fe1e0030710 comp rx=0 tx=0).stop 2024-08-31T06:07:31.360 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.358+0000 7fe1fd3e5740 1 -- 172.21.15.16:0/2925206558 shutdown_connections 2024-08-31T06:07:31.360 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.358+0000 7fe1fd3e5740 1 --2- 172.21.15.16:0/2925206558 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f6414fcd80 0x55f6419076d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:31.361 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.358+0000 7fe1fd3e5740 1 --2- 172.21.15.16:0/2925206558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f64151d4d0 0x55f6418f5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:31.361 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.358+0000 7fe1d8c00640 0 client.25559 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:31.361 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.358+0000 7fe1fd3e5740 1 --2- 172.21.15.16:0/2925206558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f64153e1b0 0x55f641901ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:31.361 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.358+0000 7fe1fd3e5740 1 --2- 172.21.15.16:0/2925206558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6415152f0 0x55f6418f57c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:31.361 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.358+0000 7fe1fd3e5740 1 -- 172.21.15.16:0/2925206558 >> 172.21.15.16:0/2925206558 conn(0x55f641532300 msgr2=0x55f6418eb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:31.361 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.359+0000 7fe1fd3e5740 1 -- 172.21.15.16:0/2925206558 shutdown_connections 2024-08-31T06:07:31.361 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.359+0000 7fe1fd3e5740 1 -- 172.21.15.16:0/2925206558 wait complete. 2024-08-31T06:07:31.361 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.359+0000 7fe1fd3e5740 20 client.25559 trim_cache size 0 max 16384 2024-08-31T06:07:31.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:30 smithi155 ceph-mon[21742]: pgmap v372: 97 pgs: 97 active+clean; 457 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 62 MiB/s wr, 168 op/s 2024-08-31T06:07:31.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.386+0000 7fe1fd3e5740 1 Processor -- start 2024-08-31T06:07:31.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.387+0000 7fe1fd3e5740 1 -- start start 2024-08-31T06:07:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.387+0000 7fe1fd3e5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6414fcd80 0x55f641901a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.387+0000 7fe1fd3e5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6415152f0 0x55f6418f73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.387+0000 7fe1fd3e5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f64151d4d0 0x55f6418f7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.387+0000 7fe1fd3e5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f01af9f0 con 0x55f6415152f0 2024-08-31T06:07:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.387+0000 7fe1fd3e5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f01ad290 con 0x55f6414fcd80 2024-08-31T06:07:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.387+0000 7fe1fd3e5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f01ab0e0 con 0x55f64151d4d0 2024-08-31T06:07:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.388+0000 7fe1ef400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6414fcd80 0x55f641901a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.388+0000 7fe1ef400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6414fcd80 0x55f641901a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:56668/0 (socket says 172.21.15.16:56668) 2024-08-31T06:07:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.388+0000 7fe1ef400640 1 -- 172.21.15.16:0/3029099388 learned_addr learned my addr 172.21.15.16:0/3029099388 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.388+0000 7fe1efe00640 1 --2- 172.21.15.16:0/3029099388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f64151d4d0 0x55f6418f7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.388+0000 7fe1eea00640 1 --2- 172.21.15.16:0/3029099388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6415152f0 0x55f6418f73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.388+0000 7fe1ef400640 1 -- 172.21.15.16:0/3029099388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f64151d4d0 msgr2=0x55f6418f7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.388+0000 7fe1ef400640 1 --2- 172.21.15.16:0/3029099388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f64151d4d0 0x55f6418f7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.388+0000 7fe1ef400640 1 -- 172.21.15.16:0/3029099388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6415152f0 msgr2=0x55f6418f73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.388+0000 7fe1ef400640 1 --2- 172.21.15.16:0/3029099388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6415152f0 0x55f6418f73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.388+0000 7fe1ef400640 1 -- 172.21.15.16:0/3029099388 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6418f8130 con 0x55f6414fcd80 2024-08-31T06:07:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.389+0000 7fe1eea00640 1 --2- 172.21.15.16:0/3029099388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6415152f0 0x55f6418f73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:07:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.389+0000 7fe1ef400640 1 --2- 172.21.15.16:0/3029099388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6414fcd80 0x55f641901a30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0058fd0 tx=0x7fe1e0005d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0fa4b239cd09bab server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.389+0000 7fe1db400640 1 -- 172.21.15.16:0/3029099388 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0030b10 con 0x55f6414fcd80 2024-08-31T06:07:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.389+0000 7fe1db400640 1 -- 172.21.15.16:0/3029099388 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1e005d020 con 0x55f6414fcd80 2024-08-31T06:07:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.389+0000 7fe1db400640 1 -- 172.21.15.16:0/3029099388 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0044960 con 0x55f6414fcd80 2024-08-31T06:07:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.390+0000 7fe1fd3e5740 1 -- 172.21.15.16:0/3029099388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6414fcd80 msgr2=0x55f641901a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.391+0000 7fe1fd3e5740 1 --2- 172.21.15.16:0/3029099388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6414fcd80 0x55f641901a30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0058fd0 tx=0x7fe1e0005d80 comp rx=0 tx=0).stop 2024-08-31T06:07:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.391+0000 7fe1fd3e5740 1 -- 172.21.15.16:0/3029099388 shutdown_connections 2024-08-31T06:07:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.391+0000 7fe1fd3e5740 1 --2- 172.21.15.16:0/3029099388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f64151d4d0 0x55f6418f7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:31.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.391+0000 7fe1fd3e5740 1 --2- 172.21.15.16:0/3029099388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6415152f0 0x55f6418f73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:31.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.391+0000 7fe1fd3e5740 1 --2- 172.21.15.16:0/3029099388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6414fcd80 0x55f641901a30 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:31.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.391+0000 7fe1fd3e5740 1 -- 172.21.15.16:0/3029099388 >> 172.21.15.16:0/3029099388 conn(0x55f641532300 msgr2=0x55f6418f6ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:31.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.391+0000 7fe1fd3e5740 1 -- 172.21.15.16:0/3029099388 shutdown_connections 2024-08-31T06:07:31.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.391+0000 7fe1fd3e5740 1 -- 172.21.15.16:0/3029099388 wait complete. 2024-08-31T06:07:31.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.392+0000 7fe1fd3e5740 1 -- start start 2024-08-31T06:07:31.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.392+0000 7fe1fd3e5740 1 Processor -- start 2024-08-31T06:07:31.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.393+0000 7fe1fd3e5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6414fcd80 0x55f641908a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:31.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.393+0000 7fe1fd3e5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6415152f0 0x55f641908fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:31.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.393+0000 7fe1fd3e5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f64151d4d0 0x55f641970e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.393+0000 7fe1ef400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6414fcd80 0x55f641908a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.393+0000 7fe1ef400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6414fcd80 0x55f641908a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:36610/0 (socket says 172.21.15.16:36610) 2024-08-31T06:07:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.393+0000 7fe1eea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6415152f0 0x55f641908fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.394+0000 7fe1eea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6415152f0 0x55f641908fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:56682/0 (socket says 172.21.15.16:56682) 2024-08-31T06:07:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.394+0000 7fe1fd3e5740 1 -- 172.21.15.16:0/1067054629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f01ab750 con 0x55f64151d4d0 2024-08-31T06:07:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.394+0000 7fe1ef400640 1 -- 172.21.15.16:0/1067054629 learned_addr learned my addr 172.21.15.16:0/1067054629 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.394+0000 7fe1fd3e5740 1 -- 172.21.15.16:0/1067054629 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f01acbc0 con 0x55f6415152f0 2024-08-31T06:07:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.394+0000 7fe1fd3e5740 1 -- 172.21.15.16:0/1067054629 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f01abe20 con 0x55f6414fcd80 2024-08-31T06:07:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.394+0000 7fe1fd3e5740 10 client.0 _ll_register_callbacks cb 0x7fe1dc015420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:07:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.394+0000 7fe1efe00640 1 --2- 172.21.15.16:0/1067054629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f64151d4d0 0x55f641970e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.394+0000 7fe1ef400640 1 -- 172.21.15.16:0/1067054629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6415152f0 msgr2=0x55f641908fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.394+0000 7fe1ef400640 1 --2- 172.21.15.16:0/1067054629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6415152f0 0x55f641908fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.394+0000 7fe1ef400640 1 -- 172.21.15.16:0/1067054629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f64151d4d0 msgr2=0x55f641970e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.395+0000 7fe1ef400640 1 --2- 172.21.15.16:0/1067054629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f64151d4d0 0x55f641970e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.395+0000 7fe1ef400640 1 -- 172.21.15.16:0/1067054629 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f641971390 con 0x55f6414fcd80 2024-08-31T06:07:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.395+0000 7fe1eea00640 1 --2- 172.21.15.16:0/1067054629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6415152f0 0x55f641908fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:07:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.395+0000 7fe1efe00640 1 --2- 172.21.15.16:0/1067054629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f64151d4d0 0x55f641970e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:07:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.396+0000 7fe1ef400640 1 --2- 172.21.15.16:0/1067054629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6414fcd80 0x55f641908a90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe1e003d040 tx=0x7fe1e0030760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7c46eb7e3739700 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.396+0000 7fe1d8c00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:07:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.396+0000 7fe1d8c00640 1 -- 172.21.15.16:0/1067054629 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0004840 con 0x55f6414fcd80 2024-08-31T06:07:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.396+0000 7fe1d8c00640 1 -- 172.21.15.16:0/1067054629 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fe1e004a040 con 0x55f6414fcd80 2024-08-31T06:07:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.396+0000 7fe1fd3e5740 1 -- 172.21.15.16:0/1067054629 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1e005d020 con 0x55f6414fcd80 2024-08-31T06:07:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.397+0000 7fe1d8c00640 1 -- 172.21.15.16:0/1067054629 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe1e0044b70 con 0x55f6414fcd80 2024-08-31T06:07:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.397+0000 7fe1fd3e5740 10 client.25558 fetch_fsmap learned FSMap version 28 2024-08-31T06:07:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.397+0000 7fe1fd3e5740 1 -- 172.21.15.16:0/1067054629 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f641972470 con 0x55f6414fcd80 2024-08-31T06:07:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.397+0000 7fe1d8c00640 1 -- 172.21.15.16:0/1067054629 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1e005d020 con 0x55f6414fcd80 2024-08-31T06:07:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.398+0000 7fe1d8c00640 1 -- 172.21.15.16:0/1067054629 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 28) ==== 8350+0+0 (secure 0 0 0) 0x7fe1e00588b0 con 0x55f6414fcd80 2024-08-31T06:07:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.399+0000 7fe1fd3e5740 10 client.25558 fetch_fsmap finished waiting for FSMap version 28 2024-08-31T06:07:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.399+0000 7fe1fd3e5740 10 client.25558 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:07:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.399+0000 7fe1fd3e5740 20 client.25558 populate_metadata read hostname 'smithi016' 2024-08-31T06:07:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.399+0000 7fe1fd3e5740 1 --2- 172.21.15.16:0/1067054629 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f64153e1b0 0x55f641974ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.399+0000 7fe1fd3e5740 4 client.25558 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:07:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.399+0000 7fe1fd3e5740 1 -- 172.21.15.16:0/1067054629 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f641854960 con 0x55f64153e1b0 2024-08-31T06:07:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.399+0000 7fe1eea00640 1 --2- 172.21.15.16:0/1067054629 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f64153e1b0 0x55f641974ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.400+0000 7fe1eea00640 1 --2- 172.21.15.16:0/1067054629 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f64153e1b0 0x55f641974ad0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67ede5ed476e705b server_cookie=c6f07bc410dc6f35 in_seq=0 out_seq=0 2024-08-31T06:07:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.400+0000 7fe1d8c00640 10 client.25558 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.437+0000 7fe1d8c00640 1 -- 172.21.15.16:0/1067054629 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f641854960 con 0x55f64153e1b0 2024-08-31T06:07:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.437+0000 7fe1d8c00640 10 client.25558 handle_command_reply: tid=0 2024-08-31T06:07:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.438+0000 7fe1fd3e5740 1 client.25558 shutdown 2024-08-31T06:07:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.438+0000 7fe1fd3e5740 1 -- 172.21.15.16:0/1067054629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6414fcd80 msgr2=0x55f641908a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.438+0000 7fe1fd3e5740 1 --2- 172.21.15.16:0/1067054629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6414fcd80 0x55f641908a90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe1e003d040 tx=0x7fe1e0030760 comp rx=0 tx=0).stop 2024-08-31T06:07:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.439+0000 7fe1fd3e5740 1 -- 172.21.15.16:0/1067054629 shutdown_connections 2024-08-31T06:07:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.439+0000 7fe1fd3e5740 1 --2- 172.21.15.16:0/1067054629 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f64153e1b0 0x55f641974ad0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.439+0000 7fe1fd3e5740 1 --2- 172.21.15.16:0/1067054629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f64151d4d0 0x55f641970e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.439+0000 7fe1fd3e5740 1 --2- 172.21.15.16:0/1067054629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6415152f0 0x55f641908fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.439+0000 7fe1fd3e5740 1 --2- 172.21.15.16:0/1067054629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6414fcd80 0x55f641908a90 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.439+0000 7fe1fd3e5740 1 -- 172.21.15.16:0/1067054629 >> 172.21.15.16:0/1067054629 conn(0x55f641532300 msgr2=0x55f6418e7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.439+0000 7fe1d8c00640 0 client.25558 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.439+0000 7fe1fd3e5740 1 -- 172.21.15.16:0/1067054629 shutdown_connections 2024-08-31T06:07:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.440+0000 7fe1fd3e5740 1 -- 172.21.15.16:0/1067054629 wait complete. 2024-08-31T06:07:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.440+0000 7fe1fd3e5740 20 client.25558 trim_cache size 0 max 16384 2024-08-31T06:07:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:07:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:07:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:07:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:07:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.442+0000 7fe1f4c00640 1 -- 172.21.15.16:0/228329540 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe1d40b4da0 msgr2=0x7fe1d40b72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.442+0000 7fe1f4c00640 1 --2- 172.21.15.16:0/228329540 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe1d40b4da0 0x7fe1d40b72b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0078dc0 tx=0x7fe1dc009ad0 comp rx=0 tx=0).stop 2024-08-31T06:07:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.442+0000 7fe1f4c00640 1 -- 172.21.15.16:0/228329540 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1f010fb50 msgr2=0x7fe1f01a1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.442+0000 7fe1f4c00640 1 --2- 172.21.15.16:0/228329540 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1f010fb50 0x7fe1f01a1b30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe1e800c9b0 tx=0x7fe1e800ce70 comp rx=0 tx=0).stop 2024-08-31T06:07:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.442+0000 7fe1f4c00640 1 -- 172.21.15.16:0/228329540 shutdown_connections 2024-08-31T06:07:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.442+0000 7fe1f4c00640 1 --2- 172.21.15.16:0/228329540 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe1d40b4da0 0x7fe1d40b72b0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.442+0000 7fe1f4c00640 1 --2- 172.21.15.16:0/228329540 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1f010fb50 0x7fe1f01a1b30 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.442+0000 7fe1f4c00640 1 --2- 172.21.15.16:0/228329540 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1f0102060 0x7fe1f019c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.442+0000 7fe1f4c00640 1 --2- 172.21.15.16:0/228329540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1f0101710 0x7fe1f019c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.442+0000 7fe1f4c00640 1 -- 172.21.15.16:0/228329540 >> 172.21.15.16:0/228329540 conn(0x7fe1f0078510 msgr2=0x7fe1f0110260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.442+0000 7fe1f4c00640 1 -- 172.21.15.16:0/228329540 shutdown_connections 2024-08-31T06:07:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:31.443+0000 7fe1f4c00640 1 -- 172.21.15.16:0/228329540 wait complete. 2024-08-31T06:07:31.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:30 smithi184 ceph-mon[22331]: pgmap v372: 97 pgs: 97 active+clean; 457 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 62 MiB/s wr, 168 op/s 2024-08-31T06:07:31.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 5.0 s, 2604.3 tps, lat 36.611 ms stddev 40.690 2024-08-31T06:07:31.802 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:07:31.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:31.959+0000 7fa86de00640 1 -- 172.21.15.16:0/243937576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa86810b490 msgr2=0x7fa86810b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:31.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:31.959+0000 7fa86de00640 1 --2- 172.21.15.16:0/243937576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa86810b490 0x7fa86810b8a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa854009900 tx=0x7fa85402a3f0 comp rx=0 tx=0).stop 2024-08-31T06:07:31.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:31.960+0000 7fa86de00640 1 -- 172.21.15.16:0/243937576 shutdown_connections 2024-08-31T06:07:31.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:31.960+0000 7fa86de00640 1 --2- 172.21.15.16:0/243937576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa86810bde0 0x7fa868112610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:31.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:31.960+0000 7fa86de00640 1 --2- 172.21.15.16:0/243937576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa86810b490 0x7fa86810b8a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:31.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:31.960+0000 7fa86de00640 1 --2- 172.21.15.16:0/243937576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa868105b50 0x7fa868105f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:31.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:31.960+0000 7fa86de00640 1 -- 172.21.15.16:0/243937576 >> 172.21.15.16:0/243937576 conn(0x7fa868101560 msgr2=0x7fa8681039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:31.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:31.960+0000 7fa86de00640 1 -- 172.21.15.16:0/243937576 shutdown_connections 2024-08-31T06:07:31.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:31.960+0000 7fa86de00640 1 -- 172.21.15.16:0/243937576 wait complete. 2024-08-31T06:07:31.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:31.962+0000 7fa86de00640 1 Processor -- start 2024-08-31T06:07:31.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:31.962+0000 7fa86de00640 1 -- start start 2024-08-31T06:07:31.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:31.963+0000 7fa86de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa868105b50 0x7fa868112550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:31.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:31.964+0000 7fa86de00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa86810b490 0x7fa868110850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:31.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:31.964+0000 7fa86de00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa86810bde0 0x7fa868110f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:31.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:31.964+0000 7fa86de00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa868077b70 con 0x7fa868105b50 2024-08-31T06:07:31.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:31.964+0000 7fa86de00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa8680779f0 con 0x7fa86810bde0 2024-08-31T06:07:31.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:31.964+0000 7fa86de00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa868077cf0 con 0x7fa86810b490 2024-08-31T06:07:31.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:31.964+0000 7fa867400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa86810bde0 0x7fa868110f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:31.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:31.964+0000 7fa866000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa86810b490 0x7fa868110850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:31.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:31.964+0000 7fa867400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa86810bde0 0x7fa868110f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:56704/0 (socket says 172.21.15.16:56704) 2024-08-31T06:07:31.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:31.964+0000 7fa867400640 1 -- 172.21.15.16:0/2523987943 learned_addr learned my addr 172.21.15.16:0/2523987943 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:31.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:31.965+0000 7fa866a00640 1 --2- 172.21.15.16:0/2523987943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa868105b50 0x7fa868112550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:31.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:31.965+0000 7fa866000640 1 -- 172.21.15.16:0/2523987943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa86810bde0 msgr2=0x7fa868110f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:31.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:31.965+0000 7fa866000640 1 --2- 172.21.15.16:0/2523987943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa86810bde0 0x7fa868110f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:31.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:31.965+0000 7fa866000640 1 -- 172.21.15.16:0/2523987943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa868105b50 msgr2=0x7fa868112550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:31.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:31.965+0000 7fa866000640 1 --2- 172.21.15.16:0/2523987943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa868105b50 0x7fa868112550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:31.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:31.965+0000 7fa866000640 1 -- 172.21.15.16:0/2523987943 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8681acab0 con 0x7fa86810b490 2024-08-31T06:07:31.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:31.966+0000 7fa866000640 1 --2- 172.21.15.16:0/2523987943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa86810b490 0x7fa868110850 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa854009230 tx=0x7fa854003cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:31.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:31.966+0000 7fa86ca00640 1 -- 172.21.15.16:0/2523987943 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85403f070 con 0x7fa86810b490 2024-08-31T06:07:31.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:31.966+0000 7fa86de00640 1 -- 172.21.15.16:0/2523987943 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8681acc80 con 0x7fa86810b490 2024-08-31T06:07:31.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:31.966+0000 7fa86ca00640 1 -- 172.21.15.16:0/2523987943 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa854003e40 con 0x7fa86810b490 2024-08-31T06:07:31.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:31.966+0000 7fa86ca00640 1 -- 172.21.15.16:0/2523987943 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa854035030 con 0x7fa86810b490 2024-08-31T06:07:31.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:31.966+0000 7fa86de00640 1 -- 172.21.15.16:0/2523987943 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8681ad180 con 0x7fa86810b490 2024-08-31T06:07:31.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:31.968+0000 7fa86de00640 1 -- 172.21.15.16:0/2523987943 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa868105f60 con 0x7fa86810b490 2024-08-31T06:07:31.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:31.972+0000 7fa86ca00640 1 -- 172.21.15.16:0/2523987943 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fa854042070 con 0x7fa86810b490 2024-08-31T06:07:31.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:31.976+0000 7fa86ca00640 1 --2- 172.21.15.16:0/2523987943 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa84c0b4d50 0x7fa84c0b7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:31.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:31.976+0000 7fa86ca00640 1 -- 172.21.15.16:0/2523987943 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fa854105900 con 0x7fa86810b490 2024-08-31T06:07:31.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:31.977+0000 7fa866a00640 1 --2- 172.21.15.16:0/2523987943 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa84c0b4d50 0x7fa84c0b7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:31.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:31.979+0000 7fa866a00640 1 --2- 172.21.15.16:0/2523987943 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa84c0b4d50 0x7fa84c0b7260 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa8681122a0 tx=0x7fa858009440 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:31.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:31.982+0000 7fa86ca00640 1 -- 172.21.15.16:0/2523987943 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fa8540cc750 con 0x7fa86810b490 2024-08-31T06:07:32.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.383+0000 7fa86de00640 1 -- 172.21.15.16:0/2523987943 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8681ad420 con 0x7fa86810b490 2024-08-31T06:07:32.387 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:07:32.388 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-08-31T06:07:28:967322+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:07:28.329493+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24712,24724,24737]},"id":1}]} 2024-08-31T06:07:32.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.384+0000 7fa86ca00640 1 -- 172.21.15.16:0/2523987943 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11918 (secure 0 0 0) 0x7fa8540cc050 con 0x7fa86810b490 2024-08-31T06:07:32.388 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-08-31T06:07:32.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.388+0000 7fa86de00640 1 -- 172.21.15.16:0/2523987943 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa84c0b4d50 msgr2=0x7fa84c0b7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:32.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.388+0000 7fa86de00640 1 --2- 172.21.15.16:0/2523987943 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa84c0b4d50 0x7fa84c0b7260 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa8681122a0 tx=0x7fa858009440 comp rx=0 tx=0).stop 2024-08-31T06:07:32.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.388+0000 7fa86de00640 1 -- 172.21.15.16:0/2523987943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa86810b490 msgr2=0x7fa868110850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:32.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.388+0000 7fa86de00640 1 --2- 172.21.15.16:0/2523987943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa86810b490 0x7fa868110850 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa854009230 tx=0x7fa854003cd0 comp rx=0 tx=0).stop 2024-08-31T06:07:32.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.388+0000 7fa86de00640 1 -- 172.21.15.16:0/2523987943 shutdown_connections 2024-08-31T06:07:32.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.388+0000 7fa86de00640 1 --2- 172.21.15.16:0/2523987943 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa84c0b4d50 0x7fa84c0b7260 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:32.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.388+0000 7fa86de00640 1 --2- 172.21.15.16:0/2523987943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa86810b490 0x7fa868110850 secure :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa854009230 tx=0x7fa854003cd0 comp rx=0 tx=0).stop 2024-08-31T06:07:32.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.388+0000 7fa86de00640 1 --2- 172.21.15.16:0/2523987943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa86810bde0 0x7fa868110f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:32.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.388+0000 7fa86de00640 1 --2- 172.21.15.16:0/2523987943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa868105b50 0x7fa868112550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:32.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.388+0000 7fa86de00640 1 -- 172.21.15.16:0/2523987943 >> 172.21.15.16:0/2523987943 conn(0x7fa868101560 msgr2=0x7fa868107be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:32.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.390+0000 7fa86de00640 1 -- 172.21.15.16:0/2523987943 shutdown_connections 2024-08-31T06:07:32.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.390+0000 7fa86de00640 1 -- 172.21.15.16:0/2523987943 wait complete. 2024-08-31T06:07:32.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.662+0000 7f5578200640 1 -- 172.21.15.16:0/214898735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f557010aab0 msgr2=0x7f5570116d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:32.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.662+0000 7f5578200640 1 --2- 172.21.15.16:0/214898735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f557010aab0 0x7f5570116d60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5568009ca0 tx=0x7f556802c7c0 comp rx=0 tx=0).stop 2024-08-31T06:07:32.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.663+0000 7f5578200640 1 -- 172.21.15.16:0/214898735 shutdown_connections 2024-08-31T06:07:32.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.663+0000 7f5578200640 1 --2- 172.21.15.16:0/214898735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f557010aab0 0x7f5570116d60 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:32.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.663+0000 7f5578200640 1 --2- 172.21.15.16:0/214898735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f557010a160 0x7f557010a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:32.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.663+0000 7f5578200640 1 --2- 172.21.15.16:0/214898735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f557010eef0 0x7f557010f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:32.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.663+0000 7f5578200640 1 -- 172.21.15.16:0/214898735 >> 172.21.15.16:0/214898735 conn(0x7f5570071090 msgr2=0x7f5570073bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:32.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.665+0000 7f5578200640 1 -- 172.21.15.16:0/214898735 shutdown_connections 2024-08-31T06:07:32.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.665+0000 7f5578200640 1 -- 172.21.15.16:0/214898735 wait complete. 2024-08-31T06:07:32.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.667+0000 7f5578200640 1 Processor -- start 2024-08-31T06:07:32.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.668+0000 7f5578200640 1 -- start start 2024-08-31T06:07:32.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.669+0000 7f5578200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f557010a160 0x7f55701a7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:32.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.669+0000 7f5578200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f557010aab0 0x7f55701a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:32.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.669+0000 7f5578200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f557010eef0 0x7f55701a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:32.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.670+0000 7f5574e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f557010aab0 0x7f55701a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:32.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.670+0000 7f5574e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f557010aab0 0x7f55701a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:56724/0 (socket says 172.21.15.16:56724) 2024-08-31T06:07:32.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.670+0000 7f5575800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f557010a160 0x7f55701a7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:32.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.670+0000 7f5574e00640 1 -- 172.21.15.16:0/832728498 learned_addr learned my addr 172.21.15.16:0/832728498 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:32.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.670+0000 7f5578200640 1 -- 172.21.15.16:0/832728498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5570119e50 con 0x7f557010a160 2024-08-31T06:07:32.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.671+0000 7f5576200640 1 --2- 172.21.15.16:0/832728498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f557010eef0 0x7f55701a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:32.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.671+0000 7f5578200640 1 -- 172.21.15.16:0/832728498 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5570119cd0 con 0x7f557010aab0 2024-08-31T06:07:32.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.671+0000 7f5578200640 1 -- 172.21.15.16:0/832728498 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5570119fd0 con 0x7f557010eef0 2024-08-31T06:07:32.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.673+0000 7f5576200640 1 -- 172.21.15.16:0/832728498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f557010aab0 msgr2=0x7f55701a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:32.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.673+0000 7f5576200640 1 --2- 172.21.15.16:0/832728498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f557010aab0 0x7f55701a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:32.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.673+0000 7f5576200640 1 -- 172.21.15.16:0/832728498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f557010a160 msgr2=0x7f55701a7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:32.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.674+0000 7f5576200640 1 --2- 172.21.15.16:0/832728498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f557010a160 0x7f55701a7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:32.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.674+0000 7f5576200640 1 -- 172.21.15.16:0/832728498 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55701a1370 con 0x7f557010eef0 2024-08-31T06:07:32.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.674+0000 7f5575800640 1 --2- 172.21.15.16:0/832728498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f557010a160 0x7f55701a7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:07:32.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.676+0000 7f5576200640 1 --2- 172.21.15.16:0/832728498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f557010eef0 0x7f55701a0b50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5568009ca0 tx=0x7f55680339a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:32.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.677+0000 7f556e000640 1 -- 172.21.15.16:0/832728498 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5568031920 con 0x7f557010eef0 2024-08-31T06:07:32.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.678+0000 7f556e000640 1 -- 172.21.15.16:0/832728498 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5568003e20 con 0x7f557010eef0 2024-08-31T06:07:32.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.678+0000 7f556e000640 1 -- 172.21.15.16:0/832728498 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556803d440 con 0x7f557010eef0 2024-08-31T06:07:32.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.678+0000 7f5578200640 1 -- 172.21.15.16:0/832728498 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55701a5fb0 con 0x7f557010eef0 2024-08-31T06:07:32.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.678+0000 7f5578200640 1 -- 172.21.15.16:0/832728498 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55701a6490 con 0x7f557010eef0 2024-08-31T06:07:32.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.691+0000 7f557efc0740 1 Processor -- start 2024-08-31T06:07:32.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.692+0000 7f556e000640 1 -- 172.21.15.16:0/832728498 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f556803b040 con 0x7f557010eef0 2024-08-31T06:07:32.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.693+0000 7f557efc0740 1 -- start start 2024-08-31T06:07:32.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.693+0000 7f557efc0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac233cb070 0x55ac23770990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:32.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.694+0000 7f557efc0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac233a21b0 0x55ac23770ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:32.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.694+0000 7f557efc0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac233aa390 0x55ac23771410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:32.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.694+0000 7f557efc0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ac236e77b0 con 0x55ac233a21b0 2024-08-31T06:07:32.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.694+0000 7f557efc0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ac236d6d40 con 0x55ac233cb070 2024-08-31T06:07:32.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.694+0000 7f557efc0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ac236dacf0 con 0x55ac233aa390 2024-08-31T06:07:32.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.695+0000 7f5574e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac233cb070 0x55ac23770990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:32.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.695+0000 7f5574e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac233cb070 0x55ac23770990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:56726/0 (socket says 172.21.15.16:56726) 2024-08-31T06:07:32.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.695+0000 7f5574e00640 1 -- 172.21.15.16:0/1312389211 learned_addr learned my addr 172.21.15.16:0/1312389211 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:32.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.695+0000 7f556e000640 1 --2- 172.21.15.16:0/832728498 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f55580b4e30 0x7f55580b7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:32.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.695+0000 7f5574e00640 1 --2- 172.21.15.16:0/1312389211 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac233aa390 0x55ac23771410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:32.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.696+0000 7f556e000640 1 -- 172.21.15.16:0/832728498 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f55681063b0 con 0x7f557010eef0 2024-08-31T06:07:32.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.696+0000 7f5574e00640 1 -- 172.21.15.16:0/1312389211 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac233aa390 msgr2=0x55ac23771410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:32.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.696+0000 7f5574e00640 1 --2- 172.21.15.16:0/1312389211 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac233aa390 0x55ac23771410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:32.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.696+0000 7f5574e00640 1 -- 172.21.15.16:0/1312389211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac233a21b0 msgr2=0x55ac23770ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:07:32.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.696+0000 7f5574e00640 1 --2- 172.21.15.16:0/1312389211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac233a21b0 0x55ac23770ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:32.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.697+0000 7f5574e00640 1 -- 172.21.15.16:0/1312389211 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac23436650 con 0x55ac233cb070 2024-08-31T06:07:32.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.698+0000 7f5576200640 1 --2- 172.21.15.16:0/832728498 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f55580b4e30 0x7f55580b7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:32.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.699+0000 7f5574e00640 1 --2- 172.21.15.16:0/1312389211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac233cb070 0x55ac23770990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5560014650 tx=0x7f556000b990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8934bd97a67a803 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:32.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.700+0000 7f555ea00640 1 -- 172.21.15.16:0/1312389211 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556001a030 con 0x55ac233cb070 2024-08-31T06:07:32.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.700+0000 7f555ea00640 1 -- 172.21.15.16:0/1312389211 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5560014170 con 0x55ac233cb070 2024-08-31T06:07:32.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.700+0000 7f555ea00640 1 -- 172.21.15.16:0/1312389211 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556001a030 con 0x55ac233cb070 2024-08-31T06:07:32.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.702+0000 7f5576200640 1 --2- 172.21.15.16:0/832728498 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f55580b4e30 0x7f55580b7340 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5568103bc0 tx=0x7f55680043d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:32.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.702+0000 7f557efc0740 1 -- 172.21.15.16:0/1312389211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac233cb070 msgr2=0x55ac23770990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:32.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.702+0000 7f557efc0740 1 --2- 172.21.15.16:0/1312389211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac233cb070 0x55ac23770990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5560014650 tx=0x7f556000b990 comp rx=0 tx=0).stop 2024-08-31T06:07:32.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.702+0000 7f557efc0740 1 -- 172.21.15.16:0/1312389211 shutdown_connections 2024-08-31T06:07:32.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.702+0000 7f557efc0740 1 --2- 172.21.15.16:0/1312389211 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac233aa390 0x55ac23771410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:32.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.702+0000 7f557efc0740 1 --2- 172.21.15.16:0/1312389211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac233a21b0 0x55ac23770ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:32.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.702+0000 7f557efc0740 1 --2- 172.21.15.16:0/1312389211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac233cb070 0x55ac23770990 secure :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5560014650 tx=0x7f556000b990 comp rx=0 tx=0).stop 2024-08-31T06:07:32.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.702+0000 7f557efc0740 1 -- 172.21.15.16:0/1312389211 >> 172.21.15.16:0/1312389211 conn(0x55ac233bf460 msgr2=0x55ac2376e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:32.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.703+0000 7f557efc0740 1 -- 172.21.15.16:0/1312389211 shutdown_connections 2024-08-31T06:07:32.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.703+0000 7f557efc0740 1 -- 172.21.15.16:0/1312389211 wait complete. 2024-08-31T06:07:32.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.704+0000 7f557efc0740 1 -- start start 2024-08-31T06:07:32.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.704+0000 7f557efc0740 1 Processor -- start 2024-08-31T06:07:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.705+0000 7f557efc0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac233a21b0 0x55ac2377c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.705+0000 7f557efc0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac233aa390 0x55ac2377cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:32.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.705+0000 7f557efc0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac233899f0 0x55ac23788460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:32.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.705+0000 7f557efc0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ac23539700 con 0x55ac233aa390 2024-08-31T06:07:32.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.705+0000 7f557efc0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ac2365dcf0 con 0x55ac233a21b0 2024-08-31T06:07:32.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.705+0000 7f557efc0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ac2377f170 con 0x55ac233899f0 2024-08-31T06:07:32.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.705+0000 7f557efc0740 10 client.0 _ll_register_callbacks cb 0x55ac23683470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:07:32.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.706+0000 7f5575800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac233aa390 0x55ac2377cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:32.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.706+0000 7f5574e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac233a21b0 0x55ac2377c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:32.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.706+0000 7f5574e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac233a21b0 0x55ac2377c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:56730/0 (socket says 172.21.15.16:56730) 2024-08-31T06:07:32.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.706+0000 7f5574e00640 1 -- 172.21.15.16:0/3488878783 learned_addr learned my addr 172.21.15.16:0/3488878783 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:32.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.706+0000 7f5574e00640 1 --2- 172.21.15.16:0/3488878783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac233899f0 0x55ac23788460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:32.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.707+0000 7f5574e00640 1 -- 172.21.15.16:0/3488878783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac233899f0 msgr2=0x55ac23788460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:32.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.707+0000 7f5574e00640 1 --2- 172.21.15.16:0/3488878783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac233899f0 0x55ac23788460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:32.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.707+0000 7f5574e00640 1 -- 172.21.15.16:0/3488878783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac233aa390 msgr2=0x55ac2377cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:32.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.707+0000 7f5574e00640 1 --2- 172.21.15.16:0/3488878783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac233aa390 0x55ac2377cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:32.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.707+0000 7f5574e00640 1 -- 172.21.15.16:0/3488878783 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac2369e8b0 con 0x55ac233a21b0 2024-08-31T06:07:32.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.708+0000 7f5574e00640 1 --2- 172.21.15.16:0/3488878783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac233a21b0 0x55ac2377c670 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5560013040 tx=0x7f5560005320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b2ec6a78b2de2ec server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:32.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.708+0000 7f5553400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:07:32.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.708+0000 7f5553400640 1 -- 172.21.15.16:0/3488878783 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556001a040 con 0x55ac233a21b0 2024-08-31T06:07:32.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.709+0000 7f5553400640 1 -- 172.21.15.16:0/3488878783 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f5560004250 con 0x55ac233a21b0 2024-08-31T06:07:32.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.709+0000 7f557efc0740 1 -- 172.21.15.16:0/3488878783 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5560014170 con 0x55ac233a21b0 2024-08-31T06:07:32.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.709+0000 7f5553400640 1 -- 172.21.15.16:0/3488878783 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5560032050 con 0x55ac233a21b0 2024-08-31T06:07:32.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.711+0000 7f557efc0740 10 client.25583 fetch_fsmap learned FSMap version 28 2024-08-31T06:07:32.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.712+0000 7f557efc0740 1 -- 172.21.15.16:0/3488878783 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ac2378b230 con 0x55ac233a21b0 2024-08-31T06:07:32.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.712+0000 7f5553400640 1 -- 172.21.15.16:0/3488878783 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5560014170 con 0x55ac233a21b0 2024-08-31T06:07:32.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.713+0000 7f5553400640 1 -- 172.21.15.16:0/3488878783 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 28) ==== 8350+0+0 (secure 0 0 0) 0x7f5560010610 con 0x55ac233a21b0 2024-08-31T06:07:32.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.713+0000 7f557efc0740 10 client.25583 fetch_fsmap finished waiting for FSMap version 28 2024-08-31T06:07:32.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.713+0000 7f557efc0740 10 client.25583 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-31T06:07:32.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.713+0000 7f557efc0740 20 client.25583 populate_metadata read hostname 'smithi016' 2024-08-31T06:07:32.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.713+0000 7f557efc0740 1 --2- 172.21.15.16:0/3488878783 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55ac2335eda0 0x55ac2378e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:32.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.713+0000 7f557efc0740 4 client.25583 mds_command: new command op to 24724 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:07:32.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.713+0000 7f557efc0740 1 -- 172.21.15.16:0/3488878783 --> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac236e9700 con 0x55ac2335eda0 2024-08-31T06:07:32.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.714+0000 7f5575800640 1 --2- 172.21.15.16:0/3488878783 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55ac2335eda0 0x55ac2378e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:32.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.715+0000 7f5553400640 10 client.25583 ms_handle_connect on v2:172.21.15.16:6838/219664460 2024-08-31T06:07:32.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.717+0000 7f5575800640 1 --2- 172.21.15.16:0/3488878783 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55ac2335eda0 0x55ac2378e590 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9915fb287fe2bbc7 server_cookie=544efae0c937051e in_seq=0 out_seq=0 2024-08-31T06:07:32.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.719+0000 7f5553400640 1 -- 172.21.15.16:0/3488878783 <== mds.1 v2:172.21.15.16:6838/219664460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ac236e9700 con 0x55ac2335eda0 2024-08-31T06:07:32.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.719+0000 7f5553400640 10 client.25583 handle_command_reply: tid=0 2024-08-31T06:07:32.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.719+0000 7f557efc0740 1 client.25583 shutdown 2024-08-31T06:07:32.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.719+0000 7f557efc0740 1 -- 172.21.15.16:0/3488878783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac233a21b0 msgr2=0x55ac2377c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:32.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.719+0000 7f557efc0740 1 --2- 172.21.15.16:0/3488878783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac233a21b0 0x55ac2377c670 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5560013040 tx=0x7f5560005320 comp rx=0 tx=0).stop 2024-08-31T06:07:32.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.719+0000 7f557efc0740 1 -- 172.21.15.16:0/3488878783 shutdown_connections 2024-08-31T06:07:32.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.719+0000 7f557efc0740 1 --2- 172.21.15.16:0/3488878783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac233899f0 0x55ac23788460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:32.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.719+0000 7f557efc0740 1 --2- 172.21.15.16:0/3488878783 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55ac2335eda0 0x55ac2378e590 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:32.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.719+0000 7f557efc0740 1 --2- 172.21.15.16:0/3488878783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac233aa390 0x55ac2377cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:32.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.719+0000 7f557efc0740 1 --2- 172.21.15.16:0/3488878783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac233a21b0 0x55ac2377c670 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:32.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.719+0000 7f557efc0740 1 -- 172.21.15.16:0/3488878783 >> 172.21.15.16:0/3488878783 conn(0x55ac233bf460 msgr2=0x55ac23773b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:32.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.720+0000 7f5553400640 0 client.25583 ms_handle_reset on v2:172.21.15.16:6838/219664460 2024-08-31T06:07:32.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.720+0000 7f557efc0740 1 -- 172.21.15.16:0/3488878783 shutdown_connections 2024-08-31T06:07:32.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.720+0000 7f557efc0740 1 -- 172.21.15.16:0/3488878783 wait complete. 2024-08-31T06:07:32.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.721+0000 7f557efc0740 20 client.25583 trim_cache size 0 max 16384 2024-08-31T06:07:32.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.752+0000 7f557efc0740 1 Processor -- start 2024-08-31T06:07:32.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.753+0000 7f557efc0740 1 -- start start 2024-08-31T06:07:32.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.754+0000 7f557efc0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac233899f0 0x55ac2376e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:32.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.754+0000 7f557efc0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac233a21b0 0x55ac237883d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:32.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.754+0000 7f557efc0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac233aa390 0x55ac23788910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:32.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.754+0000 7f557efc0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f55701c9340 con 0x55ac233aa390 2024-08-31T06:07:32.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.754+0000 7f557efc0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f55701c6be0 con 0x55ac233899f0 2024-08-31T06:07:32.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.754+0000 7f557efc0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f55701c4a30 con 0x55ac233a21b0 2024-08-31T06:07:32.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.755+0000 7f5575800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac233a21b0 0x55ac237883d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:32.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.755+0000 7f5575800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac233a21b0 0x55ac237883d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:36662/0 (socket says 172.21.15.16:36662) 2024-08-31T06:07:32.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.755+0000 7f5575800640 1 -- 172.21.15.16:0/3625462631 learned_addr learned my addr 172.21.15.16:0/3625462631 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:32.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.755+0000 7f5574e00640 1 --2- 172.21.15.16:0/3625462631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac233aa390 0x55ac23788910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:32.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.755+0000 7f5574e00640 1 --2- 172.21.15.16:0/3625462631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac233899f0 0x55ac2376e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:32.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.755+0000 7f5575800640 1 -- 172.21.15.16:0/3625462631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac233899f0 msgr2=0x55ac2376e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:32.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.755+0000 7f5575800640 1 --2- 172.21.15.16:0/3625462631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac233899f0 0x55ac2376e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:32.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.755+0000 7f5575800640 1 -- 172.21.15.16:0/3625462631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac233aa390 msgr2=0x55ac23788910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:32.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.755+0000 7f5575800640 1 --2- 172.21.15.16:0/3625462631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac233aa390 0x55ac23788910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:32.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.755+0000 7f5575800640 1 -- 172.21.15.16:0/3625462631 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac2377d0b0 con 0x55ac233a21b0 2024-08-31T06:07:32.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.756+0000 7f5575800640 1 --2- 172.21.15.16:0/3625462631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac233a21b0 0x55ac237883d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5564005310 tx=0x7f55640092f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=259cb4010f69352b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:32.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.756+0000 7f555e000640 1 -- 172.21.15.16:0/3625462631 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5564005580 con 0x55ac233a21b0 2024-08-31T06:07:32.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.756+0000 7f555e000640 1 -- 172.21.15.16:0/3625462631 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5564004c90 con 0x55ac233a21b0 2024-08-31T06:07:32.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.756+0000 7f555e000640 1 -- 172.21.15.16:0/3625462631 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5564003ae0 con 0x55ac233a21b0 2024-08-31T06:07:32.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.758+0000 7f557efc0740 1 -- 172.21.15.16:0/3625462631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac233a21b0 msgr2=0x55ac237883d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:32.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.758+0000 7f557efc0740 1 --2- 172.21.15.16:0/3625462631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac233a21b0 0x55ac237883d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5564005310 tx=0x7f55640092f0 comp rx=0 tx=0).stop 2024-08-31T06:07:32.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.758+0000 7f557efc0740 1 -- 172.21.15.16:0/3625462631 shutdown_connections 2024-08-31T06:07:32.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.759+0000 7f557efc0740 1 --2- 172.21.15.16:0/3625462631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac233a21b0 0x55ac237883d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:32.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.759+0000 7f557efc0740 1 --2- 172.21.15.16:0/3625462631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac233aa390 0x55ac23788910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:32.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.759+0000 7f557efc0740 1 --2- 172.21.15.16:0/3625462631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac233899f0 0x55ac2376e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:32.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.759+0000 7f557efc0740 1 -- 172.21.15.16:0/3625462631 >> 172.21.15.16:0/3625462631 conn(0x55ac233bf460 msgr2=0x55ac2376e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:32.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.759+0000 7f557efc0740 1 -- 172.21.15.16:0/3625462631 shutdown_connections 2024-08-31T06:07:32.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.759+0000 7f557efc0740 1 -- 172.21.15.16:0/3625462631 wait complete. 2024-08-31T06:07:32.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.760+0000 7f557efc0740 1 -- start start 2024-08-31T06:07:32.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.761+0000 7f557efc0740 1 Processor -- start 2024-08-31T06:07:32.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.762+0000 7f557efc0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac233899f0 0x55ac2378ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:32.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.762+0000 7f557efc0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac233a21b0 0x55ac2378efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:32.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.762+0000 7f5574e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac233899f0 0x55ac2378ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:32.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.763+0000 7f557efc0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac233aa390 0x55ac237f7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:32.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.763+0000 7f5574e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac233899f0 0x55ac2378ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:56742/0 (socket says 172.21.15.16:56742) 2024-08-31T06:07:32.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.764+0000 7f5574e00640 1 -- 172.21.15.16:0/3566160615 learned_addr learned my addr 172.21.15.16:0/3566160615 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:32.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.764+0000 7f5575800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac233a21b0 0x55ac2378efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:32.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.765+0000 7f557efc0740 1 -- 172.21.15.16:0/3566160615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f55701c50a0 con 0x55ac233aa390 2024-08-31T06:07:32.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.765+0000 7f557efc0740 1 -- 172.21.15.16:0/3566160615 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f55701c6510 con 0x55ac233899f0 2024-08-31T06:07:32.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.765+0000 7f557efc0740 1 -- 172.21.15.16:0/3566160615 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f55701c5770 con 0x55ac233a21b0 2024-08-31T06:07:32.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.765+0000 7f557efc0740 10 client.0 _ll_register_callbacks cb 0x7f55600056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:07:32.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.766+0000 7f5574e00640 1 --2- 172.21.15.16:0/3566160615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac233aa390 0x55ac237f7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:32.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.767+0000 7f5575800640 1 -- 172.21.15.16:0/3566160615 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac233899f0 msgr2=0x55ac2378ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:32.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.768+0000 7f5575800640 1 --2- 172.21.15.16:0/3566160615 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac233899f0 0x55ac2378ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:32.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.768+0000 7f5575800640 1 -- 172.21.15.16:0/3566160615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac233aa390 msgr2=0x55ac237f7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:32.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.768+0000 7f5575800640 1 --2- 172.21.15.16:0/3566160615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac233aa390 0x55ac237f7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:32.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.769+0000 7f5575800640 1 -- 172.21.15.16:0/3566160615 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac237f8240 con 0x55ac233a21b0 2024-08-31T06:07:32.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.769+0000 7f5574e00640 1 --2- 172.21.15.16:0/3566160615 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac233899f0 0x55ac2378ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:07:32.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.770+0000 7f5575800640 1 --2- 172.21.15.16:0/3566160615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac233a21b0 0x55ac2378efa0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5564004700 tx=0x7f5564004e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbfea08c43c56e34 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:32.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.770+0000 7f5553400640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:07:32.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.771+0000 7f5553400640 1 -- 172.21.15.16:0/3566160615 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556402b020 con 0x55ac233a21b0 2024-08-31T06:07:32.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.771+0000 7f5553400640 1 -- 172.21.15.16:0/3566160615 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f5564004230 con 0x55ac233a21b0 2024-08-31T06:07:32.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.772+0000 7f557efc0740 1 -- 172.21.15.16:0/3566160615 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5564004c90 con 0x55ac233a21b0 2024-08-31T06:07:32.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.773+0000 7f5553400640 1 -- 172.21.15.16:0/3566160615 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5564005030 con 0x55ac233a21b0 2024-08-31T06:07:32.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.773+0000 7f557efc0740 10 client.25582 fetch_fsmap learned FSMap version 28 2024-08-31T06:07:32.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.773+0000 7f557efc0740 1 -- 172.21.15.16:0/3566160615 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ac237f9370 con 0x55ac233a21b0 2024-08-31T06:07:32.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.773+0000 7f5553400640 1 -- 172.21.15.16:0/3566160615 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5564004c90 con 0x55ac233a21b0 2024-08-31T06:07:32.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.773+0000 7f5553400640 1 -- 172.21.15.16:0/3566160615 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 28) ==== 8350+0+0 (secure 0 0 0) 0x7f556401ea00 con 0x55ac233a21b0 2024-08-31T06:07:32.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.773+0000 7f557efc0740 10 client.25582 fetch_fsmap finished waiting for FSMap version 28 2024-08-31T06:07:32.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.773+0000 7f557efc0740 10 client.25582 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-31T06:07:32.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.773+0000 7f557efc0740 20 client.25582 populate_metadata read hostname 'smithi016' 2024-08-31T06:07:32.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.773+0000 7f557efc0740 1 --2- 172.21.15.16:0/3566160615 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55ac237f9a90 0x55ac237fbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:32.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.773+0000 7f557efc0740 4 client.25582 mds_command: new command op to 24724 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:07:32.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.773+0000 7f557efc0740 1 -- 172.21.15.16:0/3566160615 --> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ac236e51a0 con 0x55ac237f9a90 2024-08-31T06:07:32.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.774+0000 7f5574e00640 1 --2- 172.21.15.16:0/3566160615 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55ac237f9a90 0x55ac237fbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:32.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.775+0000 7f5574e00640 1 --2- 172.21.15.16:0/3566160615 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55ac237f9a90 0x55ac237fbeb0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=68dc4bb70a6e3bb4 server_cookie=ee33f66ef223cf8f in_seq=0 out_seq=0 2024-08-31T06:07:32.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.775+0000 7f5553400640 10 client.25582 ms_handle_connect on v2:172.21.15.16:6838/219664460 2024-08-31T06:07:32.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.776+0000 7f5553400640 1 -- 172.21.15.16:0/3566160615 <== mds.1 v2:172.21.15.16:6838/219664460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ac236e51a0 con 0x55ac237f9a90 2024-08-31T06:07:32.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.776+0000 7f5553400640 10 client.25582 handle_command_reply: tid=0 2024-08-31T06:07:32.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.776+0000 7f557efc0740 1 client.25582 shutdown 2024-08-31T06:07:32.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.777+0000 7f557efc0740 1 -- 172.21.15.16:0/3566160615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac233a21b0 msgr2=0x55ac2378efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:32.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.777+0000 7f557efc0740 1 --2- 172.21.15.16:0/3566160615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac233a21b0 0x55ac2378efa0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5564004700 tx=0x7f5564004e40 comp rx=0 tx=0).stop 2024-08-31T06:07:32.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.777+0000 7f557efc0740 1 -- 172.21.15.16:0/3566160615 shutdown_connections 2024-08-31T06:07:32.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.777+0000 7f557efc0740 1 --2- 172.21.15.16:0/3566160615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac233a21b0 0x55ac2378efa0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:32.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.777+0000 7f557efc0740 1 --2- 172.21.15.16:0/3566160615 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55ac237f9a90 0x55ac237fbeb0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:32.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.777+0000 7f557efc0740 1 --2- 172.21.15.16:0/3566160615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac233aa390 0x55ac237f7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:32.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.777+0000 7f557efc0740 1 --2- 172.21.15.16:0/3566160615 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac233899f0 0x55ac2378ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:32.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.777+0000 7f557efc0740 1 -- 172.21.15.16:0/3566160615 >> 172.21.15.16:0/3566160615 conn(0x55ac233bf460 msgr2=0x55ac2376ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:32.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.777+0000 7f5553400640 0 client.25582 ms_handle_reset on v2:172.21.15.16:6838/219664460 2024-08-31T06:07:32.781 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:07:32.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.778+0000 7f557efc0740 1 -- 172.21.15.16:0/3566160615 shutdown_connections 2024-08-31T06:07:32.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.778+0000 7f557efc0740 1 -- 172.21.15.16:0/3566160615 wait complete. 2024-08-31T06:07:32.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.778+0000 7f557efc0740 20 client.25582 trim_cache size 0 max 16384 2024-08-31T06:07:32.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.781+0000 7f5578200640 1 -- 172.21.15.16:0/832728498 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f55580b4e30 msgr2=0x7f55580b7340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:32.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.781+0000 7f5578200640 1 --2- 172.21.15.16:0/832728498 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f55580b4e30 0x7f55580b7340 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5568103bc0 tx=0x7f55680043d0 comp rx=0 tx=0).stop 2024-08-31T06:07:32.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.781+0000 7f5578200640 1 -- 172.21.15.16:0/832728498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f557010eef0 msgr2=0x7f55701a0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:32.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.781+0000 7f5578200640 1 --2- 172.21.15.16:0/832728498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f557010eef0 0x7f55701a0b50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5568009ca0 tx=0x7f55680339a0 comp rx=0 tx=0).stop 2024-08-31T06:07:32.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.781+0000 7f5578200640 1 -- 172.21.15.16:0/832728498 shutdown_connections 2024-08-31T06:07:32.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.781+0000 7f5578200640 1 --2- 172.21.15.16:0/832728498 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f55580b4e30 0x7f55580b7340 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:32.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.781+0000 7f5578200640 1 --2- 172.21.15.16:0/832728498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f557010eef0 0x7f55701a0b50 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:32.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.781+0000 7f5578200640 1 --2- 172.21.15.16:0/832728498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f557010aab0 0x7f55701a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:32.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.781+0000 7f5578200640 1 --2- 172.21.15.16:0/832728498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f557010a160 0x7f55701a7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:32.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.781+0000 7f5578200640 1 -- 172.21.15.16:0/832728498 >> 172.21.15.16:0/832728498 conn(0x7f5570071090 msgr2=0x7f557010d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:32.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.781+0000 7f5578200640 1 -- 172.21.15.16:0/832728498 shutdown_connections 2024-08-31T06:07:32.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:32.782+0000 7f5578200640 1 -- 172.21.15.16:0/832728498 wait complete. 2024-08-31T06:07:33.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:33 smithi016 ceph-mon[17194]: pgmap v373: 97 pgs: 97 active+clean; 502 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 59 MiB/s wr, 188 op/s 2024-08-31T06:07:33.274 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:33 smithi184 ceph-mon[22331]: pgmap v373: 97 pgs: 97 active+clean; 502 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 59 MiB/s wr, 188 op/s 2024-08-31T06:07:33.274 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:33 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/2523987943' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:07:33.275 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:33 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/2523987943' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:07:33.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:33 smithi155 ceph-mon[21742]: pgmap v373: 97 pgs: 97 active+clean; 502 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 59 MiB/s wr, 188 op/s 2024-08-31T06:07:33.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:33 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/2523987943' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:07:33.988 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24737 quiesce db / --timeout 90 --expiration 146.4 --await-for 45.0' 2024-08-31T06:07:33.988 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24737 quiesce db / --timeout 90 --expiration 146.4 --await-for 45.0 2024-08-31T06:07:34.298 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:34 smithi016 ceph-mon[17194]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:07:34.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:34 smithi155 ceph-mon[21742]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:07:34.385 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T06:07:34.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:34 smithi184 ceph-mon[22331]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:07:34.643 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:07:35.081 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:35 smithi016 ceph-mon[17194]: pgmap v374: 97 pgs: 97 active+clean; 683 MiB data, 723 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 53 MiB/s wr, 221 op/s 2024-08-31T06:07:35.081 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:35 smithi016 ceph-mon[17194]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:07:35.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:35 smithi155 ceph-mon[21742]: pgmap v374: 97 pgs: 97 active+clean; 683 MiB data, 723 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 53 MiB/s wr, 221 op/s 2024-08-31T06:07:35.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:35 smithi155 ceph-mon[21742]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:07:35.458 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:07:35.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:35 smithi184 ceph-mon[22331]: pgmap v374: 97 pgs: 97 active+clean; 683 MiB data, 723 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 53 MiB/s wr, 221 op/s 2024-08-31T06:07:35.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:35 smithi184 ceph-mon[22331]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:07:35.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.531+0000 7fd4ed400640 1 -- 172.21.15.16:0/1774539197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4e81050c0 msgr2=0x7fd4e810d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:35.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.531+0000 7fd4ed400640 1 --2- 172.21.15.16:0/1774539197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4e81050c0 0x7fd4e810d610 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4009900 tx=0x7fd4d402a410 comp rx=0 tx=0).stop 2024-08-31T06:07:35.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.531+0000 7fd4ed400640 1 -- 172.21.15.16:0/1774539197 shutdown_connections 2024-08-31T06:07:35.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.531+0000 7fd4ed400640 1 --2- 172.21.15.16:0/1774539197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4e81050c0 0x7fd4e810d610 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:35.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.531+0000 7fd4ed400640 1 --2- 172.21.15.16:0/1774539197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4e810dbd0 0x7fd4e81100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:35.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.531+0000 7fd4ed400640 1 --2- 172.21.15.16:0/1774539197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4e8104770 0x7fd4e8104b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:35.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.531+0000 7fd4ed400640 1 -- 172.21.15.16:0/1774539197 >> 172.21.15.16:0/1774539197 conn(0x7fd4e8079500 msgr2=0x7fd4e8079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:35.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.532+0000 7fd4ed400640 1 -- 172.21.15.16:0/1774539197 shutdown_connections 2024-08-31T06:07:35.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.532+0000 7fd4ed400640 1 -- 172.21.15.16:0/1774539197 wait complete. 2024-08-31T06:07:35.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.534+0000 7fd4ed400640 1 Processor -- start 2024-08-31T06:07:35.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.534+0000 7fd4ed400640 1 -- start start 2024-08-31T06:07:35.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.534+0000 7fd4ed400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4e8104770 0x7fd4e819a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:35.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.534+0000 7fd4ed400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4e81050c0 0x7fd4e819a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:35.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.534+0000 7fd4ed400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4e810dbd0 0x7fd4e819fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:35.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.534+0000 7fd4ed400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e8113840 con 0x7fd4e8104770 2024-08-31T06:07:35.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.534+0000 7fd4ed400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e81136c0 con 0x7fd4e810dbd0 2024-08-31T06:07:35.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.535+0000 7fd4ed400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e81139c0 con 0x7fd4e81050c0 2024-08-31T06:07:35.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.535+0000 7fd4e6800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4e8104770 0x7fd4e819a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:35.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.535+0000 7fd4e5e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4e81050c0 0x7fd4e819a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:35.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.535+0000 7fd4e5e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4e81050c0 0x7fd4e819a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:36676/0 (socket says 172.21.15.16:36676) 2024-08-31T06:07:35.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.535+0000 7fd4e5e00640 1 -- 172.21.15.16:0/2974784537 learned_addr learned my addr 172.21.15.16:0/2974784537 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:35.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.535+0000 7fd4e7200640 1 --2- 172.21.15.16:0/2974784537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4e810dbd0 0x7fd4e819fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:35.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.536+0000 7fd4e7200640 1 -- 172.21.15.16:0/2974784537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4e81050c0 msgr2=0x7fd4e819a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:35.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.536+0000 7fd4e7200640 1 --2- 172.21.15.16:0/2974784537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4e81050c0 0x7fd4e819a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:35.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.536+0000 7fd4e7200640 1 -- 172.21.15.16:0/2974784537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4e8104770 msgr2=0x7fd4e819a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:35.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.536+0000 7fd4e7200640 1 --2- 172.21.15.16:0/2974784537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4e8104770 0x7fd4e819a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:35.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.537+0000 7fd4e7200640 1 -- 172.21.15.16:0/2974784537 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4e81a01c0 con 0x7fd4e810dbd0 2024-08-31T06:07:35.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.537+0000 7fd4e6800640 1 --2- 172.21.15.16:0/2974784537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4e8104770 0x7fd4e819a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:07:35.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.537+0000 7fd4e7200640 1 --2- 172.21.15.16:0/2974784537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4e810dbd0 0x7fd4e819fa90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd4e000e940 tx=0x7fd4e000ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:35.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.537+0000 7fd4df400640 1 -- 172.21.15.16:0/2974784537 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e0004d20 con 0x7fd4e810dbd0 2024-08-31T06:07:35.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.538+0000 7fd4df400640 1 -- 172.21.15.16:0/2974784537 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4e0005020 con 0x7fd4e810dbd0 2024-08-31T06:07:35.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.538+0000 7fd4df400640 1 -- 172.21.15.16:0/2974784537 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e00116d0 con 0x7fd4e810dbd0 2024-08-31T06:07:35.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.538+0000 7fd4ed400640 1 -- 172.21.15.16:0/2974784537 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4e81a04b0 con 0x7fd4e810dbd0 2024-08-31T06:07:35.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.538+0000 7fd4ed400640 1 -- 172.21.15.16:0/2974784537 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4e8102390 con 0x7fd4e810dbd0 2024-08-31T06:07:35.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.543+0000 7fd4df400640 1 -- 172.21.15.16:0/2974784537 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fd4e0005460 con 0x7fd4e810dbd0 2024-08-31T06:07:35.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.546+0000 7fd4df400640 1 --2- 172.21.15.16:0/2974784537 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd4cc0b4d20 0x7fd4cc0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:35.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.547+0000 7fd4df400640 1 -- 172.21.15.16:0/2974784537 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fd4e00e54b0 con 0x7fd4e810dbd0 2024-08-31T06:07:35.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.549+0000 7fd4f4202740 1 Processor -- start 2024-08-31T06:07:35.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.550+0000 7fd4e6800640 1 --2- 172.21.15.16:0/2974784537 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd4cc0b4d20 0x7fd4cc0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:35.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.551+0000 7fd4f4202740 1 -- start start 2024-08-31T06:07:35.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.553+0000 7fd4f4202740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a66cad840 0x563a67054110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:35.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.553+0000 7fd4f4202740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a66c84980 0x563a67054650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:35.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.553+0000 7fd4f4202740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a66c8cb60 0x563a67054b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:35.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.553+0000 7fd4f4202740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563a66fc9530 con 0x563a66c84980 2024-08-31T06:07:35.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.553+0000 7fd4f4202740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563a66fb8ac0 con 0x563a66cad840 2024-08-31T06:07:35.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.553+0000 7fd4f4202740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563a66fbca70 con 0x563a66c8cb60 2024-08-31T06:07:35.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.554+0000 7fd4e6800640 1 --2- 172.21.15.16:0/2974784537 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd4cc0b4d20 0x7fd4cc0b7230 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8005fd0 tx=0x7fd4d800b410 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:35.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.555+0000 7fd4e6800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a66cad840 0x563a67054110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:35.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.556+0000 7fd4e7200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a66c8cb60 0x563a67054b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:35.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.556+0000 7fd4e7200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a66c8cb60 0x563a67054b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:36688/0 (socket says 172.21.15.16:36688) 2024-08-31T06:07:35.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.556+0000 7fd4e7200640 1 -- 172.21.15.16:0/2712343414 learned_addr learned my addr 172.21.15.16:0/2712343414 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:35.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.556+0000 7fd4e5e00640 1 --2- 172.21.15.16:0/2712343414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a66c84980 0x563a67054650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:35.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.556+0000 7fd4e7200640 1 -- 172.21.15.16:0/2712343414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a66cad840 msgr2=0x563a67054110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:35.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.556+0000 7fd4e7200640 1 --2- 172.21.15.16:0/2712343414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a66cad840 0x563a67054110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:35.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.556+0000 7fd4e7200640 1 -- 172.21.15.16:0/2712343414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a66c84980 msgr2=0x563a67054650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:35.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.556+0000 7fd4e7200640 1 --2- 172.21.15.16:0/2712343414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a66c84980 0x563a67054650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:35.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.556+0000 7fd4e7200640 1 -- 172.21.15.16:0/2712343414 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a66f6b880 con 0x563a66c8cb60 2024-08-31T06:07:35.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.557+0000 7fd4e7200640 1 --2- 172.21.15.16:0/2712343414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a66c8cb60 0x563a67054b90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd4e00a7cd0 tx=0x7fd4e00a73e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee2012d9beee85c3 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:35.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.557+0000 7fd4d3400640 1 -- 172.21.15.16:0/2712343414 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e001f040 con 0x563a66c8cb60 2024-08-31T06:07:35.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.557+0000 7fd4d3400640 1 -- 172.21.15.16:0/2712343414 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4e00b0410 con 0x563a66c8cb60 2024-08-31T06:07:35.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.557+0000 7fd4d3400640 1 -- 172.21.15.16:0/2712343414 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e00aaeb0 con 0x563a66c8cb60 2024-08-31T06:07:35.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.558+0000 7fd4f4202740 1 -- 172.21.15.16:0/2712343414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a66c8cb60 msgr2=0x563a67054b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:35.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.559+0000 7fd4f4202740 1 --2- 172.21.15.16:0/2712343414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a66c8cb60 0x563a67054b90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd4e00a7cd0 tx=0x7fd4e00a73e0 comp rx=0 tx=0).stop 2024-08-31T06:07:35.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.559+0000 7fd4f4202740 1 -- 172.21.15.16:0/2712343414 shutdown_connections 2024-08-31T06:07:35.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.559+0000 7fd4f4202740 1 --2- 172.21.15.16:0/2712343414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a66c8cb60 0x563a67054b90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:35.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.559+0000 7fd4f4202740 1 --2- 172.21.15.16:0/2712343414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a66c84980 0x563a67054650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:35.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.559+0000 7fd4f4202740 1 --2- 172.21.15.16:0/2712343414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a66cad840 0x563a67054110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:35.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.559+0000 7fd4f4202740 1 -- 172.21.15.16:0/2712343414 >> 172.21.15.16:0/2712343414 conn(0x563a66ca1990 msgr2=0x563a67052000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:35.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.559+0000 7fd4f4202740 1 -- 172.21.15.16:0/2712343414 shutdown_connections 2024-08-31T06:07:35.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.559+0000 7fd4f4202740 1 -- 172.21.15.16:0/2712343414 wait complete. 2024-08-31T06:07:35.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.560+0000 7fd4f4202740 1 -- start start 2024-08-31T06:07:35.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.560+0000 7fd4f4202740 1 Processor -- start 2024-08-31T06:07:35.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.563+0000 7fd4f4202740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a66c84980 0x563a6705fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:35.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.564+0000 7fd4f4202740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a66c8cb60 0x563a670603f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:35.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.564+0000 7fd4f4202740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a66cad840 0x563a6706e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:35.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.564+0000 7fd4f4202740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563a66e1b7a0 con 0x563a66c84980 2024-08-31T06:07:35.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.564+0000 7fd4f4202740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563a66f3f7f0 con 0x563a66c8cb60 2024-08-31T06:07:35.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.564+0000 7fd4f4202740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563a67062890 con 0x563a66cad840 2024-08-31T06:07:35.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.564+0000 7fd4f4202740 10 client.0 _ll_register_callbacks cb 0x563a66f64e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:07:35.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.564+0000 7fd4e6800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a66c84980 0x563a6705fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:35.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.564+0000 7fd4e5e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a66c8cb60 0x563a670603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:35.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.565+0000 7fd4e5e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a66c8cb60 0x563a670603f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:56780/0 (socket says 172.21.15.16:56780) 2024-08-31T06:07:35.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.565+0000 7fd4e5e00640 1 -- 172.21.15.16:0/2039699607 learned_addr learned my addr 172.21.15.16:0/2039699607 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:35.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.565+0000 7fd4e7200640 1 --2- 172.21.15.16:0/2039699607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a66cad840 0x563a6706e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:35.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.565+0000 7fd4e7200640 1 -- 172.21.15.16:0/2039699607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a66c8cb60 msgr2=0x563a670603f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:35.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.565+0000 7fd4e7200640 1 --2- 172.21.15.16:0/2039699607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a66c8cb60 0x563a670603f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:35.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.566+0000 7fd4e7200640 1 -- 172.21.15.16:0/2039699607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a66c84980 msgr2=0x563a6705fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:35.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.566+0000 7fd4e7200640 1 --2- 172.21.15.16:0/2039699607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a66c84980 0x563a6705fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:35.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.566+0000 7fd4e7200640 1 -- 172.21.15.16:0/2039699607 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a66d27b90 con 0x563a66cad840 2024-08-31T06:07:35.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.566+0000 7fd4e6800640 1 --2- 172.21.15.16:0/2039699607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a66c84980 0x563a6705fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:07:35.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.566+0000 7fd4e7200640 1 --2- 172.21.15.16:0/2039699607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a66cad840 0x563a6706e360 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd4e00a8ec0 tx=0x7fd4e00e30d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e27e68d5a83348ad server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:35.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.566+0000 7fd4c7e00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:07:35.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.567+0000 7fd4c7e00640 1 -- 172.21.15.16:0/2039699607 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e001f070 con 0x563a66cad840 2024-08-31T06:07:35.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.568+0000 7fd4c7e00640 1 -- 172.21.15.16:0/2039699607 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fd4e000cd30 con 0x563a66cad840 2024-08-31T06:07:35.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.568+0000 7fd4f4202740 1 -- 172.21.15.16:0/2039699607 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4e00b0410 con 0x563a66cad840 2024-08-31T06:07:35.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.569+0000 7fd4c7e00640 1 -- 172.21.15.16:0/2039699607 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd4e00930b0 con 0x563a66cad840 2024-08-31T06:07:35.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.569+0000 7fd4f4202740 10 client.25594 fetch_fsmap learned FSMap version 30 2024-08-31T06:07:35.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.569+0000 7fd4f4202740 1 -- 172.21.15.16:0/2039699607 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563a670648c0 con 0x563a66cad840 2024-08-31T06:07:35.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.570+0000 7fd4c7e00640 1 -- 172.21.15.16:0/2039699607 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4e00930b0 con 0x563a66cad840 2024-08-31T06:07:35.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.570+0000 7fd4c7e00640 1 -- 172.21.15.16:0/2039699607 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fd4e000f040 con 0x563a66cad840 2024-08-31T06:07:35.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.570+0000 7fd4f4202740 10 client.25594 fetch_fsmap finished waiting for FSMap version 30 2024-08-31T06:07:35.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.570+0000 7fd4f4202740 10 client.25594 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:07:35.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.570+0000 7fd4f4202740 20 client.25594 populate_metadata read hostname 'smithi016' 2024-08-31T06:07:35.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.570+0000 7fd4f4202740 1 --2- 172.21.15.16:0/2039699607 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x563a66c6c1c0 0x563a67067d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:35.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.570+0000 7fd4f4202740 4 client.25594 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:07:35.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.570+0000 7fd4f4202740 1 -- 172.21.15.16:0/2039699607 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a66fcb480 con 0x563a66c6c1c0 2024-08-31T06:07:35.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.570+0000 7fd4e6800640 1 --2- 172.21.15.16:0/2039699607 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x563a66c6c1c0 0x563a67067d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:35.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.571+0000 7fd4e6800640 1 --2- 172.21.15.16:0/2039699607 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x563a66c6c1c0 0x563a67067d90 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f16670d0335ad5e0 server_cookie=e804cea70726f167 in_seq=0 out_seq=0 2024-08-31T06:07:35.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.571+0000 7fd4c7e00640 10 client.25594 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:35.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.572+0000 7fd4c7e00640 1 -- 172.21.15.16:0/2039699607 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563a66fcb480 con 0x563a66c6c1c0 2024-08-31T06:07:35.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.572+0000 7fd4c7e00640 10 client.25594 handle_command_reply: tid=0 2024-08-31T06:07:35.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.572+0000 7fd4f4202740 1 client.25594 shutdown 2024-08-31T06:07:35.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.572+0000 7fd4f4202740 1 -- 172.21.15.16:0/2039699607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a66cad840 msgr2=0x563a6706e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:35.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.572+0000 7fd4f4202740 1 --2- 172.21.15.16:0/2039699607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a66cad840 0x563a6706e360 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd4e00a8ec0 tx=0x7fd4e00e30d0 comp rx=0 tx=0).stop 2024-08-31T06:07:35.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.572+0000 7fd4f4202740 1 -- 172.21.15.16:0/2039699607 shutdown_connections 2024-08-31T06:07:35.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.573+0000 7fd4f4202740 1 --2- 172.21.15.16:0/2039699607 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x563a66c6c1c0 0x563a67067d90 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:35.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.573+0000 7fd4f4202740 1 --2- 172.21.15.16:0/2039699607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a66cad840 0x563a6706e360 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:35.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.573+0000 7fd4f4202740 1 --2- 172.21.15.16:0/2039699607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a66c8cb60 0x563a670603f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:35.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.573+0000 7fd4f4202740 1 --2- 172.21.15.16:0/2039699607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a66c84980 0x563a6705fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:35.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.573+0000 7fd4f4202740 1 -- 172.21.15.16:0/2039699607 >> 172.21.15.16:0/2039699607 conn(0x563a66ca1990 msgr2=0x563a670507e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:35.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.573+0000 7fd4c7e00640 0 client.25594 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:35.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.573+0000 7fd4f4202740 1 -- 172.21.15.16:0/2039699607 shutdown_connections 2024-08-31T06:07:35.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.573+0000 7fd4f4202740 1 -- 172.21.15.16:0/2039699607 wait complete. 2024-08-31T06:07:35.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.573+0000 7fd4f4202740 20 client.25594 trim_cache size 0 max 16384 2024-08-31T06:07:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.604+0000 7fd4f4202740 1 Processor -- start 2024-08-31T06:07:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.604+0000 7fd4f4202740 1 -- start start 2024-08-31T06:07:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.605+0000 7fd4f4202740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a66c84980 0x563a67059220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:35.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.605+0000 7fd4f4202740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a66c8cb60 0x563a67059760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:35.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.605+0000 7fd4f4202740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a66cad840 0x563a67051920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:35.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.605+0000 7fd4f4202740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e81afc00 con 0x563a66c84980 2024-08-31T06:07:35.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.605+0000 7fd4f4202740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e81ad440 con 0x563a66c8cb60 2024-08-31T06:07:35.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.605+0000 7fd4f4202740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e81ab270 con 0x563a66cad840 2024-08-31T06:07:35.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.606+0000 7fd4e6800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a66c84980 0x563a67059220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:35.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.606+0000 7fd4e7200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a66cad840 0x563a67051920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:35.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.606+0000 7fd4e5e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a66c8cb60 0x563a67059760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:35.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.606+0000 7fd4e7200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a66cad840 0x563a67051920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:36696/0 (socket says 172.21.15.16:36696) 2024-08-31T06:07:35.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.606+0000 7fd4e7200640 1 -- 172.21.15.16:0/942917975 learned_addr learned my addr 172.21.15.16:0/942917975 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:35.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.606+0000 7fd4e6800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a66c84980 0x563a67059220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45962/0 (socket says 172.21.15.16:45962) 2024-08-31T06:07:35.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.606+0000 7fd4e7200640 1 -- 172.21.15.16:0/942917975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a66c8cb60 msgr2=0x563a67059760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:35.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.606+0000 7fd4e7200640 1 --2- 172.21.15.16:0/942917975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a66c8cb60 0x563a67059760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:35.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.606+0000 7fd4e7200640 1 -- 172.21.15.16:0/942917975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a66c84980 msgr2=0x563a67059220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:35.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.606+0000 7fd4e7200640 1 --2- 172.21.15.16:0/942917975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a66c84980 0x563a67059220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:35.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.606+0000 7fd4e7200640 1 -- 172.21.15.16:0/942917975 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a6705b4c0 con 0x563a66cad840 2024-08-31T06:07:35.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.607+0000 7fd4e6800640 1 --2- 172.21.15.16:0/942917975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a66c84980 0x563a67059220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:07:35.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.607+0000 7fd4e7200640 1 --2- 172.21.15.16:0/942917975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a66cad840 0x563a67051920 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd4e000c640 tx=0x7fd4e00a73e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72c631131868a677 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:35.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.608+0000 7fd4d2a00640 1 -- 172.21.15.16:0/942917975 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e00930b0 con 0x563a66cad840 2024-08-31T06:07:35.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.608+0000 7fd4d2a00640 1 -- 172.21.15.16:0/942917975 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4e00b0410 con 0x563a66cad840 2024-08-31T06:07:35.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.608+0000 7fd4d2a00640 1 -- 172.21.15.16:0/942917975 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e00a75c0 con 0x563a66cad840 2024-08-31T06:07:35.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.609+0000 7fd4f4202740 1 -- 172.21.15.16:0/942917975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a66cad840 msgr2=0x563a67051920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:35.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.609+0000 7fd4f4202740 1 --2- 172.21.15.16:0/942917975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a66cad840 0x563a67051920 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd4e000c640 tx=0x7fd4e00a73e0 comp rx=0 tx=0).stop 2024-08-31T06:07:35.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.610+0000 7fd4f4202740 1 -- 172.21.15.16:0/942917975 shutdown_connections 2024-08-31T06:07:35.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.610+0000 7fd4f4202740 1 --2- 172.21.15.16:0/942917975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a66cad840 0x563a67051920 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:35.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.610+0000 7fd4f4202740 1 --2- 172.21.15.16:0/942917975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a66c8cb60 0x563a67059760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:35.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.610+0000 7fd4f4202740 1 --2- 172.21.15.16:0/942917975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a66c84980 0x563a67059220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:35.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.610+0000 7fd4f4202740 1 -- 172.21.15.16:0/942917975 >> 172.21.15.16:0/942917975 conn(0x563a66ca1990 msgr2=0x563a6706ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:35.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.611+0000 7fd4f4202740 1 -- 172.21.15.16:0/942917975 shutdown_connections 2024-08-31T06:07:35.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.611+0000 7fd4f4202740 1 -- 172.21.15.16:0/942917975 wait complete. 2024-08-31T06:07:35.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.612+0000 7fd4f4202740 1 -- start start 2024-08-31T06:07:35.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.613+0000 7fd4f4202740 1 Processor -- start 2024-08-31T06:07:35.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.615+0000 7fd4f4202740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a66c84980 0x563a67074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:35.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.615+0000 7fd4f4202740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a66c8cb60 0x563a67071160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:35.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.616+0000 7fd4f4202740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a66cad840 0x563a670716c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:35.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.616+0000 7fd4f4202740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e81ab900 con 0x563a66c8cb60 2024-08-31T06:07:35.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.616+0000 7fd4f4202740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e81acd70 con 0x563a66c84980 2024-08-31T06:07:35.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.616+0000 7fd4f4202740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e81abfd0 con 0x563a66cad840 2024-08-31T06:07:35.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.616+0000 7fd4f4202740 10 client.0 _ll_register_callbacks cb 0x7fd4e00a7ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:07:35.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.617+0000 7fd4e6800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a66c84980 0x563a67074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:35.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.617+0000 7fd4e7200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a66cad840 0x563a670716c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:35.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.617+0000 7fd4e5e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a66c8cb60 0x563a67071160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:35.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.617+0000 7fd4e6800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a66c84980 0x563a67074ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:56806/0 (socket says 172.21.15.16:56806) 2024-08-31T06:07:35.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.617+0000 7fd4e6800640 1 -- 172.21.15.16:0/3800469146 learned_addr learned my addr 172.21.15.16:0/3800469146 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:35.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.619+0000 7fd4e7200640 1 -- 172.21.15.16:0/3800469146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a66c84980 msgr2=0x563a67074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:35.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.619+0000 7fd4e7200640 1 --2- 172.21.15.16:0/3800469146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a66c84980 0x563a67074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:35.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.619+0000 7fd4e7200640 1 -- 172.21.15.16:0/3800469146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a66c8cb60 msgr2=0x563a67071160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:35.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.619+0000 7fd4e7200640 1 --2- 172.21.15.16:0/3800469146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a66c8cb60 0x563a67071160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:35.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.619+0000 7fd4e7200640 1 -- 172.21.15.16:0/3800469146 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a67071e50 con 0x563a66cad840 2024-08-31T06:07:35.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.620+0000 7fd4e6800640 1 --2- 172.21.15.16:0/3800469146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a66c84980 0x563a67074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:07:35.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.620+0000 7fd4c7e00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:07:35.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.620+0000 7fd4e7200640 1 --2- 172.21.15.16:0/3800469146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a66cad840 0x563a670716c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd4e00a81f0 tx=0x7fd4e000ce40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a4527fd5b759a8f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:35.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.621+0000 7fd4c7e00640 1 -- 172.21.15.16:0/3800469146 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e00b4ec0 con 0x563a66cad840 2024-08-31T06:07:35.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.621+0000 7fd4c7e00640 1 -- 172.21.15.16:0/3800469146 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fd4e00aaa40 con 0x563a66cad840 2024-08-31T06:07:35.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.621+0000 7fd4f4202740 1 -- 172.21.15.16:0/3800469146 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4e00b0410 con 0x563a66cad840 2024-08-31T06:07:35.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.621+0000 7fd4c7e00640 1 -- 172.21.15.16:0/3800469146 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd4e008a050 con 0x563a66cad840 2024-08-31T06:07:35.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.621+0000 7fd4f4202740 10 client.25600 fetch_fsmap learned FSMap version 30 2024-08-31T06:07:35.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.621+0000 7fd4f4202740 1 -- 172.21.15.16:0/3800469146 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563a67071e50 con 0x563a66cad840 2024-08-31T06:07:35.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.622+0000 7fd4c7e00640 1 -- 172.21.15.16:0/3800469146 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4e00b0410 con 0x563a66cad840 2024-08-31T06:07:35.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.622+0000 7fd4c7e00640 1 -- 172.21.15.16:0/3800469146 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fd4e0091500 con 0x563a66cad840 2024-08-31T06:07:35.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.622+0000 7fd4f4202740 10 client.25600 fetch_fsmap finished waiting for FSMap version 30 2024-08-31T06:07:35.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.622+0000 7fd4f4202740 10 client.25600 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:07:35.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.622+0000 7fd4f4202740 20 client.25600 populate_metadata read hostname 'smithi016' 2024-08-31T06:07:35.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.622+0000 7fd4f4202740 1 --2- 172.21.15.16:0/3800469146 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x563a66c41570 0x563a670ef970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:35.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.622+0000 7fd4f4202740 4 client.25600 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.4, "await_for": 45.0}] 2024-08-31T06:07:35.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.622+0000 7fd4f4202740 1 -- 172.21.15.16:0/3800469146 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.4, "await_for": 45.0}) -- 0x563a66fc6f20 con 0x563a66c41570 2024-08-31T06:07:35.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.623+0000 7fd4e6800640 1 --2- 172.21.15.16:0/3800469146 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x563a66c41570 0x563a670ef970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:35.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.624+0000 7fd4e6800640 1 --2- 172.21.15.16:0/3800469146 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x563a66c41570 0x563a670ef970 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87fa4bdf632fa421 server_cookie=c38f216445792bfd in_seq=0 out_seq=0 2024-08-31T06:07:35.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:35.625+0000 7fd4c7e00640 10 client.25600 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:36.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.185+0000 7f8525600640 1 -- 172.21.15.16:0/1271110463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8520105950 msgr2=0x7f8520105d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:36.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.185+0000 7f8525600640 1 --2- 172.21.15.16:0/1271110463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8520105950 0x7f8520105d60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8510009900 tx=0x7f851002a3f0 comp rx=0 tx=0).stop 2024-08-31T06:07:36.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.185+0000 7f8525600640 1 -- 172.21.15.16:0/1271110463 shutdown_connections 2024-08-31T06:07:36.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.185+0000 7f8525600640 1 --2- 172.21.15.16:0/1271110463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f852010bbe0 0x7f8520112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:36.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.185+0000 7f8525600640 1 --2- 172.21.15.16:0/1271110463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f852010b290 0x7f852010b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:36.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.185+0000 7f8525600640 1 --2- 172.21.15.16:0/1271110463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8520105950 0x7f8520105d60 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:36.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.185+0000 7f8525600640 1 -- 172.21.15.16:0/1271110463 >> 172.21.15.16:0/1271110463 conn(0x7f8520101420 msgr2=0x7f8520103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:36.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.185+0000 7f8525600640 1 -- 172.21.15.16:0/1271110463 shutdown_connections 2024-08-31T06:07:36.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.186+0000 7f8525600640 1 -- 172.21.15.16:0/1271110463 wait complete. 2024-08-31T06:07:36.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.187+0000 7f8525600640 1 Processor -- start 2024-08-31T06:07:36.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.187+0000 7f8525600640 1 -- start start 2024-08-31T06:07:36.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.188+0000 7f8525600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8520105950 0x7f852019c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:36.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.188+0000 7f8525600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f852010b290 0x7f852019c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:36.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.188+0000 7f8525600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f852010bbe0 0x7f85201a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:36.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.188+0000 7f8525600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f85201158f0 con 0x7f8520105950 2024-08-31T06:07:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.188+0000 7f8525600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8520115770 con 0x7f852010bbe0 2024-08-31T06:07:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.188+0000 7f8525600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8520115a70 con 0x7f852010b290 2024-08-31T06:07:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.188+0000 7f851fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8520105950 0x7f852019c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.188+0000 7f851f400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f852010b290 0x7f852019c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.188+0000 7f851fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8520105950 0x7f852019c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38596/0 (socket says 172.21.15.16:38596) 2024-08-31T06:07:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.188+0000 7f851fe00640 1 -- 172.21.15.16:0/2919443926 learned_addr learned my addr 172.21.15.16:0/2919443926 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.188+0000 7f8524c00640 1 --2- 172.21.15.16:0/2919443926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f852010bbe0 0x7f85201a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:36.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.188+0000 7f851f400640 1 -- 172.21.15.16:0/2919443926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f852010bbe0 msgr2=0x7f85201a1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:36.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.188+0000 7f851f400640 1 --2- 172.21.15.16:0/2919443926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f852010bbe0 0x7f85201a1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:36.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.188+0000 7f851f400640 1 -- 172.21.15.16:0/2919443926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8520105950 msgr2=0x7f852019c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:36.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.188+0000 7f851f400640 1 --2- 172.21.15.16:0/2919443926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8520105950 0x7f852019c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:36.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.188+0000 7f851f400640 1 -- 172.21.15.16:0/2919443926 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85201a2230 con 0x7f852010b290 2024-08-31T06:07:36.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.188+0000 7f851fe00640 1 --2- 172.21.15.16:0/2919443926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8520105950 0x7f852019c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:07:36.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.189+0000 7f851f400640 1 --2- 172.21.15.16:0/2919443926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f852010b290 0x7f852019c860 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f852006c660 tx=0x7f850c00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:36.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.189+0000 7f851cc00640 1 -- 172.21.15.16:0/2919443926 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c00cc40 con 0x7f852010b290 2024-08-31T06:07:36.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.189+0000 7f851cc00640 1 -- 172.21.15.16:0/2919443926 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f850c00cde0 con 0x7f852010b290 2024-08-31T06:07:36.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.189+0000 7f851cc00640 1 -- 172.21.15.16:0/2919443926 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c018780 con 0x7f852010b290 2024-08-31T06:07:36.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.189+0000 7f8525600640 1 -- 172.21.15.16:0/2919443926 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85201a2520 con 0x7f852010b290 2024-08-31T06:07:36.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.189+0000 7f8525600640 1 -- 172.21.15.16:0/2919443926 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85201a8c80 con 0x7f852010b290 2024-08-31T06:07:36.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.192+0000 7f852da0f740 1 Processor -- start 2024-08-31T06:07:36.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.192+0000 7f852da0f740 1 -- start start 2024-08-31T06:07:36.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.193+0000 7f852da0f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5e2e0070 0x556c5e685b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:36.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.193+0000 7f852da0f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c5e2b71b0 0x556c5e6860a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:36.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.193+0000 7f852da0f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c5e2bf390 0x556c5e6865e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:36.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.193+0000 7f852da0f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556c5e5fc7b0 con 0x556c5e2bf390 2024-08-31T06:07:36.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.193+0000 7f852da0f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556c5e5ebd40 con 0x556c5e2e0070 2024-08-31T06:07:36.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.193+0000 7f852da0f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556c5e5efcf0 con 0x556c5e2b71b0 2024-08-31T06:07:36.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.193+0000 7f8524c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5e2e0070 0x556c5e685b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:36.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.193+0000 7f851fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c5e2b71b0 0x556c5e6860a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:36.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.193+0000 7f851fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c5e2b71b0 0x556c5e6860a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:36118/0 (socket says 172.21.15.16:36118) 2024-08-31T06:07:36.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.193+0000 7f851fe00640 1 -- 172.21.15.16:0/227698274 learned_addr learned my addr 172.21.15.16:0/227698274 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:36.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.193+0000 7f8524c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5e2e0070 0x556c5e685b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:42532/0 (socket says 172.21.15.16:42532) 2024-08-31T06:07:36.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.194+0000 7f851cc00640 1 -- 172.21.15.16:0/2919443926 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f850c010a90 con 0x7f852010b290 2024-08-31T06:07:36.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.194+0000 7f851fe00640 1 -- 172.21.15.16:0/227698274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5e2e0070 msgr2=0x556c5e685b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:36.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.194+0000 7f851fe00640 1 --2- 172.21.15.16:0/227698274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5e2e0070 0x556c5e685b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:36.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.194+0000 7f851fe00640 1 -- 172.21.15.16:0/227698274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c5e2bf390 msgr2=0x556c5e6865e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:07:36.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.194+0000 7f851fe00640 1 --2- 172.21.15.16:0/227698274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c5e2bf390 0x556c5e6865e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:36.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.194+0000 7f851fe00640 1 -- 172.21.15.16:0/227698274 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c5e34b650 con 0x556c5e2b71b0 2024-08-31T06:07:36.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.194+0000 7f8524c00640 1 --2- 172.21.15.16:0/227698274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5e2e0070 0x556c5e685b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:07:36.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.194+0000 7f851fe00640 1 --2- 172.21.15.16:0/227698274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c5e2b71b0 0x556c5e6860a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8510006010 tx=0x7f85100044b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c95c9c5518c80e9b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:36.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.194+0000 7f8515200640 1 -- 172.21.15.16:0/227698274 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8510040070 con 0x556c5e2b71b0 2024-08-31T06:07:36.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.195+0000 7f8515200640 1 -- 172.21.15.16:0/227698274 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85100047a0 con 0x556c5e2b71b0 2024-08-31T06:07:36.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.195+0000 7f8515200640 1 -- 172.21.15.16:0/227698274 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8510035040 con 0x556c5e2b71b0 2024-08-31T06:07:36.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.195+0000 7f852da0f740 1 -- 172.21.15.16:0/227698274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c5e2b71b0 msgr2=0x556c5e6860a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:36.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.195+0000 7f852da0f740 1 --2- 172.21.15.16:0/227698274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c5e2b71b0 0x556c5e6860a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8510006010 tx=0x7f85100044b0 comp rx=0 tx=0).stop 2024-08-31T06:07:36.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.195+0000 7f851cc00640 1 --2- 172.21.15.16:0/2919443926 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f85040b4d20 0x7f85040b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:36.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.196+0000 7f852da0f740 1 -- 172.21.15.16:0/227698274 shutdown_connections 2024-08-31T06:07:36.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.196+0000 7f852da0f740 1 --2- 172.21.15.16:0/227698274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c5e2b71b0 0x556c5e6860a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:36.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.196+0000 7f852da0f740 1 --2- 172.21.15.16:0/227698274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c5e2bf390 0x556c5e6865e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:36.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.196+0000 7f852da0f740 1 --2- 172.21.15.16:0/227698274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5e2e0070 0x556c5e685b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:36.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.196+0000 7f852da0f740 1 -- 172.21.15.16:0/227698274 >> 172.21.15.16:0/227698274 conn(0x556c5e2d4460 msgr2=0x556c5e683a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:36.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.196+0000 7f851cc00640 1 -- 172.21.15.16:0/2919443926 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f850c0e5780 con 0x7f852010b290 2024-08-31T06:07:36.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.196+0000 7f8524c00640 1 --2- 172.21.15.16:0/2919443926 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f85040b4d20 0x7f85040b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:36.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.196+0000 7f852da0f740 1 -- 172.21.15.16:0/227698274 shutdown_connections 2024-08-31T06:07:36.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.196+0000 7f852da0f740 1 -- 172.21.15.16:0/227698274 wait complete. 2024-08-31T06:07:36.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.196+0000 7f852da0f740 1 -- start start 2024-08-31T06:07:36.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.196+0000 7f8524c00640 1 --2- 172.21.15.16:0/2919443926 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f85040b4d20 0x7f85040b7230 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8518005fd0 tx=0x7f8518009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:36.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.197+0000 7f852da0f740 1 Processor -- start 2024-08-31T06:07:36.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.197+0000 7f852da0f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c5e2b71b0 0x556c5e6917a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:36.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.197+0000 7f852da0f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5e2bf390 0x556c5e691d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:36.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.197+0000 7f852da0f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c5e2e0070 0x556c5e69d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:36.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.197+0000 7f852da0f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556c5e44e700 con 0x556c5e2e0070 2024-08-31T06:07:36.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.197+0000 7f852da0f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556c5e572cf0 con 0x556c5e2bf390 2024-08-31T06:07:36.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.197+0000 7f852da0f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556c5e6942e0 con 0x556c5e2b71b0 2024-08-31T06:07:36.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.197+0000 7f852da0f740 10 client.0 _ll_register_callbacks cb 0x556c5e598470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:07:36.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.197+0000 7f851fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c5e2b71b0 0x556c5e6917a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:36.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.197+0000 7f851fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c5e2b71b0 0x556c5e6917a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:36120/0 (socket says 172.21.15.16:36120) 2024-08-31T06:07:36.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.197+0000 7f851fe00640 1 -- 172.21.15.16:0/2384884631 learned_addr learned my addr 172.21.15.16:0/2384884631 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:36.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.197+0000 7f851f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5e2bf390 0x556c5e691d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:36.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.198+0000 7f8524c00640 1 --2- 172.21.15.16:0/2384884631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c5e2e0070 0x556c5e69d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:36.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.198+0000 7f851fe00640 1 -- 172.21.15.16:0/2384884631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5e2bf390 msgr2=0x556c5e691d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:36.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.198+0000 7f851fe00640 1 --2- 172.21.15.16:0/2384884631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5e2bf390 0x556c5e691d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:36.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.198+0000 7f851fe00640 1 -- 172.21.15.16:0/2384884631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c5e2e0070 msgr2=0x556c5e69d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:36.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.198+0000 7f851fe00640 1 --2- 172.21.15.16:0/2384884631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c5e2e0070 0x556c5e69d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:36.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.198+0000 7f851fe00640 1 -- 172.21.15.16:0/2384884631 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c5e5b38b0 con 0x556c5e2b71b0 2024-08-31T06:07:36.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.198+0000 7f851fe00640 1 --2- 172.21.15.16:0/2384884631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c5e2b71b0 0x556c5e6917a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f851002abe0 tx=0x7f8510003a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf2a8dd588e45324 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:36.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.198+0000 7f8509600640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:07:36.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.198+0000 7f8509600640 1 -- 172.21.15.16:0/2384884631 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851004d060 con 0x556c5e2b71b0 2024-08-31T06:07:36.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.199+0000 7f852da0f740 1 -- 172.21.15.16:0/2384884631 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85100047a0 con 0x556c5e2b71b0 2024-08-31T06:07:36.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.199+0000 7f8509600640 1 -- 172.21.15.16:0/2384884631 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f85100316a0 con 0x556c5e2b71b0 2024-08-31T06:07:36.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.199+0000 7f8509600640 1 -- 172.21.15.16:0/2384884631 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8510040070 con 0x556c5e2b71b0 2024-08-31T06:07:36.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.199+0000 7f852da0f740 10 client.25612 fetch_fsmap learned FSMap version 30 2024-08-31T06:07:36.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.199+0000 7f852da0f740 1 -- 172.21.15.16:0/2384884631 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556c5e6a0380 con 0x556c5e2b71b0 2024-08-31T06:07:36.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.199+0000 7f8509600640 1 -- 172.21.15.16:0/2384884631 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85100047a0 con 0x556c5e2b71b0 2024-08-31T06:07:36.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.199+0000 7f8509600640 1 -- 172.21.15.16:0/2384884631 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f851003c900 con 0x556c5e2b71b0 2024-08-31T06:07:36.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.199+0000 7f852da0f740 10 client.25612 fetch_fsmap finished waiting for FSMap version 30 2024-08-31T06:07:36.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.199+0000 7f852da0f740 10 client.25612 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:07:36.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.199+0000 7f852da0f740 20 client.25612 populate_metadata read hostname 'smithi016' 2024-08-31T06:07:36.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.199+0000 7f852da0f740 1 --2- 172.21.15.16:0/2384884631 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556c5e29e9f0 0x556c5e6a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:36.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.199+0000 7f852da0f740 4 client.25612 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:07:36.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.199+0000 7f852da0f740 1 -- 172.21.15.16:0/2384884631 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c5e5fe700 con 0x556c5e29e9f0 2024-08-31T06:07:36.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.200+0000 7f851f400640 1 --2- 172.21.15.16:0/2384884631 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556c5e29e9f0 0x556c5e6a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:36.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.200+0000 7f851f400640 1 --2- 172.21.15.16:0/2384884631 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556c5e29e9f0 0x556c5e6a36e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0b5d60aa11b07a server_cookie=698647803768df7c in_seq=0 out_seq=0 2024-08-31T06:07:36.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.200+0000 7f8509600640 10 client.25612 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:36.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.204+0000 7f8509600640 1 -- 172.21.15.16:0/2384884631 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556c5e5fe700 con 0x556c5e29e9f0 2024-08-31T06:07:36.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.204+0000 7f8509600640 10 client.25612 handle_command_reply: tid=0 2024-08-31T06:07:36.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.204+0000 7f852da0f740 1 client.25612 shutdown 2024-08-31T06:07:36.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.204+0000 7f852da0f740 1 -- 172.21.15.16:0/2384884631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c5e2b71b0 msgr2=0x556c5e6917a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:36.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.204+0000 7f852da0f740 1 --2- 172.21.15.16:0/2384884631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c5e2b71b0 0x556c5e6917a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f851002abe0 tx=0x7f8510003a50 comp rx=0 tx=0).stop 2024-08-31T06:07:36.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.204+0000 7f852da0f740 1 -- 172.21.15.16:0/2384884631 shutdown_connections 2024-08-31T06:07:36.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.204+0000 7f852da0f740 1 --2- 172.21.15.16:0/2384884631 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556c5e29e9f0 0x556c5e6a36e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:36.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.204+0000 7f852da0f740 1 --2- 172.21.15.16:0/2384884631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c5e2e0070 0x556c5e69d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:36.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.204+0000 7f852da0f740 1 --2- 172.21.15.16:0/2384884631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5e2bf390 0x556c5e691d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:36.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.204+0000 7f852da0f740 1 --2- 172.21.15.16:0/2384884631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c5e2b71b0 0x556c5e6917a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:36.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.204+0000 7f8509600640 0 client.25612 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:36.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.204+0000 7f852da0f740 1 -- 172.21.15.16:0/2384884631 >> 172.21.15.16:0/2384884631 conn(0x556c5e2d4460 msgr2=0x556c5e688d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:36.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.204+0000 7f852da0f740 1 -- 172.21.15.16:0/2384884631 shutdown_connections 2024-08-31T06:07:36.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.204+0000 7f852da0f740 1 -- 172.21.15.16:0/2384884631 wait complete. 2024-08-31T06:07:36.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.205+0000 7f852da0f740 20 client.25612 trim_cache size 0 max 16384 2024-08-31T06:07:36.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.219+0000 7f852da0f740 1 Processor -- start 2024-08-31T06:07:36.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.219+0000 7f852da0f740 1 -- start start 2024-08-31T06:07:36.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.219+0000 7f852da0f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c5e2b71b0 0x556c5e68b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:36.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.219+0000 7f852da0f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5e2bf390 0x556c5e683000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:36.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.219+0000 7f852da0f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c5e2e0070 0x556c5e683540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:36.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.219+0000 7f852da0f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f85201b3d60 con 0x556c5e2b71b0 2024-08-31T06:07:36.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.219+0000 7f852da0f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f85201b15a0 con 0x556c5e2bf390 2024-08-31T06:07:36.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.219+0000 7f852da0f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f85201af3f0 con 0x556c5e2e0070 2024-08-31T06:07:36.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.220+0000 7f851f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5e2bf390 0x556c5e683000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:36.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.220+0000 7f8524c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c5e2e0070 0x556c5e683540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:36.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.220+0000 7f851fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c5e2b71b0 0x556c5e68b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:36.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.220+0000 7f851f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5e2bf390 0x556c5e683000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:42562/0 (socket says 172.21.15.16:42562) 2024-08-31T06:07:36.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.220+0000 7f851f400640 1 -- 172.21.15.16:0/1584675359 learned_addr learned my addr 172.21.15.16:0/1584675359 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:36.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.220+0000 7f8524c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c5e2e0070 0x556c5e683540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:36136/0 (socket says 172.21.15.16:36136) 2024-08-31T06:07:36.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.220+0000 7f8524c00640 1 -- 172.21.15.16:0/1584675359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5e2bf390 msgr2=0x556c5e683000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:36.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.220+0000 7f8524c00640 1 --2- 172.21.15.16:0/1584675359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5e2bf390 0x556c5e683000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:36.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.220+0000 7f8524c00640 1 -- 172.21.15.16:0/1584675359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c5e2b71b0 msgr2=0x556c5e68b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:36.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.220+0000 7f8524c00640 1 --2- 172.21.15.16:0/1584675359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c5e2b71b0 0x556c5e68b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:36.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.220+0000 7f8524c00640 1 -- 172.21.15.16:0/1584675359 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c5e68faa0 con 0x556c5e2e0070 2024-08-31T06:07:36.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.220+0000 7f8524c00640 1 --2- 172.21.15.16:0/1584675359 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c5e2e0070 0x556c5e683540 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8518003e20 tx=0x7f8518016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c86860f557ccc3e0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:36.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.221+0000 7f850be00640 1 -- 172.21.15.16:0/1584675359 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8518004540 con 0x556c5e2e0070 2024-08-31T06:07:36.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.221+0000 7f850be00640 1 -- 172.21.15.16:0/1584675359 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85180046e0 con 0x556c5e2e0070 2024-08-31T06:07:36.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.221+0000 7f850be00640 1 -- 172.21.15.16:0/1584675359 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851800fc40 con 0x556c5e2e0070 2024-08-31T06:07:36.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.222+0000 7f852da0f740 1 -- 172.21.15.16:0/1584675359 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c5e2e0070 msgr2=0x556c5e683540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:36.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.222+0000 7f852da0f740 1 --2- 172.21.15.16:0/1584675359 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c5e2e0070 0x556c5e683540 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8518003e20 tx=0x7f8518016000 comp rx=0 tx=0).stop 2024-08-31T06:07:36.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.222+0000 7f852da0f740 1 -- 172.21.15.16:0/1584675359 shutdown_connections 2024-08-31T06:07:36.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.222+0000 7f852da0f740 1 --2- 172.21.15.16:0/1584675359 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c5e2e0070 0x556c5e683540 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:36.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.222+0000 7f852da0f740 1 --2- 172.21.15.16:0/1584675359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5e2bf390 0x556c5e683000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:36.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.222+0000 7f852da0f740 1 --2- 172.21.15.16:0/1584675359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c5e2b71b0 0x556c5e68b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:36.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.222+0000 7f852da0f740 1 -- 172.21.15.16:0/1584675359 >> 172.21.15.16:0/1584675359 conn(0x556c5e2d4460 msgr2=0x556c5e68ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:36.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.222+0000 7f852da0f740 1 -- 172.21.15.16:0/1584675359 shutdown_connections 2024-08-31T06:07:36.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.222+0000 7f852da0f740 1 -- 172.21.15.16:0/1584675359 wait complete. 2024-08-31T06:07:36.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.222+0000 7f852da0f740 1 -- start start 2024-08-31T06:07:36.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.222+0000 7f852da0f740 1 Processor -- start 2024-08-31T06:07:36.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.223+0000 7f852da0f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c5e2b71b0 0x556c5e6a6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:36.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.223+0000 7f852da0f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5e2bf390 0x556c5e6a5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:36.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.223+0000 7f852da0f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c5e2e0070 0x556c5e6a5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:36.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.223+0000 7f852da0f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f85201afa60 con 0x556c5e2b71b0 2024-08-31T06:07:36.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.223+0000 7f852da0f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f85201b0ed0 con 0x556c5e2bf390 2024-08-31T06:07:36.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.223+0000 7f852da0f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f85201b0130 con 0x556c5e2e0070 2024-08-31T06:07:36.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.223+0000 7f852da0f740 10 client.0 _ll_register_callbacks cb 0x7f85100095b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:07:36.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.223+0000 7f851f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5e2bf390 0x556c5e6a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:36.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.223+0000 7f8524c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c5e2e0070 0x556c5e6a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:36.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.223+0000 7f851f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5e2bf390 0x556c5e6a5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:42564/0 (socket says 172.21.15.16:42564) 2024-08-31T06:07:36.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.223+0000 7f851f400640 1 -- 172.21.15.16:0/1359894958 learned_addr learned my addr 172.21.15.16:0/1359894958 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:36.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.223+0000 7f851fe00640 1 --2- 172.21.15.16:0/1359894958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c5e2b71b0 0x556c5e6a6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:36.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.224+0000 7f8524c00640 1 -- 172.21.15.16:0/1359894958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5e2bf390 msgr2=0x556c5e6a5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:36.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.224+0000 7f8524c00640 1 --2- 172.21.15.16:0/1359894958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5e2bf390 0x556c5e6a5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:36.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.224+0000 7f8524c00640 1 -- 172.21.15.16:0/1359894958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c5e2b71b0 msgr2=0x556c5e6a6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:36.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.224+0000 7f8524c00640 1 --2- 172.21.15.16:0/1359894958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c5e2b71b0 0x556c5e6a6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:36.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.224+0000 7f8524c00640 1 -- 172.21.15.16:0/1359894958 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c5e6a5bc0 con 0x556c5e2e0070 2024-08-31T06:07:36.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.224+0000 7f8524c00640 1 --2- 172.21.15.16:0/1359894958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c5e2e0070 0x556c5e6a5680 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8518005c10 tx=0x7f8518004540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3dde37f498df4e95 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:36.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.224+0000 7f8509600640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:07:36.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.224+0000 7f8509600640 1 -- 172.21.15.16:0/1359894958 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8518003ae0 con 0x556c5e2e0070 2024-08-31T06:07:36.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.224+0000 7f8509600640 1 -- 172.21.15.16:0/1359894958 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f8518004a40 con 0x556c5e2e0070 2024-08-31T06:07:36.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.224+0000 7f852da0f740 1 -- 172.21.15.16:0/1359894958 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85180046e0 con 0x556c5e2e0070 2024-08-31T06:07:36.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.225+0000 7f8509600640 1 -- 172.21.15.16:0/1359894958 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8518004c60 con 0x556c5e2e0070 2024-08-31T06:07:36.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.225+0000 7f852da0f740 10 client.25618 fetch_fsmap learned FSMap version 30 2024-08-31T06:07:36.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.225+0000 7f852da0f740 1 -- 172.21.15.16:0/1359894958 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556c5e70eb10 con 0x556c5e2e0070 2024-08-31T06:07:36.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.225+0000 7f8509600640 1 -- 172.21.15.16:0/1359894958 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85180046e0 con 0x556c5e2e0070 2024-08-31T06:07:36.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.225+0000 7f8509600640 1 -- 172.21.15.16:0/1359894958 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f8518039020 con 0x556c5e2e0070 2024-08-31T06:07:36.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.225+0000 7f852da0f740 10 client.25618 fetch_fsmap finished waiting for FSMap version 30 2024-08-31T06:07:36.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.225+0000 7f852da0f740 10 client.25618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:07:36.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.225+0000 7f852da0f740 20 client.25618 populate_metadata read hostname 'smithi016' 2024-08-31T06:07:36.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.225+0000 7f852da0f740 1 --2- 172.21.15.16:0/1359894958 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556c5e273da0 0x556c5e7112a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:36.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.225+0000 7f852da0f740 4 client.25618 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:07:36.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.225+0000 7f852da0f740 1 -- 172.21.15.16:0/1359894958 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556c5e5fa1a0 con 0x556c5e273da0 2024-08-31T06:07:36.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.226+0000 7f851fe00640 1 --2- 172.21.15.16:0/1359894958 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556c5e273da0 0x556c5e7112a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:36.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.227+0000 7f851fe00640 1 --2- 172.21.15.16:0/1359894958 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556c5e273da0 0x556c5e7112a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7989b2f098608fee server_cookie=8e3edba34893089 in_seq=0 out_seq=0 2024-08-31T06:07:36.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.227+0000 7f8509600640 10 client.25618 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:36.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.231+0000 7f8509600640 1 -- 172.21.15.16:0/1359894958 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556c5e5fa1a0 con 0x556c5e273da0 2024-08-31T06:07:36.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.231+0000 7f8509600640 10 client.25618 handle_command_reply: tid=0 2024-08-31T06:07:36.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.231+0000 7f852da0f740 1 client.25618 shutdown 2024-08-31T06:07:36.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.231+0000 7f852da0f740 1 -- 172.21.15.16:0/1359894958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c5e2e0070 msgr2=0x556c5e6a5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:36.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.231+0000 7f852da0f740 1 --2- 172.21.15.16:0/1359894958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c5e2e0070 0x556c5e6a5680 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8518005c10 tx=0x7f8518004540 comp rx=0 tx=0).stop 2024-08-31T06:07:36.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.231+0000 7f852da0f740 1 -- 172.21.15.16:0/1359894958 shutdown_connections 2024-08-31T06:07:36.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.231+0000 7f852da0f740 1 --2- 172.21.15.16:0/1359894958 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556c5e273da0 0x556c5e7112a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:36.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.231+0000 7f852da0f740 1 --2- 172.21.15.16:0/1359894958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c5e2e0070 0x556c5e6a5680 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:36.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.231+0000 7f852da0f740 1 --2- 172.21.15.16:0/1359894958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c5e2bf390 0x556c5e6a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:36.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.231+0000 7f852da0f740 1 --2- 172.21.15.16:0/1359894958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c5e2b71b0 0x556c5e6a6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:36.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.231+0000 7f8509600640 0 client.25618 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:36.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.231+0000 7f852da0f740 1 -- 172.21.15.16:0/1359894958 >> 172.21.15.16:0/1359894958 conn(0x556c5e2d4460 msgr2=0x556c5e686b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:36.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.231+0000 7f852da0f740 1 -- 172.21.15.16:0/1359894958 shutdown_connections 2024-08-31T06:07:36.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.231+0000 7f852da0f740 1 -- 172.21.15.16:0/1359894958 wait complete. 2024-08-31T06:07:36.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.231+0000 7f852da0f740 20 client.25618 trim_cache size 0 max 16384 2024-08-31T06:07:36.235 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:07:36.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.232+0000 7f8525600640 1 -- 172.21.15.16:0/2919443926 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f85040b4d20 msgr2=0x7f85040b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:36.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.232+0000 7f8525600640 1 --2- 172.21.15.16:0/2919443926 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f85040b4d20 0x7f85040b7230 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8518005fd0 tx=0x7f8518009500 comp rx=0 tx=0).stop 2024-08-31T06:07:36.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.232+0000 7f8525600640 1 -- 172.21.15.16:0/2919443926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f852010b290 msgr2=0x7f852019c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:36.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.232+0000 7f8525600640 1 --2- 172.21.15.16:0/2919443926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f852010b290 0x7f852019c860 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f852006c660 tx=0x7f850c00ee50 comp rx=0 tx=0).stop 2024-08-31T06:07:36.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.232+0000 7f8525600640 1 -- 172.21.15.16:0/2919443926 shutdown_connections 2024-08-31T06:07:36.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.232+0000 7f8525600640 1 --2- 172.21.15.16:0/2919443926 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f85040b4d20 0x7f85040b7230 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:36.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.232+0000 7f8525600640 1 --2- 172.21.15.16:0/2919443926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f852010b290 0x7f852019c860 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:36.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.232+0000 7f8525600640 1 --2- 172.21.15.16:0/2919443926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f852010bbe0 0x7f85201a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:36.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.232+0000 7f8525600640 1 --2- 172.21.15.16:0/2919443926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8520105950 0x7f852019c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:36.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.232+0000 7f8525600640 1 -- 172.21.15.16:0/2919443926 >> 172.21.15.16:0/2919443926 conn(0x7f8520101420 msgr2=0x7f8520103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:36.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.232+0000 7f8525600640 1 -- 172.21.15.16:0/2919443926 shutdown_connections 2024-08-31T06:07:36.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.232+0000 7f8525600640 1 -- 172.21.15.16:0/2919443926 wait complete. 2024-08-31T06:07:36.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.619+0000 7fd4c7e00640 1 -- 172.21.15.16:0/3800469146 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x563a66fc6f20 con 0x563a66c41570 2024-08-31T06:07:36.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.619+0000 7fd4c7e00640 10 client.25600 handle_command_reply: tid=0 2024-08-31T06:07:36.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.619+0000 7fd4f4202740 1 client.25600 shutdown 2024-08-31T06:07:36.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.619+0000 7fd4f4202740 1 -- 172.21.15.16:0/3800469146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a66cad840 msgr2=0x563a670716c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:36.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.619+0000 7fd4f4202740 1 --2- 172.21.15.16:0/3800469146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a66cad840 0x563a670716c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd4e00a81f0 tx=0x7fd4e000ce40 comp rx=0 tx=0).stop 2024-08-31T06:07:36.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.619+0000 7fd4f4202740 1 -- 172.21.15.16:0/3800469146 shutdown_connections 2024-08-31T06:07:36.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.619+0000 7fd4f4202740 1 --2- 172.21.15.16:0/3800469146 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x563a66c41570 0x563a670ef970 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:36.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.619+0000 7fd4f4202740 1 --2- 172.21.15.16:0/3800469146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a66cad840 0x563a670716c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:36.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.619+0000 7fd4f4202740 1 --2- 172.21.15.16:0/3800469146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a66c8cb60 0x563a67071160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:36.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.619+0000 7fd4f4202740 1 --2- 172.21.15.16:0/3800469146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a66c84980 0x563a67074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:36.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.619+0000 7fd4c7e00640 0 client.25600 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:36.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.619+0000 7fd4f4202740 1 -- 172.21.15.16:0/3800469146 >> 172.21.15.16:0/3800469146 conn(0x563a66ca1990 msgr2=0x563a6706ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:36.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.619+0000 7fd4f4202740 1 -- 172.21.15.16:0/3800469146 shutdown_connections 2024-08-31T06:07:36.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.619+0000 7fd4f4202740 1 -- 172.21.15.16:0/3800469146 wait complete. 2024-08-31T06:07:36.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.620+0000 7fd4f4202740 20 client.25600 trim_cache size 0 max 16384 2024-08-31T06:07:36.623 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:07:36.623 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 30, 2024-08-31T06:07:36.623 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24737, 2024-08-31T06:07:36.623 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 2, 2024-08-31T06:07:36.623 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-08-31T06:07:36.624 INFO:teuthology.orchestra.run.smithi016.stdout: "d396c487": { 2024-08-31T06:07:36.624 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 2, 2024-08-31T06:07:36.624 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-08-31T06:07:36.624 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:07:36.624 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-08-31T06:07:36.624 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:07:36.624 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:07:36.624 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-08-31T06:07:36.624 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 146.4, 2024-08-31T06:07:36.624 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-08-31T06:07:36.624 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-08-31T06:07:36.624 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-08-31T06:07:36.624 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:07:36.624 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-08-31T06:07:36.625 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:07:36.625 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:07:36.625 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:07:36.625 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:07:36.625 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:07:36.625 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:07:36.625 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:07:36.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.620+0000 7fd4ed400640 1 -- 172.21.15.16:0/2974784537 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd4cc0b4d20 msgr2=0x7fd4cc0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:36.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.620+0000 7fd4ed400640 1 --2- 172.21.15.16:0/2974784537 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd4cc0b4d20 0x7fd4cc0b7230 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8005fd0 tx=0x7fd4d800b410 comp rx=0 tx=0).stop 2024-08-31T06:07:36.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.620+0000 7fd4ed400640 1 -- 172.21.15.16:0/2974784537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4e810dbd0 msgr2=0x7fd4e819fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:36.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.620+0000 7fd4ed400640 1 --2- 172.21.15.16:0/2974784537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4e810dbd0 0x7fd4e819fa90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd4e000e940 tx=0x7fd4e000ee00 comp rx=0 tx=0).stop 2024-08-31T06:07:36.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.621+0000 7fd4ed400640 1 -- 172.21.15.16:0/2974784537 shutdown_connections 2024-08-31T06:07:36.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.621+0000 7fd4ed400640 1 --2- 172.21.15.16:0/2974784537 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd4cc0b4d20 0x7fd4cc0b7230 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:36.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.621+0000 7fd4ed400640 1 --2- 172.21.15.16:0/2974784537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4e81050c0 0x7fd4e819a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:36.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.621+0000 7fd4ed400640 1 --2- 172.21.15.16:0/2974784537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4e810dbd0 0x7fd4e819fa90 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:36.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.621+0000 7fd4ed400640 1 --2- 172.21.15.16:0/2974784537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4e8104770 0x7fd4e819a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:36.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.621+0000 7fd4ed400640 1 -- 172.21.15.16:0/2974784537 >> 172.21.15.16:0/2974784537 conn(0x7fd4e8079500 msgr2=0x7fd4e8100580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:36.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.621+0000 7fd4ed400640 1 -- 172.21.15.16:0/2974784537 shutdown_connections 2024-08-31T06:07:36.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:36.621+0000 7fd4ed400640 1 -- 172.21.15.16:0/2974784537 wait complete. 2024-08-31T06:07:36.627 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 10.0 s, 2188.8 tps, lat 36.895 ms stddev 46.521 2024-08-31T06:07:37.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:37 smithi155 ceph-mon[21742]: pgmap v375: 97 pgs: 97 active+clean; 881 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 41 MiB/s wr, 274 op/s 2024-08-31T06:07:37.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:37 smithi016 ceph-mon[17194]: pgmap v375: 97 pgs: 97 active+clean; 881 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 41 MiB/s wr, 274 op/s 2024-08-31T06:07:37.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:37 smithi184 ceph-mon[22331]: pgmap v375: 97 pgs: 97 active+clean; 881 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 41 MiB/s wr, 274 op/s 2024-08-31T06:07:37.913 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd396c487', quiesce took 7.6 seconds. Will release after: 18.799999999999997 2024-08-31T06:07:38.574 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T06:07:38.574 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T06:07:38.972 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:07:39.295 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:39 smithi016 ceph-mon[17194]: pgmap v376: 97 pgs: 97 active+clean; 887 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 22 MiB/s wr, 351 op/s 2024-08-31T06:07:39.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:39 smithi155 ceph-mon[21742]: pgmap v376: 97 pgs: 97 active+clean; 887 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 22 MiB/s wr, 351 op/s 2024-08-31T06:07:39.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:39 smithi184 ceph-mon[22331]: pgmap v376: 97 pgs: 97 active+clean; 887 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 22 MiB/s wr, 351 op/s 2024-08-31T06:07:39.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.604+0000 7ff05b600640 1 -- 172.21.15.16:0/3601781113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff05410b460 msgr2=0x7ff05410b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:39.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.604+0000 7ff05b600640 1 --2- 172.21.15.16:0/3601781113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff05410b460 0x7ff05410b870 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff040009900 tx=0x7ff04002a3f0 comp rx=0 tx=0).stop 2024-08-31T06:07:39.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.604+0000 7ff05b600640 1 -- 172.21.15.16:0/3601781113 shutdown_connections 2024-08-31T06:07:39.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.604+0000 7ff05b600640 1 --2- 172.21.15.16:0/3601781113 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff05410bdb0 0x7ff0541125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:39.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.604+0000 7ff05b600640 1 --2- 172.21.15.16:0/3601781113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff05410b460 0x7ff05410b870 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:39.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.604+0000 7ff05b600640 1 --2- 172.21.15.16:0/3601781113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff054105b20 0x7ff054105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:39.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.604+0000 7ff05b600640 1 -- 172.21.15.16:0/3601781113 >> 172.21.15.16:0/3601781113 conn(0x7ff0541015b0 msgr2=0x7ff054103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:39.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.604+0000 7ff05b600640 1 -- 172.21.15.16:0/3601781113 shutdown_connections 2024-08-31T06:07:39.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.605+0000 7ff05b600640 1 -- 172.21.15.16:0/3601781113 wait complete. 2024-08-31T06:07:39.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.606+0000 7ff05b600640 1 Processor -- start 2024-08-31T06:07:39.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.606+0000 7ff05b600640 1 -- start start 2024-08-31T06:07:39.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.606+0000 7ff05b600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff054105b20 0x7ff05419c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:39.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.606+0000 7ff05b600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff05410b460 0x7ff05419cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:39.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.606+0000 7ff05b600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff05410bdb0 0x7ff0541a1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:39.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.606+0000 7ff05b600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff054115a90 con 0x7ff054105b20 2024-08-31T06:07:39.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.606+0000 7ff05b600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff054115910 con 0x7ff05410bdb0 2024-08-31T06:07:39.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.606+0000 7ff05b600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff054115c10 con 0x7ff05410b460 2024-08-31T06:07:39.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.607+0000 7ff058c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff054105b20 0x7ff05419c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:39.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.607+0000 7ff053e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff05410b460 0x7ff05419cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:39.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.607+0000 7ff058c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff054105b20 0x7ff05419c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38654/0 (socket says 172.21.15.16:38654) 2024-08-31T06:07:39.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.607+0000 7ff058c00640 1 -- 172.21.15.16:0/1687655593 learned_addr learned my addr 172.21.15.16:0/1687655593 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:39.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.607+0000 7ff059600640 1 --2- 172.21.15.16:0/1687655593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff05410bdb0 0x7ff0541a1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:39.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.607+0000 7ff058c00640 1 -- 172.21.15.16:0/1687655593 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff05410b460 msgr2=0x7ff05419cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:39.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.607+0000 7ff058c00640 1 --2- 172.21.15.16:0/1687655593 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff05410b460 0x7ff05419cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:39.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.607+0000 7ff058c00640 1 -- 172.21.15.16:0/1687655593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff05410bdb0 msgr2=0x7ff0541a1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:39.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.607+0000 7ff058c00640 1 --2- 172.21.15.16:0/1687655593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff05410bdb0 0x7ff0541a1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:39.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.607+0000 7ff058c00640 1 -- 172.21.15.16:0/1687655593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0541a24b0 con 0x7ff054105b20 2024-08-31T06:07:39.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.607+0000 7ff059600640 1 --2- 172.21.15.16:0/1687655593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff05410bdb0 0x7ff0541a1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:07:39.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.607+0000 7ff058c00640 1 --2- 172.21.15.16:0/1687655593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff054105b20 0x7ff05419c540 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff04800ea60 tx=0x7ff04800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:39.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.608+0000 7ff051600640 1 -- 172.21.15.16:0/1687655593 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04800cde0 con 0x7ff054105b20 2024-08-31T06:07:39.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.608+0000 7ff051600640 1 -- 172.21.15.16:0/1687655593 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0480035f0 con 0x7ff054105b20 2024-08-31T06:07:39.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.608+0000 7ff05b600640 1 -- 172.21.15.16:0/1687655593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0541a27a0 con 0x7ff054105b20 2024-08-31T06:07:39.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.608+0000 7ff051600640 1 -- 172.21.15.16:0/1687655593 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff048010690 con 0x7ff054105b20 2024-08-31T06:07:39.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.608+0000 7ff05b600640 1 -- 172.21.15.16:0/1687655593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0541a8f00 con 0x7ff054105b20 2024-08-31T06:07:39.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.611+0000 7ff06226b740 1 Processor -- start 2024-08-31T06:07:39.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.611+0000 7ff06226b740 1 -- start start 2024-08-31T06:07:39.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.612+0000 7ff051600640 1 -- 172.21.15.16:0/1687655593 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7ff0480037a0 con 0x7ff054105b20 2024-08-31T06:07:39.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.612+0000 7ff06226b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55beed942350 0x55beedce8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:39.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.612+0000 7ff06226b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55beed919490 0x55beedce8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:39.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.612+0000 7ff06226b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55beed921670 0x55beedce9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:39.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.612+0000 7ff06226b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55beedc5e750 con 0x55beed919490 2024-08-31T06:07:39.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.612+0000 7ff06226b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55beedc4dce0 con 0x55beed921670 2024-08-31T06:07:39.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.612+0000 7ff06226b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55beedc51c90 con 0x55beed942350 2024-08-31T06:07:39.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.612+0000 7ff058c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55beed919490 0x55beedce8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:39.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.612+0000 7ff058c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55beed919490 0x55beedce8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38662/0 (socket says 172.21.15.16:38662) 2024-08-31T06:07:39.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.612+0000 7ff058c00640 1 -- 172.21.15.16:0/2669848249 learned_addr learned my addr 172.21.15.16:0/2669848249 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:39.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.612+0000 7ff053e00640 1 --2- 172.21.15.16:0/2669848249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55beed921670 0x55beedce9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:39.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.612+0000 7ff059600640 1 --2- 172.21.15.16:0/2669848249 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55beed942350 0x55beedce8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:39.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.613+0000 7ff058c00640 1 -- 172.21.15.16:0/2669848249 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55beed942350 msgr2=0x55beedce8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:39.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.613+0000 7ff058c00640 1 --2- 172.21.15.16:0/2669848249 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55beed942350 0x55beedce8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:39.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.613+0000 7ff058c00640 1 -- 172.21.15.16:0/2669848249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55beed921670 msgr2=0x55beedce9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:39.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.613+0000 7ff058c00640 1 --2- 172.21.15.16:0/2669848249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55beed921670 0x55beedce9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:39.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.613+0000 7ff058c00640 1 -- 172.21.15.16:0/2669848249 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55beed9ad930 con 0x55beed919490 2024-08-31T06:07:39.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.613+0000 7ff058c00640 1 --2- 172.21.15.16:0/2669848249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55beed919490 0x55beedce8b40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff0480e2ef0 tx=0x7ff0480ebef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6416d5d89e0948fe server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:39.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.613+0000 7ff045c00640 1 -- 172.21.15.16:0/2669848249 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0480f4070 con 0x55beed919490 2024-08-31T06:07:39.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.613+0000 7ff045c00640 1 -- 172.21.15.16:0/2669848249 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0480ede10 con 0x55beed919490 2024-08-31T06:07:39.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.614+0000 7ff051600640 1 --2- 172.21.15.16:0/1687655593 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff03c0b4ca0 0x7ff03c0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:39.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.614+0000 7ff045c00640 1 -- 172.21.15.16:0/2669848249 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0480f07d0 con 0x55beed919490 2024-08-31T06:07:39.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.614+0000 7ff051600640 1 -- 172.21.15.16:0/1687655593 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7ff0480e5860 con 0x7ff054105b20 2024-08-31T06:07:39.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.614+0000 7ff059600640 1 --2- 172.21.15.16:0/1687655593 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff03c0b4ca0 0x7ff03c0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:39.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.614+0000 7ff06226b740 1 -- 172.21.15.16:0/2669848249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55beed919490 msgr2=0x55beedce8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:39.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.614+0000 7ff06226b740 1 --2- 172.21.15.16:0/2669848249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55beed919490 0x55beedce8b40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff0480e2ef0 tx=0x7ff0480ebef0 comp rx=0 tx=0).stop 2024-08-31T06:07:39.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.615+0000 7ff06226b740 1 -- 172.21.15.16:0/2669848249 shutdown_connections 2024-08-31T06:07:39.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.615+0000 7ff06226b740 1 --2- 172.21.15.16:0/2669848249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55beed921670 0x55beedce9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:39.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.615+0000 7ff06226b740 1 --2- 172.21.15.16:0/2669848249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55beed919490 0x55beedce8b40 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:39.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.615+0000 7ff06226b740 1 --2- 172.21.15.16:0/2669848249 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55beed942350 0x55beedce8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:39.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.615+0000 7ff06226b740 1 -- 172.21.15.16:0/2669848249 >> 172.21.15.16:0/2669848249 conn(0x55beed936740 msgr2=0x55beedce64f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:39.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.615+0000 7ff059600640 1 --2- 172.21.15.16:0/1687655593 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff03c0b4ca0 0x7ff03c0b71b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff04c00a870 tx=0x7ff04c00d040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:39.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.615+0000 7ff06226b740 1 -- 172.21.15.16:0/2669848249 shutdown_connections 2024-08-31T06:07:39.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.615+0000 7ff06226b740 1 -- 172.21.15.16:0/2669848249 wait complete. 2024-08-31T06:07:39.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.615+0000 7ff06226b740 1 -- start start 2024-08-31T06:07:39.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.616+0000 7ff06226b740 1 Processor -- start 2024-08-31T06:07:39.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.616+0000 7ff06226b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55beed919490 0x55beedcf4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:39.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.616+0000 7ff06226b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55beed921670 0x55beedcf4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:39.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.616+0000 7ff06226b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55beed942350 0x55beedd00150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:39.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.616+0000 7ff06226b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55beedab0890 con 0x55beed942350 2024-08-31T06:07:39.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.616+0000 7ff06226b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55beedbd4cd0 con 0x55beed921670 2024-08-31T06:07:39.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.616+0000 7ff06226b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55beedcf6e30 con 0x55beed919490 2024-08-31T06:07:39.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.616+0000 7ff06226b740 10 client.0 _ll_register_callbacks cb 0x55beedc48ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:07:39.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.616+0000 7ff059600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55beed942350 0x55beedd00150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:39.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.617+0000 7ff059600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55beed942350 0x55beedd00150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38678/0 (socket says 172.21.15.16:38678) 2024-08-31T06:07:39.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.617+0000 7ff059600640 1 -- 172.21.15.16:0/2014208086 learned_addr learned my addr 172.21.15.16:0/2014208086 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:39.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.617+0000 7ff053e00640 1 --2- 172.21.15.16:0/2014208086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55beed921670 0x55beedcf4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:39.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.617+0000 7ff058c00640 1 --2- 172.21.15.16:0/2014208086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55beed919490 0x55beedcf4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:39.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.617+0000 7ff059600640 1 -- 172.21.15.16:0/2014208086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55beed919490 msgr2=0x55beedcf4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:39.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.617+0000 7ff059600640 1 --2- 172.21.15.16:0/2014208086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55beed919490 0x55beedcf4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:39.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.617+0000 7ff059600640 1 -- 172.21.15.16:0/2014208086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55beed921670 msgr2=0x55beedcf4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:39.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.617+0000 7ff059600640 1 --2- 172.21.15.16:0/2014208086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55beed921670 0x55beedcf4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:39.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.617+0000 7ff059600640 1 -- 172.21.15.16:0/2014208086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55beedc1e1b0 con 0x55beed942350 2024-08-31T06:07:39.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.617+0000 7ff059600640 1 --2- 172.21.15.16:0/2014208086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55beed942350 0x55beedd00150 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff04c00a650 tx=0x7ff04c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ebbfa015346d32d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:39.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.617+0000 7ff03a000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:07:39.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.618+0000 7ff03a000640 1 -- 172.21.15.16:0/2014208086 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04c027060 con 0x55beed942350 2024-08-31T06:07:39.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.618+0000 7ff06226b740 1 -- 172.21.15.16:0/2014208086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0480ede10 con 0x55beed942350 2024-08-31T06:07:39.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.618+0000 7ff03a000640 1 -- 172.21.15.16:0/2014208086 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7ff04c0225f0 con 0x55beed942350 2024-08-31T06:07:39.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.618+0000 7ff03a000640 1 -- 172.21.15.16:0/2014208086 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff04c0227f0 con 0x55beed942350 2024-08-31T06:07:39.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.618+0000 7ff06226b740 10 client.15591 fetch_fsmap learned FSMap version 30 2024-08-31T06:07:39.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.618+0000 7ff06226b740 1 -- 172.21.15.16:0/2014208086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55beedd02de0 con 0x55beed942350 2024-08-31T06:07:39.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.618+0000 7ff03a000640 1 -- 172.21.15.16:0/2014208086 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0480ede10 con 0x55beed942350 2024-08-31T06:07:39.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.619+0000 7ff03a000640 1 -- 172.21.15.16:0/2014208086 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7ff04c016a10 con 0x55beed942350 2024-08-31T06:07:39.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.619+0000 7ff06226b740 10 client.15591 fetch_fsmap finished waiting for FSMap version 30 2024-08-31T06:07:39.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.619+0000 7ff06226b740 10 client.15591 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:07:39.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.619+0000 7ff06226b740 20 client.15591 populate_metadata read hostname 'smithi016' 2024-08-31T06:07:39.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.619+0000 7ff06226b740 1 --2- 172.21.15.16:0/2014208086 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55beed900cd0 0x55beedd063b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:39.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.619+0000 7ff06226b740 4 client.15591 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:07:39.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.619+0000 7ff06226b740 1 -- 172.21.15.16:0/2014208086 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55beedc606a0 con 0x55beed900cd0 2024-08-31T06:07:39.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.619+0000 7ff058c00640 1 --2- 172.21.15.16:0/2014208086 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55beed900cd0 0x55beedd063b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:39.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.620+0000 7ff058c00640 1 --2- 172.21.15.16:0/2014208086 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55beed900cd0 0x55beedd063b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff06529cf41e101c server_cookie=8498d5c2099263a2 in_seq=0 out_seq=0 2024-08-31T06:07:39.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.620+0000 7ff03a000640 10 client.15591 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:39.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.621+0000 7ff03a000640 1 -- 172.21.15.16:0/2014208086 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55beedc606a0 con 0x55beed900cd0 2024-08-31T06:07:39.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.621+0000 7ff03a000640 10 client.15591 handle_command_reply: tid=0 2024-08-31T06:07:39.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.621+0000 7ff06226b740 1 client.15591 shutdown 2024-08-31T06:07:39.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.621+0000 7ff06226b740 1 -- 172.21.15.16:0/2014208086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55beed942350 msgr2=0x55beedd00150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:39.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.621+0000 7ff06226b740 1 --2- 172.21.15.16:0/2014208086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55beed942350 0x55beedd00150 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff04c00a650 tx=0x7ff04c004a40 comp rx=0 tx=0).stop 2024-08-31T06:07:39.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.621+0000 7ff06226b740 1 -- 172.21.15.16:0/2014208086 shutdown_connections 2024-08-31T06:07:39.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.621+0000 7ff06226b740 1 --2- 172.21.15.16:0/2014208086 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55beed900cd0 0x55beedd063b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:39.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.621+0000 7ff06226b740 1 --2- 172.21.15.16:0/2014208086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55beed942350 0x55beedd00150 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:39.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.621+0000 7ff06226b740 1 --2- 172.21.15.16:0/2014208086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55beed921670 0x55beedcf4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:39.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.621+0000 7ff06226b740 1 --2- 172.21.15.16:0/2014208086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55beed919490 0x55beedcf4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:39.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.621+0000 7ff06226b740 1 -- 172.21.15.16:0/2014208086 >> 172.21.15.16:0/2014208086 conn(0x55beed936740 msgr2=0x55beedce9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:39.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.621+0000 7ff03a000640 0 client.15591 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:39.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.622+0000 7ff06226b740 1 -- 172.21.15.16:0/2014208086 shutdown_connections 2024-08-31T06:07:39.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.622+0000 7ff06226b740 1 -- 172.21.15.16:0/2014208086 wait complete. 2024-08-31T06:07:39.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.622+0000 7ff06226b740 20 client.15591 trim_cache size 0 max 16384 2024-08-31T06:07:39.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.640+0000 7ff06226b740 1 Processor -- start 2024-08-31T06:07:39.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.640+0000 7ff06226b740 1 -- start start 2024-08-31T06:07:39.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.641+0000 7ff06226b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55beed919490 0x55beedce49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:39.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.641+0000 7ff06226b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55beed921670 0x55beedce4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:39.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.641+0000 7ff06226b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55beed942350 0x55beedcf27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:39.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.641+0000 7ff06226b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff0541b3ff0 con 0x55beed921670 2024-08-31T06:07:39.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.641+0000 7ff06226b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff0541b1830 con 0x55beed942350 2024-08-31T06:07:39.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.641+0000 7ff06226b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff0541af680 con 0x55beed919490 2024-08-31T06:07:39.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.641+0000 7ff053e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55beed921670 0x55beedce4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:39.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.641+0000 7ff053e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55beed921670 0x55beedce4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38684/0 (socket says 172.21.15.16:38684) 2024-08-31T06:07:39.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.641+0000 7ff053e00640 1 -- 172.21.15.16:0/978804507 learned_addr learned my addr 172.21.15.16:0/978804507 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:39.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.641+0000 7ff058c00640 1 --2- 172.21.15.16:0/978804507 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55beed919490 0x55beedce49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:39.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.641+0000 7ff059600640 1 --2- 172.21.15.16:0/978804507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55beed942350 0x55beedcf27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:39.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.642+0000 7ff053e00640 1 -- 172.21.15.16:0/978804507 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55beed919490 msgr2=0x55beedce49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:39.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.642+0000 7ff053e00640 1 --2- 172.21.15.16:0/978804507 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55beed919490 0x55beedce49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:39.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.642+0000 7ff053e00640 1 -- 172.21.15.16:0/978804507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55beed942350 msgr2=0x55beedcf27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:39.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.642+0000 7ff053e00640 1 --2- 172.21.15.16:0/978804507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55beed942350 0x55beedcf27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:39.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.642+0000 7ff053e00640 1 -- 172.21.15.16:0/978804507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55beedcef7b0 con 0x55beed921670 2024-08-31T06:07:39.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.642+0000 7ff053e00640 1 --2- 172.21.15.16:0/978804507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55beed921670 0x55beedce4f10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff040009900 tx=0x7ff040000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b96b169a6588638b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:39.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.642+0000 7ff045200640 1 -- 172.21.15.16:0/978804507 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff040035040 con 0x55beed921670 2024-08-31T06:07:39.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.642+0000 7ff045200640 1 -- 172.21.15.16:0/978804507 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff040031760 con 0x55beed921670 2024-08-31T06:07:39.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.642+0000 7ff045200640 1 -- 172.21.15.16:0/978804507 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0400049b0 con 0x55beed921670 2024-08-31T06:07:39.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.643+0000 7ff06226b740 1 -- 172.21.15.16:0/978804507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55beed921670 msgr2=0x55beedce4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:39.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.643+0000 7ff06226b740 1 --2- 172.21.15.16:0/978804507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55beed921670 0x55beedce4f10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff040009900 tx=0x7ff040000bc0 comp rx=0 tx=0).stop 2024-08-31T06:07:39.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.643+0000 7ff06226b740 1 -- 172.21.15.16:0/978804507 shutdown_connections 2024-08-31T06:07:39.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.643+0000 7ff06226b740 1 --2- 172.21.15.16:0/978804507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55beed942350 0x55beedcf27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:39.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.643+0000 7ff06226b740 1 --2- 172.21.15.16:0/978804507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55beed921670 0x55beedce4f10 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:39.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.643+0000 7ff06226b740 1 --2- 172.21.15.16:0/978804507 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55beed919490 0x55beedce49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:39.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.643+0000 7ff06226b740 1 -- 172.21.15.16:0/978804507 >> 172.21.15.16:0/978804507 conn(0x55beed936740 msgr2=0x55beedd06380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:39.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.643+0000 7ff06226b740 1 -- 172.21.15.16:0/978804507 shutdown_connections 2024-08-31T06:07:39.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.643+0000 7ff06226b740 1 -- 172.21.15.16:0/978804507 wait complete. 2024-08-31T06:07:39.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.644+0000 7ff06226b740 1 -- start start 2024-08-31T06:07:39.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.644+0000 7ff06226b740 1 Processor -- start 2024-08-31T06:07:39.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.645+0000 7ff06226b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55beed919490 0x55beedcfc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:39.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.645+0000 7ff06226b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55beed921670 0x55beedcfcac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:39.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.645+0000 7ff06226b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55beed942350 0x55beedcf7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:39.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.645+0000 7ff06226b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff0541afcf0 con 0x55beed921670 2024-08-31T06:07:39.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.645+0000 7ff06226b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff0541b1160 con 0x55beed942350 2024-08-31T06:07:39.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.645+0000 7ff06226b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff0541b03c0 con 0x55beed919490 2024-08-31T06:07:39.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.645+0000 7ff06226b740 10 client.0 _ll_register_callbacks cb 0x7ff04c009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:07:39.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.645+0000 7ff053e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55beed921670 0x55beedcfcac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:39.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.645+0000 7ff053e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55beed921670 0x55beedcfcac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38690/0 (socket says 172.21.15.16:38690) 2024-08-31T06:07:39.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.645+0000 7ff053e00640 1 -- 172.21.15.16:0/690700395 learned_addr learned my addr 172.21.15.16:0/690700395 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:39.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.645+0000 7ff058c00640 1 --2- 172.21.15.16:0/690700395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55beed919490 0x55beedcfc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:39.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.645+0000 7ff059600640 1 --2- 172.21.15.16:0/690700395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55beed942350 0x55beedcf7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:39.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.646+0000 7ff053e00640 1 -- 172.21.15.16:0/690700395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55beed919490 msgr2=0x55beedcfc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:39.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.646+0000 7ff053e00640 1 --2- 172.21.15.16:0/690700395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55beed919490 0x55beedcfc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:39.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.646+0000 7ff053e00640 1 -- 172.21.15.16:0/690700395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55beed942350 msgr2=0x55beedcf7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:39.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.646+0000 7ff053e00640 1 --2- 172.21.15.16:0/690700395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55beed942350 0x55beedcf7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:39.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.646+0000 7ff053e00640 1 -- 172.21.15.16:0/690700395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55beedcf79b0 con 0x55beed921670 2024-08-31T06:07:39.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.646+0000 7ff053e00640 1 --2- 172.21.15.16:0/690700395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55beed921670 0x55beedcfcac0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff040004620 tx=0x7ff040031910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34b43f04f85a7fce server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:39.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.646+0000 7ff03a000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:07:39.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.646+0000 7ff03a000640 1 -- 172.21.15.16:0/690700395 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff040050070 con 0x55beed921670 2024-08-31T06:07:39.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.646+0000 7ff03a000640 1 -- 172.21.15.16:0/690700395 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7ff040047690 con 0x55beed921670 2024-08-31T06:07:39.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.646+0000 7ff06226b740 1 -- 172.21.15.16:0/690700395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff040031760 con 0x55beed921670 2024-08-31T06:07:39.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.646+0000 7ff03a000640 1 -- 172.21.15.16:0/690700395 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff04005c050 con 0x55beed921670 2024-08-31T06:07:39.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.647+0000 7ff06226b740 10 client.15597 fetch_fsmap learned FSMap version 30 2024-08-31T06:07:39.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.647+0000 7ff06226b740 1 -- 172.21.15.16:0/690700395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55beedd719b0 con 0x55beed921670 2024-08-31T06:07:39.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.647+0000 7ff03a000640 1 -- 172.21.15.16:0/690700395 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff040031760 con 0x55beed921670 2024-08-31T06:07:39.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.647+0000 7ff03a000640 1 -- 172.21.15.16:0/690700395 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7ff04003da90 con 0x55beed921670 2024-08-31T06:07:39.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.647+0000 7ff06226b740 10 client.15597 fetch_fsmap finished waiting for FSMap version 30 2024-08-31T06:07:39.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.647+0000 7ff06226b740 10 client.15597 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:07:39.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.647+0000 7ff06226b740 20 client.15597 populate_metadata read hostname 'smithi016' 2024-08-31T06:07:39.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.647+0000 7ff06226b740 1 --2- 172.21.15.16:0/690700395 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55beed8d6080 0x55beedd74120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:39.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.647+0000 7ff06226b740 4 client.15597 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T06:07:39.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.647+0000 7ff06226b740 1 -- 172.21.15.16:0/690700395 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55beedc5c140 con 0x55beed8d6080 2024-08-31T06:07:39.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.648+0000 7ff058c00640 1 --2- 172.21.15.16:0/690700395 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55beed8d6080 0x55beedd74120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:39.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.648+0000 7ff058c00640 1 --2- 172.21.15.16:0/690700395 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55beed8d6080 0x55beedd74120 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc4fb9b1fb1535ed server_cookie=598d4c4b30e412c8 in_seq=0 out_seq=0 2024-08-31T06:07:39.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.648+0000 7ff03a000640 10 client.15597 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:39.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.649+0000 7ff03a000640 1 -- 172.21.15.16:0/690700395 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55beedc5c140 con 0x55beed8d6080 2024-08-31T06:07:39.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.649+0000 7ff03a000640 10 client.15597 handle_command_reply: tid=0 2024-08-31T06:07:39.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.649+0000 7ff06226b740 1 client.15597 shutdown 2024-08-31T06:07:39.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.649+0000 7ff06226b740 1 -- 172.21.15.16:0/690700395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55beed921670 msgr2=0x55beedcfcac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:39.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.649+0000 7ff06226b740 1 --2- 172.21.15.16:0/690700395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55beed921670 0x55beedcfcac0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff040004620 tx=0x7ff040031910 comp rx=0 tx=0).stop 2024-08-31T06:07:39.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.649+0000 7ff06226b740 1 -- 172.21.15.16:0/690700395 shutdown_connections 2024-08-31T06:07:39.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.649+0000 7ff06226b740 1 --2- 172.21.15.16:0/690700395 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55beed8d6080 0x55beedd74120 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:39.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.649+0000 7ff06226b740 1 --2- 172.21.15.16:0/690700395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55beed942350 0x55beedcf7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:39.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.649+0000 7ff06226b740 1 --2- 172.21.15.16:0/690700395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55beed921670 0x55beedcfcac0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:39.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.649+0000 7ff03a000640 0 client.15597 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:39.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.649+0000 7ff06226b740 1 --2- 172.21.15.16:0/690700395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55beed919490 0x55beedcfc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:39.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.649+0000 7ff06226b740 1 -- 172.21.15.16:0/690700395 >> 172.21.15.16:0/690700395 conn(0x55beed936740 msgr2=0x55beedcf26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:39.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.650+0000 7ff06226b740 1 -- 172.21.15.16:0/690700395 shutdown_connections 2024-08-31T06:07:39.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.650+0000 7ff06226b740 1 -- 172.21.15.16:0/690700395 wait complete. 2024-08-31T06:07:39.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.650+0000 7ff06226b740 20 client.15597 trim_cache size 0 max 16384 2024-08-31T06:07:39.655 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:07:39.655 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-08-31T06:07:39.655 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "88d7bdb8-cb1c-45e5-ad29-4cd8da6c6003", 2024-08-31T06:07:39.655 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-08-31T06:07:39.655 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:07:39.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.650+0000 7ff05b600640 1 -- 172.21.15.16:0/1687655593 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff03c0b4ca0 msgr2=0x7ff03c0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:39.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.650+0000 7ff05b600640 1 --2- 172.21.15.16:0/1687655593 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff03c0b4ca0 0x7ff03c0b71b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff04c00a870 tx=0x7ff04c00d040 comp rx=0 tx=0).stop 2024-08-31T06:07:39.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.650+0000 7ff05b600640 1 -- 172.21.15.16:0/1687655593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff054105b20 msgr2=0x7ff05419c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:39.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.650+0000 7ff05b600640 1 --2- 172.21.15.16:0/1687655593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff054105b20 0x7ff05419c540 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff04800ea60 tx=0x7ff04800ef20 comp rx=0 tx=0).stop 2024-08-31T06:07:39.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.651+0000 7ff05b600640 1 -- 172.21.15.16:0/1687655593 shutdown_connections 2024-08-31T06:07:39.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.651+0000 7ff05b600640 1 --2- 172.21.15.16:0/1687655593 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff03c0b4ca0 0x7ff03c0b71b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:39.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.651+0000 7ff05b600640 1 --2- 172.21.15.16:0/1687655593 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff05410b460 0x7ff05419cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:39.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.651+0000 7ff05b600640 1 --2- 172.21.15.16:0/1687655593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff05410bdb0 0x7ff0541a1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:39.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.651+0000 7ff05b600640 1 --2- 172.21.15.16:0/1687655593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff054105b20 0x7ff05419c540 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:39.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.651+0000 7ff05b600640 1 -- 172.21.15.16:0/1687655593 >> 172.21.15.16:0/1687655593 conn(0x7ff0541015b0 msgr2=0x7ff0541039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:39.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.651+0000 7ff05b600640 1 -- 172.21.15.16:0/1687655593 shutdown_connections 2024-08-31T06:07:39.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:39.651+0000 7ff05b600640 1 -- 172.21.15.16:0/1687655593 wait complete. 2024-08-31T06:07:40.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:40 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:07:40.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:40 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:07:40.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:40 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:07:40.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:40 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:07:40.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:40 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:07:40.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:40 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:07:40.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:40 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:07:40.379 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:40 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:07:40.380 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:40 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:07:40.380 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:40 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:07:40.380 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:40 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:07:40.380 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:40 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:07:40.380 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:40 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:07:40.381 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:40 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:07:40.416 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T06:07:40.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:40 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:07:40.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:40 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:07:40.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:40 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:07:40.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:40 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:07:40.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:40 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:07:40.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:40 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:07:40.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:40 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:07:40.794 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:07:41.241 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:41 smithi016 ceph-mon[17194]: pgmap v377: 97 pgs: 97 active+clean; 899 MiB data, 776 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 18 MiB/s wr, 454 op/s 2024-08-31T06:07:41.241 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:41 smithi016 ceph-mon[17194]: scrub queued for path: / 2024-08-31T06:07:41.242 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:41 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:07:41.242 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:41 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:07:41.242 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:41 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:07:41.242 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:41 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:07:41.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:41 smithi155 ceph-mon[21742]: pgmap v377: 97 pgs: 97 active+clean; 899 MiB data, 776 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 18 MiB/s wr, 454 op/s 2024-08-31T06:07:41.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:41 smithi155 ceph-mon[21742]: scrub queued for path: / 2024-08-31T06:07:41.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:41 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:07:41.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:41 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:07:41.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:41 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:07:41.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:41 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:07:41.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.421+0000 7f291e400640 1 -- 172.21.15.16:0/2360596959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2918105950 msgr2=0x7f2918105d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:41.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.421+0000 7f291e400640 1 --2- 172.21.15.16:0/2360596959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2918105950 0x7f2918105d60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f290c009900 tx=0x7f290c02a3f0 comp rx=0 tx=0).stop 2024-08-31T06:07:41.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.421+0000 7f291e400640 1 -- 172.21.15.16:0/2360596959 shutdown_connections 2024-08-31T06:07:41.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.421+0000 7f291e400640 1 --2- 172.21.15.16:0/2360596959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f291810b290 0x7f291810b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:41.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.421+0000 7f291e400640 1 --2- 172.21.15.16:0/2360596959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f291810bbe0 0x7f2918112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:41.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.421+0000 7f291e400640 1 --2- 172.21.15.16:0/2360596959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2918105950 0x7f2918105d60 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:41.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.421+0000 7f291e400640 1 -- 172.21.15.16:0/2360596959 >> 172.21.15.16:0/2360596959 conn(0x7f2918101420 msgr2=0x7f2918103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:41.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.421+0000 7f291e400640 1 -- 172.21.15.16:0/2360596959 shutdown_connections 2024-08-31T06:07:41.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.421+0000 7f291e400640 1 -- 172.21.15.16:0/2360596959 wait complete. 2024-08-31T06:07:41.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.422+0000 7f291e400640 1 Processor -- start 2024-08-31T06:07:41.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.423+0000 7f291e400640 1 -- start start 2024-08-31T06:07:41.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.423+0000 7f291e400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2918105950 0x7f291819c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:41.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.423+0000 7f291e400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f291810b290 0x7f291819c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:41.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.423+0000 7f291e400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f291810bbe0 0x7f29181a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:41.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.423+0000 7f291e400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f29181158a0 con 0x7f291810bbe0 2024-08-31T06:07:41.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.423+0000 7f291e400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2918115720 con 0x7f2918105950 2024-08-31T06:07:41.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.423+0000 7f291e400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2918115a20 con 0x7f291810b290 2024-08-31T06:07:41.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.423+0000 7f291da00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f291810bbe0 0x7f29181a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:41.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.423+0000 7f291da00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f291810bbe0 0x7f29181a1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38704/0 (socket says 172.21.15.16:38704) 2024-08-31T06:07:41.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.423+0000 7f291da00640 1 -- 172.21.15.16:0/3196687005 learned_addr learned my addr 172.21.15.16:0/3196687005 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:41.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.423+0000 7f291d000640 1 --2- 172.21.15.16:0/3196687005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2918105950 0x7f291819c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:41.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.423+0000 7f2917e00640 1 --2- 172.21.15.16:0/3196687005 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f291810b290 0x7f291819c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:41.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.424+0000 7f291da00640 1 -- 172.21.15.16:0/3196687005 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f291810b290 msgr2=0x7f291819c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:41.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.424+0000 7f291da00640 1 --2- 172.21.15.16:0/3196687005 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f291810b290 0x7f291819c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:41.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.424+0000 7f291da00640 1 -- 172.21.15.16:0/3196687005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2918105950 msgr2=0x7f291819c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:41.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.424+0000 7f291da00640 1 --2- 172.21.15.16:0/3196687005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2918105950 0x7f291819c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:41.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.424+0000 7f291da00640 1 -- 172.21.15.16:0/3196687005 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29181a2220 con 0x7f291810bbe0 2024-08-31T06:07:41.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.424+0000 7f291da00640 1 --2- 172.21.15.16:0/3196687005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f291810bbe0 0x7f29181a1af0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f291000eb10 tx=0x7f291000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:41.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.424+0000 7f2915600640 1 -- 172.21.15.16:0/3196687005 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2910004b20 con 0x7f291810bbe0 2024-08-31T06:07:41.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.424+0000 7f291e400640 1 -- 172.21.15.16:0/3196687005 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29181a2510 con 0x7f291810bbe0 2024-08-31T06:07:41.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.424+0000 7f2915600640 1 -- 172.21.15.16:0/3196687005 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2910004cc0 con 0x7f291810bbe0 2024-08-31T06:07:41.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.424+0000 7f2915600640 1 -- 172.21.15.16:0/3196687005 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2910010670 con 0x7f291810bbe0 2024-08-31T06:07:41.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.425+0000 7f291e400640 1 -- 172.21.15.16:0/3196687005 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29181a8c70 con 0x7f291810bbe0 2024-08-31T06:07:41.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.428+0000 7f29267d6740 1 Processor -- start 2024-08-31T06:07:41.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.428+0000 7f2915600640 1 -- 172.21.15.16:0/3196687005 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f2910005020 con 0x7f291810bbe0 2024-08-31T06:07:41.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.428+0000 7f29267d6740 1 -- start start 2024-08-31T06:07:41.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.429+0000 7f29267d6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dba4cc1070 0x55dba5066b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:41.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.429+0000 7f29267d6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dba4c981b0 0x55dba5067060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:41.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.429+0000 7f29267d6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba4ca0390 0x55dba50675a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:41.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.429+0000 7f29267d6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dba4fdd770 con 0x55dba4ca0390 2024-08-31T06:07:41.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.429+0000 7f29267d6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55dba4fccd00 con 0x55dba4cc1070 2024-08-31T06:07:41.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.429+0000 7f29267d6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55dba4fd0cb0 con 0x55dba4c981b0 2024-08-31T06:07:41.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.429+0000 7f2917e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dba4cc1070 0x55dba5066b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:41.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.429+0000 7f291d000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dba4c981b0 0x55dba5067060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:41.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.429+0000 7f291d000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dba4c981b0 0x55dba5067060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:36224/0 (socket says 172.21.15.16:36224) 2024-08-31T06:07:41.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.429+0000 7f291d000640 1 -- 172.21.15.16:0/1591069704 learned_addr learned my addr 172.21.15.16:0/1591069704 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:41.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.429+0000 7f2917e00640 1 --2- 172.21.15.16:0/1591069704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba4ca0390 0x55dba50675a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:41.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.429+0000 7f291d000640 1 -- 172.21.15.16:0/1591069704 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dba4cc1070 msgr2=0x55dba5066b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:41.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.429+0000 7f291d000640 1 --2- 172.21.15.16:0/1591069704 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dba4cc1070 0x55dba5066b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:41.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.429+0000 7f291d000640 1 -- 172.21.15.16:0/1591069704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba4ca0390 msgr2=0x55dba50675a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:41.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.429+0000 7f291d000640 1 --2- 172.21.15.16:0/1591069704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba4ca0390 0x55dba50675a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:41.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.429+0000 7f291d000640 1 -- 172.21.15.16:0/1591069704 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dba4d2c650 con 0x55dba4c981b0 2024-08-31T06:07:41.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.429+0000 7f2917e00640 1 --2- 172.21.15.16:0/1591069704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba4ca0390 0x55dba50675a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:07:41.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.430+0000 7f291d000640 1 --2- 172.21.15.16:0/1591069704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dba4c981b0 0x55dba5067060 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f290c009900 tx=0x7f290c032b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=302e6af5ae27402f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:41.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.430+0000 7f2915600640 1 --2- 172.21.15.16:0/3196687005 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f29000b4ca0 0x7f29000b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:41.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.430+0000 7f290a000640 1 -- 172.21.15.16:0/1591069704 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c040070 con 0x55dba4c981b0 2024-08-31T06:07:41.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.430+0000 7f2915600640 1 -- 172.21.15.16:0/3196687005 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f29100e5b30 con 0x7f291810bbe0 2024-08-31T06:07:41.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.430+0000 7f290a000640 1 -- 172.21.15.16:0/1591069704 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f290c004030 con 0x55dba4c981b0 2024-08-31T06:07:41.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.430+0000 7f290a000640 1 -- 172.21.15.16:0/1591069704 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c035040 con 0x55dba4c981b0 2024-08-31T06:07:41.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.430+0000 7f2917e00640 1 --2- 172.21.15.16:0/3196687005 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f29000b4ca0 0x7f29000b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:41.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.431+0000 7f29267d6740 1 -- 172.21.15.16:0/1591069704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dba4c981b0 msgr2=0x55dba5067060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:41.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.431+0000 7f2917e00640 1 --2- 172.21.15.16:0/3196687005 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f29000b4ca0 0x7f29000b71b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f291810ac90 tx=0x7f2904013040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:41.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.431+0000 7f29267d6740 1 --2- 172.21.15.16:0/1591069704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dba4c981b0 0x55dba5067060 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f290c009900 tx=0x7f290c032b90 comp rx=0 tx=0).stop 2024-08-31T06:07:41.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.431+0000 7f29267d6740 1 -- 172.21.15.16:0/1591069704 shutdown_connections 2024-08-31T06:07:41.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.431+0000 7f29267d6740 1 --2- 172.21.15.16:0/1591069704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dba4c981b0 0x55dba5067060 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:41.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.431+0000 7f29267d6740 1 --2- 172.21.15.16:0/1591069704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba4ca0390 0x55dba50675a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:41.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.431+0000 7f29267d6740 1 --2- 172.21.15.16:0/1591069704 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dba4cc1070 0x55dba5066b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:41.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.431+0000 7f29267d6740 1 -- 172.21.15.16:0/1591069704 >> 172.21.15.16:0/1591069704 conn(0x55dba4cb5460 msgr2=0x55dba5064a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:41.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.432+0000 7f29267d6740 1 -- 172.21.15.16:0/1591069704 shutdown_connections 2024-08-31T06:07:41.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.432+0000 7f29267d6740 1 -- 172.21.15.16:0/1591069704 wait complete. 2024-08-31T06:07:41.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.432+0000 7f29267d6740 1 -- start start 2024-08-31T06:07:41.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.432+0000 7f29267d6740 1 Processor -- start 2024-08-31T06:07:41.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.433+0000 7f29267d6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dba4c981b0 0x55dba5072660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:41.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.433+0000 7f29267d6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dba4ca0390 0x55dba5072ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:41.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.433+0000 7f29267d6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba4cc1070 0x55dba507e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:41.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.433+0000 7f29267d6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dba4e2f700 con 0x55dba4cc1070 2024-08-31T06:07:41.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.433+0000 7f29267d6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55dba4f53cb0 con 0x55dba4ca0390 2024-08-31T06:07:41.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.433+0000 7f29267d6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55dba50752a0 con 0x55dba4c981b0 2024-08-31T06:07:41.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.433+0000 7f29267d6740 10 client.0 _ll_register_callbacks cb 0x55dba4f79430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:07:41.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.433+0000 7f291da00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba4cc1070 0x55dba507e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:41.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.433+0000 7f291da00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba4cc1070 0x55dba507e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38722/0 (socket says 172.21.15.16:38722) 2024-08-31T06:07:41.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.433+0000 7f291da00640 1 -- 172.21.15.16:0/126401281 learned_addr learned my addr 172.21.15.16:0/126401281 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:41.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.433+0000 7f2917e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dba4ca0390 0x55dba5072ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:41.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.433+0000 7f291d000640 1 --2- 172.21.15.16:0/126401281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dba4c981b0 0x55dba5072660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:41.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.434+0000 7f291da00640 1 -- 172.21.15.16:0/126401281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dba4c981b0 msgr2=0x55dba5072660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:41.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.434+0000 7f291da00640 1 --2- 172.21.15.16:0/126401281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dba4c981b0 0x55dba5072660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:41.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.434+0000 7f291da00640 1 -- 172.21.15.16:0/126401281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dba4ca0390 msgr2=0x55dba5072ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:41.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.434+0000 7f291da00640 1 --2- 172.21.15.16:0/126401281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dba4ca0390 0x55dba5072ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:41.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.434+0000 7f291da00640 1 -- 172.21.15.16:0/126401281 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dba4f94870 con 0x55dba4cc1070 2024-08-31T06:07:41.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.434+0000 7f291d000640 1 --2- 172.21.15.16:0/126401281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dba4c981b0 0x55dba5072660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:07:41.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.434+0000 7f291da00640 1 --2- 172.21.15.16:0/126401281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba4cc1070 0x55dba507e5c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f29100ac1b0 tx=0x7f29100ac070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d461b111643772 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:41.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.434+0000 7f28fea00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:07:41.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.434+0000 7f28fea00640 1 -- 172.21.15.16:0/126401281 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29100abd30 con 0x55dba4cc1070 2024-08-31T06:07:41.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.434+0000 7f28fea00640 1 -- 172.21.15.16:0/126401281 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f29100ac690 con 0x55dba4cc1070 2024-08-31T06:07:41.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.434+0000 7f29267d6740 1 -- 172.21.15.16:0/126401281 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f290c004030 con 0x55dba4cc1070 2024-08-31T06:07:41.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.435+0000 7f28fea00640 1 -- 172.21.15.16:0/126401281 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f29100ac920 con 0x55dba4cc1070 2024-08-31T06:07:41.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.435+0000 7f29267d6740 10 client.15609 fetch_fsmap learned FSMap version 30 2024-08-31T06:07:41.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.435+0000 7f29267d6740 1 -- 172.21.15.16:0/126401281 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dba50812c0 con 0x55dba4cc1070 2024-08-31T06:07:41.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.435+0000 7f28fea00640 1 -- 172.21.15.16:0/126401281 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f290c004030 con 0x55dba4cc1070 2024-08-31T06:07:41.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.435+0000 7f28fea00640 1 -- 172.21.15.16:0/126401281 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f29100a7b90 con 0x55dba4cc1070 2024-08-31T06:07:41.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.435+0000 7f29267d6740 10 client.15609 fetch_fsmap finished waiting for FSMap version 30 2024-08-31T06:07:41.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.435+0000 7f29267d6740 10 client.15609 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:07:41.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.435+0000 7f29267d6740 20 client.15609 populate_metadata read hostname 'smithi016' 2024-08-31T06:07:41.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.435+0000 7f29267d6740 1 --2- 172.21.15.16:0/126401281 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55dba4c7f9f0 0x55dba5084760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:41.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.435+0000 7f29267d6740 4 client.15609 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:07:41.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.435+0000 7f29267d6740 1 -- 172.21.15.16:0/126401281 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dba4fdf6c0 con 0x55dba4c7f9f0 2024-08-31T06:07:41.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.436+0000 7f291d000640 1 --2- 172.21.15.16:0/126401281 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55dba4c7f9f0 0x55dba5084760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:41.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.436+0000 7f291d000640 1 --2- 172.21.15.16:0/126401281 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55dba4c7f9f0 0x55dba5084760 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2e266530fd0d1ac server_cookie=8245616828e8b13b in_seq=0 out_seq=0 2024-08-31T06:07:41.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.436+0000 7f28fea00640 10 client.15609 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:41.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.437+0000 7f28fea00640 1 -- 172.21.15.16:0/126401281 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dba4fdf6c0 con 0x55dba4c7f9f0 2024-08-31T06:07:41.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.437+0000 7f28fea00640 10 client.15609 handle_command_reply: tid=0 2024-08-31T06:07:41.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.437+0000 7f29267d6740 1 client.15609 shutdown 2024-08-31T06:07:41.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.438+0000 7f29267d6740 1 -- 172.21.15.16:0/126401281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba4cc1070 msgr2=0x55dba507e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:41.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.438+0000 7f29267d6740 1 --2- 172.21.15.16:0/126401281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba4cc1070 0x55dba507e5c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f29100ac1b0 tx=0x7f29100ac070 comp rx=0 tx=0).stop 2024-08-31T06:07:41.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.438+0000 7f29267d6740 1 -- 172.21.15.16:0/126401281 shutdown_connections 2024-08-31T06:07:41.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.438+0000 7f29267d6740 1 --2- 172.21.15.16:0/126401281 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55dba4c7f9f0 0x55dba5084760 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:41.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.438+0000 7f29267d6740 1 --2- 172.21.15.16:0/126401281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba4cc1070 0x55dba507e5c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:41.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.438+0000 7f29267d6740 1 --2- 172.21.15.16:0/126401281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dba4ca0390 0x55dba5072ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:41.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.438+0000 7f29267d6740 1 --2- 172.21.15.16:0/126401281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dba4c981b0 0x55dba5072660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:41.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.438+0000 7f29267d6740 1 -- 172.21.15.16:0/126401281 >> 172.21.15.16:0/126401281 conn(0x55dba4cb5460 msgr2=0x55dba5069cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:41.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.438+0000 7f28fea00640 0 client.15609 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:41.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.438+0000 7f29267d6740 1 -- 172.21.15.16:0/126401281 shutdown_connections 2024-08-31T06:07:41.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.438+0000 7f29267d6740 1 -- 172.21.15.16:0/126401281 wait complete. 2024-08-31T06:07:41.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.438+0000 7f29267d6740 20 client.15609 trim_cache size 0 max 16384 2024-08-31T06:07:41.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.457+0000 7f29267d6740 1 Processor -- start 2024-08-31T06:07:41.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.457+0000 7f29267d6740 1 -- start start 2024-08-31T06:07:41.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.457+0000 7f29267d6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dba4c981b0 0x55dba507e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:41.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.457+0000 7f29267d6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba4ca0390 0x55dba507ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:41.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.457+0000 7f29267d6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dba4cc1070 0x55dba506f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:41.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.457+0000 7f29267d6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f29181b3d70 con 0x55dba4ca0390 2024-08-31T06:07:41.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.457+0000 7f29267d6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f29181b15b0 con 0x55dba4c981b0 2024-08-31T06:07:41.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.457+0000 7f29267d6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f29181af400 con 0x55dba4cc1070 2024-08-31T06:07:41.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.457+0000 7f2917e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba4ca0390 0x55dba507ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:41.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.457+0000 7f2917e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba4ca0390 0x55dba507ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38730/0 (socket says 172.21.15.16:38730) 2024-08-31T06:07:41.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.457+0000 7f2917e00640 1 -- 172.21.15.16:0/262504104 learned_addr learned my addr 172.21.15.16:0/262504104 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:41.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.457+0000 7f291da00640 1 --2- 172.21.15.16:0/262504104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dba4cc1070 0x55dba506f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:41.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.457+0000 7f291d000640 1 --2- 172.21.15.16:0/262504104 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dba4c981b0 0x55dba507e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:41.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.458+0000 7f2917e00640 1 -- 172.21.15.16:0/262504104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dba4cc1070 msgr2=0x55dba506f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:41.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.458+0000 7f2917e00640 1 --2- 172.21.15.16:0/262504104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dba4cc1070 0x55dba506f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:41.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.458+0000 7f2917e00640 1 -- 172.21.15.16:0/262504104 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dba4c981b0 msgr2=0x55dba507e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:41.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.458+0000 7f2917e00640 1 --2- 172.21.15.16:0/262504104 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dba4c981b0 0x55dba507e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:41.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.458+0000 7f2917e00640 1 -- 172.21.15.16:0/262504104 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dba5069980 con 0x55dba4ca0390 2024-08-31T06:07:41.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.458+0000 7f291d000640 1 --2- 172.21.15.16:0/262504104 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dba4c981b0 0x55dba507e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:07:41.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.458+0000 7f2917e00640 1 --2- 172.21.15.16:0/262504104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba4ca0390 0x55dba507ea00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2904004140 tx=0x7f290401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fc1757d2d0b9101 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:41.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.458+0000 7f2909600640 1 -- 172.21.15.16:0/262504104 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2904003bb0 con 0x55dba4ca0390 2024-08-31T06:07:41.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.459+0000 7f2909600640 1 -- 172.21.15.16:0/262504104 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2904003d50 con 0x55dba4ca0390 2024-08-31T06:07:41.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.459+0000 7f2909600640 1 -- 172.21.15.16:0/262504104 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29040207e0 con 0x55dba4ca0390 2024-08-31T06:07:41.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.459+0000 7f29267d6740 1 -- 172.21.15.16:0/262504104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba4ca0390 msgr2=0x55dba507ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.459+0000 7f29267d6740 1 --2- 172.21.15.16:0/262504104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba4ca0390 0x55dba507ea00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2904004140 tx=0x7f290401a000 comp rx=0 tx=0).stop 2024-08-31T06:07:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.459+0000 7f29267d6740 1 -- 172.21.15.16:0/262504104 shutdown_connections 2024-08-31T06:07:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.459+0000 7f29267d6740 1 --2- 172.21.15.16:0/262504104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dba4cc1070 0x55dba506f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.459+0000 7f29267d6740 1 --2- 172.21.15.16:0/262504104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba4ca0390 0x55dba507ea00 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.459+0000 7f29267d6740 1 --2- 172.21.15.16:0/262504104 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dba4c981b0 0x55dba507e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.459+0000 7f29267d6740 1 -- 172.21.15.16:0/262504104 >> 172.21.15.16:0/262504104 conn(0x55dba4cb5460 msgr2=0x55dba5070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.459+0000 7f29267d6740 1 -- 172.21.15.16:0/262504104 shutdown_connections 2024-08-31T06:07:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.460+0000 7f29267d6740 1 -- 172.21.15.16:0/262504104 wait complete. 2024-08-31T06:07:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.460+0000 7f29267d6740 1 -- start start 2024-08-31T06:07:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.460+0000 7f29267d6740 1 Processor -- start 2024-08-31T06:07:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.461+0000 7f29267d6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dba4c981b0 0x55dba507aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.461+0000 7f29267d6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba4ca0390 0x55dba50754e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.461+0000 7f29267d6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dba4cc1070 0x55dba5075a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.461+0000 7f29267d6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f29181afa70 con 0x55dba4ca0390 2024-08-31T06:07:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.461+0000 7f29267d6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f29181b0ee0 con 0x55dba4cc1070 2024-08-31T06:07:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.461+0000 7f29267d6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f29181b0140 con 0x55dba4c981b0 2024-08-31T06:07:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.461+0000 7f29267d6740 10 client.0 _ll_register_callbacks cb 0x7f29100a80e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:07:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.461+0000 7f2917e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba4ca0390 0x55dba50754e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.461+0000 7f2917e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba4ca0390 0x55dba50754e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38732/0 (socket says 172.21.15.16:38732) 2024-08-31T06:07:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.461+0000 7f2917e00640 1 -- 172.21.15.16:0/1642115305 learned_addr learned my addr 172.21.15.16:0/1642115305 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.461+0000 7f291da00640 1 --2- 172.21.15.16:0/1642115305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dba4cc1070 0x55dba5075a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.461+0000 7f291d000640 1 --2- 172.21.15.16:0/1642115305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dba4c981b0 0x55dba507aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.461+0000 7f2917e00640 1 -- 172.21.15.16:0/1642115305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dba4c981b0 msgr2=0x55dba507aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.461+0000 7f2917e00640 1 --2- 172.21.15.16:0/1642115305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dba4c981b0 0x55dba507aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.461+0000 7f2917e00640 1 -- 172.21.15.16:0/1642115305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dba4cc1070 msgr2=0x55dba5075a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.461+0000 7f2917e00640 1 --2- 172.21.15.16:0/1642115305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dba4cc1070 0x55dba5075a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.461+0000 7f2917e00640 1 -- 172.21.15.16:0/1642115305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dba5075f60 con 0x55dba4ca0390 2024-08-31T06:07:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.462+0000 7f2917e00640 1 --2- 172.21.15.16:0/1642115305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba4ca0390 0x55dba50754e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2904005890 tx=0x7f29040096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd191b75425283bf server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.462+0000 7f28fea00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:07:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.462+0000 7f28fea00640 1 -- 172.21.15.16:0/1642115305 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290401f670 con 0x55dba4ca0390 2024-08-31T06:07:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.462+0000 7f28fea00640 1 -- 172.21.15.16:0/1642115305 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f2904019070 con 0x55dba4ca0390 2024-08-31T06:07:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.462+0000 7f29267d6740 1 -- 172.21.15.16:0/1642115305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2904003d50 con 0x55dba4ca0390 2024-08-31T06:07:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.462+0000 7f28fea00640 1 -- 172.21.15.16:0/1642115305 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f290401f810 con 0x55dba4ca0390 2024-08-31T06:07:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.462+0000 7f29267d6740 10 client.15615 fetch_fsmap learned FSMap version 30 2024-08-31T06:07:41.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.462+0000 7f29267d6740 1 -- 172.21.15.16:0/1642115305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dba50efdb0 con 0x55dba4ca0390 2024-08-31T06:07:41.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.463+0000 7f28fea00640 1 -- 172.21.15.16:0/1642115305 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2904003d50 con 0x55dba4ca0390 2024-08-31T06:07:41.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.463+0000 7f28fea00640 1 -- 172.21.15.16:0/1642115305 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f2904020b10 con 0x55dba4ca0390 2024-08-31T06:07:41.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.463+0000 7f29267d6740 10 client.15615 fetch_fsmap finished waiting for FSMap version 30 2024-08-31T06:07:41.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.463+0000 7f29267d6740 10 client.15615 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:07:41.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.463+0000 7f29267d6740 20 client.15615 populate_metadata read hostname 'smithi016' 2024-08-31T06:07:41.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.463+0000 7f29267d6740 1 --2- 172.21.15.16:0/1642115305 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55dba4c54da0 0x55dba50f24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:41.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.463+0000 7f29267d6740 4 client.15615 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T06:07:41.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.463+0000 7f29267d6740 1 -- 172.21.15.16:0/1642115305 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dba4fdb160 con 0x55dba4c54da0 2024-08-31T06:07:41.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.463+0000 7f291d000640 1 --2- 172.21.15.16:0/1642115305 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55dba4c54da0 0x55dba50f24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:41.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.464+0000 7f291d000640 1 --2- 172.21.15.16:0/1642115305 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55dba4c54da0 0x55dba50f24c0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa23ba5c16b4c54f server_cookie=608b46492d3b5760 in_seq=0 out_seq=0 2024-08-31T06:07:41.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.464+0000 7f28fea00640 10 client.15615 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:41.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.464+0000 7f28fea00640 1 -- 172.21.15.16:0/1642115305 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+854 (crc 0 0 0) 0x55dba4fdb160 con 0x55dba4c54da0 2024-08-31T06:07:41.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.464+0000 7f28fea00640 10 client.15615 handle_command_reply: tid=0 2024-08-31T06:07:41.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.464+0000 7f29267d6740 1 client.15615 shutdown 2024-08-31T06:07:41.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.464+0000 7f29267d6740 1 -- 172.21.15.16:0/1642115305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba4ca0390 msgr2=0x55dba50754e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:41.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.464+0000 7f29267d6740 1 --2- 172.21.15.16:0/1642115305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba4ca0390 0x55dba50754e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2904005890 tx=0x7f29040096a0 comp rx=0 tx=0).stop 2024-08-31T06:07:41.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.464+0000 7f29267d6740 1 -- 172.21.15.16:0/1642115305 shutdown_connections 2024-08-31T06:07:41.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.464+0000 7f29267d6740 1 --2- 172.21.15.16:0/1642115305 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55dba4c54da0 0x55dba50f24c0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:41.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.464+0000 7f29267d6740 1 --2- 172.21.15.16:0/1642115305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dba4cc1070 0x55dba5075a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:41.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.464+0000 7f29267d6740 1 --2- 172.21.15.16:0/1642115305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba4ca0390 0x55dba50754e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:41.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.464+0000 7f29267d6740 1 --2- 172.21.15.16:0/1642115305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dba4c981b0 0x55dba507aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:41.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.464+0000 7f29267d6740 1 -- 172.21.15.16:0/1642115305 >> 172.21.15.16:0/1642115305 conn(0x55dba4cb5460 msgr2=0x55dba5062fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:41.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.464+0000 7f28fea00640 0 client.15615 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:41.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.465+0000 7f29267d6740 1 -- 172.21.15.16:0/1642115305 shutdown_connections 2024-08-31T06:07:41.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.465+0000 7f29267d6740 1 -- 172.21.15.16:0/1642115305 wait complete. 2024-08-31T06:07:41.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.465+0000 7f29267d6740 20 client.15615 trim_cache size 0 max 16384 2024-08-31T06:07:41.471 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:07:41.471 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T06:07:41.471 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-08-31T06:07:41.471 INFO:teuthology.orchestra.run.smithi016.stdout: "88d7bdb8-cb1c-45e5-ad29-4cd8da6c6003": { 2024-08-31T06:07:41.471 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:07:41.471 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "88d7bdb8-cb1c-45e5-ad29-4cd8da6c6003", 2024-08-31T06:07:41.471 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-08-31T06:07:41.471 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:07:41.471 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:07:41.471 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-08-31T06:07:41.471 INFO:teuthology.orchestra.run.smithi016.stdout: "88d7bdb8-cb1c-45e5-ad29-4cd8da6c6003": { 2024-08-31T06:07:41.471 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:07:39.648942+0000", 2024-08-31T06:07:41.472 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:07:41.472 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:07:41.472 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:07:41.472 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:07:41.472 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:07:41.472 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:07:41.472 INFO:teuthology.orchestra.run.smithi016.stdout: "c300b7e8-f3fd-482c-a909-9329bc61e570": { 2024-08-31T06:07:41.472 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:06:49.868046+0000", 2024-08-31T06:07:41.472 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:07:41.472 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:07:41.472 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:07:41.472 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:07:41.473 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:07:41.473 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:07:41.473 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:07:41.473 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:07:41.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:41 smithi184 ceph-mon[22331]: pgmap v377: 97 pgs: 97 active+clean; 899 MiB data, 776 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 18 MiB/s wr, 454 op/s 2024-08-31T06:07:41.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:41 smithi184 ceph-mon[22331]: scrub queued for path: / 2024-08-31T06:07:41.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:41 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:07:41.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:41 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:07:41.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:41 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:07:41.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:41 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:07:41.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.465+0000 7f291e400640 1 -- 172.21.15.16:0/3196687005 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f29000b4ca0 msgr2=0x7f29000b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:41.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.465+0000 7f291e400640 1 --2- 172.21.15.16:0/3196687005 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f29000b4ca0 0x7f29000b71b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f291810ac90 tx=0x7f2904013040 comp rx=0 tx=0).stop 2024-08-31T06:07:41.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.465+0000 7f291e400640 1 -- 172.21.15.16:0/3196687005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f291810bbe0 msgr2=0x7f29181a1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:41.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.465+0000 7f291e400640 1 --2- 172.21.15.16:0/3196687005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f291810bbe0 0x7f29181a1af0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f291000eb10 tx=0x7f291000c2d0 comp rx=0 tx=0).stop 2024-08-31T06:07:41.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.466+0000 7f291e400640 1 -- 172.21.15.16:0/3196687005 shutdown_connections 2024-08-31T06:07:41.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.466+0000 7f291e400640 1 --2- 172.21.15.16:0/3196687005 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f29000b4ca0 0x7f29000b71b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:41.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.466+0000 7f291e400640 1 --2- 172.21.15.16:0/3196687005 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f291810b290 0x7f291819c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:41.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.466+0000 7f291e400640 1 --2- 172.21.15.16:0/3196687005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f291810bbe0 0x7f29181a1af0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:41.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.466+0000 7f291e400640 1 --2- 172.21.15.16:0/3196687005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2918105950 0x7f291819c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:41.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.466+0000 7f291e400640 1 -- 172.21.15.16:0/3196687005 >> 172.21.15.16:0/3196687005 conn(0x7f2918101420 msgr2=0x7f2918103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:41.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.466+0000 7f291e400640 1 -- 172.21.15.16:0/3196687005 shutdown_connections 2024-08-31T06:07:41.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:41.466+0000 7f291e400640 1 -- 172.21.15.16:0/3196687005 wait complete. 2024-08-31T06:07:41.627 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 15.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-31T06:07:42.229 INFO:tasks.cephfs.filesystem:scrub status for tag:88d7bdb8-cb1c-45e5-ad29-4cd8da6c6003 - {'path': '/', 'tag': '88d7bdb8-cb1c-45e5-ad29-4cd8da6c6003', 'options': 'recursive,force'} 2024-08-31T06:07:43.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:43 smithi016 ceph-mon[17194]: pgmap v378: 97 pgs: 97 active+clean; 903 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 10 MiB/s wr, 512 op/s 2024-08-31T06:07:43.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:43 smithi155 ceph-mon[21742]: pgmap v378: 97 pgs: 97 active+clean; 903 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 10 MiB/s wr, 512 op/s 2024-08-31T06:07:43.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:43 smithi184 ceph-mon[22331]: pgmap v378: 97 pgs: 97 active+clean; 903 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 10 MiB/s wr, 512 op/s 2024-08-31T06:07:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.661+0000 7fbd97200640 1 -- 172.21.15.16:0/3779823953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd98104e50 msgr2=0x7fbd9810d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.661+0000 7fbd97200640 1 --2- 172.21.15.16:0/3779823953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd98104e50 0x7fbd9810d390 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fbd80009900 tx=0x7fbd8002a3f0 comp rx=0 tx=0).stop 2024-08-31T06:07:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.661+0000 7fbd97200640 1 -- 172.21.15.16:0/3779823953 shutdown_connections 2024-08-31T06:07:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.661+0000 7fbd97200640 1 --2- 172.21.15.16:0/3779823953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd9810d960 0x7fbd9810fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.661+0000 7fbd97200640 1 --2- 172.21.15.16:0/3779823953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd98104e50 0x7fbd9810d390 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.661+0000 7fbd97200640 1 --2- 172.21.15.16:0/3779823953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd98104500 0x7fbd98104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.661+0000 7fbd97200640 1 -- 172.21.15.16:0/3779823953 >> 172.21.15.16:0/3779823953 conn(0x7fbd980fdfd0 msgr2=0x7fbd98100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:44.665 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.661+0000 7fbd97200640 1 -- 172.21.15.16:0/3779823953 shutdown_connections 2024-08-31T06:07:44.665 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.661+0000 7fbd97200640 1 -- 172.21.15.16:0/3779823953 wait complete. 2024-08-31T06:07:44.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.662+0000 7fbd97200640 1 Processor -- start 2024-08-31T06:07:44.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.662+0000 7fbd97200640 1 -- start start 2024-08-31T06:07:44.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.663+0000 7fbd97200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd98104500 0x7fbd9819c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:44.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.663+0000 7fbd97200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd98104e50 0x7fbd9819c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:44.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.663+0000 7fbd97200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd9810d960 0x7fbd981a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:44.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.663+0000 7fbd97200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbd981135f0 con 0x7fbd9810d960 2024-08-31T06:07:44.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.663+0000 7fbd97200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbd98113470 con 0x7fbd98104e50 2024-08-31T06:07:44.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.663+0000 7fbd97200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbd98113770 con 0x7fbd98104500 2024-08-31T06:07:44.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.663+0000 7fbd96800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd9810d960 0x7fbd981a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:44.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.663+0000 7fbd96800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd9810d960 0x7fbd981a1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38760/0 (socket says 172.21.15.16:38760) 2024-08-31T06:07:44.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.663+0000 7fbd96800640 1 -- 172.21.15.16:0/673101259 learned_addr learned my addr 172.21.15.16:0/673101259 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:44.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.663+0000 7fbd95e00640 1 --2- 172.21.15.16:0/673101259 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd98104500 0x7fbd9819c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:44.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.663+0000 7fbd8d400640 1 --2- 172.21.15.16:0/673101259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd98104e50 0x7fbd9819c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:44.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.664+0000 7fbd96800640 1 -- 172.21.15.16:0/673101259 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd98104500 msgr2=0x7fbd9819c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:44.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.664+0000 7fbd96800640 1 --2- 172.21.15.16:0/673101259 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd98104500 0x7fbd9819c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:44.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.664+0000 7fbd96800640 1 -- 172.21.15.16:0/673101259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd98104e50 msgr2=0x7fbd9819c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:44.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.664+0000 7fbd96800640 1 --2- 172.21.15.16:0/673101259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd98104e50 0x7fbd9819c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:44.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.664+0000 7fbd96800640 1 -- 172.21.15.16:0/673101259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd981a2200 con 0x7fbd9810d960 2024-08-31T06:07:44.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.664+0000 7fbd96800640 1 --2- 172.21.15.16:0/673101259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd9810d960 0x7fbd981a1b00 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fbd9000eb10 tx=0x7fbd9000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:44.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.664+0000 7fbd8f400640 1 -- 172.21.15.16:0/673101259 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd90004b20 con 0x7fbd9810d960 2024-08-31T06:07:44.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.664+0000 7fbd8f400640 1 -- 172.21.15.16:0/673101259 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd90004cc0 con 0x7fbd9810d960 2024-08-31T06:07:44.669 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.664+0000 7fbd8f400640 1 -- 172.21.15.16:0/673101259 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd90010670 con 0x7fbd9810d960 2024-08-31T06:07:44.669 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.664+0000 7fbd97200640 1 -- 172.21.15.16:0/673101259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd981a24f0 con 0x7fbd9810d960 2024-08-31T06:07:44.669 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.664+0000 7fbd97200640 1 -- 172.21.15.16:0/673101259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd981a8c80 con 0x7fbd9810d960 2024-08-31T06:07:44.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.668+0000 7fbda3c1f740 1 Processor -- start 2024-08-31T06:07:44.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.668+0000 7fbd8f400640 1 -- 172.21.15.16:0/673101259 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fbd90005020 con 0x7fbd9810d960 2024-08-31T06:07:44.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.668+0000 7fbda3c1f740 1 -- start start 2024-08-31T06:07:44.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.668+0000 7fbda3c1f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620722121b0 0x5620725b58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:44.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.669+0000 7fbda3c1f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620721e92f0 0x5620725b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:44.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.669+0000 7fbda3c1f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620721f14d0 0x5620725b6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:44.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.669+0000 7fbda3c1f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56207252af70 con 0x5620721e92f0 2024-08-31T06:07:44.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.669+0000 7fbda3c1f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56207251a500 con 0x5620721f14d0 2024-08-31T06:07:44.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.669+0000 7fbda3c1f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56207251e4b0 con 0x5620722121b0 2024-08-31T06:07:44.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.669+0000 7fbd95e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620721e92f0 0x5620725b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:44.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.669+0000 7fbd95e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620721e92f0 0x5620725b5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38768/0 (socket says 172.21.15.16:38768) 2024-08-31T06:07:44.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.669+0000 7fbd95e00640 1 -- 172.21.15.16:0/1898791809 learned_addr learned my addr 172.21.15.16:0/1898791809 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:44.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.669+0000 7fbd8d400640 1 --2- 172.21.15.16:0/1898791809 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620722121b0 0x5620725b58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:44.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.669+0000 7fbd8d400640 1 --2- 172.21.15.16:0/1898791809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620721f14d0 0x5620725b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:44.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.670+0000 7fbd95e00640 1 -- 172.21.15.16:0/1898791809 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620722121b0 msgr2=0x5620725b58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:44.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.670+0000 7fbd95e00640 1 --2- 172.21.15.16:0/1898791809 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620722121b0 0x5620725b58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:44.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.670+0000 7fbd95e00640 1 -- 172.21.15.16:0/1898791809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620721f14d0 msgr2=0x5620725b6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:44.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.670+0000 7fbd95e00640 1 --2- 172.21.15.16:0/1898791809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620721f14d0 0x5620725b6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:44.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.670+0000 7fbd95e00640 1 -- 172.21.15.16:0/1898791809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620724d4c10 con 0x5620721e92f0 2024-08-31T06:07:44.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.670+0000 7fbd8d400640 1 --2- 172.21.15.16:0/1898791809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620721f14d0 0x5620725b6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:07:44.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.670+0000 7fbd95e00640 1 --2- 172.21.15.16:0/1898791809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620721e92f0 0x5620725b5de0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fbd88009d20 tx=0x7fbd88011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d353cfeb79a305d5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:44.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.670+0000 7fbd8f400640 1 --2- 172.21.15.16:0/673101259 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbd7c0b4d20 0x7fbd7c0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:44.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.670+0000 7fbd87000640 1 -- 172.21.15.16:0/1898791809 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8801e070 con 0x5620721e92f0 2024-08-31T06:07:44.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.670+0000 7fbd87000640 1 -- 172.21.15.16:0/1898791809 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd88019ae0 con 0x5620721e92f0 2024-08-31T06:07:44.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.670+0000 7fbd87000640 1 -- 172.21.15.16:0/1898791809 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8800eab0 con 0x5620721e92f0 2024-08-31T06:07:44.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.670+0000 7fbd8f400640 1 -- 172.21.15.16:0/673101259 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fbd900e5a20 con 0x7fbd9810d960 2024-08-31T06:07:44.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.670+0000 7fbd8d400640 1 --2- 172.21.15.16:0/673101259 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbd7c0b4d20 0x7fbd7c0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:44.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.671+0000 7fbda3c1f740 1 -- 172.21.15.16:0/1898791809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620721e92f0 msgr2=0x5620725b5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:44.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.671+0000 7fbda3c1f740 1 --2- 172.21.15.16:0/1898791809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620721e92f0 0x5620725b5de0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fbd88009d20 tx=0x7fbd88011810 comp rx=0 tx=0).stop 2024-08-31T06:07:44.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.671+0000 7fbd8d400640 1 --2- 172.21.15.16:0/673101259 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbd7c0b4d20 0x7fbd7c0b7230 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbd8002a9e0 tx=0x7fbd80005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.671+0000 7fbda3c1f740 1 -- 172.21.15.16:0/1898791809 shutdown_connections 2024-08-31T06:07:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.671+0000 7fbda3c1f740 1 --2- 172.21.15.16:0/1898791809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620721f14d0 0x5620725b6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.671+0000 7fbda3c1f740 1 --2- 172.21.15.16:0/1898791809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620721e92f0 0x5620725b5de0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.671+0000 7fbda3c1f740 1 --2- 172.21.15.16:0/1898791809 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620722121b0 0x5620725b58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.671+0000 7fbda3c1f740 1 -- 172.21.15.16:0/1898791809 >> 172.21.15.16:0/1898791809 conn(0x562072206300 msgr2=0x5620725b3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.671+0000 7fbda3c1f740 1 -- 172.21.15.16:0/1898791809 shutdown_connections 2024-08-31T06:07:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.671+0000 7fbda3c1f740 1 -- 172.21.15.16:0/1898791809 wait complete. 2024-08-31T06:07:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.672+0000 7fbda3c1f740 1 -- start start 2024-08-31T06:07:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.672+0000 7fbda3c1f740 1 Processor -- start 2024-08-31T06:07:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.673+0000 7fbda3c1f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620721e92f0 0x5620725c0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.673+0000 7fbda3c1f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620721f14d0 0x5620725c1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.673+0000 7fbda3c1f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620722121b0 0x5620725c1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.673+0000 7fbda3c1f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562072453ab0 con 0x5620721e92f0 2024-08-31T06:07:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.673+0000 7fbda3c1f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5620724c1320 con 0x5620722121b0 2024-08-31T06:07:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.673+0000 7fbda3c1f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5620725c4020 con 0x5620721f14d0 2024-08-31T06:07:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.673+0000 7fbda3c1f740 10 client.0 _ll_register_callbacks cb 0x5620724fae80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:07:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.673+0000 7fbd95e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620721e92f0 0x5620725c0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.673+0000 7fbd96800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620722121b0 0x5620725c1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.673+0000 7fbd95e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620721e92f0 0x5620725c0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38778/0 (socket says 172.21.15.16:38778) 2024-08-31T06:07:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.673+0000 7fbd96800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620722121b0 0x5620725c1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:42672/0 (socket says 172.21.15.16:42672) 2024-08-31T06:07:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.673+0000 7fbd95e00640 1 -- 172.21.15.16:0/90753699 learned_addr learned my addr 172.21.15.16:0/90753699 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.673+0000 7fbd8d400640 1 --2- 172.21.15.16:0/90753699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620721f14d0 0x5620725c1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.673+0000 7fbd96800640 1 -- 172.21.15.16:0/90753699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620721f14d0 msgr2=0x5620725c1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.673+0000 7fbd96800640 1 --2- 172.21.15.16:0/90753699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620721f14d0 0x5620725c1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.673+0000 7fbd96800640 1 -- 172.21.15.16:0/90753699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620721e92f0 msgr2=0x5620725c0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.673+0000 7fbd96800640 1 --2- 172.21.15.16:0/90753699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620721e92f0 0x5620725c0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.673+0000 7fbd96800640 1 -- 172.21.15.16:0/90753699 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56207227d790 con 0x5620722121b0 2024-08-31T06:07:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.673+0000 7fbd8d400640 1 --2- 172.21.15.16:0/90753699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620721f14d0 0x5620725c1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:07:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.673+0000 7fbd95e00640 1 --2- 172.21.15.16:0/90753699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620721e92f0 0x5620725c0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:07:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.674+0000 7fbd96800640 1 --2- 172.21.15.16:0/90753699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620722121b0 0x5620725c1880 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbd900e3170 tx=0x7fbd900abb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1763f4c4e963b6e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.674+0000 7fbd7b400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:07:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.674+0000 7fbd7b400640 1 -- 172.21.15.16:0/90753699 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd900ac110 con 0x5620722121b0 2024-08-31T06:07:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.674+0000 7fbda3c1f740 1 -- 172.21.15.16:0/90753699 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd88019ae0 con 0x5620722121b0 2024-08-31T06:07:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.674+0000 7fbd7b400640 1 -- 172.21.15.16:0/90753699 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fbd900ac580 con 0x5620722121b0 2024-08-31T06:07:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.674+0000 7fbd7b400640 1 -- 172.21.15.16:0/90753699 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbd900b0980 con 0x5620722121b0 2024-08-31T06:07:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.674+0000 7fbda3c1f740 10 client.25667 fetch_fsmap learned FSMap version 30 2024-08-31T06:07:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.674+0000 7fbda3c1f740 1 -- 172.21.15.16:0/90753699 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5620722f6620 con 0x5620722121b0 2024-08-31T06:07:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.674+0000 7fbd7b400640 1 -- 172.21.15.16:0/90753699 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd88019ae0 con 0x5620722121b0 2024-08-31T06:07:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.675+0000 7fbd7b400640 1 -- 172.21.15.16:0/90753699 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fbd900a7a80 con 0x5620722121b0 2024-08-31T06:07:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.675+0000 7fbda3c1f740 10 client.25667 fetch_fsmap finished waiting for FSMap version 30 2024-08-31T06:07:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.675+0000 7fbda3c1f740 10 client.25667 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:07:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.675+0000 7fbda3c1f740 20 client.25667 populate_metadata read hostname 'smithi016' 2024-08-31T06:07:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.675+0000 7fbda3c1f740 1 --2- 172.21.15.16:0/90753699 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5620721d0d80 0x5620725d3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.675+0000 7fbda3c1f740 4 client.25667 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:07:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.675+0000 7fbda3c1f740 1 -- 172.21.15.16:0/90753699 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56207252cec0 con 0x5620721d0d80 2024-08-31T06:07:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.675+0000 7fbd95e00640 1 --2- 172.21.15.16:0/90753699 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5620721d0d80 0x5620725d3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:44.682 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.676+0000 7fbd95e00640 1 --2- 172.21.15.16:0/90753699 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5620721d0d80 0x5620725d3080 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9475e77ebf8d4b8a server_cookie=4bdca7e6d7b51230 in_seq=0 out_seq=0 2024-08-31T06:07:44.682 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.676+0000 7fbd7b400640 10 client.25667 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:44.682 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.677+0000 7fbd7b400640 1 -- 172.21.15.16:0/90753699 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56207252cec0 con 0x5620721d0d80 2024-08-31T06:07:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.677+0000 7fbd7b400640 10 client.25667 handle_command_reply: tid=0 2024-08-31T06:07:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.677+0000 7fbda3c1f740 1 client.25667 shutdown 2024-08-31T06:07:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.677+0000 7fbda3c1f740 1 -- 172.21.15.16:0/90753699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620722121b0 msgr2=0x5620725c1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.677+0000 7fbda3c1f740 1 --2- 172.21.15.16:0/90753699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620722121b0 0x5620725c1880 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbd900e3170 tx=0x7fbd900abb20 comp rx=0 tx=0).stop 2024-08-31T06:07:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.677+0000 7fbda3c1f740 1 -- 172.21.15.16:0/90753699 shutdown_connections 2024-08-31T06:07:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.677+0000 7fbda3c1f740 1 --2- 172.21.15.16:0/90753699 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5620721d0d80 0x5620725d3080 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.677+0000 7fbda3c1f740 1 --2- 172.21.15.16:0/90753699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620721f14d0 0x5620725c1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.677+0000 7fbda3c1f740 1 --2- 172.21.15.16:0/90753699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620722121b0 0x5620725c1880 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.677+0000 7fbda3c1f740 1 --2- 172.21.15.16:0/90753699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620721e92f0 0x5620725c0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.677+0000 7fbd7b400640 0 client.25667 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.677+0000 7fbda3c1f740 1 -- 172.21.15.16:0/90753699 >> 172.21.15.16:0/90753699 conn(0x562072206300 msgr2=0x5620725b68d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.677+0000 7fbda3c1f740 1 -- 172.21.15.16:0/90753699 shutdown_connections 2024-08-31T06:07:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.677+0000 7fbda3c1f740 1 -- 172.21.15.16:0/90753699 wait complete. 2024-08-31T06:07:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.677+0000 7fbda3c1f740 20 client.25667 trim_cache size 0 max 16384 2024-08-31T06:07:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.693+0000 7fbda3c1f740 1 Processor -- start 2024-08-31T06:07:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.693+0000 7fbda3c1f740 1 -- start start 2024-08-31T06:07:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.693+0000 7fbda3c1f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620721e92f0 0x5620725b2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.693+0000 7fbda3c1f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620721f14d0 0x5620725bdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.693+0000 7fbda3c1f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620722121b0 0x5620725be510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:44.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.693+0000 7fbda3c1f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbd981b3ca0 con 0x5620721e92f0 2024-08-31T06:07:44.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.693+0000 7fbda3c1f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbd981b14e0 con 0x5620722121b0 2024-08-31T06:07:44.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.693+0000 7fbda3c1f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbd981af330 con 0x5620721f14d0 2024-08-31T06:07:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.693+0000 7fbd95e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620721e92f0 0x5620725b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.693+0000 7fbd8d400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620721f14d0 0x5620725bdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.693+0000 7fbd96800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620722121b0 0x5620725be510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.693+0000 7fbd95e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620721e92f0 0x5620725b2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38794/0 (socket says 172.21.15.16:38794) 2024-08-31T06:07:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.693+0000 7fbd95e00640 1 -- 172.21.15.16:0/3091633715 learned_addr learned my addr 172.21.15.16:0/3091633715 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.694+0000 7fbd8d400640 1 -- 172.21.15.16:0/3091633715 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620722121b0 msgr2=0x5620725be510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.694+0000 7fbd8d400640 1 --2- 172.21.15.16:0/3091633715 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620722121b0 0x5620725be510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.694+0000 7fbd8d400640 1 -- 172.21.15.16:0/3091633715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620721e92f0 msgr2=0x5620725b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.694+0000 7fbd8d400640 1 --2- 172.21.15.16:0/3091633715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620721e92f0 0x5620725b2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.694+0000 7fbd8d400640 1 -- 172.21.15.16:0/3091633715 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620725b4130 con 0x5620721f14d0 2024-08-31T06:07:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.694+0000 7fbd95e00640 1 --2- 172.21.15.16:0/3091633715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620721e92f0 0x5620725b2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:07:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.694+0000 7fbd8d400640 1 --2- 172.21.15.16:0/3091633715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620721f14d0 0x5620725bdfd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbd80009230 tx=0x7fbd8002a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e52dba352584af89 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.694+0000 7fbd86600640 1 -- 172.21.15.16:0/3091633715 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd800314a0 con 0x5620721f14d0 2024-08-31T06:07:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.694+0000 7fbd86600640 1 -- 172.21.15.16:0/3091633715 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd80031af0 con 0x5620721f14d0 2024-08-31T06:07:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.694+0000 7fbd86600640 1 -- 172.21.15.16:0/3091633715 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8003db80 con 0x5620721f14d0 2024-08-31T06:07:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.695+0000 7fbda3c1f740 1 -- 172.21.15.16:0/3091633715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620721f14d0 msgr2=0x5620725bdfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.695+0000 7fbda3c1f740 1 --2- 172.21.15.16:0/3091633715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620721f14d0 0x5620725bdfd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbd80009230 tx=0x7fbd8002a3f0 comp rx=0 tx=0).stop 2024-08-31T06:07:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.695+0000 7fbda3c1f740 1 -- 172.21.15.16:0/3091633715 shutdown_connections 2024-08-31T06:07:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.695+0000 7fbda3c1f740 1 --2- 172.21.15.16:0/3091633715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620721f14d0 0x5620725bdfd0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.695+0000 7fbda3c1f740 1 --2- 172.21.15.16:0/3091633715 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620722121b0 0x5620725be510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.695+0000 7fbda3c1f740 1 --2- 172.21.15.16:0/3091633715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620721e92f0 0x5620725b2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.695+0000 7fbda3c1f740 1 -- 172.21.15.16:0/3091633715 >> 172.21.15.16:0/3091633715 conn(0x562072206300 msgr2=0x5620725b1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.695+0000 7fbda3c1f740 1 -- 172.21.15.16:0/3091633715 shutdown_connections 2024-08-31T06:07:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.695+0000 7fbda3c1f740 1 -- 172.21.15.16:0/3091633715 wait complete. 2024-08-31T06:07:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.696+0000 7fbda3c1f740 1 -- start start 2024-08-31T06:07:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.696+0000 7fbda3c1f740 1 Processor -- start 2024-08-31T06:07:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.696+0000 7fbda3c1f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620721e92f0 0x5620725c6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.696+0000 7fbda3c1f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620721f14d0 0x5620725c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.696+0000 7fbda3c1f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620722121b0 0x562072634a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.696+0000 7fbda3c1f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbd981af9a0 con 0x5620721f14d0 2024-08-31T06:07:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.696+0000 7fbda3c1f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbd981b0e10 con 0x5620721e92f0 2024-08-31T06:07:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.696+0000 7fbda3c1f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbd981b0070 con 0x5620722121b0 2024-08-31T06:07:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.696+0000 7fbda3c1f740 10 client.0 _ll_register_callbacks cb 0x7fbd900abee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:07:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.697+0000 7fbd8d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620721f14d0 0x5620725c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.697+0000 7fbd8d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620721f14d0 0x5620725c72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38810/0 (socket says 172.21.15.16:38810) 2024-08-31T06:07:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.697+0000 7fbd95e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620721e92f0 0x5620725c6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.697+0000 7fbd8d400640 1 -- 172.21.15.16:0/3812963109 learned_addr learned my addr 172.21.15.16:0/3812963109 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.697+0000 7fbd96800640 1 --2- 172.21.15.16:0/3812963109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620722121b0 0x562072634a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.697+0000 7fbd8d400640 1 -- 172.21.15.16:0/3812963109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620722121b0 msgr2=0x562072634a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.697+0000 7fbd8d400640 1 --2- 172.21.15.16:0/3812963109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620722121b0 0x562072634a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.697+0000 7fbd8d400640 1 -- 172.21.15.16:0/3812963109 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620721e92f0 msgr2=0x5620725c6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.697+0000 7fbd8d400640 1 --2- 172.21.15.16:0/3812963109 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620721e92f0 0x5620725c6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.697+0000 7fbd8d400640 1 -- 172.21.15.16:0/3812963109 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620726350e0 con 0x5620721f14d0 2024-08-31T06:07:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.697+0000 7fbd96800640 1 --2- 172.21.15.16:0/3812963109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620722121b0 0x562072634a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:07:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.697+0000 7fbd8d400640 1 --2- 172.21.15.16:0/3812963109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620721f14d0 0x5620725c72f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbd80005ab0 tx=0x7fbd80031520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d4235df16bad44 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.697+0000 7fbd7b400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:07:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.698+0000 7fbd7b400640 1 -- 172.21.15.16:0/3812963109 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8005c070 con 0x5620721f14d0 2024-08-31T06:07:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.698+0000 7fbd7b400640 1 -- 172.21.15.16:0/3812963109 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fbd8003d650 con 0x5620721f14d0 2024-08-31T06:07:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.698+0000 7fbda3c1f740 1 -- 172.21.15.16:0/3812963109 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd80031af0 con 0x5620721f14d0 2024-08-31T06:07:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.698+0000 7fbd7b400640 1 -- 172.21.15.16:0/3812963109 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbd80068050 con 0x5620721f14d0 2024-08-31T06:07:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.698+0000 7fbda3c1f740 10 client.15633 fetch_fsmap learned FSMap version 30 2024-08-31T06:07:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.698+0000 7fbda3c1f740 1 -- 172.21.15.16:0/3812963109 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56207263d570 con 0x5620721f14d0 2024-08-31T06:07:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.698+0000 7fbd7b400640 1 -- 172.21.15.16:0/3812963109 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd80031af0 con 0x5620721f14d0 2024-08-31T06:07:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.698+0000 7fbd7b400640 1 -- 172.21.15.16:0/3812963109 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fbd8003c9e0 con 0x5620721f14d0 2024-08-31T06:07:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.698+0000 7fbda3c1f740 10 client.15633 fetch_fsmap finished waiting for FSMap version 30 2024-08-31T06:07:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.698+0000 7fbda3c1f740 10 client.15633 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:07:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.698+0000 7fbda3c1f740 20 client.15633 populate_metadata read hostname 'smithi016' 2024-08-31T06:07:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.698+0000 7fbda3c1f740 1 --2- 172.21.15.16:0/3812963109 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5620721a61c0 0x56207263fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.699+0000 7fbda3c1f740 4 client.15633 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:07:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.699+0000 7fbda3c1f740 1 -- 172.21.15.16:0/3812963109 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562072528960 con 0x5620721a61c0 2024-08-31T06:07:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.699+0000 7fbd95e00640 1 --2- 172.21.15.16:0/3812963109 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5620721a61c0 0x56207263fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.699+0000 7fbd95e00640 1 --2- 172.21.15.16:0/3812963109 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5620721a61c0 0x56207263fd10 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d495c05018a5d server_cookie=afbc3a541e4499bb in_seq=0 out_seq=0 2024-08-31T06:07:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.699+0000 7fbd7b400640 10 client.15633 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.736+0000 7fbd7b400640 1 -- 172.21.15.16:0/3812963109 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562072528960 con 0x5620721a61c0 2024-08-31T06:07:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.736+0000 7fbd7b400640 10 client.15633 handle_command_reply: tid=0 2024-08-31T06:07:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.737+0000 7fbda3c1f740 1 client.15633 shutdown 2024-08-31T06:07:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.737+0000 7fbda3c1f740 1 -- 172.21.15.16:0/3812963109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620721f14d0 msgr2=0x5620725c72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.737+0000 7fbda3c1f740 1 --2- 172.21.15.16:0/3812963109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620721f14d0 0x5620725c72f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbd80005ab0 tx=0x7fbd80031520 comp rx=0 tx=0).stop 2024-08-31T06:07:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.737+0000 7fbda3c1f740 1 -- 172.21.15.16:0/3812963109 shutdown_connections 2024-08-31T06:07:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.737+0000 7fbda3c1f740 1 --2- 172.21.15.16:0/3812963109 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5620721a61c0 0x56207263fd10 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.737+0000 7fbda3c1f740 1 --2- 172.21.15.16:0/3812963109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5620722121b0 0x562072634a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.737+0000 7fbda3c1f740 1 --2- 172.21.15.16:0/3812963109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620721f14d0 0x5620725c72f0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.737+0000 7fbda3c1f740 1 --2- 172.21.15.16:0/3812963109 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620721e92f0 0x5620725c6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.737+0000 7fbd7b400640 0 client.15633 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.737+0000 7fbda3c1f740 1 -- 172.21.15.16:0/3812963109 >> 172.21.15.16:0/3812963109 conn(0x562072206300 msgr2=0x5620725b68d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.737+0000 7fbda3c1f740 1 -- 172.21.15.16:0/3812963109 shutdown_connections 2024-08-31T06:07:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.737+0000 7fbda3c1f740 1 -- 172.21.15.16:0/3812963109 wait complete. 2024-08-31T06:07:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.737+0000 7fbda3c1f740 20 client.15633 trim_cache size 0 max 16384 2024-08-31T06:07:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:07:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:07:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:07:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:07:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.738+0000 7fbd97200640 1 -- 172.21.15.16:0/673101259 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbd7c0b4d20 msgr2=0x7fbd7c0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.738+0000 7fbd97200640 1 --2- 172.21.15.16:0/673101259 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbd7c0b4d20 0x7fbd7c0b7230 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbd8002a9e0 tx=0x7fbd80005a20 comp rx=0 tx=0).stop 2024-08-31T06:07:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.738+0000 7fbd97200640 1 -- 172.21.15.16:0/673101259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd9810d960 msgr2=0x7fbd981a1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.738+0000 7fbd97200640 1 --2- 172.21.15.16:0/673101259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd9810d960 0x7fbd981a1b00 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fbd9000eb10 tx=0x7fbd9000c2d0 comp rx=0 tx=0).stop 2024-08-31T06:07:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.738+0000 7fbd97200640 1 -- 172.21.15.16:0/673101259 shutdown_connections 2024-08-31T06:07:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.738+0000 7fbd97200640 1 --2- 172.21.15.16:0/673101259 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbd7c0b4d20 0x7fbd7c0b7230 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.738+0000 7fbd97200640 1 --2- 172.21.15.16:0/673101259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd9810d960 0x7fbd981a1b00 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.738+0000 7fbd97200640 1 --2- 172.21.15.16:0/673101259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd98104e50 0x7fbd9819c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.738+0000 7fbd97200640 1 --2- 172.21.15.16:0/673101259 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd98104500 0x7fbd9819c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.738+0000 7fbd97200640 1 -- 172.21.15.16:0/673101259 >> 172.21.15.16:0/673101259 conn(0x7fbd980fdfd0 msgr2=0x7fbd98102140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.738+0000 7fbd97200640 1 -- 172.21.15.16:0/673101259 shutdown_connections 2024-08-31T06:07:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:44.738+0000 7fbd97200640 1 -- 172.21.15.16:0/673101259 wait complete. 2024-08-31T06:07:45.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:45 smithi155 ceph-mon[21742]: pgmap v379: 97 pgs: 97 active+clean; 903 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 4.3 MiB/s wr, 485 op/s 2024-08-31T06:07:45.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:45 smithi016 ceph-mon[17194]: pgmap v379: 97 pgs: 97 active+clean; 903 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 4.3 MiB/s wr, 485 op/s 2024-08-31T06:07:45.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:45 smithi184 ceph-mon[22331]: pgmap v379: 97 pgs: 97 active+clean; 903 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 4.3 MiB/s wr, 485 op/s 2024-08-31T06:07:46.627 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 20.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-31T06:07:47.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:47 smithi155 ceph-mon[21742]: pgmap v380: 97 pgs: 97 active+clean; 903 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 2.6 MiB/s wr, 431 op/s 2024-08-31T06:07:47.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:47 smithi016 ceph-mon[17194]: pgmap v380: 97 pgs: 97 active+clean; 903 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 2.6 MiB/s wr, 431 op/s 2024-08-31T06:07:47.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:47 smithi184 ceph-mon[22331]: pgmap v380: 97 pgs: 97 active+clean; 903 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 2.6 MiB/s wr, 431 op/s 2024-08-31T06:07:49.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:49 smithi155 ceph-mon[21742]: pgmap v381: 97 pgs: 97 active+clean; 903 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.9 MiB/s wr, 340 op/s 2024-08-31T06:07:49.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:49 smithi016 ceph-mon[17194]: pgmap v381: 97 pgs: 97 active+clean; 903 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.9 MiB/s wr, 340 op/s 2024-08-31T06:07:49.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:49 smithi184 ceph-mon[22331]: pgmap v381: 97 pgs: 97 active+clean; 903 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.9 MiB/s wr, 340 op/s 2024-08-31T06:07:51.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:51 smithi155 ceph-mon[21742]: pgmap v382: 97 pgs: 97 active+clean; 904 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.4 MiB/s wr, 256 op/s 2024-08-31T06:07:51.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:51 smithi016 ceph-mon[17194]: pgmap v382: 97 pgs: 97 active+clean; 904 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.4 MiB/s wr, 256 op/s 2024-08-31T06:07:51.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:51 smithi184 ceph-mon[22331]: pgmap v382: 97 pgs: 97 active+clean; 904 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.4 MiB/s wr, 256 op/s 2024-08-31T06:07:51.628 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 25.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-31T06:07:52.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:52 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:07:52.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:52 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:07:52.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:52 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:07:52.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:52 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:07:52.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:52 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:07:52.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:52 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:07:52.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:52 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:07:52.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:52 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:07:52.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:52 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:07:53.333 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:53 smithi016 ceph-mon[17194]: pgmap v383: 97 pgs: 97 active+clean; 904 MiB data, 777 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 919 KiB/s wr, 155 op/s 2024-08-31T06:07:53.333 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:53 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:07:53.334 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:53 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:07:53.334 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:53 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:07:53.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:53 smithi155 ceph-mon[21742]: pgmap v383: 97 pgs: 97 active+clean; 904 MiB data, 777 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 919 KiB/s wr, 155 op/s 2024-08-31T06:07:53.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:53 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:07:53.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:53 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:07:53.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:53 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:07:53.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:53 smithi184 ceph-mon[22331]: pgmap v383: 97 pgs: 97 active+clean; 904 MiB data, 777 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 919 KiB/s wr, 155 op/s 2024-08-31T06:07:53.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:53 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:07:53.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:53 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:07:53.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:53 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:07:55.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:55 smithi155 ceph-mon[21742]: pgmap v384: 97 pgs: 97 active+clean; 904 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 1015 KiB/s wr, 90 op/s 2024-08-31T06:07:55.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:55 smithi016 ceph-mon[17194]: pgmap v384: 97 pgs: 97 active+clean; 904 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 1015 KiB/s wr, 90 op/s 2024-08-31T06:07:55.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:55 smithi184 ceph-mon[22331]: pgmap v384: 97 pgs: 97 active+clean; 904 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 1015 KiB/s wr, 90 op/s 2024-08-31T06:07:56.629 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 30.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-31T06:07:56.714 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:07:57.107 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:07:57.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:57 smithi155 ceph-mon[21742]: pgmap v385: 97 pgs: 97 active+clean; 921 MiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 3.3 MiB/s wr, 118 op/s 2024-08-31T06:07:57.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:57 smithi016 ceph-mon[17194]: pgmap v385: 97 pgs: 97 active+clean; 921 MiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 3.3 MiB/s wr, 118 op/s 2024-08-31T06:07:57.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:57 smithi184 ceph-mon[22331]: pgmap v385: 97 pgs: 97 active+clean; 921 MiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 3.3 MiB/s wr, 118 op/s 2024-08-31T06:07:57.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.727+0000 7f01cd600640 1 -- 172.21.15.16:0/3412702176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01c810fe60 msgr2=0x7f01c81122a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:57.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.727+0000 7f01cd600640 1 --2- 172.21.15.16:0/3412702176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01c810fe60 0x7f01c81122a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f01c0009ca0 tx=0x7f01c002a8c0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.727+0000 7f01cd600640 1 -- 172.21.15.16:0/3412702176 shutdown_connections 2024-08-31T06:07:57.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.727+0000 7f01cd600640 1 --2- 172.21.15.16:0/3412702176 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01c8075ea0 0x7f01c810f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.727+0000 7f01cd600640 1 --2- 172.21.15.16:0/3412702176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01c810fe60 0x7f01c81122a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.727+0000 7f01cd600640 1 --2- 172.21.15.16:0/3412702176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01c8075550 0x7f01c8075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.727+0000 7f01cd600640 1 -- 172.21.15.16:0/3412702176 >> 172.21.15.16:0/3412702176 conn(0x7f01c807b5f0 msgr2=0x7f01c807ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:57.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.727+0000 7f01cd600640 1 -- 172.21.15.16:0/3412702176 shutdown_connections 2024-08-31T06:07:57.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.727+0000 7f01cd600640 1 -- 172.21.15.16:0/3412702176 wait complete. 2024-08-31T06:07:57.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.728+0000 7f01cd600640 1 Processor -- start 2024-08-31T06:07:57.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.728+0000 7f01cd600640 1 -- start start 2024-08-31T06:07:57.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.729+0000 7f01cd600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01c8075550 0x7f01c819c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:57.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.729+0000 7f01cd600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01c8075ea0 0x7f01c819ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:57.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.729+0000 7f01cd600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01c810fe60 0x7f01c81a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:57.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.729+0000 7f01cd600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f01c8115a60 con 0x7f01c810fe60 2024-08-31T06:07:57.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.729+0000 7f01cd600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f01c81158e0 con 0x7f01c8075ea0 2024-08-31T06:07:57.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.729+0000 7f01cd600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f01c8115be0 con 0x7f01c8075550 2024-08-31T06:07:57.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.729+0000 7f01c7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01c810fe60 0x7f01c81a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:57.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.729+0000 7f01c6a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01c8075550 0x7f01c819c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:57.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.729+0000 7f01c6a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01c8075550 0x7f01c819c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:40014/0 (socket says 172.21.15.16:40014) 2024-08-31T06:07:57.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.729+0000 7f01c6a00640 1 -- 172.21.15.16:0/1543050883 learned_addr learned my addr 172.21.15.16:0/1543050883 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:57.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.729+0000 7f01c6000640 1 --2- 172.21.15.16:0/1543050883 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01c8075ea0 0x7f01c819ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:57.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.730+0000 7f01c6a00640 1 -- 172.21.15.16:0/1543050883 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01c8075ea0 msgr2=0x7f01c819ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:57.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.730+0000 7f01c6a00640 1 --2- 172.21.15.16:0/1543050883 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01c8075ea0 0x7f01c819ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.730+0000 7f01c6a00640 1 -- 172.21.15.16:0/1543050883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01c810fe60 msgr2=0x7f01c81a1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:57.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.730+0000 7f01c6a00640 1 --2- 172.21.15.16:0/1543050883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01c810fe60 0x7f01c81a1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.730+0000 7f01c6a00640 1 -- 172.21.15.16:0/1543050883 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c81a23e0 con 0x7f01c8075550 2024-08-31T06:07:57.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.730+0000 7f01c6a00640 1 --2- 172.21.15.16:0/1543050883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01c8075550 0x7f01c819c470 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f01b800d950 tx=0x7f01b800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:57.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.730+0000 7f01bf400640 1 -- 172.21.15.16:0/1543050883 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b8017030 con 0x7f01c8075550 2024-08-31T06:07:57.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.731+0000 7f01bf400640 1 -- 172.21.15.16:0/1543050883 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01b8004020 con 0x7f01c8075550 2024-08-31T06:07:57.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.731+0000 7f01cd600640 1 -- 172.21.15.16:0/1543050883 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01c81a26d0 con 0x7f01c8075550 2024-08-31T06:07:57.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.731+0000 7f01bf400640 1 -- 172.21.15.16:0/1543050883 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b80047a0 con 0x7f01c8075550 2024-08-31T06:07:57.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.731+0000 7f01cd600640 1 -- 172.21.15.16:0/1543050883 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01c81a8e30 con 0x7f01c8075550 2024-08-31T06:07:57.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.732+0000 7f01cd600640 1 -- 172.21.15.16:0/1543050883 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01c8075960 con 0x7f01c8075550 2024-08-31T06:07:57.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.735+0000 7f01bf400640 1 -- 172.21.15.16:0/1543050883 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f01b8009a30 con 0x7f01c8075550 2024-08-31T06:07:57.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.736+0000 7f01bf400640 1 --2- 172.21.15.16:0/1543050883 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f01ac0b4d20 0x7f01ac0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:57.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.736+0000 7f01bf400640 1 -- 172.21.15.16:0/1543050883 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f01b80e54d0 con 0x7f01c8075550 2024-08-31T06:07:57.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.736+0000 7f01c6000640 1 --2- 172.21.15.16:0/1543050883 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f01ac0b4d20 0x7f01ac0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:57.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.737+0000 7f01c6000640 1 --2- 172.21.15.16:0/1543050883 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f01ac0b4d20 0x7f01ac0b7230 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f01c8111b20 tx=0x7f01b4005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:57.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.738+0000 7f01bf400640 1 -- 172.21.15.16:0/1543050883 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f01b80ac340 con 0x7f01c8075550 2024-08-31T06:07:57.933 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.930+0000 7f5a15e00640 1 -- 172.21.15.16:0/1366229750 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a10105020 msgr2=0x7f5a1010d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:57.933 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.930+0000 7f5a15e00640 1 --2- 172.21.15.16:0/1366229750 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a10105020 0x7f5a1010d560 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f59f8009900 tx=0x7f59f802a3f0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.933 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.930+0000 7f5a15e00640 1 -- 172.21.15.16:0/1366229750 shutdown_connections 2024-08-31T06:07:57.933 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.930+0000 7f5a15e00640 1 --2- 172.21.15.16:0/1366229750 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a10105020 0x7f5a1010d560 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.933 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.930+0000 7f5a15e00640 1 --2- 172.21.15.16:0/1366229750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a1010db30 0x7f5a10110020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.933 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.930+0000 7f5a15e00640 1 --2- 172.21.15.16:0/1366229750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a101046d0 0x7f5a10104ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.933 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.930+0000 7f5a15e00640 1 -- 172.21.15.16:0/1366229750 >> 172.21.15.16:0/1366229750 conn(0x7f5a100fe160 msgr2=0x7f5a101005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:57.933 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.930+0000 7f5a15e00640 1 -- 172.21.15.16:0/1366229750 shutdown_connections 2024-08-31T06:07:57.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.930+0000 7f5a15e00640 1 -- 172.21.15.16:0/1366229750 wait complete. 2024-08-31T06:07:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.931+0000 7f5a15e00640 1 Processor -- start 2024-08-31T06:07:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.931+0000 7f5a15e00640 1 -- start start 2024-08-31T06:07:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.932+0000 7f5a15e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a101046d0 0x7f5a10102400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.932+0000 7f5a15e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a10105020 0x7f5a10102940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.932+0000 7f5a15e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a1010db30 0x7f5a10102e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.932+0000 7f5a15e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5a101137b0 con 0x7f5a1010db30 2024-08-31T06:07:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.932+0000 7f5a15e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5a10113630 con 0x7f5a101046d0 2024-08-31T06:07:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.932+0000 7f5a15e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5a10113930 con 0x7f5a10105020 2024-08-31T06:07:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.932+0000 7f5a0f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a1010db30 0x7f5a10102e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.932+0000 7f5a0f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a1010db30 0x7f5a10102e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55422/0 (socket says 172.21.15.16:55422) 2024-08-31T06:07:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.932+0000 7f5a0f400640 1 -- 172.21.15.16:0/2684668590 learned_addr learned my addr 172.21.15.16:0/2684668590 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.932+0000 7f5a0ea00640 1 --2- 172.21.15.16:0/2684668590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a101046d0 0x7f5a10102400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.932+0000 7f5a07e00640 1 --2- 172.21.15.16:0/2684668590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a10105020 0x7f5a10102940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.933+0000 7f5a0f400640 1 -- 172.21.15.16:0/2684668590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a10105020 msgr2=0x7f5a10102940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.933+0000 7f5a0f400640 1 --2- 172.21.15.16:0/2684668590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a10105020 0x7f5a10102940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.933+0000 7f5a0f400640 1 -- 172.21.15.16:0/2684668590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a101046d0 msgr2=0x7f5a10102400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.933+0000 7f5a0f400640 1 --2- 172.21.15.16:0/2684668590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a101046d0 0x7f5a10102400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.933+0000 7f5a0f400640 1 -- 172.21.15.16:0/2684668590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a101a8500 con 0x7f5a1010db30 2024-08-31T06:07:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.933+0000 7f5a0ea00640 1 --2- 172.21.15.16:0/2684668590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a101046d0 0x7f5a10102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:07:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.933+0000 7f5a0f400640 1 --2- 172.21.15.16:0/2684668590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a1010db30 0x7f5a10102e80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5a08009fd0 tx=0x7f5a0800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.933+0000 7f5a14a00640 1 -- 172.21.15.16:0/2684668590 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a08017070 con 0x7f5a1010db30 2024-08-31T06:07:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.933+0000 7f5a15e00640 1 -- 172.21.15.16:0/2684668590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a101a87f0 con 0x7f5a1010db30 2024-08-31T06:07:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.933+0000 7f5a14a00640 1 -- 172.21.15.16:0/2684668590 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a08003e20 con 0x7f5a1010db30 2024-08-31T06:07:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.934+0000 7f5a14a00640 1 -- 172.21.15.16:0/2684668590 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a08010460 con 0x7f5a1010db30 2024-08-31T06:07:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.934+0000 7f5a15e00640 1 -- 172.21.15.16:0/2684668590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a101a8d10 con 0x7f5a1010db30 2024-08-31T06:07:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.936+0000 7f5a1ce6e740 1 Processor -- start 2024-08-31T06:07:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.939+0000 7f5a14a00640 1 -- 172.21.15.16:0/2684668590 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f5a08003970 con 0x7f5a1010db30 2024-08-31T06:07:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.939+0000 7f5a1ce6e740 1 -- start start 2024-08-31T06:07:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.940+0000 7f5a1ce6e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617ad9751b0 0x5617add18920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.940+0000 7f5a1ce6e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617ad94c2f0 0x5617add18e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.940+0000 7f5a1ce6e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617ad9544d0 0x5617add193a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.940+0000 7f5a1ce6e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5617adc8df70 con 0x5617ad94c2f0 2024-08-31T06:07:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.940+0000 7f5a1ce6e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5617adc7d500 con 0x5617ad9751b0 2024-08-31T06:07:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.940+0000 7f5a1ce6e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5617adc814b0 con 0x5617ad9544d0 2024-08-31T06:07:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.940+0000 7f5a0ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617ad94c2f0 0x5617add18e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.940+0000 7f5a0ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617ad94c2f0 0x5617add18e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55428/0 (socket says 172.21.15.16:55428) 2024-08-31T06:07:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.940+0000 7f5a0ea00640 1 -- 172.21.15.16:0/1331908343 learned_addr learned my addr 172.21.15.16:0/1331908343 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.940+0000 7f5a07e00640 1 --2- 172.21.15.16:0/1331908343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617ad9751b0 0x5617add18920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.940+0000 7f5a07e00640 1 --2- 172.21.15.16:0/1331908343 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617ad9544d0 0x5617add193a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.940+0000 7f5a0ea00640 1 -- 172.21.15.16:0/1331908343 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617ad9544d0 msgr2=0x5617add193a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.940+0000 7f5a0ea00640 1 --2- 172.21.15.16:0/1331908343 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617ad9544d0 0x5617add193a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.940+0000 7f5a0ea00640 1 -- 172.21.15.16:0/1331908343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617ad9751b0 msgr2=0x5617add18920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.940+0000 7f5a0ea00640 1 --2- 172.21.15.16:0/1331908343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617ad9751b0 0x5617add18920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.941+0000 7f5a0ea00640 1 -- 172.21.15.16:0/1331908343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617adc37c10 con 0x5617ad94c2f0 2024-08-31T06:07:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.941+0000 7f5a0ea00640 1 --2- 172.21.15.16:0/1331908343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617ad94c2f0 0x5617add18e60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5a000114f0 tx=0x7f5a000119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90d1cabbb764ecca server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.941+0000 7f5a14a00640 1 --2- 172.21.15.16:0/2684668590 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f59f40b4d50 0x7f59f40b7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.941+0000 7f59ffe00640 1 -- 172.21.15.16:0/1331908343 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a00011250 con 0x5617ad94c2f0 2024-08-31T06:07:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.941+0000 7f59ffe00640 1 -- 172.21.15.16:0/1331908343 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a000054d0 con 0x5617ad94c2f0 2024-08-31T06:07:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.941+0000 7f5a14a00640 1 -- 172.21.15.16:0/2684668590 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f5a080e50f0 con 0x7f5a1010db30 2024-08-31T06:07:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.941+0000 7f59ffe00640 1 -- 172.21.15.16:0/1331908343 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0000f820 con 0x5617ad94c2f0 2024-08-31T06:07:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.941+0000 7f5a07e00640 1 --2- 172.21.15.16:0/2684668590 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f59f40b4d50 0x7f59f40b7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.942+0000 7f5a1ce6e740 1 -- 172.21.15.16:0/1331908343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617ad94c2f0 msgr2=0x5617add18e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.942+0000 7f5a1ce6e740 1 --2- 172.21.15.16:0/1331908343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617ad94c2f0 0x5617add18e60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5a000114f0 tx=0x7f5a000119b0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.942+0000 7f5a07e00640 1 --2- 172.21.15.16:0/2684668590 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f59f40b4d50 0x7f59f40b7260 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f59f802a9e0 tx=0x7f59f8041040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.942+0000 7f5a1ce6e740 1 -- 172.21.15.16:0/1331908343 shutdown_connections 2024-08-31T06:07:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.942+0000 7f5a1ce6e740 1 --2- 172.21.15.16:0/1331908343 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617ad9544d0 0x5617add193a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.942+0000 7f5a1ce6e740 1 --2- 172.21.15.16:0/1331908343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617ad94c2f0 0x5617add18e60 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.942+0000 7f5a1ce6e740 1 --2- 172.21.15.16:0/1331908343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617ad9751b0 0x5617add18920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.942+0000 7f5a1ce6e740 1 -- 172.21.15.16:0/1331908343 >> 172.21.15.16:0/1331908343 conn(0x5617ad969300 msgr2=0x5617add16810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.943+0000 7f5a1ce6e740 1 -- 172.21.15.16:0/1331908343 shutdown_connections 2024-08-31T06:07:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.943+0000 7f5a1ce6e740 1 -- 172.21.15.16:0/1331908343 wait complete. 2024-08-31T06:07:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.943+0000 7f5a1ce6e740 1 -- start start 2024-08-31T06:07:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.943+0000 7f5a1ce6e740 1 Processor -- start 2024-08-31T06:07:57.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.944+0000 7f5a1ce6e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617ad94c2f0 0x5617add23e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:57.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.944+0000 7f5a1ce6e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617ad9544d0 0x5617add243a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:57.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.944+0000 7f5a1ce6e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617ad9751b0 0x5617add248e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:57.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.944+0000 7f5a1ce6e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5617adbb6ab0 con 0x5617ad94c2f0 2024-08-31T06:07:57.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.944+0000 7f5a1ce6e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5617adc24320 con 0x5617ad9544d0 2024-08-31T06:07:57.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.944+0000 7f5a1ce6e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5617add270a0 con 0x5617ad9751b0 2024-08-31T06:07:57.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.944+0000 7f5a1ce6e740 10 client.0 _ll_register_callbacks cb 0x5617adc2a6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:07:57.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.944+0000 7f5a0ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617ad94c2f0 0x5617add23e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:57.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.944+0000 7f5a0ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617ad94c2f0 0x5617add23e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55438/0 (socket says 172.21.15.16:55438) 2024-08-31T06:07:57.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.944+0000 7f5a0ea00640 1 -- 172.21.15.16:0/2342291788 learned_addr learned my addr 172.21.15.16:0/2342291788 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:57.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.944+0000 7f5a07e00640 1 --2- 172.21.15.16:0/2342291788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617ad9544d0 0x5617add243a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:57.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.944+0000 7f5a0f400640 1 --2- 172.21.15.16:0/2342291788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617ad9751b0 0x5617add248e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:57.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.945+0000 7f5a0ea00640 1 -- 172.21.15.16:0/2342291788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617ad9751b0 msgr2=0x5617add248e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:57.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.945+0000 7f5a0ea00640 1 --2- 172.21.15.16:0/2342291788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617ad9751b0 0x5617add248e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.945+0000 7f5a0ea00640 1 -- 172.21.15.16:0/2342291788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617ad9544d0 msgr2=0x5617add243a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.945+0000 7f5a0ea00640 1 --2- 172.21.15.16:0/2342291788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617ad9544d0 0x5617add243a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.945+0000 7f5a0ea00640 1 -- 172.21.15.16:0/2342291788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617ad9e0790 con 0x5617ad94c2f0 2024-08-31T06:07:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.945+0000 7f5a0ea00640 1 --2- 172.21.15.16:0/2342291788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617ad94c2f0 0x5617add23e60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5a00004be0 tx=0x7f5a00011300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ebcb0ea5115813e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.945+0000 7f59fcc00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:07:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.945+0000 7f59fcc00640 1 -- 172.21.15.16:0/2342291788 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a00028070 con 0x5617ad94c2f0 2024-08-31T06:07:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.945+0000 7f59fcc00640 1 -- 172.21.15.16:0/2342291788 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f5a00004250 con 0x5617ad94c2f0 2024-08-31T06:07:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.945+0000 7f5a1ce6e740 1 -- 172.21.15.16:0/2342291788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a000054d0 con 0x5617ad94c2f0 2024-08-31T06:07:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.945+0000 7f59fcc00640 1 -- 172.21.15.16:0/2342291788 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5a00034050 con 0x5617ad94c2f0 2024-08-31T06:07:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.945+0000 7f5a1ce6e740 10 client.15651 fetch_fsmap learned FSMap version 30 2024-08-31T06:07:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.946+0000 7f5a1ce6e740 1 -- 172.21.15.16:0/2342291788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5617ada59620 con 0x5617ad94c2f0 2024-08-31T06:07:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.946+0000 7f59fcc00640 1 -- 172.21.15.16:0/2342291788 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a000054d0 con 0x5617ad94c2f0 2024-08-31T06:07:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.946+0000 7f59fcc00640 1 -- 172.21.15.16:0/2342291788 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f5a000037f0 con 0x5617ad94c2f0 2024-08-31T06:07:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.946+0000 7f5a1ce6e740 10 client.15651 fetch_fsmap finished waiting for FSMap version 30 2024-08-31T06:07:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.946+0000 7f5a1ce6e740 10 client.15651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:07:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.946+0000 7f5a1ce6e740 20 client.15651 populate_metadata read hostname 'smithi016' 2024-08-31T06:07:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.946+0000 7f5a1ce6e740 1 --2- 172.21.15.16:0/2342291788 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5617ad933d80 0x5617add35f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.946+0000 7f5a1ce6e740 4 client.15651 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:07:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.946+0000 7f5a1ce6e740 1 -- 172.21.15.16:0/2342291788 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617adc8fec0 con 0x5617ad933d80 2024-08-31T06:07:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.947+0000 7f5a07e00640 1 --2- 172.21.15.16:0/2342291788 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5617ad933d80 0x5617add35f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.947+0000 7f59fcc00640 10 client.15651 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.947+0000 7f5a07e00640 1 --2- 172.21.15.16:0/2342291788 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5617ad933d80 0x5617add35f00 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baeffec9d8c3d155 server_cookie=e771ecbe33176876 in_seq=0 out_seq=0 2024-08-31T06:07:57.953 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.948+0000 7f59fcc00640 1 -- 172.21.15.16:0/2342291788 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5617adc8fec0 con 0x5617ad933d80 2024-08-31T06:07:57.953 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.948+0000 7f59fcc00640 10 client.15651 handle_command_reply: tid=0 2024-08-31T06:07:57.953 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.948+0000 7f5a1ce6e740 1 client.15651 shutdown 2024-08-31T06:07:57.953 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.949+0000 7f5a1ce6e740 1 -- 172.21.15.16:0/2342291788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617ad94c2f0 msgr2=0x5617add23e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.949+0000 7f5a1ce6e740 1 --2- 172.21.15.16:0/2342291788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617ad94c2f0 0x5617add23e60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5a00004be0 tx=0x7f5a00011300 comp rx=0 tx=0).stop 2024-08-31T06:07:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.949+0000 7f5a1ce6e740 1 -- 172.21.15.16:0/2342291788 shutdown_connections 2024-08-31T06:07:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.949+0000 7f5a1ce6e740 1 --2- 172.21.15.16:0/2342291788 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5617ad933d80 0x5617add35f00 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.949+0000 7f5a1ce6e740 1 --2- 172.21.15.16:0/2342291788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617ad9751b0 0x5617add248e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.949+0000 7f59fcc00640 0 client.15651 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.949+0000 7f5a1ce6e740 1 --2- 172.21.15.16:0/2342291788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617ad9544d0 0x5617add243a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.949+0000 7f5a1ce6e740 1 --2- 172.21.15.16:0/2342291788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617ad94c2f0 0x5617add23e60 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.949+0000 7f5a1ce6e740 1 -- 172.21.15.16:0/2342291788 >> 172.21.15.16:0/2342291788 conn(0x5617ad969300 msgr2=0x5617add19950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.949+0000 7f5a1ce6e740 1 -- 172.21.15.16:0/2342291788 shutdown_connections 2024-08-31T06:07:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.949+0000 7f5a1ce6e740 1 -- 172.21.15.16:0/2342291788 wait complete. 2024-08-31T06:07:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.949+0000 7f5a1ce6e740 20 client.15651 trim_cache size 0 max 16384 2024-08-31T06:07:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.964+0000 7f5a1ce6e740 1 Processor -- start 2024-08-31T06:07:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.964+0000 7f5a1ce6e740 1 -- start start 2024-08-31T06:07:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.965+0000 7f5a1ce6e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617ad94c2f0 0x5617add169a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.965+0000 7f5a1ce6e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617ad9544d0 0x5617add2f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.965+0000 7f5a1ce6e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617ad9751b0 0x5617add21030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.965+0000 7f5a1ce6e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5a101b3f90 con 0x5617ad9751b0 2024-08-31T06:07:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.965+0000 7f5a1ce6e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5a101b17d0 con 0x5617ad9544d0 2024-08-31T06:07:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.965+0000 7f5a1ce6e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5a101af620 con 0x5617ad94c2f0 2024-08-31T06:07:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.965+0000 7f5a0f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617ad9751b0 0x5617add21030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.965+0000 7f5a07e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617ad9544d0 0x5617add2f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.965+0000 7f5a0f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617ad9751b0 0x5617add21030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55448/0 (socket says 172.21.15.16:55448) 2024-08-31T06:07:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.965+0000 7f5a0f400640 1 -- 172.21.15.16:0/3988485322 learned_addr learned my addr 172.21.15.16:0/3988485322 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.965+0000 7f5a0ea00640 1 --2- 172.21.15.16:0/3988485322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617ad94c2f0 0x5617add169a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.965+0000 7f5a0f400640 1 -- 172.21.15.16:0/3988485322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617ad94c2f0 msgr2=0x5617add169a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.965+0000 7f5a0f400640 1 --2- 172.21.15.16:0/3988485322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617ad94c2f0 0x5617add169a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.965+0000 7f5a0f400640 1 -- 172.21.15.16:0/3988485322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617ad9544d0 msgr2=0x5617add2f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.965+0000 7f5a0f400640 1 --2- 172.21.15.16:0/3988485322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617ad9544d0 0x5617add2f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.965+0000 7f5a0f400640 1 -- 172.21.15.16:0/3988485322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617add10990 con 0x5617ad9751b0 2024-08-31T06:07:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.965+0000 7f5a07e00640 1 --2- 172.21.15.16:0/3988485322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617ad9544d0 0x5617add2f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:07:57.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.966+0000 7f5a0f400640 1 --2- 172.21.15.16:0/3988485322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617ad9751b0 0x5617add21030 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5a080e28a0 tx=0x7f5a080ab0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7243bedbb2413b66 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:57.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.966+0000 7f59ff400640 1 -- 172.21.15.16:0/3988485322 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a08020040 con 0x5617ad9751b0 2024-08-31T06:07:57.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.966+0000 7f59ff400640 1 -- 172.21.15.16:0/3988485322 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a080ab3a0 con 0x5617ad9751b0 2024-08-31T06:07:57.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.966+0000 7f59ff400640 1 -- 172.21.15.16:0/3988485322 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a080acb10 con 0x5617ad9751b0 2024-08-31T06:07:57.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.967+0000 7f5a1ce6e740 1 -- 172.21.15.16:0/3988485322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617ad9751b0 msgr2=0x5617add21030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:57.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.967+0000 7f5a1ce6e740 1 --2- 172.21.15.16:0/3988485322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617ad9751b0 0x5617add21030 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5a080e28a0 tx=0x7f5a080ab0c0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.967+0000 7f5a1ce6e740 1 -- 172.21.15.16:0/3988485322 shutdown_connections 2024-08-31T06:07:57.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.967+0000 7f5a1ce6e740 1 --2- 172.21.15.16:0/3988485322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617ad9751b0 0x5617add21030 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.967+0000 7f5a1ce6e740 1 --2- 172.21.15.16:0/3988485322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617ad9544d0 0x5617add2f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.967+0000 7f5a1ce6e740 1 --2- 172.21.15.16:0/3988485322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617ad94c2f0 0x5617add169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.967+0000 7f5a1ce6e740 1 -- 172.21.15.16:0/3988485322 >> 172.21.15.16:0/3988485322 conn(0x5617ad969300 msgr2=0x5617add2f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:57.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.967+0000 7f5a1ce6e740 1 -- 172.21.15.16:0/3988485322 shutdown_connections 2024-08-31T06:07:57.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.967+0000 7f5a1ce6e740 1 -- 172.21.15.16:0/3988485322 wait complete. 2024-08-31T06:07:57.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.967+0000 7f5a1ce6e740 1 -- start start 2024-08-31T06:07:57.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.968+0000 7f5a1ce6e740 1 Processor -- start 2024-08-31T06:07:57.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.968+0000 7f5a1ce6e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617ad94c2f0 0x5617add2c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:57.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.968+0000 7f5a1ce6e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617ad9544d0 0x5617add29d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:57.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.968+0000 7f5a1ce6e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617ad9751b0 0x5617add2a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:57.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.968+0000 7f5a1ce6e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5a101afc90 con 0x5617ad9544d0 2024-08-31T06:07:57.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.968+0000 7f5a1ce6e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5a101b1100 con 0x5617ad94c2f0 2024-08-31T06:07:57.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.968+0000 7f5a1ce6e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5a101b0360 con 0x5617ad9751b0 2024-08-31T06:07:57.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.968+0000 7f5a1ce6e740 10 client.0 _ll_register_callbacks cb 0x7f5a00005d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:07:57.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.968+0000 7f5a07e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617ad9544d0 0x5617add29d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:57.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.968+0000 7f5a07e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617ad9544d0 0x5617add29d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55458/0 (socket says 172.21.15.16:55458) 2024-08-31T06:07:57.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.968+0000 7f5a07e00640 1 -- 172.21.15.16:0/3244275688 learned_addr learned my addr 172.21.15.16:0/3244275688 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:57.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.968+0000 7f5a0ea00640 1 --2- 172.21.15.16:0/3244275688 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617ad94c2f0 0x5617add2c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:57.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.969+0000 7f5a0f400640 1 --2- 172.21.15.16:0/3244275688 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617ad9751b0 0x5617add2a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.969+0000 7f5a07e00640 1 -- 172.21.15.16:0/3244275688 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617ad9751b0 msgr2=0x5617add2a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.969+0000 7f5a07e00640 1 --2- 172.21.15.16:0/3244275688 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617ad9751b0 0x5617add2a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.969+0000 7f5a07e00640 1 -- 172.21.15.16:0/3244275688 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617ad94c2f0 msgr2=0x5617add2c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.969+0000 7f5a07e00640 1 --2- 172.21.15.16:0/3244275688 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617ad94c2f0 0x5617add2c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.969+0000 7f5a07e00640 1 -- 172.21.15.16:0/3244275688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617add1ae50 con 0x5617ad9544d0 2024-08-31T06:07:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.969+0000 7f5a0f400640 1 --2- 172.21.15.16:0/3244275688 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617ad9751b0 0x5617add2a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:07:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.969+0000 7f5a07e00640 1 --2- 172.21.15.16:0/3244275688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617ad9544d0 0x5617add29d70 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f59f8005ab0 tx=0x7f59f802a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1de9c099436240 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.969+0000 7f59fcc00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:07:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.969+0000 7f59fcc00640 1 -- 172.21.15.16:0/3244275688 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f8046070 con 0x5617ad9544d0 2024-08-31T06:07:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.969+0000 7f59fcc00640 1 -- 172.21.15.16:0/3244275688 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f59f80318b0 con 0x5617ad9544d0 2024-08-31T06:07:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.969+0000 7f5a1ce6e740 1 -- 172.21.15.16:0/3244275688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a080ab3a0 con 0x5617ad9544d0 2024-08-31T06:07:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.970+0000 7f59fcc00640 1 -- 172.21.15.16:0/3244275688 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f59f8031ab0 con 0x5617ad9544d0 2024-08-31T06:07:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.970+0000 7f5a1ce6e740 10 client.15657 fetch_fsmap learned FSMap version 30 2024-08-31T06:07:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.970+0000 7f5a1ce6e740 1 -- 172.21.15.16:0/3244275688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5617adda0360 con 0x5617ad9544d0 2024-08-31T06:07:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.970+0000 7f59fcc00640 1 -- 172.21.15.16:0/3244275688 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a080ab3a0 con 0x5617ad9544d0 2024-08-31T06:07:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.970+0000 7f59fcc00640 1 -- 172.21.15.16:0/3244275688 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f59f8057070 con 0x5617ad9544d0 2024-08-31T06:07:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.970+0000 7f5a1ce6e740 10 client.15657 fetch_fsmap finished waiting for FSMap version 30 2024-08-31T06:07:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.970+0000 7f5a1ce6e740 10 client.15657 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:07:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.970+0000 7f5a1ce6e740 20 client.15657 populate_metadata read hostname 'smithi016' 2024-08-31T06:07:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.970+0000 7f5a1ce6e740 1 --2- 172.21.15.16:0/3244275688 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5617ad9091c0 0x5617adda2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.970+0000 7f5a1ce6e740 4 client.15657 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:07:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.970+0000 7f5a1ce6e740 1 -- 172.21.15.16:0/3244275688 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5617adc8b960 con 0x5617ad9091c0 2024-08-31T06:07:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.971+0000 7f5a0ea00640 1 --2- 172.21.15.16:0/3244275688 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5617ad9091c0 0x5617adda2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:57.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.971+0000 7f5a0ea00640 1 --2- 172.21.15.16:0/3244275688 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5617ad9091c0 0x5617adda2a50 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8247cb9f65f58a23 server_cookie=f2d3e6f52073ed09 in_seq=0 out_seq=0 2024-08-31T06:07:57.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.971+0000 7f59fcc00640 10 client.15657 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:57.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.983+0000 7f01cd600640 1 -- 172.21.15.16:0/1543050883 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01c819d5a0 con 0x7f01c8075550 2024-08-31T06:07:57.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.984+0000 7f01bf400640 1 -- 172.21.15.16:0/1543050883 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11921 (secure 0 0 0) 0x7f01b80abc40 con 0x7f01c8075550 2024-08-31T06:07:57.987 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:07:57.987 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":30,"btime":"2024-08-31T06:07:34:013454+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:07:33.329391+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24712,24724,24737]},"id":1}]} 2024-08-31T06:07:57.987 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 30 2024-08-31T06:07:57.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.986+0000 7f01cd600640 1 -- 172.21.15.16:0/1543050883 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f01ac0b4d20 msgr2=0x7f01ac0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:57.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.986+0000 7f01cd600640 1 --2- 172.21.15.16:0/1543050883 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f01ac0b4d20 0x7f01ac0b7230 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f01c8111b20 tx=0x7f01b4005f90 comp rx=0 tx=0).stop 2024-08-31T06:07:57.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.986+0000 7f01cd600640 1 -- 172.21.15.16:0/1543050883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01c8075550 msgr2=0x7f01c819c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:57.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.986+0000 7f01cd600640 1 --2- 172.21.15.16:0/1543050883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01c8075550 0x7f01c819c470 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f01b800d950 tx=0x7f01b800de10 comp rx=0 tx=0).stop 2024-08-31T06:07:57.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.986+0000 7f01cd600640 1 -- 172.21.15.16:0/1543050883 shutdown_connections 2024-08-31T06:07:57.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.986+0000 7f01cd600640 1 --2- 172.21.15.16:0/1543050883 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f01ac0b4d20 0x7f01ac0b7230 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.987+0000 7f01cd600640 1 --2- 172.21.15.16:0/1543050883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01c810fe60 0x7f01c81a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.987+0000 7f01cd600640 1 --2- 172.21.15.16:0/1543050883 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01c8075ea0 0x7f01c819ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.987+0000 7f01cd600640 1 --2- 172.21.15.16:0/1543050883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01c8075550 0x7f01c819c470 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.987+0000 7f01cd600640 1 -- 172.21.15.16:0/1543050883 >> 172.21.15.16:0/1543050883 conn(0x7f01c807b5f0 msgr2=0x7f01c8108320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:57.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.987+0000 7f01cd600640 1 -- 172.21.15.16:0/1543050883 shutdown_connections 2024-08-31T06:07:57.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:57.987+0000 7f01cd600640 1 -- 172.21.15.16:0/1543050883 wait complete. 2024-08-31T06:07:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.990+0000 7f59fcc00640 1 -- 172.21.15.16:0/3244275688 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5617adc8b960 con 0x5617ad9091c0 2024-08-31T06:07:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.990+0000 7f59fcc00640 10 client.15657 handle_command_reply: tid=0 2024-08-31T06:07:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.990+0000 7f5a1ce6e740 1 client.15657 shutdown 2024-08-31T06:07:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.990+0000 7f5a1ce6e740 1 -- 172.21.15.16:0/3244275688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617ad9544d0 msgr2=0x5617add29d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.990+0000 7f5a1ce6e740 1 --2- 172.21.15.16:0/3244275688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617ad9544d0 0x5617add29d70 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f59f8005ab0 tx=0x7f59f802a3f0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.990+0000 7f5a1ce6e740 1 -- 172.21.15.16:0/3244275688 shutdown_connections 2024-08-31T06:07:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.990+0000 7f5a1ce6e740 1 --2- 172.21.15.16:0/3244275688 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5617ad9091c0 0x5617adda2a50 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.990+0000 7f5a1ce6e740 1 --2- 172.21.15.16:0/3244275688 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617ad9751b0 0x5617add2a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.990+0000 7f5a1ce6e740 1 --2- 172.21.15.16:0/3244275688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617ad9544d0 0x5617add29d70 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.990+0000 7f5a1ce6e740 1 --2- 172.21.15.16:0/3244275688 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617ad94c2f0 0x5617add2c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.990+0000 7f5a1ce6e740 1 -- 172.21.15.16:0/3244275688 >> 172.21.15.16:0/3244275688 conn(0x5617ad969300 msgr2=0x5617add166a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.990+0000 7f59fcc00640 0 client.15657 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.991+0000 7f5a1ce6e740 1 -- 172.21.15.16:0/3244275688 shutdown_connections 2024-08-31T06:07:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.991+0000 7f5a1ce6e740 1 -- 172.21.15.16:0/3244275688 wait complete. 2024-08-31T06:07:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.991+0000 7f5a1ce6e740 20 client.15657 trim_cache size 0 max 16384 2024-08-31T06:07:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:07:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:07:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:07:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:07:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.991+0000 7f5a15e00640 1 -- 172.21.15.16:0/2684668590 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f59f40b4d50 msgr2=0x7f59f40b7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.991+0000 7f5a15e00640 1 --2- 172.21.15.16:0/2684668590 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f59f40b4d50 0x7f59f40b7260 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f59f802a9e0 tx=0x7f59f8041040 comp rx=0 tx=0).stop 2024-08-31T06:07:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.991+0000 7f5a15e00640 1 -- 172.21.15.16:0/2684668590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a1010db30 msgr2=0x7f5a10102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.991+0000 7f5a15e00640 1 --2- 172.21.15.16:0/2684668590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a1010db30 0x7f5a10102e80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5a08009fd0 tx=0x7f5a0800b260 comp rx=0 tx=0).stop 2024-08-31T06:07:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.991+0000 7f5a15e00640 1 -- 172.21.15.16:0/2684668590 shutdown_connections 2024-08-31T06:07:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.991+0000 7f5a15e00640 1 --2- 172.21.15.16:0/2684668590 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f59f40b4d50 0x7f59f40b7260 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.991+0000 7f5a15e00640 1 --2- 172.21.15.16:0/2684668590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a10105020 0x7f5a10102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.991+0000 7f5a15e00640 1 --2- 172.21.15.16:0/2684668590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a1010db30 0x7f5a10102e80 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.991+0000 7f5a15e00640 1 --2- 172.21.15.16:0/2684668590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a101046d0 0x7f5a10102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.991+0000 7f5a15e00640 1 -- 172.21.15.16:0/2684668590 >> 172.21.15.16:0/2684668590 conn(0x7f5a100fe160 msgr2=0x7f5a10107640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.992+0000 7f5a15e00640 1 -- 172.21.15.16:0/2684668590 shutdown_connections 2024-08-31T06:07:57.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:07:57.992+0000 7f5a15e00640 1 -- 172.21.15.16:0/2684668590 wait complete. 2024-08-31T06:07:58.340 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:58 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/1543050883' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:07:58.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:58 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/1543050883' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:07:58.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:58 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/1543050883' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:07:58.809 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24737 quiesce db --set-id d396c487 --release --await' 2024-08-31T06:07:58.810 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24737 quiesce db --set-id d396c487 --release --await 2024-08-31T06:07:59.236 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:07:59.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:07:59 smithi155 ceph-mon[21742]: pgmap v386: 97 pgs: 97 active+clean; 929 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.2 MiB/s wr, 107 op/s 2024-08-31T06:07:59.375 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:07:59 smithi016 ceph-mon[17194]: pgmap v386: 97 pgs: 97 active+clean; 929 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.2 MiB/s wr, 107 op/s 2024-08-31T06:07:59.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:07:59 smithi184 ceph-mon[22331]: pgmap v386: 97 pgs: 97 active+clean; 929 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.2 MiB/s wr, 107 op/s 2024-08-31T06:07:59.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.888+0000 7f0164600640 1 -- 172.21.15.16:0/2762033565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f015c10bdb0 msgr2=0x7f015c1125e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:59.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.888+0000 7f0164600640 1 --2- 172.21.15.16:0/2762033565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f015c10bdb0 0x7f015c1125e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0154009ca0 tx=0x7f015402a8c0 comp rx=0 tx=0).stop 2024-08-31T06:07:59.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.888+0000 7f0164600640 1 -- 172.21.15.16:0/2762033565 shutdown_connections 2024-08-31T06:07:59.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.888+0000 7f0164600640 1 --2- 172.21.15.16:0/2762033565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f015c10bdb0 0x7f015c1125e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:59.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.888+0000 7f0164600640 1 --2- 172.21.15.16:0/2762033565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f015c10b460 0x7f015c10b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:59.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.888+0000 7f0164600640 1 --2- 172.21.15.16:0/2762033565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f015c105b20 0x7f015c105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:59.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.888+0000 7f0164600640 1 -- 172.21.15.16:0/2762033565 >> 172.21.15.16:0/2762033565 conn(0x7f015c1015b0 msgr2=0x7f015c103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:59.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.888+0000 7f0164600640 1 -- 172.21.15.16:0/2762033565 shutdown_connections 2024-08-31T06:07:59.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.888+0000 7f0164600640 1 -- 172.21.15.16:0/2762033565 wait complete. 2024-08-31T06:07:59.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.889+0000 7f0164600640 1 Processor -- start 2024-08-31T06:07:59.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.889+0000 7f0164600640 1 -- start start 2024-08-31T06:07:59.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.890+0000 7f0164600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f015c105b20 0x7f015c19c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:59.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.890+0000 7f0164600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f015c10b460 0x7f015c19c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:59.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.890+0000 7f0164600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f015c10bdb0 0x7f015c1a1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:59.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.890+0000 7f0164600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f015c115a70 con 0x7f015c105b20 2024-08-31T06:07:59.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.890+0000 7f0164600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f015c1158f0 con 0x7f015c10b460 2024-08-31T06:07:59.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.890+0000 7f0164600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f015c115bf0 con 0x7f015c10bdb0 2024-08-31T06:07:59.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.890+0000 7f0161c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f015c105b20 0x7f015c19c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:59.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.890+0000 7f0161c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f015c105b20 0x7f015c19c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55472/0 (socket says 172.21.15.16:55472) 2024-08-31T06:07:59.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.890+0000 7f0161c00640 1 -- 172.21.15.16:0/1611640678 learned_addr learned my addr 172.21.15.16:0/1611640678 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:59.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.890+0000 7f0161200640 1 --2- 172.21.15.16:0/1611640678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f015c10b460 0x7f015c19c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:59.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.890+0000 7f0162600640 1 --2- 172.21.15.16:0/1611640678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f015c10bdb0 0x7f015c1a1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:59.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.891+0000 7f0161c00640 1 -- 172.21.15.16:0/1611640678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f015c10bdb0 msgr2=0x7f015c1a1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:59.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.891+0000 7f0161c00640 1 --2- 172.21.15.16:0/1611640678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f015c10bdb0 0x7f015c1a1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:59.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.891+0000 7f0161c00640 1 -- 172.21.15.16:0/1611640678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f015c10b460 msgr2=0x7f015c19c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:59.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.891+0000 7f0161c00640 1 --2- 172.21.15.16:0/1611640678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f015c10b460 0x7f015c19c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:59.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.891+0000 7f0161c00640 1 -- 172.21.15.16:0/1611640678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f015c1a2370 con 0x7f015c105b20 2024-08-31T06:07:59.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.891+0000 7f0162600640 1 --2- 172.21.15.16:0/1611640678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f015c10bdb0 0x7f015c1a1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:07:59.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.891+0000 7f0161200640 1 --2- 172.21.15.16:0/1611640678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f015c10b460 0x7f015c19c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:07:59.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.891+0000 7f0161c00640 1 --2- 172.21.15.16:0/1611640678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f015c105b20 0x7f015c19c400 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f015000e9f0 tx=0x7f015000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:59.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.891+0000 7f015a000640 1 -- 172.21.15.16:0/1611640678 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015000cd80 con 0x7f015c105b20 2024-08-31T06:07:59.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.891+0000 7f015a000640 1 -- 172.21.15.16:0/1611640678 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01500054d0 con 0x7f015c105b20 2024-08-31T06:07:59.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.891+0000 7f0164600640 1 -- 172.21.15.16:0/1611640678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f015c1a2660 con 0x7f015c105b20 2024-08-31T06:07:59.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.891+0000 7f015a000640 1 -- 172.21.15.16:0/1611640678 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0150010690 con 0x7f015c105b20 2024-08-31T06:07:59.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.891+0000 7f0164600640 1 -- 172.21.15.16:0/1611640678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f015c1a8dc0 con 0x7f015c105b20 2024-08-31T06:07:59.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.895+0000 7f016b219740 1 Processor -- start 2024-08-31T06:07:59.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.895+0000 7f016b219740 1 -- start start 2024-08-31T06:07:59.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.895+0000 7f015a000640 1 -- 172.21.15.16:0/1611640678 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f0150004e90 con 0x7f015c105b20 2024-08-31T06:07:59.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.895+0000 7f016b219740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627ddcbd520 0x5627de062ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:59.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.895+0000 7f016b219740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627ddc94660 0x5627de063430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:59.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.895+0000 7f016b219740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ddc9c840 0x5627de063970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:59.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.895+0000 7f016b219740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5627ddfd9a90 con 0x5627ddc9c840 2024-08-31T06:07:59.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.895+0000 7f016b219740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5627ddfc9020 con 0x5627ddc94660 2024-08-31T06:07:59.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.896+0000 7f016b219740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5627ddfccfd0 con 0x5627ddcbd520 2024-08-31T06:07:59.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.896+0000 7f0162600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627ddcbd520 0x5627de062ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:59.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.896+0000 7f0162600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627ddcbd520 0x5627de062ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:40106/0 (socket says 172.21.15.16:40106) 2024-08-31T06:07:59.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.896+0000 7f0162600640 1 -- 172.21.15.16:0/432898148 learned_addr learned my addr 172.21.15.16:0/432898148 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:59.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.896+0000 7f0161200640 1 --2- 172.21.15.16:0/432898148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ddc9c840 0x5627de063970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:59.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.896+0000 7f0161c00640 1 --2- 172.21.15.16:0/432898148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627ddc94660 0x5627de063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:59.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.896+0000 7f0162600640 1 -- 172.21.15.16:0/432898148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627ddc94660 msgr2=0x5627de063430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:59.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.896+0000 7f0162600640 1 --2- 172.21.15.16:0/432898148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627ddc94660 0x5627de063430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:59.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.896+0000 7f0162600640 1 -- 172.21.15.16:0/432898148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ddc9c840 msgr2=0x5627de063970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:59.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.896+0000 7f0162600640 1 --2- 172.21.15.16:0/432898148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ddc9c840 0x5627de063970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:59.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.896+0000 7f0162600640 1 -- 172.21.15.16:0/432898148 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627ddd28b00 con 0x5627ddcbd520 2024-08-31T06:07:59.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.897+0000 7f0162600640 1 --2- 172.21.15.16:0/432898148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627ddcbd520 0x5627de062ef0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0154009470 tx=0x7f01540094a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5af1e1239021a2a5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:59.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.897+0000 7f014ea00640 1 -- 172.21.15.16:0/432898148 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0154031cf0 con 0x5627ddcbd520 2024-08-31T06:07:59.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.897+0000 7f014ea00640 1 -- 172.21.15.16:0/432898148 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0154036040 con 0x5627ddcbd520 2024-08-31T06:07:59.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.897+0000 7f014ea00640 1 -- 172.21.15.16:0/432898148 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0154004260 con 0x5627ddcbd520 2024-08-31T06:07:59.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.898+0000 7f015a000640 1 --2- 172.21.15.16:0/1611640678 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f01440b4d20 0x7f01440b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:59.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.898+0000 7f0161c00640 1 --2- 172.21.15.16:0/1611640678 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f01440b4d20 0x7f01440b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:59.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.898+0000 7f015a000640 1 -- 172.21.15.16:0/1611640678 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f01500e59b0 con 0x7f015c105b20 2024-08-31T06:07:59.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.898+0000 7f016b219740 1 -- 172.21.15.16:0/432898148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627ddcbd520 msgr2=0x5627de062ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:59.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.898+0000 7f016b219740 1 --2- 172.21.15.16:0/432898148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627ddcbd520 0x5627de062ef0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0154009470 tx=0x7f01540094a0 comp rx=0 tx=0).stop 2024-08-31T06:07:59.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.898+0000 7f0161c00640 1 --2- 172.21.15.16:0/1611640678 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f01440b4d20 0x7f01440b7230 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x5627de061b70 tx=0x7f01500e8f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:59.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.899+0000 7f016b219740 1 -- 172.21.15.16:0/432898148 shutdown_connections 2024-08-31T06:07:59.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.899+0000 7f016b219740 1 --2- 172.21.15.16:0/432898148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ddc9c840 0x5627de063970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:59.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.899+0000 7f016b219740 1 --2- 172.21.15.16:0/432898148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627ddc94660 0x5627de063430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:59.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.899+0000 7f016b219740 1 --2- 172.21.15.16:0/432898148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627ddcbd520 0x5627de062ef0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:59.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.899+0000 7f016b219740 1 -- 172.21.15.16:0/432898148 >> 172.21.15.16:0/432898148 conn(0x5627ddcb1910 msgr2=0x5627de060de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:59.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.899+0000 7f016b219740 1 -- 172.21.15.16:0/432898148 shutdown_connections 2024-08-31T06:07:59.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.899+0000 7f016b219740 1 -- 172.21.15.16:0/432898148 wait complete. 2024-08-31T06:07:59.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.899+0000 7f016b219740 1 -- start start 2024-08-31T06:07:59.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.900+0000 7f016b219740 1 Processor -- start 2024-08-31T06:07:59.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.900+0000 7f016b219740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627ddc94660 0x5627de06e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:59.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.900+0000 7f016b219740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627ddc9c840 0x5627de06e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:59.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.900+0000 7f016b219740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ddcbd520 0x5627de06eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:59.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.900+0000 7f016b219740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5627dde2bbb0 con 0x5627ddcbd520 2024-08-31T06:07:59.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.900+0000 7f016b219740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5627ddf4ff90 con 0x5627ddc9c840 2024-08-31T06:07:59.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.900+0000 7f016b219740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5627de071670 con 0x5627ddc94660 2024-08-31T06:07:59.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.900+0000 7f016b219740 10 client.0 _ll_register_callbacks cb 0x5627ddf75710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:07:59.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.900+0000 7f0162600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627ddc94660 0x5627de06e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:59.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.900+0000 7f0161200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627ddc9c840 0x5627de06e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:59.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.901+0000 7f0161200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627ddc9c840 0x5627de06e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:37970/0 (socket says 172.21.15.16:37970) 2024-08-31T06:07:59.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.901+0000 7f0161200640 1 -- 172.21.15.16:0/3885270956 learned_addr learned my addr 172.21.15.16:0/3885270956 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:59.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.901+0000 7f0162600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627ddc94660 0x5627de06e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:40122/0 (socket says 172.21.15.16:40122) 2024-08-31T06:07:59.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.901+0000 7f0162600640 1 --2- 172.21.15.16:0/3885270956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ddcbd520 0x5627de06eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:59.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.901+0000 7f0161200640 1 -- 172.21.15.16:0/3885270956 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627ddc94660 msgr2=0x5627de06e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:59.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.901+0000 7f0161200640 1 --2- 172.21.15.16:0/3885270956 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627ddc94660 0x5627de06e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:59.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.901+0000 7f0161200640 1 -- 172.21.15.16:0/3885270956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ddcbd520 msgr2=0x5627de06eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:59.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.901+0000 7f0161200640 1 --2- 172.21.15.16:0/3885270956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ddcbd520 0x5627de06eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:59.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.901+0000 7f0161200640 1 -- 172.21.15.16:0/3885270956 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627ddd9f960 con 0x5627ddc9c840 2024-08-31T06:07:59.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.901+0000 7f0161200640 1 --2- 172.21.15.16:0/3885270956 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627ddc9c840 0x5627de06e760 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0148012d20 tx=0x7f014800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bf436ee0cb6164f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:59.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.901+0000 7f0143400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:07:59.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.902+0000 7f0143400640 1 -- 172.21.15.16:0/3885270956 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0148012110 con 0x5627ddc9c840 2024-08-31T06:07:59.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.902+0000 7f0143400640 1 -- 172.21.15.16:0/3885270956 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f0148004050 con 0x5627ddc9c840 2024-08-31T06:07:59.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.902+0000 7f016b219740 1 -- 172.21.15.16:0/3885270956 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5627ddf67e30 con 0x5627ddc9c840 2024-08-31T06:07:59.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.902+0000 7f0143400640 1 -- 172.21.15.16:0/3885270956 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f01480122b0 con 0x5627ddc9c840 2024-08-31T06:07:59.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.902+0000 7f016b219740 10 client.25709 fetch_fsmap learned FSMap version 30 2024-08-31T06:07:59.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.902+0000 7f016b219740 1 -- 172.21.15.16:0/3885270956 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5627de07d0d0 con 0x5627ddc9c840 2024-08-31T06:07:59.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.902+0000 7f0143400640 1 -- 172.21.15.16:0/3885270956 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5627ddf67e30 con 0x5627ddc9c840 2024-08-31T06:07:59.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.903+0000 7f0143400640 1 -- 172.21.15.16:0/3885270956 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f0148026470 con 0x5627ddc9c840 2024-08-31T06:07:59.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.903+0000 7f016b219740 10 client.25709 fetch_fsmap finished waiting for FSMap version 30 2024-08-31T06:07:59.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.903+0000 7f016b219740 10 client.25709 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:07:59.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.903+0000 7f016b219740 20 client.25709 populate_metadata read hostname 'smithi016' 2024-08-31T06:07:59.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.903+0000 7f016b219740 1 --2- 172.21.15.16:0/3885270956 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5627ddc7bea0 0x5627de080610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:59.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.903+0000 7f016b219740 4 client.25709 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:07:59.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.903+0000 7f016b219740 1 -- 172.21.15.16:0/3885270956 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627ddfdb9e0 con 0x5627ddc7bea0 2024-08-31T06:07:59.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.903+0000 7f0162600640 1 --2- 172.21.15.16:0/3885270956 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5627ddc7bea0 0x5627de080610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:59.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.904+0000 7f0162600640 1 --2- 172.21.15.16:0/3885270956 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5627ddc7bea0 0x5627de080610 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86e5986dfb60afb6 server_cookie=b09c0aaae693dc2 in_seq=0 out_seq=0 2024-08-31T06:07:59.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.904+0000 7f0143400640 10 client.25709 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:59.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.905+0000 7f0143400640 1 -- 172.21.15.16:0/3885270956 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5627ddfdb9e0 con 0x5627ddc7bea0 2024-08-31T06:07:59.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.905+0000 7f0143400640 10 client.25709 handle_command_reply: tid=0 2024-08-31T06:07:59.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.905+0000 7f016b219740 1 client.25709 shutdown 2024-08-31T06:07:59.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.905+0000 7f016b219740 1 -- 172.21.15.16:0/3885270956 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627ddc9c840 msgr2=0x5627de06e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:59.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.905+0000 7f016b219740 1 --2- 172.21.15.16:0/3885270956 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627ddc9c840 0x5627de06e760 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0148012d20 tx=0x7f014800f790 comp rx=0 tx=0).stop 2024-08-31T06:07:59.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.905+0000 7f016b219740 1 -- 172.21.15.16:0/3885270956 shutdown_connections 2024-08-31T06:07:59.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.905+0000 7f016b219740 1 --2- 172.21.15.16:0/3885270956 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5627ddc7bea0 0x5627de080610 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:59.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.905+0000 7f016b219740 1 --2- 172.21.15.16:0/3885270956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ddcbd520 0x5627de06eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:59.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.905+0000 7f016b219740 1 --2- 172.21.15.16:0/3885270956 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627ddc9c840 0x5627de06e760 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:59.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.905+0000 7f0143400640 0 client.25709 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:07:59.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.905+0000 7f016b219740 1 --2- 172.21.15.16:0/3885270956 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627ddc94660 0x5627de06e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:59.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.905+0000 7f016b219740 1 -- 172.21.15.16:0/3885270956 >> 172.21.15.16:0/3885270956 conn(0x5627ddcb1910 msgr2=0x5627de066080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:59.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.905+0000 7f016b219740 1 -- 172.21.15.16:0/3885270956 shutdown_connections 2024-08-31T06:07:59.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.905+0000 7f016b219740 1 -- 172.21.15.16:0/3885270956 wait complete. 2024-08-31T06:07:59.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.906+0000 7f016b219740 20 client.25709 trim_cache size 0 max 16384 2024-08-31T06:07:59.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.924+0000 7f016b219740 1 Processor -- start 2024-08-31T06:07:59.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.924+0000 7f016b219740 1 -- start start 2024-08-31T06:07:59.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.925+0000 7f016b219740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ddc94660 0x5627de070100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:59.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.925+0000 7f016b219740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627ddc9c840 0x5627de07a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:59.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.925+0000 7f016b219740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627ddcbd520 0x5627de06e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:59.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.925+0000 7f016b219740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f015c1b3ee0 con 0x5627ddc94660 2024-08-31T06:07:59.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.925+0000 7f016b219740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f015c1b1720 con 0x5627ddcbd520 2024-08-31T06:07:59.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.925+0000 7f016b219740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f015c1af570 con 0x5627ddc9c840 2024-08-31T06:07:59.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.925+0000 7f0162600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ddc94660 0x5627de070100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:59.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.925+0000 7f0162600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ddc94660 0x5627de070100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55498/0 (socket says 172.21.15.16:55498) 2024-08-31T06:07:59.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.925+0000 7f0162600640 1 -- 172.21.15.16:0/2554012900 learned_addr learned my addr 172.21.15.16:0/2554012900 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:59.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.925+0000 7f0161200640 1 --2- 172.21.15.16:0/2554012900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627ddc9c840 0x5627de07a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:59.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.925+0000 7f0162600640 1 --2- 172.21.15.16:0/2554012900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627ddcbd520 0x5627de06e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:59.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.925+0000 7f0161200640 1 -- 172.21.15.16:0/2554012900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627ddcbd520 msgr2=0x5627de06e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:59.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.926+0000 7f0161200640 1 --2- 172.21.15.16:0/2554012900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627ddcbd520 0x5627de06e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:59.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.926+0000 7f0161200640 1 -- 172.21.15.16:0/2554012900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ddc94660 msgr2=0x5627de070100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:59.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.926+0000 7f0161200640 1 --2- 172.21.15.16:0/2554012900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ddc94660 0x5627de070100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:59.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.926+0000 7f0161200640 1 -- 172.21.15.16:0/2554012900 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627de065d50 con 0x5627ddc9c840 2024-08-31T06:07:59.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.926+0000 7f0162600640 1 --2- 172.21.15.16:0/2554012900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ddc94660 0x5627de070100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:07:59.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.926+0000 7f0161200640 1 --2- 172.21.15.16:0/2554012900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627ddc9c840 0x5627de07a890 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f01480268c0 tx=0x7f0148005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4323a51ca5cc9b34 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:59.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.926+0000 7f014e000640 1 -- 172.21.15.16:0/2554012900 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0148026530 con 0x5627ddc9c840 2024-08-31T06:07:59.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.926+0000 7f014e000640 1 -- 172.21.15.16:0/2554012900 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01480055c0 con 0x5627ddc9c840 2024-08-31T06:07:59.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.926+0000 7f014e000640 1 -- 172.21.15.16:0/2554012900 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0148003ac0 con 0x5627ddc9c840 2024-08-31T06:07:59.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.927+0000 7f016b219740 1 -- 172.21.15.16:0/2554012900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627ddc9c840 msgr2=0x5627de07a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:07:59.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.927+0000 7f016b219740 1 --2- 172.21.15.16:0/2554012900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627ddc9c840 0x5627de07a890 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f01480268c0 tx=0x7f0148005850 comp rx=0 tx=0).stop 2024-08-31T06:07:59.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.927+0000 7f016b219740 1 -- 172.21.15.16:0/2554012900 shutdown_connections 2024-08-31T06:07:59.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.927+0000 7f016b219740 1 --2- 172.21.15.16:0/2554012900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627ddc9c840 0x5627de07a890 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:59.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.927+0000 7f016b219740 1 --2- 172.21.15.16:0/2554012900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627ddcbd520 0x5627de06e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:59.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.927+0000 7f016b219740 1 --2- 172.21.15.16:0/2554012900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ddc94660 0x5627de070100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:59.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.927+0000 7f016b219740 1 -- 172.21.15.16:0/2554012900 >> 172.21.15.16:0/2554012900 conn(0x5627ddcb1910 msgr2=0x5627de06e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:07:59.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.927+0000 7f016b219740 1 -- 172.21.15.16:0/2554012900 shutdown_connections 2024-08-31T06:07:59.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.927+0000 7f016b219740 1 -- 172.21.15.16:0/2554012900 wait complete. 2024-08-31T06:07:59.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.928+0000 7f016b219740 1 -- start start 2024-08-31T06:07:59.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.928+0000 7f016b219740 1 Processor -- start 2024-08-31T06:07:59.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.929+0000 7f016b219740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ddc94660 0x5627de076e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:59.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.929+0000 7f016b219740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627ddc9c840 0x5627de073330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:59.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.929+0000 7f016b219740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627ddcbd520 0x5627de073870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:59.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.929+0000 7f016b219740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f015c1afbe0 con 0x5627ddc94660 2024-08-31T06:07:59.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.929+0000 7f016b219740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f015c1b1050 con 0x5627ddc9c840 2024-08-31T06:07:59.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.929+0000 7f016b219740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f015c1b02b0 con 0x5627ddcbd520 2024-08-31T06:07:59.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.929+0000 7f016b219740 10 client.0 _ll_register_callbacks cb 0x7f0148012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:07:59.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.929+0000 7f0162600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ddc94660 0x5627de076e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:59.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.929+0000 7f0162600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ddc94660 0x5627de076e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55508/0 (socket says 172.21.15.16:55508) 2024-08-31T06:07:59.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.929+0000 7f0162600640 1 -- 172.21.15.16:0/684343221 learned_addr learned my addr 172.21.15.16:0/684343221 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:07:59.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.929+0000 7f0161200640 1 --2- 172.21.15.16:0/684343221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627ddc9c840 0x5627de073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:59.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.929+0000 7f0162600640 1 --2- 172.21.15.16:0/684343221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627ddcbd520 0x5627de073870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:59.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.929+0000 7f0162600640 1 -- 172.21.15.16:0/684343221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627ddcbd520 msgr2=0x5627de073870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:59.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.929+0000 7f0162600640 1 --2- 172.21.15.16:0/684343221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627ddcbd520 0x5627de073870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:59.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.929+0000 7f0162600640 1 -- 172.21.15.16:0/684343221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627ddc9c840 msgr2=0x5627de073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:07:59.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.929+0000 7f0162600640 1 --2- 172.21.15.16:0/684343221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627ddc9c840 0x5627de073330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:07:59.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.929+0000 7f0162600640 1 -- 172.21.15.16:0/684343221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627de073db0 con 0x5627ddc94660 2024-08-31T06:07:59.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.930+0000 7f0162600640 1 --2- 172.21.15.16:0/684343221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ddc94660 0x5627de076e70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f015402ade0 tx=0x7f0154003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc459db7abd7df90 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:07:59.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.930+0000 7f0143400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:07:59.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.930+0000 7f0143400640 1 -- 172.21.15.16:0/684343221 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015405d070 con 0x5627ddc94660 2024-08-31T06:07:59.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.930+0000 7f0143400640 1 -- 172.21.15.16:0/684343221 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f0154003cd0 con 0x5627ddc94660 2024-08-31T06:07:59.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.930+0000 7f016b219740 1 -- 172.21.15.16:0/684343221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01480055c0 con 0x5627ddc94660 2024-08-31T06:07:59.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.930+0000 7f0143400640 1 -- 172.21.15.16:0/684343221 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f015404a040 con 0x5627ddc94660 2024-08-31T06:07:59.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.930+0000 7f016b219740 10 client.15675 fetch_fsmap learned FSMap version 30 2024-08-31T06:07:59.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.930+0000 7f016b219740 1 -- 172.21.15.16:0/684343221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5627de0ebcd0 con 0x5627ddc94660 2024-08-31T06:07:59.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.931+0000 7f0143400640 1 -- 172.21.15.16:0/684343221 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01480055c0 con 0x5627ddc94660 2024-08-31T06:07:59.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.931+0000 7f0143400640 1 -- 172.21.15.16:0/684343221 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f0154050040 con 0x5627ddc94660 2024-08-31T06:07:59.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.931+0000 7f016b219740 10 client.15675 fetch_fsmap finished waiting for FSMap version 30 2024-08-31T06:07:59.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.931+0000 7f016b219740 10 client.15675 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:07:59.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.931+0000 7f016b219740 20 client.15675 populate_metadata read hostname 'smithi016' 2024-08-31T06:07:59.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.931+0000 7f016b219740 1 --2- 172.21.15.16:0/684343221 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5627ddc51250 0x5627de0ee040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:07:59.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.931+0000 7f016b219740 4 client.15675 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d396c487", "release": true, "await": true}] 2024-08-31T06:07:59.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.931+0000 7f016b219740 1 -- 172.21.15.16:0/684343221 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d396c487", "release": true, "await": true}) -- 0x5627ddfd7480 con 0x5627ddc51250 2024-08-31T06:07:59.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.932+0000 7f0161200640 1 --2- 172.21.15.16:0/684343221 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5627ddc51250 0x5627de0ee040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:07:59.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.932+0000 7f0161200640 1 --2- 172.21.15.16:0/684343221 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5627ddc51250 0x5627de0ee040 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1df5d5977593023 server_cookie=8b4b5c47d403df3d in_seq=0 out_seq=0 2024-08-31T06:07:59.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:07:59.932+0000 7f0143400640 10 client.15675 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:00.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:00.147+0000 7f0143400640 1 -- 172.21.15.16:0/684343221 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5627ddfd7480 con 0x5627ddc51250 2024-08-31T06:08:00.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:00.147+0000 7f0143400640 10 client.15675 handle_command_reply: tid=0 2024-08-31T06:08:00.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:00.147+0000 7f016b219740 1 client.15675 shutdown 2024-08-31T06:08:00.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:00.148+0000 7f016b219740 1 -- 172.21.15.16:0/684343221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ddc94660 msgr2=0x5627de076e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:00.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:00.148+0000 7f016b219740 1 --2- 172.21.15.16:0/684343221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ddc94660 0x5627de076e70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f015402ade0 tx=0x7f0154003f90 comp rx=0 tx=0).stop 2024-08-31T06:08:00.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:00.148+0000 7f016b219740 1 -- 172.21.15.16:0/684343221 shutdown_connections 2024-08-31T06:08:00.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:00.148+0000 7f016b219740 1 --2- 172.21.15.16:0/684343221 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5627ddc51250 0x5627de0ee040 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:00.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:00.148+0000 7f016b219740 1 --2- 172.21.15.16:0/684343221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627ddcbd520 0x5627de073870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:00.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:00.148+0000 7f016b219740 1 --2- 172.21.15.16:0/684343221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627ddc9c840 0x5627de073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:00.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:00.148+0000 7f016b219740 1 --2- 172.21.15.16:0/684343221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ddc94660 0x5627de076e70 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:00.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:00.148+0000 7f016b219740 1 -- 172.21.15.16:0/684343221 >> 172.21.15.16:0/684343221 conn(0x5627ddcb1910 msgr2=0x5627de06eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:00.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:00.148+0000 7f0143400640 0 client.15675 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:00.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:00.149+0000 7f016b219740 1 -- 172.21.15.16:0/684343221 shutdown_connections 2024-08-31T06:08:00.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:00.149+0000 7f016b219740 1 -- 172.21.15.16:0/684343221 wait complete. 2024-08-31T06:08:00.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:00.150+0000 7f016b219740 20 client.15675 trim_cache size 0 max 16384 2024-08-31T06:08:00.153 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:08:00.153 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 30, 2024-08-31T06:08:00.153 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24737, 2024-08-31T06:08:00.153 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 4, 2024-08-31T06:08:00.153 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-08-31T06:08:00.154 INFO:teuthology.orchestra.run.smithi016.stdout: "d396c487": { 2024-08-31T06:08:00.154 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 4, 2024-08-31T06:08:00.154 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-08-31T06:08:00.154 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:08:00.154 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-08-31T06:08:00.154 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:08:00.154 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:08:00.154 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-08-31T06:08:00.154 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 146.4, 2024-08-31T06:08:00.154 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-08-31T06:08:00.154 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-08-31T06:08:00.154 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-08-31T06:08:00.154 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:08:00.154 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-08-31T06:08:00.155 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:08:00.155 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:08:00.155 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:08:00.155 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:08:00.155 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:08:00.155 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:08:00.155 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:08:00.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:00.151+0000 7f0164600640 1 -- 172.21.15.16:0/1611640678 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f01440b4d20 msgr2=0x7f01440b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:00.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:00.151+0000 7f0164600640 1 --2- 172.21.15.16:0/1611640678 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f01440b4d20 0x7f01440b7230 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x5627de061b70 tx=0x7f01500e8f90 comp rx=0 tx=0).stop 2024-08-31T06:08:00.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:00.151+0000 7f0164600640 1 -- 172.21.15.16:0/1611640678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f015c105b20 msgr2=0x7f015c19c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:00.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:00.151+0000 7f0164600640 1 --2- 172.21.15.16:0/1611640678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f015c105b20 0x7f015c19c400 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f015000e9f0 tx=0x7f015000edf0 comp rx=0 tx=0).stop 2024-08-31T06:08:00.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:00.151+0000 7f0164600640 1 -- 172.21.15.16:0/1611640678 shutdown_connections 2024-08-31T06:08:00.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:00.151+0000 7f0164600640 1 --2- 172.21.15.16:0/1611640678 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f01440b4d20 0x7f01440b7230 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:00.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:00.151+0000 7f0164600640 1 --2- 172.21.15.16:0/1611640678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f015c10bdb0 0x7f015c1a1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:00.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:00.151+0000 7f0164600640 1 --2- 172.21.15.16:0/1611640678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f015c10b460 0x7f015c19c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:00.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:00.151+0000 7f0164600640 1 --2- 172.21.15.16:0/1611640678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f015c105b20 0x7f015c19c400 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:00.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:00.151+0000 7f0164600640 1 -- 172.21.15.16:0/1611640678 >> 172.21.15.16:0/1611640678 conn(0x7f015c1015b0 msgr2=0x7f015c108a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:00.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:00.152+0000 7f0164600640 1 -- 172.21.15.16:0/1611640678 shutdown_connections 2024-08-31T06:08:00.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:00.152+0000 7f0164600640 1 -- 172.21.15.16:0/1611640678 wait complete. 2024-08-31T06:08:00.975 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd396c487', total seconds elapsed: 30.6 2024-08-31T06:08:00.975 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.4 2024-08-31T06:08:00.975 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:08:01.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:01 smithi155 ceph-mon[21742]: pgmap v387: 97 pgs: 97 active+clean; 952 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.8 MiB/s wr, 146 op/s 2024-08-31T06:08:01.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:01 smithi016 ceph-mon[17194]: pgmap v387: 97 pgs: 97 active+clean; 952 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.8 MiB/s wr, 146 op/s 2024-08-31T06:08:01.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:01 smithi184 ceph-mon[22331]: pgmap v387: 97 pgs: 97 active+clean; 952 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.8 MiB/s wr, 146 op/s 2024-08-31T06:08:01.497 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:08:01.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 35.0 s, 125.5 tps, lat 4112.006 ms stddev 8929.952 2024-08-31T06:08:02.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.223+0000 7f10b1800640 1 -- 172.21.15.16:0/2252103562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10ac10b2a0 msgr2=0x7f10ac10b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:02.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.223+0000 7f10b1800640 1 --2- 172.21.15.16:0/2252103562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10ac10b2a0 0x7f10ac10b6b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1098009900 tx=0x7f109802a3f0 comp rx=0 tx=0).stop 2024-08-31T06:08:02.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.225+0000 7f10b1800640 1 -- 172.21.15.16:0/2252103562 shutdown_connections 2024-08-31T06:08:02.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.225+0000 7f10b1800640 1 --2- 172.21.15.16:0/2252103562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10ac10bbf0 0x7f10ac112420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:02.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.225+0000 7f10b1800640 1 --2- 172.21.15.16:0/2252103562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10ac10b2a0 0x7f10ac10b6b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:02.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.225+0000 7f10b1800640 1 --2- 172.21.15.16:0/2252103562 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10ac105960 0x7f10ac105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:02.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.225+0000 7f10b1800640 1 -- 172.21.15.16:0/2252103562 >> 172.21.15.16:0/2252103562 conn(0x7f10ac1013d0 msgr2=0x7f10ac103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:02.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.227+0000 7f10b1800640 1 -- 172.21.15.16:0/2252103562 shutdown_connections 2024-08-31T06:08:02.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.227+0000 7f10b1800640 1 -- 172.21.15.16:0/2252103562 wait complete. 2024-08-31T06:08:02.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.229+0000 7f10b1800640 1 Processor -- start 2024-08-31T06:08:02.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.229+0000 7f10b1800640 1 -- start start 2024-08-31T06:08:02.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.229+0000 7f10b1800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10ac105960 0x7f10ac111380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:02.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.230+0000 7f10b1800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10ac10b2a0 0x7f10ac111920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:02.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.230+0000 7f10b1800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10ac10bbf0 0x7f10ac1a5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:02.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.230+0000 7f10b1800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f10ac078740 con 0x7f10ac10bbf0 2024-08-31T06:08:02.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.230+0000 7f10b1800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f10ac0785c0 con 0x7f10ac10b2a0 2024-08-31T06:08:02.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.230+0000 7f10b1800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f10ac0788c0 con 0x7f10ac105960 2024-08-31T06:08:02.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.230+0000 7f10abe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10ac105960 0x7f10ac111380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:02.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.230+0000 7f10abe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10ac105960 0x7f10ac111380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:40150/0 (socket says 172.21.15.16:40150) 2024-08-31T06:08:02.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.230+0000 7f10abe00640 1 -- 172.21.15.16:0/1643197785 learned_addr learned my addr 172.21.15.16:0/1643197785 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:02.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.231+0000 7f10abe00640 1 -- 172.21.15.16:0/1643197785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10ac10b2a0 msgr2=0x7f10ac111920 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-31T06:08:02.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.231+0000 7f10abe00640 1 --2- 172.21.15.16:0/1643197785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10ac10b2a0 0x7f10ac111920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:02.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.231+0000 7f10abe00640 1 -- 172.21.15.16:0/1643197785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10ac10bbf0 msgr2=0x7f10ac1a5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:02.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.231+0000 7f10abe00640 1 --2- 172.21.15.16:0/1643197785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10ac10bbf0 0x7f10ac1a5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:02.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.231+0000 7f10abe00640 1 -- 172.21.15.16:0/1643197785 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10ac1a6690 con 0x7f10ac105960 2024-08-31T06:08:02.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.231+0000 7f10abe00640 1 --2- 172.21.15.16:0/1643197785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10ac105960 0x7f10ac111380 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f109c00d950 tx=0x7f109c00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:02.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.232+0000 7f10a8c00640 1 -- 172.21.15.16:0/1643197785 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109c017070 con 0x7f10ac105960 2024-08-31T06:08:02.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.232+0000 7f10a8c00640 1 -- 172.21.15.16:0/1643197785 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f109c009c30 con 0x7f10ac105960 2024-08-31T06:08:02.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.232+0000 7f10a8c00640 1 -- 172.21.15.16:0/1643197785 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109c00f620 con 0x7f10ac105960 2024-08-31T06:08:02.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.232+0000 7f10b1800640 1 -- 172.21.15.16:0/1643197785 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10ac1a6980 con 0x7f10ac105960 2024-08-31T06:08:02.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.232+0000 7f10b1800640 1 -- 172.21.15.16:0/1643197785 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10ac1ad0e0 con 0x7f10ac105960 2024-08-31T06:08:02.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.234+0000 7f10b1800640 1 -- 172.21.15.16:0/1643197785 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10ac105d70 con 0x7f10ac105960 2024-08-31T06:08:02.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.243+0000 7f10a8c00640 1 -- 172.21.15.16:0/1643197785 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f109c005030 con 0x7f10ac105960 2024-08-31T06:08:02.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.247+0000 7f10a8c00640 1 --2- 172.21.15.16:0/1643197785 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f10900b4e70 0x7f10900b7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:02.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.247+0000 7f10a8c00640 1 -- 172.21.15.16:0/1643197785 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f109c0e50e0 con 0x7f10ac105960 2024-08-31T06:08:02.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.248+0000 7f10ab400640 1 --2- 172.21.15.16:0/1643197785 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f10900b4e70 0x7f10900b7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:02.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.248+0000 7f10a8c00640 1 -- 172.21.15.16:0/1643197785 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f109c0e55a0 con 0x7f10ac105960 2024-08-31T06:08:02.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.249+0000 7f10ab400640 1 --2- 172.21.15.16:0/1643197785 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f10900b4e70 0x7f10900b7380 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1098009230 tx=0x7f1098005fd0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:02.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.533+0000 7f10b1800640 1 -- 172.21.15.16:0/1643197785 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10ac02e430 con 0x7f10ac105960 2024-08-31T06:08:02.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.534+0000 7f10a8c00640 1 -- 172.21.15.16:0/1643197785 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11921 (secure 0 0 0) 0x7f109c0abfd0 con 0x7f10ac105960 2024-08-31T06:08:02.537 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:08:02.537 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":30,"btime":"2024-08-31T06:07:34:013454+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:07:33.329391+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24712,24724,24737]},"id":1}]} 2024-08-31T06:08:02.537 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 30 2024-08-31T06:08:02.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.537+0000 7f10b1800640 1 -- 172.21.15.16:0/1643197785 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f10900b4e70 msgr2=0x7f10900b7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:02.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.537+0000 7f10b1800640 1 --2- 172.21.15.16:0/1643197785 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f10900b4e70 0x7f10900b7380 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1098009230 tx=0x7f1098005fd0 comp rx=0 tx=0).stop 2024-08-31T06:08:02.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.537+0000 7f10b1800640 1 -- 172.21.15.16:0/1643197785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10ac105960 msgr2=0x7f10ac111380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:02.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.537+0000 7f10b1800640 1 --2- 172.21.15.16:0/1643197785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10ac105960 0x7f10ac111380 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f109c00d950 tx=0x7f109c00de10 comp rx=0 tx=0).stop 2024-08-31T06:08:02.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.537+0000 7f10b1800640 1 -- 172.21.15.16:0/1643197785 shutdown_connections 2024-08-31T06:08:02.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.537+0000 7f10b1800640 1 --2- 172.21.15.16:0/1643197785 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f10900b4e70 0x7f10900b7380 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:02.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.537+0000 7f10b1800640 1 --2- 172.21.15.16:0/1643197785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10ac10bbf0 0x7f10ac1a5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:02.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.537+0000 7f10b1800640 1 --2- 172.21.15.16:0/1643197785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10ac10b2a0 0x7f10ac111920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:02.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.537+0000 7f10b1800640 1 --2- 172.21.15.16:0/1643197785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10ac105960 0x7f10ac111380 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:02.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.537+0000 7f10b1800640 1 -- 172.21.15.16:0/1643197785 >> 172.21.15.16:0/1643197785 conn(0x7f10ac1013d0 msgr2=0x7f10ac07aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:02.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.537+0000 7f10b1800640 1 -- 172.21.15.16:0/1643197785 shutdown_connections 2024-08-31T06:08:02.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:02.537+0000 7f10b1800640 1 -- 172.21.15.16:0/1643197785 wait complete. 2024-08-31T06:08:03.328 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:03 smithi016 ceph-mon[17194]: pgmap v388: 97 pgs: 97 active+clean; 959 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.4 MiB/s wr, 143 op/s 2024-08-31T06:08:03.329 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:03 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/1643197785' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:08:03.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:03 smithi155 ceph-mon[21742]: pgmap v388: 97 pgs: 97 active+clean; 959 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.4 MiB/s wr, 143 op/s 2024-08-31T06:08:03.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:03 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/1643197785' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:08:03.398 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24737 quiesce db / --timeout 90 --expiration 146.4 --await-for 45.0' 2024-08-31T06:08:03.398 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24737 quiesce db / --timeout 90 --expiration 146.4 --await-for 45.0 2024-08-31T06:08:03.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:03 smithi184 ceph-mon[22331]: pgmap v388: 97 pgs: 97 active+clean; 959 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.4 MiB/s wr, 143 op/s 2024-08-31T06:08:03.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:03 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/1643197785' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:08:04.030 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:08:05.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:05 smithi155 ceph-mon[21742]: pgmap v389: 97 pgs: 97 active+clean; 948 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 12 MiB/s wr, 228 op/s 2024-08-31T06:08:05.374 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:05 smithi016 ceph-mon[17194]: pgmap v389: 97 pgs: 97 active+clean; 948 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 12 MiB/s wr, 228 op/s 2024-08-31T06:08:05.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:05 smithi184 ceph-mon[22331]: pgmap v389: 97 pgs: 97 active+clean; 948 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 12 MiB/s wr, 228 op/s 2024-08-31T06:08:05.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.571+0000 7febe9600640 1 -- 172.21.15.16:0/2958138806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febe410bdf0 msgr2=0x7febe4112620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:05.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.571+0000 7febe9600640 1 --2- 172.21.15.16:0/2958138806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febe410bdf0 0x7febe4112620 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7febd4009860 tx=0x7febd402a360 comp rx=0 tx=0).stop 2024-08-31T06:08:05.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.571+0000 7febe9600640 1 -- 172.21.15.16:0/2958138806 shutdown_connections 2024-08-31T06:08:05.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.571+0000 7febe9600640 1 --2- 172.21.15.16:0/2958138806 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febe410b4a0 0x7febe410b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:05.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.571+0000 7febe9600640 1 --2- 172.21.15.16:0/2958138806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febe410bdf0 0x7febe4112620 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:05.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.571+0000 7febe9600640 1 --2- 172.21.15.16:0/2958138806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febe4105b60 0x7febe4105f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:05.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.571+0000 7febe9600640 1 -- 172.21.15.16:0/2958138806 >> 172.21.15.16:0/2958138806 conn(0x7febe4101570 msgr2=0x7febe41039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:05.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.572+0000 7febe9600640 1 -- 172.21.15.16:0/2958138806 shutdown_connections 2024-08-31T06:08:05.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.572+0000 7febe9600640 1 -- 172.21.15.16:0/2958138806 wait complete. 2024-08-31T06:08:05.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.573+0000 7febe9600640 1 Processor -- start 2024-08-31T06:08:05.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.573+0000 7febe9600640 1 -- start start 2024-08-31T06:08:05.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.574+0000 7febe9600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febe4105b60 0x7febe407b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:05.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.574+0000 7febe9600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febe410b4a0 0x7febe40776c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:05.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.574+0000 7febe9600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febe410bdf0 0x7febe4077de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:05.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.574+0000 7febe9600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7febe4115ab0 con 0x7febe410b4a0 2024-08-31T06:08:05.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.574+0000 7febe9600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7febe4115930 con 0x7febe4105b60 2024-08-31T06:08:05.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.574+0000 7febe9600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7febe4115c30 con 0x7febe410bdf0 2024-08-31T06:08:05.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.575+0000 7febe2000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febe410b4a0 0x7febe40776c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:05.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.575+0000 7febe2000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febe410b4a0 0x7febe40776c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55540/0 (socket says 172.21.15.16:55540) 2024-08-31T06:08:05.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.575+0000 7febe2000640 1 -- 172.21.15.16:0/105672667 learned_addr learned my addr 172.21.15.16:0/105672667 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:05.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.576+0000 7febe2a00640 1 --2- 172.21.15.16:0/105672667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febe4105b60 0x7febe407b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:05.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.576+0000 7febe3400640 1 --2- 172.21.15.16:0/105672667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febe410bdf0 0x7febe4077de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:05.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.577+0000 7febe2000640 1 -- 172.21.15.16:0/105672667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febe410bdf0 msgr2=0x7febe4077de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:05.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.577+0000 7febe2000640 1 --2- 172.21.15.16:0/105672667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febe410bdf0 0x7febe4077de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:05.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.577+0000 7febe2000640 1 -- 172.21.15.16:0/105672667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febe4105b60 msgr2=0x7febe407b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:05.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.577+0000 7febe2000640 1 --2- 172.21.15.16:0/105672667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febe4105b60 0x7febe407b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:05.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.577+0000 7febe2000640 1 -- 172.21.15.16:0/105672667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe41ac9f0 con 0x7febe410b4a0 2024-08-31T06:08:05.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.577+0000 7febe3400640 1 --2- 172.21.15.16:0/105672667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febe410bdf0 0x7febe4077de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:08:05.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.578+0000 7febe2000640 1 --2- 172.21.15.16:0/105672667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febe410b4a0 0x7febe40776c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7febd000e9c0 tx=0x7febd000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:05.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.579+0000 7febdb400640 1 -- 172.21.15.16:0/105672667 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd000ccc0 con 0x7febe410b4a0 2024-08-31T06:08:05.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.579+0000 7febdb400640 1 -- 172.21.15.16:0/105672667 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febd00108e0 con 0x7febe410b4a0 2024-08-31T06:08:05.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.579+0000 7febe9600640 1 -- 172.21.15.16:0/105672667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febe41acc20 con 0x7febe410b4a0 2024-08-31T06:08:05.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.579+0000 7febe9600640 1 -- 172.21.15.16:0/105672667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febe41ad160 con 0x7febe410b4a0 2024-08-31T06:08:05.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.579+0000 7febdb400640 1 -- 172.21.15.16:0/105672667 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd00187e0 con 0x7febe410b4a0 2024-08-31T06:08:05.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.584+0000 7febdb400640 1 -- 172.21.15.16:0/105672667 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7febd0010430 con 0x7febe410b4a0 2024-08-31T06:08:05.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.585+0000 7febf0244740 1 Processor -- start 2024-08-31T06:08:05.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.586+0000 7febf0244740 1 -- start start 2024-08-31T06:08:05.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.586+0000 7febf0244740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f14e668840 0x55f14ea0f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:05.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.586+0000 7febf0244740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f14e63f980 0x55f14ea0f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:05.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.586+0000 7febf0244740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f14e647b60 0x55f14ea0fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:05.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.586+0000 7febf0244740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f14e984530 con 0x55f14e647b60 2024-08-31T06:08:05.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.586+0000 7febf0244740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f14e973ac0 con 0x55f14e668840 2024-08-31T06:08:05.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.586+0000 7febf0244740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f14e977a70 con 0x55f14e63f980 2024-08-31T06:08:05.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.587+0000 7febe2a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f14e63f980 0x55f14ea0f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:05.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.587+0000 7febe2a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f14e63f980 0x55f14ea0f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:40178/0 (socket says 172.21.15.16:40178) 2024-08-31T06:08:05.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.587+0000 7febe2a00640 1 -- 172.21.15.16:0/2269918595 learned_addr learned my addr 172.21.15.16:0/2269918595 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:05.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.587+0000 7febdb400640 1 --2- 172.21.15.16:0/105672667 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7febc80b4d70 0x7febc80b7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:05.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.587+0000 7febe2a00640 1 -- 172.21.15.16:0/2269918595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f14e668840 msgr2=0x55f14ea0f110 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T06:08:05.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.587+0000 7febe2a00640 1 --2- 172.21.15.16:0/2269918595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f14e668840 0x55f14ea0f110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:05.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.587+0000 7febe2a00640 1 -- 172.21.15.16:0/2269918595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f14e647b60 msgr2=0x55f14ea0fb90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:08:05.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.587+0000 7febe2a00640 1 --2- 172.21.15.16:0/2269918595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f14e647b60 0x55f14ea0fb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:05.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.587+0000 7febe2a00640 1 -- 172.21.15.16:0/2269918595 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f14e926880 con 0x55f14e63f980 2024-08-31T06:08:05.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.587+0000 7febdb400640 1 -- 172.21.15.16:0/105672667 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7febd00e5720 con 0x7febe410b4a0 2024-08-31T06:08:05.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.588+0000 7febe2a00640 1 --2- 172.21.15.16:0/2269918595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f14e63f980 0x55f14ea0f650 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7febdc009fd0 tx=0x7febdc003e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56b12a8f37ff1b81 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:05.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.588+0000 7febcf400640 1 -- 172.21.15.16:0/2269918595 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febdc012450 con 0x55f14e63f980 2024-08-31T06:08:05.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.588+0000 7febcf400640 1 -- 172.21.15.16:0/2269918595 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febdc012aa0 con 0x55f14e63f980 2024-08-31T06:08:05.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.588+0000 7febe3400640 1 --2- 172.21.15.16:0/105672667 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7febc80b4d70 0x7febc80b7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:05.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.588+0000 7febcf400640 1 -- 172.21.15.16:0/2269918595 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febdc011b90 con 0x55f14e63f980 2024-08-31T06:08:05.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.592+0000 7febf0244740 1 -- 172.21.15.16:0/2269918595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f14e63f980 msgr2=0x55f14ea0f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:05.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.592+0000 7febf0244740 1 --2- 172.21.15.16:0/2269918595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f14e63f980 0x55f14ea0f650 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7febdc009fd0 tx=0x7febdc003e80 comp rx=0 tx=0).stop 2024-08-31T06:08:05.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.592+0000 7febe3400640 1 --2- 172.21.15.16:0/105672667 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7febc80b4d70 0x7febc80b7280 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7febd4009830 tx=0x7febd4009390 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:05.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.592+0000 7febf0244740 1 -- 172.21.15.16:0/2269918595 shutdown_connections 2024-08-31T06:08:05.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.592+0000 7febf0244740 1 --2- 172.21.15.16:0/2269918595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f14e63f980 0x55f14ea0f650 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:05.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.592+0000 7febf0244740 1 --2- 172.21.15.16:0/2269918595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f14e647b60 0x55f14ea0fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:05.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.592+0000 7febf0244740 1 --2- 172.21.15.16:0/2269918595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f14e668840 0x55f14ea0f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:05.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.592+0000 7febf0244740 1 -- 172.21.15.16:0/2269918595 >> 172.21.15.16:0/2269918595 conn(0x55f14e65c990 msgr2=0x55f14ea0d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:05.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.592+0000 7febf0244740 1 -- 172.21.15.16:0/2269918595 shutdown_connections 2024-08-31T06:08:05.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.593+0000 7febf0244740 1 -- 172.21.15.16:0/2269918595 wait complete. 2024-08-31T06:08:05.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.593+0000 7febf0244740 1 -- start start 2024-08-31T06:08:05.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.594+0000 7febf0244740 1 Processor -- start 2024-08-31T06:08:05.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.594+0000 7febf0244740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f14e63f980 0x55f14ea1ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:05.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.594+0000 7febf0244740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f14e647b60 0x55f14ea1b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:05.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.594+0000 7febf0244740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f14e668840 0x55f14ea26c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:05.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.595+0000 7febe2a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f14e63f980 0x55f14ea1ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:05.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.595+0000 7febe2000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f14e647b60 0x55f14ea1b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:05.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.595+0000 7febf0244740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f14e7d67a0 con 0x55f14e647b60 2024-08-31T06:08:05.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.595+0000 7febe2000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f14e647b60 0x55f14ea1b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55542/0 (socket says 172.21.15.16:55542) 2024-08-31T06:08:05.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.595+0000 7febe2000640 1 -- 172.21.15.16:0/4289367330 learned_addr learned my addr 172.21.15.16:0/4289367330 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:05.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.595+0000 7febf0244740 1 -- 172.21.15.16:0/4289367330 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f14e8fa7f0 con 0x55f14e63f980 2024-08-31T06:08:05.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.595+0000 7febe3400640 1 --2- 172.21.15.16:0/4289367330 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f14e668840 0x55f14ea26c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:05.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.595+0000 7febf0244740 1 -- 172.21.15.16:0/4289367330 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f14ea1d8f0 con 0x55f14e668840 2024-08-31T06:08:05.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.595+0000 7febf0244740 10 client.0 _ll_register_callbacks cb 0x55f14e91fe60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:08:05.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.596+0000 7febe2000640 1 -- 172.21.15.16:0/4289367330 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f14e668840 msgr2=0x55f14ea26c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:05.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.596+0000 7febe2000640 1 --2- 172.21.15.16:0/4289367330 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f14e668840 0x55f14ea26c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:05.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.596+0000 7febe2000640 1 -- 172.21.15.16:0/4289367330 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f14e63f980 msgr2=0x55f14ea1ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:05.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.596+0000 7febe2000640 1 --2- 172.21.15.16:0/4289367330 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f14e63f980 0x55f14ea1ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:05.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.596+0000 7febe2000640 1 -- 172.21.15.16:0/4289367330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f14e6e2b90 con 0x55f14e647b60 2024-08-31T06:08:05.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.596+0000 7febe2000640 1 --2- 172.21.15.16:0/4289367330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f14e647b60 0x55f14ea1b2e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7febd00abe00 tx=0x7febd0010870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29b2f9b42383f81a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:05.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.596+0000 7febc3e00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:08:05.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.596+0000 7febc3e00640 1 -- 172.21.15.16:0/4289367330 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd00b06e0 con 0x55f14e647b60 2024-08-31T06:08:05.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.597+0000 7febc3e00640 1 -- 172.21.15.16:0/4289367330 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7febd00a8100 con 0x55f14e647b60 2024-08-31T06:08:05.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.597+0000 7febf0244740 1 -- 172.21.15.16:0/4289367330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febdc012aa0 con 0x55f14e647b60 2024-08-31T06:08:05.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.597+0000 7febc3e00640 1 -- 172.21.15.16:0/4289367330 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7febd00b5190 con 0x55f14e647b60 2024-08-31T06:08:05.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.597+0000 7febf0244740 10 client.15687 fetch_fsmap learned FSMap version 30 2024-08-31T06:08:05.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.597+0000 7febf0244740 1 -- 172.21.15.16:0/4289367330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f14ea29a60 con 0x55f14e647b60 2024-08-31T06:08:05.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.597+0000 7febc3e00640 1 -- 172.21.15.16:0/4289367330 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febdc012aa0 con 0x55f14e647b60 2024-08-31T06:08:05.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.597+0000 7febc3e00640 1 -- 172.21.15.16:0/4289367330 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7febd0093170 con 0x55f14e647b60 2024-08-31T06:08:05.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.598+0000 7febf0244740 10 client.15687 fetch_fsmap finished waiting for FSMap version 30 2024-08-31T06:08:05.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.598+0000 7febf0244740 10 client.15687 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:08:05.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.598+0000 7febf0244740 20 client.15687 populate_metadata read hostname 'smithi016' 2024-08-31T06:08:05.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.598+0000 7febf0244740 1 --2- 172.21.15.16:0/4289367330 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f14e6271c0 0x55f14ea2cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:05.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.598+0000 7febf0244740 4 client.15687 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:08:05.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.598+0000 7febf0244740 1 -- 172.21.15.16:0/4289367330 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f14e986480 con 0x55f14e6271c0 2024-08-31T06:08:05.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.600+0000 7febe2a00640 1 --2- 172.21.15.16:0/4289367330 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f14e6271c0 0x55f14ea2cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:05.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.601+0000 7febe2a00640 1 --2- 172.21.15.16:0/4289367330 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f14e6271c0 0x55f14ea2cfd0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d4ebabdc2529b3b server_cookie=a81cdd0b5d773fcf in_seq=0 out_seq=0 2024-08-31T06:08:05.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.601+0000 7febc3e00640 10 client.15687 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:05.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.602+0000 7febc3e00640 1 -- 172.21.15.16:0/4289367330 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f14e986480 con 0x55f14e6271c0 2024-08-31T06:08:05.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.602+0000 7febc3e00640 10 client.15687 handle_command_reply: tid=0 2024-08-31T06:08:05.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.603+0000 7febf0244740 1 client.15687 shutdown 2024-08-31T06:08:05.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.603+0000 7febf0244740 1 -- 172.21.15.16:0/4289367330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f14e647b60 msgr2=0x55f14ea1b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:05.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.603+0000 7febf0244740 1 --2- 172.21.15.16:0/4289367330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f14e647b60 0x55f14ea1b2e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7febd00abe00 tx=0x7febd0010870 comp rx=0 tx=0).stop 2024-08-31T06:08:05.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.603+0000 7febf0244740 1 -- 172.21.15.16:0/4289367330 shutdown_connections 2024-08-31T06:08:05.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.603+0000 7febf0244740 1 --2- 172.21.15.16:0/4289367330 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f14e6271c0 0x55f14ea2cfd0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:05.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.603+0000 7febf0244740 1 --2- 172.21.15.16:0/4289367330 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f14e668840 0x55f14ea26c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:05.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.603+0000 7febf0244740 1 --2- 172.21.15.16:0/4289367330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f14e647b60 0x55f14ea1b2e0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:05.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.603+0000 7febf0244740 1 --2- 172.21.15.16:0/4289367330 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f14e63f980 0x55f14ea1ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:05.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.603+0000 7febc3e00640 0 client.15687 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:05.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.603+0000 7febf0244740 1 -- 172.21.15.16:0/4289367330 >> 172.21.15.16:0/4289367330 conn(0x55f14e65c990 msgr2=0x55f14ea0b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:05.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.604+0000 7febf0244740 1 -- 172.21.15.16:0/4289367330 shutdown_connections 2024-08-31T06:08:05.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.604+0000 7febf0244740 1 -- 172.21.15.16:0/4289367330 wait complete. 2024-08-31T06:08:05.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.604+0000 7febf0244740 20 client.15687 trim_cache size 0 max 16384 2024-08-31T06:08:05.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.636+0000 7febf0244740 1 Processor -- start 2024-08-31T06:08:05.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.636+0000 7febf0244740 1 -- start start 2024-08-31T06:08:05.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.636+0000 7febf0244740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f14e63f980 0x55f14ea14930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:05.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.636+0000 7febf0244740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f14e647b60 0x55f14ea26b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:05.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.636+0000 7febf0244740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f14e668840 0x55f14ea0c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:05.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.636+0000 7febf0244740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7febe41b83b0 con 0x55f14e63f980 2024-08-31T06:08:05.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.636+0000 7febf0244740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7febe41b5bf0 con 0x55f14e647b60 2024-08-31T06:08:05.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.636+0000 7febf0244740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7febe41b3a40 con 0x55f14e668840 2024-08-31T06:08:05.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.636+0000 7febe2000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f14e647b60 0x55f14ea26b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:05.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.637+0000 7febe2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f14e63f980 0x55f14ea14930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:05.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.637+0000 7febe2000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f14e647b60 0x55f14ea26b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:38070/0 (socket says 172.21.15.16:38070) 2024-08-31T06:08:05.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.637+0000 7febe2000640 1 -- 172.21.15.16:0/661026096 learned_addr learned my addr 172.21.15.16:0/661026096 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:05.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.637+0000 7febe3400640 1 --2- 172.21.15.16:0/661026096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f14e668840 0x55f14ea0c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:05.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.637+0000 7febe2000640 1 -- 172.21.15.16:0/661026096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f14e668840 msgr2=0x55f14ea0c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:05.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.637+0000 7febe2000640 1 --2- 172.21.15.16:0/661026096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f14e668840 0x55f14ea0c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:05.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.637+0000 7febe2000640 1 -- 172.21.15.16:0/661026096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f14e63f980 msgr2=0x55f14ea14930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:05.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.637+0000 7febe2000640 1 --2- 172.21.15.16:0/661026096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f14e63f980 0x55f14ea14930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:05.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.637+0000 7febe2000640 1 -- 172.21.15.16:0/661026096 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f14ea163d0 con 0x55f14e647b60 2024-08-31T06:08:05.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.637+0000 7febe2a00640 1 --2- 172.21.15.16:0/661026096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f14e63f980 0x55f14ea14930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:08:05.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.638+0000 7febe2000640 1 --2- 172.21.15.16:0/661026096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f14e647b60 0x55f14ea26b10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7febd00a90e0 tx=0x7febd00abff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bd699adfcae2d28 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:05.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.639+0000 7febcea00640 1 -- 172.21.15.16:0/661026096 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd00abc40 con 0x55f14e647b60 2024-08-31T06:08:05.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.639+0000 7febcea00640 1 -- 172.21.15.16:0/661026096 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febd0097170 con 0x55f14e647b60 2024-08-31T06:08:05.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.639+0000 7febcea00640 1 -- 172.21.15.16:0/661026096 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd00aab40 con 0x55f14e647b60 2024-08-31T06:08:05.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.640+0000 7febf0244740 1 -- 172.21.15.16:0/661026096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f14e647b60 msgr2=0x55f14ea26b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:05.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.640+0000 7febf0244740 1 --2- 172.21.15.16:0/661026096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f14e647b60 0x55f14ea26b10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7febd00a90e0 tx=0x7febd00abff0 comp rx=0 tx=0).stop 2024-08-31T06:08:05.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.641+0000 7febf0244740 1 -- 172.21.15.16:0/661026096 shutdown_connections 2024-08-31T06:08:05.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.641+0000 7febf0244740 1 --2- 172.21.15.16:0/661026096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f14e668840 0x55f14ea0c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:05.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.641+0000 7febf0244740 1 --2- 172.21.15.16:0/661026096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f14e647b60 0x55f14ea26b10 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:05.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.641+0000 7febf0244740 1 --2- 172.21.15.16:0/661026096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f14e63f980 0x55f14ea14930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:05.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.641+0000 7febf0244740 1 -- 172.21.15.16:0/661026096 >> 172.21.15.16:0/661026096 conn(0x55f14e65c990 msgr2=0x55f14ea14220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:05.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.642+0000 7febf0244740 1 -- 172.21.15.16:0/661026096 shutdown_connections 2024-08-31T06:08:05.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.642+0000 7febf0244740 1 -- 172.21.15.16:0/661026096 wait complete. 2024-08-31T06:08:05.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.644+0000 7febf0244740 1 -- start start 2024-08-31T06:08:05.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.644+0000 7febf0244740 1 Processor -- start 2024-08-31T06:08:05.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.645+0000 7febf0244740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f14e63f980 0x55f14ea22f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:05.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.645+0000 7febf0244740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f14e647b60 0x55f14ea23530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:05.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.645+0000 7febf0244740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f14e668840 0x55f14ea91370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:05.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.645+0000 7febf0244740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7febe41b40b0 con 0x55f14e63f980 2024-08-31T06:08:05.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.645+0000 7febf0244740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7febe41b5520 con 0x55f14e647b60 2024-08-31T06:08:05.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.645+0000 7febf0244740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7febe41b4780 con 0x55f14e668840 2024-08-31T06:08:05.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.645+0000 7febf0244740 10 client.0 _ll_register_callbacks cb 0x7febd00ab600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:08:05.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.646+0000 7febe2000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f14e647b60 0x55f14ea23530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:05.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.646+0000 7febe2000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f14e647b60 0x55f14ea23530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:38086/0 (socket says 172.21.15.16:38086) 2024-08-31T06:08:05.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.646+0000 7febe2000640 1 -- 172.21.15.16:0/349678537 learned_addr learned my addr 172.21.15.16:0/349678537 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:05.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.646+0000 7febe2a00640 1 --2- 172.21.15.16:0/349678537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f14e63f980 0x55f14ea22f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:05.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.646+0000 7febe3400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f14e668840 0x55f14ea91370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:05.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.648+0000 7febe2000640 1 -- 172.21.15.16:0/349678537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f14e668840 msgr2=0x55f14ea91370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:05.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.648+0000 7febe2000640 1 --2- 172.21.15.16:0/349678537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f14e668840 0x55f14ea91370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:05.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.648+0000 7febe2000640 1 -- 172.21.15.16:0/349678537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f14e63f980 msgr2=0x55f14ea22f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:05.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.648+0000 7febe2000640 1 --2- 172.21.15.16:0/349678537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f14e63f980 0x55f14ea22f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:05.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.648+0000 7febe2000640 1 -- 172.21.15.16:0/349678537 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f14ea2d340 con 0x55f14e647b60 2024-08-31T06:08:05.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.648+0000 7febe3400640 1 --2- 172.21.15.16:0/349678537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f14e668840 0x55f14ea91370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:08:05.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.648+0000 7febe2a00640 1 --2- 172.21.15.16:0/349678537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f14e63f980 0x55f14ea22f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:08:05.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.649+0000 7febe2000640 1 --2- 172.21.15.16:0/349678537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f14e647b60 0x55f14ea23530 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7febd0009230 tx=0x7febd001f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d96b573546ab0579 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:05.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.649+0000 7febc3e00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:08:05.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.649+0000 7febc3e00640 1 -- 172.21.15.16:0/349678537 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd00abc40 con 0x55f14e647b60 2024-08-31T06:08:05.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.649+0000 7febc3e00640 1 -- 172.21.15.16:0/349678537 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7febd00a8100 con 0x55f14e647b60 2024-08-31T06:08:05.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.649+0000 7febf0244740 1 -- 172.21.15.16:0/349678537 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febd0097170 con 0x55f14e647b60 2024-08-31T06:08:05.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.650+0000 7febc3e00640 1 -- 172.21.15.16:0/349678537 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7febd00cd2d0 con 0x55f14e647b60 2024-08-31T06:08:05.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.650+0000 7febf0244740 10 client.25733 fetch_fsmap learned FSMap version 30 2024-08-31T06:08:05.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.650+0000 7febf0244740 1 -- 172.21.15.16:0/349678537 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f14ea2d340 con 0x55f14e647b60 2024-08-31T06:08:05.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.650+0000 7febc3e00640 1 -- 172.21.15.16:0/349678537 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febd0097170 con 0x55f14e647b60 2024-08-31T06:08:05.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.650+0000 7febc3e00640 1 -- 172.21.15.16:0/349678537 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7febd00b1900 con 0x55f14e647b60 2024-08-31T06:08:05.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.651+0000 7febf0244740 10 client.25733 fetch_fsmap finished waiting for FSMap version 30 2024-08-31T06:08:05.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.651+0000 7febf0244740 10 client.25733 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:08:05.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.651+0000 7febf0244740 20 client.25733 populate_metadata read hostname 'smithi016' 2024-08-31T06:08:05.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.651+0000 7febf0244740 1 --2- 172.21.15.16:0/349678537 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f14e5fc570 0x55f14eaab6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:05.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.651+0000 7febf0244740 4 client.25733 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.4, "await_for": 45.0}] 2024-08-31T06:08:05.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.651+0000 7febf0244740 1 -- 172.21.15.16:0/349678537 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.4, "await_for": 45.0}) -- 0x55f14e981f20 con 0x55f14e5fc570 2024-08-31T06:08:05.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.652+0000 7febe2a00640 1 --2- 172.21.15.16:0/349678537 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f14e5fc570 0x55f14eaab6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:05.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.655+0000 7febe2a00640 1 --2- 172.21.15.16:0/349678537 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f14e5fc570 0x55f14eaab6f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42f63d5d5c959f2a server_cookie=89bd57c6b4a84bb5 in_seq=0 out_seq=0 2024-08-31T06:08:05.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.655+0000 7febc3e00640 10 client.25733 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:05.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.883+0000 7febc3e00640 1 -- 172.21.15.16:0/349678537 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55f14e981f20 con 0x55f14e5fc570 2024-08-31T06:08:05.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.883+0000 7febc3e00640 10 client.25733 handle_command_reply: tid=0 2024-08-31T06:08:05.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.883+0000 7febf0244740 1 client.25733 shutdown 2024-08-31T06:08:05.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.883+0000 7febf0244740 1 -- 172.21.15.16:0/349678537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f14e647b60 msgr2=0x55f14ea23530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:05.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.883+0000 7febf0244740 1 --2- 172.21.15.16:0/349678537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f14e647b60 0x55f14ea23530 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7febd0009230 tx=0x7febd001f000 comp rx=0 tx=0).stop 2024-08-31T06:08:05.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.883+0000 7febf0244740 1 -- 172.21.15.16:0/349678537 shutdown_connections 2024-08-31T06:08:05.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.883+0000 7febf0244740 1 --2- 172.21.15.16:0/349678537 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f14e5fc570 0x55f14eaab6f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:05.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.883+0000 7febf0244740 1 --2- 172.21.15.16:0/349678537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f14e668840 0x55f14ea91370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:05.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.883+0000 7febf0244740 1 --2- 172.21.15.16:0/349678537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f14e647b60 0x55f14ea23530 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:05.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.883+0000 7febf0244740 1 --2- 172.21.15.16:0/349678537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f14e63f980 0x55f14ea22f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:05.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.883+0000 7febc3e00640 0 client.25733 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:05.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.883+0000 7febf0244740 1 -- 172.21.15.16:0/349678537 >> 172.21.15.16:0/349678537 conn(0x55f14e65c990 msgr2=0x55f14ea0c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:05.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.883+0000 7febf0244740 1 -- 172.21.15.16:0/349678537 shutdown_connections 2024-08-31T06:08:05.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.883+0000 7febf0244740 1 -- 172.21.15.16:0/349678537 wait complete. 2024-08-31T06:08:05.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.883+0000 7febf0244740 20 client.25733 trim_cache size 0 max 16384 2024-08-31T06:08:05.887 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:08:05.887 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 30, 2024-08-31T06:08:05.887 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24737, 2024-08-31T06:08:05.887 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 6, 2024-08-31T06:08:05.888 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-08-31T06:08:05.888 INFO:teuthology.orchestra.run.smithi016.stdout: "a4339836": { 2024-08-31T06:08:05.888 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 6, 2024-08-31T06:08:05.888 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-08-31T06:08:05.888 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:08:05.888 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-08-31T06:08:05.888 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:08:05.888 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:08:05.888 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-08-31T06:08:05.888 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 146.4, 2024-08-31T06:08:05.888 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-08-31T06:08:05.888 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-08-31T06:08:05.888 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-08-31T06:08:05.889 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:08:05.889 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-08-31T06:08:05.889 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:08:05.889 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:08:05.889 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:08:05.889 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:08:05.889 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:08:05.889 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:08:05.889 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:08:05.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.884+0000 7febe9600640 1 -- 172.21.15.16:0/105672667 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7febc80b4d70 msgr2=0x7febc80b7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:05.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.884+0000 7febe9600640 1 --2- 172.21.15.16:0/105672667 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7febc80b4d70 0x7febc80b7280 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7febd4009830 tx=0x7febd4009390 comp rx=0 tx=0).stop 2024-08-31T06:08:05.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.884+0000 7febe9600640 1 -- 172.21.15.16:0/105672667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febe410b4a0 msgr2=0x7febe40776c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:05.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.884+0000 7febe9600640 1 --2- 172.21.15.16:0/105672667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febe410b4a0 0x7febe40776c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7febd000e9c0 tx=0x7febd000ee80 comp rx=0 tx=0).stop 2024-08-31T06:08:05.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.884+0000 7febe9600640 1 -- 172.21.15.16:0/105672667 shutdown_connections 2024-08-31T06:08:05.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.884+0000 7febe9600640 1 --2- 172.21.15.16:0/105672667 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7febc80b4d70 0x7febc80b7280 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:05.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.884+0000 7febe9600640 1 --2- 172.21.15.16:0/105672667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febe410bdf0 0x7febe4077de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:05.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.884+0000 7febe9600640 1 --2- 172.21.15.16:0/105672667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febe410b4a0 0x7febe40776c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:05.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.884+0000 7febe9600640 1 --2- 172.21.15.16:0/105672667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febe4105b60 0x7febe407b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:05.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.884+0000 7febe9600640 1 -- 172.21.15.16:0/105672667 >> 172.21.15.16:0/105672667 conn(0x7febe4101570 msgr2=0x7febe410aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:05.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.885+0000 7febe9600640 1 -- 172.21.15.16:0/105672667 shutdown_connections 2024-08-31T06:08:05.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:05.885+0000 7febe9600640 1 -- 172.21.15.16:0/105672667 wait complete. 2024-08-31T06:08:06.625 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 40.0 s, 796.9 tps, lat 106.459 ms stddev 227.544 2024-08-31T06:08:06.670 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a4339836', quiesce took 5.7 seconds. Will release after: 20.7 2024-08-31T06:08:07.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:07 smithi155 ceph-mon[21742]: pgmap v390: 97 pgs: 97 active+clean; 927 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 28 MiB/s wr, 453 op/s 2024-08-31T06:08:07.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:07 smithi016 ceph-mon[17194]: pgmap v390: 97 pgs: 97 active+clean; 927 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 28 MiB/s wr, 453 op/s 2024-08-31T06:08:07.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:07 smithi184 ceph-mon[22331]: pgmap v390: 97 pgs: 97 active+clean; 927 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 28 MiB/s wr, 453 op/s 2024-08-31T06:08:09.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:09 smithi155 ceph-mon[21742]: pgmap v391: 97 pgs: 97 active+clean; 931 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 MiB/s wr, 462 op/s 2024-08-31T06:08:09.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:09 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:08:09.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:09 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:08:09.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:09 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:08:09.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:09 smithi016 ceph-mon[17194]: pgmap v391: 97 pgs: 97 active+clean; 931 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 MiB/s wr, 462 op/s 2024-08-31T06:08:09.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:09 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:08:09.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:09 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:08:09.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:09 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:08:09.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:09 smithi184 ceph-mon[22331]: pgmap v391: 97 pgs: 97 active+clean; 931 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 MiB/s wr, 462 op/s 2024-08-31T06:08:09.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:09 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:08:09.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:09 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:08:09.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:09 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:08:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.195+0000 7f7dc4a00640 1 -- 172.21.15.16:0/1346318683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dc0109920 msgr2=0x7f7dc0110150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.195+0000 7f7dc4a00640 1 --2- 172.21.15.16:0/1346318683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dc0109920 0x7f7dc0110150 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7db8009ca0 tx=0x7f7db802a8c0 comp rx=0 tx=0).stop 2024-08-31T06:08:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.195+0000 7f7dc4a00640 1 -- 172.21.15.16:0/1346318683 shutdown_connections 2024-08-31T06:08:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.195+0000 7f7dc4a00640 1 --2- 172.21.15.16:0/1346318683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7dc0108fd0 0x7f7dc01093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.195+0000 7f7dc4a00640 1 --2- 172.21.15.16:0/1346318683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dc0109920 0x7f7dc0110150 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.195+0000 7f7dc4a00640 1 --2- 172.21.15.16:0/1346318683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7dc01036a0 0x7f7dc0103ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:11.198 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.195+0000 7f7dc4a00640 1 -- 172.21.15.16:0/1346318683 >> 172.21.15.16:0/1346318683 conn(0x7f7dc00ff100 msgr2=0x7f7dc0101550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:11.198 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.195+0000 7f7dc4a00640 1 -- 172.21.15.16:0/1346318683 shutdown_connections 2024-08-31T06:08:11.198 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.195+0000 7f7dc4a00640 1 -- 172.21.15.16:0/1346318683 wait complete. 2024-08-31T06:08:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.196+0000 7f7dc4a00640 1 Processor -- start 2024-08-31T06:08:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.197+0000 7f7dc4a00640 1 -- start start 2024-08-31T06:08:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.197+0000 7f7dc4a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7dc01036a0 0x7f7dc019c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.197+0000 7f7dc4a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dc0108fd0 0x7f7dc019c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.197+0000 7f7dc4a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7dc0109920 0x7f7dc01a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.197+0000 7f7dc4a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc01135d0 con 0x7f7dc0108fd0 2024-08-31T06:08:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.197+0000 7f7dc4a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc0113450 con 0x7f7dc0109920 2024-08-31T06:08:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.197+0000 7f7dc4a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc0113750 con 0x7f7dc01036a0 2024-08-31T06:08:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.197+0000 7f7db7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dc0108fd0 0x7f7dc019c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.197+0000 7f7dbf400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7dc01036a0 0x7f7dc019c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.197+0000 7f7dbfe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7dc0109920 0x7f7dc01a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.197+0000 7f7dbf400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7dc01036a0 0x7f7dc019c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:41880/0 (socket says 172.21.15.16:41880) 2024-08-31T06:08:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.197+0000 7f7dbf400640 1 -- 172.21.15.16:0/2060636558 learned_addr learned my addr 172.21.15.16:0/2060636558 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.198+0000 7f7dbf400640 1 -- 172.21.15.16:0/2060636558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7dc0109920 msgr2=0x7f7dc01a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.198+0000 7f7dbf400640 1 --2- 172.21.15.16:0/2060636558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7dc0109920 0x7f7dc01a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.198+0000 7f7dbf400640 1 -- 172.21.15.16:0/2060636558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dc0108fd0 msgr2=0x7f7dc019c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.198+0000 7f7dbf400640 1 --2- 172.21.15.16:0/2060636558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dc0108fd0 0x7f7dc019c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.198+0000 7f7dbf400640 1 -- 172.21.15.16:0/2060636558 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dc01a2140 con 0x7f7dc01036a0 2024-08-31T06:08:11.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.198+0000 7f7db7e00640 1 --2- 172.21.15.16:0/2060636558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dc0108fd0 0x7f7dc019c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:08:11.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.198+0000 7f7dbfe00640 1 --2- 172.21.15.16:0/2060636558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7dc0109920 0x7f7dc01a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:08:11.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.198+0000 7f7dbf400640 1 --2- 172.21.15.16:0/2060636558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7dc01036a0 0x7f7dc019c210 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7db000d950 tx=0x7f7db000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:11.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.198+0000 7f7dbcc00640 1 -- 172.21.15.16:0/2060636558 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db0017070 con 0x7f7dc01036a0 2024-08-31T06:08:11.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.198+0000 7f7dbcc00640 1 -- 172.21.15.16:0/2060636558 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7db0009c30 con 0x7f7dc01036a0 2024-08-31T06:08:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.198+0000 7f7dc4a00640 1 -- 172.21.15.16:0/2060636558 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dc01a2430 con 0x7f7dc01036a0 2024-08-31T06:08:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.198+0000 7f7dbcc00640 1 -- 172.21.15.16:0/2060636558 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db000f620 con 0x7f7dc01036a0 2024-08-31T06:08:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.198+0000 7f7dc4a00640 1 -- 172.21.15.16:0/2060636558 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dc01a8bc0 con 0x7f7dc01036a0 2024-08-31T06:08:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.201+0000 7f7dcd0ac740 1 Processor -- start 2024-08-31T06:08:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.202+0000 7f7dcd0ac740 1 -- start start 2024-08-31T06:08:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.202+0000 7f7dcd0ac740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561dc8af91b0 0x561dc8e9cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.202+0000 7f7dcd0ac740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dc8ad02f0 0x561dc8e9d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.202+0000 7f7dcd0ac740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc8ad84d0 0x561dc8e9d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.202+0000 7f7dcd0ac740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561dc8e11f70 con 0x561dc8ad02f0 2024-08-31T06:08:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.202+0000 7f7dcd0ac740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561dc8e01500 con 0x561dc8ad84d0 2024-08-31T06:08:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.202+0000 7f7dcd0ac740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x561dc8e054b0 con 0x561dc8af91b0 2024-08-31T06:08:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.202+0000 7f7dbcc00640 1 -- 172.21.15.16:0/2060636558 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f7db0010460 con 0x7f7dc01036a0 2024-08-31T06:08:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.202+0000 7f7dbfe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561dc8af91b0 0x561dc8e9cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.202+0000 7f7dbfe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561dc8af91b0 0x561dc8e9cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:41886/0 (socket says 172.21.15.16:41886) 2024-08-31T06:08:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.202+0000 7f7dbfe00640 1 -- 172.21.15.16:0/1535055108 learned_addr learned my addr 172.21.15.16:0/1535055108 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.203+0000 7f7db7e00640 1 --2- 172.21.15.16:0/1535055108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc8ad84d0 0x561dc8e9d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.203+0000 7f7dbf400640 1 --2- 172.21.15.16:0/1535055108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dc8ad02f0 0x561dc8e9d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.203+0000 7f7dbfe00640 1 -- 172.21.15.16:0/1535055108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc8ad84d0 msgr2=0x561dc8e9d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.203+0000 7f7dbfe00640 1 --2- 172.21.15.16:0/1535055108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc8ad84d0 0x561dc8e9d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.203+0000 7f7dbfe00640 1 -- 172.21.15.16:0/1535055108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dc8ad02f0 msgr2=0x561dc8e9d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.203+0000 7f7dbfe00640 1 --2- 172.21.15.16:0/1535055108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dc8ad02f0 0x561dc8e9d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.203+0000 7f7dbfe00640 1 -- 172.21.15.16:0/1535055108 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dc8dbbc10 con 0x561dc8af91b0 2024-08-31T06:08:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.203+0000 7f7dbf400640 1 --2- 172.21.15.16:0/1535055108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dc8ad02f0 0x561dc8e9d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:08:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.203+0000 7f7dbfe00640 1 --2- 172.21.15.16:0/1535055108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561dc8af91b0 0x561dc8e9cb00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7db802f430 tx=0x7f7db802f460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb52f613b9ea7d2f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.203+0000 7f7dafe00640 1 -- 172.21.15.16:0/1535055108 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db8031650 con 0x561dc8af91b0 2024-08-31T06:08:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.204+0000 7f7dafe00640 1 -- 172.21.15.16:0/1535055108 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7db8036040 con 0x561dc8af91b0 2024-08-31T06:08:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.204+0000 7f7dafe00640 1 -- 172.21.15.16:0/1535055108 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db8004600 con 0x561dc8af91b0 2024-08-31T06:08:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.204+0000 7f7dbcc00640 1 --2- 172.21.15.16:0/2060636558 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7da40b4ca0 0x7f7da40b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.204+0000 7f7dbcc00640 1 -- 172.21.15.16:0/2060636558 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f7db00e50f0 con 0x7f7dc01036a0 2024-08-31T06:08:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.204+0000 7f7dbf400640 1 --2- 172.21.15.16:0/2060636558 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7da40b4ca0 0x7f7da40b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.204+0000 7f7dcd0ac740 1 -- 172.21.15.16:0/1535055108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561dc8af91b0 msgr2=0x561dc8e9cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.204+0000 7f7dcd0ac740 1 --2- 172.21.15.16:0/1535055108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561dc8af91b0 0x561dc8e9cb00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7db802f430 tx=0x7f7db802f460 comp rx=0 tx=0).stop 2024-08-31T06:08:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.205+0000 7f7dcd0ac740 1 -- 172.21.15.16:0/1535055108 shutdown_connections 2024-08-31T06:08:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.205+0000 7f7dcd0ac740 1 --2- 172.21.15.16:0/1535055108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc8ad84d0 0x561dc8e9d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.205+0000 7f7dcd0ac740 1 --2- 172.21.15.16:0/1535055108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dc8ad02f0 0x561dc8e9d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.205+0000 7f7dcd0ac740 1 --2- 172.21.15.16:0/1535055108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561dc8af91b0 0x561dc8e9cb00 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.205+0000 7f7dcd0ac740 1 -- 172.21.15.16:0/1535055108 >> 172.21.15.16:0/1535055108 conn(0x561dc8aed300 msgr2=0x561dc8e9a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.205+0000 7f7dcd0ac740 1 -- 172.21.15.16:0/1535055108 shutdown_connections 2024-08-31T06:08:11.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.205+0000 7f7dcd0ac740 1 -- 172.21.15.16:0/1535055108 wait complete. 2024-08-31T06:08:11.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.205+0000 7f7dbf400640 1 --2- 172.21.15.16:0/2060636558 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7da40b4ca0 0x7f7da40b71b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x561dc8e9b780 tx=0x7f7db0000bc0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:11.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.205+0000 7f7dcd0ac740 1 -- start start 2024-08-31T06:08:11.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.206+0000 7f7dcd0ac740 1 Processor -- start 2024-08-31T06:08:11.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.206+0000 7f7dcd0ac740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dc8ad02f0 0x561dc8ea8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:11.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.206+0000 7f7dcd0ac740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561dc8ad84d0 0x561dc8ea8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:11.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.206+0000 7f7dcd0ac740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc8af91b0 0x561dc8ea8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:11.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.206+0000 7f7dcd0ac740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561dc8d3aab0 con 0x561dc8ad02f0 2024-08-31T06:08:11.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.206+0000 7f7dcd0ac740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561dc8da8320 con 0x561dc8af91b0 2024-08-31T06:08:11.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.206+0000 7f7dcd0ac740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x561dc8eab0a0 con 0x561dc8ad84d0 2024-08-31T06:08:11.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.206+0000 7f7dcd0ac740 10 client.0 _ll_register_callbacks cb 0x561dc8dae6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:08:11.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.206+0000 7f7dbfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dc8ad02f0 0x561dc8ea8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:11.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.206+0000 7f7db7e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561dc8ad84d0 0x561dc8ea8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.206+0000 7f7dbfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dc8ad02f0 0x561dc8ea8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60616/0 (socket says 172.21.15.16:60616) 2024-08-31T06:08:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.206+0000 7f7dbfe00640 1 -- 172.21.15.16:0/2078754816 learned_addr learned my addr 172.21.15.16:0/2078754816 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.207+0000 7f7dbfe00640 1 --2- 172.21.15.16:0/2078754816 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc8af91b0 0x561dc8ea8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.207+0000 7f7db7e00640 1 -- 172.21.15.16:0/2078754816 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc8af91b0 msgr2=0x561dc8ea8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.207+0000 7f7db7e00640 1 --2- 172.21.15.16:0/2078754816 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc8af91b0 0x561dc8ea8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.207+0000 7f7db7e00640 1 -- 172.21.15.16:0/2078754816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dc8ad02f0 msgr2=0x561dc8ea8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.207+0000 7f7db7e00640 1 --2- 172.21.15.16:0/2078754816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dc8ad02f0 0x561dc8ea8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.207+0000 7f7db7e00640 1 -- 172.21.15.16:0/2078754816 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dc8b64790 con 0x561dc8ad84d0 2024-08-31T06:08:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.207+0000 7f7dbfe00640 1 --2- 172.21.15.16:0/2078754816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dc8ad02f0 0x561dc8ea8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:08:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.207+0000 7f7db7e00640 1 --2- 172.21.15.16:0/2078754816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561dc8ad84d0 0x561dc8ea8650 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7da8005d00 tx=0x7f7da8011c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bc8547ccb2523c1 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.207+0000 7f7dacc00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:08:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.208+0000 7f7dacc00640 1 -- 172.21.15.16:0/2078754816 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da8010c30 con 0x561dc8ad84d0 2024-08-31T06:08:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.208+0000 7f7dcd0ac740 1 -- 172.21.15.16:0/2078754816 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7db8036040 con 0x561dc8ad84d0 2024-08-31T06:08:11.213 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.208+0000 7f7dacc00640 1 -- 172.21.15.16:0/2078754816 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f7da8010dd0 con 0x561dc8ad84d0 2024-08-31T06:08:11.213 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.208+0000 7f7dacc00640 1 -- 172.21.15.16:0/2078754816 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7da8004020 con 0x561dc8ad84d0 2024-08-31T06:08:11.213 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.208+0000 7f7dcd0ac740 10 client.25747 fetch_fsmap learned FSMap version 30 2024-08-31T06:08:11.213 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.208+0000 7f7dcd0ac740 1 -- 172.21.15.16:0/2078754816 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561dc8bdd620 con 0x561dc8ad84d0 2024-08-31T06:08:11.213 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.208+0000 7f7dacc00640 1 -- 172.21.15.16:0/2078754816 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7db8036040 con 0x561dc8ad84d0 2024-08-31T06:08:11.215 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.209+0000 7f7dacc00640 1 -- 172.21.15.16:0/2078754816 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f7da801a410 con 0x561dc8ad84d0 2024-08-31T06:08:11.215 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.209+0000 7f7dcd0ac740 10 client.25747 fetch_fsmap finished waiting for FSMap version 30 2024-08-31T06:08:11.215 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.209+0000 7f7dcd0ac740 10 client.25747 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:08:11.215 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.209+0000 7f7dcd0ac740 20 client.25747 populate_metadata read hostname 'smithi016' 2024-08-31T06:08:11.215 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.209+0000 7f7dcd0ac740 1 --2- 172.21.15.16:0/2078754816 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x561dc8ab7d80 0x561dc8eba140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:11.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.209+0000 7f7dcd0ac740 4 client.25747 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:08:11.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.209+0000 7f7dcd0ac740 1 -- 172.21.15.16:0/2078754816 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561dc8e13ec0 con 0x561dc8ab7d80 2024-08-31T06:08:11.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.209+0000 7f7dbfe00640 1 --2- 172.21.15.16:0/2078754816 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x561dc8ab7d80 0x561dc8eba140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:11.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.210+0000 7f7dbfe00640 1 --2- 172.21.15.16:0/2078754816 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x561dc8ab7d80 0x561dc8eba140 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d79296185344129 server_cookie=a1f4cac09485d076 in_seq=0 out_seq=0 2024-08-31T06:08:11.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.210+0000 7f7dacc00640 10 client.25747 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:11.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.211+0000 7f7dacc00640 1 -- 172.21.15.16:0/2078754816 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561dc8e13ec0 con 0x561dc8ab7d80 2024-08-31T06:08:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.211+0000 7f7dacc00640 10 client.25747 handle_command_reply: tid=0 2024-08-31T06:08:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.211+0000 7f7dcd0ac740 1 client.25747 shutdown 2024-08-31T06:08:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.211+0000 7f7dcd0ac740 1 -- 172.21.15.16:0/2078754816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561dc8ad84d0 msgr2=0x561dc8ea8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.211+0000 7f7dcd0ac740 1 --2- 172.21.15.16:0/2078754816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561dc8ad84d0 0x561dc8ea8650 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7da8005d00 tx=0x7f7da8011c90 comp rx=0 tx=0).stop 2024-08-31T06:08:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.211+0000 7f7dcd0ac740 1 -- 172.21.15.16:0/2078754816 shutdown_connections 2024-08-31T06:08:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.211+0000 7f7dcd0ac740 1 --2- 172.21.15.16:0/2078754816 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x561dc8ab7d80 0x561dc8eba140 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.211+0000 7f7dcd0ac740 1 --2- 172.21.15.16:0/2078754816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561dc8ad84d0 0x561dc8ea8650 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.211+0000 7f7dcd0ac740 1 --2- 172.21.15.16:0/2078754816 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc8af91b0 0x561dc8ea8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.211+0000 7f7dcd0ac740 1 --2- 172.21.15.16:0/2078754816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dc8ad02f0 0x561dc8ea8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.211+0000 7f7dacc00640 0 client.25747 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.211+0000 7f7dcd0ac740 1 -- 172.21.15.16:0/2078754816 >> 172.21.15.16:0/2078754816 conn(0x561dc8aed300 msgr2=0x561dc8e9db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.211+0000 7f7dcd0ac740 1 -- 172.21.15.16:0/2078754816 shutdown_connections 2024-08-31T06:08:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.212+0000 7f7dcd0ac740 1 -- 172.21.15.16:0/2078754816 wait complete. 2024-08-31T06:08:11.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.212+0000 7f7dcd0ac740 20 client.25747 trim_cache size 0 max 16384 2024-08-31T06:08:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.227+0000 7f7dcd0ac740 1 Processor -- start 2024-08-31T06:08:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.227+0000 7f7dcd0ac740 1 -- start start 2024-08-31T06:08:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.227+0000 7f7dcd0ac740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dc8ad02f0 0x561dc8e9e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.227+0000 7f7dcd0ac740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc8ad84d0 0x561dc8ea9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.227+0000 7f7dcd0ac740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561dc8af91b0 0x561dc8ea52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.227+0000 7f7dcd0ac740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc01b3ce0 con 0x561dc8ad02f0 2024-08-31T06:08:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.227+0000 7f7dcd0ac740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc01b1520 con 0x561dc8ad84d0 2024-08-31T06:08:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.227+0000 7f7dcd0ac740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc01af370 con 0x561dc8af91b0 2024-08-31T06:08:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.228+0000 7f7dbfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dc8ad02f0 0x561dc8e9e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.228+0000 7f7dbfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dc8ad02f0 0x561dc8e9e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60618/0 (socket says 172.21.15.16:60618) 2024-08-31T06:08:11.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.228+0000 7f7dbfe00640 1 -- 172.21.15.16:0/2158744947 learned_addr learned my addr 172.21.15.16:0/2158744947 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:11.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.228+0000 7f7db7e00640 1 --2- 172.21.15.16:0/2158744947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc8ad84d0 0x561dc8ea9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:11.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.228+0000 7f7dbfe00640 1 --2- 172.21.15.16:0/2158744947 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561dc8af91b0 0x561dc8ea52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:11.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.228+0000 7f7dbfe00640 1 -- 172.21.15.16:0/2158744947 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561dc8af91b0 msgr2=0x561dc8ea52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:11.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.228+0000 7f7dbfe00640 1 --2- 172.21.15.16:0/2158744947 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561dc8af91b0 0x561dc8ea52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:11.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.228+0000 7f7dbfe00640 1 -- 172.21.15.16:0/2158744947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc8ad84d0 msgr2=0x561dc8ea9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:11.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.228+0000 7f7dbfe00640 1 --2- 172.21.15.16:0/2158744947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc8ad84d0 0x561dc8ea9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:11.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.228+0000 7f7dbfe00640 1 -- 172.21.15.16:0/2158744947 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dc8eaac50 con 0x561dc8ad02f0 2024-08-31T06:08:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.228+0000 7f7db7e00640 1 --2- 172.21.15.16:0/2158744947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc8ad84d0 0x561dc8ea9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:08:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.228+0000 7f7dbfe00640 1 --2- 172.21.15.16:0/2158744947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dc8ad02f0 0x561dc8e9e4f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7db8009ca0 tx=0x7f7db8004260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2b393376c4066fa server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.229+0000 7f7daf400640 1 -- 172.21.15.16:0/2158744947 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db805f070 con 0x561dc8ad02f0 2024-08-31T06:08:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.229+0000 7f7daf400640 1 -- 172.21.15.16:0/2158744947 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7db8036070 con 0x561dc8ad02f0 2024-08-31T06:08:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.229+0000 7f7daf400640 1 -- 172.21.15.16:0/2158744947 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db804b040 con 0x561dc8ad02f0 2024-08-31T06:08:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.229+0000 7f7dcd0ac740 1 -- 172.21.15.16:0/2158744947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dc8ad02f0 msgr2=0x561dc8e9e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.229+0000 7f7dcd0ac740 1 --2- 172.21.15.16:0/2158744947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dc8ad02f0 0x561dc8e9e4f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7db8009ca0 tx=0x7f7db8004260 comp rx=0 tx=0).stop 2024-08-31T06:08:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.229+0000 7f7dcd0ac740 1 -- 172.21.15.16:0/2158744947 shutdown_connections 2024-08-31T06:08:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.229+0000 7f7dcd0ac740 1 --2- 172.21.15.16:0/2158744947 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561dc8af91b0 0x561dc8ea52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.229+0000 7f7dcd0ac740 1 --2- 172.21.15.16:0/2158744947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc8ad84d0 0x561dc8ea9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.229+0000 7f7dcd0ac740 1 --2- 172.21.15.16:0/2158744947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dc8ad02f0 0x561dc8e9e4f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.229+0000 7f7dcd0ac740 1 -- 172.21.15.16:0/2158744947 >> 172.21.15.16:0/2158744947 conn(0x561dc8aed300 msgr2=0x561dc8ea9630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.229+0000 7f7dcd0ac740 1 -- 172.21.15.16:0/2158744947 shutdown_connections 2024-08-31T06:08:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.230+0000 7f7dcd0ac740 1 -- 172.21.15.16:0/2158744947 wait complete. 2024-08-31T06:08:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.230+0000 7f7dcd0ac740 1 -- start start 2024-08-31T06:08:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.230+0000 7f7dcd0ac740 1 Processor -- start 2024-08-31T06:08:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.231+0000 7f7dcd0ac740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561dc8ad02f0 0x561dc8eb0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.231+0000 7f7dcd0ac740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dc8ad84d0 0x561dc8eae170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.231+0000 7f7dcd0ac740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc8af91b0 0x561dc8eae880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.231+0000 7f7dcd0ac740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc01af9e0 con 0x561dc8ad84d0 2024-08-31T06:08:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.231+0000 7f7dcd0ac740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc01b0e50 con 0x561dc8af91b0 2024-08-31T06:08:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.231+0000 7f7dcd0ac740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc01b00b0 con 0x561dc8ad02f0 2024-08-31T06:08:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.231+0000 7f7dcd0ac740 10 client.0 _ll_register_callbacks cb 0x7f7da8011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:08:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.231+0000 7f7db7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dc8ad84d0 0x561dc8eae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.231+0000 7f7db7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dc8ad84d0 0x561dc8eae170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60628/0 (socket says 172.21.15.16:60628) 2024-08-31T06:08:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.231+0000 7f7db7e00640 1 -- 172.21.15.16:0/1366573594 learned_addr learned my addr 172.21.15.16:0/1366573594 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.231+0000 7f7dbfe00640 1 --2- 172.21.15.16:0/1366573594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561dc8ad02f0 0x561dc8eb0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.231+0000 7f7dbfe00640 1 --2- 172.21.15.16:0/1366573594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc8af91b0 0x561dc8eae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.231+0000 7f7db7e00640 1 -- 172.21.15.16:0/1366573594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561dc8ad02f0 msgr2=0x561dc8eb0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.231+0000 7f7db7e00640 1 --2- 172.21.15.16:0/1366573594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561dc8ad02f0 0x561dc8eb0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.231+0000 7f7db7e00640 1 -- 172.21.15.16:0/1366573594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc8af91b0 msgr2=0x561dc8eae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.231+0000 7f7db7e00640 1 --2- 172.21.15.16:0/1366573594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc8af91b0 0x561dc8eae880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.231+0000 7f7db7e00640 1 -- 172.21.15.16:0/1366573594 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dc8f23b10 con 0x561dc8ad84d0 2024-08-31T06:08:11.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.232+0000 7f7db7e00640 1 --2- 172.21.15.16:0/1366573594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dc8ad84d0 0x561dc8eae170 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7da801aa20 tx=0x7f7da800beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28de38da8754ebe1 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:11.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.232+0000 7f7dacc00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:08:11.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.232+0000 7f7dacc00640 1 -- 172.21.15.16:0/1366573594 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da8030070 con 0x561dc8ad84d0 2024-08-31T06:08:11.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.232+0000 7f7dacc00640 1 -- 172.21.15.16:0/1366573594 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f7da8010e40 con 0x561dc8ad84d0 2024-08-31T06:08:11.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.232+0000 7f7dcd0ac740 1 -- 172.21.15.16:0/1366573594 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7db8036070 con 0x561dc8ad84d0 2024-08-31T06:08:11.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.232+0000 7f7dacc00640 1 -- 172.21.15.16:0/1366573594 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7da803c050 con 0x561dc8ad84d0 2024-08-31T06:08:11.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.232+0000 7f7dcd0ac740 10 client.15711 fetch_fsmap learned FSMap version 30 2024-08-31T06:08:11.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.232+0000 7f7dcd0ac740 1 -- 172.21.15.16:0/1366573594 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561dc8f24c70 con 0x561dc8ad84d0 2024-08-31T06:08:11.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.233+0000 7f7dacc00640 1 -- 172.21.15.16:0/1366573594 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7db8036070 con 0x561dc8ad84d0 2024-08-31T06:08:11.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.233+0000 7f7dacc00640 1 -- 172.21.15.16:0/1366573594 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f7da8004660 con 0x561dc8ad84d0 2024-08-31T06:08:11.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.233+0000 7f7dcd0ac740 10 client.15711 fetch_fsmap finished waiting for FSMap version 30 2024-08-31T06:08:11.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.233+0000 7f7dcd0ac740 10 client.15711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:08:11.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.233+0000 7f7dcd0ac740 20 client.15711 populate_metadata read hostname 'smithi016' 2024-08-31T06:08:11.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.233+0000 7f7dcd0ac740 1 --2- 172.21.15.16:0/1366573594 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x561dc8a8d1c0 0x561dc8f274f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:11.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.233+0000 7f7dcd0ac740 4 client.15711 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:08:11.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.233+0000 7f7dcd0ac740 1 -- 172.21.15.16:0/1366573594 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561dc8e0f960 con 0x561dc8a8d1c0 2024-08-31T06:08:11.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.233+0000 7f7dbfe00640 1 --2- 172.21.15.16:0/1366573594 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x561dc8a8d1c0 0x561dc8f274f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:11.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.234+0000 7f7dbfe00640 1 --2- 172.21.15.16:0/1366573594 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x561dc8a8d1c0 0x561dc8f274f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb4097da8e64781b server_cookie=1d93655179745e0c in_seq=0 out_seq=0 2024-08-31T06:08:11.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.234+0000 7f7dacc00640 10 client.15711 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.251+0000 7f7dacc00640 1 -- 172.21.15.16:0/1366573594 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561dc8e0f960 con 0x561dc8a8d1c0 2024-08-31T06:08:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.251+0000 7f7dacc00640 10 client.15711 handle_command_reply: tid=0 2024-08-31T06:08:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.251+0000 7f7dcd0ac740 1 client.15711 shutdown 2024-08-31T06:08:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.251+0000 7f7dcd0ac740 1 -- 172.21.15.16:0/1366573594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dc8ad84d0 msgr2=0x561dc8eae170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.251+0000 7f7dcd0ac740 1 --2- 172.21.15.16:0/1366573594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dc8ad84d0 0x561dc8eae170 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7da801aa20 tx=0x7f7da800beb0 comp rx=0 tx=0).stop 2024-08-31T06:08:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.251+0000 7f7dcd0ac740 1 -- 172.21.15.16:0/1366573594 shutdown_connections 2024-08-31T06:08:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.251+0000 7f7dcd0ac740 1 --2- 172.21.15.16:0/1366573594 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x561dc8a8d1c0 0x561dc8f274f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.251+0000 7f7dcd0ac740 1 --2- 172.21.15.16:0/1366573594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dc8af91b0 0x561dc8eae880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.251+0000 7f7dcd0ac740 1 --2- 172.21.15.16:0/1366573594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dc8ad84d0 0x561dc8eae170 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.251+0000 7f7dacc00640 0 client.15711 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.251+0000 7f7dcd0ac740 1 --2- 172.21.15.16:0/1366573594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561dc8ad02f0 0x561dc8eb0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.251+0000 7f7dcd0ac740 1 -- 172.21.15.16:0/1366573594 >> 172.21.15.16:0/1366573594 conn(0x561dc8aed300 msgr2=0x561dc8e9e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.252+0000 7f7dcd0ac740 1 -- 172.21.15.16:0/1366573594 shutdown_connections 2024-08-31T06:08:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.252+0000 7f7dcd0ac740 1 -- 172.21.15.16:0/1366573594 wait complete. 2024-08-31T06:08:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.252+0000 7f7dcd0ac740 20 client.15711 trim_cache size 0 max 16384 2024-08-31T06:08:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:08:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:08:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:08:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:08:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.252+0000 7f7dc4a00640 1 -- 172.21.15.16:0/2060636558 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7da40b4ca0 msgr2=0x7f7da40b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.252+0000 7f7dc4a00640 1 --2- 172.21.15.16:0/2060636558 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7da40b4ca0 0x7f7da40b71b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x561dc8e9b780 tx=0x7f7db0000bc0 comp rx=0 tx=0).stop 2024-08-31T06:08:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.252+0000 7f7dc4a00640 1 -- 172.21.15.16:0/2060636558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7dc01036a0 msgr2=0x7f7dc019c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.252+0000 7f7dc4a00640 1 --2- 172.21.15.16:0/2060636558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7dc01036a0 0x7f7dc019c210 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7db000d950 tx=0x7f7db000de10 comp rx=0 tx=0).stop 2024-08-31T06:08:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.253+0000 7f7dc4a00640 1 -- 172.21.15.16:0/2060636558 shutdown_connections 2024-08-31T06:08:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.253+0000 7f7dc4a00640 1 --2- 172.21.15.16:0/2060636558 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7da40b4ca0 0x7f7da40b71b0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.253+0000 7f7dc4a00640 1 --2- 172.21.15.16:0/2060636558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7dc0109920 0x7f7dc01a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:11.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.253+0000 7f7dc4a00640 1 --2- 172.21.15.16:0/2060636558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dc0108fd0 0x7f7dc019c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:11.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.253+0000 7f7dc4a00640 1 --2- 172.21.15.16:0/2060636558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7dc01036a0 0x7f7dc019c210 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:11.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.253+0000 7f7dc4a00640 1 -- 172.21.15.16:0/2060636558 >> 172.21.15.16:0/2060636558 conn(0x7f7dc00ff100 msgr2=0x7f7dc0107630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:11.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.253+0000 7f7dc4a00640 1 -- 172.21.15.16:0/2060636558 shutdown_connections 2024-08-31T06:08:11.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:11.253+0000 7f7dc4a00640 1 -- 172.21.15.16:0/2060636558 wait complete. 2024-08-31T06:08:11.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:11 smithi155 ceph-mon[21742]: pgmap v392: 97 pgs: 97 active+clean; 938 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 36 MiB/s wr, 502 op/s 2024-08-31T06:08:11.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:11 smithi016 ceph-mon[17194]: pgmap v392: 97 pgs: 97 active+clean; 938 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 36 MiB/s wr, 502 op/s 2024-08-31T06:08:11.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:11 smithi184 ceph-mon[22331]: pgmap v392: 97 pgs: 97 active+clean; 938 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 36 MiB/s wr, 502 op/s 2024-08-31T06:08:11.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 45.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-31T06:08:12.230 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T06:08:12.623 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:08:13.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.300+0000 7f144aa00640 1 -- 172.21.15.16:0/1197608815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1444109240 msgr2=0x7f1444109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:13.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.300+0000 7f144aa00640 1 --2- 172.21.15.16:0/1197608815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1444109240 0x7f1444109650 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f142c009900 tx=0x7f142c02a410 comp rx=0 tx=0).stop 2024-08-31T06:08:13.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.300+0000 7f144aa00640 1 -- 172.21.15.16:0/1197608815 shutdown_connections 2024-08-31T06:08:13.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.300+0000 7f144aa00640 1 --2- 172.21.15.16:0/1197608815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1444109b90 0x7f14441103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:13.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.300+0000 7f144aa00640 1 --2- 172.21.15.16:0/1197608815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1444109240 0x7f1444109650 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:13.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.300+0000 7f144aa00640 1 --2- 172.21.15.16:0/1197608815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1444103900 0x7f1444103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:13.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.300+0000 7f144aa00640 1 -- 172.21.15.16:0/1197608815 >> 172.21.15.16:0/1197608815 conn(0x7f144407b5f0 msgr2=0x7f144407ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:13.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.300+0000 7f144aa00640 1 -- 172.21.15.16:0/1197608815 shutdown_connections 2024-08-31T06:08:13.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.300+0000 7f144aa00640 1 -- 172.21.15.16:0/1197608815 wait complete. 2024-08-31T06:08:13.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.301+0000 7f144aa00640 1 Processor -- start 2024-08-31T06:08:13.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.301+0000 7f144aa00640 1 -- start start 2024-08-31T06:08:13.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.302+0000 7f144aa00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1444103900 0x7f144419c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:13.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.302+0000 7f144aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1444109240 0x7f144419ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:13.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.302+0000 7f144aa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1444109b90 0x7f14441a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:13.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.302+0000 7f144aa00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f14441138a0 con 0x7f1444109240 2024-08-31T06:08:13.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.302+0000 7f144aa00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1444113720 con 0x7f1444109b90 2024-08-31T06:08:13.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.302+0000 7f144aa00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1444113a20 con 0x7f1444103900 2024-08-31T06:08:13.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.302+0000 7f143be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1444109240 0x7f144419ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:13.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.302+0000 7f1448a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1444109b90 0x7f14441a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:13.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.302+0000 7f1443e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1444103900 0x7f144419c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:13.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.302+0000 7f1448a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1444109b90 0x7f14441a1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:38738/0 (socket says 172.21.15.16:38738) 2024-08-31T06:08:13.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.302+0000 7f143be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1444109240 0x7f144419ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60648/0 (socket says 172.21.15.16:60648) 2024-08-31T06:08:13.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.302+0000 7f1448a00640 1 -- 172.21.15.16:0/1596998406 learned_addr learned my addr 172.21.15.16:0/1596998406 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:13.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.303+0000 7f1448a00640 1 -- 172.21.15.16:0/1596998406 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1444103900 msgr2=0x7f144419c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:13.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.303+0000 7f1448a00640 1 --2- 172.21.15.16:0/1596998406 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1444103900 0x7f144419c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:13.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.303+0000 7f1448a00640 1 -- 172.21.15.16:0/1596998406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1444109240 msgr2=0x7f144419ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:13.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.303+0000 7f1448a00640 1 --2- 172.21.15.16:0/1596998406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1444109240 0x7f144419ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:13.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.303+0000 7f1448a00640 1 -- 172.21.15.16:0/1596998406 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14441a2440 con 0x7f1444109b90 2024-08-31T06:08:13.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.303+0000 7f143be00640 1 --2- 172.21.15.16:0/1596998406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1444109240 0x7f144419ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:08:13.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.303+0000 7f1448a00640 1 --2- 172.21.15.16:0/1596998406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1444109b90 0x7f14441a1d10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f143c00eb10 tx=0x7f143c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:13.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.303+0000 7f1441600640 1 -- 172.21.15.16:0/1596998406 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c004b20 con 0x7f1444109b90 2024-08-31T06:08:13.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.303+0000 7f144aa00640 1 -- 172.21.15.16:0/1596998406 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14441a2730 con 0x7f1444109b90 2024-08-31T06:08:13.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.303+0000 7f1441600640 1 -- 172.21.15.16:0/1596998406 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f143c004cc0 con 0x7f1444109b90 2024-08-31T06:08:13.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.303+0000 7f1441600640 1 -- 172.21.15.16:0/1596998406 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c010670 con 0x7f1444109b90 2024-08-31T06:08:13.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.303+0000 7f144aa00640 1 -- 172.21.15.16:0/1596998406 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14441a8e90 con 0x7f1444109b90 2024-08-31T06:08:13.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.307+0000 7f1451648740 1 Processor -- start 2024-08-31T06:08:13.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.308+0000 7f1441600640 1 -- 172.21.15.16:0/1596998406 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f143c005020 con 0x7f1444109b90 2024-08-31T06:08:13.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.308+0000 7f1451648740 1 -- start start 2024-08-31T06:08:13.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.309+0000 7f1451648740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650778db070 0x565077c80950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:13.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.309+0000 7f1451648740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650778b21b0 0x565077c80e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:13.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.309+0000 7f1451648740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650778ba390 0x565077c813d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:13.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.309+0000 7f1451648740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x565077bf7770 con 0x5650778b21b0 2024-08-31T06:08:13.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.309+0000 7f1451648740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x565077be6d00 con 0x5650778db070 2024-08-31T06:08:13.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.309+0000 7f1451648740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x565077beacb0 con 0x5650778ba390 2024-08-31T06:08:13.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.309+0000 7f1443e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650778b21b0 0x565077c80e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:13.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.309+0000 7f143be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650778db070 0x565077c80950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:13.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.309+0000 7f1443e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650778b21b0 0x565077c80e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60652/0 (socket says 172.21.15.16:60652) 2024-08-31T06:08:13.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.309+0000 7f1443e00640 1 -- 172.21.15.16:0/1132847080 learned_addr learned my addr 172.21.15.16:0/1132847080 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:13.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.309+0000 7f143be00640 1 --2- 172.21.15.16:0/1132847080 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650778ba390 0x565077c813d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:13.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.310+0000 7f1443e00640 1 -- 172.21.15.16:0/1132847080 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650778ba390 msgr2=0x565077c813d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:13.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.310+0000 7f1443e00640 1 --2- 172.21.15.16:0/1132847080 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650778ba390 0x565077c813d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:13.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.310+0000 7f1443e00640 1 -- 172.21.15.16:0/1132847080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650778db070 msgr2=0x565077c80950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:13.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.310+0000 7f1443e00640 1 --2- 172.21.15.16:0/1132847080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650778db070 0x565077c80950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:13.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.310+0000 7f1443e00640 1 -- 172.21.15.16:0/1132847080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565077946650 con 0x5650778b21b0 2024-08-31T06:08:13.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.310+0000 7f143be00640 1 --2- 172.21.15.16:0/1132847080 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650778ba390 0x565077c813d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:08:13.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.310+0000 7f1441600640 1 --2- 172.21.15.16:0/1596998406 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f14280b4ca0 0x7f14280b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:13.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.310+0000 7f1443e00640 1 --2- 172.21.15.16:0/1132847080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650778b21b0 0x565077c80e90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f143400bd10 tx=0x7f14340107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76c4b6c6fe666c98 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:13.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.310+0000 7f1441600640 1 -- 172.21.15.16:0/1596998406 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f143c0e5a70 con 0x7f1444109b90 2024-08-31T06:08:13.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.310+0000 7f1439200640 1 -- 172.21.15.16:0/1132847080 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1434004030 con 0x5650778b21b0 2024-08-31T06:08:13.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.310+0000 7f1439200640 1 -- 172.21.15.16:0/1132847080 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14340035f0 con 0x5650778b21b0 2024-08-31T06:08:13.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.310+0000 7f1439200640 1 -- 172.21.15.16:0/1132847080 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1434011750 con 0x5650778b21b0 2024-08-31T06:08:13.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.310+0000 7f143be00640 1 --2- 172.21.15.16:0/1596998406 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f14280b4ca0 0x7f14280b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:13.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.311+0000 7f1451648740 1 -- 172.21.15.16:0/1132847080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650778b21b0 msgr2=0x565077c80e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:13.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.311+0000 7f143be00640 1 --2- 172.21.15.16:0/1596998406 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f14280b4ca0 0x7f14280b71b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f142c02aa00 tx=0x7f142c041040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:13.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.311+0000 7f1451648740 1 --2- 172.21.15.16:0/1132847080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650778b21b0 0x565077c80e90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f143400bd10 tx=0x7f14340107a0 comp rx=0 tx=0).stop 2024-08-31T06:08:13.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.311+0000 7f1451648740 1 -- 172.21.15.16:0/1132847080 shutdown_connections 2024-08-31T06:08:13.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.312+0000 7f1451648740 1 --2- 172.21.15.16:0/1132847080 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650778ba390 0x565077c813d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:13.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.312+0000 7f1451648740 1 --2- 172.21.15.16:0/1132847080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650778b21b0 0x565077c80e90 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:13.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.312+0000 7f1451648740 1 --2- 172.21.15.16:0/1132847080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650778db070 0x565077c80950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:13.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.312+0000 7f1451648740 1 -- 172.21.15.16:0/1132847080 >> 172.21.15.16:0/1132847080 conn(0x5650778cf460 msgr2=0x565077c7e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:13.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.312+0000 7f1451648740 1 -- 172.21.15.16:0/1132847080 shutdown_connections 2024-08-31T06:08:13.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.312+0000 7f1451648740 1 -- 172.21.15.16:0/1132847080 wait complete. 2024-08-31T06:08:13.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.312+0000 7f1451648740 1 -- start start 2024-08-31T06:08:13.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.312+0000 7f1451648740 1 Processor -- start 2024-08-31T06:08:13.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.313+0000 7f1451648740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650778b21b0 0x565077c8c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:13.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.313+0000 7f1451648740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650778ba390 0x565077c8ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:13.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.313+0000 7f1451648740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650778db070 0x565077c98480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:13.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.313+0000 7f1451648740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x565077a49700 con 0x5650778b21b0 2024-08-31T06:08:13.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.313+0000 7f1451648740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x565077b6dcb0 con 0x5650778db070 2024-08-31T06:08:13.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.313+0000 7f1451648740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x565077c8f130 con 0x5650778ba390 2024-08-31T06:08:13.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.313+0000 7f1451648740 10 client.0 _ll_register_callbacks cb 0x565077b93430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:08:13.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.313+0000 7f1443e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650778b21b0 0x565077c8c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:13.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.313+0000 7f1443e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650778b21b0 0x565077c8c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60654/0 (socket says 172.21.15.16:60654) 2024-08-31T06:08:13.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.313+0000 7f1443e00640 1 -- 172.21.15.16:0/4068129217 learned_addr learned my addr 172.21.15.16:0/4068129217 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:13.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.313+0000 7f143be00640 1 --2- 172.21.15.16:0/4068129217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650778ba390 0x565077c8ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:13.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.313+0000 7f1448a00640 1 --2- 172.21.15.16:0/4068129217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650778db070 0x565077c98480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:13.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.314+0000 7f143be00640 1 -- 172.21.15.16:0/4068129217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650778db070 msgr2=0x565077c98480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:13.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.314+0000 7f143be00640 1 --2- 172.21.15.16:0/4068129217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650778db070 0x565077c98480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:13.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.314+0000 7f143be00640 1 -- 172.21.15.16:0/4068129217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650778b21b0 msgr2=0x565077c8c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:13.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.314+0000 7f143be00640 1 --2- 172.21.15.16:0/4068129217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650778b21b0 0x565077c8c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:13.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.314+0000 7f143be00640 1 -- 172.21.15.16:0/4068129217 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565077bae870 con 0x5650778ba390 2024-08-31T06:08:13.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.314+0000 7f1443e00640 1 --2- 172.21.15.16:0/4068129217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650778b21b0 0x565077c8c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:08:13.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.314+0000 7f143be00640 1 --2- 172.21.15.16:0/4068129217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650778ba390 0x565077c8ca00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f142c009230 tx=0x7f142c02a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ac6c0b5f2123a52 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:13.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.314+0000 7f1431600640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:08:13.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.315+0000 7f1431600640 1 -- 172.21.15.16:0/4068129217 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142c051070 con 0x5650778ba390 2024-08-31T06:08:13.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.315+0000 7f1451648740 1 -- 172.21.15.16:0/4068129217 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14340035f0 con 0x5650778ba390 2024-08-31T06:08:13.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.315+0000 7f1431600640 1 -- 172.21.15.16:0/4068129217 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f142c0318b0 con 0x5650778ba390 2024-08-31T06:08:13.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.315+0000 7f1431600640 1 -- 172.21.15.16:0/4068129217 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f142c05d050 con 0x5650778ba390 2024-08-31T06:08:13.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.315+0000 7f1451648740 10 client.25765 fetch_fsmap learned FSMap version 30 2024-08-31T06:08:13.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.315+0000 7f1451648740 1 -- 172.21.15.16:0/4068129217 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x565077c9b140 con 0x5650778ba390 2024-08-31T06:08:13.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.315+0000 7f1431600640 1 -- 172.21.15.16:0/4068129217 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14340035f0 con 0x5650778ba390 2024-08-31T06:08:13.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.316+0000 7f1431600640 1 -- 172.21.15.16:0/4068129217 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f142c03db50 con 0x5650778ba390 2024-08-31T06:08:13.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.316+0000 7f1451648740 10 client.25765 fetch_fsmap finished waiting for FSMap version 30 2024-08-31T06:08:13.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.316+0000 7f1451648740 10 client.25765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:08:13.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.316+0000 7f1451648740 20 client.25765 populate_metadata read hostname 'smithi016' 2024-08-31T06:08:13.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.316+0000 7f1451648740 1 --2- 172.21.15.16:0/4068129217 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5650778999f0 0x565077c9e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:13.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.316+0000 7f1451648740 4 client.25765 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:08:13.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.316+0000 7f1451648740 1 -- 172.21.15.16:0/4068129217 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565077bf96c0 con 0x5650778999f0 2024-08-31T06:08:13.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.316+0000 7f1443e00640 1 --2- 172.21.15.16:0/4068129217 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5650778999f0 0x565077c9e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:13.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.317+0000 7f1443e00640 1 --2- 172.21.15.16:0/4068129217 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5650778999f0 0x565077c9e640 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa10c899df849d28 server_cookie=5dfaf72797065090 in_seq=0 out_seq=0 2024-08-31T06:08:13.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.317+0000 7f1431600640 10 client.25765 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:13.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.317+0000 7f1431600640 1 -- 172.21.15.16:0/4068129217 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x565077bf96c0 con 0x5650778999f0 2024-08-31T06:08:13.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.317+0000 7f1431600640 10 client.25765 handle_command_reply: tid=0 2024-08-31T06:08:13.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.318+0000 7f1451648740 1 client.25765 shutdown 2024-08-31T06:08:13.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.318+0000 7f1451648740 1 -- 172.21.15.16:0/4068129217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650778ba390 msgr2=0x565077c8ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:13.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.318+0000 7f1451648740 1 --2- 172.21.15.16:0/4068129217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650778ba390 0x565077c8ca00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f142c009230 tx=0x7f142c02a410 comp rx=0 tx=0).stop 2024-08-31T06:08:13.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.318+0000 7f1451648740 1 -- 172.21.15.16:0/4068129217 shutdown_connections 2024-08-31T06:08:13.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.318+0000 7f1451648740 1 --2- 172.21.15.16:0/4068129217 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5650778999f0 0x565077c9e640 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:13.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.318+0000 7f1451648740 1 --2- 172.21.15.16:0/4068129217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650778ba390 0x565077c8ca00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:13.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.318+0000 7f1451648740 1 --2- 172.21.15.16:0/4068129217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650778db070 0x565077c98480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:13.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.318+0000 7f1451648740 1 --2- 172.21.15.16:0/4068129217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650778b21b0 0x565077c8c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:13.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.318+0000 7f1451648740 1 -- 172.21.15.16:0/4068129217 >> 172.21.15.16:0/4068129217 conn(0x5650778cf460 msgr2=0x565077c83ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:13.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.318+0000 7f1431600640 0 client.25765 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:13.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.318+0000 7f1451648740 1 -- 172.21.15.16:0/4068129217 shutdown_connections 2024-08-31T06:08:13.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.318+0000 7f1451648740 1 -- 172.21.15.16:0/4068129217 wait complete. 2024-08-31T06:08:13.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.318+0000 7f1451648740 20 client.25765 trim_cache size 0 max 16384 2024-08-31T06:08:13.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.337+0000 7f1451648740 1 Processor -- start 2024-08-31T06:08:13.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.337+0000 7f1451648740 1 -- start start 2024-08-31T06:08:13.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.337+0000 7f1451648740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650778b21b0 0x565077c82090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:13.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.337+0000 7f1451648740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650778ba390 0x565077c98380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:13.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.337+0000 7f1451648740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650778db070 0x565077c988c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:13.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.337+0000 7f1451648740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f14441b3ef0 con 0x5650778ba390 2024-08-31T06:08:13.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.337+0000 7f1451648740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f14441b1730 con 0x5650778b21b0 2024-08-31T06:08:13.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.337+0000 7f1451648740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f14441af580 con 0x5650778db070 2024-08-31T06:08:13.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.337+0000 7f143be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650778ba390 0x565077c98380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:13.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.337+0000 7f143be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650778ba390 0x565077c98380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60660/0 (socket says 172.21.15.16:60660) 2024-08-31T06:08:13.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.337+0000 7f143be00640 1 -- 172.21.15.16:0/924823938 learned_addr learned my addr 172.21.15.16:0/924823938 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:13.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.338+0000 7f1443e00640 1 --2- 172.21.15.16:0/924823938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650778b21b0 0x565077c82090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:13.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.338+0000 7f1448a00640 1 --2- 172.21.15.16:0/924823938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650778db070 0x565077c988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:13.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.338+0000 7f143be00640 1 -- 172.21.15.16:0/924823938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650778db070 msgr2=0x565077c988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:13.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.338+0000 7f143be00640 1 --2- 172.21.15.16:0/924823938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650778db070 0x565077c988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:13.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.338+0000 7f143be00640 1 -- 172.21.15.16:0/924823938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650778b21b0 msgr2=0x565077c82090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:13.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.338+0000 7f143be00640 1 --2- 172.21.15.16:0/924823938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650778b21b0 0x565077c82090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:13.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.338+0000 7f143be00640 1 -- 172.21.15.16:0/924823938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565077c78a40 con 0x5650778ba390 2024-08-31T06:08:13.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.338+0000 7f1448a00640 1 --2- 172.21.15.16:0/924823938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650778db070 0x565077c988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:08:13.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.338+0000 7f143be00640 1 --2- 172.21.15.16:0/924823938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650778ba390 0x565077c98380 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f142c0096c0 tx=0x7f142c005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a393965cce9d8137 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:13.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.338+0000 7f1433e00640 1 -- 172.21.15.16:0/924823938 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142c066070 con 0x5650778ba390 2024-08-31T06:08:13.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.339+0000 7f1433e00640 1 -- 172.21.15.16:0/924823938 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f142c0316b0 con 0x5650778ba390 2024-08-31T06:08:13.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.339+0000 7f1433e00640 1 -- 172.21.15.16:0/924823938 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142c051040 con 0x5650778ba390 2024-08-31T06:08:13.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.339+0000 7f1451648740 1 -- 172.21.15.16:0/924823938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650778ba390 msgr2=0x565077c98380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:13.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.339+0000 7f1451648740 1 --2- 172.21.15.16:0/924823938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650778ba390 0x565077c98380 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f142c0096c0 tx=0x7f142c005780 comp rx=0 tx=0).stop 2024-08-31T06:08:13.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.339+0000 7f1451648740 1 -- 172.21.15.16:0/924823938 shutdown_connections 2024-08-31T06:08:13.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.339+0000 7f1451648740 1 --2- 172.21.15.16:0/924823938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650778db070 0x565077c988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:13.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.339+0000 7f1451648740 1 --2- 172.21.15.16:0/924823938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650778ba390 0x565077c98380 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:13.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.339+0000 7f1451648740 1 --2- 172.21.15.16:0/924823938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650778b21b0 0x565077c82090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:13.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.339+0000 7f1451648740 1 -- 172.21.15.16:0/924823938 >> 172.21.15.16:0/924823938 conn(0x5650778cf460 msgr2=0x565077c81980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:13.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.340+0000 7f1451648740 1 -- 172.21.15.16:0/924823938 shutdown_connections 2024-08-31T06:08:13.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.340+0000 7f1451648740 1 -- 172.21.15.16:0/924823938 wait complete. 2024-08-31T06:08:13.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.340+0000 7f1451648740 1 -- start start 2024-08-31T06:08:13.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.340+0000 7f1451648740 1 Processor -- start 2024-08-31T06:08:13.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.341+0000 7f1451648740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650778b21b0 0x565077c905c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:13.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.341+0000 7f1451648740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650778ba390 0x565077c90b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:13.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.341+0000 7f1451648740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650778db070 0x565077c91040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:13.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.341+0000 7f1451648740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f14441afbf0 con 0x5650778b21b0 2024-08-31T06:08:13.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.341+0000 7f1451648740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f14441b1060 con 0x5650778db070 2024-08-31T06:08:13.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.341+0000 7f1451648740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f14441b02c0 con 0x5650778ba390 2024-08-31T06:08:13.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.341+0000 7f1451648740 10 client.0 _ll_register_callbacks cb 0x7f142c004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:08:13.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.341+0000 7f143be00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650778ba390 0x565077c90b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:13.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.341+0000 7f1443e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650778b21b0 0x565077c905c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:13.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.341+0000 7f143be00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650778ba390 0x565077c90b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:41964/0 (socket says 172.21.15.16:41964) 2024-08-31T06:08:13.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.341+0000 7f143be00640 1 -- 172.21.15.16:0/3915507470 learned_addr learned my addr 172.21.15.16:0/3915507470 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:13.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.341+0000 7f1448a00640 1 --2- 172.21.15.16:0/3915507470 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650778db070 0x565077c91040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:13.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.341+0000 7f143be00640 1 -- 172.21.15.16:0/3915507470 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650778db070 msgr2=0x565077c91040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:13.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.341+0000 7f143be00640 1 --2- 172.21.15.16:0/3915507470 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650778db070 0x565077c91040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:13.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.341+0000 7f143be00640 1 -- 172.21.15.16:0/3915507470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650778b21b0 msgr2=0x565077c905c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:13.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.341+0000 7f143be00640 1 --2- 172.21.15.16:0/3915507470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650778b21b0 0x565077c905c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:13.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.341+0000 7f143be00640 1 -- 172.21.15.16:0/3915507470 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565077d081f0 con 0x5650778ba390 2024-08-31T06:08:13.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.341+0000 7f1448a00640 1 --2- 172.21.15.16:0/3915507470 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650778db070 0x565077c91040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:08:13.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.341+0000 7f1443e00640 1 --2- 172.21.15.16:0/3915507470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650778b21b0 0x565077c905c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:08:13.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.342+0000 7f143be00640 1 --2- 172.21.15.16:0/3915507470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650778ba390 0x565077c90b00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f142c031440 tx=0x7f142c04db10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=716fb3c7f34b0c6a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:13.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.342+0000 7f1431600640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:08:13.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.342+0000 7f1431600640 1 -- 172.21.15.16:0/3915507470 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142c066070 con 0x5650778ba390 2024-08-31T06:08:13.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.342+0000 7f1431600640 1 -- 172.21.15.16:0/3915507470 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f142c031a60 con 0x5650778ba390 2024-08-31T06:08:13.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.342+0000 7f1451648740 1 -- 172.21.15.16:0/3915507470 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f142c0316b0 con 0x5650778ba390 2024-08-31T06:08:13.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.343+0000 7f1431600640 1 -- 172.21.15.16:0/3915507470 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f142c051040 con 0x5650778ba390 2024-08-31T06:08:13.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.343+0000 7f1451648740 10 client.25771 fetch_fsmap learned FSMap version 30 2024-08-31T06:08:13.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.343+0000 7f1451648740 1 -- 172.21.15.16:0/3915507470 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x565077d09480 con 0x5650778ba390 2024-08-31T06:08:13.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.343+0000 7f1431600640 1 -- 172.21.15.16:0/3915507470 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f142c0316b0 con 0x5650778ba390 2024-08-31T06:08:13.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.343+0000 7f1431600640 1 -- 172.21.15.16:0/3915507470 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f142c03db70 con 0x5650778ba390 2024-08-31T06:08:13.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.343+0000 7f1451648740 10 client.25771 fetch_fsmap finished waiting for FSMap version 30 2024-08-31T06:08:13.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.343+0000 7f1451648740 10 client.25771 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:08:13.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.343+0000 7f1451648740 20 client.25771 populate_metadata read hostname 'smithi016' 2024-08-31T06:08:13.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.343+0000 7f1451648740 1 --2- 172.21.15.16:0/3915507470 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56507786eda0 0x565077d0bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:13.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.343+0000 7f1451648740 4 client.25771 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T06:08:13.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.343+0000 7f1451648740 1 -- 172.21.15.16:0/3915507470 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub status"}) -- 0x565077bf5160 con 0x56507786eda0 2024-08-31T06:08:13.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.344+0000 7f1443e00640 1 --2- 172.21.15.16:0/3915507470 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56507786eda0 0x565077d0bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:13.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.344+0000 7f1443e00640 1 --2- 172.21.15.16:0/3915507470 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56507786eda0 0x565077d0bb60 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f57ea935473717eb server_cookie=bdd4cfaa45cb16b2 in_seq=0 out_seq=0 2024-08-31T06:08:13.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.344+0000 7f1431600640 10 client.25771 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:13.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.344+0000 7f1431600640 1 -- 172.21.15.16:0/3915507470 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+658 (crc 0 0 0) 0x565077bf5160 con 0x56507786eda0 2024-08-31T06:08:13.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.344+0000 7f1431600640 10 client.25771 handle_command_reply: tid=0 2024-08-31T06:08:13.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.344+0000 7f1451648740 1 client.25771 shutdown 2024-08-31T06:08:13.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.345+0000 7f1451648740 1 -- 172.21.15.16:0/3915507470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650778ba390 msgr2=0x565077c90b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:13.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.345+0000 7f1451648740 1 --2- 172.21.15.16:0/3915507470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650778ba390 0x565077c90b00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f142c031440 tx=0x7f142c04db10 comp rx=0 tx=0).stop 2024-08-31T06:08:13.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.345+0000 7f1451648740 1 -- 172.21.15.16:0/3915507470 shutdown_connections 2024-08-31T06:08:13.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.345+0000 7f1451648740 1 --2- 172.21.15.16:0/3915507470 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56507786eda0 0x565077d0bb60 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:13.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.345+0000 7f1451648740 1 --2- 172.21.15.16:0/3915507470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650778ba390 0x565077c90b00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:13.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.345+0000 7f1451648740 1 --2- 172.21.15.16:0/3915507470 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650778db070 0x565077c91040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:13.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.345+0000 7f1451648740 1 --2- 172.21.15.16:0/3915507470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650778b21b0 0x565077c905c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:13.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.345+0000 7f1451648740 1 -- 172.21.15.16:0/3915507470 >> 172.21.15.16:0/3915507470 conn(0x5650778cf460 msgr2=0x565077c8aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:13.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.345+0000 7f1431600640 0 client.25771 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:13.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.345+0000 7f1451648740 1 -- 172.21.15.16:0/3915507470 shutdown_connections 2024-08-31T06:08:13.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.345+0000 7f1451648740 1 -- 172.21.15.16:0/3915507470 wait complete. 2024-08-31T06:08:13.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.345+0000 7f1451648740 20 client.25771 trim_cache size 0 max 16384 2024-08-31T06:08:13.350 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:08:13.350 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-08-31T06:08:13.350 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {}, 2024-08-31T06:08:13.350 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-08-31T06:08:13.350 INFO:teuthology.orchestra.run.smithi016.stdout: "88d7bdb8-cb1c-45e5-ad29-4cd8da6c6003": { 2024-08-31T06:08:13.351 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:07:39.648942+0000", 2024-08-31T06:08:13.351 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:08:13.351 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:08:13.351 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:08:13.351 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:08:13.351 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:08:13.351 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:08:13.351 INFO:teuthology.orchestra.run.smithi016.stdout: "c300b7e8-f3fd-482c-a909-9329bc61e570": { 2024-08-31T06:08:13.351 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:06:49.868046+0000", 2024-08-31T06:08:13.351 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:08:13.351 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:08:13.351 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:08:13.351 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:08:13.351 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:08:13.351 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:08:13.352 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:08:13.352 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:08:13.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.346+0000 7f144aa00640 1 -- 172.21.15.16:0/1596998406 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f14280b4ca0 msgr2=0x7f14280b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:13.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.346+0000 7f144aa00640 1 --2- 172.21.15.16:0/1596998406 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f14280b4ca0 0x7f14280b71b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f142c02aa00 tx=0x7f142c041040 comp rx=0 tx=0).stop 2024-08-31T06:08:13.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.346+0000 7f144aa00640 1 -- 172.21.15.16:0/1596998406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1444109b90 msgr2=0x7f14441a1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:13.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.346+0000 7f144aa00640 1 --2- 172.21.15.16:0/1596998406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1444109b90 0x7f14441a1d10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f143c00eb10 tx=0x7f143c00c2d0 comp rx=0 tx=0).stop 2024-08-31T06:08:13.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.346+0000 7f144aa00640 1 -- 172.21.15.16:0/1596998406 shutdown_connections 2024-08-31T06:08:13.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.346+0000 7f144aa00640 1 --2- 172.21.15.16:0/1596998406 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f14280b4ca0 0x7f14280b71b0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:13.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.346+0000 7f144aa00640 1 --2- 172.21.15.16:0/1596998406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1444109b90 0x7f14441a1d10 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:13.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.346+0000 7f144aa00640 1 --2- 172.21.15.16:0/1596998406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1444109240 0x7f144419ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:13.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.346+0000 7f144aa00640 1 --2- 172.21.15.16:0/1596998406 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1444103900 0x7f144419c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:13.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.346+0000 7f144aa00640 1 -- 172.21.15.16:0/1596998406 >> 172.21.15.16:0/1596998406 conn(0x7f144407b5f0 msgr2=0x7f14441070f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:13.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.346+0000 7f144aa00640 1 -- 172.21.15.16:0/1596998406 shutdown_connections 2024-08-31T06:08:13.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:13.346+0000 7f144aa00640 1 -- 172.21.15.16:0/1596998406 wait complete. 2024-08-31T06:08:13.366 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:13 smithi016 ceph-mon[17194]: pgmap v393: 97 pgs: 97 active+clean; 940 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 34 MiB/s wr, 482 op/s 2024-08-31T06:08:13.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:13 smithi155 ceph-mon[21742]: pgmap v393: 97 pgs: 97 active+clean; 940 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 34 MiB/s wr, 482 op/s 2024-08-31T06:08:13.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:13 smithi184 ceph-mon[22331]: pgmap v393: 97 pgs: 97 active+clean; 940 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 34 MiB/s wr, 482 op/s 2024-08-31T06:08:14.178 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T06:08:14.179 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:08:14.565 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:08:15.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:15 smithi016 ceph-mon[17194]: pgmap v394: 97 pgs: 97 active+clean; 940 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 33 MiB/s wr, 478 op/s 2024-08-31T06:08:15.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.191+0000 7f4290c00640 1 -- 172.21.15.16:0/974233735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f428c103900 msgr2=0x7f428c103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:15.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.191+0000 7f4290c00640 1 --2- 172.21.15.16:0/974233735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f428c103900 0x7f428c103d10 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f427c009900 tx=0x7f427c02a3f0 comp rx=0 tx=0).stop 2024-08-31T06:08:15.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.191+0000 7f4290c00640 1 -- 172.21.15.16:0/974233735 shutdown_connections 2024-08-31T06:08:15.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.191+0000 7f4290c00640 1 --2- 172.21.15.16:0/974233735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f428c109b90 0x7f428c1103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:15.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.191+0000 7f4290c00640 1 --2- 172.21.15.16:0/974233735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f428c109240 0x7f428c109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:15.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.191+0000 7f4290c00640 1 --2- 172.21.15.16:0/974233735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f428c103900 0x7f428c103d10 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:15.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.191+0000 7f4290c00640 1 -- 172.21.15.16:0/974233735 >> 172.21.15.16:0/974233735 conn(0x7f428c07b5f0 msgr2=0x7f428c07ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:15.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.191+0000 7f4290c00640 1 -- 172.21.15.16:0/974233735 shutdown_connections 2024-08-31T06:08:15.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.191+0000 7f4290c00640 1 -- 172.21.15.16:0/974233735 wait complete. 2024-08-31T06:08:15.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.192+0000 7f4290c00640 1 Processor -- start 2024-08-31T06:08:15.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.193+0000 7f4290c00640 1 -- start start 2024-08-31T06:08:15.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.193+0000 7f4290c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f428c103900 0x7f428c0754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:15.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.193+0000 7f4290c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f428c109240 0x7f428c0759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:15.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.193+0000 7f4290c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f428c109b90 0x7f428c075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:15.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.193+0000 7f4290c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f428c113850 con 0x7f428c103900 2024-08-31T06:08:15.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.193+0000 7f4290c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f428c1136d0 con 0x7f428c109240 2024-08-31T06:08:15.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.193+0000 7f4290c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f428c1139d0 con 0x7f428c109b90 2024-08-31T06:08:15.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.193+0000 7f4289e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f428c103900 0x7f428c0754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:15.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.194+0000 7f4289e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f428c103900 0x7f428c0754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60696/0 (socket says 172.21.15.16:60696) 2024-08-31T06:08:15.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.194+0000 7f4289e00640 1 -- 172.21.15.16:0/1847520329 learned_addr learned my addr 172.21.15.16:0/1847520329 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:15.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.194+0000 7f428a800640 1 --2- 172.21.15.16:0/1847520329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f428c109b90 0x7f428c075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:15.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.194+0000 7f4289400640 1 --2- 172.21.15.16:0/1847520329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f428c109240 0x7f428c0759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:15.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.194+0000 7f428a800640 1 -- 172.21.15.16:0/1847520329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f428c109240 msgr2=0x7f428c0759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:15.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.194+0000 7f428a800640 1 --2- 172.21.15.16:0/1847520329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f428c109240 0x7f428c0759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:15.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.194+0000 7f428a800640 1 -- 172.21.15.16:0/1847520329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f428c103900 msgr2=0x7f428c0754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:15.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.194+0000 7f428a800640 1 --2- 172.21.15.16:0/1847520329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f428c103900 0x7f428c0754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:15.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.194+0000 7f428a800640 1 -- 172.21.15.16:0/1847520329 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f428c1a8500 con 0x7f428c109b90 2024-08-31T06:08:15.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.194+0000 7f4289e00640 1 --2- 172.21.15.16:0/1847520329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f428c103900 0x7f428c0754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:08:15.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.194+0000 7f428a800640 1 --2- 172.21.15.16:0/1847520329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f428c109b90 0x7f428c075f30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f428400e940 tx=0x7f428400ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:15.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.195+0000 7f4282a00640 1 -- 172.21.15.16:0/1847520329 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4284004d20 con 0x7f428c109b90 2024-08-31T06:08:15.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.195+0000 7f4282a00640 1 -- 172.21.15.16:0/1847520329 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4284005020 con 0x7f428c109b90 2024-08-31T06:08:15.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.195+0000 7f4282a00640 1 -- 172.21.15.16:0/1847520329 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42840116d0 con 0x7f428c109b90 2024-08-31T06:08:15.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.195+0000 7f4290c00640 1 -- 172.21.15.16:0/1847520329 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f428c1a87f0 con 0x7f428c109b90 2024-08-31T06:08:15.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.195+0000 7f4290c00640 1 -- 172.21.15.16:0/1847520329 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f428c1a8ce0 con 0x7f428c109b90 2024-08-31T06:08:15.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.196+0000 7f4290c00640 1 -- 172.21.15.16:0/1847520329 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f428c103d10 con 0x7f428c109b90 2024-08-31T06:08:15.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.198+0000 7f4282a00640 1 -- 172.21.15.16:0/1847520329 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f4284005460 con 0x7f428c109b90 2024-08-31T06:08:15.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.200+0000 7f4282a00640 1 --2- 172.21.15.16:0/1847520329 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f42700b4d20 0x7f42700b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:15.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.200+0000 7f4282a00640 1 -- 172.21.15.16:0/1847520329 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f42840e5530 con 0x7f428c109b90 2024-08-31T06:08:15.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.200+0000 7f4289e00640 1 --2- 172.21.15.16:0/1847520329 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f42700b4d20 0x7f42700b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:15.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.201+0000 7f4289e00640 1 --2- 172.21.15.16:0/1847520329 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f42700b4d20 0x7f42700b7230 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f427c009230 tx=0x7f427c0058d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:15.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.202+0000 7f4282a00640 1 -- 172.21.15.16:0/1847520329 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f4284016030 con 0x7f428c109b90 2024-08-31T06:08:15.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:15 smithi155 ceph-mon[21742]: pgmap v394: 97 pgs: 97 active+clean; 940 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 33 MiB/s wr, 478 op/s 2024-08-31T06:08:15.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.435+0000 7f4290c00640 1 -- 172.21.15.16:0/1847520329 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f428c0767b0 con 0x7f428c109b90 2024-08-31T06:08:15.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.436+0000 7f4282a00640 1 -- 172.21.15.16:0/1847520329 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11921 (secure 0 0 0) 0x7f428c0767b0 con 0x7f428c109b90 2024-08-31T06:08:15.438 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:08:15.438 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":30,"btime":"2024-08-31T06:07:34:013454+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:07:33.329391+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24712,24724,24737]},"id":1}]} 2024-08-31T06:08:15.439 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 30 2024-08-31T06:08:15.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.439+0000 7f4290c00640 1 -- 172.21.15.16:0/1847520329 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f42700b4d20 msgr2=0x7f42700b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:15.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.439+0000 7f4290c00640 1 --2- 172.21.15.16:0/1847520329 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f42700b4d20 0x7f42700b7230 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f427c009230 tx=0x7f427c0058d0 comp rx=0 tx=0).stop 2024-08-31T06:08:15.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.439+0000 7f4290c00640 1 -- 172.21.15.16:0/1847520329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f428c109b90 msgr2=0x7f428c075f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:15.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.439+0000 7f4290c00640 1 --2- 172.21.15.16:0/1847520329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f428c109b90 0x7f428c075f30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f428400e940 tx=0x7f428400ee00 comp rx=0 tx=0).stop 2024-08-31T06:08:15.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.439+0000 7f4290c00640 1 -- 172.21.15.16:0/1847520329 shutdown_connections 2024-08-31T06:08:15.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.439+0000 7f4290c00640 1 --2- 172.21.15.16:0/1847520329 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f42700b4d20 0x7f42700b7230 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:15.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.439+0000 7f4290c00640 1 --2- 172.21.15.16:0/1847520329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f428c109b90 0x7f428c075f30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:15.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.439+0000 7f4290c00640 1 --2- 172.21.15.16:0/1847520329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f428c109240 0x7f428c0759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:15.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.439+0000 7f4290c00640 1 --2- 172.21.15.16:0/1847520329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f428c103900 0x7f428c0754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:15.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.439+0000 7f4290c00640 1 -- 172.21.15.16:0/1847520329 >> 172.21.15.16:0/1847520329 conn(0x7f428c07b5f0 msgr2=0x7f428c1072d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:15.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.439+0000 7f4290c00640 1 -- 172.21.15.16:0/1847520329 shutdown_connections 2024-08-31T06:08:15.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:15.439+0000 7f4290c00640 1 -- 172.21.15.16:0/1847520329 wait complete. 2024-08-31T06:08:15.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:15 smithi184 ceph-mon[22331]: pgmap v394: 97 pgs: 97 active+clean; 940 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 33 MiB/s wr, 478 op/s 2024-08-31T06:08:16.166 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T06:08:16.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:16 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/1847520329' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:08:16.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:16 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/1847520329' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:08:16.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:16 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/1847520329' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:08:16.543 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:08:16.629 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 50.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-31T06:08:17.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.183+0000 7f0b99200640 1 -- 172.21.15.16:0/3049077219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b94109240 msgr2=0x7f0b94109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:17.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.183+0000 7f0b99200640 1 --2- 172.21.15.16:0/3049077219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b94109240 0x7f0b94109650 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c009900 tx=0x7f0b7c02a410 comp rx=0 tx=0).stop 2024-08-31T06:08:17.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.183+0000 7f0b99200640 1 -- 172.21.15.16:0/3049077219 shutdown_connections 2024-08-31T06:08:17.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.183+0000 7f0b99200640 1 --2- 172.21.15.16:0/3049077219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b94109b90 0x7f0b941103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:17.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.183+0000 7f0b99200640 1 --2- 172.21.15.16:0/3049077219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b94109240 0x7f0b94109650 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:17.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.183+0000 7f0b99200640 1 --2- 172.21.15.16:0/3049077219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b94103900 0x7f0b94103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:17.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.183+0000 7f0b99200640 1 -- 172.21.15.16:0/3049077219 >> 172.21.15.16:0/3049077219 conn(0x7f0b9407b5f0 msgr2=0x7f0b9407ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:17.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.183+0000 7f0b99200640 1 -- 172.21.15.16:0/3049077219 shutdown_connections 2024-08-31T06:08:17.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.183+0000 7f0b99200640 1 -- 172.21.15.16:0/3049077219 wait complete. 2024-08-31T06:08:17.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.184+0000 7f0b99200640 1 Processor -- start 2024-08-31T06:08:17.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.184+0000 7f0b99200640 1 -- start start 2024-08-31T06:08:17.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.184+0000 7f0b99200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b94103900 0x7f0b9419c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:17.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.184+0000 7f0b99200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b94109240 0x7f0b9419c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:17.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.184+0000 7f0b99200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b94109b90 0x7f0b941a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:17.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.184+0000 7f0b99200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0b941138a0 con 0x7f0b94103900 2024-08-31T06:08:17.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.184+0000 7f0b99200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0b94113720 con 0x7f0b94109b90 2024-08-31T06:08:17.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.184+0000 7f0b99200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0b94113a20 con 0x7f0b94109240 2024-08-31T06:08:17.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.185+0000 7f0b91e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b94103900 0x7f0b9419c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:17.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.185+0000 7f0b91e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b94103900 0x7f0b9419c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56290/0 (socket says 172.21.15.16:56290) 2024-08-31T06:08:17.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.185+0000 7f0b91e00640 1 -- 172.21.15.16:0/366776535 learned_addr learned my addr 172.21.15.16:0/366776535 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:17.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.185+0000 7f0b8be00640 1 --2- 172.21.15.16:0/366776535 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b94109240 0x7f0b9419c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:17.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.185+0000 7f0b92800640 1 --2- 172.21.15.16:0/366776535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b94109b90 0x7f0b941a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:17.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.185+0000 7f0b8be00640 1 -- 172.21.15.16:0/366776535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b94109b90 msgr2=0x7f0b941a1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:17.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.185+0000 7f0b8be00640 1 --2- 172.21.15.16:0/366776535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b94109b90 0x7f0b941a1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:17.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.185+0000 7f0b8be00640 1 -- 172.21.15.16:0/366776535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b94103900 msgr2=0x7f0b9419c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:17.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.185+0000 7f0b8be00640 1 --2- 172.21.15.16:0/366776535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b94103900 0x7f0b9419c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:17.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.185+0000 7f0b8be00640 1 -- 172.21.15.16:0/366776535 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b941a23c0 con 0x7f0b94109240 2024-08-31T06:08:17.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.185+0000 7f0b91e00640 1 --2- 172.21.15.16:0/366776535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b94103900 0x7f0b9419c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:08:17.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.186+0000 7f0b8be00640 1 --2- 172.21.15.16:0/366776535 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b94109240 0x7f0b9419c9f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c009230 tx=0x7f0b7c003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:17.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.186+0000 7f0b8aa00640 1 -- 172.21.15.16:0/366776535 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c03f070 con 0x7f0b94109240 2024-08-31T06:08:17.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.186+0000 7f0b99200640 1 -- 172.21.15.16:0/366776535 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b941a2650 con 0x7f0b94109240 2024-08-31T06:08:17.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.186+0000 7f0b8aa00640 1 -- 172.21.15.16:0/366776535 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b7c031b60 con 0x7f0b94109240 2024-08-31T06:08:17.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.186+0000 7f0b8aa00640 1 -- 172.21.15.16:0/366776535 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c035030 con 0x7f0b94109240 2024-08-31T06:08:17.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.186+0000 7f0b99200640 1 -- 172.21.15.16:0/366776535 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b941a8e10 con 0x7f0b94109240 2024-08-31T06:08:17.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.189+0000 7f0b9ffc5740 1 Processor -- start 2024-08-31T06:08:17.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.189+0000 7f0b9ffc5740 1 -- start start 2024-08-31T06:08:17.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.190+0000 7f0b9ffc5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e4a6ad070 0x563e4aa52990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:17.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.190+0000 7f0b9ffc5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e4a6841b0 0x563e4aa52ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:17.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.190+0000 7f0b9ffc5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4a68c390 0x563e4aa53410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:17.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.190+0000 7f0b9ffc5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563e4a9c97b0 con 0x563e4a68c390 2024-08-31T06:08:17.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.190+0000 7f0b9ffc5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563e4a9b8d40 con 0x563e4a6ad070 2024-08-31T06:08:17.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.190+0000 7f0b9ffc5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563e4a9bccf0 con 0x563e4a6841b0 2024-08-31T06:08:17.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.190+0000 7f0b8aa00640 1 -- 172.21.15.16:0/366776535 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f0b7c042070 con 0x7f0b94109240 2024-08-31T06:08:17.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.190+0000 7f0b8be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4a68c390 0x563e4aa53410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:17.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.190+0000 7f0b92800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e4a6ad070 0x563e4aa52990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:17.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.190+0000 7f0b8be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4a68c390 0x563e4aa53410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56304/0 (socket says 172.21.15.16:56304) 2024-08-31T06:08:17.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.190+0000 7f0b8be00640 1 -- 172.21.15.16:0/2892579223 learned_addr learned my addr 172.21.15.16:0/2892579223 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:17.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.190+0000 7f0b91e00640 1 --2- 172.21.15.16:0/2892579223 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e4a6841b0 0x563e4aa52ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:17.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.191+0000 7f0b91e00640 1 -- 172.21.15.16:0/2892579223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e4a6ad070 msgr2=0x563e4aa52990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:17.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.191+0000 7f0b91e00640 1 --2- 172.21.15.16:0/2892579223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e4a6ad070 0x563e4aa52990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:17.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.191+0000 7f0b91e00640 1 -- 172.21.15.16:0/2892579223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4a68c390 msgr2=0x563e4aa53410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:17.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.191+0000 7f0b91e00640 1 --2- 172.21.15.16:0/2892579223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4a68c390 0x563e4aa53410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:17.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.191+0000 7f0b91e00640 1 -- 172.21.15.16:0/2892579223 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e4a718650 con 0x563e4a6841b0 2024-08-31T06:08:17.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.191+0000 7f0b92800640 1 --2- 172.21.15.16:0/2892579223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e4a6ad070 0x563e4aa52990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:08:17.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.191+0000 7f0b91e00640 1 --2- 172.21.15.16:0/2892579223 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e4a6841b0 0x563e4aa52ed0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0b84010550 tx=0x7f0b84010a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c53c1b3ac568fbf server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:17.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.191+0000 7f0b83400640 1 -- 172.21.15.16:0/2892579223 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8400e7d0 con 0x563e4a6841b0 2024-08-31T06:08:17.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.191+0000 7f0b83400640 1 -- 172.21.15.16:0/2892579223 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b8400ee20 con 0x563e4a6841b0 2024-08-31T06:08:17.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.191+0000 7f0b83400640 1 -- 172.21.15.16:0/2892579223 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b840193d0 con 0x563e4a6841b0 2024-08-31T06:08:17.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.192+0000 7f0b8aa00640 1 --2- 172.21.15.16:0/366776535 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f0b780b4ca0 0x7f0b780b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:17.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.192+0000 7f0b9ffc5740 1 -- 172.21.15.16:0/2892579223 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e4a6841b0 msgr2=0x563e4aa52ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:17.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.192+0000 7f0b9ffc5740 1 --2- 172.21.15.16:0/2892579223 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e4a6841b0 0x563e4aa52ed0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0b84010550 tx=0x7f0b84010a10 comp rx=0 tx=0).stop 2024-08-31T06:08:17.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.192+0000 7f0b8aa00640 1 -- 172.21.15.16:0/366776535 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f0b7c105b30 con 0x7f0b94109240 2024-08-31T06:08:17.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.192+0000 7f0b9ffc5740 1 -- 172.21.15.16:0/2892579223 shutdown_connections 2024-08-31T06:08:17.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.192+0000 7f0b9ffc5740 1 --2- 172.21.15.16:0/2892579223 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e4a6841b0 0x563e4aa52ed0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:17.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.192+0000 7f0b9ffc5740 1 --2- 172.21.15.16:0/2892579223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4a68c390 0x563e4aa53410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:17.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.192+0000 7f0b9ffc5740 1 --2- 172.21.15.16:0/2892579223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e4a6ad070 0x563e4aa52990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:17.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.193+0000 7f0b9ffc5740 1 -- 172.21.15.16:0/2892579223 >> 172.21.15.16:0/2892579223 conn(0x563e4a6a1460 msgr2=0x563e4aa50880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:17.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.193+0000 7f0b9ffc5740 1 -- 172.21.15.16:0/2892579223 shutdown_connections 2024-08-31T06:08:17.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.193+0000 7f0b92800640 1 --2- 172.21.15.16:0/366776535 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f0b780b4ca0 0x7f0b780b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:17.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.193+0000 7f0b9ffc5740 1 -- 172.21.15.16:0/2892579223 wait complete. 2024-08-31T06:08:17.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.193+0000 7f0b9ffc5740 1 -- start start 2024-08-31T06:08:17.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.193+0000 7f0b92800640 1 --2- 172.21.15.16:0/366776535 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f0b780b4ca0 0x7f0b780b71b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c005fd0 tx=0x7f0b8c009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:17.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.194+0000 7f0b9ffc5740 1 Processor -- start 2024-08-31T06:08:17.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.194+0000 7f0b9ffc5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e4a6841b0 0x563e4aa5e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:17.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.194+0000 7f0b9ffc5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e4a68c390 0x563e4aa5ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:17.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.194+0000 7f0b9ffc5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4a6ad070 0x563e4aa6a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:17.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.194+0000 7f0b9ffc5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563e4a81b700 con 0x563e4a6ad070 2024-08-31T06:08:17.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.194+0000 7f0b9ffc5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563e4a93fcf0 con 0x563e4a68c390 2024-08-31T06:08:17.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.194+0000 7f0b9ffc5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563e4aa61170 con 0x563e4a6841b0 2024-08-31T06:08:17.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.194+0000 7f0b9ffc5740 10 client.0 _ll_register_callbacks cb 0x563e4a965470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:08:17.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.195+0000 7f0b91e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e4a6841b0 0x563e4aa5e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:17.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.195+0000 7f0b91e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e4a6841b0 0x563e4aa5e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:58288/0 (socket says 172.21.15.16:58288) 2024-08-31T06:08:17.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.195+0000 7f0b91e00640 1 -- 172.21.15.16:0/656865001 learned_addr learned my addr 172.21.15.16:0/656865001 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:17.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.195+0000 7f0b8be00640 1 --2- 172.21.15.16:0/656865001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e4a68c390 0x563e4aa5ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:17.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.195+0000 7f0b92800640 1 --2- 172.21.15.16:0/656865001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4a6ad070 0x563e4aa6a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:17.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.195+0000 7f0b91e00640 1 -- 172.21.15.16:0/656865001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e4a68c390 msgr2=0x563e4aa5ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:17.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.195+0000 7f0b91e00640 1 --2- 172.21.15.16:0/656865001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e4a68c390 0x563e4aa5ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:17.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.195+0000 7f0b91e00640 1 -- 172.21.15.16:0/656865001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4a6ad070 msgr2=0x563e4aa6a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:17.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.195+0000 7f0b91e00640 1 --2- 172.21.15.16:0/656865001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4a6ad070 0x563e4aa6a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:17.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.195+0000 7f0b91e00640 1 -- 172.21.15.16:0/656865001 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e4a9808b0 con 0x563e4a6841b0 2024-08-31T06:08:17.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.195+0000 7f0b91e00640 1 --2- 172.21.15.16:0/656865001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e4a6841b0 0x563e4aa5e510 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0b8400b740 tx=0x7f0b8400bf00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e6071a36208470b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:17.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.195+0000 7f0b77e00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:08:17.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.196+0000 7f0b77e00640 1 -- 172.21.15.16:0/656865001 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b84011d70 con 0x563e4a6841b0 2024-08-31T06:08:17.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.196+0000 7f0b9ffc5740 1 -- 172.21.15.16:0/656865001 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b8400ee20 con 0x563e4a6841b0 2024-08-31T06:08:17.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.196+0000 7f0b77e00640 1 -- 172.21.15.16:0/656865001 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f0b84011440 con 0x563e4a6841b0 2024-08-31T06:08:17.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.196+0000 7f0b77e00640 1 -- 172.21.15.16:0/656865001 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0b84031050 con 0x563e4a6841b0 2024-08-31T06:08:17.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.196+0000 7f0b9ffc5740 10 client.25789 fetch_fsmap learned FSMap version 30 2024-08-31T06:08:17.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.196+0000 7f0b9ffc5740 1 -- 172.21.15.16:0/656865001 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563e4aa6d150 con 0x563e4a6841b0 2024-08-31T06:08:17.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.196+0000 7f0b77e00640 1 -- 172.21.15.16:0/656865001 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b8400ee20 con 0x563e4a6841b0 2024-08-31T06:08:17.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.197+0000 7f0b77e00640 1 -- 172.21.15.16:0/656865001 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f0b84005380 con 0x563e4a6841b0 2024-08-31T06:08:17.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.197+0000 7f0b9ffc5740 10 client.25789 fetch_fsmap finished waiting for FSMap version 30 2024-08-31T06:08:17.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.197+0000 7f0b9ffc5740 10 client.25789 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T06:08:17.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.197+0000 7f0b9ffc5740 20 client.25789 populate_metadata read hostname 'smithi016' 2024-08-31T06:08:17.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.197+0000 7f0b9ffc5740 1 --2- 172.21.15.16:0/656865001 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x563e4a66b9f0 0x563e4aa70510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:17.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.197+0000 7f0b9ffc5740 4 client.25789 mds_command: new command op to 24712 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:08:17.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.197+0000 7f0b9ffc5740 1 -- 172.21.15.16:0/656865001 --> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e4a9cb700 con 0x563e4a66b9f0 2024-08-31T06:08:17.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.197+0000 7f0b8be00640 1 --2- 172.21.15.16:0/656865001 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x563e4a66b9f0 0x563e4aa70510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:17.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.198+0000 7f0b8be00640 1 --2- 172.21.15.16:0/656865001 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x563e4a66b9f0 0x563e4aa70510 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=99a3f0b7cd942654 server_cookie=3e409bb70e5cacb3 in_seq=0 out_seq=0 2024-08-31T06:08:17.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.198+0000 7f0b77e00640 10 client.25789 ms_handle_connect on v2:172.21.15.184:6838/4011051479 2024-08-31T06:08:17.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.199+0000 7f0b77e00640 1 -- 172.21.15.16:0/656865001 <== mds.2 v2:172.21.15.184:6838/4011051479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563e4a9cb700 con 0x563e4a66b9f0 2024-08-31T06:08:17.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.199+0000 7f0b77e00640 10 client.25789 handle_command_reply: tid=0 2024-08-31T06:08:17.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.199+0000 7f0b9ffc5740 1 client.25789 shutdown 2024-08-31T06:08:17.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.199+0000 7f0b9ffc5740 1 -- 172.21.15.16:0/656865001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e4a6841b0 msgr2=0x563e4aa5e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:17.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.199+0000 7f0b9ffc5740 1 --2- 172.21.15.16:0/656865001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e4a6841b0 0x563e4aa5e510 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0b8400b740 tx=0x7f0b8400bf00 comp rx=0 tx=0).stop 2024-08-31T06:08:17.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.199+0000 7f0b9ffc5740 1 -- 172.21.15.16:0/656865001 shutdown_connections 2024-08-31T06:08:17.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.199+0000 7f0b9ffc5740 1 --2- 172.21.15.16:0/656865001 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x563e4a66b9f0 0x563e4aa70510 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:17.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.199+0000 7f0b9ffc5740 1 --2- 172.21.15.16:0/656865001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4a6ad070 0x563e4aa6a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:17.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.199+0000 7f0b9ffc5740 1 --2- 172.21.15.16:0/656865001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e4a68c390 0x563e4aa5ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:17.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.199+0000 7f0b9ffc5740 1 --2- 172.21.15.16:0/656865001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e4a6841b0 0x563e4aa5e510 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:17.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.199+0000 7f0b77e00640 0 client.25789 ms_handle_reset on v2:172.21.15.184:6838/4011051479 2024-08-31T06:08:17.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.199+0000 7f0b9ffc5740 1 -- 172.21.15.16:0/656865001 >> 172.21.15.16:0/656865001 conn(0x563e4a6a1460 msgr2=0x563e4aa55b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:17.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.199+0000 7f0b9ffc5740 1 -- 172.21.15.16:0/656865001 shutdown_connections 2024-08-31T06:08:17.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.199+0000 7f0b9ffc5740 1 -- 172.21.15.16:0/656865001 wait complete. 2024-08-31T06:08:17.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.199+0000 7f0b9ffc5740 20 client.25789 trim_cache size 0 max 16384 2024-08-31T06:08:17.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.213+0000 7f0b9ffc5740 1 Processor -- start 2024-08-31T06:08:17.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.213+0000 7f0b9ffc5740 1 -- start start 2024-08-31T06:08:17.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.214+0000 7f0b9ffc5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4a6841b0 0x563e4aa561a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:17.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.214+0000 7f0b9ffc5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e4a68c390 0x563e4aa6a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:17.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.214+0000 7f0b9ffc5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e4a6ad070 0x563e4aa6a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:17.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.214+0000 7f0b9ffc5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0b941b3ec0 con 0x563e4a6841b0 2024-08-31T06:08:17.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.214+0000 7f0b9ffc5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0b941b1700 con 0x563e4a68c390 2024-08-31T06:08:17.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.214+0000 7f0b9ffc5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0b941af550 con 0x563e4a6ad070 2024-08-31T06:08:17.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.214+0000 7f0b91e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4a6841b0 0x563e4aa561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:17.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.214+0000 7f0b91e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4a6841b0 0x563e4aa561a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56322/0 (socket says 172.21.15.16:56322) 2024-08-31T06:08:17.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.214+0000 7f0b91e00640 1 -- 172.21.15.16:0/3119069372 learned_addr learned my addr 172.21.15.16:0/3119069372 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:17.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.214+0000 7f0b8be00640 1 --2- 172.21.15.16:0/3119069372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e4a68c390 0x563e4aa6a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:17.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.214+0000 7f0b92800640 1 --2- 172.21.15.16:0/3119069372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e4a6ad070 0x563e4aa6a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:17.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.215+0000 7f0b8be00640 1 -- 172.21.15.16:0/3119069372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e4a6ad070 msgr2=0x563e4aa6a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:17.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.215+0000 7f0b8be00640 1 --2- 172.21.15.16:0/3119069372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e4a6ad070 0x563e4aa6a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:17.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.215+0000 7f0b8be00640 1 -- 172.21.15.16:0/3119069372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4a6841b0 msgr2=0x563e4aa561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:17.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.215+0000 7f0b8be00640 1 --2- 172.21.15.16:0/3119069372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4a6841b0 0x563e4aa561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:17.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.215+0000 7f0b8be00640 1 -- 172.21.15.16:0/3119069372 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e4aa54c80 con 0x563e4a68c390 2024-08-31T06:08:17.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.215+0000 7f0b91e00640 1 --2- 172.21.15.16:0/3119069372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4a6841b0 0x563e4aa561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:08:17.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.215+0000 7f0b8be00640 1 --2- 172.21.15.16:0/3119069372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e4a68c390 0x563e4aa6a3d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c0c9540 tx=0x7f0b7c0ef040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb1cd468b70aa302 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:17.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.215+0000 7f0b82a00640 1 -- 172.21.15.16:0/3119069372 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c049040 con 0x563e4a68c390 2024-08-31T06:08:17.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.215+0000 7f0b82a00640 1 -- 172.21.15.16:0/3119069372 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b7c10b070 con 0x563e4a68c390 2024-08-31T06:08:17.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.215+0000 7f0b82a00640 1 -- 172.21.15.16:0/3119069372 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c0d5540 con 0x563e4a68c390 2024-08-31T06:08:17.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.216+0000 7f0b9ffc5740 1 -- 172.21.15.16:0/3119069372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e4a68c390 msgr2=0x563e4aa6a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:17.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.216+0000 7f0b9ffc5740 1 --2- 172.21.15.16:0/3119069372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e4a68c390 0x563e4aa6a3d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c0c9540 tx=0x7f0b7c0ef040 comp rx=0 tx=0).stop 2024-08-31T06:08:17.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.216+0000 7f0b9ffc5740 1 -- 172.21.15.16:0/3119069372 shutdown_connections 2024-08-31T06:08:17.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.216+0000 7f0b9ffc5740 1 --2- 172.21.15.16:0/3119069372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e4a6ad070 0x563e4aa6a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:17.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.216+0000 7f0b9ffc5740 1 --2- 172.21.15.16:0/3119069372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e4a68c390 0x563e4aa6a3d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:17.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.216+0000 7f0b9ffc5740 1 --2- 172.21.15.16:0/3119069372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4a6841b0 0x563e4aa561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:17.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.216+0000 7f0b9ffc5740 1 -- 172.21.15.16:0/3119069372 >> 172.21.15.16:0/3119069372 conn(0x563e4a6a1460 msgr2=0x563e4aa55a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:17.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.216+0000 7f0b9ffc5740 1 -- 172.21.15.16:0/3119069372 shutdown_connections 2024-08-31T06:08:17.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.216+0000 7f0b9ffc5740 1 -- 172.21.15.16:0/3119069372 wait complete. 2024-08-31T06:08:17.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.217+0000 7f0b9ffc5740 1 -- start start 2024-08-31T06:08:17.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.217+0000 7f0b9ffc5740 1 Processor -- start 2024-08-31T06:08:17.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.218+0000 7f0b9ffc5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4a6841b0 0x563e4aa71770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:17.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.218+0000 7f0b9ffc5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e4a68c390 0x563e4aa71cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:17.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.218+0000 7f0b9ffc5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e4a6ad070 0x563e4aa72230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:17.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.218+0000 7f0b9ffc5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0b941afbc0 con 0x563e4a6841b0 2024-08-31T06:08:17.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.218+0000 7f0b9ffc5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0b941b1030 con 0x563e4a68c390 2024-08-31T06:08:17.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.218+0000 7f0b9ffc5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0b941b0290 con 0x563e4a6ad070 2024-08-31T06:08:17.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.218+0000 7f0b9ffc5740 10 client.0 _ll_register_callbacks cb 0x7f0b840103d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:08:17.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.218+0000 7f0b91e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4a6841b0 0x563e4aa71770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:17.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.218+0000 7f0b91e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4a6841b0 0x563e4aa71770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56336/0 (socket says 172.21.15.16:56336) 2024-08-31T06:08:17.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.218+0000 7f0b91e00640 1 -- 172.21.15.16:0/1051156083 learned_addr learned my addr 172.21.15.16:0/1051156083 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:17.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.218+0000 7f0b8be00640 1 --2- 172.21.15.16:0/1051156083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e4a68c390 0x563e4aa71cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:17.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.218+0000 7f0b92800640 1 --2- 172.21.15.16:0/1051156083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e4a6ad070 0x563e4aa72230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:17.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.218+0000 7f0b91e00640 1 -- 172.21.15.16:0/1051156083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e4a6ad070 msgr2=0x563e4aa72230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:17.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.218+0000 7f0b91e00640 1 --2- 172.21.15.16:0/1051156083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e4a6ad070 0x563e4aa72230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:17.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.219+0000 7f0b91e00640 1 -- 172.21.15.16:0/1051156083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e4a68c390 msgr2=0x563e4aa71cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:17.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.219+0000 7f0b91e00640 1 --2- 172.21.15.16:0/1051156083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e4a68c390 0x563e4aa71cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:17.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.219+0000 7f0b91e00640 1 -- 172.21.15.16:0/1051156083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e4aad9ee0 con 0x563e4a6841b0 2024-08-31T06:08:17.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.219+0000 7f0b91e00640 1 --2- 172.21.15.16:0/1051156083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4a6841b0 0x563e4aa71770 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0b840104a0 tx=0x7f0b8400be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=949fc0033571af82 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:17.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.219+0000 7f0b77e00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:08:17.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.219+0000 7f0b77e00640 1 -- 172.21.15.16:0/1051156083 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8400ee20 con 0x563e4a6841b0 2024-08-31T06:08:17.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.219+0000 7f0b77e00640 1 -- 172.21.15.16:0/1051156083 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f0b84011440 con 0x563e4a6841b0 2024-08-31T06:08:17.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.219+0000 7f0b9ffc5740 1 -- 172.21.15.16:0/1051156083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b7c10b070 con 0x563e4a6841b0 2024-08-31T06:08:17.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.220+0000 7f0b77e00640 1 -- 172.21.15.16:0/1051156083 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0b84011640 con 0x563e4a6841b0 2024-08-31T06:08:17.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.220+0000 7f0b9ffc5740 10 client.15753 fetch_fsmap learned FSMap version 30 2024-08-31T06:08:17.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.220+0000 7f0b9ffc5740 1 -- 172.21.15.16:0/1051156083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563e4aadb160 con 0x563e4a6841b0 2024-08-31T06:08:17.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.220+0000 7f0b77e00640 1 -- 172.21.15.16:0/1051156083 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b7c10b070 con 0x563e4a6841b0 2024-08-31T06:08:17.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.220+0000 7f0b77e00640 1 -- 172.21.15.16:0/1051156083 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f0b8401a5a0 con 0x563e4a6841b0 2024-08-31T06:08:17.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.220+0000 7f0b9ffc5740 10 client.15753 fetch_fsmap finished waiting for FSMap version 30 2024-08-31T06:08:17.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.220+0000 7f0b9ffc5740 10 client.15753 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T06:08:17.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.220+0000 7f0b9ffc5740 20 client.15753 populate_metadata read hostname 'smithi016' 2024-08-31T06:08:17.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.220+0000 7f0b9ffc5740 1 --2- 172.21.15.16:0/1051156083 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x563e4a640da0 0x563e4aadd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:17.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.220+0000 7f0b9ffc5740 4 client.15753 mds_command: new command op to 24712 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:08:17.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.220+0000 7f0b9ffc5740 1 -- 172.21.15.16:0/1051156083 --> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563e4a9c71a0 con 0x563e4a640da0 2024-08-31T06:08:17.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.221+0000 7f0b8be00640 1 --2- 172.21.15.16:0/1051156083 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x563e4a640da0 0x563e4aadd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:17.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.221+0000 7f0b8be00640 1 --2- 172.21.15.16:0/1051156083 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x563e4a640da0 0x563e4aadd8b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b17a879980d1821c server_cookie=3d9af7e49ba699b0 in_seq=0 out_seq=0 2024-08-31T06:08:17.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.221+0000 7f0b77e00640 10 client.15753 ms_handle_connect on v2:172.21.15.184:6838/4011051479 2024-08-31T06:08:17.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.221+0000 7f0b77e00640 1 -- 172.21.15.16:0/1051156083 <== mds.2 v2:172.21.15.184:6838/4011051479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563e4a9c71a0 con 0x563e4a640da0 2024-08-31T06:08:17.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.221+0000 7f0b77e00640 10 client.15753 handle_command_reply: tid=0 2024-08-31T06:08:17.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.221+0000 7f0b9ffc5740 1 client.15753 shutdown 2024-08-31T06:08:17.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.222+0000 7f0b9ffc5740 1 -- 172.21.15.16:0/1051156083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4a6841b0 msgr2=0x563e4aa71770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:17.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.222+0000 7f0b9ffc5740 1 --2- 172.21.15.16:0/1051156083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4a6841b0 0x563e4aa71770 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0b840104a0 tx=0x7f0b8400be70 comp rx=0 tx=0).stop 2024-08-31T06:08:17.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.222+0000 7f0b9ffc5740 1 -- 172.21.15.16:0/1051156083 shutdown_connections 2024-08-31T06:08:17.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.222+0000 7f0b9ffc5740 1 --2- 172.21.15.16:0/1051156083 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x563e4a640da0 0x563e4aadd8b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:17.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.222+0000 7f0b9ffc5740 1 --2- 172.21.15.16:0/1051156083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e4a6ad070 0x563e4aa72230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:17.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.222+0000 7f0b9ffc5740 1 --2- 172.21.15.16:0/1051156083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e4a68c390 0x563e4aa71cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:17.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.222+0000 7f0b9ffc5740 1 --2- 172.21.15.16:0/1051156083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4a6841b0 0x563e4aa71770 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:17.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.222+0000 7f0b77e00640 0 client.15753 ms_handle_reset on v2:172.21.15.184:6838/4011051479 2024-08-31T06:08:17.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.222+0000 7f0b9ffc5740 1 -- 172.21.15.16:0/1051156083 >> 172.21.15.16:0/1051156083 conn(0x563e4a6a1460 msgr2=0x563e4aa539c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:17.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.222+0000 7f0b9ffc5740 1 -- 172.21.15.16:0/1051156083 shutdown_connections 2024-08-31T06:08:17.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.222+0000 7f0b9ffc5740 1 -- 172.21.15.16:0/1051156083 wait complete. 2024-08-31T06:08:17.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.222+0000 7f0b9ffc5740 20 client.15753 trim_cache size 0 max 16384 2024-08-31T06:08:17.228 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:08:17.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.223+0000 7f0b99200640 1 -- 172.21.15.16:0/366776535 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f0b780b4ca0 msgr2=0x7f0b780b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:17.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.223+0000 7f0b99200640 1 --2- 172.21.15.16:0/366776535 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f0b780b4ca0 0x7f0b780b71b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c005fd0 tx=0x7f0b8c009500 comp rx=0 tx=0).stop 2024-08-31T06:08:17.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.223+0000 7f0b99200640 1 -- 172.21.15.16:0/366776535 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b94109240 msgr2=0x7f0b9419c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:17.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.223+0000 7f0b99200640 1 --2- 172.21.15.16:0/366776535 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b94109240 0x7f0b9419c9f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c009230 tx=0x7f0b7c003c10 comp rx=0 tx=0).stop 2024-08-31T06:08:17.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.223+0000 7f0b99200640 1 -- 172.21.15.16:0/366776535 shutdown_connections 2024-08-31T06:08:17.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.223+0000 7f0b99200640 1 --2- 172.21.15.16:0/366776535 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f0b780b4ca0 0x7f0b780b71b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:17.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.223+0000 7f0b99200640 1 --2- 172.21.15.16:0/366776535 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b94109240 0x7f0b9419c9f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:17.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.223+0000 7f0b99200640 1 --2- 172.21.15.16:0/366776535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b94109b90 0x7f0b941a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:17.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.223+0000 7f0b99200640 1 --2- 172.21.15.16:0/366776535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b94103900 0x7f0b9419c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:17.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.223+0000 7f0b99200640 1 -- 172.21.15.16:0/366776535 >> 172.21.15.16:0/366776535 conn(0x7f0b9407b5f0 msgr2=0x7f0b94107050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:17.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.223+0000 7f0b99200640 1 -- 172.21.15.16:0/366776535 shutdown_connections 2024-08-31T06:08:17.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:17.223+0000 7f0b99200640 1 -- 172.21.15.16:0/366776535 wait complete. 2024-08-31T06:08:17.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:17 smithi155 ceph-mon[21742]: pgmap v395: 97 pgs: 97 active+clean; 940 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 28 MiB/s wr, 394 op/s 2024-08-31T06:08:17.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:17 smithi016 ceph-mon[17194]: pgmap v395: 97 pgs: 97 active+clean; 940 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 28 MiB/s wr, 394 op/s 2024-08-31T06:08:17.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:17 smithi184 ceph-mon[22331]: pgmap v395: 97 pgs: 97 active+clean; 940 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 28 MiB/s wr, 394 op/s 2024-08-31T06:08:18.014 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T06:08:18.398 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:08:18.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:18 smithi016 ceph-mon[17194]: pgmap v396: 97 pgs: 97 active+clean; 940 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 166 op/s 2024-08-31T06:08:18.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:18 smithi184 ceph-mon[22331]: pgmap v396: 97 pgs: 97 active+clean; 940 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 166 op/s 2024-08-31T06:08:18.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:18 smithi155 ceph-mon[21742]: pgmap v396: 97 pgs: 97 active+clean; 940 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 166 op/s 2024-08-31T06:08:19.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.018+0000 7f3ca1800640 1 -- 172.21.15.16:0/3390238981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c9c10b490 msgr2=0x7f3c9c10b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:19.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.018+0000 7f3ca1800640 1 --2- 172.21.15.16:0/3390238981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c9c10b490 0x7f3c9c10b8a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3c84009860 tx=0x7f3c8402a340 comp rx=0 tx=0).stop 2024-08-31T06:08:19.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.018+0000 7f3ca1800640 1 -- 172.21.15.16:0/3390238981 shutdown_connections 2024-08-31T06:08:19.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.018+0000 7f3ca1800640 1 --2- 172.21.15.16:0/3390238981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c9c10bde0 0x7f3c9c112610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:19.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.018+0000 7f3ca1800640 1 --2- 172.21.15.16:0/3390238981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c9c10b490 0x7f3c9c10b8a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:19.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.018+0000 7f3ca1800640 1 --2- 172.21.15.16:0/3390238981 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c9c105b50 0x7f3c9c105f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:19.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.018+0000 7f3ca1800640 1 -- 172.21.15.16:0/3390238981 >> 172.21.15.16:0/3390238981 conn(0x7f3c9c101580 msgr2=0x7f3c9c1039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:19.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.018+0000 7f3ca1800640 1 -- 172.21.15.16:0/3390238981 shutdown_connections 2024-08-31T06:08:19.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.018+0000 7f3ca1800640 1 -- 172.21.15.16:0/3390238981 wait complete. 2024-08-31T06:08:19.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.020+0000 7f3ca1800640 1 Processor -- start 2024-08-31T06:08:19.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.020+0000 7f3ca1800640 1 -- start start 2024-08-31T06:08:19.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.020+0000 7f3ca1800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c9c105b50 0x7f3c9c19c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:19.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.020+0000 7f3ca1800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c9c10b490 0x7f3c9c19ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:19.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.020+0000 7f3ca1800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c9c10bde0 0x7f3c9c1a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:19.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.020+0000 7f3ca1800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c115aa0 con 0x7f3c9c105b50 2024-08-31T06:08:19.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.020+0000 7f3ca1800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c115920 con 0x7f3c9c10b490 2024-08-31T06:08:19.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.020+0000 7f3ca1800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c115c20 con 0x7f3c9c10bde0 2024-08-31T06:08:19.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.021+0000 7f3c9aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c9c105b50 0x7f3c9c19c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:19.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.021+0000 7f3c9aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c9c105b50 0x7f3c9c19c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56352/0 (socket says 172.21.15.16:56352) 2024-08-31T06:08:19.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.021+0000 7f3c9aa00640 1 -- 172.21.15.16:0/1002834016 learned_addr learned my addr 172.21.15.16:0/1002834016 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:19.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.021+0000 7f3c93e00640 1 --2- 172.21.15.16:0/1002834016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c9c10b490 0x7f3c9c19ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:19.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.021+0000 7f3c9b400640 1 --2- 172.21.15.16:0/1002834016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c9c10bde0 0x7f3c9c1a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:19.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.021+0000 7f3c9aa00640 1 -- 172.21.15.16:0/1002834016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c9c10bde0 msgr2=0x7f3c9c1a1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:19.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.021+0000 7f3c9aa00640 1 --2- 172.21.15.16:0/1002834016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c9c10bde0 0x7f3c9c1a1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:19.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.021+0000 7f3c9aa00640 1 -- 172.21.15.16:0/1002834016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c9c10b490 msgr2=0x7f3c9c19ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:19.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.021+0000 7f3c9aa00640 1 --2- 172.21.15.16:0/1002834016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c9c10b490 0x7f3c9c19ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:19.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.021+0000 7f3c9aa00640 1 -- 172.21.15.16:0/1002834016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c9c1a2400 con 0x7f3c9c105b50 2024-08-31T06:08:19.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.022+0000 7f3c9aa00640 1 --2- 172.21.15.16:0/1002834016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c9c105b50 0x7f3c9c19c490 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c00e9f0 tx=0x7f3c8c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:19.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.022+0000 7f3c93400640 1 -- 172.21.15.16:0/1002834016 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c00cd80 con 0x7f3c9c105b50 2024-08-31T06:08:19.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.022+0000 7f3c93400640 1 -- 172.21.15.16:0/1002834016 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c8c0054d0 con 0x7f3c9c105b50 2024-08-31T06:08:19.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.022+0000 7f3c93400640 1 -- 172.21.15.16:0/1002834016 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c010690 con 0x7f3c9c105b50 2024-08-31T06:08:19.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.022+0000 7f3ca1800640 1 -- 172.21.15.16:0/1002834016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c9c1a26f0 con 0x7f3c9c105b50 2024-08-31T06:08:19.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.022+0000 7f3ca1800640 1 -- 172.21.15.16:0/1002834016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c9c1a8e50 con 0x7f3c9c105b50 2024-08-31T06:08:19.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.025+0000 7f3ca848f740 1 Processor -- start 2024-08-31T06:08:19.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.025+0000 7f3ca848f740 1 -- start start 2024-08-31T06:08:19.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.026+0000 7f3c93400640 1 -- 172.21.15.16:0/1002834016 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f3c8c004e90 con 0x7f3c9c105b50 2024-08-31T06:08:19.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.026+0000 7f3ca848f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c394f4070 0x562c39899990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:19.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.026+0000 7f3ca848f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c394cb1b0 0x562c39899ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:19.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.026+0000 7f3ca848f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c394d3390 0x562c3989a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:19.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.026+0000 7f3ca848f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562c398107b0 con 0x562c394f4070 2024-08-31T06:08:19.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.026+0000 7f3ca848f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562c397ffd40 con 0x562c394d3390 2024-08-31T06:08:19.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.026+0000 7f3ca848f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562c39803cf0 con 0x562c394cb1b0 2024-08-31T06:08:19.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.026+0000 7f3c9b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c394f4070 0x562c39899990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:19.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.026+0000 7f3c9b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c394f4070 0x562c39899990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56366/0 (socket says 172.21.15.16:56366) 2024-08-31T06:08:19.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.026+0000 7f3c9b400640 1 -- 172.21.15.16:0/3670787042 learned_addr learned my addr 172.21.15.16:0/3670787042 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:19.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.026+0000 7f3c9aa00640 1 --2- 172.21.15.16:0/3670787042 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c394cb1b0 0x562c39899ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:19.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.026+0000 7f3c93e00640 1 --2- 172.21.15.16:0/3670787042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c394d3390 0x562c3989a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:19.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.027+0000 7f3c9b400640 1 -- 172.21.15.16:0/3670787042 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c394cb1b0 msgr2=0x562c39899ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:19.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.027+0000 7f3c9b400640 1 --2- 172.21.15.16:0/3670787042 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c394cb1b0 0x562c39899ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:19.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.027+0000 7f3c9b400640 1 -- 172.21.15.16:0/3670787042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c394d3390 msgr2=0x562c3989a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:19.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.027+0000 7f3c9b400640 1 --2- 172.21.15.16:0/3670787042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c394d3390 0x562c3989a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:19.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.027+0000 7f3c9b400640 1 -- 172.21.15.16:0/3670787042 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c3955f650 con 0x562c394f4070 2024-08-31T06:08:19.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.027+0000 7f3c9b400640 1 --2- 172.21.15.16:0/3670787042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c394f4070 0x562c39899990 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3c9400e580 tx=0x7f3c940095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b90272eebb88510d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:19.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.027+0000 7f3c8be00640 1 -- 172.21.15.16:0/3670787042 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c94003970 con 0x562c394f4070 2024-08-31T06:08:19.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.028+0000 7f3c8be00640 1 -- 172.21.15.16:0/3670787042 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c9400d830 con 0x562c394f4070 2024-08-31T06:08:19.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.028+0000 7f3c8be00640 1 -- 172.21.15.16:0/3670787042 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c94004460 con 0x562c394f4070 2024-08-31T06:08:19.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.028+0000 7f3c93400640 1 --2- 172.21.15.16:0/1002834016 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3c800b4ca0 0x7f3c800b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:19.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.028+0000 7f3c93400640 1 -- 172.21.15.16:0/1002834016 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f3c8c0e5990 con 0x7f3c9c105b50 2024-08-31T06:08:19.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.028+0000 7f3c9aa00640 1 --2- 172.21.15.16:0/1002834016 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3c800b4ca0 0x7f3c800b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:19.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.029+0000 7f3ca848f740 1 -- 172.21.15.16:0/3670787042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c394f4070 msgr2=0x562c39899990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:19.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.029+0000 7f3ca848f740 1 --2- 172.21.15.16:0/3670787042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c394f4070 0x562c39899990 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3c9400e580 tx=0x7f3c940095c0 comp rx=0 tx=0).stop 2024-08-31T06:08:19.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.029+0000 7f3ca848f740 1 -- 172.21.15.16:0/3670787042 shutdown_connections 2024-08-31T06:08:19.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.029+0000 7f3ca848f740 1 --2- 172.21.15.16:0/3670787042 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c394cb1b0 0x562c39899ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:19.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.029+0000 7f3ca848f740 1 --2- 172.21.15.16:0/3670787042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c394d3390 0x562c3989a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:19.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.029+0000 7f3ca848f740 1 --2- 172.21.15.16:0/3670787042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c394f4070 0x562c39899990 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:19.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.029+0000 7f3ca848f740 1 -- 172.21.15.16:0/3670787042 >> 172.21.15.16:0/3670787042 conn(0x562c394e8460 msgr2=0x562c39897880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:19.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.029+0000 7f3c9aa00640 1 --2- 172.21.15.16:0/1002834016 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3c800b4ca0 0x7f3c800b71b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x562c39898610 tx=0x7f3c8c0eb530 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:19.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.029+0000 7f3ca848f740 1 -- 172.21.15.16:0/3670787042 shutdown_connections 2024-08-31T06:08:19.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.029+0000 7f3ca848f740 1 -- 172.21.15.16:0/3670787042 wait complete. 2024-08-31T06:08:19.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.029+0000 7f3ca848f740 1 -- start start 2024-08-31T06:08:19.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.030+0000 7f3ca848f740 1 Processor -- start 2024-08-31T06:08:19.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.030+0000 7f3ca848f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c394cb1b0 0x562c398a5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:19.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.030+0000 7f3ca848f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c394d3390 0x562c398a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:19.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.030+0000 7f3ca848f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c394f4070 0x562c398b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:19.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.030+0000 7f3ca848f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562c39662700 con 0x562c394d3390 2024-08-31T06:08:19.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.030+0000 7f3ca848f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562c39786cf0 con 0x562c394f4070 2024-08-31T06:08:19.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.030+0000 7f3ca848f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562c398a8170 con 0x562c394cb1b0 2024-08-31T06:08:19.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.030+0000 7f3ca848f740 10 client.0 _ll_register_callbacks cb 0x562c397ac470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:08:19.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.030+0000 7f3c93e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c394d3390 0x562c398a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:19.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.031+0000 7f3c93e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c394d3390 0x562c398a5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56374/0 (socket says 172.21.15.16:56374) 2024-08-31T06:08:19.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.031+0000 7f3c9b400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c394cb1b0 0x562c398a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:19.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.031+0000 7f3c93e00640 1 -- 172.21.15.16:0/2675084163 learned_addr learned my addr 172.21.15.16:0/2675084163 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:19.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.031+0000 7f3c9b400640 1 --2- 172.21.15.16:0/2675084163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c394f4070 0x562c398b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:19.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.031+0000 7f3c93e00640 1 -- 172.21.15.16:0/2675084163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c394cb1b0 msgr2=0x562c398a5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:19.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.031+0000 7f3c93e00640 1 --2- 172.21.15.16:0/2675084163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c394cb1b0 0x562c398a5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:19.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.031+0000 7f3c93e00640 1 -- 172.21.15.16:0/2675084163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c394f4070 msgr2=0x562c398b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:19.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.031+0000 7f3c93e00640 1 --2- 172.21.15.16:0/2675084163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c394f4070 0x562c398b1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:19.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.031+0000 7f3c93e00640 1 -- 172.21.15.16:0/2675084163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c397c78b0 con 0x562c394d3390 2024-08-31T06:08:19.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.031+0000 7f3c9b400640 1 --2- 172.21.15.16:0/2675084163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c394f4070 0x562c398b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:08:19.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.031+0000 7f3c93e00640 1 --2- 172.21.15.16:0/2675084163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c394d3390 0x562c398a5a40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3c84000c00 tx=0x7f3c84030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd880e3568c17aa server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:19.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.031+0000 7f3c88c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:08:19.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.032+0000 7f3c88c00640 1 -- 172.21.15.16:0/2675084163 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c84042070 con 0x562c394d3390 2024-08-31T06:08:19.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.032+0000 7f3c88c00640 1 -- 172.21.15.16:0/2675084163 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f3c84004240 con 0x562c394d3390 2024-08-31T06:08:19.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.032+0000 7f3ca848f740 1 -- 172.21.15.16:0/2675084163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c9400d830 con 0x562c394d3390 2024-08-31T06:08:19.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.032+0000 7f3c88c00640 1 -- 172.21.15.16:0/2675084163 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3c8404e050 con 0x562c394d3390 2024-08-31T06:08:19.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.032+0000 7f3ca848f740 10 client.15765 fetch_fsmap learned FSMap version 30 2024-08-31T06:08:19.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.032+0000 7f3ca848f740 1 -- 172.21.15.16:0/2675084163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562c398b4100 con 0x562c394d3390 2024-08-31T06:08:19.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.032+0000 7f3c88c00640 1 -- 172.21.15.16:0/2675084163 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c9400d830 con 0x562c394d3390 2024-08-31T06:08:19.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.033+0000 7f3c88c00640 1 -- 172.21.15.16:0/2675084163 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f3c84003b10 con 0x562c394d3390 2024-08-31T06:08:19.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.033+0000 7f3ca848f740 10 client.15765 fetch_fsmap finished waiting for FSMap version 30 2024-08-31T06:08:19.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.033+0000 7f3ca848f740 10 client.15765 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-31T06:08:19.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.033+0000 7f3ca848f740 20 client.15765 populate_metadata read hostname 'smithi016' 2024-08-31T06:08:19.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.033+0000 7f3ca848f740 1 --2- 172.21.15.16:0/2675084163 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x562c394b29f0 0x562c398b7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:19.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.033+0000 7f3ca848f740 4 client.15765 mds_command: new command op to 24724 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:08:19.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.033+0000 7f3ca848f740 1 -- 172.21.15.16:0/2675084163 --> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c39812700 con 0x562c394b29f0 2024-08-31T06:08:19.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.033+0000 7f3c9b400640 1 --2- 172.21.15.16:0/2675084163 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x562c394b29f0 0x562c398b7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:19.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.034+0000 7f3c9b400640 1 --2- 172.21.15.16:0/2675084163 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x562c394b29f0 0x562c398b7630 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8e012d4fa68924af server_cookie=e4e6778619cbdbaa in_seq=0 out_seq=0 2024-08-31T06:08:19.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.034+0000 7f3c88c00640 10 client.15765 ms_handle_connect on v2:172.21.15.16:6838/219664460 2024-08-31T06:08:19.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.034+0000 7f3c88c00640 1 -- 172.21.15.16:0/2675084163 <== mds.1 v2:172.21.15.16:6838/219664460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562c39812700 con 0x562c394b29f0 2024-08-31T06:08:19.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.034+0000 7f3c88c00640 10 client.15765 handle_command_reply: tid=0 2024-08-31T06:08:19.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.035+0000 7f3ca848f740 1 client.15765 shutdown 2024-08-31T06:08:19.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.035+0000 7f3ca848f740 1 -- 172.21.15.16:0/2675084163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c394d3390 msgr2=0x562c398a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:19.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.035+0000 7f3ca848f740 1 --2- 172.21.15.16:0/2675084163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c394d3390 0x562c398a5a40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3c84000c00 tx=0x7f3c84030d80 comp rx=0 tx=0).stop 2024-08-31T06:08:19.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.035+0000 7f3ca848f740 1 -- 172.21.15.16:0/2675084163 shutdown_connections 2024-08-31T06:08:19.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.035+0000 7f3ca848f740 1 --2- 172.21.15.16:0/2675084163 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x562c394b29f0 0x562c398b7630 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:19.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.035+0000 7f3ca848f740 1 --2- 172.21.15.16:0/2675084163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c394f4070 0x562c398b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:19.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.035+0000 7f3ca848f740 1 --2- 172.21.15.16:0/2675084163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c394d3390 0x562c398a5a40 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:19.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.035+0000 7f3ca848f740 1 --2- 172.21.15.16:0/2675084163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c394cb1b0 0x562c398a5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:19.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.035+0000 7f3ca848f740 1 -- 172.21.15.16:0/2675084163 >> 172.21.15.16:0/2675084163 conn(0x562c394e8460 msgr2=0x562c3989cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:19.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.035+0000 7f3c88c00640 0 client.15765 ms_handle_reset on v2:172.21.15.16:6838/219664460 2024-08-31T06:08:19.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.035+0000 7f3ca848f740 1 -- 172.21.15.16:0/2675084163 shutdown_connections 2024-08-31T06:08:19.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.035+0000 7f3ca848f740 1 -- 172.21.15.16:0/2675084163 wait complete. 2024-08-31T06:08:19.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.035+0000 7f3ca848f740 20 client.15765 trim_cache size 0 max 16384 2024-08-31T06:08:19.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.049+0000 7f3ca848f740 1 Processor -- start 2024-08-31T06:08:19.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.049+0000 7f3ca848f740 1 -- start start 2024-08-31T06:08:19.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.050+0000 7f3ca848f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c394cb1b0 0x562c398b1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:19.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.050+0000 7f3ca848f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c394d3390 0x562c398b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:19.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.050+0000 7f3ca848f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c394f4070 0x562c398a24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:19.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.050+0000 7f3ca848f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c1b3f70 con 0x562c394cb1b0 2024-08-31T06:08:19.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.050+0000 7f3ca848f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c1b17b0 con 0x562c394d3390 2024-08-31T06:08:19.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.050+0000 7f3ca848f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c1af600 con 0x562c394f4070 2024-08-31T06:08:19.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.050+0000 7f3c9b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c394cb1b0 0x562c398b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:19.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.050+0000 7f3c9b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c394cb1b0 0x562c398b1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56378/0 (socket says 172.21.15.16:56378) 2024-08-31T06:08:19.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.050+0000 7f3c9b400640 1 -- 172.21.15.16:0/3797294991 learned_addr learned my addr 172.21.15.16:0/3797294991 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:19.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.050+0000 7f3c93e00640 1 --2- 172.21.15.16:0/3797294991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c394d3390 0x562c398b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:19.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.050+0000 7f3c9b400640 1 --2- 172.21.15.16:0/3797294991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c394f4070 0x562c398a24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:19.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.050+0000 7f3c9b400640 1 -- 172.21.15.16:0/3797294991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c394f4070 msgr2=0x562c398a24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:19.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.050+0000 7f3c9b400640 1 --2- 172.21.15.16:0/3797294991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c394f4070 0x562c398a24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:19.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.050+0000 7f3c9b400640 1 -- 172.21.15.16:0/3797294991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c394d3390 msgr2=0x562c398b18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:19.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.050+0000 7f3c9b400640 1 --2- 172.21.15.16:0/3797294991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c394d3390 0x562c398b18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:19.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.050+0000 7f3c9b400640 1 -- 172.21.15.16:0/3797294991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c398a65c0 con 0x562c394cb1b0 2024-08-31T06:08:19.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.051+0000 7f3c9b400640 1 --2- 172.21.15.16:0/3797294991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c394cb1b0 0x562c398b1390 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3c9400e550 tx=0x7f3c94005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d68c6fbde1e41ba server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:19.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.051+0000 7f3c8b400640 1 -- 172.21.15.16:0/3797294991 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9400d830 con 0x562c394cb1b0 2024-08-31T06:08:19.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.051+0000 7f3c8b400640 1 -- 172.21.15.16:0/3797294991 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c9400dce0 con 0x562c394cb1b0 2024-08-31T06:08:19.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.051+0000 7f3c8b400640 1 -- 172.21.15.16:0/3797294991 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c94003c30 con 0x562c394cb1b0 2024-08-31T06:08:19.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.052+0000 7f3ca848f740 1 -- 172.21.15.16:0/3797294991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c394cb1b0 msgr2=0x562c398b1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:19.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.052+0000 7f3ca848f740 1 --2- 172.21.15.16:0/3797294991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c394cb1b0 0x562c398b1390 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3c9400e550 tx=0x7f3c94005fd0 comp rx=0 tx=0).stop 2024-08-31T06:08:19.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.052+0000 7f3ca848f740 1 -- 172.21.15.16:0/3797294991 shutdown_connections 2024-08-31T06:08:19.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.052+0000 7f3ca848f740 1 --2- 172.21.15.16:0/3797294991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c394f4070 0x562c398a24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:19.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.052+0000 7f3ca848f740 1 --2- 172.21.15.16:0/3797294991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c394d3390 0x562c398b18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:19.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.052+0000 7f3ca848f740 1 --2- 172.21.15.16:0/3797294991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c394cb1b0 0x562c398b1390 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:19.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.052+0000 7f3ca848f740 1 -- 172.21.15.16:0/3797294991 >> 172.21.15.16:0/3797294991 conn(0x562c394e8460 msgr2=0x562c398a3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:19.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.052+0000 7f3ca848f740 1 -- 172.21.15.16:0/3797294991 shutdown_connections 2024-08-31T06:08:19.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.052+0000 7f3ca848f740 1 -- 172.21.15.16:0/3797294991 wait complete. 2024-08-31T06:08:19.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.052+0000 7f3ca848f740 1 -- start start 2024-08-31T06:08:19.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.053+0000 7f3ca848f740 1 Processor -- start 2024-08-31T06:08:19.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.053+0000 7f3ca848f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c394cb1b0 0x562c398a9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:19.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.053+0000 7f3ca848f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c394d3390 0x562c398aa110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:19.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.053+0000 7f3ca848f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c394f4070 0x562c39921030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:19.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.053+0000 7f3ca848f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c1afc70 con 0x562c394d3390 2024-08-31T06:08:19.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.053+0000 7f3ca848f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c1b10e0 con 0x562c394cb1b0 2024-08-31T06:08:19.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.053+0000 7f3ca848f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c1b0340 con 0x562c394f4070 2024-08-31T06:08:19.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.053+0000 7f3ca848f740 10 client.0 _ll_register_callbacks cb 0x7f3c84005cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:08:19.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.053+0000 7f3c93e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c394d3390 0x562c398aa110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:19.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.053+0000 7f3c93e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c394d3390 0x562c398aa110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56386/0 (socket says 172.21.15.16:56386) 2024-08-31T06:08:19.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.053+0000 7f3c93e00640 1 -- 172.21.15.16:0/1884040733 learned_addr learned my addr 172.21.15.16:0/1884040733 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:19.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.054+0000 7f3c9b400640 1 --2- 172.21.15.16:0/1884040733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c394cb1b0 0x562c398a9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:19.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.054+0000 7f3c9b400640 1 --2- 172.21.15.16:0/1884040733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c394f4070 0x562c39921030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:19.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.054+0000 7f3c93e00640 1 -- 172.21.15.16:0/1884040733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c394f4070 msgr2=0x562c39921030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:19.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.054+0000 7f3c93e00640 1 --2- 172.21.15.16:0/1884040733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c394f4070 0x562c39921030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:19.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.054+0000 7f3c93e00640 1 -- 172.21.15.16:0/1884040733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c394cb1b0 msgr2=0x562c398a9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:19.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.054+0000 7f3c93e00640 1 --2- 172.21.15.16:0/1884040733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c394cb1b0 0x562c398a9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:19.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.054+0000 7f3c93e00640 1 -- 172.21.15.16:0/1884040733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c39921620 con 0x562c394d3390 2024-08-31T06:08:19.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.054+0000 7f3c9b400640 1 --2- 172.21.15.16:0/1884040733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c394cb1b0 0x562c398a9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:08:19.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.054+0000 7f3c93e00640 1 --2- 172.21.15.16:0/1884040733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c394d3390 0x562c398aa110 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3c84003ee0 tx=0x7f3c84004210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6876b29d6fb7ed6d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:19.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.054+0000 7f3c88c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:08:19.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.054+0000 7f3c88c00640 1 -- 172.21.15.16:0/1884040733 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c84057060 con 0x562c394d3390 2024-08-31T06:08:19.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.055+0000 7f3ca848f740 1 -- 172.21.15.16:0/1884040733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c9400dce0 con 0x562c394d3390 2024-08-31T06:08:19.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.055+0000 7f3c88c00640 1 -- 172.21.15.16:0/1884040733 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f3c84030650 con 0x562c394d3390 2024-08-31T06:08:19.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.055+0000 7f3c88c00640 1 -- 172.21.15.16:0/1884040733 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3c84042040 con 0x562c394d3390 2024-08-31T06:08:19.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.055+0000 7f3ca848f740 10 client.15771 fetch_fsmap learned FSMap version 30 2024-08-31T06:08:19.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.055+0000 7f3ca848f740 1 -- 172.21.15.16:0/1884040733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562c399228d0 con 0x562c394d3390 2024-08-31T06:08:19.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.055+0000 7f3c88c00640 1 -- 172.21.15.16:0/1884040733 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c9400dce0 con 0x562c394d3390 2024-08-31T06:08:19.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.055+0000 7f3c88c00640 1 -- 172.21.15.16:0/1884040733 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f3c84005240 con 0x562c394d3390 2024-08-31T06:08:19.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.055+0000 7f3ca848f740 10 client.15771 fetch_fsmap finished waiting for FSMap version 30 2024-08-31T06:08:19.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.055+0000 7f3ca848f740 10 client.15771 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-31T06:08:19.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.055+0000 7f3ca848f740 20 client.15771 populate_metadata read hostname 'smithi016' 2024-08-31T06:08:19.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.055+0000 7f3ca848f740 1 --2- 172.21.15.16:0/1884040733 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x562c39487da0 0x562c399250f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:19.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.056+0000 7f3ca848f740 4 client.15771 mds_command: new command op to 24724 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:08:19.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.056+0000 7f3ca848f740 1 -- 172.21.15.16:0/1884040733 --> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562c3980e1a0 con 0x562c39487da0 2024-08-31T06:08:19.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.056+0000 7f3c9b400640 1 --2- 172.21.15.16:0/1884040733 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x562c39487da0 0x562c399250f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:19.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.056+0000 7f3c9b400640 1 --2- 172.21.15.16:0/1884040733 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x562c39487da0 0x562c399250f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8233b61b9925078c server_cookie=a624453895eaa371 in_seq=0 out_seq=0 2024-08-31T06:08:19.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.056+0000 7f3c88c00640 10 client.15771 ms_handle_connect on v2:172.21.15.16:6838/219664460 2024-08-31T06:08:19.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.057+0000 7f3c88c00640 1 -- 172.21.15.16:0/1884040733 <== mds.1 v2:172.21.15.16:6838/219664460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562c3980e1a0 con 0x562c39487da0 2024-08-31T06:08:19.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.057+0000 7f3c88c00640 10 client.15771 handle_command_reply: tid=0 2024-08-31T06:08:19.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.057+0000 7f3ca848f740 1 client.15771 shutdown 2024-08-31T06:08:19.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.057+0000 7f3ca848f740 1 -- 172.21.15.16:0/1884040733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c394d3390 msgr2=0x562c398aa110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:19.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.057+0000 7f3ca848f740 1 --2- 172.21.15.16:0/1884040733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c394d3390 0x562c398aa110 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3c84003ee0 tx=0x7f3c84004210 comp rx=0 tx=0).stop 2024-08-31T06:08:19.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.057+0000 7f3ca848f740 1 -- 172.21.15.16:0/1884040733 shutdown_connections 2024-08-31T06:08:19.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.057+0000 7f3ca848f740 1 --2- 172.21.15.16:0/1884040733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562c394f4070 0x562c39921030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:19.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.057+0000 7f3ca848f740 1 --2- 172.21.15.16:0/1884040733 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x562c39487da0 0x562c399250f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:19.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.057+0000 7f3ca848f740 1 --2- 172.21.15.16:0/1884040733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c394d3390 0x562c398aa110 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:19.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.057+0000 7f3ca848f740 1 --2- 172.21.15.16:0/1884040733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c394cb1b0 0x562c398a9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:19.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.057+0000 7f3ca848f740 1 -- 172.21.15.16:0/1884040733 >> 172.21.15.16:0/1884040733 conn(0x562c394e8460 msgr2=0x562c398a6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:19.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.057+0000 7f3c88c00640 0 client.15771 ms_handle_reset on v2:172.21.15.16:6838/219664460 2024-08-31T06:08:19.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.057+0000 7f3ca848f740 1 -- 172.21.15.16:0/1884040733 shutdown_connections 2024-08-31T06:08:19.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.057+0000 7f3ca848f740 1 -- 172.21.15.16:0/1884040733 wait complete. 2024-08-31T06:08:19.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.057+0000 7f3ca848f740 20 client.15771 trim_cache size 0 max 16384 2024-08-31T06:08:19.063 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:08:19.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.058+0000 7f3ca1800640 1 -- 172.21.15.16:0/1002834016 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3c800b4ca0 msgr2=0x7f3c800b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:19.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.058+0000 7f3ca1800640 1 --2- 172.21.15.16:0/1002834016 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3c800b4ca0 0x7f3c800b71b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x562c39898610 tx=0x7f3c8c0eb530 comp rx=0 tx=0).stop 2024-08-31T06:08:19.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.058+0000 7f3ca1800640 1 -- 172.21.15.16:0/1002834016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c9c105b50 msgr2=0x7f3c9c19c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:19.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.058+0000 7f3ca1800640 1 --2- 172.21.15.16:0/1002834016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c9c105b50 0x7f3c9c19c490 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c00e9f0 tx=0x7f3c8c00edf0 comp rx=0 tx=0).stop 2024-08-31T06:08:19.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.058+0000 7f3ca1800640 1 -- 172.21.15.16:0/1002834016 shutdown_connections 2024-08-31T06:08:19.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.058+0000 7f3ca1800640 1 --2- 172.21.15.16:0/1002834016 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3c800b4ca0 0x7f3c800b71b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:19.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.058+0000 7f3ca1800640 1 --2- 172.21.15.16:0/1002834016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c9c10bde0 0x7f3c9c1a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:19.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.058+0000 7f3ca1800640 1 --2- 172.21.15.16:0/1002834016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c9c10b490 0x7f3c9c19ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:19.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.058+0000 7f3ca1800640 1 --2- 172.21.15.16:0/1002834016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c9c105b50 0x7f3c9c19c490 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:19.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.058+0000 7f3ca1800640 1 -- 172.21.15.16:0/1002834016 >> 172.21.15.16:0/1002834016 conn(0x7f3c9c101580 msgr2=0x7f3c9c108ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:19.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.058+0000 7f3ca1800640 1 -- 172.21.15.16:0/1002834016 shutdown_connections 2024-08-31T06:08:19.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:19.058+0000 7f3ca1800640 1 -- 172.21.15.16:0/1002834016 wait complete. 2024-08-31T06:08:19.941 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T06:08:20.322 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:08:20.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:20 smithi016 ceph-mon[17194]: pgmap v397: 97 pgs: 97 active+clean; 940 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.4 MiB/s wr, 95 op/s 2024-08-31T06:08:20.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:20 smithi184 ceph-mon[22331]: pgmap v397: 97 pgs: 97 active+clean; 940 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.4 MiB/s wr, 95 op/s 2024-08-31T06:08:20.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:20 smithi155 ceph-mon[21742]: pgmap v397: 97 pgs: 97 active+clean; 940 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.4 MiB/s wr, 95 op/s 2024-08-31T06:08:21.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.189+0000 7ff0a0000640 1 -- 172.21.15.16:0/1567034526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff098109240 msgr2=0x7ff098109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:21.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.189+0000 7ff0a0000640 1 --2- 172.21.15.16:0/1567034526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff098109240 0x7ff098109650 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff084009900 tx=0x7ff08402a410 comp rx=0 tx=0).stop 2024-08-31T06:08:21.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.189+0000 7ff0a0000640 1 -- 172.21.15.16:0/1567034526 shutdown_connections 2024-08-31T06:08:21.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.189+0000 7ff0a0000640 1 --2- 172.21.15.16:0/1567034526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff098109b90 0x7ff0981103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:21.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.189+0000 7ff0a0000640 1 --2- 172.21.15.16:0/1567034526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff098109240 0x7ff098109650 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:21.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.189+0000 7ff0a0000640 1 --2- 172.21.15.16:0/1567034526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff098103900 0x7ff098103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:21.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.189+0000 7ff0a0000640 1 -- 172.21.15.16:0/1567034526 >> 172.21.15.16:0/1567034526 conn(0x7ff09807b5f0 msgr2=0x7ff09807ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:21.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.189+0000 7ff0a0000640 1 -- 172.21.15.16:0/1567034526 shutdown_connections 2024-08-31T06:08:21.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.189+0000 7ff0a0000640 1 -- 172.21.15.16:0/1567034526 wait complete. 2024-08-31T06:08:21.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.190+0000 7ff0a0000640 1 Processor -- start 2024-08-31T06:08:21.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.190+0000 7ff0a0000640 1 -- start start 2024-08-31T06:08:21.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.191+0000 7ff0a0000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff098103900 0x7ff098076ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:21.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.191+0000 7ff0a0000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff098109240 0x7ff0980754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:21.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.191+0000 7ff0a0000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff098109b90 0x7ff0980759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:21.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.191+0000 7ff0a0000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff098113850 con 0x7ff098109b90 2024-08-31T06:08:21.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.191+0000 7ff0a0000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff0981136d0 con 0x7ff098103900 2024-08-31T06:08:21.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.191+0000 7ff0a0000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff0981139d0 con 0x7ff098109240 2024-08-31T06:08:21.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.191+0000 7ff09d600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff098103900 0x7ff098076ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:21.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.191+0000 7ff09e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff098109b90 0x7ff0980759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:21.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.191+0000 7ff09d600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff098103900 0x7ff098076ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:54924/0 (socket says 172.21.15.16:54924) 2024-08-31T06:08:21.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.191+0000 7ff09d600640 1 -- 172.21.15.16:0/3191307701 learned_addr learned my addr 172.21.15.16:0/3191307701 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:21.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.191+0000 7ff09cc00640 1 --2- 172.21.15.16:0/3191307701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff098109240 0x7ff0980754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:21.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.192+0000 7ff09d600640 1 -- 172.21.15.16:0/3191307701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff098109240 msgr2=0x7ff0980754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:21.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.192+0000 7ff09d600640 1 --2- 172.21.15.16:0/3191307701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff098109240 0x7ff0980754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:21.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.192+0000 7ff09d600640 1 -- 172.21.15.16:0/3191307701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff098109b90 msgr2=0x7ff0980759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:21.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.192+0000 7ff09d600640 1 --2- 172.21.15.16:0/3191307701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff098109b90 0x7ff0980759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:21.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.192+0000 7ff09d600640 1 -- 172.21.15.16:0/3191307701 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff098076210 con 0x7ff098103900 2024-08-31T06:08:21.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.192+0000 7ff09d600640 1 --2- 172.21.15.16:0/3191307701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff098103900 0x7ff098076ec0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00e9f0 tx=0x7ff08c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:21.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.193+0000 7ff096000640 1 -- 172.21.15.16:0/3191307701 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c00ccc0 con 0x7ff098103900 2024-08-31T06:08:21.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.193+0000 7ff096000640 1 -- 172.21.15.16:0/3191307701 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff08c00ce60 con 0x7ff098103900 2024-08-31T06:08:21.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.193+0000 7ff096000640 1 -- 172.21.15.16:0/3191307701 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c010430 con 0x7ff098103900 2024-08-31T06:08:21.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.193+0000 7ff0a0000640 1 -- 172.21.15.16:0/3191307701 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0981acc20 con 0x7ff098103900 2024-08-31T06:08:21.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.193+0000 7ff0a0000640 1 -- 172.21.15.16:0/3191307701 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0981ad0e0 con 0x7ff098103900 2024-08-31T06:08:21.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.196+0000 7ff0a6c3a740 1 Processor -- start 2024-08-31T06:08:21.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.196+0000 7ff0a6c3a740 1 -- start start 2024-08-31T06:08:21.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.196+0000 7ff0a6c3a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e8d8ae070 0x557e8dc538b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:21.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.197+0000 7ff0a6c3a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e8d8851b0 0x557e8dc53df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:21.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.197+0000 7ff0a6c3a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e8d88d390 0x557e8dc54330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:21.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.197+0000 7ff0a6c3a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557e8dbca7b0 con 0x557e8d88d390 2024-08-31T06:08:21.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.197+0000 7ff0a6c3a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557e8dbb9d40 con 0x557e8d8ae070 2024-08-31T06:08:21.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.197+0000 7ff0a6c3a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557e8dbbdcf0 con 0x557e8d8851b0 2024-08-31T06:08:21.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.197+0000 7ff09e000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e8d8ae070 0x557e8dc538b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:21.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.197+0000 7ff09e000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e8d8ae070 0x557e8dc538b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:54934/0 (socket says 172.21.15.16:54934) 2024-08-31T06:08:21.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.197+0000 7ff09e000640 1 -- 172.21.15.16:0/3256372429 learned_addr learned my addr 172.21.15.16:0/3256372429 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:21.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.197+0000 7ff09cc00640 1 --2- 172.21.15.16:0/3256372429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e8d88d390 0x557e8dc54330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:21.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.197+0000 7ff096000640 1 -- 172.21.15.16:0/3191307701 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7ff08c0046c0 con 0x7ff098103900 2024-08-31T06:08:21.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.197+0000 7ff09e000640 1 -- 172.21.15.16:0/3256372429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e8d8851b0 msgr2=0x557e8dc53df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:08:21.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.197+0000 7ff09e000640 1 --2- 172.21.15.16:0/3256372429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e8d8851b0 0x557e8dc53df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:21.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.197+0000 7ff09e000640 1 -- 172.21.15.16:0/3256372429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e8d88d390 msgr2=0x557e8dc54330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:21.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.197+0000 7ff09e000640 1 --2- 172.21.15.16:0/3256372429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e8d88d390 0x557e8dc54330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:21.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.197+0000 7ff09e000640 1 -- 172.21.15.16:0/3256372429 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e8d919650 con 0x557e8d8ae070 2024-08-31T06:08:21.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.198+0000 7ff09cc00640 1 --2- 172.21.15.16:0/3256372429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e8d88d390 0x557e8dc54330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:08:21.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.198+0000 7ff09e000640 1 --2- 172.21.15.16:0/3256372429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e8d8ae070 0x557e8dc538b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff090009fa0 tx=0x7ff0900118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28781dd829fed53f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:21.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.198+0000 7ff08a600640 1 -- 172.21.15.16:0/3256372429 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff090004d20 con 0x557e8d8ae070 2024-08-31T06:08:21.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.198+0000 7ff08a600640 1 -- 172.21.15.16:0/3256372429 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff090004050 con 0x557e8d8ae070 2024-08-31T06:08:21.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.198+0000 7ff08a600640 1 -- 172.21.15.16:0/3256372429 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff090005260 con 0x557e8d8ae070 2024-08-31T06:08:21.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.199+0000 7ff0a6c3a740 1 -- 172.21.15.16:0/3256372429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e8d8ae070 msgr2=0x557e8dc538b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:21.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.199+0000 7ff0a6c3a740 1 --2- 172.21.15.16:0/3256372429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e8d8ae070 0x557e8dc538b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff090009fa0 tx=0x7ff0900118d0 comp rx=0 tx=0).stop 2024-08-31T06:08:21.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.199+0000 7ff0a6c3a740 1 -- 172.21.15.16:0/3256372429 shutdown_connections 2024-08-31T06:08:21.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.199+0000 7ff0a6c3a740 1 --2- 172.21.15.16:0/3256372429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e8d8851b0 0x557e8dc53df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:21.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.199+0000 7ff0a6c3a740 1 --2- 172.21.15.16:0/3256372429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e8d88d390 0x557e8dc54330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:21.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.199+0000 7ff0a6c3a740 1 --2- 172.21.15.16:0/3256372429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e8d8ae070 0x557e8dc538b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:21.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.199+0000 7ff0a6c3a740 1 -- 172.21.15.16:0/3256372429 >> 172.21.15.16:0/3256372429 conn(0x557e8d8a2460 msgr2=0x557e8dc517a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:21.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.199+0000 7ff0a6c3a740 1 -- 172.21.15.16:0/3256372429 shutdown_connections 2024-08-31T06:08:21.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.199+0000 7ff0a6c3a740 1 -- 172.21.15.16:0/3256372429 wait complete. 2024-08-31T06:08:21.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.199+0000 7ff096000640 1 --2- 172.21.15.16:0/3191307701 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff0800b4c30 0x7ff0800b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:21.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.199+0000 7ff096000640 1 -- 172.21.15.16:0/3191307701 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7ff08c0e5840 con 0x7ff098103900 2024-08-31T06:08:21.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.200+0000 7ff09cc00640 1 --2- 172.21.15.16:0/3191307701 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff0800b4c30 0x7ff0800b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:21.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.200+0000 7ff0a6c3a740 1 -- start start 2024-08-31T06:08:21.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.200+0000 7ff0a6c3a740 1 Processor -- start 2024-08-31T06:08:21.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.200+0000 7ff09cc00640 1 --2- 172.21.15.16:0/3191307701 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff0800b4c30 0x7ff0800b7140 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff084009230 tx=0x7ff084031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:21.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.201+0000 7ff0a6c3a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e8d8851b0 0x557e8dc5f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:21.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.201+0000 7ff0a6c3a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e8d88d390 0x557e8dc5f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:21.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.201+0000 7ff0a6c3a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e8d8ae070 0x557e8dc6b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:21.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.201+0000 7ff0a6c3a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557e8da1c700 con 0x557e8d8851b0 2024-08-31T06:08:21.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.201+0000 7ff0a6c3a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557e8db40cf0 con 0x557e8d88d390 2024-08-31T06:08:21.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.201+0000 7ff0a6c3a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557e8dc62090 con 0x557e8d8ae070 2024-08-31T06:08:21.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.201+0000 7ff0a6c3a740 10 client.0 _ll_register_callbacks cb 0x557e8db66470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:08:21.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.201+0000 7ff09d600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e8d8851b0 0x557e8dc5f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:21.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.201+0000 7ff09d600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e8d8851b0 0x557e8dc5f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56420/0 (socket says 172.21.15.16:56420) 2024-08-31T06:08:21.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.201+0000 7ff09d600640 1 -- 172.21.15.16:0/766061930 learned_addr learned my addr 172.21.15.16:0/766061930 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:21.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.201+0000 7ff09cc00640 1 --2- 172.21.15.16:0/766061930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e8d88d390 0x557e8dc5f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:21.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.201+0000 7ff09e000640 1 --2- 172.21.15.16:0/766061930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e8d8ae070 0x557e8dc6b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:21.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.201+0000 7ff09d600640 1 -- 172.21.15.16:0/766061930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e8d8ae070 msgr2=0x557e8dc6b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:21.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.201+0000 7ff09d600640 1 --2- 172.21.15.16:0/766061930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e8d8ae070 0x557e8dc6b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:21.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.201+0000 7ff09d600640 1 -- 172.21.15.16:0/766061930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e8d88d390 msgr2=0x557e8dc5f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:21.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.201+0000 7ff09d600640 1 --2- 172.21.15.16:0/766061930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e8d88d390 0x557e8dc5f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:21.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.201+0000 7ff09d600640 1 -- 172.21.15.16:0/766061930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e8db818b0 con 0x557e8d8851b0 2024-08-31T06:08:21.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.202+0000 7ff09d600640 1 --2- 172.21.15.16:0/766061930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e8d8851b0 0x557e8dc5f470 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff08c0e30a0 tx=0x7ff08c0ab7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b37e035dde28d87a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:21.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.202+0000 7ff07ea00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:08:21.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.202+0000 7ff07ea00640 1 -- 172.21.15.16:0/766061930 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c0abf90 con 0x557e8d8851b0 2024-08-31T06:08:21.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.202+0000 7ff07ea00640 1 -- 172.21.15.16:0/766061930 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7ff08c0b5030 con 0x557e8d8851b0 2024-08-31T06:08:21.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.202+0000 7ff0a6c3a740 1 -- 172.21.15.16:0/766061930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff090004050 con 0x557e8d8851b0 2024-08-31T06:08:21.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.202+0000 7ff07ea00640 1 -- 172.21.15.16:0/766061930 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff08c0b0800 con 0x557e8d8851b0 2024-08-31T06:08:21.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.202+0000 7ff0a6c3a740 10 client.15783 fetch_fsmap learned FSMap version 30 2024-08-31T06:08:21.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.202+0000 7ff0a6c3a740 1 -- 172.21.15.16:0/766061930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557e8dc6e170 con 0x557e8d8851b0 2024-08-31T06:08:21.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.203+0000 7ff07ea00640 1 -- 172.21.15.16:0/766061930 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff090004050 con 0x557e8d8851b0 2024-08-31T06:08:21.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.203+0000 7ff07ea00640 1 -- 172.21.15.16:0/766061930 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7ff08c0930e0 con 0x557e8d8851b0 2024-08-31T06:08:21.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.203+0000 7ff0a6c3a740 10 client.15783 fetch_fsmap finished waiting for FSMap version 30 2024-08-31T06:08:21.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.203+0000 7ff0a6c3a740 10 client.15783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:08:21.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.203+0000 7ff0a6c3a740 20 client.15783 populate_metadata read hostname 'smithi016' 2024-08-31T06:08:21.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.203+0000 7ff0a6c3a740 1 --2- 172.21.15.16:0/766061930 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557e8d86c9f0 0x557e8dc71680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:21.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.203+0000 7ff0a6c3a740 4 client.15783 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:08:21.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.203+0000 7ff0a6c3a740 1 -- 172.21.15.16:0/766061930 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e8dbcc700 con 0x557e8d86c9f0 2024-08-31T06:08:21.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.203+0000 7ff09cc00640 1 --2- 172.21.15.16:0/766061930 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557e8d86c9f0 0x557e8dc71680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:21.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.204+0000 7ff09cc00640 1 --2- 172.21.15.16:0/766061930 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557e8d86c9f0 0x557e8dc71680 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a99734fb65d4be59 server_cookie=1a1acb4641c4324f in_seq=0 out_seq=0 2024-08-31T06:08:21.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.204+0000 7ff07ea00640 10 client.15783 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:21.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.205+0000 7ff07ea00640 1 -- 172.21.15.16:0/766061930 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557e8dbcc700 con 0x557e8d86c9f0 2024-08-31T06:08:21.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.205+0000 7ff07ea00640 10 client.15783 handle_command_reply: tid=0 2024-08-31T06:08:21.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.205+0000 7ff0a6c3a740 1 client.15783 shutdown 2024-08-31T06:08:21.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.205+0000 7ff0a6c3a740 1 -- 172.21.15.16:0/766061930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e8d8851b0 msgr2=0x557e8dc5f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:21.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.205+0000 7ff0a6c3a740 1 --2- 172.21.15.16:0/766061930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e8d8851b0 0x557e8dc5f470 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff08c0e30a0 tx=0x7ff08c0ab7c0 comp rx=0 tx=0).stop 2024-08-31T06:08:21.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.205+0000 7ff0a6c3a740 1 -- 172.21.15.16:0/766061930 shutdown_connections 2024-08-31T06:08:21.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.205+0000 7ff0a6c3a740 1 --2- 172.21.15.16:0/766061930 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557e8d86c9f0 0x557e8dc71680 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:21.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.205+0000 7ff0a6c3a740 1 --2- 172.21.15.16:0/766061930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e8d8ae070 0x557e8dc6b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:21.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.205+0000 7ff0a6c3a740 1 --2- 172.21.15.16:0/766061930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e8d88d390 0x557e8dc5f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:21.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.205+0000 7ff0a6c3a740 1 --2- 172.21.15.16:0/766061930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e8d8851b0 0x557e8dc5f470 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:21.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.205+0000 7ff0a6c3a740 1 -- 172.21.15.16:0/766061930 >> 172.21.15.16:0/766061930 conn(0x557e8d8a2460 msgr2=0x557e8dc56a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:21.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.205+0000 7ff07ea00640 0 client.15783 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:21.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.205+0000 7ff0a6c3a740 1 -- 172.21.15.16:0/766061930 shutdown_connections 2024-08-31T06:08:21.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.206+0000 7ff0a6c3a740 1 -- 172.21.15.16:0/766061930 wait complete. 2024-08-31T06:08:21.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.206+0000 7ff0a6c3a740 20 client.15783 trim_cache size 0 max 16384 2024-08-31T06:08:21.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.220+0000 7ff0a6c3a740 1 Processor -- start 2024-08-31T06:08:21.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.220+0000 7ff0a6c3a740 1 -- start start 2024-08-31T06:08:21.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.220+0000 7ff0a6c3a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e8d8851b0 0x557e8dc599d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:21.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.220+0000 7ff0a6c3a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e8d88d390 0x557e8dc51030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:21.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.220+0000 7ff0a6c3a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e8d8ae070 0x557e8dc51570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:21.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.220+0000 7ff0a6c3a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff0981b81f0 con 0x557e8d8ae070 2024-08-31T06:08:21.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.220+0000 7ff0a6c3a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff0981b5a30 con 0x557e8d8851b0 2024-08-31T06:08:21.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.220+0000 7ff0a6c3a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff0981b3880 con 0x557e8d88d390 2024-08-31T06:08:21.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.220+0000 7ff09e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e8d8ae070 0x557e8dc51570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:21.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.221+0000 7ff09e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e8d8ae070 0x557e8dc51570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56434/0 (socket says 172.21.15.16:56434) 2024-08-31T06:08:21.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.221+0000 7ff09e000640 1 -- 172.21.15.16:0/3615060381 learned_addr learned my addr 172.21.15.16:0/3615060381 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:21.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.221+0000 7ff09d600640 1 --2- 172.21.15.16:0/3615060381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e8d8851b0 0x557e8dc599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:21.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.221+0000 7ff09cc00640 1 --2- 172.21.15.16:0/3615060381 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e8d88d390 0x557e8dc51030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:21.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.221+0000 7ff09e000640 1 -- 172.21.15.16:0/3615060381 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e8d88d390 msgr2=0x557e8dc51030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:21.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.221+0000 7ff09e000640 1 --2- 172.21.15.16:0/3615060381 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e8d88d390 0x557e8dc51030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:21.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.221+0000 7ff09e000640 1 -- 172.21.15.16:0/3615060381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e8d8851b0 msgr2=0x557e8dc599d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:21.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.221+0000 7ff09e000640 1 --2- 172.21.15.16:0/3615060381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e8d8851b0 0x557e8dc599d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:21.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.221+0000 7ff09e000640 1 -- 172.21.15.16:0/3615060381 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e8dc56710 con 0x557e8d8ae070 2024-08-31T06:08:21.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.221+0000 7ff09d600640 1 --2- 172.21.15.16:0/3615060381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e8d8851b0 0x557e8dc599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:08:21.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.221+0000 7ff09e000640 1 --2- 172.21.15.16:0/3615060381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e8d8ae070 0x557e8dc51570 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff090009f40 tx=0x7ff090003a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f348cd7e945ad49 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:21.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.222+0000 7ff089c00640 1 -- 172.21.15.16:0/3615060381 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09001d040 con 0x557e8d8ae070 2024-08-31T06:08:21.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.222+0000 7ff089c00640 1 -- 172.21.15.16:0/3615060381 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0900054f0 con 0x557e8d8ae070 2024-08-31T06:08:21.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.222+0000 7ff089c00640 1 -- 172.21.15.16:0/3615060381 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09000fa70 con 0x557e8d8ae070 2024-08-31T06:08:21.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.222+0000 7ff0a6c3a740 1 -- 172.21.15.16:0/3615060381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e8d8ae070 msgr2=0x557e8dc51570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:21.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.222+0000 7ff0a6c3a740 1 --2- 172.21.15.16:0/3615060381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e8d8ae070 0x557e8dc51570 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff090009f40 tx=0x7ff090003a30 comp rx=0 tx=0).stop 2024-08-31T06:08:21.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.223+0000 7ff0a6c3a740 1 -- 172.21.15.16:0/3615060381 shutdown_connections 2024-08-31T06:08:21.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.223+0000 7ff0a6c3a740 1 --2- 172.21.15.16:0/3615060381 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e8d88d390 0x557e8dc51030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:21.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.223+0000 7ff0a6c3a740 1 --2- 172.21.15.16:0/3615060381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e8d8ae070 0x557e8dc51570 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:21.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.223+0000 7ff0a6c3a740 1 --2- 172.21.15.16:0/3615060381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e8d8851b0 0x557e8dc599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:21.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.223+0000 7ff0a6c3a740 1 -- 172.21.15.16:0/3615060381 >> 172.21.15.16:0/3615060381 conn(0x557e8d8a2460 msgr2=0x557e8dc5daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:21.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.223+0000 7ff0a6c3a740 1 -- 172.21.15.16:0/3615060381 shutdown_connections 2024-08-31T06:08:21.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.223+0000 7ff0a6c3a740 1 -- 172.21.15.16:0/3615060381 wait complete. 2024-08-31T06:08:21.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.223+0000 7ff0a6c3a740 1 -- start start 2024-08-31T06:08:21.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.223+0000 7ff0a6c3a740 1 Processor -- start 2024-08-31T06:08:21.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.224+0000 7ff0a6c3a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e8d8851b0 0x557e8dc5c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:21.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.224+0000 7ff0a6c3a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e8d88d390 0x557e8dc5c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:21.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.224+0000 7ff0a6c3a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e8d8ae070 0x557e8dc5ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:21.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.224+0000 7ff0a6c3a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff0981b3ef0 con 0x557e8d88d390 2024-08-31T06:08:21.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.224+0000 7ff0a6c3a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff0981b5360 con 0x557e8d8ae070 2024-08-31T06:08:21.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.224+0000 7ff0a6c3a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff0981b45c0 con 0x557e8d8851b0 2024-08-31T06:08:21.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.224+0000 7ff0a6c3a740 10 client.0 _ll_register_callbacks cb 0x7ff08c0ab900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:08:21.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.224+0000 7ff09cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e8d88d390 0x557e8dc5c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:21.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.224+0000 7ff09cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e8d88d390 0x557e8dc5c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56444/0 (socket says 172.21.15.16:56444) 2024-08-31T06:08:21.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.224+0000 7ff09cc00640 1 -- 172.21.15.16:0/1397512186 learned_addr learned my addr 172.21.15.16:0/1397512186 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:21.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.224+0000 7ff09d600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e8d8851b0 0x557e8dc5c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:21.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.224+0000 7ff09e000640 1 --2- 172.21.15.16:0/1397512186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e8d8ae070 0x557e8dc5ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:21.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.225+0000 7ff09d600640 1 -- 172.21.15.16:0/1397512186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e8d8ae070 msgr2=0x557e8dc5ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:21.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.225+0000 7ff09d600640 1 --2- 172.21.15.16:0/1397512186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e8d8ae070 0x557e8dc5ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:21.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.225+0000 7ff09d600640 1 -- 172.21.15.16:0/1397512186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e8d88d390 msgr2=0x557e8dc5c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:21.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.225+0000 7ff09d600640 1 --2- 172.21.15.16:0/1397512186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e8d88d390 0x557e8dc5c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:21.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.225+0000 7ff09d600640 1 -- 172.21.15.16:0/1397512186 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e8dc626d0 con 0x557e8d8851b0 2024-08-31T06:08:21.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.225+0000 7ff09cc00640 1 --2- 172.21.15.16:0/1397512186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e8d88d390 0x557e8dc5c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:08:21.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.225+0000 7ff09d600640 1 --2- 172.21.15.16:0/1397512186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e8d8851b0 0x557e8dc5c410 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00c6a0 tx=0x7ff08c0a7ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f88d8e65c8cd2c93 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:21.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.225+0000 7ff07ea00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:08:21.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.225+0000 7ff07ea00640 1 -- 172.21.15.16:0/1397512186 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c0b52b0 con 0x557e8d8851b0 2024-08-31T06:08:21.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.225+0000 7ff0a6c3a740 1 -- 172.21.15.16:0/1397512186 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0900054f0 con 0x557e8d8851b0 2024-08-31T06:08:21.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.225+0000 7ff07ea00640 1 -- 172.21.15.16:0/1397512186 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7ff08c0a8580 con 0x557e8d8851b0 2024-08-31T06:08:21.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.226+0000 7ff07ea00640 1 -- 172.21.15.16:0/1397512186 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff08c0cd320 con 0x557e8d8851b0 2024-08-31T06:08:21.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.226+0000 7ff0a6c3a740 10 client.25828 fetch_fsmap learned FSMap version 30 2024-08-31T06:08:21.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.226+0000 7ff0a6c3a740 1 -- 172.21.15.16:0/1397512186 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557e8dcdc250 con 0x557e8d8851b0 2024-08-31T06:08:21.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.226+0000 7ff07ea00640 1 -- 172.21.15.16:0/1397512186 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0900054f0 con 0x557e8d8851b0 2024-08-31T06:08:21.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.226+0000 7ff07ea00640 1 -- 172.21.15.16:0/1397512186 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7ff08c0a7900 con 0x557e8d8851b0 2024-08-31T06:08:21.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.226+0000 7ff0a6c3a740 10 client.25828 fetch_fsmap finished waiting for FSMap version 30 2024-08-31T06:08:21.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.226+0000 7ff0a6c3a740 10 client.25828 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:08:21.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.226+0000 7ff0a6c3a740 20 client.25828 populate_metadata read hostname 'smithi016' 2024-08-31T06:08:21.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.226+0000 7ff0a6c3a740 1 --2- 172.21.15.16:0/1397512186 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557e8d841da0 0x557e8dcdea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:21.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.226+0000 7ff0a6c3a740 4 client.25828 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:08:21.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.226+0000 7ff0a6c3a740 1 -- 172.21.15.16:0/1397512186 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557e8dbc81a0 con 0x557e8d841da0 2024-08-31T06:08:21.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.227+0000 7ff09cc00640 1 --2- 172.21.15.16:0/1397512186 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557e8d841da0 0x557e8dcdea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:21.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.227+0000 7ff09cc00640 1 --2- 172.21.15.16:0/1397512186 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557e8d841da0 0x557e8dcdea70 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c0812a67be15881 server_cookie=24c49132371f3777 in_seq=0 out_seq=0 2024-08-31T06:08:21.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.227+0000 7ff07ea00640 10 client.25828 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:21.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.228+0000 7ff07ea00640 1 -- 172.21.15.16:0/1397512186 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557e8dbc81a0 con 0x557e8d841da0 2024-08-31T06:08:21.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.228+0000 7ff07ea00640 10 client.25828 handle_command_reply: tid=0 2024-08-31T06:08:21.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.228+0000 7ff0a6c3a740 1 client.25828 shutdown 2024-08-31T06:08:21.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.228+0000 7ff0a6c3a740 1 -- 172.21.15.16:0/1397512186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e8d8851b0 msgr2=0x557e8dc5c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:21.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.228+0000 7ff0a6c3a740 1 --2- 172.21.15.16:0/1397512186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e8d8851b0 0x557e8dc5c410 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00c6a0 tx=0x7ff08c0a7ed0 comp rx=0 tx=0).stop 2024-08-31T06:08:21.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.228+0000 7ff0a6c3a740 1 -- 172.21.15.16:0/1397512186 shutdown_connections 2024-08-31T06:08:21.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.228+0000 7ff0a6c3a740 1 --2- 172.21.15.16:0/1397512186 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557e8d841da0 0x557e8dcdea70 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:21.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.228+0000 7ff0a6c3a740 1 --2- 172.21.15.16:0/1397512186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e8d8ae070 0x557e8dc5ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:21.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.228+0000 7ff0a6c3a740 1 --2- 172.21.15.16:0/1397512186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e8d88d390 0x557e8dc5c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:21.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.228+0000 7ff07ea00640 0 client.25828 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:21.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.228+0000 7ff0a6c3a740 1 --2- 172.21.15.16:0/1397512186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e8d8851b0 0x557e8dc5c410 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:21.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.228+0000 7ff0a6c3a740 1 -- 172.21.15.16:0/1397512186 >> 172.21.15.16:0/1397512186 conn(0x557e8d8a2460 msgr2=0x557e8dc51030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:21.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.228+0000 7ff0a6c3a740 1 -- 172.21.15.16:0/1397512186 shutdown_connections 2024-08-31T06:08:21.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.228+0000 7ff0a6c3a740 1 -- 172.21.15.16:0/1397512186 wait complete. 2024-08-31T06:08:21.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.228+0000 7ff0a6c3a740 20 client.25828 trim_cache size 0 max 16384 2024-08-31T06:08:21.234 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:08:21.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.229+0000 7ff0a0000640 1 -- 172.21.15.16:0/3191307701 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff0800b4c30 msgr2=0x7ff0800b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:21.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.229+0000 7ff0a0000640 1 --2- 172.21.15.16:0/3191307701 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff0800b4c30 0x7ff0800b7140 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff084009230 tx=0x7ff084031400 comp rx=0 tx=0).stop 2024-08-31T06:08:21.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.229+0000 7ff0a0000640 1 -- 172.21.15.16:0/3191307701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff098103900 msgr2=0x7ff098076ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:21.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.229+0000 7ff0a0000640 1 --2- 172.21.15.16:0/3191307701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff098103900 0x7ff098076ec0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00e9f0 tx=0x7ff08c00edf0 comp rx=0 tx=0).stop 2024-08-31T06:08:21.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.229+0000 7ff0a0000640 1 -- 172.21.15.16:0/3191307701 shutdown_connections 2024-08-31T06:08:21.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.229+0000 7ff0a0000640 1 --2- 172.21.15.16:0/3191307701 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff0800b4c30 0x7ff0800b7140 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:21.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.229+0000 7ff0a0000640 1 --2- 172.21.15.16:0/3191307701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff098109240 0x7ff0980754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:21.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.229+0000 7ff0a0000640 1 --2- 172.21.15.16:0/3191307701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff098109b90 0x7ff0980759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:21.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.229+0000 7ff0a0000640 1 --2- 172.21.15.16:0/3191307701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff098103900 0x7ff098076ec0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:21.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.229+0000 7ff0a0000640 1 -- 172.21.15.16:0/3191307701 >> 172.21.15.16:0/3191307701 conn(0x7ff09807b5f0 msgr2=0x7ff0981052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:21.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.229+0000 7ff0a0000640 1 -- 172.21.15.16:0/3191307701 shutdown_connections 2024-08-31T06:08:21.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:21.229+0000 7ff0a0000640 1 -- 172.21.15.16:0/3191307701 wait complete. 2024-08-31T06:08:21.629 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 55.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-31T06:08:22.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:22 smithi016 ceph-mon[17194]: pgmap v398: 97 pgs: 97 active+clean; 947 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 791 KiB/s wr, 98 op/s 2024-08-31T06:08:22.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:22 smithi184 ceph-mon[22331]: pgmap v398: 97 pgs: 97 active+clean; 947 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 791 KiB/s wr, 98 op/s 2024-08-31T06:08:22.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:22 smithi155 ceph-mon[21742]: pgmap v398: 97 pgs: 97 active+clean; 947 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 791 KiB/s wr, 98 op/s 2024-08-31T06:08:22.952 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T06:08:22.953 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T06:08:23.346 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:08:23.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:23.990+0000 7f2b3ac00640 1 -- 172.21.15.16:0/3206821952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b34107480 msgr2=0x7f2b34107890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:23.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:23.990+0000 7f2b3ac00640 1 --2- 172.21.15.16:0/3206821952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b34107480 0x7f2b34107890 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2b200074e0 tx=0x7f2b2002b070 comp rx=0 tx=0).stop 2024-08-31T06:08:23.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:23.991+0000 7f2b3ac00640 1 -- 172.21.15.16:0/3206821952 shutdown_connections 2024-08-31T06:08:23.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:23.991+0000 7f2b3ac00640 1 --2- 172.21.15.16:0/3206821952 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b34107dd0 0x7f2b3410e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:23.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:23.991+0000 7f2b3ac00640 1 --2- 172.21.15.16:0/3206821952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b34107480 0x7f2b34107890 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:23.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:23.991+0000 7f2b3ac00640 1 --2- 172.21.15.16:0/3206821952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b34101b40 0x7f2b34101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:23.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:23.991+0000 7f2b3ac00640 1 -- 172.21.15.16:0/3206821952 >> 172.21.15.16:0/3206821952 conn(0x7f2b340775a0 msgr2=0x7f2b340779e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:23.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:23.991+0000 7f2b3ac00640 1 -- 172.21.15.16:0/3206821952 shutdown_connections 2024-08-31T06:08:23.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:23.991+0000 7f2b3ac00640 1 -- 172.21.15.16:0/3206821952 wait complete. 2024-08-31T06:08:23.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:23.992+0000 7f2b3ac00640 1 Processor -- start 2024-08-31T06:08:23.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:23.992+0000 7f2b3ac00640 1 -- start start 2024-08-31T06:08:23.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:23.992+0000 7f2b3ac00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b34101b40 0x7f2b3419c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:23.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:23.992+0000 7f2b3ac00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b34107480 0x7f2b3419ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:23.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:23.992+0000 7f2b3ac00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b34107dd0 0x7f2b341a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:23.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:23.993+0000 7f2b3ac00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2b34111640 con 0x7f2b34101b40 2024-08-31T06:08:23.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:23.993+0000 7f2b3ac00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2b341114c0 con 0x7f2b34107dd0 2024-08-31T06:08:23.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:23.993+0000 7f2b3ac00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2b341117c0 con 0x7f2b34107480 2024-08-31T06:08:23.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:23.993+0000 7f2b33e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b34101b40 0x7f2b3419c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:23.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:23.993+0000 7f2b33e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b34101b40 0x7f2b3419c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56460/0 (socket says 172.21.15.16:56460) 2024-08-31T06:08:23.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:23.993+0000 7f2b33e00640 1 -- 172.21.15.16:0/420480556 learned_addr learned my addr 172.21.15.16:0/420480556 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:23.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:23.993+0000 7f2b33400640 1 --2- 172.21.15.16:0/420480556 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b34107480 0x7f2b3419ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:23.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:23.993+0000 7f2b38c00640 1 --2- 172.21.15.16:0/420480556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b34107dd0 0x7f2b341a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:23.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:23.993+0000 7f2b33e00640 1 -- 172.21.15.16:0/420480556 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b34107480 msgr2=0x7f2b3419ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:23.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:23.993+0000 7f2b33e00640 1 --2- 172.21.15.16:0/420480556 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b34107480 0x7f2b3419ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:23.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:23.993+0000 7f2b33e00640 1 -- 172.21.15.16:0/420480556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b34107dd0 msgr2=0x7f2b341a1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:23.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:23.993+0000 7f2b33e00640 1 --2- 172.21.15.16:0/420480556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b34107dd0 0x7f2b341a1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:23.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:23.993+0000 7f2b33e00640 1 -- 172.21.15.16:0/420480556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b341a23d0 con 0x7f2b34101b40 2024-08-31T06:08:23.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:23.994+0000 7f2b33e00640 1 --2- 172.21.15.16:0/420480556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b34101b40 0x7f2b3419c460 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2b2400e9f0 tx=0x7f2b2400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:23.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:23.994+0000 7f2b30c00640 1 -- 172.21.15.16:0/420480556 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2400cd80 con 0x7f2b34101b40 2024-08-31T06:08:23.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:23.994+0000 7f2b30c00640 1 -- 172.21.15.16:0/420480556 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b240108e0 con 0x7f2b34101b40 2024-08-31T06:08:23.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:23.994+0000 7f2b3ac00640 1 -- 172.21.15.16:0/420480556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b341a26c0 con 0x7f2b34101b40 2024-08-31T06:08:23.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:23.994+0000 7f2b30c00640 1 -- 172.21.15.16:0/420480556 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b24018980 con 0x7f2b34101b40 2024-08-31T06:08:23.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:23.994+0000 7f2b3ac00640 1 -- 172.21.15.16:0/420480556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b341a8e20 con 0x7f2b34101b40 2024-08-31T06:08:24.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:23.998+0000 7f2b4188f740 1 Processor -- start 2024-08-31T06:08:24.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:23.999+0000 7f2b4188f740 1 -- start start 2024-08-31T06:08:24.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:23.999+0000 7f2b4188f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6350f2350 0x55a635498600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:24.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:23.999+0000 7f2b4188f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6350c9490 0x55a635498b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:24.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:23.999+0000 7f2b4188f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6350d1670 0x55a635499080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:24.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:23.999+0000 7f2b4188f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a63540e750 con 0x55a6350c9490 2024-08-31T06:08:24.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:23.999+0000 7f2b4188f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a6353fdce0 con 0x55a6350f2350 2024-08-31T06:08:24.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:23.999+0000 7f2b4188f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a635401c90 con 0x55a6350d1670 2024-08-31T06:08:24.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.000+0000 7f2b30c00640 1 -- 172.21.15.16:0/420480556 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f2b24010430 con 0x7f2b34101b40 2024-08-31T06:08:24.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.000+0000 7f2b38c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6350f2350 0x55a635498600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:24.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.000+0000 7f2b38c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6350f2350 0x55a635498600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:54990/0 (socket says 172.21.15.16:54990) 2024-08-31T06:08:24.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.000+0000 7f2b38c00640 1 -- 172.21.15.16:0/565810466 learned_addr learned my addr 172.21.15.16:0/565810466 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:24.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.000+0000 7f2b33400640 1 --2- 172.21.15.16:0/565810466 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6350d1670 0x55a635499080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:24.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.000+0000 7f2b33e00640 1 --2- 172.21.15.16:0/565810466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6350c9490 0x55a635498b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:24.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.000+0000 7f2b33400640 1 -- 172.21.15.16:0/565810466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6350f2350 msgr2=0x55a635498600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:24.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.000+0000 7f2b33400640 1 --2- 172.21.15.16:0/565810466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6350f2350 0x55a635498600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.000+0000 7f2b33400640 1 -- 172.21.15.16:0/565810466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6350c9490 msgr2=0x55a635498b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:24.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.000+0000 7f2b33400640 1 --2- 172.21.15.16:0/565810466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6350c9490 0x55a635498b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.000+0000 7f2b33400640 1 -- 172.21.15.16:0/565810466 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a63515d930 con 0x55a6350d1670 2024-08-31T06:08:24.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.000+0000 7f2b38c00640 1 --2- 172.21.15.16:0/565810466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6350f2350 0x55a635498600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:08:24.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.001+0000 7f2b33400640 1 --2- 172.21.15.16:0/565810466 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6350d1670 0x55a635499080 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2b2002b590 tx=0x7f2b20007290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb298682d9d9bea2 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:24.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.001+0000 7f2b29200640 1 -- 172.21.15.16:0/565810466 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b20041030 con 0x55a6350d1670 2024-08-31T06:08:24.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.001+0000 7f2b29200640 1 -- 172.21.15.16:0/565810466 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b20004920 con 0x55a6350d1670 2024-08-31T06:08:24.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.001+0000 7f2b29200640 1 -- 172.21.15.16:0/565810466 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2002bc40 con 0x55a6350d1670 2024-08-31T06:08:24.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.001+0000 7f2b30c00640 1 --2- 172.21.15.16:0/420480556 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f2b180b4d20 0x7f2b180b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:24.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.002+0000 7f2b30c00640 1 -- 172.21.15.16:0/420480556 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f2b240e57d0 con 0x7f2b34101b40 2024-08-31T06:08:24.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.002+0000 7f2b38c00640 1 --2- 172.21.15.16:0/420480556 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f2b180b4d20 0x7f2b180b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:24.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.002+0000 7f2b4188f740 1 -- 172.21.15.16:0/565810466 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6350d1670 msgr2=0x55a635499080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:24.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.002+0000 7f2b4188f740 1 --2- 172.21.15.16:0/565810466 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6350d1670 0x55a635499080 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2b2002b590 tx=0x7f2b20007290 comp rx=0 tx=0).stop 2024-08-31T06:08:24.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.002+0000 7f2b4188f740 1 -- 172.21.15.16:0/565810466 shutdown_connections 2024-08-31T06:08:24.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.002+0000 7f2b4188f740 1 --2- 172.21.15.16:0/565810466 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6350d1670 0x55a635499080 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.002+0000 7f2b4188f740 1 --2- 172.21.15.16:0/565810466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6350c9490 0x55a635498b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.002+0000 7f2b4188f740 1 --2- 172.21.15.16:0/565810466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6350f2350 0x55a635498600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.002+0000 7f2b4188f740 1 -- 172.21.15.16:0/565810466 >> 172.21.15.16:0/565810466 conn(0x55a6350e6740 msgr2=0x55a6354964f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:24.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.002+0000 7f2b4188f740 1 -- 172.21.15.16:0/565810466 shutdown_connections 2024-08-31T06:08:24.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.002+0000 7f2b4188f740 1 -- 172.21.15.16:0/565810466 wait complete. 2024-08-31T06:08:24.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.003+0000 7f2b38c00640 1 --2- 172.21.15.16:0/420480556 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f2b180b4d20 0x7f2b180b7230 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c005fd0 tx=0x7f2b2c009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.003+0000 7f2b4188f740 1 -- start start 2024-08-31T06:08:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.003+0000 7f2b4188f740 1 Processor -- start 2024-08-31T06:08:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.004+0000 7f2b4188f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6350c9490 0x55a6354a4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.004+0000 7f2b4188f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6350d1670 0x55a6354a4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.004+0000 7f2b4188f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6350f2350 0x55a6354b0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.004+0000 7f2b4188f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a635260890 con 0x55a6350c9490 2024-08-31T06:08:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.004+0000 7f2b4188f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a635384cd0 con 0x55a6350d1670 2024-08-31T06:08:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.004+0000 7f2b4188f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a6354a6d80 con 0x55a6350f2350 2024-08-31T06:08:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.004+0000 7f2b4188f740 10 client.0 _ll_register_callbacks cb 0x55a6353f8ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:08:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.004+0000 7f2b33400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6350d1670 0x55a6354a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.004+0000 7f2b33400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6350d1670 0x55a6354a4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:55000/0 (socket says 172.21.15.16:55000) 2024-08-31T06:08:24.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.004+0000 7f2b33400640 1 -- 172.21.15.16:0/723583586 learned_addr learned my addr 172.21.15.16:0/723583586 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:24.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.004+0000 7f2b38c00640 1 --2- 172.21.15.16:0/723583586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6350f2350 0x55a6354b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:24.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.005+0000 7f2b33400640 1 -- 172.21.15.16:0/723583586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6350f2350 msgr2=0x55a6354b0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:24.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.005+0000 7f2b33400640 1 --2- 172.21.15.16:0/723583586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6350f2350 0x55a6354b0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.005+0000 7f2b33400640 1 -- 172.21.15.16:0/723583586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6350c9490 msgr2=0x55a6354a4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:24.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.005+0000 7f2b33400640 1 --2- 172.21.15.16:0/723583586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6350c9490 0x55a6354a4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.005+0000 7f2b33400640 1 -- 172.21.15.16:0/723583586 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6353ce1b0 con 0x55a6350d1670 2024-08-31T06:08:24.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.005+0000 7f2b33400640 1 --2- 172.21.15.16:0/723583586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6350d1670 0x55a6354a4690 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2b2002ba30 tx=0x7f2b20008a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=468a1a378f57497f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:24.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.005+0000 7f2b1d600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:08:24.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.005+0000 7f2b1d600640 1 -- 172.21.15.16:0/723583586 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2004a020 con 0x55a6350d1670 2024-08-31T06:08:24.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.005+0000 7f2b4188f740 1 -- 172.21.15.16:0/723583586 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b20004920 con 0x55a6350d1670 2024-08-31T06:08:24.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.005+0000 7f2b1d600640 1 -- 172.21.15.16:0/723583586 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f2b2002bdc0 con 0x55a6350d1670 2024-08-31T06:08:24.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.006+0000 7f2b1d600640 1 -- 172.21.15.16:0/723583586 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2b20055050 con 0x55a6350d1670 2024-08-31T06:08:24.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.006+0000 7f2b4188f740 10 client.25841 fetch_fsmap learned FSMap version 30 2024-08-31T06:08:24.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.006+0000 7f2b4188f740 1 -- 172.21.15.16:0/723583586 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a6354b2d40 con 0x55a6350d1670 2024-08-31T06:08:24.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.006+0000 7f2b1d600640 1 -- 172.21.15.16:0/723583586 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b20004920 con 0x55a6350d1670 2024-08-31T06:08:24.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.006+0000 7f2b1d600640 1 -- 172.21.15.16:0/723583586 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f2b2000a4c0 con 0x55a6350d1670 2024-08-31T06:08:24.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.006+0000 7f2b4188f740 10 client.25841 fetch_fsmap finished waiting for FSMap version 30 2024-08-31T06:08:24.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.006+0000 7f2b4188f740 10 client.25841 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:08:24.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.006+0000 7f2b4188f740 20 client.25841 populate_metadata read hostname 'smithi016' 2024-08-31T06:08:24.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.006+0000 7f2b4188f740 1 --2- 172.21.15.16:0/723583586 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a6350b0cd0 0x55a6354b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:24.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.006+0000 7f2b4188f740 4 client.25841 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:08:24.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.006+0000 7f2b4188f740 1 -- 172.21.15.16:0/723583586 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6354106a0 con 0x55a6350b0cd0 2024-08-31T06:08:24.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.007+0000 7f2b33e00640 1 --2- 172.21.15.16:0/723583586 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a6350b0cd0 0x55a6354b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:24.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.007+0000 7f2b33e00640 1 --2- 172.21.15.16:0/723583586 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a6350b0cd0 0x55a6354b60a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b06047b34088567f server_cookie=19f6e5fe5910135a in_seq=0 out_seq=0 2024-08-31T06:08:24.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.007+0000 7f2b1d600640 10 client.25841 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:24.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.008+0000 7f2b1d600640 1 -- 172.21.15.16:0/723583586 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a6354106a0 con 0x55a6350b0cd0 2024-08-31T06:08:24.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.008+0000 7f2b1d600640 10 client.25841 handle_command_reply: tid=0 2024-08-31T06:08:24.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.008+0000 7f2b4188f740 1 client.25841 shutdown 2024-08-31T06:08:24.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.008+0000 7f2b4188f740 1 -- 172.21.15.16:0/723583586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6350d1670 msgr2=0x55a6354a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:24.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.008+0000 7f2b4188f740 1 --2- 172.21.15.16:0/723583586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6350d1670 0x55a6354a4690 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2b2002ba30 tx=0x7f2b20008a60 comp rx=0 tx=0).stop 2024-08-31T06:08:24.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.008+0000 7f2b4188f740 1 -- 172.21.15.16:0/723583586 shutdown_connections 2024-08-31T06:08:24.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.008+0000 7f2b4188f740 1 --2- 172.21.15.16:0/723583586 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a6350b0cd0 0x55a6354b60a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.008+0000 7f2b4188f740 1 --2- 172.21.15.16:0/723583586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6350f2350 0x55a6354b0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.008+0000 7f2b4188f740 1 --2- 172.21.15.16:0/723583586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6350d1670 0x55a6354a4690 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.008+0000 7f2b1d600640 0 client.25841 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:24.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.008+0000 7f2b4188f740 1 --2- 172.21.15.16:0/723583586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6350c9490 0x55a6354a4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.008+0000 7f2b4188f740 1 -- 172.21.15.16:0/723583586 >> 172.21.15.16:0/723583586 conn(0x55a6350e6740 msgr2=0x55a63549b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:24.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.009+0000 7f2b4188f740 1 -- 172.21.15.16:0/723583586 shutdown_connections 2024-08-31T06:08:24.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.009+0000 7f2b4188f740 1 -- 172.21.15.16:0/723583586 wait complete. 2024-08-31T06:08:24.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.009+0000 7f2b4188f740 20 client.25841 trim_cache size 0 max 16384 2024-08-31T06:08:24.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.027+0000 7f2b4188f740 1 Processor -- start 2024-08-31T06:08:24.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.027+0000 7f2b4188f740 1 -- start start 2024-08-31T06:08:24.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.028+0000 7f2b4188f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6350c9490 0x55a6354a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:24.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.028+0000 7f2b4188f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6350d1670 0x55a6354affe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:24.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.028+0000 7f2b4188f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6350f2350 0x55a6354b0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:24.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.028+0000 7f2b4188f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2b341b3e20 con 0x55a6350d1670 2024-08-31T06:08:24.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.028+0000 7f2b4188f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2b341b1660 con 0x55a6350c9490 2024-08-31T06:08:24.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.028+0000 7f2b4188f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2b341af4b0 con 0x55a6350f2350 2024-08-31T06:08:24.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.028+0000 7f2b33400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6350d1670 0x55a6354affe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:24.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.028+0000 7f2b33e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6350c9490 0x55a6354a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:24.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.028+0000 7f2b33400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6350d1670 0x55a6354affe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56484/0 (socket says 172.21.15.16:56484) 2024-08-31T06:08:24.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.028+0000 7f2b33400640 1 -- 172.21.15.16:0/2412390565 learned_addr learned my addr 172.21.15.16:0/2412390565 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:24.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.028+0000 7f2b38c00640 1 --2- 172.21.15.16:0/2412390565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6350f2350 0x55a6354b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:24.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.028+0000 7f2b33e00640 1 -- 172.21.15.16:0/2412390565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6350f2350 msgr2=0x55a6354b0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:24.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.028+0000 7f2b33e00640 1 --2- 172.21.15.16:0/2412390565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6350f2350 0x55a6354b0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.028+0000 7f2b33e00640 1 -- 172.21.15.16:0/2412390565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6350d1670 msgr2=0x55a6354affe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:24.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.028+0000 7f2b33e00640 1 --2- 172.21.15.16:0/2412390565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6350d1670 0x55a6354affe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.028+0000 7f2b33e00640 1 -- 172.21.15.16:0/2412390565 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6354a6cc0 con 0x55a6350c9490 2024-08-31T06:08:24.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.028+0000 7f2b33400640 1 --2- 172.21.15.16:0/2412390565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6350d1670 0x55a6354affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:08:24.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.029+0000 7f2b33e00640 1 --2- 172.21.15.16:0/2412390565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6350c9490 0x55a6354a5b80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2b240e2f20 tx=0x7f2b240e7c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30fdf4271d593dd3 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:24.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.029+0000 7f2b1fe00640 1 -- 172.21.15.16:0/2412390565 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b240b51e0 con 0x55a6350c9490 2024-08-31T06:08:24.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.029+0000 7f2b1fe00640 1 -- 172.21.15.16:0/2412390565 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b24010a90 con 0x55a6350c9490 2024-08-31T06:08:24.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.029+0000 7f2b1fe00640 1 -- 172.21.15.16:0/2412390565 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b240b4fb0 con 0x55a6350c9490 2024-08-31T06:08:24.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.030+0000 7f2b4188f740 1 -- 172.21.15.16:0/2412390565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6350c9490 msgr2=0x55a6354a5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:24.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.030+0000 7f2b4188f740 1 --2- 172.21.15.16:0/2412390565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6350c9490 0x55a6354a5b80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2b240e2f20 tx=0x7f2b240e7c20 comp rx=0 tx=0).stop 2024-08-31T06:08:24.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.030+0000 7f2b4188f740 1 -- 172.21.15.16:0/2412390565 shutdown_connections 2024-08-31T06:08:24.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.030+0000 7f2b4188f740 1 --2- 172.21.15.16:0/2412390565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6350f2350 0x55a6354b0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.030+0000 7f2b4188f740 1 --2- 172.21.15.16:0/2412390565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6350d1670 0x55a6354affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.030+0000 7f2b4188f740 1 --2- 172.21.15.16:0/2412390565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6350c9490 0x55a6354a5b80 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.030+0000 7f2b4188f740 1 -- 172.21.15.16:0/2412390565 >> 172.21.15.16:0/2412390565 conn(0x55a6350e6740 msgr2=0x55a6354a5470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:24.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.030+0000 7f2b4188f740 1 -- 172.21.15.16:0/2412390565 shutdown_connections 2024-08-31T06:08:24.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.030+0000 7f2b4188f740 1 -- 172.21.15.16:0/2412390565 wait complete. 2024-08-31T06:08:24.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.030+0000 7f2b4188f740 1 -- start start 2024-08-31T06:08:24.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.031+0000 7f2b4188f740 1 Processor -- start 2024-08-31T06:08:24.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.031+0000 7f2b4188f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6350c9490 0x55a6354b7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:24.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.031+0000 7f2b4188f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6350d1670 0x55a6354b81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:24.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.031+0000 7f2b4188f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6350f2350 0x55a635518850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:24.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.031+0000 7f2b4188f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2b341afb20 con 0x55a6350f2350 2024-08-31T06:08:24.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.031+0000 7f2b4188f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2b341b0f90 con 0x55a6350d1670 2024-08-31T06:08:24.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.031+0000 7f2b4188f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2b341b01f0 con 0x55a6350c9490 2024-08-31T06:08:24.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.031+0000 7f2b4188f740 10 client.0 _ll_register_callbacks cb 0x7f2b2c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:08:24.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.031+0000 7f2b38c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6350f2350 0x55a635518850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:24.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.032+0000 7f2b33e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6350c9490 0x55a6354b7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:24.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.032+0000 7f2b38c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6350f2350 0x55a635518850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56500/0 (socket says 172.21.15.16:56500) 2024-08-31T06:08:24.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.032+0000 7f2b38c00640 1 -- 172.21.15.16:0/3402050270 learned_addr learned my addr 172.21.15.16:0/3402050270 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:24.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.032+0000 7f2b33400640 1 --2- 172.21.15.16:0/3402050270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6350d1670 0x55a6354b81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:24.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.032+0000 7f2b38c00640 1 -- 172.21.15.16:0/3402050270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6350c9490 msgr2=0x55a6354b7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:24.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.032+0000 7f2b38c00640 1 --2- 172.21.15.16:0/3402050270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6350c9490 0x55a6354b7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.032+0000 7f2b38c00640 1 -- 172.21.15.16:0/3402050270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6350d1670 msgr2=0x55a6354b81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:24.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.032+0000 7f2b38c00640 1 --2- 172.21.15.16:0/3402050270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6350d1670 0x55a6354b81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.032+0000 7f2b38c00640 1 -- 172.21.15.16:0/3402050270 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a635518ec0 con 0x55a6350f2350 2024-08-31T06:08:24.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.032+0000 7f2b38c00640 1 --2- 172.21.15.16:0/3402050270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6350f2350 0x55a635518850 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c003f50 tx=0x7f2b2c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84cef464cae46652 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:24.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.032+0000 7f2b1d600640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:08:24.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.033+0000 7f2b1d600640 1 -- 172.21.15.16:0/3402050270 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c02e060 con 0x55a6350f2350 2024-08-31T06:08:24.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.033+0000 7f2b4188f740 1 -- 172.21.15.16:0/3402050270 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b24010a90 con 0x55a6350f2350 2024-08-31T06:08:24.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.033+0000 7f2b1d600640 1 -- 172.21.15.16:0/3402050270 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f2b2c0044c0 con 0x55a6350f2350 2024-08-31T06:08:24.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.033+0000 7f2b1d600640 1 -- 172.21.15.16:0/3402050270 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2b2c0046c0 con 0x55a6350f2350 2024-08-31T06:08:24.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.033+0000 7f2b4188f740 10 client.15804 fetch_fsmap learned FSMap version 30 2024-08-31T06:08:24.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.033+0000 7f2b4188f740 1 -- 172.21.15.16:0/3402050270 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a635521500 con 0x55a6350f2350 2024-08-31T06:08:24.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.033+0000 7f2b1d600640 1 -- 172.21.15.16:0/3402050270 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b24010a90 con 0x55a6350f2350 2024-08-31T06:08:24.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.033+0000 7f2b1d600640 1 -- 172.21.15.16:0/3402050270 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f2b2c01e430 con 0x55a6350f2350 2024-08-31T06:08:24.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.033+0000 7f2b4188f740 10 client.15804 fetch_fsmap finished waiting for FSMap version 30 2024-08-31T06:08:24.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.033+0000 7f2b4188f740 10 client.15804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:08:24.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.033+0000 7f2b4188f740 20 client.15804 populate_metadata read hostname 'smithi016' 2024-08-31T06:08:24.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.034+0000 7f2b4188f740 1 --2- 172.21.15.16:0/3402050270 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a635086080 0x55a635523d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:24.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.034+0000 7f2b4188f740 4 client.15804 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T06:08:24.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.034+0000 7f2b4188f740 1 -- 172.21.15.16:0/3402050270 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a63540c140 con 0x55a635086080 2024-08-31T06:08:24.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.034+0000 7f2b33e00640 1 --2- 172.21.15.16:0/3402050270 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a635086080 0x55a635523d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:24.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.035+0000 7f2b1d600640 10 client.15804 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:24.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.035+0000 7f2b33e00640 1 --2- 172.21.15.16:0/3402050270 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a635086080 0x55a635523d30 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abfafc4825a88269 server_cookie=41b7b7b0821bb487 in_seq=0 out_seq=0 2024-08-31T06:08:24.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.036+0000 7f2b1d600640 1 -- 172.21.15.16:0/3402050270 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a63540c140 con 0x55a635086080 2024-08-31T06:08:24.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.036+0000 7f2b1d600640 10 client.15804 handle_command_reply: tid=0 2024-08-31T06:08:24.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.036+0000 7f2b4188f740 1 client.15804 shutdown 2024-08-31T06:08:24.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.036+0000 7f2b4188f740 1 -- 172.21.15.16:0/3402050270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6350f2350 msgr2=0x55a635518850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:24.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.036+0000 7f2b4188f740 1 --2- 172.21.15.16:0/3402050270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6350f2350 0x55a635518850 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c003f50 tx=0x7f2b2c004a40 comp rx=0 tx=0).stop 2024-08-31T06:08:24.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.036+0000 7f2b4188f740 1 -- 172.21.15.16:0/3402050270 shutdown_connections 2024-08-31T06:08:24.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.037+0000 7f2b4188f740 1 --2- 172.21.15.16:0/3402050270 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a635086080 0x55a635523d30 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.037+0000 7f2b4188f740 1 --2- 172.21.15.16:0/3402050270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6350f2350 0x55a635518850 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.037+0000 7f2b4188f740 1 --2- 172.21.15.16:0/3402050270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6350d1670 0x55a6354b81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.037+0000 7f2b4188f740 1 --2- 172.21.15.16:0/3402050270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6350c9490 0x55a6354b7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.037+0000 7f2b4188f740 1 -- 172.21.15.16:0/3402050270 >> 172.21.15.16:0/3402050270 conn(0x55a6350e6740 msgr2=0x55a6354a5dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:24.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.037+0000 7f2b1d600640 0 client.15804 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:24.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.037+0000 7f2b4188f740 1 -- 172.21.15.16:0/3402050270 shutdown_connections 2024-08-31T06:08:24.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.037+0000 7f2b4188f740 1 -- 172.21.15.16:0/3402050270 wait complete. 2024-08-31T06:08:24.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.037+0000 7f2b4188f740 20 client.15804 trim_cache size 0 max 16384 2024-08-31T06:08:24.043 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:08:24.043 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-08-31T06:08:24.043 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "c8455d74-2fe7-4d93-bcf2-19b419d16016", 2024-08-31T06:08:24.043 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-08-31T06:08:24.043 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:08:24.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.038+0000 7f2b3ac00640 1 -- 172.21.15.16:0/420480556 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f2b180b4d20 msgr2=0x7f2b180b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:24.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.038+0000 7f2b3ac00640 1 --2- 172.21.15.16:0/420480556 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f2b180b4d20 0x7f2b180b7230 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c005fd0 tx=0x7f2b2c009500 comp rx=0 tx=0).stop 2024-08-31T06:08:24.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.038+0000 7f2b3ac00640 1 -- 172.21.15.16:0/420480556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b34101b40 msgr2=0x7f2b3419c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:24.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.038+0000 7f2b3ac00640 1 --2- 172.21.15.16:0/420480556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b34101b40 0x7f2b3419c460 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2b2400e9f0 tx=0x7f2b2400eeb0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.038+0000 7f2b3ac00640 1 -- 172.21.15.16:0/420480556 shutdown_connections 2024-08-31T06:08:24.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.038+0000 7f2b3ac00640 1 --2- 172.21.15.16:0/420480556 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f2b180b4d20 0x7f2b180b7230 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.038+0000 7f2b3ac00640 1 --2- 172.21.15.16:0/420480556 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b34107480 0x7f2b3419ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.038+0000 7f2b3ac00640 1 --2- 172.21.15.16:0/420480556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b34107dd0 0x7f2b341a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.038+0000 7f2b3ac00640 1 --2- 172.21.15.16:0/420480556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b34101b40 0x7f2b3419c460 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.038+0000 7f2b3ac00640 1 -- 172.21.15.16:0/420480556 >> 172.21.15.16:0/420480556 conn(0x7f2b340775a0 msgr2=0x7f2b34105330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:24.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.038+0000 7f2b3ac00640 1 -- 172.21.15.16:0/420480556 shutdown_connections 2024-08-31T06:08:24.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:24.038+0000 7f2b3ac00640 1 -- 172.21.15.16:0/420480556 wait complete. 2024-08-31T06:08:24.451 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.448+0000 7f1a35200640 1 -- 172.21.15.16:0/115432414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a301058b0 msgr2=0x7f1a30105cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:24.451 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.448+0000 7f1a35200640 1 --2- 172.21.15.16:0/115432414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a301058b0 0x7f1a30105cc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1a20009900 tx=0x7f1a2002a400 comp rx=0 tx=0).stop 2024-08-31T06:08:24.451 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.449+0000 7f1a35200640 1 -- 172.21.15.16:0/115432414 shutdown_connections 2024-08-31T06:08:24.451 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.449+0000 7f1a35200640 1 --2- 172.21.15.16:0/115432414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a3010b1e0 0x7f1a3010b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.451 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.449+0000 7f1a35200640 1 --2- 172.21.15.16:0/115432414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a3010bb30 0x7f1a30112360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.451 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.449+0000 7f1a35200640 1 --2- 172.21.15.16:0/115432414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a301058b0 0x7f1a30105cc0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.452 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.449+0000 7f1a35200640 1 -- 172.21.15.16:0/115432414 >> 172.21.15.16:0/115432414 conn(0x7f1a301012f0 msgr2=0x7f1a30103740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:24.452 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.449+0000 7f1a35200640 1 -- 172.21.15.16:0/115432414 shutdown_connections 2024-08-31T06:08:24.452 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.449+0000 7f1a35200640 1 -- 172.21.15.16:0/115432414 wait complete. 2024-08-31T06:08:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.450+0000 7f1a35200640 1 Processor -- start 2024-08-31T06:08:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.450+0000 7f1a35200640 1 -- start start 2024-08-31T06:08:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.450+0000 7f1a35200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a301058b0 0x7f1a3019c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.450+0000 7f1a35200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a3010b1e0 0x7f1a3019c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.450+0000 7f1a35200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a3010bb30 0x7f1a301a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.450+0000 7f1a35200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1a301157f0 con 0x7f1a301058b0 2024-08-31T06:08:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.450+0000 7f1a35200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1a30115670 con 0x7f1a3010b1e0 2024-08-31T06:08:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.450+0000 7f1a35200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1a30115970 con 0x7f1a3010bb30 2024-08-31T06:08:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.451+0000 7f1a2f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a301058b0 0x7f1a3019c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.451+0000 7f1a2fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a3010bb30 0x7f1a301a1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.451+0000 7f1a2f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a301058b0 0x7f1a3019c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56522/0 (socket says 172.21.15.16:56522) 2024-08-31T06:08:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.451+0000 7f1a2f400640 1 -- 172.21.15.16:0/3852040109 learned_addr learned my addr 172.21.15.16:0/3852040109 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.451+0000 7f1a2ea00640 1 --2- 172.21.15.16:0/3852040109 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a3010b1e0 0x7f1a3019c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.451+0000 7f1a2fe00640 1 -- 172.21.15.16:0/3852040109 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a3010b1e0 msgr2=0x7f1a3019c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.451+0000 7f1a2fe00640 1 --2- 172.21.15.16:0/3852040109 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a3010b1e0 0x7f1a3019c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.451+0000 7f1a2fe00640 1 -- 172.21.15.16:0/3852040109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a301058b0 msgr2=0x7f1a3019c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.451+0000 7f1a2fe00640 1 --2- 172.21.15.16:0/3852040109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a301058b0 0x7f1a3019c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.451+0000 7f1a2fe00640 1 -- 172.21.15.16:0/3852040109 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a301a2180 con 0x7f1a3010bb30 2024-08-31T06:08:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.451+0000 7f1a2f400640 1 --2- 172.21.15.16:0/3852040109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a301058b0 0x7f1a3019c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:08:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.451+0000 7f1a2fe00640 1 --2- 172.21.15.16:0/3852040109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a3010bb30 0x7f1a301a1a80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1a28009fd0 tx=0x7f1a2800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.452+0000 7f1a27e00640 1 -- 172.21.15.16:0/3852040109 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2800ce20 con 0x7f1a3010bb30 2024-08-31T06:08:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.452+0000 7f1a27e00640 1 -- 172.21.15.16:0/3852040109 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a28005020 con 0x7f1a3010bb30 2024-08-31T06:08:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.452+0000 7f1a35200640 1 -- 172.21.15.16:0/3852040109 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a301a2470 con 0x7f1a3010bb30 2024-08-31T06:08:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.452+0000 7f1a27e00640 1 -- 172.21.15.16:0/3852040109 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a28010690 con 0x7f1a3010bb30 2024-08-31T06:08:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.452+0000 7f1a35200640 1 -- 172.21.15.16:0/3852040109 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a301a8c00 con 0x7f1a3010bb30 2024-08-31T06:08:24.457 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.455+0000 7f1a3da03740 1 Processor -- start 2024-08-31T06:08:24.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.456+0000 7f1a27e00640 1 -- 172.21.15.16:0/3852040109 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f1a280054d0 con 0x7f1a3010bb30 2024-08-31T06:08:24.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.457+0000 7f1a3da03740 1 -- start start 2024-08-31T06:08:24.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.457+0000 7f1a3da03740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589ef28c1b0 0x5589ef62fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:24.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.457+0000 7f1a3da03740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ef2632f0 0x5589ef630040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:24.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.457+0000 7f1a3da03740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589ef26b4d0 0x5589ef630580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:24.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.457+0000 7f1a3da03740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5589ef5a4f70 con 0x5589ef28c1b0 2024-08-31T06:08:24.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.457+0000 7f1a3da03740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5589ef594500 con 0x5589ef2632f0 2024-08-31T06:08:24.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.457+0000 7f1a3da03740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5589ef5984b0 con 0x5589ef26b4d0 2024-08-31T06:08:24.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.457+0000 7f1a2ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589ef28c1b0 0x5589ef62fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:24.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.457+0000 7f1a2ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589ef28c1b0 0x5589ef62fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56538/0 (socket says 172.21.15.16:56538) 2024-08-31T06:08:24.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.457+0000 7f1a2ea00640 1 -- 172.21.15.16:0/3340622200 learned_addr learned my addr 172.21.15.16:0/3340622200 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:24.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.458+0000 7f1a2ea00640 1 --2- 172.21.15.16:0/3340622200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589ef26b4d0 0x5589ef630580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:24.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.458+0000 7f1a2ea00640 1 -- 172.21.15.16:0/3340622200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589ef26b4d0 msgr2=0x5589ef630580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:24.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.458+0000 7f1a2ea00640 1 --2- 172.21.15.16:0/3340622200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589ef26b4d0 0x5589ef630580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.458+0000 7f1a2ea00640 1 -- 172.21.15.16:0/3340622200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ef2632f0 msgr2=0x5589ef630040 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T06:08:24.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.458+0000 7f1a2ea00640 1 --2- 172.21.15.16:0/3340622200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ef2632f0 0x5589ef630040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.458+0000 7f1a2ea00640 1 -- 172.21.15.16:0/3340622200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589ef54ec10 con 0x5589ef28c1b0 2024-08-31T06:08:24.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.458+0000 7f1a2ea00640 1 --2- 172.21.15.16:0/3340622200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589ef28c1b0 0x5589ef62fb00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c0117a0 tx=0x7f1a1c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66ded897c541d40e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:24.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.458+0000 7f1a27e00640 1 --2- 172.21.15.16:0/3852040109 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f1a140b4d20 0x7f1a140b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:24.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.458+0000 7f1a27e00640 1 -- 172.21.15.16:0/3852040109 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f1a280e5bd0 con 0x7f1a3010bb30 2024-08-31T06:08:24.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.458+0000 7f1a24c00640 1 -- 172.21.15.16:0/3340622200 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c004050 con 0x5589ef28c1b0 2024-08-31T06:08:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.459+0000 7f1a24c00640 1 -- 172.21.15.16:0/3340622200 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a1c0054d0 con 0x5589ef28c1b0 2024-08-31T06:08:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.459+0000 7f1a24c00640 1 -- 172.21.15.16:0/3340622200 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c004c00 con 0x5589ef28c1b0 2024-08-31T06:08:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.459+0000 7f1a2f400640 1 --2- 172.21.15.16:0/3852040109 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f1a140b4d20 0x7f1a140b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.460+0000 7f1a3da03740 1 -- 172.21.15.16:0/3340622200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589ef28c1b0 msgr2=0x5589ef62fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.460+0000 7f1a2f400640 1 --2- 172.21.15.16:0/3852040109 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f1a140b4d20 0x7f1a140b7230 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1a20031440 tx=0x7f1a200058d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.460+0000 7f1a3da03740 1 --2- 172.21.15.16:0/3340622200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589ef28c1b0 0x5589ef62fb00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c0117a0 tx=0x7f1a1c004730 comp rx=0 tx=0).stop 2024-08-31T06:08:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.460+0000 7f1a3da03740 1 -- 172.21.15.16:0/3340622200 shutdown_connections 2024-08-31T06:08:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.460+0000 7f1a3da03740 1 --2- 172.21.15.16:0/3340622200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589ef26b4d0 0x5589ef630580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.460+0000 7f1a3da03740 1 --2- 172.21.15.16:0/3340622200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ef2632f0 0x5589ef630040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.460+0000 7f1a3da03740 1 --2- 172.21.15.16:0/3340622200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589ef28c1b0 0x5589ef62fb00 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.460+0000 7f1a3da03740 1 -- 172.21.15.16:0/3340622200 >> 172.21.15.16:0/3340622200 conn(0x5589ef280300 msgr2=0x5589ef62d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.460+0000 7f1a3da03740 1 -- 172.21.15.16:0/3340622200 shutdown_connections 2024-08-31T06:08:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.460+0000 7f1a3da03740 1 -- 172.21.15.16:0/3340622200 wait complete. 2024-08-31T06:08:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.460+0000 7f1a3da03740 1 -- start start 2024-08-31T06:08:24.468 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.461+0000 7f1a3da03740 1 Processor -- start 2024-08-31T06:08:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.461+0000 7f1a3da03740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ef2632f0 0x5589ef63b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.461+0000 7f1a3da03740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589ef26b4d0 0x5589ef63b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.461+0000 7f1a3da03740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589ef28c1b0 0x5589ef63bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.461+0000 7f1a3da03740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5589ef4cdab0 con 0x5589ef26b4d0 2024-08-31T06:08:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.461+0000 7f1a3da03740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5589ef53b320 con 0x5589ef2632f0 2024-08-31T06:08:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.461+0000 7f1a3da03740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5589ef63e280 con 0x5589ef28c1b0 2024-08-31T06:08:24.474 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.462+0000 7f1a3da03740 10 client.0 _ll_register_callbacks cb 0x5589ef5416a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:08:24.474 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.462+0000 7f1a2f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ef2632f0 0x5589ef63b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:24.474 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.462+0000 7f1a2ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589ef26b4d0 0x5589ef63b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:24.474 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.462+0000 7f1a2f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ef2632f0 0x5589ef63b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:55062/0 (socket says 172.21.15.16:55062) 2024-08-31T06:08:24.474 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.462+0000 7f1a2f400640 1 -- 172.21.15.16:0/3015261777 learned_addr learned my addr 172.21.15.16:0/3015261777 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:24.474 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.462+0000 7f1a2fe00640 1 --2- 172.21.15.16:0/3015261777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589ef28c1b0 0x5589ef63bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:24.474 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.462+0000 7f1a2f400640 1 -- 172.21.15.16:0/3015261777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589ef28c1b0 msgr2=0x5589ef63bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:24.474 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.462+0000 7f1a2f400640 1 --2- 172.21.15.16:0/3015261777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589ef28c1b0 0x5589ef63bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.474 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.462+0000 7f1a2f400640 1 -- 172.21.15.16:0/3015261777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589ef26b4d0 msgr2=0x5589ef63b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:24.474 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.462+0000 7f1a2f400640 1 --2- 172.21.15.16:0/3015261777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589ef26b4d0 0x5589ef63b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.474 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.462+0000 7f1a2f400640 1 -- 172.21.15.16:0/3015261777 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589ef2f7790 con 0x5589ef2632f0 2024-08-31T06:08:24.474 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.462+0000 7f1a2ea00640 1 --2- 172.21.15.16:0/3015261777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589ef26b4d0 0x5589ef63b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:08:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.463+0000 7f1a2f400640 1 --2- 172.21.15.16:0/3015261777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ef2632f0 0x5589ef63b090 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1a20000c00 tx=0x7f1a20009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=328bb4f419fbe1ad server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.463+0000 7f1a19600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:08:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.463+0000 7f1a19600640 1 -- 172.21.15.16:0/3015261777 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2004c070 con 0x5589ef2632f0 2024-08-31T06:08:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.463+0000 7f1a19600640 1 -- 172.21.15.16:0/3015261777 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f1a20005220 con 0x5589ef2632f0 2024-08-31T06:08:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.463+0000 7f1a3da03740 1 -- 172.21.15.16:0/3015261777 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a1c0054d0 con 0x5589ef2632f0 2024-08-31T06:08:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.463+0000 7f1a19600640 1 -- 172.21.15.16:0/3015261777 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1a20035040 con 0x5589ef2632f0 2024-08-31T06:08:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.463+0000 7f1a3da03740 10 client.25856 fetch_fsmap learned FSMap version 30 2024-08-31T06:08:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.463+0000 7f1a3da03740 1 -- 172.21.15.16:0/3015261777 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5589ef370620 con 0x5589ef2632f0 2024-08-31T06:08:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.464+0000 7f1a19600640 1 -- 172.21.15.16:0/3015261777 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a1c0054d0 con 0x5589ef2632f0 2024-08-31T06:08:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.464+0000 7f1a19600640 1 -- 172.21.15.16:0/3015261777 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f1a20004810 con 0x5589ef2632f0 2024-08-31T06:08:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.464+0000 7f1a3da03740 10 client.25856 fetch_fsmap finished waiting for FSMap version 30 2024-08-31T06:08:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.464+0000 7f1a3da03740 10 client.25856 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:08:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.464+0000 7f1a3da03740 20 client.25856 populate_metadata read hostname 'smithi016' 2024-08-31T06:08:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.464+0000 7f1a3da03740 1 --2- 172.21.15.16:0/3015261777 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5589ef24ad80 0x5589ef64d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.464+0000 7f1a3da03740 4 client.25856 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:08:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.464+0000 7f1a3da03740 1 -- 172.21.15.16:0/3015261777 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589ef5a6ec0 con 0x5589ef24ad80 2024-08-31T06:08:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.464+0000 7f1a2ea00640 1 --2- 172.21.15.16:0/3015261777 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5589ef24ad80 0x5589ef64d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.465+0000 7f1a19600640 10 client.25856 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.465+0000 7f1a2ea00640 1 --2- 172.21.15.16:0/3015261777 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5589ef24ad80 0x5589ef64d280 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4ae9df2f54fb75b server_cookie=69c98244b1ea6a0e in_seq=0 out_seq=0 2024-08-31T06:08:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.466+0000 7f1a19600640 1 -- 172.21.15.16:0/3015261777 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5589ef5a6ec0 con 0x5589ef24ad80 2024-08-31T06:08:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.466+0000 7f1a19600640 10 client.25856 handle_command_reply: tid=0 2024-08-31T06:08:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.466+0000 7f1a3da03740 1 client.25856 shutdown 2024-08-31T06:08:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.466+0000 7f1a3da03740 1 -- 172.21.15.16:0/3015261777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ef2632f0 msgr2=0x5589ef63b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.466+0000 7f1a3da03740 1 --2- 172.21.15.16:0/3015261777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ef2632f0 0x5589ef63b090 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1a20000c00 tx=0x7f1a20009260 comp rx=0 tx=0).stop 2024-08-31T06:08:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.466+0000 7f1a3da03740 1 -- 172.21.15.16:0/3015261777 shutdown_connections 2024-08-31T06:08:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.466+0000 7f1a3da03740 1 --2- 172.21.15.16:0/3015261777 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5589ef24ad80 0x5589ef64d280 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.466+0000 7f1a3da03740 1 --2- 172.21.15.16:0/3015261777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589ef28c1b0 0x5589ef63bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.466+0000 7f1a3da03740 1 --2- 172.21.15.16:0/3015261777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589ef26b4d0 0x5589ef63b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.466+0000 7f1a3da03740 1 --2- 172.21.15.16:0/3015261777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ef2632f0 0x5589ef63b090 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.466+0000 7f1a19600640 0 client.25856 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.466+0000 7f1a3da03740 1 -- 172.21.15.16:0/3015261777 >> 172.21.15.16:0/3015261777 conn(0x5589ef280300 msgr2=0x5589ef630b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.466+0000 7f1a3da03740 1 -- 172.21.15.16:0/3015261777 shutdown_connections 2024-08-31T06:08:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.466+0000 7f1a3da03740 1 -- 172.21.15.16:0/3015261777 wait complete. 2024-08-31T06:08:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.466+0000 7f1a3da03740 20 client.25856 trim_cache size 0 max 16384 2024-08-31T06:08:24.497 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.493+0000 7f1a3da03740 1 Processor -- start 2024-08-31T06:08:24.497 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.493+0000 7f1a3da03740 1 -- start start 2024-08-31T06:08:24.497 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.494+0000 7f1a3da03740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589ef2632f0 0x5589ef63cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:24.497 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.494+0000 7f1a3da03740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589ef26b4d0 0x5589ef646b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:24.497 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.494+0000 7f1a3da03740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ef28c1b0 0x5589ef63b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:24.497 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.494+0000 7f1a3da03740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1a301b3b90 con 0x5589ef2632f0 2024-08-31T06:08:24.497 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.494+0000 7f1a3da03740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1a301b13d0 con 0x5589ef28c1b0 2024-08-31T06:08:24.497 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.494+0000 7f1a3da03740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1a301af220 con 0x5589ef26b4d0 2024-08-31T06:08:24.497 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.494+0000 7f1a2f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589ef2632f0 0x5589ef63cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:24.497 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.494+0000 7f1a2f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589ef2632f0 0x5589ef63cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56562/0 (socket says 172.21.15.16:56562) 2024-08-31T06:08:24.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.494+0000 7f1a2f400640 1 -- 172.21.15.16:0/383054386 learned_addr learned my addr 172.21.15.16:0/383054386 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:24.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.494+0000 7f1a2fe00640 1 --2- 172.21.15.16:0/383054386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ef28c1b0 0x5589ef63b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:24.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.494+0000 7f1a2ea00640 1 --2- 172.21.15.16:0/383054386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589ef26b4d0 0x5589ef646b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:24.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.494+0000 7f1a2f400640 1 -- 172.21.15.16:0/383054386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589ef26b4d0 msgr2=0x5589ef646b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:24.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.494+0000 7f1a2f400640 1 --2- 172.21.15.16:0/383054386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589ef26b4d0 0x5589ef646b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.494+0000 7f1a2f400640 1 -- 172.21.15.16:0/383054386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ef28c1b0 msgr2=0x5589ef63b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:24.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.494+0000 7f1a2f400640 1 --2- 172.21.15.16:0/383054386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ef28c1b0 0x5589ef63b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.494+0000 7f1a2f400640 1 -- 172.21.15.16:0/383054386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589ef64d630 con 0x5589ef2632f0 2024-08-31T06:08:24.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.494+0000 7f1a2fe00640 1 --2- 172.21.15.16:0/383054386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ef28c1b0 0x5589ef63b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:08:24.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.495+0000 7f1a2f400640 1 --2- 172.21.15.16:0/383054386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589ef2632f0 0x5589ef63cf70 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1a20004dd0 tx=0x7f1a2003c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a0d45eab29f8e6f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:24.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.495+0000 7f1a1be00640 1 -- 172.21.15.16:0/383054386 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2004c070 con 0x5589ef2632f0 2024-08-31T06:08:24.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.495+0000 7f1a1be00640 1 -- 172.21.15.16:0/383054386 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a20003cb0 con 0x5589ef2632f0 2024-08-31T06:08:24.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.495+0000 7f1a1be00640 1 -- 172.21.15.16:0/383054386 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a20005530 con 0x5589ef2632f0 2024-08-31T06:08:24.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.496+0000 7f1a3da03740 1 -- 172.21.15.16:0/383054386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589ef2632f0 msgr2=0x5589ef63cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:24.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.496+0000 7f1a3da03740 1 --2- 172.21.15.16:0/383054386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589ef2632f0 0x5589ef63cf70 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1a20004dd0 tx=0x7f1a2003c5d0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.496+0000 7f1a3da03740 1 -- 172.21.15.16:0/383054386 shutdown_connections 2024-08-31T06:08:24.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.496+0000 7f1a3da03740 1 --2- 172.21.15.16:0/383054386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589ef26b4d0 0x5589ef646b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.496+0000 7f1a3da03740 1 --2- 172.21.15.16:0/383054386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ef28c1b0 0x5589ef63b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.496+0000 7f1a3da03740 1 --2- 172.21.15.16:0/383054386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589ef2632f0 0x5589ef63cf70 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.496+0000 7f1a3da03740 1 -- 172.21.15.16:0/383054386 >> 172.21.15.16:0/383054386 conn(0x5589ef280300 msgr2=0x5589ef646460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:24.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.496+0000 7f1a3da03740 1 -- 172.21.15.16:0/383054386 shutdown_connections 2024-08-31T06:08:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.496+0000 7f1a3da03740 1 -- 172.21.15.16:0/383054386 wait complete. 2024-08-31T06:08:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.496+0000 7f1a3da03740 1 -- start start 2024-08-31T06:08:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.497+0000 7f1a3da03740 1 Processor -- start 2024-08-31T06:08:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.497+0000 7f1a3da03740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ef2632f0 0x5589ef641f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.497+0000 7f1a3da03740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589ef26b4d0 0x5589ef6424b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.497+0000 7f1a3da03740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589ef28c1b0 0x5589ef6b0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.497+0000 7f1a3da03740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1a301af890 con 0x5589ef28c1b0 2024-08-31T06:08:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.497+0000 7f1a3da03740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1a301b0d00 con 0x5589ef2632f0 2024-08-31T06:08:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.497+0000 7f1a3da03740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1a301aff60 con 0x5589ef26b4d0 2024-08-31T06:08:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.497+0000 7f1a3da03740 10 client.0 _ll_register_callbacks cb 0x7f1a20009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:08:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.497+0000 7f1a2ea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589ef26b4d0 0x5589ef6424b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.497+0000 7f1a2f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ef2632f0 0x5589ef641f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.497+0000 7f1a2ea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589ef26b4d0 0x5589ef6424b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:58536/0 (socket says 172.21.15.16:58536) 2024-08-31T06:08:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.497+0000 7f1a2ea00640 1 -- 172.21.15.16:0/2091056407 learned_addr learned my addr 172.21.15.16:0/2091056407 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.497+0000 7f1a2fe00640 1 --2- 172.21.15.16:0/2091056407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589ef28c1b0 0x5589ef6b0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.498+0000 7f1a2f400640 1 -- 172.21.15.16:0/2091056407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589ef26b4d0 msgr2=0x5589ef6424b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.498+0000 7f1a2f400640 1 --2- 172.21.15.16:0/2091056407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589ef26b4d0 0x5589ef6424b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.498+0000 7f1a2f400640 1 -- 172.21.15.16:0/2091056407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589ef28c1b0 msgr2=0x5589ef6b0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.498+0000 7f1a2f400640 1 --2- 172.21.15.16:0/2091056407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589ef28c1b0 0x5589ef6b0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.498+0000 7f1a2f400640 1 -- 172.21.15.16:0/2091056407 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589ef6b1300 con 0x5589ef2632f0 2024-08-31T06:08:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.498+0000 7f1a2ea00640 1 --2- 172.21.15.16:0/2091056407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589ef26b4d0 0x5589ef6424b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:08:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.498+0000 7f1a2f400640 1 --2- 172.21.15.16:0/2091056407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ef2632f0 0x5589ef641f10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1a20000c00 tx=0x7f1a20009230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=503eb1a3c7079e51 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.498+0000 7f1a19600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:08:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.498+0000 7f1a19600640 1 -- 172.21.15.16:0/2091056407 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2004c070 con 0x5589ef2632f0 2024-08-31T06:08:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.498+0000 7f1a3da03740 1 -- 172.21.15.16:0/2091056407 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a20003cb0 con 0x5589ef2632f0 2024-08-31T06:08:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.498+0000 7f1a19600640 1 -- 172.21.15.16:0/2091056407 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f1a200052b0 con 0x5589ef2632f0 2024-08-31T06:08:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.499+0000 7f1a19600640 1 -- 172.21.15.16:0/2091056407 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1a20072050 con 0x5589ef2632f0 2024-08-31T06:08:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.499+0000 7f1a3da03740 10 client.25862 fetch_fsmap learned FSMap version 30 2024-08-31T06:08:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.499+0000 7f1a3da03740 1 -- 172.21.15.16:0/2091056407 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5589ef6b8720 con 0x5589ef2632f0 2024-08-31T06:08:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.499+0000 7f1a19600640 1 -- 172.21.15.16:0/2091056407 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a20003cb0 con 0x5589ef2632f0 2024-08-31T06:08:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.499+0000 7f1a19600640 1 -- 172.21.15.16:0/2091056407 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f1a2006a020 con 0x5589ef2632f0 2024-08-31T06:08:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.499+0000 7f1a3da03740 10 client.25862 fetch_fsmap finished waiting for FSMap version 30 2024-08-31T06:08:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.499+0000 7f1a3da03740 10 client.25862 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:08:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.499+0000 7f1a3da03740 20 client.25862 populate_metadata read hostname 'smithi016' 2024-08-31T06:08:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.499+0000 7f1a3da03740 1 --2- 172.21.15.16:0/2091056407 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5589ef2201c0 0x5589ef6bae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.499+0000 7f1a3da03740 4 client.25862 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:08:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.499+0000 7f1a3da03740 1 -- 172.21.15.16:0/2091056407 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5589ef5a2960 con 0x5589ef2201c0 2024-08-31T06:08:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.500+0000 7f1a2ea00640 1 --2- 172.21.15.16:0/2091056407 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5589ef2201c0 0x5589ef6bae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:24.506 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.500+0000 7f1a2ea00640 1 --2- 172.21.15.16:0/2091056407 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5589ef2201c0 0x5589ef6bae40 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=406e315b031ea7f server_cookie=a00710d25fa9040a in_seq=0 out_seq=0 2024-08-31T06:08:24.506 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.500+0000 7f1a19600640 10 client.25862 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:24.516 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.514+0000 7f1a19600640 1 -- 172.21.15.16:0/2091056407 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5589ef5a2960 con 0x5589ef2201c0 2024-08-31T06:08:24.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.514+0000 7f1a19600640 10 client.25862 handle_command_reply: tid=0 2024-08-31T06:08:24.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.514+0000 7f1a3da03740 1 client.25862 shutdown 2024-08-31T06:08:24.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.514+0000 7f1a3da03740 1 -- 172.21.15.16:0/2091056407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ef2632f0 msgr2=0x5589ef641f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:24.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.514+0000 7f1a3da03740 1 --2- 172.21.15.16:0/2091056407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ef2632f0 0x5589ef641f10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1a20000c00 tx=0x7f1a20009230 comp rx=0 tx=0).stop 2024-08-31T06:08:24.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.514+0000 7f1a3da03740 1 -- 172.21.15.16:0/2091056407 shutdown_connections 2024-08-31T06:08:24.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.514+0000 7f1a3da03740 1 --2- 172.21.15.16:0/2091056407 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5589ef2201c0 0x5589ef6bae40 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.514+0000 7f1a3da03740 1 --2- 172.21.15.16:0/2091056407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589ef26b4d0 0x5589ef6424b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.514+0000 7f1a19600640 0 client.25862 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:24.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.514+0000 7f1a3da03740 1 --2- 172.21.15.16:0/2091056407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589ef28c1b0 0x5589ef6b0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.514+0000 7f1a3da03740 1 --2- 172.21.15.16:0/2091056407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589ef2632f0 0x5589ef641f10 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.514+0000 7f1a3da03740 1 -- 172.21.15.16:0/2091056407 >> 172.21.15.16:0/2091056407 conn(0x5589ef280300 msgr2=0x5589ef63c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:24.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.514+0000 7f1a3da03740 1 -- 172.21.15.16:0/2091056407 shutdown_connections 2024-08-31T06:08:24.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.514+0000 7f1a3da03740 1 -- 172.21.15.16:0/2091056407 wait complete. 2024-08-31T06:08:24.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.514+0000 7f1a3da03740 20 client.25862 trim_cache size 0 max 16384 2024-08-31T06:08:24.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:08:24.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:08:24.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:08:24.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:08:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.515+0000 7f1a35200640 1 -- 172.21.15.16:0/3852040109 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f1a140b4d20 msgr2=0x7f1a140b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.515+0000 7f1a35200640 1 --2- 172.21.15.16:0/3852040109 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f1a140b4d20 0x7f1a140b7230 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1a20031440 tx=0x7f1a200058d0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.515+0000 7f1a35200640 1 -- 172.21.15.16:0/3852040109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a3010bb30 msgr2=0x7f1a301a1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.515+0000 7f1a35200640 1 --2- 172.21.15.16:0/3852040109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a3010bb30 0x7f1a301a1a80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1a28009fd0 tx=0x7f1a2800ef10 comp rx=0 tx=0).stop 2024-08-31T06:08:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.515+0000 7f1a35200640 1 -- 172.21.15.16:0/3852040109 shutdown_connections 2024-08-31T06:08:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.515+0000 7f1a35200640 1 --2- 172.21.15.16:0/3852040109 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f1a140b4d20 0x7f1a140b7230 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.515+0000 7f1a35200640 1 --2- 172.21.15.16:0/3852040109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a3010bb30 0x7f1a301a1a80 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.515+0000 7f1a35200640 1 --2- 172.21.15.16:0/3852040109 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a3010b1e0 0x7f1a3019c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.515+0000 7f1a35200640 1 --2- 172.21.15.16:0/3852040109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a301058b0 0x7f1a3019c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.515+0000 7f1a35200640 1 -- 172.21.15.16:0/3852040109 >> 172.21.15.16:0/3852040109 conn(0x7f1a301012f0 msgr2=0x7f1a30103710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.515+0000 7f1a35200640 1 -- 172.21.15.16:0/3852040109 shutdown_connections 2024-08-31T06:08:24.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:24.515+0000 7f1a35200640 1 -- 172.21.15.16:0/3852040109 wait complete. 2024-08-31T06:08:24.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:24 smithi016 ceph-mon[17194]: pgmap v399: 97 pgs: 97 active+clean; 958 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 1.5 MiB/s wr, 188 op/s 2024-08-31T06:08:24.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:24 smithi016 ceph-mon[17194]: scrub queued for path: / 2024-08-31T06:08:24.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:24 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:08:24.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:24 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:08:24.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:24 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:08:24.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:24 smithi184 ceph-mon[22331]: pgmap v399: 97 pgs: 97 active+clean; 958 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 1.5 MiB/s wr, 188 op/s 2024-08-31T06:08:24.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:24 smithi184 ceph-mon[22331]: scrub queued for path: / 2024-08-31T06:08:24.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:24 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:08:24.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:24 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:08:24.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:24 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:08:24.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:24 smithi155 ceph-mon[21742]: pgmap v399: 97 pgs: 97 active+clean; 958 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 1.5 MiB/s wr, 188 op/s 2024-08-31T06:08:24.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:24 smithi155 ceph-mon[21742]: scrub queued for path: / 2024-08-31T06:08:24.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:24 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:08:24.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:24 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:08:24.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:24 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:08:24.897 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T06:08:25.276 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:08:25.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:25 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:08:25.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:25 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:08:25.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:25 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:08:25.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:25 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:08:25.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:25 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:08:25.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:25 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:08:25.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:25 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:08:25.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:25 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:08:25.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:25 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:08:25.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:25 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:08:25.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:25 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:08:25.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:25 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:08:25.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:25 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:08:25.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:25 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:08:25.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:25 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:08:25.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:25 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:08:25.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:25 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:08:25.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:25 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:08:25.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:25 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:08:25.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:25 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:08:25.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:25 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:08:25.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:25 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:08:25.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:25 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:08:25.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:25 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:08:25.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.935+0000 7fa6c5200640 1 -- 172.21.15.16:0/3345744272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6c0075550 msgr2=0x7fa6c0075960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:25.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.935+0000 7fa6c5200640 1 --2- 172.21.15.16:0/3345744272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6c0075550 0x7fa6c0075960 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0009900 tx=0x7fa6b002a3f0 comp rx=0 tx=0).stop 2024-08-31T06:08:25.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.935+0000 7fa6c5200640 1 -- 172.21.15.16:0/3345744272 shutdown_connections 2024-08-31T06:08:25.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.935+0000 7fa6c5200640 1 --2- 172.21.15.16:0/3345744272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6c0075ea0 0x7fa6c010f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:25.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.935+0000 7fa6c5200640 1 --2- 172.21.15.16:0/3345744272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6c010fe60 0x7fa6c01122a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:25.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.935+0000 7fa6c5200640 1 --2- 172.21.15.16:0/3345744272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6c0075550 0x7fa6c0075960 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:25.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.935+0000 7fa6c5200640 1 -- 172.21.15.16:0/3345744272 >> 172.21.15.16:0/3345744272 conn(0x7fa6c007b5f0 msgr2=0x7fa6c007ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:25.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.935+0000 7fa6c5200640 1 -- 172.21.15.16:0/3345744272 shutdown_connections 2024-08-31T06:08:25.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.935+0000 7fa6c5200640 1 -- 172.21.15.16:0/3345744272 wait complete. 2024-08-31T06:08:25.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.936+0000 7fa6c5200640 1 Processor -- start 2024-08-31T06:08:25.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.936+0000 7fa6c5200640 1 -- start start 2024-08-31T06:08:25.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.937+0000 7fa6c5200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6c0075550 0x7fa6c019c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:25.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.937+0000 7fa6c5200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6c0075ea0 0x7fa6c019c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:25.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.937+0000 7fa6c5200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6c010fe60 0x7fa6c01a1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:25.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.937+0000 7fa6c5200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c0115a60 con 0x7fa6c010fe60 2024-08-31T06:08:25.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.937+0000 7fa6c5200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c01158e0 con 0x7fa6c0075ea0 2024-08-31T06:08:25.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.937+0000 7fa6c5200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c0115be0 con 0x7fa6c0075550 2024-08-31T06:08:25.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.937+0000 7fa6bde00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6c0075550 0x7fa6c019c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:25.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.937+0000 7fa6bde00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6c0075550 0x7fa6c019c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:34842/0 (socket says 172.21.15.16:34842) 2024-08-31T06:08:25.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.937+0000 7fa6bde00640 1 -- 172.21.15.16:0/3335808554 learned_addr learned my addr 172.21.15.16:0/3335808554 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:25.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.937+0000 7fa6be800640 1 --2- 172.21.15.16:0/3335808554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6c010fe60 0x7fa6c01a1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:25.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.937+0000 7fa6b7e00640 1 --2- 172.21.15.16:0/3335808554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6c0075ea0 0x7fa6c019c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:25.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.938+0000 7fa6be800640 1 -- 172.21.15.16:0/3335808554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6c0075550 msgr2=0x7fa6c019c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:25.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.938+0000 7fa6be800640 1 --2- 172.21.15.16:0/3335808554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6c0075550 0x7fa6c019c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:25.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.938+0000 7fa6be800640 1 -- 172.21.15.16:0/3335808554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6c0075ea0 msgr2=0x7fa6c019c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:25.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.938+0000 7fa6be800640 1 --2- 172.21.15.16:0/3335808554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6c0075ea0 0x7fa6c019c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:25.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.938+0000 7fa6be800640 1 -- 172.21.15.16:0/3335808554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c01a2360 con 0x7fa6c010fe60 2024-08-31T06:08:25.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.938+0000 7fa6bde00640 1 --2- 172.21.15.16:0/3335808554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6c0075550 0x7fa6c019c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:08:25.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.938+0000 7fa6be800640 1 --2- 172.21.15.16:0/3335808554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6c010fe60 0x7fa6c01a1c30 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8009fd0 tx=0x7fa6b800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:25.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.938+0000 7fa6b6a00640 1 -- 172.21.15.16:0/3335808554 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b800ce20 con 0x7fa6c010fe60 2024-08-31T06:08:25.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.938+0000 7fa6b6a00640 1 -- 172.21.15.16:0/3335808554 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6b8005020 con 0x7fa6c010fe60 2024-08-31T06:08:25.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.938+0000 7fa6b6a00640 1 -- 172.21.15.16:0/3335808554 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8010690 con 0x7fa6c010fe60 2024-08-31T06:08:25.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.938+0000 7fa6c5200640 1 -- 172.21.15.16:0/3335808554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6c01a2650 con 0x7fa6c010fe60 2024-08-31T06:08:25.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.939+0000 7fa6c5200640 1 -- 172.21.15.16:0/3335808554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6c01a8db0 con 0x7fa6c010fe60 2024-08-31T06:08:25.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.942+0000 7fa6cbe2e740 1 Processor -- start 2024-08-31T06:08:25.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.942+0000 7fa6b6a00640 1 -- 172.21.15.16:0/3335808554 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fa6b80054d0 con 0x7fa6c010fe60 2024-08-31T06:08:25.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.943+0000 7fa6cbe2e740 1 -- start start 2024-08-31T06:08:25.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.943+0000 7fa6cbe2e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfb4b3a070 0x55cfb4edf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:25.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.943+0000 7fa6cbe2e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfb4b111b0 0x55cfb4edfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:25.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.943+0000 7fa6cbe2e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cfb4b19390 0x55cfb4ee03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:25.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.943+0000 7fa6cbe2e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cfb4e56770 con 0x55cfb4b111b0 2024-08-31T06:08:25.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.943+0000 7fa6cbe2e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55cfb4e45d00 con 0x55cfb4b19390 2024-08-31T06:08:25.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.943+0000 7fa6cbe2e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55cfb4e49cb0 con 0x55cfb4b3a070 2024-08-31T06:08:25.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.944+0000 7fa6b7e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfb4b3a070 0x55cfb4edf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:25.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.944+0000 7fa6b7e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfb4b3a070 0x55cfb4edf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:34858/0 (socket says 172.21.15.16:34858) 2024-08-31T06:08:25.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.944+0000 7fa6b7e00640 1 -- 172.21.15.16:0/1604086576 learned_addr learned my addr 172.21.15.16:0/1604086576 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:25.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.944+0000 7fa6b7e00640 1 --2- 172.21.15.16:0/1604086576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cfb4b19390 0x55cfb4ee03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:25.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.944+0000 7fa6bde00640 1 --2- 172.21.15.16:0/1604086576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfb4b111b0 0x55cfb4edfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:25.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.944+0000 7fa6b7e00640 1 -- 172.21.15.16:0/1604086576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfb4b3a070 msgr2=0x55cfb4edf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:25.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.944+0000 7fa6b7e00640 1 --2- 172.21.15.16:0/1604086576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfb4b3a070 0x55cfb4edf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:25.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.944+0000 7fa6b7e00640 1 -- 172.21.15.16:0/1604086576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfb4b111b0 msgr2=0x55cfb4edfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:25.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.944+0000 7fa6b7e00640 1 --2- 172.21.15.16:0/1604086576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfb4b111b0 0x55cfb4edfe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:25.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.944+0000 7fa6b7e00640 1 -- 172.21.15.16:0/1604086576 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfb4ba5650 con 0x55cfb4b19390 2024-08-31T06:08:25.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.945+0000 7fa6b7e00640 1 --2- 172.21.15.16:0/1604086576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cfb4b19390 0x55cfb4ee03d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa6a80058c0 tx=0x7fa6a8005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cee1e571e5b90da7 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:25.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.945+0000 7fa6b6a00640 1 --2- 172.21.15.16:0/3335808554 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa6a40b4d20 0x7fa6a40b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:25.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.945+0000 7fa6b6a00640 1 -- 172.21.15.16:0/3335808554 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fa6b80e5bd0 con 0x7fa6c010fe60 2024-08-31T06:08:25.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.945+0000 7fa6af400640 1 -- 172.21.15.16:0/1604086576 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a8011d30 con 0x55cfb4b19390 2024-08-31T06:08:25.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.945+0000 7fa6bde00640 1 --2- 172.21.15.16:0/3335808554 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa6a40b4d20 0x7fa6a40b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:25.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.945+0000 7fa6af400640 1 -- 172.21.15.16:0/1604086576 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6a80054d0 con 0x55cfb4b19390 2024-08-31T06:08:25.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.945+0000 7fa6af400640 1 -- 172.21.15.16:0/1604086576 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a80047a0 con 0x55cfb4b19390 2024-08-31T06:08:25.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.946+0000 7fa6cbe2e740 1 -- 172.21.15.16:0/1604086576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cfb4b19390 msgr2=0x55cfb4ee03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:25.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.946+0000 7fa6bde00640 1 --2- 172.21.15.16:0/3335808554 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa6a40b4d20 0x7fa6a40b7230 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x55cfb4ede5d0 tx=0x7fa6b0005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:25.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.946+0000 7fa6cbe2e740 1 --2- 172.21.15.16:0/1604086576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cfb4b19390 0x55cfb4ee03d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa6a80058c0 tx=0x7fa6a8005f90 comp rx=0 tx=0).stop 2024-08-31T06:08:25.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.946+0000 7fa6cbe2e740 1 -- 172.21.15.16:0/1604086576 shutdown_connections 2024-08-31T06:08:25.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.946+0000 7fa6cbe2e740 1 --2- 172.21.15.16:0/1604086576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cfb4b19390 0x55cfb4ee03d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:25.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.946+0000 7fa6cbe2e740 1 --2- 172.21.15.16:0/1604086576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfb4b111b0 0x55cfb4edfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:25.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.946+0000 7fa6cbe2e740 1 --2- 172.21.15.16:0/1604086576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfb4b3a070 0x55cfb4edf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:25.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.946+0000 7fa6cbe2e740 1 -- 172.21.15.16:0/1604086576 >> 172.21.15.16:0/1604086576 conn(0x55cfb4b2e460 msgr2=0x55cfb4edd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:25.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.946+0000 7fa6cbe2e740 1 -- 172.21.15.16:0/1604086576 shutdown_connections 2024-08-31T06:08:25.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.946+0000 7fa6cbe2e740 1 -- 172.21.15.16:0/1604086576 wait complete. 2024-08-31T06:08:25.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.947+0000 7fa6cbe2e740 1 -- start start 2024-08-31T06:08:25.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.947+0000 7fa6cbe2e740 1 Processor -- start 2024-08-31T06:08:25.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.948+0000 7fa6cbe2e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfb4b111b0 0x55cfb4eeb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:25.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.948+0000 7fa6cbe2e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cfb4b19390 0x55cfb4eebaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:25.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.948+0000 7fa6cbe2e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfb4b3a070 0x55cfb4ef7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:25.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.948+0000 7fa6cbe2e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cfb4ca8700 con 0x55cfb4b3a070 2024-08-31T06:08:25.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.948+0000 7fa6cbe2e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55cfb4dcccb0 con 0x55cfb4b19390 2024-08-31T06:08:25.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.948+0000 7fa6cbe2e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55cfb4eee130 con 0x55cfb4b111b0 2024-08-31T06:08:25.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.948+0000 7fa6cbe2e740 10 client.0 _ll_register_callbacks cb 0x55cfb4df2430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:08:25.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.948+0000 7fa6b7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cfb4b19390 0x55cfb4eebaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:25.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.948+0000 7fa6b7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cfb4b19390 0x55cfb4eebaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:33528/0 (socket says 172.21.15.16:33528) 2024-08-31T06:08:25.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.948+0000 7fa6b7e00640 1 -- 172.21.15.16:0/3239979767 learned_addr learned my addr 172.21.15.16:0/3239979767 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:25.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.948+0000 7fa6bde00640 1 --2- 172.21.15.16:0/3239979767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfb4b111b0 0x55cfb4eeb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:25.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.948+0000 7fa6be800640 1 --2- 172.21.15.16:0/3239979767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfb4b3a070 0x55cfb4ef7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:25.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.949+0000 7fa6b7e00640 1 -- 172.21.15.16:0/3239979767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfb4b111b0 msgr2=0x55cfb4eeb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:25.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.949+0000 7fa6b7e00640 1 --2- 172.21.15.16:0/3239979767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfb4b111b0 0x55cfb4eeb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:25.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.949+0000 7fa6b7e00640 1 -- 172.21.15.16:0/3239979767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfb4b3a070 msgr2=0x55cfb4ef7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:25.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.949+0000 7fa6b7e00640 1 --2- 172.21.15.16:0/3239979767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfb4b3a070 0x55cfb4ef7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:25.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.949+0000 7fa6b7e00640 1 -- 172.21.15.16:0/3239979767 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfb4e0d870 con 0x55cfb4b19390 2024-08-31T06:08:25.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.949+0000 7fa6b7e00640 1 --2- 172.21.15.16:0/3239979767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cfb4b19390 0x55cfb4eebaf0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa6a800fb00 tx=0x7fa6a8009de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff17d1826d1c2f05 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:25.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.949+0000 7fa6a3e00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:08:25.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.949+0000 7fa6a3e00640 1 -- 172.21.15.16:0/3239979767 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a80046c0 con 0x55cfb4b19390 2024-08-31T06:08:25.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.949+0000 7fa6cbe2e740 1 -- 172.21.15.16:0/3239979767 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6a80054d0 con 0x55cfb4b19390 2024-08-31T06:08:25.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.949+0000 7fa6a3e00640 1 -- 172.21.15.16:0/3239979767 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fa6a8004860 con 0x55cfb4b19390 2024-08-31T06:08:25.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.950+0000 7fa6a3e00640 1 -- 172.21.15.16:0/3239979767 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa6a802e050 con 0x55cfb4b19390 2024-08-31T06:08:25.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.950+0000 7fa6cbe2e740 10 client.25874 fetch_fsmap learned FSMap version 30 2024-08-31T06:08:25.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.950+0000 7fa6cbe2e740 1 -- 172.21.15.16:0/3239979767 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cfb4efa1a0 con 0x55cfb4b19390 2024-08-31T06:08:25.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.950+0000 7fa6a3e00640 1 -- 172.21.15.16:0/3239979767 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6a80054d0 con 0x55cfb4b19390 2024-08-31T06:08:25.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.950+0000 7fa6a3e00640 1 -- 172.21.15.16:0/3239979767 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fa6a801d030 con 0x55cfb4b19390 2024-08-31T06:08:25.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.950+0000 7fa6cbe2e740 10 client.25874 fetch_fsmap finished waiting for FSMap version 30 2024-08-31T06:08:25.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.950+0000 7fa6cbe2e740 10 client.25874 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:08:25.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.950+0000 7fa6cbe2e740 20 client.25874 populate_metadata read hostname 'smithi016' 2024-08-31T06:08:25.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.950+0000 7fa6cbe2e740 1 --2- 172.21.15.16:0/3239979767 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55cfb4af89f0 0x55cfb4efd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:25.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.950+0000 7fa6cbe2e740 4 client.25874 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:08:25.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.950+0000 7fa6cbe2e740 1 -- 172.21.15.16:0/3239979767 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfb4e586c0 con 0x55cfb4af89f0 2024-08-31T06:08:25.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.951+0000 7fa6bde00640 1 --2- 172.21.15.16:0/3239979767 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55cfb4af89f0 0x55cfb4efd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:25.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.952+0000 7fa6bde00640 1 --2- 172.21.15.16:0/3239979767 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55cfb4af89f0 0x55cfb4efd500 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=773e4fd45ade937c server_cookie=2731fc38e884befb in_seq=0 out_seq=0 2024-08-31T06:08:25.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.952+0000 7fa6a3e00640 10 client.25874 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:25.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.952+0000 7fa6a3e00640 1 -- 172.21.15.16:0/3239979767 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cfb4e586c0 con 0x55cfb4af89f0 2024-08-31T06:08:25.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.952+0000 7fa6a3e00640 10 client.25874 handle_command_reply: tid=0 2024-08-31T06:08:25.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.952+0000 7fa6cbe2e740 1 client.25874 shutdown 2024-08-31T06:08:25.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.953+0000 7fa6cbe2e740 1 -- 172.21.15.16:0/3239979767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cfb4b19390 msgr2=0x55cfb4eebaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:25.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.953+0000 7fa6cbe2e740 1 --2- 172.21.15.16:0/3239979767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cfb4b19390 0x55cfb4eebaf0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa6a800fb00 tx=0x7fa6a8009de0 comp rx=0 tx=0).stop 2024-08-31T06:08:25.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.953+0000 7fa6cbe2e740 1 -- 172.21.15.16:0/3239979767 shutdown_connections 2024-08-31T06:08:25.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.953+0000 7fa6cbe2e740 1 --2- 172.21.15.16:0/3239979767 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55cfb4af89f0 0x55cfb4efd500 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:25.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.953+0000 7fa6cbe2e740 1 --2- 172.21.15.16:0/3239979767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfb4b3a070 0x55cfb4ef7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:25.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.953+0000 7fa6cbe2e740 1 --2- 172.21.15.16:0/3239979767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cfb4b19390 0x55cfb4eebaf0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:25.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.953+0000 7fa6a3e00640 0 client.25874 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:25.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.953+0000 7fa6cbe2e740 1 --2- 172.21.15.16:0/3239979767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfb4b111b0 0x55cfb4eeb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:25.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.953+0000 7fa6cbe2e740 1 -- 172.21.15.16:0/3239979767 >> 172.21.15.16:0/3239979767 conn(0x55cfb4b2e460 msgr2=0x55cfb4ee2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:25.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.953+0000 7fa6cbe2e740 1 -- 172.21.15.16:0/3239979767 shutdown_connections 2024-08-31T06:08:25.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.953+0000 7fa6cbe2e740 1 -- 172.21.15.16:0/3239979767 wait complete. 2024-08-31T06:08:25.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.953+0000 7fa6cbe2e740 20 client.25874 trim_cache size 0 max 16384 2024-08-31T06:08:25.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.971+0000 7fa6cbe2e740 1 Processor -- start 2024-08-31T06:08:25.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.972+0000 7fa6cbe2e740 1 -- start start 2024-08-31T06:08:25.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.972+0000 7fa6cbe2e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cfb4b111b0 0x55cfb4ee1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:25.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.972+0000 7fa6cbe2e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfb4b19390 0x55cfb4edcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:25.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.972+0000 7fa6cbe2e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfb4b3a070 0x55cfb4edd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:25.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.972+0000 7fa6cbe2e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c01b3e10 con 0x55cfb4b19390 2024-08-31T06:08:25.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.972+0000 7fa6cbe2e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c01b1650 con 0x55cfb4b111b0 2024-08-31T06:08:25.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.972+0000 7fa6cbe2e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c01af4a0 con 0x55cfb4b3a070 2024-08-31T06:08:25.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.972+0000 7fa6b7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfb4b19390 0x55cfb4edcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:25.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.972+0000 7fa6bde00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cfb4b111b0 0x55cfb4ee1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:25.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.972+0000 7fa6b7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfb4b19390 0x55cfb4edcdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59608/0 (socket says 172.21.15.16:59608) 2024-08-31T06:08:25.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.972+0000 7fa6b7e00640 1 -- 172.21.15.16:0/891944134 learned_addr learned my addr 172.21.15.16:0/891944134 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:25.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.973+0000 7fa6be800640 1 --2- 172.21.15.16:0/891944134 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfb4b3a070 0x55cfb4edd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:25.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.973+0000 7fa6bde00640 1 -- 172.21.15.16:0/891944134 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfb4b3a070 msgr2=0x55cfb4edd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:25.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.973+0000 7fa6bde00640 1 --2- 172.21.15.16:0/891944134 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfb4b3a070 0x55cfb4edd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:25.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.973+0000 7fa6bde00640 1 -- 172.21.15.16:0/891944134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfb4b19390 msgr2=0x55cfb4edcdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:25.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.973+0000 7fa6bde00640 1 --2- 172.21.15.16:0/891944134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfb4b19390 0x55cfb4edcdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:25.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.973+0000 7fa6bde00640 1 -- 172.21.15.16:0/891944134 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfb4eec430 con 0x55cfb4b111b0 2024-08-31T06:08:25.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.973+0000 7fa6b7e00640 1 --2- 172.21.15.16:0/891944134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfb4b19390 0x55cfb4edcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:08:25.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.973+0000 7fa6be800640 1 --2- 172.21.15.16:0/891944134 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfb4b3a070 0x55cfb4edd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:08:25.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.973+0000 7fa6bde00640 1 --2- 172.21.15.16:0/891944134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cfb4b111b0 0x55cfb4ee1090 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0031440 tx=0x7fa6b0040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36e802bf98798579 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:25.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.973+0000 7fa6aea00640 1 -- 172.21.15.16:0/891944134 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b0035040 con 0x55cfb4b111b0 2024-08-31T06:08:25.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.974+0000 7fa6aea00640 1 -- 172.21.15.16:0/891944134 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6b0031da0 con 0x55cfb4b111b0 2024-08-31T06:08:25.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.974+0000 7fa6aea00640 1 -- 172.21.15.16:0/891944134 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b003dc80 con 0x55cfb4b111b0 2024-08-31T06:08:25.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.974+0000 7fa6cbe2e740 1 -- 172.21.15.16:0/891944134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cfb4b111b0 msgr2=0x55cfb4ee1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:25.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.974+0000 7fa6cbe2e740 1 --2- 172.21.15.16:0/891944134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cfb4b111b0 0x55cfb4ee1090 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0031440 tx=0x7fa6b0040000 comp rx=0 tx=0).stop 2024-08-31T06:08:25.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.974+0000 7fa6cbe2e740 1 -- 172.21.15.16:0/891944134 shutdown_connections 2024-08-31T06:08:25.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.974+0000 7fa6cbe2e740 1 --2- 172.21.15.16:0/891944134 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfb4b3a070 0x55cfb4edd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:25.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.974+0000 7fa6cbe2e740 1 --2- 172.21.15.16:0/891944134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfb4b19390 0x55cfb4edcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:25.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.974+0000 7fa6cbe2e740 1 --2- 172.21.15.16:0/891944134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cfb4b111b0 0x55cfb4ee1090 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:25.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.974+0000 7fa6cbe2e740 1 -- 172.21.15.16:0/891944134 >> 172.21.15.16:0/891944134 conn(0x55cfb4b2e460 msgr2=0x55cfb4ee0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:25.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.975+0000 7fa6cbe2e740 1 -- 172.21.15.16:0/891944134 shutdown_connections 2024-08-31T06:08:25.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.975+0000 7fa6cbe2e740 1 -- 172.21.15.16:0/891944134 wait complete. 2024-08-31T06:08:25.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.975+0000 7fa6cbe2e740 1 -- start start 2024-08-31T06:08:25.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.975+0000 7fa6cbe2e740 1 Processor -- start 2024-08-31T06:08:25.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.976+0000 7fa6cbe2e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfb4b111b0 0x55cfb4eef470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:25.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.976+0000 7fa6cbe2e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cfb4b19390 0x55cfb4eef9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:25.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.976+0000 7fa6cbe2e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfb4b3a070 0x55cfb4eefef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:25.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.976+0000 7fa6cbe2e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c01afb10 con 0x55cfb4b111b0 2024-08-31T06:08:25.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.976+0000 7fa6cbe2e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c01b0f80 con 0x55cfb4b19390 2024-08-31T06:08:25.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.976+0000 7fa6cbe2e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c01b01e0 con 0x55cfb4b3a070 2024-08-31T06:08:25.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.976+0000 7fa6cbe2e740 10 client.0 _ll_register_callbacks cb 0x7fa6a80116e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:08:25.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.976+0000 7fa6bde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfb4b111b0 0x55cfb4eef470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:25.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.976+0000 7fa6b7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cfb4b19390 0x55cfb4eef9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:25.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.976+0000 7fa6be800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfb4b3a070 0x55cfb4eefef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:25.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.976+0000 7fa6b7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cfb4b19390 0x55cfb4eef9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:33546/0 (socket says 172.21.15.16:33546) 2024-08-31T06:08:25.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.976+0000 7fa6b7e00640 1 -- 172.21.15.16:0/466880044 learned_addr learned my addr 172.21.15.16:0/466880044 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:25.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.976+0000 7fa6b7e00640 1 -- 172.21.15.16:0/466880044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfb4b3a070 msgr2=0x55cfb4eefef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:25.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.977+0000 7fa6b7e00640 1 --2- 172.21.15.16:0/466880044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfb4b3a070 0x55cfb4eefef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:25.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.977+0000 7fa6b7e00640 1 -- 172.21.15.16:0/466880044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfb4b111b0 msgr2=0x55cfb4eef470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:25.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.977+0000 7fa6b7e00640 1 --2- 172.21.15.16:0/466880044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfb4b111b0 0x55cfb4eef470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:25.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.977+0000 7fa6b7e00640 1 -- 172.21.15.16:0/466880044 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfb4f66e50 con 0x55cfb4b19390 2024-08-31T06:08:25.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.977+0000 7fa6bde00640 1 --2- 172.21.15.16:0/466880044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfb4b111b0 0x55cfb4eef470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:08:25.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.977+0000 7fa6b7e00640 1 --2- 172.21.15.16:0/466880044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cfb4b19390 0x55cfb4eef9b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8027a10 tx=0x7fa6a8027c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0235c2582629534 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:25.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.977+0000 7fa6a3e00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:08:25.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.977+0000 7fa6a3e00640 1 -- 172.21.15.16:0/466880044 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a800fac0 con 0x55cfb4b19390 2024-08-31T06:08:25.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.977+0000 7fa6a3e00640 1 -- 172.21.15.16:0/466880044 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fa6a8027630 con 0x55cfb4b19390 2024-08-31T06:08:25.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.977+0000 7fa6cbe2e740 1 -- 172.21.15.16:0/466880044 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6b0031da0 con 0x55cfb4b19390 2024-08-31T06:08:25.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.978+0000 7fa6a3e00640 1 -- 172.21.15.16:0/466880044 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa6a802b030 con 0x55cfb4b19390 2024-08-31T06:08:25.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.978+0000 7fa6cbe2e740 10 client.25880 fetch_fsmap learned FSMap version 30 2024-08-31T06:08:25.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.978+0000 7fa6cbe2e740 1 -- 172.21.15.16:0/466880044 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cfb4f680c0 con 0x55cfb4b19390 2024-08-31T06:08:25.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.978+0000 7fa6a3e00640 1 -- 172.21.15.16:0/466880044 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6b0031da0 con 0x55cfb4b19390 2024-08-31T06:08:25.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.978+0000 7fa6a3e00640 1 -- 172.21.15.16:0/466880044 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fa6a803a020 con 0x55cfb4b19390 2024-08-31T06:08:25.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.978+0000 7fa6cbe2e740 10 client.25880 fetch_fsmap finished waiting for FSMap version 30 2024-08-31T06:08:25.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.978+0000 7fa6cbe2e740 10 client.25880 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:08:25.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.978+0000 7fa6cbe2e740 20 client.25880 populate_metadata read hostname 'smithi016' 2024-08-31T06:08:25.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.978+0000 7fa6cbe2e740 1 --2- 172.21.15.16:0/466880044 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55cfb4acdda0 0x55cfb4f6a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:25.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.978+0000 7fa6cbe2e740 4 client.25880 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T06:08:25.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.979+0000 7fa6cbe2e740 1 -- 172.21.15.16:0/466880044 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cfb4e54160 con 0x55cfb4acdda0 2024-08-31T06:08:25.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.979+0000 7fa6bde00640 1 --2- 172.21.15.16:0/466880044 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55cfb4acdda0 0x55cfb4f6a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:25.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.979+0000 7fa6bde00640 1 --2- 172.21.15.16:0/466880044 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55cfb4acdda0 0x55cfb4f6a8b0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2222c4b628b6a4ec server_cookie=66ab52c3d8e1a891 in_seq=0 out_seq=0 2024-08-31T06:08:25.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.979+0000 7fa6a3e00640 10 client.25880 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:25.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.980+0000 7fa6a3e00640 1 -- 172.21.15.16:0/466880044 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1138 (crc 0 0 0) 0x55cfb4e54160 con 0x55cfb4acdda0 2024-08-31T06:08:25.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.980+0000 7fa6a3e00640 10 client.25880 handle_command_reply: tid=0 2024-08-31T06:08:25.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.980+0000 7fa6cbe2e740 1 client.25880 shutdown 2024-08-31T06:08:25.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.980+0000 7fa6cbe2e740 1 -- 172.21.15.16:0/466880044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cfb4b19390 msgr2=0x55cfb4eef9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:25.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.980+0000 7fa6cbe2e740 1 --2- 172.21.15.16:0/466880044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cfb4b19390 0x55cfb4eef9b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8027a10 tx=0x7fa6a8027c10 comp rx=0 tx=0).stop 2024-08-31T06:08:25.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.980+0000 7fa6cbe2e740 1 -- 172.21.15.16:0/466880044 shutdown_connections 2024-08-31T06:08:25.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.980+0000 7fa6cbe2e740 1 --2- 172.21.15.16:0/466880044 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55cfb4acdda0 0x55cfb4f6a8b0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:25.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.980+0000 7fa6cbe2e740 1 --2- 172.21.15.16:0/466880044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfb4b3a070 0x55cfb4eefef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:25.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.980+0000 7fa6cbe2e740 1 --2- 172.21.15.16:0/466880044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cfb4b19390 0x55cfb4eef9b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:25.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.980+0000 7fa6cbe2e740 1 --2- 172.21.15.16:0/466880044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfb4b111b0 0x55cfb4eef470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:25.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.980+0000 7fa6a3e00640 0 client.25880 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:25.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.980+0000 7fa6cbe2e740 1 -- 172.21.15.16:0/466880044 >> 172.21.15.16:0/466880044 conn(0x55cfb4b2e460 msgr2=0x55cfb4eec650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:25.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.980+0000 7fa6cbe2e740 1 -- 172.21.15.16:0/466880044 shutdown_connections 2024-08-31T06:08:25.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.980+0000 7fa6cbe2e740 1 -- 172.21.15.16:0/466880044 wait complete. 2024-08-31T06:08:25.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.980+0000 7fa6cbe2e740 20 client.25880 trim_cache size 0 max 16384 2024-08-31T06:08:25.986 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:08:25.986 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T06:08:25.986 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-08-31T06:08:25.986 INFO:teuthology.orchestra.run.smithi016.stdout: "c8455d74-2fe7-4d93-bcf2-19b419d16016": { 2024-08-31T06:08:25.986 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:08:25.986 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "c8455d74-2fe7-4d93-bcf2-19b419d16016", 2024-08-31T06:08:25.986 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-08-31T06:08:25.986 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:08:25.987 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:08:25.987 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-08-31T06:08:25.987 INFO:teuthology.orchestra.run.smithi016.stdout: "c8455d74-2fe7-4d93-bcf2-19b419d16016": { 2024-08-31T06:08:25.987 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:08:24.035960+0000", 2024-08-31T06:08:25.987 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:08:25.987 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:08:25.987 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:08:25.987 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:08:25.987 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:08:25.987 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:08:25.987 INFO:teuthology.orchestra.run.smithi016.stdout: "88d7bdb8-cb1c-45e5-ad29-4cd8da6c6003": { 2024-08-31T06:08:25.987 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:07:39.648942+0000", 2024-08-31T06:08:25.987 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:08:25.987 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:08:25.988 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:08:25.988 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:08:25.988 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:08:25.988 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:08:25.988 INFO:teuthology.orchestra.run.smithi016.stdout: "c300b7e8-f3fd-482c-a909-9329bc61e570": { 2024-08-31T06:08:25.988 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:06:49.868046+0000", 2024-08-31T06:08:25.988 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:08:25.988 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:08:25.988 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:08:25.988 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:08:25.988 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:08:25.988 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:08:25.988 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:08:25.988 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:08:25.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.981+0000 7fa6c5200640 1 -- 172.21.15.16:0/3335808554 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa6a40b4d20 msgr2=0x7fa6a40b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:25.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.981+0000 7fa6c5200640 1 --2- 172.21.15.16:0/3335808554 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa6a40b4d20 0x7fa6a40b7230 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x55cfb4ede5d0 tx=0x7fa6b0005a20 comp rx=0 tx=0).stop 2024-08-31T06:08:25.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.981+0000 7fa6c5200640 1 -- 172.21.15.16:0/3335808554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6c010fe60 msgr2=0x7fa6c01a1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:25.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.981+0000 7fa6c5200640 1 --2- 172.21.15.16:0/3335808554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6c010fe60 0x7fa6c01a1c30 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8009fd0 tx=0x7fa6b800ef10 comp rx=0 tx=0).stop 2024-08-31T06:08:25.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.981+0000 7fa6c5200640 1 -- 172.21.15.16:0/3335808554 shutdown_connections 2024-08-31T06:08:25.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.981+0000 7fa6c5200640 1 --2- 172.21.15.16:0/3335808554 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa6a40b4d20 0x7fa6a40b7230 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:25.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.981+0000 7fa6c5200640 1 --2- 172.21.15.16:0/3335808554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6c010fe60 0x7fa6c01a1c30 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:25.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.981+0000 7fa6c5200640 1 --2- 172.21.15.16:0/3335808554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6c0075ea0 0x7fa6c019c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:25.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.981+0000 7fa6c5200640 1 --2- 172.21.15.16:0/3335808554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6c0075550 0x7fa6c019c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:25.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.981+0000 7fa6c5200640 1 -- 172.21.15.16:0/3335808554 >> 172.21.15.16:0/3335808554 conn(0x7fa6c007b5f0 msgr2=0x7fa6c01099e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:25.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.981+0000 7fa6c5200640 1 -- 172.21.15.16:0/3335808554 shutdown_connections 2024-08-31T06:08:25.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:25.981+0000 7fa6c5200640 1 -- 172.21.15.16:0/3335808554 wait complete. 2024-08-31T06:08:26.628 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 60.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-31T06:08:26.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:26 smithi016 ceph-mon[17194]: pgmap v400: 97 pgs: 97 active+clean; 964 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 2.0 MiB/s wr, 263 op/s 2024-08-31T06:08:26.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:26 smithi184 ceph-mon[22331]: pgmap v400: 97 pgs: 97 active+clean; 964 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 2.0 MiB/s wr, 263 op/s 2024-08-31T06:08:26.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:26 smithi155 ceph-mon[21742]: pgmap v400: 97 pgs: 97 active+clean; 964 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 2.0 MiB/s wr, 263 op/s 2024-08-31T06:08:27.014 INFO:tasks.cephfs.filesystem:scrub status for tag:c8455d74-2fe7-4d93-bcf2-19b419d16016 - {'path': '/', 'tag': 'c8455d74-2fe7-4d93-bcf2-19b419d16016', 'options': 'recursive,force'} 2024-08-31T06:08:27.371 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:08:27.762 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:08:28.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.421+0000 7f4173200640 1 -- 172.21.15.16:0/3339282549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4174109090 msgr2=0x7f41741094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:28.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.421+0000 7f4173200640 1 --2- 172.21.15.16:0/3339282549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4174109090 0x7f41741094a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4160009900 tx=0x7f416002a410 comp rx=0 tx=0).stop 2024-08-31T06:08:28.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.421+0000 7f4173200640 1 -- 172.21.15.16:0/3339282549 shutdown_connections 2024-08-31T06:08:28.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.421+0000 7f4173200640 1 --2- 172.21.15.16:0/3339282549 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41741099e0 0x7f4174110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:28.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.421+0000 7f4173200640 1 --2- 172.21.15.16:0/3339282549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4174109090 0x7f41741094a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:28.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.421+0000 7f4173200640 1 --2- 172.21.15.16:0/3339282549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4174103750 0x7f4174103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:28.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.421+0000 7f4173200640 1 -- 172.21.15.16:0/3339282549 >> 172.21.15.16:0/3339282549 conn(0x7f41740ff1e0 msgr2=0x7f4174101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:28.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.421+0000 7f4173200640 1 -- 172.21.15.16:0/3339282549 shutdown_connections 2024-08-31T06:08:28.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.422+0000 7f4173200640 1 -- 172.21.15.16:0/3339282549 wait complete. 2024-08-31T06:08:28.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.423+0000 7f4173200640 1 Processor -- start 2024-08-31T06:08:28.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.423+0000 7f4173200640 1 -- start start 2024-08-31T06:08:28.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.423+0000 7f4173200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4174103750 0x7f417419a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:28.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.423+0000 7f4173200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4174109090 0x7f417419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:28.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.423+0000 7f4173200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41741099e0 0x7f417419f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:28.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.423+0000 7f4173200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f41741136a0 con 0x7f4174109090 2024-08-31T06:08:28.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.423+0000 7f4173200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4174113520 con 0x7f4174103750 2024-08-31T06:08:28.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.423+0000 7f4173200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4174113820 con 0x7f41741099e0 2024-08-31T06:08:28.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.423+0000 7f4171400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4174109090 0x7f417419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:28.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.424+0000 7f4171400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4174109090 0x7f417419a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59642/0 (socket says 172.21.15.16:59642) 2024-08-31T06:08:28.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.424+0000 7f4171400640 1 -- 172.21.15.16:0/7123202 learned_addr learned my addr 172.21.15.16:0/7123202 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:28.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.424+0000 7f4171e00640 1 --2- 172.21.15.16:0/7123202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4174103750 0x7f417419a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:28.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.424+0000 7f4172800640 1 --2- 172.21.15.16:0/7123202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41741099e0 0x7f417419f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:28.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.424+0000 7f4171e00640 1 -- 172.21.15.16:0/7123202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41741099e0 msgr2=0x7f417419f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:28.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.424+0000 7f4171e00640 1 --2- 172.21.15.16:0/7123202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41741099e0 0x7f417419f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:28.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.424+0000 7f4171e00640 1 -- 172.21.15.16:0/7123202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4174109090 msgr2=0x7f417419a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:28.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.424+0000 7f4171e00640 1 --2- 172.21.15.16:0/7123202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4174109090 0x7f417419a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:28.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.424+0000 7f4171e00640 1 -- 172.21.15.16:0/7123202 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41741a0040 con 0x7f4174103750 2024-08-31T06:08:28.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.424+0000 7f4171400640 1 --2- 172.21.15.16:0/7123202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4174109090 0x7f417419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:08:28.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.424+0000 7f4171e00640 1 --2- 172.21.15.16:0/7123202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4174103750 0x7f417419a0d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f416400e9f0 tx=0x7f416400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:28.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.425+0000 7f416aa00640 1 -- 172.21.15.16:0/7123202 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416400cd80 con 0x7f4174103750 2024-08-31T06:08:28.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.425+0000 7f4173200640 1 -- 172.21.15.16:0/7123202 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41741a0330 con 0x7f4174103750 2024-08-31T06:08:28.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.425+0000 7f4173200640 1 -- 172.21.15.16:0/7123202 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41741a6a90 con 0x7f4174103750 2024-08-31T06:08:28.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.425+0000 7f416aa00640 1 -- 172.21.15.16:0/7123202 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41640035f0 con 0x7f4174103750 2024-08-31T06:08:28.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.425+0000 7f416aa00640 1 -- 172.21.15.16:0/7123202 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4164010690 con 0x7f4174103750 2024-08-31T06:08:28.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.426+0000 7f4173200640 1 -- 172.21.15.16:0/7123202 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41540052c0 con 0x7f4174103750 2024-08-31T06:08:28.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.428+0000 7f416aa00640 1 -- 172.21.15.16:0/7123202 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f4164004000 con 0x7f4174103750 2024-08-31T06:08:28.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.430+0000 7f416aa00640 1 --2- 172.21.15.16:0/7123202 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f41580b4d20 0x7f41580b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:28.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.430+0000 7f416aa00640 1 -- 172.21.15.16:0/7123202 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f41640e5580 con 0x7f4174103750 2024-08-31T06:08:28.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.430+0000 7f4171400640 1 --2- 172.21.15.16:0/7123202 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f41580b4d20 0x7f41580b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:28.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.431+0000 7f4171400640 1 --2- 172.21.15.16:0/7123202 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f41580b4d20 0x7f41580b7230 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4160006010 tx=0x7f4160005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:28.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.432+0000 7f416aa00640 1 -- 172.21.15.16:0/7123202 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f41640ac370 con 0x7f4174103750 2024-08-31T06:08:28.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:28 smithi016 ceph-mon[17194]: pgmap v401: 97 pgs: 97 active+clean; 964 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 2.0 MiB/s wr, 275 op/s 2024-08-31T06:08:28.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.663+0000 7f4173200640 1 -- 172.21.15.16:0/7123202 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41540055b0 con 0x7f4174103750 2024-08-31T06:08:28.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.664+0000 7f416aa00640 1 -- 172.21.15.16:0/7123202 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11921 (secure 0 0 0) 0x7f41640abc70 con 0x7f4174103750 2024-08-31T06:08:28.667 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:08:28.667 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":30,"btime":"2024-08-31T06:07:34:013454+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:07:33.329391+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24712,24724,24737]},"id":1}]} 2024-08-31T06:08:28.667 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 30 2024-08-31T06:08:28.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.666+0000 7f4173200640 1 -- 172.21.15.16:0/7123202 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f41580b4d20 msgr2=0x7f41580b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:28.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.666+0000 7f4173200640 1 --2- 172.21.15.16:0/7123202 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f41580b4d20 0x7f41580b7230 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4160006010 tx=0x7f4160005a20 comp rx=0 tx=0).stop 2024-08-31T06:08:28.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.667+0000 7f4173200640 1 -- 172.21.15.16:0/7123202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4174103750 msgr2=0x7f417419a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:28.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.667+0000 7f4173200640 1 --2- 172.21.15.16:0/7123202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4174103750 0x7f417419a0d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f416400e9f0 tx=0x7f416400eeb0 comp rx=0 tx=0).stop 2024-08-31T06:08:28.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.667+0000 7f4173200640 1 -- 172.21.15.16:0/7123202 shutdown_connections 2024-08-31T06:08:28.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.667+0000 7f4173200640 1 --2- 172.21.15.16:0/7123202 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f41580b4d20 0x7f41580b7230 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:28.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.667+0000 7f4173200640 1 --2- 172.21.15.16:0/7123202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41741099e0 0x7f417419f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:28.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.667+0000 7f4173200640 1 --2- 172.21.15.16:0/7123202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4174109090 0x7f417419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:28.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.667+0000 7f4173200640 1 --2- 172.21.15.16:0/7123202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4174103750 0x7f417419a0d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:28.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.667+0000 7f4173200640 1 -- 172.21.15.16:0/7123202 >> 172.21.15.16:0/7123202 conn(0x7f41740ff1e0 msgr2=0x7f4174101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:28.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.667+0000 7f4173200640 1 -- 172.21.15.16:0/7123202 shutdown_connections 2024-08-31T06:08:28.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:28.667+0000 7f4173200640 1 -- 172.21.15.16:0/7123202 wait complete. 2024-08-31T06:08:28.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:28 smithi184 ceph-mon[22331]: pgmap v401: 97 pgs: 97 active+clean; 964 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 2.0 MiB/s wr, 275 op/s 2024-08-31T06:08:28.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:28 smithi155 ceph-mon[21742]: pgmap v401: 97 pgs: 97 active+clean; 964 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 2.0 MiB/s wr, 275 op/s 2024-08-31T06:08:29.508 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24737 quiesce db --set-id a4339836 --release --await' 2024-08-31T06:08:29.508 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24737 quiesce db --set-id a4339836 --release --await 2024-08-31T06:08:29.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:29 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/7123202' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:08:29.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:29 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/7123202' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:08:29.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:29 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/7123202' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:08:29.890 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:08:30.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.519+0000 7ff6f3200640 1 -- 172.21.15.16:0/4007329537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6f4103750 msgr2=0x7ff6f4103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:30.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.519+0000 7ff6f3200640 1 --2- 172.21.15.16:0/4007329537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6f4103750 0x7ff6f4103b60 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4009900 tx=0x7ff6e402a3f0 comp rx=0 tx=0).stop 2024-08-31T06:08:30.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.519+0000 7ff6f3200640 1 -- 172.21.15.16:0/4007329537 shutdown_connections 2024-08-31T06:08:30.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.519+0000 7ff6f3200640 1 --2- 172.21.15.16:0/4007329537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6f41099e0 0x7ff6f4110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:30.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.519+0000 7ff6f3200640 1 --2- 172.21.15.16:0/4007329537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6f4109090 0x7ff6f41094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:30.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.519+0000 7ff6f3200640 1 --2- 172.21.15.16:0/4007329537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6f4103750 0x7ff6f4103b60 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:30.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.519+0000 7ff6f3200640 1 -- 172.21.15.16:0/4007329537 >> 172.21.15.16:0/4007329537 conn(0x7ff6f40ff1e0 msgr2=0x7ff6f4101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:30.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.519+0000 7ff6f3200640 1 -- 172.21.15.16:0/4007329537 shutdown_connections 2024-08-31T06:08:30.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.519+0000 7ff6f3200640 1 -- 172.21.15.16:0/4007329537 wait complete. 2024-08-31T06:08:30.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.521+0000 7ff6f3200640 1 Processor -- start 2024-08-31T06:08:30.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.521+0000 7ff6f3200640 1 -- start start 2024-08-31T06:08:30.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.521+0000 7ff6f3200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6f4103750 0x7ff6f419c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:30.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.521+0000 7ff6f3200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6f4109090 0x7ff6f419c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:30.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.521+0000 7ff6f3200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6f41099e0 0x7ff6f41a1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:30.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.521+0000 7ff6f3200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f41136e0 con 0x7ff6f4103750 2024-08-31T06:08:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.521+0000 7ff6f3200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f4113560 con 0x7ff6f41099e0 2024-08-31T06:08:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.521+0000 7ff6f3200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f4113860 con 0x7ff6f4109090 2024-08-31T06:08:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.522+0000 7ff6f1e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6f4103750 0x7ff6f419c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.522+0000 7ff6f1e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6f4103750 0x7ff6f419c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59668/0 (socket says 172.21.15.16:59668) 2024-08-31T06:08:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.522+0000 7ff6f1e00640 1 -- 172.21.15.16:0/1315287558 learned_addr learned my addr 172.21.15.16:0/1315287558 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.522+0000 7ff6f2800640 1 --2- 172.21.15.16:0/1315287558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6f41099e0 0x7ff6f41a1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.522+0000 7ff6f1400640 1 --2- 172.21.15.16:0/1315287558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6f4109090 0x7ff6f419c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:30.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.522+0000 7ff6f1e00640 1 -- 172.21.15.16:0/1315287558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6f4109090 msgr2=0x7ff6f419c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:30.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.522+0000 7ff6f1e00640 1 --2- 172.21.15.16:0/1315287558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6f4109090 0x7ff6f419c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:30.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.522+0000 7ff6f1e00640 1 -- 172.21.15.16:0/1315287558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6f41099e0 msgr2=0x7ff6f41a1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:30.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.522+0000 7ff6f1e00640 1 --2- 172.21.15.16:0/1315287558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6f41099e0 0x7ff6f41a1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:30.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.522+0000 7ff6f1e00640 1 -- 172.21.15.16:0/1315287558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6f41a2260 con 0x7ff6f4103750 2024-08-31T06:08:30.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.523+0000 7ff6f1e00640 1 --2- 172.21.15.16:0/1315287558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6f4103750 0x7ff6f419c2f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4005ab0 tx=0x7ff6e4003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:30.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.523+0000 7ff6eaa00640 1 -- 172.21.15.16:0/1315287558 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e403f070 con 0x7ff6f4103750 2024-08-31T06:08:30.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.523+0000 7ff6f3200640 1 -- 172.21.15.16:0/1315287558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6f41a24f0 con 0x7ff6f4103750 2024-08-31T06:08:30.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.523+0000 7ff6eaa00640 1 -- 172.21.15.16:0/1315287558 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6e4031d60 con 0x7ff6f4103750 2024-08-31T06:08:30.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.523+0000 7ff6eaa00640 1 -- 172.21.15.16:0/1315287558 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e4035030 con 0x7ff6f4103750 2024-08-31T06:08:30.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.523+0000 7ff6f3200640 1 -- 172.21.15.16:0/1315287558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6f41a8cb0 con 0x7ff6f4103750 2024-08-31T06:08:30.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.527+0000 7ff6ff680740 1 Processor -- start 2024-08-31T06:08:30.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.527+0000 7ff6ff680740 1 -- start start 2024-08-31T06:08:30.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.527+0000 7ff6ff680740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f54cee6520 0x55f54d28c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:30.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.527+0000 7ff6ff680740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f54cebd660 0x55f54d28c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:30.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.527+0000 7ff6eaa00640 1 -- 172.21.15.16:0/1315287558 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7ff6e4042070 con 0x7ff6f4103750 2024-08-31T06:08:30.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.527+0000 7ff6ff680740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54cec5840 0x55f54d28cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:30.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.527+0000 7ff6ff680740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f54d202a90 con 0x55f54cec5840 2024-08-31T06:08:30.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.527+0000 7ff6ff680740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f54d1f2020 con 0x55f54cebd660 2024-08-31T06:08:30.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.527+0000 7ff6ff680740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f54d1f5fd0 con 0x55f54cee6520 2024-08-31T06:08:30.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.528+0000 7ff6f2800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f54cee6520 0x55f54d28c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:30.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.528+0000 7ff6f1e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f54cebd660 0x55f54d28c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:30.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.528+0000 7ff6f2800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f54cee6520 0x55f54d28c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:34932/0 (socket says 172.21.15.16:34932) 2024-08-31T06:08:30.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.528+0000 7ff6f2800640 1 -- 172.21.15.16:0/2234779408 learned_addr learned my addr 172.21.15.16:0/2234779408 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:30.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.528+0000 7ff6f1400640 1 --2- 172.21.15.16:0/2234779408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54cec5840 0x55f54d28cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:30.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.528+0000 7ff6f2800640 1 -- 172.21.15.16:0/2234779408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f54cebd660 msgr2=0x55f54d28c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:30.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.528+0000 7ff6f2800640 1 --2- 172.21.15.16:0/2234779408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f54cebd660 0x55f54d28c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:30.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.528+0000 7ff6f2800640 1 -- 172.21.15.16:0/2234779408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54cec5840 msgr2=0x55f54d28cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:30.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.528+0000 7ff6f2800640 1 --2- 172.21.15.16:0/2234779408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54cec5840 0x55f54d28cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:30.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.528+0000 7ff6f2800640 1 -- 172.21.15.16:0/2234779408 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f54cf51b00 con 0x55f54cee6520 2024-08-31T06:08:30.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.529+0000 7ff6f2800640 1 --2- 172.21.15.16:0/2234779408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f54cee6520 0x55f54d28c090 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec010370 tx=0x7ff6ec010830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6732aed7c42082e4 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:30.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.529+0000 7ff6df000640 1 -- 172.21.15.16:0/2234779408 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec016070 con 0x55f54cee6520 2024-08-31T06:08:30.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.529+0000 7ff6df000640 1 -- 172.21.15.16:0/2234779408 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6ec004d20 con 0x55f54cee6520 2024-08-31T06:08:30.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.529+0000 7ff6df000640 1 -- 172.21.15.16:0/2234779408 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec005350 con 0x55f54cee6520 2024-08-31T06:08:30.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.529+0000 7ff6eaa00640 1 --2- 172.21.15.16:0/1315287558 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff6d80b4ca0 0x7ff6d80b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:30.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.529+0000 7ff6eaa00640 1 -- 172.21.15.16:0/1315287558 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7ff6e4105d20 con 0x7ff6f4103750 2024-08-31T06:08:30.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.530+0000 7ff6f1e00640 1 --2- 172.21.15.16:0/1315287558 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff6d80b4ca0 0x7ff6d80b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:30.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.530+0000 7ff6ff680740 1 -- 172.21.15.16:0/2234779408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f54cee6520 msgr2=0x55f54d28c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:30.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.530+0000 7ff6ff680740 1 --2- 172.21.15.16:0/2234779408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f54cee6520 0x55f54d28c090 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec010370 tx=0x7ff6ec010830 comp rx=0 tx=0).stop 2024-08-31T06:08:30.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.530+0000 7ff6ff680740 1 -- 172.21.15.16:0/2234779408 shutdown_connections 2024-08-31T06:08:30.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.530+0000 7ff6ff680740 1 --2- 172.21.15.16:0/2234779408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54cec5840 0x55f54d28cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:30.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.530+0000 7ff6ff680740 1 --2- 172.21.15.16:0/2234779408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f54cebd660 0x55f54d28c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:30.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.530+0000 7ff6ff680740 1 --2- 172.21.15.16:0/2234779408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f54cee6520 0x55f54d28c090 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:30.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.530+0000 7ff6ff680740 1 -- 172.21.15.16:0/2234779408 >> 172.21.15.16:0/2234779408 conn(0x55f54ceda910 msgr2=0x55f54d289f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:30.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.530+0000 7ff6f1e00640 1 --2- 172.21.15.16:0/1315287558 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff6d80b4ca0 0x7ff6d80b71b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55f54d28ad10 tx=0x7ff6e4031f10 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:30.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.531+0000 7ff6ff680740 1 -- 172.21.15.16:0/2234779408 shutdown_connections 2024-08-31T06:08:30.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.531+0000 7ff6ff680740 1 -- 172.21.15.16:0/2234779408 wait complete. 2024-08-31T06:08:30.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.531+0000 7ff6ff680740 1 -- start start 2024-08-31T06:08:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.531+0000 7ff6ff680740 1 Processor -- start 2024-08-31T06:08:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.532+0000 7ff6ff680740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54cebd660 0x55f54d2973e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.532+0000 7ff6ff680740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f54cec5840 0x55f54d297920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.532+0000 7ff6ff680740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f54cee6520 0x55f54d297e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.532+0000 7ff6ff680740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f54d054bb0 con 0x55f54cebd660 2024-08-31T06:08:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.532+0000 7ff6ff680740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f54d178f90 con 0x55f54cec5840 2024-08-31T06:08:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.532+0000 7ff6ff680740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f54d29a870 con 0x55f54cee6520 2024-08-31T06:08:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.532+0000 7ff6ff680740 10 client.0 _ll_register_callbacks cb 0x55f54d19e710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:08:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.532+0000 7ff6f1400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f54cec5840 0x55f54d297920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.532+0000 7ff6f1400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f54cec5840 0x55f54d297920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:33588/0 (socket says 172.21.15.16:33588) 2024-08-31T06:08:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.532+0000 7ff6f2800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54cebd660 0x55f54d2973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.532+0000 7ff6f1400640 1 -- 172.21.15.16:0/226462836 learned_addr learned my addr 172.21.15.16:0/226462836 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:30.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.533+0000 7ff6f2800640 1 --2- 172.21.15.16:0/226462836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f54cee6520 0x55f54d297e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:30.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.533+0000 7ff6f2800640 1 -- 172.21.15.16:0/226462836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f54cee6520 msgr2=0x55f54d297e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:30.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.533+0000 7ff6f2800640 1 --2- 172.21.15.16:0/226462836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f54cee6520 0x55f54d297e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:30.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.533+0000 7ff6f2800640 1 -- 172.21.15.16:0/226462836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f54cec5840 msgr2=0x55f54d297920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:30.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.533+0000 7ff6f2800640 1 --2- 172.21.15.16:0/226462836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f54cec5840 0x55f54d297920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:30.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.533+0000 7ff6f2800640 1 -- 172.21.15.16:0/226462836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f54cfc8960 con 0x55f54cebd660 2024-08-31T06:08:30.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.533+0000 7ff6f2800640 1 --2- 172.21.15.16:0/226462836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54cebd660 0x55f54d2973e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec010370 tx=0x7ff6ec012ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ef5f5cb4fe090bc server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:30.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.533+0000 7ff6d3400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:08:30.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.534+0000 7ff6d3400640 1 -- 172.21.15.16:0/226462836 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec016040 con 0x55f54cebd660 2024-08-31T06:08:30.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.534+0000 7ff6d3400640 1 -- 172.21.15.16:0/226462836 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7ff6ec003b80 con 0x55f54cebd660 2024-08-31T06:08:30.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.534+0000 7ff6ff680740 1 -- 172.21.15.16:0/226462836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f54d190e30 con 0x55f54cebd660 2024-08-31T06:08:30.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.534+0000 7ff6d3400640 1 -- 172.21.15.16:0/226462836 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff6ec012740 con 0x55f54cebd660 2024-08-31T06:08:30.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.534+0000 7ff6ff680740 10 client.15858 fetch_fsmap learned FSMap version 30 2024-08-31T06:08:30.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.534+0000 7ff6ff680740 1 -- 172.21.15.16:0/226462836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f54d2a6280 con 0x55f54cebd660 2024-08-31T06:08:30.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.534+0000 7ff6d3400640 1 -- 172.21.15.16:0/226462836 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f54d190e30 con 0x55f54cebd660 2024-08-31T06:08:30.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.535+0000 7ff6d3400640 1 -- 172.21.15.16:0/226462836 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7ff6ec028040 con 0x55f54cebd660 2024-08-31T06:08:30.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.535+0000 7ff6ff680740 10 client.15858 fetch_fsmap finished waiting for FSMap version 30 2024-08-31T06:08:30.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.535+0000 7ff6ff680740 10 client.15858 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:08:30.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.535+0000 7ff6ff680740 20 client.15858 populate_metadata read hostname 'smithi016' 2024-08-31T06:08:30.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.535+0000 7ff6ff680740 1 --2- 172.21.15.16:0/226462836 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f54cea4ea0 0x55f54d2a9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:30.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.535+0000 7ff6ff680740 4 client.15858 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:08:30.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.535+0000 7ff6ff680740 1 -- 172.21.15.16:0/226462836 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f54d2049e0 con 0x55f54cea4ea0 2024-08-31T06:08:30.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.535+0000 7ff6f1400640 1 --2- 172.21.15.16:0/226462836 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f54cea4ea0 0x55f54d2a9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:30.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.536+0000 7ff6f1400640 1 --2- 172.21.15.16:0/226462836 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f54cea4ea0 0x55f54d2a9640 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ee24875b87fc2a1 server_cookie=5bf4037cf2535e0d in_seq=0 out_seq=0 2024-08-31T06:08:30.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.536+0000 7ff6d3400640 10 client.15858 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:30.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.537+0000 7ff6d3400640 1 -- 172.21.15.16:0/226462836 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f54d2049e0 con 0x55f54cea4ea0 2024-08-31T06:08:30.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.537+0000 7ff6d3400640 10 client.15858 handle_command_reply: tid=0 2024-08-31T06:08:30.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.537+0000 7ff6ff680740 1 client.15858 shutdown 2024-08-31T06:08:30.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.537+0000 7ff6ff680740 1 -- 172.21.15.16:0/226462836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54cebd660 msgr2=0x55f54d2973e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:30.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.537+0000 7ff6ff680740 1 --2- 172.21.15.16:0/226462836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54cebd660 0x55f54d2973e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec010370 tx=0x7ff6ec012ea0 comp rx=0 tx=0).stop 2024-08-31T06:08:30.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.537+0000 7ff6ff680740 1 -- 172.21.15.16:0/226462836 shutdown_connections 2024-08-31T06:08:30.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.537+0000 7ff6ff680740 1 --2- 172.21.15.16:0/226462836 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f54cea4ea0 0x55f54d2a9640 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:30.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.537+0000 7ff6ff680740 1 --2- 172.21.15.16:0/226462836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f54cee6520 0x55f54d297e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:30.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.537+0000 7ff6ff680740 1 --2- 172.21.15.16:0/226462836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f54cec5840 0x55f54d297920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:30.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.537+0000 7ff6ff680740 1 --2- 172.21.15.16:0/226462836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54cebd660 0x55f54d2973e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:30.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.537+0000 7ff6ff680740 1 -- 172.21.15.16:0/226462836 >> 172.21.15.16:0/226462836 conn(0x55f54ceda910 msgr2=0x55f54d28f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:30.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.538+0000 7ff6d3400640 0 client.15858 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:30.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.538+0000 7ff6ff680740 1 -- 172.21.15.16:0/226462836 shutdown_connections 2024-08-31T06:08:30.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.538+0000 7ff6ff680740 1 -- 172.21.15.16:0/226462836 wait complete. 2024-08-31T06:08:30.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.539+0000 7ff6ff680740 20 client.15858 trim_cache size 0 max 16384 2024-08-31T06:08:30.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.557+0000 7ff6ff680740 1 Processor -- start 2024-08-31T06:08:30.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.557+0000 7ff6ff680740 1 -- start start 2024-08-31T06:08:30.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.558+0000 7ff6ff680740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f54cebd660 0x55f54d292bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:30.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.558+0000 7ff6ff680740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54cec5840 0x55f54d2a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:30.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.558+0000 7ff6ff680740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f54cee6520 0x55f54d28d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:30.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.558+0000 7ff6ff680740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f41b3d70 con 0x55f54cec5840 2024-08-31T06:08:30.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.558+0000 7ff6ff680740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f41b15b0 con 0x55f54cee6520 2024-08-31T06:08:30.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.558+0000 7ff6ff680740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f41af400 con 0x55f54cebd660 2024-08-31T06:08:30.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.558+0000 7ff6f1400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54cec5840 0x55f54d2a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:30.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.558+0000 7ff6f1400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54cec5840 0x55f54d2a3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59702/0 (socket says 172.21.15.16:59702) 2024-08-31T06:08:30.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.558+0000 7ff6f1400640 1 -- 172.21.15.16:0/2921722534 learned_addr learned my addr 172.21.15.16:0/2921722534 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:30.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.558+0000 7ff6f2800640 1 --2- 172.21.15.16:0/2921722534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f54cebd660 0x55f54d292bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:30.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.558+0000 7ff6f2800640 1 --2- 172.21.15.16:0/2921722534 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f54cee6520 0x55f54d28d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:30.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.559+0000 7ff6f1400640 1 -- 172.21.15.16:0/2921722534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f54cebd660 msgr2=0x55f54d292bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:30.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.559+0000 7ff6f1400640 1 --2- 172.21.15.16:0/2921722534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f54cebd660 0x55f54d292bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:30.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.559+0000 7ff6f1400640 1 -- 172.21.15.16:0/2921722534 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f54cee6520 msgr2=0x55f54d28d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:30.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.559+0000 7ff6f1400640 1 --2- 172.21.15.16:0/2921722534 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f54cee6520 0x55f54d28d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:30.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.559+0000 7ff6f1400640 1 -- 172.21.15.16:0/2921722534 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f54cea50d0 con 0x55f54cec5840 2024-08-31T06:08:30.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.559+0000 7ff6f2800640 1 --2- 172.21.15.16:0/2921722534 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f54cee6520 0x55f54d28d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:08:30.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.559+0000 7ff6f1400640 1 --2- 172.21.15.16:0/2921722534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54cec5840 0x55f54d2a3ac0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0009fd0 tx=0x7ff6e0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f22317ec7366ba5f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:30.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.559+0000 7ff6de600640 1 -- 172.21.15.16:0/2921722534 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e0004020 con 0x55f54cec5840 2024-08-31T06:08:30.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.559+0000 7ff6de600640 1 -- 172.21.15.16:0/2921722534 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6e00041c0 con 0x55f54cec5840 2024-08-31T06:08:30.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.559+0000 7ff6de600640 1 -- 172.21.15.16:0/2921722534 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e0010690 con 0x55f54cec5840 2024-08-31T06:08:30.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.560+0000 7ff6ff680740 1 -- 172.21.15.16:0/2921722534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54cec5840 msgr2=0x55f54d2a3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:30.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.560+0000 7ff6ff680740 1 --2- 172.21.15.16:0/2921722534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54cec5840 0x55f54d2a3ac0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0009fd0 tx=0x7ff6e0005f90 comp rx=0 tx=0).stop 2024-08-31T06:08:30.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.560+0000 7ff6ff680740 1 -- 172.21.15.16:0/2921722534 shutdown_connections 2024-08-31T06:08:30.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.560+0000 7ff6ff680740 1 --2- 172.21.15.16:0/2921722534 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f54cee6520 0x55f54d28d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:30.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.560+0000 7ff6ff680740 1 --2- 172.21.15.16:0/2921722534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54cec5840 0x55f54d2a3ac0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:30.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.560+0000 7ff6ff680740 1 --2- 172.21.15.16:0/2921722534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f54cebd660 0x55f54d292bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:30.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.560+0000 7ff6ff680740 1 -- 172.21.15.16:0/2921722534 >> 172.21.15.16:0/2921722534 conn(0x55f54ceda910 msgr2=0x55f54d28d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:30.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.561+0000 7ff6ff680740 1 -- 172.21.15.16:0/2921722534 shutdown_connections 2024-08-31T06:08:30.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.561+0000 7ff6ff680740 1 -- 172.21.15.16:0/2921722534 wait complete. 2024-08-31T06:08:30.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.561+0000 7ff6ff680740 1 -- start start 2024-08-31T06:08:30.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.561+0000 7ff6ff680740 1 Processor -- start 2024-08-31T06:08:30.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.562+0000 7ff6ff680740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f54cebd660 0x55f54d29be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:30.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.562+0000 7ff6ff680740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54cec5840 0x55f54d29c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:30.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.562+0000 7ff6ff680740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f54cee6520 0x55f54d312f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:30.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.562+0000 7ff6ff680740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f41afa70 con 0x55f54cec5840 2024-08-31T06:08:30.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.562+0000 7ff6ff680740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f41b0ee0 con 0x55f54cee6520 2024-08-31T06:08:30.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.562+0000 7ff6ff680740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f41b0140 con 0x55f54cebd660 2024-08-31T06:08:30.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.562+0000 7ff6ff680740 10 client.0 _ll_register_callbacks cb 0x7ff6ec004e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:08:30.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.562+0000 7ff6f1400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54cec5840 0x55f54d29c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:30.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.562+0000 7ff6f2800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f54cebd660 0x55f54d29be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:30.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.562+0000 7ff6f1400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54cec5840 0x55f54d29c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59708/0 (socket says 172.21.15.16:59708) 2024-08-31T06:08:30.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.562+0000 7ff6f1400640 1 -- 172.21.15.16:0/4150128326 learned_addr learned my addr 172.21.15.16:0/4150128326 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:30.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.562+0000 7ff6f2800640 1 --2- 172.21.15.16:0/4150128326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f54cee6520 0x55f54d312f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:30.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.563+0000 7ff6f1400640 1 -- 172.21.15.16:0/4150128326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f54cebd660 msgr2=0x55f54d29be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:30.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.563+0000 7ff6f1400640 1 --2- 172.21.15.16:0/4150128326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f54cebd660 0x55f54d29be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:30.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.563+0000 7ff6f1400640 1 -- 172.21.15.16:0/4150128326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f54cee6520 msgr2=0x55f54d312f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:30.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.563+0000 7ff6f1400640 1 --2- 172.21.15.16:0/4150128326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f54cee6520 0x55f54d312f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:30.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.563+0000 7ff6f1400640 1 -- 172.21.15.16:0/4150128326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f54d29cbf0 con 0x55f54cec5840 2024-08-31T06:08:30.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.563+0000 7ff6f2800640 1 --2- 172.21.15.16:0/4150128326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f54cebd660 0x55f54d29be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:08:30.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.563+0000 7ff6f1400640 1 --2- 172.21.15.16:0/4150128326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54cec5840 0x55f54d29c350 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff6e00040a0 tx=0x7ff6e00040d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca7f6dca3e077a97 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:30.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.563+0000 7ff6d3400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:08:30.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.563+0000 7ff6d3400640 1 -- 172.21.15.16:0/4150128326 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e00035f0 con 0x55f54cec5840 2024-08-31T06:08:30.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.564+0000 7ff6d3400640 1 -- 172.21.15.16:0/4150128326 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7ff6e0003790 con 0x55f54cec5840 2024-08-31T06:08:30.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.564+0000 7ff6ff680740 1 -- 172.21.15.16:0/4150128326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6e00041c0 con 0x55f54cec5840 2024-08-31T06:08:30.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.564+0000 7ff6d3400640 1 -- 172.21.15.16:0/4150128326 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff6e003b050 con 0x55f54cec5840 2024-08-31T06:08:30.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.564+0000 7ff6ff680740 10 client.15864 fetch_fsmap learned FSMap version 30 2024-08-31T06:08:30.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.564+0000 7ff6ff680740 1 -- 172.21.15.16:0/4150128326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f54d314510 con 0x55f54cec5840 2024-08-31T06:08:30.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.564+0000 7ff6d3400640 1 -- 172.21.15.16:0/4150128326 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6e00041c0 con 0x55f54cec5840 2024-08-31T06:08:30.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.564+0000 7ff6d3400640 1 -- 172.21.15.16:0/4150128326 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7ff6e00046c0 con 0x55f54cec5840 2024-08-31T06:08:30.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.565+0000 7ff6ff680740 10 client.15864 fetch_fsmap finished waiting for FSMap version 30 2024-08-31T06:08:30.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.565+0000 7ff6ff680740 10 client.15864 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:08:30.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.565+0000 7ff6ff680740 20 client.15864 populate_metadata read hostname 'smithi016' 2024-08-31T06:08:30.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.565+0000 7ff6ff680740 1 --2- 172.21.15.16:0/4150128326 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f54ce7a250 0x55f54d316c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:30.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.565+0000 7ff6ff680740 4 client.15864 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a4339836", "release": true, "await": true}] 2024-08-31T06:08:30.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.565+0000 7ff6ff680740 1 -- 172.21.15.16:0/4150128326 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a4339836", "release": true, "await": true}) -- 0x55f54d200480 con 0x55f54ce7a250 2024-08-31T06:08:30.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.565+0000 7ff6f2800640 1 --2- 172.21.15.16:0/4150128326 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f54ce7a250 0x55f54d316c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:30.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.566+0000 7ff6f2800640 1 --2- 172.21.15.16:0/4150128326 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f54ce7a250 0x55f54d316c70 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a42b4c521c94bb4a server_cookie=974404f38834f2f in_seq=0 out_seq=0 2024-08-31T06:08:30.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.566+0000 7ff6d3400640 10 client.15864 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:30.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:30 smithi016 ceph-mon[17194]: pgmap v402: 97 pgs: 97 active+clean; 964 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 2.0 MiB/s wr, 324 op/s 2024-08-31T06:08:30.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:30 smithi184 ceph-mon[22331]: pgmap v402: 97 pgs: 97 active+clean; 964 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 2.0 MiB/s wr, 324 op/s 2024-08-31T06:08:30.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.804+0000 7ff6d3400640 1 -- 172.21.15.16:0/4150128326 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55f54d200480 con 0x55f54ce7a250 2024-08-31T06:08:30.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.804+0000 7ff6d3400640 10 client.15864 handle_command_reply: tid=0 2024-08-31T06:08:30.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.804+0000 7ff6ff680740 1 client.15864 shutdown 2024-08-31T06:08:30.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.805+0000 7ff6ff680740 1 -- 172.21.15.16:0/4150128326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54cec5840 msgr2=0x55f54d29c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:30.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.805+0000 7ff6ff680740 1 --2- 172.21.15.16:0/4150128326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54cec5840 0x55f54d29c350 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff6e00040a0 tx=0x7ff6e00040d0 comp rx=0 tx=0).stop 2024-08-31T06:08:30.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.806+0000 7ff6ff680740 1 -- 172.21.15.16:0/4150128326 shutdown_connections 2024-08-31T06:08:30.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.806+0000 7ff6ff680740 1 --2- 172.21.15.16:0/4150128326 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f54ce7a250 0x55f54d316c70 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:30.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.806+0000 7ff6ff680740 1 --2- 172.21.15.16:0/4150128326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f54cee6520 0x55f54d312f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:30.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.806+0000 7ff6ff680740 1 --2- 172.21.15.16:0/4150128326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54cec5840 0x55f54d29c350 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:30.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.806+0000 7ff6ff680740 1 --2- 172.21.15.16:0/4150128326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f54cebd660 0x55f54d29be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:30.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.806+0000 7ff6ff680740 1 -- 172.21.15.16:0/4150128326 >> 172.21.15.16:0/4150128326 conn(0x55f54ceda910 msgr2=0x55f54d28cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:30.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.806+0000 7ff6d3400640 0 client.15864 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:30.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.807+0000 7ff6ff680740 1 -- 172.21.15.16:0/4150128326 shutdown_connections 2024-08-31T06:08:30.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.807+0000 7ff6ff680740 1 -- 172.21.15.16:0/4150128326 wait complete. 2024-08-31T06:08:30.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.807+0000 7ff6ff680740 20 client.15864 trim_cache size 0 max 16384 2024-08-31T06:08:30.812 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:08:30.812 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 30, 2024-08-31T06:08:30.812 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24737, 2024-08-31T06:08:30.813 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 8, 2024-08-31T06:08:30.813 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-08-31T06:08:30.813 INFO:teuthology.orchestra.run.smithi016.stdout: "a4339836": { 2024-08-31T06:08:30.813 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 8, 2024-08-31T06:08:30.813 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-08-31T06:08:30.813 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:08:30.813 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-08-31T06:08:30.813 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:08:30.813 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:08:30.813 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-08-31T06:08:30.813 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 146.4, 2024-08-31T06:08:30.814 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-08-31T06:08:30.814 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-08-31T06:08:30.814 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-08-31T06:08:30.814 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:08:30.814 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-08-31T06:08:30.814 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:08:30.814 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:08:30.814 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:08:30.814 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:08:30.814 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:08:30.814 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:08:30.814 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:08:30.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.814+0000 7ff6f3200640 1 -- 172.21.15.16:0/1315287558 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff6d80b4ca0 msgr2=0x7ff6d80b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:30.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.814+0000 7ff6f3200640 1 --2- 172.21.15.16:0/1315287558 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff6d80b4ca0 0x7ff6d80b71b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55f54d28ad10 tx=0x7ff6e4031f10 comp rx=0 tx=0).stop 2024-08-31T06:08:30.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.814+0000 7ff6f3200640 1 -- 172.21.15.16:0/1315287558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6f4103750 msgr2=0x7ff6f419c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:30.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.814+0000 7ff6f3200640 1 --2- 172.21.15.16:0/1315287558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6f4103750 0x7ff6f419c2f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4005ab0 tx=0x7ff6e4003c80 comp rx=0 tx=0).stop 2024-08-31T06:08:30.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.815+0000 7ff6f3200640 1 -- 172.21.15.16:0/1315287558 shutdown_connections 2024-08-31T06:08:30.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.815+0000 7ff6f3200640 1 --2- 172.21.15.16:0/1315287558 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff6d80b4ca0 0x7ff6d80b71b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:30.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.815+0000 7ff6f3200640 1 --2- 172.21.15.16:0/1315287558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6f4109090 0x7ff6f419c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:30.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.815+0000 7ff6f3200640 1 --2- 172.21.15.16:0/1315287558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6f41099e0 0x7ff6f41a1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:30.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.815+0000 7ff6f3200640 1 --2- 172.21.15.16:0/1315287558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6f4103750 0x7ff6f419c2f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:30.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.815+0000 7ff6f3200640 1 -- 172.21.15.16:0/1315287558 >> 172.21.15.16:0/1315287558 conn(0x7ff6f40ff1e0 msgr2=0x7ff6f4108790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:30.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.816+0000 7ff6f3200640 1 -- 172.21.15.16:0/1315287558 shutdown_connections 2024-08-31T06:08:30.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:30.816+0000 7ff6f3200640 1 -- 172.21.15.16:0/1315287558 wait complete. 2024-08-31T06:08:30.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:30 smithi155 ceph-mon[21742]: pgmap v402: 97 pgs: 97 active+clean; 964 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 2.0 MiB/s wr, 324 op/s 2024-08-31T06:08:31.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 65.0 s, 264.2 tps, lat 1963.356 ms stddev 6666.115 2024-08-31T06:08:31.993 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a4339836', total seconds elapsed: 31.0 2024-08-31T06:08:32.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:32 smithi016 ceph-mon[17194]: pgmap v403: 97 pgs: 97 active+clean; 964 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 2.0 MiB/s wr, 325 op/s 2024-08-31T06:08:32.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:32 smithi184 ceph-mon[22331]: pgmap v403: 97 pgs: 97 active+clean; 964 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 2.0 MiB/s wr, 325 op/s 2024-08-31T06:08:32.871 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:32 smithi155 ceph-mon[21742]: pgmap v403: 97 pgs: 97 active+clean; 964 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 2.0 MiB/s wr, 325 op/s 2024-08-31T06:08:34.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:34 smithi184 ceph-mon[22331]: pgmap v404: 97 pgs: 97 active+clean; 972 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 2.1 MiB/s wr, 294 op/s 2024-08-31T06:08:34.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:34 smithi155 ceph-mon[21742]: pgmap v404: 97 pgs: 97 active+clean; 972 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 2.1 MiB/s wr, 294 op/s 2024-08-31T06:08:34.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:34 smithi016 ceph-mon[17194]: pgmap v404: 97 pgs: 97 active+clean; 972 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 2.1 MiB/s wr, 294 op/s 2024-08-31T06:08:36.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 70.0 s, 3064.4 tps, lat 32.808 ms stddev 40.394 2024-08-31T06:08:36.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:36 smithi184 ceph-mon[22331]: pgmap v405: 97 pgs: 97 active+clean; 995 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 4.0 MiB/s wr, 359 op/s 2024-08-31T06:08:36.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:36 smithi155 ceph-mon[21742]: pgmap v405: 97 pgs: 97 active+clean; 995 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 4.0 MiB/s wr, 359 op/s 2024-08-31T06:08:36.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:36 smithi016 ceph-mon[17194]: pgmap v405: 97 pgs: 97 active+clean; 995 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 4.0 MiB/s wr, 359 op/s 2024-08-31T06:08:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.802+0000 7fa95be00640 1 -- 172.21.15.16:0/2052046389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa95c1058b0 msgr2=0x7fa95c105cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.802+0000 7fa95be00640 1 --2- 172.21.15.16:0/2052046389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa95c1058b0 0x7fa95c105cc0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa94c009900 tx=0x7fa94c02a3f0 comp rx=0 tx=0).stop 2024-08-31T06:08:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.802+0000 7fa95be00640 1 -- 172.21.15.16:0/2052046389 shutdown_connections 2024-08-31T06:08:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.802+0000 7fa95be00640 1 --2- 172.21.15.16:0/2052046389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa95c10bb30 0x7fa95c112360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.802+0000 7fa95be00640 1 --2- 172.21.15.16:0/2052046389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa95c10b1e0 0x7fa95c10b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.802+0000 7fa95be00640 1 --2- 172.21.15.16:0/2052046389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa95c1058b0 0x7fa95c105cc0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:37.806 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.802+0000 7fa95be00640 1 -- 172.21.15.16:0/2052046389 >> 172.21.15.16:0/2052046389 conn(0x7fa95c101290 msgr2=0x7fa95c1036e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:37.806 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.802+0000 7fa95be00640 1 -- 172.21.15.16:0/2052046389 shutdown_connections 2024-08-31T06:08:37.806 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.803+0000 7fa95be00640 1 -- 172.21.15.16:0/2052046389 wait complete. 2024-08-31T06:08:37.806 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.804+0000 7fa95be00640 1 Processor -- start 2024-08-31T06:08:37.806 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.804+0000 7fa95be00640 1 -- start start 2024-08-31T06:08:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.805+0000 7fa95be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa95c1058b0 0x7fa95c110e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.805+0000 7fa95be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa95c10b1e0 0x7fa95c074be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.806+0000 7fa95be00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa95c10bb30 0x7fa95c111390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.806+0000 7fa95aa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa95c1058b0 0x7fa95c110e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.806+0000 7fa95aa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa95c1058b0 0x7fa95c110e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:36654/0 (socket says 172.21.15.16:36654) 2024-08-31T06:08:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.806+0000 7fa95aa00640 1 -- 172.21.15.16:0/732345563 learned_addr learned my addr 172.21.15.16:0/732345563 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.806+0000 7fa95be00640 1 -- 172.21.15.16:0/732345563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa95c078250 con 0x7fa95c10b1e0 2024-08-31T06:08:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.806+0000 7fa95be00640 1 -- 172.21.15.16:0/732345563 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa95c0780d0 con 0x7fa95c1058b0 2024-08-31T06:08:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.806+0000 7fa95be00640 1 -- 172.21.15.16:0/732345563 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa95c0783d0 con 0x7fa95c10bb30 2024-08-31T06:08:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.806+0000 7fa95a000640 1 --2- 172.21.15.16:0/732345563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa95c10b1e0 0x7fa95c074be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.806+0000 7fa95aa00640 1 -- 172.21.15.16:0/732345563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa95c10bb30 msgr2=0x7fa95c111390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.806+0000 7fa95b400640 1 --2- 172.21.15.16:0/732345563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa95c10bb30 0x7fa95c111390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.807+0000 7fa95aa00640 1 --2- 172.21.15.16:0/732345563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa95c10bb30 0x7fa95c111390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.807+0000 7fa95aa00640 1 -- 172.21.15.16:0/732345563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa95c10b1e0 msgr2=0x7fa95c074be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.807+0000 7fa95aa00640 1 --2- 172.21.15.16:0/732345563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa95c10b1e0 0x7fa95c074be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.807+0000 7fa95aa00640 1 -- 172.21.15.16:0/732345563 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa95c1b50b0 con 0x7fa95c1058b0 2024-08-31T06:08:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.807+0000 7fa95a000640 1 --2- 172.21.15.16:0/732345563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa95c10b1e0 0x7fa95c074be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:08:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.807+0000 7fa95aa00640 1 --2- 172.21.15.16:0/732345563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa95c1058b0 0x7fa95c110e50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa94c005ab0 tx=0x7fa94c003c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.807+0000 7fa960a00640 1 -- 172.21.15.16:0/732345563 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c03f070 con 0x7fa95c1058b0 2024-08-31T06:08:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.807+0000 7fa960a00640 1 -- 172.21.15.16:0/732345563 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa94c031b50 con 0x7fa95c1058b0 2024-08-31T06:08:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.807+0000 7fa960a00640 1 -- 172.21.15.16:0/732345563 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c035030 con 0x7fa95c1058b0 2024-08-31T06:08:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.807+0000 7fa95be00640 1 -- 172.21.15.16:0/732345563 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa95c1b5280 con 0x7fa95c1058b0 2024-08-31T06:08:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.808+0000 7fa95be00640 1 -- 172.21.15.16:0/732345563 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa95c1b5630 con 0x7fa95c1058b0 2024-08-31T06:08:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.812+0000 7fa968e0b740 1 Processor -- start 2024-08-31T06:08:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.812+0000 7fa968e0b740 1 -- start start 2024-08-31T06:08:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.813+0000 7fa968e0b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dece921b0 0x556ded2358a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.813+0000 7fa968e0b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556dece692f0 0x556ded235de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.813+0000 7fa968e0b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dece714d0 0x556ded236320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.813+0000 7fa968e0b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556ded1aaf70 con 0x556dece921b0 2024-08-31T06:08:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.813+0000 7fa968e0b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556ded19a500 con 0x556dece692f0 2024-08-31T06:08:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.813+0000 7fa968e0b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556ded19e4b0 con 0x556dece714d0 2024-08-31T06:08:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.813+0000 7fa960a00640 1 -- 172.21.15.16:0/732345563 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fa94c042070 con 0x7fa95c1058b0 2024-08-31T06:08:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.813+0000 7fa95aa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556dece692f0 0x556ded235de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.813+0000 7fa95aa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556dece692f0 0x556ded235de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:36668/0 (socket says 172.21.15.16:36668) 2024-08-31T06:08:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.813+0000 7fa95aa00640 1 -- 172.21.15.16:0/57954346 learned_addr learned my addr 172.21.15.16:0/57954346 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.814+0000 7fa95aa00640 1 -- 172.21.15.16:0/57954346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dece714d0 msgr2=0x556ded236320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:08:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.814+0000 7fa95aa00640 1 --2- 172.21.15.16:0/57954346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dece714d0 0x556ded236320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.814+0000 7fa95aa00640 1 -- 172.21.15.16:0/57954346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dece921b0 msgr2=0x556ded2358a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:08:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.814+0000 7fa95aa00640 1 --2- 172.21.15.16:0/57954346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dece921b0 0x556ded2358a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.814+0000 7fa95aa00640 1 -- 172.21.15.16:0/57954346 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ded154c10 con 0x556dece692f0 2024-08-31T06:08:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.814+0000 7fa95aa00640 1 --2- 172.21.15.16:0/57954346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556dece692f0 0x556ded235de0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa94c10bb40 tx=0x7fa94c10bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97cadc8f2f83973a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.815+0000 7fa947e00640 1 -- 172.21.15.16:0/57954346 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c114070 con 0x556dece692f0 2024-08-31T06:08:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.815+0000 7fa960a00640 1 --2- 172.21.15.16:0/732345563 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa9400b4dd0 0x7fa9400b72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.816+0000 7fa947e00640 1 -- 172.21.15.16:0/57954346 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa94c10dc90 con 0x556dece692f0 2024-08-31T06:08:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.816+0000 7fa947e00640 1 -- 172.21.15.16:0/57954346 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c10f5e0 con 0x556dece692f0 2024-08-31T06:08:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.817+0000 7fa968e0b740 1 -- 172.21.15.16:0/57954346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556dece692f0 msgr2=0x556ded235de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.817+0000 7fa968e0b740 1 --2- 172.21.15.16:0/57954346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556dece692f0 0x556ded235de0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa94c10bb40 tx=0x7fa94c10bc30 comp rx=0 tx=0).stop 2024-08-31T06:08:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.817+0000 7fa960a00640 1 -- 172.21.15.16:0/732345563 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fa94c1059f0 con 0x7fa95c1058b0 2024-08-31T06:08:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.818+0000 7fa95b400640 1 --2- 172.21.15.16:0/732345563 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa9400b4dd0 0x7fa9400b72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.819+0000 7fa95b400640 1 --2- 172.21.15.16:0/732345563 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa9400b4dd0 0x7fa9400b72e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa954005fd0 tx=0x7fa9540093c0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.819+0000 7fa968e0b740 1 -- 172.21.15.16:0/57954346 shutdown_connections 2024-08-31T06:08:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.819+0000 7fa968e0b740 1 --2- 172.21.15.16:0/57954346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dece714d0 0x556ded236320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.819+0000 7fa968e0b740 1 --2- 172.21.15.16:0/57954346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556dece692f0 0x556ded235de0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.819+0000 7fa968e0b740 1 --2- 172.21.15.16:0/57954346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dece921b0 0x556ded2358a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.819+0000 7fa968e0b740 1 -- 172.21.15.16:0/57954346 >> 172.21.15.16:0/57954346 conn(0x556dece86300 msgr2=0x556ded233790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.819+0000 7fa968e0b740 1 -- 172.21.15.16:0/57954346 shutdown_connections 2024-08-31T06:08:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.819+0000 7fa968e0b740 1 -- 172.21.15.16:0/57954346 wait complete. 2024-08-31T06:08:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.820+0000 7fa968e0b740 1 -- start start 2024-08-31T06:08:37.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.821+0000 7fa968e0b740 1 Processor -- start 2024-08-31T06:08:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.822+0000 7fa968e0b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dece692f0 0x556ded240ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.822+0000 7fa968e0b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dece714d0 0x556ded241420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.822+0000 7fa968e0b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556dece921b0 0x556ded24d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.823+0000 7fa968e0b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556ded0d3ab0 con 0x556dece692f0 2024-08-31T06:08:37.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.823+0000 7fa95a000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dece714d0 0x556ded241420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:37.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.823+0000 7fa95b400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556dece921b0 0x556ded24d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:37.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.823+0000 7fa95a000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dece714d0 0x556ded241420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:39712/0 (socket says 172.21.15.16:39712) 2024-08-31T06:08:37.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.823+0000 7fa95a000640 1 -- 172.21.15.16:0/3277762518 learned_addr learned my addr 172.21.15.16:0/3277762518 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:37.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.823+0000 7fa95aa00640 1 --2- 172.21.15.16:0/3277762518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dece692f0 0x556ded240ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:37.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.823+0000 7fa968e0b740 1 -- 172.21.15.16:0/3277762518 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556ded141320 con 0x556dece921b0 2024-08-31T06:08:37.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.823+0000 7fa968e0b740 1 -- 172.21.15.16:0/3277762518 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556ded243de0 con 0x556dece714d0 2024-08-31T06:08:37.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.824+0000 7fa968e0b740 10 client.0 _ll_register_callbacks cb 0x556ded17ae80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:08:37.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.824+0000 7fa95aa00640 1 -- 172.21.15.16:0/3277762518 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dece714d0 msgr2=0x556ded241420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:37.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.824+0000 7fa95aa00640 1 --2- 172.21.15.16:0/3277762518 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dece714d0 0x556ded241420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:37.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.824+0000 7fa95aa00640 1 -- 172.21.15.16:0/3277762518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556dece921b0 msgr2=0x556ded24d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:37.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.824+0000 7fa95aa00640 1 --2- 172.21.15.16:0/3277762518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556dece921b0 0x556ded24d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:37.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.824+0000 7fa95aa00640 1 -- 172.21.15.16:0/3277762518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556decefd790 con 0x556dece692f0 2024-08-31T06:08:37.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.825+0000 7fa95b400640 1 --2- 172.21.15.16:0/3277762518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556dece921b0 0x556ded24d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:08:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.826+0000 7fa95aa00640 1 --2- 172.21.15.16:0/3277762518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dece692f0 0x556ded240ee0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa94c031440 tx=0x7fa94c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7000205e70d773b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.826+0000 7fa944c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:08:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.826+0000 7fa944c00640 1 -- 172.21.15.16:0/3277762518 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c049070 con 0x556dece692f0 2024-08-31T06:08:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.826+0000 7fa944c00640 1 -- 172.21.15.16:0/3277762518 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fa94c10de40 con 0x556dece692f0 2024-08-31T06:08:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.826+0000 7fa968e0b740 1 -- 172.21.15.16:0/3277762518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa94c10dc90 con 0x556dece692f0 2024-08-31T06:08:37.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.827+0000 7fa944c00640 1 -- 172.21.15.16:0/3277762518 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa94c114030 con 0x556dece692f0 2024-08-31T06:08:37.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.827+0000 7fa968e0b740 10 client.15873 fetch_fsmap learned FSMap version 30 2024-08-31T06:08:37.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.827+0000 7fa968e0b740 1 -- 172.21.15.16:0/3277762518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556decf76620 con 0x556dece692f0 2024-08-31T06:08:37.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.827+0000 7fa944c00640 1 -- 172.21.15.16:0/3277762518 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa94c10dc90 con 0x556dece692f0 2024-08-31T06:08:37.830 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.827+0000 7fa944c00640 1 -- 172.21.15.16:0/3277762518 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fa94c04c060 con 0x556dece692f0 2024-08-31T06:08:37.830 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.827+0000 7fa968e0b740 10 client.15873 fetch_fsmap finished waiting for FSMap version 30 2024-08-31T06:08:37.830 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.827+0000 7fa968e0b740 10 client.15873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:08:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.828+0000 7fa968e0b740 20 client.15873 populate_metadata read hostname 'smithi016' 2024-08-31T06:08:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.828+0000 7fa968e0b740 1 --2- 172.21.15.16:0/3277762518 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556dece50d80 0x556ded252b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.828+0000 7fa968e0b740 4 client.15873 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:08:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.828+0000 7fa968e0b740 1 -- 172.21.15.16:0/3277762518 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ded1acec0 con 0x556dece50d80 2024-08-31T06:08:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.828+0000 7fa95a000640 1 --2- 172.21.15.16:0/3277762518 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556dece50d80 0x556ded252b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.829+0000 7fa95a000640 1 --2- 172.21.15.16:0/3277762518 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556dece50d80 0x556ded252b70 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19bd09182c4c86a3 server_cookie=54c1b82ca13f5fe6 in_seq=0 out_seq=0 2024-08-31T06:08:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.829+0000 7fa944c00640 10 client.15873 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.830+0000 7fa944c00640 1 -- 172.21.15.16:0/3277762518 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556ded1acec0 con 0x556dece50d80 2024-08-31T06:08:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.830+0000 7fa944c00640 10 client.15873 handle_command_reply: tid=0 2024-08-31T06:08:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.830+0000 7fa968e0b740 1 client.15873 shutdown 2024-08-31T06:08:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.831+0000 7fa968e0b740 1 -- 172.21.15.16:0/3277762518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dece692f0 msgr2=0x556ded240ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.831+0000 7fa968e0b740 1 --2- 172.21.15.16:0/3277762518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dece692f0 0x556ded240ee0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa94c031440 tx=0x7fa94c000bc0 comp rx=0 tx=0).stop 2024-08-31T06:08:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.832+0000 7fa968e0b740 1 -- 172.21.15.16:0/3277762518 shutdown_connections 2024-08-31T06:08:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.832+0000 7fa968e0b740 1 --2- 172.21.15.16:0/3277762518 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556dece50d80 0x556ded252b70 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.832+0000 7fa968e0b740 1 --2- 172.21.15.16:0/3277762518 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dece714d0 0x556ded241420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.832+0000 7fa968e0b740 1 --2- 172.21.15.16:0/3277762518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556dece921b0 0x556ded24d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.832+0000 7fa968e0b740 1 --2- 172.21.15.16:0/3277762518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dece692f0 0x556ded240ee0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.832+0000 7fa968e0b740 1 -- 172.21.15.16:0/3277762518 >> 172.21.15.16:0/3277762518 conn(0x556dece86300 msgr2=0x556ded2368d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.832+0000 7fa944c00640 0 client.15873 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.832+0000 7fa968e0b740 1 -- 172.21.15.16:0/3277762518 shutdown_connections 2024-08-31T06:08:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.832+0000 7fa968e0b740 1 -- 172.21.15.16:0/3277762518 wait complete. 2024-08-31T06:08:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.832+0000 7fa968e0b740 20 client.15873 trim_cache size 0 max 16384 2024-08-31T06:08:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.870+0000 7fa968e0b740 1 Processor -- start 2024-08-31T06:08:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.870+0000 7fa968e0b740 1 -- start start 2024-08-31T06:08:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.871+0000 7fa968e0b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556dece692f0 0x556ded232730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.871+0000 7fa968e0b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dece714d0 0x556ded236fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.871+0000 7fa968e0b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dece921b0 0x556ded23e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.871+0000 7fa968e0b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa95c1c08b0 con 0x556dece714d0 2024-08-31T06:08:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.871+0000 7fa968e0b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa95c1be0f0 con 0x556dece692f0 2024-08-31T06:08:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.871+0000 7fa968e0b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa95c1bbf40 con 0x556dece921b0 2024-08-31T06:08:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.871+0000 7fa95a000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dece714d0 0x556ded236fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.871+0000 7fa95b400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dece921b0 0x556ded23e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.871+0000 7fa95aa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556dece692f0 0x556ded232730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.871+0000 7fa95b400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dece921b0 0x556ded23e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:39714/0 (socket says 172.21.15.16:39714) 2024-08-31T06:08:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.871+0000 7fa95aa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556dece692f0 0x556ded232730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:36688/0 (socket says 172.21.15.16:36688) 2024-08-31T06:08:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.871+0000 7fa95b400640 1 -- 172.21.15.16:0/483994354 learned_addr learned my addr 172.21.15.16:0/483994354 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.871+0000 7fa95a000640 1 --2- 172.21.15.16:0/483994354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dece714d0 0x556ded236fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56452/0 (socket says 172.21.15.16:56452) 2024-08-31T06:08:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.871+0000 7fa95aa00640 1 -- 172.21.15.16:0/483994354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dece921b0 msgr2=0x556ded23e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.871+0000 7fa95aa00640 1 --2- 172.21.15.16:0/483994354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dece921b0 0x556ded23e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.872+0000 7fa95aa00640 1 -- 172.21.15.16:0/483994354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dece714d0 msgr2=0x556ded236fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.872+0000 7fa95aa00640 1 --2- 172.21.15.16:0/483994354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dece714d0 0x556ded236fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.872+0000 7fa95aa00640 1 -- 172.21.15.16:0/483994354 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ded237ba0 con 0x556dece692f0 2024-08-31T06:08:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.872+0000 7fa95aa00640 1 --2- 172.21.15.16:0/483994354 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556dece692f0 0x556ded232730 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa94c0cbde0 tx=0x7fa94c02a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be199e970f922e7b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.872+0000 7fa947400640 1 -- 172.21.15.16:0/483994354 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c0b7070 con 0x556dece692f0 2024-08-31T06:08:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.872+0000 7fa947400640 1 -- 172.21.15.16:0/483994354 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa94c10d670 con 0x556dece692f0 2024-08-31T06:08:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.872+0000 7fa947400640 1 -- 172.21.15.16:0/483994354 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c10b770 con 0x556dece692f0 2024-08-31T06:08:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.873+0000 7fa968e0b740 1 -- 172.21.15.16:0/483994354 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556dece692f0 msgr2=0x556ded232730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.873+0000 7fa968e0b740 1 --2- 172.21.15.16:0/483994354 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556dece692f0 0x556ded232730 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa94c0cbde0 tx=0x7fa94c02a3f0 comp rx=0 tx=0).stop 2024-08-31T06:08:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.874+0000 7fa968e0b740 1 -- 172.21.15.16:0/483994354 shutdown_connections 2024-08-31T06:08:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.874+0000 7fa968e0b740 1 --2- 172.21.15.16:0/483994354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dece921b0 0x556ded23e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.874+0000 7fa968e0b740 1 --2- 172.21.15.16:0/483994354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dece714d0 0x556ded236fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.874+0000 7fa968e0b740 1 --2- 172.21.15.16:0/483994354 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556dece692f0 0x556ded232730 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.874+0000 7fa968e0b740 1 -- 172.21.15.16:0/483994354 >> 172.21.15.16:0/483994354 conn(0x556dece86300 msgr2=0x556ded2368d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.874+0000 7fa968e0b740 1 -- 172.21.15.16:0/483994354 shutdown_connections 2024-08-31T06:08:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.874+0000 7fa968e0b740 1 -- 172.21.15.16:0/483994354 wait complete. 2024-08-31T06:08:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.875+0000 7fa968e0b740 1 -- start start 2024-08-31T06:08:37.878 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.875+0000 7fa968e0b740 1 Processor -- start 2024-08-31T06:08:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.876+0000 7fa968e0b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dece692f0 0x556ded246950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.876+0000 7fa968e0b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dece714d0 0x556ded246ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.876+0000 7fa968e0b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556dece921b0 0x556ded2b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.876+0000 7fa968e0b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa95c1bc5b0 con 0x556dece692f0 2024-08-31T06:08:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.876+0000 7fa968e0b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa95c1bda20 con 0x556dece921b0 2024-08-31T06:08:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.876+0000 7fa968e0b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa95c1bcc80 con 0x556dece714d0 2024-08-31T06:08:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.876+0000 7fa968e0b740 10 client.0 _ll_register_callbacks cb 0x7fa94c10b9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:08:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.877+0000 7fa95b400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556dece921b0 0x556ded2b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.877+0000 7fa95b400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556dece921b0 0x556ded2b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:36702/0 (socket says 172.21.15.16:36702) 2024-08-31T06:08:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.877+0000 7fa95b400640 1 -- 172.21.15.16:0/2708223931 learned_addr learned my addr 172.21.15.16:0/2708223931 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.877+0000 7fa95a000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dece714d0 0x556ded246ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.878+0000 7fa95aa00640 1 --2- 172.21.15.16:0/2708223931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dece692f0 0x556ded246950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.878+0000 7fa95b400640 1 -- 172.21.15.16:0/2708223931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dece714d0 msgr2=0x556ded246ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.878+0000 7fa95b400640 1 --2- 172.21.15.16:0/2708223931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dece714d0 0x556ded246ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.878+0000 7fa95b400640 1 -- 172.21.15.16:0/2708223931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dece692f0 msgr2=0x556ded246950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.879+0000 7fa95b400640 1 --2- 172.21.15.16:0/2708223931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dece692f0 0x556ded246950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.879+0000 7fa95b400640 1 -- 172.21.15.16:0/2708223931 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ded2b5b90 con 0x556dece921b0 2024-08-31T06:08:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.879+0000 7fa95aa00640 1 --2- 172.21.15.16:0/2708223931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dece692f0 0x556ded246950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:08:37.883 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.881+0000 7fa95b400640 1 --2- 172.21.15.16:0/2708223931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556dece921b0 0x556ded2b5520 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa954020810 tx=0x7fa954004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=949e4cc9f70ff461 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:37.883 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.881+0000 7fa944c00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:08:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.882+0000 7fa944c00640 1 -- 172.21.15.16:0/2708223931 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa954029440 con 0x556dece921b0 2024-08-31T06:08:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.882+0000 7fa968e0b740 1 -- 172.21.15.16:0/2708223931 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa94c10d670 con 0x556dece921b0 2024-08-31T06:08:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.883+0000 7fa944c00640 1 -- 172.21.15.16:0/2708223931 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fa954011c90 con 0x556dece921b0 2024-08-31T06:08:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.883+0000 7fa944c00640 1 -- 172.21.15.16:0/2708223931 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa954029440 con 0x556dece921b0 2024-08-31T06:08:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.883+0000 7fa968e0b740 10 client.25922 fetch_fsmap learned FSMap version 30 2024-08-31T06:08:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.883+0000 7fa968e0b740 1 -- 172.21.15.16:0/2708223931 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556ded2cdfa0 con 0x556dece921b0 2024-08-31T06:08:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.884+0000 7fa944c00640 1 -- 172.21.15.16:0/2708223931 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa94c10d670 con 0x556dece921b0 2024-08-31T06:08:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.885+0000 7fa944c00640 1 -- 172.21.15.16:0/2708223931 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fa954027070 con 0x556dece921b0 2024-08-31T06:08:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.885+0000 7fa968e0b740 10 client.25922 fetch_fsmap finished waiting for FSMap version 30 2024-08-31T06:08:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.885+0000 7fa968e0b740 10 client.25922 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:08:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.885+0000 7fa968e0b740 20 client.25922 populate_metadata read hostname 'smithi016' 2024-08-31T06:08:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.885+0000 7fa968e0b740 1 --2- 172.21.15.16:0/2708223931 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556dece261c0 0x556ded2d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.885+0000 7fa968e0b740 4 client.25922 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:08:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.885+0000 7fa968e0b740 1 -- 172.21.15.16:0/2708223931 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556ded1a8960 con 0x556dece261c0 2024-08-31T06:08:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.886+0000 7fa95aa00640 1 --2- 172.21.15.16:0/2708223931 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556dece261c0 0x556ded2d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.887+0000 7fa95aa00640 1 --2- 172.21.15.16:0/2708223931 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556dece261c0 0x556ded2d07c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8591d46e3c19c67 server_cookie=593cc6f46562024f in_seq=0 out_seq=0 2024-08-31T06:08:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.887+0000 7fa944c00640 10 client.25922 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.900+0000 7fa944c00640 1 -- 172.21.15.16:0/2708223931 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556ded1a8960 con 0x556dece261c0 2024-08-31T06:08:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.901+0000 7fa944c00640 10 client.25922 handle_command_reply: tid=0 2024-08-31T06:08:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.901+0000 7fa968e0b740 1 client.25922 shutdown 2024-08-31T06:08:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.902+0000 7fa968e0b740 1 -- 172.21.15.16:0/2708223931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556dece921b0 msgr2=0x556ded2b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.902+0000 7fa968e0b740 1 --2- 172.21.15.16:0/2708223931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556dece921b0 0x556ded2b5520 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa954020810 tx=0x7fa954004ab0 comp rx=0 tx=0).stop 2024-08-31T06:08:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.902+0000 7fa968e0b740 1 -- 172.21.15.16:0/2708223931 shutdown_connections 2024-08-31T06:08:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.902+0000 7fa968e0b740 1 --2- 172.21.15.16:0/2708223931 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556dece261c0 0x556ded2d07c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.902+0000 7fa968e0b740 1 --2- 172.21.15.16:0/2708223931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dece714d0 0x556ded246ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.902+0000 7fa968e0b740 1 --2- 172.21.15.16:0/2708223931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556dece921b0 0x556ded2b5520 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.902+0000 7fa968e0b740 1 --2- 172.21.15.16:0/2708223931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dece692f0 0x556ded246950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.902+0000 7fa944c00640 0 client.25922 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.902+0000 7fa968e0b740 1 -- 172.21.15.16:0/2708223931 >> 172.21.15.16:0/2708223931 conn(0x556dece86300 msgr2=0x556ded242d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.903+0000 7fa968e0b740 1 -- 172.21.15.16:0/2708223931 shutdown_connections 2024-08-31T06:08:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.903+0000 7fa968e0b740 1 -- 172.21.15.16:0/2708223931 wait complete. 2024-08-31T06:08:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.903+0000 7fa968e0b740 20 client.25922 trim_cache size 0 max 16384 2024-08-31T06:08:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:08:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:08:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:08:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:08:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.905+0000 7fa95be00640 1 -- 172.21.15.16:0/732345563 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa9400b4dd0 msgr2=0x7fa9400b72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.905+0000 7fa95be00640 1 --2- 172.21.15.16:0/732345563 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa9400b4dd0 0x7fa9400b72e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa954005fd0 tx=0x7fa9540093c0 comp rx=0 tx=0).stop 2024-08-31T06:08:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.905+0000 7fa95be00640 1 -- 172.21.15.16:0/732345563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa95c1058b0 msgr2=0x7fa95c110e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.905+0000 7fa95be00640 1 --2- 172.21.15.16:0/732345563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa95c1058b0 0x7fa95c110e50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa94c005ab0 tx=0x7fa94c003c20 comp rx=0 tx=0).stop 2024-08-31T06:08:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.905+0000 7fa95be00640 1 -- 172.21.15.16:0/732345563 shutdown_connections 2024-08-31T06:08:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.905+0000 7fa95be00640 1 --2- 172.21.15.16:0/732345563 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa9400b4dd0 0x7fa9400b72e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.905+0000 7fa95be00640 1 --2- 172.21.15.16:0/732345563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa95c10bb30 0x7fa95c111390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.905+0000 7fa95be00640 1 --2- 172.21.15.16:0/732345563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa95c10b1e0 0x7fa95c074be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.905+0000 7fa95be00640 1 --2- 172.21.15.16:0/732345563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa95c1058b0 0x7fa95c110e50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.905+0000 7fa95be00640 1 -- 172.21.15.16:0/732345563 >> 172.21.15.16:0/732345563 conn(0x7fa95c101290 msgr2=0x7fa95c1036b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.906+0000 7fa95be00640 1 -- 172.21.15.16:0/732345563 shutdown_connections 2024-08-31T06:08:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:37.906+0000 7fa95be00640 1 -- 172.21.15.16:0/732345563 wait complete. 2024-08-31T06:08:38.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:38 smithi184 ceph-mon[22331]: pgmap v406: 97 pgs: 97 active+clean; 1003 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 4.1 MiB/s wr, 358 op/s 2024-08-31T06:08:38.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:38 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:08:38.871 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:38 smithi155 ceph-mon[21742]: pgmap v406: 97 pgs: 97 active+clean; 1003 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 4.1 MiB/s wr, 358 op/s 2024-08-31T06:08:38.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:38 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:08:38.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:38 smithi016 ceph-mon[17194]: pgmap v406: 97 pgs: 97 active+clean; 1003 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 4.1 MiB/s wr, 358 op/s 2024-08-31T06:08:38.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:38 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:08:39.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:39 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:08:39.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:39 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:08:39.773 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:39 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:08:39.773 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:39 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:08:39.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:39 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:08:39.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:39 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:08:40.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:40 smithi184 ceph-mon[22331]: pgmap v407: 97 pgs: 97 active+clean; 1020 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 6.7 MiB/s wr, 580 op/s 2024-08-31T06:08:40.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:40 smithi155 ceph-mon[21742]: pgmap v407: 97 pgs: 97 active+clean; 1020 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 6.7 MiB/s wr, 580 op/s 2024-08-31T06:08:40.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:40 smithi016 ceph-mon[17194]: pgmap v407: 97 pgs: 97 active+clean; 1020 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 6.7 MiB/s wr, 580 op/s 2024-08-31T06:08:41.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 75.0 s, 3194.8 tps, lat 31.317 ms stddev 32.813 2024-08-31T06:08:42.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:42 smithi016 ceph-mon[17194]: pgmap v408: 97 pgs: 97 active+clean; 1.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 7.1 MiB/s wr, 591 op/s 2024-08-31T06:08:42.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:42 smithi184 ceph-mon[22331]: pgmap v408: 97 pgs: 97 active+clean; 1.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 7.1 MiB/s wr, 591 op/s 2024-08-31T06:08:42.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:42 smithi155 ceph-mon[21742]: pgmap v408: 97 pgs: 97 active+clean; 1.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 7.1 MiB/s wr, 591 op/s 2024-08-31T06:08:44.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:44 smithi184 ceph-mon[22331]: pgmap v409: 97 pgs: 97 active+clean; 1.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 8.7 MiB/s wr, 708 op/s 2024-08-31T06:08:44.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:44 smithi155 ceph-mon[21742]: pgmap v409: 97 pgs: 97 active+clean; 1.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 8.7 MiB/s wr, 708 op/s 2024-08-31T06:08:44.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:44 smithi016 ceph-mon[17194]: pgmap v409: 97 pgs: 97 active+clean; 1.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 8.7 MiB/s wr, 708 op/s 2024-08-31T06:08:46.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 80.0 s, 2849.8 tps, lat 34.947 ms stddev 41.453 2024-08-31T06:08:46.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:46 smithi184 ceph-mon[22331]: pgmap v410: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 11 MiB/s wr, 869 op/s 2024-08-31T06:08:46.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:46 smithi155 ceph-mon[21742]: pgmap v410: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 11 MiB/s wr, 869 op/s 2024-08-31T06:08:46.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:46 smithi016 ceph-mon[17194]: pgmap v410: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 11 MiB/s wr, 869 op/s 2024-08-31T06:08:48.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:48 smithi184 ceph-mon[22331]: pgmap v411: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.7 MiB/s wr, 739 op/s 2024-08-31T06:08:48.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:48 smithi184 ceph-mon[22331]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:08:48.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:48 smithi155 ceph-mon[21742]: pgmap v411: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.7 MiB/s wr, 739 op/s 2024-08-31T06:08:48.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:48 smithi155 ceph-mon[21742]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:08:48.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:48 smithi016 ceph-mon[17194]: pgmap v411: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.7 MiB/s wr, 739 op/s 2024-08-31T06:08:48.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:48 smithi016 ceph-mon[17194]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:08:50.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:50 smithi155 ceph-mon[21742]: pgmap v412: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 10 MiB/s wr, 859 op/s 2024-08-31T06:08:50.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:50 smithi016 ceph-mon[17194]: pgmap v412: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 10 MiB/s wr, 859 op/s 2024-08-31T06:08:50.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:50 smithi184 ceph-mon[22331]: pgmap v412: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 10 MiB/s wr, 859 op/s 2024-08-31T06:08:51.194 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.192+0000 7f5598a00640 1 -- 172.21.15.16:0/4038557098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5594104f90 msgr2=0x7f559410d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:51.194 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.192+0000 7f5598a00640 1 --2- 172.21.15.16:0/4038557098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5594104f90 0x7f559410d5d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5580009900 tx=0x7f558002a3f0 comp rx=0 tx=0).stop 2024-08-31T06:08:51.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.192+0000 7f5598a00640 1 -- 172.21.15.16:0/4038557098 shutdown_connections 2024-08-31T06:08:51.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.192+0000 7f5598a00640 1 --2- 172.21.15.16:0/4038557098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5594104f90 0x7f559410d5d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:51.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.192+0000 7f5598a00640 1 --2- 172.21.15.16:0/4038557098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f559410db10 0x7f5594110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:51.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.192+0000 7f5598a00640 1 --2- 172.21.15.16:0/4038557098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5594104560 0x7f5594104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:51.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.192+0000 7f5598a00640 1 -- 172.21.15.16:0/4038557098 >> 172.21.15.16:0/4038557098 conn(0x7f559407a4e0 msgr2=0x7f559407a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:51.196 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.193+0000 7f5598a00640 1 -- 172.21.15.16:0/4038557098 shutdown_connections 2024-08-31T06:08:51.196 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.193+0000 7f5598a00640 1 -- 172.21.15.16:0/4038557098 wait complete. 2024-08-31T06:08:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.195+0000 7f5598a00640 1 Processor -- start 2024-08-31T06:08:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.196+0000 7f5598a00640 1 -- start start 2024-08-31T06:08:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.197+0000 7f5598a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5594104560 0x7f5594074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.197+0000 7f5598a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5594104f90 0x7f559406df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.198+0000 7f5591e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5594104560 0x7f5594074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.198+0000 7f5591e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5594104560 0x7f5594074c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:46038/0 (socket says 172.21.15.16:46038) 2024-08-31T06:08:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.198+0000 7f5598a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f559410db10 0x7f559406e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.198+0000 7f5591400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5594104f90 0x7f559406df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.198+0000 7f5591e00640 1 -- 172.21.15.16:0/2036010201 learned_addr learned my addr 172.21.15.16:0/2036010201 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.198+0000 7f5598a00640 1 -- 172.21.15.16:0/2036010201 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5594113790 con 0x7f559410db10 2024-08-31T06:08:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.198+0000 7f5598a00640 1 -- 172.21.15.16:0/2036010201 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5594113610 con 0x7f5594104560 2024-08-31T06:08:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.198+0000 7f5592800640 1 --2- 172.21.15.16:0/2036010201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f559410db10 0x7f559406e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.199+0000 7f5598a00640 1 -- 172.21.15.16:0/2036010201 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5594113910 con 0x7f5594104f90 2024-08-31T06:08:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.199+0000 7f5591400640 1 -- 172.21.15.16:0/2036010201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5594104560 msgr2=0x7f5594074c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.199+0000 7f5591400640 1 --2- 172.21.15.16:0/2036010201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5594104560 0x7f5594074c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.199+0000 7f5591400640 1 -- 172.21.15.16:0/2036010201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f559410db10 msgr2=0x7f559406e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:51.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.199+0000 7f5591400640 1 --2- 172.21.15.16:0/2036010201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f559410db10 0x7f559406e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:51.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.199+0000 7f5591400640 1 -- 172.21.15.16:0/2036010201 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f559406ed00 con 0x7f5594104f90 2024-08-31T06:08:51.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.200+0000 7f5591400640 1 --2- 172.21.15.16:0/2036010201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5594104f90 0x7f559406df10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5580009230 tx=0x7f5580003be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:51.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.200+0000 7f558aa00640 1 -- 172.21.15.16:0/2036010201 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558003f070 con 0x7f5594104f90 2024-08-31T06:08:51.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.200+0000 7f558aa00640 1 -- 172.21.15.16:0/2036010201 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5580003e10 con 0x7f5594104f90 2024-08-31T06:08:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.201+0000 7f5598a00640 1 -- 172.21.15.16:0/2036010201 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5594073970 con 0x7f5594104f90 2024-08-31T06:08:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.201+0000 7f5598a00640 1 -- 172.21.15.16:0/2036010201 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5594073de0 con 0x7f5594104f90 2024-08-31T06:08:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.201+0000 7f558aa00640 1 -- 172.21.15.16:0/2036010201 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5580035030 con 0x7f5594104f90 2024-08-31T06:08:51.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.208+0000 7f559fbe2740 1 Processor -- start 2024-08-31T06:08:51.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.208+0000 7f558aa00640 1 -- 172.21.15.16:0/2036010201 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f5580042070 con 0x7f5594104f90 2024-08-31T06:08:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.208+0000 7f559fbe2740 1 -- start start 2024-08-31T06:08:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.208+0000 7f559fbe2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1eac41b0 0x559e1ee67920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.208+0000 7f559fbe2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e1ea9b2f0 0x559e1ee67e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.208+0000 7f559fbe2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e1eaa34d0 0x559e1ee683a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.208+0000 7f559fbe2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559e1eddcf70 con 0x559e1eac41b0 2024-08-31T06:08:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.208+0000 7f559fbe2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559e1edcc500 con 0x559e1ea9b2f0 2024-08-31T06:08:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.208+0000 7f559fbe2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559e1edd04b0 con 0x559e1eaa34d0 2024-08-31T06:08:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.209+0000 7f5591e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e1ea9b2f0 0x559e1ee67e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.209+0000 7f5592800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1eac41b0 0x559e1ee67920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.209+0000 7f5591e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e1ea9b2f0 0x559e1ee67e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:46042/0 (socket says 172.21.15.16:46042) 2024-08-31T06:08:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.209+0000 7f5591e00640 1 -- 172.21.15.16:0/1502388954 learned_addr learned my addr 172.21.15.16:0/1502388954 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.209+0000 7f5591400640 1 --2- 172.21.15.16:0/1502388954 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e1eaa34d0 0x559e1ee683a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.209+0000 7f5592800640 1 -- 172.21.15.16:0/1502388954 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e1eaa34d0 msgr2=0x559e1ee683a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.209+0000 7f5592800640 1 --2- 172.21.15.16:0/1502388954 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e1eaa34d0 0x559e1ee683a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.209+0000 7f5592800640 1 -- 172.21.15.16:0/1502388954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e1ea9b2f0 msgr2=0x559e1ee67e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.209+0000 7f5592800640 1 --2- 172.21.15.16:0/1502388954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e1ea9b2f0 0x559e1ee67e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.209+0000 7f5592800640 1 -- 172.21.15.16:0/1502388954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e1ed86c10 con 0x559e1eac41b0 2024-08-31T06:08:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.210+0000 7f5591e00640 1 --2- 172.21.15.16:0/1502388954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e1ea9b2f0 0x559e1ee67e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:08:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.210+0000 7f558aa00640 1 --2- 172.21.15.16:0/2036010201 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f55780b9240 0x7f55780bb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.210+0000 7f5592800640 1 --2- 172.21.15.16:0/1502388954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1eac41b0 0x559e1ee67920 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f558c009f00 tx=0x7f558c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5f94133466d4d4d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.210+0000 7f558aa00640 1 -- 172.21.15.16:0/2036010201 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f5580105a90 con 0x7f5594104f90 2024-08-31T06:08:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.210+0000 7f557f000640 1 -- 172.21.15.16:0/1502388954 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c004d20 con 0x559e1eac41b0 2024-08-31T06:08:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.210+0000 7f557f000640 1 -- 172.21.15.16:0/1502388954 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f558c004030 con 0x559e1eac41b0 2024-08-31T06:08:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.210+0000 7f557f000640 1 -- 172.21.15.16:0/1502388954 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c005260 con 0x559e1eac41b0 2024-08-31T06:08:51.215 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.210+0000 7f5591e00640 1 --2- 172.21.15.16:0/2036010201 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f55780b9240 0x7f55780bb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:51.215 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.211+0000 7f559fbe2740 1 -- 172.21.15.16:0/1502388954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1eac41b0 msgr2=0x559e1ee67920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:51.215 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.211+0000 7f559fbe2740 1 --2- 172.21.15.16:0/1502388954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1eac41b0 0x559e1ee67920 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f558c009f00 tx=0x7f558c011810 comp rx=0 tx=0).stop 2024-08-31T06:08:51.215 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.212+0000 7f5591e00640 1 --2- 172.21.15.16:0/2036010201 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f55780b9240 0x7f55780bb750 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x559e1ee665a0 tx=0x7f558400b480 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:51.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.212+0000 7f559fbe2740 1 -- 172.21.15.16:0/1502388954 shutdown_connections 2024-08-31T06:08:51.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.212+0000 7f559fbe2740 1 --2- 172.21.15.16:0/1502388954 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e1eaa34d0 0x559e1ee683a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:51.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.212+0000 7f559fbe2740 1 --2- 172.21.15.16:0/1502388954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e1ea9b2f0 0x559e1ee67e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:51.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.212+0000 7f559fbe2740 1 --2- 172.21.15.16:0/1502388954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1eac41b0 0x559e1ee67920 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:51.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.212+0000 7f559fbe2740 1 -- 172.21.15.16:0/1502388954 >> 172.21.15.16:0/1502388954 conn(0x559e1eab8300 msgr2=0x559e1ee65810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:51.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.213+0000 7f559fbe2740 1 -- 172.21.15.16:0/1502388954 shutdown_connections 2024-08-31T06:08:51.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.213+0000 7f559fbe2740 1 -- 172.21.15.16:0/1502388954 wait complete. 2024-08-31T06:08:51.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.214+0000 7f559fbe2740 1 -- start start 2024-08-31T06:08:51.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.214+0000 7f559fbe2740 1 Processor -- start 2024-08-31T06:08:51.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.215+0000 7f559fbe2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1ea9b2f0 0x559e1ee72f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:51.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.216+0000 7f559fbe2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e1eaa34d0 0x559e1ee73440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:51.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.216+0000 7f559fbe2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e1eac41b0 0x559e1ee73980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:51.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.216+0000 7f5591e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1ea9b2f0 0x559e1ee72f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:51.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.216+0000 7f5591e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1ea9b2f0 0x559e1ee72f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52212/0 (socket says 172.21.15.16:52212) 2024-08-31T06:08:51.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.216+0000 7f5591400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e1eaa34d0 0x559e1ee73440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:51.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.216+0000 7f5591400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e1eaa34d0 0x559e1ee73440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:46058/0 (socket says 172.21.15.16:46058) 2024-08-31T06:08:51.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.216+0000 7f559fbe2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559e1ed05ab0 con 0x559e1ea9b2f0 2024-08-31T06:08:51.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.216+0000 7f559fbe2740 1 -- 172.21.15.16:0/2969138792 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559e1ed73320 con 0x559e1eaa34d0 2024-08-31T06:08:51.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.216+0000 7f5591e00640 1 -- 172.21.15.16:0/2969138792 learned_addr learned my addr 172.21.15.16:0/2969138792 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:51.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.216+0000 7f559fbe2740 1 -- 172.21.15.16:0/2969138792 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559e1ee760a0 con 0x559e1eac41b0 2024-08-31T06:08:51.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.216+0000 7f5592800640 1 --2- 172.21.15.16:0/2969138792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e1eac41b0 0x559e1ee73980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:51.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.216+0000 7f559fbe2740 10 client.0 _ll_register_callbacks cb 0x559e1ed796a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:08:51.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.217+0000 7f5591e00640 1 -- 172.21.15.16:0/2969138792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e1eac41b0 msgr2=0x559e1ee73980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:51.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.217+0000 7f5591e00640 1 --2- 172.21.15.16:0/2969138792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e1eac41b0 0x559e1ee73980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:51.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.217+0000 7f5591e00640 1 -- 172.21.15.16:0/2969138792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e1eaa34d0 msgr2=0x559e1ee73440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:51.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.217+0000 7f5591e00640 1 --2- 172.21.15.16:0/2969138792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e1eaa34d0 0x559e1ee73440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:51.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.217+0000 7f5591e00640 1 -- 172.21.15.16:0/2969138792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e1eb2f790 con 0x559e1ea9b2f0 2024-08-31T06:08:51.222 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.217+0000 7f5591400640 1 --2- 172.21.15.16:0/2969138792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e1eaa34d0 0x559e1ee73440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:08:51.222 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.218+0000 7f5591e00640 1 --2- 172.21.15.16:0/2969138792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1ea9b2f0 0x559e1ee72f00 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f55840058c0 tx=0x7f5584004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f553557e8eda3fd8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:51.222 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.218+0000 7f5573400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:08:51.222 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.218+0000 7f5573400640 1 -- 172.21.15.16:0/2969138792 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5584005020 con 0x559e1ea9b2f0 2024-08-31T06:08:51.222 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.218+0000 7f559fbe2740 1 -- 172.21.15.16:0/2969138792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f558c004030 con 0x559e1ea9b2f0 2024-08-31T06:08:51.222 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.218+0000 7f5573400640 1 -- 172.21.15.16:0/2969138792 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f5584017030 con 0x559e1ea9b2f0 2024-08-31T06:08:51.223 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.219+0000 7f5573400640 1 -- 172.21.15.16:0/2969138792 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f55840051c0 con 0x559e1ea9b2f0 2024-08-31T06:08:51.223 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.219+0000 7f559fbe2740 10 client.15891 fetch_fsmap learned FSMap version 31 2024-08-31T06:08:51.223 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.219+0000 7f559fbe2740 1 -- 172.21.15.16:0/2969138792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559e1eba8620 con 0x559e1ea9b2f0 2024-08-31T06:08:51.223 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.220+0000 7f5573400640 1 -- 172.21.15.16:0/2969138792 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f558c004030 con 0x559e1ea9b2f0 2024-08-31T06:08:51.223 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.221+0000 7f5573400640 1 -- 172.21.15.16:0/2969138792 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f5584027410 con 0x559e1ea9b2f0 2024-08-31T06:08:51.223 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.221+0000 7f559fbe2740 10 client.15891 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:08:51.224 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.221+0000 7f559fbe2740 10 client.15891 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:08:51.224 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.221+0000 7f559fbe2740 20 client.15891 populate_metadata read hostname 'smithi016' 2024-08-31T06:08:51.224 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.221+0000 7f559fbe2740 1 --2- 172.21.15.16:0/2969138792 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559e1ea82d80 0x559e1ee85190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.221+0000 7f559fbe2740 4 client.15891 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:08:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.221+0000 7f559fbe2740 1 -- 172.21.15.16:0/2969138792 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e1eddeec0 con 0x559e1ea82d80 2024-08-31T06:08:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.221+0000 7f5591400640 1 --2- 172.21.15.16:0/2969138792 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559e1ea82d80 0x559e1ee85190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.223+0000 7f5573400640 10 client.15891 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.224+0000 7f5591400640 1 --2- 172.21.15.16:0/2969138792 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559e1ea82d80 0x559e1ee85190 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=505de5fb47d5863c server_cookie=67d5678e37af4bcc in_seq=0 out_seq=0 2024-08-31T06:08:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.225+0000 7f5573400640 1 -- 172.21.15.16:0/2969138792 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559e1eddeec0 con 0x559e1ea82d80 2024-08-31T06:08:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.225+0000 7f5573400640 10 client.15891 handle_command_reply: tid=0 2024-08-31T06:08:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.225+0000 7f559fbe2740 1 client.15891 shutdown 2024-08-31T06:08:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.226+0000 7f559fbe2740 1 -- 172.21.15.16:0/2969138792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1ea9b2f0 msgr2=0x559e1ee72f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.226+0000 7f559fbe2740 1 --2- 172.21.15.16:0/2969138792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1ea9b2f0 0x559e1ee72f00 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f55840058c0 tx=0x7f5584004730 comp rx=0 tx=0).stop 2024-08-31T06:08:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.227+0000 7f559fbe2740 1 -- 172.21.15.16:0/2969138792 shutdown_connections 2024-08-31T06:08:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.227+0000 7f559fbe2740 1 --2- 172.21.15.16:0/2969138792 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559e1ea82d80 0x559e1ee85190 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.228+0000 7f559fbe2740 1 --2- 172.21.15.16:0/2969138792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e1eac41b0 0x559e1ee73980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.228+0000 7f559fbe2740 1 --2- 172.21.15.16:0/2969138792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e1eaa34d0 0x559e1ee73440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.228+0000 7f559fbe2740 1 --2- 172.21.15.16:0/2969138792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1ea9b2f0 0x559e1ee72f00 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.228+0000 7f559fbe2740 1 -- 172.21.15.16:0/2969138792 >> 172.21.15.16:0/2969138792 conn(0x559e1eab8300 msgr2=0x559e1ee68950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.228+0000 7f5573400640 0 client.15891 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.229+0000 7f559fbe2740 1 -- 172.21.15.16:0/2969138792 shutdown_connections 2024-08-31T06:08:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.229+0000 7f559fbe2740 1 -- 172.21.15.16:0/2969138792 wait complete. 2024-08-31T06:08:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.229+0000 7f559fbe2740 20 client.15891 trim_cache size 0 max 16384 2024-08-31T06:08:51.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.259+0000 7f559fbe2740 1 Processor -- start 2024-08-31T06:08:51.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.259+0000 7f559fbe2740 1 -- start start 2024-08-31T06:08:51.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.260+0000 7f559fbe2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e1ea9b2f0 0x559e1ee69310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.260+0000 7f559fbe2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e1eaa34d0 0x559e1ee74b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.260+0000 7f559fbe2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1eac41b0 0x559e1ee700d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.260+0000 7f559fbe2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f55941b8290 con 0x559e1eac41b0 2024-08-31T06:08:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.260+0000 7f559fbe2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f55941b5aa0 con 0x559e1eaa34d0 2024-08-31T06:08:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.260+0000 7f559fbe2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f55941b38f0 con 0x559e1ea9b2f0 2024-08-31T06:08:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.261+0000 7f5591e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e1ea9b2f0 0x559e1ee69310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.261+0000 7f5591e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e1ea9b2f0 0x559e1ee69310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:35878/0 (socket says 172.21.15.16:35878) 2024-08-31T06:08:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.261+0000 7f5591400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e1eaa34d0 0x559e1ee74b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.261+0000 7f5591e00640 1 -- 172.21.15.16:0/556784972 learned_addr learned my addr 172.21.15.16:0/556784972 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.261+0000 7f5591e00640 1 -- 172.21.15.16:0/556784972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e1eaa34d0 msgr2=0x559e1ee74b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.261+0000 7f5591e00640 1 --2- 172.21.15.16:0/556784972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e1eaa34d0 0x559e1ee74b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.261+0000 7f5591e00640 1 -- 172.21.15.16:0/556784972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1eac41b0 msgr2=0x559e1ee700d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.261+0000 7f5591e00640 1 --2- 172.21.15.16:0/556784972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1eac41b0 0x559e1ee700d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.261+0000 7f5591e00640 1 -- 172.21.15.16:0/556784972 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e1ee69c20 con 0x559e1ea9b2f0 2024-08-31T06:08:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.262+0000 7f5591e00640 1 --2- 172.21.15.16:0/556784972 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e1ea9b2f0 0x559e1ee69310 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5584003a80 tx=0x7f558400b4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46bc950bf2c988a0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.263+0000 7f557e600640 1 -- 172.21.15.16:0/556784972 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55840273e0 con 0x559e1ea9b2f0 2024-08-31T06:08:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.263+0000 7f557e600640 1 -- 172.21.15.16:0/556784972 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5584027e50 con 0x559e1ea9b2f0 2024-08-31T06:08:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.263+0000 7f557e600640 1 -- 172.21.15.16:0/556784972 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5584004d80 con 0x559e1ea9b2f0 2024-08-31T06:08:51.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.265+0000 7f559fbe2740 1 -- 172.21.15.16:0/556784972 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e1ea9b2f0 msgr2=0x559e1ee69310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:51.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.265+0000 7f559fbe2740 1 --2- 172.21.15.16:0/556784972 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e1ea9b2f0 0x559e1ee69310 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5584003a80 tx=0x7f558400b4f0 comp rx=0 tx=0).stop 2024-08-31T06:08:51.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.266+0000 7f559fbe2740 1 -- 172.21.15.16:0/556784972 shutdown_connections 2024-08-31T06:08:51.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.266+0000 7f559fbe2740 1 --2- 172.21.15.16:0/556784972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1eac41b0 0x559e1ee700d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:51.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.266+0000 7f559fbe2740 1 --2- 172.21.15.16:0/556784972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e1eaa34d0 0x559e1ee74b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:51.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.266+0000 7f559fbe2740 1 --2- 172.21.15.16:0/556784972 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e1ea9b2f0 0x559e1ee69310 secure :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5584003a80 tx=0x7f558400b4f0 comp rx=0 tx=0).stop 2024-08-31T06:08:51.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.266+0000 7f559fbe2740 1 -- 172.21.15.16:0/556784972 >> 172.21.15.16:0/556784972 conn(0x559e1eab8300 msgr2=0x559e1ee74420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:51.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.267+0000 7f559fbe2740 1 -- 172.21.15.16:0/556784972 shutdown_connections 2024-08-31T06:08:51.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.272+0000 7f559fbe2740 1 -- 172.21.15.16:0/556784972 wait complete. 2024-08-31T06:08:51.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.272+0000 7f559fbe2740 1 -- start start 2024-08-31T06:08:51.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.273+0000 7f559fbe2740 1 Processor -- start 2024-08-31T06:08:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.277+0000 7f559fbe2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1eaa34d0 0x559e1ee7bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.278+0000 7f5591e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1eaa34d0 0x559e1ee7bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.279+0000 7f5591e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1eaa34d0 0x559e1ee7bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52234/0 (socket says 172.21.15.16:52234) 2024-08-31T06:08:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.278+0000 7f559fbe2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e1eac41b0 0x559e1ee79100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.282+0000 7f559fbe2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e1ea581c0 0x559e1ee797a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.282+0000 7f559fbe2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f55941b3f60 con 0x559e1eaa34d0 2024-08-31T06:08:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.282+0000 7f559fbe2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f55941b53d0 con 0x559e1eac41b0 2024-08-31T06:08:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.282+0000 7f559fbe2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f55941b4630 con 0x559e1ea581c0 2024-08-31T06:08:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.282+0000 7f559fbe2740 10 client.0 _ll_register_callbacks cb 0x7f5584003cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:08:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.282+0000 7f5591e00640 1 -- 172.21.15.16:0/1251849917 learned_addr learned my addr 172.21.15.16:0/1251849917 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.283+0000 7f5591e00640 1 -- 172.21.15.16:0/1251849917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e1ea581c0 msgr2=0x559e1ee797a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.283+0000 7f5591e00640 1 --2- 172.21.15.16:0/1251849917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e1ea581c0 0x559e1ee797a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.283+0000 7f5591400640 1 --2- 172.21.15.16:0/1251849917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e1eac41b0 0x559e1ee79100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.284+0000 7f5591e00640 1 -- 172.21.15.16:0/1251849917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e1eac41b0 msgr2=0x559e1ee79100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.284+0000 7f5591e00640 1 --2- 172.21.15.16:0/1251849917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e1eac41b0 0x559e1ee79100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.284+0000 7f5591e00640 1 -- 172.21.15.16:0/1251849917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e1ee79f30 con 0x559e1eaa34d0 2024-08-31T06:08:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.285+0000 7f5591e00640 1 --2- 172.21.15.16:0/1251849917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1eaa34d0 0x559e1ee7bc40 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f55840059d0 tx=0x7f558400bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0136944547d3d1a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.285+0000 7f5573400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:08:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.288+0000 7f5573400640 1 -- 172.21.15.16:0/1251849917 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5584047070 con 0x559e1eaa34d0 2024-08-31T06:08:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.288+0000 7f5573400640 1 -- 172.21.15.16:0/1251849917 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f5584027c20 con 0x559e1eaa34d0 2024-08-31T06:08:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.288+0000 7f559fbe2740 1 -- 172.21.15.16:0/1251849917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5584027e50 con 0x559e1eaa34d0 2024-08-31T06:08:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.289+0000 7f5573400640 1 -- 172.21.15.16:0/1251849917 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5584005360 con 0x559e1eaa34d0 2024-08-31T06:08:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.289+0000 7f559fbe2740 10 client.15894 fetch_fsmap learned FSMap version 31 2024-08-31T06:08:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.289+0000 7f559fbe2740 1 -- 172.21.15.16:0/1251849917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559e1eeefc60 con 0x559e1eaa34d0 2024-08-31T06:08:51.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.291+0000 7f5573400640 1 -- 172.21.15.16:0/1251849917 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5584027e50 con 0x559e1eaa34d0 2024-08-31T06:08:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.291+0000 7f5573400640 1 -- 172.21.15.16:0/1251849917 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f5584056600 con 0x559e1eaa34d0 2024-08-31T06:08:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.292+0000 7f559fbe2740 10 client.15894 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:08:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.292+0000 7f559fbe2740 10 client.15894 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:08:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.292+0000 7f559fbe2740 20 client.15894 populate_metadata read hostname 'smithi016' 2024-08-31T06:08:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.292+0000 7f559fbe2740 1 --2- 172.21.15.16:0/1251849917 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559e1eef0290 0x559e1eef26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.292+0000 7f559fbe2740 4 client.15894 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:08:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.292+0000 7f559fbe2740 1 -- 172.21.15.16:0/1251849917 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559e1edda960 con 0x559e1eef0290 2024-08-31T06:08:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.292+0000 7f5591400640 1 --2- 172.21.15.16:0/1251849917 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559e1eef0290 0x559e1eef26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.294+0000 7f5591400640 1 --2- 172.21.15.16:0/1251849917 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559e1eef0290 0x559e1eef26b0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65927fcc53c00566 server_cookie=2c3a6c792e65f6a4 in_seq=0 out_seq=0 2024-08-31T06:08:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.294+0000 7f5573400640 10 client.15894 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.333+0000 7f5573400640 1 -- 172.21.15.16:0/1251849917 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559e1edda960 con 0x559e1eef0290 2024-08-31T06:08:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.333+0000 7f5573400640 10 client.15894 handle_command_reply: tid=0 2024-08-31T06:08:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.333+0000 7f559fbe2740 1 client.15894 shutdown 2024-08-31T06:08:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.333+0000 7f559fbe2740 1 -- 172.21.15.16:0/1251849917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1eaa34d0 msgr2=0x559e1ee7bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.333+0000 7f559fbe2740 1 --2- 172.21.15.16:0/1251849917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1eaa34d0 0x559e1ee7bc40 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f55840059d0 tx=0x7f558400bfd0 comp rx=0 tx=0).stop 2024-08-31T06:08:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.333+0000 7f559fbe2740 1 -- 172.21.15.16:0/1251849917 shutdown_connections 2024-08-31T06:08:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.333+0000 7f559fbe2740 1 --2- 172.21.15.16:0/1251849917 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559e1eef0290 0x559e1eef26b0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.333+0000 7f559fbe2740 1 --2- 172.21.15.16:0/1251849917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e1ea581c0 0x559e1ee797a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.333+0000 7f559fbe2740 1 --2- 172.21.15.16:0/1251849917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e1eac41b0 0x559e1ee79100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.333+0000 7f559fbe2740 1 --2- 172.21.15.16:0/1251849917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1eaa34d0 0x559e1ee7bc40 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.333+0000 7f5573400640 0 client.15894 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.333+0000 7f559fbe2740 1 -- 172.21.15.16:0/1251849917 >> 172.21.15.16:0/1251849917 conn(0x559e1eab8300 msgr2=0x559e1ee74d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.334+0000 7f559fbe2740 1 -- 172.21.15.16:0/1251849917 shutdown_connections 2024-08-31T06:08:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.334+0000 7f559fbe2740 1 -- 172.21.15.16:0/1251849917 wait complete. 2024-08-31T06:08:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.334+0000 7f559fbe2740 20 client.15894 trim_cache size 0 max 16384 2024-08-31T06:08:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:08:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:08:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:08:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:08:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.336+0000 7f5598a00640 1 -- 172.21.15.16:0/2036010201 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f55780b9240 msgr2=0x7f55780bb750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:51.339 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.336+0000 7f5598a00640 1 --2- 172.21.15.16:0/2036010201 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f55780b9240 0x7f55780bb750 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x559e1ee665a0 tx=0x7f558400b480 comp rx=0 tx=0).stop 2024-08-31T06:08:51.339 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.336+0000 7f5598a00640 1 -- 172.21.15.16:0/2036010201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5594104f90 msgr2=0x7f559406df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:51.339 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.336+0000 7f5598a00640 1 --2- 172.21.15.16:0/2036010201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5594104f90 0x7f559406df10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5580009230 tx=0x7f5580003be0 comp rx=0 tx=0).stop 2024-08-31T06:08:51.339 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.336+0000 7f5598a00640 1 -- 172.21.15.16:0/2036010201 shutdown_connections 2024-08-31T06:08:51.339 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.336+0000 7f5598a00640 1 --2- 172.21.15.16:0/2036010201 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f55780b9240 0x7f55780bb750 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:51.339 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.336+0000 7f5598a00640 1 --2- 172.21.15.16:0/2036010201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5594104f90 0x7f559406df10 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.336+0000 7f5598a00640 1 --2- 172.21.15.16:0/2036010201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f559410db10 0x7f559406e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.336+0000 7f5598a00640 1 --2- 172.21.15.16:0/2036010201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5594104560 0x7f5594074c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.336+0000 7f5598a00640 1 -- 172.21.15.16:0/2036010201 >> 172.21.15.16:0/2036010201 conn(0x7f559407a4e0 msgr2=0x7f55941005e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.337+0000 7f5598a00640 1 -- 172.21.15.16:0/2036010201 shutdown_connections 2024-08-31T06:08:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:08:51.337+0000 7f5598a00640 1 -- 172.21.15.16:0/2036010201 wait complete. 2024-08-31T06:08:51.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 85.0 s, 2948.9 tps, lat 34.043 ms stddev 39.049 2024-08-31T06:08:52.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:52 smithi155 ceph-mon[21742]: pgmap v413: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.0 MiB/s wr, 678 op/s 2024-08-31T06:08:52.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:52 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:08:52.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:52 smithi016 ceph-mon[17194]: pgmap v413: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.0 MiB/s wr, 678 op/s 2024-08-31T06:08:52.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:52 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:08:52.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:52 smithi184 ceph-mon[22331]: pgmap v413: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.0 MiB/s wr, 678 op/s 2024-08-31T06:08:52.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:52 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:08:54.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:54 smithi155 ceph-mon[21742]: pgmap v414: 97 pgs: 97 active+clean; 1.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.9 MiB/s wr, 757 op/s 2024-08-31T06:08:54.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:54 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:08:54.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:54 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:08:54.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:54 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:08:54.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:54 smithi016 ceph-mon[17194]: pgmap v414: 97 pgs: 97 active+clean; 1.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.9 MiB/s wr, 757 op/s 2024-08-31T06:08:54.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:54 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:08:54.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:54 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:08:54.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:54 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:08:54.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:54 smithi184 ceph-mon[22331]: pgmap v414: 97 pgs: 97 active+clean; 1.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.9 MiB/s wr, 757 op/s 2024-08-31T06:08:54.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:54 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:08:54.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:54 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:08:54.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:54 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:08:56.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 90.0 s, 3131.5 tps, lat 31.769 ms stddev 34.480 2024-08-31T06:08:56.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:56 smithi155 ceph-mon[21742]: pgmap v415: 97 pgs: 97 active+clean; 1.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.5 MiB/s wr, 774 op/s 2024-08-31T06:08:56.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:56 smithi016 ceph-mon[17194]: pgmap v415: 97 pgs: 97 active+clean; 1.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.5 MiB/s wr, 774 op/s 2024-08-31T06:08:56.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:56 smithi184 ceph-mon[22331]: pgmap v415: 97 pgs: 97 active+clean; 1.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.5 MiB/s wr, 774 op/s 2024-08-31T06:08:57.020 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T06:08:57.675 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:08:58.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.515+0000 7f7509400640 1 -- 172.21.15.16:0/914953763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7504077120 msgr2=0x7f7504075550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:58.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.515+0000 7f7509400640 1 --2- 172.21.15.16:0/914953763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7504077120 0x7f7504075550 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f74f4009900 tx=0x7f74f402a410 comp rx=0 tx=0).stop 2024-08-31T06:08:58.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.515+0000 7f7509400640 1 -- 172.21.15.16:0/914953763 shutdown_connections 2024-08-31T06:08:58.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.515+0000 7f7509400640 1 --2- 172.21.15.16:0/914953763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7504075b70 0x7f7504075f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:58.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.515+0000 7f7509400640 1 --2- 172.21.15.16:0/914953763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7504114020 0x7f75041164f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:58.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.515+0000 7f7509400640 1 --2- 172.21.15.16:0/914953763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7504077120 0x7f7504075550 secure :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f74f4009900 tx=0x7f74f402a410 comp rx=0 tx=0).stop 2024-08-31T06:08:58.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.515+0000 7f7509400640 1 -- 172.21.15.16:0/914953763 >> 172.21.15.16:0/914953763 conn(0x7f75040ff1e0 msgr2=0x7f7504101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:58.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.516+0000 7f7509400640 1 -- 172.21.15.16:0/914953763 shutdown_connections 2024-08-31T06:08:58.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.516+0000 7f7509400640 1 -- 172.21.15.16:0/914953763 wait complete. 2024-08-31T06:08:58.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.518+0000 7f7509400640 1 Processor -- start 2024-08-31T06:08:58.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.518+0000 7f7509400640 1 -- start start 2024-08-31T06:08:58.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.519+0000 7f7509400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7504075b70 0x7f7504113940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:58.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.519+0000 7f7509400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7504114020 0x7f750410cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:58.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.520+0000 7f7503e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7504075b70 0x7f7504113940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:58.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.520+0000 7f7503e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7504075b70 0x7f7504113940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:34348/0 (socket says 172.21.15.16:34348) 2024-08-31T06:08:58.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.520+0000 7f7503e00640 1 -- 172.21.15.16:0/2950561857 learned_addr learned my addr 172.21.15.16:0/2950561857 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:58.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.520+0000 7f7503400640 1 --2- 172.21.15.16:0/2950561857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7504114020 0x7f750410cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:58.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.520+0000 7f7509400640 1 --2- 172.21.15.16:0/2950561857 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f750410d1b0 0x7f75041124f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:58.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.521+0000 7f7509400640 1 -- 172.21.15.16:0/2950561857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7504119cb0 con 0x7f7504114020 2024-08-31T06:08:58.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.521+0000 7f7509400640 1 -- 172.21.15.16:0/2950561857 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7504119b30 con 0x7f7504075b70 2024-08-31T06:08:58.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.521+0000 7f7508a00640 1 --2- 172.21.15.16:0/2950561857 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f750410d1b0 0x7f75041124f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:58.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.522+0000 7f7509400640 1 -- 172.21.15.16:0/2950561857 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7504119e30 con 0x7f750410d1b0 2024-08-31T06:08:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.523+0000 7f7503e00640 1 -- 172.21.15.16:0/2950561857 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f750410d1b0 msgr2=0x7f75041124f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:58.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.524+0000 7f7503e00640 1 --2- 172.21.15.16:0/2950561857 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f750410d1b0 0x7f75041124f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:58.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.524+0000 7f7503e00640 1 -- 172.21.15.16:0/2950561857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7504114020 msgr2=0x7f750410cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:58.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.524+0000 7f7503e00640 1 --2- 172.21.15.16:0/2950561857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7504114020 0x7f750410cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:58.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.524+0000 7f7503e00640 1 -- 172.21.15.16:0/2950561857 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f750410d8c0 con 0x7f7504075b70 2024-08-31T06:08:58.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.524+0000 7f7508a00640 1 --2- 172.21.15.16:0/2950561857 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f750410d1b0 0x7f75041124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:08:58.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.524+0000 7f7503400640 1 --2- 172.21.15.16:0/2950561857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7504114020 0x7f750410cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:08:58.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.525+0000 7f7503e00640 1 --2- 172.21.15.16:0/2950561857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7504075b70 0x7f7504113940 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f74f402aa00 tx=0x7f74f4003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:58.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.525+0000 7f7500c00640 1 -- 172.21.15.16:0/2950561857 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f403f020 con 0x7f7504075b70 2024-08-31T06:08:58.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.525+0000 7f7500c00640 1 -- 172.21.15.16:0/2950561857 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74f4031d60 con 0x7f7504075b70 2024-08-31T06:08:58.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.525+0000 7f7500c00640 1 -- 172.21.15.16:0/2950561857 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f4035070 con 0x7f7504075b70 2024-08-31T06:08:58.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.525+0000 7f7509400640 1 -- 172.21.15.16:0/2950561857 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7504112a30 con 0x7f7504075b70 2024-08-31T06:08:58.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.526+0000 7f7509400640 1 -- 172.21.15.16:0/2950561857 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7504112f20 con 0x7f7504075b70 2024-08-31T06:08:58.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.538+0000 7f7500c00640 1 -- 172.21.15.16:0/2950561857 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f74f4042020 con 0x7f7504075b70 2024-08-31T06:08:58.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.540+0000 7f75116aa740 1 Processor -- start 2024-08-31T06:08:58.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.540+0000 7f75116aa740 1 -- start start 2024-08-31T06:08:58.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.541+0000 7f75116aa740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bc5bb8070 0x564bc5f5db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:58.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.541+0000 7f75116aa740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bc5b8f1b0 0x564bc5f5e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:58.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.542+0000 7f7508a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bc5bb8070 0x564bc5f5db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:58.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.542+0000 7f7508a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bc5bb8070 0x564bc5f5db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:34352/0 (socket says 172.21.15.16:34352) 2024-08-31T06:08:58.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.542+0000 7f7503e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bc5b8f1b0 0x564bc5f5e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:58.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.542+0000 7f7503e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bc5b8f1b0 0x564bc5f5e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:55356/0 (socket says 172.21.15.16:55356) 2024-08-31T06:08:58.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.542+0000 7f7508a00640 1 -- 172.21.15.16:0/606490693 learned_addr learned my addr 172.21.15.16:0/606490693 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:58.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.543+0000 7f75116aa740 1 --2- 172.21.15.16:0/606490693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc5b97390 0x564bc5f5e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:58.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.543+0000 7f75116aa740 1 -- 172.21.15.16:0/606490693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564bc5ed4770 con 0x564bc5b97390 2024-08-31T06:08:58.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.543+0000 7f75116aa740 1 -- 172.21.15.16:0/606490693 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x564bc5ec3d00 con 0x564bc5bb8070 2024-08-31T06:08:58.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.543+0000 7f75116aa740 1 -- 172.21.15.16:0/606490693 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564bc5ec7cb0 con 0x564bc5b8f1b0 2024-08-31T06:08:58.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.544+0000 7f7503e00640 1 -- 172.21.15.16:0/606490693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bc5bb8070 msgr2=0x564bc5f5db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:58.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.544+0000 7f7503e00640 1 --2- 172.21.15.16:0/606490693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bc5bb8070 0x564bc5f5db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:58.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.544+0000 7f7500c00640 1 --2- 172.21.15.16:0/2950561857 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f74e80b4e70 0x7f74e80b7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:58.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.544+0000 7f7503e00640 1 -- 172.21.15.16:0/606490693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc5b97390 msgr2=0x564bc5f5e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:58.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.544+0000 7f7503e00640 1 --2- 172.21.15.16:0/606490693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc5b97390 0x564bc5f5e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:58.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.545+0000 7f7503e00640 1 -- 172.21.15.16:0/606490693 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bc5c23650 con 0x564bc5b8f1b0 2024-08-31T06:08:58.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.545+0000 7f7500c00640 1 -- 172.21.15.16:0/2950561857 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f74f41057a0 con 0x7f7504075b70 2024-08-31T06:08:58.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.546+0000 7f7503e00640 1 --2- 172.21.15.16:0/606490693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bc5b8f1b0 0x564bc5f5e060 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f74f4102ef0 tx=0x7f74f40908c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2f58e6bc2dfe883 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:58.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.546+0000 7f7508a00640 1 --2- 172.21.15.16:0/2950561857 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f74e80b4e70 0x7f74e80b7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:58.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.547+0000 7f74f9200640 1 -- 172.21.15.16:0/606490693 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f40cb7e0 con 0x564bc5b8f1b0 2024-08-31T06:08:58.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.547+0000 7f74f9200640 1 -- 172.21.15.16:0/606490693 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74f4099070 con 0x564bc5b8f1b0 2024-08-31T06:08:58.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.547+0000 7f74f9200640 1 -- 172.21.15.16:0/606490693 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f40cb600 con 0x564bc5b8f1b0 2024-08-31T06:08:58.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.552+0000 7f7508a00640 1 --2- 172.21.15.16:0/2950561857 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f74e80b4e70 0x7f74e80b7380 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f74fc010890 tx=0x7f74fc007520 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:58.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.552+0000 7f75116aa740 1 -- 172.21.15.16:0/606490693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bc5b8f1b0 msgr2=0x564bc5f5e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:58.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.552+0000 7f75116aa740 1 --2- 172.21.15.16:0/606490693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bc5b8f1b0 0x564bc5f5e060 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f74f4102ef0 tx=0x7f74f40908c0 comp rx=0 tx=0).stop 2024-08-31T06:08:58.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.553+0000 7f75116aa740 1 -- 172.21.15.16:0/606490693 shutdown_connections 2024-08-31T06:08:58.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.553+0000 7f75116aa740 1 --2- 172.21.15.16:0/606490693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bc5b8f1b0 0x564bc5f5e060 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:58.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.553+0000 7f75116aa740 1 --2- 172.21.15.16:0/606490693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc5b97390 0x564bc5f5e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:58.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.553+0000 7f75116aa740 1 --2- 172.21.15.16:0/606490693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bc5bb8070 0x564bc5f5db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:58.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.553+0000 7f75116aa740 1 -- 172.21.15.16:0/606490693 >> 172.21.15.16:0/606490693 conn(0x564bc5bac460 msgr2=0x564bc5f5ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:58.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.554+0000 7f75116aa740 1 -- 172.21.15.16:0/606490693 shutdown_connections 2024-08-31T06:08:58.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.554+0000 7f75116aa740 1 -- 172.21.15.16:0/606490693 wait complete. 2024-08-31T06:08:58.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.555+0000 7f75116aa740 1 -- start start 2024-08-31T06:08:58.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.556+0000 7f75116aa740 1 Processor -- start 2024-08-31T06:08:58.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.556+0000 7f75116aa740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bc5b8f1b0 0x564bc5f697c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:58.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.556+0000 7f75116aa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc5b97390 0x564bc5f69da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:58.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.556+0000 7f75116aa740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bc5bb8070 0x564bc5f6f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:58.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.556+0000 7f75116aa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564bc5d26700 con 0x564bc5b97390 2024-08-31T06:08:58.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.556+0000 7f75116aa740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x564bc5e4acb0 con 0x564bc5bb8070 2024-08-31T06:08:58.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.557+0000 7f75116aa740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564bc5ec7cb0 con 0x564bc5b8f1b0 2024-08-31T06:08:58.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.557+0000 7f75116aa740 10 client.0 _ll_register_callbacks cb 0x564bc5e70430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:08:58.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.557+0000 7f7503e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bc5b8f1b0 0x564bc5f697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:58.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.557+0000 7f7503e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bc5b8f1b0 0x564bc5f697c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:55366/0 (socket says 172.21.15.16:55366) 2024-08-31T06:08:58.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.557+0000 7f7503e00640 1 -- 172.21.15.16:0/1486286506 learned_addr learned my addr 172.21.15.16:0/1486286506 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:58.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.558+0000 7f7503400640 1 --2- 172.21.15.16:0/1486286506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc5b97390 0x564bc5f69da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:58.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.558+0000 7f7508a00640 1 --2- 172.21.15.16:0/1486286506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bc5bb8070 0x564bc5f6f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:58.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.558+0000 7f7503e00640 1 -- 172.21.15.16:0/1486286506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bc5bb8070 msgr2=0x564bc5f6f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:58.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.558+0000 7f7503e00640 1 --2- 172.21.15.16:0/1486286506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bc5bb8070 0x564bc5f6f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:58.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.558+0000 7f7503e00640 1 -- 172.21.15.16:0/1486286506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc5b97390 msgr2=0x564bc5f69da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:58.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.558+0000 7f7503e00640 1 --2- 172.21.15.16:0/1486286506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc5b97390 0x564bc5f69da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:58.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.558+0000 7f7503e00640 1 -- 172.21.15.16:0/1486286506 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bc5e8b870 con 0x564bc5b8f1b0 2024-08-31T06:08:58.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.560+0000 7f7503e00640 1 --2- 172.21.15.16:0/1486286506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bc5b8f1b0 0x564bc5f697c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f74f40c9100 tx=0x7f74f403b460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7b60e01c3cf248 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:58.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.560+0000 7f74ed600640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:08:58.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.561+0000 7f74ed600640 1 -- 172.21.15.16:0/1486286506 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f40cb600 con 0x564bc5b8f1b0 2024-08-31T06:08:58.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.561+0000 7f74ed600640 1 -- 172.21.15.16:0/1486286506 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f74f403b4f0 con 0x564bc5b8f1b0 2024-08-31T06:08:58.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.561+0000 7f75116aa740 1 -- 172.21.15.16:0/1486286506 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74f4099070 con 0x564bc5b8f1b0 2024-08-31T06:08:58.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.562+0000 7f74ed600640 1 -- 172.21.15.16:0/1486286506 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f74f40cb7e0 con 0x564bc5b8f1b0 2024-08-31T06:08:58.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.562+0000 7f75116aa740 10 client.25945 fetch_fsmap learned FSMap version 31 2024-08-31T06:08:58.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.562+0000 7f75116aa740 1 -- 172.21.15.16:0/1486286506 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564bc5f88f30 con 0x564bc5b8f1b0 2024-08-31T06:08:58.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.563+0000 7f74ed600640 1 -- 172.21.15.16:0/1486286506 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74f4099070 con 0x564bc5b8f1b0 2024-08-31T06:08:58.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.563+0000 7f74ed600640 1 -- 172.21.15.16:0/1486286506 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f74f40c7800 con 0x564bc5b8f1b0 2024-08-31T06:08:58.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.564+0000 7f75116aa740 10 client.25945 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:08:58.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.564+0000 7f75116aa740 10 client.25945 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:08:58.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.564+0000 7f75116aa740 20 client.25945 populate_metadata read hostname 'smithi016' 2024-08-31T06:08:58.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.564+0000 7f75116aa740 1 --2- 172.21.15.16:0/1486286506 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564bc5b769f0 0x564bc5f8c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:58.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.564+0000 7f75116aa740 4 client.25945 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:08:58.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.564+0000 7f75116aa740 1 -- 172.21.15.16:0/1486286506 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564bc5ed66c0 con 0x564bc5b769f0 2024-08-31T06:08:58.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.565+0000 7f7503400640 1 --2- 172.21.15.16:0/1486286506 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564bc5b769f0 0x564bc5f8c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:58.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.566+0000 7f7503400640 1 --2- 172.21.15.16:0/1486286506 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564bc5b769f0 0x564bc5f8c290 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7777e087e08e421a server_cookie=fc208353849d8564 in_seq=0 out_seq=0 2024-08-31T06:08:58.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.566+0000 7f74ed600640 10 client.25945 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:58.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.567+0000 7f74ed600640 1 -- 172.21.15.16:0/1486286506 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564bc5ed66c0 con 0x564bc5b769f0 2024-08-31T06:08:58.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.567+0000 7f74ed600640 10 client.25945 handle_command_reply: tid=0 2024-08-31T06:08:58.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.567+0000 7f75116aa740 1 client.25945 shutdown 2024-08-31T06:08:58.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.568+0000 7f75116aa740 1 -- 172.21.15.16:0/1486286506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bc5b8f1b0 msgr2=0x564bc5f697c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:58.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.568+0000 7f75116aa740 1 --2- 172.21.15.16:0/1486286506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bc5b8f1b0 0x564bc5f697c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f74f40c9100 tx=0x7f74f403b460 comp rx=0 tx=0).stop 2024-08-31T06:08:58.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.569+0000 7f75116aa740 1 -- 172.21.15.16:0/1486286506 shutdown_connections 2024-08-31T06:08:58.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.569+0000 7f75116aa740 1 --2- 172.21.15.16:0/1486286506 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564bc5b769f0 0x564bc5f8c290 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:58.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.569+0000 7f75116aa740 1 --2- 172.21.15.16:0/1486286506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bc5bb8070 0x564bc5f6f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:58.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.569+0000 7f75116aa740 1 --2- 172.21.15.16:0/1486286506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc5b97390 0x564bc5f69da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:58.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.569+0000 7f75116aa740 1 --2- 172.21.15.16:0/1486286506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bc5b8f1b0 0x564bc5f697c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:58.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.569+0000 7f75116aa740 1 -- 172.21.15.16:0/1486286506 >> 172.21.15.16:0/1486286506 conn(0x564bc5bac460 msgr2=0x564bc5f60d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:58.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.569+0000 7f74ed600640 0 client.25945 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:58.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.569+0000 7f75116aa740 1 -- 172.21.15.16:0/1486286506 shutdown_connections 2024-08-31T06:08:58.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.569+0000 7f75116aa740 1 -- 172.21.15.16:0/1486286506 wait complete. 2024-08-31T06:08:58.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.570+0000 7f75116aa740 20 client.25945 trim_cache size 0 max 16384 2024-08-31T06:08:58.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.608+0000 7f75116aa740 1 Processor -- start 2024-08-31T06:08:58.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.608+0000 7f75116aa740 1 -- start start 2024-08-31T06:08:58.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.609+0000 7f75116aa740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bc5b8f1b0 0x564bc5f6f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:58.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.609+0000 7f75116aa740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bc5b97390 0x564bc5f62c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:58.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.609+0000 7f75116aa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc5bb8070 0x564bc5f63170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:58.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.609+0000 7f75116aa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f75041b8700 con 0x564bc5bb8070 2024-08-31T06:08:58.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.609+0000 7f75116aa740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7504075100 con 0x564bc5b97390 2024-08-31T06:08:58.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.609+0000 7f75116aa740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7504072f10 con 0x564bc5b8f1b0 2024-08-31T06:08:58.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.610+0000 7f7503400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bc5b97390 0x564bc5f62c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:58.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.610+0000 7f7503400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bc5b97390 0x564bc5f62c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:34370/0 (socket says 172.21.15.16:34370) 2024-08-31T06:08:58.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.610+0000 7f7503400640 1 -- 172.21.15.16:0/3616806921 learned_addr learned my addr 172.21.15.16:0/3616806921 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:58.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.610+0000 7f7508a00640 1 --2- 172.21.15.16:0/3616806921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc5bb8070 0x564bc5f63170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:58.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.610+0000 7f7503e00640 1 --2- 172.21.15.16:0/3616806921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bc5b8f1b0 0x564bc5f6f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:58.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.611+0000 7f7503400640 1 -- 172.21.15.16:0/3616806921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bc5b8f1b0 msgr2=0x564bc5f6f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:58.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.611+0000 7f7503400640 1 --2- 172.21.15.16:0/3616806921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bc5b8f1b0 0x564bc5f6f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:58.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.611+0000 7f7503400640 1 -- 172.21.15.16:0/3616806921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc5bb8070 msgr2=0x564bc5f63170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:58.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.611+0000 7f7503400640 1 --2- 172.21.15.16:0/3616806921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc5bb8070 0x564bc5f63170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:58.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.611+0000 7f7503400640 1 -- 172.21.15.16:0/3616806921 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bc5f67ac0 con 0x564bc5b97390 2024-08-31T06:08:58.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.611+0000 7f7508a00640 1 --2- 172.21.15.16:0/3616806921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc5bb8070 0x564bc5f63170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:08:58.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.612+0000 7f7503400640 1 --2- 172.21.15.16:0/3616806921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bc5b97390 0x564bc5f62c30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f74f000bf50 tx=0x7f74f000b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae44fc1eacf73c43 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:58.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.612+0000 7f74efe00640 1 -- 172.21.15.16:0/3616806921 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f0004220 con 0x564bc5b97390 2024-08-31T06:08:58.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.613+0000 7f74efe00640 1 -- 172.21.15.16:0/3616806921 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x564bc5ed66c0 con 0x564bc5b97390 2024-08-31T06:08:58.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.613+0000 7f74efe00640 1 -- 172.21.15.16:0/3616806921 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f0004cd0 con 0x564bc5b97390 2024-08-31T06:08:58.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.614+0000 7f75116aa740 1 -- 172.21.15.16:0/3616806921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bc5b97390 msgr2=0x564bc5f62c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:58.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.614+0000 7f75116aa740 1 --2- 172.21.15.16:0/3616806921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bc5b97390 0x564bc5f62c30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f74f000bf50 tx=0x7f74f000b390 comp rx=0 tx=0).stop 2024-08-31T06:08:58.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.614+0000 7f75116aa740 1 -- 172.21.15.16:0/3616806921 shutdown_connections 2024-08-31T06:08:58.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.614+0000 7f75116aa740 1 --2- 172.21.15.16:0/3616806921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc5bb8070 0x564bc5f63170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:58.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.614+0000 7f75116aa740 1 --2- 172.21.15.16:0/3616806921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bc5b97390 0x564bc5f62c30 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:58.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.614+0000 7f75116aa740 1 --2- 172.21.15.16:0/3616806921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bc5b8f1b0 0x564bc5f6f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:58.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.615+0000 7f75116aa740 1 -- 172.21.15.16:0/3616806921 >> 172.21.15.16:0/3616806921 conn(0x564bc5bac460 msgr2=0x564bc5f67e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:58.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.615+0000 7f75116aa740 1 -- 172.21.15.16:0/3616806921 shutdown_connections 2024-08-31T06:08:58.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.615+0000 7f75116aa740 1 -- 172.21.15.16:0/3616806921 wait complete. 2024-08-31T06:08:58.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.615+0000 7f75116aa740 1 -- start start 2024-08-31T06:08:58.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.616+0000 7f75116aa740 1 Processor -- start 2024-08-31T06:08:58.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.617+0000 7f75116aa740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bc5b8f1b0 0x564bc5f75910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:58.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.617+0000 7f75116aa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc5b97390 0x564bc5f71dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:58.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.617+0000 7f75116aa740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bc5bb8070 0x564bc5f72310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:58.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.617+0000 7f75116aa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f75040735c0 con 0x564bc5b97390 2024-08-31T06:08:58.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.617+0000 7f75116aa740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7504074a30 con 0x564bc5bb8070 2024-08-31T06:08:58.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.617+0000 7f75116aa740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7504073c90 con 0x564bc5b8f1b0 2024-08-31T06:08:58.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.617+0000 7f75116aa740 10 client.0 _ll_register_callbacks cb 0x7f74f40c8390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:08:58.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.618+0000 7f7503e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bc5b8f1b0 0x564bc5f75910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:58.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.618+0000 7f7503e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bc5b8f1b0 0x564bc5f75910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:55386/0 (socket says 172.21.15.16:55386) 2024-08-31T06:08:58.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.618+0000 7f7503e00640 1 -- 172.21.15.16:0/3896799156 learned_addr learned my addr 172.21.15.16:0/3896799156 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:08:58.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.618+0000 7f7503400640 1 --2- 172.21.15.16:0/3896799156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc5b97390 0x564bc5f71dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:58.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.618+0000 7f7508a00640 1 --2- 172.21.15.16:0/3896799156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bc5bb8070 0x564bc5f72310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:58.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.619+0000 7f7503400640 1 -- 172.21.15.16:0/3896799156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bc5b8f1b0 msgr2=0x564bc5f75910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:58.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.619+0000 7f7503400640 1 --2- 172.21.15.16:0/3896799156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bc5b8f1b0 0x564bc5f75910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:58.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.619+0000 7f7503400640 1 -- 172.21.15.16:0/3896799156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bc5bb8070 msgr2=0x564bc5f72310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:08:58.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.619+0000 7f7503400640 1 --2- 172.21.15.16:0/3896799156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bc5bb8070 0x564bc5f72310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:58.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.619+0000 7f7503400640 1 -- 172.21.15.16:0/3896799156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bc5f72a00 con 0x564bc5b97390 2024-08-31T06:08:58.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.619+0000 7f7503e00640 1 --2- 172.21.15.16:0/3896799156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bc5b8f1b0 0x564bc5f75910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:08:58.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.620+0000 7f7503400640 1 --2- 172.21.15.16:0/3896799156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc5b97390 0x564bc5f71dd0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f74f0000c00 tx=0x7f74f0005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b2c7763fbf50935 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:08:58.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.620+0000 7f74ed600640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:08:58.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.620+0000 7f74ed600640 1 -- 172.21.15.16:0/3896799156 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f0033060 con 0x564bc5b97390 2024-08-31T06:08:58.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.621+0000 7f74ed600640 1 -- 172.21.15.16:0/3896799156 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f74f0004c60 con 0x564bc5b97390 2024-08-31T06:08:58.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.621+0000 7f75116aa740 1 -- 172.21.15.16:0/3896799156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564bc5ed66c0 con 0x564bc5b97390 2024-08-31T06:08:58.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.621+0000 7f74ed600640 1 -- 172.21.15.16:0/3896799156 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f74f0004e60 con 0x564bc5b97390 2024-08-31T06:08:58.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.621+0000 7f75116aa740 10 client.15906 fetch_fsmap learned FSMap version 31 2024-08-31T06:08:58.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.621+0000 7f75116aa740 1 -- 172.21.15.16:0/3896799156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564bc5ff7af0 con 0x564bc5b97390 2024-08-31T06:08:58.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.622+0000 7f74ed600640 1 -- 172.21.15.16:0/3896799156 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x564bc5ed66c0 con 0x564bc5b97390 2024-08-31T06:08:58.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.622+0000 7f74ed600640 1 -- 172.21.15.16:0/3896799156 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f74f0010bd0 con 0x564bc5b97390 2024-08-31T06:08:58.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.622+0000 7f75116aa740 10 client.15906 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:08:58.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.622+0000 7f75116aa740 10 client.15906 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:08:58.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.622+0000 7f75116aa740 20 client.15906 populate_metadata read hostname 'smithi016' 2024-08-31T06:08:58.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.622+0000 7f75116aa740 1 --2- 172.21.15.16:0/3896799156 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564bc5b4bda0 0x564bc5ffa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:08:58.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.622+0000 7f75116aa740 4 client.15906 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T06:08:58.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.622+0000 7f75116aa740 1 -- 172.21.15.16:0/3896799156 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564bc5ed2160 con 0x564bc5b4bda0 2024-08-31T06:08:58.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.623+0000 7f7503e00640 1 --2- 172.21.15.16:0/3896799156 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564bc5b4bda0 0x564bc5ffa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:08:58.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.623+0000 7f74ed600640 10 client.15906 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:58.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.623+0000 7f7503e00640 1 --2- 172.21.15.16:0/3896799156 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564bc5b4bda0 0x564bc5ffa1e0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5eb16146a40edb41 server_cookie=94c385de8e07f1f4 in_seq=0 out_seq=0 2024-08-31T06:08:58.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.624+0000 7f74ed600640 1 -- 172.21.15.16:0/3896799156 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+942 (crc 0 0 0) 0x564bc5ed2160 con 0x564bc5b4bda0 2024-08-31T06:08:58.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.624+0000 7f74ed600640 10 client.15906 handle_command_reply: tid=0 2024-08-31T06:08:58.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.624+0000 7f75116aa740 1 client.15906 shutdown 2024-08-31T06:08:58.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.624+0000 7f75116aa740 1 -- 172.21.15.16:0/3896799156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc5b97390 msgr2=0x564bc5f71dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:58.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.624+0000 7f75116aa740 1 --2- 172.21.15.16:0/3896799156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc5b97390 0x564bc5f71dd0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f74f0000c00 tx=0x7f74f0005e50 comp rx=0 tx=0).stop 2024-08-31T06:08:58.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.624+0000 7f75116aa740 1 -- 172.21.15.16:0/3896799156 shutdown_connections 2024-08-31T06:08:58.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.624+0000 7f75116aa740 1 --2- 172.21.15.16:0/3896799156 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564bc5b4bda0 0x564bc5ffa1e0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:58.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.624+0000 7f75116aa740 1 --2- 172.21.15.16:0/3896799156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bc5bb8070 0x564bc5f72310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:58.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.624+0000 7f75116aa740 1 --2- 172.21.15.16:0/3896799156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc5b97390 0x564bc5f71dd0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:58.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.624+0000 7f75116aa740 1 --2- 172.21.15.16:0/3896799156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bc5b8f1b0 0x564bc5f75910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:58.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.624+0000 7f75116aa740 1 -- 172.21.15.16:0/3896799156 >> 172.21.15.16:0/3896799156 conn(0x564bc5bac460 msgr2=0x564bc5f6aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:58.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.624+0000 7f74ed600640 0 client.15906 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:08:58.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.625+0000 7f75116aa740 1 -- 172.21.15.16:0/3896799156 shutdown_connections 2024-08-31T06:08:58.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.625+0000 7f75116aa740 1 -- 172.21.15.16:0/3896799156 wait complete. 2024-08-31T06:08:58.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.625+0000 7f75116aa740 20 client.15906 trim_cache size 0 max 16384 2024-08-31T06:08:58.629 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:08:58.629 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-08-31T06:08:58.629 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {}, 2024-08-31T06:08:58.629 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-08-31T06:08:58.629 INFO:teuthology.orchestra.run.smithi016.stdout: "c8455d74-2fe7-4d93-bcf2-19b419d16016": { 2024-08-31T06:08:58.629 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:08:24.035960+0000", 2024-08-31T06:08:58.629 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:08:58.629 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:08:58.629 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:08:58.629 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:08:58.629 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:08:58.629 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:08:58.630 INFO:teuthology.orchestra.run.smithi016.stdout: "88d7bdb8-cb1c-45e5-ad29-4cd8da6c6003": { 2024-08-31T06:08:58.630 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:07:39.648942+0000", 2024-08-31T06:08:58.630 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:08:58.630 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:08:58.630 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:08:58.630 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:08:58.630 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:08:58.630 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:08:58.630 INFO:teuthology.orchestra.run.smithi016.stdout: "c300b7e8-f3fd-482c-a909-9329bc61e570": { 2024-08-31T06:08:58.630 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:06:49.868046+0000", 2024-08-31T06:08:58.630 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:08:58.630 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:08:58.630 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:08:58.630 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:08:58.631 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:08:58.631 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:08:58.631 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:08:58.631 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:08:58.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.626+0000 7f7509400640 1 -- 172.21.15.16:0/2950561857 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f74e80b4e70 msgr2=0x7f74e80b7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:58.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.626+0000 7f7509400640 1 --2- 172.21.15.16:0/2950561857 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f74e80b4e70 0x7f74e80b7380 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f74fc010890 tx=0x7f74fc007520 comp rx=0 tx=0).stop 2024-08-31T06:08:58.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.626+0000 7f7509400640 1 -- 172.21.15.16:0/2950561857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7504075b70 msgr2=0x7f7504113940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:08:58.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.626+0000 7f7509400640 1 --2- 172.21.15.16:0/2950561857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7504075b70 0x7f7504113940 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f74f402aa00 tx=0x7f74f4003cd0 comp rx=0 tx=0).stop 2024-08-31T06:08:58.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.626+0000 7f7509400640 1 -- 172.21.15.16:0/2950561857 shutdown_connections 2024-08-31T06:08:58.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.626+0000 7f7509400640 1 --2- 172.21.15.16:0/2950561857 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f74e80b4e70 0x7f74e80b7380 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:58.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.626+0000 7f7509400640 1 --2- 172.21.15.16:0/2950561857 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f750410d1b0 0x7f75041124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:58.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.626+0000 7f7509400640 1 --2- 172.21.15.16:0/2950561857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7504114020 0x7f750410cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:58.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.626+0000 7f7509400640 1 --2- 172.21.15.16:0/2950561857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7504075b70 0x7f7504113940 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:08:58.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.626+0000 7f7509400640 1 -- 172.21.15.16:0/2950561857 >> 172.21.15.16:0/2950561857 conn(0x7f75040ff1e0 msgr2=0x7f7504103180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:08:58.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.629+0000 7f7509400640 1 -- 172.21.15.16:0/2950561857 shutdown_connections 2024-08-31T06:08:58.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:08:58.629+0000 7f7509400640 1 -- 172.21.15.16:0/2950561857 wait complete. 2024-08-31T06:08:58.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:08:58 smithi155 ceph-mon[21742]: pgmap v416: 97 pgs: 97 active+clean; 1.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.1 MiB/s wr, 693 op/s 2024-08-31T06:08:58.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:08:58 smithi016 ceph-mon[17194]: pgmap v416: 97 pgs: 97 active+clean; 1.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.1 MiB/s wr, 693 op/s 2024-08-31T06:08:58.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:08:58 smithi184 ceph-mon[22331]: pgmap v416: 97 pgs: 97 active+clean; 1.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.1 MiB/s wr, 693 op/s 2024-08-31T06:08:59.680 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T06:08:59.680 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:09:00.313 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:09:00.822 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:00 smithi016 ceph-mon[17194]: pgmap v417: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.1 MiB/s wr, 784 op/s 2024-08-31T06:09:00.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:00 smithi155 ceph-mon[21742]: pgmap v417: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.1 MiB/s wr, 784 op/s 2024-08-31T06:09:00.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:00 smithi184 ceph-mon[22331]: pgmap v417: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.1 MiB/s wr, 784 op/s 2024-08-31T06:09:01.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.061+0000 7f790d800640 1 -- 172.21.15.16:0/2378679820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7908104750 msgr2=0x7f7908104b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:01.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.061+0000 7f790d800640 1 --2- 172.21.15.16:0/2378679820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7908104750 0x7f7908104b60 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f78f8009860 tx=0x7f78f802a340 comp rx=0 tx=0).stop 2024-08-31T06:09:01.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.062+0000 7f790d800640 1 -- 172.21.15.16:0/2378679820 shutdown_connections 2024-08-31T06:09:01.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.062+0000 7f790d800640 1 --2- 172.21.15.16:0/2378679820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f790810dbb0 0x7f79081100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:01.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.062+0000 7f790d800640 1 --2- 172.21.15.16:0/2378679820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79081050a0 0x7f790810d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:01.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.062+0000 7f790d800640 1 --2- 172.21.15.16:0/2378679820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7908104750 0x7f7908104b60 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:01.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.062+0000 7f790d800640 1 -- 172.21.15.16:0/2378679820 >> 172.21.15.16:0/2378679820 conn(0x7f79080794d0 msgr2=0x7f79080798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:01.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.062+0000 7f790d800640 1 -- 172.21.15.16:0/2378679820 shutdown_connections 2024-08-31T06:09:01.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.062+0000 7f790d800640 1 -- 172.21.15.16:0/2378679820 wait complete. 2024-08-31T06:09:01.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.064+0000 7f790d800640 1 Processor -- start 2024-08-31T06:09:01.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.064+0000 7f790d800640 1 -- start start 2024-08-31T06:09:01.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.065+0000 7f790d800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7908104750 0x7f790819c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:01.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.065+0000 7f790d800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79081050a0 0x7f790819cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:01.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.065+0000 7f78fea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7908104750 0x7f790819c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:01.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.065+0000 7f78fea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7908104750 0x7f790819c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:55416/0 (socket says 172.21.15.16:55416) 2024-08-31T06:09:01.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.066+0000 7f790d800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f790810dbb0 0x7f79081a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:01.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.066+0000 7f790d800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7908113830 con 0x7f79081050a0 2024-08-31T06:09:01.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.066+0000 7f790d800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f79081136b0 con 0x7f790810dbb0 2024-08-31T06:09:01.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.066+0000 7f790d800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f79081139b0 con 0x7f7908104750 2024-08-31T06:09:01.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.066+0000 7f78fea00640 1 -- 172.21.15.16:0/502232903 learned_addr learned my addr 172.21.15.16:0/502232903 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:01.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.066+0000 7f7906a00640 1 --2- 172.21.15.16:0/502232903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79081050a0 0x7f790819cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:01.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.067+0000 7f78fea00640 1 -- 172.21.15.16:0/502232903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f790810dbb0 msgr2=0x7f79081a1dc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-31T06:09:01.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.067+0000 7f78fea00640 1 --2- 172.21.15.16:0/502232903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f790810dbb0 0x7f79081a1dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:01.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.067+0000 7f78fea00640 1 -- 172.21.15.16:0/502232903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79081050a0 msgr2=0x7f790819cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:01.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.067+0000 7f78fea00640 1 --2- 172.21.15.16:0/502232903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79081050a0 0x7f790819cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:01.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.067+0000 7f78fea00640 1 -- 172.21.15.16:0/502232903 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79081a24f0 con 0x7f7908104750 2024-08-31T06:09:01.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.067+0000 7f7906a00640 1 --2- 172.21.15.16:0/502232903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79081050a0 0x7f790819cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:09:01.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.067+0000 7f78fea00640 1 --2- 172.21.15.16:0/502232903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7908104750 0x7f790819c580 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f78f8035040 tx=0x7f78f8032dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:01.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.068+0000 7f78ffe00640 1 -- 172.21.15.16:0/502232903 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f8037040 con 0x7f7908104750 2024-08-31T06:09:01.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.068+0000 7f78ffe00640 1 -- 172.21.15.16:0/502232903 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78f8041070 con 0x7f7908104750 2024-08-31T06:09:01.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.068+0000 7f78ffe00640 1 -- 172.21.15.16:0/502232903 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f8003850 con 0x7f7908104750 2024-08-31T06:09:01.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.068+0000 7f790d800640 1 -- 172.21.15.16:0/502232903 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79081a2780 con 0x7f7908104750 2024-08-31T06:09:01.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.068+0000 7f790d800640 1 -- 172.21.15.16:0/502232903 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79081a2bb0 con 0x7f7908104750 2024-08-31T06:09:01.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.069+0000 7f790d800640 1 -- 172.21.15.16:0/502232903 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78e40052c0 con 0x7f7908104750 2024-08-31T06:09:01.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.076+0000 7f78ffe00640 1 -- 172.21.15.16:0/502232903 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f78f8030450 con 0x7f7908104750 2024-08-31T06:09:01.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.079+0000 7f78ffe00640 1 --2- 172.21.15.16:0/502232903 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f78ec0b4da0 0x7f78ec0b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:01.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.079+0000 7f78ffe00640 1 -- 172.21.15.16:0/502232903 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f78f8105320 con 0x7f7908104750 2024-08-31T06:09:01.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.079+0000 7f78ffe00640 1 -- 172.21.15.16:0/502232903 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f78f804a060 con 0x7f7908104750 2024-08-31T06:09:01.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.080+0000 7f7906a00640 1 --2- 172.21.15.16:0/502232903 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f78ec0b4da0 0x7f78ec0b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:01.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.082+0000 7f7906a00640 1 --2- 172.21.15.16:0/502232903 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f78ec0b4da0 0x7f78ec0b72b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f790810f950 tx=0x7f78f0009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.421+0000 7f790d800640 1 -- 172.21.15.16:0/502232903 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78e40055b0 con 0x7f7908104750 2024-08-31T06:09:01.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.422+0000 7f78ffe00640 1 -- 172.21.15.16:0/502232903 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11923 (secure 0 0 0) 0x7f78f80cc1a0 con 0x7f7908104750 2024-08-31T06:09:01.425 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:09:01.425 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":31,"btime":"2024-08-31T06:08:47:452652+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:08:47.004817+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24737,24724,24712]},"id":1}]} 2024-08-31T06:09:01.426 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 31 2024-08-31T06:09:01.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.426+0000 7f790d800640 1 -- 172.21.15.16:0/502232903 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f78ec0b4da0 msgr2=0x7f78ec0b72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:01.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.426+0000 7f790d800640 1 --2- 172.21.15.16:0/502232903 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f78ec0b4da0 0x7f78ec0b72b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f790810f950 tx=0x7f78f0009260 comp rx=0 tx=0).stop 2024-08-31T06:09:01.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.426+0000 7f790d800640 1 -- 172.21.15.16:0/502232903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7908104750 msgr2=0x7f790819c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:01.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.426+0000 7f790d800640 1 --2- 172.21.15.16:0/502232903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7908104750 0x7f790819c580 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f78f8035040 tx=0x7f78f8032dc0 comp rx=0 tx=0).stop 2024-08-31T06:09:01.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.426+0000 7f790d800640 1 -- 172.21.15.16:0/502232903 shutdown_connections 2024-08-31T06:09:01.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.426+0000 7f790d800640 1 --2- 172.21.15.16:0/502232903 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f78ec0b4da0 0x7f78ec0b72b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:01.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.426+0000 7f790d800640 1 --2- 172.21.15.16:0/502232903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f790810dbb0 0x7f79081a1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:01.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.426+0000 7f790d800640 1 --2- 172.21.15.16:0/502232903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79081050a0 0x7f790819cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:01.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.426+0000 7f790d800640 1 --2- 172.21.15.16:0/502232903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7908104750 0x7f790819c580 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:01.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.426+0000 7f790d800640 1 -- 172.21.15.16:0/502232903 >> 172.21.15.16:0/502232903 conn(0x7f79080794d0 msgr2=0x7f79080ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:01.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.427+0000 7f790d800640 1 -- 172.21.15.16:0/502232903 shutdown_connections 2024-08-31T06:09:01.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:01.427+0000 7f790d800640 1 -- 172.21.15.16:0/502232903 wait complete. 2024-08-31T06:09:01.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 95.0 s, 2092.0 tps, lat 46.637 ms stddev 63.391 2024-08-31T06:09:01.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:01 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/502232903' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:09:01.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:01 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/502232903' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:09:01.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:01 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/502232903' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:09:02.244 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T06:09:02.509 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:02 smithi016 ceph-mon[17194]: pgmap v418: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.4 MiB/s wr, 645 op/s 2024-08-31T06:09:02.744 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:09:02.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:02 smithi155 ceph-mon[21742]: pgmap v418: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.4 MiB/s wr, 645 op/s 2024-08-31T06:09:02.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:02 smithi184 ceph-mon[22331]: pgmap v418: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.4 MiB/s wr, 645 op/s 2024-08-31T06:09:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.458+0000 7fe9c2400640 1 -- 172.21.15.16:0/834878118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9bc10b2a0 msgr2=0x7fe9bc10b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.458+0000 7fe9c2400640 1 --2- 172.21.15.16:0/834878118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9bc10b2a0 0x7fe9bc10b6b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8009860 tx=0x7fe9a802a340 comp rx=0 tx=0).stop 2024-08-31T06:09:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.458+0000 7fe9c2400640 1 -- 172.21.15.16:0/834878118 shutdown_connections 2024-08-31T06:09:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.458+0000 7fe9c2400640 1 --2- 172.21.15.16:0/834878118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9bc10bbf0 0x7fe9bc112420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.458+0000 7fe9c2400640 1 --2- 172.21.15.16:0/834878118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9bc10b2a0 0x7fe9bc10b6b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:03.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.458+0000 7fe9c2400640 1 --2- 172.21.15.16:0/834878118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9bc105960 0x7fe9bc105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:03.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.458+0000 7fe9c2400640 1 -- 172.21.15.16:0/834878118 >> 172.21.15.16:0/834878118 conn(0x7fe9bc1013d0 msgr2=0x7fe9bc103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:03.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.458+0000 7fe9c2400640 1 -- 172.21.15.16:0/834878118 shutdown_connections 2024-08-31T06:09:03.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.458+0000 7fe9c2400640 1 -- 172.21.15.16:0/834878118 wait complete. 2024-08-31T06:09:03.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.460+0000 7fe9c2400640 1 Processor -- start 2024-08-31T06:09:03.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.460+0000 7fe9c2400640 1 -- start start 2024-08-31T06:09:03.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.460+0000 7fe9c2400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9bc105960 0x7fe9bc19c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:03.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.460+0000 7fe9c2400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9bc10b2a0 0x7fe9bc19c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:03.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.460+0000 7fe9c2400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9bc10bbf0 0x7fe9bc1a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:03.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.460+0000 7fe9c2400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe9bc1158b0 con 0x7fe9bc105960 2024-08-31T06:09:03.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.460+0000 7fe9c2400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe9bc115730 con 0x7fe9bc10b2a0 2024-08-31T06:09:03.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.460+0000 7fe9c2400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe9bc115a30 con 0x7fe9bc10bbf0 2024-08-31T06:09:03.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.461+0000 7fe9c1a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9bc10bbf0 0x7fe9bc1a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:03.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.461+0000 7fe9c1000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9bc105960 0x7fe9bc19c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:03.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.461+0000 7fe9c1a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9bc10bbf0 0x7fe9bc1a1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:55424/0 (socket says 172.21.15.16:55424) 2024-08-31T06:09:03.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.461+0000 7fe9c1a00640 1 -- 172.21.15.16:0/4230465036 learned_addr learned my addr 172.21.15.16:0/4230465036 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:03.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.461+0000 7fe9bbe00640 1 --2- 172.21.15.16:0/4230465036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9bc10b2a0 0x7fe9bc19c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:03.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.461+0000 7fe9c1a00640 1 -- 172.21.15.16:0/4230465036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9bc10b2a0 msgr2=0x7fe9bc19c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:03.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.461+0000 7fe9c1a00640 1 --2- 172.21.15.16:0/4230465036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9bc10b2a0 0x7fe9bc19c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:03.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.461+0000 7fe9c1a00640 1 -- 172.21.15.16:0/4230465036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9bc105960 msgr2=0x7fe9bc19c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:03.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.461+0000 7fe9c1a00640 1 --2- 172.21.15.16:0/4230465036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9bc105960 0x7fe9bc19c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:03.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.461+0000 7fe9c1a00640 1 -- 172.21.15.16:0/4230465036 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9bc1a21f0 con 0x7fe9bc10bbf0 2024-08-31T06:09:03.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.461+0000 7fe9c1000640 1 --2- 172.21.15.16:0/4230465036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9bc105960 0x7fe9bc19c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:09:03.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.462+0000 7fe9c1a00640 1 --2- 172.21.15.16:0/4230465036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9bc10bbf0 0x7fe9bc1a1ac0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe9b400eb10 tx=0x7fe9b400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:03.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.462+0000 7fe9b9600640 1 -- 172.21.15.16:0/4230465036 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b4004b20 con 0x7fe9bc10bbf0 2024-08-31T06:09:03.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.462+0000 7fe9b9600640 1 -- 172.21.15.16:0/4230465036 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9b4004cc0 con 0x7fe9bc10bbf0 2024-08-31T06:09:03.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.462+0000 7fe9b9600640 1 -- 172.21.15.16:0/4230465036 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b4010670 con 0x7fe9bc10bbf0 2024-08-31T06:09:03.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.462+0000 7fe9c2400640 1 -- 172.21.15.16:0/4230465036 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9bc1a24e0 con 0x7fe9bc10bbf0 2024-08-31T06:09:03.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.462+0000 7fe9c2400640 1 -- 172.21.15.16:0/4230465036 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9bc1a8c40 con 0x7fe9bc10bbf0 2024-08-31T06:09:03.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.466+0000 7fe9ca612740 1 Processor -- start 2024-08-31T06:09:03.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.466+0000 7fe9b9600640 1 -- 172.21.15.16:0/4230465036 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fe9b4005020 con 0x7fe9bc10bbf0 2024-08-31T06:09:03.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.466+0000 7fe9ca612740 1 -- start start 2024-08-31T06:09:03.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.467+0000 7fe9ca612740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565278f3a070 0x5652792dfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:03.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.467+0000 7fe9ca612740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565278f111b0 0x5652792dffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:03.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.467+0000 7fe9ca612740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565278f19390 0x5652792e04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:03.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.467+0000 7fe9ca612740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5652792567b0 con 0x565278f19390 2024-08-31T06:09:03.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.467+0000 7fe9ca612740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x565279245d40 con 0x565278f111b0 2024-08-31T06:09:03.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.467+0000 7fe9ca612740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x565279249cf0 con 0x565278f3a070 2024-08-31T06:09:03.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.467+0000 7fe9bbe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565278f3a070 0x5652792dfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:03.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.467+0000 7fe9bbe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565278f3a070 0x5652792dfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:55438/0 (socket says 172.21.15.16:55438) 2024-08-31T06:09:03.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.467+0000 7fe9bbe00640 1 -- 172.21.15.16:0/482419854 learned_addr learned my addr 172.21.15.16:0/482419854 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:03.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.467+0000 7fe9bbe00640 1 --2- 172.21.15.16:0/482419854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565278f19390 0x5652792e04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:03.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.467+0000 7fe9c1000640 1 --2- 172.21.15.16:0/482419854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565278f111b0 0x5652792dffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:03.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.468+0000 7fe9bbe00640 1 -- 172.21.15.16:0/482419854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565278f111b0 msgr2=0x5652792dffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:03.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.468+0000 7fe9bbe00640 1 --2- 172.21.15.16:0/482419854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565278f111b0 0x5652792dffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:03.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.468+0000 7fe9b9600640 1 --2- 172.21.15.16:0/4230465036 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe9a40b4d20 0x7fe9a40b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:03.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.468+0000 7fe9bbe00640 1 -- 172.21.15.16:0/482419854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565278f19390 msgr2=0x5652792e04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:03.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.468+0000 7fe9bbe00640 1 --2- 172.21.15.16:0/482419854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565278f19390 0x5652792e04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:03.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.468+0000 7fe9bbe00640 1 -- 172.21.15.16:0/482419854 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565278fa5650 con 0x565278f3a070 2024-08-31T06:09:03.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.468+0000 7fe9b9600640 1 -- 172.21.15.16:0/4230465036 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fe9b40e5a90 con 0x7fe9bc10bbf0 2024-08-31T06:09:03.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.469+0000 7fe9bbe00640 1 --2- 172.21.15.16:0/482419854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565278f3a070 0x5652792dfa70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe9a802a930 tx=0x7fe9a8005780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1eef71237856b80 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:03.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.469+0000 7fe9c1a00640 1 --2- 172.21.15.16:0/4230465036 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe9a40b4d20 0x7fe9a40b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:03.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.469+0000 7fe9ae000640 1 -- 172.21.15.16:0/482419854 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8037040 con 0x565278f3a070 2024-08-31T06:09:03.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.469+0000 7fe9ae000640 1 -- 172.21.15.16:0/482419854 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9a8030450 con 0x565278f3a070 2024-08-31T06:09:03.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.469+0000 7fe9ae000640 1 -- 172.21.15.16:0/482419854 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a80037f0 con 0x565278f3a070 2024-08-31T06:09:03.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.470+0000 7fe9ca612740 1 -- 172.21.15.16:0/482419854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565278f3a070 msgr2=0x5652792dfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:03.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.470+0000 7fe9c1a00640 1 --2- 172.21.15.16:0/4230465036 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe9a40b4d20 0x7fe9a40b7230 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe9b40a93f0 tx=0x7fe9b40cf040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:03.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.470+0000 7fe9ca612740 1 --2- 172.21.15.16:0/482419854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565278f3a070 0x5652792dfa70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe9a802a930 tx=0x7fe9a8005780 comp rx=0 tx=0).stop 2024-08-31T06:09:03.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.470+0000 7fe9ca612740 1 -- 172.21.15.16:0/482419854 shutdown_connections 2024-08-31T06:09:03.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.470+0000 7fe9ca612740 1 --2- 172.21.15.16:0/482419854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565278f19390 0x5652792e04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:03.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.470+0000 7fe9ca612740 1 --2- 172.21.15.16:0/482419854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565278f111b0 0x5652792dffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:03.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.470+0000 7fe9ca612740 1 --2- 172.21.15.16:0/482419854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565278f3a070 0x5652792dfa70 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:03.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.470+0000 7fe9ca612740 1 -- 172.21.15.16:0/482419854 >> 172.21.15.16:0/482419854 conn(0x565278f2e460 msgr2=0x5652792dd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:03.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.471+0000 7fe9ca612740 1 -- 172.21.15.16:0/482419854 shutdown_connections 2024-08-31T06:09:03.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.471+0000 7fe9ca612740 1 -- 172.21.15.16:0/482419854 wait complete. 2024-08-31T06:09:03.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.471+0000 7fe9ca612740 1 -- start start 2024-08-31T06:09:03.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.472+0000 7fe9ca612740 1 Processor -- start 2024-08-31T06:09:03.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.472+0000 7fe9ca612740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565278f111b0 0x5652792f3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:03.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.472+0000 7fe9ca612740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565278f19390 0x5652792f44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:03.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.472+0000 7fe9ca612740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565278f3a070 0x5652792ffc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:03.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.472+0000 7fe9ca612740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5652790a8700 con 0x565278f111b0 2024-08-31T06:09:03.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.472+0000 7fe9ca612740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5652791cccf0 con 0x565278f19390 2024-08-31T06:09:03.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.472+0000 7fe9ca612740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5652792f6920 con 0x565278f3a070 2024-08-31T06:09:03.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.472+0000 7fe9ca612740 10 client.0 _ll_register_callbacks cb 0x5652791f2470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:09:03.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.473+0000 7fe9c1000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565278f19390 0x5652792f44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:03.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.473+0000 7fe9c1000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565278f19390 0x5652792f44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:34450/0 (socket says 172.21.15.16:34450) 2024-08-31T06:09:03.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.473+0000 7fe9c1000640 1 -- 172.21.15.16:0/3514306268 learned_addr learned my addr 172.21.15.16:0/3514306268 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:03.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.473+0000 7fe9c1000640 1 -- 172.21.15.16:0/3514306268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565278f3a070 msgr2=0x5652792ffc50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:09:03.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.473+0000 7fe9c1000640 1 --2- 172.21.15.16:0/3514306268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565278f3a070 0x5652792ffc50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:03.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.473+0000 7fe9c1000640 1 -- 172.21.15.16:0/3514306268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565278f111b0 msgr2=0x5652792f3ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T06:09:03.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.473+0000 7fe9bbe00640 1 --2- 172.21.15.16:0/3514306268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565278f111b0 0x5652792f3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:03.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.473+0000 7fe9c1000640 1 --2- 172.21.15.16:0/3514306268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565278f111b0 0x5652792f3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:03.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.473+0000 7fe9c1000640 1 -- 172.21.15.16:0/3514306268 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56527920d8b0 con 0x565278f19390 2024-08-31T06:09:03.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.474+0000 7fe9c1000640 1 --2- 172.21.15.16:0/3514306268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565278f19390 0x5652792f44d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0012a10 tx=0x7fe9b0012ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ad5d53c2dedf62b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:03.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.474+0000 7fe9a2a00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:09:03.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.474+0000 7fe9a2a00640 1 -- 172.21.15.16:0/3514306268 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b000fa40 con 0x565278f19390 2024-08-31T06:09:03.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.474+0000 7fe9a2a00640 1 -- 172.21.15.16:0/3514306268 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fe9b000fbe0 con 0x565278f19390 2024-08-31T06:09:03.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.474+0000 7fe9ca612740 1 -- 172.21.15.16:0/3514306268 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9a8030450 con 0x565278f19390 2024-08-31T06:09:03.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.474+0000 7fe9a2a00640 1 -- 172.21.15.16:0/3514306268 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe9b000fde0 con 0x565278f19390 2024-08-31T06:09:03.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.474+0000 7fe9ca612740 10 client.25970 fetch_fsmap learned FSMap version 31 2024-08-31T06:09:03.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.474+0000 7fe9ca612740 1 -- 172.21.15.16:0/3514306268 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x565279302a80 con 0x565278f19390 2024-08-31T06:09:03.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.475+0000 7fe9a2a00640 1 -- 172.21.15.16:0/3514306268 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9a8030450 con 0x565278f19390 2024-08-31T06:09:03.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.475+0000 7fe9a2a00640 1 -- 172.21.15.16:0/3514306268 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fe9b001c420 con 0x565278f19390 2024-08-31T06:09:03.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.475+0000 7fe9ca612740 10 client.25970 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:09:03.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.475+0000 7fe9ca612740 10 client.25970 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T06:09:03.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.475+0000 7fe9ca612740 20 client.25970 populate_metadata read hostname 'smithi016' 2024-08-31T06:09:03.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.475+0000 7fe9ca612740 1 --2- 172.21.15.16:0/3514306268 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x565278ef89f0 0x565279306010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:03.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.475+0000 7fe9ca612740 4 client.25970 mds_command: new command op to 24712 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:09:03.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.475+0000 7fe9ca612740 1 -- 172.21.15.16:0/3514306268 --> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565279258700 con 0x565278ef89f0 2024-08-31T06:09:03.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.476+0000 7fe9bbe00640 1 --2- 172.21.15.16:0/3514306268 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x565278ef89f0 0x565279306010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:03.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.476+0000 7fe9bbe00640 1 --2- 172.21.15.16:0/3514306268 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x565278ef89f0 0x565279306010 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a2e20968c02bb19 server_cookie=bbbf8e1dcb1b3c72 in_seq=0 out_seq=0 2024-08-31T06:09:03.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.476+0000 7fe9a2a00640 10 client.25970 ms_handle_connect on v2:172.21.15.184:6838/4011051479 2024-08-31T06:09:03.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.477+0000 7fe9a2a00640 1 -- 172.21.15.16:0/3514306268 <== mds.2 v2:172.21.15.184:6838/4011051479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x565279258700 con 0x565278ef89f0 2024-08-31T06:09:03.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.477+0000 7fe9a2a00640 10 client.25970 handle_command_reply: tid=0 2024-08-31T06:09:03.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.477+0000 7fe9ca612740 1 client.25970 shutdown 2024-08-31T06:09:03.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.478+0000 7fe9ca612740 1 -- 172.21.15.16:0/3514306268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565278f19390 msgr2=0x5652792f44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:03.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.478+0000 7fe9ca612740 1 --2- 172.21.15.16:0/3514306268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565278f19390 0x5652792f44d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0012a10 tx=0x7fe9b0012ed0 comp rx=0 tx=0).stop 2024-08-31T06:09:03.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.478+0000 7fe9ca612740 1 -- 172.21.15.16:0/3514306268 shutdown_connections 2024-08-31T06:09:03.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.478+0000 7fe9ca612740 1 --2- 172.21.15.16:0/3514306268 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x565278ef89f0 0x565279306010 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:03.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.478+0000 7fe9ca612740 1 --2- 172.21.15.16:0/3514306268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565278f3a070 0x5652792ffc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:03.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.478+0000 7fe9ca612740 1 --2- 172.21.15.16:0/3514306268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565278f19390 0x5652792f44d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:03.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.478+0000 7fe9ca612740 1 --2- 172.21.15.16:0/3514306268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565278f111b0 0x5652792f3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:03.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.478+0000 7fe9ca612740 1 -- 172.21.15.16:0/3514306268 >> 172.21.15.16:0/3514306268 conn(0x565278f2e460 msgr2=0x5652792dbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:03.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.478+0000 7fe9a2a00640 0 client.25970 ms_handle_reset on v2:172.21.15.184:6838/4011051479 2024-08-31T06:09:03.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.478+0000 7fe9ca612740 1 -- 172.21.15.16:0/3514306268 shutdown_connections 2024-08-31T06:09:03.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.478+0000 7fe9ca612740 1 -- 172.21.15.16:0/3514306268 wait complete. 2024-08-31T06:09:03.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.478+0000 7fe9ca612740 20 client.25970 trim_cache size 0 max 16384 2024-08-31T06:09:03.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.495+0000 7fe9ca612740 1 Processor -- start 2024-08-31T06:09:03.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.495+0000 7fe9ca612740 1 -- start start 2024-08-31T06:09:03.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.496+0000 7fe9ca612740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565278f111b0 0x5652792ffb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:03.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.496+0000 7fe9ca612740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565278f19390 0x5652792e4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:03.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.496+0000 7fe9ca612740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565278f3a070 0x5652792e5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:03.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.496+0000 7fe9ca612740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe9bc1af980 con 0x565278f111b0 2024-08-31T06:09:03.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.496+0000 7fe9ca612740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe9bc1ad190 con 0x565278f19390 2024-08-31T06:09:03.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.496+0000 7fe9ca612740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe9bc1aafe0 con 0x565278f3a070 2024-08-31T06:09:03.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.496+0000 7fe9c1000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565278f19390 0x5652792e4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:03.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.496+0000 7fe9c1000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565278f19390 0x5652792e4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:34464/0 (socket says 172.21.15.16:34464) 2024-08-31T06:09:03.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.496+0000 7fe9c1000640 1 -- 172.21.15.16:0/1363496501 learned_addr learned my addr 172.21.15.16:0/1363496501 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:03.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.496+0000 7fe9bbe00640 1 --2- 172.21.15.16:0/1363496501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565278f111b0 0x5652792ffb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:03.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.497+0000 7fe9c1000640 1 -- 172.21.15.16:0/1363496501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565278f3a070 msgr2=0x5652792e5070 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T06:09:03.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.497+0000 7fe9c1000640 1 --2- 172.21.15.16:0/1363496501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565278f3a070 0x5652792e5070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:03.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.497+0000 7fe9c1000640 1 -- 172.21.15.16:0/1363496501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565278f111b0 msgr2=0x5652792ffb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:03.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.497+0000 7fe9c1000640 1 --2- 172.21.15.16:0/1363496501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565278f111b0 0x5652792ffb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:03.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.497+0000 7fe9c1000640 1 -- 172.21.15.16:0/1363496501 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652792e2880 con 0x565278f19390 2024-08-31T06:09:03.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.497+0000 7fe9c1000640 1 --2- 172.21.15.16:0/1363496501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565278f19390 0x5652792e4b30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0000c00 tx=0x7fe9b000fcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ff2c02db484e2c9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:03.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.497+0000 7fe9ad600640 1 -- 172.21.15.16:0/1363496501 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b0029030 con 0x565278f19390 2024-08-31T06:09:03.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.497+0000 7fe9ad600640 1 -- 172.21.15.16:0/1363496501 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9b001cb70 con 0x565278f19390 2024-08-31T06:09:03.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.497+0000 7fe9ad600640 1 -- 172.21.15.16:0/1363496501 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b0004660 con 0x565278f19390 2024-08-31T06:09:03.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.498+0000 7fe9ca612740 1 -- 172.21.15.16:0/1363496501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565278f19390 msgr2=0x5652792e4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:03.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.498+0000 7fe9ca612740 1 --2- 172.21.15.16:0/1363496501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565278f19390 0x5652792e4b30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0000c00 tx=0x7fe9b000fcc0 comp rx=0 tx=0).stop 2024-08-31T06:09:03.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.499+0000 7fe9ca612740 1 -- 172.21.15.16:0/1363496501 shutdown_connections 2024-08-31T06:09:03.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.499+0000 7fe9ca612740 1 --2- 172.21.15.16:0/1363496501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565278f3a070 0x5652792e5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:03.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.499+0000 7fe9ca612740 1 --2- 172.21.15.16:0/1363496501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565278f19390 0x5652792e4b30 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:03.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.499+0000 7fe9ca612740 1 --2- 172.21.15.16:0/1363496501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565278f111b0 0x5652792ffb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:03.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.499+0000 7fe9ca612740 1 -- 172.21.15.16:0/1363496501 >> 172.21.15.16:0/1363496501 conn(0x565278f2e460 msgr2=0x5652792dbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:03.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.499+0000 7fe9ca612740 1 -- 172.21.15.16:0/1363496501 shutdown_connections 2024-08-31T06:09:03.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.499+0000 7fe9ca612740 1 -- 172.21.15.16:0/1363496501 wait complete. 2024-08-31T06:09:03.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.499+0000 7fe9ca612740 1 -- start start 2024-08-31T06:09:03.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.500+0000 7fe9ca612740 1 Processor -- start 2024-08-31T06:09:03.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.500+0000 7fe9ca612740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565278f111b0 0x5652792f82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:03.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.500+0000 7fe9ca612740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565278f19390 0x5652792f8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:03.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.500+0000 7fe9ca612740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565278f3a070 0x56527936fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:03.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.500+0000 7fe9ca612740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe9bc1ab650 con 0x565278f3a070 2024-08-31T06:09:03.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.500+0000 7fe9ca612740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe9bc1acac0 con 0x565278f19390 2024-08-31T06:09:03.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.500+0000 7fe9ca612740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe9bc1abd20 con 0x565278f111b0 2024-08-31T06:09:03.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.500+0000 7fe9ca612740 10 client.0 _ll_register_callbacks cb 0x7fe9b0012620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:09:03.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.501+0000 7fe9bbe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565278f111b0 0x5652792f82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:03.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.501+0000 7fe9bbe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565278f111b0 0x5652792f82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:55458/0 (socket says 172.21.15.16:55458) 2024-08-31T06:09:03.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.501+0000 7fe9bbe00640 1 -- 172.21.15.16:0/893940565 learned_addr learned my addr 172.21.15.16:0/893940565 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:03.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.501+0000 7fe9c1000640 1 --2- 172.21.15.16:0/893940565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565278f19390 0x5652792f8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:03.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.501+0000 7fe9bbe00640 1 --2- 172.21.15.16:0/893940565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565278f3a070 0x56527936fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:03.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.502+0000 7fe9bbe00640 1 -- 172.21.15.16:0/893940565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565278f111b0 msgr2=0x5652792f82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:03.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.503+0000 7fe9bbe00640 1 --2- 172.21.15.16:0/893940565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565278f111b0 0x5652792f82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:03.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.503+0000 7fe9bbe00640 1 -- 172.21.15.16:0/893940565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565278f19390 msgr2=0x5652792f8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:03.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.503+0000 7fe9bbe00640 1 --2- 172.21.15.16:0/893940565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565278f19390 0x5652792f8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:03.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.503+0000 7fe9bbe00640 1 -- 172.21.15.16:0/893940565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56527936ff80 con 0x565278f3a070 2024-08-31T06:09:03.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.504+0000 7fe9bbe00640 1 --2- 172.21.15.16:0/893940565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565278f3a070 0x56527936fa10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x5652792ef5f0 tx=0x7fe9a8005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b80e90fe286aef0e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:03.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.504+0000 7fe9a2a00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:09:03.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.506+0000 7fe9a2a00640 1 -- 172.21.15.16:0/893940565 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8037040 con 0x565278f3a070 2024-08-31T06:09:03.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.506+0000 7fe9a2a00640 1 -- 172.21.15.16:0/893940565 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fe9a8004240 con 0x565278f3a070 2024-08-31T06:09:03.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.506+0000 7fe9ca612740 1 -- 172.21.15.16:0/893940565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9b001cb70 con 0x565278f3a070 2024-08-31T06:09:03.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.507+0000 7fe9a2a00640 1 -- 172.21.15.16:0/893940565 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe9a8004440 con 0x565278f3a070 2024-08-31T06:09:03.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.507+0000 7fe9ca612740 10 client.15927 fetch_fsmap learned FSMap version 31 2024-08-31T06:09:03.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.507+0000 7fe9ca612740 1 -- 172.21.15.16:0/893940565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x565279371200 con 0x565278f3a070 2024-08-31T06:09:03.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.508+0000 7fe9a2a00640 1 -- 172.21.15.16:0/893940565 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9b001cb70 con 0x565278f3a070 2024-08-31T06:09:03.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.508+0000 7fe9a2a00640 1 -- 172.21.15.16:0/893940565 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fe9a8051070 con 0x565278f3a070 2024-08-31T06:09:03.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.508+0000 7fe9ca612740 10 client.15927 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:09:03.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.508+0000 7fe9ca612740 10 client.15927 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T06:09:03.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.508+0000 7fe9ca612740 20 client.15927 populate_metadata read hostname 'smithi016' 2024-08-31T06:09:03.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.508+0000 7fe9ca612740 1 --2- 172.21.15.16:0/893940565 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x565278ecdda0 0x565279373490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:03.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.508+0000 7fe9ca612740 4 client.15927 mds_command: new command op to 24712 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:09:03.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.508+0000 7fe9ca612740 1 -- 172.21.15.16:0/893940565 --> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5652792541a0 con 0x565278ecdda0 2024-08-31T06:09:03.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.509+0000 7fe9c1000640 1 --2- 172.21.15.16:0/893940565 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x565278ecdda0 0x565279373490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:03.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.509+0000 7fe9c1000640 1 --2- 172.21.15.16:0/893940565 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x565278ecdda0 0x565279373490 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a92e32920b3e0eee server_cookie=c10966cb1c54ee2d in_seq=0 out_seq=0 2024-08-31T06:09:03.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.509+0000 7fe9a2a00640 10 client.15927 ms_handle_connect on v2:172.21.15.184:6838/4011051479 2024-08-31T06:09:03.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.510+0000 7fe9a2a00640 1 -- 172.21.15.16:0/893940565 <== mds.2 v2:172.21.15.184:6838/4011051479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5652792541a0 con 0x565278ecdda0 2024-08-31T06:09:03.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.510+0000 7fe9a2a00640 10 client.15927 handle_command_reply: tid=0 2024-08-31T06:09:03.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.510+0000 7fe9ca612740 1 client.15927 shutdown 2024-08-31T06:09:03.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.511+0000 7fe9ca612740 1 -- 172.21.15.16:0/893940565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565278f3a070 msgr2=0x56527936fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:03.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.511+0000 7fe9ca612740 1 --2- 172.21.15.16:0/893940565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565278f3a070 0x56527936fa10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x5652792ef5f0 tx=0x7fe9a8005e10 comp rx=0 tx=0).stop 2024-08-31T06:09:03.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.512+0000 7fe9ca612740 1 -- 172.21.15.16:0/893940565 shutdown_connections 2024-08-31T06:09:03.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.512+0000 7fe9ca612740 1 --2- 172.21.15.16:0/893940565 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x565278ecdda0 0x565279373490 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:03.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.512+0000 7fe9ca612740 1 --2- 172.21.15.16:0/893940565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565278f3a070 0x56527936fa10 secure :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0x5652792ef5f0 tx=0x7fe9a8005e10 comp rx=0 tx=0).stop 2024-08-31T06:09:03.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.512+0000 7fe9ca612740 1 --2- 172.21.15.16:0/893940565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565278f19390 0x5652792f8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:03.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.512+0000 7fe9ca612740 1 --2- 172.21.15.16:0/893940565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565278f111b0 0x5652792f82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:03.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.512+0000 7fe9a2a00640 0 client.15927 ms_handle_reset on v2:172.21.15.184:6838/4011051479 2024-08-31T06:09:03.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.512+0000 7fe9ca612740 1 -- 172.21.15.16:0/893940565 >> 172.21.15.16:0/893940565 conn(0x565278f2e460 msgr2=0x5652792e4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:03.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.513+0000 7fe9ca612740 1 -- 172.21.15.16:0/893940565 shutdown_connections 2024-08-31T06:09:03.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.513+0000 7fe9ca612740 1 -- 172.21.15.16:0/893940565 wait complete. 2024-08-31T06:09:03.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.514+0000 7fe9ca612740 20 client.15927 trim_cache size 0 max 16384 2024-08-31T06:09:03.517 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:09:03.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.515+0000 7fe9c2400640 1 -- 172.21.15.16:0/4230465036 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe9a40b4d20 msgr2=0x7fe9a40b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:03.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.515+0000 7fe9c2400640 1 --2- 172.21.15.16:0/4230465036 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe9a40b4d20 0x7fe9a40b7230 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe9b40a93f0 tx=0x7fe9b40cf040 comp rx=0 tx=0).stop 2024-08-31T06:09:03.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.515+0000 7fe9c2400640 1 -- 172.21.15.16:0/4230465036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9bc10bbf0 msgr2=0x7fe9bc1a1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:03.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.515+0000 7fe9c2400640 1 --2- 172.21.15.16:0/4230465036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9bc10bbf0 0x7fe9bc1a1ac0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe9b400eb10 tx=0x7fe9b400c2d0 comp rx=0 tx=0).stop 2024-08-31T06:09:03.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.515+0000 7fe9c2400640 1 -- 172.21.15.16:0/4230465036 shutdown_connections 2024-08-31T06:09:03.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.515+0000 7fe9c2400640 1 --2- 172.21.15.16:0/4230465036 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe9a40b4d20 0x7fe9a40b7230 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:03.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.515+0000 7fe9c2400640 1 --2- 172.21.15.16:0/4230465036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9bc10bbf0 0x7fe9bc1a1ac0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:03.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.515+0000 7fe9c2400640 1 --2- 172.21.15.16:0/4230465036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9bc10b2a0 0x7fe9bc19c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:03.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.515+0000 7fe9c2400640 1 --2- 172.21.15.16:0/4230465036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9bc105960 0x7fe9bc19c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:03.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.515+0000 7fe9c2400640 1 -- 172.21.15.16:0/4230465036 >> 172.21.15.16:0/4230465036 conn(0x7fe9bc1013d0 msgr2=0x7fe9bc1037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:03.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.515+0000 7fe9c2400640 1 -- 172.21.15.16:0/4230465036 shutdown_connections 2024-08-31T06:09:03.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:03.515+0000 7fe9c2400640 1 -- 172.21.15.16:0/4230465036 wait complete. 2024-08-31T06:09:04.431 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T06:09:04.636 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.633+0000 7fe8f0600640 1 -- 172.21.15.16:0/2642599252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8e810a340 msgr2=0x7fe8e810a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:04.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.633+0000 7fe8f0600640 1 --2- 172.21.15.16:0/2642599252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8e810a340 0x7fe8e810a750 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0009be0 tx=0x7fe8e002a6c0 comp rx=0 tx=0).stop 2024-08-31T06:09:04.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.634+0000 7fe8f0600640 1 -- 172.21.15.16:0/2642599252 shutdown_connections 2024-08-31T06:09:04.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.634+0000 7fe8f0600640 1 --2- 172.21.15.16:0/2642599252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8e8116340 0x7fe8e8118820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:04.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.634+0000 7fe8f0600640 1 --2- 172.21.15.16:0/2642599252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8e810ac90 0x7fe8e8115d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:04.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.634+0000 7fe8f0600640 1 --2- 172.21.15.16:0/2642599252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8e810a340 0x7fe8e810a750 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:04.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.634+0000 7fe8f0600640 1 -- 172.21.15.16:0/2642599252 >> 172.21.15.16:0/2642599252 conn(0x7fe8e8070b70 msgr2=0x7fe8e8070f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:04.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.634+0000 7fe8f0600640 1 -- 172.21.15.16:0/2642599252 shutdown_connections 2024-08-31T06:09:04.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.634+0000 7fe8f0600640 1 -- 172.21.15.16:0/2642599252 wait complete. 2024-08-31T06:09:04.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.635+0000 7fe8f0600640 1 Processor -- start 2024-08-31T06:09:04.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.635+0000 7fe8f0600640 1 -- start start 2024-08-31T06:09:04.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.636+0000 7fe8f0600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8e810a340 0x7fe8e81a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:04.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.636+0000 7fe8f0600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8e810ac90 0x7fe8e81a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.636+0000 7fe8f0600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8e8116340 0x7fe8e81aa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.636+0000 7fe8f0600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e811bfa0 con 0x7fe8e810ac90 2024-08-31T06:09:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.636+0000 7fe8f0600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e811be20 con 0x7fe8e8116340 2024-08-31T06:09:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.636+0000 7fe8f0600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e811c120 con 0x7fe8e810a340 2024-08-31T06:09:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.636+0000 7fe8edc00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8e810a340 0x7fe8e81a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.636+0000 7fe8edc00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8e810a340 0x7fe8e81a4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:55480/0 (socket says 172.21.15.16:55480) 2024-08-31T06:09:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.636+0000 7fe8edc00640 1 -- 172.21.15.16:0/4188831248 learned_addr learned my addr 172.21.15.16:0/4188831248 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:04.640 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.637+0000 7fe8ee600640 1 --2- 172.21.15.16:0/4188831248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8e8116340 0x7fe8e81aa4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:04.640 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.637+0000 7fe8ed200640 1 --2- 172.21.15.16:0/4188831248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8e810ac90 0x7fe8e81a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:04.640 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.637+0000 7fe8edc00640 1 -- 172.21.15.16:0/4188831248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8e8116340 msgr2=0x7fe8e81aa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.637+0000 7fe8edc00640 1 --2- 172.21.15.16:0/4188831248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8e8116340 0x7fe8e81aa4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.637+0000 7fe8edc00640 1 -- 172.21.15.16:0/4188831248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8e810ac90 msgr2=0x7fe8e81a5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.637+0000 7fe8edc00640 1 --2- 172.21.15.16:0/4188831248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8e810ac90 0x7fe8e81a5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.637+0000 7fe8edc00640 1 -- 172.21.15.16:0/4188831248 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e81aabf0 con 0x7fe8e810a340 2024-08-31T06:09:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.637+0000 7fe8ed200640 1 --2- 172.21.15.16:0/4188831248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8e810ac90 0x7fe8e81a5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:09:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.638+0000 7fe8edc00640 1 --2- 172.21.15.16:0/4188831248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8e810a340 0x7fe8e81a4cc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe8e002acb0 tx=0x7fe8e003dc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.638+0000 7fe8e6000640 1 -- 172.21.15.16:0/4188831248 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e004d070 con 0x7fe8e810a340 2024-08-31T06:09:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.638+0000 7fe8e6000640 1 -- 172.21.15.16:0/4188831248 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8e0003e20 con 0x7fe8e810a340 2024-08-31T06:09:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.638+0000 7fe8e6000640 1 -- 172.21.15.16:0/4188831248 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0030bd0 con 0x7fe8e810a340 2024-08-31T06:09:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.638+0000 7fe8f0600640 1 -- 172.21.15.16:0/4188831248 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8e81aae80 con 0x7fe8e810a340 2024-08-31T06:09:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.638+0000 7fe8f0600640 1 -- 172.21.15.16:0/4188831248 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8e81ab320 con 0x7fe8e810a340 2024-08-31T06:09:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.642+0000 7fe8e6000640 1 -- 172.21.15.16:0/4188831248 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fe8e0050070 con 0x7fe8e810a340 2024-08-31T06:09:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.642+0000 7fe8f7623740 1 Processor -- start 2024-08-31T06:09:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.643+0000 7fe8f7623740 1 -- start start 2024-08-31T06:09:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.643+0000 7fe8f7623740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958e6e71b0 0x55958ea8a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.643+0000 7fe8f7623740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958e6be2f0 0x55958ea8ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.643+0000 7fe8f7623740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958e6c64d0 0x55958ea8b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.643+0000 7fe8f7623740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55958e9fff70 con 0x55958e6e71b0 2024-08-31T06:09:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.643+0000 7fe8f7623740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55958e9ef500 con 0x55958e6c64d0 2024-08-31T06:09:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.643+0000 7fe8f7623740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55958e9f34b0 con 0x55958e6be2f0 2024-08-31T06:09:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.644+0000 7fe8ee600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958e6e71b0 0x55958ea8a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.644+0000 7fe8edc00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958e6be2f0 0x55958ea8ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.644+0000 7fe8ee600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958e6e71b0 0x55958ea8a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33970/0 (socket says 172.21.15.16:33970) 2024-08-31T06:09:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.644+0000 7fe8ee600640 1 -- 172.21.15.16:0/1186986187 learned_addr learned my addr 172.21.15.16:0/1186986187 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.644+0000 7fe8ed200640 1 --2- 172.21.15.16:0/1186986187 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958e6c64d0 0x55958ea8b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.644+0000 7fe8ee600640 1 -- 172.21.15.16:0/1186986187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958e6be2f0 msgr2=0x55958ea8ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.644+0000 7fe8ee600640 1 --2- 172.21.15.16:0/1186986187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958e6be2f0 0x55958ea8ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.644+0000 7fe8ee600640 1 -- 172.21.15.16:0/1186986187 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958e6c64d0 msgr2=0x55958ea8b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.644+0000 7fe8ee600640 1 --2- 172.21.15.16:0/1186986187 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958e6c64d0 0x55958ea8b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.644+0000 7fe8ee600640 1 -- 172.21.15.16:0/1186986187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55958e9a9c10 con 0x55958e6e71b0 2024-08-31T06:09:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.645+0000 7fe8ee600640 1 --2- 172.21.15.16:0/1186986187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958e6e71b0 0x55958ea8a920 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc009be0 tx=0x7fe8dc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5c8bda6a515bcdf server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.645+0000 7fe8e6000640 1 --2- 172.21.15.16:0/4188831248 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe8d00b4d20 0x7fe8d00b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.645+0000 7fe8daa00640 1 -- 172.21.15.16:0/1186986187 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc01e070 con 0x55958e6e71b0 2024-08-31T06:09:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.645+0000 7fe8e6000640 1 -- 172.21.15.16:0/4188831248 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fe8e010d920 con 0x7fe8e810a340 2024-08-31T06:09:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.645+0000 7fe8daa00640 1 -- 172.21.15.16:0/1186986187 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8dc009e50 con 0x55958e6e71b0 2024-08-31T06:09:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.645+0000 7fe8daa00640 1 -- 172.21.15.16:0/1186986187 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc00e930 con 0x55958e6e71b0 2024-08-31T06:09:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.646+0000 7fe8edc00640 1 --2- 172.21.15.16:0/4188831248 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe8d00b4d20 0x7fe8d00b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.646+0000 7fe8f7623740 1 -- 172.21.15.16:0/1186986187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958e6e71b0 msgr2=0x55958ea8a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.646+0000 7fe8f7623740 1 --2- 172.21.15.16:0/1186986187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958e6e71b0 0x55958ea8a920 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc009be0 tx=0x7fe8dc011810 comp rx=0 tx=0).stop 2024-08-31T06:09:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.647+0000 7fe8edc00640 1 --2- 172.21.15.16:0/4188831248 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe8d00b4d20 0x7fe8d00b7230 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x55958ea895a0 tx=0x7fe8e003d4b0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.647+0000 7fe8f7623740 1 -- 172.21.15.16:0/1186986187 shutdown_connections 2024-08-31T06:09:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.647+0000 7fe8f7623740 1 --2- 172.21.15.16:0/1186986187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958e6be2f0 0x55958ea8ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.647+0000 7fe8f7623740 1 --2- 172.21.15.16:0/1186986187 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958e6c64d0 0x55958ea8b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.647+0000 7fe8f7623740 1 --2- 172.21.15.16:0/1186986187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958e6e71b0 0x55958ea8a920 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.647+0000 7fe8f7623740 1 -- 172.21.15.16:0/1186986187 >> 172.21.15.16:0/1186986187 conn(0x55958e6db300 msgr2=0x55958ea88810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.647+0000 7fe8f7623740 1 -- 172.21.15.16:0/1186986187 shutdown_connections 2024-08-31T06:09:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.647+0000 7fe8f7623740 1 -- 172.21.15.16:0/1186986187 wait complete. 2024-08-31T06:09:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.647+0000 7fe8f7623740 1 -- start start 2024-08-31T06:09:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.648+0000 7fe8f7623740 1 Processor -- start 2024-08-31T06:09:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.648+0000 7fe8f7623740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958e6be2f0 0x55958ea95f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.648+0000 7fe8f7623740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958e6c64d0 0x55958ea96490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.648+0000 7fe8f7623740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958e6e71b0 0x55958ea96a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.648+0000 7fe8f7623740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55958e928ab0 con 0x55958e6c64d0 2024-08-31T06:09:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.648+0000 7fe8f7623740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55958e996320 con 0x55958e6e71b0 2024-08-31T06:09:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.648+0000 7fe8f7623740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55958eaaa0b0 con 0x55958e6be2f0 2024-08-31T06:09:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.648+0000 7fe8f7623740 10 client.0 _ll_register_callbacks cb 0x55958e99c6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:09:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.649+0000 7fe8ee600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958e6be2f0 0x55958ea95f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.649+0000 7fe8ee600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958e6be2f0 0x55958ea95f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:55496/0 (socket says 172.21.15.16:55496) 2024-08-31T06:09:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.649+0000 7fe8ee600640 1 -- 172.21.15.16:0/3678930994 learned_addr learned my addr 172.21.15.16:0/3678930994 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.649+0000 7fe8ee600640 1 --2- 172.21.15.16:0/3678930994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958e6e71b0 0x55958ea96a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.649+0000 7fe8ed200640 1 --2- 172.21.15.16:0/3678930994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958e6c64d0 0x55958ea96490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.649+0000 7fe8ee600640 1 -- 172.21.15.16:0/3678930994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958e6e71b0 msgr2=0x55958ea96a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.649+0000 7fe8ee600640 1 --2- 172.21.15.16:0/3678930994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958e6e71b0 0x55958ea96a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.649+0000 7fe8ee600640 1 -- 172.21.15.16:0/3678930994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958e6c64d0 msgr2=0x55958ea96490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.649+0000 7fe8ee600640 1 --2- 172.21.15.16:0/3678930994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958e6c64d0 0x55958ea96490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.649+0000 7fe8ee600640 1 -- 172.21.15.16:0/3678930994 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55958e752790 con 0x55958e6be2f0 2024-08-31T06:09:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.650+0000 7fe8ee600640 1 --2- 172.21.15.16:0/3678930994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958e6be2f0 0x55958ea95f50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc011d10 tx=0x7fe8dc00e590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fed148d5c0c86760 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.650+0000 7fe8cf400640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:09:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.650+0000 7fe8cf400640 1 -- 172.21.15.16:0/3678930994 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc01e070 con 0x55958e6be2f0 2024-08-31T06:09:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.650+0000 7fe8f7623740 1 -- 172.21.15.16:0/3678930994 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8dc009e50 con 0x55958e6be2f0 2024-08-31T06:09:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.650+0000 7fe8cf400640 1 -- 172.21.15.16:0/3678930994 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fe8dc00ed10 con 0x55958e6be2f0 2024-08-31T06:09:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.650+0000 7fe8cf400640 1 -- 172.21.15.16:0/3678930994 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe8dc037050 con 0x55958e6be2f0 2024-08-31T06:09:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.651+0000 7fe8f7623740 10 client.25981 fetch_fsmap learned FSMap version 31 2024-08-31T06:09:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.651+0000 7fe8f7623740 1 -- 172.21.15.16:0/3678930994 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55958e7cb620 con 0x55958e6be2f0 2024-08-31T06:09:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.651+0000 7fe8cf400640 1 -- 172.21.15.16:0/3678930994 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8dc009e50 con 0x55958e6be2f0 2024-08-31T06:09:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.652+0000 7fe8cf400640 1 -- 172.21.15.16:0/3678930994 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fe8dc0194c0 con 0x55958e6be2f0 2024-08-31T06:09:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.652+0000 7fe8f7623740 10 client.25981 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:09:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.652+0000 7fe8f7623740 10 client.25981 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:09:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.652+0000 7fe8f7623740 20 client.25981 populate_metadata read hostname 'smithi016' 2024-08-31T06:09:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.652+0000 7fe8f7623740 1 --2- 172.21.15.16:0/3678930994 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55958e6a5d80 0x55958eab8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.652+0000 7fe8f7623740 4 client.25981 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:09:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.652+0000 7fe8f7623740 1 -- 172.21.15.16:0/3678930994 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55958ea01ec0 con 0x55958e6a5d80 2024-08-31T06:09:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.653+0000 7fe8ed200640 1 --2- 172.21.15.16:0/3678930994 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55958e6a5d80 0x55958eab8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.654+0000 7fe8ed200640 1 --2- 172.21.15.16:0/3678930994 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55958e6a5d80 0x55958eab8fa0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf2cc105a9b8af06 server_cookie=7b548834e8244512 in_seq=0 out_seq=0 2024-08-31T06:09:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.654+0000 7fe8cf400640 10 client.25981 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:09:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.655+0000 7fe8cf400640 1 -- 172.21.15.16:0/3678930994 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55958ea01ec0 con 0x55958e6a5d80 2024-08-31T06:09:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.655+0000 7fe8cf400640 10 client.25981 handle_command_reply: tid=0 2024-08-31T06:09:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.655+0000 7fe8f7623740 1 client.25981 shutdown 2024-08-31T06:09:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.655+0000 7fe8f7623740 1 -- 172.21.15.16:0/3678930994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958e6be2f0 msgr2=0x55958ea95f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.655+0000 7fe8f7623740 1 --2- 172.21.15.16:0/3678930994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958e6be2f0 0x55958ea95f50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc011d10 tx=0x7fe8dc00e590 comp rx=0 tx=0).stop 2024-08-31T06:09:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.655+0000 7fe8f7623740 1 -- 172.21.15.16:0/3678930994 shutdown_connections 2024-08-31T06:09:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.655+0000 7fe8f7623740 1 --2- 172.21.15.16:0/3678930994 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55958e6a5d80 0x55958eab8fa0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.655+0000 7fe8f7623740 1 --2- 172.21.15.16:0/3678930994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958e6e71b0 0x55958ea96a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.655+0000 7fe8f7623740 1 --2- 172.21.15.16:0/3678930994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958e6c64d0 0x55958ea96490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.655+0000 7fe8cf400640 0 client.25981 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:09:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.655+0000 7fe8f7623740 1 --2- 172.21.15.16:0/3678930994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958e6be2f0 0x55958ea95f50 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.655+0000 7fe8f7623740 1 -- 172.21.15.16:0/3678930994 >> 172.21.15.16:0/3678930994 conn(0x55958e6db300 msgr2=0x55958ea8b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.656+0000 7fe8f7623740 1 -- 172.21.15.16:0/3678930994 shutdown_connections 2024-08-31T06:09:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.656+0000 7fe8f7623740 1 -- 172.21.15.16:0/3678930994 wait complete. 2024-08-31T06:09:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.656+0000 7fe8f7623740 20 client.25981 trim_cache size 0 max 16384 2024-08-31T06:09:04.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:04 smithi016 ceph-mon[17194]: pgmap v419: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.4 MiB/s wr, 774 op/s 2024-08-31T06:09:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.673+0000 7fe8f7623740 1 Processor -- start 2024-08-31T06:09:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.673+0000 7fe8f7623740 1 -- start start 2024-08-31T06:09:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.673+0000 7fe8f7623740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958e6be2f0 0x55958ea97e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.673+0000 7fe8f7623740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958e6c64d0 0x55958ea8c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.673+0000 7fe8f7623740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958e6e71b0 0x55958ea93120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.673+0000 7fe8f7623740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e81b8300 con 0x55958e6be2f0 2024-08-31T06:09:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.674+0000 7fe8f7623740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e81b5b10 con 0x55958e6c64d0 2024-08-31T06:09:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.674+0000 7fe8f7623740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e81b3980 con 0x55958e6e71b0 2024-08-31T06:09:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.674+0000 7fe8ee600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958e6be2f0 0x55958ea97e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:04.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.674+0000 7fe8ee600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958e6be2f0 0x55958ea97e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33988/0 (socket says 172.21.15.16:33988) 2024-08-31T06:09:04.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.674+0000 7fe8ee600640 1 -- 172.21.15.16:0/3218241837 learned_addr learned my addr 172.21.15.16:0/3218241837 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:04.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.675+0000 7fe8ee600640 1 --2- 172.21.15.16:0/3218241837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958e6e71b0 0x55958ea93120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:04.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.675+0000 7fe8ed200640 1 --2- 172.21.15.16:0/3218241837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958e6c64d0 0x55958ea8c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.675+0000 7fe8ee600640 1 -- 172.21.15.16:0/3218241837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958e6c64d0 msgr2=0x55958ea8c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.675+0000 7fe8ee600640 1 --2- 172.21.15.16:0/3218241837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958e6c64d0 0x55958ea8c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.675+0000 7fe8ee600640 1 -- 172.21.15.16:0/3218241837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958e6be2f0 msgr2=0x55958ea97e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.675+0000 7fe8ee600640 1 --2- 172.21.15.16:0/3218241837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958e6be2f0 0x55958ea97e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.675+0000 7fe8ee600640 1 -- 172.21.15.16:0/3218241837 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55958ea8d730 con 0x55958e6e71b0 2024-08-31T06:09:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.676+0000 7fe8ee600640 1 --2- 172.21.15.16:0/3218241837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958e6e71b0 0x55958ea93120 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc0059f0 tx=0x7fe8dc00e4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f999a4c5c5c73bd server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.676+0000 7fe8da000640 1 -- 172.21.15.16:0/3218241837 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc01e070 con 0x55958e6e71b0 2024-08-31T06:09:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.676+0000 7fe8da000640 1 -- 172.21.15.16:0/3218241837 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8dc02e070 con 0x55958e6e71b0 2024-08-31T06:09:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.676+0000 7fe8da000640 1 -- 172.21.15.16:0/3218241837 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc019690 con 0x55958e6e71b0 2024-08-31T06:09:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.677+0000 7fe8f7623740 1 -- 172.21.15.16:0/3218241837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958e6e71b0 msgr2=0x55958ea93120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.677+0000 7fe8f7623740 1 --2- 172.21.15.16:0/3218241837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958e6e71b0 0x55958ea93120 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc0059f0 tx=0x7fe8dc00e4f0 comp rx=0 tx=0).stop 2024-08-31T06:09:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.677+0000 7fe8f7623740 1 -- 172.21.15.16:0/3218241837 shutdown_connections 2024-08-31T06:09:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.677+0000 7fe8f7623740 1 --2- 172.21.15.16:0/3218241837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958e6e71b0 0x55958ea93120 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.677+0000 7fe8f7623740 1 --2- 172.21.15.16:0/3218241837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958e6c64d0 0x55958ea8c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.677+0000 7fe8f7623740 1 --2- 172.21.15.16:0/3218241837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958e6be2f0 0x55958ea97e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.677+0000 7fe8f7623740 1 -- 172.21.15.16:0/3218241837 >> 172.21.15.16:0/3218241837 conn(0x55958e6db300 msgr2=0x55958ea8b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.678+0000 7fe8f7623740 1 -- 172.21.15.16:0/3218241837 shutdown_connections 2024-08-31T06:09:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.678+0000 7fe8f7623740 1 -- 172.21.15.16:0/3218241837 wait complete. 2024-08-31T06:09:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.678+0000 7fe8f7623740 1 -- start start 2024-08-31T06:09:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.678+0000 7fe8f7623740 1 Processor -- start 2024-08-31T06:09:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.679+0000 7fe8f7623740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958e6be2f0 0x55958eaafac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.679+0000 7fe8f7623740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958e6c64d0 0x55958eaacfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.679+0000 7fe8f7623740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958e6e71b0 0x55958eaad510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.679+0000 7fe8f7623740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e81b3fd0 con 0x55958e6e71b0 2024-08-31T06:09:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.679+0000 7fe8f7623740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e81b5440 con 0x55958e6be2f0 2024-08-31T06:09:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.679+0000 7fe8f7623740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e81b46a0 con 0x55958e6c64d0 2024-08-31T06:09:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.679+0000 7fe8f7623740 10 client.0 _ll_register_callbacks cb 0x7fe8dc009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:09:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.679+0000 7fe8ee600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958e6be2f0 0x55958eaafac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.680+0000 7fe8ee600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958e6be2f0 0x55958eaafac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:34514/0 (socket says 172.21.15.16:34514) 2024-08-31T06:09:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.680+0000 7fe8ee600640 1 -- 172.21.15.16:0/1747709760 learned_addr learned my addr 172.21.15.16:0/1747709760 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.680+0000 7fe8ee600640 1 --2- 172.21.15.16:0/1747709760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958e6e71b0 0x55958eaad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.680+0000 7fe8ed200640 1 --2- 172.21.15.16:0/1747709760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958e6c64d0 0x55958eaacfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.681+0000 7fe8ee600640 1 -- 172.21.15.16:0/1747709760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958e6c64d0 msgr2=0x55958eaacfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.681+0000 7fe8ee600640 1 --2- 172.21.15.16:0/1747709760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958e6c64d0 0x55958eaacfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.681+0000 7fe8ee600640 1 -- 172.21.15.16:0/1747709760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958e6e71b0 msgr2=0x55958eaad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.681+0000 7fe8ee600640 1 --2- 172.21.15.16:0/1747709760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958e6e71b0 0x55958eaad510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.681+0000 7fe8ee600640 1 -- 172.21.15.16:0/1747709760 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55958eaadc80 con 0x55958e6be2f0 2024-08-31T06:09:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.681+0000 7fe8ed200640 1 --2- 172.21.15.16:0/1747709760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958e6c64d0 0x55958eaacfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:09:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.681+0000 7fe8ee600640 1 --2- 172.21.15.16:0/1747709760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958e6be2f0 0x55958eaafac0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc009be0 tx=0x7fe8dc009600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a18f25a573a7039a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.681+0000 7fe8cf400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:09:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.682+0000 7fe8cf400640 1 -- 172.21.15.16:0/1747709760 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc01e070 con 0x55958e6be2f0 2024-08-31T06:09:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.682+0000 7fe8cf400640 1 -- 172.21.15.16:0/1747709760 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fe8dc00eb10 con 0x55958e6be2f0 2024-08-31T06:09:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.682+0000 7fe8f7623740 1 -- 172.21.15.16:0/1747709760 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8dc02e070 con 0x55958e6be2f0 2024-08-31T06:09:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.683+0000 7fe8cf400640 1 -- 172.21.15.16:0/1747709760 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe8dc019660 con 0x55958e6be2f0 2024-08-31T06:09:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.683+0000 7fe8f7623740 10 client.25994 fetch_fsmap learned FSMap version 31 2024-08-31T06:09:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.683+0000 7fe8f7623740 1 -- 172.21.15.16:0/1747709760 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55958eb23790 con 0x55958e6be2f0 2024-08-31T06:09:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.683+0000 7fe8cf400640 1 -- 172.21.15.16:0/1747709760 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8dc02e070 con 0x55958e6be2f0 2024-08-31T06:09:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.684+0000 7fe8cf400640 1 -- 172.21.15.16:0/1747709760 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fe8dc033450 con 0x55958e6be2f0 2024-08-31T06:09:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.684+0000 7fe8f7623740 10 client.25994 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:09:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.684+0000 7fe8f7623740 10 client.25994 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:09:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.684+0000 7fe8f7623740 20 client.25994 populate_metadata read hostname 'smithi016' 2024-08-31T06:09:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.684+0000 7fe8f7623740 1 --2- 172.21.15.16:0/1747709760 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55958e67b1c0 0x55958eb25eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.684+0000 7fe8f7623740 4 client.25994 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:09:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.684+0000 7fe8f7623740 1 -- 172.21.15.16:0/1747709760 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55958e9fd960 con 0x55958e67b1c0 2024-08-31T06:09:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.684+0000 7fe8ed200640 1 --2- 172.21.15.16:0/1747709760 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55958e67b1c0 0x55958eb25eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.685+0000 7fe8cf400640 10 client.25994 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:09:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.685+0000 7fe8ed200640 1 --2- 172.21.15.16:0/1747709760 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55958e67b1c0 0x55958eb25eb0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cceb089701592355 server_cookie=e309a0940df7e1cd in_seq=0 out_seq=0 2024-08-31T06:09:04.728 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.725+0000 7fe8cf400640 1 -- 172.21.15.16:0/1747709760 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55958e9fd960 con 0x55958e67b1c0 2024-08-31T06:09:04.728 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.725+0000 7fe8cf400640 10 client.25994 handle_command_reply: tid=0 2024-08-31T06:09:04.728 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.725+0000 7fe8f7623740 1 client.25994 shutdown 2024-08-31T06:09:04.728 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.725+0000 7fe8f7623740 1 -- 172.21.15.16:0/1747709760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958e6be2f0 msgr2=0x55958eaafac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:04.728 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.725+0000 7fe8f7623740 1 --2- 172.21.15.16:0/1747709760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958e6be2f0 0x55958eaafac0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc009be0 tx=0x7fe8dc009600 comp rx=0 tx=0).stop 2024-08-31T06:09:04.729 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.726+0000 7fe8f7623740 1 -- 172.21.15.16:0/1747709760 shutdown_connections 2024-08-31T06:09:04.729 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.726+0000 7fe8f7623740 1 --2- 172.21.15.16:0/1747709760 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55958e67b1c0 0x55958eb25eb0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:04.729 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.726+0000 7fe8f7623740 1 --2- 172.21.15.16:0/1747709760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958e6c64d0 0x55958eaacfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:04.729 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.726+0000 7fe8f7623740 1 --2- 172.21.15.16:0/1747709760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958e6e71b0 0x55958eaad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:04.729 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.726+0000 7fe8f7623740 1 --2- 172.21.15.16:0/1747709760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958e6be2f0 0x55958eaafac0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.726+0000 7fe8f7623740 1 -- 172.21.15.16:0/1747709760 >> 172.21.15.16:0/1747709760 conn(0x55958e6db300 msgr2=0x55958ea8b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.727+0000 7fe8cf400640 0 client.25994 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:09:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.727+0000 7fe8f7623740 1 -- 172.21.15.16:0/1747709760 shutdown_connections 2024-08-31T06:09:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.727+0000 7fe8f7623740 1 -- 172.21.15.16:0/1747709760 wait complete. 2024-08-31T06:09:04.732 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.730+0000 7fe8f7623740 20 client.25994 trim_cache size 0 max 16384 2024-08-31T06:09:04.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:09:04.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:09:04.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:09:04.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:09:04.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.731+0000 7fe8f0600640 1 -- 172.21.15.16:0/4188831248 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe8d00b4d20 msgr2=0x7fe8d00b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:04.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.731+0000 7fe8f0600640 1 --2- 172.21.15.16:0/4188831248 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe8d00b4d20 0x7fe8d00b7230 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x55958ea895a0 tx=0x7fe8e003d4b0 comp rx=0 tx=0).stop 2024-08-31T06:09:04.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.731+0000 7fe8f0600640 1 -- 172.21.15.16:0/4188831248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8e810a340 msgr2=0x7fe8e81a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:04.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.731+0000 7fe8f0600640 1 --2- 172.21.15.16:0/4188831248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8e810a340 0x7fe8e81a4cc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe8e002acb0 tx=0x7fe8e003dc00 comp rx=0 tx=0).stop 2024-08-31T06:09:04.734 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.731+0000 7fe8f0600640 1 -- 172.21.15.16:0/4188831248 shutdown_connections 2024-08-31T06:09:04.734 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.731+0000 7fe8f0600640 1 --2- 172.21.15.16:0/4188831248 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe8d00b4d20 0x7fe8d00b7230 secure :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0x55958ea895a0 tx=0x7fe8e003d4b0 comp rx=0 tx=0).stop 2024-08-31T06:09:04.734 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.731+0000 7fe8f0600640 1 --2- 172.21.15.16:0/4188831248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8e8116340 0x7fe8e81aa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:04.734 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.731+0000 7fe8f0600640 1 --2- 172.21.15.16:0/4188831248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8e810ac90 0x7fe8e81a5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:04.734 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.731+0000 7fe8f0600640 1 --2- 172.21.15.16:0/4188831248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8e810a340 0x7fe8e81a4cc0 secure :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe8e002acb0 tx=0x7fe8e003dc00 comp rx=0 tx=0).stop 2024-08-31T06:09:04.734 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.731+0000 7fe8f0600640 1 -- 172.21.15.16:0/4188831248 >> 172.21.15.16:0/4188831248 conn(0x7fe8e8070b70 msgr2=0x7fe8e810edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:04.738 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.736+0000 7fe8f0600640 1 -- 172.21.15.16:0/4188831248 shutdown_connections 2024-08-31T06:09:04.738 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:04.736+0000 7fe8f0600640 1 -- 172.21.15.16:0/4188831248 wait complete. 2024-08-31T06:09:04.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:04 smithi155 ceph-mon[21742]: pgmap v419: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.4 MiB/s wr, 774 op/s 2024-08-31T06:09:04.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:04 smithi184 ceph-mon[22331]: pgmap v419: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.4 MiB/s wr, 774 op/s 2024-08-31T06:09:05.196 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:09:06.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.177+0000 7f59c9e00640 1 -- 172.21.15.16:0/3908743031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59c410b470 msgr2=0x7f59c410b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:06.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.177+0000 7f59c9e00640 1 --2- 172.21.15.16:0/3908743031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59c410b470 0x7f59c410b880 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f59bc009900 tx=0x7f59bc02a410 comp rx=0 tx=0).stop 2024-08-31T06:09:06.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.177+0000 7f59c9e00640 1 -- 172.21.15.16:0/3908743031 shutdown_connections 2024-08-31T06:09:06.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.177+0000 7f59c9e00640 1 --2- 172.21.15.16:0/3908743031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59c410b470 0x7f59c410b880 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:06.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.177+0000 7f59c9e00640 1 --2- 172.21.15.16:0/3908743031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59c410bdc0 0x7f59c41125f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:06.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.177+0000 7f59c9e00640 1 --2- 172.21.15.16:0/3908743031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59c4105b30 0x7f59c4105f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:06.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.177+0000 7f59c9e00640 1 -- 172.21.15.16:0/3908743031 >> 172.21.15.16:0/3908743031 conn(0x7f59c4101600 msgr2=0x7f59c4103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:06.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.178+0000 7f59c9e00640 1 -- 172.21.15.16:0/3908743031 shutdown_connections 2024-08-31T06:09:06.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.178+0000 7f59c9e00640 1 -- 172.21.15.16:0/3908743031 wait complete. 2024-08-31T06:09:06.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.180+0000 7f59c9e00640 1 Processor -- start 2024-08-31T06:09:06.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.180+0000 7f59c9e00640 1 -- start start 2024-08-31T06:09:06.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.181+0000 7f59c9e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59c4105b30 0x7f59c419c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:06.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.181+0000 7f59c9e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59c410b470 0x7f59c419ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:06.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.181+0000 7f59c9e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59c410bdc0 0x7f59c41a1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:06.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.181+0000 7f59c9e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f59c4115af0 con 0x7f59c410bdc0 2024-08-31T06:09:06.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.181+0000 7f59c9e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f59c4115970 con 0x7f59c4105b30 2024-08-31T06:09:06.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.181+0000 7f59c9e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f59c4115c70 con 0x7f59c410b470 2024-08-31T06:09:06.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.181+0000 7f59c2000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59c410b470 0x7f59c419ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:06.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.181+0000 7f59c2a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59c4105b30 0x7f59c419c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:06.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.181+0000 7f59c2000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59c410b470 0x7f59c419ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:42130/0 (socket says 172.21.15.16:42130) 2024-08-31T06:09:06.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.181+0000 7f59c2000640 1 -- 172.21.15.16:0/2681927836 learned_addr learned my addr 172.21.15.16:0/2681927836 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:06.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.183+0000 7f59c2000640 1 -- 172.21.15.16:0/2681927836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59c4105b30 msgr2=0x7f59c419c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:06.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.183+0000 7f59c2000640 1 --2- 172.21.15.16:0/2681927836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59c4105b30 0x7f59c419c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:06.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.183+0000 7f59c2000640 1 -- 172.21.15.16:0/2681927836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59c410bdc0 msgr2=0x7f59c41a1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:06.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.183+0000 7f59c2000640 1 --2- 172.21.15.16:0/2681927836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59c410bdc0 0x7f59c41a1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:06.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.183+0000 7f59c2000640 1 -- 172.21.15.16:0/2681927836 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c41a2430 con 0x7f59c410b470 2024-08-31T06:09:06.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.184+0000 7f59c2000640 1 --2- 172.21.15.16:0/2681927836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59c410b470 0x7f59c419ca60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f59bc031440 tx=0x7f59bc003c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:06.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.184+0000 7f59c8a00640 1 -- 172.21.15.16:0/2681927836 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc03f070 con 0x7f59c410b470 2024-08-31T06:09:06.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.185+0000 7f59c8a00640 1 -- 172.21.15.16:0/2681927836 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59bc031b50 con 0x7f59c410b470 2024-08-31T06:09:06.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.185+0000 7f59c8a00640 1 -- 172.21.15.16:0/2681927836 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc035030 con 0x7f59c410b470 2024-08-31T06:09:06.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.185+0000 7f59c9e00640 1 -- 172.21.15.16:0/2681927836 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59c41a26c0 con 0x7f59c410b470 2024-08-31T06:09:06.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.185+0000 7f59c9e00640 1 -- 172.21.15.16:0/2681927836 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59c41a8e80 con 0x7f59c410b470 2024-08-31T06:09:06.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.190+0000 7f59d0aa2740 1 Processor -- start 2024-08-31T06:09:06.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.191+0000 7f59d0aa2740 1 -- start start 2024-08-31T06:09:06.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.191+0000 7f59c8a00640 1 -- 172.21.15.16:0/2681927836 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f59bc042070 con 0x7f59c410b470 2024-08-31T06:09:06.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.191+0000 7f59d0aa2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3fe452070 0x55c3fe7f7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:06.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.191+0000 7f59d0aa2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3fe4291b0 0x55c3fe7f7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:06.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.191+0000 7f59d0aa2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3fe431390 0x55c3fe7f8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:06.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.191+0000 7f59d0aa2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c3fe76e7b0 con 0x55c3fe431390 2024-08-31T06:09:06.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.191+0000 7f59d0aa2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c3fe75dd40 con 0x55c3fe4291b0 2024-08-31T06:09:06.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.191+0000 7f59d0aa2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c3fe761cf0 con 0x55c3fe452070 2024-08-31T06:09:06.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.192+0000 7f59c3400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3fe452070 0x55c3fe7f7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:06.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.192+0000 7f59c3400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3fe452070 0x55c3fe7f7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:42140/0 (socket says 172.21.15.16:42140) 2024-08-31T06:09:06.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.192+0000 7f59c3400640 1 -- 172.21.15.16:0/745418518 learned_addr learned my addr 172.21.15.16:0/745418518 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:06.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.192+0000 7f59c2a00640 1 --2- 172.21.15.16:0/745418518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3fe4291b0 0x55c3fe7f7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:06.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.192+0000 7f59c2000640 1 --2- 172.21.15.16:0/745418518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3fe431390 0x55c3fe7f8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:06.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.193+0000 7f59c3400640 1 -- 172.21.15.16:0/745418518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3fe4291b0 msgr2=0x55c3fe7f7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:06.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.193+0000 7f59c3400640 1 --2- 172.21.15.16:0/745418518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3fe4291b0 0x55c3fe7f7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:06.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.193+0000 7f59c3400640 1 -- 172.21.15.16:0/745418518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3fe431390 msgr2=0x55c3fe7f8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:06.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.193+0000 7f59c3400640 1 --2- 172.21.15.16:0/745418518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3fe431390 0x55c3fe7f8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:06.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.193+0000 7f59c3400640 1 -- 172.21.15.16:0/745418518 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3fe4bd650 con 0x55c3fe452070 2024-08-31T06:09:06.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.193+0000 7f59c2000640 1 --2- 172.21.15.16:0/745418518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3fe431390 0x55c3fe7f8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:09:06.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.194+0000 7f59c8a00640 1 --2- 172.21.15.16:0/2681927836 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f59a80b4e30 0x7f59a80b7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:06.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.194+0000 7f59c8a00640 1 -- 172.21.15.16:0/2681927836 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f59bc105c40 con 0x7f59c410b470 2024-08-31T06:09:06.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.194+0000 7f59c3400640 1 --2- 172.21.15.16:0/745418518 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3fe452070 0x55c3fe7f7990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f59b4009d00 tx=0x7f59b400b420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eee6d6deaeb1bf01 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:06.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.195+0000 7f59c2a00640 1 --2- 172.21.15.16:0/2681927836 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f59a80b4e30 0x7f59a80b7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:06.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.195+0000 7f59b3e00640 1 -- 172.21.15.16:0/745418518 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b400bd40 con 0x55c3fe452070 2024-08-31T06:09:06.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.195+0000 7f59b3e00640 1 -- 172.21.15.16:0/745418518 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59b4017070 con 0x55c3fe452070 2024-08-31T06:09:06.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.195+0000 7f59b3e00640 1 -- 172.21.15.16:0/745418518 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b40115f0 con 0x55c3fe452070 2024-08-31T06:09:06.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.196+0000 7f59d0aa2740 1 -- 172.21.15.16:0/745418518 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3fe452070 msgr2=0x55c3fe7f7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:06.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.196+0000 7f59d0aa2740 1 --2- 172.21.15.16:0/745418518 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3fe452070 0x55c3fe7f7990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f59b4009d00 tx=0x7f59b400b420 comp rx=0 tx=0).stop 2024-08-31T06:09:06.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.196+0000 7f59c2a00640 1 --2- 172.21.15.16:0/2681927836 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f59a80b4e30 0x7f59a80b7340 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x55c3fe7f6610 tx=0x7f59ac005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:06.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.197+0000 7f59d0aa2740 1 -- 172.21.15.16:0/745418518 shutdown_connections 2024-08-31T06:09:06.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.197+0000 7f59d0aa2740 1 --2- 172.21.15.16:0/745418518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3fe431390 0x55c3fe7f8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:06.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.197+0000 7f59d0aa2740 1 --2- 172.21.15.16:0/745418518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3fe4291b0 0x55c3fe7f7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:06.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.197+0000 7f59d0aa2740 1 --2- 172.21.15.16:0/745418518 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3fe452070 0x55c3fe7f7990 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:06.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.197+0000 7f59d0aa2740 1 -- 172.21.15.16:0/745418518 >> 172.21.15.16:0/745418518 conn(0x55c3fe446460 msgr2=0x55c3fe7f5880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:06.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.197+0000 7f59d0aa2740 1 -- 172.21.15.16:0/745418518 shutdown_connections 2024-08-31T06:09:06.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.197+0000 7f59d0aa2740 1 -- 172.21.15.16:0/745418518 wait complete. 2024-08-31T06:09:06.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.198+0000 7f59d0aa2740 1 -- start start 2024-08-31T06:09:06.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.199+0000 7f59d0aa2740 1 Processor -- start 2024-08-31T06:09:06.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.200+0000 7f59d0aa2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3fe4291b0 0x55c3fe803c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:06.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.200+0000 7f59d0aa2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3fe431390 0x55c3fe8041d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:06.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.200+0000 7f59d0aa2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3fe452070 0x55c3fe809210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:06.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.200+0000 7f59d0aa2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c3fe5c0700 con 0x55c3fe4291b0 2024-08-31T06:09:06.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.200+0000 7f59d0aa2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c3fe6e4cf0 con 0x55c3fe431390 2024-08-31T06:09:06.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.200+0000 7f59d0aa2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c3fe7fe040 con 0x55c3fe452070 2024-08-31T06:09:06.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.200+0000 7f59d0aa2740 10 client.0 _ll_register_callbacks cb 0x55c3fe70a470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:09:06.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.201+0000 7f59c3400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3fe452070 0x55c3fe809210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:06.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.201+0000 7f59c3400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3fe452070 0x55c3fe809210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:42148/0 (socket says 172.21.15.16:42148) 2024-08-31T06:09:06.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.201+0000 7f59c3400640 1 -- 172.21.15.16:0/4048134819 learned_addr learned my addr 172.21.15.16:0/4048134819 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:06.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.201+0000 7f59c2a00640 1 --2- 172.21.15.16:0/4048134819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3fe4291b0 0x55c3fe803c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:06.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.201+0000 7f59c2000640 1 --2- 172.21.15.16:0/4048134819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3fe431390 0x55c3fe8041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:06.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.201+0000 7f59c3400640 1 -- 172.21.15.16:0/4048134819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3fe431390 msgr2=0x55c3fe8041d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:06.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.202+0000 7f59c3400640 1 --2- 172.21.15.16:0/4048134819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3fe431390 0x55c3fe8041d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:06.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.202+0000 7f59c3400640 1 -- 172.21.15.16:0/4048134819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3fe4291b0 msgr2=0x55c3fe803c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:06.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.202+0000 7f59c3400640 1 --2- 172.21.15.16:0/4048134819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3fe4291b0 0x55c3fe803c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:06.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.202+0000 7f59c3400640 1 -- 172.21.15.16:0/4048134819 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3fe7258b0 con 0x55c3fe452070 2024-08-31T06:09:06.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.202+0000 7f59c2000640 1 --2- 172.21.15.16:0/4048134819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3fe431390 0x55c3fe8041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:09:06.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.202+0000 7f59c2a00640 1 --2- 172.21.15.16:0/4048134819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3fe4291b0 0x55c3fe803c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:09:06.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.203+0000 7f59c3400640 1 --2- 172.21.15.16:0/4048134819 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3fe452070 0x55c3fe809210 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f59b4012e00 tx=0x7f59b4007190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc26ee224f1f4ba9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:06.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.203+0000 7f59b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:09:06.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.204+0000 7f59b0c00640 1 -- 172.21.15.16:0/4048134819 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b4021070 con 0x55c3fe452070 2024-08-31T06:09:06.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.204+0000 7f59b0c00640 1 -- 172.21.15.16:0/4048134819 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f59b401b800 con 0x55c3fe452070 2024-08-31T06:09:06.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.204+0000 7f59d0aa2740 1 -- 172.21.15.16:0/4048134819 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59b4017070 con 0x55c3fe452070 2024-08-31T06:09:06.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.205+0000 7f59b0c00640 1 -- 172.21.15.16:0/4048134819 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f59b401ba80 con 0x55c3fe452070 2024-08-31T06:09:06.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.205+0000 7f59d0aa2740 10 client.25999 fetch_fsmap learned FSMap version 31 2024-08-31T06:09:06.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.205+0000 7f59d0aa2740 1 -- 172.21.15.16:0/4048134819 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c3fe8230f0 con 0x55c3fe452070 2024-08-31T06:09:06.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.205+0000 7f59b0c00640 1 -- 172.21.15.16:0/4048134819 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59b4017070 con 0x55c3fe452070 2024-08-31T06:09:06.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.205+0000 7f59b0c00640 1 -- 172.21.15.16:0/4048134819 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f59b402c020 con 0x55c3fe452070 2024-08-31T06:09:06.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.206+0000 7f59d0aa2740 10 client.25999 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:09:06.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.206+0000 7f59d0aa2740 10 client.25999 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-31T06:09:06.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.206+0000 7f59d0aa2740 20 client.25999 populate_metadata read hostname 'smithi016' 2024-08-31T06:09:06.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.206+0000 7f59d0aa2740 1 --2- 172.21.15.16:0/4048134819 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55c3fe4109f0 0x55c3fe8264b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:06.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.206+0000 7f59d0aa2740 4 client.25999 mds_command: new command op to 24724 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:09:06.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.206+0000 7f59d0aa2740 1 -- 172.21.15.16:0/4048134819 --> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3fe770700 con 0x55c3fe4109f0 2024-08-31T06:09:06.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.206+0000 7f59c2a00640 1 --2- 172.21.15.16:0/4048134819 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55c3fe4109f0 0x55c3fe8264b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:06.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.207+0000 7f59b0c00640 10 client.25999 ms_handle_connect on v2:172.21.15.16:6838/219664460 2024-08-31T06:09:06.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.208+0000 7f59c2a00640 1 --2- 172.21.15.16:0/4048134819 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55c3fe4109f0 0x55c3fe8264b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=de6c848e5336b155 server_cookie=9f0c72675826771b in_seq=0 out_seq=0 2024-08-31T06:09:06.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.209+0000 7f59b0c00640 1 -- 172.21.15.16:0/4048134819 <== mds.1 v2:172.21.15.16:6838/219664460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c3fe770700 con 0x55c3fe4109f0 2024-08-31T06:09:06.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.209+0000 7f59b0c00640 10 client.25999 handle_command_reply: tid=0 2024-08-31T06:09:06.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.209+0000 7f59d0aa2740 1 client.25999 shutdown 2024-08-31T06:09:06.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.210+0000 7f59d0aa2740 1 -- 172.21.15.16:0/4048134819 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3fe452070 msgr2=0x55c3fe809210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:06.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.210+0000 7f59d0aa2740 1 --2- 172.21.15.16:0/4048134819 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3fe452070 0x55c3fe809210 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f59b4012e00 tx=0x7f59b4007190 comp rx=0 tx=0).stop 2024-08-31T06:09:06.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.210+0000 7f59d0aa2740 1 -- 172.21.15.16:0/4048134819 shutdown_connections 2024-08-31T06:09:06.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.210+0000 7f59d0aa2740 1 --2- 172.21.15.16:0/4048134819 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3fe452070 0x55c3fe809210 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:06.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.210+0000 7f59d0aa2740 1 --2- 172.21.15.16:0/4048134819 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55c3fe4109f0 0x55c3fe8264b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:06.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.210+0000 7f59d0aa2740 1 --2- 172.21.15.16:0/4048134819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3fe431390 0x55c3fe8041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:06.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.210+0000 7f59d0aa2740 1 --2- 172.21.15.16:0/4048134819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3fe4291b0 0x55c3fe803c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:06.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.210+0000 7f59d0aa2740 1 -- 172.21.15.16:0/4048134819 >> 172.21.15.16:0/4048134819 conn(0x55c3fe446460 msgr2=0x55c3fe7f3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:06.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.210+0000 7f59b0c00640 0 client.25999 ms_handle_reset on v2:172.21.15.16:6838/219664460 2024-08-31T06:09:06.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.210+0000 7f59d0aa2740 1 -- 172.21.15.16:0/4048134819 shutdown_connections 2024-08-31T06:09:06.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.210+0000 7f59d0aa2740 1 -- 172.21.15.16:0/4048134819 wait complete. 2024-08-31T06:09:06.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.211+0000 7f59d0aa2740 20 client.25999 trim_cache size 0 max 16384 2024-08-31T06:09:06.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.232+0000 7f59d0aa2740 1 Processor -- start 2024-08-31T06:09:06.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.232+0000 7f59d0aa2740 1 -- start start 2024-08-31T06:09:06.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.232+0000 7f59d0aa2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3fe4291b0 0x55c3fe808780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:06.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.232+0000 7f59d0aa2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3fe431390 0x55c3fe807020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:06.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.232+0000 7f59d0aa2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3fe452070 0x55c3fe807560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:06.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.232+0000 7f59d0aa2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f59c41afc30 con 0x55c3fe452070 2024-08-31T06:09:06.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.232+0000 7f59d0aa2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f59c41ad440 con 0x55c3fe4291b0 2024-08-31T06:09:06.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.232+0000 7f59d0aa2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f59c41ab2b0 con 0x55c3fe431390 2024-08-31T06:09:06.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.233+0000 7f59c2a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3fe4291b0 0x55c3fe808780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:06.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.233+0000 7f59c2a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3fe4291b0 0x55c3fe808780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:36134/0 (socket says 172.21.15.16:36134) 2024-08-31T06:09:06.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.233+0000 7f59c2a00640 1 -- 172.21.15.16:0/2383153579 learned_addr learned my addr 172.21.15.16:0/2383153579 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:06.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.233+0000 7f59c2000640 1 --2- 172.21.15.16:0/2383153579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3fe431390 0x55c3fe807020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:06.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.233+0000 7f59c3400640 1 --2- 172.21.15.16:0/2383153579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3fe452070 0x55c3fe807560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:06.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.234+0000 7f59c2000640 1 -- 172.21.15.16:0/2383153579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3fe4291b0 msgr2=0x55c3fe808780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:06.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.234+0000 7f59c2000640 1 --2- 172.21.15.16:0/2383153579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3fe4291b0 0x55c3fe808780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:06.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.234+0000 7f59c2000640 1 -- 172.21.15.16:0/2383153579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3fe452070 msgr2=0x55c3fe807560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:06.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.234+0000 7f59c2000640 1 --2- 172.21.15.16:0/2383153579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3fe452070 0x55c3fe807560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:06.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.234+0000 7f59c2000640 1 -- 172.21.15.16:0/2383153579 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3fe7fa7a0 con 0x55c3fe431390 2024-08-31T06:09:06.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.234+0000 7f59c2a00640 1 --2- 172.21.15.16:0/2383153579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3fe4291b0 0x55c3fe808780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:09:06.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.235+0000 7f59c2000640 1 --2- 172.21.15.16:0/2383153579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3fe431390 0x55c3fe807020 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f59bc1033f0 tx=0x7f59bc031f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d92e7c0890eba338 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:06.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.235+0000 7f59b3400640 1 -- 172.21.15.16:0/2383153579 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc0cc390 con 0x55c3fe431390 2024-08-31T06:09:06.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.235+0000 7f59b3400640 1 -- 172.21.15.16:0/2383153579 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59bc0d0c00 con 0x55c3fe431390 2024-08-31T06:09:06.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.235+0000 7f59b3400640 1 -- 172.21.15.16:0/2383153579 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc103670 con 0x55c3fe431390 2024-08-31T06:09:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.236+0000 7f59d0aa2740 1 -- 172.21.15.16:0/2383153579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3fe431390 msgr2=0x55c3fe807020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.236+0000 7f59d0aa2740 1 --2- 172.21.15.16:0/2383153579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3fe431390 0x55c3fe807020 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f59bc1033f0 tx=0x7f59bc031f50 comp rx=0 tx=0).stop 2024-08-31T06:09:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.237+0000 7f59d0aa2740 1 -- 172.21.15.16:0/2383153579 shutdown_connections 2024-08-31T06:09:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.237+0000 7f59d0aa2740 1 --2- 172.21.15.16:0/2383153579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3fe431390 0x55c3fe807020 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.237+0000 7f59d0aa2740 1 --2- 172.21.15.16:0/2383153579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3fe452070 0x55c3fe807560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.237+0000 7f59d0aa2740 1 --2- 172.21.15.16:0/2383153579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3fe4291b0 0x55c3fe808780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.237+0000 7f59d0aa2740 1 -- 172.21.15.16:0/2383153579 >> 172.21.15.16:0/2383153579 conn(0x55c3fe446460 msgr2=0x55c3fe808070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.237+0000 7f59d0aa2740 1 -- 172.21.15.16:0/2383153579 shutdown_connections 2024-08-31T06:09:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.237+0000 7f59d0aa2740 1 -- 172.21.15.16:0/2383153579 wait complete. 2024-08-31T06:09:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.237+0000 7f59d0aa2740 1 -- start start 2024-08-31T06:09:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.238+0000 7f59d0aa2740 1 Processor -- start 2024-08-31T06:09:06.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.238+0000 7f59d0aa2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3fe4291b0 0x55c3fe837d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:06.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.238+0000 7f59d0aa2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3fe431390 0x55c3fe838330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:06.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.238+0000 7f59d0aa2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3fe452070 0x55c3fe832ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:06.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.238+0000 7f59d0aa2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f59c41ab900 con 0x55c3fe4291b0 2024-08-31T06:09:06.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.238+0000 7f59d0aa2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f59c41acd70 con 0x55c3fe452070 2024-08-31T06:09:06.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.238+0000 7f59d0aa2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f59c41abfd0 con 0x55c3fe431390 2024-08-31T06:09:06.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.238+0000 7f59d0aa2740 10 client.0 _ll_register_callbacks cb 0x7f59b40127d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:09:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.239+0000 7f59c3400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3fe452070 0x55c3fe832ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.239+0000 7f59c3400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3fe452070 0x55c3fe832ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:36138/0 (socket says 172.21.15.16:36138) 2024-08-31T06:09:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.239+0000 7f59c3400640 1 -- 172.21.15.16:0/3460249684 learned_addr learned my addr 172.21.15.16:0/3460249684 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.239+0000 7f59c2000640 1 --2- 172.21.15.16:0/3460249684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3fe431390 0x55c3fe838330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.239+0000 7f59c2a00640 1 --2- 172.21.15.16:0/3460249684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3fe4291b0 0x55c3fe837d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:06.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.240+0000 7f59c3400640 1 -- 172.21.15.16:0/3460249684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3fe431390 msgr2=0x55c3fe838330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:06.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.240+0000 7f59c3400640 1 --2- 172.21.15.16:0/3460249684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3fe431390 0x55c3fe838330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:06.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.240+0000 7f59c3400640 1 -- 172.21.15.16:0/3460249684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3fe4291b0 msgr2=0x55c3fe837d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:06.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.240+0000 7f59c3400640 1 --2- 172.21.15.16:0/3460249684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3fe4291b0 0x55c3fe837d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:06.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.240+0000 7f59c3400640 1 -- 172.21.15.16:0/3460249684 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3fe833220 con 0x55c3fe452070 2024-08-31T06:09:06.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.241+0000 7f59c2a00640 1 --2- 172.21.15.16:0/3460249684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3fe4291b0 0x55c3fe837d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:09:06.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.241+0000 7f59c3400640 1 --2- 172.21.15.16:0/3460249684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3fe452070 0x55c3fe832ab0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f59b4009f50 tx=0x7f59b401b970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59506e9579d7d9ed server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:06.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.241+0000 7f59b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:09:06.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.242+0000 7f59b0c00640 1 -- 172.21.15.16:0/3460249684 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b4021040 con 0x55c3fe452070 2024-08-31T06:09:06.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.242+0000 7f59b0c00640 1 -- 172.21.15.16:0/3460249684 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f59b4012880 con 0x55c3fe452070 2024-08-31T06:09:06.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.242+0000 7f59d0aa2740 1 -- 172.21.15.16:0/3460249684 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59bc0d0c00 con 0x55c3fe452070 2024-08-31T06:09:06.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.242+0000 7f59b0c00640 1 -- 172.21.15.16:0/3460249684 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f59b4012a80 con 0x55c3fe452070 2024-08-31T06:09:06.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.243+0000 7f59d0aa2740 10 client.26009 fetch_fsmap learned FSMap version 31 2024-08-31T06:09:06.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.243+0000 7f59d0aa2740 1 -- 172.21.15.16:0/3460249684 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c3fe891740 con 0x55c3fe452070 2024-08-31T06:09:06.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.243+0000 7f59b0c00640 1 -- 172.21.15.16:0/3460249684 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59b4017040 con 0x55c3fe452070 2024-08-31T06:09:06.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.243+0000 7f59b0c00640 1 -- 172.21.15.16:0/3460249684 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f59b402c040 con 0x55c3fe452070 2024-08-31T06:09:06.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.244+0000 7f59d0aa2740 10 client.26009 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:09:06.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.244+0000 7f59d0aa2740 10 client.26009 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-31T06:09:06.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.244+0000 7f59d0aa2740 20 client.26009 populate_metadata read hostname 'smithi016' 2024-08-31T06:09:06.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.244+0000 7f59d0aa2740 1 --2- 172.21.15.16:0/3460249684 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55c3fe3e5da0 0x55c3fe893f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:06.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.244+0000 7f59d0aa2740 4 client.26009 mds_command: new command op to 24724 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:09:06.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.244+0000 7f59d0aa2740 1 -- 172.21.15.16:0/3460249684 --> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f59bc0d0c00 con 0x55c3fe3e5da0 2024-08-31T06:09:06.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.244+0000 7f59c2a00640 1 --2- 172.21.15.16:0/3460249684 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55c3fe3e5da0 0x55c3fe893f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:06.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.246+0000 7f59c2a00640 1 --2- 172.21.15.16:0/3460249684 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55c3fe3e5da0 0x55c3fe893f00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1bd5d47e778c8645 server_cookie=9153fae3fbab9285 in_seq=0 out_seq=0 2024-08-31T06:09:06.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.246+0000 7f59b0c00640 10 client.26009 ms_handle_connect on v2:172.21.15.16:6838/219664460 2024-08-31T06:09:06.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.246+0000 7f59b0c00640 1 -- 172.21.15.16:0/3460249684 <== mds.1 v2:172.21.15.16:6838/219664460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f59bc0d0c00 con 0x55c3fe3e5da0 2024-08-31T06:09:06.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.246+0000 7f59b0c00640 10 client.26009 handle_command_reply: tid=0 2024-08-31T06:09:06.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.247+0000 7f59d0aa2740 1 client.26009 shutdown 2024-08-31T06:09:06.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.247+0000 7f59d0aa2740 1 -- 172.21.15.16:0/3460249684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3fe452070 msgr2=0x55c3fe832ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:06.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.247+0000 7f59d0aa2740 1 --2- 172.21.15.16:0/3460249684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3fe452070 0x55c3fe832ab0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f59b4009f50 tx=0x7f59b401b970 comp rx=0 tx=0).stop 2024-08-31T06:09:06.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.247+0000 7f59d0aa2740 1 -- 172.21.15.16:0/3460249684 shutdown_connections 2024-08-31T06:09:06.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.247+0000 7f59d0aa2740 1 --2- 172.21.15.16:0/3460249684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c3fe431390 0x55c3fe838330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:06.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.247+0000 7f59d0aa2740 1 --2- 172.21.15.16:0/3460249684 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55c3fe3e5da0 0x55c3fe893f00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:06.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.247+0000 7f59d0aa2740 1 --2- 172.21.15.16:0/3460249684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3fe452070 0x55c3fe832ab0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:06.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.247+0000 7f59d0aa2740 1 --2- 172.21.15.16:0/3460249684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3fe4291b0 0x55c3fe837d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:06.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.247+0000 7f59d0aa2740 1 -- 172.21.15.16:0/3460249684 >> 172.21.15.16:0/3460249684 conn(0x55c3fe446460 msgr2=0x55c3fe8077b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:06.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.248+0000 7f59b0c00640 0 client.26009 ms_handle_reset on v2:172.21.15.16:6838/219664460 2024-08-31T06:09:06.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.248+0000 7f59d0aa2740 1 -- 172.21.15.16:0/3460249684 shutdown_connections 2024-08-31T06:09:06.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.248+0000 7f59d0aa2740 1 -- 172.21.15.16:0/3460249684 wait complete. 2024-08-31T06:09:06.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.248+0000 7f59d0aa2740 20 client.26009 trim_cache size 0 max 16384 2024-08-31T06:09:06.252 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:09:06.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.250+0000 7f59c9e00640 1 -- 172.21.15.16:0/2681927836 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f59a80b4e30 msgr2=0x7f59a80b7340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:06.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.250+0000 7f59c9e00640 1 --2- 172.21.15.16:0/2681927836 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f59a80b4e30 0x7f59a80b7340 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x55c3fe7f6610 tx=0x7f59ac005f90 comp rx=0 tx=0).stop 2024-08-31T06:09:06.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.250+0000 7f59c9e00640 1 -- 172.21.15.16:0/2681927836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59c410b470 msgr2=0x7f59c419ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:06.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.250+0000 7f59c9e00640 1 --2- 172.21.15.16:0/2681927836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59c410b470 0x7f59c419ca60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f59bc031440 tx=0x7f59bc003c00 comp rx=0 tx=0).stop 2024-08-31T06:09:06.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.250+0000 7f59c9e00640 1 -- 172.21.15.16:0/2681927836 shutdown_connections 2024-08-31T06:09:06.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.250+0000 7f59c9e00640 1 --2- 172.21.15.16:0/2681927836 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f59a80b4e30 0x7f59a80b7340 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:06.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.250+0000 7f59c9e00640 1 --2- 172.21.15.16:0/2681927836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59c410b470 0x7f59c419ca60 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:06.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.250+0000 7f59c9e00640 1 --2- 172.21.15.16:0/2681927836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59c410bdc0 0x7f59c41a1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:06.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.250+0000 7f59c9e00640 1 --2- 172.21.15.16:0/2681927836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59c4105b30 0x7f59c419c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:06.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.250+0000 7f59c9e00640 1 -- 172.21.15.16:0/2681927836 >> 172.21.15.16:0/2681927836 conn(0x7f59c4101600 msgr2=0x7f59c410ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:06.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.250+0000 7f59c9e00640 1 -- 172.21.15.16:0/2681927836 shutdown_connections 2024-08-31T06:09:06.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:06.250+0000 7f59c9e00640 1 -- 172.21.15.16:0/2681927836 wait complete. 2024-08-31T06:09:06.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 100.0 s, 328.8 tps, lat 298.508 ms stddev 497.341 2024-08-31T06:09:06.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:06 smithi155 ceph-mon[21742]: pgmap v420: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.6 MiB/s wr, 847 op/s 2024-08-31T06:09:06.909 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:06 smithi016 ceph-mon[17194]: pgmap v420: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.6 MiB/s wr, 847 op/s 2024-08-31T06:09:06.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:06 smithi184 ceph-mon[22331]: pgmap v420: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.6 MiB/s wr, 847 op/s 2024-08-31T06:09:08.261 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T06:09:08.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:08 smithi155 ceph-mon[21742]: pgmap v421: 97 pgs: 97 active+clean; 1.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 1.00k op/s 2024-08-31T06:09:08.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:08 smithi016 ceph-mon[17194]: pgmap v421: 97 pgs: 97 active+clean; 1.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 1.00k op/s 2024-08-31T06:09:08.934 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:09:08.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:08 smithi184 ceph-mon[22331]: pgmap v421: 97 pgs: 97 active+clean; 1.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 1.00k op/s 2024-08-31T06:09:09.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.777+0000 7fa44f400640 1 -- 172.21.15.16:0/308015053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa448103900 msgr2=0x7fa448103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:09.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.777+0000 7fa44f400640 1 --2- 172.21.15.16:0/308015053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa448103900 0x7fa448103d10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa438009900 tx=0x7fa43802a3f0 comp rx=0 tx=0).stop 2024-08-31T06:09:09.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.778+0000 7fa44f400640 1 -- 172.21.15.16:0/308015053 shutdown_connections 2024-08-31T06:09:09.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.778+0000 7fa44f400640 1 --2- 172.21.15.16:0/308015053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa448109240 0x7fa448109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:09.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.778+0000 7fa44f400640 1 --2- 172.21.15.16:0/308015053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa448109b90 0x7fa4481103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:09.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.778+0000 7fa44f400640 1 --2- 172.21.15.16:0/308015053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa448103900 0x7fa448103d10 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:09.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.778+0000 7fa44f400640 1 -- 172.21.15.16:0/308015053 >> 172.21.15.16:0/308015053 conn(0x7fa44807b5f0 msgr2=0x7fa44807ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:09.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.778+0000 7fa44f400640 1 -- 172.21.15.16:0/308015053 shutdown_connections 2024-08-31T06:09:09.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.778+0000 7fa44f400640 1 -- 172.21.15.16:0/308015053 wait complete. 2024-08-31T06:09:09.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.780+0000 7fa44f400640 1 Processor -- start 2024-08-31T06:09:09.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.780+0000 7fa44f400640 1 -- start start 2024-08-31T06:09:09.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.780+0000 7fa44f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa448103900 0x7fa448077000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:09.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.780+0000 7fa44f400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa448109240 0x7fa4480722d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:09.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.780+0000 7fa44f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa448109b90 0x7fa448072a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:09.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.781+0000 7fa44f400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa4481138b0 con 0x7fa448103900 2024-08-31T06:09:09.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.781+0000 7fa44f400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa448113730 con 0x7fa448109b90 2024-08-31T06:09:09.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.781+0000 7fa44f400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa448113a30 con 0x7fa448109240 2024-08-31T06:09:09.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.781+0000 7fa43fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa448109240 0x7fa4480722d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:09.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.781+0000 7fa43fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa448109240 0x7fa4480722d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:42188/0 (socket says 172.21.15.16:42188) 2024-08-31T06:09:09.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.781+0000 7fa43fe00640 1 -- 172.21.15.16:0/3976655669 learned_addr learned my addr 172.21.15.16:0/3976655669 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:09.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.781+0000 7fa44d400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa448109b90 0x7fa448072a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:09.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.781+0000 7fa44ca00640 1 --2- 172.21.15.16:0/3976655669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa448103900 0x7fa448077000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:09.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.782+0000 7fa43fe00640 1 -- 172.21.15.16:0/3976655669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa448109b90 msgr2=0x7fa448072a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:09.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.782+0000 7fa43fe00640 1 --2- 172.21.15.16:0/3976655669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa448109b90 0x7fa448072a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:09.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.782+0000 7fa43fe00640 1 -- 172.21.15.16:0/3976655669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa448103900 msgr2=0x7fa448077000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:09.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.782+0000 7fa43fe00640 1 --2- 172.21.15.16:0/3976655669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa448103900 0x7fa448077000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:09.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.782+0000 7fa43fe00640 1 -- 172.21.15.16:0/3976655669 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4481b0dd0 con 0x7fa448109240 2024-08-31T06:09:09.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.783+0000 7fa43fe00640 1 --2- 172.21.15.16:0/3976655669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa448109240 0x7fa4480722d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa43000d740 tx=0x7fa43000dc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:09.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.783+0000 7fa446000640 1 -- 172.21.15.16:0/3976655669 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa430017070 con 0x7fa448109240 2024-08-31T06:09:09.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.783+0000 7fa446000640 1 -- 172.21.15.16:0/3976655669 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa43000de50 con 0x7fa448109240 2024-08-31T06:09:09.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.783+0000 7fa446000640 1 -- 172.21.15.16:0/3976655669 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43000f620 con 0x7fa448109240 2024-08-31T06:09:09.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.784+0000 7fa44f400640 1 -- 172.21.15.16:0/3976655669 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4481b1060 con 0x7fa448109240 2024-08-31T06:09:09.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.784+0000 7fa44f400640 1 -- 172.21.15.16:0/3976655669 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4481b15a0 con 0x7fa448109240 2024-08-31T06:09:09.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.788+0000 7fa455fe9740 1 Processor -- start 2024-08-31T06:09:09.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.788+0000 7fa455fe9740 1 -- start start 2024-08-31T06:09:09.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.789+0000 7fa455fe9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d8e900070 0x558d8eca58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:09.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.789+0000 7fa455fe9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d8e8d71b0 0x558d8eca5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:09.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.789+0000 7fa455fe9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d8e8df390 0x558d8eca6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:09.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.789+0000 7fa455fe9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558d8ec1c7b0 con 0x558d8e900070 2024-08-31T06:09:09.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.789+0000 7fa455fe9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x558d8ec0bd40 con 0x558d8e8d71b0 2024-08-31T06:09:09.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.789+0000 7fa455fe9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558d8ec0fcf0 con 0x558d8e8df390 2024-08-31T06:09:09.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.789+0000 7fa446000640 1 -- 172.21.15.16:0/3976655669 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fa43001a070 con 0x7fa448109240 2024-08-31T06:09:09.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.789+0000 7fa43fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d8e8df390 0x558d8eca6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:09.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.790+0000 7fa43fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d8e8df390 0x558d8eca6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:42200/0 (socket says 172.21.15.16:42200) 2024-08-31T06:09:09.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.790+0000 7fa43fe00640 1 -- 172.21.15.16:0/1884343876 learned_addr learned my addr 172.21.15.16:0/1884343876 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:09.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.790+0000 7fa44ca00640 1 --2- 172.21.15.16:0/1884343876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d8e8d71b0 0x558d8eca5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:09.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.790+0000 7fa44d400640 1 --2- 172.21.15.16:0/1884343876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d8e900070 0x558d8eca58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:09.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.790+0000 7fa43fe00640 1 -- 172.21.15.16:0/1884343876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d8e8d71b0 msgr2=0x558d8eca5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:09.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.790+0000 7fa43fe00640 1 --2- 172.21.15.16:0/1884343876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d8e8d71b0 0x558d8eca5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:09.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.790+0000 7fa43fe00640 1 -- 172.21.15.16:0/1884343876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d8e900070 msgr2=0x558d8eca58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:09.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.790+0000 7fa43fe00640 1 --2- 172.21.15.16:0/1884343876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d8e900070 0x558d8eca58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:09.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.790+0000 7fa43fe00640 1 -- 172.21.15.16:0/1884343876 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d8e96b650 con 0x558d8e8df390 2024-08-31T06:09:09.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.791+0000 7fa43fe00640 1 --2- 172.21.15.16:0/1884343876 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d8e8df390 0x558d8eca6330 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa4300e2630 tx=0x7fa4300ed570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60216e0464f78401 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:09.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.791+0000 7fa43dc00640 1 -- 172.21.15.16:0/1884343876 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4300f9070 con 0x558d8e8df390 2024-08-31T06:09:09.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.792+0000 7fa43dc00640 1 -- 172.21.15.16:0/1884343876 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4300e8250 con 0x558d8e8df390 2024-08-31T06:09:09.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.792+0000 7fa43dc00640 1 -- 172.21.15.16:0/1884343876 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4300e57d0 con 0x558d8e8df390 2024-08-31T06:09:09.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.792+0000 7fa446000640 1 --2- 172.21.15.16:0/3976655669 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa4280b4da0 0x7fa4280b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:09.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.792+0000 7fa446000640 1 -- 172.21.15.16:0/3976655669 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fa4300e4fa0 con 0x7fa448109240 2024-08-31T06:09:09.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.793+0000 7fa455fe9740 1 -- 172.21.15.16:0/1884343876 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d8e8df390 msgr2=0x558d8eca6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:09.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.793+0000 7fa455fe9740 1 --2- 172.21.15.16:0/1884343876 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d8e8df390 0x558d8eca6330 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa4300e2630 tx=0x7fa4300ed570 comp rx=0 tx=0).stop 2024-08-31T06:09:09.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.793+0000 7fa44d400640 1 --2- 172.21.15.16:0/3976655669 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa4280b4da0 0x7fa4280b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:09.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.793+0000 7fa455fe9740 1 -- 172.21.15.16:0/1884343876 shutdown_connections 2024-08-31T06:09:09.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.793+0000 7fa455fe9740 1 --2- 172.21.15.16:0/1884343876 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d8e8df390 0x558d8eca6330 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:09.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.793+0000 7fa455fe9740 1 --2- 172.21.15.16:0/1884343876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d8e8d71b0 0x558d8eca5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:09.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.793+0000 7fa455fe9740 1 --2- 172.21.15.16:0/1884343876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d8e900070 0x558d8eca58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:09.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.793+0000 7fa455fe9740 1 -- 172.21.15.16:0/1884343876 >> 172.21.15.16:0/1884343876 conn(0x558d8e8f4460 msgr2=0x558d8eca37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:09.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.794+0000 7fa455fe9740 1 -- 172.21.15.16:0/1884343876 shutdown_connections 2024-08-31T06:09:09.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.794+0000 7fa455fe9740 1 -- 172.21.15.16:0/1884343876 wait complete. 2024-08-31T06:09:09.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.795+0000 7fa455fe9740 1 -- start start 2024-08-31T06:09:09.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.796+0000 7fa44d400640 1 --2- 172.21.15.16:0/3976655669 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa4280b4da0 0x7fa4280b72b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa440009fa0 tx=0x7fa44000e040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:09.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.796+0000 7fa455fe9740 1 Processor -- start 2024-08-31T06:09:09.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.797+0000 7fa455fe9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d8e8d71b0 0x558d8ecb9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:09.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.797+0000 7fa455fe9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d8e8df390 0x558d8ecba240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:09.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.797+0000 7fa455fe9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d8e900070 0x558d8ecc5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:09.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.797+0000 7fa43fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d8e8df390 0x558d8ecba240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:09.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.797+0000 7fa455fe9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558d8ea6e700 con 0x558d8e900070 2024-08-31T06:09:09.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.797+0000 7fa455fe9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x558d8eb92cf0 con 0x558d8e8df390 2024-08-31T06:09:09.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.797+0000 7fa455fe9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558d8ecbc7c0 con 0x558d8e8d71b0 2024-08-31T06:09:09.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.797+0000 7fa43fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d8e8df390 0x558d8ecba240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:36174/0 (socket says 172.21.15.16:36174) 2024-08-31T06:09:09.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.797+0000 7fa43fe00640 1 -- 172.21.15.16:0/3053052800 learned_addr learned my addr 172.21.15.16:0/3053052800 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:09.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.797+0000 7fa455fe9740 10 client.0 _ll_register_callbacks cb 0x558d8ebb8470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:09:09.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.798+0000 7fa44d400640 1 --2- 172.21.15.16:0/3053052800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d8e900070 0x558d8ecc5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:09.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.798+0000 7fa44ca00640 1 --2- 172.21.15.16:0/3053052800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d8e8d71b0 0x558d8ecb9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:09.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.798+0000 7fa43fe00640 1 -- 172.21.15.16:0/3053052800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d8e8d71b0 msgr2=0x558d8ecb9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:09.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.798+0000 7fa43fe00640 1 --2- 172.21.15.16:0/3053052800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d8e8d71b0 0x558d8ecb9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:09.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.798+0000 7fa43fe00640 1 -- 172.21.15.16:0/3053052800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d8e900070 msgr2=0x558d8ecc5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:09.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.798+0000 7fa43fe00640 1 --2- 172.21.15.16:0/3053052800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d8e900070 0x558d8ecc5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:09.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.798+0000 7fa43fe00640 1 -- 172.21.15.16:0/3053052800 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d8ebd38b0 con 0x558d8e8df390 2024-08-31T06:09:09.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.799+0000 7fa43fe00640 1 --2- 172.21.15.16:0/3053052800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d8e8df390 0x558d8ecba240 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa4300ab390 tx=0x7fa430005cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f102f3feadb98c6b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:09.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.799+0000 7fa436000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:09:09.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.799+0000 7fa44ca00640 1 --2- 172.21.15.16:0/3053052800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d8e8d71b0 0x558d8ecb9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:09:09.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.799+0000 7fa436000640 1 -- 172.21.15.16:0/3053052800 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa430021030 con 0x558d8e8df390 2024-08-31T06:09:09.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.799+0000 7fa436000640 1 -- 172.21.15.16:0/3053052800 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fa4300a7c80 con 0x558d8e8df390 2024-08-31T06:09:09.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.799+0000 7fa455fe9740 1 -- 172.21.15.16:0/3053052800 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4300e8250 con 0x558d8e8df390 2024-08-31T06:09:09.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.799+0000 7fa436000640 1 -- 172.21.15.16:0/3053052800 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa4300a7550 con 0x558d8e8df390 2024-08-31T06:09:09.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.799+0000 7fa455fe9740 10 client.26018 fetch_fsmap learned FSMap version 31 2024-08-31T06:09:09.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.799+0000 7fa455fe9740 1 -- 172.21.15.16:0/3053052800 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558d8ecc8820 con 0x558d8e8df390 2024-08-31T06:09:09.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.800+0000 7fa436000640 1 -- 172.21.15.16:0/3053052800 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4300e8250 con 0x558d8e8df390 2024-08-31T06:09:09.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.800+0000 7fa436000640 1 -- 172.21.15.16:0/3053052800 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fa4300a7920 con 0x558d8e8df390 2024-08-31T06:09:09.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.800+0000 7fa455fe9740 10 client.26018 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:09:09.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.800+0000 7fa455fe9740 10 client.26018 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:09:09.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.800+0000 7fa455fe9740 20 client.26018 populate_metadata read hostname 'smithi016' 2024-08-31T06:09:09.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.800+0000 7fa455fe9740 1 --2- 172.21.15.16:0/3053052800 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x558d8e8be9f0 0x558d8eccbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:09.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.800+0000 7fa455fe9740 4 client.26018 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:09:09.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.800+0000 7fa455fe9740 1 -- 172.21.15.16:0/3053052800 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d8ec1e700 con 0x558d8e8be9f0 2024-08-31T06:09:09.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.801+0000 7fa44ca00640 1 --2- 172.21.15.16:0/3053052800 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x558d8e8be9f0 0x558d8eccbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:09.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.801+0000 7fa436000640 10 client.26018 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:09:09.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.801+0000 7fa44ca00640 1 --2- 172.21.15.16:0/3053052800 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x558d8e8be9f0 0x558d8eccbb80 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eef5f4566f24abc1 server_cookie=b457c6e60c29f6a9 in_seq=0 out_seq=0 2024-08-31T06:09:09.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.802+0000 7fa436000640 1 -- 172.21.15.16:0/3053052800 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558d8ec1e700 con 0x558d8e8be9f0 2024-08-31T06:09:09.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.802+0000 7fa436000640 10 client.26018 handle_command_reply: tid=0 2024-08-31T06:09:09.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.802+0000 7fa455fe9740 1 client.26018 shutdown 2024-08-31T06:09:09.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.802+0000 7fa455fe9740 1 -- 172.21.15.16:0/3053052800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d8e8df390 msgr2=0x558d8ecba240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:09.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.802+0000 7fa455fe9740 1 --2- 172.21.15.16:0/3053052800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d8e8df390 0x558d8ecba240 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa4300ab390 tx=0x7fa430005cc0 comp rx=0 tx=0).stop 2024-08-31T06:09:09.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.803+0000 7fa455fe9740 1 -- 172.21.15.16:0/3053052800 shutdown_connections 2024-08-31T06:09:09.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.803+0000 7fa455fe9740 1 --2- 172.21.15.16:0/3053052800 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x558d8e8be9f0 0x558d8eccbb80 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:09.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.803+0000 7fa436000640 0 client.26018 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:09:09.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.803+0000 7fa455fe9740 1 --2- 172.21.15.16:0/3053052800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d8e900070 0x558d8ecc5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:09.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.803+0000 7fa455fe9740 1 --2- 172.21.15.16:0/3053052800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d8e8df390 0x558d8ecba240 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:09.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.803+0000 7fa455fe9740 1 --2- 172.21.15.16:0/3053052800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d8e8d71b0 0x558d8ecb9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:09.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.803+0000 7fa455fe9740 1 -- 172.21.15.16:0/3053052800 >> 172.21.15.16:0/3053052800 conn(0x558d8e8f4460 msgr2=0x558d8eca1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:09.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.803+0000 7fa455fe9740 1 -- 172.21.15.16:0/3053052800 shutdown_connections 2024-08-31T06:09:09.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.803+0000 7fa455fe9740 1 -- 172.21.15.16:0/3053052800 wait complete. 2024-08-31T06:09:09.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.803+0000 7fa455fe9740 20 client.26018 trim_cache size 0 max 16384 2024-08-31T06:09:09.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.826+0000 7fa455fe9740 1 Processor -- start 2024-08-31T06:09:09.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.826+0000 7fa455fe9740 1 -- start start 2024-08-31T06:09:09.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.826+0000 7fa455fe9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d8e8d71b0 0x558d8eca37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:09.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.826+0000 7fa455fe9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d8e8df390 0x558d8eca5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:09.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.826+0000 7fa455fe9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d8e900070 0x558d8eca6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:09.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.826+0000 7fa455fe9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa4481b8550 con 0x558d8e900070 2024-08-31T06:09:09.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.826+0000 7fa455fe9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa4481b5d60 con 0x558d8e8d71b0 2024-08-31T06:09:09.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.826+0000 7fa455fe9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa4481b3bb0 con 0x558d8e8df390 2024-08-31T06:09:09.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.827+0000 7fa44ca00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d8e8d71b0 0x558d8eca37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:09.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.827+0000 7fa44ca00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d8e8d71b0 0x558d8eca37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:36190/0 (socket says 172.21.15.16:36190) 2024-08-31T06:09:09.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.827+0000 7fa44ca00640 1 -- 172.21.15.16:0/2386364795 learned_addr learned my addr 172.21.15.16:0/2386364795 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:09.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.827+0000 7fa43fe00640 1 --2- 172.21.15.16:0/2386364795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d8e8df390 0x558d8eca5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:09.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.827+0000 7fa44d400640 1 --2- 172.21.15.16:0/2386364795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d8e900070 0x558d8eca6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:09.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.828+0000 7fa43fe00640 1 -- 172.21.15.16:0/2386364795 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d8e8d71b0 msgr2=0x558d8eca37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:09.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.828+0000 7fa43fe00640 1 --2- 172.21.15.16:0/2386364795 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d8e8d71b0 0x558d8eca37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:09.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.828+0000 7fa43fe00640 1 -- 172.21.15.16:0/2386364795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d8e900070 msgr2=0x558d8eca6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:09.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.828+0000 7fa43fe00640 1 --2- 172.21.15.16:0/2386364795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d8e900070 0x558d8eca6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:09.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.828+0000 7fa43fe00640 1 -- 172.21.15.16:0/2386364795 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d8eca2e60 con 0x558d8e8df390 2024-08-31T06:09:09.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.828+0000 7fa43fe00640 1 --2- 172.21.15.16:0/2386364795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d8e8df390 0x558d8eca5de0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa4300a8900 tx=0x7fa4300edfb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2af798951ea70971 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:09.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.828+0000 7fa44d400640 1 --2- 172.21.15.16:0/2386364795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d8e900070 0x558d8eca6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:09:09.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.829+0000 7fa43d200640 1 -- 172.21.15.16:0/2386364795 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa430021070 con 0x558d8e8df390 2024-08-31T06:09:09.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.829+0000 7fa43d200640 1 -- 172.21.15.16:0/2386364795 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4300b42f0 con 0x558d8e8df390 2024-08-31T06:09:09.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.829+0000 7fa43d200640 1 -- 172.21.15.16:0/2386364795 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4300aae00 con 0x558d8e8df390 2024-08-31T06:09:09.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.830+0000 7fa455fe9740 1 -- 172.21.15.16:0/2386364795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d8e8df390 msgr2=0x558d8eca5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:09.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.830+0000 7fa455fe9740 1 --2- 172.21.15.16:0/2386364795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d8e8df390 0x558d8eca5de0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa4300a8900 tx=0x7fa4300edfb0 comp rx=0 tx=0).stop 2024-08-31T06:09:09.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.830+0000 7fa455fe9740 1 -- 172.21.15.16:0/2386364795 shutdown_connections 2024-08-31T06:09:09.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.830+0000 7fa455fe9740 1 --2- 172.21.15.16:0/2386364795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d8e8df390 0x558d8eca5de0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:09.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.830+0000 7fa455fe9740 1 --2- 172.21.15.16:0/2386364795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d8e900070 0x558d8eca6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:09.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.830+0000 7fa455fe9740 1 --2- 172.21.15.16:0/2386364795 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d8e8d71b0 0x558d8eca37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:09.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.830+0000 7fa455fe9740 1 -- 172.21.15.16:0/2386364795 >> 172.21.15.16:0/2386364795 conn(0x558d8e8f4460 msgr2=0x558d8eca30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:09.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.830+0000 7fa455fe9740 1 -- 172.21.15.16:0/2386364795 shutdown_connections 2024-08-31T06:09:09.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.830+0000 7fa455fe9740 1 -- 172.21.15.16:0/2386364795 wait complete. 2024-08-31T06:09:09.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.831+0000 7fa455fe9740 1 -- start start 2024-08-31T06:09:09.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.831+0000 7fa455fe9740 1 Processor -- start 2024-08-31T06:09:09.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.831+0000 7fa455fe9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d8e8d71b0 0x558d8eccebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:09.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.831+0000 7fa455fe9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d8e8df390 0x558d8eccf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:09.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.831+0000 7fa455fe9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d8e900070 0x558d8eccbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:09.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.831+0000 7fa455fe9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa4481b4220 con 0x558d8e8df390 2024-08-31T06:09:09.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.831+0000 7fa455fe9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa4481b5690 con 0x558d8e900070 2024-08-31T06:09:09.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.831+0000 7fa455fe9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa4481b48f0 con 0x558d8e8d71b0 2024-08-31T06:09:09.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.831+0000 7fa455fe9740 10 client.0 _ll_register_callbacks cb 0x7fa430009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:09:09.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.832+0000 7fa44d400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d8e900070 0x558d8eccbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:09.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.832+0000 7fa44ca00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d8e8d71b0 0x558d8eccebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:09.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.832+0000 7fa44d400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d8e900070 0x558d8eccbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:36198/0 (socket says 172.21.15.16:36198) 2024-08-31T06:09:09.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.832+0000 7fa44d400640 1 -- 172.21.15.16:0/3149885178 learned_addr learned my addr 172.21.15.16:0/3149885178 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:09.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.832+0000 7fa43fe00640 1 --2- 172.21.15.16:0/3149885178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d8e8df390 0x558d8eccf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:09.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.832+0000 7fa44d400640 1 -- 172.21.15.16:0/3149885178 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d8e8d71b0 msgr2=0x558d8eccebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:09.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.832+0000 7fa44d400640 1 --2- 172.21.15.16:0/3149885178 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d8e8d71b0 0x558d8eccebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:09.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.832+0000 7fa44d400640 1 -- 172.21.15.16:0/3149885178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d8e8df390 msgr2=0x558d8eccf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:09.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.832+0000 7fa44d400640 1 --2- 172.21.15.16:0/3149885178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d8e8df390 0x558d8eccf0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:09.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.832+0000 7fa44d400640 1 -- 172.21.15.16:0/3149885178 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d8eccf8c0 con 0x558d8e900070 2024-08-31T06:09:09.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.833+0000 7fa44d400640 1 --2- 172.21.15.16:0/3149885178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d8e900070 0x558d8eccbb80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa440019fb0 tx=0x7fa440004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6853e9a118d09ecc server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.833+0000 7fa436000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:09:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.833+0000 7fa436000640 1 -- 172.21.15.16:0/3149885178 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa440025610 con 0x558d8e900070 2024-08-31T06:09:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.833+0000 7fa436000640 1 -- 172.21.15.16:0/3149885178 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fa4400257b0 con 0x558d8e900070 2024-08-31T06:09:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.833+0000 7fa455fe9740 1 -- 172.21.15.16:0/3149885178 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558d8ecba5d0 con 0x558d8e900070 2024-08-31T06:09:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.835+0000 7fa436000640 1 -- 172.21.15.16:0/3149885178 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa4400259b0 con 0x558d8e900070 2024-08-31T06:09:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.835+0000 7fa455fe9740 10 client.26024 fetch_fsmap learned FSMap version 31 2024-08-31T06:09:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.835+0000 7fa455fe9740 1 -- 172.21.15.16:0/3149885178 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558d8ed36810 con 0x558d8e900070 2024-08-31T06:09:09.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.835+0000 7fa436000640 1 -- 172.21.15.16:0/3149885178 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x558d8ecba5d0 con 0x558d8e900070 2024-08-31T06:09:09.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.836+0000 7fa436000640 1 -- 172.21.15.16:0/3149885178 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fa440016070 con 0x558d8e900070 2024-08-31T06:09:09.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.836+0000 7fa455fe9740 10 client.26024 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:09:09.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.836+0000 7fa455fe9740 10 client.26024 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:09:09.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.836+0000 7fa455fe9740 20 client.26024 populate_metadata read hostname 'smithi016' 2024-08-31T06:09:09.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.836+0000 7fa455fe9740 1 --2- 172.21.15.16:0/3149885178 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x558d8e893da0 0x558d8ed390e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:09.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.836+0000 7fa455fe9740 4 client.26024 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:09:09.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.836+0000 7fa455fe9740 1 -- 172.21.15.16:0/3149885178 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558d8ec1a1a0 con 0x558d8e893da0 2024-08-31T06:09:09.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.837+0000 7fa44ca00640 1 --2- 172.21.15.16:0/3149885178 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x558d8e893da0 0x558d8ed390e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:09.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.837+0000 7fa44ca00640 1 --2- 172.21.15.16:0/3149885178 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x558d8e893da0 0x558d8ed390e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afe317670ca13a61 server_cookie=d197728a1567215e in_seq=0 out_seq=0 2024-08-31T06:09:09.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.837+0000 7fa436000640 10 client.26024 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:09:09.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.838+0000 7fa436000640 1 -- 172.21.15.16:0/3149885178 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558d8ec1a1a0 con 0x558d8e893da0 2024-08-31T06:09:09.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.838+0000 7fa436000640 10 client.26024 handle_command_reply: tid=0 2024-08-31T06:09:09.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.838+0000 7fa455fe9740 1 client.26024 shutdown 2024-08-31T06:09:09.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.838+0000 7fa455fe9740 1 -- 172.21.15.16:0/3149885178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d8e900070 msgr2=0x558d8eccbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:09.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.838+0000 7fa455fe9740 1 --2- 172.21.15.16:0/3149885178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d8e900070 0x558d8eccbb80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa440019fb0 tx=0x7fa440004a40 comp rx=0 tx=0).stop 2024-08-31T06:09:09.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.839+0000 7fa455fe9740 1 -- 172.21.15.16:0/3149885178 shutdown_connections 2024-08-31T06:09:09.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.839+0000 7fa455fe9740 1 --2- 172.21.15.16:0/3149885178 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x558d8e893da0 0x558d8ed390e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:09.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.839+0000 7fa455fe9740 1 --2- 172.21.15.16:0/3149885178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558d8e900070 0x558d8eccbb80 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:09.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.839+0000 7fa455fe9740 1 --2- 172.21.15.16:0/3149885178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d8e8df390 0x558d8eccf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:09.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.839+0000 7fa455fe9740 1 --2- 172.21.15.16:0/3149885178 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558d8e8d71b0 0x558d8eccebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:09.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.839+0000 7fa436000640 0 client.26024 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:09:09.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.839+0000 7fa455fe9740 1 -- 172.21.15.16:0/3149885178 >> 172.21.15.16:0/3149885178 conn(0x558d8e8f4460 msgr2=0x558d8eca68e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:09.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.839+0000 7fa455fe9740 1 -- 172.21.15.16:0/3149885178 shutdown_connections 2024-08-31T06:09:09.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.839+0000 7fa455fe9740 1 -- 172.21.15.16:0/3149885178 wait complete. 2024-08-31T06:09:09.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.839+0000 7fa455fe9740 20 client.26024 trim_cache size 0 max 16384 2024-08-31T06:09:09.843 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:09:09.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.841+0000 7fa44f400640 1 -- 172.21.15.16:0/3976655669 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa4280b4da0 msgr2=0x7fa4280b72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:09.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.841+0000 7fa44f400640 1 --2- 172.21.15.16:0/3976655669 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa4280b4da0 0x7fa4280b72b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa440009fa0 tx=0x7fa44000e040 comp rx=0 tx=0).stop 2024-08-31T06:09:09.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.841+0000 7fa44f400640 1 -- 172.21.15.16:0/3976655669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa448109240 msgr2=0x7fa4480722d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:09.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.841+0000 7fa44f400640 1 --2- 172.21.15.16:0/3976655669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa448109240 0x7fa4480722d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa43000d740 tx=0x7fa43000dc00 comp rx=0 tx=0).stop 2024-08-31T06:09:09.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.842+0000 7fa44f400640 1 -- 172.21.15.16:0/3976655669 shutdown_connections 2024-08-31T06:09:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.842+0000 7fa44f400640 1 --2- 172.21.15.16:0/3976655669 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa4280b4da0 0x7fa4280b72b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.842+0000 7fa44f400640 1 --2- 172.21.15.16:0/3976655669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa448109240 0x7fa4480722d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.842+0000 7fa44f400640 1 --2- 172.21.15.16:0/3976655669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa448109b90 0x7fa448072a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.842+0000 7fa44f400640 1 --2- 172.21.15.16:0/3976655669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa448103900 0x7fa448077000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.842+0000 7fa44f400640 1 -- 172.21.15.16:0/3976655669 >> 172.21.15.16:0/3976655669 conn(0x7fa44807b5f0 msgr2=0x7fa4481072d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.842+0000 7fa44f400640 1 -- 172.21.15.16:0/3976655669 shutdown_connections 2024-08-31T06:09:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:09.842+0000 7fa44f400640 1 -- 172.21.15.16:0/3976655669 wait complete. 2024-08-31T06:09:10.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:10 smithi155 ceph-mon[21742]: pgmap v422: 97 pgs: 97 active+clean; 1.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 27 MiB/s wr, 1.53k op/s 2024-08-31T06:09:10.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:10 smithi016 ceph-mon[17194]: pgmap v422: 97 pgs: 97 active+clean; 1.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 27 MiB/s wr, 1.53k op/s 2024-08-31T06:09:10.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:10 smithi184 ceph-mon[22331]: pgmap v422: 97 pgs: 97 active+clean; 1.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 27 MiB/s wr, 1.53k op/s 2024-08-31T06:09:11.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 105.0 s, 2380.3 tps, lat 43.963 ms stddev 56.241 2024-08-31T06:09:11.794 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T06:09:11.795 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T06:09:12.439 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:09:12.824 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:12 smithi016 ceph-mon[17194]: pgmap v423: 97 pgs: 97 active+clean; 1.4 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 31 MiB/s wr, 1.69k op/s 2024-08-31T06:09:12.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:12 smithi155 ceph-mon[21742]: pgmap v423: 97 pgs: 97 active+clean; 1.4 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 31 MiB/s wr, 1.69k op/s 2024-08-31T06:09:12.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:12 smithi184 ceph-mon[22331]: pgmap v423: 97 pgs: 97 active+clean; 1.4 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 31 MiB/s wr, 1.69k op/s 2024-08-31T06:09:13.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.467+0000 7f0eefe00640 1 -- 172.21.15.16:0/3127854721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ef01099e0 msgr2=0x7f0ef0110210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:13.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.467+0000 7f0eefe00640 1 --2- 172.21.15.16:0/3127854721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ef01099e0 0x7f0ef0110210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0edc009900 tx=0x7f0edc02a3f0 comp rx=0 tx=0).stop 2024-08-31T06:09:13.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.468+0000 7f0eefe00640 1 -- 172.21.15.16:0/3127854721 shutdown_connections 2024-08-31T06:09:13.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.468+0000 7f0eefe00640 1 --2- 172.21.15.16:0/3127854721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ef01099e0 0x7f0ef0110210 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:13.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.468+0000 7f0eefe00640 1 --2- 172.21.15.16:0/3127854721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ef0109090 0x7f0ef01094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:13.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.468+0000 7f0eefe00640 1 --2- 172.21.15.16:0/3127854721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ef0103750 0x7f0ef0103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:13.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.468+0000 7f0eefe00640 1 -- 172.21.15.16:0/3127854721 >> 172.21.15.16:0/3127854721 conn(0x7f0ef00ff1e0 msgr2=0x7f0ef0101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:13.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.468+0000 7f0eefe00640 1 -- 172.21.15.16:0/3127854721 shutdown_connections 2024-08-31T06:09:13.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.469+0000 7f0eefe00640 1 -- 172.21.15.16:0/3127854721 wait complete. 2024-08-31T06:09:13.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.470+0000 7f0eefe00640 1 Processor -- start 2024-08-31T06:09:13.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.470+0000 7f0eefe00640 1 -- start start 2024-08-31T06:09:13.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.471+0000 7f0eefe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ef0103750 0x7f0ef0197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:13.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.471+0000 7f0eefe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ef0109090 0x7f0ef0198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:13.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.471+0000 7f0eefe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ef01099e0 0x7f0ef019d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:13.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.472+0000 7f0eefe00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef01114e0 con 0x7f0ef01099e0 2024-08-31T06:09:13.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.472+0000 7f0eefe00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef0111360 con 0x7f0ef0103750 2024-08-31T06:09:13.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.472+0000 7f0eefe00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef0111660 con 0x7f0ef0109090 2024-08-31T06:09:13.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.472+0000 7f0eeea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ef0103750 0x7f0ef0197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:13.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.472+0000 7f0eeea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ef0103750 0x7f0ef0197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:36224/0 (socket says 172.21.15.16:36224) 2024-08-31T06:09:13.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.472+0000 7f0eeea00640 1 -- 172.21.15.16:0/1594018613 learned_addr learned my addr 172.21.15.16:0/1594018613 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:13.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.472+0000 7f0eee000640 1 --2- 172.21.15.16:0/1594018613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ef0109090 0x7f0ef0198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:13.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.472+0000 7f0eee000640 1 -- 172.21.15.16:0/1594018613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ef0103750 msgr2=0x7f0ef0197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:13.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.472+0000 7f0eee000640 1 --2- 172.21.15.16:0/1594018613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ef0103750 0x7f0ef0197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:13.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.472+0000 7f0eee000640 1 -- 172.21.15.16:0/1594018613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ef01099e0 msgr2=0x7f0ef019d6d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-31T06:09:13.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.473+0000 7f0eee000640 1 --2- 172.21.15.16:0/1594018613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ef01099e0 0x7f0ef019d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:13.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.473+0000 7f0eee000640 1 -- 172.21.15.16:0/1594018613 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef019de00 con 0x7f0ef0109090 2024-08-31T06:09:13.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.473+0000 7f0eee000640 1 --2- 172.21.15.16:0/1594018613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ef0109090 0x7f0ef0198430 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0ef006b730 tx=0x7f0ee000db30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:13.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.473+0000 7f0ee7400640 1 -- 172.21.15.16:0/1594018613 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee0017070 con 0x7f0ef0109090 2024-08-31T06:09:13.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.474+0000 7f0ee7400640 1 -- 172.21.15.16:0/1594018613 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ee0009820 con 0x7f0ef0109090 2024-08-31T06:09:13.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.474+0000 7f0ee7400640 1 -- 172.21.15.16:0/1594018613 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee0004b90 con 0x7f0ef0109090 2024-08-31T06:09:13.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.474+0000 7f0eefe00640 1 -- 172.21.15.16:0/1594018613 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ef019e0f0 con 0x7f0ef0109090 2024-08-31T06:09:13.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.474+0000 7f0eefe00640 1 -- 172.21.15.16:0/1594018613 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ef01a4850 con 0x7f0ef0109090 2024-08-31T06:09:13.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.479+0000 7f0efc7db740 1 Processor -- start 2024-08-31T06:09:13.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.479+0000 7f0efc7db740 1 -- start start 2024-08-31T06:09:13.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.480+0000 7f0efc7db740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cddaf0a350 0x55cddb2b8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:13.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.480+0000 7f0efc7db740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cddaee1490 0x55cddb2b9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:13.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.480+0000 7f0efc7db740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddaee9670 0x55cddb2b9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:13.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.480+0000 7f0efc7db740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cddb226750 con 0x55cddaee1490 2024-08-31T06:09:13.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.480+0000 7f0efc7db740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55cddb215ce0 con 0x55cddaee9670 2024-08-31T06:09:13.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.480+0000 7f0efc7db740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55cddb219c90 con 0x55cddaf0a350 2024-08-31T06:09:13.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.480+0000 7f0eef400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cddaf0a350 0x55cddb2b8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:13.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.480+0000 7f0ee7400640 1 -- 172.21.15.16:0/1594018613 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f0ee0004d30 con 0x7f0ef0109090 2024-08-31T06:09:13.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.480+0000 7f0eef400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cddaf0a350 0x55cddb2b8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:42244/0 (socket says 172.21.15.16:42244) 2024-08-31T06:09:13.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.480+0000 7f0eef400640 1 -- 172.21.15.16:0/2975043973 learned_addr learned my addr 172.21.15.16:0/2975043973 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:13.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.481+0000 7f0eeea00640 1 --2- 172.21.15.16:0/2975043973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cddaee1490 0x55cddb2b9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:13.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.481+0000 7f0eef400640 1 -- 172.21.15.16:0/2975043973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddaee9670 msgr2=0x55cddb2b9a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:09:13.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.481+0000 7f0eef400640 1 --2- 172.21.15.16:0/2975043973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddaee9670 0x55cddb2b9a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:13.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.481+0000 7f0eef400640 1 -- 172.21.15.16:0/2975043973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cddaee1490 msgr2=0x55cddb2b9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:13.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.481+0000 7f0eef400640 1 --2- 172.21.15.16:0/2975043973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cddaee1490 0x55cddb2b9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:13.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.481+0000 7f0eef400640 1 -- 172.21.15.16:0/2975043973 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cddaf75930 con 0x55cddaf0a350 2024-08-31T06:09:13.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.481+0000 7f0eef400640 1 --2- 172.21.15.16:0/2975043973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cddaf0a350 0x55cddb2b8fe0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0edc0040f0 tx=0x7f0edc004120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c26e9c73ce0babd0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:13.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.482+0000 7f0edbe00640 1 -- 172.21.15.16:0/2975043973 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0edc03e030 con 0x55cddaf0a350 2024-08-31T06:09:13.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.482+0000 7f0edbe00640 1 -- 172.21.15.16:0/2975043973 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0edc0042c0 con 0x55cddaf0a350 2024-08-31T06:09:13.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.482+0000 7f0edbe00640 1 -- 172.21.15.16:0/2975043973 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0edc035040 con 0x55cddaf0a350 2024-08-31T06:09:13.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.483+0000 7f0efc7db740 1 -- 172.21.15.16:0/2975043973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cddaf0a350 msgr2=0x55cddb2b8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:13.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.483+0000 7f0efc7db740 1 --2- 172.21.15.16:0/2975043973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cddaf0a350 0x55cddb2b8fe0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0edc0040f0 tx=0x7f0edc004120 comp rx=0 tx=0).stop 2024-08-31T06:09:13.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.485+0000 7f0ee7400640 1 --2- 172.21.15.16:0/1594018613 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f0ed40b4d20 0x7f0ed40b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:13.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.485+0000 7f0eef400640 1 --2- 172.21.15.16:0/1594018613 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f0ed40b4d20 0x7f0ed40b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:13.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.486+0000 7f0ee7400640 1 -- 172.21.15.16:0/1594018613 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f0ee00e4d90 con 0x7f0ef0109090 2024-08-31T06:09:13.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.486+0000 7f0efc7db740 1 -- 172.21.15.16:0/2975043973 shutdown_connections 2024-08-31T06:09:13.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.486+0000 7f0efc7db740 1 --2- 172.21.15.16:0/2975043973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddaee9670 0x55cddb2b9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:13.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.486+0000 7f0efc7db740 1 --2- 172.21.15.16:0/2975043973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cddaee1490 0x55cddb2b9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:13.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.486+0000 7f0efc7db740 1 --2- 172.21.15.16:0/2975043973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cddaf0a350 0x55cddb2b8fe0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:13.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.486+0000 7f0efc7db740 1 -- 172.21.15.16:0/2975043973 >> 172.21.15.16:0/2975043973 conn(0x55cddaefe740 msgr2=0x55cddb2b6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:13.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.487+0000 7f0eef400640 1 --2- 172.21.15.16:0/1594018613 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f0ed40b4d20 0x7f0ed40b7230 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0edc0040f0 tx=0x7f0edc005df0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:13.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.487+0000 7f0efc7db740 1 -- 172.21.15.16:0/2975043973 shutdown_connections 2024-08-31T06:09:13.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.487+0000 7f0efc7db740 1 -- 172.21.15.16:0/2975043973 wait complete. 2024-08-31T06:09:13.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.488+0000 7f0efc7db740 1 -- start start 2024-08-31T06:09:13.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.488+0000 7f0efc7db740 1 Processor -- start 2024-08-31T06:09:13.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.489+0000 7f0efc7db740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddaee1490 0x55cddb2c4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:13.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.489+0000 7f0efc7db740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cddaee9670 0x55cddb2c51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:13.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.489+0000 7f0efc7db740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cddaf0a350 0x55cddb2d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:13.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.489+0000 7f0efc7db740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cddb078890 con 0x55cddaf0a350 2024-08-31T06:09:13.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.489+0000 7f0efc7db740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55cddb19ccd0 con 0x55cddaee1490 2024-08-31T06:09:13.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.489+0000 7f0efc7db740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55cddb2c7760 con 0x55cddaee9670 2024-08-31T06:09:13.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.489+0000 7f0efc7db740 10 client.0 _ll_register_callbacks cb 0x55cddb210ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:09:13.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.489+0000 7f0eeea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddaee1490 0x55cddb2c4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:13.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.489+0000 7f0eeea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddaee1490 0x55cddb2c4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:36238/0 (socket says 172.21.15.16:36238) 2024-08-31T06:09:13.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.489+0000 7f0eeea00640 1 -- 172.21.15.16:0/3615128949 learned_addr learned my addr 172.21.15.16:0/3615128949 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:13.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.489+0000 7f0eee000640 1 --2- 172.21.15.16:0/3615128949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cddaee9670 0x55cddb2c51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:13.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.490+0000 7f0eeea00640 1 -- 172.21.15.16:0/3615128949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cddaee9670 msgr2=0x55cddb2c51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:13.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.490+0000 7f0eeea00640 1 --2- 172.21.15.16:0/3615128949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cddaee9670 0x55cddb2c51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:13.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.490+0000 7f0eeea00640 1 -- 172.21.15.16:0/3615128949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cddaf0a350 msgr2=0x55cddb2d0a50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:09:13.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.490+0000 7f0eeea00640 1 --2- 172.21.15.16:0/3615128949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cddaf0a350 0x55cddb2d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:13.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.490+0000 7f0eeea00640 1 -- 172.21.15.16:0/3615128949 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cddb1e61b0 con 0x55cddaee1490 2024-08-31T06:09:13.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.490+0000 7f0eeea00640 1 --2- 172.21.15.16:0/3615128949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddaee1490 0x55cddb2c4c10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8010aa0 tx=0x7f0ee8010f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50449bd9cf96f3f3 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:13.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.490+0000 7f0ed8c00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:09:13.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.490+0000 7f0ed8c00640 1 -- 172.21.15.16:0/3615128949 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee800ee90 con 0x55cddaee1490 2024-08-31T06:09:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.490+0000 7f0ed8c00640 1 -- 172.21.15.16:0/3615128949 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f0ee8004b20 con 0x55cddaee1490 2024-08-31T06:09:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.491+0000 7f0efc7db740 1 -- 172.21.15.16:0/3615128949 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0edc0042c0 con 0x55cddaee1490 2024-08-31T06:09:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.491+0000 7f0ed8c00640 1 -- 172.21.15.16:0/3615128949 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0ee8004d20 con 0x55cddaee1490 2024-08-31T06:09:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.491+0000 7f0efc7db740 10 client.26030 fetch_fsmap learned FSMap version 31 2024-08-31T06:09:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.491+0000 7f0efc7db740 1 -- 172.21.15.16:0/3615128949 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55cddb2d3850 con 0x55cddaee1490 2024-08-31T06:09:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.491+0000 7f0ed8c00640 1 -- 172.21.15.16:0/3615128949 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0edc0042c0 con 0x55cddaee1490 2024-08-31T06:09:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.492+0000 7f0ed8c00640 1 -- 172.21.15.16:0/3615128949 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f0ee801b640 con 0x55cddaee1490 2024-08-31T06:09:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.492+0000 7f0efc7db740 10 client.26030 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:09:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.492+0000 7f0efc7db740 10 client.26030 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:09:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.492+0000 7f0efc7db740 20 client.26030 populate_metadata read hostname 'smithi016' 2024-08-31T06:09:13.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.492+0000 7f0efc7db740 1 --2- 172.21.15.16:0/3615128949 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55cddaec8cd0 0x55cddb2d6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:13.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.492+0000 7f0efc7db740 4 client.26030 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:09:13.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.492+0000 7f0efc7db740 1 -- 172.21.15.16:0/3615128949 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cddb2286a0 con 0x55cddaec8cd0 2024-08-31T06:09:13.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.493+0000 7f0eee000640 1 --2- 172.21.15.16:0/3615128949 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55cddaec8cd0 0x55cddb2d6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:13.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.494+0000 7f0eee000640 1 --2- 172.21.15.16:0/3615128949 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55cddaec8cd0 0x55cddb2d6e50 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93330d5eac9b9b59 server_cookie=ccef86a94f290c33 in_seq=0 out_seq=0 2024-08-31T06:09:13.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.494+0000 7f0ed8c00640 10 client.26030 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:09:13.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.494+0000 7f0ed8c00640 1 -- 172.21.15.16:0/3615128949 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cddb2286a0 con 0x55cddaec8cd0 2024-08-31T06:09:13.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.494+0000 7f0ed8c00640 10 client.26030 handle_command_reply: tid=0 2024-08-31T06:09:13.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.495+0000 7f0efc7db740 1 client.26030 shutdown 2024-08-31T06:09:13.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.495+0000 7f0efc7db740 1 -- 172.21.15.16:0/3615128949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddaee1490 msgr2=0x55cddb2c4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:13.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.495+0000 7f0efc7db740 1 --2- 172.21.15.16:0/3615128949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddaee1490 0x55cddb2c4c10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8010aa0 tx=0x7f0ee8010f60 comp rx=0 tx=0).stop 2024-08-31T06:09:13.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.495+0000 7f0efc7db740 1 -- 172.21.15.16:0/3615128949 shutdown_connections 2024-08-31T06:09:13.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.495+0000 7f0efc7db740 1 --2- 172.21.15.16:0/3615128949 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55cddaec8cd0 0x55cddb2d6e50 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:13.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.495+0000 7f0efc7db740 1 --2- 172.21.15.16:0/3615128949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cddaee9670 0x55cddb2c51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:13.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.495+0000 7f0efc7db740 1 --2- 172.21.15.16:0/3615128949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cddaf0a350 0x55cddb2d0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:13.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.495+0000 7f0ed8c00640 0 client.26030 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:09:13.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.495+0000 7f0efc7db740 1 --2- 172.21.15.16:0/3615128949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddaee1490 0x55cddb2c4c10 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:13.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.495+0000 7f0efc7db740 1 -- 172.21.15.16:0/3615128949 >> 172.21.15.16:0/3615128949 conn(0x55cddaefe740 msgr2=0x55cddb2bc170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:13.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.496+0000 7f0efc7db740 1 -- 172.21.15.16:0/3615128949 shutdown_connections 2024-08-31T06:09:13.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.496+0000 7f0efc7db740 1 -- 172.21.15.16:0/3615128949 wait complete. 2024-08-31T06:09:13.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.496+0000 7f0efc7db740 20 client.26030 trim_cache size 0 max 16384 2024-08-31T06:09:13.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.530+0000 7f0efc7db740 1 Processor -- start 2024-08-31T06:09:13.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.530+0000 7f0efc7db740 1 -- start start 2024-08-31T06:09:13.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.531+0000 7f0efc7db740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddaee1490 0x55cddb2ba9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:13.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.531+0000 7f0efc7db740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cddaee9670 0x55cddb2bf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:13.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.531+0000 7f0efc7db740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cddaf0a350 0x55cddb2ad240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:13.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.531+0000 7f0eeea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddaee1490 0x55cddb2ba9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:13.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.532+0000 7f0efc7db740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef01af970 con 0x55cddaee9670 2024-08-31T06:09:13.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.532+0000 7f0efc7db740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef01ad180 con 0x55cddaee1490 2024-08-31T06:09:13.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.532+0000 7f0efc7db740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef01aafd0 con 0x55cddaf0a350 2024-08-31T06:09:13.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.532+0000 7f0eeea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddaee1490 0x55cddb2ba9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:36242/0 (socket says 172.21.15.16:36242) 2024-08-31T06:09:13.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.532+0000 7f0eeea00640 1 -- 172.21.15.16:0/2616650803 learned_addr learned my addr 172.21.15.16:0/2616650803 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:13.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.532+0000 7f0eee000640 1 --2- 172.21.15.16:0/2616650803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cddaee9670 0x55cddb2bf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:13.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.532+0000 7f0eef400640 1 --2- 172.21.15.16:0/2616650803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cddaf0a350 0x55cddb2ad240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:13.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.532+0000 7f0eeea00640 1 -- 172.21.15.16:0/2616650803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cddaf0a350 msgr2=0x55cddb2ad240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:13.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.532+0000 7f0eeea00640 1 --2- 172.21.15.16:0/2616650803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cddaf0a350 0x55cddb2ad240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:13.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.532+0000 7f0eeea00640 1 -- 172.21.15.16:0/2616650803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cddaee9670 msgr2=0x55cddb2bf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:13.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.532+0000 7f0eeea00640 1 --2- 172.21.15.16:0/2616650803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cddaee9670 0x55cddb2bf100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:13.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.532+0000 7f0eeea00640 1 -- 172.21.15.16:0/2616650803 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cddb2a8800 con 0x55cddaee1490 2024-08-31T06:09:13.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.533+0000 7f0eeea00640 1 --2- 172.21.15.16:0/2616650803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddaee1490 0x55cddb2ba9d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8002230 tx=0x7f0ee80078b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df08f4a60dcb60e6 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:13.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.534+0000 7f0edb400640 1 -- 172.21.15.16:0/2616650803 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee8033070 con 0x55cddaee1490 2024-08-31T06:09:13.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.534+0000 7f0edb400640 1 -- 172.21.15.16:0/2616650803 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ee800ec90 con 0x55cddaee1490 2024-08-31T06:09:13.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.534+0000 7f0edb400640 1 -- 172.21.15.16:0/2616650803 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee802a070 con 0x55cddaee1490 2024-08-31T06:09:13.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.535+0000 7f0efc7db740 1 -- 172.21.15.16:0/2616650803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddaee1490 msgr2=0x55cddb2ba9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:13.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.535+0000 7f0efc7db740 1 --2- 172.21.15.16:0/2616650803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddaee1490 0x55cddb2ba9d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8002230 tx=0x7f0ee80078b0 comp rx=0 tx=0).stop 2024-08-31T06:09:13.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.535+0000 7f0efc7db740 1 -- 172.21.15.16:0/2616650803 shutdown_connections 2024-08-31T06:09:13.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.535+0000 7f0efc7db740 1 --2- 172.21.15.16:0/2616650803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cddaf0a350 0x55cddb2ad240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:13.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.535+0000 7f0efc7db740 1 --2- 172.21.15.16:0/2616650803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cddaee9670 0x55cddb2bf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:13.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.535+0000 7f0efc7db740 1 --2- 172.21.15.16:0/2616650803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddaee1490 0x55cddb2ba9d0 secure :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8002230 tx=0x7f0ee80078b0 comp rx=0 tx=0).stop 2024-08-31T06:09:13.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.535+0000 7f0efc7db740 1 -- 172.21.15.16:0/2616650803 >> 172.21.15.16:0/2616650803 conn(0x55cddaefe740 msgr2=0x55cddb2acb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:13.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.536+0000 7f0efc7db740 1 -- 172.21.15.16:0/2616650803 shutdown_connections 2024-08-31T06:09:13.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.536+0000 7f0efc7db740 1 -- 172.21.15.16:0/2616650803 wait complete. 2024-08-31T06:09:13.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.537+0000 7f0efc7db740 1 -- start start 2024-08-31T06:09:13.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.537+0000 7f0efc7db740 1 Processor -- start 2024-08-31T06:09:13.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.538+0000 7f0efc7db740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddaee9670 0x55cddb2ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:13.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.539+0000 7f0efc7db740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cddaf0a350 0x55cddb2ca730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:13.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.539+0000 7f0eeea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddaee9670 0x55cddb2ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:13.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.539+0000 7f0eeea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddaee9670 0x55cddb2ca1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:36258/0 (socket says 172.21.15.16:36258) 2024-08-31T06:09:13.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.539+0000 7f0eeea00640 1 -- 172.21.15.16:0/1109506449 learned_addr learned my addr 172.21.15.16:0/1109506449 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:13.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.539+0000 7f0efc7db740 1 --2- 172.21.15.16:0/1109506449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cddae9e080 0x55cddb341860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:13.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.539+0000 7f0eee000640 1 --2- 172.21.15.16:0/1109506449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cddaf0a350 0x55cddb2ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:13.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.540+0000 7f0eef400640 1 --2- 172.21.15.16:0/1109506449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cddae9e080 0x55cddb341860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:13.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.540+0000 7f0efc7db740 1 -- 172.21.15.16:0/1109506449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef01ab640 con 0x55cddaf0a350 2024-08-31T06:09:13.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.540+0000 7f0efc7db740 1 -- 172.21.15.16:0/1109506449 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef01acab0 con 0x55cddaee9670 2024-08-31T06:09:13.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.540+0000 7f0efc7db740 1 -- 172.21.15.16:0/1109506449 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef01abd10 con 0x55cddae9e080 2024-08-31T06:09:13.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.541+0000 7f0efc7db740 10 client.0 _ll_register_callbacks cb 0x7f0ee8007600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:09:13.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.541+0000 7f0eeea00640 1 -- 172.21.15.16:0/1109506449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cddae9e080 msgr2=0x55cddb341860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:13.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.541+0000 7f0eeea00640 1 --2- 172.21.15.16:0/1109506449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cddae9e080 0x55cddb341860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:13.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.541+0000 7f0eeea00640 1 -- 172.21.15.16:0/1109506449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cddaf0a350 msgr2=0x55cddb2ca730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:13.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.541+0000 7f0eeea00640 1 --2- 172.21.15.16:0/1109506449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cddaf0a350 0x55cddb2ca730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:13.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.541+0000 7f0eeea00640 1 -- 172.21.15.16:0/1109506449 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cddb2bf640 con 0x55cddaee9670 2024-08-31T06:09:13.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.541+0000 7f0eef400640 1 --2- 172.21.15.16:0/1109506449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cddae9e080 0x55cddb341860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:09:13.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.541+0000 7f0efc7db740 1 -- 172.21.15.16:0/1109506449 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ee800ec90 con 0x55cddaee9670 2024-08-31T06:09:13.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.541+0000 7f0eeea00640 1 --2- 172.21.15.16:0/1109506449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddaee9670 0x55cddb2ca1b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8010aa0 tx=0x7f0ee8007ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e341824c369f845 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:13.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.541+0000 7f0ed8c00640 10 client.26036 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:09:13.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.542+0000 7f0ed8c00640 1 -- 172.21.15.16:0/1109506449 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee8033040 con 0x55cddaee9670 2024-08-31T06:09:13.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.543+0000 7f0ed8c00640 1 -- 172.21.15.16:0/1109506449 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f0ee801b430 con 0x55cddaee9670 2024-08-31T06:09:13.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.543+0000 7f0ed8c00640 1 -- 172.21.15.16:0/1109506449 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0ee802a040 con 0x55cddaee9670 2024-08-31T06:09:13.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.543+0000 7f0efc7db740 10 client.26036 fetch_fsmap learned FSMap version 31 2024-08-31T06:09:13.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.543+0000 7f0efc7db740 1 -- 172.21.15.16:0/1109506449 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({fsmap=31}) -- 0x55cddb342e60 con 0x55cddaee9670 2024-08-31T06:09:13.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.543+0000 7f0ed8c00640 1 -- 172.21.15.16:0/1109506449 <== mon.2 v2:172.21.15.155:3300/0 4 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f0ee801b6b0 con 0x55cddaee9670 2024-08-31T06:09:13.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.544+0000 7f0efc7db740 10 client.26036 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:09:13.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.544+0000 7f0efc7db740 10 client.26036 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:09:13.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.544+0000 7f0efc7db740 20 client.26036 populate_metadata read hostname 'smithi016' 2024-08-31T06:09:13.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.544+0000 7f0efc7db740 1 --2- 172.21.15.16:0/1109506449 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55cddb3433b0 0x55cddb3457d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:13.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.544+0000 7f0efc7db740 4 client.26036 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T06:09:13.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.544+0000 7f0efc7db740 1 -- 172.21.15.16:0/1109506449 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cddb224140 con 0x55cddb3433b0 2024-08-31T06:09:13.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.544+0000 7f0eee000640 1 --2- 172.21.15.16:0/1109506449 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55cddb3433b0 0x55cddb3457d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:13.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.546+0000 7f0eee000640 1 --2- 172.21.15.16:0/1109506449 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55cddb3433b0 0x55cddb3457d0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9d574e4d28772f2 server_cookie=371da01d177fdb37 in_seq=0 out_seq=0 2024-08-31T06:09:13.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.546+0000 7f0ed8c00640 10 client.26036 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:09:13.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.548+0000 7f0ed8c00640 1 -- 172.21.15.16:0/1109506449 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cddb224140 con 0x55cddb3433b0 2024-08-31T06:09:13.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.548+0000 7f0ed8c00640 10 client.26036 handle_command_reply: tid=0 2024-08-31T06:09:13.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.548+0000 7f0efc7db740 1 client.26036 shutdown 2024-08-31T06:09:13.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.549+0000 7f0efc7db740 1 -- 172.21.15.16:0/1109506449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddaee9670 msgr2=0x55cddb2ca1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:13.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.549+0000 7f0efc7db740 1 --2- 172.21.15.16:0/1109506449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddaee9670 0x55cddb2ca1b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8010aa0 tx=0x7f0ee8007ef0 comp rx=0 tx=0).stop 2024-08-31T06:09:13.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.549+0000 7f0efc7db740 1 -- 172.21.15.16:0/1109506449 shutdown_connections 2024-08-31T06:09:13.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.549+0000 7f0efc7db740 1 --2- 172.21.15.16:0/1109506449 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55cddb3433b0 0x55cddb3457d0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:13.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.549+0000 7f0efc7db740 1 --2- 172.21.15.16:0/1109506449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cddae9e080 0x55cddb341860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:13.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.549+0000 7f0efc7db740 1 --2- 172.21.15.16:0/1109506449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cddaf0a350 0x55cddb2ca730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:13.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.549+0000 7f0ed8c00640 0 client.26036 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:09:13.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.549+0000 7f0efc7db740 1 --2- 172.21.15.16:0/1109506449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddaee9670 0x55cddb2ca1b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:13.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.549+0000 7f0efc7db740 1 -- 172.21.15.16:0/1109506449 >> 172.21.15.16:0/1109506449 conn(0x55cddaefe740 msgr2=0x55cddb2ba7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:13.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.549+0000 7f0efc7db740 1 -- 172.21.15.16:0/1109506449 shutdown_connections 2024-08-31T06:09:13.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.550+0000 7f0efc7db740 1 -- 172.21.15.16:0/1109506449 wait complete. 2024-08-31T06:09:13.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.550+0000 7f0efc7db740 20 client.26036 trim_cache size 0 max 16384 2024-08-31T06:09:13.553 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:09:13.553 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-08-31T06:09:13.553 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "78d6ef02-8551-45e8-ad60-9627eb573fc1", 2024-08-31T06:09:13.554 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-08-31T06:09:13.554 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:09:13.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.551+0000 7f0eefe00640 1 -- 172.21.15.16:0/1594018613 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f0ed40b4d20 msgr2=0x7f0ed40b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:13.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.551+0000 7f0eefe00640 1 --2- 172.21.15.16:0/1594018613 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f0ed40b4d20 0x7f0ed40b7230 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0edc0040f0 tx=0x7f0edc005df0 comp rx=0 tx=0).stop 2024-08-31T06:09:13.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.551+0000 7f0eefe00640 1 -- 172.21.15.16:0/1594018613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ef0109090 msgr2=0x7f0ef0198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:13.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.551+0000 7f0eefe00640 1 --2- 172.21.15.16:0/1594018613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ef0109090 0x7f0ef0198430 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0ef006b730 tx=0x7f0ee000db30 comp rx=0 tx=0).stop 2024-08-31T06:09:13.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.551+0000 7f0eefe00640 1 -- 172.21.15.16:0/1594018613 shutdown_connections 2024-08-31T06:09:13.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.551+0000 7f0eefe00640 1 --2- 172.21.15.16:0/1594018613 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f0ed40b4d20 0x7f0ed40b7230 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:13.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.551+0000 7f0eefe00640 1 --2- 172.21.15.16:0/1594018613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ef0109090 0x7f0ef0198430 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:13.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.551+0000 7f0eefe00640 1 --2- 172.21.15.16:0/1594018613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ef01099e0 0x7f0ef019d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:13.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.551+0000 7f0eefe00640 1 --2- 172.21.15.16:0/1594018613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ef0103750 0x7f0ef0197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:13.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.551+0000 7f0eefe00640 1 -- 172.21.15.16:0/1594018613 >> 172.21.15.16:0/1594018613 conn(0x7f0ef00ff1e0 msgr2=0x7f0ef01086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:13.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.552+0000 7f0eefe00640 1 -- 172.21.15.16:0/1594018613 shutdown_connections 2024-08-31T06:09:13.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:13.552+0000 7f0eefe00640 1 -- 172.21.15.16:0/1594018613 wait complete. 2024-08-31T06:09:14.637 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T06:09:14.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:14 smithi155 ceph-mon[21742]: pgmap v424: 97 pgs: 97 active+clean; 1.4 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 32 MiB/s wr, 1.79k op/s 2024-08-31T06:09:14.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:14 smithi155 ceph-mon[21742]: scrub queued for path: / 2024-08-31T06:09:14.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:14 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:09:14.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:14 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:09:14.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:14 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:09:14.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:14 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:09:14.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:14 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:09:14.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:14 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:09:14.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:14 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:09:14.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:14 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:09:14.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:14 smithi016 ceph-mon[17194]: pgmap v424: 97 pgs: 97 active+clean; 1.4 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 32 MiB/s wr, 1.79k op/s 2024-08-31T06:09:14.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:14 smithi016 ceph-mon[17194]: scrub queued for path: / 2024-08-31T06:09:14.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:14 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:09:14.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:14 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:09:14.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:14 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:09:14.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:14 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:09:14.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:14 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:09:14.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:14 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:09:14.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:14 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:09:14.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:14 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:09:14.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:14 smithi184 ceph-mon[22331]: pgmap v424: 97 pgs: 97 active+clean; 1.4 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 32 MiB/s wr, 1.79k op/s 2024-08-31T06:09:14.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:14 smithi184 ceph-mon[22331]: scrub queued for path: / 2024-08-31T06:09:14.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:14 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:09:14.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:14 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:09:14.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:14 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:09:14.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:14 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:09:14.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:14 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:09:14.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:14 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:09:14.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:14 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:09:14.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:14 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:09:15.255 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:09:15.689 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:15 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:09:15.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:15 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:09:15.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:15 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:09:16.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.078+0000 7f9b62e00640 1 -- 172.21.15.16:0/2346828421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b5c107f50 msgr2=0x7f9b5c108360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:16.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.078+0000 7f9b62e00640 1 --2- 172.21.15.16:0/2346828421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b5c107f50 0x7f9b5c108360 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9b4801a910 tx=0x7f9b4803b430 comp rx=0 tx=0).stop 2024-08-31T06:09:16.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.079+0000 7f9b62e00640 1 -- 172.21.15.16:0/2346828421 shutdown_connections 2024-08-31T06:09:16.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.079+0000 7f9b62e00640 1 --2- 172.21.15.16:0/2346828421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b5c107f50 0x7f9b5c108360 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:16.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.079+0000 7f9b62e00640 1 --2- 172.21.15.16:0/2346828421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b5c1088a0 0x7f9b5c10f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:16.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.079+0000 7f9b62e00640 1 --2- 172.21.15.16:0/2346828421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b5c102610 0x7f9b5c102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:16.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.079+0000 7f9b62e00640 1 -- 172.21.15.16:0/2346828421 >> 172.21.15.16:0/2346828421 conn(0x7f9b5c0fe060 msgr2=0x7f9b5c1004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:16.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.079+0000 7f9b62e00640 1 -- 172.21.15.16:0/2346828421 shutdown_connections 2024-08-31T06:09:16.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.079+0000 7f9b62e00640 1 -- 172.21.15.16:0/2346828421 wait complete. 2024-08-31T06:09:16.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.081+0000 7f9b62e00640 1 Processor -- start 2024-08-31T06:09:16.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.081+0000 7f9b62e00640 1 -- start start 2024-08-31T06:09:16.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.082+0000 7f9b62e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b5c102610 0x7f9b5c10e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:16.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.082+0000 7f9b62e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b5c107f50 0x7f9b5c10ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:16.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.082+0000 7f9b62e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b5c1088a0 0x7f9b5c1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:16.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.082+0000 7f9b62e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5c06f430 con 0x7f9b5c107f50 2024-08-31T06:09:16.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.082+0000 7f9b62e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5c06f2b0 con 0x7f9b5c1088a0 2024-08-31T06:09:16.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.082+0000 7f9b62e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5c06f5b0 con 0x7f9b5c102610 2024-08-31T06:09:16.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.082+0000 7f9b61a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b5c102610 0x7f9b5c10e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:16.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.083+0000 7f9b61000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b5c107f50 0x7f9b5c10ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:16.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.083+0000 7f9b61a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b5c102610 0x7f9b5c10e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:39928/0 (socket says 172.21.15.16:39928) 2024-08-31T06:09:16.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.083+0000 7f9b61a00640 1 -- 172.21.15.16:0/124298855 learned_addr learned my addr 172.21.15.16:0/124298855 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:16.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.083+0000 7f9b62400640 1 --2- 172.21.15.16:0/124298855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b5c1088a0 0x7f9b5c1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:16.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.084+0000 7f9b61a00640 1 -- 172.21.15.16:0/124298855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b5c1088a0 msgr2=0x7f9b5c1992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:16.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.084+0000 7f9b61a00640 1 --2- 172.21.15.16:0/124298855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b5c1088a0 0x7f9b5c1992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:16.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.084+0000 7f9b61a00640 1 -- 172.21.15.16:0/124298855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b5c107f50 msgr2=0x7f9b5c10ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:16.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.084+0000 7f9b61a00640 1 --2- 172.21.15.16:0/124298855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b5c107f50 0x7f9b5c10ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:16.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.084+0000 7f9b61a00640 1 -- 172.21.15.16:0/124298855 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b5c199a20 con 0x7f9b5c102610 2024-08-31T06:09:16.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.084+0000 7f9b61a00640 1 --2- 172.21.15.16:0/124298855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b5c102610 0x7f9b5c10e690 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9b5000ea60 tx=0x7f9b5000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:16.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.087+0000 7f9b5a000640 1 -- 172.21.15.16:0/124298855 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5000cde0 con 0x7f9b5c102610 2024-08-31T06:09:16.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.087+0000 7f9b5a000640 1 -- 172.21.15.16:0/124298855 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b500035f0 con 0x7f9b5c102610 2024-08-31T06:09:16.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.087+0000 7f9b5a000640 1 -- 172.21.15.16:0/124298855 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b50010690 con 0x7f9b5c102610 2024-08-31T06:09:16.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.088+0000 7f9b62e00640 1 -- 172.21.15.16:0/124298855 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b5c199d10 con 0x7f9b5c102610 2024-08-31T06:09:16.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.088+0000 7f9b62e00640 1 -- 172.21.15.16:0/124298855 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b5c1a0470 con 0x7f9b5c102610 2024-08-31T06:09:16.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.096+0000 7f9b5a000640 1 -- 172.21.15.16:0/124298855 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f9b50010830 con 0x7f9b5c102610 2024-08-31T06:09:16.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.098+0000 7f9b5a000640 1 --2- 172.21.15.16:0/124298855 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9b440b4e30 0x7f9b440b7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:16.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.100+0000 7f9b6b037740 1 Processor -- start 2024-08-31T06:09:16.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.101+0000 7f9b61000640 1 --2- 172.21.15.16:0/124298855 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9b440b4e30 0x7f9b440b7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:16.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.099+0000 7f9b5a000640 1 -- 172.21.15.16:0/124298855 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f9b50003ac0 con 0x7f9b5c102610 2024-08-31T06:09:16.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.102+0000 7f9b6b037740 1 -- start start 2024-08-31T06:09:16.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.104+0000 7f9b6b037740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1b937070 0x557c1bcdcb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:16.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.104+0000 7f9b6b037740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c1b90e1b0 0x557c1bcdd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:16.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.104+0000 7f9b61000640 1 --2- 172.21.15.16:0/124298855 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9b440b4e30 0x7f9b440b7340 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9b48006010 tx=0x7f9b48000bc0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:16.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.104+0000 7f9b6b037740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1b916390 0x557c1bcdd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:16.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.104+0000 7f9b6b037740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557c1bc53770 con 0x557c1b937070 2024-08-31T06:09:16.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.104+0000 7f9b6b037740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557c1bc42d00 con 0x557c1b916390 2024-08-31T06:09:16.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.104+0000 7f9b6b037740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557c1bc46cb0 con 0x557c1b90e1b0 2024-08-31T06:09:16.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.104+0000 7f9b62400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1b916390 0x557c1bcdd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:16.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.104+0000 7f9b62400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1b916390 0x557c1bcdd5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:40258/0 (socket says 172.21.15.16:40258) 2024-08-31T06:09:16.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.104+0000 7f9b62400640 1 -- 172.21.15.16:0/2166671502 learned_addr learned my addr 172.21.15.16:0/2166671502 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:16.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.104+0000 7f9b61a00640 1 --2- 172.21.15.16:0/2166671502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1b937070 0x557c1bcdcb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:16.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.105+0000 7f9b62400640 1 -- 172.21.15.16:0/2166671502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c1b90e1b0 msgr2=0x557c1bcdd060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:09:16.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.105+0000 7f9b62400640 1 --2- 172.21.15.16:0/2166671502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c1b90e1b0 0x557c1bcdd060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:16.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.105+0000 7f9b62400640 1 -- 172.21.15.16:0/2166671502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1b937070 msgr2=0x557c1bcdcb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:16.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.105+0000 7f9b62400640 1 --2- 172.21.15.16:0/2166671502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1b937070 0x557c1bcdcb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:16.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.105+0000 7f9b62400640 1 -- 172.21.15.16:0/2166671502 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c1b9a2650 con 0x557c1b916390 2024-08-31T06:09:16.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.105+0000 7f9b61a00640 1 --2- 172.21.15.16:0/2166671502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1b937070 0x557c1bcdcb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:09:16.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.106+0000 7f9b62400640 1 --2- 172.21.15.16:0/2166671502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1b916390 0x557c1bcdd5a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9b54009fb0 tx=0x7f9b5400a890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e095b8e188e68e30 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:16.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.106+0000 7f9b4ea00640 1 -- 172.21.15.16:0/2166671502 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b540074c0 con 0x557c1b916390 2024-08-31T06:09:16.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.106+0000 7f9b4ea00640 1 -- 172.21.15.16:0/2166671502 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b54007b10 con 0x557c1b916390 2024-08-31T06:09:16.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.106+0000 7f9b4ea00640 1 -- 172.21.15.16:0/2166671502 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b54013c10 con 0x557c1b916390 2024-08-31T06:09:16.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.108+0000 7f9b6b037740 1 -- 172.21.15.16:0/2166671502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1b916390 msgr2=0x557c1bcdd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:16.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.108+0000 7f9b6b037740 1 --2- 172.21.15.16:0/2166671502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1b916390 0x557c1bcdd5a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9b54009fb0 tx=0x7f9b5400a890 comp rx=0 tx=0).stop 2024-08-31T06:09:16.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.108+0000 7f9b6b037740 1 -- 172.21.15.16:0/2166671502 shutdown_connections 2024-08-31T06:09:16.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.108+0000 7f9b6b037740 1 --2- 172.21.15.16:0/2166671502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c1b90e1b0 0x557c1bcdd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:16.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.108+0000 7f9b6b037740 1 --2- 172.21.15.16:0/2166671502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1b916390 0x557c1bcdd5a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:16.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.108+0000 7f9b6b037740 1 --2- 172.21.15.16:0/2166671502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1b937070 0x557c1bcdcb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:16.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.108+0000 7f9b6b037740 1 -- 172.21.15.16:0/2166671502 >> 172.21.15.16:0/2166671502 conn(0x557c1b92b460 msgr2=0x557c1bcdaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:16.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.108+0000 7f9b6b037740 1 -- 172.21.15.16:0/2166671502 shutdown_connections 2024-08-31T06:09:16.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.108+0000 7f9b6b037740 1 -- 172.21.15.16:0/2166671502 wait complete. 2024-08-31T06:09:16.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.109+0000 7f9b6b037740 1 -- start start 2024-08-31T06:09:16.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.110+0000 7f9b6b037740 1 Processor -- start 2024-08-31T06:09:16.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.111+0000 7f9b6b037740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1b90e1b0 0x557c1bce8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:16.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.111+0000 7f9b6b037740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c1b916390 0x557c1bce8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:16.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.111+0000 7f9b6b037740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1b937070 0x557c1bcf45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:16.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.112+0000 7f9b61000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c1b916390 0x557c1bce8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:16.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.112+0000 7f9b6b037740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557c1baa5700 con 0x557c1b90e1b0 2024-08-31T06:09:16.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.112+0000 7f9b6b037740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557c1bbc9cb0 con 0x557c1b937070 2024-08-31T06:09:16.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.112+0000 7f9b6b037740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557c1bceb300 con 0x557c1b916390 2024-08-31T06:09:16.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.112+0000 7f9b62400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1b937070 0x557c1bcf45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:16.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.112+0000 7f9b62400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1b937070 0x557c1bcf45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:40262/0 (socket says 172.21.15.16:40262) 2024-08-31T06:09:16.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.112+0000 7f9b62400640 1 -- 172.21.15.16:0/3521391432 learned_addr learned my addr 172.21.15.16:0/3521391432 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:16.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.112+0000 7f9b6b037740 10 client.0 _ll_register_callbacks cb 0x557c1bbef430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:09:16.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.112+0000 7f9b61a00640 1 --2- 172.21.15.16:0/3521391432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1b90e1b0 0x557c1bce8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:16.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.113+0000 7f9b61000640 1 -- 172.21.15.16:0/3521391432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1b937070 msgr2=0x557c1bcf45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:16.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.113+0000 7f9b61000640 1 --2- 172.21.15.16:0/3521391432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1b937070 0x557c1bcf45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:16.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.113+0000 7f9b61000640 1 -- 172.21.15.16:0/3521391432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1b90e1b0 msgr2=0x557c1bce8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:16.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.113+0000 7f9b61000640 1 --2- 172.21.15.16:0/3521391432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1b90e1b0 0x557c1bce8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:16.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.113+0000 7f9b61000640 1 -- 172.21.15.16:0/3521391432 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c1bc0a870 con 0x557c1b916390 2024-08-31T06:09:16.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.114+0000 7f9b61000640 1 --2- 172.21.15.16:0/3521391432 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c1b916390 0x557c1bce8cb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9b4801a860 tx=0x7f9b4801a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f39a94d904ddddbd server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:16.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.116+0000 7f9b43400640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:09:16.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.116+0000 7f9b43400640 1 -- 172.21.15.16:0/3521391432 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4805e020 con 0x557c1b916390 2024-08-31T06:09:16.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.117+0000 7f9b43400640 1 -- 172.21.15.16:0/3521391432 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f9b48040510 con 0x557c1b916390 2024-08-31T06:09:16.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.117+0000 7f9b6b037740 1 -- 172.21.15.16:0/3521391432 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b54007b10 con 0x557c1b916390 2024-08-31T06:09:16.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.118+0000 7f9b43400640 1 -- 172.21.15.16:0/3521391432 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9b48069050 con 0x557c1b916390 2024-08-31T06:09:16.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.118+0000 7f9b6b037740 10 client.26050 fetch_fsmap learned FSMap version 31 2024-08-31T06:09:16.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.118+0000 7f9b6b037740 1 -- 172.21.15.16:0/3521391432 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557c1bcf7340 con 0x557c1b916390 2024-08-31T06:09:16.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.118+0000 7f9b43400640 1 -- 172.21.15.16:0/3521391432 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b54007b10 con 0x557c1b916390 2024-08-31T06:09:16.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.120+0000 7f9b43400640 1 -- 172.21.15.16:0/3521391432 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f9b4805a610 con 0x557c1b916390 2024-08-31T06:09:16.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.121+0000 7f9b6b037740 10 client.26050 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:09:16.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.121+0000 7f9b6b037740 10 client.26050 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:09:16.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.121+0000 7f9b6b037740 20 client.26050 populate_metadata read hostname 'smithi016' 2024-08-31T06:09:16.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.121+0000 7f9b6b037740 1 --2- 172.21.15.16:0/3521391432 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557c1b8f59f0 0x557c1bcfa950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:16.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.121+0000 7f9b6b037740 4 client.26050 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:09:16.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.121+0000 7f9b6b037740 1 -- 172.21.15.16:0/3521391432 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c1bc556c0 con 0x557c1b8f59f0 2024-08-31T06:09:16.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.122+0000 7f9b61a00640 1 --2- 172.21.15.16:0/3521391432 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557c1b8f59f0 0x557c1bcfa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:16.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.123+0000 7f9b61a00640 1 --2- 172.21.15.16:0/3521391432 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557c1b8f59f0 0x557c1bcfa950 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e09a7967f8cac6f0 server_cookie=7ec8304916955c0a in_seq=0 out_seq=0 2024-08-31T06:09:16.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.123+0000 7f9b43400640 10 client.26050 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:09:16.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.123+0000 7f9b43400640 1 -- 172.21.15.16:0/3521391432 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557c1bc556c0 con 0x557c1b8f59f0 2024-08-31T06:09:16.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.123+0000 7f9b43400640 10 client.26050 handle_command_reply: tid=0 2024-08-31T06:09:16.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.124+0000 7f9b6b037740 1 client.26050 shutdown 2024-08-31T06:09:16.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.125+0000 7f9b6b037740 1 -- 172.21.15.16:0/3521391432 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c1b916390 msgr2=0x557c1bce8cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:16.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.125+0000 7f9b6b037740 1 --2- 172.21.15.16:0/3521391432 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c1b916390 0x557c1bce8cb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9b4801a860 tx=0x7f9b4801a270 comp rx=0 tx=0).stop 2024-08-31T06:09:16.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.125+0000 7f9b6b037740 1 -- 172.21.15.16:0/3521391432 shutdown_connections 2024-08-31T06:09:16.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.125+0000 7f9b6b037740 1 --2- 172.21.15.16:0/3521391432 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557c1b8f59f0 0x557c1bcfa950 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:16.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.125+0000 7f9b6b037740 1 --2- 172.21.15.16:0/3521391432 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c1b916390 0x557c1bce8cb0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:16.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.125+0000 7f9b6b037740 1 --2- 172.21.15.16:0/3521391432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1b937070 0x557c1bcf45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:16.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.125+0000 7f9b6b037740 1 --2- 172.21.15.16:0/3521391432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1b90e1b0 0x557c1bce8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:16.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.125+0000 7f9b6b037740 1 -- 172.21.15.16:0/3521391432 >> 172.21.15.16:0/3521391432 conn(0x557c1b92b460 msgr2=0x557c1bcdfd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:16.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.125+0000 7f9b43400640 0 client.26050 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:09:16.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.126+0000 7f9b6b037740 1 -- 172.21.15.16:0/3521391432 shutdown_connections 2024-08-31T06:09:16.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.126+0000 7f9b6b037740 1 -- 172.21.15.16:0/3521391432 wait complete. 2024-08-31T06:09:16.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.126+0000 7f9b6b037740 20 client.26050 trim_cache size 0 max 16384 2024-08-31T06:09:16.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.152+0000 7f9b6b037740 1 Processor -- start 2024-08-31T06:09:16.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.152+0000 7f9b6b037740 1 -- start start 2024-08-31T06:09:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.152+0000 7f9b6b037740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1b90e1b0 0x557c1bce6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.152+0000 7f9b6b037740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c1b916390 0x557c1bce72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:16.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.153+0000 7f9b6b037740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1b937070 0x557c1bce1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:16.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.153+0000 7f9b6b037740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5c1a7220 con 0x557c1b937070 2024-08-31T06:09:16.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.153+0000 7f9b6b037740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5c1a4a30 con 0x557c1b90e1b0 2024-08-31T06:09:16.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.153+0000 7f9b6b037740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5c1a28a0 con 0x557c1b916390 2024-08-31T06:09:16.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.153+0000 7f9b61a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1b90e1b0 0x557c1bce6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:16.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.153+0000 7f9b62400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1b937070 0x557c1bce1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:16.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.153+0000 7f9b62400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1b937070 0x557c1bce1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35018/0 (socket says 172.21.15.16:35018) 2024-08-31T06:09:16.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.153+0000 7f9b62400640 1 -- 172.21.15.16:0/2758171040 learned_addr learned my addr 172.21.15.16:0/2758171040 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:16.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.153+0000 7f9b61000640 1 --2- 172.21.15.16:0/2758171040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c1b916390 0x557c1bce72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:16.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.154+0000 7f9b61000640 1 -- 172.21.15.16:0/2758171040 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1b90e1b0 msgr2=0x557c1bce6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:16.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.154+0000 7f9b61000640 1 --2- 172.21.15.16:0/2758171040 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1b90e1b0 0x557c1bce6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:16.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.154+0000 7f9b61000640 1 -- 172.21.15.16:0/2758171040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1b937070 msgr2=0x557c1bce1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:16.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.154+0000 7f9b61000640 1 --2- 172.21.15.16:0/2758171040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1b937070 0x557c1bce1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:16.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.154+0000 7f9b61000640 1 -- 172.21.15.16:0/2758171040 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c1bce95d0 con 0x557c1b916390 2024-08-31T06:09:16.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.155+0000 7f9b62400640 1 --2- 172.21.15.16:0/2758171040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1b937070 0x557c1bce1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:09:16.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.155+0000 7f9b61a00640 1 --2- 172.21.15.16:0/2758171040 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1b90e1b0 0x557c1bce6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:09:16.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.155+0000 7f9b61000640 1 --2- 172.21.15.16:0/2758171040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c1b916390 0x557c1bce72f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9b48004950 tx=0x7f9b480053c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c17a3007740272d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:16.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.156+0000 7f9b4e000640 1 -- 172.21.15.16:0/2758171040 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b48071020 con 0x557c1b916390 2024-08-31T06:09:16.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.156+0000 7f9b4e000640 1 -- 172.21.15.16:0/2758171040 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b4805a680 con 0x557c1b916390 2024-08-31T06:09:16.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.156+0000 7f9b4e000640 1 -- 172.21.15.16:0/2758171040 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4805e070 con 0x557c1b916390 2024-08-31T06:09:16.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.157+0000 7f9b6b037740 1 -- 172.21.15.16:0/2758171040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c1b916390 msgr2=0x557c1bce72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:16.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.157+0000 7f9b6b037740 1 --2- 172.21.15.16:0/2758171040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c1b916390 0x557c1bce72f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9b48004950 tx=0x7f9b480053c0 comp rx=0 tx=0).stop 2024-08-31T06:09:16.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.157+0000 7f9b6b037740 1 -- 172.21.15.16:0/2758171040 shutdown_connections 2024-08-31T06:09:16.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.157+0000 7f9b6b037740 1 --2- 172.21.15.16:0/2758171040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c1b916390 0x557c1bce72f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:16.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.157+0000 7f9b6b037740 1 --2- 172.21.15.16:0/2758171040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1b937070 0x557c1bce1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:16.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.157+0000 7f9b6b037740 1 --2- 172.21.15.16:0/2758171040 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1b90e1b0 0x557c1bce6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:16.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.157+0000 7f9b6b037740 1 -- 172.21.15.16:0/2758171040 >> 172.21.15.16:0/2758171040 conn(0x557c1b92b460 msgr2=0x557c1bce3670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:16.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.157+0000 7f9b6b037740 1 -- 172.21.15.16:0/2758171040 shutdown_connections 2024-08-31T06:09:16.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.157+0000 7f9b6b037740 1 -- 172.21.15.16:0/2758171040 wait complete. 2024-08-31T06:09:16.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.158+0000 7f9b6b037740 1 -- start start 2024-08-31T06:09:16.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.158+0000 7f9b6b037740 1 Processor -- start 2024-08-31T06:09:16.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.159+0000 7f9b6b037740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c1b90e1b0 0x557c1bcf0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:16.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.159+0000 7f9b6b037740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1b916390 0x557c1bcec2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:16.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.159+0000 7f9b6b037740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1b937070 0x557c1bcec910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:16.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.159+0000 7f9b6b037740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5c1a2ef0 con 0x557c1b937070 2024-08-31T06:09:16.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.159+0000 7f9b6b037740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5c1a4360 con 0x557c1b916390 2024-08-31T06:09:16.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.159+0000 7f9b6b037740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5c1a35c0 con 0x557c1b90e1b0 2024-08-31T06:09:16.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.159+0000 7f9b6b037740 10 client.0 _ll_register_callbacks cb 0x7f9b48004a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:09:16.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.160+0000 7f9b61000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1b916390 0x557c1bcec2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:16.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.160+0000 7f9b61000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1b916390 0x557c1bcec2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:40278/0 (socket says 172.21.15.16:40278) 2024-08-31T06:09:16.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.160+0000 7f9b61000640 1 -- 172.21.15.16:0/2225977047 learned_addr learned my addr 172.21.15.16:0/2225977047 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:16.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.160+0000 7f9b61000640 1 -- 172.21.15.16:0/2225977047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c1b90e1b0 msgr2=0x557c1bcf0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:16.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.160+0000 7f9b61a00640 1 --2- 172.21.15.16:0/2225977047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c1b90e1b0 0x557c1bcf0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:16.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.160+0000 7f9b61000640 1 --2- 172.21.15.16:0/2225977047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c1b90e1b0 0x557c1bcf0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:16.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.160+0000 7f9b61000640 1 -- 172.21.15.16:0/2225977047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1b937070 msgr2=0x557c1bcec910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:16.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.161+0000 7f9b62400640 1 --2- 172.21.15.16:0/2225977047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1b937070 0x557c1bcec910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:16.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.162+0000 7f9b61000640 1 --2- 172.21.15.16:0/2225977047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1b937070 0x557c1bcec910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:16.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.162+0000 7f9b61000640 1 -- 172.21.15.16:0/2225977047 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c1bced080 con 0x557c1b916390 2024-08-31T06:09:16.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.162+0000 7f9b61a00640 1 --2- 172.21.15.16:0/2225977047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c1b90e1b0 0x557c1bcf0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:09:16.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.162+0000 7f9b62400640 1 --2- 172.21.15.16:0/2225977047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1b937070 0x557c1bcec910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:09:16.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.162+0000 7f9b61000640 1 --2- 172.21.15.16:0/2225977047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1b916390 0x557c1bcec2c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9b4803baf0 tx=0x7f9b48040fe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d485b92fdc6e825c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:16.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.162+0000 7f9b43400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:09:16.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.163+0000 7f9b43400640 1 -- 172.21.15.16:0/2225977047 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b48071030 con 0x557c1b916390 2024-08-31T06:09:16.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.163+0000 7f9b43400640 1 -- 172.21.15.16:0/2225977047 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f9b48040720 con 0x557c1b916390 2024-08-31T06:09:16.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.163+0000 7f9b6b037740 1 -- 172.21.15.16:0/2225977047 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b4805a680 con 0x557c1b916390 2024-08-31T06:09:16.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.164+0000 7f9b43400640 1 -- 172.21.15.16:0/2225977047 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9b4805e040 con 0x557c1b916390 2024-08-31T06:09:16.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.164+0000 7f9b6b037740 10 client.26054 fetch_fsmap learned FSMap version 31 2024-08-31T06:09:16.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.164+0000 7f9b6b037740 1 -- 172.21.15.16:0/2225977047 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557c1bd65330 con 0x557c1b916390 2024-08-31T06:09:16.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.164+0000 7f9b43400640 1 -- 172.21.15.16:0/2225977047 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b4805a680 con 0x557c1b916390 2024-08-31T06:09:16.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.164+0000 7f9b43400640 1 -- 172.21.15.16:0/2225977047 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f9b48076bf0 con 0x557c1b916390 2024-08-31T06:09:16.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.164+0000 7f9b6b037740 10 client.26054 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:09:16.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.164+0000 7f9b6b037740 10 client.26054 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:09:16.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.164+0000 7f9b6b037740 20 client.26054 populate_metadata read hostname 'smithi016' 2024-08-31T06:09:16.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.164+0000 7f9b6b037740 1 --2- 172.21.15.16:0/2225977047 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557c1b8cada0 0x557c1bd679b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:16.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.165+0000 7f9b6b037740 4 client.26054 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T06:09:16.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.165+0000 7f9b6b037740 1 -- 172.21.15.16:0/2225977047 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557c1bc51160 con 0x557c1b8cada0 2024-08-31T06:09:16.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.165+0000 7f9b61a00640 1 --2- 172.21.15.16:0/2225977047 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557c1b8cada0 0x557c1bd679b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:16.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.165+0000 7f9b43400640 10 client.26054 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:09:16.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.166+0000 7f9b61a00640 1 --2- 172.21.15.16:0/2225977047 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557c1b8cada0 0x557c1bd679b0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9d180d66b7fcd4c server_cookie=9e44120994eac2c7 in_seq=0 out_seq=0 2024-08-31T06:09:16.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.166+0000 7f9b43400640 1 -- 172.21.15.16:0/2225977047 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1422 (crc 0 0 0) 0x557c1bc51160 con 0x557c1b8cada0 2024-08-31T06:09:16.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.166+0000 7f9b43400640 10 client.26054 handle_command_reply: tid=0 2024-08-31T06:09:16.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.166+0000 7f9b6b037740 1 client.26054 shutdown 2024-08-31T06:09:16.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.166+0000 7f9b6b037740 1 -- 172.21.15.16:0/2225977047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1b916390 msgr2=0x557c1bcec2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:16.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.166+0000 7f9b6b037740 1 --2- 172.21.15.16:0/2225977047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1b916390 0x557c1bcec2c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9b4803baf0 tx=0x7f9b48040fe0 comp rx=0 tx=0).stop 2024-08-31T06:09:16.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.166+0000 7f9b6b037740 1 -- 172.21.15.16:0/2225977047 shutdown_connections 2024-08-31T06:09:16.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.166+0000 7f9b6b037740 1 --2- 172.21.15.16:0/2225977047 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557c1b8cada0 0x557c1bd679b0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:16.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.166+0000 7f9b6b037740 1 --2- 172.21.15.16:0/2225977047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1b937070 0x557c1bcec910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:16.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.166+0000 7f9b6b037740 1 --2- 172.21.15.16:0/2225977047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1b916390 0x557c1bcec2c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:16.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.166+0000 7f9b6b037740 1 --2- 172.21.15.16:0/2225977047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c1b90e1b0 0x557c1bcf0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:16.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.166+0000 7f9b6b037740 1 -- 172.21.15.16:0/2225977047 >> 172.21.15.16:0/2225977047 conn(0x557c1b92b460 msgr2=0x557c1bce3670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:16.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.166+0000 7f9b43400640 0 client.26054 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:09:16.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.167+0000 7f9b6b037740 1 -- 172.21.15.16:0/2225977047 shutdown_connections 2024-08-31T06:09:16.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.167+0000 7f9b6b037740 1 -- 172.21.15.16:0/2225977047 wait complete. 2024-08-31T06:09:16.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.167+0000 7f9b6b037740 20 client.26054 trim_cache size 0 max 16384 2024-08-31T06:09:16.170 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:09:16.170 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T06:09:16.170 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-08-31T06:09:16.170 INFO:teuthology.orchestra.run.smithi016.stdout: "78d6ef02-8551-45e8-ad60-9627eb573fc1": { 2024-08-31T06:09:16.170 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:09:16.170 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "78d6ef02-8551-45e8-ad60-9627eb573fc1", 2024-08-31T06:09:16.170 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-08-31T06:09:16.170 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:09:16.170 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:09:16.170 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-08-31T06:09:16.171 INFO:teuthology.orchestra.run.smithi016.stdout: "78d6ef02-8551-45e8-ad60-9627eb573fc1": { 2024-08-31T06:09:16.171 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:09:13.547865+0000", 2024-08-31T06:09:16.171 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:09:16.171 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:09:16.171 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:09:16.171 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:09:16.171 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:09:16.171 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:09:16.171 INFO:teuthology.orchestra.run.smithi016.stdout: "c8455d74-2fe7-4d93-bcf2-19b419d16016": { 2024-08-31T06:09:16.171 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:08:24.035960+0000", 2024-08-31T06:09:16.171 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:09:16.171 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:09:16.171 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:09:16.171 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:09:16.172 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:09:16.172 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:09:16.172 INFO:teuthology.orchestra.run.smithi016.stdout: "88d7bdb8-cb1c-45e5-ad29-4cd8da6c6003": { 2024-08-31T06:09:16.172 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:07:39.648942+0000", 2024-08-31T06:09:16.172 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:09:16.172 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:09:16.172 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:09:16.172 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:09:16.172 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:09:16.172 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:09:16.172 INFO:teuthology.orchestra.run.smithi016.stdout: "c300b7e8-f3fd-482c-a909-9329bc61e570": { 2024-08-31T06:09:16.172 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:06:49.868046+0000", 2024-08-31T06:09:16.172 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:09:16.172 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:09:16.173 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:09:16.173 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:09:16.173 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:09:16.173 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:09:16.173 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:09:16.173 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:09:16.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.168+0000 7f9b62e00640 1 -- 172.21.15.16:0/124298855 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9b440b4e30 msgr2=0x7f9b440b7340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:16.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.168+0000 7f9b62e00640 1 --2- 172.21.15.16:0/124298855 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9b440b4e30 0x7f9b440b7340 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9b48006010 tx=0x7f9b48000bc0 comp rx=0 tx=0).stop 2024-08-31T06:09:16.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.168+0000 7f9b62e00640 1 -- 172.21.15.16:0/124298855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b5c102610 msgr2=0x7f9b5c10e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:16.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.168+0000 7f9b62e00640 1 --2- 172.21.15.16:0/124298855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b5c102610 0x7f9b5c10e690 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9b5000ea60 tx=0x7f9b5000ef20 comp rx=0 tx=0).stop 2024-08-31T06:09:16.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.169+0000 7f9b62e00640 1 -- 172.21.15.16:0/124298855 shutdown_connections 2024-08-31T06:09:16.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.169+0000 7f9b62e00640 1 --2- 172.21.15.16:0/124298855 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9b440b4e30 0x7f9b440b7340 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:16.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.169+0000 7f9b62e00640 1 --2- 172.21.15.16:0/124298855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b5c1088a0 0x7f9b5c1992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:16.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.169+0000 7f9b62e00640 1 --2- 172.21.15.16:0/124298855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b5c107f50 0x7f9b5c10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:16.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.169+0000 7f9b62e00640 1 --2- 172.21.15.16:0/124298855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b5c102610 0x7f9b5c10e690 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:16.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.169+0000 7f9b62e00640 1 -- 172.21.15.16:0/124298855 >> 172.21.15.16:0/124298855 conn(0x7f9b5c0fe060 msgr2=0x7f9b5c074000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:16.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.169+0000 7f9b62e00640 1 -- 172.21.15.16:0/124298855 shutdown_connections 2024-08-31T06:09:16.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:16.169+0000 7f9b62e00640 1 -- 172.21.15.16:0/124298855 wait complete. 2024-08-31T06:09:16.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 110.0 s, 2642.4 tps, lat 37.578 ms stddev 39.958 2024-08-31T06:09:16.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:16 smithi155 ceph-mon[21742]: pgmap v425: 97 pgs: 97 active+clean; 1.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 34 MiB/s wr, 1.81k op/s 2024-08-31T06:09:16.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:16 smithi016 ceph-mon[17194]: pgmap v425: 97 pgs: 97 active+clean; 1.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 34 MiB/s wr, 1.81k op/s 2024-08-31T06:09:16.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:16 smithi184 ceph-mon[22331]: pgmap v425: 97 pgs: 97 active+clean; 1.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 34 MiB/s wr, 1.81k op/s 2024-08-31T06:09:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.049+0000 7fd88e200640 1 -- 172.21.15.16:0/3853966982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd88810fea0 msgr2=0x7fd8881122c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.049+0000 7fd88e200640 1 --2- 172.21.15.16:0/3853966982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd88810fea0 0x7fd8881122c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd880009be0 tx=0x7fd88002a750 comp rx=0 tx=0).stop 2024-08-31T06:09:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.049+0000 7fd88e200640 1 -- 172.21.15.16:0/3853966982 shutdown_connections 2024-08-31T06:09:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.049+0000 7fd88e200640 1 --2- 172.21.15.16:0/3853966982 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd888102c40 0x7fd88810f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.049+0000 7fd88e200640 1 --2- 172.21.15.16:0/3853966982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd88810fea0 0x7fd8881122c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.049+0000 7fd88e200640 1 --2- 172.21.15.16:0/3853966982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8881022f0 0x7fd888102700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.049+0000 7fd88e200640 1 -- 172.21.15.16:0/3853966982 >> 172.21.15.16:0/3853966982 conn(0x7fd88807a4e0 msgr2=0x7fd88807a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.050+0000 7fd88e200640 1 -- 172.21.15.16:0/3853966982 shutdown_connections 2024-08-31T06:09:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.050+0000 7fd88e200640 1 -- 172.21.15.16:0/3853966982 wait complete. 2024-08-31T06:09:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.053+0000 7fd88e200640 1 Processor -- start 2024-08-31T06:09:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.053+0000 7fd88e200640 1 -- start start 2024-08-31T06:09:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.055+0000 7fd88e200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8881022f0 0x7fd8881a1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.055+0000 7fd88e200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd888102c40 0x7fd8881a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.056+0000 7fd88e200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd88810fea0 0x7fd8881a1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.056+0000 7fd88e200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd888115960 con 0x7fd8881022f0 2024-08-31T06:09:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.056+0000 7fd88e200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd8881157e0 con 0x7fd88810fea0 2024-08-31T06:09:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.056+0000 7fd88e200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd888115ae0 con 0x7fd888102c40 2024-08-31T06:09:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.056+0000 7fd887400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8881022f0 0x7fd8881a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.056+0000 7fd887400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8881022f0 0x7fd8881a1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35040/0 (socket says 172.21.15.16:35040) 2024-08-31T06:09:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.056+0000 7fd887400640 1 -- 172.21.15.16:0/3976004304 learned_addr learned my addr 172.21.15.16:0/3976004304 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.056+0000 7fd886a00640 1 --2- 172.21.15.16:0/3976004304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd888102c40 0x7fd8881a1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.056+0000 7fd887e00640 1 --2- 172.21.15.16:0/3976004304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd88810fea0 0x7fd8881a1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.057+0000 7fd887400640 1 -- 172.21.15.16:0/3976004304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd888102c40 msgr2=0x7fd8881a1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.057+0000 7fd887400640 1 --2- 172.21.15.16:0/3976004304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd888102c40 0x7fd8881a1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.057+0000 7fd887400640 1 -- 172.21.15.16:0/3976004304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd88810fea0 msgr2=0x7fd8881a1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.057+0000 7fd887400640 1 --2- 172.21.15.16:0/3976004304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd88810fea0 0x7fd8881a1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.057+0000 7fd887400640 1 -- 172.21.15.16:0/3976004304 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd88810e8a0 con 0x7fd8881022f0 2024-08-31T06:09:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.057+0000 7fd887e00640 1 --2- 172.21.15.16:0/3976004304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd88810fea0 0x7fd8881a1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:09:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.057+0000 7fd887400640 1 --2- 172.21.15.16:0/3976004304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8881022f0 0x7fd8881a1540 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd87000e9c0 tx=0x7fd87000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.058+0000 7fd87fe00640 1 -- 172.21.15.16:0/3976004304 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87000ccc0 con 0x7fd8881022f0 2024-08-31T06:09:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.058+0000 7fd87fe00640 1 -- 172.21.15.16:0/3976004304 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8700054d0 con 0x7fd8881022f0 2024-08-31T06:09:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.058+0000 7fd87fe00640 1 -- 172.21.15.16:0/3976004304 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd870010430 con 0x7fd8881022f0 2024-08-31T06:09:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.058+0000 7fd88e200640 1 -- 172.21.15.16:0/3976004304 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd88810eb90 con 0x7fd8881022f0 2024-08-31T06:09:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.058+0000 7fd88e200640 1 -- 172.21.15.16:0/3976004304 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd88810f090 con 0x7fd8881022f0 2024-08-31T06:09:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.064+0000 7fd8953ca740 1 Processor -- start 2024-08-31T06:09:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.065+0000 7fd8953ca740 1 -- start start 2024-08-31T06:09:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.065+0000 7fd8953ca740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c936c9b1b0 0x55c93703e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.065+0000 7fd8953ca740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c936c722f0 0x55c93703ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.065+0000 7fd8953ca740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c936c7a4d0 0x55c93703f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.065+0000 7fd8953ca740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c936fb3f70 con 0x55c936c9b1b0 2024-08-31T06:09:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.065+0000 7fd8953ca740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c936fa3500 con 0x55c936c722f0 2024-08-31T06:09:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.065+0000 7fd8953ca740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c936fa74b0 con 0x55c936c7a4d0 2024-08-31T06:09:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.066+0000 7fd887e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c936c9b1b0 0x55c93703e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.066+0000 7fd887e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c936c9b1b0 0x55c93703e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35050/0 (socket says 172.21.15.16:35050) 2024-08-31T06:09:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.066+0000 7fd887e00640 1 -- 172.21.15.16:0/2279695326 learned_addr learned my addr 172.21.15.16:0/2279695326 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.066+0000 7fd886a00640 1 --2- 172.21.15.16:0/2279695326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c936c7a4d0 0x55c93703f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.066+0000 7fd87fe00640 1 -- 172.21.15.16:0/3976004304 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fd870004f80 con 0x7fd8881022f0 2024-08-31T06:09:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.066+0000 7fd887e00640 1 -- 172.21.15.16:0/2279695326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c936c7a4d0 msgr2=0x55c93703f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.066+0000 7fd887e00640 1 --2- 172.21.15.16:0/2279695326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c936c7a4d0 0x55c93703f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.066+0000 7fd887e00640 1 -- 172.21.15.16:0/2279695326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c936c722f0 msgr2=0x55c93703ebd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T06:09:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.066+0000 7fd887e00640 1 --2- 172.21.15.16:0/2279695326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c936c722f0 0x55c93703ebd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.066+0000 7fd887e00640 1 -- 172.21.15.16:0/2279695326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c936f5dc10 con 0x55c936c9b1b0 2024-08-31T06:09:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.067+0000 7fd886a00640 1 --2- 172.21.15.16:0/2279695326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c936c7a4d0 0x55c93703f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:09:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.067+0000 7fd887e00640 1 --2- 172.21.15.16:0/2279695326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c936c9b1b0 0x55c93703e690 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd88002ad40 tx=0x7fd880030420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=307ef769fd4f77af server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.068+0000 7fd877e00640 1 -- 172.21.15.16:0/2279695326 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd880040030 con 0x55c936c9b1b0 2024-08-31T06:09:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.068+0000 7fd877e00640 1 -- 172.21.15.16:0/2279695326 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd880003e20 con 0x55c936c9b1b0 2024-08-31T06:09:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.068+0000 7fd877e00640 1 -- 172.21.15.16:0/2279695326 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd880036040 con 0x55c936c9b1b0 2024-08-31T06:09:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.069+0000 7fd8953ca740 1 -- 172.21.15.16:0/2279695326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c936c9b1b0 msgr2=0x55c93703e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.069+0000 7fd8953ca740 1 --2- 172.21.15.16:0/2279695326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c936c9b1b0 0x55c93703e690 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd88002ad40 tx=0x7fd880030420 comp rx=0 tx=0).stop 2024-08-31T06:09:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.070+0000 7fd8953ca740 1 -- 172.21.15.16:0/2279695326 shutdown_connections 2024-08-31T06:09:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.070+0000 7fd8953ca740 1 --2- 172.21.15.16:0/2279695326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c936c7a4d0 0x55c93703f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.070+0000 7fd8953ca740 1 --2- 172.21.15.16:0/2279695326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c936c722f0 0x55c93703ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.070+0000 7fd8953ca740 1 --2- 172.21.15.16:0/2279695326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c936c9b1b0 0x55c93703e690 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.070+0000 7fd8953ca740 1 -- 172.21.15.16:0/2279695326 >> 172.21.15.16:0/2279695326 conn(0x55c936c8f300 msgr2=0x55c93703c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.070+0000 7fd8953ca740 1 -- 172.21.15.16:0/2279695326 shutdown_connections 2024-08-31T06:09:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.070+0000 7fd8953ca740 1 -- 172.21.15.16:0/2279695326 wait complete. 2024-08-31T06:09:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.071+0000 7fd87fe00640 1 --2- 172.21.15.16:0/3976004304 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd86c0b4c30 0x7fd86c0b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.072+0000 7fd87fe00640 1 -- 172.21.15.16:0/3976004304 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fd8700e58f0 con 0x7fd8881022f0 2024-08-31T06:09:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.072+0000 7fd887e00640 1 --2- 172.21.15.16:0/3976004304 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd86c0b4c30 0x7fd86c0b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.072+0000 7fd8953ca740 1 -- start start 2024-08-31T06:09:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.073+0000 7fd8953ca740 1 Processor -- start 2024-08-31T06:09:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.074+0000 7fd8953ca740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c936c722f0 0x55c937049cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.074+0000 7fd8953ca740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c936c7a4d0 0x55c93704a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.074+0000 7fd8953ca740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c936c9b1b0 0x55c937056100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.074+0000 7fd8953ca740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c936edcab0 con 0x55c936c722f0 2024-08-31T06:09:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.075+0000 7fd8953ca740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c936f4a320 con 0x55c936c7a4d0 2024-08-31T06:09:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.075+0000 7fd887e00640 1 --2- 172.21.15.16:0/3976004304 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd86c0b4c30 0x7fd86c0b7140 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd88002ac70 tx=0x7fd880004ab0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.075+0000 7fd8953ca740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c93704cdb0 con 0x55c936c9b1b0 2024-08-31T06:09:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.075+0000 7fd887400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c936c722f0 0x55c937049cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.075+0000 7fd8953ca740 10 client.0 _ll_register_callbacks cb 0x55c936f83e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:09:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.075+0000 7fd887400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c936c722f0 0x55c937049cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35054/0 (socket says 172.21.15.16:35054) 2024-08-31T06:09:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.075+0000 7fd887400640 1 -- 172.21.15.16:0/1568236007 learned_addr learned my addr 172.21.15.16:0/1568236007 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.075+0000 7fd887e00640 1 --2- 172.21.15.16:0/1568236007 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c936c9b1b0 0x55c937056100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.075+0000 7fd886a00640 1 --2- 172.21.15.16:0/1568236007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c936c7a4d0 0x55c93704a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.076+0000 7fd887400640 1 -- 172.21.15.16:0/1568236007 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c936c9b1b0 msgr2=0x55c937056100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.076+0000 7fd887400640 1 --2- 172.21.15.16:0/1568236007 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c936c9b1b0 0x55c937056100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.076+0000 7fd887400640 1 -- 172.21.15.16:0/1568236007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c936c7a4d0 msgr2=0x55c93704a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.076+0000 7fd887400640 1 --2- 172.21.15.16:0/1568236007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c936c7a4d0 0x55c93704a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.076+0000 7fd887400640 1 -- 172.21.15.16:0/1568236007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c936d06790 con 0x55c936c722f0 2024-08-31T06:09:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.076+0000 7fd886a00640 1 --2- 172.21.15.16:0/1568236007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c936c7a4d0 0x55c93704a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:09:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.076+0000 7fd887e00640 1 --2- 172.21.15.16:0/1568236007 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c936c9b1b0 0x55c937056100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:09:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.076+0000 7fd887400640 1 --2- 172.21.15.16:0/1568236007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c936c722f0 0x55c937049cf0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd870000c00 tx=0x7fd8700ab8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e01df4681cdfb16 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.076+0000 7fd874c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:09:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.077+0000 7fd874c00640 1 -- 172.21.15.16:0/1568236007 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8700ac040 con 0x55c936c722f0 2024-08-31T06:09:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.077+0000 7fd8953ca740 1 -- 172.21.15.16:0/1568236007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd880003e20 con 0x55c936c722f0 2024-08-31T06:09:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.077+0000 7fd874c00640 1 -- 172.21.15.16:0/1568236007 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fd8700a82d0 con 0x55c936c722f0 2024-08-31T06:09:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.077+0000 7fd874c00640 1 -- 172.21.15.16:0/1568236007 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd8700b08b0 con 0x55c936c722f0 2024-08-31T06:09:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.077+0000 7fd8953ca740 10 client.15999 fetch_fsmap learned FSMap version 31 2024-08-31T06:09:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.077+0000 7fd8953ca740 1 -- 172.21.15.16:0/1568236007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c936d7f620 con 0x55c936c722f0 2024-08-31T06:09:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.078+0000 7fd874c00640 1 -- 172.21.15.16:0/1568236007 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd880003e20 con 0x55c936c722f0 2024-08-31T06:09:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.078+0000 7fd874c00640 1 -- 172.21.15.16:0/1568236007 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fd8700a7ad0 con 0x55c936c722f0 2024-08-31T06:09:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.078+0000 7fd8953ca740 10 client.15999 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:09:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.078+0000 7fd8953ca740 10 client.15999 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:09:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.078+0000 7fd8953ca740 20 client.15999 populate_metadata read hostname 'smithi016' 2024-08-31T06:09:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.078+0000 7fd8953ca740 1 --2- 172.21.15.16:0/1568236007 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55c936c59d80 0x55c93705be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.079+0000 7fd8953ca740 4 client.15999 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:09:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.079+0000 7fd8953ca740 1 -- 172.21.15.16:0/1568236007 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c936fb5ec0 con 0x55c936c59d80 2024-08-31T06:09:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.079+0000 7fd886a00640 1 --2- 172.21.15.16:0/1568236007 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55c936c59d80 0x55c93705be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.081+0000 7fd886a00640 1 --2- 172.21.15.16:0/1568236007 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55c936c59d80 0x55c93705be30 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb60a65b7f14ac86 server_cookie=d31107ff3a077759 in_seq=0 out_seq=0 2024-08-31T06:09:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.081+0000 7fd874c00640 10 client.15999 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:09:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.082+0000 7fd874c00640 1 -- 172.21.15.16:0/1568236007 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c936fb5ec0 con 0x55c936c59d80 2024-08-31T06:09:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.082+0000 7fd874c00640 10 client.15999 handle_command_reply: tid=0 2024-08-31T06:09:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.082+0000 7fd8953ca740 1 client.15999 shutdown 2024-08-31T06:09:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.083+0000 7fd8953ca740 1 -- 172.21.15.16:0/1568236007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c936c722f0 msgr2=0x55c937049cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.083+0000 7fd8953ca740 1 --2- 172.21.15.16:0/1568236007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c936c722f0 0x55c937049cf0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd870000c00 tx=0x7fd8700ab8a0 comp rx=0 tx=0).stop 2024-08-31T06:09:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.083+0000 7fd8953ca740 1 -- 172.21.15.16:0/1568236007 shutdown_connections 2024-08-31T06:09:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.083+0000 7fd8953ca740 1 --2- 172.21.15.16:0/1568236007 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55c936c59d80 0x55c93705be30 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.083+0000 7fd8953ca740 1 --2- 172.21.15.16:0/1568236007 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c936c9b1b0 0x55c937056100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.083+0000 7fd8953ca740 1 --2- 172.21.15.16:0/1568236007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c936c7a4d0 0x55c93704a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.083+0000 7fd8953ca740 1 --2- 172.21.15.16:0/1568236007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c936c722f0 0x55c937049cf0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.083+0000 7fd8953ca740 1 -- 172.21.15.16:0/1568236007 >> 172.21.15.16:0/1568236007 conn(0x55c936c8f300 msgr2=0x55c93703f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.083+0000 7fd874c00640 0 client.15999 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:09:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.085+0000 7fd8953ca740 1 -- 172.21.15.16:0/1568236007 shutdown_connections 2024-08-31T06:09:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.085+0000 7fd8953ca740 1 -- 172.21.15.16:0/1568236007 wait complete. 2024-08-31T06:09:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.086+0000 7fd8953ca740 20 client.15999 trim_cache size 0 max 16384 2024-08-31T06:09:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.112+0000 7fd8953ca740 1 Processor -- start 2024-08-31T06:09:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.112+0000 7fd8953ca740 1 -- start start 2024-08-31T06:09:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.113+0000 7fd8953ca740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c936c722f0 0x55c93704a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.113+0000 7fd8953ca740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c936c7a4d0 0x55c93704bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.113+0000 7fd8953ca740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c936c9b1b0 0x55c93703fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.113+0000 7fd8953ca740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd8881c0a60 con 0x55c936c9b1b0 2024-08-31T06:09:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.113+0000 7fd8953ca740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd8881be270 con 0x55c936c722f0 2024-08-31T06:09:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.113+0000 7fd8953ca740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd8881bc0c0 con 0x55c936c7a4d0 2024-08-31T06:09:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.114+0000 7fd887400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c936c722f0 0x55c93704a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.114+0000 7fd887400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c936c722f0 0x55c93704a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:40310/0 (socket says 172.21.15.16:40310) 2024-08-31T06:09:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.114+0000 7fd886a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c936c7a4d0 0x55c93704bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.114+0000 7fd887400640 1 -- 172.21.15.16:0/3417681704 learned_addr learned my addr 172.21.15.16:0/3417681704 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.114+0000 7fd887e00640 1 --2- 172.21.15.16:0/3417681704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c936c9b1b0 0x55c93703fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.114+0000 7fd887400640 1 -- 172.21.15.16:0/3417681704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c936c7a4d0 msgr2=0x55c93704bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.114+0000 7fd887400640 1 --2- 172.21.15.16:0/3417681704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c936c7a4d0 0x55c93704bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.114+0000 7fd887400640 1 -- 172.21.15.16:0/3417681704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c936c9b1b0 msgr2=0x55c93703fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.114+0000 7fd887400640 1 --2- 172.21.15.16:0/3417681704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c936c9b1b0 0x55c93703fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.114+0000 7fd887400640 1 -- 172.21.15.16:0/3417681704 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c93704afb0 con 0x55c936c722f0 2024-08-31T06:09:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.114+0000 7fd887e00640 1 --2- 172.21.15.16:0/3417681704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c936c9b1b0 0x55c93703fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:09:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.114+0000 7fd887400640 1 --2- 172.21.15.16:0/3417681704 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c936c722f0 0x55c93704a1f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd8700a92b0 tx=0x7fd870004a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f43105fe2c408209 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.115+0000 7fd877400640 1 -- 172.21.15.16:0/3417681704 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8700abb50 con 0x55c936c722f0 2024-08-31T06:09:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.115+0000 7fd877400640 1 -- 172.21.15.16:0/3417681704 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8700ac040 con 0x55c936c722f0 2024-08-31T06:09:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.115+0000 7fd877400640 1 -- 172.21.15.16:0/3417681704 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8700a79b0 con 0x55c936c722f0 2024-08-31T06:09:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.117+0000 7fd8953ca740 1 -- 172.21.15.16:0/3417681704 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c936c722f0 msgr2=0x55c93704a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.117+0000 7fd8953ca740 1 --2- 172.21.15.16:0/3417681704 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c936c722f0 0x55c93704a1f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd8700a92b0 tx=0x7fd870004a20 comp rx=0 tx=0).stop 2024-08-31T06:09:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.118+0000 7fd8953ca740 1 -- 172.21.15.16:0/3417681704 shutdown_connections 2024-08-31T06:09:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.118+0000 7fd8953ca740 1 --2- 172.21.15.16:0/3417681704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c936c7a4d0 0x55c93704bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.118+0000 7fd8953ca740 1 --2- 172.21.15.16:0/3417681704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c936c9b1b0 0x55c93703fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.118+0000 7fd8953ca740 1 --2- 172.21.15.16:0/3417681704 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c936c722f0 0x55c93704a1f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.118+0000 7fd8953ca740 1 -- 172.21.15.16:0/3417681704 >> 172.21.15.16:0/3417681704 conn(0x55c936c8f300 msgr2=0x55c93703f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.119+0000 7fd8953ca740 1 -- 172.21.15.16:0/3417681704 shutdown_connections 2024-08-31T06:09:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.119+0000 7fd8953ca740 1 -- 172.21.15.16:0/3417681704 wait complete. 2024-08-31T06:09:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.119+0000 7fd8953ca740 1 -- start start 2024-08-31T06:09:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.120+0000 7fd8953ca740 1 Processor -- start 2024-08-31T06:09:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.121+0000 7fd8953ca740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c936c722f0 0x55c937052850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.121+0000 7fd8953ca740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c936c7a4d0 0x55c93704dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.121+0000 7fd8953ca740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c936c9b1b0 0x55c93704e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.121+0000 7fd887400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c936c722f0 0x55c937052850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.121+0000 7fd887e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c936c9b1b0 0x55c93704e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.122+0000 7fd887e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c936c9b1b0 0x55c93704e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35084/0 (socket says 172.21.15.16:35084) 2024-08-31T06:09:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.122+0000 7fd887e00640 1 -- 172.21.15.16:0/2568161398 learned_addr learned my addr 172.21.15.16:0/2568161398 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.122+0000 7fd8953ca740 1 -- 172.21.15.16:0/2568161398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd8881bc730 con 0x55c936c9b1b0 2024-08-31T06:09:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.122+0000 7fd8953ca740 1 -- 172.21.15.16:0/2568161398 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd8881bdba0 con 0x55c936c7a4d0 2024-08-31T06:09:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.122+0000 7fd8953ca740 1 -- 172.21.15.16:0/2568161398 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd8881bce00 con 0x55c936c722f0 2024-08-31T06:09:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.122+0000 7fd8953ca740 10 client.0 _ll_register_callbacks cb 0x7fd870004920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:09:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.122+0000 7fd886a00640 1 --2- 172.21.15.16:0/2568161398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c936c7a4d0 0x55c93704dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.122+0000 7fd887400640 1 -- 172.21.15.16:0/2568161398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c936c7a4d0 msgr2=0x55c93704dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.122+0000 7fd887400640 1 --2- 172.21.15.16:0/2568161398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c936c7a4d0 0x55c93704dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.122+0000 7fd887400640 1 -- 172.21.15.16:0/2568161398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c936c9b1b0 msgr2=0x55c93704e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.122+0000 7fd887400640 1 --2- 172.21.15.16:0/2568161398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c936c9b1b0 0x55c93704e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.122+0000 7fd887400640 1 -- 172.21.15.16:0/2568161398 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c93704e9c0 con 0x55c936c722f0 2024-08-31T06:09:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.123+0000 7fd887400640 1 --2- 172.21.15.16:0/2568161398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c936c722f0 0x55c937052850 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd8700e36a0 tx=0x7fd8700abf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80f87ab66ea0ea71 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.123+0000 7fd874c00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:09:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.123+0000 7fd874c00640 1 -- 172.21.15.16:0/2568161398 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8700cb060 con 0x55c936c722f0 2024-08-31T06:09:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.123+0000 7fd8953ca740 1 -- 172.21.15.16:0/2568161398 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8700ac040 con 0x55c936c722f0 2024-08-31T06:09:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.123+0000 7fd874c00640 1 -- 172.21.15.16:0/2568161398 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fd8700ab4b0 con 0x55c936c722f0 2024-08-31T06:09:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.124+0000 7fd874c00640 1 -- 172.21.15.16:0/2568161398 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd87008a050 con 0x55c936c722f0 2024-08-31T06:09:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.124+0000 7fd8953ca740 10 client.26071 fetch_fsmap learned FSMap version 31 2024-08-31T06:09:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.124+0000 7fd8953ca740 1 -- 172.21.15.16:0/2568161398 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c93704e9c0 con 0x55c936c722f0 2024-08-31T06:09:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.124+0000 7fd874c00640 1 -- 172.21.15.16:0/2568161398 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8700ac040 con 0x55c936c722f0 2024-08-31T06:09:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.124+0000 7fd874c00640 1 -- 172.21.15.16:0/2568161398 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fd8700cd2b0 con 0x55c936c722f0 2024-08-31T06:09:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.124+0000 7fd8953ca740 10 client.26071 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:09:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.124+0000 7fd8953ca740 10 client.26071 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:09:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.124+0000 7fd8953ca740 20 client.26071 populate_metadata read hostname 'smithi016' 2024-08-31T06:09:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.124+0000 7fd8953ca740 1 --2- 172.21.15.16:0/2568161398 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55c936c2f1c0 0x55c9370c8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.124+0000 7fd8953ca740 4 client.26071 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:09:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.124+0000 7fd8953ca740 1 -- 172.21.15.16:0/2568161398 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c936fb1960 con 0x55c936c2f1c0 2024-08-31T06:09:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.125+0000 7fd886a00640 1 --2- 172.21.15.16:0/2568161398 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55c936c2f1c0 0x55c9370c8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.125+0000 7fd886a00640 1 --2- 172.21.15.16:0/2568161398 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55c936c2f1c0 0x55c9370c8af0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bc6ee7dd38ccf29 server_cookie=a585f72e6638b445 in_seq=0 out_seq=0 2024-08-31T06:09:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.125+0000 7fd874c00640 10 client.26071 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:09:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.144+0000 7fd874c00640 1 -- 172.21.15.16:0/2568161398 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c936fb1960 con 0x55c936c2f1c0 2024-08-31T06:09:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.144+0000 7fd874c00640 10 client.26071 handle_command_reply: tid=0 2024-08-31T06:09:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.144+0000 7fd8953ca740 1 client.26071 shutdown 2024-08-31T06:09:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.145+0000 7fd8953ca740 1 -- 172.21.15.16:0/2568161398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c936c722f0 msgr2=0x55c937052850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.145+0000 7fd8953ca740 1 --2- 172.21.15.16:0/2568161398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c936c722f0 0x55c937052850 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd8700e36a0 tx=0x7fd8700abf10 comp rx=0 tx=0).stop 2024-08-31T06:09:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.145+0000 7fd8953ca740 1 -- 172.21.15.16:0/2568161398 shutdown_connections 2024-08-31T06:09:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.145+0000 7fd8953ca740 1 --2- 172.21.15.16:0/2568161398 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55c936c2f1c0 0x55c9370c8af0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.145+0000 7fd8953ca740 1 --2- 172.21.15.16:0/2568161398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c936c9b1b0 0x55c93704e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.145+0000 7fd8953ca740 1 --2- 172.21.15.16:0/2568161398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c936c7a4d0 0x55c93704dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.145+0000 7fd8953ca740 1 --2- 172.21.15.16:0/2568161398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c936c722f0 0x55c937052850 secure :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd8700e36a0 tx=0x7fd8700abf10 comp rx=0 tx=0).stop 2024-08-31T06:09:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.145+0000 7fd8953ca740 1 -- 172.21.15.16:0/2568161398 >> 172.21.15.16:0/2568161398 conn(0x55c936c8f300 msgr2=0x55c93703f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.145+0000 7fd874c00640 0 client.26071 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:09:18.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.146+0000 7fd8953ca740 1 -- 172.21.15.16:0/2568161398 shutdown_connections 2024-08-31T06:09:18.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.146+0000 7fd8953ca740 1 -- 172.21.15.16:0/2568161398 wait complete. 2024-08-31T06:09:18.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.147+0000 7fd8953ca740 20 client.26071 trim_cache size 0 max 16384 2024-08-31T06:09:18.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:09:18.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:09:18.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:09:18.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:09:18.149 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.148+0000 7fd88e200640 1 -- 172.21.15.16:0/3976004304 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd86c0b4c30 msgr2=0x7fd86c0b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:18.149 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.148+0000 7fd88e200640 1 --2- 172.21.15.16:0/3976004304 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd86c0b4c30 0x7fd86c0b7140 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd88002ac70 tx=0x7fd880004ab0 comp rx=0 tx=0).stop 2024-08-31T06:09:18.149 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.148+0000 7fd88e200640 1 -- 172.21.15.16:0/3976004304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8881022f0 msgr2=0x7fd8881a1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:18.149 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.148+0000 7fd88e200640 1 --2- 172.21.15.16:0/3976004304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8881022f0 0x7fd8881a1540 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd87000e9c0 tx=0x7fd87000ee80 comp rx=0 tx=0).stop 2024-08-31T06:09:18.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.148+0000 7fd88e200640 1 -- 172.21.15.16:0/3976004304 shutdown_connections 2024-08-31T06:09:18.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.148+0000 7fd88e200640 1 --2- 172.21.15.16:0/3976004304 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd86c0b4c30 0x7fd86c0b7140 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:18.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.148+0000 7fd88e200640 1 --2- 172.21.15.16:0/3976004304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd888102c40 0x7fd8881a1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:18.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.148+0000 7fd88e200640 1 --2- 172.21.15.16:0/3976004304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd88810fea0 0x7fd8881a1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:18.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.148+0000 7fd88e200640 1 --2- 172.21.15.16:0/3976004304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8881022f0 0x7fd8881a1540 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:18.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.148+0000 7fd88e200640 1 -- 172.21.15.16:0/3976004304 >> 172.21.15.16:0/3976004304 conn(0x7fd88807a4e0 msgr2=0x7fd888104500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:18.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.149+0000 7fd88e200640 1 -- 172.21.15.16:0/3976004304 shutdown_connections 2024-08-31T06:09:18.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:18.149+0000 7fd88e200640 1 -- 172.21.15.16:0/3976004304 wait complete. 2024-08-31T06:09:18.593 INFO:tasks.cephfs.filesystem:scrub status for tag:78d6ef02-8551-45e8-ad60-9627eb573fc1 - {'path': '/', 'tag': '78d6ef02-8551-45e8-ad60-9627eb573fc1', 'options': 'recursive,force'} 2024-08-31T06:09:18.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:18 smithi155 ceph-mon[21742]: pgmap v426: 97 pgs: 97 active+clean; 1.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 32 MiB/s wr, 1.66k op/s 2024-08-31T06:09:18.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:18 smithi016 ceph-mon[17194]: pgmap v426: 97 pgs: 97 active+clean; 1.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 32 MiB/s wr, 1.66k op/s 2024-08-31T06:09:18.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:18 smithi184 ceph-mon[22331]: pgmap v426: 97 pgs: 97 active+clean; 1.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 32 MiB/s wr, 1.66k op/s 2024-08-31T06:09:20.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:20 smithi155 ceph-mon[21742]: pgmap v427: 97 pgs: 97 active+clean; 1.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 27 MiB/s wr, 1.55k op/s 2024-08-31T06:09:20.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:20 smithi016 ceph-mon[17194]: pgmap v427: 97 pgs: 97 active+clean; 1.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 27 MiB/s wr, 1.55k op/s 2024-08-31T06:09:20.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:20 smithi184 ceph-mon[22331]: pgmap v427: 97 pgs: 97 active+clean; 1.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 27 MiB/s wr, 1.55k op/s 2024-08-31T06:09:21.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 115.0 s, 2890.0 tps, lat 34.763 ms stddev 45.366 2024-08-31T06:09:22.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:22 smithi155 ceph-mon[21742]: pgmap v428: 97 pgs: 97 active+clean; 1.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 14 MiB/s wr, 964 op/s 2024-08-31T06:09:22.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:22 smithi016 ceph-mon[17194]: pgmap v428: 97 pgs: 97 active+clean; 1.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 14 MiB/s wr, 964 op/s 2024-08-31T06:09:22.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:22 smithi184 ceph-mon[22331]: pgmap v428: 97 pgs: 97 active+clean; 1.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 14 MiB/s wr, 964 op/s 2024-08-31T06:09:23.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:23 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:09:23.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:23 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:09:23.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:23 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:09:24.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:24 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:09:24.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:24 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:09:24.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:24 smithi155 ceph-mon[21742]: pgmap v429: 97 pgs: 97 active+clean; 1.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 8.9 MiB/s wr, 823 op/s 2024-08-31T06:09:24.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:24 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:09:24.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:24 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:09:24.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:24 smithi016 ceph-mon[17194]: pgmap v429: 97 pgs: 97 active+clean; 1.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 8.9 MiB/s wr, 823 op/s 2024-08-31T06:09:24.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:24 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:09:24.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:24 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:09:24.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:24 smithi184 ceph-mon[22331]: pgmap v429: 97 pgs: 97 active+clean; 1.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 8.9 MiB/s wr, 823 op/s 2024-08-31T06:09:26.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 120.0 s, 2711.2 tps, lat 36.928 ms stddev 49.489 2024-08-31T06:09:26.871 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:26 smithi155 ceph-mon[21742]: pgmap v430: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 9.0 MiB/s wr, 842 op/s 2024-08-31T06:09:26.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:26 smithi016 ceph-mon[17194]: pgmap v430: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 9.0 MiB/s wr, 842 op/s 2024-08-31T06:09:26.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:26 smithi184 ceph-mon[22331]: pgmap v430: 97 pgs: 97 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 9.0 MiB/s wr, 842 op/s 2024-08-31T06:09:28.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:28 smithi155 ceph-mon[21742]: pgmap v431: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 6.7 MiB/s wr, 677 op/s 2024-08-31T06:09:28.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:28 smithi016 ceph-mon[17194]: pgmap v431: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 6.7 MiB/s wr, 677 op/s 2024-08-31T06:09:28.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:28 smithi184 ceph-mon[22331]: pgmap v431: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 6.7 MiB/s wr, 677 op/s 2024-08-31T06:09:30.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:30 smithi155 ceph-mon[21742]: pgmap v432: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 8.1 MiB/s wr, 748 op/s 2024-08-31T06:09:30.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:30 smithi016 ceph-mon[17194]: pgmap v432: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 8.1 MiB/s wr, 748 op/s 2024-08-31T06:09:30.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:30 smithi184 ceph-mon[22331]: pgmap v432: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 8.1 MiB/s wr, 748 op/s 2024-08-31T06:09:31.471 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.468+0000 7f1911400640 1 -- 172.21.15.16:0/2264361280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f190c109b00 msgr2=0x7f190c110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:31.471 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.468+0000 7f1911400640 1 --2- 172.21.15.16:0/2264361280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f190c109b00 0x7f190c110330 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f18f4009860 tx=0x7f18f402a340 comp rx=0 tx=0).stop 2024-08-31T06:09:31.471 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.468+0000 7f1911400640 1 -- 172.21.15.16:0/2264361280 shutdown_connections 2024-08-31T06:09:31.471 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.468+0000 7f1911400640 1 --2- 172.21.15.16:0/2264361280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f190c109b00 0x7f190c110330 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:31.471 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.468+0000 7f1911400640 1 --2- 172.21.15.16:0/2264361280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f190c1091b0 0x7f190c1095c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:31.471 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.468+0000 7f1911400640 1 --2- 172.21.15.16:0/2264361280 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f190c103880 0x7f190c103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:31.471 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.468+0000 7f1911400640 1 -- 172.21.15.16:0/2264361280 >> 172.21.15.16:0/2264361280 conn(0x7f190c0ff2c0 msgr2=0x7f190c101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:31.472 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.469+0000 7f1911400640 1 -- 172.21.15.16:0/2264361280 shutdown_connections 2024-08-31T06:09:31.472 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.469+0000 7f1911400640 1 -- 172.21.15.16:0/2264361280 wait complete. 2024-08-31T06:09:31.473 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.471+0000 7f1911400640 1 Processor -- start 2024-08-31T06:09:31.473 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.471+0000 7f1911400640 1 -- start start 2024-08-31T06:09:31.475 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.472+0000 7f1911400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f190c103880 0x7f190c1a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:31.475 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.472+0000 7f1911400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f190c1091b0 0x7f190c1a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:31.475 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.472+0000 7f1911400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f190c109b00 0x7f190c1a5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:31.475 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.472+0000 7f1911400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f190c119de0 con 0x7f190c109b00 2024-08-31T06:09:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.472+0000 7f1911400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f190c119c60 con 0x7f190c103880 2024-08-31T06:09:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.472+0000 7f1911400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f190c119f60 con 0x7f190c1091b0 2024-08-31T06:09:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.473+0000 7f1909e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f190c1091b0 0x7f190c1a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.473+0000 7f1909e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f190c1091b0 0x7f190c1a0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:48812/0 (socket says 172.21.15.16:48812) 2024-08-31T06:09:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.473+0000 7f1909e00640 1 -- 172.21.15.16:0/442978678 learned_addr learned my addr 172.21.15.16:0/442978678 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.473+0000 7f190b200640 1 --2- 172.21.15.16:0/442978678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f190c109b00 0x7f190c1a5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.473+0000 7f190a800640 1 --2- 172.21.15.16:0/442978678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f190c103880 0x7f190c1a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.474+0000 7f1909e00640 1 -- 172.21.15.16:0/442978678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f190c103880 msgr2=0x7f190c1a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.474+0000 7f1909e00640 1 --2- 172.21.15.16:0/442978678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f190c103880 0x7f190c1a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.474+0000 7f1909e00640 1 -- 172.21.15.16:0/442978678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f190c109b00 msgr2=0x7f190c1a5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.474+0000 7f1909e00640 1 --2- 172.21.15.16:0/442978678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f190c109b00 0x7f190c1a5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.474+0000 7f1909e00640 1 -- 172.21.15.16:0/442978678 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f190c1a66b0 con 0x7f190c1091b0 2024-08-31T06:09:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.475+0000 7f1909e00640 1 --2- 172.21.15.16:0/442978678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f190c1091b0 0x7f190c1a0d20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00bb20 tx=0x7f18fc0091e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.476+0000 7f1903400640 1 -- 172.21.15.16:0/442978678 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18fc009af0 con 0x7f190c1091b0 2024-08-31T06:09:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.476+0000 7f1903400640 1 -- 172.21.15.16:0/442978678 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18fc009c90 con 0x7f190c1091b0 2024-08-31T06:09:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.476+0000 7f1911400640 1 -- 172.21.15.16:0/442978678 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f190c1a69a0 con 0x7f190c1091b0 2024-08-31T06:09:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.476+0000 7f1911400640 1 -- 172.21.15.16:0/442978678 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f190c1ad130 con 0x7f190c1091b0 2024-08-31T06:09:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.477+0000 7f1903400640 1 -- 172.21.15.16:0/442978678 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18fc015620 con 0x7f190c1091b0 2024-08-31T06:09:31.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.484+0000 7f1903400640 1 -- 172.21.15.16:0/442978678 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f18fc016420 con 0x7f190c1091b0 2024-08-31T06:09:31.490 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.488+0000 7f191861a740 1 Processor -- start 2024-08-31T06:09:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.490+0000 7f1903400640 1 --2- 172.21.15.16:0/442978678 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f18f00b4e70 0x7f18f00b7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:31.494 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.490+0000 7f1903400640 1 -- 172.21.15.16:0/442978678 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f18fc0e5050 con 0x7f190c1091b0 2024-08-31T06:09:31.494 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.491+0000 7f191861a740 1 -- start start 2024-08-31T06:09:31.494 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.491+0000 7f190b200640 1 --2- 172.21.15.16:0/442978678 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f18f00b4e70 0x7f18f00b7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.493+0000 7f190b200640 1 --2- 172.21.15.16:0/442978678 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f18f00b4e70 0x7f18f00b7380 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f18f40303f0 tx=0x7f18f4030380 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.493+0000 7f191861a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d39908a1b0 0x55d39942d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.493+0000 7f191861a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3990612f0 0x55d39942dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.493+0000 7f191861a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3990694d0 0x55d39942e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.493+0000 7f191861a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d3993a2f70 con 0x55d3990694d0 2024-08-31T06:09:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.493+0000 7f191861a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d399392500 con 0x55d39908a1b0 2024-08-31T06:09:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.493+0000 7f191861a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d3993964b0 con 0x55d3990612f0 2024-08-31T06:09:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.494+0000 7f1909e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3990612f0 0x55d39942dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.494+0000 7f1909e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3990612f0 0x55d39942dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:48816/0 (socket says 172.21.15.16:48816) 2024-08-31T06:09:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.494+0000 7f1909e00640 1 -- 172.21.15.16:0/4120356119 learned_addr learned my addr 172.21.15.16:0/4120356119 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.494+0000 7f1909e00640 1 -- 172.21.15.16:0/4120356119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d39908a1b0 msgr2=0x55d39942d690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:09:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.494+0000 7f1909e00640 1 --2- 172.21.15.16:0/4120356119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d39908a1b0 0x55d39942d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.494+0000 7f1909e00640 1 -- 172.21.15.16:0/4120356119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3990694d0 msgr2=0x55d39942e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.494+0000 7f190b200640 1 --2- 172.21.15.16:0/4120356119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3990694d0 0x55d39942e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.495+0000 7f1909e00640 1 --2- 172.21.15.16:0/4120356119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3990694d0 0x55d39942e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.495+0000 7f1909e00640 1 -- 172.21.15.16:0/4120356119 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d39934cc10 con 0x55d3990612f0 2024-08-31T06:09:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.496+0000 7f1909e00640 1 --2- 172.21.15.16:0/4120356119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3990612f0 0x55d39942dbd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f18fc0ab6d0 tx=0x7f18fc009e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b63180a67f272bb4 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:31.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.496+0000 7f18fb400640 1 -- 172.21.15.16:0/4120356119 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18fc0a5e60 con 0x55d3990612f0 2024-08-31T06:09:31.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.496+0000 7f18fb400640 1 -- 172.21.15.16:0/4120356119 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18fc0ab250 con 0x55d3990612f0 2024-08-31T06:09:31.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.496+0000 7f18fb400640 1 -- 172.21.15.16:0/4120356119 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18fc016710 con 0x55d3990612f0 2024-08-31T06:09:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.498+0000 7f191861a740 1 -- 172.21.15.16:0/4120356119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3990612f0 msgr2=0x55d39942dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.498+0000 7f191861a740 1 --2- 172.21.15.16:0/4120356119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3990612f0 0x55d39942dbd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f18fc0ab6d0 tx=0x7f18fc009e40 comp rx=0 tx=0).stop 2024-08-31T06:09:31.501 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.498+0000 7f191861a740 1 -- 172.21.15.16:0/4120356119 shutdown_connections 2024-08-31T06:09:31.501 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.498+0000 7f191861a740 1 --2- 172.21.15.16:0/4120356119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3990612f0 0x55d39942dbd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:31.501 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.498+0000 7f191861a740 1 --2- 172.21.15.16:0/4120356119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3990694d0 0x55d39942e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:31.501 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.498+0000 7f191861a740 1 --2- 172.21.15.16:0/4120356119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d39908a1b0 0x55d39942d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:31.501 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.498+0000 7f191861a740 1 -- 172.21.15.16:0/4120356119 >> 172.21.15.16:0/4120356119 conn(0x55d39907e300 msgr2=0x55d39942b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:31.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.499+0000 7f191861a740 1 -- 172.21.15.16:0/4120356119 shutdown_connections 2024-08-31T06:09:31.502 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.500+0000 7f191861a740 1 -- 172.21.15.16:0/4120356119 wait complete. 2024-08-31T06:09:31.505 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.502+0000 7f191861a740 1 -- start start 2024-08-31T06:09:31.505 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.502+0000 7f191861a740 1 Processor -- start 2024-08-31T06:09:31.506 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.503+0000 7f191861a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3990612f0 0x55d399438c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.504+0000 7f191861a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3990694d0 0x55d3994391d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.504+0000 7f191861a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d39908a1b0 0x55d399439730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.504+0000 7f191861a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d3992cbab0 con 0x55d3990612f0 2024-08-31T06:09:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.504+0000 7f191861a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d399339320 con 0x55d3990694d0 2024-08-31T06:09:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.504+0000 7f191861a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d39943bd50 con 0x55d39908a1b0 2024-08-31T06:09:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.504+0000 7f191861a740 10 client.0 _ll_register_callbacks cb 0x55d399372e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:09:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.504+0000 7f1909e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3990694d0 0x55d3994391d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.504+0000 7f1909e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3990694d0 0x55d3994391d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:33840/0 (socket says 172.21.15.16:33840) 2024-08-31T06:09:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.504+0000 7f1909e00640 1 -- 172.21.15.16:0/1788532583 learned_addr learned my addr 172.21.15.16:0/1788532583 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:31.508 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.505+0000 7f1909e00640 1 -- 172.21.15.16:0/1788532583 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d39908a1b0 msgr2=0x55d399439730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:09:31.508 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.505+0000 7f1909e00640 1 --2- 172.21.15.16:0/1788532583 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d39908a1b0 0x55d399439730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:31.508 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.505+0000 7f1909e00640 1 -- 172.21.15.16:0/1788532583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3990612f0 msgr2=0x55d399438c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:09:31.508 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.505+0000 7f190a800640 1 --2- 172.21.15.16:0/1788532583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3990612f0 0x55d399438c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:31.508 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.505+0000 7f1909e00640 1 --2- 172.21.15.16:0/1788532583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3990612f0 0x55d399438c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:31.508 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.505+0000 7f1909e00640 1 -- 172.21.15.16:0/1788532583 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3990f5790 con 0x55d3990694d0 2024-08-31T06:09:31.508 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.506+0000 7f190a800640 1 --2- 172.21.15.16:0/1788532583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3990612f0 0x55d399438c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:09:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.507+0000 7f1909e00640 1 --2- 172.21.15.16:0/1788532583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3990694d0 0x55d3994391d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f18fc0a89b0 tx=0x7f18fc0ab510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebebd86ce1f01aa4 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.507+0000 7f18efe00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:09:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.508+0000 7f18efe00640 1 -- 172.21.15.16:0/1788532583 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18fc096050 con 0x55d3990694d0 2024-08-31T06:09:31.511 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.508+0000 7f18efe00640 1 -- 172.21.15.16:0/1788532583 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f18fc0aabb0 con 0x55d3990694d0 2024-08-31T06:09:31.511 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.509+0000 7f191861a740 1 -- 172.21.15.16:0/1788532583 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18fc0ab250 con 0x55d3990694d0 2024-08-31T06:09:31.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.509+0000 7f18efe00640 1 -- 172.21.15.16:0/1788532583 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f18fc0a5e60 con 0x55d3990694d0 2024-08-31T06:09:31.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.509+0000 7f191861a740 10 client.26078 fetch_fsmap learned FSMap version 31 2024-08-31T06:09:31.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.509+0000 7f191861a740 1 -- 172.21.15.16:0/1788532583 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d39916e620 con 0x55d3990694d0 2024-08-31T06:09:31.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.510+0000 7f18efe00640 1 -- 172.21.15.16:0/1788532583 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18fc0ab250 con 0x55d3990694d0 2024-08-31T06:09:31.513 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.510+0000 7f18efe00640 1 -- 172.21.15.16:0/1788532583 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f18fc0c6140 con 0x55d3990694d0 2024-08-31T06:09:31.513 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.511+0000 7f191861a740 10 client.26078 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:09:31.513 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.511+0000 7f191861a740 10 client.26078 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:09:31.513 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.511+0000 7f191861a740 20 client.26078 populate_metadata read hostname 'smithi016' 2024-08-31T06:09:31.513 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.511+0000 7f191861a740 1 --2- 172.21.15.16:0/1788532583 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d399048d80 0x55d39944abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:31.513 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.511+0000 7f191861a740 4 client.26078 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:09:31.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.511+0000 7f191861a740 1 -- 172.21.15.16:0/1788532583 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3993a4ec0 con 0x55d399048d80 2024-08-31T06:09:31.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.512+0000 7f190a800640 1 --2- 172.21.15.16:0/1788532583 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d399048d80 0x55d39944abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:31.515 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.513+0000 7f190a800640 1 --2- 172.21.15.16:0/1788532583 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d399048d80 0x55d39944abc0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fe474f05fe2aa73 server_cookie=2741fd421daf373 in_seq=0 out_seq=0 2024-08-31T06:09:31.516 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.514+0000 7f18efe00640 10 client.26078 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:09:31.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.514+0000 7f18efe00640 1 -- 172.21.15.16:0/1788532583 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d3993a4ec0 con 0x55d399048d80 2024-08-31T06:09:31.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.514+0000 7f18efe00640 10 client.26078 handle_command_reply: tid=0 2024-08-31T06:09:31.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.515+0000 7f191861a740 1 client.26078 shutdown 2024-08-31T06:09:31.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.516+0000 7f191861a740 1 -- 172.21.15.16:0/1788532583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3990694d0 msgr2=0x55d3994391d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:31.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.516+0000 7f191861a740 1 --2- 172.21.15.16:0/1788532583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3990694d0 0x55d3994391d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f18fc0a89b0 tx=0x7f18fc0ab510 comp rx=0 tx=0).stop 2024-08-31T06:09:31.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.516+0000 7f191861a740 1 -- 172.21.15.16:0/1788532583 shutdown_connections 2024-08-31T06:09:31.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.516+0000 7f191861a740 1 --2- 172.21.15.16:0/1788532583 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d399048d80 0x55d39944abc0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:31.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.516+0000 7f191861a740 1 --2- 172.21.15.16:0/1788532583 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d39908a1b0 0x55d399439730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:31.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.516+0000 7f191861a740 1 --2- 172.21.15.16:0/1788532583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3990694d0 0x55d3994391d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:31.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.516+0000 7f191861a740 1 --2- 172.21.15.16:0/1788532583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3990612f0 0x55d399438c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:31.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.516+0000 7f191861a740 1 -- 172.21.15.16:0/1788532583 >> 172.21.15.16:0/1788532583 conn(0x55d39907e300 msgr2=0x55d39942e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:31.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.516+0000 7f18efe00640 0 client.26078 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:09:31.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.517+0000 7f191861a740 1 -- 172.21.15.16:0/1788532583 shutdown_connections 2024-08-31T06:09:31.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.517+0000 7f191861a740 1 -- 172.21.15.16:0/1788532583 wait complete. 2024-08-31T06:09:31.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.517+0000 7f191861a740 20 client.26078 trim_cache size 0 max 16384 2024-08-31T06:09:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.566+0000 7f191861a740 1 Processor -- start 2024-08-31T06:09:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.566+0000 7f191861a740 1 -- start start 2024-08-31T06:09:31.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.566+0000 7f191861a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3990612f0 0x55d39942a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:31.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.566+0000 7f191861a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3990694d0 0x55d39943a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:31.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.566+0000 7f191861a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d39908a1b0 0x55d399435e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:31.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.567+0000 7f1909e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3990694d0 0x55d39943a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:31.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.567+0000 7f190b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d39908a1b0 0x55d399435e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:31.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.567+0000 7f190a800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3990612f0 0x55d39942a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:31.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.568+0000 7f190a800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3990612f0 0x55d39942a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:48830/0 (socket says 172.21.15.16:48830) 2024-08-31T06:09:31.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.568+0000 7f190a800640 1 -- 172.21.15.16:0/2839771107 learned_addr learned my addr 172.21.15.16:0/2839771107 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:31.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.568+0000 7f191861a740 1 -- 172.21.15.16:0/2839771107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f18e80110f0 con 0x55d39908a1b0 2024-08-31T06:09:31.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.568+0000 7f191861a740 1 -- 172.21.15.16:0/2839771107 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f18e800e900 con 0x55d3990694d0 2024-08-31T06:09:31.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.568+0000 7f191861a740 1 -- 172.21.15.16:0/2839771107 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f18e800c750 con 0x55d3990612f0 2024-08-31T06:09:31.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.568+0000 7f1909e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3990694d0 0x55d39943a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:33844/0 (socket says 172.21.15.16:33844) 2024-08-31T06:09:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.568+0000 7f190a800640 1 -- 172.21.15.16:0/2839771107 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3990694d0 msgr2=0x55d39943a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.568+0000 7f190a800640 1 --2- 172.21.15.16:0/2839771107 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3990694d0 0x55d39943a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.568+0000 7f190a800640 1 -- 172.21.15.16:0/2839771107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d39908a1b0 msgr2=0x55d399435e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.569+0000 7f190a800640 1 --2- 172.21.15.16:0/2839771107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d39908a1b0 0x55d399435e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.569+0000 7f190a800640 1 -- 172.21.15.16:0/2839771107 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d399439f50 con 0x55d3990612f0 2024-08-31T06:09:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.569+0000 7f190b200640 1 --2- 172.21.15.16:0/2839771107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d39908a1b0 0x55d399435e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:09:31.572 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.570+0000 7f190a800640 1 --2- 172.21.15.16:0/2839771107 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3990612f0 0x55d39942a4e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1904010580 tx=0x7f19040119f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82f9eb5ebfb1b6b5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.570+0000 7f18faa00640 1 -- 172.21.15.16:0/2839771107 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190401f040 con 0x55d3990612f0 2024-08-31T06:09:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.570+0000 7f18faa00640 1 -- 172.21.15.16:0/2839771107 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1904003b70 con 0x55d3990612f0 2024-08-31T06:09:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.571+0000 7f18faa00640 1 -- 172.21.15.16:0/2839771107 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190401f040 con 0x55d3990612f0 2024-08-31T06:09:31.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.572+0000 7f191861a740 1 -- 172.21.15.16:0/2839771107 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3990612f0 msgr2=0x55d39942a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:31.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.572+0000 7f191861a740 1 --2- 172.21.15.16:0/2839771107 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3990612f0 0x55d39942a4e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1904010580 tx=0x7f19040119f0 comp rx=0 tx=0).stop 2024-08-31T06:09:31.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.572+0000 7f191861a740 1 -- 172.21.15.16:0/2839771107 shutdown_connections 2024-08-31T06:09:31.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.572+0000 7f191861a740 1 --2- 172.21.15.16:0/2839771107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d39908a1b0 0x55d399435e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:31.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.572+0000 7f191861a740 1 --2- 172.21.15.16:0/2839771107 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3990694d0 0x55d39943a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:31.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.572+0000 7f191861a740 1 --2- 172.21.15.16:0/2839771107 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3990612f0 0x55d39942a4e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:31.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.572+0000 7f191861a740 1 -- 172.21.15.16:0/2839771107 >> 172.21.15.16:0/2839771107 conn(0x55d39907e300 msgr2=0x55d39943a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:31.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.573+0000 7f191861a740 1 -- 172.21.15.16:0/2839771107 shutdown_connections 2024-08-31T06:09:31.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.573+0000 7f191861a740 1 -- 172.21.15.16:0/2839771107 wait complete. 2024-08-31T06:09:31.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.575+0000 7f191861a740 1 -- start start 2024-08-31T06:09:31.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.575+0000 7f191861a740 1 Processor -- start 2024-08-31T06:09:31.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.578+0000 7f191861a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3990612f0 0x55d39944da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:31.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.578+0000 7f191861a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3990694d0 0x55d39944dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:31.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.578+0000 7f191861a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d39908a1b0 0x55d3994aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:31.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.579+0000 7f190b200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d39908a1b0 0x55d3994aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:31.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.579+0000 7f190b200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d39908a1b0 0x55d3994aa570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:33860/0 (socket says 172.21.15.16:33860) 2024-08-31T06:09:31.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.579+0000 7f190a800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3990612f0 0x55d39944da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:31.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.579+0000 7f190a800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3990612f0 0x55d39944da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:48834/0 (socket says 172.21.15.16:48834) 2024-08-31T06:09:31.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.579+0000 7f190b200640 1 -- 172.21.15.16:0/215921492 learned_addr learned my addr 172.21.15.16:0/215921492 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:31.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.579+0000 7f191861a740 1 -- 172.21.15.16:0/215921492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f18e800cdc0 con 0x55d3990694d0 2024-08-31T06:09:31.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.579+0000 7f191861a740 1 -- 172.21.15.16:0/215921492 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f18e800e230 con 0x55d39908a1b0 2024-08-31T06:09:31.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.579+0000 7f191861a740 1 -- 172.21.15.16:0/215921492 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f18e800d490 con 0x55d3990612f0 2024-08-31T06:09:31.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.580+0000 7f191861a740 10 client.0 _ll_register_callbacks cb 0x7f18fc0ab120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:09:31.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.580+0000 7f190b200640 1 -- 172.21.15.16:0/215921492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3990612f0 msgr2=0x55d39944da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:31.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.580+0000 7f190b200640 1 --2- 172.21.15.16:0/215921492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3990612f0 0x55d39944da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:31.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.580+0000 7f190b200640 1 -- 172.21.15.16:0/215921492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3990694d0 msgr2=0x55d39944dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:31.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.580+0000 7f190b200640 1 --2- 172.21.15.16:0/215921492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3990694d0 0x55d39944dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:31.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.580+0000 7f190b200640 1 -- 172.21.15.16:0/215921492 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3994aabe0 con 0x55d39908a1b0 2024-08-31T06:09:31.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.580+0000 7f190b200640 1 --2- 172.21.15.16:0/215921492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d39908a1b0 0x55d3994aa570 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f18f4009230 tx=0x7f18f4005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=383b48f9177b3ff4 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:31.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.580+0000 7f18efe00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:09:31.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.581+0000 7f18efe00640 1 -- 172.21.15.16:0/215921492 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f404d030 con 0x55d39908a1b0 2024-08-31T06:09:31.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.581+0000 7f18efe00640 1 -- 172.21.15.16:0/215921492 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f18f4003800 con 0x55d39908a1b0 2024-08-31T06:09:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.581+0000 7f191861a740 1 -- 172.21.15.16:0/215921492 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1904003b70 con 0x55d39908a1b0 2024-08-31T06:09:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.581+0000 7f18efe00640 1 -- 172.21.15.16:0/215921492 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f18f4037040 con 0x55d39908a1b0 2024-08-31T06:09:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.582+0000 7f191861a740 10 client.26084 fetch_fsmap learned FSMap version 31 2024-08-31T06:09:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.582+0000 7f191861a740 1 -- 172.21.15.16:0/215921492 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d3994b5220 con 0x55d39908a1b0 2024-08-31T06:09:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.582+0000 7f18efe00640 1 -- 172.21.15.16:0/215921492 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1904003b70 con 0x55d39908a1b0 2024-08-31T06:09:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.582+0000 7f18efe00640 1 -- 172.21.15.16:0/215921492 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f18f40328d0 con 0x55d39908a1b0 2024-08-31T06:09:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.582+0000 7f191861a740 10 client.26084 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:09:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.582+0000 7f191861a740 10 client.26084 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:09:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.583+0000 7f191861a740 20 client.26084 populate_metadata read hostname 'smithi016' 2024-08-31T06:09:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.583+0000 7f191861a740 1 --2- 172.21.15.16:0/215921492 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d39901e1c0 0x55d3994b7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.583+0000 7f191861a740 4 client.26084 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:09:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.583+0000 7f191861a740 1 -- 172.21.15.16:0/215921492 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d3993a0960 con 0x55d39901e1c0 2024-08-31T06:09:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.583+0000 7f190a800640 1 --2- 172.21.15.16:0/215921492 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d39901e1c0 0x55d3994b7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.584+0000 7f18efe00640 10 client.26084 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:09:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.585+0000 7f190a800640 1 --2- 172.21.15.16:0/215921492 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d39901e1c0 0x55d3994b7a90 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3eab07a122589cc4 server_cookie=c93de554d84121f in_seq=0 out_seq=0 2024-08-31T06:09:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.601+0000 7f18efe00640 1 -- 172.21.15.16:0/215921492 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d3993a0960 con 0x55d39901e1c0 2024-08-31T06:09:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.601+0000 7f18efe00640 10 client.26084 handle_command_reply: tid=0 2024-08-31T06:09:31.604 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.602+0000 7f191861a740 1 client.26084 shutdown 2024-08-31T06:09:31.605 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.602+0000 7f191861a740 1 -- 172.21.15.16:0/215921492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d39908a1b0 msgr2=0x55d3994aa570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:31.605 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.602+0000 7f191861a740 1 --2- 172.21.15.16:0/215921492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d39908a1b0 0x55d3994aa570 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f18f4009230 tx=0x7f18f4005f90 comp rx=0 tx=0).stop 2024-08-31T06:09:31.605 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.603+0000 7f191861a740 1 -- 172.21.15.16:0/215921492 shutdown_connections 2024-08-31T06:09:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.603+0000 7f191861a740 1 --2- 172.21.15.16:0/215921492 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d39901e1c0 0x55d3994b7a90 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.603+0000 7f191861a740 1 --2- 172.21.15.16:0/215921492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d39908a1b0 0x55d3994aa570 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.603+0000 7f191861a740 1 --2- 172.21.15.16:0/215921492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3990694d0 0x55d39944dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.603+0000 7f191861a740 1 --2- 172.21.15.16:0/215921492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3990612f0 0x55d39944da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.603+0000 7f191861a740 1 -- 172.21.15.16:0/215921492 >> 172.21.15.16:0/215921492 conn(0x55d39907e300 msgr2=0x55d39942a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.603+0000 7f18efe00640 0 client.26084 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:09:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.604+0000 7f191861a740 1 -- 172.21.15.16:0/215921492 shutdown_connections 2024-08-31T06:09:31.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.604+0000 7f191861a740 1 -- 172.21.15.16:0/215921492 wait complete. 2024-08-31T06:09:31.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.604+0000 7f191861a740 20 client.26084 trim_cache size 0 max 16384 2024-08-31T06:09:31.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:09:31.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:09:31.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:09:31.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:09:31.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.605+0000 7f1911400640 1 -- 172.21.15.16:0/442978678 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f18f00b4e70 msgr2=0x7f18f00b7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:31.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.606+0000 7f1911400640 1 --2- 172.21.15.16:0/442978678 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f18f00b4e70 0x7f18f00b7380 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f18f40303f0 tx=0x7f18f4030380 comp rx=0 tx=0).stop 2024-08-31T06:09:31.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.606+0000 7f1911400640 1 -- 172.21.15.16:0/442978678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f190c1091b0 msgr2=0x7f190c1a0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:31.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.606+0000 7f1911400640 1 --2- 172.21.15.16:0/442978678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f190c1091b0 0x7f190c1a0d20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00bb20 tx=0x7f18fc0091e0 comp rx=0 tx=0).stop 2024-08-31T06:09:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.606+0000 7f1911400640 1 -- 172.21.15.16:0/442978678 shutdown_connections 2024-08-31T06:09:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.606+0000 7f1911400640 1 --2- 172.21.15.16:0/442978678 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f18f00b4e70 0x7f18f00b7380 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.606+0000 7f1911400640 1 --2- 172.21.15.16:0/442978678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f190c1091b0 0x7f190c1a0d20 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.606+0000 7f1911400640 1 --2- 172.21.15.16:0/442978678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f190c109b00 0x7f190c1a5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.606+0000 7f1911400640 1 --2- 172.21.15.16:0/442978678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f190c103880 0x7f190c1a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.606+0000 7f1911400640 1 -- 172.21.15.16:0/442978678 >> 172.21.15.16:0/442978678 conn(0x7f190c0ff2c0 msgr2=0x7f190c1016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.606+0000 7f1911400640 1 -- 172.21.15.16:0/442978678 shutdown_connections 2024-08-31T06:09:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:31.607+0000 7f1911400640 1 -- 172.21.15.16:0/442978678 wait complete. 2024-08-31T06:09:31.625 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 125.0 s, 2837.1 tps, lat 35.233 ms stddev 38.923 2024-08-31T06:09:32.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:32 smithi155 ceph-mon[21742]: pgmap v433: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.8 MiB/s wr, 683 op/s 2024-08-31T06:09:32.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:32 smithi016 ceph-mon[17194]: pgmap v433: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.8 MiB/s wr, 683 op/s 2024-08-31T06:09:32.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:32 smithi184 ceph-mon[22331]: pgmap v433: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.8 MiB/s wr, 683 op/s 2024-08-31T06:09:34.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:34 smithi016 ceph-mon[17194]: pgmap v434: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.4 MiB/s wr, 744 op/s 2024-08-31T06:09:34.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:34 smithi184 ceph-mon[22331]: pgmap v434: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.4 MiB/s wr, 744 op/s 2024-08-31T06:09:35.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:34 smithi155 ceph-mon[21742]: pgmap v434: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.4 MiB/s wr, 744 op/s 2024-08-31T06:09:36.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 130.0 s, 2966.2 tps, lat 33.405 ms stddev 37.886 2024-08-31T06:09:36.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:36 smithi016 ceph-mon[17194]: pgmap v435: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 8.1 MiB/s wr, 730 op/s 2024-08-31T06:09:36.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:36 smithi184 ceph-mon[22331]: pgmap v435: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 8.1 MiB/s wr, 730 op/s 2024-08-31T06:09:37.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:36 smithi155 ceph-mon[21742]: pgmap v435: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 8.1 MiB/s wr, 730 op/s 2024-08-31T06:09:38.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:38 smithi016 ceph-mon[17194]: pgmap v436: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 7.9 MiB/s wr, 660 op/s 2024-08-31T06:09:38.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:38 smithi184 ceph-mon[22331]: pgmap v436: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 7.9 MiB/s wr, 660 op/s 2024-08-31T06:09:39.121 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:38 smithi155 ceph-mon[21742]: pgmap v436: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 7.9 MiB/s wr, 660 op/s 2024-08-31T06:09:40.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:40 smithi016 ceph-mon[17194]: pgmap v437: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 9.5 MiB/s wr, 789 op/s 2024-08-31T06:09:40.923 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:40 smithi155 ceph-mon[21742]: pgmap v437: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 9.5 MiB/s wr, 789 op/s 2024-08-31T06:09:40.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:40 smithi184 ceph-mon[22331]: pgmap v437: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 9.5 MiB/s wr, 789 op/s 2024-08-31T06:09:41.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 135.0 s, 3128.5 tps, lat 32.068 ms stddev 36.851 2024-08-31T06:09:42.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:42 smithi016 ceph-mon[17194]: pgmap v438: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 8.5 MiB/s wr, 716 op/s 2024-08-31T06:09:42.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:42 smithi184 ceph-mon[22331]: pgmap v438: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 8.5 MiB/s wr, 716 op/s 2024-08-31T06:09:43.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:42 smithi155 ceph-mon[21742]: pgmap v438: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 8.5 MiB/s wr, 716 op/s 2024-08-31T06:09:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.903+0000 7f0827800640 1 -- 172.21.15.16:0/1976101754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f082010fdd0 msgr2=0x7f0820112230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.903+0000 7f0827800640 1 --2- 172.21.15.16:0/1976101754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f082010fdd0 0x7f0820112230 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0810009ca0 tx=0x7f081002a8c0 comp rx=0 tx=0).stop 2024-08-31T06:09:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.904+0000 7f0827800640 1 -- 172.21.15.16:0/1976101754 shutdown_connections 2024-08-31T06:09:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.904+0000 7f0827800640 1 --2- 172.21.15.16:0/1976101754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f082010fdd0 0x7f0820112230 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.904+0000 7f0827800640 1 --2- 172.21.15.16:0/1976101754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0820075ea0 0x7f082010f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.904+0000 7f0827800640 1 --2- 172.21.15.16:0/1976101754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0820075550 0x7f0820075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:44.907 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.904+0000 7f0827800640 1 -- 172.21.15.16:0/1976101754 >> 172.21.15.16:0/1976101754 conn(0x7f08200ff2c0 msgr2=0x7f0820101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:44.907 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.905+0000 7f0827800640 1 -- 172.21.15.16:0/1976101754 shutdown_connections 2024-08-31T06:09:44.907 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.905+0000 7f0827800640 1 -- 172.21.15.16:0/1976101754 wait complete. 2024-08-31T06:09:44.909 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.907+0000 7f0827800640 1 Processor -- start 2024-08-31T06:09:44.909 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.907+0000 7f0827800640 1 -- start start 2024-08-31T06:09:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.908+0000 7f0827800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0820075550 0x7f0820106820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.908+0000 7f0827800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0820075ea0 0x7f0820106d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.908+0000 7f0827800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f082010fdd0 0x7f08201072a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.908+0000 7f0827800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f08201159f0 con 0x7f0820075550 2024-08-31T06:09:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.908+0000 7f0827800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0820115870 con 0x7f082010fdd0 2024-08-31T06:09:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.908+0000 7f0827800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0820115b70 con 0x7f0820075ea0 2024-08-31T06:09:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.908+0000 7f0825800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f082010fdd0 0x7f08201072a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.908+0000 7f0825800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f082010fdd0 0x7f08201072a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:54464/0 (socket says 172.21.15.16:54464) 2024-08-31T06:09:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.908+0000 7f0824e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0820075550 0x7f0820106820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.908+0000 7f0825800640 1 -- 172.21.15.16:0/2861336112 learned_addr learned my addr 172.21.15.16:0/2861336112 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.908+0000 7f081fe00640 1 --2- 172.21.15.16:0/2861336112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0820075ea0 0x7f0820106d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.909+0000 7f0825800640 1 -- 172.21.15.16:0/2861336112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0820075ea0 msgr2=0x7f0820106d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.909+0000 7f0825800640 1 --2- 172.21.15.16:0/2861336112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0820075ea0 0x7f0820106d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.909+0000 7f0825800640 1 -- 172.21.15.16:0/2861336112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0820075550 msgr2=0x7f0820106820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:44.913 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.909+0000 7f0825800640 1 --2- 172.21.15.16:0/2861336112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0820075550 0x7f0820106820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:44.913 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.909+0000 7f0825800640 1 -- 172.21.15.16:0/2861336112 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08201ac8c0 con 0x7f082010fdd0 2024-08-31T06:09:44.913 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.909+0000 7f0825800640 1 --2- 172.21.15.16:0/2861336112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f082010fdd0 0x7f08201072a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f08100319f0 tx=0x7f0810031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:44.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.909+0000 7f081d600640 1 -- 172.21.15.16:0/2861336112 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0810003c30 con 0x7f082010fdd0 2024-08-31T06:09:44.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.909+0000 7f0827800640 1 -- 172.21.15.16:0/2861336112 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08201acb50 con 0x7f082010fdd0 2024-08-31T06:09:44.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.909+0000 7f081d600640 1 -- 172.21.15.16:0/2861336112 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0810003dd0 con 0x7f082010fdd0 2024-08-31T06:09:44.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.909+0000 7f081d600640 1 -- 172.21.15.16:0/2861336112 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081002fad0 con 0x7f082010fdd0 2024-08-31T06:09:44.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.910+0000 7f0827800640 1 -- 172.21.15.16:0/2861336112 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08201ad0f0 con 0x7f082010fdd0 2024-08-31T06:09:44.916 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:44 smithi016 ceph-mon[17194]: pgmap v439: 97 pgs: 97 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 8.6 MiB/s wr, 739 op/s 2024-08-31T06:09:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.914+0000 7f081d600640 1 -- 172.21.15.16:0/2861336112 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f081002fc70 con 0x7f082010fdd0 2024-08-31T06:09:44.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.916+0000 7f082e819740 1 Processor -- start 2024-08-31T06:09:44.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.916+0000 7f082e819740 1 -- start start 2024-08-31T06:09:44.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.917+0000 7f082e819740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ceb291b0 0x55e8ceecc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:44.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.917+0000 7f082e819740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8ceb002f0 0x55e8ceecce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:44.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.917+0000 7f082e819740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e8ceb084d0 0x55e8ceecd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:44.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.917+0000 7f082e819740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e8cee41f70 con 0x55e8ceb291b0 2024-08-31T06:09:44.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.917+0000 7f082e819740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e8cee31500 con 0x55e8ceb002f0 2024-08-31T06:09:44.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.917+0000 7f082e819740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e8cee354b0 con 0x55e8ceb084d0 2024-08-31T06:09:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.917+0000 7f0824e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8ceb002f0 0x55e8ceecce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.917+0000 7f0824e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8ceb002f0 0x55e8ceecce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:54474/0 (socket says 172.21.15.16:54474) 2024-08-31T06:09:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.917+0000 7f0824e00640 1 -- 172.21.15.16:0/2953585020 learned_addr learned my addr 172.21.15.16:0/2953585020 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.917+0000 7f081fe00640 1 --2- 172.21.15.16:0/2953585020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ceb291b0 0x55e8ceecc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.917+0000 7f081d600640 1 --2- 172.21.15.16:0/2861336112 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f08000b4d20 0x7f08000b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.917+0000 7f081fe00640 1 --2- 172.21.15.16:0/2953585020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e8ceb084d0 0x55e8ceecd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.918+0000 7f081d600640 1 -- 172.21.15.16:0/2861336112 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f08101069f0 con 0x7f082010fdd0 2024-08-31T06:09:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.918+0000 7f0824e00640 1 -- 172.21.15.16:0/2953585020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e8ceb084d0 msgr2=0x55e8ceecd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.918+0000 7f0824e00640 1 --2- 172.21.15.16:0/2953585020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e8ceb084d0 0x55e8ceecd3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.918+0000 7f0824e00640 1 -- 172.21.15.16:0/2953585020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ceb291b0 msgr2=0x55e8ceecc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.918+0000 7f0825800640 1 --2- 172.21.15.16:0/2861336112 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f08000b4d20 0x7f08000b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.918+0000 7f0824e00640 1 --2- 172.21.15.16:0/2953585020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ceb291b0 0x55e8ceecc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.918+0000 7f0824e00640 1 -- 172.21.15.16:0/2953585020 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8cedebc10 con 0x55e8ceb002f0 2024-08-31T06:09:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.918+0000 7f081fe00640 1 --2- 172.21.15.16:0/2953585020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ceb291b0 0x55e8ceecc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:09:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.919+0000 7f0824e00640 1 --2- 172.21.15.16:0/2953585020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8ceb002f0 0x55e8ceecce60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f081800bd40 tx=0x7f08180107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70a9f1e787eb1f3a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.919+0000 7f0816000640 1 -- 172.21.15.16:0/2953585020 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0818005220 con 0x55e8ceb002f0 2024-08-31T06:09:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.920+0000 7f0816000640 1 -- 172.21.15.16:0/2953585020 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0818004c50 con 0x55e8ceb002f0 2024-08-31T06:09:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.920+0000 7f0816000640 1 -- 172.21.15.16:0/2953585020 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0818011840 con 0x55e8ceb002f0 2024-08-31T06:09:44.923 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.921+0000 7f0825800640 1 --2- 172.21.15.16:0/2861336112 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f08000b4d20 0x7f08000b7230 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f08100cd070 tx=0x7f081004d000 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:44.923 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.921+0000 7f082e819740 1 -- 172.21.15.16:0/2953585020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8ceb002f0 msgr2=0x55e8ceecce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:44.923 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.921+0000 7f082e819740 1 --2- 172.21.15.16:0/2953585020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8ceb002f0 0x55e8ceecce60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f081800bd40 tx=0x7f08180107a0 comp rx=0 tx=0).stop 2024-08-31T06:09:44.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.922+0000 7f082e819740 1 -- 172.21.15.16:0/2953585020 shutdown_connections 2024-08-31T06:09:44.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.922+0000 7f082e819740 1 --2- 172.21.15.16:0/2953585020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e8ceb084d0 0x55e8ceecd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:44.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.922+0000 7f082e819740 1 --2- 172.21.15.16:0/2953585020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8ceb002f0 0x55e8ceecce60 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:44.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.922+0000 7f082e819740 1 --2- 172.21.15.16:0/2953585020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ceb291b0 0x55e8ceecc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:44.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.922+0000 7f082e819740 1 -- 172.21.15.16:0/2953585020 >> 172.21.15.16:0/2953585020 conn(0x55e8ceb1d300 msgr2=0x55e8ceeca810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:44.925 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.922+0000 7f082e819740 1 -- 172.21.15.16:0/2953585020 shutdown_connections 2024-08-31T06:09:44.926 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.923+0000 7f082e819740 1 -- 172.21.15.16:0/2953585020 wait complete. 2024-08-31T06:09:44.926 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.924+0000 7f082e819740 1 -- start start 2024-08-31T06:09:44.926 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.924+0000 7f082e819740 1 Processor -- start 2024-08-31T06:09:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.925+0000 7f082e819740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e8ceb002f0 0x55e8ceed7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.925+0000 7f082e819740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ceb084d0 0x55e8ceed8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.925+0000 7f082e819740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8ceb291b0 0x55e8ceed89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.925+0000 7f082e819740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e8ced6aab0 con 0x55e8ceb084d0 2024-08-31T06:09:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.925+0000 7f082e819740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e8cedd8320 con 0x55e8ceb291b0 2024-08-31T06:09:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.925+0000 7f082e819740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e8ceedb0a0 con 0x55e8ceb002f0 2024-08-31T06:09:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.925+0000 7f082e819740 10 client.0 _ll_register_callbacks cb 0x55e8cedde6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:09:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.925+0000 7f081fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e8ceb002f0 0x55e8ceed7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.925+0000 7f081fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e8ceb002f0 0x55e8ceed7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:33410/0 (socket says 172.21.15.16:33410) 2024-08-31T06:09:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.925+0000 7f081fe00640 1 -- 172.21.15.16:0/2959847522 learned_addr learned my addr 172.21.15.16:0/2959847522 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.925+0000 7f081fe00640 1 --2- 172.21.15.16:0/2959847522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8ceb291b0 0x55e8ceed89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.925+0000 7f0824e00640 1 --2- 172.21.15.16:0/2959847522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ceb084d0 0x55e8ceed8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.926+0000 7f081fe00640 1 -- 172.21.15.16:0/2959847522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8ceb291b0 msgr2=0x55e8ceed89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.926+0000 7f081fe00640 1 --2- 172.21.15.16:0/2959847522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8ceb291b0 0x55e8ceed89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.926+0000 7f081fe00640 1 -- 172.21.15.16:0/2959847522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ceb084d0 msgr2=0x55e8ceed8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.926+0000 7f081fe00640 1 --2- 172.21.15.16:0/2959847522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ceb084d0 0x55e8ceed8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.926+0000 7f081fe00640 1 -- 172.21.15.16:0/2959847522 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8ceb94790 con 0x55e8ceb002f0 2024-08-31T06:09:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.926+0000 7f081fe00640 1 --2- 172.21.15.16:0/2959847522 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e8ceb002f0 0x55e8ceed7f20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f08080045e0 tx=0x7f0808004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e80d2595cf48a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.926+0000 7f080ea00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:09:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.926+0000 7f080ea00640 1 -- 172.21.15.16:0/2959847522 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080801b040 con 0x55e8ceb002f0 2024-08-31T06:09:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.926+0000 7f080ea00640 1 -- 172.21.15.16:0/2959847522 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f0808005020 con 0x55e8ceb002f0 2024-08-31T06:09:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.926+0000 7f082e819740 1 -- 172.21.15.16:0/2959847522 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0818004c50 con 0x55e8ceb002f0 2024-08-31T06:09:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.927+0000 7f080ea00640 1 -- 172.21.15.16:0/2959847522 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0808027050 con 0x55e8ceb002f0 2024-08-31T06:09:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.927+0000 7f082e819740 10 client.26098 fetch_fsmap learned FSMap version 31 2024-08-31T06:09:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.927+0000 7f082e819740 1 -- 172.21.15.16:0/2959847522 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e8cec0d620 con 0x55e8ceb002f0 2024-08-31T06:09:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.927+0000 7f080ea00640 1 -- 172.21.15.16:0/2959847522 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0818004c50 con 0x55e8ceb002f0 2024-08-31T06:09:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.927+0000 7f080ea00640 1 -- 172.21.15.16:0/2959847522 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f0808012ce0 con 0x55e8ceb002f0 2024-08-31T06:09:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.927+0000 7f082e819740 10 client.26098 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:09:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.928+0000 7f082e819740 10 client.26098 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:09:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.928+0000 7f082e819740 20 client.26098 populate_metadata read hostname 'smithi016' 2024-08-31T06:09:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.928+0000 7f082e819740 1 --2- 172.21.15.16:0/2959847522 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55e8ceae7d80 0x55e8ceeea170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.928+0000 7f082e819740 4 client.26098 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:09:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.928+0000 7f082e819740 1 -- 172.21.15.16:0/2959847522 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8cee43ec0 con 0x55e8ceae7d80 2024-08-31T06:09:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.928+0000 7f0824e00640 1 --2- 172.21.15.16:0/2959847522 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55e8ceae7d80 0x55e8ceeea170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.929+0000 7f0824e00640 1 --2- 172.21.15.16:0/2959847522 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55e8ceae7d80 0x55e8ceeea170 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e72c522680efa8a server_cookie=b585b9cfe6c1fac in_seq=0 out_seq=0 2024-08-31T06:09:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.929+0000 7f080ea00640 10 client.26098 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:09:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.930+0000 7f080ea00640 1 -- 172.21.15.16:0/2959847522 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e8cee43ec0 con 0x55e8ceae7d80 2024-08-31T06:09:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.930+0000 7f080ea00640 10 client.26098 handle_command_reply: tid=0 2024-08-31T06:09:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.930+0000 7f082e819740 1 client.26098 shutdown 2024-08-31T06:09:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.930+0000 7f082e819740 1 -- 172.21.15.16:0/2959847522 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e8ceb002f0 msgr2=0x55e8ceed7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.930+0000 7f082e819740 1 --2- 172.21.15.16:0/2959847522 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e8ceb002f0 0x55e8ceed7f20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f08080045e0 tx=0x7f0808004730 comp rx=0 tx=0).stop 2024-08-31T06:09:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.930+0000 7f082e819740 1 -- 172.21.15.16:0/2959847522 shutdown_connections 2024-08-31T06:09:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.930+0000 7f082e819740 1 --2- 172.21.15.16:0/2959847522 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55e8ceae7d80 0x55e8ceeea170 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.930+0000 7f082e819740 1 --2- 172.21.15.16:0/2959847522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8ceb291b0 0x55e8ceed89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.930+0000 7f082e819740 1 --2- 172.21.15.16:0/2959847522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ceb084d0 0x55e8ceed8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.930+0000 7f080ea00640 0 client.26098 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:09:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.930+0000 7f082e819740 1 --2- 172.21.15.16:0/2959847522 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e8ceb002f0 0x55e8ceed7f20 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.930+0000 7f082e819740 1 -- 172.21.15.16:0/2959847522 >> 172.21.15.16:0/2959847522 conn(0x55e8ceb1d300 msgr2=0x55e8ceecd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.931+0000 7f082e819740 1 -- 172.21.15.16:0/2959847522 shutdown_connections 2024-08-31T06:09:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.931+0000 7f082e819740 1 -- 172.21.15.16:0/2959847522 wait complete. 2024-08-31T06:09:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.931+0000 7f082e819740 20 client.26098 trim_cache size 0 max 16384 2024-08-31T06:09:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.953+0000 7f082e819740 1 Processor -- start 2024-08-31T06:09:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.953+0000 7f082e819740 1 -- start start 2024-08-31T06:09:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.954+0000 7f082e819740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ceb002f0 0x55e8ceed8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.954+0000 7f082e819740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e8ceb084d0 0x55e8ceed9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.954+0000 7f082e819740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8ceb291b0 0x55e8ceece060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.954+0000 7f082e819740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f08201b8340 con 0x55e8ceb002f0 2024-08-31T06:09:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.954+0000 7f082e819740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f08201b5b50 con 0x55e8ceb291b0 2024-08-31T06:09:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.954+0000 7f082e819740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f08201b39a0 con 0x55e8ceb084d0 2024-08-31T06:09:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.954+0000 7f081fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ceb002f0 0x55e8ceed8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.954+0000 7f0824e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e8ceb084d0 0x55e8ceed9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.954+0000 7f081fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ceb002f0 0x55e8ceed8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33960/0 (socket says 172.21.15.16:33960) 2024-08-31T06:09:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.954+0000 7f081fe00640 1 -- 172.21.15.16:0/2571229748 learned_addr learned my addr 172.21.15.16:0/2571229748 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.955+0000 7f081fe00640 1 --2- 172.21.15.16:0/2571229748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8ceb291b0 0x55e8ceece060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.955+0000 7f0824e00640 1 -- 172.21.15.16:0/2571229748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8ceb291b0 msgr2=0x55e8ceece060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.955+0000 7f0824e00640 1 --2- 172.21.15.16:0/2571229748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8ceb291b0 0x55e8ceece060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.955+0000 7f0824e00640 1 -- 172.21.15.16:0/2571229748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ceb002f0 msgr2=0x55e8ceed8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.955+0000 7f0824e00640 1 --2- 172.21.15.16:0/2571229748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ceb002f0 0x55e8ceed8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.955+0000 7f0824e00640 1 -- 172.21.15.16:0/2571229748 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8ceed3a60 con 0x55e8ceb084d0 2024-08-31T06:09:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.956+0000 7f081fe00640 1 --2- 172.21.15.16:0/2571229748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ceb002f0 0x55e8ceed8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:09:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.956+0000 7f0824e00640 1 --2- 172.21.15.16:0/2571229748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e8ceb084d0 0x55e8ceed9e00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0818005d00 tx=0x7f0818010fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70b5de9d257e6c3d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.957+0000 7f0815600640 1 -- 172.21.15.16:0/2571229748 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0818004220 con 0x55e8ceb084d0 2024-08-31T06:09:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.957+0000 7f0815600640 1 -- 172.21.15.16:0/2571229748 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08180043c0 con 0x55e8ceb084d0 2024-08-31T06:09:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.957+0000 7f0815600640 1 -- 172.21.15.16:0/2571229748 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0818003850 con 0x55e8ceb084d0 2024-08-31T06:09:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.958+0000 7f082e819740 1 -- 172.21.15.16:0/2571229748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e8ceb084d0 msgr2=0x55e8ceed9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.958+0000 7f082e819740 1 --2- 172.21.15.16:0/2571229748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e8ceb084d0 0x55e8ceed9e00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0818005d00 tx=0x7f0818010fd0 comp rx=0 tx=0).stop 2024-08-31T06:09:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.958+0000 7f082e819740 1 -- 172.21.15.16:0/2571229748 shutdown_connections 2024-08-31T06:09:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.958+0000 7f082e819740 1 --2- 172.21.15.16:0/2571229748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e8ceb084d0 0x55e8ceed9e00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.958+0000 7f082e819740 1 --2- 172.21.15.16:0/2571229748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8ceb291b0 0x55e8ceece060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.958+0000 7f082e819740 1 --2- 172.21.15.16:0/2571229748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ceb002f0 0x55e8ceed8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.958+0000 7f082e819740 1 -- 172.21.15.16:0/2571229748 >> 172.21.15.16:0/2571229748 conn(0x55e8ceb1d300 msgr2=0x55e8ceecd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.959+0000 7f082e819740 1 -- 172.21.15.16:0/2571229748 shutdown_connections 2024-08-31T06:09:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.959+0000 7f082e819740 1 -- 172.21.15.16:0/2571229748 wait complete. 2024-08-31T06:09:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.959+0000 7f082e819740 1 -- start start 2024-08-31T06:09:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.960+0000 7f082e819740 1 Processor -- start 2024-08-31T06:09:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.960+0000 7f082e819740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ceb002f0 0x55e8ceedde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.960+0000 7f082e819740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8ceb084d0 0x55e8ceede3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.960+0000 7f082e819740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e8ceb291b0 0x55e8cef4bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.960+0000 7f082e819740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f08201b4010 con 0x55e8ceb002f0 2024-08-31T06:09:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.960+0000 7f082e819740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f08201b5480 con 0x55e8ceb084d0 2024-08-31T06:09:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.960+0000 7f082e819740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f08201b46e0 con 0x55e8ceb291b0 2024-08-31T06:09:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.960+0000 7f082e819740 10 client.0 _ll_register_callbacks cb 0x7f08080044a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:09:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.961+0000 7f0824e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8ceb084d0 0x55e8ceede3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.961+0000 7f0824e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8ceb084d0 0x55e8ceede3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:54502/0 (socket says 172.21.15.16:54502) 2024-08-31T06:09:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.961+0000 7f0824e00640 1 -- 172.21.15.16:0/1425464108 learned_addr learned my addr 172.21.15.16:0/1425464108 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.961+0000 7f081fe00640 1 --2- 172.21.15.16:0/1425464108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ceb002f0 0x55e8ceedde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.961+0000 7f081fe00640 1 --2- 172.21.15.16:0/1425464108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e8ceb291b0 0x55e8cef4bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.961+0000 7f0824e00640 1 -- 172.21.15.16:0/1425464108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e8ceb291b0 msgr2=0x55e8cef4bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.961+0000 7f0824e00640 1 --2- 172.21.15.16:0/1425464108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e8ceb291b0 0x55e8cef4bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.961+0000 7f0824e00640 1 -- 172.21.15.16:0/1425464108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ceb002f0 msgr2=0x55e8ceedde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.961+0000 7f0824e00640 1 --2- 172.21.15.16:0/1425464108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ceb002f0 0x55e8ceedde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.961+0000 7f0824e00640 1 -- 172.21.15.16:0/1425464108 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8cef4c1f0 con 0x55e8ceb084d0 2024-08-31T06:09:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.961+0000 7f0824e00640 1 --2- 172.21.15.16:0/1425464108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8ceb084d0 0x55e8ceede3b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0818010ca0 tx=0x7f08180056f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f77dd7daeff4b6b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.962+0000 7f080ea00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:09:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.962+0000 7f080ea00640 1 -- 172.21.15.16:0/1425464108 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0818036020 con 0x55e8ceb084d0 2024-08-31T06:09:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.962+0000 7f080ea00640 1 -- 172.21.15.16:0/1425464108 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f08180118d0 con 0x55e8ceb084d0 2024-08-31T06:09:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.962+0000 7f082e819740 1 -- 172.21.15.16:0/1425464108 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08180043c0 con 0x55e8ceb084d0 2024-08-31T06:09:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.962+0000 7f080ea00640 1 -- 172.21.15.16:0/1425464108 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0818011b10 con 0x55e8ceb084d0 2024-08-31T06:09:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.962+0000 7f082e819740 10 client.26102 fetch_fsmap learned FSMap version 31 2024-08-31T06:09:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.962+0000 7f082e819740 1 -- 172.21.15.16:0/1425464108 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e8cef54690 con 0x55e8ceb084d0 2024-08-31T06:09:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.963+0000 7f080ea00640 1 -- 172.21.15.16:0/1425464108 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08180043c0 con 0x55e8ceb084d0 2024-08-31T06:09:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.963+0000 7f080ea00640 1 -- 172.21.15.16:0/1425464108 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f081802c070 con 0x55e8ceb084d0 2024-08-31T06:09:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.963+0000 7f082e819740 10 client.26102 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:09:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.963+0000 7f082e819740 10 client.26102 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:09:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.963+0000 7f082e819740 20 client.26102 populate_metadata read hostname 'smithi016' 2024-08-31T06:09:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.963+0000 7f082e819740 1 --2- 172.21.15.16:0/1425464108 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55e8ceabd1c0 0x55e8cef56bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.963+0000 7f082e819740 4 client.26102 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:09:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.963+0000 7f082e819740 1 -- 172.21.15.16:0/1425464108 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e8cee3f960 con 0x55e8ceabd1c0 2024-08-31T06:09:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.963+0000 7f081fe00640 1 --2- 172.21.15.16:0/1425464108 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55e8ceabd1c0 0x55e8cef56bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.964+0000 7f081fe00640 1 --2- 172.21.15.16:0/1425464108 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55e8ceabd1c0 0x55e8cef56bb0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14c1737dfe05335d server_cookie=919c86d83041e357 in_seq=0 out_seq=0 2024-08-31T06:09:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.964+0000 7f080ea00640 10 client.26102 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:09:44.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:44 smithi184 ceph-mon[22331]: pgmap v439: 97 pgs: 97 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 8.6 MiB/s wr, 739 op/s 2024-08-31T06:09:44.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.980+0000 7f080ea00640 1 -- 172.21.15.16:0/1425464108 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e8cee3f960 con 0x55e8ceabd1c0 2024-08-31T06:09:44.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.980+0000 7f080ea00640 10 client.26102 handle_command_reply: tid=0 2024-08-31T06:09:44.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.980+0000 7f082e819740 1 client.26102 shutdown 2024-08-31T06:09:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.980+0000 7f082e819740 1 -- 172.21.15.16:0/1425464108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8ceb084d0 msgr2=0x55e8ceede3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.980+0000 7f082e819740 1 --2- 172.21.15.16:0/1425464108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8ceb084d0 0x55e8ceede3b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0818010ca0 tx=0x7f08180056f0 comp rx=0 tx=0).stop 2024-08-31T06:09:44.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.981+0000 7f082e819740 1 -- 172.21.15.16:0/1425464108 shutdown_connections 2024-08-31T06:09:44.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.981+0000 7f082e819740 1 --2- 172.21.15.16:0/1425464108 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55e8ceabd1c0 0x55e8cef56bb0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:44.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.981+0000 7f082e819740 1 --2- 172.21.15.16:0/1425464108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e8ceb291b0 0x55e8cef4bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:44.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.981+0000 7f082e819740 1 --2- 172.21.15.16:0/1425464108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8ceb084d0 0x55e8ceede3b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:44.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.981+0000 7f082e819740 1 --2- 172.21.15.16:0/1425464108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ceb002f0 0x55e8ceedde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:44.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.981+0000 7f080ea00640 0 client.26102 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:09:44.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.981+0000 7f082e819740 1 -- 172.21.15.16:0/1425464108 >> 172.21.15.16:0/1425464108 conn(0x55e8ceb1d300 msgr2=0x55e8ceed9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:44.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.981+0000 7f082e819740 1 -- 172.21.15.16:0/1425464108 shutdown_connections 2024-08-31T06:09:44.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.981+0000 7f082e819740 1 -- 172.21.15.16:0/1425464108 wait complete. 2024-08-31T06:09:44.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.981+0000 7f082e819740 20 client.26102 trim_cache size 0 max 16384 2024-08-31T06:09:44.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:09:44.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:09:44.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:09:44.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:09:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.982+0000 7f0827800640 1 -- 172.21.15.16:0/2861336112 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f08000b4d20 msgr2=0x7f08000b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.982+0000 7f0827800640 1 --2- 172.21.15.16:0/2861336112 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f08000b4d20 0x7f08000b7230 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f08100cd070 tx=0x7f081004d000 comp rx=0 tx=0).stop 2024-08-31T06:09:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.982+0000 7f0827800640 1 -- 172.21.15.16:0/2861336112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f082010fdd0 msgr2=0x7f08201072a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.982+0000 7f0827800640 1 --2- 172.21.15.16:0/2861336112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f082010fdd0 0x7f08201072a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f08100319f0 tx=0x7f0810031a20 comp rx=0 tx=0).stop 2024-08-31T06:09:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.982+0000 7f0827800640 1 -- 172.21.15.16:0/2861336112 shutdown_connections 2024-08-31T06:09:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.982+0000 7f0827800640 1 --2- 172.21.15.16:0/2861336112 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f08000b4d20 0x7f08000b7230 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.982+0000 7f0827800640 1 --2- 172.21.15.16:0/2861336112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0820075ea0 0x7f0820106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.982+0000 7f0827800640 1 --2- 172.21.15.16:0/2861336112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f082010fdd0 0x7f08201072a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.982+0000 7f0827800640 1 --2- 172.21.15.16:0/2861336112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0820075550 0x7f0820106820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.982+0000 7f0827800640 1 -- 172.21.15.16:0/2861336112 >> 172.21.15.16:0/2861336112 conn(0x7f08200ff2c0 msgr2=0x7f08201098e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.982+0000 7f0827800640 1 -- 172.21.15.16:0/2861336112 shutdown_connections 2024-08-31T06:09:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:44.982+0000 7f0827800640 1 -- 172.21.15.16:0/2861336112 wait complete. 2024-08-31T06:09:45.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:44 smithi155 ceph-mon[21742]: pgmap v439: 97 pgs: 97 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 8.6 MiB/s wr, 739 op/s 2024-08-31T06:09:46.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 140.0 s, 2943.2 tps, lat 34.001 ms stddev 37.376 2024-08-31T06:09:46.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:46 smithi016 ceph-mon[17194]: pgmap v440: 97 pgs: 97 active+clean; 1.5 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 9.1 MiB/s wr, 784 op/s 2024-08-31T06:09:46.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:46 smithi184 ceph-mon[22331]: pgmap v440: 97 pgs: 97 active+clean; 1.5 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 9.1 MiB/s wr, 784 op/s 2024-08-31T06:09:47.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:46 smithi155 ceph-mon[21742]: pgmap v440: 97 pgs: 97 active+clean; 1.5 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 9.1 MiB/s wr, 784 op/s 2024-08-31T06:09:48.595 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T06:09:48.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:48 smithi016 ceph-mon[17194]: pgmap v441: 97 pgs: 97 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.3 MiB/s wr, 749 op/s 2024-08-31T06:09:48.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:48 smithi184 ceph-mon[22331]: pgmap v441: 97 pgs: 97 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.3 MiB/s wr, 749 op/s 2024-08-31T06:09:49.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:48 smithi155 ceph-mon[21742]: pgmap v441: 97 pgs: 97 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.3 MiB/s wr, 749 op/s 2024-08-31T06:09:49.146 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:09:50.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.293+0000 7fb9ea600640 1 -- 172.21.15.16:0/4134143968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9e4077120 msgr2=0x7fb9e4075550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:50.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.293+0000 7fb9ea600640 1 --2- 172.21.15.16:0/4134143968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9e4077120 0x7fb9e4075550 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4009860 tx=0x7fb9d402a340 comp rx=0 tx=0).stop 2024-08-31T06:09:50.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.294+0000 7fb9ea600640 1 -- 172.21.15.16:0/4134143968 shutdown_connections 2024-08-31T06:09:50.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.294+0000 7fb9ea600640 1 --2- 172.21.15.16:0/4134143968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9e4075b70 0x7fb9e4075f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:50.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.294+0000 7fb9ea600640 1 --2- 172.21.15.16:0/4134143968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9e41141e0 0x7fb9e41166b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:50.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.294+0000 7fb9ea600640 1 --2- 172.21.15.16:0/4134143968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9e4077120 0x7fb9e4075550 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:50.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.294+0000 7fb9ea600640 1 -- 172.21.15.16:0/4134143968 >> 172.21.15.16:0/4134143968 conn(0x7fb9e40ff380 msgr2=0x7fb9e41017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:50.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.294+0000 7fb9ea600640 1 -- 172.21.15.16:0/4134143968 shutdown_connections 2024-08-31T06:09:50.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.294+0000 7fb9ea600640 1 -- 172.21.15.16:0/4134143968 wait complete. 2024-08-31T06:09:50.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.295+0000 7fb9ea600640 1 Processor -- start 2024-08-31T06:09:50.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.296+0000 7fb9ea600640 1 -- start start 2024-08-31T06:09:50.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.296+0000 7fb9ea600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9e4075b70 0x7fb9e41a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:50.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.296+0000 7fb9ea600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9e4077120 0x7fb9e41a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:50.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.296+0000 7fb9ea600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9e41141e0 0x7fb9e41a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:50.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.296+0000 7fb9ea600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e4119e20 con 0x7fb9e4075b70 2024-08-31T06:09:50.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.297+0000 7fb9ea600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e4119ca0 con 0x7fb9e4077120 2024-08-31T06:09:50.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.297+0000 7fb9ea600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e4119fa0 con 0x7fb9e41141e0 2024-08-31T06:09:50.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.297+0000 7fb9e2a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9e4077120 0x7fb9e41a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:50.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.297+0000 7fb9e3e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9e41141e0 0x7fb9e41a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:50.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.297+0000 7fb9e3e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9e41141e0 0x7fb9e41a6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:33362/0 (socket says 172.21.15.16:33362) 2024-08-31T06:09:50.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.297+0000 7fb9e3e00640 1 -- 172.21.15.16:0/1303980643 learned_addr learned my addr 172.21.15.16:0/1303980643 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:50.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.297+0000 7fb9e3e00640 1 -- 172.21.15.16:0/1303980643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9e4077120 msgr2=0x7fb9e41a0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:50.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.297+0000 7fb9e3400640 1 --2- 172.21.15.16:0/1303980643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9e4075b70 0x7fb9e41a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:50.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.298+0000 7fb9e3e00640 1 --2- 172.21.15.16:0/1303980643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9e4077120 0x7fb9e41a0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:50.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.298+0000 7fb9e3e00640 1 -- 172.21.15.16:0/1303980643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9e4075b70 msgr2=0x7fb9e41a0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:50.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.298+0000 7fb9e3e00640 1 --2- 172.21.15.16:0/1303980643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9e4075b70 0x7fb9e41a0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:50.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.298+0000 7fb9e3e00640 1 -- 172.21.15.16:0/1303980643 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9e41a6770 con 0x7fb9e41141e0 2024-08-31T06:09:50.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.298+0000 7fb9e3400640 1 --2- 172.21.15.16:0/1303980643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9e4075b70 0x7fb9e41a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:09:50.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.298+0000 7fb9e2a00640 1 --2- 172.21.15.16:0/1303980643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9e4077120 0x7fb9e41a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:09:50.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.299+0000 7fb9e3e00640 1 --2- 172.21.15.16:0/1303980643 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9e41141e0 0x7fb9e41a6070 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc00c370 tx=0x7fb9dc00c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:50.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.299+0000 7fb9dbe00640 1 -- 172.21.15.16:0/1303980643 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc003970 con 0x7fb9e41141e0 2024-08-31T06:09:50.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.299+0000 7fb9ea600640 1 -- 172.21.15.16:0/1303980643 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9e41a6a60 con 0x7fb9e41141e0 2024-08-31T06:09:50.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.299+0000 7fb9dbe00640 1 -- 172.21.15.16:0/1303980643 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9dc004020 con 0x7fb9e41141e0 2024-08-31T06:09:50.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.299+0000 7fb9dbe00640 1 -- 172.21.15.16:0/1303980643 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc00b960 con 0x7fb9e41141e0 2024-08-31T06:09:50.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.299+0000 7fb9ea600640 1 -- 172.21.15.16:0/1303980643 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9e41078d0 con 0x7fb9e41141e0 2024-08-31T06:09:50.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.307+0000 7fb9dbe00640 1 -- 172.21.15.16:0/1303980643 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fb9dc01d020 con 0x7fb9e41141e0 2024-08-31T06:09:50.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.307+0000 7fb9f11c1740 1 Processor -- start 2024-08-31T06:09:50.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.308+0000 7fb9f11c1740 1 -- start start 2024-08-31T06:09:50.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.308+0000 7fb9f11c1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f9e18f070 0x556f9e534950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:50.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.308+0000 7fb9f11c1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f9e1661b0 0x556f9e534e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:50.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.308+0000 7fb9f11c1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f9e16e390 0x556f9e5353d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:50.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.308+0000 7fb9f11c1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556f9e4ab770 con 0x556f9e1661b0 2024-08-31T06:09:50.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.308+0000 7fb9f11c1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556f9e49ad00 con 0x556f9e16e390 2024-08-31T06:09:50.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.308+0000 7fb9f11c1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556f9e49ecb0 con 0x556f9e18f070 2024-08-31T06:09:50.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.308+0000 7fb9e2a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f9e18f070 0x556f9e534950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:50.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.309+0000 7fb9e2a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f9e18f070 0x556f9e534950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:33378/0 (socket says 172.21.15.16:33378) 2024-08-31T06:09:50.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.309+0000 7fb9e2a00640 1 -- 172.21.15.16:0/706478631 learned_addr learned my addr 172.21.15.16:0/706478631 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:50.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.309+0000 7fb9e2a00640 1 --2- 172.21.15.16:0/706478631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f9e16e390 0x556f9e5353d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:50.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.309+0000 7fb9e3400640 1 --2- 172.21.15.16:0/706478631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f9e1661b0 0x556f9e534e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:50.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.309+0000 7fb9dbe00640 1 --2- 172.21.15.16:0/1303980643 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb9c80b4da0 0x7fb9c80b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:50.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.309+0000 7fb9e2a00640 1 -- 172.21.15.16:0/706478631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f9e16e390 msgr2=0x556f9e5353d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:50.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.310+0000 7fb9e2a00640 1 --2- 172.21.15.16:0/706478631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f9e16e390 0x556f9e5353d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:50.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.310+0000 7fb9dbe00640 1 -- 172.21.15.16:0/1303980643 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fb9dc0e5490 con 0x7fb9e41141e0 2024-08-31T06:09:50.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.310+0000 7fb9e2a00640 1 -- 172.21.15.16:0/706478631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f9e1661b0 msgr2=0x556f9e534e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:50.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.310+0000 7fb9e3e00640 1 --2- 172.21.15.16:0/1303980643 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb9c80b4da0 0x7fb9c80b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:50.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.310+0000 7fb9e2a00640 1 --2- 172.21.15.16:0/706478631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f9e1661b0 0x556f9e534e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:50.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.310+0000 7fb9e2a00640 1 -- 172.21.15.16:0/706478631 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f9e1fa650 con 0x556f9e18f070 2024-08-31T06:09:50.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.311+0000 7fb9e3400640 1 --2- 172.21.15.16:0/706478631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f9e1661b0 0x556f9e534e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:09:50.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.312+0000 7fb9e2a00640 1 --2- 172.21.15.16:0/706478631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f9e18f070 0x556f9e534950 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0014460 tx=0x7fb9d0009870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87ae011142da58a1 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:50.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.313+0000 7fb9d8c00640 1 -- 172.21.15.16:0/706478631 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d00140d0 con 0x556f9e18f070 2024-08-31T06:09:50.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.313+0000 7fb9d8c00640 1 -- 172.21.15.16:0/706478631 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9d0012910 con 0x556f9e18f070 2024-08-31T06:09:50.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.314+0000 7fb9d8c00640 1 -- 172.21.15.16:0/706478631 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d00140d0 con 0x556f9e18f070 2024-08-31T06:09:50.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.315+0000 7fb9f11c1740 1 -- 172.21.15.16:0/706478631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f9e18f070 msgr2=0x556f9e534950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:50.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.316+0000 7fb9f11c1740 1 --2- 172.21.15.16:0/706478631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f9e18f070 0x556f9e534950 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0014460 tx=0x7fb9d0009870 comp rx=0 tx=0).stop 2024-08-31T06:09:50.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.316+0000 7fb9e3e00640 1 --2- 172.21.15.16:0/1303980643 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb9c80b4da0 0x7fb9c80b72b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc004af0 tx=0x7fb9dc004a40 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:50.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.317+0000 7fb9f11c1740 1 -- 172.21.15.16:0/706478631 shutdown_connections 2024-08-31T06:09:50.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.317+0000 7fb9f11c1740 1 --2- 172.21.15.16:0/706478631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f9e16e390 0x556f9e5353d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:50.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.317+0000 7fb9f11c1740 1 --2- 172.21.15.16:0/706478631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f9e1661b0 0x556f9e534e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:50.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.317+0000 7fb9f11c1740 1 --2- 172.21.15.16:0/706478631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f9e18f070 0x556f9e534950 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:50.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.317+0000 7fb9f11c1740 1 -- 172.21.15.16:0/706478631 >> 172.21.15.16:0/706478631 conn(0x556f9e183460 msgr2=0x556f9e532840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:50.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.318+0000 7fb9f11c1740 1 -- 172.21.15.16:0/706478631 shutdown_connections 2024-08-31T06:09:50.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.318+0000 7fb9f11c1740 1 -- 172.21.15.16:0/706478631 wait complete. 2024-08-31T06:09:50.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.319+0000 7fb9f11c1740 1 -- start start 2024-08-31T06:09:50.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.321+0000 7fb9f11c1740 1 Processor -- start 2024-08-31T06:09:50.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.322+0000 7fb9f11c1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f9e1661b0 0x556f9e5404a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:50.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.322+0000 7fb9f11c1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f9e16e390 0x556f9e5409e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:50.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.322+0000 7fb9f11c1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f9e18f070 0x556f9e54c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:50.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.322+0000 7fb9f11c1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556f9e2fd700 con 0x556f9e1661b0 2024-08-31T06:09:50.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.322+0000 7fb9f11c1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556f9e421cb0 con 0x556f9e18f070 2024-08-31T06:09:50.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.322+0000 7fb9f11c1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556f9e5431d0 con 0x556f9e16e390 2024-08-31T06:09:50.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.323+0000 7fb9f11c1740 10 client.0 _ll_register_callbacks cb 0x556f9e447430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:09:50.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.324+0000 7fb9e3400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f9e16e390 0x556f9e5409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:50.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.324+0000 7fb9e3400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f9e16e390 0x556f9e5409e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:33382/0 (socket says 172.21.15.16:33382) 2024-08-31T06:09:50.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.324+0000 7fb9e3400640 1 -- 172.21.15.16:0/192093154 learned_addr learned my addr 172.21.15.16:0/192093154 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:50.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.324+0000 7fb9e2a00640 1 --2- 172.21.15.16:0/192093154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f9e1661b0 0x556f9e5404a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:50.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.326+0000 7fb9e2a00640 1 --2- 172.21.15.16:0/192093154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f9e18f070 0x556f9e54c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:50.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.326+0000 7fb9e3400640 1 -- 172.21.15.16:0/192093154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f9e18f070 msgr2=0x556f9e54c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:50.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.326+0000 7fb9e3400640 1 --2- 172.21.15.16:0/192093154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f9e18f070 0x556f9e54c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:50.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.326+0000 7fb9e3400640 1 -- 172.21.15.16:0/192093154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f9e1661b0 msgr2=0x556f9e5404a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:50.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.326+0000 7fb9e3400640 1 --2- 172.21.15.16:0/192093154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f9e1661b0 0x556f9e5404a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:50.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.327+0000 7fb9e3400640 1 -- 172.21.15.16:0/192093154 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f9e462870 con 0x556f9e16e390 2024-08-31T06:09:50.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.327+0000 7fb9e2a00640 1 --2- 172.21.15.16:0/192093154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f9e18f070 0x556f9e54c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:09:50.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.328+0000 7fb9e3400640 1 --2- 172.21.15.16:0/192093154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f9e16e390 0x556f9e5409e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4009830 tx=0x7fb9d4004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc86fb22318a601c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:50.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.328+0000 7fb9cd600640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:09:50.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.329+0000 7fb9cd600640 1 -- 172.21.15.16:0/192093154 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d4037040 con 0x556f9e16e390 2024-08-31T06:09:50.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.329+0000 7fb9cd600640 1 -- 172.21.15.16:0/192093154 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fb9d4030450 con 0x556f9e16e390 2024-08-31T06:09:50.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.329+0000 7fb9f11c1740 1 -- 172.21.15.16:0/192093154 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9d0012910 con 0x556f9e16e390 2024-08-31T06:09:50.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.329+0000 7fb9cd600640 1 -- 172.21.15.16:0/192093154 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb9d4005d00 con 0x556f9e16e390 2024-08-31T06:09:50.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.330+0000 7fb9f11c1740 10 client.26116 fetch_fsmap learned FSMap version 31 2024-08-31T06:09:50.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.330+0000 7fb9f11c1740 1 -- 172.21.15.16:0/192093154 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556f9e54f1e0 con 0x556f9e16e390 2024-08-31T06:09:50.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.330+0000 7fb9cd600640 1 -- 172.21.15.16:0/192093154 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9d0012910 con 0x556f9e16e390 2024-08-31T06:09:50.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.330+0000 7fb9cd600640 1 -- 172.21.15.16:0/192093154 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fb9d40048c0 con 0x556f9e16e390 2024-08-31T06:09:50.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.331+0000 7fb9f11c1740 10 client.26116 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:09:50.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.331+0000 7fb9f11c1740 10 client.26116 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:09:50.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.331+0000 7fb9f11c1740 20 client.26116 populate_metadata read hostname 'smithi016' 2024-08-31T06:09:50.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.331+0000 7fb9f11c1740 1 --2- 172.21.15.16:0/192093154 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556f9e14d9f0 0x556f9e552bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:50.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.331+0000 7fb9f11c1740 4 client.26116 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:09:50.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.331+0000 7fb9f11c1740 1 -- 172.21.15.16:0/192093154 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f9e4ad6c0 con 0x556f9e14d9f0 2024-08-31T06:09:50.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.331+0000 7fb9e2a00640 1 --2- 172.21.15.16:0/192093154 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556f9e14d9f0 0x556f9e552bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:50.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.333+0000 7fb9e2a00640 1 --2- 172.21.15.16:0/192093154 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556f9e14d9f0 0x556f9e552bb0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3320fb75a8368835 server_cookie=7d38a19b2d131224 in_seq=0 out_seq=0 2024-08-31T06:09:50.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.333+0000 7fb9cd600640 10 client.26116 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:09:50.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.335+0000 7fb9cd600640 1 -- 172.21.15.16:0/192093154 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556f9e4ad6c0 con 0x556f9e14d9f0 2024-08-31T06:09:50.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.335+0000 7fb9cd600640 10 client.26116 handle_command_reply: tid=0 2024-08-31T06:09:50.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.335+0000 7fb9f11c1740 1 client.26116 shutdown 2024-08-31T06:09:50.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.337+0000 7fb9f11c1740 1 -- 172.21.15.16:0/192093154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f9e16e390 msgr2=0x556f9e5409e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:50.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.337+0000 7fb9f11c1740 1 --2- 172.21.15.16:0/192093154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f9e16e390 0x556f9e5409e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4009830 tx=0x7fb9d4004270 comp rx=0 tx=0).stop 2024-08-31T06:09:50.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.337+0000 7fb9f11c1740 1 -- 172.21.15.16:0/192093154 shutdown_connections 2024-08-31T06:09:50.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.337+0000 7fb9f11c1740 1 --2- 172.21.15.16:0/192093154 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556f9e14d9f0 0x556f9e552bb0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:50.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.337+0000 7fb9f11c1740 1 --2- 172.21.15.16:0/192093154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f9e16e390 0x556f9e5409e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:50.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.337+0000 7fb9f11c1740 1 --2- 172.21.15.16:0/192093154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f9e18f070 0x556f9e54c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:50.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.337+0000 7fb9f11c1740 1 --2- 172.21.15.16:0/192093154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f9e1661b0 0x556f9e5404a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:50.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.337+0000 7fb9f11c1740 1 -- 172.21.15.16:0/192093154 >> 172.21.15.16:0/192093154 conn(0x556f9e183460 msgr2=0x556f9e535980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:50.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.337+0000 7fb9cd600640 0 client.26116 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:09:50.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.337+0000 7fb9f11c1740 1 -- 172.21.15.16:0/192093154 shutdown_connections 2024-08-31T06:09:50.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.338+0000 7fb9f11c1740 1 -- 172.21.15.16:0/192093154 wait complete. 2024-08-31T06:09:50.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.338+0000 7fb9f11c1740 20 client.26116 trim_cache size 0 max 16384 2024-08-31T06:09:50.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.378+0000 7fb9f11c1740 1 Processor -- start 2024-08-31T06:09:50.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.379+0000 7fb9f11c1740 1 -- start start 2024-08-31T06:09:50.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.380+0000 7fb9f11c1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f9e1661b0 0x556f9e536340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:50.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.380+0000 7fb9f11c1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f9e16e390 0x556f9e532390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:50.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.380+0000 7fb9f11c1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f9e18f070 0x556f9e54c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:50.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.380+0000 7fb9f11c1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e41c0b60 con 0x556f9e18f070 2024-08-31T06:09:50.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.380+0000 7fb9f11c1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e41be370 con 0x556f9e16e390 2024-08-31T06:09:50.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.380+0000 7fb9f11c1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e410e0a0 con 0x556f9e1661b0 2024-08-31T06:09:50.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.380+0000 7fb9e2a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f9e1661b0 0x556f9e536340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:50.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.380+0000 7fb9e2a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f9e1661b0 0x556f9e536340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:33390/0 (socket says 172.21.15.16:33390) 2024-08-31T06:09:50.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.380+0000 7fb9e2a00640 1 -- 172.21.15.16:0/1640730803 learned_addr learned my addr 172.21.15.16:0/1640730803 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:50.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.380+0000 7fb9e3400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f9e16e390 0x556f9e532390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:50.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.381+0000 7fb9e2a00640 1 --2- 172.21.15.16:0/1640730803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f9e18f070 0x556f9e54c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:50.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.381+0000 7fb9e2a00640 1 -- 172.21.15.16:0/1640730803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f9e16e390 msgr2=0x556f9e532390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:50.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.381+0000 7fb9e2a00640 1 --2- 172.21.15.16:0/1640730803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f9e16e390 0x556f9e532390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:50.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.381+0000 7fb9e2a00640 1 -- 172.21.15.16:0/1640730803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f9e18f070 msgr2=0x556f9e54c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:50.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.381+0000 7fb9e2a00640 1 --2- 172.21.15.16:0/1640730803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f9e18f070 0x556f9e54c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:50.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.381+0000 7fb9e2a00640 1 -- 172.21.15.16:0/1640730803 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f9e536c40 con 0x556f9e1661b0 2024-08-31T06:09:50.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.381+0000 7fb9e2a00640 1 --2- 172.21.15.16:0/1640730803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f9e1661b0 0x556f9e536340 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0005e00 tx=0x7fb9d0009750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8646d0020ceb7bdb server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:50.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.382+0000 7fb9cfe00640 1 -- 172.21.15.16:0/1640730803 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d00050c0 con 0x556f9e1661b0 2024-08-31T06:09:50.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.382+0000 7fb9cfe00640 1 -- 172.21.15.16:0/1640730803 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9d0029040 con 0x556f9e1661b0 2024-08-31T06:09:50.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.382+0000 7fb9cfe00640 1 -- 172.21.15.16:0/1640730803 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0003b20 con 0x556f9e1661b0 2024-08-31T06:09:50.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.383+0000 7fb9f11c1740 1 -- 172.21.15.16:0/1640730803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f9e1661b0 msgr2=0x556f9e536340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:50.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.383+0000 7fb9f11c1740 1 --2- 172.21.15.16:0/1640730803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f9e1661b0 0x556f9e536340 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0005e00 tx=0x7fb9d0009750 comp rx=0 tx=0).stop 2024-08-31T06:09:50.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.384+0000 7fb9f11c1740 1 -- 172.21.15.16:0/1640730803 shutdown_connections 2024-08-31T06:09:50.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.384+0000 7fb9f11c1740 1 --2- 172.21.15.16:0/1640730803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f9e18f070 0x556f9e54c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:50.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.384+0000 7fb9f11c1740 1 --2- 172.21.15.16:0/1640730803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f9e16e390 0x556f9e532390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:50.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.384+0000 7fb9f11c1740 1 --2- 172.21.15.16:0/1640730803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f9e1661b0 0x556f9e536340 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:50.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.384+0000 7fb9f11c1740 1 -- 172.21.15.16:0/1640730803 >> 172.21.15.16:0/1640730803 conn(0x556f9e183460 msgr2=0x556f9e53b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:50.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.384+0000 7fb9f11c1740 1 -- 172.21.15.16:0/1640730803 shutdown_connections 2024-08-31T06:09:50.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.384+0000 7fb9f11c1740 1 -- 172.21.15.16:0/1640730803 wait complete. 2024-08-31T06:09:50.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.385+0000 7fb9f11c1740 1 -- start start 2024-08-31T06:09:50.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.385+0000 7fb9f11c1740 1 Processor -- start 2024-08-31T06:09:50.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.386+0000 7fb9f11c1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f9e1661b0 0x556f9e544dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:50.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.386+0000 7fb9f11c1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f9e16e390 0x556f9e5453a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:50.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.386+0000 7fb9f11c1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f9e18f070 0x556f9e5b65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:50.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.386+0000 7fb9f11c1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e410e710 con 0x556f9e18f070 2024-08-31T06:09:50.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.386+0000 7fb9f11c1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e410fb80 con 0x556f9e16e390 2024-08-31T06:09:50.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.386+0000 7fb9f11c1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e410ede0 con 0x556f9e1661b0 2024-08-31T06:09:50.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.386+0000 7fb9f11c1740 10 client.0 _ll_register_callbacks cb 0x7fb9d0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:09:50.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.387+0000 7fb9e2a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f9e1661b0 0x556f9e544dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:50.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.387+0000 7fb9e2a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f9e1661b0 0x556f9e544dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:33392/0 (socket says 172.21.15.16:33392) 2024-08-31T06:09:50.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.387+0000 7fb9e2a00640 1 -- 172.21.15.16:0/4131087824 learned_addr learned my addr 172.21.15.16:0/4131087824 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:50.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.388+0000 7fb9e3400640 1 --2- 172.21.15.16:0/4131087824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f9e16e390 0x556f9e5453a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:50.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.388+0000 7fb9e3400640 1 -- 172.21.15.16:0/4131087824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f9e1661b0 msgr2=0x556f9e544dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:50.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.389+0000 7fb9e3400640 1 --2- 172.21.15.16:0/4131087824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f9e1661b0 0x556f9e544dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:50.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.389+0000 7fb9e3400640 1 -- 172.21.15.16:0/4131087824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f9e18f070 msgr2=0x556f9e5b65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:50.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.389+0000 7fb9e3400640 1 --2- 172.21.15.16:0/4131087824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f9e18f070 0x556f9e5b65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:50.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.389+0000 7fb9e3400640 1 -- 172.21.15.16:0/4131087824 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f9e5b6c60 con 0x556f9e16e390 2024-08-31T06:09:50.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.389+0000 7fb9e2a00640 1 --2- 172.21.15.16:0/4131087824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f9e1661b0 0x556f9e544dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:09:50.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.390+0000 7fb9e3400640 1 --2- 172.21.15.16:0/4131087824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f9e16e390 0x556f9e5453a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4031420 tx=0x7fb9d402a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b7d39e446d369df server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:50.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.390+0000 7fb9cd600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:09:50.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.390+0000 7fb9cd600640 1 -- 172.21.15.16:0/4131087824 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d4053070 con 0x556f9e16e390 2024-08-31T06:09:50.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.390+0000 7fb9cd600640 1 -- 172.21.15.16:0/4131087824 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fb9d403cc90 con 0x556f9e16e390 2024-08-31T06:09:50.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.391+0000 7fb9f11c1740 1 -- 172.21.15.16:0/4131087824 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9d0029040 con 0x556f9e16e390 2024-08-31T06:09:50.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.391+0000 7fb9cd600640 1 -- 172.21.15.16:0/4131087824 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb9d405c050 con 0x556f9e16e390 2024-08-31T06:09:50.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.392+0000 7fb9f11c1740 10 client.26117 fetch_fsmap learned FSMap version 31 2024-08-31T06:09:50.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.392+0000 7fb9f11c1740 1 -- 172.21.15.16:0/4131087824 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556f9e5bdfb0 con 0x556f9e16e390 2024-08-31T06:09:50.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.392+0000 7fb9cd600640 1 -- 172.21.15.16:0/4131087824 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9d0029040 con 0x556f9e16e390 2024-08-31T06:09:50.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.392+0000 7fb9cd600640 1 -- 172.21.15.16:0/4131087824 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fb9d40048c0 con 0x556f9e16e390 2024-08-31T06:09:50.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.392+0000 7fb9f11c1740 10 client.26117 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:09:50.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.392+0000 7fb9f11c1740 10 client.26117 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:09:50.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.392+0000 7fb9f11c1740 20 client.26117 populate_metadata read hostname 'smithi016' 2024-08-31T06:09:50.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.392+0000 7fb9f11c1740 1 --2- 172.21.15.16:0/4131087824 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556f9e122da0 0x556f9e5c0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:50.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.393+0000 7fb9f11c1740 4 client.26117 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T06:09:50.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.393+0000 7fb9f11c1740 1 -- 172.21.15.16:0/4131087824 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556f9e4a9160 con 0x556f9e122da0 2024-08-31T06:09:50.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.393+0000 7fb9e2a00640 1 --2- 172.21.15.16:0/4131087824 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556f9e122da0 0x556f9e5c0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:50.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.394+0000 7fb9e2a00640 1 --2- 172.21.15.16:0/4131087824 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556f9e122da0 0x556f9e5c0850 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bfee19ce522e987 server_cookie=1827503419682575 in_seq=0 out_seq=0 2024-08-31T06:09:50.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.394+0000 7fb9cd600640 10 client.26117 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:09:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.394+0000 7fb9cd600640 1 -- 172.21.15.16:0/4131087824 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1226 (crc 0 0 0) 0x556f9e4a9160 con 0x556f9e122da0 2024-08-31T06:09:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.394+0000 7fb9cd600640 10 client.26117 handle_command_reply: tid=0 2024-08-31T06:09:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.394+0000 7fb9f11c1740 1 client.26117 shutdown 2024-08-31T06:09:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.394+0000 7fb9f11c1740 1 -- 172.21.15.16:0/4131087824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f9e16e390 msgr2=0x556f9e5453a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.394+0000 7fb9f11c1740 1 --2- 172.21.15.16:0/4131087824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f9e16e390 0x556f9e5453a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4031420 tx=0x7fb9d402a340 comp rx=0 tx=0).stop 2024-08-31T06:09:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.395+0000 7fb9f11c1740 1 -- 172.21.15.16:0/4131087824 shutdown_connections 2024-08-31T06:09:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.395+0000 7fb9f11c1740 1 --2- 172.21.15.16:0/4131087824 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556f9e122da0 0x556f9e5c0850 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.395+0000 7fb9f11c1740 1 --2- 172.21.15.16:0/4131087824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f9e18f070 0x556f9e5b65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.395+0000 7fb9f11c1740 1 --2- 172.21.15.16:0/4131087824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f9e16e390 0x556f9e5453a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:50.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.395+0000 7fb9f11c1740 1 --2- 172.21.15.16:0/4131087824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f9e1661b0 0x556f9e544dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:50.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.395+0000 7fb9f11c1740 1 -- 172.21.15.16:0/4131087824 >> 172.21.15.16:0/4131087824 conn(0x556f9e183460 msgr2=0x556f9e532390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:50.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.395+0000 7fb9cd600640 0 client.26117 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:09:50.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.395+0000 7fb9f11c1740 1 -- 172.21.15.16:0/4131087824 shutdown_connections 2024-08-31T06:09:50.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.395+0000 7fb9f11c1740 1 -- 172.21.15.16:0/4131087824 wait complete. 2024-08-31T06:09:50.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.396+0000 7fb9f11c1740 20 client.26117 trim_cache size 0 max 16384 2024-08-31T06:09:50.399 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:09:50.399 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-08-31T06:09:50.399 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {}, 2024-08-31T06:09:50.399 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-08-31T06:09:50.399 INFO:teuthology.orchestra.run.smithi016.stdout: "78d6ef02-8551-45e8-ad60-9627eb573fc1": { 2024-08-31T06:09:50.399 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:09:13.547865+0000", 2024-08-31T06:09:50.400 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:09:50.400 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:09:50.400 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:09:50.400 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:09:50.400 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:09:50.400 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:09:50.400 INFO:teuthology.orchestra.run.smithi016.stdout: "c8455d74-2fe7-4d93-bcf2-19b419d16016": { 2024-08-31T06:09:50.400 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:08:24.035960+0000", 2024-08-31T06:09:50.400 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:09:50.400 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:09:50.400 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:09:50.400 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:09:50.400 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:09:50.400 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:09:50.401 INFO:teuthology.orchestra.run.smithi016.stdout: "88d7bdb8-cb1c-45e5-ad29-4cd8da6c6003": { 2024-08-31T06:09:50.401 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:07:39.648942+0000", 2024-08-31T06:09:50.401 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:09:50.401 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:09:50.401 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:09:50.401 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:09:50.401 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:09:50.401 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:09:50.401 INFO:teuthology.orchestra.run.smithi016.stdout: "c300b7e8-f3fd-482c-a909-9329bc61e570": { 2024-08-31T06:09:50.401 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:06:49.868046+0000", 2024-08-31T06:09:50.401 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:09:50.401 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:09:50.401 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:09:50.401 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:09:50.402 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:09:50.402 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:09:50.402 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:09:50.402 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:09:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.397+0000 7fb9ea600640 1 -- 172.21.15.16:0/1303980643 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb9c80b4da0 msgr2=0x7fb9c80b72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.397+0000 7fb9ea600640 1 --2- 172.21.15.16:0/1303980643 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb9c80b4da0 0x7fb9c80b72b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc004af0 tx=0x7fb9dc004a40 comp rx=0 tx=0).stop 2024-08-31T06:09:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.397+0000 7fb9ea600640 1 -- 172.21.15.16:0/1303980643 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9e41141e0 msgr2=0x7fb9e41a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.397+0000 7fb9ea600640 1 --2- 172.21.15.16:0/1303980643 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9e41141e0 0x7fb9e41a6070 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc00c370 tx=0x7fb9dc00c830 comp rx=0 tx=0).stop 2024-08-31T06:09:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.397+0000 7fb9ea600640 1 -- 172.21.15.16:0/1303980643 shutdown_connections 2024-08-31T06:09:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.397+0000 7fb9ea600640 1 --2- 172.21.15.16:0/1303980643 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb9c80b4da0 0x7fb9c80b72b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.397+0000 7fb9ea600640 1 --2- 172.21.15.16:0/1303980643 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9e41141e0 0x7fb9e41a6070 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.397+0000 7fb9ea600640 1 --2- 172.21.15.16:0/1303980643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9e4077120 0x7fb9e41a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.397+0000 7fb9ea600640 1 --2- 172.21.15.16:0/1303980643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9e4075b70 0x7fb9e41a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.397+0000 7fb9ea600640 1 -- 172.21.15.16:0/1303980643 >> 172.21.15.16:0/1303980643 conn(0x7fb9e40ff380 msgr2=0x7fb9e41100e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.398+0000 7fb9ea600640 1 -- 172.21.15.16:0/1303980643 shutdown_connections 2024-08-31T06:09:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:50.398+0000 7fb9ea600640 1 -- 172.21.15.16:0/1303980643 wait complete. 2024-08-31T06:09:50.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:50 smithi016 ceph-mon[17194]: pgmap v442: 97 pgs: 97 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.6 MiB/s wr, 749 op/s 2024-08-31T06:09:50.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:50 smithi184 ceph-mon[22331]: pgmap v442: 97 pgs: 97 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.6 MiB/s wr, 749 op/s 2024-08-31T06:09:51.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:50 smithi155 ceph-mon[21742]: pgmap v442: 97 pgs: 97 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.6 MiB/s wr, 749 op/s 2024-08-31T06:09:51.393 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T06:09:51.393 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:09:51.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 145.0 s, 2490.6 tps, lat 39.926 ms stddev 48.700 2024-08-31T06:09:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:09:52.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:52.854+0000 7fc2cb400640 1 -- 172.21.15.16:0/170015206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2c41050a0 msgr2=0x7fc2c410d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:52.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:52.854+0000 7fc2cb400640 1 --2- 172.21.15.16:0/170015206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2c41050a0 0x7fc2c410d5f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0009900 tx=0x7fc2b002a3f0 comp rx=0 tx=0).stop 2024-08-31T06:09:52.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:52.855+0000 7fc2cb400640 1 -- 172.21.15.16:0/170015206 shutdown_connections 2024-08-31T06:09:52.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:52.855+0000 7fc2cb400640 1 --2- 172.21.15.16:0/170015206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2c410dbb0 0x7fc2c41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:52.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:52.855+0000 7fc2cb400640 1 --2- 172.21.15.16:0/170015206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2c41050a0 0x7fc2c410d5f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:52.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:52.855+0000 7fc2cb400640 1 --2- 172.21.15.16:0/170015206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2c4104750 0x7fc2c4104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:52.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:52.855+0000 7fc2cb400640 1 -- 172.21.15.16:0/170015206 >> 172.21.15.16:0/170015206 conn(0x7fc2c40794d0 msgr2=0x7fc2c40798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:52.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:52.855+0000 7fc2cb400640 1 -- 172.21.15.16:0/170015206 shutdown_connections 2024-08-31T06:09:52.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:52 smithi016 ceph-mon[17194]: pgmap v443: 97 pgs: 97 active+clean; 1.6 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.1 MiB/s wr, 693 op/s 2024-08-31T06:09:52.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:52.855+0000 7fc2cb400640 1 -- 172.21.15.16:0/170015206 wait complete. 2024-08-31T06:09:52.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:52.857+0000 7fc2cb400640 1 Processor -- start 2024-08-31T06:09:52.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:52.857+0000 7fc2cb400640 1 -- start start 2024-08-31T06:09:52.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:52.858+0000 7fc2cb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2c4104750 0x7fc2c4102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:52.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:52.858+0000 7fc2cb400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2c41050a0 0x7fc2c41029d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:52.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:52.858+0000 7fc2cb400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2c410dbb0 0x7fc2c4102f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:52.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:52.858+0000 7fc2cb400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c41138e0 con 0x7fc2c4104750 2024-08-31T06:09:52.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:52.858+0000 7fc2cb400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c4113760 con 0x7fc2c410dbb0 2024-08-31T06:09:52.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:52.858+0000 7fc2cb400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c4113a60 con 0x7fc2c41050a0 2024-08-31T06:09:52.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:52.859+0000 7fc2c9400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2c410dbb0 0x7fc2c4102f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:52.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:52.859+0000 7fc2c8a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2c4104750 0x7fc2c4102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:52.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:52.859+0000 7fc2c9400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2c410dbb0 0x7fc2c4102f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:53354/0 (socket says 172.21.15.16:53354) 2024-08-31T06:09:52.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:52.859+0000 7fc2c9400640 1 -- 172.21.15.16:0/1615078708 learned_addr learned my addr 172.21.15.16:0/1615078708 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:52.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:52.859+0000 7fc2c3e00640 1 --2- 172.21.15.16:0/1615078708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2c41050a0 0x7fc2c41029d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:52.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:52.859+0000 7fc2c9400640 1 -- 172.21.15.16:0/1615078708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2c41050a0 msgr2=0x7fc2c41029d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:52.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:52.859+0000 7fc2c9400640 1 --2- 172.21.15.16:0/1615078708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2c41050a0 0x7fc2c41029d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:52.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:52.859+0000 7fc2c9400640 1 -- 172.21.15.16:0/1615078708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2c4104750 msgr2=0x7fc2c4102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:52.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:52.859+0000 7fc2c9400640 1 --2- 172.21.15.16:0/1615078708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2c4104750 0x7fc2c4102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:52.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:52.859+0000 7fc2c9400640 1 -- 172.21.15.16:0/1615078708 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2c41a85d0 con 0x7fc2c410dbb0 2024-08-31T06:09:52.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:52.860+0000 7fc2c8a00640 1 --2- 172.21.15.16:0/1615078708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2c4104750 0x7fc2c4102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:09:52.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:52.860+0000 7fc2c9400640 1 --2- 172.21.15.16:0/1615078708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2c410dbb0 0x7fc2c4102f10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00eb10 tx=0x7fc2bc00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:52.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:52.860+0000 7fc2c1600640 1 -- 172.21.15.16:0/1615078708 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc004b20 con 0x7fc2c410dbb0 2024-08-31T06:09:52.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:52.860+0000 7fc2c1600640 1 -- 172.21.15.16:0/1615078708 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2bc004cc0 con 0x7fc2c410dbb0 2024-08-31T06:09:52.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:52.860+0000 7fc2cb400640 1 -- 172.21.15.16:0/1615078708 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2c419a080 con 0x7fc2c410dbb0 2024-08-31T06:09:52.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:52.861+0000 7fc2cb400640 1 -- 172.21.15.16:0/1615078708 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2c419a610 con 0x7fc2c410dbb0 2024-08-31T06:09:52.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:52.861+0000 7fc2c1600640 1 -- 172.21.15.16:0/1615078708 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc010720 con 0x7fc2c410dbb0 2024-08-31T06:09:52.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:52.863+0000 7fc2cb400640 1 -- 172.21.15.16:0/1615078708 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2a4005290 con 0x7fc2c410dbb0 2024-08-31T06:09:52.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:52.868+0000 7fc2c1600640 1 -- 172.21.15.16:0/1615078708 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fc2bc0108c0 con 0x7fc2c410dbb0 2024-08-31T06:09:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:52.871+0000 7fc2c1600640 1 --2- 172.21.15.16:0/1615078708 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc2ac0b4df0 0x7fc2ac0b7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:52.871+0000 7fc2c1600640 1 -- 172.21.15.16:0/1615078708 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fc2bc0e5840 con 0x7fc2c410dbb0 2024-08-31T06:09:52.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:52.871+0000 7fc2c1600640 1 -- 172.21.15.16:0/1615078708 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fc2bc1161b0 con 0x7fc2c410dbb0 2024-08-31T06:09:52.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:52.871+0000 7fc2c8a00640 1 --2- 172.21.15.16:0/1615078708 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc2ac0b4df0 0x7fc2ac0b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:52.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:52.874+0000 7fc2c8a00640 1 --2- 172.21.15.16:0/1615078708 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc2ac0b4df0 0x7fc2ac0b7300 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4103ee0 tx=0x7fc2b8008040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:52.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:52 smithi184 ceph-mon[22331]: pgmap v443: 97 pgs: 97 active+clean; 1.6 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.1 MiB/s wr, 693 op/s 2024-08-31T06:09:53.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:52 smithi155 ceph-mon[21742]: pgmap v443: 97 pgs: 97 active+clean; 1.6 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.1 MiB/s wr, 693 op/s 2024-08-31T06:09:53.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:53.269+0000 7fc2cb400640 1 -- 172.21.15.16:0/1615078708 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc2a4005580 con 0x7fc2c410dbb0 2024-08-31T06:09:53.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:53.270+0000 7fc2c1600640 1 -- 172.21.15.16:0/1615078708 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11923 (secure 0 0 0) 0x7fc2bc0ac710 con 0x7fc2c410dbb0 2024-08-31T06:09:53.273 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:09:53.273 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 31 2024-08-31T06:09:53.273 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":31,"btime":"2024-08-31T06:08:47:452652+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:08:47.004817+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24737,24724,24712]},"id":1}]} 2024-08-31T06:09:53.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:53.276+0000 7fc2cb400640 1 -- 172.21.15.16:0/1615078708 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc2ac0b4df0 msgr2=0x7fc2ac0b7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:53.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:53.276+0000 7fc2cb400640 1 --2- 172.21.15.16:0/1615078708 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc2ac0b4df0 0x7fc2ac0b7300 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4103ee0 tx=0x7fc2b8008040 comp rx=0 tx=0).stop 2024-08-31T06:09:53.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:53.276+0000 7fc2cb400640 1 -- 172.21.15.16:0/1615078708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2c410dbb0 msgr2=0x7fc2c4102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:53.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:53.276+0000 7fc2cb400640 1 --2- 172.21.15.16:0/1615078708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2c410dbb0 0x7fc2c4102f10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00eb10 tx=0x7fc2bc00c2d0 comp rx=0 tx=0).stop 2024-08-31T06:09:53.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:53.278+0000 7fc2cb400640 1 -- 172.21.15.16:0/1615078708 shutdown_connections 2024-08-31T06:09:53.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:53.278+0000 7fc2cb400640 1 --2- 172.21.15.16:0/1615078708 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc2ac0b4df0 0x7fc2ac0b7300 secure :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4103ee0 tx=0x7fc2b8008040 comp rx=0 tx=0).stop 2024-08-31T06:09:53.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:53.278+0000 7fc2cb400640 1 --2- 172.21.15.16:0/1615078708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2c41050a0 0x7fc2c41029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:53.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:53.278+0000 7fc2cb400640 1 --2- 172.21.15.16:0/1615078708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2c410dbb0 0x7fc2c4102f10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:53.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:53.278+0000 7fc2cb400640 1 --2- 172.21.15.16:0/1615078708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2c4104750 0x7fc2c4102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:53.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:53.278+0000 7fc2cb400640 1 -- 172.21.15.16:0/1615078708 >> 172.21.15.16:0/1615078708 conn(0x7fc2c40794d0 msgr2=0x7fc2c40ff2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:53.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:53.279+0000 7fc2cb400640 1 -- 172.21.15.16:0/1615078708 shutdown_connections 2024-08-31T06:09:53.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:53.279+0000 7fc2cb400640 1 -- 172.21.15.16:0/1615078708 wait complete. 2024-08-31T06:09:53.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:53 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/1615078708' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:09:54.121 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:53 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/1615078708' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:09:54.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:53 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/1615078708' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:09:54.420 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T06:09:54.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:54 smithi016 ceph-mon[17194]: pgmap v444: 97 pgs: 97 active+clean; 1.6 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.1 MiB/s wr, 726 op/s 2024-08-31T06:09:54.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:54 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:09:54.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:54 smithi184 ceph-mon[22331]: pgmap v444: 97 pgs: 97 active+clean; 1.6 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.1 MiB/s wr, 726 op/s 2024-08-31T06:09:54.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:54 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:09:55.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:54 smithi155 ceph-mon[21742]: pgmap v444: 97 pgs: 97 active+clean; 1.6 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.1 MiB/s wr, 726 op/s 2024-08-31T06:09:55.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:54 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:09:55.126 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:09:55.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:55 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:09:55.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:55 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:09:55.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:55 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:09:55.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.946+0000 7fddef600640 1 -- 172.21.15.16:0/459599441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdde81050a0 msgr2=0x7fdde810d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:55.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.946+0000 7fddef600640 1 --2- 172.21.15.16:0/459599441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdde81050a0 0x7fdde810d5f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fddd4009900 tx=0x7fddd402a3f0 comp rx=0 tx=0).stop 2024-08-31T06:09:55.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.947+0000 7fddef600640 1 -- 172.21.15.16:0/459599441 shutdown_connections 2024-08-31T06:09:55.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.947+0000 7fddef600640 1 --2- 172.21.15.16:0/459599441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdde810dbb0 0x7fdde81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:55.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.947+0000 7fddef600640 1 --2- 172.21.15.16:0/459599441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdde81050a0 0x7fdde810d5f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:55.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.947+0000 7fddef600640 1 --2- 172.21.15.16:0/459599441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdde8104750 0x7fdde8104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:55.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.947+0000 7fddef600640 1 -- 172.21.15.16:0/459599441 >> 172.21.15.16:0/459599441 conn(0x7fdde80794d0 msgr2=0x7fdde80798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:55.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.947+0000 7fddef600640 1 -- 172.21.15.16:0/459599441 shutdown_connections 2024-08-31T06:09:55.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.947+0000 7fddef600640 1 -- 172.21.15.16:0/459599441 wait complete. 2024-08-31T06:09:55.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.951+0000 7fddef600640 1 Processor -- start 2024-08-31T06:09:55.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.951+0000 7fddef600640 1 -- start start 2024-08-31T06:09:55.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.952+0000 7fddef600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdde8104750 0x7fdde810ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:55.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.952+0000 7fddef600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdde81050a0 0x7fdde810b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:55.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.952+0000 7fddef600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdde810dbb0 0x7fdde810b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:55.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.952+0000 7fddef600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdde8103f00 con 0x7fdde810dbb0 2024-08-31T06:09:55.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.952+0000 7fddef600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdde8103d80 con 0x7fdde81050a0 2024-08-31T06:09:55.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.952+0000 7fddef600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdde8104080 con 0x7fdde8104750 2024-08-31T06:09:55.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.952+0000 7fdde7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdde81050a0 0x7fdde810b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:55.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.952+0000 7fdde7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdde81050a0 0x7fdde810b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:50798/0 (socket says 172.21.15.16:50798) 2024-08-31T06:09:55.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.952+0000 7fdde7e00640 1 -- 172.21.15.16:0/4073807118 learned_addr learned my addr 172.21.15.16:0/4073807118 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:55.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.953+0000 7fdded600640 1 --2- 172.21.15.16:0/4073807118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdde810dbb0 0x7fdde810b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:55.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.953+0000 7fddecc00640 1 --2- 172.21.15.16:0/4073807118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdde8104750 0x7fdde810ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:55.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.953+0000 7fddecc00640 1 -- 172.21.15.16:0/4073807118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdde81050a0 msgr2=0x7fdde810b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:55.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.953+0000 7fddecc00640 1 --2- 172.21.15.16:0/4073807118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdde81050a0 0x7fdde810b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:55.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.953+0000 7fddecc00640 1 -- 172.21.15.16:0/4073807118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdde810dbb0 msgr2=0x7fdde810b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:55.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.953+0000 7fddecc00640 1 --2- 172.21.15.16:0/4073807118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdde810dbb0 0x7fdde810b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:55.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.954+0000 7fddecc00640 1 -- 172.21.15.16:0/4073807118 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdde81ac900 con 0x7fdde8104750 2024-08-31T06:09:55.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.954+0000 7fdded600640 1 --2- 172.21.15.16:0/4073807118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdde810dbb0 0x7fdde810b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:09:55.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.954+0000 7fddecc00640 1 --2- 172.21.15.16:0/4073807118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdde8104750 0x7fdde810ac20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdddc00e9c0 tx=0x7fdddc00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:55.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.955+0000 7fdde5600640 1 -- 172.21.15.16:0/4073807118 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdddc00ccc0 con 0x7fdde8104750 2024-08-31T06:09:55.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.955+0000 7fddef600640 1 -- 172.21.15.16:0/4073807118 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdde81acbf0 con 0x7fdde8104750 2024-08-31T06:09:55.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.955+0000 7fddef600640 1 -- 172.21.15.16:0/4073807118 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdde81ad180 con 0x7fdde8104750 2024-08-31T06:09:55.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.955+0000 7fdde5600640 1 -- 172.21.15.16:0/4073807118 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdddc0035f0 con 0x7fdde8104750 2024-08-31T06:09:55.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.955+0000 7fdde5600640 1 -- 172.21.15.16:0/4073807118 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdddc00ce60 con 0x7fdde8104750 2024-08-31T06:09:55.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.960+0000 7fddf62a3740 1 Processor -- start 2024-08-31T06:09:55.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.960+0000 7fddf62a3740 1 -- start start 2024-08-31T06:09:55.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.961+0000 7fdde5600640 1 -- 172.21.15.16:0/4073807118 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fdddc010690 con 0x7fdde8104750 2024-08-31T06:09:55.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.961+0000 7fddf62a3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5558364ef070 0x555836894990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:55.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.961+0000 7fddf62a3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5558364c61b0 0x555836894ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:55.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.961+0000 7fddf62a3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558364ce390 0x555836895410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:55.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.961+0000 7fddf62a3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55583680b7b0 con 0x5558364ce390 2024-08-31T06:09:55.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.961+0000 7fddf62a3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5558367fad40 con 0x5558364ef070 2024-08-31T06:09:55.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.961+0000 7fddf62a3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5558367fecf0 con 0x5558364c61b0 2024-08-31T06:09:55.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.962+0000 7fddecc00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5558364c61b0 0x555836894ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:55.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.962+0000 7fddecc00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5558364c61b0 0x555836894ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:44200/0 (socket says 172.21.15.16:44200) 2024-08-31T06:09:55.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.962+0000 7fddecc00640 1 -- 172.21.15.16:0/2765406181 learned_addr learned my addr 172.21.15.16:0/2765406181 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:55.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.962+0000 7fdded600640 1 --2- 172.21.15.16:0/2765406181 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5558364ef070 0x555836894990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:55.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.962+0000 7fdde7e00640 1 --2- 172.21.15.16:0/2765406181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558364ce390 0x555836895410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:55.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.962+0000 7fddecc00640 1 -- 172.21.15.16:0/2765406181 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5558364ef070 msgr2=0x555836894990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:55.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.962+0000 7fddecc00640 1 --2- 172.21.15.16:0/2765406181 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5558364ef070 0x555836894990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:55.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.962+0000 7fddecc00640 1 -- 172.21.15.16:0/2765406181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558364ce390 msgr2=0x555836895410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:55.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.962+0000 7fddecc00640 1 --2- 172.21.15.16:0/2765406181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558364ce390 0x555836895410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:55.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.962+0000 7fddecc00640 1 -- 172.21.15.16:0/2765406181 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55583655a650 con 0x5558364c61b0 2024-08-31T06:09:55.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.962+0000 7fdde7e00640 1 --2- 172.21.15.16:0/2765406181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558364ce390 0x555836895410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:09:55.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.963+0000 7fddecc00640 1 --2- 172.21.15.16:0/2765406181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5558364c61b0 0x555836894ed0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdddc0e2e10 tx=0x7fdddc0ebef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a1cd08e8b0df399 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:55.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.964+0000 7fddd9c00640 1 -- 172.21.15.16:0/2765406181 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdddc0f4070 con 0x5558364c61b0 2024-08-31T06:09:55.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.964+0000 7fddd9c00640 1 -- 172.21.15.16:0/2765406181 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdddc0e70d0 con 0x5558364c61b0 2024-08-31T06:09:55.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.964+0000 7fddd9c00640 1 -- 172.21.15.16:0/2765406181 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdddc0f0680 con 0x5558364c61b0 2024-08-31T06:09:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.966+0000 7fddf62a3740 1 -- 172.21.15.16:0/2765406181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5558364c61b0 msgr2=0x555836894ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.966+0000 7fddf62a3740 1 --2- 172.21.15.16:0/2765406181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5558364c61b0 0x555836894ed0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdddc0e2e10 tx=0x7fdddc0ebef0 comp rx=0 tx=0).stop 2024-08-31T06:09:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.967+0000 7fddf62a3740 1 -- 172.21.15.16:0/2765406181 shutdown_connections 2024-08-31T06:09:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.967+0000 7fddf62a3740 1 --2- 172.21.15.16:0/2765406181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5558364c61b0 0x555836894ed0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:55.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.967+0000 7fddf62a3740 1 --2- 172.21.15.16:0/2765406181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558364ce390 0x555836895410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:55.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.967+0000 7fddf62a3740 1 --2- 172.21.15.16:0/2765406181 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5558364ef070 0x555836894990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:55.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.967+0000 7fddf62a3740 1 -- 172.21.15.16:0/2765406181 >> 172.21.15.16:0/2765406181 conn(0x5558364e3460 msgr2=0x555836892880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:55.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.967+0000 7fddf62a3740 1 -- 172.21.15.16:0/2765406181 shutdown_connections 2024-08-31T06:09:55.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.967+0000 7fddf62a3740 1 -- 172.21.15.16:0/2765406181 wait complete. 2024-08-31T06:09:55.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.968+0000 7fdde5600640 1 --2- 172.21.15.16:0/4073807118 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fddd00b4cb0 0x7fddd00b71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:55.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.968+0000 7fdde5600640 1 -- 172.21.15.16:0/4073807118 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fdddc0e5780 con 0x7fdde8104750 2024-08-31T06:09:55.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.968+0000 7fddf62a3740 1 -- start start 2024-08-31T06:09:55.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.968+0000 7fddf62a3740 1 Processor -- start 2024-08-31T06:09:55.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.969+0000 7fdde7e00640 1 --2- 172.21.15.16:0/4073807118 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fddd00b4cb0 0x7fddd00b71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:55.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:55 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:09:55.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:55 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:09:55.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:55 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:09:55.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.970+0000 7fddf62a3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5558364c61b0 0x5558368a0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:55.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.970+0000 7fddf62a3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558364ce390 0x5558368a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:55.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.970+0000 7fddf62a3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5558364ef070 0x5558368ac4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:55.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.970+0000 7fddf62a3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55583665d700 con 0x5558364ce390 2024-08-31T06:09:55.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.970+0000 7fddf62a3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555836781cf0 con 0x5558364ef070 2024-08-31T06:09:55.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.970+0000 7fddf62a3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5558368a3170 con 0x5558364c61b0 2024-08-31T06:09:55.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.970+0000 7fddf62a3740 10 client.0 _ll_register_callbacks cb 0x5558367a7470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:09:55.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.971+0000 7fdded600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5558364ef070 0x5558368ac4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:55.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.971+0000 7fdded600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5558364ef070 0x5558368ac4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:50824/0 (socket says 172.21.15.16:50824) 2024-08-31T06:09:55.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.971+0000 7fdded600640 1 -- 172.21.15.16:0/2325687928 learned_addr learned my addr 172.21.15.16:0/2325687928 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:55.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.971+0000 7fddecc00640 1 --2- 172.21.15.16:0/2325687928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5558364c61b0 0x5558368a0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:55.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.971+0000 7fdde7e00640 1 --2- 172.21.15.16:0/2325687928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558364ce390 0x5558368a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:55.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.972+0000 7fdded600640 1 -- 172.21.15.16:0/2325687928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5558364c61b0 msgr2=0x5558368a0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:55.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.972+0000 7fdded600640 1 --2- 172.21.15.16:0/2325687928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5558364c61b0 0x5558368a0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:55.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.972+0000 7fdded600640 1 -- 172.21.15.16:0/2325687928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558364ce390 msgr2=0x5558368a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:55.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.972+0000 7fdded600640 1 --2- 172.21.15.16:0/2325687928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558364ce390 0x5558368a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:55.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.972+0000 7fdded600640 1 -- 172.21.15.16:0/2325687928 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558367c28b0 con 0x5558364ef070 2024-08-31T06:09:55.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.972+0000 7fdde7e00640 1 --2- 172.21.15.16:0/2325687928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558364ce390 0x5558368a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:09:55.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.972+0000 7fddecc00640 1 --2- 172.21.15.16:0/2325687928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5558364c61b0 0x5558368a0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:09:55.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.972+0000 7fdded600640 1 --2- 172.21.15.16:0/2325687928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5558364ef070 0x5558368ac4f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdde0010b90 tx=0x7fdde0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=717aefbab637780e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:55.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.972+0000 7fddce000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:09:55.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.973+0000 7fdde7e00640 1 --2- 172.21.15.16:0/4073807118 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fddd00b4cb0 0x7fddd00b71c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fddd4005ab0 tx=0x7fddd4031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:55.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.974+0000 7fddce000640 1 -- 172.21.15.16:0/2325687928 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde001b420 con 0x5558364ef070 2024-08-31T06:09:55.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.974+0000 7fddce000640 1 -- 172.21.15.16:0/2325687928 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fdde001a3c0 con 0x5558364ef070 2024-08-31T06:09:55.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.974+0000 7fddf62a3740 1 -- 172.21.15.16:0/2325687928 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdddc0e70d0 con 0x5558364ef070 2024-08-31T06:09:55.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.974+0000 7fddce000640 1 -- 172.21.15.16:0/2325687928 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdde001b5c0 con 0x5558364ef070 2024-08-31T06:09:55.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.974+0000 7fddf62a3740 10 client.26135 fetch_fsmap learned FSMap version 31 2024-08-31T06:09:55.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.974+0000 7fddf62a3740 1 -- 172.21.15.16:0/2325687928 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5558368c0170 con 0x5558364ef070 2024-08-31T06:09:55.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.975+0000 7fddce000640 1 -- 172.21.15.16:0/2325687928 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdddc0e70d0 con 0x5558364ef070 2024-08-31T06:09:55.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.975+0000 7fddce000640 1 -- 172.21.15.16:0/2325687928 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fdde001a930 con 0x5558364ef070 2024-08-31T06:09:55.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.975+0000 7fddf62a3740 10 client.26135 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:09:55.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.975+0000 7fddf62a3740 10 client.26135 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T06:09:55.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.975+0000 7fddf62a3740 20 client.26135 populate_metadata read hostname 'smithi016' 2024-08-31T06:09:55.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.975+0000 7fddf62a3740 1 --2- 172.21.15.16:0/2325687928 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x5558364ad9f0 0x5558368c3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:55.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.975+0000 7fddf62a3740 4 client.26135 mds_command: new command op to 24712 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:09:55.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.975+0000 7fddf62a3740 1 -- 172.21.15.16:0/2325687928 --> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55583680d700 con 0x5558364ad9f0 2024-08-31T06:09:55.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.976+0000 7fddecc00640 1 --2- 172.21.15.16:0/2325687928 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x5558364ad9f0 0x5558368c3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:55.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.977+0000 7fddce000640 10 client.26135 ms_handle_connect on v2:172.21.15.184:6838/4011051479 2024-08-31T06:09:55.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.977+0000 7fddecc00640 1 --2- 172.21.15.16:0/2325687928 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x5558364ad9f0 0x5558368c3b00 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=710ab3b7a71095bd server_cookie=c297f383818c069b in_seq=0 out_seq=0 2024-08-31T06:09:55.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.978+0000 7fddce000640 1 -- 172.21.15.16:0/2325687928 <== mds.2 v2:172.21.15.184:6838/4011051479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55583680d700 con 0x5558364ad9f0 2024-08-31T06:09:55.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.978+0000 7fddce000640 10 client.26135 handle_command_reply: tid=0 2024-08-31T06:09:55.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.978+0000 7fddf62a3740 1 client.26135 shutdown 2024-08-31T06:09:55.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.978+0000 7fddf62a3740 1 -- 172.21.15.16:0/2325687928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5558364ef070 msgr2=0x5558368ac4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:55.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.978+0000 7fddf62a3740 1 --2- 172.21.15.16:0/2325687928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5558364ef070 0x5558368ac4f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdde0010b90 tx=0x7fdde0011810 comp rx=0 tx=0).stop 2024-08-31T06:09:55.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.979+0000 7fddf62a3740 1 -- 172.21.15.16:0/2325687928 shutdown_connections 2024-08-31T06:09:55.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.979+0000 7fddf62a3740 1 --2- 172.21.15.16:0/2325687928 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x5558364ad9f0 0x5558368c3b00 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:55.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.979+0000 7fddf62a3740 1 --2- 172.21.15.16:0/2325687928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5558364ef070 0x5558368ac4f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:55.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.979+0000 7fddf62a3740 1 --2- 172.21.15.16:0/2325687928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558364ce390 0x5558368a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:55.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.979+0000 7fddf62a3740 1 --2- 172.21.15.16:0/2325687928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5558364c61b0 0x5558368a0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:55.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.979+0000 7fddce000640 0 client.26135 ms_handle_reset on v2:172.21.15.184:6838/4011051479 2024-08-31T06:09:55.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.979+0000 7fddf62a3740 1 -- 172.21.15.16:0/2325687928 >> 172.21.15.16:0/2325687928 conn(0x5558364e3460 msgr2=0x555836897b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:55.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.979+0000 7fddf62a3740 1 -- 172.21.15.16:0/2325687928 shutdown_connections 2024-08-31T06:09:55.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.980+0000 7fddf62a3740 1 -- 172.21.15.16:0/2325687928 wait complete. 2024-08-31T06:09:55.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:55.980+0000 7fddf62a3740 20 client.26135 trim_cache size 0 max 16384 2024-08-31T06:09:56.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.001+0000 7fddf62a3740 1 Processor -- start 2024-08-31T06:09:56.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.002+0000 7fddf62a3740 1 -- start start 2024-08-31T06:09:56.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.002+0000 7fddf62a3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5558364c61b0 0x55583689a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:56.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.002+0000 7fddf62a3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5558364ce390 0x5558368ac3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:56.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.002+0000 7fddf62a3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558364ef070 0x5558368ac930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:56.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.002+0000 7fddf62a3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fddc8011070 con 0x5558364ef070 2024-08-31T06:09:56.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.002+0000 7fddf62a3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fddc800e880 con 0x5558364ce390 2024-08-31T06:09:56.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.002+0000 7fddf62a3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fddc800c690 con 0x5558364c61b0 2024-08-31T06:09:56.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.002+0000 7fddecc00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5558364c61b0 0x55583689a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:56.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.002+0000 7fdde7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5558364ce390 0x5558368ac3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:56.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.002+0000 7fdde7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5558364ce390 0x5558368ac3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:50834/0 (socket says 172.21.15.16:50834) 2024-08-31T06:09:56.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.002+0000 7fdde7e00640 1 -- 172.21.15.16:0/1825104902 learned_addr learned my addr 172.21.15.16:0/1825104902 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:56.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.003+0000 7fdded600640 1 --2- 172.21.15.16:0/1825104902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558364ef070 0x5558368ac930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:56.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.003+0000 7fddecc00640 1 -- 172.21.15.16:0/1825104902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5558364ce390 msgr2=0x5558368ac3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:56.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.003+0000 7fddecc00640 1 --2- 172.21.15.16:0/1825104902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5558364ce390 0x5558368ac3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:56.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.003+0000 7fddecc00640 1 -- 172.21.15.16:0/1825104902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558364ef070 msgr2=0x5558368ac930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:56.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.003+0000 7fddecc00640 1 --2- 172.21.15.16:0/1825104902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558364ef070 0x5558368ac930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:56.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.003+0000 7fddecc00640 1 -- 172.21.15.16:0/1825104902 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558368977f0 con 0x5558364c61b0 2024-08-31T06:09:56.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.003+0000 7fddecc00640 1 --2- 172.21.15.16:0/1825104902 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5558364c61b0 0x55583689a1b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdddc0abb70 tx=0x7fdddc0e79f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=548532559743aa82 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:56.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.004+0000 7fddd9200640 1 -- 172.21.15.16:0/1825104902 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdddc0f4030 con 0x5558364c61b0 2024-08-31T06:09:56.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.004+0000 7fddd9200640 1 -- 172.21.15.16:0/1825104902 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55583680d700 con 0x5558364c61b0 2024-08-31T06:09:56.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.004+0000 7fddd9200640 1 -- 172.21.15.16:0/1825104902 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdddc0ab980 con 0x5558364c61b0 2024-08-31T06:09:56.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.005+0000 7fddf62a3740 1 -- 172.21.15.16:0/1825104902 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5558364c61b0 msgr2=0x55583689a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:56.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.005+0000 7fddf62a3740 1 --2- 172.21.15.16:0/1825104902 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5558364c61b0 0x55583689a1b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdddc0abb70 tx=0x7fdddc0e79f0 comp rx=0 tx=0).stop 2024-08-31T06:09:56.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.006+0000 7fddf62a3740 1 -- 172.21.15.16:0/1825104902 shutdown_connections 2024-08-31T06:09:56.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.006+0000 7fddf62a3740 1 --2- 172.21.15.16:0/1825104902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558364ef070 0x5558368ac930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.007+0000 7fddf62a3740 1 --2- 172.21.15.16:0/1825104902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5558364ce390 0x5558368ac3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.007+0000 7fddf62a3740 1 --2- 172.21.15.16:0/1825104902 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5558364c61b0 0x55583689a1b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.007+0000 7fddf62a3740 1 -- 172.21.15.16:0/1825104902 >> 172.21.15.16:0/1825104902 conn(0x5558364e3460 msgr2=0x555836899aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.007+0000 7fddf62a3740 1 -- 172.21.15.16:0/1825104902 shutdown_connections 2024-08-31T06:09:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.007+0000 7fddf62a3740 1 -- 172.21.15.16:0/1825104902 wait complete. 2024-08-31T06:09:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.008+0000 7fddf62a3740 1 -- start start 2024-08-31T06:09:56.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.008+0000 7fddf62a3740 1 Processor -- start 2024-08-31T06:09:56.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.008+0000 7fddf62a3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558364c61b0 0x5558368a4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:56.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.008+0000 7fddf62a3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5558364ce390 0x5558368a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:56.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.008+0000 7fddf62a3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5558364ef070 0x55583692d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:56.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.008+0000 7fddf62a3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fddc800cd40 con 0x5558364c61b0 2024-08-31T06:09:56.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.008+0000 7fddf62a3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fddc800e1b0 con 0x5558364ef070 2024-08-31T06:09:56.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.008+0000 7fddf62a3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fddc800d410 con 0x5558364ce390 2024-08-31T06:09:56.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.008+0000 7fddf62a3740 10 client.0 _ll_register_callbacks cb 0x7fdde0010660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:09:56.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.009+0000 7fdde7e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5558364ce390 0x5558368a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:56.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.009+0000 7fddecc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558364c61b0 0x5558368a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:56.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.009+0000 7fdde7e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5558364ce390 0x5558368a4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:44218/0 (socket says 172.21.15.16:44218) 2024-08-31T06:09:56.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.009+0000 7fdde7e00640 1 -- 172.21.15.16:0/2060277184 learned_addr learned my addr 172.21.15.16:0/2060277184 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:56.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.009+0000 7fdded600640 1 --2- 172.21.15.16:0/2060277184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5558364ef070 0x55583692d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:56.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.009+0000 7fdde7e00640 1 -- 172.21.15.16:0/2060277184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5558364ef070 msgr2=0x55583692d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:56.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.009+0000 7fdde7e00640 1 --2- 172.21.15.16:0/2060277184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5558364ef070 0x55583692d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:56.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.009+0000 7fdde7e00640 1 -- 172.21.15.16:0/2060277184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558364c61b0 msgr2=0x5558368a4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:56.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.009+0000 7fdde7e00640 1 --2- 172.21.15.16:0/2060277184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558364c61b0 0x5558368a4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:56.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.010+0000 7fdde7e00640 1 -- 172.21.15.16:0/2060277184 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558368a5580 con 0x5558364ce390 2024-08-31T06:09:56.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.010+0000 7fdded600640 1 --2- 172.21.15.16:0/2060277184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5558364ef070 0x55583692d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:09:56.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.010+0000 7fddecc00640 1 --2- 172.21.15.16:0/2060277184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558364c61b0 0x5558368a4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:09:56.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.010+0000 7fdde7e00640 1 --2- 172.21.15.16:0/2060277184 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5558364ce390 0x5558368a4cb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fddd402a9e0 tx=0x7fddd402a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3193003c0d9d7b7e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:56.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.010+0000 7fddce000640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:09:56.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.011+0000 7fddce000640 1 -- 172.21.15.16:0/2060277184 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd4035050 con 0x5558364ce390 2024-08-31T06:09:56.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.011+0000 7fddce000640 1 -- 172.21.15.16:0/2060277184 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fddd40037f0 con 0x5558364ce390 2024-08-31T06:09:56.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.012+0000 7fddf62a3740 1 -- 172.21.15.16:0/2060277184 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55583680d700 con 0x5558364ce390 2024-08-31T06:09:56.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.012+0000 7fddce000640 1 -- 172.21.15.16:0/2060277184 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fddd4004e50 con 0x5558364ce390 2024-08-31T06:09:56.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.013+0000 7fddf62a3740 10 client.26146 fetch_fsmap learned FSMap version 31 2024-08-31T06:09:56.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.013+0000 7fddf62a3740 1 -- 172.21.15.16:0/2060277184 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55583692eac0 con 0x5558364ce390 2024-08-31T06:09:56.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.013+0000 7fddce000640 1 -- 172.21.15.16:0/2060277184 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55583680d700 con 0x5558364ce390 2024-08-31T06:09:56.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.013+0000 7fddce000640 1 -- 172.21.15.16:0/2060277184 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fddd404f020 con 0x5558364ce390 2024-08-31T06:09:56.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.013+0000 7fddf62a3740 10 client.26146 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:09:56.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.013+0000 7fddf62a3740 10 client.26146 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T06:09:56.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.013+0000 7fddf62a3740 20 client.26146 populate_metadata read hostname 'smithi016' 2024-08-31T06:09:56.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.013+0000 7fddf62a3740 1 --2- 172.21.15.16:0/2060277184 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x555836482da0 0x5558369312b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:56.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.013+0000 7fddf62a3740 4 client.26146 mds_command: new command op to 24712 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:09:56.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.013+0000 7fddf62a3740 1 -- 172.21.15.16:0/2060277184 --> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5558368091a0 con 0x555836482da0 2024-08-31T06:09:56.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.014+0000 7fddecc00640 1 --2- 172.21.15.16:0/2060277184 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x555836482da0 0x5558369312b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.014+0000 7fddecc00640 1 --2- 172.21.15.16:0/2060277184 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x555836482da0 0x5558369312b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6f24247a9f1ef528 server_cookie=6409d75cf40e86be in_seq=0 out_seq=0 2024-08-31T06:09:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.014+0000 7fddce000640 10 client.26146 ms_handle_connect on v2:172.21.15.184:6838/4011051479 2024-08-31T06:09:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.014+0000 7fddce000640 1 -- 172.21.15.16:0/2060277184 <== mds.2 v2:172.21.15.184:6838/4011051479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5558368091a0 con 0x555836482da0 2024-08-31T06:09:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.014+0000 7fddce000640 10 client.26146 handle_command_reply: tid=0 2024-08-31T06:09:56.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.014+0000 7fddf62a3740 1 client.26146 shutdown 2024-08-31T06:09:56.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.015+0000 7fddf62a3740 1 -- 172.21.15.16:0/2060277184 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5558364ce390 msgr2=0x5558368a4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:56.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.015+0000 7fddf62a3740 1 --2- 172.21.15.16:0/2060277184 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5558364ce390 0x5558368a4cb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fddd402a9e0 tx=0x7fddd402a3f0 comp rx=0 tx=0).stop 2024-08-31T06:09:56.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.015+0000 7fddf62a3740 1 -- 172.21.15.16:0/2060277184 shutdown_connections 2024-08-31T06:09:56.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.015+0000 7fddf62a3740 1 --2- 172.21.15.16:0/2060277184 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x555836482da0 0x5558369312b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:56.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.015+0000 7fddf62a3740 1 --2- 172.21.15.16:0/2060277184 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5558364ce390 0x5558368a4cb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:56.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.015+0000 7fddf62a3740 1 --2- 172.21.15.16:0/2060277184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5558364ef070 0x55583692d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:56.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.015+0000 7fddf62a3740 1 --2- 172.21.15.16:0/2060277184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558364c61b0 0x5558368a4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:56.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.015+0000 7fddf62a3740 1 -- 172.21.15.16:0/2060277184 >> 172.21.15.16:0/2060277184 conn(0x5558364e3460 msgr2=0x5558368959c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:56.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.015+0000 7fddce000640 0 client.26146 ms_handle_reset on v2:172.21.15.184:6838/4011051479 2024-08-31T06:09:56.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.016+0000 7fddf62a3740 1 -- 172.21.15.16:0/2060277184 shutdown_connections 2024-08-31T06:09:56.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.016+0000 7fddf62a3740 1 -- 172.21.15.16:0/2060277184 wait complete. 2024-08-31T06:09:56.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.016+0000 7fddf62a3740 20 client.26146 trim_cache size 0 max 16384 2024-08-31T06:09:56.020 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:09:56.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.018+0000 7fddef600640 1 -- 172.21.15.16:0/4073807118 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fddd00b4cb0 msgr2=0x7fddd00b71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:56.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.018+0000 7fddef600640 1 --2- 172.21.15.16:0/4073807118 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fddd00b4cb0 0x7fddd00b71c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fddd4005ab0 tx=0x7fddd4031400 comp rx=0 tx=0).stop 2024-08-31T06:09:56.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.018+0000 7fddef600640 1 -- 172.21.15.16:0/4073807118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdde8104750 msgr2=0x7fdde810ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:56.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.018+0000 7fddef600640 1 --2- 172.21.15.16:0/4073807118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdde8104750 0x7fdde810ac20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdddc00e9c0 tx=0x7fdddc00ee80 comp rx=0 tx=0).stop 2024-08-31T06:09:56.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.019+0000 7fddef600640 1 -- 172.21.15.16:0/4073807118 shutdown_connections 2024-08-31T06:09:56.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.019+0000 7fddef600640 1 --2- 172.21.15.16:0/4073807118 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fddd00b4cb0 0x7fddd00b71c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:56.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.019+0000 7fddef600640 1 --2- 172.21.15.16:0/4073807118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdde810dbb0 0x7fdde810b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:56.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.019+0000 7fddef600640 1 --2- 172.21.15.16:0/4073807118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdde81050a0 0x7fdde810b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:56.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.019+0000 7fddef600640 1 --2- 172.21.15.16:0/4073807118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdde8104750 0x7fdde810ac20 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:56.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.019+0000 7fddef600640 1 -- 172.21.15.16:0/4073807118 >> 172.21.15.16:0/4073807118 conn(0x7fdde80794d0 msgr2=0x7fdde8100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:56.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.019+0000 7fddef600640 1 -- 172.21.15.16:0/4073807118 shutdown_connections 2024-08-31T06:09:56.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:56.019+0000 7fddef600640 1 -- 172.21.15.16:0/4073807118 wait complete. 2024-08-31T06:09:56.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:55 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:09:56.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:55 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:09:56.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:55 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:09:56.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 150.0 s, 2083.4 tps, lat 48.208 ms stddev 51.644 2024-08-31T06:09:57.025 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T06:09:57.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:57 smithi016 ceph-mon[17194]: pgmap v445: 97 pgs: 97 active+clean; 1.6 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.7 MiB/s wr, 780 op/s 2024-08-31T06:09:57.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:57 smithi184 ceph-mon[22331]: pgmap v445: 97 pgs: 97 active+clean; 1.6 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.7 MiB/s wr, 780 op/s 2024-08-31T06:09:57.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:57 smithi155 ceph-mon[21742]: pgmap v445: 97 pgs: 97 active+clean; 1.6 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.7 MiB/s wr, 780 op/s 2024-08-31T06:09:57.660 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:09:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.273+0000 7ff308600640 1 -- 172.21.15.16:0/363290839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff30010fc00 msgr2=0x7ff300110010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.273+0000 7ff308600640 1 --2- 172.21.15.16:0/363290839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff30010fc00 0x7ff300110010 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec009900 tx=0x7ff2ec02a410 comp rx=0 tx=0).stop 2024-08-31T06:09:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.273+0000 7ff308600640 1 -- 172.21.15.16:0/363290839 shutdown_connections 2024-08-31T06:09:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.273+0000 7ff308600640 1 --2- 172.21.15.16:0/363290839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff30010fc00 0x7ff300110010 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.273+0000 7ff308600640 1 --2- 172.21.15.16:0/363290839 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff300110550 0x7ff300116d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.273+0000 7ff308600640 1 --2- 172.21.15.16:0/363290839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff30010a400 0x7ff30010a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.273+0000 7ff308600640 1 -- 172.21.15.16:0/363290839 >> 172.21.15.16:0/363290839 conn(0x7ff300070fc0 msgr2=0x7ff300073af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.274+0000 7ff308600640 1 -- 172.21.15.16:0/363290839 shutdown_connections 2024-08-31T06:09:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.274+0000 7ff308600640 1 -- 172.21.15.16:0/363290839 wait complete. 2024-08-31T06:09:58.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.275+0000 7ff308600640 1 Processor -- start 2024-08-31T06:09:58.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.276+0000 7ff308600640 1 -- start start 2024-08-31T06:09:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.277+0000 7ff308600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff30010a400 0x7ff30011ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.278+0000 7ff308600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff30010fc00 0x7ff30011af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.278+0000 7ff308600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff300110550 0x7ff30011b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.278+0000 7ff308600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff300119f40 con 0x7ff30010fc00 2024-08-31T06:09:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.278+0000 7ff308600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff300119dc0 con 0x7ff30010a400 2024-08-31T06:09:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.278+0000 7ff308600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff30011a0c0 con 0x7ff300110550 2024-08-31T06:09:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.278+0000 7ff305c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff30010a400 0x7ff30011ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.278+0000 7ff305c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff30010a400 0x7ff30011ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:50860/0 (socket says 172.21.15.16:50860) 2024-08-31T06:09:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.278+0000 7ff305c00640 1 -- 172.21.15.16:0/1497856154 learned_addr learned my addr 172.21.15.16:0/1497856154 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.279+0000 7ff306600640 1 --2- 172.21.15.16:0/1497856154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff300110550 0x7ff30011b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.279+0000 7ff305c00640 1 -- 172.21.15.16:0/1497856154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff300110550 msgr2=0x7ff30011b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.279+0000 7ff305c00640 1 --2- 172.21.15.16:0/1497856154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff300110550 0x7ff30011b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.279+0000 7ff305c00640 1 -- 172.21.15.16:0/1497856154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff30010fc00 msgr2=0x7ff30011af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.279+0000 7ff305200640 1 --2- 172.21.15.16:0/1497856154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff30010fc00 0x7ff30011af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.279+0000 7ff305c00640 1 --2- 172.21.15.16:0/1497856154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff30010fc00 0x7ff30011af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.279+0000 7ff305c00640 1 -- 172.21.15.16:0/1497856154 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3001b51d0 con 0x7ff30010a400 2024-08-31T06:09:58.283 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.280+0000 7ff305c00640 1 --2- 172.21.15.16:0/1497856154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff30010a400 0x7ff30011ecc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff2f400c920 tx=0x7ff2f400cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:58.283 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.280+0000 7ff2fe000640 1 -- 172.21.15.16:0/1497856154 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4007c70 con 0x7ff30010a400 2024-08-31T06:09:58.283 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.280+0000 7ff2fe000640 1 -- 172.21.15.16:0/1497856154 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2f4007e10 con 0x7ff30010a400 2024-08-31T06:09:58.283 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.280+0000 7ff2fe000640 1 -- 172.21.15.16:0/1497856154 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4018820 con 0x7ff30010a400 2024-08-31T06:09:58.283 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.280+0000 7ff308600640 1 -- 172.21.15.16:0/1497856154 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3001b53a0 con 0x7ff30010a400 2024-08-31T06:09:58.283 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.280+0000 7ff308600640 1 -- 172.21.15.16:0/1497856154 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3001b5870 con 0x7ff30010a400 2024-08-31T06:09:58.289 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.286+0000 7ff2fe000640 1 -- 172.21.15.16:0/1497856154 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7ff2f40189c0 con 0x7ff30010a400 2024-08-31T06:09:58.289 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.287+0000 7ff30f655740 1 Processor -- start 2024-08-31T06:09:58.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.288+0000 7ff2fe000640 1 --2- 172.21.15.16:0/1497856154 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff2e80b4d20 0x7ff2e80b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:58.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.288+0000 7ff2fe000640 1 -- 172.21.15.16:0/1497856154 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7ff2f40e56d0 con 0x7ff30010a400 2024-08-31T06:09:58.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.288+0000 7ff30f655740 1 -- start start 2024-08-31T06:09:58.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.289+0000 7ff30f655740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fda51681b0 0x55fda550b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:58.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.289+0000 7ff30f655740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fda513f2f0 0x55fda550be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:58.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.289+0000 7ff305c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fda51681b0 0x55fda550b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:58.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.289+0000 7ff305c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fda51681b0 0x55fda550b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:50870/0 (socket says 172.21.15.16:50870) 2024-08-31T06:09:58.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.289+0000 7ff30f655740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fda51474d0 0x55fda550c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:58.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.290+0000 7ff30f655740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fda5480f70 con 0x55fda51474d0 2024-08-31T06:09:58.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.290+0000 7ff30f655740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55fda5470500 con 0x55fda51681b0 2024-08-31T06:09:58.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.290+0000 7ff30f655740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55fda54744b0 con 0x55fda513f2f0 2024-08-31T06:09:58.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.290+0000 7ff306600640 1 --2- 172.21.15.16:0/1497856154 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff2e80b4d20 0x7ff2e80b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:58.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.291+0000 7ff305c00640 1 -- 172.21.15.16:0/687444365 learned_addr learned my addr 172.21.15.16:0/687444365 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:58.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.291+0000 7ff305c00640 1 -- 172.21.15.16:0/687444365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fda513f2f0 msgr2=0x55fda550be60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T06:09:58.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.291+0000 7ff305c00640 1 --2- 172.21.15.16:0/687444365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fda513f2f0 0x55fda550be60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.291+0000 7ff305c00640 1 -- 172.21.15.16:0/687444365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fda51474d0 msgr2=0x55fda550c3a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:09:58.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.292+0000 7ff306600640 1 --2- 172.21.15.16:0/687444365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fda51474d0 0x55fda550c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:58.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.292+0000 7ff305c00640 1 --2- 172.21.15.16:0/687444365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fda51474d0 0x55fda550c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.292+0000 7ff305c00640 1 -- 172.21.15.16:0/687444365 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fda542ac10 con 0x55fda51681b0 2024-08-31T06:09:58.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.293+0000 7ff306600640 1 --2- 172.21.15.16:0/687444365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fda51474d0 0x55fda550c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:09:58.295 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.293+0000 7ff305c00640 1 --2- 172.21.15.16:0/687444365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fda51681b0 0x55fda550b920 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff2f400c8f0 tx=0x7ff2f40aba50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4d77c0e8acae09 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.294+0000 7ff2f2a00640 1 -- 172.21.15.16:0/687444365 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f40b0670 con 0x55fda51681b0 2024-08-31T06:09:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.294+0000 7ff2f2a00640 1 -- 172.21.15.16:0/687444365 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2f40b5120 con 0x55fda51681b0 2024-08-31T06:09:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.294+0000 7ff2f2a00640 1 -- 172.21.15.16:0/687444365 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4093260 con 0x55fda51681b0 2024-08-31T06:09:58.297 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.295+0000 7ff30f655740 1 -- 172.21.15.16:0/687444365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fda51681b0 msgr2=0x55fda550b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:58.297 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.295+0000 7ff306600640 1 --2- 172.21.15.16:0/1497856154 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff2e80b4d20 0x7ff2e80b7230 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8005fd0 tx=0x7ff2f8009840 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:58.297 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.295+0000 7ff30f655740 1 --2- 172.21.15.16:0/687444365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fda51681b0 0x55fda550b920 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff2f400c8f0 tx=0x7ff2f40aba50 comp rx=0 tx=0).stop 2024-08-31T06:09:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.296+0000 7ff30f655740 1 -- 172.21.15.16:0/687444365 shutdown_connections 2024-08-31T06:09:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.296+0000 7ff30f655740 1 --2- 172.21.15.16:0/687444365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fda513f2f0 0x55fda550be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.296+0000 7ff30f655740 1 --2- 172.21.15.16:0/687444365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fda51474d0 0x55fda550c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.296+0000 7ff30f655740 1 --2- 172.21.15.16:0/687444365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fda51681b0 0x55fda550b920 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.296+0000 7ff30f655740 1 -- 172.21.15.16:0/687444365 >> 172.21.15.16:0/687444365 conn(0x55fda515c300 msgr2=0x55fda5509810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.296+0000 7ff30f655740 1 -- 172.21.15.16:0/687444365 shutdown_connections 2024-08-31T06:09:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.296+0000 7ff30f655740 1 -- 172.21.15.16:0/687444365 wait complete. 2024-08-31T06:09:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.297+0000 7ff30f655740 1 -- start start 2024-08-31T06:09:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.297+0000 7ff30f655740 1 Processor -- start 2024-08-31T06:09:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.298+0000 7ff30f655740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fda513f2f0 0x55fda5516ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.298+0000 7ff30f655740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fda51474d0 0x55fda5517430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.298+0000 7ff30f655740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fda51681b0 0x55fda5517970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.298+0000 7ff30f655740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fda53a9ab0 con 0x55fda51474d0 2024-08-31T06:09:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.298+0000 7ff30f655740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55fda5417320 con 0x55fda513f2f0 2024-08-31T06:09:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.298+0000 7ff30f655740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55fda551a0a0 con 0x55fda51681b0 2024-08-31T06:09:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.299+0000 7ff30f655740 10 client.0 _ll_register_callbacks cb 0x55fda541d6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:09:58.303 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.299+0000 7ff305c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fda513f2f0 0x55fda5516ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:58.303 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.299+0000 7ff305c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fda513f2f0 0x55fda5516ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:50886/0 (socket says 172.21.15.16:50886) 2024-08-31T06:09:58.303 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.299+0000 7ff305c00640 1 -- 172.21.15.16:0/276936090 learned_addr learned my addr 172.21.15.16:0/276936090 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:58.303 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.300+0000 7ff305200640 1 --2- 172.21.15.16:0/276936090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fda51474d0 0x55fda5517430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:58.303 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.299+0000 7ff306600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fda51681b0 0x55fda5517970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:58.303 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.300+0000 7ff305c00640 1 -- 172.21.15.16:0/276936090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fda51681b0 msgr2=0x55fda5517970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:58.303 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.300+0000 7ff305c00640 1 --2- 172.21.15.16:0/276936090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fda51681b0 0x55fda5517970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.303 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.300+0000 7ff305c00640 1 -- 172.21.15.16:0/276936090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fda51474d0 msgr2=0x55fda5517430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.300+0000 7ff305c00640 1 --2- 172.21.15.16:0/276936090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fda51474d0 0x55fda5517430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.300+0000 7ff305c00640 1 -- 172.21.15.16:0/276936090 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fda51d3790 con 0x55fda513f2f0 2024-08-31T06:09:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.300+0000 7ff306600640 1 --2- 172.21.15.16:0/276936090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fda51681b0 0x55fda5517970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:09:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.300+0000 7ff305c00640 1 --2- 172.21.15.16:0/276936090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fda513f2f0 0x55fda5516ef0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40a7e80 tx=0x7ff2f40a82c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98d6656863e03206 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.300+0000 7ff2e7400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:09:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.301+0000 7ff2e7400640 1 -- 172.21.15.16:0/276936090 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f405c020 con 0x55fda513f2f0 2024-08-31T06:09:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.301+0000 7ff2e7400640 1 -- 172.21.15.16:0/276936090 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7ff2f40a83f0 con 0x55fda513f2f0 2024-08-31T06:09:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.301+0000 7ff30f655740 1 -- 172.21.15.16:0/276936090 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2f40b5120 con 0x55fda513f2f0 2024-08-31T06:09:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.301+0000 7ff2e7400640 1 -- 172.21.15.16:0/276936090 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff2f40b0670 con 0x55fda513f2f0 2024-08-31T06:09:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.302+0000 7ff30f655740 10 client.26153 fetch_fsmap learned FSMap version 31 2024-08-31T06:09:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.302+0000 7ff30f655740 1 -- 172.21.15.16:0/276936090 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55fda524c620 con 0x55fda513f2f0 2024-08-31T06:09:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.303+0000 7ff2e7400640 1 -- 172.21.15.16:0/276936090 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2f40b5120 con 0x55fda513f2f0 2024-08-31T06:09:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.303+0000 7ff2e7400640 1 -- 172.21.15.16:0/276936090 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7ff2f40aaad0 con 0x55fda513f2f0 2024-08-31T06:09:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.303+0000 7ff30f655740 10 client.26153 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:09:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.303+0000 7ff30f655740 10 client.26153 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:09:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.303+0000 7ff30f655740 20 client.26153 populate_metadata read hostname 'smithi016' 2024-08-31T06:09:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.303+0000 7ff30f655740 1 --2- 172.21.15.16:0/276936090 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fda5126d80 0x55fda5528dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.303+0000 7ff30f655740 4 client.26153 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:09:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.303+0000 7ff30f655740 1 -- 172.21.15.16:0/276936090 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fda5482ec0 con 0x55fda5126d80 2024-08-31T06:09:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.304+0000 7ff305200640 1 --2- 172.21.15.16:0/276936090 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fda5126d80 0x55fda5528dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.305+0000 7ff305200640 1 --2- 172.21.15.16:0/276936090 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fda5126d80 0x55fda5528dd0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88d0f377f8ff2092 server_cookie=1f416ead048d285e in_seq=0 out_seq=0 2024-08-31T06:09:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.305+0000 7ff2e7400640 10 client.26153 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:09:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.306+0000 7ff2e7400640 1 -- 172.21.15.16:0/276936090 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fda5482ec0 con 0x55fda5126d80 2024-08-31T06:09:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.306+0000 7ff2e7400640 10 client.26153 handle_command_reply: tid=0 2024-08-31T06:09:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.306+0000 7ff30f655740 1 client.26153 shutdown 2024-08-31T06:09:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.307+0000 7ff30f655740 1 -- 172.21.15.16:0/276936090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fda513f2f0 msgr2=0x55fda5516ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.307+0000 7ff30f655740 1 --2- 172.21.15.16:0/276936090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fda513f2f0 0x55fda5516ef0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40a7e80 tx=0x7ff2f40a82c0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.307+0000 7ff30f655740 1 -- 172.21.15.16:0/276936090 shutdown_connections 2024-08-31T06:09:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.307+0000 7ff30f655740 1 --2- 172.21.15.16:0/276936090 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fda5126d80 0x55fda5528dd0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.307+0000 7ff2e7400640 0 client.26153 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:09:58.311 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.307+0000 7ff30f655740 1 --2- 172.21.15.16:0/276936090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fda51681b0 0x55fda5517970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.311 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.307+0000 7ff30f655740 1 --2- 172.21.15.16:0/276936090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fda51474d0 0x55fda5517430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.311 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.307+0000 7ff30f655740 1 --2- 172.21.15.16:0/276936090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fda513f2f0 0x55fda5516ef0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.311 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.307+0000 7ff30f655740 1 -- 172.21.15.16:0/276936090 >> 172.21.15.16:0/276936090 conn(0x55fda515c300 msgr2=0x55fda550c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:58.311 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.308+0000 7ff30f655740 1 -- 172.21.15.16:0/276936090 shutdown_connections 2024-08-31T06:09:58.311 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.308+0000 7ff30f655740 1 -- 172.21.15.16:0/276936090 wait complete. 2024-08-31T06:09:58.311 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.308+0000 7ff30f655740 20 client.26153 trim_cache size 0 max 16384 2024-08-31T06:09:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.333+0000 7ff30f655740 1 Processor -- start 2024-08-31T06:09:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.333+0000 7ff30f655740 1 -- start start 2024-08-31T06:09:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.334+0000 7ff30f655740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fda5126d80 0x55fda550d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.334+0000 7ff30f655740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fda513f2f0 0x55fda5518b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.334+0000 7ff30f655740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fda51474d0 0x55fda55140c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:58.337 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.334+0000 7ff30f655740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff3001c0b60 con 0x55fda513f2f0 2024-08-31T06:09:58.337 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.334+0000 7ff30f655740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff3001be370 con 0x55fda51474d0 2024-08-31T06:09:58.337 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.334+0000 7ff30f655740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff3001bc1c0 con 0x55fda5126d80 2024-08-31T06:09:58.337 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.335+0000 7ff305c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fda5126d80 0x55fda550d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:58.337 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.335+0000 7ff305c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fda5126d80 0x55fda550d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:44260/0 (socket says 172.21.15.16:44260) 2024-08-31T06:09:58.337 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.335+0000 7ff305c00640 1 -- 172.21.15.16:0/3675278295 learned_addr learned my addr 172.21.15.16:0/3675278295 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:58.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.335+0000 7ff305200640 1 --2- 172.21.15.16:0/3675278295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fda513f2f0 0x55fda5518b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:58.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.335+0000 7ff306600640 1 --2- 172.21.15.16:0/3675278295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fda51474d0 0x55fda55140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:58.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.335+0000 7ff305c00640 1 -- 172.21.15.16:0/3675278295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fda51474d0 msgr2=0x55fda55140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:58.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.335+0000 7ff305c00640 1 --2- 172.21.15.16:0/3675278295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fda51474d0 0x55fda55140c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.335+0000 7ff305c00640 1 -- 172.21.15.16:0/3675278295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fda513f2f0 msgr2=0x55fda5518b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.335+0000 7ff305c00640 1 --2- 172.21.15.16:0/3675278295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fda513f2f0 0x55fda5518b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.335+0000 7ff305c00640 1 -- 172.21.15.16:0/3675278295 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fda55153d0 con 0x55fda5126d80 2024-08-31T06:09:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.336+0000 7ff305c00640 1 --2- 172.21.15.16:0/3675278295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fda5126d80 0x55fda550d310 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40a9070 tx=0x7ff2f40aba50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdc49d827772e41 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.336+0000 7ff2f2000640 1 -- 172.21.15.16:0/3675278295 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f405c070 con 0x55fda5126d80 2024-08-31T06:09:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.337+0000 7ff2f2000640 1 -- 172.21.15.16:0/3675278295 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2f4058e50 con 0x55fda5126d80 2024-08-31T06:09:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.337+0000 7ff2f2000640 1 -- 172.21.15.16:0/3675278295 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f40931f0 con 0x55fda5126d80 2024-08-31T06:09:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.338+0000 7ff30f655740 1 -- 172.21.15.16:0/3675278295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fda5126d80 msgr2=0x55fda550d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.338+0000 7ff30f655740 1 --2- 172.21.15.16:0/3675278295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fda5126d80 0x55fda550d310 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40a9070 tx=0x7ff2f40aba50 comp rx=0 tx=0).stop 2024-08-31T06:09:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.338+0000 7ff30f655740 1 -- 172.21.15.16:0/3675278295 shutdown_connections 2024-08-31T06:09:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.338+0000 7ff30f655740 1 --2- 172.21.15.16:0/3675278295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fda51474d0 0x55fda55140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.338+0000 7ff30f655740 1 --2- 172.21.15.16:0/3675278295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fda513f2f0 0x55fda5518b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.338+0000 7ff30f655740 1 --2- 172.21.15.16:0/3675278295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fda5126d80 0x55fda550d310 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.338+0000 7ff30f655740 1 -- 172.21.15.16:0/3675278295 >> 172.21.15.16:0/3675278295 conn(0x55fda515c300 msgr2=0x55fda5518410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.339+0000 7ff30f655740 1 -- 172.21.15.16:0/3675278295 shutdown_connections 2024-08-31T06:09:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.339+0000 7ff30f655740 1 -- 172.21.15.16:0/3675278295 wait complete. 2024-08-31T06:09:58.342 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.339+0000 7ff30f655740 1 -- start start 2024-08-31T06:09:58.342 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.340+0000 7ff30f655740 1 Processor -- start 2024-08-31T06:09:58.342 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.340+0000 7ff30f655740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fda5126d80 0x55fda552b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.340+0000 7ff30f655740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fda513f2f0 0x55fda552bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.340+0000 7ff30f655740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fda51474d0 0x55fda55884f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.341+0000 7ff30f655740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff3001bc830 con 0x55fda5126d80 2024-08-31T06:09:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.341+0000 7ff30f655740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff3001bdca0 con 0x55fda51474d0 2024-08-31T06:09:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.341+0000 7ff30f655740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff3001bcf00 con 0x55fda513f2f0 2024-08-31T06:09:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.341+0000 7ff30f655740 10 client.0 _ll_register_callbacks cb 0x7ff2f40971c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:09:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.341+0000 7ff305200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fda513f2f0 0x55fda552bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.341+0000 7ff305c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fda5126d80 0x55fda552b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.341+0000 7ff305c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fda5126d80 0x55fda552b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36000/0 (socket says 172.21.15.16:36000) 2024-08-31T06:09:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.341+0000 7ff305c00640 1 -- 172.21.15.16:0/4210464080 learned_addr learned my addr 172.21.15.16:0/4210464080 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.342+0000 7ff306600640 1 --2- 172.21.15.16:0/4210464080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fda51474d0 0x55fda55884f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.342+0000 7ff305c00640 1 -- 172.21.15.16:0/4210464080 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fda513f2f0 msgr2=0x55fda552bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.342+0000 7ff305c00640 1 --2- 172.21.15.16:0/4210464080 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fda513f2f0 0x55fda552bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.342+0000 7ff305c00640 1 -- 172.21.15.16:0/4210464080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fda51474d0 msgr2=0x55fda55884f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.342+0000 7ff305c00640 1 --2- 172.21.15.16:0/4210464080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fda51474d0 0x55fda55884f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.342+0000 7ff305c00640 1 -- 172.21.15.16:0/4210464080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fda5588b60 con 0x55fda5126d80 2024-08-31T06:09:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.343+0000 7ff305c00640 1 --2- 172.21.15.16:0/4210464080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fda5126d80 0x55fda552b9f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff2f401a040 tx=0x7ff2f400fca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a7e0de549bf66e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:58.346 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.343+0000 7ff2e7400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:09:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.344+0000 7ff2e7400640 1 -- 172.21.15.16:0/4210464080 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f405c040 con 0x55fda5126d80 2024-08-31T06:09:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.345+0000 7ff30f655740 1 -- 172.21.15.16:0/4210464080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2f4058e50 con 0x55fda5126d80 2024-08-31T06:09:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.345+0000 7ff2e7400640 1 -- 172.21.15.16:0/4210464080 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7ff2f40a83f0 con 0x55fda5126d80 2024-08-31T06:09:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.345+0000 7ff2e7400640 1 -- 172.21.15.16:0/4210464080 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff2f40b1800 con 0x55fda5126d80 2024-08-31T06:09:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.346+0000 7ff30f655740 10 client.16062 fetch_fsmap learned FSMap version 31 2024-08-31T06:09:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.346+0000 7ff30f655740 1 -- 172.21.15.16:0/4210464080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55fda5588b60 con 0x55fda5126d80 2024-08-31T06:09:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.346+0000 7ff2e7400640 1 -- 172.21.15.16:0/4210464080 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2f4058e50 con 0x55fda5126d80 2024-08-31T06:09:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.347+0000 7ff2e7400640 1 -- 172.21.15.16:0/4210464080 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7ff2f40b4c90 con 0x55fda5126d80 2024-08-31T06:09:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.348+0000 7ff30f655740 10 client.16062 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:09:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.348+0000 7ff30f655740 10 client.16062 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:09:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.348+0000 7ff30f655740 20 client.16062 populate_metadata read hostname 'smithi016' 2024-08-31T06:09:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.348+0000 7ff30f655740 1 --2- 172.21.15.16:0/4210464080 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fda51681b0 0x55fda5595560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.348+0000 7ff30f655740 4 client.16062 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:09:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.348+0000 7ff30f655740 1 -- 172.21.15.16:0/4210464080 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fda547e960 con 0x55fda51681b0 2024-08-31T06:09:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.348+0000 7ff305200640 1 --2- 172.21.15.16:0/4210464080 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fda51681b0 0x55fda5595560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.349+0000 7ff305200640 1 --2- 172.21.15.16:0/4210464080 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fda51681b0 0x55fda5595560 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89b092afa81c8e02 server_cookie=ead3a63aaf41bf6b in_seq=0 out_seq=0 2024-08-31T06:09:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.350+0000 7ff2e7400640 10 client.16062 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:09:58.368 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.365+0000 7ff2e7400640 1 -- 172.21.15.16:0/4210464080 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fda547e960 con 0x55fda51681b0 2024-08-31T06:09:58.368 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.366+0000 7ff2e7400640 10 client.16062 handle_command_reply: tid=0 2024-08-31T06:09:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.366+0000 7ff30f655740 1 client.16062 shutdown 2024-08-31T06:09:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.367+0000 7ff30f655740 1 -- 172.21.15.16:0/4210464080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fda5126d80 msgr2=0x55fda552b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.367+0000 7ff30f655740 1 --2- 172.21.15.16:0/4210464080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fda5126d80 0x55fda552b9f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff2f401a040 tx=0x7ff2f400fca0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.367+0000 7ff30f655740 1 -- 172.21.15.16:0/4210464080 shutdown_connections 2024-08-31T06:09:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.367+0000 7ff30f655740 1 --2- 172.21.15.16:0/4210464080 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fda51681b0 0x55fda5595560 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.367+0000 7ff30f655740 1 --2- 172.21.15.16:0/4210464080 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fda513f2f0 0x55fda552bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.367+0000 7ff30f655740 1 --2- 172.21.15.16:0/4210464080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fda51474d0 0x55fda55884f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.367+0000 7ff30f655740 1 --2- 172.21.15.16:0/4210464080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fda5126d80 0x55fda552b9f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.367+0000 7ff30f655740 1 -- 172.21.15.16:0/4210464080 >> 172.21.15.16:0/4210464080 conn(0x55fda515c300 msgr2=0x55fda5522cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.368+0000 7ff2e7400640 0 client.16062 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:09:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.368+0000 7ff30f655740 1 -- 172.21.15.16:0/4210464080 shutdown_connections 2024-08-31T06:09:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.368+0000 7ff30f655740 1 -- 172.21.15.16:0/4210464080 wait complete. 2024-08-31T06:09:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.369+0000 7ff30f655740 20 client.16062 trim_cache size 0 max 16384 2024-08-31T06:09:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:09:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:09:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:09:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:09:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.370+0000 7ff308600640 1 -- 172.21.15.16:0/1497856154 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff2e80b4d20 msgr2=0x7ff2e80b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.370+0000 7ff308600640 1 --2- 172.21.15.16:0/1497856154 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff2e80b4d20 0x7ff2e80b7230 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8005fd0 tx=0x7ff2f8009840 comp rx=0 tx=0).stop 2024-08-31T06:09:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.370+0000 7ff308600640 1 -- 172.21.15.16:0/1497856154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff30010a400 msgr2=0x7ff30011ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.370+0000 7ff308600640 1 --2- 172.21.15.16:0/1497856154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff30010a400 0x7ff30011ecc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff2f400c920 tx=0x7ff2f400cde0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.371+0000 7ff308600640 1 -- 172.21.15.16:0/1497856154 shutdown_connections 2024-08-31T06:09:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.371+0000 7ff308600640 1 --2- 172.21.15.16:0/1497856154 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff2e80b4d20 0x7ff2e80b7230 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.371+0000 7ff308600640 1 --2- 172.21.15.16:0/1497856154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff300110550 0x7ff30011b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.371+0000 7ff308600640 1 --2- 172.21.15.16:0/1497856154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff30010fc00 0x7ff30011af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.371+0000 7ff308600640 1 --2- 172.21.15.16:0/1497856154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff30010a400 0x7ff30011ecc0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.371+0000 7ff308600640 1 -- 172.21.15.16:0/1497856154 >> 172.21.15.16:0/1497856154 conn(0x7ff300070fc0 msgr2=0x7ff300069a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.371+0000 7ff308600640 1 -- 172.21.15.16:0/1497856154 shutdown_connections 2024-08-31T06:09:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:09:58.371+0000 7ff308600640 1 -- 172.21.15.16:0/1497856154 wait complete. 2024-08-31T06:09:58.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.497+0000 7f15e3000640 1 -- 172.21.15.16:0/2500238073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15dc109b90 msgr2=0x7f15dc1103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:58.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.497+0000 7f15e3000640 1 --2- 172.21.15.16:0/2500238073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15dc109b90 0x7f15dc1103c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f15d4009be0 tx=0x7f15d402a6c0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.497+0000 7f15e3000640 1 -- 172.21.15.16:0/2500238073 shutdown_connections 2024-08-31T06:09:58.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.497+0000 7f15e3000640 1 --2- 172.21.15.16:0/2500238073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15dc109240 0x7f15dc109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.497+0000 7f15e3000640 1 --2- 172.21.15.16:0/2500238073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15dc109b90 0x7f15dc1103c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.497+0000 7f15e3000640 1 --2- 172.21.15.16:0/2500238073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15dc103900 0x7f15dc103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.497+0000 7f15e3000640 1 -- 172.21.15.16:0/2500238073 >> 172.21.15.16:0/2500238073 conn(0x7f15dc07b5f0 msgr2=0x7f15dc07ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:58.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.498+0000 7f15e3000640 1 -- 172.21.15.16:0/2500238073 shutdown_connections 2024-08-31T06:09:58.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.498+0000 7f15e3000640 1 -- 172.21.15.16:0/2500238073 wait complete. 2024-08-31T06:09:58.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.500+0000 7f15e3000640 1 Processor -- start 2024-08-31T06:09:58.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.500+0000 7f15e3000640 1 -- start start 2024-08-31T06:09:58.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.501+0000 7f15e3000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15dc103900 0x7f15dc198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:58.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.501+0000 7f15e3000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15dc109240 0x7f15dc199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:58.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.502+0000 7f15e3000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15dc109b90 0x7f15dc19fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:58.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.502+0000 7f15e3000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f15dc113850 con 0x7f15dc109240 2024-08-31T06:09:58.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.502+0000 7f15e1000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15dc109b90 0x7f15dc19fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:58.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.502+0000 7f15e1000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15dc109b90 0x7f15dc19fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:50914/0 (socket says 172.21.15.16:50914) 2024-08-31T06:09:58.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.502+0000 7f15e1000640 1 -- 172.21.15.16:0/910998536 learned_addr learned my addr 172.21.15.16:0/910998536 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:58.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.503+0000 7f15dbe00640 1 --2- 172.21.15.16:0/910998536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15dc103900 0x7f15dc198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:58.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.503+0000 7f15e3000640 1 -- 172.21.15.16:0/910998536 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f15dc1136d0 con 0x7f15dc109b90 2024-08-31T06:09:58.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.503+0000 7f15db400640 1 --2- 172.21.15.16:0/910998536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15dc109240 0x7f15dc199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:58.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.503+0000 7f15e3000640 1 -- 172.21.15.16:0/910998536 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f15dc1139d0 con 0x7f15dc103900 2024-08-31T06:09:58.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.504+0000 7f15e1000640 1 -- 172.21.15.16:0/910998536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15dc103900 msgr2=0x7f15dc198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:58.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.504+0000 7f15e1000640 1 --2- 172.21.15.16:0/910998536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15dc103900 0x7f15dc198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.504+0000 7f15e1000640 1 -- 172.21.15.16:0/910998536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15dc109240 msgr2=0x7f15dc199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:58.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.504+0000 7f15e1000640 1 --2- 172.21.15.16:0/910998536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15dc109240 0x7f15dc199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.504+0000 7f15e1000640 1 -- 172.21.15.16:0/910998536 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15dc199a60 con 0x7f15dc109b90 2024-08-31T06:09:58.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.504+0000 7f15db400640 1 --2- 172.21.15.16:0/910998536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15dc109240 0x7f15dc199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:09:58.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.505+0000 7f15e1000640 1 --2- 172.21.15.16:0/910998536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15dc109b90 0x7f15dc19fd60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f15d4003cc0 tx=0x7f15d4003cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:58.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.505+0000 7f15d8c00640 1 -- 172.21.15.16:0/910998536 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d4040020 con 0x7f15dc109b90 2024-08-31T06:09:58.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.506+0000 7f15d8c00640 1 -- 172.21.15.16:0/910998536 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15d4003e10 con 0x7f15dc109b90 2024-08-31T06:09:58.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.506+0000 7f15e3000640 1 -- 172.21.15.16:0/910998536 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15dc1a02a0 con 0x7f15dc109b90 2024-08-31T06:09:58.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.506+0000 7f15d8c00640 1 -- 172.21.15.16:0/910998536 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d4036040 con 0x7f15dc109b90 2024-08-31T06:09:58.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.506+0000 7f15e3000640 1 -- 172.21.15.16:0/910998536 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15dc1a07d0 con 0x7f15dc109b90 2024-08-31T06:09:58.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.511+0000 7f15d8c00640 1 -- 172.21.15.16:0/910998536 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f15d4043020 con 0x7f15dc109b90 2024-08-31T06:09:58.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.514+0000 7f15d8c00640 1 --2- 172.21.15.16:0/910998536 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f15c00b4dd0 0x7f15c00b72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:58.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.514+0000 7f15e9c47740 1 Processor -- start 2024-08-31T06:09:58.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.515+0000 7f15d8c00640 1 -- 172.21.15.16:0/910998536 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f15d4106170 con 0x7f15dc109b90 2024-08-31T06:09:58.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.515+0000 7f15dbe00640 1 --2- 172.21.15.16:0/910998536 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f15c00b4dd0 0x7f15c00b72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:58.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.515+0000 7f15e9c47740 1 -- start start 2024-08-31T06:09:58.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.516+0000 7f15e9c47740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607c98a4070 0x5607c9c49990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:58.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.516+0000 7f15e9c47740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c987b1b0 0x5607c9c49ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:58.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.516+0000 7f15e9c47740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c9883390 0x5607c9c4a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:58.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.516+0000 7f15e1000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c9883390 0x5607c9c4a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:58.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.516+0000 7f15e1000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c9883390 0x5607c9c4a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:44308/0 (socket says 172.21.15.16:44308) 2024-08-31T06:09:58.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.516+0000 7f15e9c47740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5607c9bc07b0 con 0x5607c987b1b0 2024-08-31T06:09:58.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.516+0000 7f15e9c47740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5607c9bafd40 con 0x5607c98a4070 2024-08-31T06:09:58.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.516+0000 7f15e9c47740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5607c9bb3cf0 con 0x5607c9883390 2024-08-31T06:09:58.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.516+0000 7f15e1000640 1 -- 172.21.15.16:0/2551506423 learned_addr learned my addr 172.21.15.16:0/2551506423 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:58.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.516+0000 7f15dbe00640 1 --2- 172.21.15.16:0/2551506423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607c98a4070 0x5607c9c49990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:58.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.517+0000 7f15db400640 1 --2- 172.21.15.16:0/2551506423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c987b1b0 0x5607c9c49ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:58.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.517+0000 7f15dbe00640 1 -- 172.21.15.16:0/2551506423 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c9883390 msgr2=0x5607c9c4a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:58.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.517+0000 7f15dbe00640 1 --2- 172.21.15.16:0/2551506423 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c9883390 0x5607c9c4a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.517+0000 7f15dbe00640 1 -- 172.21.15.16:0/2551506423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c987b1b0 msgr2=0x5607c9c49ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:58.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.517+0000 7f15dbe00640 1 --2- 172.21.15.16:0/2551506423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c987b1b0 0x5607c9c49ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.517+0000 7f15dbe00640 1 -- 172.21.15.16:0/2551506423 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607c990f650 con 0x5607c98a4070 2024-08-31T06:09:58.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.517+0000 7f15dbe00640 1 --2- 172.21.15.16:0/910998536 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f15c00b4dd0 0x7f15c00b72e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f15dc110470 tx=0x7f15cc00a2d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:58.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.517+0000 7f15e1000640 1 --2- 172.21.15.16:0/2551506423 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c9883390 0x5607c9c4a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:09:58.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.518+0000 7f15dbe00640 1 --2- 172.21.15.16:0/2551506423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607c98a4070 0x5607c9c49990 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f15cc00f850 tx=0x7f15cc01a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b871eebf9e40ef server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:58.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.518+0000 7f15d1600640 1 -- 172.21.15.16:0/2551506423 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc016070 con 0x5607c98a4070 2024-08-31T06:09:58.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.518+0000 7f15d1600640 1 -- 172.21.15.16:0/2551506423 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15cc010cf0 con 0x5607c98a4070 2024-08-31T06:09:58.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.518+0000 7f15d1600640 1 -- 172.21.15.16:0/2551506423 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc023e00 con 0x5607c98a4070 2024-08-31T06:09:58.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.519+0000 7f15e9c47740 1 -- 172.21.15.16:0/2551506423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607c98a4070 msgr2=0x5607c9c49990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:58.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.519+0000 7f15e9c47740 1 --2- 172.21.15.16:0/2551506423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607c98a4070 0x5607c9c49990 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f15cc00f850 tx=0x7f15cc01a000 comp rx=0 tx=0).stop 2024-08-31T06:09:58.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.520+0000 7f15e9c47740 1 -- 172.21.15.16:0/2551506423 shutdown_connections 2024-08-31T06:09:58.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.520+0000 7f15e9c47740 1 --2- 172.21.15.16:0/2551506423 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c9883390 0x5607c9c4a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.520+0000 7f15e9c47740 1 --2- 172.21.15.16:0/2551506423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c987b1b0 0x5607c9c49ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.520+0000 7f15e9c47740 1 --2- 172.21.15.16:0/2551506423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607c98a4070 0x5607c9c49990 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.520+0000 7f15e9c47740 1 -- 172.21.15.16:0/2551506423 >> 172.21.15.16:0/2551506423 conn(0x5607c9898460 msgr2=0x5607c9c47880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:58.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.520+0000 7f15e9c47740 1 -- 172.21.15.16:0/2551506423 shutdown_connections 2024-08-31T06:09:58.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.520+0000 7f15e9c47740 1 -- 172.21.15.16:0/2551506423 wait complete. 2024-08-31T06:09:58.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.521+0000 7f15e9c47740 1 -- start start 2024-08-31T06:09:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.521+0000 7f15e9c47740 1 Processor -- start 2024-08-31T06:09:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.522+0000 7f15e9c47740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c987b1b0 0x5607c9c55640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.522+0000 7f15e9c47740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607c9883390 0x5607c9c55c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.522+0000 7f15e9c47740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c98a4070 0x5607c9c61460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.522+0000 7f15e9c47740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5607c9a12700 con 0x5607c987b1b0 2024-08-31T06:09:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.522+0000 7f15e9c47740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5607c9b36cf0 con 0x5607c9883390 2024-08-31T06:09:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.522+0000 7f15e9c47740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5607c9c58170 con 0x5607c98a4070 2024-08-31T06:09:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.522+0000 7f15e9c47740 10 client.0 _ll_register_callbacks cb 0x5607c9b5c470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:09:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.522+0000 7f15dbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c987b1b0 0x5607c9c55640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.522+0000 7f15db400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607c9883390 0x5607c9c55c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.522+0000 7f15db400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607c9883390 0x5607c9c55c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:50928/0 (socket says 172.21.15.16:50928) 2024-08-31T06:09:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.522+0000 7f15db400640 1 -- 172.21.15.16:0/774545941 learned_addr learned my addr 172.21.15.16:0/774545941 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:58.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.522+0000 7f15e1000640 1 --2- 172.21.15.16:0/774545941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c98a4070 0x5607c9c61460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:58.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.523+0000 7f15e1000640 1 -- 172.21.15.16:0/774545941 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607c9883390 msgr2=0x5607c9c55c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:58.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.523+0000 7f15e1000640 1 --2- 172.21.15.16:0/774545941 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607c9883390 0x5607c9c55c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.523+0000 7f15e1000640 1 -- 172.21.15.16:0/774545941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c987b1b0 msgr2=0x5607c9c55640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:58.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.523+0000 7f15e1000640 1 --2- 172.21.15.16:0/774545941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c987b1b0 0x5607c9c55640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.523+0000 7f15e1000640 1 -- 172.21.15.16:0/774545941 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607c9b778b0 con 0x5607c98a4070 2024-08-31T06:09:58.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.524+0000 7f15e1000640 1 --2- 172.21.15.16:0/774545941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c98a4070 0x5607c9c61460 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f15d40edfd0 tx=0x7f15d40cc430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=caebad1541b527b4 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:58.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.524+0000 7f15c6000640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:09:58.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.524+0000 7f15c6000640 1 -- 172.21.15.16:0/774545941 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d40d10d0 con 0x5607c98a4070 2024-08-31T06:09:58.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.524+0000 7f15c6000640 1 -- 172.21.15.16:0/774545941 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f15d4103b40 con 0x5607c98a4070 2024-08-31T06:09:58.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.524+0000 7f15e9c47740 1 -- 172.21.15.16:0/774545941 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15cc010cf0 con 0x5607c98a4070 2024-08-31T06:09:58.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.525+0000 7f15c6000640 1 -- 172.21.15.16:0/774545941 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f15d40e3070 con 0x5607c98a4070 2024-08-31T06:09:58.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.525+0000 7f15e9c47740 10 client.26173 fetch_fsmap learned FSMap version 31 2024-08-31T06:09:58.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.525+0000 7f15e9c47740 1 -- 172.21.15.16:0/774545941 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5607c9c64210 con 0x5607c98a4070 2024-08-31T06:09:58.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.525+0000 7f15c6000640 1 -- 172.21.15.16:0/774545941 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15cc010cf0 con 0x5607c98a4070 2024-08-31T06:09:58.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.525+0000 7f15c6000640 1 -- 172.21.15.16:0/774545941 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f15d4088190 con 0x5607c98a4070 2024-08-31T06:09:58.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.525+0000 7f15e9c47740 10 client.26173 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:09:58.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.525+0000 7f15e9c47740 10 client.26173 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-31T06:09:58.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.525+0000 7f15e9c47740 20 client.26173 populate_metadata read hostname 'smithi016' 2024-08-31T06:09:58.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.525+0000 7f15e9c47740 1 --2- 172.21.15.16:0/774545941 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x5607c98629f0 0x5607c9c677d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:58.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.525+0000 7f15e9c47740 4 client.26173 mds_command: new command op to 24724 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:09:58.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.525+0000 7f15e9c47740 1 -- 172.21.15.16:0/774545941 --> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607c9bc2700 con 0x5607c98629f0 2024-08-31T06:09:58.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.526+0000 7f15dbe00640 1 --2- 172.21.15.16:0/774545941 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x5607c98629f0 0x5607c9c677d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:58.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.527+0000 7f15dbe00640 1 --2- 172.21.15.16:0/774545941 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x5607c98629f0 0x5607c9c677d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c24f4316ba413fce server_cookie=b60f3eb46d81459d in_seq=0 out_seq=0 2024-08-31T06:09:58.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.527+0000 7f15c6000640 10 client.26173 ms_handle_connect on v2:172.21.15.16:6838/219664460 2024-08-31T06:09:58.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.529+0000 7f15c6000640 1 -- 172.21.15.16:0/774545941 <== mds.1 v2:172.21.15.16:6838/219664460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5607c9bc2700 con 0x5607c98629f0 2024-08-31T06:09:58.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.529+0000 7f15c6000640 10 client.26173 handle_command_reply: tid=0 2024-08-31T06:09:58.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.529+0000 7f15e9c47740 1 client.26173 shutdown 2024-08-31T06:09:58.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.530+0000 7f15e9c47740 1 -- 172.21.15.16:0/774545941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c98a4070 msgr2=0x5607c9c61460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:58.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.530+0000 7f15e9c47740 1 --2- 172.21.15.16:0/774545941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c98a4070 0x5607c9c61460 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f15d40edfd0 tx=0x7f15d40cc430 comp rx=0 tx=0).stop 2024-08-31T06:09:58.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.530+0000 7f15e9c47740 1 -- 172.21.15.16:0/774545941 shutdown_connections 2024-08-31T06:09:58.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.530+0000 7f15e9c47740 1 --2- 172.21.15.16:0/774545941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c98a4070 0x5607c9c61460 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.530+0000 7f15e9c47740 1 --2- 172.21.15.16:0/774545941 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x5607c98629f0 0x5607c9c677d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.530+0000 7f15e9c47740 1 --2- 172.21.15.16:0/774545941 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607c9883390 0x5607c9c55c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.530+0000 7f15e9c47740 1 --2- 172.21.15.16:0/774545941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c987b1b0 0x5607c9c55640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.530+0000 7f15e9c47740 1 -- 172.21.15.16:0/774545941 >> 172.21.15.16:0/774545941 conn(0x5607c9898460 msgr2=0x5607c9c4cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:58.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.530+0000 7f15c6000640 0 client.26173 ms_handle_reset on v2:172.21.15.16:6838/219664460 2024-08-31T06:09:58.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.530+0000 7f15e9c47740 1 -- 172.21.15.16:0/774545941 shutdown_connections 2024-08-31T06:09:58.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.530+0000 7f15e9c47740 1 -- 172.21.15.16:0/774545941 wait complete. 2024-08-31T06:09:58.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.530+0000 7f15e9c47740 20 client.26173 trim_cache size 0 max 16384 2024-08-31T06:09:58.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.558+0000 7f15e9c47740 1 Processor -- start 2024-08-31T06:09:58.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.559+0000 7f15e9c47740 1 -- start start 2024-08-31T06:09:58.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.559+0000 7f15e9c47740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607c987b1b0 0x5607c9c478f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:58.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.559+0000 7f15e9c47740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c9883390 0x5607c9c4eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:58.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.559+0000 7f15e9c47740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c98a4070 0x5607c9c4efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:58.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.559+0000 7f15e9c47740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f15dc1c0b80 con 0x5607c9883390 2024-08-31T06:09:58.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.559+0000 7f15e9c47740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f15dc1be390 con 0x5607c987b1b0 2024-08-31T06:09:58.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.559+0000 7f15e9c47740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f15dc1bc1a0 con 0x5607c98a4070 2024-08-31T06:09:58.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.560+0000 7f15dbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607c987b1b0 0x5607c9c478f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:58.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.560+0000 7f15dbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607c987b1b0 0x5607c9c478f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:50944/0 (socket says 172.21.15.16:50944) 2024-08-31T06:09:58.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.560+0000 7f15dbe00640 1 -- 172.21.15.16:0/2083231723 learned_addr learned my addr 172.21.15.16:0/2083231723 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:58.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.560+0000 7f15dbe00640 1 -- 172.21.15.16:0/2083231723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c98a4070 msgr2=0x5607c9c4efe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:09:58.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.560+0000 7f15dbe00640 1 --2- 172.21.15.16:0/2083231723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c98a4070 0x5607c9c4efe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.560+0000 7f15dbe00640 1 -- 172.21.15.16:0/2083231723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c9883390 msgr2=0x5607c9c4eaa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:09:58.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.561+0000 7f15dbe00640 1 --2- 172.21.15.16:0/2083231723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c9883390 0x5607c9c4eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.561+0000 7f15dbe00640 1 -- 172.21.15.16:0/2083231723 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607c9c56700 con 0x5607c987b1b0 2024-08-31T06:09:58.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.561+0000 7f15dbe00640 1 --2- 172.21.15.16:0/2083231723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607c987b1b0 0x5607c9c478f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f15cc00fd20 tx=0x7f15cc0238c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c4296fb9e84b29d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:58.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.562+0000 7f15d0c00640 1 -- 172.21.15.16:0/2083231723 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc03b070 con 0x5607c987b1b0 2024-08-31T06:09:58.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.562+0000 7f15d0c00640 1 -- 172.21.15.16:0/2083231723 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15cc023e50 con 0x5607c987b1b0 2024-08-31T06:09:58.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.562+0000 7f15d0c00640 1 -- 172.21.15.16:0/2083231723 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc016040 con 0x5607c987b1b0 2024-08-31T06:09:58.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.563+0000 7f15e9c47740 1 -- 172.21.15.16:0/2083231723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607c987b1b0 msgr2=0x5607c9c478f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:58.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.563+0000 7f15e9c47740 1 --2- 172.21.15.16:0/2083231723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607c987b1b0 0x5607c9c478f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f15cc00fd20 tx=0x7f15cc0238c0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.563+0000 7f15e9c47740 1 -- 172.21.15.16:0/2083231723 shutdown_connections 2024-08-31T06:09:58.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.563+0000 7f15e9c47740 1 --2- 172.21.15.16:0/2083231723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c98a4070 0x5607c9c4efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.563+0000 7f15e9c47740 1 --2- 172.21.15.16:0/2083231723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c9883390 0x5607c9c4eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.563+0000 7f15e9c47740 1 --2- 172.21.15.16:0/2083231723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607c987b1b0 0x5607c9c478f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.563+0000 7f15e9c47740 1 -- 172.21.15.16:0/2083231723 >> 172.21.15.16:0/2083231723 conn(0x5607c9898460 msgr2=0x5607c9c471e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:58.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.564+0000 7f15e9c47740 1 -- 172.21.15.16:0/2083231723 shutdown_connections 2024-08-31T06:09:58.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.564+0000 7f15e9c47740 1 -- 172.21.15.16:0/2083231723 wait complete. 2024-08-31T06:09:58.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.564+0000 7f15e9c47740 1 -- start start 2024-08-31T06:09:58.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.565+0000 7f15e9c47740 1 Processor -- start 2024-08-31T06:09:58.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.565+0000 7f15e9c47740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c987b1b0 0x5607c9c49990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:58.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.565+0000 7f15e9c47740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607c9883390 0x5607c9c49ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:58.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.565+0000 7f15e9c47740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c98a4070 0x5607c9c4a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:58.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.565+0000 7f15e9c47740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f15dc1bc850 con 0x5607c98a4070 2024-08-31T06:09:58.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.565+0000 7f15e9c47740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f15dc1bdcc0 con 0x5607c9883390 2024-08-31T06:09:58.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.565+0000 7f15e9c47740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f15dc1bcf20 con 0x5607c987b1b0 2024-08-31T06:09:58.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.565+0000 7f15e9c47740 10 client.0 _ll_register_callbacks cb 0x7f15d40c8070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:09:58.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.566+0000 7f15db400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607c9883390 0x5607c9c49ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:58.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.566+0000 7f15dbe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c987b1b0 0x5607c9c49990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:58.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.566+0000 7f15dbe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c987b1b0 0x5607c9c49990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:44332/0 (socket says 172.21.15.16:44332) 2024-08-31T06:09:58.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.566+0000 7f15db400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607c9883390 0x5607c9c49ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:50950/0 (socket says 172.21.15.16:50950) 2024-08-31T06:09:58.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.566+0000 7f15dbe00640 1 -- 172.21.15.16:0/3716644120 learned_addr learned my addr 172.21.15.16:0/3716644120 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:09:58.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.566+0000 7f15db400640 1 -- 172.21.15.16:0/3716644120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c987b1b0 msgr2=0x5607c9c49990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:58.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.567+0000 7f15db400640 1 --2- 172.21.15.16:0/3716644120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c987b1b0 0x5607c9c49990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.567+0000 7f15db400640 1 -- 172.21.15.16:0/3716644120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c98a4070 msgr2=0x5607c9c4a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:09:58.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.567+0000 7f15e1000640 1 --2- 172.21.15.16:0/3716644120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c98a4070 0x5607c9c4a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:58.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.568+0000 7f15db400640 1 --2- 172.21.15.16:0/3716644120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c98a4070 0x5607c9c4a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.568+0000 7f15db400640 1 -- 172.21.15.16:0/3716644120 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607c9c5cab0 con 0x5607c9883390 2024-08-31T06:09:58.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.568+0000 7f15e1000640 1 --2- 172.21.15.16:0/3716644120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c98a4070 0x5607c9c4a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:09:58.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.568+0000 7f15db400640 1 --2- 172.21.15.16:0/3716644120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607c9883390 0x5607c9c49ed0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f15c8005a70 tx=0x7f15c8013c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=150f40fee4ab9df7 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:09:58.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.568+0000 7f15c6000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:09:58.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.569+0000 7f15c6000640 1 -- 172.21.15.16:0/3716644120 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c8004240 con 0x5607c9883390 2024-08-31T06:09:58.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.569+0000 7f15c6000640 1 -- 172.21.15.16:0/3716644120 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f15c80043e0 con 0x5607c9883390 2024-08-31T06:09:58.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.569+0000 7f15e9c47740 1 -- 172.21.15.16:0/3716644120 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15cc023e50 con 0x5607c9883390 2024-08-31T06:09:58.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.570+0000 7f15c6000640 1 -- 172.21.15.16:0/3716644120 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f15c80118f0 con 0x5607c9883390 2024-08-31T06:09:58.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.570+0000 7f15e9c47740 10 client.26177 fetch_fsmap learned FSMap version 31 2024-08-31T06:09:58.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.570+0000 7f15e9c47740 1 -- 172.21.15.16:0/3716644120 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5607c9cd2450 con 0x5607c9883390 2024-08-31T06:09:58.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.570+0000 7f15c6000640 1 -- 172.21.15.16:0/3716644120 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15cc023e50 con 0x5607c9883390 2024-08-31T06:09:58.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.570+0000 7f15c6000640 1 -- 172.21.15.16:0/3716644120 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f15c801d420 con 0x5607c9883390 2024-08-31T06:09:58.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.571+0000 7f15e9c47740 10 client.26177 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:09:58.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.571+0000 7f15e9c47740 10 client.26177 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-31T06:09:58.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.571+0000 7f15e9c47740 20 client.26177 populate_metadata read hostname 'smithi016' 2024-08-31T06:09:58.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.571+0000 7f15e9c47740 1 --2- 172.21.15.16:0/3716644120 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x5607c9837da0 0x5607c9cd4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:09:58.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.571+0000 7f15e9c47740 4 client.26177 mds_command: new command op to 24724 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:09:58.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.571+0000 7f15e9c47740 1 -- 172.21.15.16:0/3716644120 --> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5607c9bbe1a0 con 0x5607c9837da0 2024-08-31T06:09:58.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.571+0000 7f15dbe00640 1 --2- 172.21.15.16:0/3716644120 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x5607c9837da0 0x5607c9cd4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:09:58.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.572+0000 7f15dbe00640 1 --2- 172.21.15.16:0/3716644120 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x5607c9837da0 0x5607c9cd4cf0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9c2987fd649a7381 server_cookie=e0844f0d9ab67ad6 in_seq=0 out_seq=0 2024-08-31T06:09:58.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.572+0000 7f15c6000640 10 client.26177 ms_handle_connect on v2:172.21.15.16:6838/219664460 2024-08-31T06:09:58.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.572+0000 7f15c6000640 1 -- 172.21.15.16:0/3716644120 <== mds.1 v2:172.21.15.16:6838/219664460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5607c9bbe1a0 con 0x5607c9837da0 2024-08-31T06:09:58.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.572+0000 7f15c6000640 10 client.26177 handle_command_reply: tid=0 2024-08-31T06:09:58.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.572+0000 7f15e9c47740 1 client.26177 shutdown 2024-08-31T06:09:58.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.573+0000 7f15e9c47740 1 -- 172.21.15.16:0/3716644120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607c9883390 msgr2=0x5607c9c49ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:58.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.573+0000 7f15e9c47740 1 --2- 172.21.15.16:0/3716644120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607c9883390 0x5607c9c49ed0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f15c8005a70 tx=0x7f15c8013c00 comp rx=0 tx=0).stop 2024-08-31T06:09:58.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.573+0000 7f15e9c47740 1 -- 172.21.15.16:0/3716644120 shutdown_connections 2024-08-31T06:09:58.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.573+0000 7f15e9c47740 1 --2- 172.21.15.16:0/3716644120 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x5607c9837da0 0x5607c9cd4cf0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.573+0000 7f15e9c47740 1 --2- 172.21.15.16:0/3716644120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c98a4070 0x5607c9c4a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.573+0000 7f15e9c47740 1 --2- 172.21.15.16:0/3716644120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607c9883390 0x5607c9c49ed0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.573+0000 7f15e9c47740 1 --2- 172.21.15.16:0/3716644120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c987b1b0 0x5607c9c49990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.573+0000 7f15e9c47740 1 -- 172.21.15.16:0/3716644120 >> 172.21.15.16:0/3716644120 conn(0x5607c9898460 msgr2=0x5607c9c4a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:58.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.573+0000 7f15c6000640 0 client.26177 ms_handle_reset on v2:172.21.15.16:6838/219664460 2024-08-31T06:09:58.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.574+0000 7f15e9c47740 1 -- 172.21.15.16:0/3716644120 shutdown_connections 2024-08-31T06:09:58.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.574+0000 7f15e9c47740 1 -- 172.21.15.16:0/3716644120 wait complete. 2024-08-31T06:09:58.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.574+0000 7f15e9c47740 20 client.26177 trim_cache size 0 max 16384 2024-08-31T06:09:58.577 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:09:58.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.575+0000 7f15e3000640 1 -- 172.21.15.16:0/910998536 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f15c00b4dd0 msgr2=0x7f15c00b72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:58.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.575+0000 7f15e3000640 1 --2- 172.21.15.16:0/910998536 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f15c00b4dd0 0x7f15c00b72e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f15dc110470 tx=0x7f15cc00a2d0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.575+0000 7f15e3000640 1 -- 172.21.15.16:0/910998536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15dc109b90 msgr2=0x7f15dc19fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:09:58.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.575+0000 7f15e3000640 1 --2- 172.21.15.16:0/910998536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15dc109b90 0x7f15dc19fd60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f15d4003cc0 tx=0x7f15d4003cf0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.575+0000 7f15e3000640 1 -- 172.21.15.16:0/910998536 shutdown_connections 2024-08-31T06:09:58.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.575+0000 7f15e3000640 1 --2- 172.21.15.16:0/910998536 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f15c00b4dd0 0x7f15c00b72e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.575+0000 7f15e3000640 1 --2- 172.21.15.16:0/910998536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15dc109b90 0x7f15dc19fd60 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.575+0000 7f15e3000640 1 --2- 172.21.15.16:0/910998536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15dc109240 0x7f15dc199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.575+0000 7f15e3000640 1 --2- 172.21.15.16:0/910998536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15dc103900 0x7f15dc198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:09:58.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.575+0000 7f15e3000640 1 -- 172.21.15.16:0/910998536 >> 172.21.15.16:0/910998536 conn(0x7f15dc07b5f0 msgr2=0x7f15dc1070f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:09:58.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.576+0000 7f15e3000640 1 -- 172.21.15.16:0/910998536 shutdown_connections 2024-08-31T06:09:58.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:09:58.576+0000 7f15e3000640 1 -- 172.21.15.16:0/910998536 wait complete. 2024-08-31T06:09:59.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:09:59 smithi016 ceph-mon[17194]: pgmap v446: 97 pgs: 97 active+clean; 1.6 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.4 MiB/s wr, 652 op/s 2024-08-31T06:09:59.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:09:59 smithi184 ceph-mon[22331]: pgmap v446: 97 pgs: 97 active+clean; 1.6 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.4 MiB/s wr, 652 op/s 2024-08-31T06:09:59.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:09:59 smithi155 ceph-mon[21742]: pgmap v446: 97 pgs: 97 active+clean; 1.6 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.4 MiB/s wr, 652 op/s 2024-08-31T06:10:00.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:00 smithi016 ceph-mon[17194]: overall HEALTH_OK 2024-08-31T06:10:00.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:00 smithi184 ceph-mon[22331]: overall HEALTH_OK 2024-08-31T06:10:00.621 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:00 smithi155 ceph-mon[21742]: overall HEALTH_OK 2024-08-31T06:10:01.064 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T06:10:01.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:01 smithi016 ceph-mon[17194]: pgmap v447: 97 pgs: 97 active+clean; 1.6 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.7 MiB/s wr, 716 op/s 2024-08-31T06:10:01.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:01 smithi184 ceph-mon[22331]: pgmap v447: 97 pgs: 97 active+clean; 1.6 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.7 MiB/s wr, 716 op/s 2024-08-31T06:10:01.525 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:10:01.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:01 smithi155 ceph-mon[21742]: pgmap v447: 97 pgs: 97 active+clean; 1.6 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.7 MiB/s wr, 716 op/s 2024-08-31T06:10:01.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 155.0 s, 2180.0 tps, lat 44.493 ms stddev 62.089 2024-08-31T06:10:02.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.258+0000 7fc574e00640 1 -- 172.21.15.16:0/3511117154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc570109250 msgr2=0x7fc570109660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:02.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.258+0000 7fc574e00640 1 --2- 172.21.15.16:0/3511117154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc570109250 0x7fc570109660 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc55c009900 tx=0x7fc55c02a410 comp rx=0 tx=0).stop 2024-08-31T06:10:02.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.258+0000 7fc574e00640 1 -- 172.21.15.16:0/3511117154 shutdown_connections 2024-08-31T06:10:02.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.258+0000 7fc574e00640 1 --2- 172.21.15.16:0/3511117154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc570109ba0 0x7fc5701103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:02.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.258+0000 7fc574e00640 1 --2- 172.21.15.16:0/3511117154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc570109250 0x7fc570109660 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:02.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.258+0000 7fc574e00640 1 --2- 172.21.15.16:0/3511117154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc570103910 0x7fc570103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:02.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.258+0000 7fc574e00640 1 -- 172.21.15.16:0/3511117154 >> 172.21.15.16:0/3511117154 conn(0x7fc5700ff380 msgr2=0x7fc5701017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:02.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.259+0000 7fc574e00640 1 -- 172.21.15.16:0/3511117154 shutdown_connections 2024-08-31T06:10:02.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.259+0000 7fc574e00640 1 -- 172.21.15.16:0/3511117154 wait complete. 2024-08-31T06:10:02.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.260+0000 7fc574e00640 1 Processor -- start 2024-08-31T06:10:02.263 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:02 smithi016 ceph-mon[17194]: pgmap v448: 97 pgs: 97 active+clean; 1.6 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.1 MiB/s wr, 678 op/s 2024-08-31T06:10:02.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.260+0000 7fc574e00640 1 -- start start 2024-08-31T06:10:02.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.260+0000 7fc574e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc570103910 0x7fc57019c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:02.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.260+0000 7fc574e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc570109250 0x7fc57019caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:02.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.261+0000 7fc574e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc570109ba0 0x7fc5701a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:02.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.261+0000 7fc574e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc5701138a0 con 0x7fc570109ba0 2024-08-31T06:10:02.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.261+0000 7fc574e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc570113720 con 0x7fc570109250 2024-08-31T06:10:02.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.261+0000 7fc574e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc570113a20 con 0x7fc570103910 2024-08-31T06:10:02.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.261+0000 7fc56d400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc570109250 0x7fc57019caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:02.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.261+0000 7fc56de00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc570103910 0x7fc57019c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:02.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.261+0000 7fc56d400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc570109250 0x7fc57019caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:50974/0 (socket says 172.21.15.16:50974) 2024-08-31T06:10:02.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.261+0000 7fc56d400640 1 -- 172.21.15.16:0/3798375036 learned_addr learned my addr 172.21.15.16:0/3798375036 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:02.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.261+0000 7fc56d400640 1 -- 172.21.15.16:0/3798375036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc570103910 msgr2=0x7fc57019c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:02.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.261+0000 7fc56d400640 1 --2- 172.21.15.16:0/3798375036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc570103910 0x7fc57019c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:02.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.261+0000 7fc56d400640 1 -- 172.21.15.16:0/3798375036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc570109ba0 msgr2=0x7fc5701a1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:02.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.261+0000 7fc56d400640 1 --2- 172.21.15.16:0/3798375036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc570109ba0 0x7fc5701a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:02.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.261+0000 7fc56d400640 1 -- 172.21.15.16:0/3798375036 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5701a24c0 con 0x7fc570109250 2024-08-31T06:10:02.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.262+0000 7fc56d400640 1 --2- 172.21.15.16:0/3798375036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc570109250 0x7fc57019caf0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc55c02a930 tx=0x7fc55c004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:02.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.263+0000 7fc566a00640 1 -- 172.21.15.16:0/3798375036 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc55c03f030 con 0x7fc570109250 2024-08-31T06:10:02.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.263+0000 7fc574e00640 1 -- 172.21.15.16:0/3798375036 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5701a2750 con 0x7fc570109250 2024-08-31T06:10:02.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.263+0000 7fc566a00640 1 -- 172.21.15.16:0/3798375036 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc55c003aa0 con 0x7fc570109250 2024-08-31T06:10:02.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.263+0000 7fc566a00640 1 -- 172.21.15.16:0/3798375036 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc55c035070 con 0x7fc570109250 2024-08-31T06:10:02.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.263+0000 7fc574e00640 1 -- 172.21.15.16:0/3798375036 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5701a8f10 con 0x7fc570109250 2024-08-31T06:10:02.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.268+0000 7fc566a00640 1 -- 172.21.15.16:0/3798375036 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fc55c042030 con 0x7fc570109250 2024-08-31T06:10:02.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.268+0000 7fc57bbd5740 1 Processor -- start 2024-08-31T06:10:02.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.269+0000 7fc57bbd5740 1 -- start start 2024-08-31T06:10:02.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.269+0000 7fc57bbd5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac3bfcc070 0x55ac3c371990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:02.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.269+0000 7fc57bbd5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac3bfa31b0 0x55ac3c371ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:02.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.269+0000 7fc57bbd5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac3bfab390 0x55ac3c372410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:02.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.269+0000 7fc57bbd5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ac3c2e87b0 con 0x55ac3bfa31b0 2024-08-31T06:10:02.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.269+0000 7fc57bbd5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ac3c2d7d40 con 0x55ac3bfcc070 2024-08-31T06:10:02.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.269+0000 7fc57bbd5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ac3c2dbcf0 con 0x55ac3bfab390 2024-08-31T06:10:02.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.269+0000 7fc56e800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac3bfcc070 0x55ac3c371990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:02.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.269+0000 7fc56de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac3bfa31b0 0x55ac3c371ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:02.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.269+0000 7fc56e800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac3bfcc070 0x55ac3c371990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:50990/0 (socket says 172.21.15.16:50990) 2024-08-31T06:10:02.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.269+0000 7fc56e800640 1 -- 172.21.15.16:0/1483090836 learned_addr learned my addr 172.21.15.16:0/1483090836 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:02.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.269+0000 7fc56e800640 1 -- 172.21.15.16:0/1483090836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac3bfab390 msgr2=0x55ac3c372410 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T06:10:02.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.269+0000 7fc56e800640 1 --2- 172.21.15.16:0/1483090836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac3bfab390 0x55ac3c372410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:02.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.269+0000 7fc56e800640 1 -- 172.21.15.16:0/1483090836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac3bfa31b0 msgr2=0x55ac3c371ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:02.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.269+0000 7fc56e800640 1 --2- 172.21.15.16:0/1483090836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac3bfa31b0 0x55ac3c371ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:02.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.269+0000 7fc56e800640 1 -- 172.21.15.16:0/1483090836 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac3c037650 con 0x55ac3bfcc070 2024-08-31T06:10:02.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.269+0000 7fc56de00640 1 --2- 172.21.15.16:0/1483090836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac3bfa31b0 0x55ac3c371ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:10:02.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.269+0000 7fc56e800640 1 --2- 172.21.15.16:0/1483090836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac3bfcc070 0x55ac3c371990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc568009d80 tx=0x7fc56800e6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d87b092a8944d454 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:02.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.269+0000 7fc55b400640 1 -- 172.21.15.16:0/1483090836 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc568016070 con 0x55ac3bfcc070 2024-08-31T06:10:02.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.269+0000 7fc55b400640 1 -- 172.21.15.16:0/1483090836 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc568005020 con 0x55ac3bfcc070 2024-08-31T06:10:02.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.269+0000 7fc55b400640 1 -- 172.21.15.16:0/1483090836 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5680116d0 con 0x55ac3bfcc070 2024-08-31T06:10:02.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.269+0000 7fc57bbd5740 1 -- 172.21.15.16:0/1483090836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac3bfcc070 msgr2=0x55ac3c371990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:02.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.269+0000 7fc57bbd5740 1 --2- 172.21.15.16:0/1483090836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac3bfcc070 0x55ac3c371990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc568009d80 tx=0x7fc56800e6c0 comp rx=0 tx=0).stop 2024-08-31T06:10:02.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.269+0000 7fc57bbd5740 1 -- 172.21.15.16:0/1483090836 shutdown_connections 2024-08-31T06:10:02.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.269+0000 7fc57bbd5740 1 --2- 172.21.15.16:0/1483090836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac3bfab390 0x55ac3c372410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:02.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.269+0000 7fc57bbd5740 1 --2- 172.21.15.16:0/1483090836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac3bfa31b0 0x55ac3c371ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:02.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.269+0000 7fc57bbd5740 1 --2- 172.21.15.16:0/1483090836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac3bfcc070 0x55ac3c371990 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:02.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.269+0000 7fc57bbd5740 1 -- 172.21.15.16:0/1483090836 >> 172.21.15.16:0/1483090836 conn(0x55ac3bfc0460 msgr2=0x55ac3c36f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:02.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.269+0000 7fc566a00640 1 --2- 172.21.15.16:0/3798375036 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc5540b4d20 0x7fc5540b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:02.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.269+0000 7fc566a00640 1 -- 172.21.15.16:0/3798375036 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fc55c105840 con 0x7fc570109250 2024-08-31T06:10:02.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.269+0000 7fc57bbd5740 1 -- 172.21.15.16:0/1483090836 shutdown_connections 2024-08-31T06:10:02.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.269+0000 7fc57bbd5740 1 -- 172.21.15.16:0/1483090836 wait complete. 2024-08-31T06:10:02.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.269+0000 7fc56de00640 1 --2- 172.21.15.16:0/3798375036 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc5540b4d20 0x7fc5540b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:02.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.275+0000 7fc57bbd5740 1 -- start start 2024-08-31T06:10:02.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.275+0000 7fc57bbd5740 1 Processor -- start 2024-08-31T06:10:02.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.275+0000 7fc56de00640 1 --2- 172.21.15.16:0/3798375036 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc5540b4d20 0x7fc5540b7230 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc560006fd0 tx=0x7fc5600090e0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:02.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.276+0000 7fc57bbd5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac3bfa31b0 0x55ac3c385d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:02.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.276+0000 7fc57bbd5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac3bfab390 0x55ac3c386260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:02.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.276+0000 7fc57bbd5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac3bfcc070 0x55ac3c391c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:02.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.277+0000 7fc56de00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac3bfa31b0 0x55ac3c385d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:02.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.277+0000 7fc56de00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac3bfa31b0 0x55ac3c385d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:44388/0 (socket says 172.21.15.16:44388) 2024-08-31T06:10:02.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.277+0000 7fc56de00640 1 -- 172.21.15.16:0/3058506262 learned_addr learned my addr 172.21.15.16:0/3058506262 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:02.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.277+0000 7fc56d400640 1 --2- 172.21.15.16:0/3058506262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac3bfab390 0x55ac3c386260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:02.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.277+0000 7fc56e800640 1 --2- 172.21.15.16:0/3058506262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac3bfcc070 0x55ac3c391c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:02.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.277+0000 7fc57bbd5740 1 -- 172.21.15.16:0/3058506262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ac3c13a700 con 0x55ac3bfcc070 2024-08-31T06:10:02.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.277+0000 7fc57bbd5740 1 -- 172.21.15.16:0/3058506262 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ac3c25ecf0 con 0x55ac3bfab390 2024-08-31T06:10:02.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.277+0000 7fc57bbd5740 1 -- 172.21.15.16:0/3058506262 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ac3c3888a0 con 0x55ac3bfa31b0 2024-08-31T06:10:02.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.278+0000 7fc57bbd5740 10 client.0 _ll_register_callbacks cb 0x55ac3c284470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:10:02.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.278+0000 7fc56de00640 1 -- 172.21.15.16:0/3058506262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac3bfab390 msgr2=0x55ac3c386260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:02.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.278+0000 7fc56de00640 1 --2- 172.21.15.16:0/3058506262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac3bfab390 0x55ac3c386260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:02.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.278+0000 7fc56de00640 1 -- 172.21.15.16:0/3058506262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac3bfcc070 msgr2=0x55ac3c391c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:02.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.278+0000 7fc56de00640 1 --2- 172.21.15.16:0/3058506262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac3bfcc070 0x55ac3c391c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:02.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.278+0000 7fc56de00640 1 -- 172.21.15.16:0/3058506262 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac3c29f8b0 con 0x55ac3bfa31b0 2024-08-31T06:10:02.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.278+0000 7fc56de00640 1 --2- 172.21.15.16:0/3058506262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac3bfa31b0 0x55ac3c385d20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc5600147e0 tx=0x7fc5600046e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89ce549feaf4af26 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:02.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.278+0000 7fc54fe00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:10:02.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.279+0000 7fc54fe00640 1 -- 172.21.15.16:0/3058506262 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc560025070 con 0x55ac3bfa31b0 2024-08-31T06:10:02.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.279+0000 7fc54fe00640 1 -- 172.21.15.16:0/3058506262 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fc560021600 con 0x55ac3bfa31b0 2024-08-31T06:10:02.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.279+0000 7fc57bbd5740 1 -- 172.21.15.16:0/3058506262 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc568005020 con 0x55ac3bfa31b0 2024-08-31T06:10:02.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.279+0000 7fc54fe00640 1 -- 172.21.15.16:0/3058506262 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc560021800 con 0x55ac3bfa31b0 2024-08-31T06:10:02.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.279+0000 7fc57bbd5740 10 client.26185 fetch_fsmap learned FSMap version 31 2024-08-31T06:10:02.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.279+0000 7fc57bbd5740 1 -- 172.21.15.16:0/3058506262 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ac3c394930 con 0x55ac3bfa31b0 2024-08-31T06:10:02.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.279+0000 7fc54fe00640 1 -- 172.21.15.16:0/3058506262 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc568005020 con 0x55ac3bfa31b0 2024-08-31T06:10:02.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.279+0000 7fc54fe00640 1 -- 172.21.15.16:0/3058506262 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fc560020070 con 0x55ac3bfa31b0 2024-08-31T06:10:02.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.279+0000 7fc57bbd5740 10 client.26185 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:10:02.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.279+0000 7fc57bbd5740 10 client.26185 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:10:02.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.279+0000 7fc57bbd5740 20 client.26185 populate_metadata read hostname 'smithi016' 2024-08-31T06:10:02.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.279+0000 7fc57bbd5740 1 --2- 172.21.15.16:0/3058506262 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55ac3bf8a9f0 0x55ac3c397ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:02.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.279+0000 7fc57bbd5740 4 client.26185 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:10:02.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.279+0000 7fc57bbd5740 1 -- 172.21.15.16:0/3058506262 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac3c2ea700 con 0x55ac3bf8a9f0 2024-08-31T06:10:02.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.279+0000 7fc56d400640 1 --2- 172.21.15.16:0/3058506262 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55ac3bf8a9f0 0x55ac3c397ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:02.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.301+0000 7fc56d400640 1 --2- 172.21.15.16:0/3058506262 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55ac3bf8a9f0 0x55ac3c397ed0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eecccc77d3d849c6 server_cookie=8a4aa09f29c6245b in_seq=0 out_seq=0 2024-08-31T06:10:02.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.301+0000 7fc54fe00640 10 client.26185 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:02.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.302+0000 7fc54fe00640 1 -- 172.21.15.16:0/3058506262 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ac3c2ea700 con 0x55ac3bf8a9f0 2024-08-31T06:10:02.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.302+0000 7fc54fe00640 10 client.26185 handle_command_reply: tid=0 2024-08-31T06:10:02.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.302+0000 7fc57bbd5740 1 client.26185 shutdown 2024-08-31T06:10:02.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.302+0000 7fc57bbd5740 1 -- 172.21.15.16:0/3058506262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac3bfa31b0 msgr2=0x55ac3c385d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:02.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.302+0000 7fc57bbd5740 1 --2- 172.21.15.16:0/3058506262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac3bfa31b0 0x55ac3c385d20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc5600147e0 tx=0x7fc5600046e0 comp rx=0 tx=0).stop 2024-08-31T06:10:02.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.302+0000 7fc57bbd5740 1 -- 172.21.15.16:0/3058506262 shutdown_connections 2024-08-31T06:10:02.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.302+0000 7fc57bbd5740 1 --2- 172.21.15.16:0/3058506262 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55ac3bf8a9f0 0x55ac3c397ed0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:02.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.302+0000 7fc57bbd5740 1 --2- 172.21.15.16:0/3058506262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac3bfcc070 0x55ac3c391c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:02.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.302+0000 7fc57bbd5740 1 --2- 172.21.15.16:0/3058506262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac3bfab390 0x55ac3c386260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:02.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.302+0000 7fc57bbd5740 1 --2- 172.21.15.16:0/3058506262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac3bfa31b0 0x55ac3c385d20 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:02.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.302+0000 7fc57bbd5740 1 -- 172.21.15.16:0/3058506262 >> 172.21.15.16:0/3058506262 conn(0x55ac3bfc0460 msgr2=0x55ac3c374a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:02.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.302+0000 7fc54fe00640 0 client.26185 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:02.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.313+0000 7fc57bbd5740 1 -- 172.21.15.16:0/3058506262 shutdown_connections 2024-08-31T06:10:02.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.313+0000 7fc57bbd5740 1 -- 172.21.15.16:0/3058506262 wait complete. 2024-08-31T06:10:02.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.313+0000 7fc57bbd5740 20 client.26185 trim_cache size 0 max 16384 2024-08-31T06:10:02.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.328+0000 7fc57bbd5740 1 Processor -- start 2024-08-31T06:10:02.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.328+0000 7fc57bbd5740 1 -- start start 2024-08-31T06:10:02.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.328+0000 7fc57bbd5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac3bfa31b0 0x55ac3c376c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:02.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.328+0000 7fc57bbd5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac3bfab390 0x55ac3c3771c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:02.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.328+0000 7fc57bbd5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac3bfcc070 0x55ac3c3843a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:02.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.328+0000 7fc57bbd5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc5701afc40 con 0x55ac3bfab390 2024-08-31T06:10:02.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.328+0000 7fc57bbd5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc5701ad450 con 0x55ac3bfa31b0 2024-08-31T06:10:02.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.328+0000 7fc57bbd5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc5701ab2e0 con 0x55ac3bfcc070 2024-08-31T06:10:02.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.328+0000 7fc56de00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac3bfa31b0 0x55ac3c376c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:02.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.328+0000 7fc56de00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac3bfa31b0 0x55ac3c376c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51012/0 (socket says 172.21.15.16:51012) 2024-08-31T06:10:02.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.328+0000 7fc56de00640 1 -- 172.21.15.16:0/3246169407 learned_addr learned my addr 172.21.15.16:0/3246169407 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:02.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.328+0000 7fc56d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac3bfab390 0x55ac3c3771c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:02.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.329+0000 7fc56e800640 1 --2- 172.21.15.16:0/3246169407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac3bfcc070 0x55ac3c3843a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:02.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.329+0000 7fc56de00640 1 -- 172.21.15.16:0/3246169407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac3bfcc070 msgr2=0x55ac3c3843a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:02.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.329+0000 7fc56de00640 1 --2- 172.21.15.16:0/3246169407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac3bfcc070 0x55ac3c3843a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:02.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.329+0000 7fc56de00640 1 -- 172.21.15.16:0/3246169407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac3bfab390 msgr2=0x55ac3c3771c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:02.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.329+0000 7fc56de00640 1 --2- 172.21.15.16:0/3246169407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac3bfab390 0x55ac3c3771c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:02.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.329+0000 7fc56de00640 1 -- 172.21.15.16:0/3246169407 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac3c381690 con 0x55ac3bfa31b0 2024-08-31T06:10:02.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.329+0000 7fc56de00640 1 --2- 172.21.15.16:0/3246169407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac3bfa31b0 0x55ac3c376c80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc560029d10 tx=0x7fc560013450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9946de7212530bb8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:02.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.330+0000 7fc55aa00640 1 -- 172.21.15.16:0/3246169407 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc560036070 con 0x55ac3bfa31b0 2024-08-31T06:10:02.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.330+0000 7fc55aa00640 1 -- 172.21.15.16:0/3246169407 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc56001b070 con 0x55ac3bfa31b0 2024-08-31T06:10:02.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.330+0000 7fc55aa00640 1 -- 172.21.15.16:0/3246169407 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc560025040 con 0x55ac3bfa31b0 2024-08-31T06:10:02.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.332+0000 7fc57bbd5740 1 -- 172.21.15.16:0/3246169407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac3bfa31b0 msgr2=0x55ac3c376c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:02.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.332+0000 7fc57bbd5740 1 --2- 172.21.15.16:0/3246169407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac3bfa31b0 0x55ac3c376c80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc560029d10 tx=0x7fc560013450 comp rx=0 tx=0).stop 2024-08-31T06:10:02.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.332+0000 7fc57bbd5740 1 -- 172.21.15.16:0/3246169407 shutdown_connections 2024-08-31T06:10:02.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.332+0000 7fc57bbd5740 1 --2- 172.21.15.16:0/3246169407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac3bfcc070 0x55ac3c3843a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:02.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.332+0000 7fc57bbd5740 1 --2- 172.21.15.16:0/3246169407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac3bfab390 0x55ac3c3771c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:02.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.332+0000 7fc57bbd5740 1 --2- 172.21.15.16:0/3246169407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac3bfa31b0 0x55ac3c376c80 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:02.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.332+0000 7fc57bbd5740 1 -- 172.21.15.16:0/3246169407 >> 172.21.15.16:0/3246169407 conn(0x55ac3bfc0460 msgr2=0x55ac3c385d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:02.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.332+0000 7fc57bbd5740 1 -- 172.21.15.16:0/3246169407 shutdown_connections 2024-08-31T06:10:02.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.332+0000 7fc57bbd5740 1 -- 172.21.15.16:0/3246169407 wait complete. 2024-08-31T06:10:02.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.333+0000 7fc57bbd5740 1 -- start start 2024-08-31T06:10:02.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.333+0000 7fc57bbd5740 1 Processor -- start 2024-08-31T06:10:02.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.334+0000 7fc57bbd5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac3bfa31b0 0x55ac3c38db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:02.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.334+0000 7fc57bbd5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac3bfab390 0x55ac3c38e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:02.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.334+0000 7fc57bbd5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac3bfcc070 0x55ac3c388b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:02.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.334+0000 7fc57bbd5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc5701ab930 con 0x55ac3bfab390 2024-08-31T06:10:02.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.334+0000 7fc57bbd5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc5701acd80 con 0x55ac3bfcc070 2024-08-31T06:10:02.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.334+0000 7fc57bbd5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc5701abfe0 con 0x55ac3bfa31b0 2024-08-31T06:10:02.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.334+0000 7fc57bbd5740 10 client.0 _ll_register_callbacks cb 0x7fc560014460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:10:02.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.334+0000 7fc56de00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac3bfa31b0 0x55ac3c38db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:02.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.334+0000 7fc56e800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac3bfcc070 0x55ac3c388b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:02.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.334+0000 7fc56de00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac3bfa31b0 0x55ac3c38db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:44408/0 (socket says 172.21.15.16:44408) 2024-08-31T06:10:02.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.334+0000 7fc56de00640 1 -- 172.21.15.16:0/1355174655 learned_addr learned my addr 172.21.15.16:0/1355174655 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:02.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.334+0000 7fc56de00640 1 -- 172.21.15.16:0/1355174655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac3bfcc070 msgr2=0x55ac3c388b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:02.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.334+0000 7fc56d400640 1 --2- 172.21.15.16:0/1355174655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac3bfab390 0x55ac3c38e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:02.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.335+0000 7fc56de00640 1 --2- 172.21.15.16:0/1355174655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac3bfcc070 0x55ac3c388b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:02.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.335+0000 7fc56de00640 1 -- 172.21.15.16:0/1355174655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac3bfab390 msgr2=0x55ac3c38e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:02.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.335+0000 7fc56de00640 1 --2- 172.21.15.16:0/1355174655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac3bfab390 0x55ac3c38e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:02.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.335+0000 7fc56de00640 1 -- 172.21.15.16:0/1355174655 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac3c38e900 con 0x55ac3bfa31b0 2024-08-31T06:10:02.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.335+0000 7fc56d400640 1 --2- 172.21.15.16:0/1355174655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac3bfab390 0x55ac3c38e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:10:02.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.335+0000 7fc56de00640 1 --2- 172.21.15.16:0/1355174655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac3bfa31b0 0x55ac3c38db70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc560005f20 tx=0x7fc560029f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=324c3c4c8ce54a81 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:02.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.335+0000 7fc54fe00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:10:02.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.335+0000 7fc54fe00640 1 -- 172.21.15.16:0/1355174655 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc560025040 con 0x55ac3bfa31b0 2024-08-31T06:10:02.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.336+0000 7fc57bbd5740 1 -- 172.21.15.16:0/1355174655 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc56001b070 con 0x55ac3bfa31b0 2024-08-31T06:10:02.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.336+0000 7fc54fe00640 1 -- 172.21.15.16:0/1355174655 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fc560004030 con 0x55ac3bfa31b0 2024-08-31T06:10:02.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.336+0000 7fc54fe00640 1 -- 172.21.15.16:0/1355174655 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc56004c050 con 0x55ac3bfa31b0 2024-08-31T06:10:02.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.336+0000 7fc57bbd5740 10 client.26191 fetch_fsmap learned FSMap version 31 2024-08-31T06:10:02.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.336+0000 7fc57bbd5740 1 -- 172.21.15.16:0/1355174655 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ac3c402e80 con 0x55ac3bfa31b0 2024-08-31T06:10:02.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.336+0000 7fc54fe00640 1 -- 172.21.15.16:0/1355174655 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc56001b070 con 0x55ac3bfa31b0 2024-08-31T06:10:02.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.338+0000 7fc54fe00640 1 -- 172.21.15.16:0/1355174655 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fc560020070 con 0x55ac3bfa31b0 2024-08-31T06:10:02.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.338+0000 7fc57bbd5740 10 client.26191 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:10:02.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.338+0000 7fc57bbd5740 10 client.26191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:10:02.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.338+0000 7fc57bbd5740 20 client.26191 populate_metadata read hostname 'smithi016' 2024-08-31T06:10:02.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.338+0000 7fc57bbd5740 1 --2- 172.21.15.16:0/1355174655 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55ac3bf5fda0 0x55ac3c405600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:02.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.338+0000 7fc57bbd5740 4 client.26191 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:10:02.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.338+0000 7fc57bbd5740 1 -- 172.21.15.16:0/1355174655 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ac3c2e61a0 con 0x55ac3bf5fda0 2024-08-31T06:10:02.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.338+0000 7fc56d400640 1 --2- 172.21.15.16:0/1355174655 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55ac3bf5fda0 0x55ac3c405600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:02.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.339+0000 7fc56d400640 1 --2- 172.21.15.16:0/1355174655 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55ac3bf5fda0 0x55ac3c405600 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af5d3785e45e5fce server_cookie=89d741a4fed2c1a1 in_seq=0 out_seq=0 2024-08-31T06:10:02.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.339+0000 7fc54fe00640 10 client.26191 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:02.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.339+0000 7fc54fe00640 1 -- 172.21.15.16:0/1355174655 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ac3c2e61a0 con 0x55ac3bf5fda0 2024-08-31T06:10:02.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.339+0000 7fc54fe00640 10 client.26191 handle_command_reply: tid=0 2024-08-31T06:10:02.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.339+0000 7fc57bbd5740 1 client.26191 shutdown 2024-08-31T06:10:02.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.339+0000 7fc57bbd5740 1 -- 172.21.15.16:0/1355174655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac3bfa31b0 msgr2=0x55ac3c38db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:02.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.339+0000 7fc57bbd5740 1 --2- 172.21.15.16:0/1355174655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac3bfa31b0 0x55ac3c38db70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc560005f20 tx=0x7fc560029f70 comp rx=0 tx=0).stop 2024-08-31T06:10:02.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.339+0000 7fc57bbd5740 1 -- 172.21.15.16:0/1355174655 shutdown_connections 2024-08-31T06:10:02.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.339+0000 7fc57bbd5740 1 --2- 172.21.15.16:0/1355174655 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55ac3bf5fda0 0x55ac3c405600 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:02.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.339+0000 7fc57bbd5740 1 --2- 172.21.15.16:0/1355174655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac3bfcc070 0x55ac3c388b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:02.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.339+0000 7fc54fe00640 0 client.26191 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:02.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.339+0000 7fc57bbd5740 1 --2- 172.21.15.16:0/1355174655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac3bfab390 0x55ac3c38e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:02.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.339+0000 7fc57bbd5740 1 --2- 172.21.15.16:0/1355174655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac3bfa31b0 0x55ac3c38db70 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:02.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.339+0000 7fc57bbd5740 1 -- 172.21.15.16:0/1355174655 >> 172.21.15.16:0/1355174655 conn(0x55ac3bfc0460 msgr2=0x55ac3c384360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:02.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.340+0000 7fc57bbd5740 1 -- 172.21.15.16:0/1355174655 shutdown_connections 2024-08-31T06:10:02.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.340+0000 7fc57bbd5740 1 -- 172.21.15.16:0/1355174655 wait complete. 2024-08-31T06:10:02.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.340+0000 7fc57bbd5740 20 client.26191 trim_cache size 0 max 16384 2024-08-31T06:10:02.343 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:10:02.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.341+0000 7fc574e00640 1 -- 172.21.15.16:0/3798375036 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc5540b4d20 msgr2=0x7fc5540b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:02.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.341+0000 7fc574e00640 1 --2- 172.21.15.16:0/3798375036 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc5540b4d20 0x7fc5540b7230 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc560006fd0 tx=0x7fc5600090e0 comp rx=0 tx=0).stop 2024-08-31T06:10:02.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.341+0000 7fc574e00640 1 -- 172.21.15.16:0/3798375036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc570109250 msgr2=0x7fc57019caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:02.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.341+0000 7fc574e00640 1 --2- 172.21.15.16:0/3798375036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc570109250 0x7fc57019caf0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc55c02a930 tx=0x7fc55c004270 comp rx=0 tx=0).stop 2024-08-31T06:10:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.341+0000 7fc574e00640 1 -- 172.21.15.16:0/3798375036 shutdown_connections 2024-08-31T06:10:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.341+0000 7fc574e00640 1 --2- 172.21.15.16:0/3798375036 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc5540b4d20 0x7fc5540b7230 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.341+0000 7fc574e00640 1 --2- 172.21.15.16:0/3798375036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc570109ba0 0x7fc5701a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.341+0000 7fc574e00640 1 --2- 172.21.15.16:0/3798375036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc570109250 0x7fc57019caf0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.341+0000 7fc574e00640 1 --2- 172.21.15.16:0/3798375036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc570103910 0x7fc57019c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.341+0000 7fc574e00640 1 -- 172.21.15.16:0/3798375036 >> 172.21.15.16:0/3798375036 conn(0x7fc5700ff380 msgr2=0x7fc570107930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.341+0000 7fc574e00640 1 -- 172.21.15.16:0/3798375036 shutdown_connections 2024-08-31T06:10:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:02.341+0000 7fc574e00640 1 -- 172.21.15.16:0/3798375036 wait complete. 2024-08-31T06:10:02.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:02 smithi184 ceph-mon[22331]: pgmap v448: 97 pgs: 97 active+clean; 1.6 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.1 MiB/s wr, 678 op/s 2024-08-31T06:10:02.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:02 smithi155 ceph-mon[21742]: pgmap v448: 97 pgs: 97 active+clean; 1.6 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.1 MiB/s wr, 678 op/s 2024-08-31T06:10:04.331 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T06:10:04.332 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T06:10:04.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:04 smithi016 ceph-mon[17194]: pgmap v449: 97 pgs: 97 active+clean; 1.6 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.1 MiB/s wr, 664 op/s 2024-08-31T06:10:04.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:04 smithi184 ceph-mon[22331]: pgmap v449: 97 pgs: 97 active+clean; 1.6 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.1 MiB/s wr, 664 op/s 2024-08-31T06:10:04.798 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:10:04.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:04 smithi155 ceph-mon[21742]: pgmap v449: 97 pgs: 97 active+clean; 1.6 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.1 MiB/s wr, 664 op/s 2024-08-31T06:10:06.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.615+0000 7fbeb7200640 1 -- 172.21.15.16:0/2282952835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbeb81099e0 msgr2=0x7fbeb8110210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:06.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.615+0000 7fbeb7200640 1 --2- 172.21.15.16:0/2282952835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbeb81099e0 0x7fbeb8110210 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0009ca0 tx=0x7fbeb002a8c0 comp rx=0 tx=0).stop 2024-08-31T06:10:06.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.615+0000 7fbeb7200640 1 -- 172.21.15.16:0/2282952835 shutdown_connections 2024-08-31T06:10:06.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.615+0000 7fbeb7200640 1 --2- 172.21.15.16:0/2282952835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbeb8109090 0x7fbeb81094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:06.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.615+0000 7fbeb7200640 1 --2- 172.21.15.16:0/2282952835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbeb81099e0 0x7fbeb8110210 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:06.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.615+0000 7fbeb7200640 1 --2- 172.21.15.16:0/2282952835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbeb8103750 0x7fbeb8103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:06.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.615+0000 7fbeb7200640 1 -- 172.21.15.16:0/2282952835 >> 172.21.15.16:0/2282952835 conn(0x7fbeb80ff1e0 msgr2=0x7fbeb8101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:06.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.615+0000 7fbeb7200640 1 -- 172.21.15.16:0/2282952835 shutdown_connections 2024-08-31T06:10:06.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.615+0000 7fbeb7200640 1 -- 172.21.15.16:0/2282952835 wait complete. 2024-08-31T06:10:06.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.616+0000 7fbeb7200640 1 Processor -- start 2024-08-31T06:10:06.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.616+0000 7fbeb7200640 1 -- start start 2024-08-31T06:10:06.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.617+0000 7fbeb7200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbeb8103750 0x7fbeb819c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:06.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.617+0000 7fbeb7200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbeb8109090 0x7fbeb819c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:06.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.617+0000 7fbeb7200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbeb81099e0 0x7fbeb81a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:06.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.617+0000 7fbeb7200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb8113700 con 0x7fbeb81099e0 2024-08-31T06:10:06.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.617+0000 7fbeb7200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb8113580 con 0x7fbeb8103750 2024-08-31T06:10:06.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.617+0000 7fbeb7200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb8113880 con 0x7fbeb8109090 2024-08-31T06:10:06.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.617+0000 7fbeb5e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbeb8103750 0x7fbeb819c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:06.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.617+0000 7fbeb5400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbeb8109090 0x7fbeb819c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:06.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.617+0000 7fbeb5400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbeb8109090 0x7fbeb819c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:46374/0 (socket says 172.21.15.16:46374) 2024-08-31T06:10:06.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.617+0000 7fbeb5e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbeb8103750 0x7fbeb819c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:59318/0 (socket says 172.21.15.16:59318) 2024-08-31T06:10:06.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.618+0000 7fbeb5400640 1 -- 172.21.15.16:0/1959809899 learned_addr learned my addr 172.21.15.16:0/1959809899 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:06.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.618+0000 7fbeb6800640 1 --2- 172.21.15.16:0/1959809899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbeb81099e0 0x7fbeb81a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:06.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.618+0000 7fbeb5400640 1 -- 172.21.15.16:0/1959809899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbeb8103750 msgr2=0x7fbeb819c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:06.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.618+0000 7fbeb5400640 1 --2- 172.21.15.16:0/1959809899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbeb8103750 0x7fbeb819c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:06.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.618+0000 7fbeb5400640 1 -- 172.21.15.16:0/1959809899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbeb81099e0 msgr2=0x7fbeb81a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:06.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.618+0000 7fbeb5400640 1 --2- 172.21.15.16:0/1959809899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbeb81099e0 0x7fbeb81a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:06.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.618+0000 7fbeb5400640 1 -- 172.21.15.16:0/1959809899 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb81a2180 con 0x7fbeb8109090 2024-08-31T06:10:06.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.618+0000 7fbeb5e00640 1 --2- 172.21.15.16:0/1959809899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbeb8103750 0x7fbeb819c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:10:06.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.618+0000 7fbeb6800640 1 --2- 172.21.15.16:0/1959809899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbeb81099e0 0x7fbeb81a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:10:06.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.618+0000 7fbeb5400640 1 --2- 172.21.15.16:0/1959809899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbeb8109090 0x7fbeb819c7b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbea400e9c0 tx=0x7fbea400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:06.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.619+0000 7fbeaea00640 1 -- 172.21.15.16:0/1959809899 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea400ccc0 con 0x7fbeb8109090 2024-08-31T06:10:06.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.619+0000 7fbeaea00640 1 -- 172.21.15.16:0/1959809899 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbea40054d0 con 0x7fbeb8109090 2024-08-31T06:10:06.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.619+0000 7fbeaea00640 1 -- 172.21.15.16:0/1959809899 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea4010430 con 0x7fbeb8109090 2024-08-31T06:10:06.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.619+0000 7fbeb7200640 1 -- 172.21.15.16:0/1959809899 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeb81a2470 con 0x7fbeb8109090 2024-08-31T06:10:06.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.619+0000 7fbeb7200640 1 -- 172.21.15.16:0/1959809899 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeb81a8bd0 con 0x7fbeb8109090 2024-08-31T06:10:06.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 160.0 s, 343.6 tps, lat 286.798 ms stddev 177.895 2024-08-31T06:10:06.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.623+0000 7fbec3ebe740 1 Processor -- start 2024-08-31T06:10:06.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.623+0000 7fbec3ebe740 1 -- start start 2024-08-31T06:10:06.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.623+0000 7fbec3ebe740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565122a8b350 0x565122e317d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:06.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.623+0000 7fbeaea00640 1 -- 172.21.15.16:0/1959809899 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fbea4005020 con 0x7fbeb8109090 2024-08-31T06:10:06.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.623+0000 7fbec3ebe740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565122a62490 0x565122e31d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:06.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.623+0000 7fbec3ebe740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565122a6a670 0x565122e32250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:06.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.623+0000 7fbec3ebe740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x565122da7750 con 0x565122a8b350 2024-08-31T06:10:06.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.623+0000 7fbec3ebe740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x565122d96ce0 con 0x565122a6a670 2024-08-31T06:10:06.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.623+0000 7fbec3ebe740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x565122d9ac90 con 0x565122a62490 2024-08-31T06:10:06.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.624+0000 7fbeb5e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565122a62490 0x565122e31d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:06.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.624+0000 7fbeb6800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565122a8b350 0x565122e317d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:06.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.624+0000 7fbeb5400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565122a6a670 0x565122e32250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:06.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.624+0000 7fbeb5e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565122a62490 0x565122e31d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:46390/0 (socket says 172.21.15.16:46390) 2024-08-31T06:10:06.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.624+0000 7fbeb5e00640 1 -- 172.21.15.16:0/3697655852 learned_addr learned my addr 172.21.15.16:0/3697655852 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:06.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.624+0000 7fbeb5e00640 1 -- 172.21.15.16:0/3697655852 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565122a6a670 msgr2=0x565122e32250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:06.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.624+0000 7fbeb5e00640 1 --2- 172.21.15.16:0/3697655852 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565122a6a670 0x565122e32250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:06.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.624+0000 7fbeb5e00640 1 -- 172.21.15.16:0/3697655852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565122a8b350 msgr2=0x565122e317d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:06.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.624+0000 7fbeb5e00640 1 --2- 172.21.15.16:0/3697655852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565122a8b350 0x565122e317d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:06.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.624+0000 7fbeb5e00640 1 -- 172.21.15.16:0/3697655852 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565122af6930 con 0x565122a62490 2024-08-31T06:10:06.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.624+0000 7fbeb5e00640 1 --2- 172.21.15.16:0/3697655852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565122a62490 0x565122e31d10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbea80115f0 tx=0x7fbea8011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a183b04b934a4b30 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:06.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.625+0000 7fbea3400640 1 -- 172.21.15.16:0/3697655852 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea801e070 con 0x565122a62490 2024-08-31T06:10:06.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.625+0000 7fbea3400640 1 -- 172.21.15.16:0/3697655852 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbea800faf0 con 0x565122a62490 2024-08-31T06:10:06.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.625+0000 7fbea3400640 1 -- 172.21.15.16:0/3697655852 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea8019bc0 con 0x565122a62490 2024-08-31T06:10:06.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.626+0000 7fbeaea00640 1 --2- 172.21.15.16:0/1959809899 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbe9c0b4ca0 0x7fbe9c0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:06.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.626+0000 7fbec3ebe740 1 -- 172.21.15.16:0/3697655852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565122a62490 msgr2=0x565122e31d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:06.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.626+0000 7fbec3ebe740 1 --2- 172.21.15.16:0/3697655852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565122a62490 0x565122e31d10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbea80115f0 tx=0x7fbea8011ab0 comp rx=0 tx=0).stop 2024-08-31T06:10:06.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.626+0000 7fbeaea00640 1 -- 172.21.15.16:0/1959809899 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fbea40e5920 con 0x7fbeb8109090 2024-08-31T06:10:06.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.627+0000 7fbeb6800640 1 --2- 172.21.15.16:0/1959809899 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbe9c0b4ca0 0x7fbe9c0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:06.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.628+0000 7fbec3ebe740 1 -- 172.21.15.16:0/3697655852 shutdown_connections 2024-08-31T06:10:06.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.628+0000 7fbec3ebe740 1 --2- 172.21.15.16:0/3697655852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565122a62490 0x565122e31d10 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:06.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.628+0000 7fbec3ebe740 1 --2- 172.21.15.16:0/3697655852 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565122a6a670 0x565122e32250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:06.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.628+0000 7fbec3ebe740 1 --2- 172.21.15.16:0/3697655852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565122a8b350 0x565122e317d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:06.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.628+0000 7fbec3ebe740 1 -- 172.21.15.16:0/3697655852 >> 172.21.15.16:0/3697655852 conn(0x565122a7f740 msgr2=0x565122e2f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:06.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.628+0000 7fbec3ebe740 1 -- 172.21.15.16:0/3697655852 shutdown_connections 2024-08-31T06:10:06.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.628+0000 7fbeb6800640 1 --2- 172.21.15.16:0/1959809899 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbe9c0b4ca0 0x7fbe9c0b71b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0006010 tx=0x7fbeb002f3f0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:06.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.628+0000 7fbec3ebe740 1 -- 172.21.15.16:0/3697655852 wait complete. 2024-08-31T06:10:06.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.629+0000 7fbec3ebe740 1 -- start start 2024-08-31T06:10:06.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.630+0000 7fbec3ebe740 1 Processor -- start 2024-08-31T06:10:06.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.631+0000 7fbec3ebe740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565122a62490 0x565122e46240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:06.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.631+0000 7fbec3ebe740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565122a6a670 0x565122e46820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:06.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.631+0000 7fbec3ebe740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565122a8b350 0x565122e4b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:06.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.631+0000 7fbec3ebe740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x565122bf9890 con 0x565122a8b350 2024-08-31T06:10:06.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.631+0000 7fbec3ebe740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x565122d1dcd0 con 0x565122a6a670 2024-08-31T06:10:06.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.631+0000 7fbec3ebe740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x565122e3ffb0 con 0x565122a62490 2024-08-31T06:10:06.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.631+0000 7fbec3ebe740 10 client.0 _ll_register_callbacks cb 0x565122d91ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:10:06.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.631+0000 7fbeb5400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565122a6a670 0x565122e46820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:06.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.631+0000 7fbeb5400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565122a6a670 0x565122e46820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:59340/0 (socket says 172.21.15.16:59340) 2024-08-31T06:10:06.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.631+0000 7fbeb5400640 1 -- 172.21.15.16:0/2469593605 learned_addr learned my addr 172.21.15.16:0/2469593605 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:06.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.631+0000 7fbeb5400640 1 -- 172.21.15.16:0/2469593605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565122a62490 msgr2=0x565122e46240 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T06:10:06.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.631+0000 7fbeb5400640 1 --2- 172.21.15.16:0/2469593605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565122a62490 0x565122e46240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:06.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.631+0000 7fbeb5400640 1 -- 172.21.15.16:0/2469593605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565122a8b350 msgr2=0x565122e4b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:06.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.632+0000 7fbeb6800640 1 --2- 172.21.15.16:0/2469593605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565122a8b350 0x565122e4b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:06.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.632+0000 7fbeb5400640 1 --2- 172.21.15.16:0/2469593605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565122a8b350 0x565122e4b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:06.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.632+0000 7fbeb5400640 1 -- 172.21.15.16:0/2469593605 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565122d671b0 con 0x565122a6a670 2024-08-31T06:10:06.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.632+0000 7fbeb6800640 1 --2- 172.21.15.16:0/2469593605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565122a8b350 0x565122e4b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:10:06.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.632+0000 7fbeb5400640 1 --2- 172.21.15.16:0/2469593605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565122a6a670 0x565122e46820 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbea4004650 tx=0x7fbea40abde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3c3fc0e20a5c1d0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:06.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.632+0000 7fbe97e00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:10:06.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.632+0000 7fbe97e00640 1 -- 172.21.15.16:0/2469593605 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea400ce60 con 0x565122a6a670 2024-08-31T06:10:06.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.633+0000 7fbe97e00640 1 -- 172.21.15.16:0/2469593605 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fbea40a8600 con 0x565122a6a670 2024-08-31T06:10:06.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.633+0000 7fbec3ebe740 1 -- 172.21.15.16:0/2469593605 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbea800faf0 con 0x565122a6a670 2024-08-31T06:10:06.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.633+0000 7fbe97e00640 1 -- 172.21.15.16:0/2469593605 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fbea40e3500 con 0x565122a6a670 2024-08-31T06:10:06.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.633+0000 7fbec3ebe740 10 client.26204 fetch_fsmap learned FSMap version 31 2024-08-31T06:10:06.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.633+0000 7fbec3ebe740 1 -- 172.21.15.16:0/2469593605 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x565122e547f0 con 0x565122a6a670 2024-08-31T06:10:06.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.633+0000 7fbe97e00640 1 -- 172.21.15.16:0/2469593605 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbea800faf0 con 0x565122a6a670 2024-08-31T06:10:06.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.633+0000 7fbe97e00640 1 -- 172.21.15.16:0/2469593605 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fbea40971d0 con 0x565122a6a670 2024-08-31T06:10:06.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.634+0000 7fbec3ebe740 10 client.26204 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:10:06.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.634+0000 7fbec3ebe740 10 client.26204 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:10:06.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.634+0000 7fbec3ebe740 20 client.26204 populate_metadata read hostname 'smithi016' 2024-08-31T06:10:06.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.634+0000 7fbec3ebe740 1 --2- 172.21.15.16:0/2469593605 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x565122a49cd0 0x565122e57cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:06.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.634+0000 7fbec3ebe740 4 client.26204 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:10:06.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.634+0000 7fbec3ebe740 1 -- 172.21.15.16:0/2469593605 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565122da96a0 con 0x565122a49cd0 2024-08-31T06:10:06.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.634+0000 7fbeb5e00640 1 --2- 172.21.15.16:0/2469593605 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x565122a49cd0 0x565122e57cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:06.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.635+0000 7fbeb5e00640 1 --2- 172.21.15.16:0/2469593605 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x565122a49cd0 0x565122e57cc0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75f4e5fd6a7cf504 server_cookie=3f2e9af49cc4edee in_seq=0 out_seq=0 2024-08-31T06:10:06.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.635+0000 7fbe97e00640 10 client.26204 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:06.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.637+0000 7fbe97e00640 1 -- 172.21.15.16:0/2469593605 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x565122da96a0 con 0x565122a49cd0 2024-08-31T06:10:06.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.637+0000 7fbe97e00640 10 client.26204 handle_command_reply: tid=0 2024-08-31T06:10:06.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.637+0000 7fbec3ebe740 1 client.26204 shutdown 2024-08-31T06:10:06.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.639+0000 7fbec3ebe740 1 -- 172.21.15.16:0/2469593605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565122a6a670 msgr2=0x565122e46820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:06.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.639+0000 7fbec3ebe740 1 --2- 172.21.15.16:0/2469593605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565122a6a670 0x565122e46820 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbea4004650 tx=0x7fbea40abde0 comp rx=0 tx=0).stop 2024-08-31T06:10:06.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.639+0000 7fbec3ebe740 1 -- 172.21.15.16:0/2469593605 shutdown_connections 2024-08-31T06:10:06.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.639+0000 7fbec3ebe740 1 --2- 172.21.15.16:0/2469593605 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x565122a49cd0 0x565122e57cc0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:06.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.639+0000 7fbe97e00640 0 client.26204 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:06.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.639+0000 7fbec3ebe740 1 --2- 172.21.15.16:0/2469593605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565122a8b350 0x565122e4b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:06.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.639+0000 7fbec3ebe740 1 --2- 172.21.15.16:0/2469593605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565122a6a670 0x565122e46820 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:06.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.639+0000 7fbec3ebe740 1 --2- 172.21.15.16:0/2469593605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565122a62490 0x565122e46240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:06.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.639+0000 7fbec3ebe740 1 -- 172.21.15.16:0/2469593605 >> 172.21.15.16:0/2469593605 conn(0x565122a7f740 msgr2=0x565122e349b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:06.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.639+0000 7fbec3ebe740 1 -- 172.21.15.16:0/2469593605 shutdown_connections 2024-08-31T06:10:06.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.639+0000 7fbec3ebe740 1 -- 172.21.15.16:0/2469593605 wait complete. 2024-08-31T06:10:06.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.640+0000 7fbec3ebe740 20 client.26204 trim_cache size 0 max 16384 2024-08-31T06:10:06.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:06 smithi016 ceph-mon[17194]: pgmap v450: 97 pgs: 97 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.0 MiB/s wr, 789 op/s 2024-08-31T06:10:06.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.667+0000 7fbec3ebe740 1 Processor -- start 2024-08-31T06:10:06.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.667+0000 7fbec3ebe740 1 -- start start 2024-08-31T06:10:06.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.667+0000 7fbec3ebe740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565122a62490 0x565122e40860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:06.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.667+0000 7fbec3ebe740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565122a6a670 0x565122e32f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:06.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.667+0000 7fbec3ebe740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565122a8b350 0x565122e43230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:06.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.667+0000 7fbec3ebe740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb81afa60 con 0x565122a62490 2024-08-31T06:10:06.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.667+0000 7fbec3ebe740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb81ad270 con 0x565122a6a670 2024-08-31T06:10:06.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.667+0000 7fbec3ebe740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb81ab140 con 0x565122a8b350 2024-08-31T06:10:06.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.668+0000 7fbeb5400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565122a6a670 0x565122e32f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:06.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.668+0000 7fbeb5400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565122a6a670 0x565122e32f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:59344/0 (socket says 172.21.15.16:59344) 2024-08-31T06:10:06.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.668+0000 7fbeb5400640 1 -- 172.21.15.16:0/2186877329 learned_addr learned my addr 172.21.15.16:0/2186877329 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:06.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.668+0000 7fbeb6800640 1 --2- 172.21.15.16:0/2186877329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565122a8b350 0x565122e43230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:06.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.668+0000 7fbeb5e00640 1 --2- 172.21.15.16:0/2186877329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565122a62490 0x565122e40860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:06.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.668+0000 7fbeb5400640 1 -- 172.21.15.16:0/2186877329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565122a8b350 msgr2=0x565122e43230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:06.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.668+0000 7fbeb5400640 1 --2- 172.21.15.16:0/2186877329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565122a8b350 0x565122e43230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:06.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.668+0000 7fbeb5400640 1 -- 172.21.15.16:0/2186877329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565122a62490 msgr2=0x565122e40860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:06.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.668+0000 7fbeb5400640 1 --2- 172.21.15.16:0/2186877329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565122a62490 0x565122e40860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:06.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.668+0000 7fbeb5400640 1 -- 172.21.15.16:0/2186877329 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565122e3fae0 con 0x565122a6a670 2024-08-31T06:10:06.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.668+0000 7fbeb5400640 1 --2- 172.21.15.16:0/2186877329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565122a6a670 0x565122e32f10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbea40abd10 tx=0x7fbea40a78d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2199bd75629bacb server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:06.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.669+0000 7fbea2a00640 1 -- 172.21.15.16:0/2186877329 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea400ce60 con 0x565122a6a670 2024-08-31T06:10:06.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.669+0000 7fbea2a00640 1 -- 172.21.15.16:0/2186877329 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbea4005310 con 0x565122a6a670 2024-08-31T06:10:06.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.669+0000 7fbea2a00640 1 -- 172.21.15.16:0/2186877329 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea40a6730 con 0x565122a6a670 2024-08-31T06:10:06.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.670+0000 7fbec3ebe740 1 -- 172.21.15.16:0/2186877329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565122a6a670 msgr2=0x565122e32f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:06.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.670+0000 7fbec3ebe740 1 --2- 172.21.15.16:0/2186877329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565122a6a670 0x565122e32f10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbea40abd10 tx=0x7fbea40a78d0 comp rx=0 tx=0).stop 2024-08-31T06:10:06.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.670+0000 7fbec3ebe740 1 -- 172.21.15.16:0/2186877329 shutdown_connections 2024-08-31T06:10:06.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.670+0000 7fbec3ebe740 1 --2- 172.21.15.16:0/2186877329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565122a8b350 0x565122e43230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:06.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.670+0000 7fbec3ebe740 1 --2- 172.21.15.16:0/2186877329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565122a6a670 0x565122e32f10 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:06.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.670+0000 7fbec3ebe740 1 --2- 172.21.15.16:0/2186877329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565122a62490 0x565122e40860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:06.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.670+0000 7fbec3ebe740 1 -- 172.21.15.16:0/2186877329 >> 172.21.15.16:0/2186877329 conn(0x565122a7f740 msgr2=0x565122e32800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:06.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.670+0000 7fbec3ebe740 1 -- 172.21.15.16:0/2186877329 shutdown_connections 2024-08-31T06:10:06.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.670+0000 7fbec3ebe740 1 -- 172.21.15.16:0/2186877329 wait complete. 2024-08-31T06:10:06.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.671+0000 7fbec3ebe740 1 -- start start 2024-08-31T06:10:06.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.671+0000 7fbec3ebe740 1 Processor -- start 2024-08-31T06:10:06.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.672+0000 7fbec3ebe740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565122a62490 0x565122e52430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:06.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.672+0000 7fbec3ebe740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565122a6a670 0x565122e4e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:06.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.672+0000 7fbec3ebe740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565122a8b350 0x565122e4ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:06.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.672+0000 7fbec3ebe740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb81ab790 con 0x565122a8b350 2024-08-31T06:10:06.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.672+0000 7fbec3ebe740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb81acba0 con 0x565122a62490 2024-08-31T06:10:06.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.672+0000 7fbec3ebe740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb81abe00 con 0x565122a6a670 2024-08-31T06:10:06.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.672+0000 7fbec3ebe740 10 client.0 _ll_register_callbacks cb 0x7fbea40eb220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:10:06.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.672+0000 7fbeb5400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565122a6a670 0x565122e4e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:06.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.672+0000 7fbeb5e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565122a62490 0x565122e52430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:06.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.672+0000 7fbeb5400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565122a6a670 0x565122e4e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:46420/0 (socket says 172.21.15.16:46420) 2024-08-31T06:10:06.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.672+0000 7fbeb5400640 1 -- 172.21.15.16:0/2676681069 learned_addr learned my addr 172.21.15.16:0/2676681069 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:06.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.672+0000 7fbeb6800640 1 --2- 172.21.15.16:0/2676681069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565122a8b350 0x565122e4ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:06.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.673+0000 7fbeb5e00640 1 -- 172.21.15.16:0/2676681069 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565122a6a670 msgr2=0x565122e4e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:06.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.673+0000 7fbeb5e00640 1 --2- 172.21.15.16:0/2676681069 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565122a6a670 0x565122e4e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:06.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.673+0000 7fbeb5e00640 1 -- 172.21.15.16:0/2676681069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565122a8b350 msgr2=0x565122e4ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:06.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.673+0000 7fbeb5e00640 1 --2- 172.21.15.16:0/2676681069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565122a8b350 0x565122e4ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:06.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.673+0000 7fbeb5e00640 1 -- 172.21.15.16:0/2676681069 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565122e358e0 con 0x565122a62490 2024-08-31T06:10:06.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.673+0000 7fbeb5e00640 1 --2- 172.21.15.16:0/2676681069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565122a62490 0x565122e52430 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x565122e583a0 tx=0x7fbea8019700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b325cfdc224e6e8b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:06.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.673+0000 7fbe97e00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:10:06.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.675+0000 7fbe97e00640 1 -- 172.21.15.16:0/2676681069 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea802e070 con 0x565122a62490 2024-08-31T06:10:06.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.675+0000 7fbe97e00640 1 -- 172.21.15.16:0/2676681069 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fbea8019410 con 0x565122a62490 2024-08-31T06:10:06.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.675+0000 7fbec3ebe740 1 -- 172.21.15.16:0/2676681069 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x565122da5140 con 0x565122a62490 2024-08-31T06:10:06.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.675+0000 7fbe97e00640 1 -- 172.21.15.16:0/2676681069 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fbea8037050 con 0x565122a62490 2024-08-31T06:10:06.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.676+0000 7fbec3ebe740 10 client.26210 fetch_fsmap learned FSMap version 31 2024-08-31T06:10:06.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.676+0000 7fbec3ebe740 1 -- 172.21.15.16:0/2676681069 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x565122ec32d0 con 0x565122a62490 2024-08-31T06:10:06.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.676+0000 7fbe97e00640 1 -- 172.21.15.16:0/2676681069 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x565122da5140 con 0x565122a62490 2024-08-31T06:10:06.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.676+0000 7fbe97e00640 1 -- 172.21.15.16:0/2676681069 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fbea8031020 con 0x565122a62490 2024-08-31T06:10:06.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.676+0000 7fbec3ebe740 10 client.26210 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:10:06.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.676+0000 7fbec3ebe740 10 client.26210 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:10:06.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.676+0000 7fbec3ebe740 20 client.26210 populate_metadata read hostname 'smithi016' 2024-08-31T06:10:06.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.676+0000 7fbec3ebe740 1 --2- 172.21.15.16:0/2676681069 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x565122a1f080 0x565122ec5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:06.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.676+0000 7fbec3ebe740 4 client.26210 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T06:10:06.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.676+0000 7fbec3ebe740 1 -- 172.21.15.16:0/2676681069 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x565122da2880 con 0x565122a1f080 2024-08-31T06:10:06.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.677+0000 7fbeb5400640 1 --2- 172.21.15.16:0/2676681069 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x565122a1f080 0x565122ec5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:06.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.677+0000 7fbeb5400640 1 --2- 172.21.15.16:0/2676681069 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x565122a1f080 0x565122ec5950 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65e763a8dce3b8fa server_cookie=b70e2122250d046f in_seq=0 out_seq=0 2024-08-31T06:10:06.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.677+0000 7fbe97e00640 10 client.26210 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:06.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.679+0000 7fbe97e00640 1 -- 172.21.15.16:0/2676681069 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x565122da2880 con 0x565122a1f080 2024-08-31T06:10:06.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.679+0000 7fbe97e00640 10 client.26210 handle_command_reply: tid=0 2024-08-31T06:10:06.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.679+0000 7fbec3ebe740 1 client.26210 shutdown 2024-08-31T06:10:06.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.680+0000 7fbec3ebe740 1 -- 172.21.15.16:0/2676681069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565122a62490 msgr2=0x565122e52430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:06.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.680+0000 7fbec3ebe740 1 --2- 172.21.15.16:0/2676681069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565122a62490 0x565122e52430 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x565122e583a0 tx=0x7fbea8019700 comp rx=0 tx=0).stop 2024-08-31T06:10:06.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.680+0000 7fbec3ebe740 1 -- 172.21.15.16:0/2676681069 shutdown_connections 2024-08-31T06:10:06.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.680+0000 7fbec3ebe740 1 --2- 172.21.15.16:0/2676681069 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x565122a1f080 0x565122ec5950 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:06.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.680+0000 7fbec3ebe740 1 --2- 172.21.15.16:0/2676681069 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565122a6a670 0x565122e4e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:06.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.680+0000 7fbec3ebe740 1 --2- 172.21.15.16:0/2676681069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565122a8b350 0x565122e4ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:06.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.680+0000 7fbec3ebe740 1 --2- 172.21.15.16:0/2676681069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565122a62490 0x565122e52430 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:06.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.680+0000 7fbec3ebe740 1 -- 172.21.15.16:0/2676681069 >> 172.21.15.16:0/2676681069 conn(0x565122a7f740 msgr2=0x565122e32800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:06.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.680+0000 7fbe97e00640 0 client.26210 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:06.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.680+0000 7fbec3ebe740 1 -- 172.21.15.16:0/2676681069 shutdown_connections 2024-08-31T06:10:06.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.681+0000 7fbec3ebe740 1 -- 172.21.15.16:0/2676681069 wait complete. 2024-08-31T06:10:06.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.681+0000 7fbec3ebe740 20 client.26210 trim_cache size 0 max 16384 2024-08-31T06:10:06.684 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:10:06.684 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-08-31T06:10:06.684 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "c1fd6904-2e02-47fa-939a-47349b9fe36b", 2024-08-31T06:10:06.685 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-08-31T06:10:06.685 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:10:06.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.682+0000 7fbeb7200640 1 -- 172.21.15.16:0/1959809899 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbe9c0b4ca0 msgr2=0x7fbe9c0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:06.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.682+0000 7fbeb7200640 1 --2- 172.21.15.16:0/1959809899 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbe9c0b4ca0 0x7fbe9c0b71b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0006010 tx=0x7fbeb002f3f0 comp rx=0 tx=0).stop 2024-08-31T06:10:06.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.682+0000 7fbeb7200640 1 -- 172.21.15.16:0/1959809899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbeb8109090 msgr2=0x7fbeb819c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:06.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.682+0000 7fbeb7200640 1 --2- 172.21.15.16:0/1959809899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbeb8109090 0x7fbeb819c7b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbea400e9c0 tx=0x7fbea400ee80 comp rx=0 tx=0).stop 2024-08-31T06:10:06.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.682+0000 7fbeb7200640 1 -- 172.21.15.16:0/1959809899 shutdown_connections 2024-08-31T06:10:06.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.682+0000 7fbeb7200640 1 --2- 172.21.15.16:0/1959809899 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbe9c0b4ca0 0x7fbe9c0b71b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:06.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.682+0000 7fbeb7200640 1 --2- 172.21.15.16:0/1959809899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbeb8109090 0x7fbeb819c7b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:06.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.682+0000 7fbeb7200640 1 --2- 172.21.15.16:0/1959809899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbeb81099e0 0x7fbeb81a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:06.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.682+0000 7fbeb7200640 1 --2- 172.21.15.16:0/1959809899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbeb8103750 0x7fbeb819c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:06.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.682+0000 7fbeb7200640 1 -- 172.21.15.16:0/1959809899 >> 172.21.15.16:0/1959809899 conn(0x7fbeb80ff1e0 msgr2=0x7fbeb8108620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:06.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.682+0000 7fbeb7200640 1 -- 172.21.15.16:0/1959809899 shutdown_connections 2024-08-31T06:10:06.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:06.682+0000 7fbeb7200640 1 -- 172.21.15.16:0/1959809899 wait complete. 2024-08-31T06:10:06.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:06 smithi184 ceph-mon[22331]: pgmap v450: 97 pgs: 97 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.0 MiB/s wr, 789 op/s 2024-08-31T06:10:06.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:06 smithi155 ceph-mon[21742]: pgmap v450: 97 pgs: 97 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.0 MiB/s wr, 789 op/s 2024-08-31T06:10:07.623 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T06:10:07.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:07 smithi016 ceph-mon[17194]: scrub queued for path: / 2024-08-31T06:10:07.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:07 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:10:07.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:07 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:10:07.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:07 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:10:07.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:07 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:10:07.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:07 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:10:07.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:07 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:10:07.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:07 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:10:07.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:07 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:10:07.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:07 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:10:07.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:07 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:10:07.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:07 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:10:07.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:07 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:10:07.724 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:07 smithi184 ceph-mon[22331]: scrub queued for path: / 2024-08-31T06:10:07.725 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:07 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:10:07.725 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:07 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:10:07.725 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:07 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:10:07.725 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:07 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:10:07.725 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:07 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:10:07.726 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:07 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:10:07.726 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:07 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:10:07.726 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:07 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:10:07.726 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:07 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:10:07.726 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:07 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:10:07.726 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:07 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:10:07.727 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:07 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:10:07.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:07 smithi155 ceph-mon[21742]: scrub queued for path: / 2024-08-31T06:10:07.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:07 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:10:07.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:07 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:10:07.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:07 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:10:07.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:07 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:10:07.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:07 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:10:07.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:07 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:10:07.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:07 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:10:07.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:07 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:10:07.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:07 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:10:07.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:07 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:10:07.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:07 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:10:07.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:07 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:10:08.071 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:10:08.579 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:08 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:10:08.580 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:08 smithi016 ceph-mon[17194]: pgmap v451: 97 pgs: 97 active+clean; 1.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.2 MiB/s wr, 655 op/s 2024-08-31T06:10:08.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:08 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:10:08.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:08 smithi184 ceph-mon[22331]: pgmap v451: 97 pgs: 97 active+clean; 1.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.2 MiB/s wr, 655 op/s 2024-08-31T06:10:08.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.812+0000 7ff5f0c00640 1 -- 172.21.15.16:0/2936372251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5ec104770 msgr2=0x7ff5ec104b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:08.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.812+0000 7ff5f0c00640 1 --2- 172.21.15.16:0/2936372251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5ec104770 0x7ff5ec104b80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc009900 tx=0x7ff5dc02a3f0 comp rx=0 tx=0).stop 2024-08-31T06:10:08.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.813+0000 7ff5f0c00640 1 -- 172.21.15.16:0/2936372251 shutdown_connections 2024-08-31T06:10:08.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.813+0000 7ff5f0c00640 1 --2- 172.21.15.16:0/2936372251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5ec10dbd0 0x7ff5ec1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:08.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.813+0000 7ff5f0c00640 1 --2- 172.21.15.16:0/2936372251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5ec1050c0 0x7ff5ec10d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:08.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.813+0000 7ff5f0c00640 1 --2- 172.21.15.16:0/2936372251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5ec104770 0x7ff5ec104b80 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:08.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.813+0000 7ff5f0c00640 1 -- 172.21.15.16:0/2936372251 >> 172.21.15.16:0/2936372251 conn(0x7ff5ec0794e0 msgr2=0x7ff5ec0798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:08.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.814+0000 7ff5f0c00640 1 -- 172.21.15.16:0/2936372251 shutdown_connections 2024-08-31T06:10:08.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.814+0000 7ff5f0c00640 1 -- 172.21.15.16:0/2936372251 wait complete. 2024-08-31T06:10:08.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.816+0000 7ff5f0c00640 1 Processor -- start 2024-08-31T06:10:08.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.816+0000 7ff5f0c00640 1 -- start start 2024-08-31T06:10:08.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.816+0000 7ff5f0c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5ec104770 0x7ff5ec1099b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:08.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.816+0000 7ff5f0c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5ec1050c0 0x7ff5ec109ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:08.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.816+0000 7ff5f0c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5ec10dbd0 0x7ff5ec10a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:08.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.816+0000 7ff5f0c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ec06d480 con 0x7ff5ec1050c0 2024-08-31T06:10:08.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.816+0000 7ff5f0c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ec06d300 con 0x7ff5ec10dbd0 2024-08-31T06:10:08.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.816+0000 7ff5f0c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ec06d600 con 0x7ff5ec104770 2024-08-31T06:10:08.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.817+0000 7ff5e9e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5ec104770 0x7ff5ec1099b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:08.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.817+0000 7ff5e9e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5ec104770 0x7ff5ec1099b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:46428/0 (socket says 172.21.15.16:46428) 2024-08-31T06:10:08.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.817+0000 7ff5e9e00640 1 -- 172.21.15.16:0/2674383273 learned_addr learned my addr 172.21.15.16:0/2674383273 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:08.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.817+0000 7ff5ea800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5ec10dbd0 0x7ff5ec10a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:08.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.817+0000 7ff5e3e00640 1 --2- 172.21.15.16:0/2674383273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5ec1050c0 0x7ff5ec109ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:08.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.817+0000 7ff5e9e00640 1 -- 172.21.15.16:0/2674383273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5ec10dbd0 msgr2=0x7ff5ec10a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:08.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.817+0000 7ff5e9e00640 1 --2- 172.21.15.16:0/2674383273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5ec10dbd0 0x7ff5ec10a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:08.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.817+0000 7ff5e9e00640 1 -- 172.21.15.16:0/2674383273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5ec1050c0 msgr2=0x7ff5ec109ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:08.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.817+0000 7ff5e9e00640 1 --2- 172.21.15.16:0/2674383273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5ec1050c0 0x7ff5ec109ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:08.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.817+0000 7ff5e9e00640 1 -- 172.21.15.16:0/2674383273 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5ec1b0d40 con 0x7ff5ec104770 2024-08-31T06:10:08.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.817+0000 7ff5e9e00640 1 --2- 172.21.15.16:0/2674383273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5ec104770 0x7ff5ec1099b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc005ab0 tx=0x7ff5dc003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:08.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.818+0000 7ff5e2a00640 1 -- 172.21.15.16:0/2674383273 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc03f070 con 0x7ff5ec104770 2024-08-31T06:10:08.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.818+0000 7ff5e2a00640 1 -- 172.21.15.16:0/2674383273 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5dc031b50 con 0x7ff5ec104770 2024-08-31T06:10:08.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.818+0000 7ff5f0c00640 1 -- 172.21.15.16:0/2674383273 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5ec1b0fd0 con 0x7ff5ec104770 2024-08-31T06:10:08.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.818+0000 7ff5e2a00640 1 -- 172.21.15.16:0/2674383273 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc035030 con 0x7ff5ec104770 2024-08-31T06:10:08.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.818+0000 7ff5f0c00640 1 -- 172.21.15.16:0/2674383273 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5ec1b1420 con 0x7ff5ec104770 2024-08-31T06:10:08.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.823+0000 7ff5f7897740 1 Processor -- start 2024-08-31T06:10:08.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.823+0000 7ff5e2a00640 1 -- 172.21.15.16:0/2674383273 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7ff5dc042070 con 0x7ff5ec104770 2024-08-31T06:10:08.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.824+0000 7ff5f7897740 1 -- start start 2024-08-31T06:10:08.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.824+0000 7ff5f7897740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f107417070 0x55f1077c5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:08.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.825+0000 7ff5f7897740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1073ee1b0 0x55f1077c55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:08.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.825+0000 7ff5ea800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f107417070 0x55f1077c5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:08.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.825+0000 7ff5ea800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f107417070 0x55f1077c5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:46442/0 (socket says 172.21.15.16:46442) 2024-08-31T06:10:08.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.825+0000 7ff5ea800640 1 -- 172.21.15.16:0/3688440127 learned_addr learned my addr 172.21.15.16:0/3688440127 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:08.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.825+0000 7ff5f7897740 1 --2- 172.21.15.16:0/3688440127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1073f6390 0x55f1077c5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:08.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.826+0000 7ff5f7897740 1 -- 172.21.15.16:0/3688440127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f107733770 con 0x55f1073ee1b0 2024-08-31T06:10:08.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.826+0000 7ff5f7897740 1 -- 172.21.15.16:0/3688440127 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f107722d00 con 0x55f1073f6390 2024-08-31T06:10:08.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.826+0000 7ff5f7897740 1 -- 172.21.15.16:0/3688440127 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f107726cb0 con 0x55f107417070 2024-08-31T06:10:08.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.826+0000 7ff5e3e00640 1 --2- 172.21.15.16:0/3688440127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1073f6390 0x55f1077c5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:08.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.826+0000 7ff5ea800640 1 -- 172.21.15.16:0/3688440127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1073f6390 msgr2=0x55f1077c5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:08.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.826+0000 7ff5ea800640 1 --2- 172.21.15.16:0/3688440127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1073f6390 0x55f1077c5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:08.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.826+0000 7ff5ea800640 1 -- 172.21.15.16:0/3688440127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1073ee1b0 msgr2=0x55f1077c55c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:10:08.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.829+0000 7ff5e9e00640 1 --2- 172.21.15.16:0/3688440127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1073ee1b0 0x55f1077c55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:08.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.829+0000 7ff5ea800640 1 --2- 172.21.15.16:0/3688440127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1073ee1b0 0x55f1077c55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:08.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.829+0000 7ff5ea800640 1 -- 172.21.15.16:0/3688440127 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f107482650 con 0x55f107417070 2024-08-31T06:10:08.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.829+0000 7ff5e9e00640 1 --2- 172.21.15.16:0/3688440127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1073ee1b0 0x55f1077c55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:10:08.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.830+0000 7ff5ea800640 1 --2- 172.21.15.16:0/3688440127 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f107417070 0x55f1077c5080 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4011440 tx=0x7ff5e4011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3df180dd7cc15b5d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:08.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.830+0000 7ff5db000640 1 -- 172.21.15.16:0/3688440127 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e4003c30 con 0x55f107417070 2024-08-31T06:10:08.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.830+0000 7ff5db000640 1 -- 172.21.15.16:0/3688440127 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5e4003dd0 con 0x55f107417070 2024-08-31T06:10:08.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.830+0000 7ff5db000640 1 -- 172.21.15.16:0/3688440127 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e400fae0 con 0x55f107417070 2024-08-31T06:10:08.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.830+0000 7ff5e2a00640 1 --2- 172.21.15.16:0/2674383273 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff5d00b4d20 0x7ff5d00b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:08.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.831+0000 7ff5e2a00640 1 -- 172.21.15.16:0/2674383273 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7ff5dc105b20 con 0x7ff5ec104770 2024-08-31T06:10:08.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.831+0000 7ff5e9e00640 1 --2- 172.21.15.16:0/2674383273 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff5d00b4d20 0x7ff5d00b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:08.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.831+0000 7ff5f7897740 1 -- 172.21.15.16:0/3688440127 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f107417070 msgr2=0x55f1077c5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:08.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.831+0000 7ff5f7897740 1 --2- 172.21.15.16:0/3688440127 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f107417070 0x55f1077c5080 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4011440 tx=0x7ff5e4011900 comp rx=0 tx=0).stop 2024-08-31T06:10:08.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.832+0000 7ff5f7897740 1 -- 172.21.15.16:0/3688440127 shutdown_connections 2024-08-31T06:10:08.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.832+0000 7ff5f7897740 1 --2- 172.21.15.16:0/3688440127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1073f6390 0x55f1077c5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:08.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.832+0000 7ff5f7897740 1 --2- 172.21.15.16:0/3688440127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1073ee1b0 0x55f1077c55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:08.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.832+0000 7ff5f7897740 1 --2- 172.21.15.16:0/3688440127 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f107417070 0x55f1077c5080 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:08.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.832+0000 7ff5f7897740 1 -- 172.21.15.16:0/3688440127 >> 172.21.15.16:0/3688440127 conn(0x55f10740b460 msgr2=0x55f1077c2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:08.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.832+0000 7ff5f7897740 1 -- 172.21.15.16:0/3688440127 shutdown_connections 2024-08-31T06:10:08.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.832+0000 7ff5f7897740 1 -- 172.21.15.16:0/3688440127 wait complete. 2024-08-31T06:10:08.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.833+0000 7ff5f7897740 1 -- start start 2024-08-31T06:10:08.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.833+0000 7ff5f7897740 1 Processor -- start 2024-08-31T06:10:08.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.834+0000 7ff5e9e00640 1 --2- 172.21.15.16:0/2674383273 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff5d00b4d20 0x7ff5d00b7230 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x55f1077c3d00 tx=0x7ff5dc000bc0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:08.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.834+0000 7ff5f7897740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1073ee1b0 0x55f1077d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:08.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.835+0000 7ff5ea800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1073ee1b0 0x55f1077d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:08.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.835+0000 7ff5ea800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1073ee1b0 0x55f1077d0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37620/0 (socket says 172.21.15.16:37620) 2024-08-31T06:10:08.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.835+0000 7ff5ea800640 1 -- 172.21.15.16:0/2284260368 learned_addr learned my addr 172.21.15.16:0/2284260368 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:08.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.835+0000 7ff5f7897740 1 --2- 172.21.15.16:0/2284260368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1073f6390 0x55f1077d1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:08.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.836+0000 7ff5f7897740 1 --2- 172.21.15.16:0/2284260368 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f107417070 0x55f1077dcb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:08.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.836+0000 7ff5f7897740 1 -- 172.21.15.16:0/2284260368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f107585700 con 0x55f1073ee1b0 2024-08-31T06:10:08.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.836+0000 7ff5e3e00640 1 --2- 172.21.15.16:0/2284260368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1073f6390 0x55f1077d1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:08.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.836+0000 7ff5f7897740 1 -- 172.21.15.16:0/2284260368 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f1076a9cb0 con 0x55f1073f6390 2024-08-31T06:10:08.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.836+0000 7ff5f7897740 1 -- 172.21.15.16:0/2284260368 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f1077d3860 con 0x55f107417070 2024-08-31T06:10:08.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.837+0000 7ff5e3e00640 1 -- 172.21.15.16:0/2284260368 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f107417070 msgr2=0x55f1077dcb50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:10:08.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.837+0000 7ff5e3e00640 1 --2- 172.21.15.16:0/2284260368 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f107417070 0x55f1077dcb50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:08.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.837+0000 7ff5e3e00640 1 -- 172.21.15.16:0/2284260368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1073ee1b0 msgr2=0x55f1077d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:08.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.837+0000 7ff5e3e00640 1 --2- 172.21.15.16:0/2284260368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1073ee1b0 0x55f1077d0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:08.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.837+0000 7ff5e3e00640 1 -- 172.21.15.16:0/2284260368 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1076ea870 con 0x55f1073f6390 2024-08-31T06:10:08.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.837+0000 7ff5ea800640 1 --2- 172.21.15.16:0/2284260368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1073ee1b0 0x55f1077d0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:10:08.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.837+0000 7ff5f7897740 10 client.0 _ll_register_callbacks cb 0x55f1076cf430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:10:08.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.838+0000 7ff5f7897740 1 -- 172.21.15.16:0/2284260368 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5e4003dd0 con 0x55f1073f6390 2024-08-31T06:10:08.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.838+0000 7ff5e3e00640 1 --2- 172.21.15.16:0/2284260368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1073f6390 0x55f1077d1290 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec06b460 tx=0x7ff5d4012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4476f7a62b1ed4cf server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:08.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.839+0000 7ff5cf400640 10 client.26222 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:10:08.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.839+0000 7ff5cf400640 1 -- 172.21.15.16:0/2284260368 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d4004050 con 0x55f1073f6390 2024-08-31T06:10:08.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.839+0000 7ff5cf400640 1 -- 172.21.15.16:0/2284260368 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7ff5d40041f0 con 0x55f1073f6390 2024-08-31T06:10:08.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.839+0000 7ff5cf400640 1 -- 172.21.15.16:0/2284260368 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff5d4004490 con 0x55f1073f6390 2024-08-31T06:10:08.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.839+0000 7ff5f7897740 10 client.26222 fetch_fsmap learned FSMap version 31 2024-08-31T06:10:08.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.839+0000 7ff5f7897740 1 -- 172.21.15.16:0/2284260368 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({fsmap=31}) -- 0x55f1077df7a0 con 0x55f1073f6390 2024-08-31T06:10:08.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.840+0000 7ff5cf400640 1 -- 172.21.15.16:0/2284260368 <== mon.2 v2:172.21.15.155:3300/0 4 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7ff5d400f870 con 0x55f1073f6390 2024-08-31T06:10:08.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.840+0000 7ff5f7897740 10 client.26222 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:10:08.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.840+0000 7ff5f7897740 10 client.26222 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:10:08.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.840+0000 7ff5f7897740 20 client.26222 populate_metadata read hostname 'smithi016' 2024-08-31T06:10:08.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.840+0000 7ff5f7897740 1 --2- 172.21.15.16:0/2284260368 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f1073d59f0 0x55f1077e2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:08.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.840+0000 7ff5f7897740 4 client.26222 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:10:08.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.840+0000 7ff5f7897740 1 -- 172.21.15.16:0/2284260368 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1077356c0 con 0x55f1073d59f0 2024-08-31T06:10:08.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.841+0000 7ff5ea800640 1 --2- 172.21.15.16:0/2284260368 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f1073d59f0 0x55f1077e2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:08.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.842+0000 7ff5cf400640 10 client.26222 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:08.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.842+0000 7ff5ea800640 1 --2- 172.21.15.16:0/2284260368 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f1073d59f0 0x55f1077e2c50 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d97223ce4357ca3 server_cookie=c2c4767cd56f4dc2 in_seq=0 out_seq=0 2024-08-31T06:10:08.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.844+0000 7ff5cf400640 1 -- 172.21.15.16:0/2284260368 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f1077356c0 con 0x55f1073d59f0 2024-08-31T06:10:08.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.844+0000 7ff5cf400640 10 client.26222 handle_command_reply: tid=0 2024-08-31T06:10:08.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.844+0000 7ff5f7897740 1 client.26222 shutdown 2024-08-31T06:10:08.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.845+0000 7ff5f7897740 1 -- 172.21.15.16:0/2284260368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1073f6390 msgr2=0x55f1077d1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:08.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.845+0000 7ff5f7897740 1 --2- 172.21.15.16:0/2284260368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1073f6390 0x55f1077d1290 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec06b460 tx=0x7ff5d4012b60 comp rx=0 tx=0).stop 2024-08-31T06:10:08.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.845+0000 7ff5f7897740 1 -- 172.21.15.16:0/2284260368 shutdown_connections 2024-08-31T06:10:08.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.845+0000 7ff5f7897740 1 --2- 172.21.15.16:0/2284260368 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f1073d59f0 0x55f1077e2c50 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:08.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.845+0000 7ff5f7897740 1 --2- 172.21.15.16:0/2284260368 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f107417070 0x55f1077dcb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:08.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.845+0000 7ff5f7897740 1 --2- 172.21.15.16:0/2284260368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1073f6390 0x55f1077d1290 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:08.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.845+0000 7ff5f7897740 1 --2- 172.21.15.16:0/2284260368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1073ee1b0 0x55f1077d0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:08.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.845+0000 7ff5f7897740 1 -- 172.21.15.16:0/2284260368 >> 172.21.15.16:0/2284260368 conn(0x55f10740b460 msgr2=0x55f1077c8210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:08.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.845+0000 7ff5cf400640 0 client.26222 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:08.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.846+0000 7ff5f7897740 1 -- 172.21.15.16:0/2284260368 shutdown_connections 2024-08-31T06:10:08.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.846+0000 7ff5f7897740 1 -- 172.21.15.16:0/2284260368 wait complete. 2024-08-31T06:10:08.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.846+0000 7ff5f7897740 20 client.26222 trim_cache size 0 max 16384 2024-08-31T06:10:08.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:08 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:10:08.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:08 smithi155 ceph-mon[21742]: pgmap v451: 97 pgs: 97 active+clean; 1.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.2 MiB/s wr, 655 op/s 2024-08-31T06:10:08.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.869+0000 7ff5f7897740 1 Processor -- start 2024-08-31T06:10:08.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.870+0000 7ff5f7897740 1 -- start start 2024-08-31T06:10:08.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.870+0000 7ff5f7897740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1073ee1b0 0x55f1077c8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:08.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.870+0000 7ff5f7897740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f1073f6390 0x55f1077c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:08.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.870+0000 7ff5f7897740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f107417070 0x55f1077c2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:08.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.870+0000 7ff5f7897740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ec1b8420 con 0x55f1073ee1b0 2024-08-31T06:10:08.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.870+0000 7ff5f7897740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ec1b5c30 con 0x55f107417070 2024-08-31T06:10:08.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.870+0000 7ff5f7897740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ec1b3aa0 con 0x55f1073f6390 2024-08-31T06:10:08.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.870+0000 7ff5ea800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1073ee1b0 0x55f1077c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:08.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.870+0000 7ff5e3e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f1073f6390 0x55f1077c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:08.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.870+0000 7ff5e3e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f1073f6390 0x55f1077c86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:46456/0 (socket says 172.21.15.16:46456) 2024-08-31T06:10:08.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.870+0000 7ff5ea800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1073ee1b0 0x55f1077c8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37632/0 (socket says 172.21.15.16:37632) 2024-08-31T06:10:08.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.870+0000 7ff5e3e00640 1 -- 172.21.15.16:0/2160165262 learned_addr learned my addr 172.21.15.16:0/2160165262 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:08.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.871+0000 7ff5ea800640 1 --2- 172.21.15.16:0/2160165262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f107417070 0x55f1077c2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:08.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.871+0000 7ff5e3e00640 1 -- 172.21.15.16:0/2160165262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f107417070 msgr2=0x55f1077c2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:08.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.871+0000 7ff5e3e00640 1 --2- 172.21.15.16:0/2160165262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f107417070 0x55f1077c2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:08.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.871+0000 7ff5e3e00640 1 -- 172.21.15.16:0/2160165262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1073ee1b0 msgr2=0x55f1077c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:08.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.871+0000 7ff5e3e00640 1 --2- 172.21.15.16:0/2160165262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1073ee1b0 0x55f1077c8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:08.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.871+0000 7ff5e3e00640 1 -- 172.21.15.16:0/2160165262 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1077b9db0 con 0x55f1073f6390 2024-08-31T06:10:08.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.871+0000 7ff5e3e00640 1 --2- 172.21.15.16:0/2160165262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f1073f6390 0x55f1077c86c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4004690 tx=0x7ff5d4013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=481114459250fed server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.872+0000 7ff5da600640 1 -- 172.21.15.16:0/2160165262 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d400f870 con 0x55f1073f6390 2024-08-31T06:10:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.872+0000 7ff5da600640 1 -- 172.21.15.16:0/2160165262 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5e4003dd0 con 0x55f1073f6390 2024-08-31T06:10:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.872+0000 7ff5da600640 1 -- 172.21.15.16:0/2160165262 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d4005530 con 0x55f1073f6390 2024-08-31T06:10:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.873+0000 7ff5f7897740 1 -- 172.21.15.16:0/2160165262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f1073f6390 msgr2=0x55f1077c86c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.873+0000 7ff5f7897740 1 --2- 172.21.15.16:0/2160165262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f1073f6390 0x55f1077c86c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4004690 tx=0x7ff5d4013000 comp rx=0 tx=0).stop 2024-08-31T06:10:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.873+0000 7ff5f7897740 1 -- 172.21.15.16:0/2160165262 shutdown_connections 2024-08-31T06:10:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.873+0000 7ff5f7897740 1 --2- 172.21.15.16:0/2160165262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f1073f6390 0x55f1077c86c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.873+0000 7ff5f7897740 1 --2- 172.21.15.16:0/2160165262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f107417070 0x55f1077c2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.873+0000 7ff5f7897740 1 --2- 172.21.15.16:0/2160165262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1073ee1b0 0x55f1077c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.873+0000 7ff5f7897740 1 -- 172.21.15.16:0/2160165262 >> 172.21.15.16:0/2160165262 conn(0x55f10740b460 msgr2=0x55f1077cf370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.873+0000 7ff5f7897740 1 -- 172.21.15.16:0/2160165262 shutdown_connections 2024-08-31T06:10:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.873+0000 7ff5f7897740 1 -- 172.21.15.16:0/2160165262 wait complete. 2024-08-31T06:10:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.874+0000 7ff5f7897740 1 -- start start 2024-08-31T06:10:08.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.874+0000 7ff5f7897740 1 Processor -- start 2024-08-31T06:10:08.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.875+0000 7ff5f7897740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1073ee1b0 0x55f1077d9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:08.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.875+0000 7ff5f7897740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f1073f6390 0x55f1077c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:08.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.875+0000 7ff5f7897740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f107417070 0x55f1077ca500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:08.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.875+0000 7ff5f7897740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ec1b40f0 con 0x55f1073ee1b0 2024-08-31T06:10:08.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.875+0000 7ff5f7897740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ec1b5560 con 0x55f107417070 2024-08-31T06:10:08.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.875+0000 7ff5f7897740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ec1b47c0 con 0x55f1073f6390 2024-08-31T06:10:08.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.875+0000 7ff5f7897740 10 client.0 _ll_register_callbacks cb 0x7ff5d4009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:10:08.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.875+0000 7ff5ea800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1073ee1b0 0x55f1077d9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:08.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.875+0000 7ff5e3e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f1073f6390 0x55f1077c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:08.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.875+0000 7ff5ea800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1073ee1b0 0x55f1077d9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37638/0 (socket says 172.21.15.16:37638) 2024-08-31T06:10:08.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.875+0000 7ff5ea800640 1 -- 172.21.15.16:0/546312193 learned_addr learned my addr 172.21.15.16:0/546312193 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:08.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.876+0000 7ff5ea800640 1 --2- 172.21.15.16:0/546312193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f107417070 0x55f1077ca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:08.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.876+0000 7ff5ea800640 1 -- 172.21.15.16:0/546312193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f1073f6390 msgr2=0x55f1077c9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:08.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.876+0000 7ff5ea800640 1 --2- 172.21.15.16:0/546312193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f1073f6390 0x55f1077c9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:08.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.876+0000 7ff5ea800640 1 -- 172.21.15.16:0/546312193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f107417070 msgr2=0x55f1077ca500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:08.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.876+0000 7ff5ea800640 1 --2- 172.21.15.16:0/546312193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f107417070 0x55f1077ca500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:08.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.876+0000 7ff5ea800640 1 -- 172.21.15.16:0/546312193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1077cac70 con 0x55f1073ee1b0 2024-08-31T06:10:08.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.877+0000 7ff5ea800640 1 --2- 172.21.15.16:0/546312193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1073ee1b0 0x55f1077d9260 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4011440 tx=0x7ff5e400f5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b5183705bd3502a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:08.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.877+0000 7ff5cf400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:10:08.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.877+0000 7ff5cf400640 1 -- 172.21.15.16:0/546312193 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e400fd20 con 0x55f1073ee1b0 2024-08-31T06:10:08.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.877+0000 7ff5cf400640 1 -- 172.21.15.16:0/546312193 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7ff5e401f040 con 0x55f1073ee1b0 2024-08-31T06:10:08.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.877+0000 7ff5f7897740 1 -- 172.21.15.16:0/546312193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5e4003dd0 con 0x55f1073ee1b0 2024-08-31T06:10:08.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.878+0000 7ff5cf400640 1 -- 172.21.15.16:0/546312193 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff5e4019430 con 0x55f1073ee1b0 2024-08-31T06:10:08.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.878+0000 7ff5f7897740 10 client.16113 fetch_fsmap learned FSMap version 31 2024-08-31T06:10:08.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.878+0000 7ff5f7897740 1 -- 172.21.15.16:0/546312193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f10784d6a0 con 0x55f1073ee1b0 2024-08-31T06:10:08.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.878+0000 7ff5cf400640 1 -- 172.21.15.16:0/546312193 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5e4003dd0 con 0x55f1073ee1b0 2024-08-31T06:10:08.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.879+0000 7ff5cf400640 1 -- 172.21.15.16:0/546312193 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7ff5e4038400 con 0x55f1073ee1b0 2024-08-31T06:10:08.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.879+0000 7ff5f7897740 10 client.16113 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:10:08.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.879+0000 7ff5f7897740 10 client.16113 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:10:08.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.879+0000 7ff5f7897740 20 client.16113 populate_metadata read hostname 'smithi016' 2024-08-31T06:10:08.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.880+0000 7ff5f7897740 1 --2- 172.21.15.16:0/546312193 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f1073aada0 0x55f10784fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:08.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.880+0000 7ff5f7897740 4 client.16113 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T06:10:08.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.880+0000 7ff5f7897740 1 -- 172.21.15.16:0/546312193 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f107731160 con 0x55f1073aada0 2024-08-31T06:10:08.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.880+0000 7ff5e3e00640 1 --2- 172.21.15.16:0/546312193 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f1073aada0 0x55f10784fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:08.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.881+0000 7ff5e3e00640 1 --2- 172.21.15.16:0/546312193 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f1073aada0 0x55f10784fb80 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79ffd1502c107828 server_cookie=b1e4ca85ffc7e066 in_seq=0 out_seq=0 2024-08-31T06:10:08.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.881+0000 7ff5cf400640 10 client.16113 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:08.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.881+0000 7ff5cf400640 1 -- 172.21.15.16:0/546312193 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1706 (crc 0 0 0) 0x55f107731160 con 0x55f1073aada0 2024-08-31T06:10:08.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.881+0000 7ff5cf400640 10 client.16113 handle_command_reply: tid=0 2024-08-31T06:10:08.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.881+0000 7ff5f7897740 1 client.16113 shutdown 2024-08-31T06:10:08.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.881+0000 7ff5f7897740 1 -- 172.21.15.16:0/546312193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1073ee1b0 msgr2=0x55f1077d9260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:08.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.881+0000 7ff5f7897740 1 --2- 172.21.15.16:0/546312193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1073ee1b0 0x55f1077d9260 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4011440 tx=0x7ff5e400f5e0 comp rx=0 tx=0).stop 2024-08-31T06:10:08.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.882+0000 7ff5f7897740 1 -- 172.21.15.16:0/546312193 shutdown_connections 2024-08-31T06:10:08.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.882+0000 7ff5f7897740 1 --2- 172.21.15.16:0/546312193 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f1073aada0 0x55f10784fb80 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:08.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.882+0000 7ff5f7897740 1 --2- 172.21.15.16:0/546312193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f1073f6390 0x55f1077c9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:08.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.882+0000 7ff5f7897740 1 --2- 172.21.15.16:0/546312193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f107417070 0x55f1077ca500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:08.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.882+0000 7ff5f7897740 1 --2- 172.21.15.16:0/546312193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1073ee1b0 0x55f1077d9260 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:08.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.882+0000 7ff5f7897740 1 -- 172.21.15.16:0/546312193 >> 172.21.15.16:0/546312193 conn(0x55f10740b460 msgr2=0x55f1077c8180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:08.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.882+0000 7ff5cf400640 0 client.16113 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:08.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.882+0000 7ff5f7897740 1 -- 172.21.15.16:0/546312193 shutdown_connections 2024-08-31T06:10:08.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.882+0000 7ff5f7897740 1 -- 172.21.15.16:0/546312193 wait complete. 2024-08-31T06:10:08.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.882+0000 7ff5f7897740 20 client.16113 trim_cache size 0 max 16384 2024-08-31T06:10:08.889 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:10:08.889 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T06:10:08.889 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-08-31T06:10:08.889 INFO:teuthology.orchestra.run.smithi016.stdout: "c1fd6904-2e02-47fa-939a-47349b9fe36b": { 2024-08-31T06:10:08.889 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:10:08.889 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "c1fd6904-2e02-47fa-939a-47349b9fe36b", 2024-08-31T06:10:08.889 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-08-31T06:10:08.889 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:10:08.889 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:10:08.890 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-08-31T06:10:08.890 INFO:teuthology.orchestra.run.smithi016.stdout: "c1fd6904-2e02-47fa-939a-47349b9fe36b": { 2024-08-31T06:10:08.890 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:06.678691+0000", 2024-08-31T06:10:08.890 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:10:08.890 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:10:08.890 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:10:08.890 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:10:08.890 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:10:08.890 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:10:08.890 INFO:teuthology.orchestra.run.smithi016.stdout: "78d6ef02-8551-45e8-ad60-9627eb573fc1": { 2024-08-31T06:10:08.890 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:09:13.547865+0000", 2024-08-31T06:10:08.890 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:10:08.891 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:10:08.891 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:10:08.891 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:10:08.891 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:10:08.891 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:10:08.891 INFO:teuthology.orchestra.run.smithi016.stdout: "c8455d74-2fe7-4d93-bcf2-19b419d16016": { 2024-08-31T06:10:08.891 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:08:24.035960+0000", 2024-08-31T06:10:08.891 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:10:08.891 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:10:08.891 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:10:08.891 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:10:08.891 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:10:08.891 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:10:08.892 INFO:teuthology.orchestra.run.smithi016.stdout: "88d7bdb8-cb1c-45e5-ad29-4cd8da6c6003": { 2024-08-31T06:10:08.892 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:07:39.648942+0000", 2024-08-31T06:10:08.892 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:10:08.892 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:10:08.892 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:10:08.892 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:10:08.892 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:10:08.892 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:10:08.892 INFO:teuthology.orchestra.run.smithi016.stdout: "c300b7e8-f3fd-482c-a909-9329bc61e570": { 2024-08-31T06:10:08.892 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:06:49.868046+0000", 2024-08-31T06:10:08.892 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:10:08.892 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:10:08.893 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:10:08.893 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:10:08.893 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:10:08.893 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:10:08.893 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:10:08.893 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:10:08.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.883+0000 7ff5f0c00640 1 -- 172.21.15.16:0/2674383273 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff5d00b4d20 msgr2=0x7ff5d00b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:08.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.883+0000 7ff5f0c00640 1 --2- 172.21.15.16:0/2674383273 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff5d00b4d20 0x7ff5d00b7230 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x55f1077c3d00 tx=0x7ff5dc000bc0 comp rx=0 tx=0).stop 2024-08-31T06:10:08.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.883+0000 7ff5f0c00640 1 -- 172.21.15.16:0/2674383273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5ec104770 msgr2=0x7ff5ec1099b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:08.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.883+0000 7ff5f0c00640 1 --2- 172.21.15.16:0/2674383273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5ec104770 0x7ff5ec1099b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc005ab0 tx=0x7ff5dc003b90 comp rx=0 tx=0).stop 2024-08-31T06:10:08.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.884+0000 7ff5f0c00640 1 -- 172.21.15.16:0/2674383273 shutdown_connections 2024-08-31T06:10:08.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.884+0000 7ff5f0c00640 1 --2- 172.21.15.16:0/2674383273 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff5d00b4d20 0x7ff5d00b7230 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:08.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.884+0000 7ff5f0c00640 1 --2- 172.21.15.16:0/2674383273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5ec10dbd0 0x7ff5ec10a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:08.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.884+0000 7ff5f0c00640 1 --2- 172.21.15.16:0/2674383273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5ec1050c0 0x7ff5ec109ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:08.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.884+0000 7ff5f0c00640 1 --2- 172.21.15.16:0/2674383273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5ec104770 0x7ff5ec1099b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:08.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.884+0000 7ff5f0c00640 1 -- 172.21.15.16:0/2674383273 >> 172.21.15.16:0/2674383273 conn(0x7ff5ec0794e0 msgr2=0x7ff5ec100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:08.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.884+0000 7ff5f0c00640 1 -- 172.21.15.16:0/2674383273 shutdown_connections 2024-08-31T06:10:08.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:08.884+0000 7ff5f0c00640 1 -- 172.21.15.16:0/2674383273 wait complete. 2024-08-31T06:10:09.906 INFO:tasks.cephfs.filesystem:scrub status for tag:c1fd6904-2e02-47fa-939a-47349b9fe36b - {'path': '/', 'tag': 'c1fd6904-2e02-47fa-939a-47349b9fe36b', 'options': 'recursive,force'} 2024-08-31T06:10:10.051 INFO:tasks.quiescer.fs.[cephfs]:Total duration (43.6) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-08-31T06:10:10.051 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.8 2024-08-31T06:10:10.052 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:10:10.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:10 smithi184 ceph-mon[22331]: pgmap v452: 97 pgs: 97 active+clean; 1.7 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 8.7 MiB/s wr, 873 op/s 2024-08-31T06:10:10.776 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:10:10.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:10 smithi155 ceph-mon[21742]: pgmap v452: 97 pgs: 97 active+clean; 1.7 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 8.7 MiB/s wr, 873 op/s 2024-08-31T06:10:10.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:10 smithi016 ceph-mon[17194]: pgmap v452: 97 pgs: 97 active+clean; 1.7 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 8.7 MiB/s wr, 873 op/s 2024-08-31T06:10:11.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 165.0 s, 735.4 tps, lat 140.981 ms stddev 259.725 2024-08-31T06:10:11.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.678+0000 7f2cada00640 1 -- 172.21.15.16:0/67702754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ca81036b0 msgr2=0x7f2ca8103ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:11.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.678+0000 7f2cada00640 1 --2- 172.21.15.16:0/67702754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ca81036b0 0x7f2ca8103ac0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2c98009900 tx=0x7f2c9802a400 comp rx=0 tx=0).stop 2024-08-31T06:10:11.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.678+0000 7f2cada00640 1 -- 172.21.15.16:0/67702754 shutdown_connections 2024-08-31T06:10:11.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.678+0000 7f2cada00640 1 --2- 172.21.15.16:0/67702754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ca8109930 0x7f2ca8110160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:11.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.678+0000 7f2cada00640 1 --2- 172.21.15.16:0/67702754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ca8108fe0 0x7f2ca81093f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:11.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.678+0000 7f2cada00640 1 --2- 172.21.15.16:0/67702754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ca81036b0 0x7f2ca8103ac0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:11.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.678+0000 7f2cada00640 1 -- 172.21.15.16:0/67702754 >> 172.21.15.16:0/67702754 conn(0x7f2ca80ff0d0 msgr2=0x7f2ca8101520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:11.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.678+0000 7f2cada00640 1 -- 172.21.15.16:0/67702754 shutdown_connections 2024-08-31T06:10:11.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.678+0000 7f2cada00640 1 -- 172.21.15.16:0/67702754 wait complete. 2024-08-31T06:10:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.680+0000 7f2cada00640 1 Processor -- start 2024-08-31T06:10:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.680+0000 7f2cada00640 1 -- start start 2024-08-31T06:10:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.681+0000 7f2cada00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ca81036b0 0x7f2ca810d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:11.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.681+0000 7f2cada00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ca8108fe0 0x7f2ca810ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:11.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.681+0000 7f2cada00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ca8109930 0x7f2ca81ac3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:11.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.681+0000 7f2cada00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca8076fa0 con 0x7f2ca8109930 2024-08-31T06:10:11.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.681+0000 7f2cada00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca8076e20 con 0x7f2ca8108fe0 2024-08-31T06:10:11.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.681+0000 7f2cada00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca8077120 con 0x7f2ca81036b0 2024-08-31T06:10:11.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.682+0000 7f2ca7e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ca81036b0 0x7f2ca810d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:11.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.682+0000 7f2ca7e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ca81036b0 0x7f2ca810d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:46484/0 (socket says 172.21.15.16:46484) 2024-08-31T06:10:11.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.682+0000 7f2ca7e00640 1 -- 172.21.15.16:0/2114207398 learned_addr learned my addr 172.21.15.16:0/2114207398 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:11.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.682+0000 7f2c9fe00640 1 --2- 172.21.15.16:0/2114207398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ca8108fe0 0x7f2ca810ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:11.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.682+0000 7f2ca7e00640 1 -- 172.21.15.16:0/2114207398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ca8108fe0 msgr2=0x7f2ca810ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:11.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.683+0000 7f2ca7e00640 1 --2- 172.21.15.16:0/2114207398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ca8108fe0 0x7f2ca810ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:11.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.683+0000 7f2ca7e00640 1 -- 172.21.15.16:0/2114207398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ca8109930 msgr2=0x7f2ca81ac3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:11.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.683+0000 7f2cad000640 1 --2- 172.21.15.16:0/2114207398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ca8109930 0x7f2ca81ac3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:11.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.683+0000 7f2ca7e00640 1 --2- 172.21.15.16:0/2114207398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ca8109930 0x7f2ca81ac3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:11.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.683+0000 7f2ca7e00640 1 -- 172.21.15.16:0/2114207398 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca81ac8f0 con 0x7f2ca81036b0 2024-08-31T06:10:11.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.683+0000 7f2cad000640 1 --2- 172.21.15.16:0/2114207398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ca8109930 0x7f2ca81ac3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:10:11.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.684+0000 7f2ca7e00640 1 --2- 172.21.15.16:0/2114207398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ca81036b0 0x7f2ca810d880 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2c98005ab0 tx=0x7f2c98003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:11.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.685+0000 7f2ca5600640 1 -- 172.21.15.16:0/2114207398 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9803f070 con 0x7f2ca81036b0 2024-08-31T06:10:11.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.685+0000 7f2ca5600640 1 -- 172.21.15.16:0/2114207398 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c98031b60 con 0x7f2ca81036b0 2024-08-31T06:10:11.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.685+0000 7f2ca5600640 1 -- 172.21.15.16:0/2114207398 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c98035030 con 0x7f2ca81036b0 2024-08-31T06:10:11.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.685+0000 7f2cada00640 1 -- 172.21.15.16:0/2114207398 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ca81acac0 con 0x7f2ca81036b0 2024-08-31T06:10:11.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.686+0000 7f2cada00640 1 -- 172.21.15.16:0/2114207398 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ca81acf30 con 0x7f2ca81036b0 2024-08-31T06:10:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.691+0000 7f2cb6055740 1 Processor -- start 2024-08-31T06:10:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.691+0000 7f2cb6055740 1 -- start start 2024-08-31T06:10:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.691+0000 7f2cb6055740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e243df1b0 0x561e24782b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.691+0000 7f2cb6055740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e243b62f0 0x561e24783040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.691+0000 7f2cb6055740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e243be4d0 0x561e24783580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.691+0000 7f2cb6055740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561e246f7f70 con 0x561e243df1b0 2024-08-31T06:10:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.692+0000 7f2cb6055740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561e246e7500 con 0x561e243b62f0 2024-08-31T06:10:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.692+0000 7f2cb6055740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x561e246eb4b0 con 0x561e243be4d0 2024-08-31T06:10:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.692+0000 7f2ca5600640 1 -- 172.21.15.16:0/2114207398 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f2c98042070 con 0x7f2ca81036b0 2024-08-31T06:10:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.692+0000 7f2cad000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e243df1b0 0x561e24782b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.692+0000 7f2cad000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e243df1b0 0x561e24782b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37664/0 (socket says 172.21.15.16:37664) 2024-08-31T06:10:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.692+0000 7f2cad000640 1 -- 172.21.15.16:0/3571743823 learned_addr learned my addr 172.21.15.16:0/3571743823 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.693+0000 7f2cad000640 1 -- 172.21.15.16:0/3571743823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e243be4d0 msgr2=0x561e24783580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:10:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.693+0000 7f2ca7e00640 1 --2- 172.21.15.16:0/3571743823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e243b62f0 0x561e24783040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.693+0000 7f2c9fe00640 1 --2- 172.21.15.16:0/3571743823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e243be4d0 0x561e24783580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.693+0000 7f2cad000640 1 --2- 172.21.15.16:0/3571743823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e243be4d0 0x561e24783580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.693+0000 7f2cad000640 1 -- 172.21.15.16:0/3571743823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e243b62f0 msgr2=0x561e24783040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.693+0000 7f2cad000640 1 --2- 172.21.15.16:0/3571743823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e243b62f0 0x561e24783040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.693+0000 7f2ca5600640 1 --2- 172.21.15.16:0/2114207398 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f2c8c0b4d20 0x7f2c8c0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.693+0000 7f2cad000640 1 -- 172.21.15.16:0/3571743823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e246a1c10 con 0x561e243df1b0 2024-08-31T06:10:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.693+0000 7f2ca7e00640 1 --2- 172.21.15.16:0/3571743823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e243b62f0 0x561e24783040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:10:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.693+0000 7f2ca5600640 1 -- 172.21.15.16:0/2114207398 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f2c98105970 con 0x7f2ca81036b0 2024-08-31T06:10:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.694+0000 7f2c9fe00640 1 --2- 172.21.15.16:0/3571743823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e243be4d0 0x561e24783580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:10:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.694+0000 7f2cad000640 1 --2- 172.21.15.16:0/2114207398 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f2c8c0b4d20 0x7f2c8c0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.694+0000 7f2cad000640 1 --2- 172.21.15.16:0/3571743823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e243df1b0 0x561e24782b00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0009f10 tx=0x7f2ca000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ea83947f1f2d519 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.694+0000 7f2c9d600640 1 -- 172.21.15.16:0/3571743823 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0011a40 con 0x561e243df1b0 2024-08-31T06:10:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.695+0000 7f2cad000640 1 --2- 172.21.15.16:0/2114207398 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f2c8c0b4d20 0x7f2c8c0b7230 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2ca001e000 tx=0x7f2ca0021000 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.695+0000 7f2c9d600640 1 -- 172.21.15.16:0/3571743823 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ca0011d40 con 0x561e243df1b0 2024-08-31T06:10:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.695+0000 7f2c9d600640 1 -- 172.21.15.16:0/3571743823 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca00103d0 con 0x561e243df1b0 2024-08-31T06:10:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.697+0000 7f2cb6055740 1 -- 172.21.15.16:0/3571743823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e243df1b0 msgr2=0x561e24782b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.697+0000 7f2cb6055740 1 --2- 172.21.15.16:0/3571743823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e243df1b0 0x561e24782b00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0009f10 tx=0x7f2ca000f730 comp rx=0 tx=0).stop 2024-08-31T06:10:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.698+0000 7f2cb6055740 1 -- 172.21.15.16:0/3571743823 shutdown_connections 2024-08-31T06:10:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.698+0000 7f2cb6055740 1 --2- 172.21.15.16:0/3571743823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e243be4d0 0x561e24783580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.698+0000 7f2cb6055740 1 --2- 172.21.15.16:0/3571743823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e243b62f0 0x561e24783040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.698+0000 7f2cb6055740 1 --2- 172.21.15.16:0/3571743823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e243df1b0 0x561e24782b00 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.698+0000 7f2cb6055740 1 -- 172.21.15.16:0/3571743823 >> 172.21.15.16:0/3571743823 conn(0x561e243d3300 msgr2=0x561e247809f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.698+0000 7f2cb6055740 1 -- 172.21.15.16:0/3571743823 shutdown_connections 2024-08-31T06:10:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.698+0000 7f2cb6055740 1 -- 172.21.15.16:0/3571743823 wait complete. 2024-08-31T06:10:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.699+0000 7f2cb6055740 1 -- start start 2024-08-31T06:10:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.699+0000 7f2cb6055740 1 Processor -- start 2024-08-31T06:10:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.700+0000 7f2cb6055740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e243b62f0 0x561e2478e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:11.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.700+0000 7f2cb6055740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e243be4d0 0x561e2478e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:11.703 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.701+0000 7f2cb6055740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e243df1b0 0x561e2478eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.701+0000 7f2cb6055740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561e24620ab0 con 0x561e243df1b0 2024-08-31T06:10:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.701+0000 7f2cb6055740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561e2468e320 con 0x561e243b62f0 2024-08-31T06:10:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.701+0000 7f2cb6055740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x561e247a2290 con 0x561e243be4d0 2024-08-31T06:10:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.701+0000 7f2c9fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e243be4d0 0x561e2478e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.701+0000 7f2c9fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e243be4d0 0x561e2478e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:46504/0 (socket says 172.21.15.16:46504) 2024-08-31T06:10:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.701+0000 7f2c9fe00640 1 -- 172.21.15.16:0/1998197178 learned_addr learned my addr 172.21.15.16:0/1998197178 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.701+0000 7f2ca7e00640 1 --2- 172.21.15.16:0/1998197178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e243b62f0 0x561e2478e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.701+0000 7f2cb6055740 10 client.0 _ll_register_callbacks cb 0x561e246946a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:10:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.701+0000 7f2cad000640 1 --2- 172.21.15.16:0/1998197178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e243df1b0 0x561e2478eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.702+0000 7f2c9fe00640 1 -- 172.21.15.16:0/1998197178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e243b62f0 msgr2=0x561e2478e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.702+0000 7f2c9fe00640 1 --2- 172.21.15.16:0/1998197178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e243b62f0 0x561e2478e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.702+0000 7f2c9fe00640 1 -- 172.21.15.16:0/1998197178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e243df1b0 msgr2=0x561e2478eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.702+0000 7f2c9fe00640 1 --2- 172.21.15.16:0/1998197178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e243df1b0 0x561e2478eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.702+0000 7f2c9fe00640 1 -- 172.21.15.16:0/1998197178 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e2444a790 con 0x561e243be4d0 2024-08-31T06:10:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.702+0000 7f2cad000640 1 --2- 172.21.15.16:0/1998197178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e243df1b0 0x561e2478eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:10:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.703+0000 7f2c9fe00640 1 --2- 172.21.15.16:0/1998197178 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e243be4d0 0x561e2478e630 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2c900136e0 tx=0x7f2c90009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=311986ab414b65bd server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.703+0000 7f2c96000640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:10:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.703+0000 7f2c96000640 1 -- 172.21.15.16:0/1998197178 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c90004460 con 0x561e243be4d0 2024-08-31T06:10:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.703+0000 7f2cb6055740 1 -- 172.21.15.16:0/1998197178 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ca0011d40 con 0x561e243be4d0 2024-08-31T06:10:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.703+0000 7f2c96000640 1 -- 172.21.15.16:0/1998197178 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f2c900119c0 con 0x561e243be4d0 2024-08-31T06:10:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.703+0000 7f2c96000640 1 -- 172.21.15.16:0/1998197178 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2c90011bc0 con 0x561e243be4d0 2024-08-31T06:10:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.703+0000 7f2cb6055740 10 client.26230 fetch_fsmap learned FSMap version 31 2024-08-31T06:10:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.703+0000 7f2cb6055740 1 -- 172.21.15.16:0/1998197178 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561e244c3620 con 0x561e243be4d0 2024-08-31T06:10:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.704+0000 7f2c96000640 1 -- 172.21.15.16:0/1998197178 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ca0011d40 con 0x561e243be4d0 2024-08-31T06:10:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.706+0000 7f2c96000640 1 -- 172.21.15.16:0/1998197178 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f2c9001a070 con 0x561e243be4d0 2024-08-31T06:10:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.706+0000 7f2cb6055740 10 client.26230 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:10:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.706+0000 7f2cb6055740 10 client.26230 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:10:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.706+0000 7f2cb6055740 20 client.26230 populate_metadata read hostname 'smithi016' 2024-08-31T06:10:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.706+0000 7f2cb6055740 1 --2- 172.21.15.16:0/1998197178 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x561e2439dd80 0x561e247b1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.706+0000 7f2cb6055740 4 client.26230 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:10:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.706+0000 7f2cb6055740 1 -- 172.21.15.16:0/1998197178 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e246f9ec0 con 0x561e2439dd80 2024-08-31T06:10:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.707+0000 7f2ca7e00640 1 --2- 172.21.15.16:0/1998197178 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x561e2439dd80 0x561e247b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.707+0000 7f2ca7e00640 1 --2- 172.21.15.16:0/1998197178 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x561e2439dd80 0x561e247b1330 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20a71e0af4aeb353 server_cookie=521544329c0ce8e4 in_seq=0 out_seq=0 2024-08-31T06:10:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.708+0000 7f2c96000640 10 client.26230 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.708+0000 7f2c96000640 1 -- 172.21.15.16:0/1998197178 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561e246f9ec0 con 0x561e2439dd80 2024-08-31T06:10:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.708+0000 7f2c96000640 10 client.26230 handle_command_reply: tid=0 2024-08-31T06:10:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.708+0000 7f2cb6055740 1 client.26230 shutdown 2024-08-31T06:10:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.708+0000 7f2cb6055740 1 -- 172.21.15.16:0/1998197178 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e243be4d0 msgr2=0x561e2478e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.708+0000 7f2cb6055740 1 --2- 172.21.15.16:0/1998197178 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e243be4d0 0x561e2478e630 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2c900136e0 tx=0x7f2c90009f90 comp rx=0 tx=0).stop 2024-08-31T06:10:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.709+0000 7f2cb6055740 1 -- 172.21.15.16:0/1998197178 shutdown_connections 2024-08-31T06:10:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.709+0000 7f2cb6055740 1 --2- 172.21.15.16:0/1998197178 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x561e2439dd80 0x561e247b1330 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.709+0000 7f2cb6055740 1 --2- 172.21.15.16:0/1998197178 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e243be4d0 0x561e2478e630 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.709+0000 7f2cb6055740 1 --2- 172.21.15.16:0/1998197178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e243df1b0 0x561e2478eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.709+0000 7f2cb6055740 1 --2- 172.21.15.16:0/1998197178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e243b62f0 0x561e2478e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.709+0000 7f2cb6055740 1 -- 172.21.15.16:0/1998197178 >> 172.21.15.16:0/1998197178 conn(0x561e243d3300 msgr2=0x561e24783b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.709+0000 7f2c96000640 0 client.26230 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.709+0000 7f2cb6055740 1 -- 172.21.15.16:0/1998197178 shutdown_connections 2024-08-31T06:10:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.709+0000 7f2cb6055740 1 -- 172.21.15.16:0/1998197178 wait complete. 2024-08-31T06:10:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.709+0000 7f2cb6055740 20 client.26230 trim_cache size 0 max 16384 2024-08-31T06:10:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.741+0000 7f2cb6055740 1 Processor -- start 2024-08-31T06:10:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.741+0000 7f2cb6055740 1 -- start start 2024-08-31T06:10:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.742+0000 7f2cb6055740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e243b62f0 0x561e247844f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.742+0000 7f2cb6055740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e243be4d0 0x561e2478fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.742+0000 7f2cb6055740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e243df1b0 0x561e247ab500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.742+0000 7f2cb6055740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca81af9a0 con 0x561e243df1b0 2024-08-31T06:10:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.742+0000 7f2cb6055740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca81ad1b0 con 0x561e243b62f0 2024-08-31T06:10:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.742+0000 7f2cb6055740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca8073240 con 0x561e243be4d0 2024-08-31T06:10:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.743+0000 7f2ca7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e243b62f0 0x561e247844f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.743+0000 7f2ca7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e243b62f0 0x561e247844f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:59458/0 (socket says 172.21.15.16:59458) 2024-08-31T06:10:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.743+0000 7f2ca7e00640 1 -- 172.21.15.16:0/2106006119 learned_addr learned my addr 172.21.15.16:0/2106006119 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.743+0000 7f2c9fe00640 1 --2- 172.21.15.16:0/2106006119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e243be4d0 0x561e2478fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.743+0000 7f2cad000640 1 --2- 172.21.15.16:0/2106006119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e243df1b0 0x561e247ab500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.743+0000 7f2ca7e00640 1 -- 172.21.15.16:0/2106006119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e243be4d0 msgr2=0x561e2478fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.743+0000 7f2ca7e00640 1 --2- 172.21.15.16:0/2106006119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e243be4d0 0x561e2478fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.743+0000 7f2ca7e00640 1 -- 172.21.15.16:0/2106006119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e243df1b0 msgr2=0x561e247ab500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.743+0000 7f2ca7e00640 1 --2- 172.21.15.16:0/2106006119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e243df1b0 0x561e247ab500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.743+0000 7f2ca7e00640 1 -- 172.21.15.16:0/2106006119 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e2439dfb0 con 0x561e243b62f0 2024-08-31T06:10:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.743+0000 7f2cad000640 1 --2- 172.21.15.16:0/2106006119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e243df1b0 0x561e247ab500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:10:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.744+0000 7f2ca7e00640 1 --2- 172.21.15.16:0/2106006119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e243b62f0 0x561e247844f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2c98000c00 tx=0x7f2c980045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99526c9c7f69e38d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.744+0000 7f2c9cc00640 1 -- 172.21.15.16:0/2106006119 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c98049040 con 0x561e243b62f0 2024-08-31T06:10:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.744+0000 7f2c9cc00640 1 -- 172.21.15.16:0/2106006119 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c980b1e50 con 0x561e243b62f0 2024-08-31T06:10:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.744+0000 7f2c9cc00640 1 -- 172.21.15.16:0/2106006119 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c980c82f0 con 0x561e243b62f0 2024-08-31T06:10:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.745+0000 7f2cb6055740 1 -- 172.21.15.16:0/2106006119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e243b62f0 msgr2=0x561e247844f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.745+0000 7f2cb6055740 1 --2- 172.21.15.16:0/2106006119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e243b62f0 0x561e247844f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2c98000c00 tx=0x7f2c980045e0 comp rx=0 tx=0).stop 2024-08-31T06:10:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.746+0000 7f2cb6055740 1 -- 172.21.15.16:0/2106006119 shutdown_connections 2024-08-31T06:10:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.746+0000 7f2cb6055740 1 --2- 172.21.15.16:0/2106006119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e243be4d0 0x561e2478fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.746+0000 7f2cb6055740 1 --2- 172.21.15.16:0/2106006119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e243df1b0 0x561e247ab500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.746+0000 7f2cb6055740 1 --2- 172.21.15.16:0/2106006119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e243b62f0 0x561e247844f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.746+0000 7f2cb6055740 1 -- 172.21.15.16:0/2106006119 >> 172.21.15.16:0/2106006119 conn(0x561e243d3300 msgr2=0x561e2478f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.746+0000 7f2cb6055740 1 -- 172.21.15.16:0/2106006119 shutdown_connections 2024-08-31T06:10:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.746+0000 7f2cb6055740 1 -- 172.21.15.16:0/2106006119 wait complete. 2024-08-31T06:10:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.747+0000 7f2cb6055740 1 -- start start 2024-08-31T06:10:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.747+0000 7f2cb6055740 1 Processor -- start 2024-08-31T06:10:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.749+0000 7f2cb6055740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e243b62f0 0x561e247a4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:11.752 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.749+0000 7f2cb6055740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e243be4d0 0x561e247a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:11.752 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.749+0000 7f2cb6055740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e243df1b0 0x561e24819d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.750+0000 7f2cb6055740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca8073890 con 0x561e243b62f0 2024-08-31T06:10:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.750+0000 7f2cb6055740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca8074d00 con 0x561e243df1b0 2024-08-31T06:10:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.750+0000 7f2cb6055740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca8073f60 con 0x561e243be4d0 2024-08-31T06:10:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.750+0000 7f2cb6055740 10 client.0 _ll_register_callbacks cb 0x7f2c90009e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:10:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.750+0000 7f2c9fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e243be4d0 0x561e247a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.751+0000 7f2c9fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e243be4d0 0x561e247a4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:46524/0 (socket says 172.21.15.16:46524) 2024-08-31T06:10:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.751+0000 7f2c9fe00640 1 -- 172.21.15.16:0/3822623866 learned_addr learned my addr 172.21.15.16:0/3822623866 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.751+0000 7f2cad000640 1 --2- 172.21.15.16:0/3822623866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e243df1b0 0x561e24819d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.751+0000 7f2ca7e00640 1 --2- 172.21.15.16:0/3822623866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e243b62f0 0x561e247a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.751+0000 7f2c9fe00640 1 -- 172.21.15.16:0/3822623866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e243df1b0 msgr2=0x561e24819d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.751+0000 7f2c9fe00640 1 --2- 172.21.15.16:0/3822623866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e243df1b0 0x561e24819d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.751+0000 7f2c9fe00640 1 -- 172.21.15.16:0/3822623866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e243b62f0 msgr2=0x561e247a4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.751+0000 7f2c9fe00640 1 --2- 172.21.15.16:0/3822623866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e243b62f0 0x561e247a4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.751+0000 7f2c9fe00640 1 -- 172.21.15.16:0/3822623866 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e247a55d0 con 0x561e243be4d0 2024-08-31T06:10:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.751+0000 7f2ca7e00640 1 --2- 172.21.15.16:0/3822623866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e243b62f0 0x561e247a4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:10:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.752+0000 7f2c9fe00640 1 --2- 172.21.15.16:0/3822623866 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e243be4d0 0x561e247a4d50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2c90001eb0 tx=0x7f2c90011b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f37fd9b0cf5e4c12 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.752+0000 7f2c96000640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:10:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.752+0000 7f2c96000640 1 -- 172.21.15.16:0/3822623866 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c90029040 con 0x561e243be4d0 2024-08-31T06:10:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.752+0000 7f2c96000640 1 -- 172.21.15.16:0/3822623866 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f2c900052e0 con 0x561e243be4d0 2024-08-31T06:10:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.752+0000 7f2cb6055740 1 -- 172.21.15.16:0/3822623866 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c980b1e50 con 0x561e243be4d0 2024-08-31T06:10:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.753+0000 7f2c96000640 1 -- 172.21.15.16:0/3822623866 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2c900054e0 con 0x561e243be4d0 2024-08-31T06:10:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.753+0000 7f2cb6055740 10 client.26236 fetch_fsmap learned FSMap version 31 2024-08-31T06:10:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.753+0000 7f2cb6055740 1 -- 172.21.15.16:0/3822623866 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561e2481b270 con 0x561e243be4d0 2024-08-31T06:10:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.754+0000 7f2c96000640 1 -- 172.21.15.16:0/3822623866 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c980b1e50 con 0x561e243be4d0 2024-08-31T06:10:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.754+0000 7f2c96000640 1 -- 172.21.15.16:0/3822623866 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f2c9001a040 con 0x561e243be4d0 2024-08-31T06:10:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.754+0000 7f2cb6055740 10 client.26236 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:10:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.754+0000 7f2cb6055740 10 client.26236 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:10:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.754+0000 7f2cb6055740 20 client.26236 populate_metadata read hostname 'smithi016' 2024-08-31T06:10:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.754+0000 7f2cb6055740 1 --2- 172.21.15.16:0/3822623866 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x561e243731c0 0x561e2481dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.755+0000 7f2cb6055740 4 client.26236 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:10:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.755+0000 7f2cb6055740 1 -- 172.21.15.16:0/3822623866 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561e246f5960 con 0x561e243731c0 2024-08-31T06:10:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.755+0000 7f2ca7e00640 1 --2- 172.21.15.16:0/3822623866 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x561e243731c0 0x561e2481dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.756+0000 7f2ca7e00640 1 --2- 172.21.15.16:0/3822623866 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x561e243731c0 0x561e2481dab0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b21367f567d17151 server_cookie=cd6752e13f4a7b38 in_seq=0 out_seq=0 2024-08-31T06:10:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.756+0000 7f2c96000640 10 client.26236 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.786+0000 7f2c96000640 1 -- 172.21.15.16:0/3822623866 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561e246f5960 con 0x561e243731c0 2024-08-31T06:10:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.786+0000 7f2c96000640 10 client.26236 handle_command_reply: tid=0 2024-08-31T06:10:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.786+0000 7f2cb6055740 1 client.26236 shutdown 2024-08-31T06:10:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.786+0000 7f2cb6055740 1 -- 172.21.15.16:0/3822623866 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e243be4d0 msgr2=0x561e247a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.786+0000 7f2cb6055740 1 --2- 172.21.15.16:0/3822623866 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e243be4d0 0x561e247a4d50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2c90001eb0 tx=0x7f2c90011b20 comp rx=0 tx=0).stop 2024-08-31T06:10:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.786+0000 7f2cb6055740 1 -- 172.21.15.16:0/3822623866 shutdown_connections 2024-08-31T06:10:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.786+0000 7f2cb6055740 1 --2- 172.21.15.16:0/3822623866 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x561e243731c0 0x561e2481dab0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.786+0000 7f2cb6055740 1 --2- 172.21.15.16:0/3822623866 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e243be4d0 0x561e247a4d50 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.787+0000 7f2cb6055740 1 --2- 172.21.15.16:0/3822623866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e243df1b0 0x561e24819d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.787+0000 7f2cb6055740 1 --2- 172.21.15.16:0/3822623866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e243b62f0 0x561e247a4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.787+0000 7f2cb6055740 1 -- 172.21.15.16:0/3822623866 >> 172.21.15.16:0/3822623866 conn(0x561e243d3300 msgr2=0x561e247842c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.787+0000 7f2c96000640 0 client.26236 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.787+0000 7f2cb6055740 1 -- 172.21.15.16:0/3822623866 shutdown_connections 2024-08-31T06:10:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.787+0000 7f2cb6055740 1 -- 172.21.15.16:0/3822623866 wait complete. 2024-08-31T06:10:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.787+0000 7f2cb6055740 20 client.26236 trim_cache size 0 max 16384 2024-08-31T06:10:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:10:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:10:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:10:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:10:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.788+0000 7f2cada00640 1 -- 172.21.15.16:0/2114207398 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f2c8c0b4d20 msgr2=0x7f2c8c0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.788+0000 7f2cada00640 1 --2- 172.21.15.16:0/2114207398 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f2c8c0b4d20 0x7f2c8c0b7230 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2ca001e000 tx=0x7f2ca0021000 comp rx=0 tx=0).stop 2024-08-31T06:10:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.788+0000 7f2cada00640 1 -- 172.21.15.16:0/2114207398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ca81036b0 msgr2=0x7f2ca810d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.788+0000 7f2cada00640 1 --2- 172.21.15.16:0/2114207398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ca81036b0 0x7f2ca810d880 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2c98005ab0 tx=0x7f2c98003c10 comp rx=0 tx=0).stop 2024-08-31T06:10:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.788+0000 7f2cada00640 1 -- 172.21.15.16:0/2114207398 shutdown_connections 2024-08-31T06:10:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.788+0000 7f2cada00640 1 --2- 172.21.15.16:0/2114207398 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f2c8c0b4d20 0x7f2c8c0b7230 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.788+0000 7f2cada00640 1 --2- 172.21.15.16:0/2114207398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ca8109930 0x7f2ca81ac3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.788+0000 7f2cada00640 1 --2- 172.21.15.16:0/2114207398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ca8108fe0 0x7f2ca810ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.788+0000 7f2cada00640 1 --2- 172.21.15.16:0/2114207398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ca81036b0 0x7f2ca810d880 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.788+0000 7f2cada00640 1 -- 172.21.15.16:0/2114207398 >> 172.21.15.16:0/2114207398 conn(0x7f2ca80ff0d0 msgr2=0x7f2ca81014f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.789+0000 7f2cada00640 1 -- 172.21.15.16:0/2114207398 shutdown_connections 2024-08-31T06:10:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:11.789+0000 7f2cada00640 1 -- 172.21.15.16:0/2114207398 wait complete. 2024-08-31T06:10:12.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.446+0000 7fd932200640 1 -- 172.21.15.16:0/2278403870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd92c103920 msgr2=0x7fd92c103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:12.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.446+0000 7fd932200640 1 --2- 172.21.15.16:0/2278403870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd92c103920 0x7fd92c103d30 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd918009900 tx=0x7fd91802a440 comp rx=0 tx=0).stop 2024-08-31T06:10:12.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.449+0000 7fd932200640 1 -- 172.21.15.16:0/2278403870 shutdown_connections 2024-08-31T06:10:12.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.449+0000 7fd932200640 1 --2- 172.21.15.16:0/2278403870 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd92c109bb0 0x7fd92c1103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:12.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.449+0000 7fd932200640 1 --2- 172.21.15.16:0/2278403870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd92c109260 0x7fd92c109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:12.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.449+0000 7fd932200640 1 --2- 172.21.15.16:0/2278403870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd92c103920 0x7fd92c103d30 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:12.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.449+0000 7fd932200640 1 -- 172.21.15.16:0/2278403870 >> 172.21.15.16:0/2278403870 conn(0x7fd92c0ff330 msgr2=0x7fd92c101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:12.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.449+0000 7fd932200640 1 -- 172.21.15.16:0/2278403870 shutdown_connections 2024-08-31T06:10:12.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.450+0000 7fd932200640 1 -- 172.21.15.16:0/2278403870 wait complete. 2024-08-31T06:10:12.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.451+0000 7fd932200640 1 Processor -- start 2024-08-31T06:10:12.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.452+0000 7fd932200640 1 -- start start 2024-08-31T06:10:12.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.453+0000 7fd932200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd92c103920 0x7fd92c198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:12.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.453+0000 7fd932200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd92c109260 0x7fd92c1992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:12.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.454+0000 7fd932200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd92c109bb0 0x7fd92c199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:12.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.454+0000 7fd932200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd92c1138c0 con 0x7fd92c109bb0 2024-08-31T06:10:12.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.454+0000 7fd932200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd92c113740 con 0x7fd92c109260 2024-08-31T06:10:12.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.454+0000 7fd92be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd92c109bb0 0x7fd92c199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:12.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.454+0000 7fd92b400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd92c103920 0x7fd92c198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:12.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.454+0000 7fd92aa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd92c109260 0x7fd92c1992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:12.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.455+0000 7fd92b400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd92c103920 0x7fd92c198d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:46542/0 (socket says 172.21.15.16:46542) 2024-08-31T06:10:12.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.455+0000 7fd92b400640 1 -- 172.21.15.16:0/2083278307 learned_addr learned my addr 172.21.15.16:0/2083278307 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:12.459 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:12 smithi016 ceph-mon[17194]: pgmap v453: 97 pgs: 97 active+clean; 1.7 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 13 MiB/s wr, 985 op/s 2024-08-31T06:10:12.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.456+0000 7fd932200640 1 -- 172.21.15.16:0/2083278307 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd92c113a40 con 0x7fd92c103920 2024-08-31T06:10:12.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.457+0000 7fd92b400640 1 -- 172.21.15.16:0/2083278307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd92c109260 msgr2=0x7fd92c1992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:12.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.457+0000 7fd92b400640 1 --2- 172.21.15.16:0/2083278307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd92c109260 0x7fd92c1992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:12.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.457+0000 7fd92b400640 1 -- 172.21.15.16:0/2083278307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd92c109bb0 msgr2=0x7fd92c199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:12.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.457+0000 7fd92b400640 1 --2- 172.21.15.16:0/2083278307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd92c109bb0 0x7fd92c199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:12.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.457+0000 7fd92b400640 1 -- 172.21.15.16:0/2083278307 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd92c1a0000 con 0x7fd92c103920 2024-08-31T06:10:12.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.458+0000 7fd92aa00640 1 --2- 172.21.15.16:0/2083278307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd92c109260 0x7fd92c1992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:10:12.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.458+0000 7fd92b400640 1 --2- 172.21.15.16:0/2083278307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd92c103920 0x7fd92c198d70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd918000c00 tx=0x7fd918003fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:12.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.459+0000 7fd923e00640 1 -- 172.21.15.16:0/2083278307 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd918040030 con 0x7fd92c103920 2024-08-31T06:10:12.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.459+0000 7fd923e00640 1 -- 172.21.15.16:0/2083278307 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd91803c8f0 con 0x7fd92c103920 2024-08-31T06:10:12.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.459+0000 7fd923e00640 1 -- 172.21.15.16:0/2083278307 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9180319d0 con 0x7fd92c103920 2024-08-31T06:10:12.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.459+0000 7fd932200640 1 -- 172.21.15.16:0/2083278307 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd92c1a0290 con 0x7fd92c103920 2024-08-31T06:10:12.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.459+0000 7fd932200640 1 -- 172.21.15.16:0/2083278307 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd92c1a0720 con 0x7fd92c103920 2024-08-31T06:10:12.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.461+0000 7fd932200640 1 -- 172.21.15.16:0/2083278307 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd92c103d30 con 0x7fd92c103920 2024-08-31T06:10:12.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.467+0000 7fd923e00640 1 -- 172.21.15.16:0/2083278307 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fd918043030 con 0x7fd92c103920 2024-08-31T06:10:12.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.470+0000 7fd923e00640 1 --2- 172.21.15.16:0/2083278307 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd9100b4d50 0x7fd9100b7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:12.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.470+0000 7fd923e00640 1 -- 172.21.15.16:0/2083278307 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fd918106340 con 0x7fd92c103920 2024-08-31T06:10:12.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.471+0000 7fd923e00640 1 -- 172.21.15.16:0/2083278307 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fd918106860 con 0x7fd92c103920 2024-08-31T06:10:12.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.471+0000 7fd92aa00640 1 --2- 172.21.15.16:0/2083278307 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd9100b4d50 0x7fd9100b7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:12.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.474+0000 7fd92aa00640 1 --2- 172.21.15.16:0/2083278307 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd9100b4d50 0x7fd9100b7260 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd92c109ae0 tx=0x7fd91c005d90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:12.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:12 smithi184 ceph-mon[22331]: pgmap v453: 97 pgs: 97 active+clean; 1.7 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 13 MiB/s wr, 985 op/s 2024-08-31T06:10:12.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.856+0000 7fd932200640 1 -- 172.21.15.16:0/2083278307 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd92c10fab0 con 0x7fd92c103920 2024-08-31T06:10:12.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.862+0000 7fd923e00640 1 -- 172.21.15.16:0/2083278307 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11923 (secure 0 0 0) 0x7fd92c10fab0 con 0x7fd92c103920 2024-08-31T06:10:12.867 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:10:12.867 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":31,"btime":"2024-08-31T06:08:47:452652+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:08:47.004817+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24737,24724,24712]},"id":1}]} 2024-08-31T06:10:12.867 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 31 2024-08-31T06:10:12.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.867+0000 7fd932200640 1 -- 172.21.15.16:0/2083278307 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd9100b4d50 msgr2=0x7fd9100b7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:12.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.867+0000 7fd932200640 1 --2- 172.21.15.16:0/2083278307 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd9100b4d50 0x7fd9100b7260 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd92c109ae0 tx=0x7fd91c005d90 comp rx=0 tx=0).stop 2024-08-31T06:10:12.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.867+0000 7fd932200640 1 -- 172.21.15.16:0/2083278307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd92c103920 msgr2=0x7fd92c198d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:12.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.867+0000 7fd932200640 1 --2- 172.21.15.16:0/2083278307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd92c103920 0x7fd92c198d70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd918000c00 tx=0x7fd918003fd0 comp rx=0 tx=0).stop 2024-08-31T06:10:12.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.868+0000 7fd932200640 1 -- 172.21.15.16:0/2083278307 shutdown_connections 2024-08-31T06:10:12.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.868+0000 7fd932200640 1 --2- 172.21.15.16:0/2083278307 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd9100b4d50 0x7fd9100b7260 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:12.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.868+0000 7fd932200640 1 --2- 172.21.15.16:0/2083278307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd92c109bb0 0x7fd92c199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:12.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.868+0000 7fd932200640 1 --2- 172.21.15.16:0/2083278307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd92c109260 0x7fd92c1992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:12.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.868+0000 7fd932200640 1 --2- 172.21.15.16:0/2083278307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd92c103920 0x7fd92c198d70 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:12.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.868+0000 7fd932200640 1 -- 172.21.15.16:0/2083278307 >> 172.21.15.16:0/2083278307 conn(0x7fd92c0ff330 msgr2=0x7fd92c107a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:12.871 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:12 smithi155 ceph-mon[21742]: pgmap v453: 97 pgs: 97 active+clean; 1.7 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 13 MiB/s wr, 985 op/s 2024-08-31T06:10:12.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.869+0000 7fd932200640 1 -- 172.21.15.16:0/2083278307 shutdown_connections 2024-08-31T06:10:12.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:12.869+0000 7fd932200640 1 -- 172.21.15.16:0/2083278307 wait complete. 2024-08-31T06:10:13.457 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:13 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/2083278307' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:10:13.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:13 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/2083278307' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:10:13.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:13 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:10:13.791 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:13 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:10:13.853 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24737 quiesce db / --timeout 90 --expiration 141.8 --await-for 45.0' 2024-08-31T06:10:13.854 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24737 quiesce db / --timeout 90 --expiration 141.8 --await-for 45.0 2024-08-31T06:10:13.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:13 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/2083278307' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:10:13.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:13 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:10:14.542 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:10:14.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:14 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:10:14.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:14 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:10:14.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:14 smithi184 ceph-mon[22331]: pgmap v454: 97 pgs: 97 active+clean; 1.8 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 26 MiB/s wr, 1.45k op/s 2024-08-31T06:10:14.741 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:14 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:10:14.741 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:14 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:10:14.741 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:14 smithi016 ceph-mon[17194]: pgmap v454: 97 pgs: 97 active+clean; 1.8 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 26 MiB/s wr, 1.45k op/s 2024-08-31T06:10:14.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:14 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:10:14.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:14 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:10:14.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:14 smithi155 ceph-mon[21742]: pgmap v454: 97 pgs: 97 active+clean; 1.8 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 26 MiB/s wr, 1.45k op/s 2024-08-31T06:10:15.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.806+0000 7f4f0ec00640 1 -- 172.21.15.16:0/669111738 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f08102640 msgr2=0x7f4f08102a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:15.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.806+0000 7f4f0ec00640 1 --2- 172.21.15.16:0/669111738 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f08102640 0x7f4f08102a50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4efc009900 tx=0x7f4efc02a3f0 comp rx=0 tx=0).stop 2024-08-31T06:10:15.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.807+0000 7f4f0ec00640 1 -- 172.21.15.16:0/669111738 shutdown_connections 2024-08-31T06:10:15.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.807+0000 7f4f0ec00640 1 --2- 172.21.15.16:0/669111738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f081088d0 0x7f4f0810f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:15.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.807+0000 7f4f0ec00640 1 --2- 172.21.15.16:0/669111738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f08107f80 0x7f4f08108390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:15.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.807+0000 7f4f0ec00640 1 --2- 172.21.15.16:0/669111738 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f08102640 0x7f4f08102a50 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:15.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.807+0000 7f4f0ec00640 1 -- 172.21.15.16:0/669111738 >> 172.21.15.16:0/669111738 conn(0x7f4f080fe0d0 msgr2=0x7f4f08100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:15.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.809+0000 7f4f0ec00640 1 -- 172.21.15.16:0/669111738 shutdown_connections 2024-08-31T06:10:15.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.809+0000 7f4f0ec00640 1 -- 172.21.15.16:0/669111738 wait complete. 2024-08-31T06:10:15.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.812+0000 7f4f0ec00640 1 Processor -- start 2024-08-31T06:10:15.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.813+0000 7f4f0ec00640 1 -- start start 2024-08-31T06:10:15.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.814+0000 7f4f0ec00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f08102640 0x7f4f0819c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:15.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.815+0000 7f4f0ec00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f08107f80 0x7f4f0819c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:15.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.815+0000 7f4f0ec00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f081088d0 0x7f4f081a1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:15.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.816+0000 7f4f0ec00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4f081136f0 con 0x7f4f081088d0 2024-08-31T06:10:15.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.816+0000 7f4f0ec00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4f08113570 con 0x7f4f08107f80 2024-08-31T06:10:15.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.816+0000 7f4f0ce00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f08107f80 0x7f4f0819c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:15.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.816+0000 7f4f0ce00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f08107f80 0x7f4f0819c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:38660/0 (socket says 172.21.15.16:38660) 2024-08-31T06:10:15.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.816+0000 7f4f0ce00640 1 -- 172.21.15.16:0/3956388004 learned_addr learned my addr 172.21.15.16:0/3956388004 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:15.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.816+0000 7f4f0ec00640 1 -- 172.21.15.16:0/3956388004 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4f08113870 con 0x7f4f08102640 2024-08-31T06:10:15.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.816+0000 7f4f0d800640 1 --2- 172.21.15.16:0/3956388004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f08102640 0x7f4f0819c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:15.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.817+0000 7f4f0e200640 1 --2- 172.21.15.16:0/3956388004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f081088d0 0x7f4f081a1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:15.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.817+0000 7f4f0ce00640 1 -- 172.21.15.16:0/3956388004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f08102640 msgr2=0x7f4f0819c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:15.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.817+0000 7f4f0ce00640 1 --2- 172.21.15.16:0/3956388004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f08102640 0x7f4f0819c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:15.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.817+0000 7f4f0ce00640 1 -- 172.21.15.16:0/3956388004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f081088d0 msgr2=0x7f4f081a1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:15.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.817+0000 7f4f0ce00640 1 --2- 172.21.15.16:0/3956388004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f081088d0 0x7f4f081a1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:15.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.817+0000 7f4f0ce00640 1 -- 172.21.15.16:0/3956388004 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f081a2290 con 0x7f4f08107f80 2024-08-31T06:10:15.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.817+0000 7f4f0e200640 1 --2- 172.21.15.16:0/3956388004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f081088d0 0x7f4f081a1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:10:15.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.818+0000 7f4f0ce00640 1 --2- 172.21.15.16:0/3956388004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f08107f80 0x7f4f0819c8c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4ef400d740 tx=0x7f4ef400dc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:15.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.819+0000 7f4f06000640 1 -- 172.21.15.16:0/3956388004 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef4017070 con 0x7f4f08107f80 2024-08-31T06:10:15.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.820+0000 7f4f06000640 1 -- 172.21.15.16:0/3956388004 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ef400de50 con 0x7f4f08107f80 2024-08-31T06:10:15.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.820+0000 7f4f06000640 1 -- 172.21.15.16:0/3956388004 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef400f620 con 0x7f4f08107f80 2024-08-31T06:10:15.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.820+0000 7f4f0ec00640 1 -- 172.21.15.16:0/3956388004 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f081a2580 con 0x7f4f08107f80 2024-08-31T06:10:15.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.820+0000 7f4f0ec00640 1 -- 172.21.15.16:0/3956388004 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f081a8ce0 con 0x7f4f08107f80 2024-08-31T06:10:15.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.824+0000 7f4f16e0d740 1 Processor -- start 2024-08-31T06:10:15.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.829+0000 7f4f16e0d740 1 -- start start 2024-08-31T06:10:15.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.830+0000 7f4f16e0d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614ad8b5840 0x5614adc5c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:15.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.830+0000 7f4f16e0d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614ad88c980 0x5614adc5c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:15.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.830+0000 7f4f16e0d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614ad894b60 0x5614adc5cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:15.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.830+0000 7f4f16e0d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5614adbd1530 con 0x5614ad8b5840 2024-08-31T06:10:15.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.830+0000 7f4f16e0d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5614adbc0ac0 con 0x5614ad88c980 2024-08-31T06:10:15.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.830+0000 7f4f16e0d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5614adbc4a70 con 0x5614ad894b60 2024-08-31T06:10:15.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.831+0000 7f4f0d800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614ad88c980 0x5614adc5c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:15.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.831+0000 7f4f0d800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614ad88c980 0x5614adc5c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:38662/0 (socket says 172.21.15.16:38662) 2024-08-31T06:10:15.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.831+0000 7f4f0d800640 1 -- 172.21.15.16:0/2435478556 learned_addr learned my addr 172.21.15.16:0/2435478556 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:15.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.832+0000 7f4f0d800640 1 -- 172.21.15.16:0/2435478556 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614ad894b60 msgr2=0x5614adc5cd50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T06:10:15.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.832+0000 7f4f0d800640 1 --2- 172.21.15.16:0/2435478556 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614ad894b60 0x5614adc5cd50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:15.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.832+0000 7f4f0d800640 1 -- 172.21.15.16:0/2435478556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614ad8b5840 msgr2=0x5614adc5c2d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:10:15.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.832+0000 7f4f0e200640 1 --2- 172.21.15.16:0/2435478556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614ad8b5840 0x5614adc5c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:15.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.832+0000 7f4f0d800640 1 --2- 172.21.15.16:0/2435478556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614ad8b5840 0x5614adc5c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:15.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.832+0000 7f4f0d800640 1 -- 172.21.15.16:0/2435478556 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614adb73880 con 0x5614ad88c980 2024-08-31T06:10:15.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.833+0000 7f4f0d800640 1 --2- 172.21.15.16:0/2435478556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614ad88c980 0x5614adc5c810 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4efc009230 tx=0x7f4efc004150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ee8f6524565255c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:15.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.833+0000 7f4f06000640 1 -- 172.21.15.16:0/3956388004 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f4ef401a070 con 0x7f4f08107f80 2024-08-31T06:10:15.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.834+0000 7f4efaa00640 1 -- 172.21.15.16:0/2435478556 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4efc035030 con 0x5614ad88c980 2024-08-31T06:10:15.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.834+0000 7f4efaa00640 1 -- 172.21.15.16:0/2435478556 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4efc0054f0 con 0x5614ad88c980 2024-08-31T06:10:15.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.834+0000 7f4efaa00640 1 -- 172.21.15.16:0/2435478556 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4efc0035f0 con 0x5614ad88c980 2024-08-31T06:10:15.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.840+0000 7f4f16e0d740 1 -- 172.21.15.16:0/2435478556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614ad88c980 msgr2=0x5614adc5c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:15.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.840+0000 7f4f16e0d740 1 --2- 172.21.15.16:0/2435478556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614ad88c980 0x5614adc5c810 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4efc009230 tx=0x7f4efc004150 comp rx=0 tx=0).stop 2024-08-31T06:10:15.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.840+0000 7f4f06000640 1 --2- 172.21.15.16:0/3956388004 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f4ef00b4d90 0x7f4ef00b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:15.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.840+0000 7f4f06000640 1 -- 172.21.15.16:0/3956388004 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f4ef40e4df0 con 0x7f4f08107f80 2024-08-31T06:10:15.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.841+0000 7f4f16e0d740 1 -- 172.21.15.16:0/2435478556 shutdown_connections 2024-08-31T06:10:15.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.841+0000 7f4f16e0d740 1 --2- 172.21.15.16:0/2435478556 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614ad894b60 0x5614adc5cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:15.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.841+0000 7f4f16e0d740 1 --2- 172.21.15.16:0/2435478556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614ad88c980 0x5614adc5c810 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:15.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.841+0000 7f4f16e0d740 1 --2- 172.21.15.16:0/2435478556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614ad8b5840 0x5614adc5c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:15.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.841+0000 7f4f16e0d740 1 -- 172.21.15.16:0/2435478556 >> 172.21.15.16:0/2435478556 conn(0x5614ad8a9990 msgr2=0x5614adc5a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:15.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.841+0000 7f4f16e0d740 1 -- 172.21.15.16:0/2435478556 shutdown_connections 2024-08-31T06:10:15.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.842+0000 7f4f16e0d740 1 -- 172.21.15.16:0/2435478556 wait complete. 2024-08-31T06:10:15.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.842+0000 7f4f16e0d740 1 -- start start 2024-08-31T06:10:15.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.843+0000 7f4f16e0d740 1 Processor -- start 2024-08-31T06:10:15.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.844+0000 7f4f0e200640 1 --2- 172.21.15.16:0/3956388004 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f4ef00b4d90 0x7f4ef00b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:15.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.845+0000 7f4f16e0d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614ad88c980 0x5614adc67e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:15.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.845+0000 7f4f16e0d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614ad894b60 0x5614adc683b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:15.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.845+0000 7f4f0e200640 1 --2- 172.21.15.16:0/3956388004 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f4ef00b4d90 0x7f4ef00b72a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4f00005fd0 tx=0x7f4f00009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:15.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.845+0000 7f4f16e0d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614ad8b5840 0x5614adc7c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:15.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.845+0000 7f4f16e0d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5614ada237a0 con 0x5614ad894b60 2024-08-31T06:10:15.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.845+0000 7f4f16e0d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5614adb477f0 con 0x5614ad8b5840 2024-08-31T06:10:15.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.845+0000 7f4f16e0d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5614adc6aa50 con 0x5614ad88c980 2024-08-31T06:10:15.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.846+0000 7f4f16e0d740 10 client.0 _ll_register_callbacks cb 0x5614adb6ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:10:15.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.846+0000 7f4f0d800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614ad88c980 0x5614adc67e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:15.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.846+0000 7f4f0d800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614ad88c980 0x5614adc67e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:39962/0 (socket says 172.21.15.16:39962) 2024-08-31T06:10:15.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.846+0000 7f4f0d800640 1 -- 172.21.15.16:0/3637441597 learned_addr learned my addr 172.21.15.16:0/3637441597 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:15.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.846+0000 7f4f0e200640 1 --2- 172.21.15.16:0/3637441597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614ad8b5840 0x5614adc7c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:15.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.846+0000 7f4f0d800640 1 -- 172.21.15.16:0/3637441597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614ad8b5840 msgr2=0x5614adc7c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:15.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.846+0000 7f4f0d800640 1 --2- 172.21.15.16:0/3637441597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614ad8b5840 0x5614adc7c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:15.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.846+0000 7f4f0d800640 1 -- 172.21.15.16:0/3637441597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614ad894b60 msgr2=0x5614adc683b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:15.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.846+0000 7f4f0d800640 1 --2- 172.21.15.16:0/3637441597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614ad894b60 0x5614adc683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:15.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.846+0000 7f4f0d800640 1 -- 172.21.15.16:0/3637441597 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614ad92fb90 con 0x5614ad88c980 2024-08-31T06:10:15.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.847+0000 7f4f0d800640 1 --2- 172.21.15.16:0/3637441597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614ad88c980 0x5614adc67e70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4efc005ab0 tx=0x7f4efc0058b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21dd7b31a01096d8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:15.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.847+0000 7f4eef400640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:10:15.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.847+0000 7f4eef400640 1 -- 172.21.15.16:0/3637441597 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4efc035040 con 0x5614ad88c980 2024-08-31T06:10:15.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.847+0000 7f4eef400640 1 -- 172.21.15.16:0/3637441597 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f4efc0035f0 con 0x5614ad88c980 2024-08-31T06:10:15.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.847+0000 7f4f16e0d740 1 -- 172.21.15.16:0/3637441597 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4efc0054f0 con 0x5614ad88c980 2024-08-31T06:10:15.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.849+0000 7f4eef400640 1 -- 172.21.15.16:0/3637441597 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4efc004e30 con 0x5614ad88c980 2024-08-31T06:10:15.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.849+0000 7f4f16e0d740 10 client.26248 fetch_fsmap learned FSMap version 31 2024-08-31T06:10:15.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.849+0000 7f4f16e0d740 1 -- 172.21.15.16:0/3637441597 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5614adc7f260 con 0x5614ad88c980 2024-08-31T06:10:15.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.849+0000 7f4eef400640 1 -- 172.21.15.16:0/3637441597 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4efc0054f0 con 0x5614ad88c980 2024-08-31T06:10:15.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.851+0000 7f4eef400640 1 -- 172.21.15.16:0/3637441597 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f4efc04b070 con 0x5614ad88c980 2024-08-31T06:10:15.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.851+0000 7f4f16e0d740 10 client.26248 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:10:15.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.851+0000 7f4f16e0d740 10 client.26248 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:10:15.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.851+0000 7f4f16e0d740 20 client.26248 populate_metadata read hostname 'smithi016' 2024-08-31T06:10:15.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.851+0000 7f4f16e0d740 1 --2- 172.21.15.16:0/3637441597 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5614ad8741c0 0x5614adc825c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:15.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.851+0000 7f4f16e0d740 4 client.26248 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:10:15.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.851+0000 7f4f16e0d740 1 -- 172.21.15.16:0/3637441597 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614adbd3480 con 0x5614ad8741c0 2024-08-31T06:10:15.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.854+0000 7f4f0ce00640 1 --2- 172.21.15.16:0/3637441597 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5614ad8741c0 0x5614adc825c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:15.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.854+0000 7f4eef400640 10 client.26248 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:15.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.854+0000 7f4f0ce00640 1 --2- 172.21.15.16:0/3637441597 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5614ad8741c0 0x5614adc825c0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83f0fd477547200d server_cookie=65e60a382bbb4cf4 in_seq=0 out_seq=0 2024-08-31T06:10:15.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.855+0000 7f4eef400640 1 -- 172.21.15.16:0/3637441597 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5614adbd3480 con 0x5614ad8741c0 2024-08-31T06:10:15.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.855+0000 7f4eef400640 10 client.26248 handle_command_reply: tid=0 2024-08-31T06:10:15.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.856+0000 7f4f16e0d740 1 client.26248 shutdown 2024-08-31T06:10:15.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.856+0000 7f4f16e0d740 1 -- 172.21.15.16:0/3637441597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614ad88c980 msgr2=0x5614adc67e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:15.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.856+0000 7f4f16e0d740 1 --2- 172.21.15.16:0/3637441597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614ad88c980 0x5614adc67e70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4efc005ab0 tx=0x7f4efc0058b0 comp rx=0 tx=0).stop 2024-08-31T06:10:15.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.856+0000 7f4f16e0d740 1 -- 172.21.15.16:0/3637441597 shutdown_connections 2024-08-31T06:10:15.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.856+0000 7f4f16e0d740 1 --2- 172.21.15.16:0/3637441597 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5614ad8741c0 0x5614adc825c0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:15.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.856+0000 7f4f16e0d740 1 --2- 172.21.15.16:0/3637441597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614ad8b5840 0x5614adc7c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:15.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.856+0000 7f4eef400640 0 client.26248 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:15.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.856+0000 7f4f16e0d740 1 --2- 172.21.15.16:0/3637441597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614ad894b60 0x5614adc683b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:15.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.856+0000 7f4f16e0d740 1 --2- 172.21.15.16:0/3637441597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614ad88c980 0x5614adc67e70 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:15.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.856+0000 7f4f16e0d740 1 -- 172.21.15.16:0/3637441597 >> 172.21.15.16:0/3637441597 conn(0x5614ad8a9990 msgr2=0x5614adc5f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:15.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.857+0000 7f4f16e0d740 1 -- 172.21.15.16:0/3637441597 shutdown_connections 2024-08-31T06:10:15.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.857+0000 7f4f16e0d740 1 -- 172.21.15.16:0/3637441597 wait complete. 2024-08-31T06:10:15.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.857+0000 7f4f16e0d740 20 client.26248 trim_cache size 0 max 16384 2024-08-31T06:10:15.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.888+0000 7f4f16e0d740 1 Processor -- start 2024-08-31T06:10:15.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.889+0000 7f4f16e0d740 1 -- start start 2024-08-31T06:10:15.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.889+0000 7f4f16e0d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614ad88c980 0x5614adc591a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:15.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.889+0000 7f4f16e0d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614ad894b60 0x5614adc62e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:15.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.889+0000 7f4f16e0d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614ad8b5840 0x5614adc7c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:15.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.889+0000 7f4f16e0d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee8011030 con 0x5614ad8b5840 2024-08-31T06:10:15.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.889+0000 7f4f16e0d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee800e840 con 0x5614ad88c980 2024-08-31T06:10:15.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.889+0000 7f4f16e0d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee800c690 con 0x5614ad894b60 2024-08-31T06:10:15.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.890+0000 7f4f0ce00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614ad894b60 0x5614adc62e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:15.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.890+0000 7f4f0ce00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614ad894b60 0x5614adc62e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:39978/0 (socket says 172.21.15.16:39978) 2024-08-31T06:10:15.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.890+0000 7f4f0ce00640 1 -- 172.21.15.16:0/1144010449 learned_addr learned my addr 172.21.15.16:0/1144010449 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:15.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.890+0000 7f4f0e200640 1 --2- 172.21.15.16:0/1144010449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614ad8b5840 0x5614adc7c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:15.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.890+0000 7f4f0d800640 1 --2- 172.21.15.16:0/1144010449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614ad88c980 0x5614adc591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:15.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.890+0000 7f4f0ce00640 1 -- 172.21.15.16:0/1144010449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614ad88c980 msgr2=0x5614adc591a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:15.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.890+0000 7f4f0ce00640 1 --2- 172.21.15.16:0/1144010449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614ad88c980 0x5614adc591a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:15.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.890+0000 7f4f0ce00640 1 -- 172.21.15.16:0/1144010449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614ad8b5840 msgr2=0x5614adc7c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:15.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.890+0000 7f4f0ce00640 1 --2- 172.21.15.16:0/1144010449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614ad8b5840 0x5614adc7c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:15.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.890+0000 7f4f0ce00640 1 -- 172.21.15.16:0/1144010449 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614adc5a200 con 0x5614ad894b60 2024-08-31T06:10:15.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.890+0000 7f4f0ce00640 1 --2- 172.21.15.16:0/1144010449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614ad894b60 0x5614adc62e20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4009610 tx=0x7f4ef4010db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc9d88cac28c8c68 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:15.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.891+0000 7f4efa000640 1 -- 172.21.15.16:0/1144010449 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef4021040 con 0x5614ad894b60 2024-08-31T06:10:15.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.891+0000 7f4efa000640 1 -- 172.21.15.16:0/1144010449 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ef40afd50 con 0x5614ad894b60 2024-08-31T06:10:15.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.891+0000 7f4efa000640 1 -- 172.21.15.16:0/1144010449 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef40ce0c0 con 0x5614ad894b60 2024-08-31T06:10:15.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.892+0000 7f4f16e0d740 1 -- 172.21.15.16:0/1144010449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614ad894b60 msgr2=0x5614adc62e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:15.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.892+0000 7f4f16e0d740 1 --2- 172.21.15.16:0/1144010449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614ad894b60 0x5614adc62e20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4009610 tx=0x7f4ef4010db0 comp rx=0 tx=0).stop 2024-08-31T06:10:15.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.892+0000 7f4f16e0d740 1 -- 172.21.15.16:0/1144010449 shutdown_connections 2024-08-31T06:10:15.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.892+0000 7f4f16e0d740 1 --2- 172.21.15.16:0/1144010449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614ad894b60 0x5614adc62e20 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:15.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.892+0000 7f4f16e0d740 1 --2- 172.21.15.16:0/1144010449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614ad8b5840 0x5614adc7c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:15.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.892+0000 7f4f16e0d740 1 --2- 172.21.15.16:0/1144010449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614ad88c980 0x5614adc591a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:15.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.892+0000 7f4f16e0d740 1 -- 172.21.15.16:0/1144010449 >> 172.21.15.16:0/1144010449 conn(0x5614ad8a9990 msgr2=0x5614adc664f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:15.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.892+0000 7f4f16e0d740 1 -- 172.21.15.16:0/1144010449 shutdown_connections 2024-08-31T06:10:15.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.892+0000 7f4f16e0d740 1 -- 172.21.15.16:0/1144010449 wait complete. 2024-08-31T06:10:15.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.892+0000 7f4f16e0d740 1 -- start start 2024-08-31T06:10:15.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.893+0000 7f4f16e0d740 1 Processor -- start 2024-08-31T06:10:15.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.893+0000 7f4f16e0d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614ad88c980 0x5614adc78e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:15.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.893+0000 7f4f16e0d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614ad894b60 0x5614adc75330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:15.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.893+0000 7f4f16e0d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614ad8b5840 0x5614adc75930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:15.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.893+0000 7f4f16e0d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee800cd00 con 0x5614ad88c980 2024-08-31T06:10:15.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.893+0000 7f4f16e0d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee800e170 con 0x5614ad894b60 2024-08-31T06:10:15.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.893+0000 7f4f16e0d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee800d3d0 con 0x5614ad8b5840 2024-08-31T06:10:15.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.893+0000 7f4f16e0d740 10 client.0 _ll_register_callbacks cb 0x7f4efc0053e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:10:15.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.894+0000 7f4f0ce00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614ad894b60 0x5614adc75330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:15.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.894+0000 7f4f0d800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614ad88c980 0x5614adc78e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:15.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.894+0000 7f4f0ce00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614ad894b60 0x5614adc75330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:38686/0 (socket says 172.21.15.16:38686) 2024-08-31T06:10:15.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.894+0000 7f4f0ce00640 1 -- 172.21.15.16:0/1480647403 learned_addr learned my addr 172.21.15.16:0/1480647403 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:15.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.894+0000 7f4f0e200640 1 --2- 172.21.15.16:0/1480647403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614ad8b5840 0x5614adc75930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:15.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.894+0000 7f4f0ce00640 1 -- 172.21.15.16:0/1480647403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614ad8b5840 msgr2=0x5614adc75930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:15.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.894+0000 7f4f0ce00640 1 --2- 172.21.15.16:0/1480647403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614ad8b5840 0x5614adc75930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:15.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.894+0000 7f4f0ce00640 1 -- 172.21.15.16:0/1480647403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614ad88c980 msgr2=0x5614adc78e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:15.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.894+0000 7f4f0ce00640 1 --2- 172.21.15.16:0/1480647403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614ad88c980 0x5614adc78e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:15.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.894+0000 7f4f0ce00640 1 -- 172.21.15.16:0/1480647403 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614adc760c0 con 0x5614ad894b60 2024-08-31T06:10:15.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.895+0000 7f4f0ce00640 1 --2- 172.21.15.16:0/1480647403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614ad894b60 0x5614adc75330 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4ef40ab470 tx=0x7f4ef40a79a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fa28319ee11ee45 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:15.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.895+0000 7f4eef400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:10:15.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.895+0000 7f4eef400640 1 -- 172.21.15.16:0/1480647403 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef40b4800 con 0x5614ad894b60 2024-08-31T06:10:15.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.895+0000 7f4eef400640 1 -- 172.21.15.16:0/1480647403 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f4ef405b170 con 0x5614ad894b60 2024-08-31T06:10:15.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.896+0000 7f4f16e0d740 1 -- 172.21.15.16:0/1480647403 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ef40afd50 con 0x5614ad894b60 2024-08-31T06:10:15.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.896+0000 7f4eef400640 1 -- 172.21.15.16:0/1480647403 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4ef40e27c0 con 0x5614ad894b60 2024-08-31T06:10:15.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.896+0000 7f4f16e0d740 10 client.26261 fetch_fsmap learned FSMap version 31 2024-08-31T06:10:15.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.896+0000 7f4f16e0d740 1 -- 172.21.15.16:0/1480647403 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5614adcedd60 con 0x5614ad894b60 2024-08-31T06:10:15.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.897+0000 7f4eef400640 1 -- 172.21.15.16:0/1480647403 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ef40afd50 con 0x5614ad894b60 2024-08-31T06:10:15.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.897+0000 7f4eef400640 1 -- 172.21.15.16:0/1480647403 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f4ef40a6e50 con 0x5614ad894b60 2024-08-31T06:10:15.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.897+0000 7f4f16e0d740 10 client.26261 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:10:15.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.897+0000 7f4f16e0d740 10 client.26261 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:10:15.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.898+0000 7f4f16e0d740 20 client.26261 populate_metadata read hostname 'smithi016' 2024-08-31T06:10:15.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.898+0000 7f4f16e0d740 1 --2- 172.21.15.16:0/1480647403 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5614ad849570 0x5614adcf0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:15.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.898+0000 7f4f16e0d740 4 client.26261 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.8, "await_for": 45.0}] 2024-08-31T06:10:15.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.898+0000 7f4f16e0d740 1 -- 172.21.15.16:0/1480647403 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.8, "await_for": 45.0}) -- 0x5614adbcef20 con 0x5614ad849570 2024-08-31T06:10:15.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.898+0000 7f4f0d800640 1 --2- 172.21.15.16:0/1480647403 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5614ad849570 0x5614adcf0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:15.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.898+0000 7f4f0d800640 1 --2- 172.21.15.16:0/1480647403 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5614ad849570 0x5614adcf0500 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=253eea42f610a9df server_cookie=7bae39bbe7e9ab74 in_seq=0 out_seq=0 2024-08-31T06:10:15.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:15.898+0000 7f4eef400640 10 client.26261 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:16.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:16.103+0000 7f4eef400640 1 -- 172.21.15.16:0/1480647403 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5614adbcef20 con 0x5614ad849570 2024-08-31T06:10:16.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:16.103+0000 7f4eef400640 10 client.26261 handle_command_reply: tid=0 2024-08-31T06:10:16.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:16.103+0000 7f4f16e0d740 1 client.26261 shutdown 2024-08-31T06:10:16.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:16.103+0000 7f4f16e0d740 1 -- 172.21.15.16:0/1480647403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614ad894b60 msgr2=0x5614adc75330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:16.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:16.103+0000 7f4f16e0d740 1 --2- 172.21.15.16:0/1480647403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614ad894b60 0x5614adc75330 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4ef40ab470 tx=0x7f4ef40a79a0 comp rx=0 tx=0).stop 2024-08-31T06:10:16.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:16.103+0000 7f4f16e0d740 1 -- 172.21.15.16:0/1480647403 shutdown_connections 2024-08-31T06:10:16.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:16.103+0000 7f4f16e0d740 1 --2- 172.21.15.16:0/1480647403 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5614ad849570 0x5614adcf0500 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:16.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:16.103+0000 7f4f16e0d740 1 --2- 172.21.15.16:0/1480647403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614ad8b5840 0x5614adc75930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:16.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:16.103+0000 7f4f16e0d740 1 --2- 172.21.15.16:0/1480647403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614ad894b60 0x5614adc75330 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:16.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:16.103+0000 7f4f16e0d740 1 --2- 172.21.15.16:0/1480647403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614ad88c980 0x5614adc78e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:16.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:16.103+0000 7f4eef400640 0 client.26261 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:16.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:16.103+0000 7f4f16e0d740 1 -- 172.21.15.16:0/1480647403 >> 172.21.15.16:0/1480647403 conn(0x5614ad8a9990 msgr2=0x5614adc5d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:16.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:16.104+0000 7f4f16e0d740 1 -- 172.21.15.16:0/1480647403 shutdown_connections 2024-08-31T06:10:16.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:16.104+0000 7f4f16e0d740 1 -- 172.21.15.16:0/1480647403 wait complete. 2024-08-31T06:10:16.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:16.104+0000 7f4f16e0d740 20 client.26261 trim_cache size 0 max 16384 2024-08-31T06:10:16.108 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:10:16.108 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 31, 2024-08-31T06:10:16.108 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24737, 2024-08-31T06:10:16.108 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 10, 2024-08-31T06:10:16.108 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-08-31T06:10:16.108 INFO:teuthology.orchestra.run.smithi016.stdout: "53a757a7": { 2024-08-31T06:10:16.108 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 10, 2024-08-31T06:10:16.108 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-08-31T06:10:16.108 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:10:16.108 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-08-31T06:10:16.108 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:10:16.108 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:10:16.108 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-08-31T06:10:16.109 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 141.8, 2024-08-31T06:10:16.109 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-08-31T06:10:16.109 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-08-31T06:10:16.109 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-08-31T06:10:16.109 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:10:16.109 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-08-31T06:10:16.109 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:10:16.109 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:10:16.109 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:10:16.109 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:10:16.109 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:10:16.109 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:10:16.109 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:10:16.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:16.105+0000 7f4f0ec00640 1 -- 172.21.15.16:0/3956388004 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f4ef00b4d90 msgr2=0x7f4ef00b72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:16.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:16.105+0000 7f4f0ec00640 1 --2- 172.21.15.16:0/3956388004 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f4ef00b4d90 0x7f4ef00b72a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4f00005fd0 tx=0x7f4f00009500 comp rx=0 tx=0).stop 2024-08-31T06:10:16.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:16.105+0000 7f4f0ec00640 1 -- 172.21.15.16:0/3956388004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f08107f80 msgr2=0x7f4f0819c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:16.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:16.105+0000 7f4f0ec00640 1 --2- 172.21.15.16:0/3956388004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f08107f80 0x7f4f0819c8c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4ef400d740 tx=0x7f4ef400dc00 comp rx=0 tx=0).stop 2024-08-31T06:10:16.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:16.105+0000 7f4f0ec00640 1 -- 172.21.15.16:0/3956388004 shutdown_connections 2024-08-31T06:10:16.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:16.105+0000 7f4f0ec00640 1 --2- 172.21.15.16:0/3956388004 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f4ef00b4d90 0x7f4ef00b72a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:16.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:16.105+0000 7f4f0ec00640 1 --2- 172.21.15.16:0/3956388004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f081088d0 0x7f4f081a1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:16.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:16.105+0000 7f4f0ec00640 1 --2- 172.21.15.16:0/3956388004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f08107f80 0x7f4f0819c8c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:16.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:16.105+0000 7f4f0ec00640 1 --2- 172.21.15.16:0/3956388004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f08102640 0x7f4f0819c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:16.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:16.105+0000 7f4f0ec00640 1 -- 172.21.15.16:0/3956388004 >> 172.21.15.16:0/3956388004 conn(0x7f4f080fe0d0 msgr2=0x7f4f081075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:16.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:16.105+0000 7f4f0ec00640 1 -- 172.21.15.16:0/3956388004 shutdown_connections 2024-08-31T06:10:16.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:16.105+0000 7f4f0ec00640 1 -- 172.21.15.16:0/3956388004 wait complete. 2024-08-31T06:10:16.625 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 170.0 s, 2185.7 tps, lat 39.494 ms stddev 47.628 2024-08-31T06:10:16.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:16 smithi184 ceph-mon[22331]: pgmap v455: 97 pgs: 97 active+clean; 1.9 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 34 MiB/s wr, 1.76k op/s 2024-08-31T06:10:16.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:16 smithi155 ceph-mon[21742]: pgmap v455: 97 pgs: 97 active+clean; 1.9 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 34 MiB/s wr, 1.76k op/s 2024-08-31T06:10:16.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:16 smithi016 ceph-mon[17194]: pgmap v455: 97 pgs: 97 active+clean; 1.9 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 34 MiB/s wr, 1.76k op/s 2024-08-31T06:10:17.395 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '53a757a7', quiesce took 7.3 seconds. Will release after: 14.5 2024-08-31T06:10:18.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:18 smithi155 ceph-mon[21742]: pgmap v456: 97 pgs: 97 active+clean; 1.9 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 32 MiB/s wr, 1.58k op/s 2024-08-31T06:10:18.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:18 smithi016 ceph-mon[17194]: pgmap v456: 97 pgs: 97 active+clean; 1.9 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 32 MiB/s wr, 1.58k op/s 2024-08-31T06:10:18.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:18 smithi184 ceph-mon[22331]: pgmap v456: 97 pgs: 97 active+clean; 1.9 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 32 MiB/s wr, 1.58k op/s 2024-08-31T06:10:20.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:20 smithi155 ceph-mon[21742]: pgmap v457: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 32 MiB/s wr, 1.57k op/s 2024-08-31T06:10:20.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:20 smithi016 ceph-mon[17194]: pgmap v457: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 32 MiB/s wr, 1.57k op/s 2024-08-31T06:10:20.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:20 smithi184 ceph-mon[22331]: pgmap v457: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 32 MiB/s wr, 1.57k op/s 2024-08-31T06:10:21.629 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 175.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-31T06:10:22.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:22 smithi155 ceph-mon[21742]: pgmap v458: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 30 MiB/s wr, 1.34k op/s 2024-08-31T06:10:22.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:22 smithi016 ceph-mon[17194]: pgmap v458: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 30 MiB/s wr, 1.34k op/s 2024-08-31T06:10:22.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:22 smithi184 ceph-mon[22331]: pgmap v458: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 30 MiB/s wr, 1.34k op/s 2024-08-31T06:10:24.747 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:24 smithi016 ceph-mon[17194]: pgmap v459: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 23 MiB/s wr, 1.05k op/s 2024-08-31T06:10:24.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:24 smithi155 ceph-mon[21742]: pgmap v459: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 23 MiB/s wr, 1.05k op/s 2024-08-31T06:10:24.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:24 smithi184 ceph-mon[22331]: pgmap v459: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 23 MiB/s wr, 1.05k op/s 2024-08-31T06:10:24.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:24.993+0000 7fa481200640 1 -- 172.21.15.16:0/2169763739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa47c103870 msgr2=0x7fa47c103c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:24.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:24.993+0000 7fa481200640 1 --2- 172.21.15.16:0/2169763739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa47c103870 0x7fa47c103c80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa46c009900 tx=0x7fa46c02a410 comp rx=0 tx=0).stop 2024-08-31T06:10:24.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:24.994+0000 7fa481200640 1 -- 172.21.15.16:0/2169763739 shutdown_connections 2024-08-31T06:10:24.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:24.994+0000 7fa481200640 1 --2- 172.21.15.16:0/2169763739 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa47c109af0 0x7fa47c110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:24.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:24.994+0000 7fa481200640 1 --2- 172.21.15.16:0/2169763739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa47c1091a0 0x7fa47c1095b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:24.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:24.994+0000 7fa481200640 1 --2- 172.21.15.16:0/2169763739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa47c103870 0x7fa47c103c80 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:24.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:24.994+0000 7fa481200640 1 -- 172.21.15.16:0/2169763739 >> 172.21.15.16:0/2169763739 conn(0x7fa47c0ff2b0 msgr2=0x7fa47c101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:24.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:24.994+0000 7fa481200640 1 -- 172.21.15.16:0/2169763739 shutdown_connections 2024-08-31T06:10:24.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:24.994+0000 7fa481200640 1 -- 172.21.15.16:0/2169763739 wait complete. 2024-08-31T06:10:24.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:24.995+0000 7fa481200640 1 Processor -- start 2024-08-31T06:10:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:24.995+0000 7fa481200640 1 -- start start 2024-08-31T06:10:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:24.996+0000 7fa481200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa47c103870 0x7fa47c19c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:24.996+0000 7fa481200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa47c1091a0 0x7fa47c19c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:24.996+0000 7fa481200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa47c109af0 0x7fa47c1a1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:24.996+0000 7fa481200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c1137c0 con 0x7fa47c1091a0 2024-08-31T06:10:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:24.996+0000 7fa481200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c113640 con 0x7fa47c103870 2024-08-31T06:10:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:24.996+0000 7fa481200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c113940 con 0x7fa47c109af0 2024-08-31T06:10:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:24.996+0000 7fa479400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa47c1091a0 0x7fa47c19c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:24.996+0000 7fa479400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa47c1091a0 0x7fa47c19c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44462/0 (socket says 172.21.15.16:44462) 2024-08-31T06:10:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:24.996+0000 7fa479400640 1 -- 172.21.15.16:0/1776782921 learned_addr learned my addr 172.21.15.16:0/1776782921 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:24.996+0000 7fa479e00640 1 --2- 172.21.15.16:0/1776782921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa47c103870 0x7fa47c19c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:24.996+0000 7fa47a800640 1 --2- 172.21.15.16:0/1776782921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa47c109af0 0x7fa47c1a1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:24.997+0000 7fa479400640 1 -- 172.21.15.16:0/1776782921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa47c109af0 msgr2=0x7fa47c1a1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:24.997+0000 7fa479400640 1 --2- 172.21.15.16:0/1776782921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa47c109af0 0x7fa47c1a1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:24.997+0000 7fa479400640 1 -- 172.21.15.16:0/1776782921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa47c103870 msgr2=0x7fa47c19c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:24.997+0000 7fa479400640 1 --2- 172.21.15.16:0/1776782921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa47c103870 0x7fa47c19c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:24.997+0000 7fa479400640 1 -- 172.21.15.16:0/1776782921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa47c1a2330 con 0x7fa47c1091a0 2024-08-31T06:10:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:24.997+0000 7fa479e00640 1 --2- 172.21.15.16:0/1776782921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa47c103870 0x7fa47c19c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:10:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:24.997+0000 7fa479400640 1 --2- 172.21.15.16:0/1776782921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa47c1091a0 0x7fa47c19c9a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa46800e9f0 tx=0x7fa46800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:24.997+0000 7fa472a00640 1 -- 172.21.15.16:0/1776782921 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46800ccc0 con 0x7fa47c1091a0 2024-08-31T06:10:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:24.997+0000 7fa472a00640 1 -- 172.21.15.16:0/1776782921 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4680108e0 con 0x7fa47c1091a0 2024-08-31T06:10:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:24.997+0000 7fa481200640 1 -- 172.21.15.16:0/1776782921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa47c1a2620 con 0x7fa47c1091a0 2024-08-31T06:10:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:24.997+0000 7fa472a00640 1 -- 172.21.15.16:0/1776782921 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa468018860 con 0x7fa47c1091a0 2024-08-31T06:10:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:24.997+0000 7fa481200640 1 -- 172.21.15.16:0/1776782921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa47c1a8db0 con 0x7fa47c1091a0 2024-08-31T06:10:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.002+0000 7fa4882a0740 1 Processor -- start 2024-08-31T06:10:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.002+0000 7fa472a00640 1 -- 172.21.15.16:0/1776782921 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fa468010430 con 0x7fa47c1091a0 2024-08-31T06:10:25.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.002+0000 7fa4882a0740 1 -- start start 2024-08-31T06:10:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.003+0000 7fa4882a0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556050cd11b0 0x556051074920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.003+0000 7fa4882a0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556050ca82f0 0x556051074e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.003+0000 7fa4882a0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556050cb04d0 0x5560510753a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.003+0000 7fa4882a0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556050fe9f70 con 0x556050cd11b0 2024-08-31T06:10:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.003+0000 7fa4882a0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556050fd9500 con 0x556050cb04d0 2024-08-31T06:10:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.003+0000 7fa4882a0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556050fdd4b0 con 0x556050ca82f0 2024-08-31T06:10:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.003+0000 7fa47a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556050cd11b0 0x556051074920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.003+0000 7fa47a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556050cd11b0 0x556051074920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44470/0 (socket says 172.21.15.16:44470) 2024-08-31T06:10:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.003+0000 7fa47a800640 1 -- 172.21.15.16:0/3219467055 learned_addr learned my addr 172.21.15.16:0/3219467055 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.003+0000 7fa479400640 1 --2- 172.21.15.16:0/3219467055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556050cb04d0 0x5560510753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.003+0000 7fa479e00640 1 --2- 172.21.15.16:0/3219467055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556050ca82f0 0x556051074e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.004+0000 7fa47a800640 1 -- 172.21.15.16:0/3219467055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556050ca82f0 msgr2=0x556051074e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.004+0000 7fa47a800640 1 --2- 172.21.15.16:0/3219467055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556050ca82f0 0x556051074e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.004+0000 7fa47a800640 1 -- 172.21.15.16:0/3219467055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556050cb04d0 msgr2=0x5560510753a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.004+0000 7fa47a800640 1 --2- 172.21.15.16:0/3219467055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556050cb04d0 0x5560510753a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.004+0000 7fa47a800640 1 -- 172.21.15.16:0/3219467055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556050f93c10 con 0x556050cd11b0 2024-08-31T06:10:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.004+0000 7fa472a00640 1 --2- 172.21.15.16:0/1776782921 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa4600b4d20 0x7fa4600b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.004+0000 7fa47a800640 1 --2- 172.21.15.16:0/3219467055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556050cd11b0 0x556051074920 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa474009fd0 tx=0x7fa4740118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52076c03826c5e40 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.004+0000 7fa472a00640 1 -- 172.21.15.16:0/1776782921 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fa4680e5700 con 0x7fa47c1091a0 2024-08-31T06:10:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.004+0000 7fa479e00640 1 --2- 172.21.15.16:0/1776782921 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa4600b4d20 0x7fa4600b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.004+0000 7fa467400640 1 -- 172.21.15.16:0/3219467055 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa474004d20 con 0x556050cd11b0 2024-08-31T06:10:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.004+0000 7fa467400640 1 -- 172.21.15.16:0/3219467055 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa474004050 con 0x556050cd11b0 2024-08-31T06:10:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.004+0000 7fa467400640 1 -- 172.21.15.16:0/3219467055 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa474005220 con 0x556050cd11b0 2024-08-31T06:10:25.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.006+0000 7fa4882a0740 1 -- 172.21.15.16:0/3219467055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556050cd11b0 msgr2=0x556051074920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:25.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.006+0000 7fa479e00640 1 --2- 172.21.15.16:0/1776782921 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa4600b4d20 0x7fa4600b7230 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa46c031440 tx=0x7fa46c032ed0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:25.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.006+0000 7fa4882a0740 1 --2- 172.21.15.16:0/3219467055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556050cd11b0 0x556051074920 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa474009fd0 tx=0x7fa4740118d0 comp rx=0 tx=0).stop 2024-08-31T06:10:25.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.006+0000 7fa4882a0740 1 -- 172.21.15.16:0/3219467055 shutdown_connections 2024-08-31T06:10:25.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.006+0000 7fa4882a0740 1 --2- 172.21.15.16:0/3219467055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556050ca82f0 0x556051074e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:25.011 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.006+0000 7fa4882a0740 1 --2- 172.21.15.16:0/3219467055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556050cb04d0 0x5560510753a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:25.011 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.006+0000 7fa4882a0740 1 --2- 172.21.15.16:0/3219467055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556050cd11b0 0x556051074920 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:25.011 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.006+0000 7fa4882a0740 1 -- 172.21.15.16:0/3219467055 >> 172.21.15.16:0/3219467055 conn(0x556050cc5300 msgr2=0x556051072810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:25.011 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.006+0000 7fa4882a0740 1 -- 172.21.15.16:0/3219467055 shutdown_connections 2024-08-31T06:10:25.011 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.006+0000 7fa4882a0740 1 -- 172.21.15.16:0/3219467055 wait complete. 2024-08-31T06:10:25.011 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.007+0000 7fa4882a0740 1 -- start start 2024-08-31T06:10:25.011 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.007+0000 7fa4882a0740 1 Processor -- start 2024-08-31T06:10:25.011 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.008+0000 7fa4882a0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556050ca82f0 0x55605107fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:25.011 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.008+0000 7fa4882a0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556050cb04d0 0x556051080410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:25.011 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.008+0000 7fa4882a0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556050cd11b0 0x556051080950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:25.011 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.008+0000 7fa4882a0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556050f12ab0 con 0x556050ca82f0 2024-08-31T06:10:25.012 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.008+0000 7fa4882a0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556050f80320 con 0x556050cb04d0 2024-08-31T06:10:25.012 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.008+0000 7fa4882a0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5560510830a0 con 0x556050cd11b0 2024-08-31T06:10:25.012 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.008+0000 7fa4882a0740 10 client.0 _ll_register_callbacks cb 0x556050f866a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:10:25.012 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.008+0000 7fa479e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556050ca82f0 0x55605107fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:25.012 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.008+0000 7fa479e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556050ca82f0 0x55605107fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44478/0 (socket says 172.21.15.16:44478) 2024-08-31T06:10:25.012 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.008+0000 7fa479e00640 1 -- 172.21.15.16:0/220056094 learned_addr learned my addr 172.21.15.16:0/220056094 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:25.012 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.008+0000 7fa479400640 1 --2- 172.21.15.16:0/220056094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556050cb04d0 0x556051080410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:25.012 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.008+0000 7fa47a800640 1 --2- 172.21.15.16:0/220056094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556050cd11b0 0x556051080950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:25.012 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.009+0000 7fa479e00640 1 -- 172.21.15.16:0/220056094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556050cd11b0 msgr2=0x556051080950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:25.012 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.009+0000 7fa479e00640 1 --2- 172.21.15.16:0/220056094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556050cd11b0 0x556051080950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:25.012 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.009+0000 7fa479e00640 1 -- 172.21.15.16:0/220056094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556050cb04d0 msgr2=0x556051080410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:25.013 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.009+0000 7fa479e00640 1 --2- 172.21.15.16:0/220056094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556050cb04d0 0x556051080410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:25.013 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.009+0000 7fa479e00640 1 -- 172.21.15.16:0/220056094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556050d3c790 con 0x556050ca82f0 2024-08-31T06:10:25.015 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.009+0000 7fa479e00640 1 --2- 172.21.15.16:0/220056094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556050ca82f0 0x55605107fed0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa46c033000 tx=0x7fa46c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a77168516f53d14 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:25.015 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.009+0000 7fa45be00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:10:25.015 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.009+0000 7fa45be00640 1 -- 172.21.15.16:0/220056094 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c04d070 con 0x556050ca82f0 2024-08-31T06:10:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.010+0000 7fa4882a0740 1 -- 172.21.15.16:0/220056094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa474004050 con 0x556050ca82f0 2024-08-31T06:10:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.010+0000 7fa45be00640 1 -- 172.21.15.16:0/220056094 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fa46c004050 con 0x556050ca82f0 2024-08-31T06:10:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.010+0000 7fa45be00640 1 -- 172.21.15.16:0/220056094 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa46c059050 con 0x556050ca82f0 2024-08-31T06:10:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.010+0000 7fa4882a0740 10 client.16155 fetch_fsmap learned FSMap version 31 2024-08-31T06:10:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.010+0000 7fa4882a0740 1 -- 172.21.15.16:0/220056094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556050db5620 con 0x556050ca82f0 2024-08-31T06:10:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.010+0000 7fa45be00640 1 -- 172.21.15.16:0/220056094 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa474004050 con 0x556050ca82f0 2024-08-31T06:10:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.010+0000 7fa45be00640 1 -- 172.21.15.16:0/220056094 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fa46c0036f0 con 0x556050ca82f0 2024-08-31T06:10:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.010+0000 7fa4882a0740 10 client.16155 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:10:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.010+0000 7fa4882a0740 10 client.16155 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:10:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.010+0000 7fa4882a0740 20 client.16155 populate_metadata read hostname 'smithi016' 2024-08-31T06:10:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.011+0000 7fa4882a0740 1 --2- 172.21.15.16:0/220056094 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556050c8fd80 0x556051092130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.011+0000 7fa4882a0740 4 client.16155 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:10:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.011+0000 7fa4882a0740 1 -- 172.21.15.16:0/220056094 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556050febec0 con 0x556050c8fd80 2024-08-31T06:10:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.011+0000 7fa479400640 1 --2- 172.21.15.16:0/220056094 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556050c8fd80 0x556051092130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.011+0000 7fa479400640 1 --2- 172.21.15.16:0/220056094 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556050c8fd80 0x556051092130 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fb3863687956042 server_cookie=ba1c95deecf73b8e in_seq=0 out_seq=0 2024-08-31T06:10:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.011+0000 7fa45be00640 10 client.16155 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.012+0000 7fa45be00640 1 -- 172.21.15.16:0/220056094 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556050febec0 con 0x556050c8fd80 2024-08-31T06:10:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.012+0000 7fa45be00640 10 client.16155 handle_command_reply: tid=0 2024-08-31T06:10:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.012+0000 7fa4882a0740 1 client.16155 shutdown 2024-08-31T06:10:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.012+0000 7fa4882a0740 1 -- 172.21.15.16:0/220056094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556050ca82f0 msgr2=0x55605107fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.012+0000 7fa4882a0740 1 --2- 172.21.15.16:0/220056094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556050ca82f0 0x55605107fed0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa46c033000 tx=0x7fa46c009260 comp rx=0 tx=0).stop 2024-08-31T06:10:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.013+0000 7fa4882a0740 1 -- 172.21.15.16:0/220056094 shutdown_connections 2024-08-31T06:10:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.013+0000 7fa4882a0740 1 --2- 172.21.15.16:0/220056094 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556050c8fd80 0x556051092130 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.013+0000 7fa4882a0740 1 --2- 172.21.15.16:0/220056094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556050cd11b0 0x556051080950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.013+0000 7fa45be00640 0 client.16155 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.013+0000 7fa4882a0740 1 --2- 172.21.15.16:0/220056094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556050cb04d0 0x556051080410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.013+0000 7fa4882a0740 1 --2- 172.21.15.16:0/220056094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556050ca82f0 0x55605107fed0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.013+0000 7fa4882a0740 1 -- 172.21.15.16:0/220056094 >> 172.21.15.16:0/220056094 conn(0x556050cc5300 msgr2=0x556051075950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.013+0000 7fa4882a0740 1 -- 172.21.15.16:0/220056094 shutdown_connections 2024-08-31T06:10:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.013+0000 7fa4882a0740 1 -- 172.21.15.16:0/220056094 wait complete. 2024-08-31T06:10:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.013+0000 7fa4882a0740 20 client.16155 trim_cache size 0 max 16384 2024-08-31T06:10:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.028+0000 7fa4882a0740 1 Processor -- start 2024-08-31T06:10:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.028+0000 7fa4882a0740 1 -- start start 2024-08-31T06:10:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.029+0000 7fa4882a0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556050ca82f0 0x556051076310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.029+0000 7fa4882a0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556050cb04d0 0x556051081b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.029+0000 7fa4882a0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556050cd11b0 0x55605107d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.029+0000 7fa4882a0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c1b3f30 con 0x556050cd11b0 2024-08-31T06:10:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.029+0000 7fa4882a0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c1b1740 con 0x556050ca82f0 2024-08-31T06:10:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.029+0000 7fa4882a0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c1af590 con 0x556050cb04d0 2024-08-31T06:10:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.029+0000 7fa47a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556050cd11b0 0x55605107d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.029+0000 7fa47a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556050cd11b0 0x55605107d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44494/0 (socket says 172.21.15.16:44494) 2024-08-31T06:10:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.029+0000 7fa47a800640 1 -- 172.21.15.16:0/2163380652 learned_addr learned my addr 172.21.15.16:0/2163380652 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.029+0000 7fa479400640 1 --2- 172.21.15.16:0/2163380652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556050cb04d0 0x556051081b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.029+0000 7fa479e00640 1 --2- 172.21.15.16:0/2163380652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556050ca82f0 0x556051076310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:25.033 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.030+0000 7fa479400640 1 -- 172.21.15.16:0/2163380652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556050ca82f0 msgr2=0x556051076310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:25.033 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.030+0000 7fa479400640 1 --2- 172.21.15.16:0/2163380652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556050ca82f0 0x556051076310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:25.033 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.030+0000 7fa479400640 1 -- 172.21.15.16:0/2163380652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556050cd11b0 msgr2=0x55605107d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:25.033 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.030+0000 7fa479400640 1 --2- 172.21.15.16:0/2163380652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556050cd11b0 0x55605107d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:25.033 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.030+0000 7fa479400640 1 -- 172.21.15.16:0/2163380652 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556051076c20 con 0x556050cb04d0 2024-08-31T06:10:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.030+0000 7fa47a800640 1 --2- 172.21.15.16:0/2163380652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556050cd11b0 0x55605107d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:10:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.030+0000 7fa479400640 1 --2- 172.21.15.16:0/2163380652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556050cb04d0 0x556051081b00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa4680e2eb0 tx=0x7fa4680eb550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69961e5dd6f05413 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.030+0000 7fa466a00640 1 -- 172.21.15.16:0/2163380652 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4680aba10 con 0x556050cb04d0 2024-08-31T06:10:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.030+0000 7fa466a00640 1 -- 172.21.15.16:0/2163380652 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4680abf00 con 0x556050cb04d0 2024-08-31T06:10:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.030+0000 7fa466a00640 1 -- 172.21.15.16:0/2163380652 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4680e3130 con 0x556050cb04d0 2024-08-31T06:10:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.031+0000 7fa4882a0740 1 -- 172.21.15.16:0/2163380652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556050cb04d0 msgr2=0x556051081b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.031+0000 7fa4882a0740 1 --2- 172.21.15.16:0/2163380652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556050cb04d0 0x556051081b00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa4680e2eb0 tx=0x7fa4680eb550 comp rx=0 tx=0).stop 2024-08-31T06:10:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.031+0000 7fa4882a0740 1 -- 172.21.15.16:0/2163380652 shutdown_connections 2024-08-31T06:10:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.031+0000 7fa4882a0740 1 --2- 172.21.15.16:0/2163380652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556050cb04d0 0x556051081b00 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.031+0000 7fa4882a0740 1 --2- 172.21.15.16:0/2163380652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556050cd11b0 0x55605107d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.031+0000 7fa4882a0740 1 --2- 172.21.15.16:0/2163380652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556050ca82f0 0x556051076310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.031+0000 7fa4882a0740 1 -- 172.21.15.16:0/2163380652 >> 172.21.15.16:0/2163380652 conn(0x556050cc5300 msgr2=0x5560510813f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.031+0000 7fa4882a0740 1 -- 172.21.15.16:0/2163380652 shutdown_connections 2024-08-31T06:10:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.031+0000 7fa4882a0740 1 -- 172.21.15.16:0/2163380652 wait complete. 2024-08-31T06:10:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.032+0000 7fa4882a0740 1 -- start start 2024-08-31T06:10:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.032+0000 7fa4882a0740 1 Processor -- start 2024-08-31T06:10:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.033+0000 7fa4882a0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556050ca82f0 0x556051088980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.033+0000 7fa4882a0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556050cb04d0 0x556051085e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.033+0000 7fa4882a0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556050cd11b0 0x556051086380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.033+0000 7fa4882a0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c1afc00 con 0x556050cd11b0 2024-08-31T06:10:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.033+0000 7fa4882a0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c1b1070 con 0x556050cb04d0 2024-08-31T06:10:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.033+0000 7fa4882a0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c1b02d0 con 0x556050ca82f0 2024-08-31T06:10:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.033+0000 7fa4882a0740 10 client.0 _ll_register_callbacks cb 0x7fa46c0317e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:10:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.033+0000 7fa479400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556050cb04d0 0x556051085e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.033+0000 7fa479400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556050cb04d0 0x556051085e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:38742/0 (socket says 172.21.15.16:38742) 2024-08-31T06:10:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.033+0000 7fa479400640 1 -- 172.21.15.16:0/2830695035 learned_addr learned my addr 172.21.15.16:0/2830695035 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.033+0000 7fa47a800640 1 --2- 172.21.15.16:0/2830695035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556050cd11b0 0x556051086380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.033+0000 7fa479e00640 1 --2- 172.21.15.16:0/2830695035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556050ca82f0 0x556051088980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.033+0000 7fa479400640 1 -- 172.21.15.16:0/2830695035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556050ca82f0 msgr2=0x556051088980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.033+0000 7fa479400640 1 --2- 172.21.15.16:0/2830695035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556050ca82f0 0x556051088980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.033+0000 7fa479400640 1 -- 172.21.15.16:0/2830695035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556050cd11b0 msgr2=0x556051086380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.033+0000 7fa479400640 1 --2- 172.21.15.16:0/2830695035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556050cd11b0 0x556051086380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.033+0000 7fa479400640 1 -- 172.21.15.16:0/2830695035 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560510869c0 con 0x556050cb04d0 2024-08-31T06:10:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.033+0000 7fa479e00640 1 --2- 172.21.15.16:0/2830695035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556050ca82f0 0x556051088980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:10:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.034+0000 7fa479400640 1 --2- 172.21.15.16:0/2830695035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556050cb04d0 0x556051085e40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa4680a7f80 tx=0x7fa46800cef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52d17efea48c4c9d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.034+0000 7fa45be00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:10:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.034+0000 7fa45be00640 1 -- 172.21.15.16:0/2830695035 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4680b06c0 con 0x556050cb04d0 2024-08-31T06:10:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.034+0000 7fa45be00640 1 -- 172.21.15.16:0/2830695035 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fa4680a8190 con 0x556050cb04d0 2024-08-31T06:10:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.034+0000 7fa4882a0740 1 -- 172.21.15.16:0/2830695035 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4680abf00 con 0x556050cb04d0 2024-08-31T06:10:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.034+0000 7fa45be00640 1 -- 172.21.15.16:0/2830695035 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa4680e3130 con 0x556050cb04d0 2024-08-31T06:10:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.034+0000 7fa4882a0740 10 client.26279 fetch_fsmap learned FSMap version 31 2024-08-31T06:10:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.034+0000 7fa4882a0740 1 -- 172.21.15.16:0/2830695035 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5560510fc900 con 0x556050cb04d0 2024-08-31T06:10:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.034+0000 7fa45be00640 1 -- 172.21.15.16:0/2830695035 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4680abf00 con 0x556050cb04d0 2024-08-31T06:10:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.034+0000 7fa45be00640 1 -- 172.21.15.16:0/2830695035 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fa4680930b0 con 0x556050cb04d0 2024-08-31T06:10:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.035+0000 7fa4882a0740 10 client.26279 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:10:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.035+0000 7fa4882a0740 10 client.26279 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:10:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.035+0000 7fa4882a0740 20 client.26279 populate_metadata read hostname 'smithi016' 2024-08-31T06:10:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.035+0000 7fa4882a0740 1 --2- 172.21.15.16:0/2830695035 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556050c651c0 0x5560510ff0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.035+0000 7fa4882a0740 4 client.26279 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:10:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.035+0000 7fa4882a0740 1 -- 172.21.15.16:0/2830695035 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556050fe7960 con 0x556050c651c0 2024-08-31T06:10:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.035+0000 7fa479e00640 1 --2- 172.21.15.16:0/2830695035 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556050c651c0 0x5560510ff0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.035+0000 7fa45be00640 10 client.26279 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.035+0000 7fa479e00640 1 --2- 172.21.15.16:0/2830695035 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556050c651c0 0x5560510ff0b0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b42978bb4ce3926 server_cookie=8d5120615d09272 in_seq=0 out_seq=0 2024-08-31T06:10:25.054 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.052+0000 7fa45be00640 1 -- 172.21.15.16:0/2830695035 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556050fe7960 con 0x556050c651c0 2024-08-31T06:10:25.054 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.052+0000 7fa45be00640 10 client.26279 handle_command_reply: tid=0 2024-08-31T06:10:25.054 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.052+0000 7fa4882a0740 1 client.26279 shutdown 2024-08-31T06:10:25.054 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.053+0000 7fa4882a0740 1 -- 172.21.15.16:0/2830695035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556050cb04d0 msgr2=0x556051085e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.053+0000 7fa4882a0740 1 --2- 172.21.15.16:0/2830695035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556050cb04d0 0x556051085e40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa4680a7f80 tx=0x7fa46800cef0 comp rx=0 tx=0).stop 2024-08-31T06:10:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.053+0000 7fa4882a0740 1 -- 172.21.15.16:0/2830695035 shutdown_connections 2024-08-31T06:10:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.053+0000 7fa4882a0740 1 --2- 172.21.15.16:0/2830695035 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556050c651c0 0x5560510ff0b0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.053+0000 7fa4882a0740 1 --2- 172.21.15.16:0/2830695035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556050cd11b0 0x556051086380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.053+0000 7fa45be00640 0 client.26279 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.053+0000 7fa4882a0740 1 --2- 172.21.15.16:0/2830695035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556050cb04d0 0x556051085e40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.053+0000 7fa4882a0740 1 --2- 172.21.15.16:0/2830695035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556050ca82f0 0x556051088980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.053+0000 7fa4882a0740 1 -- 172.21.15.16:0/2830695035 >> 172.21.15.16:0/2830695035 conn(0x556050cc5300 msgr2=0x5560510760e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.053+0000 7fa4882a0740 1 -- 172.21.15.16:0/2830695035 shutdown_connections 2024-08-31T06:10:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.053+0000 7fa4882a0740 1 -- 172.21.15.16:0/2830695035 wait complete. 2024-08-31T06:10:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.053+0000 7fa4882a0740 20 client.26279 trim_cache size 0 max 16384 2024-08-31T06:10:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:10:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:10:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:10:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:10:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.054+0000 7fa481200640 1 -- 172.21.15.16:0/1776782921 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa4600b4d20 msgr2=0x7fa4600b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.054+0000 7fa481200640 1 --2- 172.21.15.16:0/1776782921 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa4600b4d20 0x7fa4600b7230 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa46c031440 tx=0x7fa46c032ed0 comp rx=0 tx=0).stop 2024-08-31T06:10:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.054+0000 7fa481200640 1 -- 172.21.15.16:0/1776782921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa47c1091a0 msgr2=0x7fa47c19c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.054+0000 7fa481200640 1 --2- 172.21.15.16:0/1776782921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa47c1091a0 0x7fa47c19c9a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa46800e9f0 tx=0x7fa46800eeb0 comp rx=0 tx=0).stop 2024-08-31T06:10:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.054+0000 7fa481200640 1 -- 172.21.15.16:0/1776782921 shutdown_connections 2024-08-31T06:10:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.054+0000 7fa481200640 1 --2- 172.21.15.16:0/1776782921 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa4600b4d20 0x7fa4600b7230 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.054+0000 7fa481200640 1 --2- 172.21.15.16:0/1776782921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa47c109af0 0x7fa47c1a1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.054+0000 7fa481200640 1 --2- 172.21.15.16:0/1776782921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa47c1091a0 0x7fa47c19c9a0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.054+0000 7fa481200640 1 --2- 172.21.15.16:0/1776782921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa47c103870 0x7fa47c19c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.054+0000 7fa481200640 1 -- 172.21.15.16:0/1776782921 >> 172.21.15.16:0/1776782921 conn(0x7fa47c0ff2b0 msgr2=0x7fa47c1067a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.054+0000 7fa481200640 1 -- 172.21.15.16:0/1776782921 shutdown_connections 2024-08-31T06:10:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:25.054+0000 7fa481200640 1 -- 172.21.15.16:0/1776782921 wait complete. 2024-08-31T06:10:26.629 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 180.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-31T06:10:26.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:26 smithi016 ceph-mon[17194]: pgmap v460: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 10 MiB/s wr, 531 op/s 2024-08-31T06:10:26.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:26 smithi155 ceph-mon[21742]: pgmap v460: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 10 MiB/s wr, 531 op/s 2024-08-31T06:10:26.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:26 smithi184 ceph-mon[22331]: pgmap v460: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 10 MiB/s wr, 531 op/s 2024-08-31T06:10:28.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:28 smithi155 ceph-mon[21742]: pgmap v461: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 906 KiB/s wr, 109 op/s 2024-08-31T06:10:28.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:28 smithi016 ceph-mon[17194]: pgmap v461: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 906 KiB/s wr, 109 op/s 2024-08-31T06:10:28.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:28 smithi184 ceph-mon[22331]: pgmap v461: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 906 KiB/s wr, 109 op/s 2024-08-31T06:10:30.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:30 smithi155 ceph-mon[21742]: pgmap v462: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 670 KiB/s wr, 81 op/s 2024-08-31T06:10:30.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:30 smithi016 ceph-mon[17194]: pgmap v462: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 670 KiB/s wr, 81 op/s 2024-08-31T06:10:30.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:30 smithi184 ceph-mon[22331]: pgmap v462: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 670 KiB/s wr, 81 op/s 2024-08-31T06:10:31.628 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 185.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-31T06:10:31.896 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:10:32.293 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:10:32.739 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:32 smithi016 ceph-mon[17194]: pgmap v463: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 281 KiB/s wr, 35 op/s 2024-08-31T06:10:32.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:32 smithi155 ceph-mon[21742]: pgmap v463: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 281 KiB/s wr, 35 op/s 2024-08-31T06:10:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:32.916+0000 7faea7000640 1 -- 172.21.15.16:0/1944582159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faea0105950 msgr2=0x7faea0105d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:32.916+0000 7faea7000640 1 --2- 172.21.15.16:0/1944582159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faea0105950 0x7faea0105d60 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fae94009900 tx=0x7fae9402a3f0 comp rx=0 tx=0).stop 2024-08-31T06:10:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:32.916+0000 7faea7000640 1 -- 172.21.15.16:0/1944582159 shutdown_connections 2024-08-31T06:10:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:32.916+0000 7faea7000640 1 --2- 172.21.15.16:0/1944582159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faea010b290 0x7faea010b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:32.916+0000 7faea7000640 1 --2- 172.21.15.16:0/1944582159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faea010bbe0 0x7faea0112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:32.916+0000 7faea7000640 1 --2- 172.21.15.16:0/1944582159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faea0105950 0x7faea0105d60 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:32.916+0000 7faea7000640 1 -- 172.21.15.16:0/1944582159 >> 172.21.15.16:0/1944582159 conn(0x7faea0101420 msgr2=0x7faea0103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:32.916+0000 7faea7000640 1 -- 172.21.15.16:0/1944582159 shutdown_connections 2024-08-31T06:10:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:32.916+0000 7faea7000640 1 -- 172.21.15.16:0/1944582159 wait complete. 2024-08-31T06:10:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:32.917+0000 7faea7000640 1 Processor -- start 2024-08-31T06:10:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:32.918+0000 7faea7000640 1 -- start start 2024-08-31T06:10:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:32.918+0000 7faea7000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faea0105950 0x7faea019c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:32.918+0000 7faea7000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faea010b290 0x7faea019c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:32.918+0000 7faea7000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faea010bbe0 0x7faea01a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:32.918+0000 7faea7000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faea01158f0 con 0x7faea0105950 2024-08-31T06:10:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:32.918+0000 7faea7000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faea0115770 con 0x7faea010b290 2024-08-31T06:10:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:32.918+0000 7faea7000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faea0115a70 con 0x7faea010bbe0 2024-08-31T06:10:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:32.918+0000 7faea5c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faea0105950 0x7faea019c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:32.918+0000 7faea5200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faea010b290 0x7faea019c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:32.918+0000 7faea5c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faea0105950 0x7faea019c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49706/0 (socket says 172.21.15.16:49706) 2024-08-31T06:10:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:32.918+0000 7faea5c00640 1 -- 172.21.15.16:0/493610093 learned_addr learned my addr 172.21.15.16:0/493610093 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:32.919+0000 7faea6600640 1 --2- 172.21.15.16:0/493610093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faea010bbe0 0x7faea01a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:32.919+0000 7faea5200640 1 -- 172.21.15.16:0/493610093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faea010bbe0 msgr2=0x7faea01a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:32.919+0000 7faea5200640 1 --2- 172.21.15.16:0/493610093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faea010bbe0 0x7faea01a1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:32.919+0000 7faea5200640 1 -- 172.21.15.16:0/493610093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faea0105950 msgr2=0x7faea019c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:32.919+0000 7faea5200640 1 --2- 172.21.15.16:0/493610093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faea0105950 0x7faea019c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:32.919+0000 7faea5200640 1 -- 172.21.15.16:0/493610093 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faea01a2250 con 0x7faea010b290 2024-08-31T06:10:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:32.919+0000 7faea5c00640 1 --2- 172.21.15.16:0/493610093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faea0105950 0x7faea019c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:10:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:32.919+0000 7faea5200640 1 --2- 172.21.15.16:0/493610093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faea010b290 0x7faea019c880 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7faea006c3d0 tx=0x7fae8c00dbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:32.920+0000 7fae9e000640 1 -- 172.21.15.16:0/493610093 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c017070 con 0x7faea010b290 2024-08-31T06:10:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:32.920+0000 7faea7000640 1 -- 172.21.15.16:0/493610093 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faea01a2540 con 0x7faea010b290 2024-08-31T06:10:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:32.920+0000 7fae9e000640 1 -- 172.21.15.16:0/493610093 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae8c0099a0 con 0x7faea010b290 2024-08-31T06:10:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:32.920+0000 7fae9e000640 1 -- 172.21.15.16:0/493610093 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c0047a0 con 0x7faea010b290 2024-08-31T06:10:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:32.920+0000 7faea7000640 1 -- 172.21.15.16:0/493610093 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faea01a8ca0 con 0x7faea010b290 2024-08-31T06:10:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:32.921+0000 7faea7000640 1 -- 172.21.15.16:0/493610093 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faea0105d60 con 0x7faea010b290 2024-08-31T06:10:32.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:32.924+0000 7fae9e000640 1 -- 172.21.15.16:0/493610093 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fae8c004940 con 0x7faea010b290 2024-08-31T06:10:32.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:32.925+0000 7fae9e000640 1 --2- 172.21.15.16:0/493610093 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fae880b4ca0 0x7fae880b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:32.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:32.925+0000 7fae9e000640 1 -- 172.21.15.16:0/493610093 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fae8c0e5540 con 0x7faea010b290 2024-08-31T06:10:32.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:32.925+0000 7faea5c00640 1 --2- 172.21.15.16:0/493610093 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fae880b4ca0 0x7fae880b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:32.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:32.926+0000 7faea5c00640 1 --2- 172.21.15.16:0/493610093 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fae880b4ca0 0x7fae880b71b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fae940096c0 tx=0x7fae94005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:32.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:32.927+0000 7fae9e000640 1 -- 172.21.15.16:0/493610093 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fae8c020020 con 0x7faea010b290 2024-08-31T06:10:32.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:32 smithi184 ceph-mon[22331]: pgmap v463: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 281 KiB/s wr, 35 op/s 2024-08-31T06:10:33.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:33.160+0000 7faea7000640 1 -- 172.21.15.16:0/493610093 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faea010da10 con 0x7faea010b290 2024-08-31T06:10:33.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:33.161+0000 7fae9e000640 1 -- 172.21.15.16:0/493610093 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11923 (secure 0 0 0) 0x7fae8c0ac330 con 0x7faea010b290 2024-08-31T06:10:33.164 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:10:33.164 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":31,"btime":"2024-08-31T06:08:47:452652+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:08:47.004817+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24737,24724,24712]},"id":1}]} 2024-08-31T06:10:33.165 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 31 2024-08-31T06:10:33.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:33.164+0000 7faea7000640 1 -- 172.21.15.16:0/493610093 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fae880b4ca0 msgr2=0x7fae880b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:33.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:33.164+0000 7faea7000640 1 --2- 172.21.15.16:0/493610093 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fae880b4ca0 0x7fae880b71b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fae940096c0 tx=0x7fae94005a20 comp rx=0 tx=0).stop 2024-08-31T06:10:33.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:33.165+0000 7faea7000640 1 -- 172.21.15.16:0/493610093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faea010b290 msgr2=0x7faea019c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:33.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:33.165+0000 7faea7000640 1 --2- 172.21.15.16:0/493610093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faea010b290 0x7faea019c880 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7faea006c3d0 tx=0x7fae8c00dbb0 comp rx=0 tx=0).stop 2024-08-31T06:10:33.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:33.165+0000 7faea7000640 1 -- 172.21.15.16:0/493610093 shutdown_connections 2024-08-31T06:10:33.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:33.165+0000 7faea7000640 1 --2- 172.21.15.16:0/493610093 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fae880b4ca0 0x7fae880b71b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:33.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:33.165+0000 7faea7000640 1 --2- 172.21.15.16:0/493610093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faea010bbe0 0x7faea01a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:33.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:33.165+0000 7faea7000640 1 --2- 172.21.15.16:0/493610093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faea010b290 0x7faea019c880 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:33.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:33.165+0000 7faea7000640 1 --2- 172.21.15.16:0/493610093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faea0105950 0x7faea019c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:33.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:33.165+0000 7faea7000640 1 -- 172.21.15.16:0/493610093 >> 172.21.15.16:0/493610093 conn(0x7faea0101420 msgr2=0x7faea0103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:33.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:33.165+0000 7faea7000640 1 -- 172.21.15.16:0/493610093 shutdown_connections 2024-08-31T06:10:33.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:33.165+0000 7faea7000640 1 -- 172.21.15.16:0/493610093 wait complete. 2024-08-31T06:10:33.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:33 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/493610093' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:10:33.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:33 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/493610093' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:10:33.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:33 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/493610093' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:10:34.024 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24737 quiesce db --set-id 53a757a7 --release --await' 2024-08-31T06:10:34.024 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24737 quiesce db --set-id 53a757a7 --release --await 2024-08-31T06:10:34.410 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:10:34.869 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:34 smithi016 ceph-mon[17194]: pgmap v464: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 9 op/s 2024-08-31T06:10:34.949 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:34 smithi184 ceph-mon[22331]: pgmap v464: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 9 op/s 2024-08-31T06:10:34.949 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:34 smithi155 ceph-mon[21742]: pgmap v464: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 9 op/s 2024-08-31T06:10:35.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.214+0000 7f9c02e00640 1 -- 172.21.15.16:0/3129811042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bfc109240 msgr2=0x7f9bfc109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:35.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.214+0000 7f9c02e00640 1 --2- 172.21.15.16:0/3129811042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bfc109240 0x7f9bfc109650 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f9be8009900 tx=0x7f9be802a410 comp rx=0 tx=0).stop 2024-08-31T06:10:35.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.214+0000 7f9c02e00640 1 -- 172.21.15.16:0/3129811042 shutdown_connections 2024-08-31T06:10:35.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.214+0000 7f9c02e00640 1 --2- 172.21.15.16:0/3129811042 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9bfc109b90 0x7f9bfc1103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:35.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.214+0000 7f9c02e00640 1 --2- 172.21.15.16:0/3129811042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bfc109240 0x7f9bfc109650 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:35.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.214+0000 7f9c02e00640 1 --2- 172.21.15.16:0/3129811042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9bfc103900 0x7f9bfc103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:35.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.214+0000 7f9c02e00640 1 -- 172.21.15.16:0/3129811042 >> 172.21.15.16:0/3129811042 conn(0x7f9bfc07b5f0 msgr2=0x7f9bfc07ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:35.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.214+0000 7f9c02e00640 1 -- 172.21.15.16:0/3129811042 shutdown_connections 2024-08-31T06:10:35.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.215+0000 7f9c02e00640 1 -- 172.21.15.16:0/3129811042 wait complete. 2024-08-31T06:10:35.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.215+0000 7f9c02e00640 1 Processor -- start 2024-08-31T06:10:35.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.216+0000 7f9c02e00640 1 -- start start 2024-08-31T06:10:35.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.216+0000 7f9c02e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9bfc103900 0x7f9bfc19c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:35.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.216+0000 7f9c02e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bfc109240 0x7f9bfc19c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:35.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.216+0000 7f9c02e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9bfc109b90 0x7f9bfc1a1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:35.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.216+0000 7f9c02e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9bfc113850 con 0x7f9bfc109240 2024-08-31T06:10:35.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.216+0000 7f9c02e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9bfc1136d0 con 0x7f9bfc109b90 2024-08-31T06:10:35.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.216+0000 7f9c02e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9bfc1139d0 con 0x7f9bfc103900 2024-08-31T06:10:35.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.216+0000 7f9bfb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bfc109240 0x7f9bfc19c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:35.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.216+0000 7f9bfb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bfc109240 0x7f9bfc19c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49724/0 (socket says 172.21.15.16:49724) 2024-08-31T06:10:35.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.216+0000 7f9bfb400640 1 -- 172.21.15.16:0/1587145193 learned_addr learned my addr 172.21.15.16:0/1587145193 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:35.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.216+0000 7f9bfbe00640 1 --2- 172.21.15.16:0/1587145193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9bfc103900 0x7f9bfc19c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:35.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.217+0000 7f9c00e00640 1 --2- 172.21.15.16:0/1587145193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9bfc109b90 0x7f9bfc1a1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:35.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.217+0000 7f9bfb400640 1 -- 172.21.15.16:0/1587145193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9bfc103900 msgr2=0x7f9bfc19c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:35.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.217+0000 7f9bfb400640 1 --2- 172.21.15.16:0/1587145193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9bfc103900 0x7f9bfc19c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:35.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.217+0000 7f9bfb400640 1 -- 172.21.15.16:0/1587145193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9bfc109b90 msgr2=0x7f9bfc1a1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:35.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.217+0000 7f9bfb400640 1 --2- 172.21.15.16:0/1587145193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9bfc109b90 0x7f9bfc1a1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:35.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.217+0000 7f9bfb400640 1 -- 172.21.15.16:0/1587145193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bfc1a2390 con 0x7f9bfc109240 2024-08-31T06:10:35.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.217+0000 7f9bfb400640 1 --2- 172.21.15.16:0/1587145193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bfc109240 0x7f9bfc19c9c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f9be802a930 tx=0x7f9be8031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:35.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.218+0000 7f9bf8c00640 1 -- 172.21.15.16:0/1587145193 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be803f030 con 0x7f9bfc109240 2024-08-31T06:10:35.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.218+0000 7f9bf8c00640 1 -- 172.21.15.16:0/1587145193 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9be8004490 con 0x7f9bfc109240 2024-08-31T06:10:35.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.218+0000 7f9c02e00640 1 -- 172.21.15.16:0/1587145193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bfc1a2620 con 0x7f9bfc109240 2024-08-31T06:10:35.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.218+0000 7f9bf8c00640 1 -- 172.21.15.16:0/1587145193 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be8035070 con 0x7f9bfc109240 2024-08-31T06:10:35.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.218+0000 7f9c02e00640 1 -- 172.21.15.16:0/1587145193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bfc1a8de0 con 0x7f9bfc109240 2024-08-31T06:10:35.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.222+0000 7f9c09bce740 1 Processor -- start 2024-08-31T06:10:35.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.222+0000 7f9c09bce740 1 -- start start 2024-08-31T06:10:35.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.223+0000 7f9c09bce740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557482a71520 0x557482e16ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:35.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.223+0000 7f9c09bce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557482a48660 0x557482e17430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:35.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.223+0000 7f9c09bce740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557482a50840 0x557482e17970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:35.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.223+0000 7f9c09bce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557482d8da90 con 0x557482a48660 2024-08-31T06:10:35.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.223+0000 7f9c09bce740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557482d7d020 con 0x557482a71520 2024-08-31T06:10:35.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.223+0000 7f9c09bce740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557482d80fd0 con 0x557482a50840 2024-08-31T06:10:35.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.223+0000 7f9bf8c00640 1 -- 172.21.15.16:0/1587145193 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f9be8042030 con 0x7f9bfc109240 2024-08-31T06:10:35.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.223+0000 7f9bfbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557482a48660 0x557482e17430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:35.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.223+0000 7f9bfbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557482a48660 0x557482e17430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49738/0 (socket says 172.21.15.16:49738) 2024-08-31T06:10:35.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.223+0000 7f9bfbe00640 1 -- 172.21.15.16:0/929131583 learned_addr learned my addr 172.21.15.16:0/929131583 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:35.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.223+0000 7f9c00e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557482a71520 0x557482e16ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:35.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.223+0000 7f9bfb400640 1 --2- 172.21.15.16:0/929131583 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557482a50840 0x557482e17970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:35.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.224+0000 7f9c00e00640 1 -- 172.21.15.16:0/929131583 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557482a50840 msgr2=0x557482e17970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:35.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.224+0000 7f9c00e00640 1 --2- 172.21.15.16:0/929131583 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557482a50840 0x557482e17970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:35.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.224+0000 7f9c00e00640 1 -- 172.21.15.16:0/929131583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557482a48660 msgr2=0x557482e17430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:35.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.224+0000 7f9c00e00640 1 --2- 172.21.15.16:0/929131583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557482a48660 0x557482e17430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:35.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.224+0000 7f9c00e00640 1 -- 172.21.15.16:0/929131583 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557482adcb00 con 0x557482a71520 2024-08-31T06:10:35.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.224+0000 7f9bfbe00640 1 --2- 172.21.15.16:0/929131583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557482a48660 0x557482e17430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:10:35.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.224+0000 7f9c00e00640 1 --2- 172.21.15.16:0/929131583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557482a71520 0x557482e16ef0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4009ed0 tx=0x7f9bf40107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b478ad686ba16004 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:35.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.224+0000 7f9bf1600640 1 -- 172.21.15.16:0/929131583 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4003b70 con 0x557482a71520 2024-08-31T06:10:35.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.224+0000 7f9bf1600640 1 -- 172.21.15.16:0/929131583 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bf4015070 con 0x557482a71520 2024-08-31T06:10:35.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.224+0000 7f9bf1600640 1 -- 172.21.15.16:0/929131583 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf40117b0 con 0x557482a71520 2024-08-31T06:10:35.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.225+0000 7f9bf8c00640 1 --2- 172.21.15.16:0/1587145193 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9be00b4ca0 0x7f9be00b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:35.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.225+0000 7f9bf8c00640 1 -- 172.21.15.16:0/1587145193 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f9be8105870 con 0x7f9bfc109240 2024-08-31T06:10:35.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.225+0000 7f9bfbe00640 1 --2- 172.21.15.16:0/1587145193 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9be00b4ca0 0x7f9be00b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:35.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.225+0000 7f9c09bce740 1 -- 172.21.15.16:0/929131583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557482a71520 msgr2=0x557482e16ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:35.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.225+0000 7f9c09bce740 1 --2- 172.21.15.16:0/929131583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557482a71520 0x557482e16ef0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4009ed0 tx=0x7f9bf40107a0 comp rx=0 tx=0).stop 2024-08-31T06:10:35.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.225+0000 7f9c09bce740 1 -- 172.21.15.16:0/929131583 shutdown_connections 2024-08-31T06:10:35.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.225+0000 7f9c09bce740 1 --2- 172.21.15.16:0/929131583 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557482a50840 0x557482e17970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:35.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.225+0000 7f9c09bce740 1 --2- 172.21.15.16:0/929131583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557482a48660 0x557482e17430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:35.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.225+0000 7f9c09bce740 1 --2- 172.21.15.16:0/929131583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557482a71520 0x557482e16ef0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:35.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.225+0000 7f9c09bce740 1 -- 172.21.15.16:0/929131583 >> 172.21.15.16:0/929131583 conn(0x557482a65910 msgr2=0x557482e14de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:35.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.226+0000 7f9c09bce740 1 -- 172.21.15.16:0/929131583 shutdown_connections 2024-08-31T06:10:35.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.226+0000 7f9c09bce740 1 -- 172.21.15.16:0/929131583 wait complete. 2024-08-31T06:10:35.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.226+0000 7f9bfbe00640 1 --2- 172.21.15.16:0/1587145193 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9be00b4ca0 0x7f9be00b71b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9bec009ab0 tx=0x7f9bec005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:35.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.226+0000 7f9c09bce740 1 -- start start 2024-08-31T06:10:35.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.227+0000 7f9c09bce740 1 Processor -- start 2024-08-31T06:10:35.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.227+0000 7f9c09bce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557482a48660 0x557482e22310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:35.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.227+0000 7f9c09bce740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557482a50840 0x557482e22850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:35.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.227+0000 7f9c09bce740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557482a71520 0x557482e22d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:35.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.227+0000 7f9c09bce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557482bdfbb0 con 0x557482a48660 2024-08-31T06:10:35.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.227+0000 7f9c09bce740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557482d03f90 con 0x557482a71520 2024-08-31T06:10:35.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.227+0000 7f9c09bce740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557482e256d0 con 0x557482a50840 2024-08-31T06:10:35.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.227+0000 7f9c09bce740 10 client.0 _ll_register_callbacks cb 0x557482d29710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:10:35.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.228+0000 7f9bfb400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557482a50840 0x557482e22850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:35.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.228+0000 7f9bfb400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557482a50840 0x557482e22850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:34074/0 (socket says 172.21.15.16:34074) 2024-08-31T06:10:35.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.228+0000 7f9bfb400640 1 -- 172.21.15.16:0/1451263129 learned_addr learned my addr 172.21.15.16:0/1451263129 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:35.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.228+0000 7f9c00e00640 1 --2- 172.21.15.16:0/1451263129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557482a71520 0x557482e22d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:35.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.228+0000 7f9bfbe00640 1 --2- 172.21.15.16:0/1451263129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557482a48660 0x557482e22310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:35.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.228+0000 7f9bfb400640 1 -- 172.21.15.16:0/1451263129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557482a71520 msgr2=0x557482e22d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:35.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.228+0000 7f9bfb400640 1 --2- 172.21.15.16:0/1451263129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557482a71520 0x557482e22d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:35.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.228+0000 7f9bfb400640 1 -- 172.21.15.16:0/1451263129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557482a48660 msgr2=0x557482e22310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:35.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.228+0000 7f9bfb400640 1 --2- 172.21.15.16:0/1451263129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557482a48660 0x557482e22310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:35.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.228+0000 7f9bfb400640 1 -- 172.21.15.16:0/1451263129 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557482b53960 con 0x557482a50840 2024-08-31T06:10:35.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.228+0000 7f9bfbe00640 1 --2- 172.21.15.16:0/1451263129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557482a48660 0x557482e22310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:10:35.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.228+0000 7f9bfb400640 1 --2- 172.21.15.16:0/1451263129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557482a50840 0x557482e22850 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9be8006010 tx=0x7f9be8009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f771f786d8c2109 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:35.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.228+0000 7f9be6000640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:10:35.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.229+0000 7f9be6000640 1 -- 172.21.15.16:0/1451263129 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be8049050 con 0x557482a50840 2024-08-31T06:10:35.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.229+0000 7f9be6000640 1 -- 172.21.15.16:0/1451263129 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f9be80c8250 con 0x557482a50840 2024-08-31T06:10:35.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.229+0000 7f9c09bce740 1 -- 172.21.15.16:0/1451263129 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557482d1be30 con 0x557482a50840 2024-08-31T06:10:35.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.229+0000 7f9be6000640 1 -- 172.21.15.16:0/1451263129 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9be80cbad0 con 0x557482a50840 2024-08-31T06:10:35.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.229+0000 7f9c09bce740 10 client.26290 fetch_fsmap learned FSMap version 31 2024-08-31T06:10:35.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.229+0000 7f9c09bce740 1 -- 172.21.15.16:0/1451263129 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557482e31090 con 0x557482a50840 2024-08-31T06:10:35.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.229+0000 7f9be6000640 1 -- 172.21.15.16:0/1451263129 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557482d1be30 con 0x557482a50840 2024-08-31T06:10:35.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.229+0000 7f9be6000640 1 -- 172.21.15.16:0/1451263129 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f9be80ce080 con 0x557482a50840 2024-08-31T06:10:35.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.229+0000 7f9c09bce740 10 client.26290 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:10:35.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.229+0000 7f9c09bce740 10 client.26290 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:10:35.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.229+0000 7f9c09bce740 20 client.26290 populate_metadata read hostname 'smithi016' 2024-08-31T06:10:35.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.230+0000 7f9c09bce740 1 --2- 172.21.15.16:0/1451263129 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557482a2fea0 0x557482e345a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:35.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.230+0000 7f9c09bce740 4 client.26290 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:10:35.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.230+0000 7f9c09bce740 1 -- 172.21.15.16:0/1451263129 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557482d8f9e0 con 0x557482a2fea0 2024-08-31T06:10:35.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.230+0000 7f9bfbe00640 1 --2- 172.21.15.16:0/1451263129 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557482a2fea0 0x557482e345a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:35.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.230+0000 7f9bfbe00640 1 --2- 172.21.15.16:0/1451263129 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557482a2fea0 0x557482e345a0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec58e881d0b18026 server_cookie=7e0a9f90debb1c72 in_seq=0 out_seq=0 2024-08-31T06:10:35.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.230+0000 7f9be6000640 10 client.26290 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:35.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.231+0000 7f9be6000640 1 -- 172.21.15.16:0/1451263129 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557482d8f9e0 con 0x557482a2fea0 2024-08-31T06:10:35.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.231+0000 7f9be6000640 10 client.26290 handle_command_reply: tid=0 2024-08-31T06:10:35.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.231+0000 7f9c09bce740 1 client.26290 shutdown 2024-08-31T06:10:35.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.232+0000 7f9c09bce740 1 -- 172.21.15.16:0/1451263129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557482a50840 msgr2=0x557482e22850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:35.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.232+0000 7f9c09bce740 1 --2- 172.21.15.16:0/1451263129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557482a50840 0x557482e22850 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9be8006010 tx=0x7f9be8009260 comp rx=0 tx=0).stop 2024-08-31T06:10:35.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.232+0000 7f9c09bce740 1 -- 172.21.15.16:0/1451263129 shutdown_connections 2024-08-31T06:10:35.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.232+0000 7f9c09bce740 1 --2- 172.21.15.16:0/1451263129 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557482a2fea0 0x557482e345a0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:35.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.232+0000 7f9c09bce740 1 --2- 172.21.15.16:0/1451263129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557482a50840 0x557482e22850 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:35.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.232+0000 7f9c09bce740 1 --2- 172.21.15.16:0/1451263129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557482a71520 0x557482e22d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:35.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.232+0000 7f9c09bce740 1 --2- 172.21.15.16:0/1451263129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557482a48660 0x557482e22310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:35.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.232+0000 7f9be6000640 0 client.26290 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:35.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.232+0000 7f9c09bce740 1 -- 172.21.15.16:0/1451263129 >> 172.21.15.16:0/1451263129 conn(0x557482a65910 msgr2=0x557482e1a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:35.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.232+0000 7f9c09bce740 1 -- 172.21.15.16:0/1451263129 shutdown_connections 2024-08-31T06:10:35.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.232+0000 7f9c09bce740 1 -- 172.21.15.16:0/1451263129 wait complete. 2024-08-31T06:10:35.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.232+0000 7f9c09bce740 20 client.26290 trim_cache size 0 max 16384 2024-08-31T06:10:35.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.250+0000 7f9c09bce740 1 Processor -- start 2024-08-31T06:10:35.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.251+0000 7f9c09bce740 1 -- start start 2024-08-31T06:10:35.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.251+0000 7f9c09bce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557482a48660 0x557482e14e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:35.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.251+0000 7f9c09bce740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557482a50840 0x557482e1f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:35.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.251+0000 7f9c09bce740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557482a71520 0x557482e1fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:35.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.251+0000 7f9c09bce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9bfc1b3e60 con 0x557482a48660 2024-08-31T06:10:35.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.251+0000 7f9c09bce740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9bfc1b1670 con 0x557482a71520 2024-08-31T06:10:35.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.251+0000 7f9c09bce740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9bfc1af4c0 con 0x557482a50840 2024-08-31T06:10:35.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.251+0000 7f9bfbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557482a48660 0x557482e14e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:35.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.251+0000 7f9bfbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557482a48660 0x557482e14e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49750/0 (socket says 172.21.15.16:49750) 2024-08-31T06:10:35.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.251+0000 7f9bfbe00640 1 -- 172.21.15.16:0/1811911927 learned_addr learned my addr 172.21.15.16:0/1811911927 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:35.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.251+0000 7f9c00e00640 1 --2- 172.21.15.16:0/1811911927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557482a71520 0x557482e1fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:35.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.251+0000 7f9bfb400640 1 --2- 172.21.15.16:0/1811911927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557482a50840 0x557482e1f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:35.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.252+0000 7f9bfbe00640 1 -- 172.21.15.16:0/1811911927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557482a50840 msgr2=0x557482e1f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:35.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.252+0000 7f9bfbe00640 1 --2- 172.21.15.16:0/1811911927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557482a50840 0x557482e1f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:35.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.252+0000 7f9bfbe00640 1 -- 172.21.15.16:0/1811911927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557482a71520 msgr2=0x557482e1fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:35.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.252+0000 7f9bfbe00640 1 --2- 172.21.15.16:0/1811911927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557482a71520 0x557482e1fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:35.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.252+0000 7f9bfbe00640 1 -- 172.21.15.16:0/1811911927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557482e194b0 con 0x557482a48660 2024-08-31T06:10:35.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.252+0000 7f9bfbe00640 1 --2- 172.21.15.16:0/1811911927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557482a48660 0x557482e14e30 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9bec012fe0 tx=0x7f9bec01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546627db2bedf379 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:35.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.252+0000 7f9bf0c00640 1 -- 172.21.15.16:0/1811911927 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec02f400 con 0x557482a48660 2024-08-31T06:10:35.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.252+0000 7f9bf0c00640 1 -- 172.21.15.16:0/1811911927 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bec034070 con 0x557482a48660 2024-08-31T06:10:35.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.252+0000 7f9bf0c00640 1 -- 172.21.15.16:0/1811911927 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec029670 con 0x557482a48660 2024-08-31T06:10:35.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.253+0000 7f9c09bce740 1 -- 172.21.15.16:0/1811911927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557482a48660 msgr2=0x557482e14e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:35.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.253+0000 7f9c09bce740 1 --2- 172.21.15.16:0/1811911927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557482a48660 0x557482e14e30 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9bec012fe0 tx=0x7f9bec01b000 comp rx=0 tx=0).stop 2024-08-31T06:10:35.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.253+0000 7f9c09bce740 1 -- 172.21.15.16:0/1811911927 shutdown_connections 2024-08-31T06:10:35.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.253+0000 7f9c09bce740 1 --2- 172.21.15.16:0/1811911927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557482a50840 0x557482e1f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:35.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.253+0000 7f9c09bce740 1 --2- 172.21.15.16:0/1811911927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557482a71520 0x557482e1fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:35.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.253+0000 7f9c09bce740 1 --2- 172.21.15.16:0/1811911927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557482a48660 0x557482e14e30 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:35.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.253+0000 7f9c09bce740 1 -- 172.21.15.16:0/1811911927 >> 172.21.15.16:0/1811911927 conn(0x557482a65910 msgr2=0x557482e187c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:35.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.253+0000 7f9c09bce740 1 -- 172.21.15.16:0/1811911927 shutdown_connections 2024-08-31T06:10:35.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.253+0000 7f9c09bce740 1 -- 172.21.15.16:0/1811911927 wait complete. 2024-08-31T06:10:35.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.254+0000 7f9c09bce740 1 -- start start 2024-08-31T06:10:35.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.254+0000 7f9c09bce740 1 Processor -- start 2024-08-31T06:10:35.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.254+0000 7f9c09bce740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557482a48660 0x557482e27200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:35.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.255+0000 7f9c09bce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557482a50840 0x557482e277e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:35.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.255+0000 7f9c09bce740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557482a71520 0x557482e9dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:35.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.255+0000 7f9c09bce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9bfc1afb30 con 0x557482a50840 2024-08-31T06:10:35.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.255+0000 7f9c09bce740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9bfc1b0fa0 con 0x557482a48660 2024-08-31T06:10:35.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.255+0000 7f9c09bce740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9bfc1b0200 con 0x557482a71520 2024-08-31T06:10:35.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.255+0000 7f9c09bce740 10 client.0 _ll_register_callbacks cb 0x7f9be80c7f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:10:35.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.255+0000 7f9bfb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557482a50840 0x557482e277e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:35.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.255+0000 7f9bfbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557482a48660 0x557482e27200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:35.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.255+0000 7f9bfb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557482a50840 0x557482e277e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49762/0 (socket says 172.21.15.16:49762) 2024-08-31T06:10:35.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.255+0000 7f9bfb400640 1 -- 172.21.15.16:0/3413233337 learned_addr learned my addr 172.21.15.16:0/3413233337 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:35.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.255+0000 7f9c00e00640 1 --2- 172.21.15.16:0/3413233337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557482a71520 0x557482e9dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:35.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.255+0000 7f9bfb400640 1 -- 172.21.15.16:0/3413233337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557482a71520 msgr2=0x557482e9dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:35.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.255+0000 7f9bfb400640 1 --2- 172.21.15.16:0/3413233337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557482a71520 0x557482e9dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:35.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.255+0000 7f9bfb400640 1 -- 172.21.15.16:0/3413233337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557482a48660 msgr2=0x557482e27200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:35.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.255+0000 7f9bfb400640 1 --2- 172.21.15.16:0/3413233337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557482a48660 0x557482e27200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:35.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.255+0000 7f9bfb400640 1 -- 172.21.15.16:0/3413233337 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557482e9e610 con 0x557482a50840 2024-08-31T06:10:35.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.255+0000 7f9c00e00640 1 --2- 172.21.15.16:0/3413233337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557482a71520 0x557482e9dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:10:35.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.256+0000 7f9bfb400640 1 --2- 172.21.15.16:0/3413233337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557482a50840 0x557482e277e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f9be80cbf50 tx=0x7f9be80cc590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51193f2c5ca16226 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:35.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.256+0000 7f9be6000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:10:35.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.256+0000 7f9be6000640 1 -- 172.21.15.16:0/3413233337 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be8049070 con 0x557482a50840 2024-08-31T06:10:35.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.256+0000 7f9be6000640 1 -- 172.21.15.16:0/3413233337 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f9be80d5060 con 0x557482a50840 2024-08-31T06:10:35.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.256+0000 7f9c09bce740 1 -- 172.21.15.16:0/3413233337 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bec034070 con 0x557482a50840 2024-08-31T06:10:35.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.256+0000 7f9be6000640 1 -- 172.21.15.16:0/3413233337 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9be80cb910 con 0x557482a50840 2024-08-31T06:10:35.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.256+0000 7f9c09bce740 10 client.16182 fetch_fsmap learned FSMap version 31 2024-08-31T06:10:35.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.256+0000 7f9c09bce740 1 -- 172.21.15.16:0/3413233337 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557482e9f900 con 0x557482a50840 2024-08-31T06:10:35.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.256+0000 7f9be6000640 1 -- 172.21.15.16:0/3413233337 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9be807a020 con 0x557482a50840 2024-08-31T06:10:35.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.257+0000 7f9be6000640 1 -- 172.21.15.16:0/3413233337 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f9be80c7930 con 0x557482a50840 2024-08-31T06:10:35.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.257+0000 7f9c09bce740 10 client.16182 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:10:35.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.257+0000 7f9c09bce740 10 client.16182 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:10:35.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.257+0000 7f9c09bce740 20 client.16182 populate_metadata read hostname 'smithi016' 2024-08-31T06:10:35.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.257+0000 7f9c09bce740 1 --2- 172.21.15.16:0/3413233337 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557482a05250 0x557482ea1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:35.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.257+0000 7f9c09bce740 4 client.16182 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "53a757a7", "release": true, "await": true}] 2024-08-31T06:10:35.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.257+0000 7f9c09bce740 1 -- 172.21.15.16:0/3413233337 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "quiesce db", "set_id": "53a757a7", "release": true, "await": true}) -- 0x557482d8b480 con 0x557482a05250 2024-08-31T06:10:35.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.257+0000 7f9bfbe00640 1 --2- 172.21.15.16:0/3413233337 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557482a05250 0x557482ea1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:35.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.258+0000 7f9bfbe00640 1 --2- 172.21.15.16:0/3413233337 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557482a05250 0x557482ea1fb0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b6a346745868761 server_cookie=95d992eda7166667 in_seq=0 out_seq=0 2024-08-31T06:10:35.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.258+0000 7f9be6000640 10 client.16182 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:35.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.284+0000 7f9be6000640 1 -- 172.21.15.16:0/3413233337 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557482d8b480 con 0x557482a05250 2024-08-31T06:10:35.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.284+0000 7f9be6000640 10 client.16182 handle_command_reply: tid=0 2024-08-31T06:10:35.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.285+0000 7f9c09bce740 1 client.16182 shutdown 2024-08-31T06:10:35.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.286+0000 7f9c09bce740 1 -- 172.21.15.16:0/3413233337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557482a50840 msgr2=0x557482e277e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:35.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.286+0000 7f9c09bce740 1 --2- 172.21.15.16:0/3413233337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557482a50840 0x557482e277e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f9be80cbf50 tx=0x7f9be80cc590 comp rx=0 tx=0).stop 2024-08-31T06:10:35.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.287+0000 7f9c09bce740 1 -- 172.21.15.16:0/3413233337 shutdown_connections 2024-08-31T06:10:35.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.287+0000 7f9c09bce740 1 --2- 172.21.15.16:0/3413233337 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557482a05250 0x557482ea1fb0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:35.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.287+0000 7f9c09bce740 1 --2- 172.21.15.16:0/3413233337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557482a71520 0x557482e9dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:35.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.287+0000 7f9c09bce740 1 --2- 172.21.15.16:0/3413233337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557482a50840 0x557482e277e0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:35.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.287+0000 7f9c09bce740 1 --2- 172.21.15.16:0/3413233337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557482a48660 0x557482e27200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:35.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.287+0000 7f9c09bce740 1 -- 172.21.15.16:0/3413233337 >> 172.21.15.16:0/3413233337 conn(0x557482a65910 msgr2=0x557482e237c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:35.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.287+0000 7f9be6000640 0 client.16182 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:35.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.287+0000 7f9c09bce740 1 -- 172.21.15.16:0/3413233337 shutdown_connections 2024-08-31T06:10:35.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.292+0000 7f9c09bce740 1 -- 172.21.15.16:0/3413233337 wait complete. 2024-08-31T06:10:35.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.292+0000 7f9c09bce740 20 client.16182 trim_cache size 0 max 16384 2024-08-31T06:10:35.295 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:10:35.295 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 31, 2024-08-31T06:10:35.296 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24737, 2024-08-31T06:10:35.296 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 12, 2024-08-31T06:10:35.296 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-08-31T06:10:35.296 INFO:teuthology.orchestra.run.smithi016.stdout: "53a757a7": { 2024-08-31T06:10:35.296 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 12, 2024-08-31T06:10:35.296 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-08-31T06:10:35.296 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:10:35.296 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-08-31T06:10:35.296 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:10:35.296 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:10:35.296 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-08-31T06:10:35.296 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 141.8, 2024-08-31T06:10:35.296 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-08-31T06:10:35.296 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-08-31T06:10:35.297 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-08-31T06:10:35.297 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:10:35.297 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-08-31T06:10:35.297 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:10:35.297 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:10:35.297 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:10:35.297 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:10:35.297 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:10:35.298 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:10:35.298 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:10:35.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.294+0000 7f9c02e00640 1 -- 172.21.15.16:0/1587145193 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9be00b4ca0 msgr2=0x7f9be00b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:35.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.294+0000 7f9c02e00640 1 --2- 172.21.15.16:0/1587145193 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9be00b4ca0 0x7f9be00b71b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9bec009ab0 tx=0x7f9bec005f90 comp rx=0 tx=0).stop 2024-08-31T06:10:35.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.294+0000 7f9c02e00640 1 -- 172.21.15.16:0/1587145193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bfc109240 msgr2=0x7f9bfc19c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:35.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.294+0000 7f9c02e00640 1 --2- 172.21.15.16:0/1587145193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bfc109240 0x7f9bfc19c9c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f9be802a930 tx=0x7f9be8031a20 comp rx=0 tx=0).stop 2024-08-31T06:10:35.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.294+0000 7f9c02e00640 1 -- 172.21.15.16:0/1587145193 shutdown_connections 2024-08-31T06:10:35.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.294+0000 7f9c02e00640 1 --2- 172.21.15.16:0/1587145193 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9be00b4ca0 0x7f9be00b71b0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:35.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.294+0000 7f9c02e00640 1 --2- 172.21.15.16:0/1587145193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9bfc109b90 0x7f9bfc1a1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:35.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.294+0000 7f9c02e00640 1 --2- 172.21.15.16:0/1587145193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bfc109240 0x7f9bfc19c9c0 secure :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f9be802a930 tx=0x7f9be8031a20 comp rx=0 tx=0).stop 2024-08-31T06:10:35.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.294+0000 7f9c02e00640 1 --2- 172.21.15.16:0/1587145193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9bfc103900 0x7f9bfc19c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:35.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.294+0000 7f9c02e00640 1 -- 172.21.15.16:0/1587145193 >> 172.21.15.16:0/1587145193 conn(0x7f9bfc07b5f0 msgr2=0x7f9bfc107050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:35.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.294+0000 7f9c02e00640 1 -- 172.21.15.16:0/1587145193 shutdown_connections 2024-08-31T06:10:35.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:35.294+0000 7f9c02e00640 1 -- 172.21.15.16:0/1587145193 wait complete. 2024-08-31T06:10:36.197 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '53a757a7', total seconds elapsed: 26.1 2024-08-31T06:10:36.197 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.8 2024-08-31T06:10:36.197 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:10:36.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 190.0 s, 582.1 tps, lat 711.874 ms stddev 3536.179 2024-08-31T06:10:36.781 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:10:36.871 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:36 smithi155 ceph-mon[21742]: pgmap v465: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 862 KiB/s wr, 95 op/s 2024-08-31T06:10:36.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:36 smithi016 ceph-mon[17194]: pgmap v465: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 862 KiB/s wr, 95 op/s 2024-08-31T06:10:36.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:36 smithi184 ceph-mon[22331]: pgmap v465: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 862 KiB/s wr, 95 op/s 2024-08-31T06:10:37.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:37.586+0000 7feae5e00640 1 -- 172.21.15.16:0/2407942557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feae010bde0 msgr2=0x7feae0112610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:37.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:37.586+0000 7feae5e00640 1 --2- 172.21.15.16:0/2407942557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feae010bde0 0x7feae0112610 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fead8009ca0 tx=0x7fead802a8c0 comp rx=0 tx=0).stop 2024-08-31T06:10:37.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:37.588+0000 7feae5e00640 1 -- 172.21.15.16:0/2407942557 shutdown_connections 2024-08-31T06:10:37.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:37.588+0000 7feae5e00640 1 --2- 172.21.15.16:0/2407942557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feae010bde0 0x7feae0112610 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:37.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:37.588+0000 7feae5e00640 1 --2- 172.21.15.16:0/2407942557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feae010b490 0x7feae010b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:37.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:37.588+0000 7feae5e00640 1 --2- 172.21.15.16:0/2407942557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feae0105b50 0x7feae0105f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:37.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:37.588+0000 7feae5e00640 1 -- 172.21.15.16:0/2407942557 >> 172.21.15.16:0/2407942557 conn(0x7feae0101580 msgr2=0x7feae01039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:37.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:37.588+0000 7feae5e00640 1 -- 172.21.15.16:0/2407942557 shutdown_connections 2024-08-31T06:10:37.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:37.589+0000 7feae5e00640 1 -- 172.21.15.16:0/2407942557 wait complete. 2024-08-31T06:10:37.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:37.592+0000 7feae5e00640 1 Processor -- start 2024-08-31T06:10:37.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:37.593+0000 7feae5e00640 1 -- start start 2024-08-31T06:10:37.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:37.594+0000 7feae5e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feae0105b50 0x7feae019c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:37.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:37.594+0000 7feae5e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feae010b490 0x7feae019ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:37.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:37.594+0000 7feae5e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feae010bde0 0x7feae01a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:37.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:37.595+0000 7feade000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feae010b490 0x7feae019ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:37.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:37.595+0000 7feade000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feae010b490 0x7feae019ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:35728/0 (socket says 172.21.15.16:35728) 2024-08-31T06:10:37.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:37.597+0000 7feadea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feae0105b50 0x7feae019c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:37.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:37.598+0000 7feade000640 1 -- 172.21.15.16:0/638787532 learned_addr learned my addr 172.21.15.16:0/638787532 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:37.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:37.598+0000 7feae5e00640 1 -- 172.21.15.16:0/638787532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feae0115aa0 con 0x7feae010bde0 2024-08-31T06:10:37.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:37.598+0000 7feae5e00640 1 -- 172.21.15.16:0/638787532 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7feae0115920 con 0x7feae0105b50 2024-08-31T06:10:37.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:37.598+0000 7feae5e00640 1 -- 172.21.15.16:0/638787532 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7feae0115c20 con 0x7feae010b490 2024-08-31T06:10:37.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:37.598+0000 7feadf400640 1 --2- 172.21.15.16:0/638787532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feae010bde0 0x7feae01a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:37.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:37.598+0000 7feadea00640 1 -- 172.21.15.16:0/638787532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feae010b490 msgr2=0x7feae019ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:37.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:37.598+0000 7feadea00640 1 --2- 172.21.15.16:0/638787532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feae010b490 0x7feae019ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:37.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:37.598+0000 7feadea00640 1 -- 172.21.15.16:0/638787532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feae010bde0 msgr2=0x7feae01a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:37.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:37.600+0000 7feadea00640 1 --2- 172.21.15.16:0/638787532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feae010bde0 0x7feae01a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:37.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:37.600+0000 7feadea00640 1 -- 172.21.15.16:0/638787532 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feae01a2420 con 0x7feae0105b50 2024-08-31T06:10:37.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:37.600+0000 7feade000640 1 --2- 172.21.15.16:0/638787532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feae010b490 0x7feae019ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:10:37.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:37.600+0000 7feadf400640 1 --2- 172.21.15.16:0/638787532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feae010bde0 0x7feae01a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:10:37.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:37.601+0000 7feadea00640 1 --2- 172.21.15.16:0/638787532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feae0105b50 0x7feae019c4b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7feacc00c970 tx=0x7feacc00ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:37.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:37.603+0000 7feae4a00640 1 -- 172.21.15.16:0/638787532 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc007bb0 con 0x7feae0105b50 2024-08-31T06:10:37.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:37.604+0000 7feae4a00640 1 -- 172.21.15.16:0/638787532 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feacc007d50 con 0x7feae0105b50 2024-08-31T06:10:37.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:37.604+0000 7feae4a00640 1 -- 172.21.15.16:0/638787532 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc004df0 con 0x7feae0105b50 2024-08-31T06:10:37.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:37.604+0000 7feae5e00640 1 -- 172.21.15.16:0/638787532 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feae01a2710 con 0x7feae0105b50 2024-08-31T06:10:37.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:37.604+0000 7feae5e00640 1 -- 172.21.15.16:0/638787532 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feae01a2b10 con 0x7feae0105b50 2024-08-31T06:10:37.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:37.606+0000 7feae5e00640 1 -- 172.21.15.16:0/638787532 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feae0105f60 con 0x7feae0105b50 2024-08-31T06:10:37.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:37.621+0000 7feae4a00640 1 -- 172.21.15.16:0/638787532 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7feacc046880 con 0x7feae0105b50 2024-08-31T06:10:37.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:37.624+0000 7feae4a00640 1 --2- 172.21.15.16:0/638787532 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7feac40b4dd0 0x7feac40b72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:37.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:37.624+0000 7feae4a00640 1 -- 172.21.15.16:0/638787532 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7feacc0e51e0 con 0x7feae0105b50 2024-08-31T06:10:37.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:37.624+0000 7feade000640 1 --2- 172.21.15.16:0/638787532 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7feac40b4dd0 0x7feac40b72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:37.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:37.624+0000 7feae4a00640 1 -- 172.21.15.16:0/638787532 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7feacc014030 con 0x7feae0105b50 2024-08-31T06:10:37.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:37.625+0000 7feade000640 1 --2- 172.21.15.16:0/638787532 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7feac40b4dd0 0x7feac40b72e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7feae0104dd0 tx=0x7fead0005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:38.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:38.023+0000 7feae5e00640 1 -- 172.21.15.16:0/638787532 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feae019d5e0 con 0x7feae0105b50 2024-08-31T06:10:38.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:38.024+0000 7feae4a00640 1 -- 172.21.15.16:0/638787532 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11923 (secure 0 0 0) 0x7feacc0ac050 con 0x7feae0105b50 2024-08-31T06:10:38.031 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:10:38.032 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 31 2024-08-31T06:10:38.032 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":31,"btime":"2024-08-31T06:08:47:452652+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:08:47.004817+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24737,24724,24712]},"id":1}]} 2024-08-31T06:10:38.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:38.030+0000 7feae5e00640 1 -- 172.21.15.16:0/638787532 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7feac40b4dd0 msgr2=0x7feac40b72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:38.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:38.030+0000 7feae5e00640 1 --2- 172.21.15.16:0/638787532 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7feac40b4dd0 0x7feac40b72e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7feae0104dd0 tx=0x7fead0005f90 comp rx=0 tx=0).stop 2024-08-31T06:10:38.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:38.030+0000 7feae5e00640 1 -- 172.21.15.16:0/638787532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feae0105b50 msgr2=0x7feae019c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:38.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:38.030+0000 7feae5e00640 1 --2- 172.21.15.16:0/638787532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feae0105b50 0x7feae019c4b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7feacc00c970 tx=0x7feacc00ce30 comp rx=0 tx=0).stop 2024-08-31T06:10:38.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:38.031+0000 7feae5e00640 1 -- 172.21.15.16:0/638787532 shutdown_connections 2024-08-31T06:10:38.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:38.031+0000 7feae5e00640 1 --2- 172.21.15.16:0/638787532 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7feac40b4dd0 0x7feac40b72e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:38.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:38.031+0000 7feae5e00640 1 --2- 172.21.15.16:0/638787532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feae010b490 0x7feae019ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:38.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:38.031+0000 7feae5e00640 1 --2- 172.21.15.16:0/638787532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feae010bde0 0x7feae01a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:38.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:38.031+0000 7feae5e00640 1 --2- 172.21.15.16:0/638787532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feae0105b50 0x7feae019c4b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:38.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:38.031+0000 7feae5e00640 1 -- 172.21.15.16:0/638787532 >> 172.21.15.16:0/638787532 conn(0x7feae0101580 msgr2=0x7feae010aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:38.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:38.032+0000 7feae5e00640 1 -- 172.21.15.16:0/638787532 shutdown_connections 2024-08-31T06:10:38.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:38.032+0000 7feae5e00640 1 -- 172.21.15.16:0/638787532 wait complete. 2024-08-31T06:10:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.332+0000 7fc7db800640 1 -- 172.21.15.16:0/2987861048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7d4110100 msgr2=0x7fc7d4116930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.332+0000 7fc7db800640 1 --2- 172.21.15.16:0/2987861048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7d4110100 0x7fc7d4116930 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc009ca0 tx=0x7fc7cc02a8c0 comp rx=0 tx=0).stop 2024-08-31T06:10:38.335 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.332+0000 7fc7db800640 1 -- 172.21.15.16:0/2987861048 shutdown_connections 2024-08-31T06:10:38.335 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.332+0000 7fc7db800640 1 --2- 172.21.15.16:0/2987861048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7d4110100 0x7fc7d4116930 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:38.335 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.332+0000 7fc7db800640 1 --2- 172.21.15.16:0/2987861048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7d410f7b0 0x7fc7d410fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:38.335 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.332+0000 7fc7db800640 1 --2- 172.21.15.16:0/2987861048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7d4109e80 0x7fc7d410a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:38.335 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.332+0000 7fc7db800640 1 -- 172.21.15.16:0/2987861048 >> 172.21.15.16:0/2987861048 conn(0x7fc7d40793d0 msgr2=0x7fc7d4107f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:38.335 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.332+0000 7fc7db800640 1 -- 172.21.15.16:0/2987861048 shutdown_connections 2024-08-31T06:10:38.335 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.332+0000 7fc7db800640 1 -- 172.21.15.16:0/2987861048 wait complete. 2024-08-31T06:10:38.336 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.334+0000 7fc7db800640 1 Processor -- start 2024-08-31T06:10:38.336 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.334+0000 7fc7db800640 1 -- start start 2024-08-31T06:10:38.336 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.334+0000 7fc7db800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7d4109e80 0x7fc7d4074bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:38.337 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.334+0000 7fc7db800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7d410f7b0 0x7fc7d406deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:38.337 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.334+0000 7fc7db800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7d4110100 0x7fc7d406e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:38.337 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.335+0000 7fc7db800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d4119e10 con 0x7fc7d410f7b0 2024-08-31T06:10:38.337 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.335+0000 7fc7db800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d4119c90 con 0x7fc7d4110100 2024-08-31T06:10:38.337 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.335+0000 7fc7d8e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7d4109e80 0x7fc7d4074bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.335+0000 7fc7db800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d4119f90 con 0x7fc7d4109e80 2024-08-31T06:10:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.335+0000 7fc7d8e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7d4109e80 0x7fc7d4074bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:35746/0 (socket says 172.21.15.16:35746) 2024-08-31T06:10:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.335+0000 7fc7d8e00640 1 -- 172.21.15.16:0/3347481168 learned_addr learned my addr 172.21.15.16:0/3347481168 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.335+0000 7fc7d9800640 1 --2- 172.21.15.16:0/3347481168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7d4110100 0x7fc7d406e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.335+0000 7fc7d3e00640 1 --2- 172.21.15.16:0/3347481168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7d410f7b0 0x7fc7d406deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.336+0000 7fc7d9800640 1 -- 172.21.15.16:0/3347481168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7d4109e80 msgr2=0x7fc7d4074bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.336+0000 7fc7d9800640 1 --2- 172.21.15.16:0/3347481168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7d4109e80 0x7fc7d4074bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.336+0000 7fc7d9800640 1 -- 172.21.15.16:0/3347481168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7d410f7b0 msgr2=0x7fc7d406deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.336+0000 7fc7d9800640 1 --2- 172.21.15.16:0/3347481168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7d410f7b0 0x7fc7d406deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.336+0000 7fc7d9800640 1 -- 172.21.15.16:0/3347481168 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7d406ec50 con 0x7fc7d4110100 2024-08-31T06:10:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.336+0000 7fc7d3e00640 1 --2- 172.21.15.16:0/3347481168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7d410f7b0 0x7fc7d406deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:10:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.336+0000 7fc7d9800640 1 --2- 172.21.15.16:0/3347481168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7d4110100 0x7fc7d406e430 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc005e30 tx=0x7fc7cc0319f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.337+0000 7fc7d1600640 1 -- 172.21.15.16:0/3347481168 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc003bf0 con 0x7fc7d4110100 2024-08-31T06:10:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.337+0000 7fc7d1600640 1 -- 172.21.15.16:0/3347481168 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7cc003d90 con 0x7fc7d4110100 2024-08-31T06:10:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.337+0000 7fc7d1600640 1 -- 172.21.15.16:0/3347481168 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc02fb40 con 0x7fc7d4110100 2024-08-31T06:10:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.337+0000 7fc7db800640 1 -- 172.21.15.16:0/3347481168 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7d40738b0 con 0x7fc7d4110100 2024-08-31T06:10:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.337+0000 7fc7db800640 1 -- 172.21.15.16:0/3347481168 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7d4073db0 con 0x7fc7d4110100 2024-08-31T06:10:38.344 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.342+0000 7fc7e2a27740 1 Processor -- start 2024-08-31T06:10:38.345 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.343+0000 7fc7d1600640 1 -- 172.21.15.16:0/3347481168 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fc7cc004660 con 0x7fc7d4110100 2024-08-31T06:10:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.344+0000 7fc7e2a27740 1 -- start start 2024-08-31T06:10:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.344+0000 7fc7e2a27740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55768e93b1b0 0x55768ecde920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.344+0000 7fc7e2a27740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55768e9122f0 0x55768ecdee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.344+0000 7fc7e2a27740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55768e91a4d0 0x55768ecdf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.344+0000 7fc7e2a27740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55768ec53f70 con 0x55768e9122f0 2024-08-31T06:10:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.344+0000 7fc7e2a27740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55768ec43500 con 0x55768e93b1b0 2024-08-31T06:10:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.344+0000 7fc7e2a27740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55768ec474b0 con 0x55768e91a4d0 2024-08-31T06:10:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.345+0000 7fc7d8e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55768e9122f0 0x55768ecdee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.345+0000 7fc7d3e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55768e91a4d0 0x55768ecdf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.345+0000 7fc7d3e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55768e91a4d0 0x55768ecdf3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:35750/0 (socket says 172.21.15.16:35750) 2024-08-31T06:10:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.345+0000 7fc7d3e00640 1 -- 172.21.15.16:0/77418431 learned_addr learned my addr 172.21.15.16:0/77418431 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.345+0000 7fc7d3e00640 1 --2- 172.21.15.16:0/77418431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55768e93b1b0 0x55768ecde920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.346+0000 7fc7d3e00640 1 -- 172.21.15.16:0/77418431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55768e93b1b0 msgr2=0x55768ecde920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.346+0000 7fc7d3e00640 1 --2- 172.21.15.16:0/77418431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55768e93b1b0 0x55768ecde920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.346+0000 7fc7d3e00640 1 -- 172.21.15.16:0/77418431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55768e9122f0 msgr2=0x55768ecdee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.346+0000 7fc7d1600640 1 --2- 172.21.15.16:0/3347481168 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc7bc0b4d20 0x7fc7bc0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.346+0000 7fc7d3e00640 1 --2- 172.21.15.16:0/77418431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55768e9122f0 0x55768ecdee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.346+0000 7fc7d3e00640 1 -- 172.21.15.16:0/77418431 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55768ebfdc10 con 0x55768e91a4d0 2024-08-31T06:10:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.346+0000 7fc7d1600640 1 -- 172.21.15.16:0/3347481168 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fc7cc106960 con 0x7fc7d4110100 2024-08-31T06:10:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.346+0000 7fc7d8e00640 1 --2- 172.21.15.16:0/77418431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55768e9122f0 0x55768ecdee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:10:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.347+0000 7fc7d3e00640 1 --2- 172.21.15.16:0/3347481168 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc7bc0b4d20 0x7fc7bc0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:38.350 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.347+0000 7fc7d3e00640 1 --2- 172.21.15.16:0/77418431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55768e91a4d0 0x55768ecdf3a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc7c00135b0 tx=0x7fc7c000c6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f16a89224cfc153e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:38.350 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.348+0000 7fc7c6000640 1 -- 172.21.15.16:0/77418431 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c0019040 con 0x55768e91a4d0 2024-08-31T06:10:38.350 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.348+0000 7fc7c6000640 1 -- 172.21.15.16:0/77418431 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7c0021430 con 0x55768e91a4d0 2024-08-31T06:10:38.350 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.348+0000 7fc7c6000640 1 -- 172.21.15.16:0/77418431 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c000f6a0 con 0x55768e91a4d0 2024-08-31T06:10:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.349+0000 7fc7e2a27740 1 -- 172.21.15.16:0/77418431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55768e91a4d0 msgr2=0x55768ecdf3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.349+0000 7fc7e2a27740 1 --2- 172.21.15.16:0/77418431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55768e91a4d0 0x55768ecdf3a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc7c00135b0 tx=0x7fc7c000c6b0 comp rx=0 tx=0).stop 2024-08-31T06:10:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.349+0000 7fc7d3e00640 1 --2- 172.21.15.16:0/3347481168 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc7bc0b4d20 0x7fc7bc0b7230 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0020000 tx=0x7fc7c000f980 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.349+0000 7fc7e2a27740 1 -- 172.21.15.16:0/77418431 shutdown_connections 2024-08-31T06:10:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.349+0000 7fc7e2a27740 1 --2- 172.21.15.16:0/77418431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55768e91a4d0 0x55768ecdf3a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.349+0000 7fc7e2a27740 1 --2- 172.21.15.16:0/77418431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55768e9122f0 0x55768ecdee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.349+0000 7fc7e2a27740 1 --2- 172.21.15.16:0/77418431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55768e93b1b0 0x55768ecde920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.349+0000 7fc7e2a27740 1 -- 172.21.15.16:0/77418431 >> 172.21.15.16:0/77418431 conn(0x55768e92f300 msgr2=0x55768ecdc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:38.353 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.350+0000 7fc7e2a27740 1 -- 172.21.15.16:0/77418431 shutdown_connections 2024-08-31T06:10:38.353 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.350+0000 7fc7e2a27740 1 -- 172.21.15.16:0/77418431 wait complete. 2024-08-31T06:10:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.352+0000 7fc7e2a27740 1 -- start start 2024-08-31T06:10:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.352+0000 7fc7e2a27740 1 Processor -- start 2024-08-31T06:10:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.353+0000 7fc7e2a27740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55768e9122f0 0x55768ece9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.353+0000 7fc7e2a27740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55768e91a4d0 0x55768ecea4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.353+0000 7fc7e2a27740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55768e93b1b0 0x55768ecf6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.353+0000 7fc7e2a27740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55768eb7cab0 con 0x55768e91a4d0 2024-08-31T06:10:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.353+0000 7fc7e2a27740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55768ebea320 con 0x55768e9122f0 2024-08-31T06:10:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.353+0000 7fc7e2a27740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55768eced0a0 con 0x55768e93b1b0 2024-08-31T06:10:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.353+0000 7fc7e2a27740 10 client.0 _ll_register_callbacks cb 0x55768ebf06a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:10:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.353+0000 7fc7d8e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55768e9122f0 0x55768ece9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.353+0000 7fc7d3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55768e91a4d0 0x55768ecea4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.353+0000 7fc7d8e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55768e9122f0 0x55768ece9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:34366/0 (socket says 172.21.15.16:34366) 2024-08-31T06:10:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.353+0000 7fc7d8e00640 1 -- 172.21.15.16:0/1453244624 learned_addr learned my addr 172.21.15.16:0/1453244624 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.354+0000 7fc7d9800640 1 --2- 172.21.15.16:0/1453244624 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55768e93b1b0 0x55768ecf6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.354+0000 7fc7d8e00640 1 -- 172.21.15.16:0/1453244624 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55768e93b1b0 msgr2=0x55768ecf6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.354+0000 7fc7d8e00640 1 --2- 172.21.15.16:0/1453244624 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55768e93b1b0 0x55768ecf6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.354+0000 7fc7d8e00640 1 -- 172.21.15.16:0/1453244624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55768e91a4d0 msgr2=0x55768ecea4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.354+0000 7fc7d8e00640 1 --2- 172.21.15.16:0/1453244624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55768e91a4d0 0x55768ecea4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.354+0000 7fc7d8e00640 1 -- 172.21.15.16:0/1453244624 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55768e9a6790 con 0x55768e9122f0 2024-08-31T06:10:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.354+0000 7fc7d9800640 1 --2- 172.21.15.16:0/1453244624 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55768e93b1b0 0x55768ecf6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:10:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.354+0000 7fc7d3e00640 1 --2- 172.21.15.16:0/1453244624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55768e91a4d0 0x55768ecea4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:10:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.356+0000 7fc7d8e00640 1 --2- 172.21.15.16:0/1453244624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55768e9122f0 0x55768ece9f70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc7c80114e0 tx=0x7fc7c80119a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7146457fb3e211ca server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.356+0000 7fc7baa00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:10:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.356+0000 7fc7baa00640 1 -- 172.21.15.16:0/1453244624 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c8010880 con 0x55768e9122f0 2024-08-31T06:10:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.356+0000 7fc7baa00640 1 -- 172.21.15.16:0/1453244624 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fc7c8010a20 con 0x55768e9122f0 2024-08-31T06:10:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.356+0000 7fc7e2a27740 1 -- 172.21.15.16:0/1453244624 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7c0021430 con 0x55768e9122f0 2024-08-31T06:10:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.357+0000 7fc7baa00640 1 -- 172.21.15.16:0/1453244624 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc7c8010c20 con 0x55768e9122f0 2024-08-31T06:10:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.357+0000 7fc7e2a27740 10 client.26318 fetch_fsmap learned FSMap version 31 2024-08-31T06:10:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.357+0000 7fc7e2a27740 1 -- 172.21.15.16:0/1453244624 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55768ea1f620 con 0x55768e9122f0 2024-08-31T06:10:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.357+0000 7fc7baa00640 1 -- 172.21.15.16:0/1453244624 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7c0021430 con 0x55768e9122f0 2024-08-31T06:10:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.357+0000 7fc7baa00640 1 -- 172.21.15.16:0/1453244624 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fc7c800f3c0 con 0x55768e9122f0 2024-08-31T06:10:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.359+0000 7fc7e2a27740 10 client.26318 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:10:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.359+0000 7fc7e2a27740 10 client.26318 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:10:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.359+0000 7fc7e2a27740 20 client.26318 populate_metadata read hostname 'smithi016' 2024-08-31T06:10:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.359+0000 7fc7e2a27740 1 --2- 172.21.15.16:0/1453244624 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55768e8f9d80 0x55768ecfc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.359+0000 7fc7e2a27740 4 client.26318 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:10:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.359+0000 7fc7e2a27740 1 -- 172.21.15.16:0/1453244624 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55768ec55ec0 con 0x55768e8f9d80 2024-08-31T06:10:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.359+0000 7fc7d3e00640 1 --2- 172.21.15.16:0/1453244624 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55768e8f9d80 0x55768ecfc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.360+0000 7fc7d3e00640 1 --2- 172.21.15.16:0/1453244624 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55768e8f9d80 0x55768ecfc180 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f97776103df6644c server_cookie=4fa8afe658c72470 in_seq=0 out_seq=0 2024-08-31T06:10:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.360+0000 7fc7baa00640 10 client.26318 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.361+0000 7fc7baa00640 1 -- 172.21.15.16:0/1453244624 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55768ec55ec0 con 0x55768e8f9d80 2024-08-31T06:10:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.361+0000 7fc7baa00640 10 client.26318 handle_command_reply: tid=0 2024-08-31T06:10:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.361+0000 7fc7e2a27740 1 client.26318 shutdown 2024-08-31T06:10:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.361+0000 7fc7e2a27740 1 -- 172.21.15.16:0/1453244624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55768e9122f0 msgr2=0x55768ece9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.361+0000 7fc7e2a27740 1 --2- 172.21.15.16:0/1453244624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55768e9122f0 0x55768ece9f70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc7c80114e0 tx=0x7fc7c80119a0 comp rx=0 tx=0).stop 2024-08-31T06:10:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.362+0000 7fc7e2a27740 1 -- 172.21.15.16:0/1453244624 shutdown_connections 2024-08-31T06:10:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.362+0000 7fc7e2a27740 1 --2- 172.21.15.16:0/1453244624 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55768e8f9d80 0x55768ecfc180 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.362+0000 7fc7e2a27740 1 --2- 172.21.15.16:0/1453244624 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55768e93b1b0 0x55768ecf6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.362+0000 7fc7e2a27740 1 --2- 172.21.15.16:0/1453244624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55768e91a4d0 0x55768ecea4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.362+0000 7fc7e2a27740 1 --2- 172.21.15.16:0/1453244624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55768e9122f0 0x55768ece9f70 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.362+0000 7fc7baa00640 0 client.26318 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.362+0000 7fc7e2a27740 1 -- 172.21.15.16:0/1453244624 >> 172.21.15.16:0/1453244624 conn(0x55768e92f300 msgr2=0x55768ecdf950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.362+0000 7fc7e2a27740 1 -- 172.21.15.16:0/1453244624 shutdown_connections 2024-08-31T06:10:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.362+0000 7fc7e2a27740 1 -- 172.21.15.16:0/1453244624 wait complete. 2024-08-31T06:10:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.362+0000 7fc7e2a27740 20 client.26318 trim_cache size 0 max 16384 2024-08-31T06:10:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.388+0000 7fc7e2a27740 1 Processor -- start 2024-08-31T06:10:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.388+0000 7fc7e2a27740 1 -- start start 2024-08-31T06:10:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.389+0000 7fc7e2a27740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55768e9122f0 0x55768ecf6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.389+0000 7fc7e2a27740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55768e91a4d0 0x55768ed01940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.389+0000 7fc7e2a27740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55768e93b1b0 0x55768ed01e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.389+0000 7fc7e2a27740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d41c0b90 con 0x55768e93b1b0 2024-08-31T06:10:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.389+0000 7fc7e2a27740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d41be3a0 con 0x55768e91a4d0 2024-08-31T06:10:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.389+0000 7fc7e2a27740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d41bc1f0 con 0x55768e9122f0 2024-08-31T06:10:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.389+0000 7fc7d8e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55768e9122f0 0x55768ecf6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.389+0000 7fc7d8e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55768e9122f0 0x55768ecf6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:35768/0 (socket says 172.21.15.16:35768) 2024-08-31T06:10:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.389+0000 7fc7d8e00640 1 -- 172.21.15.16:0/1743945680 learned_addr learned my addr 172.21.15.16:0/1743945680 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.390+0000 7fc7d3e00640 1 --2- 172.21.15.16:0/1743945680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55768e91a4d0 0x55768ed01940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.390+0000 7fc7d9800640 1 --2- 172.21.15.16:0/1743945680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55768e93b1b0 0x55768ed01e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.391+0000 7fc7d8e00640 1 -- 172.21.15.16:0/1743945680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55768e91a4d0 msgr2=0x55768ed01940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.391+0000 7fc7d8e00640 1 --2- 172.21.15.16:0/1743945680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55768e91a4d0 0x55768ed01940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.391+0000 7fc7d8e00640 1 -- 172.21.15.16:0/1743945680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55768e93b1b0 msgr2=0x55768ed01e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:38.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.391+0000 7fc7d8e00640 1 --2- 172.21.15.16:0/1743945680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55768e93b1b0 0x55768ed01e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:38.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.391+0000 7fc7d8e00640 1 -- 172.21.15.16:0/1743945680 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55768ecd6990 con 0x55768e9122f0 2024-08-31T06:10:38.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.392+0000 7fc7d9800640 1 --2- 172.21.15.16:0/1743945680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55768e93b1b0 0x55768ed01e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:10:38.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.392+0000 7fc7d8e00640 1 --2- 172.21.15.16:0/1743945680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55768e9122f0 0x55768ecf6290 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8001eb0 tx=0x7fc7c800b630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47f071aa55ac2987 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:38.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.393+0000 7fc7c5600640 1 -- 172.21.15.16:0/1743945680 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c8010dc0 con 0x55768e9122f0 2024-08-31T06:10:38.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.393+0000 7fc7c5600640 1 -- 172.21.15.16:0/1743945680 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7c80054f0 con 0x55768e9122f0 2024-08-31T06:10:38.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.393+0000 7fc7c5600640 1 -- 172.21.15.16:0/1743945680 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c800fa50 con 0x55768e9122f0 2024-08-31T06:10:38.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.394+0000 7fc7e2a27740 1 -- 172.21.15.16:0/1743945680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55768e9122f0 msgr2=0x55768ecf6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:38.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.394+0000 7fc7e2a27740 1 --2- 172.21.15.16:0/1743945680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55768e9122f0 0x55768ecf6290 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8001eb0 tx=0x7fc7c800b630 comp rx=0 tx=0).stop 2024-08-31T06:10:38.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.394+0000 7fc7e2a27740 1 -- 172.21.15.16:0/1743945680 shutdown_connections 2024-08-31T06:10:38.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.394+0000 7fc7e2a27740 1 --2- 172.21.15.16:0/1743945680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55768e93b1b0 0x55768ed01e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:38.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.394+0000 7fc7e2a27740 1 --2- 172.21.15.16:0/1743945680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55768e91a4d0 0x55768ed01940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:38.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.394+0000 7fc7e2a27740 1 --2- 172.21.15.16:0/1743945680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55768e9122f0 0x55768ecf6290 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:38.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.394+0000 7fc7e2a27740 1 -- 172.21.15.16:0/1743945680 >> 172.21.15.16:0/1743945680 conn(0x55768e92f300 msgr2=0x55768ece0310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:38.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.395+0000 7fc7e2a27740 1 -- 172.21.15.16:0/1743945680 shutdown_connections 2024-08-31T06:10:38.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.395+0000 7fc7e2a27740 1 -- 172.21.15.16:0/1743945680 wait complete. 2024-08-31T06:10:38.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.395+0000 7fc7e2a27740 1 -- start start 2024-08-31T06:10:38.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.396+0000 7fc7e2a27740 1 Processor -- start 2024-08-31T06:10:38.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.396+0000 7fc7e2a27740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55768e9122f0 0x55768ecf21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:38.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.396+0000 7fc7e2a27740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55768e91a4d0 0x55768ecf27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:38.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.396+0000 7fc7e2a27740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55768e93b1b0 0x55768ed5f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:38.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.396+0000 7fc7e2a27740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d41bc860 con 0x55768e91a4d0 2024-08-31T06:10:38.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.396+0000 7fc7e2a27740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d41bdcd0 con 0x55768e93b1b0 2024-08-31T06:10:38.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.396+0000 7fc7e2a27740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d41bcf30 con 0x55768e9122f0 2024-08-31T06:10:38.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.397+0000 7fc7e2a27740 10 client.0 _ll_register_callbacks cb 0x7fc7c800bca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:10:38.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.397+0000 7fc7d8e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55768e9122f0 0x55768ecf21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:38.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.397+0000 7fc7d8e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55768e9122f0 0x55768ecf21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:35770/0 (socket says 172.21.15.16:35770) 2024-08-31T06:10:38.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.397+0000 7fc7d8e00640 1 -- 172.21.15.16:0/199622525 learned_addr learned my addr 172.21.15.16:0/199622525 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:38.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.397+0000 7fc7d9800640 1 --2- 172.21.15.16:0/199622525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55768e93b1b0 0x55768ed5f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:38.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.397+0000 7fc7d3e00640 1 --2- 172.21.15.16:0/199622525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55768e91a4d0 0x55768ecf27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:38.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.398+0000 7fc7d9800640 1 -- 172.21.15.16:0/199622525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55768e9122f0 msgr2=0x55768ecf21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:38.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.398+0000 7fc7d9800640 1 --2- 172.21.15.16:0/199622525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55768e9122f0 0x55768ecf21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:38.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.398+0000 7fc7d9800640 1 -- 172.21.15.16:0/199622525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55768e91a4d0 msgr2=0x55768ecf27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:38.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.398+0000 7fc7d9800640 1 --2- 172.21.15.16:0/199622525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55768e91a4d0 0x55768ecf27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:38.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.399+0000 7fc7d9800640 1 -- 172.21.15.16:0/199622525 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55768ed5f6f0 con 0x55768e93b1b0 2024-08-31T06:10:38.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.399+0000 7fc7d3e00640 1 --2- 172.21.15.16:0/199622525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55768e91a4d0 0x55768ecf27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:10:38.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.399+0000 7fc7d9800640 1 --2- 172.21.15.16:0/199622525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55768e93b1b0 0x55768ed5f050 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc104110 tx=0x7fc7cc0ccb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd2e61feab93515 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:38.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.399+0000 7fc7baa00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:10:38.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.400+0000 7fc7baa00640 1 -- 172.21.15.16:0/199622525 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc0cd140 con 0x55768e93b1b0 2024-08-31T06:10:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.400+0000 7fc7baa00640 1 -- 172.21.15.16:0/199622525 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fc7cc02fce0 con 0x55768e93b1b0 2024-08-31T06:10:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.400+0000 7fc7e2a27740 1 -- 172.21.15.16:0/199622525 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7c80054f0 con 0x55768e93b1b0 2024-08-31T06:10:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.400+0000 7fc7baa00640 1 -- 172.21.15.16:0/199622525 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc7cc0e7e60 con 0x55768e93b1b0 2024-08-31T06:10:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.400+0000 7fc7e2a27740 10 client.26324 fetch_fsmap learned FSMap version 31 2024-08-31T06:10:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.400+0000 7fc7e2a27740 1 -- 172.21.15.16:0/199622525 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55768ed66ab0 con 0x55768e93b1b0 2024-08-31T06:10:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.401+0000 7fc7baa00640 1 -- 172.21.15.16:0/199622525 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7c80054f0 con 0x55768e93b1b0 2024-08-31T06:10:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.401+0000 7fc7baa00640 1 -- 172.21.15.16:0/199622525 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fc7cc0d5f40 con 0x55768e93b1b0 2024-08-31T06:10:38.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.401+0000 7fc7e2a27740 10 client.26324 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:10:38.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.401+0000 7fc7e2a27740 10 client.26324 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:10:38.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.401+0000 7fc7e2a27740 20 client.26324 populate_metadata read hostname 'smithi016' 2024-08-31T06:10:38.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.401+0000 7fc7e2a27740 1 --2- 172.21.15.16:0/199622525 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55768e8cf1c0 0x55768ed692d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:38.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.401+0000 7fc7e2a27740 4 client.26324 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:10:38.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.401+0000 7fc7e2a27740 1 -- 172.21.15.16:0/199622525 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55768ec51960 con 0x55768e8cf1c0 2024-08-31T06:10:38.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.402+0000 7fc7d8e00640 1 --2- 172.21.15.16:0/199622525 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55768e8cf1c0 0x55768ed692d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:38.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.403+0000 7fc7d8e00640 1 --2- 172.21.15.16:0/199622525 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55768e8cf1c0 0x55768ed692d0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e6db97731a08661 server_cookie=4791a4be376828ee in_seq=0 out_seq=0 2024-08-31T06:10:38.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.403+0000 7fc7baa00640 10 client.26324 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.416+0000 7fc7baa00640 1 -- 172.21.15.16:0/199622525 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55768ec51960 con 0x55768e8cf1c0 2024-08-31T06:10:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.416+0000 7fc7baa00640 10 client.26324 handle_command_reply: tid=0 2024-08-31T06:10:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.416+0000 7fc7e2a27740 1 client.26324 shutdown 2024-08-31T06:10:38.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.417+0000 7fc7e2a27740 1 -- 172.21.15.16:0/199622525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55768e93b1b0 msgr2=0x55768ed5f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:38.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.417+0000 7fc7e2a27740 1 --2- 172.21.15.16:0/199622525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55768e93b1b0 0x55768ed5f050 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc104110 tx=0x7fc7cc0ccb50 comp rx=0 tx=0).stop 2024-08-31T06:10:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.418+0000 7fc7e2a27740 1 -- 172.21.15.16:0/199622525 shutdown_connections 2024-08-31T06:10:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.418+0000 7fc7e2a27740 1 --2- 172.21.15.16:0/199622525 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55768e8cf1c0 0x55768ed692d0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.418+0000 7fc7e2a27740 1 --2- 172.21.15.16:0/199622525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55768e93b1b0 0x55768ed5f050 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.418+0000 7fc7e2a27740 1 --2- 172.21.15.16:0/199622525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55768e91a4d0 0x55768ecf27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.418+0000 7fc7e2a27740 1 --2- 172.21.15.16:0/199622525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55768e9122f0 0x55768ecf21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.418+0000 7fc7e2a27740 1 -- 172.21.15.16:0/199622525 >> 172.21.15.16:0/199622525 conn(0x55768e92f300 msgr2=0x55768ece00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.418+0000 7fc7baa00640 0 client.26324 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.418+0000 7fc7e2a27740 1 -- 172.21.15.16:0/199622525 shutdown_connections 2024-08-31T06:10:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.418+0000 7fc7e2a27740 1 -- 172.21.15.16:0/199622525 wait complete. 2024-08-31T06:10:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.418+0000 7fc7e2a27740 20 client.26324 trim_cache size 0 max 16384 2024-08-31T06:10:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:10:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:10:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:10:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:10:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.425+0000 7fc7db800640 1 -- 172.21.15.16:0/3347481168 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc7bc0b4d20 msgr2=0x7fc7bc0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.425+0000 7fc7db800640 1 --2- 172.21.15.16:0/3347481168 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc7bc0b4d20 0x7fc7bc0b7230 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0020000 tx=0x7fc7c000f980 comp rx=0 tx=0).stop 2024-08-31T06:10:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.425+0000 7fc7db800640 1 -- 172.21.15.16:0/3347481168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7d4110100 msgr2=0x7fc7d406e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:38.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.425+0000 7fc7db800640 1 --2- 172.21.15.16:0/3347481168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7d4110100 0x7fc7d406e430 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc005e30 tx=0x7fc7cc0319f0 comp rx=0 tx=0).stop 2024-08-31T06:10:38.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.425+0000 7fc7db800640 1 -- 172.21.15.16:0/3347481168 shutdown_connections 2024-08-31T06:10:38.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.425+0000 7fc7db800640 1 --2- 172.21.15.16:0/3347481168 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc7bc0b4d20 0x7fc7bc0b7230 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:38.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.425+0000 7fc7db800640 1 --2- 172.21.15.16:0/3347481168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7d4110100 0x7fc7d406e430 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:38.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.425+0000 7fc7db800640 1 --2- 172.21.15.16:0/3347481168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7d410f7b0 0x7fc7d406deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:38.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.425+0000 7fc7db800640 1 --2- 172.21.15.16:0/3347481168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7d4109e80 0x7fc7d4074bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:38.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.425+0000 7fc7db800640 1 -- 172.21.15.16:0/3347481168 >> 172.21.15.16:0/3347481168 conn(0x7fc7d40793d0 msgr2=0x7fc7d410b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:38.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.426+0000 7fc7db800640 1 -- 172.21.15.16:0/3347481168 shutdown_connections 2024-08-31T06:10:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:38.426+0000 7fc7db800640 1 -- 172.21.15.16:0/3347481168 wait complete. 2024-08-31T06:10:38.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:38 smithi016 ceph-mon[17194]: pgmap v466: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-08-31T06:10:38.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:38 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/638787532' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:10:38.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:38 smithi155 ceph-mon[21742]: pgmap v466: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-08-31T06:10:38.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:38 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/638787532' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:10:38.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:38 smithi184 ceph-mon[22331]: pgmap v466: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-08-31T06:10:38.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:38 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/638787532' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:10:39.005 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24737 quiesce db / --timeout 90 --expiration 141.8 --await-for 45.0' 2024-08-31T06:10:39.006 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24737 quiesce db / --timeout 90 --expiration 141.8 --await-for 45.0 2024-08-31T06:10:39.644 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:10:39.907 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T06:10:40.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.494+0000 7f6e12400640 1 -- 172.21.15.16:0/3849256299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e0c10bdf0 msgr2=0x7f6e0c112620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:40.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.494+0000 7f6e12400640 1 --2- 172.21.15.16:0/3849256299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e0c10bdf0 0x7f6e0c112620 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6e04009ca0 tx=0x7f6e0402a8c0 comp rx=0 tx=0).stop 2024-08-31T06:10:40.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.494+0000 7f6e12400640 1 -- 172.21.15.16:0/3849256299 shutdown_connections 2024-08-31T06:10:40.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.494+0000 7f6e12400640 1 --2- 172.21.15.16:0/3849256299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e0c10bdf0 0x7f6e0c112620 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:40.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.494+0000 7f6e12400640 1 --2- 172.21.15.16:0/3849256299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e0c10b4a0 0x7f6e0c10b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:40.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.494+0000 7f6e12400640 1 --2- 172.21.15.16:0/3849256299 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e0c105b60 0x7f6e0c105f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:40.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.494+0000 7f6e12400640 1 -- 172.21.15.16:0/3849256299 >> 172.21.15.16:0/3849256299 conn(0x7f6e0c101570 msgr2=0x7f6e0c1039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:40.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.494+0000 7f6e12400640 1 -- 172.21.15.16:0/3849256299 shutdown_connections 2024-08-31T06:10:40.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.495+0000 7f6e12400640 1 -- 172.21.15.16:0/3849256299 wait complete. 2024-08-31T06:10:40.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.497+0000 7f6e12400640 1 Processor -- start 2024-08-31T06:10:40.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.498+0000 7f6e12400640 1 -- start start 2024-08-31T06:10:40.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.499+0000 7f6e12400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e0c105b60 0x7f6e0c074b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:40.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.499+0000 7f6e12400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e0c10b4a0 0x7f6e0c06de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:40.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.499+0000 7f6e12400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e0c10bdf0 0x7f6e0c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:40.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.499+0000 7f6e12400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0c115ab0 con 0x7f6e0c105b60 2024-08-31T06:10:40.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.499+0000 7f6e0b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e0c105b60 0x7f6e0c074b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:40.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.499+0000 7f6e03e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e0c10b4a0 0x7f6e0c06de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:40.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.499+0000 7f6e03e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e0c10b4a0 0x7f6e0c06de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:34412/0 (socket says 172.21.15.16:34412) 2024-08-31T06:10:40.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.499+0000 7f6e03e00640 1 -- 172.21.15.16:0/942061671 learned_addr learned my addr 172.21.15.16:0/942061671 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:40.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.500+0000 7f6e12400640 1 -- 172.21.15.16:0/942061671 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0c115930 con 0x7f6e0c10b4a0 2024-08-31T06:10:40.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.500+0000 7f6e12400640 1 -- 172.21.15.16:0/942061671 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0c115c30 con 0x7f6e0c10bdf0 2024-08-31T06:10:40.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.500+0000 7f6e0be00640 1 --2- 172.21.15.16:0/942061671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e0c10bdf0 0x7f6e0c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:40.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.503+0000 7f6e03e00640 1 -- 172.21.15.16:0/942061671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e0c10bdf0 msgr2=0x7f6e0c06e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:40.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.503+0000 7f6e03e00640 1 --2- 172.21.15.16:0/942061671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e0c10bdf0 0x7f6e0c06e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:40.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.503+0000 7f6e03e00640 1 -- 172.21.15.16:0/942061671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e0c105b60 msgr2=0x7f6e0c074b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:40.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.503+0000 7f6e03e00640 1 --2- 172.21.15.16:0/942061671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e0c105b60 0x7f6e0c074b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:40.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.503+0000 7f6e03e00640 1 -- 172.21.15.16:0/942061671 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e0c06eb40 con 0x7f6e0c10b4a0 2024-08-31T06:10:40.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.503+0000 7f6e03e00640 1 --2- 172.21.15.16:0/942061671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e0c10b4a0 0x7f6e0c06de30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6df400d950 tx=0x7f6df400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:40.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.503+0000 7f6e08c00640 1 -- 172.21.15.16:0/942061671 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df4017070 con 0x7f6e0c10b4a0 2024-08-31T06:10:40.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.504+0000 7f6e08c00640 1 -- 172.21.15.16:0/942061671 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6df4009c30 con 0x7f6e0c10b4a0 2024-08-31T06:10:40.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.504+0000 7f6e08c00640 1 -- 172.21.15.16:0/942061671 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df40047a0 con 0x7f6e0c10b4a0 2024-08-31T06:10:40.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.504+0000 7f6e12400640 1 -- 172.21.15.16:0/942061671 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e0c0737d0 con 0x7f6e0c10b4a0 2024-08-31T06:10:40.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.505+0000 7f6e12400640 1 -- 172.21.15.16:0/942061671 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e0c073cc0 con 0x7f6e0c10b4a0 2024-08-31T06:10:40.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.515+0000 7f6e08c00640 1 -- 172.21.15.16:0/942061671 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f6df4004940 con 0x7f6e0c10b4a0 2024-08-31T06:10:40.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.518+0000 7f6e1902e740 1 Processor -- start 2024-08-31T06:10:40.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.519+0000 7f6e1902e740 1 -- start start 2024-08-31T06:10:40.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.519+0000 7f6e1902e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e0fcf840 0x55a0e1376110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:40.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.519+0000 7f6e1902e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0e0fa6980 0x55a0e1376650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:40.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.519+0000 7f6e1902e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0e0faeb60 0x55a0e1376b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:40.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.519+0000 7f6e1902e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a0e12eb530 con 0x55a0e0fcf840 2024-08-31T06:10:40.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.519+0000 7f6e1902e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a0e12daac0 con 0x55a0e0faeb60 2024-08-31T06:10:40.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.519+0000 7f6e1902e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a0e12dea70 con 0x55a0e0fa6980 2024-08-31T06:10:40.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.520+0000 7f6e0b400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0e0fa6980 0x55a0e1376650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:40.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.520+0000 7f6e03e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0e0faeb60 0x55a0e1376b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:40.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.520+0000 7f6e0b400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0e0fa6980 0x55a0e1376650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:35800/0 (socket says 172.21.15.16:35800) 2024-08-31T06:10:40.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.520+0000 7f6e0b400640 1 -- 172.21.15.16:0/438757864 learned_addr learned my addr 172.21.15.16:0/438757864 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:40.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.520+0000 7f6e03e00640 1 -- 172.21.15.16:0/438757864 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0e0fa6980 msgr2=0x55a0e1376650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:40.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.520+0000 7f6e03e00640 1 --2- 172.21.15.16:0/438757864 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0e0fa6980 0x55a0e1376650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:40.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.520+0000 7f6e03e00640 1 -- 172.21.15.16:0/438757864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e0fcf840 msgr2=0x55a0e1376110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:40.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.520+0000 7f6e03e00640 1 --2- 172.21.15.16:0/438757864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e0fcf840 0x55a0e1376110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:40.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.521+0000 7f6e03e00640 1 -- 172.21.15.16:0/438757864 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0e128d880 con 0x55a0e0faeb60 2024-08-31T06:10:40.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.521+0000 7f6e03e00640 1 --2- 172.21.15.16:0/438757864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0e0faeb60 0x55a0e1376b90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6df40e37a0 tx=0x7f6df4012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6efc64393c7a971 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:40.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.521+0000 7f6e00c00640 1 -- 172.21.15.16:0/438757864 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df40e7610 con 0x55a0e0faeb60 2024-08-31T06:10:40.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.522+0000 7f6e00c00640 1 -- 172.21.15.16:0/438757864 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6df40ec0d0 con 0x55a0e0faeb60 2024-08-31T06:10:40.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.522+0000 7f6e08c00640 1 --2- 172.21.15.16:0/942061671 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6dec0b4da0 0x7f6dec0b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:40.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.523+0000 7f6e08c00640 1 -- 172.21.15.16:0/942061671 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f6df40e6050 con 0x7f6e0c10b4a0 2024-08-31T06:10:40.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.523+0000 7f6e1902e740 1 -- 172.21.15.16:0/438757864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0e0faeb60 msgr2=0x55a0e1376b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:40.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.523+0000 7f6e1902e740 1 --2- 172.21.15.16:0/438757864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0e0faeb60 0x55a0e1376b90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6df40e37a0 tx=0x7f6df4012000 comp rx=0 tx=0).stop 2024-08-31T06:10:40.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.524+0000 7f6e0be00640 1 --2- 172.21.15.16:0/942061671 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6dec0b4da0 0x7f6dec0b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:40.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.524+0000 7f6e1902e740 1 -- 172.21.15.16:0/438757864 shutdown_connections 2024-08-31T06:10:40.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.524+0000 7f6e1902e740 1 --2- 172.21.15.16:0/438757864 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0e0fa6980 0x55a0e1376650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:40.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.524+0000 7f6e1902e740 1 --2- 172.21.15.16:0/438757864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0e0faeb60 0x55a0e1376b90 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:40.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.524+0000 7f6e1902e740 1 --2- 172.21.15.16:0/438757864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e0fcf840 0x55a0e1376110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:40.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.524+0000 7f6e1902e740 1 -- 172.21.15.16:0/438757864 >> 172.21.15.16:0/438757864 conn(0x55a0e0fc3990 msgr2=0x55a0e1374000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:40.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.525+0000 7f6e1902e740 1 -- 172.21.15.16:0/438757864 shutdown_connections 2024-08-31T06:10:40.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.525+0000 7f6e1902e740 1 -- 172.21.15.16:0/438757864 wait complete. 2024-08-31T06:10:40.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.526+0000 7f6e1902e740 1 -- start start 2024-08-31T06:10:40.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.526+0000 7f6e1902e740 1 Processor -- start 2024-08-31T06:10:40.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.526+0000 7f6e0be00640 1 --2- 172.21.15.16:0/942061671 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6dec0b4da0 0x7f6dec0b72b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6e040094e0 tx=0x7f6e040095c0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:40.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.527+0000 7f6e1902e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0e0fa6980 0x55a0e138a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:40.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.527+0000 7f6e1902e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e0faeb60 0x55a0e138a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:40.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.527+0000 7f6e1902e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0e0fcf840 0x55a0e1396440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:40.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.527+0000 7f6e1902e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a0e113d7a0 con 0x55a0e0faeb60 2024-08-31T06:10:40.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.527+0000 7f6e1902e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a0e12617f0 con 0x55a0e0fcf840 2024-08-31T06:10:40.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.527+0000 7f6e1902e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a0e138d060 con 0x55a0e0fa6980 2024-08-31T06:10:40.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.527+0000 7f6e1902e740 10 client.0 _ll_register_callbacks cb 0x55a0e1286e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:10:40.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.528+0000 7f6e0be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0e0fcf840 0x55a0e1396440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:40.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.528+0000 7f6e0b400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0e0fa6980 0x55a0e138a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:40.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.528+0000 7f6e0b400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0e0fa6980 0x55a0e138a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:35804/0 (socket says 172.21.15.16:35804) 2024-08-31T06:10:40.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.528+0000 7f6e0b400640 1 -- 172.21.15.16:0/3612312155 learned_addr learned my addr 172.21.15.16:0/3612312155 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:40.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.529+0000 7f6e03e00640 1 --2- 172.21.15.16:0/3612312155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e0faeb60 0x55a0e138a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:40.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.529+0000 7f6e0be00640 1 -- 172.21.15.16:0/3612312155 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0e0fa6980 msgr2=0x55a0e138a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:40.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.529+0000 7f6e0be00640 1 --2- 172.21.15.16:0/3612312155 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0e0fa6980 0x55a0e138a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:40.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.529+0000 7f6e0be00640 1 -- 172.21.15.16:0/3612312155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e0faeb60 msgr2=0x55a0e138a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:40.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.529+0000 7f6e0be00640 1 --2- 172.21.15.16:0/3612312155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e0faeb60 0x55a0e138a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:40.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.529+0000 7f6e0be00640 1 -- 172.21.15.16:0/3612312155 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0e1049b90 con 0x55a0e0fcf840 2024-08-31T06:10:40.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.529+0000 7f6e0b400640 1 --2- 172.21.15.16:0/3612312155 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0e0fa6980 0x55a0e138a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:10:40.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.529+0000 7f6df9600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:10:40.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.529+0000 7f6e0be00640 1 --2- 172.21.15.16:0/3612312155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0e0fcf840 0x55a0e1396440 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6e04006010 tx=0x7f6e0402a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b951d5e8722993b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:40.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.530+0000 7f6df9600640 1 -- 172.21.15.16:0/3612312155 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0404e070 con 0x55a0e0fcf840 2024-08-31T06:10:40.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.531+0000 7f6df9600640 1 -- 172.21.15.16:0/3612312155 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f6e04004040 con 0x55a0e0fcf840 2024-08-31T06:10:40.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.532+0000 7f6e1902e740 1 -- 172.21.15.16:0/3612312155 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6df40ec0d0 con 0x55a0e0fcf840 2024-08-31T06:10:40.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.543+0000 7f6df9600640 1 -- 172.21.15.16:0/3612312155 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6e04058050 con 0x55a0e0fcf840 2024-08-31T06:10:40.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.544+0000 7f6e1902e740 10 client.26336 fetch_fsmap learned FSMap version 31 2024-08-31T06:10:40.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.544+0000 7f6e1902e740 1 -- 172.21.15.16:0/3612312155 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a0e137eba0 con 0x55a0e0fcf840 2024-08-31T06:10:40.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.545+0000 7f6df9600640 1 -- 172.21.15.16:0/3612312155 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e040046f0 con 0x55a0e0fcf840 2024-08-31T06:10:40.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.545+0000 7f6df9600640 1 -- 172.21.15.16:0/3612312155 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f6e04043040 con 0x55a0e0fcf840 2024-08-31T06:10:40.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.545+0000 7f6e1902e740 10 client.26336 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:10:40.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.545+0000 7f6e1902e740 10 client.26336 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:10:40.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.545+0000 7f6e1902e740 20 client.26336 populate_metadata read hostname 'smithi016' 2024-08-31T06:10:40.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.546+0000 7f6e1902e740 1 --2- 172.21.15.16:0/3612312155 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a0e0f8e1c0 0x55a0e13821a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:40.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.546+0000 7f6e1902e740 4 client.26336 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:10:40.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.546+0000 7f6e1902e740 1 -- 172.21.15.16:0/3612312155 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6df40ec0d0 con 0x55a0e0f8e1c0 2024-08-31T06:10:40.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.546+0000 7f6e0b400640 1 --2- 172.21.15.16:0/3612312155 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a0e0f8e1c0 0x55a0e13821a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:40.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.547+0000 7f6df9600640 10 client.26336 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:40.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.548+0000 7f6e0b400640 1 --2- 172.21.15.16:0/3612312155 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a0e0f8e1c0 0x55a0e13821a0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=377c284b1282c2e4 server_cookie=8079952ada7c2d1c in_seq=0 out_seq=0 2024-08-31T06:10:40.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.549+0000 7f6df9600640 1 -- 172.21.15.16:0/3612312155 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f6df40ec0d0 con 0x55a0e0f8e1c0 2024-08-31T06:10:40.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.549+0000 7f6df9600640 10 client.26336 handle_command_reply: tid=0 2024-08-31T06:10:40.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.549+0000 7f6e1902e740 1 client.26336 shutdown 2024-08-31T06:10:40.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.550+0000 7f6e1902e740 1 -- 172.21.15.16:0/3612312155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0e0fcf840 msgr2=0x55a0e1396440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:40.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.550+0000 7f6e1902e740 1 --2- 172.21.15.16:0/3612312155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0e0fcf840 0x55a0e1396440 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6e04006010 tx=0x7f6e0402a8c0 comp rx=0 tx=0).stop 2024-08-31T06:10:40.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.550+0000 7f6e1902e740 1 -- 172.21.15.16:0/3612312155 shutdown_connections 2024-08-31T06:10:40.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.550+0000 7f6e1902e740 1 --2- 172.21.15.16:0/3612312155 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a0e0f8e1c0 0x55a0e13821a0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:40.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.550+0000 7f6e1902e740 1 --2- 172.21.15.16:0/3612312155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0e0fcf840 0x55a0e1396440 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:40.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.550+0000 7f6e1902e740 1 --2- 172.21.15.16:0/3612312155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e0faeb60 0x55a0e138a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:40.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.550+0000 7f6e1902e740 1 --2- 172.21.15.16:0/3612312155 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0e0fa6980 0x55a0e138a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:40.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.550+0000 7f6e1902e740 1 -- 172.21.15.16:0/3612312155 >> 172.21.15.16:0/3612312155 conn(0x55a0e0fc3990 msgr2=0x55a0e13727e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:40.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.550+0000 7f6df9600640 0 client.26336 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:40.554 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:10:40.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.550+0000 7f6e1902e740 1 -- 172.21.15.16:0/3612312155 shutdown_connections 2024-08-31T06:10:40.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.551+0000 7f6e1902e740 1 -- 172.21.15.16:0/3612312155 wait complete. 2024-08-31T06:10:40.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.551+0000 7f6e1902e740 20 client.26336 trim_cache size 0 max 16384 2024-08-31T06:10:40.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.581+0000 7f6e1902e740 1 Processor -- start 2024-08-31T06:10:40.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.581+0000 7f6e1902e740 1 -- start start 2024-08-31T06:10:40.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.581+0000 7f6e1902e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e0fa6980 0x55a0e138c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:40.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.581+0000 7f6e1902e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0e0faeb60 0x55a0e137c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:40.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.581+0000 7f6e1902e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0e0fcf840 0x55a0e1387460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:40.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.581+0000 7f6e1902e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6de8011120 con 0x55a0e0fa6980 2024-08-31T06:10:40.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.581+0000 7f6e1902e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6de800e930 con 0x55a0e0fcf840 2024-08-31T06:10:40.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.581+0000 7f6e1902e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6de800c780 con 0x55a0e0faeb60 2024-08-31T06:10:40.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.582+0000 7f6e0b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e0fa6980 0x55a0e138c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:40.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.582+0000 7f6e0be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0e0fcf840 0x55a0e1387460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:40.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.582+0000 7f6e0b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e0fa6980 0x55a0e138c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54602/0 (socket says 172.21.15.16:54602) 2024-08-31T06:10:40.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.582+0000 7f6e0b400640 1 -- 172.21.15.16:0/984777069 learned_addr learned my addr 172.21.15.16:0/984777069 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:40.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.582+0000 7f6e03e00640 1 --2- 172.21.15.16:0/984777069 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0e0faeb60 0x55a0e137c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:40.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.582+0000 7f6e0b400640 1 -- 172.21.15.16:0/984777069 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0e0faeb60 msgr2=0x55a0e137c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:40.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.582+0000 7f6e0b400640 1 --2- 172.21.15.16:0/984777069 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0e0faeb60 0x55a0e137c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:40.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.582+0000 7f6e0b400640 1 -- 172.21.15.16:0/984777069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0e0fcf840 msgr2=0x55a0e1387460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:40.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.582+0000 7f6e0b400640 1 --2- 172.21.15.16:0/984777069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0e0fcf840 0x55a0e1387460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:40.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.582+0000 7f6e0b400640 1 -- 172.21.15.16:0/984777069 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0e1378f20 con 0x55a0e0fa6980 2024-08-31T06:10:40.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.582+0000 7f6e03e00640 1 --2- 172.21.15.16:0/984777069 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0e0faeb60 0x55a0e137c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:10:40.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.582+0000 7f6e0be00640 1 --2- 172.21.15.16:0/984777069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0e0fcf840 0x55a0e1387460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:10:40.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.583+0000 7f6e0b400640 1 --2- 172.21.15.16:0/984777069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e0fa6980 0x55a0e138c130 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc00b920 tx=0x7f6dfc00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e13e54d1816f723d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:40.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.583+0000 7f6dfbe00640 1 -- 172.21.15.16:0/984777069 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc005220 con 0x55a0e0fa6980 2024-08-31T06:10:40.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.583+0000 7f6dfbe00640 1 -- 172.21.15.16:0/984777069 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dfc00be30 con 0x55a0e0fa6980 2024-08-31T06:10:40.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.583+0000 7f6dfbe00640 1 -- 172.21.15.16:0/984777069 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc0117c0 con 0x55a0e0fa6980 2024-08-31T06:10:40.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.584+0000 7f6e1902e740 1 -- 172.21.15.16:0/984777069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e0fa6980 msgr2=0x55a0e138c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:40.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.584+0000 7f6e1902e740 1 --2- 172.21.15.16:0/984777069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e0fa6980 0x55a0e138c130 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc00b920 tx=0x7f6dfc00b390 comp rx=0 tx=0).stop 2024-08-31T06:10:40.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.584+0000 7f6e1902e740 1 -- 172.21.15.16:0/984777069 shutdown_connections 2024-08-31T06:10:40.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.584+0000 7f6e1902e740 1 --2- 172.21.15.16:0/984777069 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0e0faeb60 0x55a0e137c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:40.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.584+0000 7f6e1902e740 1 --2- 172.21.15.16:0/984777069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0e0fcf840 0x55a0e1387460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:40.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.584+0000 7f6e1902e740 1 --2- 172.21.15.16:0/984777069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e0fa6980 0x55a0e138c130 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:40.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.584+0000 7f6e1902e740 1 -- 172.21.15.16:0/984777069 >> 172.21.15.16:0/984777069 conn(0x55a0e0fc3990 msgr2=0x55a0e1373d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:40.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.585+0000 7f6e1902e740 1 -- 172.21.15.16:0/984777069 shutdown_connections 2024-08-31T06:10:40.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.585+0000 7f6e1902e740 1 -- 172.21.15.16:0/984777069 wait complete. 2024-08-31T06:10:40.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.586+0000 7f6e1902e740 1 -- start start 2024-08-31T06:10:40.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.586+0000 7f6e1902e740 1 Processor -- start 2024-08-31T06:10:40.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.587+0000 7f6e1902e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0e0faeb60 0x55a0e1392ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:40.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.587+0000 7f6e1902e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e0fcf840 0x55a0e138f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:40.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.587+0000 7f6e1902e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0e0f63570 0x55a0e138f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:40.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.587+0000 7f6e1902e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6de800cdf0 con 0x55a0e0fcf840 2024-08-31T06:10:40.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.587+0000 7f6e1902e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6de800e260 con 0x55a0e0f63570 2024-08-31T06:10:40.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.587+0000 7f6e1902e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6de800d4c0 con 0x55a0e0faeb60 2024-08-31T06:10:40.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.587+0000 7f6e1902e740 10 client.0 _ll_register_callbacks cb 0x7f6e04031f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:10:40.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.587+0000 7f6e0b400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0e0faeb60 0x55a0e1392ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:40.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.587+0000 7f6e0b400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0e0faeb60 0x55a0e1392ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:35826/0 (socket says 172.21.15.16:35826) 2024-08-31T06:10:40.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.587+0000 7f6e0b400640 1 -- 172.21.15.16:0/1479951846 learned_addr learned my addr 172.21.15.16:0/1479951846 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:40.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.587+0000 7f6e03e00640 1 --2- 172.21.15.16:0/1479951846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e0fcf840 0x55a0e138f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:40.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.588+0000 7f6e0be00640 1 --2- 172.21.15.16:0/1479951846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0e0f63570 0x55a0e138f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:40.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.588+0000 7f6e0be00640 1 -- 172.21.15.16:0/1479951846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0e0faeb60 msgr2=0x55a0e1392ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:40.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.588+0000 7f6e0be00640 1 --2- 172.21.15.16:0/1479951846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0e0faeb60 0x55a0e1392ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:40.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.588+0000 7f6e0be00640 1 -- 172.21.15.16:0/1479951846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e0fcf840 msgr2=0x55a0e138f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:40.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.588+0000 7f6e0be00640 1 --2- 172.21.15.16:0/1479951846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e0fcf840 0x55a0e138f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:40.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.588+0000 7f6e0be00640 1 -- 172.21.15.16:0/1479951846 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0e138ffd0 con 0x55a0e0f63570 2024-08-31T06:10:40.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.588+0000 7f6e0b400640 1 --2- 172.21.15.16:0/1479951846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0e0faeb60 0x55a0e1392ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:10:40.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.589+0000 7f6e0be00640 1 --2- 172.21.15.16:0/1479951846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0e0f63570 0x55a0e138f840 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6e04032010 tx=0x7f6e04003dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fdbcd67afa700fd server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:40.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.589+0000 7f6df9600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:10:40.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.590+0000 7f6df9600640 1 -- 172.21.15.16:0/1479951846 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e04061070 con 0x55a0e0f63570 2024-08-31T06:10:40.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.590+0000 7f6df9600640 1 -- 172.21.15.16:0/1479951846 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f6e04030c70 con 0x55a0e0f63570 2024-08-31T06:10:40.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.590+0000 7f6e1902e740 1 -- 172.21.15.16:0/1479951846 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dfc00be30 con 0x55a0e0f63570 2024-08-31T06:10:40.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.590+0000 7f6df9600640 1 -- 172.21.15.16:0/1479951846 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6e0406d050 con 0x55a0e0f63570 2024-08-31T06:10:40.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.591+0000 7f6e1902e740 10 client.26342 fetch_fsmap learned FSMap version 31 2024-08-31T06:10:40.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.591+0000 7f6e1902e740 1 -- 172.21.15.16:0/1479951846 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a0e1410c50 con 0x55a0e0f63570 2024-08-31T06:10:40.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.591+0000 7f6df9600640 1 -- 172.21.15.16:0/1479951846 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dfc00be30 con 0x55a0e0f63570 2024-08-31T06:10:40.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.591+0000 7f6df9600640 1 -- 172.21.15.16:0/1479951846 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f6e04043040 con 0x55a0e0f63570 2024-08-31T06:10:40.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.591+0000 7f6e1902e740 10 client.26342 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:10:40.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.591+0000 7f6e1902e740 10 client.26342 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:10:40.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.591+0000 7f6e1902e740 20 client.26342 populate_metadata read hostname 'smithi016' 2024-08-31T06:10:40.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.592+0000 7f6e1902e740 1 --2- 172.21.15.16:0/1479951846 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a0e1411440 0x55a0e1413860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:40.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.592+0000 7f6e1902e740 4 client.26342 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.8, "await_for": 45.0}] 2024-08-31T06:10:40.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.592+0000 7f6e1902e740 1 -- 172.21.15.16:0/1479951846 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.8, "await_for": 45.0}) -- 0x55a0e12ed480 con 0x55a0e1411440 2024-08-31T06:10:40.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.592+0000 7f6e0b400640 1 --2- 172.21.15.16:0/1479951846 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a0e1411440 0x55a0e1413860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:40.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.595+0000 7f6e0b400640 1 --2- 172.21.15.16:0/1479951846 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a0e1411440 0x55a0e1413860 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f4d106fbbb0d49b server_cookie=9c3fe72cddc745d3 in_seq=0 out_seq=0 2024-08-31T06:10:40.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.595+0000 7f6df9600640 10 client.26342 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:40.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.814+0000 7f6df9600640 1 -- 172.21.15.16:0/1479951846 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a0e12ed480 con 0x55a0e1411440 2024-08-31T06:10:40.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.814+0000 7f6df9600640 10 client.26342 handle_command_reply: tid=0 2024-08-31T06:10:40.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.814+0000 7f6e1902e740 1 client.26342 shutdown 2024-08-31T06:10:40.817 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:40 smithi016 ceph-mon[17194]: pgmap v467: 97 pgs: 97 active+clean; 2.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 2.8 MiB/s wr, 289 op/s 2024-08-31T06:10:40.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.814+0000 7f6e1902e740 1 -- 172.21.15.16:0/1479951846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0e0f63570 msgr2=0x55a0e138f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:40.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.814+0000 7f6e1902e740 1 --2- 172.21.15.16:0/1479951846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0e0f63570 0x55a0e138f840 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6e04032010 tx=0x7f6e04003dc0 comp rx=0 tx=0).stop 2024-08-31T06:10:40.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.815+0000 7f6e1902e740 1 -- 172.21.15.16:0/1479951846 shutdown_connections 2024-08-31T06:10:40.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.815+0000 7f6e1902e740 1 --2- 172.21.15.16:0/1479951846 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a0e1411440 0x55a0e1413860 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:40.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.815+0000 7f6e1902e740 1 --2- 172.21.15.16:0/1479951846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0e0f63570 0x55a0e138f840 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:40.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.815+0000 7f6e1902e740 1 --2- 172.21.15.16:0/1479951846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e0fcf840 0x55a0e138f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:40.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.815+0000 7f6e1902e740 1 --2- 172.21.15.16:0/1479951846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0e0faeb60 0x55a0e1392ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:40.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.815+0000 7f6e1902e740 1 -- 172.21.15.16:0/1479951846 >> 172.21.15.16:0/1479951846 conn(0x55a0e0fc3990 msgr2=0x55a0e1377040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:40.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.815+0000 7f6df9600640 0 client.26342 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:40.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.815+0000 7f6e1902e740 1 -- 172.21.15.16:0/1479951846 shutdown_connections 2024-08-31T06:10:40.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.815+0000 7f6e1902e740 1 -- 172.21.15.16:0/1479951846 wait complete. 2024-08-31T06:10:40.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.815+0000 7f6e1902e740 20 client.26342 trim_cache size 0 max 16384 2024-08-31T06:10:40.820 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:10:40.820 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 31, 2024-08-31T06:10:40.820 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24737, 2024-08-31T06:10:40.820 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 14, 2024-08-31T06:10:40.820 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-08-31T06:10:40.820 INFO:teuthology.orchestra.run.smithi016.stdout: "9b804def": { 2024-08-31T06:10:40.820 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 14, 2024-08-31T06:10:40.820 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-08-31T06:10:40.820 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:10:40.820 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-08-31T06:10:40.821 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:10:40.821 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:10:40.821 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-08-31T06:10:40.821 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 141.8, 2024-08-31T06:10:40.821 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-08-31T06:10:40.821 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-08-31T06:10:40.821 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-08-31T06:10:40.821 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:10:40.821 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-08-31T06:10:40.821 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:10:40.821 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:10:40.821 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:10:40.821 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:10:40.821 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:10:40.822 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:10:40.822 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:10:40.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.816+0000 7f6e12400640 1 -- 172.21.15.16:0/942061671 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6dec0b4da0 msgr2=0x7f6dec0b72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:40.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.816+0000 7f6e12400640 1 --2- 172.21.15.16:0/942061671 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6dec0b4da0 0x7f6dec0b72b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6e040094e0 tx=0x7f6e040095c0 comp rx=0 tx=0).stop 2024-08-31T06:10:40.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.816+0000 7f6e12400640 1 -- 172.21.15.16:0/942061671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e0c10b4a0 msgr2=0x7f6e0c06de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:40.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.816+0000 7f6e12400640 1 --2- 172.21.15.16:0/942061671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e0c10b4a0 0x7f6e0c06de30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6df400d950 tx=0x7f6df400de10 comp rx=0 tx=0).stop 2024-08-31T06:10:40.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.816+0000 7f6e12400640 1 -- 172.21.15.16:0/942061671 shutdown_connections 2024-08-31T06:10:40.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.816+0000 7f6e12400640 1 --2- 172.21.15.16:0/942061671 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6dec0b4da0 0x7f6dec0b72b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:40.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.816+0000 7f6e12400640 1 --2- 172.21.15.16:0/942061671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e0c10bdf0 0x7f6e0c06e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:40.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.816+0000 7f6e12400640 1 --2- 172.21.15.16:0/942061671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e0c10b4a0 0x7f6e0c06de30 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:40.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.816+0000 7f6e12400640 1 --2- 172.21.15.16:0/942061671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e0c105b60 0x7f6e0c074b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:40.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.816+0000 7f6e12400640 1 -- 172.21.15.16:0/942061671 >> 172.21.15.16:0/942061671 conn(0x7f6e0c101570 msgr2=0x7f6e0c1089a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:40.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.817+0000 7f6e12400640 1 -- 172.21.15.16:0/942061671 shutdown_connections 2024-08-31T06:10:40.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:40.817+0000 7f6e12400640 1 -- 172.21.15.16:0/942061671 wait complete. 2024-08-31T06:10:40.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:40 smithi155 ceph-mon[21742]: pgmap v467: 97 pgs: 97 active+clean; 2.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 2.8 MiB/s wr, 289 op/s 2024-08-31T06:10:40.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:40 smithi184 ceph-mon[22331]: pgmap v467: 97 pgs: 97 active+clean; 2.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 2.8 MiB/s wr, 289 op/s 2024-08-31T06:10:41.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 195.0 s, 1670.8 tps, lat 47.872 ms stddev 59.067 2024-08-31T06:10:41.676 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9b804def', quiesce took 5.5 seconds. Will release after: 16.3 2024-08-31T06:10:42.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.015+0000 7f1c7d600640 1 -- 172.21.15.16:0/2914132018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c78103730 msgr2=0x7f1c78103b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:42.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.015+0000 7f1c7d600640 1 --2- 172.21.15.16:0/2914132018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c78103730 0x7f1c78103b40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1c68009900 tx=0x7f1c6802a3f0 comp rx=0 tx=0).stop 2024-08-31T06:10:42.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.015+0000 7f1c7d600640 1 -- 172.21.15.16:0/2914132018 shutdown_connections 2024-08-31T06:10:42.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.015+0000 7f1c7d600640 1 --2- 172.21.15.16:0/2914132018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c78109070 0x7f1c78109480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:42.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.015+0000 7f1c7d600640 1 --2- 172.21.15.16:0/2914132018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c781099c0 0x7f1c781101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:42.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.015+0000 7f1c7d600640 1 --2- 172.21.15.16:0/2914132018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c78103730 0x7f1c78103b40 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:42.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.015+0000 7f1c7d600640 1 -- 172.21.15.16:0/2914132018 >> 172.21.15.16:0/2914132018 conn(0x7f1c780ff180 msgr2=0x7f1c781015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:42.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.015+0000 7f1c7d600640 1 -- 172.21.15.16:0/2914132018 shutdown_connections 2024-08-31T06:10:42.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.015+0000 7f1c7d600640 1 -- 172.21.15.16:0/2914132018 wait complete. 2024-08-31T06:10:42.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.016+0000 7f1c7d600640 1 Processor -- start 2024-08-31T06:10:42.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.017+0000 7f1c7d600640 1 -- start start 2024-08-31T06:10:42.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.017+0000 7f1c7d600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c78103730 0x7f1c7819c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:42.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.017+0000 7f1c7d600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c78109070 0x7f1c7819c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:42.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.017+0000 7f1c7d600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c781099c0 0x7f1c781a1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:42.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.017+0000 7f1c7d600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1c781136e0 con 0x7f1c78103730 2024-08-31T06:10:42.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.017+0000 7f1c7d600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1c78113560 con 0x7f1c78109070 2024-08-31T06:10:42.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.017+0000 7f1c7d600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1c78113860 con 0x7f1c781099c0 2024-08-31T06:10:42.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.017+0000 7f1c77e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c78103730 0x7f1c7819c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:42.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.017+0000 7f1c77400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c78109070 0x7f1c7819c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:42.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.017+0000 7f1c77400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c78109070 0x7f1c7819c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:34474/0 (socket says 172.21.15.16:34474) 2024-08-31T06:10:42.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.017+0000 7f1c77400640 1 -- 172.21.15.16:0/2245564683 learned_addr learned my addr 172.21.15.16:0/2245564683 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:42.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.017+0000 7f1c77e00640 1 --2- 172.21.15.16:0/2245564683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c78103730 0x7f1c7819c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54630/0 (socket says 172.21.15.16:54630) 2024-08-31T06:10:42.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.018+0000 7f1c7cc00640 1 --2- 172.21.15.16:0/2245564683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c781099c0 0x7f1c781a1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:42.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.018+0000 7f1c77400640 1 -- 172.21.15.16:0/2245564683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c781099c0 msgr2=0x7f1c781a1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:42.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.018+0000 7f1c77400640 1 --2- 172.21.15.16:0/2245564683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c781099c0 0x7f1c781a1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:42.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.018+0000 7f1c77400640 1 -- 172.21.15.16:0/2245564683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c78103730 msgr2=0x7f1c7819c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:42.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.018+0000 7f1c77400640 1 --2- 172.21.15.16:0/2245564683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c78103730 0x7f1c7819c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:42.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.018+0000 7f1c77400640 1 -- 172.21.15.16:0/2245564683 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c781a22a0 con 0x7f1c78109070 2024-08-31T06:10:42.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.018+0000 7f1c77400640 1 --2- 172.21.15.16:0/2245564683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c78109070 0x7f1c7819c8d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1c6400e9c0 tx=0x7f1c6400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:42.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.018+0000 7f1c74c00640 1 -- 172.21.15.16:0/2245564683 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6400ccc0 con 0x7f1c78109070 2024-08-31T06:10:42.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.018+0000 7f1c7d600640 1 -- 172.21.15.16:0/2245564683 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c781a2590 con 0x7f1c78109070 2024-08-31T06:10:42.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.018+0000 7f1c74c00640 1 -- 172.21.15.16:0/2245564683 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c640054d0 con 0x7f1c78109070 2024-08-31T06:10:42.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.018+0000 7f1c74c00640 1 -- 172.21.15.16:0/2245564683 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c64010430 con 0x7f1c78109070 2024-08-31T06:10:42.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.019+0000 7f1c7d600640 1 -- 172.21.15.16:0/2245564683 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c781a8cf0 con 0x7f1c78109070 2024-08-31T06:10:42.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.022+0000 7f1c85801740 1 Processor -- start 2024-08-31T06:10:42.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.022+0000 7f1c85801740 1 -- start start 2024-08-31T06:10:42.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.022+0000 7f1c85801740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eea606070 0x562eea9aba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:42.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.022+0000 7f1c85801740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eea5dd1b0 0x562eea9abf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:42.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.022+0000 7f1c85801740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562eea5e5390 0x562eea9ac4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:42.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.022+0000 7f1c85801740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562eea922770 con 0x562eea5e5390 2024-08-31T06:10:42.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.022+0000 7f1c85801740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562eea911d00 con 0x562eea5dd1b0 2024-08-31T06:10:42.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.022+0000 7f1c85801740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562eea915cb0 con 0x562eea606070 2024-08-31T06:10:42.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.023+0000 7f1c77e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eea5dd1b0 0x562eea9abf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:42.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.023+0000 7f1c7cc00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eea606070 0x562eea9aba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:42.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.023+0000 7f1c74c00640 1 -- 172.21.15.16:0/2245564683 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f1c64005020 con 0x7f1c78109070 2024-08-31T06:10:42.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.023+0000 7f1c7cc00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eea606070 0x562eea9aba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:35854/0 (socket says 172.21.15.16:35854) 2024-08-31T06:10:42.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.023+0000 7f1c7cc00640 1 -- 172.21.15.16:0/1934669651 learned_addr learned my addr 172.21.15.16:0/1934669651 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:42.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.023+0000 7f1c77e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eea5dd1b0 0x562eea9abf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:34488/0 (socket says 172.21.15.16:34488) 2024-08-31T06:10:42.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.023+0000 7f1c77400640 1 --2- 172.21.15.16:0/1934669651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562eea5e5390 0x562eea9ac4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:42.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.023+0000 7f1c77e00640 1 -- 172.21.15.16:0/1934669651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eea606070 msgr2=0x562eea9aba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:42.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.023+0000 7f1c77e00640 1 --2- 172.21.15.16:0/1934669651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eea606070 0x562eea9aba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:42.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.023+0000 7f1c77e00640 1 -- 172.21.15.16:0/1934669651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562eea5e5390 msgr2=0x562eea9ac4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:42.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.023+0000 7f1c77e00640 1 --2- 172.21.15.16:0/1934669651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562eea5e5390 0x562eea9ac4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:42.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.023+0000 7f1c77e00640 1 -- 172.21.15.16:0/1934669651 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eea671650 con 0x562eea5dd1b0 2024-08-31T06:10:42.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.023+0000 7f1c77400640 1 --2- 172.21.15.16:0/1934669651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562eea5e5390 0x562eea9ac4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:10:42.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.023+0000 7f1c77e00640 1 --2- 172.21.15.16:0/1934669651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eea5dd1b0 0x562eea9abf70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1c6802a9e0 tx=0x7f1c68004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37d595056effa0a9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:42.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.024+0000 7f1c6d200640 1 -- 172.21.15.16:0/1934669651 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c68035040 con 0x562eea5dd1b0 2024-08-31T06:10:42.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.024+0000 7f1c6d200640 1 -- 172.21.15.16:0/1934669651 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c680054f0 con 0x562eea5dd1b0 2024-08-31T06:10:42.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.024+0000 7f1c6d200640 1 -- 172.21.15.16:0/1934669651 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c680035f0 con 0x562eea5dd1b0 2024-08-31T06:10:42.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.025+0000 7f1c85801740 1 -- 172.21.15.16:0/1934669651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eea5dd1b0 msgr2=0x562eea9abf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:42.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.025+0000 7f1c85801740 1 --2- 172.21.15.16:0/1934669651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eea5dd1b0 0x562eea9abf70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1c6802a9e0 tx=0x7f1c68004040 comp rx=0 tx=0).stop 2024-08-31T06:10:42.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.025+0000 7f1c85801740 1 -- 172.21.15.16:0/1934669651 shutdown_connections 2024-08-31T06:10:42.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.025+0000 7f1c85801740 1 --2- 172.21.15.16:0/1934669651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562eea5e5390 0x562eea9ac4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:42.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.025+0000 7f1c74c00640 1 --2- 172.21.15.16:0/2245564683 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f1c5c0b4ca0 0x7f1c5c0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:42.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.025+0000 7f1c85801740 1 --2- 172.21.15.16:0/1934669651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eea5dd1b0 0x562eea9abf70 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:42.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.025+0000 7f1c85801740 1 --2- 172.21.15.16:0/1934669651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eea606070 0x562eea9aba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:42.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.025+0000 7f1c85801740 1 -- 172.21.15.16:0/1934669651 >> 172.21.15.16:0/1934669651 conn(0x562eea5fa460 msgr2=0x562eea9a9920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:42.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.025+0000 7f1c74c00640 1 -- 172.21.15.16:0/2245564683 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f1c640e5870 con 0x7f1c78109070 2024-08-31T06:10:42.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.025+0000 7f1c85801740 1 -- 172.21.15.16:0/1934669651 shutdown_connections 2024-08-31T06:10:42.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.025+0000 7f1c85801740 1 -- 172.21.15.16:0/1934669651 wait complete. 2024-08-31T06:10:42.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.025+0000 7f1c7cc00640 1 --2- 172.21.15.16:0/2245564683 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f1c5c0b4ca0 0x7f1c5c0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:42.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.026+0000 7f1c85801740 1 -- start start 2024-08-31T06:10:42.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.026+0000 7f1c85801740 1 Processor -- start 2024-08-31T06:10:42.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.026+0000 7f1c7cc00640 1 --2- 172.21.15.16:0/2245564683 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f1c5c0b4ca0 0x7f1c5c0b71b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1c70005fd0 tx=0x7f1c70009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:42.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.026+0000 7f1c85801740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562eea5dd1b0 0x562eea9b7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:42.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.027+0000 7f1c85801740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eea5e5390 0x562eea9b7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:42.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.027+0000 7f1c85801740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eea606070 0x562eea9c34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:42.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.027+0000 7f1c85801740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562eea774700 con 0x562eea5dd1b0 2024-08-31T06:10:42.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.027+0000 7f1c85801740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562eea898cb0 con 0x562eea606070 2024-08-31T06:10:42.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.027+0000 7f1c85801740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562eea9ba1b0 con 0x562eea5e5390 2024-08-31T06:10:42.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.027+0000 7f1c85801740 10 client.0 _ll_register_callbacks cb 0x562eea8be430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:10:42.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.027+0000 7f1c7cc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eea606070 0x562eea9c34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:42.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.027+0000 7f1c7cc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eea606070 0x562eea9c34a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:34496/0 (socket says 172.21.15.16:34496) 2024-08-31T06:10:42.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.027+0000 7f1c7cc00640 1 -- 172.21.15.16:0/3593060011 learned_addr learned my addr 172.21.15.16:0/3593060011 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:42.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.027+0000 7f1c77e00640 1 --2- 172.21.15.16:0/3593060011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562eea5dd1b0 0x562eea9b7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:42.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.027+0000 7f1c77400640 1 --2- 172.21.15.16:0/3593060011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eea5e5390 0x562eea9b7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:42.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.027+0000 7f1c7cc00640 1 -- 172.21.15.16:0/3593060011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eea5e5390 msgr2=0x562eea9b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:42.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.027+0000 7f1c7cc00640 1 --2- 172.21.15.16:0/3593060011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eea5e5390 0x562eea9b7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:42.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.027+0000 7f1c7cc00640 1 -- 172.21.15.16:0/3593060011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562eea5dd1b0 msgr2=0x562eea9b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:42.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.027+0000 7f1c7cc00640 1 --2- 172.21.15.16:0/3593060011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562eea5dd1b0 0x562eea9b7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:42.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.027+0000 7f1c7cc00640 1 -- 172.21.15.16:0/3593060011 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eea8d9870 con 0x562eea606070 2024-08-31T06:10:42.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.027+0000 7f1c77400640 1 --2- 172.21.15.16:0/3593060011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eea5e5390 0x562eea9b7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:10:42.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.027+0000 7f1c77e00640 1 --2- 172.21.15.16:0/3593060011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562eea5dd1b0 0x562eea9b7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:10:42.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.028+0000 7f1c7cc00640 1 --2- 172.21.15.16:0/3593060011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eea606070 0x562eea9c34a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1c70004070 tx=0x7f1c70004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7ccd619d2c7a92e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:42.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.028+0000 7f1c61600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:10:42.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.028+0000 7f1c61600640 1 -- 172.21.15.16:0/3593060011 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7000f910 con 0x562eea606070 2024-08-31T06:10:42.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.028+0000 7f1c85801740 1 -- 172.21.15.16:0/3593060011 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c680054f0 con 0x562eea606070 2024-08-31T06:10:42.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.028+0000 7f1c61600640 1 -- 172.21.15.16:0/3593060011 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f1c7000fab0 con 0x562eea606070 2024-08-31T06:10:42.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.028+0000 7f1c61600640 1 -- 172.21.15.16:0/3593060011 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1c7000fcb0 con 0x562eea606070 2024-08-31T06:10:42.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.028+0000 7f1c85801740 10 client.26354 fetch_fsmap learned FSMap version 31 2024-08-31T06:10:42.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.028+0000 7f1c85801740 1 -- 172.21.15.16:0/3593060011 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562eea9c6260 con 0x562eea606070 2024-08-31T06:10:42.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.028+0000 7f1c61600640 1 -- 172.21.15.16:0/3593060011 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c680054f0 con 0x562eea606070 2024-08-31T06:10:42.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.029+0000 7f1c61600640 1 -- 172.21.15.16:0/3593060011 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f1c70026020 con 0x562eea606070 2024-08-31T06:10:42.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.029+0000 7f1c85801740 10 client.26354 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:10:42.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.029+0000 7f1c85801740 10 client.26354 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:10:42.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.029+0000 7f1c85801740 20 client.26354 populate_metadata read hostname 'smithi016' 2024-08-31T06:10:42.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.029+0000 7f1c85801740 1 --2- 172.21.15.16:0/3593060011 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x562eea5c49f0 0x562eea9c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:42.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.029+0000 7f1c85801740 4 client.26354 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:10:42.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.029+0000 7f1c85801740 1 -- 172.21.15.16:0/3593060011 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562eea9246c0 con 0x562eea5c49f0 2024-08-31T06:10:42.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.030+0000 7f1c77e00640 1 --2- 172.21.15.16:0/3593060011 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x562eea5c49f0 0x562eea9c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:42.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.030+0000 7f1c77e00640 1 --2- 172.21.15.16:0/3593060011 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x562eea5c49f0 0x562eea9c9770 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5faf9c2685d712b server_cookie=cb3b31d689a08c75 in_seq=0 out_seq=0 2024-08-31T06:10:42.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.030+0000 7f1c61600640 10 client.26354 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:42.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.032+0000 7f1c61600640 1 -- 172.21.15.16:0/3593060011 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562eea9246c0 con 0x562eea5c49f0 2024-08-31T06:10:42.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.032+0000 7f1c61600640 10 client.26354 handle_command_reply: tid=0 2024-08-31T06:10:42.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.032+0000 7f1c85801740 1 client.26354 shutdown 2024-08-31T06:10:42.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.032+0000 7f1c85801740 1 -- 172.21.15.16:0/3593060011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eea606070 msgr2=0x562eea9c34a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:42.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.032+0000 7f1c85801740 1 --2- 172.21.15.16:0/3593060011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eea606070 0x562eea9c34a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1c70004070 tx=0x7f1c70004ab0 comp rx=0 tx=0).stop 2024-08-31T06:10:42.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.032+0000 7f1c85801740 1 -- 172.21.15.16:0/3593060011 shutdown_connections 2024-08-31T06:10:42.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.032+0000 7f1c85801740 1 --2- 172.21.15.16:0/3593060011 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x562eea5c49f0 0x562eea9c9770 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:42.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.032+0000 7f1c85801740 1 --2- 172.21.15.16:0/3593060011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eea5e5390 0x562eea9b7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:42.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.032+0000 7f1c85801740 1 --2- 172.21.15.16:0/3593060011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eea606070 0x562eea9c34a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:42.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.032+0000 7f1c61600640 0 client.26354 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:42.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.032+0000 7f1c85801740 1 --2- 172.21.15.16:0/3593060011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562eea5dd1b0 0x562eea9b7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:42.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.032+0000 7f1c85801740 1 -- 172.21.15.16:0/3593060011 >> 172.21.15.16:0/3593060011 conn(0x562eea5fa460 msgr2=0x562eea9aebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:42.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.032+0000 7f1c85801740 1 -- 172.21.15.16:0/3593060011 shutdown_connections 2024-08-31T06:10:42.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.032+0000 7f1c85801740 1 -- 172.21.15.16:0/3593060011 wait complete. 2024-08-31T06:10:42.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.032+0000 7f1c85801740 20 client.26354 trim_cache size 0 max 16384 2024-08-31T06:10:42.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.052+0000 7f1c85801740 1 Processor -- start 2024-08-31T06:10:42.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.052+0000 7f1c85801740 1 -- start start 2024-08-31T06:10:42.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.053+0000 7f1c85801740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eea5dd1b0 0x562eea9b9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:42.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.053+0000 7f1c85801740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eea5e5390 0x562eea9a8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:42.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.053+0000 7f1c85801740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562eea606070 0x562eea9a9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:42.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.053+0000 7f1c85801740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1c781b3d60 con 0x562eea606070 2024-08-31T06:10:42.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.053+0000 7f1c85801740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1c781b1570 con 0x562eea5e5390 2024-08-31T06:10:42.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.053+0000 7f1c85801740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1c781af3c0 con 0x562eea5dd1b0 2024-08-31T06:10:42.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.053+0000 7f1c77400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eea5e5390 0x562eea9a8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:42.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.053+0000 7f1c77e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eea5dd1b0 0x562eea9b9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:42.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.053+0000 7f1c77400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eea5e5390 0x562eea9a8f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:34508/0 (socket says 172.21.15.16:34508) 2024-08-31T06:10:42.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.053+0000 7f1c77e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eea5dd1b0 0x562eea9b9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:35858/0 (socket says 172.21.15.16:35858) 2024-08-31T06:10:42.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.053+0000 7f1c77400640 1 -- 172.21.15.16:0/1074597478 learned_addr learned my addr 172.21.15.16:0/1074597478 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:42.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.053+0000 7f1c7cc00640 1 --2- 172.21.15.16:0/1074597478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562eea606070 0x562eea9a9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:42.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.053+0000 7f1c77400640 1 -- 172.21.15.16:0/1074597478 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eea5dd1b0 msgr2=0x562eea9b9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:42.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.053+0000 7f1c77400640 1 --2- 172.21.15.16:0/1074597478 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eea5dd1b0 0x562eea9b9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:42.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.053+0000 7f1c77400640 1 -- 172.21.15.16:0/1074597478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562eea606070 msgr2=0x562eea9a9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:42.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.053+0000 7f1c77400640 1 --2- 172.21.15.16:0/1074597478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562eea606070 0x562eea9a9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:42.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.053+0000 7f1c77400640 1 -- 172.21.15.16:0/1074597478 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eea9add20 con 0x562eea5e5390 2024-08-31T06:10:42.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.053+0000 7f1c77e00640 1 --2- 172.21.15.16:0/1074597478 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eea5dd1b0 0x562eea9b9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:10:42.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.054+0000 7f1c77400640 1 --2- 172.21.15.16:0/1074597478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eea5e5390 0x562eea9a8f10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1c640abef0 tx=0x7f1c640ab970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4877a2022a7d2b82 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:42.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.054+0000 7f1c63e00640 1 -- 172.21.15.16:0/1074597478 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c64005680 con 0x562eea5e5390 2024-08-31T06:10:42.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.054+0000 7f1c63e00640 1 -- 172.21.15.16:0/1074597478 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c640e8e50 con 0x562eea5e5390 2024-08-31T06:10:42.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.054+0000 7f1c63e00640 1 -- 172.21.15.16:0/1074597478 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c640e3240 con 0x562eea5e5390 2024-08-31T06:10:42.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.055+0000 7f1c85801740 1 -- 172.21.15.16:0/1074597478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eea5e5390 msgr2=0x562eea9a8f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:42.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.055+0000 7f1c85801740 1 --2- 172.21.15.16:0/1074597478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eea5e5390 0x562eea9a8f10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1c640abef0 tx=0x7f1c640ab970 comp rx=0 tx=0).stop 2024-08-31T06:10:42.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.055+0000 7f1c85801740 1 -- 172.21.15.16:0/1074597478 shutdown_connections 2024-08-31T06:10:42.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.055+0000 7f1c85801740 1 --2- 172.21.15.16:0/1074597478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562eea606070 0x562eea9a9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:42.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.055+0000 7f1c85801740 1 --2- 172.21.15.16:0/1074597478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eea5e5390 0x562eea9a8f10 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:42.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.055+0000 7f1c85801740 1 --2- 172.21.15.16:0/1074597478 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eea5dd1b0 0x562eea9b9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:42.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.055+0000 7f1c85801740 1 -- 172.21.15.16:0/1074597478 >> 172.21.15.16:0/1074597478 conn(0x562eea5fa460 msgr2=0x562eea9b8980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:42.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.055+0000 7f1c85801740 1 -- 172.21.15.16:0/1074597478 shutdown_connections 2024-08-31T06:10:42.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.055+0000 7f1c85801740 1 -- 172.21.15.16:0/1074597478 wait complete. 2024-08-31T06:10:42.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.055+0000 7f1c85801740 1 -- start start 2024-08-31T06:10:42.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.056+0000 7f1c85801740 1 Processor -- start 2024-08-31T06:10:42.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.056+0000 7f1c85801740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eea5dd1b0 0x562eea9bbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:42.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.056+0000 7f1c85801740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eea5e5390 0x562eea9bc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:42.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.056+0000 7f1c85801740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562eea606070 0x562eeaa2d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:42.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.056+0000 7f1c85801740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1c781afa30 con 0x562eea606070 2024-08-31T06:10:42.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.056+0000 7f1c85801740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1c781b0ea0 con 0x562eea5e5390 2024-08-31T06:10:42.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.056+0000 7f1c85801740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1c781b0100 con 0x562eea5dd1b0 2024-08-31T06:10:42.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.056+0000 7f1c85801740 10 client.0 _ll_register_callbacks cb 0x7f1c70003e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:10:42.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.056+0000 7f1c77400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eea5e5390 0x562eea9bc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:42.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.057+0000 7f1c7cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562eea606070 0x562eeaa2d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:42.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.057+0000 7f1c77e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eea5dd1b0 0x562eea9bbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:42.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.057+0000 7f1c77400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eea5e5390 0x562eea9bc490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:34518/0 (socket says 172.21.15.16:34518) 2024-08-31T06:10:42.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.057+0000 7f1c77400640 1 -- 172.21.15.16:0/2181168264 learned_addr learned my addr 172.21.15.16:0/2181168264 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:42.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.057+0000 7f1c77400640 1 -- 172.21.15.16:0/2181168264 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eea5dd1b0 msgr2=0x562eea9bbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:42.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.057+0000 7f1c77400640 1 --2- 172.21.15.16:0/2181168264 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eea5dd1b0 0x562eea9bbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:42.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.057+0000 7f1c77400640 1 -- 172.21.15.16:0/2181168264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562eea606070 msgr2=0x562eeaa2d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:42.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.057+0000 7f1c77400640 1 --2- 172.21.15.16:0/2181168264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562eea606070 0x562eeaa2d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:42.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.057+0000 7f1c77400640 1 -- 172.21.15.16:0/2181168264 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eeaa2d950 con 0x562eea5e5390 2024-08-31T06:10:42.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.057+0000 7f1c77e00640 1 --2- 172.21.15.16:0/2181168264 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eea5dd1b0 0x562eea9bbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:10:42.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.057+0000 7f1c7cc00640 1 --2- 172.21.15.16:0/2181168264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562eea606070 0x562eeaa2d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:10:42.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.057+0000 7f1c77400640 1 --2- 172.21.15.16:0/2181168264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eea5e5390 0x562eea9bc490 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1c6400c6a0 tx=0x7f1c640e34f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9640808e70738e3 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:42.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.057+0000 7f1c61600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:10:42.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.057+0000 7f1c61600640 1 -- 172.21.15.16:0/2181168264 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c64005680 con 0x562eea5e5390 2024-08-31T06:10:42.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.058+0000 7f1c85801740 1 -- 172.21.15.16:0/2181168264 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c640e8e50 con 0x562eea5e5390 2024-08-31T06:10:42.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.058+0000 7f1c61600640 1 -- 172.21.15.16:0/2181168264 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f1c640b5000 con 0x562eea5e5390 2024-08-31T06:10:42.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.058+0000 7f1c61600640 1 -- 172.21.15.16:0/2181168264 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1c640abf60 con 0x562eea5e5390 2024-08-31T06:10:42.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.058+0000 7f1c85801740 10 client.26360 fetch_fsmap learned FSMap version 31 2024-08-31T06:10:42.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.058+0000 7f1c85801740 1 -- 172.21.15.16:0/2181168264 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562eeaa34d50 con 0x562eea5e5390 2024-08-31T06:10:42.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.058+0000 7f1c61600640 1 -- 172.21.15.16:0/2181168264 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c640e8e50 con 0x562eea5e5390 2024-08-31T06:10:42.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.058+0000 7f1c61600640 1 -- 172.21.15.16:0/2181168264 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f1c640af070 con 0x562eea5e5390 2024-08-31T06:10:42.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.058+0000 7f1c85801740 10 client.26360 fetch_fsmap finished waiting for FSMap version 31 2024-08-31T06:10:42.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.058+0000 7f1c85801740 10 client.26360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:10:42.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.058+0000 7f1c85801740 20 client.26360 populate_metadata read hostname 'smithi016' 2024-08-31T06:10:42.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.058+0000 7f1c85801740 1 --2- 172.21.15.16:0/2181168264 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x562eea599da0 0x562eeaa373f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:42.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.058+0000 7f1c85801740 4 client.26360 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T06:10:42.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.058+0000 7f1c85801740 1 -- 172.21.15.16:0/2181168264 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562eea920160 con 0x562eea599da0 2024-08-31T06:10:42.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.059+0000 7f1c77e00640 1 --2- 172.21.15.16:0/2181168264 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x562eea599da0 0x562eeaa373f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:42.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.059+0000 7f1c77e00640 1 --2- 172.21.15.16:0/2181168264 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x562eea599da0 0x562eeaa373f0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bb64801898b5c13 server_cookie=3772eba555ce8a84 in_seq=0 out_seq=0 2024-08-31T06:10:42.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.059+0000 7f1c61600640 10 client.26360 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:42.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.059+0000 7f1c61600640 1 -- 172.21.15.16:0/2181168264 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1510 (crc 0 0 0) 0x562eea920160 con 0x562eea599da0 2024-08-31T06:10:42.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.059+0000 7f1c61600640 10 client.26360 handle_command_reply: tid=0 2024-08-31T06:10:42.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.059+0000 7f1c85801740 1 client.26360 shutdown 2024-08-31T06:10:42.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.060+0000 7f1c85801740 1 -- 172.21.15.16:0/2181168264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eea5e5390 msgr2=0x562eea9bc490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:42.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.060+0000 7f1c85801740 1 --2- 172.21.15.16:0/2181168264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eea5e5390 0x562eea9bc490 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1c6400c6a0 tx=0x7f1c640e34f0 comp rx=0 tx=0).stop 2024-08-31T06:10:42.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.060+0000 7f1c85801740 1 -- 172.21.15.16:0/2181168264 shutdown_connections 2024-08-31T06:10:42.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.060+0000 7f1c85801740 1 --2- 172.21.15.16:0/2181168264 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x562eea599da0 0x562eeaa373f0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:42.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.060+0000 7f1c85801740 1 --2- 172.21.15.16:0/2181168264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562eea606070 0x562eeaa2d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:42.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.060+0000 7f1c85801740 1 --2- 172.21.15.16:0/2181168264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eea5e5390 0x562eea9bc490 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:42.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.060+0000 7f1c85801740 1 --2- 172.21.15.16:0/2181168264 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eea5dd1b0 0x562eea9bbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:42.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.060+0000 7f1c85801740 1 -- 172.21.15.16:0/2181168264 >> 172.21.15.16:0/2181168264 conn(0x562eea5fa460 msgr2=0x562eea9b8870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:42.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.060+0000 7f1c61600640 0 client.26360 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:42.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.060+0000 7f1c85801740 1 -- 172.21.15.16:0/2181168264 shutdown_connections 2024-08-31T06:10:42.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.060+0000 7f1c85801740 1 -- 172.21.15.16:0/2181168264 wait complete. 2024-08-31T06:10:42.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.060+0000 7f1c85801740 20 client.26360 trim_cache size 0 max 16384 2024-08-31T06:10:42.067 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:10:42.067 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-08-31T06:10:42.067 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {}, 2024-08-31T06:10:42.067 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-08-31T06:10:42.067 INFO:teuthology.orchestra.run.smithi016.stdout: "c1fd6904-2e02-47fa-939a-47349b9fe36b": { 2024-08-31T06:10:42.067 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:06.678691+0000", 2024-08-31T06:10:42.067 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:10:42.067 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:10:42.067 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:10:42.067 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:10:42.067 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:10:42.067 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:10:42.068 INFO:teuthology.orchestra.run.smithi016.stdout: "78d6ef02-8551-45e8-ad60-9627eb573fc1": { 2024-08-31T06:10:42.068 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:09:13.547865+0000", 2024-08-31T06:10:42.068 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:10:42.068 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:10:42.068 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:10:42.068 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:10:42.068 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:10:42.068 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:10:42.068 INFO:teuthology.orchestra.run.smithi016.stdout: "c8455d74-2fe7-4d93-bcf2-19b419d16016": { 2024-08-31T06:10:42.068 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:08:24.035960+0000", 2024-08-31T06:10:42.068 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:10:42.068 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:10:42.068 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:10:42.068 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:10:42.069 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:10:42.069 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:10:42.069 INFO:teuthology.orchestra.run.smithi016.stdout: "88d7bdb8-cb1c-45e5-ad29-4cd8da6c6003": { 2024-08-31T06:10:42.069 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:07:39.648942+0000", 2024-08-31T06:10:42.069 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:10:42.069 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:10:42.069 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:10:42.069 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:10:42.069 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:10:42.069 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:10:42.069 INFO:teuthology.orchestra.run.smithi016.stdout: "c300b7e8-f3fd-482c-a909-9329bc61e570": { 2024-08-31T06:10:42.069 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:06:49.868046+0000", 2024-08-31T06:10:42.069 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:10:42.069 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:10:42.070 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:10:42.070 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:10:42.070 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:10:42.070 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:10:42.070 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:10:42.070 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:10:42.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.061+0000 7f1c7d600640 1 -- 172.21.15.16:0/2245564683 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f1c5c0b4ca0 msgr2=0x7f1c5c0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:42.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.061+0000 7f1c7d600640 1 --2- 172.21.15.16:0/2245564683 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f1c5c0b4ca0 0x7f1c5c0b71b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1c70005fd0 tx=0x7f1c70009500 comp rx=0 tx=0).stop 2024-08-31T06:10:42.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.061+0000 7f1c7d600640 1 -- 172.21.15.16:0/2245564683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c78109070 msgr2=0x7f1c7819c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:42.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.061+0000 7f1c7d600640 1 --2- 172.21.15.16:0/2245564683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c78109070 0x7f1c7819c8d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1c6400e9c0 tx=0x7f1c6400ee80 comp rx=0 tx=0).stop 2024-08-31T06:10:42.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.061+0000 7f1c7d600640 1 -- 172.21.15.16:0/2245564683 shutdown_connections 2024-08-31T06:10:42.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.061+0000 7f1c7d600640 1 --2- 172.21.15.16:0/2245564683 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f1c5c0b4ca0 0x7f1c5c0b71b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:42.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.061+0000 7f1c7d600640 1 --2- 172.21.15.16:0/2245564683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c781099c0 0x7f1c781a1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:42.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.061+0000 7f1c7d600640 1 --2- 172.21.15.16:0/2245564683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c78109070 0x7f1c7819c8d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:42.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.061+0000 7f1c7d600640 1 --2- 172.21.15.16:0/2245564683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c78103730 0x7f1c7819c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:42.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.061+0000 7f1c7d600640 1 -- 172.21.15.16:0/2245564683 >> 172.21.15.16:0/2245564683 conn(0x7f1c780ff180 msgr2=0x7f1c78108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:42.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.061+0000 7f1c7d600640 1 -- 172.21.15.16:0/2245564683 shutdown_connections 2024-08-31T06:10:42.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:42.061+0000 7f1c7d600640 1 -- 172.21.15.16:0/2245564683 wait complete. 2024-08-31T06:10:42.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:42 smithi155 ceph-mon[21742]: pgmap v468: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 3.4 MiB/s wr, 362 op/s 2024-08-31T06:10:42.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:42 smithi016 ceph-mon[17194]: pgmap v468: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 3.4 MiB/s wr, 362 op/s 2024-08-31T06:10:42.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:42 smithi184 ceph-mon[22331]: pgmap v468: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 3.4 MiB/s wr, 362 op/s 2024-08-31T06:10:43.472 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T06:10:43.473 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:10:43.868 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:10:43.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:43 smithi155 ceph-mon[21742]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:10:43.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:43 smithi016 ceph-mon[17194]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:10:43.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:43 smithi184 ceph-mon[22331]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:10:44.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.549+0000 7f7815600640 1 -- 172.21.15.16:0/3510068229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7810103750 msgr2=0x7f7810103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:44.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.549+0000 7f7815600640 1 --2- 172.21.15.16:0/3510068229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7810103750 0x7f7810103b60 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7800009900 tx=0x7f780002a3f0 comp rx=0 tx=0).stop 2024-08-31T06:10:44.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.549+0000 7f7815600640 1 -- 172.21.15.16:0/3510068229 shutdown_connections 2024-08-31T06:10:44.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.549+0000 7f7815600640 1 --2- 172.21.15.16:0/3510068229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78101099e0 0x7f7810110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:44.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.549+0000 7f7815600640 1 --2- 172.21.15.16:0/3510068229 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7810109090 0x7f78101094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:44.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.549+0000 7f7815600640 1 --2- 172.21.15.16:0/3510068229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7810103750 0x7f7810103b60 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:44.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.549+0000 7f7815600640 1 -- 172.21.15.16:0/3510068229 >> 172.21.15.16:0/3510068229 conn(0x7f78100ff1e0 msgr2=0x7f7810101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:44.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.550+0000 7f7815600640 1 -- 172.21.15.16:0/3510068229 shutdown_connections 2024-08-31T06:10:44.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.550+0000 7f7815600640 1 -- 172.21.15.16:0/3510068229 wait complete. 2024-08-31T06:10:44.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.551+0000 7f7815600640 1 Processor -- start 2024-08-31T06:10:44.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.551+0000 7f7815600640 1 -- start start 2024-08-31T06:10:44.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.551+0000 7f7815600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7810103750 0x7f7810076ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:44.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.551+0000 7f7815600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7810109090 0x7f78100754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:44.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.551+0000 7f7815600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78101099e0 0x7f78100759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:44.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.551+0000 7f7815600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7810113690 con 0x7f78101099e0 2024-08-31T06:10:44.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.551+0000 7f7815600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7810113510 con 0x7f7810103750 2024-08-31T06:10:44.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.551+0000 7f7815600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7810113810 con 0x7f7810109090 2024-08-31T06:10:44.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.552+0000 7f780fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7810103750 0x7f7810076ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:44.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.552+0000 7f780fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7810103750 0x7f7810076ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:34530/0 (socket says 172.21.15.16:34530) 2024-08-31T06:10:44.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.552+0000 7f780fe00640 1 -- 172.21.15.16:0/3792487991 learned_addr learned my addr 172.21.15.16:0/3792487991 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:44.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.552+0000 7f7807e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7810109090 0x7f78100754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:44.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.552+0000 7f7814c00640 1 --2- 172.21.15.16:0/3792487991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78101099e0 0x7f78100759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:44.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.552+0000 7f7807e00640 1 -- 172.21.15.16:0/3792487991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7810103750 msgr2=0x7f7810076ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:44.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.552+0000 7f7807e00640 1 --2- 172.21.15.16:0/3792487991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7810103750 0x7f7810076ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:44.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.552+0000 7f7807e00640 1 -- 172.21.15.16:0/3792487991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78101099e0 msgr2=0x7f78100759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:44.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.552+0000 7f7807e00640 1 --2- 172.21.15.16:0/3792487991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78101099e0 0x7f78100759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:44.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.552+0000 7f7807e00640 1 -- 172.21.15.16:0/3792487991 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7810076210 con 0x7f7810109090 2024-08-31T06:10:44.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.552+0000 7f780fe00640 1 --2- 172.21.15.16:0/3792487991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7810103750 0x7f7810076ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:10:44.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.552+0000 7f7814c00640 1 --2- 172.21.15.16:0/3792487991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78101099e0 0x7f78100759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:10:44.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.553+0000 7f7807e00640 1 --2- 172.21.15.16:0/3792487991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7810109090 0x7f78100754b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f77f8005fd0 tx=0x7f77f800dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:44.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.554+0000 7f780d600640 1 -- 172.21.15.16:0/3792487991 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f8009c00 con 0x7f7810109090 2024-08-31T06:10:44.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.554+0000 7f780d600640 1 -- 172.21.15.16:0/3792487991 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77f8009da0 con 0x7f7810109090 2024-08-31T06:10:44.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.554+0000 7f7815600640 1 -- 172.21.15.16:0/3792487991 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78101aca80 con 0x7f7810109090 2024-08-31T06:10:44.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.554+0000 7f780d600640 1 -- 172.21.15.16:0/3792487991 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f8003db0 con 0x7f7810109090 2024-08-31T06:10:44.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.554+0000 7f7815600640 1 -- 172.21.15.16:0/3792487991 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78101acfc0 con 0x7f7810109090 2024-08-31T06:10:44.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.555+0000 7f7815600640 1 -- 172.21.15.16:0/3792487991 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7810103b60 con 0x7f7810109090 2024-08-31T06:10:44.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.557+0000 7f780d600640 1 -- 172.21.15.16:0/3792487991 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f77f8003f50 con 0x7f7810109090 2024-08-31T06:10:44.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.558+0000 7f780d600640 1 --2- 172.21.15.16:0/3792487991 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f77f40b4ca0 0x7f77f40b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:44.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.559+0000 7f780d600640 1 -- 172.21.15.16:0/3792487991 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f77f80e4fc0 con 0x7f7810109090 2024-08-31T06:10:44.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.559+0000 7f780fe00640 1 --2- 172.21.15.16:0/3792487991 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f77f40b4ca0 0x7f77f40b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:44.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.559+0000 7f780fe00640 1 --2- 172.21.15.16:0/3792487991 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f77f40b4ca0 0x7f77f40b71b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7800009230 tx=0x7f78000058d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:44.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.560+0000 7f780d600640 1 -- 172.21.15.16:0/3792487991 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f77f8017020 con 0x7f7810109090 2024-08-31T06:10:44.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:44 smithi016 ceph-mon[17194]: pgmap v469: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.3 MiB/s wr, 463 op/s 2024-08-31T06:10:44.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.792+0000 7f7815600640 1 -- 172.21.15.16:0/3792487991 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f781002e430 con 0x7f7810109090 2024-08-31T06:10:44.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.793+0000 7f780d600640 1 -- 172.21.15.16:0/3792487991 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11921 (secure 0 0 0) 0x7f77f80cce20 con 0x7f7810109090 2024-08-31T06:10:44.796 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:10:44.796 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":32,"btime":"2024-08-31T06:10:42:535148+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:10:42.006856+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24712,24724,24737]},"id":1}]} 2024-08-31T06:10:44.797 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 32 2024-08-31T06:10:44.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.796+0000 7f7815600640 1 -- 172.21.15.16:0/3792487991 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f77f40b4ca0 msgr2=0x7f77f40b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:44.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.796+0000 7f7815600640 1 --2- 172.21.15.16:0/3792487991 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f77f40b4ca0 0x7f77f40b71b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7800009230 tx=0x7f78000058d0 comp rx=0 tx=0).stop 2024-08-31T06:10:44.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.796+0000 7f7815600640 1 -- 172.21.15.16:0/3792487991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7810109090 msgr2=0x7f78100754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:44.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.796+0000 7f7815600640 1 --2- 172.21.15.16:0/3792487991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7810109090 0x7f78100754b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f77f8005fd0 tx=0x7f77f800dde0 comp rx=0 tx=0).stop 2024-08-31T06:10:44.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.796+0000 7f7815600640 1 -- 172.21.15.16:0/3792487991 shutdown_connections 2024-08-31T06:10:44.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.796+0000 7f7815600640 1 --2- 172.21.15.16:0/3792487991 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f77f40b4ca0 0x7f77f40b71b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:44.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.796+0000 7f7815600640 1 --2- 172.21.15.16:0/3792487991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7810109090 0x7f78100754b0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:44.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.796+0000 7f7815600640 1 --2- 172.21.15.16:0/3792487991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78101099e0 0x7f78100759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:44.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.796+0000 7f7815600640 1 --2- 172.21.15.16:0/3792487991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7810103750 0x7f7810076ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:44.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.796+0000 7f7815600640 1 -- 172.21.15.16:0/3792487991 >> 172.21.15.16:0/3792487991 conn(0x7f78100ff1e0 msgr2=0x7f7810101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:44.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.796+0000 7f7815600640 1 -- 172.21.15.16:0/3792487991 shutdown_connections 2024-08-31T06:10:44.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:44.796+0000 7f7815600640 1 -- 172.21.15.16:0/3792487991 wait complete. 2024-08-31T06:10:44.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:44 smithi155 ceph-mon[21742]: pgmap v469: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.3 MiB/s wr, 463 op/s 2024-08-31T06:10:44.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:44 smithi184 ceph-mon[22331]: pgmap v469: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.3 MiB/s wr, 463 op/s 2024-08-31T06:10:45.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:45 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/3792487991' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:10:45.666 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T06:10:45.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:45 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/3792487991' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:10:45.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:45 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/3792487991' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:10:46.046 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:10:46.629 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 200.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-31T06:10:46.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.720+0000 7f7b73200640 1 -- 172.21.15.16:0/3944839112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b74103730 msgr2=0x7f7b74103b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:46.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.720+0000 7f7b73200640 1 --2- 172.21.15.16:0/3944839112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b74103730 0x7f7b74103b40 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009900 tx=0x7f7b6402a3f0 comp rx=0 tx=0).stop 2024-08-31T06:10:46.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.721+0000 7f7b73200640 1 -- 172.21.15.16:0/3944839112 shutdown_connections 2024-08-31T06:10:46.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.721+0000 7f7b73200640 1 --2- 172.21.15.16:0/3944839112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b741099c0 0x7f7b741101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:46.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.721+0000 7f7b73200640 1 --2- 172.21.15.16:0/3944839112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b74109070 0x7f7b74109480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:46.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.721+0000 7f7b73200640 1 --2- 172.21.15.16:0/3944839112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b74103730 0x7f7b74103b40 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:46.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.721+0000 7f7b73200640 1 -- 172.21.15.16:0/3944839112 >> 172.21.15.16:0/3944839112 conn(0x7f7b740ff180 msgr2=0x7f7b741015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:46.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.721+0000 7f7b73200640 1 -- 172.21.15.16:0/3944839112 shutdown_connections 2024-08-31T06:10:46.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.721+0000 7f7b73200640 1 -- 172.21.15.16:0/3944839112 wait complete. 2024-08-31T06:10:46.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.722+0000 7f7b73200640 1 Processor -- start 2024-08-31T06:10:46.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.722+0000 7f7b73200640 1 -- start start 2024-08-31T06:10:46.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.722+0000 7f7b73200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b74103730 0x7f7b7419c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:46.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.722+0000 7f7b73200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b74109070 0x7f7b7419c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:46.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.722+0000 7f7b73200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b741099c0 0x7f7b741a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:46.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.722+0000 7f7b73200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7b74113670 con 0x7f7b74103730 2024-08-31T06:10:46.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.722+0000 7f7b73200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7b741134f0 con 0x7f7b741099c0 2024-08-31T06:10:46.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.722+0000 7f7b73200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7b741137f0 con 0x7f7b74109070 2024-08-31T06:10:46.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.723+0000 7f7b71e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b74103730 0x7f7b7419c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:46.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.723+0000 7f7b6be00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b74109070 0x7f7b7419c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:46.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.723+0000 7f7b72800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b741099c0 0x7f7b741a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:46.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.723+0000 7f7b6be00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b74109070 0x7f7b7419c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:49320/0 (socket says 172.21.15.16:49320) 2024-08-31T06:10:46.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.723+0000 7f7b71e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b74103730 0x7f7b7419c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36948/0 (socket says 172.21.15.16:36948) 2024-08-31T06:10:46.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.723+0000 7f7b6be00640 1 -- 172.21.15.16:0/446477910 learned_addr learned my addr 172.21.15.16:0/446477910 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:46.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.723+0000 7f7b72800640 1 -- 172.21.15.16:0/446477910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b74109070 msgr2=0x7f7b7419c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:46.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.723+0000 7f7b72800640 1 --2- 172.21.15.16:0/446477910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b74109070 0x7f7b7419c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:46.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.723+0000 7f7b72800640 1 -- 172.21.15.16:0/446477910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b74103730 msgr2=0x7f7b7419c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:46.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.723+0000 7f7b72800640 1 --2- 172.21.15.16:0/446477910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b74103730 0x7f7b7419c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:46.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.723+0000 7f7b72800640 1 -- 172.21.15.16:0/446477910 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b741a2250 con 0x7f7b741099c0 2024-08-31T06:10:46.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.723+0000 7f7b71e00640 1 --2- 172.21.15.16:0/446477910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b74103730 0x7f7b7419c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:10:46.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.724+0000 7f7b72800640 1 --2- 172.21.15.16:0/446477910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b741099c0 0x7f7b741a1b20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c00da10 tx=0x7f7b6c00ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:46.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.724+0000 7f7b6aa00640 1 -- 172.21.15.16:0/446477910 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c00bd70 con 0x7f7b741099c0 2024-08-31T06:10:46.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.724+0000 7f7b73200640 1 -- 172.21.15.16:0/446477910 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b741a2540 con 0x7f7b741099c0 2024-08-31T06:10:46.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.724+0000 7f7b6aa00640 1 -- 172.21.15.16:0/446477910 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b6c003e20 con 0x7f7b741099c0 2024-08-31T06:10:46.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.724+0000 7f7b6aa00640 1 -- 172.21.15.16:0/446477910 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c0106c0 con 0x7f7b741099c0 2024-08-31T06:10:46.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.724+0000 7f7b73200640 1 -- 172.21.15.16:0/446477910 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b741a8ca0 con 0x7f7b741099c0 2024-08-31T06:10:46.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.727+0000 7f7b7fe5b740 1 Processor -- start 2024-08-31T06:10:46.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.728+0000 7f7b6aa00640 1 -- 172.21.15.16:0/446477910 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f7b6c004270 con 0x7f7b741099c0 2024-08-31T06:10:46.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.728+0000 7f7b7fe5b740 1 -- start start 2024-08-31T06:10:46.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.728+0000 7f7b7fe5b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b12fce070 0x557b13373b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:46.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.728+0000 7f7b7fe5b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b12fa51b0 0x557b133740a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:46.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.728+0000 7f7b7fe5b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b12fad390 0x557b133745e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:46.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.728+0000 7f7b7fe5b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557b132ea7b0 con 0x557b12fa51b0 2024-08-31T06:10:46.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.728+0000 7f7b7fe5b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557b132d9d40 con 0x557b12fad390 2024-08-31T06:10:46.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.728+0000 7f7b7fe5b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557b132ddcf0 con 0x557b12fce070 2024-08-31T06:10:46.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.729+0000 7f7b71e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b12fa51b0 0x557b133740a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:46.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.729+0000 7f7b6be00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b12fce070 0x557b13373b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:46.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.729+0000 7f7b71e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b12fa51b0 0x557b133740a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36960/0 (socket says 172.21.15.16:36960) 2024-08-31T06:10:46.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.729+0000 7f7b71e00640 1 -- 172.21.15.16:0/641011428 learned_addr learned my addr 172.21.15.16:0/641011428 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:46.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.729+0000 7f7b6be00640 1 --2- 172.21.15.16:0/641011428 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b12fad390 0x557b133745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:46.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.729+0000 7f7b71e00640 1 -- 172.21.15.16:0/641011428 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b12fce070 msgr2=0x557b13373b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:46.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.729+0000 7f7b71e00640 1 --2- 172.21.15.16:0/641011428 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b12fce070 0x557b13373b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:46.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.729+0000 7f7b71e00640 1 -- 172.21.15.16:0/641011428 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b12fad390 msgr2=0x557b133745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:46.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.729+0000 7f7b71e00640 1 --2- 172.21.15.16:0/641011428 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b12fad390 0x557b133745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:46.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.729+0000 7f7b71e00640 1 -- 172.21.15.16:0/641011428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b13039650 con 0x557b12fa51b0 2024-08-31T06:10:46.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.730+0000 7f7b71e00640 1 --2- 172.21.15.16:0/641011428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b12fa51b0 0x557b133740a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009900 tx=0x7f7b64005ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=503cc354c9864e79 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:46.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.730+0000 7f7b6aa00640 1 --2- 172.21.15.16:0/446477910 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7b580b4ca0 0x7f7b580b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:46.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.730+0000 7f7b63400640 1 -- 172.21.15.16:0/641011428 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b64040070 con 0x557b12fa51b0 2024-08-31T06:10:46.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.730+0000 7f7b6aa00640 1 -- 172.21.15.16:0/446477910 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f7b6c0e5910 con 0x7f7b741099c0 2024-08-31T06:10:46.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.730+0000 7f7b63400640 1 -- 172.21.15.16:0/641011428 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b64031820 con 0x557b12fa51b0 2024-08-31T06:10:46.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.730+0000 7f7b63400640 1 -- 172.21.15.16:0/641011428 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b64035040 con 0x557b12fa51b0 2024-08-31T06:10:46.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.730+0000 7f7b6be00640 1 --2- 172.21.15.16:0/446477910 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7b580b4ca0 0x7f7b580b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:46.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.731+0000 7f7b7fe5b740 1 -- 172.21.15.16:0/641011428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b12fa51b0 msgr2=0x557b133740a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:46.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.731+0000 7f7b7fe5b740 1 --2- 172.21.15.16:0/641011428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b12fa51b0 0x557b133740a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009900 tx=0x7f7b64005ed0 comp rx=0 tx=0).stop 2024-08-31T06:10:46.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.731+0000 7f7b6be00640 1 --2- 172.21.15.16:0/446477910 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7b580b4ca0 0x7f7b580b71b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c009fd0 tx=0x7f7b5c004730 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:46.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.731+0000 7f7b7fe5b740 1 -- 172.21.15.16:0/641011428 shutdown_connections 2024-08-31T06:10:46.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.731+0000 7f7b7fe5b740 1 --2- 172.21.15.16:0/641011428 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b12fad390 0x557b133745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:46.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.731+0000 7f7b7fe5b740 1 --2- 172.21.15.16:0/641011428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b12fa51b0 0x557b133740a0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:46.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.731+0000 7f7b7fe5b740 1 --2- 172.21.15.16:0/641011428 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b12fce070 0x557b13373b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:46.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.731+0000 7f7b7fe5b740 1 -- 172.21.15.16:0/641011428 >> 172.21.15.16:0/641011428 conn(0x557b12fc2460 msgr2=0x557b13371a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:46.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.731+0000 7f7b7fe5b740 1 -- 172.21.15.16:0/641011428 shutdown_connections 2024-08-31T06:10:46.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.731+0000 7f7b7fe5b740 1 -- 172.21.15.16:0/641011428 wait complete. 2024-08-31T06:10:46.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.732+0000 7f7b7fe5b740 1 -- start start 2024-08-31T06:10:46.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.732+0000 7f7b7fe5b740 1 Processor -- start 2024-08-31T06:10:46.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.733+0000 7f7b7fe5b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b12fa51b0 0x557b1337f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:46.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.733+0000 7f7b7fe5b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b12fad390 0x557b1337fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:46.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.733+0000 7f7b7fe5b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b12fce070 0x557b1338b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:46.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.733+0000 7f7b7fe5b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557b1313c700 con 0x557b12fa51b0 2024-08-31T06:10:46.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.733+0000 7f7b7fe5b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557b13260cf0 con 0x557b12fce070 2024-08-31T06:10:46.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.733+0000 7f7b7fe5b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557b133822e0 con 0x557b12fad390 2024-08-31T06:10:46.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.733+0000 7f7b7fe5b740 10 client.0 _ll_register_callbacks cb 0x557b13286470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:10:46.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.733+0000 7f7b72800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b12fce070 0x557b1338b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:46.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.733+0000 7f7b71e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b12fa51b0 0x557b1337f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:46.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.733+0000 7f7b72800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b12fce070 0x557b1338b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:36598/0 (socket says 172.21.15.16:36598) 2024-08-31T06:10:46.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.733+0000 7f7b72800640 1 -- 172.21.15.16:0/824927123 learned_addr learned my addr 172.21.15.16:0/824927123 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:46.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.733+0000 7f7b6be00640 1 --2- 172.21.15.16:0/824927123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b12fad390 0x557b1337fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:46.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.733+0000 7f7b72800640 1 -- 172.21.15.16:0/824927123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b12fad390 msgr2=0x557b1337fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:46.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.733+0000 7f7b72800640 1 --2- 172.21.15.16:0/824927123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b12fad390 0x557b1337fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:46.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.733+0000 7f7b72800640 1 -- 172.21.15.16:0/824927123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b12fa51b0 msgr2=0x557b1337f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:46.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.733+0000 7f7b72800640 1 --2- 172.21.15.16:0/824927123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b12fa51b0 0x557b1337f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:46.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.733+0000 7f7b72800640 1 -- 172.21.15.16:0/824927123 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b132a18b0 con 0x557b12fce070 2024-08-31T06:10:46.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.734+0000 7f7b72800640 1 --2- 172.21.15.16:0/824927123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b12fce070 0x557b1338b600 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c0abf90 tx=0x7f7b6c0abc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6663fed6c48c19f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:46.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.734+0000 7f7b57e00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:10:46.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.734+0000 7f7b57e00640 1 -- 172.21.15.16:0/824927123 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c0b5320 con 0x557b12fce070 2024-08-31T06:10:46.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.734+0000 7f7b7fe5b740 1 -- 172.21.15.16:0/824927123 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b64031820 con 0x557b12fce070 2024-08-31T06:10:46.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.734+0000 7f7b57e00640 1 -- 172.21.15.16:0/824927123 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f7b6c0e32e0 con 0x557b12fce070 2024-08-31T06:10:46.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.734+0000 7f7b57e00640 1 -- 172.21.15.16:0/824927123 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7b6c0a8290 con 0x557b12fce070 2024-08-31T06:10:46.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.734+0000 7f7b7fe5b740 10 client.26378 fetch_fsmap learned FSMap version 32 2024-08-31T06:10:46.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.734+0000 7f7b7fe5b740 1 -- 172.21.15.16:0/824927123 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557b1338e3e0 con 0x557b12fce070 2024-08-31T06:10:46.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.734+0000 7f7b57e00640 1 -- 172.21.15.16:0/824927123 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b6c0d1020 con 0x557b12fce070 2024-08-31T06:10:46.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.735+0000 7f7b57e00640 1 -- 172.21.15.16:0/824927123 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f7b6c024060 con 0x557b12fce070 2024-08-31T06:10:46.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.735+0000 7f7b7fe5b740 10 client.26378 fetch_fsmap finished waiting for FSMap version 32 2024-08-31T06:10:46.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.735+0000 7f7b7fe5b740 10 client.26378 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T06:10:46.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.735+0000 7f7b7fe5b740 20 client.26378 populate_metadata read hostname 'smithi016' 2024-08-31T06:10:46.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.735+0000 7f7b7fe5b740 1 --2- 172.21.15.16:0/824927123 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x557b12f8c9f0 0x557b13391900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:46.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.735+0000 7f7b7fe5b740 4 client.26378 mds_command: new command op to 24712 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:10:46.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.735+0000 7f7b7fe5b740 1 -- 172.21.15.16:0/824927123 --> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b132ec700 con 0x557b12f8c9f0 2024-08-31T06:10:46.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.735+0000 7f7b71e00640 1 --2- 172.21.15.16:0/824927123 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x557b12f8c9f0 0x557b13391900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:46.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.736+0000 7f7b71e00640 1 --2- 172.21.15.16:0/824927123 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x557b12f8c9f0 0x557b13391900 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=576c48ae2e7fae37 server_cookie=9456ffade5fe5600 in_seq=0 out_seq=0 2024-08-31T06:10:46.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.736+0000 7f7b57e00640 10 client.26378 ms_handle_connect on v2:172.21.15.184:6838/4011051479 2024-08-31T06:10:46.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.737+0000 7f7b57e00640 1 -- 172.21.15.16:0/824927123 <== mds.2 v2:172.21.15.184:6838/4011051479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557b132ec700 con 0x557b12f8c9f0 2024-08-31T06:10:46.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.737+0000 7f7b57e00640 10 client.26378 handle_command_reply: tid=0 2024-08-31T06:10:46.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.737+0000 7f7b7fe5b740 1 client.26378 shutdown 2024-08-31T06:10:46.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.737+0000 7f7b7fe5b740 1 -- 172.21.15.16:0/824927123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b12fce070 msgr2=0x557b1338b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:46.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.737+0000 7f7b7fe5b740 1 --2- 172.21.15.16:0/824927123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b12fce070 0x557b1338b600 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c0abf90 tx=0x7f7b6c0abc50 comp rx=0 tx=0).stop 2024-08-31T06:10:46.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.737+0000 7f7b7fe5b740 1 -- 172.21.15.16:0/824927123 shutdown_connections 2024-08-31T06:10:46.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.737+0000 7f7b7fe5b740 1 --2- 172.21.15.16:0/824927123 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x557b12f8c9f0 0x557b13391900 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:46.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.737+0000 7f7b7fe5b740 1 --2- 172.21.15.16:0/824927123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b12fad390 0x557b1337fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:46.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.737+0000 7f7b7fe5b740 1 --2- 172.21.15.16:0/824927123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b12fce070 0x557b1338b600 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:46.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.737+0000 7f7b7fe5b740 1 --2- 172.21.15.16:0/824927123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b12fa51b0 0x557b1337f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:46.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.737+0000 7f7b57e00640 0 client.26378 ms_handle_reset on v2:172.21.15.184:6838/4011051479 2024-08-31T06:10:46.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.737+0000 7f7b7fe5b740 1 -- 172.21.15.16:0/824927123 >> 172.21.15.16:0/824927123 conn(0x557b12fc2460 msgr2=0x557b13376cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:46.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.738+0000 7f7b7fe5b740 1 -- 172.21.15.16:0/824927123 shutdown_connections 2024-08-31T06:10:46.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.738+0000 7f7b7fe5b740 1 -- 172.21.15.16:0/824927123 wait complete. 2024-08-31T06:10:46.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.738+0000 7f7b7fe5b740 20 client.26378 trim_cache size 0 max 16384 2024-08-31T06:10:46.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.752+0000 7f7b7fe5b740 1 Processor -- start 2024-08-31T06:10:46.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.752+0000 7f7b7fe5b740 1 -- start start 2024-08-31T06:10:46.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.752+0000 7f7b7fe5b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b12fa51b0 0x557b13379380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:46.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.752+0000 7f7b7fe5b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b12fad390 0x557b13371000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:46.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.752+0000 7f7b7fe5b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b12fce070 0x557b13371540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:46.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.752+0000 7f7b7fe5b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7b741b3d60 con 0x557b12fce070 2024-08-31T06:10:46.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.752+0000 7f7b7fe5b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7b741b15a0 con 0x557b12fad390 2024-08-31T06:10:46.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.752+0000 7f7b7fe5b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7b741af3f0 con 0x557b12fa51b0 2024-08-31T06:10:46.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.752+0000 7f7b71e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b12fa51b0 0x557b13379380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:46.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.752+0000 7f7b72800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b12fce070 0x557b13371540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:46.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.753+0000 7f7b71e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b12fa51b0 0x557b13379380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:49364/0 (socket says 172.21.15.16:49364) 2024-08-31T06:10:46.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.753+0000 7f7b6be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b12fad390 0x557b13371000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:46.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.753+0000 7f7b72800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b12fce070 0x557b13371540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36980/0 (socket says 172.21.15.16:36980) 2024-08-31T06:10:46.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.753+0000 7f7b71e00640 1 -- 172.21.15.16:0/3790998724 learned_addr learned my addr 172.21.15.16:0/3790998724 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:46.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.753+0000 7f7b71e00640 1 -- 172.21.15.16:0/3790998724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b12fad390 msgr2=0x557b13371000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:46.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.753+0000 7f7b71e00640 1 --2- 172.21.15.16:0/3790998724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b12fad390 0x557b13371000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:46.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.753+0000 7f7b71e00640 1 -- 172.21.15.16:0/3790998724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b12fce070 msgr2=0x557b13371540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:46.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.753+0000 7f7b71e00640 1 --2- 172.21.15.16:0/3790998724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b12fce070 0x557b13371540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:46.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.753+0000 7f7b71e00640 1 -- 172.21.15.16:0/3790998724 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b13391cb0 con 0x557b12fa51b0 2024-08-31T06:10:46.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.753+0000 7f7b6be00640 1 --2- 172.21.15.16:0/3790998724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b12fad390 0x557b13371000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:10:46.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.753+0000 7f7b71e00640 1 --2- 172.21.15.16:0/3790998724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b12fa51b0 0x557b13379380 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7b64033000 tx=0x7f7b64005530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17c6754a6d636113 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:46.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.754+0000 7f7b62a00640 1 -- 172.21.15.16:0/3790998724 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b64035040 con 0x557b12fa51b0 2024-08-31T06:10:46.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.754+0000 7f7b62a00640 1 -- 172.21.15.16:0/3790998724 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b640035f0 con 0x557b12fa51b0 2024-08-31T06:10:46.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.754+0000 7f7b62a00640 1 -- 172.21.15.16:0/3790998724 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b64040040 con 0x557b12fa51b0 2024-08-31T06:10:46.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.754+0000 7f7b7fe5b740 1 -- 172.21.15.16:0/3790998724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b12fa51b0 msgr2=0x557b13379380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:46.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.754+0000 7f7b7fe5b740 1 --2- 172.21.15.16:0/3790998724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b12fa51b0 0x557b13379380 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7b64033000 tx=0x7f7b64005530 comp rx=0 tx=0).stop 2024-08-31T06:10:46.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.755+0000 7f7b7fe5b740 1 -- 172.21.15.16:0/3790998724 shutdown_connections 2024-08-31T06:10:46.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.755+0000 7f7b7fe5b740 1 --2- 172.21.15.16:0/3790998724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b12fce070 0x557b13371540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:46.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.755+0000 7f7b7fe5b740 1 --2- 172.21.15.16:0/3790998724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b12fad390 0x557b13371000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:46.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.755+0000 7f7b7fe5b740 1 --2- 172.21.15.16:0/3790998724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b12fa51b0 0x557b13379380 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:46.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.755+0000 7f7b7fe5b740 1 -- 172.21.15.16:0/3790998724 >> 172.21.15.16:0/3790998724 conn(0x557b12fc2460 msgr2=0x557b13378c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:46.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.755+0000 7f7b7fe5b740 1 -- 172.21.15.16:0/3790998724 shutdown_connections 2024-08-31T06:10:46.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.755+0000 7f7b7fe5b740 1 -- 172.21.15.16:0/3790998724 wait complete. 2024-08-31T06:10:46.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.755+0000 7f7b7fe5b740 1 -- start start 2024-08-31T06:10:46.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.755+0000 7f7b7fe5b740 1 Processor -- start 2024-08-31T06:10:46.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.756+0000 7f7b7fe5b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b12fa51b0 0x557b13387a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:46.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.756+0000 7f7b7fe5b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b12fad390 0x557b13383f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:46.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.756+0000 7f7b7fe5b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b12fce070 0x557b133844b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:46.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.756+0000 7f7b7fe5b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7b741afa60 con 0x557b12fad390 2024-08-31T06:10:46.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.756+0000 7f7b7fe5b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7b741b0ed0 con 0x557b12fce070 2024-08-31T06:10:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.756+0000 7f7b7fe5b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7b741b0130 con 0x557b12fa51b0 2024-08-31T06:10:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.756+0000 7f7b7fe5b740 10 client.0 _ll_register_callbacks cb 0x7f7b6c0ab770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:10:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.756+0000 7f7b6be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b12fad390 0x557b13383f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.756+0000 7f7b71e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b12fa51b0 0x557b13387a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.756+0000 7f7b71e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b12fa51b0 0x557b13387a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:49372/0 (socket says 172.21.15.16:49372) 2024-08-31T06:10:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.756+0000 7f7b71e00640 1 -- 172.21.15.16:0/3721431597 learned_addr learned my addr 172.21.15.16:0/3721431597 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.756+0000 7f7b72800640 1 --2- 172.21.15.16:0/3721431597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b12fce070 0x557b133844b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.757+0000 7f7b71e00640 1 -- 172.21.15.16:0/3721431597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b12fce070 msgr2=0x557b133844b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.757+0000 7f7b71e00640 1 --2- 172.21.15.16:0/3721431597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b12fce070 0x557b133844b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.757+0000 7f7b71e00640 1 -- 172.21.15.16:0/3721431597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b12fad390 msgr2=0x557b13383f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.757+0000 7f7b71e00640 1 --2- 172.21.15.16:0/3721431597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b12fad390 0x557b13383f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.757+0000 7f7b71e00640 1 -- 172.21.15.16:0/3721431597 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b133849f0 con 0x557b12fa51b0 2024-08-31T06:10:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.757+0000 7f7b72800640 1 --2- 172.21.15.16:0/3721431597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b12fce070 0x557b133844b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:10:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.757+0000 7f7b71e00640 1 --2- 172.21.15.16:0/3721431597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b12fa51b0 0x557b13387a60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009800 tx=0x7f7b64005490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8be4302cb49f0d66 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.757+0000 7f7b57e00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:10:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.758+0000 7f7b57e00640 1 -- 172.21.15.16:0/3721431597 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b64040070 con 0x557b12fa51b0 2024-08-31T06:10:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.758+0000 7f7b7fe5b740 1 -- 172.21.15.16:0/3721431597 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b640035f0 con 0x557b12fa51b0 2024-08-31T06:10:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.758+0000 7f7b57e00640 1 -- 172.21.15.16:0/3721431597 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f7b640037a0 con 0x557b12fa51b0 2024-08-31T06:10:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.758+0000 7f7b57e00640 1 -- 172.21.15.16:0/3721431597 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7b64067050 con 0x557b12fa51b0 2024-08-31T06:10:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.758+0000 7f7b7fe5b740 10 client.26377 fetch_fsmap learned FSMap version 32 2024-08-31T06:10:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.758+0000 7f7b7fe5b740 1 -- 172.21.15.16:0/3721431597 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557b133fcf20 con 0x557b12fa51b0 2024-08-31T06:10:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.758+0000 7f7b57e00640 1 -- 172.21.15.16:0/3721431597 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b640035f0 con 0x557b12fa51b0 2024-08-31T06:10:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.758+0000 7f7b57e00640 1 -- 172.21.15.16:0/3721431597 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f7b64057b00 con 0x557b12fa51b0 2024-08-31T06:10:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.759+0000 7f7b7fe5b740 10 client.26377 fetch_fsmap finished waiting for FSMap version 32 2024-08-31T06:10:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.759+0000 7f7b7fe5b740 10 client.26377 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T06:10:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.759+0000 7f7b7fe5b740 20 client.26377 populate_metadata read hostname 'smithi016' 2024-08-31T06:10:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.759+0000 7f7b7fe5b740 1 --2- 172.21.15.16:0/3721431597 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x557b12f61da0 0x557b133ff1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:46.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.759+0000 7f7b7fe5b740 4 client.26377 mds_command: new command op to 24712 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:10:46.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.759+0000 7f7b7fe5b740 1 -- 172.21.15.16:0/3721431597 --> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557b132e81a0 con 0x557b12f61da0 2024-08-31T06:10:46.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.759+0000 7f7b6be00640 1 --2- 172.21.15.16:0/3721431597 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x557b12f61da0 0x557b133ff1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:46.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.759+0000 7f7b6be00640 1 --2- 172.21.15.16:0/3721431597 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x557b12f61da0 0x557b133ff1b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=facc8480a6f023af server_cookie=39829bebf58b9482 in_seq=0 out_seq=0 2024-08-31T06:10:46.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.759+0000 7f7b57e00640 10 client.26377 ms_handle_connect on v2:172.21.15.184:6838/4011051479 2024-08-31T06:10:46.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.760+0000 7f7b57e00640 1 -- 172.21.15.16:0/3721431597 <== mds.2 v2:172.21.15.184:6838/4011051479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557b132e81a0 con 0x557b12f61da0 2024-08-31T06:10:46.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.760+0000 7f7b57e00640 10 client.26377 handle_command_reply: tid=0 2024-08-31T06:10:46.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.760+0000 7f7b7fe5b740 1 client.26377 shutdown 2024-08-31T06:10:46.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.760+0000 7f7b7fe5b740 1 -- 172.21.15.16:0/3721431597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b12fa51b0 msgr2=0x557b13387a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:46.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.760+0000 7f7b7fe5b740 1 --2- 172.21.15.16:0/3721431597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b12fa51b0 0x557b13387a60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009800 tx=0x7f7b64005490 comp rx=0 tx=0).stop 2024-08-31T06:10:46.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.760+0000 7f7b7fe5b740 1 -- 172.21.15.16:0/3721431597 shutdown_connections 2024-08-31T06:10:46.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.760+0000 7f7b7fe5b740 1 --2- 172.21.15.16:0/3721431597 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x557b12f61da0 0x557b133ff1b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:46.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.760+0000 7f7b7fe5b740 1 --2- 172.21.15.16:0/3721431597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b12fce070 0x557b133844b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:46.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.760+0000 7f7b57e00640 0 client.26377 ms_handle_reset on v2:172.21.15.184:6838/4011051479 2024-08-31T06:10:46.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.760+0000 7f7b7fe5b740 1 --2- 172.21.15.16:0/3721431597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b12fad390 0x557b13383f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:46.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.760+0000 7f7b7fe5b740 1 --2- 172.21.15.16:0/3721431597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b12fa51b0 0x557b13387a60 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:46.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.760+0000 7f7b7fe5b740 1 -- 172.21.15.16:0/3721431597 >> 172.21.15.16:0/3721431597 conn(0x557b12fc2460 msgr2=0x557b13374b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:46.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.760+0000 7f7b7fe5b740 1 -- 172.21.15.16:0/3721431597 shutdown_connections 2024-08-31T06:10:46.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.760+0000 7f7b7fe5b740 1 -- 172.21.15.16:0/3721431597 wait complete. 2024-08-31T06:10:46.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.760+0000 7f7b7fe5b740 20 client.26377 trim_cache size 0 max 16384 2024-08-31T06:10:46.766 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:10:46.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.761+0000 7f7b73200640 1 -- 172.21.15.16:0/446477910 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7b580b4ca0 msgr2=0x7f7b580b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:46.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.761+0000 7f7b73200640 1 --2- 172.21.15.16:0/446477910 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7b580b4ca0 0x7f7b580b71b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c009fd0 tx=0x7f7b5c004730 comp rx=0 tx=0).stop 2024-08-31T06:10:46.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.761+0000 7f7b73200640 1 -- 172.21.15.16:0/446477910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b741099c0 msgr2=0x7f7b741a1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:46.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.761+0000 7f7b73200640 1 --2- 172.21.15.16:0/446477910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b741099c0 0x7f7b741a1b20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c00da10 tx=0x7f7b6c00ded0 comp rx=0 tx=0).stop 2024-08-31T06:10:46.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.761+0000 7f7b73200640 1 -- 172.21.15.16:0/446477910 shutdown_connections 2024-08-31T06:10:46.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.761+0000 7f7b73200640 1 --2- 172.21.15.16:0/446477910 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7b580b4ca0 0x7f7b580b71b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:46.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.761+0000 7f7b73200640 1 --2- 172.21.15.16:0/446477910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b74109070 0x7f7b7419c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:46.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.761+0000 7f7b73200640 1 --2- 172.21.15.16:0/446477910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b741099c0 0x7f7b741a1b20 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:46.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.761+0000 7f7b73200640 1 --2- 172.21.15.16:0/446477910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b74103730 0x7f7b7419c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:46.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.761+0000 7f7b73200640 1 -- 172.21.15.16:0/446477910 >> 172.21.15.16:0/446477910 conn(0x7f7b740ff180 msgr2=0x7f7b7410e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:46.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.761+0000 7f7b73200640 1 -- 172.21.15.16:0/446477910 shutdown_connections 2024-08-31T06:10:46.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:46.761+0000 7f7b73200640 1 -- 172.21.15.16:0/446477910 wait complete. 2024-08-31T06:10:46.828 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:46 smithi016 ceph-mon[17194]: pgmap v470: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 4.5 MiB/s wr, 496 op/s 2024-08-31T06:10:46.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:46 smithi155 ceph-mon[21742]: pgmap v470: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 4.5 MiB/s wr, 496 op/s 2024-08-31T06:10:46.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:46 smithi184 ceph-mon[22331]: pgmap v470: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 4.5 MiB/s wr, 496 op/s 2024-08-31T06:10:47.726 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T06:10:48.109 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:10:48.846 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:48 smithi016 ceph-mon[17194]: pgmap v471: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 3.7 MiB/s wr, 411 op/s 2024-08-31T06:10:48.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:48 smithi155 ceph-mon[21742]: pgmap v471: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 3.7 MiB/s wr, 411 op/s 2024-08-31T06:10:48.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.893+0000 7ff106800640 1 -- 172.21.15.16:0/2544352440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff100103750 msgr2=0x7ff100103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:48.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.893+0000 7ff106800640 1 --2- 172.21.15.16:0/2544352440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff100103750 0x7ff100103b60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4009ca0 tx=0x7ff0f402a8c0 comp rx=0 tx=0).stop 2024-08-31T06:10:48.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.893+0000 7ff106800640 1 -- 172.21.15.16:0/2544352440 shutdown_connections 2024-08-31T06:10:48.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.893+0000 7ff106800640 1 --2- 172.21.15.16:0/2544352440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1001099e0 0x7ff100110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:48.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.893+0000 7ff106800640 1 --2- 172.21.15.16:0/2544352440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff100109090 0x7ff1001094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:48.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.893+0000 7ff106800640 1 --2- 172.21.15.16:0/2544352440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff100103750 0x7ff100103b60 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:48.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.893+0000 7ff106800640 1 -- 172.21.15.16:0/2544352440 >> 172.21.15.16:0/2544352440 conn(0x7ff1000ff1e0 msgr2=0x7ff100101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:48.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.893+0000 7ff106800640 1 -- 172.21.15.16:0/2544352440 shutdown_connections 2024-08-31T06:10:48.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.893+0000 7ff106800640 1 -- 172.21.15.16:0/2544352440 wait complete. 2024-08-31T06:10:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.894+0000 7ff106800640 1 Processor -- start 2024-08-31T06:10:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.895+0000 7ff106800640 1 -- start start 2024-08-31T06:10:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.895+0000 7ff106800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff100103750 0x7ff1000754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.895+0000 7ff106800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff100109090 0x7ff1000759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.895+0000 7ff106800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1001099e0 0x7ff100075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.895+0000 7ff106800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff1001136e0 con 0x7ff1001099e0 2024-08-31T06:10:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.895+0000 7ff106800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff100113560 con 0x7ff100109090 2024-08-31T06:10:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.895+0000 7ff106800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff100113860 con 0x7ff100103750 2024-08-31T06:10:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.895+0000 7ff105e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1001099e0 0x7ff100075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.895+0000 7ff105400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff100103750 0x7ff1000754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.896+0000 7ff105e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1001099e0 0x7ff100075f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37010/0 (socket says 172.21.15.16:37010) 2024-08-31T06:10:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.896+0000 7ff105e00640 1 -- 172.21.15.16:0/4159896977 learned_addr learned my addr 172.21.15.16:0/4159896977 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.896+0000 7ff104a00640 1 --2- 172.21.15.16:0/4159896977 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff100109090 0x7ff1000759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.896+0000 7ff105e00640 1 -- 172.21.15.16:0/4159896977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff100103750 msgr2=0x7ff1000754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.896+0000 7ff105e00640 1 --2- 172.21.15.16:0/4159896977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff100103750 0x7ff1000754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.896+0000 7ff105e00640 1 -- 172.21.15.16:0/4159896977 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff100109090 msgr2=0x7ff1000759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.896+0000 7ff105e00640 1 --2- 172.21.15.16:0/4159896977 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff100109090 0x7ff1000759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.896+0000 7ff105e00640 1 -- 172.21.15.16:0/4159896977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1001a83b0 con 0x7ff1001099e0 2024-08-31T06:10:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.896+0000 7ff105400640 1 --2- 172.21.15.16:0/4159896977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff100103750 0x7ff1000754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:10:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.896+0000 7ff104a00640 1 --2- 172.21.15.16:0/4159896977 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff100109090 0x7ff1000759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:10:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.896+0000 7ff105e00640 1 --2- 172.21.15.16:0/4159896977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1001099e0 0x7ff100075f30 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff0f800d950 tx=0x7ff0f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.897+0000 7ff0fe000640 1 -- 172.21.15.16:0/4159896977 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8017030 con 0x7ff1001099e0 2024-08-31T06:10:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.897+0000 7ff0fe000640 1 -- 172.21.15.16:0/4159896977 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0f8004020 con 0x7ff1001099e0 2024-08-31T06:10:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.897+0000 7ff106800640 1 -- 172.21.15.16:0/4159896977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1001a86a0 con 0x7ff1001099e0 2024-08-31T06:10:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.897+0000 7ff0fe000640 1 -- 172.21.15.16:0/4159896977 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8003b50 con 0x7ff1001099e0 2024-08-31T06:10:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.897+0000 7ff106800640 1 -- 172.21.15.16:0/4159896977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1001a8be0 con 0x7ff1001099e0 2024-08-31T06:10:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.900+0000 7ff10ebe6740 1 Processor -- start 2024-08-31T06:10:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.900+0000 7ff0fe000640 1 -- 172.21.15.16:0/4159896977 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7ff0f801a070 con 0x7ff1001099e0 2024-08-31T06:10:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.901+0000 7ff10ebe6740 1 -- start start 2024-08-31T06:10:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.901+0000 7ff10ebe6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6a8741070 0x55f6a8ae6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:48.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.901+0000 7ff10ebe6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6a87181b0 0x55f6a8ae6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:48.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.901+0000 7ff10ebe6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6a8720390 0x55f6a8ae74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:48.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.901+0000 7ff10ebe6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f6a8a5d7b0 con 0x55f6a87181b0 2024-08-31T06:10:48.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.901+0000 7ff10ebe6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f6a8a4cd40 con 0x55f6a8741070 2024-08-31T06:10:48.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.901+0000 7ff10ebe6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f6a8a50cf0 con 0x55f6a8720390 2024-08-31T06:10:48.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.901+0000 7ff105400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6a87181b0 0x55f6a8ae6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:48.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.901+0000 7ff105400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6a87181b0 0x55f6a8ae6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37020/0 (socket says 172.21.15.16:37020) 2024-08-31T06:10:48.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.901+0000 7ff105400640 1 -- 172.21.15.16:0/1834625141 learned_addr learned my addr 172.21.15.16:0/1834625141 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:48.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.901+0000 7ff104a00640 1 --2- 172.21.15.16:0/1834625141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6a8741070 0x55f6a8ae6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:48.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.902+0000 7ff104a00640 1 --2- 172.21.15.16:0/1834625141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6a8720390 0x55f6a8ae74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:48.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.902+0000 7ff105400640 1 -- 172.21.15.16:0/1834625141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6a8720390 msgr2=0x55f6a8ae74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:48.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.902+0000 7ff105400640 1 --2- 172.21.15.16:0/1834625141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6a8720390 0x55f6a8ae74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:48.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.902+0000 7ff105400640 1 -- 172.21.15.16:0/1834625141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6a8741070 msgr2=0x55f6a8ae6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.902+0000 7ff105400640 1 --2- 172.21.15.16:0/1834625141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6a8741070 0x55f6a8ae6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.902+0000 7ff105400640 1 -- 172.21.15.16:0/1834625141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6a87ac650 con 0x55f6a87181b0 2024-08-31T06:10:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.902+0000 7ff104a00640 1 --2- 172.21.15.16:0/1834625141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6a8720390 0x55f6a8ae74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:10:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.902+0000 7ff105400640 1 --2- 172.21.15.16:0/1834625141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6a87181b0 0x55f6a8ae6fb0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff0f402ade0 tx=0x7ff0f4032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=204c99bd88d10db5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.902+0000 7ff0fe000640 1 --2- 172.21.15.16:0/4159896977 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff0e80b4d20 0x7ff0e80b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.903+0000 7ff0f2600640 1 -- 172.21.15.16:0/1834625141 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f4031650 con 0x55f6a87181b0 2024-08-31T06:10:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.903+0000 7ff0f2600640 1 -- 172.21.15.16:0/1834625141 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0f4036040 con 0x55f6a87181b0 2024-08-31T06:10:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.903+0000 7ff0f2600640 1 -- 172.21.15.16:0/1834625141 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f4004550 con 0x55f6a87181b0 2024-08-31T06:10:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.903+0000 7ff0fe000640 1 -- 172.21.15.16:0/4159896977 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7ff0f80e4c30 con 0x7ff1001099e0 2024-08-31T06:10:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.903+0000 7ff104a00640 1 --2- 172.21.15.16:0/4159896977 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff0e80b4d20 0x7ff0e80b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:48.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.904+0000 7ff10ebe6740 1 -- 172.21.15.16:0/1834625141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6a87181b0 msgr2=0x55f6a8ae6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:48.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.904+0000 7ff10ebe6740 1 --2- 172.21.15.16:0/1834625141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6a87181b0 0x55f6a8ae6fb0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff0f402ade0 tx=0x7ff0f4032ff0 comp rx=0 tx=0).stop 2024-08-31T06:10:48.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.904+0000 7ff104a00640 1 --2- 172.21.15.16:0/4159896977 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff0e80b4d20 0x7ff0e80b7230 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec0058c0 tx=0x7ff0ec005850 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:48.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.904+0000 7ff10ebe6740 1 -- 172.21.15.16:0/1834625141 shutdown_connections 2024-08-31T06:10:48.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.904+0000 7ff10ebe6740 1 --2- 172.21.15.16:0/1834625141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6a8720390 0x55f6a8ae74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:48.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.904+0000 7ff10ebe6740 1 --2- 172.21.15.16:0/1834625141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6a87181b0 0x55f6a8ae6fb0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:48.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.904+0000 7ff10ebe6740 1 --2- 172.21.15.16:0/1834625141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6a8741070 0x55f6a8ae6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:48.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.904+0000 7ff10ebe6740 1 -- 172.21.15.16:0/1834625141 >> 172.21.15.16:0/1834625141 conn(0x55f6a8735460 msgr2=0x55f6a8ae4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:48.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.904+0000 7ff10ebe6740 1 -- 172.21.15.16:0/1834625141 shutdown_connections 2024-08-31T06:10:48.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.904+0000 7ff10ebe6740 1 -- 172.21.15.16:0/1834625141 wait complete. 2024-08-31T06:10:48.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.905+0000 7ff10ebe6740 1 -- start start 2024-08-31T06:10:48.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.905+0000 7ff10ebe6740 1 Processor -- start 2024-08-31T06:10:48.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.905+0000 7ff10ebe6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6a87181b0 0x55f6a8af25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:48.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.905+0000 7ff10ebe6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6a8720390 0x55f6a8af2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:48.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.905+0000 7ff10ebe6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6a8741070 0x55f6a8afe3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:48.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.905+0000 7ff10ebe6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f6a88af700 con 0x55f6a8741070 2024-08-31T06:10:48.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.905+0000 7ff10ebe6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f6a89d3cf0 con 0x55f6a87181b0 2024-08-31T06:10:48.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.905+0000 7ff10ebe6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f6a8af5070 con 0x55f6a8720390 2024-08-31T06:10:48.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.906+0000 7ff10ebe6740 10 client.0 _ll_register_callbacks cb 0x55f6a89f9470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:10:48.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.906+0000 7ff105400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6a87181b0 0x55f6a8af25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:48.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.906+0000 7ff104a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6a8720390 0x55f6a8af2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:48.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.906+0000 7ff105e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6a8741070 0x55f6a8afe3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:48.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.906+0000 7ff105400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6a87181b0 0x55f6a8af25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:36634/0 (socket says 172.21.15.16:36634) 2024-08-31T06:10:48.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.906+0000 7ff105400640 1 -- 172.21.15.16:0/560928360 learned_addr learned my addr 172.21.15.16:0/560928360 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:48.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.906+0000 7ff104a00640 1 -- 172.21.15.16:0/560928360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6a87181b0 msgr2=0x55f6a8af25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:48.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.906+0000 7ff104a00640 1 --2- 172.21.15.16:0/560928360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6a87181b0 0x55f6a8af25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:48.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.906+0000 7ff104a00640 1 -- 172.21.15.16:0/560928360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6a8741070 msgr2=0x55f6a8afe3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:48.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.906+0000 7ff104a00640 1 --2- 172.21.15.16:0/560928360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6a8741070 0x55f6a8afe3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:48.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.906+0000 7ff104a00640 1 -- 172.21.15.16:0/560928360 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6a8a148b0 con 0x55f6a8720390 2024-08-31T06:10:48.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.906+0000 7ff105e00640 1 --2- 172.21.15.16:0/560928360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6a8741070 0x55f6a8afe3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:10:48.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.907+0000 7ff104a00640 1 --2- 172.21.15.16:0/560928360 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6a8720390 0x55f6a8af2ae0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff10006b4a0 tx=0x7ff0ec004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ecfde9de701e6fd server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:48.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.907+0000 7ff0e6a00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:10:48.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.907+0000 7ff0e6a00640 1 -- 172.21.15.16:0/560928360 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec02d060 con 0x55f6a8720390 2024-08-31T06:10:48.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.907+0000 7ff10ebe6740 1 -- 172.21.15.16:0/560928360 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0f4036040 con 0x55f6a8720390 2024-08-31T06:10:48.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.907+0000 7ff0e6a00640 1 -- 172.21.15.16:0/560928360 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7ff0ec01a030 con 0x55f6a8720390 2024-08-31T06:10:48.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.907+0000 7ff0e6a00640 1 -- 172.21.15.16:0/560928360 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff0ec01d5f0 con 0x55f6a8720390 2024-08-31T06:10:48.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.907+0000 7ff10ebe6740 10 client.26389 fetch_fsmap learned FSMap version 32 2024-08-31T06:10:48.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.907+0000 7ff10ebe6740 1 -- 172.21.15.16:0/560928360 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f6a8b010d0 con 0x55f6a8720390 2024-08-31T06:10:48.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.908+0000 7ff0e6a00640 1 -- 172.21.15.16:0/560928360 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0f4036040 con 0x55f6a8720390 2024-08-31T06:10:48.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.908+0000 7ff0e6a00640 1 -- 172.21.15.16:0/560928360 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7ff0ec032950 con 0x55f6a8720390 2024-08-31T06:10:48.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.908+0000 7ff10ebe6740 10 client.26389 fetch_fsmap finished waiting for FSMap version 32 2024-08-31T06:10:48.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.908+0000 7ff10ebe6740 10 client.26389 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-31T06:10:48.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.908+0000 7ff10ebe6740 20 client.26389 populate_metadata read hostname 'smithi016' 2024-08-31T06:10:48.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.908+0000 7ff10ebe6740 1 --2- 172.21.15.16:0/560928360 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55f6a86ff9f0 0x55f6a8b04610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:48.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.908+0000 7ff10ebe6740 4 client.26389 mds_command: new command op to 24724 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:10:48.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.908+0000 7ff10ebe6740 1 -- 172.21.15.16:0/560928360 --> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6a8a5f700 con 0x55f6a86ff9f0 2024-08-31T06:10:48.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.908+0000 7ff105400640 1 --2- 172.21.15.16:0/560928360 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55f6a86ff9f0 0x55f6a8b04610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:48.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.909+0000 7ff105400640 1 --2- 172.21.15.16:0/560928360 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55f6a86ff9f0 0x55f6a8b04610 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=132bd275f75d8d6d server_cookie=fdf73b28ecc0eb7e in_seq=0 out_seq=0 2024-08-31T06:10:48.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.909+0000 7ff0e6a00640 10 client.26389 ms_handle_connect on v2:172.21.15.16:6838/219664460 2024-08-31T06:10:48.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.910+0000 7ff0e6a00640 1 -- 172.21.15.16:0/560928360 <== mds.1 v2:172.21.15.16:6838/219664460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f6a8a5f700 con 0x55f6a86ff9f0 2024-08-31T06:10:48.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.910+0000 7ff0e6a00640 10 client.26389 handle_command_reply: tid=0 2024-08-31T06:10:48.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.910+0000 7ff10ebe6740 1 client.26389 shutdown 2024-08-31T06:10:48.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.910+0000 7ff10ebe6740 1 -- 172.21.15.16:0/560928360 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6a8720390 msgr2=0x55f6a8af2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:48.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.910+0000 7ff10ebe6740 1 --2- 172.21.15.16:0/560928360 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6a8720390 0x55f6a8af2ae0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff10006b4a0 tx=0x7ff0ec004730 comp rx=0 tx=0).stop 2024-08-31T06:10:48.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.910+0000 7ff10ebe6740 1 -- 172.21.15.16:0/560928360 shutdown_connections 2024-08-31T06:10:48.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.910+0000 7ff10ebe6740 1 --2- 172.21.15.16:0/560928360 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6a8720390 0x55f6a8af2ae0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:48.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.910+0000 7ff10ebe6740 1 --2- 172.21.15.16:0/560928360 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55f6a86ff9f0 0x55f6a8b04610 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:48.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.910+0000 7ff10ebe6740 1 --2- 172.21.15.16:0/560928360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6a8741070 0x55f6a8afe3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:48.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.910+0000 7ff10ebe6740 1 --2- 172.21.15.16:0/560928360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6a87181b0 0x55f6a8af25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:48.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.910+0000 7ff0e6a00640 0 client.26389 ms_handle_reset on v2:172.21.15.16:6838/219664460 2024-08-31T06:10:48.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.910+0000 7ff10ebe6740 1 -- 172.21.15.16:0/560928360 >> 172.21.15.16:0/560928360 conn(0x55f6a8735460 msgr2=0x55f6a8ae9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:48.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.910+0000 7ff10ebe6740 1 -- 172.21.15.16:0/560928360 shutdown_connections 2024-08-31T06:10:48.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.910+0000 7ff10ebe6740 1 -- 172.21.15.16:0/560928360 wait complete. 2024-08-31T06:10:48.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.911+0000 7ff10ebe6740 20 client.26389 trim_cache size 0 max 16384 2024-08-31T06:10:48.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.925+0000 7ff10ebe6740 1 Processor -- start 2024-08-31T06:10:48.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.925+0000 7ff10ebe6740 1 -- start start 2024-08-31T06:10:48.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.925+0000 7ff10ebe6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6a87181b0 0x55f6a8af1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:48.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.925+0000 7ff10ebe6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6a8720390 0x55f6a8ae3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:48.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.925+0000 7ff10ebe6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6a8741070 0x55f6a8afe2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:48.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.925+0000 7ff10ebe6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff1001b3d00 con 0x55f6a87181b0 2024-08-31T06:10:48.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.925+0000 7ff10ebe6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff1001b1540 con 0x55f6a8720390 2024-08-31T06:10:48.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.925+0000 7ff10ebe6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff1001af390 con 0x55f6a8741070 2024-08-31T06:10:48.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.925+0000 7ff105400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6a87181b0 0x55f6a8af1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:48.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.925+0000 7ff105400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6a87181b0 0x55f6a8af1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37034/0 (socket says 172.21.15.16:37034) 2024-08-31T06:10:48.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.925+0000 7ff105400640 1 -- 172.21.15.16:0/3548715996 learned_addr learned my addr 172.21.15.16:0/3548715996 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:48.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.925+0000 7ff105e00640 1 --2- 172.21.15.16:0/3548715996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6a8741070 0x55f6a8afe2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:48.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.926+0000 7ff104a00640 1 --2- 172.21.15.16:0/3548715996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6a8720390 0x55f6a8ae3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:48.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.926+0000 7ff105400640 1 -- 172.21.15.16:0/3548715996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6a8741070 msgr2=0x55f6a8afe2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:48.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.926+0000 7ff105400640 1 --2- 172.21.15.16:0/3548715996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6a8741070 0x55f6a8afe2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:48.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.926+0000 7ff105400640 1 -- 172.21.15.16:0/3548715996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6a8720390 msgr2=0x55f6a8ae3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:48.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.926+0000 7ff105400640 1 --2- 172.21.15.16:0/3548715996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6a8720390 0x55f6a8ae3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:48.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.926+0000 7ff105400640 1 -- 172.21.15.16:0/3548715996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6a8b00ee0 con 0x55f6a87181b0 2024-08-31T06:10:48.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.926+0000 7ff105e00640 1 --2- 172.21.15.16:0/3548715996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6a8741070 0x55f6a8afe2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:10:48.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.926+0000 7ff104a00640 1 --2- 172.21.15.16:0/3548715996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6a8720390 0x55f6a8ae3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:10:48.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.926+0000 7ff105400640 1 --2- 172.21.15.16:0/3548715996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6a87181b0 0x55f6a8af1290 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4000f80 tx=0x7ff0f4004230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b46ef26aee219a33 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:48.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.926+0000 7ff0f1c00640 1 -- 172.21.15.16:0/3548715996 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f405f070 con 0x55f6a87181b0 2024-08-31T06:10:48.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.927+0000 7ff0f1c00640 1 -- 172.21.15.16:0/3548715996 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0f4036070 con 0x55f6a87181b0 2024-08-31T06:10:48.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.927+0000 7ff0f1c00640 1 -- 172.21.15.16:0/3548715996 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f4003970 con 0x55f6a87181b0 2024-08-31T06:10:48.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.927+0000 7ff10ebe6740 1 -- 172.21.15.16:0/3548715996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6a87181b0 msgr2=0x55f6a8af1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:48.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.927+0000 7ff10ebe6740 1 --2- 172.21.15.16:0/3548715996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6a87181b0 0x55f6a8af1290 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4000f80 tx=0x7ff0f4004230 comp rx=0 tx=0).stop 2024-08-31T06:10:48.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.928+0000 7ff10ebe6740 1 -- 172.21.15.16:0/3548715996 shutdown_connections 2024-08-31T06:10:48.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.928+0000 7ff10ebe6740 1 --2- 172.21.15.16:0/3548715996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6a8741070 0x55f6a8afe2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:48.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.928+0000 7ff10ebe6740 1 --2- 172.21.15.16:0/3548715996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6a8720390 0x55f6a8ae3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:48.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.928+0000 7ff10ebe6740 1 --2- 172.21.15.16:0/3548715996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6a87181b0 0x55f6a8af1290 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:48.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.928+0000 7ff10ebe6740 1 -- 172.21.15.16:0/3548715996 >> 172.21.15.16:0/3548715996 conn(0x55f6a8735460 msgr2=0x55f6a8ae2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:48.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.928+0000 7ff10ebe6740 1 -- 172.21.15.16:0/3548715996 shutdown_connections 2024-08-31T06:10:48.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.928+0000 7ff10ebe6740 1 -- 172.21.15.16:0/3548715996 wait complete. 2024-08-31T06:10:48.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.928+0000 7ff10ebe6740 1 -- start start 2024-08-31T06:10:48.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.928+0000 7ff10ebe6740 1 Processor -- start 2024-08-31T06:10:48.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.929+0000 7ff10ebe6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6a87181b0 0x55f6a8afa360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:48.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.929+0000 7ff10ebe6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6a8720390 0x55f6a8afa900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:48.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.929+0000 7ff10ebe6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6a8741070 0x55f6a8af5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:48.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.929+0000 7ff10ebe6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff1001afa00 con 0x55f6a8720390 2024-08-31T06:10:48.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.929+0000 7ff10ebe6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff1001b0e70 con 0x55f6a87181b0 2024-08-31T06:10:48.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.929+0000 7ff10ebe6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff1001b00d0 con 0x55f6a8741070 2024-08-31T06:10:48.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.929+0000 7ff10ebe6740 10 client.0 _ll_register_callbacks cb 0x7ff0ec00fde0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:10:48.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.929+0000 7ff104a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6a8720390 0x55f6a8afa900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:48.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.929+0000 7ff105400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6a87181b0 0x55f6a8afa360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:48.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.929+0000 7ff105e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6a8741070 0x55f6a8af5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:48.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.929+0000 7ff104a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6a8720390 0x55f6a8afa900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37040/0 (socket says 172.21.15.16:37040) 2024-08-31T06:10:48.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.929+0000 7ff105400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6a87181b0 0x55f6a8afa360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:36656/0 (socket says 172.21.15.16:36656) 2024-08-31T06:10:48.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.929+0000 7ff104a00640 1 -- 172.21.15.16:0/2104952793 learned_addr learned my addr 172.21.15.16:0/2104952793 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:48.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.930+0000 7ff105400640 1 -- 172.21.15.16:0/2104952793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6a8741070 msgr2=0x55f6a8af5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:48.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.930+0000 7ff105400640 1 --2- 172.21.15.16:0/2104952793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6a8741070 0x55f6a8af5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:48.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.930+0000 7ff105400640 1 -- 172.21.15.16:0/2104952793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6a8720390 msgr2=0x55f6a8afa900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:48.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.930+0000 7ff105400640 1 --2- 172.21.15.16:0/2104952793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6a8720390 0x55f6a8afa900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:48.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.930+0000 7ff105400640 1 -- 172.21.15.16:0/2104952793 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6a8af5850 con 0x55f6a87181b0 2024-08-31T06:10:48.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.930+0000 7ff104a00640 1 --2- 172.21.15.16:0/2104952793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6a8720390 0x55f6a8afa900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:10:48.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.930+0000 7ff105400640 1 --2- 172.21.15.16:0/2104952793 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6a87181b0 0x55f6a8afa360 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4009620 tx=0x7ff0f40099a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb3bdb53a08ce410 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:48.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.930+0000 7ff0e6a00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:10:48.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.931+0000 7ff0e6a00640 1 -- 172.21.15.16:0/2104952793 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f405f040 con 0x55f6a87181b0 2024-08-31T06:10:48.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.931+0000 7ff10ebe6740 1 -- 172.21.15.16:0/2104952793 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0f4036070 con 0x55f6a87181b0 2024-08-31T06:10:48.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.931+0000 7ff0e6a00640 1 -- 172.21.15.16:0/2104952793 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7ff0f40044a0 con 0x55f6a87181b0 2024-08-31T06:10:48.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.931+0000 7ff0e6a00640 1 -- 172.21.15.16:0/2104952793 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff0f404b040 con 0x55f6a87181b0 2024-08-31T06:10:48.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.931+0000 7ff10ebe6740 10 client.26402 fetch_fsmap learned FSMap version 32 2024-08-31T06:10:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.931+0000 7ff10ebe6740 1 -- 172.21.15.16:0/2104952793 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f6a8b6f730 con 0x55f6a87181b0 2024-08-31T06:10:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.931+0000 7ff0e6a00640 1 -- 172.21.15.16:0/2104952793 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0f4036070 con 0x55f6a87181b0 2024-08-31T06:10:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.931+0000 7ff0e6a00640 1 -- 172.21.15.16:0/2104952793 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7ff0f4044070 con 0x55f6a87181b0 2024-08-31T06:10:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.932+0000 7ff10ebe6740 10 client.26402 fetch_fsmap finished waiting for FSMap version 32 2024-08-31T06:10:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.932+0000 7ff10ebe6740 10 client.26402 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-31T06:10:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.932+0000 7ff10ebe6740 20 client.26402 populate_metadata read hostname 'smithi016' 2024-08-31T06:10:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.932+0000 7ff10ebe6740 1 --2- 172.21.15.16:0/2104952793 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55f6a86d4da0 0x55f6a8b71ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.932+0000 7ff10ebe6740 4 client.26402 mds_command: new command op to 24724 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:10:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.932+0000 7ff10ebe6740 1 -- 172.21.15.16:0/2104952793 --> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f6a8a5b1a0 con 0x55f6a86d4da0 2024-08-31T06:10:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.932+0000 7ff104a00640 1 --2- 172.21.15.16:0/2104952793 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55f6a86d4da0 0x55f6a8b71ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.932+0000 7ff104a00640 1 --2- 172.21.15.16:0/2104952793 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55f6a86d4da0 0x55f6a8b71ad0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1a386d28f42be566 server_cookie=a077c3332b843788 in_seq=0 out_seq=0 2024-08-31T06:10:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.932+0000 7ff0e6a00640 10 client.26402 ms_handle_connect on v2:172.21.15.16:6838/219664460 2024-08-31T06:10:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.933+0000 7ff0e6a00640 1 -- 172.21.15.16:0/2104952793 <== mds.1 v2:172.21.15.16:6838/219664460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f6a8a5b1a0 con 0x55f6a86d4da0 2024-08-31T06:10:48.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.933+0000 7ff0e6a00640 10 client.26402 handle_command_reply: tid=0 2024-08-31T06:10:48.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.933+0000 7ff10ebe6740 1 client.26402 shutdown 2024-08-31T06:10:48.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.933+0000 7ff10ebe6740 1 -- 172.21.15.16:0/2104952793 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6a87181b0 msgr2=0x55f6a8afa360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:48.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.933+0000 7ff10ebe6740 1 --2- 172.21.15.16:0/2104952793 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6a87181b0 0x55f6a8afa360 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4009620 tx=0x7ff0f40099a0 comp rx=0 tx=0).stop 2024-08-31T06:10:48.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.933+0000 7ff10ebe6740 1 -- 172.21.15.16:0/2104952793 shutdown_connections 2024-08-31T06:10:48.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.933+0000 7ff10ebe6740 1 --2- 172.21.15.16:0/2104952793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f6a8741070 0x55f6a8af5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:48.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.933+0000 7ff10ebe6740 1 --2- 172.21.15.16:0/2104952793 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55f6a86d4da0 0x55f6a8b71ad0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:48.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.933+0000 7ff10ebe6740 1 --2- 172.21.15.16:0/2104952793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6a8720390 0x55f6a8afa900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:48.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.933+0000 7ff10ebe6740 1 --2- 172.21.15.16:0/2104952793 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6a87181b0 0x55f6a8afa360 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:48.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.933+0000 7ff0e6a00640 0 client.26402 ms_handle_reset on v2:172.21.15.16:6838/219664460 2024-08-31T06:10:48.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.933+0000 7ff10ebe6740 1 -- 172.21.15.16:0/2104952793 >> 172.21.15.16:0/2104952793 conn(0x55f6a8735460 msgr2=0x55f6a8ae7aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:48.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.933+0000 7ff10ebe6740 1 -- 172.21.15.16:0/2104952793 shutdown_connections 2024-08-31T06:10:48.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.933+0000 7ff10ebe6740 1 -- 172.21.15.16:0/2104952793 wait complete. 2024-08-31T06:10:48.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.933+0000 7ff10ebe6740 20 client.26402 trim_cache size 0 max 16384 2024-08-31T06:10:48.940 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:10:48.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.934+0000 7ff106800640 1 -- 172.21.15.16:0/4159896977 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff0e80b4d20 msgr2=0x7ff0e80b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:48.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.934+0000 7ff106800640 1 --2- 172.21.15.16:0/4159896977 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff0e80b4d20 0x7ff0e80b7230 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec0058c0 tx=0x7ff0ec005850 comp rx=0 tx=0).stop 2024-08-31T06:10:48.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.934+0000 7ff106800640 1 -- 172.21.15.16:0/4159896977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1001099e0 msgr2=0x7ff100075f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:48.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.934+0000 7ff106800640 1 --2- 172.21.15.16:0/4159896977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1001099e0 0x7ff100075f30 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff0f800d950 tx=0x7ff0f800de10 comp rx=0 tx=0).stop 2024-08-31T06:10:48.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.934+0000 7ff106800640 1 -- 172.21.15.16:0/4159896977 shutdown_connections 2024-08-31T06:10:48.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.934+0000 7ff106800640 1 --2- 172.21.15.16:0/4159896977 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff0e80b4d20 0x7ff0e80b7230 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:48.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.934+0000 7ff106800640 1 --2- 172.21.15.16:0/4159896977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1001099e0 0x7ff100075f30 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:48.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.934+0000 7ff106800640 1 --2- 172.21.15.16:0/4159896977 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff100109090 0x7ff1000759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:48.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.934+0000 7ff106800640 1 --2- 172.21.15.16:0/4159896977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff100103750 0x7ff1000754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:48.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.934+0000 7ff106800640 1 -- 172.21.15.16:0/4159896977 >> 172.21.15.16:0/4159896977 conn(0x7ff1000ff1e0 msgr2=0x7ff100101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:48.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.934+0000 7ff106800640 1 -- 172.21.15.16:0/4159896977 shutdown_connections 2024-08-31T06:10:48.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:48.934+0000 7ff106800640 1 -- 172.21.15.16:0/4159896977 wait complete. 2024-08-31T06:10:48.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:48 smithi184 ceph-mon[22331]: pgmap v471: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 3.7 MiB/s wr, 411 op/s 2024-08-31T06:10:49.798 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T06:10:50.178 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:10:50.615 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:50 smithi016 ceph-mon[17194]: pgmap v472: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 3.4 MiB/s wr, 375 op/s 2024-08-31T06:10:50.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.793+0000 7f533b000640 1 -- 172.21.15.16:0/958842163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f533410b470 msgr2=0x7f533410b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:50.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.793+0000 7f533b000640 1 --2- 172.21.15.16:0/958842163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f533410b470 0x7f533410b880 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f531c009900 tx=0x7f531c02a410 comp rx=0 tx=0).stop 2024-08-31T06:10:50.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.793+0000 7f533b000640 1 -- 172.21.15.16:0/958842163 shutdown_connections 2024-08-31T06:10:50.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.793+0000 7f533b000640 1 --2- 172.21.15.16:0/958842163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f533410bdc0 0x7f53341125f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:50.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.793+0000 7f533b000640 1 --2- 172.21.15.16:0/958842163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f533410b470 0x7f533410b880 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:50.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.793+0000 7f533b000640 1 --2- 172.21.15.16:0/958842163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5334105b30 0x7f5334105f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:50.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.793+0000 7f533b000640 1 -- 172.21.15.16:0/958842163 >> 172.21.15.16:0/958842163 conn(0x7f5334101600 msgr2=0x7f5334103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:50.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.793+0000 7f533b000640 1 -- 172.21.15.16:0/958842163 shutdown_connections 2024-08-31T06:10:50.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.793+0000 7f533b000640 1 -- 172.21.15.16:0/958842163 wait complete. 2024-08-31T06:10:50.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.794+0000 7f533b000640 1 Processor -- start 2024-08-31T06:10:50.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.795+0000 7f533b000640 1 -- start start 2024-08-31T06:10:50.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.795+0000 7f533b000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5334105b30 0x7f533419c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:50.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.795+0000 7f533b000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f533410b470 0x7f533419ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:50.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.795+0000 7f533b000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f533410bdc0 0x7f53341a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:50.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.795+0000 7f533b000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5334115a80 con 0x7f533410bdc0 2024-08-31T06:10:50.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.795+0000 7f533b000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5334115900 con 0x7f5334105b30 2024-08-31T06:10:50.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.795+0000 7f533b000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5334115c00 con 0x7f533410b470 2024-08-31T06:10:50.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.795+0000 7f5339000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f533410bdc0 0x7f53341a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:50.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.795+0000 7f5339000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f533410bdc0 0x7f53341a1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37064/0 (socket says 172.21.15.16:37064) 2024-08-31T06:10:50.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.795+0000 7f5339000640 1 -- 172.21.15.16:0/4239488432 learned_addr learned my addr 172.21.15.16:0/4239488432 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:50.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.795+0000 7f5333e00640 1 --2- 172.21.15.16:0/4239488432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5334105b30 0x7f533419c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:50.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.796+0000 7f532b400640 1 --2- 172.21.15.16:0/4239488432 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f533410b470 0x7f533419ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:50.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.796+0000 7f5339000640 1 -- 172.21.15.16:0/4239488432 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f533410b470 msgr2=0x7f533419ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:50.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.796+0000 7f5339000640 1 --2- 172.21.15.16:0/4239488432 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f533410b470 0x7f533419ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:50.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.796+0000 7f5339000640 1 -- 172.21.15.16:0/4239488432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5334105b30 msgr2=0x7f533419c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:50.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.796+0000 7f5339000640 1 --2- 172.21.15.16:0/4239488432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5334105b30 0x7f533419c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:50.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.796+0000 7f5339000640 1 -- 172.21.15.16:0/4239488432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53341a23e0 con 0x7f533410bdc0 2024-08-31T06:10:50.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.796+0000 7f5339000640 1 --2- 172.21.15.16:0/4239488432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f533410bdc0 0x7f53341a1cb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f532c00eb10 tx=0x7f532c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:50.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.797+0000 7f5331600640 1 -- 172.21.15.16:0/4239488432 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c004b20 con 0x7f533410bdc0 2024-08-31T06:10:50.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.797+0000 7f5331600640 1 -- 172.21.15.16:0/4239488432 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f532c004cc0 con 0x7f533410bdc0 2024-08-31T06:10:50.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.797+0000 7f533b000640 1 -- 172.21.15.16:0/4239488432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53341a26d0 con 0x7f533410bdc0 2024-08-31T06:10:50.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.797+0000 7f5331600640 1 -- 172.21.15.16:0/4239488432 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c010670 con 0x7f533410bdc0 2024-08-31T06:10:50.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.797+0000 7f533b000640 1 -- 172.21.15.16:0/4239488432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53341a8e30 con 0x7f533410bdc0 2024-08-31T06:10:50.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.800+0000 7f5341dd7740 1 Processor -- start 2024-08-31T06:10:50.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.800+0000 7f5331600640 1 -- 172.21.15.16:0/4239488432 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f532c005020 con 0x7f533410bdc0 2024-08-31T06:10:50.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.801+0000 7f5341dd7740 1 -- start start 2024-08-31T06:10:50.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.801+0000 7f5341dd7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f22eb2070 0x556f23257990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:50.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.801+0000 7f5341dd7740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f22e891b0 0x556f23257ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:50.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.801+0000 7f5341dd7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f22e91390 0x556f23258410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:50.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.801+0000 7f5341dd7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556f231ce7b0 con 0x556f22e91390 2024-08-31T06:10:50.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.801+0000 7f5341dd7740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556f231bdd40 con 0x556f22e891b0 2024-08-31T06:10:50.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.801+0000 7f5341dd7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556f231c1cf0 con 0x556f22eb2070 2024-08-31T06:10:50.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.801+0000 7f532b400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f22eb2070 0x556f23257990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:50.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.801+0000 7f5333e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f22e891b0 0x556f23257ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:50.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.801+0000 7f532b400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f22eb2070 0x556f23257990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:49440/0 (socket says 172.21.15.16:49440) 2024-08-31T06:10:50.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.801+0000 7f532b400640 1 -- 172.21.15.16:0/2315231160 learned_addr learned my addr 172.21.15.16:0/2315231160 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:50.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.801+0000 7f532b400640 1 --2- 172.21.15.16:0/2315231160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f22e91390 0x556f23258410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:50.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.802+0000 7f5333e00640 1 -- 172.21.15.16:0/2315231160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f22eb2070 msgr2=0x556f23257990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:50.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.802+0000 7f5333e00640 1 --2- 172.21.15.16:0/2315231160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f22eb2070 0x556f23257990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:50.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.802+0000 7f5333e00640 1 -- 172.21.15.16:0/2315231160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f22e91390 msgr2=0x556f23258410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:50.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.802+0000 7f5333e00640 1 --2- 172.21.15.16:0/2315231160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f22e91390 0x556f23258410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:50.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.802+0000 7f5333e00640 1 -- 172.21.15.16:0/2315231160 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f22f1d650 con 0x556f22e891b0 2024-08-31T06:10:50.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.802+0000 7f5331600640 1 --2- 172.21.15.16:0/4239488432 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f53180b4ca0 0x7f53180b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:50.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.802+0000 7f5333e00640 1 --2- 172.21.15.16:0/2315231160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f22e891b0 0x556f23257ed0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5324009c20 tx=0x7f5324011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de680f5209ef19f9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:50.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.802+0000 7f5331600640 1 -- 172.21.15.16:0/4239488432 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f532c0e5af0 con 0x7f533410bdc0 2024-08-31T06:10:50.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.802+0000 7f532b400640 1 --2- 172.21.15.16:0/4239488432 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f53180b4ca0 0x7f53180b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:50.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.803+0000 7f5329600640 1 -- 172.21.15.16:0/2315231160 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5324004050 con 0x556f22e891b0 2024-08-31T06:10:50.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.803+0000 7f5329600640 1 -- 172.21.15.16:0/2315231160 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53240054d0 con 0x556f22e891b0 2024-08-31T06:10:50.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.803+0000 7f5329600640 1 -- 172.21.15.16:0/2315231160 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5324004c60 con 0x556f22e891b0 2024-08-31T06:10:50.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.803+0000 7f5341dd7740 1 -- 172.21.15.16:0/2315231160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f22e891b0 msgr2=0x556f23257ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:50.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.803+0000 7f532b400640 1 --2- 172.21.15.16:0/4239488432 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f53180b4ca0 0x7f53180b71b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f531c0096c0 tx=0x7f531c031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:50.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.804+0000 7f5341dd7740 1 --2- 172.21.15.16:0/2315231160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f22e891b0 0x556f23257ed0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5324009c20 tx=0x7f5324011810 comp rx=0 tx=0).stop 2024-08-31T06:10:50.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.804+0000 7f5341dd7740 1 -- 172.21.15.16:0/2315231160 shutdown_connections 2024-08-31T06:10:50.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.804+0000 7f5341dd7740 1 --2- 172.21.15.16:0/2315231160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f22e91390 0x556f23258410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:50.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.804+0000 7f5341dd7740 1 --2- 172.21.15.16:0/2315231160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f22e891b0 0x556f23257ed0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:50.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.804+0000 7f5341dd7740 1 --2- 172.21.15.16:0/2315231160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f22eb2070 0x556f23257990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:50.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.804+0000 7f5341dd7740 1 -- 172.21.15.16:0/2315231160 >> 172.21.15.16:0/2315231160 conn(0x556f22ea6460 msgr2=0x556f23255880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:50.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.804+0000 7f5341dd7740 1 -- 172.21.15.16:0/2315231160 shutdown_connections 2024-08-31T06:10:50.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.804+0000 7f5341dd7740 1 -- 172.21.15.16:0/2315231160 wait complete. 2024-08-31T06:10:50.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.804+0000 7f5341dd7740 1 -- start start 2024-08-31T06:10:50.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.805+0000 7f5341dd7740 1 Processor -- start 2024-08-31T06:10:50.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.805+0000 7f5341dd7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f22e891b0 0x556f23263500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:50.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.805+0000 7f5341dd7740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f22e91390 0x556f23263a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:50.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.805+0000 7f5341dd7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f22eb2070 0x556f2326f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:50.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.805+0000 7f5341dd7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556f23020700 con 0x556f22eb2070 2024-08-31T06:10:50.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.805+0000 7f5341dd7740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556f23144cf0 con 0x556f22e91390 2024-08-31T06:10:50.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.805+0000 7f5341dd7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556f23266170 con 0x556f22e891b0 2024-08-31T06:10:50.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.805+0000 7f5341dd7740 10 client.0 _ll_register_callbacks cb 0x556f2316a470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:10:50.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.805+0000 7f532b400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f22e91390 0x556f23263a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:50.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.805+0000 7f5339000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f22eb2070 0x556f2326f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:50.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.805+0000 7f532b400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f22e91390 0x556f23263a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:36680/0 (socket says 172.21.15.16:36680) 2024-08-31T06:10:50.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.805+0000 7f532b400640 1 -- 172.21.15.16:0/111083542 learned_addr learned my addr 172.21.15.16:0/111083542 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:50.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.806+0000 7f5333e00640 1 --2- 172.21.15.16:0/111083542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f22e891b0 0x556f23263500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:50.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.806+0000 7f532b400640 1 -- 172.21.15.16:0/111083542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f22e891b0 msgr2=0x556f23263500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:50.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.806+0000 7f532b400640 1 --2- 172.21.15.16:0/111083542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f22e891b0 0x556f23263500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:50.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.806+0000 7f532b400640 1 -- 172.21.15.16:0/111083542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f22eb2070 msgr2=0x556f2326f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:50.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.806+0000 7f532b400640 1 --2- 172.21.15.16:0/111083542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f22eb2070 0x556f2326f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:50.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.806+0000 7f532b400640 1 -- 172.21.15.16:0/111083542 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f231858b0 con 0x556f22e91390 2024-08-31T06:10:50.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.806+0000 7f5339000640 1 --2- 172.21.15.16:0/111083542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f22eb2070 0x556f2326f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:10:50.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.806+0000 7f532b400640 1 --2- 172.21.15.16:0/111083542 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f22e91390 0x556f23263a40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f531c005f70 tx=0x7f531c02a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=422040341e2e7575 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:50.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.806+0000 7f5322000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:10:50.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.807+0000 7f5322000640 1 -- 172.21.15.16:0/111083542 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c04f030 con 0x556f22e91390 2024-08-31T06:10:50.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.807+0000 7f5341dd7740 1 -- 172.21.15.16:0/111083542 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53240054d0 con 0x556f22e91390 2024-08-31T06:10:50.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.807+0000 7f5322000640 1 -- 172.21.15.16:0/111083542 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f531c004040 con 0x556f22e91390 2024-08-31T06:10:50.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.807+0000 7f5322000640 1 -- 172.21.15.16:0/111083542 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f531c035040 con 0x556f22e91390 2024-08-31T06:10:50.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.807+0000 7f5341dd7740 10 client.26414 fetch_fsmap learned FSMap version 32 2024-08-31T06:10:50.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.807+0000 7f5341dd7740 1 -- 172.21.15.16:0/111083542 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556f232721b0 con 0x556f22e91390 2024-08-31T06:10:50.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.807+0000 7f5322000640 1 -- 172.21.15.16:0/111083542 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53240054d0 con 0x556f22e91390 2024-08-31T06:10:50.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.807+0000 7f5322000640 1 -- 172.21.15.16:0/111083542 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f531c048050 con 0x556f22e91390 2024-08-31T06:10:50.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.808+0000 7f5341dd7740 10 client.26414 fetch_fsmap finished waiting for FSMap version 32 2024-08-31T06:10:50.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.808+0000 7f5341dd7740 10 client.26414 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:10:50.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.808+0000 7f5341dd7740 20 client.26414 populate_metadata read hostname 'smithi016' 2024-08-31T06:10:50.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.808+0000 7f5341dd7740 1 --2- 172.21.15.16:0/111083542 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556f22e709f0 0x556f23275690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:50.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.808+0000 7f5341dd7740 4 client.26414 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:10:50.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.808+0000 7f5341dd7740 1 -- 172.21.15.16:0/111083542 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f231d0700 con 0x556f22e709f0 2024-08-31T06:10:50.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.808+0000 7f5333e00640 1 --2- 172.21.15.16:0/111083542 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556f22e709f0 0x556f23275690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:50.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.809+0000 7f5333e00640 1 --2- 172.21.15.16:0/111083542 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556f22e709f0 0x556f23275690 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90aae2894355c22 server_cookie=c4a8e006c3776f97 in_seq=0 out_seq=0 2024-08-31T06:10:50.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.809+0000 7f5322000640 10 client.26414 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:50.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.809+0000 7f5322000640 1 -- 172.21.15.16:0/111083542 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556f231d0700 con 0x556f22e709f0 2024-08-31T06:10:50.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.809+0000 7f5322000640 10 client.26414 handle_command_reply: tid=0 2024-08-31T06:10:50.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.810+0000 7f5341dd7740 1 client.26414 shutdown 2024-08-31T06:10:50.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.810+0000 7f5341dd7740 1 -- 172.21.15.16:0/111083542 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f22e91390 msgr2=0x556f23263a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:50.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.810+0000 7f5341dd7740 1 --2- 172.21.15.16:0/111083542 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f22e91390 0x556f23263a40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f531c005f70 tx=0x7f531c02a410 comp rx=0 tx=0).stop 2024-08-31T06:10:50.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.810+0000 7f5341dd7740 1 -- 172.21.15.16:0/111083542 shutdown_connections 2024-08-31T06:10:50.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.810+0000 7f5341dd7740 1 --2- 172.21.15.16:0/111083542 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556f22e709f0 0x556f23275690 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:50.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.810+0000 7f5341dd7740 1 --2- 172.21.15.16:0/111083542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f22eb2070 0x556f2326f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:50.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.810+0000 7f5341dd7740 1 --2- 172.21.15.16:0/111083542 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f22e91390 0x556f23263a40 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:50.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.810+0000 7f5341dd7740 1 --2- 172.21.15.16:0/111083542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f22e891b0 0x556f23263500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:50.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.810+0000 7f5341dd7740 1 -- 172.21.15.16:0/111083542 >> 172.21.15.16:0/111083542 conn(0x556f22ea6460 msgr2=0x556f2325ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:50.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.810+0000 7f5322000640 0 client.26414 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:50.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.810+0000 7f5341dd7740 1 -- 172.21.15.16:0/111083542 shutdown_connections 2024-08-31T06:10:50.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.810+0000 7f5341dd7740 1 -- 172.21.15.16:0/111083542 wait complete. 2024-08-31T06:10:50.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.810+0000 7f5341dd7740 20 client.26414 trim_cache size 0 max 16384 2024-08-31T06:10:50.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.824+0000 7f5341dd7740 1 Processor -- start 2024-08-31T06:10:50.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.825+0000 7f5341dd7740 1 -- start start 2024-08-31T06:10:50.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.825+0000 7f5341dd7740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f22e891b0 0x556f2326f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:50.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.825+0000 7f5341dd7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f22e91390 0x556f232604f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:50.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.825+0000 7f5341dd7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f22eb2070 0x556f23260a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:50.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.825+0000 7f5341dd7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f53341b3fc0 con 0x556f22e91390 2024-08-31T06:10:50.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.825+0000 7f5341dd7740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f53341b1800 con 0x556f22e891b0 2024-08-31T06:10:50.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.825+0000 7f5341dd7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f53341af650 con 0x556f22eb2070 2024-08-31T06:10:50.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.825+0000 7f532b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f22e91390 0x556f232604f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:50.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.825+0000 7f532b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f22e91390 0x556f232604f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37092/0 (socket says 172.21.15.16:37092) 2024-08-31T06:10:50.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.825+0000 7f532b400640 1 -- 172.21.15.16:0/278018568 learned_addr learned my addr 172.21.15.16:0/278018568 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:50.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.825+0000 7f5333e00640 1 --2- 172.21.15.16:0/278018568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f22e891b0 0x556f2326f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:50.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.825+0000 7f5339000640 1 --2- 172.21.15.16:0/278018568 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f22eb2070 0x556f23260a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:50.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.826+0000 7f5333e00640 1 -- 172.21.15.16:0/278018568 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f22eb2070 msgr2=0x556f23260a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:50.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.826+0000 7f5333e00640 1 --2- 172.21.15.16:0/278018568 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f22eb2070 0x556f23260a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:50.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.826+0000 7f5333e00640 1 -- 172.21.15.16:0/278018568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f22e91390 msgr2=0x556f232604f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:50.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.826+0000 7f5333e00640 1 --2- 172.21.15.16:0/278018568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f22e91390 0x556f232604f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:50.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.826+0000 7f5333e00640 1 -- 172.21.15.16:0/278018568 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f232643a0 con 0x556f22e891b0 2024-08-31T06:10:50.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.826+0000 7f5333e00640 1 --2- 172.21.15.16:0/278018568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f22e891b0 0x556f2326f390 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5324004880 tx=0x7f5324009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef88987aa3868abf server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:50.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.826+0000 7f5328c00640 1 -- 172.21.15.16:0/278018568 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5324004220 con 0x556f22e891b0 2024-08-31T06:10:50.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.826+0000 7f5328c00640 1 -- 172.21.15.16:0/278018568 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53240043c0 con 0x556f22e891b0 2024-08-31T06:10:50.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.826+0000 7f5328c00640 1 -- 172.21.15.16:0/278018568 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53240196b0 con 0x556f22e891b0 2024-08-31T06:10:50.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.827+0000 7f5341dd7740 1 -- 172.21.15.16:0/278018568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f22e891b0 msgr2=0x556f2326f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:50.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.827+0000 7f5341dd7740 1 --2- 172.21.15.16:0/278018568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f22e891b0 0x556f2326f390 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5324004880 tx=0x7f5324009bb0 comp rx=0 tx=0).stop 2024-08-31T06:10:50.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.827+0000 7f5341dd7740 1 -- 172.21.15.16:0/278018568 shutdown_connections 2024-08-31T06:10:50.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.827+0000 7f5341dd7740 1 --2- 172.21.15.16:0/278018568 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f22eb2070 0x556f23260a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:50.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.827+0000 7f5341dd7740 1 --2- 172.21.15.16:0/278018568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f22e91390 0x556f232604f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:50.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.827+0000 7f5341dd7740 1 --2- 172.21.15.16:0/278018568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f22e891b0 0x556f2326f390 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:50.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.827+0000 7f5341dd7740 1 -- 172.21.15.16:0/278018568 >> 172.21.15.16:0/278018568 conn(0x556f22ea6460 msgr2=0x556f23261b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:50.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.827+0000 7f5341dd7740 1 -- 172.21.15.16:0/278018568 shutdown_connections 2024-08-31T06:10:50.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.828+0000 7f5341dd7740 1 -- 172.21.15.16:0/278018568 wait complete. 2024-08-31T06:10:50.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.828+0000 7f5341dd7740 1 -- start start 2024-08-31T06:10:50.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.828+0000 7f5341dd7740 1 Processor -- start 2024-08-31T06:10:50.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.829+0000 7f5341dd7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f22e891b0 0x556f23267ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:50.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.829+0000 7f5341dd7740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f22e91390 0x556f23268480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:50.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.829+0000 7f5341dd7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f22eb2070 0x556f232d9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:50.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.829+0000 7f5341dd7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f53341afcc0 con 0x556f22e891b0 2024-08-31T06:10:50.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.829+0000 7f5341dd7740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f53341b1130 con 0x556f22e91390 2024-08-31T06:10:50.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.829+0000 7f5341dd7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f53341b0390 con 0x556f22eb2070 2024-08-31T06:10:50.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.829+0000 7f5341dd7740 10 client.0 _ll_register_callbacks cb 0x7f531c005e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:10:50.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.829+0000 7f5333e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f22e891b0 0x556f23267ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:50.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.829+0000 7f5333e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f22e891b0 0x556f23267ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37096/0 (socket says 172.21.15.16:37096) 2024-08-31T06:10:50.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.829+0000 7f5333e00640 1 -- 172.21.15.16:0/4067841773 learned_addr learned my addr 172.21.15.16:0/4067841773 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:50.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.829+0000 7f532b400640 1 --2- 172.21.15.16:0/4067841773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f22e91390 0x556f23268480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:50.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.829+0000 7f5339000640 1 --2- 172.21.15.16:0/4067841773 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f22eb2070 0x556f232d9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:50.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.829+0000 7f5333e00640 1 -- 172.21.15.16:0/4067841773 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f22eb2070 msgr2=0x556f232d9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:50.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.829+0000 7f5333e00640 1 --2- 172.21.15.16:0/4067841773 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f22eb2070 0x556f232d9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:50.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.829+0000 7f5333e00640 1 -- 172.21.15.16:0/4067841773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f22e91390 msgr2=0x556f23268480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:50.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.829+0000 7f5333e00640 1 --2- 172.21.15.16:0/4067841773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f22e91390 0x556f23268480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:50.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.829+0000 7f5333e00640 1 -- 172.21.15.16:0/4067841773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f232d9800 con 0x556f22e891b0 2024-08-31T06:10:50.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.830+0000 7f5333e00640 1 --2- 172.21.15.16:0/4067841773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f22e891b0 0x556f23267ea0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5324009b40 tx=0x7f5324009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84003286c1ebc152 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:50.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.830+0000 7f5322000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:10:50.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.830+0000 7f5322000640 1 -- 172.21.15.16:0/4067841773 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532403a070 con 0x556f22e891b0 2024-08-31T06:10:50.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.830+0000 7f5341dd7740 1 -- 172.21.15.16:0/4067841773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53240043c0 con 0x556f22e891b0 2024-08-31T06:10:50.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.830+0000 7f5322000640 1 -- 172.21.15.16:0/4067841773 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f5324003fa0 con 0x556f22e891b0 2024-08-31T06:10:50.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.830+0000 7f5322000640 1 -- 172.21.15.16:0/4067841773 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f53240041a0 con 0x556f22e891b0 2024-08-31T06:10:50.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.830+0000 7f5341dd7740 10 client.16293 fetch_fsmap learned FSMap version 32 2024-08-31T06:10:50.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.830+0000 7f5341dd7740 1 -- 172.21.15.16:0/4067841773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556f232e0bf0 con 0x556f22e891b0 2024-08-31T06:10:50.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.831+0000 7f5322000640 1 -- 172.21.15.16:0/4067841773 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53240043c0 con 0x556f22e891b0 2024-08-31T06:10:50.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.831+0000 7f5322000640 1 -- 172.21.15.16:0/4067841773 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f5324019980 con 0x556f22e891b0 2024-08-31T06:10:50.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.831+0000 7f5341dd7740 10 client.16293 fetch_fsmap finished waiting for FSMap version 32 2024-08-31T06:10:50.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.831+0000 7f5341dd7740 10 client.16293 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:10:50.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.831+0000 7f5341dd7740 20 client.16293 populate_metadata read hostname 'smithi016' 2024-08-31T06:10:50.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.831+0000 7f5341dd7740 1 --2- 172.21.15.16:0/4067841773 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556f22e45da0 0x556f232e3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:50.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.831+0000 7f5341dd7740 4 client.16293 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:10:50.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.831+0000 7f5341dd7740 1 -- 172.21.15.16:0/4067841773 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556f231cc1a0 con 0x556f22e45da0 2024-08-31T06:10:50.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.831+0000 7f532b400640 1 --2- 172.21.15.16:0/4067841773 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556f22e45da0 0x556f232e3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:50.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.832+0000 7f532b400640 1 --2- 172.21.15.16:0/4067841773 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556f22e45da0 0x556f232e3210 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d4634ffe36cbede server_cookie=367efec83d16faa8 in_seq=0 out_seq=0 2024-08-31T06:10:50.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.832+0000 7f5322000640 10 client.16293 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:50.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.832+0000 7f5322000640 1 -- 172.21.15.16:0/4067841773 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556f231cc1a0 con 0x556f22e45da0 2024-08-31T06:10:50.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.832+0000 7f5322000640 10 client.16293 handle_command_reply: tid=0 2024-08-31T06:10:50.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.832+0000 7f5341dd7740 1 client.16293 shutdown 2024-08-31T06:10:50.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.832+0000 7f5341dd7740 1 -- 172.21.15.16:0/4067841773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f22e891b0 msgr2=0x556f23267ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:50.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.832+0000 7f5341dd7740 1 --2- 172.21.15.16:0/4067841773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f22e891b0 0x556f23267ea0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5324009b40 tx=0x7f5324009640 comp rx=0 tx=0).stop 2024-08-31T06:10:50.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.833+0000 7f5341dd7740 1 -- 172.21.15.16:0/4067841773 shutdown_connections 2024-08-31T06:10:50.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.833+0000 7f5341dd7740 1 --2- 172.21.15.16:0/4067841773 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556f22e45da0 0x556f232e3210 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:50.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.833+0000 7f5341dd7740 1 --2- 172.21.15.16:0/4067841773 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f22eb2070 0x556f232d9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:50.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.833+0000 7f5341dd7740 1 --2- 172.21.15.16:0/4067841773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f22e91390 0x556f23268480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:50.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.833+0000 7f5341dd7740 1 --2- 172.21.15.16:0/4067841773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f22e891b0 0x556f23267ea0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:50.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.833+0000 7f5341dd7740 1 -- 172.21.15.16:0/4067841773 >> 172.21.15.16:0/4067841773 conn(0x556f22ea6460 msgr2=0x556f232589c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:50.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.833+0000 7f5322000640 0 client.16293 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:50.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.833+0000 7f5341dd7740 1 -- 172.21.15.16:0/4067841773 shutdown_connections 2024-08-31T06:10:50.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.833+0000 7f5341dd7740 1 -- 172.21.15.16:0/4067841773 wait complete. 2024-08-31T06:10:50.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.833+0000 7f5341dd7740 20 client.16293 trim_cache size 0 max 16384 2024-08-31T06:10:50.838 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:10:50.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.833+0000 7f533b000640 1 -- 172.21.15.16:0/4239488432 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f53180b4ca0 msgr2=0x7f53180b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:50.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.833+0000 7f533b000640 1 --2- 172.21.15.16:0/4239488432 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f53180b4ca0 0x7f53180b71b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f531c0096c0 tx=0x7f531c031400 comp rx=0 tx=0).stop 2024-08-31T06:10:50.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.833+0000 7f533b000640 1 -- 172.21.15.16:0/4239488432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f533410bdc0 msgr2=0x7f53341a1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:50.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.833+0000 7f533b000640 1 --2- 172.21.15.16:0/4239488432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f533410bdc0 0x7f53341a1cb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f532c00eb10 tx=0x7f532c00c2d0 comp rx=0 tx=0).stop 2024-08-31T06:10:50.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.834+0000 7f533b000640 1 -- 172.21.15.16:0/4239488432 shutdown_connections 2024-08-31T06:10:50.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.834+0000 7f533b000640 1 --2- 172.21.15.16:0/4239488432 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f53180b4ca0 0x7f53180b71b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:50.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.834+0000 7f533b000640 1 --2- 172.21.15.16:0/4239488432 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f533410b470 0x7f533419ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:50.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.834+0000 7f533b000640 1 --2- 172.21.15.16:0/4239488432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f533410bdc0 0x7f53341a1cb0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:50.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.834+0000 7f533b000640 1 --2- 172.21.15.16:0/4239488432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5334105b30 0x7f533419c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:50.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.834+0000 7f533b000640 1 -- 172.21.15.16:0/4239488432 >> 172.21.15.16:0/4239488432 conn(0x7f5334101600 msgr2=0x7f5334103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:50.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.834+0000 7f533b000640 1 -- 172.21.15.16:0/4239488432 shutdown_connections 2024-08-31T06:10:50.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:50.834+0000 7f533b000640 1 -- 172.21.15.16:0/4239488432 wait complete. 2024-08-31T06:10:50.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:50 smithi155 ceph-mon[21742]: pgmap v472: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 3.4 MiB/s wr, 375 op/s 2024-08-31T06:10:50.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:50 smithi184 ceph-mon[22331]: pgmap v472: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 3.4 MiB/s wr, 375 op/s 2024-08-31T06:10:51.628 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 205.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-31T06:10:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.636+0000 7f9cd3600640 1 -- 172.21.15.16:0/3922500588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ccc10b3c0 msgr2=0x7f9ccc10b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.636+0000 7f9cd3600640 1 --2- 172.21.15.16:0/3922500588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ccc10b3c0 0x7f9ccc10b7d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8009900 tx=0x7f9cb802a410 comp rx=0 tx=0).stop 2024-08-31T06:10:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.636+0000 7f9cd3600640 1 -- 172.21.15.16:0/3922500588 shutdown_connections 2024-08-31T06:10:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.636+0000 7f9cd3600640 1 --2- 172.21.15.16:0/3922500588 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ccc10bd10 0x7f9ccc112540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.636+0000 7f9cd3600640 1 --2- 172.21.15.16:0/3922500588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ccc10b3c0 0x7f9ccc10b7d0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.636+0000 7f9cd3600640 1 --2- 172.21.15.16:0/3922500588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ccc105a90 0x7f9ccc105ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.636+0000 7f9cd3600640 1 -- 172.21.15.16:0/3922500588 >> 172.21.15.16:0/3922500588 conn(0x7f9ccc101470 msgr2=0x7f9ccc1038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.636+0000 7f9cd3600640 1 -- 172.21.15.16:0/3922500588 shutdown_connections 2024-08-31T06:10:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.636+0000 7f9cd3600640 1 -- 172.21.15.16:0/3922500588 wait complete. 2024-08-31T06:10:51.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.637+0000 7f9cd3600640 1 Processor -- start 2024-08-31T06:10:51.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.637+0000 7f9cd3600640 1 -- start start 2024-08-31T06:10:51.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.638+0000 7f9cd3600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ccc105a90 0x7f9ccc19c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:51.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.638+0000 7f9cd3600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ccc10b3c0 0x7f9ccc19c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:51.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.638+0000 7f9cd3600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ccc10bd10 0x7f9ccc1a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:51.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.638+0000 7f9cd3600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc115a20 con 0x7f9ccc105a90 2024-08-31T06:10:51.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.638+0000 7f9cd3600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc1158a0 con 0x7f9ccc10bd10 2024-08-31T06:10:51.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.638+0000 7f9cd3600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc115ba0 con 0x7f9ccc10b3c0 2024-08-31T06:10:51.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.638+0000 7f9cd0c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ccc105a90 0x7f9ccc19c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.638+0000 7f9cd0c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ccc105a90 0x7f9ccc19c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37116/0 (socket says 172.21.15.16:37116) 2024-08-31T06:10:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.638+0000 7f9cd0c00640 1 -- 172.21.15.16:0/3400596187 learned_addr learned my addr 172.21.15.16:0/3400596187 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.638+0000 7f9ccbe00640 1 --2- 172.21.15.16:0/3400596187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ccc10b3c0 0x7f9ccc19c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.638+0000 7f9cd1600640 1 --2- 172.21.15.16:0/3400596187 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ccc10bd10 0x7f9ccc1a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.638+0000 7f9cd0c00640 1 -- 172.21.15.16:0/3400596187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ccc10b3c0 msgr2=0x7f9ccc19c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.638+0000 7f9cd0c00640 1 --2- 172.21.15.16:0/3400596187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ccc10b3c0 0x7f9ccc19c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.639+0000 7f9cd0c00640 1 -- 172.21.15.16:0/3400596187 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ccc10bd10 msgr2=0x7f9ccc1a1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.639+0000 7f9cd0c00640 1 --2- 172.21.15.16:0/3400596187 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ccc10bd10 0x7f9ccc1a1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.639+0000 7f9cd0c00640 1 -- 172.21.15.16:0/3400596187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ccc1a2350 con 0x7f9ccc105a90 2024-08-31T06:10:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.639+0000 7f9cd0c00640 1 --2- 172.21.15.16:0/3400596187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ccc105a90 0x7f9ccc19c420 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f9cc000d950 tx=0x7f9cc000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.639+0000 7f9cc9600640 1 -- 172.21.15.16:0/3400596187 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc0017030 con 0x7f9ccc105a90 2024-08-31T06:10:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.639+0000 7f9cd3600640 1 -- 172.21.15.16:0/3400596187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ccc1a2640 con 0x7f9ccc105a90 2024-08-31T06:10:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.639+0000 7f9cc9600640 1 -- 172.21.15.16:0/3400596187 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cc0004020 con 0x7f9ccc105a90 2024-08-31T06:10:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.639+0000 7f9cc9600640 1 -- 172.21.15.16:0/3400596187 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc00047a0 con 0x7f9ccc105a90 2024-08-31T06:10:51.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.639+0000 7f9cd3600640 1 -- 172.21.15.16:0/3400596187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ccc1a8dd0 con 0x7f9ccc105a90 2024-08-31T06:10:51.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.642+0000 7f9cda833740 1 Processor -- start 2024-08-31T06:10:51.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.643+0000 7f9cda833740 1 -- start start 2024-08-31T06:10:51.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.643+0000 7f9cda833740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ddc181b0 0x55d0ddfbb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:51.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.643+0000 7f9cda833740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0ddbef2f0 0x55d0ddfbbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:51.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.643+0000 7f9cda833740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0ddbf74d0 0x55d0ddfbc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:51.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.643+0000 7f9cda833740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d0ddf30f70 con 0x55d0ddbf74d0 2024-08-31T06:10:51.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.643+0000 7f9cda833740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d0ddf20500 con 0x55d0ddbef2f0 2024-08-31T06:10:51.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.643+0000 7f9cda833740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d0ddf244b0 con 0x55d0ddc181b0 2024-08-31T06:10:51.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.643+0000 7f9ccbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0ddbf74d0 0x55d0ddfbc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:51.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.643+0000 7f9ccbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0ddbf74d0 0x55d0ddfbc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37118/0 (socket says 172.21.15.16:37118) 2024-08-31T06:10:51.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.643+0000 7f9ccbe00640 1 -- 172.21.15.16:0/2126981130 learned_addr learned my addr 172.21.15.16:0/2126981130 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.643+0000 7f9cc9600640 1 -- 172.21.15.16:0/3400596187 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f9cc0009a30 con 0x7f9ccc105a90 2024-08-31T06:10:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.643+0000 7f9cd1600640 1 --2- 172.21.15.16:0/2126981130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ddc181b0 0x55d0ddfbb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.644+0000 7f9ccbe00640 1 -- 172.21.15.16:0/2126981130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ddc181b0 msgr2=0x55d0ddfbb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.644+0000 7f9ccbe00640 1 --2- 172.21.15.16:0/2126981130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ddc181b0 0x55d0ddfbb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.644+0000 7f9ccbe00640 1 -- 172.21.15.16:0/2126981130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0ddbef2f0 msgr2=0x55d0ddfbbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.644+0000 7f9ccbe00640 1 --2- 172.21.15.16:0/2126981130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0ddbef2f0 0x55d0ddfbbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.644+0000 7f9ccbe00640 1 -- 172.21.15.16:0/2126981130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0ddedac10 con 0x55d0ddbf74d0 2024-08-31T06:10:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.644+0000 7f9ccbe00640 1 --2- 172.21.15.16:0/2126981130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0ddbf74d0 0x55d0ddfbc110 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8005ab0 tx=0x7f9cb8004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b9cd998deb0e143 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.644+0000 7f9cbdc00640 1 -- 172.21.15.16:0/2126981130 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb8035040 con 0x55d0ddbf74d0 2024-08-31T06:10:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.644+0000 7f9cbdc00640 1 -- 172.21.15.16:0/2126981130 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cb80054f0 con 0x55d0ddbf74d0 2024-08-31T06:10:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.644+0000 7f9cbdc00640 1 -- 172.21.15.16:0/2126981130 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb80037f0 con 0x55d0ddbf74d0 2024-08-31T06:10:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.645+0000 7f9cc9600640 1 --2- 172.21.15.16:0/3400596187 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9cb40b4ca0 0x7f9cb40b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:51.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.645+0000 7f9cda833740 1 -- 172.21.15.16:0/2126981130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0ddbf74d0 msgr2=0x55d0ddfbc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:51.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.645+0000 7f9cda833740 1 --2- 172.21.15.16:0/2126981130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0ddbf74d0 0x55d0ddfbc110 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8005ab0 tx=0x7f9cb8004180 comp rx=0 tx=0).stop 2024-08-31T06:10:51.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.645+0000 7f9cc9600640 1 -- 172.21.15.16:0/3400596187 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f9cc00e4f60 con 0x7f9ccc105a90 2024-08-31T06:10:51.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.645+0000 7f9cda833740 1 -- 172.21.15.16:0/2126981130 shutdown_connections 2024-08-31T06:10:51.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.645+0000 7f9cda833740 1 --2- 172.21.15.16:0/2126981130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0ddbf74d0 0x55d0ddfbc110 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:51.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.645+0000 7f9cda833740 1 --2- 172.21.15.16:0/2126981130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0ddbef2f0 0x55d0ddfbbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:51.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.645+0000 7f9cda833740 1 --2- 172.21.15.16:0/2126981130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ddc181b0 0x55d0ddfbb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:51.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.645+0000 7f9cda833740 1 -- 172.21.15.16:0/2126981130 >> 172.21.15.16:0/2126981130 conn(0x55d0ddc0c300 msgr2=0x55d0ddfb9580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:51.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.645+0000 7f9cd1600640 1 --2- 172.21.15.16:0/3400596187 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9cb40b4ca0 0x7f9cb40b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:51.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.645+0000 7f9cda833740 1 -- 172.21.15.16:0/2126981130 shutdown_connections 2024-08-31T06:10:51.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.645+0000 7f9cda833740 1 -- 172.21.15.16:0/2126981130 wait complete. 2024-08-31T06:10:51.654 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.646+0000 7f9cda833740 1 -- start start 2024-08-31T06:10:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.646+0000 7f9cd1600640 1 --2- 172.21.15.16:0/3400596187 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9cb40b4ca0 0x7f9cb40b71b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4005fd0 tx=0x7f9cc4009600 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.646+0000 7f9cda833740 1 Processor -- start 2024-08-31T06:10:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.647+0000 7f9cda833740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0ddbef2f0 0x55d0ddfc6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.647+0000 7f9cda833740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0ddbf74d0 0x55d0ddfc7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.647+0000 7f9cda833740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ddc181b0 0x55d0ddfc7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.647+0000 7f9cda833740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d0dde59ab0 con 0x55d0ddbf74d0 2024-08-31T06:10:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.647+0000 7f9cda833740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d0ddec7320 con 0x55d0ddbef2f0 2024-08-31T06:10:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.647+0000 7f9cda833740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d0ddfc9db0 con 0x55d0ddc181b0 2024-08-31T06:10:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.647+0000 7f9cda833740 10 client.0 _ll_register_callbacks cb 0x55d0ddf00e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:10:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.647+0000 7f9ccbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0ddbf74d0 0x55d0ddfc7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.647+0000 7f9cd0c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0ddbef2f0 0x55d0ddfc6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.647+0000 7f9ccbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0ddbf74d0 0x55d0ddfc7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37128/0 (socket says 172.21.15.16:37128) 2024-08-31T06:10:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.647+0000 7f9ccbe00640 1 -- 172.21.15.16:0/1031549428 learned_addr learned my addr 172.21.15.16:0/1031549428 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:51.656 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.647+0000 7f9cd0c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0ddbef2f0 0x55d0ddfc6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:36738/0 (socket says 172.21.15.16:36738) 2024-08-31T06:10:51.656 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.647+0000 7f9cd1600640 1 --2- 172.21.15.16:0/1031549428 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ddc181b0 0x55d0ddfc7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:51.656 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.648+0000 7f9cd0c00640 1 -- 172.21.15.16:0/1031549428 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ddc181b0 msgr2=0x55d0ddfc7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:51.656 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.648+0000 7f9cd0c00640 1 --2- 172.21.15.16:0/1031549428 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ddc181b0 0x55d0ddfc7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:51.656 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.648+0000 7f9cd0c00640 1 -- 172.21.15.16:0/1031549428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0ddbf74d0 msgr2=0x55d0ddfc7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:51.656 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.648+0000 7f9cd0c00640 1 --2- 172.21.15.16:0/1031549428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0ddbf74d0 0x55d0ddfc7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:51.656 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.648+0000 7f9cd0c00640 1 -- 172.21.15.16:0/1031549428 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0ddc83790 con 0x55d0ddbef2f0 2024-08-31T06:10:51.656 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.648+0000 7f9cd1600640 1 --2- 172.21.15.16:0/1031549428 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ddc181b0 0x55d0ddfc7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:10:51.656 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.648+0000 7f9ccbe00640 1 --2- 172.21.15.16:0/1031549428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0ddbf74d0 0x55d0ddfc7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:10:51.656 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.648+0000 7f9cd0c00640 1 --2- 172.21.15.16:0/1031549428 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0ddbef2f0 0x55d0ddfc6cc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00e26b0 tx=0x7f9cc0009f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd75b2d0fe8972f1 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:51.656 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.648+0000 7f9cb2000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:10:51.656 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.648+0000 7f9cb2000640 1 -- 172.21.15.16:0/1031549428 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc00afec0 con 0x55d0ddbef2f0 2024-08-31T06:10:51.656 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.648+0000 7f9cda833740 1 -- 172.21.15.16:0/1031549428 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cb80054f0 con 0x55d0ddbef2f0 2024-08-31T06:10:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.648+0000 7f9cb2000640 1 -- 172.21.15.16:0/1031549428 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f9cc00a7c40 con 0x55d0ddbef2f0 2024-08-31T06:10:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.648+0000 7f9cb2000640 1 -- 172.21.15.16:0/1031549428 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9cc00b4970 con 0x55d0ddbef2f0 2024-08-31T06:10:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.648+0000 7f9cda833740 10 client.26429 fetch_fsmap learned FSMap version 32 2024-08-31T06:10:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.648+0000 7f9cda833740 1 -- 172.21.15.16:0/1031549428 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d0ddcfc620 con 0x55d0ddbef2f0 2024-08-31T06:10:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.649+0000 7f9cb2000640 1 -- 172.21.15.16:0/1031549428 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cb80054f0 con 0x55d0ddbef2f0 2024-08-31T06:10:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.649+0000 7f9cb2000640 1 -- 172.21.15.16:0/1031549428 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f9cc00a6fc0 con 0x55d0ddbef2f0 2024-08-31T06:10:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.649+0000 7f9cda833740 10 client.26429 fetch_fsmap finished waiting for FSMap version 32 2024-08-31T06:10:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.649+0000 7f9cda833740 10 client.26429 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:10:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.649+0000 7f9cda833740 20 client.26429 populate_metadata read hostname 'smithi016' 2024-08-31T06:10:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.649+0000 7f9cda833740 1 --2- 172.21.15.16:0/1031549428 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d0ddbd6d80 0x55d0ddfd8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.649+0000 7f9cda833740 4 client.26429 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:10:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.649+0000 7f9cda833740 1 -- 172.21.15.16:0/1031549428 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0ddf32ec0 con 0x55d0ddbd6d80 2024-08-31T06:10:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.649+0000 7f9ccbe00640 1 --2- 172.21.15.16:0/1031549428 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d0ddbd6d80 0x55d0ddfd8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.650+0000 7f9ccbe00640 1 --2- 172.21.15.16:0/1031549428 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d0ddbd6d80 0x55d0ddfd8f20 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=deffc9c85a3562b0 server_cookie=e00d4d0512458a67 in_seq=0 out_seq=0 2024-08-31T06:10:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.650+0000 7f9cb2000640 10 client.26429 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.651+0000 7f9cb2000640 1 -- 172.21.15.16:0/1031549428 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d0ddf32ec0 con 0x55d0ddbd6d80 2024-08-31T06:10:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.651+0000 7f9cb2000640 10 client.26429 handle_command_reply: tid=0 2024-08-31T06:10:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.651+0000 7f9cda833740 1 client.26429 shutdown 2024-08-31T06:10:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.651+0000 7f9cda833740 1 -- 172.21.15.16:0/1031549428 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0ddbef2f0 msgr2=0x55d0ddfc6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.651+0000 7f9cda833740 1 --2- 172.21.15.16:0/1031549428 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0ddbef2f0 0x55d0ddfc6cc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00e26b0 tx=0x7f9cc0009f50 comp rx=0 tx=0).stop 2024-08-31T06:10:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.652+0000 7f9cda833740 1 -- 172.21.15.16:0/1031549428 shutdown_connections 2024-08-31T06:10:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.652+0000 7f9cda833740 1 --2- 172.21.15.16:0/1031549428 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d0ddbd6d80 0x55d0ddfd8f20 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.652+0000 7f9cda833740 1 --2- 172.21.15.16:0/1031549428 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ddc181b0 0x55d0ddfc7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.652+0000 7f9cb2000640 0 client.26429 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.652+0000 7f9cda833740 1 --2- 172.21.15.16:0/1031549428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0ddbf74d0 0x55d0ddfc7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.652+0000 7f9cda833740 1 --2- 172.21.15.16:0/1031549428 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0ddbef2f0 0x55d0ddfc6cc0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.652+0000 7f9cda833740 1 -- 172.21.15.16:0/1031549428 >> 172.21.15.16:0/1031549428 conn(0x55d0ddc0c300 msgr2=0x55d0ddfbc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.652+0000 7f9cda833740 1 -- 172.21.15.16:0/1031549428 shutdown_connections 2024-08-31T06:10:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.652+0000 7f9cda833740 1 -- 172.21.15.16:0/1031549428 wait complete. 2024-08-31T06:10:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.652+0000 7f9cda833740 20 client.26429 trim_cache size 0 max 16384 2024-08-31T06:10:51.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.668+0000 7f9cda833740 1 Processor -- start 2024-08-31T06:10:51.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.668+0000 7f9cda833740 1 -- start start 2024-08-31T06:10:51.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.668+0000 7f9cda833740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0ddbef2f0 0x55d0ddfb84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:51.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.669+0000 7f9cda833740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0ddbf74d0 0x55d0ddfbcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:51.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.669+0000 7f9cda833740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ddc181b0 0x55d0ddfde6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:51.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.669+0000 7f9cda833740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc1b3e50 con 0x55d0ddbf74d0 2024-08-31T06:10:51.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.669+0000 7f9cda833740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc1b1690 con 0x55d0ddbef2f0 2024-08-31T06:10:51.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.669+0000 7f9cda833740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc1af4e0 con 0x55d0ddc181b0 2024-08-31T06:10:51.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.669+0000 7f9cd0c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0ddbef2f0 0x55d0ddfb84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.669+0000 7f9cd0c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0ddbef2f0 0x55d0ddfb84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:36740/0 (socket says 172.21.15.16:36740) 2024-08-31T06:10:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.669+0000 7f9cd0c00640 1 -- 172.21.15.16:0/2942933705 learned_addr learned my addr 172.21.15.16:0/2942933705 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.669+0000 7f9ccbe00640 1 --2- 172.21.15.16:0/2942933705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0ddbf74d0 0x55d0ddfbcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.669+0000 7f9cd1600640 1 --2- 172.21.15.16:0/2942933705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ddc181b0 0x55d0ddfde6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.669+0000 7f9cd0c00640 1 -- 172.21.15.16:0/2942933705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ddc181b0 msgr2=0x55d0ddfde6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.669+0000 7f9cd0c00640 1 --2- 172.21.15.16:0/2942933705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ddc181b0 0x55d0ddfde6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.669+0000 7f9cd0c00640 1 -- 172.21.15.16:0/2942933705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0ddbf74d0 msgr2=0x55d0ddfbcdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.669+0000 7f9cd0c00640 1 --2- 172.21.15.16:0/2942933705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0ddbf74d0 0x55d0ddfbcdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.669+0000 7f9cd0c00640 1 -- 172.21.15.16:0/2942933705 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0ddfc7b90 con 0x55d0ddbef2f0 2024-08-31T06:10:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.670+0000 7f9cd0c00640 1 --2- 172.21.15.16:0/2942933705 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0ddbef2f0 0x55d0ddfb84e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00a88c0 tx=0x7f9cc00043e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24b6fcdec04b9cc6 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.670+0000 7f9cbd200640 1 -- 172.21.15.16:0/2942933705 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc00abd50 con 0x55d0ddbef2f0 2024-08-31T06:10:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.670+0000 7f9cbd200640 1 -- 172.21.15.16:0/2942933705 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cc00afec0 con 0x55d0ddbef2f0 2024-08-31T06:10:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.670+0000 7f9cbd200640 1 -- 172.21.15.16:0/2942933705 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc005b1c0 con 0x55d0ddbef2f0 2024-08-31T06:10:51.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.671+0000 7f9cda833740 1 -- 172.21.15.16:0/2942933705 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0ddbef2f0 msgr2=0x55d0ddfb84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:51.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.671+0000 7f9cda833740 1 --2- 172.21.15.16:0/2942933705 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0ddbef2f0 0x55d0ddfb84e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00a88c0 tx=0x7f9cc00043e0 comp rx=0 tx=0).stop 2024-08-31T06:10:51.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.671+0000 7f9cda833740 1 -- 172.21.15.16:0/2942933705 shutdown_connections 2024-08-31T06:10:51.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.671+0000 7f9cda833740 1 --2- 172.21.15.16:0/2942933705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ddc181b0 0x55d0ddfde6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:51.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.671+0000 7f9cda833740 1 --2- 172.21.15.16:0/2942933705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0ddbf74d0 0x55d0ddfbcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:51.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.671+0000 7f9cda833740 1 --2- 172.21.15.16:0/2942933705 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0ddbef2f0 0x55d0ddfb84e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:51.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.671+0000 7f9cda833740 1 -- 172.21.15.16:0/2942933705 >> 172.21.15.16:0/2942933705 conn(0x55d0ddc0c300 msgr2=0x55d0ddfbc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:51.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.671+0000 7f9cda833740 1 -- 172.21.15.16:0/2942933705 shutdown_connections 2024-08-31T06:10:51.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.671+0000 7f9cda833740 1 -- 172.21.15.16:0/2942933705 wait complete. 2024-08-31T06:10:51.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.671+0000 7f9cda833740 1 -- start start 2024-08-31T06:10:51.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.672+0000 7f9cda833740 1 Processor -- start 2024-08-31T06:10:51.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.672+0000 7f9cda833740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ddbef2f0 0x55d0ddfcd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:51.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.672+0000 7f9cda833740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0ddbf74d0 0x55d0ddfcdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.672+0000 7f9cda833740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0ddc181b0 0x55d0de0428f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.672+0000 7f9cda833740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc1afb50 con 0x55d0ddbf74d0 2024-08-31T06:10:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.672+0000 7f9cda833740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc1b0fc0 con 0x55d0ddc181b0 2024-08-31T06:10:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.672+0000 7f9cda833740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc1b0220 con 0x55d0ddbef2f0 2024-08-31T06:10:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.672+0000 7f9cda833740 10 client.0 _ll_register_callbacks cb 0x7f9cc00044d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:10:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.672+0000 7f9ccbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0ddbf74d0 0x55d0ddfcdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.672+0000 7f9cd0c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ddbef2f0 0x55d0ddfcd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.672+0000 7f9ccbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0ddbf74d0 0x55d0ddfcdcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37144/0 (socket says 172.21.15.16:37144) 2024-08-31T06:10:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.672+0000 7f9cd0c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ddbef2f0 0x55d0ddfcd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:49534/0 (socket says 172.21.15.16:49534) 2024-08-31T06:10:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.672+0000 7f9ccbe00640 1 -- 172.21.15.16:0/2358649800 learned_addr learned my addr 172.21.15.16:0/2358649800 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.673+0000 7f9cd1600640 1 --2- 172.21.15.16:0/2358649800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0ddc181b0 0x55d0de0428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.673+0000 7f9cd0c00640 1 -- 172.21.15.16:0/2358649800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0ddc181b0 msgr2=0x55d0de0428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.673+0000 7f9cd0c00640 1 --2- 172.21.15.16:0/2358649800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0ddc181b0 0x55d0de0428f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.673+0000 7f9cd0c00640 1 -- 172.21.15.16:0/2358649800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0ddbf74d0 msgr2=0x55d0ddfcdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.673+0000 7f9cd0c00640 1 --2- 172.21.15.16:0/2358649800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0ddbf74d0 0x55d0ddfcdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.673+0000 7f9cd0c00640 1 -- 172.21.15.16:0/2358649800 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0de042e60 con 0x55d0ddbef2f0 2024-08-31T06:10:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.673+0000 7f9ccbe00640 1 --2- 172.21.15.16:0/2358649800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0ddbf74d0 0x55d0ddfcdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:10:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.673+0000 7f9cd0c00640 1 --2- 172.21.15.16:0/2358649800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ddbef2f0 0x55d0ddfcd780 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0000c00 tx=0x7f9cc00a7510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2992e040ad368216 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.673+0000 7f9cb2000640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:10:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.673+0000 7f9cb2000640 1 -- 172.21.15.16:0/2358649800 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc00a78e0 con 0x55d0ddbef2f0 2024-08-31T06:10:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.673+0000 7f9cda833740 1 -- 172.21.15.16:0/2358649800 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cc00afec0 con 0x55d0ddbef2f0 2024-08-31T06:10:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.673+0000 7f9cb2000640 1 -- 172.21.15.16:0/2358649800 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f9cc00a7c40 con 0x55d0ddbef2f0 2024-08-31T06:10:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.674+0000 7f9cb2000640 1 -- 172.21.15.16:0/2358649800 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9cc00c6200 con 0x55d0ddbef2f0 2024-08-31T06:10:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.674+0000 7f9cda833740 10 client.26431 fetch_fsmap learned FSMap version 32 2024-08-31T06:10:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.674+0000 7f9cda833740 1 -- 172.21.15.16:0/2358649800 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d0de0440d0 con 0x55d0ddbef2f0 2024-08-31T06:10:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.674+0000 7f9cb2000640 1 -- 172.21.15.16:0/2358649800 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cc00afec0 con 0x55d0ddbef2f0 2024-08-31T06:10:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.674+0000 7f9cb2000640 1 -- 172.21.15.16:0/2358649800 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f9cc00aa490 con 0x55d0ddbef2f0 2024-08-31T06:10:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.674+0000 7f9cda833740 10 client.26431 fetch_fsmap finished waiting for FSMap version 32 2024-08-31T06:10:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.674+0000 7f9cda833740 10 client.26431 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:10:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.674+0000 7f9cda833740 20 client.26431 populate_metadata read hostname 'smithi016' 2024-08-31T06:10:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.674+0000 7f9cda833740 1 --2- 172.21.15.16:0/2358649800 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d0ddbac1c0 0x55d0de046620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.674+0000 7f9cda833740 4 client.26431 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:10:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.674+0000 7f9cda833740 1 -- 172.21.15.16:0/2358649800 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d0ddf2e960 con 0x55d0ddbac1c0 2024-08-31T06:10:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.675+0000 7f9ccbe00640 1 --2- 172.21.15.16:0/2358649800 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d0ddbac1c0 0x55d0de046620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.675+0000 7f9ccbe00640 1 --2- 172.21.15.16:0/2358649800 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d0ddbac1c0 0x55d0de046620 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=818a92cb5b2ea019 server_cookie=86d6573d289bc47c in_seq=0 out_seq=0 2024-08-31T06:10:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.675+0000 7f9cb2000640 10 client.26431 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.691+0000 7f9cb2000640 1 -- 172.21.15.16:0/2358649800 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d0ddf2e960 con 0x55d0ddbac1c0 2024-08-31T06:10:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.691+0000 7f9cb2000640 10 client.26431 handle_command_reply: tid=0 2024-08-31T06:10:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.691+0000 7f9cda833740 1 client.26431 shutdown 2024-08-31T06:10:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.691+0000 7f9cda833740 1 -- 172.21.15.16:0/2358649800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ddbef2f0 msgr2=0x55d0ddfcd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.691+0000 7f9cda833740 1 --2- 172.21.15.16:0/2358649800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ddbef2f0 0x55d0ddfcd780 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0000c00 tx=0x7f9cc00a7510 comp rx=0 tx=0).stop 2024-08-31T06:10:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.691+0000 7f9cda833740 1 -- 172.21.15.16:0/2358649800 shutdown_connections 2024-08-31T06:10:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.691+0000 7f9cda833740 1 --2- 172.21.15.16:0/2358649800 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d0ddbac1c0 0x55d0de046620 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.691+0000 7f9cda833740 1 --2- 172.21.15.16:0/2358649800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0ddc181b0 0x55d0de0428f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.691+0000 7f9cda833740 1 --2- 172.21.15.16:0/2358649800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0ddbf74d0 0x55d0ddfcdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.691+0000 7f9cda833740 1 --2- 172.21.15.16:0/2358649800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ddbef2f0 0x55d0ddfcd780 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.691+0000 7f9cb2000640 0 client.26431 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.691+0000 7f9cda833740 1 -- 172.21.15.16:0/2358649800 >> 172.21.15.16:0/2358649800 conn(0x55d0ddc0c300 msgr2=0x55d0ddfb82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.692+0000 7f9cda833740 1 -- 172.21.15.16:0/2358649800 shutdown_connections 2024-08-31T06:10:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.692+0000 7f9cda833740 1 -- 172.21.15.16:0/2358649800 wait complete. 2024-08-31T06:10:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.692+0000 7f9cda833740 20 client.26431 trim_cache size 0 max 16384 2024-08-31T06:10:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:10:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:10:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:10:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:10:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.692+0000 7f9cd3600640 1 -- 172.21.15.16:0/3400596187 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9cb40b4ca0 msgr2=0x7f9cb40b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.692+0000 7f9cd3600640 1 --2- 172.21.15.16:0/3400596187 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9cb40b4ca0 0x7f9cb40b71b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4005fd0 tx=0x7f9cc4009600 comp rx=0 tx=0).stop 2024-08-31T06:10:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.692+0000 7f9cd3600640 1 -- 172.21.15.16:0/3400596187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ccc105a90 msgr2=0x7f9ccc19c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.692+0000 7f9cd3600640 1 --2- 172.21.15.16:0/3400596187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ccc105a90 0x7f9ccc19c420 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f9cc000d950 tx=0x7f9cc000de10 comp rx=0 tx=0).stop 2024-08-31T06:10:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.692+0000 7f9cd3600640 1 -- 172.21.15.16:0/3400596187 shutdown_connections 2024-08-31T06:10:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.692+0000 7f9cd3600640 1 --2- 172.21.15.16:0/3400596187 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9cb40b4ca0 0x7f9cb40b71b0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.692+0000 7f9cd3600640 1 --2- 172.21.15.16:0/3400596187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ccc10b3c0 0x7f9ccc19c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.692+0000 7f9cd3600640 1 --2- 172.21.15.16:0/3400596187 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ccc10bd10 0x7f9ccc1a1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.692+0000 7f9cd3600640 1 --2- 172.21.15.16:0/3400596187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ccc105a90 0x7f9ccc19c420 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.692+0000 7f9cd3600640 1 -- 172.21.15.16:0/3400596187 >> 172.21.15.16:0/3400596187 conn(0x7f9ccc101470 msgr2=0x7f9ccc104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.693+0000 7f9cd3600640 1 -- 172.21.15.16:0/3400596187 shutdown_connections 2024-08-31T06:10:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:10:51.693+0000 7f9cd3600640 1 -- 172.21.15.16:0/3400596187 wait complete. 2024-08-31T06:10:52.835 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T06:10:52.836 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T06:10:52.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:52 smithi155 ceph-mon[21742]: pgmap v473: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 1.8 MiB/s wr, 218 op/s 2024-08-31T06:10:52.891 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:52 smithi016 ceph-mon[17194]: pgmap v473: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 1.8 MiB/s wr, 218 op/s 2024-08-31T06:10:52.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:52 smithi184 ceph-mon[22331]: pgmap v473: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 1.8 MiB/s wr, 218 op/s 2024-08-31T06:10:53.226 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:10:54.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.040+0000 7fcf9ec00640 1 -- 172.21.15.16:0/4096242219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf98105b20 msgr2=0x7fcf98105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:54.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.040+0000 7fcf9ec00640 1 --2- 172.21.15.16:0/4096242219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf98105b20 0x7fcf98105f30 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fcf88009900 tx=0x7fcf8802a410 comp rx=0 tx=0).stop 2024-08-31T06:10:54.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.040+0000 7fcf9ec00640 1 -- 172.21.15.16:0/4096242219 shutdown_connections 2024-08-31T06:10:54.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.040+0000 7fcf9ec00640 1 --2- 172.21.15.16:0/4096242219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf9810b460 0x7fcf9810b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:54.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.040+0000 7fcf9ec00640 1 --2- 172.21.15.16:0/4096242219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf9810bdb0 0x7fcf981125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:54.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.040+0000 7fcf9ec00640 1 --2- 172.21.15.16:0/4096242219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf98105b20 0x7fcf98105f30 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:54.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.040+0000 7fcf9ec00640 1 -- 172.21.15.16:0/4096242219 >> 172.21.15.16:0/4096242219 conn(0x7fcf98101570 msgr2=0x7fcf981039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:54.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.040+0000 7fcf9ec00640 1 -- 172.21.15.16:0/4096242219 shutdown_connections 2024-08-31T06:10:54.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.040+0000 7fcf9ec00640 1 -- 172.21.15.16:0/4096242219 wait complete. 2024-08-31T06:10:54.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.041+0000 7fcf9ec00640 1 Processor -- start 2024-08-31T06:10:54.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.041+0000 7fcf9ec00640 1 -- start start 2024-08-31T06:10:54.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.042+0000 7fcf9ec00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf98105b20 0x7fcf9819c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:54.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.042+0000 7fcf9ec00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf9810b460 0x7fcf9819ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:54.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.042+0000 7fcf9ec00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf9810bdb0 0x7fcf981a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:54.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.042+0000 7fcf9ec00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcf98115ac0 con 0x7fcf9810bdb0 2024-08-31T06:10:54.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.042+0000 7fcf9ec00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcf98115940 con 0x7fcf9810b460 2024-08-31T06:10:54.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.042+0000 7fcf9ec00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcf98115c40 con 0x7fcf98105b20 2024-08-31T06:10:54.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.042+0000 7fcf9cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf9810bdb0 0x7fcf981a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:54.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.042+0000 7fcf97e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf98105b20 0x7fcf9819c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:54.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.042+0000 7fcf97e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf98105b20 0x7fcf9819c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:49556/0 (socket says 172.21.15.16:49556) 2024-08-31T06:10:54.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.042+0000 7fcf9cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf9810bdb0 0x7fcf981a1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37164/0 (socket says 172.21.15.16:37164) 2024-08-31T06:10:54.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.042+0000 7fcf97e00640 1 -- 172.21.15.16:0/1356243912 learned_addr learned my addr 172.21.15.16:0/1356243912 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:54.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.042+0000 7fcf8fe00640 1 --2- 172.21.15.16:0/1356243912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf9810b460 0x7fcf9819ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:54.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.043+0000 7fcf97e00640 1 -- 172.21.15.16:0/1356243912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf9810b460 msgr2=0x7fcf9819ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:54.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.043+0000 7fcf97e00640 1 --2- 172.21.15.16:0/1356243912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf9810b460 0x7fcf9819ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:54.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.043+0000 7fcf97e00640 1 -- 172.21.15.16:0/1356243912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf9810bdb0 msgr2=0x7fcf981a1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:54.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.043+0000 7fcf97e00640 1 --2- 172.21.15.16:0/1356243912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf9810bdb0 0x7fcf981a1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:54.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.043+0000 7fcf97e00640 1 -- 172.21.15.16:0/1356243912 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf981a2400 con 0x7fcf98105b20 2024-08-31T06:10:54.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.043+0000 7fcf9cc00640 1 --2- 172.21.15.16:0/1356243912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf9810bdb0 0x7fcf981a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:10:54.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.043+0000 7fcf97e00640 1 --2- 172.21.15.16:0/1356243912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf98105b20 0x7fcf9819c490 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcf88005ab0 tx=0x7fcf88003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:54.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.043+0000 7fcf95600640 1 -- 172.21.15.16:0/1356243912 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8803f070 con 0x7fcf98105b20 2024-08-31T06:10:54.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.044+0000 7fcf9ec00640 1 -- 172.21.15.16:0/1356243912 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf981a2690 con 0x7fcf98105b20 2024-08-31T06:10:54.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.044+0000 7fcf95600640 1 -- 172.21.15.16:0/1356243912 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf88031b60 con 0x7fcf98105b20 2024-08-31T06:10:54.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.044+0000 7fcf95600640 1 -- 172.21.15.16:0/1356243912 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf88035030 con 0x7fcf98105b20 2024-08-31T06:10:54.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.044+0000 7fcf9ec00640 1 -- 172.21.15.16:0/1356243912 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf981a8e50 con 0x7fcf98105b20 2024-08-31T06:10:54.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.047+0000 7fcfa581e740 1 Processor -- start 2024-08-31T06:10:54.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.048+0000 7fcfa581e740 1 -- start start 2024-08-31T06:10:54.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.048+0000 7fcfa581e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f73c4350 0x55e5f776a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:54.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.048+0000 7fcfa581e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5f739b490 0x55e5f776ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:54.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.048+0000 7fcfa581e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5f73a3670 0x55e5f776b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:54.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.048+0000 7fcfa581e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e5f76e0750 con 0x55e5f73c4350 2024-08-31T06:10:54.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.048+0000 7fcfa581e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e5f76cfce0 con 0x55e5f73a3670 2024-08-31T06:10:54.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.048+0000 7fcfa581e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e5f76d3c90 con 0x55e5f739b490 2024-08-31T06:10:54.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.048+0000 7fcf9cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f73c4350 0x55e5f776a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:54.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.048+0000 7fcf9cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f73c4350 0x55e5f776a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37180/0 (socket says 172.21.15.16:37180) 2024-08-31T06:10:54.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.049+0000 7fcf9cc00640 1 -- 172.21.15.16:0/2464220710 learned_addr learned my addr 172.21.15.16:0/2464220710 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:54.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.049+0000 7fcf8fe00640 1 --2- 172.21.15.16:0/2464220710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5f73a3670 0x55e5f776b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:54.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.049+0000 7fcf9cc00640 1 -- 172.21.15.16:0/2464220710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5f739b490 msgr2=0x55e5f776ab40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:10:54.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.049+0000 7fcf9cc00640 1 --2- 172.21.15.16:0/2464220710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5f739b490 0x55e5f776ab40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:54.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.049+0000 7fcf9cc00640 1 -- 172.21.15.16:0/2464220710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5f73a3670 msgr2=0x55e5f776b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:54.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.049+0000 7fcf9cc00640 1 --2- 172.21.15.16:0/2464220710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5f73a3670 0x55e5f776b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:54.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.049+0000 7fcf95600640 1 -- 172.21.15.16:0/1356243912 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fcf88042070 con 0x7fcf98105b20 2024-08-31T06:10:54.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.049+0000 7fcf9cc00640 1 -- 172.21.15.16:0/2464220710 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5f742f930 con 0x55e5f73c4350 2024-08-31T06:10:54.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.049+0000 7fcf9cc00640 1 --2- 172.21.15.16:0/2464220710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f73c4350 0x55e5f776a600 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fcf900117e0 tx=0x7fcf90011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b623412ab0b316c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:54.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.050+0000 7fcf8d200640 1 -- 172.21.15.16:0/2464220710 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf90011310 con 0x55e5f73c4350 2024-08-31T06:10:54.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.050+0000 7fcf8d200640 1 -- 172.21.15.16:0/2464220710 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf90003e20 con 0x55e5f73c4350 2024-08-31T06:10:54.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.050+0000 7fcf8d200640 1 -- 172.21.15.16:0/2464220710 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf90004490 con 0x55e5f73c4350 2024-08-31T06:10:54.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.050+0000 7fcfa581e740 1 -- 172.21.15.16:0/2464220710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f73c4350 msgr2=0x55e5f776a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:54.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.050+0000 7fcfa581e740 1 --2- 172.21.15.16:0/2464220710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f73c4350 0x55e5f776a600 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fcf900117e0 tx=0x7fcf90011ca0 comp rx=0 tx=0).stop 2024-08-31T06:10:54.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.051+0000 7fcfa581e740 1 -- 172.21.15.16:0/2464220710 shutdown_connections 2024-08-31T06:10:54.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.051+0000 7fcfa581e740 1 --2- 172.21.15.16:0/2464220710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5f739b490 0x55e5f776ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:54.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.051+0000 7fcfa581e740 1 --2- 172.21.15.16:0/2464220710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5f73a3670 0x55e5f776b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:54.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.051+0000 7fcfa581e740 1 --2- 172.21.15.16:0/2464220710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f73c4350 0x55e5f776a600 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:54.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.051+0000 7fcfa581e740 1 -- 172.21.15.16:0/2464220710 >> 172.21.15.16:0/2464220710 conn(0x55e5f73b8740 msgr2=0x55e5f77684f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:54.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.051+0000 7fcf95600640 1 --2- 172.21.15.16:0/1356243912 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fcf7c0b4c50 0x7fcf7c0b7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:54.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.051+0000 7fcfa581e740 1 -- 172.21.15.16:0/2464220710 shutdown_connections 2024-08-31T06:10:54.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.051+0000 7fcfa581e740 1 -- 172.21.15.16:0/2464220710 wait complete. 2024-08-31T06:10:54.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.051+0000 7fcf95600640 1 -- 172.21.15.16:0/1356243912 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fcf881058c0 con 0x7fcf98105b20 2024-08-31T06:10:54.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.051+0000 7fcf8fe00640 1 --2- 172.21.15.16:0/1356243912 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fcf7c0b4c50 0x7fcf7c0b7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:54.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.052+0000 7fcfa581e740 1 -- start start 2024-08-31T06:10:54.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.052+0000 7fcf8fe00640 1 --2- 172.21.15.16:0/1356243912 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fcf7c0b4c50 0x7fcf7c0b7160 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcf9810ae00 tx=0x7fcf80005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:54.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.052+0000 7fcfa581e740 1 Processor -- start 2024-08-31T06:10:54.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.053+0000 7fcfa581e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5f739b490 0x55e5f7776240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:54.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.053+0000 7fcfa581e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5f73a3670 0x55e5f7776800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:54.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.053+0000 7fcfa581e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f73c4350 0x55e5f77820d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:54.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.053+0000 7fcfa581e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e5f7532890 con 0x55e5f73c4350 2024-08-31T06:10:54.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.053+0000 7fcfa581e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e5f7656cd0 con 0x55e5f739b490 2024-08-31T06:10:54.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.053+0000 7fcfa581e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e5f7778de0 con 0x55e5f73a3670 2024-08-31T06:10:54.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.053+0000 7fcfa581e740 10 client.0 _ll_register_callbacks cb 0x55e5f76caef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:10:54.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.053+0000 7fcf97e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5f739b490 0x55e5f7776240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:54.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.053+0000 7fcf9cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f73c4350 0x55e5f77820d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:54.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.053+0000 7fcf97e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5f739b490 0x55e5f7776240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:36788/0 (socket says 172.21.15.16:36788) 2024-08-31T06:10:54.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.053+0000 7fcf97e00640 1 -- 172.21.15.16:0/2159491537 learned_addr learned my addr 172.21.15.16:0/2159491537 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:54.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.053+0000 7fcf8fe00640 1 --2- 172.21.15.16:0/2159491537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5f73a3670 0x55e5f7776800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:54.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.053+0000 7fcf97e00640 1 -- 172.21.15.16:0/2159491537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5f73a3670 msgr2=0x55e5f7776800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:54.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.053+0000 7fcf97e00640 1 --2- 172.21.15.16:0/2159491537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5f73a3670 0x55e5f7776800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:54.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.054+0000 7fcf97e00640 1 -- 172.21.15.16:0/2159491537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f73c4350 msgr2=0x55e5f77820d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:54.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.054+0000 7fcf97e00640 1 --2- 172.21.15.16:0/2159491537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f73c4350 0x55e5f77820d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:54.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.054+0000 7fcf97e00640 1 -- 172.21.15.16:0/2159491537 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5f76a01b0 con 0x55e5f739b490 2024-08-31T06:10:54.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.054+0000 7fcf9cc00640 1 --2- 172.21.15.16:0/2159491537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f73c4350 0x55e5f77820d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:10:54.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.054+0000 7fcf97e00640 1 --2- 172.21.15.16:0/2159491537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5f739b490 0x55e5f7776240 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fcf88031440 tx=0x7fcf880cbae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6a58f7ae44ae7a1 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:54.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.054+0000 7fcf85600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:10:54.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.054+0000 7fcf85600640 1 -- 172.21.15.16:0/2159491537 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf88049030 con 0x55e5f739b490 2024-08-31T06:10:54.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.054+0000 7fcf85600640 1 -- 172.21.15.16:0/2159491537 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fcf880d0470 con 0x55e5f739b490 2024-08-31T06:10:54.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.054+0000 7fcfa581e740 1 -- 172.21.15.16:0/2159491537 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf90003e20 con 0x55e5f739b490 2024-08-31T06:10:54.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.054+0000 7fcf85600640 1 -- 172.21.15.16:0/2159491537 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcf880d0820 con 0x55e5f739b490 2024-08-31T06:10:54.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.055+0000 7fcfa581e740 10 client.26447 fetch_fsmap learned FSMap version 32 2024-08-31T06:10:54.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.055+0000 7fcfa581e740 1 -- 172.21.15.16:0/2159491537 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e5f7784e80 con 0x55e5f739b490 2024-08-31T06:10:54.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.055+0000 7fcf85600640 1 -- 172.21.15.16:0/2159491537 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf88079070 con 0x55e5f739b490 2024-08-31T06:10:54.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.055+0000 7fcf85600640 1 -- 172.21.15.16:0/2159491537 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7fcf8804c060 con 0x55e5f739b490 2024-08-31T06:10:54.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.055+0000 7fcfa581e740 10 client.26447 fetch_fsmap finished waiting for FSMap version 32 2024-08-31T06:10:54.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.055+0000 7fcfa581e740 10 client.26447 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:10:54.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.055+0000 7fcfa581e740 20 client.26447 populate_metadata read hostname 'smithi016' 2024-08-31T06:10:54.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.055+0000 7fcfa581e740 1 --2- 172.21.15.16:0/2159491537 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55e5f7382cd0 0x55e5f7788280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:54.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.055+0000 7fcfa581e740 4 client.26447 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:10:54.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.055+0000 7fcfa581e740 1 -- 172.21.15.16:0/2159491537 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5f76e26a0 con 0x55e5f7382cd0 2024-08-31T06:10:54.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.056+0000 7fcf8fe00640 1 --2- 172.21.15.16:0/2159491537 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55e5f7382cd0 0x55e5f7788280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:54.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.056+0000 7fcf8fe00640 1 --2- 172.21.15.16:0/2159491537 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55e5f7382cd0 0x55e5f7788280 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6dab6ba06bd6ca4 server_cookie=34d09a2e0333582 in_seq=0 out_seq=0 2024-08-31T06:10:54.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.056+0000 7fcf85600640 10 client.26447 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:54.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.057+0000 7fcf85600640 1 -- 172.21.15.16:0/2159491537 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e5f76e26a0 con 0x55e5f7382cd0 2024-08-31T06:10:54.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.057+0000 7fcf85600640 10 client.26447 handle_command_reply: tid=0 2024-08-31T06:10:54.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.057+0000 7fcfa581e740 1 client.26447 shutdown 2024-08-31T06:10:54.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.057+0000 7fcfa581e740 1 -- 172.21.15.16:0/2159491537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5f739b490 msgr2=0x55e5f7776240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:54.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.057+0000 7fcfa581e740 1 --2- 172.21.15.16:0/2159491537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5f739b490 0x55e5f7776240 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fcf88031440 tx=0x7fcf880cbae0 comp rx=0 tx=0).stop 2024-08-31T06:10:54.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.057+0000 7fcfa581e740 1 -- 172.21.15.16:0/2159491537 shutdown_connections 2024-08-31T06:10:54.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.057+0000 7fcfa581e740 1 --2- 172.21.15.16:0/2159491537 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55e5f7382cd0 0x55e5f7788280 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:54.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.057+0000 7fcfa581e740 1 --2- 172.21.15.16:0/2159491537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5f73a3670 0x55e5f7776800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:54.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.057+0000 7fcfa581e740 1 --2- 172.21.15.16:0/2159491537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f73c4350 0x55e5f77820d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:54.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.057+0000 7fcfa581e740 1 --2- 172.21.15.16:0/2159491537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5f739b490 0x55e5f7776240 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:54.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.057+0000 7fcf85600640 0 client.26447 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:54.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.057+0000 7fcfa581e740 1 -- 172.21.15.16:0/2159491537 >> 172.21.15.16:0/2159491537 conn(0x55e5f73b8740 msgr2=0x55e5f776d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:54.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.058+0000 7fcfa581e740 1 -- 172.21.15.16:0/2159491537 shutdown_connections 2024-08-31T06:10:54.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.058+0000 7fcfa581e740 1 -- 172.21.15.16:0/2159491537 wait complete. 2024-08-31T06:10:54.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.058+0000 7fcfa581e740 20 client.26447 trim_cache size 0 max 16384 2024-08-31T06:10:54.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.076+0000 7fcfa581e740 1 Processor -- start 2024-08-31T06:10:54.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.076+0000 7fcfa581e740 1 -- start start 2024-08-31T06:10:54.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.077+0000 7fcfa581e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f739b490 0x55e5f776f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:54.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.077+0000 7fcfa581e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5f73a3670 0x55e5f776fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:54.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.077+0000 7fcfa581e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5f73c4350 0x55e5f776d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:54.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.077+0000 7fcfa581e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981b3f10 con 0x55e5f739b490 2024-08-31T06:10:54.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.077+0000 7fcfa581e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981b1750 con 0x55e5f73a3670 2024-08-31T06:10:54.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.077+0000 7fcfa581e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981af5a0 con 0x55e5f73c4350 2024-08-31T06:10:54.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.077+0000 7fcf97e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f739b490 0x55e5f776f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:54.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.077+0000 7fcf97e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f739b490 0x55e5f776f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37192/0 (socket says 172.21.15.16:37192) 2024-08-31T06:10:54.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.077+0000 7fcf97e00640 1 -- 172.21.15.16:0/2750226422 learned_addr learned my addr 172.21.15.16:0/2750226422 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:54.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.077+0000 7fcf8fe00640 1 --2- 172.21.15.16:0/2750226422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5f73a3670 0x55e5f776fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:54.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.077+0000 7fcf9cc00640 1 --2- 172.21.15.16:0/2750226422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5f73c4350 0x55e5f776d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:54.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.077+0000 7fcf97e00640 1 -- 172.21.15.16:0/2750226422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5f73c4350 msgr2=0x55e5f776d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:54.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.077+0000 7fcf97e00640 1 --2- 172.21.15.16:0/2750226422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5f73c4350 0x55e5f776d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:54.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.077+0000 7fcf97e00640 1 -- 172.21.15.16:0/2750226422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5f73a3670 msgr2=0x55e5f776fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:54.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.077+0000 7fcf97e00640 1 --2- 172.21.15.16:0/2750226422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5f73a3670 0x55e5f776fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:54.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.077+0000 7fcf97e00640 1 -- 172.21.15.16:0/2750226422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5f77718d0 con 0x55e5f739b490 2024-08-31T06:10:54.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.077+0000 7fcf8fe00640 1 --2- 172.21.15.16:0/2750226422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5f73a3670 0x55e5f776fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:10:54.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.078+0000 7fcf97e00640 1 --2- 172.21.15.16:0/2750226422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f739b490 0x55e5f776f880 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fcf880cc520 tx=0x7fcf880cb9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc67e091ddf512c6 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:54.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.078+0000 7fcf87e00640 1 -- 172.21.15.16:0/2750226422 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf88049030 con 0x55e5f739b490 2024-08-31T06:10:54.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.078+0000 7fcf87e00640 1 -- 172.21.15.16:0/2750226422 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf88079070 con 0x55e5f739b490 2024-08-31T06:10:54.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.078+0000 7fcf87e00640 1 -- 172.21.15.16:0/2750226422 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8803b460 con 0x55e5f739b490 2024-08-31T06:10:54.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.079+0000 7fcfa581e740 1 -- 172.21.15.16:0/2750226422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f739b490 msgr2=0x55e5f776f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:54.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.079+0000 7fcfa581e740 1 --2- 172.21.15.16:0/2750226422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f739b490 0x55e5f776f880 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fcf880cc520 tx=0x7fcf880cb9b0 comp rx=0 tx=0).stop 2024-08-31T06:10:54.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.079+0000 7fcfa581e740 1 -- 172.21.15.16:0/2750226422 shutdown_connections 2024-08-31T06:10:54.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.079+0000 7fcfa581e740 1 --2- 172.21.15.16:0/2750226422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5f73c4350 0x55e5f776d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:54.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.079+0000 7fcfa581e740 1 --2- 172.21.15.16:0/2750226422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5f73a3670 0x55e5f776fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:54.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.079+0000 7fcfa581e740 1 --2- 172.21.15.16:0/2750226422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f739b490 0x55e5f776f880 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:54.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.079+0000 7fcfa581e740 1 -- 172.21.15.16:0/2750226422 >> 172.21.15.16:0/2750226422 conn(0x55e5f73b8740 msgr2=0x55e5f77748c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:54.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.079+0000 7fcfa581e740 1 -- 172.21.15.16:0/2750226422 shutdown_connections 2024-08-31T06:10:54.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.079+0000 7fcfa581e740 1 -- 172.21.15.16:0/2750226422 wait complete. 2024-08-31T06:10:54.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.079+0000 7fcfa581e740 1 -- start start 2024-08-31T06:10:54.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.080+0000 7fcfa581e740 1 Processor -- start 2024-08-31T06:10:54.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.080+0000 7fcfa581e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5f739b490 0x55e5f777aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:54.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.080+0000 7fcfa581e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f73a3670 0x55e5f777b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:54.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.080+0000 7fcfa581e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5f73c4350 0x55e5f77f44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:54.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.080+0000 7fcfa581e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981afc10 con 0x55e5f73a3670 2024-08-31T06:10:54.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.080+0000 7fcfa581e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981b1080 con 0x55e5f739b490 2024-08-31T06:10:54.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.080+0000 7fcfa581e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981b02e0 con 0x55e5f73c4350 2024-08-31T06:10:54.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.080+0000 7fcfa581e740 10 client.0 _ll_register_callbacks cb 0x7fcf88005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:10:54.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.081+0000 7fcf97e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5f739b490 0x55e5f777aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:54.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.081+0000 7fcf8fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f73a3670 0x55e5f777b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:54.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.081+0000 7fcf97e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5f739b490 0x55e5f777aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:36798/0 (socket says 172.21.15.16:36798) 2024-08-31T06:10:54.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.081+0000 7fcf97e00640 1 -- 172.21.15.16:0/3039475701 learned_addr learned my addr 172.21.15.16:0/3039475701 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:54.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.081+0000 7fcf9cc00640 1 --2- 172.21.15.16:0/3039475701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5f73c4350 0x55e5f77f44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:54.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.081+0000 7fcf97e00640 1 -- 172.21.15.16:0/3039475701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5f73c4350 msgr2=0x55e5f77f44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:54.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.081+0000 7fcf97e00640 1 --2- 172.21.15.16:0/3039475701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5f73c4350 0x55e5f77f44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:54.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.081+0000 7fcf97e00640 1 -- 172.21.15.16:0/3039475701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f73a3670 msgr2=0x55e5f777b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:54.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.081+0000 7fcf97e00640 1 --2- 172.21.15.16:0/3039475701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f73a3670 0x55e5f777b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:54.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.081+0000 7fcf97e00640 1 -- 172.21.15.16:0/3039475701 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5f77f4b20 con 0x55e5f739b490 2024-08-31T06:10:54.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.081+0000 7fcf97e00640 1 --2- 172.21.15.16:0/3039475701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5f739b490 0x55e5f777aa60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcf8802a9d0 tx=0x7fcf88005cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ca952c24e4cc8f5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:54.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.081+0000 7fcf85600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:10:54.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.082+0000 7fcf85600640 1 -- 172.21.15.16:0/3039475701 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf88049070 con 0x55e5f739b490 2024-08-31T06:10:54.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.082+0000 7fcf85600640 1 -- 172.21.15.16:0/3039475701 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fcf8803b460 con 0x55e5f739b490 2024-08-31T06:10:54.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.082+0000 7fcfa581e740 1 -- 172.21.15.16:0/3039475701 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf88079070 con 0x55e5f739b490 2024-08-31T06:10:54.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.082+0000 7fcf85600640 1 -- 172.21.15.16:0/3039475701 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcf880cbe30 con 0x55e5f739b490 2024-08-31T06:10:54.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.082+0000 7fcfa581e740 10 client.26453 fetch_fsmap learned FSMap version 32 2024-08-31T06:10:54.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.082+0000 7fcfa581e740 1 -- 172.21.15.16:0/3039475701 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e5f77f4b20 con 0x55e5f739b490 2024-08-31T06:10:54.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.082+0000 7fcf85600640 1 -- 172.21.15.16:0/3039475701 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf88079070 con 0x55e5f739b490 2024-08-31T06:10:54.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.082+0000 7fcf85600640 1 -- 172.21.15.16:0/3039475701 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7fcf880ed300 con 0x55e5f739b490 2024-08-31T06:10:54.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.082+0000 7fcfa581e740 10 client.26453 fetch_fsmap finished waiting for FSMap version 32 2024-08-31T06:10:54.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.082+0000 7fcfa581e740 10 client.26453 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:10:54.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.082+0000 7fcfa581e740 20 client.26453 populate_metadata read hostname 'smithi016' 2024-08-31T06:10:54.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.082+0000 7fcfa581e740 1 --2- 172.21.15.16:0/3039475701 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55e5f7358080 0x55e5f77fe340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:54.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.082+0000 7fcfa581e740 4 client.26453 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T06:10:54.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.082+0000 7fcfa581e740 1 -- 172.21.15.16:0/3039475701 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e5f76de140 con 0x55e5f7358080 2024-08-31T06:10:54.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.083+0000 7fcf8fe00640 1 --2- 172.21.15.16:0/3039475701 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55e5f7358080 0x55e5f77fe340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:54.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.083+0000 7fcf8fe00640 1 --2- 172.21.15.16:0/3039475701 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55e5f7358080 0x55e5f77fe340 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=560b0a4f85790856 server_cookie=54ca32fb7a8accc5 in_seq=0 out_seq=0 2024-08-31T06:10:54.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.083+0000 7fcf85600640 10 client.26453 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:54.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.085+0000 7fcf85600640 1 -- 172.21.15.16:0/3039475701 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e5f76de140 con 0x55e5f7358080 2024-08-31T06:10:54.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.085+0000 7fcf85600640 10 client.26453 handle_command_reply: tid=0 2024-08-31T06:10:54.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.085+0000 7fcfa581e740 1 client.26453 shutdown 2024-08-31T06:10:54.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.085+0000 7fcfa581e740 1 -- 172.21.15.16:0/3039475701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5f739b490 msgr2=0x55e5f777aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:54.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.085+0000 7fcfa581e740 1 --2- 172.21.15.16:0/3039475701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5f739b490 0x55e5f777aa60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcf8802a9d0 tx=0x7fcf88005cb0 comp rx=0 tx=0).stop 2024-08-31T06:10:54.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.085+0000 7fcfa581e740 1 -- 172.21.15.16:0/3039475701 shutdown_connections 2024-08-31T06:10:54.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.085+0000 7fcfa581e740 1 --2- 172.21.15.16:0/3039475701 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55e5f7358080 0x55e5f77fe340 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:54.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.085+0000 7fcfa581e740 1 --2- 172.21.15.16:0/3039475701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5f73c4350 0x55e5f77f44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:54.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.085+0000 7fcfa581e740 1 --2- 172.21.15.16:0/3039475701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f73a3670 0x55e5f777b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:54.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.085+0000 7fcfa581e740 1 --2- 172.21.15.16:0/3039475701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5f739b490 0x55e5f777aa60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:54.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.085+0000 7fcfa581e740 1 -- 172.21.15.16:0/3039475701 >> 172.21.15.16:0/3039475701 conn(0x55e5f73b8740 msgr2=0x55e5f776d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:54.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.085+0000 7fcf85600640 0 client.26453 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:54.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.086+0000 7fcfa581e740 1 -- 172.21.15.16:0/3039475701 shutdown_connections 2024-08-31T06:10:54.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.086+0000 7fcfa581e740 1 -- 172.21.15.16:0/3039475701 wait complete. 2024-08-31T06:10:54.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.086+0000 7fcfa581e740 20 client.26453 trim_cache size 0 max 16384 2024-08-31T06:10:54.090 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:10:54.091 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-08-31T06:10:54.091 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "e739a16e-8ace-43a8-a1ce-ac27d39ef8a3", 2024-08-31T06:10:54.091 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-08-31T06:10:54.091 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:10:54.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.087+0000 7fcf9ec00640 1 -- 172.21.15.16:0/1356243912 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fcf7c0b4c50 msgr2=0x7fcf7c0b7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:54.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.087+0000 7fcf9ec00640 1 --2- 172.21.15.16:0/1356243912 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fcf7c0b4c50 0x7fcf7c0b7160 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcf9810ae00 tx=0x7fcf80005f90 comp rx=0 tx=0).stop 2024-08-31T06:10:54.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.087+0000 7fcf9ec00640 1 -- 172.21.15.16:0/1356243912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf98105b20 msgr2=0x7fcf9819c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:54.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.087+0000 7fcf9ec00640 1 --2- 172.21.15.16:0/1356243912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf98105b20 0x7fcf9819c490 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcf88005ab0 tx=0x7fcf88003c10 comp rx=0 tx=0).stop 2024-08-31T06:10:54.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.087+0000 7fcf9ec00640 1 -- 172.21.15.16:0/1356243912 shutdown_connections 2024-08-31T06:10:54.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.087+0000 7fcf9ec00640 1 --2- 172.21.15.16:0/1356243912 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fcf7c0b4c50 0x7fcf7c0b7160 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:54.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.087+0000 7fcf9ec00640 1 --2- 172.21.15.16:0/1356243912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf9810bdb0 0x7fcf981a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:54.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.087+0000 7fcf9ec00640 1 --2- 172.21.15.16:0/1356243912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf9810b460 0x7fcf9819ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:54.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.087+0000 7fcf9ec00640 1 --2- 172.21.15.16:0/1356243912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf98105b20 0x7fcf9819c490 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:54.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.087+0000 7fcf9ec00640 1 -- 172.21.15.16:0/1356243912 >> 172.21.15.16:0/1356243912 conn(0x7fcf98101570 msgr2=0x7fcf98103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:54.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.087+0000 7fcf9ec00640 1 -- 172.21.15.16:0/1356243912 shutdown_connections 2024-08-31T06:10:54.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:54.087+0000 7fcf9ec00640 1 -- 172.21.15.16:0/1356243912 wait complete. 2024-08-31T06:10:54.372 INFO:journalctl@ceph.mds.b.smithi155.stdout:Aug 31 06:10:54 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-b[53353]: 2024-08-31T06:10:54.097+0000 7f6c22b03640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000019 [...6,head] /volumes/qa/sv_1/ba5423bc-5c83-4d64-ae1c-ad8850110ce4/postgres/data/pg_stat_tmp/ auth{1=1,2=1} v1356 ap=3 f(v3 m2024-08-31T06:10:40.594789+0000 4=4+0) n(v6 rc2024-08-31T06:10:40.594789+0000 b14082 5=4+1) old_inodes=3 (inest sync->mix g=1,2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:3737 nref=3)) mcw={1=AsLsXsFs} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55849899d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000019:[<0x10000000006/pg_stat_tmp v438>,<0x10000000005/data v106>,<0x100000001fb/postgres v19>,<0x100000001fa/ba5423bc-5c83-4d64-ae1c-ad8850110ce4 v35>,<0x10000000001/sv_1 v56>,<0x10000000000/qa v44>,<0x1/volumes v40>]//[]","memoryvalue":"(2)0x10000000019:[<0x10000000006/pg_stat_tmp v1356>,<0x10000000005/data v386>,<0x100000001fb/postgres v199>,<0x100000001fa/ba5423bc-5c83-4d64-ae1c-ad8850110ce4 v220>,<0x10000000001/sv_1 v238>,<0x10000000000/qa v224>,<0x1/volumes v208>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-08-31T06:09:31.242851+0000 3=3+0)","ondisk_value.rstat":"n(v0 rc2024-08-31T06:10:40.594789+0000 b14082 5=4+1)","memory_value.dirstat":"f(v3 m2024-08-31T06:10:40.594789+0000 4=4+0)","memory_value.rstat":"n(v6 rc2024-08-31T06:10:40.594789+0000 b14082 5=4+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-08-31T06:10:54.372 INFO:journalctl@ceph.mds.b.smithi155.stdout:Aug 31 06:10:54 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-b[53353]: 2024-08-31T06:10:54.103+0000 7f6c22b03640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...6,head] /volumes/qa/sv_1/ba5423bc-5c83-4d64-ae1c-ad8850110ce4/postgres/data/pg_wal/ auth{1=1,2=1} v1362 ap=3 f(v8 m2024-08-31T06:10:12.475219+0000 35=34+1) n(v14 rc2024-08-31T06:10:40.626789+0000 b570425344 36=34+2) old_inodes=3 (inest sync->mix g=1,2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:3750 nref=3)) caps={15237=pAsLsXsFs/pAsLsXsFsx@207} mcw={1=AsLsXsFs} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x5584989b8c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000007:[<0x10000000006/pg_wal v604>,<0x10000000005/data v106>,<0x100000001fb/postgres v19>,<0x100000001fa/ba5423bc-5c83-4d64-ae1c-ad8850110ce4 v35>,<0x10000000001/sv_1 v56>,<0x10000000000/qa v44>,<0x1/volumes v40>]//[]","memoryvalue":"(2)0x10000000007:[<0x10000000006/pg_wal v1362>,<0x10000000005/data v386>,<0x100000001fb/postgres v199>,<0x100000001fa/ba5423bc-5c83-4d64-ae1c-ad8850110ce4 v220>,<0x10000000001/sv_1 v238>,<0x10000000000/qa v224>,<0x1/volumes v208>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-08-31T06:09:48.905580+0000 34=33+1)","ondisk_value.rstat":"n(v0 rc2024-08-31T06:10:40.626789+0000 b570425344 36=34+2)","memory_value.dirstat":"f(v8 m2024-08-31T06:10:12.475219+0000 35=34+1)","memory_value.rstat":"n(v14 rc2024-08-31T06:10:40.626789+0000 b570425344 36=34+2)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-08-31T06:10:54.372 INFO:journalctl@ceph.mds.b.smithi155.stdout:Aug 31 06:10:54 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-b[53353]: 2024-08-31T06:10:54.114+0000 7f6c22b03640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000000f [...6,head] /volumes/qa/sv_1/ba5423bc-5c83-4d64-ae1c-ad8850110ce4/postgres/data/pg_subtrans/ auth{1=1,2=1} v1364 ap=3 f(v3 m2024-08-31T06:09:59.724415+0000 4=4+0) n(v11 rc2024-08-31T06:10:39.779802+0000 b909312 5=4+1) old_inodes=3 (inest sync->mix g=1,2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:3756 nref=3)) caps={15237=pAsLsXsFs/pAsLsXsFsx@13} mcw={1=AsLsXsFs} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x558498a41600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000000f:[<0x10000000006/pg_subtrans v252>,<0x10000000005/data v106>,<0x100000001fb/postgres v19>,<0x100000001fa/ba5423bc-5c83-4d64-ae1c-ad8850110ce4 v35>,<0x10000000001/sv_1 v56>,<0x10000000000/qa v44>,<0x1/volumes v40>]//[]","memoryvalue":"(2)0x1000000000f:[<0x10000000006/pg_subtrans v1364>,<0x10000000005/data v386>,<0x100000001fb/postgres v199>,<0x100000001fa/ba5423bc-5c83-4d64-ae1c-ad8850110ce4 v220>,<0x10000000001/sv_1 v238>,<0x10000000000/qa v224>,<0x1/volumes v208>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-08-31T06:09:35.028793+0000 3=3+0)","ondisk_value.rstat":"n(v0 rc2024-08-31T06:10:39.779802+0000 b909312 5=4+1)","memory_value.dirstat":"f(v3 m2024-08-31T06:09:59.724415+0000 4=4+0)","memory_value.rstat":"n(v11 rc2024-08-31T06:10:39.779802+0000 b909312 5=4+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-08-31T06:10:54.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:54 smithi155 ceph-mon[21742]: pgmap v474: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 1.2 MiB/s wr, 145 op/s 2024-08-31T06:10:54.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:54 smithi155 ceph-mon[21742]: scrub queued for path: / 2024-08-31T06:10:54.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:54 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:10:54.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:54 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:10:54.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:54 smithi155 ceph-mon[21742]: Scrub error on inode 0x10000000019 (/volumes/qa/sv_1/ba5423bc-5c83-4d64-ae1c-ad8850110ce4/postgres/data/pg_stat_tmp) see mds.b log and `damage ls` output for details 2024-08-31T06:10:54.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:54 smithi155 ceph-mon[21742]: Scrub error on inode 0x10000000007 (/volumes/qa/sv_1/ba5423bc-5c83-4d64-ae1c-ad8850110ce4/postgres/data/pg_wal) see mds.b log and `damage ls` output for details 2024-08-31T06:10:54.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:54 smithi155 ceph-mon[21742]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_1/ba5423bc-5c83-4d64-ae1c-ad8850110ce4/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-08-31T06:10:54.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:54 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:10:54.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:54 smithi016 ceph-mon[17194]: pgmap v474: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 1.2 MiB/s wr, 145 op/s 2024-08-31T06:10:54.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:54 smithi016 ceph-mon[17194]: scrub queued for path: / 2024-08-31T06:10:54.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:54 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:10:54.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:54 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:10:54.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:54 smithi016 ceph-mon[17194]: Scrub error on inode 0x10000000019 (/volumes/qa/sv_1/ba5423bc-5c83-4d64-ae1c-ad8850110ce4/postgres/data/pg_stat_tmp) see mds.b log and `damage ls` output for details 2024-08-31T06:10:54.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:54 smithi016 ceph-mon[17194]: Scrub error on inode 0x10000000007 (/volumes/qa/sv_1/ba5423bc-5c83-4d64-ae1c-ad8850110ce4/postgres/data/pg_wal) see mds.b log and `damage ls` output for details 2024-08-31T06:10:54.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:54 smithi016 ceph-mon[17194]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_1/ba5423bc-5c83-4d64-ae1c-ad8850110ce4/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-08-31T06:10:54.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:54 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:10:54.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:54 smithi184 ceph-mon[22331]: pgmap v474: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 1.2 MiB/s wr, 145 op/s 2024-08-31T06:10:54.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:54 smithi184 ceph-mon[22331]: scrub queued for path: / 2024-08-31T06:10:54.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:54 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:10:54.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:54 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:10:54.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:54 smithi184 ceph-mon[22331]: Scrub error on inode 0x10000000019 (/volumes/qa/sv_1/ba5423bc-5c83-4d64-ae1c-ad8850110ce4/postgres/data/pg_stat_tmp) see mds.b log and `damage ls` output for details 2024-08-31T06:10:54.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:54 smithi184 ceph-mon[22331]: Scrub error on inode 0x10000000007 (/volumes/qa/sv_1/ba5423bc-5c83-4d64-ae1c-ad8850110ce4/postgres/data/pg_wal) see mds.b log and `damage ls` output for details 2024-08-31T06:10:54.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:54 smithi184 ceph-mon[22331]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_1/ba5423bc-5c83-4d64-ae1c-ad8850110ce4/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-08-31T06:10:54.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:54 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:10:55.002 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T06:10:55.381 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:10:55.822 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:55 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:10:55.822 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:55 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:10:55.822 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:55 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:10:55.822 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:55 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:10:55.823 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:55 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:10:55.823 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:55 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:10:55.823 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:55 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:10:55.823 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:55 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:10:55.823 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:55 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:10:55.823 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:55 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:10:55.823 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:55 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:10:55.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:55 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:10:55.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:55 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:10:55.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:55 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:10:55.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:55 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:10:55.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:55 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:10:55.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:55 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:10:55.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:55 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:10:55.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:55 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:10:55.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:55 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:10:55.874 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:55 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:10:55.874 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:55 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:10:55.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:55 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:10:55.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:55 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:10:55.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:55 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:10:55.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:55 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:10:55.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:55 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:10:55.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:55 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:10:55.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:55 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:10:55.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:55 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:10:55.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:55 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:10:55.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:55 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:10:55.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:55 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:10:56.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.001+0000 7fefe8000640 1 -- 172.21.15.16:0/3263160948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefe0109260 msgr2=0x7fefe0109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:56.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.001+0000 7fefe8000640 1 --2- 172.21.15.16:0/3263160948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefe0109260 0x7fefe0109670 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fefcc009860 tx=0x7fefcc02a340 comp rx=0 tx=0).stop 2024-08-31T06:10:56.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.001+0000 7fefe8000640 1 -- 172.21.15.16:0/3263160948 shutdown_connections 2024-08-31T06:10:56.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.001+0000 7fefe8000640 1 --2- 172.21.15.16:0/3263160948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefe0109bb0 0x7fefe01103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:56.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.001+0000 7fefe8000640 1 --2- 172.21.15.16:0/3263160948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefe0109260 0x7fefe0109670 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:56.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.001+0000 7fefe8000640 1 --2- 172.21.15.16:0/3263160948 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefe0103920 0x7fefe0103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:56.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.001+0000 7fefe8000640 1 -- 172.21.15.16:0/3263160948 >> 172.21.15.16:0/3263160948 conn(0x7fefe00ff330 msgr2=0x7fefe0101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:56.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.001+0000 7fefe8000640 1 -- 172.21.15.16:0/3263160948 shutdown_connections 2024-08-31T06:10:56.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.001+0000 7fefe8000640 1 -- 172.21.15.16:0/3263160948 wait complete. 2024-08-31T06:10:56.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.002+0000 7fefe8000640 1 Processor -- start 2024-08-31T06:10:56.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.002+0000 7fefe8000640 1 -- start start 2024-08-31T06:10:56.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.003+0000 7fefe8000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefe0103920 0x7fefe00754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:56.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.003+0000 7fefe8000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefe0109260 0x7fefe00759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:56.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.003+0000 7fefe8000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefe0109bb0 0x7fefe0075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:56.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.003+0000 7fefe8000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fefe0113870 con 0x7fefe0103920 2024-08-31T06:10:56.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.003+0000 7fefe8000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fefe01136f0 con 0x7fefe0109bb0 2024-08-31T06:10:56.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.003+0000 7fefe8000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fefe01139f0 con 0x7fefe0109260 2024-08-31T06:10:56.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.003+0000 7fefe5600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefe0103920 0x7fefe00754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:56.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.003+0000 7fefe5600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefe0103920 0x7fefe00754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47364/0 (socket says 172.21.15.16:47364) 2024-08-31T06:10:56.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.003+0000 7fefe5600640 1 -- 172.21.15.16:0/2477573875 learned_addr learned my addr 172.21.15.16:0/2477573875 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:56.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.003+0000 7fefe4c00640 1 --2- 172.21.15.16:0/2477573875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefe0109260 0x7fefe00759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:56.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.003+0000 7fefe6000640 1 --2- 172.21.15.16:0/2477573875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefe0109bb0 0x7fefe0075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:56.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.004+0000 7fefe5600640 1 -- 172.21.15.16:0/2477573875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefe0109260 msgr2=0x7fefe00759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:56.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.004+0000 7fefe5600640 1 --2- 172.21.15.16:0/2477573875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefe0109260 0x7fefe00759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:56.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.004+0000 7fefe5600640 1 -- 172.21.15.16:0/2477573875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefe0109bb0 msgr2=0x7fefe0075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:56.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.004+0000 7fefe5600640 1 --2- 172.21.15.16:0/2477573875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefe0109bb0 0x7fefe0075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.004+0000 7fefe5600640 1 -- 172.21.15.16:0/2477573875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe01a85a0 con 0x7fefe0103920 2024-08-31T06:10:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.004+0000 7fefe6000640 1 --2- 172.21.15.16:0/2477573875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefe0109bb0 0x7fefe0075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:10:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.004+0000 7fefe5600640 1 --2- 172.21.15.16:0/2477573875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefe0103920 0x7fefe00754b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fefd400e9f0 tx=0x7fefd400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.004+0000 7fefde000640 1 -- 172.21.15.16:0/2477573875 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd400cd80 con 0x7fefe0103920 2024-08-31T06:10:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.004+0000 7fefde000640 1 -- 172.21.15.16:0/2477573875 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefd40108e0 con 0x7fefe0103920 2024-08-31T06:10:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.004+0000 7fefde000640 1 -- 172.21.15.16:0/2477573875 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd4018980 con 0x7fefe0103920 2024-08-31T06:10:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.004+0000 7fefe8000640 1 -- 172.21.15.16:0/2477573875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefe01a8890 con 0x7fefe0103920 2024-08-31T06:10:56.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.005+0000 7fefe8000640 1 -- 172.21.15.16:0/2477573875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefe01a8df0 con 0x7fefe0103920 2024-08-31T06:10:56.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.009+0000 7fefeedf4740 1 Processor -- start 2024-08-31T06:10:56.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.009+0000 7fefeedf4740 1 -- start start 2024-08-31T06:10:56.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.009+0000 7fefeedf4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a60db1070 0x559a61156950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.009+0000 7fefeedf4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a60d881b0 0x559a61156e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.009+0000 7fefeedf4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a60d90390 0x559a611573d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:56.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.009+0000 7fefeedf4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559a610cd770 con 0x559a60d881b0 2024-08-31T06:10:56.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.009+0000 7fefeedf4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559a610bcd00 con 0x559a60d90390 2024-08-31T06:10:56.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.009+0000 7fefeedf4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559a610c0cb0 con 0x559a60db1070 2024-08-31T06:10:56.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.010+0000 7fefe6000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a60db1070 0x559a61156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:56.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.010+0000 7fefe6000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a60db1070 0x559a61156950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:35708/0 (socket says 172.21.15.16:35708) 2024-08-31T06:10:56.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.010+0000 7fefe6000640 1 -- 172.21.15.16:0/2696313637 learned_addr learned my addr 172.21.15.16:0/2696313637 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:56.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.010+0000 7fefe4c00640 1 --2- 172.21.15.16:0/2696313637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a60d90390 0x559a611573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:56.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.010+0000 7fefde000640 1 -- 172.21.15.16:0/2477573875 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fefd4010430 con 0x7fefe0103920 2024-08-31T06:10:56.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.010+0000 7fefe6000640 1 -- 172.21.15.16:0/2696313637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a60d90390 msgr2=0x559a611573d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:56.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.010+0000 7fefe6000640 1 --2- 172.21.15.16:0/2696313637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a60d90390 0x559a611573d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:56.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.010+0000 7fefe6000640 1 -- 172.21.15.16:0/2696313637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a60d881b0 msgr2=0x559a61156e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:56.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.010+0000 7fefe5600640 1 --2- 172.21.15.16:0/2696313637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a60d881b0 0x559a61156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:56.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.010+0000 7fefe6000640 1 --2- 172.21.15.16:0/2696313637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a60d881b0 0x559a61156e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:56.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.010+0000 7fefe6000640 1 -- 172.21.15.16:0/2696313637 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a60e1c650 con 0x559a60db1070 2024-08-31T06:10:56.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.011+0000 7fefe6000640 1 --2- 172.21.15.16:0/2696313637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a60db1070 0x559a61156950 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fefd800e870 tx=0x7fefd80095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=feed2b9d0de40ba8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:56.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.011+0000 7fefd2600640 1 -- 172.21.15.16:0/2696313637 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd800ddd0 con 0x559a60db1070 2024-08-31T06:10:56.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.011+0000 7fefd2600640 1 -- 172.21.15.16:0/2696313637 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefd801c070 con 0x559a60db1070 2024-08-31T06:10:56.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.011+0000 7fefd2600640 1 -- 172.21.15.16:0/2696313637 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd8016580 con 0x559a60db1070 2024-08-31T06:10:56.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.012+0000 7fefde000640 1 --2- 172.21.15.16:0/2477573875 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fefc80b4d20 0x7fefc80b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:56.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.012+0000 7fefeedf4740 1 -- 172.21.15.16:0/2696313637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a60db1070 msgr2=0x559a61156950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:56.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.012+0000 7fefeedf4740 1 --2- 172.21.15.16:0/2696313637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a60db1070 0x559a61156950 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fefd800e870 tx=0x7fefd80095c0 comp rx=0 tx=0).stop 2024-08-31T06:10:56.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.012+0000 7fefde000640 1 -- 172.21.15.16:0/2477573875 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fefd40e57d0 con 0x7fefe0103920 2024-08-31T06:10:56.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.012+0000 7fefeedf4740 1 -- 172.21.15.16:0/2696313637 shutdown_connections 2024-08-31T06:10:56.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.012+0000 7fefeedf4740 1 --2- 172.21.15.16:0/2696313637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a60d90390 0x559a611573d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:56.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.012+0000 7fefeedf4740 1 --2- 172.21.15.16:0/2696313637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a60d881b0 0x559a61156e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:56.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.013+0000 7fefe5600640 1 --2- 172.21.15.16:0/2477573875 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fefc80b4d20 0x7fefc80b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:56.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.013+0000 7fefeedf4740 1 --2- 172.21.15.16:0/2696313637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a60db1070 0x559a61156950 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:56.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.013+0000 7fefeedf4740 1 -- 172.21.15.16:0/2696313637 >> 172.21.15.16:0/2696313637 conn(0x559a60da5460 msgr2=0x559a61154840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:56.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.013+0000 7fefeedf4740 1 -- 172.21.15.16:0/2696313637 shutdown_connections 2024-08-31T06:10:56.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.013+0000 7fefeedf4740 1 -- 172.21.15.16:0/2696313637 wait complete. 2024-08-31T06:10:56.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.013+0000 7fefe5600640 1 --2- 172.21.15.16:0/2477573875 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fefc80b4d20 0x7fefc80b7230 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x559a611555d0 tx=0x7fefd4000bc0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:56.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.014+0000 7fefeedf4740 1 -- start start 2024-08-31T06:10:56.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.014+0000 7fefeedf4740 1 Processor -- start 2024-08-31T06:10:56.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.015+0000 7fefeedf4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a60d881b0 0x559a611624f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:56.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.015+0000 7fefeedf4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a60d90390 0x559a61162a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:56.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.015+0000 7fefeedf4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a60db1070 0x559a6116e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:56.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.015+0000 7fefeedf4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559a60f1f700 con 0x559a60db1070 2024-08-31T06:10:56.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.015+0000 7fefeedf4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559a61043cb0 con 0x559a60d881b0 2024-08-31T06:10:56.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.015+0000 7fefeedf4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559a61165130 con 0x559a60d90390 2024-08-31T06:10:56.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.015+0000 7fefeedf4740 10 client.0 _ll_register_callbacks cb 0x559a61069430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:10:56.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.015+0000 7fefe4c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a60d90390 0x559a61162a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:56.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.015+0000 7fefe4c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a60d90390 0x559a61162a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:35716/0 (socket says 172.21.15.16:35716) 2024-08-31T06:10:56.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.015+0000 7fefe4c00640 1 -- 172.21.15.16:0/193632238 learned_addr learned my addr 172.21.15.16:0/193632238 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:56.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.015+0000 7fefe6000640 1 --2- 172.21.15.16:0/193632238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a60d881b0 0x559a611624f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:56.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.015+0000 7fefe6000640 1 --2- 172.21.15.16:0/193632238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a60db1070 0x559a6116e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:56.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.015+0000 7fefe4c00640 1 -- 172.21.15.16:0/193632238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a60d881b0 msgr2=0x559a611624f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:56.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.015+0000 7fefe4c00640 1 --2- 172.21.15.16:0/193632238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a60d881b0 0x559a611624f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:56.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.015+0000 7fefe4c00640 1 -- 172.21.15.16:0/193632238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a60db1070 msgr2=0x559a6116e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:56.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.015+0000 7fefe4c00640 1 --2- 172.21.15.16:0/193632238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a60db1070 0x559a6116e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:56.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.015+0000 7fefe4c00640 1 -- 172.21.15.16:0/193632238 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a61084870 con 0x559a60d90390 2024-08-31T06:10:56.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.015+0000 7fefe6000640 1 --2- 172.21.15.16:0/193632238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a60db1070 0x559a6116e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:10:56.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.016+0000 7fefe4c00640 1 --2- 172.21.15.16:0/193632238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a60d90390 0x559a61162a30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fefe006a830 tx=0x7fefcc005e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2e99cd302cab6d3 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:56.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.016+0000 7fefc6a00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:10:56.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.016+0000 7fefc6a00640 1 -- 172.21.15.16:0/193632238 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefcc042070 con 0x559a60d90390 2024-08-31T06:10:56.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.016+0000 7fefeedf4740 1 -- 172.21.15.16:0/193632238 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefd801c070 con 0x559a60d90390 2024-08-31T06:10:56.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.016+0000 7fefc6a00640 1 -- 172.21.15.16:0/193632238 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fefcc004240 con 0x559a60d90390 2024-08-31T06:10:56.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.016+0000 7fefc6a00640 1 -- 172.21.15.16:0/193632238 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fefcc04e050 con 0x559a60d90390 2024-08-31T06:10:56.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.017+0000 7fefeedf4740 10 client.26458 fetch_fsmap learned FSMap version 32 2024-08-31T06:10:56.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.017+0000 7fefeedf4740 1 -- 172.21.15.16:0/193632238 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559a61171170 con 0x559a60d90390 2024-08-31T06:10:56.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.017+0000 7fefc6a00640 1 -- 172.21.15.16:0/193632238 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefd801c070 con 0x559a60d90390 2024-08-31T06:10:56.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.017+0000 7fefc6a00640 1 -- 172.21.15.16:0/193632238 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7fefcc003b10 con 0x559a60d90390 2024-08-31T06:10:56.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.017+0000 7fefeedf4740 10 client.26458 fetch_fsmap finished waiting for FSMap version 32 2024-08-31T06:10:56.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.017+0000 7fefeedf4740 10 client.26458 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:10:56.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.017+0000 7fefeedf4740 20 client.26458 populate_metadata read hostname 'smithi016' 2024-08-31T06:10:56.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.017+0000 7fefeedf4740 1 --2- 172.21.15.16:0/193632238 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559a60d6f9f0 0x559a61174710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:56.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.017+0000 7fefeedf4740 4 client.26458 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:10:56.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.017+0000 7fefeedf4740 1 -- 172.21.15.16:0/193632238 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a610cf6c0 con 0x559a60d6f9f0 2024-08-31T06:10:56.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.018+0000 7fefe6000640 1 --2- 172.21.15.16:0/193632238 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559a60d6f9f0 0x559a61174710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:56.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.018+0000 7fefe6000640 1 --2- 172.21.15.16:0/193632238 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559a60d6f9f0 0x559a61174710 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2803139d23eeb56 server_cookie=bcef5cae99a12694 in_seq=0 out_seq=0 2024-08-31T06:10:56.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.018+0000 7fefc6a00640 10 client.26458 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:56.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.019+0000 7fefc6a00640 1 -- 172.21.15.16:0/193632238 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559a610cf6c0 con 0x559a60d6f9f0 2024-08-31T06:10:56.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.019+0000 7fefc6a00640 10 client.26458 handle_command_reply: tid=0 2024-08-31T06:10:56.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.019+0000 7fefeedf4740 1 client.26458 shutdown 2024-08-31T06:10:56.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.019+0000 7fefeedf4740 1 -- 172.21.15.16:0/193632238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a60d90390 msgr2=0x559a61162a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:56.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.019+0000 7fefeedf4740 1 --2- 172.21.15.16:0/193632238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a60d90390 0x559a61162a30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fefe006a830 tx=0x7fefcc005e70 comp rx=0 tx=0).stop 2024-08-31T06:10:56.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.019+0000 7fefeedf4740 1 -- 172.21.15.16:0/193632238 shutdown_connections 2024-08-31T06:10:56.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.019+0000 7fefeedf4740 1 --2- 172.21.15.16:0/193632238 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559a60d6f9f0 0x559a61174710 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:56.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.019+0000 7fefeedf4740 1 --2- 172.21.15.16:0/193632238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a60d90390 0x559a61162a30 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:56.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.019+0000 7fefeedf4740 1 --2- 172.21.15.16:0/193632238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a60db1070 0x559a6116e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:56.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.019+0000 7fefc6a00640 0 client.26458 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:56.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.019+0000 7fefeedf4740 1 --2- 172.21.15.16:0/193632238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a60d881b0 0x559a611624f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:56.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.019+0000 7fefeedf4740 1 -- 172.21.15.16:0/193632238 >> 172.21.15.16:0/193632238 conn(0x559a60da5460 msgr2=0x559a61159ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:56.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.020+0000 7fefeedf4740 1 -- 172.21.15.16:0/193632238 shutdown_connections 2024-08-31T06:10:56.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.020+0000 7fefeedf4740 1 -- 172.21.15.16:0/193632238 wait complete. 2024-08-31T06:10:56.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.020+0000 7fefeedf4740 20 client.26458 trim_cache size 0 max 16384 2024-08-31T06:10:56.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.057+0000 7fefeedf4740 1 Processor -- start 2024-08-31T06:10:56.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.057+0000 7fefeedf4740 1 -- start start 2024-08-31T06:10:56.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.057+0000 7fefeedf4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a60d881b0 0x559a61153df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:56.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.057+0000 7fefeedf4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a60d90390 0x559a61154330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:56.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.057+0000 7fefeedf4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a60db1070 0x559a6115f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:56.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.057+0000 7fefeedf4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fefe01b3fb0 con 0x559a60d90390 2024-08-31T06:10:56.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.057+0000 7fefeedf4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fefe01b17f0 con 0x559a60db1070 2024-08-31T06:10:56.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.058+0000 7fefeedf4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fefe01af640 con 0x559a60d881b0 2024-08-31T06:10:56.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.058+0000 7fefe4c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a60d90390 0x559a61154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:56.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.058+0000 7fefe6000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a60d881b0 0x559a61153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:56.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.058+0000 7fefe4c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a60d90390 0x559a61154330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47384/0 (socket says 172.21.15.16:47384) 2024-08-31T06:10:56.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.058+0000 7fefe4c00640 1 -- 172.21.15.16:0/3776277351 learned_addr learned my addr 172.21.15.16:0/3776277351 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:56.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.058+0000 7fefe6000640 1 --2- 172.21.15.16:0/3776277351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a60db1070 0x559a6115f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:56.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.058+0000 7fefe4c00640 1 -- 172.21.15.16:0/3776277351 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a60d881b0 msgr2=0x559a61153df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:56.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.058+0000 7fefe4c00640 1 --2- 172.21.15.16:0/3776277351 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a60d881b0 0x559a61153df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:56.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.058+0000 7fefe4c00640 1 -- 172.21.15.16:0/3776277351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a60db1070 msgr2=0x559a6115f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:56.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.058+0000 7fefe4c00640 1 --2- 172.21.15.16:0/3776277351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a60db1070 0x559a6115f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:56.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.058+0000 7fefe4c00640 1 -- 172.21.15.16:0/3776277351 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a61158c40 con 0x559a60d90390 2024-08-31T06:10:56.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.059+0000 7fefe4c00640 1 --2- 172.21.15.16:0/3776277351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a60d90390 0x559a61154330 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fefcc0057c0 tx=0x7fefcc030fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e3aff2dada41706 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:56.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.059+0000 7fefd1c00640 1 -- 172.21.15.16:0/3776277351 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefcc042040 con 0x559a60d90390 2024-08-31T06:10:56.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.059+0000 7fefd1c00640 1 -- 172.21.15.16:0/3776277351 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefcc030900 con 0x559a60d90390 2024-08-31T06:10:56.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.059+0000 7fefd1c00640 1 -- 172.21.15.16:0/3776277351 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefcc037040 con 0x559a60d90390 2024-08-31T06:10:56.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.060+0000 7fefeedf4740 1 -- 172.21.15.16:0/3776277351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a60d90390 msgr2=0x559a61154330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:56.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.060+0000 7fefeedf4740 1 --2- 172.21.15.16:0/3776277351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a60d90390 0x559a61154330 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fefcc0057c0 tx=0x7fefcc030fc0 comp rx=0 tx=0).stop 2024-08-31T06:10:56.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.060+0000 7fefeedf4740 1 -- 172.21.15.16:0/3776277351 shutdown_connections 2024-08-31T06:10:56.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.060+0000 7fefeedf4740 1 --2- 172.21.15.16:0/3776277351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a60db1070 0x559a6115f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:56.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.060+0000 7fefeedf4740 1 --2- 172.21.15.16:0/3776277351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a60d90390 0x559a61154330 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:56.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.060+0000 7fefeedf4740 1 --2- 172.21.15.16:0/3776277351 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a60d881b0 0x559a61153df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:56.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.060+0000 7fefeedf4740 1 -- 172.21.15.16:0/3776277351 >> 172.21.15.16:0/3776277351 conn(0x559a60da5460 msgr2=0x559a61160b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:56.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.060+0000 7fefeedf4740 1 -- 172.21.15.16:0/3776277351 shutdown_connections 2024-08-31T06:10:56.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.060+0000 7fefeedf4740 1 -- 172.21.15.16:0/3776277351 wait complete. 2024-08-31T06:10:56.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.061+0000 7fefeedf4740 1 -- start start 2024-08-31T06:10:56.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.061+0000 7fefeedf4740 1 Processor -- start 2024-08-31T06:10:56.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.061+0000 7fefeedf4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a60d881b0 0x559a611665b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:56.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.062+0000 7fefeedf4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a60d90390 0x559a61166af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:56.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.062+0000 7fefeedf4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a60db1070 0x559a61167030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:56.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.062+0000 7fefeedf4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fefe01afcb0 con 0x559a60d881b0 2024-08-31T06:10:56.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.062+0000 7fefeedf4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fefe01b1120 con 0x559a60db1070 2024-08-31T06:10:56.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.062+0000 7fefeedf4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fefe01b0380 con 0x559a60d90390 2024-08-31T06:10:56.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.062+0000 7fefeedf4740 10 client.0 _ll_register_callbacks cb 0x7fefcc005d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:10:56.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.062+0000 7fefe6000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a60d881b0 0x559a611665b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:56.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.062+0000 7fefe4c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a60d90390 0x559a61166af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:56.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.062+0000 7fefe6000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a60d881b0 0x559a611665b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47398/0 (socket says 172.21.15.16:47398) 2024-08-31T06:10:56.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.062+0000 7fefe6000640 1 -- 172.21.15.16:0/1291083558 learned_addr learned my addr 172.21.15.16:0/1291083558 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:56.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.062+0000 7fefe4c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a60d90390 0x559a61166af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:35722/0 (socket says 172.21.15.16:35722) 2024-08-31T06:10:56.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.062+0000 7fefe6000640 1 --2- 172.21.15.16:0/1291083558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a60db1070 0x559a61167030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:56.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.062+0000 7fefe4c00640 1 -- 172.21.15.16:0/1291083558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a60db1070 msgr2=0x559a61167030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:56.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.062+0000 7fefe4c00640 1 --2- 172.21.15.16:0/1291083558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a60db1070 0x559a61167030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:56.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.062+0000 7fefe4c00640 1 -- 172.21.15.16:0/1291083558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a60d881b0 msgr2=0x559a611665b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:10:56.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.062+0000 7fefe4c00640 1 --2- 172.21.15.16:0/1291083558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a60d881b0 0x559a611665b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:56.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.062+0000 7fefe4c00640 1 -- 172.21.15.16:0/1291083558 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a611de2d0 con 0x559a60d90390 2024-08-31T06:10:56.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.062+0000 7fefe6000640 1 --2- 172.21.15.16:0/1291083558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a60d881b0 0x559a611665b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:10:56.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.063+0000 7fefe4c00640 1 --2- 172.21.15.16:0/1291083558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a60d90390 0x559a61166af0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fefcc02a840 tx=0x7fefcc005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d553c3fb8992d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:56.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.063+0000 7fefc6a00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:10:56.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.063+0000 7fefc6a00640 1 -- 172.21.15.16:0/1291083558 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefcc037040 con 0x559a60d90390 2024-08-31T06:10:56.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.063+0000 7fefc6a00640 1 -- 172.21.15.16:0/1291083558 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fefcc030450 con 0x559a60d90390 2024-08-31T06:10:56.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.063+0000 7fefeedf4740 1 -- 172.21.15.16:0/1291083558 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefcc030900 con 0x559a60d90390 2024-08-31T06:10:56.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.064+0000 7fefc6a00640 1 -- 172.21.15.16:0/1291083558 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fefcc042040 con 0x559a60d90390 2024-08-31T06:10:56.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.064+0000 7fefeedf4740 10 client.26464 fetch_fsmap learned FSMap version 32 2024-08-31T06:10:56.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.064+0000 7fefeedf4740 1 -- 172.21.15.16:0/1291083558 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559a611df560 con 0x559a60d90390 2024-08-31T06:10:56.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.064+0000 7fefc6a00640 1 -- 172.21.15.16:0/1291083558 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefcc030900 con 0x559a60d90390 2024-08-31T06:10:56.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.064+0000 7fefc6a00640 1 -- 172.21.15.16:0/1291083558 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7fefcc03eb40 con 0x559a60d90390 2024-08-31T06:10:56.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.064+0000 7fefeedf4740 10 client.26464 fetch_fsmap finished waiting for FSMap version 32 2024-08-31T06:10:56.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.064+0000 7fefeedf4740 10 client.26464 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:10:56.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.064+0000 7fefeedf4740 20 client.26464 populate_metadata read hostname 'smithi016' 2024-08-31T06:10:56.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.064+0000 7fefeedf4740 1 --2- 172.21.15.16:0/1291083558 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559a60d44da0 0x559a611e1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:56.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.064+0000 7fefeedf4740 4 client.26464 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T06:10:56.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.064+0000 7fefeedf4740 1 -- 172.21.15.16:0/1291083558 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559a610cb160 con 0x559a60d44da0 2024-08-31T06:10:56.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.065+0000 7fefe6000640 1 --2- 172.21.15.16:0/1291083558 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559a60d44da0 0x559a611e1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:56.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.065+0000 7fefe6000640 1 --2- 172.21.15.16:0/1291083558 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559a60d44da0 0x559a611e1bd0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd63532892485e4 server_cookie=abe57a959e12ad3a in_seq=0 out_seq=0 2024-08-31T06:10:56.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.065+0000 7fefc6a00640 10 client.26464 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:56.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.066+0000 7fefc6a00640 1 -- 172.21.15.16:0/1291083558 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1990 (crc 0 0 0) 0x559a610cb160 con 0x559a60d44da0 2024-08-31T06:10:56.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.066+0000 7fefc6a00640 10 client.26464 handle_command_reply: tid=0 2024-08-31T06:10:56.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.066+0000 7fefeedf4740 1 client.26464 shutdown 2024-08-31T06:10:56.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.066+0000 7fefeedf4740 1 -- 172.21.15.16:0/1291083558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a60d90390 msgr2=0x559a61166af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:56.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.066+0000 7fefeedf4740 1 --2- 172.21.15.16:0/1291083558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a60d90390 0x559a61166af0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fefcc02a840 tx=0x7fefcc005f10 comp rx=0 tx=0).stop 2024-08-31T06:10:56.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.066+0000 7fefeedf4740 1 -- 172.21.15.16:0/1291083558 shutdown_connections 2024-08-31T06:10:56.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.066+0000 7fefeedf4740 1 --2- 172.21.15.16:0/1291083558 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559a60d44da0 0x559a611e1bd0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:56.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.066+0000 7fefeedf4740 1 --2- 172.21.15.16:0/1291083558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a60d90390 0x559a61166af0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:56.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.066+0000 7fefc6a00640 0 client.26464 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:10:56.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.066+0000 7fefeedf4740 1 --2- 172.21.15.16:0/1291083558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a60db1070 0x559a61167030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:56.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.066+0000 7fefeedf4740 1 --2- 172.21.15.16:0/1291083558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a60d881b0 0x559a611665b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:56.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.066+0000 7fefeedf4740 1 -- 172.21.15.16:0/1291083558 >> 172.21.15.16:0/1291083558 conn(0x559a60da5460 msgr2=0x559a61159a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:56.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.066+0000 7fefeedf4740 1 -- 172.21.15.16:0/1291083558 shutdown_connections 2024-08-31T06:10:56.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.066+0000 7fefeedf4740 1 -- 172.21.15.16:0/1291083558 wait complete. 2024-08-31T06:10:56.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.066+0000 7fefeedf4740 20 client.26464 trim_cache size 0 max 16384 2024-08-31T06:10:56.071 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:10:56.072 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T06:10:56.072 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-08-31T06:10:56.072 INFO:teuthology.orchestra.run.smithi016.stdout: "e739a16e-8ace-43a8-a1ce-ac27d39ef8a3": { 2024-08-31T06:10:56.072 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:10:56.072 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "e739a16e-8ace-43a8-a1ce-ac27d39ef8a3", 2024-08-31T06:10:56.072 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-08-31T06:10:56.072 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:10:56.072 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:10:56.072 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-08-31T06:10:56.072 INFO:teuthology.orchestra.run.smithi016.stdout: "e739a16e-8ace-43a8-a1ce-ac27d39ef8a3": { 2024-08-31T06:10:56.072 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:54.084646+0000", 2024-08-31T06:10:56.072 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:10:56.073 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:10:56.073 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:10:56.073 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:10:56.073 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:10:56.073 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:10:56.073 INFO:teuthology.orchestra.run.smithi016.stdout: "c1fd6904-2e02-47fa-939a-47349b9fe36b": { 2024-08-31T06:10:56.073 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:06.678691+0000", 2024-08-31T06:10:56.073 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:10:56.073 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:10:56.073 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:10:56.073 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:10:56.073 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:10:56.073 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:10:56.073 INFO:teuthology.orchestra.run.smithi016.stdout: "78d6ef02-8551-45e8-ad60-9627eb573fc1": { 2024-08-31T06:10:56.074 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:09:13.547865+0000", 2024-08-31T06:10:56.074 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:10:56.074 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:10:56.074 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:10:56.074 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:10:56.074 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:10:56.074 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:10:56.074 INFO:teuthology.orchestra.run.smithi016.stdout: "c8455d74-2fe7-4d93-bcf2-19b419d16016": { 2024-08-31T06:10:56.074 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:08:24.035960+0000", 2024-08-31T06:10:56.074 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:10:56.074 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:10:56.074 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:10:56.074 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:10:56.074 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:10:56.075 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:10:56.075 INFO:teuthology.orchestra.run.smithi016.stdout: "88d7bdb8-cb1c-45e5-ad29-4cd8da6c6003": { 2024-08-31T06:10:56.075 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:07:39.648942+0000", 2024-08-31T06:10:56.075 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:10:56.075 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:10:56.075 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:10:56.075 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:10:56.075 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:10:56.075 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:10:56.075 INFO:teuthology.orchestra.run.smithi016.stdout: "c300b7e8-f3fd-482c-a909-9329bc61e570": { 2024-08-31T06:10:56.075 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:06:49.868046+0000", 2024-08-31T06:10:56.075 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:10:56.075 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:10:56.076 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:10:56.076 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:10:56.076 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:10:56.076 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:10:56.076 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:10:56.076 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:10:56.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.067+0000 7fefe8000640 1 -- 172.21.15.16:0/2477573875 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fefc80b4d20 msgr2=0x7fefc80b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:56.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.067+0000 7fefe8000640 1 --2- 172.21.15.16:0/2477573875 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fefc80b4d20 0x7fefc80b7230 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x559a611555d0 tx=0x7fefd4000bc0 comp rx=0 tx=0).stop 2024-08-31T06:10:56.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.067+0000 7fefe8000640 1 -- 172.21.15.16:0/2477573875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefe0103920 msgr2=0x7fefe00754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:56.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.067+0000 7fefe8000640 1 --2- 172.21.15.16:0/2477573875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefe0103920 0x7fefe00754b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fefd400e9f0 tx=0x7fefd400eeb0 comp rx=0 tx=0).stop 2024-08-31T06:10:56.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.067+0000 7fefe8000640 1 -- 172.21.15.16:0/2477573875 shutdown_connections 2024-08-31T06:10:56.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.067+0000 7fefe8000640 1 --2- 172.21.15.16:0/2477573875 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fefc80b4d20 0x7fefc80b7230 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:56.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.067+0000 7fefe8000640 1 --2- 172.21.15.16:0/2477573875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefe0109260 0x7fefe00759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:56.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.067+0000 7fefe8000640 1 --2- 172.21.15.16:0/2477573875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefe0109bb0 0x7fefe0075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:56.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.067+0000 7fefe8000640 1 --2- 172.21.15.16:0/2477573875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefe0103920 0x7fefe00754b0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:56.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.067+0000 7fefe8000640 1 -- 172.21.15.16:0/2477573875 >> 172.21.15.16:0/2477573875 conn(0x7fefe00ff330 msgr2=0x7fefe01059b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:56.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.067+0000 7fefe8000640 1 -- 172.21.15.16:0/2477573875 shutdown_connections 2024-08-31T06:10:56.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:56.068+0000 7fefe8000640 1 -- 172.21.15.16:0/2477573875 wait complete. 2024-08-31T06:10:56.628 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 210.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-31T06:10:56.773 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:56 smithi016 ceph-mon[17194]: pgmap v475: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 326 KiB/s wr, 68 op/s 2024-08-31T06:10:56.774 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:56 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:10:56.774 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:56 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:10:56.774 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:56 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:10:56.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:56 smithi155 ceph-mon[21742]: pgmap v475: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 326 KiB/s wr, 68 op/s 2024-08-31T06:10:56.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:56 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:10:56.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:56 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:10:56.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:56 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:10:56.951 INFO:tasks.cephfs.filesystem:scrub status for tag:e739a16e-8ace-43a8-a1ce-ac27d39ef8a3 - {'path': '/', 'tag': 'e739a16e-8ace-43a8-a1ce-ac27d39ef8a3', 'options': 'recursive,force'} 2024-08-31T06:10:56.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:56 smithi184 ceph-mon[22331]: pgmap v475: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 326 KiB/s wr, 68 op/s 2024-08-31T06:10:56.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:56 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:10:56.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:56 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:10:56.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:56 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:10:57.978 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:10:58.361 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:10:58.833 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:58 smithi016 ceph-mon[17194]: pgmap v476: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 60 KiB/s wr, 52 op/s 2024-08-31T06:10:58.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:58 smithi155 ceph-mon[21742]: pgmap v476: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 60 KiB/s wr, 52 op/s 2024-08-31T06:10:58.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:58 smithi184 ceph-mon[22331]: pgmap v476: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 60 KiB/s wr, 52 op/s 2024-08-31T06:10:59.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.065+0000 7fb9c6e00640 1 -- 172.21.15.16:0/1729193206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9c010ad30 msgr2=0x7fb9c0115d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:59.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.065+0000 7fb9c6e00640 1 --2- 172.21.15.16:0/1729193206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9c010ad30 0x7fb9c0115d70 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4009be0 tx=0x7fb9b402a6f0 comp rx=0 tx=0).stop 2024-08-31T06:10:59.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.067+0000 7fb9c6e00640 1 -- 172.21.15.16:0/1729193206 shutdown_connections 2024-08-31T06:10:59.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.067+0000 7fb9c6e00640 1 --2- 172.21.15.16:0/1729193206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9c01162b0 0x7fb9c0118730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:59.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.067+0000 7fb9c6e00640 1 --2- 172.21.15.16:0/1729193206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9c010ad30 0x7fb9c0115d70 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:59.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.067+0000 7fb9c6e00640 1 --2- 172.21.15.16:0/1729193206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9c010a3e0 0x7fb9c010a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:59.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.068+0000 7fb9c6e00640 1 -- 172.21.15.16:0/1729193206 >> 172.21.15.16:0/1729193206 conn(0x7fb9c006e790 msgr2=0x7fb9c006ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:59.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.070+0000 7fb9c6e00640 1 -- 172.21.15.16:0/1729193206 shutdown_connections 2024-08-31T06:10:59.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.070+0000 7fb9c6e00640 1 -- 172.21.15.16:0/1729193206 wait complete. 2024-08-31T06:10:59.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.070+0000 7fb9c6e00640 1 Processor -- start 2024-08-31T06:10:59.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.070+0000 7fb9c6e00640 1 -- start start 2024-08-31T06:10:59.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.075+0000 7fb9c6e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9c010a3e0 0x7fb9c01a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:59.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.075+0000 7fb9c6e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9c010ad30 0x7fb9c01a2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:59.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.075+0000 7fb9c5a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9c010a3e0 0x7fb9c01a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:59.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.075+0000 7fb9c5a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9c010a3e0 0x7fb9c01a29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47428/0 (socket says 172.21.15.16:47428) 2024-08-31T06:10:59.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.075+0000 7fb9c6e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9c01162b0 0x7fb9c01a8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:59.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.075+0000 7fb9c5a00640 1 -- 172.21.15.16:0/2197864766 learned_addr learned my addr 172.21.15.16:0/2197864766 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:10:59.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.075+0000 7fb9c6e00640 1 -- 172.21.15.16:0/2197864766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c011be90 con 0x7fb9c010a3e0 2024-08-31T06:10:59.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.075+0000 7fb9c6e00640 1 -- 172.21.15.16:0/2197864766 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c011bd10 con 0x7fb9c010ad30 2024-08-31T06:10:59.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.075+0000 7fb9c6e00640 1 -- 172.21.15.16:0/2197864766 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c011c010 con 0x7fb9c01162b0 2024-08-31T06:10:59.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.075+0000 7fb9c5000640 1 --2- 172.21.15.16:0/2197864766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9c010ad30 0x7fb9c01a2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:59.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.075+0000 7fb9c6400640 1 --2- 172.21.15.16:0/2197864766 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9c01162b0 0x7fb9c01a8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:59.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.075+0000 7fb9c5000640 1 -- 172.21.15.16:0/2197864766 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9c01162b0 msgr2=0x7fb9c01a8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:59.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.075+0000 7fb9c5000640 1 --2- 172.21.15.16:0/2197864766 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9c01162b0 0x7fb9c01a8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:59.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.075+0000 7fb9c5000640 1 -- 172.21.15.16:0/2197864766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9c010a3e0 msgr2=0x7fb9c01a29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:59.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.075+0000 7fb9c5000640 1 --2- 172.21.15.16:0/2197864766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9c010a3e0 0x7fb9c01a29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:59.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.075+0000 7fb9c5000640 1 -- 172.21.15.16:0/2197864766 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9c01a8950 con 0x7fb9c010ad30 2024-08-31T06:10:59.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.075+0000 7fb9c5a00640 1 --2- 172.21.15.16:0/2197864766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9c010a3e0 0x7fb9c01a29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:10:59.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.075+0000 7fb9c5000640 1 --2- 172.21.15.16:0/2197864766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9c010ad30 0x7fb9c01a2f80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4034040 tx=0x7fb9b40305b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:59.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.080+0000 7fb9be000640 1 -- 172.21.15.16:0/2197864766 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4041020 con 0x7fb9c010ad30 2024-08-31T06:10:59.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.080+0000 7fb9c6e00640 1 -- 172.21.15.16:0/2197864766 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9c01a8be0 con 0x7fb9c010ad30 2024-08-31T06:10:59.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.080+0000 7fb9c6e00640 1 -- 172.21.15.16:0/2197864766 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9c01a9020 con 0x7fb9c010ad30 2024-08-31T06:10:59.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.080+0000 7fb9be000640 1 -- 172.21.15.16:0/2197864766 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9b4030e60 con 0x7fb9c010ad30 2024-08-31T06:10:59.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.080+0000 7fb9be000640 1 -- 172.21.15.16:0/2197864766 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4036040 con 0x7fb9c010ad30 2024-08-31T06:10:59.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.082+0000 7fb9c6e00640 1 -- 172.21.15.16:0/2197864766 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9a4005290 con 0x7fb9c010ad30 2024-08-31T06:10:59.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.086+0000 7fb9be000640 1 -- 172.21.15.16:0/2197864766 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fb9b4044020 con 0x7fb9c010ad30 2024-08-31T06:10:59.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.090+0000 7fb9be000640 1 --2- 172.21.15.16:0/2197864766 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb9a80b4d50 0x7fb9a80b7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:10:59.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.091+0000 7fb9be000640 1 -- 172.21.15.16:0/2197864766 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fb9b4105cc0 con 0x7fb9c010ad30 2024-08-31T06:10:59.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.091+0000 7fb9c5a00640 1 --2- 172.21.15.16:0/2197864766 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb9a80b4d50 0x7fb9a80b7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:10:59.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.094+0000 7fb9be000640 1 -- 172.21.15.16:0/2197864766 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fb9b404b030 con 0x7fb9c010ad30 2024-08-31T06:10:59.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.096+0000 7fb9c5a00640 1 --2- 172.21.15.16:0/2197864766 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb9a80b4d50 0x7fb9a80b7260 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb9b80058c0 tx=0x7fb9b8009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:10:59.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.351+0000 7fb9c6e00640 1 -- 172.21.15.16:0/2197864766 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb9a4005850 con 0x7fb9c010ad30 2024-08-31T06:10:59.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.352+0000 7fb9be000640 1 -- 172.21.15.16:0/2197864766 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11921 (secure 0 0 0) 0x7fb9b40ccb10 con 0x7fb9c010ad30 2024-08-31T06:10:59.355 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:10:59.355 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":32,"btime":"2024-08-31T06:10:42:535148+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:10:42.006856+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24712,24724,24737]},"id":1}]} 2024-08-31T06:10:59.355 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 32 2024-08-31T06:10:59.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.355+0000 7fb9c6e00640 1 -- 172.21.15.16:0/2197864766 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb9a80b4d50 msgr2=0x7fb9a80b7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:59.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.355+0000 7fb9c6e00640 1 --2- 172.21.15.16:0/2197864766 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb9a80b4d50 0x7fb9a80b7260 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb9b80058c0 tx=0x7fb9b8009260 comp rx=0 tx=0).stop 2024-08-31T06:10:59.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.355+0000 7fb9c6e00640 1 -- 172.21.15.16:0/2197864766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9c010ad30 msgr2=0x7fb9c01a2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:10:59.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.355+0000 7fb9c6e00640 1 --2- 172.21.15.16:0/2197864766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9c010ad30 0x7fb9c01a2f80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4034040 tx=0x7fb9b40305b0 comp rx=0 tx=0).stop 2024-08-31T06:10:59.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.355+0000 7fb9c6e00640 1 -- 172.21.15.16:0/2197864766 shutdown_connections 2024-08-31T06:10:59.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.355+0000 7fb9c6e00640 1 --2- 172.21.15.16:0/2197864766 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb9a80b4d50 0x7fb9a80b7260 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:59.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.355+0000 7fb9c6e00640 1 --2- 172.21.15.16:0/2197864766 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9c01162b0 0x7fb9c01a8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:59.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.355+0000 7fb9c6e00640 1 --2- 172.21.15.16:0/2197864766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9c010ad30 0x7fb9c01a2f80 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:59.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.355+0000 7fb9c6e00640 1 --2- 172.21.15.16:0/2197864766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9c010a3e0 0x7fb9c01a29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:10:59.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.355+0000 7fb9c6e00640 1 -- 172.21.15.16:0/2197864766 >> 172.21.15.16:0/2197864766 conn(0x7fb9c006e790 msgr2=0x7fb9c010ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:10:59.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.355+0000 7fb9c6e00640 1 -- 172.21.15.16:0/2197864766 shutdown_connections 2024-08-31T06:10:59.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:10:59.355+0000 7fb9c6e00640 1 -- 172.21.15.16:0/2197864766 wait complete. 2024-08-31T06:10:59.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:10:59 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/2197864766' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:10:59.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:10:59 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/2197864766' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:10:59.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:10:59 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/2197864766' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:11:00.452 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24737 quiesce db --set-id 9b804def --release --await' 2024-08-31T06:11:00.453 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24737 quiesce db --set-id 9b804def --release --await 2024-08-31T06:11:00.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:00 smithi016 ceph-mon[17194]: pgmap v477: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 430 KiB/s wr, 127 op/s 2024-08-31T06:11:00.850 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:11:00.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:00 smithi155 ceph-mon[21742]: pgmap v477: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 430 KiB/s wr, 127 op/s 2024-08-31T06:11:00.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:00 smithi184 ceph-mon[22331]: pgmap v477: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 430 KiB/s wr, 127 op/s 2024-08-31T06:11:01.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.521+0000 7f5131600640 1 -- 172.21.15.16:0/1279977941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f512c102eb0 msgr2=0x7f512c1073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:01.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.521+0000 7f5131600640 1 --2- 172.21.15.16:0/1279977941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f512c102eb0 0x7f512c1073e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5118009930 tx=0x7f511802a420 comp rx=0 tx=0).stop 2024-08-31T06:11:01.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.521+0000 7f5131600640 1 -- 172.21.15.16:0/1279977941 shutdown_connections 2024-08-31T06:11:01.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.521+0000 7f5131600640 1 --2- 172.21.15.16:0/1279977941 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f512c1079a0 0x7f512c107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:01.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.521+0000 7f5131600640 1 --2- 172.21.15.16:0/1279977941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f512c102eb0 0x7f512c1073e0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:01.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.521+0000 7f5131600640 1 --2- 172.21.15.16:0/1279977941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f512c102560 0x7f512c102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:01.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.521+0000 7f5131600640 1 -- 172.21.15.16:0/1279977941 >> 172.21.15.16:0/1279977941 conn(0x7f512c0794e0 msgr2=0x7f512c0798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:01.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.521+0000 7f5131600640 1 -- 172.21.15.16:0/1279977941 shutdown_connections 2024-08-31T06:11:01.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.521+0000 7f5131600640 1 -- 172.21.15.16:0/1279977941 wait complete. 2024-08-31T06:11:01.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.522+0000 7f5131600640 1 Processor -- start 2024-08-31T06:11:01.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.523+0000 7f5131600640 1 -- start start 2024-08-31T06:11:01.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.523+0000 7f5131600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f512c102560 0x7f512c1a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:01.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.523+0000 7f5131600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f512c102eb0 0x7f512c1a0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:01.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.523+0000 7f5131600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f512c1079a0 0x7f512c1a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:01.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.523+0000 7f5131600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f512c119e40 con 0x7f512c1079a0 2024-08-31T06:11:01.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.523+0000 7f5131600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f512c119cc0 con 0x7f512c102eb0 2024-08-31T06:11:01.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.523+0000 7f5131600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f512c119fc0 con 0x7f512c102560 2024-08-31T06:11:01.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.524+0000 7f512b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f512c1079a0 0x7f512c1a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:01.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.524+0000 7f512aa00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f512c102560 0x7f512c1a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:01.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.524+0000 7f512b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f512c1079a0 0x7f512c1a6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47442/0 (socket says 172.21.15.16:47442) 2024-08-31T06:11:01.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.524+0000 7f512b400640 1 -- 172.21.15.16:0/4200020622 learned_addr learned my addr 172.21.15.16:0/4200020622 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:01.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.524+0000 7f512a000640 1 --2- 172.21.15.16:0/4200020622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f512c102eb0 0x7f512c1a0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:01.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.524+0000 7f512aa00640 1 -- 172.21.15.16:0/4200020622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f512c102eb0 msgr2=0x7f512c1a0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:01.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.524+0000 7f512aa00640 1 --2- 172.21.15.16:0/4200020622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f512c102eb0 0x7f512c1a0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:01.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.524+0000 7f512aa00640 1 -- 172.21.15.16:0/4200020622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f512c1079a0 msgr2=0x7f512c1a6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:01.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.524+0000 7f512aa00640 1 --2- 172.21.15.16:0/4200020622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f512c1079a0 0x7f512c1a6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:01.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.524+0000 7f512aa00640 1 -- 172.21.15.16:0/4200020622 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f512c1a68c0 con 0x7f512c102560 2024-08-31T06:11:01.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.524+0000 7f512b400640 1 --2- 172.21.15.16:0/4200020622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f512c1079a0 0x7f512c1a6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:11:01.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.524+0000 7f512a000640 1 --2- 172.21.15.16:0/4200020622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f512c102eb0 0x7f512c1a0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:11:01.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.524+0000 7f512aa00640 1 --2- 172.21.15.16:0/4200020622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f512c102560 0x7f512c1a0950 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f512c06c6e0 tx=0x7f511c00dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:01.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.525+0000 7f5123400640 1 -- 172.21.15.16:0/4200020622 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c017070 con 0x7f512c102560 2024-08-31T06:11:01.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.525+0000 7f5131600640 1 -- 172.21.15.16:0/4200020622 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f512c1a6bb0 con 0x7f512c102560 2024-08-31T06:11:01.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.525+0000 7f5123400640 1 -- 172.21.15.16:0/4200020622 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f511c009c30 con 0x7f512c102560 2024-08-31T06:11:01.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.525+0000 7f5123400640 1 -- 172.21.15.16:0/4200020622 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c00f620 con 0x7f512c102560 2024-08-31T06:11:01.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.525+0000 7f5131600640 1 -- 172.21.15.16:0/4200020622 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f512c1ad310 con 0x7f512c102560 2024-08-31T06:11:01.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.528+0000 7f51381ca740 1 Processor -- start 2024-08-31T06:11:01.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.528+0000 7f51381ca740 1 -- start start 2024-08-31T06:11:01.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.528+0000 7f51381ca740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f573ee8520 0x55f57428def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:01.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.528+0000 7f51381ca740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f573ebf660 0x55f57428e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:01.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.529+0000 7f51381ca740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f573ec7840 0x55f57428e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:01.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.529+0000 7f51381ca740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f574204a90 con 0x55f573ebf660 2024-08-31T06:11:01.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.529+0000 7f51381ca740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f5741f4020 con 0x55f573ee8520 2024-08-31T06:11:01.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.529+0000 7f51381ca740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f5741f7fd0 con 0x55f573ec7840 2024-08-31T06:11:01.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.529+0000 7f5123400640 1 -- 172.21.15.16:0/4200020622 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f511c01a070 con 0x7f512c102560 2024-08-31T06:11:01.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.529+0000 7f512aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f573ebf660 0x55f57428e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:01.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.529+0000 7f512b400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f573ee8520 0x55f57428def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:01.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.529+0000 7f512b400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f573ee8520 0x55f57428def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:45912/0 (socket says 172.21.15.16:45912) 2024-08-31T06:11:01.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.529+0000 7f512b400640 1 -- 172.21.15.16:0/3275313307 learned_addr learned my addr 172.21.15.16:0/3275313307 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:01.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.529+0000 7f512a000640 1 --2- 172.21.15.16:0/3275313307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f573ec7840 0x55f57428e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:01.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.529+0000 7f512b400640 1 -- 172.21.15.16:0/3275313307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f573ec7840 msgr2=0x55f57428e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:01.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.529+0000 7f512b400640 1 --2- 172.21.15.16:0/3275313307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f573ec7840 0x55f57428e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:01.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.529+0000 7f512b400640 1 -- 172.21.15.16:0/3275313307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f573ebf660 msgr2=0x55f57428e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:01.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.529+0000 7f512b400640 1 --2- 172.21.15.16:0/3275313307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f573ebf660 0x55f57428e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:01.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.529+0000 7f512b400640 1 -- 172.21.15.16:0/3275313307 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f573f53b00 con 0x55f573ee8520 2024-08-31T06:11:01.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.530+0000 7f512b400640 1 --2- 172.21.15.16:0/3275313307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f573ee8520 0x55f57428def0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5124011850 tx=0x7f5124011d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=709b9f0baf609793 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:01.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.530+0000 7f5117400640 1 -- 172.21.15.16:0/3275313307 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5124011330 con 0x55f573ee8520 2024-08-31T06:11:01.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.530+0000 7f5117400640 1 -- 172.21.15.16:0/3275313307 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f512400fe50 con 0x55f573ee8520 2024-08-31T06:11:01.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.530+0000 7f5117400640 1 -- 172.21.15.16:0/3275313307 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51240193e0 con 0x55f573ee8520 2024-08-31T06:11:01.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.531+0000 7f5123400640 1 --2- 172.21.15.16:0/4200020622 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f51100b4ca0 0x7f51100b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:01.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.531+0000 7f5123400640 1 -- 172.21.15.16:0/4200020622 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f511c0e4f30 con 0x7f512c102560 2024-08-31T06:11:01.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.531+0000 7f512aa00640 1 --2- 172.21.15.16:0/4200020622 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f51100b4ca0 0x7f51100b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:01.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.531+0000 7f51381ca740 1 -- 172.21.15.16:0/3275313307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f573ee8520 msgr2=0x55f57428def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:01.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.531+0000 7f51381ca740 1 --2- 172.21.15.16:0/3275313307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f573ee8520 0x55f57428def0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5124011850 tx=0x7f5124011d10 comp rx=0 tx=0).stop 2024-08-31T06:11:01.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.532+0000 7f51381ca740 1 -- 172.21.15.16:0/3275313307 shutdown_connections 2024-08-31T06:11:01.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.532+0000 7f51381ca740 1 --2- 172.21.15.16:0/3275313307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f573ec7840 0x55f57428e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:01.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.532+0000 7f51381ca740 1 --2- 172.21.15.16:0/3275313307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f573ebf660 0x55f57428e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:01.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.532+0000 7f51381ca740 1 --2- 172.21.15.16:0/3275313307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f573ee8520 0x55f57428def0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:01.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.532+0000 7f51381ca740 1 -- 172.21.15.16:0/3275313307 >> 172.21.15.16:0/3275313307 conn(0x55f573edc910 msgr2=0x55f57428bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:01.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.532+0000 7f51381ca740 1 -- 172.21.15.16:0/3275313307 shutdown_connections 2024-08-31T06:11:01.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.532+0000 7f51381ca740 1 -- 172.21.15.16:0/3275313307 wait complete. 2024-08-31T06:11:01.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.532+0000 7f512aa00640 1 --2- 172.21.15.16:0/4200020622 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f51100b4ca0 0x7f51100b71b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x55f57428cb70 tx=0x7f511c0e89e0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:01.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.532+0000 7f51381ca740 1 -- start start 2024-08-31T06:11:01.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.533+0000 7f51381ca740 1 Processor -- start 2024-08-31T06:11:01.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.533+0000 7f51381ca740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f573ebf660 0x55f57429bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:01.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.533+0000 7f51381ca740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f573ec7840 0x55f574299200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:01.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.533+0000 7f51381ca740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f573ee8520 0x55f5742998f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:01.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.533+0000 7f51381ca740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f574056bb0 con 0x55f573ebf660 2024-08-31T06:11:01.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.533+0000 7f51381ca740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f57417af90 con 0x55f573ee8520 2024-08-31T06:11:01.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.533+0000 7f51381ca740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f57429c6d0 con 0x55f573ec7840 2024-08-31T06:11:01.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.533+0000 7f51381ca740 10 client.0 _ll_register_callbacks cb 0x55f5741a0710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:11:01.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.533+0000 7f512b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f573ebf660 0x55f57429bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:01.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.534+0000 7f512b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f573ebf660 0x55f57429bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47448/0 (socket says 172.21.15.16:47448) 2024-08-31T06:11:01.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.534+0000 7f512b400640 1 -- 172.21.15.16:0/2248287118 learned_addr learned my addr 172.21.15.16:0/2248287118 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:01.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.534+0000 7f512a000640 1 --2- 172.21.15.16:0/2248287118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f573ec7840 0x55f574299200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:01.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.534+0000 7f512b400640 1 --2- 172.21.15.16:0/2248287118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f573ee8520 0x55f5742998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:01.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.534+0000 7f512a000640 1 -- 172.21.15.16:0/2248287118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f573ee8520 msgr2=0x55f5742998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:01.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.534+0000 7f512a000640 1 --2- 172.21.15.16:0/2248287118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f573ee8520 0x55f5742998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:01.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.534+0000 7f512a000640 1 -- 172.21.15.16:0/2248287118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f573ebf660 msgr2=0x55f57429bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:01.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.534+0000 7f512a000640 1 --2- 172.21.15.16:0/2248287118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f573ebf660 0x55f57429bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:01.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.534+0000 7f512a000640 1 -- 172.21.15.16:0/2248287118 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f573fca960 con 0x55f573ec7840 2024-08-31T06:11:01.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.535+0000 7f512a000640 1 --2- 172.21.15.16:0/2248287118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f573ec7840 0x55f574299200 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f511802a940 tx=0x7f51180316d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f39f0a21ce2c1fdc server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:01.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.535+0000 7f510be00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:11:01.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.535+0000 7f510be00640 1 -- 172.21.15.16:0/2248287118 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5118035040 con 0x55f573ec7840 2024-08-31T06:11:01.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.535+0000 7f51381ca740 1 -- 172.21.15.16:0/2248287118 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f574192e30 con 0x55f573ec7840 2024-08-31T06:11:01.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.535+0000 7f510be00640 1 -- 172.21.15.16:0/2248287118 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f5118004240 con 0x55f573ec7840 2024-08-31T06:11:01.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.535+0000 7f510be00640 1 -- 172.21.15.16:0/2248287118 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5118004440 con 0x55f573ec7840 2024-08-31T06:11:01.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.535+0000 7f51381ca740 10 client.26482 fetch_fsmap learned FSMap version 32 2024-08-31T06:11:01.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.535+0000 7f51381ca740 1 -- 172.21.15.16:0/2248287118 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f5742a7fb0 con 0x55f573ec7840 2024-08-31T06:11:01.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.536+0000 7f510be00640 1 -- 172.21.15.16:0/2248287118 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f574192e30 con 0x55f573ec7840 2024-08-31T06:11:01.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.536+0000 7f510be00640 1 -- 172.21.15.16:0/2248287118 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f5118004990 con 0x55f573ec7840 2024-08-31T06:11:01.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.536+0000 7f51381ca740 10 client.26482 fetch_fsmap finished waiting for FSMap version 32 2024-08-31T06:11:01.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.536+0000 7f51381ca740 10 client.26482 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:11:01.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.536+0000 7f51381ca740 20 client.26482 populate_metadata read hostname 'smithi016' 2024-08-31T06:11:01.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.536+0000 7f51381ca740 1 --2- 172.21.15.16:0/2248287118 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f573ea6ea0 0x55f5742ab450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:01.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.536+0000 7f51381ca740 4 client.26482 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:11:01.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.536+0000 7f51381ca740 1 -- 172.21.15.16:0/2248287118 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5742069e0 con 0x55f573ea6ea0 2024-08-31T06:11:01.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.536+0000 7f512b400640 1 --2- 172.21.15.16:0/2248287118 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f573ea6ea0 0x55f5742ab450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:01.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.537+0000 7f512b400640 1 --2- 172.21.15.16:0/2248287118 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f573ea6ea0 0x55f5742ab450 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=873a342d5f887242 server_cookie=19b64f0db9276013 in_seq=0 out_seq=0 2024-08-31T06:11:01.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.537+0000 7f510be00640 10 client.26482 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:11:01.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.538+0000 7f510be00640 1 -- 172.21.15.16:0/2248287118 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f5742069e0 con 0x55f573ea6ea0 2024-08-31T06:11:01.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.538+0000 7f510be00640 10 client.26482 handle_command_reply: tid=0 2024-08-31T06:11:01.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.538+0000 7f51381ca740 1 client.26482 shutdown 2024-08-31T06:11:01.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.538+0000 7f51381ca740 1 -- 172.21.15.16:0/2248287118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f573ec7840 msgr2=0x55f574299200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:01.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.538+0000 7f51381ca740 1 --2- 172.21.15.16:0/2248287118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f573ec7840 0x55f574299200 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f511802a940 tx=0x7f51180316d0 comp rx=0 tx=0).stop 2024-08-31T06:11:01.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.538+0000 7f51381ca740 1 -- 172.21.15.16:0/2248287118 shutdown_connections 2024-08-31T06:11:01.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.538+0000 7f51381ca740 1 --2- 172.21.15.16:0/2248287118 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f573ea6ea0 0x55f5742ab450 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:01.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.538+0000 7f51381ca740 1 --2- 172.21.15.16:0/2248287118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f573ec7840 0x55f574299200 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:01.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.538+0000 7f51381ca740 1 --2- 172.21.15.16:0/2248287118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f573ee8520 0x55f5742998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:01.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.538+0000 7f51381ca740 1 --2- 172.21.15.16:0/2248287118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f573ebf660 0x55f57429bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:01.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.538+0000 7f510be00640 0 client.26482 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:11:01.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.538+0000 7f51381ca740 1 -- 172.21.15.16:0/2248287118 >> 172.21.15.16:0/2248287118 conn(0x55f573edc910 msgr2=0x55f5742910d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:01.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.539+0000 7f51381ca740 1 -- 172.21.15.16:0/2248287118 shutdown_connections 2024-08-31T06:11:01.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.539+0000 7f51381ca740 1 -- 172.21.15.16:0/2248287118 wait complete. 2024-08-31T06:11:01.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.539+0000 7f51381ca740 20 client.26482 trim_cache size 0 max 16384 2024-08-31T06:11:01.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.557+0000 7f51381ca740 1 Processor -- start 2024-08-31T06:11:01.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.557+0000 7f51381ca740 1 -- start start 2024-08-31T06:11:01.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.557+0000 7f51381ca740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f573ebf660 0x55f57428b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:01.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.557+0000 7f51381ca740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f573ec7840 0x55f57428b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:01.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.557+0000 7f51381ca740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f573ee8520 0x55f5742a4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:01.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.557+0000 7f51381ca740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f512c1b8450 con 0x55f573ee8520 2024-08-31T06:11:01.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.557+0000 7f51381ca740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f512c1b5c90 con 0x55f573ec7840 2024-08-31T06:11:01.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.557+0000 7f51381ca740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f512c1b3ae0 con 0x55f573ebf660 2024-08-31T06:11:01.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.558+0000 7f512b400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f573ebf660 0x55f57428b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:01.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.558+0000 7f512b400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f573ebf660 0x55f57428b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:35778/0 (socket says 172.21.15.16:35778) 2024-08-31T06:11:01.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.558+0000 7f512b400640 1 -- 172.21.15.16:0/3446406699 learned_addr learned my addr 172.21.15.16:0/3446406699 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:01.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.558+0000 7f512a000640 1 --2- 172.21.15.16:0/3446406699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f573ec7840 0x55f57428b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:01.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.558+0000 7f512b400640 1 --2- 172.21.15.16:0/3446406699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f573ee8520 0x55f5742a4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:01.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.558+0000 7f512b400640 1 -- 172.21.15.16:0/3446406699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f573ec7840 msgr2=0x55f57428b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:01.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.558+0000 7f512b400640 1 --2- 172.21.15.16:0/3446406699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f573ec7840 0x55f57428b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:01.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.558+0000 7f512b400640 1 -- 172.21.15.16:0/3446406699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f573ee8520 msgr2=0x55f5742a4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:01.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.558+0000 7f512b400640 1 --2- 172.21.15.16:0/3446406699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f573ee8520 0x55f5742a4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:01.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.558+0000 7f512b400640 1 -- 172.21.15.16:0/3446406699 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5742904d0 con 0x55f573ebf660 2024-08-31T06:11:01.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.558+0000 7f512a000640 1 --2- 172.21.15.16:0/3446406699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f573ec7840 0x55f57428b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:11:01.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.558+0000 7f512b400640 1 --2- 172.21.15.16:0/3446406699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f573ebf660 0x55f57428b3d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5124011850 tx=0x7f5124011290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5212effaea482a57 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:01.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.559+0000 7f5116a00640 1 -- 172.21.15.16:0/3446406699 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5124010fe0 con 0x55f573ebf660 2024-08-31T06:11:01.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.559+0000 7f5116a00640 1 -- 172.21.15.16:0/3446406699 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5124010430 con 0x55f573ebf660 2024-08-31T06:11:01.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.559+0000 7f5116a00640 1 -- 172.21.15.16:0/3446406699 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512400f9a0 con 0x55f573ebf660 2024-08-31T06:11:01.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.560+0000 7f51381ca740 1 -- 172.21.15.16:0/3446406699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f573ebf660 msgr2=0x55f57428b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:01.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.560+0000 7f51381ca740 1 --2- 172.21.15.16:0/3446406699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f573ebf660 0x55f57428b3d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5124011850 tx=0x7f5124011290 comp rx=0 tx=0).stop 2024-08-31T06:11:01.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.560+0000 7f51381ca740 1 -- 172.21.15.16:0/3446406699 shutdown_connections 2024-08-31T06:11:01.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.560+0000 7f51381ca740 1 --2- 172.21.15.16:0/3446406699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f573ee8520 0x55f5742a4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:01.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.560+0000 7f51381ca740 1 --2- 172.21.15.16:0/3446406699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f573ec7840 0x55f57428b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:01.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.560+0000 7f51381ca740 1 --2- 172.21.15.16:0/3446406699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f573ebf660 0x55f57428b3d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:01.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.560+0000 7f51381ca740 1 -- 172.21.15.16:0/3446406699 >> 172.21.15.16:0/3446406699 conn(0x55f573edc910 msgr2=0x55f5742a48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:01.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.560+0000 7f51381ca740 1 -- 172.21.15.16:0/3446406699 shutdown_connections 2024-08-31T06:11:01.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.560+0000 7f51381ca740 1 -- 172.21.15.16:0/3446406699 wait complete. 2024-08-31T06:11:01.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.560+0000 7f51381ca740 1 -- start start 2024-08-31T06:11:01.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.560+0000 7f51381ca740 1 Processor -- start 2024-08-31T06:11:01.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.561+0000 7f51381ca740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f573ebf660 0x55f57429e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:01.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.561+0000 7f51381ca740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f573ec7840 0x55f57429e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:01.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.561+0000 7f51381ca740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f573ee8520 0x55f57430eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:01.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.561+0000 7f51381ca740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f512c1b4150 con 0x55f573ee8520 2024-08-31T06:11:01.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.561+0000 7f51381ca740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f512c1b55c0 con 0x55f573ebf660 2024-08-31T06:11:01.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.561+0000 7f51381ca740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f512c1b4820 con 0x55f573ec7840 2024-08-31T06:11:01.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.561+0000 7f51381ca740 10 client.0 _ll_register_callbacks cb 0x7f5118032ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:11:01.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.561+0000 7f512b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f573ee8520 0x55f57430eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:01.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.561+0000 7f512a000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f573ec7840 0x55f57429e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:01.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.561+0000 7f512a000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f573ec7840 0x55f57429e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:35780/0 (socket says 172.21.15.16:35780) 2024-08-31T06:11:01.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.561+0000 7f512a000640 1 -- 172.21.15.16:0/634909482 learned_addr learned my addr 172.21.15.16:0/634909482 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:01.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.561+0000 7f512b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f573ee8520 0x55f57430eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47460/0 (socket says 172.21.15.16:47460) 2024-08-31T06:11:01.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.562+0000 7f512b400640 1 --2- 172.21.15.16:0/634909482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f573ebf660 0x55f57429e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:01.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.562+0000 7f512a000640 1 -- 172.21.15.16:0/634909482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f573ebf660 msgr2=0x55f57429e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:01.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.562+0000 7f512a000640 1 --2- 172.21.15.16:0/634909482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f573ebf660 0x55f57429e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:01.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.562+0000 7f512a000640 1 -- 172.21.15.16:0/634909482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f573ee8520 msgr2=0x55f57430eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:01.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.562+0000 7f512a000640 1 --2- 172.21.15.16:0/634909482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f573ee8520 0x55f57430eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:01.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.562+0000 7f512a000640 1 -- 172.21.15.16:0/634909482 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f57430f530 con 0x55f573ec7840 2024-08-31T06:11:01.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.562+0000 7f512a000640 1 --2- 172.21.15.16:0/634909482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f573ec7840 0x55f57429e950 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f511802a940 tx=0x7f511802a420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f7aaddca06f1d4c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:01.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.562+0000 7f510be00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:11:01.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.562+0000 7f510be00640 1 -- 172.21.15.16:0/634909482 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5118035040 con 0x55f573ec7840 2024-08-31T06:11:01.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.563+0000 7f51381ca740 1 -- 172.21.15.16:0/634909482 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5124010430 con 0x55f573ec7840 2024-08-31T06:11:01.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.563+0000 7f510be00640 1 -- 172.21.15.16:0/634909482 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f5118031980 con 0x55f573ec7840 2024-08-31T06:11:01.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.563+0000 7f510be00640 1 -- 172.21.15.16:0/634909482 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5118031b80 con 0x55f573ec7840 2024-08-31T06:11:01.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.563+0000 7f51381ca740 10 client.26488 fetch_fsmap learned FSMap version 32 2024-08-31T06:11:01.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.563+0000 7f51381ca740 1 -- 172.21.15.16:0/634909482 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f574316940 con 0x55f573ec7840 2024-08-31T06:11:01.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.563+0000 7f510be00640 1 -- 172.21.15.16:0/634909482 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5124010430 con 0x55f573ec7840 2024-08-31T06:11:01.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.563+0000 7f510be00640 1 -- 172.21.15.16:0/634909482 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f5118003a60 con 0x55f573ec7840 2024-08-31T06:11:01.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.563+0000 7f51381ca740 10 client.26488 fetch_fsmap finished waiting for FSMap version 32 2024-08-31T06:11:01.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.563+0000 7f51381ca740 10 client.26488 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:11:01.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.563+0000 7f51381ca740 20 client.26488 populate_metadata read hostname 'smithi016' 2024-08-31T06:11:01.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.563+0000 7f51381ca740 1 --2- 172.21.15.16:0/634909482 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f573e7c250 0x55f5743191b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:01.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.563+0000 7f51381ca740 4 client.26488 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9b804def", "release": true, "await": true}] 2024-08-31T06:11:01.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.564+0000 7f51381ca740 1 -- 172.21.15.16:0/634909482 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9b804def", "release": true, "await": true}) -- 0x55f574202480 con 0x55f573e7c250 2024-08-31T06:11:01.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.564+0000 7f512b400640 1 --2- 172.21.15.16:0/634909482 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f573e7c250 0x55f5743191b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:01.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.564+0000 7f512b400640 1 --2- 172.21.15.16:0/634909482 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f573e7c250 0x55f5743191b0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d0f7952518ea8cf server_cookie=f71e1f2bd8ab917 in_seq=0 out_seq=0 2024-08-31T06:11:01.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.564+0000 7f510be00640 10 client.26488 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:11:01.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 215.0 s, 0.8 tps, lat 21081.635 ms stddev 74.849 2024-08-31T06:11:01.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.756+0000 7f510be00640 1 -- 172.21.15.16:0/634909482 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f574202480 con 0x55f573e7c250 2024-08-31T06:11:01.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.756+0000 7f510be00640 10 client.26488 handle_command_reply: tid=0 2024-08-31T06:11:01.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.757+0000 7f51381ca740 1 client.26488 shutdown 2024-08-31T06:11:01.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.757+0000 7f51381ca740 1 -- 172.21.15.16:0/634909482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f573ec7840 msgr2=0x55f57429e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:01.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.757+0000 7f51381ca740 1 --2- 172.21.15.16:0/634909482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f573ec7840 0x55f57429e950 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f511802a940 tx=0x7f511802a420 comp rx=0 tx=0).stop 2024-08-31T06:11:01.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.757+0000 7f51381ca740 1 -- 172.21.15.16:0/634909482 shutdown_connections 2024-08-31T06:11:01.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.757+0000 7f51381ca740 1 --2- 172.21.15.16:0/634909482 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f573e7c250 0x55f5743191b0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:01.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.757+0000 7f510be00640 0 client.26488 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:11:01.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.757+0000 7f51381ca740 1 --2- 172.21.15.16:0/634909482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f573ec7840 0x55f57429e950 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:01.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.758+0000 7f51381ca740 1 --2- 172.21.15.16:0/634909482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f573ee8520 0x55f57430eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:01.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.758+0000 7f51381ca740 1 --2- 172.21.15.16:0/634909482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f573ebf660 0x55f57429e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:01.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.758+0000 7f51381ca740 1 -- 172.21.15.16:0/634909482 >> 172.21.15.16:0/634909482 conn(0x55f573edc910 msgr2=0x55f5742a5270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:01.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.758+0000 7f51381ca740 1 -- 172.21.15.16:0/634909482 shutdown_connections 2024-08-31T06:11:01.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.758+0000 7f51381ca740 1 -- 172.21.15.16:0/634909482 wait complete. 2024-08-31T06:11:01.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.758+0000 7f51381ca740 20 client.26488 trim_cache size 0 max 16384 2024-08-31T06:11:01.762 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:11:01.762 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 32, 2024-08-31T06:11:01.762 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24737, 2024-08-31T06:11:01.762 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 16, 2024-08-31T06:11:01.762 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-08-31T06:11:01.762 INFO:teuthology.orchestra.run.smithi016.stdout: "9b804def": { 2024-08-31T06:11:01.763 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 16, 2024-08-31T06:11:01.763 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-08-31T06:11:01.763 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:11:01.763 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-08-31T06:11:01.763 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:11:01.763 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:11:01.763 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-08-31T06:11:01.763 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 141.8, 2024-08-31T06:11:01.763 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-08-31T06:11:01.763 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-08-31T06:11:01.763 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-08-31T06:11:01.763 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:11:01.763 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-08-31T06:11:01.763 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:11:01.764 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:11:01.764 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:11:01.764 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:11:01.764 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:11:01.764 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:11:01.764 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:11:01.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.760+0000 7f5131600640 1 -- 172.21.15.16:0/4200020622 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f51100b4ca0 msgr2=0x7f51100b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:01.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.760+0000 7f5131600640 1 --2- 172.21.15.16:0/4200020622 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f51100b4ca0 0x7f51100b71b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x55f57428cb70 tx=0x7f511c0e89e0 comp rx=0 tx=0).stop 2024-08-31T06:11:01.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.760+0000 7f5131600640 1 -- 172.21.15.16:0/4200020622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f512c102560 msgr2=0x7f512c1a0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:01.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.760+0000 7f5131600640 1 --2- 172.21.15.16:0/4200020622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f512c102560 0x7f512c1a0950 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f512c06c6e0 tx=0x7f511c00dde0 comp rx=0 tx=0).stop 2024-08-31T06:11:01.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.761+0000 7f5131600640 1 -- 172.21.15.16:0/4200020622 shutdown_connections 2024-08-31T06:11:01.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.761+0000 7f5131600640 1 --2- 172.21.15.16:0/4200020622 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f51100b4ca0 0x7f51100b71b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:01.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.761+0000 7f5131600640 1 --2- 172.21.15.16:0/4200020622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f512c1079a0 0x7f512c1a6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:01.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.761+0000 7f5131600640 1 --2- 172.21.15.16:0/4200020622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f512c102eb0 0x7f512c1a0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:01.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.761+0000 7f5131600640 1 --2- 172.21.15.16:0/4200020622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f512c102560 0x7f512c1a0950 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:01.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.761+0000 7f5131600640 1 -- 172.21.15.16:0/4200020622 >> 172.21.15.16:0/4200020622 conn(0x7f512c0794e0 msgr2=0x7f512c1053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:01.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.761+0000 7f5131600640 1 -- 172.21.15.16:0/4200020622 shutdown_connections 2024-08-31T06:11:01.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:01.763+0000 7f5131600640 1 -- 172.21.15.16:0/4200020622 wait complete. 2024-08-31T06:11:02.598 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:02 smithi016 ceph-mon[17194]: pgmap v478: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 873 KiB/s wr, 170 op/s 2024-08-31T06:11:02.846 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9b804def', total seconds elapsed: 26.6 2024-08-31T06:11:02.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:02 smithi155 ceph-mon[21742]: pgmap v478: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 873 KiB/s wr, 170 op/s 2024-08-31T06:11:02.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:02 smithi184 ceph-mon[22331]: pgmap v478: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 873 KiB/s wr, 170 op/s 2024-08-31T06:11:03.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:03 smithi155 ceph-mon[21742]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:11:03.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:03 smithi016 ceph-mon[17194]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:11:03.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:03 smithi184 ceph-mon[22331]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:11:04.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:04 smithi155 ceph-mon[21742]: pgmap v479: 97 pgs: 97 active+clean; 2.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 5.2 MiB/s wr, 569 op/s 2024-08-31T06:11:04.873 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:04 smithi184 ceph-mon[22331]: pgmap v479: 97 pgs: 97 active+clean; 2.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 5.2 MiB/s wr, 569 op/s 2024-08-31T06:11:04.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:04 smithi016 ceph-mon[17194]: pgmap v479: 97 pgs: 97 active+clean; 2.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 5.2 MiB/s wr, 569 op/s 2024-08-31T06:11:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.980+0000 7fbcfa800640 1 -- 172.21.15.16:0/3944858429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcf4102150 msgr2=0x7fbcf4102560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.980+0000 7fbcfa800640 1 --2- 172.21.15.16:0/3944858429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcf4102150 0x7fbcf4102560 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbce4009be0 tx=0x7fbce402a6f0 comp rx=0 tx=0).stop 2024-08-31T06:11:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.980+0000 7fbcfa800640 1 -- 172.21.15.16:0/3944858429 shutdown_connections 2024-08-31T06:11:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.980+0000 7fbcfa800640 1 --2- 172.21.15.16:0/3944858429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcf410fd00 0x7fbcf4112120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.980+0000 7fbcfa800640 1 --2- 172.21.15.16:0/3944858429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcf4102aa0 0x7fbcf410f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.980+0000 7fbcfa800640 1 --2- 172.21.15.16:0/3944858429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcf4102150 0x7fbcf4102560 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.980+0000 7fbcfa800640 1 -- 172.21.15.16:0/3944858429 >> 172.21.15.16:0/3944858429 conn(0x7fbcf40fdfd0 msgr2=0x7fbcf4100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.980+0000 7fbcfa800640 1 -- 172.21.15.16:0/3944858429 shutdown_connections 2024-08-31T06:11:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.980+0000 7fbcfa800640 1 -- 172.21.15.16:0/3944858429 wait complete. 2024-08-31T06:11:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.982+0000 7fbcfa800640 1 Processor -- start 2024-08-31T06:11:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.982+0000 7fbcfa800640 1 -- start start 2024-08-31T06:11:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.983+0000 7fbcfa800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcf4102150 0x7fbcf419c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.983+0000 7fbcfa800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcf4102aa0 0x7fbcf419c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.983+0000 7fbcfa800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcf410fd00 0x7fbcf41a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.983+0000 7fbcfa800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf4115860 con 0x7fbcf410fd00 2024-08-31T06:11:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.983+0000 7fbcfa800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf41156e0 con 0x7fbcf4102150 2024-08-31T06:11:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.983+0000 7fbcfa800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf41159e0 con 0x7fbcf4102aa0 2024-08-31T06:11:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.983+0000 7fbcf8a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcf4102aa0 0x7fbcf419c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.983+0000 7fbcf9e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcf410fd00 0x7fbcf41a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.983+0000 7fbcf8a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcf4102aa0 0x7fbcf419c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:35802/0 (socket says 172.21.15.16:35802) 2024-08-31T06:11:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.983+0000 7fbcf8a00640 1 -- 172.21.15.16:0/2296456629 learned_addr learned my addr 172.21.15.16:0/2296456629 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.983+0000 7fbcf9400640 1 --2- 172.21.15.16:0/2296456629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcf4102150 0x7fbcf419c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.984+0000 7fbcf8a00640 1 -- 172.21.15.16:0/2296456629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcf4102150 msgr2=0x7fbcf419c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.984+0000 7fbcf8a00640 1 --2- 172.21.15.16:0/2296456629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcf4102150 0x7fbcf419c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.984+0000 7fbcf8a00640 1 -- 172.21.15.16:0/2296456629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcf410fd00 msgr2=0x7fbcf41a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.984+0000 7fbcf8a00640 1 --2- 172.21.15.16:0/2296456629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcf410fd00 0x7fbcf41a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.984+0000 7fbcf8a00640 1 -- 172.21.15.16:0/2296456629 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf41a21f0 con 0x7fbcf4102aa0 2024-08-31T06:11:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.984+0000 7fbcf9400640 1 --2- 172.21.15.16:0/2296456629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcf4102150 0x7fbcf419c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:11:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.984+0000 7fbcf9e00640 1 --2- 172.21.15.16:0/2296456629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcf410fd00 0x7fbcf41a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:11:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.985+0000 7fbcf8a00640 1 --2- 172.21.15.16:0/2296456629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcf4102aa0 0x7fbcf419c860 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbce00058c0 tx=0x7fbce000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.985+0000 7fbcf2000640 1 -- 172.21.15.16:0/2296456629 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce000cd30 con 0x7fbcf4102aa0 2024-08-31T06:11:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.985+0000 7fbcf2000640 1 -- 172.21.15.16:0/2296456629 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbce00035f0 con 0x7fbcf4102aa0 2024-08-31T06:11:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.985+0000 7fbcf2000640 1 -- 172.21.15.16:0/2296456629 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce00044e0 con 0x7fbcf4102aa0 2024-08-31T06:11:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.985+0000 7fbcfa800640 1 -- 172.21.15.16:0/2296456629 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcf41a24e0 con 0x7fbcf4102aa0 2024-08-31T06:11:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.985+0000 7fbcfa800640 1 -- 172.21.15.16:0/2296456629 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcf4104360 con 0x7fbcf4102aa0 2024-08-31T06:11:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.990+0000 7fbd02e1f740 1 Processor -- start 2024-08-31T06:11:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.991+0000 7fbd02e1f740 1 -- start start 2024-08-31T06:11:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.991+0000 7fbd02e1f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b1efe51b0 0x557b1f3888a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.992+0000 7fbd02e1f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b1efbc2f0 0x557b1f388de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:04.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.992+0000 7fbd02e1f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1efc44d0 0x557b1f389320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:04.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.992+0000 7fbd02e1f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557b1f2fdf70 con 0x557b1efc44d0 2024-08-31T06:11:04.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.992+0000 7fbd02e1f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557b1f2ed500 con 0x557b1efe51b0 2024-08-31T06:11:04.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.992+0000 7fbd02e1f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557b1f2f14b0 con 0x557b1efbc2f0 2024-08-31T06:11:04.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.992+0000 7fbcf9e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b1efe51b0 0x557b1f3888a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:04.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.992+0000 7fbcf9e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b1efe51b0 0x557b1f3888a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:45948/0 (socket says 172.21.15.16:45948) 2024-08-31T06:11:04.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.992+0000 7fbcf9e00640 1 -- 172.21.15.16:0/57963149 learned_addr learned my addr 172.21.15.16:0/57963149 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:04.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.992+0000 7fbcf9400640 1 --2- 172.21.15.16:0/57963149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b1efbc2f0 0x557b1f388de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:04.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.993+0000 7fbcf2000640 1 -- 172.21.15.16:0/2296456629 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fbce0003aa0 con 0x7fbcf4102aa0 2024-08-31T06:11:04.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.993+0000 7fbcf8a00640 1 --2- 172.21.15.16:0/57963149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1efc44d0 0x557b1f389320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:04.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.993+0000 7fbcf9e00640 1 -- 172.21.15.16:0/57963149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b1efbc2f0 msgr2=0x557b1f388de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:04.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.993+0000 7fbcf9e00640 1 --2- 172.21.15.16:0/57963149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b1efbc2f0 0x557b1f388de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:04.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.993+0000 7fbcf9e00640 1 -- 172.21.15.16:0/57963149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1efc44d0 msgr2=0x557b1f389320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:04.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.993+0000 7fbcf9e00640 1 --2- 172.21.15.16:0/57963149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1efc44d0 0x557b1f389320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:04.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.994+0000 7fbcf9e00640 1 -- 172.21.15.16:0/57963149 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b1f2a7c10 con 0x557b1efe51b0 2024-08-31T06:11:04.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.994+0000 7fbcf9400640 1 --2- 172.21.15.16:0/57963149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b1efbc2f0 0x557b1f388de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:11:04.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.994+0000 7fbcf9e00640 1 --2- 172.21.15.16:0/57963149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b1efe51b0 0x557b1f3888a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbcec009da0 tx=0x7fbcec00e5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aeb0a629e41a4685 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:04.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.994+0000 7fbcea600640 1 -- 172.21.15.16:0/57963149 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcec017070 con 0x557b1efe51b0 2024-08-31T06:11:04.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.995+0000 7fbcea600640 1 -- 172.21.15.16:0/57963149 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcec013920 con 0x557b1efe51b0 2024-08-31T06:11:04.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.995+0000 7fbcea600640 1 -- 172.21.15.16:0/57963149 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcec0128d0 con 0x557b1efe51b0 2024-08-31T06:11:04.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.996+0000 7fbd02e1f740 1 -- 172.21.15.16:0/57963149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b1efe51b0 msgr2=0x557b1f3888a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:04.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.996+0000 7fbd02e1f740 1 --2- 172.21.15.16:0/57963149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b1efe51b0 0x557b1f3888a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbcec009da0 tx=0x7fbcec00e5a0 comp rx=0 tx=0).stop 2024-08-31T06:11:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.996+0000 7fbcf2000640 1 --2- 172.21.15.16:0/2296456629 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbcd80b4d20 0x7fbcd80b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.996+0000 7fbd02e1f740 1 -- 172.21.15.16:0/57963149 shutdown_connections 2024-08-31T06:11:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.996+0000 7fbd02e1f740 1 --2- 172.21.15.16:0/57963149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b1efbc2f0 0x557b1f388de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.996+0000 7fbd02e1f740 1 --2- 172.21.15.16:0/57963149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1efc44d0 0x557b1f389320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.996+0000 7fbd02e1f740 1 --2- 172.21.15.16:0/57963149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b1efe51b0 0x557b1f3888a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.996+0000 7fbd02e1f740 1 -- 172.21.15.16:0/57963149 >> 172.21.15.16:0/57963149 conn(0x557b1efd9300 msgr2=0x557b1f386790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.997+0000 7fbcf9e00640 1 --2- 172.21.15.16:0/2296456629 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbcd80b4d20 0x7fbcd80b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.999+0000 7fbcf2000640 1 -- 172.21.15.16:0/2296456629 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fbce00e56d0 con 0x7fbcf4102aa0 2024-08-31T06:11:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.999+0000 7fbcf9e00640 1 --2- 172.21.15.16:0/2296456629 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbcd80b4d20 0x7fbcd80b7230 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbcec00a010 tx=0x7fbcec009870 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.999+0000 7fbd02e1f740 1 -- 172.21.15.16:0/57963149 shutdown_connections 2024-08-31T06:11:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:04.999+0000 7fbd02e1f740 1 -- 172.21.15.16:0/57963149 wait complete. 2024-08-31T06:11:05.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.002+0000 7fbd02e1f740 1 -- start start 2024-08-31T06:11:05.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.002+0000 7fbd02e1f740 1 Processor -- start 2024-08-31T06:11:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.003+0000 7fbd02e1f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1efbc2f0 0x557b1f393ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.003+0000 7fbd02e1f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b1efc44d0 0x557b1f3943e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.004+0000 7fbd02e1f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b1efe51b0 0x557b1f394940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.004+0000 7fbd02e1f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557b1f226ab0 con 0x557b1efbc2f0 2024-08-31T06:11:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.004+0000 7fbd02e1f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557b1f294320 con 0x557b1efe51b0 2024-08-31T06:11:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.004+0000 7fbd02e1f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557b1f397020 con 0x557b1efc44d0 2024-08-31T06:11:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.004+0000 7fbd02e1f740 10 client.0 _ll_register_callbacks cb 0x557b1f2cde80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:11:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.004+0000 7fbcf9e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b1efe51b0 0x557b1f394940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.004+0000 7fbcf8a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b1efc44d0 0x557b1f3943e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.005+0000 7fbcf8a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b1efc44d0 0x557b1f3943e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:35820/0 (socket says 172.21.15.16:35820) 2024-08-31T06:11:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.005+0000 7fbcf9e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b1efe51b0 0x557b1f394940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:45960/0 (socket says 172.21.15.16:45960) 2024-08-31T06:11:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.005+0000 7fbcf8a00640 1 -- 172.21.15.16:0/2644996988 learned_addr learned my addr 172.21.15.16:0/2644996988 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.005+0000 7fbcf9400640 1 --2- 172.21.15.16:0/2644996988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1efbc2f0 0x557b1f393ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.005+0000 7fbcf8a00640 1 -- 172.21.15.16:0/2644996988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b1efe51b0 msgr2=0x557b1f394940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.005+0000 7fbcf8a00640 1 --2- 172.21.15.16:0/2644996988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b1efe51b0 0x557b1f394940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.005+0000 7fbcf8a00640 1 -- 172.21.15.16:0/2644996988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1efbc2f0 msgr2=0x557b1f393ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.005+0000 7fbcf8a00640 1 --2- 172.21.15.16:0/2644996988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1efbc2f0 0x557b1f393ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.006+0000 7fbcf8a00640 1 -- 172.21.15.16:0/2644996988 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b1f050790 con 0x557b1efc44d0 2024-08-31T06:11:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.007+0000 7fbcf8a00640 1 --2- 172.21.15.16:0/2644996988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b1efc44d0 0x557b1f3943e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbce0000c00 tx=0x7fbce00ab590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4d7bc160b9f5d3b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.007+0000 7fbcdea00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:11:05.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.007+0000 7fbcdea00640 1 -- 172.21.15.16:0/2644996988 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce00932c0 con 0x557b1efc44d0 2024-08-31T06:11:05.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.007+0000 7fbcdea00640 1 -- 172.21.15.16:0/2644996988 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fbce00b02e0 con 0x557b1efc44d0 2024-08-31T06:11:05.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.007+0000 7fbd02e1f740 1 -- 172.21.15.16:0/2644996988 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcec013920 con 0x557b1efc44d0 2024-08-31T06:11:05.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.008+0000 7fbcdea00640 1 -- 172.21.15.16:0/2644996988 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbce00ac520 con 0x557b1efc44d0 2024-08-31T06:11:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.008+0000 7fbd02e1f740 10 client.26500 fetch_fsmap learned FSMap version 33 2024-08-31T06:11:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.008+0000 7fbd02e1f740 1 -- 172.21.15.16:0/2644996988 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557b1f0c9620 con 0x557b1efc44d0 2024-08-31T06:11:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.009+0000 7fbcdea00640 1 -- 172.21.15.16:0/2644996988 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcec013920 con 0x557b1efc44d0 2024-08-31T06:11:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.010+0000 7fbcdea00640 1 -- 172.21.15.16:0/2644996988 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7fbce00b4cb0 con 0x557b1efc44d0 2024-08-31T06:11:05.013 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.010+0000 7fbd02e1f740 10 client.26500 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T06:11:05.013 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.010+0000 7fbd02e1f740 10 client.26500 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:11:05.014 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.010+0000 7fbd02e1f740 20 client.26500 populate_metadata read hostname 'smithi016' 2024-08-31T06:11:05.014 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.010+0000 7fbd02e1f740 1 --2- 172.21.15.16:0/2644996988 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557b1efa3d80 0x557b1f3a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:05.014 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.010+0000 7fbd02e1f740 4 client.26500 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:11:05.014 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.010+0000 7fbd02e1f740 1 -- 172.21.15.16:0/2644996988 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b1f2ffec0 con 0x557b1efa3d80 2024-08-31T06:11:05.014 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.011+0000 7fbcf9400640 1 --2- 172.21.15.16:0/2644996988 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557b1efa3d80 0x557b1f3a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:05.014 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.012+0000 7fbcf9400640 1 --2- 172.21.15.16:0/2644996988 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557b1efa3d80 0x557b1f3a5fa0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e39e1a49332aafb7 server_cookie=cbb85e173b74591a in_seq=0 out_seq=0 2024-08-31T06:11:05.014 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.012+0000 7fbcdea00640 10 client.26500 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:11:05.015 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.013+0000 7fbcdea00640 1 -- 172.21.15.16:0/2644996988 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557b1f2ffec0 con 0x557b1efa3d80 2024-08-31T06:11:05.016 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.013+0000 7fbcdea00640 10 client.26500 handle_command_reply: tid=0 2024-08-31T06:11:05.016 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.014+0000 7fbd02e1f740 1 client.26500 shutdown 2024-08-31T06:11:05.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.014+0000 7fbd02e1f740 1 -- 172.21.15.16:0/2644996988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b1efc44d0 msgr2=0x557b1f3943e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:05.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.014+0000 7fbd02e1f740 1 --2- 172.21.15.16:0/2644996988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b1efc44d0 0x557b1f3943e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbce0000c00 tx=0x7fbce00ab590 comp rx=0 tx=0).stop 2024-08-31T06:11:05.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.015+0000 7fbd02e1f740 1 -- 172.21.15.16:0/2644996988 shutdown_connections 2024-08-31T06:11:05.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.015+0000 7fbd02e1f740 1 --2- 172.21.15.16:0/2644996988 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557b1efa3d80 0x557b1f3a5fa0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:05.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.015+0000 7fbd02e1f740 1 --2- 172.21.15.16:0/2644996988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b1efc44d0 0x557b1f3943e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:05.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.015+0000 7fbd02e1f740 1 --2- 172.21.15.16:0/2644996988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b1efe51b0 0x557b1f394940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:05.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.015+0000 7fbd02e1f740 1 --2- 172.21.15.16:0/2644996988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1efbc2f0 0x557b1f393ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:05.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.015+0000 7fbd02e1f740 1 -- 172.21.15.16:0/2644996988 >> 172.21.15.16:0/2644996988 conn(0x557b1efd9300 msgr2=0x557b1f3898d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:05.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.015+0000 7fbcdea00640 0 client.26500 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:11:05.019 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.016+0000 7fbd02e1f740 1 -- 172.21.15.16:0/2644996988 shutdown_connections 2024-08-31T06:11:05.020 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.018+0000 7fbd02e1f740 1 -- 172.21.15.16:0/2644996988 wait complete. 2024-08-31T06:11:05.020 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.018+0000 7fbd02e1f740 20 client.26500 trim_cache size 0 max 16384 2024-08-31T06:11:05.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.059+0000 7fbd02e1f740 1 Processor -- start 2024-08-31T06:11:05.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.060+0000 7fbd02e1f740 1 -- start start 2024-08-31T06:11:05.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.060+0000 7fbd02e1f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b1efbc2f0 0x557b1f385480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:05.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.060+0000 7fbd02e1f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b1efc44d0 0x557b1f391070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:05.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.060+0000 7fbd02e1f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1efe51b0 0x557b1f3915b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:05.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.060+0000 7fbd02e1f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf41b81d0 con 0x557b1efe51b0 2024-08-31T06:11:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.060+0000 7fbd02e1f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf41b5a40 con 0x557b1efbc2f0 2024-08-31T06:11:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.060+0000 7fbd02e1f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf41b3890 con 0x557b1efc44d0 2024-08-31T06:11:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.061+0000 7fbcf8a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b1efc44d0 0x557b1f391070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.061+0000 7fbcf9400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b1efbc2f0 0x557b1f385480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.061+0000 7fbcf8a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b1efc44d0 0x557b1f391070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:35832/0 (socket says 172.21.15.16:35832) 2024-08-31T06:11:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.061+0000 7fbcf8a00640 1 -- 172.21.15.16:0/119864958 learned_addr learned my addr 172.21.15.16:0/119864958 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.061+0000 7fbcf8a00640 1 -- 172.21.15.16:0/119864958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b1efbc2f0 msgr2=0x557b1f385480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.061+0000 7fbcf8a00640 1 --2- 172.21.15.16:0/119864958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b1efbc2f0 0x557b1f385480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.061+0000 7fbcf8a00640 1 -- 172.21.15.16:0/119864958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1efe51b0 msgr2=0x557b1f3915b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.061+0000 7fbcf9e00640 1 --2- 172.21.15.16:0/119864958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1efe51b0 0x557b1f3915b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.061+0000 7fbcf8a00640 1 --2- 172.21.15.16:0/119864958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1efe51b0 0x557b1f3915b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.061+0000 7fbcf8a00640 1 -- 172.21.15.16:0/119864958 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b1f3a6500 con 0x557b1efc44d0 2024-08-31T06:11:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.062+0000 7fbcf8a00640 1 --2- 172.21.15.16:0/119864958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b1efc44d0 0x557b1f391070 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbce0009fd0 tx=0x7fbce00ab6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=809d88fc4aa4dc50 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.062+0000 7fbce9c00640 1 -- 172.21.15.16:0/119864958 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce00932c0 con 0x557b1efc44d0 2024-08-31T06:11:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.062+0000 7fbce9c00640 1 -- 172.21.15.16:0/119864958 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbce0009530 con 0x557b1efc44d0 2024-08-31T06:11:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.062+0000 7fbce9c00640 1 -- 172.21.15.16:0/119864958 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce00b4cb0 con 0x557b1efc44d0 2024-08-31T06:11:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.064+0000 7fbd02e1f740 1 -- 172.21.15.16:0/119864958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b1efc44d0 msgr2=0x557b1f391070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.064+0000 7fbd02e1f740 1 --2- 172.21.15.16:0/119864958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b1efc44d0 0x557b1f391070 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbce0009fd0 tx=0x7fbce00ab6c0 comp rx=0 tx=0).stop 2024-08-31T06:11:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.066+0000 7fbd02e1f740 1 -- 172.21.15.16:0/119864958 shutdown_connections 2024-08-31T06:11:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.066+0000 7fbd02e1f740 1 --2- 172.21.15.16:0/119864958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b1efc44d0 0x557b1f391070 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.066+0000 7fbd02e1f740 1 --2- 172.21.15.16:0/119864958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1efe51b0 0x557b1f3915b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.066+0000 7fbd02e1f740 1 --2- 172.21.15.16:0/119864958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b1efbc2f0 0x557b1f385480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.066+0000 7fbd02e1f740 1 -- 172.21.15.16:0/119864958 >> 172.21.15.16:0/119864958 conn(0x557b1efd9300 msgr2=0x557b1f384d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.066+0000 7fbd02e1f740 1 -- 172.21.15.16:0/119864958 shutdown_connections 2024-08-31T06:11:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.066+0000 7fbd02e1f740 1 -- 172.21.15.16:0/119864958 wait complete. 2024-08-31T06:11:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.067+0000 7fbd02e1f740 1 -- start start 2024-08-31T06:11:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.067+0000 7fbd02e1f740 1 Processor -- start 2024-08-31T06:11:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.069+0000 7fbd02e1f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1efbc2f0 0x557b1f39c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.069+0000 7fbd02e1f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b1efc44d0 0x557b1f399c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.069+0000 7fbd02e1f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b1efe51b0 0x557b1f39a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.069+0000 7fbd02e1f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf41b3f00 con 0x557b1efbc2f0 2024-08-31T06:11:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.069+0000 7fbd02e1f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf41b5370 con 0x557b1efe51b0 2024-08-31T06:11:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.069+0000 7fbd02e1f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf41b45d0 con 0x557b1efc44d0 2024-08-31T06:11:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.069+0000 7fbd02e1f740 10 client.0 _ll_register_callbacks cb 0x7fbce00abc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:11:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.070+0000 7fbcf8a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b1efc44d0 0x557b1f399c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.070+0000 7fbcf9400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1efbc2f0 0x557b1f39c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.070+0000 7fbcf9400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1efbc2f0 0x557b1f39c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47528/0 (socket says 172.21.15.16:47528) 2024-08-31T06:11:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.070+0000 7fbcf9e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b1efe51b0 0x557b1f39a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.070+0000 7fbcf9400640 1 -- 172.21.15.16:0/3598865185 learned_addr learned my addr 172.21.15.16:0/3598865185 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.071+0000 7fbcf8a00640 1 -- 172.21.15.16:0/3598865185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b1efe51b0 msgr2=0x557b1f39a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.071+0000 7fbcf8a00640 1 --2- 172.21.15.16:0/3598865185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b1efe51b0 0x557b1f39a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.071+0000 7fbcf8a00640 1 -- 172.21.15.16:0/3598865185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1efbc2f0 msgr2=0x557b1f39c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.071+0000 7fbcf8a00640 1 --2- 172.21.15.16:0/3598865185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1efbc2f0 0x557b1f39c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.071+0000 7fbcf8a00640 1 -- 172.21.15.16:0/3598865185 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b1f3a0210 con 0x557b1efc44d0 2024-08-31T06:11:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.071+0000 7fbcf8a00640 1 --2- 172.21.15.16:0/3598865185 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b1efc44d0 0x557b1f399c90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x557b1f275aa0 tx=0x7fbce00abd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25e2f3ca965bc232 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.071+0000 7fbcdea00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:11:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.071+0000 7fbcdea00640 1 -- 172.21.15.16:0/3598865185 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce00c5020 con 0x557b1efc44d0 2024-08-31T06:11:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.071+0000 7fbcdea00640 1 -- 172.21.15.16:0/3598865185 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fbce00b0360 con 0x557b1efc44d0 2024-08-31T06:11:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.072+0000 7fbd02e1f740 1 -- 172.21.15.16:0/3598865185 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557b1f2fb960 con 0x557b1efc44d0 2024-08-31T06:11:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.072+0000 7fbcdea00640 1 -- 172.21.15.16:0/3598865185 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbce00cd050 con 0x557b1efc44d0 2024-08-31T06:11:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.072+0000 7fbd02e1f740 10 client.26506 fetch_fsmap learned FSMap version 33 2024-08-31T06:11:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.072+0000 7fbd02e1f740 1 -- 172.21.15.16:0/3598865185 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557b1f410450 con 0x557b1efc44d0 2024-08-31T06:11:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.072+0000 7fbcdea00640 1 -- 172.21.15.16:0/3598865185 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557b1f2fb960 con 0x557b1efc44d0 2024-08-31T06:11:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.072+0000 7fbcdea00640 1 -- 172.21.15.16:0/3598865185 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7fbce00a7790 con 0x557b1efc44d0 2024-08-31T06:11:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.072+0000 7fbd02e1f740 10 client.26506 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T06:11:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.072+0000 7fbd02e1f740 10 client.26506 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:11:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.072+0000 7fbd02e1f740 20 client.26506 populate_metadata read hostname 'smithi016' 2024-08-31T06:11:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.072+0000 7fbd02e1f740 1 --2- 172.21.15.16:0/3598865185 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557b1ef791c0 0x557b1f4129c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.072+0000 7fbd02e1f740 4 client.26506 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:11:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.072+0000 7fbd02e1f740 1 -- 172.21.15.16:0/3598865185 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557b1f2f90a0 con 0x557b1ef791c0 2024-08-31T06:11:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.073+0000 7fbcf9400640 1 --2- 172.21.15.16:0/3598865185 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557b1ef791c0 0x557b1f4129c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.073+0000 7fbcf9400640 1 --2- 172.21.15.16:0/3598865185 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557b1ef791c0 0x557b1f4129c0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a788e0ab788b6d1d server_cookie=6e7530fe65f41e9d in_seq=0 out_seq=0 2024-08-31T06:11:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.073+0000 7fbcdea00640 10 client.26506 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:11:05.087 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.085+0000 7fbcdea00640 1 -- 172.21.15.16:0/3598865185 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557b1f2f90a0 con 0x557b1ef791c0 2024-08-31T06:11:05.087 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.085+0000 7fbcdea00640 10 client.26506 handle_command_reply: tid=0 2024-08-31T06:11:05.087 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.085+0000 7fbd02e1f740 1 client.26506 shutdown 2024-08-31T06:11:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.085+0000 7fbd02e1f740 1 -- 172.21.15.16:0/3598865185 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b1efc44d0 msgr2=0x557b1f399c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.085+0000 7fbd02e1f740 1 --2- 172.21.15.16:0/3598865185 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b1efc44d0 0x557b1f399c90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x557b1f275aa0 tx=0x7fbce00abd10 comp rx=0 tx=0).stop 2024-08-31T06:11:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.087+0000 7fbd02e1f740 1 -- 172.21.15.16:0/3598865185 shutdown_connections 2024-08-31T06:11:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.087+0000 7fbd02e1f740 1 --2- 172.21.15.16:0/3598865185 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557b1ef791c0 0x557b1f4129c0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.087+0000 7fbd02e1f740 1 --2- 172.21.15.16:0/3598865185 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557b1efc44d0 0x557b1f399c90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.087+0000 7fbd02e1f740 1 --2- 172.21.15.16:0/3598865185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557b1efe51b0 0x557b1f39a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.087+0000 7fbd02e1f740 1 --2- 172.21.15.16:0/3598865185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1efbc2f0 0x557b1f39c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.087+0000 7fbd02e1f740 1 -- 172.21.15.16:0/3598865185 >> 172.21.15.16:0/3598865185 conn(0x557b1efd9300 msgr2=0x557b1f386750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.087+0000 7fbcdea00640 0 client.26506 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:11:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.088+0000 7fbd02e1f740 1 -- 172.21.15.16:0/3598865185 shutdown_connections 2024-08-31T06:11:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.088+0000 7fbd02e1f740 1 -- 172.21.15.16:0/3598865185 wait complete. 2024-08-31T06:11:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.089+0000 7fbd02e1f740 20 client.26506 trim_cache size 0 max 16384 2024-08-31T06:11:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:11:05.092 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:11:05.092 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:11:05.092 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:11:05.092 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.090+0000 7fbcfa800640 1 -- 172.21.15.16:0/2296456629 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbcd80b4d20 msgr2=0x7fbcd80b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:05.092 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.090+0000 7fbcfa800640 1 --2- 172.21.15.16:0/2296456629 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbcd80b4d20 0x7fbcd80b7230 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbcec00a010 tx=0x7fbcec009870 comp rx=0 tx=0).stop 2024-08-31T06:11:05.092 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.090+0000 7fbcfa800640 1 -- 172.21.15.16:0/2296456629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcf4102aa0 msgr2=0x7fbcf419c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:05.092 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.090+0000 7fbcfa800640 1 --2- 172.21.15.16:0/2296456629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcf4102aa0 0x7fbcf419c860 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbce00058c0 tx=0x7fbce000ee50 comp rx=0 tx=0).stop 2024-08-31T06:11:05.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.090+0000 7fbcfa800640 1 -- 172.21.15.16:0/2296456629 shutdown_connections 2024-08-31T06:11:05.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.090+0000 7fbcfa800640 1 --2- 172.21.15.16:0/2296456629 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbcd80b4d20 0x7fbcd80b7230 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:05.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.090+0000 7fbcfa800640 1 --2- 172.21.15.16:0/2296456629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcf4102aa0 0x7fbcf419c860 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:05.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.090+0000 7fbcfa800640 1 --2- 172.21.15.16:0/2296456629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcf410fd00 0x7fbcf41a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:05.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.090+0000 7fbcfa800640 1 --2- 172.21.15.16:0/2296456629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcf4102150 0x7fbcf419c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:05.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.090+0000 7fbcfa800640 1 -- 172.21.15.16:0/2296456629 >> 172.21.15.16:0/2296456629 conn(0x7fbcf40fdfd0 msgr2=0x7fbcf4100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:05.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.091+0000 7fbcfa800640 1 -- 172.21.15.16:0/2296456629 shutdown_connections 2024-08-31T06:11:05.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:05.091+0000 7fbcfa800640 1 -- 172.21.15.16:0/2296456629 wait complete. 2024-08-31T06:11:06.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 220.0 s, 1985.0 tps, lat 253.618 ms stddev 2063.022 2024-08-31T06:11:06.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:06 smithi155 ceph-mon[21742]: pgmap v480: 97 pgs: 97 active+clean; 2.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 10 MiB/s wr, 1.04k op/s 2024-08-31T06:11:06.918 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:06 smithi016 ceph-mon[17194]: pgmap v480: 97 pgs: 97 active+clean; 2.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 10 MiB/s wr, 1.04k op/s 2024-08-31T06:11:06.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:06 smithi184 ceph-mon[22331]: pgmap v480: 97 pgs: 97 active+clean; 2.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 10 MiB/s wr, 1.04k op/s 2024-08-31T06:11:08.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:08 smithi016 ceph-mon[17194]: pgmap v481: 97 pgs: 97 active+clean; 2.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 11 MiB/s wr, 1.14k op/s 2024-08-31T06:11:08.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:08 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:11:08.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:08 smithi184 ceph-mon[22331]: pgmap v481: 97 pgs: 97 active+clean; 2.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 11 MiB/s wr, 1.14k op/s 2024-08-31T06:11:08.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:08 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:11:09.121 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:08 smithi155 ceph-mon[21742]: pgmap v481: 97 pgs: 97 active+clean; 2.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 11 MiB/s wr, 1.14k op/s 2024-08-31T06:11:09.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:08 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:11:09.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:09 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:11:09.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:09 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:11:09.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:09 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:11:09.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:09 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:11:10.023 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:09 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:11:10.023 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:09 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:11:10.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:10 smithi016 ceph-mon[17194]: pgmap v482: 97 pgs: 97 active+clean; 2.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 13 MiB/s wr, 1.31k op/s 2024-08-31T06:11:10.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:10 smithi184 ceph-mon[22331]: pgmap v482: 97 pgs: 97 active+clean; 2.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 13 MiB/s wr, 1.31k op/s 2024-08-31T06:11:11.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:10 smithi155 ceph-mon[21742]: pgmap v482: 97 pgs: 97 active+clean; 2.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 13 MiB/s wr, 1.31k op/s 2024-08-31T06:11:11.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 225.0 s, 2709.8 tps, lat 36.833 ms stddev 42.358 2024-08-31T06:11:12.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:12 smithi016 ceph-mon[17194]: pgmap v483: 97 pgs: 97 active+clean; 2.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 MiB/s wr, 1.38k op/s 2024-08-31T06:11:12.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:12 smithi184 ceph-mon[22331]: pgmap v483: 97 pgs: 97 active+clean; 2.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 MiB/s wr, 1.38k op/s 2024-08-31T06:11:13.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:12 smithi155 ceph-mon[21742]: pgmap v483: 97 pgs: 97 active+clean; 2.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 MiB/s wr, 1.38k op/s 2024-08-31T06:11:14.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:14 smithi016 ceph-mon[17194]: pgmap v484: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 16 MiB/s wr, 1.42k op/s 2024-08-31T06:11:14.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:14 smithi184 ceph-mon[22331]: pgmap v484: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 16 MiB/s wr, 1.42k op/s 2024-08-31T06:11:15.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:14 smithi155 ceph-mon[21742]: pgmap v484: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 16 MiB/s wr, 1.42k op/s 2024-08-31T06:11:16.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 230.0 s, 2301.8 tps, lat 43.400 ms stddev 50.914 2024-08-31T06:11:16.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:16 smithi016 ceph-mon[17194]: pgmap v485: 97 pgs: 97 active+clean; 2.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 MiB/s wr, 1.20k op/s 2024-08-31T06:11:16.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:16 smithi184 ceph-mon[22331]: pgmap v485: 97 pgs: 97 active+clean; 2.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 MiB/s wr, 1.20k op/s 2024-08-31T06:11:17.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:16 smithi155 ceph-mon[21742]: pgmap v485: 97 pgs: 97 active+clean; 2.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 MiB/s wr, 1.20k op/s 2024-08-31T06:11:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.441+0000 7f5b3fe00640 1 -- 172.21.15.16:0/3766974087 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b400728b0 msgr2=0x7f5b40072cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.441+0000 7f5b3fe00640 1 --2- 172.21.15.16:0/3766974087 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b400728b0 0x7f5b40072cc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c009900 tx=0x7f5b2c02a3f0 comp rx=0 tx=0).stop 2024-08-31T06:11:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.442+0000 7f5b3fe00640 1 -- 172.21.15.16:0/3766974087 shutdown_connections 2024-08-31T06:11:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.442+0000 7f5b3fe00640 1 --2- 172.21.15.16:0/3766974087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b40073200 0x7f5b40073a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.442+0000 7f5b3fe00640 1 --2- 172.21.15.16:0/3766974087 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b400728b0 0x7f5b40072cc0 secure :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c009900 tx=0x7f5b2c02a3f0 comp rx=0 tx=0).stop 2024-08-31T06:11:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.442+0000 7f5b3fe00640 1 --2- 172.21.15.16:0/3766974087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b4006cf80 0x7f5b4006d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.442+0000 7f5b3fe00640 1 -- 172.21.15.16:0/3766974087 >> 172.21.15.16:0/3766974087 conn(0x7f5b400fdfd0 msgr2=0x7f5b40100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.444+0000 7f5b3fe00640 1 -- 172.21.15.16:0/3766974087 shutdown_connections 2024-08-31T06:11:18.447 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.444+0000 7f5b3fe00640 1 -- 172.21.15.16:0/3766974087 wait complete. 2024-08-31T06:11:18.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.447+0000 7f5b3fe00640 1 Processor -- start 2024-08-31T06:11:18.450 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.448+0000 7f5b3fe00640 1 -- start start 2024-08-31T06:11:18.452 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.450+0000 7f5b3fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b4006cf80 0x7f5b401a2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.450+0000 7f5b3fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b40073200 0x7f5b401a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.450+0000 7f5b3fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b401a81b0 0x7f5b401a85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.450+0000 7f5b3fe00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4011ad30 con 0x7f5b40073200 2024-08-31T06:11:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.450+0000 7f5b3fe00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4011abb0 con 0x7f5b4006cf80 2024-08-31T06:11:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.450+0000 7f5b3fe00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4011aeb0 con 0x7f5b401a81b0 2024-08-31T06:11:18.454 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.450+0000 7f5b3ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b4006cf80 0x7f5b401a2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:18.454 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.451+0000 7f5b3ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b4006cf80 0x7f5b401a2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:56040/0 (socket says 172.21.15.16:56040) 2024-08-31T06:11:18.454 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.451+0000 7f5b3ea00640 1 -- 172.21.15.16:0/3094867887 learned_addr learned my addr 172.21.15.16:0/3094867887 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:18.454 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.451+0000 7f5b3e000640 1 --2- 172.21.15.16:0/3094867887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b40073200 0x7f5b401a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:18.454 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.451+0000 7f5b3f400640 1 --2- 172.21.15.16:0/3094867887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b401a81b0 0x7f5b401a85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.451+0000 7f5b3ea00640 1 -- 172.21.15.16:0/3094867887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b401a81b0 msgr2=0x7f5b401a85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.451+0000 7f5b3ea00640 1 --2- 172.21.15.16:0/3094867887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b401a81b0 0x7f5b401a85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.451+0000 7f5b3ea00640 1 -- 172.21.15.16:0/3094867887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b40073200 msgr2=0x7f5b401a2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.451+0000 7f5b3ea00640 1 --2- 172.21.15.16:0/3094867887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b40073200 0x7f5b401a2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.451+0000 7f5b3ea00640 1 -- 172.21.15.16:0/3094867887 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b401a8d90 con 0x7f5b4006cf80 2024-08-31T06:11:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.453+0000 7f5b3ea00640 1 --2- 172.21.15.16:0/3094867887 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b4006cf80 0x7f5b401a2a40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5b3000e9f0 tx=0x7f5b3000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.453+0000 7f5b44a00640 1 -- 172.21.15.16:0/3094867887 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3000cd80 con 0x7f5b4006cf80 2024-08-31T06:11:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.453+0000 7f5b44a00640 1 -- 172.21.15.16:0/3094867887 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b300108e0 con 0x7f5b4006cf80 2024-08-31T06:11:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.454+0000 7f5b44a00640 1 -- 172.21.15.16:0/3094867887 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b30018980 con 0x7f5b4006cf80 2024-08-31T06:11:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.454+0000 7f5b3fe00640 1 -- 172.21.15.16:0/3094867887 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b400fe3e0 con 0x7f5b4006cf80 2024-08-31T06:11:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.454+0000 7f5b3fe00640 1 -- 172.21.15.16:0/3094867887 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b400fe930 con 0x7f5b4006cf80 2024-08-31T06:11:18.470 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.467+0000 7f5b44a00640 1 -- 172.21.15.16:0/3094867887 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f5b30018b20 con 0x7f5b4006cf80 2024-08-31T06:11:18.470 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.468+0000 7f5b4ceb0740 1 Processor -- start 2024-08-31T06:11:18.470 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.468+0000 7f5b4ceb0740 1 -- start start 2024-08-31T06:11:18.472 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.470+0000 7f5b4ceb0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dbf0651b0 0x557dbf411310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:18.473 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.470+0000 7f5b4ceb0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dbf03c2f0 0x557dbf411850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:18.473 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.470+0000 7f5b4ceb0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbf0444d0 0x557dbf411d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:18.473 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.470+0000 7f5b4ceb0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557dbf37df70 con 0x557dbf0444d0 2024-08-31T06:11:18.473 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.470+0000 7f5b4ceb0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557dbf36d500 con 0x557dbf0651b0 2024-08-31T06:11:18.473 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.470+0000 7f5b4ceb0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557dbf3714b0 con 0x557dbf03c2f0 2024-08-31T06:11:18.473 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.470+0000 7f5b44a00640 1 --2- 172.21.15.16:0/3094867887 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f5b240b4d50 0x7f5b240b7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:18.474 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.470+0000 7f5b44a00640 1 -- 172.21.15.16:0/3094867887 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f5b300e58c0 con 0x7f5b4006cf80 2024-08-31T06:11:18.475 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.471+0000 7f5b3ea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dbf03c2f0 0x557dbf411850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:18.475 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.471+0000 7f5b3f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dbf0651b0 0x557dbf411310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:18.475 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.471+0000 7f5b3f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dbf0651b0 0x557dbf411310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:56050/0 (socket says 172.21.15.16:56050) 2024-08-31T06:11:18.475 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.471+0000 7f5b3ea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dbf03c2f0 0x557dbf411850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:42806/0 (socket says 172.21.15.16:42806) 2024-08-31T06:11:18.475 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.471+0000 7f5b3f400640 1 -- 172.21.15.16:0/1240034059 learned_addr learned my addr 172.21.15.16:0/1240034059 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:18.475 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.472+0000 7f5b3f400640 1 --2- 172.21.15.16:0/3094867887 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f5b240b4d50 0x7f5b240b7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:18.475 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.472+0000 7f5b3f400640 1 -- 172.21.15.16:0/1240034059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dbf03c2f0 msgr2=0x557dbf411850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:18.475 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.472+0000 7f5b3f400640 1 --2- 172.21.15.16:0/1240034059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dbf03c2f0 0x557dbf411850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.472+0000 7f5b3f400640 1 -- 172.21.15.16:0/1240034059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbf0444d0 msgr2=0x557dbf411d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.472+0000 7f5b3f400640 1 --2- 172.21.15.16:0/1240034059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbf0444d0 0x557dbf411d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.472+0000 7f5b3f400640 1 -- 172.21.15.16:0/1240034059 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dbf327c10 con 0x557dbf0651b0 2024-08-31T06:11:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.473+0000 7f5b3f400640 1 --2- 172.21.15.16:0/1240034059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dbf0651b0 0x557dbf411310 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5b3800b9e0 tx=0x7f5b3800bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82ee3aa4c0e172e5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.474+0000 7f5b2be00640 1 -- 172.21.15.16:0/1240034059 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b38019410 con 0x557dbf0651b0 2024-08-31T06:11:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.474+0000 7f5b2be00640 1 -- 172.21.15.16:0/1240034059 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b38019a60 con 0x557dbf0651b0 2024-08-31T06:11:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.474+0000 7f5b2be00640 1 -- 172.21.15.16:0/1240034059 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b38019410 con 0x557dbf0651b0 2024-08-31T06:11:18.480 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.476+0000 7f5b3f400640 1 --2- 172.21.15.16:0/3094867887 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f5b240b4d50 0x7f5b240b7260 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5b3801f000 tx=0x7f5b38018e10 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:18.480 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.477+0000 7f5b4ceb0740 1 -- 172.21.15.16:0/1240034059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dbf0651b0 msgr2=0x557dbf411310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:18.480 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.477+0000 7f5b4ceb0740 1 --2- 172.21.15.16:0/1240034059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dbf0651b0 0x557dbf411310 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5b3800b9e0 tx=0x7f5b3800bea0 comp rx=0 tx=0).stop 2024-08-31T06:11:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.478+0000 7f5b4ceb0740 1 -- 172.21.15.16:0/1240034059 shutdown_connections 2024-08-31T06:11:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.478+0000 7f5b4ceb0740 1 --2- 172.21.15.16:0/1240034059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dbf03c2f0 0x557dbf411850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.478+0000 7f5b4ceb0740 1 --2- 172.21.15.16:0/1240034059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbf0444d0 0x557dbf411d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.478+0000 7f5b4ceb0740 1 --2- 172.21.15.16:0/1240034059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dbf0651b0 0x557dbf411310 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.479+0000 7f5b4ceb0740 1 -- 172.21.15.16:0/1240034059 >> 172.21.15.16:0/1240034059 conn(0x557dbf059300 msgr2=0x557dbf40f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.479+0000 7f5b4ceb0740 1 -- 172.21.15.16:0/1240034059 shutdown_connections 2024-08-31T06:11:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.479+0000 7f5b4ceb0740 1 -- 172.21.15.16:0/1240034059 wait complete. 2024-08-31T06:11:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.480+0000 7f5b4ceb0740 1 -- start start 2024-08-31T06:11:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.480+0000 7f5b4ceb0740 1 Processor -- start 2024-08-31T06:11:18.484 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.482+0000 7f5b4ceb0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dbf03c2f0 0x557dbf41c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:18.484 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.482+0000 7f5b4ceb0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dbf0444d0 0x557dbf41ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:18.485 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.482+0000 7f5b3ea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dbf03c2f0 0x557dbf41c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.482+0000 7f5b3ea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dbf03c2f0 0x557dbf41c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:42808/0 (socket says 172.21.15.16:42808) 2024-08-31T06:11:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.482+0000 7f5b4ceb0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbf0651b0 0x557dbf41d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.482+0000 7f5b4ceb0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557dbf2a6ab0 con 0x557dbf0651b0 2024-08-31T06:11:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.482+0000 7f5b4ceb0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557dbf314320 con 0x557dbf0444d0 2024-08-31T06:11:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.482+0000 7f5b4ceb0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557dbf41fa90 con 0x557dbf03c2f0 2024-08-31T06:11:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.482+0000 7f5b3ea00640 1 -- 172.21.15.16:0/225561662 learned_addr learned my addr 172.21.15.16:0/225561662 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.483+0000 7f5b4ceb0740 10 client.0 _ll_register_callbacks cb 0x557dbf31a6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:11:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.483+0000 7f5b3f400640 1 --2- 172.21.15.16:0/225561662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbf0651b0 0x557dbf41d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.484+0000 7f5b3e000640 1 --2- 172.21.15.16:0/225561662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dbf0444d0 0x557dbf41ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:18.487 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.485+0000 7f5b3ea00640 1 -- 172.21.15.16:0/225561662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dbf0444d0 msgr2=0x557dbf41ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.485+0000 7f5b3ea00640 1 --2- 172.21.15.16:0/225561662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dbf0444d0 0x557dbf41ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.485+0000 7f5b3ea00640 1 -- 172.21.15.16:0/225561662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbf0651b0 msgr2=0x557dbf41d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.485+0000 7f5b3ea00640 1 --2- 172.21.15.16:0/225561662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbf0651b0 0x557dbf41d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.485+0000 7f5b3ea00640 1 -- 172.21.15.16:0/225561662 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dbf0d0790 con 0x557dbf03c2f0 2024-08-31T06:11:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.486+0000 7f5b3ea00640 1 --2- 172.21.15.16:0/225561662 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dbf03c2f0 0x557dbf41c990 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5b3000e9c0 tx=0x7f5b300abbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=854fb6e1d3a295f8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.486+0000 7f5b28c00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:11:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.486+0000 7f5b28c00640 1 -- 172.21.15.16:0/225561662 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b300b0880 con 0x557dbf03c2f0 2024-08-31T06:11:18.489 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.486+0000 7f5b4ceb0740 1 -- 172.21.15.16:0/225561662 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b38019a60 con 0x557dbf03c2f0 2024-08-31T06:11:18.489 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.487+0000 7f5b28c00640 1 -- 172.21.15.16:0/225561662 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f5b300a8600 con 0x557dbf03c2f0 2024-08-31T06:11:18.492 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.490+0000 7f5b28c00640 1 -- 172.21.15.16:0/225561662 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5b300b5330 con 0x557dbf03c2f0 2024-08-31T06:11:18.493 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.490+0000 7f5b4ceb0740 10 client.26518 fetch_fsmap learned FSMap version 33 2024-08-31T06:11:18.493 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.490+0000 7f5b4ceb0740 1 -- 172.21.15.16:0/225561662 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557dbf149620 con 0x557dbf03c2f0 2024-08-31T06:11:18.493 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.491+0000 7f5b28c00640 1 -- 172.21.15.16:0/225561662 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b38019a60 con 0x557dbf03c2f0 2024-08-31T06:11:18.494 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.491+0000 7f5b28c00640 1 -- 172.21.15.16:0/225561662 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f5b300bab70 con 0x557dbf03c2f0 2024-08-31T06:11:18.494 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.491+0000 7f5b4ceb0740 10 client.26518 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T06:11:18.494 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.492+0000 7f5b4ceb0740 10 client.26518 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:11:18.494 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.492+0000 7f5b4ceb0740 20 client.26518 populate_metadata read hostname 'smithi016' 2024-08-31T06:11:18.494 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.492+0000 7f5b4ceb0740 1 --2- 172.21.15.16:0/225561662 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557dbf023d80 0x557dbf42ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:18.494 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.492+0000 7f5b4ceb0740 4 client.26518 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:11:18.494 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.492+0000 7f5b4ceb0740 1 -- 172.21.15.16:0/225561662 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557dbf37fec0 con 0x557dbf023d80 2024-08-31T06:11:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.492+0000 7f5b3e000640 1 --2- 172.21.15.16:0/225561662 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557dbf023d80 0x557dbf42ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.493+0000 7f5b3e000640 1 --2- 172.21.15.16:0/225561662 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557dbf023d80 0x557dbf42ea80 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b881e1c3cc1b80b server_cookie=99709b4a7a066bf8 in_seq=0 out_seq=0 2024-08-31T06:11:18.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.494+0000 7f5b28c00640 10 client.26518 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:11:18.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.494+0000 7f5b28c00640 1 -- 172.21.15.16:0/225561662 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557dbf37fec0 con 0x557dbf023d80 2024-08-31T06:11:18.497 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.494+0000 7f5b28c00640 10 client.26518 handle_command_reply: tid=0 2024-08-31T06:11:18.497 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.494+0000 7f5b4ceb0740 1 client.26518 shutdown 2024-08-31T06:11:18.497 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.494+0000 7f5b4ceb0740 1 -- 172.21.15.16:0/225561662 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dbf03c2f0 msgr2=0x557dbf41c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:18.497 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.494+0000 7f5b4ceb0740 1 --2- 172.21.15.16:0/225561662 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dbf03c2f0 0x557dbf41c990 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5b3000e9c0 tx=0x7f5b300abbe0 comp rx=0 tx=0).stop 2024-08-31T06:11:18.497 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.495+0000 7f5b4ceb0740 1 -- 172.21.15.16:0/225561662 shutdown_connections 2024-08-31T06:11:18.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.495+0000 7f5b4ceb0740 1 --2- 172.21.15.16:0/225561662 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557dbf023d80 0x557dbf42ea80 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:18.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.495+0000 7f5b4ceb0740 1 --2- 172.21.15.16:0/225561662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbf0651b0 0x557dbf41d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:18.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.495+0000 7f5b4ceb0740 1 --2- 172.21.15.16:0/225561662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dbf0444d0 0x557dbf41ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:18.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.495+0000 7f5b4ceb0740 1 --2- 172.21.15.16:0/225561662 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dbf03c2f0 0x557dbf41c990 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:18.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.495+0000 7f5b4ceb0740 1 -- 172.21.15.16:0/225561662 >> 172.21.15.16:0/225561662 conn(0x557dbf059300 msgr2=0x557dbf412340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:18.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.495+0000 7f5b28c00640 0 client.26518 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:11:18.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.496+0000 7f5b4ceb0740 1 -- 172.21.15.16:0/225561662 shutdown_connections 2024-08-31T06:11:18.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.496+0000 7f5b4ceb0740 1 -- 172.21.15.16:0/225561662 wait complete. 2024-08-31T06:11:18.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.496+0000 7f5b4ceb0740 20 client.26518 trim_cache size 0 max 16384 2024-08-31T06:11:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.529+0000 7f5b4ceb0740 1 Processor -- start 2024-08-31T06:11:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.530+0000 7f5b4ceb0740 1 -- start start 2024-08-31T06:11:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.530+0000 7f5b4ceb0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbf03c2f0 0x557dbf40f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.530+0000 7f5b4ceb0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dbf0444d0 0x557dbf434210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.531+0000 7f5b4ceb0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dbf0651b0 0x557dbf434750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.531+0000 7f5b4ceb0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5b401b8860 con 0x557dbf03c2f0 2024-08-31T06:11:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.531+0000 7f5b4ceb0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5b401b60d0 con 0x557dbf0444d0 2024-08-31T06:11:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.531+0000 7f5b4ceb0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5b401b3f20 con 0x557dbf0651b0 2024-08-31T06:11:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.531+0000 7f5b3f400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dbf0651b0 0x557dbf434750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.532+0000 7f5b3f400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dbf0651b0 0x557dbf434750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:42810/0 (socket says 172.21.15.16:42810) 2024-08-31T06:11:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.532+0000 7f5b3f400640 1 -- 172.21.15.16:0/3406485795 learned_addr learned my addr 172.21.15.16:0/3406485795 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.532+0000 7f5b3ea00640 1 --2- 172.21.15.16:0/3406485795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbf03c2f0 0x557dbf40f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.532+0000 7f5b3e000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dbf0444d0 0x557dbf434210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.533+0000 7f5b3f400640 1 -- 172.21.15.16:0/3406485795 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dbf0444d0 msgr2=0x557dbf434210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.533+0000 7f5b3f400640 1 --2- 172.21.15.16:0/3406485795 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dbf0444d0 0x557dbf434210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.533+0000 7f5b3f400640 1 -- 172.21.15.16:0/3406485795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbf03c2f0 msgr2=0x557dbf40f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.533+0000 7f5b3f400640 1 --2- 172.21.15.16:0/3406485795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbf03c2f0 0x557dbf40f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.533+0000 7f5b3f400640 1 -- 172.21.15.16:0/3406485795 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dbf428c80 con 0x557dbf0651b0 2024-08-31T06:11:18.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.534+0000 7f5b3f400640 1 --2- 172.21.15.16:0/3406485795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dbf0651b0 0x557dbf434750 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5b38008b10 tx=0x7f5b38012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4ac0303df4703bc server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:18.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.535+0000 7f5b2b400640 1 -- 172.21.15.16:0/3406485795 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b38018370 con 0x557dbf0651b0 2024-08-31T06:11:18.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.535+0000 7f5b2b400640 1 -- 172.21.15.16:0/3406485795 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b38021920 con 0x557dbf0651b0 2024-08-31T06:11:18.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.537+0000 7f5b4ceb0740 1 -- 172.21.15.16:0/3406485795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dbf0651b0 msgr2=0x557dbf434750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:18.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.537+0000 7f5b4ceb0740 1 --2- 172.21.15.16:0/3406485795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dbf0651b0 0x557dbf434750 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5b38008b10 tx=0x7f5b38012000 comp rx=0 tx=0).stop 2024-08-31T06:11:18.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.537+0000 7f5b4ceb0740 1 -- 172.21.15.16:0/3406485795 shutdown_connections 2024-08-31T06:11:18.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.537+0000 7f5b4ceb0740 1 --2- 172.21.15.16:0/3406485795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dbf0651b0 0x557dbf434750 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:18.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.537+0000 7f5b4ceb0740 1 --2- 172.21.15.16:0/3406485795 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dbf0444d0 0x557dbf434210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:18.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.537+0000 7f5b4ceb0740 1 --2- 172.21.15.16:0/3406485795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbf03c2f0 0x557dbf40f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:18.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.537+0000 7f5b4ceb0740 1 -- 172.21.15.16:0/3406485795 >> 172.21.15.16:0/3406485795 conn(0x557dbf059300 msgr2=0x557dbf40ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:18.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.537+0000 7f5b4ceb0740 1 -- 172.21.15.16:0/3406485795 shutdown_connections 2024-08-31T06:11:18.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.538+0000 7f5b4ceb0740 1 -- 172.21.15.16:0/3406485795 wait complete. 2024-08-31T06:11:18.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.538+0000 7f5b4ceb0740 1 -- start start 2024-08-31T06:11:18.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.539+0000 7f5b4ceb0740 1 Processor -- start 2024-08-31T06:11:18.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.541+0000 7f5b4ceb0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dbf03c2f0 0x557dbf425400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:18.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.541+0000 7f5b4ceb0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbf0444d0 0x557dbf4208a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.541+0000 7f5b4ceb0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dbf0651b0 0x557dbf420de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.541+0000 7f5b4ceb0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5b401b4590 con 0x557dbf0444d0 2024-08-31T06:11:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.541+0000 7f5b4ceb0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5b401b5a00 con 0x557dbf0651b0 2024-08-31T06:11:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.541+0000 7f5b4ceb0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5b401b4c60 con 0x557dbf03c2f0 2024-08-31T06:11:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.542+0000 7f5b4ceb0740 10 client.0 _ll_register_callbacks cb 0x7f5b300ab780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:11:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.542+0000 7f5b3ea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dbf03c2f0 0x557dbf425400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.542+0000 7f5b3f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dbf0651b0 0x557dbf420de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.543+0000 7f5b3f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dbf0651b0 0x557dbf420de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:56082/0 (socket says 172.21.15.16:56082) 2024-08-31T06:11:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.543+0000 7f5b3f400640 1 -- 172.21.15.16:0/847271760 learned_addr learned my addr 172.21.15.16:0/847271760 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.543+0000 7f5b3e000640 1 --2- 172.21.15.16:0/847271760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbf0444d0 0x557dbf4208a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.544+0000 7f5b3f400640 1 -- 172.21.15.16:0/847271760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dbf03c2f0 msgr2=0x557dbf425400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.544+0000 7f5b3f400640 1 --2- 172.21.15.16:0/847271760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dbf03c2f0 0x557dbf425400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.544+0000 7f5b3f400640 1 -- 172.21.15.16:0/847271760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbf0444d0 msgr2=0x557dbf4208a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:18.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.544+0000 7f5b3f400640 1 --2- 172.21.15.16:0/847271760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbf0444d0 0x557dbf4208a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:18.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.544+0000 7f5b3f400640 1 -- 172.21.15.16:0/847271760 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dbf4214a0 con 0x557dbf0651b0 2024-08-31T06:11:18.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.544+0000 7f5b3e000640 1 --2- 172.21.15.16:0/847271760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbf0444d0 0x557dbf4208a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:11:18.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.545+0000 7f5b3f400640 1 --2- 172.21.15.16:0/847271760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dbf0651b0 0x557dbf420de0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5b38004b90 tx=0x7f5b38008790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c3ac4841b8daef2 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.545+0000 7f5b28c00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:11:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.546+0000 7f5b28c00640 1 -- 172.21.15.16:0/847271760 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b38021cd0 con 0x557dbf0651b0 2024-08-31T06:11:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.546+0000 7f5b28c00640 1 -- 172.21.15.16:0/847271760 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f5b38021e70 con 0x557dbf0651b0 2024-08-31T06:11:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.546+0000 7f5b4ceb0740 1 -- 172.21.15.16:0/847271760 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b38021920 con 0x557dbf0651b0 2024-08-31T06:11:18.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.546+0000 7f5b28c00640 1 -- 172.21.15.16:0/847271760 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5b38019410 con 0x557dbf0651b0 2024-08-31T06:11:18.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.547+0000 7f5b4ceb0740 10 client.26528 fetch_fsmap learned FSMap version 33 2024-08-31T06:11:18.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.547+0000 7f5b4ceb0740 1 -- 172.21.15.16:0/847271760 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557dbf499290 con 0x557dbf0651b0 2024-08-31T06:11:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.548+0000 7f5b28c00640 1 -- 172.21.15.16:0/847271760 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b38021920 con 0x557dbf0651b0 2024-08-31T06:11:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.549+0000 7f5b28c00640 1 -- 172.21.15.16:0/847271760 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f5b38029070 con 0x557dbf0651b0 2024-08-31T06:11:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.549+0000 7f5b4ceb0740 10 client.26528 fetch_fsmap finished waiting for FSMap version 33 2024-08-31T06:11:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.549+0000 7f5b4ceb0740 10 client.26528 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:11:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.549+0000 7f5b4ceb0740 20 client.26528 populate_metadata read hostname 'smithi016' 2024-08-31T06:11:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.550+0000 7f5b4ceb0740 1 --2- 172.21.15.16:0/847271760 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557dbeff91c0 0x557dbf49b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.550+0000 7f5b4ceb0740 4 client.26528 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:11:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.550+0000 7f5b4ceb0740 1 -- 172.21.15.16:0/847271760 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557dbf37b960 con 0x557dbeff91c0 2024-08-31T06:11:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.550+0000 7f5b3ea00640 1 --2- 172.21.15.16:0/847271760 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557dbeff91c0 0x557dbf49b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.552+0000 7f5b28c00640 10 client.26528 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:11:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.552+0000 7f5b3ea00640 1 --2- 172.21.15.16:0/847271760 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557dbeff91c0 0x557dbf49b930 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a9478816895d952 server_cookie=50e63eb734a71f7a in_seq=0 out_seq=0 2024-08-31T06:11:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.572+0000 7f5b28c00640 1 -- 172.21.15.16:0/847271760 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557dbf37b960 con 0x557dbeff91c0 2024-08-31T06:11:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.572+0000 7f5b28c00640 10 client.26528 handle_command_reply: tid=0 2024-08-31T06:11:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.574+0000 7f5b4ceb0740 1 client.26528 shutdown 2024-08-31T06:11:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.574+0000 7f5b4ceb0740 1 -- 172.21.15.16:0/847271760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dbf0651b0 msgr2=0x557dbf420de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.574+0000 7f5b4ceb0740 1 --2- 172.21.15.16:0/847271760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dbf0651b0 0x557dbf420de0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5b38004b90 tx=0x7f5b38008790 comp rx=0 tx=0).stop 2024-08-31T06:11:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.575+0000 7f5b4ceb0740 1 -- 172.21.15.16:0/847271760 shutdown_connections 2024-08-31T06:11:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.575+0000 7f5b4ceb0740 1 --2- 172.21.15.16:0/847271760 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557dbeff91c0 0x557dbf49b930 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.575+0000 7f5b4ceb0740 1 --2- 172.21.15.16:0/847271760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dbf0651b0 0x557dbf420de0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.575+0000 7f5b4ceb0740 1 --2- 172.21.15.16:0/847271760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbf0444d0 0x557dbf4208a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.575+0000 7f5b4ceb0740 1 --2- 172.21.15.16:0/847271760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557dbf03c2f0 0x557dbf425400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.575+0000 7f5b4ceb0740 1 -- 172.21.15.16:0/847271760 >> 172.21.15.16:0/847271760 conn(0x557dbf059300 msgr2=0x557dbf41c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.575+0000 7f5b28c00640 0 client.26528 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:11:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.575+0000 7f5b4ceb0740 1 -- 172.21.15.16:0/847271760 shutdown_connections 2024-08-31T06:11:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.575+0000 7f5b4ceb0740 1 -- 172.21.15.16:0/847271760 wait complete. 2024-08-31T06:11:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.575+0000 7f5b4ceb0740 20 client.26528 trim_cache size 0 max 16384 2024-08-31T06:11:18.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:11:18.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:11:18.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:11:18.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:11:18.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.577+0000 7f5b3fe00640 1 -- 172.21.15.16:0/3094867887 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f5b240b4d50 msgr2=0x7f5b240b7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:18.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.577+0000 7f5b3fe00640 1 --2- 172.21.15.16:0/3094867887 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f5b240b4d50 0x7f5b240b7260 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5b3801f000 tx=0x7f5b38018e10 comp rx=0 tx=0).stop 2024-08-31T06:11:18.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.577+0000 7f5b3fe00640 1 -- 172.21.15.16:0/3094867887 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b4006cf80 msgr2=0x7f5b401a2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:18.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.577+0000 7f5b3fe00640 1 --2- 172.21.15.16:0/3094867887 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b4006cf80 0x7f5b401a2a40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5b3000e9f0 tx=0x7f5b3000eeb0 comp rx=0 tx=0).stop 2024-08-31T06:11:18.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.578+0000 7f5b3fe00640 1 -- 172.21.15.16:0/3094867887 shutdown_connections 2024-08-31T06:11:18.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.578+0000 7f5b3fe00640 1 --2- 172.21.15.16:0/3094867887 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f5b240b4d50 0x7f5b240b7260 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:18.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.578+0000 7f5b3fe00640 1 --2- 172.21.15.16:0/3094867887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b401a81b0 0x7f5b401a85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:18.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.578+0000 7f5b3fe00640 1 --2- 172.21.15.16:0/3094867887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b40073200 0x7f5b401a2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:18.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.578+0000 7f5b3fe00640 1 --2- 172.21.15.16:0/3094867887 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b4006cf80 0x7f5b401a2a40 secure :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5b3000e9f0 tx=0x7f5b3000eeb0 comp rx=0 tx=0).stop 2024-08-31T06:11:18.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.578+0000 7f5b3fe00640 1 -- 172.21.15.16:0/3094867887 >> 172.21.15.16:0/3094867887 conn(0x7f5b400fdfd0 msgr2=0x7f5b4006a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:18.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.579+0000 7f5b3fe00640 1 -- 172.21.15.16:0/3094867887 shutdown_connections 2024-08-31T06:11:18.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:18.579+0000 7f5b3fe00640 1 -- 172.21.15.16:0/3094867887 wait complete. 2024-08-31T06:11:18.915 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:18 smithi016 ceph-mon[17194]: pgmap v486: 97 pgs: 97 active+clean; 2.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.6 MiB/s wr, 769 op/s 2024-08-31T06:11:18.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:18 smithi184 ceph-mon[22331]: pgmap v486: 97 pgs: 97 active+clean; 2.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.6 MiB/s wr, 769 op/s 2024-08-31T06:11:19.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:18 smithi155 ceph-mon[21742]: pgmap v486: 97 pgs: 97 active+clean; 2.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.6 MiB/s wr, 769 op/s 2024-08-31T06:11:20.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:20 smithi184 ceph-mon[22331]: pgmap v487: 97 pgs: 97 active+clean; 2.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 1.24k op/s 2024-08-31T06:11:21.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:20 smithi155 ceph-mon[21742]: pgmap v487: 97 pgs: 97 active+clean; 2.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 1.24k op/s 2024-08-31T06:11:21.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:20 smithi016 ceph-mon[17194]: pgmap v487: 97 pgs: 97 active+clean; 2.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 1.24k op/s 2024-08-31T06:11:21.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 235.0 s, 2377.4 tps, lat 42.032 ms stddev 88.462 2024-08-31T06:11:22.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:22 smithi184 ceph-mon[22331]: pgmap v488: 97 pgs: 97 active+clean; 2.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 1.39k op/s 2024-08-31T06:11:23.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:22 smithi155 ceph-mon[21742]: pgmap v488: 97 pgs: 97 active+clean; 2.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 1.39k op/s 2024-08-31T06:11:23.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:22 smithi016 ceph-mon[17194]: pgmap v488: 97 pgs: 97 active+clean; 2.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 1.39k op/s 2024-08-31T06:11:24.966 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:24 smithi016 ceph-mon[17194]: pgmap v489: 97 pgs: 97 active+clean; 2.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 MiB/s wr, 1.65k op/s 2024-08-31T06:11:24.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:24 smithi184 ceph-mon[22331]: pgmap v489: 97 pgs: 97 active+clean; 2.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 MiB/s wr, 1.65k op/s 2024-08-31T06:11:25.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:24 smithi155 ceph-mon[21742]: pgmap v489: 97 pgs: 97 active+clean; 2.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 MiB/s wr, 1.65k op/s 2024-08-31T06:11:26.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 240.0 s, 2315.2 tps, lat 43.239 ms stddev 50.208 2024-08-31T06:11:26.952 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T06:11:26.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:26 smithi184 ceph-mon[22331]: pgmap v490: 97 pgs: 97 active+clean; 2.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 MiB/s wr, 1.74k op/s 2024-08-31T06:11:27.031 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:26 smithi016 ceph-mon[17194]: pgmap v490: 97 pgs: 97 active+clean; 2.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 MiB/s wr, 1.74k op/s 2024-08-31T06:11:27.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:26 smithi155 ceph-mon[21742]: pgmap v490: 97 pgs: 97 active+clean; 2.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 MiB/s wr, 1.74k op/s 2024-08-31T06:11:27.602 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:11:28.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:28 smithi184 ceph-mon[22331]: pgmap v491: 97 pgs: 97 active+clean; 2.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 MiB/s wr, 1.62k op/s 2024-08-31T06:11:29.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:28 smithi155 ceph-mon[21742]: pgmap v491: 97 pgs: 97 active+clean; 2.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 MiB/s wr, 1.62k op/s 2024-08-31T06:11:29.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:28 smithi016 ceph-mon[17194]: pgmap v491: 97 pgs: 97 active+clean; 2.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 MiB/s wr, 1.62k op/s 2024-08-31T06:11:29.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.213+0000 7fc811400640 1 -- 172.21.15.16:0/3809497323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc80c108a90 msgr2=0x7fc80c10f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:29.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.213+0000 7fc811400640 1 --2- 172.21.15.16:0/3809497323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc80c108a90 0x7fc80c10f2c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc00bd70 tx=0x7fc7fc02c440 comp rx=0 tx=0).stop 2024-08-31T06:11:29.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.213+0000 7fc811400640 1 -- 172.21.15.16:0/3809497323 shutdown_connections 2024-08-31T06:11:29.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.213+0000 7fc811400640 1 --2- 172.21.15.16:0/3809497323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc80c108a90 0x7fc80c10f2c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:29.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.213+0000 7fc811400640 1 --2- 172.21.15.16:0/3809497323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc80c108140 0x7fc80c108550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:29.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.213+0000 7fc811400640 1 --2- 172.21.15.16:0/3809497323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc80c102800 0x7fc80c102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:29.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.214+0000 7fc811400640 1 -- 172.21.15.16:0/3809497323 >> 172.21.15.16:0/3809497323 conn(0x7fc80c0794d0 msgr2=0x7fc80c0798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:29.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.214+0000 7fc811400640 1 -- 172.21.15.16:0/3809497323 shutdown_connections 2024-08-31T06:11:29.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.214+0000 7fc811400640 1 -- 172.21.15.16:0/3809497323 wait complete. 2024-08-31T06:11:29.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.216+0000 7fc811400640 1 Processor -- start 2024-08-31T06:11:29.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.216+0000 7fc811400640 1 -- start start 2024-08-31T06:11:29.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.216+0000 7fc811400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc80c102800 0x7fc80c074c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:29.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.216+0000 7fc811400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc80c108140 0x7fc80c06df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:29.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.216+0000 7fc811400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc80c108a90 0x7fc80c06e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:29.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.216+0000 7fc811400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c115a10 con 0x7fc80c102800 2024-08-31T06:11:29.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.216+0000 7fc811400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c115890 con 0x7fc80c108a90 2024-08-31T06:11:29.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.216+0000 7fc811400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c115b90 con 0x7fc80c108140 2024-08-31T06:11:29.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.217+0000 7fc80b200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc80c108a90 0x7fc80c06e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:29.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.217+0000 7fc80b200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc80c108a90 0x7fc80c06e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:38446/0 (socket says 172.21.15.16:38446) 2024-08-31T06:11:29.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.217+0000 7fc80b200640 1 -- 172.21.15.16:0/3940359813 learned_addr learned my addr 172.21.15.16:0/3940359813 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:29.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.217+0000 7fc809e00640 1 --2- 172.21.15.16:0/3940359813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc80c108140 0x7fc80c06df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:29.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.217+0000 7fc80b200640 1 -- 172.21.15.16:0/3940359813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc80c108140 msgr2=0x7fc80c06df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:29.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.217+0000 7fc80b200640 1 --2- 172.21.15.16:0/3940359813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc80c108140 0x7fc80c06df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:29.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.217+0000 7fc80b200640 1 -- 172.21.15.16:0/3940359813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc80c102800 msgr2=0x7fc80c074c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-31T06:11:29.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.218+0000 7fc80a800640 1 --2- 172.21.15.16:0/3940359813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc80c102800 0x7fc80c074c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:29.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.219+0000 7fc80b200640 1 --2- 172.21.15.16:0/3940359813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc80c102800 0x7fc80c074c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:29.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.219+0000 7fc80b200640 1 -- 172.21.15.16:0/3940359813 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc80c06ec60 con 0x7fc80c108a90 2024-08-31T06:11:29.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.219+0000 7fc80a800640 1 --2- 172.21.15.16:0/3940359813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc80c102800 0x7fc80c074c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:11:29.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.219+0000 7fc80b200640 1 --2- 172.21.15.16:0/3940359813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc80c108a90 0x7fc80c06e4d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc02c9b0 tx=0x7fc7fc003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:29.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.219+0000 7fc803400640 1 -- 172.21.15.16:0/3940359813 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc040020 con 0x7fc80c108a90 2024-08-31T06:11:29.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.219+0000 7fc803400640 1 -- 172.21.15.16:0/3940359813 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7fc0044f0 con 0x7fc80c108a90 2024-08-31T06:11:29.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.219+0000 7fc803400640 1 -- 172.21.15.16:0/3940359813 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc033e60 con 0x7fc80c108a90 2024-08-31T06:11:29.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.220+0000 7fc811400640 1 -- 172.21.15.16:0/3940359813 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc80c0738b0 con 0x7fc80c108a90 2024-08-31T06:11:29.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.220+0000 7fc811400640 1 -- 172.21.15.16:0/3940359813 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc80c073d10 con 0x7fc80c108a90 2024-08-31T06:11:29.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.224+0000 7fc803400640 1 -- 172.21.15.16:0/3940359813 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fc7fc043020 con 0x7fc80c108a90 2024-08-31T06:11:29.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.225+0000 7fc803400640 1 --2- 172.21.15.16:0/3940359813 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc7f00b4d50 0x7fc7f00b7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:29.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.226+0000 7fc803400640 1 -- 172.21.15.16:0/3940359813 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fc7fc105d50 con 0x7fc80c108a90 2024-08-31T06:11:29.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.227+0000 7fc80a800640 1 --2- 172.21.15.16:0/3940359813 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc7f00b4d50 0x7fc7f00b7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:29.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.230+0000 7fc818043740 1 Processor -- start 2024-08-31T06:11:29.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.231+0000 7fc818043740 1 -- start start 2024-08-31T06:11:29.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.231+0000 7fc80a800640 1 --2- 172.21.15.16:0/3940359813 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc7f00b4d50 0x7fc7f00b7260 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc80400baa0 tx=0x7fc804005d90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:29.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.231+0000 7fc818043740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc67b99070 0x55fc67f3e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:29.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.232+0000 7fc80a800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc67b99070 0x55fc67f3e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:29.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.232+0000 7fc80a800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc67b99070 0x55fc67f3e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:38448/0 (socket says 172.21.15.16:38448) 2024-08-31T06:11:29.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.232+0000 7fc818043740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc67b701b0 0x55fc67f3ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:29.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.232+0000 7fc80a800640 1 -- 172.21.15.16:0/71873177 learned_addr learned my addr 172.21.15.16:0/71873177 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:29.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.233+0000 7fc809e00640 1 --2- 172.21.15.16:0/71873177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc67b701b0 0x55fc67f3ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:29.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.233+0000 7fc818043740 1 --2- 172.21.15.16:0/71873177 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc67b78390 0x55fc67f3f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:29.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.233+0000 7fc818043740 1 -- 172.21.15.16:0/71873177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fc67eb5770 con 0x55fc67b701b0 2024-08-31T06:11:29.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.233+0000 7fc818043740 1 -- 172.21.15.16:0/71873177 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55fc67ea4d00 con 0x55fc67b99070 2024-08-31T06:11:29.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.233+0000 7fc818043740 1 -- 172.21.15.16:0/71873177 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55fc67ea8cb0 con 0x55fc67b78390 2024-08-31T06:11:29.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.234+0000 7fc80b200640 1 --2- 172.21.15.16:0/71873177 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc67b78390 0x55fc67f3f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:29.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.235+0000 7fc809e00640 1 -- 172.21.15.16:0/71873177 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc67b78390 msgr2=0x55fc67f3f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:29.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.235+0000 7fc809e00640 1 --2- 172.21.15.16:0/71873177 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc67b78390 0x55fc67f3f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:29.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.235+0000 7fc809e00640 1 -- 172.21.15.16:0/71873177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc67b99070 msgr2=0x55fc67f3e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:29.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.235+0000 7fc809e00640 1 --2- 172.21.15.16:0/71873177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc67b99070 0x55fc67f3e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:29.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.235+0000 7fc809e00640 1 -- 172.21.15.16:0/71873177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc67c04650 con 0x55fc67b701b0 2024-08-31T06:11:29.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.235+0000 7fc809e00640 1 --2- 172.21.15.16:0/71873177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc67b701b0 0x55fc67f3ee90 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc80c06a840 tx=0x7fc7f8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da750e628c36008b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:29.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.236+0000 7fc7f7400640 1 -- 172.21.15.16:0/71873177 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8004050 con 0x55fc67b701b0 2024-08-31T06:11:29.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.237+0000 7fc7f7400640 1 -- 172.21.15.16:0/71873177 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7f80041f0 con 0x55fc67b701b0 2024-08-31T06:11:29.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.237+0000 7fc7f7400640 1 -- 172.21.15.16:0/71873177 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8019690 con 0x55fc67b701b0 2024-08-31T06:11:29.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.238+0000 7fc818043740 1 -- 172.21.15.16:0/71873177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc67b701b0 msgr2=0x55fc67f3ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:29.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.238+0000 7fc818043740 1 --2- 172.21.15.16:0/71873177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc67b701b0 0x55fc67f3ee90 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc80c06a840 tx=0x7fc7f8011810 comp rx=0 tx=0).stop 2024-08-31T06:11:29.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.239+0000 7fc818043740 1 -- 172.21.15.16:0/71873177 shutdown_connections 2024-08-31T06:11:29.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.239+0000 7fc818043740 1 --2- 172.21.15.16:0/71873177 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc67b78390 0x55fc67f3f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:29.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.239+0000 7fc818043740 1 --2- 172.21.15.16:0/71873177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc67b701b0 0x55fc67f3ee90 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:29.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.239+0000 7fc818043740 1 --2- 172.21.15.16:0/71873177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc67b99070 0x55fc67f3e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:29.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.239+0000 7fc818043740 1 -- 172.21.15.16:0/71873177 >> 172.21.15.16:0/71873177 conn(0x55fc67b8d460 msgr2=0x55fc67f3c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:29.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.239+0000 7fc818043740 1 -- 172.21.15.16:0/71873177 shutdown_connections 2024-08-31T06:11:29.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.239+0000 7fc818043740 1 -- 172.21.15.16:0/71873177 wait complete. 2024-08-31T06:11:29.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.240+0000 7fc818043740 1 -- start start 2024-08-31T06:11:29.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.240+0000 7fc818043740 1 Processor -- start 2024-08-31T06:11:29.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.242+0000 7fc818043740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc67b701b0 0x55fc67f52cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:29.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.242+0000 7fc818043740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc67b78390 0x55fc67f53230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:29.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.242+0000 7fc818043740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc67b99070 0x55fc67f5eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:29.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.242+0000 7fc809e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc67b78390 0x55fc67f53230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:29.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.243+0000 7fc809e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc67b78390 0x55fc67f53230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55622/0 (socket says 172.21.15.16:55622) 2024-08-31T06:11:29.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.243+0000 7fc818043740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fc67d07700 con 0x55fc67b78390 2024-08-31T06:11:29.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.243+0000 7fc809e00640 1 -- 172.21.15.16:0/1043228537 learned_addr learned my addr 172.21.15.16:0/1043228537 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:29.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.243+0000 7fc818043740 1 -- 172.21.15.16:0/1043228537 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55fc67e2bcb0 con 0x55fc67b701b0 2024-08-31T06:11:29.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.243+0000 7fc80b200640 1 --2- 172.21.15.16:0/1043228537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc67b99070 0x55fc67f5eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:29.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.243+0000 7fc818043740 1 -- 172.21.15.16:0/1043228537 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55fc67f55860 con 0x55fc67b99070 2024-08-31T06:11:29.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.243+0000 7fc80a800640 1 --2- 172.21.15.16:0/1043228537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc67b701b0 0x55fc67f52cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:29.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.243+0000 7fc818043740 10 client.0 _ll_register_callbacks cb 0x55fc67e51430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:11:29.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.244+0000 7fc80b200640 1 -- 172.21.15.16:0/1043228537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc67b701b0 msgr2=0x55fc67f52cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:29.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.244+0000 7fc80b200640 1 --2- 172.21.15.16:0/1043228537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc67b701b0 0x55fc67f52cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:29.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.244+0000 7fc80b200640 1 -- 172.21.15.16:0/1043228537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc67b78390 msgr2=0x55fc67f53230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:29.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.244+0000 7fc80b200640 1 --2- 172.21.15.16:0/1043228537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc67b78390 0x55fc67f53230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:29.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.244+0000 7fc80b200640 1 -- 172.21.15.16:0/1043228537 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc67e6c870 con 0x55fc67b99070 2024-08-31T06:11:29.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.245+0000 7fc80b200640 1 --2- 172.21.15.16:0/1043228537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc67b99070 0x55fc67f5eb90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc1034a0 tx=0x7fc7fc0cbc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=833e981477ecc598 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:29.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.245+0000 7fc7ebe00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:11:29.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.245+0000 7fc7ebe00640 1 -- 172.21.15.16:0/1043228537 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc0cc440 con 0x55fc67b99070 2024-08-31T06:11:29.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.245+0000 7fc7ebe00640 1 -- 172.21.15.16:0/1043228537 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fc7fc0d52d0 con 0x55fc67b99070 2024-08-31T06:11:29.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.245+0000 7fc818043740 1 -- 172.21.15.16:0/1043228537 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7f80041f0 con 0x55fc67b99070 2024-08-31T06:11:29.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.246+0000 7fc7ebe00640 1 -- 172.21.15.16:0/1043228537 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc7fc0d0cb0 con 0x55fc67b99070 2024-08-31T06:11:29.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.246+0000 7fc818043740 10 client.26536 fetch_fsmap learned FSMap version 34 2024-08-31T06:11:29.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.246+0000 7fc818043740 1 -- 172.21.15.16:0/1043228537 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55fc67f61860 con 0x55fc67b99070 2024-08-31T06:11:29.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.246+0000 7fc7ebe00640 1 -- 172.21.15.16:0/1043228537 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7f80041f0 con 0x55fc67b99070 2024-08-31T06:11:29.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.246+0000 7fc7ebe00640 1 -- 172.21.15.16:0/1043228537 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7fc7fc03b030 con 0x55fc67b99070 2024-08-31T06:11:29.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.247+0000 7fc818043740 10 client.26536 fetch_fsmap finished waiting for FSMap version 34 2024-08-31T06:11:29.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.247+0000 7fc818043740 10 client.26536 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:11:29.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.247+0000 7fc818043740 20 client.26536 populate_metadata read hostname 'smithi016' 2024-08-31T06:11:29.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.247+0000 7fc818043740 1 --2- 172.21.15.16:0/1043228537 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fc67b579f0 0x55fc67f64c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:29.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.247+0000 7fc818043740 4 client.26536 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:11:29.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.247+0000 7fc818043740 1 -- 172.21.15.16:0/1043228537 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc67eb76c0 con 0x55fc67b579f0 2024-08-31T06:11:29.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.247+0000 7fc80a800640 1 --2- 172.21.15.16:0/1043228537 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fc67b579f0 0x55fc67f64c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:29.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.248+0000 7fc80a800640 1 --2- 172.21.15.16:0/1043228537 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fc67b579f0 0x55fc67f64c80 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76591e77505bb6af server_cookie=b8af3c430e807eff in_seq=0 out_seq=0 2024-08-31T06:11:29.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.248+0000 7fc7ebe00640 10 client.26536 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:11:29.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.248+0000 7fc7ebe00640 1 -- 172.21.15.16:0/1043228537 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fc67eb76c0 con 0x55fc67b579f0 2024-08-31T06:11:29.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.248+0000 7fc7ebe00640 10 client.26536 handle_command_reply: tid=0 2024-08-31T06:11:29.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.249+0000 7fc818043740 1 client.26536 shutdown 2024-08-31T06:11:29.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.249+0000 7fc818043740 1 -- 172.21.15.16:0/1043228537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc67b99070 msgr2=0x55fc67f5eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:29.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.249+0000 7fc818043740 1 --2- 172.21.15.16:0/1043228537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc67b99070 0x55fc67f5eb90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc1034a0 tx=0x7fc7fc0cbc70 comp rx=0 tx=0).stop 2024-08-31T06:11:29.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.249+0000 7fc818043740 1 -- 172.21.15.16:0/1043228537 shutdown_connections 2024-08-31T06:11:29.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.249+0000 7fc818043740 1 --2- 172.21.15.16:0/1043228537 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fc67b579f0 0x55fc67f64c80 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:29.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.249+0000 7fc7ebe00640 0 client.26536 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:11:29.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.249+0000 7fc818043740 1 --2- 172.21.15.16:0/1043228537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc67b99070 0x55fc67f5eb90 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:29.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.249+0000 7fc818043740 1 --2- 172.21.15.16:0/1043228537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc67b78390 0x55fc67f53230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:29.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.249+0000 7fc818043740 1 --2- 172.21.15.16:0/1043228537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc67b701b0 0x55fc67f52cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:29.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.249+0000 7fc818043740 1 -- 172.21.15.16:0/1043228537 >> 172.21.15.16:0/1043228537 conn(0x55fc67b8d460 msgr2=0x55fc67f3ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:29.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.250+0000 7fc818043740 1 -- 172.21.15.16:0/1043228537 shutdown_connections 2024-08-31T06:11:29.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.250+0000 7fc818043740 1 -- 172.21.15.16:0/1043228537 wait complete. 2024-08-31T06:11:29.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.250+0000 7fc818043740 20 client.26536 trim_cache size 0 max 16384 2024-08-31T06:11:29.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.283+0000 7fc818043740 1 Processor -- start 2024-08-31T06:11:29.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.283+0000 7fc818043740 1 -- start start 2024-08-31T06:11:29.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.284+0000 7fc818043740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc67b701b0 0x55fc67f3c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:29.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.284+0000 7fc818043740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc67b78390 0x55fc67f40090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:29.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.284+0000 7fc818043740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc67b99070 0x55fc67f5ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:29.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.284+0000 7fc818043740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c1b7ab0 con 0x55fc67b78390 2024-08-31T06:11:29.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.284+0000 7fc818043740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c1b81b0 con 0x55fc67b701b0 2024-08-31T06:11:29.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.284+0000 7fc818043740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c1b5a50 con 0x55fc67b99070 2024-08-31T06:11:29.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.284+0000 7fc80a800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc67b701b0 0x55fc67f3c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:29.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.284+0000 7fc80a800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc67b701b0 0x55fc67f3c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:38462/0 (socket says 172.21.15.16:38462) 2024-08-31T06:11:29.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.284+0000 7fc80a800640 1 -- 172.21.15.16:0/621280358 learned_addr learned my addr 172.21.15.16:0/621280358 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:29.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.284+0000 7fc80b200640 1 --2- 172.21.15.16:0/621280358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc67b99070 0x55fc67f5ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:29.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.285+0000 7fc809e00640 1 --2- 172.21.15.16:0/621280358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc67b78390 0x55fc67f40090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:29.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.285+0000 7fc80b200640 1 -- 172.21.15.16:0/621280358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc67b701b0 msgr2=0x55fc67f3c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:29.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.285+0000 7fc80b200640 1 --2- 172.21.15.16:0/621280358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc67b701b0 0x55fc67f3c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:29.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.285+0000 7fc80b200640 1 -- 172.21.15.16:0/621280358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc67b78390 msgr2=0x55fc67f40090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:29.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.285+0000 7fc80b200640 1 --2- 172.21.15.16:0/621280358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc67b78390 0x55fc67f40090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:29.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.285+0000 7fc80b200640 1 -- 172.21.15.16:0/621280358 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc67f53db0 con 0x55fc67b99070 2024-08-31T06:11:29.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.286+0000 7fc80b200640 1 --2- 172.21.15.16:0/621280358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc67b99070 0x55fc67f5ea90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc0cc140 tx=0x7fc7fc0c8430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3b7fb40f8c49f82 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:29.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.286+0000 7fc7f6a00640 1 -- 172.21.15.16:0/621280358 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc0cc440 con 0x55fc67b99070 2024-08-31T06:11:29.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.286+0000 7fc7f6a00640 1 -- 172.21.15.16:0/621280358 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7fc0d5760 con 0x55fc67b99070 2024-08-31T06:11:29.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.286+0000 7fc7f6a00640 1 -- 172.21.15.16:0/621280358 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc0b3ba0 con 0x55fc67b99070 2024-08-31T06:11:29.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.287+0000 7fc818043740 1 -- 172.21.15.16:0/621280358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc67b99070 msgr2=0x55fc67f5ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:29.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.287+0000 7fc818043740 1 --2- 172.21.15.16:0/621280358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc67b99070 0x55fc67f5ea90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc0cc140 tx=0x7fc7fc0c8430 comp rx=0 tx=0).stop 2024-08-31T06:11:29.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.287+0000 7fc818043740 1 -- 172.21.15.16:0/621280358 shutdown_connections 2024-08-31T06:11:29.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.287+0000 7fc818043740 1 --2- 172.21.15.16:0/621280358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc67b99070 0x55fc67f5ea90 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:29.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.287+0000 7fc818043740 1 --2- 172.21.15.16:0/621280358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc67b78390 0x55fc67f40090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:29.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.287+0000 7fc818043740 1 --2- 172.21.15.16:0/621280358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc67b701b0 0x55fc67f3c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:29.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.287+0000 7fc818043740 1 -- 172.21.15.16:0/621280358 >> 172.21.15.16:0/621280358 conn(0x55fc67b8d460 msgr2=0x55fc67f3f980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:29.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.288+0000 7fc818043740 1 -- 172.21.15.16:0/621280358 shutdown_connections 2024-08-31T06:11:29.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.288+0000 7fc818043740 1 -- 172.21.15.16:0/621280358 wait complete. 2024-08-31T06:11:29.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.288+0000 7fc818043740 1 -- start start 2024-08-31T06:11:29.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.288+0000 7fc818043740 1 Processor -- start 2024-08-31T06:11:29.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.289+0000 7fc818043740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc67b701b0 0x55fc67f5b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:29.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.289+0000 7fc818043740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc67b78390 0x55fc67f57620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:29.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.289+0000 7fc818043740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc67b99070 0x55fc67f57b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:29.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.289+0000 7fc818043740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c1b3f10 con 0x55fc67b99070 2024-08-31T06:11:29.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.289+0000 7fc818043740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c1b5380 con 0x55fc67b78390 2024-08-31T06:11:29.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.289+0000 7fc818043740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c1b45e0 con 0x55fc67b701b0 2024-08-31T06:11:29.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.289+0000 7fc818043740 10 client.0 _ll_register_callbacks cb 0x7fc7fc0cbe60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:11:29.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.289+0000 7fc80a800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc67b701b0 0x55fc67f5b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:29.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.289+0000 7fc80a800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc67b701b0 0x55fc67f5b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:59000/0 (socket says 172.21.15.16:59000) 2024-08-31T06:11:29.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.289+0000 7fc80a800640 1 -- 172.21.15.16:0/648559357 learned_addr learned my addr 172.21.15.16:0/648559357 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:29.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.290+0000 7fc809e00640 1 --2- 172.21.15.16:0/648559357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc67b78390 0x55fc67f57620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:29.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.290+0000 7fc80b200640 1 --2- 172.21.15.16:0/648559357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc67b99070 0x55fc67f57b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:29.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.291+0000 7fc80a800640 1 -- 172.21.15.16:0/648559357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc67b78390 msgr2=0x55fc67f57620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:29.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.291+0000 7fc80a800640 1 --2- 172.21.15.16:0/648559357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc67b78390 0x55fc67f57620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:29.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.291+0000 7fc80a800640 1 -- 172.21.15.16:0/648559357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc67b99070 msgr2=0x55fc67f57b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:29.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.291+0000 7fc80a800640 1 --2- 172.21.15.16:0/648559357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc67b99070 0x55fc67f57b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:29.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.291+0000 7fc80a800640 1 -- 172.21.15.16:0/648559357 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc67f58170 con 0x55fc67b701b0 2024-08-31T06:11:29.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.291+0000 7fc80b200640 1 --2- 172.21.15.16:0/648559357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc67b99070 0x55fc67f57b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:11:29.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.291+0000 7fc80a800640 1 --2- 172.21.15.16:0/648559357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc67b701b0 0x55fc67f5b160 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc80400f750 tx=0x7fc804004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=901e19468fa4c140 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:29.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.291+0000 7fc7ebe00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:11:29.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.292+0000 7fc7ebe00640 1 -- 172.21.15.16:0/648559357 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc804033060 con 0x55fc67b701b0 2024-08-31T06:11:29.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.292+0000 7fc7ebe00640 1 -- 172.21.15.16:0/648559357 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fc80401fb50 con 0x55fc67b701b0 2024-08-31T06:11:29.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.292+0000 7fc818043740 1 -- 172.21.15.16:0/648559357 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7fc0d5760 con 0x55fc67b701b0 2024-08-31T06:11:29.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.292+0000 7fc7ebe00640 1 -- 172.21.15.16:0/648559357 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc80401fd50 con 0x55fc67b701b0 2024-08-31T06:11:29.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.293+0000 7fc818043740 10 client.26542 fetch_fsmap learned FSMap version 34 2024-08-31T06:11:29.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.293+0000 7fc818043740 1 -- 172.21.15.16:0/648559357 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55fc67fd0500 con 0x55fc67b701b0 2024-08-31T06:11:29.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.293+0000 7fc7ebe00640 1 -- 172.21.15.16:0/648559357 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7fc0d5760 con 0x55fc67b701b0 2024-08-31T06:11:29.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.293+0000 7fc7ebe00640 1 -- 172.21.15.16:0/648559357 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7fc80401d430 con 0x55fc67b701b0 2024-08-31T06:11:29.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.293+0000 7fc818043740 10 client.26542 fetch_fsmap finished waiting for FSMap version 34 2024-08-31T06:11:29.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.293+0000 7fc818043740 10 client.26542 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:11:29.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.293+0000 7fc818043740 20 client.26542 populate_metadata read hostname 'smithi016' 2024-08-31T06:11:29.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.293+0000 7fc818043740 1 --2- 172.21.15.16:0/648559357 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fc67b2cda0 0x55fc67fd2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:29.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.293+0000 7fc818043740 4 client.26542 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T06:11:29.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.293+0000 7fc818043740 1 -- 172.21.15.16:0/648559357 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fc67eb3160 con 0x55fc67b2cda0 2024-08-31T06:11:29.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.294+0000 7fc809e00640 1 --2- 172.21.15.16:0/648559357 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fc67b2cda0 0x55fc67fd2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:29.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.294+0000 7fc809e00640 1 --2- 172.21.15.16:0/648559357 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fc67b2cda0 0x55fc67fd2d00 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6febf624992b8f19 server_cookie=150f30eedc6ec957 in_seq=0 out_seq=0 2024-08-31T06:11:29.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.294+0000 7fc7ebe00640 10 client.26542 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:11:29.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.295+0000 7fc7ebe00640 1 -- 172.21.15.16:0/648559357 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1794 (crc 0 0 0) 0x55fc67eb3160 con 0x55fc67b2cda0 2024-08-31T06:11:29.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.295+0000 7fc7ebe00640 10 client.26542 handle_command_reply: tid=0 2024-08-31T06:11:29.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.295+0000 7fc818043740 1 client.26542 shutdown 2024-08-31T06:11:29.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.296+0000 7fc818043740 1 -- 172.21.15.16:0/648559357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc67b701b0 msgr2=0x55fc67f5b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:29.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.296+0000 7fc818043740 1 --2- 172.21.15.16:0/648559357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc67b701b0 0x55fc67f5b160 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc80400f750 tx=0x7fc804004730 comp rx=0 tx=0).stop 2024-08-31T06:11:29.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.296+0000 7fc818043740 1 -- 172.21.15.16:0/648559357 shutdown_connections 2024-08-31T06:11:29.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.296+0000 7fc818043740 1 --2- 172.21.15.16:0/648559357 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fc67b2cda0 0x55fc67fd2d00 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:29.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.296+0000 7fc818043740 1 --2- 172.21.15.16:0/648559357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc67b99070 0x55fc67f57b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:29.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.296+0000 7fc818043740 1 --2- 172.21.15.16:0/648559357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc67b78390 0x55fc67f57620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:29.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.296+0000 7fc818043740 1 --2- 172.21.15.16:0/648559357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc67b701b0 0x55fc67f5b160 secure :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc80400f750 tx=0x7fc804004730 comp rx=0 tx=0).stop 2024-08-31T06:11:29.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.296+0000 7fc818043740 1 -- 172.21.15.16:0/648559357 >> 172.21.15.16:0/648559357 conn(0x55fc67b8d460 msgr2=0x55fc67f40340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:29.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.297+0000 7fc7ebe00640 0 client.26542 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:11:29.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.297+0000 7fc818043740 1 -- 172.21.15.16:0/648559357 shutdown_connections 2024-08-31T06:11:29.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.297+0000 7fc818043740 1 -- 172.21.15.16:0/648559357 wait complete. 2024-08-31T06:11:29.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.297+0000 7fc818043740 20 client.26542 trim_cache size 0 max 16384 2024-08-31T06:11:29.300 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:11:29.300 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-08-31T06:11:29.300 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {}, 2024-08-31T06:11:29.300 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-08-31T06:11:29.300 INFO:teuthology.orchestra.run.smithi016.stdout: "e739a16e-8ace-43a8-a1ce-ac27d39ef8a3": { 2024-08-31T06:11:29.301 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:54.084646+0000", 2024-08-31T06:11:29.301 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:11:29.301 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:11:29.301 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:11:29.301 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:11:29.301 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:11:29.301 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:11:29.301 INFO:teuthology.orchestra.run.smithi016.stdout: "c1fd6904-2e02-47fa-939a-47349b9fe36b": { 2024-08-31T06:11:29.301 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:06.678691+0000", 2024-08-31T06:11:29.301 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:11:29.301 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:11:29.302 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:11:29.302 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:11:29.302 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:11:29.302 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:11:29.302 INFO:teuthology.orchestra.run.smithi016.stdout: "78d6ef02-8551-45e8-ad60-9627eb573fc1": { 2024-08-31T06:11:29.302 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:09:13.547865+0000", 2024-08-31T06:11:29.302 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:11:29.302 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:11:29.302 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:11:29.302 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:11:29.302 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:11:29.302 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:11:29.302 INFO:teuthology.orchestra.run.smithi016.stdout: "c8455d74-2fe7-4d93-bcf2-19b419d16016": { 2024-08-31T06:11:29.303 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:08:24.035960+0000", 2024-08-31T06:11:29.303 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:11:29.303 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:11:29.303 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:11:29.303 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:11:29.303 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:11:29.303 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:11:29.303 INFO:teuthology.orchestra.run.smithi016.stdout: "88d7bdb8-cb1c-45e5-ad29-4cd8da6c6003": { 2024-08-31T06:11:29.303 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:07:39.648942+0000", 2024-08-31T06:11:29.303 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:11:29.303 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:11:29.303 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:11:29.303 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:11:29.303 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:11:29.303 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:11:29.304 INFO:teuthology.orchestra.run.smithi016.stdout: "c300b7e8-f3fd-482c-a909-9329bc61e570": { 2024-08-31T06:11:29.304 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:06:49.868046+0000", 2024-08-31T06:11:29.304 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:11:29.304 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:11:29.304 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:11:29.304 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:11:29.304 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:11:29.304 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:11:29.304 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:11:29.304 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:11:29.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.299+0000 7fc811400640 1 -- 172.21.15.16:0/3940359813 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc7f00b4d50 msgr2=0x7fc7f00b7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:29.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.299+0000 7fc811400640 1 --2- 172.21.15.16:0/3940359813 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc7f00b4d50 0x7fc7f00b7260 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc80400baa0 tx=0x7fc804005d90 comp rx=0 tx=0).stop 2024-08-31T06:11:29.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.299+0000 7fc811400640 1 -- 172.21.15.16:0/3940359813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc80c108a90 msgr2=0x7fc80c06e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:29.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.299+0000 7fc811400640 1 --2- 172.21.15.16:0/3940359813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc80c108a90 0x7fc80c06e4d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc02c9b0 tx=0x7fc7fc003ba0 comp rx=0 tx=0).stop 2024-08-31T06:11:29.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.299+0000 7fc811400640 1 -- 172.21.15.16:0/3940359813 shutdown_connections 2024-08-31T06:11:29.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.299+0000 7fc811400640 1 --2- 172.21.15.16:0/3940359813 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc7f00b4d50 0x7fc7f00b7260 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:29.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.299+0000 7fc811400640 1 --2- 172.21.15.16:0/3940359813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc80c108140 0x7fc80c06df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:29.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.299+0000 7fc811400640 1 --2- 172.21.15.16:0/3940359813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc80c108a90 0x7fc80c06e4d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:29.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.299+0000 7fc811400640 1 --2- 172.21.15.16:0/3940359813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc80c102800 0x7fc80c074c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:29.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.299+0000 7fc811400640 1 -- 172.21.15.16:0/3940359813 >> 172.21.15.16:0/3940359813 conn(0x7fc80c0794d0 msgr2=0x7fc80c1041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:29.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.300+0000 7fc811400640 1 -- 172.21.15.16:0/3940359813 shutdown_connections 2024-08-31T06:11:29.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:29.300+0000 7fc811400640 1 -- 172.21.15.16:0/3940359813 wait complete. 2024-08-31T06:11:29.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:29 smithi184 ceph-mon[22331]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:11:30.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:29 smithi155 ceph-mon[21742]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:11:30.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:29 smithi016 ceph-mon[17194]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:11:30.304 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T06:11:30.304 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:11:30.959 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:11:30.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:30 smithi184 ceph-mon[22331]: pgmap v492: 97 pgs: 97 active+clean; 2.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 MiB/s wr, 1.72k op/s 2024-08-31T06:11:31.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:30 smithi155 ceph-mon[21742]: pgmap v492: 97 pgs: 97 active+clean; 2.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 MiB/s wr, 1.72k op/s 2024-08-31T06:11:31.124 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:30 smithi016 ceph-mon[17194]: pgmap v492: 97 pgs: 97 active+clean; 2.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 MiB/s wr, 1.72k op/s 2024-08-31T06:11:31.625 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 245.0 s, 2831.1 tps, lat 35.264 ms stddev 39.773 2024-08-31T06:11:31.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:31.826+0000 7f019d800640 1 -- 172.21.15.16:0/357635260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0198104e40 msgr2=0x7f019810d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:31.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:31.826+0000 7f019d800640 1 --2- 172.21.15.16:0/357635260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0198104e40 0x7f019810d480 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0184009900 tx=0x7f018402a400 comp rx=0 tx=0).stop 2024-08-31T06:11:31.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:31.827+0000 7f019d800640 1 -- 172.21.15.16:0/357635260 shutdown_connections 2024-08-31T06:11:31.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:31.827+0000 7f019d800640 1 --2- 172.21.15.16:0/357635260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0198104e40 0x7f019810d480 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:31.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:31.827+0000 7f019d800640 1 --2- 172.21.15.16:0/357635260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f019810d9c0 0x7f019810feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:31.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:31.827+0000 7f019d800640 1 --2- 172.21.15.16:0/357635260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0198104410 0x7f0198104820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:31.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:31.827+0000 7f019d800640 1 -- 172.21.15.16:0/357635260 >> 172.21.15.16:0/357635260 conn(0x7f01980fe060 msgr2=0x7f01981004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:31.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:31.827+0000 7f019d800640 1 -- 172.21.15.16:0/357635260 shutdown_connections 2024-08-31T06:11:31.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:31.828+0000 7f019d800640 1 -- 172.21.15.16:0/357635260 wait complete. 2024-08-31T06:11:31.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:31.829+0000 7f019d800640 1 Processor -- start 2024-08-31T06:11:31.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:31.829+0000 7f019d800640 1 -- start start 2024-08-31T06:11:31.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:31.830+0000 7f019d800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0198104410 0x7f0198109240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:31.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:31.830+0000 7f019d800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0198104e40 0x7f0198105520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:31.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:31.830+0000 7f019d800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f019810d9c0 0x7f0198105c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:31.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:31.830+0000 7f019d800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0198113640 con 0x7f0198104e40 2024-08-31T06:11:31.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:31.830+0000 7f019d800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f01981134c0 con 0x7f0198104410 2024-08-31T06:11:31.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:31.830+0000 7f019d800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f01981137c0 con 0x7f019810d9c0 2024-08-31T06:11:31.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:31.831+0000 7f0197e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0198104410 0x7f0198109240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:31.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:31.831+0000 7f0197e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0198104410 0x7f0198109240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:38490/0 (socket says 172.21.15.16:38490) 2024-08-31T06:11:31.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:31.831+0000 7f0197e00640 1 -- 172.21.15.16:0/225483978 learned_addr learned my addr 172.21.15.16:0/225483978 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:31.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:31.831+0000 7f019ce00640 1 --2- 172.21.15.16:0/225483978 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f019810d9c0 0x7f0198105c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:31.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:31.831+0000 7f0197400640 1 --2- 172.21.15.16:0/225483978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0198104e40 0x7f0198105520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:31.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:31.832+0000 7f019ce00640 1 -- 172.21.15.16:0/225483978 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0198104410 msgr2=0x7f0198109240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:31.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:31.832+0000 7f019ce00640 1 --2- 172.21.15.16:0/225483978 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0198104410 0x7f0198109240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:31.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:31.832+0000 7f019ce00640 1 -- 172.21.15.16:0/225483978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0198104e40 msgr2=0x7f0198105520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:31.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:31.832+0000 7f019ce00640 1 --2- 172.21.15.16:0/225483978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0198104e40 0x7f0198105520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:31.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:31.832+0000 7f019ce00640 1 -- 172.21.15.16:0/225483978 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01981ac7e0 con 0x7f019810d9c0 2024-08-31T06:11:31.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:31.832+0000 7f019ce00640 1 --2- 172.21.15.16:0/225483978 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f019810d9c0 0x7f0198105c40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f019000ca30 tx=0x7f019000cef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:31.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:31.833+0000 7f0194c00640 1 -- 172.21.15.16:0/225483978 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0190007d90 con 0x7f019810d9c0 2024-08-31T06:11:31.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:31.833+0000 7f0194c00640 1 -- 172.21.15.16:0/225483978 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0190005020 con 0x7f019810d9c0 2024-08-31T06:11:31.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:31.833+0000 7f0194c00640 1 -- 172.21.15.16:0/225483978 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019000f690 con 0x7f019810d9c0 2024-08-31T06:11:31.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:31.833+0000 7f019d800640 1 -- 172.21.15.16:0/225483978 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01981ac9b0 con 0x7f019810d9c0 2024-08-31T06:11:31.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:31.833+0000 7f019d800640 1 -- 172.21.15.16:0/225483978 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01981ace70 con 0x7f019810d9c0 2024-08-31T06:11:31.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:31.835+0000 7f019d800640 1 -- 172.21.15.16:0/225483978 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f019806b280 con 0x7f019810d9c0 2024-08-31T06:11:31.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:31.839+0000 7f0194c00640 1 -- 172.21.15.16:0/225483978 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f01900051d0 con 0x7f019810d9c0 2024-08-31T06:11:31.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:31.841+0000 7f0194c00640 1 --2- 172.21.15.16:0/225483978 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f017c0b4ca0 0x7f017c0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:31.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:31.841+0000 7f0194c00640 1 -- 172.21.15.16:0/225483978 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f01900e57e0 con 0x7f019810d9c0 2024-08-31T06:11:31.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:31.842+0000 7f0197e00640 1 --2- 172.21.15.16:0/225483978 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f017c0b4ca0 0x7f017c0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:31.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:31.843+0000 7f0197e00640 1 --2- 172.21.15.16:0/225483978 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f017c0b4ca0 0x7f017c0b71b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0198108f90 tx=0x7f018800a380 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:31.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:31.846+0000 7f0194c00640 1 -- 172.21.15.16:0/225483978 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f0190014070 con 0x7f019810d9c0 2024-08-31T06:11:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.906+0000 7f08e1c00640 1 -- 172.21.15.16:0/3880869575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08dc107950 msgr2=0x7f08dc107dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.906+0000 7f08e1c00640 1 --2- 172.21.15.16:0/3880869575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08dc107950 0x7f08dc107dc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f08c8009900 tx=0x7f08c802a400 comp rx=0 tx=0).stop 2024-08-31T06:11:31.910 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.909+0000 7f08e1c00640 1 -- 172.21.15.16:0/3880869575 shutdown_connections 2024-08-31T06:11:31.910 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.909+0000 7f08e1c00640 1 --2- 172.21.15.16:0/3880869575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08dc107950 0x7f08dc107dc0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:31.910 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.909+0000 7f08e1c00640 1 --2- 172.21.15.16:0/3880869575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08dc102de0 0x7f08dc107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:31.910 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.909+0000 7f08e1c00640 1 --2- 172.21.15.16:0/3880869575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08dc102490 0x7f08dc1028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.909+0000 7f08e1c00640 1 -- 172.21.15.16:0/3880869575 >> 172.21.15.16:0/3880869575 conn(0x7f08dc07a4e0 msgr2=0x7f08dc07a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.909+0000 7f08e1c00640 1 -- 172.21.15.16:0/3880869575 shutdown_connections 2024-08-31T06:11:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.909+0000 7f08e1c00640 1 -- 172.21.15.16:0/3880869575 wait complete. 2024-08-31T06:11:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.911+0000 7f08e1c00640 1 Processor -- start 2024-08-31T06:11:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.911+0000 7f08e1c00640 1 -- start start 2024-08-31T06:11:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.912+0000 7f08e1c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08dc102490 0x7f08dc1a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.912+0000 7f08e1c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08dc102de0 0x7f08dc1a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.912+0000 7f08e1c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08dc107950 0x7f08dc1a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.912+0000 7f08e1c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc119d90 con 0x7f08dc102de0 2024-08-31T06:11:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.912+0000 7f08e1c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc119c10 con 0x7f08dc107950 2024-08-31T06:11:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.912+0000 7f08e1c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc119f10 con 0x7f08dc102490 2024-08-31T06:11:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.913+0000 7f08daa00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08dc102490 0x7f08dc1a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.913+0000 7f08daa00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08dc102490 0x7f08dc1a0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:59026/0 (socket says 172.21.15.16:59026) 2024-08-31T06:11:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.913+0000 7f08daa00640 1 -- 172.21.15.16:0/2798424875 learned_addr learned my addr 172.21.15.16:0/2798424875 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.913+0000 7f08da000640 1 --2- 172.21.15.16:0/2798424875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08dc102de0 0x7f08dc1a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:31.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.913+0000 7f08db400640 1 --2- 172.21.15.16:0/2798424875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08dc107950 0x7f08dc1a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:31.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.914+0000 7f08db400640 1 -- 172.21.15.16:0/2798424875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08dc102490 msgr2=0x7f08dc1a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:31.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.914+0000 7f08db400640 1 --2- 172.21.15.16:0/2798424875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08dc102490 0x7f08dc1a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:31.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.914+0000 7f08db400640 1 -- 172.21.15.16:0/2798424875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08dc102de0 msgr2=0x7f08dc1a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:31.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.914+0000 7f08db400640 1 --2- 172.21.15.16:0/2798424875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08dc102de0 0x7f08dc1a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:31.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.914+0000 7f08db400640 1 -- 172.21.15.16:0/2798424875 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08dc1a66a0 con 0x7f08dc107950 2024-08-31T06:11:31.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.915+0000 7f08daa00640 1 --2- 172.21.15.16:0/2798424875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08dc102490 0x7f08dc1a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:11:31.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.915+0000 7f08da000640 1 --2- 172.21.15.16:0/2798424875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08dc102de0 0x7f08dc1a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:11:31.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.916+0000 7f08db400640 1 --2- 172.21.15.16:0/2798424875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08dc107950 0x7f08dc1a5fa0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f08c8009230 tx=0x7f08c8003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:31.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.916+0000 7f08d3400640 1 -- 172.21.15.16:0/2798424875 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c803f070 con 0x7f08dc107950 2024-08-31T06:11:31.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.917+0000 7f08e1c00640 1 -- 172.21.15.16:0/2798424875 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08dc1a6930 con 0x7f08dc107950 2024-08-31T06:11:31.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.917+0000 7f08e1c00640 1 -- 172.21.15.16:0/2798424875 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08dc1ad120 con 0x7f08dc107950 2024-08-31T06:11:31.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.917+0000 7f08d3400640 1 -- 172.21.15.16:0/2798424875 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08c8031b50 con 0x7f08dc107950 2024-08-31T06:11:31.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.917+0000 7f08d3400640 1 -- 172.21.15.16:0/2798424875 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c8035030 con 0x7f08dc107950 2024-08-31T06:11:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.926+0000 7f08e8bde740 1 Processor -- start 2024-08-31T06:11:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.927+0000 7f08d3400640 1 -- 172.21.15.16:0/2798424875 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f08c8042070 con 0x7f08dc107950 2024-08-31T06:11:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.927+0000 7f08e8bde740 1 -- start start 2024-08-31T06:11:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.927+0000 7f08e8bde740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56031a1281b0 0x56031a4cb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.927+0000 7f08e8bde740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56031a0ff2f0 0x56031a4cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.927+0000 7f08e8bde740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56031a1074d0 0x56031a4cc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.927+0000 7f08e8bde740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56031a440f70 con 0x56031a0ff2f0 2024-08-31T06:11:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.927+0000 7f08e8bde740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56031a430500 con 0x56031a1074d0 2024-08-31T06:11:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.927+0000 7f08e8bde740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56031a4344b0 con 0x56031a1281b0 2024-08-31T06:11:31.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.928+0000 7f08da000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56031a1281b0 0x56031a4cb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:31.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.928+0000 7f08da000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56031a1281b0 0x56031a4cb6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:59042/0 (socket says 172.21.15.16:59042) 2024-08-31T06:11:31.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.928+0000 7f08da000640 1 -- 172.21.15.16:0/3340981381 learned_addr learned my addr 172.21.15.16:0/3340981381 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:31.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.928+0000 7f08da000640 1 --2- 172.21.15.16:0/3340981381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56031a1074d0 0x56031a4cc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:31.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.929+0000 7f08daa00640 1 --2- 172.21.15.16:0/3340981381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56031a0ff2f0 0x56031a4cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:31.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.930+0000 7f08da000640 1 -- 172.21.15.16:0/3340981381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56031a1074d0 msgr2=0x56031a4cc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:31.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.930+0000 7f08da000640 1 --2- 172.21.15.16:0/3340981381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56031a1074d0 0x56031a4cc140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:31.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.930+0000 7f08da000640 1 -- 172.21.15.16:0/3340981381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56031a0ff2f0 msgr2=0x56031a4cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:31.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.930+0000 7f08da000640 1 --2- 172.21.15.16:0/3340981381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56031a0ff2f0 0x56031a4cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:31.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.930+0000 7f08da000640 1 -- 172.21.15.16:0/3340981381 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56031a3eac10 con 0x56031a1281b0 2024-08-31T06:11:31.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.932+0000 7f08d3400640 1 --2- 172.21.15.16:0/2798424875 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f08c00b4d50 0x7f08c00b7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:31.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.932+0000 7f08d3400640 1 -- 172.21.15.16:0/2798424875 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f08c8105d80 con 0x7f08dc107950 2024-08-31T06:11:31.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.934+0000 7f08daa00640 1 --2- 172.21.15.16:0/2798424875 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f08c00b4d50 0x7f08c00b7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:31.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.935+0000 7f08da000640 1 --2- 172.21.15.16:0/3340981381 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56031a1281b0 0x56031a4cb6c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f08cc012970 tx=0x7f08cc00bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7766d96824fd146e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:31.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.935+0000 7f08c7e00640 1 -- 172.21.15.16:0/3340981381 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc004020 con 0x56031a1281b0 2024-08-31T06:11:31.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.935+0000 7f08c7e00640 1 -- 172.21.15.16:0/3340981381 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08cc0054d0 con 0x56031a1281b0 2024-08-31T06:11:31.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.935+0000 7f08c7e00640 1 -- 172.21.15.16:0/3340981381 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc004ba0 con 0x56031a1281b0 2024-08-31T06:11:31.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.937+0000 7f08e8bde740 1 -- 172.21.15.16:0/3340981381 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56031a1281b0 msgr2=0x56031a4cb6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:31.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.937+0000 7f08e8bde740 1 --2- 172.21.15.16:0/3340981381 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56031a1281b0 0x56031a4cb6c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f08cc012970 tx=0x7f08cc00bf90 comp rx=0 tx=0).stop 2024-08-31T06:11:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.937+0000 7f08daa00640 1 --2- 172.21.15.16:0/2798424875 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f08c00b4d50 0x7f08c00b7260 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x56031a4ca340 tx=0x7f08d40075a0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.938+0000 7f08e8bde740 1 -- 172.21.15.16:0/3340981381 shutdown_connections 2024-08-31T06:11:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.938+0000 7f08e8bde740 1 --2- 172.21.15.16:0/3340981381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56031a1074d0 0x56031a4cc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.938+0000 7f08e8bde740 1 --2- 172.21.15.16:0/3340981381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56031a0ff2f0 0x56031a4cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.938+0000 7f08e8bde740 1 --2- 172.21.15.16:0/3340981381 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56031a1281b0 0x56031a4cb6c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.938+0000 7f08e8bde740 1 -- 172.21.15.16:0/3340981381 >> 172.21.15.16:0/3340981381 conn(0x56031a11c300 msgr2=0x56031a4c95b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.939+0000 7f08e8bde740 1 -- 172.21.15.16:0/3340981381 shutdown_connections 2024-08-31T06:11:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.939+0000 7f08e8bde740 1 -- 172.21.15.16:0/3340981381 wait complete. 2024-08-31T06:11:31.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.941+0000 7f08e8bde740 1 -- start start 2024-08-31T06:11:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.943+0000 7f08e8bde740 1 Processor -- start 2024-08-31T06:11:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.945+0000 7f08e8bde740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56031a0ff2f0 0x56031a4d6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.945+0000 7f08e8bde740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56031a1074d0 0x56031a4d72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.945+0000 7f08e8bde740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56031a1281b0 0x56031a4e3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.945+0000 7f08e8bde740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56031a369ab0 con 0x56031a0ff2f0 2024-08-31T06:11:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.945+0000 7f08e8bde740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56031a3d7320 con 0x56031a1074d0 2024-08-31T06:11:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.945+0000 7f08e8bde740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56031a4d9e40 con 0x56031a1281b0 2024-08-31T06:11:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.945+0000 7f08e8bde740 10 client.0 _ll_register_callbacks cb 0x56031a410e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:11:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.946+0000 7f08da000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56031a1074d0 0x56031a4d72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.946+0000 7f08da000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56031a1074d0 0x56031a4d72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:38526/0 (socket says 172.21.15.16:38526) 2024-08-31T06:11:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.946+0000 7f08da000640 1 -- 172.21.15.16:0/2801349749 learned_addr learned my addr 172.21.15.16:0/2801349749 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.947+0000 7f08db400640 1 --2- 172.21.15.16:0/2801349749 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56031a1281b0 0x56031a4e3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.947+0000 7f08daa00640 1 --2- 172.21.15.16:0/2801349749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56031a0ff2f0 0x56031a4d6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:31.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.949+0000 7f08db400640 1 -- 172.21.15.16:0/2801349749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56031a1074d0 msgr2=0x56031a4d72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:31.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.949+0000 7f08db400640 1 --2- 172.21.15.16:0/2801349749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56031a1074d0 0x56031a4d72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:31.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.949+0000 7f08db400640 1 -- 172.21.15.16:0/2801349749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56031a0ff2f0 msgr2=0x56031a4d6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:31.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.949+0000 7f08db400640 1 --2- 172.21.15.16:0/2801349749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56031a0ff2f0 0x56031a4d6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.949+0000 7f08db400640 1 -- 172.21.15.16:0/2801349749 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56031a193790 con 0x56031a1281b0 2024-08-31T06:11:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.950+0000 7f08da000640 1 --2- 172.21.15.16:0/2801349749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56031a1074d0 0x56031a4d72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:11:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.950+0000 7f08db400640 1 --2- 172.21.15.16:0/2801349749 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56031a1281b0 0x56031a4e3130 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f08c81034d0 tx=0x7f08c80cc070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83e7c00fb847c5d3 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.951+0000 7f08c4c00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:11:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.952+0000 7f08c4c00640 1 -- 172.21.15.16:0/2801349749 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c8049050 con 0x56031a1281b0 2024-08-31T06:11:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.952+0000 7f08c4c00640 1 -- 172.21.15.16:0/2801349749 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f08c80c8ad0 con 0x56031a1281b0 2024-08-31T06:11:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.953+0000 7f08e8bde740 1 -- 172.21.15.16:0/2801349749 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08cc0054d0 con 0x56031a1281b0 2024-08-31T06:11:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.953+0000 7f08c4c00640 1 -- 172.21.15.16:0/2801349749 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f08c80d10b0 con 0x56031a1281b0 2024-08-31T06:11:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.954+0000 7f08e8bde740 10 client.26560 fetch_fsmap learned FSMap version 34 2024-08-31T06:11:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.954+0000 7f08e8bde740 1 -- 172.21.15.16:0/2801349749 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56031a20c620 con 0x56031a1281b0 2024-08-31T06:11:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.955+0000 7f08c4c00640 1 -- 172.21.15.16:0/2801349749 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08cc0054d0 con 0x56031a1281b0 2024-08-31T06:11:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.955+0000 7f08c4c00640 1 -- 172.21.15.16:0/2801349749 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f08c80316a0 con 0x56031a1281b0 2024-08-31T06:11:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.956+0000 7f08e8bde740 10 client.26560 fetch_fsmap finished waiting for FSMap version 34 2024-08-31T06:11:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.956+0000 7f08e8bde740 10 client.26560 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:11:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.956+0000 7f08e8bde740 20 client.26560 populate_metadata read hostname 'smithi016' 2024-08-31T06:11:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.956+0000 7f08e8bde740 1 --2- 172.21.15.16:0/2801349749 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56031a0e6d80 0x56031a4e8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.956+0000 7f08e8bde740 4 client.26560 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:11:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.956+0000 7f08e8bde740 1 -- 172.21.15.16:0/2801349749 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56031a442ec0 con 0x56031a0e6d80 2024-08-31T06:11:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.956+0000 7f08daa00640 1 --2- 172.21.15.16:0/2801349749 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56031a0e6d80 0x56031a4e8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.958+0000 7f08daa00640 1 --2- 172.21.15.16:0/2801349749 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56031a0e6d80 0x56031a4e8e50 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17eebd6015eb37f3 server_cookie=e9df81d7fa10d278 in_seq=0 out_seq=0 2024-08-31T06:11:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.959+0000 7f08c4c00640 10 client.26560 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:11:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.960+0000 7f08c4c00640 1 -- 172.21.15.16:0/2801349749 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56031a442ec0 con 0x56031a0e6d80 2024-08-31T06:11:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.960+0000 7f08c4c00640 10 client.26560 handle_command_reply: tid=0 2024-08-31T06:11:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.960+0000 7f08e8bde740 1 client.26560 shutdown 2024-08-31T06:11:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.961+0000 7f08e8bde740 1 -- 172.21.15.16:0/2801349749 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56031a1281b0 msgr2=0x56031a4e3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.961+0000 7f08e8bde740 1 --2- 172.21.15.16:0/2801349749 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56031a1281b0 0x56031a4e3130 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f08c81034d0 tx=0x7f08c80cc070 comp rx=0 tx=0).stop 2024-08-31T06:11:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.962+0000 7f08e8bde740 1 -- 172.21.15.16:0/2801349749 shutdown_connections 2024-08-31T06:11:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.962+0000 7f08e8bde740 1 --2- 172.21.15.16:0/2801349749 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56031a0e6d80 0x56031a4e8e50 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.962+0000 7f08e8bde740 1 --2- 172.21.15.16:0/2801349749 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56031a1281b0 0x56031a4e3130 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.962+0000 7f08e8bde740 1 --2- 172.21.15.16:0/2801349749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56031a1074d0 0x56031a4d72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.962+0000 7f08e8bde740 1 --2- 172.21.15.16:0/2801349749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56031a0ff2f0 0x56031a4d6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.962+0000 7f08e8bde740 1 -- 172.21.15.16:0/2801349749 >> 172.21.15.16:0/2801349749 conn(0x56031a11c300 msgr2=0x56031a4cc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.962+0000 7f08c4c00640 0 client.26560 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:11:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.962+0000 7f08e8bde740 1 -- 172.21.15.16:0/2801349749 shutdown_connections 2024-08-31T06:11:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.962+0000 7f08e8bde740 1 -- 172.21.15.16:0/2801349749 wait complete. 2024-08-31T06:11:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.963+0000 7f08e8bde740 20 client.26560 trim_cache size 0 max 16384 2024-08-31T06:11:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:31.999+0000 7f08e8bde740 1 Processor -- start 2024-08-31T06:11:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.000+0000 7f08e8bde740 1 -- start start 2024-08-31T06:11:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.001+0000 7f08e8bde740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56031a0ff2f0 0x56031a4c8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.001+0000 7f08e8bde740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56031a1074d0 0x56031a4d3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.001+0000 7f08e8bde740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56031a1281b0 0x56031a4d4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.001+0000 7f08e8bde740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc1b81e0 con 0x56031a0ff2f0 2024-08-31T06:11:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.001+0000 7f08e8bde740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc1b5a80 con 0x56031a1281b0 2024-08-31T06:11:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.001+0000 7f08e8bde740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc1b38d0 con 0x56031a1074d0 2024-08-31T06:11:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.002+0000 7f08db400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56031a1281b0 0x56031a4d4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.002+0000 7f08db400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56031a1281b0 0x56031a4d4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:38530/0 (socket says 172.21.15.16:38530) 2024-08-31T06:11:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.002+0000 7f08db400640 1 -- 172.21.15.16:0/3022163392 learned_addr learned my addr 172.21.15.16:0/3022163392 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.002+0000 7f08daa00640 1 --2- 172.21.15.16:0/3022163392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56031a0ff2f0 0x56031a4c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.002+0000 7f08da000640 1 --2- 172.21.15.16:0/3022163392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56031a1074d0 0x56031a4d3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.003+0000 7f08da000640 1 -- 172.21.15.16:0/3022163392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56031a1281b0 msgr2=0x56031a4d4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.003+0000 7f08da000640 1 --2- 172.21.15.16:0/3022163392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56031a1281b0 0x56031a4d4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.003+0000 7f08da000640 1 -- 172.21.15.16:0/3022163392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56031a0ff2f0 msgr2=0x56031a4c8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.003+0000 7f08da000640 1 --2- 172.21.15.16:0/3022163392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56031a0ff2f0 0x56031a4c8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.003+0000 7f08da000640 1 -- 172.21.15.16:0/3022163392 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56031a4d5240 con 0x56031a1074d0 2024-08-31T06:11:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.003+0000 7f08daa00640 1 --2- 172.21.15.16:0/3022163392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56031a0ff2f0 0x56031a4c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:11:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.003+0000 7f08db400640 1 --2- 172.21.15.16:0/3022163392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56031a1281b0 0x56031a4d4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:11:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.004+0000 7f08da000640 1 --2- 172.21.15.16:0/3022163392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56031a1074d0 0x56031a4d3f30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f08cc005d00 tx=0x7f08cc00b470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b49f6a92c9a0cdf5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.004+0000 7f08c7400640 1 -- 172.21.15.16:0/3022163392 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc00e7d0 con 0x56031a1074d0 2024-08-31T06:11:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.004+0000 7f08c7400640 1 -- 172.21.15.16:0/3022163392 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08cc00ee20 con 0x56031a1074d0 2024-08-31T06:11:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.004+0000 7f08c7400640 1 -- 172.21.15.16:0/3022163392 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc010d00 con 0x56031a1074d0 2024-08-31T06:11:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.006+0000 7f08e8bde740 1 -- 172.21.15.16:0/3022163392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56031a1074d0 msgr2=0x56031a4d3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.006+0000 7f08e8bde740 1 --2- 172.21.15.16:0/3022163392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56031a1074d0 0x56031a4d3f30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f08cc005d00 tx=0x7f08cc00b470 comp rx=0 tx=0).stop 2024-08-31T06:11:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.006+0000 7f08e8bde740 1 -- 172.21.15.16:0/3022163392 shutdown_connections 2024-08-31T06:11:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.006+0000 7f08e8bde740 1 --2- 172.21.15.16:0/3022163392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56031a1074d0 0x56031a4d3f30 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.006+0000 7f08e8bde740 1 --2- 172.21.15.16:0/3022163392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56031a1281b0 0x56031a4d4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.006+0000 7f08e8bde740 1 --2- 172.21.15.16:0/3022163392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56031a0ff2f0 0x56031a4c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.006+0000 7f08e8bde740 1 -- 172.21.15.16:0/3022163392 >> 172.21.15.16:0/3022163392 conn(0x56031a11c300 msgr2=0x56031a4c7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.006+0000 7f08e8bde740 1 -- 172.21.15.16:0/3022163392 shutdown_connections 2024-08-31T06:11:32.009 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.006+0000 7f08e8bde740 1 -- 172.21.15.16:0/3022163392 wait complete. 2024-08-31T06:11:32.009 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.007+0000 7f08e8bde740 1 -- start start 2024-08-31T06:11:32.009 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.007+0000 7f08e8bde740 1 Processor -- start 2024-08-31T06:11:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.009+0000 7f08e8bde740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56031a0ff2f0 0x56031a4df760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.009+0000 7f08e8bde740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56031a1074d0 0x56031a4dcc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.009+0000 7f08daa00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56031a0ff2f0 0x56031a4df760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.009+0000 7f08daa00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56031a0ff2f0 0x56031a4df760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:59076/0 (socket says 172.21.15.16:59076) 2024-08-31T06:11:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.009+0000 7f08e8bde740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56031a1281b0 0x56031a4dd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.010+0000 7f08db400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56031a1281b0 0x56031a4dd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.010+0000 7f08db400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56031a1281b0 0x56031a4dd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55738/0 (socket says 172.21.15.16:55738) 2024-08-31T06:11:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.010+0000 7f08daa00640 1 -- 172.21.15.16:0/9798059 learned_addr learned my addr 172.21.15.16:0/9798059 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.010+0000 7f08e8bde740 1 -- 172.21.15.16:0/9798059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc1b3f40 con 0x56031a1281b0 2024-08-31T06:11:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.011+0000 7f08e8bde740 1 -- 172.21.15.16:0/9798059 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc1b53b0 con 0x56031a1074d0 2024-08-31T06:11:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.011+0000 7f08da000640 1 --2- 172.21.15.16:0/9798059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56031a1074d0 0x56031a4dcc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.011+0000 7f08e8bde740 1 -- 172.21.15.16:0/9798059 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc1b4610 con 0x56031a0ff2f0 2024-08-31T06:11:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.011+0000 7f08e8bde740 10 client.0 _ll_register_callbacks cb 0x7f08c8009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:11:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.012+0000 7f08daa00640 1 -- 172.21.15.16:0/9798059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56031a1074d0 msgr2=0x56031a4dcc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.012+0000 7f08daa00640 1 --2- 172.21.15.16:0/9798059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56031a1074d0 0x56031a4dcc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.012+0000 7f08daa00640 1 -- 172.21.15.16:0/9798059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56031a1281b0 msgr2=0x56031a4dd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.012+0000 7f08daa00640 1 --2- 172.21.15.16:0/9798059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56031a1281b0 0x56031a4dd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.012+0000 7f08daa00640 1 -- 172.21.15.16:0/9798059 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56031a4dd7d0 con 0x56031a0ff2f0 2024-08-31T06:11:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.013+0000 7f08daa00640 1 --2- 172.21.15.16:0/9798059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56031a0ff2f0 0x56031a4df760 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f08d4011010 tx=0x7f08d4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=461a047693cf3c42 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.013+0000 7f08c4c00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:11:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.014+0000 7f08c4c00640 1 -- 172.21.15.16:0/9798059 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d4034060 con 0x56031a0ff2f0 2024-08-31T06:11:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.014+0000 7f08c4c00640 1 -- 172.21.15.16:0/9798059 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f08d402f430 con 0x56031a0ff2f0 2024-08-31T06:11:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.014+0000 7f08e8bde740 1 -- 172.21.15.16:0/9798059 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08cc00ee20 con 0x56031a0ff2f0 2024-08-31T06:11:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.015+0000 7f08c4c00640 1 -- 172.21.15.16:0/9798059 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f08d402f630 con 0x56031a0ff2f0 2024-08-31T06:11:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.015+0000 7f08e8bde740 10 client.26566 fetch_fsmap learned FSMap version 34 2024-08-31T06:11:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.015+0000 7f08e8bde740 1 -- 172.21.15.16:0/9798059 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56031a5536a0 con 0x56031a0ff2f0 2024-08-31T06:11:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.015+0000 7f08c4c00640 1 -- 172.21.15.16:0/9798059 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08cc00ee20 con 0x56031a0ff2f0 2024-08-31T06:11:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.016+0000 7f08c4c00640 1 -- 172.21.15.16:0/9798059 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f08d401e8e0 con 0x56031a0ff2f0 2024-08-31T06:11:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.016+0000 7f08e8bde740 10 client.26566 fetch_fsmap finished waiting for FSMap version 34 2024-08-31T06:11:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.016+0000 7f08e8bde740 10 client.26566 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:11:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.016+0000 7f08e8bde740 20 client.26566 populate_metadata read hostname 'smithi016' 2024-08-31T06:11:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.016+0000 7f08e8bde740 1 --2- 172.21.15.16:0/9798059 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56031a0bc1c0 0x56031a555e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.016+0000 7f08e8bde740 4 client.26566 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:11:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.016+0000 7f08e8bde740 1 -- 172.21.15.16:0/9798059 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56031a43e960 con 0x56031a0bc1c0 2024-08-31T06:11:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.017+0000 7f08da000640 1 --2- 172.21.15.16:0/9798059 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56031a0bc1c0 0x56031a555e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.017+0000 7f08da000640 1 --2- 172.21.15.16:0/9798059 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56031a0bc1c0 0x56031a555e00 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=112a8dc33cd85846 server_cookie=7fc3a1818a3400e1 in_seq=0 out_seq=0 2024-08-31T06:11:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.017+0000 7f08c4c00640 10 client.26566 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:11:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.030+0000 7f08c4c00640 1 -- 172.21.15.16:0/9798059 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56031a43e960 con 0x56031a0bc1c0 2024-08-31T06:11:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.030+0000 7f08c4c00640 10 client.26566 handle_command_reply: tid=0 2024-08-31T06:11:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.031+0000 7f08e8bde740 1 client.26566 shutdown 2024-08-31T06:11:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.031+0000 7f08e8bde740 1 -- 172.21.15.16:0/9798059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56031a0ff2f0 msgr2=0x56031a4df760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.031+0000 7f08e8bde740 1 --2- 172.21.15.16:0/9798059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56031a0ff2f0 0x56031a4df760 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f08d4011010 tx=0x7f08d4004a40 comp rx=0 tx=0).stop 2024-08-31T06:11:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.031+0000 7f08e8bde740 1 -- 172.21.15.16:0/9798059 shutdown_connections 2024-08-31T06:11:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.031+0000 7f08e8bde740 1 --2- 172.21.15.16:0/9798059 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56031a0bc1c0 0x56031a555e00 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.031+0000 7f08e8bde740 1 --2- 172.21.15.16:0/9798059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56031a1281b0 0x56031a4dd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.031+0000 7f08e8bde740 1 --2- 172.21.15.16:0/9798059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56031a1074d0 0x56031a4dcc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.031+0000 7f08e8bde740 1 --2- 172.21.15.16:0/9798059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56031a0ff2f0 0x56031a4df760 secure :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f08d4011010 tx=0x7f08d4004a40 comp rx=0 tx=0).stop 2024-08-31T06:11:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.031+0000 7f08c4c00640 0 client.26566 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:11:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.031+0000 7f08e8bde740 1 -- 172.21.15.16:0/9798059 >> 172.21.15.16:0/9798059 conn(0x56031a11c300 msgr2=0x56031a4c8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.031+0000 7f08e8bde740 1 -- 172.21.15.16:0/9798059 shutdown_connections 2024-08-31T06:11:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.032+0000 7f08e8bde740 1 -- 172.21.15.16:0/9798059 wait complete. 2024-08-31T06:11:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.032+0000 7f08e8bde740 20 client.26566 trim_cache size 0 max 16384 2024-08-31T06:11:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:11:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:11:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:11:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:11:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.034+0000 7f08e1c00640 1 -- 172.21.15.16:0/2798424875 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f08c00b4d50 msgr2=0x7f08c00b7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.034+0000 7f08e1c00640 1 --2- 172.21.15.16:0/2798424875 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f08c00b4d50 0x7f08c00b7260 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x56031a4ca340 tx=0x7f08d40075a0 comp rx=0 tx=0).stop 2024-08-31T06:11:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.034+0000 7f08e1c00640 1 -- 172.21.15.16:0/2798424875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08dc107950 msgr2=0x7f08dc1a5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.034+0000 7f08e1c00640 1 --2- 172.21.15.16:0/2798424875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08dc107950 0x7f08dc1a5fa0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f08c8009230 tx=0x7f08c8003cd0 comp rx=0 tx=0).stop 2024-08-31T06:11:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.034+0000 7f08e1c00640 1 -- 172.21.15.16:0/2798424875 shutdown_connections 2024-08-31T06:11:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.034+0000 7f08e1c00640 1 --2- 172.21.15.16:0/2798424875 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f08c00b4d50 0x7f08c00b7260 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.034+0000 7f08e1c00640 1 --2- 172.21.15.16:0/2798424875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08dc107950 0x7f08dc1a5fa0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.034+0000 7f08e1c00640 1 --2- 172.21.15.16:0/2798424875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08dc102de0 0x7f08dc1a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.034+0000 7f08e1c00640 1 --2- 172.21.15.16:0/2798424875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08dc102490 0x7f08dc1a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.034+0000 7f08e1c00640 1 -- 172.21.15.16:0/2798424875 >> 172.21.15.16:0/2798424875 conn(0x7f08dc07a4e0 msgr2=0x7f08dc1052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.035+0000 7f08e1c00640 1 -- 172.21.15.16:0/2798424875 shutdown_connections 2024-08-31T06:11:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:32.035+0000 7f08e1c00640 1 -- 172.21.15.16:0/2798424875 wait complete. 2024-08-31T06:11:32.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:32.289+0000 7f019d800640 1 -- 172.21.15.16:0/225483978 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0198106820 con 0x7f019810d9c0 2024-08-31T06:11:32.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:32.290+0000 7f0194c00640 1 -- 172.21.15.16:0/225483978 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11918 (secure 0 0 0) 0x7f0198106820 con 0x7f019810d9c0 2024-08-31T06:11:32.292 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:11:32.292 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":34,"btime":"2024-08-31T06:11:28:678375+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:11:28.338576+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24712,24724,24737]},"id":1}]} 2024-08-31T06:11:32.293 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 34 2024-08-31T06:11:32.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:32.294+0000 7f019d800640 1 -- 172.21.15.16:0/225483978 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f017c0b4ca0 msgr2=0x7f017c0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:32.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:32.294+0000 7f019d800640 1 --2- 172.21.15.16:0/225483978 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f017c0b4ca0 0x7f017c0b71b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0198108f90 tx=0x7f018800a380 comp rx=0 tx=0).stop 2024-08-31T06:11:32.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:32.294+0000 7f019d800640 1 -- 172.21.15.16:0/225483978 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f019810d9c0 msgr2=0x7f0198105c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:32.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:32.294+0000 7f019d800640 1 --2- 172.21.15.16:0/225483978 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f019810d9c0 0x7f0198105c40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f019000ca30 tx=0x7f019000cef0 comp rx=0 tx=0).stop 2024-08-31T06:11:32.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:32.295+0000 7f019d800640 1 -- 172.21.15.16:0/225483978 shutdown_connections 2024-08-31T06:11:32.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:32.295+0000 7f019d800640 1 --2- 172.21.15.16:0/225483978 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f017c0b4ca0 0x7f017c0b71b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:32.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:32.295+0000 7f019d800640 1 --2- 172.21.15.16:0/225483978 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f019810d9c0 0x7f0198105c40 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:32.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:32.295+0000 7f019d800640 1 --2- 172.21.15.16:0/225483978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0198104e40 0x7f0198105520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:32.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:32.295+0000 7f019d800640 1 --2- 172.21.15.16:0/225483978 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0198104410 0x7f0198109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:32.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:32.295+0000 7f019d800640 1 -- 172.21.15.16:0/225483978 >> 172.21.15.16:0/225483978 conn(0x7f01980fe060 msgr2=0x7f0198102230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:32.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:32.296+0000 7f019d800640 1 -- 172.21.15.16:0/225483978 shutdown_connections 2024-08-31T06:11:32.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:32.296+0000 7f019d800640 1 -- 172.21.15.16:0/225483978 wait complete. 2024-08-31T06:11:32.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:32 smithi184 ceph-mon[22331]: pgmap v493: 97 pgs: 97 active+clean; 2.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 1.20k op/s 2024-08-31T06:11:32.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:32 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/225483978' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:11:33.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:32 smithi155 ceph-mon[21742]: pgmap v493: 97 pgs: 97 active+clean; 2.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 1.20k op/s 2024-08-31T06:11:33.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:32 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/225483978' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:11:33.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:32 smithi016 ceph-mon[17194]: pgmap v493: 97 pgs: 97 active+clean; 2.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 1.20k op/s 2024-08-31T06:11:33.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:32 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/225483978' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:11:33.247 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T06:11:33.893 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:11:34.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.729+0000 7f047da00640 1 -- 172.21.15.16:0/1097577704 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04781045e0 msgr2=0x7f04781049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:34.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.730+0000 7f047da00640 1 --2- 172.21.15.16:0/1097577704 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04781045e0 0x7f04781049f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0468009900 tx=0x7f046802a410 comp rx=0 tx=0).stop 2024-08-31T06:11:34.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.730+0000 7f047da00640 1 -- 172.21.15.16:0/1097577704 shutdown_connections 2024-08-31T06:11:34.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.730+0000 7f047da00640 1 --2- 172.21.15.16:0/1097577704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0478105010 0x7f047810d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.730+0000 7f047da00640 1 --2- 172.21.15.16:0/1097577704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f047810db90 0x7f04781100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.730+0000 7f047da00640 1 --2- 172.21.15.16:0/1097577704 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04781045e0 0x7f04781049f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.730+0000 7f047da00640 1 -- 172.21.15.16:0/1097577704 >> 172.21.15.16:0/1097577704 conn(0x7f04780794d0 msgr2=0x7f04780798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.730+0000 7f047da00640 1 -- 172.21.15.16:0/1097577704 shutdown_connections 2024-08-31T06:11:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.730+0000 7f047da00640 1 -- 172.21.15.16:0/1097577704 wait complete. 2024-08-31T06:11:34.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.732+0000 7f047da00640 1 Processor -- start 2024-08-31T06:11:34.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.732+0000 7f047da00640 1 -- start start 2024-08-31T06:11:34.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.733+0000 7f047da00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04781045e0 0x7f0478074c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:34.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.733+0000 7f047da00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0478105010 0x7f047806df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:34.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.733+0000 7f047da00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f047810db90 0x7f047806e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:34.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.733+0000 7f047da00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0478113940 con 0x7f047810db90 2024-08-31T06:11:34.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.733+0000 7f047da00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f04781137c0 con 0x7f0478105010 2024-08-31T06:11:34.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.733+0000 7f047da00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0478113ac0 con 0x7f04781045e0 2024-08-31T06:11:34.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.733+0000 7f0476a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04781045e0 0x7f0478074c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:34.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.733+0000 7f0476a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04781045e0 0x7f0478074c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:59090/0 (socket says 172.21.15.16:59090) 2024-08-31T06:11:34.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.733+0000 7f0476a00640 1 -- 172.21.15.16:0/806954691 learned_addr learned my addr 172.21.15.16:0/806954691 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:34.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.733+0000 7f0476000640 1 --2- 172.21.15.16:0/806954691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0478105010 0x7f047806df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:34.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.734+0000 7f0477400640 1 --2- 172.21.15.16:0/806954691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f047810db90 0x7f047806e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:34.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.734+0000 7f0476000640 1 -- 172.21.15.16:0/806954691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04781045e0 msgr2=0x7f0478074c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:34.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.734+0000 7f0476000640 1 --2- 172.21.15.16:0/806954691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04781045e0 0x7f0478074c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:34.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.734+0000 7f0476000640 1 -- 172.21.15.16:0/806954691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f047810db90 msgr2=0x7f047806e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:34.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.734+0000 7f0476000640 1 --2- 172.21.15.16:0/806954691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f047810db90 0x7f047806e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:34.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.734+0000 7f0476000640 1 -- 172.21.15.16:0/806954691 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f047806ecf0 con 0x7f0478105010 2024-08-31T06:11:34.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.734+0000 7f0476000640 1 --2- 172.21.15.16:0/806954691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0478105010 0x7f047806df10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f047806aad0 tx=0x7f047000de20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:34.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.734+0000 7f046f400640 1 -- 172.21.15.16:0/806954691 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047000d0d0 con 0x7f0478105010 2024-08-31T06:11:34.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.735+0000 7f046f400640 1 -- 172.21.15.16:0/806954691 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f047000bb70 con 0x7f0478105010 2024-08-31T06:11:34.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.735+0000 7f046f400640 1 -- 172.21.15.16:0/806954691 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0470018760 con 0x7f0478105010 2024-08-31T06:11:34.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.735+0000 7f047da00640 1 -- 172.21.15.16:0/806954691 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0478073910 con 0x7f0478105010 2024-08-31T06:11:34.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.736+0000 7f047da00640 1 -- 172.21.15.16:0/806954691 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0478073e80 con 0x7f0478105010 2024-08-31T06:11:34.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.741+0000 7f046f400640 1 -- 172.21.15.16:0/806954691 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f047000b6c0 con 0x7f0478105010 2024-08-31T06:11:34.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.742+0000 7f0484622740 1 Processor -- start 2024-08-31T06:11:34.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.742+0000 7f0484622740 1 -- start start 2024-08-31T06:11:34.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.743+0000 7f0484622740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3120db070 0x55d312480990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:34.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.743+0000 7f0484622740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3120b21b0 0x55d312480ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:34.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.743+0000 7f0484622740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3120ba390 0x55d312481410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:34.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.743+0000 7f0484622740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d3123f77b0 con 0x55d3120ba390 2024-08-31T06:11:34.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.743+0000 7f0484622740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d3123e6d40 con 0x55d3120b21b0 2024-08-31T06:11:34.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.743+0000 7f0484622740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d3123eacf0 con 0x55d3120db070 2024-08-31T06:11:34.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.743+0000 7f046f400640 1 --2- 172.21.15.16:0/806954691 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f045c0b4d50 0x7f045c0b7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:34.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.744+0000 7f0476000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3120ba390 0x55d312481410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:34.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.744+0000 7f0476000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3120ba390 0x55d312481410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55764/0 (socket says 172.21.15.16:55764) 2024-08-31T06:11:34.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.744+0000 7f0476000640 1 -- 172.21.15.16:0/352808733 learned_addr learned my addr 172.21.15.16:0/352808733 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:34.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.744+0000 7f0476a00640 1 --2- 172.21.15.16:0/352808733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3120b21b0 0x55d312480ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:34.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.744+0000 7f046f400640 1 -- 172.21.15.16:0/806954691 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f04700e8cb0 con 0x7f0478105010 2024-08-31T06:11:34.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.745+0000 7f0476a00640 1 -- 172.21.15.16:0/352808733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3120db070 msgr2=0x55d312480990 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T06:11:34.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.745+0000 7f0476a00640 1 --2- 172.21.15.16:0/352808733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3120db070 0x55d312480990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:34.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.745+0000 7f0476a00640 1 -- 172.21.15.16:0/352808733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3120ba390 msgr2=0x55d312481410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:34.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.745+0000 7f0476a00640 1 --2- 172.21.15.16:0/352808733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3120ba390 0x55d312481410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:34.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.745+0000 7f0476a00640 1 -- 172.21.15.16:0/352808733 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d312146650 con 0x55d3120b21b0 2024-08-31T06:11:34.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.745+0000 7f0476000640 1 --2- 172.21.15.16:0/352808733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3120ba390 0x55d312481410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:11:34.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.745+0000 7f0476a00640 1 --2- 172.21.15.16:0/352808733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3120b21b0 0x55d312480ed0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0468006010 tx=0x7f0468004410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d596b88f5e06338f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:34.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.745+0000 7f0463e00640 1 -- 172.21.15.16:0/352808733 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0468040070 con 0x55d3120b21b0 2024-08-31T06:11:34.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.746+0000 7f0463e00640 1 -- 172.21.15.16:0/352808733 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04680047a0 con 0x55d3120b21b0 2024-08-31T06:11:34.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.746+0000 7f0463e00640 1 -- 172.21.15.16:0/352808733 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0468035040 con 0x55d3120b21b0 2024-08-31T06:11:34.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.746+0000 7f0477400640 1 --2- 172.21.15.16:0/806954691 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f045c0b4d50 0x7f045c0b7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:34.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.747+0000 7f0484622740 1 -- 172.21.15.16:0/352808733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3120b21b0 msgr2=0x55d312480ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:34.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.747+0000 7f0484622740 1 --2- 172.21.15.16:0/352808733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3120b21b0 0x55d312480ed0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0468006010 tx=0x7f0468004410 comp rx=0 tx=0).stop 2024-08-31T06:11:34.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.748+0000 7f0484622740 1 -- 172.21.15.16:0/352808733 shutdown_connections 2024-08-31T06:11:34.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.748+0000 7f0484622740 1 --2- 172.21.15.16:0/352808733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3120ba390 0x55d312481410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:34.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.748+0000 7f0484622740 1 --2- 172.21.15.16:0/352808733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3120b21b0 0x55d312480ed0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:34.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.748+0000 7f0484622740 1 --2- 172.21.15.16:0/352808733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3120db070 0x55d312480990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:34.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.748+0000 7f0484622740 1 -- 172.21.15.16:0/352808733 >> 172.21.15.16:0/352808733 conn(0x55d3120cf460 msgr2=0x55d31247e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:34.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.750+0000 7f0477400640 1 --2- 172.21.15.16:0/806954691 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f045c0b4d50 0x7f045c0b7260 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f046400b780 tx=0x7f0464008040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:34.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.750+0000 7f0484622740 1 -- 172.21.15.16:0/352808733 shutdown_connections 2024-08-31T06:11:34.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.750+0000 7f0484622740 1 -- 172.21.15.16:0/352808733 wait complete. 2024-08-31T06:11:34.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.751+0000 7f0484622740 1 -- start start 2024-08-31T06:11:34.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.751+0000 7f0484622740 1 Processor -- start 2024-08-31T06:11:34.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.752+0000 7f0484622740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3120b21b0 0x55d312494d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:34.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.752+0000 7f0484622740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3120ba390 0x55d312495270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:34.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.752+0000 7f0484622740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3120db070 0x55d3124a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:34.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.752+0000 7f0484622740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d312249700 con 0x55d3120b21b0 2024-08-31T06:11:34.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.752+0000 7f0484622740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d31236dcf0 con 0x55d3120db070 2024-08-31T06:11:34.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.752+0000 7f0484622740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d312497840 con 0x55d3120ba390 2024-08-31T06:11:34.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.752+0000 7f0484622740 10 client.0 _ll_register_callbacks cb 0x55d312393470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:11:34.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.753+0000 7f0476a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3120b21b0 0x55d312494d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:34.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.753+0000 7f0476a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3120b21b0 0x55d312494d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55778/0 (socket says 172.21.15.16:55778) 2024-08-31T06:11:34.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.753+0000 7f0476a00640 1 -- 172.21.15.16:0/225207661 learned_addr learned my addr 172.21.15.16:0/225207661 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:34.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.753+0000 7f0476000640 1 --2- 172.21.15.16:0/225207661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3120ba390 0x55d312495270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:34.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.753+0000 7f0476a00640 1 -- 172.21.15.16:0/225207661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3120ba390 msgr2=0x55d312495270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:34.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.753+0000 7f0476a00640 1 --2- 172.21.15.16:0/225207661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3120ba390 0x55d312495270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:34.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.753+0000 7f0476a00640 1 -- 172.21.15.16:0/225207661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3120db070 msgr2=0x55d3124a0b70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T06:11:34.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.754+0000 7f0477400640 1 --2- 172.21.15.16:0/225207661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3120db070 0x55d3124a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:34.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.754+0000 7f0476a00640 1 --2- 172.21.15.16:0/225207661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3120db070 0x55d3124a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:34.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.754+0000 7f0476a00640 1 -- 172.21.15.16:0/225207661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3123ae8b0 con 0x55d3120b21b0 2024-08-31T06:11:34.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.755+0000 7f0477400640 1 --2- 172.21.15.16:0/225207661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3120db070 0x55d3124a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:11:34.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.756+0000 7f0476a00640 1 --2- 172.21.15.16:0/225207661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3120b21b0 0x55d312494d30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f046802a450 tx=0x7f0468004b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f72e11808649a053 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:34.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.756+0000 7f0460c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:11:34.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.757+0000 7f0460c00640 1 -- 172.21.15.16:0/225207661 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0468035070 con 0x55d3120b21b0 2024-08-31T06:11:34.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.757+0000 7f0460c00640 1 -- 172.21.15.16:0/225207661 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f04680051b0 con 0x55d3120b21b0 2024-08-31T06:11:34.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.757+0000 7f0484622740 1 -- 172.21.15.16:0/225207661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04680047a0 con 0x55d3120b21b0 2024-08-31T06:11:34.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.757+0000 7f0460c00640 1 -- 172.21.15.16:0/225207661 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0468040040 con 0x55d3120b21b0 2024-08-31T06:11:34.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.758+0000 7f0484622740 10 client.16431 fetch_fsmap learned FSMap version 34 2024-08-31T06:11:34.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.758+0000 7f0484622740 1 -- 172.21.15.16:0/225207661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d3124a3860 con 0x55d3120b21b0 2024-08-31T06:11:34.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.758+0000 7f0460c00640 1 -- 172.21.15.16:0/225207661 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04680047a0 con 0x55d3120b21b0 2024-08-31T06:11:34.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.759+0000 7f0460c00640 1 -- 172.21.15.16:0/225207661 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f04680314c0 con 0x55d3120b21b0 2024-08-31T06:11:34.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.759+0000 7f0484622740 10 client.16431 fetch_fsmap finished waiting for FSMap version 34 2024-08-31T06:11:34.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.759+0000 7f0484622740 10 client.16431 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T06:11:34.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.759+0000 7f0484622740 20 client.16431 populate_metadata read hostname 'smithi016' 2024-08-31T06:11:34.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.759+0000 7f0484622740 1 --2- 172.21.15.16:0/225207661 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55d3120999f0 0x55d3124a6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:34.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.759+0000 7f0484622740 4 client.16431 mds_command: new command op to 24712 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:11:34.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.759+0000 7f0484622740 1 -- 172.21.15.16:0/225207661 --> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3123f9700 con 0x55d3120999f0 2024-08-31T06:11:34.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.760+0000 7f0476000640 1 --2- 172.21.15.16:0/225207661 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55d3120999f0 0x55d3124a6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:34.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.761+0000 7f0476000640 1 --2- 172.21.15.16:0/225207661 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55d3120999f0 0x55d3124a6bc0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f1b637031cd639f0 server_cookie=c258e334a9f8e284 in_seq=0 out_seq=0 2024-08-31T06:11:34.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.761+0000 7f0460c00640 10 client.16431 ms_handle_connect on v2:172.21.15.184:6838/4011051479 2024-08-31T06:11:34.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.762+0000 7f0460c00640 1 -- 172.21.15.16:0/225207661 <== mds.2 v2:172.21.15.184:6838/4011051479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d3123f9700 con 0x55d3120999f0 2024-08-31T06:11:34.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.762+0000 7f0460c00640 10 client.16431 handle_command_reply: tid=0 2024-08-31T06:11:34.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.762+0000 7f0484622740 1 client.16431 shutdown 2024-08-31T06:11:34.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.763+0000 7f0484622740 1 -- 172.21.15.16:0/225207661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3120b21b0 msgr2=0x55d312494d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:34.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.763+0000 7f0484622740 1 --2- 172.21.15.16:0/225207661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3120b21b0 0x55d312494d30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f046802a450 tx=0x7f0468004b80 comp rx=0 tx=0).stop 2024-08-31T06:11:34.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.763+0000 7f0484622740 1 -- 172.21.15.16:0/225207661 shutdown_connections 2024-08-31T06:11:34.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.763+0000 7f0484622740 1 --2- 172.21.15.16:0/225207661 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55d3120999f0 0x55d3124a6bc0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:34.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.764+0000 7f0484622740 1 --2- 172.21.15.16:0/225207661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3120ba390 0x55d312495270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:34.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.764+0000 7f0484622740 1 --2- 172.21.15.16:0/225207661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3120db070 0x55d3124a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:34.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.764+0000 7f0484622740 1 --2- 172.21.15.16:0/225207661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3120b21b0 0x55d312494d30 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:34.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.764+0000 7f0484622740 1 -- 172.21.15.16:0/225207661 >> 172.21.15.16:0/225207661 conn(0x55d3120cf460 msgr2=0x55d31247ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:34.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.764+0000 7f0460c00640 0 client.16431 ms_handle_reset on v2:172.21.15.184:6838/4011051479 2024-08-31T06:11:34.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.765+0000 7f0484622740 1 -- 172.21.15.16:0/225207661 shutdown_connections 2024-08-31T06:11:34.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.765+0000 7f0484622740 1 -- 172.21.15.16:0/225207661 wait complete. 2024-08-31T06:11:34.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.765+0000 7f0484622740 20 client.16431 trim_cache size 0 max 16384 2024-08-31T06:11:34.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.786+0000 7f0484622740 1 Processor -- start 2024-08-31T06:11:34.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.786+0000 7f0484622740 1 -- start start 2024-08-31T06:11:34.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.786+0000 7f0484622740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3120b21b0 0x55d31247d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:34.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.786+0000 7f0484622740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3120ba390 0x55d3124a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:34.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.786+0000 7f0484622740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3120db070 0x55d3124a1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:34.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.786+0000 7f0484622740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f04781b81a0 con 0x55d3120b21b0 2024-08-31T06:11:34.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.786+0000 7f0484622740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f04781b5a40 con 0x55d3120ba390 2024-08-31T06:11:34.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.786+0000 7f0484622740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f04781b3940 con 0x55d3120db070 2024-08-31T06:11:34.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.787+0000 7f0476000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3120ba390 0x55d3124a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:34.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.787+0000 7f0476000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3120ba390 0x55d3124a0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:38590/0 (socket says 172.21.15.16:38590) 2024-08-31T06:11:34.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.787+0000 7f0476000640 1 -- 172.21.15.16:0/1195945568 learned_addr learned my addr 172.21.15.16:0/1195945568 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:34.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.787+0000 7f0477400640 1 --2- 172.21.15.16:0/1195945568 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3120db070 0x55d3124a1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:34.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.788+0000 7f0476a00640 1 --2- 172.21.15.16:0/1195945568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3120b21b0 0x55d31247d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:34.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.788+0000 7f0476000640 1 -- 172.21.15.16:0/1195945568 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3120db070 msgr2=0x55d3124a1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:34.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.788+0000 7f0476000640 1 --2- 172.21.15.16:0/1195945568 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3120db070 0x55d3124a1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:34.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.788+0000 7f0476000640 1 -- 172.21.15.16:0/1195945568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3120b21b0 msgr2=0x55d31247d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:34.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.789+0000 7f0476000640 1 --2- 172.21.15.16:0/1195945568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3120b21b0 0x55d31247d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:34.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.789+0000 7f0476000640 1 -- 172.21.15.16:0/1195945568 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d312495df0 con 0x55d3120ba390 2024-08-31T06:11:34.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.789+0000 7f0477400640 1 --2- 172.21.15.16:0/1195945568 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3120db070 0x55d3124a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:11:34.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.789+0000 7f0476a00640 1 --2- 172.21.15.16:0/1195945568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3120b21b0 0x55d31247d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:11:34.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.789+0000 7f0476000640 1 --2- 172.21.15.16:0/1195945568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3120ba390 0x55d3124a0ae0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f047000d6d0 tx=0x7f0470079440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e389d048cb6f09fd server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:34.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.790+0000 7f0463400640 1 -- 172.21.15.16:0/1195945568 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0470026040 con 0x55d3120ba390 2024-08-31T06:11:34.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.790+0000 7f0463400640 1 -- 172.21.15.16:0/1195945568 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04700afaa0 con 0x55d3120ba390 2024-08-31T06:11:34.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.790+0000 7f0463400640 1 -- 172.21.15.16:0/1195945568 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047000bd20 con 0x55d3120ba390 2024-08-31T06:11:34.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.791+0000 7f0484622740 1 -- 172.21.15.16:0/1195945568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3120ba390 msgr2=0x55d3124a0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:34.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.791+0000 7f0484622740 1 --2- 172.21.15.16:0/1195945568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3120ba390 0x55d3124a0ae0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f047000d6d0 tx=0x7f0470079440 comp rx=0 tx=0).stop 2024-08-31T06:11:34.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.792+0000 7f0484622740 1 -- 172.21.15.16:0/1195945568 shutdown_connections 2024-08-31T06:11:34.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.792+0000 7f0484622740 1 --2- 172.21.15.16:0/1195945568 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3120db070 0x55d3124a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:34.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.792+0000 7f0484622740 1 --2- 172.21.15.16:0/1195945568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3120ba390 0x55d3124a0ae0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:34.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.792+0000 7f0484622740 1 --2- 172.21.15.16:0/1195945568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3120b21b0 0x55d31247d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:34.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.792+0000 7f0484622740 1 -- 172.21.15.16:0/1195945568 >> 172.21.15.16:0/1195945568 conn(0x55d3120cf460 msgr2=0x55d31247ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:34.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.792+0000 7f0484622740 1 -- 172.21.15.16:0/1195945568 shutdown_connections 2024-08-31T06:11:34.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.792+0000 7f0484622740 1 -- 172.21.15.16:0/1195945568 wait complete. 2024-08-31T06:11:34.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.793+0000 7f0484622740 1 -- start start 2024-08-31T06:11:34.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.793+0000 7f0484622740 1 Processor -- start 2024-08-31T06:11:34.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.795+0000 7f0484622740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3120b21b0 0x55d3124a8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:34.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.795+0000 7f0484622740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3120ba390 0x55d3124a8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:34.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.795+0000 7f0484622740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3120db070 0x55d3124a8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:34.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.796+0000 7f0484622740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f04781b3f00 con 0x55d3120b21b0 2024-08-31T06:11:34.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.796+0000 7f0484622740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f04781b5370 con 0x55d3120ba390 2024-08-31T06:11:34.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.796+0000 7f0484622740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f04781b45d0 con 0x55d3120db070 2024-08-31T06:11:34.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.796+0000 7f0476a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3120b21b0 0x55d3124a8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:34.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.796+0000 7f0476000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3120ba390 0x55d3124a8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:34.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.796+0000 7f0476000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3120ba390 0x55d3124a8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:38606/0 (socket says 172.21.15.16:38606) 2024-08-31T06:11:34.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.796+0000 7f0476000640 1 -- 172.21.15.16:0/3083602102 learned_addr learned my addr 172.21.15.16:0/3083602102 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:34.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.796+0000 7f0484622740 10 client.0 _ll_register_callbacks cb 0x7f0468004180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:11:34.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.796+0000 7f0476a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3120b21b0 0x55d3124a8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55802/0 (socket says 172.21.15.16:55802) 2024-08-31T06:11:34.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.796+0000 7f0477400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3120db070 0x55d3124a8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:34.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.797+0000 7f0476000640 1 -- 172.21.15.16:0/3083602102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3120db070 msgr2=0x55d3124a8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:34.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.797+0000 7f0476000640 1 --2- 172.21.15.16:0/3083602102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3120db070 0x55d3124a8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:34.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.797+0000 7f0476000640 1 -- 172.21.15.16:0/3083602102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3120b21b0 msgr2=0x55d3124a8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:34.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.797+0000 7f0476000640 1 --2- 172.21.15.16:0/3083602102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3120b21b0 0x55d3124a8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:34.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.797+0000 7f0476000640 1 -- 172.21.15.16:0/3083602102 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d312510490 con 0x55d3120ba390 2024-08-31T06:11:34.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.797+0000 7f0476000640 1 --2- 172.21.15.16:0/3083602102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3120ba390 0x55d3124a8770 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f047000d3e0 tx=0x7f047000bf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bab37ddbd14cc24b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:34.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.797+0000 7f0460c00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:11:34.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.798+0000 7f0460c00640 1 -- 172.21.15.16:0/3083602102 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04700af3a0 con 0x55d3120ba390 2024-08-31T06:11:34.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.798+0000 7f0460c00640 1 -- 172.21.15.16:0/3083602102 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f04700ab630 con 0x55d3120ba390 2024-08-31T06:11:34.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.798+0000 7f0484622740 1 -- 172.21.15.16:0/3083602102 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04700afaa0 con 0x55d3120ba390 2024-08-31T06:11:34.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.798+0000 7f0460c00640 1 -- 172.21.15.16:0/3083602102 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f04700b86c0 con 0x55d3120ba390 2024-08-31T06:11:34.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.798+0000 7f0484622740 10 client.26579 fetch_fsmap learned FSMap version 34 2024-08-31T06:11:34.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.798+0000 7f0484622740 1 -- 172.21.15.16:0/3083602102 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d3125116f0 con 0x55d3120ba390 2024-08-31T06:11:34.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.799+0000 7f0460c00640 1 -- 172.21.15.16:0/3083602102 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04700afaa0 con 0x55d3120ba390 2024-08-31T06:11:34.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.799+0000 7f0460c00640 1 -- 172.21.15.16:0/3083602102 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f04700b8230 con 0x55d3120ba390 2024-08-31T06:11:34.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.799+0000 7f0484622740 10 client.26579 fetch_fsmap finished waiting for FSMap version 34 2024-08-31T06:11:34.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.799+0000 7f0484622740 10 client.26579 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T06:11:34.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.799+0000 7f0484622740 20 client.26579 populate_metadata read hostname 'smithi016' 2024-08-31T06:11:34.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.799+0000 7f0484622740 1 --2- 172.21.15.16:0/3083602102 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55d31206eda0 0x55d312513d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:34.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.800+0000 7f0484622740 4 client.26579 mds_command: new command op to 24712 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:11:34.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.800+0000 7f0484622740 1 -- 172.21.15.16:0/3083602102 --> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d3123f51a0 con 0x55d31206eda0 2024-08-31T06:11:34.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.801+0000 7f0476a00640 1 --2- 172.21.15.16:0/3083602102 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55d31206eda0 0x55d312513d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:34.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.806+0000 7f0476a00640 1 --2- 172.21.15.16:0/3083602102 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55d31206eda0 0x55d312513d80 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d180e30f8375f85e server_cookie=671db30e8137dfb5 in_seq=0 out_seq=0 2024-08-31T06:11:34.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.806+0000 7f0460c00640 10 client.26579 ms_handle_connect on v2:172.21.15.184:6838/4011051479 2024-08-31T06:11:34.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.809+0000 7f0460c00640 1 -- 172.21.15.16:0/3083602102 <== mds.2 v2:172.21.15.184:6838/4011051479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d3123f51a0 con 0x55d31206eda0 2024-08-31T06:11:34.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.809+0000 7f0460c00640 10 client.26579 handle_command_reply: tid=0 2024-08-31T06:11:34.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.809+0000 7f0484622740 1 client.26579 shutdown 2024-08-31T06:11:34.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.810+0000 7f0484622740 1 -- 172.21.15.16:0/3083602102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3120ba390 msgr2=0x55d3124a8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:34.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.810+0000 7f0484622740 1 --2- 172.21.15.16:0/3083602102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3120ba390 0x55d3124a8770 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f047000d3e0 tx=0x7f047000bf50 comp rx=0 tx=0).stop 2024-08-31T06:11:34.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.810+0000 7f0484622740 1 -- 172.21.15.16:0/3083602102 shutdown_connections 2024-08-31T06:11:34.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.810+0000 7f0484622740 1 --2- 172.21.15.16:0/3083602102 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55d31206eda0 0x55d312513d80 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:34.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.810+0000 7f0484622740 1 --2- 172.21.15.16:0/3083602102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3120db070 0x55d3124a8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:34.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.810+0000 7f0484622740 1 --2- 172.21.15.16:0/3083602102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3120ba390 0x55d3124a8770 secure :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f047000d3e0 tx=0x7f047000bf50 comp rx=0 tx=0).stop 2024-08-31T06:11:34.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.810+0000 7f0484622740 1 --2- 172.21.15.16:0/3083602102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3120b21b0 0x55d3124a8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:34.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.810+0000 7f0484622740 1 -- 172.21.15.16:0/3083602102 >> 172.21.15.16:0/3083602102 conn(0x55d3120cf460 msgr2=0x55d3124819c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:34.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.811+0000 7f0460c00640 0 client.26579 ms_handle_reset on v2:172.21.15.184:6838/4011051479 2024-08-31T06:11:34.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.811+0000 7f0484622740 1 -- 172.21.15.16:0/3083602102 shutdown_connections 2024-08-31T06:11:34.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.812+0000 7f0484622740 1 -- 172.21.15.16:0/3083602102 wait complete. 2024-08-31T06:11:34.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.813+0000 7f0484622740 20 client.26579 trim_cache size 0 max 16384 2024-08-31T06:11:34.816 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:11:34.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.815+0000 7f047da00640 1 -- 172.21.15.16:0/806954691 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f045c0b4d50 msgr2=0x7f045c0b7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:34.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.815+0000 7f047da00640 1 --2- 172.21.15.16:0/806954691 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f045c0b4d50 0x7f045c0b7260 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f046400b780 tx=0x7f0464008040 comp rx=0 tx=0).stop 2024-08-31T06:11:34.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.815+0000 7f047da00640 1 -- 172.21.15.16:0/806954691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0478105010 msgr2=0x7f047806df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:34.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.815+0000 7f047da00640 1 --2- 172.21.15.16:0/806954691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0478105010 0x7f047806df10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f047806aad0 tx=0x7f047000de20 comp rx=0 tx=0).stop 2024-08-31T06:11:34.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.816+0000 7f047da00640 1 -- 172.21.15.16:0/806954691 shutdown_connections 2024-08-31T06:11:34.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.816+0000 7f047da00640 1 --2- 172.21.15.16:0/806954691 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f045c0b4d50 0x7f045c0b7260 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:34.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.816+0000 7f047da00640 1 --2- 172.21.15.16:0/806954691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f047810db90 0x7f047806e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:34.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.816+0000 7f047da00640 1 --2- 172.21.15.16:0/806954691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0478105010 0x7f047806df10 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:34.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.816+0000 7f047da00640 1 --2- 172.21.15.16:0/806954691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04781045e0 0x7f0478074c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:34.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.816+0000 7f047da00640 1 -- 172.21.15.16:0/806954691 >> 172.21.15.16:0/806954691 conn(0x7f04780794d0 msgr2=0x7f0478100580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:34.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.817+0000 7f047da00640 1 -- 172.21.15.16:0/806954691 shutdown_connections 2024-08-31T06:11:34.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:34.818+0000 7f047da00640 1 -- 172.21.15.16:0/806954691 wait complete. 2024-08-31T06:11:34.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:34 smithi184 ceph-mon[22331]: pgmap v494: 97 pgs: 97 active+clean; 2.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 MiB/s wr, 1.01k op/s 2024-08-31T06:11:35.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:34 smithi155 ceph-mon[21742]: pgmap v494: 97 pgs: 97 active+clean; 2.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 MiB/s wr, 1.01k op/s 2024-08-31T06:11:35.166 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:34 smithi016 ceph-mon[17194]: pgmap v494: 97 pgs: 97 active+clean; 2.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 MiB/s wr, 1.01k op/s 2024-08-31T06:11:35.853 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T06:11:36.479 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:11:36.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 250.0 s, 2604.8 tps, lat 38.331 ms stddev 45.035 2024-08-31T06:11:36.898 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:36 smithi016 ceph-mon[17194]: pgmap v495: 97 pgs: 97 active+clean; 2.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 8.5 MiB/s wr, 751 op/s 2024-08-31T06:11:36.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:36 smithi184 ceph-mon[22331]: pgmap v495: 97 pgs: 97 active+clean; 2.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 8.5 MiB/s wr, 751 op/s 2024-08-31T06:11:37.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:36 smithi155 ceph-mon[21742]: pgmap v495: 97 pgs: 97 active+clean; 2.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 8.5 MiB/s wr, 751 op/s 2024-08-31T06:11:37.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.329+0000 7f4105600640 1 -- 172.21.15.16:0/1432367164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4100109240 msgr2=0x7f4100109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.329+0000 7f4105600640 1 --2- 172.21.15.16:0/1432367164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4100109240 0x7f4100109650 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f40ec009900 tx=0x7f40ec02a410 comp rx=0 tx=0).stop 2024-08-31T06:11:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.330+0000 7f4105600640 1 -- 172.21.15.16:0/1432367164 shutdown_connections 2024-08-31T06:11:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.330+0000 7f4105600640 1 --2- 172.21.15.16:0/1432367164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4100109b90 0x7f41001103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.330+0000 7f4105600640 1 --2- 172.21.15.16:0/1432367164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4100109240 0x7f4100109650 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.330+0000 7f4105600640 1 --2- 172.21.15.16:0/1432367164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4100103900 0x7f4100103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:37.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.330+0000 7f4105600640 1 -- 172.21.15.16:0/1432367164 >> 172.21.15.16:0/1432367164 conn(0x7f410007b5f0 msgr2=0x7f410007ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:37.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.330+0000 7f4105600640 1 -- 172.21.15.16:0/1432367164 shutdown_connections 2024-08-31T06:11:37.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.330+0000 7f4105600640 1 -- 172.21.15.16:0/1432367164 wait complete. 2024-08-31T06:11:37.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.331+0000 7f4105600640 1 Processor -- start 2024-08-31T06:11:37.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.332+0000 7f4105600640 1 -- start start 2024-08-31T06:11:37.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.332+0000 7f4105600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4100103900 0x7f410019c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:37.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.332+0000 7f4105600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4100109240 0x7f410019c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:37.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.332+0000 7f4105600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4100109b90 0x7f41001a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:37.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.332+0000 7f4105600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4100115a80 con 0x7f4100103900 2024-08-31T06:11:37.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.332+0000 7f4105600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4100115900 con 0x7f4100109b90 2024-08-31T06:11:37.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.332+0000 7f4105600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4100115c00 con 0x7f4100109240 2024-08-31T06:11:37.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.333+0000 7f40fe000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4100109240 0x7f410019c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:37.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.333+0000 7f40ff400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4100109b90 0x7f41001a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:37.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.334+0000 7f40fe000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4100109240 0x7f410019c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:32788/0 (socket says 172.21.15.16:32788) 2024-08-31T06:11:37.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.334+0000 7f40fe000640 1 -- 172.21.15.16:0/1884055034 learned_addr learned my addr 172.21.15.16:0/1884055034 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:37.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.334+0000 7f40ff400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4100109b90 0x7f41001a1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:55300/0 (socket says 172.21.15.16:55300) 2024-08-31T06:11:37.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.334+0000 7f40fea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4100103900 0x7f410019c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:37.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.335+0000 7f40ff400640 1 -- 172.21.15.16:0/1884055034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4100109240 msgr2=0x7f410019c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:37.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.335+0000 7f40ff400640 1 --2- 172.21.15.16:0/1884055034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4100109240 0x7f410019c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:37.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.335+0000 7f40ff400640 1 -- 172.21.15.16:0/1884055034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4100103900 msgr2=0x7f410019c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:37.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.335+0000 7f40ff400640 1 --2- 172.21.15.16:0/1884055034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4100103900 0x7f410019c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:37.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.335+0000 7f40ff400640 1 -- 172.21.15.16:0/1884055034 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41001a2380 con 0x7f4100109b90 2024-08-31T06:11:37.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.336+0000 7f40ff400640 1 --2- 172.21.15.16:0/1884055034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4100109b90 0x7f41001a1c50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f40f8009fd0 tx=0x7f40f800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:37.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.336+0000 7f40f7400640 1 -- 172.21.15.16:0/1884055034 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f800ce20 con 0x7f4100109b90 2024-08-31T06:11:37.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.336+0000 7f40f7400640 1 -- 172.21.15.16:0/1884055034 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40f8005020 con 0x7f4100109b90 2024-08-31T06:11:37.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.336+0000 7f40f7400640 1 -- 172.21.15.16:0/1884055034 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f8010690 con 0x7f4100109b90 2024-08-31T06:11:37.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.336+0000 7f4105600640 1 -- 172.21.15.16:0/1884055034 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41001a2670 con 0x7f4100109b90 2024-08-31T06:11:37.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.337+0000 7f4105600640 1 -- 172.21.15.16:0/1884055034 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41001a8dd0 con 0x7f4100109b90 2024-08-31T06:11:37.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.341+0000 7f40f7400640 1 -- 172.21.15.16:0/1884055034 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f40f80054d0 con 0x7f4100109b90 2024-08-31T06:11:37.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.343+0000 7f40f7400640 1 --2- 172.21.15.16:0/1884055034 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f40e40b4d20 0x7f40e40b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:37.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.344+0000 7f40f7400640 1 -- 172.21.15.16:0/1884055034 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f40f80e5b70 con 0x7f4100109b90 2024-08-31T06:11:37.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.344+0000 7f40fea00640 1 --2- 172.21.15.16:0/1884055034 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f40e40b4d20 0x7f40e40b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:37.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.345+0000 7f40fea00640 1 --2- 172.21.15.16:0/1884055034 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f40e40b4d20 0x7f40e40b7230 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f40f0009bd0 tx=0x7f40f0005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:37.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.346+0000 7f410c273740 1 Processor -- start 2024-08-31T06:11:37.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.347+0000 7f410c273740 1 -- start start 2024-08-31T06:11:37.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.347+0000 7f410c273740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b49b3dd070 0x55b49b782990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:37.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.347+0000 7f410c273740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b49b3b41b0 0x55b49b782ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:37.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.348+0000 7f410c273740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b49b3bc390 0x55b49b783410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:37.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.348+0000 7f410c273740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b49b6f97b0 con 0x55b49b3dd070 2024-08-31T06:11:37.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.348+0000 7f410c273740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b49b6e8d40 con 0x55b49b3b41b0 2024-08-31T06:11:37.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.348+0000 7f410c273740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b49b6eccf0 con 0x55b49b3bc390 2024-08-31T06:11:37.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.348+0000 7f40fe000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b49b3b41b0 0x55b49b782ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:37.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.348+0000 7f40ff400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b49b3bc390 0x55b49b783410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:37.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.348+0000 7f40fe000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b49b3b41b0 0x55b49b782ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:55304/0 (socket says 172.21.15.16:55304) 2024-08-31T06:11:37.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.348+0000 7f40fe000640 1 -- 172.21.15.16:0/3529555268 learned_addr learned my addr 172.21.15.16:0/3529555268 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:37.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.348+0000 7f40ff400640 1 -- 172.21.15.16:0/3529555268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b49b3b41b0 msgr2=0x55b49b782ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:37.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.348+0000 7f40ff400640 1 --2- 172.21.15.16:0/3529555268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b49b3b41b0 0x55b49b782ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:37.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.348+0000 7f40ff400640 1 -- 172.21.15.16:0/3529555268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b49b3dd070 msgr2=0x55b49b782990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:11:37.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.348+0000 7f40ff400640 1 --2- 172.21.15.16:0/3529555268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b49b3dd070 0x55b49b782990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:37.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.349+0000 7f40ff400640 1 -- 172.21.15.16:0/3529555268 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b49b448650 con 0x55b49b3bc390 2024-08-31T06:11:37.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.349+0000 7f40ff400640 1 --2- 172.21.15.16:0/3529555268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b49b3bc390 0x55b49b783410 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f40f80e3320 tx=0x7f40f80acb00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=961aa06a83cb43cc server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:37.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.349+0000 7f40eb400640 1 -- 172.21.15.16:0/3529555268 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f80b55e0 con 0x55b49b3bc390 2024-08-31T06:11:37.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.349+0000 7f40eb400640 1 -- 172.21.15.16:0/3529555268 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40f80e35a0 con 0x55b49b3bc390 2024-08-31T06:11:37.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.349+0000 7f40eb400640 1 -- 172.21.15.16:0/3529555268 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f80a7c30 con 0x55b49b3bc390 2024-08-31T06:11:37.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.351+0000 7f410c273740 1 -- 172.21.15.16:0/3529555268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b49b3bc390 msgr2=0x55b49b783410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:37.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.351+0000 7f410c273740 1 --2- 172.21.15.16:0/3529555268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b49b3bc390 0x55b49b783410 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f40f80e3320 tx=0x7f40f80acb00 comp rx=0 tx=0).stop 2024-08-31T06:11:37.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.351+0000 7f410c273740 1 -- 172.21.15.16:0/3529555268 shutdown_connections 2024-08-31T06:11:37.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.351+0000 7f410c273740 1 --2- 172.21.15.16:0/3529555268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b49b3bc390 0x55b49b783410 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:37.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.351+0000 7f410c273740 1 --2- 172.21.15.16:0/3529555268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b49b3b41b0 0x55b49b782ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:37.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.351+0000 7f410c273740 1 --2- 172.21.15.16:0/3529555268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b49b3dd070 0x55b49b782990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:37.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.351+0000 7f410c273740 1 -- 172.21.15.16:0/3529555268 >> 172.21.15.16:0/3529555268 conn(0x55b49b3d1460 msgr2=0x55b49b780880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:37.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.352+0000 7f410c273740 1 -- 172.21.15.16:0/3529555268 shutdown_connections 2024-08-31T06:11:37.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.352+0000 7f410c273740 1 -- 172.21.15.16:0/3529555268 wait complete. 2024-08-31T06:11:37.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.353+0000 7f410c273740 1 -- start start 2024-08-31T06:11:37.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.353+0000 7f410c273740 1 Processor -- start 2024-08-31T06:11:37.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.354+0000 7f410c273740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b49b3b41b0 0x55b49b7d2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:37.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.354+0000 7f410c273740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b49b3bc390 0x55b49b7d3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:37.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.354+0000 7f410c273740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b49b3dd070 0x55b49b7d8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:37.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.354+0000 7f410c273740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b49b54b700 con 0x55b49b3b41b0 2024-08-31T06:11:37.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.354+0000 7f410c273740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b49b66fcf0 con 0x55b49b3bc390 2024-08-31T06:11:37.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.354+0000 7f410c273740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b49b789040 con 0x55b49b3dd070 2024-08-31T06:11:37.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.354+0000 7f410c273740 10 client.0 _ll_register_callbacks cb 0x55b49b695470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:11:37.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.354+0000 7f40fea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b49b3b41b0 0x55b49b7d2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:37.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.354+0000 7f40fe000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b49b3bc390 0x55b49b7d3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:37.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.355+0000 7f40fe000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b49b3bc390 0x55b49b7d3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:55316/0 (socket says 172.21.15.16:55316) 2024-08-31T06:11:37.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.355+0000 7f40fe000640 1 -- 172.21.15.16:0/3643330357 learned_addr learned my addr 172.21.15.16:0/3643330357 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:37.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.355+0000 7f40ff400640 1 --2- 172.21.15.16:0/3643330357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b49b3dd070 0x55b49b7d8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:37.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.355+0000 7f40fe000640 1 -- 172.21.15.16:0/3643330357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b49b3dd070 msgr2=0x55b49b7d8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:37.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.355+0000 7f40fe000640 1 --2- 172.21.15.16:0/3643330357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b49b3dd070 0x55b49b7d8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:37.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.355+0000 7f40fe000640 1 -- 172.21.15.16:0/3643330357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b49b3b41b0 msgr2=0x55b49b7d2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:37.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.355+0000 7f40fe000640 1 --2- 172.21.15.16:0/3643330357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b49b3b41b0 0x55b49b7d2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:37.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.355+0000 7f40fe000640 1 -- 172.21.15.16:0/3643330357 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b49b6b08b0 con 0x55b49b3bc390 2024-08-31T06:11:37.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.355+0000 7f40fe000640 1 --2- 172.21.15.16:0/3643330357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b49b3bc390 0x55b49b7d3130 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f40ec006010 tx=0x7f40ec0053b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1eed567c640fdd5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:37.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.356+0000 7f40dfe00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:11:37.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.356+0000 7f40dfe00640 1 -- 172.21.15.16:0/3643330357 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec035040 con 0x55b49b3bc390 2024-08-31T06:11:37.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.356+0000 7f40dfe00640 1 -- 172.21.15.16:0/3643330357 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f40ec004460 con 0x55b49b3bc390 2024-08-31T06:11:37.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.356+0000 7f410c273740 1 -- 172.21.15.16:0/3643330357 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40f80e35a0 con 0x55b49b3bc390 2024-08-31T06:11:37.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.356+0000 7f40dfe00640 1 -- 172.21.15.16:0/3643330357 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f40ec004770 con 0x55b49b3bc390 2024-08-31T06:11:37.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.356+0000 7f410c273740 10 client.26591 fetch_fsmap learned FSMap version 34 2024-08-31T06:11:37.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.356+0000 7f410c273740 1 -- 172.21.15.16:0/3643330357 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b49b7e10a0 con 0x55b49b3bc390 2024-08-31T06:11:37.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.357+0000 7f40dfe00640 1 -- 172.21.15.16:0/3643330357 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40ec041070 con 0x55b49b3bc390 2024-08-31T06:11:37.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.357+0000 7f40dfe00640 1 -- 172.21.15.16:0/3643330357 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f40ec003bf0 con 0x55b49b3bc390 2024-08-31T06:11:37.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.358+0000 7f410c273740 10 client.26591 fetch_fsmap finished waiting for FSMap version 34 2024-08-31T06:11:37.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.358+0000 7f410c273740 10 client.26591 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-31T06:11:37.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.358+0000 7f410c273740 20 client.26591 populate_metadata read hostname 'smithi016' 2024-08-31T06:11:37.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.358+0000 7f410c273740 1 --2- 172.21.15.16:0/3643330357 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55b49b39b9f0 0x55b49b7e4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:37.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.358+0000 7f410c273740 4 client.26591 mds_command: new command op to 24724 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:11:37.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.358+0000 7f410c273740 1 -- 172.21.15.16:0/3643330357 --> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f40f80e35a0 con 0x55b49b39b9f0 2024-08-31T06:11:37.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.359+0000 7f40fea00640 1 --2- 172.21.15.16:0/3643330357 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55b49b39b9f0 0x55b49b7e4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:37.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.360+0000 7f40fea00640 1 --2- 172.21.15.16:0/3643330357 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55b49b39b9f0 0x55b49b7e4610 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6e6f08be3d937a71 server_cookie=e23e83ec6397a06c in_seq=0 out_seq=0 2024-08-31T06:11:37.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.361+0000 7f40dfe00640 10 client.26591 ms_handle_connect on v2:172.21.15.16:6838/219664460 2024-08-31T06:11:37.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.363+0000 7f40dfe00640 1 -- 172.21.15.16:0/3643330357 <== mds.1 v2:172.21.15.16:6838/219664460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f40f80e35a0 con 0x55b49b39b9f0 2024-08-31T06:11:37.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.363+0000 7f40dfe00640 10 client.26591 handle_command_reply: tid=0 2024-08-31T06:11:37.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.363+0000 7f410c273740 1 client.26591 shutdown 2024-08-31T06:11:37.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.364+0000 7f410c273740 1 -- 172.21.15.16:0/3643330357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b49b3bc390 msgr2=0x55b49b7d3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:37.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.364+0000 7f410c273740 1 --2- 172.21.15.16:0/3643330357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b49b3bc390 0x55b49b7d3130 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f40ec006010 tx=0x7f40ec0053b0 comp rx=0 tx=0).stop 2024-08-31T06:11:37.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.364+0000 7f410c273740 1 -- 172.21.15.16:0/3643330357 shutdown_connections 2024-08-31T06:11:37.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.364+0000 7f410c273740 1 --2- 172.21.15.16:0/3643330357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b49b3dd070 0x55b49b7d8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:37.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.364+0000 7f410c273740 1 --2- 172.21.15.16:0/3643330357 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55b49b39b9f0 0x55b49b7e4610 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:37.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.364+0000 7f410c273740 1 --2- 172.21.15.16:0/3643330357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b49b3bc390 0x55b49b7d3130 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:37.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.364+0000 7f410c273740 1 --2- 172.21.15.16:0/3643330357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b49b3b41b0 0x55b49b7d2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:37.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.364+0000 7f410c273740 1 -- 172.21.15.16:0/3643330357 >> 172.21.15.16:0/3643330357 conn(0x55b49b3d1460 msgr2=0x55b49b785ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:37.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.364+0000 7f40dfe00640 0 client.26591 ms_handle_reset on v2:172.21.15.16:6838/219664460 2024-08-31T06:11:37.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.364+0000 7f410c273740 1 -- 172.21.15.16:0/3643330357 shutdown_connections 2024-08-31T06:11:37.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.365+0000 7f410c273740 1 -- 172.21.15.16:0/3643330357 wait complete. 2024-08-31T06:11:37.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.365+0000 7f410c273740 20 client.26591 trim_cache size 0 max 16384 2024-08-31T06:11:37.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.390+0000 7f410c273740 1 Processor -- start 2024-08-31T06:11:37.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.390+0000 7f410c273740 1 -- start start 2024-08-31T06:11:37.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.391+0000 7f410c273740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b49b3b41b0 0x55b49b77fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:37.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.391+0000 7f410c273740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b49b3bc390 0x55b49b780370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:37.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.391+0000 7f410c273740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b49b3dd070 0x55b49b7e9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:37.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.391+0000 7f410c273740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f41001af9b0 con 0x55b49b3bc390 2024-08-31T06:11:37.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.391+0000 7f410c273740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f41001ad250 con 0x55b49b3dd070 2024-08-31T06:11:37.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.391+0000 7f410c273740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f41001ab0c0 con 0x55b49b3b41b0 2024-08-31T06:11:37.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.392+0000 7f40ff400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b49b3dd070 0x55b49b7e9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:37.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.392+0000 7f40fea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b49b3b41b0 0x55b49b77fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:37.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.392+0000 7f40fea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b49b3b41b0 0x55b49b77fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:32818/0 (socket says 172.21.15.16:32818) 2024-08-31T06:11:37.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.392+0000 7f40fea00640 1 -- 172.21.15.16:0/2764399182 learned_addr learned my addr 172.21.15.16:0/2764399182 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:37.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.393+0000 7f40fea00640 1 -- 172.21.15.16:0/2764399182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b49b3dd070 msgr2=0x55b49b7e9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:37.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.393+0000 7f40fea00640 1 --2- 172.21.15.16:0/2764399182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b49b3dd070 0x55b49b7e9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:37.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.393+0000 7f40fea00640 1 -- 172.21.15.16:0/2764399182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b49b3bc390 msgr2=0x55b49b780370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:37.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.393+0000 7f40fea00640 1 --2- 172.21.15.16:0/2764399182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b49b3bc390 0x55b49b780370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:37.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.393+0000 7f40fea00640 1 -- 172.21.15.16:0/2764399182 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b49b786a70 con 0x55b49b3b41b0 2024-08-31T06:11:37.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.393+0000 7f40fea00640 1 --2- 172.21.15.16:0/2764399182 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b49b3b41b0 0x55b49b77fe30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f40f0000c00 tx=0x7f40f0014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71935fffbebb7cbf server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:37.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.394+0000 7f40eaa00640 1 -- 172.21.15.16:0/2764399182 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f0003ac0 con 0x55b49b3b41b0 2024-08-31T06:11:37.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.394+0000 7f40eaa00640 1 -- 172.21.15.16:0/2764399182 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40f0003c60 con 0x55b49b3b41b0 2024-08-31T06:11:37.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.394+0000 7f40eaa00640 1 -- 172.21.15.16:0/2764399182 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f001d670 con 0x55b49b3b41b0 2024-08-31T06:11:37.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.395+0000 7f410c273740 1 -- 172.21.15.16:0/2764399182 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b49b3b41b0 msgr2=0x55b49b77fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:37.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.395+0000 7f410c273740 1 --2- 172.21.15.16:0/2764399182 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b49b3b41b0 0x55b49b77fe30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f40f0000c00 tx=0x7f40f0014000 comp rx=0 tx=0).stop 2024-08-31T06:11:37.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.395+0000 7f410c273740 1 -- 172.21.15.16:0/2764399182 shutdown_connections 2024-08-31T06:11:37.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.395+0000 7f410c273740 1 --2- 172.21.15.16:0/2764399182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b49b3dd070 0x55b49b7e9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:37.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.395+0000 7f410c273740 1 --2- 172.21.15.16:0/2764399182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b49b3bc390 0x55b49b780370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:37.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.395+0000 7f410c273740 1 --2- 172.21.15.16:0/2764399182 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b49b3b41b0 0x55b49b77fe30 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:37.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.395+0000 7f410c273740 1 -- 172.21.15.16:0/2764399182 >> 172.21.15.16:0/2764399182 conn(0x55b49b3d1460 msgr2=0x55b49b784380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:37.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.395+0000 7f410c273740 1 -- 172.21.15.16:0/2764399182 shutdown_connections 2024-08-31T06:11:37.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.395+0000 7f410c273740 1 -- 172.21.15.16:0/2764399182 wait complete. 2024-08-31T06:11:37.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.396+0000 7f410c273740 1 -- start start 2024-08-31T06:11:37.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.396+0000 7f410c273740 1 Processor -- start 2024-08-31T06:11:37.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.397+0000 7f410c273740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b49b3b41b0 0x55b49b7df050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:37.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.397+0000 7f410c273740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b49b3bc390 0x55b49b7dd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:37.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.397+0000 7f410c273740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b49b3dd070 0x55b49b7ddcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:37.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.398+0000 7f40fea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b49b3b41b0 0x55b49b7df050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:37.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.398+0000 7f40fea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b49b3b41b0 0x55b49b7df050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:55330/0 (socket says 172.21.15.16:55330) 2024-08-31T06:11:37.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.398+0000 7f40ff400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b49b3dd070 0x55b49b7ddcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:37.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.398+0000 7f40ff400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b49b3dd070 0x55b49b7ddcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:32832/0 (socket says 172.21.15.16:32832) 2024-08-31T06:11:37.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.399+0000 7f40fe000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b49b3bc390 0x55b49b7dd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:37.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.399+0000 7f40fea00640 1 -- 172.21.15.16:0/2318392539 learned_addr learned my addr 172.21.15.16:0/2318392539 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:37.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.399+0000 7f410c273740 1 -- 172.21.15.16:0/2318392539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f41001ab710 con 0x55b49b3bc390 2024-08-31T06:11:37.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.399+0000 7f410c273740 1 -- 172.21.15.16:0/2318392539 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f41001acb80 con 0x55b49b3b41b0 2024-08-31T06:11:37.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.399+0000 7f410c273740 1 -- 172.21.15.16:0/2318392539 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f41001abde0 con 0x55b49b3dd070 2024-08-31T06:11:37.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.399+0000 7f410c273740 10 client.0 _ll_register_callbacks cb 0x7f40ec009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:11:37.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.399+0000 7f40ff400640 1 -- 172.21.15.16:0/2318392539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b49b3b41b0 msgr2=0x55b49b7df050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:37.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.399+0000 7f40ff400640 1 --2- 172.21.15.16:0/2318392539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b49b3b41b0 0x55b49b7df050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:37.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.400+0000 7f40ff400640 1 -- 172.21.15.16:0/2318392539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b49b3bc390 msgr2=0x55b49b7dd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:37.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.400+0000 7f40ff400640 1 --2- 172.21.15.16:0/2318392539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b49b3bc390 0x55b49b7dd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:37.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.400+0000 7f40ff400640 1 -- 172.21.15.16:0/2318392539 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b49b84efa0 con 0x55b49b3dd070 2024-08-31T06:11:37.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.400+0000 7f40ff400640 1 --2- 172.21.15.16:0/2318392539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b49b3dd070 0x55b49b7ddcf0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f40f80acb70 tx=0x7f40f80b5380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3658386fda625b6b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:37.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.400+0000 7f40dfe00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:11:37.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.401+0000 7f40dfe00640 1 -- 172.21.15.16:0/2318392539 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f80ac090 con 0x55b49b3dd070 2024-08-31T06:11:37.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.401+0000 7f40dfe00640 1 -- 172.21.15.16:0/2318392539 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f40f80ab730 con 0x55b49b3dd070 2024-08-31T06:11:37.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.401+0000 7f410c273740 1 -- 172.21.15.16:0/2318392539 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b49b789600 con 0x55b49b3dd070 2024-08-31T06:11:37.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.401+0000 7f40dfe00640 1 -- 172.21.15.16:0/2318392539 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f40f80cf0b0 con 0x55b49b3dd070 2024-08-31T06:11:37.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.401+0000 7f410c273740 10 client.26596 fetch_fsmap learned FSMap version 34 2024-08-31T06:11:37.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.401+0000 7f410c273740 1 -- 172.21.15.16:0/2318392539 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b49b850120 con 0x55b49b3dd070 2024-08-31T06:11:37.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.401+0000 7f40dfe00640 1 -- 172.21.15.16:0/2318392539 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b49b789600 con 0x55b49b3dd070 2024-08-31T06:11:37.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.402+0000 7f40dfe00640 1 -- 172.21.15.16:0/2318392539 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f40f80b1cc0 con 0x55b49b3dd070 2024-08-31T06:11:37.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.402+0000 7f410c273740 10 client.26596 fetch_fsmap finished waiting for FSMap version 34 2024-08-31T06:11:37.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.402+0000 7f410c273740 10 client.26596 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-31T06:11:37.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.402+0000 7f410c273740 20 client.26596 populate_metadata read hostname 'smithi016' 2024-08-31T06:11:37.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.402+0000 7f410c273740 1 --2- 172.21.15.16:0/2318392539 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55b49b370da0 0x55b49b852300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:37.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.402+0000 7f410c273740 4 client.26596 mds_command: new command op to 24724 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:11:37.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.402+0000 7f410c273740 1 -- 172.21.15.16:0/2318392539 --> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b49b6fb700 con 0x55b49b370da0 2024-08-31T06:11:37.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.402+0000 7f40fea00640 1 --2- 172.21.15.16:0/2318392539 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55b49b370da0 0x55b49b852300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:37.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.404+0000 7f40dfe00640 10 client.26596 ms_handle_connect on v2:172.21.15.16:6838/219664460 2024-08-31T06:11:37.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.404+0000 7f40fea00640 1 --2- 172.21.15.16:0/2318392539 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55b49b370da0 0x55b49b852300 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=320912319aef4d6e server_cookie=d43a238999010c6d in_seq=0 out_seq=0 2024-08-31T06:11:37.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.405+0000 7f40dfe00640 1 -- 172.21.15.16:0/2318392539 <== mds.1 v2:172.21.15.16:6838/219664460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b49b6fb700 con 0x55b49b370da0 2024-08-31T06:11:37.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.405+0000 7f40dfe00640 10 client.26596 handle_command_reply: tid=0 2024-08-31T06:11:37.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.405+0000 7f410c273740 1 client.26596 shutdown 2024-08-31T06:11:37.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.405+0000 7f410c273740 1 -- 172.21.15.16:0/2318392539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b49b3dd070 msgr2=0x55b49b7ddcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:37.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.406+0000 7f410c273740 1 --2- 172.21.15.16:0/2318392539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b49b3dd070 0x55b49b7ddcf0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f40f80acb70 tx=0x7f40f80b5380 comp rx=0 tx=0).stop 2024-08-31T06:11:37.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.406+0000 7f410c273740 1 -- 172.21.15.16:0/2318392539 shutdown_connections 2024-08-31T06:11:37.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.406+0000 7f410c273740 1 --2- 172.21.15.16:0/2318392539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b49b3dd070 0x55b49b7ddcf0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:37.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.406+0000 7f410c273740 1 --2- 172.21.15.16:0/2318392539 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55b49b370da0 0x55b49b852300 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:37.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.406+0000 7f410c273740 1 --2- 172.21.15.16:0/2318392539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b49b3bc390 0x55b49b7dd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:37.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.406+0000 7f410c273740 1 --2- 172.21.15.16:0/2318392539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b49b3b41b0 0x55b49b7df050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:37.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.406+0000 7f410c273740 1 -- 172.21.15.16:0/2318392539 >> 172.21.15.16:0/2318392539 conn(0x55b49b3d1460 msgr2=0x55b49b784150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:37.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.406+0000 7f40dfe00640 0 client.26596 ms_handle_reset on v2:172.21.15.16:6838/219664460 2024-08-31T06:11:37.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.406+0000 7f410c273740 1 -- 172.21.15.16:0/2318392539 shutdown_connections 2024-08-31T06:11:37.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.406+0000 7f410c273740 1 -- 172.21.15.16:0/2318392539 wait complete. 2024-08-31T06:11:37.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.407+0000 7f410c273740 20 client.26596 trim_cache size 0 max 16384 2024-08-31T06:11:37.417 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:11:37.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.408+0000 7f4105600640 1 -- 172.21.15.16:0/1884055034 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f40e40b4d20 msgr2=0x7f40e40b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:37.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.408+0000 7f4105600640 1 --2- 172.21.15.16:0/1884055034 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f40e40b4d20 0x7f40e40b7230 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f40f0009bd0 tx=0x7f40f0005f90 comp rx=0 tx=0).stop 2024-08-31T06:11:37.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.408+0000 7f4105600640 1 -- 172.21.15.16:0/1884055034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4100109b90 msgr2=0x7f41001a1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:37.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.408+0000 7f4105600640 1 --2- 172.21.15.16:0/1884055034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4100109b90 0x7f41001a1c50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f40f8009fd0 tx=0x7f40f800ef10 comp rx=0 tx=0).stop 2024-08-31T06:11:37.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.408+0000 7f4105600640 1 -- 172.21.15.16:0/1884055034 shutdown_connections 2024-08-31T06:11:37.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.408+0000 7f4105600640 1 --2- 172.21.15.16:0/1884055034 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f40e40b4d20 0x7f40e40b7230 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:37.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.408+0000 7f4105600640 1 --2- 172.21.15.16:0/1884055034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4100109240 0x7f410019c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:37.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.408+0000 7f4105600640 1 --2- 172.21.15.16:0/1884055034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4100109b90 0x7f41001a1c50 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:37.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.408+0000 7f4105600640 1 --2- 172.21.15.16:0/1884055034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4100103900 0x7f410019c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:37.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.408+0000 7f4105600640 1 -- 172.21.15.16:0/1884055034 >> 172.21.15.16:0/1884055034 conn(0x7f410007b5f0 msgr2=0x7f41001052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:37.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.408+0000 7f4105600640 1 -- 172.21.15.16:0/1884055034 shutdown_connections 2024-08-31T06:11:37.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:37.408+0000 7f4105600640 1 -- 172.21.15.16:0/1884055034 wait complete. 2024-08-31T06:11:38.733 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T06:11:38.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:38 smithi184 ceph-mon[22331]: pgmap v496: 97 pgs: 97 active+clean; 2.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 7.8 MiB/s wr, 658 op/s 2024-08-31T06:11:39.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:38 smithi155 ceph-mon[21742]: pgmap v496: 97 pgs: 97 active+clean; 2.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 7.8 MiB/s wr, 658 op/s 2024-08-31T06:11:39.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:38 smithi016 ceph-mon[17194]: pgmap v496: 97 pgs: 97 active+clean; 2.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 7.8 MiB/s wr, 658 op/s 2024-08-31T06:11:39.353 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:11:40.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.206+0000 7f8040e00640 1 -- 172.21.15.16:0/229891095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f803c109b90 msgr2=0x7f803c1103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:40.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.206+0000 7f8040e00640 1 --2- 172.21.15.16:0/229891095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f803c109b90 0x7f803c1103c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8034009be0 tx=0x7f803402a6c0 comp rx=0 tx=0).stop 2024-08-31T06:11:40.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.207+0000 7f8040e00640 1 -- 172.21.15.16:0/229891095 shutdown_connections 2024-08-31T06:11:40.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.207+0000 7f8040e00640 1 --2- 172.21.15.16:0/229891095 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f803c109240 0x7f803c109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:40.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.207+0000 7f8040e00640 1 --2- 172.21.15.16:0/229891095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f803c109b90 0x7f803c1103c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:40.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.207+0000 7f8040e00640 1 --2- 172.21.15.16:0/229891095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f803c103900 0x7f803c103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:40.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.207+0000 7f8040e00640 1 -- 172.21.15.16:0/229891095 >> 172.21.15.16:0/229891095 conn(0x7f803c07b5f0 msgr2=0x7f803c07ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:40.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.208+0000 7f8040e00640 1 -- 172.21.15.16:0/229891095 shutdown_connections 2024-08-31T06:11:40.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.208+0000 7f8040e00640 1 -- 172.21.15.16:0/229891095 wait complete. 2024-08-31T06:11:40.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.209+0000 7f8040e00640 1 Processor -- start 2024-08-31T06:11:40.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.209+0000 7f8040e00640 1 -- start start 2024-08-31T06:11:40.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.210+0000 7f8040e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f803c103900 0x7f803c119050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:40.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.210+0000 7f8040e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f803c109240 0x7f803c112320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:40.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.210+0000 7f8040e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f803c109b90 0x7f803c112860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:40.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.210+0000 7f8040e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f803c119e70 con 0x7f803c103900 2024-08-31T06:11:40.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.210+0000 7f8040e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f803c119cf0 con 0x7f803c109240 2024-08-31T06:11:40.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.210+0000 7f8040e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f803c119ff0 con 0x7f803c109b90 2024-08-31T06:11:40.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.210+0000 7f8039400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f803c109240 0x7f803c112320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:40.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.211+0000 7f8039400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f803c109240 0x7f803c112320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:55346/0 (socket says 172.21.15.16:55346) 2024-08-31T06:11:40.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.211+0000 7f8039400640 1 -- 172.21.15.16:0/2997958799 learned_addr learned my addr 172.21.15.16:0/2997958799 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:40.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.211+0000 7f8039e00640 1 --2- 172.21.15.16:0/2997958799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f803c103900 0x7f803c119050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:40.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.211+0000 7f803a800640 1 --2- 172.21.15.16:0/2997958799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f803c109b90 0x7f803c112860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:40.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.211+0000 7f8039400640 1 -- 172.21.15.16:0/2997958799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f803c109b90 msgr2=0x7f803c112860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:40.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.211+0000 7f8039400640 1 --2- 172.21.15.16:0/2997958799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f803c109b90 0x7f803c112860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:40.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.212+0000 7f8039400640 1 -- 172.21.15.16:0/2997958799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f803c103900 msgr2=0x7f803c119050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:40.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.212+0000 7f8039400640 1 --2- 172.21.15.16:0/2997958799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f803c103900 0x7f803c119050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:40.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.212+0000 7f8039400640 1 -- 172.21.15.16:0/2997958799 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f803c113080 con 0x7f803c109240 2024-08-31T06:11:40.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.212+0000 7f8039400640 1 --2- 172.21.15.16:0/2997958799 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f803c109240 0x7f803c112320 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f802800e9f0 tx=0x7f802800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:40.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.212+0000 7f8032a00640 1 -- 172.21.15.16:0/2997958799 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802800ccc0 con 0x7f803c109240 2024-08-31T06:11:40.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.213+0000 7f8040e00640 1 -- 172.21.15.16:0/2997958799 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f803c117d20 con 0x7f803c109240 2024-08-31T06:11:40.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.213+0000 7f8040e00640 1 -- 172.21.15.16:0/2997958799 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f803c1182b0 con 0x7f803c109240 2024-08-31T06:11:40.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.213+0000 7f8032a00640 1 -- 172.21.15.16:0/2997958799 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80280108e0 con 0x7f803c109240 2024-08-31T06:11:40.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.213+0000 7f8032a00640 1 -- 172.21.15.16:0/2997958799 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8028018860 con 0x7f803c109240 2024-08-31T06:11:40.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.219+0000 7f8047a8e740 1 Processor -- start 2024-08-31T06:11:40.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.220+0000 7f8047a8e740 1 -- start start 2024-08-31T06:11:40.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.221+0000 7f8032a00640 1 -- 172.21.15.16:0/2997958799 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f8028018a00 con 0x7f803c109240 2024-08-31T06:11:40.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.221+0000 7f8047a8e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da0b895070 0x55da0bc3a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:40.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.221+0000 7f8047a8e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da0b86c1b0 0x55da0bc3adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:40.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.222+0000 7f8047a8e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da0b874390 0x55da0bc3b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:40.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.222+0000 7f803a800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da0b895070 0x55da0bc3a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:40.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.222+0000 7f8047a8e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55da0bbb17b0 con 0x55da0b86c1b0 2024-08-31T06:11:40.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.222+0000 7f8047a8e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55da0bba0d40 con 0x55da0b874390 2024-08-31T06:11:40.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.222+0000 7f8047a8e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55da0bba4cf0 con 0x55da0b895070 2024-08-31T06:11:40.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.223+0000 7f803a800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da0b895070 0x55da0bc3a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:32862/0 (socket says 172.21.15.16:32862) 2024-08-31T06:11:40.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.223+0000 7f803a800640 1 -- 172.21.15.16:0/3144577714 learned_addr learned my addr 172.21.15.16:0/3144577714 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:40.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.223+0000 7f8039e00640 1 --2- 172.21.15.16:0/3144577714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da0b86c1b0 0x55da0bc3adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:40.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.223+0000 7f803a800640 1 -- 172.21.15.16:0/3144577714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da0b874390 msgr2=0x55da0bc3b330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:11:40.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.223+0000 7f803a800640 1 --2- 172.21.15.16:0/3144577714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da0b874390 0x55da0bc3b330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:40.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.223+0000 7f803a800640 1 -- 172.21.15.16:0/3144577714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da0b86c1b0 msgr2=0x55da0bc3adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:40.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.223+0000 7f803a800640 1 --2- 172.21.15.16:0/3144577714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da0b86c1b0 0x55da0bc3adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:40.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.223+0000 7f803a800640 1 -- 172.21.15.16:0/3144577714 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da0b900650 con 0x55da0b895070 2024-08-31T06:11:40.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.224+0000 7f803a800640 1 --2- 172.21.15.16:0/3144577714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da0b895070 0x55da0bc3a8b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8034009be0 tx=0x7f8034030430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9721bf8174c30680 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:40.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.224+0000 7f8032a00640 1 --2- 172.21.15.16:0/2997958799 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f80200b4da0 0x7f80200b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:40.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.225+0000 7f8032a00640 1 -- 172.21.15.16:0/2997958799 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f80280e5790 con 0x7f803c109240 2024-08-31T06:11:40.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.225+0000 7f8039e00640 1 --2- 172.21.15.16:0/2997958799 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f80200b4da0 0x7f80200b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:40.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.225+0000 7f8027400640 1 -- 172.21.15.16:0/3144577714 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8034040030 con 0x55da0b895070 2024-08-31T06:11:40.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.225+0000 7f8027400640 1 -- 172.21.15.16:0/3144577714 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8034003e20 con 0x55da0b895070 2024-08-31T06:11:40.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.225+0000 7f8027400640 1 -- 172.21.15.16:0/3144577714 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8034036040 con 0x55da0b895070 2024-08-31T06:11:40.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.227+0000 7f8047a8e740 1 -- 172.21.15.16:0/3144577714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da0b895070 msgr2=0x55da0bc3a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:40.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.227+0000 7f8047a8e740 1 --2- 172.21.15.16:0/3144577714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da0b895070 0x55da0bc3a8b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8034009be0 tx=0x7f8034030430 comp rx=0 tx=0).stop 2024-08-31T06:11:40.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.227+0000 7f8039e00640 1 --2- 172.21.15.16:0/2997958799 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f80200b4da0 0x7f80200b72b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x55da0bc39530 tx=0x7f802c011040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:40.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.227+0000 7f8047a8e740 1 -- 172.21.15.16:0/3144577714 shutdown_connections 2024-08-31T06:11:40.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.227+0000 7f8047a8e740 1 --2- 172.21.15.16:0/3144577714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da0b874390 0x55da0bc3b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:40.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.227+0000 7f8047a8e740 1 --2- 172.21.15.16:0/3144577714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da0b86c1b0 0x55da0bc3adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:40.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.227+0000 7f8047a8e740 1 --2- 172.21.15.16:0/3144577714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da0b895070 0x55da0bc3a8b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:40.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.227+0000 7f8047a8e740 1 -- 172.21.15.16:0/3144577714 >> 172.21.15.16:0/3144577714 conn(0x55da0b889460 msgr2=0x55da0bc387a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:40.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.228+0000 7f8047a8e740 1 -- 172.21.15.16:0/3144577714 shutdown_connections 2024-08-31T06:11:40.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.228+0000 7f8047a8e740 1 -- 172.21.15.16:0/3144577714 wait complete. 2024-08-31T06:11:40.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.228+0000 7f8047a8e740 1 -- start start 2024-08-31T06:11:40.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.229+0000 7f8047a8e740 1 Processor -- start 2024-08-31T06:11:40.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.230+0000 7f8047a8e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da0b86c1b0 0x55da0bc46520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:40.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.230+0000 7f8047a8e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da0b874390 0x55da0bc46aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:40.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.230+0000 7f8047a8e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da0b895070 0x55da0bc523b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:40.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.230+0000 7f8039e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da0b86c1b0 0x55da0bc46520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:40.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.230+0000 7f8039e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da0b86c1b0 0x55da0bc46520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:55362/0 (socket says 172.21.15.16:55362) 2024-08-31T06:11:40.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.230+0000 7f8039400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da0b874390 0x55da0bc46aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:40.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.230+0000 7f8039400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da0b874390 0x55da0bc46aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54036/0 (socket says 172.21.15.16:54036) 2024-08-31T06:11:40.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.230+0000 7f8039400640 1 -- 172.21.15.16:0/1571678354 learned_addr learned my addr 172.21.15.16:0/1571678354 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:40.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.230+0000 7f8047a8e740 1 -- 172.21.15.16:0/1571678354 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55da0ba03700 con 0x55da0b874390 2024-08-31T06:11:40.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.230+0000 7f8047a8e740 1 -- 172.21.15.16:0/1571678354 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55da0bb27cf0 con 0x55da0b86c1b0 2024-08-31T06:11:40.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.230+0000 7f8047a8e740 1 -- 172.21.15.16:0/1571678354 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55da0bc49030 con 0x55da0b895070 2024-08-31T06:11:40.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.230+0000 7f803a800640 1 --2- 172.21.15.16:0/1571678354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da0b895070 0x55da0bc523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:40.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.231+0000 7f8047a8e740 10 client.0 _ll_register_callbacks cb 0x55da0bb4d470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:11:40.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.231+0000 7f803a800640 1 -- 172.21.15.16:0/1571678354 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da0b86c1b0 msgr2=0x55da0bc46520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:40.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.231+0000 7f803a800640 1 --2- 172.21.15.16:0/1571678354 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da0b86c1b0 0x55da0bc46520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:40.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.231+0000 7f803a800640 1 -- 172.21.15.16:0/1571678354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da0b874390 msgr2=0x55da0bc46aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:40.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.231+0000 7f803a800640 1 --2- 172.21.15.16:0/1571678354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da0b874390 0x55da0bc46aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:40.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.231+0000 7f803a800640 1 -- 172.21.15.16:0/1571678354 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da0bb688b0 con 0x55da0b895070 2024-08-31T06:11:40.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.231+0000 7f8039400640 1 --2- 172.21.15.16:0/1571678354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da0b874390 0x55da0bc46aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:11:40.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.232+0000 7f803a800640 1 --2- 172.21.15.16:0/1571678354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da0b895070 0x55da0bc523b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8034006010 tx=0x7f8034030cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9aa5bac582b9798 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:40.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.232+0000 7f801be00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:11:40.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.233+0000 7f801be00640 1 -- 172.21.15.16:0/1571678354 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8034036050 con 0x55da0b895070 2024-08-31T06:11:40.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.233+0000 7f801be00640 1 -- 172.21.15.16:0/1571678354 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f8034030620 con 0x55da0b895070 2024-08-31T06:11:40.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.233+0000 7f8047a8e740 1 -- 172.21.15.16:0/1571678354 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8034003e20 con 0x55da0b895070 2024-08-31T06:11:40.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.233+0000 7f801be00640 1 -- 172.21.15.16:0/1571678354 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8034036050 con 0x55da0b895070 2024-08-31T06:11:40.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.234+0000 7f8047a8e740 10 client.26608 fetch_fsmap learned FSMap version 34 2024-08-31T06:11:40.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.234+0000 7f8047a8e740 1 -- 172.21.15.16:0/1571678354 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55da0bc55100 con 0x55da0b895070 2024-08-31T06:11:40.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.234+0000 7f801be00640 1 -- 172.21.15.16:0/1571678354 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8034003e20 con 0x55da0b895070 2024-08-31T06:11:40.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.235+0000 7f801be00640 1 -- 172.21.15.16:0/1571678354 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f80340043e0 con 0x55da0b895070 2024-08-31T06:11:40.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.236+0000 7f8047a8e740 10 client.26608 fetch_fsmap finished waiting for FSMap version 34 2024-08-31T06:11:40.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.236+0000 7f8047a8e740 10 client.26608 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:11:40.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.236+0000 7f8047a8e740 20 client.26608 populate_metadata read hostname 'smithi016' 2024-08-31T06:11:40.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.236+0000 7f8047a8e740 1 --2- 172.21.15.16:0/1571678354 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55da0b8539f0 0x55da0bc58460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:40.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.236+0000 7f8047a8e740 4 client.26608 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:11:40.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.236+0000 7f8047a8e740 1 -- 172.21.15.16:0/1571678354 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da0bbb3700 con 0x55da0b8539f0 2024-08-31T06:11:40.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.236+0000 7f8039e00640 1 --2- 172.21.15.16:0/1571678354 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55da0b8539f0 0x55da0bc58460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:40.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.237+0000 7f801be00640 10 client.26608 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:11:40.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.237+0000 7f8039e00640 1 --2- 172.21.15.16:0/1571678354 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55da0b8539f0 0x55da0bc58460 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26219d7d394f278b server_cookie=5a3bcaa42dfc2435 in_seq=0 out_seq=0 2024-08-31T06:11:40.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.238+0000 7f801be00640 1 -- 172.21.15.16:0/1571678354 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55da0bbb3700 con 0x55da0b8539f0 2024-08-31T06:11:40.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.238+0000 7f801be00640 10 client.26608 handle_command_reply: tid=0 2024-08-31T06:11:40.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.238+0000 7f8047a8e740 1 client.26608 shutdown 2024-08-31T06:11:40.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.239+0000 7f8047a8e740 1 -- 172.21.15.16:0/1571678354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da0b895070 msgr2=0x55da0bc523b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:40.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.239+0000 7f8047a8e740 1 --2- 172.21.15.16:0/1571678354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da0b895070 0x55da0bc523b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8034006010 tx=0x7f8034030cb0 comp rx=0 tx=0).stop 2024-08-31T06:11:40.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.239+0000 7f8047a8e740 1 -- 172.21.15.16:0/1571678354 shutdown_connections 2024-08-31T06:11:40.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.239+0000 7f8047a8e740 1 --2- 172.21.15.16:0/1571678354 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55da0b8539f0 0x55da0bc58460 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:40.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.239+0000 7f8047a8e740 1 --2- 172.21.15.16:0/1571678354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da0b895070 0x55da0bc523b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:40.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.239+0000 7f801be00640 0 client.26608 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:11:40.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.239+0000 7f8047a8e740 1 --2- 172.21.15.16:0/1571678354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da0b874390 0x55da0bc46aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:40.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.239+0000 7f8047a8e740 1 --2- 172.21.15.16:0/1571678354 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da0b86c1b0 0x55da0bc46520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:40.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.239+0000 7f8047a8e740 1 -- 172.21.15.16:0/1571678354 >> 172.21.15.16:0/1571678354 conn(0x55da0b889460 msgr2=0x55da0bc3da40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:40.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.240+0000 7f8047a8e740 1 -- 172.21.15.16:0/1571678354 shutdown_connections 2024-08-31T06:11:40.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.240+0000 7f8047a8e740 1 -- 172.21.15.16:0/1571678354 wait complete. 2024-08-31T06:11:40.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.240+0000 7f8047a8e740 20 client.26608 trim_cache size 0 max 16384 2024-08-31T06:11:40.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.276+0000 7f8047a8e740 1 Processor -- start 2024-08-31T06:11:40.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.280+0000 7f8047a8e740 1 -- start start 2024-08-31T06:11:40.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.280+0000 7f8047a8e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da0b86c1b0 0x55da0bc400d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:40.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.280+0000 7f8047a8e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da0b874390 0x55da0bc52320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:40.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.280+0000 7f8047a8e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da0b895070 0x55da0bc380e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:40.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.280+0000 7f8047a8e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f801c010fb0 con 0x55da0b895070 2024-08-31T06:11:40.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.280+0000 7f8047a8e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f801c00e850 con 0x55da0b874390 2024-08-31T06:11:40.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.280+0000 7f8047a8e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f801c00c6a0 con 0x55da0b86c1b0 2024-08-31T06:11:40.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.281+0000 7f8039400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da0b874390 0x55da0bc52320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:40.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.281+0000 7f8039e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da0b86c1b0 0x55da0bc400d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:40.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.281+0000 7f8039400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da0b874390 0x55da0bc52320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:55376/0 (socket says 172.21.15.16:55376) 2024-08-31T06:11:40.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.281+0000 7f8039400640 1 -- 172.21.15.16:0/1561327258 learned_addr learned my addr 172.21.15.16:0/1561327258 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:40.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.281+0000 7f803a800640 1 --2- 172.21.15.16:0/1561327258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da0b895070 0x55da0bc380e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:40.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.282+0000 7f8039400640 1 -- 172.21.15.16:0/1561327258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da0b86c1b0 msgr2=0x55da0bc400d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:40.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.282+0000 7f8039400640 1 --2- 172.21.15.16:0/1561327258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da0b86c1b0 0x55da0bc400d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:40.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.282+0000 7f8039400640 1 -- 172.21.15.16:0/1561327258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da0b895070 msgr2=0x55da0bc380e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:40.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.282+0000 7f8039400640 1 --2- 172.21.15.16:0/1561327258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da0b895070 0x55da0bc380e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:40.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.282+0000 7f8039400640 1 -- 172.21.15.16:0/1561327258 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da0bc3cba0 con 0x55da0b874390 2024-08-31T06:11:40.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.284+0000 7f8039400640 1 --2- 172.21.15.16:0/1561327258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da0b874390 0x55da0bc52320 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8028009230 tx=0x7f802800cf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3428c9f77324c0d8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:40.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.285+0000 7f8026a00640 1 -- 172.21.15.16:0/1561327258 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80280abe80 con 0x55da0b874390 2024-08-31T06:11:40.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.285+0000 7f8026a00640 1 -- 172.21.15.16:0/1561327258 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80280b06f0 con 0x55da0b874390 2024-08-31T06:11:40.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.285+0000 7f8026a00640 1 -- 172.21.15.16:0/1561327258 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80280ab0e0 con 0x55da0b874390 2024-08-31T06:11:40.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.288+0000 7f8047a8e740 1 -- 172.21.15.16:0/1561327258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da0b874390 msgr2=0x55da0bc52320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:40.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.289+0000 7f8047a8e740 1 --2- 172.21.15.16:0/1561327258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da0b874390 0x55da0bc52320 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8028009230 tx=0x7f802800cf20 comp rx=0 tx=0).stop 2024-08-31T06:11:40.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.289+0000 7f8047a8e740 1 -- 172.21.15.16:0/1561327258 shutdown_connections 2024-08-31T06:11:40.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.289+0000 7f8047a8e740 1 --2- 172.21.15.16:0/1561327258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da0b895070 0x55da0bc380e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:40.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.289+0000 7f8047a8e740 1 --2- 172.21.15.16:0/1561327258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da0b874390 0x55da0bc52320 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:40.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.289+0000 7f8047a8e740 1 --2- 172.21.15.16:0/1561327258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da0b86c1b0 0x55da0bc400d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:40.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.289+0000 7f8047a8e740 1 -- 172.21.15.16:0/1561327258 >> 172.21.15.16:0/1561327258 conn(0x55da0b889460 msgr2=0x55da0bc3f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:40.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.290+0000 7f8047a8e740 1 -- 172.21.15.16:0/1561327258 shutdown_connections 2024-08-31T06:11:40.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.290+0000 7f8047a8e740 1 -- 172.21.15.16:0/1561327258 wait complete. 2024-08-31T06:11:40.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.291+0000 7f8047a8e740 1 -- start start 2024-08-31T06:11:40.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.292+0000 7f8047a8e740 1 Processor -- start 2024-08-31T06:11:40.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.295+0000 7f8047a8e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da0b86c1b0 0x55da0bc58f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:40.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.296+0000 7f8047a8e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da0b874390 0x55da0bc594c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:40.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.296+0000 7f8047a8e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da0b895070 0x55da0bcbbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:40.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.296+0000 7f8047a8e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f801c00cd10 con 0x55da0b86c1b0 2024-08-31T06:11:40.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.296+0000 7f8047a8e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f801c00e180 con 0x55da0b874390 2024-08-31T06:11:40.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.296+0000 7f8047a8e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f801c00d3e0 con 0x55da0b895070 2024-08-31T06:11:40.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.296+0000 7f8047a8e740 10 client.0 _ll_register_callbacks cb 0x7f8034009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:11:40.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.297+0000 7f8039400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da0b874390 0x55da0bc594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:40.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.297+0000 7f8039400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da0b874390 0x55da0bc594c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:55388/0 (socket says 172.21.15.16:55388) 2024-08-31T06:11:40.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.297+0000 7f8039400640 1 -- 172.21.15.16:0/3464545710 learned_addr learned my addr 172.21.15.16:0/3464545710 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:40.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.298+0000 7f803a800640 1 --2- 172.21.15.16:0/3464545710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da0b895070 0x55da0bcbbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:40.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.298+0000 7f803a800640 1 -- 172.21.15.16:0/3464545710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da0b874390 msgr2=0x55da0bc594c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:40.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.298+0000 7f803a800640 1 --2- 172.21.15.16:0/3464545710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da0b874390 0x55da0bc594c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:40.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.298+0000 7f803a800640 1 -- 172.21.15.16:0/3464545710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da0b86c1b0 msgr2=0x55da0bc58f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:11:40.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.298+0000 7f8039e00640 1 --2- 172.21.15.16:0/3464545710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da0b86c1b0 0x55da0bc58f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:40.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.299+0000 7f803a800640 1 --2- 172.21.15.16:0/3464545710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da0b86c1b0 0x55da0bc58f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:40.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.299+0000 7f803a800640 1 -- 172.21.15.16:0/3464545710 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da0bcbc1e0 con 0x55da0b895070 2024-08-31T06:11:40.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.299+0000 7f8039e00640 1 --2- 172.21.15.16:0/3464545710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da0b86c1b0 0x55da0bc58f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:11:40.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.300+0000 7f803a800640 1 --2- 172.21.15.16:0/3464545710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da0b895070 0x55da0bcbbba0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f803402abe0 tx=0x7f80340306e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f403d9d081967434 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:40.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.300+0000 7f801be00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:11:40.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.301+0000 7f801be00640 1 -- 172.21.15.16:0/3464545710 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803405a020 con 0x55da0b895070 2024-08-31T06:11:40.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.301+0000 7f801be00640 1 -- 172.21.15.16:0/3464545710 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f80340318a0 con 0x55da0b895070 2024-08-31T06:11:40.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.301+0000 7f8047a8e740 1 -- 172.21.15.16:0/3464545710 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80280b06f0 con 0x55da0b895070 2024-08-31T06:11:40.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.302+0000 7f801be00640 1 -- 172.21.15.16:0/3464545710 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8034040040 con 0x55da0b895070 2024-08-31T06:11:40.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.302+0000 7f8047a8e740 10 client.26614 fetch_fsmap learned FSMap version 34 2024-08-31T06:11:40.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.302+0000 7f8047a8e740 1 -- 172.21.15.16:0/3464545710 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55da0bcc3510 con 0x55da0b895070 2024-08-31T06:11:40.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.304+0000 7f801be00640 1 -- 172.21.15.16:0/3464545710 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80280b06f0 con 0x55da0b895070 2024-08-31T06:11:40.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.304+0000 7f801be00640 1 -- 172.21.15.16:0/3464545710 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f8034056af0 con 0x55da0b895070 2024-08-31T06:11:40.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.304+0000 7f8047a8e740 10 client.26614 fetch_fsmap finished waiting for FSMap version 34 2024-08-31T06:11:40.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.304+0000 7f8047a8e740 10 client.26614 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:11:40.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.304+0000 7f8047a8e740 20 client.26614 populate_metadata read hostname 'smithi016' 2024-08-31T06:11:40.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.304+0000 7f8047a8e740 1 --2- 172.21.15.16:0/3464545710 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55da0b828da0 0x55da0bcc5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:40.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.305+0000 7f8047a8e740 4 client.26614 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:11:40.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.305+0000 7f8047a8e740 1 -- 172.21.15.16:0/3464545710 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55da0bbaf1a0 con 0x55da0b828da0 2024-08-31T06:11:40.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.305+0000 7f8039e00640 1 --2- 172.21.15.16:0/3464545710 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55da0b828da0 0x55da0bcc5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:40.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.306+0000 7f8039e00640 1 --2- 172.21.15.16:0/3464545710 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55da0b828da0 0x55da0bcc5bf0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a311576b02d7c83 server_cookie=7a0a787122aaaabd in_seq=0 out_seq=0 2024-08-31T06:11:40.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.306+0000 7f801be00640 10 client.26614 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:11:40.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.307+0000 7f801be00640 1 -- 172.21.15.16:0/3464545710 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55da0bbaf1a0 con 0x55da0b828da0 2024-08-31T06:11:40.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.307+0000 7f801be00640 10 client.26614 handle_command_reply: tid=0 2024-08-31T06:11:40.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.307+0000 7f8047a8e740 1 client.26614 shutdown 2024-08-31T06:11:40.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.308+0000 7f8047a8e740 1 -- 172.21.15.16:0/3464545710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da0b895070 msgr2=0x55da0bcbbba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:40.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.308+0000 7f8047a8e740 1 --2- 172.21.15.16:0/3464545710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da0b895070 0x55da0bcbbba0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f803402abe0 tx=0x7f80340306e0 comp rx=0 tx=0).stop 2024-08-31T06:11:40.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.308+0000 7f8047a8e740 1 -- 172.21.15.16:0/3464545710 shutdown_connections 2024-08-31T06:11:40.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.308+0000 7f8047a8e740 1 --2- 172.21.15.16:0/3464545710 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55da0b828da0 0x55da0bcc5bf0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:40.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.308+0000 7f8047a8e740 1 --2- 172.21.15.16:0/3464545710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da0b895070 0x55da0bcbbba0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:40.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.308+0000 7f8047a8e740 1 --2- 172.21.15.16:0/3464545710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da0b874390 0x55da0bc594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:40.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.308+0000 7f801be00640 0 client.26614 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:11:40.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.308+0000 7f8047a8e740 1 --2- 172.21.15.16:0/3464545710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da0b86c1b0 0x55da0bc58f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:40.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.308+0000 7f8047a8e740 1 -- 172.21.15.16:0/3464545710 >> 172.21.15.16:0/3464545710 conn(0x55da0b889460 msgr2=0x55da0bc38030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:40.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.308+0000 7f8047a8e740 1 -- 172.21.15.16:0/3464545710 shutdown_connections 2024-08-31T06:11:40.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.309+0000 7f8047a8e740 1 -- 172.21.15.16:0/3464545710 wait complete. 2024-08-31T06:11:40.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.309+0000 7f8047a8e740 20 client.26614 trim_cache size 0 max 16384 2024-08-31T06:11:40.312 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:11:40.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.310+0000 7f8040e00640 1 -- 172.21.15.16:0/2997958799 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f80200b4da0 msgr2=0x7f80200b72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:40.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.310+0000 7f8040e00640 1 --2- 172.21.15.16:0/2997958799 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f80200b4da0 0x7f80200b72b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x55da0bc39530 tx=0x7f802c011040 comp rx=0 tx=0).stop 2024-08-31T06:11:40.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.310+0000 7f8040e00640 1 -- 172.21.15.16:0/2997958799 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f803c109240 msgr2=0x7f803c112320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:40.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.310+0000 7f8040e00640 1 --2- 172.21.15.16:0/2997958799 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f803c109240 0x7f803c112320 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f802800e9f0 tx=0x7f802800eeb0 comp rx=0 tx=0).stop 2024-08-31T06:11:40.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.310+0000 7f8040e00640 1 -- 172.21.15.16:0/2997958799 shutdown_connections 2024-08-31T06:11:40.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.310+0000 7f8040e00640 1 --2- 172.21.15.16:0/2997958799 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f80200b4da0 0x7f80200b72b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:40.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.310+0000 7f8040e00640 1 --2- 172.21.15.16:0/2997958799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f803c109b90 0x7f803c112860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:40.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.310+0000 7f8040e00640 1 --2- 172.21.15.16:0/2997958799 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f803c109240 0x7f803c112320 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:40.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.310+0000 7f8040e00640 1 --2- 172.21.15.16:0/2997958799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f803c103900 0x7f803c119050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:40.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.310+0000 7f8040e00640 1 -- 172.21.15.16:0/2997958799 >> 172.21.15.16:0/2997958799 conn(0x7f803c07b5f0 msgr2=0x7f803c107290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:40.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.311+0000 7f8040e00640 1 -- 172.21.15.16:0/2997958799 shutdown_connections 2024-08-31T06:11:40.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:40.311+0000 7f8040e00640 1 -- 172.21.15.16:0/2997958799 wait complete. 2024-08-31T06:11:41.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:40 smithi155 ceph-mon[21742]: pgmap v497: 97 pgs: 97 active+clean; 2.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 10 MiB/s wr, 788 op/s 2024-08-31T06:11:41.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:40 smithi016 ceph-mon[17194]: pgmap v497: 97 pgs: 97 active+clean; 2.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 10 MiB/s wr, 788 op/s 2024-08-31T06:11:41.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:40 smithi184 ceph-mon[22331]: pgmap v497: 97 pgs: 97 active+clean; 2.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 10 MiB/s wr, 788 op/s 2024-08-31T06:11:41.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 255.0 s, 2032.4 tps, lat 48.989 ms stddev 56.837 2024-08-31T06:11:42.418 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T06:11:42.418 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T06:11:42.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:42 smithi016 ceph-mon[17194]: pgmap v498: 97 pgs: 97 active+clean; 2.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 9.0 MiB/s wr, 710 op/s 2024-08-31T06:11:43.071 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:11:43.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:42 smithi155 ceph-mon[21742]: pgmap v498: 97 pgs: 97 active+clean; 2.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 9.0 MiB/s wr, 710 op/s 2024-08-31T06:11:43.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:42 smithi184 ceph-mon[22331]: pgmap v498: 97 pgs: 97 active+clean; 2.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 9.0 MiB/s wr, 710 op/s 2024-08-31T06:11:43.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.887+0000 7fa3e5400640 1 -- 172.21.15.16:0/1168376165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3e0109ba0 msgr2=0x7fa3e01103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:43.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.887+0000 7fa3e5400640 1 --2- 172.21.15.16:0/1168376165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3e0109ba0 0x7fa3e01103d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8009be0 tx=0x7fa3d802a6c0 comp rx=0 tx=0).stop 2024-08-31T06:11:43.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.887+0000 7fa3e5400640 1 -- 172.21.15.16:0/1168376165 shutdown_connections 2024-08-31T06:11:43.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.887+0000 7fa3e5400640 1 --2- 172.21.15.16:0/1168376165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3e0109ba0 0x7fa3e01103d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:43.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.887+0000 7fa3e5400640 1 --2- 172.21.15.16:0/1168376165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3e0109250 0x7fa3e0109660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:43.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.887+0000 7fa3e5400640 1 --2- 172.21.15.16:0/1168376165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3e0103960 0x7fa3e0103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:43.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.887+0000 7fa3e5400640 1 -- 172.21.15.16:0/1168376165 >> 172.21.15.16:0/1168376165 conn(0x7fa3e007b6b0 msgr2=0x7fa3e007bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:43.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.889+0000 7fa3e5400640 1 -- 172.21.15.16:0/1168376165 shutdown_connections 2024-08-31T06:11:43.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.889+0000 7fa3e5400640 1 -- 172.21.15.16:0/1168376165 wait complete. 2024-08-31T06:11:43.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.890+0000 7fa3e5400640 1 Processor -- start 2024-08-31T06:11:43.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.891+0000 7fa3e5400640 1 -- start start 2024-08-31T06:11:43.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.891+0000 7fa3e5400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3e0103960 0x7fa3e019c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:43.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.891+0000 7fa3e5400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3e0109250 0x7fa3e019c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:43.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.891+0000 7fa3e5400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3e0109ba0 0x7fa3e01a1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:43.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.891+0000 7fa3e5400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e01138b0 con 0x7fa3e0109250 2024-08-31T06:11:43.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.891+0000 7fa3e5400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e0113730 con 0x7fa3e0109ba0 2024-08-31T06:11:43.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.891+0000 7fa3e5400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e0113a30 con 0x7fa3e0103960 2024-08-31T06:11:43.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.892+0000 7fa3de800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3e0103960 0x7fa3e019c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:43.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.892+0000 7fa3dde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3e0109250 0x7fa3e019c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:43.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.892+0000 7fa3de800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3e0103960 0x7fa3e019c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:32918/0 (socket says 172.21.15.16:32918) 2024-08-31T06:11:43.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.892+0000 7fa3de800640 1 -- 172.21.15.16:0/142142744 learned_addr learned my addr 172.21.15.16:0/142142744 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:43.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.892+0000 7fa3df200640 1 --2- 172.21.15.16:0/142142744 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3e0109ba0 0x7fa3e01a1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:43.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.893+0000 7fa3dde00640 1 -- 172.21.15.16:0/142142744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3e0103960 msgr2=0x7fa3e019c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:43.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.893+0000 7fa3dde00640 1 --2- 172.21.15.16:0/142142744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3e0103960 0x7fa3e019c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:43.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.893+0000 7fa3dde00640 1 -- 172.21.15.16:0/142142744 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3e0109ba0 msgr2=0x7fa3e01a1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:43.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.893+0000 7fa3dde00640 1 --2- 172.21.15.16:0/142142744 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3e0109ba0 0x7fa3e01a1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:43.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.893+0000 7fa3dde00640 1 -- 172.21.15.16:0/142142744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e01a2290 con 0x7fa3e0109250 2024-08-31T06:11:43.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.893+0000 7fa3dde00640 1 --2- 172.21.15.16:0/142142744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3e0109250 0x7fa3e019c980 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa3e006b8d0 tx=0x7fa3cc00cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:43.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.894+0000 7fa3d7400640 1 -- 172.21.15.16:0/142142744 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc007bb0 con 0x7fa3e0109250 2024-08-31T06:11:43.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.894+0000 7fa3e5400640 1 -- 172.21.15.16:0/142142744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3e01a2580 con 0x7fa3e0109250 2024-08-31T06:11:43.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.894+0000 7fa3d7400640 1 -- 172.21.15.16:0/142142744 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3cc007d50 con 0x7fa3e0109250 2024-08-31T06:11:43.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.894+0000 7fa3d7400640 1 -- 172.21.15.16:0/142142744 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc029700 con 0x7fa3e0109250 2024-08-31T06:11:43.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.895+0000 7fa3e5400640 1 -- 172.21.15.16:0/142142744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3e010a280 con 0x7fa3e0109250 2024-08-31T06:11:43.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.903+0000 7fa3d7400640 1 -- 172.21.15.16:0/142142744 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fa3cc027020 con 0x7fa3e0109250 2024-08-31T06:11:43.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.903+0000 7fa3ec045740 1 Processor -- start 2024-08-31T06:11:43.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.904+0000 7fa3ec045740 1 -- start start 2024-08-31T06:11:43.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.905+0000 7fa3ec045740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f78a508350 0x55f78a8ae600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:43.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.905+0000 7fa3ec045740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f78a4df490 0x55f78a8aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:43.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.905+0000 7fa3ec045740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f78a4e7670 0x55f78a8af080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:43.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.905+0000 7fa3ec045740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f78a824750 con 0x55f78a508350 2024-08-31T06:11:43.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.905+0000 7fa3ec045740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f78a813ce0 con 0x55f78a4df490 2024-08-31T06:11:43.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.905+0000 7fa3ec045740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f78a817c90 con 0x55f78a4e7670 2024-08-31T06:11:43.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.905+0000 7fa3de800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f78a4df490 0x55f78a8aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:43.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.905+0000 7fa3de800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f78a4df490 0x55f78a8aeb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:55420/0 (socket says 172.21.15.16:55420) 2024-08-31T06:11:43.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.905+0000 7fa3de800640 1 -- 172.21.15.16:0/1550892709 learned_addr learned my addr 172.21.15.16:0/1550892709 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:43.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.906+0000 7fa3dde00640 1 --2- 172.21.15.16:0/1550892709 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f78a4e7670 0x55f78a8af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:43.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.906+0000 7fa3dde00640 1 -- 172.21.15.16:0/1550892709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f78a4df490 msgr2=0x55f78a8aeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:43.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.906+0000 7fa3dde00640 1 --2- 172.21.15.16:0/1550892709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f78a4df490 0x55f78a8aeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:43.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.906+0000 7fa3dde00640 1 -- 172.21.15.16:0/1550892709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f78a508350 msgr2=0x55f78a8ae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:43.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.906+0000 7fa3dde00640 1 --2- 172.21.15.16:0/1550892709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f78a508350 0x55f78a8ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:43.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.906+0000 7fa3dde00640 1 -- 172.21.15.16:0/1550892709 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f78a573930 con 0x55f78a4e7670 2024-08-31T06:11:43.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.907+0000 7fa3d7400640 1 --2- 172.21.15.16:0/142142744 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa3c40b4d90 0x7fa3c40b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:43.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.907+0000 7fa3dde00640 1 --2- 172.21.15.16:0/1550892709 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f78a4e7670 0x55f78a8af080 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0f4940 tx=0x7fa3cc011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f97f25d7af224a7f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:43.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.907+0000 7fa3df200640 1 --2- 172.21.15.16:0/142142744 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa3c40b4d90 0x7fa3c40b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:43.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.908+0000 7fa3cb400640 1 -- 172.21.15.16:0/1550892709 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc0b97a0 con 0x55f78a4e7670 2024-08-31T06:11:43.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.908+0000 7fa3cb400640 1 -- 172.21.15.16:0/1550892709 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3cc0bd3f0 con 0x55f78a4e7670 2024-08-31T06:11:43.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.908+0000 7fa3cb400640 1 -- 172.21.15.16:0/1550892709 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc0bd050 con 0x55f78a4e7670 2024-08-31T06:11:43.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.908+0000 7fa3d7400640 1 -- 172.21.15.16:0/142142744 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fa3cc0f71f0 con 0x7fa3e0109250 2024-08-31T06:11:43.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.909+0000 7fa3df200640 1 --2- 172.21.15.16:0/142142744 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa3c40b4d90 0x7fa3c40b72a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa3d80099a0 tx=0x7fa3d80098f0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:43.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.909+0000 7fa3ec045740 1 -- 172.21.15.16:0/1550892709 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f78a4e7670 msgr2=0x55f78a8af080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:43.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.909+0000 7fa3ec045740 1 --2- 172.21.15.16:0/1550892709 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f78a4e7670 0x55f78a8af080 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0f4940 tx=0x7fa3cc011000 comp rx=0 tx=0).stop 2024-08-31T06:11:43.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.909+0000 7fa3ec045740 1 -- 172.21.15.16:0/1550892709 shutdown_connections 2024-08-31T06:11:43.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.909+0000 7fa3ec045740 1 --2- 172.21.15.16:0/1550892709 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f78a4e7670 0x55f78a8af080 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:43.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.909+0000 7fa3ec045740 1 --2- 172.21.15.16:0/1550892709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f78a4df490 0x55f78a8aeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:43.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.909+0000 7fa3ec045740 1 --2- 172.21.15.16:0/1550892709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f78a508350 0x55f78a8ae600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:43.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.909+0000 7fa3ec045740 1 -- 172.21.15.16:0/1550892709 >> 172.21.15.16:0/1550892709 conn(0x55f78a4fc740 msgr2=0x55f78a8ac4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:43.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.909+0000 7fa3ec045740 1 -- 172.21.15.16:0/1550892709 shutdown_connections 2024-08-31T06:11:43.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.910+0000 7fa3ec045740 1 -- 172.21.15.16:0/1550892709 wait complete. 2024-08-31T06:11:43.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.910+0000 7fa3ec045740 1 -- start start 2024-08-31T06:11:43.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.910+0000 7fa3ec045740 1 Processor -- start 2024-08-31T06:11:43.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.912+0000 7fa3ec045740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f78a4df490 0x55f78a8ba250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:43.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.912+0000 7fa3ec045740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f78a4e7670 0x55f78a8ba810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:43.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.912+0000 7fa3ec045740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f78a508350 0x55f78a8c6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:43.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.912+0000 7fa3ec045740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f78a676890 con 0x55f78a4e7670 2024-08-31T06:11:43.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.912+0000 7fa3ec045740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f78a79acd0 con 0x55f78a508350 2024-08-31T06:11:43.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.912+0000 7fa3ec045740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f78a8bce30 con 0x55f78a4df490 2024-08-31T06:11:43.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.912+0000 7fa3ec045740 10 client.0 _ll_register_callbacks cb 0x55f78a80eef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:11:43.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.913+0000 7fa3dde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f78a4e7670 0x55f78a8ba810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:43.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.913+0000 7fa3dde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f78a4e7670 0x55f78a8ba810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54096/0 (socket says 172.21.15.16:54096) 2024-08-31T06:11:43.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.913+0000 7fa3dde00640 1 -- 172.21.15.16:0/1932739264 learned_addr learned my addr 172.21.15.16:0/1932739264 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:43.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.913+0000 7fa3de800640 1 --2- 172.21.15.16:0/1932739264 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f78a4df490 0x55f78a8ba250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:43.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.914+0000 7fa3df200640 1 --2- 172.21.15.16:0/1932739264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f78a508350 0x55f78a8c6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:43.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.914+0000 7fa3dde00640 1 -- 172.21.15.16:0/1932739264 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f78a4df490 msgr2=0x55f78a8ba250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:43.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.914+0000 7fa3dde00640 1 --2- 172.21.15.16:0/1932739264 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f78a4df490 0x55f78a8ba250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:43.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.914+0000 7fa3dde00640 1 -- 172.21.15.16:0/1932739264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f78a508350 msgr2=0x55f78a8c6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:43.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.914+0000 7fa3dde00640 1 --2- 172.21.15.16:0/1932739264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f78a508350 0x55f78a8c6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:43.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.914+0000 7fa3dde00640 1 -- 172.21.15.16:0/1932739264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f78a7e41b0 con 0x55f78a4e7670 2024-08-31T06:11:43.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.915+0000 7fa3dde00640 1 --2- 172.21.15.16:0/1932739264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f78a4e7670 0x55f78a8ba810 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc00ffd0 tx=0x7fa3cc0b7ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c56660b67318b33a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:43.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.915+0000 7fa3bfe00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:11:43.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.915+0000 7fa3bfe00640 1 -- 172.21.15.16:0/1932739264 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc06d020 con 0x55f78a4e7670 2024-08-31T06:11:43.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.916+0000 7fa3bfe00640 1 -- 172.21.15.16:0/1932739264 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fa3cc0fa2e0 con 0x55f78a4e7670 2024-08-31T06:11:43.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.916+0000 7fa3ec045740 1 -- 172.21.15.16:0/1932739264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3cc0bd3f0 con 0x55f78a4e7670 2024-08-31T06:11:43.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.916+0000 7fa3bfe00640 1 -- 172.21.15.16:0/1932739264 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa3cc0bd230 con 0x55f78a4e7670 2024-08-31T06:11:43.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.916+0000 7fa3ec045740 10 client.16467 fetch_fsmap learned FSMap version 34 2024-08-31T06:11:43.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.916+0000 7fa3ec045740 1 -- 172.21.15.16:0/1932739264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f78a8c8ed0 con 0x55f78a4e7670 2024-08-31T06:11:43.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.916+0000 7fa3bfe00640 1 -- 172.21.15.16:0/1932739264 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3cc0bd3f0 con 0x55f78a4e7670 2024-08-31T06:11:43.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.917+0000 7fa3bfe00640 1 -- 172.21.15.16:0/1932739264 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7fa3cc0bdb40 con 0x55f78a4e7670 2024-08-31T06:11:43.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.917+0000 7fa3ec045740 10 client.16467 fetch_fsmap finished waiting for FSMap version 34 2024-08-31T06:11:43.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.917+0000 7fa3ec045740 10 client.16467 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:11:43.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.917+0000 7fa3ec045740 20 client.16467 populate_metadata read hostname 'smithi016' 2024-08-31T06:11:43.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.917+0000 7fa3ec045740 1 --2- 172.21.15.16:0/1932739264 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f78a4c6cd0 0x55f78a8cc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:43.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.917+0000 7fa3ec045740 4 client.16467 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:11:43.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.917+0000 7fa3ec045740 1 -- 172.21.15.16:0/1932739264 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f78a8266a0 con 0x55f78a4c6cd0 2024-08-31T06:11:43.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.917+0000 7fa3de800640 1 --2- 172.21.15.16:0/1932739264 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f78a4c6cd0 0x55f78a8cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:43.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.918+0000 7fa3de800640 1 --2- 172.21.15.16:0/1932739264 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f78a4c6cd0 0x55f78a8cc230 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=914c6a4e1a7f0521 server_cookie=32f1046f8ef10e48 in_seq=0 out_seq=0 2024-08-31T06:11:43.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.919+0000 7fa3bfe00640 10 client.16467 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:11:43.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.919+0000 7fa3bfe00640 1 -- 172.21.15.16:0/1932739264 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f78a8266a0 con 0x55f78a4c6cd0 2024-08-31T06:11:43.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.919+0000 7fa3bfe00640 10 client.16467 handle_command_reply: tid=0 2024-08-31T06:11:43.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.920+0000 7fa3ec045740 1 client.16467 shutdown 2024-08-31T06:11:43.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.922+0000 7fa3ec045740 1 -- 172.21.15.16:0/1932739264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f78a4e7670 msgr2=0x55f78a8ba810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:43.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.922+0000 7fa3ec045740 1 --2- 172.21.15.16:0/1932739264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f78a4e7670 0x55f78a8ba810 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc00ffd0 tx=0x7fa3cc0b7ef0 comp rx=0 tx=0).stop 2024-08-31T06:11:43.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.922+0000 7fa3ec045740 1 -- 172.21.15.16:0/1932739264 shutdown_connections 2024-08-31T06:11:43.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.922+0000 7fa3ec045740 1 --2- 172.21.15.16:0/1932739264 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f78a4c6cd0 0x55f78a8cc230 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:43.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.922+0000 7fa3ec045740 1 --2- 172.21.15.16:0/1932739264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f78a508350 0x55f78a8c6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:43.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.922+0000 7fa3ec045740 1 --2- 172.21.15.16:0/1932739264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f78a4e7670 0x55f78a8ba810 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:43.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.922+0000 7fa3ec045740 1 --2- 172.21.15.16:0/1932739264 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f78a4df490 0x55f78a8ba250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:43.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.922+0000 7fa3ec045740 1 -- 172.21.15.16:0/1932739264 >> 172.21.15.16:0/1932739264 conn(0x55f78a4fc740 msgr2=0x55f78a8b17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:43.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.922+0000 7fa3bfe00640 0 client.16467 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:11:43.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.923+0000 7fa3ec045740 1 -- 172.21.15.16:0/1932739264 shutdown_connections 2024-08-31T06:11:43.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.923+0000 7fa3ec045740 1 -- 172.21.15.16:0/1932739264 wait complete. 2024-08-31T06:11:43.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.923+0000 7fa3ec045740 20 client.16467 trim_cache size 0 max 16384 2024-08-31T06:11:43.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.952+0000 7fa3ec045740 1 Processor -- start 2024-08-31T06:11:43.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.952+0000 7fa3ec045740 1 -- start start 2024-08-31T06:11:43.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.953+0000 7fa3ec045740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f78a4df490 0x55f78a8b36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:43.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.953+0000 7fa3ec045740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f78a4e7670 0x55f78a8b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:43.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.953+0000 7fa3ec045740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f78a508350 0x55f78a8b1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:43.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.953+0000 7fa3ec045740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e01afa20 con 0x55f78a4e7670 2024-08-31T06:11:43.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.953+0000 7fa3ec045740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e01ad2c0 con 0x55f78a4df490 2024-08-31T06:11:43.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.953+0000 7fa3ec045740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e01ab150 con 0x55f78a508350 2024-08-31T06:11:43.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.954+0000 7fa3de800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f78a4df490 0x55f78a8b36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:43.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.954+0000 7fa3de800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f78a4df490 0x55f78a8b36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:55432/0 (socket says 172.21.15.16:55432) 2024-08-31T06:11:43.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.954+0000 7fa3de800640 1 -- 172.21.15.16:0/3683443148 learned_addr learned my addr 172.21.15.16:0/3683443148 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:43.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.955+0000 7fa3dde00640 1 --2- 172.21.15.16:0/3683443148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f78a4e7670 0x55f78a8b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:43.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.955+0000 7fa3df200640 1 --2- 172.21.15.16:0/3683443148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f78a508350 0x55f78a8b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:43.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.955+0000 7fa3de800640 1 -- 172.21.15.16:0/3683443148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f78a508350 msgr2=0x55f78a8b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:43.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.955+0000 7fa3de800640 1 --2- 172.21.15.16:0/3683443148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f78a508350 0x55f78a8b1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:43.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.955+0000 7fa3de800640 1 -- 172.21.15.16:0/3683443148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f78a4e7670 msgr2=0x55f78a8b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:43.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.955+0000 7fa3de800640 1 --2- 172.21.15.16:0/3683443148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f78a4e7670 0x55f78a8b3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:43.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.955+0000 7fa3de800640 1 -- 172.21.15.16:0/3683443148 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f78a8b58e0 con 0x55f78a4df490 2024-08-31T06:11:43.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.956+0000 7fa3de800640 1 --2- 172.21.15.16:0/3683443148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f78a4df490 0x55f78a8b36e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x55f78a8ad280 tx=0x7fa3d00044d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=753cd46204454ad4 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:43.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.956+0000 7fa3caa00640 1 -- 172.21.15.16:0/3683443148 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0003e20 con 0x55f78a4df490 2024-08-31T06:11:43.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.956+0000 7fa3caa00640 1 -- 172.21.15.16:0/3683443148 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3d00054d0 con 0x55f78a4df490 2024-08-31T06:11:43.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.956+0000 7fa3caa00640 1 -- 172.21.15.16:0/3683443148 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0016390 con 0x55f78a4df490 2024-08-31T06:11:43.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.957+0000 7fa3ec045740 1 -- 172.21.15.16:0/3683443148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f78a4df490 msgr2=0x55f78a8b36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:43.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.957+0000 7fa3ec045740 1 --2- 172.21.15.16:0/3683443148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f78a4df490 0x55f78a8b36e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x55f78a8ad280 tx=0x7fa3d00044d0 comp rx=0 tx=0).stop 2024-08-31T06:11:43.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.958+0000 7fa3ec045740 1 -- 172.21.15.16:0/3683443148 shutdown_connections 2024-08-31T06:11:43.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.958+0000 7fa3ec045740 1 --2- 172.21.15.16:0/3683443148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f78a508350 0x55f78a8b1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:43.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.958+0000 7fa3ec045740 1 --2- 172.21.15.16:0/3683443148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f78a4e7670 0x55f78a8b3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:43.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.958+0000 7fa3ec045740 1 --2- 172.21.15.16:0/3683443148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f78a4df490 0x55f78a8b36e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:43.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.958+0000 7fa3ec045740 1 -- 172.21.15.16:0/3683443148 >> 172.21.15.16:0/3683443148 conn(0x55f78a4fc740 msgr2=0x55f78a8b88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:43.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.958+0000 7fa3ec045740 1 -- 172.21.15.16:0/3683443148 shutdown_connections 2024-08-31T06:11:43.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.958+0000 7fa3ec045740 1 -- 172.21.15.16:0/3683443148 wait complete. 2024-08-31T06:11:43.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.959+0000 7fa3ec045740 1 -- start start 2024-08-31T06:11:43.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.959+0000 7fa3ec045740 1 Processor -- start 2024-08-31T06:11:43.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.960+0000 7fa3ec045740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f78a4df490 0x55f78a8cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:43.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.960+0000 7fa3ec045740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f78a4e7670 0x55f78a8cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:43.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.960+0000 7fa3ec045740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f78a508350 0x55f78a8cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:43.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.960+0000 7fa3ec045740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e01ab7a0 con 0x55f78a4e7670 2024-08-31T06:11:43.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.960+0000 7fa3ec045740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e01acbf0 con 0x55f78a4df490 2024-08-31T06:11:43.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.960+0000 7fa3ec045740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e01abe50 con 0x55f78a508350 2024-08-31T06:11:43.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.960+0000 7fa3ec045740 10 client.0 _ll_register_callbacks cb 0x7fa3cc0fa030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:11:43.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.961+0000 7fa3de800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f78a4df490 0x55f78a8cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:43.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.961+0000 7fa3de800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f78a4df490 0x55f78a8cd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:55442/0 (socket says 172.21.15.16:55442) 2024-08-31T06:11:43.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.961+0000 7fa3de800640 1 -- 172.21.15.16:0/3359445243 learned_addr learned my addr 172.21.15.16:0/3359445243 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:43.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.961+0000 7fa3de800640 1 -- 172.21.15.16:0/3359445243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f78a508350 msgr2=0x55f78a8cdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:43.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.961+0000 7fa3dde00640 1 --2- 172.21.15.16:0/3359445243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f78a4e7670 0x55f78a8cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:43.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.961+0000 7fa3df200640 1 --2- 172.21.15.16:0/3359445243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f78a508350 0x55f78a8cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:43.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.962+0000 7fa3de800640 1 --2- 172.21.15.16:0/3359445243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f78a508350 0x55f78a8cdeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:43.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.962+0000 7fa3de800640 1 -- 172.21.15.16:0/3359445243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f78a4e7670 msgr2=0x55f78a8cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:43.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.962+0000 7fa3de800640 1 --2- 172.21.15.16:0/3359445243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f78a4e7670 0x55f78a8cd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:43.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.962+0000 7fa3de800640 1 -- 172.21.15.16:0/3359445243 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f78a935b30 con 0x55f78a4df490 2024-08-31T06:11:43.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.962+0000 7fa3df200640 1 --2- 172.21.15.16:0/3359445243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f78a508350 0x55f78a8cdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:11:43.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.962+0000 7fa3dde00640 1 --2- 172.21.15.16:0/3359445243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f78a4e7670 0x55f78a8cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:11:43.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.963+0000 7fa3de800640 1 --2- 172.21.15.16:0/3359445243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f78a4df490 0x55f78a8cd430 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0009ef0 tx=0x7fa3d0009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b66a19318592651c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:43.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.963+0000 7fa3bfe00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:11:43.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.963+0000 7fa3bfe00640 1 -- 172.21.15.16:0/3359445243 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d00050c0 con 0x55f78a4df490 2024-08-31T06:11:43.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.963+0000 7fa3bfe00640 1 -- 172.21.15.16:0/3359445243 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fa3d0005260 con 0x55f78a4df490 2024-08-31T06:11:43.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.963+0000 7fa3ec045740 1 -- 172.21.15.16:0/3359445243 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3d00054d0 con 0x55f78a4df490 2024-08-31T06:11:43.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.964+0000 7fa3bfe00640 1 -- 172.21.15.16:0/3359445243 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa3d0004730 con 0x55f78a4df490 2024-08-31T06:11:43.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.964+0000 7fa3ec045740 10 client.26630 fetch_fsmap learned FSMap version 34 2024-08-31T06:11:43.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.964+0000 7fa3ec045740 1 -- 172.21.15.16:0/3359445243 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f78a936df0 con 0x55f78a4df490 2024-08-31T06:11:43.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.964+0000 7fa3bfe00640 1 -- 172.21.15.16:0/3359445243 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3d00054d0 con 0x55f78a4df490 2024-08-31T06:11:43.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.965+0000 7fa3bfe00640 1 -- 172.21.15.16:0/3359445243 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7fa3d0028070 con 0x55f78a4df490 2024-08-31T06:11:43.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.965+0000 7fa3ec045740 10 client.26630 fetch_fsmap finished waiting for FSMap version 34 2024-08-31T06:11:43.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.965+0000 7fa3ec045740 10 client.26630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:11:43.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.965+0000 7fa3ec045740 20 client.26630 populate_metadata read hostname 'smithi016' 2024-08-31T06:11:43.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.965+0000 7fa3ec045740 1 --2- 172.21.15.16:0/3359445243 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f78a49c080 0x55f78a939550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:43.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.965+0000 7fa3ec045740 4 client.26630 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T06:11:43.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.965+0000 7fa3ec045740 1 -- 172.21.15.16:0/3359445243 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f78a8c6bb0 con 0x55f78a49c080 2024-08-31T06:11:43.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.965+0000 7fa3dde00640 1 --2- 172.21.15.16:0/3359445243 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f78a49c080 0x55f78a939550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:43.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.967+0000 7fa3dde00640 1 --2- 172.21.15.16:0/3359445243 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f78a49c080 0x55f78a939550 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e831b7460f2b8c8b server_cookie=6031b60cbe88b007 in_seq=0 out_seq=0 2024-08-31T06:11:43.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.967+0000 7fa3bfe00640 10 client.26630 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:11:43.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.969+0000 7fa3bfe00640 1 -- 172.21.15.16:0/3359445243 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f78a8c6bb0 con 0x55f78a49c080 2024-08-31T06:11:43.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.969+0000 7fa3bfe00640 10 client.26630 handle_command_reply: tid=0 2024-08-31T06:11:43.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.969+0000 7fa3ec045740 1 client.26630 shutdown 2024-08-31T06:11:43.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.969+0000 7fa3ec045740 1 -- 172.21.15.16:0/3359445243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f78a4df490 msgr2=0x55f78a8cd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:43.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.969+0000 7fa3ec045740 1 --2- 172.21.15.16:0/3359445243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f78a4df490 0x55f78a8cd430 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0009ef0 tx=0x7fa3d0009b60 comp rx=0 tx=0).stop 2024-08-31T06:11:43.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.969+0000 7fa3ec045740 1 -- 172.21.15.16:0/3359445243 shutdown_connections 2024-08-31T06:11:43.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.969+0000 7fa3ec045740 1 --2- 172.21.15.16:0/3359445243 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f78a49c080 0x55f78a939550 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:43.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.969+0000 7fa3ec045740 1 --2- 172.21.15.16:0/3359445243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f78a508350 0x55f78a8cdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:43.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.969+0000 7fa3ec045740 1 --2- 172.21.15.16:0/3359445243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f78a4e7670 0x55f78a8cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:43.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.969+0000 7fa3ec045740 1 --2- 172.21.15.16:0/3359445243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f78a4df490 0x55f78a8cd430 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:43.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.969+0000 7fa3ec045740 1 -- 172.21.15.16:0/3359445243 >> 172.21.15.16:0/3359445243 conn(0x55f78a4fc740 msgr2=0x55f78a8b1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:43.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.969+0000 7fa3bfe00640 0 client.26630 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:11:43.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.970+0000 7fa3ec045740 1 -- 172.21.15.16:0/3359445243 shutdown_connections 2024-08-31T06:11:43.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.970+0000 7fa3ec045740 1 -- 172.21.15.16:0/3359445243 wait complete. 2024-08-31T06:11:43.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.970+0000 7fa3ec045740 20 client.26630 trim_cache size 0 max 16384 2024-08-31T06:11:43.973 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:11:43.973 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-08-31T06:11:43.973 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "a10cce76-99ab-46b5-b3b2-07b82899b8f9", 2024-08-31T06:11:43.974 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-08-31T06:11:43.974 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:11:43.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.972+0000 7fa3e5400640 1 -- 172.21.15.16:0/142142744 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa3c40b4d90 msgr2=0x7fa3c40b72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:43.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.972+0000 7fa3e5400640 1 --2- 172.21.15.16:0/142142744 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa3c40b4d90 0x7fa3c40b72a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa3d80099a0 tx=0x7fa3d80098f0 comp rx=0 tx=0).stop 2024-08-31T06:11:43.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.972+0000 7fa3e5400640 1 -- 172.21.15.16:0/142142744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3e0109250 msgr2=0x7fa3e019c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:43.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.972+0000 7fa3e5400640 1 --2- 172.21.15.16:0/142142744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3e0109250 0x7fa3e019c980 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa3e006b8d0 tx=0x7fa3cc00cdd0 comp rx=0 tx=0).stop 2024-08-31T06:11:43.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.972+0000 7fa3e5400640 1 -- 172.21.15.16:0/142142744 shutdown_connections 2024-08-31T06:11:43.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.972+0000 7fa3e5400640 1 --2- 172.21.15.16:0/142142744 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa3c40b4d90 0x7fa3c40b72a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:43.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.972+0000 7fa3e5400640 1 --2- 172.21.15.16:0/142142744 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3e0109ba0 0x7fa3e01a1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:43.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.972+0000 7fa3e5400640 1 --2- 172.21.15.16:0/142142744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3e0109250 0x7fa3e019c980 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:43.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.972+0000 7fa3e5400640 1 --2- 172.21.15.16:0/142142744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3e0103960 0x7fa3e019c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:43.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.972+0000 7fa3e5400640 1 -- 172.21.15.16:0/142142744 >> 172.21.15.16:0/142142744 conn(0x7fa3e007b6b0 msgr2=0x7fa3e0104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:43.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.973+0000 7fa3e5400640 1 -- 172.21.15.16:0/142142744 shutdown_connections 2024-08-31T06:11:43.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:43.973+0000 7fa3e5400640 1 -- 172.21.15.16:0/142142744 wait complete. 2024-08-31T06:11:45.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:44 smithi155 ceph-mon[21742]: pgmap v499: 97 pgs: 97 active+clean; 2.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 9.5 MiB/s wr, 752 op/s 2024-08-31T06:11:45.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:44 smithi155 ceph-mon[21742]: scrub queued for path: / 2024-08-31T06:11:45.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:44 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:11:45.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:44 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:11:45.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:44 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:11:45.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:44 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:11:45.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:44 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:11:45.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:44 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:11:45.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:44 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:11:45.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:44 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:11:45.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:44 smithi016 ceph-mon[17194]: pgmap v499: 97 pgs: 97 active+clean; 2.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 9.5 MiB/s wr, 752 op/s 2024-08-31T06:11:45.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:44 smithi016 ceph-mon[17194]: scrub queued for path: / 2024-08-31T06:11:45.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:44 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:11:45.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:44 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:11:45.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:44 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:11:45.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:44 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:11:45.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:44 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:11:45.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:44 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:11:45.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:44 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:11:45.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:44 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:11:45.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:44 smithi184 ceph-mon[22331]: pgmap v499: 97 pgs: 97 active+clean; 2.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 9.5 MiB/s wr, 752 op/s 2024-08-31T06:11:45.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:44 smithi184 ceph-mon[22331]: scrub queued for path: / 2024-08-31T06:11:45.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:44 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:11:45.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:44 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:11:45.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:44 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:11:45.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:44 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:11:45.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:44 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:11:45.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:44 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:11:45.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:44 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:11:45.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:44 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:11:45.337 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.335+0000 7f2543a00640 1 -- 172.21.15.16:0/943928143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f253c108080 msgr2=0x7f253c108490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:45.337 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.335+0000 7f2543a00640 1 --2- 172.21.15.16:0/943928143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f253c108080 0x7f253c108490 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2528009900 tx=0x7f252802a410 comp rx=0 tx=0).stop 2024-08-31T06:11:45.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.335+0000 7f2543a00640 1 -- 172.21.15.16:0/943928143 shutdown_connections 2024-08-31T06:11:45.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.335+0000 7f2543a00640 1 --2- 172.21.15.16:0/943928143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f253c1089d0 0x7f253c10f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:45.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.335+0000 7f2543a00640 1 --2- 172.21.15.16:0/943928143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f253c108080 0x7f253c108490 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:45.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.335+0000 7f2543a00640 1 --2- 172.21.15.16:0/943928143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f253c102750 0x7f253c102b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:45.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.335+0000 7f2543a00640 1 -- 172.21.15.16:0/943928143 >> 172.21.15.16:0/943928143 conn(0x7f253c07a4e0 msgr2=0x7f253c07a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:45.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.336+0000 7f2543a00640 1 -- 172.21.15.16:0/943928143 shutdown_connections 2024-08-31T06:11:45.339 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.336+0000 7f2543a00640 1 -- 172.21.15.16:0/943928143 wait complete. 2024-08-31T06:11:45.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.338+0000 7f2543a00640 1 Processor -- start 2024-08-31T06:11:45.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.338+0000 7f2543a00640 1 -- start start 2024-08-31T06:11:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.339+0000 7f2543a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f253c102750 0x7f253c10cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:45.342 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.339+0000 7f2541000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f253c102750 0x7f253c10cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:45.342 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.339+0000 7f2541000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f253c102750 0x7f253c10cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:55462/0 (socket says 172.21.15.16:55462) 2024-08-31T06:11:45.342 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.339+0000 7f2543a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f253c108080 0x7f253c1091d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:45.342 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.339+0000 7f2541000640 1 -- 172.21.15.16:0/3283377482 learned_addr learned my addr 172.21.15.16:0/3283377482 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:45.342 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.339+0000 7f2543a00640 1 --2- 172.21.15.16:0/3283377482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f253c1089d0 0x7f253c109900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.340+0000 7f2543a00640 1 -- 172.21.15.16:0/3283377482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f253c1115d0 con 0x7f253c108080 2024-08-31T06:11:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.340+0000 7f2543a00640 1 -- 172.21.15.16:0/3283377482 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f253c111450 con 0x7f253c102750 2024-08-31T06:11:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.340+0000 7f2543a00640 1 -- 172.21.15.16:0/3283377482 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f253c111750 con 0x7f253c1089d0 2024-08-31T06:11:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.340+0000 7f2541a00640 1 --2- 172.21.15.16:0/3283377482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f253c1089d0 0x7f253c109900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.340+0000 7f253be00640 1 --2- 172.21.15.16:0/3283377482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f253c108080 0x7f253c1091d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.341+0000 7f2541000640 1 -- 172.21.15.16:0/3283377482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f253c1089d0 msgr2=0x7f253c109900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.341+0000 7f2541000640 1 --2- 172.21.15.16:0/3283377482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f253c1089d0 0x7f253c109900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.341+0000 7f2541000640 1 -- 172.21.15.16:0/3283377482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f253c108080 msgr2=0x7f253c1091d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.341+0000 7f2541000640 1 --2- 172.21.15.16:0/3283377482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f253c108080 0x7f253c1091d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.341+0000 7f2541000640 1 -- 172.21.15.16:0/3283377482 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f253c1ac910 con 0x7f253c102750 2024-08-31T06:11:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.341+0000 7f2541000640 1 --2- 172.21.15.16:0/3283377482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f253c102750 0x7f253c10cf00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f253000ea60 tx=0x7f253000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.342+0000 7f2539600640 1 -- 172.21.15.16:0/3283377482 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253000cde0 con 0x7f253c102750 2024-08-31T06:11:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.342+0000 7f2539600640 1 -- 172.21.15.16:0/3283377482 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25300035f0 con 0x7f253c102750 2024-08-31T06:11:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.342+0000 7f2539600640 1 -- 172.21.15.16:0/3283377482 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2530010690 con 0x7f253c102750 2024-08-31T06:11:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.342+0000 7f2543a00640 1 -- 172.21.15.16:0/3283377482 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f253c1acb40 con 0x7f253c102750 2024-08-31T06:11:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.343+0000 7f2543a00640 1 -- 172.21.15.16:0/3283377482 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f253c1ad010 con 0x7f253c102750 2024-08-31T06:11:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.349+0000 7f254aa57740 1 Processor -- start 2024-08-31T06:11:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.350+0000 7f254aa57740 1 -- start start 2024-08-31T06:11:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.351+0000 7f254aa57740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c22cb11b0 0x558c23054920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.351+0000 7f254aa57740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c22c882f0 0x558c23054e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.351+0000 7f254aa57740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c22c904d0 0x558c230553a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.351+0000 7f254aa57740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558c22fc9f70 con 0x558c22cb11b0 2024-08-31T06:11:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.351+0000 7f254aa57740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x558c22fb9500 con 0x558c22c882f0 2024-08-31T06:11:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.351+0000 7f254aa57740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558c22fbd4b0 con 0x558c22c904d0 2024-08-31T06:11:45.355 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.351+0000 7f253be00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c22c904d0 0x558c230553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:45.355 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.351+0000 7f253be00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c22c904d0 0x558c230553a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:32990/0 (socket says 172.21.15.16:32990) 2024-08-31T06:11:45.355 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.351+0000 7f253be00640 1 -- 172.21.15.16:0/3397309103 learned_addr learned my addr 172.21.15.16:0/3397309103 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:45.355 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.352+0000 7f2541a00640 1 --2- 172.21.15.16:0/3397309103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c22cb11b0 0x558c23054920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:45.355 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.352+0000 7f253be00640 1 -- 172.21.15.16:0/3397309103 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c22c882f0 msgr2=0x558c23054e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:11:45.355 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.352+0000 7f253be00640 1 --2- 172.21.15.16:0/3397309103 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c22c882f0 0x558c23054e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:45.355 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.352+0000 7f253be00640 1 -- 172.21.15.16:0/3397309103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c22cb11b0 msgr2=0x558c23054920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:45.355 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.352+0000 7f253be00640 1 --2- 172.21.15.16:0/3397309103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c22cb11b0 0x558c23054920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.352+0000 7f253be00640 1 -- 172.21.15.16:0/3397309103 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c22f73c10 con 0x558c22c904d0 2024-08-31T06:11:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.352+0000 7f2541a00640 1 --2- 172.21.15.16:0/3397309103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c22cb11b0 0x558c23054920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:11:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.353+0000 7f2539600640 1 -- 172.21.15.16:0/3283377482 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f25300037a0 con 0x7f253c102750 2024-08-31T06:11:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.353+0000 7f253be00640 1 --2- 172.21.15.16:0/3397309103 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c22c904d0 0x558c230553a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f253c06c760 tx=0x7f2528004150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bacd77fb23cd2e01 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.353+0000 7f252e000640 1 -- 172.21.15.16:0/3397309103 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252803f070 con 0x558c22c904d0 2024-08-31T06:11:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.354+0000 7f252e000640 1 -- 172.21.15.16:0/3397309103 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2528005500 con 0x558c22c904d0 2024-08-31T06:11:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.354+0000 7f252e000640 1 -- 172.21.15.16:0/3397309103 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252803f070 con 0x558c22c904d0 2024-08-31T06:11:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.355+0000 7f254aa57740 1 -- 172.21.15.16:0/3397309103 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c22c904d0 msgr2=0x558c230553a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.355+0000 7f254aa57740 1 --2- 172.21.15.16:0/3397309103 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c22c904d0 0x558c230553a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f253c06c760 tx=0x7f2528004150 comp rx=0 tx=0).stop 2024-08-31T06:11:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.355+0000 7f2539600640 1 --2- 172.21.15.16:0/3283377482 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f25240b4df0 0x7f25240b7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.356+0000 7f2539600640 1 -- 172.21.15.16:0/3283377482 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f25300e5890 con 0x7f253c102750 2024-08-31T06:11:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.356+0000 7f254aa57740 1 -- 172.21.15.16:0/3397309103 shutdown_connections 2024-08-31T06:11:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.356+0000 7f254aa57740 1 --2- 172.21.15.16:0/3397309103 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c22c904d0 0x558c230553a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.356+0000 7f254aa57740 1 --2- 172.21.15.16:0/3397309103 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c22c882f0 0x558c23054e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.356+0000 7f254aa57740 1 --2- 172.21.15.16:0/3397309103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c22cb11b0 0x558c23054920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.356+0000 7f254aa57740 1 -- 172.21.15.16:0/3397309103 >> 172.21.15.16:0/3397309103 conn(0x558c22ca5300 msgr2=0x558c23052810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.356+0000 7f254aa57740 1 -- 172.21.15.16:0/3397309103 shutdown_connections 2024-08-31T06:11:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.357+0000 7f254aa57740 1 -- 172.21.15.16:0/3397309103 wait complete. 2024-08-31T06:11:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.357+0000 7f2541a00640 1 --2- 172.21.15.16:0/3283377482 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f25240b4df0 0x7f25240b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.357+0000 7f254aa57740 1 -- start start 2024-08-31T06:11:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.358+0000 7f254aa57740 1 Processor -- start 2024-08-31T06:11:45.363 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.359+0000 7f254aa57740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c22c882f0 0x558c2305ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:45.363 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.359+0000 7f254aa57740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c22c904d0 0x558c230604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:45.363 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.359+0000 7f254aa57740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c22cb11b0 0x558c2306c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:45.363 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.359+0000 7f254aa57740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558c22ef2ab0 con 0x558c22c904d0 2024-08-31T06:11:45.363 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.359+0000 7f254aa57740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x558c22f60320 con 0x558c22c882f0 2024-08-31T06:11:45.363 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.359+0000 7f254aa57740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558c23063040 con 0x558c22cb11b0 2024-08-31T06:11:45.363 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.359+0000 7f254aa57740 10 client.0 _ll_register_callbacks cb 0x558c22f666a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:11:45.363 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.360+0000 7f2541000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c22c882f0 0x558c2305ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:45.363 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.360+0000 7f2541000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c22c882f0 0x558c2305ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:55478/0 (socket says 172.21.15.16:55478) 2024-08-31T06:11:45.363 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.360+0000 7f2541000640 1 -- 172.21.15.16:0/170074419 learned_addr learned my addr 172.21.15.16:0/170074419 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:45.364 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.360+0000 7f2541000640 1 -- 172.21.15.16:0/170074419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c22cb11b0 msgr2=0x558c2306c360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:11:45.364 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.360+0000 7f2541000640 1 --2- 172.21.15.16:0/170074419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c22cb11b0 0x558c2306c360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:45.364 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.360+0000 7f2541000640 1 -- 172.21.15.16:0/170074419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c22c904d0 msgr2=0x558c230604b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:11:45.364 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.360+0000 7f2541000640 1 --2- 172.21.15.16:0/170074419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c22c904d0 0x558c230604b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:45.364 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.360+0000 7f2541000640 1 -- 172.21.15.16:0/170074419 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c22d1c790 con 0x558c22c882f0 2024-08-31T06:11:45.364 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.361+0000 7f2541000640 1 --2- 172.21.15.16:0/170074419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c22c882f0 0x558c2305ff70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f253000d010 tx=0x7f25300ab800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=756917d73795eb79 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:45.365 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.361+0000 7f2522a00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:11:45.365 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.361+0000 7f2522a00640 1 -- 172.21.15.16:0/170074419 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25300b07f0 con 0x558c22c882f0 2024-08-31T06:11:45.365 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.361+0000 7f2522a00640 1 -- 172.21.15.16:0/170074419 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f25300971b0 con 0x558c22c882f0 2024-08-31T06:11:45.365 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.361+0000 7f2541a00640 1 --2- 172.21.15.16:0/3283377482 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f25240b4df0 0x7f25240b7300 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2534005fd0 tx=0x7f253400b2c0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:45.365 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.362+0000 7f254aa57740 1 -- 172.21.15.16:0/170074419 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2528005500 con 0x558c22c882f0 2024-08-31T06:11:45.366 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.363+0000 7f2522a00640 1 -- 172.21.15.16:0/170074419 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f25300b52a0 con 0x558c22c882f0 2024-08-31T06:11:45.366 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.363+0000 7f254aa57740 10 client.26639 fetch_fsmap learned FSMap version 34 2024-08-31T06:11:45.366 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.363+0000 7f254aa57740 1 -- 172.21.15.16:0/170074419 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558c22d95620 con 0x558c22c882f0 2024-08-31T06:11:45.366 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.364+0000 7f2522a00640 1 -- 172.21.15.16:0/170074419 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2528005500 con 0x558c22c882f0 2024-08-31T06:11:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.364+0000 7f2522a00640 1 -- 172.21.15.16:0/170074419 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f2530004000 con 0x558c22c882f0 2024-08-31T06:11:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.364+0000 7f254aa57740 10 client.26639 fetch_fsmap finished waiting for FSMap version 34 2024-08-31T06:11:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.364+0000 7f254aa57740 10 client.26639 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:11:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.364+0000 7f254aa57740 20 client.26639 populate_metadata read hostname 'smithi016' 2024-08-31T06:11:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.364+0000 7f254aa57740 1 --2- 172.21.15.16:0/170074419 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x558c22c6fd80 0x558c230830b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.364+0000 7f254aa57740 4 client.26639 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:11:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.364+0000 7f254aa57740 1 -- 172.21.15.16:0/170074419 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c22fcbec0 con 0x558c22c6fd80 2024-08-31T06:11:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.365+0000 7f253be00640 1 --2- 172.21.15.16:0/170074419 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x558c22c6fd80 0x558c230830b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.365+0000 7f253be00640 1 --2- 172.21.15.16:0/170074419 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x558c22c6fd80 0x558c230830b0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=995099608fcd2d7 server_cookie=ea81f23fc88a4789 in_seq=0 out_seq=0 2024-08-31T06:11:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.365+0000 7f2522a00640 10 client.26639 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:11:45.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.366+0000 7f2522a00640 1 -- 172.21.15.16:0/170074419 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558c22fcbec0 con 0x558c22c6fd80 2024-08-31T06:11:45.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.366+0000 7f2522a00640 10 client.26639 handle_command_reply: tid=0 2024-08-31T06:11:45.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.367+0000 7f254aa57740 1 client.26639 shutdown 2024-08-31T06:11:45.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.367+0000 7f254aa57740 1 -- 172.21.15.16:0/170074419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c22c882f0 msgr2=0x558c2305ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:45.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.367+0000 7f254aa57740 1 --2- 172.21.15.16:0/170074419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c22c882f0 0x558c2305ff70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f253000d010 tx=0x7f25300ab800 comp rx=0 tx=0).stop 2024-08-31T06:11:45.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.367+0000 7f254aa57740 1 -- 172.21.15.16:0/170074419 shutdown_connections 2024-08-31T06:11:45.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.367+0000 7f254aa57740 1 --2- 172.21.15.16:0/170074419 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x558c22c6fd80 0x558c230830b0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:45.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.367+0000 7f254aa57740 1 --2- 172.21.15.16:0/170074419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c22cb11b0 0x558c2306c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:45.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.367+0000 7f254aa57740 1 --2- 172.21.15.16:0/170074419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c22c904d0 0x558c230604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:45.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.367+0000 7f254aa57740 1 --2- 172.21.15.16:0/170074419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c22c882f0 0x558c2305ff70 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:45.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.367+0000 7f254aa57740 1 -- 172.21.15.16:0/170074419 >> 172.21.15.16:0/170074419 conn(0x558c22ca5300 msgr2=0x558c23055950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:45.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.367+0000 7f2522a00640 0 client.26639 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:11:45.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.367+0000 7f254aa57740 1 -- 172.21.15.16:0/170074419 shutdown_connections 2024-08-31T06:11:45.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.368+0000 7f254aa57740 1 -- 172.21.15.16:0/170074419 wait complete. 2024-08-31T06:11:45.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.368+0000 7f254aa57740 20 client.26639 trim_cache size 0 max 16384 2024-08-31T06:11:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.394+0000 7f254aa57740 1 Processor -- start 2024-08-31T06:11:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.394+0000 7f254aa57740 1 -- start start 2024-08-31T06:11:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.394+0000 7f254aa57740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c22c882f0 0x558c23060470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.394+0000 7f254aa57740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c22c904d0 0x558c23061e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.394+0000 7f254aa57740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c22cb11b0 0x558c23056060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.394+0000 7f254aa57740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f253c1af870 con 0x558c22c904d0 2024-08-31T06:11:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.394+0000 7f254aa57740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f253c1ad240 con 0x558c22c882f0 2024-08-31T06:11:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.394+0000 7f254aa57740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f253c102b60 con 0x558c22cb11b0 2024-08-31T06:11:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.395+0000 7f2541a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c22cb11b0 0x558c23056060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.395+0000 7f2541000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c22c882f0 0x558c23060470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.395+0000 7f2541a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c22cb11b0 0x558c23056060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:33006/0 (socket says 172.21.15.16:33006) 2024-08-31T06:11:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.395+0000 7f2541a00640 1 -- 172.21.15.16:0/784218480 learned_addr learned my addr 172.21.15.16:0/784218480 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.395+0000 7f253be00640 1 --2- 172.21.15.16:0/784218480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c22c904d0 0x558c23061e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.395+0000 7f2541a00640 1 -- 172.21.15.16:0/784218480 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c22c882f0 msgr2=0x558c23060470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.395+0000 7f2541a00640 1 --2- 172.21.15.16:0/784218480 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c22c882f0 0x558c23060470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.395+0000 7f2541a00640 1 -- 172.21.15.16:0/784218480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c22c904d0 msgr2=0x558c23061e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.395+0000 7f2541a00640 1 --2- 172.21.15.16:0/784218480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c22c904d0 0x558c23061e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.395+0000 7f2541a00640 1 -- 172.21.15.16:0/784218480 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c2307dae0 con 0x558c22cb11b0 2024-08-31T06:11:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.395+0000 7f253be00640 1 --2- 172.21.15.16:0/784218480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c22c904d0 0x558c23061e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:11:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.395+0000 7f2541a00640 1 --2- 172.21.15.16:0/784218480 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c22cb11b0 0x558c23056060 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f253401a760 tx=0x7f2534013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ffcfafdd392428a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.396+0000 7f252d600640 1 -- 172.21.15.16:0/784218480 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2534028070 con 0x558c22cb11b0 2024-08-31T06:11:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.396+0000 7f252d600640 1 -- 172.21.15.16:0/784218480 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2534017a20 con 0x558c22cb11b0 2024-08-31T06:11:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.396+0000 7f252d600640 1 -- 172.21.15.16:0/784218480 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2534018960 con 0x558c22cb11b0 2024-08-31T06:11:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.397+0000 7f254aa57740 1 -- 172.21.15.16:0/784218480 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c22cb11b0 msgr2=0x558c23056060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.397+0000 7f254aa57740 1 --2- 172.21.15.16:0/784218480 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c22cb11b0 0x558c23056060 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f253401a760 tx=0x7f2534013000 comp rx=0 tx=0).stop 2024-08-31T06:11:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.398+0000 7f254aa57740 1 -- 172.21.15.16:0/784218480 shutdown_connections 2024-08-31T06:11:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.398+0000 7f254aa57740 1 --2- 172.21.15.16:0/784218480 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c22cb11b0 0x558c23056060 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.398+0000 7f254aa57740 1 --2- 172.21.15.16:0/784218480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c22c904d0 0x558c23061e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.398+0000 7f254aa57740 1 --2- 172.21.15.16:0/784218480 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c22c882f0 0x558c23060470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:45.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.398+0000 7f254aa57740 1 -- 172.21.15.16:0/784218480 >> 172.21.15.16:0/784218480 conn(0x558c22ca5300 msgr2=0x558c23055950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:45.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.398+0000 7f254aa57740 1 -- 172.21.15.16:0/784218480 shutdown_connections 2024-08-31T06:11:45.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.398+0000 7f254aa57740 1 -- 172.21.15.16:0/784218480 wait complete. 2024-08-31T06:11:45.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.399+0000 7f254aa57740 1 -- start start 2024-08-31T06:11:45.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.399+0000 7f254aa57740 1 Processor -- start 2024-08-31T06:11:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.400+0000 7f254aa57740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c22c882f0 0x558c230689f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.400+0000 7f254aa57740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c22c904d0 0x558c23065eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.400+0000 7f254aa57740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c22cb11b0 0x558c230663f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.401+0000 7f2541000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c22c882f0 0x558c230689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.401+0000 7f2541000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c22c882f0 0x558c230689f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:55492/0 (socket says 172.21.15.16:55492) 2024-08-31T06:11:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.401+0000 7f2541a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c22cb11b0 0x558c230663f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.401+0000 7f2541a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c22cb11b0 0x558c230663f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:33014/0 (socket says 172.21.15.16:33014) 2024-08-31T06:11:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.402+0000 7f253be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c22c904d0 0x558c23065eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.402+0000 7f253be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c22c904d0 0x558c23065eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54174/0 (socket says 172.21.15.16:54174) 2024-08-31T06:11:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.402+0000 7f254aa57740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f253c1a2c80 con 0x558c22c904d0 2024-08-31T06:11:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.402+0000 7f254aa57740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f253c1a40f0 con 0x558c22c882f0 2024-08-31T06:11:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.402+0000 7f254aa57740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f253c1a3350 con 0x558c22cb11b0 2024-08-31T06:11:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.402+0000 7f2541000640 1 -- 172.21.15.16:0/3068389132 learned_addr learned my addr 172.21.15.16:0/3068389132 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.402+0000 7f254aa57740 10 client.0 _ll_register_callbacks cb 0x7f25300ab9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:11:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.402+0000 7f2541000640 1 -- 172.21.15.16:0/3068389132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c22cb11b0 msgr2=0x558c230663f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.402+0000 7f2541000640 1 --2- 172.21.15.16:0/3068389132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c22cb11b0 0x558c230663f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.402+0000 7f2541000640 1 -- 172.21.15.16:0/3068389132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c22c904d0 msgr2=0x558c23065eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.402+0000 7f2541000640 1 --2- 172.21.15.16:0/3068389132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c22c904d0 0x558c23065eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.402+0000 7f2541000640 1 -- 172.21.15.16:0/3068389132 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c23066ae0 con 0x558c22c882f0 2024-08-31T06:11:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.403+0000 7f2541000640 1 --2- 172.21.15.16:0/3068389132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c22c882f0 0x558c230689f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f25300abc80 tx=0x7f2530003e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c1028d26190e961 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.403+0000 7f2522a00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:11:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.403+0000 7f2522a00640 1 -- 172.21.15.16:0/3068389132 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25300ca070 con 0x558c22c882f0 2024-08-31T06:11:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.403+0000 7f2522a00640 1 -- 172.21.15.16:0/3068389132 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f25300e3260 con 0x558c22c882f0 2024-08-31T06:11:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.403+0000 7f254aa57740 1 -- 172.21.15.16:0/3068389132 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2534017a20 con 0x558c22c882f0 2024-08-31T06:11:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.403+0000 7f2522a00640 1 -- 172.21.15.16:0/3068389132 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f25300b07f0 con 0x558c22c882f0 2024-08-31T06:11:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.403+0000 7f254aa57740 10 client.26645 fetch_fsmap learned FSMap version 34 2024-08-31T06:11:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.403+0000 7f254aa57740 1 -- 172.21.15.16:0/3068389132 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558c230ed880 con 0x558c22c882f0 2024-08-31T06:11:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.404+0000 7f2522a00640 1 -- 172.21.15.16:0/3068389132 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2534017a20 con 0x558c22c882f0 2024-08-31T06:11:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.405+0000 7f2522a00640 1 -- 172.21.15.16:0/3068389132 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f25300cd020 con 0x558c22c882f0 2024-08-31T06:11:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.405+0000 7f254aa57740 10 client.26645 fetch_fsmap finished waiting for FSMap version 34 2024-08-31T06:11:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.405+0000 7f254aa57740 10 client.26645 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:11:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.405+0000 7f254aa57740 20 client.26645 populate_metadata read hostname 'smithi016' 2024-08-31T06:11:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.405+0000 7f254aa57740 1 --2- 172.21.15.16:0/3068389132 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x558c22c451c0 0x558c230eff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.405+0000 7f254aa57740 4 client.26645 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:11:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.405+0000 7f254aa57740 1 -- 172.21.15.16:0/3068389132 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558c22fc7960 con 0x558c22c451c0 2024-08-31T06:11:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.405+0000 7f253be00640 1 --2- 172.21.15.16:0/3068389132 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x558c22c451c0 0x558c230eff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.406+0000 7f2522a00640 10 client.26645 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:11:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.406+0000 7f253be00640 1 --2- 172.21.15.16:0/3068389132 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x558c22c451c0 0x558c230eff00 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=787578ef6ed94789 server_cookie=d771b09ff4425bbb in_seq=0 out_seq=0 2024-08-31T06:11:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.422+0000 7f2522a00640 1 -- 172.21.15.16:0/3068389132 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558c22fc7960 con 0x558c22c451c0 2024-08-31T06:11:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.422+0000 7f2522a00640 10 client.26645 handle_command_reply: tid=0 2024-08-31T06:11:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.422+0000 7f254aa57740 1 client.26645 shutdown 2024-08-31T06:11:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.422+0000 7f254aa57740 1 -- 172.21.15.16:0/3068389132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c22c882f0 msgr2=0x558c230689f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.422+0000 7f254aa57740 1 --2- 172.21.15.16:0/3068389132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c22c882f0 0x558c230689f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f25300abc80 tx=0x7f2530003e90 comp rx=0 tx=0).stop 2024-08-31T06:11:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.422+0000 7f254aa57740 1 -- 172.21.15.16:0/3068389132 shutdown_connections 2024-08-31T06:11:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.422+0000 7f254aa57740 1 --2- 172.21.15.16:0/3068389132 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x558c22c451c0 0x558c230eff00 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.422+0000 7f254aa57740 1 --2- 172.21.15.16:0/3068389132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c22cb11b0 0x558c230663f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.422+0000 7f254aa57740 1 --2- 172.21.15.16:0/3068389132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c22c904d0 0x558c23065eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.422+0000 7f254aa57740 1 --2- 172.21.15.16:0/3068389132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c22c882f0 0x558c230689f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.422+0000 7f254aa57740 1 -- 172.21.15.16:0/3068389132 >> 172.21.15.16:0/3068389132 conn(0x558c22ca5300 msgr2=0x558c23056310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.423+0000 7f2522a00640 0 client.26645 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:11:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.423+0000 7f254aa57740 1 -- 172.21.15.16:0/3068389132 shutdown_connections 2024-08-31T06:11:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.423+0000 7f254aa57740 1 -- 172.21.15.16:0/3068389132 wait complete. 2024-08-31T06:11:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.424+0000 7f254aa57740 20 client.26645 trim_cache size 0 max 16384 2024-08-31T06:11:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:11:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:11:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:11:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:11:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.425+0000 7f2543a00640 1 -- 172.21.15.16:0/3283377482 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f25240b4df0 msgr2=0x7f25240b7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.425+0000 7f2543a00640 1 --2- 172.21.15.16:0/3283377482 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f25240b4df0 0x7f25240b7300 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2534005fd0 tx=0x7f253400b2c0 comp rx=0 tx=0).stop 2024-08-31T06:11:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.426+0000 7f2543a00640 1 -- 172.21.15.16:0/3283377482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f253c102750 msgr2=0x7f253c10cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.426+0000 7f2543a00640 1 --2- 172.21.15.16:0/3283377482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f253c102750 0x7f253c10cf00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f253000ea60 tx=0x7f253000ef20 comp rx=0 tx=0).stop 2024-08-31T06:11:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.426+0000 7f2543a00640 1 -- 172.21.15.16:0/3283377482 shutdown_connections 2024-08-31T06:11:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.426+0000 7f2543a00640 1 --2- 172.21.15.16:0/3283377482 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f25240b4df0 0x7f25240b7300 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.426+0000 7f2543a00640 1 --2- 172.21.15.16:0/3283377482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f253c1089d0 0x7f253c109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.426+0000 7f2543a00640 1 --2- 172.21.15.16:0/3283377482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f253c108080 0x7f253c1091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.426+0000 7f2543a00640 1 --2- 172.21.15.16:0/3283377482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f253c102750 0x7f253c10cf00 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.426+0000 7f2543a00640 1 -- 172.21.15.16:0/3283377482 >> 172.21.15.16:0/3283377482 conn(0x7f253c07a4e0 msgr2=0x7f253c105ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.426+0000 7f2543a00640 1 -- 172.21.15.16:0/3283377482 shutdown_connections 2024-08-31T06:11:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:45.427+0000 7f2543a00640 1 -- 172.21.15.16:0/3283377482 wait complete. 2024-08-31T06:11:45.834 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T06:11:45.917 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:45 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:11:45.917 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:45 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:11:45.917 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:45 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:11:46.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:45 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:11:46.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:45 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:11:46.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:45 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:11:46.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:45 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:11:46.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:45 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:11:46.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:45 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:11:46.492 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:11:46.625 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 260.0 s, 2067.6 tps, lat 48.606 ms stddev 55.503 2024-08-31T06:11:46.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:46 smithi016 ceph-mon[17194]: pgmap v500: 97 pgs: 97 active+clean; 2.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 9.3 MiB/s wr, 789 op/s 2024-08-31T06:11:47.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:46 smithi155 ceph-mon[21742]: pgmap v500: 97 pgs: 97 active+clean; 2.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 9.3 MiB/s wr, 789 op/s 2024-08-31T06:11:47.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:46 smithi184 ceph-mon[22331]: pgmap v500: 97 pgs: 97 active+clean; 2.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 9.3 MiB/s wr, 789 op/s 2024-08-31T06:11:47.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.436+0000 7f0029e00640 1 -- 172.21.15.16:0/734109702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0024104750 msgr2=0x7f0024104b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:47.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.437+0000 7f0029e00640 1 --2- 172.21.15.16:0/734109702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0024104750 0x7f0024104b60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0014009900 tx=0x7f001402a410 comp rx=0 tx=0).stop 2024-08-31T06:11:47.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.438+0000 7f0029e00640 1 -- 172.21.15.16:0/734109702 shutdown_connections 2024-08-31T06:11:47.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.438+0000 7f0029e00640 1 --2- 172.21.15.16:0/734109702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00241050a0 0x7f002410d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:47.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.439+0000 7f0029e00640 1 --2- 172.21.15.16:0/734109702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f002410dbb0 0x7f00241100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:47.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.439+0000 7f0029e00640 1 --2- 172.21.15.16:0/734109702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0024104750 0x7f0024104b60 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:47.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.439+0000 7f0029e00640 1 -- 172.21.15.16:0/734109702 >> 172.21.15.16:0/734109702 conn(0x7f00240794d0 msgr2=0x7f00240798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:47.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.443+0000 7f0029e00640 1 -- 172.21.15.16:0/734109702 shutdown_connections 2024-08-31T06:11:47.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.443+0000 7f0029e00640 1 -- 172.21.15.16:0/734109702 wait complete. 2024-08-31T06:11:47.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.444+0000 7f0029e00640 1 Processor -- start 2024-08-31T06:11:47.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.445+0000 7f0029e00640 1 -- start start 2024-08-31T06:11:47.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.446+0000 7f0029e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0024104750 0x7f002419a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:47.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.446+0000 7f0029e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00241050a0 0x7f002419a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:47.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.446+0000 7f0022a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0024104750 0x7f002419a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:47.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.447+0000 7f0022a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0024104750 0x7f002419a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:38722/0 (socket says 172.21.15.16:38722) 2024-08-31T06:11:47.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.447+0000 7f0029e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f002410dbb0 0x7f002419fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:47.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.447+0000 7f0022a00640 1 -- 172.21.15.16:0/2947588600 learned_addr learned my addr 172.21.15.16:0/2947588600 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:47.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.447+0000 7f0029e00640 1 -- 172.21.15.16:0/2947588600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0024113860 con 0x7f002410dbb0 2024-08-31T06:11:47.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.447+0000 7f0029e00640 1 -- 172.21.15.16:0/2947588600 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f00241136e0 con 0x7f0024104750 2024-08-31T06:11:47.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.447+0000 7f0029e00640 1 -- 172.21.15.16:0/2947588600 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f00241139e0 con 0x7f00241050a0 2024-08-31T06:11:47.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.448+0000 7f0022000640 1 --2- 172.21.15.16:0/2947588600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00241050a0 0x7f002419a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:47.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.448+0000 7f0023400640 1 --2- 172.21.15.16:0/2947588600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f002410dbb0 0x7f002419fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:47.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.449+0000 7f0022000640 1 -- 172.21.15.16:0/2947588600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0024104750 msgr2=0x7f002419a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:47.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.449+0000 7f0022000640 1 --2- 172.21.15.16:0/2947588600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0024104750 0x7f002419a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:47.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.449+0000 7f0022000640 1 -- 172.21.15.16:0/2947588600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f002410dbb0 msgr2=0x7f002419fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:47.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.449+0000 7f0022000640 1 --2- 172.21.15.16:0/2947588600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f002410dbb0 0x7f002419fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:47.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.449+0000 7f0022000640 1 -- 172.21.15.16:0/2947588600 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00241a02a0 con 0x7f00241050a0 2024-08-31T06:11:47.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.449+0000 7f0022000640 1 --2- 172.21.15.16:0/2947588600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00241050a0 0x7f002419a8d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0010005fd0 tx=0x7f001000dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:47.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.450+0000 7f0028a00640 1 -- 172.21.15.16:0/2947588600 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0010009c00 con 0x7f00241050a0 2024-08-31T06:11:47.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.450+0000 7f0028a00640 1 -- 172.21.15.16:0/2947588600 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0010009da0 con 0x7f00241050a0 2024-08-31T06:11:47.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.450+0000 7f0028a00640 1 -- 172.21.15.16:0/2947588600 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0010003d70 con 0x7f00241050a0 2024-08-31T06:11:47.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.451+0000 7f0029e00640 1 -- 172.21.15.16:0/2947588600 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00241a0590 con 0x7f00241050a0 2024-08-31T06:11:47.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.452+0000 7f0029e00640 1 -- 172.21.15.16:0/2947588600 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0024102370 con 0x7f00241050a0 2024-08-31T06:11:47.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.458+0000 7f0028a00640 1 -- 172.21.15.16:0/2947588600 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f0010003f10 con 0x7f00241050a0 2024-08-31T06:11:47.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.463+0000 7f0028a00640 1 --2- 172.21.15.16:0/2947588600 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f00080b4dd0 0x7f00080b72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:47.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.464+0000 7f0028a00640 1 -- 172.21.15.16:0/2947588600 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f00100e4470 con 0x7f00241050a0 2024-08-31T06:11:47.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.464+0000 7f0022a00640 1 --2- 172.21.15.16:0/2947588600 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f00080b4dd0 0x7f00080b72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:47.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.466+0000 7f00309c1740 1 Processor -- start 2024-08-31T06:11:47.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.466+0000 7f00309c1740 1 -- start start 2024-08-31T06:11:47.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.467+0000 7f00309c1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f936836070 0x55f936be4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:47.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.467+0000 7f00309c1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f93680d1b0 0x55f936be45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:47.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.467+0000 7f00309c1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f936815390 0x55f936be4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:47.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.467+0000 7f00309c1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f936b52770 con 0x55f936815390 2024-08-31T06:11:47.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.467+0000 7f00309c1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f936b41d00 con 0x55f936836070 2024-08-31T06:11:47.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.467+0000 7f00309c1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f936b45cb0 con 0x55f93680d1b0 2024-08-31T06:11:47.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.468+0000 7f0022a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f936836070 0x55f936be4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:47.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.468+0000 7f0022a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f936836070 0x55f936be4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:38726/0 (socket says 172.21.15.16:38726) 2024-08-31T06:11:47.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.468+0000 7f0022a00640 1 -- 172.21.15.16:0/3402330815 learned_addr learned my addr 172.21.15.16:0/3402330815 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:47.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.468+0000 7f0022a00640 1 -- 172.21.15.16:0/3402330815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f93680d1b0 msgr2=0x55f936be45c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T06:11:47.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.468+0000 7f0022a00640 1 --2- 172.21.15.16:0/3402330815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f93680d1b0 0x55f936be45c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:47.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.468+0000 7f0022a00640 1 -- 172.21.15.16:0/3402330815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f936815390 msgr2=0x55f936be4b00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:11:47.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.468+0000 7f0022a00640 1 --2- 172.21.15.16:0/3402330815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f936815390 0x55f936be4b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:47.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.468+0000 7f0022a00640 1 -- 172.21.15.16:0/3402330815 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9368a1650 con 0x55f936836070 2024-08-31T06:11:47.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.469+0000 7f0022a00640 1 --2- 172.21.15.16:0/3402330815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f936836070 0x55f936be4080 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f001402aa00 tx=0x7f0014004460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fde4d7377e4e03a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:47.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.470+0000 7f000fe00640 1 -- 172.21.15.16:0/3402330815 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0014035070 con 0x55f936836070 2024-08-31T06:11:47.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.470+0000 7f000fe00640 1 -- 172.21.15.16:0/3402330815 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0014004780 con 0x55f936836070 2024-08-31T06:11:47.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.470+0000 7f000fe00640 1 -- 172.21.15.16:0/3402330815 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0014035070 con 0x55f936836070 2024-08-31T06:11:47.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.471+0000 7f00309c1740 1 -- 172.21.15.16:0/3402330815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f936836070 msgr2=0x55f936be4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:47.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.471+0000 7f00309c1740 1 --2- 172.21.15.16:0/3402330815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f936836070 0x55f936be4080 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f001402aa00 tx=0x7f0014004460 comp rx=0 tx=0).stop 2024-08-31T06:11:47.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.472+0000 7f00309c1740 1 -- 172.21.15.16:0/3402330815 shutdown_connections 2024-08-31T06:11:47.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.472+0000 7f0022a00640 1 --2- 172.21.15.16:0/2947588600 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f00080b4dd0 0x7f00080b72e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f00140057c0 tx=0x7f0014003850 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:47.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.472+0000 7f00309c1740 1 --2- 172.21.15.16:0/3402330815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f93680d1b0 0x55f936be45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:47.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.472+0000 7f00309c1740 1 --2- 172.21.15.16:0/3402330815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f936815390 0x55f936be4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:47.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.472+0000 7f00309c1740 1 --2- 172.21.15.16:0/3402330815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f936836070 0x55f936be4080 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:47.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.472+0000 7f00309c1740 1 -- 172.21.15.16:0/3402330815 >> 172.21.15.16:0/3402330815 conn(0x55f93682a460 msgr2=0x55f936be1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:47.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.472+0000 7f00309c1740 1 -- 172.21.15.16:0/3402330815 shutdown_connections 2024-08-31T06:11:47.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.473+0000 7f00309c1740 1 -- 172.21.15.16:0/3402330815 wait complete. 2024-08-31T06:11:47.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.473+0000 7f00309c1740 1 -- start start 2024-08-31T06:11:47.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.474+0000 7f00309c1740 1 Processor -- start 2024-08-31T06:11:47.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.475+0000 7f00309c1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f93680d1b0 0x55f936bde3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:47.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.475+0000 7f00309c1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f936815390 0x55f936bde8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:47.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.475+0000 7f00309c1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f936836070 0x55f936c04420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:47.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.475+0000 7f00309c1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f9369a4700 con 0x55f93680d1b0 2024-08-31T06:11:47.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.475+0000 7f00309c1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f936ac8cb0 con 0x55f936836070 2024-08-31T06:11:47.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.475+0000 7f00309c1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f936be12f0 con 0x55f936815390 2024-08-31T06:11:47.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.475+0000 7f00309c1740 10 client.0 _ll_register_callbacks cb 0x55f936aee430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:11:47.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.476+0000 7f0023400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f936836070 0x55f936c04420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:47.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.477+0000 7f0023400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f936836070 0x55f936c04420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:38742/0 (socket says 172.21.15.16:38742) 2024-08-31T06:11:47.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.477+0000 7f0023400640 1 -- 172.21.15.16:0/2991289536 learned_addr learned my addr 172.21.15.16:0/2991289536 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:47.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.477+0000 7f0022000640 1 --2- 172.21.15.16:0/2991289536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f936815390 0x55f936bde8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:47.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.478+0000 7f0022a00640 1 --2- 172.21.15.16:0/2991289536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f93680d1b0 0x55f936bde3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:47.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.478+0000 7f0023400640 1 -- 172.21.15.16:0/2991289536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f936815390 msgr2=0x55f936bde8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:47.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.478+0000 7f0023400640 1 --2- 172.21.15.16:0/2991289536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f936815390 0x55f936bde8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:47.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.478+0000 7f0023400640 1 -- 172.21.15.16:0/2991289536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f93680d1b0 msgr2=0x55f936bde3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:47.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.478+0000 7f0023400640 1 --2- 172.21.15.16:0/2991289536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f93680d1b0 0x55f936bde3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:47.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.478+0000 7f0023400640 1 -- 172.21.15.16:0/2991289536 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f936b09870 con 0x55f936836070 2024-08-31T06:11:47.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.479+0000 7f0023400640 1 --2- 172.21.15.16:0/2991289536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f936836070 0x55f936c04420 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f001c00e620 tx=0x7f001c00eae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=793589236ea89356 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:47.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.479+0000 7f000cc00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:11:47.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.479+0000 7f000cc00640 1 -- 172.21.15.16:0/2991289536 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001c00c900 con 0x55f936836070 2024-08-31T06:11:47.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.479+0000 7f000cc00640 1 -- 172.21.15.16:0/2991289536 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f001c00caa0 con 0x55f936836070 2024-08-31T06:11:47.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.481+0000 7f00309c1740 1 -- 172.21.15.16:0/2991289536 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0014004780 con 0x55f936836070 2024-08-31T06:11:47.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.481+0000 7f000cc00640 1 -- 172.21.15.16:0/2991289536 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f001c00cca0 con 0x55f936836070 2024-08-31T06:11:47.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.481+0000 7f00309c1740 10 client.26657 fetch_fsmap learned FSMap version 34 2024-08-31T06:11:47.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.482+0000 7f00309c1740 1 -- 172.21.15.16:0/2991289536 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f936c06f00 con 0x55f936836070 2024-08-31T06:11:47.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.483+0000 7f000cc00640 1 -- 172.21.15.16:0/2991289536 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0014004780 con 0x55f936836070 2024-08-31T06:11:47.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.484+0000 7f000cc00640 1 -- 172.21.15.16:0/2991289536 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f001c00f040 con 0x55f936836070 2024-08-31T06:11:47.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.484+0000 7f00309c1740 10 client.26657 fetch_fsmap finished waiting for FSMap version 34 2024-08-31T06:11:47.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.484+0000 7f00309c1740 10 client.26657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:11:47.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.484+0000 7f00309c1740 20 client.26657 populate_metadata read hostname 'smithi016' 2024-08-31T06:11:47.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.484+0000 7f00309c1740 1 --2- 172.21.15.16:0/2991289536 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f9367f49f0 0x55f936c0a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:47.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.484+0000 7f00309c1740 4 client.26657 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:11:47.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.485+0000 7f00309c1740 1 -- 172.21.15.16:0/2991289536 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f936b546c0 con 0x55f9367f49f0 2024-08-31T06:11:47.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.485+0000 7f0022a00640 1 --2- 172.21.15.16:0/2991289536 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f9367f49f0 0x55f936c0a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:47.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.487+0000 7f0022a00640 1 --2- 172.21.15.16:0/2991289536 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f9367f49f0 0x55f936c0a410 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=957a0c42298fb5f2 server_cookie=cacb03cc01a3dd79 in_seq=0 out_seq=0 2024-08-31T06:11:47.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.487+0000 7f000cc00640 10 client.26657 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:11:47.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.488+0000 7f000cc00640 1 -- 172.21.15.16:0/2991289536 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f936b546c0 con 0x55f9367f49f0 2024-08-31T06:11:47.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.488+0000 7f000cc00640 10 client.26657 handle_command_reply: tid=0 2024-08-31T06:11:47.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.489+0000 7f00309c1740 1 client.26657 shutdown 2024-08-31T06:11:47.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.490+0000 7f00309c1740 1 -- 172.21.15.16:0/2991289536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f936836070 msgr2=0x55f936c04420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:47.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.490+0000 7f00309c1740 1 --2- 172.21.15.16:0/2991289536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f936836070 0x55f936c04420 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f001c00e620 tx=0x7f001c00eae0 comp rx=0 tx=0).stop 2024-08-31T06:11:47.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.491+0000 7f00309c1740 1 -- 172.21.15.16:0/2991289536 shutdown_connections 2024-08-31T06:11:47.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.491+0000 7f00309c1740 1 --2- 172.21.15.16:0/2991289536 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f9367f49f0 0x55f936c0a410 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:47.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.491+0000 7f00309c1740 1 --2- 172.21.15.16:0/2991289536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f936815390 0x55f936bde8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:47.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.491+0000 7f00309c1740 1 --2- 172.21.15.16:0/2991289536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f936836070 0x55f936c04420 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:47.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.491+0000 7f000cc00640 0 client.26657 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:11:47.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.491+0000 7f00309c1740 1 --2- 172.21.15.16:0/2991289536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f93680d1b0 0x55f936bde3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:47.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.491+0000 7f00309c1740 1 -- 172.21.15.16:0/2991289536 >> 172.21.15.16:0/2991289536 conn(0x55f93682a460 msgr2=0x55f936be5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:47.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.491+0000 7f00309c1740 1 -- 172.21.15.16:0/2991289536 shutdown_connections 2024-08-31T06:11:47.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.491+0000 7f00309c1740 1 -- 172.21.15.16:0/2991289536 wait complete. 2024-08-31T06:11:47.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.491+0000 7f00309c1740 20 client.26657 trim_cache size 0 max 16384 2024-08-31T06:11:47.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.529+0000 7f00309c1740 1 Processor -- start 2024-08-31T06:11:47.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.529+0000 7f00309c1740 1 -- start start 2024-08-31T06:11:47.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.530+0000 7f00309c1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f93680d1b0 0x55f936be9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:47.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.530+0000 7f00309c1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f936815390 0x55f936c043a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:47.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.530+0000 7f00309c1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f936836070 0x55f936bde150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:47.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.530+0000 7f00309c1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f00241afb90 con 0x55f936836070 2024-08-31T06:11:47.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.530+0000 7f00309c1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f00241ad430 con 0x55f93680d1b0 2024-08-31T06:11:47.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.530+0000 7f00309c1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f00241ab280 con 0x55f936815390 2024-08-31T06:11:47.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.531+0000 7f0022a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f93680d1b0 0x55f936be9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:47.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.531+0000 7f0022a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f93680d1b0 0x55f936be9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:38754/0 (socket says 172.21.15.16:38754) 2024-08-31T06:11:47.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.531+0000 7f0022a00640 1 -- 172.21.15.16:0/3576159101 learned_addr learned my addr 172.21.15.16:0/3576159101 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:47.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.531+0000 7f0022000640 1 --2- 172.21.15.16:0/3576159101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f936815390 0x55f936c043a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:47.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.531+0000 7f0023400640 1 --2- 172.21.15.16:0/3576159101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f936836070 0x55f936bde150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:47.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.532+0000 7f0022000640 1 -- 172.21.15.16:0/3576159101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f93680d1b0 msgr2=0x55f936be9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:47.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.532+0000 7f0022000640 1 --2- 172.21.15.16:0/3576159101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f93680d1b0 0x55f936be9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:47.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.532+0000 7f0022000640 1 -- 172.21.15.16:0/3576159101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f936836070 msgr2=0x55f936bde150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:47.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.532+0000 7f0022000640 1 --2- 172.21.15.16:0/3576159101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f936836070 0x55f936bde150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:47.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.532+0000 7f0022000640 1 -- 172.21.15.16:0/3576159101 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f936be6e40 con 0x55f936815390 2024-08-31T06:11:47.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.533+0000 7f0022000640 1 --2- 172.21.15.16:0/3576159101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f936815390 0x55f936c043a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f00100e1bc0 tx=0x7f00100aa670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=521ca9aa671ca59c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:47.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.534+0000 7f000f400640 1 -- 172.21.15.16:0/3576159101 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00100af3d0 con 0x55f936815390 2024-08-31T06:11:47.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.534+0000 7f000f400640 1 -- 172.21.15.16:0/3576159101 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00100b3e80 con 0x55f936815390 2024-08-31T06:11:47.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.534+0000 7f000f400640 1 -- 172.21.15.16:0/3576159101 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0010004460 con 0x55f936815390 2024-08-31T06:11:47.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.535+0000 7f00309c1740 1 -- 172.21.15.16:0/3576159101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f936815390 msgr2=0x55f936c043a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:47.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.535+0000 7f00309c1740 1 --2- 172.21.15.16:0/3576159101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f936815390 0x55f936c043a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f00100e1bc0 tx=0x7f00100aa670 comp rx=0 tx=0).stop 2024-08-31T06:11:47.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.536+0000 7f00309c1740 1 -- 172.21.15.16:0/3576159101 shutdown_connections 2024-08-31T06:11:47.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.536+0000 7f00309c1740 1 --2- 172.21.15.16:0/3576159101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f936815390 0x55f936c043a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:47.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.536+0000 7f00309c1740 1 --2- 172.21.15.16:0/3576159101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f936836070 0x55f936bde150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:47.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.536+0000 7f00309c1740 1 --2- 172.21.15.16:0/3576159101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f93680d1b0 0x55f936be9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:47.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.536+0000 7f00309c1740 1 -- 172.21.15.16:0/3576159101 >> 172.21.15.16:0/3576159101 conn(0x55f93682a460 msgr2=0x55f936be90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:47.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.536+0000 7f00309c1740 1 -- 172.21.15.16:0/3576159101 shutdown_connections 2024-08-31T06:11:47.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.536+0000 7f00309c1740 1 -- 172.21.15.16:0/3576159101 wait complete. 2024-08-31T06:11:47.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.537+0000 7f00309c1740 1 -- start start 2024-08-31T06:11:47.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.537+0000 7f00309c1740 1 Processor -- start 2024-08-31T06:11:47.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.538+0000 7f00309c1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f93680d1b0 0x55f936bfc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:47.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.538+0000 7f00309c1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f936815390 0x55f936bfcb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:47.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.538+0000 7f00309c1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f936836070 0x55f936bfd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:47.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.538+0000 7f00309c1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f00241ab8f0 con 0x55f93680d1b0 2024-08-31T06:11:47.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.538+0000 7f00309c1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f00241acd60 con 0x55f936815390 2024-08-31T06:11:47.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.538+0000 7f00309c1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f00241abfc0 con 0x55f936836070 2024-08-31T06:11:47.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.538+0000 7f00309c1740 10 client.0 _ll_register_callbacks cb 0x7f001c009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:11:47.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.539+0000 7f0022000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f936815390 0x55f936bfcb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:47.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.540+0000 7f0022000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f936815390 0x55f936bfcb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:38758/0 (socket says 172.21.15.16:38758) 2024-08-31T06:11:47.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.540+0000 7f0022000640 1 -- 172.21.15.16:0/984587491 learned_addr learned my addr 172.21.15.16:0/984587491 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:47.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.540+0000 7f0023400640 1 --2- 172.21.15.16:0/984587491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f936836070 0x55f936bfd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:47.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.540+0000 7f0022a00640 1 --2- 172.21.15.16:0/984587491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f93680d1b0 0x55f936bfc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:47.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.542+0000 7f0022000640 1 -- 172.21.15.16:0/984587491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f936836070 msgr2=0x55f936bfd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:47.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.542+0000 7f0022000640 1 --2- 172.21.15.16:0/984587491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f936836070 0x55f936bfd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:47.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.542+0000 7f0022000640 1 -- 172.21.15.16:0/984587491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f93680d1b0 msgr2=0x55f936bfc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:47.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.542+0000 7f0022000640 1 --2- 172.21.15.16:0/984587491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f93680d1b0 0x55f936bfc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:47.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.542+0000 7f0022000640 1 -- 172.21.15.16:0/984587491 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f936c74010 con 0x55f936815390 2024-08-31T06:11:47.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.543+0000 7f0023400640 1 --2- 172.21.15.16:0/984587491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f936836070 0x55f936bfd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:11:47.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.543+0000 7f0022a00640 1 --2- 172.21.15.16:0/984587491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f93680d1b0 0x55f936bfc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:11:47.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.544+0000 7f0022000640 1 --2- 172.21.15.16:0/984587491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f936815390 0x55f936bfcb10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f001000bfa0 tx=0x7f00100a7000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c741954c287476a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:47.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.544+0000 7f000cc00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:11:47.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.545+0000 7f000cc00640 1 -- 172.21.15.16:0/984587491 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00100e2050 con 0x55f936815390 2024-08-31T06:11:47.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.545+0000 7f000cc00640 1 -- 172.21.15.16:0/984587491 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f0010004400 con 0x55f936815390 2024-08-31T06:11:47.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.545+0000 7f00309c1740 1 -- 172.21.15.16:0/984587491 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00100b3e80 con 0x55f936815390 2024-08-31T06:11:47.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.546+0000 7f000cc00640 1 -- 172.21.15.16:0/984587491 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f00100cb050 con 0x55f936815390 2024-08-31T06:11:47.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.546+0000 7f00309c1740 10 client.26663 fetch_fsmap learned FSMap version 34 2024-08-31T06:11:47.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.546+0000 7f00309c1740 1 -- 172.21.15.16:0/984587491 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f936c752a0 con 0x55f936815390 2024-08-31T06:11:47.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.546+0000 7f000cc00640 1 -- 172.21.15.16:0/984587491 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00100b3e80 con 0x55f936815390 2024-08-31T06:11:47.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.547+0000 7f000cc00640 1 -- 172.21.15.16:0/984587491 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f00100a9950 con 0x55f936815390 2024-08-31T06:11:47.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.548+0000 7f00309c1740 10 client.26663 fetch_fsmap finished waiting for FSMap version 34 2024-08-31T06:11:47.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.548+0000 7f00309c1740 10 client.26663 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:11:47.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.548+0000 7f00309c1740 20 client.26663 populate_metadata read hostname 'smithi016' 2024-08-31T06:11:47.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.548+0000 7f00309c1740 1 --2- 172.21.15.16:0/984587491 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f9367c9da0 0x55f936c77980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:47.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.548+0000 7f00309c1740 4 client.26663 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T06:11:47.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.548+0000 7f00309c1740 1 -- 172.21.15.16:0/984587491 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f936b50160 con 0x55f9367c9da0 2024-08-31T06:11:47.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.549+0000 7f0022a00640 1 --2- 172.21.15.16:0/984587491 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f9367c9da0 0x55f936c77980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:47.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.553+0000 7f0022a00640 1 --2- 172.21.15.16:0/984587491 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f9367c9da0 0x55f936c77980 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bb475d0fa75b823 server_cookie=48eb1ee564d442 in_seq=0 out_seq=0 2024-08-31T06:11:47.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.553+0000 7f000cc00640 10 client.26663 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:11:47.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.553+0000 7f000cc00640 1 -- 172.21.15.16:0/984587491 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2274 (crc 0 0 0) 0x55f936b50160 con 0x55f9367c9da0 2024-08-31T06:11:47.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.553+0000 7f000cc00640 10 client.26663 handle_command_reply: tid=0 2024-08-31T06:11:47.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.553+0000 7f00309c1740 1 client.26663 shutdown 2024-08-31T06:11:47.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.553+0000 7f00309c1740 1 -- 172.21.15.16:0/984587491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f936815390 msgr2=0x55f936bfcb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:47.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.553+0000 7f00309c1740 1 --2- 172.21.15.16:0/984587491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f936815390 0x55f936bfcb10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f001000bfa0 tx=0x7f00100a7000 comp rx=0 tx=0).stop 2024-08-31T06:11:47.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.554+0000 7f00309c1740 1 -- 172.21.15.16:0/984587491 shutdown_connections 2024-08-31T06:11:47.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.554+0000 7f00309c1740 1 --2- 172.21.15.16:0/984587491 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f9367c9da0 0x55f936c77980 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:47.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.554+0000 7f00309c1740 1 --2- 172.21.15.16:0/984587491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f936836070 0x55f936bfd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:47.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.554+0000 7f00309c1740 1 --2- 172.21.15.16:0/984587491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f936815390 0x55f936bfcb10 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:47.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.554+0000 7f00309c1740 1 --2- 172.21.15.16:0/984587491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f93680d1b0 0x55f936bfc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:47.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.554+0000 7f00309c1740 1 -- 172.21.15.16:0/984587491 >> 172.21.15.16:0/984587491 conn(0x55f93682a460 msgr2=0x55f936bde150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:47.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.554+0000 7f000cc00640 0 client.26663 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:11:47.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.554+0000 7f00309c1740 1 -- 172.21.15.16:0/984587491 shutdown_connections 2024-08-31T06:11:47.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.554+0000 7f00309c1740 1 -- 172.21.15.16:0/984587491 wait complete. 2024-08-31T06:11:47.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.555+0000 7f00309c1740 20 client.26663 trim_cache size 0 max 16384 2024-08-31T06:11:47.559 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:11:47.559 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T06:11:47.560 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-08-31T06:11:47.560 INFO:teuthology.orchestra.run.smithi016.stdout: "a10cce76-99ab-46b5-b3b2-07b82899b8f9": { 2024-08-31T06:11:47.560 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:11:47.560 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "a10cce76-99ab-46b5-b3b2-07b82899b8f9", 2024-08-31T06:11:47.560 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-08-31T06:11:47.560 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:11:47.560 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:11:47.560 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-08-31T06:11:47.560 INFO:teuthology.orchestra.run.smithi016.stdout: "e739a16e-8ace-43a8-a1ce-ac27d39ef8a3": { 2024-08-31T06:11:47.560 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:54.084646+0000", 2024-08-31T06:11:47.560 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:11:47.560 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:11:47.561 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:11:47.561 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:11:47.561 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:11:47.561 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:11:47.561 INFO:teuthology.orchestra.run.smithi016.stdout: "a10cce76-99ab-46b5-b3b2-07b82899b8f9": { 2024-08-31T06:11:47.561 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:11:43.967546+0000", 2024-08-31T06:11:47.561 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:11:47.561 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:11:47.561 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:11:47.561 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:11:47.562 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:11:47.562 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:11:47.562 INFO:teuthology.orchestra.run.smithi016.stdout: "c1fd6904-2e02-47fa-939a-47349b9fe36b": { 2024-08-31T06:11:47.562 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:06.678691+0000", 2024-08-31T06:11:47.562 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:11:47.562 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:11:47.562 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:11:47.562 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:11:47.562 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:11:47.562 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:11:47.562 INFO:teuthology.orchestra.run.smithi016.stdout: "78d6ef02-8551-45e8-ad60-9627eb573fc1": { 2024-08-31T06:11:47.562 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:09:13.547865+0000", 2024-08-31T06:11:47.563 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:11:47.563 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:11:47.563 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:11:47.563 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:11:47.563 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:11:47.563 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:11:47.563 INFO:teuthology.orchestra.run.smithi016.stdout: "c8455d74-2fe7-4d93-bcf2-19b419d16016": { 2024-08-31T06:11:47.563 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:08:24.035960+0000", 2024-08-31T06:11:47.563 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:11:47.563 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:11:47.563 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:11:47.564 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:11:47.564 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:11:47.564 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:11:47.564 INFO:teuthology.orchestra.run.smithi016.stdout: "88d7bdb8-cb1c-45e5-ad29-4cd8da6c6003": { 2024-08-31T06:11:47.564 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:07:39.648942+0000", 2024-08-31T06:11:47.564 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:11:47.564 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:11:47.564 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:11:47.564 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:11:47.564 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:11:47.564 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:11:47.565 INFO:teuthology.orchestra.run.smithi016.stdout: "c300b7e8-f3fd-482c-a909-9329bc61e570": { 2024-08-31T06:11:47.565 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:06:49.868046+0000", 2024-08-31T06:11:47.565 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:11:47.565 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:11:47.565 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:11:47.565 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:11:47.565 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:11:47.565 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:11:47.565 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:11:47.565 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:11:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.556+0000 7f0029e00640 1 -- 172.21.15.16:0/2947588600 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f00080b4dd0 msgr2=0x7f00080b72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.556+0000 7f0029e00640 1 --2- 172.21.15.16:0/2947588600 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f00080b4dd0 0x7f00080b72e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f00140057c0 tx=0x7f0014003850 comp rx=0 tx=0).stop 2024-08-31T06:11:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.556+0000 7f0029e00640 1 -- 172.21.15.16:0/2947588600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00241050a0 msgr2=0x7f002419a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.556+0000 7f0029e00640 1 --2- 172.21.15.16:0/2947588600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00241050a0 0x7f002419a8d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0010005fd0 tx=0x7f001000dde0 comp rx=0 tx=0).stop 2024-08-31T06:11:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.557+0000 7f0029e00640 1 -- 172.21.15.16:0/2947588600 shutdown_connections 2024-08-31T06:11:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.557+0000 7f0029e00640 1 --2- 172.21.15.16:0/2947588600 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f00080b4dd0 0x7f00080b72e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.557+0000 7f0029e00640 1 --2- 172.21.15.16:0/2947588600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00241050a0 0x7f002419a8d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.557+0000 7f0029e00640 1 --2- 172.21.15.16:0/2947588600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f002410dbb0 0x7f002419fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:47.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.557+0000 7f0029e00640 1 --2- 172.21.15.16:0/2947588600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0024104750 0x7f002419a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:47.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.557+0000 7f0029e00640 1 -- 172.21.15.16:0/2947588600 >> 172.21.15.16:0/2947588600 conn(0x7f00240794d0 msgr2=0x7f00241005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:47.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.557+0000 7f0029e00640 1 -- 172.21.15.16:0/2947588600 shutdown_connections 2024-08-31T06:11:47.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:11:47.557+0000 7f0029e00640 1 -- 172.21.15.16:0/2947588600 wait complete. 2024-08-31T06:11:48.547 INFO:tasks.cephfs.filesystem:scrub status for tag:a10cce76-99ab-46b5-b3b2-07b82899b8f9 - {'path': '/', 'tag': 'a10cce76-99ab-46b5-b3b2-07b82899b8f9', 'options': 'recursive,force'} 2024-08-31T06:11:49.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:48 smithi155 ceph-mon[21742]: pgmap v501: 97 pgs: 97 active+clean; 2.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 8.3 MiB/s wr, 709 op/s 2024-08-31T06:11:49.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:48 smithi155 ceph-mon[21742]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:11:49.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:48 smithi016 ceph-mon[17194]: pgmap v501: 97 pgs: 97 active+clean; 2.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 8.3 MiB/s wr, 709 op/s 2024-08-31T06:11:49.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:48 smithi016 ceph-mon[17194]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:11:49.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:48 smithi184 ceph-mon[22331]: pgmap v501: 97 pgs: 97 active+clean; 2.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 8.3 MiB/s wr, 709 op/s 2024-08-31T06:11:49.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:48 smithi184 ceph-mon[22331]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:11:50.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:49 smithi155 ceph-mon[21742]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:11:50.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:49 smithi016 ceph-mon[17194]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:11:50.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:49 smithi184 ceph-mon[22331]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:11:51.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:50 smithi155 ceph-mon[21742]: pgmap v502: 97 pgs: 97 active+clean; 2.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 10 MiB/s wr, 924 op/s 2024-08-31T06:11:51.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:50 smithi016 ceph-mon[17194]: pgmap v502: 97 pgs: 97 active+clean; 2.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 10 MiB/s wr, 924 op/s 2024-08-31T06:11:51.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:50 smithi184 ceph-mon[22331]: pgmap v502: 97 pgs: 97 active+clean; 2.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 10 MiB/s wr, 924 op/s 2024-08-31T06:11:51.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 265.0 s, 2440.2 tps, lat 40.876 ms stddev 46.257 2024-08-31T06:11:53.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:52 smithi155 ceph-mon[21742]: pgmap v503: 97 pgs: 97 active+clean; 2.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 7.8 MiB/s wr, 830 op/s 2024-08-31T06:11:53.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:52 smithi016 ceph-mon[17194]: pgmap v503: 97 pgs: 97 active+clean; 2.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 7.8 MiB/s wr, 830 op/s 2024-08-31T06:11:53.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:52 smithi184 ceph-mon[22331]: pgmap v503: 97 pgs: 97 active+clean; 2.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 7.8 MiB/s wr, 830 op/s 2024-08-31T06:11:54.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:53 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:11:54.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:53 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:11:54.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:53 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:11:54.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:53 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:11:54.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:53 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:11:54.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:53 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:11:55.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:54 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:11:55.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:54 smithi155 ceph-mon[21742]: pgmap v504: 97 pgs: 97 active+clean; 2.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 8.7 MiB/s wr, 915 op/s 2024-08-31T06:11:55.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:54 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:11:55.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:54 smithi016 ceph-mon[17194]: pgmap v504: 97 pgs: 97 active+clean; 2.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 8.7 MiB/s wr, 915 op/s 2024-08-31T06:11:55.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:54 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:11:55.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:54 smithi184 ceph-mon[22331]: pgmap v504: 97 pgs: 97 active+clean; 2.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 8.7 MiB/s wr, 915 op/s 2024-08-31T06:11:56.626 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 270.0 s, 2790.4 tps, lat 35.815 ms stddev 43.377 2024-08-31T06:11:57.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:56 smithi155 ceph-mon[21742]: pgmap v505: 97 pgs: 97 active+clean; 2.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 10 MiB/s wr, 975 op/s 2024-08-31T06:11:57.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:56 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:11:57.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:56 smithi016 ceph-mon[17194]: pgmap v505: 97 pgs: 97 active+clean; 2.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 10 MiB/s wr, 975 op/s 2024-08-31T06:11:57.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:56 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:11:57.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:56 smithi184 ceph-mon[22331]: pgmap v505: 97 pgs: 97 active+clean; 2.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 10 MiB/s wr, 975 op/s 2024-08-31T06:11:57.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:56 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:11:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.728+0000 7f727f200640 1 -- 172.21.15.16:0/701785129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7280108fe0 msgr2=0x7f72801093f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.728+0000 7f727f200640 1 --2- 172.21.15.16:0/701785129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7280108fe0 0x7f72801093f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7268009860 tx=0x7f726802a340 comp rx=0 tx=0).stop 2024-08-31T06:11:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.728+0000 7f727f200640 1 -- 172.21.15.16:0/701785129 shutdown_connections 2024-08-31T06:11:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.728+0000 7f727f200640 1 --2- 172.21.15.16:0/701785129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7280109930 0x7f7280110160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.728+0000 7f727f200640 1 --2- 172.21.15.16:0/701785129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7280108fe0 0x7f72801093f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.728+0000 7f727f200640 1 --2- 172.21.15.16:0/701785129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72801036b0 0x7f7280103ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.728+0000 7f727f200640 1 -- 172.21.15.16:0/701785129 >> 172.21.15.16:0/701785129 conn(0x7f72800ff0d0 msgr2=0x7f7280101520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:58.732 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.728+0000 7f727f200640 1 -- 172.21.15.16:0/701785129 shutdown_connections 2024-08-31T06:11:58.732 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.728+0000 7f727f200640 1 -- 172.21.15.16:0/701785129 wait complete. 2024-08-31T06:11:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.731+0000 7f727f200640 1 Processor -- start 2024-08-31T06:11:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.731+0000 7f727f200640 1 -- start start 2024-08-31T06:11:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.732+0000 7f727f200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72801036b0 0x7f728010d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.732+0000 7f727f200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7280108fe0 0x7f728010dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.732+0000 7f727f200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7280109930 0x7f72801ac3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.732+0000 7f727f200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7280076f80 con 0x7f7280109930 2024-08-31T06:11:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.732+0000 7f727f200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7280076e00 con 0x7f72801036b0 2024-08-31T06:11:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.732+0000 7f727f200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7280077100 con 0x7f7280108fe0 2024-08-31T06:11:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.732+0000 7f7275400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7280108fe0 0x7f728010dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.733+0000 7f7275400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7280108fe0 0x7f728010dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:53428/0 (socket says 172.21.15.16:53428) 2024-08-31T06:11:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.733+0000 7f7275400640 1 -- 172.21.15.16:0/4195484560 learned_addr learned my addr 172.21.15.16:0/4195484560 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.733+0000 7f727de00640 1 --2- 172.21.15.16:0/4195484560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72801036b0 0x7f728010d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.733+0000 7f727e800640 1 --2- 172.21.15.16:0/4195484560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7280109930 0x7f72801ac3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.733+0000 7f727de00640 1 -- 172.21.15.16:0/4195484560 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7280108fe0 msgr2=0x7f728010dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.733+0000 7f727de00640 1 --2- 172.21.15.16:0/4195484560 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7280108fe0 0x7f728010dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.733+0000 7f727de00640 1 -- 172.21.15.16:0/4195484560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7280109930 msgr2=0x7f72801ac3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.733+0000 7f727de00640 1 --2- 172.21.15.16:0/4195484560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7280109930 0x7f72801ac3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.733+0000 7f727de00640 1 -- 172.21.15.16:0/4195484560 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72801ac8f0 con 0x7f72801036b0 2024-08-31T06:11:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.734+0000 7f727de00640 1 --2- 172.21.15.16:0/4195484560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72801036b0 0x7f728010d800 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f727000e9f0 tx=0x7f727000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.734+0000 7f7277400640 1 -- 172.21.15.16:0/4195484560 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727000cd80 con 0x7f72801036b0 2024-08-31T06:11:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.734+0000 7f7277400640 1 -- 172.21.15.16:0/4195484560 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72700035f0 con 0x7f72801036b0 2024-08-31T06:11:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.734+0000 7f7277400640 1 -- 172.21.15.16:0/4195484560 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7270010670 con 0x7f72801036b0 2024-08-31T06:11:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.734+0000 7f727f200640 1 -- 172.21.15.16:0/4195484560 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72801acac0 con 0x7f72801036b0 2024-08-31T06:11:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.734+0000 7f727f200640 1 -- 172.21.15.16:0/4195484560 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72801acf30 con 0x7f72801036b0 2024-08-31T06:11:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.741+0000 7f728c004740 1 Processor -- start 2024-08-31T06:11:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.741+0000 7f7277400640 1 -- 172.21.15.16:0/4195484560 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f72700108d0 con 0x7f72801036b0 2024-08-31T06:11:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.743+0000 7f728c004740 1 -- start start 2024-08-31T06:11:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.744+0000 7f728c004740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594878d81b0 0x559487c7b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.744+0000 7f728c004740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594878af2f0 0x559487c7bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.744+0000 7f728c004740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594878b74d0 0x559487c7c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.744+0000 7f728c004740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559487bf0f70 con 0x5594878b74d0 2024-08-31T06:11:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.744+0000 7f728c004740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559487be0500 con 0x5594878af2f0 2024-08-31T06:11:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.744+0000 7f728c004740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559487be44b0 con 0x5594878d81b0 2024-08-31T06:11:58.748 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.745+0000 7f7275400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594878b74d0 0x559487c7c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:58.748 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.745+0000 7f7275400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594878b74d0 0x559487c7c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57302/0 (socket says 172.21.15.16:57302) 2024-08-31T06:11:58.748 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.745+0000 7f727e800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594878d81b0 0x559487c7b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:58.748 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.745+0000 7f7275400640 1 -- 172.21.15.16:0/3395361373 learned_addr learned my addr 172.21.15.16:0/3395361373 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:58.748 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.745+0000 7f7275400640 1 -- 172.21.15.16:0/3395361373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594878d81b0 msgr2=0x559487c7b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:58.748 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.745+0000 7f7275400640 1 --2- 172.21.15.16:0/3395361373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594878d81b0 0x559487c7b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:58.748 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.746+0000 7f7275400640 1 -- 172.21.15.16:0/3395361373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594878af2f0 msgr2=0x559487c7bde0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:11:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.746+0000 7f7275400640 1 --2- 172.21.15.16:0/3395361373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594878af2f0 0x559487c7bde0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.746+0000 7f7275400640 1 -- 172.21.15.16:0/3395361373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559487b9ac10 con 0x5594878b74d0 2024-08-31T06:11:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.746+0000 7f727e800640 1 --2- 172.21.15.16:0/3395361373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594878d81b0 0x559487c7b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:11:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.746+0000 7f7275400640 1 --2- 172.21.15.16:0/3395361373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594878b74d0 0x559487c7c320 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f726802a8b0 tx=0x7f7268009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16d4fa5195a86921 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.746+0000 7f7277400640 1 --2- 172.21.15.16:0/4195484560 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f72640b4dd0 0x7f72640b72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.747+0000 7f7277400640 1 -- 172.21.15.16:0/4195484560 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f72700e57f0 con 0x7f72801036b0 2024-08-31T06:11:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.747+0000 7f726f400640 1 -- 172.21.15.16:0/3395361373 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7268037040 con 0x5594878b74d0 2024-08-31T06:11:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.747+0000 7f726f400640 1 -- 172.21.15.16:0/3395361373 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7268042070 con 0x5594878b74d0 2024-08-31T06:11:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.747+0000 7f726f400640 1 -- 172.21.15.16:0/3395361373 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72680052a0 con 0x5594878b74d0 2024-08-31T06:11:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.747+0000 7f727e800640 1 --2- 172.21.15.16:0/4195484560 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f72640b4dd0 0x7f72640b72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.748+0000 7f728c004740 1 -- 172.21.15.16:0/3395361373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594878b74d0 msgr2=0x559487c7c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.748+0000 7f728c004740 1 --2- 172.21.15.16:0/3395361373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594878b74d0 0x559487c7c320 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f726802a8b0 tx=0x7f7268009540 comp rx=0 tx=0).stop 2024-08-31T06:11:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.748+0000 7f728c004740 1 -- 172.21.15.16:0/3395361373 shutdown_connections 2024-08-31T06:11:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.748+0000 7f728c004740 1 --2- 172.21.15.16:0/3395361373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594878b74d0 0x559487c7c320 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.748+0000 7f728c004740 1 --2- 172.21.15.16:0/3395361373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594878af2f0 0x559487c7bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.748+0000 7f728c004740 1 --2- 172.21.15.16:0/3395361373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594878d81b0 0x559487c7b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.748+0000 7f728c004740 1 -- 172.21.15.16:0/3395361373 >> 172.21.15.16:0/3395361373 conn(0x5594878cc300 msgr2=0x559487c79790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.749+0000 7f727e800640 1 --2- 172.21.15.16:0/4195484560 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f72640b4dd0 0x7f72640b72e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7278005fd0 tx=0x7f7278011040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.749+0000 7f728c004740 1 -- 172.21.15.16:0/3395361373 shutdown_connections 2024-08-31T06:11:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.749+0000 7f728c004740 1 -- 172.21.15.16:0/3395361373 wait complete. 2024-08-31T06:11:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.750+0000 7f728c004740 1 -- start start 2024-08-31T06:11:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.750+0000 7f728c004740 1 Processor -- start 2024-08-31T06:11:58.754 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.752+0000 7f728c004740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594878af2f0 0x559487c86e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:58.755 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.753+0000 7f728c004740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594878b74d0 0x559487c873a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:58.755 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.753+0000 7f728c004740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594878d81b0 0x559487c878e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:58.756 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.753+0000 7f728c004740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559487b19ab0 con 0x5594878b74d0 2024-08-31T06:11:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.753+0000 7f728c004740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559487b87320 con 0x5594878d81b0 2024-08-31T06:11:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.753+0000 7f728c004740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559487c89fc0 con 0x5594878af2f0 2024-08-31T06:11:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.753+0000 7f728c004740 10 client.0 _ll_register_callbacks cb 0x559487bc0e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:11:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.754+0000 7f727e800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594878d81b0 0x559487c878e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.754+0000 7f727e800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594878d81b0 0x559487c878e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:38294/0 (socket says 172.21.15.16:38294) 2024-08-31T06:11:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.754+0000 7f727e800640 1 -- 172.21.15.16:0/4133902034 learned_addr learned my addr 172.21.15.16:0/4133902034 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:58.758 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.755+0000 7f7275400640 1 --2- 172.21.15.16:0/4133902034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594878b74d0 0x559487c873a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:58.758 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.755+0000 7f727e800640 1 -- 172.21.15.16:0/4133902034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594878af2f0 msgr2=0x559487c86e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:58.758 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.755+0000 7f727e800640 1 --2- 172.21.15.16:0/4133902034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594878af2f0 0x559487c86e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:58.758 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.755+0000 7f727e800640 1 -- 172.21.15.16:0/4133902034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594878b74d0 msgr2=0x559487c873a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.755+0000 7f727e800640 1 --2- 172.21.15.16:0/4133902034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594878b74d0 0x559487c873a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.755+0000 7f727e800640 1 -- 172.21.15.16:0/4133902034 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559487943790 con 0x5594878d81b0 2024-08-31T06:11:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.757+0000 7f727e800640 1 --2- 172.21.15.16:0/4133902034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594878d81b0 0x559487c878e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f727800bf00 tx=0x7f7278004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6079d067b8dcdf9f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.757+0000 7f7263e00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:11:58.764 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.762+0000 7f7263e00640 1 -- 172.21.15.16:0/4133902034 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7278029030 con 0x5594878d81b0 2024-08-31T06:11:58.764 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.762+0000 7f7263e00640 1 -- 172.21.15.16:0/4133902034 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f7278009c80 con 0x5594878d81b0 2024-08-31T06:11:58.765 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.762+0000 7f728c004740 1 -- 172.21.15.16:0/4133902034 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7268042070 con 0x5594878d81b0 2024-08-31T06:11:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.764+0000 7f7263e00640 1 -- 172.21.15.16:0/4133902034 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7278033050 con 0x5594878d81b0 2024-08-31T06:11:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.764+0000 7f728c004740 10 client.26672 fetch_fsmap learned FSMap version 36 2024-08-31T06:11:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.764+0000 7f728c004740 1 -- 172.21.15.16:0/4133902034 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5594879bc620 con 0x5594878d81b0 2024-08-31T06:11:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.766+0000 7f7263e00640 1 -- 172.21.15.16:0/4133902034 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7268042070 con 0x5594878d81b0 2024-08-31T06:11:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.766+0000 7f7263e00640 1 -- 172.21.15.16:0/4133902034 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 36) ==== 8350+0+0 (secure 0 0 0) 0x7f7278024030 con 0x5594878d81b0 2024-08-31T06:11:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.766+0000 7f728c004740 10 client.26672 fetch_fsmap finished waiting for FSMap version 36 2024-08-31T06:11:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.766+0000 7f728c004740 10 client.26672 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:11:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.766+0000 7f728c004740 20 client.26672 populate_metadata read hostname 'smithi016' 2024-08-31T06:11:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.766+0000 7f728c004740 1 --2- 172.21.15.16:0/4133902034 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559487896d80 0x559487c99180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.766+0000 7f728c004740 4 client.26672 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:11:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.766+0000 7f728c004740 1 -- 172.21.15.16:0/4133902034 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559487bf2ec0 con 0x559487896d80 2024-08-31T06:11:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.766+0000 7f727de00640 1 --2- 172.21.15.16:0/4133902034 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559487896d80 0x559487c99180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.769+0000 7f727de00640 1 --2- 172.21.15.16:0/4133902034 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559487896d80 0x559487c99180 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d20cd4a7488a96de server_cookie=2f283a3fb7700886 in_seq=0 out_seq=0 2024-08-31T06:11:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.769+0000 7f7263e00640 10 client.26672 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:11:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.771+0000 7f7263e00640 1 -- 172.21.15.16:0/4133902034 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559487bf2ec0 con 0x559487896d80 2024-08-31T06:11:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.771+0000 7f7263e00640 10 client.26672 handle_command_reply: tid=0 2024-08-31T06:11:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.771+0000 7f728c004740 1 client.26672 shutdown 2024-08-31T06:11:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.772+0000 7f728c004740 1 -- 172.21.15.16:0/4133902034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594878d81b0 msgr2=0x559487c878e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.772+0000 7f728c004740 1 --2- 172.21.15.16:0/4133902034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594878d81b0 0x559487c878e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f727800bf00 tx=0x7f7278004ab0 comp rx=0 tx=0).stop 2024-08-31T06:11:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.773+0000 7f728c004740 1 -- 172.21.15.16:0/4133902034 shutdown_connections 2024-08-31T06:11:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.773+0000 7f728c004740 1 --2- 172.21.15.16:0/4133902034 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559487896d80 0x559487c99180 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.773+0000 7f728c004740 1 --2- 172.21.15.16:0/4133902034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594878d81b0 0x559487c878e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.773+0000 7f7263e00640 0 client.26672 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:11:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.773+0000 7f728c004740 1 --2- 172.21.15.16:0/4133902034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594878b74d0 0x559487c873a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.773+0000 7f728c004740 1 --2- 172.21.15.16:0/4133902034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594878af2f0 0x559487c86e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.773+0000 7f728c004740 1 -- 172.21.15.16:0/4133902034 >> 172.21.15.16:0/4133902034 conn(0x5594878cc300 msgr2=0x559487c7c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.773+0000 7f728c004740 1 -- 172.21.15.16:0/4133902034 shutdown_connections 2024-08-31T06:11:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.773+0000 7f728c004740 1 -- 172.21.15.16:0/4133902034 wait complete. 2024-08-31T06:11:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.774+0000 7f728c004740 20 client.26672 trim_cache size 0 max 16384 2024-08-31T06:11:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.807+0000 7f728c004740 1 Processor -- start 2024-08-31T06:11:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.807+0000 7f728c004740 1 -- start start 2024-08-31T06:11:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.807+0000 7f728c004740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594878af2f0 0x559487c79790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.807+0000 7f728c004740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594878b74d0 0x559487c7d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.807+0000 7f728c004740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594878d81b0 0x559487c88a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.808+0000 7f728c004740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f72801b8140 con 0x5594878d81b0 2024-08-31T06:11:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.808+0000 7f728c004740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f72801b59e0 con 0x5594878af2f0 2024-08-31T06:11:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.808+0000 7f728c004740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f72801b3830 con 0x5594878b74d0 2024-08-31T06:11:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.808+0000 7f7275400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594878b74d0 0x559487c7d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.808+0000 7f7275400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594878b74d0 0x559487c7d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:53450/0 (socket says 172.21.15.16:53450) 2024-08-31T06:11:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.808+0000 7f7275400640 1 -- 172.21.15.16:0/2124276592 learned_addr learned my addr 172.21.15.16:0/2124276592 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:58.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.808+0000 7f727de00640 1 --2- 172.21.15.16:0/2124276592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594878af2f0 0x559487c79790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:58.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.808+0000 7f7275400640 1 -- 172.21.15.16:0/2124276592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594878af2f0 msgr2=0x559487c79790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:58.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.808+0000 7f7275400640 1 --2- 172.21.15.16:0/2124276592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594878af2f0 0x559487c79790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:58.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.808+0000 7f7275400640 1 -- 172.21.15.16:0/2124276592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594878d81b0 msgr2=0x559487c88a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:58.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.808+0000 7f7275400640 1 --2- 172.21.15.16:0/2124276592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594878d81b0 0x559487c88a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:58.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.808+0000 7f7275400640 1 -- 172.21.15.16:0/2124276592 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559487c95be0 con 0x5594878b74d0 2024-08-31T06:11:58.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.809+0000 7f7275400640 1 --2- 172.21.15.16:0/2124276592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594878b74d0 0x559487c7d290 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7268005eb0 tx=0x7f72680095b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e7933d1b48e8626 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:58.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.809+0000 7f726ea00640 1 -- 172.21.15.16:0/2124276592 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7268037040 con 0x5594878b74d0 2024-08-31T06:11:58.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.809+0000 7f726ea00640 1 -- 172.21.15.16:0/2124276592 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7268042050 con 0x5594878b74d0 2024-08-31T06:11:58.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.809+0000 7f726ea00640 1 -- 172.21.15.16:0/2124276592 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7268005430 con 0x5594878b74d0 2024-08-31T06:11:58.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.810+0000 7f728c004740 1 -- 172.21.15.16:0/2124276592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594878b74d0 msgr2=0x559487c7d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:58.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.810+0000 7f728c004740 1 --2- 172.21.15.16:0/2124276592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594878b74d0 0x559487c7d290 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7268005eb0 tx=0x7f72680095b0 comp rx=0 tx=0).stop 2024-08-31T06:11:58.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.810+0000 7f728c004740 1 -- 172.21.15.16:0/2124276592 shutdown_connections 2024-08-31T06:11:58.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.810+0000 7f728c004740 1 --2- 172.21.15.16:0/2124276592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594878b74d0 0x559487c7d290 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:58.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.810+0000 7f728c004740 1 --2- 172.21.15.16:0/2124276592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594878d81b0 0x559487c88a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:58.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.810+0000 7f728c004740 1 --2- 172.21.15.16:0/2124276592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594878af2f0 0x559487c79790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:58.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.810+0000 7f728c004740 1 -- 172.21.15.16:0/2124276592 >> 172.21.15.16:0/2124276592 conn(0x5594878cc300 msgr2=0x559487c88380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:58.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.811+0000 7f728c004740 1 -- 172.21.15.16:0/2124276592 shutdown_connections 2024-08-31T06:11:58.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.811+0000 7f728c004740 1 -- 172.21.15.16:0/2124276592 wait complete. 2024-08-31T06:11:58.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.811+0000 7f728c004740 1 -- start start 2024-08-31T06:11:58.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.812+0000 7f728c004740 1 Processor -- start 2024-08-31T06:11:58.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.813+0000 7f728c004740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594878af2f0 0x559487c8f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:58.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.813+0000 7f728c004740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594878b74d0 0x559487c8ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:58.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.813+0000 7f728c004740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594878d81b0 0x559487c8d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.813+0000 7f728c004740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f72801b3ea0 con 0x5594878af2f0 2024-08-31T06:11:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.813+0000 7f728c004740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f72801b5310 con 0x5594878d81b0 2024-08-31T06:11:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.813+0000 7f728c004740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f72801b4570 con 0x5594878b74d0 2024-08-31T06:11:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.813+0000 7f728c004740 10 client.0 _ll_register_callbacks cb 0x7f7278009bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:11:58.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.813+0000 7f727e800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594878d81b0 0x559487c8d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:58.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.813+0000 7f727e800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594878d81b0 0x559487c8d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:38314/0 (socket says 172.21.15.16:38314) 2024-08-31T06:11:58.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.813+0000 7f727e800640 1 -- 172.21.15.16:0/945515336 learned_addr learned my addr 172.21.15.16:0/945515336 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:11:58.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.813+0000 7f7275400640 1 --2- 172.21.15.16:0/945515336 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594878b74d0 0x559487c8ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:58.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.813+0000 7f727de00640 1 --2- 172.21.15.16:0/945515336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594878af2f0 0x559487c8f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:58.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.814+0000 7f727e800640 1 -- 172.21.15.16:0/945515336 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594878b74d0 msgr2=0x559487c8ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:58.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.814+0000 7f727e800640 1 --2- 172.21.15.16:0/945515336 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594878b74d0 0x559487c8ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:58.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.814+0000 7f727e800640 1 -- 172.21.15.16:0/945515336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594878af2f0 msgr2=0x559487c8f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:11:58.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.814+0000 7f727e800640 1 --2- 172.21.15.16:0/945515336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594878af2f0 0x559487c8f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:58.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.814+0000 7f727e800640 1 -- 172.21.15.16:0/945515336 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559487c8d980 con 0x5594878d81b0 2024-08-31T06:11:58.818 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.814+0000 7f727e800640 1 --2- 172.21.15.16:0/945515336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594878d81b0 0x559487c8d390 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7278005ab0 tx=0x7f7278009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a40e6ffecf97b3c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:11:58.818 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.814+0000 7f7263e00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:11:58.818 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.814+0000 7f7263e00640 1 -- 172.21.15.16:0/945515336 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727803c060 con 0x5594878d81b0 2024-08-31T06:11:58.818 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.815+0000 7f7263e00640 1 -- 172.21.15.16:0/945515336 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f727802ed50 con 0x5594878d81b0 2024-08-31T06:11:58.818 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.815+0000 7f728c004740 1 -- 172.21.15.16:0/945515336 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7268042050 con 0x5594878d81b0 2024-08-31T06:11:58.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.815+0000 7f7263e00640 1 -- 172.21.15.16:0/945515336 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7278029070 con 0x5594878d81b0 2024-08-31T06:11:58.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.815+0000 7f728c004740 10 client.26678 fetch_fsmap learned FSMap version 36 2024-08-31T06:11:58.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.815+0000 7f728c004740 1 -- 172.21.15.16:0/945515336 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x559487d03990 con 0x5594878d81b0 2024-08-31T06:11:58.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.815+0000 7f7263e00640 1 -- 172.21.15.16:0/945515336 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7268042050 con 0x5594878d81b0 2024-08-31T06:11:58.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.816+0000 7f7263e00640 1 -- 172.21.15.16:0/945515336 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 36) ==== 8350+0+0 (secure 0 0 0) 0x7f7278024040 con 0x5594878d81b0 2024-08-31T06:11:58.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.816+0000 7f728c004740 10 client.26678 fetch_fsmap finished waiting for FSMap version 36 2024-08-31T06:11:58.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.816+0000 7f728c004740 10 client.26678 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:11:58.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.816+0000 7f728c004740 20 client.26678 populate_metadata read hostname 'smithi016' 2024-08-31T06:11:58.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.816+0000 7f728c004740 1 --2- 172.21.15.16:0/945515336 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55948786c1c0 0x559487d06150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:11:58.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.816+0000 7f728c004740 4 client.26678 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:11:58.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.816+0000 7f728c004740 1 -- 172.21.15.16:0/945515336 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559487c897d0 con 0x55948786c1c0 2024-08-31T06:11:58.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.816+0000 7f727de00640 1 --2- 172.21.15.16:0/945515336 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55948786c1c0 0x559487d06150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:11:58.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.816+0000 7f727de00640 1 --2- 172.21.15.16:0/945515336 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55948786c1c0 0x559487d06150 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41cab5b63f95f2b1 server_cookie=37fa0d8ca337ce7a in_seq=0 out_seq=0 2024-08-31T06:11:58.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.817+0000 7f7263e00640 10 client.26678 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:11:58.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.843+0000 7f7263e00640 1 -- 172.21.15.16:0/945515336 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559487c897d0 con 0x55948786c1c0 2024-08-31T06:11:58.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.843+0000 7f7263e00640 10 client.26678 handle_command_reply: tid=0 2024-08-31T06:11:58.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.843+0000 7f728c004740 1 client.26678 shutdown 2024-08-31T06:11:58.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.844+0000 7f728c004740 1 -- 172.21.15.16:0/945515336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594878d81b0 msgr2=0x559487c8d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:58.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.844+0000 7f728c004740 1 --2- 172.21.15.16:0/945515336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594878d81b0 0x559487c8d390 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7278005ab0 tx=0x7f7278009650 comp rx=0 tx=0).stop 2024-08-31T06:11:58.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.844+0000 7f728c004740 1 -- 172.21.15.16:0/945515336 shutdown_connections 2024-08-31T06:11:58.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.844+0000 7f728c004740 1 --2- 172.21.15.16:0/945515336 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55948786c1c0 0x559487d06150 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:58.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.844+0000 7f728c004740 1 --2- 172.21.15.16:0/945515336 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594878b74d0 0x559487c8ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:58.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.844+0000 7f728c004740 1 --2- 172.21.15.16:0/945515336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594878d81b0 0x559487c8d390 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:58.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.844+0000 7f728c004740 1 --2- 172.21.15.16:0/945515336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594878af2f0 0x559487c8f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:58.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.844+0000 7f728c004740 1 -- 172.21.15.16:0/945515336 >> 172.21.15.16:0/945515336 conn(0x5594878cc300 msgr2=0x559487c7d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:58.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.844+0000 7f7263e00640 0 client.26678 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:11:58.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.845+0000 7f728c004740 1 -- 172.21.15.16:0/945515336 shutdown_connections 2024-08-31T06:11:58.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.845+0000 7f728c004740 1 -- 172.21.15.16:0/945515336 wait complete. 2024-08-31T06:11:58.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.845+0000 7f728c004740 20 client.26678 trim_cache size 0 max 16384 2024-08-31T06:11:58.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:11:58.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:11:58.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:11:58.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:11:58.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.846+0000 7f727f200640 1 -- 172.21.15.16:0/4195484560 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f72640b4dd0 msgr2=0x7f72640b72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:58.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.846+0000 7f727f200640 1 --2- 172.21.15.16:0/4195484560 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f72640b4dd0 0x7f72640b72e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7278005fd0 tx=0x7f7278011040 comp rx=0 tx=0).stop 2024-08-31T06:11:58.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.846+0000 7f727f200640 1 -- 172.21.15.16:0/4195484560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72801036b0 msgr2=0x7f728010d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:11:58.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.846+0000 7f727f200640 1 --2- 172.21.15.16:0/4195484560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72801036b0 0x7f728010d800 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f727000e9f0 tx=0x7f727000eeb0 comp rx=0 tx=0).stop 2024-08-31T06:11:58.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.846+0000 7f727f200640 1 -- 172.21.15.16:0/4195484560 shutdown_connections 2024-08-31T06:11:58.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.846+0000 7f727f200640 1 --2- 172.21.15.16:0/4195484560 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f72640b4dd0 0x7f72640b72e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:58.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.846+0000 7f727f200640 1 --2- 172.21.15.16:0/4195484560 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7280108fe0 0x7f728010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:58.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.846+0000 7f727f200640 1 --2- 172.21.15.16:0/4195484560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7280109930 0x7f72801ac3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:58.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.846+0000 7f727f200640 1 --2- 172.21.15.16:0/4195484560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72801036b0 0x7f728010d800 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:11:58.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.846+0000 7f727f200640 1 -- 172.21.15.16:0/4195484560 >> 172.21.15.16:0/4195484560 conn(0x7f72800ff0d0 msgr2=0x7f72801014f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:11:58.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.846+0000 7f727f200640 1 -- 172.21.15.16:0/4195484560 shutdown_connections 2024-08-31T06:11:58.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:11:58.847+0000 7f727f200640 1 -- 172.21.15.16:0/4195484560 wait complete. 2024-08-31T06:11:59.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:58 smithi155 ceph-mon[21742]: pgmap v506: 97 pgs: 97 active+clean; 2.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 9.6 MiB/s wr, 846 op/s 2024-08-31T06:11:59.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:58 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:11:59.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:58 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:11:59.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:11:58 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:11:59.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:58 smithi016 ceph-mon[17194]: pgmap v506: 97 pgs: 97 active+clean; 2.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 9.6 MiB/s wr, 846 op/s 2024-08-31T06:11:59.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:58 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:11:59.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:58 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:11:59.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:11:58 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:11:59.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:58 smithi184 ceph-mon[22331]: pgmap v506: 97 pgs: 97 active+clean; 2.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 9.6 MiB/s wr, 846 op/s 2024-08-31T06:11:59.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:58 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:11:59.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:58 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:11:59.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:11:58 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:12:01.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:12:00 smithi155 ceph-mon[21742]: pgmap v507: 97 pgs: 97 active+clean; 2.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 10 MiB/s wr, 920 op/s 2024-08-31T06:12:01.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:12:00 smithi016 ceph-mon[17194]: pgmap v507: 97 pgs: 97 active+clean; 2.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 10 MiB/s wr, 920 op/s 2024-08-31T06:12:01.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:12:00 smithi184 ceph-mon[22331]: pgmap v507: 97 pgs: 97 active+clean; 2.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 10 MiB/s wr, 920 op/s 2024-08-31T06:12:01.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 275.0 s, 2204.8 tps, lat 44.438 ms stddev 59.692 2024-08-31T06:12:03.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:12:02 smithi155 ceph-mon[21742]: pgmap v508: 97 pgs: 97 active+clean; 2.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.2 MiB/s wr, 694 op/s 2024-08-31T06:12:03.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:12:02 smithi016 ceph-mon[17194]: pgmap v508: 97 pgs: 97 active+clean; 2.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.2 MiB/s wr, 694 op/s 2024-08-31T06:12:03.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:12:02 smithi184 ceph-mon[22331]: pgmap v508: 97 pgs: 97 active+clean; 2.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.2 MiB/s wr, 694 op/s 2024-08-31T06:12:05.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:12:04 smithi155 ceph-mon[21742]: pgmap v509: 97 pgs: 97 active+clean; 2.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.0 MiB/s wr, 712 op/s 2024-08-31T06:12:05.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:12:04 smithi016 ceph-mon[17194]: pgmap v509: 97 pgs: 97 active+clean; 2.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.0 MiB/s wr, 712 op/s 2024-08-31T06:12:05.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:12:04 smithi184 ceph-mon[22331]: pgmap v509: 97 pgs: 97 active+clean; 2.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.0 MiB/s wr, 712 op/s 2024-08-31T06:12:06.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 280.0 s, 272.6 tps, lat 334.501 ms stddev 200.817 2024-08-31T06:12:07.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:12:06 smithi155 ceph-mon[21742]: pgmap v510: 97 pgs: 97 active+clean; 2.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.5 MiB/s wr, 737 op/s 2024-08-31T06:12:07.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:12:06 smithi016 ceph-mon[17194]: pgmap v510: 97 pgs: 97 active+clean; 2.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.5 MiB/s wr, 737 op/s 2024-08-31T06:12:07.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:12:06 smithi184 ceph-mon[22331]: pgmap v510: 97 pgs: 97 active+clean; 2.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.5 MiB/s wr, 737 op/s 2024-08-31T06:12:09.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:12:08 smithi155 ceph-mon[21742]: pgmap v511: 97 pgs: 97 active+clean; 2.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 8.3 MiB/s wr, 710 op/s 2024-08-31T06:12:09.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:12:08 smithi016 ceph-mon[17194]: pgmap v511: 97 pgs: 97 active+clean; 2.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 8.3 MiB/s wr, 710 op/s 2024-08-31T06:12:09.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:12:08 smithi184 ceph-mon[22331]: pgmap v511: 97 pgs: 97 active+clean; 2.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 8.3 MiB/s wr, 710 op/s 2024-08-31T06:12:11.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:12:10 smithi016 ceph-mon[17194]: pgmap v512: 97 pgs: 97 active+clean; 2.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 21 MiB/s wr, 1.16k op/s 2024-08-31T06:12:11.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:12:10 smithi184 ceph-mon[22331]: pgmap v512: 97 pgs: 97 active+clean; 2.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 21 MiB/s wr, 1.16k op/s 2024-08-31T06:12:11.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:12:10 smithi155 ceph-mon[21742]: pgmap v512: 97 pgs: 97 active+clean; 2.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 21 MiB/s wr, 1.16k op/s 2024-08-31T06:12:11.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 285.0 s, 1612.4 tps, lat 68.763 ms stddev 203.031 2024-08-31T06:12:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.123+0000 7fcc55a00640 1 -- 172.21.15.16:0/255325494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc50103890 msgr2=0x7fcc50103ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.123+0000 7fcc55a00640 1 --2- 172.21.15.16:0/255325494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc50103890 0x7fcc50103ca0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fcc40009900 tx=0x7fcc4002a400 comp rx=0 tx=0).stop 2024-08-31T06:12:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.124+0000 7fcc55a00640 1 -- 172.21.15.16:0/255325494 shutdown_connections 2024-08-31T06:12:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.124+0000 7fcc55a00640 1 --2- 172.21.15.16:0/255325494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc501091c0 0x7fcc501095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.124+0000 7fcc55a00640 1 --2- 172.21.15.16:0/255325494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc50109b10 0x7fcc50110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.124+0000 7fcc55a00640 1 --2- 172.21.15.16:0/255325494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc50103890 0x7fcc50103ca0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.124+0000 7fcc55a00640 1 -- 172.21.15.16:0/255325494 >> 172.21.15.16:0/255325494 conn(0x7fcc500ff310 msgr2=0x7fcc50101760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.125+0000 7fcc55a00640 1 -- 172.21.15.16:0/255325494 shutdown_connections 2024-08-31T06:12:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.125+0000 7fcc55a00640 1 -- 172.21.15.16:0/255325494 wait complete. 2024-08-31T06:12:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.126+0000 7fcc55a00640 1 Processor -- start 2024-08-31T06:12:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.126+0000 7fcc55a00640 1 -- start start 2024-08-31T06:12:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.127+0000 7fcc55a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc50103890 0x7fcc50074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.127+0000 7fcc55a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc501091c0 0x7fcc5006df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.128+0000 7fcc4ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc50103890 0x7fcc50074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:12.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.128+0000 7fcc4ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc50103890 0x7fcc50074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:36434/0 (socket says 172.21.15.16:36434) 2024-08-31T06:12:12.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.128+0000 7fcc4ea00640 1 -- 172.21.15.16:0/3966010343 learned_addr learned my addr 172.21.15.16:0/3966010343 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:12.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.128+0000 7fcc55a00640 1 --2- 172.21.15.16:0/3966010343 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc50109b10 0x7fcc5006e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:12.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.129+0000 7fcc55a00640 1 -- 172.21.15.16:0/3966010343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcc50113810 con 0x7fcc501091c0 2024-08-31T06:12:12.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.129+0000 7fcc55a00640 1 -- 172.21.15.16:0/3966010343 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcc50113690 con 0x7fcc50103890 2024-08-31T06:12:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.129+0000 7fcc55a00640 1 -- 172.21.15.16:0/3966010343 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcc50113990 con 0x7fcc50109b10 2024-08-31T06:12:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.129+0000 7fcc4f400640 1 --2- 172.21.15.16:0/3966010343 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc50109b10 0x7fcc5006e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.130+0000 7fcc4ea00640 1 -- 172.21.15.16:0/3966010343 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc50109b10 msgr2=0x7fcc5006e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.130+0000 7fcc4ea00640 1 --2- 172.21.15.16:0/3966010343 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc50109b10 0x7fcc5006e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.130+0000 7fcc4ea00640 1 -- 172.21.15.16:0/3966010343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc501091c0 msgr2=0x7fcc5006df20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-31T06:12:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.130+0000 7fcc4ea00640 1 --2- 172.21.15.16:0/3966010343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc501091c0 0x7fcc5006df20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.130+0000 7fcc4ea00640 1 -- 172.21.15.16:0/3966010343 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc5006eca0 con 0x7fcc50103890 2024-08-31T06:12:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.130+0000 7fcc4ea00640 1 --2- 172.21.15.16:0/3966010343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc50103890 0x7fcc50074c60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fcc40005ab0 tx=0x7fcc40003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.131+0000 7fcc47400640 1 -- 172.21.15.16:0/3966010343 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc40035030 con 0x7fcc50103890 2024-08-31T06:12:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.131+0000 7fcc47400640 1 -- 172.21.15.16:0/3966010343 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc40041070 con 0x7fcc50103890 2024-08-31T06:12:12.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.131+0000 7fcc47400640 1 -- 172.21.15.16:0/3966010343 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc40046470 con 0x7fcc50103890 2024-08-31T06:12:12.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.131+0000 7fcc55a00640 1 -- 172.21.15.16:0/3966010343 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc500738c0 con 0x7fcc50103890 2024-08-31T06:12:12.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.131+0000 7fcc55a00640 1 -- 172.21.15.16:0/3966010343 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc50073d60 con 0x7fcc50103890 2024-08-31T06:12:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.136+0000 7fcc47400640 1 -- 172.21.15.16:0/3966010343 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fcc400314a0 con 0x7fcc50103890 2024-08-31T06:12:12.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.139+0000 7fcc47400640 1 --2- 172.21.15.16:0/3966010343 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fcc340b4d20 0x7fcc340b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:12.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.139+0000 7fcc47400640 1 -- 172.21.15.16:0/3966010343 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fcc40109c50 con 0x7fcc50103890 2024-08-31T06:12:12.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.140+0000 7fcc4e000640 1 --2- 172.21.15.16:0/3966010343 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fcc340b4d20 0x7fcc340b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.140+0000 7fcc5c9e0740 1 Processor -- start 2024-08-31T06:12:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.141+0000 7fcc5c9e0740 1 -- start start 2024-08-31T06:12:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.141+0000 7fcc4e000640 1 --2- 172.21.15.16:0/3966010343 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fcc340b4d20 0x7fcc340b7230 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcc50108d10 tx=0x7fcc3c00b370 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.141+0000 7fcc5c9e0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601b79f01b0 0x5601b7d93690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.141+0000 7fcc5c9e0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601b79c72f0 0x5601b7d93bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.141+0000 7fcc5c9e0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601b79cf4d0 0x5601b7d94110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.141+0000 7fcc5c9e0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5601b7d08f70 con 0x5601b79cf4d0 2024-08-31T06:12:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.141+0000 7fcc5c9e0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5601b7cf8500 con 0x5601b79f01b0 2024-08-31T06:12:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.141+0000 7fcc5c9e0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5601b7cfc4b0 con 0x5601b79c72f0 2024-08-31T06:12:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.141+0000 7fcc4e000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601b79c72f0 0x5601b7d93bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.141+0000 7fcc4e000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601b79c72f0 0x5601b7d93bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:56946/0 (socket says 172.21.15.16:56946) 2024-08-31T06:12:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.141+0000 7fcc4e000640 1 -- 172.21.15.16:0/2915335192 learned_addr learned my addr 172.21.15.16:0/2915335192 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.142+0000 7fcc4f400640 1 --2- 172.21.15.16:0/2915335192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601b79cf4d0 0x5601b7d94110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.142+0000 7fcc4e000640 1 -- 172.21.15.16:0/2915335192 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601b79f01b0 msgr2=0x5601b7d93690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.142+0000 7fcc4ea00640 1 --2- 172.21.15.16:0/2915335192 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601b79f01b0 0x5601b7d93690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.142+0000 7fcc4e000640 1 --2- 172.21.15.16:0/2915335192 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601b79f01b0 0x5601b7d93690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.142+0000 7fcc4e000640 1 -- 172.21.15.16:0/2915335192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601b79cf4d0 msgr2=0x5601b7d94110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.142+0000 7fcc4e000640 1 --2- 172.21.15.16:0/2915335192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601b79cf4d0 0x5601b7d94110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.142+0000 7fcc4e000640 1 -- 172.21.15.16:0/2915335192 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601b7cb2c10 con 0x5601b79c72f0 2024-08-31T06:12:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.142+0000 7fcc4ea00640 1 --2- 172.21.15.16:0/2915335192 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601b79f01b0 0x5601b7d93690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:12:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.142+0000 7fcc4f400640 1 --2- 172.21.15.16:0/2915335192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601b79cf4d0 0x5601b7d94110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:12:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.143+0000 7fcc4e000640 1 --2- 172.21.15.16:0/2915335192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601b79c72f0 0x5601b7d93bd0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c00dd90 tx=0x7fcc3c010000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30555a68a5f05095 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.143+0000 7fcc3be00640 1 -- 172.21.15.16:0/2915335192 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c01d440 con 0x5601b79c72f0 2024-08-31T06:12:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.143+0000 7fcc3be00640 1 -- 172.21.15.16:0/2915335192 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc3c022070 con 0x5601b79c72f0 2024-08-31T06:12:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.143+0000 7fcc3be00640 1 -- 172.21.15.16:0/2915335192 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c00f7e0 con 0x5601b79c72f0 2024-08-31T06:12:12.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.144+0000 7fcc5c9e0740 1 -- 172.21.15.16:0/2915335192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601b79c72f0 msgr2=0x5601b7d93bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:12.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.144+0000 7fcc5c9e0740 1 --2- 172.21.15.16:0/2915335192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601b79c72f0 0x5601b7d93bd0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c00dd90 tx=0x7fcc3c010000 comp rx=0 tx=0).stop 2024-08-31T06:12:12.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.145+0000 7fcc5c9e0740 1 -- 172.21.15.16:0/2915335192 shutdown_connections 2024-08-31T06:12:12.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.145+0000 7fcc5c9e0740 1 --2- 172.21.15.16:0/2915335192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601b79c72f0 0x5601b7d93bd0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:12.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.145+0000 7fcc5c9e0740 1 --2- 172.21.15.16:0/2915335192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601b79cf4d0 0x5601b7d94110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:12.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.145+0000 7fcc5c9e0740 1 --2- 172.21.15.16:0/2915335192 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601b79f01b0 0x5601b7d93690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:12.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.145+0000 7fcc5c9e0740 1 -- 172.21.15.16:0/2915335192 >> 172.21.15.16:0/2915335192 conn(0x5601b79e4300 msgr2=0x5601b7d91580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:12.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.145+0000 7fcc5c9e0740 1 -- 172.21.15.16:0/2915335192 shutdown_connections 2024-08-31T06:12:12.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.145+0000 7fcc5c9e0740 1 -- 172.21.15.16:0/2915335192 wait complete. 2024-08-31T06:12:12.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.145+0000 7fcc5c9e0740 1 -- start start 2024-08-31T06:12:12.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.146+0000 7fcc5c9e0740 1 Processor -- start 2024-08-31T06:12:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.146+0000 7fcc5c9e0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601b79c72f0 0x5601b7d9ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.146+0000 7fcc5c9e0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601b79cf4d0 0x5601b7d9f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.146+0000 7fcc5c9e0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601b79f01b0 0x5601b7d9f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.146+0000 7fcc5c9e0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5601b7c31ab0 con 0x5601b79cf4d0 2024-08-31T06:12:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.146+0000 7fcc5c9e0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5601b7c9f320 con 0x5601b79c72f0 2024-08-31T06:12:12.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.146+0000 7fcc5c9e0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5601b7da1db0 con 0x5601b79f01b0 2024-08-31T06:12:12.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.146+0000 7fcc5c9e0740 10 client.0 _ll_register_callbacks cb 0x5601b7cd8e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:12:12.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.147+0000 7fcc4ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601b79c72f0 0x5601b7d9ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:12.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.147+0000 7fcc4ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601b79c72f0 0x5601b7d9ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:36448/0 (socket says 172.21.15.16:36448) 2024-08-31T06:12:12.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.147+0000 7fcc4ea00640 1 -- 172.21.15.16:0/1698365552 learned_addr learned my addr 172.21.15.16:0/1698365552 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:12.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.147+0000 7fcc4f400640 1 --2- 172.21.15.16:0/1698365552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601b79f01b0 0x5601b7d9f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.147+0000 7fcc4f400640 1 -- 172.21.15.16:0/1698365552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601b79c72f0 msgr2=0x5601b7d9ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.147+0000 7fcc4f400640 1 --2- 172.21.15.16:0/1698365552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601b79c72f0 0x5601b7d9ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.147+0000 7fcc4f400640 1 -- 172.21.15.16:0/1698365552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601b79cf4d0 msgr2=0x5601b7d9f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.147+0000 7fcc4f400640 1 --2- 172.21.15.16:0/1698365552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601b79cf4d0 0x5601b7d9f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.147+0000 7fcc4f400640 1 -- 172.21.15.16:0/1698365552 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601b7a5b790 con 0x5601b79f01b0 2024-08-31T06:12:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.148+0000 7fcc4f400640 1 --2- 172.21.15.16:0/1698365552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601b79f01b0 0x5601b7d9f6b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcc48011ae0 tx=0x7fcc48007ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a97d12ddfed1c937 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.149+0000 7fcc38c00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:12:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.149+0000 7fcc38c00640 1 -- 172.21.15.16:0/1698365552 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc48003970 con 0x5601b79f01b0 2024-08-31T06:12:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.149+0000 7fcc38c00640 1 -- 172.21.15.16:0/1698365552 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fcc48003b10 con 0x5601b79f01b0 2024-08-31T06:12:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.149+0000 7fcc5c9e0740 1 -- 172.21.15.16:0/1698365552 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc3c022070 con 0x5601b79f01b0 2024-08-31T06:12:12.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.149+0000 7fcc38c00640 1 -- 172.21.15.16:0/1698365552 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fcc48003d10 con 0x5601b79f01b0 2024-08-31T06:12:12.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.152+0000 7fcc5c9e0740 10 client.26686 fetch_fsmap learned FSMap version 36 2024-08-31T06:12:12.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.152+0000 7fcc5c9e0740 1 -- 172.21.15.16:0/1698365552 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5601b7ad4620 con 0x5601b79f01b0 2024-08-31T06:12:12.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.152+0000 7fcc38c00640 1 -- 172.21.15.16:0/1698365552 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc3c022070 con 0x5601b79f01b0 2024-08-31T06:12:12.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.152+0000 7fcc38c00640 1 -- 172.21.15.16:0/1698365552 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 36) ==== 8350+0+0 (secure 0 0 0) 0x7fcc48025670 con 0x5601b79f01b0 2024-08-31T06:12:12.155 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.152+0000 7fcc5c9e0740 10 client.26686 fetch_fsmap finished waiting for FSMap version 36 2024-08-31T06:12:12.155 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.152+0000 7fcc5c9e0740 10 client.26686 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:12:12.155 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.153+0000 7fcc5c9e0740 20 client.26686 populate_metadata read hostname 'smithi016' 2024-08-31T06:12:12.155 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.153+0000 7fcc5c9e0740 1 --2- 172.21.15.16:0/1698365552 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5601b79aed80 0x5601b7db0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:12.155 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.153+0000 7fcc5c9e0740 4 client.26686 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:12:12.155 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.153+0000 7fcc5c9e0740 1 -- 172.21.15.16:0/1698365552 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601b7d0aec0 con 0x5601b79aed80 2024-08-31T06:12:12.159 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.157+0000 7fcc4ea00640 1 --2- 172.21.15.16:0/1698365552 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5601b79aed80 0x5601b7db0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.158+0000 7fcc4ea00640 1 --2- 172.21.15.16:0/1698365552 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5601b79aed80 0x5601b7db0cb0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca0aadbdde04633 server_cookie=4edec29c49fbae3d in_seq=0 out_seq=0 2024-08-31T06:12:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.158+0000 7fcc38c00640 10 client.26686 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:12:12.163 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.159+0000 7fcc38c00640 1 -- 172.21.15.16:0/1698365552 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5601b7d0aec0 con 0x5601b79aed80 2024-08-31T06:12:12.163 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.159+0000 7fcc38c00640 10 client.26686 handle_command_reply: tid=0 2024-08-31T06:12:12.163 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.159+0000 7fcc5c9e0740 1 client.26686 shutdown 2024-08-31T06:12:12.163 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.161+0000 7fcc5c9e0740 1 -- 172.21.15.16:0/1698365552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601b79f01b0 msgr2=0x5601b7d9f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.161+0000 7fcc5c9e0740 1 --2- 172.21.15.16:0/1698365552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601b79f01b0 0x5601b7d9f6b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcc48011ae0 tx=0x7fcc48007ef0 comp rx=0 tx=0).stop 2024-08-31T06:12:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.162+0000 7fcc5c9e0740 1 -- 172.21.15.16:0/1698365552 shutdown_connections 2024-08-31T06:12:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.162+0000 7fcc5c9e0740 1 --2- 172.21.15.16:0/1698365552 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5601b79aed80 0x5601b7db0cb0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.162+0000 7fcc5c9e0740 1 --2- 172.21.15.16:0/1698365552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601b79f01b0 0x5601b7d9f6b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.162+0000 7fcc5c9e0740 1 --2- 172.21.15.16:0/1698365552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601b79cf4d0 0x5601b7d9f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.162+0000 7fcc5c9e0740 1 --2- 172.21.15.16:0/1698365552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601b79c72f0 0x5601b7d9ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.162+0000 7fcc38c00640 0 client.26686 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:12:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.162+0000 7fcc5c9e0740 1 -- 172.21.15.16:0/1698365552 >> 172.21.15.16:0/1698365552 conn(0x5601b79e4300 msgr2=0x5601b7d946c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.165+0000 7fcc5c9e0740 1 -- 172.21.15.16:0/1698365552 shutdown_connections 2024-08-31T06:12:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.165+0000 7fcc5c9e0740 1 -- 172.21.15.16:0/1698365552 wait complete. 2024-08-31T06:12:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.166+0000 7fcc5c9e0740 20 client.26686 trim_cache size 0 max 16384 2024-08-31T06:12:12.213 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.207+0000 7fcc5c9e0740 1 Processor -- start 2024-08-31T06:12:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.207+0000 7fcc5c9e0740 1 -- start start 2024-08-31T06:12:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.208+0000 7fcc5c9e0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601b79c72f0 0x5601b7d904e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.208+0000 7fcc5c9e0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601b79cf4d0 0x5601b7d94dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.208+0000 7fcc5c9e0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601b79f01b0 0x5601b7db64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.208+0000 7fcc5c9e0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcc501b82d0 con 0x5601b79c72f0 2024-08-31T06:12:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.208+0000 7fcc5c9e0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcc501b5b70 con 0x5601b79cf4d0 2024-08-31T06:12:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.208+0000 7fcc5c9e0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcc501b39c0 con 0x5601b79f01b0 2024-08-31T06:12:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.208+0000 7fcc4ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601b79c72f0 0x5601b7d904e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.208+0000 7fcc4e000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601b79cf4d0 0x5601b7d94dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.208+0000 7fcc4e000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601b79cf4d0 0x5601b7d94dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:36460/0 (socket says 172.21.15.16:36460) 2024-08-31T06:12:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.208+0000 7fcc4ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601b79c72f0 0x5601b7d904e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47264/0 (socket says 172.21.15.16:47264) 2024-08-31T06:12:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.208+0000 7fcc4e000640 1 -- 172.21.15.16:0/3102276422 learned_addr learned my addr 172.21.15.16:0/3102276422 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.208+0000 7fcc4f400640 1 --2- 172.21.15.16:0/3102276422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601b79f01b0 0x5601b7db64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.209+0000 7fcc4e000640 1 -- 172.21.15.16:0/3102276422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601b79f01b0 msgr2=0x5601b7db64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.209+0000 7fcc4e000640 1 --2- 172.21.15.16:0/3102276422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601b79f01b0 0x5601b7db64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.209+0000 7fcc4e000640 1 -- 172.21.15.16:0/3102276422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601b79c72f0 msgr2=0x5601b7d904e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.209+0000 7fcc4e000640 1 --2- 172.21.15.16:0/3102276422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601b79c72f0 0x5601b7d904e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.209+0000 7fcc4e000640 1 -- 172.21.15.16:0/3102276422 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601b7d9a760 con 0x5601b79cf4d0 2024-08-31T06:12:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.209+0000 7fcc4f400640 1 --2- 172.21.15.16:0/3102276422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601b79f01b0 0x5601b7db64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:12:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.209+0000 7fcc4e000640 1 --2- 172.21.15.16:0/3102276422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601b79cf4d0 0x5601b7d94dd0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c010c00 tx=0x7fcc3c005ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a70cbf60cfa06bef server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.209+0000 7fcc3b400640 1 -- 172.21.15.16:0/3102276422 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c005040 con 0x5601b79cf4d0 2024-08-31T06:12:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.210+0000 7fcc3b400640 1 -- 172.21.15.16:0/3102276422 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc3c02b070 con 0x5601b79cf4d0 2024-08-31T06:12:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.210+0000 7fcc3b400640 1 -- 172.21.15.16:0/3102276422 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c01d440 con 0x5601b79cf4d0 2024-08-31T06:12:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.211+0000 7fcc5c9e0740 1 -- 172.21.15.16:0/3102276422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601b79cf4d0 msgr2=0x5601b7d94dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.211+0000 7fcc5c9e0740 1 --2- 172.21.15.16:0/3102276422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601b79cf4d0 0x5601b7d94dd0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c010c00 tx=0x7fcc3c005ed0 comp rx=0 tx=0).stop 2024-08-31T06:12:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.212+0000 7fcc5c9e0740 1 -- 172.21.15.16:0/3102276422 shutdown_connections 2024-08-31T06:12:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.212+0000 7fcc5c9e0740 1 --2- 172.21.15.16:0/3102276422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601b79f01b0 0x5601b7db64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.212+0000 7fcc5c9e0740 1 --2- 172.21.15.16:0/3102276422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601b79cf4d0 0x5601b7d94dd0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.212+0000 7fcc5c9e0740 1 --2- 172.21.15.16:0/3102276422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601b79c72f0 0x5601b7d904e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.212+0000 7fcc5c9e0740 1 -- 172.21.15.16:0/3102276422 >> 172.21.15.16:0/3102276422 conn(0x5601b79e4300 msgr2=0x5601b7d946c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.212+0000 7fcc5c9e0740 1 -- 172.21.15.16:0/3102276422 shutdown_connections 2024-08-31T06:12:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.213+0000 7fcc5c9e0740 1 -- 172.21.15.16:0/3102276422 wait complete. 2024-08-31T06:12:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.213+0000 7fcc5c9e0740 1 -- start start 2024-08-31T06:12:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.214+0000 7fcc5c9e0740 1 Processor -- start 2024-08-31T06:12:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.214+0000 7fcc5c9e0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601b79c72f0 0x5601b7da52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.214+0000 7fcc5c9e0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601b79cf4d0 0x5601b7da5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.214+0000 7fcc5c9e0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601b79f01b0 0x5601b7e1a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.214+0000 7fcc5c9e0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcc501b4030 con 0x5601b79f01b0 2024-08-31T06:12:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.214+0000 7fcc5c9e0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcc501b54a0 con 0x5601b79cf4d0 2024-08-31T06:12:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.214+0000 7fcc5c9e0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcc501b4700 con 0x5601b79c72f0 2024-08-31T06:12:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.214+0000 7fcc5c9e0740 10 client.0 _ll_register_callbacks cb 0x7fcc48007940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:12:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.215+0000 7fcc4ea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601b79c72f0 0x5601b7da52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.215+0000 7fcc4ea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601b79c72f0 0x5601b7da52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:56970/0 (socket says 172.21.15.16:56970) 2024-08-31T06:12:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.215+0000 7fcc4ea00640 1 -- 172.21.15.16:0/4094778703 learned_addr learned my addr 172.21.15.16:0/4094778703 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.215+0000 7fcc4e000640 1 --2- 172.21.15.16:0/4094778703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601b79cf4d0 0x5601b7da5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.215+0000 7fcc4f400640 1 --2- 172.21.15.16:0/4094778703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601b79f01b0 0x5601b7e1a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:12.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.215+0000 7fcc4ea00640 1 -- 172.21.15.16:0/4094778703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601b79cf4d0 msgr2=0x5601b7da5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:12.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.215+0000 7fcc4ea00640 1 --2- 172.21.15.16:0/4094778703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601b79cf4d0 0x5601b7da5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:12.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.215+0000 7fcc4ea00640 1 -- 172.21.15.16:0/4094778703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601b79f01b0 msgr2=0x5601b7e1a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.215+0000 7fcc4ea00640 1 --2- 172.21.15.16:0/4094778703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601b79f01b0 0x5601b7e1a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.215+0000 7fcc4ea00640 1 -- 172.21.15.16:0/4094778703 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601b7da6030 con 0x5601b79c72f0 2024-08-31T06:12:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.215+0000 7fcc4e000640 1 --2- 172.21.15.16:0/4094778703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601b79cf4d0 0x5601b7da5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:12:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.215+0000 7fcc4ea00640 1 --2- 172.21.15.16:0/4094778703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601b79c72f0 0x5601b7da52c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcc40031440 tx=0x7fcc400d7f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=984ca3aabdea3832 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.215+0000 7fcc38c00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:12:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.215+0000 7fcc38c00640 1 -- 172.21.15.16:0/4094778703 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc400f7030 con 0x5601b79c72f0 2024-08-31T06:12:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.216+0000 7fcc38c00640 1 -- 172.21.15.16:0/4094778703 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fcc4003db70 con 0x5601b79c72f0 2024-08-31T06:12:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.216+0000 7fcc5c9e0740 1 -- 172.21.15.16:0/4094778703 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc3c02b070 con 0x5601b79c72f0 2024-08-31T06:12:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.216+0000 7fcc38c00640 1 -- 172.21.15.16:0/4094778703 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fcc400d4c10 con 0x5601b79c72f0 2024-08-31T06:12:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.216+0000 7fcc5c9e0740 10 client.26692 fetch_fsmap learned FSMap version 36 2024-08-31T06:12:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.216+0000 7fcc5c9e0740 1 -- 172.21.15.16:0/4094778703 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5601b7e1bc80 con 0x5601b79c72f0 2024-08-31T06:12:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.216+0000 7fcc38c00640 1 -- 172.21.15.16:0/4094778703 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc3c02b070 con 0x5601b79c72f0 2024-08-31T06:12:12.222 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.216+0000 7fcc38c00640 1 -- 172.21.15.16:0/4094778703 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 36) ==== 8350+0+0 (secure 0 0 0) 0x7fcc400b2050 con 0x5601b79c72f0 2024-08-31T06:12:12.222 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.216+0000 7fcc5c9e0740 10 client.26692 fetch_fsmap finished waiting for FSMap version 36 2024-08-31T06:12:12.222 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.217+0000 7fcc5c9e0740 10 client.26692 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:12:12.222 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.217+0000 7fcc5c9e0740 20 client.26692 populate_metadata read hostname 'smithi016' 2024-08-31T06:12:12.222 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.217+0000 7fcc5c9e0740 1 --2- 172.21.15.16:0/4094778703 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5601b79841c0 0x5601b7e1e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:12.222 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.217+0000 7fcc5c9e0740 4 client.26692 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:12:12.222 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.217+0000 7fcc5c9e0740 1 -- 172.21.15.16:0/4094778703 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5601b7d06960 con 0x5601b79841c0 2024-08-31T06:12:12.222 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.217+0000 7fcc4e000640 1 --2- 172.21.15.16:0/4094778703 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5601b79841c0 0x5601b7e1e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:12.222 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.217+0000 7fcc4e000640 1 --2- 172.21.15.16:0/4094778703 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5601b79841c0 0x5601b7e1e3a0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29768789a39f3813 server_cookie=907b35484fe3e034 in_seq=0 out_seq=0 2024-08-31T06:12:12.222 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.217+0000 7fcc38c00640 10 client.26692 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:12:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.234+0000 7fcc38c00640 1 -- 172.21.15.16:0/4094778703 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5601b7d06960 con 0x5601b79841c0 2024-08-31T06:12:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.234+0000 7fcc38c00640 10 client.26692 handle_command_reply: tid=0 2024-08-31T06:12:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.234+0000 7fcc5c9e0740 1 client.26692 shutdown 2024-08-31T06:12:12.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.235+0000 7fcc5c9e0740 1 -- 172.21.15.16:0/4094778703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601b79c72f0 msgr2=0x5601b7da52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:12.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.235+0000 7fcc5c9e0740 1 --2- 172.21.15.16:0/4094778703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601b79c72f0 0x5601b7da52c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcc40031440 tx=0x7fcc400d7f90 comp rx=0 tx=0).stop 2024-08-31T06:12:12.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.235+0000 7fcc5c9e0740 1 -- 172.21.15.16:0/4094778703 shutdown_connections 2024-08-31T06:12:12.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.235+0000 7fcc5c9e0740 1 --2- 172.21.15.16:0/4094778703 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5601b79841c0 0x5601b7e1e3a0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:12.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.235+0000 7fcc5c9e0740 1 --2- 172.21.15.16:0/4094778703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5601b79f01b0 0x5601b7e1a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:12.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.235+0000 7fcc5c9e0740 1 --2- 172.21.15.16:0/4094778703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601b79cf4d0 0x5601b7da5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:12.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.235+0000 7fcc5c9e0740 1 --2- 172.21.15.16:0/4094778703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601b79c72f0 0x5601b7da52c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:12.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.235+0000 7fcc5c9e0740 1 -- 172.21.15.16:0/4094778703 >> 172.21.15.16:0/4094778703 conn(0x5601b79e4300 msgr2=0x5601b7d902b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:12.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.236+0000 7fcc38c00640 0 client.26692 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:12:12.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.236+0000 7fcc5c9e0740 1 -- 172.21.15.16:0/4094778703 shutdown_connections 2024-08-31T06:12:12.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.236+0000 7fcc5c9e0740 1 -- 172.21.15.16:0/4094778703 wait complete. 2024-08-31T06:12:12.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.236+0000 7fcc5c9e0740 20 client.26692 trim_cache size 0 max 16384 2024-08-31T06:12:12.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:12:12.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:12:12.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:12:12.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:12:12.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.237+0000 7fcc55a00640 1 -- 172.21.15.16:0/3966010343 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fcc340b4d20 msgr2=0x7fcc340b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:12.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.237+0000 7fcc55a00640 1 --2- 172.21.15.16:0/3966010343 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fcc340b4d20 0x7fcc340b7230 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcc50108d10 tx=0x7fcc3c00b370 comp rx=0 tx=0).stop 2024-08-31T06:12:12.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.237+0000 7fcc55a00640 1 -- 172.21.15.16:0/3966010343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc50103890 msgr2=0x7fcc50074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:12.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.237+0000 7fcc55a00640 1 --2- 172.21.15.16:0/3966010343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc50103890 0x7fcc50074c60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fcc40005ab0 tx=0x7fcc40003cd0 comp rx=0 tx=0).stop 2024-08-31T06:12:12.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.238+0000 7fcc55a00640 1 -- 172.21.15.16:0/3966010343 shutdown_connections 2024-08-31T06:12:12.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.238+0000 7fcc55a00640 1 --2- 172.21.15.16:0/3966010343 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fcc340b4d20 0x7fcc340b7230 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:12.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.238+0000 7fcc55a00640 1 --2- 172.21.15.16:0/3966010343 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc50109b10 0x7fcc5006e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.238+0000 7fcc55a00640 1 --2- 172.21.15.16:0/3966010343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc501091c0 0x7fcc5006df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.238+0000 7fcc55a00640 1 --2- 172.21.15.16:0/3966010343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc50103890 0x7fcc50074c60 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.238+0000 7fcc55a00640 1 -- 172.21.15.16:0/3966010343 >> 172.21.15.16:0/3966010343 conn(0x7fcc500ff310 msgr2=0x7fcc50101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.239+0000 7fcc55a00640 1 -- 172.21.15.16:0/3966010343 shutdown_connections 2024-08-31T06:12:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:12.239+0000 7fcc55a00640 1 -- 172.21.15.16:0/3966010343 wait complete. 2024-08-31T06:12:13.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:12:12 smithi016 ceph-mon[17194]: pgmap v513: 97 pgs: 97 active+clean; 2.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 MiB/s wr, 1.35k op/s 2024-08-31T06:12:13.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:12:12 smithi184 ceph-mon[22331]: pgmap v513: 97 pgs: 97 active+clean; 2.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 MiB/s wr, 1.35k op/s 2024-08-31T06:12:13.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:12:12 smithi155 ceph-mon[21742]: pgmap v513: 97 pgs: 97 active+clean; 2.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 MiB/s wr, 1.35k op/s 2024-08-31T06:12:15.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:12:14 smithi016 ceph-mon[17194]: pgmap v514: 97 pgs: 97 active+clean; 2.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 31 MiB/s wr, 1.44k op/s 2024-08-31T06:12:15.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:12:14 smithi184 ceph-mon[22331]: pgmap v514: 97 pgs: 97 active+clean; 2.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 31 MiB/s wr, 1.44k op/s 2024-08-31T06:12:15.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:12:14 smithi155 ceph-mon[21742]: pgmap v514: 97 pgs: 97 active+clean; 2.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 31 MiB/s wr, 1.44k op/s 2024-08-31T06:12:16.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 290.0 s, 1918.2 tps, lat 50.666 ms stddev 67.221 2024-08-31T06:12:17.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:12:16 smithi016 ceph-mon[17194]: pgmap v515: 97 pgs: 97 active+clean; 2.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 33 MiB/s wr, 1.52k op/s 2024-08-31T06:12:17.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:12:16 smithi184 ceph-mon[22331]: pgmap v515: 97 pgs: 97 active+clean; 2.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 33 MiB/s wr, 1.52k op/s 2024-08-31T06:12:17.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:12:16 smithi155 ceph-mon[21742]: pgmap v515: 97 pgs: 97 active+clean; 2.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 33 MiB/s wr, 1.52k op/s 2024-08-31T06:12:18.549 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T06:12:19.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:12:18 smithi016 ceph-mon[17194]: pgmap v516: 97 pgs: 97 active+clean; 2.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 34 MiB/s wr, 1.43k op/s 2024-08-31T06:12:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:12:19.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:12:18 smithi184 ceph-mon[22331]: pgmap v516: 97 pgs: 97 active+clean; 2.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 34 MiB/s wr, 1.43k op/s 2024-08-31T06:12:19.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:12:18 smithi155 ceph-mon[21742]: pgmap v516: 97 pgs: 97 active+clean; 2.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 34 MiB/s wr, 1.43k op/s 2024-08-31T06:12:20.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.004+0000 7f4cdea00640 1 -- 172.21.15.16:0/3261415976 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4cd8102370 msgr2=0x7f4cd8102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:20.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.004+0000 7f4cdea00640 1 --2- 172.21.15.16:0/3261415976 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4cd8102370 0x7f4cd8102780 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0009be0 tx=0x7f4cd002a750 comp rx=0 tx=0).stop 2024-08-31T06:12:20.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.004+0000 7f4cdea00640 1 -- 172.21.15.16:0/3261415976 shutdown_connections 2024-08-31T06:12:20.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.004+0000 7f4cdea00640 1 --2- 172.21.15.16:0/3261415976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cd810ff20 0x7f4cd8112340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:20.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.004+0000 7f4cdea00640 1 --2- 172.21.15.16:0/3261415976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4cd8102cc0 0x7f4cd810f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:20.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.004+0000 7f4cdea00640 1 --2- 172.21.15.16:0/3261415976 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4cd8102370 0x7f4cd8102780 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:20.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.004+0000 7f4cdea00640 1 -- 172.21.15.16:0/3261415976 >> 172.21.15.16:0/3261415976 conn(0x7f4cd80794d0 msgr2=0x7f4cd80798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:20.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.005+0000 7f4cdea00640 1 -- 172.21.15.16:0/3261415976 shutdown_connections 2024-08-31T06:12:20.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.005+0000 7f4cdea00640 1 -- 172.21.15.16:0/3261415976 wait complete. 2024-08-31T06:12:20.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.006+0000 7f4cdea00640 1 Processor -- start 2024-08-31T06:12:20.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.007+0000 7f4cdea00640 1 -- start start 2024-08-31T06:12:20.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.008+0000 7f4cdea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4cd8102370 0x7f4cd819c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:20.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.008+0000 7f4cdea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cd8102cc0 0x7f4cd819c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:20.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.008+0000 7f4cdea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4cd810ff20 0x7f4cd81a1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:20.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.008+0000 7f4cdea00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd8115a60 con 0x7f4cd8102cc0 2024-08-31T06:12:20.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.008+0000 7f4cdea00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd81158e0 con 0x7f4cd8102370 2024-08-31T06:12:20.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.008+0000 7f4cdea00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd8115be0 con 0x7f4cd810ff20 2024-08-31T06:12:20.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.008+0000 7f4cd7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cd8102cc0 0x7f4cd819c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:20.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.008+0000 7f4cd7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cd8102cc0 0x7f4cd819c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52108/0 (socket says 172.21.15.16:52108) 2024-08-31T06:12:20.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.008+0000 7f4cd7400640 1 -- 172.21.15.16:0/2245946228 learned_addr learned my addr 172.21.15.16:0/2245946228 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:20.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.008+0000 7f4cdca00640 1 --2- 172.21.15.16:0/2245946228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4cd810ff20 0x7f4cd81a1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:20.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.009+0000 7f4cd7e00640 1 --2- 172.21.15.16:0/2245946228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4cd8102370 0x7f4cd819c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:20.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.010+0000 7f4cd7400640 1 -- 172.21.15.16:0/2245946228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4cd810ff20 msgr2=0x7f4cd81a1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:20.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.010+0000 7f4cd7400640 1 --2- 172.21.15.16:0/2245946228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4cd810ff20 0x7f4cd81a1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:20.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.010+0000 7f4cd7400640 1 -- 172.21.15.16:0/2245946228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4cd8102370 msgr2=0x7f4cd819c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:20.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.010+0000 7f4cd7400640 1 --2- 172.21.15.16:0/2245946228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4cd8102370 0x7f4cd819c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:20.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.010+0000 7f4cd7400640 1 -- 172.21.15.16:0/2245946228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cd8108790 con 0x7f4cd8102cc0 2024-08-31T06:12:20.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.010+0000 7f4cd7e00640 1 --2- 172.21.15.16:0/2245946228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4cd8102370 0x7f4cd819c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:12:20.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.010+0000 7f4cdca00640 1 --2- 172.21.15.16:0/2245946228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4cd810ff20 0x7f4cd81a1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:12:20.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.011+0000 7f4cd7400640 1 --2- 172.21.15.16:0/2245946228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cd8102cc0 0x7f4cd819c9d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4cc400e9f0 tx=0x7f4cc400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:20.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.014+0000 7f4cd4c00640 1 -- 172.21.15.16:0/2245946228 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc400ccc0 con 0x7f4cd8102cc0 2024-08-31T06:12:20.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.015+0000 7f4cd4c00640 1 -- 172.21.15.16:0/2245946228 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cc40054d0 con 0x7f4cd8102cc0 2024-08-31T06:12:20.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.015+0000 7f4cd4c00640 1 -- 172.21.15.16:0/2245946228 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc4010430 con 0x7f4cd8102cc0 2024-08-31T06:12:20.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.015+0000 7f4cdea00640 1 -- 172.21.15.16:0/2245946228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cd81a22d0 con 0x7f4cd8102cc0 2024-08-31T06:12:20.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.016+0000 7f4cdea00640 1 -- 172.21.15.16:0/2245946228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cd81a27e0 con 0x7f4cd8102cc0 2024-08-31T06:12:20.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.020+0000 7f4ce562c740 1 Processor -- start 2024-08-31T06:12:20.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.020+0000 7f4ce562c740 1 -- start start 2024-08-31T06:12:20.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.020+0000 7f4ce562c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611c72c2070 0x5611c7667950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:20.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.020+0000 7f4ce562c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5611c72991b0 0x5611c7667e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:20.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.020+0000 7f4ce562c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611c72a1390 0x5611c76683d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:20.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.020+0000 7f4ce562c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5611c75de770 con 0x5611c72c2070 2024-08-31T06:12:20.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.020+0000 7f4ce562c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5611c75cdd00 con 0x5611c72a1390 2024-08-31T06:12:20.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.020+0000 7f4ce562c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5611c75d1cb0 con 0x5611c72991b0 2024-08-31T06:12:20.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.021+0000 7f4cd7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611c72a1390 0x5611c76683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:20.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.021+0000 7f4cd7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611c72a1390 0x5611c76683d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:57082/0 (socket says 172.21.15.16:57082) 2024-08-31T06:12:20.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.021+0000 7f4cd7400640 1 -- 172.21.15.16:0/249224668 learned_addr learned my addr 172.21.15.16:0/249224668 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:20.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.021+0000 7f4cd7e00640 1 --2- 172.21.15.16:0/249224668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5611c72991b0 0x5611c7667e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:20.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.021+0000 7f4cd7e00640 1 -- 172.21.15.16:0/249224668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611c72a1390 msgr2=0x5611c76683d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:20.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.021+0000 7f4cd7e00640 1 --2- 172.21.15.16:0/249224668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611c72a1390 0x5611c76683d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:20.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.021+0000 7f4cd7e00640 1 -- 172.21.15.16:0/249224668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611c72c2070 msgr2=0x5611c7667950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:20.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.021+0000 7f4cd7e00640 1 --2- 172.21.15.16:0/249224668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611c72c2070 0x5611c7667950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:20.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.021+0000 7f4cd7e00640 1 -- 172.21.15.16:0/249224668 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611c732d650 con 0x5611c72991b0 2024-08-31T06:12:20.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.022+0000 7f4cd7e00640 1 --2- 172.21.15.16:0/249224668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5611c72991b0 0x5611c7667e90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4cd002ad40 tx=0x7f4cd0030750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70bc75ae06b3ca2e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:20.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.022+0000 7f4ccd200640 1 -- 172.21.15.16:0/249224668 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0041030 con 0x5611c72991b0 2024-08-31T06:12:20.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.022+0000 7f4ccd200640 1 -- 172.21.15.16:0/249224668 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cd0030970 con 0x5611c72991b0 2024-08-31T06:12:20.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.022+0000 7f4ccd200640 1 -- 172.21.15.16:0/249224668 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0036040 con 0x5611c72991b0 2024-08-31T06:12:20.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.023+0000 7f4ce562c740 1 -- 172.21.15.16:0/249224668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5611c72991b0 msgr2=0x5611c7667e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:20.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.023+0000 7f4ce562c740 1 --2- 172.21.15.16:0/249224668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5611c72991b0 0x5611c7667e90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4cd002ad40 tx=0x7f4cd0030750 comp rx=0 tx=0).stop 2024-08-31T06:12:20.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.024+0000 7f4ce562c740 1 -- 172.21.15.16:0/249224668 shutdown_connections 2024-08-31T06:12:20.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.024+0000 7f4ce562c740 1 --2- 172.21.15.16:0/249224668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5611c72991b0 0x5611c7667e90 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:20.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.024+0000 7f4ce562c740 1 --2- 172.21.15.16:0/249224668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611c72a1390 0x5611c76683d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:20.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.024+0000 7f4ce562c740 1 --2- 172.21.15.16:0/249224668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611c72c2070 0x5611c7667950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:20.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.024+0000 7f4ce562c740 1 -- 172.21.15.16:0/249224668 >> 172.21.15.16:0/249224668 conn(0x5611c72b6460 msgr2=0x5611c7665840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:20.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.027+0000 7f4cd4c00640 1 -- 172.21.15.16:0/2245946228 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f4cc40046c0 con 0x7f4cd8102cc0 2024-08-31T06:12:20.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.028+0000 7f4ce562c740 1 -- 172.21.15.16:0/249224668 shutdown_connections 2024-08-31T06:12:20.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.028+0000 7f4ce562c740 1 -- 172.21.15.16:0/249224668 wait complete. 2024-08-31T06:12:20.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.029+0000 7f4ce562c740 1 -- start start 2024-08-31T06:12:20.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.029+0000 7f4ce562c740 1 Processor -- start 2024-08-31T06:12:20.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.030+0000 7f4cd4c00640 1 --2- 172.21.15.16:0/2245946228 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f4cbc0b4d40 0x7f4cbc0b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:20.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.030+0000 7f4ce562c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611c72991b0 0x5611c7673600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:20.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.030+0000 7f4ce562c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5611c72a1390 0x5611c7673bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:20.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.030+0000 7f4ce562c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611c72c2070 0x5611c7679390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:20.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.030+0000 7f4ce562c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5611c7430700 con 0x5611c72c2070 2024-08-31T06:12:20.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.030+0000 7f4ce562c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5611c7554cb0 con 0x5611c72991b0 2024-08-31T06:12:20.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.030+0000 7f4ce562c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5611c76760d0 con 0x5611c72a1390 2024-08-31T06:12:20.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.030+0000 7f4ce562c740 10 client.0 _ll_register_callbacks cb 0x5611c757a430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:12:20.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.031+0000 7f4cd7400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5611c72a1390 0x5611c7673bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:20.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.031+0000 7f4cd7400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5611c72a1390 0x5611c7673bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:36448/0 (socket says 172.21.15.16:36448) 2024-08-31T06:12:20.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.031+0000 7f4cd7400640 1 -- 172.21.15.16:0/4261929726 learned_addr learned my addr 172.21.15.16:0/4261929726 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:20.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.031+0000 7f4cd7e00640 1 --2- 172.21.15.16:0/4261929726 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611c72991b0 0x5611c7673600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:20.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.031+0000 7f4cd7e00640 1 -- 172.21.15.16:0/4261929726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5611c72a1390 msgr2=0x5611c7673bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:20.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.031+0000 7f4cd7e00640 1 --2- 172.21.15.16:0/4261929726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5611c72a1390 0x5611c7673bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:20.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.031+0000 7f4cd7e00640 1 -- 172.21.15.16:0/4261929726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611c72c2070 msgr2=0x5611c7679390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:12:20.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.031+0000 7f4cd7e00640 1 --2- 172.21.15.16:0/4261929726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611c72c2070 0x5611c7679390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:20.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.031+0000 7f4cd7e00640 1 -- 172.21.15.16:0/4261929726 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611c7595870 con 0x5611c72991b0 2024-08-31T06:12:20.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.032+0000 7f4cd7e00640 1 --2- 172.21.15.16:0/4261929726 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611c72991b0 0x5611c7673600 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0034040 tx=0x7f4cd0009520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8468223d8fcd49e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:20.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.032+0000 7f4cc1600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:12:20.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.032+0000 7f4cc1600640 1 -- 172.21.15.16:0/4261929726 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd004a020 con 0x5611c72991b0 2024-08-31T06:12:20.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.033+0000 7f4cd4c00640 1 -- 172.21.15.16:0/2245946228 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f4cc40e5a00 con 0x7f4cd8102cc0 2024-08-31T06:12:20.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.033+0000 7f4cc1600640 1 -- 172.21.15.16:0/4261929726 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f4cd0030de0 con 0x5611c72991b0 2024-08-31T06:12:20.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.033+0000 7f4ce562c740 1 -- 172.21.15.16:0/4261929726 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cd0030970 con 0x5611c72991b0 2024-08-31T06:12:20.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.034+0000 7f4cc1600640 1 -- 172.21.15.16:0/4261929726 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4cd0036070 con 0x5611c72991b0 2024-08-31T06:12:20.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.034+0000 7f4ce562c740 10 client.26699 fetch_fsmap learned FSMap version 36 2024-08-31T06:12:20.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.034+0000 7f4ce562c740 1 -- 172.21.15.16:0/4261929726 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5611c76821f0 con 0x5611c72991b0 2024-08-31T06:12:20.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.034+0000 7f4cc1600640 1 -- 172.21.15.16:0/4261929726 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cd0030970 con 0x5611c72991b0 2024-08-31T06:12:20.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.034+0000 7f4cc1600640 1 -- 172.21.15.16:0/4261929726 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 36) ==== 8350+0+0 (secure 0 0 0) 0x7f4cd003d460 con 0x5611c72991b0 2024-08-31T06:12:20.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.035+0000 7f4ce562c740 10 client.26699 fetch_fsmap finished waiting for FSMap version 36 2024-08-31T06:12:20.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.035+0000 7f4ce562c740 10 client.26699 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:12:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.035+0000 7f4ce562c740 20 client.26699 populate_metadata read hostname 'smithi016' 2024-08-31T06:12:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.035+0000 7f4ce562c740 1 --2- 172.21.15.16:0/4261929726 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5611c72809f0 0x5611c7685550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.035+0000 7f4ce562c740 4 client.26699 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:12:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.035+0000 7f4ce562c740 1 -- 172.21.15.16:0/4261929726 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611c75e06c0 con 0x5611c72809f0 2024-08-31T06:12:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.035+0000 7f4cd7400640 1 --2- 172.21.15.16:0/4261929726 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5611c72809f0 0x5611c7685550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.036+0000 7f4cc1600640 10 client.26699 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:12:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.036+0000 7f4cd7400640 1 --2- 172.21.15.16:0/4261929726 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5611c72809f0 0x5611c7685550 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d73f372c1184e3ff server_cookie=46bef8a42c711a92 in_seq=0 out_seq=0 2024-08-31T06:12:20.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.037+0000 7f4cc1600640 1 -- 172.21.15.16:0/4261929726 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5611c75e06c0 con 0x5611c72809f0 2024-08-31T06:12:20.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.037+0000 7f4cc1600640 10 client.26699 handle_command_reply: tid=0 2024-08-31T06:12:20.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.037+0000 7f4ce562c740 1 client.26699 shutdown 2024-08-31T06:12:20.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.037+0000 7f4cdca00640 1 --2- 172.21.15.16:0/2245946228 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f4cbc0b4d40 0x7f4cbc0b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.041+0000 7f4cdca00640 1 --2- 172.21.15.16:0/2245946228 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f4cbc0b4d40 0x7f4cbc0b7250 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8009f20 tx=0x7f4cc80046c0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.041+0000 7f4ce562c740 1 -- 172.21.15.16:0/4261929726 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611c72991b0 msgr2=0x5611c7673600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.041+0000 7f4ce562c740 1 --2- 172.21.15.16:0/4261929726 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611c72991b0 0x5611c7673600 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0034040 tx=0x7f4cd0009520 comp rx=0 tx=0).stop 2024-08-31T06:12:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.041+0000 7f4ce562c740 1 -- 172.21.15.16:0/4261929726 shutdown_connections 2024-08-31T06:12:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.041+0000 7f4ce562c740 1 --2- 172.21.15.16:0/4261929726 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5611c72809f0 0x5611c7685550 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.041+0000 7f4ce562c740 1 --2- 172.21.15.16:0/4261929726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5611c72a1390 0x5611c7673bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.041+0000 7f4ce562c740 1 --2- 172.21.15.16:0/4261929726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611c72c2070 0x5611c7679390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.041+0000 7f4ce562c740 1 --2- 172.21.15.16:0/4261929726 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611c72991b0 0x5611c7673600 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.041+0000 7f4ce562c740 1 -- 172.21.15.16:0/4261929726 >> 172.21.15.16:0/4261929726 conn(0x5611c72b6460 msgr2=0x5611c766ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.041+0000 7f4cc1600640 0 client.26699 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:12:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.042+0000 7f4ce562c740 1 -- 172.21.15.16:0/4261929726 shutdown_connections 2024-08-31T06:12:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.042+0000 7f4ce562c740 1 -- 172.21.15.16:0/4261929726 wait complete. 2024-08-31T06:12:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.042+0000 7f4ce562c740 20 client.26699 trim_cache size 0 max 16384 2024-08-31T06:12:20.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.068+0000 7f4ce562c740 1 Processor -- start 2024-08-31T06:12:20.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.068+0000 7f4ce562c740 1 -- start start 2024-08-31T06:12:20.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.069+0000 7f4ce562c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611c72991b0 0x5611c7680170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:20.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.069+0000 7f4ce562c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611c72a1390 0x5611c766d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:20.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.069+0000 7f4ce562c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5611c72c2070 0x5611c766d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:20.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.069+0000 7f4ce562c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd81b81f0 con 0x5611c72991b0 2024-08-31T06:12:20.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.069+0000 7f4ce562c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd81b5a90 con 0x5611c72a1390 2024-08-31T06:12:20.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.069+0000 7f4ce562c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd81b3880 con 0x5611c72c2070 2024-08-31T06:12:20.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.070+0000 7f4cd7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611c72a1390 0x5611c766d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:20.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.070+0000 7f4cd7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611c72a1390 0x5611c766d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:57100/0 (socket says 172.21.15.16:57100) 2024-08-31T06:12:20.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.070+0000 7f4cd7400640 1 -- 172.21.15.16:0/972251434 learned_addr learned my addr 172.21.15.16:0/972251434 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:20.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.070+0000 7f4cdca00640 1 --2- 172.21.15.16:0/972251434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5611c72c2070 0x5611c766d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:20.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.070+0000 7f4cd7e00640 1 --2- 172.21.15.16:0/972251434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611c72991b0 0x5611c7680170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:20.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.071+0000 7f4cdca00640 1 -- 172.21.15.16:0/972251434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611c72a1390 msgr2=0x5611c766d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:20.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.071+0000 7f4cdca00640 1 --2- 172.21.15.16:0/972251434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611c72a1390 0x5611c766d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:20.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.071+0000 7f4cdca00640 1 -- 172.21.15.16:0/972251434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611c72991b0 msgr2=0x5611c7680170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:20.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.071+0000 7f4cdca00640 1 --2- 172.21.15.16:0/972251434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611c72991b0 0x5611c7680170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:20.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.071+0000 7f4cdca00640 1 -- 172.21.15.16:0/972251434 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611c7672360 con 0x5611c72c2070 2024-08-31T06:12:20.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.072+0000 7f4cdca00640 1 --2- 172.21.15.16:0/972251434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5611c72c2070 0x5611c766d7c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8011ca0 tx=0x7f4cc8014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d3997b12b466fe9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:20.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.072+0000 7f4cc3e00640 1 -- 172.21.15.16:0/972251434 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc801e420 con 0x5611c72c2070 2024-08-31T06:12:20.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.073+0000 7f4cc3e00640 1 -- 172.21.15.16:0/972251434 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cc801ea70 con 0x5611c72c2070 2024-08-31T06:12:20.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.073+0000 7f4cc3e00640 1 -- 172.21.15.16:0/972251434 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc801c960 con 0x5611c72c2070 2024-08-31T06:12:20.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.074+0000 7f4ce562c740 1 -- 172.21.15.16:0/972251434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5611c72c2070 msgr2=0x5611c766d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:20.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.074+0000 7f4ce562c740 1 --2- 172.21.15.16:0/972251434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5611c72c2070 0x5611c766d7c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8011ca0 tx=0x7f4cc8014000 comp rx=0 tx=0).stop 2024-08-31T06:12:20.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.074+0000 7f4ce562c740 1 -- 172.21.15.16:0/972251434 shutdown_connections 2024-08-31T06:12:20.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.074+0000 7f4ce562c740 1 --2- 172.21.15.16:0/972251434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5611c72c2070 0x5611c766d7c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:20.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.074+0000 7f4ce562c740 1 --2- 172.21.15.16:0/972251434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611c72a1390 0x5611c766d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:20.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.074+0000 7f4ce562c740 1 --2- 172.21.15.16:0/972251434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611c72991b0 0x5611c7680170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:20.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.074+0000 7f4ce562c740 1 -- 172.21.15.16:0/972251434 >> 172.21.15.16:0/972251434 conn(0x5611c72b6460 msgr2=0x5611c766ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:20.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.074+0000 7f4ce562c740 1 -- 172.21.15.16:0/972251434 shutdown_connections 2024-08-31T06:12:20.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.074+0000 7f4ce562c740 1 -- 172.21.15.16:0/972251434 wait complete. 2024-08-31T06:12:20.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.075+0000 7f4ce562c740 1 -- start start 2024-08-31T06:12:20.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.075+0000 7f4ce562c740 1 Processor -- start 2024-08-31T06:12:20.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.075+0000 7f4ce562c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611c72991b0 0x5611c767e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:20.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.075+0000 7f4ce562c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5611c72a1390 0x5611c767e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:20.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.075+0000 7f4ce562c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611c72c2070 0x5611c76e8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:20.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.075+0000 7f4ce562c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd81b3f50 con 0x5611c72991b0 2024-08-31T06:12:20.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.075+0000 7f4ce562c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd81b53c0 con 0x5611c72c2070 2024-08-31T06:12:20.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.075+0000 7f4ce562c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd81b4620 con 0x5611c72a1390 2024-08-31T06:12:20.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.075+0000 7f4ce562c740 10 client.0 _ll_register_callbacks cb 0x7f4cd0009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:12:20.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.076+0000 7f4cd7400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5611c72a1390 0x5611c767e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:20.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.076+0000 7f4cdca00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611c72c2070 0x5611c76e8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:20.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.077+0000 7f4cdca00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611c72c2070 0x5611c76e8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:57110/0 (socket says 172.21.15.16:57110) 2024-08-31T06:12:20.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.077+0000 7f4cdca00640 1 -- 172.21.15.16:0/3668059383 learned_addr learned my addr 172.21.15.16:0/3668059383 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:20.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.077+0000 7f4cd7e00640 1 --2- 172.21.15.16:0/3668059383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611c72991b0 0x5611c767e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:20.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.076+0000 7f4cd7400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5611c72a1390 0x5611c767e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:36468/0 (socket says 172.21.15.16:36468) 2024-08-31T06:12:20.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.077+0000 7f4cdca00640 1 -- 172.21.15.16:0/3668059383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5611c72a1390 msgr2=0x5611c767e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:20.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.077+0000 7f4cdca00640 1 --2- 172.21.15.16:0/3668059383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5611c72a1390 0x5611c767e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:20.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.077+0000 7f4cdca00640 1 -- 172.21.15.16:0/3668059383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611c72991b0 msgr2=0x5611c767e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:20.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.077+0000 7f4cdca00640 1 --2- 172.21.15.16:0/3668059383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611c72991b0 0x5611c767e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:20.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.077+0000 7f4cdca00640 1 -- 172.21.15.16:0/3668059383 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611c76e9310 con 0x5611c72c2070 2024-08-31T06:12:20.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.078+0000 7f4cd7e00640 1 --2- 172.21.15.16:0/3668059383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611c72991b0 0x5611c767e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:12:20.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.078+0000 7f4cdca00640 1 --2- 172.21.15.16:0/3668059383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611c72c2070 0x5611c76e8ca0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4cc801c500 tx=0x7f4cc8005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41c364f97f2826ad server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:20.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.078+0000 7f4cc1600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:12:20.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.079+0000 7f4cc1600640 1 -- 172.21.15.16:0/3668059383 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc801c870 con 0x5611c72c2070 2024-08-31T06:12:20.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.079+0000 7f4ce562c740 1 -- 172.21.15.16:0/3668059383 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cc801ea70 con 0x5611c72c2070 2024-08-31T06:12:20.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.079+0000 7f4cc1600640 1 -- 172.21.15.16:0/3668059383 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f4cc801ca10 con 0x5611c72c2070 2024-08-31T06:12:20.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.080+0000 7f4cc1600640 1 -- 172.21.15.16:0/3668059383 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4cc803e050 con 0x5611c72c2070 2024-08-31T06:12:20.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.080+0000 7f4ce562c740 10 client.26705 fetch_fsmap learned FSMap version 36 2024-08-31T06:12:20.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.080+0000 7f4ce562c740 1 -- 172.21.15.16:0/3668059383 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5611c76f06f0 con 0x5611c72c2070 2024-08-31T06:12:20.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.080+0000 7f4cc1600640 1 -- 172.21.15.16:0/3668059383 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cc801ea70 con 0x5611c72c2070 2024-08-31T06:12:20.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.080+0000 7f4cc1600640 1 -- 172.21.15.16:0/3668059383 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 36) ==== 8350+0+0 (secure 0 0 0) 0x7f4cc8030cb0 con 0x5611c72c2070 2024-08-31T06:12:20.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.081+0000 7f4ce562c740 10 client.26705 fetch_fsmap finished waiting for FSMap version 36 2024-08-31T06:12:20.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.081+0000 7f4ce562c740 10 client.26705 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:12:20.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.081+0000 7f4ce562c740 20 client.26705 populate_metadata read hostname 'smithi016' 2024-08-31T06:12:20.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.081+0000 7f4ce562c740 1 --2- 172.21.15.16:0/3668059383 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5611c7255da0 0x5611c76f2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:20.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.081+0000 7f4ce562c740 4 client.26705 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T06:12:20.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.081+0000 7f4ce562c740 1 -- 172.21.15.16:0/3668059383 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5611c75dc160 con 0x5611c7255da0 2024-08-31T06:12:20.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.081+0000 7f4cd7e00640 1 --2- 172.21.15.16:0/3668059383 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5611c7255da0 0x5611c76f2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:20.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.082+0000 7f4cd7e00640 1 --2- 172.21.15.16:0/3668059383 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5611c7255da0 0x5611c76f2d70 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d73509f859727892 server_cookie=bb0f0bbd455e2a58 in_seq=0 out_seq=0 2024-08-31T06:12:20.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.082+0000 7f4cc1600640 10 client.26705 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:12:20.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.082+0000 7f4cc1600640 1 -- 172.21.15.16:0/3668059383 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2078 (crc 0 0 0) 0x5611c75dc160 con 0x5611c7255da0 2024-08-31T06:12:20.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.082+0000 7f4cc1600640 10 client.26705 handle_command_reply: tid=0 2024-08-31T06:12:20.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.082+0000 7f4ce562c740 1 client.26705 shutdown 2024-08-31T06:12:20.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.083+0000 7f4ce562c740 1 -- 172.21.15.16:0/3668059383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611c72c2070 msgr2=0x5611c76e8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:20.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.083+0000 7f4ce562c740 1 --2- 172.21.15.16:0/3668059383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611c72c2070 0x5611c76e8ca0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4cc801c500 tx=0x7f4cc8005740 comp rx=0 tx=0).stop 2024-08-31T06:12:20.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.083+0000 7f4ce562c740 1 -- 172.21.15.16:0/3668059383 shutdown_connections 2024-08-31T06:12:20.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.083+0000 7f4ce562c740 1 --2- 172.21.15.16:0/3668059383 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5611c7255da0 0x5611c76f2d70 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:20.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.083+0000 7f4ce562c740 1 --2- 172.21.15.16:0/3668059383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5611c72a1390 0x5611c767e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:20.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.083+0000 7f4ce562c740 1 --2- 172.21.15.16:0/3668059383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611c72c2070 0x5611c76e8ca0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:20.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.083+0000 7f4ce562c740 1 --2- 172.21.15.16:0/3668059383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611c72991b0 0x5611c767e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:20.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.083+0000 7f4ce562c740 1 -- 172.21.15.16:0/3668059383 >> 172.21.15.16:0/3668059383 conn(0x5611c72b6460 msgr2=0x5611c7668980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:20.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.083+0000 7f4cc1600640 0 client.26705 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:12:20.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.083+0000 7f4ce562c740 1 -- 172.21.15.16:0/3668059383 shutdown_connections 2024-08-31T06:12:20.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.083+0000 7f4ce562c740 1 -- 172.21.15.16:0/3668059383 wait complete. 2024-08-31T06:12:20.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.084+0000 7f4ce562c740 20 client.26705 trim_cache size 0 max 16384 2024-08-31T06:12:20.088 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:12:20.088 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-08-31T06:12:20.088 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {}, 2024-08-31T06:12:20.088 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-08-31T06:12:20.088 INFO:teuthology.orchestra.run.smithi016.stdout: "e739a16e-8ace-43a8-a1ce-ac27d39ef8a3": { 2024-08-31T06:12:20.089 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:54.084646+0000", 2024-08-31T06:12:20.089 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:12:20.089 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:12:20.089 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:12:20.089 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:12:20.089 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:12:20.089 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:12:20.089 INFO:teuthology.orchestra.run.smithi016.stdout: "a10cce76-99ab-46b5-b3b2-07b82899b8f9": { 2024-08-31T06:12:20.089 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:11:43.967546+0000", 2024-08-31T06:12:20.089 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:12:20.089 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:12:20.089 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:12:20.089 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:12:20.089 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:12:20.090 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:12:20.090 INFO:teuthology.orchestra.run.smithi016.stdout: "c1fd6904-2e02-47fa-939a-47349b9fe36b": { 2024-08-31T06:12:20.090 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:06.678691+0000", 2024-08-31T06:12:20.090 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:12:20.090 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:12:20.090 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:12:20.090 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:12:20.090 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:12:20.090 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:12:20.090 INFO:teuthology.orchestra.run.smithi016.stdout: "78d6ef02-8551-45e8-ad60-9627eb573fc1": { 2024-08-31T06:12:20.090 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:09:13.547865+0000", 2024-08-31T06:12:20.090 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:12:20.090 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:12:20.090 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:12:20.090 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:12:20.091 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:12:20.091 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:12:20.091 INFO:teuthology.orchestra.run.smithi016.stdout: "c8455d74-2fe7-4d93-bcf2-19b419d16016": { 2024-08-31T06:12:20.091 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:08:24.035960+0000", 2024-08-31T06:12:20.091 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:12:20.091 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:12:20.091 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:12:20.091 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:12:20.091 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:12:20.091 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:12:20.091 INFO:teuthology.orchestra.run.smithi016.stdout: "88d7bdb8-cb1c-45e5-ad29-4cd8da6c6003": { 2024-08-31T06:12:20.091 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:07:39.648942+0000", 2024-08-31T06:12:20.091 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:12:20.091 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:12:20.092 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:12:20.092 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:12:20.092 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:12:20.092 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:12:20.092 INFO:teuthology.orchestra.run.smithi016.stdout: "c300b7e8-f3fd-482c-a909-9329bc61e570": { 2024-08-31T06:12:20.092 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:06:49.868046+0000", 2024-08-31T06:12:20.092 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:12:20.092 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:12:20.092 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:12:20.092 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:12:20.092 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:12:20.092 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:12:20.092 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:12:20.092 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:12:20.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.086+0000 7f4cdea00640 1 -- 172.21.15.16:0/2245946228 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f4cbc0b4d40 msgr2=0x7f4cbc0b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:20.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.086+0000 7f4cdea00640 1 --2- 172.21.15.16:0/2245946228 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f4cbc0b4d40 0x7f4cbc0b7250 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8009f20 tx=0x7f4cc80046c0 comp rx=0 tx=0).stop 2024-08-31T06:12:20.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.086+0000 7f4cdea00640 1 -- 172.21.15.16:0/2245946228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cd8102cc0 msgr2=0x7f4cd819c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:20.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.086+0000 7f4cdea00640 1 --2- 172.21.15.16:0/2245946228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cd8102cc0 0x7f4cd819c9d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4cc400e9f0 tx=0x7f4cc400eeb0 comp rx=0 tx=0).stop 2024-08-31T06:12:20.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.087+0000 7f4cdea00640 1 -- 172.21.15.16:0/2245946228 shutdown_connections 2024-08-31T06:12:20.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.087+0000 7f4cdea00640 1 --2- 172.21.15.16:0/2245946228 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f4cbc0b4d40 0x7f4cbc0b7250 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:20.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.087+0000 7f4cdea00640 1 --2- 172.21.15.16:0/2245946228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4cd810ff20 0x7f4cd81a1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:20.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.087+0000 7f4cdea00640 1 --2- 172.21.15.16:0/2245946228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cd8102cc0 0x7f4cd819c9d0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:20.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.087+0000 7f4cdea00640 1 --2- 172.21.15.16:0/2245946228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4cd8102370 0x7f4cd819c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:20.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.087+0000 7f4cdea00640 1 -- 172.21.15.16:0/2245946228 >> 172.21.15.16:0/2245946228 conn(0x7f4cd80794d0 msgr2=0x7f4cd81113a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:20.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.087+0000 7f4cdea00640 1 -- 172.21.15.16:0/2245946228 shutdown_connections 2024-08-31T06:12:20.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:20.087+0000 7f4cdea00640 1 -- 172.21.15.16:0/2245946228 wait complete. 2024-08-31T06:12:21.088 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T06:12:21.089 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:12:21.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:12:20 smithi016 ceph-mon[17194]: pgmap v517: 97 pgs: 97 active+clean; 2.7 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 36 MiB/s wr, 1.47k op/s 2024-08-31T06:12:21.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:12:20 smithi184 ceph-mon[22331]: pgmap v517: 97 pgs: 97 active+clean; 2.7 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 36 MiB/s wr, 1.47k op/s 2024-08-31T06:12:21.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:12:20 smithi155 ceph-mon[21742]: pgmap v517: 97 pgs: 97 active+clean; 2.7 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 36 MiB/s wr, 1.47k op/s 2024-08-31T06:12:21.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 295.0 s, 1693.4 tps, lat 60.588 ms stddev 90.582 2024-08-31T06:12:21.759 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:12:22.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:22.694+0000 7f167cc00640 1 -- 172.21.15.16:0/372399032 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1678107f50 msgr2=0x7f1678108360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:22.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:22.694+0000 7f167cc00640 1 --2- 172.21.15.16:0/372399032 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1678107f50 0x7f1678108360 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f166401a910 tx=0x7f166403b430 comp rx=0 tx=0).stop 2024-08-31T06:12:22.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:22.695+0000 7f167cc00640 1 -- 172.21.15.16:0/372399032 shutdown_connections 2024-08-31T06:12:22.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:22.695+0000 7f167cc00640 1 --2- 172.21.15.16:0/372399032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16781088a0 0x7f167810f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:22.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:22.695+0000 7f167cc00640 1 --2- 172.21.15.16:0/372399032 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1678107f50 0x7f1678108360 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:22.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:22.695+0000 7f167cc00640 1 --2- 172.21.15.16:0/372399032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1678102610 0x7f1678102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:22.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:22.695+0000 7f167cc00640 1 -- 172.21.15.16:0/372399032 >> 172.21.15.16:0/372399032 conn(0x7f16780fe060 msgr2=0x7f16781004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:22.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:22.695+0000 7f167cc00640 1 -- 172.21.15.16:0/372399032 shutdown_connections 2024-08-31T06:12:22.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:22.695+0000 7f167cc00640 1 -- 172.21.15.16:0/372399032 wait complete. 2024-08-31T06:12:22.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:22.697+0000 7f167cc00640 1 Processor -- start 2024-08-31T06:12:22.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:22.698+0000 7f167cc00640 1 -- start start 2024-08-31T06:12:22.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:22.698+0000 7f167cc00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1678102610 0x7f167810efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:22.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:22.698+0000 7f167cc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1678107f50 0x7f167810c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:22.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:22.698+0000 7f167cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16781088a0 0x7f167810c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:22.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:22.699+0000 7f1676a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1678107f50 0x7f167810c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:22.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:22.699+0000 7f167cc00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f167806f410 con 0x7f16781088a0 2024-08-31T06:12:22.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:22.699+0000 7f167cc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f167806f290 con 0x7f1678107f50 2024-08-31T06:12:22.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:22.699+0000 7f167cc00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f167806f590 con 0x7f1678102610 2024-08-31T06:12:22.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:22.699+0000 7f1676a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1678107f50 0x7f167810c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:57142/0 (socket says 172.21.15.16:57142) 2024-08-31T06:12:22.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:22.699+0000 7f1676a00640 1 -- 172.21.15.16:0/637808185 learned_addr learned my addr 172.21.15.16:0/637808185 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:22.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:22.699+0000 7f1677e00640 1 --2- 172.21.15.16:0/637808185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16781088a0 0x7f167810c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:22.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:22.700+0000 7f1677400640 1 --2- 172.21.15.16:0/637808185 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1678102610 0x7f167810efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:22.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:22.701+0000 7f1677e00640 1 -- 172.21.15.16:0/637808185 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1678102610 msgr2=0x7f167810efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:22.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:22.701+0000 7f1677e00640 1 --2- 172.21.15.16:0/637808185 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1678102610 0x7f167810efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:22.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:22.701+0000 7f1677e00640 1 -- 172.21.15.16:0/637808185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1678107f50 msgr2=0x7f167810c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:22.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:22.701+0000 7f1677e00640 1 --2- 172.21.15.16:0/637808185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1678107f50 0x7f167810c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:22.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:22.701+0000 7f1677e00640 1 -- 172.21.15.16:0/637808185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f167819da90 con 0x7f16781088a0 2024-08-31T06:12:22.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:22.701+0000 7f1677400640 1 --2- 172.21.15.16:0/637808185 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1678102610 0x7f167810efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:12:22.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:22.702+0000 7f1677e00640 1 --2- 172.21.15.16:0/637808185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16781088a0 0x7f167810c930 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f167000e940 tx=0x7f167000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:22.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:22.703+0000 7f166fe00640 1 -- 172.21.15.16:0/637808185 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1670004d20 con 0x7f16781088a0 2024-08-31T06:12:22.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:22.703+0000 7f167cc00640 1 -- 172.21.15.16:0/637808185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16780fe470 con 0x7f16781088a0 2024-08-31T06:12:22.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:22.704+0000 7f167cc00640 1 -- 172.21.15.16:0/637808185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16780fea00 con 0x7f16781088a0 2024-08-31T06:12:22.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:22.704+0000 7f166fe00640 1 -- 172.21.15.16:0/637808185 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1670005020 con 0x7f16781088a0 2024-08-31T06:12:22.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:22.704+0000 7f166fe00640 1 -- 172.21.15.16:0/637808185 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16700116d0 con 0x7f16781088a0 2024-08-31T06:12:22.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:22.705+0000 7f167cc00640 1 -- 172.21.15.16:0/637808185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16580052c0 con 0x7f16781088a0 2024-08-31T06:12:22.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:22.713+0000 7f166fe00640 1 -- 172.21.15.16:0/637808185 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f167000cb20 con 0x7f16781088a0 2024-08-31T06:12:22.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:22.715+0000 7f166fe00640 1 --2- 172.21.15.16:0/637808185 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f165c0b4da0 0x7f165c0b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:22.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:22.716+0000 7f166fe00640 1 -- 172.21.15.16:0/637808185 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f16700e56d0 con 0x7f16781088a0 2024-08-31T06:12:22.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:22.716+0000 7f1677400640 1 --2- 172.21.15.16:0/637808185 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f165c0b4da0 0x7f165c0b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:22.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:22.717+0000 7f1677400640 1 --2- 172.21.15.16:0/637808185 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f165c0b4da0 0x7f165c0b72b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f167810ed00 tx=0x7f1668005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:22.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:22.725+0000 7f166fe00640 1 -- 172.21.15.16:0/637808185 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f1670016030 con 0x7f16781088a0 2024-08-31T06:12:23.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:23.123+0000 7f167cc00640 1 -- 172.21.15.16:0/637808185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1658005880 con 0x7f16781088a0 2024-08-31T06:12:23.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:23.124+0000 7f166fe00640 1 -- 172.21.15.16:0/637808185 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11918 (secure 0 0 0) 0x7f16700ac4c0 con 0x7f16781088a0 2024-08-31T06:12:23.127 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:12:23.127 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":36,"btime":"2024-08-31T06:11:48:781243+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:11:48.335944+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24712,24724,24737]},"id":1}]} 2024-08-31T06:12:23.128 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 36 2024-08-31T06:12:23.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:23.127+0000 7f167cc00640 1 -- 172.21.15.16:0/637808185 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f165c0b4da0 msgr2=0x7f165c0b72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:23.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:23.127+0000 7f167cc00640 1 --2- 172.21.15.16:0/637808185 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f165c0b4da0 0x7f165c0b72b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f167810ed00 tx=0x7f1668005f90 comp rx=0 tx=0).stop 2024-08-31T06:12:23.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:23.127+0000 7f167cc00640 1 -- 172.21.15.16:0/637808185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16781088a0 msgr2=0x7f167810c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:23.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:23.127+0000 7f167cc00640 1 --2- 172.21.15.16:0/637808185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16781088a0 0x7f167810c930 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f167000e940 tx=0x7f167000ee00 comp rx=0 tx=0).stop 2024-08-31T06:12:23.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:23.128+0000 7f167cc00640 1 -- 172.21.15.16:0/637808185 shutdown_connections 2024-08-31T06:12:23.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:23.128+0000 7f167cc00640 1 --2- 172.21.15.16:0/637808185 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f165c0b4da0 0x7f165c0b72b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:23.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:23.128+0000 7f167cc00640 1 --2- 172.21.15.16:0/637808185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16781088a0 0x7f167810c930 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:23.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:23.128+0000 7f167cc00640 1 --2- 172.21.15.16:0/637808185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1678107f50 0x7f167810c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:23.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:23.128+0000 7f167cc00640 1 --2- 172.21.15.16:0/637808185 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1678102610 0x7f167810efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:23.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:23.128+0000 7f167cc00640 1 -- 172.21.15.16:0/637808185 >> 172.21.15.16:0/637808185 conn(0x7f16780fe060 msgr2=0x7f1678101100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:23.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:23.128+0000 7f167cc00640 1 -- 172.21.15.16:0/637808185 shutdown_connections 2024-08-31T06:12:23.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:23.128+0000 7f167cc00640 1 -- 172.21.15.16:0/637808185 wait complete. 2024-08-31T06:12:23.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:12:22 smithi016 ceph-mon[17194]: pgmap v518: 97 pgs: 97 active+clean; 2.7 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 28 MiB/s wr, 1.06k op/s 2024-08-31T06:12:23.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:12:22 smithi184 ceph-mon[22331]: pgmap v518: 97 pgs: 97 active+clean; 2.7 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 28 MiB/s wr, 1.06k op/s 2024-08-31T06:12:23.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:12:22 smithi155 ceph-mon[21742]: pgmap v518: 97 pgs: 97 active+clean; 2.7 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 28 MiB/s wr, 1.06k op/s 2024-08-31T06:12:23.819 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 32.7 2024-08-31T06:12:23.819 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:12:24.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:12:23 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/637808185' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:12:24.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:12:23 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/637808185' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:12:24.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:12:23 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/637808185' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:12:24.376 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T06:12:24.524 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:12:25.070 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:12:24 smithi016 ceph-mon[17194]: pgmap v519: 97 pgs: 97 active+clean; 2.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 21 MiB/s wr, 813 op/s 2024-08-31T06:12:25.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:12:24 smithi184 ceph-mon[22331]: pgmap v519: 97 pgs: 97 active+clean; 2.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 21 MiB/s wr, 813 op/s 2024-08-31T06:12:25.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.357+0000 7fd961600640 1 -- 172.21.15.16:0/2249724061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd95c075e40 msgr2=0x7fd95c10f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:25.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.357+0000 7fd961600640 1 --2- 172.21.15.16:0/2249724061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd95c075e40 0x7fd95c10f7d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd944009900 tx=0x7fd94402a400 comp rx=0 tx=0).stop 2024-08-31T06:12:25.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.358+0000 7fd961600640 1 -- 172.21.15.16:0/2249724061 shutdown_connections 2024-08-31T06:12:25.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.358+0000 7fd961600640 1 --2- 172.21.15.16:0/2249724061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd95c10fd10 0x7fd95c112130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:25.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.358+0000 7fd961600640 1 --2- 172.21.15.16:0/2249724061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd95c075e40 0x7fd95c10f7d0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:25.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.358+0000 7fd961600640 1 --2- 172.21.15.16:0/2249724061 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd95c0754f0 0x7fd95c075900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:25.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.358+0000 7fd961600640 1 -- 172.21.15.16:0/2249724061 >> 172.21.15.16:0/2249724061 conn(0x7fd95c0ff1a0 msgr2=0x7fd95c1015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:25.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.358+0000 7fd961600640 1 -- 172.21.15.16:0/2249724061 shutdown_connections 2024-08-31T06:12:25.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.359+0000 7fd961600640 1 -- 172.21.15.16:0/2249724061 wait complete. 2024-08-31T06:12:25.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.361+0000 7fd961600640 1 Processor -- start 2024-08-31T06:12:25.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.366+0000 7fd961600640 1 -- start start 2024-08-31T06:12:25.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:12:24 smithi155 ceph-mon[21742]: pgmap v519: 97 pgs: 97 active+clean; 2.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 21 MiB/s wr, 813 op/s 2024-08-31T06:12:25.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.370+0000 7fd961600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd95c0754f0 0x7fd95c074da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:25.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.371+0000 7fd961600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd95c10fd10 0x7fd95c06dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:25.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.372+0000 7fd95be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd95c0754f0 0x7fd95c074da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:25.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.373+0000 7fd961600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd95c06e5e0 0x7fd95c073950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:25.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.373+0000 7fd961600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd95c1158d0 con 0x7fd95c10fd10 2024-08-31T06:12:25.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.373+0000 7fd961600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd95c115750 con 0x7fd95c0754f0 2024-08-31T06:12:25.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.373+0000 7fd961600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd95c115a50 con 0x7fd95c06e5e0 2024-08-31T06:12:25.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.373+0000 7fd95be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd95c0754f0 0x7fd95c074da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:57160/0 (socket says 172.21.15.16:57160) 2024-08-31T06:12:25.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.373+0000 7fd95be00640 1 -- 172.21.15.16:0/967683936 learned_addr learned my addr 172.21.15.16:0/967683936 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:25.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.373+0000 7fd960c00640 1 --2- 172.21.15.16:0/967683936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd95c06e5e0 0x7fd95c073950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:25.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.373+0000 7fd95be00640 1 -- 172.21.15.16:0/967683936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd95c06e5e0 msgr2=0x7fd95c073950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:25.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.374+0000 7fd95be00640 1 --2- 172.21.15.16:0/967683936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd95c06e5e0 0x7fd95c073950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:25.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.374+0000 7fd95be00640 1 -- 172.21.15.16:0/967683936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd95c10fd10 msgr2=0x7fd95c06dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:25.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.374+0000 7fd95be00640 1 --2- 172.21.15.16:0/967683936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd95c10fd10 0x7fd95c06dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:25.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.374+0000 7fd95be00640 1 -- 172.21.15.16:0/967683936 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd95c06edc0 con 0x7fd95c0754f0 2024-08-31T06:12:25.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.374+0000 7fd95be00640 1 --2- 172.21.15.16:0/967683936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd95c0754f0 0x7fd95c074da0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd94c00c890 tx=0x7fd94c00cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:25.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.375+0000 7fd959600640 1 -- 172.21.15.16:0/967683936 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94c004220 con 0x7fd95c0754f0 2024-08-31T06:12:25.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.375+0000 7fd959600640 1 -- 172.21.15.16:0/967683936 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd94c0043c0 con 0x7fd95c0754f0 2024-08-31T06:12:25.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.375+0000 7fd959600640 1 -- 172.21.15.16:0/967683936 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94c004a00 con 0x7fd95c0754f0 2024-08-31T06:12:25.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.375+0000 7fd961600640 1 -- 172.21.15.16:0/967683936 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd95c073fb0 con 0x7fd95c0754f0 2024-08-31T06:12:25.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.375+0000 7fd961600640 1 -- 172.21.15.16:0/967683936 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd95c074470 con 0x7fd95c0754f0 2024-08-31T06:12:25.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.378+0000 7fd961600640 1 -- 172.21.15.16:0/967683936 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd95c074760 con 0x7fd95c0754f0 2024-08-31T06:12:25.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.385+0000 7fd959600640 1 -- 172.21.15.16:0/967683936 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fd94c0050b0 con 0x7fd95c0754f0 2024-08-31T06:12:25.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.389+0000 7fd959600640 1 --2- 172.21.15.16:0/967683936 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd9400b4d50 0x7fd9400b7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:25.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.389+0000 7fd959600640 1 -- 172.21.15.16:0/967683936 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fd94c0e5510 con 0x7fd95c0754f0 2024-08-31T06:12:25.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.390+0000 7fd953e00640 1 --2- 172.21.15.16:0/967683936 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd9400b4d50 0x7fd9400b7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:25.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.396+0000 7fd959600640 1 -- 172.21.15.16:0/967683936 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fd94c014030 con 0x7fd95c0754f0 2024-08-31T06:12:25.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.396+0000 7fd953e00640 1 --2- 172.21.15.16:0/967683936 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd9400b4d50 0x7fd9400b7260 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd944008000 tx=0x7fd944031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:25.430 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:12:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.605+0000 7f5ede800640 1 -- 172.21.15.16:0/1056296798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ed8102de0 msgr2=0x7f5ed8107300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.605+0000 7f5ede800640 1 --2- 172.21.15.16:0/1056296798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ed8102de0 0x7f5ed8107300 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5ec8009900 tx=0x7f5ec802a3f0 comp rx=0 tx=0).stop 2024-08-31T06:12:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.606+0000 7f5ede800640 1 -- 172.21.15.16:0/1056296798 shutdown_connections 2024-08-31T06:12:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.606+0000 7f5ede800640 1 --2- 172.21.15.16:0/1056296798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ed8107950 0x7f5ed8107dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.606+0000 7f5ede800640 1 --2- 172.21.15.16:0/1056296798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ed8102de0 0x7f5ed8107300 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.606+0000 7f5ede800640 1 --2- 172.21.15.16:0/1056296798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ed8102490 0x7f5ed81028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.606+0000 7f5ede800640 1 -- 172.21.15.16:0/1056296798 >> 172.21.15.16:0/1056296798 conn(0x7f5ed807a4e0 msgr2=0x7f5ed807a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.606+0000 7f5ede800640 1 -- 172.21.15.16:0/1056296798 shutdown_connections 2024-08-31T06:12:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.606+0000 7f5ede800640 1 -- 172.21.15.16:0/1056296798 wait complete. 2024-08-31T06:12:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.608+0000 7f5ede800640 1 Processor -- start 2024-08-31T06:12:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.610+0000 7f5ede800640 1 -- start start 2024-08-31T06:12:25.614 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.612+0000 7f5ede800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ed8102490 0x7f5ed8074160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:25.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.613+0000 7f5ede800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ed8102de0 0x7f5ed80746e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:25.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.613+0000 7f5ede800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ed8107950 0x7f5ed810ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:25.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.613+0000 7f5ede800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed8109170 con 0x7f5ed8102de0 2024-08-31T06:12:25.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.613+0000 7f5ede800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed8108ff0 con 0x7f5ed8107950 2024-08-31T06:12:25.617 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.613+0000 7f5ede800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed81092f0 con 0x7f5ed8102490 2024-08-31T06:12:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.614+0000 7f5ecfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ed8102de0 0x7f5ed80746e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.615+0000 7f5ecfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ed8102de0 0x7f5ed80746e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52168/0 (socket says 172.21.15.16:52168) 2024-08-31T06:12:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.615+0000 7f5ecfe00640 1 -- 172.21.15.16:0/208595126 learned_addr learned my addr 172.21.15.16:0/208595126 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.615+0000 7f5ed7400640 1 --2- 172.21.15.16:0/208595126 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ed8102490 0x7f5ed8074160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:25.619 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.615+0000 7f5ecfe00640 1 -- 172.21.15.16:0/208595126 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ed8102490 msgr2=0x7f5ed8074160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:25.619 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.615+0000 7f5ecfe00640 1 --2- 172.21.15.16:0/208595126 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ed8102490 0x7f5ed8074160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:25.619 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.615+0000 7f5ecfe00640 1 -- 172.21.15.16:0/208595126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ed8107950 msgr2=0x7f5ed810ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:25.619 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.615+0000 7f5ecfe00640 1 --2- 172.21.15.16:0/208595126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ed8107950 0x7f5ed810ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:25.619 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.615+0000 7f5ecfe00640 1 -- 172.21.15.16:0/208595126 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed810d4e0 con 0x7f5ed8102de0 2024-08-31T06:12:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.618+0000 7f5ecfe00640 1 --2- 172.21.15.16:0/208595126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ed8102de0 0x7f5ed80746e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5ec8031440 tx=0x7f5ec8003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.621+0000 7f5ed4c00640 1 -- 172.21.15.16:0/208595126 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec803f070 con 0x7f5ed8102de0 2024-08-31T06:12:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.622+0000 7f5ed4c00640 1 -- 172.21.15.16:0/208595126 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ec8031b50 con 0x7f5ed8102de0 2024-08-31T06:12:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.623+0000 7f5ed4c00640 1 -- 172.21.15.16:0/208595126 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec8035030 con 0x7f5ed8102de0 2024-08-31T06:12:25.626 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.623+0000 7f5ede800640 1 -- 172.21.15.16:0/208595126 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ed810d770 con 0x7f5ed8102de0 2024-08-31T06:12:25.626 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.623+0000 7f5ede800640 1 -- 172.21.15.16:0/208595126 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ed810dc10 con 0x7f5ed8102de0 2024-08-31T06:12:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.630+0000 7f5ee5a05740 1 Processor -- start 2024-08-31T06:12:25.633 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.631+0000 7f5ee5a05740 1 -- start start 2024-08-31T06:12:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.632+0000 7f5ee5a05740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576bdb1b1b0 0x5576bdebe920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.632+0000 7f5ee5a05740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576bdaf22f0 0x5576bdebee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.632+0000 7f5ee5a05740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576bdafa4d0 0x5576bdebf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.632+0000 7f5ee5a05740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5576bde33f70 con 0x5576bdaf22f0 2024-08-31T06:12:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.632+0000 7f5ee5a05740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5576bde23500 con 0x5576bdb1b1b0 2024-08-31T06:12:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.632+0000 7f5ee5a05740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5576bde274b0 con 0x5576bdafa4d0 2024-08-31T06:12:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.632+0000 7f5ed4c00640 1 -- 172.21.15.16:0/208595126 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f5ec8042070 con 0x7f5ed8102de0 2024-08-31T06:12:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.634+0000 7f5ed7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576bdaf22f0 0x5576bdebee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.634+0000 7f5ed7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576bdaf22f0 0x5576bdebee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52182/0 (socket says 172.21.15.16:52182) 2024-08-31T06:12:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.634+0000 7f5ed7400640 1 -- 172.21.15.16:0/3213576346 learned_addr learned my addr 172.21.15.16:0/3213576346 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.634+0000 7f5ed7e00640 1 --2- 172.21.15.16:0/3213576346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576bdb1b1b0 0x5576bdebe920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.634+0000 7f5ecfe00640 1 --2- 172.21.15.16:0/3213576346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576bdafa4d0 0x5576bdebf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.635+0000 7f5ed7e00640 1 -- 172.21.15.16:0/3213576346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576bdafa4d0 msgr2=0x5576bdebf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.635+0000 7f5ed7e00640 1 --2- 172.21.15.16:0/3213576346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576bdafa4d0 0x5576bdebf3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.635+0000 7f5ed7e00640 1 -- 172.21.15.16:0/3213576346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576bdaf22f0 msgr2=0x5576bdebee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.635+0000 7f5ed7e00640 1 --2- 172.21.15.16:0/3213576346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576bdaf22f0 0x5576bdebee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.635+0000 7f5ed7e00640 1 -- 172.21.15.16:0/3213576346 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576bddddc10 con 0x5576bdb1b1b0 2024-08-31T06:12:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.635+0000 7f5ed4c00640 1 --2- 172.21.15.16:0/208595126 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f5ebc0b4d50 0x7f5ebc0b7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.635+0000 7f5ed4c00640 1 -- 172.21.15.16:0/208595126 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f5ec8105aa0 con 0x7f5ed8102de0 2024-08-31T06:12:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.636+0000 7f5ed7e00640 1 --2- 172.21.15.16:0/3213576346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576bdb1b1b0 0x5576bdebe920 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0005fd0 tx=0x7f5ed000e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbdf89d2322862d8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.636+0000 7f5eccc00640 1 -- 172.21.15.16:0/3213576346 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed0003b70 con 0x5576bdb1b1b0 2024-08-31T06:12:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.636+0000 7f5eccc00640 1 -- 172.21.15.16:0/3213576346 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ed0003d10 con 0x5576bdb1b1b0 2024-08-31T06:12:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.636+0000 7f5eccc00640 1 -- 172.21.15.16:0/3213576346 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed00108a0 con 0x5576bdb1b1b0 2024-08-31T06:12:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.638+0000 7f5ee5a05740 1 -- 172.21.15.16:0/3213576346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576bdb1b1b0 msgr2=0x5576bdebe920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.638+0000 7f5ee5a05740 1 --2- 172.21.15.16:0/3213576346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576bdb1b1b0 0x5576bdebe920 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0005fd0 tx=0x7f5ed000e720 comp rx=0 tx=0).stop 2024-08-31T06:12:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.639+0000 7f5ed7400640 1 --2- 172.21.15.16:0/208595126 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f5ebc0b4d50 0x7f5ebc0b7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:25.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.640+0000 7f5ee5a05740 1 -- 172.21.15.16:0/3213576346 shutdown_connections 2024-08-31T06:12:25.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.640+0000 7f5ee5a05740 1 --2- 172.21.15.16:0/3213576346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576bdafa4d0 0x5576bdebf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:25.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.640+0000 7f5ee5a05740 1 --2- 172.21.15.16:0/3213576346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576bdaf22f0 0x5576bdebee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.640+0000 7f5ee5a05740 1 --2- 172.21.15.16:0/3213576346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576bdb1b1b0 0x5576bdebe920 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.641+0000 7f5ee5a05740 1 -- 172.21.15.16:0/3213576346 >> 172.21.15.16:0/3213576346 conn(0x5576bdb0f300 msgr2=0x5576bdebc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.641+0000 7f5ed7400640 1 --2- 172.21.15.16:0/208595126 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f5ebc0b4d50 0x7f5ebc0b7260 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x5576bdebd5a0 tx=0x7f5ec0005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.641+0000 7f5ee5a05740 1 -- 172.21.15.16:0/3213576346 shutdown_connections 2024-08-31T06:12:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.642+0000 7f5ee5a05740 1 -- 172.21.15.16:0/3213576346 wait complete. 2024-08-31T06:12:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.643+0000 7f5ee5a05740 1 -- start start 2024-08-31T06:12:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.643+0000 7f5ee5a05740 1 Processor -- start 2024-08-31T06:12:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.644+0000 7f5ee5a05740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576bdaf22f0 0x5576bdec9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.644+0000 7f5ee5a05740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576bdafa4d0 0x5576bdeca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.644+0000 7f5ee5a05740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576bdb1b1b0 0x5576bded6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.644+0000 7f5ee5a05740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5576bdd5cab0 con 0x5576bdaf22f0 2024-08-31T06:12:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.644+0000 7f5ee5a05740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5576bddca320 con 0x5576bdafa4d0 2024-08-31T06:12:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.644+0000 7f5ee5a05740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5576bdecd0a0 con 0x5576bdb1b1b0 2024-08-31T06:12:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.644+0000 7f5ee5a05740 10 client.0 _ll_register_callbacks cb 0x5576bddd06a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:12:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.645+0000 7f5ecfe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576bdafa4d0 0x5576bdeca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.645+0000 7f5ecfe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576bdafa4d0 0x5576bdeca4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:57204/0 (socket says 172.21.15.16:57204) 2024-08-31T06:12:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.645+0000 7f5ecfe00640 1 -- 172.21.15.16:0/3604135584 learned_addr learned my addr 172.21.15.16:0/3604135584 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.645+0000 7f5ed7e00640 1 --2- 172.21.15.16:0/3604135584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576bdb1b1b0 0x5576bded6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.645+0000 7f5ecfe00640 1 -- 172.21.15.16:0/3604135584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576bdb1b1b0 msgr2=0x5576bded6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:25.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.645+0000 7f5ecfe00640 1 --2- 172.21.15.16:0/3604135584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576bdb1b1b0 0x5576bded6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:25.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.645+0000 7f5ed7400640 1 --2- 172.21.15.16:0/3604135584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576bdaf22f0 0x5576bdec9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:25.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.645+0000 7f5ecfe00640 1 -- 172.21.15.16:0/3604135584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576bdaf22f0 msgr2=0x5576bdec9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:25.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.646+0000 7f5ecfe00640 1 --2- 172.21.15.16:0/3604135584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576bdaf22f0 0x5576bdec9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:25.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.646+0000 7f5ecfe00640 1 -- 172.21.15.16:0/3604135584 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576bdb86790 con 0x5576bdafa4d0 2024-08-31T06:12:25.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.646+0000 7f5ed7400640 1 --2- 172.21.15.16:0/3604135584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576bdaf22f0 0x5576bdec9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:12:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.646+0000 7f5ecfe00640 1 --2- 172.21.15.16:0/3604135584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576bdafa4d0 0x5576bdeca4a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5ec80c8210 tx=0x7f5ec803b640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b22a8da946e4463 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.646+0000 7f5ec5600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:12:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.647+0000 7f5ec5600640 1 -- 172.21.15.16:0/3604135584 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec80cbca0 con 0x5576bdafa4d0 2024-08-31T06:12:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.647+0000 7f5ec5600640 1 -- 172.21.15.16:0/3604135584 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f5ec80c8420 con 0x5576bdafa4d0 2024-08-31T06:12:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.647+0000 7f5ee5a05740 1 -- 172.21.15.16:0/3604135584 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ed0003d10 con 0x5576bdafa4d0 2024-08-31T06:12:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.647+0000 7f5ec5600640 1 -- 172.21.15.16:0/3604135584 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f5ec80cc890 con 0x5576bdafa4d0 2024-08-31T06:12:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.648+0000 7f5ee5a05740 10 client.26726 fetch_fsmap learned FSMap version 36 2024-08-31T06:12:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.648+0000 7f5ee5a05740 1 -- 172.21.15.16:0/3604135584 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5576bdbff620 con 0x5576bdafa4d0 2024-08-31T06:12:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.648+0000 7f5ec5600640 1 -- 172.21.15.16:0/3604135584 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ed0003d10 con 0x5576bdafa4d0 2024-08-31T06:12:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.648+0000 7f5ec5600640 1 -- 172.21.15.16:0/3604135584 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 36) ==== 8350+0+0 (secure 0 0 0) 0x7f5ec80d5020 con 0x5576bdafa4d0 2024-08-31T06:12:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.649+0000 7f5ee5a05740 10 client.26726 fetch_fsmap finished waiting for FSMap version 36 2024-08-31T06:12:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.649+0000 7f5ee5a05740 10 client.26726 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:12:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.649+0000 7f5ee5a05740 20 client.26726 populate_metadata read hostname 'smithi016' 2024-08-31T06:12:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.649+0000 7f5ee5a05740 1 --2- 172.21.15.16:0/3604135584 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5576bdad9d80 0x5576bdedc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.649+0000 7f5ee5a05740 4 client.26726 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:12:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.649+0000 7f5ee5a05740 1 -- 172.21.15.16:0/3604135584 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576bde35ec0 con 0x5576bdad9d80 2024-08-31T06:12:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.649+0000 7f5ed7400640 1 --2- 172.21.15.16:0/3604135584 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5576bdad9d80 0x5576bdedc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:25.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.650+0000 7f5ed7400640 1 --2- 172.21.15.16:0/3604135584 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5576bdad9d80 0x5576bdedc0c0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe869352704da490 server_cookie=30512df281fb1475 in_seq=0 out_seq=0 2024-08-31T06:12:25.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.650+0000 7f5ec5600640 10 client.26726 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:12:25.654 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.651+0000 7f5ec5600640 1 -- 172.21.15.16:0/3604135584 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5576bde35ec0 con 0x5576bdad9d80 2024-08-31T06:12:25.654 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.652+0000 7f5ec5600640 10 client.26726 handle_command_reply: tid=0 2024-08-31T06:12:25.654 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.652+0000 7f5ee5a05740 1 client.26726 shutdown 2024-08-31T06:12:25.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.652+0000 7f5ee5a05740 1 -- 172.21.15.16:0/3604135584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576bdafa4d0 msgr2=0x5576bdeca4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:25.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.652+0000 7f5ee5a05740 1 --2- 172.21.15.16:0/3604135584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576bdafa4d0 0x5576bdeca4a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5ec80c8210 tx=0x7f5ec803b640 comp rx=0 tx=0).stop 2024-08-31T06:12:25.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.653+0000 7f5ee5a05740 1 -- 172.21.15.16:0/3604135584 shutdown_connections 2024-08-31T06:12:25.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.653+0000 7f5ee5a05740 1 --2- 172.21.15.16:0/3604135584 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5576bdad9d80 0x5576bdedc0c0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:25.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.653+0000 7f5ee5a05740 1 --2- 172.21.15.16:0/3604135584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576bdb1b1b0 0x5576bded6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:25.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.653+0000 7f5ec5600640 0 client.26726 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:12:25.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.653+0000 7f5ee5a05740 1 --2- 172.21.15.16:0/3604135584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576bdafa4d0 0x5576bdeca4a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:25.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.653+0000 7f5ee5a05740 1 --2- 172.21.15.16:0/3604135584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576bdaf22f0 0x5576bdec9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:25.656 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.653+0000 7f5ee5a05740 1 -- 172.21.15.16:0/3604135584 >> 172.21.15.16:0/3604135584 conn(0x5576bdb0f300 msgr2=0x5576bdebf950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:25.656 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.654+0000 7f5ee5a05740 1 -- 172.21.15.16:0/3604135584 shutdown_connections 2024-08-31T06:12:25.656 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.654+0000 7f5ee5a05740 1 -- 172.21.15.16:0/3604135584 wait complete. 2024-08-31T06:12:25.656 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.654+0000 7f5ee5a05740 20 client.26726 trim_cache size 0 max 16384 2024-08-31T06:12:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.686+0000 7f5ee5a05740 1 Processor -- start 2024-08-31T06:12:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.687+0000 7f5ee5a05740 1 -- start start 2024-08-31T06:12:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.687+0000 7f5ee5a05740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576bdaf22f0 0x5576bdeca440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.687+0000 7f5ee5a05740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576bdafa4d0 0x5576bdec0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.687+0000 7f5ee5a05740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576bdb1b1b0 0x5576bdecbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.687+0000 7f5ee5a05740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed81c9380 con 0x5576bdb1b1b0 2024-08-31T06:12:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.687+0000 7f5ee5a05740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed81c6c20 con 0x5576bdafa4d0 2024-08-31T06:12:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.687+0000 7f5ee5a05740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed81c4a70 con 0x5576bdaf22f0 2024-08-31T06:12:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.687+0000 7f5ecfe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576bdafa4d0 0x5576bdec0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.687+0000 7f5ecfe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576bdafa4d0 0x5576bdec0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:57206/0 (socket says 172.21.15.16:57206) 2024-08-31T06:12:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.687+0000 7f5ecfe00640 1 -- 172.21.15.16:0/2395819473 learned_addr learned my addr 172.21.15.16:0/2395819473 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.688+0000 7f5ed7400640 1 --2- 172.21.15.16:0/2395819473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576bdaf22f0 0x5576bdeca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.688+0000 7f5ecfe00640 1 -- 172.21.15.16:0/2395819473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576bdaf22f0 msgr2=0x5576bdeca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.688+0000 7f5ecfe00640 1 --2- 172.21.15.16:0/2395819473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576bdaf22f0 0x5576bdeca440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.688+0000 7f5ecfe00640 1 -- 172.21.15.16:0/2395819473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576bdb1b1b0 msgr2=0x5576bdecbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.688+0000 7f5ecfe00640 1 --2- 172.21.15.16:0/2395819473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576bdb1b1b0 0x5576bdecbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.688+0000 7f5ecfe00640 1 -- 172.21.15.16:0/2395819473 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576bdad9fb0 con 0x5576bdafa4d0 2024-08-31T06:12:25.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.688+0000 7f5ed7400640 1 --2- 172.21.15.16:0/2395819473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576bdaf22f0 0x5576bdeca440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:12:25.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.688+0000 7f5ecfe00640 1 --2- 172.21.15.16:0/2395819473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576bdafa4d0 0x5576bdec0310 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5ec802a9e0 tx=0x7f5ec802a8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c0294fc8afcf6ee server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:25.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.690+0000 7f5ec7e00640 1 -- 172.21.15.16:0/2395819473 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec8103680 con 0x5576bdafa4d0 2024-08-31T06:12:25.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.690+0000 7f5ec7e00640 1 -- 172.21.15.16:0/2395819473 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ec8105e50 con 0x5576bdafa4d0 2024-08-31T06:12:25.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.690+0000 7f5ec7e00640 1 -- 172.21.15.16:0/2395819473 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec80d5020 con 0x5576bdafa4d0 2024-08-31T06:12:25.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.691+0000 7f5ee5a05740 1 -- 172.21.15.16:0/2395819473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576bdafa4d0 msgr2=0x5576bdec0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:25.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.691+0000 7f5ee5a05740 1 --2- 172.21.15.16:0/2395819473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576bdafa4d0 0x5576bdec0310 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5ec802a9e0 tx=0x7f5ec802a8f0 comp rx=0 tx=0).stop 2024-08-31T06:12:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.691+0000 7f5ee5a05740 1 -- 172.21.15.16:0/2395819473 shutdown_connections 2024-08-31T06:12:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.691+0000 7f5ee5a05740 1 --2- 172.21.15.16:0/2395819473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576bdb1b1b0 0x5576bdecbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.691+0000 7f5ee5a05740 1 --2- 172.21.15.16:0/2395819473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576bdafa4d0 0x5576bdec0310 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.691+0000 7f5ee5a05740 1 --2- 172.21.15.16:0/2395819473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576bdaf22f0 0x5576bdeca440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.691+0000 7f5ee5a05740 1 -- 172.21.15.16:0/2395819473 >> 172.21.15.16:0/2395819473 conn(0x5576bdb0f300 msgr2=0x5576bdecb480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.692+0000 7f5ee5a05740 1 -- 172.21.15.16:0/2395819473 shutdown_connections 2024-08-31T06:12:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.692+0000 7f5ee5a05740 1 -- 172.21.15.16:0/2395819473 wait complete. 2024-08-31T06:12:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.692+0000 7f5ee5a05740 1 -- start start 2024-08-31T06:12:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.693+0000 7f5ee5a05740 1 Processor -- start 2024-08-31T06:12:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.693+0000 7f5ee5a05740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576bdaf22f0 0x5576bded2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.693+0000 7f5ee5a05740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576bdafa4d0 0x5576bdecfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.693+0000 7f5ee5a05740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576bdb1b1b0 0x5576bded0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.693+0000 7f5ee5a05740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed81c50e0 con 0x5576bdafa4d0 2024-08-31T06:12:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.693+0000 7f5ee5a05740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed81c6550 con 0x5576bdb1b1b0 2024-08-31T06:12:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.693+0000 7f5ee5a05740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed81c57b0 con 0x5576bdaf22f0 2024-08-31T06:12:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.693+0000 7f5ee5a05740 10 client.0 _ll_register_callbacks cb 0x7f5ec80c8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:12:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.694+0000 7f5ed7400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576bdaf22f0 0x5576bded2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.694+0000 7f5ed7400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576bdaf22f0 0x5576bded2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:36556/0 (socket says 172.21.15.16:36556) 2024-08-31T06:12:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.694+0000 7f5ed7400640 1 -- 172.21.15.16:0/561022566 learned_addr learned my addr 172.21.15.16:0/561022566 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.694+0000 7f5ed7e00640 1 --2- 172.21.15.16:0/561022566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576bdb1b1b0 0x5576bded0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.695+0000 7f5ed7400640 1 -- 172.21.15.16:0/561022566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576bdb1b1b0 msgr2=0x5576bded0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.695+0000 7f5ed7400640 1 --2- 172.21.15.16:0/561022566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576bdb1b1b0 0x5576bded0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.695+0000 7f5ed7400640 1 -- 172.21.15.16:0/561022566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576bdafa4d0 msgr2=0x5576bdecfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.695+0000 7f5ed7400640 1 --2- 172.21.15.16:0/561022566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576bdafa4d0 0x5576bdecfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.695+0000 7f5ed7400640 1 -- 172.21.15.16:0/561022566 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576bded0920 con 0x5576bdaf22f0 2024-08-31T06:12:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.695+0000 7f5ed7400640 1 --2- 172.21.15.16:0/561022566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576bdaf22f0 0x5576bded2930 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5ec001f840 tx=0x7f5ec0015040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e671e51ce54839b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.695+0000 7f5ec5600640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:12:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.696+0000 7f5ec5600640 1 -- 172.21.15.16:0/561022566 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec001dc80 con 0x5576bdaf22f0 2024-08-31T06:12:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.696+0000 7f5ec5600640 1 -- 172.21.15.16:0/561022566 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f5ec001de20 con 0x5576bdaf22f0 2024-08-31T06:12:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.696+0000 7f5ee5a05740 1 -- 172.21.15.16:0/561022566 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ec8105e50 con 0x5576bdaf22f0 2024-08-31T06:12:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.696+0000 7f5ec5600640 1 -- 172.21.15.16:0/561022566 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f5ec0005020 con 0x5576bdaf22f0 2024-08-31T06:12:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.696+0000 7f5ee5a05740 10 client.26728 fetch_fsmap learned FSMap version 36 2024-08-31T06:12:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.696+0000 7f5ee5a05740 1 -- 172.21.15.16:0/561022566 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5576bdf46820 con 0x5576bdaf22f0 2024-08-31T06:12:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.697+0000 7f5ec5600640 1 -- 172.21.15.16:0/561022566 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ec8105e50 con 0x5576bdaf22f0 2024-08-31T06:12:25.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.697+0000 7f5ec5600640 1 -- 172.21.15.16:0/561022566 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 36) ==== 8350+0+0 (secure 0 0 0) 0x7f5ec000f040 con 0x5576bdaf22f0 2024-08-31T06:12:25.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.698+0000 7f5ee5a05740 10 client.26728 fetch_fsmap finished waiting for FSMap version 36 2024-08-31T06:12:25.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.698+0000 7f5ee5a05740 10 client.26728 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:12:25.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.698+0000 7f5ee5a05740 20 client.26728 populate_metadata read hostname 'smithi016' 2024-08-31T06:12:25.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.698+0000 7f5ee5a05740 1 --2- 172.21.15.16:0/561022566 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5576bdaaf1c0 0x5576bdf48fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:25.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.698+0000 7f5ee5a05740 4 client.26728 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:12:25.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.698+0000 7f5ee5a05740 1 -- 172.21.15.16:0/561022566 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5576bde31960 con 0x5576bdaaf1c0 2024-08-31T06:12:25.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.698+0000 7f5ecfe00640 1 --2- 172.21.15.16:0/561022566 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5576bdaaf1c0 0x5576bdf48fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:25.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.699+0000 7f5ecfe00640 1 --2- 172.21.15.16:0/561022566 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5576bdaaf1c0 0x5576bdf48fc0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcc233b7709daef2 server_cookie=d42452cbe8e139d in_seq=0 out_seq=0 2024-08-31T06:12:25.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.699+0000 7f5ec5600640 10 client.26728 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:12:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.728+0000 7f5ec5600640 1 -- 172.21.15.16:0/561022566 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5576bde31960 con 0x5576bdaaf1c0 2024-08-31T06:12:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.729+0000 7f5ec5600640 10 client.26728 handle_command_reply: tid=0 2024-08-31T06:12:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.729+0000 7f5ee5a05740 1 client.26728 shutdown 2024-08-31T06:12:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.729+0000 7f5ee5a05740 1 -- 172.21.15.16:0/561022566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576bdaf22f0 msgr2=0x5576bded2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.729+0000 7f5ee5a05740 1 --2- 172.21.15.16:0/561022566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576bdaf22f0 0x5576bded2930 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5ec001f840 tx=0x7f5ec0015040 comp rx=0 tx=0).stop 2024-08-31T06:12:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.729+0000 7f5ee5a05740 1 -- 172.21.15.16:0/561022566 shutdown_connections 2024-08-31T06:12:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.729+0000 7f5ee5a05740 1 --2- 172.21.15.16:0/561022566 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5576bdaaf1c0 0x5576bdf48fc0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.729+0000 7f5ee5a05740 1 --2- 172.21.15.16:0/561022566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576bdb1b1b0 0x5576bded0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.729+0000 7f5ee5a05740 1 --2- 172.21.15.16:0/561022566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576bdafa4d0 0x5576bdecfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.729+0000 7f5ee5a05740 1 --2- 172.21.15.16:0/561022566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576bdaf22f0 0x5576bded2930 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.729+0000 7f5ec5600640 0 client.26728 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:12:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.729+0000 7f5ee5a05740 1 -- 172.21.15.16:0/561022566 >> 172.21.15.16:0/561022566 conn(0x5576bdb0f300 msgr2=0x5576bdecbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.730+0000 7f5ee5a05740 1 -- 172.21.15.16:0/561022566 shutdown_connections 2024-08-31T06:12:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.730+0000 7f5ee5a05740 1 -- 172.21.15.16:0/561022566 wait complete. 2024-08-31T06:12:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.731+0000 7f5ee5a05740 20 client.26728 trim_cache size 0 max 16384 2024-08-31T06:12:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:12:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:12:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:12:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:12:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.732+0000 7f5ede800640 1 -- 172.21.15.16:0/208595126 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f5ebc0b4d50 msgr2=0x7f5ebc0b7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.732+0000 7f5ede800640 1 --2- 172.21.15.16:0/208595126 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f5ebc0b4d50 0x7f5ebc0b7260 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x5576bdebd5a0 tx=0x7f5ec0005f90 comp rx=0 tx=0).stop 2024-08-31T06:12:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.732+0000 7f5ede800640 1 -- 172.21.15.16:0/208595126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ed8102de0 msgr2=0x7f5ed80746e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.732+0000 7f5ede800640 1 --2- 172.21.15.16:0/208595126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ed8102de0 0x7f5ed80746e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5ec8031440 tx=0x7f5ec8003b20 comp rx=0 tx=0).stop 2024-08-31T06:12:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.733+0000 7f5ede800640 1 -- 172.21.15.16:0/208595126 shutdown_connections 2024-08-31T06:12:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.733+0000 7f5ede800640 1 --2- 172.21.15.16:0/208595126 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f5ebc0b4d50 0x7f5ebc0b7260 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.733+0000 7f5ede800640 1 --2- 172.21.15.16:0/208595126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ed8107950 0x7f5ed810ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.733+0000 7f5ede800640 1 --2- 172.21.15.16:0/208595126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ed8102de0 0x7f5ed80746e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.733+0000 7f5ede800640 1 --2- 172.21.15.16:0/208595126 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ed8102490 0x7f5ed8074160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.733+0000 7f5ede800640 1 -- 172.21.15.16:0/208595126 >> 172.21.15.16:0/208595126 conn(0x7f5ed807a4e0 msgr2=0x7f5ed81052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.733+0000 7f5ede800640 1 -- 172.21.15.16:0/208595126 shutdown_connections 2024-08-31T06:12:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:25.733+0000 7f5ede800640 1 -- 172.21.15.16:0/208595126 wait complete. 2024-08-31T06:12:25.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.845+0000 7fd961600640 1 -- 172.21.15.16:0/967683936 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd95c10aec0 con 0x7fd95c0754f0 2024-08-31T06:12:25.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.846+0000 7fd959600640 1 -- 172.21.15.16:0/967683936 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11918 (secure 0 0 0) 0x7fd94c0ac360 con 0x7fd95c0754f0 2024-08-31T06:12:25.849 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:12:25.849 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":36,"btime":"2024-08-31T06:11:48:781243+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:11:48.335944+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24712,24724,24737]},"id":1}]} 2024-08-31T06:12:25.849 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 36 2024-08-31T06:12:25.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.850+0000 7fd961600640 1 -- 172.21.15.16:0/967683936 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd9400b4d50 msgr2=0x7fd9400b7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:25.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.850+0000 7fd961600640 1 --2- 172.21.15.16:0/967683936 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd9400b4d50 0x7fd9400b7260 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd944008000 tx=0x7fd944031400 comp rx=0 tx=0).stop 2024-08-31T06:12:25.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.850+0000 7fd961600640 1 -- 172.21.15.16:0/967683936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd95c0754f0 msgr2=0x7fd95c074da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:25.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.850+0000 7fd961600640 1 --2- 172.21.15.16:0/967683936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd95c0754f0 0x7fd95c074da0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd94c00c890 tx=0x7fd94c00cd50 comp rx=0 tx=0).stop 2024-08-31T06:12:25.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.850+0000 7fd961600640 1 -- 172.21.15.16:0/967683936 shutdown_connections 2024-08-31T06:12:25.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.850+0000 7fd961600640 1 --2- 172.21.15.16:0/967683936 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd9400b4d50 0x7fd9400b7260 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:25.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.850+0000 7fd961600640 1 --2- 172.21.15.16:0/967683936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd95c06e5e0 0x7fd95c073950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:25.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.850+0000 7fd961600640 1 --2- 172.21.15.16:0/967683936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd95c10fd10 0x7fd95c06dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:25.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.850+0000 7fd961600640 1 --2- 172.21.15.16:0/967683936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd95c0754f0 0x7fd95c074da0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:25.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.851+0000 7fd961600640 1 -- 172.21.15.16:0/967683936 >> 172.21.15.16:0/967683936 conn(0x7fd95c0ff1a0 msgr2=0x7fd95c0ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:25.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.851+0000 7fd961600640 1 -- 172.21.15.16:0/967683936 shutdown_connections 2024-08-31T06:12:25.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:25.851+0000 7fd961600640 1 -- 172.21.15.16:0/967683936 wait complete. 2024-08-31T06:12:26.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:12:25 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/967683936' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:12:26.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:12:25 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/967683936' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:12:26.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:12:25 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/967683936' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:12:26.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 300.0 s, 1744.2 tps, lat 57.432 ms stddev 70.639 2024-08-31T06:12:27.202 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:12:26 smithi016 ceph-mon[17194]: pgmap v520: 97 pgs: 97 active+clean; 2.8 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 28 MiB/s wr, 843 op/s 2024-08-31T06:12:27.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:12:26 smithi184 ceph-mon[22331]: pgmap v520: 97 pgs: 97 active+clean; 2.8 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 28 MiB/s wr, 843 op/s 2024-08-31T06:12:27.276 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24737 quiesce db / --timeout 90 --expiration 152.7 --await-for 45.0' 2024-08-31T06:12:27.277 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24737 quiesce db / --timeout 90 --expiration 152.7 --await-for 45.0 2024-08-31T06:12:27.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:12:26 smithi155 ceph-mon[21742]: pgmap v520: 97 pgs: 97 active+clean; 2.8 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 28 MiB/s wr, 843 op/s 2024-08-31T06:12:27.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.866+0000 7f0786000640 1 -- 172.21.15.16:0/1290630293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0780109270 msgr2=0x7f0780109680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:27.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.866+0000 7f0786000640 1 --2- 172.21.15.16:0/1290630293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0780109270 0x7f0780109680 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f076c009900 tx=0x7f076c02a3f0 comp rx=0 tx=0).stop 2024-08-31T06:12:27.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.867+0000 7f0786000640 1 -- 172.21.15.16:0/1290630293 shutdown_connections 2024-08-31T06:12:27.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.867+0000 7f0786000640 1 --2- 172.21.15.16:0/1290630293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0780109bc0 0x7f07801103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:27.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.867+0000 7f0786000640 1 --2- 172.21.15.16:0/1290630293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0780109270 0x7f0780109680 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:27.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.867+0000 7f0786000640 1 --2- 172.21.15.16:0/1290630293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0780103930 0x7f0780103d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:27.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.867+0000 7f0786000640 1 -- 172.21.15.16:0/1290630293 >> 172.21.15.16:0/1290630293 conn(0x7f07800ff380 msgr2=0x7f07801017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:27.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.868+0000 7f0786000640 1 -- 172.21.15.16:0/1290630293 shutdown_connections 2024-08-31T06:12:27.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.868+0000 7f0786000640 1 -- 172.21.15.16:0/1290630293 wait complete. 2024-08-31T06:12:27.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.870+0000 7f0786000640 1 Processor -- start 2024-08-31T06:12:27.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.871+0000 7f0786000640 1 -- start start 2024-08-31T06:12:27.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.873+0000 7f0786000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0780103930 0x7f0780074be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:27.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.873+0000 7f0786000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0780109270 0x7f078006deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:27.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.873+0000 7f0786000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0780109bc0 0x7f078006e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:27.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.874+0000 7f0786000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f07801138d0 con 0x7f0780103930 2024-08-31T06:12:27.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.874+0000 7f0786000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0780113750 con 0x7f0780109bc0 2024-08-31T06:12:27.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.874+0000 7f0786000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0780113a50 con 0x7f0780109270 2024-08-31T06:12:27.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.874+0000 7f077fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0780109bc0 0x7f078006e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:27.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.874+0000 7f077fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0780109bc0 0x7f078006e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:45684/0 (socket says 172.21.15.16:45684) 2024-08-31T06:12:27.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.874+0000 7f077fe00640 1 -- 172.21.15.16:0/1237996926 learned_addr learned my addr 172.21.15.16:0/1237996926 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:27.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.874+0000 7f077ea00640 1 --2- 172.21.15.16:0/1237996926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0780109270 0x7f078006deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:27.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.874+0000 7f077fe00640 1 -- 172.21.15.16:0/1237996926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0780109270 msgr2=0x7f078006deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:27.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.874+0000 7f077fe00640 1 --2- 172.21.15.16:0/1237996926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0780109270 0x7f078006deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:27.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.874+0000 7f077fe00640 1 -- 172.21.15.16:0/1237996926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0780103930 msgr2=0x7f0780074be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:27.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.874+0000 7f077fe00640 1 --2- 172.21.15.16:0/1237996926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0780103930 0x7f0780074be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:27.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.875+0000 7f077fe00640 1 -- 172.21.15.16:0/1237996926 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f078006ec30 con 0x7f0780109bc0 2024-08-31T06:12:27.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.875+0000 7f077fe00640 1 --2- 172.21.15.16:0/1237996926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0780109bc0 0x7f078006e410 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f077800efd0 tx=0x7f077800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:27.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.875+0000 7f0777e00640 1 -- 172.21.15.16:0/1237996926 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0778003b70 con 0x7f0780109bc0 2024-08-31T06:12:27.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.875+0000 7f0777e00640 1 -- 172.21.15.16:0/1237996926 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0778003d10 con 0x7f0780109bc0 2024-08-31T06:12:27.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.875+0000 7f0777e00640 1 -- 172.21.15.16:0/1237996926 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07780108d0 con 0x7f0780109bc0 2024-08-31T06:12:27.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.875+0000 7f0786000640 1 -- 172.21.15.16:0/1237996926 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07800738b0 con 0x7f0780109bc0 2024-08-31T06:12:27.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.875+0000 7f0786000640 1 -- 172.21.15.16:0/1237996926 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0780073d70 con 0x7f0780109bc0 2024-08-31T06:12:27.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.884+0000 7f078cbf0740 1 Processor -- start 2024-08-31T06:12:27.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.884+0000 7f0777e00640 1 -- 172.21.15.16:0/1237996926 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f0778004560 con 0x7f0780109bc0 2024-08-31T06:12:27.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.886+0000 7f078cbf0740 1 -- start start 2024-08-31T06:12:27.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.887+0000 7f078cbf0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a29d4a1070 0x55a29d846990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:27.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.887+0000 7f078cbf0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a29d4781b0 0x55a29d846ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:27.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.887+0000 7f078cbf0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a29d480390 0x55a29d847410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:27.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.887+0000 7f078cbf0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a29d7bd7b0 con 0x55a29d4781b0 2024-08-31T06:12:27.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.887+0000 7f078cbf0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a29d7acd40 con 0x55a29d480390 2024-08-31T06:12:27.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.887+0000 7f078cbf0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a29d7b0cf0 con 0x55a29d4a1070 2024-08-31T06:12:27.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.887+0000 7f077ea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a29d4a1070 0x55a29d846990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:27.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.887+0000 7f0777e00640 1 --2- 172.21.15.16:0/1237996926 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f07640b4d20 0x7f07640b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:27.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.887+0000 7f077ea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a29d4a1070 0x55a29d846990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:37202/0 (socket says 172.21.15.16:37202) 2024-08-31T06:12:27.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.887+0000 7f077ea00640 1 -- 172.21.15.16:0/1237535920 learned_addr learned my addr 172.21.15.16:0/1237535920 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:27.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.888+0000 7f077ea00640 1 --2- 172.21.15.16:0/1237535920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a29d480390 0x55a29d847410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:27.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.888+0000 7f077f400640 1 --2- 172.21.15.16:0/1237535920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a29d4781b0 0x55a29d846ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:27.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.888+0000 7f077ea00640 1 -- 172.21.15.16:0/1237535920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a29d480390 msgr2=0x55a29d847410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:27.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.888+0000 7f077ea00640 1 --2- 172.21.15.16:0/1237535920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a29d480390 0x55a29d847410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:27.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.888+0000 7f077ea00640 1 -- 172.21.15.16:0/1237535920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a29d4781b0 msgr2=0x55a29d846ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:27.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.888+0000 7f077ea00640 1 --2- 172.21.15.16:0/1237535920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a29d4781b0 0x55a29d846ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:27.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.888+0000 7f077ea00640 1 -- 172.21.15.16:0/1237535920 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a29d50c650 con 0x55a29d4a1070 2024-08-31T06:12:27.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.889+0000 7f077f400640 1 --2- 172.21.15.16:0/1237535920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a29d4781b0 0x55a29d846ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:12:27.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.889+0000 7f077ea00640 1 --2- 172.21.15.16:0/1237535920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a29d4a1070 0x55a29d846990 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f076c0057c0 tx=0x7f076c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3d936242380d82 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:27.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.891+0000 7f076be00640 1 -- 172.21.15.16:0/1237535920 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076c035040 con 0x55a29d4a1070 2024-08-31T06:12:27.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.891+0000 7f076be00640 1 -- 172.21.15.16:0/1237535920 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f076c0054d0 con 0x55a29d4a1070 2024-08-31T06:12:27.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.891+0000 7f076be00640 1 -- 172.21.15.16:0/1237535920 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076c03b5c0 con 0x55a29d4a1070 2024-08-31T06:12:27.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.892+0000 7f078cbf0740 1 -- 172.21.15.16:0/1237535920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a29d4a1070 msgr2=0x55a29d846990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:27.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.892+0000 7f078cbf0740 1 --2- 172.21.15.16:0/1237535920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a29d4a1070 0x55a29d846990 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f076c0057c0 tx=0x7f076c000bc0 comp rx=0 tx=0).stop 2024-08-31T06:12:27.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.892+0000 7f0777e00640 1 -- 172.21.15.16:0/1237996926 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f07780e5a30 con 0x7f0780109bc0 2024-08-31T06:12:27.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.893+0000 7f078cbf0740 1 -- 172.21.15.16:0/1237535920 shutdown_connections 2024-08-31T06:12:27.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.893+0000 7f078cbf0740 1 --2- 172.21.15.16:0/1237535920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a29d480390 0x55a29d847410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:27.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.893+0000 7f078cbf0740 1 --2- 172.21.15.16:0/1237535920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a29d4781b0 0x55a29d846ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:27.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.893+0000 7f078cbf0740 1 --2- 172.21.15.16:0/1237535920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a29d4a1070 0x55a29d846990 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:27.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.893+0000 7f078cbf0740 1 -- 172.21.15.16:0/1237535920 >> 172.21.15.16:0/1237535920 conn(0x55a29d495460 msgr2=0x55a29d844880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:27.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.893+0000 7f077fe00640 1 --2- 172.21.15.16:0/1237996926 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f07640b4d20 0x7f07640b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:27.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.894+0000 7f078cbf0740 1 -- 172.21.15.16:0/1237535920 shutdown_connections 2024-08-31T06:12:27.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.894+0000 7f078cbf0740 1 -- 172.21.15.16:0/1237535920 wait complete. 2024-08-31T06:12:27.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.895+0000 7f077fe00640 1 --2- 172.21.15.16:0/1237996926 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f07640b4d20 0x7f07640b7230 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f07780ac110 tx=0x7f0778004850 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:27.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.895+0000 7f078cbf0740 1 -- start start 2024-08-31T06:12:27.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.896+0000 7f078cbf0740 1 Processor -- start 2024-08-31T06:12:27.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.897+0000 7f078cbf0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a29d4781b0 0x55a29d852630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:27.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.897+0000 7f078cbf0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a29d480390 0x55a29d852bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:27.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.897+0000 7f078cbf0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a29d4a1070 0x55a29d85e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:27.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.897+0000 7f078cbf0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a29d60f700 con 0x55a29d480390 2024-08-31T06:12:27.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.897+0000 7f078cbf0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a29d733cf0 con 0x55a29d4781b0 2024-08-31T06:12:27.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.897+0000 7f078cbf0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a29d855110 con 0x55a29d4a1070 2024-08-31T06:12:27.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.897+0000 7f078cbf0740 10 client.0 _ll_register_callbacks cb 0x55a29d759470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:12:27.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.897+0000 7f077f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a29d480390 0x55a29d852bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:27.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.897+0000 7f077ea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a29d4a1070 0x55a29d85e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:27.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.897+0000 7f077f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a29d480390 0x55a29d852bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51412/0 (socket says 172.21.15.16:51412) 2024-08-31T06:12:27.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.897+0000 7f077f400640 1 -- 172.21.15.16:0/4254277545 learned_addr learned my addr 172.21.15.16:0/4254277545 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:27.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.898+0000 7f077ea00640 1 --2- 172.21.15.16:0/4254277545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a29d4781b0 0x55a29d852630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:27.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.898+0000 7f077ea00640 1 -- 172.21.15.16:0/4254277545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a29d4a1070 msgr2=0x55a29d85e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:27.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.899+0000 7f077ea00640 1 --2- 172.21.15.16:0/4254277545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a29d4a1070 0x55a29d85e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:27.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.899+0000 7f077ea00640 1 -- 172.21.15.16:0/4254277545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a29d480390 msgr2=0x55a29d852bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:27.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.899+0000 7f077ea00640 1 --2- 172.21.15.16:0/4254277545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a29d480390 0x55a29d852bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:27.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.899+0000 7f077ea00640 1 -- 172.21.15.16:0/4254277545 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a29d7748b0 con 0x55a29d4781b0 2024-08-31T06:12:27.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.900+0000 7f077f400640 1 --2- 172.21.15.16:0/4254277545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a29d480390 0x55a29d852bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:12:27.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.901+0000 7f077ea00640 1 --2- 172.21.15.16:0/4254277545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a29d4781b0 0x55a29d852630 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f076c0092a0 tx=0x7f076c005e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44ee2599ca18071 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:27.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.901+0000 7f0768c00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:12:27.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.901+0000 7f0768c00640 1 -- 172.21.15.16:0/4254277545 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076c035070 con 0x55a29d4781b0 2024-08-31T06:12:27.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.902+0000 7f0768c00640 1 -- 172.21.15.16:0/4254277545 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f076c0314a0 con 0x55a29d4781b0 2024-08-31T06:12:27.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.902+0000 7f078cbf0740 1 -- 172.21.15.16:0/4254277545 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f076c0054d0 con 0x55a29d4781b0 2024-08-31T06:12:27.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.902+0000 7f0768c00640 1 -- 172.21.15.16:0/4254277545 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f076c0316a0 con 0x55a29d4781b0 2024-08-31T06:12:27.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.902+0000 7f078cbf0740 10 client.26744 fetch_fsmap learned FSMap version 36 2024-08-31T06:12:27.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.902+0000 7f078cbf0740 1 -- 172.21.15.16:0/4254277545 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a29d861150 con 0x55a29d4781b0 2024-08-31T06:12:27.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.903+0000 7f0768c00640 1 -- 172.21.15.16:0/4254277545 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f076c0054d0 con 0x55a29d4781b0 2024-08-31T06:12:27.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.903+0000 7f0768c00640 1 -- 172.21.15.16:0/4254277545 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 36) ==== 8350+0+0 (secure 0 0 0) 0x7f076c048040 con 0x55a29d4781b0 2024-08-31T06:12:27.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.903+0000 7f078cbf0740 10 client.26744 fetch_fsmap finished waiting for FSMap version 36 2024-08-31T06:12:27.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.903+0000 7f078cbf0740 10 client.26744 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T06:12:27.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.903+0000 7f078cbf0740 20 client.26744 populate_metadata read hostname 'smithi016' 2024-08-31T06:12:27.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.903+0000 7f078cbf0740 1 --2- 172.21.15.16:0/4254277545 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55a29d45f9f0 0x55a29d8644b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:27.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.904+0000 7f078cbf0740 4 client.26744 mds_command: new command op to 24712 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:12:27.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.904+0000 7f078cbf0740 1 -- 172.21.15.16:0/4254277545 --> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a29d7bf700 con 0x55a29d45f9f0 2024-08-31T06:12:27.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.904+0000 7f077f400640 1 --2- 172.21.15.16:0/4254277545 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55a29d45f9f0 0x55a29d8644b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:27.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.905+0000 7f077f400640 1 --2- 172.21.15.16:0/4254277545 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55a29d45f9f0 0x55a29d8644b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5669fe01379f902c server_cookie=8b8d7366b8f329d1 in_seq=0 out_seq=0 2024-08-31T06:12:27.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.905+0000 7f0768c00640 10 client.26744 ms_handle_connect on v2:172.21.15.184:6838/4011051479 2024-08-31T06:12:27.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.906+0000 7f0768c00640 1 -- 172.21.15.16:0/4254277545 <== mds.2 v2:172.21.15.184:6838/4011051479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a29d7bf700 con 0x55a29d45f9f0 2024-08-31T06:12:27.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.906+0000 7f0768c00640 10 client.26744 handle_command_reply: tid=0 2024-08-31T06:12:27.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.906+0000 7f078cbf0740 1 client.26744 shutdown 2024-08-31T06:12:27.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.907+0000 7f078cbf0740 1 -- 172.21.15.16:0/4254277545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a29d4781b0 msgr2=0x55a29d852630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:27.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.907+0000 7f078cbf0740 1 --2- 172.21.15.16:0/4254277545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a29d4781b0 0x55a29d852630 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f076c0092a0 tx=0x7f076c005e30 comp rx=0 tx=0).stop 2024-08-31T06:12:27.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.907+0000 7f078cbf0740 1 -- 172.21.15.16:0/4254277545 shutdown_connections 2024-08-31T06:12:27.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.907+0000 7f078cbf0740 1 --2- 172.21.15.16:0/4254277545 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55a29d45f9f0 0x55a29d8644b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:27.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.907+0000 7f078cbf0740 1 --2- 172.21.15.16:0/4254277545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a29d4a1070 0x55a29d85e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:27.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.907+0000 7f078cbf0740 1 --2- 172.21.15.16:0/4254277545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a29d480390 0x55a29d852bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:27.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.907+0000 7f078cbf0740 1 --2- 172.21.15.16:0/4254277545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a29d4781b0 0x55a29d852630 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:27.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.907+0000 7f078cbf0740 1 -- 172.21.15.16:0/4254277545 >> 172.21.15.16:0/4254277545 conn(0x55a29d495460 msgr2=0x55a29d849b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:27.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.908+0000 7f0768c00640 0 client.26744 ms_handle_reset on v2:172.21.15.184:6838/4011051479 2024-08-31T06:12:27.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.908+0000 7f078cbf0740 1 -- 172.21.15.16:0/4254277545 shutdown_connections 2024-08-31T06:12:27.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.908+0000 7f078cbf0740 1 -- 172.21.15.16:0/4254277545 wait complete. 2024-08-31T06:12:27.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.909+0000 7f078cbf0740 20 client.26744 trim_cache size 0 max 16384 2024-08-31T06:12:27.916 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:12:27.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.943+0000 7f078cbf0740 1 Processor -- start 2024-08-31T06:12:27.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.944+0000 7f078cbf0740 1 -- start start 2024-08-31T06:12:27.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.944+0000 7f078cbf0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a29d4781b0 0x55a29d85e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:27.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.944+0000 7f078cbf0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a29d480390 0x55a29d8441a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:27.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.944+0000 7f078cbf0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a29d4a1070 0x55a29d8446e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:27.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.944+0000 7f078cbf0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f07801b8230 con 0x55a29d4781b0 2024-08-31T06:12:27.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.944+0000 7f078cbf0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f07801b5ad0 con 0x55a29d480390 2024-08-31T06:12:27.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.944+0000 7f078cbf0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f07801b3940 con 0x55a29d4a1070 2024-08-31T06:12:27.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.945+0000 7f077ea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a29d4a1070 0x55a29d8446e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:27.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.945+0000 7f077ea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a29d4a1070 0x55a29d8446e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:37228/0 (socket says 172.21.15.16:37228) 2024-08-31T06:12:27.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.945+0000 7f077ea00640 1 -- 172.21.15.16:0/63471253 learned_addr learned my addr 172.21.15.16:0/63471253 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:27.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.945+0000 7f077ea00640 1 -- 172.21.15.16:0/63471253 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a29d480390 msgr2=0x55a29d8441a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:27.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.945+0000 7f077ea00640 1 --2- 172.21.15.16:0/63471253 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a29d480390 0x55a29d8441a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:27.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.945+0000 7f077ea00640 1 -- 172.21.15.16:0/63471253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a29d4781b0 msgr2=0x55a29d85e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:27.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.945+0000 7f077ea00640 1 --2- 172.21.15.16:0/63471253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a29d4781b0 0x55a29d85e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:27.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.945+0000 7f077ea00640 1 -- 172.21.15.16:0/63471253 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a29d83fed0 con 0x55a29d4a1070 2024-08-31T06:12:27.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.946+0000 7f077ea00640 1 --2- 172.21.15.16:0/63471253 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a29d4a1070 0x55a29d8446e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f076c0096c0 tx=0x7f076c031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=716ef6e01269b959 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:27.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.947+0000 7f076b400640 1 -- 172.21.15.16:0/63471253 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076c035040 con 0x55a29d4a1070 2024-08-31T06:12:27.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.947+0000 7f076b400640 1 -- 172.21.15.16:0/63471253 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f076c03b520 con 0x55a29d4a1070 2024-08-31T06:12:27.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.947+0000 7f076b400640 1 -- 172.21.15.16:0/63471253 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076c0048d0 con 0x55a29d4a1070 2024-08-31T06:12:27.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.949+0000 7f078cbf0740 1 -- 172.21.15.16:0/63471253 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a29d4a1070 msgr2=0x55a29d8446e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:27.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.949+0000 7f078cbf0740 1 --2- 172.21.15.16:0/63471253 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a29d4a1070 0x55a29d8446e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f076c0096c0 tx=0x7f076c031400 comp rx=0 tx=0).stop 2024-08-31T06:12:27.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.949+0000 7f078cbf0740 1 -- 172.21.15.16:0/63471253 shutdown_connections 2024-08-31T06:12:27.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.949+0000 7f078cbf0740 1 --2- 172.21.15.16:0/63471253 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a29d4a1070 0x55a29d8446e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:27.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.949+0000 7f078cbf0740 1 --2- 172.21.15.16:0/63471253 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a29d480390 0x55a29d8441a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:27.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.950+0000 7f078cbf0740 1 --2- 172.21.15.16:0/63471253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a29d4781b0 0x55a29d85e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:27.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.950+0000 7f078cbf0740 1 -- 172.21.15.16:0/63471253 >> 172.21.15.16:0/63471253 conn(0x55a29d495460 msgr2=0x55a29d850cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:27.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.950+0000 7f078cbf0740 1 -- 172.21.15.16:0/63471253 shutdown_connections 2024-08-31T06:12:27.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.950+0000 7f078cbf0740 1 -- 172.21.15.16:0/63471253 wait complete. 2024-08-31T06:12:27.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.952+0000 7f078cbf0740 1 -- start start 2024-08-31T06:12:27.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.952+0000 7f078cbf0740 1 Processor -- start 2024-08-31T06:12:27.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.954+0000 7f078cbf0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a29d4781b0 0x55a29d864590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:27.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.954+0000 7f077ea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a29d4781b0 0x55a29d864590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:27.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.954+0000 7f077ea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a29d4781b0 0x55a29d864590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:37238/0 (socket says 172.21.15.16:37238) 2024-08-31T06:12:27.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.954+0000 7f077ea00640 1 -- 172.21.15.16:0/4121655569 learned_addr learned my addr 172.21.15.16:0/4121655569 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:27.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.957+0000 7f078cbf0740 1 --2- 172.21.15.16:0/4121655569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a29d480390 0x55a29d864ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:27.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.957+0000 7f078cbf0740 1 --2- 172.21.15.16:0/4121655569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a29d4a1070 0x55a29d865010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:27.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.957+0000 7f078cbf0740 1 -- 172.21.15.16:0/4121655569 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f07801b3f90 con 0x55a29d4a1070 2024-08-31T06:12:27.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.957+0000 7f078cbf0740 1 -- 172.21.15.16:0/4121655569 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f07801b5400 con 0x55a29d480390 2024-08-31T06:12:27.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.957+0000 7f078cbf0740 1 -- 172.21.15.16:0/4121655569 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f07801b4660 con 0x55a29d4781b0 2024-08-31T06:12:27.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.957+0000 7f078cbf0740 10 client.0 _ll_register_callbacks cb 0x7f076c005020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:12:27.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.958+0000 7f077f400640 1 --2- 172.21.15.16:0/4121655569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a29d480390 0x55a29d864ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:27.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.958+0000 7f077ea00640 1 -- 172.21.15.16:0/4121655569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a29d480390 msgr2=0x55a29d864ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:27.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.958+0000 7f077ea00640 1 --2- 172.21.15.16:0/4121655569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a29d480390 0x55a29d864ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:27.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.958+0000 7f077ea00640 1 -- 172.21.15.16:0/4121655569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a29d4a1070 msgr2=0x55a29d865010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:27.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.958+0000 7f077ea00640 1 --2- 172.21.15.16:0/4121655569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a29d4a1070 0x55a29d865010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:27.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.958+0000 7f077ea00640 1 -- 172.21.15.16:0/4121655569 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a29d848f50 con 0x55a29d4781b0 2024-08-31T06:12:27.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.959+0000 7f077ea00640 1 --2- 172.21.15.16:0/4121655569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a29d4781b0 0x55a29d864590 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f076c009230 tx=0x7f076c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45131b093e5cb90d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:27.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.959+0000 7f0768c00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:12:27.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.960+0000 7f0768c00640 1 -- 172.21.15.16:0/4121655569 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076c035040 con 0x55a29d4781b0 2024-08-31T06:12:27.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.960+0000 7f0768c00640 1 -- 172.21.15.16:0/4121655569 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f076c0048d0 con 0x55a29d4781b0 2024-08-31T06:12:27.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.960+0000 7f078cbf0740 1 -- 172.21.15.16:0/4121655569 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f076c03b520 con 0x55a29d4781b0 2024-08-31T06:12:27.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.962+0000 7f0768c00640 1 -- 172.21.15.16:0/4121655569 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f076c004ad0 con 0x55a29d4781b0 2024-08-31T06:12:27.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.962+0000 7f078cbf0740 10 client.26746 fetch_fsmap learned FSMap version 36 2024-08-31T06:12:27.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.962+0000 7f078cbf0740 1 -- 172.21.15.16:0/4121655569 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a29d8cebe0 con 0x55a29d4781b0 2024-08-31T06:12:27.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.962+0000 7f0768c00640 1 -- 172.21.15.16:0/4121655569 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f076c03b520 con 0x55a29d4781b0 2024-08-31T06:12:27.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.964+0000 7f0768c00640 1 -- 172.21.15.16:0/4121655569 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 36) ==== 8350+0+0 (secure 0 0 0) 0x7f076c058450 con 0x55a29d4781b0 2024-08-31T06:12:27.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.964+0000 7f078cbf0740 10 client.26746 fetch_fsmap finished waiting for FSMap version 36 2024-08-31T06:12:27.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.964+0000 7f078cbf0740 10 client.26746 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T06:12:27.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.964+0000 7f078cbf0740 20 client.26746 populate_metadata read hostname 'smithi016' 2024-08-31T06:12:27.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.964+0000 7f078cbf0740 1 --2- 172.21.15.16:0/4121655569 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55a29d434da0 0x55a29d8d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:27.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.964+0000 7f078cbf0740 4 client.26746 mds_command: new command op to 24712 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:12:27.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.964+0000 7f078cbf0740 1 -- 172.21.15.16:0/4121655569 --> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a29d7bb1a0 con 0x55a29d434da0 2024-08-31T06:12:27.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.964+0000 7f077f400640 1 --2- 172.21.15.16:0/4121655569 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55a29d434da0 0x55a29d8d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:27.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.965+0000 7f077f400640 1 --2- 172.21.15.16:0/4121655569 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55a29d434da0 0x55a29d8d11b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e11cdda8d7c90540 server_cookie=7eaa64144e793599 in_seq=0 out_seq=0 2024-08-31T06:12:27.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.965+0000 7f0768c00640 10 client.26746 ms_handle_connect on v2:172.21.15.184:6838/4011051479 2024-08-31T06:12:27.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.966+0000 7f0768c00640 1 -- 172.21.15.16:0/4121655569 <== mds.2 v2:172.21.15.184:6838/4011051479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a29d7bb1a0 con 0x55a29d434da0 2024-08-31T06:12:27.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.966+0000 7f0768c00640 10 client.26746 handle_command_reply: tid=0 2024-08-31T06:12:27.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.966+0000 7f078cbf0740 1 client.26746 shutdown 2024-08-31T06:12:27.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.966+0000 7f078cbf0740 1 -- 172.21.15.16:0/4121655569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a29d4781b0 msgr2=0x55a29d864590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:27.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.966+0000 7f078cbf0740 1 --2- 172.21.15.16:0/4121655569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a29d4781b0 0x55a29d864590 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f076c009230 tx=0x7f076c009260 comp rx=0 tx=0).stop 2024-08-31T06:12:27.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.966+0000 7f078cbf0740 1 -- 172.21.15.16:0/4121655569 shutdown_connections 2024-08-31T06:12:27.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.966+0000 7f078cbf0740 1 --2- 172.21.15.16:0/4121655569 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55a29d434da0 0x55a29d8d11b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:27.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.966+0000 7f078cbf0740 1 --2- 172.21.15.16:0/4121655569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a29d4a1070 0x55a29d865010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:27.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.966+0000 7f078cbf0740 1 --2- 172.21.15.16:0/4121655569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a29d480390 0x55a29d864ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:27.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.966+0000 7f078cbf0740 1 --2- 172.21.15.16:0/4121655569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a29d4781b0 0x55a29d864590 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:27.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.966+0000 7f078cbf0740 1 -- 172.21.15.16:0/4121655569 >> 172.21.15.16:0/4121655569 conn(0x55a29d495460 msgr2=0x55a29d84ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:27.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.966+0000 7f0768c00640 0 client.26746 ms_handle_reset on v2:172.21.15.184:6838/4011051479 2024-08-31T06:12:27.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.967+0000 7f078cbf0740 1 -- 172.21.15.16:0/4121655569 shutdown_connections 2024-08-31T06:12:27.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.967+0000 7f078cbf0740 1 -- 172.21.15.16:0/4121655569 wait complete. 2024-08-31T06:12:27.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.967+0000 7f078cbf0740 20 client.26746 trim_cache size 0 max 16384 2024-08-31T06:12:27.973 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:12:27.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.970+0000 7f0786000640 1 -- 172.21.15.16:0/1237996926 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f07640b4d20 msgr2=0x7f07640b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:27.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.970+0000 7f0786000640 1 --2- 172.21.15.16:0/1237996926 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f07640b4d20 0x7f07640b7230 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f07780ac110 tx=0x7f0778004850 comp rx=0 tx=0).stop 2024-08-31T06:12:27.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.970+0000 7f0786000640 1 -- 172.21.15.16:0/1237996926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0780109bc0 msgr2=0x7f078006e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:27.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.970+0000 7f0786000640 1 --2- 172.21.15.16:0/1237996926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0780109bc0 0x7f078006e410 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f077800efd0 tx=0x7f077800c6a0 comp rx=0 tx=0).stop 2024-08-31T06:12:27.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.970+0000 7f0786000640 1 -- 172.21.15.16:0/1237996926 shutdown_connections 2024-08-31T06:12:27.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.970+0000 7f0786000640 1 --2- 172.21.15.16:0/1237996926 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f07640b4d20 0x7f07640b7230 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:27.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.970+0000 7f0786000640 1 --2- 172.21.15.16:0/1237996926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0780109270 0x7f078006deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:27.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.970+0000 7f0786000640 1 --2- 172.21.15.16:0/1237996926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0780109bc0 0x7f078006e410 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:27.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.970+0000 7f0786000640 1 --2- 172.21.15.16:0/1237996926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0780103930 0x7f0780074be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:27.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.970+0000 7f0786000640 1 -- 172.21.15.16:0/1237996926 >> 172.21.15.16:0/1237996926 conn(0x7f07800ff380 msgr2=0x7f07801017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:27.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.971+0000 7f0786000640 1 -- 172.21.15.16:0/1237996926 shutdown_connections 2024-08-31T06:12:27.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:27.971+0000 7f0786000640 1 -- 172.21.15.16:0/1237996926 wait complete. 2024-08-31T06:12:29.204 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:12:28 smithi016 ceph-mon[17194]: pgmap v521: 97 pgs: 97 active+clean; 2.8 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 29 MiB/s wr, 716 op/s 2024-08-31T06:12:29.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:12:28 smithi184 ceph-mon[22331]: pgmap v521: 97 pgs: 97 active+clean; 2.8 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 29 MiB/s wr, 716 op/s 2024-08-31T06:12:29.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:12:28 smithi155 ceph-mon[21742]: pgmap v521: 97 pgs: 97 active+clean; 2.8 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 29 MiB/s wr, 716 op/s 2024-08-31T06:12:29.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.496+0000 7f34f3600640 1 -- 172.21.15.16:0/3143993877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f34ec11ce80 msgr2=0x7f34ec11f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:29.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.496+0000 7f34f3600640 1 --2- 172.21.15.16:0/3143993877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f34ec11ce80 0x7f34ec11f2a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f34e4009be0 tx=0x7f34e402a6c0 comp rx=0 tx=0).stop 2024-08-31T06:12:29.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.497+0000 7f34f3600640 1 -- 172.21.15.16:0/3143993877 shutdown_connections 2024-08-31T06:12:29.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.497+0000 7f34f3600640 1 --2- 172.21.15.16:0/3143993877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f34ec11ce80 0x7f34ec11f2a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:29.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.497+0000 7f34f3600640 1 --2- 172.21.15.16:0/3143993877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34ec10aab0 0x7f34ec11c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:29.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.497+0000 7f34f3600640 1 --2- 172.21.15.16:0/3143993877 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34ec10a160 0x7f34ec10a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:29.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.497+0000 7f34f3600640 1 -- 172.21.15.16:0/3143993877 >> 172.21.15.16:0/3143993877 conn(0x7f34ec070c20 msgr2=0x7f34ec071030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:29.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.498+0000 7f34f3600640 1 -- 172.21.15.16:0/3143993877 shutdown_connections 2024-08-31T06:12:29.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.498+0000 7f34f3600640 1 -- 172.21.15.16:0/3143993877 wait complete. 2024-08-31T06:12:29.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.499+0000 7f34f3600640 1 Processor -- start 2024-08-31T06:12:29.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.499+0000 7f34f3600640 1 -- start start 2024-08-31T06:12:29.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.500+0000 7f34f3600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34ec10a160 0x7f34ec1a9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:29.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.500+0000 7f34f3600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f34ec10aab0 0x7f34ec1a95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:29.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.500+0000 7f34f3600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34ec11ce80 0x7f34ec1ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:29.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.500+0000 7f34f3600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f34ec122650 con 0x7f34ec11ce80 2024-08-31T06:12:29.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.500+0000 7f34f3600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f34ec1224d0 con 0x7f34ec10aab0 2024-08-31T06:12:29.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.500+0000 7f34f3600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f34ec1227d0 con 0x7f34ec10a160 2024-08-31T06:12:29.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.501+0000 7f34ebe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f34ec10aab0 0x7f34ec1a95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:29.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.501+0000 7f34f0c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34ec10a160 0x7f34ec1a9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:29.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.501+0000 7f34ebe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f34ec10aab0 0x7f34ec1a95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:45730/0 (socket says 172.21.15.16:45730) 2024-08-31T06:12:29.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.501+0000 7f34ebe00640 1 -- 172.21.15.16:0/59973756 learned_addr learned my addr 172.21.15.16:0/59973756 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:29.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.501+0000 7f34f1600640 1 --2- 172.21.15.16:0/59973756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34ec11ce80 0x7f34ec1ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:29.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.501+0000 7f34f0c00640 1 -- 172.21.15.16:0/59973756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f34ec10aab0 msgr2=0x7f34ec1a95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:29.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.501+0000 7f34f0c00640 1 --2- 172.21.15.16:0/59973756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f34ec10aab0 0x7f34ec1a95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:29.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.501+0000 7f34f0c00640 1 -- 172.21.15.16:0/59973756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34ec11ce80 msgr2=0x7f34ec1ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:29.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.501+0000 7f34f0c00640 1 --2- 172.21.15.16:0/59973756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34ec11ce80 0x7f34ec1ae840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:29.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.501+0000 7f34f0c00640 1 -- 172.21.15.16:0/59973756 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34ec1aeeb0 con 0x7f34ec10a160 2024-08-31T06:12:29.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.502+0000 7f34f0c00640 1 --2- 172.21.15.16:0/59973756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34ec10a160 0x7f34ec1a9000 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f34e000c940 tx=0x7f34e000ce00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:29.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.503+0000 7f34e9600640 1 -- 172.21.15.16:0/59973756 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e0007bb0 con 0x7f34ec10a160 2024-08-31T06:12:29.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.503+0000 7f34e9600640 1 -- 172.21.15.16:0/59973756 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34e0007d50 con 0x7f34ec10a160 2024-08-31T06:12:29.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.503+0000 7f34f3600640 1 -- 172.21.15.16:0/59973756 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34ec1af1a0 con 0x7f34ec10a160 2024-08-31T06:12:29.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.503+0000 7f34f3600640 1 -- 172.21.15.16:0/59973756 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34ec1b59c0 con 0x7f34ec10a160 2024-08-31T06:12:29.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.504+0000 7f34e9600640 1 -- 172.21.15.16:0/59973756 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e00186f0 con 0x7f34ec10a160 2024-08-31T06:12:29.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.508+0000 7f34fa2a0740 1 Processor -- start 2024-08-31T06:12:29.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.508+0000 7f34fa2a0740 1 -- start start 2024-08-31T06:12:29.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.508+0000 7f34fa2a0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617597f3840 0x561759b9a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:29.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.508+0000 7f34fa2a0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617597ca980 0x561759b9a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:29.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.508+0000 7f34fa2a0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617597d2b60 0x561759b9ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:29.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.508+0000 7f34fa2a0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561759b0f530 con 0x5617597ca980 2024-08-31T06:12:29.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.508+0000 7f34fa2a0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561759afeac0 con 0x5617597d2b60 2024-08-31T06:12:29.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.508+0000 7f34fa2a0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x561759b02a70 con 0x5617597f3840 2024-08-31T06:12:29.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.509+0000 7f34f1600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617597f3840 0x561759b9a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:29.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.509+0000 7f34f1600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617597f3840 0x561759b9a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:37254/0 (socket says 172.21.15.16:37254) 2024-08-31T06:12:29.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.509+0000 7f34f1600640 1 -- 172.21.15.16:0/11908541 learned_addr learned my addr 172.21.15.16:0/11908541 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:29.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.510+0000 7f34f1600640 1 -- 172.21.15.16:0/11908541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617597d2b60 msgr2=0x561759b9ab90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T06:12:29.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.510+0000 7f34f1600640 1 --2- 172.21.15.16:0/11908541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617597d2b60 0x561759b9ab90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:29.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.510+0000 7f34f1600640 1 -- 172.21.15.16:0/11908541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617597ca980 msgr2=0x561759b9a650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:12:29.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.510+0000 7f34f1600640 1 --2- 172.21.15.16:0/11908541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617597ca980 0x561759b9a650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:29.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.510+0000 7f34f1600640 1 -- 172.21.15.16:0/11908541 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561759ab1880 con 0x5617597f3840 2024-08-31T06:12:29.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.510+0000 7f34f1600640 1 --2- 172.21.15.16:0/11908541 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617597f3840 0x561759b9a110 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f34e4034040 tx=0x7f34e40305c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12e372b0596fcd8c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:29.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.511+0000 7f34ddc00640 1 -- 172.21.15.16:0/11908541 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e4041020 con 0x5617597f3840 2024-08-31T06:12:29.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.511+0000 7f34ddc00640 1 -- 172.21.15.16:0/11908541 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34e4037040 con 0x5617597f3840 2024-08-31T06:12:29.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.511+0000 7f34ddc00640 1 -- 172.21.15.16:0/11908541 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e4036070 con 0x5617597f3840 2024-08-31T06:12:29.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.513+0000 7f34fa2a0740 1 -- 172.21.15.16:0/11908541 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617597f3840 msgr2=0x561759b9a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:29.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.513+0000 7f34fa2a0740 1 --2- 172.21.15.16:0/11908541 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617597f3840 0x561759b9a110 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f34e4034040 tx=0x7f34e40305c0 comp rx=0 tx=0).stop 2024-08-31T06:12:29.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.513+0000 7f34fa2a0740 1 -- 172.21.15.16:0/11908541 shutdown_connections 2024-08-31T06:12:29.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.513+0000 7f34fa2a0740 1 --2- 172.21.15.16:0/11908541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617597d2b60 0x561759b9ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:29.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.513+0000 7f34fa2a0740 1 --2- 172.21.15.16:0/11908541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617597ca980 0x561759b9a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:29.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.513+0000 7f34fa2a0740 1 --2- 172.21.15.16:0/11908541 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617597f3840 0x561759b9a110 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:29.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.513+0000 7f34fa2a0740 1 -- 172.21.15.16:0/11908541 >> 172.21.15.16:0/11908541 conn(0x5617597e7990 msgr2=0x561759b98000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:29.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.514+0000 7f34e9600640 1 -- 172.21.15.16:0/59973756 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f34e000f450 con 0x7f34ec10a160 2024-08-31T06:12:29.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.515+0000 7f34fa2a0740 1 -- 172.21.15.16:0/11908541 shutdown_connections 2024-08-31T06:12:29.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.515+0000 7f34fa2a0740 1 -- 172.21.15.16:0/11908541 wait complete. 2024-08-31T06:12:29.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.516+0000 7f34fa2a0740 1 -- start start 2024-08-31T06:12:29.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.516+0000 7f34fa2a0740 1 Processor -- start 2024-08-31T06:12:29.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.517+0000 7f34e9600640 1 --2- 172.21.15.16:0/59973756 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f34d40b4d60 0x7f34d40b7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:29.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.517+0000 7f34e9600640 1 -- 172.21.15.16:0/59973756 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f34e00e52d0 con 0x7f34ec10a160 2024-08-31T06:12:29.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.518+0000 7f34fa2a0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617597d2b60 0x561759ba5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:29.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.518+0000 7f34fa2a0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617597f3840 0x561759ba62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:29.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.518+0000 7f34fa2a0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617597b21c0 0x561759bb1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:29.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.518+0000 7f34fa2a0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5617599617a0 con 0x5617597b21c0 2024-08-31T06:12:29.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.518+0000 7f34fa2a0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561759a857f0 con 0x5617597d2b60 2024-08-31T06:12:29.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.518+0000 7f34fa2a0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x561759ba8890 con 0x5617597f3840 2024-08-31T06:12:29.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.518+0000 7f34fa2a0740 10 client.0 _ll_register_callbacks cb 0x561759aaae60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:12:29.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.518+0000 7f34f0c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617597d2b60 0x561759ba5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:29.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.518+0000 7f34f1600640 1 --2- 172.21.15.16:0/59973756 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f34d40b4d60 0x7f34d40b7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:29.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.518+0000 7f34f0c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617597d2b60 0x561759ba5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:45762/0 (socket says 172.21.15.16:45762) 2024-08-31T06:12:29.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.518+0000 7f34f0c00640 1 -- 172.21.15.16:0/2440204085 learned_addr learned my addr 172.21.15.16:0/2440204085 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:29.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.519+0000 7f34f0c00640 1 -- 172.21.15.16:0/2440204085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617597f3840 msgr2=0x561759ba62e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T06:12:29.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.519+0000 7f34f0c00640 1 --2- 172.21.15.16:0/2440204085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617597f3840 0x561759ba62e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:29.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.519+0000 7f34f0c00640 1 -- 172.21.15.16:0/2440204085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617597b21c0 msgr2=0x561759bb1c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:12:29.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.519+0000 7f34f0c00640 1 --2- 172.21.15.16:0/2440204085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617597b21c0 0x561759bb1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:29.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.519+0000 7f34f0c00640 1 -- 172.21.15.16:0/2440204085 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56175986db90 con 0x5617597d2b60 2024-08-31T06:12:29.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.519+0000 7f34f0c00640 1 --2- 172.21.15.16:0/2440204085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617597d2b60 0x561759ba5d60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f34e00a8c90 tx=0x7f34e00ab190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31e9b5ac05863e37 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:29.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.520+0000 7f34d2000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:12:29.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.520+0000 7f34d2000640 1 -- 172.21.15.16:0/2440204085 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e001f500 con 0x5617597d2b60 2024-08-31T06:12:29.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.520+0000 7f34d2000640 1 -- 172.21.15.16:0/2440204085 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f34e00a8010 con 0x5617597d2b60 2024-08-31T06:12:29.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.521+0000 7f34fa2a0740 1 -- 172.21.15.16:0/2440204085 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34e4037040 con 0x5617597d2b60 2024-08-31T06:12:29.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.521+0000 7f34f1600640 1 --2- 172.21.15.16:0/59973756 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f34d40b4d60 0x7f34d40b7270 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f34e40099a0 tx=0x7f34e4030e20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:29.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.521+0000 7f34d2000640 1 -- 172.21.15.16:0/2440204085 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f34e00ac120 con 0x5617597d2b60 2024-08-31T06:12:29.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.521+0000 7f34fa2a0740 10 client.26756 fetch_fsmap learned FSMap version 37 2024-08-31T06:12:29.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.521+0000 7f34fa2a0740 1 -- 172.21.15.16:0/2440204085 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x561759bb49e0 con 0x5617597d2b60 2024-08-31T06:12:29.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.522+0000 7f34d2000640 1 -- 172.21.15.16:0/2440204085 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34e4037040 con 0x5617597d2b60 2024-08-31T06:12:29.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.523+0000 7f34d2000640 1 -- 172.21.15.16:0/2440204085 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f34e0020030 con 0x5617597d2b60 2024-08-31T06:12:29.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.523+0000 7f34fa2a0740 10 client.26756 fetch_fsmap finished waiting for FSMap version 37 2024-08-31T06:12:29.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.523+0000 7f34fa2a0740 10 client.26756 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:12:29.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.523+0000 7f34fa2a0740 20 client.26756 populate_metadata read hostname 'smithi016' 2024-08-31T06:12:29.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.523+0000 7f34fa2a0740 1 --2- 172.21.15.16:0/2440204085 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x561759787570 0x561759bb7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:29.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.524+0000 7f34fa2a0740 4 client.26756 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:12:29.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.524+0000 7f34fa2a0740 1 -- 172.21.15.16:0/2440204085 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561759b11480 con 0x561759787570 2024-08-31T06:12:29.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.529+0000 7f34ebe00640 1 --2- 172.21.15.16:0/2440204085 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x561759787570 0x561759bb7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:29.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.532+0000 7f34ebe00640 1 --2- 172.21.15.16:0/2440204085 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x561759787570 0x561759bb7eb0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d35c5599d881f5cf server_cookie=a0d699a84eac14bb in_seq=0 out_seq=0 2024-08-31T06:12:29.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.532+0000 7f34d2000640 10 client.26756 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:12:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.533+0000 7f34d2000640 1 -- 172.21.15.16:0/2440204085 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561759b11480 con 0x561759787570 2024-08-31T06:12:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.533+0000 7f34d2000640 10 client.26756 handle_command_reply: tid=0 2024-08-31T06:12:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.533+0000 7f34fa2a0740 1 client.26756 shutdown 2024-08-31T06:12:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.537+0000 7f34fa2a0740 1 -- 172.21.15.16:0/2440204085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617597d2b60 msgr2=0x561759ba5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.537+0000 7f34fa2a0740 1 --2- 172.21.15.16:0/2440204085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617597d2b60 0x561759ba5d60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f34e00a8c90 tx=0x7f34e00ab190 comp rx=0 tx=0).stop 2024-08-31T06:12:29.541 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T06:12:29.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.537+0000 7f34fa2a0740 1 -- 172.21.15.16:0/2440204085 shutdown_connections 2024-08-31T06:12:29.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.537+0000 7f34fa2a0740 1 --2- 172.21.15.16:0/2440204085 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x561759787570 0x561759bb7eb0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:29.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.537+0000 7f34fa2a0740 1 --2- 172.21.15.16:0/2440204085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617597f3840 0x561759ba62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:29.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.537+0000 7f34fa2a0740 1 --2- 172.21.15.16:0/2440204085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617597b21c0 0x561759bb1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:29.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.537+0000 7f34fa2a0740 1 --2- 172.21.15.16:0/2440204085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617597d2b60 0x561759ba5d60 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:29.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.537+0000 7f34fa2a0740 1 -- 172.21.15.16:0/2440204085 >> 172.21.15.16:0/2440204085 conn(0x5617597e7990 msgr2=0x561759b9b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:29.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.538+0000 7f34d2000640 0 client.26756 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:12:29.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.538+0000 7f34fa2a0740 1 -- 172.21.15.16:0/2440204085 shutdown_connections 2024-08-31T06:12:29.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.538+0000 7f34fa2a0740 1 -- 172.21.15.16:0/2440204085 wait complete. 2024-08-31T06:12:29.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.538+0000 7f34fa2a0740 20 client.26756 trim_cache size 0 max 16384 2024-08-31T06:12:29.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.571+0000 7f34fa2a0740 1 Processor -- start 2024-08-31T06:12:29.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.571+0000 7f34fa2a0740 1 -- start start 2024-08-31T06:12:29.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.571+0000 7f34fa2a0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617597b21c0 0x561759b9b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:29.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.571+0000 7f34fa2a0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617597d2b60 0x561759bb1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:29.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.571+0000 7f34fa2a0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617597f3840 0x561759ba2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:29.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.571+0000 7f34fa2a0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc010fa0 con 0x5617597f3840 2024-08-31T06:12:29.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.571+0000 7f34fa2a0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc00e810 con 0x5617597d2b60 2024-08-31T06:12:29.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.571+0000 7f34fa2a0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc00c660 con 0x5617597b21c0 2024-08-31T06:12:29.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.572+0000 7f34ebe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617597d2b60 0x561759bb1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:29.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.572+0000 7f34ebe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617597d2b60 0x561759bb1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:45768/0 (socket says 172.21.15.16:45768) 2024-08-31T06:12:29.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.572+0000 7f34ebe00640 1 -- 172.21.15.16:0/1291164416 learned_addr learned my addr 172.21.15.16:0/1291164416 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:29.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.572+0000 7f34f1600640 1 --2- 172.21.15.16:0/1291164416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617597f3840 0x561759ba2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:29.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.573+0000 7f34ebe00640 1 -- 172.21.15.16:0/1291164416 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617597b21c0 msgr2=0x561759b9b770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:12:29.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.573+0000 7f34ebe00640 1 --2- 172.21.15.16:0/1291164416 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617597b21c0 0x561759b9b770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:29.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.573+0000 7f34ebe00640 1 -- 172.21.15.16:0/1291164416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617597f3840 msgr2=0x561759ba2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:29.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.573+0000 7f34ebe00640 1 --2- 172.21.15.16:0/1291164416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617597f3840 0x561759ba2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:29.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.573+0000 7f34ebe00640 1 -- 172.21.15.16:0/1291164416 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561759b9cf70 con 0x5617597d2b60 2024-08-31T06:12:29.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.575+0000 7f34ebe00640 1 --2- 172.21.15.16:0/1291164416 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617597d2b60 0x561759bb1b10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x561759b98ed0 tx=0x7f34d8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=950dce8626aea92b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:29.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.577+0000 7f34dd200640 1 -- 172.21.15.16:0/1291164416 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d8023620 con 0x5617597d2b60 2024-08-31T06:12:29.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.577+0000 7f34dd200640 1 -- 172.21.15.16:0/1291164416 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34d801e030 con 0x5617597d2b60 2024-08-31T06:12:29.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.577+0000 7f34dd200640 1 -- 172.21.15.16:0/1291164416 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d80117c0 con 0x5617597d2b60 2024-08-31T06:12:29.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.578+0000 7f34fa2a0740 1 -- 172.21.15.16:0/1291164416 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617597d2b60 msgr2=0x561759bb1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:29.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.578+0000 7f34fa2a0740 1 --2- 172.21.15.16:0/1291164416 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617597d2b60 0x561759bb1b10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x561759b98ed0 tx=0x7f34d8009260 comp rx=0 tx=0).stop 2024-08-31T06:12:29.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.581+0000 7f34fa2a0740 1 -- 172.21.15.16:0/1291164416 shutdown_connections 2024-08-31T06:12:29.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.581+0000 7f34fa2a0740 1 --2- 172.21.15.16:0/1291164416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617597f3840 0x561759ba2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:29.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.581+0000 7f34fa2a0740 1 --2- 172.21.15.16:0/1291164416 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617597d2b60 0x561759bb1b10 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:29.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.581+0000 7f34fa2a0740 1 --2- 172.21.15.16:0/1291164416 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617597b21c0 0x561759b9b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:29.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.581+0000 7f34fa2a0740 1 -- 172.21.15.16:0/1291164416 >> 172.21.15.16:0/1291164416 conn(0x5617597e7990 msgr2=0x561759b9b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:29.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.583+0000 7f34fa2a0740 1 -- 172.21.15.16:0/1291164416 shutdown_connections 2024-08-31T06:12:29.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.583+0000 7f34fa2a0740 1 -- 172.21.15.16:0/1291164416 wait complete. 2024-08-31T06:12:29.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.584+0000 7f34fa2a0740 1 -- start start 2024-08-31T06:12:29.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.584+0000 7f34fa2a0740 1 Processor -- start 2024-08-31T06:12:29.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.586+0000 7f34fa2a0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617597b21c0 0x561759bae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:29.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.586+0000 7f34fa2a0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617597d2b60 0x561759baab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:29.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.586+0000 7f34fa2a0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617597f3840 0x561759bab080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:29.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.587+0000 7f34fa2a0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc00ccd0 con 0x5617597f3840 2024-08-31T06:12:29.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.587+0000 7f34fa2a0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc00e140 con 0x5617597b21c0 2024-08-31T06:12:29.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.587+0000 7f34fa2a0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc00d3a0 con 0x5617597d2b60 2024-08-31T06:12:29.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.587+0000 7f34fa2a0740 10 client.0 _ll_register_callbacks cb 0x7f34e00ab370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:12:29.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.587+0000 7f34f1600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617597f3840 0x561759bab080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:29.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.586+0000 7f34f0c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617597b21c0 0x561759bae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:29.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.587+0000 7f34f0c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617597b21c0 0x561759bae1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:45774/0 (socket says 172.21.15.16:45774) 2024-08-31T06:12:29.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.587+0000 7f34f0c00640 1 -- 172.21.15.16:0/2725596257 learned_addr learned my addr 172.21.15.16:0/2725596257 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:29.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.587+0000 7f34ebe00640 1 --2- 172.21.15.16:0/2725596257 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617597d2b60 0x561759baab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:29.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.588+0000 7f34f0c00640 1 -- 172.21.15.16:0/2725596257 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617597d2b60 msgr2=0x561759baab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:29.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.588+0000 7f34f0c00640 1 --2- 172.21.15.16:0/2725596257 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617597d2b60 0x561759baab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:29.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.588+0000 7f34f0c00640 1 -- 172.21.15.16:0/2725596257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617597f3840 msgr2=0x561759bab080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:29.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.588+0000 7f34f0c00640 1 --2- 172.21.15.16:0/2725596257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617597f3840 0x561759bab080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:29.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.588+0000 7f34f0c00640 1 -- 172.21.15.16:0/2725596257 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561759bab6c0 con 0x5617597b21c0 2024-08-31T06:12:29.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.589+0000 7f34f0c00640 1 --2- 172.21.15.16:0/2725596257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617597b21c0 0x561759bae1f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f34e00e55f0 tx=0x7f34e000cf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80e8acb6d3a5a006 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:29.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.589+0000 7f34d2000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:12:29.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.591+0000 7f34d2000640 1 -- 172.21.15.16:0/2725596257 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e00ca070 con 0x5617597b21c0 2024-08-31T06:12:29.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.591+0000 7f34d2000640 1 -- 172.21.15.16:0/2725596257 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f34e00a8010 con 0x5617597b21c0 2024-08-31T06:12:29.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.592+0000 7f34fa2a0740 1 -- 172.21.15.16:0/2725596257 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34d801e030 con 0x5617597b21c0 2024-08-31T06:12:29.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.592+0000 7f34d2000640 1 -- 172.21.15.16:0/2725596257 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f34e001f500 con 0x5617597b21c0 2024-08-31T06:12:29.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.592+0000 7f34fa2a0740 10 client.26762 fetch_fsmap learned FSMap version 37 2024-08-31T06:12:29.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.592+0000 7f34fa2a0740 1 -- 172.21.15.16:0/2725596257 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x561759c23680 con 0x5617597b21c0 2024-08-31T06:12:29.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.593+0000 7f34d2000640 1 -- 172.21.15.16:0/2725596257 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34d801e030 con 0x5617597b21c0 2024-08-31T06:12:29.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.593+0000 7f34d2000640 1 -- 172.21.15.16:0/2725596257 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f34e00aa6f0 con 0x5617597b21c0 2024-08-31T06:12:29.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.593+0000 7f34fa2a0740 10 client.26762 fetch_fsmap finished waiting for FSMap version 37 2024-08-31T06:12:29.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.593+0000 7f34fa2a0740 10 client.26762 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:12:29.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.593+0000 7f34fa2a0740 20 client.26762 populate_metadata read hostname 'smithi016' 2024-08-31T06:12:29.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.593+0000 7f34fa2a0740 1 --2- 172.21.15.16:0/2725596257 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x561759c23c60 0x561759c26080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:29.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.593+0000 7f34fa2a0740 4 client.26762 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 152.7, "await_for": 45.0}] 2024-08-31T06:12:29.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.593+0000 7f34fa2a0740 1 -- 172.21.15.16:0/2725596257 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 152.7, "await_for": 45.0}) -- 0x561759b0cf20 con 0x561759c23c60 2024-08-31T06:12:29.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.593+0000 7f34ebe00640 1 --2- 172.21.15.16:0/2725596257 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x561759c23c60 0x561759c26080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:29.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.595+0000 7f34ebe00640 1 --2- 172.21.15.16:0/2725596257 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x561759c23c60 0x561759c26080 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4633279878109c6 server_cookie=58a3920bd305bcfd in_seq=0 out_seq=0 2024-08-31T06:12:29.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.595+0000 7f34d2000640 10 client.26762 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:12:29.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.914+0000 7f34d2000640 1 -- 172.21.15.16:0/2725596257 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561759b0cf20 con 0x561759c23c60 2024-08-31T06:12:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.914+0000 7f34d2000640 10 client.26762 handle_command_reply: tid=0 2024-08-31T06:12:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.914+0000 7f34fa2a0740 1 client.26762 shutdown 2024-08-31T06:12:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.915+0000 7f34fa2a0740 1 -- 172.21.15.16:0/2725596257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617597b21c0 msgr2=0x561759bae1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.915+0000 7f34fa2a0740 1 --2- 172.21.15.16:0/2725596257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617597b21c0 0x561759bae1f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f34e00e55f0 tx=0x7f34e000cf30 comp rx=0 tx=0).stop 2024-08-31T06:12:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.915+0000 7f34fa2a0740 1 -- 172.21.15.16:0/2725596257 shutdown_connections 2024-08-31T06:12:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.915+0000 7f34fa2a0740 1 --2- 172.21.15.16:0/2725596257 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x561759c23c60 0x561759c26080 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.915+0000 7f34fa2a0740 1 --2- 172.21.15.16:0/2725596257 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5617597d2b60 0x561759baab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.915+0000 7f34fa2a0740 1 --2- 172.21.15.16:0/2725596257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5617597f3840 0x561759bab080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.915+0000 7f34fa2a0740 1 --2- 172.21.15.16:0/2725596257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617597b21c0 0x561759bae1f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.915+0000 7f34fa2a0740 1 -- 172.21.15.16:0/2725596257 >> 172.21.15.16:0/2725596257 conn(0x5617597e7990 msgr2=0x561759b97960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.915+0000 7f34d2000640 0 client.26762 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:12:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.915+0000 7f34fa2a0740 1 -- 172.21.15.16:0/2725596257 shutdown_connections 2024-08-31T06:12:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.915+0000 7f34fa2a0740 1 -- 172.21.15.16:0/2725596257 wait complete. 2024-08-31T06:12:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.915+0000 7f34fa2a0740 20 client.26762 trim_cache size 0 max 16384 2024-08-31T06:12:29.919 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:12:29.919 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 37, 2024-08-31T06:12:29.919 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24737, 2024-08-31T06:12:29.919 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 18, 2024-08-31T06:12:29.919 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-08-31T06:12:29.919 INFO:teuthology.orchestra.run.smithi016.stdout: "2facd010": { 2024-08-31T06:12:29.920 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 18, 2024-08-31T06:12:29.920 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-08-31T06:12:29.920 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:12:29.920 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-08-31T06:12:29.920 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:12:29.920 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:12:29.920 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-08-31T06:12:29.920 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 152.7, 2024-08-31T06:12:29.920 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-08-31T06:12:29.920 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-08-31T06:12:29.920 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-08-31T06:12:29.920 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:12:29.920 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-08-31T06:12:29.920 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:12:29.921 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:12:29.921 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:12:29.921 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:12:29.921 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:12:29.921 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:12:29.921 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:12:29.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.916+0000 7f34f3600640 1 -- 172.21.15.16:0/59973756 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f34d40b4d60 msgr2=0x7f34d40b7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:29.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.916+0000 7f34f3600640 1 --2- 172.21.15.16:0/59973756 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f34d40b4d60 0x7f34d40b7270 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f34e40099a0 tx=0x7f34e4030e20 comp rx=0 tx=0).stop 2024-08-31T06:12:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.916+0000 7f34f3600640 1 -- 172.21.15.16:0/59973756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34ec10a160 msgr2=0x7f34ec1a9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.916+0000 7f34f3600640 1 --2- 172.21.15.16:0/59973756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34ec10a160 0x7f34ec1a9000 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f34e000c940 tx=0x7f34e000ce00 comp rx=0 tx=0).stop 2024-08-31T06:12:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.916+0000 7f34f3600640 1 -- 172.21.15.16:0/59973756 shutdown_connections 2024-08-31T06:12:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.916+0000 7f34f3600640 1 --2- 172.21.15.16:0/59973756 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f34d40b4d60 0x7f34d40b7270 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.916+0000 7f34f3600640 1 --2- 172.21.15.16:0/59973756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34ec11ce80 0x7f34ec1ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.916+0000 7f34f3600640 1 --2- 172.21.15.16:0/59973756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f34ec10aab0 0x7f34ec1a95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.916+0000 7f34f3600640 1 --2- 172.21.15.16:0/59973756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34ec10a160 0x7f34ec1a9000 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.916+0000 7f34f3600640 1 -- 172.21.15.16:0/59973756 >> 172.21.15.16:0/59973756 conn(0x7f34ec070c20 msgr2=0x7f34ec071b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.919+0000 7f34f3600640 1 -- 172.21.15.16:0/59973756 shutdown_connections 2024-08-31T06:12:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:29.919+0000 7f34f3600640 1 -- 172.21.15.16:0/59973756 wait complete. 2024-08-31T06:12:30.276 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:12:29 smithi016 ceph-mon[17194]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:12:30.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:12:29 smithi155 ceph-mon[21742]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:12:30.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:12:29 smithi184 ceph-mon[22331]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:12:31.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:12:30 smithi155 ceph-mon[21742]: pgmap v522: 97 pgs: 97 active+clean; 2.8 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 31 MiB/s wr, 763 op/s 2024-08-31T06:12:31.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:12:30 smithi016 ceph-mon[17194]: pgmap v522: 97 pgs: 97 active+clean; 2.8 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 31 MiB/s wr, 763 op/s 2024-08-31T06:12:31.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:12:30 smithi184 ceph-mon[22331]: pgmap v522: 97 pgs: 97 active+clean; 2.8 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 31 MiB/s wr, 763 op/s 2024-08-31T06:12:31.626 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 305.0 s, 1151.0 tps, lat 55.302 ms stddev 77.072 2024-08-31T06:12:31.960 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2facd010', quiesce took 8.1 seconds. Will release after: 24.6 2024-08-31T06:12:31.988 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:12:32.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.651+0000 7fd702c00640 1 -- 172.21.15.16:0/3852630273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6fc10b290 msgr2=0x7fd6fc10b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:32.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.651+0000 7fd702c00640 1 --2- 172.21.15.16:0/3852630273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6fc10b290 0x7fd6fc10b6a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8009900 tx=0x7fd6e802a3f0 comp rx=0 tx=0).stop 2024-08-31T06:12:32.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.651+0000 7fd702c00640 1 -- 172.21.15.16:0/3852630273 shutdown_connections 2024-08-31T06:12:32.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.651+0000 7fd702c00640 1 --2- 172.21.15.16:0/3852630273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6fc10bbe0 0x7fd6fc112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:32.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.651+0000 7fd702c00640 1 --2- 172.21.15.16:0/3852630273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6fc10b290 0x7fd6fc10b6a0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:32.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.651+0000 7fd702c00640 1 --2- 172.21.15.16:0/3852630273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6fc105950 0x7fd6fc105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:32.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.651+0000 7fd702c00640 1 -- 172.21.15.16:0/3852630273 >> 172.21.15.16:0/3852630273 conn(0x7fd6fc101420 msgr2=0x7fd6fc103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:32.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.651+0000 7fd702c00640 1 -- 172.21.15.16:0/3852630273 shutdown_connections 2024-08-31T06:12:32.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.651+0000 7fd702c00640 1 -- 172.21.15.16:0/3852630273 wait complete. 2024-08-31T06:12:32.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.652+0000 7fd702c00640 1 Processor -- start 2024-08-31T06:12:32.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.652+0000 7fd702c00640 1 -- start start 2024-08-31T06:12:32.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.653+0000 7fd702c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6fc105950 0x7fd6fc19c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:32.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.653+0000 7fd702c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6fc10b290 0x7fd6fc19c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:32.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.653+0000 7fd702c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6fc10bbe0 0x7fd6fc1a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:32.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.653+0000 7fd702c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd6fc1158f0 con 0x7fd6fc105950 2024-08-31T06:12:32.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.653+0000 7fd702c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd6fc115770 con 0x7fd6fc10b290 2024-08-31T06:12:32.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.653+0000 7fd702c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd6fc115a70 con 0x7fd6fc10bbe0 2024-08-31T06:12:32.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.653+0000 7fd701800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6fc105950 0x7fd6fc19c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:32.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.653+0000 7fd700e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6fc10b290 0x7fd6fc19c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:32.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.653+0000 7fd701800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6fc105950 0x7fd6fc19c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51474/0 (socket says 172.21.15.16:51474) 2024-08-31T06:12:32.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.653+0000 7fd701800640 1 -- 172.21.15.16:0/3519336292 learned_addr learned my addr 172.21.15.16:0/3519336292 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:32.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.653+0000 7fd702200640 1 --2- 172.21.15.16:0/3519336292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6fc10bbe0 0x7fd6fc1a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:32.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.653+0000 7fd701800640 1 -- 172.21.15.16:0/3519336292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6fc10bbe0 msgr2=0x7fd6fc1a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:32.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.653+0000 7fd701800640 1 --2- 172.21.15.16:0/3519336292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6fc10bbe0 0x7fd6fc1a1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:32.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.653+0000 7fd701800640 1 -- 172.21.15.16:0/3519336292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6fc10b290 msgr2=0x7fd6fc19c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:32.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.653+0000 7fd701800640 1 --2- 172.21.15.16:0/3519336292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6fc10b290 0x7fd6fc19c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:32.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.653+0000 7fd701800640 1 -- 172.21.15.16:0/3519336292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6fc1a2190 con 0x7fd6fc105950 2024-08-31T06:12:32.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.654+0000 7fd701800640 1 --2- 172.21.15.16:0/3519336292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6fc105950 0x7fd6fc19c2e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000d950 tx=0x7fd6f000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:32.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.654+0000 7fd6fa000640 1 -- 172.21.15.16:0/3519336292 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f0017030 con 0x7fd6fc105950 2024-08-31T06:12:32.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.654+0000 7fd6fa000640 1 -- 172.21.15.16:0/3519336292 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6f0004020 con 0x7fd6fc105950 2024-08-31T06:12:32.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.654+0000 7fd702c00640 1 -- 172.21.15.16:0/3519336292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6fc1a2480 con 0x7fd6fc105950 2024-08-31T06:12:32.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.654+0000 7fd6fa000640 1 -- 172.21.15.16:0/3519336292 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f00047a0 con 0x7fd6fc105950 2024-08-31T06:12:32.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.654+0000 7fd702c00640 1 -- 172.21.15.16:0/3519336292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6fc1a8ca0 con 0x7fd6fc105950 2024-08-31T06:12:32.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.658+0000 7fd70aeaf740 1 Processor -- start 2024-08-31T06:12:32.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.658+0000 7fd70aeaf740 1 -- start start 2024-08-31T06:12:32.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.658+0000 7fd70aeaf740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a7e837070 0x555a7ebdca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:32.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.658+0000 7fd70aeaf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a7e80e1b0 0x555a7ebdcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:32.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.658+0000 7fd70aeaf740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a7e816390 0x555a7ebdd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:32.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.658+0000 7fd70aeaf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555a7eb537b0 con 0x555a7e80e1b0 2024-08-31T06:12:32.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.658+0000 7fd70aeaf740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555a7eb42d40 con 0x555a7e816390 2024-08-31T06:12:32.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.658+0000 7fd70aeaf740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x555a7eb46cf0 con 0x555a7e837070 2024-08-31T06:12:32.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.659+0000 7fd700e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a7e816390 0x555a7ebdd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:32.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.659+0000 7fd702200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a7e837070 0x555a7ebdca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:32.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.659+0000 7fd700e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a7e816390 0x555a7ebdd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:45794/0 (socket says 172.21.15.16:45794) 2024-08-31T06:12:32.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.659+0000 7fd700e00640 1 -- 172.21.15.16:0/3109335350 learned_addr learned my addr 172.21.15.16:0/3109335350 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:32.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.659+0000 7fd6fa000640 1 -- 172.21.15.16:0/3519336292 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fd6f0009a30 con 0x7fd6fc105950 2024-08-31T06:12:32.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.659+0000 7fd700e00640 1 -- 172.21.15.16:0/3109335350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a7e837070 msgr2=0x555a7ebdca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:32.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.659+0000 7fd700e00640 1 --2- 172.21.15.16:0/3109335350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a7e837070 0x555a7ebdca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:32.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.659+0000 7fd700e00640 1 -- 172.21.15.16:0/3109335350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a7e80e1b0 msgr2=0x555a7ebdcfb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:12:32.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.659+0000 7fd700e00640 1 --2- 172.21.15.16:0/3109335350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a7e80e1b0 0x555a7ebdcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:32.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.659+0000 7fd700e00640 1 -- 172.21.15.16:0/3109335350 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a7e8a2650 con 0x555a7e816390 2024-08-31T06:12:32.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.660+0000 7fd700e00640 1 --2- 172.21.15.16:0/3109335350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a7e816390 0x555a7ebdd4f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8005ab0 tx=0x7fd6e8004390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8674010a3a12c059 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:32.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.660+0000 7fd6eea00640 1 -- 172.21.15.16:0/3109335350 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e8035040 con 0x555a7e816390 2024-08-31T06:12:32.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.660+0000 7fd6eea00640 1 -- 172.21.15.16:0/3109335350 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6e8004750 con 0x555a7e816390 2024-08-31T06:12:32.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.660+0000 7fd6eea00640 1 -- 172.21.15.16:0/3109335350 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e8003850 con 0x555a7e816390 2024-08-31T06:12:32.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.661+0000 7fd6fa000640 1 --2- 172.21.15.16:0/3519336292 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd6e40b4d50 0x7fd6e40b7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:32.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.661+0000 7fd6fa000640 1 -- 172.21.15.16:0/3519336292 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fd6f00e4f50 con 0x7fd6fc105950 2024-08-31T06:12:32.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.661+0000 7fd702200640 1 --2- 172.21.15.16:0/3519336292 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd6e40b4d50 0x7fd6e40b7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:32.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.661+0000 7fd70aeaf740 1 -- 172.21.15.16:0/3109335350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a7e816390 msgr2=0x555a7ebdd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:32.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.661+0000 7fd70aeaf740 1 --2- 172.21.15.16:0/3109335350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a7e816390 0x555a7ebdd4f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8005ab0 tx=0x7fd6e8004390 comp rx=0 tx=0).stop 2024-08-31T06:12:32.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.661+0000 7fd70aeaf740 1 -- 172.21.15.16:0/3109335350 shutdown_connections 2024-08-31T06:12:32.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.661+0000 7fd70aeaf740 1 --2- 172.21.15.16:0/3109335350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a7e816390 0x555a7ebdd4f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:32.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.661+0000 7fd70aeaf740 1 --2- 172.21.15.16:0/3109335350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a7e80e1b0 0x555a7ebdcfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:32.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.661+0000 7fd70aeaf740 1 --2- 172.21.15.16:0/3109335350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a7e837070 0x555a7ebdca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:32.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.661+0000 7fd70aeaf740 1 -- 172.21.15.16:0/3109335350 >> 172.21.15.16:0/3109335350 conn(0x555a7e82b460 msgr2=0x555a7ebda960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:32.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.661+0000 7fd70aeaf740 1 -- 172.21.15.16:0/3109335350 shutdown_connections 2024-08-31T06:12:32.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.662+0000 7fd70aeaf740 1 -- 172.21.15.16:0/3109335350 wait complete. 2024-08-31T06:12:32.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.662+0000 7fd702200640 1 --2- 172.21.15.16:0/3519336292 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd6e40b4d50 0x7fd6e40b7260 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4007e50 tx=0x7fd6f4007600 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:32.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.662+0000 7fd70aeaf740 1 -- start start 2024-08-31T06:12:32.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.662+0000 7fd70aeaf740 1 Processor -- start 2024-08-31T06:12:32.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.663+0000 7fd70aeaf740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a7e80e1b0 0x555a7ebe86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:32.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.663+0000 7fd70aeaf740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a7e816390 0x555a7ebe8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:32.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.663+0000 7fd70aeaf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a7e837070 0x555a7ebf4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:32.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.663+0000 7fd70aeaf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555a7e9a5700 con 0x555a7e837070 2024-08-31T06:12:32.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.663+0000 7fd70aeaf740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555a7eac9cf0 con 0x555a7e80e1b0 2024-08-31T06:12:32.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.663+0000 7fd70aeaf740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x555a7ebeb1f0 con 0x555a7e816390 2024-08-31T06:12:32.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.663+0000 7fd70aeaf740 10 client.0 _ll_register_callbacks cb 0x555a7eaef470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:12:32.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.663+0000 7fd702200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a7e837070 0x555a7ebf4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:32.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.663+0000 7fd701800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a7e80e1b0 0x555a7ebe86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:32.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.663+0000 7fd702200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a7e837070 0x555a7ebf4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51492/0 (socket says 172.21.15.16:51492) 2024-08-31T06:12:32.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.663+0000 7fd700e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a7e816390 0x555a7ebe8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:32.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.663+0000 7fd702200640 1 -- 172.21.15.16:0/3564649242 learned_addr learned my addr 172.21.15.16:0/3564649242 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:32.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.664+0000 7fd700e00640 1 -- 172.21.15.16:0/3564649242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a7e80e1b0 msgr2=0x555a7ebe86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:32.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.664+0000 7fd700e00640 1 --2- 172.21.15.16:0/3564649242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a7e80e1b0 0x555a7ebe86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:32.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.664+0000 7fd700e00640 1 -- 172.21.15.16:0/3564649242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a7e837070 msgr2=0x555a7ebf4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:32.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.664+0000 7fd700e00640 1 --2- 172.21.15.16:0/3564649242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a7e837070 0x555a7ebf4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:32.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.664+0000 7fd700e00640 1 -- 172.21.15.16:0/3564649242 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a7eb0a8b0 con 0x555a7e816390 2024-08-31T06:12:32.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.664+0000 7fd702200640 1 --2- 172.21.15.16:0/3564649242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a7e837070 0x555a7ebf4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:12:32.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.664+0000 7fd700e00640 1 --2- 172.21.15.16:0/3564649242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a7e816390 0x555a7ebe8c60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd6e802a910 tx=0x7fd6e8004b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3147e50588fab9f0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:32.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.664+0000 7fd6e3400640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:12:32.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.665+0000 7fd6e3400640 1 -- 172.21.15.16:0/3564649242 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e8051020 con 0x555a7e816390 2024-08-31T06:12:32.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.665+0000 7fd70aeaf740 1 -- 172.21.15.16:0/3564649242 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6e8004750 con 0x555a7e816390 2024-08-31T06:12:32.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.665+0000 7fd6e3400640 1 -- 172.21.15.16:0/3564649242 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fd6e80051e0 con 0x555a7e816390 2024-08-31T06:12:32.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.665+0000 7fd6e3400640 1 -- 172.21.15.16:0/3564649242 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd6e8035040 con 0x555a7e816390 2024-08-31T06:12:32.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.665+0000 7fd70aeaf740 10 client.26761 fetch_fsmap learned FSMap version 37 2024-08-31T06:12:32.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.665+0000 7fd70aeaf740 1 -- 172.21.15.16:0/3564649242 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555a7ebf7280 con 0x555a7e816390 2024-08-31T06:12:32.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.665+0000 7fd6e3400640 1 -- 172.21.15.16:0/3564649242 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6e8004750 con 0x555a7e816390 2024-08-31T06:12:32.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.666+0000 7fd6e3400640 1 -- 172.21.15.16:0/3564649242 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fd6e804c070 con 0x555a7e816390 2024-08-31T06:12:32.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.666+0000 7fd70aeaf740 10 client.26761 fetch_fsmap finished waiting for FSMap version 37 2024-08-31T06:12:32.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.666+0000 7fd70aeaf740 10 client.26761 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-31T06:12:32.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.666+0000 7fd70aeaf740 20 client.26761 populate_metadata read hostname 'smithi016' 2024-08-31T06:12:32.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.666+0000 7fd70aeaf740 1 --2- 172.21.15.16:0/3564649242 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x555a7e7f59f0 0x555a7ebfa5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:32.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.666+0000 7fd70aeaf740 4 client.26761 mds_command: new command op to 24724 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:12:32.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.666+0000 7fd70aeaf740 1 -- 172.21.15.16:0/3564649242 --> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a7eb55700 con 0x555a7e7f59f0 2024-08-31T06:12:32.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.666+0000 7fd701800640 1 --2- 172.21.15.16:0/3564649242 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x555a7e7f59f0 0x555a7ebfa5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:32.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.667+0000 7fd701800640 1 --2- 172.21.15.16:0/3564649242 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x555a7e7f59f0 0x555a7ebfa5e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f41297c0910b2ba3 server_cookie=6e48bd4299e17617 in_seq=0 out_seq=0 2024-08-31T06:12:32.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.667+0000 7fd6e3400640 10 client.26761 ms_handle_connect on v2:172.21.15.16:6838/219664460 2024-08-31T06:12:32.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.667+0000 7fd6e3400640 1 -- 172.21.15.16:0/3564649242 <== mds.1 v2:172.21.15.16:6838/219664460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555a7eb55700 con 0x555a7e7f59f0 2024-08-31T06:12:32.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.667+0000 7fd6e3400640 10 client.26761 handle_command_reply: tid=0 2024-08-31T06:12:32.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.668+0000 7fd70aeaf740 1 client.26761 shutdown 2024-08-31T06:12:32.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.668+0000 7fd70aeaf740 1 -- 172.21.15.16:0/3564649242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a7e816390 msgr2=0x555a7ebe8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:32.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.668+0000 7fd70aeaf740 1 --2- 172.21.15.16:0/3564649242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a7e816390 0x555a7ebe8c60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd6e802a910 tx=0x7fd6e8004b30 comp rx=0 tx=0).stop 2024-08-31T06:12:32.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.668+0000 7fd70aeaf740 1 -- 172.21.15.16:0/3564649242 shutdown_connections 2024-08-31T06:12:32.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.668+0000 7fd70aeaf740 1 --2- 172.21.15.16:0/3564649242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a7e816390 0x555a7ebe8c60 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:32.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.668+0000 7fd70aeaf740 1 --2- 172.21.15.16:0/3564649242 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x555a7e7f59f0 0x555a7ebfa5e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:32.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.668+0000 7fd70aeaf740 1 --2- 172.21.15.16:0/3564649242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a7e837070 0x555a7ebf4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:32.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.668+0000 7fd70aeaf740 1 --2- 172.21.15.16:0/3564649242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a7e80e1b0 0x555a7ebe86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:32.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.668+0000 7fd6e3400640 0 client.26761 ms_handle_reset on v2:172.21.15.16:6838/219664460 2024-08-31T06:12:32.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.668+0000 7fd70aeaf740 1 -- 172.21.15.16:0/3564649242 >> 172.21.15.16:0/3564649242 conn(0x555a7e82b460 msgr2=0x555a7ebdfc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:32.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.668+0000 7fd70aeaf740 1 -- 172.21.15.16:0/3564649242 shutdown_connections 2024-08-31T06:12:32.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.668+0000 7fd70aeaf740 1 -- 172.21.15.16:0/3564649242 wait complete. 2024-08-31T06:12:32.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.668+0000 7fd70aeaf740 20 client.26761 trim_cache size 0 max 16384 2024-08-31T06:12:32.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.682+0000 7fd70aeaf740 1 Processor -- start 2024-08-31T06:12:32.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.682+0000 7fd70aeaf740 1 -- start start 2024-08-31T06:12:32.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.682+0000 7fd70aeaf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a7e80e1b0 0x555a7ebe2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:32.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.682+0000 7fd70aeaf740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a7e816390 0x555a7ebf4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:32.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.683+0000 7fd70aeaf740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a7e837070 0x555a7ebf49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:32.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.683+0000 7fd70aeaf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd6fc1b3c40 con 0x555a7e80e1b0 2024-08-31T06:12:32.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.683+0000 7fd70aeaf740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd6fc1b14b0 con 0x555a7e837070 2024-08-31T06:12:32.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.683+0000 7fd70aeaf740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd6fc1af300 con 0x555a7e816390 2024-08-31T06:12:32.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.683+0000 7fd701800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a7e80e1b0 0x555a7ebe2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:32.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.683+0000 7fd700e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a7e816390 0x555a7ebf4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:32.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.683+0000 7fd701800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a7e80e1b0 0x555a7ebe2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51508/0 (socket says 172.21.15.16:51508) 2024-08-31T06:12:32.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.683+0000 7fd700e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a7e816390 0x555a7ebf4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:37314/0 (socket says 172.21.15.16:37314) 2024-08-31T06:12:32.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.683+0000 7fd701800640 1 -- 172.21.15.16:0/1555752515 learned_addr learned my addr 172.21.15.16:0/1555752515 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:32.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.683+0000 7fd702200640 1 --2- 172.21.15.16:0/1555752515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a7e837070 0x555a7ebf49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:32.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.683+0000 7fd700e00640 1 -- 172.21.15.16:0/1555752515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a7e837070 msgr2=0x555a7ebf49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:32.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.683+0000 7fd700e00640 1 --2- 172.21.15.16:0/1555752515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a7e837070 0x555a7ebf49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:32.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.683+0000 7fd700e00640 1 -- 172.21.15.16:0/1555752515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a7e80e1b0 msgr2=0x555a7ebe2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:32.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.683+0000 7fd700e00640 1 --2- 172.21.15.16:0/1555752515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a7e80e1b0 0x555a7ebe2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:32.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.683+0000 7fd700e00640 1 -- 172.21.15.16:0/1555752515 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a7ebded60 con 0x555a7e816390 2024-08-31T06:12:32.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.683+0000 7fd701800640 1 --2- 172.21.15.16:0/1555752515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a7e80e1b0 0x555a7ebe2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:12:32.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.684+0000 7fd700e00640 1 --2- 172.21.15.16:0/1555752515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a7e816390 0x555a7ebf4480 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8005f70 tx=0x7fd6e802a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5823d3c4cfba66f4 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:32.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.684+0000 7fd6ee000640 1 -- 172.21.15.16:0/1555752515 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e8051040 con 0x555a7e816390 2024-08-31T06:12:32.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.684+0000 7fd6ee000640 1 -- 172.21.15.16:0/1555752515 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6e80055e0 con 0x555a7e816390 2024-08-31T06:12:32.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.684+0000 7fd6ee000640 1 -- 172.21.15.16:0/1555752515 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e803c560 con 0x555a7e816390 2024-08-31T06:12:32.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.685+0000 7fd70aeaf740 1 -- 172.21.15.16:0/1555752515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a7e816390 msgr2=0x555a7ebf4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:32.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.685+0000 7fd70aeaf740 1 --2- 172.21.15.16:0/1555752515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a7e816390 0x555a7ebf4480 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8005f70 tx=0x7fd6e802a3f0 comp rx=0 tx=0).stop 2024-08-31T06:12:32.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.685+0000 7fd70aeaf740 1 -- 172.21.15.16:0/1555752515 shutdown_connections 2024-08-31T06:12:32.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.685+0000 7fd70aeaf740 1 --2- 172.21.15.16:0/1555752515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a7e816390 0x555a7ebf4480 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:32.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.685+0000 7fd70aeaf740 1 --2- 172.21.15.16:0/1555752515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a7e837070 0x555a7ebf49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:32.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.685+0000 7fd70aeaf740 1 --2- 172.21.15.16:0/1555752515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a7e80e1b0 0x555a7ebe2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:32.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.685+0000 7fd70aeaf740 1 -- 172.21.15.16:0/1555752515 >> 172.21.15.16:0/1555752515 conn(0x555a7e82b460 msgr2=0x555a7ebe1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:32.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.685+0000 7fd70aeaf740 1 -- 172.21.15.16:0/1555752515 shutdown_connections 2024-08-31T06:12:32.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.685+0000 7fd70aeaf740 1 -- 172.21.15.16:0/1555752515 wait complete. 2024-08-31T06:12:32.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.686+0000 7fd70aeaf740 1 -- start start 2024-08-31T06:12:32.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.686+0000 7fd70aeaf740 1 Processor -- start 2024-08-31T06:12:32.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.686+0000 7fd70aeaf740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a7e80e1b0 0x555a7ebfbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:32.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.686+0000 7fd70aeaf740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a7e816390 0x555a7ebfc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:32.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.686+0000 7fd70aeaf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a7e837070 0x555a7ec63d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:32.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.687+0000 7fd70aeaf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd6fc1af970 con 0x555a7e837070 2024-08-31T06:12:32.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.687+0000 7fd70aeaf740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd6fc1b0de0 con 0x555a7e816390 2024-08-31T06:12:32.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.687+0000 7fd70aeaf740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd6fc1b0040 con 0x555a7e80e1b0 2024-08-31T06:12:32.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.687+0000 7fd70aeaf740 10 client.0 _ll_register_callbacks cb 0x7fd6e8005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:12:32.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.687+0000 7fd701800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a7e80e1b0 0x555a7ebfbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:32.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.687+0000 7fd701800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a7e80e1b0 0x555a7ebfbd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:37320/0 (socket says 172.21.15.16:37320) 2024-08-31T06:12:32.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.687+0000 7fd701800640 1 -- 172.21.15.16:0/2234778265 learned_addr learned my addr 172.21.15.16:0/2234778265 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:32.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.687+0000 7fd700e00640 1 --2- 172.21.15.16:0/2234778265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a7e816390 0x555a7ebfc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:32.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.687+0000 7fd702200640 1 --2- 172.21.15.16:0/2234778265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a7e837070 0x555a7ec63d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:32.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.687+0000 7fd701800640 1 -- 172.21.15.16:0/2234778265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a7e816390 msgr2=0x555a7ebfc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:32.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.687+0000 7fd701800640 1 --2- 172.21.15.16:0/2234778265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a7e816390 0x555a7ebfc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:32.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.687+0000 7fd701800640 1 -- 172.21.15.16:0/2234778265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a7e837070 msgr2=0x555a7ec63d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:32.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.687+0000 7fd701800640 1 --2- 172.21.15.16:0/2234778265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a7e837070 0x555a7ec63d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:32.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.687+0000 7fd701800640 1 -- 172.21.15.16:0/2234778265 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a7ec642f0 con 0x555a7e80e1b0 2024-08-31T06:12:32.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.687+0000 7fd700e00640 1 --2- 172.21.15.16:0/2234778265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a7e816390 0x555a7ebfc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:12:32.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.688+0000 7fd701800640 1 --2- 172.21.15.16:0/2234778265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a7e80e1b0 0x555a7ebfbd80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0005fd0 tx=0x7fd6f00ab470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83c5e1d059c308d5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:32.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.688+0000 7fd6e3400640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:12:32.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.688+0000 7fd6e3400640 1 -- 172.21.15.16:0/2234778265 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f00ab6a0 con 0x555a7e80e1b0 2024-08-31T06:12:32.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.688+0000 7fd6e3400640 1 -- 172.21.15.16:0/2234778265 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fd6f00a7c90 con 0x555a7e80e1b0 2024-08-31T06:12:32.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.688+0000 7fd70aeaf740 1 -- 172.21.15.16:0/2234778265 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6e80055e0 con 0x555a7e80e1b0 2024-08-31T06:12:32.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.688+0000 7fd6e3400640 1 -- 172.21.15.16:0/2234778265 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd6f00aff10 con 0x555a7e80e1b0 2024-08-31T06:12:32.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.688+0000 7fd70aeaf740 10 client.26767 fetch_fsmap learned FSMap version 37 2024-08-31T06:12:32.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.688+0000 7fd70aeaf740 1 -- 172.21.15.16:0/2234778265 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555a7ec65550 con 0x555a7e80e1b0 2024-08-31T06:12:32.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.688+0000 7fd6e3400640 1 -- 172.21.15.16:0/2234778265 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6e80055e0 con 0x555a7e80e1b0 2024-08-31T06:12:32.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.689+0000 7fd6e3400640 1 -- 172.21.15.16:0/2234778265 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fd6f00e70b0 con 0x555a7e80e1b0 2024-08-31T06:12:32.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.689+0000 7fd70aeaf740 10 client.26767 fetch_fsmap finished waiting for FSMap version 37 2024-08-31T06:12:32.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.689+0000 7fd70aeaf740 10 client.26767 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-31T06:12:32.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.689+0000 7fd70aeaf740 20 client.26767 populate_metadata read hostname 'smithi016' 2024-08-31T06:12:32.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.689+0000 7fd70aeaf740 1 --2- 172.21.15.16:0/2234778265 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x555a7e7cada0 0x555a7ec67d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:32.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.689+0000 7fd70aeaf740 4 client.26767 mds_command: new command op to 24724 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:12:32.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.689+0000 7fd70aeaf740 1 -- 172.21.15.16:0/2234778265 --> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555a7eb511a0 con 0x555a7e7cada0 2024-08-31T06:12:32.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.689+0000 7fd700e00640 1 --2- 172.21.15.16:0/2234778265 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x555a7e7cada0 0x555a7ec67d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:32.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.690+0000 7fd6e3400640 10 client.26767 ms_handle_connect on v2:172.21.15.16:6838/219664460 2024-08-31T06:12:32.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.690+0000 7fd700e00640 1 --2- 172.21.15.16:0/2234778265 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x555a7e7cada0 0x555a7ec67d20 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ec4b7818ec4946be server_cookie=9a7a9a57e46e0f70 in_seq=0 out_seq=0 2024-08-31T06:12:32.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.690+0000 7fd6e3400640 1 -- 172.21.15.16:0/2234778265 <== mds.1 v2:172.21.15.16:6838/219664460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555a7eb511a0 con 0x555a7e7cada0 2024-08-31T06:12:32.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.690+0000 7fd6e3400640 10 client.26767 handle_command_reply: tid=0 2024-08-31T06:12:32.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.690+0000 7fd70aeaf740 1 client.26767 shutdown 2024-08-31T06:12:32.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.690+0000 7fd70aeaf740 1 -- 172.21.15.16:0/2234778265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a7e80e1b0 msgr2=0x555a7ebfbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:32.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.690+0000 7fd70aeaf740 1 --2- 172.21.15.16:0/2234778265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a7e80e1b0 0x555a7ebfbd80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0005fd0 tx=0x7fd6f00ab470 comp rx=0 tx=0).stop 2024-08-31T06:12:32.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.690+0000 7fd70aeaf740 1 -- 172.21.15.16:0/2234778265 shutdown_connections 2024-08-31T06:12:32.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.690+0000 7fd70aeaf740 1 --2- 172.21.15.16:0/2234778265 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x555a7e7cada0 0x555a7ec67d20 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:32.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.690+0000 7fd70aeaf740 1 --2- 172.21.15.16:0/2234778265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a7e837070 0x555a7ec63d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:32.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.690+0000 7fd70aeaf740 1 --2- 172.21.15.16:0/2234778265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a7e816390 0x555a7ebfc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:32.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.690+0000 7fd70aeaf740 1 --2- 172.21.15.16:0/2234778265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a7e80e1b0 0x555a7ebfbd80 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:32.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.690+0000 7fd6e3400640 0 client.26767 ms_handle_reset on v2:172.21.15.16:6838/219664460 2024-08-31T06:12:32.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.690+0000 7fd70aeaf740 1 -- 172.21.15.16:0/2234778265 >> 172.21.15.16:0/2234778265 conn(0x555a7e82b460 msgr2=0x555a7ebe9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:32.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.690+0000 7fd70aeaf740 1 -- 172.21.15.16:0/2234778265 shutdown_connections 2024-08-31T06:12:32.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.690+0000 7fd70aeaf740 1 -- 172.21.15.16:0/2234778265 wait complete. 2024-08-31T06:12:32.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.691+0000 7fd70aeaf740 20 client.26767 trim_cache size 0 max 16384 2024-08-31T06:12:32.695 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:12:32.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.691+0000 7fd702c00640 1 -- 172.21.15.16:0/3519336292 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd6e40b4d50 msgr2=0x7fd6e40b7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:32.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.691+0000 7fd702c00640 1 --2- 172.21.15.16:0/3519336292 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd6e40b4d50 0x7fd6e40b7260 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4007e50 tx=0x7fd6f4007600 comp rx=0 tx=0).stop 2024-08-31T06:12:32.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.691+0000 7fd702c00640 1 -- 172.21.15.16:0/3519336292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6fc105950 msgr2=0x7fd6fc19c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:32.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.691+0000 7fd702c00640 1 --2- 172.21.15.16:0/3519336292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6fc105950 0x7fd6fc19c2e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000d950 tx=0x7fd6f000de10 comp rx=0 tx=0).stop 2024-08-31T06:12:32.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.691+0000 7fd702c00640 1 -- 172.21.15.16:0/3519336292 shutdown_connections 2024-08-31T06:12:32.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.691+0000 7fd702c00640 1 --2- 172.21.15.16:0/3519336292 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd6e40b4d50 0x7fd6e40b7260 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:32.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.691+0000 7fd702c00640 1 --2- 172.21.15.16:0/3519336292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6fc10bbe0 0x7fd6fc1a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:32.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.691+0000 7fd702c00640 1 --2- 172.21.15.16:0/3519336292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6fc10b290 0x7fd6fc19c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:32.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.691+0000 7fd702c00640 1 --2- 172.21.15.16:0/3519336292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6fc105950 0x7fd6fc19c2e0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:32.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.691+0000 7fd702c00640 1 -- 172.21.15.16:0/3519336292 >> 172.21.15.16:0/3519336292 conn(0x7fd6fc101420 msgr2=0x7fd6fc103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:32.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.692+0000 7fd702c00640 1 -- 172.21.15.16:0/3519336292 shutdown_connections 2024-08-31T06:12:32.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:32.692+0000 7fd702c00640 1 -- 172.21.15.16:0/3519336292 wait complete. 2024-08-31T06:12:33.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:12:32 smithi155 ceph-mon[21742]: pgmap v523: 97 pgs: 97 active+clean; 2.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 28 MiB/s wr, 666 op/s 2024-08-31T06:12:33.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:12:32 smithi016 ceph-mon[17194]: pgmap v523: 97 pgs: 97 active+clean; 2.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 28 MiB/s wr, 666 op/s 2024-08-31T06:12:33.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:12:32 smithi184 ceph-mon[22331]: pgmap v523: 97 pgs: 97 active+clean; 2.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 28 MiB/s wr, 666 op/s 2024-08-31T06:12:33.581 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T06:12:34.010 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:12:34.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.791+0000 7f2812400640 1 -- 172.21.15.16:0/86414854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f280c109240 msgr2=0x7f280c109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:34.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.791+0000 7f2812400640 1 --2- 172.21.15.16:0/86414854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f280c109240 0x7f280c109650 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f27f4009900 tx=0x7f27f402a410 comp rx=0 tx=0).stop 2024-08-31T06:12:34.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.791+0000 7f2812400640 1 -- 172.21.15.16:0/86414854 shutdown_connections 2024-08-31T06:12:34.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.791+0000 7f2812400640 1 --2- 172.21.15.16:0/86414854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f280c109b90 0x7f280c1103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:34.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.791+0000 7f2812400640 1 --2- 172.21.15.16:0/86414854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f280c109240 0x7f280c109650 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:34.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.791+0000 7f2812400640 1 --2- 172.21.15.16:0/86414854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f280c103900 0x7f280c103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:34.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.791+0000 7f2812400640 1 -- 172.21.15.16:0/86414854 >> 172.21.15.16:0/86414854 conn(0x7f280c07b5f0 msgr2=0x7f280c07ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:34.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.791+0000 7f2812400640 1 -- 172.21.15.16:0/86414854 shutdown_connections 2024-08-31T06:12:34.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.791+0000 7f2812400640 1 -- 172.21.15.16:0/86414854 wait complete. 2024-08-31T06:12:34.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.792+0000 7f2812400640 1 Processor -- start 2024-08-31T06:12:34.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.792+0000 7f2812400640 1 -- start start 2024-08-31T06:12:34.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.793+0000 7f2812400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f280c103900 0x7f280c0754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:34.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.793+0000 7f2812400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f280c109240 0x7f280c0759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:34.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.793+0000 7f2812400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f280c109b90 0x7f280c075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:34.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.793+0000 7f2812400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f280c113850 con 0x7f280c109b90 2024-08-31T06:12:34.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.793+0000 7f2812400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f280c1136d0 con 0x7f280c109240 2024-08-31T06:12:34.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.793+0000 7f2812400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f280c1139d0 con 0x7f280c103900 2024-08-31T06:12:34.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.793+0000 7f2803e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f280c109240 0x7f280c0759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:34.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.793+0000 7f280be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f280c109b90 0x7f280c075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:34.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.793+0000 7f280b400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f280c103900 0x7f280c0754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:34.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.793+0000 7f2803e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f280c109240 0x7f280c0759f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:45860/0 (socket says 172.21.15.16:45860) 2024-08-31T06:12:34.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.793+0000 7f2803e00640 1 -- 172.21.15.16:0/1763256195 learned_addr learned my addr 172.21.15.16:0/1763256195 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:34.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.794+0000 7f280b400640 1 -- 172.21.15.16:0/1763256195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f280c109240 msgr2=0x7f280c0759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:34.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.794+0000 7f280b400640 1 --2- 172.21.15.16:0/1763256195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f280c109240 0x7f280c0759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:34.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.794+0000 7f280b400640 1 -- 172.21.15.16:0/1763256195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f280c109b90 msgr2=0x7f280c075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:34.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.794+0000 7f280b400640 1 --2- 172.21.15.16:0/1763256195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f280c109b90 0x7f280c075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:34.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.794+0000 7f280b400640 1 -- 172.21.15.16:0/1763256195 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f280c1a84e0 con 0x7f280c103900 2024-08-31T06:12:34.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.794+0000 7f280b400640 1 --2- 172.21.15.16:0/1763256195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f280c103900 0x7f280c0754b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f27fc00e9f0 tx=0x7f27fc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:34.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.794+0000 7f2808c00640 1 -- 172.21.15.16:0/1763256195 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27fc00ccc0 con 0x7f280c103900 2024-08-31T06:12:34.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.794+0000 7f2812400640 1 -- 172.21.15.16:0/1763256195 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f280c1a87d0 con 0x7f280c103900 2024-08-31T06:12:34.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.794+0000 7f2808c00640 1 -- 172.21.15.16:0/1763256195 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27fc0108e0 con 0x7f280c103900 2024-08-31T06:12:34.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.795+0000 7f2808c00640 1 -- 172.21.15.16:0/1763256195 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27fc018880 con 0x7f280c103900 2024-08-31T06:12:34.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.795+0000 7f2812400640 1 -- 172.21.15.16:0/1763256195 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f280c1a8d60 con 0x7f280c103900 2024-08-31T06:12:34.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.798+0000 7f2819024740 1 Processor -- start 2024-08-31T06:12:34.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.798+0000 7f2819024740 1 -- start start 2024-08-31T06:12:34.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.798+0000 7f2819024740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9bf40d070 0x55f9bf7b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:34.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.798+0000 7f2819024740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9bf3e41b0 0x55f9bf7b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:34.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.798+0000 7f2819024740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bf3ec390 0x55f9bf7b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:34.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.798+0000 7f2819024740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f9bf7297b0 con 0x55f9bf3ec390 2024-08-31T06:12:34.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.798+0000 7f2819024740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f9bf718d40 con 0x55f9bf40d070 2024-08-31T06:12:34.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.798+0000 7f2819024740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f9bf71ccf0 con 0x55f9bf3e41b0 2024-08-31T06:12:34.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.799+0000 7f2803e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bf3ec390 0x55f9bf7b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:34.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.799+0000 7f280be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9bf40d070 0x55f9bf7b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:34.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.799+0000 7f280be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9bf40d070 0x55f9bf7b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:45864/0 (socket says 172.21.15.16:45864) 2024-08-31T06:12:34.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.799+0000 7f280be00640 1 -- 172.21.15.16:0/210016883 learned_addr learned my addr 172.21.15.16:0/210016883 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:34.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.799+0000 7f2808c00640 1 -- 172.21.15.16:0/1763256195 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f27fc010430 con 0x7f280c103900 2024-08-31T06:12:34.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.799+0000 7f280b400640 1 --2- 172.21.15.16:0/210016883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9bf3e41b0 0x55f9bf7b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:34.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.799+0000 7f280be00640 1 -- 172.21.15.16:0/210016883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9bf3e41b0 msgr2=0x55f9bf7b2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:34.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.799+0000 7f280be00640 1 --2- 172.21.15.16:0/210016883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9bf3e41b0 0x55f9bf7b2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:34.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.799+0000 7f280be00640 1 -- 172.21.15.16:0/210016883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bf3ec390 msgr2=0x55f9bf7b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:34.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.799+0000 7f280be00640 1 --2- 172.21.15.16:0/210016883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bf3ec390 0x55f9bf7b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:34.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.799+0000 7f280be00640 1 -- 172.21.15.16:0/210016883 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9bf478650 con 0x55f9bf40d070 2024-08-31T06:12:34.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.799+0000 7f2803e00640 1 --2- 172.21.15.16:0/210016883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bf3ec390 0x55f9bf7b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:12:34.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.799+0000 7f280b400640 1 --2- 172.21.15.16:0/210016883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9bf3e41b0 0x55f9bf7b2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:12:34.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.800+0000 7f280be00640 1 --2- 172.21.15.16:0/210016883 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9bf40d070 0x55f9bf7b2990 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2804009fd0 tx=0x7f28040118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=104fdd1b953b4ed5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:34.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.800+0000 7f2800c00640 1 -- 172.21.15.16:0/210016883 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2804004d20 con 0x55f9bf40d070 2024-08-31T06:12:34.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.800+0000 7f2800c00640 1 -- 172.21.15.16:0/210016883 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2804004050 con 0x55f9bf40d070 2024-08-31T06:12:34.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.800+0000 7f2800c00640 1 -- 172.21.15.16:0/210016883 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2804005280 con 0x55f9bf40d070 2024-08-31T06:12:34.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.800+0000 7f2808c00640 1 --2- 172.21.15.16:0/1763256195 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f27f00b4ca0 0x7f27f00b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:34.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.800+0000 7f2808c00640 1 -- 172.21.15.16:0/1763256195 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f27fc0e5890 con 0x7f280c103900 2024-08-31T06:12:34.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.801+0000 7f280b400640 1 --2- 172.21.15.16:0/1763256195 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f27f00b4ca0 0x7f27f00b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:34.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.801+0000 7f2819024740 1 -- 172.21.15.16:0/210016883 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9bf40d070 msgr2=0x55f9bf7b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:34.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.801+0000 7f2819024740 1 --2- 172.21.15.16:0/210016883 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9bf40d070 0x55f9bf7b2990 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2804009fd0 tx=0x7f28040118d0 comp rx=0 tx=0).stop 2024-08-31T06:12:34.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.801+0000 7f2819024740 1 -- 172.21.15.16:0/210016883 shutdown_connections 2024-08-31T06:12:34.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.801+0000 7f2819024740 1 --2- 172.21.15.16:0/210016883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9bf3e41b0 0x55f9bf7b2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:34.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.801+0000 7f2819024740 1 --2- 172.21.15.16:0/210016883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bf3ec390 0x55f9bf7b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:34.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.801+0000 7f2819024740 1 --2- 172.21.15.16:0/210016883 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9bf40d070 0x55f9bf7b2990 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:34.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.801+0000 7f2819024740 1 -- 172.21.15.16:0/210016883 >> 172.21.15.16:0/210016883 conn(0x55f9bf401460 msgr2=0x55f9bf7b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:34.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.801+0000 7f2819024740 1 -- 172.21.15.16:0/210016883 shutdown_connections 2024-08-31T06:12:34.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.801+0000 7f2819024740 1 -- 172.21.15.16:0/210016883 wait complete. 2024-08-31T06:12:34.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.801+0000 7f280b400640 1 --2- 172.21.15.16:0/1763256195 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f27f00b4ca0 0x7f27f00b71b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x55f9bf7b1610 tx=0x7f27fc009700 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:34.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.802+0000 7f2819024740 1 -- start start 2024-08-31T06:12:34.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.802+0000 7f2819024740 1 Processor -- start 2024-08-31T06:12:34.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.803+0000 7f2819024740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9bf3e41b0 0x55f9bf7be5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:34.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.803+0000 7f2819024740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9bf3ec390 0x55f9bf7beb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:34.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.803+0000 7f2819024740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bf40d070 0x55f9bf7ca460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:34.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.803+0000 7f2819024740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f9bf57b700 con 0x55f9bf40d070 2024-08-31T06:12:34.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.803+0000 7f2819024740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f9bf69fcf0 con 0x55f9bf3e41b0 2024-08-31T06:12:34.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.803+0000 7f2819024740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f9bf7c1170 con 0x55f9bf3ec390 2024-08-31T06:12:34.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.803+0000 7f2819024740 10 client.0 _ll_register_callbacks cb 0x55f9bf6c5470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:12:34.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.803+0000 7f280be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9bf3e41b0 0x55f9bf7be5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:34.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.803+0000 7f280be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bf40d070 0x55f9bf7ca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:34.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.803+0000 7f2803e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9bf3ec390 0x55f9bf7beb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:34.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.803+0000 7f280be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bf40d070 0x55f9bf7ca460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51544/0 (socket says 172.21.15.16:51544) 2024-08-31T06:12:34.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.803+0000 7f280be00640 1 -- 172.21.15.16:0/4272544086 learned_addr learned my addr 172.21.15.16:0/4272544086 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:34.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.804+0000 7f280be00640 1 -- 172.21.15.16:0/4272544086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9bf3ec390 msgr2=0x55f9bf7beb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:34.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.804+0000 7f280be00640 1 --2- 172.21.15.16:0/4272544086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9bf3ec390 0x55f9bf7beb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:34.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.804+0000 7f280be00640 1 -- 172.21.15.16:0/4272544086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9bf3e41b0 msgr2=0x55f9bf7be5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:34.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.804+0000 7f280be00640 1 --2- 172.21.15.16:0/4272544086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9bf3e41b0 0x55f9bf7be5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:34.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.804+0000 7f280be00640 1 -- 172.21.15.16:0/4272544086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9bf6e08b0 con 0x55f9bf40d070 2024-08-31T06:12:34.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.804+0000 7f280be00640 1 --2- 172.21.15.16:0/4272544086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bf40d070 0x55f9bf7ca460 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2804009fa0 tx=0x7f2804009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd5d5c6b834a8901 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:34.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.804+0000 7f27f9600640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:12:34.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.804+0000 7f27f9600640 1 -- 172.21.15.16:0/4272544086 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280402d060 con 0x55f9bf40d070 2024-08-31T06:12:34.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.804+0000 7f27f9600640 1 -- 172.21.15.16:0/4272544086 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f2804004c40 con 0x55f9bf40d070 2024-08-31T06:12:34.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.804+0000 7f2819024740 1 -- 172.21.15.16:0/4272544086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2804004050 con 0x55f9bf40d070 2024-08-31T06:12:34.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.805+0000 7f27f9600640 1 -- 172.21.15.16:0/4272544086 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f2804004e40 con 0x55f9bf40d070 2024-08-31T06:12:34.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.805+0000 7f2819024740 10 client.16578 fetch_fsmap learned FSMap version 38 2024-08-31T06:12:34.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.805+0000 7f2819024740 1 -- 172.21.15.16:0/4272544086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55f9bf7cd170 con 0x55f9bf40d070 2024-08-31T06:12:34.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.805+0000 7f27f9600640 1 -- 172.21.15.16:0/4272544086 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2804004050 con 0x55f9bf40d070 2024-08-31T06:12:34.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.805+0000 7f27f9600640 1 -- 172.21.15.16:0/4272544086 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f2804019430 con 0x55f9bf40d070 2024-08-31T06:12:34.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.805+0000 7f2819024740 10 client.16578 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T06:12:34.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.805+0000 7f2819024740 10 client.16578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:12:34.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.805+0000 7f2819024740 20 client.16578 populate_metadata read hostname 'smithi016' 2024-08-31T06:12:34.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.805+0000 7f2819024740 1 --2- 172.21.15.16:0/4272544086 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f9bf3cb9f0 0x55f9bf7d05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:34.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.805+0000 7f2819024740 4 client.16578 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:12:34.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.805+0000 7f2819024740 1 -- 172.21.15.16:0/4272544086 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9bf72b700 con 0x55f9bf3cb9f0 2024-08-31T06:12:34.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.806+0000 7f2803e00640 1 --2- 172.21.15.16:0/4272544086 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f9bf3cb9f0 0x55f9bf7d05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:34.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.806+0000 7f2803e00640 1 --2- 172.21.15.16:0/4272544086 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f9bf3cb9f0 0x55f9bf7d05e0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecb4ce5c71e0c12d server_cookie=33c414f44d67e493 in_seq=0 out_seq=0 2024-08-31T06:12:34.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.806+0000 7f27f9600640 10 client.16578 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:12:34.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.807+0000 7f27f9600640 1 -- 172.21.15.16:0/4272544086 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f9bf72b700 con 0x55f9bf3cb9f0 2024-08-31T06:12:34.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.807+0000 7f27f9600640 10 client.16578 handle_command_reply: tid=0 2024-08-31T06:12:34.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.807+0000 7f2819024740 1 client.16578 shutdown 2024-08-31T06:12:34.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.808+0000 7f2819024740 1 -- 172.21.15.16:0/4272544086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bf40d070 msgr2=0x55f9bf7ca460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:34.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.808+0000 7f2819024740 1 --2- 172.21.15.16:0/4272544086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bf40d070 0x55f9bf7ca460 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2804009fa0 tx=0x7f2804009500 comp rx=0 tx=0).stop 2024-08-31T06:12:34.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.808+0000 7f2819024740 1 -- 172.21.15.16:0/4272544086 shutdown_connections 2024-08-31T06:12:34.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.808+0000 7f2819024740 1 --2- 172.21.15.16:0/4272544086 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f9bf3cb9f0 0x55f9bf7d05e0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:34.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.808+0000 7f2819024740 1 --2- 172.21.15.16:0/4272544086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9bf3ec390 0x55f9bf7beb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:34.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.808+0000 7f27f9600640 0 client.16578 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:12:34.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.808+0000 7f2819024740 1 --2- 172.21.15.16:0/4272544086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bf40d070 0x55f9bf7ca460 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:34.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.808+0000 7f2819024740 1 --2- 172.21.15.16:0/4272544086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9bf3e41b0 0x55f9bf7be5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:34.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.808+0000 7f2819024740 1 -- 172.21.15.16:0/4272544086 >> 172.21.15.16:0/4272544086 conn(0x55f9bf401460 msgr2=0x55f9bf7b5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:34.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.808+0000 7f2819024740 1 -- 172.21.15.16:0/4272544086 shutdown_connections 2024-08-31T06:12:34.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.808+0000 7f2819024740 1 -- 172.21.15.16:0/4272544086 wait complete. 2024-08-31T06:12:34.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.808+0000 7f2819024740 20 client.16578 trim_cache size 0 max 16384 2024-08-31T06:12:34.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.823+0000 7f2819024740 1 Processor -- start 2024-08-31T06:12:34.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.823+0000 7f2819024740 1 -- start start 2024-08-31T06:12:34.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.823+0000 7f2819024740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bf3e41b0 0x55f9bf7ca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:34.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.823+0000 7f2819024740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9bf3ec390 0x55f9bf7bcc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:34.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.823+0000 7f2819024740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9bf40d070 0x55f9bf7bd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:34.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.823+0000 7f2819024740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f280c1b3fa0 con 0x55f9bf3e41b0 2024-08-31T06:12:34.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.823+0000 7f2819024740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f280c1b17e0 con 0x55f9bf40d070 2024-08-31T06:12:34.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.823+0000 7f2819024740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f280c1af630 con 0x55f9bf3ec390 2024-08-31T06:12:34.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.824+0000 7f280be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bf3e41b0 0x55f9bf7ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:34.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.824+0000 7f280be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bf3e41b0 0x55f9bf7ca360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51552/0 (socket says 172.21.15.16:51552) 2024-08-31T06:12:34.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.824+0000 7f280be00640 1 -- 172.21.15.16:0/2204853036 learned_addr learned my addr 172.21.15.16:0/2204853036 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:34.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.824+0000 7f2803e00640 1 --2- 172.21.15.16:0/2204853036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9bf3ec390 0x55f9bf7bcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:34.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.824+0000 7f280be00640 1 --2- 172.21.15.16:0/2204853036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9bf40d070 0x55f9bf7bd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:34.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.824+0000 7f2803e00640 1 -- 172.21.15.16:0/2204853036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9bf40d070 msgr2=0x55f9bf7bd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:34.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.824+0000 7f2803e00640 1 --2- 172.21.15.16:0/2204853036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9bf40d070 0x55f9bf7bd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:34.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.824+0000 7f2803e00640 1 -- 172.21.15.16:0/2204853036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bf3e41b0 msgr2=0x55f9bf7ca360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:34.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.824+0000 7f2803e00640 1 --2- 172.21.15.16:0/2204853036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bf3e41b0 0x55f9bf7ca360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:34.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.824+0000 7f2803e00640 1 -- 172.21.15.16:0/2204853036 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9bf7be920 con 0x55f9bf3ec390 2024-08-31T06:12:34.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.824+0000 7f280be00640 1 --2- 172.21.15.16:0/2204853036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9bf40d070 0x55f9bf7bd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:12:34.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.824+0000 7f2803e00640 1 --2- 172.21.15.16:0/2204853036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9bf3ec390 0x55f9bf7bcc30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f27f4031440 tx=0x7f27f4005890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1013fd5ae09668e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:34.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.825+0000 7f27fbe00640 1 -- 172.21.15.16:0/2204853036 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f404a070 con 0x55f9bf3ec390 2024-08-31T06:12:34.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.825+0000 7f27fbe00640 1 -- 172.21.15.16:0/2204853036 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27f4031950 con 0x55f9bf3ec390 2024-08-31T06:12:34.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.825+0000 7f27fbe00640 1 -- 172.21.15.16:0/2204853036 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f4035040 con 0x55f9bf3ec390 2024-08-31T06:12:34.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.826+0000 7f2819024740 1 -- 172.21.15.16:0/2204853036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9bf3ec390 msgr2=0x55f9bf7bcc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:34.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.826+0000 7f2819024740 1 --2- 172.21.15.16:0/2204853036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9bf3ec390 0x55f9bf7bcc30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f27f4031440 tx=0x7f27f4005890 comp rx=0 tx=0).stop 2024-08-31T06:12:34.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.826+0000 7f2819024740 1 -- 172.21.15.16:0/2204853036 shutdown_connections 2024-08-31T06:12:34.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.826+0000 7f2819024740 1 --2- 172.21.15.16:0/2204853036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9bf3ec390 0x55f9bf7bcc30 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:34.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.826+0000 7f2819024740 1 --2- 172.21.15.16:0/2204853036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9bf40d070 0x55f9bf7bd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:34.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.826+0000 7f2819024740 1 --2- 172.21.15.16:0/2204853036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bf3e41b0 0x55f9bf7ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:34.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.826+0000 7f2819024740 1 -- 172.21.15.16:0/2204853036 >> 172.21.15.16:0/2204853036 conn(0x55f9bf401460 msgr2=0x55f9bf7b94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:34.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.826+0000 7f2819024740 1 -- 172.21.15.16:0/2204853036 shutdown_connections 2024-08-31T06:12:34.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.826+0000 7f2819024740 1 -- 172.21.15.16:0/2204853036 wait complete. 2024-08-31T06:12:34.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.826+0000 7f2819024740 1 -- start start 2024-08-31T06:12:34.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.827+0000 7f2819024740 1 Processor -- start 2024-08-31T06:12:34.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.827+0000 7f2819024740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9bf3e41b0 0x55f9bf7c6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:34.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.827+0000 7f2819024740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bf3ec390 0x55f9bf7c1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:34.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.827+0000 7f2819024740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9bf40d070 0x55f9bf7c2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:34.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.827+0000 7f2819024740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f280c1afca0 con 0x55f9bf3ec390 2024-08-31T06:12:34.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.827+0000 7f2819024740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f280c1b1110 con 0x55f9bf3e41b0 2024-08-31T06:12:34.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.827+0000 7f2819024740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f280c1b0370 con 0x55f9bf40d070 2024-08-31T06:12:34.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.827+0000 7f2819024740 10 client.0 _ll_register_callbacks cb 0x7f2804003df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:12:34.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.827+0000 7f2803e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bf3ec390 0x55f9bf7c1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:34.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.827+0000 7f2803e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bf3ec390 0x55f9bf7c1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51560/0 (socket says 172.21.15.16:51560) 2024-08-31T06:12:34.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.827+0000 7f2803e00640 1 -- 172.21.15.16:0/2369584940 learned_addr learned my addr 172.21.15.16:0/2369584940 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:34.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.827+0000 7f280be00640 1 --2- 172.21.15.16:0/2369584940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9bf3e41b0 0x55f9bf7c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:34.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.828+0000 7f280be00640 1 --2- 172.21.15.16:0/2369584940 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9bf40d070 0x55f9bf7c2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:34.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.828+0000 7f2803e00640 1 -- 172.21.15.16:0/2369584940 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9bf40d070 msgr2=0x55f9bf7c2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:34.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.828+0000 7f2803e00640 1 --2- 172.21.15.16:0/2369584940 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9bf40d070 0x55f9bf7c2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:34.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.828+0000 7f2803e00640 1 -- 172.21.15.16:0/2369584940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9bf3e41b0 msgr2=0x55f9bf7c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:34.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.828+0000 7f2803e00640 1 --2- 172.21.15.16:0/2369584940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9bf3e41b0 0x55f9bf7c6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:34.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.828+0000 7f2803e00640 1 -- 172.21.15.16:0/2369584940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9bf7c2890 con 0x55f9bf3ec390 2024-08-31T06:12:34.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.828+0000 7f280be00640 1 --2- 172.21.15.16:0/2369584940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9bf3e41b0 0x55f9bf7c6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:12:34.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.828+0000 7f2803e00640 1 --2- 172.21.15.16:0/2369584940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bf3ec390 0x55f9bf7c1e10 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f27f402ac00 tx=0x7f27f402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6586efad65e0252 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:34.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.828+0000 7f27f9600640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:12:34.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.828+0000 7f27f9600640 1 -- 172.21.15.16:0/2369584940 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f4057060 con 0x55f9bf3ec390 2024-08-31T06:12:34.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.829+0000 7f2819024740 1 -- 172.21.15.16:0/2369584940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f9bf7271a0 con 0x55f9bf3ec390 2024-08-31T06:12:34.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.829+0000 7f27f9600640 1 -- 172.21.15.16:0/2369584940 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f27f4005220 con 0x55f9bf3ec390 2024-08-31T06:12:34.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.829+0000 7f27f9600640 1 -- 172.21.15.16:0/2369584940 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f27f4035050 con 0x55f9bf3ec390 2024-08-31T06:12:34.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.829+0000 7f2819024740 10 client.16584 fetch_fsmap learned FSMap version 38 2024-08-31T06:12:34.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.829+0000 7f2819024740 1 -- 172.21.15.16:0/2369584940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55f9bf83ab60 con 0x55f9bf3ec390 2024-08-31T06:12:34.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.829+0000 7f27f9600640 1 -- 172.21.15.16:0/2369584940 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f9bf7271a0 con 0x55f9bf3ec390 2024-08-31T06:12:34.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.829+0000 7f27f9600640 1 -- 172.21.15.16:0/2369584940 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f27f403d450 con 0x55f9bf3ec390 2024-08-31T06:12:34.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.829+0000 7f2819024740 10 client.16584 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T06:12:34.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.829+0000 7f2819024740 10 client.16584 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:12:34.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.829+0000 7f2819024740 20 client.16584 populate_metadata read hostname 'smithi016' 2024-08-31T06:12:34.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.830+0000 7f2819024740 1 --2- 172.21.15.16:0/2369584940 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f9bf3a0da0 0x55f9bf83d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:34.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.830+0000 7f2819024740 4 client.16584 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:12:34.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.830+0000 7f2819024740 1 -- 172.21.15.16:0/2369584940 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f9bf7248e0 con 0x55f9bf3a0da0 2024-08-31T06:12:34.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.830+0000 7f280be00640 1 --2- 172.21.15.16:0/2369584940 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f9bf3a0da0 0x55f9bf83d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:34.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.830+0000 7f280be00640 1 --2- 172.21.15.16:0/2369584940 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f9bf3a0da0 0x55f9bf83d2d0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66e0a8f033655725 server_cookie=11e68e0ea756cfb4 in_seq=0 out_seq=0 2024-08-31T06:12:34.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.831+0000 7f27f9600640 10 client.16584 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:12:34.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.831+0000 7f27f9600640 1 -- 172.21.15.16:0/2369584940 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f9bf7248e0 con 0x55f9bf3a0da0 2024-08-31T06:12:34.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.831+0000 7f27f9600640 10 client.16584 handle_command_reply: tid=0 2024-08-31T06:12:34.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.831+0000 7f2819024740 1 client.16584 shutdown 2024-08-31T06:12:34.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.831+0000 7f2819024740 1 -- 172.21.15.16:0/2369584940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bf3ec390 msgr2=0x55f9bf7c1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:34.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.831+0000 7f2819024740 1 --2- 172.21.15.16:0/2369584940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bf3ec390 0x55f9bf7c1e10 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f27f402ac00 tx=0x7f27f402a410 comp rx=0 tx=0).stop 2024-08-31T06:12:34.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.831+0000 7f2819024740 1 -- 172.21.15.16:0/2369584940 shutdown_connections 2024-08-31T06:12:34.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.831+0000 7f2819024740 1 --2- 172.21.15.16:0/2369584940 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f9bf3a0da0 0x55f9bf83d2d0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:34.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.831+0000 7f2819024740 1 --2- 172.21.15.16:0/2369584940 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9bf40d070 0x55f9bf7c2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:34.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.831+0000 7f2819024740 1 --2- 172.21.15.16:0/2369584940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9bf3ec390 0x55f9bf7c1e10 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:34.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.831+0000 7f2819024740 1 --2- 172.21.15.16:0/2369584940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9bf3e41b0 0x55f9bf7c6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:34.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.831+0000 7f2819024740 1 -- 172.21.15.16:0/2369584940 >> 172.21.15.16:0/2369584940 conn(0x55f9bf401460 msgr2=0x55f9bf7b0040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:34.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.831+0000 7f27f9600640 0 client.16584 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:12:34.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.831+0000 7f2819024740 1 -- 172.21.15.16:0/2369584940 shutdown_connections 2024-08-31T06:12:34.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.831+0000 7f2819024740 1 -- 172.21.15.16:0/2369584940 wait complete. 2024-08-31T06:12:34.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.832+0000 7f2819024740 20 client.16584 trim_cache size 0 max 16384 2024-08-31T06:12:34.837 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:12:34.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.832+0000 7f2812400640 1 -- 172.21.15.16:0/1763256195 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f27f00b4ca0 msgr2=0x7f27f00b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:34.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.832+0000 7f2812400640 1 --2- 172.21.15.16:0/1763256195 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f27f00b4ca0 0x7f27f00b71b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x55f9bf7b1610 tx=0x7f27fc009700 comp rx=0 tx=0).stop 2024-08-31T06:12:34.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.832+0000 7f2812400640 1 -- 172.21.15.16:0/1763256195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f280c103900 msgr2=0x7f280c0754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:34.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.832+0000 7f2812400640 1 --2- 172.21.15.16:0/1763256195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f280c103900 0x7f280c0754b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f27fc00e9f0 tx=0x7f27fc00eeb0 comp rx=0 tx=0).stop 2024-08-31T06:12:34.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.832+0000 7f2812400640 1 -- 172.21.15.16:0/1763256195 shutdown_connections 2024-08-31T06:12:34.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.832+0000 7f2812400640 1 --2- 172.21.15.16:0/1763256195 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f27f00b4ca0 0x7f27f00b71b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:34.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.832+0000 7f2812400640 1 --2- 172.21.15.16:0/1763256195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f280c109b90 0x7f280c075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:34.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.832+0000 7f2812400640 1 --2- 172.21.15.16:0/1763256195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f280c109240 0x7f280c0759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:34.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.832+0000 7f2812400640 1 --2- 172.21.15.16:0/1763256195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f280c103900 0x7f280c0754b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:34.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.832+0000 7f2812400640 1 -- 172.21.15.16:0/1763256195 >> 172.21.15.16:0/1763256195 conn(0x7f280c07b5f0 msgr2=0x7f280c107330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:34.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.833+0000 7f2812400640 1 -- 172.21.15.16:0/1763256195 shutdown_connections 2024-08-31T06:12:34.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:34.833+0000 7f2812400640 1 -- 172.21.15.16:0/1763256195 wait complete. 2024-08-31T06:12:35.275 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:12:34 smithi184 ceph-mon[22331]: pgmap v524: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 24 MiB/s wr, 598 op/s 2024-08-31T06:12:35.275 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:12:34 smithi184 ceph-mon[22331]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:12:35.324 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:12:34 smithi016 ceph-mon[17194]: pgmap v524: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 24 MiB/s wr, 598 op/s 2024-08-31T06:12:35.324 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:12:34 smithi016 ceph-mon[17194]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:12:35.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:12:34 smithi155 ceph-mon[21742]: pgmap v524: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 24 MiB/s wr, 598 op/s 2024-08-31T06:12:35.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:12:34 smithi155 ceph-mon[21742]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:12:36.627 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 310.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-31T06:12:36.707 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T06:12:36.707 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T06:12:37.090 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:12:37.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:12:36 smithi016 ceph-mon[17194]: pgmap v525: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 23 MiB/s wr, 526 op/s 2024-08-31T06:12:37.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:12:36 smithi155 ceph-mon[21742]: pgmap v525: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 23 MiB/s wr, 526 op/s 2024-08-31T06:12:37.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:12:36 smithi184 ceph-mon[22331]: pgmap v525: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 23 MiB/s wr, 526 op/s 2024-08-31T06:12:37.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.955+0000 7f45cba00640 1 -- 172.21.15.16:0/545283937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45c410f890 msgr2=0x7f45c410fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:37.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.955+0000 7f45cba00640 1 --2- 172.21.15.16:0/545283937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45c410f890 0x7f45c410fca0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f45b0009900 tx=0x7f45b002a410 comp rx=0 tx=0).stop 2024-08-31T06:12:37.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.955+0000 7f45cba00640 1 -- 172.21.15.16:0/545283937 shutdown_connections 2024-08-31T06:12:37.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.955+0000 7f45cba00640 1 --2- 172.21.15.16:0/545283937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45c41101e0 0x7f45c4116a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:37.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.955+0000 7f45cba00640 1 --2- 172.21.15.16:0/545283937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45c410f890 0x7f45c410fca0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:37.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.955+0000 7f45cba00640 1 --2- 172.21.15.16:0/545283937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45c4109f50 0x7f45c410a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:37.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.955+0000 7f45cba00640 1 -- 172.21.15.16:0/545283937 >> 172.21.15.16:0/545283937 conn(0x7f45c4077e80 msgr2=0x7f45c4107f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:37.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.955+0000 7f45cba00640 1 -- 172.21.15.16:0/545283937 shutdown_connections 2024-08-31T06:12:37.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.955+0000 7f45cba00640 1 -- 172.21.15.16:0/545283937 wait complete. 2024-08-31T06:12:37.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.956+0000 7f45cba00640 1 Processor -- start 2024-08-31T06:12:37.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.956+0000 7f45cba00640 1 -- start start 2024-08-31T06:12:37.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.956+0000 7f45cba00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45c4109f50 0x7f45c41a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:37.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.957+0000 7f45cba00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45c410f890 0x7f45c41a0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:37.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.957+0000 7f45cba00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45c41101e0 0x7f45c41a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:37.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.957+0000 7f45cba00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f45c4119ea0 con 0x7f45c4109f50 2024-08-31T06:12:37.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.957+0000 7f45cba00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f45c4119d20 con 0x7f45c410f890 2024-08-31T06:12:37.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.957+0000 7f45cba00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f45c411a020 con 0x7f45c41101e0 2024-08-31T06:12:37.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.957+0000 7f45c9000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45c4109f50 0x7f45c41a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:37.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.957+0000 7f45c9000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45c4109f50 0x7f45c41a0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33314/0 (socket says 172.21.15.16:33314) 2024-08-31T06:12:37.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.957+0000 7f45c9000640 1 -- 172.21.15.16:0/320095062 learned_addr learned my addr 172.21.15.16:0/320095062 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:37.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.957+0000 7f45c3e00640 1 --2- 172.21.15.16:0/320095062 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45c410f890 0x7f45c41a0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:37.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.957+0000 7f45c9a00640 1 --2- 172.21.15.16:0/320095062 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45c41101e0 0x7f45c41a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:37.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.957+0000 7f45c9000640 1 -- 172.21.15.16:0/320095062 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45c41101e0 msgr2=0x7f45c41a6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:37.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.957+0000 7f45c9000640 1 --2- 172.21.15.16:0/320095062 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45c41101e0 0x7f45c41a6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:37.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.957+0000 7f45c9000640 1 -- 172.21.15.16:0/320095062 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45c410f890 msgr2=0x7f45c41a0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:37.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.957+0000 7f45c9000640 1 --2- 172.21.15.16:0/320095062 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45c410f890 0x7f45c41a0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:37.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.957+0000 7f45c9000640 1 -- 172.21.15.16:0/320095062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c41a68a0 con 0x7f45c4109f50 2024-08-31T06:12:37.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.958+0000 7f45c9000640 1 --2- 172.21.15.16:0/320095062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45c4109f50 0x7f45c41a0930 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f45b800e9f0 tx=0x7f45b800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:37.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.958+0000 7f45c1600640 1 -- 172.21.15.16:0/320095062 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b800cd80 con 0x7f45c4109f50 2024-08-31T06:12:37.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.958+0000 7f45c1600640 1 -- 172.21.15.16:0/320095062 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45b80108e0 con 0x7f45c4109f50 2024-08-31T06:12:37.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.958+0000 7f45cba00640 1 -- 172.21.15.16:0/320095062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45c41a6b90 con 0x7f45c4109f50 2024-08-31T06:12:37.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.958+0000 7f45c1600640 1 -- 172.21.15.16:0/320095062 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b8018980 con 0x7f45c4109f50 2024-08-31T06:12:37.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.958+0000 7f45cba00640 1 -- 172.21.15.16:0/320095062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45c41ad2f0 con 0x7f45c4109f50 2024-08-31T06:12:37.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.962+0000 7f45d2631740 1 Processor -- start 2024-08-31T06:12:37.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.962+0000 7f45d2631740 1 -- start start 2024-08-31T06:12:37.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.963+0000 7f45d2631740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4d59a8350 0x55c4d5d4e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:37.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.963+0000 7f45d2631740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4d597f490 0x55c4d5d4eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:37.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.963+0000 7f45d2631740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4d5987670 0x55c4d5d4f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:37.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.963+0000 7f45d2631740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c4d5cc4750 con 0x55c4d5987670 2024-08-31T06:12:37.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.963+0000 7f45d2631740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c4d5cb3ce0 con 0x55c4d597f490 2024-08-31T06:12:37.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.963+0000 7f45d2631740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c4d5cb7c90 con 0x55c4d59a8350 2024-08-31T06:12:37.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.963+0000 7f45c3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4d5987670 0x55c4d5d4f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:37.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.963+0000 7f45c9a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4d59a8350 0x55c4d5d4e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:37.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.963+0000 7f45c9a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4d59a8350 0x55c4d5d4e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:47792/0 (socket says 172.21.15.16:47792) 2024-08-31T06:12:37.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.963+0000 7f45c3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4d5987670 0x55c4d5d4f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33322/0 (socket says 172.21.15.16:33322) 2024-08-31T06:12:37.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.963+0000 7f45c9a00640 1 -- 172.21.15.16:0/1400029580 learned_addr learned my addr 172.21.15.16:0/1400029580 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:37.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.963+0000 7f45c1600640 1 -- 172.21.15.16:0/320095062 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f45b8010430 con 0x7f45c4109f50 2024-08-31T06:12:37.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.964+0000 7f45c9000640 1 --2- 172.21.15.16:0/1400029580 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4d597f490 0x55c4d5d4eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:37.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.964+0000 7f45c9a00640 1 -- 172.21.15.16:0/1400029580 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4d597f490 msgr2=0x55c4d5d4eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:37.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.964+0000 7f45c9a00640 1 --2- 172.21.15.16:0/1400029580 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4d597f490 0x55c4d5d4eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:37.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.964+0000 7f45c9a00640 1 -- 172.21.15.16:0/1400029580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4d5987670 msgr2=0x55c4d5d4f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:37.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.964+0000 7f45c9a00640 1 --2- 172.21.15.16:0/1400029580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4d5987670 0x55c4d5d4f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:37.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.964+0000 7f45c9a00640 1 -- 172.21.15.16:0/1400029580 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4d5a13930 con 0x55c4d59a8350 2024-08-31T06:12:37.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.964+0000 7f45c9000640 1 --2- 172.21.15.16:0/1400029580 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4d597f490 0x55c4d5d4eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:12:37.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.964+0000 7f45c3e00640 1 --2- 172.21.15.16:0/1400029580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4d5987670 0x55c4d5d4f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:12:37.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.964+0000 7f45c9a00640 1 --2- 172.21.15.16:0/1400029580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4d59a8350 0x55c4d5d4e600 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f45bc009f70 tx=0x7f45bc010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b268d79938ab0c5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:37.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.965+0000 7f45b6000640 1 -- 172.21.15.16:0/1400029580 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45bc016070 con 0x55c4d59a8350 2024-08-31T06:12:37.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.965+0000 7f45b6000640 1 -- 172.21.15.16:0/1400029580 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45bc004d20 con 0x55c4d59a8350 2024-08-31T06:12:37.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.965+0000 7f45b6000640 1 -- 172.21.15.16:0/1400029580 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45bc012790 con 0x55c4d59a8350 2024-08-31T06:12:37.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.965+0000 7f45c1600640 1 --2- 172.21.15.16:0/320095062 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f45ac0b4d20 0x7f45ac0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:37.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.965+0000 7f45c1600640 1 -- 172.21.15.16:0/320095062 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f45b80e57d0 con 0x7f45c4109f50 2024-08-31T06:12:37.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.965+0000 7f45c9000640 1 --2- 172.21.15.16:0/320095062 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f45ac0b4d20 0x7f45ac0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:37.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.966+0000 7f45d2631740 1 -- 172.21.15.16:0/1400029580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4d59a8350 msgr2=0x55c4d5d4e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:37.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.966+0000 7f45d2631740 1 --2- 172.21.15.16:0/1400029580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4d59a8350 0x55c4d5d4e600 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f45bc009f70 tx=0x7f45bc010800 comp rx=0 tx=0).stop 2024-08-31T06:12:37.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.966+0000 7f45d2631740 1 -- 172.21.15.16:0/1400029580 shutdown_connections 2024-08-31T06:12:37.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.966+0000 7f45d2631740 1 --2- 172.21.15.16:0/1400029580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4d5987670 0x55c4d5d4f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:37.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.966+0000 7f45d2631740 1 --2- 172.21.15.16:0/1400029580 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4d597f490 0x55c4d5d4eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:37.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.966+0000 7f45d2631740 1 --2- 172.21.15.16:0/1400029580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4d59a8350 0x55c4d5d4e600 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:37.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.966+0000 7f45d2631740 1 -- 172.21.15.16:0/1400029580 >> 172.21.15.16:0/1400029580 conn(0x55c4d599c740 msgr2=0x55c4d5d4c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:37.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.966+0000 7f45d2631740 1 -- 172.21.15.16:0/1400029580 shutdown_connections 2024-08-31T06:12:37.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.966+0000 7f45c9000640 1 --2- 172.21.15.16:0/320095062 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f45ac0b4d20 0x7f45ac0b7230 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x55c4d5d4d280 tx=0x7f45b8000bc0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:37.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.966+0000 7f45d2631740 1 -- 172.21.15.16:0/1400029580 wait complete. 2024-08-31T06:12:37.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.967+0000 7f45d2631740 1 -- start start 2024-08-31T06:12:37.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.967+0000 7f45d2631740 1 Processor -- start 2024-08-31T06:12:37.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.967+0000 7f45d2631740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4d597f490 0x55c4d5d5a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:37.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.967+0000 7f45d2631740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4d5987670 0x55c4d5d5a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:37.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.968+0000 7f45d2631740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4d59a8350 0x55c4d5d660d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.968+0000 7f45d2631740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c4d5b16890 con 0x55c4d597f490 2024-08-31T06:12:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.968+0000 7f45d2631740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c4d5c3acd0 con 0x55c4d5987670 2024-08-31T06:12:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.968+0000 7f45d2631740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c4d5d5cde0 con 0x55c4d59a8350 2024-08-31T06:12:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.968+0000 7f45d2631740 10 client.0 _ll_register_callbacks cb 0x55c4d5caeef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:12:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.968+0000 7f45c9a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4d597f490 0x55c4d5d5a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.968+0000 7f45c3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4d5987670 0x55c4d5d5a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.968+0000 7f45c9a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4d597f490 0x55c4d5d5a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33338/0 (socket says 172.21.15.16:33338) 2024-08-31T06:12:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.968+0000 7f45c9a00640 1 -- 172.21.15.16:0/1265382822 learned_addr learned my addr 172.21.15.16:0/1265382822 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.968+0000 7f45c3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4d5987670 0x55c4d5d5a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:36176/0 (socket says 172.21.15.16:36176) 2024-08-31T06:12:37.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.968+0000 7f45c9a00640 1 --2- 172.21.15.16:0/1265382822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4d59a8350 0x55c4d5d660d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:37.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.968+0000 7f45c3e00640 1 -- 172.21.15.16:0/1265382822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4d59a8350 msgr2=0x55c4d5d660d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:37.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.968+0000 7f45c3e00640 1 --2- 172.21.15.16:0/1265382822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4d59a8350 0x55c4d5d660d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:37.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.968+0000 7f45c3e00640 1 -- 172.21.15.16:0/1265382822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4d597f490 msgr2=0x55c4d5d5a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:37.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.968+0000 7f45c3e00640 1 --2- 172.21.15.16:0/1265382822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4d597f490 0x55c4d5d5a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:37.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.968+0000 7f45c3e00640 1 -- 172.21.15.16:0/1265382822 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4d5c841b0 con 0x55c4d5987670 2024-08-31T06:12:37.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.969+0000 7f45c9a00640 1 --2- 172.21.15.16:0/1265382822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4d597f490 0x55c4d5d5a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:12:37.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.969+0000 7f45c3e00640 1 --2- 172.21.15.16:0/1265382822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4d5987670 0x55c4d5d5a6e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f45b002a930 tx=0x7f45b0032e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7235059a253329e8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:37.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.969+0000 7f45aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:12:37.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.969+0000 7f45aaa00640 1 -- 172.21.15.16:0/1265382822 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b0035040 con 0x55c4d5987670 2024-08-31T06:12:37.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.969+0000 7f45d2631740 1 -- 172.21.15.16:0/1265382822 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45bc004d20 con 0x55c4d5987670 2024-08-31T06:12:37.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.969+0000 7f45aaa00640 1 -- 172.21.15.16:0/1265382822 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f45b0031760 con 0x55c4d5987670 2024-08-31T06:12:37.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.970+0000 7f45aaa00640 1 -- 172.21.15.16:0/1265382822 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f45b0004040 con 0x55c4d5987670 2024-08-31T06:12:37.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.970+0000 7f45d2631740 10 client.26807 fetch_fsmap learned FSMap version 38 2024-08-31T06:12:37.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.970+0000 7f45d2631740 1 -- 172.21.15.16:0/1265382822 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55c4d5d68db0 con 0x55c4d5987670 2024-08-31T06:12:37.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.970+0000 7f45aaa00640 1 -- 172.21.15.16:0/1265382822 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45bc004d20 con 0x55c4d5987670 2024-08-31T06:12:37.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.970+0000 7f45aaa00640 1 -- 172.21.15.16:0/1265382822 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f45b00047e0 con 0x55c4d5987670 2024-08-31T06:12:37.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.970+0000 7f45d2631740 10 client.26807 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T06:12:37.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.970+0000 7f45d2631740 10 client.26807 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:12:37.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.970+0000 7f45d2631740 20 client.26807 populate_metadata read hostname 'smithi016' 2024-08-31T06:12:37.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.970+0000 7f45d2631740 1 --2- 172.21.15.16:0/1265382822 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55c4d5966cd0 0x55c4d5d6c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:37.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.970+0000 7f45d2631740 4 client.26807 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:12:37.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.970+0000 7f45d2631740 1 -- 172.21.15.16:0/1265382822 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4d5cc66a0 con 0x55c4d5966cd0 2024-08-31T06:12:37.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.971+0000 7f45c9a00640 1 --2- 172.21.15.16:0/1265382822 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55c4d5966cd0 0x55c4d5d6c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:37.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.971+0000 7f45c9a00640 1 --2- 172.21.15.16:0/1265382822 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55c4d5966cd0 0x55c4d5d6c1f0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d33caa2a9aa058f3 server_cookie=e20190d021934e6 in_seq=0 out_seq=0 2024-08-31T06:12:37.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.971+0000 7f45aaa00640 10 client.26807 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:12:37.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.972+0000 7f45aaa00640 1 -- 172.21.15.16:0/1265382822 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c4d5cc66a0 con 0x55c4d5966cd0 2024-08-31T06:12:37.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.972+0000 7f45aaa00640 10 client.26807 handle_command_reply: tid=0 2024-08-31T06:12:37.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.972+0000 7f45d2631740 1 client.26807 shutdown 2024-08-31T06:12:37.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.973+0000 7f45d2631740 1 -- 172.21.15.16:0/1265382822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4d5987670 msgr2=0x55c4d5d5a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:37.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.973+0000 7f45d2631740 1 --2- 172.21.15.16:0/1265382822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4d5987670 0x55c4d5d5a6e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f45b002a930 tx=0x7f45b0032e90 comp rx=0 tx=0).stop 2024-08-31T06:12:37.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.973+0000 7f45d2631740 1 -- 172.21.15.16:0/1265382822 shutdown_connections 2024-08-31T06:12:37.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.973+0000 7f45d2631740 1 --2- 172.21.15.16:0/1265382822 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55c4d5966cd0 0x55c4d5d6c1f0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:37.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.973+0000 7f45d2631740 1 --2- 172.21.15.16:0/1265382822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4d59a8350 0x55c4d5d660d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:37.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.973+0000 7f45d2631740 1 --2- 172.21.15.16:0/1265382822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4d5987670 0x55c4d5d5a6e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:37.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.973+0000 7f45d2631740 1 --2- 172.21.15.16:0/1265382822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4d597f490 0x55c4d5d5a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:37.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.973+0000 7f45aaa00640 0 client.26807 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:12:37.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.973+0000 7f45d2631740 1 -- 172.21.15.16:0/1265382822 >> 172.21.15.16:0/1265382822 conn(0x55c4d599c740 msgr2=0x55c4d5d51790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:37.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.973+0000 7f45d2631740 1 -- 172.21.15.16:0/1265382822 shutdown_connections 2024-08-31T06:12:37.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.973+0000 7f45d2631740 1 -- 172.21.15.16:0/1265382822 wait complete. 2024-08-31T06:12:37.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.973+0000 7f45d2631740 20 client.26807 trim_cache size 0 max 16384 2024-08-31T06:12:37.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.991+0000 7f45d2631740 1 Processor -- start 2024-08-31T06:12:37.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.992+0000 7f45d2631740 1 -- start start 2024-08-31T06:12:37.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.992+0000 7f45d2631740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4d597f490 0x55c4d5d4fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:37.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.992+0000 7f45d2631740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4d5987670 0x55c4d5d53860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:37.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.992+0000 7f45d2631740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4d59a8350 0x55c4d5d53da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:37.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.992+0000 7f45d2631740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f45c41b83f0 con 0x55c4d597f490 2024-08-31T06:12:37.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.992+0000 7f45d2631740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f45c41b5c30 con 0x55c4d59a8350 2024-08-31T06:12:37.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.992+0000 7f45d2631740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f45c41b3a80 con 0x55c4d5987670 2024-08-31T06:12:37.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.992+0000 7f45c9a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4d597f490 0x55c4d5d4fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:37.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.992+0000 7f45c9a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4d597f490 0x55c4d5d4fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33350/0 (socket says 172.21.15.16:33350) 2024-08-31T06:12:37.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.992+0000 7f45c9a00640 1 -- 172.21.15.16:0/299238477 learned_addr learned my addr 172.21.15.16:0/299238477 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:37.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.992+0000 7f45c3e00640 1 --2- 172.21.15.16:0/299238477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4d5987670 0x55c4d5d53860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:37.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.993+0000 7f45c9a00640 1 --2- 172.21.15.16:0/299238477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4d59a8350 0x55c4d5d53da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:37.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.993+0000 7f45c9a00640 1 -- 172.21.15.16:0/299238477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4d5987670 msgr2=0x55c4d5d53860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:37.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.993+0000 7f45c9a00640 1 --2- 172.21.15.16:0/299238477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4d5987670 0x55c4d5d53860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:37.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.993+0000 7f45c9a00640 1 -- 172.21.15.16:0/299238477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4d59a8350 msgr2=0x55c4d5d53da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:37.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.993+0000 7f45c9a00640 1 --2- 172.21.15.16:0/299238477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4d59a8350 0x55c4d5d53da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:37.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.993+0000 7f45c9a00640 1 -- 172.21.15.16:0/299238477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4d5d58f00 con 0x55c4d597f490 2024-08-31T06:12:37.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.993+0000 7f45c9a00640 1 --2- 172.21.15.16:0/299238477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4d597f490 0x55c4d5d4fff0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f45bc000f60 tx=0x7f45bc009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daefd3b802af563 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:37.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.993+0000 7f45b5600640 1 -- 172.21.15.16:0/299238477 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45bc016070 con 0x55c4d597f490 2024-08-31T06:12:37.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.993+0000 7f45b5600640 1 -- 172.21.15.16:0/299238477 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45bc025040 con 0x55c4d597f490 2024-08-31T06:12:37.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.993+0000 7f45b5600640 1 -- 172.21.15.16:0/299238477 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45bc012450 con 0x55c4d597f490 2024-08-31T06:12:37.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.994+0000 7f45d2631740 1 -- 172.21.15.16:0/299238477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4d597f490 msgr2=0x55c4d5d4fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:37.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.994+0000 7f45d2631740 1 --2- 172.21.15.16:0/299238477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4d597f490 0x55c4d5d4fff0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f45bc000f60 tx=0x7f45bc009800 comp rx=0 tx=0).stop 2024-08-31T06:12:37.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.994+0000 7f45d2631740 1 -- 172.21.15.16:0/299238477 shutdown_connections 2024-08-31T06:12:37.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.994+0000 7f45d2631740 1 --2- 172.21.15.16:0/299238477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4d5987670 0x55c4d5d53860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:37.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.994+0000 7f45d2631740 1 --2- 172.21.15.16:0/299238477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4d59a8350 0x55c4d5d53da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:37.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.994+0000 7f45d2631740 1 --2- 172.21.15.16:0/299238477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4d597f490 0x55c4d5d4fff0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:37.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.994+0000 7f45d2631740 1 -- 172.21.15.16:0/299238477 >> 172.21.15.16:0/299238477 conn(0x55c4d599c740 msgr2=0x55c4d5d55150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:37.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.994+0000 7f45d2631740 1 -- 172.21.15.16:0/299238477 shutdown_connections 2024-08-31T06:12:37.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.995+0000 7f45d2631740 1 -- 172.21.15.16:0/299238477 wait complete. 2024-08-31T06:12:37.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.995+0000 7f45d2631740 1 -- start start 2024-08-31T06:12:37.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.995+0000 7f45d2631740 1 Processor -- start 2024-08-31T06:12:38.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.996+0000 7f45d2631740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4d597f490 0x55c4d5d5d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:38.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.996+0000 7f45d2631740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4d5987670 0x55c4d5d5d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:38.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.996+0000 7f45d2631740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4d59a8350 0x55c4d5dd6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:38.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.996+0000 7f45d2631740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f45c41b40f0 con 0x55c4d597f490 2024-08-31T06:12:38.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.996+0000 7f45d2631740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f45c41b5560 con 0x55c4d5987670 2024-08-31T06:12:38.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.996+0000 7f45d2631740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f45c41b47c0 con 0x55c4d59a8350 2024-08-31T06:12:38.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.996+0000 7f45d2631740 10 client.0 _ll_register_callbacks cb 0x7f45b0009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:12:38.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.996+0000 7f45c3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4d5987670 0x55c4d5d5d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:38.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.996+0000 7f45c3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4d5987670 0x55c4d5d5d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:36200/0 (socket says 172.21.15.16:36200) 2024-08-31T06:12:38.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.996+0000 7f45c3e00640 1 -- 172.21.15.16:0/8304013 learned_addr learned my addr 172.21.15.16:0/8304013 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:38.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.996+0000 7f45c9a00640 1 --2- 172.21.15.16:0/8304013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4d597f490 0x55c4d5d5d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:38.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.997+0000 7f45c9a00640 1 --2- 172.21.15.16:0/8304013 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4d59a8350 0x55c4d5dd6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:38.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.997+0000 7f45c3e00640 1 -- 172.21.15.16:0/8304013 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4d59a8350 msgr2=0x55c4d5dd6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:38.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.997+0000 7f45c3e00640 1 --2- 172.21.15.16:0/8304013 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4d59a8350 0x55c4d5dd6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:38.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.997+0000 7f45c3e00640 1 -- 172.21.15.16:0/8304013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4d597f490 msgr2=0x55c4d5d5d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:38.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.997+0000 7f45c3e00640 1 --2- 172.21.15.16:0/8304013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4d597f490 0x55c4d5d5d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:38.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.997+0000 7f45c3e00640 1 -- 172.21.15.16:0/8304013 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4d5d5e200 con 0x55c4d5987670 2024-08-31T06:12:38.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.997+0000 7f45c9a00640 1 --2- 172.21.15.16:0/8304013 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4d59a8350 0x55c4d5dd6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:12:38.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.997+0000 7f45c3e00640 1 --2- 172.21.15.16:0/8304013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4d5987670 0x55c4d5d5d960 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f45b0006010 tx=0x7f45b00048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=540a4e0c90ec51ea server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:38.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:37.997+0000 7f45aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:12:38.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:38.000+0000 7f45aaa00640 1 -- 172.21.15.16:0/8304013 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b0035040 con 0x55c4d5987670 2024-08-31T06:12:38.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:38.000+0000 7f45aaa00640 1 -- 172.21.15.16:0/8304013 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f45b0031560 con 0x55c4d5987670 2024-08-31T06:12:38.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:38.000+0000 7f45d2631740 1 -- 172.21.15.16:0/8304013 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45bc025040 con 0x55c4d5987670 2024-08-31T06:12:38.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:38.001+0000 7f45aaa00640 1 -- 172.21.15.16:0/8304013 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f45b0060050 con 0x55c4d5987670 2024-08-31T06:12:38.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:38.001+0000 7f45d2631740 10 client.26813 fetch_fsmap learned FSMap version 38 2024-08-31T06:12:38.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:38.001+0000 7f45d2631740 1 -- 172.21.15.16:0/8304013 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55c4d5dd81f0 con 0x55c4d5987670 2024-08-31T06:12:38.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:38.001+0000 7f45aaa00640 1 -- 172.21.15.16:0/8304013 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45bc025040 con 0x55c4d5987670 2024-08-31T06:12:38.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:38.001+0000 7f45aaa00640 1 -- 172.21.15.16:0/8304013 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f45b0004dd0 con 0x55c4d5987670 2024-08-31T06:12:38.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:38.001+0000 7f45d2631740 10 client.26813 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T06:12:38.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:38.001+0000 7f45d2631740 10 client.26813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:12:38.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:38.001+0000 7f45d2631740 20 client.26813 populate_metadata read hostname 'smithi016' 2024-08-31T06:12:38.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:38.001+0000 7f45d2631740 1 --2- 172.21.15.16:0/8304013 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55c4d593c080 0x55c4d5ddaa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:38.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:38.001+0000 7f45d2631740 4 client.26813 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T06:12:38.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:38.001+0000 7f45d2631740 1 -- 172.21.15.16:0/8304013 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c4d5cc2140 con 0x55c4d593c080 2024-08-31T06:12:38.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:38.002+0000 7f45c9a00640 1 --2- 172.21.15.16:0/8304013 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55c4d593c080 0x55c4d5ddaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:38.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:38.002+0000 7f45c9a00640 1 --2- 172.21.15.16:0/8304013 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55c4d593c080 0x55c4d5ddaa90 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9a8c39586e0d558 server_cookie=8d5d262e0778efd8 in_seq=0 out_seq=0 2024-08-31T06:12:38.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:38.002+0000 7f45aaa00640 10 client.26813 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:12:38.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:38.004+0000 7f45aaa00640 1 -- 172.21.15.16:0/8304013 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c4d5cc2140 con 0x55c4d593c080 2024-08-31T06:12:38.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:38.004+0000 7f45aaa00640 10 client.26813 handle_command_reply: tid=0 2024-08-31T06:12:38.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:38.004+0000 7f45d2631740 1 client.26813 shutdown 2024-08-31T06:12:38.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:38.004+0000 7f45d2631740 1 -- 172.21.15.16:0/8304013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4d5987670 msgr2=0x55c4d5d5d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:38.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:38.004+0000 7f45d2631740 1 --2- 172.21.15.16:0/8304013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4d5987670 0x55c4d5d5d960 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f45b0006010 tx=0x7f45b00048f0 comp rx=0 tx=0).stop 2024-08-31T06:12:38.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:38.004+0000 7f45d2631740 1 -- 172.21.15.16:0/8304013 shutdown_connections 2024-08-31T06:12:38.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:38.004+0000 7f45d2631740 1 --2- 172.21.15.16:0/8304013 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55c4d593c080 0x55c4d5ddaa90 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:38.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:38.004+0000 7f45d2631740 1 --2- 172.21.15.16:0/8304013 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4d59a8350 0x55c4d5dd6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:38.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:38.004+0000 7f45d2631740 1 --2- 172.21.15.16:0/8304013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4d5987670 0x55c4d5d5d960 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:38.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:38.004+0000 7f45aaa00640 0 client.26813 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:12:38.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:38.004+0000 7f45d2631740 1 --2- 172.21.15.16:0/8304013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4d597f490 0x55c4d5d5d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:38.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:38.004+0000 7f45d2631740 1 -- 172.21.15.16:0/8304013 >> 172.21.15.16:0/8304013 conn(0x55c4d599c740 msgr2=0x55c4d5d55150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:38.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:38.004+0000 7f45d2631740 1 -- 172.21.15.16:0/8304013 shutdown_connections 2024-08-31T06:12:38.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:38.004+0000 7f45d2631740 1 -- 172.21.15.16:0/8304013 wait complete. 2024-08-31T06:12:38.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:38.005+0000 7f45d2631740 20 client.26813 trim_cache size 0 max 16384 2024-08-31T06:12:38.008 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:12:38.008 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-08-31T06:12:38.008 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "6a03046e-cdf4-4a14-b9bd-b560b8968467", 2024-08-31T06:12:38.008 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-08-31T06:12:38.008 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:12:38.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:38.005+0000 7f45cba00640 1 -- 172.21.15.16:0/320095062 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f45ac0b4d20 msgr2=0x7f45ac0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:38.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:38.005+0000 7f45cba00640 1 --2- 172.21.15.16:0/320095062 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f45ac0b4d20 0x7f45ac0b7230 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x55c4d5d4d280 tx=0x7f45b8000bc0 comp rx=0 tx=0).stop 2024-08-31T06:12:38.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:38.005+0000 7f45cba00640 1 -- 172.21.15.16:0/320095062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45c4109f50 msgr2=0x7f45c41a0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:38.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:38.005+0000 7f45cba00640 1 --2- 172.21.15.16:0/320095062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45c4109f50 0x7f45c41a0930 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f45b800e9f0 tx=0x7f45b800eeb0 comp rx=0 tx=0).stop 2024-08-31T06:12:38.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:38.005+0000 7f45cba00640 1 -- 172.21.15.16:0/320095062 shutdown_connections 2024-08-31T06:12:38.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:38.005+0000 7f45cba00640 1 --2- 172.21.15.16:0/320095062 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f45ac0b4d20 0x7f45ac0b7230 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:38.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:38.005+0000 7f45cba00640 1 --2- 172.21.15.16:0/320095062 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45c41101e0 0x7f45c41a6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:38.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:38.005+0000 7f45cba00640 1 --2- 172.21.15.16:0/320095062 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45c410f890 0x7f45c41a0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:38.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:38.005+0000 7f45cba00640 1 --2- 172.21.15.16:0/320095062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45c4109f50 0x7f45c41a0930 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:38.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:38.005+0000 7f45cba00640 1 -- 172.21.15.16:0/320095062 >> 172.21.15.16:0/320095062 conn(0x7f45c4077e80 msgr2=0x7f45c4108370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:38.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:38.006+0000 7f45cba00640 1 -- 172.21.15.16:0/320095062 shutdown_connections 2024-08-31T06:12:38.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:38.006+0000 7f45cba00640 1 -- 172.21.15.16:0/320095062 wait complete. 2024-08-31T06:12:38.927 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T06:12:38.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.942+0000 7fdadb000640 1 -- 172.21.15.16:0/383610824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdad410b390 msgr2=0x7fdad410b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:38.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.942+0000 7fdadb000640 1 --2- 172.21.15.16:0/383610824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdad410b390 0x7fdad410b7a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fdac0009900 tx=0x7fdac002a3f0 comp rx=0 tx=0).stop 2024-08-31T06:12:38.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.942+0000 7fdadb000640 1 -- 172.21.15.16:0/383610824 shutdown_connections 2024-08-31T06:12:38.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.942+0000 7fdadb000640 1 --2- 172.21.15.16:0/383610824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdad410bce0 0x7fdad4112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:38.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.942+0000 7fdadb000640 1 --2- 172.21.15.16:0/383610824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdad410b390 0x7fdad410b7a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:38.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.942+0000 7fdadb000640 1 --2- 172.21.15.16:0/383610824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdad4105a60 0x7fdad4105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:38.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.942+0000 7fdadb000640 1 -- 172.21.15.16:0/383610824 >> 172.21.15.16:0/383610824 conn(0x7fdad4101480 msgr2=0x7fdad41038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:38.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.943+0000 7fdadb000640 1 -- 172.21.15.16:0/383610824 shutdown_connections 2024-08-31T06:12:38.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.943+0000 7fdadb000640 1 -- 172.21.15.16:0/383610824 wait complete. 2024-08-31T06:12:38.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.943+0000 7fdadb000640 1 Processor -- start 2024-08-31T06:12:38.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.944+0000 7fdadb000640 1 -- start start 2024-08-31T06:12:38.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.944+0000 7fdadb000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdad4105a60 0x7fdad419c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:38.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.944+0000 7fdadb000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdad410b390 0x7fdad419c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.944+0000 7fdadb000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdad410bce0 0x7fdad41a1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.944+0000 7fdadb000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdad41159a0 con 0x7fdad4105a60 2024-08-31T06:12:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.944+0000 7fdadb000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdad4115820 con 0x7fdad410b390 2024-08-31T06:12:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.944+0000 7fdadb000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdad4115b20 con 0x7fdad410bce0 2024-08-31T06:12:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.944+0000 7fdad3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdad4105a60 0x7fdad419c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.944+0000 7fdad3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdad4105a60 0x7fdad419c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33388/0 (socket says 172.21.15.16:33388) 2024-08-31T06:12:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.944+0000 7fdad3e00640 1 -- 172.21.15.16:0/3243595528 learned_addr learned my addr 172.21.15.16:0/3243595528 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.944+0000 7fdad3400640 1 --2- 172.21.15.16:0/3243595528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdad410b390 0x7fdad419c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.945+0000 7fdad9000640 1 --2- 172.21.15.16:0/3243595528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdad410bce0 0x7fdad41a1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.945+0000 7fdad3e00640 1 -- 172.21.15.16:0/3243595528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdad410bce0 msgr2=0x7fdad41a1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.945+0000 7fdad3e00640 1 --2- 172.21.15.16:0/3243595528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdad410bce0 0x7fdad41a1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.945+0000 7fdad3e00640 1 -- 172.21.15.16:0/3243595528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdad410b390 msgr2=0x7fdad419c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.945+0000 7fdad3e00640 1 --2- 172.21.15.16:0/3243595528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdad410b390 0x7fdad419c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.945+0000 7fdad3e00640 1 -- 172.21.15.16:0/3243595528 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad41a2330 con 0x7fdad4105a60 2024-08-31T06:12:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.945+0000 7fdad3400640 1 --2- 172.21.15.16:0/3243595528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdad410b390 0x7fdad419c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:12:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.945+0000 7fdad9000640 1 --2- 172.21.15.16:0/3243595528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdad410bce0 0x7fdad41a1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:12:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.945+0000 7fdad3e00640 1 --2- 172.21.15.16:0/3243595528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdad4105a60 0x7fdad419c400 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fdac400e9f0 tx=0x7fdac400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.945+0000 7fdad0c00640 1 -- 172.21.15.16:0/3243595528 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac400cd80 con 0x7fdad4105a60 2024-08-31T06:12:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.945+0000 7fdad0c00640 1 -- 172.21.15.16:0/3243595528 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdac40054d0 con 0x7fdad4105a60 2024-08-31T06:12:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.945+0000 7fdadb000640 1 -- 172.21.15.16:0/3243595528 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdad41a2620 con 0x7fdad4105a60 2024-08-31T06:12:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.945+0000 7fdad0c00640 1 -- 172.21.15.16:0/3243595528 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4010690 con 0x7fdad4105a60 2024-08-31T06:12:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.946+0000 7fdadb000640 1 -- 172.21.15.16:0/3243595528 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdad41a8db0 con 0x7fdad4105a60 2024-08-31T06:12:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.949+0000 7fdae2251740 1 Processor -- start 2024-08-31T06:12:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.950+0000 7fdae2251740 1 -- start start 2024-08-31T06:12:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.950+0000 7fdae2251740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56280891d1b0 0x562808cc0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.950+0000 7fdae2251740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5628088f42f0 0x562808cc0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.950+0000 7fdae2251740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628088fc4d0 0x562808cc13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.950+0000 7fdae2251740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562808c35f70 con 0x5628088fc4d0 2024-08-31T06:12:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.950+0000 7fdae2251740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562808c25500 con 0x56280891d1b0 2024-08-31T06:12:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.950+0000 7fdae2251740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562808c294b0 con 0x5628088f42f0 2024-08-31T06:12:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.951+0000 7fdad3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628088fc4d0 0x562808cc13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.951+0000 7fdad9000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56280891d1b0 0x562808cc0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.951+0000 7fdad9000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56280891d1b0 0x562808cc0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:36238/0 (socket says 172.21.15.16:36238) 2024-08-31T06:12:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.951+0000 7fdad9000640 1 -- 172.21.15.16:0/3844887184 learned_addr learned my addr 172.21.15.16:0/3844887184 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.951+0000 7fdad3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628088fc4d0 0x562808cc13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33398/0 (socket says 172.21.15.16:33398) 2024-08-31T06:12:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.951+0000 7fdad9000640 1 -- 172.21.15.16:0/3844887184 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5628088f42f0 msgr2=0x562808cc0e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:12:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.951+0000 7fdad9000640 1 --2- 172.21.15.16:0/3844887184 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5628088f42f0 0x562808cc0e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.951+0000 7fdad9000640 1 -- 172.21.15.16:0/3844887184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628088fc4d0 msgr2=0x562808cc13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.951+0000 7fdad9000640 1 --2- 172.21.15.16:0/3844887184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628088fc4d0 0x562808cc13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.951+0000 7fdad9000640 1 -- 172.21.15.16:0/3844887184 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562808bdfc10 con 0x56280891d1b0 2024-08-31T06:12:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.951+0000 7fdad0c00640 1 -- 172.21.15.16:0/3243595528 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fdac4004e90 con 0x7fdad4105a60 2024-08-31T06:12:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.951+0000 7fdad9000640 1 --2- 172.21.15.16:0/3844887184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56280891d1b0 0x562808cc0920 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fdacc010450 tx=0x7fdacc010910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=733ed969f96dd519 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.952+0000 7fdac9600640 1 -- 172.21.15.16:0/3844887184 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc017070 con 0x56280891d1b0 2024-08-31T06:12:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.952+0000 7fdac9600640 1 -- 172.21.15.16:0/3844887184 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdacc011c10 con 0x56280891d1b0 2024-08-31T06:12:38.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.952+0000 7fdac9600640 1 -- 172.21.15.16:0/3844887184 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc01ac80 con 0x56280891d1b0 2024-08-31T06:12:38.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.953+0000 7fdae2251740 1 -- 172.21.15.16:0/3844887184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56280891d1b0 msgr2=0x562808cc0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:38.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.953+0000 7fdae2251740 1 --2- 172.21.15.16:0/3844887184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56280891d1b0 0x562808cc0920 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fdacc010450 tx=0x7fdacc010910 comp rx=0 tx=0).stop 2024-08-31T06:12:38.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.953+0000 7fdae2251740 1 -- 172.21.15.16:0/3844887184 shutdown_connections 2024-08-31T06:12:38.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.953+0000 7fdae2251740 1 --2- 172.21.15.16:0/3844887184 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5628088f42f0 0x562808cc0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:38.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.953+0000 7fdae2251740 1 --2- 172.21.15.16:0/3844887184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628088fc4d0 0x562808cc13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:38.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.953+0000 7fdad0c00640 1 --2- 172.21.15.16:0/3243595528 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdab80b4ca0 0x7fdab80b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:38.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.953+0000 7fdae2251740 1 --2- 172.21.15.16:0/3844887184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56280891d1b0 0x562808cc0920 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:38.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.953+0000 7fdae2251740 1 -- 172.21.15.16:0/3844887184 >> 172.21.15.16:0/3844887184 conn(0x562808911300 msgr2=0x562808cbe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:38.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.953+0000 7fdae2251740 1 -- 172.21.15.16:0/3844887184 shutdown_connections 2024-08-31T06:12:38.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.953+0000 7fdad0c00640 1 -- 172.21.15.16:0/3243595528 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fdac40e5a40 con 0x7fdad4105a60 2024-08-31T06:12:38.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.953+0000 7fdae2251740 1 -- 172.21.15.16:0/3844887184 wait complete. 2024-08-31T06:12:38.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.953+0000 7fdad9000640 1 --2- 172.21.15.16:0/3243595528 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdab80b4ca0 0x7fdab80b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:38.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.954+0000 7fdae2251740 1 -- start start 2024-08-31T06:12:38.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.954+0000 7fdad9000640 1 --2- 172.21.15.16:0/3243595528 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdab80b4ca0 0x7fdab80b71b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdacc005bf0 tx=0x7fdacc01a740 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:38.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.954+0000 7fdae2251740 1 Processor -- start 2024-08-31T06:12:38.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.954+0000 7fdae2251740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5628088f42f0 0x562808cd5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:38.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.955+0000 7fdae2251740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628088fc4d0 0x562808cd5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:38.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.955+0000 7fdae2251740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56280891d1b0 0x562808cda660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:38.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.955+0000 7fdae2251740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562808b5eab0 con 0x5628088fc4d0 2024-08-31T06:12:38.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.955+0000 7fdae2251740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562808bcc320 con 0x56280891d1b0 2024-08-31T06:12:38.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.955+0000 7fdae2251740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562808ccf0a0 con 0x5628088f42f0 2024-08-31T06:12:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.955+0000 7fdae2251740 10 client.0 _ll_register_callbacks cb 0x562808bd26a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:12:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.955+0000 7fdad3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628088fc4d0 0x562808cd5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.955+0000 7fdad9000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56280891d1b0 0x562808cda660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.955+0000 7fdad3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628088fc4d0 0x562808cd5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33402/0 (socket says 172.21.15.16:33402) 2024-08-31T06:12:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.955+0000 7fdad3400640 1 -- 172.21.15.16:0/2487248766 learned_addr learned my addr 172.21.15.16:0/2487248766 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.955+0000 7fdad3e00640 1 --2- 172.21.15.16:0/2487248766 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5628088f42f0 0x562808cd5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.955+0000 7fdad3400640 1 -- 172.21.15.16:0/2487248766 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5628088f42f0 msgr2=0x562808cd5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.955+0000 7fdad3400640 1 --2- 172.21.15.16:0/2487248766 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5628088f42f0 0x562808cd5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.955+0000 7fdad3400640 1 -- 172.21.15.16:0/2487248766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56280891d1b0 msgr2=0x562808cda660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.955+0000 7fdad3400640 1 --2- 172.21.15.16:0/2487248766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56280891d1b0 0x562808cda660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.955+0000 7fdad3400640 1 -- 172.21.15.16:0/2487248766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562808988790 con 0x5628088fc4d0 2024-08-31T06:12:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.955+0000 7fdad9000640 1 --2- 172.21.15.16:0/2487248766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56280891d1b0 0x562808cda660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:12:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.956+0000 7fdad3400640 1 --2- 172.21.15.16:0/2487248766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628088fc4d0 0x562808cd5650 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fdac0009900 tx=0x7fdac00316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a684570bdc26a7 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.956+0000 7fdabe000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:12:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.956+0000 7fdabe000640 1 -- 172.21.15.16:0/2487248766 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac0035040 con 0x5628088fc4d0 2024-08-31T06:12:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.956+0000 7fdae2251740 1 -- 172.21.15.16:0/2487248766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdacc011c10 con 0x5628088fc4d0 2024-08-31T06:12:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.956+0000 7fdabe000640 1 -- 172.21.15.16:0/2487248766 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fdac0004240 con 0x5628088fc4d0 2024-08-31T06:12:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.956+0000 7fdabe000640 1 -- 172.21.15.16:0/2487248766 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fdac0031900 con 0x5628088fc4d0 2024-08-31T06:12:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.956+0000 7fdae2251740 10 client.16614 fetch_fsmap learned FSMap version 38 2024-08-31T06:12:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.957+0000 7fdae2251740 1 -- 172.21.15.16:0/2487248766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x562808a01620 con 0x5628088fc4d0 2024-08-31T06:12:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.957+0000 7fdabe000640 1 -- 172.21.15.16:0/2487248766 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdacc011c10 con 0x5628088fc4d0 2024-08-31T06:12:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.957+0000 7fdabe000640 1 -- 172.21.15.16:0/2487248766 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7fdac0004b80 con 0x5628088fc4d0 2024-08-31T06:12:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.957+0000 7fdae2251740 10 client.16614 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T06:12:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.957+0000 7fdae2251740 10 client.16614 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:12:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.957+0000 7fdae2251740 20 client.16614 populate_metadata read hostname 'smithi016' 2024-08-31T06:12:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.957+0000 7fdae2251740 1 --2- 172.21.15.16:0/2487248766 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5628088dbd80 0x562808ce6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.957+0000 7fdae2251740 4 client.16614 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:12:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.957+0000 7fdae2251740 1 -- 172.21.15.16:0/2487248766 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562808c37ec0 con 0x5628088dbd80 2024-08-31T06:12:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.958+0000 7fdad3e00640 1 --2- 172.21.15.16:0/2487248766 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5628088dbd80 0x562808ce6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.958+0000 7fdad3e00640 1 --2- 172.21.15.16:0/2487248766 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5628088dbd80 0x562808ce6950 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2c9e986e0e10f24 server_cookie=31d1d8ab48891a7a in_seq=0 out_seq=0 2024-08-31T06:12:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.958+0000 7fdabe000640 10 client.16614 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:12:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.959+0000 7fdabe000640 1 -- 172.21.15.16:0/2487248766 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562808c37ec0 con 0x5628088dbd80 2024-08-31T06:12:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.959+0000 7fdabe000640 10 client.16614 handle_command_reply: tid=0 2024-08-31T06:12:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.959+0000 7fdae2251740 1 client.16614 shutdown 2024-08-31T06:12:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.959+0000 7fdae2251740 1 -- 172.21.15.16:0/2487248766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628088fc4d0 msgr2=0x562808cd5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.959+0000 7fdae2251740 1 --2- 172.21.15.16:0/2487248766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628088fc4d0 0x562808cd5650 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fdac0009900 tx=0x7fdac00316d0 comp rx=0 tx=0).stop 2024-08-31T06:12:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.959+0000 7fdae2251740 1 -- 172.21.15.16:0/2487248766 shutdown_connections 2024-08-31T06:12:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.959+0000 7fdae2251740 1 --2- 172.21.15.16:0/2487248766 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5628088dbd80 0x562808ce6950 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.959+0000 7fdae2251740 1 --2- 172.21.15.16:0/2487248766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56280891d1b0 0x562808cda660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.960+0000 7fdae2251740 1 --2- 172.21.15.16:0/2487248766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628088fc4d0 0x562808cd5650 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.960+0000 7fdae2251740 1 --2- 172.21.15.16:0/2487248766 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5628088f42f0 0x562808cd5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.960+0000 7fdae2251740 1 -- 172.21.15.16:0/2487248766 >> 172.21.15.16:0/2487248766 conn(0x562808911300 msgr2=0x562808cc1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.960+0000 7fdabe000640 0 client.16614 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:12:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.960+0000 7fdae2251740 1 -- 172.21.15.16:0/2487248766 shutdown_connections 2024-08-31T06:12:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.960+0000 7fdae2251740 1 -- 172.21.15.16:0/2487248766 wait complete. 2024-08-31T06:12:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.960+0000 7fdae2251740 20 client.16614 trim_cache size 0 max 16384 2024-08-31T06:12:38.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.975+0000 7fdae2251740 1 Processor -- start 2024-08-31T06:12:38.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.975+0000 7fdae2251740 1 -- start start 2024-08-31T06:12:38.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.976+0000 7fdae2251740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5628088f42f0 0x562808cbe860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.976+0000 7fdae2251740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628088fc4d0 0x562808cd6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.976+0000 7fdae2251740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56280891d1b0 0x562808cd02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.976+0000 7fdae2251740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdad41afa30 con 0x5628088fc4d0 2024-08-31T06:12:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.976+0000 7fdae2251740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdad41ad270 con 0x56280891d1b0 2024-08-31T06:12:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.976+0000 7fdae2251740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdad41ab0c0 con 0x5628088f42f0 2024-08-31T06:12:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.976+0000 7fdad3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628088fc4d0 0x562808cd6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.976+0000 7fdad9000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56280891d1b0 0x562808cd02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.976+0000 7fdad3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628088fc4d0 0x562808cd6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33418/0 (socket says 172.21.15.16:33418) 2024-08-31T06:12:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.976+0000 7fdad3400640 1 -- 172.21.15.16:0/2339202977 learned_addr learned my addr 172.21.15.16:0/2339202977 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.976+0000 7fdad3e00640 1 --2- 172.21.15.16:0/2339202977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5628088f42f0 0x562808cbe860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.977+0000 7fdad3400640 1 -- 172.21.15.16:0/2339202977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5628088f42f0 msgr2=0x562808cbe860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.977+0000 7fdad3400640 1 --2- 172.21.15.16:0/2339202977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5628088f42f0 0x562808cbe860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.977+0000 7fdad3400640 1 -- 172.21.15.16:0/2339202977 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56280891d1b0 msgr2=0x562808cd02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.977+0000 7fdad3400640 1 --2- 172.21.15.16:0/2339202977 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56280891d1b0 0x562808cd02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.977+0000 7fdad3400640 1 -- 172.21.15.16:0/2339202977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562808cd7230 con 0x5628088fc4d0 2024-08-31T06:12:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.977+0000 7fdad3e00640 1 --2- 172.21.15.16:0/2339202977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5628088f42f0 0x562808cbe860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:12:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.977+0000 7fdad9000640 1 --2- 172.21.15.16:0/2339202977 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56280891d1b0 0x562808cd02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:12:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.977+0000 7fdad3400640 1 --2- 172.21.15.16:0/2339202977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628088fc4d0 0x562808cd6380 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fdac0005190 tx=0x7fdac002a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16fc28f9addc1f7f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.977+0000 7fdac8c00640 1 -- 172.21.15.16:0/2339202977 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac0000e60 con 0x5628088fc4d0 2024-08-31T06:12:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.977+0000 7fdac8c00640 1 -- 172.21.15.16:0/2339202977 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdac0005570 con 0x5628088fc4d0 2024-08-31T06:12:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.977+0000 7fdac8c00640 1 -- 172.21.15.16:0/2339202977 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac0035070 con 0x5628088fc4d0 2024-08-31T06:12:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.978+0000 7fdae2251740 1 -- 172.21.15.16:0/2339202977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628088fc4d0 msgr2=0x562808cd6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.978+0000 7fdae2251740 1 --2- 172.21.15.16:0/2339202977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628088fc4d0 0x562808cd6380 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fdac0005190 tx=0x7fdac002a3f0 comp rx=0 tx=0).stop 2024-08-31T06:12:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.978+0000 7fdae2251740 1 -- 172.21.15.16:0/2339202977 shutdown_connections 2024-08-31T06:12:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.978+0000 7fdae2251740 1 --2- 172.21.15.16:0/2339202977 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56280891d1b0 0x562808cd02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.978+0000 7fdae2251740 1 --2- 172.21.15.16:0/2339202977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628088fc4d0 0x562808cd6380 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.978+0000 7fdae2251740 1 --2- 172.21.15.16:0/2339202977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5628088f42f0 0x562808cbe860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.978+0000 7fdae2251740 1 -- 172.21.15.16:0/2339202977 >> 172.21.15.16:0/2339202977 conn(0x562808911300 msgr2=0x562808cc1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.978+0000 7fdae2251740 1 -- 172.21.15.16:0/2339202977 shutdown_connections 2024-08-31T06:12:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.979+0000 7fdae2251740 1 -- 172.21.15.16:0/2339202977 wait complete. 2024-08-31T06:12:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.979+0000 7fdae2251740 1 -- start start 2024-08-31T06:12:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.979+0000 7fdae2251740 1 Processor -- start 2024-08-31T06:12:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.980+0000 7fdae2251740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628088f42f0 0x562808cdb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.980+0000 7fdae2251740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628088fc4d0 0x562808cdbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.980+0000 7fdae2251740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56280891d1b0 0x562808cdc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.980+0000 7fdae2251740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdad41ab730 con 0x5628088f42f0 2024-08-31T06:12:38.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.980+0000 7fdae2251740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdad41acba0 con 0x5628088fc4d0 2024-08-31T06:12:38.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.980+0000 7fdae2251740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdad41abe00 con 0x56280891d1b0 2024-08-31T06:12:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.980+0000 7fdae2251740 10 client.0 _ll_register_callbacks cb 0x7fdac0005e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:12:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.980+0000 7fdad3400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628088fc4d0 0x562808cdbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.980+0000 7fdad3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628088f42f0 0x562808cdb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.980+0000 7fdad3400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628088fc4d0 0x562808cdbb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:36260/0 (socket says 172.21.15.16:36260) 2024-08-31T06:12:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.980+0000 7fdad3400640 1 -- 172.21.15.16:0/1201759081 learned_addr learned my addr 172.21.15.16:0/1201759081 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.980+0000 7fdad9000640 1 --2- 172.21.15.16:0/1201759081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56280891d1b0 0x562808cdc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.980+0000 7fdad3400640 1 -- 172.21.15.16:0/1201759081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56280891d1b0 msgr2=0x562808cdc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.980+0000 7fdad3400640 1 --2- 172.21.15.16:0/1201759081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56280891d1b0 0x562808cdc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.980+0000 7fdad3400640 1 -- 172.21.15.16:0/1201759081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628088f42f0 msgr2=0x562808cdb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.980+0000 7fdad3400640 1 --2- 172.21.15.16:0/1201759081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628088f42f0 0x562808cdb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.980+0000 7fdad3400640 1 -- 172.21.15.16:0/1201759081 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562808ce0720 con 0x5628088fc4d0 2024-08-31T06:12:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.981+0000 7fdad3e00640 1 --2- 172.21.15.16:0/1201759081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628088f42f0 0x562808cdb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:12:38.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.981+0000 7fdad3400640 1 --2- 172.21.15.16:0/1201759081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628088fc4d0 0x562808cdbb90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fdac0009900 tx=0x7fdac0009640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90ccb8c19dbacc81 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:38.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.981+0000 7fdabe000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:12:38.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.981+0000 7fdabe000640 1 -- 172.21.15.16:0/1201759081 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac0035040 con 0x5628088fc4d0 2024-08-31T06:12:38.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.981+0000 7fdabe000640 1 -- 172.21.15.16:0/1201759081 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fdac00035f0 con 0x5628088fc4d0 2024-08-31T06:12:38.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.981+0000 7fdae2251740 1 -- 172.21.15.16:0/1201759081 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdac0005570 con 0x5628088fc4d0 2024-08-31T06:12:38.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.982+0000 7fdabe000640 1 -- 172.21.15.16:0/1201759081 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fdac0046680 con 0x5628088fc4d0 2024-08-31T06:12:38.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.982+0000 7fdae2251740 10 client.26831 fetch_fsmap learned FSMap version 38 2024-08-31T06:12:38.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.982+0000 7fdae2251740 1 -- 172.21.15.16:0/1201759081 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x562808d51590 con 0x5628088fc4d0 2024-08-31T06:12:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.982+0000 7fdabe000640 1 -- 172.21.15.16:0/1201759081 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdac0005570 con 0x5628088fc4d0 2024-08-31T06:12:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.982+0000 7fdabe000640 1 -- 172.21.15.16:0/1201759081 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7fdac0050850 con 0x5628088fc4d0 2024-08-31T06:12:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.982+0000 7fdae2251740 10 client.26831 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T06:12:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.982+0000 7fdae2251740 10 client.26831 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:12:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.982+0000 7fdae2251740 20 client.26831 populate_metadata read hostname 'smithi016' 2024-08-31T06:12:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.982+0000 7fdae2251740 1 --2- 172.21.15.16:0/1201759081 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5628088b11c0 0x562808d53c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.982+0000 7fdae2251740 4 client.26831 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:12:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.982+0000 7fdae2251740 1 -- 172.21.15.16:0/1201759081 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562808c33960 con 0x5628088b11c0 2024-08-31T06:12:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.983+0000 7fdad3e00640 1 --2- 172.21.15.16:0/1201759081 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5628088b11c0 0x562808d53c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.983+0000 7fdad3e00640 1 --2- 172.21.15.16:0/1201759081 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5628088b11c0 0x562808d53c60 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=feab5b95cd6a12b9 server_cookie=b168effefb279fe1 in_seq=0 out_seq=0 2024-08-31T06:12:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:38.983+0000 7fdabe000640 10 client.26831 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:12:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:39.003+0000 7fdabe000640 1 -- 172.21.15.16:0/1201759081 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562808c33960 con 0x5628088b11c0 2024-08-31T06:12:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:39.003+0000 7fdabe000640 10 client.26831 handle_command_reply: tid=0 2024-08-31T06:12:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:39.003+0000 7fdae2251740 1 client.26831 shutdown 2024-08-31T06:12:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:39.004+0000 7fdae2251740 1 -- 172.21.15.16:0/1201759081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628088fc4d0 msgr2=0x562808cdbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:39.004+0000 7fdae2251740 1 --2- 172.21.15.16:0/1201759081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628088fc4d0 0x562808cdbb90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fdac0009900 tx=0x7fdac0009640 comp rx=0 tx=0).stop 2024-08-31T06:12:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:39.004+0000 7fdae2251740 1 -- 172.21.15.16:0/1201759081 shutdown_connections 2024-08-31T06:12:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:39.004+0000 7fdae2251740 1 --2- 172.21.15.16:0/1201759081 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5628088b11c0 0x562808d53c60 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:39.004+0000 7fdae2251740 1 --2- 172.21.15.16:0/1201759081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56280891d1b0 0x562808cdc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:39.004+0000 7fdae2251740 1 --2- 172.21.15.16:0/1201759081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628088fc4d0 0x562808cdbb90 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:39.004+0000 7fdabe000640 0 client.26831 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:12:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:39.004+0000 7fdae2251740 1 --2- 172.21.15.16:0/1201759081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628088f42f0 0x562808cdb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:39.004+0000 7fdae2251740 1 -- 172.21.15.16:0/1201759081 >> 172.21.15.16:0/1201759081 conn(0x562808911300 msgr2=0x562808cd9430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:39.004+0000 7fdae2251740 1 -- 172.21.15.16:0/1201759081 shutdown_connections 2024-08-31T06:12:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:39.004+0000 7fdae2251740 1 -- 172.21.15.16:0/1201759081 wait complete. 2024-08-31T06:12:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:39.004+0000 7fdae2251740 20 client.26831 trim_cache size 0 max 16384 2024-08-31T06:12:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:12:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:12:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:12:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:12:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:39.005+0000 7fdadb000640 1 -- 172.21.15.16:0/3243595528 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdab80b4ca0 msgr2=0x7fdab80b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:39.005+0000 7fdadb000640 1 --2- 172.21.15.16:0/3243595528 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdab80b4ca0 0x7fdab80b71b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdacc005bf0 tx=0x7fdacc01a740 comp rx=0 tx=0).stop 2024-08-31T06:12:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:39.005+0000 7fdadb000640 1 -- 172.21.15.16:0/3243595528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdad4105a60 msgr2=0x7fdad419c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:39.005+0000 7fdadb000640 1 --2- 172.21.15.16:0/3243595528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdad4105a60 0x7fdad419c400 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fdac400e9f0 tx=0x7fdac400edf0 comp rx=0 tx=0).stop 2024-08-31T06:12:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:39.005+0000 7fdadb000640 1 -- 172.21.15.16:0/3243595528 shutdown_connections 2024-08-31T06:12:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:39.005+0000 7fdadb000640 1 --2- 172.21.15.16:0/3243595528 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdab80b4ca0 0x7fdab80b71b0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:39.005+0000 7fdadb000640 1 --2- 172.21.15.16:0/3243595528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdad410bce0 0x7fdad41a1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:39.005+0000 7fdadb000640 1 --2- 172.21.15.16:0/3243595528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdad410b390 0x7fdad419c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:39.005+0000 7fdadb000640 1 --2- 172.21.15.16:0/3243595528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdad4105a60 0x7fdad419c400 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:39.005+0000 7fdadb000640 1 -- 172.21.15.16:0/3243595528 >> 172.21.15.16:0/3243595528 conn(0x7fdad4101480 msgr2=0x7fdad41038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:39.005+0000 7fdadb000640 1 -- 172.21.15.16:0/3243595528 shutdown_connections 2024-08-31T06:12:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:39.005+0000 7fdadb000640 1 -- 172.21.15.16:0/3243595528 wait complete. 2024-08-31T06:12:39.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:12:38 smithi016 ceph-mon[17194]: pgmap v526: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 13 MiB/s wr, 371 op/s 2024-08-31T06:12:39.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:12:38 smithi016 ceph-mon[17194]: scrub queued for path: / 2024-08-31T06:12:39.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:12:38 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:12:39.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:12:38 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:12:39.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:12:38 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:12:39.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:12:38 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:12:39.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:12:38 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:12:39.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:12:38 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:12:39.327 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:12:39.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:12:38 smithi155 ceph-mon[21742]: pgmap v526: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 13 MiB/s wr, 371 op/s 2024-08-31T06:12:39.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:12:38 smithi155 ceph-mon[21742]: scrub queued for path: / 2024-08-31T06:12:39.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:12:38 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:12:39.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:12:38 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:12:39.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:12:38 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:12:39.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:12:38 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:12:39.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:12:38 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:12:39.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:12:38 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:12:39.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:12:38 smithi184 ceph-mon[22331]: pgmap v526: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 13 MiB/s wr, 371 op/s 2024-08-31T06:12:39.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:12:38 smithi184 ceph-mon[22331]: scrub queued for path: / 2024-08-31T06:12:39.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:12:38 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:12:39.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:12:38 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:12:39.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:12:38 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:12:39.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:12:38 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:12:39.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:12:38 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:12:39.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:12:38 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:12:40.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.035+0000 7f31a0c00640 1 -- 172.21.15.16:0/4288163292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f319c075e40 msgr2=0x7f319c10f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:40.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.035+0000 7f31a0c00640 1 --2- 172.21.15.16:0/4288163292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f319c075e40 0x7f319c10f7d0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3188009900 tx=0x7f318802a3f0 comp rx=0 tx=0).stop 2024-08-31T06:12:40.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.035+0000 7f31a0c00640 1 -- 172.21.15.16:0/4288163292 shutdown_connections 2024-08-31T06:12:40.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.035+0000 7f31a0c00640 1 --2- 172.21.15.16:0/4288163292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f319c10fd10 0x7f319c112130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:40.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.035+0000 7f31a0c00640 1 --2- 172.21.15.16:0/4288163292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f319c075e40 0x7f319c10f7d0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:40.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.035+0000 7f31a0c00640 1 --2- 172.21.15.16:0/4288163292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f319c0754f0 0x7f319c075900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:40.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.035+0000 7f31a0c00640 1 -- 172.21.15.16:0/4288163292 >> 172.21.15.16:0/4288163292 conn(0x7f319c0ff1a0 msgr2=0x7f319c1015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:40.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.036+0000 7f31a0c00640 1 -- 172.21.15.16:0/4288163292 shutdown_connections 2024-08-31T06:12:40.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.036+0000 7f31a0c00640 1 -- 172.21.15.16:0/4288163292 wait complete. 2024-08-31T06:12:40.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.037+0000 7f31a0c00640 1 Processor -- start 2024-08-31T06:12:40.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.037+0000 7f31a0c00640 1 -- start start 2024-08-31T06:12:40.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.037+0000 7f31a0c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f319c0754f0 0x7f319c19c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:40.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.037+0000 7f31a0c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f319c075e40 0x7f319c19c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:40.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.037+0000 7f31a0c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f319c10fd10 0x7f319c1a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:40.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.038+0000 7f31a0c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f319c115880 con 0x7f319c075e40 2024-08-31T06:12:40.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.038+0000 7f31a0c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f319c115700 con 0x7f319c0754f0 2024-08-31T06:12:40.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.038+0000 7f31a0c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f319c115a00 con 0x7f319c10fd10 2024-08-31T06:12:40.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.038+0000 7f319b400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f319c0754f0 0x7f319c19c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:40.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.038+0000 7f319aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f319c075e40 0x7f319c19c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:40.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.038+0000 7f319b400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f319c0754f0 0x7f319c19c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:36286/0 (socket says 172.21.15.16:36286) 2024-08-31T06:12:40.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.038+0000 7f319b400640 1 -- 172.21.15.16:0/1164998113 learned_addr learned my addr 172.21.15.16:0/1164998113 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:40.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.038+0000 7f319be00640 1 --2- 172.21.15.16:0/1164998113 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f319c10fd10 0x7f319c1a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:40.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.038+0000 7f319b400640 1 -- 172.21.15.16:0/1164998113 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f319c10fd10 msgr2=0x7f319c1a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:40.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.038+0000 7f319b400640 1 --2- 172.21.15.16:0/1164998113 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f319c10fd10 0x7f319c1a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:40.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.038+0000 7f319b400640 1 -- 172.21.15.16:0/1164998113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f319c075e40 msgr2=0x7f319c19c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:40.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.038+0000 7f319b400640 1 --2- 172.21.15.16:0/1164998113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f319c075e40 0x7f319c19c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:40.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.038+0000 7f319b400640 1 -- 172.21.15.16:0/1164998113 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f319c1a2220 con 0x7f319c0754f0 2024-08-31T06:12:40.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.039+0000 7f319b400640 1 --2- 172.21.15.16:0/1164998113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f319c0754f0 0x7f319c19c2b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f318c00e9f0 tx=0x7f318c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:40.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.039+0000 7f3193e00640 1 -- 172.21.15.16:0/1164998113 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c00cd80 con 0x7f319c0754f0 2024-08-31T06:12:40.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.039+0000 7f31a0c00640 1 -- 172.21.15.16:0/1164998113 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f319c1a2510 con 0x7f319c0754f0 2024-08-31T06:12:40.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.039+0000 7f3193e00640 1 -- 172.21.15.16:0/1164998113 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f318c0054d0 con 0x7f319c0754f0 2024-08-31T06:12:40.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.039+0000 7f3193e00640 1 -- 172.21.15.16:0/1164998113 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c010690 con 0x7f319c0754f0 2024-08-31T06:12:40.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.039+0000 7f31a0c00640 1 -- 172.21.15.16:0/1164998113 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f319c1a8c70 con 0x7f319c0754f0 2024-08-31T06:12:40.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.042+0000 7f31a8e35740 1 Processor -- start 2024-08-31T06:12:40.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.042+0000 7f31a8e35740 1 -- start start 2024-08-31T06:12:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.043+0000 7f3193e00640 1 -- 172.21.15.16:0/1164998113 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f318c004e90 con 0x7f319c0754f0 2024-08-31T06:12:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.043+0000 7f31a8e35740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a07bf9070 0x559a07f9eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.043+0000 7f31a8e35740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a07bd01b0 0x559a07f9f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.043+0000 7f31a8e35740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a07bd8390 0x559a07f9f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.043+0000 7f31a8e35740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559a07f15770 con 0x559a07bf9070 2024-08-31T06:12:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.043+0000 7f31a8e35740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559a07f04d00 con 0x559a07bd01b0 2024-08-31T06:12:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.043+0000 7f31a8e35740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559a07f08cb0 con 0x559a07bd8390 2024-08-31T06:12:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.043+0000 7f319be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a07bf9070 0x559a07f9eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.043+0000 7f319be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a07bf9070 0x559a07f9eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33442/0 (socket says 172.21.15.16:33442) 2024-08-31T06:12:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.043+0000 7f319be00640 1 -- 172.21.15.16:0/3435725596 learned_addr learned my addr 172.21.15.16:0/3435725596 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.043+0000 7f319aa00640 1 --2- 172.21.15.16:0/3435725596 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a07bd8390 0x559a07f9f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.043+0000 7f319b400640 1 --2- 172.21.15.16:0/3435725596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a07bd01b0 0x559a07f9f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.044+0000 7f319be00640 1 -- 172.21.15.16:0/3435725596 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a07bd8390 msgr2=0x559a07f9f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.044+0000 7f319be00640 1 --2- 172.21.15.16:0/3435725596 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a07bd8390 0x559a07f9f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.044+0000 7f319be00640 1 -- 172.21.15.16:0/3435725596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a07bd01b0 msgr2=0x559a07f9f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.044+0000 7f319be00640 1 --2- 172.21.15.16:0/3435725596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a07bd01b0 0x559a07f9f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.044+0000 7f319be00640 1 -- 172.21.15.16:0/3435725596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a07c64650 con 0x559a07bf9070 2024-08-31T06:12:40.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.044+0000 7f319be00640 1 --2- 172.21.15.16:0/3435725596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a07bf9070 0x559a07f9eb20 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3194009fd0 tx=0x7f31940118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed138b6187fcf400 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:40.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.044+0000 7f3187e00640 1 -- 172.21.15.16:0/3435725596 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3194004d20 con 0x559a07bf9070 2024-08-31T06:12:40.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.044+0000 7f3187e00640 1 -- 172.21.15.16:0/3435725596 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3194004050 con 0x559a07bf9070 2024-08-31T06:12:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.044+0000 7f3187e00640 1 -- 172.21.15.16:0/3435725596 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3194005240 con 0x559a07bf9070 2024-08-31T06:12:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.045+0000 7f3193e00640 1 --2- 172.21.15.16:0/1164998113 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f31800b4ca0 0x7f31800b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.045+0000 7f3193e00640 1 -- 172.21.15.16:0/1164998113 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f318c0e5930 con 0x7f319c0754f0 2024-08-31T06:12:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.045+0000 7f319b400640 1 --2- 172.21.15.16:0/1164998113 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f31800b4ca0 0x7f31800b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.045+0000 7f31a8e35740 1 -- 172.21.15.16:0/3435725596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a07bf9070 msgr2=0x559a07f9eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.045+0000 7f31a8e35740 1 --2- 172.21.15.16:0/3435725596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a07bf9070 0x559a07f9eb20 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3194009fd0 tx=0x7f31940118d0 comp rx=0 tx=0).stop 2024-08-31T06:12:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.045+0000 7f31a8e35740 1 -- 172.21.15.16:0/3435725596 shutdown_connections 2024-08-31T06:12:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.045+0000 7f31a8e35740 1 --2- 172.21.15.16:0/3435725596 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a07bd8390 0x559a07f9f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.045+0000 7f31a8e35740 1 --2- 172.21.15.16:0/3435725596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a07bd01b0 0x559a07f9f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.045+0000 7f31a8e35740 1 --2- 172.21.15.16:0/3435725596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a07bf9070 0x559a07f9eb20 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.045+0000 7f31a8e35740 1 -- 172.21.15.16:0/3435725596 >> 172.21.15.16:0/3435725596 conn(0x559a07bed460 msgr2=0x559a07f9ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.046+0000 7f31a8e35740 1 -- 172.21.15.16:0/3435725596 shutdown_connections 2024-08-31T06:12:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.046+0000 7f31a8e35740 1 -- 172.21.15.16:0/3435725596 wait complete. 2024-08-31T06:12:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.046+0000 7f319b400640 1 --2- 172.21.15.16:0/1164998113 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f31800b4ca0 0x7f31800b71b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f318c0e3190 tx=0x7f318c004a90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:40.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.046+0000 7f31a8e35740 1 -- start start 2024-08-31T06:12:40.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.047+0000 7f31a8e35740 1 Processor -- start 2024-08-31T06:12:40.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.047+0000 7f31a8e35740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a07bd01b0 0x559a07faa6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:40.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.047+0000 7f31a8e35740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a07bd8390 0x559a07faac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:40.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.047+0000 7f31a8e35740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a07bf9070 0x559a07fb65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:40.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.047+0000 7f31a8e35740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559a07d67700 con 0x559a07bf9070 2024-08-31T06:12:40.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.047+0000 7f31a8e35740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559a07e8bcb0 con 0x559a07bd01b0 2024-08-31T06:12:40.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.047+0000 7f31a8e35740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559a07fad300 con 0x559a07bd8390 2024-08-31T06:12:40.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.047+0000 7f31a8e35740 10 client.0 _ll_register_callbacks cb 0x559a07eb1430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:12:40.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.048+0000 7f319be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a07bd01b0 0x559a07faa6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:40.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.048+0000 7f319aa00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a07bd8390 0x559a07faac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:40.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.048+0000 7f319be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a07bf9070 0x559a07fb65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:40.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.048+0000 7f319aa00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a07bd8390 0x559a07faac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:47902/0 (socket says 172.21.15.16:47902) 2024-08-31T06:12:40.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.048+0000 7f319aa00640 1 -- 172.21.15.16:0/1396004623 learned_addr learned my addr 172.21.15.16:0/1396004623 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:40.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.048+0000 7f319aa00640 1 -- 172.21.15.16:0/1396004623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a07bd01b0 msgr2=0x559a07faa6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:40.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.048+0000 7f319aa00640 1 --2- 172.21.15.16:0/1396004623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a07bd01b0 0x559a07faa6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:40.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.048+0000 7f319aa00640 1 -- 172.21.15.16:0/1396004623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a07bf9070 msgr2=0x559a07fb65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:40.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.048+0000 7f319aa00640 1 --2- 172.21.15.16:0/1396004623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a07bf9070 0x559a07fb65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:40.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.048+0000 7f319aa00640 1 -- 172.21.15.16:0/1396004623 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a07ecc870 con 0x559a07bd8390 2024-08-31T06:12:40.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.049+0000 7f319aa00640 1 --2- 172.21.15.16:0/1396004623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a07bd8390 0x559a07faac20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3188009900 tx=0x7f3188032e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b66f7bd9c8b3b47 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:40.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.049+0000 7f3184c00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:12:40.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.049+0000 7f3184c00640 1 -- 172.21.15.16:0/1396004623 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3188035040 con 0x559a07bd8390 2024-08-31T06:12:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.049+0000 7f31a8e35740 1 -- 172.21.15.16:0/1396004623 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3194004050 con 0x559a07bd8390 2024-08-31T06:12:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.049+0000 7f3184c00640 1 -- 172.21.15.16:0/1396004623 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f3188031760 con 0x559a07bd8390 2024-08-31T06:12:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.049+0000 7f3184c00640 1 -- 172.21.15.16:0/1396004623 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f3188004040 con 0x559a07bd8390 2024-08-31T06:12:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.049+0000 7f31a8e35740 10 client.26824 fetch_fsmap learned FSMap version 38 2024-08-31T06:12:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.049+0000 7f31a8e35740 1 -- 172.21.15.16:0/1396004623 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x559a07fb9370 con 0x559a07bd8390 2024-08-31T06:12:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.050+0000 7f3184c00640 1 -- 172.21.15.16:0/1396004623 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3194004050 con 0x559a07bd8390 2024-08-31T06:12:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.050+0000 7f3184c00640 1 -- 172.21.15.16:0/1396004623 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f3188004750 con 0x559a07bd8390 2024-08-31T06:12:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.050+0000 7f31a8e35740 10 client.26824 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T06:12:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.050+0000 7f31a8e35740 10 client.26824 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:12:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.050+0000 7f31a8e35740 20 client.26824 populate_metadata read hostname 'smithi016' 2024-08-31T06:12:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.050+0000 7f31a8e35740 1 --2- 172.21.15.16:0/1396004623 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559a07bb79f0 0x559a07fbc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.050+0000 7f31a8e35740 4 client.26824 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:12:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.050+0000 7f31a8e35740 1 -- 172.21.15.16:0/1396004623 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a07f176c0 con 0x559a07bb79f0 2024-08-31T06:12:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.051+0000 7f319be00640 1 --2- 172.21.15.16:0/1396004623 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559a07bb79f0 0x559a07fbc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.051+0000 7f319be00640 1 --2- 172.21.15.16:0/1396004623 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559a07bb79f0 0x559a07fbc860 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb25427b89ce2d89 server_cookie=7013cbda070b3639 in_seq=0 out_seq=0 2024-08-31T06:12:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.051+0000 7f3184c00640 10 client.26824 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:12:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.052+0000 7f3184c00640 1 -- 172.21.15.16:0/1396004623 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559a07f176c0 con 0x559a07bb79f0 2024-08-31T06:12:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.052+0000 7f3184c00640 10 client.26824 handle_command_reply: tid=0 2024-08-31T06:12:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.052+0000 7f31a8e35740 1 client.26824 shutdown 2024-08-31T06:12:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.052+0000 7f31a8e35740 1 -- 172.21.15.16:0/1396004623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a07bd8390 msgr2=0x559a07faac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.052+0000 7f31a8e35740 1 --2- 172.21.15.16:0/1396004623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a07bd8390 0x559a07faac20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3188009900 tx=0x7f3188032e90 comp rx=0 tx=0).stop 2024-08-31T06:12:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.052+0000 7f31a8e35740 1 -- 172.21.15.16:0/1396004623 shutdown_connections 2024-08-31T06:12:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.052+0000 7f31a8e35740 1 --2- 172.21.15.16:0/1396004623 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559a07bb79f0 0x559a07fbc860 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.052+0000 7f31a8e35740 1 --2- 172.21.15.16:0/1396004623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a07bd8390 0x559a07faac20 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.052+0000 7f31a8e35740 1 --2- 172.21.15.16:0/1396004623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a07bf9070 0x559a07fb65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.052+0000 7f31a8e35740 1 --2- 172.21.15.16:0/1396004623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a07bd01b0 0x559a07faa6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.052+0000 7f3184c00640 0 client.26824 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:12:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.052+0000 7f31a8e35740 1 -- 172.21.15.16:0/1396004623 >> 172.21.15.16:0/1396004623 conn(0x559a07bed460 msgr2=0x559a07fa1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.053+0000 7f31a8e35740 1 -- 172.21.15.16:0/1396004623 shutdown_connections 2024-08-31T06:12:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.053+0000 7f31a8e35740 1 -- 172.21.15.16:0/1396004623 wait complete. 2024-08-31T06:12:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.053+0000 7f31a8e35740 20 client.26824 trim_cache size 0 max 16384 2024-08-31T06:12:40.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.071+0000 7f31a8e35740 1 Processor -- start 2024-08-31T06:12:40.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.071+0000 7f31a8e35740 1 -- start start 2024-08-31T06:12:40.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.072+0000 7f31a8e35740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a07bd01b0 0x559a07fb64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:40.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.072+0000 7f31a8e35740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a07bd8390 0x559a07fb6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:40.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.072+0000 7f31a8e35740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a07bf9070 0x559a07fa76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:40.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.072+0000 7f31a8e35740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1b3d50 con 0x559a07bd01b0 2024-08-31T06:12:40.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.072+0000 7f31a8e35740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1b1590 con 0x559a07bd8390 2024-08-31T06:12:40.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.072+0000 7f31a8e35740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1af3e0 con 0x559a07bf9070 2024-08-31T06:12:40.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.072+0000 7f319be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a07bd01b0 0x559a07fb64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:40.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.072+0000 7f319be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a07bd01b0 0x559a07fb64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33464/0 (socket says 172.21.15.16:33464) 2024-08-31T06:12:40.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.072+0000 7f319be00640 1 -- 172.21.15.16:0/3036625546 learned_addr learned my addr 172.21.15.16:0/3036625546 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:40.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.072+0000 7f319be00640 1 --2- 172.21.15.16:0/3036625546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a07bf9070 0x559a07fa76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:40.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.072+0000 7f319aa00640 1 --2- 172.21.15.16:0/3036625546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a07bd8390 0x559a07fb6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:40.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.073+0000 7f319be00640 1 -- 172.21.15.16:0/3036625546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a07bf9070 msgr2=0x559a07fa76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:40.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.073+0000 7f319be00640 1 --2- 172.21.15.16:0/3036625546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a07bf9070 0x559a07fa76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:40.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.073+0000 7f319be00640 1 -- 172.21.15.16:0/3036625546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a07bd8390 msgr2=0x559a07fb6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:40.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.073+0000 7f319be00640 1 --2- 172.21.15.16:0/3036625546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a07bd8390 0x559a07fb6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:40.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.073+0000 7f319be00640 1 -- 172.21.15.16:0/3036625546 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a07fab7a0 con 0x559a07bd01b0 2024-08-31T06:12:40.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.073+0000 7f319aa00640 1 --2- 172.21.15.16:0/3036625546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a07bd8390 0x559a07fb6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:12:40.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.073+0000 7f319be00640 1 --2- 172.21.15.16:0/3036625546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a07bd01b0 0x559a07fb64f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f3194014040 tx=0x7f3194009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7824defe98c6748c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:40.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.073+0000 7f3187400640 1 -- 172.21.15.16:0/3036625546 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3194004e40 con 0x559a07bd01b0 2024-08-31T06:12:40.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.073+0000 7f3187400640 1 -- 172.21.15.16:0/3036625546 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31940054f0 con 0x559a07bd01b0 2024-08-31T06:12:40.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.073+0000 7f3187400640 1 -- 172.21.15.16:0/3036625546 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3194004200 con 0x559a07bd01b0 2024-08-31T06:12:40.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.074+0000 7f31a8e35740 1 -- 172.21.15.16:0/3036625546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a07bd01b0 msgr2=0x559a07fb64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:40.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.074+0000 7f31a8e35740 1 --2- 172.21.15.16:0/3036625546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a07bd01b0 0x559a07fb64f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f3194014040 tx=0x7f3194009500 comp rx=0 tx=0).stop 2024-08-31T06:12:40.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.074+0000 7f31a8e35740 1 -- 172.21.15.16:0/3036625546 shutdown_connections 2024-08-31T06:12:40.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.074+0000 7f31a8e35740 1 --2- 172.21.15.16:0/3036625546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a07bf9070 0x559a07fa76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:40.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.074+0000 7f31a8e35740 1 --2- 172.21.15.16:0/3036625546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a07bd8390 0x559a07fb6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:40.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.074+0000 7f31a8e35740 1 --2- 172.21.15.16:0/3036625546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a07bd01b0 0x559a07fb64f0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:40.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.074+0000 7f31a8e35740 1 -- 172.21.15.16:0/3036625546 >> 172.21.15.16:0/3036625546 conn(0x559a07bed460 msgr2=0x559a07fa3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:40.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.075+0000 7f31a8e35740 1 -- 172.21.15.16:0/3036625546 shutdown_connections 2024-08-31T06:12:40.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.075+0000 7f31a8e35740 1 -- 172.21.15.16:0/3036625546 wait complete. 2024-08-31T06:12:40.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.075+0000 7f31a8e35740 1 -- start start 2024-08-31T06:12:40.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.075+0000 7f31a8e35740 1 Processor -- start 2024-08-31T06:12:40.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.076+0000 7f31a8e35740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a07bd01b0 0x559a07fae760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:40.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.076+0000 7f31a8e35740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a07bd8390 0x559a07faeca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:40.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.076+0000 7f31a8e35740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a07bf9070 0x559a07faf220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:40.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.076+0000 7f31a8e35740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1afa50 con 0x559a07bd8390 2024-08-31T06:12:40.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.076+0000 7f31a8e35740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1b0ec0 con 0x559a07bd01b0 2024-08-31T06:12:40.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.076+0000 7f31a8e35740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1b0120 con 0x559a07bf9070 2024-08-31T06:12:40.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.076+0000 7f31a8e35740 10 client.0 _ll_register_callbacks cb 0x7f3188032b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:12:40.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.076+0000 7f319aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a07bd8390 0x559a07faeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:40.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.076+0000 7f319be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a07bd01b0 0x559a07fae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:40.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.076+0000 7f319aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a07bd8390 0x559a07faeca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33466/0 (socket says 172.21.15.16:33466) 2024-08-31T06:12:40.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.076+0000 7f319aa00640 1 -- 172.21.15.16:0/2251802833 learned_addr learned my addr 172.21.15.16:0/2251802833 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:40.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.076+0000 7f319be00640 1 --2- 172.21.15.16:0/2251802833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a07bf9070 0x559a07faf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:40.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.077+0000 7f319aa00640 1 -- 172.21.15.16:0/2251802833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a07bf9070 msgr2=0x559a07faf220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:40.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.077+0000 7f319aa00640 1 --2- 172.21.15.16:0/2251802833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a07bf9070 0x559a07faf220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:40.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.077+0000 7f319aa00640 1 -- 172.21.15.16:0/2251802833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a07bd01b0 msgr2=0x559a07fae760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:40.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.077+0000 7f319aa00640 1 --2- 172.21.15.16:0/2251802833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a07bd01b0 0x559a07fae760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:40.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.077+0000 7f319aa00640 1 -- 172.21.15.16:0/2251802833 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a080264b0 con 0x559a07bd8390 2024-08-31T06:12:40.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.077+0000 7f319be00640 1 --2- 172.21.15.16:0/2251802833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a07bf9070 0x559a07faf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:12:40.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.077+0000 7f319aa00640 1 --2- 172.21.15.16:0/2251802833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a07bd8390 0x559a07faeca0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3188004b50 tx=0x7f31880048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc5ece7d975087b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:40.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.077+0000 7f3184c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:12:40.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.077+0000 7f3184c00640 1 -- 172.21.15.16:0/2251802833 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3188035040 con 0x559a07bd8390 2024-08-31T06:12:40.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.077+0000 7f3184c00640 1 -- 172.21.15.16:0/2251802833 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f3188003640 con 0x559a07bd8390 2024-08-31T06:12:40.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.077+0000 7f31a8e35740 1 -- 172.21.15.16:0/2251802833 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31940054f0 con 0x559a07bd8390 2024-08-31T06:12:40.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.078+0000 7f3184c00640 1 -- 172.21.15.16:0/2251802833 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f3188003840 con 0x559a07bd8390 2024-08-31T06:12:40.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.078+0000 7f31a8e35740 10 client.16638 fetch_fsmap learned FSMap version 38 2024-08-31T06:12:40.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.078+0000 7f31a8e35740 1 -- 172.21.15.16:0/2251802833 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x559a08027770 con 0x559a07bd8390 2024-08-31T06:12:40.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.078+0000 7f3184c00640 1 -- 172.21.15.16:0/2251802833 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31940054f0 con 0x559a07bd8390 2024-08-31T06:12:40.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.078+0000 7f3184c00640 1 -- 172.21.15.16:0/2251802833 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f3188004f40 con 0x559a07bd8390 2024-08-31T06:12:40.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.078+0000 7f31a8e35740 10 client.16638 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T06:12:40.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.078+0000 7f31a8e35740 10 client.16638 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:12:40.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.078+0000 7f31a8e35740 20 client.16638 populate_metadata read hostname 'smithi016' 2024-08-31T06:12:40.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.078+0000 7f31a8e35740 1 --2- 172.21.15.16:0/2251802833 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559a07b8cda0 0x559a08029fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:40.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.078+0000 7f31a8e35740 4 client.16638 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T06:12:40.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.078+0000 7f31a8e35740 1 -- 172.21.15.16:0/2251802833 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559a07f13160 con 0x559a07b8cda0 2024-08-31T06:12:40.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.079+0000 7f319be00640 1 --2- 172.21.15.16:0/2251802833 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559a07b8cda0 0x559a08029fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:40.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.079+0000 7f3184c00640 10 client.16638 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:12:40.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.079+0000 7f319be00640 1 --2- 172.21.15.16:0/2251802833 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559a07b8cda0 0x559a08029fc0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a8351e61cdf5f73 server_cookie=a3b2b203c2c1f49d in_seq=0 out_seq=0 2024-08-31T06:12:40.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.080+0000 7f3184c00640 1 -- 172.21.15.16:0/2251802833 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2558 (crc 0 0 0) 0x559a07f13160 con 0x559a07b8cda0 2024-08-31T06:12:40.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.080+0000 7f3184c00640 10 client.16638 handle_command_reply: tid=0 2024-08-31T06:12:40.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.080+0000 7f31a8e35740 1 client.16638 shutdown 2024-08-31T06:12:40.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.080+0000 7f31a8e35740 1 -- 172.21.15.16:0/2251802833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a07bd8390 msgr2=0x559a07faeca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:40.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.080+0000 7f31a8e35740 1 --2- 172.21.15.16:0/2251802833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a07bd8390 0x559a07faeca0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3188004b50 tx=0x7f31880048d0 comp rx=0 tx=0).stop 2024-08-31T06:12:40.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.080+0000 7f31a8e35740 1 -- 172.21.15.16:0/2251802833 shutdown_connections 2024-08-31T06:12:40.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.080+0000 7f31a8e35740 1 --2- 172.21.15.16:0/2251802833 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559a07b8cda0 0x559a08029fc0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:40.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.080+0000 7f31a8e35740 1 --2- 172.21.15.16:0/2251802833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a07bf9070 0x559a07faf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:40.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.080+0000 7f31a8e35740 1 --2- 172.21.15.16:0/2251802833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a07bd8390 0x559a07faeca0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.080+0000 7f31a8e35740 1 --2- 172.21.15.16:0/2251802833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a07bd01b0 0x559a07fae760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.080+0000 7f3184c00640 0 client.16638 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:12:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.080+0000 7f31a8e35740 1 -- 172.21.15.16:0/2251802833 >> 172.21.15.16:0/2251802833 conn(0x559a07bed460 msgr2=0x559a07f9fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.080+0000 7f31a8e35740 1 -- 172.21.15.16:0/2251802833 shutdown_connections 2024-08-31T06:12:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.080+0000 7f31a8e35740 1 -- 172.21.15.16:0/2251802833 wait complete. 2024-08-31T06:12:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.081+0000 7f31a8e35740 20 client.16638 trim_cache size 0 max 16384 2024-08-31T06:12:40.086 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:12:40.086 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T06:12:40.086 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-08-31T06:12:40.086 INFO:teuthology.orchestra.run.smithi016.stdout: "6a03046e-cdf4-4a14-b9bd-b560b8968467": { 2024-08-31T06:12:40.086 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:12:40.087 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "6a03046e-cdf4-4a14-b9bd-b560b8968467", 2024-08-31T06:12:40.087 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-08-31T06:12:40.087 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:12:40.087 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:12:40.087 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-08-31T06:12:40.087 INFO:teuthology.orchestra.run.smithi016.stdout: "e739a16e-8ace-43a8-a1ce-ac27d39ef8a3": { 2024-08-31T06:12:40.087 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:54.084646+0000", 2024-08-31T06:12:40.087 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:12:40.087 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:12:40.087 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:12:40.087 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:12:40.087 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:12:40.087 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:12:40.088 INFO:teuthology.orchestra.run.smithi016.stdout: "a10cce76-99ab-46b5-b3b2-07b82899b8f9": { 2024-08-31T06:12:40.088 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:11:43.967546+0000", 2024-08-31T06:12:40.088 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:12:40.088 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:12:40.088 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:12:40.088 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:12:40.088 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:12:40.088 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:12:40.088 INFO:teuthology.orchestra.run.smithi016.stdout: "c1fd6904-2e02-47fa-939a-47349b9fe36b": { 2024-08-31T06:12:40.088 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:06.678691+0000", 2024-08-31T06:12:40.088 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:12:40.088 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:12:40.088 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:12:40.088 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:12:40.089 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:12:40.089 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:12:40.089 INFO:teuthology.orchestra.run.smithi016.stdout: "78d6ef02-8551-45e8-ad60-9627eb573fc1": { 2024-08-31T06:12:40.089 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:09:13.547865+0000", 2024-08-31T06:12:40.089 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:12:40.089 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:12:40.089 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:12:40.089 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:12:40.089 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:12:40.089 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:12:40.089 INFO:teuthology.orchestra.run.smithi016.stdout: "c8455d74-2fe7-4d93-bcf2-19b419d16016": { 2024-08-31T06:12:40.089 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:08:24.035960+0000", 2024-08-31T06:12:40.089 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:12:40.089 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:12:40.090 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:12:40.090 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:12:40.090 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:12:40.090 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:12:40.090 INFO:teuthology.orchestra.run.smithi016.stdout: "88d7bdb8-cb1c-45e5-ad29-4cd8da6c6003": { 2024-08-31T06:12:40.090 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:07:39.648942+0000", 2024-08-31T06:12:40.090 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:12:40.090 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:12:40.090 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:12:40.090 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:12:40.090 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:12:40.090 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:12:40.090 INFO:teuthology.orchestra.run.smithi016.stdout: "6a03046e-cdf4-4a14-b9bd-b560b8968467": { 2024-08-31T06:12:40.090 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:12:38.002354+0000", 2024-08-31T06:12:40.091 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:12:40.091 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:12:40.091 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:12:40.091 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:12:40.091 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:12:40.091 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:12:40.091 INFO:teuthology.orchestra.run.smithi016.stdout: "c300b7e8-f3fd-482c-a909-9329bc61e570": { 2024-08-31T06:12:40.091 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:06:49.868046+0000", 2024-08-31T06:12:40.091 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:12:40.091 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:12:40.091 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:12:40.091 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:12:40.091 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:12:40.091 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:12:40.091 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:12:40.092 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:12:40.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.081+0000 7f31a0c00640 1 -- 172.21.15.16:0/1164998113 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f31800b4ca0 msgr2=0x7f31800b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:40.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.081+0000 7f31a0c00640 1 --2- 172.21.15.16:0/1164998113 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f31800b4ca0 0x7f31800b71b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f318c0e3190 tx=0x7f318c004a90 comp rx=0 tx=0).stop 2024-08-31T06:12:40.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.081+0000 7f31a0c00640 1 -- 172.21.15.16:0/1164998113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f319c0754f0 msgr2=0x7f319c19c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:40.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.081+0000 7f31a0c00640 1 --2- 172.21.15.16:0/1164998113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f319c0754f0 0x7f319c19c2b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f318c00e9f0 tx=0x7f318c00edf0 comp rx=0 tx=0).stop 2024-08-31T06:12:40.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.081+0000 7f31a0c00640 1 -- 172.21.15.16:0/1164998113 shutdown_connections 2024-08-31T06:12:40.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.081+0000 7f31a0c00640 1 --2- 172.21.15.16:0/1164998113 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f31800b4ca0 0x7f31800b71b0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:40.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.081+0000 7f31a0c00640 1 --2- 172.21.15.16:0/1164998113 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f319c10fd10 0x7f319c1a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:40.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.081+0000 7f31a0c00640 1 --2- 172.21.15.16:0/1164998113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f319c075e40 0x7f319c19c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:40.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.081+0000 7f31a0c00640 1 --2- 172.21.15.16:0/1164998113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f319c0754f0 0x7f319c19c2b0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:40.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.081+0000 7f31a0c00640 1 -- 172.21.15.16:0/1164998113 >> 172.21.15.16:0/1164998113 conn(0x7f319c0ff1a0 msgr2=0x7f319c1087a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:40.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.082+0000 7f31a0c00640 1 -- 172.21.15.16:0/1164998113 shutdown_connections 2024-08-31T06:12:40.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:40.082+0000 7f31a0c00640 1 -- 172.21.15.16:0/1164998113 wait complete. 2024-08-31T06:12:40.273 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:12:40 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:12:40.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:12:40 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:12:40.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:12:40 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:12:41.046 INFO:tasks.cephfs.filesystem:scrub status for tag:6a03046e-cdf4-4a14-b9bd-b560b8968467 - {'path': '/', 'tag': '6a03046e-cdf4-4a14-b9bd-b560b8968467', 'options': 'recursive,force'} 2024-08-31T06:12:41.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:12:41 smithi155 ceph-mon[21742]: pgmap v527: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 11 MiB/s wr, 447 op/s 2024-08-31T06:12:41.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:12:41 smithi016 ceph-mon[17194]: pgmap v527: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 11 MiB/s wr, 447 op/s 2024-08-31T06:12:41.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:12:41 smithi184 ceph-mon[22331]: pgmap v527: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 11 MiB/s wr, 447 op/s 2024-08-31T06:12:41.629 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 315.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-31T06:12:43.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:12:43 smithi155 ceph-mon[21742]: pgmap v528: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 7.6 MiB/s wr, 410 op/s 2024-08-31T06:12:43.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:12:43 smithi016 ceph-mon[17194]: pgmap v528: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 7.6 MiB/s wr, 410 op/s 2024-08-31T06:12:43.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:12:43 smithi184 ceph-mon[22331]: pgmap v528: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 7.6 MiB/s wr, 410 op/s 2024-08-31T06:12:45.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:12:45 smithi155 ceph-mon[21742]: pgmap v529: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 4.4 MiB/s wr, 354 op/s 2024-08-31T06:12:45.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:12:45 smithi016 ceph-mon[17194]: pgmap v529: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 4.4 MiB/s wr, 354 op/s 2024-08-31T06:12:45.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:12:45 smithi184 ceph-mon[22331]: pgmap v529: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 4.4 MiB/s wr, 354 op/s 2024-08-31T06:12:46.627 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 320.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-31T06:12:47.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:12:47 smithi155 ceph-mon[21742]: pgmap v530: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 2.8 MiB/s wr, 316 op/s 2024-08-31T06:12:47.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:12:47 smithi016 ceph-mon[17194]: pgmap v530: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 2.8 MiB/s wr, 316 op/s 2024-08-31T06:12:47.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:12:47 smithi184 ceph-mon[22331]: pgmap v530: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 2.8 MiB/s wr, 316 op/s 2024-08-31T06:12:49.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:12:49 smithi155 ceph-mon[21742]: pgmap v531: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 2.3 MiB/s wr, 301 op/s 2024-08-31T06:12:49.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:12:49 smithi016 ceph-mon[17194]: pgmap v531: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 2.3 MiB/s wr, 301 op/s 2024-08-31T06:12:49.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:12:49 smithi184 ceph-mon[22331]: pgmap v531: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 2.3 MiB/s wr, 301 op/s 2024-08-31T06:12:51.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:12:51 smithi155 ceph-mon[21742]: pgmap v532: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 1.9 MiB/s wr, 264 op/s 2024-08-31T06:12:51.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:12:51 smithi016 ceph-mon[17194]: pgmap v532: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 1.9 MiB/s wr, 264 op/s 2024-08-31T06:12:51.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:12:51 smithi184 ceph-mon[22331]: pgmap v532: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 1.9 MiB/s wr, 264 op/s 2024-08-31T06:12:51.629 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 325.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-31T06:12:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.212+0000 7f5b23200640 1 -- 172.21.15.16:0/1225261912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b1c108fe0 msgr2=0x7f5b1c1093f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.212+0000 7f5b23200640 1 --2- 172.21.15.16:0/1225261912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b1c108fe0 0x7f5b1c1093f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c009900 tx=0x7f5b0c02a3f0 comp rx=0 tx=0).stop 2024-08-31T06:12:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.212+0000 7f5b23200640 1 -- 172.21.15.16:0/1225261912 shutdown_connections 2024-08-31T06:12:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.212+0000 7f5b23200640 1 --2- 172.21.15.16:0/1225261912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b1c109930 0x7f5b1c110160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.212+0000 7f5b23200640 1 --2- 172.21.15.16:0/1225261912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b1c108fe0 0x7f5b1c1093f0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.212+0000 7f5b23200640 1 --2- 172.21.15.16:0/1225261912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b1c1036b0 0x7f5b1c103ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.212+0000 7f5b23200640 1 -- 172.21.15.16:0/1225261912 >> 172.21.15.16:0/1225261912 conn(0x7f5b1c0ff0d0 msgr2=0x7f5b1c101520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.212+0000 7f5b23200640 1 -- 172.21.15.16:0/1225261912 shutdown_connections 2024-08-31T06:12:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.212+0000 7f5b23200640 1 -- 172.21.15.16:0/1225261912 wait complete. 2024-08-31T06:12:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.213+0000 7f5b23200640 1 Processor -- start 2024-08-31T06:12:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.214+0000 7f5b23200640 1 -- start start 2024-08-31T06:12:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.214+0000 7f5b23200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b1c1036b0 0x7f5b1c0754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.214+0000 7f5b23200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b1c108fe0 0x7f5b1c0759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.214+0000 7f5b23200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b1c109930 0x7f5b1c075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.214+0000 7f5b23200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1c1135f0 con 0x7f5b1c1036b0 2024-08-31T06:12:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.214+0000 7f5b23200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1c113470 con 0x7f5b1c108fe0 2024-08-31T06:12:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.214+0000 7f5b23200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1c113770 con 0x7f5b1c109930 2024-08-31T06:12:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.214+0000 7f5b21e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b1c1036b0 0x7f5b1c0754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.214+0000 7f5b21e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b1c1036b0 0x7f5b1c0754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44548/0 (socket says 172.21.15.16:44548) 2024-08-31T06:12:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.214+0000 7f5b21e00640 1 -- 172.21.15.16:0/73996121 learned_addr learned my addr 172.21.15.16:0/73996121 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.215+0000 7f5b22800640 1 --2- 172.21.15.16:0/73996121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b1c109930 0x7f5b1c075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.215+0000 7f5b21400640 1 --2- 172.21.15.16:0/73996121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b1c108fe0 0x7f5b1c0759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.215+0000 7f5b21e00640 1 -- 172.21.15.16:0/73996121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b1c109930 msgr2=0x7f5b1c075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.215+0000 7f5b21e00640 1 --2- 172.21.15.16:0/73996121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b1c109930 0x7f5b1c075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.215+0000 7f5b21e00640 1 -- 172.21.15.16:0/73996121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b1c108fe0 msgr2=0x7f5b1c0759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.215+0000 7f5b21e00640 1 --2- 172.21.15.16:0/73996121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b1c108fe0 0x7f5b1c0759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.215+0000 7f5b21e00640 1 -- 172.21.15.16:0/73996121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b1c1a8300 con 0x7f5b1c1036b0 2024-08-31T06:12:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.215+0000 7f5b22800640 1 --2- 172.21.15.16:0/73996121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b1c109930 0x7f5b1c075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:12:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.215+0000 7f5b21400640 1 --2- 172.21.15.16:0/73996121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b1c108fe0 0x7f5b1c0759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:12:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.215+0000 7f5b21e00640 1 --2- 172.21.15.16:0/73996121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b1c1036b0 0x7f5b1c0754b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5b1000e9f0 tx=0x7f5b1000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.215+0000 7f5b16a00640 1 -- 172.21.15.16:0/73996121 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1000cd80 con 0x7f5b1c1036b0 2024-08-31T06:12:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.215+0000 7f5b23200640 1 -- 172.21.15.16:0/73996121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b1c1a85f0 con 0x7f5b1c1036b0 2024-08-31T06:12:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.215+0000 7f5b16a00640 1 -- 172.21.15.16:0/73996121 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b100108e0 con 0x7f5b1c1036b0 2024-08-31T06:12:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.215+0000 7f5b16a00640 1 -- 172.21.15.16:0/73996121 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b10018980 con 0x7f5b1c1036b0 2024-08-31T06:12:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.216+0000 7f5b23200640 1 -- 172.21.15.16:0/73996121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b1c1a8b10 con 0x7f5b1c1036b0 2024-08-31T06:12:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.219+0000 7f5b2b804740 1 Processor -- start 2024-08-31T06:12:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.220+0000 7f5b2b804740 1 -- start start 2024-08-31T06:12:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.220+0000 7f5b2b804740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642b57571b0 0x5642b5afa8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.220+0000 7f5b2b804740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642b572e2f0 0x5642b5afade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.220+0000 7f5b2b804740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5642b57364d0 0x5642b5afb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.220+0000 7f5b2b804740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5642b5a6ff70 con 0x5642b57571b0 2024-08-31T06:12:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.220+0000 7f5b2b804740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5642b5a5f500 con 0x5642b57364d0 2024-08-31T06:12:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.220+0000 7f5b2b804740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5642b5a634b0 con 0x5642b572e2f0 2024-08-31T06:12:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.220+0000 7f5b22800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642b57571b0 0x5642b5afa8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.220+0000 7f5b22800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642b57571b0 0x5642b5afa8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44556/0 (socket says 172.21.15.16:44556) 2024-08-31T06:12:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.220+0000 7f5b22800640 1 -- 172.21.15.16:0/2817518819 learned_addr learned my addr 172.21.15.16:0/2817518819 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.221+0000 7f5b21400640 1 --2- 172.21.15.16:0/2817518819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5642b57364d0 0x5642b5afb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.221+0000 7f5b16a00640 1 -- 172.21.15.16:0/73996121 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f5b10010430 con 0x7f5b1c1036b0 2024-08-31T06:12:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.221+0000 7f5b21e00640 1 --2- 172.21.15.16:0/2817518819 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642b572e2f0 0x5642b5afade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.221+0000 7f5b21400640 1 -- 172.21.15.16:0/2817518819 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642b572e2f0 msgr2=0x5642b5afade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.221+0000 7f5b21400640 1 --2- 172.21.15.16:0/2817518819 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642b572e2f0 0x5642b5afade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.221+0000 7f5b21400640 1 -- 172.21.15.16:0/2817518819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642b57571b0 msgr2=0x5642b5afa8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.221+0000 7f5b21400640 1 --2- 172.21.15.16:0/2817518819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642b57571b0 0x5642b5afa8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.221+0000 7f5b21400640 1 -- 172.21.15.16:0/2817518819 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642b5a19c10 con 0x5642b57364d0 2024-08-31T06:12:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.221+0000 7f5b21e00640 1 --2- 172.21.15.16:0/2817518819 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642b572e2f0 0x5642b5afade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:12:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.221+0000 7f5b21400640 1 --2- 172.21.15.16:0/2817518819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5642b57364d0 0x5642b5afb320 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c06c1a0 tx=0x7f5b0c004170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40a2dd1386362b09 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.222+0000 7f5b0b000640 1 -- 172.21.15.16:0/2817518819 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c03f070 con 0x5642b57364d0 2024-08-31T06:12:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.222+0000 7f5b0b000640 1 -- 172.21.15.16:0/2817518819 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b0c005500 con 0x5642b57364d0 2024-08-31T06:12:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.222+0000 7f5b0b000640 1 -- 172.21.15.16:0/2817518819 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c035070 con 0x5642b57364d0 2024-08-31T06:12:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.222+0000 7f5b16a00640 1 --2- 172.21.15.16:0/73996121 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f5b040b4cd0 0x7f5b040b71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.223+0000 7f5b16a00640 1 -- 172.21.15.16:0/73996121 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f5b100e5990 con 0x7f5b1c1036b0 2024-08-31T06:12:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.223+0000 7f5b22800640 1 --2- 172.21.15.16:0/73996121 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f5b040b4cd0 0x7f5b040b71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.223+0000 7f5b2b804740 1 -- 172.21.15.16:0/2817518819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5642b57364d0 msgr2=0x5642b5afb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.223+0000 7f5b2b804740 1 --2- 172.21.15.16:0/2817518819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5642b57364d0 0x5642b5afb320 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c06c1a0 tx=0x7f5b0c004170 comp rx=0 tx=0).stop 2024-08-31T06:12:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.223+0000 7f5b2b804740 1 -- 172.21.15.16:0/2817518819 shutdown_connections 2024-08-31T06:12:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.223+0000 7f5b2b804740 1 --2- 172.21.15.16:0/2817518819 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642b572e2f0 0x5642b5afade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.223+0000 7f5b2b804740 1 --2- 172.21.15.16:0/2817518819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5642b57364d0 0x5642b5afb320 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.223+0000 7f5b2b804740 1 --2- 172.21.15.16:0/2817518819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642b57571b0 0x5642b5afa8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.223+0000 7f5b2b804740 1 -- 172.21.15.16:0/2817518819 >> 172.21.15.16:0/2817518819 conn(0x5642b574b300 msgr2=0x5642b5af8790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.223+0000 7f5b2b804740 1 -- 172.21.15.16:0/2817518819 shutdown_connections 2024-08-31T06:12:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.223+0000 7f5b2b804740 1 -- 172.21.15.16:0/2817518819 wait complete. 2024-08-31T06:12:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.223+0000 7f5b22800640 1 --2- 172.21.15.16:0/73996121 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f5b040b4cd0 0x7f5b040b71e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5b180099a0 tx=0x7f5b18009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.224+0000 7f5b2b804740 1 -- start start 2024-08-31T06:12:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.224+0000 7f5b2b804740 1 Processor -- start 2024-08-31T06:12:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.224+0000 7f5b2b804740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642b572e2f0 0x5642b5b05ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.224+0000 7f5b2b804740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5642b57364d0 0x5642b5b06410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.224+0000 7f5b2b804740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642b57571b0 0x5642b5b06950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.224+0000 7f5b2b804740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5642b5998ab0 con 0x5642b572e2f0 2024-08-31T06:12:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.224+0000 7f5b2b804740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5642b5a06320 con 0x5642b57364d0 2024-08-31T06:12:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.224+0000 7f5b2b804740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5642b5b08fc0 con 0x5642b57571b0 2024-08-31T06:12:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.224+0000 7f5b2b804740 10 client.0 _ll_register_callbacks cb 0x5642b5a3fe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:12:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.225+0000 7f5b21e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642b572e2f0 0x5642b5b05ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.225+0000 7f5b21e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642b572e2f0 0x5642b5b05ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44558/0 (socket says 172.21.15.16:44558) 2024-08-31T06:12:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.225+0000 7f5b21e00640 1 -- 172.21.15.16:0/2096420633 learned_addr learned my addr 172.21.15.16:0/2096420633 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.225+0000 7f5b22800640 1 --2- 172.21.15.16:0/2096420633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642b57571b0 0x5642b5b06950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.225+0000 7f5b21400640 1 --2- 172.21.15.16:0/2096420633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5642b57364d0 0x5642b5b06410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.225+0000 7f5b21e00640 1 -- 172.21.15.16:0/2096420633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642b57571b0 msgr2=0x5642b5b06950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.225+0000 7f5b21e00640 1 --2- 172.21.15.16:0/2096420633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642b57571b0 0x5642b5b06950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.225+0000 7f5b21e00640 1 -- 172.21.15.16:0/2096420633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5642b57364d0 msgr2=0x5642b5b06410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.225+0000 7f5b21e00640 1 --2- 172.21.15.16:0/2096420633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5642b57364d0 0x5642b5b06410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.225+0000 7f5b21e00640 1 -- 172.21.15.16:0/2096420633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642b57c2790 con 0x5642b572e2f0 2024-08-31T06:12:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.225+0000 7f5b21e00640 1 --2- 172.21.15.16:0/2096420633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642b572e2f0 0x5642b5b05ed0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5b10000c00 tx=0x7f5b100ab8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26775f20b42fcba6 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.225+0000 7f5aff400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:12:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.226+0000 7f5aff400640 1 -- 172.21.15.16:0/2096420633 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b100e3570 con 0x5642b572e2f0 2024-08-31T06:12:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.226+0000 7f5aff400640 1 -- 172.21.15.16:0/2096420633 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f5b100b0540 con 0x5642b572e2f0 2024-08-31T06:12:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.226+0000 7f5b2b804740 1 -- 172.21.15.16:0/2096420633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b0c005500 con 0x5642b572e2f0 2024-08-31T06:12:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.226+0000 7f5aff400640 1 -- 172.21.15.16:0/2096420633 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f5b100ac080 con 0x5642b572e2f0 2024-08-31T06:12:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.226+0000 7f5b2b804740 10 client.16650 fetch_fsmap learned FSMap version 38 2024-08-31T06:12:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.226+0000 7f5b2b804740 1 -- 172.21.15.16:0/2096420633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5642b583b620 con 0x5642b572e2f0 2024-08-31T06:12:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.226+0000 7f5aff400640 1 -- 172.21.15.16:0/2096420633 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b0c005500 con 0x5642b572e2f0 2024-08-31T06:12:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.226+0000 7f5aff400640 1 -- 172.21.15.16:0/2096420633 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f5b10018b20 con 0x5642b572e2f0 2024-08-31T06:12:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.227+0000 7f5b2b804740 10 client.16650 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T06:12:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.227+0000 7f5b2b804740 10 client.16650 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:12:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.227+0000 7f5b2b804740 20 client.16650 populate_metadata read hostname 'smithi016' 2024-08-31T06:12:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.227+0000 7f5b2b804740 1 --2- 172.21.15.16:0/2096420633 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5642b5715d80 0x5642b5b17e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.227+0000 7f5b2b804740 4 client.16650 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:12:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.227+0000 7f5b2b804740 1 -- 172.21.15.16:0/2096420633 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642b5a71ec0 con 0x5642b5715d80 2024-08-31T06:12:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.227+0000 7f5b21400640 1 --2- 172.21.15.16:0/2096420633 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5642b5715d80 0x5642b5b17e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.228+0000 7f5b21400640 1 --2- 172.21.15.16:0/2096420633 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5642b5715d80 0x5642b5b17e40 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6291e3f526056f6 server_cookie=46a7c7f947d5880c in_seq=0 out_seq=0 2024-08-31T06:12:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.228+0000 7f5aff400640 10 client.16650 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:12:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.229+0000 7f5aff400640 1 -- 172.21.15.16:0/2096420633 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5642b5a71ec0 con 0x5642b5715d80 2024-08-31T06:12:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.229+0000 7f5aff400640 10 client.16650 handle_command_reply: tid=0 2024-08-31T06:12:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.229+0000 7f5b2b804740 1 client.16650 shutdown 2024-08-31T06:12:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.229+0000 7f5b2b804740 1 -- 172.21.15.16:0/2096420633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642b572e2f0 msgr2=0x5642b5b05ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.229+0000 7f5b2b804740 1 --2- 172.21.15.16:0/2096420633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642b572e2f0 0x5642b5b05ed0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5b10000c00 tx=0x7f5b100ab8e0 comp rx=0 tx=0).stop 2024-08-31T06:12:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.229+0000 7f5b2b804740 1 -- 172.21.15.16:0/2096420633 shutdown_connections 2024-08-31T06:12:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.229+0000 7f5b2b804740 1 --2- 172.21.15.16:0/2096420633 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5642b5715d80 0x5642b5b17e40 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.229+0000 7f5b2b804740 1 --2- 172.21.15.16:0/2096420633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642b57571b0 0x5642b5b06950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.229+0000 7f5b2b804740 1 --2- 172.21.15.16:0/2096420633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5642b57364d0 0x5642b5b06410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.229+0000 7f5b2b804740 1 --2- 172.21.15.16:0/2096420633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642b572e2f0 0x5642b5b05ed0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.229+0000 7f5aff400640 0 client.16650 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:12:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.229+0000 7f5b2b804740 1 -- 172.21.15.16:0/2096420633 >> 172.21.15.16:0/2096420633 conn(0x5642b574b300 msgr2=0x5642b5afb8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.229+0000 7f5b2b804740 1 -- 172.21.15.16:0/2096420633 shutdown_connections 2024-08-31T06:12:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.229+0000 7f5b2b804740 1 -- 172.21.15.16:0/2096420633 wait complete. 2024-08-31T06:12:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.229+0000 7f5b2b804740 20 client.16650 trim_cache size 0 max 16384 2024-08-31T06:12:52.248 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.245+0000 7f5b2b804740 1 Processor -- start 2024-08-31T06:12:52.248 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.245+0000 7f5b2b804740 1 -- start start 2024-08-31T06:12:52.248 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.245+0000 7f5b2b804740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642b572e2f0 0x5642b5af7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.245+0000 7f5b2b804740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5642b57364d0 0x5642b5afbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.245+0000 7f5b2b804740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642b57571b0 0x5642b5b063b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.245+0000 7f5b2b804740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1c1b3d10 con 0x5642b57571b0 2024-08-31T06:12:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.245+0000 7f5b2b804740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1c1b1550 con 0x5642b57364d0 2024-08-31T06:12:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.245+0000 7f5b2b804740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1c1af3a0 con 0x5642b572e2f0 2024-08-31T06:12:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.245+0000 7f5b22800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642b57571b0 0x5642b5b063b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.245+0000 7f5b22800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642b57571b0 0x5642b5b063b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44562/0 (socket says 172.21.15.16:44562) 2024-08-31T06:12:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.245+0000 7f5b22800640 1 -- 172.21.15.16:0/688587584 learned_addr learned my addr 172.21.15.16:0/688587584 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.245+0000 7f5b21400640 1 --2- 172.21.15.16:0/688587584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5642b57364d0 0x5642b5afbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.246+0000 7f5b21e00640 1 --2- 172.21.15.16:0/688587584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642b572e2f0 0x5642b5af7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.246+0000 7f5b22800640 1 -- 172.21.15.16:0/688587584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642b572e2f0 msgr2=0x5642b5af7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.246+0000 7f5b22800640 1 --2- 172.21.15.16:0/688587584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642b572e2f0 0x5642b5af7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.246+0000 7f5b22800640 1 -- 172.21.15.16:0/688587584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5642b57364d0 msgr2=0x5642b5afbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.246+0000 7f5b22800640 1 --2- 172.21.15.16:0/688587584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5642b57364d0 0x5642b5afbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.246+0000 7f5b22800640 1 -- 172.21.15.16:0/688587584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642b5afcba0 con 0x5642b57571b0 2024-08-31T06:12:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.246+0000 7f5b21400640 1 --2- 172.21.15.16:0/688587584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5642b57364d0 0x5642b5afbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:12:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.246+0000 7f5b21e00640 1 --2- 172.21.15.16:0/688587584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642b572e2f0 0x5642b5af7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:12:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.246+0000 7f5b22800640 1 --2- 172.21.15.16:0/688587584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642b57571b0 0x5642b5b063b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f5b18010a50 tx=0x7f5b18019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28bff80c836cc636 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.246+0000 7f5b0a600640 1 -- 172.21.15.16:0/688587584 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b18004b20 con 0x5642b57571b0 2024-08-31T06:12:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.246+0000 7f5b0a600640 1 -- 172.21.15.16:0/688587584 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b18004cc0 con 0x5642b57571b0 2024-08-31T06:12:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.247+0000 7f5b0a600640 1 -- 172.21.15.16:0/688587584 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b180058d0 con 0x5642b57571b0 2024-08-31T06:12:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.247+0000 7f5b2b804740 1 -- 172.21.15.16:0/688587584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642b57571b0 msgr2=0x5642b5b063b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.247+0000 7f5b2b804740 1 --2- 172.21.15.16:0/688587584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642b57571b0 0x5642b5b063b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f5b18010a50 tx=0x7f5b18019000 comp rx=0 tx=0).stop 2024-08-31T06:12:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.247+0000 7f5b2b804740 1 -- 172.21.15.16:0/688587584 shutdown_connections 2024-08-31T06:12:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.247+0000 7f5b2b804740 1 --2- 172.21.15.16:0/688587584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642b57571b0 0x5642b5b063b0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.247+0000 7f5b2b804740 1 --2- 172.21.15.16:0/688587584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5642b57364d0 0x5642b5afbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.247+0000 7f5b2b804740 1 --2- 172.21.15.16:0/688587584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642b572e2f0 0x5642b5af7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.247+0000 7f5b2b804740 1 -- 172.21.15.16:0/688587584 >> 172.21.15.16:0/688587584 conn(0x5642b574b300 msgr2=0x5642b5afb8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.247+0000 7f5b2b804740 1 -- 172.21.15.16:0/688587584 shutdown_connections 2024-08-31T06:12:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.247+0000 7f5b2b804740 1 -- 172.21.15.16:0/688587584 wait complete. 2024-08-31T06:12:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.248+0000 7f5b2b804740 1 -- start start 2024-08-31T06:12:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.248+0000 7f5b2b804740 1 Processor -- start 2024-08-31T06:12:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.248+0000 7f5b2b804740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642b572e2f0 0x5642b5b0ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.249+0000 7f5b2b804740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5642b57364d0 0x5642b5b0b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.249+0000 7f5b2b804740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642b57571b0 0x5642b5b7b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.249+0000 7f5b2b804740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1c1afa10 con 0x5642b57571b0 2024-08-31T06:12:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.249+0000 7f5b2b804740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1c1b0e80 con 0x5642b57364d0 2024-08-31T06:12:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.249+0000 7f5b2b804740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1c1b00e0 con 0x5642b572e2f0 2024-08-31T06:12:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.249+0000 7f5b2b804740 10 client.0 _ll_register_callbacks cb 0x7f5b10009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:12:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.249+0000 7f5b22800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642b57571b0 0x5642b5b7b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.249+0000 7f5b22800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642b57571b0 0x5642b5b7b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44568/0 (socket says 172.21.15.16:44568) 2024-08-31T06:12:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.249+0000 7f5b22800640 1 -- 172.21.15.16:0/827336747 learned_addr learned my addr 172.21.15.16:0/827336747 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.249+0000 7f5b21e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642b572e2f0 0x5642b5b0ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.249+0000 7f5b21400640 1 --2- 172.21.15.16:0/827336747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5642b57364d0 0x5642b5b0b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.249+0000 7f5b22800640 1 -- 172.21.15.16:0/827336747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642b572e2f0 msgr2=0x5642b5b0ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.249+0000 7f5b22800640 1 --2- 172.21.15.16:0/827336747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642b572e2f0 0x5642b5b0ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.249+0000 7f5b22800640 1 -- 172.21.15.16:0/827336747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5642b57364d0 msgr2=0x5642b5b0b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.249+0000 7f5b22800640 1 --2- 172.21.15.16:0/827336747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5642b57364d0 0x5642b5b0b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.249+0000 7f5b22800640 1 -- 172.21.15.16:0/827336747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642b5b7bfb0 con 0x5642b57571b0 2024-08-31T06:12:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.250+0000 7f5b22800640 1 --2- 172.21.15.16:0/827336747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642b57571b0 0x5642b5b7b940 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f5b18018040 tx=0x7f5b180054d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6272767207b4402e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.250+0000 7f5aff400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:12:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.250+0000 7f5aff400640 1 -- 172.21.15.16:0/827336747 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b18036060 con 0x5642b57571b0 2024-08-31T06:12:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.250+0000 7f5aff400640 1 -- 172.21.15.16:0/827336747 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f5b1801ec60 con 0x5642b57571b0 2024-08-31T06:12:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.250+0000 7f5b2b804740 1 -- 172.21.15.16:0/827336747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b18004cc0 con 0x5642b57571b0 2024-08-31T06:12:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.250+0000 7f5aff400640 1 -- 172.21.15.16:0/827336747 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f5b1801ee60 con 0x5642b57571b0 2024-08-31T06:12:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.250+0000 7f5b2b804740 10 client.16656 fetch_fsmap learned FSMap version 38 2024-08-31T06:12:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.250+0000 7f5b2b804740 1 -- 172.21.15.16:0/827336747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5642b5b833d0 con 0x5642b57571b0 2024-08-31T06:12:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.250+0000 7f5aff400640 1 -- 172.21.15.16:0/827336747 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b18004cc0 con 0x5642b57571b0 2024-08-31T06:12:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.251+0000 7f5aff400640 1 -- 172.21.15.16:0/827336747 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f5b18030030 con 0x5642b57571b0 2024-08-31T06:12:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.251+0000 7f5b2b804740 10 client.16656 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T06:12:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.251+0000 7f5b2b804740 10 client.16656 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:12:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.251+0000 7f5b2b804740 20 client.16656 populate_metadata read hostname 'smithi016' 2024-08-31T06:12:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.251+0000 7f5b2b804740 1 --2- 172.21.15.16:0/827336747 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5642b56eb1c0 0x5642b5b85ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.251+0000 7f5b2b804740 4 client.16656 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:12:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.251+0000 7f5b2b804740 1 -- 172.21.15.16:0/827336747 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5642b5a6d960 con 0x5642b56eb1c0 2024-08-31T06:12:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.251+0000 7f5b21e00640 1 --2- 172.21.15.16:0/827336747 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5642b56eb1c0 0x5642b5b85ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.252+0000 7f5aff400640 10 client.16656 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:12:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.252+0000 7f5b21e00640 1 --2- 172.21.15.16:0/827336747 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5642b56eb1c0 0x5642b5b85ab0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63a2d4fea0117b83 server_cookie=e86561018df58f73 in_seq=0 out_seq=0 2024-08-31T06:12:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.266+0000 7f5aff400640 1 -- 172.21.15.16:0/827336747 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5642b5a6d960 con 0x5642b56eb1c0 2024-08-31T06:12:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.266+0000 7f5aff400640 10 client.16656 handle_command_reply: tid=0 2024-08-31T06:12:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.266+0000 7f5b2b804740 1 client.16656 shutdown 2024-08-31T06:12:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.266+0000 7f5b2b804740 1 -- 172.21.15.16:0/827336747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642b57571b0 msgr2=0x5642b5b7b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.266+0000 7f5b2b804740 1 --2- 172.21.15.16:0/827336747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642b57571b0 0x5642b5b7b940 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f5b18018040 tx=0x7f5b180054d0 comp rx=0 tx=0).stop 2024-08-31T06:12:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.266+0000 7f5b2b804740 1 -- 172.21.15.16:0/827336747 shutdown_connections 2024-08-31T06:12:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.266+0000 7f5b2b804740 1 --2- 172.21.15.16:0/827336747 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5642b56eb1c0 0x5642b5b85ab0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.266+0000 7f5b2b804740 1 --2- 172.21.15.16:0/827336747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642b57571b0 0x5642b5b7b940 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.266+0000 7f5b2b804740 1 --2- 172.21.15.16:0/827336747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5642b57364d0 0x5642b5b0b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.266+0000 7f5aff400640 0 client.16656 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:12:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.266+0000 7f5b2b804740 1 --2- 172.21.15.16:0/827336747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642b572e2f0 0x5642b5b0ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.266+0000 7f5b2b804740 1 -- 172.21.15.16:0/827336747 >> 172.21.15.16:0/827336747 conn(0x5642b574b300 msgr2=0x5642b5af7500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.266+0000 7f5b2b804740 1 -- 172.21.15.16:0/827336747 shutdown_connections 2024-08-31T06:12:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.266+0000 7f5b2b804740 1 -- 172.21.15.16:0/827336747 wait complete. 2024-08-31T06:12:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.266+0000 7f5b2b804740 20 client.16656 trim_cache size 0 max 16384 2024-08-31T06:12:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:12:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:12:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:12:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:12:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.267+0000 7f5b23200640 1 -- 172.21.15.16:0/73996121 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f5b040b4cd0 msgr2=0x7f5b040b71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.267+0000 7f5b23200640 1 --2- 172.21.15.16:0/73996121 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f5b040b4cd0 0x7f5b040b71e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5b180099a0 tx=0x7f5b18009500 comp rx=0 tx=0).stop 2024-08-31T06:12:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.267+0000 7f5b23200640 1 -- 172.21.15.16:0/73996121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b1c1036b0 msgr2=0x7f5b1c0754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.267+0000 7f5b23200640 1 --2- 172.21.15.16:0/73996121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b1c1036b0 0x7f5b1c0754b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5b1000e9f0 tx=0x7f5b1000eeb0 comp rx=0 tx=0).stop 2024-08-31T06:12:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.267+0000 7f5b23200640 1 -- 172.21.15.16:0/73996121 shutdown_connections 2024-08-31T06:12:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.267+0000 7f5b23200640 1 --2- 172.21.15.16:0/73996121 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f5b040b4cd0 0x7f5b040b71e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.267+0000 7f5b23200640 1 --2- 172.21.15.16:0/73996121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b1c109930 0x7f5b1c075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:52.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.267+0000 7f5b23200640 1 --2- 172.21.15.16:0/73996121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b1c108fe0 0x7f5b1c0759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:52.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.267+0000 7f5b23200640 1 --2- 172.21.15.16:0/73996121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b1c1036b0 0x7f5b1c0754b0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:52.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.267+0000 7f5b23200640 1 -- 172.21.15.16:0/73996121 >> 172.21.15.16:0/73996121 conn(0x7f5b1c0ff0d0 msgr2=0x7f5b1c1014f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:52.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.267+0000 7f5b23200640 1 -- 172.21.15.16:0/73996121 shutdown_connections 2024-08-31T06:12:52.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:12:52.267+0000 7f5b23200640 1 -- 172.21.15.16:0/73996121 wait complete. 2024-08-31T06:12:53.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:12:53 smithi155 ceph-mon[21742]: pgmap v533: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 689 KiB/s wr, 119 op/s 2024-08-31T06:12:53.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:12:53 smithi016 ceph-mon[17194]: pgmap v533: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 689 KiB/s wr, 119 op/s 2024-08-31T06:12:53.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:12:53 smithi184 ceph-mon[22331]: pgmap v533: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 689 KiB/s wr, 119 op/s 2024-08-31T06:12:55.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:12:55 smithi155 ceph-mon[21742]: pgmap v534: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 30 KiB/s wr, 32 op/s 2024-08-31T06:12:55.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:12:55 smithi016 ceph-mon[17194]: pgmap v534: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 30 KiB/s wr, 32 op/s 2024-08-31T06:12:55.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:12:55 smithi184 ceph-mon[22331]: pgmap v534: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 30 KiB/s wr, 32 op/s 2024-08-31T06:12:56.561 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:12:56.629 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 330.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-31T06:12:56.953 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:12:57.294 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:12:57 smithi016 ceph-mon[17194]: pgmap v535: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-08-31T06:12:57.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:12:57 smithi155 ceph-mon[21742]: pgmap v535: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-08-31T06:12:57.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:12:57 smithi184 ceph-mon[22331]: pgmap v535: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-08-31T06:12:57.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.588+0000 7fe47b000640 1 -- 172.21.15.16:0/4087767628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe474103900 msgr2=0x7fe474103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:57.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.588+0000 7fe47b000640 1 --2- 172.21.15.16:0/4087767628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe474103900 0x7fe474103d10 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fe464009900 tx=0x7fe46402a3f0 comp rx=0 tx=0).stop 2024-08-31T06:12:57.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.588+0000 7fe47b000640 1 -- 172.21.15.16:0/4087767628 shutdown_connections 2024-08-31T06:12:57.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.588+0000 7fe47b000640 1 --2- 172.21.15.16:0/4087767628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe474109b90 0x7fe4741103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:57.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.588+0000 7fe47b000640 1 --2- 172.21.15.16:0/4087767628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe474109240 0x7fe474109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:57.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.588+0000 7fe47b000640 1 --2- 172.21.15.16:0/4087767628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe474103900 0x7fe474103d10 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:57.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.588+0000 7fe47b000640 1 -- 172.21.15.16:0/4087767628 >> 172.21.15.16:0/4087767628 conn(0x7fe47407b5f0 msgr2=0x7fe47407ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:57.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.588+0000 7fe47b000640 1 -- 172.21.15.16:0/4087767628 shutdown_connections 2024-08-31T06:12:57.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.588+0000 7fe47b000640 1 -- 172.21.15.16:0/4087767628 wait complete. 2024-08-31T06:12:57.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.589+0000 7fe47b000640 1 Processor -- start 2024-08-31T06:12:57.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.589+0000 7fe47b000640 1 -- start start 2024-08-31T06:12:57.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.590+0000 7fe47b000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe474103900 0x7fe47419a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:57.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.590+0000 7fe47b000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe474109240 0x7fe47419a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:57.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.590+0000 7fe47b000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe474109b90 0x7fe47419fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:57.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.590+0000 7fe47b000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe474113850 con 0x7fe474103900 2024-08-31T06:12:57.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.590+0000 7fe47b000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe4741136d0 con 0x7fe474109b90 2024-08-31T06:12:57.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.590+0000 7fe47b000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe4741139d0 con 0x7fe474109240 2024-08-31T06:12:57.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.590+0000 7fe473e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe474103900 0x7fe47419a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:57.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.590+0000 7fe473e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe474103900 0x7fe47419a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36216/0 (socket says 172.21.15.16:36216) 2024-08-31T06:12:57.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.590+0000 7fe473e00640 1 -- 172.21.15.16:0/1269045678 learned_addr learned my addr 172.21.15.16:0/1269045678 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:57.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.590+0000 7fe46be00640 1 --2- 172.21.15.16:0/1269045678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe474109240 0x7fe47419a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:57.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.590+0000 7fe479000640 1 --2- 172.21.15.16:0/1269045678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe474109b90 0x7fe47419fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:57.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.591+0000 7fe473e00640 1 -- 172.21.15.16:0/1269045678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe474109240 msgr2=0x7fe47419a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:57.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.591+0000 7fe473e00640 1 --2- 172.21.15.16:0/1269045678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe474109240 0x7fe47419a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:57.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.591+0000 7fe473e00640 1 -- 172.21.15.16:0/1269045678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe474109b90 msgr2=0x7fe47419fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:57.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.591+0000 7fe473e00640 1 --2- 172.21.15.16:0/1269045678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe474109b90 0x7fe47419fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:57.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.591+0000 7fe473e00640 1 -- 172.21.15.16:0/1269045678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4741a0190 con 0x7fe474103900 2024-08-31T06:12:57.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.591+0000 7fe473e00640 1 --2- 172.21.15.16:0/1269045678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe474103900 0x7fe47419a220 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe464005940 tx=0x7fe464004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:57.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.591+0000 7fe471600640 1 -- 172.21.15.16:0/1269045678 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe46403f030 con 0x7fe474103900 2024-08-31T06:12:57.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.591+0000 7fe471600640 1 -- 172.21.15.16:0/1269045678 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe464003aa0 con 0x7fe474103900 2024-08-31T06:12:57.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.591+0000 7fe47b000640 1 -- 172.21.15.16:0/1269045678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4741a0420 con 0x7fe474103900 2024-08-31T06:12:57.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.591+0000 7fe471600640 1 -- 172.21.15.16:0/1269045678 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe464035070 con 0x7fe474103900 2024-08-31T06:12:57.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.592+0000 7fe47b000640 1 -- 172.21.15.16:0/1269045678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4741a6be0 con 0x7fe474103900 2024-08-31T06:12:57.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.593+0000 7fe47b000640 1 -- 172.21.15.16:0/1269045678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe474103d10 con 0x7fe474103900 2024-08-31T06:12:57.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.595+0000 7fe471600640 1 -- 172.21.15.16:0/1269045678 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fe464042030 con 0x7fe474103900 2024-08-31T06:12:57.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.597+0000 7fe471600640 1 --2- 172.21.15.16:0/1269045678 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe4580b4ca0 0x7fe4580b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:57.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.597+0000 7fe471600640 1 -- 172.21.15.16:0/1269045678 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7fe464105d50 con 0x7fe474103900 2024-08-31T06:12:57.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.597+0000 7fe46be00640 1 --2- 172.21.15.16:0/1269045678 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe4580b4ca0 0x7fe4580b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:57.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.598+0000 7fe46be00640 1 --2- 172.21.15.16:0/1269045678 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe4580b4ca0 0x7fe4580b71b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe4741061f0 tx=0x7fe45c00b410 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:57.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.599+0000 7fe471600640 1 -- 172.21.15.16:0/1269045678 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fe4640cc4f0 con 0x7fe474103900 2024-08-31T06:12:57.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.831+0000 7fe47b000640 1 -- 172.21.15.16:0/1269045678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe47419b350 con 0x7fe474103900 2024-08-31T06:12:57.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.832+0000 7fe471600640 1 -- 172.21.15.16:0/1269045678 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11921 (secure 0 0 0) 0x7fe4640d0cb0 con 0x7fe474103900 2024-08-31T06:12:57.834 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:12:57.835 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":38,"btime":"2024-08-31T06:12:33:984731+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:12:33.336879+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24712,24724,24737]},"id":1}]} 2024-08-31T06:12:57.835 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 38 2024-08-31T06:12:57.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.834+0000 7fe47b000640 1 -- 172.21.15.16:0/1269045678 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe4580b4ca0 msgr2=0x7fe4580b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:57.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.834+0000 7fe47b000640 1 --2- 172.21.15.16:0/1269045678 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe4580b4ca0 0x7fe4580b71b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe4741061f0 tx=0x7fe45c00b410 comp rx=0 tx=0).stop 2024-08-31T06:12:57.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.834+0000 7fe47b000640 1 -- 172.21.15.16:0/1269045678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe474103900 msgr2=0x7fe47419a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:57.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.834+0000 7fe47b000640 1 --2- 172.21.15.16:0/1269045678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe474103900 0x7fe47419a220 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe464005940 tx=0x7fe464004270 comp rx=0 tx=0).stop 2024-08-31T06:12:57.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.834+0000 7fe47b000640 1 -- 172.21.15.16:0/1269045678 shutdown_connections 2024-08-31T06:12:57.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.834+0000 7fe47b000640 1 --2- 172.21.15.16:0/1269045678 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe4580b4ca0 0x7fe4580b71b0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:57.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.834+0000 7fe47b000640 1 --2- 172.21.15.16:0/1269045678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe474109240 0x7fe47419a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:57.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.834+0000 7fe47b000640 1 --2- 172.21.15.16:0/1269045678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe474109b90 0x7fe47419fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:57.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.834+0000 7fe47b000640 1 --2- 172.21.15.16:0/1269045678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe474103900 0x7fe47419a220 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:57.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.834+0000 7fe47b000640 1 -- 172.21.15.16:0/1269045678 >> 172.21.15.16:0/1269045678 conn(0x7fe47407b5f0 msgr2=0x7fe474107330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:57.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.834+0000 7fe47b000640 1 -- 172.21.15.16:0/1269045678 shutdown_connections 2024-08-31T06:12:57.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:57.834+0000 7fe47b000640 1 -- 172.21.15.16:0/1269045678 wait complete. 2024-08-31T06:12:58.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:12:58 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/1269045678' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:12:58.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:12:58 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/1269045678' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:12:58.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:12:58 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/1269045678' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:12:58.698 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24737 quiesce db --set-id 2facd010 --release --await' 2024-08-31T06:12:58.699 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24737 quiesce db --set-id 2facd010 --release --await 2024-08-31T06:12:59.092 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:12:59.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:12:59 smithi155 ceph-mon[21742]: pgmap v536: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-08-31T06:12:59.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:12:59 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:12:59.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:12:59 smithi016 ceph-mon[17194]: pgmap v536: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-08-31T06:12:59.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:12:59 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:12:59.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:12:59 smithi184 ceph-mon[22331]: pgmap v536: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-08-31T06:12:59.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:12:59 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:12:59.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.731+0000 7f8e96000640 1 -- 172.21.15.16:0/1318987029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e900776e0 msgr2=0x7f8e90077af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:59.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.731+0000 7f8e96000640 1 --2- 172.21.15.16:0/1318987029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e900776e0 0x7f8e90077af0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8e84009900 tx=0x7f8e8402a410 comp rx=0 tx=0).stop 2024-08-31T06:12:59.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.731+0000 7f8e96000640 1 -- 172.21.15.16:0/1318987029 shutdown_connections 2024-08-31T06:12:59.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.731+0000 7f8e96000640 1 --2- 172.21.15.16:0/1318987029 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e90114310 0x7f8e90116730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:59.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.731+0000 7f8e96000640 1 --2- 172.21.15.16:0/1318987029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e90078030 0x7f8e90113dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:59.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.731+0000 7f8e96000640 1 --2- 172.21.15.16:0/1318987029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e900776e0 0x7f8e90077af0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:59.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.731+0000 7f8e96000640 1 -- 172.21.15.16:0/1318987029 >> 172.21.15.16:0/1318987029 conn(0x7f8e90101420 msgr2=0x7f8e90103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:59.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.731+0000 7f8e96000640 1 -- 172.21.15.16:0/1318987029 shutdown_connections 2024-08-31T06:12:59.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.731+0000 7f8e96000640 1 -- 172.21.15.16:0/1318987029 wait complete. 2024-08-31T06:12:59.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.732+0000 7f8e96000640 1 Processor -- start 2024-08-31T06:12:59.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.732+0000 7f8e96000640 1 -- start start 2024-08-31T06:12:59.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.733+0000 7f8e96000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e900776e0 0x7f8e901a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:59.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.733+0000 7f8e96000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e90078030 0x7f8e901a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:59.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.733+0000 7f8e96000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e90114310 0x7f8e901a5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:59.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.733+0000 7f8e96000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8e90119c90 con 0x7f8e90078030 2024-08-31T06:12:59.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.733+0000 7f8e96000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8e90119b10 con 0x7f8e90114310 2024-08-31T06:12:59.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.733+0000 7f8e96000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8e90119e10 con 0x7f8e900776e0 2024-08-31T06:12:59.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.733+0000 7f8e94c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e900776e0 0x7f8e901a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:59.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.733+0000 7f8e94c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e900776e0 0x7f8e901a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:41804/0 (socket says 172.21.15.16:41804) 2024-08-31T06:12:59.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.733+0000 7f8e94c00640 1 -- 172.21.15.16:0/1470027295 learned_addr learned my addr 172.21.15.16:0/1470027295 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:59.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.733+0000 7f8e8fe00640 1 --2- 172.21.15.16:0/1470027295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e90078030 0x7f8e901a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:59.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.733+0000 7f8e95600640 1 --2- 172.21.15.16:0/1470027295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e90114310 0x7f8e901a5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:59.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.733+0000 7f8e8fe00640 1 -- 172.21.15.16:0/1470027295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e900776e0 msgr2=0x7f8e901a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:59.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.733+0000 7f8e8fe00640 1 --2- 172.21.15.16:0/1470027295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e900776e0 0x7f8e901a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:59.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.733+0000 7f8e8fe00640 1 -- 172.21.15.16:0/1470027295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e90114310 msgr2=0x7f8e901a5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:59.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.733+0000 7f8e8fe00640 1 --2- 172.21.15.16:0/1470027295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e90114310 0x7f8e901a5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:59.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.733+0000 7f8e8fe00640 1 -- 172.21.15.16:0/1470027295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e901a6610 con 0x7f8e90078030 2024-08-31T06:12:59.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.734+0000 7f8e8fe00640 1 --2- 172.21.15.16:0/1470027295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e90078030 0x7f8e901a0c40 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f8e9006c660 tx=0x7f8e7c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:59.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.734+0000 7f8e8d600640 1 -- 172.21.15.16:0/1470027295 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c00cc40 con 0x7f8e90078030 2024-08-31T06:12:59.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.734+0000 7f8e8d600640 1 -- 172.21.15.16:0/1470027295 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e7c00cde0 con 0x7f8e90078030 2024-08-31T06:12:59.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.734+0000 7f8e96000640 1 -- 172.21.15.16:0/1470027295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e901a6900 con 0x7f8e90078030 2024-08-31T06:12:59.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.734+0000 7f8e8d600640 1 -- 172.21.15.16:0/1470027295 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c018780 con 0x7f8e90078030 2024-08-31T06:12:59.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.734+0000 7f8e96000640 1 -- 172.21.15.16:0/1470027295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e901ad060 con 0x7f8e90078030 2024-08-31T06:12:59.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.737+0000 7f8e9e1bc740 1 Processor -- start 2024-08-31T06:12:59.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.737+0000 7f8e9e1bc740 1 -- start start 2024-08-31T06:12:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.738+0000 7f8e8d600640 1 -- 172.21.15.16:0/1470027295 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f8e7c018920 con 0x7f8e90078030 2024-08-31T06:12:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.738+0000 7f8e9e1bc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561eab12b520 0x561eab4d1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.738+0000 7f8e9e1bc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561eab102660 0x561eab4d15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.738+0000 7f8e9e1bc740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561eab10a840 0x561eab4d1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.738+0000 7f8e9e1bc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561eab447a90 con 0x561eab102660 2024-08-31T06:12:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.738+0000 7f8e9e1bc740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561eab437020 con 0x561eab10a840 2024-08-31T06:12:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.738+0000 7f8e9e1bc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x561eab43afd0 con 0x561eab12b520 2024-08-31T06:12:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.738+0000 7f8e94c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561eab102660 0x561eab4d15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:59.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.738+0000 7f8e8fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561eab10a840 0x561eab4d1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:59.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.738+0000 7f8e95600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561eab12b520 0x561eab4d1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:59.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.738+0000 7f8e94c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561eab102660 0x561eab4d15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36238/0 (socket says 172.21.15.16:36238) 2024-08-31T06:12:59.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.738+0000 7f8e94c00640 1 -- 172.21.15.16:0/2512836968 learned_addr learned my addr 172.21.15.16:0/2512836968 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:59.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.739+0000 7f8e95600640 1 -- 172.21.15.16:0/2512836968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561eab10a840 msgr2=0x561eab4d1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:59.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.739+0000 7f8e95600640 1 --2- 172.21.15.16:0/2512836968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561eab10a840 0x561eab4d1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:59.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.739+0000 7f8e95600640 1 -- 172.21.15.16:0/2512836968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561eab102660 msgr2=0x561eab4d15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:59.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.739+0000 7f8e95600640 1 --2- 172.21.15.16:0/2512836968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561eab102660 0x561eab4d15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:59.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.739+0000 7f8e95600640 1 -- 172.21.15.16:0/2512836968 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561eab196b00 con 0x561eab12b520 2024-08-31T06:12:59.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.739+0000 7f8e95600640 1 --2- 172.21.15.16:0/2512836968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561eab12b520 0x561eab4d1090 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8e88009f00 tx=0x7f8e88011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdaf082de5cab70e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:59.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.739+0000 7f8e81c00640 1 -- 172.21.15.16:0/2512836968 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e88004d20 con 0x561eab12b520 2024-08-31T06:12:59.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.739+0000 7f8e81c00640 1 -- 172.21.15.16:0/2512836968 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e88004030 con 0x561eab12b520 2024-08-31T06:12:59.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.739+0000 7f8e81c00640 1 -- 172.21.15.16:0/2512836968 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e88005260 con 0x561eab12b520 2024-08-31T06:12:59.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.740+0000 7f8e8d600640 1 --2- 172.21.15.16:0/1470027295 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f8e780b4ca0 0x7f8e780b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:59.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.740+0000 7f8e8d600640 1 -- 172.21.15.16:0/1470027295 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f8e7c0e5600 con 0x7f8e90078030 2024-08-31T06:12:59.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.740+0000 7f8e94c00640 1 --2- 172.21.15.16:0/1470027295 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f8e780b4ca0 0x7f8e780b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:59.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.740+0000 7f8e9e1bc740 1 -- 172.21.15.16:0/2512836968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561eab12b520 msgr2=0x561eab4d1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:59.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.740+0000 7f8e9e1bc740 1 --2- 172.21.15.16:0/2512836968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561eab12b520 0x561eab4d1090 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8e88009f00 tx=0x7f8e88011810 comp rx=0 tx=0).stop 2024-08-31T06:12:59.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.741+0000 7f8e9e1bc740 1 -- 172.21.15.16:0/2512836968 shutdown_connections 2024-08-31T06:12:59.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.741+0000 7f8e9e1bc740 1 --2- 172.21.15.16:0/2512836968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561eab10a840 0x561eab4d1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:59.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.741+0000 7f8e9e1bc740 1 --2- 172.21.15.16:0/2512836968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561eab102660 0x561eab4d15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:59.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.741+0000 7f8e9e1bc740 1 --2- 172.21.15.16:0/2512836968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561eab12b520 0x561eab4d1090 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:59.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.741+0000 7f8e9e1bc740 1 -- 172.21.15.16:0/2512836968 >> 172.21.15.16:0/2512836968 conn(0x561eab11f910 msgr2=0x561eab4cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:59.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.741+0000 7f8e9e1bc740 1 -- 172.21.15.16:0/2512836968 shutdown_connections 2024-08-31T06:12:59.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.741+0000 7f8e9e1bc740 1 -- 172.21.15.16:0/2512836968 wait complete. 2024-08-31T06:12:59.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.741+0000 7f8e94c00640 1 --2- 172.21.15.16:0/1470027295 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f8e780b4ca0 0x7f8e780b71b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8e840096c0 tx=0x7f8e840058d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:59.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.741+0000 7f8e9e1bc740 1 -- start start 2024-08-31T06:12:59.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.742+0000 7f8e9e1bc740 1 Processor -- start 2024-08-31T06:12:59.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.742+0000 7f8e9e1bc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561eab102660 0x561eab4dee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:59.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.742+0000 7f8e9e1bc740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561eab10a840 0x561eab4dc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:59.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.742+0000 7f8e9e1bc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561eab12b520 0x561eab4dca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:59.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.742+0000 7f8e9e1bc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561eab299bb0 con 0x561eab12b520 2024-08-31T06:12:59.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.742+0000 7f8e9e1bc740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561eab3bdf90 con 0x561eab10a840 2024-08-31T06:12:59.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.742+0000 7f8e9e1bc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x561eab4df870 con 0x561eab102660 2024-08-31T06:12:59.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.742+0000 7f8e9e1bc740 10 client.0 _ll_register_callbacks cb 0x561eab3e3710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:12:59.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.742+0000 7f8e95600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561eab12b520 0x561eab4dca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:59.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.743+0000 7f8e95600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561eab12b520 0x561eab4dca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36240/0 (socket says 172.21.15.16:36240) 2024-08-31T06:12:59.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.743+0000 7f8e95600640 1 -- 172.21.15.16:0/1588161329 learned_addr learned my addr 172.21.15.16:0/1588161329 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:59.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.743+0000 7f8e94c00640 1 --2- 172.21.15.16:0/1588161329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561eab102660 0x561eab4dee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:59.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.743+0000 7f8e8fe00640 1 --2- 172.21.15.16:0/1588161329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561eab10a840 0x561eab4dc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:59.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.743+0000 7f8e95600640 1 -- 172.21.15.16:0/1588161329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561eab102660 msgr2=0x561eab4dee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:59.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.743+0000 7f8e95600640 1 --2- 172.21.15.16:0/1588161329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561eab102660 0x561eab4dee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:59.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.743+0000 7f8e95600640 1 -- 172.21.15.16:0/1588161329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561eab10a840 msgr2=0x561eab4dc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:59.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.743+0000 7f8e95600640 1 --2- 172.21.15.16:0/1588161329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561eab10a840 0x561eab4dc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:59.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.743+0000 7f8e95600640 1 -- 172.21.15.16:0/1588161329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561eab20d960 con 0x561eab12b520 2024-08-31T06:12:59.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.743+0000 7f8e95600640 1 --2- 172.21.15.16:0/1588161329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561eab12b520 0x561eab4dca10 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f8e88006010 tx=0x7f8e88004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d06d6d3f3e8c7924 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:59.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.743+0000 7f8e76000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:12:59.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.744+0000 7f8e76000640 1 -- 172.21.15.16:0/1588161329 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e88027020 con 0x561eab12b520 2024-08-31T06:12:59.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.744+0000 7f8e9e1bc740 1 -- 172.21.15.16:0/1588161329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561eab3d5e30 con 0x561eab12b520 2024-08-31T06:12:59.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.744+0000 7f8e76000640 1 -- 172.21.15.16:0/1588161329 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f8e88005310 con 0x561eab12b520 2024-08-31T06:12:59.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.744+0000 7f8e76000640 1 -- 172.21.15.16:0/1588161329 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f8e88032050 con 0x561eab12b520 2024-08-31T06:12:59.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.744+0000 7f8e9e1bc740 10 client.16674 fetch_fsmap learned FSMap version 38 2024-08-31T06:12:59.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.744+0000 7f8e9e1bc740 1 -- 172.21.15.16:0/1588161329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x561eab4eb1a0 con 0x561eab12b520 2024-08-31T06:12:59.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.744+0000 7f8e76000640 1 -- 172.21.15.16:0/1588161329 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561eab3d5e30 con 0x561eab12b520 2024-08-31T06:12:59.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.745+0000 7f8e76000640 1 -- 172.21.15.16:0/1588161329 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f8e88018070 con 0x561eab12b520 2024-08-31T06:12:59.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.745+0000 7f8e9e1bc740 10 client.16674 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T06:12:59.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.745+0000 7f8e9e1bc740 10 client.16674 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:12:59.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.745+0000 7f8e9e1bc740 20 client.16674 populate_metadata read hostname 'smithi016' 2024-08-31T06:12:59.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.745+0000 7f8e9e1bc740 1 --2- 172.21.15.16:0/1588161329 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x561eab0e9ea0 0x561eab4ee600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:59.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.745+0000 7f8e9e1bc740 4 client.16674 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:12:59.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.745+0000 7f8e9e1bc740 1 -- 172.21.15.16:0/1588161329 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561eab4499e0 con 0x561eab0e9ea0 2024-08-31T06:12:59.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.745+0000 7f8e94c00640 1 --2- 172.21.15.16:0/1588161329 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x561eab0e9ea0 0x561eab4ee600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:59.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.746+0000 7f8e94c00640 1 --2- 172.21.15.16:0/1588161329 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x561eab0e9ea0 0x561eab4ee600 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95c1b973ebe85cb0 server_cookie=c6ba76725c85ef49 in_seq=0 out_seq=0 2024-08-31T06:12:59.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.746+0000 7f8e76000640 10 client.16674 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:12:59.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.747+0000 7f8e76000640 1 -- 172.21.15.16:0/1588161329 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561eab4499e0 con 0x561eab0e9ea0 2024-08-31T06:12:59.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.747+0000 7f8e76000640 10 client.16674 handle_command_reply: tid=0 2024-08-31T06:12:59.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.747+0000 7f8e9e1bc740 1 client.16674 shutdown 2024-08-31T06:12:59.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.747+0000 7f8e9e1bc740 1 -- 172.21.15.16:0/1588161329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561eab12b520 msgr2=0x561eab4dca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:59.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.747+0000 7f8e9e1bc740 1 --2- 172.21.15.16:0/1588161329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561eab12b520 0x561eab4dca10 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f8e88006010 tx=0x7f8e88004f00 comp rx=0 tx=0).stop 2024-08-31T06:12:59.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.747+0000 7f8e9e1bc740 1 -- 172.21.15.16:0/1588161329 shutdown_connections 2024-08-31T06:12:59.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.747+0000 7f8e9e1bc740 1 --2- 172.21.15.16:0/1588161329 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x561eab0e9ea0 0x561eab4ee600 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:59.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.747+0000 7f8e9e1bc740 1 --2- 172.21.15.16:0/1588161329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561eab12b520 0x561eab4dca10 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:59.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.747+0000 7f8e9e1bc740 1 --2- 172.21.15.16:0/1588161329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561eab10a840 0x561eab4dc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:59.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.747+0000 7f8e9e1bc740 1 --2- 172.21.15.16:0/1588161329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561eab102660 0x561eab4dee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:59.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.747+0000 7f8e76000640 0 client.16674 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:12:59.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.747+0000 7f8e9e1bc740 1 -- 172.21.15.16:0/1588161329 >> 172.21.15.16:0/1588161329 conn(0x561eab11f910 msgr2=0x561eab4d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:59.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.748+0000 7f8e9e1bc740 1 -- 172.21.15.16:0/1588161329 shutdown_connections 2024-08-31T06:12:59.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.748+0000 7f8e9e1bc740 1 -- 172.21.15.16:0/1588161329 wait complete. 2024-08-31T06:12:59.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.748+0000 7f8e9e1bc740 20 client.16674 trim_cache size 0 max 16384 2024-08-31T06:12:59.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.766+0000 7f8e9e1bc740 1 Processor -- start 2024-08-31T06:12:59.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.766+0000 7f8e9e1bc740 1 -- start start 2024-08-31T06:12:59.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.766+0000 7f8e9e1bc740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561eab102660 0x561eab4d4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:59.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.766+0000 7f8e9e1bc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561eab10a840 0x561eab4d9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:59.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.766+0000 7f8e9e1bc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561eab12b520 0x561eab4d9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:59.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.766+0000 7f8e9e1bc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8e70010ff0 con 0x561eab10a840 2024-08-31T06:12:59.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.766+0000 7f8e9e1bc740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7000e830 con 0x561eab102660 2024-08-31T06:12:59.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.766+0000 7f8e9e1bc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7000c680 con 0x561eab12b520 2024-08-31T06:12:59.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.767+0000 7f8e8fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561eab10a840 0x561eab4d9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:59.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.767+0000 7f8e94c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561eab102660 0x561eab4d4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:59.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.767+0000 7f8e8fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561eab10a840 0x561eab4d9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36242/0 (socket says 172.21.15.16:36242) 2024-08-31T06:12:59.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.767+0000 7f8e8fe00640 1 -- 172.21.15.16:0/4199897303 learned_addr learned my addr 172.21.15.16:0/4199897303 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:59.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.767+0000 7f8e94c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561eab102660 0x561eab4d4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:38142/0 (socket says 172.21.15.16:38142) 2024-08-31T06:12:59.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.767+0000 7f8e95600640 1 --2- 172.21.15.16:0/4199897303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561eab12b520 0x561eab4d9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:59.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.767+0000 7f8e8fe00640 1 -- 172.21.15.16:0/4199897303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561eab12b520 msgr2=0x561eab4d9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:59.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.767+0000 7f8e8fe00640 1 --2- 172.21.15.16:0/4199897303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561eab12b520 0x561eab4d9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:59.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.767+0000 7f8e8fe00640 1 -- 172.21.15.16:0/4199897303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561eab102660 msgr2=0x561eab4d4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:59.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.767+0000 7f8e8fe00640 1 --2- 172.21.15.16:0/4199897303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561eab102660 0x561eab4d4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:59.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.767+0000 7f8e8fe00640 1 -- 172.21.15.16:0/4199897303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561eab4c9260 con 0x561eab10a840 2024-08-31T06:12:59.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.767+0000 7f8e95600640 1 --2- 172.21.15.16:0/4199897303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561eab12b520 0x561eab4d9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:12:59.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.767+0000 7f8e8fe00640 1 --2- 172.21.15.16:0/4199897303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561eab10a840 0x561eab4d9530 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c000c00 tx=0x7f8e7c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4aa8dcf3fe1971d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:59.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.768+0000 7f8e81200640 1 -- 172.21.15.16:0/4199897303 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c0ab510 con 0x561eab10a840 2024-08-31T06:12:59.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.768+0000 7f8e81200640 1 -- 172.21.15.16:0/4199897303 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e7c0ab8b0 con 0x561eab10a840 2024-08-31T06:12:59.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.768+0000 7f8e81200640 1 -- 172.21.15.16:0/4199897303 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c093200 con 0x561eab10a840 2024-08-31T06:12:59.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.768+0000 7f8e9e1bc740 1 -- 172.21.15.16:0/4199897303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561eab10a840 msgr2=0x561eab4d9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:59.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.768+0000 7f8e9e1bc740 1 --2- 172.21.15.16:0/4199897303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561eab10a840 0x561eab4d9530 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c000c00 tx=0x7f8e7c00cf90 comp rx=0 tx=0).stop 2024-08-31T06:12:59.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.769+0000 7f8e9e1bc740 1 -- 172.21.15.16:0/4199897303 shutdown_connections 2024-08-31T06:12:59.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.769+0000 7f8e9e1bc740 1 --2- 172.21.15.16:0/4199897303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561eab12b520 0x561eab4d9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:59.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.769+0000 7f8e9e1bc740 1 --2- 172.21.15.16:0/4199897303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561eab10a840 0x561eab4d9530 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:59.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.769+0000 7f8e9e1bc740 1 --2- 172.21.15.16:0/4199897303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561eab102660 0x561eab4d4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:59.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.769+0000 7f8e9e1bc740 1 -- 172.21.15.16:0/4199897303 >> 172.21.15.16:0/4199897303 conn(0x561eab11f910 msgr2=0x561eab4dc760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:59.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.769+0000 7f8e9e1bc740 1 -- 172.21.15.16:0/4199897303 shutdown_connections 2024-08-31T06:12:59.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.769+0000 7f8e9e1bc740 1 -- 172.21.15.16:0/4199897303 wait complete. 2024-08-31T06:12:59.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.769+0000 7f8e9e1bc740 1 -- start start 2024-08-31T06:12:59.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.769+0000 7f8e9e1bc740 1 Processor -- start 2024-08-31T06:12:59.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.770+0000 7f8e9e1bc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561eab102660 0x561eab4e1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:59.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.770+0000 7f8e9e1bc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561eab10a840 0x561eab4e17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:59.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.770+0000 7f8e9e1bc740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561eab12b520 0x561eab558000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:59.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.770+0000 7f8e9e1bc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7000ccf0 con 0x561eab10a840 2024-08-31T06:12:59.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.770+0000 7f8e9e1bc740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7000e160 con 0x561eab12b520 2024-08-31T06:12:59.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.770+0000 7f8e9e1bc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7000d3c0 con 0x561eab102660 2024-08-31T06:12:59.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.770+0000 7f8e9e1bc740 10 client.0 _ll_register_callbacks cb 0x7f8e88009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:12:59.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.770+0000 7f8e8fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561eab10a840 0x561eab4e17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:59.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.770+0000 7f8e8fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561eab10a840 0x561eab4e17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36244/0 (socket says 172.21.15.16:36244) 2024-08-31T06:12:59.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.770+0000 7f8e8fe00640 1 -- 172.21.15.16:0/1165271125 learned_addr learned my addr 172.21.15.16:0/1165271125 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:12:59.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.770+0000 7f8e94c00640 1 --2- 172.21.15.16:0/1165271125 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561eab102660 0x561eab4e1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:59.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.771+0000 7f8e95600640 1 --2- 172.21.15.16:0/1165271125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561eab12b520 0x561eab558000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:59.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.771+0000 7f8e8fe00640 1 -- 172.21.15.16:0/1165271125 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561eab102660 msgr2=0x561eab4e1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:59.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.771+0000 7f8e8fe00640 1 --2- 172.21.15.16:0/1165271125 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561eab102660 0x561eab4e1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:59.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.771+0000 7f8e8fe00640 1 -- 172.21.15.16:0/1165271125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561eab12b520 msgr2=0x561eab558000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:12:59.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.771+0000 7f8e8fe00640 1 --2- 172.21.15.16:0/1165271125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561eab12b520 0x561eab558000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:59.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.771+0000 7f8e8fe00640 1 -- 172.21.15.16:0/1165271125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561eab4dce70 con 0x561eab10a840 2024-08-31T06:12:59.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.771+0000 7f8e8fe00640 1 --2- 172.21.15.16:0/1165271125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561eab10a840 0x561eab4e17a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c0a9010 tx=0x7f8e7c0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e19175812330b3cd server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:12:59.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.771+0000 7f8e76000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:12:59.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.771+0000 7f8e76000640 1 -- 172.21.15.16:0/1165271125 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c0abda0 con 0x561eab10a840 2024-08-31T06:12:59.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.771+0000 7f8e76000640 1 -- 172.21.15.16:0/1165271125 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f8e7c091da0 con 0x561eab10a840 2024-08-31T06:12:59.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.771+0000 7f8e9e1bc740 1 -- 172.21.15.16:0/1165271125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e7c0ab8b0 con 0x561eab10a840 2024-08-31T06:12:59.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.772+0000 7f8e76000640 1 -- 172.21.15.16:0/1165271125 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f8e7c0b0560 con 0x561eab10a840 2024-08-31T06:12:59.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.772+0000 7f8e9e1bc740 10 client.16680 fetch_fsmap learned FSMap version 38 2024-08-31T06:12:59.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.772+0000 7f8e9e1bc740 1 -- 172.21.15.16:0/1165271125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x561eab559630 con 0x561eab10a840 2024-08-31T06:12:59.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.772+0000 7f8e76000640 1 -- 172.21.15.16:0/1165271125 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e7c0ab8b0 con 0x561eab10a840 2024-08-31T06:12:59.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.772+0000 7f8e76000640 1 -- 172.21.15.16:0/1165271125 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f8e7c0a77a0 con 0x561eab10a840 2024-08-31T06:12:59.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.772+0000 7f8e9e1bc740 10 client.16680 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T06:12:59.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.772+0000 7f8e9e1bc740 10 client.16680 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:12:59.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.772+0000 7f8e9e1bc740 20 client.16680 populate_metadata read hostname 'smithi016' 2024-08-31T06:12:59.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.772+0000 7f8e9e1bc740 1 --2- 172.21.15.16:0/1165271125 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x561eab0bf250 0x561eab55bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:12:59.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.772+0000 7f8e9e1bc740 4 client.16680 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2facd010", "release": true, "await": true}] 2024-08-31T06:12:59.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.772+0000 7f8e9e1bc740 1 -- 172.21.15.16:0/1165271125 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2facd010", "release": true, "await": true}) -- 0x561eab445480 con 0x561eab0bf250 2024-08-31T06:12:59.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.773+0000 7f8e94c00640 1 --2- 172.21.15.16:0/1165271125 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x561eab0bf250 0x561eab55bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:12:59.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.773+0000 7f8e94c00640 1 --2- 172.21.15.16:0/1165271125 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x561eab0bf250 0x561eab55bce0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cd6a662ad322772 server_cookie=a0e062a55359a4e7 in_seq=0 out_seq=0 2024-08-31T06:12:59.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.773+0000 7f8e76000640 10 client.16680 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:12:59.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.939+0000 7f8e76000640 1 -- 172.21.15.16:0/1165271125 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561eab445480 con 0x561eab0bf250 2024-08-31T06:12:59.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.939+0000 7f8e76000640 10 client.16680 handle_command_reply: tid=0 2024-08-31T06:12:59.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.939+0000 7f8e9e1bc740 1 client.16680 shutdown 2024-08-31T06:12:59.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.940+0000 7f8e9e1bc740 1 -- 172.21.15.16:0/1165271125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561eab10a840 msgr2=0x561eab4e17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:59.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.940+0000 7f8e9e1bc740 1 --2- 172.21.15.16:0/1165271125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561eab10a840 0x561eab4e17a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c0a9010 tx=0x7f8e7c0056d0 comp rx=0 tx=0).stop 2024-08-31T06:12:59.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.940+0000 7f8e9e1bc740 1 -- 172.21.15.16:0/1165271125 shutdown_connections 2024-08-31T06:12:59.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.940+0000 7f8e9e1bc740 1 --2- 172.21.15.16:0/1165271125 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x561eab0bf250 0x561eab55bce0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:59.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.940+0000 7f8e9e1bc740 1 --2- 172.21.15.16:0/1165271125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561eab12b520 0x561eab558000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:59.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.940+0000 7f8e9e1bc740 1 --2- 172.21.15.16:0/1165271125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561eab10a840 0x561eab4e17a0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:59.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.940+0000 7f8e9e1bc740 1 --2- 172.21.15.16:0/1165271125 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561eab102660 0x561eab4e1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:59.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.940+0000 7f8e9e1bc740 1 -- 172.21.15.16:0/1165271125 >> 172.21.15.16:0/1165271125 conn(0x561eab11f910 msgr2=0x561eab4dc760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:59.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.940+0000 7f8e76000640 0 client.16680 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:12:59.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.940+0000 7f8e9e1bc740 1 -- 172.21.15.16:0/1165271125 shutdown_connections 2024-08-31T06:12:59.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.940+0000 7f8e9e1bc740 1 -- 172.21.15.16:0/1165271125 wait complete. 2024-08-31T06:12:59.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.940+0000 7f8e9e1bc740 20 client.16680 trim_cache size 0 max 16384 2024-08-31T06:12:59.944 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:12:59.945 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 38, 2024-08-31T06:12:59.945 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24737, 2024-08-31T06:12:59.945 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 20, 2024-08-31T06:12:59.945 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-08-31T06:12:59.945 INFO:teuthology.orchestra.run.smithi016.stdout: "2facd010": { 2024-08-31T06:12:59.945 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 20, 2024-08-31T06:12:59.945 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-08-31T06:12:59.945 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:12:59.945 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-08-31T06:12:59.945 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:12:59.945 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:12:59.945 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-08-31T06:12:59.945 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 152.7, 2024-08-31T06:12:59.945 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-08-31T06:12:59.946 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-08-31T06:12:59.946 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-08-31T06:12:59.946 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:12:59.946 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-08-31T06:12:59.946 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:12:59.946 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:12:59.946 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:12:59.946 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:12:59.946 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:12:59.946 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:12:59.946 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:12:59.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.941+0000 7f8e96000640 1 -- 172.21.15.16:0/1470027295 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f8e780b4ca0 msgr2=0x7f8e780b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:59.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.941+0000 7f8e96000640 1 --2- 172.21.15.16:0/1470027295 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f8e780b4ca0 0x7f8e780b71b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8e840096c0 tx=0x7f8e840058d0 comp rx=0 tx=0).stop 2024-08-31T06:12:59.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.941+0000 7f8e96000640 1 -- 172.21.15.16:0/1470027295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e90078030 msgr2=0x7f8e901a0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:12:59.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.941+0000 7f8e96000640 1 --2- 172.21.15.16:0/1470027295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e90078030 0x7f8e901a0c40 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f8e9006c660 tx=0x7f8e7c00ee50 comp rx=0 tx=0).stop 2024-08-31T06:12:59.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.941+0000 7f8e96000640 1 -- 172.21.15.16:0/1470027295 shutdown_connections 2024-08-31T06:12:59.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.941+0000 7f8e96000640 1 --2- 172.21.15.16:0/1470027295 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f8e780b4ca0 0x7f8e780b71b0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:59.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.941+0000 7f8e96000640 1 --2- 172.21.15.16:0/1470027295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e90114310 0x7f8e901a5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:59.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.941+0000 7f8e96000640 1 --2- 172.21.15.16:0/1470027295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e90078030 0x7f8e901a0c40 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:59.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.941+0000 7f8e96000640 1 --2- 172.21.15.16:0/1470027295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e900776e0 0x7f8e901a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:12:59.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.941+0000 7f8e96000640 1 -- 172.21.15.16:0/1470027295 >> 172.21.15.16:0/1470027295 conn(0x7f8e90101420 msgr2=0x7f8e901163a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:12:59.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.942+0000 7f8e96000640 1 -- 172.21.15.16:0/1470027295 shutdown_connections 2024-08-31T06:12:59.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:12:59.942+0000 7f8e96000640 1 -- 172.21.15.16:0/1470027295 wait complete. 2024-08-31T06:13:00.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:00 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:00.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:00 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:00.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:00 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:00.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:00 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:00.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:00 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:00.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:00 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:00.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:00 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:00.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:00 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:00.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:00 smithi155 ceph-mon[21742]: Adjusting osd_memory_target on smithi016 to 177.5M 2024-08-31T06:13:00.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:00 smithi155 ceph-mon[21742]: Unable to set osd_memory_target on smithi016 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-08-31T06:13:00.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:00 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:13:00.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:00 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:13:00.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:00 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:13:00.381 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:00 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:00.381 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:00 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:00.381 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:00 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:00.381 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:00 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:00.381 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:00 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:00.381 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:00 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:00.382 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:00 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:00.382 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:00 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:00.382 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:00 smithi016 ceph-mon[17194]: Adjusting osd_memory_target on smithi016 to 177.5M 2024-08-31T06:13:00.382 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:00 smithi016 ceph-mon[17194]: Unable to set osd_memory_target on smithi016 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-08-31T06:13:00.382 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:00 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:13:00.382 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:00 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:13:00.382 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:00 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:13:00.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:00 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:00.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:00 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:00.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:00 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:00.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:00 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:00.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:00 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:00.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:00 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:00.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:00 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:00.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:00 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:00.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:00 smithi184 ceph-mon[22331]: Adjusting osd_memory_target on smithi016 to 177.5M 2024-08-31T06:13:00.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:00 smithi184 ceph-mon[22331]: Unable to set osd_memory_target on smithi016 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-08-31T06:13:00.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:00 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:13:00.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:00 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:13:00.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:00 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:13:01.091 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2facd010', total seconds elapsed: 37.3 2024-08-31T06:13:01.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:01 smithi155 ceph-mon[21742]: pgmap v537: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 8 op/s 2024-08-31T06:13:01.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:01 smithi016 ceph-mon[17194]: pgmap v537: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 8 op/s 2024-08-31T06:13:01.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:01 smithi184 ceph-mon[22331]: pgmap v537: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 8 op/s 2024-08-31T06:13:01.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 335.0 s, 580.6 tps, lat 1095.990 ms stddev 5518.271 2024-08-31T06:13:03.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:03 smithi155 ceph-mon[21742]: pgmap v538: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 98 KiB/s wr, 12 op/s 2024-08-31T06:13:03.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:03 smithi016 ceph-mon[17194]: pgmap v538: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 98 KiB/s wr, 12 op/s 2024-08-31T06:13:03.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:03 smithi184 ceph-mon[22331]: pgmap v538: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 98 KiB/s wr, 12 op/s 2024-08-31T06:13:05.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:05 smithi155 ceph-mon[21742]: pgmap v539: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 3.8 MiB/s wr, 190 op/s 2024-08-31T06:13:05.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:05 smithi016 ceph-mon[17194]: pgmap v539: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 3.8 MiB/s wr, 190 op/s 2024-08-31T06:13:05.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:05 smithi184 ceph-mon[22331]: pgmap v539: 97 pgs: 97 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 3.8 MiB/s wr, 190 op/s 2024-08-31T06:13:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.550+0000 7f8076400640 1 -- 172.21.15.16:0/57972794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8070109000 msgr2=0x7f8070109410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.550+0000 7f8076400640 1 --2- 172.21.15.16:0/57972794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8070109000 0x7f8070109410 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f805c009900 tx=0x7f805c02a3f0 comp rx=0 tx=0).stop 2024-08-31T06:13:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.551+0000 7f8076400640 1 -- 172.21.15.16:0/57972794 shutdown_connections 2024-08-31T06:13:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.551+0000 7f8076400640 1 --2- 172.21.15.16:0/57972794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8070109000 0x7f8070109410 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.551+0000 7f8076400640 1 --2- 172.21.15.16:0/57972794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8070109950 0x7f8070110180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.551+0000 7f8076400640 1 --2- 172.21.15.16:0/57972794 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80701036d0 0x7f8070103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.551+0000 7f8076400640 1 -- 172.21.15.16:0/57972794 >> 172.21.15.16:0/57972794 conn(0x7f80700ff130 msgr2=0x7f8070101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.551+0000 7f8076400640 1 -- 172.21.15.16:0/57972794 shutdown_connections 2024-08-31T06:13:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.551+0000 7f8076400640 1 -- 172.21.15.16:0/57972794 wait complete. 2024-08-31T06:13:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.553+0000 7f8076400640 1 Processor -- start 2024-08-31T06:13:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.553+0000 7f8076400640 1 -- start start 2024-08-31T06:13:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.553+0000 7f8076400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80701036d0 0x7f807019c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.554+0000 7f8076400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8070109000 0x7f807019c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.554+0000 7f8076400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8070109950 0x7f80701a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.554+0000 7f8076400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8070113610 con 0x7f8070109950 2024-08-31T06:13:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.554+0000 7f8076400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8070113490 con 0x7f80701036d0 2024-08-31T06:13:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.554+0000 7f8076400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8070113790 con 0x7f8070109000 2024-08-31T06:13:05.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.554+0000 7f8075000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80701036d0 0x7f807019c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:05.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.554+0000 7f8075000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80701036d0 0x7f807019c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:38156/0 (socket says 172.21.15.16:38156) 2024-08-31T06:13:05.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.554+0000 7f8075000640 1 -- 172.21.15.16:0/812759634 learned_addr learned my addr 172.21.15.16:0/812759634 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:05.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.554+0000 7f806fe00640 1 --2- 172.21.15.16:0/812759634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8070109000 0x7f807019c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:05.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.554+0000 7f8075000640 1 -- 172.21.15.16:0/812759634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8070109000 msgr2=0x7f807019c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:05.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.554+0000 7f8075a00640 1 --2- 172.21.15.16:0/812759634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8070109950 0x7f80701a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:05.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.554+0000 7f8075000640 1 --2- 172.21.15.16:0/812759634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8070109000 0x7f807019c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.554+0000 7f8075000640 1 -- 172.21.15.16:0/812759634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8070109950 msgr2=0x7f80701a1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.554+0000 7f8075000640 1 --2- 172.21.15.16:0/812759634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8070109950 0x7f80701a1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.554+0000 7f8075000640 1 -- 172.21.15.16:0/812759634 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80701a21c0 con 0x7f80701036d0 2024-08-31T06:13:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.557+0000 7f8075000640 1 --2- 172.21.15.16:0/812759634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80701036d0 0x7f807019c290 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f806400d950 tx=0x7f806400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.558+0000 7f806d600640 1 -- 172.21.15.16:0/812759634 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8064017070 con 0x7f80701036d0 2024-08-31T06:13:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.558+0000 7f806d600640 1 -- 172.21.15.16:0/812759634 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8064009c30 con 0x7f80701036d0 2024-08-31T06:13:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.558+0000 7f806d600640 1 -- 172.21.15.16:0/812759634 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806400f620 con 0x7f80701036d0 2024-08-31T06:13:05.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.558+0000 7f8076400640 1 -- 172.21.15.16:0/812759634 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80701a24b0 con 0x7f80701036d0 2024-08-31T06:13:05.561 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.558+0000 7f8076400640 1 -- 172.21.15.16:0/812759634 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80701a8c40 con 0x7f80701036d0 2024-08-31T06:13:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.564+0000 7f807eaaa740 1 Processor -- start 2024-08-31T06:13:05.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.566+0000 7f806d600640 1 -- 172.21.15.16:0/812759634 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f8064010460 con 0x7f80701036d0 2024-08-31T06:13:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.566+0000 7f807eaaa740 1 -- start start 2024-08-31T06:13:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.566+0000 7f807eaaa740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642c1e811b0 0x5642c2224b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.566+0000 7f807eaaa740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5642c1e582f0 0x5642c2225040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.567+0000 7f807eaaa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642c1e604d0 0x5642c2225580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.567+0000 7f807eaaa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5642c2199f70 con 0x5642c1e604d0 2024-08-31T06:13:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.567+0000 7f807eaaa740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5642c2189500 con 0x5642c1e582f0 2024-08-31T06:13:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.567+0000 7f807eaaa740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5642c218d4b0 con 0x5642c1e811b0 2024-08-31T06:13:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.567+0000 7f8075a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642c1e811b0 0x5642c2224b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.567+0000 7f8075a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642c1e811b0 0x5642c2224b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:41886/0 (socket says 172.21.15.16:41886) 2024-08-31T06:13:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.567+0000 7f8075a00640 1 -- 172.21.15.16:0/2547412669 learned_addr learned my addr 172.21.15.16:0/2547412669 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.567+0000 7f8075000640 1 --2- 172.21.15.16:0/2547412669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5642c1e582f0 0x5642c2225040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.567+0000 7f806fe00640 1 --2- 172.21.15.16:0/2547412669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642c1e604d0 0x5642c2225580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.567+0000 7f8075a00640 1 -- 172.21.15.16:0/2547412669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5642c1e582f0 msgr2=0x5642c2225040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.568+0000 7f8075a00640 1 --2- 172.21.15.16:0/2547412669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5642c1e582f0 0x5642c2225040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.568+0000 7f8075a00640 1 -- 172.21.15.16:0/2547412669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642c1e604d0 msgr2=0x5642c2225580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.568+0000 7f8075a00640 1 --2- 172.21.15.16:0/2547412669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642c1e604d0 0x5642c2225580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.568+0000 7f8075a00640 1 -- 172.21.15.16:0/2547412669 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642c2143c10 con 0x5642c1e811b0 2024-08-31T06:13:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.568+0000 7f8075a00640 1 --2- 172.21.15.16:0/2547412669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642c1e811b0 0x5642c2224b00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8068009eb0 tx=0x7f806800f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3480954979907ce6 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.568+0000 7f8062000640 1 -- 172.21.15.16:0/2547412669 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806800fea0 con 0x5642c1e811b0 2024-08-31T06:13:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.568+0000 7f8062000640 1 -- 172.21.15.16:0/2547412669 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8068011af0 con 0x5642c1e811b0 2024-08-31T06:13:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.568+0000 7f8062000640 1 -- 172.21.15.16:0/2547412669 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806801bc40 con 0x5642c1e811b0 2024-08-31T06:13:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.570+0000 7f807eaaa740 1 -- 172.21.15.16:0/2547412669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642c1e811b0 msgr2=0x5642c2224b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.570+0000 7f806d600640 1 --2- 172.21.15.16:0/812759634 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f80580b4e70 0x7f80580b7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:05.573 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.570+0000 7f807eaaa740 1 --2- 172.21.15.16:0/2547412669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642c1e811b0 0x5642c2224b00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8068009eb0 tx=0x7f806800f730 comp rx=0 tx=0).stop 2024-08-31T06:13:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.570+0000 7f806d600640 1 -- 172.21.15.16:0/812759634 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f80640e5160 con 0x7f80701036d0 2024-08-31T06:13:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.571+0000 7f8075000640 1 --2- 172.21.15.16:0/812759634 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f80580b4e70 0x7f80580b7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.571+0000 7f807eaaa740 1 -- 172.21.15.16:0/2547412669 shutdown_connections 2024-08-31T06:13:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.571+0000 7f807eaaa740 1 --2- 172.21.15.16:0/2547412669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642c1e604d0 0x5642c2225580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.571+0000 7f807eaaa740 1 --2- 172.21.15.16:0/2547412669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5642c1e582f0 0x5642c2225040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.571+0000 7f807eaaa740 1 --2- 172.21.15.16:0/2547412669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642c1e811b0 0x5642c2224b00 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.571+0000 7f807eaaa740 1 -- 172.21.15.16:0/2547412669 >> 172.21.15.16:0/2547412669 conn(0x5642c1e75300 msgr2=0x5642c22229f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.571+0000 7f807eaaa740 1 -- 172.21.15.16:0/2547412669 shutdown_connections 2024-08-31T06:13:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.572+0000 7f807eaaa740 1 -- 172.21.15.16:0/2547412669 wait complete. 2024-08-31T06:13:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.572+0000 7f8075000640 1 --2- 172.21.15.16:0/812759634 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f80580b4e70 0x7f80580b7380 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f80640ab550 tx=0x7f8064005880 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.573+0000 7f807eaaa740 1 -- start start 2024-08-31T06:13:05.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.573+0000 7f807eaaa740 1 Processor -- start 2024-08-31T06:13:05.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.574+0000 7f807eaaa740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5642c1e582f0 0x5642c22309b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:05.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.574+0000 7f807eaaa740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642c1e604d0 0x5642c223b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:05.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.574+0000 7f807eaaa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642c1e811b0 0x5642c2231120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:05.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.574+0000 7f807eaaa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5642c20c2ab0 con 0x5642c1e811b0 2024-08-31T06:13:05.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.574+0000 7f807eaaa740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5642c2130320 con 0x5642c1e582f0 2024-08-31T06:13:05.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.574+0000 7f807eaaa740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5642c222b1e0 con 0x5642c1e604d0 2024-08-31T06:13:05.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.574+0000 7f807eaaa740 10 client.0 _ll_register_callbacks cb 0x5642c21366a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:13:05.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.575+0000 7f806fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642c1e604d0 0x5642c223b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:05.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.575+0000 7f806fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642c1e604d0 0x5642c223b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:41898/0 (socket says 172.21.15.16:41898) 2024-08-31T06:13:05.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.575+0000 7f806fe00640 1 -- 172.21.15.16:0/2173777206 learned_addr learned my addr 172.21.15.16:0/2173777206 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:05.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.575+0000 7f8075a00640 1 --2- 172.21.15.16:0/2173777206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5642c1e582f0 0x5642c22309b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:05.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.575+0000 7f8075a00640 1 --2- 172.21.15.16:0/2173777206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642c1e811b0 0x5642c2231120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:05.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.575+0000 7f806fe00640 1 -- 172.21.15.16:0/2173777206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5642c1e582f0 msgr2=0x5642c22309b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:05.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.575+0000 7f806fe00640 1 --2- 172.21.15.16:0/2173777206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5642c1e582f0 0x5642c22309b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:05.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.575+0000 7f806fe00640 1 -- 172.21.15.16:0/2173777206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642c1e811b0 msgr2=0x5642c2231120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:05.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.575+0000 7f806fe00640 1 --2- 172.21.15.16:0/2173777206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642c1e811b0 0x5642c2231120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:05.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.575+0000 7f806fe00640 1 -- 172.21.15.16:0/2173777206 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642c1eec790 con 0x5642c1e604d0 2024-08-31T06:13:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.576+0000 7f806fe00640 1 --2- 172.21.15.16:0/2173777206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642c1e604d0 0x5642c223b130 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f805c006010 tx=0x7f805c005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa541ab5e5fead85 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.576+0000 7f8056a00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:13:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.576+0000 7f8056a00640 1 -- 172.21.15.16:0/2173777206 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c035030 con 0x5642c1e604d0 2024-08-31T06:13:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.576+0000 7f8056a00640 1 -- 172.21.15.16:0/2173777206 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f805c0046c0 con 0x5642c1e604d0 2024-08-31T06:13:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.577+0000 7f807eaaa740 1 -- 172.21.15.16:0/2173777206 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8068011af0 con 0x5642c1e604d0 2024-08-31T06:13:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.577+0000 7f8056a00640 1 -- 172.21.15.16:0/2173777206 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f805c0048c0 con 0x5642c1e604d0 2024-08-31T06:13:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.577+0000 7f807eaaa740 10 client.26881 fetch_fsmap learned FSMap version 38 2024-08-31T06:13:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.577+0000 7f807eaaa740 1 -- 172.21.15.16:0/2173777206 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5642c1f65620 con 0x5642c1e604d0 2024-08-31T06:13:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.577+0000 7f8056a00640 1 -- 172.21.15.16:0/2173777206 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8068011af0 con 0x5642c1e604d0 2024-08-31T06:13:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.578+0000 7f8056a00640 1 -- 172.21.15.16:0/2173777206 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f805c003c60 con 0x5642c1e604d0 2024-08-31T06:13:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.578+0000 7f807eaaa740 10 client.26881 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T06:13:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.578+0000 7f807eaaa740 10 client.26881 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:13:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.578+0000 7f807eaaa740 20 client.26881 populate_metadata read hostname 'smithi016' 2024-08-31T06:13:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.578+0000 7f807eaaa740 1 --2- 172.21.15.16:0/2173777206 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5642c1e3fd80 0x5642c2253250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.578+0000 7f807eaaa740 4 client.26881 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:13:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.578+0000 7f807eaaa740 1 -- 172.21.15.16:0/2173777206 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642c219bec0 con 0x5642c1e3fd80 2024-08-31T06:13:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.580+0000 7f8075a00640 1 --2- 172.21.15.16:0/2173777206 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5642c1e3fd80 0x5642c2253250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.581+0000 7f8075a00640 1 --2- 172.21.15.16:0/2173777206 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5642c1e3fd80 0x5642c2253250 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3539e0d880ca81fa server_cookie=c0db5145f9ff64e6 in_seq=0 out_seq=0 2024-08-31T06:13:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.582+0000 7f8056a00640 10 client.26881 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:13:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.582+0000 7f8056a00640 1 -- 172.21.15.16:0/2173777206 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5642c219bec0 con 0x5642c1e3fd80 2024-08-31T06:13:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.582+0000 7f8056a00640 10 client.26881 handle_command_reply: tid=0 2024-08-31T06:13:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.582+0000 7f807eaaa740 1 client.26881 shutdown 2024-08-31T06:13:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.583+0000 7f807eaaa740 1 -- 172.21.15.16:0/2173777206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642c1e604d0 msgr2=0x5642c223b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.583+0000 7f807eaaa740 1 --2- 172.21.15.16:0/2173777206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642c1e604d0 0x5642c223b130 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f805c006010 tx=0x7f805c005270 comp rx=0 tx=0).stop 2024-08-31T06:13:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.583+0000 7f807eaaa740 1 -- 172.21.15.16:0/2173777206 shutdown_connections 2024-08-31T06:13:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.583+0000 7f807eaaa740 1 --2- 172.21.15.16:0/2173777206 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5642c1e3fd80 0x5642c2253250 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.583+0000 7f807eaaa740 1 --2- 172.21.15.16:0/2173777206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642c1e604d0 0x5642c223b130 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.583+0000 7f8056a00640 0 client.26881 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:13:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.583+0000 7f807eaaa740 1 --2- 172.21.15.16:0/2173777206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642c1e811b0 0x5642c2231120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.583+0000 7f807eaaa740 1 --2- 172.21.15.16:0/2173777206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5642c1e582f0 0x5642c22309b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.583+0000 7f807eaaa740 1 -- 172.21.15.16:0/2173777206 >> 172.21.15.16:0/2173777206 conn(0x5642c1e75300 msgr2=0x5642c2227c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.583+0000 7f807eaaa740 1 -- 172.21.15.16:0/2173777206 shutdown_connections 2024-08-31T06:13:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.583+0000 7f807eaaa740 1 -- 172.21.15.16:0/2173777206 wait complete. 2024-08-31T06:13:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.583+0000 7f807eaaa740 20 client.26881 trim_cache size 0 max 16384 2024-08-31T06:13:05.612 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.610+0000 7f807eaaa740 1 Processor -- start 2024-08-31T06:13:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.610+0000 7f807eaaa740 1 -- start start 2024-08-31T06:13:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.611+0000 7f807eaaa740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642c1e582f0 0x5642c2230380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.611+0000 7f807eaaa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642c1e604d0 0x5642c222d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.611+0000 7f807eaaa740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5642c1e811b0 0x5642c222deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.611+0000 7f807eaaa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f80701af9d0 con 0x5642c1e604d0 2024-08-31T06:13:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.611+0000 7f807eaaa740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f80701ad210 con 0x5642c1e811b0 2024-08-31T06:13:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.611+0000 7f807eaaa740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f80701ab0c0 con 0x5642c1e582f0 2024-08-31T06:13:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.611+0000 7f8075a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642c1e582f0 0x5642c2230380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.611+0000 7f8075a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642c1e582f0 0x5642c2230380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:41910/0 (socket says 172.21.15.16:41910) 2024-08-31T06:13:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.611+0000 7f8075a00640 1 -- 172.21.15.16:0/3883059686 learned_addr learned my addr 172.21.15.16:0/3883059686 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.611+0000 7f8075a00640 1 --2- 172.21.15.16:0/3883059686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5642c1e811b0 0x5642c222deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.611+0000 7f806fe00640 1 --2- 172.21.15.16:0/3883059686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642c1e604d0 0x5642c222d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.612+0000 7f8075a00640 1 -- 172.21.15.16:0/3883059686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5642c1e811b0 msgr2=0x5642c222deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.612+0000 7f8075a00640 1 --2- 172.21.15.16:0/3883059686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5642c1e811b0 0x5642c222deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.612+0000 7f8075a00640 1 -- 172.21.15.16:0/3883059686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642c1e604d0 msgr2=0x5642c222d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.612+0000 7f8075a00640 1 --2- 172.21.15.16:0/3883059686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642c1e604d0 0x5642c222d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.612+0000 7f8075a00640 1 -- 172.21.15.16:0/3883059686 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642c222ec80 con 0x5642c1e582f0 2024-08-31T06:13:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.613+0000 7f806fe00640 1 --2- 172.21.15.16:0/3883059686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642c1e604d0 0x5642c222d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:13:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.613+0000 7f8075a00640 1 --2- 172.21.15.16:0/3883059686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642c1e582f0 0x5642c2230380 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8068009b60 tx=0x7f8068000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f20a49ff3adaecc4 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.613+0000 7f8061600640 1 -- 172.21.15.16:0/3883059686 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8068003a60 con 0x5642c1e582f0 2024-08-31T06:13:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.613+0000 7f8061600640 1 -- 172.21.15.16:0/3883059686 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f806802c040 con 0x5642c1e582f0 2024-08-31T06:13:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.613+0000 7f8061600640 1 -- 172.21.15.16:0/3883059686 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8068011730 con 0x5642c1e582f0 2024-08-31T06:13:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.614+0000 7f807eaaa740 1 -- 172.21.15.16:0/3883059686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642c1e582f0 msgr2=0x5642c2230380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.614+0000 7f807eaaa740 1 --2- 172.21.15.16:0/3883059686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642c1e582f0 0x5642c2230380 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8068009b60 tx=0x7f8068000f40 comp rx=0 tx=0).stop 2024-08-31T06:13:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.615+0000 7f807eaaa740 1 -- 172.21.15.16:0/3883059686 shutdown_connections 2024-08-31T06:13:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.615+0000 7f807eaaa740 1 --2- 172.21.15.16:0/3883059686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5642c1e811b0 0x5642c222deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.615+0000 7f807eaaa740 1 --2- 172.21.15.16:0/3883059686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642c1e604d0 0x5642c222d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.615+0000 7f807eaaa740 1 --2- 172.21.15.16:0/3883059686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642c1e582f0 0x5642c2230380 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.615+0000 7f807eaaa740 1 -- 172.21.15.16:0/3883059686 >> 172.21.15.16:0/3883059686 conn(0x5642c1e75300 msgr2=0x5642c222fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.615+0000 7f807eaaa740 1 -- 172.21.15.16:0/3883059686 shutdown_connections 2024-08-31T06:13:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.615+0000 7f807eaaa740 1 -- 172.21.15.16:0/3883059686 wait complete. 2024-08-31T06:13:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.616+0000 7f807eaaa740 1 -- start start 2024-08-31T06:13:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.616+0000 7f807eaaa740 1 Processor -- start 2024-08-31T06:13:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.617+0000 7f807eaaa740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642c1e582f0 0x5642c22608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.617+0000 7f807eaaa740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5642c1e604d0 0x5642c2260df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.617+0000 7f807eaaa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642c1e811b0 0x5642c2261330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.617+0000 7f807eaaa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f80701ab710 con 0x5642c1e811b0 2024-08-31T06:13:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.617+0000 7f807eaaa740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f80701acb40 con 0x5642c1e604d0 2024-08-31T06:13:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.617+0000 7f807eaaa740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f80701abda0 con 0x5642c1e582f0 2024-08-31T06:13:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.617+0000 7f807eaaa740 10 client.0 _ll_register_callbacks cb 0x7f805c004500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:13:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.617+0000 7f8075a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642c1e582f0 0x5642c22608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.617+0000 7f8075a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642c1e582f0 0x5642c22608b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:41926/0 (socket says 172.21.15.16:41926) 2024-08-31T06:13:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.617+0000 7f8075a00640 1 -- 172.21.15.16:0/1880677256 learned_addr learned my addr 172.21.15.16:0/1880677256 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.617+0000 7f806fe00640 1 --2- 172.21.15.16:0/1880677256 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5642c1e604d0 0x5642c2260df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.617+0000 7f8075a00640 1 --2- 172.21.15.16:0/1880677256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642c1e811b0 0x5642c2261330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.618+0000 7f8075a00640 1 -- 172.21.15.16:0/1880677256 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5642c1e604d0 msgr2=0x5642c2260df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.618+0000 7f8075a00640 1 --2- 172.21.15.16:0/1880677256 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5642c1e604d0 0x5642c2260df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.618+0000 7f8075a00640 1 -- 172.21.15.16:0/1880677256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642c1e811b0 msgr2=0x5642c2261330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.618+0000 7f8075a00640 1 --2- 172.21.15.16:0/1880677256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642c1e811b0 0x5642c2261330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.618+0000 7f8075a00640 1 -- 172.21.15.16:0/1880677256 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642c22bcbc0 con 0x5642c1e582f0 2024-08-31T06:13:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.618+0000 7f806fe00640 1 --2- 172.21.15.16:0/1880677256 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5642c1e604d0 0x5642c2260df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:13:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.618+0000 7f8075a00640 1 --2- 172.21.15.16:0/1880677256 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642c1e582f0 0x5642c22608b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8068027040 tx=0x7f8068003f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11720cf1e797e780 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.618+0000 7f8056a00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:13:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.618+0000 7f8056a00640 1 -- 172.21.15.16:0/1880677256 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8068003a60 con 0x5642c1e582f0 2024-08-31T06:13:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.618+0000 7f8056a00640 1 -- 172.21.15.16:0/1880677256 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8426+0+0 (secure 0 0 0) 0x7f8068003c00 con 0x5642c1e582f0 2024-08-31T06:13:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.618+0000 7f807eaaa740 1 -- 172.21.15.16:0/1880677256 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f806802c040 con 0x5642c1e582f0 2024-08-31T06:13:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.619+0000 7f8056a00640 1 -- 172.21.15.16:0/1880677256 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f806801b810 con 0x5642c1e582f0 2024-08-31T06:13:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.619+0000 7f807eaaa740 10 client.26887 fetch_fsmap learned FSMap version 38 2024-08-31T06:13:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.619+0000 7f807eaaa740 1 -- 172.21.15.16:0/1880677256 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5642c22bddb0 con 0x5642c1e582f0 2024-08-31T06:13:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.619+0000 7f8056a00640 1 -- 172.21.15.16:0/1880677256 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f806802c040 con 0x5642c1e582f0 2024-08-31T06:13:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.621+0000 7f8056a00640 1 -- 172.21.15.16:0/1880677256 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f8068030070 con 0x5642c1e582f0 2024-08-31T06:13:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.621+0000 7f807eaaa740 10 client.26887 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T06:13:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.621+0000 7f807eaaa740 10 client.26887 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:13:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.621+0000 7f807eaaa740 20 client.26887 populate_metadata read hostname 'smithi016' 2024-08-31T06:13:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.622+0000 7f807eaaa740 1 --2- 172.21.15.16:0/1880677256 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5642c1e151c0 0x5642c22c0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.622+0000 7f807eaaa740 4 client.26887 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:13:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.622+0000 7f807eaaa740 1 -- 172.21.15.16:0/1880677256 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5642c2197960 con 0x5642c1e151c0 2024-08-31T06:13:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.622+0000 7f806fe00640 1 --2- 172.21.15.16:0/1880677256 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5642c1e151c0 0x5642c22c0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.626+0000 7f806fe00640 1 --2- 172.21.15.16:0/1880677256 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5642c1e151c0 0x5642c22c0290 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbf2059cacd7c45c server_cookie=b67d74f2731eadfa in_seq=0 out_seq=0 2024-08-31T06:13:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.626+0000 7f8056a00640 10 client.26887 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:13:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.656+0000 7f8056a00640 1 -- 172.21.15.16:0/1880677256 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5642c2197960 con 0x5642c1e151c0 2024-08-31T06:13:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.656+0000 7f8056a00640 10 client.26887 handle_command_reply: tid=0 2024-08-31T06:13:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.656+0000 7f807eaaa740 1 client.26887 shutdown 2024-08-31T06:13:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.657+0000 7f807eaaa740 1 -- 172.21.15.16:0/1880677256 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642c1e582f0 msgr2=0x5642c22608b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.657+0000 7f807eaaa740 1 --2- 172.21.15.16:0/1880677256 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642c1e582f0 0x5642c22608b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8068027040 tx=0x7f8068003f60 comp rx=0 tx=0).stop 2024-08-31T06:13:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.658+0000 7f807eaaa740 1 -- 172.21.15.16:0/1880677256 shutdown_connections 2024-08-31T06:13:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.658+0000 7f807eaaa740 1 --2- 172.21.15.16:0/1880677256 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5642c1e151c0 0x5642c22c0290 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.658+0000 7f807eaaa740 1 --2- 172.21.15.16:0/1880677256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642c1e811b0 0x5642c2261330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.658+0000 7f807eaaa740 1 --2- 172.21.15.16:0/1880677256 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5642c1e604d0 0x5642c2260df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.658+0000 7f807eaaa740 1 --2- 172.21.15.16:0/1880677256 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5642c1e582f0 0x5642c22608b0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.658+0000 7f807eaaa740 1 -- 172.21.15.16:0/1880677256 >> 172.21.15.16:0/1880677256 conn(0x5642c1e75300 msgr2=0x5642c2225b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.659+0000 7f8056a00640 0 client.26887 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:13:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.659+0000 7f807eaaa740 1 -- 172.21.15.16:0/1880677256 shutdown_connections 2024-08-31T06:13:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.659+0000 7f807eaaa740 1 -- 172.21.15.16:0/1880677256 wait complete. 2024-08-31T06:13:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.660+0000 7f807eaaa740 20 client.26887 trim_cache size 0 max 16384 2024-08-31T06:13:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:13:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:13:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:13:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:13:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.662+0000 7f8076400640 1 -- 172.21.15.16:0/812759634 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f80580b4e70 msgr2=0x7f80580b7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.662+0000 7f8076400640 1 --2- 172.21.15.16:0/812759634 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f80580b4e70 0x7f80580b7380 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f80640ab550 tx=0x7f8064005880 comp rx=0 tx=0).stop 2024-08-31T06:13:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.663+0000 7f8076400640 1 -- 172.21.15.16:0/812759634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80701036d0 msgr2=0x7f807019c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.663+0000 7f8076400640 1 --2- 172.21.15.16:0/812759634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80701036d0 0x7f807019c290 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f806400d950 tx=0x7f806400de10 comp rx=0 tx=0).stop 2024-08-31T06:13:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.663+0000 7f8076400640 1 -- 172.21.15.16:0/812759634 shutdown_connections 2024-08-31T06:13:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.663+0000 7f8076400640 1 --2- 172.21.15.16:0/812759634 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f80580b4e70 0x7f80580b7380 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.663+0000 7f8076400640 1 --2- 172.21.15.16:0/812759634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8070109000 0x7f807019c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.663+0000 7f8076400640 1 --2- 172.21.15.16:0/812759634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8070109950 0x7f80701a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.663+0000 7f8076400640 1 --2- 172.21.15.16:0/812759634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80701036d0 0x7f807019c290 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.663+0000 7f8076400640 1 -- 172.21.15.16:0/812759634 >> 172.21.15.16:0/812759634 conn(0x7f80700ff130 msgr2=0x7f8070108700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.663+0000 7f8076400640 1 -- 172.21.15.16:0/812759634 shutdown_connections 2024-08-31T06:13:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:05.663+0000 7f8076400640 1 -- 172.21.15.16:0/812759634 wait complete. 2024-08-31T06:13:06.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 340.0 s, 1464.8 tps, lat 68.637 ms stddev 117.802 2024-08-31T06:13:07.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:07 smithi016 ceph-mon[17194]: pgmap v540: 97 pgs: 97 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 13 MiB/s wr, 922 op/s 2024-08-31T06:13:07.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:07 smithi184 ceph-mon[22331]: pgmap v540: 97 pgs: 97 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 13 MiB/s wr, 922 op/s 2024-08-31T06:13:07.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:07 smithi155 ceph-mon[21742]: pgmap v540: 97 pgs: 97 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 13 MiB/s wr, 922 op/s 2024-08-31T06:13:09.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:09 smithi016 ceph-mon[17194]: pgmap v541: 97 pgs: 97 active+clean; 3.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 18 MiB/s wr, 1.22k op/s 2024-08-31T06:13:09.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:09 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:13:09.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:09 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:13:09.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:09 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:13:09.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:09 smithi184 ceph-mon[22331]: pgmap v541: 97 pgs: 97 active+clean; 3.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 18 MiB/s wr, 1.22k op/s 2024-08-31T06:13:09.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:09 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:13:09.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:09 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:13:09.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:09 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:13:09.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:09 smithi155 ceph-mon[21742]: pgmap v541: 97 pgs: 97 active+clean; 3.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 18 MiB/s wr, 1.22k op/s 2024-08-31T06:13:09.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:09 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:13:09.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:09 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:13:09.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:09 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:13:10.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:10 smithi016 ceph-mon[17194]: osdmap e82: 12 total, 12 up, 12 in 2024-08-31T06:13:10.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:10 smithi184 ceph-mon[22331]: osdmap e82: 12 total, 12 up, 12 in 2024-08-31T06:13:10.523 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:10 smithi155 ceph-mon[21742]: osdmap e82: 12 total, 12 up, 12 in 2024-08-31T06:13:11.048 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T06:13:11.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:11 smithi016 ceph-mon[17194]: pgmap v543: 97 pgs: 97 active+clean; 3.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 31 MiB/s wr, 1.77k op/s 2024-08-31T06:13:11.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:11 smithi184 ceph-mon[22331]: pgmap v543: 97 pgs: 97 active+clean; 3.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 31 MiB/s wr, 1.77k op/s 2024-08-31T06:13:11.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:11 smithi155 ceph-mon[21742]: pgmap v543: 97 pgs: 97 active+clean; 3.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 31 MiB/s wr, 1.77k op/s 2024-08-31T06:13:11.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 345.0 s, 2212.4 tps, lat 45.215 ms stddev 56.957 2024-08-31T06:13:11.660 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:13:12.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.720+0000 7f2235200640 1 -- 172.21.15.16:0/238615284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2230109090 msgr2=0x7f22301094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:12.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.720+0000 7f2235200640 1 --2- 172.21.15.16:0/238615284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2230109090 0x7f22301094a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f221c009860 tx=0x7f221c02a340 comp rx=0 tx=0).stop 2024-08-31T06:13:12.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.720+0000 7f2235200640 1 -- 172.21.15.16:0/238615284 shutdown_connections 2024-08-31T06:13:12.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.721+0000 7f2235200640 1 --2- 172.21.15.16:0/238615284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22301099e0 0x7f2230110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:12.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.721+0000 7f2235200640 1 --2- 172.21.15.16:0/238615284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2230109090 0x7f22301094a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:12.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.721+0000 7f2235200640 1 --2- 172.21.15.16:0/238615284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2230103750 0x7f2230103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:12.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.721+0000 7f2235200640 1 -- 172.21.15.16:0/238615284 >> 172.21.15.16:0/238615284 conn(0x7f22300ff1e0 msgr2=0x7f2230101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:12.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.721+0000 7f2235200640 1 -- 172.21.15.16:0/238615284 shutdown_connections 2024-08-31T06:13:12.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.722+0000 7f2235200640 1 -- 172.21.15.16:0/238615284 wait complete. 2024-08-31T06:13:12.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.724+0000 7f2235200640 1 Processor -- start 2024-08-31T06:13:12.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.724+0000 7f2235200640 1 -- start start 2024-08-31T06:13:12.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.726+0000 7f2235200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2230103750 0x7f223019a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:12.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.726+0000 7f2235200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2230109090 0x7f223019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:12.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.726+0000 7f2235200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22301099e0 0x7f223019f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:12.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.726+0000 7f2235200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2230113650 con 0x7f2230103750 2024-08-31T06:13:12.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.726+0000 7f2235200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f22301134d0 con 0x7f2230109090 2024-08-31T06:13:12.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.726+0000 7f2235200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f22301137d0 con 0x7f22301099e0 2024-08-31T06:13:12.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.726+0000 7f222fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22301099e0 0x7f223019f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:12.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.726+0000 7f222fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22301099e0 0x7f223019f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:36380/0 (socket says 172.21.15.16:36380) 2024-08-31T06:13:12.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.726+0000 7f222fe00640 1 -- 172.21.15.16:0/2225370129 learned_addr learned my addr 172.21.15.16:0/2225370129 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:12.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.727+0000 7f222ea00640 1 --2- 172.21.15.16:0/2225370129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2230109090 0x7f223019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:12.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.727+0000 7f222fe00640 1 -- 172.21.15.16:0/2225370129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2230109090 msgr2=0x7f223019a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:12.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.727+0000 7f222f400640 1 --2- 172.21.15.16:0/2225370129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2230103750 0x7f223019a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:12.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.727+0000 7f222fe00640 1 --2- 172.21.15.16:0/2225370129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2230109090 0x7f223019a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:12.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.728+0000 7f222fe00640 1 -- 172.21.15.16:0/2225370129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2230103750 msgr2=0x7f223019a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:12.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.728+0000 7f222fe00640 1 --2- 172.21.15.16:0/2225370129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2230103750 0x7f223019a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:12.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.728+0000 7f222fe00640 1 -- 172.21.15.16:0/2225370129 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22301a0010 con 0x7f22301099e0 2024-08-31T06:13:12.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.729+0000 7f222f400640 1 --2- 172.21.15.16:0/2225370129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2230103750 0x7f223019a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:13:12.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.729+0000 7f222fe00640 1 --2- 172.21.15.16:0/2225370129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22301099e0 0x7f223019f8e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f222800eaa0 tx=0x7f222800ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:12.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.729+0000 7f222ea00640 1 --2- 172.21.15.16:0/2225370129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2230109090 0x7f223019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:13:12.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.729+0000 7f2227e00640 1 -- 172.21.15.16:0/2225370129 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222800cdb0 con 0x7f22301099e0 2024-08-31T06:13:12.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.729+0000 7f2227e00640 1 -- 172.21.15.16:0/2225370129 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2228005020 con 0x7f22301099e0 2024-08-31T06:13:12.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.729+0000 7f2227e00640 1 -- 172.21.15.16:0/2225370129 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22280116b0 con 0x7f22301099e0 2024-08-31T06:13:12.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.729+0000 7f2235200640 1 -- 172.21.15.16:0/2225370129 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22301a0300 con 0x7f22301099e0 2024-08-31T06:13:12.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.730+0000 7f2235200640 1 -- 172.21.15.16:0/2225370129 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2230075390 con 0x7f22301099e0 2024-08-31T06:13:12.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.735+0000 7f2227e00640 1 -- 172.21.15.16:0/2225370129 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f22280054b0 con 0x7f22301099e0 2024-08-31T06:13:12.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.736+0000 7f223d44f740 1 Processor -- start 2024-08-31T06:13:12.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.736+0000 7f223d44f740 1 -- start start 2024-08-31T06:13:12.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.737+0000 7f223d44f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55916febc070 0x55917026a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:12.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.737+0000 7f223d44f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55916fe931b0 0x55917026a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:12.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.737+0000 7f223d44f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55916fe9b390 0x55917026adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:12.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.737+0000 7f223d44f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5591701d8770 con 0x55916fe931b0 2024-08-31T06:13:12.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.737+0000 7f223d44f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5591701c7d00 con 0x55916febc070 2024-08-31T06:13:12.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.737+0000 7f223d44f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5591701cbcb0 con 0x55916fe9b390 2024-08-31T06:13:12.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.739+0000 7f222ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55916febc070 0x55917026a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:12.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.739+0000 7f222f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55916fe931b0 0x55917026a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:12.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.739+0000 7f222ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55916febc070 0x55917026a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:56686/0 (socket says 172.21.15.16:56686) 2024-08-31T06:13:12.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.739+0000 7f222ea00640 1 -- 172.21.15.16:0/2591471119 learned_addr learned my addr 172.21.15.16:0/2591471119 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:12.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.739+0000 7f222ea00640 1 --2- 172.21.15.16:0/2591471119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55916fe9b390 0x55917026adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:12.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.740+0000 7f2227e00640 1 --2- 172.21.15.16:0/2225370129 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f22140b4e70 0x7f22140b7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:12.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.740+0000 7f222ea00640 1 -- 172.21.15.16:0/2591471119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55916fe9b390 msgr2=0x55917026adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:12.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.740+0000 7f222ea00640 1 --2- 172.21.15.16:0/2591471119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55916fe9b390 0x55917026adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:12.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.740+0000 7f222ea00640 1 -- 172.21.15.16:0/2591471119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55916fe931b0 msgr2=0x55917026a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:12.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.740+0000 7f222ea00640 1 --2- 172.21.15.16:0/2591471119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55916fe931b0 0x55917026a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:12.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.740+0000 7f222ea00640 1 -- 172.21.15.16:0/2591471119 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55916ff27650 con 0x55916febc070 2024-08-31T06:13:12.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.741+0000 7f222ea00640 1 --2- 172.21.15.16:0/2591471119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55916febc070 0x55917026a330 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f221c02a860 tx=0x7f221c009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56c7d7b7d928e2cd server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:12.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.741+0000 7f2227e00640 1 -- 172.21.15.16:0/2225370129 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8514+0+0 (secure 0 0 0) 0x7f22280e5740 con 0x7f22301099e0 2024-08-31T06:13:12.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.741+0000 7f222fe00640 1 --2- 172.21.15.16:0/2225370129 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f22140b4e70 0x7f22140b7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:12.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.741+0000 7f2224c00640 1 -- 172.21.15.16:0/2591471119 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221c037070 con 0x55916febc070 2024-08-31T06:13:12.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.742+0000 7f2224c00640 1 -- 172.21.15.16:0/2591471119 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f221c030900 con 0x55916febc070 2024-08-31T06:13:12.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.742+0000 7f2224c00640 1 -- 172.21.15.16:0/2591471119 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221c0046c0 con 0x55916febc070 2024-08-31T06:13:12.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.743+0000 7f223d44f740 1 -- 172.21.15.16:0/2591471119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55916febc070 msgr2=0x55917026a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:12.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.743+0000 7f223d44f740 1 --2- 172.21.15.16:0/2591471119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55916febc070 0x55917026a330 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f221c02a860 tx=0x7f221c009500 comp rx=0 tx=0).stop 2024-08-31T06:13:12.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.743+0000 7f222fe00640 1 --2- 172.21.15.16:0/2225370129 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f22140b4e70 0x7f22140b7380 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2228009270 tx=0x7f222800cf50 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:12.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.744+0000 7f223d44f740 1 -- 172.21.15.16:0/2591471119 shutdown_connections 2024-08-31T06:13:12.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.744+0000 7f223d44f740 1 --2- 172.21.15.16:0/2591471119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55916fe9b390 0x55917026adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:12.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.744+0000 7f223d44f740 1 --2- 172.21.15.16:0/2591471119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55916fe931b0 0x55917026a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:12.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.744+0000 7f223d44f740 1 --2- 172.21.15.16:0/2591471119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55916febc070 0x55917026a330 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:12.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.744+0000 7f223d44f740 1 -- 172.21.15.16:0/2591471119 >> 172.21.15.16:0/2591471119 conn(0x55916feb0460 msgr2=0x559170268220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:12.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.745+0000 7f223d44f740 1 -- 172.21.15.16:0/2591471119 shutdown_connections 2024-08-31T06:13:12.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.745+0000 7f223d44f740 1 -- 172.21.15.16:0/2591471119 wait complete. 2024-08-31T06:13:12.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.746+0000 7f223d44f740 1 -- start start 2024-08-31T06:13:12.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.746+0000 7f223d44f740 1 Processor -- start 2024-08-31T06:13:12.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.749+0000 7f223d44f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55916fe931b0 0x559170275f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:12.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.749+0000 7f223d44f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55916fe9b390 0x559170276590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:12.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.749+0000 7f223d44f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55916febc070 0x559170281d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:12.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.749+0000 7f223d44f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55917002a700 con 0x55916fe931b0 2024-08-31T06:13:12.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.749+0000 7f223d44f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55917014ecb0 con 0x55916fe9b390 2024-08-31T06:13:12.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.749+0000 7f223d44f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559170278aa0 con 0x55916febc070 2024-08-31T06:13:12.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.749+0000 7f223d44f740 10 client.0 _ll_register_callbacks cb 0x559170174430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:13:12.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.750+0000 7f222f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55916fe9b390 0x559170276590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:12.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.750+0000 7f222f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55916fe9b390 0x559170276590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:56692/0 (socket says 172.21.15.16:56692) 2024-08-31T06:13:12.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.750+0000 7f222f400640 1 -- 172.21.15.16:0/638847108 learned_addr learned my addr 172.21.15.16:0/638847108 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:12.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.750+0000 7f222ea00640 1 --2- 172.21.15.16:0/638847108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55916fe931b0 0x559170275f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:12.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.751+0000 7f222ea00640 1 --2- 172.21.15.16:0/638847108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55916febc070 0x559170281d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:12.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.752+0000 7f222f400640 1 -- 172.21.15.16:0/638847108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55916febc070 msgr2=0x559170281d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:12.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.752+0000 7f222f400640 1 --2- 172.21.15.16:0/638847108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55916febc070 0x559170281d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:12.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.752+0000 7f222f400640 1 -- 172.21.15.16:0/638847108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55916fe931b0 msgr2=0x559170275f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:12.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.752+0000 7f222f400640 1 --2- 172.21.15.16:0/638847108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55916fe931b0 0x559170275f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:12.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.752+0000 7f222f400640 1 -- 172.21.15.16:0/638847108 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55917018f870 con 0x55916fe9b390 2024-08-31T06:13:12.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.753+0000 7f222ea00640 1 --2- 172.21.15.16:0/638847108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55916febc070 0x559170281d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:13:12.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.753+0000 7f222f400640 1 --2- 172.21.15.16:0/638847108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55916fe9b390 0x559170276590 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f22200126a0 tx=0x7f2220012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bae533bccfbd7888 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:12.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.753+0000 7f2219600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:13:12.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.754+0000 7f2219600640 1 -- 172.21.15.16:0/638847108 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2220010e20 con 0x55916fe9b390 2024-08-31T06:13:12.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.754+0000 7f2219600640 1 -- 172.21.15.16:0/638847108 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8514+0+0 (secure 0 0 0) 0x7f2220004020 con 0x55916fe9b390 2024-08-31T06:13:12.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.754+0000 7f223d44f740 1 -- 172.21.15.16:0/638847108 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f221c030900 con 0x55916fe9b390 2024-08-31T06:13:12.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.754+0000 7f2219600640 1 -- 172.21.15.16:0/638847108 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f2220004220 con 0x55916fe9b390 2024-08-31T06:13:12.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.754+0000 7f223d44f740 10 client.26918 fetch_fsmap learned FSMap version 38 2024-08-31T06:13:12.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.754+0000 7f223d44f740 1 -- 172.21.15.16:0/638847108 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x559170284bc0 con 0x55916fe9b390 2024-08-31T06:13:12.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.754+0000 7f2219600640 1 -- 172.21.15.16:0/638847108 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f221c030900 con 0x55916fe9b390 2024-08-31T06:13:12.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.755+0000 7f2219600640 1 -- 172.21.15.16:0/638847108 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f2220019030 con 0x55916fe9b390 2024-08-31T06:13:12.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.755+0000 7f223d44f740 10 client.26918 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T06:13:12.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.755+0000 7f223d44f740 10 client.26918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:13:12.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.755+0000 7f223d44f740 20 client.26918 populate_metadata read hostname 'smithi016' 2024-08-31T06:13:12.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.755+0000 7f223d44f740 1 --2- 172.21.15.16:0/638847108 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55916fe7a9f0 0x559170288100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:12.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.755+0000 7f223d44f740 4 client.26918 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:13:12.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.755+0000 7f223d44f740 1 -- 172.21.15.16:0/638847108 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591701da6c0 con 0x55916fe7a9f0 2024-08-31T06:13:12.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.755+0000 7f222ea00640 1 --2- 172.21.15.16:0/638847108 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55916fe7a9f0 0x559170288100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:12.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.756+0000 7f222ea00640 1 --2- 172.21.15.16:0/638847108 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55916fe7a9f0 0x559170288100 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd8761f330450c8d server_cookie=d8b9a27123cc781b in_seq=0 out_seq=0 2024-08-31T06:13:12.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.756+0000 7f2219600640 10 client.26918 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:13:12.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.757+0000 7f2219600640 1 -- 172.21.15.16:0/638847108 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5591701da6c0 con 0x55916fe7a9f0 2024-08-31T06:13:12.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.757+0000 7f2219600640 10 client.26918 handle_command_reply: tid=0 2024-08-31T06:13:12.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.758+0000 7f223d44f740 1 client.26918 shutdown 2024-08-31T06:13:12.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.758+0000 7f223d44f740 1 -- 172.21.15.16:0/638847108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55916fe9b390 msgr2=0x559170276590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:12.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.758+0000 7f223d44f740 1 --2- 172.21.15.16:0/638847108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55916fe9b390 0x559170276590 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f22200126a0 tx=0x7f2220012b60 comp rx=0 tx=0).stop 2024-08-31T06:13:12.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.759+0000 7f223d44f740 1 -- 172.21.15.16:0/638847108 shutdown_connections 2024-08-31T06:13:12.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.759+0000 7f223d44f740 1 --2- 172.21.15.16:0/638847108 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55916fe7a9f0 0x559170288100 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:12.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.759+0000 7f223d44f740 1 --2- 172.21.15.16:0/638847108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55916febc070 0x559170281d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:12.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.759+0000 7f223d44f740 1 --2- 172.21.15.16:0/638847108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55916fe9b390 0x559170276590 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:12.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.759+0000 7f223d44f740 1 --2- 172.21.15.16:0/638847108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55916fe931b0 0x559170275f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:12.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.759+0000 7f223d44f740 1 -- 172.21.15.16:0/638847108 >> 172.21.15.16:0/638847108 conn(0x55916feb0460 msgr2=0x55917026d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:12.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.759+0000 7f2219600640 0 client.26918 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:13:12.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.759+0000 7f223d44f740 1 -- 172.21.15.16:0/638847108 shutdown_connections 2024-08-31T06:13:12.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.759+0000 7f223d44f740 1 -- 172.21.15.16:0/638847108 wait complete. 2024-08-31T06:13:12.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.759+0000 7f223d44f740 20 client.26918 trim_cache size 0 max 16384 2024-08-31T06:13:12.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.792+0000 7f223d44f740 1 Processor -- start 2024-08-31T06:13:12.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.792+0000 7f223d44f740 1 -- start start 2024-08-31T06:13:12.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.793+0000 7f223d44f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55916fe931b0 0x55917026fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:12.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.793+0000 7f223d44f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55916fe9b390 0x559170281c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:12.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.793+0000 7f223d44f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55916febc070 0x5591702821d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:12.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.793+0000 7f223d44f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f22301afb00 con 0x55916febc070 2024-08-31T06:13:12.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.793+0000 7f223d44f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f22301ad340 con 0x55916fe9b390 2024-08-31T06:13:12.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.793+0000 7f223d44f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f22301ab1b0 con 0x55916fe931b0 2024-08-31T06:13:12.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.793+0000 7f222f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55916fe9b390 0x559170281c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:12.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.794+0000 7f222f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55916fe9b390 0x559170281c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:56704/0 (socket says 172.21.15.16:56704) 2024-08-31T06:13:12.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.794+0000 7f222f400640 1 -- 172.21.15.16:0/634003060 learned_addr learned my addr 172.21.15.16:0/634003060 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:12.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.794+0000 7f222ea00640 1 --2- 172.21.15.16:0/634003060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55916fe931b0 0x55917026fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:12.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.794+0000 7f222ea00640 1 --2- 172.21.15.16:0/634003060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55916febc070 0x5591702821d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:12.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.794+0000 7f222f400640 1 -- 172.21.15.16:0/634003060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55916fe931b0 msgr2=0x55917026fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:12.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.794+0000 7f222f400640 1 --2- 172.21.15.16:0/634003060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55916fe931b0 0x55917026fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:12.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.794+0000 7f222f400640 1 -- 172.21.15.16:0/634003060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55916febc070 msgr2=0x5591702821d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:12.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.794+0000 7f222f400640 1 --2- 172.21.15.16:0/634003060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55916febc070 0x5591702821d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:12.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.794+0000 7f222f400640 1 -- 172.21.15.16:0/634003060 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55917026c850 con 0x55916fe9b390 2024-08-31T06:13:12.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.794+0000 7f222ea00640 1 --2- 172.21.15.16:0/634003060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55916febc070 0x5591702821d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:13:12.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.795+0000 7f222f400640 1 --2- 172.21.15.16:0/634003060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55916fe9b390 0x559170281c90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f22200092d0 tx=0x7f2220009300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c94ee4818fd706f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:12.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.795+0000 7f221be00640 1 -- 172.21.15.16:0/634003060 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2220012110 con 0x55916fe9b390 2024-08-31T06:13:12.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.795+0000 7f221be00640 1 -- 172.21.15.16:0/634003060 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f222001a5e0 con 0x55916fe9b390 2024-08-31T06:13:12.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.795+0000 7f221be00640 1 -- 172.21.15.16:0/634003060 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2220004680 con 0x55916fe9b390 2024-08-31T06:13:12.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.796+0000 7f223d44f740 1 -- 172.21.15.16:0/634003060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55916fe9b390 msgr2=0x559170281c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:12.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.796+0000 7f223d44f740 1 --2- 172.21.15.16:0/634003060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55916fe9b390 0x559170281c90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f22200092d0 tx=0x7f2220009300 comp rx=0 tx=0).stop 2024-08-31T06:13:12.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.796+0000 7f223d44f740 1 -- 172.21.15.16:0/634003060 shutdown_connections 2024-08-31T06:13:12.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.796+0000 7f223d44f740 1 --2- 172.21.15.16:0/634003060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55916febc070 0x5591702821d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:12.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.796+0000 7f223d44f740 1 --2- 172.21.15.16:0/634003060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55916fe9b390 0x559170281c90 secure :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f22200092d0 tx=0x7f2220009300 comp rx=0 tx=0).stop 2024-08-31T06:13:12.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.796+0000 7f223d44f740 1 --2- 172.21.15.16:0/634003060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55916fe931b0 0x55917026fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:12.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.796+0000 7f223d44f740 1 -- 172.21.15.16:0/634003060 >> 172.21.15.16:0/634003060 conn(0x55916feb0460 msgr2=0x55917026f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:12.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.797+0000 7f223d44f740 1 -- 172.21.15.16:0/634003060 shutdown_connections 2024-08-31T06:13:12.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.797+0000 7f223d44f740 1 -- 172.21.15.16:0/634003060 wait complete. 2024-08-31T06:13:12.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.798+0000 7f223d44f740 1 -- start start 2024-08-31T06:13:12.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.798+0000 7f223d44f740 1 Processor -- start 2024-08-31T06:13:12.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.799+0000 7f223d44f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55916fe931b0 0x559170279f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:12.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.799+0000 7f223d44f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55916febc070 0x55917027a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:12.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.799+0000 7f223d44f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55916fe4fda0 0x5591702f1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:12.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.799+0000 7f223d44f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f22301ab800 con 0x55916fe4fda0 2024-08-31T06:13:12.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.799+0000 7f223d44f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f22301acc70 con 0x55916fe931b0 2024-08-31T06:13:12.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.799+0000 7f223d44f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f22301abed0 con 0x55916febc070 2024-08-31T06:13:12.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.799+0000 7f223d44f740 10 client.0 _ll_register_callbacks cb 0x7f2220009d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:13:12.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.799+0000 7f222ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55916fe931b0 0x559170279f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:12.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.799+0000 7f222f400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55916febc070 0x55917027a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:12.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.799+0000 7f222ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55916fe4fda0 0x5591702f1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:12.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.799+0000 7f222f400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55916febc070 0x55917027a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:36422/0 (socket says 172.21.15.16:36422) 2024-08-31T06:13:12.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.799+0000 7f222f400640 1 -- 172.21.15.16:0/1911915542 learned_addr learned my addr 172.21.15.16:0/1911915542 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:12.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.800+0000 7f222ea00640 1 -- 172.21.15.16:0/1911915542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55916febc070 msgr2=0x55917027a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:12.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.800+0000 7f222ea00640 1 --2- 172.21.15.16:0/1911915542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55916febc070 0x55917027a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:12.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.800+0000 7f222ea00640 1 -- 172.21.15.16:0/1911915542 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55916fe931b0 msgr2=0x559170279f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:12.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.800+0000 7f222ea00640 1 --2- 172.21.15.16:0/1911915542 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55916fe931b0 0x559170279f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:12.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.800+0000 7f222ea00640 1 -- 172.21.15.16:0/1911915542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55917027ad20 con 0x55916fe4fda0 2024-08-31T06:13:12.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.801+0000 7f222ea00640 1 --2- 172.21.15.16:0/1911915542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55916fe4fda0 0x5591702f1b00 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f221c0095f0 tx=0x7f221c030bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cce92e708e30af9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:12.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.801+0000 7f2219600640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:13:12.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.802+0000 7f2219600640 1 -- 172.21.15.16:0/1911915542 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221c037040 con 0x55916fe4fda0 2024-08-31T06:13:12.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.802+0000 7f2219600640 1 -- 172.21.15.16:0/1911915542 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8514+0+0 (secure 0 0 0) 0x7f221c048040 con 0x55916fe4fda0 2024-08-31T06:13:12.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.802+0000 7f223d44f740 1 -- 172.21.15.16:0/1911915542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f222001a5e0 con 0x55916fe4fda0 2024-08-31T06:13:12.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.806+0000 7f2219600640 1 -- 172.21.15.16:0/1911915542 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f221c064050 con 0x55916fe4fda0 2024-08-31T06:13:12.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.808+0000 7f223d44f740 10 client.16710 fetch_fsmap learned FSMap version 38 2024-08-31T06:13:12.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.808+0000 7f223d44f740 1 -- 172.21.15.16:0/1911915542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5591702f3060 con 0x55916fe4fda0 2024-08-31T06:13:12.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.809+0000 7f2219600640 1 -- 172.21.15.16:0/1911915542 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f222001a5e0 con 0x55916fe4fda0 2024-08-31T06:13:12.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.809+0000 7f2219600640 1 -- 172.21.15.16:0/1911915542 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f221c0035f0 con 0x55916fe4fda0 2024-08-31T06:13:12.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.810+0000 7f223d44f740 10 client.16710 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T06:13:12.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.810+0000 7f223d44f740 10 client.16710 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:13:12.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.810+0000 7f223d44f740 20 client.16710 populate_metadata read hostname 'smithi016' 2024-08-31T06:13:12.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.810+0000 7f223d44f740 1 --2- 172.21.15.16:0/1911915542 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5591702f3340 0x5591702f5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:12.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.810+0000 7f223d44f740 4 client.16710 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T06:13:12.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.810+0000 7f223d44f740 1 -- 172.21.15.16:0/1911915542 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5591701d6160 con 0x5591702f3340 2024-08-31T06:13:12.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.810+0000 7f222f400640 1 --2- 172.21.15.16:0/1911915542 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5591702f3340 0x5591702f5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:12.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.812+0000 7f222f400640 1 --2- 172.21.15.16:0/1911915542 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5591702f3340 0x5591702f5760 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3a50d6663941897 server_cookie=e4fa0c54a49a8aef in_seq=0 out_seq=0 2024-08-31T06:13:12.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.813+0000 7f2219600640 10 client.16710 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:13:12.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.813+0000 7f2219600640 1 -- 172.21.15.16:0/1911915542 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2362 (crc 0 0 0) 0x5591701d6160 con 0x5591702f3340 2024-08-31T06:13:12.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.813+0000 7f2219600640 10 client.16710 handle_command_reply: tid=0 2024-08-31T06:13:12.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.813+0000 7f223d44f740 1 client.16710 shutdown 2024-08-31T06:13:12.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.814+0000 7f223d44f740 1 -- 172.21.15.16:0/1911915542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55916fe4fda0 msgr2=0x5591702f1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:12.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.814+0000 7f223d44f740 1 --2- 172.21.15.16:0/1911915542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55916fe4fda0 0x5591702f1b00 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f221c0095f0 tx=0x7f221c030bd0 comp rx=0 tx=0).stop 2024-08-31T06:13:12.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.814+0000 7f223d44f740 1 -- 172.21.15.16:0/1911915542 shutdown_connections 2024-08-31T06:13:12.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.814+0000 7f223d44f740 1 --2- 172.21.15.16:0/1911915542 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5591702f3340 0x5591702f5760 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:12.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.814+0000 7f223d44f740 1 --2- 172.21.15.16:0/1911915542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55916febc070 0x55917027a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:12.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.814+0000 7f223d44f740 1 --2- 172.21.15.16:0/1911915542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55916fe4fda0 0x5591702f1b00 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:12.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.814+0000 7f223d44f740 1 --2- 172.21.15.16:0/1911915542 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55916fe931b0 0x559170279f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:12.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.814+0000 7f223d44f740 1 -- 172.21.15.16:0/1911915542 >> 172.21.15.16:0/1911915542 conn(0x55916feb0460 msgr2=0x559170275f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:12.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.814+0000 7f2219600640 0 client.16710 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:13:12.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.816+0000 7f223d44f740 1 -- 172.21.15.16:0/1911915542 shutdown_connections 2024-08-31T06:13:12.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.816+0000 7f223d44f740 1 -- 172.21.15.16:0/1911915542 wait complete. 2024-08-31T06:13:12.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.816+0000 7f223d44f740 20 client.16710 trim_cache size 0 max 16384 2024-08-31T06:13:12.820 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:13:12.821 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-08-31T06:13:12.821 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {}, 2024-08-31T06:13:12.821 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-08-31T06:13:12.821 INFO:teuthology.orchestra.run.smithi016.stdout: "e739a16e-8ace-43a8-a1ce-ac27d39ef8a3": { 2024-08-31T06:13:12.821 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:54.084646+0000", 2024-08-31T06:13:12.821 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:13:12.821 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:13:12.821 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:13:12.821 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:13:12.821 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:13:12.821 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:13:12.821 INFO:teuthology.orchestra.run.smithi016.stdout: "a10cce76-99ab-46b5-b3b2-07b82899b8f9": { 2024-08-31T06:13:12.821 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:11:43.967546+0000", 2024-08-31T06:13:12.822 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:13:12.822 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:13:12.822 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:13:12.822 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:13:12.822 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:13:12.822 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:13:12.822 INFO:teuthology.orchestra.run.smithi016.stdout: "c1fd6904-2e02-47fa-939a-47349b9fe36b": { 2024-08-31T06:13:12.822 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:06.678691+0000", 2024-08-31T06:13:12.822 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:13:12.822 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:13:12.822 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:13:12.822 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:13:12.822 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:13:12.822 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:13:12.823 INFO:teuthology.orchestra.run.smithi016.stdout: "78d6ef02-8551-45e8-ad60-9627eb573fc1": { 2024-08-31T06:13:12.823 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:09:13.547865+0000", 2024-08-31T06:13:12.823 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:13:12.823 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:13:12.823 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:13:12.823 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:13:12.823 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:13:12.823 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:13:12.823 INFO:teuthology.orchestra.run.smithi016.stdout: "c8455d74-2fe7-4d93-bcf2-19b419d16016": { 2024-08-31T06:13:12.823 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:08:24.035960+0000", 2024-08-31T06:13:12.823 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:13:12.823 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:13:12.823 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:13:12.823 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:13:12.824 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:13:12.824 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:13:12.824 INFO:teuthology.orchestra.run.smithi016.stdout: "88d7bdb8-cb1c-45e5-ad29-4cd8da6c6003": { 2024-08-31T06:13:12.824 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:07:39.648942+0000", 2024-08-31T06:13:12.824 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:13:12.824 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:13:12.824 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:13:12.824 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:13:12.824 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:13:12.824 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:13:12.824 INFO:teuthology.orchestra.run.smithi016.stdout: "6a03046e-cdf4-4a14-b9bd-b560b8968467": { 2024-08-31T06:13:12.824 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:12:38.002354+0000", 2024-08-31T06:13:12.824 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:13:12.824 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:13:12.825 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:13:12.825 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:13:12.825 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:13:12.825 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:13:12.825 INFO:teuthology.orchestra.run.smithi016.stdout: "c300b7e8-f3fd-482c-a909-9329bc61e570": { 2024-08-31T06:13:12.825 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:06:49.868046+0000", 2024-08-31T06:13:12.825 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:13:12.825 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:13:12.825 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:13:12.825 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:13:12.825 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:13:12.825 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:13:12.825 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:13:12.825 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:13:12.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.819+0000 7f2235200640 1 -- 172.21.15.16:0/2225370129 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f22140b4e70 msgr2=0x7f22140b7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:12.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.819+0000 7f2235200640 1 --2- 172.21.15.16:0/2225370129 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f22140b4e70 0x7f22140b7380 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2228009270 tx=0x7f222800cf50 comp rx=0 tx=0).stop 2024-08-31T06:13:12.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.819+0000 7f2235200640 1 -- 172.21.15.16:0/2225370129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22301099e0 msgr2=0x7f223019f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:12.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.819+0000 7f2235200640 1 --2- 172.21.15.16:0/2225370129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22301099e0 0x7f223019f8e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f222800eaa0 tx=0x7f222800ef60 comp rx=0 tx=0).stop 2024-08-31T06:13:12.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.819+0000 7f2235200640 1 -- 172.21.15.16:0/2225370129 shutdown_connections 2024-08-31T06:13:12.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.819+0000 7f2235200640 1 --2- 172.21.15.16:0/2225370129 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f22140b4e70 0x7f22140b7380 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:12.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.819+0000 7f2235200640 1 --2- 172.21.15.16:0/2225370129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22301099e0 0x7f223019f8e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:12.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.819+0000 7f2235200640 1 --2- 172.21.15.16:0/2225370129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2230109090 0x7f223019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:12.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.819+0000 7f2235200640 1 --2- 172.21.15.16:0/2225370129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2230103750 0x7f223019a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:12.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.819+0000 7f2235200640 1 -- 172.21.15.16:0/2225370129 >> 172.21.15.16:0/2225370129 conn(0x7f22300ff1e0 msgr2=0x7f2230101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:12.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.819+0000 7f2235200640 1 -- 172.21.15.16:0/2225370129 shutdown_connections 2024-08-31T06:13:12.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:12.820+0000 7f2235200640 1 -- 172.21.15.16:0/2225370129 wait complete. 2024-08-31T06:13:13.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:13 smithi016 ceph-mon[17194]: pgmap v544: 97 pgs: 97 active+clean; 3.2 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 34 MiB/s wr, 1.91k op/s 2024-08-31T06:13:13.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:13 smithi184 ceph-mon[22331]: pgmap v544: 97 pgs: 97 active+clean; 3.2 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 34 MiB/s wr, 1.91k op/s 2024-08-31T06:13:13.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:13 smithi155 ceph-mon[21742]: pgmap v544: 97 pgs: 97 active+clean; 3.2 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 34 MiB/s wr, 1.91k op/s 2024-08-31T06:13:13.926 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T06:13:13.926 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:13:14.548 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:13:15.377 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:15 smithi016 ceph-mon[17194]: pgmap v545: 97 pgs: 97 active+clean; 3.2 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 32 MiB/s wr, 1.81k op/s 2024-08-31T06:13:15.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:15 smithi184 ceph-mon[22331]: pgmap v545: 97 pgs: 97 active+clean; 3.2 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 32 MiB/s wr, 1.81k op/s 2024-08-31T06:13:15.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:15 smithi155 ceph-mon[21742]: pgmap v545: 97 pgs: 97 active+clean; 3.2 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 32 MiB/s wr, 1.81k op/s 2024-08-31T06:13:15.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:15.655+0000 7fb4e5000640 1 -- 172.21.15.16:0/2006386167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4e0109250 msgr2=0x7fb4e0109660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:15.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:15.655+0000 7fb4e5000640 1 --2- 172.21.15.16:0/2006386167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4e0109250 0x7fb4e0109660 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc009860 tx=0x7fb4cc02a340 comp rx=0 tx=0).stop 2024-08-31T06:13:15.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:15.655+0000 7fb4e5000640 1 -- 172.21.15.16:0/2006386167 shutdown_connections 2024-08-31T06:13:15.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:15.655+0000 7fb4e5000640 1 --2- 172.21.15.16:0/2006386167 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4e0109ba0 0x7fb4e01103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:15.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:15.655+0000 7fb4e5000640 1 --2- 172.21.15.16:0/2006386167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4e0109250 0x7fb4e0109660 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:15.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:15.655+0000 7fb4e5000640 1 --2- 172.21.15.16:0/2006386167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e0103910 0x7fb4e0103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:15.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:15.655+0000 7fb4e5000640 1 -- 172.21.15.16:0/2006386167 >> 172.21.15.16:0/2006386167 conn(0x7fb4e00ff380 msgr2=0x7fb4e01017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:15.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:15.656+0000 7fb4e5000640 1 -- 172.21.15.16:0/2006386167 shutdown_connections 2024-08-31T06:13:15.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:15.656+0000 7fb4e5000640 1 -- 172.21.15.16:0/2006386167 wait complete. 2024-08-31T06:13:15.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:15.658+0000 7fb4e5000640 1 Processor -- start 2024-08-31T06:13:15.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:15.659+0000 7fb4e5000640 1 -- start start 2024-08-31T06:13:15.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:15.659+0000 7fb4e5000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4e0103910 0x7fb4e019a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:15.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:15.659+0000 7fb4e5000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e0109250 0x7fb4e019a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:15.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:15.659+0000 7fb4e5000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4e0109ba0 0x7fb4e019fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:15.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:15.659+0000 7fb4e5000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e0113860 con 0x7fb4e0109250 2024-08-31T06:13:15.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:15.659+0000 7fb4e5000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e01136e0 con 0x7fb4e0103910 2024-08-31T06:13:15.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:15.659+0000 7fb4e5000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e01139e0 con 0x7fb4e0109ba0 2024-08-31T06:13:15.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:15.660+0000 7fb4de800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4e0109ba0 0x7fb4e019fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:15.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:15.660+0000 7fb4dde00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4e0103910 0x7fb4e019a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:15.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:15.660+0000 7fb4de800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4e0109ba0 0x7fb4e019fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:36442/0 (socket says 172.21.15.16:36442) 2024-08-31T06:13:15.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:15.660+0000 7fb4de800640 1 -- 172.21.15.16:0/4073511869 learned_addr learned my addr 172.21.15.16:0/4073511869 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:15.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:15.660+0000 7fb4dd400640 1 --2- 172.21.15.16:0/4073511869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e0109250 0x7fb4e019a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:15.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:15.661+0000 7fb4dde00640 1 -- 172.21.15.16:0/4073511869 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4e0109ba0 msgr2=0x7fb4e019fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:15.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:15.661+0000 7fb4dde00640 1 --2- 172.21.15.16:0/4073511869 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4e0109ba0 0x7fb4e019fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:15.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:15.661+0000 7fb4dde00640 1 -- 172.21.15.16:0/4073511869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e0109250 msgr2=0x7fb4e019a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:15.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:15.661+0000 7fb4dde00640 1 --2- 172.21.15.16:0/4073511869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e0109250 0x7fb4e019a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:15.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:15.661+0000 7fb4dde00640 1 -- 172.21.15.16:0/4073511869 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4e01a0180 con 0x7fb4e0103910 2024-08-31T06:13:15.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:15.661+0000 7fb4dde00640 1 --2- 172.21.15.16:0/4073511869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4e0103910 0x7fb4e019a210 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0005fd0 tx=0x7fb4d000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:15.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:15.662+0000 7fb4d6a00640 1 -- 172.21.15.16:0/4073511869 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d000cd00 con 0x7fb4e0103910 2024-08-31T06:13:15.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:15.662+0000 7fb4d6a00640 1 -- 172.21.15.16:0/4073511869 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4d00035f0 con 0x7fb4e0103910 2024-08-31T06:13:15.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:15.662+0000 7fb4d6a00640 1 -- 172.21.15.16:0/4073511869 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d0010670 con 0x7fb4e0103910 2024-08-31T06:13:15.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:15.662+0000 7fb4e5000640 1 -- 172.21.15.16:0/4073511869 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4e01a0470 con 0x7fb4e0103910 2024-08-31T06:13:15.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:15.662+0000 7fb4e5000640 1 -- 172.21.15.16:0/4073511869 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4e01a6bd0 con 0x7fb4e0103910 2024-08-31T06:13:15.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:15.664+0000 7fb4e5000640 1 -- 172.21.15.16:0/4073511869 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4e0104b90 con 0x7fb4e0103910 2024-08-31T06:13:15.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:15.668+0000 7fb4d6a00640 1 -- 172.21.15.16:0/4073511869 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fb4d00037a0 con 0x7fb4e0103910 2024-08-31T06:13:15.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:15.672+0000 7fb4d6a00640 1 --2- 172.21.15.16:0/4073511869 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb4c40b4df0 0x7fb4c40b7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:15.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:15.672+0000 7fb4dd400640 1 --2- 172.21.15.16:0/4073511869 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb4c40b4df0 0x7fb4c40b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:15.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:15.673+0000 7fb4d6a00640 1 -- 172.21.15.16:0/4073511869 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8514+0+0 (secure 0 0 0) 0x7fb4d00e57d0 con 0x7fb4e0103910 2024-08-31T06:13:15.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:15.673+0000 7fb4d6a00640 1 -- 172.21.15.16:0/4073511869 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fb4d01161b0 con 0x7fb4e0103910 2024-08-31T06:13:15.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:15.675+0000 7fb4dd400640 1 --2- 172.21.15.16:0/4073511869 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb4c40b4df0 0x7fb4c40b7300 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc035040 tx=0x7fb4cc000bc0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:16.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:16.077+0000 7fb4e5000640 1 -- 172.21.15.16:0/4073511869 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb4e0103d20 con 0x7fb4e0103910 2024-08-31T06:13:16.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:16.078+0000 7fb4d6a00640 1 -- 172.21.15.16:0/4073511869 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11921 (secure 0 0 0) 0x7fb4d0009510 con 0x7fb4e0103910 2024-08-31T06:13:16.082 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:13:16.082 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":38,"btime":"2024-08-31T06:12:33:984731+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:12:33.336879+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24712,24724,24737]},"id":1}]} 2024-08-31T06:13:16.083 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 38 2024-08-31T06:13:16.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:16.081+0000 7fb4e5000640 1 -- 172.21.15.16:0/4073511869 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb4c40b4df0 msgr2=0x7fb4c40b7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:16.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:16.081+0000 7fb4e5000640 1 --2- 172.21.15.16:0/4073511869 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb4c40b4df0 0x7fb4c40b7300 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc035040 tx=0x7fb4cc000bc0 comp rx=0 tx=0).stop 2024-08-31T06:13:16.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:16.081+0000 7fb4e5000640 1 -- 172.21.15.16:0/4073511869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4e0103910 msgr2=0x7fb4e019a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:16.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:16.081+0000 7fb4e5000640 1 --2- 172.21.15.16:0/4073511869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4e0103910 0x7fb4e019a210 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0005fd0 tx=0x7fb4d000ee50 comp rx=0 tx=0).stop 2024-08-31T06:13:16.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:16.082+0000 7fb4e5000640 1 -- 172.21.15.16:0/4073511869 shutdown_connections 2024-08-31T06:13:16.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:16.082+0000 7fb4e5000640 1 --2- 172.21.15.16:0/4073511869 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb4c40b4df0 0x7fb4c40b7300 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:16.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:16.082+0000 7fb4e5000640 1 --2- 172.21.15.16:0/4073511869 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4e0109ba0 0x7fb4e019fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:16.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:16.082+0000 7fb4e5000640 1 --2- 172.21.15.16:0/4073511869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e0109250 0x7fb4e019a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:16.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:16.082+0000 7fb4e5000640 1 --2- 172.21.15.16:0/4073511869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4e0103910 0x7fb4e019a210 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:16.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:16.082+0000 7fb4e5000640 1 -- 172.21.15.16:0/4073511869 >> 172.21.15.16:0/4073511869 conn(0x7fb4e00ff380 msgr2=0x7fb4e01017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:16.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:16.082+0000 7fb4e5000640 1 -- 172.21.15.16:0/4073511869 shutdown_connections 2024-08-31T06:13:16.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:16.082+0000 7fb4e5000640 1 -- 172.21.15.16:0/4073511869 wait complete. 2024-08-31T06:13:16.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:16 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/4073511869' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:13:16.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:16 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/4073511869' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:13:16.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:16 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/4073511869' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:13:16.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 350.0 s, 1901.6 tps, lat 52.326 ms stddev 65.046 2024-08-31T06:13:17.185 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T06:13:17.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:17 smithi016 ceph-mon[17194]: pgmap v546: 97 pgs: 97 active+clean; 3.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 29 MiB/s wr, 1.17k op/s 2024-08-31T06:13:17.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:17 smithi184 ceph-mon[22331]: pgmap v546: 97 pgs: 97 active+clean; 3.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 29 MiB/s wr, 1.17k op/s 2024-08-31T06:13:17.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:17 smithi155 ceph-mon[21742]: pgmap v546: 97 pgs: 97 active+clean; 3.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 29 MiB/s wr, 1.17k op/s 2024-08-31T06:13:17.894 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:13:18.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.806+0000 7f4359600640 1 -- 172.21.15.16:0/4089692889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4354104480 msgr2=0x7f4354104890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:18.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.806+0000 7f4359600640 1 --2- 172.21.15.16:0/4089692889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4354104480 0x7f4354104890 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f434c009ca0 tx=0x7f434c02a8e0 comp rx=0 tx=0).stop 2024-08-31T06:13:18.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.807+0000 7f4359600640 1 -- 172.21.15.16:0/4089692889 shutdown_connections 2024-08-31T06:13:18.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.807+0000 7f4359600640 1 --2- 172.21.15.16:0/4089692889 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43541026b0 0x7f4354112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:18.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.807+0000 7f4359600640 1 --2- 172.21.15.16:0/4089692889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4354101c00 0x7f4354102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:18.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.807+0000 7f4359600640 1 --2- 172.21.15.16:0/4089692889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4354104480 0x7f4354104890 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:18.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.807+0000 7f4359600640 1 -- 172.21.15.16:0/4089692889 >> 172.21.15.16:0/4089692889 conn(0x7f4354077a00 msgr2=0x7f4354077e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:18.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.807+0000 7f4359600640 1 -- 172.21.15.16:0/4089692889 shutdown_connections 2024-08-31T06:13:18.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.807+0000 7f4359600640 1 -- 172.21.15.16:0/4089692889 wait complete. 2024-08-31T06:13:18.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.810+0000 7f4359600640 1 Processor -- start 2024-08-31T06:13:18.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.810+0000 7f4359600640 1 -- start start 2024-08-31T06:13:18.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.811+0000 7f4359600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4354101c00 0x7f435419c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:18.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.811+0000 7f4359600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43541026b0 0x7f435419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:18.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.811+0000 7f4359600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43541a1ea0 0x7f43541a22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:18.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.811+0000 7f4359600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4354115b90 con 0x7f43541026b0 2024-08-31T06:13:18.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.811+0000 7f4359600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4354115a10 con 0x7f4354101c00 2024-08-31T06:13:18.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.811+0000 7f4359600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4354115d10 con 0x7f43541a1ea0 2024-08-31T06:13:18.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.812+0000 7f4352a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4354101c00 0x7f435419c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:18.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.812+0000 7f4352a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4354101c00 0x7f435419c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:57420/0 (socket says 172.21.15.16:57420) 2024-08-31T06:13:18.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.812+0000 7f4352a00640 1 -- 172.21.15.16:0/2589122847 learned_addr learned my addr 172.21.15.16:0/2589122847 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:18.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.812+0000 7f4352000640 1 --2- 172.21.15.16:0/2589122847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43541026b0 0x7f435419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:18.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.812+0000 7f4353400640 1 --2- 172.21.15.16:0/2589122847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43541a1ea0 0x7f43541a22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:18.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.813+0000 7f4353400640 1 -- 172.21.15.16:0/2589122847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4354101c00 msgr2=0x7f435419c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:18.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.813+0000 7f4353400640 1 --2- 172.21.15.16:0/2589122847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4354101c00 0x7f435419c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:18.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.813+0000 7f4353400640 1 -- 172.21.15.16:0/2589122847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43541026b0 msgr2=0x7f435419ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:18.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.813+0000 7f4353400640 1 --2- 172.21.15.16:0/2589122847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43541026b0 0x7f435419ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:18.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.813+0000 7f4353400640 1 -- 172.21.15.16:0/2589122847 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43541a2a80 con 0x7f43541a1ea0 2024-08-31T06:13:18.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.814+0000 7f4353400640 1 --2- 172.21.15.16:0/2589122847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43541a1ea0 0x7f43541a22b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f434400b3e0 tx=0x7f434400b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:18.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.814+0000 7f434b400640 1 -- 172.21.15.16:0/2589122847 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4344015020 con 0x7f43541a1ea0 2024-08-31T06:13:18.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.814+0000 7f434b400640 1 -- 172.21.15.16:0/2589122847 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4344011ce0 con 0x7f43541a1ea0 2024-08-31T06:13:18.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.815+0000 7f4359600640 1 -- 172.21.15.16:0/2589122847 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43541a9020 con 0x7f43541a1ea0 2024-08-31T06:13:18.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.815+0000 7f4359600640 1 -- 172.21.15.16:0/2589122847 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43541a9420 con 0x7f43541a1ea0 2024-08-31T06:13:18.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.815+0000 7f434b400640 1 -- 172.21.15.16:0/2589122847 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4344015020 con 0x7f43541a1ea0 2024-08-31T06:13:18.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.819+0000 7f43603e7740 1 Processor -- start 2024-08-31T06:13:18.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.821+0000 7f434b400640 1 -- 172.21.15.16:0/2589122847 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f4344018020 con 0x7f43541a1ea0 2024-08-31T06:13:18.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.822+0000 7f43603e7740 1 -- start start 2024-08-31T06:13:18.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.823+0000 7f43603e7740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562df5a39070 0x562df5dde990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:18.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.823+0000 7f43603e7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562df5a101b0 0x562df5ddeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:18.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.823+0000 7f43603e7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562df5a18390 0x562df5ddf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:18.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.823+0000 7f43603e7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562df5d557b0 con 0x562df5a101b0 2024-08-31T06:13:18.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.823+0000 7f43603e7740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562df5d44d40 con 0x562df5a39070 2024-08-31T06:13:18.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.823+0000 7f43603e7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562df5d48cf0 con 0x562df5a18390 2024-08-31T06:13:18.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.823+0000 7f4352000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562df5a39070 0x562df5dde990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:18.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.823+0000 7f4352000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562df5a39070 0x562df5dde990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:57426/0 (socket says 172.21.15.16:57426) 2024-08-31T06:13:18.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.823+0000 7f4352000640 1 -- 172.21.15.16:0/115559074 learned_addr learned my addr 172.21.15.16:0/115559074 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:18.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.823+0000 7f4352000640 1 --2- 172.21.15.16:0/115559074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562df5a18390 0x562df5ddf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:18.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.824+0000 7f4352000640 1 -- 172.21.15.16:0/115559074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562df5a18390 msgr2=0x562df5ddf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:18.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.824+0000 7f4352000640 1 --2- 172.21.15.16:0/115559074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562df5a18390 0x562df5ddf410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:18.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.824+0000 7f4352000640 1 -- 172.21.15.16:0/115559074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562df5a101b0 msgr2=0x562df5ddeed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:13:18.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.824+0000 7f434b400640 1 --2- 172.21.15.16:0/2589122847 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f43380b4fd0 0x7f43380b74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:18.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.824+0000 7f4352a00640 1 --2- 172.21.15.16:0/115559074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562df5a101b0 0x562df5ddeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:18.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.824+0000 7f4352000640 1 --2- 172.21.15.16:0/115559074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562df5a101b0 0x562df5ddeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:18.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.824+0000 7f4352000640 1 -- 172.21.15.16:0/115559074 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562df5aa4650 con 0x562df5a39070 2024-08-31T06:13:18.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.824+0000 7f434b400640 1 -- 172.21.15.16:0/2589122847 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8514+0+0 (secure 0 0 0) 0x7f43440e4600 con 0x7f43541a1ea0 2024-08-31T06:13:18.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.825+0000 7f4352a00640 1 --2- 172.21.15.16:0/115559074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562df5a101b0 0x562df5ddeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:13:18.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.825+0000 7f4352000640 1 --2- 172.21.15.16:0/115559074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562df5a39070 0x562df5dde990 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f433c0116b0 tx=0x7f433c00a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39d8a7d79684896d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:18.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.825+0000 7f4352000640 1 --2- 172.21.15.16:0/2589122847 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f43380b4fd0 0x7f43380b74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:18.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.825+0000 7f4343400640 1 -- 172.21.15.16:0/115559074 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433c00f600 con 0x562df5a39070 2024-08-31T06:13:18.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.825+0000 7f4343400640 1 -- 172.21.15.16:0/115559074 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f433c00fc50 con 0x562df5a39070 2024-08-31T06:13:18.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.826+0000 7f4343400640 1 -- 172.21.15.16:0/115559074 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433c01c370 con 0x562df5a39070 2024-08-31T06:13:18.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.827+0000 7f43603e7740 1 -- 172.21.15.16:0/115559074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562df5a39070 msgr2=0x562df5dde990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:18.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.827+0000 7f43603e7740 1 --2- 172.21.15.16:0/115559074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562df5a39070 0x562df5dde990 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f433c0116b0 tx=0x7f433c00a640 comp rx=0 tx=0).stop 2024-08-31T06:13:18.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.827+0000 7f4352000640 1 --2- 172.21.15.16:0/2589122847 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f43380b4fd0 0x7f43380b74e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f433c01d000 tx=0x7f433c01ef30 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:18.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.828+0000 7f43603e7740 1 -- 172.21.15.16:0/115559074 shutdown_connections 2024-08-31T06:13:18.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.828+0000 7f43603e7740 1 --2- 172.21.15.16:0/115559074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562df5a18390 0x562df5ddf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:18.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.828+0000 7f43603e7740 1 --2- 172.21.15.16:0/115559074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562df5a101b0 0x562df5ddeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:18.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.828+0000 7f43603e7740 1 --2- 172.21.15.16:0/115559074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562df5a39070 0x562df5dde990 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:18.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.828+0000 7f43603e7740 1 -- 172.21.15.16:0/115559074 >> 172.21.15.16:0/115559074 conn(0x562df5a2d460 msgr2=0x562df5ddc880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:18.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.828+0000 7f43603e7740 1 -- 172.21.15.16:0/115559074 shutdown_connections 2024-08-31T06:13:18.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.829+0000 7f43603e7740 1 -- 172.21.15.16:0/115559074 wait complete. 2024-08-31T06:13:18.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.830+0000 7f43603e7740 1 -- start start 2024-08-31T06:13:18.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.831+0000 7f43603e7740 1 Processor -- start 2024-08-31T06:13:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.832+0000 7f43603e7740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562df5a101b0 0x562df5dea470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.832+0000 7f43603e7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562df5a18390 0x562df5dea9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.832+0000 7f43603e7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562df5a39070 0x562df5dfed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.832+0000 7f43603e7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562df5ba7700 con 0x562df5a18390 2024-08-31T06:13:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.832+0000 7f43603e7740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562df5ccbcf0 con 0x562df5a101b0 2024-08-31T06:13:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.832+0000 7f43603e7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562df5df5980 con 0x562df5a39070 2024-08-31T06:13:18.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.832+0000 7f43603e7740 10 client.0 _ll_register_callbacks cb 0x562df5cf1470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:13:18.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.833+0000 7f4352a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562df5a101b0 0x562df5dea470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:18.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.833+0000 7f4352a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562df5a101b0 0x562df5dea470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:57428/0 (socket says 172.21.15.16:57428) 2024-08-31T06:13:18.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.833+0000 7f4352a00640 1 -- 172.21.15.16:0/1211831635 learned_addr learned my addr 172.21.15.16:0/1211831635 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:18.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.834+0000 7f4353400640 1 --2- 172.21.15.16:0/1211831635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562df5a39070 0x562df5dfed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:18.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.834+0000 7f4352000640 1 --2- 172.21.15.16:0/1211831635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562df5a18390 0x562df5dea9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:18.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.834+0000 7f4352a00640 1 -- 172.21.15.16:0/1211831635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562df5a39070 msgr2=0x562df5dfed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:18.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.834+0000 7f4352a00640 1 --2- 172.21.15.16:0/1211831635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562df5a39070 0x562df5dfed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:18.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.834+0000 7f4352a00640 1 -- 172.21.15.16:0/1211831635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562df5a18390 msgr2=0x562df5dea9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:18.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.835+0000 7f4352a00640 1 --2- 172.21.15.16:0/1211831635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562df5a18390 0x562df5dea9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:18.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.835+0000 7f4352a00640 1 -- 172.21.15.16:0/1211831635 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562df5d0c8b0 con 0x562df5a101b0 2024-08-31T06:13:18.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.835+0000 7f4352000640 1 --2- 172.21.15.16:0/1211831635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562df5a18390 0x562df5dea9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:13:18.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.835+0000 7f4337e00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:13:18.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.835+0000 7f4352a00640 1 --2- 172.21.15.16:0/1211831635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562df5a101b0 0x562df5dea470 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f434c009bf0 tx=0x7f434c031d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec45300c3399cbf5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:18.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.836+0000 7f4337e00640 1 -- 172.21.15.16:0/1211831635 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c049060 con 0x562df5a101b0 2024-08-31T06:13:18.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.836+0000 7f4337e00640 1 -- 172.21.15.16:0/1211831635 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8514+0+0 (secure 0 0 0) 0x7f434c041070 con 0x562df5a101b0 2024-08-31T06:13:18.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.836+0000 7f43603e7740 1 -- 172.21.15.16:0/1211831635 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f433c00fc50 con 0x562df5a101b0 2024-08-31T06:13:18.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.837+0000 7f4337e00640 1 -- 172.21.15.16:0/1211831635 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f434c049060 con 0x562df5a101b0 2024-08-31T06:13:18.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.837+0000 7f43603e7740 10 client.26936 fetch_fsmap learned FSMap version 38 2024-08-31T06:13:18.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.837+0000 7f43603e7740 1 -- 172.21.15.16:0/1211831635 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x562df5e12890 con 0x562df5a101b0 2024-08-31T06:13:18.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.838+0000 7f4337e00640 1 -- 172.21.15.16:0/1211831635 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f433c00fc50 con 0x562df5a101b0 2024-08-31T06:13:18.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.838+0000 7f4337e00640 1 -- 172.21.15.16:0/1211831635 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f434c0305e0 con 0x562df5a101b0 2024-08-31T06:13:18.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.839+0000 7f43603e7740 10 client.26936 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T06:13:18.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.839+0000 7f43603e7740 10 client.26936 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T06:13:18.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.839+0000 7f43603e7740 20 client.26936 populate_metadata read hostname 'smithi016' 2024-08-31T06:13:18.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.839+0000 7f43603e7740 1 --2- 172.21.15.16:0/1211831635 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x562df59f79f0 0x562df5e15ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:18.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.839+0000 7f43603e7740 4 client.26936 mds_command: new command op to 24712 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:13:18.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.839+0000 7f43603e7740 1 -- 172.21.15.16:0/1211831635 --> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562df5d57700 con 0x562df59f79f0 2024-08-31T06:13:18.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.841+0000 7f4352000640 1 --2- 172.21.15.16:0/1211831635 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x562df59f79f0 0x562df5e15ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:18.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.841+0000 7f4352000640 1 --2- 172.21.15.16:0/1211831635 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x562df59f79f0 0x562df5e15ec0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f0e4bc3d2e7569dd server_cookie=25d4d936daae575b in_seq=0 out_seq=0 2024-08-31T06:13:18.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.841+0000 7f4337e00640 10 client.26936 ms_handle_connect on v2:172.21.15.184:6838/4011051479 2024-08-31T06:13:18.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.843+0000 7f4337e00640 1 -- 172.21.15.16:0/1211831635 <== mds.2 v2:172.21.15.184:6838/4011051479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562df5d57700 con 0x562df59f79f0 2024-08-31T06:13:18.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.843+0000 7f4337e00640 10 client.26936 handle_command_reply: tid=0 2024-08-31T06:13:18.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.844+0000 7f43603e7740 1 client.26936 shutdown 2024-08-31T06:13:18.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.845+0000 7f43603e7740 1 -- 172.21.15.16:0/1211831635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562df5a101b0 msgr2=0x562df5dea470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:18.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.845+0000 7f43603e7740 1 --2- 172.21.15.16:0/1211831635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562df5a101b0 0x562df5dea470 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f434c009bf0 tx=0x7f434c031d80 comp rx=0 tx=0).stop 2024-08-31T06:13:18.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.846+0000 7f43603e7740 1 -- 172.21.15.16:0/1211831635 shutdown_connections 2024-08-31T06:13:18.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.846+0000 7f43603e7740 1 --2- 172.21.15.16:0/1211831635 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x562df59f79f0 0x562df5e15ec0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:18.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.846+0000 7f43603e7740 1 --2- 172.21.15.16:0/1211831635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562df5a39070 0x562df5dfed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:18.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.846+0000 7f43603e7740 1 --2- 172.21.15.16:0/1211831635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562df5a18390 0x562df5dea9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:18.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.846+0000 7f43603e7740 1 --2- 172.21.15.16:0/1211831635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562df5a101b0 0x562df5dea470 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:18.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.846+0000 7f43603e7740 1 -- 172.21.15.16:0/1211831635 >> 172.21.15.16:0/1211831635 conn(0x562df5a2d460 msgr2=0x562df5de1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:18.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.846+0000 7f4337e00640 0 client.26936 ms_handle_reset on v2:172.21.15.184:6838/4011051479 2024-08-31T06:13:18.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.846+0000 7f43603e7740 1 -- 172.21.15.16:0/1211831635 shutdown_connections 2024-08-31T06:13:18.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.847+0000 7f43603e7740 1 -- 172.21.15.16:0/1211831635 wait complete. 2024-08-31T06:13:18.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.847+0000 7f43603e7740 20 client.26936 trim_cache size 0 max 16384 2024-08-31T06:13:18.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.868+0000 7f43603e7740 1 Processor -- start 2024-08-31T06:13:18.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.869+0000 7f43603e7740 1 -- start start 2024-08-31T06:13:18.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.869+0000 7f43603e7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562df5a101b0 0x562df5dfec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:18.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.869+0000 7f43603e7740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562df5a18390 0x562df5ddc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:18.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.869+0000 7f43603e7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562df5a39070 0x562df5de7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:18.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.870+0000 7f4352a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562df5a101b0 0x562df5dfec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.870+0000 7f43603e7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4330011180 con 0x562df5a39070 2024-08-31T06:13:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.870+0000 7f43603e7740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f433000e9c0 con 0x562df5a18390 2024-08-31T06:13:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.870+0000 7f43603e7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f433000c830 con 0x562df5a101b0 2024-08-31T06:13:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.870+0000 7f4352000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562df5a18390 0x562df5ddc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:18.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.870+0000 7f4353400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562df5a39070 0x562df5de7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:18.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.870+0000 7f4353400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562df5a39070 0x562df5de7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60810/0 (socket says 172.21.15.16:60810) 2024-08-31T06:13:18.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.870+0000 7f4353400640 1 -- 172.21.15.16:0/3695973224 learned_addr learned my addr 172.21.15.16:0/3695973224 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:18.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.870+0000 7f4352a00640 1 -- 172.21.15.16:0/3695973224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562df5a18390 msgr2=0x562df5ddc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:18.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.870+0000 7f4352a00640 1 --2- 172.21.15.16:0/3695973224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562df5a18390 0x562df5ddc780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:18.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.870+0000 7f4352a00640 1 -- 172.21.15.16:0/3695973224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562df5a39070 msgr2=0x562df5de7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:18.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.870+0000 7f4352a00640 1 --2- 172.21.15.16:0/3695973224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562df5a39070 0x562df5de7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:18.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.870+0000 7f4352a00640 1 -- 172.21.15.16:0/3695973224 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562df5decc20 con 0x562df5a101b0 2024-08-31T06:13:18.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.871+0000 7f4352a00640 1 --2- 172.21.15.16:0/3695973224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562df5a101b0 0x562df5dfec00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f434c002230 tx=0x7f434c000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c63115ee6890e6b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:18.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.871+0000 7f4342a00640 1 -- 172.21.15.16:0/3695973224 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c0305e0 con 0x562df5a101b0 2024-08-31T06:13:18.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.872+0000 7f4342a00640 1 -- 172.21.15.16:0/3695973224 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f434c036050 con 0x562df5a101b0 2024-08-31T06:13:18.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.872+0000 7f4342a00640 1 -- 172.21.15.16:0/3695973224 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c045c80 con 0x562df5a101b0 2024-08-31T06:13:18.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.873+0000 7f43603e7740 1 -- 172.21.15.16:0/3695973224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562df5a101b0 msgr2=0x562df5dfec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:18.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.873+0000 7f43603e7740 1 --2- 172.21.15.16:0/3695973224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562df5a101b0 0x562df5dfec00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f434c002230 tx=0x7f434c000f40 comp rx=0 tx=0).stop 2024-08-31T06:13:18.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.873+0000 7f43603e7740 1 -- 172.21.15.16:0/3695973224 shutdown_connections 2024-08-31T06:13:18.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.873+0000 7f43603e7740 1 --2- 172.21.15.16:0/3695973224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562df5a39070 0x562df5de7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:18.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.873+0000 7f43603e7740 1 --2- 172.21.15.16:0/3695973224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562df5a18390 0x562df5ddc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:18.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.873+0000 7f43603e7740 1 --2- 172.21.15.16:0/3695973224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562df5a101b0 0x562df5dfec00 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:18.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.874+0000 7f43603e7740 1 -- 172.21.15.16:0/3695973224 >> 172.21.15.16:0/3695973224 conn(0x562df5a2d460 msgr2=0x562df5ddc070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:18.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.874+0000 7f43603e7740 1 -- 172.21.15.16:0/3695973224 shutdown_connections 2024-08-31T06:13:18.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.875+0000 7f43603e7740 1 -- 172.21.15.16:0/3695973224 wait complete. 2024-08-31T06:13:18.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.875+0000 7f43603e7740 1 -- start start 2024-08-31T06:13:18.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.876+0000 7f43603e7740 1 Processor -- start 2024-08-31T06:13:18.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.879+0000 7f43603e7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562df5a101b0 0x562df5dfac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:18.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.879+0000 7f43603e7740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562df5a18390 0x562df5dfb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:18.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.879+0000 7f43603e7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562df5a39070 0x562df5df5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:18.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.879+0000 7f43603e7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f433000ce80 con 0x562df5a39070 2024-08-31T06:13:18.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.879+0000 7f43603e7740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f433000e2f0 con 0x562df5a18390 2024-08-31T06:13:18.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.879+0000 7f43603e7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f433000d550 con 0x562df5a101b0 2024-08-31T06:13:18.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.879+0000 7f43603e7740 10 client.0 _ll_register_callbacks cb 0x7f434c031a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:13:18.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.880+0000 7f4352a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562df5a101b0 0x562df5dfac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:18.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.881+0000 7f4352a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562df5a101b0 0x562df5dfac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:42862/0 (socket says 172.21.15.16:42862) 2024-08-31T06:13:18.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.881+0000 7f4352a00640 1 -- 172.21.15.16:0/2221129492 learned_addr learned my addr 172.21.15.16:0/2221129492 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:18.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.880+0000 7f4352000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562df5a18390 0x562df5dfb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:18.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.881+0000 7f4353400640 1 --2- 172.21.15.16:0/2221129492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562df5a39070 0x562df5df5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:18.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.882+0000 7f4352a00640 1 -- 172.21.15.16:0/2221129492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562df5a18390 msgr2=0x562df5dfb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:18.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.882+0000 7f4352a00640 1 --2- 172.21.15.16:0/2221129492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562df5a18390 0x562df5dfb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:18.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.882+0000 7f4352a00640 1 -- 172.21.15.16:0/2221129492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562df5a39070 msgr2=0x562df5df5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:18.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.882+0000 7f4352a00640 1 --2- 172.21.15.16:0/2221129492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562df5a39070 0x562df5df5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:18.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.882+0000 7f4352a00640 1 -- 172.21.15.16:0/2221129492 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562df5df6160 con 0x562df5a101b0 2024-08-31T06:13:18.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.884+0000 7f4352a00640 1 --2- 172.21.15.16:0/2221129492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562df5a101b0 0x562df5dfac70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f434c009430 tx=0x7f434c031ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ac081b0be0473da server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:18.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.884+0000 7f4337e00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:13:18.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.885+0000 7f4337e00640 1 -- 172.21.15.16:0/2221129492 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c05b070 con 0x562df5a101b0 2024-08-31T06:13:18.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.886+0000 7f4337e00640 1 -- 172.21.15.16:0/2221129492 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8514+0+0 (secure 0 0 0) 0x7f434c045690 con 0x562df5a101b0 2024-08-31T06:13:18.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.887+0000 7f43603e7740 1 -- 172.21.15.16:0/2221129492 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f434c036050 con 0x562df5a101b0 2024-08-31T06:13:18.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.888+0000 7f4337e00640 1 -- 172.21.15.16:0/2221129492 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f434c045890 con 0x562df5a101b0 2024-08-31T06:13:18.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.888+0000 7f43603e7740 10 client.26923 fetch_fsmap learned FSMap version 38 2024-08-31T06:13:18.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.888+0000 7f43603e7740 1 -- 172.21.15.16:0/2221129492 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x562df5e81000 con 0x562df5a101b0 2024-08-31T06:13:18.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.889+0000 7f4337e00640 1 -- 172.21.15.16:0/2221129492 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f434c036050 con 0x562df5a101b0 2024-08-31T06:13:18.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.889+0000 7f4337e00640 1 -- 172.21.15.16:0/2221129492 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f434c055030 con 0x562df5a101b0 2024-08-31T06:13:18.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.890+0000 7f43603e7740 10 client.26923 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T06:13:18.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.890+0000 7f43603e7740 10 client.26923 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T06:13:18.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.890+0000 7f43603e7740 20 client.26923 populate_metadata read hostname 'smithi016' 2024-08-31T06:13:18.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.890+0000 7f43603e7740 1 --2- 172.21.15.16:0/2221129492 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x562df59ccda0 0x562df5e836c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:18.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.891+0000 7f43603e7740 4 client.26923 mds_command: new command op to 24712 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:13:18.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.891+0000 7f43603e7740 1 -- 172.21.15.16:0/2221129492 --> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562df5d531a0 con 0x562df59ccda0 2024-08-31T06:13:18.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.895+0000 7f4352000640 1 --2- 172.21.15.16:0/2221129492 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x562df59ccda0 0x562df5e836c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:18.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.895+0000 7f4352000640 1 --2- 172.21.15.16:0/2221129492 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x562df59ccda0 0x562df5e836c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1f51a498d4d44101 server_cookie=c69fd653b4e40add in_seq=0 out_seq=0 2024-08-31T06:13:18.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.895+0000 7f4337e00640 10 client.26923 ms_handle_connect on v2:172.21.15.184:6838/4011051479 2024-08-31T06:13:18.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.896+0000 7f4337e00640 1 -- 172.21.15.16:0/2221129492 <== mds.2 v2:172.21.15.184:6838/4011051479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562df5d531a0 con 0x562df59ccda0 2024-08-31T06:13:18.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.896+0000 7f4337e00640 10 client.26923 handle_command_reply: tid=0 2024-08-31T06:13:18.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.896+0000 7f43603e7740 1 client.26923 shutdown 2024-08-31T06:13:18.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.896+0000 7f43603e7740 1 -- 172.21.15.16:0/2221129492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562df5a101b0 msgr2=0x562df5dfac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:18.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.896+0000 7f43603e7740 1 --2- 172.21.15.16:0/2221129492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562df5a101b0 0x562df5dfac70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f434c009430 tx=0x7f434c031ec0 comp rx=0 tx=0).stop 2024-08-31T06:13:18.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.896+0000 7f43603e7740 1 -- 172.21.15.16:0/2221129492 shutdown_connections 2024-08-31T06:13:18.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.896+0000 7f43603e7740 1 --2- 172.21.15.16:0/2221129492 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x562df59ccda0 0x562df5e836c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:18.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.897+0000 7f43603e7740 1 --2- 172.21.15.16:0/2221129492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562df5a39070 0x562df5df5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:18.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.897+0000 7f43603e7740 1 --2- 172.21.15.16:0/2221129492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562df5a18390 0x562df5dfb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:18.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.897+0000 7f4337e00640 0 client.26923 ms_handle_reset on v2:172.21.15.184:6838/4011051479 2024-08-31T06:13:18.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.897+0000 7f43603e7740 1 --2- 172.21.15.16:0/2221129492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562df5a101b0 0x562df5dfac70 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:18.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.897+0000 7f43603e7740 1 -- 172.21.15.16:0/2221129492 >> 172.21.15.16:0/2221129492 conn(0x562df5a2d460 msgr2=0x562df5ddae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:18.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.897+0000 7f43603e7740 1 -- 172.21.15.16:0/2221129492 shutdown_connections 2024-08-31T06:13:18.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.897+0000 7f43603e7740 1 -- 172.21.15.16:0/2221129492 wait complete. 2024-08-31T06:13:18.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.898+0000 7f43603e7740 20 client.26923 trim_cache size 0 max 16384 2024-08-31T06:13:18.901 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:13:18.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.899+0000 7f4359600640 1 -- 172.21.15.16:0/2589122847 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f43380b4fd0 msgr2=0x7f43380b74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:18.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.899+0000 7f4359600640 1 --2- 172.21.15.16:0/2589122847 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f43380b4fd0 0x7f43380b74e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f433c01d000 tx=0x7f433c01ef30 comp rx=0 tx=0).stop 2024-08-31T06:13:18.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.899+0000 7f4359600640 1 -- 172.21.15.16:0/2589122847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43541a1ea0 msgr2=0x7f43541a22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:18.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.899+0000 7f4359600640 1 --2- 172.21.15.16:0/2589122847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43541a1ea0 0x7f43541a22b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f434400b3e0 tx=0x7f434400b8a0 comp rx=0 tx=0).stop 2024-08-31T06:13:18.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.900+0000 7f4359600640 1 -- 172.21.15.16:0/2589122847 shutdown_connections 2024-08-31T06:13:18.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.900+0000 7f4359600640 1 --2- 172.21.15.16:0/2589122847 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f43380b4fd0 0x7f43380b74e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:18.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.900+0000 7f4359600640 1 --2- 172.21.15.16:0/2589122847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43541a1ea0 0x7f43541a22b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:18.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.900+0000 7f4359600640 1 --2- 172.21.15.16:0/2589122847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43541026b0 0x7f435419ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:18.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.900+0000 7f4359600640 1 --2- 172.21.15.16:0/2589122847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4354101c00 0x7f435419c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:18.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.900+0000 7f4359600640 1 -- 172.21.15.16:0/2589122847 >> 172.21.15.16:0/2589122847 conn(0x7f4354077a00 msgr2=0x7f4354100a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:18.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.900+0000 7f4359600640 1 -- 172.21.15.16:0/2589122847 shutdown_connections 2024-08-31T06:13:18.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:18.900+0000 7f4359600640 1 -- 172.21.15.16:0/2589122847 wait complete. 2024-08-31T06:13:18.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:18.988+0000 7f78d2e00640 1 -- 172.21.15.16:0/163319286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78cc104700 msgr2=0x7f78cc104b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:18.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:18.988+0000 7f78d2e00640 1 --2- 172.21.15.16:0/163319286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78cc104700 0x7f78cc104b10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f78bc009900 tx=0x7f78bc02a3f0 comp rx=0 tx=0).stop 2024-08-31T06:13:18.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:18.988+0000 7f78d2e00640 1 -- 172.21.15.16:0/163319286 shutdown_connections 2024-08-31T06:13:18.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:18.988+0000 7f78d2e00640 1 --2- 172.21.15.16:0/163319286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78cc10db20 0x7f78cc110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:18.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:18.988+0000 7f78d2e00640 1 --2- 172.21.15.16:0/163319286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78cc105050 0x7f78cc10d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:18.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:18.988+0000 7f78d2e00640 1 --2- 172.21.15.16:0/163319286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78cc104700 0x7f78cc104b10 secure :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f78bc009900 tx=0x7f78bc02a3f0 comp rx=0 tx=0).stop 2024-08-31T06:13:18.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:18.988+0000 7f78d2e00640 1 -- 172.21.15.16:0/163319286 >> 172.21.15.16:0/163319286 conn(0x7f78cc07a4e0 msgr2=0x7f78cc07a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:18.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:18.989+0000 7f78d2e00640 1 -- 172.21.15.16:0/163319286 shutdown_connections 2024-08-31T06:13:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:18.989+0000 7f78d2e00640 1 -- 172.21.15.16:0/163319286 wait complete. 2024-08-31T06:13:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:18.992+0000 7f78d2e00640 1 Processor -- start 2024-08-31T06:13:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:18.992+0000 7f78d2e00640 1 -- start start 2024-08-31T06:13:18.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:18.994+0000 7f78d2e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78cc105050 0x7f78cc19c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:18.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:18.994+0000 7f78d2e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78cc10db20 0x7f78cc19cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:18.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:18.994+0000 7f78d2e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78cc1a1d40 0x7f78cc1a2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:18.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:18.995+0000 7f78d2e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc1137f0 con 0x7f78cc105050 2024-08-31T06:13:18.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:18.995+0000 7f78cb400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78cc10db20 0x7f78cc19cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:18.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:18.995+0000 7f78cb400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78cc10db20 0x7f78cc19cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:42880/0 (socket says 172.21.15.16:42880) 2024-08-31T06:13:18.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:18.995+0000 7f78cb400640 1 -- 172.21.15.16:0/1616654669 learned_addr learned my addr 172.21.15.16:0/1616654669 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:18.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:18.995+0000 7f78d2e00640 1 -- 172.21.15.16:0/1616654669 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc113670 con 0x7f78cc1a1d40 2024-08-31T06:13:18.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:18.995+0000 7f78d2e00640 1 -- 172.21.15.16:0/1616654669 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc113970 con 0x7f78cc10db20 2024-08-31T06:13:18.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:18.995+0000 7f78d0e00640 1 --2- 172.21.15.16:0/1616654669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78cc1a1d40 0x7f78cc1a2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:18.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:18.996+0000 7f78cb400640 1 -- 172.21.15.16:0/1616654669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78cc1a1d40 msgr2=0x7f78cc1a2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:18.996+0000 7f78cb400640 1 --2- 172.21.15.16:0/1616654669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78cc1a1d40 0x7f78cc1a2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:18.996+0000 7f78cb400640 1 -- 172.21.15.16:0/1616654669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78cc105050 msgr2=0x7f78cc19c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:18.996+0000 7f78cbe00640 1 --2- 172.21.15.16:0/1616654669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78cc105050 0x7f78cc19c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:18.996+0000 7f78cb400640 1 --2- 172.21.15.16:0/1616654669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78cc105050 0x7f78cc19c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:18.996+0000 7f78cb400640 1 -- 172.21.15.16:0/1616654669 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78cc1a2920 con 0x7f78cc10db20 2024-08-31T06:13:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:18.996+0000 7f78cbe00640 1 --2- 172.21.15.16:0/1616654669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78cc105050 0x7f78cc19c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:13:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:18.997+0000 7f78cb400640 1 --2- 172.21.15.16:0/1616654669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78cc10db20 0x7f78cc19cb70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f78b800e9f0 tx=0x7f78b800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:18.997+0000 7f78c8c00640 1 -- 172.21.15.16:0/1616654669 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b800ccc0 con 0x7f78cc10db20 2024-08-31T06:13:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:18.997+0000 7f78c8c00640 1 -- 172.21.15.16:0/1616654669 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78b80108e0 con 0x7f78cc10db20 2024-08-31T06:13:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:18.997+0000 7f78d2e00640 1 -- 172.21.15.16:0/1616654669 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78cc1a8ec0 con 0x7f78cc10db20 2024-08-31T06:13:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:18.997+0000 7f78c8c00640 1 -- 172.21.15.16:0/1616654669 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b8018860 con 0x7f78cc10db20 2024-08-31T06:13:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:18.997+0000 7f78d2e00640 1 -- 172.21.15.16:0/1616654669 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78cc1a92f0 con 0x7f78cc10db20 2024-08-31T06:13:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.003+0000 7f78c8c00640 1 -- 172.21.15.16:0/1616654669 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f78b8010430 con 0x7f78cc10db20 2024-08-31T06:13:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.004+0000 7f78d9e16740 1 Processor -- start 2024-08-31T06:13:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.004+0000 7f78d9e16740 1 -- start start 2024-08-31T06:13:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.006+0000 7f78d9e16740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef2a9d1b0 0x55eef2e49130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.006+0000 7f78d9e16740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eef2a742f0 0x55eef2e49670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.006+0000 7f78d9e16740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef2a7c4d0 0x55eef2e49bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.006+0000 7f78d9e16740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55eef2db5f70 con 0x55eef2a9d1b0 2024-08-31T06:13:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.006+0000 7f78d9e16740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55eef2da5500 con 0x55eef2a7c4d0 2024-08-31T06:13:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.006+0000 7f78d9e16740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55eef2da94b0 con 0x55eef2a742f0 2024-08-31T06:13:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.006+0000 7f78c8c00640 1 --2- 172.21.15.16:0/1616654669 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f78b00b4df0 0x7f78b00b7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.007+0000 7f78c8c00640 1 -- 172.21.15.16:0/1616654669 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8514+0+0 (secure 0 0 0) 0x7f78b80e58a0 con 0x7f78cc10db20 2024-08-31T06:13:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.007+0000 7f78d0e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef2a9d1b0 0x55eef2e49130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.007+0000 7f78d0e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef2a9d1b0 0x55eef2e49130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60842/0 (socket says 172.21.15.16:60842) 2024-08-31T06:13:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.007+0000 7f78d0e00640 1 -- 172.21.15.16:0/3096838381 learned_addr learned my addr 172.21.15.16:0/3096838381 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.007+0000 7f78d0e00640 1 --2- 172.21.15.16:0/1616654669 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f78b00b4df0 0x7f78b00b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.007+0000 7f78cb400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef2a7c4d0 0x55eef2e49bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.007+0000 7f78cbe00640 1 --2- 172.21.15.16:0/3096838381 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eef2a742f0 0x55eef2e49670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:19.011 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.008+0000 7f78cbe00640 1 -- 172.21.15.16:0/3096838381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef2a7c4d0 msgr2=0x55eef2e49bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:19.011 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.008+0000 7f78cbe00640 1 --2- 172.21.15.16:0/3096838381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef2a7c4d0 0x55eef2e49bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:19.011 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.008+0000 7f78cbe00640 1 -- 172.21.15.16:0/3096838381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef2a9d1b0 msgr2=0x55eef2e49130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:19.011 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.008+0000 7f78cbe00640 1 --2- 172.21.15.16:0/3096838381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef2a9d1b0 0x55eef2e49130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:19.011 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.008+0000 7f78cbe00640 1 -- 172.21.15.16:0/3096838381 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eef2d5fc10 con 0x55eef2a742f0 2024-08-31T06:13:19.011 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.008+0000 7f78d0e00640 1 --2- 172.21.15.16:0/3096838381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef2a9d1b0 0x55eef2e49130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:13:19.011 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.008+0000 7f78cb400640 1 --2- 172.21.15.16:0/3096838381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef2a7c4d0 0x55eef2e49bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:13:19.012 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.008+0000 7f78cbe00640 1 --2- 172.21.15.16:0/3096838381 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eef2a742f0 0x55eef2e49670 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f78cc06aa00 tx=0x7f78bc005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61f741935808da39 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:19.012 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.008+0000 7f78d0e00640 1 --2- 172.21.15.16:0/1616654669 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f78b00b4df0 0x7f78b00b7300 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f78c4006d20 tx=0x7f78c4005df0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:19.012 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.009+0000 7f78c1600640 1 -- 172.21.15.16:0/3096838381 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78bc035070 con 0x55eef2a742f0 2024-08-31T06:13:19.012 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.009+0000 7f78c1600640 1 -- 172.21.15.16:0/3096838381 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78bc040070 con 0x55eef2a742f0 2024-08-31T06:13:19.012 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.009+0000 7f78c1600640 1 -- 172.21.15.16:0/3096838381 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78bc031c20 con 0x55eef2a742f0 2024-08-31T06:13:19.013 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.011+0000 7f78d9e16740 1 -- 172.21.15.16:0/3096838381 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eef2a742f0 msgr2=0x55eef2e49670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:19.013 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.011+0000 7f78d9e16740 1 --2- 172.21.15.16:0/3096838381 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eef2a742f0 0x55eef2e49670 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f78cc06aa00 tx=0x7f78bc005f10 comp rx=0 tx=0).stop 2024-08-31T06:13:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.012+0000 7f78d9e16740 1 -- 172.21.15.16:0/3096838381 shutdown_connections 2024-08-31T06:13:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.012+0000 7f78d9e16740 1 --2- 172.21.15.16:0/3096838381 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eef2a742f0 0x55eef2e49670 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.012+0000 7f78d9e16740 1 --2- 172.21.15.16:0/3096838381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef2a7c4d0 0x55eef2e49bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.012+0000 7f78d9e16740 1 --2- 172.21.15.16:0/3096838381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef2a9d1b0 0x55eef2e49130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.013+0000 7f78d9e16740 1 -- 172.21.15.16:0/3096838381 >> 172.21.15.16:0/3096838381 conn(0x55eef2a91300 msgr2=0x55eef2e47020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.013+0000 7f78d9e16740 1 -- 172.21.15.16:0/3096838381 shutdown_connections 2024-08-31T06:13:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.013+0000 7f78d9e16740 1 -- 172.21.15.16:0/3096838381 wait complete. 2024-08-31T06:13:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.014+0000 7f78d9e16740 1 -- start start 2024-08-31T06:13:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.014+0000 7f78d9e16740 1 Processor -- start 2024-08-31T06:13:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.015+0000 7f78d9e16740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eef2a742f0 0x55eef2e54780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.015+0000 7f78d9e16740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef2a7c4d0 0x55eef2e54cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.016+0000 7f78cbe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eef2a742f0 0x55eef2e54780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.016+0000 7f78cbe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eef2a742f0 0x55eef2e54780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:42910/0 (socket says 172.21.15.16:42910) 2024-08-31T06:13:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.016+0000 7f78cb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef2a7c4d0 0x55eef2e54cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.016+0000 7f78cb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef2a7c4d0 0x55eef2e54cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60854/0 (socket says 172.21.15.16:60854) 2024-08-31T06:13:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.017+0000 7f78d9e16740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef2a9d1b0 0x55eef2e60b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.017+0000 7f78d9e16740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55eef2cdeab0 con 0x55eef2a7c4d0 2024-08-31T06:13:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.017+0000 7f78d9e16740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55eef2d4c320 con 0x55eef2a9d1b0 2024-08-31T06:13:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.017+0000 7f78d9e16740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55eef2e57850 con 0x55eef2a742f0 2024-08-31T06:13:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.017+0000 7f78d9e16740 10 client.0 _ll_register_callbacks cb 0x55eef2d526a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:13:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.017+0000 7f78cb400640 1 -- 172.21.15.16:0/3218226373 learned_addr learned my addr 172.21.15.16:0/3218226373 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.017+0000 7f78d0e00640 1 --2- 172.21.15.16:0/3218226373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef2a9d1b0 0x55eef2e60b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.018+0000 7f78d0e00640 1 -- 172.21.15.16:0/3218226373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eef2a742f0 msgr2=0x55eef2e54780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.018+0000 7f78d0e00640 1 --2- 172.21.15.16:0/3218226373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eef2a742f0 0x55eef2e54780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.018+0000 7f78d0e00640 1 -- 172.21.15.16:0/3218226373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef2a7c4d0 msgr2=0x55eef2e54cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.018+0000 7f78d0e00640 1 --2- 172.21.15.16:0/3218226373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef2a7c4d0 0x55eef2e54cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.018+0000 7f78d0e00640 1 -- 172.21.15.16:0/3218226373 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eef2b08790 con 0x55eef2a9d1b0 2024-08-31T06:13:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.018+0000 7f78cbe00640 1 --2- 172.21.15.16:0/3218226373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eef2a742f0 0x55eef2e54780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:13:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.018+0000 7f78cb400640 1 --2- 172.21.15.16:0/3218226373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef2a7c4d0 0x55eef2e54cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:13:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.019+0000 7f78d0e00640 1 --2- 172.21.15.16:0/3218226373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef2a9d1b0 0x55eef2e60b40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f78c4017010 tx=0x7f78c4004b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab523a6d19ad676e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.019+0000 7f78b6000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:13:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.019+0000 7f78b6000640 1 -- 172.21.15.16:0/3218226373 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c4028060 con 0x55eef2a9d1b0 2024-08-31T06:13:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.020+0000 7f78b6000640 1 -- 172.21.15.16:0/3218226373 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8514+0+0 (secure 0 0 0) 0x7f78c401a0d0 con 0x55eef2a9d1b0 2024-08-31T06:13:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.020+0000 7f78d9e16740 1 -- 172.21.15.16:0/3218226373 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78bc040070 con 0x55eef2a9d1b0 2024-08-31T06:13:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.020+0000 7f78b6000640 1 -- 172.21.15.16:0/3218226373 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f78c4018430 con 0x55eef2a9d1b0 2024-08-31T06:13:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.021+0000 7f78d9e16740 10 client.26948 fetch_fsmap learned FSMap version 38 2024-08-31T06:13:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.021+0000 7f78d9e16740 1 -- 172.21.15.16:0/3218226373 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55eef2b81620 con 0x55eef2a9d1b0 2024-08-31T06:13:19.026 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.022+0000 7f78b6000640 1 -- 172.21.15.16:0/3218226373 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78bc040070 con 0x55eef2a9d1b0 2024-08-31T06:13:19.027 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.022+0000 7f78b6000640 1 -- 172.21.15.16:0/3218226373 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f78c4023690 con 0x55eef2a9d1b0 2024-08-31T06:13:19.027 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.023+0000 7f78d9e16740 10 client.26948 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T06:13:19.027 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.023+0000 7f78d9e16740 10 client.26948 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:13:19.027 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.023+0000 7f78d9e16740 20 client.26948 populate_metadata read hostname 'smithi016' 2024-08-31T06:13:19.027 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.023+0000 7f78d9e16740 1 --2- 172.21.15.16:0/3218226373 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55eef2a5bd80 0x55eef2e66cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:19.027 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.023+0000 7f78d9e16740 4 client.26948 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:13:19.027 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.023+0000 7f78d9e16740 1 -- 172.21.15.16:0/3218226373 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eef2db7ec0 con 0x55eef2a5bd80 2024-08-31T06:13:19.028 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.023+0000 7f78cbe00640 1 --2- 172.21.15.16:0/3218226373 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55eef2a5bd80 0x55eef2e66cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:19.028 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.025+0000 7f78cbe00640 1 --2- 172.21.15.16:0/3218226373 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55eef2a5bd80 0x55eef2e66cf0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ade3fe25ef1b8f6 server_cookie=199b0c3fbe16a66b in_seq=0 out_seq=0 2024-08-31T06:13:19.028 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.025+0000 7f78b6000640 10 client.26948 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:13:19.028 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.025+0000 7f78b6000640 1 -- 172.21.15.16:0/3218226373 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55eef2db7ec0 con 0x55eef2a5bd80 2024-08-31T06:13:19.028 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.025+0000 7f78b6000640 10 client.26948 handle_command_reply: tid=0 2024-08-31T06:13:19.028 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.026+0000 7f78d9e16740 1 client.26948 shutdown 2024-08-31T06:13:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.026+0000 7f78d9e16740 1 -- 172.21.15.16:0/3218226373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef2a9d1b0 msgr2=0x55eef2e60b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.026+0000 7f78d9e16740 1 --2- 172.21.15.16:0/3218226373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef2a9d1b0 0x55eef2e60b40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f78c4017010 tx=0x7f78c4004b50 comp rx=0 tx=0).stop 2024-08-31T06:13:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.026+0000 7f78d9e16740 1 -- 172.21.15.16:0/3218226373 shutdown_connections 2024-08-31T06:13:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.026+0000 7f78d9e16740 1 --2- 172.21.15.16:0/3218226373 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55eef2a5bd80 0x55eef2e66cf0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.026+0000 7f78d9e16740 1 --2- 172.21.15.16:0/3218226373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef2a9d1b0 0x55eef2e60b40 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.026+0000 7f78d9e16740 1 --2- 172.21.15.16:0/3218226373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef2a7c4d0 0x55eef2e54cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.026+0000 7f78d9e16740 1 --2- 172.21.15.16:0/3218226373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eef2a742f0 0x55eef2e54780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.026+0000 7f78d9e16740 1 -- 172.21.15.16:0/3218226373 >> 172.21.15.16:0/3218226373 conn(0x55eef2a91300 msgr2=0x55eef2e4a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.026+0000 7f78b6000640 0 client.26948 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:13:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.027+0000 7f78d9e16740 1 -- 172.21.15.16:0/3218226373 shutdown_connections 2024-08-31T06:13:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.027+0000 7f78d9e16740 1 -- 172.21.15.16:0/3218226373 wait complete. 2024-08-31T06:13:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.027+0000 7f78d9e16740 20 client.26948 trim_cache size 0 max 16384 2024-08-31T06:13:19.060 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.058+0000 7f78d9e16740 1 Processor -- start 2024-08-31T06:13:19.060 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.058+0000 7f78d9e16740 1 -- start start 2024-08-31T06:13:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.060+0000 7f78d9e16740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eef2a742f0 0x55eef2e54c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.060+0000 7f78d9e16740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef2a7c4d0 0x55eef2e3d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.060+0000 7f78d9e16740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef2a9d1b0 0x55eef2e60a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.060+0000 7f78d9e16740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc1b0300 con 0x55eef2a7c4d0 2024-08-31T06:13:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.060+0000 7f78d9e16740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc1adb40 con 0x55eef2a9d1b0 2024-08-31T06:13:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.060+0000 7f78d9e16740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc1ab9d0 con 0x55eef2a742f0 2024-08-31T06:13:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.060+0000 7f78cbe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eef2a742f0 0x55eef2e54c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.061+0000 7f78cbe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eef2a742f0 0x55eef2e54c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:42912/0 (socket says 172.21.15.16:42912) 2024-08-31T06:13:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.061+0000 7f78cbe00640 1 -- 172.21.15.16:0/430360484 learned_addr learned my addr 172.21.15.16:0/430360484 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.060+0000 7f78cb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef2a7c4d0 0x55eef2e3d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.061+0000 7f78d0e00640 1 --2- 172.21.15.16:0/430360484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef2a9d1b0 0x55eef2e60a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.061+0000 7f78cbe00640 1 -- 172.21.15.16:0/430360484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef2a9d1b0 msgr2=0x55eef2e60a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.061+0000 7f78cbe00640 1 --2- 172.21.15.16:0/430360484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef2a9d1b0 0x55eef2e60a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.061+0000 7f78cbe00640 1 -- 172.21.15.16:0/430360484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef2a7c4d0 msgr2=0x55eef2e3d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.061+0000 7f78cbe00640 1 --2- 172.21.15.16:0/430360484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef2a7c4d0 0x55eef2e3d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.061+0000 7f78cbe00640 1 -- 172.21.15.16:0/430360484 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eef2e39df0 con 0x55eef2a742f0 2024-08-31T06:13:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.061+0000 7f78cbe00640 1 --2- 172.21.15.16:0/430360484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eef2a742f0 0x55eef2e54c60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f78cc06aa00 tx=0x7f78bc02a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d7d570a29c8baa server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.062+0000 7f78c0c00640 1 -- 172.21.15.16:0/430360484 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78bc035040 con 0x55eef2a742f0 2024-08-31T06:13:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.062+0000 7f78c0c00640 1 -- 172.21.15.16:0/430360484 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78bc040040 con 0x55eef2a742f0 2024-08-31T06:13:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.062+0000 7f78c0c00640 1 -- 172.21.15.16:0/430360484 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78bc031d40 con 0x55eef2a742f0 2024-08-31T06:13:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.064+0000 7f78d9e16740 1 -- 172.21.15.16:0/430360484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eef2a742f0 msgr2=0x55eef2e54c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.064+0000 7f78d9e16740 1 --2- 172.21.15.16:0/430360484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eef2a742f0 0x55eef2e54c60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f78cc06aa00 tx=0x7f78bc02a3f0 comp rx=0 tx=0).stop 2024-08-31T06:13:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.064+0000 7f78d9e16740 1 -- 172.21.15.16:0/430360484 shutdown_connections 2024-08-31T06:13:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.064+0000 7f78d9e16740 1 --2- 172.21.15.16:0/430360484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef2a9d1b0 0x55eef2e60a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.064+0000 7f78d9e16740 1 --2- 172.21.15.16:0/430360484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef2a7c4d0 0x55eef2e3d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.064+0000 7f78d9e16740 1 --2- 172.21.15.16:0/430360484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eef2a742f0 0x55eef2e54c60 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.064+0000 7f78d9e16740 1 -- 172.21.15.16:0/430360484 >> 172.21.15.16:0/430360484 conn(0x55eef2a91300 msgr2=0x55eef2e3ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.064+0000 7f78d9e16740 1 -- 172.21.15.16:0/430360484 shutdown_connections 2024-08-31T06:13:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.064+0000 7f78d9e16740 1 -- 172.21.15.16:0/430360484 wait complete. 2024-08-31T06:13:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.065+0000 7f78d9e16740 1 -- start start 2024-08-31T06:13:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.065+0000 7f78d9e16740 1 Processor -- start 2024-08-31T06:13:19.068 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.065+0000 7f78d9e16740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eef2a742f0 0x55eef2e57c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.066+0000 7f78d9e16740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef2a7c4d0 0x55eef2e581d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.066+0000 7f78d9e16740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef2a9d1b0 0x55eef2e58750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.066+0000 7f78d9e16740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc1ac020 con 0x55eef2a7c4d0 2024-08-31T06:13:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.066+0000 7f78d9e16740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc1ad470 con 0x55eef2a9d1b0 2024-08-31T06:13:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.066+0000 7f78d9e16740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc1ac6d0 con 0x55eef2a742f0 2024-08-31T06:13:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.066+0000 7f78d9e16740 10 client.0 _ll_register_callbacks cb 0x7f78c4016f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:13:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.066+0000 7f78cbe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eef2a742f0 0x55eef2e57c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.066+0000 7f78cbe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eef2a742f0 0x55eef2e57c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:42914/0 (socket says 172.21.15.16:42914) 2024-08-31T06:13:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.066+0000 7f78cbe00640 1 -- 172.21.15.16:0/2995318429 learned_addr learned my addr 172.21.15.16:0/2995318429 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.066+0000 7f78cb400640 1 --2- 172.21.15.16:0/2995318429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef2a7c4d0 0x55eef2e581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.066+0000 7f78d0e00640 1 --2- 172.21.15.16:0/2995318429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef2a9d1b0 0x55eef2e58750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:19.071 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.067+0000 7f78cbe00640 1 -- 172.21.15.16:0/2995318429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef2a9d1b0 msgr2=0x55eef2e58750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:19.071 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.067+0000 7f78cbe00640 1 --2- 172.21.15.16:0/2995318429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef2a9d1b0 0x55eef2e58750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:19.071 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.067+0000 7f78cbe00640 1 -- 172.21.15.16:0/2995318429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef2a7c4d0 msgr2=0x55eef2e581d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:19.071 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.067+0000 7f78cbe00640 1 --2- 172.21.15.16:0/2995318429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef2a7c4d0 0x55eef2e581d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:19.071 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.067+0000 7f78cbe00640 1 -- 172.21.15.16:0/2995318429 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eef2ecf8f0 con 0x55eef2a742f0 2024-08-31T06:13:19.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.069+0000 7f78cbe00640 1 --2- 172.21.15.16:0/2995318429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eef2a742f0 0x55eef2e57c90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f78bc031430 tx=0x7f78bc0098c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba4e810e06be4457 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:19.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.069+0000 7f78b6000640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:13:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.071+0000 7f78b6000640 1 -- 172.21.15.16:0/2995318429 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78bc035070 con 0x55eef2a742f0 2024-08-31T06:13:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.071+0000 7f78b6000640 1 -- 172.21.15.16:0/2995318429 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8514+0+0 (secure 0 0 0) 0x7f78bc031cc0 con 0x55eef2a742f0 2024-08-31T06:13:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.071+0000 7f78d9e16740 1 -- 172.21.15.16:0/2995318429 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78bc040040 con 0x55eef2a742f0 2024-08-31T06:13:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.071+0000 7f78b6000640 1 -- 172.21.15.16:0/2995318429 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f78bc005030 con 0x55eef2a742f0 2024-08-31T06:13:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.071+0000 7f78d9e16740 10 client.26941 fetch_fsmap learned FSMap version 38 2024-08-31T06:13:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.072+0000 7f78d9e16740 1 -- 172.21.15.16:0/2995318429 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55eef2ed0b80 con 0x55eef2a742f0 2024-08-31T06:13:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.072+0000 7f78b6000640 1 -- 172.21.15.16:0/2995318429 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78bc040040 con 0x55eef2a742f0 2024-08-31T06:13:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.073+0000 7f78b6000640 1 -- 172.21.15.16:0/2995318429 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f78bc056070 con 0x55eef2a742f0 2024-08-31T06:13:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.073+0000 7f78d9e16740 10 client.26941 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T06:13:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.073+0000 7f78d9e16740 10 client.26941 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:13:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.073+0000 7f78d9e16740 20 client.26941 populate_metadata read hostname 'smithi016' 2024-08-31T06:13:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.073+0000 7f78d9e16740 1 --2- 172.21.15.16:0/2995318429 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55eef2a311c0 0x55eef2ed2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.073+0000 7f78d9e16740 4 client.26941 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:13:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.073+0000 7f78d9e16740 1 -- 172.21.15.16:0/2995318429 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eef2db3960 con 0x55eef2a311c0 2024-08-31T06:13:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.074+0000 7f78cb400640 1 --2- 172.21.15.16:0/2995318429 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55eef2a311c0 0x55eef2ed2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.075+0000 7f78cb400640 1 --2- 172.21.15.16:0/2995318429 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55eef2a311c0 0x55eef2ed2f20 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68c49ae0e4179bc0 server_cookie=261d401afc430333 in_seq=0 out_seq=0 2024-08-31T06:13:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.075+0000 7f78b6000640 10 client.26941 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:13:19.125 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.123+0000 7f78b6000640 1 -- 172.21.15.16:0/2995318429 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eef2db3960 con 0x55eef2a311c0 2024-08-31T06:13:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.123+0000 7f78b6000640 10 client.26941 handle_command_reply: tid=0 2024-08-31T06:13:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.123+0000 7f78d9e16740 1 client.26941 shutdown 2024-08-31T06:13:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.124+0000 7f78d9e16740 1 -- 172.21.15.16:0/2995318429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eef2a742f0 msgr2=0x55eef2e57c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.124+0000 7f78d9e16740 1 --2- 172.21.15.16:0/2995318429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eef2a742f0 0x55eef2e57c90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f78bc031430 tx=0x7f78bc0098c0 comp rx=0 tx=0).stop 2024-08-31T06:13:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.124+0000 7f78d9e16740 1 -- 172.21.15.16:0/2995318429 shutdown_connections 2024-08-31T06:13:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.124+0000 7f78d9e16740 1 --2- 172.21.15.16:0/2995318429 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55eef2a311c0 0x55eef2ed2f20 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.124+0000 7f78d9e16740 1 --2- 172.21.15.16:0/2995318429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eef2a9d1b0 0x55eef2e58750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.124+0000 7f78d9e16740 1 --2- 172.21.15.16:0/2995318429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eef2a7c4d0 0x55eef2e581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.124+0000 7f78d9e16740 1 --2- 172.21.15.16:0/2995318429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eef2a742f0 0x55eef2e57c90 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.124+0000 7f78b6000640 0 client.26941 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:13:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.124+0000 7f78d9e16740 1 -- 172.21.15.16:0/2995318429 >> 172.21.15.16:0/2995318429 conn(0x55eef2a91300 msgr2=0x55eef2e4a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.125+0000 7f78d9e16740 1 -- 172.21.15.16:0/2995318429 shutdown_connections 2024-08-31T06:13:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.125+0000 7f78d9e16740 1 -- 172.21.15.16:0/2995318429 wait complete. 2024-08-31T06:13:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.125+0000 7f78d9e16740 20 client.26941 trim_cache size 0 max 16384 2024-08-31T06:13:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:13:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:13:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:13:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:13:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.126+0000 7f78d2e00640 1 -- 172.21.15.16:0/1616654669 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f78b00b4df0 msgr2=0x7f78b00b7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.126+0000 7f78d2e00640 1 --2- 172.21.15.16:0/1616654669 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f78b00b4df0 0x7f78b00b7300 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f78c4006d20 tx=0x7f78c4005df0 comp rx=0 tx=0).stop 2024-08-31T06:13:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.126+0000 7f78d2e00640 1 -- 172.21.15.16:0/1616654669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78cc10db20 msgr2=0x7f78cc19cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.126+0000 7f78d2e00640 1 --2- 172.21.15.16:0/1616654669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78cc10db20 0x7f78cc19cb70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f78b800e9f0 tx=0x7f78b800eeb0 comp rx=0 tx=0).stop 2024-08-31T06:13:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.126+0000 7f78d2e00640 1 -- 172.21.15.16:0/1616654669 shutdown_connections 2024-08-31T06:13:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.126+0000 7f78d2e00640 1 --2- 172.21.15.16:0/1616654669 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f78b00b4df0 0x7f78b00b7300 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.126+0000 7f78d2e00640 1 --2- 172.21.15.16:0/1616654669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78cc10db20 0x7f78cc19cb70 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.126+0000 7f78d2e00640 1 --2- 172.21.15.16:0/1616654669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78cc1a1d40 0x7f78cc1a2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.126+0000 7f78d2e00640 1 --2- 172.21.15.16:0/1616654669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78cc105050 0x7f78cc19c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.126+0000 7f78d2e00640 1 -- 172.21.15.16:0/1616654669 >> 172.21.15.16:0/1616654669 conn(0x7f78cc07a4e0 msgr2=0x7f78cc100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.126+0000 7f78d2e00640 1 -- 172.21.15.16:0/1616654669 shutdown_connections 2024-08-31T06:13:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:19.126+0000 7f78d2e00640 1 -- 172.21.15.16:0/1616654669 wait complete. 2024-08-31T06:13:19.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:19 smithi184 ceph-mon[22331]: pgmap v547: 97 pgs: 97 active+clean; 3.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 25 MiB/s wr, 876 op/s 2024-08-31T06:13:19.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:19 smithi155 ceph-mon[21742]: pgmap v547: 97 pgs: 97 active+clean; 3.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 25 MiB/s wr, 876 op/s 2024-08-31T06:13:19.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:19 smithi016 ceph-mon[17194]: pgmap v547: 97 pgs: 97 active+clean; 3.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 25 MiB/s wr, 876 op/s 2024-08-31T06:13:20.298 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T06:13:20.954 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:13:21.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:21 smithi016 ceph-mon[17194]: pgmap v548: 97 pgs: 97 active+clean; 3.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 26 MiB/s wr, 1.24k op/s 2024-08-31T06:13:21.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:21 smithi016 ceph-mon[17194]: osdmap e83: 12 total, 12 up, 12 in 2024-08-31T06:13:21.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:21 smithi184 ceph-mon[22331]: pgmap v548: 97 pgs: 97 active+clean; 3.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 26 MiB/s wr, 1.24k op/s 2024-08-31T06:13:21.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:21 smithi184 ceph-mon[22331]: osdmap e83: 12 total, 12 up, 12 in 2024-08-31T06:13:21.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:21 smithi155 ceph-mon[21742]: pgmap v548: 97 pgs: 97 active+clean; 3.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 26 MiB/s wr, 1.24k op/s 2024-08-31T06:13:21.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:21 smithi155 ceph-mon[21742]: osdmap e83: 12 total, 12 up, 12 in 2024-08-31T06:13:21.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 355.0 s, 1838.6 tps, lat 54.381 ms stddev 95.392 2024-08-31T06:13:21.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.749+0000 7fc9baa00640 1 -- 172.21.15.16:0/4130415260 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a80318a0 con 0x7fc9bc113c50 2024-08-31T06:13:21.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.750+0000 7fc9c2e00640 1 -- 172.21.15.16:0/4130415260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9bc113c50 msgr2=0x7fc9bc114060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:21.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.750+0000 7fc9c2e00640 1 --2- 172.21.15.16:0/4130415260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9bc113c50 0x7fc9bc114060 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc9a8009900 tx=0x7fc9a802a410 comp rx=0 tx=0).stop 2024-08-31T06:13:21.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.750+0000 7fc9c2e00640 1 -- 172.21.15.16:0/4130415260 shutdown_connections 2024-08-31T06:13:21.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.750+0000 7fc9c2e00640 1 --2- 172.21.15.16:0/4130415260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9bc113c50 0x7fc9bc114060 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:21.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.750+0000 7fc9c2e00640 1 --2- 172.21.15.16:0/4130415260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9bc1145a0 0x7fc9bc11add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:21.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.750+0000 7fc9c2e00640 1 --2- 172.21.15.16:0/4130415260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9bc10e360 0x7fc9bc10e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:21.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.750+0000 7fc9c2e00640 1 -- 172.21.15.16:0/4130415260 >> 172.21.15.16:0/4130415260 conn(0x7fc9bc10a160 msgr2=0x7fc9bc10c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:21.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.750+0000 7fc9c2e00640 1 -- 172.21.15.16:0/4130415260 shutdown_connections 2024-08-31T06:13:21.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.750+0000 7fc9c2e00640 1 -- 172.21.15.16:0/4130415260 wait complete. 2024-08-31T06:13:21.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.751+0000 7fc9c2e00640 1 Processor -- start 2024-08-31T06:13:21.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.752+0000 7fc9c2e00640 1 -- start start 2024-08-31T06:13:21.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.752+0000 7fc9c2e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9bc10e360 0x7fc9bc072b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:21.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.752+0000 7fc9c2e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9bc113c50 0x7fc9bc06ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:21.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.752+0000 7fc9c2e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9bc1145a0 0x7fc9bc06f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:21.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.752+0000 7fc9c2e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc9bc11e230 con 0x7fc9bc1145a0 2024-08-31T06:13:21.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.752+0000 7fc9c2e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc9bc11e0b0 con 0x7fc9bc113c50 2024-08-31T06:13:21.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.752+0000 7fc9c2e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc9bc11e3b0 con 0x7fc9bc10e360 2024-08-31T06:13:21.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.752+0000 7fc9bbe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9bc10e360 0x7fc9bc072b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:21.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.752+0000 7fc9bb400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9bc113c50 0x7fc9bc06ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:21.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.752+0000 7fc9bbe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9bc10e360 0x7fc9bc072b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:42938/0 (socket says 172.21.15.16:42938) 2024-08-31T06:13:21.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.752+0000 7fc9bbe00640 1 -- 172.21.15.16:0/2811401089 learned_addr learned my addr 172.21.15.16:0/2811401089 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:21.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.753+0000 7fc9c0e00640 1 --2- 172.21.15.16:0/2811401089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9bc1145a0 0x7fc9bc06f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:21.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.753+0000 7fc9bbe00640 1 -- 172.21.15.16:0/2811401089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9bc113c50 msgr2=0x7fc9bc06ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:21.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.753+0000 7fc9bbe00640 1 --2- 172.21.15.16:0/2811401089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9bc113c50 0x7fc9bc06ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:21.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.753+0000 7fc9bbe00640 1 -- 172.21.15.16:0/2811401089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9bc1145a0 msgr2=0x7fc9bc06f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:21.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.753+0000 7fc9bbe00640 1 --2- 172.21.15.16:0/2811401089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9bc1145a0 0x7fc9bc06f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:21.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.753+0000 7fc9bbe00640 1 -- 172.21.15.16:0/2811401089 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9bc1b5190 con 0x7fc9bc10e360 2024-08-31T06:13:21.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.754+0000 7fc9bbe00640 1 --2- 172.21.15.16:0/2811401089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9bc10e360 0x7fc9bc072b70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac00e9f0 tx=0x7fc9ac00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:21.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.754+0000 7fc9b8c00640 1 -- 172.21.15.16:0/2811401089 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac00cd80 con 0x7fc9bc10e360 2024-08-31T06:13:21.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.754+0000 7fc9b8c00640 1 -- 172.21.15.16:0/2811401089 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9ac0108e0 con 0x7fc9bc10e360 2024-08-31T06:13:21.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.754+0000 7fc9b8c00640 1 -- 172.21.15.16:0/2811401089 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac018980 con 0x7fc9bc10e360 2024-08-31T06:13:21.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.754+0000 7fc9c2e00640 1 -- 172.21.15.16:0/2811401089 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9bc1b53c0 con 0x7fc9bc10e360 2024-08-31T06:13:21.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.755+0000 7fc9c2e00640 1 -- 172.21.15.16:0/2811401089 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9bc1b5880 con 0x7fc9bc10e360 2024-08-31T06:13:21.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.760+0000 7fc9b8c00640 1 -- 172.21.15.16:0/2811401089 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fc9ac010430 con 0x7fc9bc10e360 2024-08-31T06:13:21.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.760+0000 7fc9c9bef740 1 Processor -- start 2024-08-31T06:13:21.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.761+0000 7fc9c9bef740 1 -- start start 2024-08-31T06:13:21.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.761+0000 7fc9c9bef740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a73414f070 0x55a7344f48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:21.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.761+0000 7fc9c9bef740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7341261b0 0x55a7344f4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:21.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.761+0000 7fc9c9bef740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73412e390 0x55a7344f5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:21.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.761+0000 7fc9c9bef740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a73446b7b0 con 0x55a73412e390 2024-08-31T06:13:21.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.761+0000 7fc9c9bef740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a73445ad40 con 0x55a73414f070 2024-08-31T06:13:21.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.761+0000 7fc9c9bef740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a73445ecf0 con 0x55a7341261b0 2024-08-31T06:13:21.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.762+0000 7fc9c0e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a73414f070 0x55a7344f48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:21.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.762+0000 7fc9bbe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7341261b0 0x55a7344f4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:21.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.762+0000 7fc9c0e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a73414f070 0x55a7344f48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:57518/0 (socket says 172.21.15.16:57518) 2024-08-31T06:13:21.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.762+0000 7fc9c0e00640 1 -- 172.21.15.16:0/3763550524 learned_addr learned my addr 172.21.15.16:0/3763550524 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:21.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.762+0000 7fc9bbe00640 1 -- 172.21.15.16:0/3763550524 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a73414f070 msgr2=0x55a7344f48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:21.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.762+0000 7fc9bbe00640 1 --2- 172.21.15.16:0/3763550524 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a73414f070 0x55a7344f48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:21.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.762+0000 7fc9bbe00640 1 -- 172.21.15.16:0/3763550524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73412e390 msgr2=0x55a7344f5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:21.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.762+0000 7fc9bbe00640 1 --2- 172.21.15.16:0/3763550524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73412e390 0x55a7344f5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:21.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.762+0000 7fc9bbe00640 1 -- 172.21.15.16:0/3763550524 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7341ba650 con 0x55a7341261b0 2024-08-31T06:13:21.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.763+0000 7fc9bbe00640 1 --2- 172.21.15.16:0/3763550524 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7341261b0 0x55a7344f4df0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac0e30a0 tx=0x7fc9ac0ed690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a8a303b82a112a2 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:21.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.763+0000 7fc9b1600640 1 -- 172.21.15.16:0/3763550524 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac0f4070 con 0x55a7341261b0 2024-08-31T06:13:21.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.764+0000 7fc9b8c00640 1 --2- 172.21.15.16:0/2811401089 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc9a00b4d20 0x7fc9a00b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:21.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.764+0000 7fc9b1600640 1 -- 172.21.15.16:0/3763550524 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9ac0f0af0 con 0x55a7341261b0 2024-08-31T06:13:21.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.764+0000 7fc9b1600640 1 -- 172.21.15.16:0/3763550524 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac0ef960 con 0x55a7341261b0 2024-08-31T06:13:21.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.764+0000 7fc9b8c00640 1 -- 172.21.15.16:0/2811401089 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8470+0+0 (secure 0 0 0) 0x7fc9ac0e5920 con 0x7fc9bc10e360 2024-08-31T06:13:21.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.765+0000 7fc9c0e00640 1 --2- 172.21.15.16:0/2811401089 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc9a00b4d20 0x7fc9a00b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:21.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.765+0000 7fc9c9bef740 1 -- 172.21.15.16:0/3763550524 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7341261b0 msgr2=0x55a7344f4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:21.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.765+0000 7fc9c9bef740 1 --2- 172.21.15.16:0/3763550524 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7341261b0 0x55a7344f4df0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac0e30a0 tx=0x7fc9ac0ed690 comp rx=0 tx=0).stop 2024-08-31T06:13:21.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.765+0000 7fc9c9bef740 1 -- 172.21.15.16:0/3763550524 shutdown_connections 2024-08-31T06:13:21.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.765+0000 7fc9c9bef740 1 --2- 172.21.15.16:0/3763550524 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7341261b0 0x55a7344f4df0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:21.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.765+0000 7fc9c9bef740 1 --2- 172.21.15.16:0/3763550524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73412e390 0x55a7344f5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:21.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.765+0000 7fc9c9bef740 1 --2- 172.21.15.16:0/3763550524 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a73414f070 0x55a7344f48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:21.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.765+0000 7fc9c9bef740 1 -- 172.21.15.16:0/3763550524 >> 172.21.15.16:0/3763550524 conn(0x55a734143460 msgr2=0x55a7344f27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:21.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.765+0000 7fc9c9bef740 1 -- 172.21.15.16:0/3763550524 shutdown_connections 2024-08-31T06:13:21.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.765+0000 7fc9c9bef740 1 -- 172.21.15.16:0/3763550524 wait complete. 2024-08-31T06:13:21.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.766+0000 7fc9c0e00640 1 --2- 172.21.15.16:0/2811401089 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc9a00b4d20 0x7fc9a00b7230 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4011670 tx=0x7fc9b4009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:21.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.767+0000 7fc9c9bef740 1 -- start start 2024-08-31T06:13:21.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.767+0000 7fc9c9bef740 1 Processor -- start 2024-08-31T06:13:21.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.768+0000 7fc9c9bef740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7341261b0 0x55a734500580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:21.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.768+0000 7fc9c9bef740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a73412e390 0x55a734500b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:21.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.768+0000 7fc9c9bef740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a73414f070 0x55a73450c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:21.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.768+0000 7fc9c9bef740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a7342bd700 con 0x55a7341261b0 2024-08-31T06:13:21.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.768+0000 7fc9c9bef740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a7343e1cf0 con 0x55a73414f070 2024-08-31T06:13:21.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.768+0000 7fc9c9bef740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a734503090 con 0x55a73412e390 2024-08-31T06:13:21.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.768+0000 7fc9c9bef740 10 client.0 _ll_register_callbacks cb 0x55a734407470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:13:21.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.768+0000 7fc9bbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7341261b0 0x55a734500580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:21.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.768+0000 7fc9c0e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a73414f070 0x55a73450c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:21.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.768+0000 7fc9c0e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a73414f070 0x55a73450c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:57520/0 (socket says 172.21.15.16:57520) 2024-08-31T06:13:21.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.768+0000 7fc9c0e00640 1 -- 172.21.15.16:0/2613578366 learned_addr learned my addr 172.21.15.16:0/2613578366 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:21.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.768+0000 7fc9bb400640 1 --2- 172.21.15.16:0/2613578366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a73412e390 0x55a734500b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:21.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.769+0000 7fc9c0e00640 1 -- 172.21.15.16:0/2613578366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a73412e390 msgr2=0x55a734500b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:21.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.769+0000 7fc9c0e00640 1 --2- 172.21.15.16:0/2613578366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a73412e390 0x55a734500b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:21.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.769+0000 7fc9c0e00640 1 -- 172.21.15.16:0/2613578366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7341261b0 msgr2=0x55a734500580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:21.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.769+0000 7fc9c0e00640 1 --2- 172.21.15.16:0/2613578366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7341261b0 0x55a734500580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:21.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.769+0000 7fc9c0e00640 1 -- 172.21.15.16:0/2613578366 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7344228b0 con 0x55a73414f070 2024-08-31T06:13:21.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.769+0000 7fc9bbe00640 1 --2- 172.21.15.16:0/2613578366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7341261b0 0x55a734500580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:13:21.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.770+0000 7fc9c0e00640 1 --2- 172.21.15.16:0/2613578366 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a73414f070 0x55a73450c3e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4003f60 tx=0x7fc9b4004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d45f7032ba03f213 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:21.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.770+0000 7fc9a6000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:13:21.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.770+0000 7fc9a6000640 1 -- 172.21.15.16:0/2613578366 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b400f9d0 con 0x55a73414f070 2024-08-31T06:13:21.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.771+0000 7fc9a6000640 1 -- 172.21.15.16:0/2613578366 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8470+0+0 (secure 0 0 0) 0x7fc9b400fb70 con 0x55a73414f070 2024-08-31T06:13:21.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.771+0000 7fc9c9bef740 1 -- 172.21.15.16:0/2613578366 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9ac0f0af0 con 0x55a73414f070 2024-08-31T06:13:21.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.771+0000 7fc9a6000640 1 -- 172.21.15.16:0/2613578366 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fc9b400fd70 con 0x55a73414f070 2024-08-31T06:13:21.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.772+0000 7fc9c9bef740 10 client.26966 fetch_fsmap learned FSMap version 38 2024-08-31T06:13:21.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.772+0000 7fc9c9bef740 1 -- 172.21.15.16:0/2613578366 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55a73450f260 con 0x55a73414f070 2024-08-31T06:13:21.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.772+0000 7fc9a6000640 1 -- 172.21.15.16:0/2613578366 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9ac0f0af0 con 0x55a73414f070 2024-08-31T06:13:21.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.772+0000 7fc9a6000640 1 -- 172.21.15.16:0/2613578366 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7fc9b4026070 con 0x55a73414f070 2024-08-31T06:13:21.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.773+0000 7fc9c9bef740 10 client.26966 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T06:13:21.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.773+0000 7fc9c9bef740 10 client.26966 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-31T06:13:21.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.773+0000 7fc9c9bef740 20 client.26966 populate_metadata read hostname 'smithi016' 2024-08-31T06:13:21.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.773+0000 7fc9c9bef740 1 --2- 172.21.15.16:0/2613578366 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55a73410d9f0 0x55a734512800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:21.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.773+0000 7fc9c9bef740 4 client.26966 mds_command: new command op to 24724 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:13:21.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.773+0000 7fc9c9bef740 1 -- 172.21.15.16:0/2613578366 --> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a73446d700 con 0x55a73410d9f0 2024-08-31T06:13:21.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.775+0000 7fc9bbe00640 1 --2- 172.21.15.16:0/2613578366 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55a73410d9f0 0x55a734512800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:21.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.777+0000 7fc9bbe00640 1 --2- 172.21.15.16:0/2613578366 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55a73410d9f0 0x55a734512800 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=46e62dba47f0b516 server_cookie=2dc4713740b89124 in_seq=0 out_seq=0 2024-08-31T06:13:21.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.777+0000 7fc9a6000640 10 client.26966 ms_handle_connect on v2:172.21.15.16:6838/219664460 2024-08-31T06:13:21.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.777+0000 7fc9a6000640 1 -- 172.21.15.16:0/2613578366 <== mds.1 v2:172.21.15.16:6838/219664460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a73446d700 con 0x55a73410d9f0 2024-08-31T06:13:21.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.777+0000 7fc9a6000640 10 client.26966 handle_command_reply: tid=0 2024-08-31T06:13:21.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.777+0000 7fc9c9bef740 1 client.26966 shutdown 2024-08-31T06:13:21.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.778+0000 7fc9c9bef740 1 -- 172.21.15.16:0/2613578366 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a73414f070 msgr2=0x55a73450c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:21.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.778+0000 7fc9c9bef740 1 --2- 172.21.15.16:0/2613578366 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a73414f070 0x55a73450c3e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4003f60 tx=0x7fc9b4004ab0 comp rx=0 tx=0).stop 2024-08-31T06:13:21.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.778+0000 7fc9c9bef740 1 -- 172.21.15.16:0/2613578366 shutdown_connections 2024-08-31T06:13:21.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.778+0000 7fc9c9bef740 1 --2- 172.21.15.16:0/2613578366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a73412e390 0x55a734500b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:21.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.778+0000 7fc9c9bef740 1 --2- 172.21.15.16:0/2613578366 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55a73410d9f0 0x55a734512800 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:21.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.778+0000 7fc9c9bef740 1 --2- 172.21.15.16:0/2613578366 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a73414f070 0x55a73450c3e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:21.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.778+0000 7fc9c9bef740 1 --2- 172.21.15.16:0/2613578366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7341261b0 0x55a734500580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:21.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.778+0000 7fc9c9bef740 1 -- 172.21.15.16:0/2613578366 >> 172.21.15.16:0/2613578366 conn(0x55a734143460 msgr2=0x55a7344f7a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:21.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.778+0000 7fc9a6000640 0 client.26966 ms_handle_reset on v2:172.21.15.16:6838/219664460 2024-08-31T06:13:21.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.778+0000 7fc9c9bef740 1 -- 172.21.15.16:0/2613578366 shutdown_connections 2024-08-31T06:13:21.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.779+0000 7fc9c9bef740 1 -- 172.21.15.16:0/2613578366 wait complete. 2024-08-31T06:13:21.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.779+0000 7fc9c9bef740 20 client.26966 trim_cache size 0 max 16384 2024-08-31T06:13:21.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.802+0000 7fc9c9bef740 1 Processor -- start 2024-08-31T06:13:21.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.802+0000 7fc9c9bef740 1 -- start start 2024-08-31T06:13:21.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.803+0000 7fc9c9bef740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7341261b0 0x55a7344fec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:21.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.803+0000 7fc9c9bef740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73412e390 0x55a7344ff140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:21.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.803+0000 7fc9c9bef740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a73414f070 0x55a73450c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:21.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.803+0000 7fc9c9bef740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc9bc1c0c80 con 0x55a73412e390 2024-08-31T06:13:21.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.803+0000 7fc9c9bef740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc9bc1be4c0 con 0x55a73414f070 2024-08-31T06:13:21.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.803+0000 7fc9c9bef740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc9bc1bc330 con 0x55a7341261b0 2024-08-31T06:13:21.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.803+0000 7fc9bbe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7341261b0 0x55a7344fec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:21.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.803+0000 7fc9bbe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7341261b0 0x55a7344fec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:42962/0 (socket says 172.21.15.16:42962) 2024-08-31T06:13:21.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.803+0000 7fc9bbe00640 1 -- 172.21.15.16:0/1744160092 learned_addr learned my addr 172.21.15.16:0/1744160092 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:21.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.803+0000 7fc9c0e00640 1 --2- 172.21.15.16:0/1744160092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a73414f070 0x55a73450c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:21.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.803+0000 7fc9bb400640 1 --2- 172.21.15.16:0/1744160092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73412e390 0x55a7344ff140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:21.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.804+0000 7fc9c0e00640 1 -- 172.21.15.16:0/1744160092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7341261b0 msgr2=0x55a7344fec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:21.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.804+0000 7fc9c0e00640 1 --2- 172.21.15.16:0/1744160092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7341261b0 0x55a7344fec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:21.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.804+0000 7fc9c0e00640 1 -- 172.21.15.16:0/1744160092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73412e390 msgr2=0x55a7344ff140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:21.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.804+0000 7fc9c0e00640 1 --2- 172.21.15.16:0/1744160092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73412e390 0x55a7344ff140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:21.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.804+0000 7fc9c0e00640 1 -- 172.21.15.16:0/1744160092 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a73450f070 con 0x55a73414f070 2024-08-31T06:13:21.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.804+0000 7fc9bb400640 1 --2- 172.21.15.16:0/1744160092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73412e390 0x55a7344ff140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:13:21.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.804+0000 7fc9bbe00640 1 --2- 172.21.15.16:0/1744160092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7341261b0 0x55a7344fec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:13:21.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.804+0000 7fc9c0e00640 1 --2- 172.21.15.16:0/1744160092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a73414f070 0x55a73450c2e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc9b401e710 tx=0x7fc9b400fb60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10d67c979fc1dd37 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:21.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.804+0000 7fc9b0c00640 1 -- 172.21.15.16:0/1744160092 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b4035070 con 0x55a73414f070 2024-08-31T06:13:21.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.804+0000 7fc9b0c00640 1 -- 172.21.15.16:0/1744160092 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9b401e990 con 0x55a73414f070 2024-08-31T06:13:21.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.804+0000 7fc9b0c00640 1 -- 172.21.15.16:0/1744160092 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b401edc0 con 0x55a73414f070 2024-08-31T06:13:21.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.805+0000 7fc9c9bef740 1 -- 172.21.15.16:0/1744160092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a73414f070 msgr2=0x55a73450c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:21.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.805+0000 7fc9c9bef740 1 --2- 172.21.15.16:0/1744160092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a73414f070 0x55a73450c2e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc9b401e710 tx=0x7fc9b400fb60 comp rx=0 tx=0).stop 2024-08-31T06:13:21.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.806+0000 7fc9c9bef740 1 -- 172.21.15.16:0/1744160092 shutdown_connections 2024-08-31T06:13:21.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.806+0000 7fc9c9bef740 1 --2- 172.21.15.16:0/1744160092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a73414f070 0x55a73450c2e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:21.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.806+0000 7fc9c9bef740 1 --2- 172.21.15.16:0/1744160092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73412e390 0x55a7344ff140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:21.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.806+0000 7fc9c9bef740 1 --2- 172.21.15.16:0/1744160092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7341261b0 0x55a7344fec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:21.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.806+0000 7fc9c9bef740 1 -- 172.21.15.16:0/1744160092 >> 172.21.15.16:0/1744160092 conn(0x55a734143460 msgr2=0x55a7344fb530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:21.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.806+0000 7fc9c9bef740 1 -- 172.21.15.16:0/1744160092 shutdown_connections 2024-08-31T06:13:21.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.806+0000 7fc9c9bef740 1 -- 172.21.15.16:0/1744160092 wait complete. 2024-08-31T06:13:21.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.807+0000 7fc9c9bef740 1 -- start start 2024-08-31T06:13:21.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.807+0000 7fc9c9bef740 1 Processor -- start 2024-08-31T06:13:21.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.807+0000 7fc9c9bef740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a7341261b0 0x55a734508350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:21.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.807+0000 7fc9c9bef740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73412e390 0x55a734508890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:21.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.807+0000 7fc9c9bef740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a73414f070 0x55a734503300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:21.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.807+0000 7fc9c9bef740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc9bc1bc980 con 0x55a73412e390 2024-08-31T06:13:21.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.807+0000 7fc9c9bef740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc9bc1bddf0 con 0x55a7341261b0 2024-08-31T06:13:21.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.807+0000 7fc9c9bef740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc9bc1bd050 con 0x55a73414f070 2024-08-31T06:13:21.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.808+0000 7fc9c9bef740 10 client.0 _ll_register_callbacks cb 0x7fc9b4003e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:13:21.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.810+0000 7fc9bb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73412e390 0x55a734508890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:21.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.810+0000 7fc9bb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73412e390 0x55a734508890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60936/0 (socket says 172.21.15.16:60936) 2024-08-31T06:13:21.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.810+0000 7fc9bb400640 1 -- 172.21.15.16:0/2117722842 learned_addr learned my addr 172.21.15.16:0/2117722842 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:21.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.811+0000 7fc9bbe00640 1 --2- 172.21.15.16:0/2117722842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a7341261b0 0x55a734508350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:21.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.812+0000 7fc9c0e00640 1 --2- 172.21.15.16:0/2117722842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a73414f070 0x55a734503300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:21.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.814+0000 7fc9bbe00640 1 -- 172.21.15.16:0/2117722842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a73414f070 msgr2=0x55a734503300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:21.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.814+0000 7fc9bbe00640 1 --2- 172.21.15.16:0/2117722842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a73414f070 0x55a734503300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:21.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.814+0000 7fc9bbe00640 1 -- 172.21.15.16:0/2117722842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73412e390 msgr2=0x55a734508890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:21.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.814+0000 7fc9bbe00640 1 --2- 172.21.15.16:0/2117722842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73412e390 0x55a734508890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:21.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.815+0000 7fc9bbe00640 1 -- 172.21.15.16:0/2117722842 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a734509040 con 0x55a7341261b0 2024-08-31T06:13:21.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.815+0000 7fc9bbe00640 1 --2- 172.21.15.16:0/2117722842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a7341261b0 0x55a734508350 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac0a80c0 tx=0x7fc9ac0ebe60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d240d610744bba6d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:21.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.816+0000 7fc9a6000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:13:21.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.817+0000 7fc9a6000640 1 -- 172.21.15.16:0/2117722842 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac0f4040 con 0x55a7341261b0 2024-08-31T06:13:21.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.817+0000 7fc9a6000640 1 -- 172.21.15.16:0/2117722842 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8470+0+0 (secure 0 0 0) 0x7fc9ac010a90 con 0x55a7341261b0 2024-08-31T06:13:21.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.818+0000 7fc9c9bef740 1 -- 172.21.15.16:0/2117722842 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9b401e990 con 0x55a7341261b0 2024-08-31T06:13:21.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.818+0000 7fc9a6000640 1 -- 172.21.15.16:0/2117722842 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fc9ac0ab990 con 0x55a7341261b0 2024-08-31T06:13:21.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.819+0000 7fc9c9bef740 10 client.26972 fetch_fsmap learned FSMap version 38 2024-08-31T06:13:21.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.819+0000 7fc9c9bef740 1 -- 172.21.15.16:0/2117722842 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55a73457d700 con 0x55a7341261b0 2024-08-31T06:13:21.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.820+0000 7fc9a6000640 1 -- 172.21.15.16:0/2117722842 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9b401e990 con 0x55a7341261b0 2024-08-31T06:13:21.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.820+0000 7fc9a6000640 1 -- 172.21.15.16:0/2117722842 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7fc9ac0e3320 con 0x55a7341261b0 2024-08-31T06:13:21.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.821+0000 7fc9c9bef740 10 client.26972 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T06:13:21.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.821+0000 7fc9c9bef740 10 client.26972 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-31T06:13:21.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.821+0000 7fc9c9bef740 20 client.26972 populate_metadata read hostname 'smithi016' 2024-08-31T06:13:21.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.821+0000 7fc9c9bef740 1 --2- 172.21.15.16:0/2117722842 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55a7340e2da0 0x55a73457f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:21.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.821+0000 7fc9c9bef740 4 client.26972 mds_command: new command op to 24724 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:13:21.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.821+0000 7fc9c9bef740 1 -- 172.21.15.16:0/2117722842 --> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a7344691a0 con 0x55a7340e2da0 2024-08-31T06:13:21.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.826+0000 7fc9bb400640 1 --2- 172.21.15.16:0/2117722842 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55a7340e2da0 0x55a73457f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:21.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.828+0000 7fc9bb400640 1 --2- 172.21.15.16:0/2117722842 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55a7340e2da0 0x55a73457f8e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a3f40637c3bae7e8 server_cookie=19329a08ad737d1a in_seq=0 out_seq=0 2024-08-31T06:13:21.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.829+0000 7fc9a6000640 10 client.26972 ms_handle_connect on v2:172.21.15.16:6838/219664460 2024-08-31T06:13:21.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.829+0000 7fc9a6000640 1 -- 172.21.15.16:0/2117722842 <== mds.1 v2:172.21.15.16:6838/219664460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a7344691a0 con 0x55a7340e2da0 2024-08-31T06:13:21.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.829+0000 7fc9a6000640 10 client.26972 handle_command_reply: tid=0 2024-08-31T06:13:21.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.829+0000 7fc9c9bef740 1 client.26972 shutdown 2024-08-31T06:13:21.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.830+0000 7fc9c9bef740 1 -- 172.21.15.16:0/2117722842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a7341261b0 msgr2=0x55a734508350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:21.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.830+0000 7fc9c9bef740 1 --2- 172.21.15.16:0/2117722842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a7341261b0 0x55a734508350 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac0a80c0 tx=0x7fc9ac0ebe60 comp rx=0 tx=0).stop 2024-08-31T06:13:21.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.830+0000 7fc9c9bef740 1 -- 172.21.15.16:0/2117722842 shutdown_connections 2024-08-31T06:13:21.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.830+0000 7fc9c9bef740 1 --2- 172.21.15.16:0/2117722842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a73414f070 0x55a734503300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:21.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.830+0000 7fc9c9bef740 1 --2- 172.21.15.16:0/2117722842 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55a7340e2da0 0x55a73457f8e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:21.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.830+0000 7fc9c9bef740 1 --2- 172.21.15.16:0/2117722842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a73412e390 0x55a734508890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:21.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.830+0000 7fc9c9bef740 1 --2- 172.21.15.16:0/2117722842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a7341261b0 0x55a734508350 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:21.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.830+0000 7fc9c9bef740 1 -- 172.21.15.16:0/2117722842 >> 172.21.15.16:0/2117722842 conn(0x55a734143460 msgr2=0x55a7344feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:21.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.832+0000 7fc9a6000640 0 client.26972 ms_handle_reset on v2:172.21.15.16:6838/219664460 2024-08-31T06:13:21.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.832+0000 7fc9c9bef740 1 -- 172.21.15.16:0/2117722842 shutdown_connections 2024-08-31T06:13:21.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.832+0000 7fc9c9bef740 1 -- 172.21.15.16:0/2117722842 wait complete. 2024-08-31T06:13:21.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.833+0000 7fc9c9bef740 20 client.26972 trim_cache size 0 max 16384 2024-08-31T06:13:21.836 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:13:21.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.835+0000 7fc9c2e00640 1 -- 172.21.15.16:0/2811401089 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc9a00b4d20 msgr2=0x7fc9a00b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:21.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.835+0000 7fc9c2e00640 1 --2- 172.21.15.16:0/2811401089 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc9a00b4d20 0x7fc9a00b7230 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4011670 tx=0x7fc9b4009500 comp rx=0 tx=0).stop 2024-08-31T06:13:21.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.835+0000 7fc9c2e00640 1 -- 172.21.15.16:0/2811401089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9bc10e360 msgr2=0x7fc9bc072b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:21.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.835+0000 7fc9c2e00640 1 --2- 172.21.15.16:0/2811401089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9bc10e360 0x7fc9bc072b70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac00e9f0 tx=0x7fc9ac00eeb0 comp rx=0 tx=0).stop 2024-08-31T06:13:21.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.835+0000 7fc9c2e00640 1 -- 172.21.15.16:0/2811401089 shutdown_connections 2024-08-31T06:13:21.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.835+0000 7fc9c2e00640 1 --2- 172.21.15.16:0/2811401089 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc9a00b4d20 0x7fc9a00b7230 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:21.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.835+0000 7fc9c2e00640 1 --2- 172.21.15.16:0/2811401089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9bc1145a0 0x7fc9bc06f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:21.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.835+0000 7fc9c2e00640 1 --2- 172.21.15.16:0/2811401089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9bc113c50 0x7fc9bc06ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:21.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.835+0000 7fc9c2e00640 1 --2- 172.21.15.16:0/2811401089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9bc10e360 0x7fc9bc072b70 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:21.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.835+0000 7fc9c2e00640 1 -- 172.21.15.16:0/2811401089 >> 172.21.15.16:0/2811401089 conn(0x7fc9bc10a160 msgr2=0x7fc9bc113240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:21.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.836+0000 7fc9c2e00640 1 -- 172.21.15.16:0/2811401089 shutdown_connections 2024-08-31T06:13:21.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:21.836+0000 7fc9c2e00640 1 -- 172.21.15.16:0/2811401089 wait complete. 2024-08-31T06:13:22.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:22 smithi184 ceph-mon[22331]: pgmap v550: 97 pgs: 97 active+clean; 3.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 27 MiB/s wr, 1.47k op/s 2024-08-31T06:13:22.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:22 smithi155 ceph-mon[21742]: pgmap v550: 97 pgs: 97 active+clean; 3.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 27 MiB/s wr, 1.47k op/s 2024-08-31T06:13:22.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:22 smithi016 ceph-mon[17194]: pgmap v550: 97 pgs: 97 active+clean; 3.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 27 MiB/s wr, 1.47k op/s 2024-08-31T06:13:22.811 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T06:13:23.650 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:13:24.509 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:24 smithi016 ceph-mon[17194]: pgmap v551: 97 pgs: 97 active+clean; 3.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 30 MiB/s wr, 1.72k op/s 2024-08-31T06:13:24.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.835+0000 7fc67dc00640 1 -- 172.21.15.16:0/2178450122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc678109d60 msgr2=0x7fc67810a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:24.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.835+0000 7fc67dc00640 1 --2- 172.21.15.16:0/2178450122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc678109d60 0x7fc67810a170 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fc670009860 tx=0x7fc67002a340 comp rx=0 tx=0).stop 2024-08-31T06:13:24.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.835+0000 7fc67dc00640 1 -- 172.21.15.16:0/2178450122 shutdown_connections 2024-08-31T06:13:24.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.835+0000 7fc67dc00640 1 --2- 172.21.15.16:0/2178450122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc67810f6a0 0x7fc67810fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:24.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.835+0000 7fc67dc00640 1 --2- 172.21.15.16:0/2178450122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc67810fff0 0x7fc678116820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:24.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.835+0000 7fc67dc00640 1 --2- 172.21.15.16:0/2178450122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc678109d60 0x7fc67810a170 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:24.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.835+0000 7fc67dc00640 1 -- 172.21.15.16:0/2178450122 >> 172.21.15.16:0/2178450122 conn(0x7fc678079360 msgr2=0x7fc678107f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:24.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.835+0000 7fc67dc00640 1 -- 172.21.15.16:0/2178450122 shutdown_connections 2024-08-31T06:13:24.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.837+0000 7fc67dc00640 1 -- 172.21.15.16:0/2178450122 wait complete. 2024-08-31T06:13:24.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.839+0000 7fc67dc00640 1 Processor -- start 2024-08-31T06:13:24.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.839+0000 7fc67dc00640 1 -- start start 2024-08-31T06:13:24.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.841+0000 7fc67dc00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc678109d60 0x7fc6781a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:24.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.841+0000 7fc67dc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc67810f6a0 0x7fc6781a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:24.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.841+0000 7fc67dc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc67810fff0 0x7fc6781a5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:24.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.841+0000 7fc67dc00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc678119cb0 con 0x7fc67810f6a0 2024-08-31T06:13:24.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.841+0000 7fc67dc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc678119b30 con 0x7fc67810fff0 2024-08-31T06:13:24.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.841+0000 7fc67dc00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc678119e30 con 0x7fc678109d60 2024-08-31T06:13:24.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.842+0000 7fc677400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc67810f6a0 0x7fc6781a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:24.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.842+0000 7fc677400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc67810f6a0 0x7fc6781a0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60946/0 (socket says 172.21.15.16:60946) 2024-08-31T06:13:24.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.842+0000 7fc677400640 1 -- 172.21.15.16:0/1639745980 learned_addr learned my addr 172.21.15.16:0/1639745980 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:24.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.842+0000 7fc677e00640 1 --2- 172.21.15.16:0/1639745980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc678109d60 0x7fc6781a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:24.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.842+0000 7fc677400640 1 -- 172.21.15.16:0/1639745980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc678109d60 msgr2=0x7fc6781a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:24.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.842+0000 7fc677400640 1 --2- 172.21.15.16:0/1639745980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc678109d60 0x7fc6781a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:24.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.843+0000 7fc677400640 1 -- 172.21.15.16:0/1639745980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc67810fff0 msgr2=0x7fc6781a5ee0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-31T06:13:24.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.843+0000 7fc677400640 1 --2- 172.21.15.16:0/1639745980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc67810fff0 0x7fc6781a5ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:24.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.843+0000 7fc677400640 1 -- 172.21.15.16:0/1639745980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6781a6610 con 0x7fc67810f6a0 2024-08-31T06:13:24.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.843+0000 7fc677e00640 1 --2- 172.21.15.16:0/1639745980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc678109d60 0x7fc6781a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:13:24.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.843+0000 7fc677400640 1 --2- 172.21.15.16:0/1639745980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc67810f6a0 0x7fc6781a0c40 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc66800ea50 tx=0x7fc66800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:24.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.844+0000 7fc674c00640 1 -- 172.21.15.16:0/1639745980 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66800cdc0 con 0x7fc67810f6a0 2024-08-31T06:13:24.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.844+0000 7fc674c00640 1 -- 172.21.15.16:0/1639745980 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc668011920 con 0x7fc67810f6a0 2024-08-31T06:13:24.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.844+0000 7fc674c00640 1 -- 172.21.15.16:0/1639745980 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66801aa60 con 0x7fc67810f6a0 2024-08-31T06:13:24.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.844+0000 7fc67dc00640 1 -- 172.21.15.16:0/1639745980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6781a6900 con 0x7fc67810f6a0 2024-08-31T06:13:24.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.844+0000 7fc67dc00640 1 -- 172.21.15.16:0/1639745980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6781ad060 con 0x7fc67810f6a0 2024-08-31T06:13:24.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.850+0000 7fc685ed8740 1 Processor -- start 2024-08-31T06:13:24.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.851+0000 7fc674c00640 1 -- 172.21.15.16:0/1639745980 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fc66801ac00 con 0x7fc67810f6a0 2024-08-31T06:13:24.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.851+0000 7fc685ed8740 1 -- start start 2024-08-31T06:13:24.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.851+0000 7fc685ed8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd66979070 0x55fd66d1eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:24.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.851+0000 7fc685ed8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd669501b0 0x55fd66d1f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:24.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.852+0000 7fc685ed8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd66958390 0x55fd66d1f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:24.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.852+0000 7fc685ed8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fd66c957b0 con 0x55fd66979070 2024-08-31T06:13:24.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.852+0000 7fc685ed8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55fd66c84d40 con 0x55fd669501b0 2024-08-31T06:13:24.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.852+0000 7fc685ed8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55fd66c88cf0 con 0x55fd66958390 2024-08-31T06:13:24.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.852+0000 7fc677e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd669501b0 0x55fd66d1f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:24.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.852+0000 7fc677400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd66958390 0x55fd66d1f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:24.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.853+0000 7fc677400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd66958390 0x55fd66d1f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:42996/0 (socket says 172.21.15.16:42996) 2024-08-31T06:13:24.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.853+0000 7fc677400640 1 -- 172.21.15.16:0/64926309 learned_addr learned my addr 172.21.15.16:0/64926309 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:24.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.854+0000 7fc67d200640 1 --2- 172.21.15.16:0/64926309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd66979070 0x55fd66d1eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:24.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.854+0000 7fc677400640 1 -- 172.21.15.16:0/64926309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd669501b0 msgr2=0x55fd66d1f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:24.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.854+0000 7fc677400640 1 --2- 172.21.15.16:0/64926309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd669501b0 0x55fd66d1f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:24.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.854+0000 7fc677400640 1 -- 172.21.15.16:0/64926309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd66979070 msgr2=0x55fd66d1eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:24.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.855+0000 7fc677400640 1 --2- 172.21.15.16:0/64926309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd66979070 0x55fd66d1eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:24.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.855+0000 7fc677400640 1 -- 172.21.15.16:0/64926309 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd669e4650 con 0x55fd66958390 2024-08-31T06:13:24.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.855+0000 7fc677e00640 1 --2- 172.21.15.16:0/64926309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd669501b0 0x55fd66d1f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:13:24.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.855+0000 7fc67d200640 1 --2- 172.21.15.16:0/64926309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd66979070 0x55fd66d1eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:13:24.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.856+0000 7fc677400640 1 --2- 172.21.15.16:0/64926309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd66958390 0x55fd66d1f5e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc6680e3080 tx=0x7fc6680ed670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b947431b060f376 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:24.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.856+0000 7fc66d600640 1 -- 172.21.15.16:0/64926309 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6680f4070 con 0x55fd66958390 2024-08-31T06:13:24.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.857+0000 7fc66d600640 1 -- 172.21.15.16:0/64926309 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6680f0ad0 con 0x55fd66958390 2024-08-31T06:13:24.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.858+0000 7fc66d600640 1 -- 172.21.15.16:0/64926309 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6680ef940 con 0x55fd66958390 2024-08-31T06:13:24.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.861+0000 7fc685ed8740 1 -- 172.21.15.16:0/64926309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd66958390 msgr2=0x55fd66d1f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:24.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.861+0000 7fc685ed8740 1 --2- 172.21.15.16:0/64926309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd66958390 0x55fd66d1f5e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc6680e3080 tx=0x7fc6680ed670 comp rx=0 tx=0).stop 2024-08-31T06:13:24.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.862+0000 7fc685ed8740 1 -- 172.21.15.16:0/64926309 shutdown_connections 2024-08-31T06:13:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.862+0000 7fc685ed8740 1 --2- 172.21.15.16:0/64926309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd66958390 0x55fd66d1f5e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.862+0000 7fc685ed8740 1 --2- 172.21.15.16:0/64926309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd669501b0 0x55fd66d1f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.862+0000 7fc685ed8740 1 --2- 172.21.15.16:0/64926309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd66979070 0x55fd66d1eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.862+0000 7fc685ed8740 1 -- 172.21.15.16:0/64926309 >> 172.21.15.16:0/64926309 conn(0x55fd6696d460 msgr2=0x55fd66d1ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.862+0000 7fc685ed8740 1 -- 172.21.15.16:0/64926309 shutdown_connections 2024-08-31T06:13:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.863+0000 7fc685ed8740 1 -- 172.21.15.16:0/64926309 wait complete. 2024-08-31T06:13:24.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.864+0000 7fc685ed8740 1 -- start start 2024-08-31T06:13:24.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.864+0000 7fc674c00640 1 --2- 172.21.15.16:0/1639745980 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc65c0b4d10 0x7fc65c0b7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:24.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.864+0000 7fc685ed8740 1 Processor -- start 2024-08-31T06:13:24.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.865+0000 7fc674c00640 1 -- 172.21.15.16:0/1639745980 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8470+0+0 (secure 0 0 0) 0x7fc6680e5a20 con 0x7fc67810f6a0 2024-08-31T06:13:24.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.867+0000 7fc685ed8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd669501b0 0x55fd66d2a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:24.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.867+0000 7fc685ed8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd66958390 0x55fd66d2acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:24.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.868+0000 7fc67d200640 1 --2- 172.21.15.16:0/1639745980 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc65c0b4d10 0x7fc65c0b7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:24.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.868+0000 7fc685ed8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd66979070 0x55fd66d36630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:24.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.868+0000 7fc685ed8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fd66ae7700 con 0x55fd669501b0 2024-08-31T06:13:24.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.868+0000 7fc685ed8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55fd66c0bcf0 con 0x55fd66979070 2024-08-31T06:13:24.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.868+0000 7fc685ed8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55fd66d2d340 con 0x55fd66958390 2024-08-31T06:13:24.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.868+0000 7fc685ed8740 10 client.0 _ll_register_callbacks cb 0x55fd66c31470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:13:24.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.868+0000 7fc67d200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd66979070 0x55fd66d36630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:24.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.868+0000 7fc67d200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd66979070 0x55fd66d36630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:57572/0 (socket says 172.21.15.16:57572) 2024-08-31T06:13:24.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.868+0000 7fc67d200640 1 -- 172.21.15.16:0/1879636537 learned_addr learned my addr 172.21.15.16:0/1879636537 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:24.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.868+0000 7fc677e00640 1 --2- 172.21.15.16:0/1879636537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd669501b0 0x55fd66d2a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:24.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.869+0000 7fc677400640 1 --2- 172.21.15.16:0/1879636537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd66958390 0x55fd66d2acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:24.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.869+0000 7fc67d200640 1 -- 172.21.15.16:0/1879636537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd66958390 msgr2=0x55fd66d2acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:24.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.869+0000 7fc67d200640 1 --2- 172.21.15.16:0/1879636537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd66958390 0x55fd66d2acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:24.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.869+0000 7fc67d200640 1 -- 172.21.15.16:0/1879636537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd669501b0 msgr2=0x55fd66d2a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:24.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.869+0000 7fc67d200640 1 --2- 172.21.15.16:0/1879636537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd669501b0 0x55fd66d2a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:24.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.869+0000 7fc67d200640 1 -- 172.21.15.16:0/1879636537 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd66c4c8b0 con 0x55fd66979070 2024-08-31T06:13:24.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.870+0000 7fc677400640 1 --2- 172.21.15.16:0/1879636537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd66958390 0x55fd66d2acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:13:24.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:24 smithi155 ceph-mon[21742]: pgmap v551: 97 pgs: 97 active+clean; 3.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 30 MiB/s wr, 1.72k op/s 2024-08-31T06:13:24.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.870+0000 7fc67d200640 1 --2- 172.21.15.16:0/1879636537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd66979070 0x55fd66d36630 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc664015f50 tx=0x7fc664012820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a00f10f2fa09aef7 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:24.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.870+0000 7fc67d200640 1 --2- 172.21.15.16:0/1639745980 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc65c0b4d10 0x7fc65c0b7220 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc6640058c0 tx=0x7fc66400d000 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:24.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.870+0000 7fc662000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:13:24.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.871+0000 7fc662000640 1 -- 172.21.15.16:0/1879636537 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc664009a60 con 0x55fd66979070 2024-08-31T06:13:24.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.871+0000 7fc662000640 1 -- 172.21.15.16:0/1879636537 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8470+0+0 (secure 0 0 0) 0x7fc664015050 con 0x55fd66979070 2024-08-31T06:13:24.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.871+0000 7fc685ed8740 1 -- 172.21.15.16:0/1879636537 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6680f0ad0 con 0x55fd66979070 2024-08-31T06:13:24.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.873+0000 7fc662000640 1 -- 172.21.15.16:0/1879636537 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fc664009d70 con 0x55fd66979070 2024-08-31T06:13:24.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.873+0000 7fc685ed8740 10 client.26978 fetch_fsmap learned FSMap version 38 2024-08-31T06:13:24.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.873+0000 7fc685ed8740 1 -- 172.21.15.16:0/1879636537 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55fd66d39290 con 0x55fd66979070 2024-08-31T06:13:24.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.873+0000 7fc662000640 1 -- 172.21.15.16:0/1879636537 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6680f0ad0 con 0x55fd66979070 2024-08-31T06:13:24.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.873+0000 7fc662000640 1 -- 172.21.15.16:0/1879636537 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7fc66401e070 con 0x55fd66979070 2024-08-31T06:13:24.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.874+0000 7fc685ed8740 10 client.26978 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T06:13:24.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.874+0000 7fc685ed8740 10 client.26978 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:13:24.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.874+0000 7fc685ed8740 20 client.26978 populate_metadata read hostname 'smithi016' 2024-08-31T06:13:24.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.874+0000 7fc685ed8740 1 --2- 172.21.15.16:0/1879636537 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fd669379f0 0x55fd66d3c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:24.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.874+0000 7fc685ed8740 4 client.26978 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:13:24.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.874+0000 7fc685ed8740 1 -- 172.21.15.16:0/1879636537 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd66c97700 con 0x55fd669379f0 2024-08-31T06:13:24.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.875+0000 7fc677e00640 1 --2- 172.21.15.16:0/1879636537 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fd669379f0 0x55fd66d3c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:24.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.876+0000 7fc662000640 10 client.26978 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:13:24.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.879+0000 7fc677e00640 1 --2- 172.21.15.16:0/1879636537 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fd669379f0 0x55fd66d3c850 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afe8512b3ab3390e server_cookie=a6aec9159f6ed8e1 in_seq=0 out_seq=0 2024-08-31T06:13:24.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.880+0000 7fc662000640 1 -- 172.21.15.16:0/1879636537 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fd66c97700 con 0x55fd669379f0 2024-08-31T06:13:24.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.880+0000 7fc662000640 10 client.26978 handle_command_reply: tid=0 2024-08-31T06:13:24.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.881+0000 7fc685ed8740 1 client.26978 shutdown 2024-08-31T06:13:24.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.881+0000 7fc685ed8740 1 -- 172.21.15.16:0/1879636537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd66979070 msgr2=0x55fd66d36630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:24.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.881+0000 7fc685ed8740 1 --2- 172.21.15.16:0/1879636537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd66979070 0x55fd66d36630 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc664015f50 tx=0x7fc664012820 comp rx=0 tx=0).stop 2024-08-31T06:13:24.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.881+0000 7fc685ed8740 1 -- 172.21.15.16:0/1879636537 shutdown_connections 2024-08-31T06:13:24.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.882+0000 7fc685ed8740 1 --2- 172.21.15.16:0/1879636537 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fd669379f0 0x55fd66d3c850 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:24.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.882+0000 7fc685ed8740 1 --2- 172.21.15.16:0/1879636537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd66958390 0x55fd66d2acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:24.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.882+0000 7fc685ed8740 1 --2- 172.21.15.16:0/1879636537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd66979070 0x55fd66d36630 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:24.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.882+0000 7fc685ed8740 1 --2- 172.21.15.16:0/1879636537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd669501b0 0x55fd66d2a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:24.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.882+0000 7fc685ed8740 1 -- 172.21.15.16:0/1879636537 >> 172.21.15.16:0/1879636537 conn(0x55fd6696d460 msgr2=0x55fd66d21d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:24.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.882+0000 7fc662000640 0 client.26978 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:13:24.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.884+0000 7fc685ed8740 1 -- 172.21.15.16:0/1879636537 shutdown_connections 2024-08-31T06:13:24.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.884+0000 7fc685ed8740 1 -- 172.21.15.16:0/1879636537 wait complete. 2024-08-31T06:13:24.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.884+0000 7fc685ed8740 20 client.26978 trim_cache size 0 max 16384 2024-08-31T06:13:24.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.909+0000 7fc685ed8740 1 Processor -- start 2024-08-31T06:13:24.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.910+0000 7fc685ed8740 1 -- start start 2024-08-31T06:13:24.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.910+0000 7fc685ed8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd669501b0 0x55fd66d25d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:24.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.911+0000 7fc685ed8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd66958390 0x55fd66d202a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:24.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.911+0000 7fc677400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd66958390 0x55fd66d202a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:24.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.911+0000 7fc677400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd66958390 0x55fd66d202a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:43020/0 (socket says 172.21.15.16:43020) 2024-08-31T06:13:24.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.911+0000 7fc677e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd669501b0 0x55fd66d25d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:24.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.911+0000 7fc677400640 1 -- 172.21.15.16:0/3373625918 learned_addr learned my addr 172.21.15.16:0/3373625918 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:24.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.912+0000 7fc685ed8740 1 --2- 172.21.15.16:0/3373625918 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd66979070 0x55fd66d1c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:24.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.912+0000 7fc67d200640 1 --2- 172.21.15.16:0/3373625918 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd66979070 0x55fd66d1c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:24.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.912+0000 7fc685ed8740 1 -- 172.21.15.16:0/3373625918 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc6781b8360 con 0x55fd669501b0 2024-08-31T06:13:24.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.912+0000 7fc685ed8740 1 -- 172.21.15.16:0/3373625918 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc6781b5ba0 con 0x55fd66979070 2024-08-31T06:13:24.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.912+0000 7fc685ed8740 1 -- 172.21.15.16:0/3373625918 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc6781b3a10 con 0x55fd66958390 2024-08-31T06:13:24.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.913+0000 7fc677e00640 1 -- 172.21.15.16:0/3373625918 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd66958390 msgr2=0x55fd66d202a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:24.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.913+0000 7fc677e00640 1 --2- 172.21.15.16:0/3373625918 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd66958390 0x55fd66d202a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:24.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.913+0000 7fc677e00640 1 -- 172.21.15.16:0/3373625918 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd66979070 msgr2=0x55fd66d1c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:24.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.913+0000 7fc677e00640 1 --2- 172.21.15.16:0/3373625918 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd66979070 0x55fd66d1c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:24.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.913+0000 7fc677e00640 1 -- 172.21.15.16:0/3373625918 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd66d20e50 con 0x55fd669501b0 2024-08-31T06:13:24.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.914+0000 7fc677e00640 1 --2- 172.21.15.16:0/3373625918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd669501b0 0x55fd66d25d30 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc670035040 tx=0x7fc67002aac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=343fb007317d7190 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:24.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.916+0000 7fc66cc00640 1 -- 172.21.15.16:0/3373625918 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc670049030 con 0x55fd669501b0 2024-08-31T06:13:24.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.916+0000 7fc66cc00640 1 -- 172.21.15.16:0/3373625918 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc670030e60 con 0x55fd669501b0 2024-08-31T06:13:24.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.916+0000 7fc66cc00640 1 -- 172.21.15.16:0/3373625918 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc670049030 con 0x55fd669501b0 2024-08-31T06:13:24.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.917+0000 7fc685ed8740 1 -- 172.21.15.16:0/3373625918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd669501b0 msgr2=0x55fd66d25d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:24.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.917+0000 7fc685ed8740 1 --2- 172.21.15.16:0/3373625918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd669501b0 0x55fd66d25d30 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc670035040 tx=0x7fc67002aac0 comp rx=0 tx=0).stop 2024-08-31T06:13:24.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.917+0000 7fc685ed8740 1 -- 172.21.15.16:0/3373625918 shutdown_connections 2024-08-31T06:13:24.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.917+0000 7fc685ed8740 1 --2- 172.21.15.16:0/3373625918 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd66958390 0x55fd66d202a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:24.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.917+0000 7fc685ed8740 1 --2- 172.21.15.16:0/3373625918 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd66979070 0x55fd66d1c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:24.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.917+0000 7fc685ed8740 1 --2- 172.21.15.16:0/3373625918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd669501b0 0x55fd66d25d30 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:24.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.917+0000 7fc685ed8740 1 -- 172.21.15.16:0/3373625918 >> 172.21.15.16:0/3373625918 conn(0x55fd6696d460 msgr2=0x55fd66d1fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:24.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.918+0000 7fc685ed8740 1 -- 172.21.15.16:0/3373625918 shutdown_connections 2024-08-31T06:13:24.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.918+0000 7fc685ed8740 1 -- 172.21.15.16:0/3373625918 wait complete. 2024-08-31T06:13:24.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.918+0000 7fc685ed8740 1 -- start start 2024-08-31T06:13:24.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.919+0000 7fc685ed8740 1 Processor -- start 2024-08-31T06:13:24.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.920+0000 7fc685ed8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd669501b0 0x55fd66d2e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:24.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.920+0000 7fc685ed8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd66958390 0x55fd66d2edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:24.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.920+0000 7fc685ed8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd66979070 0x55fd66da6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:24.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.920+0000 7fc685ed8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc6781b4060 con 0x55fd66958390 2024-08-31T06:13:24.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.921+0000 7fc67d200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd66979070 0x55fd66da6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:24.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.921+0000 7fc677e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd669501b0 0x55fd66d2e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:24.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.921+0000 7fc67d200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd66979070 0x55fd66da6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:57588/0 (socket says 172.21.15.16:57588) 2024-08-31T06:13:24.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.921+0000 7fc67d200640 1 -- 172.21.15.16:0/1435895361 learned_addr learned my addr 172.21.15.16:0/1435895361 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:24.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.921+0000 7fc677400640 1 --2- 172.21.15.16:0/1435895361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd66958390 0x55fd66d2edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:24.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.920+0000 7fc685ed8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc6781b54d0 con 0x55fd66979070 2024-08-31T06:13:24.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.921+0000 7fc685ed8740 1 -- 172.21.15.16:0/1435895361 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc6781b4730 con 0x55fd669501b0 2024-08-31T06:13:24.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.921+0000 7fc685ed8740 10 client.0 _ll_register_callbacks cb 0x7fc664015b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:13:24.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.922+0000 7fc67d200640 1 -- 172.21.15.16:0/1435895361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd669501b0 msgr2=0x55fd66d2e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:24.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.922+0000 7fc67d200640 1 --2- 172.21.15.16:0/1435895361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd669501b0 0x55fd66d2e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:24.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.922+0000 7fc67d200640 1 -- 172.21.15.16:0/1435895361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd66958390 msgr2=0x55fd66d2edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:24.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.923+0000 7fc67d200640 1 --2- 172.21.15.16:0/1435895361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd66958390 0x55fd66d2edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:24.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.923+0000 7fc67d200640 1 -- 172.21.15.16:0/1435895361 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd66d2f6c0 con 0x55fd66979070 2024-08-31T06:13:24.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.923+0000 7fc677400640 1 --2- 172.21.15.16:0/1435895361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd66958390 0x55fd66d2edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:13:24.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.923+0000 7fc67d200640 1 --2- 172.21.15.16:0/1435895361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd66979070 0x55fd66da6210 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc664005890 tx=0x7fc664005df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc15743a98b13c4c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:24.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.923+0000 7fc662000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:13:24.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.924+0000 7fc662000640 1 -- 172.21.15.16:0/1435895361 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc664005640 con 0x55fd66979070 2024-08-31T06:13:24.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.924+0000 7fc662000640 1 -- 172.21.15.16:0/1435895361 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8470+0+0 (secure 0 0 0) 0x7fc664003800 con 0x55fd66979070 2024-08-31T06:13:24.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.924+0000 7fc685ed8740 1 -- 172.21.15.16:0/1435895361 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc670030e60 con 0x55fd66979070 2024-08-31T06:13:24.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.924+0000 7fc662000640 1 -- 172.21.15.16:0/1435895361 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fc664042050 con 0x55fd66979070 2024-08-31T06:13:24.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.924+0000 7fc685ed8740 10 client.26984 fetch_fsmap learned FSMap version 38 2024-08-31T06:13:24.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.924+0000 7fc685ed8740 1 -- 172.21.15.16:0/1435895361 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55fd66da77d0 con 0x55fd66979070 2024-08-31T06:13:24.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.925+0000 7fc662000640 1 -- 172.21.15.16:0/1435895361 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc670030e60 con 0x55fd66979070 2024-08-31T06:13:24.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.925+0000 7fc662000640 1 -- 172.21.15.16:0/1435895361 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7fc66401e070 con 0x55fd66979070 2024-08-31T06:13:24.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.927+0000 7fc685ed8740 10 client.26984 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T06:13:24.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.927+0000 7fc685ed8740 10 client.26984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:13:24.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.927+0000 7fc685ed8740 20 client.26984 populate_metadata read hostname 'smithi016' 2024-08-31T06:13:24.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.927+0000 7fc685ed8740 1 --2- 172.21.15.16:0/1435895361 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fd6690cda0 0x55fd66da9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:24.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.927+0000 7fc685ed8740 4 client.26984 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:13:24.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.927+0000 7fc685ed8740 1 -- 172.21.15.16:0/1435895361 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fd66c931a0 con 0x55fd6690cda0 2024-08-31T06:13:24.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.928+0000 7fc677e00640 1 --2- 172.21.15.16:0/1435895361 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fd6690cda0 0x55fd66da9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:24.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.929+0000 7fc677e00640 1 --2- 172.21.15.16:0/1435895361 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fd6690cda0 0x55fd66da9f90 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a74c7e6b00d004ba server_cookie=48725e67f75fd2f9 in_seq=0 out_seq=0 2024-08-31T06:13:24.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.929+0000 7fc662000640 10 client.26984 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:13:24.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.930+0000 7fc662000640 1 -- 172.21.15.16:0/1435895361 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fd66c931a0 con 0x55fd6690cda0 2024-08-31T06:13:24.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.930+0000 7fc662000640 10 client.26984 handle_command_reply: tid=0 2024-08-31T06:13:24.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.930+0000 7fc685ed8740 1 client.26984 shutdown 2024-08-31T06:13:24.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.931+0000 7fc685ed8740 1 -- 172.21.15.16:0/1435895361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd66979070 msgr2=0x55fd66da6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:24.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.931+0000 7fc685ed8740 1 --2- 172.21.15.16:0/1435895361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd66979070 0x55fd66da6210 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc664005890 tx=0x7fc664005df0 comp rx=0 tx=0).stop 2024-08-31T06:13:24.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.931+0000 7fc685ed8740 1 -- 172.21.15.16:0/1435895361 shutdown_connections 2024-08-31T06:13:24.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.931+0000 7fc685ed8740 1 --2- 172.21.15.16:0/1435895361 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fd6690cda0 0x55fd66da9f90 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:24.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.931+0000 7fc685ed8740 1 --2- 172.21.15.16:0/1435895361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd66979070 0x55fd66da6210 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:24.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.931+0000 7fc685ed8740 1 --2- 172.21.15.16:0/1435895361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd66958390 0x55fd66d2edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:24.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.931+0000 7fc685ed8740 1 --2- 172.21.15.16:0/1435895361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd669501b0 0x55fd66d2e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:24.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.931+0000 7fc685ed8740 1 -- 172.21.15.16:0/1435895361 >> 172.21.15.16:0/1435895361 conn(0x55fd6696d460 msgr2=0x55fd66d1fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:24.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.931+0000 7fc662000640 0 client.26984 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:13:24.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.932+0000 7fc685ed8740 1 -- 172.21.15.16:0/1435895361 shutdown_connections 2024-08-31T06:13:24.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.932+0000 7fc685ed8740 1 -- 172.21.15.16:0/1435895361 wait complete. 2024-08-31T06:13:24.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.932+0000 7fc685ed8740 20 client.26984 trim_cache size 0 max 16384 2024-08-31T06:13:24.935 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:13:24.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.933+0000 7fc67dc00640 1 -- 172.21.15.16:0/1639745980 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc65c0b4d10 msgr2=0x7fc65c0b7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:24.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.933+0000 7fc67dc00640 1 --2- 172.21.15.16:0/1639745980 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc65c0b4d10 0x7fc65c0b7220 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc6640058c0 tx=0x7fc66400d000 comp rx=0 tx=0).stop 2024-08-31T06:13:24.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.933+0000 7fc67dc00640 1 -- 172.21.15.16:0/1639745980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc67810f6a0 msgr2=0x7fc6781a0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:24.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.933+0000 7fc67dc00640 1 --2- 172.21.15.16:0/1639745980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc67810f6a0 0x7fc6781a0c40 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc66800ea50 tx=0x7fc66800ef10 comp rx=0 tx=0).stop 2024-08-31T06:13:24.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.934+0000 7fc67dc00640 1 -- 172.21.15.16:0/1639745980 shutdown_connections 2024-08-31T06:13:24.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.934+0000 7fc67dc00640 1 --2- 172.21.15.16:0/1639745980 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc65c0b4d10 0x7fc65c0b7220 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:24.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.934+0000 7fc67dc00640 1 --2- 172.21.15.16:0/1639745980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc67810fff0 0x7fc6781a5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.934+0000 7fc67dc00640 1 --2- 172.21.15.16:0/1639745980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc67810f6a0 0x7fc6781a0c40 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.934+0000 7fc67dc00640 1 --2- 172.21.15.16:0/1639745980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc678109d60 0x7fc6781a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.934+0000 7fc67dc00640 1 -- 172.21.15.16:0/1639745980 >> 172.21.15.16:0/1639745980 conn(0x7fc678079360 msgr2=0x7fc678079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.934+0000 7fc67dc00640 1 -- 172.21.15.16:0/1639745980 shutdown_connections 2024-08-31T06:13:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:24.934+0000 7fc67dc00640 1 -- 172.21.15.16:0/1639745980 wait complete. 2024-08-31T06:13:24.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:24 smithi184 ceph-mon[22331]: pgmap v551: 97 pgs: 97 active+clean; 3.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 30 MiB/s wr, 1.72k op/s 2024-08-31T06:13:26.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 360.0 s, 1974.4 tps, lat 50.932 ms stddev 72.440 2024-08-31T06:13:26.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:26 smithi155 ceph-mon[21742]: pgmap v552: 97 pgs: 97 active+clean; 3.4 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 31 MiB/s wr, 2.13k op/s 2024-08-31T06:13:26.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:26 smithi016 ceph-mon[17194]: pgmap v552: 97 pgs: 97 active+clean; 3.4 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 31 MiB/s wr, 2.13k op/s 2024-08-31T06:13:26.945 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T06:13:26.945 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T06:13:26.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:26 smithi184 ceph-mon[22331]: pgmap v552: 97 pgs: 97 active+clean; 3.4 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 31 MiB/s wr, 2.13k op/s 2024-08-31T06:13:27.660 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:13:28.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.564+0000 7f8643a00640 1 -- 172.21.15.16:0/1613839020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f863c102cc0 msgr2=0x7f863c10f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:28.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.564+0000 7f8643a00640 1 --2- 172.21.15.16:0/1613839020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f863c102cc0 0x7f863c10f9e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8628009900 tx=0x7f862802a490 comp rx=0 tx=0).stop 2024-08-31T06:13:28.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.565+0000 7f8643a00640 1 -- 172.21.15.16:0/1613839020 shutdown_connections 2024-08-31T06:13:28.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.565+0000 7f8643a00640 1 --2- 172.21.15.16:0/1613839020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f863c10ff20 0x7f863c112340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:28.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.565+0000 7f8643a00640 1 --2- 172.21.15.16:0/1613839020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f863c102cc0 0x7f863c10f9e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:28.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.565+0000 7f8643a00640 1 --2- 172.21.15.16:0/1613839020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f863c102370 0x7f863c102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:28.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.565+0000 7f8643a00640 1 -- 172.21.15.16:0/1613839020 >> 172.21.15.16:0/1613839020 conn(0x7f863c0794d0 msgr2=0x7f863c0798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:28.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.565+0000 7f8643a00640 1 -- 172.21.15.16:0/1613839020 shutdown_connections 2024-08-31T06:13:28.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.565+0000 7f8643a00640 1 -- 172.21.15.16:0/1613839020 wait complete. 2024-08-31T06:13:28.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.567+0000 7f8643a00640 1 Processor -- start 2024-08-31T06:13:28.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.568+0000 7f8643a00640 1 -- start start 2024-08-31T06:13:28.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.569+0000 7f8643a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f863c102370 0x7f863c074c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:28.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.569+0000 7f8643a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f863c102cc0 0x7f863c06def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:28.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.570+0000 7f8643a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f863c10ff20 0x7f863c06e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:28.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.570+0000 7f8641000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f863c102370 0x7f863c074c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:28.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.570+0000 7f8641000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f863c102370 0x7f863c074c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57928/0 (socket says 172.21.15.16:57928) 2024-08-31T06:13:28.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.570+0000 7f8641000640 1 -- 172.21.15.16:0/3982768681 learned_addr learned my addr 172.21.15.16:0/3982768681 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:28.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.570+0000 7f863be00640 1 --2- 172.21.15.16:0/3982768681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f863c102cc0 0x7f863c06def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:28.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.570+0000 7f8641a00640 1 --2- 172.21.15.16:0/3982768681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f863c10ff20 0x7f863c06e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:28.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.570+0000 7f8643a00640 1 -- 172.21.15.16:0/3982768681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f863c115a10 con 0x7f863c102370 2024-08-31T06:13:28.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.570+0000 7f8643a00640 1 -- 172.21.15.16:0/3982768681 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f863c115890 con 0x7f863c10ff20 2024-08-31T06:13:28.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.570+0000 7f8643a00640 1 -- 172.21.15.16:0/3982768681 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f863c115b90 con 0x7f863c102cc0 2024-08-31T06:13:28.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.571+0000 7f8641000640 1 -- 172.21.15.16:0/3982768681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f863c102cc0 msgr2=0x7f863c06def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:28.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.571+0000 7f8641000640 1 --2- 172.21.15.16:0/3982768681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f863c102cc0 0x7f863c06def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:28.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.571+0000 7f8641000640 1 -- 172.21.15.16:0/3982768681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f863c10ff20 msgr2=0x7f863c06e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:28.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.571+0000 7f8641000640 1 --2- 172.21.15.16:0/3982768681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f863c10ff20 0x7f863c06e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:28.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.571+0000 7f8641000640 1 -- 172.21.15.16:0/3982768681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f863c06ec20 con 0x7f863c102370 2024-08-31T06:13:28.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.571+0000 7f8641a00640 1 --2- 172.21.15.16:0/3982768681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f863c10ff20 0x7f863c06e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:13:28.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.571+0000 7f8641000640 1 --2- 172.21.15.16:0/3982768681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f863c102370 0x7f863c074c20 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f863000c920 tx=0x7f863000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:28.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.573+0000 7f8639600640 1 -- 172.21.15.16:0/3982768681 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8630007c70 con 0x7f863c102370 2024-08-31T06:13:28.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.573+0000 7f8639600640 1 -- 172.21.15.16:0/3982768681 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8630007e10 con 0x7f863c102370 2024-08-31T06:13:28.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.573+0000 7f8639600640 1 -- 172.21.15.16:0/3982768681 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8630018820 con 0x7f863c102370 2024-08-31T06:13:28.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.573+0000 7f8643a00640 1 -- 172.21.15.16:0/3982768681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f863c0738f0 con 0x7f863c102370 2024-08-31T06:13:28.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.573+0000 7f8643a00640 1 -- 172.21.15.16:0/3982768681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f863c073e00 con 0x7f863c102370 2024-08-31T06:13:28.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.580+0000 7f864a681740 1 Processor -- start 2024-08-31T06:13:28.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.581+0000 7f864a681740 1 -- start start 2024-08-31T06:13:28.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.582+0000 7f864a681740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f27a545350 0x55f27a8eb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:28.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.582+0000 7f864a681740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f27a51c490 0x55f27a8ebb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:28.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.582+0000 7f864a681740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f27a524670 0x55f27a8ec080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:28.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.582+0000 7f864a681740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f27a861750 con 0x55f27a524670 2024-08-31T06:13:28.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.582+0000 7f864a681740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f27a850ce0 con 0x55f27a545350 2024-08-31T06:13:28.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.582+0000 7f864a681740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f27a854c90 con 0x55f27a51c490 2024-08-31T06:13:28.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.582+0000 7f8639600640 1 -- 172.21.15.16:0/3982768681 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f863000f450 con 0x7f863c102370 2024-08-31T06:13:28.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.583+0000 7f8641a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f27a545350 0x55f27a8eb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:28.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.583+0000 7f8641a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f27a545350 0x55f27a8eb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:54928/0 (socket says 172.21.15.16:54928) 2024-08-31T06:13:28.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.583+0000 7f8641a00640 1 -- 172.21.15.16:0/4172269084 learned_addr learned my addr 172.21.15.16:0/4172269084 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:28.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.583+0000 7f8641000640 1 --2- 172.21.15.16:0/4172269084 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f27a51c490 0x55f27a8ebb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:28.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.583+0000 7f863be00640 1 --2- 172.21.15.16:0/4172269084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f27a524670 0x55f27a8ec080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:28.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.583+0000 7f8641000640 1 -- 172.21.15.16:0/4172269084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f27a545350 msgr2=0x55f27a8eb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:28.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.584+0000 7f8641000640 1 --2- 172.21.15.16:0/4172269084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f27a545350 0x55f27a8eb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:28.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.584+0000 7f8641000640 1 -- 172.21.15.16:0/4172269084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f27a524670 msgr2=0x55f27a8ec080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:28.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.584+0000 7f8641000640 1 --2- 172.21.15.16:0/4172269084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f27a524670 0x55f27a8ec080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:28.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.584+0000 7f8641000640 1 -- 172.21.15.16:0/4172269084 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f27a5b0930 con 0x55f27a51c490 2024-08-31T06:13:28.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.585+0000 7f8641a00640 1 --2- 172.21.15.16:0/4172269084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f27a545350 0x55f27a8eb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:13:28.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.586+0000 7f8639600640 1 --2- 172.21.15.16:0/3982768681 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f86240b4d20 0x7f86240b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:28.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.586+0000 7f863be00640 1 --2- 172.21.15.16:0/4172269084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f27a524670 0x55f27a8ec080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:13:28.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.586+0000 7f8639600640 1 -- 172.21.15.16:0/3982768681 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8470+0+0 (secure 0 0 0) 0x7f86300e5650 con 0x7f863c102370 2024-08-31T06:13:28.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.587+0000 7f8641000640 1 --2- 172.21.15.16:0/4172269084 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f27a51c490 0x55f27a8ebb40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f86300e2d70 tx=0x7f86300ebcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51c0c66fc74606d5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:28.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.588+0000 7f8641a00640 1 --2- 172.21.15.16:0/3982768681 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f86240b4d20 0x7f86240b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:28.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.588+0000 7f862e000640 1 -- 172.21.15.16:0/4172269084 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86300a7bd0 con 0x55f27a51c490 2024-08-31T06:13:28.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.588+0000 7f862e000640 1 -- 172.21.15.16:0/4172269084 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86300ab820 con 0x55f27a51c490 2024-08-31T06:13:28.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.588+0000 7f862e000640 1 -- 172.21.15.16:0/4172269084 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86300ab480 con 0x55f27a51c490 2024-08-31T06:13:28.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.590+0000 7f864a681740 1 -- 172.21.15.16:0/4172269084 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f27a51c490 msgr2=0x55f27a8ebb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:28.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.590+0000 7f864a681740 1 --2- 172.21.15.16:0/4172269084 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f27a51c490 0x55f27a8ebb40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f86300e2d70 tx=0x7f86300ebcb0 comp rx=0 tx=0).stop 2024-08-31T06:13:28.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.590+0000 7f8641a00640 1 --2- 172.21.15.16:0/3982768681 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f86240b4d20 0x7f86240b7230 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8634005cc0 tx=0x7f86340079f0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:28.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.590+0000 7f864a681740 1 -- 172.21.15.16:0/4172269084 shutdown_connections 2024-08-31T06:13:28.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.590+0000 7f864a681740 1 --2- 172.21.15.16:0/4172269084 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f27a51c490 0x55f27a8ebb40 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:28.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.591+0000 7f864a681740 1 --2- 172.21.15.16:0/4172269084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f27a524670 0x55f27a8ec080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:28.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.591+0000 7f864a681740 1 --2- 172.21.15.16:0/4172269084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f27a545350 0x55f27a8eb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:28.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.591+0000 7f864a681740 1 -- 172.21.15.16:0/4172269084 >> 172.21.15.16:0/4172269084 conn(0x55f27a539740 msgr2=0x55f27a8e94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:28.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.591+0000 7f864a681740 1 -- 172.21.15.16:0/4172269084 shutdown_connections 2024-08-31T06:13:28.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.591+0000 7f864a681740 1 -- 172.21.15.16:0/4172269084 wait complete. 2024-08-31T06:13:28.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.592+0000 7f864a681740 1 -- start start 2024-08-31T06:13:28.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.592+0000 7f864a681740 1 Processor -- start 2024-08-31T06:13:28.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.595+0000 7f864a681740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f27a51c490 0x55f27a8f71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:28.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.595+0000 7f864a681740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f27a524670 0x55f27a8f76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:28.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.595+0000 7f864a681740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f27a545350 0x55f27a90b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:28.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.595+0000 7f864a681740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f27a6b3890 con 0x55f27a524670 2024-08-31T06:13:28.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.595+0000 7f864a681740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f27a7d7cd0 con 0x55f27a545350 2024-08-31T06:13:28.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.595+0000 7f864a681740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f27a8f9d80 con 0x55f27a51c490 2024-08-31T06:13:28.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.595+0000 7f864a681740 10 client.0 _ll_register_callbacks cb 0x55f27a84bef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:13:28.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.595+0000 7f863be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f27a524670 0x55f27a8f76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:28.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.595+0000 7f8641a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f27a545350 0x55f27a90b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:28.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.595+0000 7f8641a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f27a545350 0x55f27a90b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:54932/0 (socket says 172.21.15.16:54932) 2024-08-31T06:13:28.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.595+0000 7f8641a00640 1 -- 172.21.15.16:0/3503347369 learned_addr learned my addr 172.21.15.16:0/3503347369 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:28.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.595+0000 7f8641000640 1 --2- 172.21.15.16:0/3503347369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f27a51c490 0x55f27a8f71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:28.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.596+0000 7f8641a00640 1 -- 172.21.15.16:0/3503347369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f27a51c490 msgr2=0x55f27a8f71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:28.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.596+0000 7f8641a00640 1 --2- 172.21.15.16:0/3503347369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f27a51c490 0x55f27a8f71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:28.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.596+0000 7f8641a00640 1 -- 172.21.15.16:0/3503347369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f27a524670 msgr2=0x55f27a8f76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:28.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.596+0000 7f8641a00640 1 --2- 172.21.15.16:0/3503347369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f27a524670 0x55f27a8f76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:28.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.596+0000 7f8641a00640 1 -- 172.21.15.16:0/3503347369 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f27a8211b0 con 0x55f27a545350 2024-08-31T06:13:28.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.596+0000 7f8641a00640 1 --2- 172.21.15.16:0/3503347369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f27a545350 0x55f27a90b880 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8634005c90 tx=0x7f8634004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e5c290defa76f6f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:28.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.596+0000 7f8622a00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:13:28.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.596+0000 7f8622a00640 1 -- 172.21.15.16:0/3503347369 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8634003b70 con 0x55f27a545350 2024-08-31T06:13:28.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.596+0000 7f8622a00640 1 -- 172.21.15.16:0/3503347369 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8470+0+0 (secure 0 0 0) 0x7f8634003d10 con 0x55f27a545350 2024-08-31T06:13:28.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.597+0000 7f864a681740 1 -- 172.21.15.16:0/3503347369 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86300ab820 con 0x55f27a545350 2024-08-31T06:13:28.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.597+0000 7f8622a00640 1 -- 172.21.15.16:0/3503347369 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f8634004020 con 0x55f27a545350 2024-08-31T06:13:28.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.597+0000 7f864a681740 10 client.26996 fetch_fsmap learned FSMap version 38 2024-08-31T06:13:28.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.597+0000 7f864a681740 1 -- 172.21.15.16:0/3503347369 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55f27a8fc850 con 0x55f27a545350 2024-08-31T06:13:28.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.597+0000 7f8622a00640 1 -- 172.21.15.16:0/3503347369 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86300ab820 con 0x55f27a545350 2024-08-31T06:13:28.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.599+0000 7f8622a00640 1 -- 172.21.15.16:0/3503347369 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f863402f7c0 con 0x55f27a545350 2024-08-31T06:13:28.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.599+0000 7f864a681740 10 client.26996 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T06:13:28.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.599+0000 7f864a681740 10 client.26996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:13:28.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.599+0000 7f864a681740 20 client.26996 populate_metadata read hostname 'smithi016' 2024-08-31T06:13:28.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.599+0000 7f864a681740 1 --2- 172.21.15.16:0/3503347369 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f27a503cd0 0x55f27a8ffd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:28.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.599+0000 7f864a681740 4 client.26996 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:13:28.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.599+0000 7f864a681740 1 -- 172.21.15.16:0/3503347369 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f27a8636a0 con 0x55f27a503cd0 2024-08-31T06:13:28.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.600+0000 7f8641000640 1 --2- 172.21.15.16:0/3503347369 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f27a503cd0 0x55f27a8ffd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:28.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.601+0000 7f8641000640 1 --2- 172.21.15.16:0/3503347369 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f27a503cd0 0x55f27a8ffd40 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=382aa93d10efdf8b server_cookie=f4e4fbf9f725919a in_seq=0 out_seq=0 2024-08-31T06:13:28.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.601+0000 7f8622a00640 10 client.26996 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:13:28.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.603+0000 7f8622a00640 1 -- 172.21.15.16:0/3503347369 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f27a8636a0 con 0x55f27a503cd0 2024-08-31T06:13:28.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.603+0000 7f8622a00640 10 client.26996 handle_command_reply: tid=0 2024-08-31T06:13:28.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.603+0000 7f864a681740 1 client.26996 shutdown 2024-08-31T06:13:28.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.605+0000 7f864a681740 1 -- 172.21.15.16:0/3503347369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f27a545350 msgr2=0x55f27a90b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:28.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.605+0000 7f864a681740 1 --2- 172.21.15.16:0/3503347369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f27a545350 0x55f27a90b880 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8634005c90 tx=0x7f8634004a40 comp rx=0 tx=0).stop 2024-08-31T06:13:28.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.605+0000 7f864a681740 1 -- 172.21.15.16:0/3503347369 shutdown_connections 2024-08-31T06:13:28.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.605+0000 7f864a681740 1 --2- 172.21.15.16:0/3503347369 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f27a503cd0 0x55f27a8ffd40 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:28.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.605+0000 7f864a681740 1 --2- 172.21.15.16:0/3503347369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f27a545350 0x55f27a90b880 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:28.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.605+0000 7f864a681740 1 --2- 172.21.15.16:0/3503347369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f27a524670 0x55f27a8f76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:28.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.605+0000 7f864a681740 1 --2- 172.21.15.16:0/3503347369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f27a51c490 0x55f27a8f71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:28.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.605+0000 7f8622a00640 0 client.26996 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:13:28.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.605+0000 7f864a681740 1 -- 172.21.15.16:0/3503347369 >> 172.21.15.16:0/3503347369 conn(0x55f27a539740 msgr2=0x55f27a8ee7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:28.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.605+0000 7f864a681740 1 -- 172.21.15.16:0/3503347369 shutdown_connections 2024-08-31T06:13:28.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.605+0000 7f864a681740 1 -- 172.21.15.16:0/3503347369 wait complete. 2024-08-31T06:13:28.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.606+0000 7f864a681740 20 client.26996 trim_cache size 0 max 16384 2024-08-31T06:13:28.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.639+0000 7f864a681740 1 Processor -- start 2024-08-31T06:13:28.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.640+0000 7f864a681740 1 -- start start 2024-08-31T06:13:28.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:28 smithi016 ceph-mon[17194]: pgmap v553: 97 pgs: 97 active+clean; 3.4 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 31 MiB/s wr, 2.26k op/s 2024-08-31T06:13:28.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.641+0000 7f864a681740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f27a51c490 0x55f27a8f8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:28.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.641+0000 7f864a681740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f27a524670 0x55f27a90b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:28.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.641+0000 7f864a681740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f27a545350 0x55f27a8f06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:28.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.641+0000 7f864a681740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f863c1b8310 con 0x55f27a524670 2024-08-31T06:13:28.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.641+0000 7f864a681740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f863c1b5b50 con 0x55f27a545350 2024-08-31T06:13:28.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.641+0000 7f864a681740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f863c1b3a20 con 0x55f27a51c490 2024-08-31T06:13:28.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.642+0000 7f863be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f27a524670 0x55f27a90b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:28.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.642+0000 7f8641a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f27a545350 0x55f27a8f06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:28.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.642+0000 7f8641000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f27a51c490 0x55f27a8f8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:28.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.642+0000 7f863be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f27a524670 0x55f27a90b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57974/0 (socket says 172.21.15.16:57974) 2024-08-31T06:13:28.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.642+0000 7f863be00640 1 -- 172.21.15.16:0/2630946016 learned_addr learned my addr 172.21.15.16:0/2630946016 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:28.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.643+0000 7f863be00640 1 -- 172.21.15.16:0/2630946016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f27a51c490 msgr2=0x55f27a8f8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:28.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.643+0000 7f863be00640 1 --2- 172.21.15.16:0/2630946016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f27a51c490 0x55f27a8f8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:28.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.643+0000 7f863be00640 1 -- 172.21.15.16:0/2630946016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f27a545350 msgr2=0x55f27a8f06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:28.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.643+0000 7f863be00640 1 --2- 172.21.15.16:0/2630946016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f27a545350 0x55f27a8f06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:28.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.643+0000 7f863be00640 1 -- 172.21.15.16:0/2630946016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f27a8e4ad0 con 0x55f27a524670 2024-08-31T06:13:28.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.644+0000 7f863be00640 1 --2- 172.21.15.16:0/2630946016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f27a524670 0x55f27a90b4a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f862802a9b0 tx=0x7f8628031e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c173aeea1e0cb8f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:28.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.645+0000 7f862d600640 1 -- 172.21.15.16:0/2630946016 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8628035040 con 0x55f27a524670 2024-08-31T06:13:28.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.645+0000 7f862d600640 1 -- 172.21.15.16:0/2630946016 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8628031760 con 0x55f27a524670 2024-08-31T06:13:28.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.645+0000 7f862d600640 1 -- 172.21.15.16:0/2630946016 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862803b430 con 0x55f27a524670 2024-08-31T06:13:28.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.646+0000 7f864a681740 1 -- 172.21.15.16:0/2630946016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f27a524670 msgr2=0x55f27a90b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:28.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.646+0000 7f864a681740 1 --2- 172.21.15.16:0/2630946016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f27a524670 0x55f27a90b4a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f862802a9b0 tx=0x7f8628031e20 comp rx=0 tx=0).stop 2024-08-31T06:13:28.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.647+0000 7f864a681740 1 -- 172.21.15.16:0/2630946016 shutdown_connections 2024-08-31T06:13:28.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.647+0000 7f864a681740 1 --2- 172.21.15.16:0/2630946016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f27a545350 0x55f27a8f06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:28.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.647+0000 7f864a681740 1 --2- 172.21.15.16:0/2630946016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f27a524670 0x55f27a90b4a0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:28.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.647+0000 7f864a681740 1 --2- 172.21.15.16:0/2630946016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f27a51c490 0x55f27a8f8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:28.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.647+0000 7f864a681740 1 -- 172.21.15.16:0/2630946016 >> 172.21.15.16:0/2630946016 conn(0x55f27a539740 msgr2=0x55f27a8e9480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:28.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.647+0000 7f864a681740 1 -- 172.21.15.16:0/2630946016 shutdown_connections 2024-08-31T06:13:28.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.647+0000 7f864a681740 1 -- 172.21.15.16:0/2630946016 wait complete. 2024-08-31T06:13:28.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.648+0000 7f864a681740 1 -- start start 2024-08-31T06:13:28.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.648+0000 7f864a681740 1 Processor -- start 2024-08-31T06:13:28.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.650+0000 7f864a681740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f27a51c490 0x55f27a900620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:28.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.650+0000 7f864a681740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f27a545350 0x55f27a900b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:28.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.650+0000 7f864a681740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f27a4d9080 0x55f27a9010a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:28.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.651+0000 7f863be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f27a545350 0x55f27a900b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:28.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.651+0000 7f863be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f27a545350 0x55f27a900b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:54960/0 (socket says 172.21.15.16:54960) 2024-08-31T06:13:28.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.651+0000 7f8641a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f27a4d9080 0x55f27a9010a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:28.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.651+0000 7f8641a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f27a4d9080 0x55f27a9010a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:52726/0 (socket says 172.21.15.16:52726) 2024-08-31T06:13:28.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.652+0000 7f8641000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f27a51c490 0x55f27a900620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:28.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.652+0000 7f8641000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f27a51c490 0x55f27a900620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57984/0 (socket says 172.21.15.16:57984) 2024-08-31T06:13:28.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.652+0000 7f863be00640 1 -- 172.21.15.16:0/1735557188 learned_addr learned my addr 172.21.15.16:0/1735557188 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:28.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.652+0000 7f864a681740 1 -- 172.21.15.16:0/1735557188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f863c1b4070 con 0x55f27a51c490 2024-08-31T06:13:28.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.652+0000 7f864a681740 1 -- 172.21.15.16:0/1735557188 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f863c1b5480 con 0x55f27a545350 2024-08-31T06:13:28.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.652+0000 7f864a681740 1 -- 172.21.15.16:0/1735557188 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f863c1b46e0 con 0x55f27a4d9080 2024-08-31T06:13:28.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.653+0000 7f8641a00640 1 -- 172.21.15.16:0/1735557188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f27a545350 msgr2=0x55f27a900b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:28.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.653+0000 7f8641a00640 1 --2- 172.21.15.16:0/1735557188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f27a545350 0x55f27a900b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:28.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.654+0000 7f8641a00640 1 -- 172.21.15.16:0/1735557188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f27a51c490 msgr2=0x55f27a900620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:28.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.654+0000 7f8641a00640 1 --2- 172.21.15.16:0/1735557188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f27a51c490 0x55f27a900620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:28.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.654+0000 7f8641a00640 1 -- 172.21.15.16:0/1735557188 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f27a985150 con 0x55f27a4d9080 2024-08-31T06:13:28.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.654+0000 7f8641000640 1 --2- 172.21.15.16:0/1735557188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f27a51c490 0x55f27a900620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:13:28.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.654+0000 7f864a681740 10 client.0 _ll_register_callbacks cb 0x7f8634010f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:13:28.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.655+0000 7f864a681740 1 -- 172.21.15.16:0/1735557188 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8628031760 con 0x55f27a4d9080 2024-08-31T06:13:28.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.655+0000 7f8641a00640 1 --2- 172.21.15.16:0/1735557188 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f27a4d9080 0x55f27a9010a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8634005b40 tx=0x7f863400eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be00eb106490345b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:28.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.655+0000 7f8622a00640 10 client.26989 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:13:28.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.656+0000 7f8622a00640 1 -- 172.21.15.16:0/1735557188 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8634003d80 con 0x55f27a4d9080 2024-08-31T06:13:28.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.656+0000 7f8622a00640 1 -- 172.21.15.16:0/1735557188 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8470+0+0 (secure 0 0 0) 0x7f863402f450 con 0x55f27a4d9080 2024-08-31T06:13:28.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.657+0000 7f8622a00640 1 -- 172.21.15.16:0/1735557188 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f863402f690 con 0x55f27a4d9080 2024-08-31T06:13:28.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.657+0000 7f864a681740 10 client.26989 fetch_fsmap learned FSMap version 38 2024-08-31T06:13:28.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.657+0000 7f864a681740 1 -- 172.21.15.16:0/1735557188 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({fsmap=38}) -- 0x55f27a986240 con 0x55f27a4d9080 2024-08-31T06:13:28.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.658+0000 7f8622a00640 1 -- 172.21.15.16:0/1735557188 <== mon.1 v2:172.21.15.184:3300/0 4 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f8634042030 con 0x55f27a4d9080 2024-08-31T06:13:28.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.658+0000 7f864a681740 10 client.26989 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T06:13:28.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.658+0000 7f864a681740 10 client.26989 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:13:28.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.658+0000 7f864a681740 20 client.26989 populate_metadata read hostname 'smithi016' 2024-08-31T06:13:28.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.658+0000 7f864a681740 1 --2- 172.21.15.16:0/1735557188 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f27a9868c0 0x55f27a988d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:28.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.658+0000 7f864a681740 4 client.26989 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T06:13:28.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.658+0000 7f864a681740 1 -- 172.21.15.16:0/1735557188 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f27a85f140 con 0x55f27a9868c0 2024-08-31T06:13:28.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.661+0000 7f8641000640 1 --2- 172.21.15.16:0/1735557188 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f27a9868c0 0x55f27a988d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:28.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.662+0000 7f8641000640 1 --2- 172.21.15.16:0/1735557188 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f27a9868c0 0x55f27a988d10 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58de43ec6d03099e server_cookie=dc18eca0e04e3bcb in_seq=0 out_seq=0 2024-08-31T06:13:28.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.662+0000 7f8622a00640 10 client.26989 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:13:28.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.665+0000 7f8622a00640 1 -- 172.21.15.16:0/1735557188 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f27a85f140 con 0x55f27a9868c0 2024-08-31T06:13:28.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.665+0000 7f8622a00640 10 client.26989 handle_command_reply: tid=0 2024-08-31T06:13:28.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.667+0000 7f864a681740 1 client.26989 shutdown 2024-08-31T06:13:28.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.667+0000 7f864a681740 1 -- 172.21.15.16:0/1735557188 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f27a4d9080 msgr2=0x55f27a9010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:28.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.667+0000 7f864a681740 1 --2- 172.21.15.16:0/1735557188 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f27a4d9080 0x55f27a9010a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8634005b40 tx=0x7f863400eca0 comp rx=0 tx=0).stop 2024-08-31T06:13:28.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.667+0000 7f864a681740 1 -- 172.21.15.16:0/1735557188 shutdown_connections 2024-08-31T06:13:28.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.667+0000 7f864a681740 1 --2- 172.21.15.16:0/1735557188 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f27a9868c0 0x55f27a988d10 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:28.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.667+0000 7f864a681740 1 --2- 172.21.15.16:0/1735557188 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f27a4d9080 0x55f27a9010a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:28.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.667+0000 7f864a681740 1 --2- 172.21.15.16:0/1735557188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f27a545350 0x55f27a900b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:28.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.667+0000 7f864a681740 1 --2- 172.21.15.16:0/1735557188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f27a51c490 0x55f27a900620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:28.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.667+0000 7f864a681740 1 -- 172.21.15.16:0/1735557188 >> 172.21.15.16:0/1735557188 conn(0x55f27a539740 msgr2=0x55f27a8e9480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:28.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.667+0000 7f8622a00640 0 client.26989 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:13:28.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.668+0000 7f864a681740 1 -- 172.21.15.16:0/1735557188 shutdown_connections 2024-08-31T06:13:28.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.668+0000 7f864a681740 1 -- 172.21.15.16:0/1735557188 wait complete. 2024-08-31T06:13:28.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.668+0000 7f864a681740 20 client.26989 trim_cache size 0 max 16384 2024-08-31T06:13:28.672 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:13:28.672 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-08-31T06:13:28.672 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "91b72a34-4ba9-4126-a414-8b862f8c5f78", 2024-08-31T06:13:28.672 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-08-31T06:13:28.672 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:13:28.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.670+0000 7f8643a00640 1 -- 172.21.15.16:0/3982768681 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f86240b4d20 msgr2=0x7f86240b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:28.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.670+0000 7f8643a00640 1 --2- 172.21.15.16:0/3982768681 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f86240b4d20 0x7f86240b7230 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8634005cc0 tx=0x7f86340079f0 comp rx=0 tx=0).stop 2024-08-31T06:13:28.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.670+0000 7f8643a00640 1 -- 172.21.15.16:0/3982768681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f863c102370 msgr2=0x7f863c074c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:28.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.670+0000 7f8643a00640 1 --2- 172.21.15.16:0/3982768681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f863c102370 0x7f863c074c20 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f863000c920 tx=0x7f863000cde0 comp rx=0 tx=0).stop 2024-08-31T06:13:28.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.670+0000 7f8643a00640 1 -- 172.21.15.16:0/3982768681 shutdown_connections 2024-08-31T06:13:28.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.670+0000 7f8643a00640 1 --2- 172.21.15.16:0/3982768681 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f86240b4d20 0x7f86240b7230 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:28.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.670+0000 7f8643a00640 1 --2- 172.21.15.16:0/3982768681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f863c102cc0 0x7f863c06def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:28.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.670+0000 7f8643a00640 1 --2- 172.21.15.16:0/3982768681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f863c10ff20 0x7f863c06e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:28.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.670+0000 7f8643a00640 1 --2- 172.21.15.16:0/3982768681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f863c102370 0x7f863c074c20 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:28.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.670+0000 7f8643a00640 1 -- 172.21.15.16:0/3982768681 >> 172.21.15.16:0/3982768681 conn(0x7f863c0794d0 msgr2=0x7f863c109a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:28.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.670+0000 7f8643a00640 1 -- 172.21.15.16:0/3982768681 shutdown_connections 2024-08-31T06:13:28.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:28.670+0000 7f8643a00640 1 -- 172.21.15.16:0/3982768681 wait complete. 2024-08-31T06:13:28.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:28 smithi155 ceph-mon[21742]: pgmap v553: 97 pgs: 97 active+clean; 3.4 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 31 MiB/s wr, 2.26k op/s 2024-08-31T06:13:28.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:28 smithi184 ceph-mon[22331]: pgmap v553: 97 pgs: 97 active+clean; 3.4 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 31 MiB/s wr, 2.26k op/s 2024-08-31T06:13:29.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:29 smithi016 ceph-mon[17194]: scrub queued for path: / 2024-08-31T06:13:29.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:29 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:13:29.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:29 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:13:29.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:29 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:13:29.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:29 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:13:29.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:29 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:13:29.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:29 smithi155 ceph-mon[21742]: scrub queued for path: / 2024-08-31T06:13:29.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:29 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:13:29.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:29 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:13:29.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:29 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:13:29.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:29 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:13:29.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:29 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:13:29.937 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T06:13:29.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:29 smithi184 ceph-mon[22331]: scrub queued for path: / 2024-08-31T06:13:29.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:29 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:13:29.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:29 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:13:29.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:29 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:13:29.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:29 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:13:29.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:29 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:13:30.627 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:13:30.798 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:30 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:13:30.799 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:30 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:13:30.799 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:30 smithi016 ceph-mon[17194]: pgmap v554: 97 pgs: 97 active+clean; 3.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 26 MiB/s wr, 1.87k op/s 2024-08-31T06:13:30.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:30 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:13:30.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:30 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:13:30.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:30 smithi155 ceph-mon[21742]: pgmap v554: 97 pgs: 97 active+clean; 3.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 26 MiB/s wr, 1.87k op/s 2024-08-31T06:13:30.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:30 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:13:30.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:30 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:13:30.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:30 smithi184 ceph-mon[22331]: pgmap v554: 97 pgs: 97 active+clean; 3.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 26 MiB/s wr, 1.87k op/s 2024-08-31T06:13:31.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.500+0000 7f73a3e00640 1 -- 172.21.15.16:0/499534164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73a4109460 msgr2=0x7f73a4109870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:31.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.500+0000 7f73a3e00640 1 --2- 172.21.15.16:0/499534164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73a4109460 0x7f73a4109870 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f7394009cd0 tx=0x7f739402a8f0 comp rx=0 tx=0).stop 2024-08-31T06:13:31.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.501+0000 7f73a3e00640 1 -- 172.21.15.16:0/499534164 shutdown_connections 2024-08-31T06:13:31.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.501+0000 7f73a3e00640 1 --2- 172.21.15.16:0/499534164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73a4101470 0x7f73a4101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:31.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.501+0000 7f73a3e00640 1 --2- 172.21.15.16:0/499534164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73a4101dc0 0x7f73a4110130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:31.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.501+0000 7f73a3e00640 1 --2- 172.21.15.16:0/499534164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73a4109460 0x7f73a4109870 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:31.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.501+0000 7f73a3e00640 1 -- 172.21.15.16:0/499534164 >> 172.21.15.16:0/499534164 conn(0x7f73a4078420 msgr2=0x7f73a4078860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:31.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.501+0000 7f73a3e00640 1 -- 172.21.15.16:0/499534164 shutdown_connections 2024-08-31T06:13:31.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.501+0000 7f73a3e00640 1 -- 172.21.15.16:0/499534164 wait complete. 2024-08-31T06:13:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.502+0000 7f73a3e00640 1 Processor -- start 2024-08-31T06:13:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.503+0000 7f73a3e00640 1 -- start start 2024-08-31T06:13:31.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.503+0000 7f73a3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73a4101470 0x7f73a41a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:31.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.503+0000 7f73a3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73a4101dc0 0x7f73a41a67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:31.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.503+0000 7f73a3e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73a4109460 0x7f73a410cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:31.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.503+0000 7f73a3e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f73a4103ec0 con 0x7f73a4101dc0 2024-08-31T06:13:31.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.503+0000 7f73a3e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f73a4103d40 con 0x7f73a4101470 2024-08-31T06:13:31.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.503+0000 7f73a3e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f73a4104040 con 0x7f73a4109460 2024-08-31T06:13:31.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.504+0000 7f73a2a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73a4101470 0x7f73a41a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:31.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.504+0000 7f73a2a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73a4101470 0x7f73a41a6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:54990/0 (socket says 172.21.15.16:54990) 2024-08-31T06:13:31.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.504+0000 7f73a2a00640 1 -- 172.21.15.16:0/2964801640 learned_addr learned my addr 172.21.15.16:0/2964801640 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:31.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.504+0000 7f73a3400640 1 --2- 172.21.15.16:0/2964801640 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73a4109460 0x7f73a410cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:31.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.504+0000 7f739a000640 1 --2- 172.21.15.16:0/2964801640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73a4101dc0 0x7f73a41a67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:31.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.504+0000 7f73a2a00640 1 -- 172.21.15.16:0/2964801640 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73a4109460 msgr2=0x7f73a410cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:31.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.504+0000 7f73a2a00640 1 --2- 172.21.15.16:0/2964801640 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73a4109460 0x7f73a410cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:31.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.504+0000 7f73a2a00640 1 -- 172.21.15.16:0/2964801640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73a4101dc0 msgr2=0x7f73a41a67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:31.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.504+0000 7f73a2a00640 1 --2- 172.21.15.16:0/2964801640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73a4101dc0 0x7f73a41a67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:31.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.504+0000 7f73a2a00640 1 -- 172.21.15.16:0/2964801640 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a410d440 con 0x7f73a4101470 2024-08-31T06:13:31.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.504+0000 7f73a3400640 1 --2- 172.21.15.16:0/2964801640 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73a4109460 0x7f73a410cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:13:31.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.505+0000 7f73a2a00640 1 --2- 172.21.15.16:0/2964801640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73a4101470 0x7f73a41a6240 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7394000f80 tx=0x7f7394030da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:31.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.505+0000 7f739be00640 1 -- 172.21.15.16:0/2964801640 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7394003e00 con 0x7f73a4101470 2024-08-31T06:13:31.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.506+0000 7f739be00640 1 -- 172.21.15.16:0/2964801640 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7394040070 con 0x7f73a4101470 2024-08-31T06:13:31.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.506+0000 7f739be00640 1 -- 172.21.15.16:0/2964801640 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7394003e00 con 0x7f73a4101470 2024-08-31T06:13:31.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.506+0000 7f73a3e00640 1 -- 172.21.15.16:0/2964801640 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73a410d6d0 con 0x7f73a4101470 2024-08-31T06:13:31.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.506+0000 7f73a3e00640 1 -- 172.21.15.16:0/2964801640 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73a41ad060 con 0x7f73a4101470 2024-08-31T06:13:31.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.511+0000 7f73b0886740 1 Processor -- start 2024-08-31T06:13:31.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.511+0000 7f739be00640 1 -- 172.21.15.16:0/2964801640 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f73940317a0 con 0x7f73a4101470 2024-08-31T06:13:31.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.511+0000 7f73b0886740 1 -- start start 2024-08-31T06:13:31.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.512+0000 7f73b0886740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ac2689070 0x559ac2a2eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:31.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.512+0000 7f73b0886740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ac26601b0 0x559ac2a2f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:31.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.512+0000 7f73b0886740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ac2668390 0x559ac2a2f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:31.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.512+0000 7f73b0886740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559ac29a5770 con 0x559ac2668390 2024-08-31T06:13:31.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.512+0000 7f73b0886740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559ac2994d00 con 0x559ac2689070 2024-08-31T06:13:31.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.512+0000 7f73b0886740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559ac2998cb0 con 0x559ac26601b0 2024-08-31T06:13:31.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.512+0000 7f73a2a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ac26601b0 0x559ac2a2f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:31.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.512+0000 7f73a2a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ac26601b0 0x559ac2a2f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:52754/0 (socket says 172.21.15.16:52754) 2024-08-31T06:13:31.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.512+0000 7f73a2a00640 1 -- 172.21.15.16:0/3451511725 learned_addr learned my addr 172.21.15.16:0/3451511725 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:31.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.513+0000 7f73a2a00640 1 -- 172.21.15.16:0/3451511725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ac2689070 msgr2=0x559ac2a2eb20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T06:13:31.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.513+0000 7f73a2a00640 1 --2- 172.21.15.16:0/3451511725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ac2689070 0x559ac2a2eb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:31.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.513+0000 7f73a2a00640 1 -- 172.21.15.16:0/3451511725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ac2668390 msgr2=0x559ac2a2f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:31.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.513+0000 7f739a000640 1 --2- 172.21.15.16:0/3451511725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ac2668390 0x559ac2a2f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:31.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.513+0000 7f73a2a00640 1 --2- 172.21.15.16:0/3451511725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ac2668390 0x559ac2a2f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:31.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.513+0000 7f73a2a00640 1 -- 172.21.15.16:0/3451511725 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ac26f4650 con 0x559ac26601b0 2024-08-31T06:13:31.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.513+0000 7f739a000640 1 --2- 172.21.15.16:0/3451511725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ac2668390 0x559ac2a2f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:13:31.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.514+0000 7f739be00640 1 --2- 172.21.15.16:0/2964801640 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f73880b4f50 0x7f73880b7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:31.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.514+0000 7f739be00640 1 -- 172.21.15.16:0/2964801640 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8470+0+0 (secure 0 0 0) 0x7f7394106090 con 0x7f73a4101470 2024-08-31T06:13:31.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.514+0000 7f73a3400640 1 --2- 172.21.15.16:0/2964801640 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f73880b4f50 0x7f73880b7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:31.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.514+0000 7f73a2a00640 1 --2- 172.21.15.16:0/3451511725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ac26601b0 0x559ac2a2f060 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7394103820 tx=0x7f7394085a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd45873cea50b1ae server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:31.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.515+0000 7f7393e00640 1 -- 172.21.15.16:0/3451511725 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73940cc620 con 0x559ac26601b0 2024-08-31T06:13:31.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.515+0000 7f7393e00640 1 -- 172.21.15.16:0/3451511725 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7394036070 con 0x559ac26601b0 2024-08-31T06:13:31.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.515+0000 7f7393e00640 1 -- 172.21.15.16:0/3451511725 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7394109d60 con 0x559ac26601b0 2024-08-31T06:13:31.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.517+0000 7f73b0886740 1 -- 172.21.15.16:0/3451511725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ac26601b0 msgr2=0x559ac2a2f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:31.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.517+0000 7f73a3400640 1 --2- 172.21.15.16:0/2964801640 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f73880b4f50 0x7f73880b7460 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f739c0095f0 tx=0x7f739c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:31.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.517+0000 7f73b0886740 1 --2- 172.21.15.16:0/3451511725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ac26601b0 0x559ac2a2f060 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7394103820 tx=0x7f7394085a60 comp rx=0 tx=0).stop 2024-08-31T06:13:31.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.519+0000 7f73b0886740 1 -- 172.21.15.16:0/3451511725 shutdown_connections 2024-08-31T06:13:31.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.519+0000 7f73b0886740 1 --2- 172.21.15.16:0/3451511725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ac26601b0 0x559ac2a2f060 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:31.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.519+0000 7f73b0886740 1 --2- 172.21.15.16:0/3451511725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ac2668390 0x559ac2a2f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:31.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.519+0000 7f73b0886740 1 --2- 172.21.15.16:0/3451511725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ac2689070 0x559ac2a2eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:31.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.519+0000 7f73b0886740 1 -- 172.21.15.16:0/3451511725 >> 172.21.15.16:0/3451511725 conn(0x559ac267d460 msgr2=0x559ac2a2ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:31.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.519+0000 7f73b0886740 1 -- 172.21.15.16:0/3451511725 shutdown_connections 2024-08-31T06:13:31.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.519+0000 7f73b0886740 1 -- 172.21.15.16:0/3451511725 wait complete. 2024-08-31T06:13:31.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.520+0000 7f73b0886740 1 -- start start 2024-08-31T06:13:31.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.520+0000 7f73b0886740 1 Processor -- start 2024-08-31T06:13:31.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.521+0000 7f73b0886740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ac26601b0 0x559ac2a3a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:31.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.521+0000 7f73b0886740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ac2668390 0x559ac2a3ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:31.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.521+0000 7f73b0886740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ac2689070 0x559ac2a462e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:31.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.521+0000 7f73b0886740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559ac27f7700 con 0x559ac2668390 2024-08-31T06:13:31.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.521+0000 7f73b0886740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559ac291bcb0 con 0x559ac26601b0 2024-08-31T06:13:31.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.521+0000 7f73b0886740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559ac2a3d0c0 con 0x559ac2689070 2024-08-31T06:13:31.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.521+0000 7f73b0886740 10 client.0 _ll_register_callbacks cb 0x559ac2941430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:13:31.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.521+0000 7f73a3400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ac2689070 0x559ac2a462e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:31.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.521+0000 7f73a3400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ac2689070 0x559ac2a462e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:52764/0 (socket says 172.21.15.16:52764) 2024-08-31T06:13:31.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.521+0000 7f73a3400640 1 -- 172.21.15.16:0/1635832687 learned_addr learned my addr 172.21.15.16:0/1635832687 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:31.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.522+0000 7f73a2a00640 1 --2- 172.21.15.16:0/1635832687 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ac26601b0 0x559ac2a3a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:31.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.522+0000 7f739a000640 1 --2- 172.21.15.16:0/1635832687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ac2668390 0x559ac2a3ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:31.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.522+0000 7f73a3400640 1 -- 172.21.15.16:0/1635832687 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ac26601b0 msgr2=0x559ac2a3a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:31.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.522+0000 7f73a3400640 1 --2- 172.21.15.16:0/1635832687 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ac26601b0 0x559ac2a3a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:31.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.522+0000 7f73a3400640 1 -- 172.21.15.16:0/1635832687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ac2668390 msgr2=0x559ac2a3ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:31.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.522+0000 7f73a3400640 1 --2- 172.21.15.16:0/1635832687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ac2668390 0x559ac2a3ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:31.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.522+0000 7f73a3400640 1 -- 172.21.15.16:0/1635832687 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ac295c870 con 0x559ac2689070 2024-08-31T06:13:31.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.522+0000 7f73a3400640 1 --2- 172.21.15.16:0/1635832687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ac2689070 0x559ac2a462e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f739c0038a0 tx=0x7f739c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cf84376aa4b2dd5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:31.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.522+0000 7f7390c00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:13:31.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.523+0000 7f7390c00640 1 -- 172.21.15.16:0/1635832687 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739c00fd50 con 0x559ac2689070 2024-08-31T06:13:31.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.523+0000 7f7390c00640 1 -- 172.21.15.16:0/1635832687 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8470+0+0 (secure 0 0 0) 0x7f739c01e420 con 0x559ac2689070 2024-08-31T06:13:31.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.523+0000 7f73b0886740 1 -- 172.21.15.16:0/1635832687 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7394036070 con 0x559ac2689070 2024-08-31T06:13:31.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.524+0000 7f7390c00640 1 -- 172.21.15.16:0/1635832687 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f739c003e20 con 0x559ac2689070 2024-08-31T06:13:31.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.524+0000 7f73b0886740 10 client.27001 fetch_fsmap learned FSMap version 38 2024-08-31T06:13:31.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.524+0000 7f73b0886740 1 -- 172.21.15.16:0/1635832687 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x559ac2a49020 con 0x559ac2689070 2024-08-31T06:13:31.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.524+0000 7f7390c00640 1 -- 172.21.15.16:0/1635832687 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7394036070 con 0x559ac2689070 2024-08-31T06:13:31.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.525+0000 7f7390c00640 1 -- 172.21.15.16:0/1635832687 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f739c02f680 con 0x559ac2689070 2024-08-31T06:13:31.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.525+0000 7f73b0886740 10 client.27001 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T06:13:31.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.525+0000 7f73b0886740 10 client.27001 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:13:31.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.525+0000 7f73b0886740 20 client.27001 populate_metadata read hostname 'smithi016' 2024-08-31T06:13:31.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.525+0000 7f73b0886740 1 --2- 172.21.15.16:0/1635832687 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559ac26479f0 0x559ac2a4c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:31.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.525+0000 7f73b0886740 4 client.27001 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:13:31.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.525+0000 7f73b0886740 1 -- 172.21.15.16:0/1635832687 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ac29a76c0 con 0x559ac26479f0 2024-08-31T06:13:31.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.525+0000 7f73a2a00640 1 --2- 172.21.15.16:0/1635832687 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559ac26479f0 0x559ac2a4c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:31.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.526+0000 7f7390c00640 10 client.27001 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:13:31.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.526+0000 7f73a2a00640 1 --2- 172.21.15.16:0/1635832687 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559ac26479f0 0x559ac2a4c500 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23fe13436b4d8295 server_cookie=e118a705ae9bd11e in_seq=0 out_seq=0 2024-08-31T06:13:31.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.527+0000 7f7390c00640 1 -- 172.21.15.16:0/1635832687 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559ac29a76c0 con 0x559ac26479f0 2024-08-31T06:13:31.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.527+0000 7f7390c00640 10 client.27001 handle_command_reply: tid=0 2024-08-31T06:13:31.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.527+0000 7f73b0886740 1 client.27001 shutdown 2024-08-31T06:13:31.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.528+0000 7f73b0886740 1 -- 172.21.15.16:0/1635832687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ac2689070 msgr2=0x559ac2a462e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:31.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.528+0000 7f73b0886740 1 --2- 172.21.15.16:0/1635832687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ac2689070 0x559ac2a462e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f739c0038a0 tx=0x7f739c004730 comp rx=0 tx=0).stop 2024-08-31T06:13:31.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.528+0000 7f73b0886740 1 -- 172.21.15.16:0/1635832687 shutdown_connections 2024-08-31T06:13:31.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.528+0000 7f73b0886740 1 --2- 172.21.15.16:0/1635832687 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559ac26479f0 0x559ac2a4c500 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:31.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.529+0000 7f73b0886740 1 --2- 172.21.15.16:0/1635832687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ac2689070 0x559ac2a462e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:31.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.529+0000 7f73b0886740 1 --2- 172.21.15.16:0/1635832687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ac2668390 0x559ac2a3ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:31.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.529+0000 7f73b0886740 1 --2- 172.21.15.16:0/1635832687 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ac26601b0 0x559ac2a3a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:31.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.529+0000 7f73b0886740 1 -- 172.21.15.16:0/1635832687 >> 172.21.15.16:0/1635832687 conn(0x559ac267d460 msgr2=0x559ac2a31cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:31.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.529+0000 7f7390c00640 0 client.27001 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:13:31.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.529+0000 7f73b0886740 1 -- 172.21.15.16:0/1635832687 shutdown_connections 2024-08-31T06:13:31.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.529+0000 7f73b0886740 1 -- 172.21.15.16:0/1635832687 wait complete. 2024-08-31T06:13:31.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.529+0000 7f73b0886740 20 client.27001 trim_cache size 0 max 16384 2024-08-31T06:13:31.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.559+0000 7f73b0886740 1 Processor -- start 2024-08-31T06:13:31.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.560+0000 7f73b0886740 1 -- start start 2024-08-31T06:13:31.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.560+0000 7f73b0886740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ac26601b0 0x559ac2a34340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:31.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.560+0000 7f73b0886740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ac2668390 0x559ac2a2c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:31.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.560+0000 7f73b0886740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ac2689070 0x559ac2a46220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:31.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.560+0000 7f73b0886740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f73a41b8370 con 0x559ac2689070 2024-08-31T06:13:31.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.560+0000 7f73b0886740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f73a41b5bb0 con 0x559ac2668390 2024-08-31T06:13:31.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.560+0000 7f73b0886740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f73a41b3a20 con 0x559ac26601b0 2024-08-31T06:13:31.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.561+0000 7f739a000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ac2668390 0x559ac2a2c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:31.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.561+0000 7f73a2a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ac26601b0 0x559ac2a34340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:31.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.561+0000 7f73a2a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ac26601b0 0x559ac2a34340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:52772/0 (socket says 172.21.15.16:52772) 2024-08-31T06:13:31.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.561+0000 7f73a2a00640 1 -- 172.21.15.16:0/2686471932 learned_addr learned my addr 172.21.15.16:0/2686471932 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:31.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.562+0000 7f73a3400640 1 --2- 172.21.15.16:0/2686471932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ac2689070 0x559ac2a46220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:31.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.562+0000 7f73a2a00640 1 -- 172.21.15.16:0/2686471932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ac2668390 msgr2=0x559ac2a2c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:31.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.562+0000 7f73a2a00640 1 --2- 172.21.15.16:0/2686471932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ac2668390 0x559ac2a2c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:31.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.562+0000 7f73a2a00640 1 -- 172.21.15.16:0/2686471932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ac2689070 msgr2=0x559ac2a46220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:31.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.562+0000 7f73a2a00640 1 --2- 172.21.15.16:0/2686471932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ac2689070 0x559ac2a46220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:31.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.562+0000 7f73a2a00640 1 -- 172.21.15.16:0/2686471932 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ac2a39470 con 0x559ac26601b0 2024-08-31T06:13:31.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.562+0000 7f739a000640 1 --2- 172.21.15.16:0/2686471932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ac2668390 0x559ac2a2c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:13:31.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.562+0000 7f73a2a00640 1 --2- 172.21.15.16:0/2686471932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ac26601b0 0x559ac2a34340 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f73940c8880 tx=0x7f73940c8810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11079dc6d1f0c3a5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:31.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.563+0000 7f7393400640 1 -- 172.21.15.16:0/2686471932 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73940cbfc0 con 0x559ac26601b0 2024-08-31T06:13:31.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.563+0000 7f7393400640 1 -- 172.21.15.16:0/2686471932 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7394036040 con 0x559ac26601b0 2024-08-31T06:13:31.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.563+0000 7f7393400640 1 -- 172.21.15.16:0/2686471932 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73940cc1a0 con 0x559ac26601b0 2024-08-31T06:13:31.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.564+0000 7f73b0886740 1 -- 172.21.15.16:0/2686471932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ac26601b0 msgr2=0x559ac2a34340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:31.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.564+0000 7f73b0886740 1 --2- 172.21.15.16:0/2686471932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ac26601b0 0x559ac2a34340 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f73940c8880 tx=0x7f73940c8810 comp rx=0 tx=0).stop 2024-08-31T06:13:31.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.564+0000 7f73b0886740 1 -- 172.21.15.16:0/2686471932 shutdown_connections 2024-08-31T06:13:31.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.564+0000 7f73b0886740 1 --2- 172.21.15.16:0/2686471932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ac2689070 0x559ac2a46220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:31.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.564+0000 7f73b0886740 1 --2- 172.21.15.16:0/2686471932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ac2668390 0x559ac2a2c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:31.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.564+0000 7f73b0886740 1 --2- 172.21.15.16:0/2686471932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ac26601b0 0x559ac2a34340 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:31.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.564+0000 7f73b0886740 1 -- 172.21.15.16:0/2686471932 >> 172.21.15.16:0/2686471932 conn(0x559ac267d460 msgr2=0x559ac2a33c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:31.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.564+0000 7f73b0886740 1 -- 172.21.15.16:0/2686471932 shutdown_connections 2024-08-31T06:13:31.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.564+0000 7f73b0886740 1 -- 172.21.15.16:0/2686471932 wait complete. 2024-08-31T06:13:31.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.565+0000 7f73b0886740 1 -- start start 2024-08-31T06:13:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.565+0000 7f73b0886740 1 Processor -- start 2024-08-31T06:13:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.566+0000 7f73b0886740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ac26601b0 0x559ac2a42b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.566+0000 7f73b0886740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ac2668390 0x559ac2a3eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.566+0000 7f73b0886740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ac2689070 0x559ac2a3f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.566+0000 7f73b0886740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f73a41b4070 con 0x559ac2668390 2024-08-31T06:13:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.566+0000 7f73b0886740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f73a41b54e0 con 0x559ac2689070 2024-08-31T06:13:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.566+0000 7f73b0886740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f73a41b4740 con 0x559ac26601b0 2024-08-31T06:13:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.566+0000 7f73b0886740 10 client.0 _ll_register_callbacks cb 0x7f739c0044e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:13:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.567+0000 7f739a000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ac2668390 0x559ac2a3eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.567+0000 7f739a000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ac2668390 0x559ac2a3eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58020/0 (socket says 172.21.15.16:58020) 2024-08-31T06:13:31.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.567+0000 7f739a000640 1 -- 172.21.15.16:0/3130035039 learned_addr learned my addr 172.21.15.16:0/3130035039 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:31.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.567+0000 7f73a2a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ac26601b0 0x559ac2a42b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:31.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.568+0000 7f73a3400640 1 --2- 172.21.15.16:0/3130035039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ac2689070 0x559ac2a3f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:31.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.568+0000 7f739a000640 1 -- 172.21.15.16:0/3130035039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ac26601b0 msgr2=0x559ac2a42b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:31.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.568+0000 7f739a000640 1 --2- 172.21.15.16:0/3130035039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ac26601b0 0x559ac2a42b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:31.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.568+0000 7f739a000640 1 -- 172.21.15.16:0/3130035039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ac2689070 msgr2=0x559ac2a3f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:31.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.568+0000 7f739a000640 1 --2- 172.21.15.16:0/3130035039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ac2689070 0x559ac2a3f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:31.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.568+0000 7f739a000640 1 -- 172.21.15.16:0/3130035039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ac2a3aa80 con 0x559ac2668390 2024-08-31T06:13:31.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.568+0000 7f73a2a00640 1 --2- 172.21.15.16:0/3130035039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ac26601b0 0x559ac2a42b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:13:31.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.569+0000 7f739a000640 1 --2- 172.21.15.16:0/3130035039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ac2668390 0x559ac2a3eff0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f73a406c7c0 tx=0x7f738c012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79d96870f94bc8e5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:31.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.569+0000 7f7390c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:13:31.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.570+0000 7f7390c00640 1 -- 172.21.15.16:0/3130035039 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738c01f020 con 0x559ac2668390 2024-08-31T06:13:31.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.570+0000 7f7390c00640 1 -- 172.21.15.16:0/3130035039 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8470+0+0 (secure 0 0 0) 0x7f738c010bb0 con 0x559ac2668390 2024-08-31T06:13:31.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.570+0000 7f73b0886740 1 -- 172.21.15.16:0/3130035039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7394036040 con 0x559ac2668390 2024-08-31T06:13:31.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.570+0000 7f7390c00640 1 -- 172.21.15.16:0/3130035039 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f738c010df0 con 0x559ac2668390 2024-08-31T06:13:31.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.570+0000 7f73b0886740 10 client.16794 fetch_fsmap learned FSMap version 38 2024-08-31T06:13:31.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.570+0000 7f73b0886740 1 -- 172.21.15.16:0/3130035039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x559ac2ab7de0 con 0x559ac2668390 2024-08-31T06:13:31.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.571+0000 7f7390c00640 1 -- 172.21.15.16:0/3130035039 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7394036040 con 0x559ac2668390 2024-08-31T06:13:31.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.572+0000 7f7390c00640 1 -- 172.21.15.16:0/3130035039 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f738c01a070 con 0x559ac2668390 2024-08-31T06:13:31.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.572+0000 7f73b0886740 10 client.16794 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T06:13:31.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.572+0000 7f73b0886740 10 client.16794 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:13:31.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.572+0000 7f73b0886740 20 client.16794 populate_metadata read hostname 'smithi016' 2024-08-31T06:13:31.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.572+0000 7f73b0886740 1 --2- 172.21.15.16:0/3130035039 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559ac261cda0 0x559ac2aba6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:31.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.572+0000 7f73b0886740 4 client.16794 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T06:13:31.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.572+0000 7f73b0886740 1 -- 172.21.15.16:0/3130035039 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559ac29a3160 con 0x559ac261cda0 2024-08-31T06:13:31.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.573+0000 7f73a2a00640 1 --2- 172.21.15.16:0/3130035039 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559ac261cda0 0x559ac2aba6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:31.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.573+0000 7f73a2a00640 1 --2- 172.21.15.16:0/3130035039 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559ac261cda0 0x559ac2aba6b0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3f16e4e8f3ed3f5 server_cookie=d1137acb677b0846 in_seq=0 out_seq=0 2024-08-31T06:13:31.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.573+0000 7f7390c00640 10 client.16794 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:13:31.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.574+0000 7f7390c00640 1 -- 172.21.15.16:0/3130035039 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2842 (crc 0 0 0) 0x559ac29a3160 con 0x559ac261cda0 2024-08-31T06:13:31.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.574+0000 7f7390c00640 10 client.16794 handle_command_reply: tid=0 2024-08-31T06:13:31.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.574+0000 7f73b0886740 1 client.16794 shutdown 2024-08-31T06:13:31.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.574+0000 7f73b0886740 1 -- 172.21.15.16:0/3130035039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ac2668390 msgr2=0x559ac2a3eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:31.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.574+0000 7f73b0886740 1 --2- 172.21.15.16:0/3130035039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ac2668390 0x559ac2a3eff0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f73a406c7c0 tx=0x7f738c012b60 comp rx=0 tx=0).stop 2024-08-31T06:13:31.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.574+0000 7f73b0886740 1 -- 172.21.15.16:0/3130035039 shutdown_connections 2024-08-31T06:13:31.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.574+0000 7f73b0886740 1 --2- 172.21.15.16:0/3130035039 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559ac261cda0 0x559ac2aba6b0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:31.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.575+0000 7f73b0886740 1 --2- 172.21.15.16:0/3130035039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ac2689070 0x559ac2a3f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:31.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.575+0000 7f73b0886740 1 --2- 172.21.15.16:0/3130035039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ac2668390 0x559ac2a3eff0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:31.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.575+0000 7f73b0886740 1 --2- 172.21.15.16:0/3130035039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ac26601b0 0x559ac2a42b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:31.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.575+0000 7f73b0886740 1 -- 172.21.15.16:0/3130035039 >> 172.21.15.16:0/3130035039 conn(0x559ac267d460 msgr2=0x559ac2a2fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:31.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.575+0000 7f7390c00640 0 client.16794 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:13:31.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.576+0000 7f73b0886740 1 -- 172.21.15.16:0/3130035039 shutdown_connections 2024-08-31T06:13:31.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.577+0000 7f73b0886740 1 -- 172.21.15.16:0/3130035039 wait complete. 2024-08-31T06:13:31.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.577+0000 7f73b0886740 20 client.16794 trim_cache size 0 max 16384 2024-08-31T06:13:31.581 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:13:31.581 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T06:13:31.582 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-08-31T06:13:31.582 INFO:teuthology.orchestra.run.smithi016.stdout: "91b72a34-4ba9-4126-a414-8b862f8c5f78": { 2024-08-31T06:13:31.582 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:13:31.582 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "91b72a34-4ba9-4126-a414-8b862f8c5f78", 2024-08-31T06:13:31.582 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-08-31T06:13:31.582 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:13:31.582 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:13:31.582 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-08-31T06:13:31.582 INFO:teuthology.orchestra.run.smithi016.stdout: "e739a16e-8ace-43a8-a1ce-ac27d39ef8a3": { 2024-08-31T06:13:31.582 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:54.084646+0000", 2024-08-31T06:13:31.583 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:13:31.583 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:13:31.583 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:13:31.583 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:13:31.583 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:13:31.583 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:13:31.583 INFO:teuthology.orchestra.run.smithi016.stdout: "91b72a34-4ba9-4126-a414-8b862f8c5f78": { 2024-08-31T06:13:31.583 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:13:28.663237+0000", 2024-08-31T06:13:31.583 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:13:31.583 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:13:31.583 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:13:31.584 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:13:31.584 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:13:31.584 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:13:31.584 INFO:teuthology.orchestra.run.smithi016.stdout: "a10cce76-99ab-46b5-b3b2-07b82899b8f9": { 2024-08-31T06:13:31.584 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:11:43.967546+0000", 2024-08-31T06:13:31.584 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:13:31.584 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:13:31.584 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:13:31.584 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:13:31.584 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:13:31.585 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:13:31.585 INFO:teuthology.orchestra.run.smithi016.stdout: "c1fd6904-2e02-47fa-939a-47349b9fe36b": { 2024-08-31T06:13:31.585 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:06.678691+0000", 2024-08-31T06:13:31.585 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:13:31.585 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:13:31.585 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:13:31.585 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:13:31.585 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:13:31.586 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:13:31.586 INFO:teuthology.orchestra.run.smithi016.stdout: "78d6ef02-8551-45e8-ad60-9627eb573fc1": { 2024-08-31T06:13:31.586 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:09:13.547865+0000", 2024-08-31T06:13:31.586 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:13:31.586 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:13:31.586 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:13:31.586 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:13:31.586 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:13:31.586 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:13:31.586 INFO:teuthology.orchestra.run.smithi016.stdout: "c8455d74-2fe7-4d93-bcf2-19b419d16016": { 2024-08-31T06:13:31.586 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:08:24.035960+0000", 2024-08-31T06:13:31.586 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:13:31.587 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:13:31.587 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:13:31.587 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:13:31.587 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:13:31.587 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:13:31.587 INFO:teuthology.orchestra.run.smithi016.stdout: "88d7bdb8-cb1c-45e5-ad29-4cd8da6c6003": { 2024-08-31T06:13:31.587 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:07:39.648942+0000", 2024-08-31T06:13:31.587 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:13:31.587 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:13:31.587 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:13:31.588 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:13:31.588 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:13:31.588 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:13:31.588 INFO:teuthology.orchestra.run.smithi016.stdout: "6a03046e-cdf4-4a14-b9bd-b560b8968467": { 2024-08-31T06:13:31.588 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:12:38.002354+0000", 2024-08-31T06:13:31.588 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:13:31.588 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:13:31.588 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:13:31.588 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:13:31.588 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:13:31.588 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:13:31.588 INFO:teuthology.orchestra.run.smithi016.stdout: "c300b7e8-f3fd-482c-a909-9329bc61e570": { 2024-08-31T06:13:31.589 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:06:49.868046+0000", 2024-08-31T06:13:31.589 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:13:31.589 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:13:31.589 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:13:31.589 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:13:31.589 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:13:31.589 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:13:31.589 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:13:31.589 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:13:31.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.579+0000 7f73a3e00640 1 -- 172.21.15.16:0/2964801640 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f73880b4f50 msgr2=0x7f73880b7460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:31.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.579+0000 7f73a3e00640 1 --2- 172.21.15.16:0/2964801640 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f73880b4f50 0x7f73880b7460 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f739c0095f0 tx=0x7f739c005f90 comp rx=0 tx=0).stop 2024-08-31T06:13:31.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.579+0000 7f73a3e00640 1 -- 172.21.15.16:0/2964801640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73a4101470 msgr2=0x7f73a41a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:31.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.579+0000 7f73a3e00640 1 --2- 172.21.15.16:0/2964801640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73a4101470 0x7f73a41a6240 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7394000f80 tx=0x7f7394030da0 comp rx=0 tx=0).stop 2024-08-31T06:13:31.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.579+0000 7f73a3e00640 1 -- 172.21.15.16:0/2964801640 shutdown_connections 2024-08-31T06:13:31.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.579+0000 7f73a3e00640 1 --2- 172.21.15.16:0/2964801640 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f73880b4f50 0x7f73880b7460 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:31.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.579+0000 7f73a3e00640 1 --2- 172.21.15.16:0/2964801640 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73a4109460 0x7f73a410cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:31.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.579+0000 7f73a3e00640 1 --2- 172.21.15.16:0/2964801640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73a4101dc0 0x7f73a41a67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:31.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.579+0000 7f73a3e00640 1 --2- 172.21.15.16:0/2964801640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73a4101470 0x7f73a41a6240 secure :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7394000f80 tx=0x7f7394030da0 comp rx=0 tx=0).stop 2024-08-31T06:13:31.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.579+0000 7f73a3e00640 1 -- 172.21.15.16:0/2964801640 >> 172.21.15.16:0/2964801640 conn(0x7f73a4078420 msgr2=0x7f73a4114630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:31.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.580+0000 7f73a3e00640 1 -- 172.21.15.16:0/2964801640 shutdown_connections 2024-08-31T06:13:31.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:13:31.580+0000 7f73a3e00640 1 -- 172.21.15.16:0/2964801640 wait complete. 2024-08-31T06:13:31.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 365.0 s, 2055.0 tps, lat 48.478 ms stddev 57.390 2024-08-31T06:13:32.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.425+0000 7f808e800640 1 -- 172.21.15.16:0/1247618039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80881044e0 msgr2=0x7f80881048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:32.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.425+0000 7f808e800640 1 --2- 172.21.15.16:0/1247618039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80881044e0 0x7f80881048f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f807c009860 tx=0x7f807c02a340 comp rx=0 tx=0).stop 2024-08-31T06:13:32.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.426+0000 7f808e800640 1 -- 172.21.15.16:0/1247618039 shutdown_connections 2024-08-31T06:13:32.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.426+0000 7f808e800640 1 --2- 172.21.15.16:0/1247618039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f808810d940 0x7f808810fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:32.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.426+0000 7f808e800640 1 --2- 172.21.15.16:0/1247618039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8088104e30 0x7f808810d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:32.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.426+0000 7f808e800640 1 --2- 172.21.15.16:0/1247618039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80881044e0 0x7f80881048f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:32.430 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.426+0000 7f808e800640 1 -- 172.21.15.16:0/1247618039 >> 172.21.15.16:0/1247618039 conn(0x7f80880fdfd0 msgr2=0x7f8088100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:32.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.430+0000 7f808e800640 1 -- 172.21.15.16:0/1247618039 shutdown_connections 2024-08-31T06:13:32.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.430+0000 7f808e800640 1 -- 172.21.15.16:0/1247618039 wait complete. 2024-08-31T06:13:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.432+0000 7f808e800640 1 Processor -- start 2024-08-31T06:13:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.432+0000 7f808e800640 1 -- start start 2024-08-31T06:13:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.433+0000 7f808e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80881044e0 0x7f8088114520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.433+0000 7f808e800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8088104e30 0x7f80881107f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.433+0000 7f808e800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f808810d940 0x7f8088110f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.433+0000 7f808e800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f80881157d0 con 0x7f80881044e0 2024-08-31T06:13:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.433+0000 7f808e800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8088115650 con 0x7f808810d940 2024-08-31T06:13:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.433+0000 7f808e800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8088115950 con 0x7f8088104e30 2024-08-31T06:13:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.433+0000 7f808d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80881044e0 0x7f8088114520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.434+0000 7f808ca00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8088104e30 0x7f80881107f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.434+0000 7f808ca00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8088104e30 0x7f80881107f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:52796/0 (socket says 172.21.15.16:52796) 2024-08-31T06:13:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.434+0000 7f808ca00640 1 -- 172.21.15.16:0/1624476947 learned_addr learned my addr 172.21.15.16:0/1624476947 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.435+0000 7f808d400640 1 -- 172.21.15.16:0/1624476947 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8088104e30 msgr2=0x7f80881107f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.435+0000 7f808d400640 1 --2- 172.21.15.16:0/1624476947 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8088104e30 0x7f80881107f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.435+0000 7f808d400640 1 -- 172.21.15.16:0/1624476947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f808810d940 msgr2=0x7f8088110f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.435+0000 7f808d400640 1 --2- 172.21.15.16:0/1624476947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f808810d940 0x7f8088110f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.435+0000 7f808d400640 1 -- 172.21.15.16:0/1624476947 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80881ac750 con 0x7f80881044e0 2024-08-31T06:13:32.439 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.436+0000 7f808d400640 1 --2- 172.21.15.16:0/1624476947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80881044e0 0x7f8088114520 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f807c035040 tx=0x7f807c032dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:32.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.437+0000 7f8086000640 1 -- 172.21.15.16:0/1624476947 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807c037040 con 0x7f80881044e0 2024-08-31T06:13:32.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.437+0000 7f8086000640 1 -- 172.21.15.16:0/1624476947 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f807c030bd0 con 0x7f80881044e0 2024-08-31T06:13:32.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.437+0000 7f8086000640 1 -- 172.21.15.16:0/1624476947 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807c0035f0 con 0x7f80881044e0 2024-08-31T06:13:32.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.437+0000 7f808e800640 1 -- 172.21.15.16:0/1624476947 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80881ac920 con 0x7f80881044e0 2024-08-31T06:13:32.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.438+0000 7f808e800640 1 -- 172.21.15.16:0/1624476947 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80881acd60 con 0x7f80881044e0 2024-08-31T06:13:32.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.453+0000 7f8086000640 1 -- 172.21.15.16:0/1624476947 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f807c043070 con 0x7f80881044e0 2024-08-31T06:13:32.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.457+0000 7f8086000640 1 --2- 172.21.15.16:0/1624476947 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f80700b4d20 0x7f80700b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:32.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.457+0000 7f8086000640 1 -- 172.21.15.16:0/1624476947 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8470+0+0 (secure 0 0 0) 0x7f807c104640 con 0x7f80881044e0 2024-08-31T06:13:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.459+0000 7f8097010740 1 Processor -- start 2024-08-31T06:13:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.460+0000 7f8097010740 1 -- start start 2024-08-31T06:13:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.460+0000 7f808ca00640 1 --2- 172.21.15.16:0/1624476947 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f80700b4d20 0x7f80700b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.460+0000 7f8097010740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5571eb4011b0 0x5571eb7a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.460+0000 7f8097010740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571eb3d82f0 0x5571eb7a5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.460+0000 7f8097010740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571eb3e04d0 0x5571eb7a5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.461+0000 7f808d400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5571eb4011b0 0x5571eb7a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.461+0000 7f8097010740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5571eb719f70 con 0x5571eb3e04d0 2024-08-31T06:13:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.461+0000 7f8097010740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5571eb709500 con 0x5571eb3d82f0 2024-08-31T06:13:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.461+0000 7f8097010740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5571eb70d4b0 con 0x5571eb4011b0 2024-08-31T06:13:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.462+0000 7f808ca00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571eb3d82f0 0x5571eb7a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.462+0000 7f808ca00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571eb3d82f0 0x5571eb7a5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:55044/0 (socket says 172.21.15.16:55044) 2024-08-31T06:13:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.462+0000 7f808ca00640 1 -- 172.21.15.16:0/3642913990 learned_addr learned my addr 172.21.15.16:0/3642913990 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.462+0000 7f808de00640 1 --2- 172.21.15.16:0/3642913990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571eb3e04d0 0x5571eb7a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.463+0000 7f808ca00640 1 -- 172.21.15.16:0/3642913990 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5571eb4011b0 msgr2=0x5571eb7a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.463+0000 7f808ca00640 1 --2- 172.21.15.16:0/3642913990 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5571eb4011b0 0x5571eb7a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.463+0000 7f808ca00640 1 -- 172.21.15.16:0/3642913990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571eb3e04d0 msgr2=0x5571eb7a5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.463+0000 7f808ca00640 1 --2- 172.21.15.16:0/3642913990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571eb3e04d0 0x5571eb7a5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.463+0000 7f808ca00640 1 -- 172.21.15.16:0/3642913990 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571eb6c3c10 con 0x5571eb3d82f0 2024-08-31T06:13:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.463+0000 7f808d400640 1 --2- 172.21.15.16:0/3642913990 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5571eb4011b0 0x5571eb7a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:13:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.463+0000 7f808de00640 1 --2- 172.21.15.16:0/3642913990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571eb3e04d0 0x5571eb7a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:13:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.464+0000 7f808ca00640 1 --2- 172.21.15.16:0/1624476947 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f80700b4d20 0x7f80700b7230 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8074011630 tx=0x7f8074009f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.464+0000 7f808ca00640 1 --2- 172.21.15.16:0/3642913990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571eb3d82f0 0x5571eb7a5040 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f807400f8a0 tx=0x7f8074020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e63e6be67f48f64 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.465+0000 7f807a600640 1 -- 172.21.15.16:0/3642913990 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807401a040 con 0x5571eb3d82f0 2024-08-31T06:13:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.466+0000 7f807a600640 1 -- 172.21.15.16:0/3642913990 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8074024030 con 0x5571eb3d82f0 2024-08-31T06:13:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.466+0000 7f807a600640 1 -- 172.21.15.16:0/3642913990 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80740038e0 con 0x5571eb3d82f0 2024-08-31T06:13:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.468+0000 7f8097010740 1 -- 172.21.15.16:0/3642913990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571eb3d82f0 msgr2=0x5571eb7a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.468+0000 7f8097010740 1 --2- 172.21.15.16:0/3642913990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571eb3d82f0 0x5571eb7a5040 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f807400f8a0 tx=0x7f8074020000 comp rx=0 tx=0).stop 2024-08-31T06:13:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.470+0000 7f8097010740 1 -- 172.21.15.16:0/3642913990 shutdown_connections 2024-08-31T06:13:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.470+0000 7f8097010740 1 --2- 172.21.15.16:0/3642913990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571eb3e04d0 0x5571eb7a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:32.473 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.470+0000 7f8097010740 1 --2- 172.21.15.16:0/3642913990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571eb3d82f0 0x5571eb7a5040 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:32.473 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.470+0000 7f8097010740 1 --2- 172.21.15.16:0/3642913990 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5571eb4011b0 0x5571eb7a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:32.473 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.470+0000 7f8097010740 1 -- 172.21.15.16:0/3642913990 >> 172.21.15.16:0/3642913990 conn(0x5571eb3f5300 msgr2=0x5571eb7a29f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:32.473 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.471+0000 7f8097010740 1 -- 172.21.15.16:0/3642913990 shutdown_connections 2024-08-31T06:13:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.471+0000 7f8097010740 1 -- 172.21.15.16:0/3642913990 wait complete. 2024-08-31T06:13:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.472+0000 7f8097010740 1 -- start start 2024-08-31T06:13:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.472+0000 7f8097010740 1 Processor -- start 2024-08-31T06:13:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.473+0000 7f8097010740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571eb3d82f0 0x5571eb7b0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.474+0000 7f8097010740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5571eb3e04d0 0x5571eb7b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.474+0000 7f8097010740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571eb4011b0 0x5571eb7b0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.474+0000 7f8097010740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5571eb642ab0 con 0x5571eb3d82f0 2024-08-31T06:13:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.474+0000 7f8097010740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5571eb6b0320 con 0x5571eb4011b0 2024-08-31T06:13:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.474+0000 7f8097010740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5571eb7b3280 con 0x5571eb3e04d0 2024-08-31T06:13:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.474+0000 7f8097010740 10 client.0 _ll_register_callbacks cb 0x5571eb6b66a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:13:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.474+0000 7f808d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571eb3d82f0 0x5571eb7b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.474+0000 7f808de00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571eb4011b0 0x5571eb7b0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.474+0000 7f808d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571eb3d82f0 0x5571eb7b0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58072/0 (socket says 172.21.15.16:58072) 2024-08-31T06:13:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.474+0000 7f808d400640 1 -- 172.21.15.16:0/3503691320 learned_addr learned my addr 172.21.15.16:0/3503691320 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.475+0000 7f808ca00640 1 --2- 172.21.15.16:0/3503691320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5571eb3e04d0 0x5571eb7b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.475+0000 7f808de00640 1 -- 172.21.15.16:0/3503691320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5571eb3e04d0 msgr2=0x5571eb7b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.475+0000 7f808de00640 1 --2- 172.21.15.16:0/3503691320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5571eb3e04d0 0x5571eb7b05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:32.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.475+0000 7f808de00640 1 -- 172.21.15.16:0/3503691320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571eb3d82f0 msgr2=0x5571eb7b0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:32.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.475+0000 7f808de00640 1 --2- 172.21.15.16:0/3503691320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571eb3d82f0 0x5571eb7b0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:32.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.475+0000 7f808de00640 1 -- 172.21.15.16:0/3503691320 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571eb46c790 con 0x5571eb4011b0 2024-08-31T06:13:32.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.476+0000 7f808de00640 1 --2- 172.21.15.16:0/3503691320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571eb4011b0 0x5571eb7b0af0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8080005fd0 tx=0x7f8080010b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=166d09fd47a1f46f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:32.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.476+0000 7f806ea00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:13:32.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.476+0000 7f806ea00640 1 -- 172.21.15.16:0/3503691320 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8080004d20 con 0x5571eb4011b0 2024-08-31T06:13:32.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.476+0000 7f806ea00640 1 -- 172.21.15.16:0/3503691320 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8470+0+0 (secure 0 0 0) 0x7f8080004b20 con 0x5571eb4011b0 2024-08-31T06:13:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.477+0000 7f8097010740 1 -- 172.21.15.16:0/3503691320 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8074024030 con 0x5571eb4011b0 2024-08-31T06:13:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.477+0000 7f806ea00640 1 -- 172.21.15.16:0/3503691320 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f8080004d20 con 0x5571eb4011b0 2024-08-31T06:13:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.477+0000 7f8097010740 10 client.27023 fetch_fsmap learned FSMap version 38 2024-08-31T06:13:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.477+0000 7f8097010740 1 -- 172.21.15.16:0/3503691320 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5571eb4e5620 con 0x5571eb4011b0 2024-08-31T06:13:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.478+0000 7f806ea00640 1 -- 172.21.15.16:0/3503691320 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8080003e20 con 0x5571eb4011b0 2024-08-31T06:13:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.478+0000 7f806ea00640 1 -- 172.21.15.16:0/3503691320 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f8080024020 con 0x5571eb4011b0 2024-08-31T06:13:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.481+0000 7f8097010740 10 client.27023 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T06:13:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.481+0000 7f8097010740 10 client.27023 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:13:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.481+0000 7f8097010740 20 client.27023 populate_metadata read hostname 'smithi016' 2024-08-31T06:13:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.481+0000 7f8097010740 1 --2- 172.21.15.16:0/3503691320 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5571eb3bfd80 0x5571eb7c2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.481+0000 7f8097010740 4 client.27023 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:13:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.481+0000 7f8097010740 1 -- 172.21.15.16:0/3503691320 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8074024030 con 0x5571eb3bfd80 2024-08-31T06:13:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.482+0000 7f808d400640 1 --2- 172.21.15.16:0/3503691320 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5571eb3bfd80 0x5571eb7c2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.483+0000 7f808d400640 1 --2- 172.21.15.16:0/3503691320 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5571eb3bfd80 0x5571eb7c2240 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7274d0b4dc92b0f8 server_cookie=290d51072e50ae6e in_seq=0 out_seq=0 2024-08-31T06:13:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.483+0000 7f806ea00640 10 client.27023 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:13:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.484+0000 7f806ea00640 1 -- 172.21.15.16:0/3503691320 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f8074024030 con 0x5571eb3bfd80 2024-08-31T06:13:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.484+0000 7f806ea00640 10 client.27023 handle_command_reply: tid=0 2024-08-31T06:13:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.484+0000 7f8097010740 1 client.27023 shutdown 2024-08-31T06:13:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.485+0000 7f8097010740 1 -- 172.21.15.16:0/3503691320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571eb4011b0 msgr2=0x5571eb7b0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.485+0000 7f8097010740 1 --2- 172.21.15.16:0/3503691320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571eb4011b0 0x5571eb7b0af0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8080005fd0 tx=0x7f8080010b90 comp rx=0 tx=0).stop 2024-08-31T06:13:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.486+0000 7f8097010740 1 -- 172.21.15.16:0/3503691320 shutdown_connections 2024-08-31T06:13:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.486+0000 7f8097010740 1 --2- 172.21.15.16:0/3503691320 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5571eb3bfd80 0x5571eb7c2240 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.486+0000 7f8097010740 1 --2- 172.21.15.16:0/3503691320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5571eb3e04d0 0x5571eb7b05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.486+0000 7f8097010740 1 --2- 172.21.15.16:0/3503691320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571eb4011b0 0x5571eb7b0af0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.486+0000 7f8097010740 1 --2- 172.21.15.16:0/3503691320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571eb3d82f0 0x5571eb7b0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.486+0000 7f8097010740 1 -- 172.21.15.16:0/3503691320 >> 172.21.15.16:0/3503691320 conn(0x5571eb3f5300 msgr2=0x5571eb7a5b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.486+0000 7f806ea00640 0 client.27023 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:13:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.487+0000 7f8097010740 1 -- 172.21.15.16:0/3503691320 shutdown_connections 2024-08-31T06:13:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.487+0000 7f8097010740 1 -- 172.21.15.16:0/3503691320 wait complete. 2024-08-31T06:13:32.490 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.487+0000 7f8097010740 20 client.27023 trim_cache size 0 max 16384 2024-08-31T06:13:32.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.523+0000 7f8097010740 1 Processor -- start 2024-08-31T06:13:32.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.524+0000 7f8097010740 1 -- start start 2024-08-31T06:13:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.525+0000 7f8097010740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571eb3d82f0 0x5571eb7b1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.525+0000 7f8097010740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5571eb3e04d0 0x5571eb7ad240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:32.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.525+0000 7f8097010740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571eb4011b0 0x5571eb7ad780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:32.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.525+0000 7f8097010740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f80881b80c0 con 0x5571eb4011b0 2024-08-31T06:13:32.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.525+0000 7f8097010740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f80881b5900 con 0x5571eb3d82f0 2024-08-31T06:13:32.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.525+0000 7f8097010740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f80881b3770 con 0x5571eb3e04d0 2024-08-31T06:13:32.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.525+0000 7f808ca00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5571eb3e04d0 0x5571eb7ad240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:32.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.525+0000 7f808ca00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5571eb3e04d0 0x5571eb7ad240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:52818/0 (socket says 172.21.15.16:52818) 2024-08-31T06:13:32.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.525+0000 7f808ca00640 1 -- 172.21.15.16:0/1451051888 learned_addr learned my addr 172.21.15.16:0/1451051888 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:32.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.525+0000 7f808de00640 1 --2- 172.21.15.16:0/1451051888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571eb4011b0 0x5571eb7ad780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:32.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.525+0000 7f808d400640 1 --2- 172.21.15.16:0/1451051888 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571eb3d82f0 0x5571eb7b1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:32.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.526+0000 7f808ca00640 1 -- 172.21.15.16:0/1451051888 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571eb3d82f0 msgr2=0x5571eb7b1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:32.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.526+0000 7f808ca00640 1 --2- 172.21.15.16:0/1451051888 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571eb3d82f0 0x5571eb7b1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:32.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.526+0000 7f808ca00640 1 -- 172.21.15.16:0/1451051888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571eb4011b0 msgr2=0x5571eb7ad780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:32.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.526+0000 7f808ca00640 1 --2- 172.21.15.16:0/1451051888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571eb4011b0 0x5571eb7ad780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.526+0000 7f808ca00640 1 -- 172.21.15.16:0/1451051888 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571eb7ae550 con 0x5571eb3e04d0 2024-08-31T06:13:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.526+0000 7f808de00640 1 --2- 172.21.15.16:0/1451051888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571eb4011b0 0x5571eb7ad780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:13:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.526+0000 7f808ca00640 1 --2- 172.21.15.16:0/1451051888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5571eb3e04d0 0x5571eb7ad240 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8074004610 tx=0x7f8074005670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=517450e391052669 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.526+0000 7f8079c00640 1 -- 172.21.15.16:0/1451051888 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807401a040 con 0x5571eb3e04d0 2024-08-31T06:13:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.526+0000 7f8079c00640 1 -- 172.21.15.16:0/1451051888 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8074024040 con 0x5571eb3e04d0 2024-08-31T06:13:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.526+0000 7f8079c00640 1 -- 172.21.15.16:0/1451051888 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8074028a20 con 0x5571eb3e04d0 2024-08-31T06:13:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.528+0000 7f8097010740 1 -- 172.21.15.16:0/1451051888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5571eb3e04d0 msgr2=0x5571eb7ad240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.528+0000 7f8097010740 1 --2- 172.21.15.16:0/1451051888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5571eb3e04d0 0x5571eb7ad240 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8074004610 tx=0x7f8074005670 comp rx=0 tx=0).stop 2024-08-31T06:13:32.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.528+0000 7f8097010740 1 -- 172.21.15.16:0/1451051888 shutdown_connections 2024-08-31T06:13:32.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.528+0000 7f8097010740 1 --2- 172.21.15.16:0/1451051888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5571eb3e04d0 0x5571eb7ad240 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:32.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.528+0000 7f8097010740 1 --2- 172.21.15.16:0/1451051888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571eb4011b0 0x5571eb7ad780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:32.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.528+0000 7f8097010740 1 --2- 172.21.15.16:0/1451051888 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571eb3d82f0 0x5571eb7b1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:32.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.529+0000 7f8097010740 1 -- 172.21.15.16:0/1451051888 >> 172.21.15.16:0/1451051888 conn(0x5571eb3f5300 msgr2=0x5571eb7bb4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:32.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.529+0000 7f8097010740 1 -- 172.21.15.16:0/1451051888 shutdown_connections 2024-08-31T06:13:32.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.529+0000 7f8097010740 1 -- 172.21.15.16:0/1451051888 wait complete. 2024-08-31T06:13:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.529+0000 7f8097010740 1 -- start start 2024-08-31T06:13:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.530+0000 7f8097010740 1 Processor -- start 2024-08-31T06:13:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.530+0000 7f8097010740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5571eb3d82f0 0x5571eb7b8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.530+0000 7f8097010740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571eb3e04d0 0x5571eb7b6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.530+0000 7f8097010740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571eb4011b0 0x5571eb7b6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.530+0000 7f8097010740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f80881b3dc0 con 0x5571eb4011b0 2024-08-31T06:13:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.530+0000 7f8097010740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f80881b5230 con 0x5571eb3e04d0 2024-08-31T06:13:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.530+0000 7f8097010740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f80881b4490 con 0x5571eb3d82f0 2024-08-31T06:13:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.530+0000 7f8097010740 10 client.0 _ll_register_callbacks cb 0x5571eb6b66a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:13:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.531+0000 7f808d400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5571eb3d82f0 0x5571eb7b8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.531+0000 7f808d400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5571eb3d82f0 0x5571eb7b8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:52832/0 (socket says 172.21.15.16:52832) 2024-08-31T06:13:32.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.531+0000 7f808d400640 1 -- 172.21.15.16:0/1761622372 learned_addr learned my addr 172.21.15.16:0/1761622372 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:32.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.531+0000 7f808de00640 1 --2- 172.21.15.16:0/1761622372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571eb4011b0 0x5571eb7b6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:32.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.531+0000 7f808ca00640 1 --2- 172.21.15.16:0/1761622372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571eb3e04d0 0x5571eb7b6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:32.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.531+0000 7f808d400640 1 -- 172.21.15.16:0/1761622372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571eb3e04d0 msgr2=0x5571eb7b6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:32.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.531+0000 7f808d400640 1 --2- 172.21.15.16:0/1761622372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571eb3e04d0 0x5571eb7b6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:32.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.531+0000 7f808d400640 1 -- 172.21.15.16:0/1761622372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571eb4011b0 msgr2=0x5571eb7b6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:32.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.532+0000 7f808d400640 1 --2- 172.21.15.16:0/1761622372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571eb4011b0 0x5571eb7b6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:32.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.532+0000 7f808d400640 1 -- 172.21.15.16:0/1761622372 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571eb82bc50 con 0x5571eb3d82f0 2024-08-31T06:13:32.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.532+0000 7f808de00640 1 --2- 172.21.15.16:0/1761622372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571eb4011b0 0x5571eb7b6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:13:32.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.532+0000 7f808d400640 1 --2- 172.21.15.16:0/1761622372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5571eb3d82f0 0x5571eb7b8f20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f807c009780 tx=0x7f807c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc72cd5a58480ddd server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:32.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.532+0000 7f806ea00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:13:32.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.532+0000 7f806ea00640 1 -- 172.21.15.16:0/1761622372 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807c0b6170 con 0x5571eb3d82f0 2024-08-31T06:13:32.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.533+0000 7f806ea00640 1 -- 172.21.15.16:0/1761622372 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8470+0+0 (secure 0 0 0) 0x7f807c0c7070 con 0x5571eb3d82f0 2024-08-31T06:13:32.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.533+0000 7f8097010740 1 -- 172.21.15.16:0/1761622372 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8074024040 con 0x5571eb3d82f0 2024-08-31T06:13:32.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.533+0000 7f806ea00640 1 -- 172.21.15.16:0/1761622372 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f807c0cf650 con 0x5571eb3d82f0 2024-08-31T06:13:32.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.534+0000 7f8097010740 10 client.27019 fetch_fsmap learned FSMap version 38 2024-08-31T06:13:32.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.534+0000 7f8097010740 1 -- 172.21.15.16:0/1761622372 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5571eb82cdf0 con 0x5571eb3d82f0 2024-08-31T06:13:32.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.534+0000 7f806ea00640 1 -- 172.21.15.16:0/1761622372 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8074024040 con 0x5571eb3d82f0 2024-08-31T06:13:32.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.534+0000 7f806ea00640 1 -- 172.21.15.16:0/1761622372 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f807c0cf190 con 0x5571eb3d82f0 2024-08-31T06:13:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.534+0000 7f8097010740 10 client.27019 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T06:13:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.534+0000 7f8097010740 10 client.27019 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:13:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.534+0000 7f8097010740 20 client.27019 populate_metadata read hostname 'smithi016' 2024-08-31T06:13:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.534+0000 7f8097010740 1 --2- 172.21.15.16:0/1761622372 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5571eb3951c0 0x5571eb82f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.534+0000 7f8097010740 4 client.27019 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:13:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.534+0000 7f8097010740 1 -- 172.21.15.16:0/1761622372 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5571eb71bec0 con 0x5571eb3951c0 2024-08-31T06:13:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.535+0000 7f808ca00640 1 --2- 172.21.15.16:0/1761622372 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5571eb3951c0 0x5571eb82f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.536+0000 7f806ea00640 10 client.27019 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:13:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:32.537+0000 7f808ca00640 1 --2- 172.21.15.16:0/1761622372 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5571eb3951c0 0x5571eb82f5e0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80f970a76cc2551c server_cookie=ca77f500e90fb576 in_seq=0 out_seq=0 2024-08-31T06:13:32.802 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:32 smithi016 ceph-mon[17194]: pgmap v555: 97 pgs: 97 active+clean; 3.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 20 MiB/s wr, 1.50k op/s 2024-08-31T06:13:32.837 INFO:tasks.cephfs.filesystem:scrub status for tag:91b72a34-4ba9-4126-a414-8b862f8c5f78 - {'path': '/', 'tag': '91b72a34-4ba9-4126-a414-8b862f8c5f78', 'options': 'recursive,force'} 2024-08-31T06:13:32.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:32 smithi155 ceph-mon[21742]: pgmap v555: 97 pgs: 97 active+clean; 3.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 20 MiB/s wr, 1.50k op/s 2024-08-31T06:13:32.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:32 smithi184 ceph-mon[22331]: pgmap v555: 97 pgs: 97 active+clean; 3.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 20 MiB/s wr, 1.50k op/s 2024-08-31T06:13:34.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:34 smithi155 ceph-mon[21742]: pgmap v556: 97 pgs: 97 active+clean; 3.5 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 20 MiB/s wr, 1.47k op/s 2024-08-31T06:13:34.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:34 smithi016 ceph-mon[17194]: pgmap v556: 97 pgs: 97 active+clean; 3.5 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 20 MiB/s wr, 1.47k op/s 2024-08-31T06:13:34.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:34 smithi184 ceph-mon[22331]: pgmap v556: 97 pgs: 97 active+clean; 3.5 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 20 MiB/s wr, 1.47k op/s 2024-08-31T06:13:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 370.0 s, 481.2 tps, lat 39.288 ms stddev 45.796 2024-08-31T06:13:36.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:36 smithi155 ceph-mon[21742]: pgmap v557: 97 pgs: 97 active+clean; 3.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 17 MiB/s wr, 1.37k op/s 2024-08-31T06:13:36.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:36 smithi016 ceph-mon[17194]: pgmap v557: 97 pgs: 97 active+clean; 3.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 17 MiB/s wr, 1.37k op/s 2024-08-31T06:13:36.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:36 smithi184 ceph-mon[22331]: pgmap v557: 97 pgs: 97 active+clean; 3.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 17 MiB/s wr, 1.37k op/s 2024-08-31T06:13:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:38.380+0000 7f806ea00640 1 -- 172.21.15.16:0/1761622372 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5571eb71bec0 con 0x5571eb3951c0 2024-08-31T06:13:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:38.380+0000 7f806ea00640 10 client.27019 handle_command_reply: tid=0 2024-08-31T06:13:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:38.380+0000 7f8097010740 1 client.27019 shutdown 2024-08-31T06:13:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:38.380+0000 7f8097010740 1 -- 172.21.15.16:0/1761622372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5571eb3d82f0 msgr2=0x5571eb7b8f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:38.380+0000 7f8097010740 1 --2- 172.21.15.16:0/1761622372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5571eb3d82f0 0x5571eb7b8f20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f807c009780 tx=0x7f807c005f90 comp rx=0 tx=0).stop 2024-08-31T06:13:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:38.380+0000 7f8097010740 1 -- 172.21.15.16:0/1761622372 shutdown_connections 2024-08-31T06:13:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:38.380+0000 7f8097010740 1 --2- 172.21.15.16:0/1761622372 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5571eb3951c0 0x5571eb82f5e0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:38.380+0000 7f8097010740 1 --2- 172.21.15.16:0/1761622372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571eb4011b0 0x5571eb7b6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:38.380+0000 7f8097010740 1 --2- 172.21.15.16:0/1761622372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5571eb3e04d0 0x5571eb7b6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:38.380+0000 7f8097010740 1 --2- 172.21.15.16:0/1761622372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5571eb3d82f0 0x5571eb7b8f20 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:38.380+0000 7f806ea00640 0 client.27019 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:13:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:38.380+0000 7f8097010740 1 -- 172.21.15.16:0/1761622372 >> 172.21.15.16:0/1761622372 conn(0x5571eb3f5300 msgr2=0x5571eb7b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:38.381+0000 7f8097010740 1 -- 172.21.15.16:0/1761622372 shutdown_connections 2024-08-31T06:13:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:38.381+0000 7f8097010740 1 -- 172.21.15.16:0/1761622372 wait complete. 2024-08-31T06:13:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:38.381+0000 7f8097010740 20 client.27019 trim_cache size 0 max 16384 2024-08-31T06:13:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:13:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:13:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:13:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:13:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:38.382+0000 7f808e800640 1 -- 172.21.15.16:0/1624476947 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f80700b4d20 msgr2=0x7f80700b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:38.382+0000 7f808e800640 1 --2- 172.21.15.16:0/1624476947 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f80700b4d20 0x7f80700b7230 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8074011630 tx=0x7f8074009f90 comp rx=0 tx=0).stop 2024-08-31T06:13:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:38.382+0000 7f808e800640 1 -- 172.21.15.16:0/1624476947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80881044e0 msgr2=0x7f8088114520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:38.382+0000 7f808e800640 1 --2- 172.21.15.16:0/1624476947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80881044e0 0x7f8088114520 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f807c035040 tx=0x7f807c032dc0 comp rx=0 tx=0).stop 2024-08-31T06:13:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:38.382+0000 7f808e800640 1 -- 172.21.15.16:0/1624476947 shutdown_connections 2024-08-31T06:13:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:38.382+0000 7f808e800640 1 --2- 172.21.15.16:0/1624476947 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f80700b4d20 0x7f80700b7230 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:38.382+0000 7f808e800640 1 --2- 172.21.15.16:0/1624476947 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8088104e30 0x7f80881107f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:38.382+0000 7f808e800640 1 --2- 172.21.15.16:0/1624476947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f808810d940 0x7f8088110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:38.382+0000 7f808e800640 1 --2- 172.21.15.16:0/1624476947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80881044e0 0x7f8088114520 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:38.383+0000 7f808e800640 1 -- 172.21.15.16:0/1624476947 >> 172.21.15.16:0/1624476947 conn(0x7f80880fdfd0 msgr2=0x7f8088107450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:38.386 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:38.383+0000 7f808e800640 1 -- 172.21.15.16:0/1624476947 shutdown_connections 2024-08-31T06:13:38.386 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:38.383+0000 7f808e800640 1 -- 172.21.15.16:0/1624476947 wait complete. 2024-08-31T06:13:38.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:38 smithi155 ceph-mon[21742]: pgmap v558: 97 pgs: 97 active+clean; 3.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 10 MiB/s wr, 855 op/s 2024-08-31T06:13:38.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:38 smithi016 ceph-mon[17194]: pgmap v558: 97 pgs: 97 active+clean; 3.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 10 MiB/s wr, 855 op/s 2024-08-31T06:13:38.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:38 smithi184 ceph-mon[22331]: pgmap v558: 97 pgs: 97 active+clean; 3.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 10 MiB/s wr, 855 op/s 2024-08-31T06:13:40.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:40 smithi155 ceph-mon[21742]: pgmap v559: 97 pgs: 97 active+clean; 3.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 9.0 MiB/s wr, 770 op/s 2024-08-31T06:13:40.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:40 smithi016 ceph-mon[17194]: pgmap v559: 97 pgs: 97 active+clean; 3.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 9.0 MiB/s wr, 770 op/s 2024-08-31T06:13:40.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:40 smithi184 ceph-mon[22331]: pgmap v559: 97 pgs: 97 active+clean; 3.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 9.0 MiB/s wr, 770 op/s 2024-08-31T06:13:41.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 375.0 s, 1499.0 tps, lat 121.116 ms stddev 674.044 2024-08-31T06:13:42.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:42 smithi155 ceph-mon[21742]: pgmap v560: 97 pgs: 97 active+clean; 3.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 6.9 MiB/s wr, 603 op/s 2024-08-31T06:13:42.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:42 smithi016 ceph-mon[17194]: pgmap v560: 97 pgs: 97 active+clean; 3.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 6.9 MiB/s wr, 603 op/s 2024-08-31T06:13:42.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:42 smithi184 ceph-mon[22331]: pgmap v560: 97 pgs: 97 active+clean; 3.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 6.9 MiB/s wr, 603 op/s 2024-08-31T06:13:43.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:43 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:13:43.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:43 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:13:43.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:43 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:13:44.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:44 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:13:44.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:44 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:13:44.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:44 smithi155 ceph-mon[21742]: pgmap v561: 97 pgs: 97 active+clean; 3.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 5.9 MiB/s wr, 555 op/s 2024-08-31T06:13:44.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:44 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:13:44.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:44 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:13:44.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:44 smithi016 ceph-mon[17194]: pgmap v561: 97 pgs: 97 active+clean; 3.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 5.9 MiB/s wr, 555 op/s 2024-08-31T06:13:44.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:44 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:13:44.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:44 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:13:44.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:44 smithi184 ceph-mon[22331]: pgmap v561: 97 pgs: 97 active+clean; 3.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 5.9 MiB/s wr, 555 op/s 2024-08-31T06:13:46.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 380.0 s, 2118.6 tps, lat 46.607 ms stddev 53.592 2024-08-31T06:13:46.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:46 smithi155 ceph-mon[21742]: pgmap v562: 97 pgs: 97 active+clean; 3.5 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 7.2 MiB/s wr, 642 op/s 2024-08-31T06:13:46.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:46 smithi016 ceph-mon[17194]: pgmap v562: 97 pgs: 97 active+clean; 3.5 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 7.2 MiB/s wr, 642 op/s 2024-08-31T06:13:46.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:46 smithi184 ceph-mon[22331]: pgmap v562: 97 pgs: 97 active+clean; 3.5 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 7.2 MiB/s wr, 642 op/s 2024-08-31T06:13:48.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:48 smithi155 ceph-mon[21742]: pgmap v563: 97 pgs: 97 active+clean; 3.5 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.2 MiB/s wr, 505 op/s 2024-08-31T06:13:48.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:48 smithi016 ceph-mon[17194]: pgmap v563: 97 pgs: 97 active+clean; 3.5 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.2 MiB/s wr, 505 op/s 2024-08-31T06:13:48.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:48 smithi184 ceph-mon[22331]: pgmap v563: 97 pgs: 97 active+clean; 3.5 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.2 MiB/s wr, 505 op/s 2024-08-31T06:13:50.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:50 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-31T06:13:50.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:50 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T06:13:50.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:50 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T06:13:50.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:50 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:13:50.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:50 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:50 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:50 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:50 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:50 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:50 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:50 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:50 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:50 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:50 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:50 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:50 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:50 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.874 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:50 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.874 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:50 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.874 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:50 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.874 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:50 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:13:50.874 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:50 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:13:50.874 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:50 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:13:50.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:50 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-31T06:13:50.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:50 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T06:13:50.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:50 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T06:13:50.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:50 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:13:50.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:50 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:50 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:50 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:50 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:50 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:50 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:50 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:50 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:50 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:50 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:50 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:50 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:50 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:50 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:50 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:50 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:50 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:13:50.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:50 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:13:50.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:50 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:13:50.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:50 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-31T06:13:50.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:50 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-31T06:13:50.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:50 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-31T06:13:50.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:50 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:13:50.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:50 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:50 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:50 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:50 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:50 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:50 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:50 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:50 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:50 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:50 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:50 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:50 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:50 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:50 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:50 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.975 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:50 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-31T06:13:50.975 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:50 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:13:50.975 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:50 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:13:50.975 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:50 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:13:51.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 385.0 s, 2297.0 tps, lat 43.862 ms stddev 52.889 2024-08-31T06:13:51.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.669+0000 7fd4cea00640 1 -- 172.21.15.16:0/1983598866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4c81089d0 msgr2=0x7fd4c810f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:51.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.669+0000 7fd4cea00640 1 --2- 172.21.15.16:0/1983598866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4c81089d0 0x7fd4c810f200 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0009be0 tx=0x7fd4c002c650 comp rx=0 tx=0).stop 2024-08-31T06:13:51.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.669+0000 7fd4cea00640 1 -- 172.21.15.16:0/1983598866 shutdown_connections 2024-08-31T06:13:51.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.669+0000 7fd4cea00640 1 --2- 172.21.15.16:0/1983598866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4c81089d0 0x7fd4c810f200 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:51.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.669+0000 7fd4cea00640 1 --2- 172.21.15.16:0/1983598866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4c8108080 0x7fd4c8108490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:51.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.669+0000 7fd4cea00640 1 --2- 172.21.15.16:0/1983598866 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4c8102750 0x7fd4c8102b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:51.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.669+0000 7fd4cea00640 1 -- 172.21.15.16:0/1983598866 >> 172.21.15.16:0/1983598866 conn(0x7fd4c807a4e0 msgr2=0x7fd4c807a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:51.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.669+0000 7fd4cea00640 1 -- 172.21.15.16:0/1983598866 shutdown_connections 2024-08-31T06:13:51.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.669+0000 7fd4cea00640 1 -- 172.21.15.16:0/1983598866 wait complete. 2024-08-31T06:13:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.670+0000 7fd4cea00640 1 Processor -- start 2024-08-31T06:13:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.671+0000 7fd4cea00640 1 -- start start 2024-08-31T06:13:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.671+0000 7fd4cea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4c8102750 0x7fd4c8197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:51.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.671+0000 7fd4cea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4c8108080 0x7fd4c8198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:51.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.671+0000 7fd4cea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4c81089d0 0x7fd4c819d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:51.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.671+0000 7fd4cea00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c81115d0 con 0x7fd4c8108080 2024-08-31T06:13:51.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.671+0000 7fd4cea00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c8111450 con 0x7fd4c81089d0 2024-08-31T06:13:51.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.671+0000 7fd4cea00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c8111750 con 0x7fd4c8102750 2024-08-31T06:13:51.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.672+0000 7fd4c7e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4c8102750 0x7fd4c8197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:51.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.672+0000 7fd4bf400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4c8108080 0x7fd4c8198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:51.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.672+0000 7fd4bf400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4c8108080 0x7fd4c8198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52006/0 (socket says 172.21.15.16:52006) 2024-08-31T06:13:51.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.672+0000 7fd4bf400640 1 -- 172.21.15.16:0/3636933903 learned_addr learned my addr 172.21.15.16:0/3636933903 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:51.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.672+0000 7fd4cca00640 1 --2- 172.21.15.16:0/3636933903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4c81089d0 0x7fd4c819d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:51.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.673+0000 7fd4cca00640 1 -- 172.21.15.16:0/3636933903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4c8102750 msgr2=0x7fd4c8197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:51.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.673+0000 7fd4cca00640 1 --2- 172.21.15.16:0/3636933903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4c8102750 0x7fd4c8197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:51.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.673+0000 7fd4cca00640 1 -- 172.21.15.16:0/3636933903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4c8108080 msgr2=0x7fd4c8198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:51.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.673+0000 7fd4cca00640 1 --2- 172.21.15.16:0/3636933903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4c8108080 0x7fd4c8198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:51.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.673+0000 7fd4cca00640 1 -- 172.21.15.16:0/3636933903 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4c819de40 con 0x7fd4c81089d0 2024-08-31T06:13:51.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.674+0000 7fd4cca00640 1 --2- 172.21.15.16:0/3636933903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4c81089d0 0x7fd4c819d7d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0036040 tx=0x7fd4c0003cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:51.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.674+0000 7fd4c5600640 1 -- 172.21.15.16:0/3636933903 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0042020 con 0x7fd4c81089d0 2024-08-31T06:13:51.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.674+0000 7fd4c5600640 1 -- 172.21.15.16:0/3636933903 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4c00338f0 con 0x7fd4c81089d0 2024-08-31T06:13:51.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.674+0000 7fd4c5600640 1 -- 172.21.15.16:0/3636933903 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0042020 con 0x7fd4c81089d0 2024-08-31T06:13:51.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.675+0000 7fd4cea00640 1 -- 172.21.15.16:0/3636933903 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4c819e0d0 con 0x7fd4c81089d0 2024-08-31T06:13:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.675+0000 7fd4cea00640 1 -- 172.21.15.16:0/3636933903 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4c819e4b0 con 0x7fd4c81089d0 2024-08-31T06:13:51.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.681+0000 7fd4c5600640 1 -- 172.21.15.16:0/3636933903 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fd4c0045020 con 0x7fd4c81089d0 2024-08-31T06:13:51.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.681+0000 7fd4d59f0740 1 Processor -- start 2024-08-31T06:13:51.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.681+0000 7fd4d59f0740 1 -- start start 2024-08-31T06:13:51.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.682+0000 7fd4d59f0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b314eab1b0 0x55b31524e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:51.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.682+0000 7fd4d59f0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b314e822f0 0x55b31524ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:51.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.682+0000 7fd4d59f0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b314e8a4d0 0x55b31524f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:51.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.682+0000 7fd4d59f0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b3151c3f70 con 0x55b314e8a4d0 2024-08-31T06:13:51.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.682+0000 7fd4d59f0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b3151b3500 con 0x55b314e822f0 2024-08-31T06:13:51.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.682+0000 7fd4d59f0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b3151b74b0 con 0x55b314eab1b0 2024-08-31T06:13:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.682+0000 7fd4c7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b314e822f0 0x55b31524ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.683+0000 7fd4c7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b314e822f0 0x55b31524ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:35208/0 (socket says 172.21.15.16:35208) 2024-08-31T06:13:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.683+0000 7fd4c7e00640 1 -- 172.21.15.16:0/1119086938 learned_addr learned my addr 172.21.15.16:0/1119086938 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.683+0000 7fd4bf400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b314eab1b0 0x55b31524e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.683+0000 7fd4bf400640 1 --2- 172.21.15.16:0/1119086938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b314e8a4d0 0x55b31524f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:51.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.683+0000 7fd4c7e00640 1 -- 172.21.15.16:0/1119086938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b314eab1b0 msgr2=0x55b31524e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:51.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.683+0000 7fd4c7e00640 1 --2- 172.21.15.16:0/1119086938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b314eab1b0 0x55b31524e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:51.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.683+0000 7fd4c7e00640 1 -- 172.21.15.16:0/1119086938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b314e8a4d0 msgr2=0x55b31524f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:51.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.683+0000 7fd4c7e00640 1 --2- 172.21.15.16:0/1119086938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b314e8a4d0 0x55b31524f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:51.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.683+0000 7fd4c7e00640 1 -- 172.21.15.16:0/1119086938 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b31516dc10 con 0x55b314e822f0 2024-08-31T06:13:51.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.683+0000 7fd4bf400640 1 --2- 172.21.15.16:0/1119086938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b314e8a4d0 0x55b31524f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:13:51.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.684+0000 7fd4c5600640 1 --2- 172.21.15.16:0/3636933903 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd4ac0b4f50 0x7fd4ac0b7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:51.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.684+0000 7fd4c7e00640 1 --2- 172.21.15.16:0/1119086938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b314e822f0 0x55b31524ebd0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd4b0009d00 tx=0x7fd4b0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfabb85086f83056 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:51.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.684+0000 7fd4bf400640 1 --2- 172.21.15.16:0/3636933903 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd4ac0b4f50 0x7fd4ac0b7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:51.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.684+0000 7fd4c5600640 1 -- 172.21.15.16:0/3636933903 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8470+0+0 (secure 0 0 0) 0x7fd4c01059b0 con 0x7fd4c81089d0 2024-08-31T06:13:51.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.685+0000 7fd4bd200640 1 -- 172.21.15.16:0/1119086938 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b001e070 con 0x55b314e822f0 2024-08-31T06:13:51.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.685+0000 7fd4bd200640 1 -- 172.21.15.16:0/1119086938 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4b0019ae0 con 0x55b314e822f0 2024-08-31T06:13:51.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.685+0000 7fd4bd200640 1 -- 172.21.15.16:0/1119086938 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b000eab0 con 0x55b314e822f0 2024-08-31T06:13:51.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.686+0000 7fd4d59f0740 1 -- 172.21.15.16:0/1119086938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b314e822f0 msgr2=0x55b31524ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:51.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.686+0000 7fd4d59f0740 1 --2- 172.21.15.16:0/1119086938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b314e822f0 0x55b31524ebd0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd4b0009d00 tx=0x7fd4b0011810 comp rx=0 tx=0).stop 2024-08-31T06:13:51.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.687+0000 7fd4bf400640 1 --2- 172.21.15.16:0/3636933903 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd4ac0b4f50 0x7fd4ac0b7460 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd4b80058c0 tx=0x7fd4b800a380 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:51.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.687+0000 7fd4d59f0740 1 -- 172.21.15.16:0/1119086938 shutdown_connections 2024-08-31T06:13:51.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.687+0000 7fd4d59f0740 1 --2- 172.21.15.16:0/1119086938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b314e8a4d0 0x55b31524f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.687+0000 7fd4d59f0740 1 --2- 172.21.15.16:0/1119086938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b314e822f0 0x55b31524ebd0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.687+0000 7fd4d59f0740 1 --2- 172.21.15.16:0/1119086938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b314eab1b0 0x55b31524e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.687+0000 7fd4d59f0740 1 -- 172.21.15.16:0/1119086938 >> 172.21.15.16:0/1119086938 conn(0x55b314e9f300 msgr2=0x55b31524c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.687+0000 7fd4d59f0740 1 -- 172.21.15.16:0/1119086938 shutdown_connections 2024-08-31T06:13:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.687+0000 7fd4d59f0740 1 -- 172.21.15.16:0/1119086938 wait complete. 2024-08-31T06:13:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.688+0000 7fd4d59f0740 1 -- start start 2024-08-31T06:13:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.688+0000 7fd4d59f0740 1 Processor -- start 2024-08-31T06:13:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.689+0000 7fd4d59f0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b314e822f0 0x55b315259cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.689+0000 7fd4d59f0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b314e8a4d0 0x55b31525a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.689+0000 7fd4d59f0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b314eab1b0 0x55b31525a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.689+0000 7fd4d59f0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b3150ecab0 con 0x55b314e8a4d0 2024-08-31T06:13:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.689+0000 7fd4d59f0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b31515a320 con 0x55b314e822f0 2024-08-31T06:13:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.689+0000 7fd4d59f0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b31525ce10 con 0x55b314eab1b0 2024-08-31T06:13:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.689+0000 7fd4d59f0740 10 client.0 _ll_register_callbacks cb 0x55b315193e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:13:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.690+0000 7fd4c7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b314e822f0 0x55b315259cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.690+0000 7fd4c7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b314e822f0 0x55b315259cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:35210/0 (socket says 172.21.15.16:35210) 2024-08-31T06:13:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.690+0000 7fd4c7e00640 1 -- 172.21.15.16:0/2827739949 learned_addr learned my addr 172.21.15.16:0/2827739949 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.690+0000 7fd4bf400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b314e8a4d0 0x55b31525a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.690+0000 7fd4cca00640 1 --2- 172.21.15.16:0/2827739949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b314eab1b0 0x55b31525a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.691+0000 7fd4c7e00640 1 -- 172.21.15.16:0/2827739949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b314eab1b0 msgr2=0x55b31525a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.691+0000 7fd4c7e00640 1 --2- 172.21.15.16:0/2827739949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b314eab1b0 0x55b31525a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.691+0000 7fd4c7e00640 1 -- 172.21.15.16:0/2827739949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b314e8a4d0 msgr2=0x55b31525a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.691+0000 7fd4c7e00640 1 --2- 172.21.15.16:0/2827739949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b314e8a4d0 0x55b31525a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.691+0000 7fd4c7e00640 1 -- 172.21.15.16:0/2827739949 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b314f16790 con 0x55b314e822f0 2024-08-31T06:13:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.691+0000 7fd4cca00640 1 --2- 172.21.15.16:0/2827739949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b314eab1b0 0x55b31525a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:13:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.691+0000 7fd4bf400640 1 --2- 172.21.15.16:0/2827739949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b314e8a4d0 0x55b31525a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:13:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.691+0000 7fd4c7e00640 1 --2- 172.21.15.16:0/2827739949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b314e822f0 0x55b315259cb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd4b00094d0 tx=0x7fd4b0009930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e41ec9d6b232d7a2 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.691+0000 7fd4b5600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:13:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.692+0000 7fd4b5600640 1 -- 172.21.15.16:0/2827739949 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b0027070 con 0x55b314e822f0 2024-08-31T06:13:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.692+0000 7fd4d59f0740 1 -- 172.21.15.16:0/2827739949 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4b0019ae0 con 0x55b314e822f0 2024-08-31T06:13:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.692+0000 7fd4b5600640 1 -- 172.21.15.16:0/2827739949 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8470+0+0 (secure 0 0 0) 0x7fd4b0019840 con 0x55b314e822f0 2024-08-31T06:13:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.693+0000 7fd4b5600640 1 -- 172.21.15.16:0/2827739949 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fd4b0032050 con 0x55b314e822f0 2024-08-31T06:13:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.694+0000 7fd4d59f0740 10 client.27041 fetch_fsmap learned FSMap version 38 2024-08-31T06:13:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.694+0000 7fd4d59f0740 1 -- 172.21.15.16:0/2827739949 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55b314f8f620 con 0x55b314e822f0 2024-08-31T06:13:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.694+0000 7fd4b5600640 1 -- 172.21.15.16:0/2827739949 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4b0019ae0 con 0x55b314e822f0 2024-08-31T06:13:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.694+0000 7fd4b5600640 1 -- 172.21.15.16:0/2827739949 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7fd4b000e7b0 con 0x55b314e822f0 2024-08-31T06:13:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.694+0000 7fd4d59f0740 10 client.27041 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T06:13:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.694+0000 7fd4d59f0740 10 client.27041 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:13:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.694+0000 7fd4d59f0740 20 client.27041 populate_metadata read hostname 'smithi016' 2024-08-31T06:13:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.695+0000 7fd4d59f0740 1 --2- 172.21.15.16:0/2827739949 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55b314e69d80 0x55b3152744b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.695+0000 7fd4d59f0740 4 client.27041 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:13:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.695+0000 7fd4d59f0740 1 -- 172.21.15.16:0/2827739949 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3151c5ec0 con 0x55b314e69d80 2024-08-31T06:13:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.695+0000 7fd4bf400640 1 --2- 172.21.15.16:0/2827739949 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55b314e69d80 0x55b3152744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.696+0000 7fd4bf400640 1 --2- 172.21.15.16:0/2827739949 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55b314e69d80 0x55b3152744b0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47758ca5d0ac0 server_cookie=78953ac0b83582b9 in_seq=0 out_seq=0 2024-08-31T06:13:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.696+0000 7fd4b5600640 10 client.27041 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:13:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.697+0000 7fd4b5600640 1 -- 172.21.15.16:0/2827739949 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b3151c5ec0 con 0x55b314e69d80 2024-08-31T06:13:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.697+0000 7fd4b5600640 10 client.27041 handle_command_reply: tid=0 2024-08-31T06:13:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.697+0000 7fd4d59f0740 1 client.27041 shutdown 2024-08-31T06:13:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.697+0000 7fd4d59f0740 1 -- 172.21.15.16:0/2827739949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b314e822f0 msgr2=0x55b315259cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.697+0000 7fd4d59f0740 1 --2- 172.21.15.16:0/2827739949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b314e822f0 0x55b315259cb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd4b00094d0 tx=0x7fd4b0009930 comp rx=0 tx=0).stop 2024-08-31T06:13:51.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.697+0000 7fd4d59f0740 1 -- 172.21.15.16:0/2827739949 shutdown_connections 2024-08-31T06:13:51.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.697+0000 7fd4d59f0740 1 --2- 172.21.15.16:0/2827739949 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55b314e69d80 0x55b3152744b0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.697+0000 7fd4d59f0740 1 --2- 172.21.15.16:0/2827739949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b314eab1b0 0x55b31525a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.697+0000 7fd4d59f0740 1 --2- 172.21.15.16:0/2827739949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b314e8a4d0 0x55b31525a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.697+0000 7fd4d59f0740 1 --2- 172.21.15.16:0/2827739949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b314e822f0 0x55b315259cb0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.697+0000 7fd4d59f0740 1 -- 172.21.15.16:0/2827739949 >> 172.21.15.16:0/2827739949 conn(0x55b314e9f300 msgr2=0x55b31524f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.698+0000 7fd4b5600640 0 client.27041 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:13:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.698+0000 7fd4d59f0740 1 -- 172.21.15.16:0/2827739949 shutdown_connections 2024-08-31T06:13:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.698+0000 7fd4d59f0740 1 -- 172.21.15.16:0/2827739949 wait complete. 2024-08-31T06:13:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.698+0000 7fd4d59f0740 20 client.27041 trim_cache size 0 max 16384 2024-08-31T06:13:51.725 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.723+0000 7fd4d59f0740 1 Processor -- start 2024-08-31T06:13:51.725 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.723+0000 7fd4d59f0740 1 -- start start 2024-08-31T06:13:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.724+0000 7fd4d59f0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b314e822f0 0x55b315250080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.724+0000 7fd4d59f0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b314e8a4d0 0x55b31525b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.724+0000 7fd4d59f0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b314eab1b0 0x55b315256e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.724+0000 7fd4d59f0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c81afa90 con 0x55b314eab1b0 2024-08-31T06:13:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.724+0000 7fd4d59f0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c81ad2d0 con 0x55b314e8a4d0 2024-08-31T06:13:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.724+0000 7fd4d59f0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c81ab140 con 0x55b314e822f0 2024-08-31T06:13:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.725+0000 7fd4bf400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b314e8a4d0 0x55b31525b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.725+0000 7fd4c7e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b314e822f0 0x55b315250080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.725+0000 7fd4bf400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b314e8a4d0 0x55b31525b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:35214/0 (socket says 172.21.15.16:35214) 2024-08-31T06:13:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.725+0000 7fd4bf400640 1 -- 172.21.15.16:0/2058737973 learned_addr learned my addr 172.21.15.16:0/2058737973 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.725+0000 7fd4cca00640 1 --2- 172.21.15.16:0/2058737973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b314eab1b0 0x55b315256e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.725+0000 7fd4c7e00640 1 -- 172.21.15.16:0/2058737973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b314e8a4d0 msgr2=0x55b31525b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.725+0000 7fd4c7e00640 1 --2- 172.21.15.16:0/2058737973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b314e8a4d0 0x55b31525b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.725+0000 7fd4c7e00640 1 -- 172.21.15.16:0/2058737973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b314eab1b0 msgr2=0x55b315256e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.725+0000 7fd4c7e00640 1 --2- 172.21.15.16:0/2058737973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b314eab1b0 0x55b315256e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.726+0000 7fd4c7e00640 1 -- 172.21.15.16:0/2058737973 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b314e69fb0 con 0x55b314e822f0 2024-08-31T06:13:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.726+0000 7fd4c7e00640 1 --2- 172.21.15.16:0/2058737973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b314e822f0 0x55b315250080 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd4b00094d0 tx=0x7fd4b0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed0dff5bafbb683e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.726+0000 7fd4b7e00640 1 -- 172.21.15.16:0/2058737973 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b0027040 con 0x55b314e822f0 2024-08-31T06:13:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.726+0000 7fd4b7e00640 1 -- 172.21.15.16:0/2058737973 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4b000e880 con 0x55b314e822f0 2024-08-31T06:13:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.726+0000 7fd4b7e00640 1 -- 172.21.15.16:0/2058737973 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b001e040 con 0x55b314e822f0 2024-08-31T06:13:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.727+0000 7fd4d59f0740 1 -- 172.21.15.16:0/2058737973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b314e822f0 msgr2=0x55b315250080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.727+0000 7fd4d59f0740 1 --2- 172.21.15.16:0/2058737973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b314e822f0 0x55b315250080 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd4b00094d0 tx=0x7fd4b0011810 comp rx=0 tx=0).stop 2024-08-31T06:13:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.728+0000 7fd4d59f0740 1 -- 172.21.15.16:0/2058737973 shutdown_connections 2024-08-31T06:13:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.728+0000 7fd4d59f0740 1 --2- 172.21.15.16:0/2058737973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b314eab1b0 0x55b315256e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.728+0000 7fd4d59f0740 1 --2- 172.21.15.16:0/2058737973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b314e8a4d0 0x55b31525b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.728+0000 7fd4d59f0740 1 --2- 172.21.15.16:0/2058737973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b314e822f0 0x55b315250080 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.728+0000 7fd4d59f0740 1 -- 172.21.15.16:0/2058737973 >> 172.21.15.16:0/2058737973 conn(0x55b314e9f300 msgr2=0x55b31525b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.728+0000 7fd4d59f0740 1 -- 172.21.15.16:0/2058737973 shutdown_connections 2024-08-31T06:13:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.728+0000 7fd4d59f0740 1 -- 172.21.15.16:0/2058737973 wait complete. 2024-08-31T06:13:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.729+0000 7fd4d59f0740 1 -- start start 2024-08-31T06:13:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.729+0000 7fd4d59f0740 1 Processor -- start 2024-08-31T06:13:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.730+0000 7fd4d59f0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b314e822f0 0x55b31525f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.730+0000 7fd4d59f0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b314e8a4d0 0x55b31525fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.730+0000 7fd4d59f0740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b314eab1b0 0x55b3152dccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.730+0000 7fd4d59f0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c81ab790 con 0x55b314e822f0 2024-08-31T06:13:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.730+0000 7fd4d59f0740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c81acc00 con 0x55b314eab1b0 2024-08-31T06:13:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.730+0000 7fd4d59f0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c81abe60 con 0x55b314e8a4d0 2024-08-31T06:13:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.730+0000 7fd4d59f0740 10 client.0 _ll_register_callbacks cb 0x7fd4b001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:13:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.731+0000 7fd4bf400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b314e8a4d0 0x55b31525fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.731+0000 7fd4bf400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b314e8a4d0 0x55b31525fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:58362/0 (socket says 172.21.15.16:58362) 2024-08-31T06:13:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.731+0000 7fd4bf400640 1 -- 172.21.15.16:0/3968037386 learned_addr learned my addr 172.21.15.16:0/3968037386 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:13:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.731+0000 7fd4cca00640 1 --2- 172.21.15.16:0/3968037386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b314eab1b0 0x55b3152dccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.731+0000 7fd4c7e00640 1 --2- 172.21.15.16:0/3968037386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b314e822f0 0x55b31525f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.732+0000 7fd4bf400640 1 -- 172.21.15.16:0/3968037386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b314eab1b0 msgr2=0x55b3152dccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.732+0000 7fd4bf400640 1 --2- 172.21.15.16:0/3968037386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b314eab1b0 0x55b3152dccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.732+0000 7fd4bf400640 1 -- 172.21.15.16:0/3968037386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b314e822f0 msgr2=0x55b31525f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:13:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.732+0000 7fd4bf400640 1 --2- 172.21.15.16:0/3968037386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b314e822f0 0x55b31525f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.732+0000 7fd4bf400640 1 -- 172.21.15.16:0/3968037386 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3152dd1f0 con 0x55b314e8a4d0 2024-08-31T06:13:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.732+0000 7fd4c7e00640 1 --2- 172.21.15.16:0/3968037386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b314e822f0 0x55b31525f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:13:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.732+0000 7fd4bf400640 1 --2- 172.21.15.16:0/3968037386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b314e8a4d0 0x55b31525fca0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd4b800fca0 tx=0x7fd4b800a720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d93601d9ec92864f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:13:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.732+0000 7fd4b5600640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:13:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.733+0000 7fd4b5600640 1 -- 172.21.15.16:0/3968037386 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8037060 con 0x55b314e8a4d0 2024-08-31T06:13:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.733+0000 7fd4b5600640 1 -- 172.21.15.16:0/3968037386 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8470+0+0 (secure 0 0 0) 0x7fd4b8020b20 con 0x55b314e8a4d0 2024-08-31T06:13:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.733+0000 7fd4d59f0740 1 -- 172.21.15.16:0/3968037386 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4b000e880 con 0x55b314e8a4d0 2024-08-31T06:13:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.734+0000 7fd4b5600640 1 -- 172.21.15.16:0/3968037386 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fd4b8020d20 con 0x55b314e8a4d0 2024-08-31T06:13:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.734+0000 7fd4d59f0740 10 client.27037 fetch_fsmap learned FSMap version 38 2024-08-31T06:13:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.734+0000 7fd4d59f0740 1 -- 172.21.15.16:0/3968037386 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55b3152de4f0 con 0x55b314e8a4d0 2024-08-31T06:13:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.735+0000 7fd4b5600640 1 -- 172.21.15.16:0/3968037386 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4b000e880 con 0x55b314e8a4d0 2024-08-31T06:13:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.735+0000 7fd4b5600640 1 -- 172.21.15.16:0/3968037386 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7fd4b801f400 con 0x55b314e8a4d0 2024-08-31T06:13:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.735+0000 7fd4d59f0740 10 client.27037 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T06:13:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.735+0000 7fd4d59f0740 10 client.27037 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:13:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.735+0000 7fd4d59f0740 20 client.27037 populate_metadata read hostname 'smithi016' 2024-08-31T06:13:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.735+0000 7fd4d59f0740 1 --2- 172.21.15.16:0/3968037386 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55b314e3f1c0 0x55b3152e0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:13:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.735+0000 7fd4d59f0740 4 client.27037 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:13:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.735+0000 7fd4d59f0740 1 -- 172.21.15.16:0/3968037386 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b3151c1960 con 0x55b314e3f1c0 2024-08-31T06:13:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.735+0000 7fd4c7e00640 1 --2- 172.21.15.16:0/3968037386 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55b314e3f1c0 0x55b3152e0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:13:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.736+0000 7fd4b5600640 10 client.27037 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:13:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.737+0000 7fd4c7e00640 1 --2- 172.21.15.16:0/3968037386 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55b314e3f1c0 0x55b3152e0d70 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39e84ee11d84f062 server_cookie=567334fcf16c2768 in_seq=0 out_seq=0 2024-08-31T06:13:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.760+0000 7fd4b5600640 1 -- 172.21.15.16:0/3968037386 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b3151c1960 con 0x55b314e3f1c0 2024-08-31T06:13:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.760+0000 7fd4b5600640 10 client.27037 handle_command_reply: tid=0 2024-08-31T06:13:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.761+0000 7fd4d59f0740 1 client.27037 shutdown 2024-08-31T06:13:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.761+0000 7fd4d59f0740 1 -- 172.21.15.16:0/3968037386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b314e8a4d0 msgr2=0x55b31525fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.761+0000 7fd4d59f0740 1 --2- 172.21.15.16:0/3968037386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b314e8a4d0 0x55b31525fca0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd4b800fca0 tx=0x7fd4b800a720 comp rx=0 tx=0).stop 2024-08-31T06:13:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.761+0000 7fd4d59f0740 1 -- 172.21.15.16:0/3968037386 shutdown_connections 2024-08-31T06:13:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.761+0000 7fd4d59f0740 1 --2- 172.21.15.16:0/3968037386 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55b314e3f1c0 0x55b3152e0d70 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.761+0000 7fd4d59f0740 1 --2- 172.21.15.16:0/3968037386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b314e8a4d0 0x55b31525fca0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.761+0000 7fd4d59f0740 1 --2- 172.21.15.16:0/3968037386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b314eab1b0 0x55b3152dccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.761+0000 7fd4d59f0740 1 --2- 172.21.15.16:0/3968037386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b314e822f0 0x55b31525f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.762+0000 7fd4d59f0740 1 -- 172.21.15.16:0/3968037386 >> 172.21.15.16:0/3968037386 conn(0x55b314e9f300 msgr2=0x55b31524fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.762+0000 7fd4b5600640 0 client.27037 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:13:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.762+0000 7fd4d59f0740 1 -- 172.21.15.16:0/3968037386 shutdown_connections 2024-08-31T06:13:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.762+0000 7fd4d59f0740 1 -- 172.21.15.16:0/3968037386 wait complete. 2024-08-31T06:13:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.762+0000 7fd4d59f0740 20 client.27037 trim_cache size 0 max 16384 2024-08-31T06:13:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:13:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:13:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:13:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:13:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.763+0000 7fd4cea00640 1 -- 172.21.15.16:0/3636933903 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd4ac0b4f50 msgr2=0x7fd4ac0b7460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.763+0000 7fd4cea00640 1 --2- 172.21.15.16:0/3636933903 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd4ac0b4f50 0x7fd4ac0b7460 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd4b80058c0 tx=0x7fd4b800a380 comp rx=0 tx=0).stop 2024-08-31T06:13:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.763+0000 7fd4cea00640 1 -- 172.21.15.16:0/3636933903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4c81089d0 msgr2=0x7fd4c819d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:13:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.763+0000 7fd4cea00640 1 --2- 172.21.15.16:0/3636933903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4c81089d0 0x7fd4c819d7d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0036040 tx=0x7fd4c0003cc0 comp rx=0 tx=0).stop 2024-08-31T06:13:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.764+0000 7fd4cea00640 1 -- 172.21.15.16:0/3636933903 shutdown_connections 2024-08-31T06:13:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.764+0000 7fd4cea00640 1 --2- 172.21.15.16:0/3636933903 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd4ac0b4f50 0x7fd4ac0b7460 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.764+0000 7fd4cea00640 1 --2- 172.21.15.16:0/3636933903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4c81089d0 0x7fd4c819d7d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.764+0000 7fd4cea00640 1 --2- 172.21.15.16:0/3636933903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4c8108080 0x7fd4c8198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.764+0000 7fd4cea00640 1 --2- 172.21.15.16:0/3636933903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4c8102750 0x7fd4c8197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:13:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.764+0000 7fd4cea00640 1 -- 172.21.15.16:0/3636933903 >> 172.21.15.16:0/3636933903 conn(0x7fd4c807a4e0 msgr2=0x7fd4c81040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:13:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.764+0000 7fd4cea00640 1 -- 172.21.15.16:0/3636933903 shutdown_connections 2024-08-31T06:13:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:13:51.765+0000 7fd4cea00640 1 -- 172.21.15.16:0/3636933903 wait complete. 2024-08-31T06:13:55.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:54 smithi155 ceph-mon[21742]: Adjusting osd_memory_target on smithi184 to 176.0M 2024-08-31T06:13:55.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:54 smithi155 ceph-mon[21742]: Adjusting osd_memory_target on smithi155 to 176.0M 2024-08-31T06:13:55.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:54 smithi155 ceph-mon[21742]: Unable to set osd_memory_target on smithi184 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-08-31T06:13:55.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:54 smithi155 ceph-mon[21742]: Unable to set osd_memory_target on smithi155 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-08-31T06:13:55.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:54 smithi155 ceph-mon[21742]: pgmap v565: 97 pgs: 97 active+clean; 3.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.0 MiB/s wr, 608 op/s 2024-08-31T06:13:55.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:54 smithi155 ceph-mon[21742]: pgmap v566: 97 pgs: 97 active+clean; 3.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.7 MiB/s wr, 647 op/s 2024-08-31T06:13:55.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:54 smithi016 ceph-mon[17194]: Adjusting osd_memory_target on smithi184 to 176.0M 2024-08-31T06:13:55.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:54 smithi016 ceph-mon[17194]: Adjusting osd_memory_target on smithi155 to 176.0M 2024-08-31T06:13:55.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:54 smithi016 ceph-mon[17194]: Unable to set osd_memory_target on smithi184 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-08-31T06:13:55.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:54 smithi016 ceph-mon[17194]: Unable to set osd_memory_target on smithi155 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-08-31T06:13:55.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:54 smithi016 ceph-mon[17194]: pgmap v565: 97 pgs: 97 active+clean; 3.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.0 MiB/s wr, 608 op/s 2024-08-31T06:13:55.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:54 smithi016 ceph-mon[17194]: pgmap v566: 97 pgs: 97 active+clean; 3.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.7 MiB/s wr, 647 op/s 2024-08-31T06:13:55.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:54 smithi184 ceph-mon[22331]: Adjusting osd_memory_target on smithi184 to 176.0M 2024-08-31T06:13:55.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:54 smithi184 ceph-mon[22331]: Adjusting osd_memory_target on smithi155 to 176.0M 2024-08-31T06:13:55.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:54 smithi184 ceph-mon[22331]: Unable to set osd_memory_target on smithi184 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-08-31T06:13:55.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:54 smithi184 ceph-mon[22331]: Unable to set osd_memory_target on smithi155 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-08-31T06:13:55.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:54 smithi184 ceph-mon[22331]: pgmap v565: 97 pgs: 97 active+clean; 3.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.0 MiB/s wr, 608 op/s 2024-08-31T06:13:55.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:54 smithi184 ceph-mon[22331]: pgmap v566: 97 pgs: 97 active+clean; 3.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.7 MiB/s wr, 647 op/s 2024-08-31T06:13:56.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 390.0 s, 2873.5 tps, lat 34.806 ms stddev 39.824 2024-08-31T06:13:56.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:56 smithi184 ceph-mon[22331]: pgmap v567: 97 pgs: 97 active+clean; 3.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 11 MiB/s wr, 784 op/s 2024-08-31T06:13:57.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:56 smithi155 ceph-mon[21742]: pgmap v567: 97 pgs: 97 active+clean; 3.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 11 MiB/s wr, 784 op/s 2024-08-31T06:13:57.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:56 smithi016 ceph-mon[17194]: pgmap v567: 97 pgs: 97 active+clean; 3.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 11 MiB/s wr, 784 op/s 2024-08-31T06:13:58.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:13:58 smithi184 ceph-mon[22331]: pgmap v568: 97 pgs: 97 active+clean; 3.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.2 MiB/s wr, 637 op/s 2024-08-31T06:13:59.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:13:58 smithi155 ceph-mon[21742]: pgmap v568: 97 pgs: 97 active+clean; 3.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.2 MiB/s wr, 637 op/s 2024-08-31T06:13:59.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:13:58 smithi016 ceph-mon[17194]: pgmap v568: 97 pgs: 97 active+clean; 3.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.2 MiB/s wr, 637 op/s 2024-08-31T06:14:00.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:00 smithi184 ceph-mon[22331]: pgmap v569: 97 pgs: 97 active+clean; 3.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 9.4 MiB/s wr, 730 op/s 2024-08-31T06:14:01.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:00 smithi155 ceph-mon[21742]: pgmap v569: 97 pgs: 97 active+clean; 3.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 9.4 MiB/s wr, 730 op/s 2024-08-31T06:14:01.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:00 smithi016 ceph-mon[17194]: pgmap v569: 97 pgs: 97 active+clean; 3.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 9.4 MiB/s wr, 730 op/s 2024-08-31T06:14:01.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 395.0 s, 1626.2 tps, lat 57.318 ms stddev 77.479 2024-08-31T06:14:01.820 INFO:tasks.quiescer.fs.[cephfs]:Total duration (56.0) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-08-31T06:14:01.820 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 28.0 2024-08-31T06:14:01.821 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:14:02.575 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:14:02.838 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T06:14:02.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:02 smithi184 ceph-mon[22331]: pgmap v570: 97 pgs: 97 active+clean; 3.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.1 MiB/s wr, 654 op/s 2024-08-31T06:14:03.053 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:02 smithi016 ceph-mon[17194]: pgmap v570: 97 pgs: 97 active+clean; 3.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.1 MiB/s wr, 654 op/s 2024-08-31T06:14:03.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:02 smithi155 ceph-mon[21742]: pgmap v570: 97 pgs: 97 active+clean; 3.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.1 MiB/s wr, 654 op/s 2024-08-31T06:14:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:03.553+0000 7fed7ca00640 1 -- 172.21.15.16:0/2464772643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed7810d990 msgr2=0x7fed7810dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:03.553+0000 7fed7ca00640 1 --2- 172.21.15.16:0/2464772643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed7810d990 0x7fed7810dda0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fed64009900 tx=0x7fed6402a410 comp rx=0 tx=0).stop 2024-08-31T06:14:03.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:03.553+0000 7fed7ca00640 1 -- 172.21.15.16:0/2464772643 shutdown_connections 2024-08-31T06:14:03.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:03.553+0000 7fed7ca00640 1 --2- 172.21.15.16:0/2464772643 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed7811c940 0x7fed7811ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:03.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:03.553+0000 7fed7ca00640 1 --2- 172.21.15.16:0/2464772643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed7810d990 0x7fed7810dda0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:03.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:03.553+0000 7fed7ca00640 1 --2- 172.21.15.16:0/2464772643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed78111a70 0x7fed7810d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:03.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:03.553+0000 7fed7ca00640 1 -- 172.21.15.16:0/2464772643 >> 172.21.15.16:0/2464772643 conn(0x7fed7806e790 msgr2=0x7fed7806ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:03.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:03.554+0000 7fed7ca00640 1 -- 172.21.15.16:0/2464772643 shutdown_connections 2024-08-31T06:14:03.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:03.554+0000 7fed7ca00640 1 -- 172.21.15.16:0/2464772643 wait complete. 2024-08-31T06:14:03.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:03.555+0000 7fed7ca00640 1 Processor -- start 2024-08-31T06:14:03.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:03.555+0000 7fed7ca00640 1 -- start start 2024-08-31T06:14:03.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:03.556+0000 7fed7ca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed7810d990 0x7fed7811c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:03.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:03.556+0000 7fed7ca00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed78111a70 0x7fed781154f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:03.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:03.556+0000 7fed7ca00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed7811c940 0x7fed78115ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:03.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:03.556+0000 7fed7ca00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fed78122470 con 0x7fed7810d990 2024-08-31T06:14:03.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:03.556+0000 7fed7ca00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fed781222f0 con 0x7fed78111a70 2024-08-31T06:14:03.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:03.556+0000 7fed7ca00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fed781225f0 con 0x7fed7811c940 2024-08-31T06:14:03.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:03.556+0000 7fed77200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed7811c940 0x7fed78115ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:03.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:03.556+0000 7fed77200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed7811c940 0x7fed78115ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:52506/0 (socket says 172.21.15.16:52506) 2024-08-31T06:14:03.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:03.556+0000 7fed77200640 1 -- 172.21.15.16:0/1143596115 learned_addr learned my addr 172.21.15.16:0/1143596115 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:03.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:03.557+0000 7fed75e00640 1 --2- 172.21.15.16:0/1143596115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed78111a70 0x7fed781154f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:03.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:03.557+0000 7fed77200640 1 -- 172.21.15.16:0/1143596115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed78111a70 msgr2=0x7fed781154f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:03.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:03.557+0000 7fed77200640 1 --2- 172.21.15.16:0/1143596115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed78111a70 0x7fed781154f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:03.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:03.557+0000 7fed77200640 1 -- 172.21.15.16:0/1143596115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed7810d990 msgr2=0x7fed7811c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:03.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:03.557+0000 7fed77200640 1 --2- 172.21.15.16:0/1143596115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed7810d990 0x7fed7811c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:03.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:03.557+0000 7fed77200640 1 -- 172.21.15.16:0/1143596115 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed78116300 con 0x7fed7811c940 2024-08-31T06:14:03.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:03.557+0000 7fed77200640 1 --2- 172.21.15.16:0/1143596115 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed7811c940 0x7fed78115ae0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fed7000eaa0 tx=0x7fed7000ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:03.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:03.558+0000 7fed6f400640 1 -- 172.21.15.16:0/1143596115 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7000cdb0 con 0x7fed7811c940 2024-08-31T06:14:03.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:03.558+0000 7fed7ca00640 1 -- 172.21.15.16:0/1143596115 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed7811afb0 con 0x7fed7811c940 2024-08-31T06:14:03.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:03.558+0000 7fed7ca00640 1 -- 172.21.15.16:0/1143596115 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed7811b510 con 0x7fed7811c940 2024-08-31T06:14:03.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:03.559+0000 7fed6f400640 1 -- 172.21.15.16:0/1143596115 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed70005020 con 0x7fed7811c940 2024-08-31T06:14:03.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:03.559+0000 7fed6f400640 1 -- 172.21.15.16:0/1143596115 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed700116d0 con 0x7fed7811c940 2024-08-31T06:14:03.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:03.560+0000 7fed7ca00640 1 -- 172.21.15.16:0/1143596115 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed580052c0 con 0x7fed7811c940 2024-08-31T06:14:03.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:03.565+0000 7fed6f400640 1 -- 172.21.15.16:0/1143596115 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fed70004a40 con 0x7fed7811c940 2024-08-31T06:14:03.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:03.567+0000 7fed6f400640 1 --2- 172.21.15.16:0/1143596115 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fed5c0b4da0 0x7fed5c0b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:03.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:03.568+0000 7fed6f400640 1 -- 172.21.15.16:0/1143596115 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8470+0+0 (secure 0 0 0) 0x7fed700e59d0 con 0x7fed7811c940 2024-08-31T06:14:03.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:03.569+0000 7fed76800640 1 --2- 172.21.15.16:0/1143596115 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fed5c0b4da0 0x7fed5c0b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:03.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:03.570+0000 7fed6f400640 1 -- 172.21.15.16:0/1143596115 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fed70016030 con 0x7fed7811c940 2024-08-31T06:14:03.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:03.571+0000 7fed76800640 1 --2- 172.21.15.16:0/1143596115 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fed5c0b4da0 0x7fed5c0b72b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fed68005fd0 tx=0x7fed6800b410 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:03.634 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:14:04.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.001+0000 7fed7ca00640 1 -- 172.21.15.16:0/1143596115 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed580055b0 con 0x7fed7811c940 2024-08-31T06:14:04.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.003+0000 7fed6f400640 1 -- 172.21.15.16:0/1143596115 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11921 (secure 0 0 0) 0x7fed700ac810 con 0x7fed7811c940 2024-08-31T06:14:04.009 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:14:04.009 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":38,"btime":"2024-08-31T06:12:33:984731+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:12:33.336879+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24712,24724,24737]},"id":1}]} 2024-08-31T06:14:04.010 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 38 2024-08-31T06:14:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.007+0000 7fed7ca00640 1 -- 172.21.15.16:0/1143596115 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fed5c0b4da0 msgr2=0x7fed5c0b72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.007+0000 7fed7ca00640 1 --2- 172.21.15.16:0/1143596115 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fed5c0b4da0 0x7fed5c0b72b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fed68005fd0 tx=0x7fed6800b410 comp rx=0 tx=0).stop 2024-08-31T06:14:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.007+0000 7fed7ca00640 1 -- 172.21.15.16:0/1143596115 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed7811c940 msgr2=0x7fed78115ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.007+0000 7fed7ca00640 1 --2- 172.21.15.16:0/1143596115 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed7811c940 0x7fed78115ae0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fed7000eaa0 tx=0x7fed7000ef60 comp rx=0 tx=0).stop 2024-08-31T06:14:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.008+0000 7fed7ca00640 1 -- 172.21.15.16:0/1143596115 shutdown_connections 2024-08-31T06:14:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.008+0000 7fed7ca00640 1 --2- 172.21.15.16:0/1143596115 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fed5c0b4da0 0x7fed5c0b72b0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.008+0000 7fed7ca00640 1 --2- 172.21.15.16:0/1143596115 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed7811c940 0x7fed78115ae0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:04.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.008+0000 7fed7ca00640 1 --2- 172.21.15.16:0/1143596115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed78111a70 0x7fed781154f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:04.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.008+0000 7fed7ca00640 1 --2- 172.21.15.16:0/1143596115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed7810d990 0x7fed7811c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:04.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.008+0000 7fed7ca00640 1 -- 172.21.15.16:0/1143596115 >> 172.21.15.16:0/1143596115 conn(0x7fed7806e790 msgr2=0x7fed78110720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:04.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.008+0000 7fed7ca00640 1 -- 172.21.15.16:0/1143596115 shutdown_connections 2024-08-31T06:14:04.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.009+0000 7fed7ca00640 1 -- 172.21.15.16:0/1143596115 wait complete. 2024-08-31T06:14:04.783 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:04 smithi016 ceph-mon[17194]: pgmap v571: 97 pgs: 97 active+clean; 3.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 12 MiB/s wr, 884 op/s 2024-08-31T06:14:04.783 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:04 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/1143596115' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:14:04.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.815+0000 7f7c49000640 1 -- 172.21.15.16:0/3244724399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c441145a0 msgr2=0x7f7c4411add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:04.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.816+0000 7f7c49000640 1 --2- 172.21.15.16:0/3244724399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c441145a0 0x7f7c4411add0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c009ca0 tx=0x7f7c3c02a8f0 comp rx=0 tx=0).stop 2024-08-31T06:14:04.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.817+0000 7f7c49000640 1 -- 172.21.15.16:0/3244724399 shutdown_connections 2024-08-31T06:14:04.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.818+0000 7f7c49000640 1 --2- 172.21.15.16:0/3244724399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c441145a0 0x7f7c4411add0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:04.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.818+0000 7f7c49000640 1 --2- 172.21.15.16:0/3244724399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c44113c50 0x7f7c44114060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:04.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.818+0000 7f7c49000640 1 --2- 172.21.15.16:0/3244724399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c4410e360 0x7f7c4410e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:04.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.818+0000 7f7c49000640 1 -- 172.21.15.16:0/3244724399 >> 172.21.15.16:0/3244724399 conn(0x7f7c4410a160 msgr2=0x7f7c4410c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:04.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.819+0000 7f7c49000640 1 -- 172.21.15.16:0/3244724399 shutdown_connections 2024-08-31T06:14:04.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.819+0000 7f7c49000640 1 -- 172.21.15.16:0/3244724399 wait complete. 2024-08-31T06:14:04.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.823+0000 7f7c49000640 1 Processor -- start 2024-08-31T06:14:04.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.823+0000 7f7c49000640 1 -- start start 2024-08-31T06:14:04.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.824+0000 7f7c49000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c4410e360 0x7f7c441a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:04.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.825+0000 7f7c49000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c44113c50 0x7f7c441a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:04.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.826+0000 7f7c41400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c44113c50 0x7f7c441a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:04.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.826+0000 7f7c41400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c44113c50 0x7f7c441a51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50464/0 (socket says 172.21.15.16:50464) 2024-08-31T06:14:04.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.826+0000 7f7c41400640 1 -- 172.21.15.16:0/2924885710 learned_addr learned my addr 172.21.15.16:0/2924885710 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:04.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.827+0000 7f7c49000640 1 --2- 172.21.15.16:0/2924885710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c441145a0 0x7f7c441aa490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:04.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.828+0000 7f7c49000640 1 -- 172.21.15.16:0/2924885710 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4411e2c0 con 0x7f7c44113c50 2024-08-31T06:14:04.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.828+0000 7f7c49000640 1 -- 172.21.15.16:0/2924885710 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4411e140 con 0x7f7c4410e360 2024-08-31T06:14:04.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.828+0000 7f7c49000640 1 -- 172.21.15.16:0/2924885710 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4411e440 con 0x7f7c441145a0 2024-08-31T06:14:04.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.828+0000 7f7c42800640 1 --2- 172.21.15.16:0/2924885710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c441145a0 0x7f7c441aa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:04.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.829+0000 7f7c42800640 1 -- 172.21.15.16:0/2924885710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c4410e360 msgr2=0x7f7c441a4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:04.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.829+0000 7f7c41e00640 1 --2- 172.21.15.16:0/2924885710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c4410e360 0x7f7c441a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:04.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.829+0000 7f7c42800640 1 --2- 172.21.15.16:0/2924885710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c4410e360 0x7f7c441a4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:04.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.830+0000 7f7c42800640 1 -- 172.21.15.16:0/2924885710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c44113c50 msgr2=0x7f7c441a51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:04.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.830+0000 7f7c42800640 1 --2- 172.21.15.16:0/2924885710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c44113c50 0x7f7c441a51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:04.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.830+0000 7f7c42800640 1 -- 172.21.15.16:0/2924885710 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c441aabc0 con 0x7f7c441145a0 2024-08-31T06:14:04.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.831+0000 7f7c41e00640 1 --2- 172.21.15.16:0/2924885710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c4410e360 0x7f7c441a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:14:04.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.831+0000 7f7c42800640 1 --2- 172.21.15.16:0/2924885710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c441145a0 0x7f7c441aa490 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c009bf0 tx=0x7f7c3c031f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:04.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.832+0000 7f7c3aa00640 1 -- 172.21.15.16:0/2924885710 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c030b40 con 0x7f7c441145a0 2024-08-31T06:14:04.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.833+0000 7f7c3aa00640 1 -- 172.21.15.16:0/2924885710 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c3c02d070 con 0x7f7c441145a0 2024-08-31T06:14:04.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.833+0000 7f7c3aa00640 1 -- 172.21.15.16:0/2924885710 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c03a400 con 0x7f7c441145a0 2024-08-31T06:14:04.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.833+0000 7f7c49000640 1 -- 172.21.15.16:0/2924885710 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c441aae50 con 0x7f7c441145a0 2024-08-31T06:14:04.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.835+0000 7f7c49000640 1 -- 172.21.15.16:0/2924885710 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c441b1610 con 0x7f7c441145a0 2024-08-31T06:14:04.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.859+0000 7f7c4fcc8740 1 Processor -- start 2024-08-31T06:14:04.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.860+0000 7f7c4fcc8740 1 -- start start 2024-08-31T06:14:04.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.862+0000 7f7c4fcc8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d342aa3070 0x55d342e51160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:04.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.862+0000 7f7c4fcc8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d342a7a1b0 0x55d342e516a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:04.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.862+0000 7f7c4fcc8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d342a82390 0x55d342e51be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:04.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.862+0000 7f7c4fcc8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d342dbf770 con 0x55d342a82390 2024-08-31T06:14:04.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.862+0000 7f7c4fcc8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d342daed00 con 0x55d342aa3070 2024-08-31T06:14:04.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.862+0000 7f7c4fcc8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d342db2cb0 con 0x55d342a7a1b0 2024-08-31T06:14:04.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.862+0000 7f7c41e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d342a7a1b0 0x55d342e516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:04.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.863+0000 7f7c41e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d342a7a1b0 0x55d342e516a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:52524/0 (socket says 172.21.15.16:52524) 2024-08-31T06:14:04.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.863+0000 7f7c41e00640 1 -- 172.21.15.16:0/265899061 learned_addr learned my addr 172.21.15.16:0/265899061 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:04.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.863+0000 7f7c41400640 1 --2- 172.21.15.16:0/265899061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d342aa3070 0x55d342e51160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:04.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.863+0000 7f7c41e00640 1 -- 172.21.15.16:0/265899061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d342aa3070 msgr2=0x55d342e51160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:04.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.863+0000 7f7c41e00640 1 --2- 172.21.15.16:0/265899061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d342aa3070 0x55d342e51160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:04.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.863+0000 7f7c41400640 1 --2- 172.21.15.16:0/265899061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d342a82390 0x55d342e51be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:04.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.863+0000 7f7c41e00640 1 -- 172.21.15.16:0/265899061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d342a82390 msgr2=0x55d342e51be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:04.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.863+0000 7f7c41e00640 1 --2- 172.21.15.16:0/265899061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d342a82390 0x55d342e51be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:04.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.863+0000 7f7c41e00640 1 -- 172.21.15.16:0/265899061 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d342b0e650 con 0x55d342a7a1b0 2024-08-31T06:14:04.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.864+0000 7f7c41400640 1 --2- 172.21.15.16:0/265899061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d342a82390 0x55d342e51be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:14:04.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.864+0000 7f7c41e00640 1 --2- 172.21.15.16:0/265899061 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d342a7a1b0 0x55d342e516a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7c3400aa10 tx=0x7f7c3400aed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb00054db0bb919e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:04.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.864+0000 7f7c2f400640 1 -- 172.21.15.16:0/265899061 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c34015070 con 0x55d342a7a1b0 2024-08-31T06:14:04.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.864+0000 7f7c2f400640 1 -- 172.21.15.16:0/265899061 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c3400cc60 con 0x55d342a7a1b0 2024-08-31T06:14:04.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.864+0000 7f7c2f400640 1 -- 172.21.15.16:0/265899061 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c34010590 con 0x55d342a7a1b0 2024-08-31T06:14:04.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.865+0000 7f7c3aa00640 1 -- 172.21.15.16:0/2924885710 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f7c3c03b400 con 0x7f7c441145a0 2024-08-31T06:14:04.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.866+0000 7f7c4fcc8740 1 -- 172.21.15.16:0/265899061 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d342a7a1b0 msgr2=0x55d342e516a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:04.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.866+0000 7f7c4fcc8740 1 --2- 172.21.15.16:0/265899061 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d342a7a1b0 0x55d342e516a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7c3400aa10 tx=0x7f7c3400aed0 comp rx=0 tx=0).stop 2024-08-31T06:14:04.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.867+0000 7f7c4fcc8740 1 -- 172.21.15.16:0/265899061 shutdown_connections 2024-08-31T06:14:04.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.867+0000 7f7c4fcc8740 1 --2- 172.21.15.16:0/265899061 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d342a7a1b0 0x55d342e516a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:04.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.867+0000 7f7c4fcc8740 1 --2- 172.21.15.16:0/265899061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d342a82390 0x55d342e51be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:04.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.867+0000 7f7c4fcc8740 1 --2- 172.21.15.16:0/265899061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d342aa3070 0x55d342e51160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:04.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.867+0000 7f7c4fcc8740 1 -- 172.21.15.16:0/265899061 >> 172.21.15.16:0/265899061 conn(0x55d342a97460 msgr2=0x55d342e4f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:04.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.867+0000 7f7c4fcc8740 1 -- 172.21.15.16:0/265899061 shutdown_connections 2024-08-31T06:14:04.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.868+0000 7f7c4fcc8740 1 -- 172.21.15.16:0/265899061 wait complete. 2024-08-31T06:14:04.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.869+0000 7f7c4fcc8740 1 -- start start 2024-08-31T06:14:04.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.870+0000 7f7c4fcc8740 1 Processor -- start 2024-08-31T06:14:04.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.871+0000 7f7c4fcc8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d342a7a1b0 0x55d342e5cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:04.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.871+0000 7f7c4fcc8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d342a82390 0x55d342e5d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:04.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.871+0000 7f7c4fcc8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d342aa3070 0x55d342e68c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:04.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.871+0000 7f7c4fcc8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d342c11700 con 0x55d342aa3070 2024-08-31T06:14:04.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.871+0000 7f7c4fcc8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d342d35cb0 con 0x55d342a82390 2024-08-31T06:14:04.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.871+0000 7f7c4fcc8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d342e5f940 con 0x55d342a7a1b0 2024-08-31T06:14:04.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.871+0000 7f7c4fcc8740 10 client.0 _ll_register_callbacks cb 0x55d342d5b430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:14:04.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.872+0000 7f7c41e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d342a82390 0x55d342e5d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:04.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.872+0000 7f7c41e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d342a82390 0x55d342e5d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:46342/0 (socket says 172.21.15.16:46342) 2024-08-31T06:14:04.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.872+0000 7f7c41e00640 1 -- 172.21.15.16:0/3941475613 learned_addr learned my addr 172.21.15.16:0/3941475613 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:04.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.872+0000 7f7c41400640 1 --2- 172.21.15.16:0/3941475613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d342a7a1b0 0x55d342e5cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:04.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.872+0000 7f7c3aa00640 1 --2- 172.21.15.16:0/2924885710 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7c280b4cf0 0x7f7c280b7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:04.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.872+0000 7f7c41400640 1 --2- 172.21.15.16:0/3941475613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d342aa3070 0x55d342e68c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:04.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.873+0000 7f7c41e00640 1 -- 172.21.15.16:0/3941475613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d342a7a1b0 msgr2=0x55d342e5cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:04.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.873+0000 7f7c41e00640 1 --2- 172.21.15.16:0/3941475613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d342a7a1b0 0x55d342e5cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:04.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.873+0000 7f7c41e00640 1 -- 172.21.15.16:0/3941475613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d342aa3070 msgr2=0x55d342e68c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:04.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.874+0000 7f7c41e00640 1 --2- 172.21.15.16:0/3941475613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d342aa3070 0x55d342e68c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:04.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.874+0000 7f7c41e00640 1 -- 172.21.15.16:0/3941475613 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d342d76870 con 0x55d342a82390 2024-08-31T06:14:04.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.874+0000 7f7c3aa00640 1 -- 172.21.15.16:0/2924885710 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8470+0+0 (secure 0 0 0) 0x7f7c3c105a40 con 0x7f7c441145a0 2024-08-31T06:14:04.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.874+0000 7f7c41400640 1 --2- 172.21.15.16:0/3941475613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d342aa3070 0x55d342e68c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:14:04.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.875+0000 7f7c41e00640 1 --2- 172.21.15.16:0/3941475613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d342a82390 0x55d342e5d2d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7c34011e30 tx=0x7f7c34011bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=add0efcca91db618 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:04.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.875+0000 7f7c23e00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:14:04.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.876+0000 7f7c23e00640 1 -- 172.21.15.16:0/3941475613 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c34020070 con 0x55d342a82390 2024-08-31T06:14:04.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.876+0000 7f7c23e00640 1 -- 172.21.15.16:0/3941475613 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8470+0+0 (secure 0 0 0) 0x7f7c3401a450 con 0x55d342a82390 2024-08-31T06:14:04.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.877+0000 7f7c4fcc8740 1 -- 172.21.15.16:0/3941475613 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c3400cc60 con 0x55d342a82390 2024-08-31T06:14:04.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.877+0000 7f7c23e00640 1 -- 172.21.15.16:0/3941475613 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f7c34015040 con 0x55d342a82390 2024-08-31T06:14:04.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.878+0000 7f7c4fcc8740 10 client.27062 fetch_fsmap learned FSMap version 38 2024-08-31T06:14:04.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.878+0000 7f7c4fcc8740 1 -- 172.21.15.16:0/3941475613 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55d342e6b940 con 0x55d342a82390 2024-08-31T06:14:04.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.878+0000 7f7c42800640 1 --2- 172.21.15.16:0/2924885710 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7c280b4cf0 0x7f7c280b7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.879+0000 7f7c23e00640 1 -- 172.21.15.16:0/3941475613 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c3400cc60 con 0x55d342a82390 2024-08-31T06:14:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.879+0000 7f7c23e00640 1 -- 172.21.15.16:0/3941475613 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f7c3402e640 con 0x55d342a82390 2024-08-31T06:14:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.879+0000 7f7c4fcc8740 10 client.27062 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T06:14:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.879+0000 7f7c4fcc8740 10 client.27062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:14:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.879+0000 7f7c4fcc8740 20 client.27062 populate_metadata read hostname 'smithi016' 2024-08-31T06:14:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.879+0000 7f7c4fcc8740 1 --2- 172.21.15.16:0/3941475613 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d342a619f0 0x55d342e6eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.879+0000 7f7c4fcc8740 4 client.27062 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:14:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.879+0000 7f7c4fcc8740 1 -- 172.21.15.16:0/3941475613 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d342dc16c0 con 0x55d342a619f0 2024-08-31T06:14:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.879+0000 7f7c41400640 1 --2- 172.21.15.16:0/3941475613 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d342a619f0 0x55d342e6eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.880+0000 7f7c23e00640 10 client.27062 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:04.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.882+0000 7f7c41400640 1 --2- 172.21.15.16:0/3941475613 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d342a619f0 0x55d342e6eca0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfcb0384737f376f server_cookie=32deb558ab2d6c8f in_seq=0 out_seq=0 2024-08-31T06:14:04.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.882+0000 7f7c42800640 1 --2- 172.21.15.16:0/2924885710 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7c280b4cf0 0x7f7c280b7200 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c0094e0 tx=0x7f7c3c02adf0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:04.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.883+0000 7f7c23e00640 1 -- 172.21.15.16:0/3941475613 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d342dc16c0 con 0x55d342a619f0 2024-08-31T06:14:04.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.883+0000 7f7c23e00640 10 client.27062 handle_command_reply: tid=0 2024-08-31T06:14:04.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.883+0000 7f7c4fcc8740 1 client.27062 shutdown 2024-08-31T06:14:04.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.883+0000 7f7c4fcc8740 1 -- 172.21.15.16:0/3941475613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d342a82390 msgr2=0x55d342e5d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:04.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.883+0000 7f7c4fcc8740 1 --2- 172.21.15.16:0/3941475613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d342a82390 0x55d342e5d2d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7c34011e30 tx=0x7f7c34011bf0 comp rx=0 tx=0).stop 2024-08-31T06:14:04.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.883+0000 7f7c4fcc8740 1 -- 172.21.15.16:0/3941475613 shutdown_connections 2024-08-31T06:14:04.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.883+0000 7f7c4fcc8740 1 --2- 172.21.15.16:0/3941475613 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d342a619f0 0x55d342e6eca0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:04.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.883+0000 7f7c4fcc8740 1 --2- 172.21.15.16:0/3941475613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d342aa3070 0x55d342e68c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:04.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.883+0000 7f7c4fcc8740 1 --2- 172.21.15.16:0/3941475613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d342a82390 0x55d342e5d2d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:04.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.883+0000 7f7c4fcc8740 1 --2- 172.21.15.16:0/3941475613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d342a7a1b0 0x55d342e5cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:04.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.883+0000 7f7c4fcc8740 1 -- 172.21.15.16:0/3941475613 >> 172.21.15.16:0/3941475613 conn(0x55d342a97460 msgr2=0x55d342e542f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:04.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.883+0000 7f7c23e00640 0 client.27062 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:04.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.884+0000 7f7c4fcc8740 1 -- 172.21.15.16:0/3941475613 shutdown_connections 2024-08-31T06:14:04.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.884+0000 7f7c4fcc8740 1 -- 172.21.15.16:0/3941475613 wait complete. 2024-08-31T06:14:04.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.884+0000 7f7c4fcc8740 20 client.27062 trim_cache size 0 max 16384 2024-08-31T06:14:04.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.923+0000 7f7c4fcc8740 1 Processor -- start 2024-08-31T06:14:04.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.923+0000 7f7c4fcc8740 1 -- start start 2024-08-31T06:14:04.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.924+0000 7f7c4fcc8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d342a7a1b0 0x55d342e5b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:04.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.924+0000 7f7c4fcc8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d342a82390 0x55d342e5b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:04.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.924+0000 7f7c4fcc8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d342aa3070 0x55d342e56240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:04.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.924+0000 7f7c4fcc8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7c441b8390 con 0x55d342aa3070 2024-08-31T06:14:04.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.924+0000 7f7c4fcc8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7c441b5bd0 con 0x55d342a7a1b0 2024-08-31T06:14:04.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.924+0000 7f7c4fcc8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7c441b3a40 con 0x55d342a82390 2024-08-31T06:14:04.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.924+0000 7f7c41400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d342a7a1b0 0x55d342e5b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:04.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.924+0000 7f7c41400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d342a7a1b0 0x55d342e5b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:46356/0 (socket says 172.21.15.16:46356) 2024-08-31T06:14:04.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.924+0000 7f7c41400640 1 -- 172.21.15.16:0/419819837 learned_addr learned my addr 172.21.15.16:0/419819837 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:04.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.924+0000 7f7c41e00640 1 --2- 172.21.15.16:0/419819837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d342a82390 0x55d342e5b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:04.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.925+0000 7f7c41400640 1 --2- 172.21.15.16:0/419819837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d342aa3070 0x55d342e56240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:04.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.925+0000 7f7c41400640 1 -- 172.21.15.16:0/419819837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d342a82390 msgr2=0x55d342e5b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:04.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.925+0000 7f7c41400640 1 --2- 172.21.15.16:0/419819837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d342a82390 0x55d342e5b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:04.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.925+0000 7f7c41400640 1 -- 172.21.15.16:0/419819837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d342aa3070 msgr2=0x55d342e56240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:04.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.925+0000 7f7c41400640 1 --2- 172.21.15.16:0/419819837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d342aa3070 0x55d342e56240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:04.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.925+0000 7f7c41400640 1 -- 172.21.15.16:0/419819837 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d342e53fc0 con 0x55d342a7a1b0 2024-08-31T06:14:04.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.925+0000 7f7c41400640 1 --2- 172.21.15.16:0/419819837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d342a7a1b0 0x55d342e5b3d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x55d342e53a10 tx=0x7f7c30013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45663e07615067f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:04.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.926+0000 7f7c2ea00640 1 -- 172.21.15.16:0/419819837 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3001a040 con 0x55d342a7a1b0 2024-08-31T06:14:04.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.926+0000 7f7c2ea00640 1 -- 172.21.15.16:0/419819837 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c30003d80 con 0x55d342a7a1b0 2024-08-31T06:14:04.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.926+0000 7f7c2ea00640 1 -- 172.21.15.16:0/419819837 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c30004d00 con 0x55d342a7a1b0 2024-08-31T06:14:04.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.927+0000 7f7c4fcc8740 1 -- 172.21.15.16:0/419819837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d342a7a1b0 msgr2=0x55d342e5b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:04.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.928+0000 7f7c4fcc8740 1 --2- 172.21.15.16:0/419819837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d342a7a1b0 0x55d342e5b3d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x55d342e53a10 tx=0x7f7c30013040 comp rx=0 tx=0).stop 2024-08-31T06:14:04.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.928+0000 7f7c4fcc8740 1 -- 172.21.15.16:0/419819837 shutdown_connections 2024-08-31T06:14:04.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.928+0000 7f7c4fcc8740 1 --2- 172.21.15.16:0/419819837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d342a82390 0x55d342e5b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:04.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.928+0000 7f7c4fcc8740 1 --2- 172.21.15.16:0/419819837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d342aa3070 0x55d342e56240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:04.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.928+0000 7f7c4fcc8740 1 --2- 172.21.15.16:0/419819837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d342a7a1b0 0x55d342e5b3d0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:04.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.928+0000 7f7c4fcc8740 1 -- 172.21.15.16:0/419819837 >> 172.21.15.16:0/419819837 conn(0x55d342a97460 msgr2=0x55d342e5cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:04.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.929+0000 7f7c4fcc8740 1 -- 172.21.15.16:0/419819837 shutdown_connections 2024-08-31T06:14:04.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.929+0000 7f7c4fcc8740 1 -- 172.21.15.16:0/419819837 wait complete. 2024-08-31T06:14:04.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.929+0000 7f7c4fcc8740 1 -- start start 2024-08-31T06:14:04.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.930+0000 7f7c4fcc8740 1 Processor -- start 2024-08-31T06:14:04.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.932+0000 7f7c4fcc8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d342a7a1b0 0x55d342e6d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:04.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.933+0000 7f7c4fcc8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d342a82390 0x55d342e6db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:04.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.933+0000 7f7c4fcc8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d342aa3070 0x55d342ed1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:04.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.933+0000 7f7c4fcc8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7c441b4090 con 0x55d342a82390 2024-08-31T06:14:04.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.933+0000 7f7c4fcc8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7c441b5500 con 0x55d342a7a1b0 2024-08-31T06:14:04.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.933+0000 7f7c4fcc8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7c441b4760 con 0x55d342aa3070 2024-08-31T06:14:04.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.933+0000 7f7c4fcc8740 10 client.0 _ll_register_callbacks cb 0x7f7c34011880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:14:04.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.933+0000 7f7c41e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d342a82390 0x55d342e6db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:04.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.933+0000 7f7c41e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d342a82390 0x55d342e6db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50502/0 (socket says 172.21.15.16:50502) 2024-08-31T06:14:04.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.933+0000 7f7c41e00640 1 -- 172.21.15.16:0/666956595 learned_addr learned my addr 172.21.15.16:0/666956595 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:04.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.934+0000 7f7c41e00640 1 -- 172.21.15.16:0/666956595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d342aa3070 msgr2=0x55d342ed1440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:14:04.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.934+0000 7f7c41e00640 1 --2- 172.21.15.16:0/666956595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d342aa3070 0x55d342ed1440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:04.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.934+0000 7f7c41e00640 1 -- 172.21.15.16:0/666956595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d342a7a1b0 msgr2=0x55d342e6d5c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T06:14:04.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.934+0000 7f7c41e00640 1 --2- 172.21.15.16:0/666956595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d342a7a1b0 0x55d342e6d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:04.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.934+0000 7f7c41e00640 1 -- 172.21.15.16:0/666956595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d342ed1ab0 con 0x55d342a82390 2024-08-31T06:14:04.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.936+0000 7f7c41e00640 1 --2- 172.21.15.16:0/666956595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d342a82390 0x55d342e6db80 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7c34001eb0 tx=0x7f7c34011c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7db22703ff781c55 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:04.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.936+0000 7f7c23e00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:14:04.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.937+0000 7f7c23e00640 1 -- 172.21.15.16:0/666956595 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c34020040 con 0x55d342a82390 2024-08-31T06:14:04.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.938+0000 7f7c4fcc8740 1 -- 172.21.15.16:0/666956595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c30003d80 con 0x55d342a82390 2024-08-31T06:14:04.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.938+0000 7f7c23e00640 1 -- 172.21.15.16:0/666956595 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8470+0+0 (secure 0 0 0) 0x7f7c34006db0 con 0x55d342a82390 2024-08-31T06:14:04.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.938+0000 7f7c23e00640 1 -- 172.21.15.16:0/666956595 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f7c34041050 con 0x55d342a82390 2024-08-31T06:14:04.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.938+0000 7f7c4fcc8740 10 client.16842 fetch_fsmap learned FSMap version 38 2024-08-31T06:14:04.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.938+0000 7f7c4fcc8740 1 -- 172.21.15.16:0/666956595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55d342ed8dc0 con 0x55d342a82390 2024-08-31T06:14:04.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.939+0000 7f7c23e00640 1 -- 172.21.15.16:0/666956595 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c30003d80 con 0x55d342a82390 2024-08-31T06:14:04.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.940+0000 7f7c23e00640 1 -- 172.21.15.16:0/666956595 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f7c3401a9c0 con 0x55d342a82390 2024-08-31T06:14:04.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.940+0000 7f7c4fcc8740 10 client.16842 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T06:14:04.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.940+0000 7f7c4fcc8740 10 client.16842 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:14:04.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.940+0000 7f7c4fcc8740 20 client.16842 populate_metadata read hostname 'smithi016' 2024-08-31T06:14:04.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.940+0000 7f7c4fcc8740 1 --2- 172.21.15.16:0/666956595 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d342a36da0 0x55d342edb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:04.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.940+0000 7f7c4fcc8740 4 client.16842 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T06:14:04.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.940+0000 7f7c4fcc8740 1 -- 172.21.15.16:0/666956595 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d342dbd160 con 0x55d342a36da0 2024-08-31T06:14:04.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.941+0000 7f7c41400640 1 --2- 172.21.15.16:0/666956595 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d342a36da0 0x55d342edb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:04.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.942+0000 7f7c41400640 1 --2- 172.21.15.16:0/666956595 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d342a36da0 0x55d342edb520 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99a52396eaf4f7cb server_cookie=5a0a61f6d723195b in_seq=0 out_seq=0 2024-08-31T06:14:04.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.942+0000 7f7c23e00640 10 client.16842 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:04.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.943+0000 7f7c23e00640 1 -- 172.21.15.16:0/666956595 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2646 (crc 0 0 0) 0x55d342dbd160 con 0x55d342a36da0 2024-08-31T06:14:04.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.943+0000 7f7c23e00640 10 client.16842 handle_command_reply: tid=0 2024-08-31T06:14:04.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.943+0000 7f7c4fcc8740 1 client.16842 shutdown 2024-08-31T06:14:04.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.943+0000 7f7c4fcc8740 1 -- 172.21.15.16:0/666956595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d342a82390 msgr2=0x55d342e6db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:04.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.943+0000 7f7c4fcc8740 1 --2- 172.21.15.16:0/666956595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d342a82390 0x55d342e6db80 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7c34001eb0 tx=0x7f7c34011c60 comp rx=0 tx=0).stop 2024-08-31T06:14:04.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.944+0000 7f7c4fcc8740 1 -- 172.21.15.16:0/666956595 shutdown_connections 2024-08-31T06:14:04.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.944+0000 7f7c4fcc8740 1 --2- 172.21.15.16:0/666956595 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d342a36da0 0x55d342edb520 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:04.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.944+0000 7f7c4fcc8740 1 --2- 172.21.15.16:0/666956595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d342aa3070 0x55d342ed1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:04.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.944+0000 7f7c4fcc8740 1 --2- 172.21.15.16:0/666956595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d342a82390 0x55d342e6db80 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:04.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.944+0000 7f7c4fcc8740 1 --2- 172.21.15.16:0/666956595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d342a7a1b0 0x55d342e6d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:04.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.944+0000 7f7c23e00640 0 client.16842 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:04.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.944+0000 7f7c4fcc8740 1 -- 172.21.15.16:0/666956595 >> 172.21.15.16:0/666956595 conn(0x55d342a97460 msgr2=0x55d342e5cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:04.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.944+0000 7f7c4fcc8740 1 -- 172.21.15.16:0/666956595 shutdown_connections 2024-08-31T06:14:04.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.945+0000 7f7c4fcc8740 1 -- 172.21.15.16:0/666956595 wait complete. 2024-08-31T06:14:04.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.945+0000 7f7c4fcc8740 20 client.16842 trim_cache size 0 max 16384 2024-08-31T06:14:04.949 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:14:04.949 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-08-31T06:14:04.949 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {}, 2024-08-31T06:14:04.950 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-08-31T06:14:04.950 INFO:teuthology.orchestra.run.smithi016.stdout: "e739a16e-8ace-43a8-a1ce-ac27d39ef8a3": { 2024-08-31T06:14:04.950 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:54.084646+0000", 2024-08-31T06:14:04.950 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:14:04.950 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:14:04.950 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:14:04.950 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:14:04.950 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:14:04.950 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:14:04.950 INFO:teuthology.orchestra.run.smithi016.stdout: "91b72a34-4ba9-4126-a414-8b862f8c5f78": { 2024-08-31T06:14:04.950 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:13:28.663237+0000", 2024-08-31T06:14:04.950 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:14:04.950 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:14:04.951 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:14:04.951 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:14:04.951 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:14:04.951 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:14:04.951 INFO:teuthology.orchestra.run.smithi016.stdout: "a10cce76-99ab-46b5-b3b2-07b82899b8f9": { 2024-08-31T06:14:04.951 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:11:43.967546+0000", 2024-08-31T06:14:04.951 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:14:04.951 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:14:04.951 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:14:04.951 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:14:04.951 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:14:04.951 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:14:04.951 INFO:teuthology.orchestra.run.smithi016.stdout: "c1fd6904-2e02-47fa-939a-47349b9fe36b": { 2024-08-31T06:14:04.952 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:06.678691+0000", 2024-08-31T06:14:04.952 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:14:04.952 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:14:04.952 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:14:04.952 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:14:04.952 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:14:04.952 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:14:04.952 INFO:teuthology.orchestra.run.smithi016.stdout: "78d6ef02-8551-45e8-ad60-9627eb573fc1": { 2024-08-31T06:14:04.952 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:09:13.547865+0000", 2024-08-31T06:14:04.952 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:14:04.952 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:14:04.952 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:14:04.952 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:14:04.953 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:14:04.953 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:14:04.953 INFO:teuthology.orchestra.run.smithi016.stdout: "c8455d74-2fe7-4d93-bcf2-19b419d16016": { 2024-08-31T06:14:04.953 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:08:24.035960+0000", 2024-08-31T06:14:04.953 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:14:04.953 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:14:04.953 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:14:04.953 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:14:04.953 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:14:04.953 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:14:04.953 INFO:teuthology.orchestra.run.smithi016.stdout: "88d7bdb8-cb1c-45e5-ad29-4cd8da6c6003": { 2024-08-31T06:14:04.953 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:07:39.648942+0000", 2024-08-31T06:14:04.953 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:14:04.953 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:14:04.954 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:14:04.954 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:14:04.954 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:14:04.954 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:14:04.954 INFO:teuthology.orchestra.run.smithi016.stdout: "6a03046e-cdf4-4a14-b9bd-b560b8968467": { 2024-08-31T06:14:04.954 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:12:38.002354+0000", 2024-08-31T06:14:04.954 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:14:04.954 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:14:04.954 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:14:04.954 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:14:04.954 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:14:04.954 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:14:04.954 INFO:teuthology.orchestra.run.smithi016.stdout: "c300b7e8-f3fd-482c-a909-9329bc61e570": { 2024-08-31T06:14:04.954 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:06:49.868046+0000", 2024-08-31T06:14:04.955 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:14:04.955 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:14:04.955 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:14:04.955 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:14:04.955 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:14:04.955 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:14:04.955 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:14:04.955 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:14:04.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.948+0000 7f7c49000640 1 -- 172.21.15.16:0/2924885710 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7c280b4cf0 msgr2=0x7f7c280b7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:04.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.948+0000 7f7c49000640 1 --2- 172.21.15.16:0/2924885710 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7c280b4cf0 0x7f7c280b7200 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c0094e0 tx=0x7f7c3c02adf0 comp rx=0 tx=0).stop 2024-08-31T06:14:04.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.948+0000 7f7c49000640 1 -- 172.21.15.16:0/2924885710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c441145a0 msgr2=0x7f7c441aa490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:04.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.948+0000 7f7c49000640 1 --2- 172.21.15.16:0/2924885710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c441145a0 0x7f7c441aa490 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c009bf0 tx=0x7f7c3c031f80 comp rx=0 tx=0).stop 2024-08-31T06:14:04.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.948+0000 7f7c49000640 1 -- 172.21.15.16:0/2924885710 shutdown_connections 2024-08-31T06:14:04.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.948+0000 7f7c49000640 1 --2- 172.21.15.16:0/2924885710 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7c280b4cf0 0x7f7c280b7200 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:04.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.948+0000 7f7c49000640 1 --2- 172.21.15.16:0/2924885710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c441145a0 0x7f7c441aa490 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:04.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.948+0000 7f7c49000640 1 --2- 172.21.15.16:0/2924885710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c44113c50 0x7f7c441a51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:04.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.948+0000 7f7c49000640 1 --2- 172.21.15.16:0/2924885710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c4410e360 0x7f7c441a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:04.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.948+0000 7f7c49000640 1 -- 172.21.15.16:0/2924885710 >> 172.21.15.16:0/2924885710 conn(0x7f7c4410a160 msgr2=0x7f7c44111150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:04.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.949+0000 7f7c49000640 1 -- 172.21.15.16:0/2924885710 shutdown_connections 2024-08-31T06:14:04.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:04.949+0000 7f7c49000640 1 -- 172.21.15.16:0/2924885710 wait complete. 2024-08-31T06:14:04.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:04 smithi184 ceph-mon[22331]: pgmap v571: 97 pgs: 97 active+clean; 3.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 12 MiB/s wr, 884 op/s 2024-08-31T06:14:04.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:04 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/1143596115' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:14:05.108 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.106+0000 7fe52a600640 1 -- 172.21.15.16:0/4150528869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe524102c60 msgr2=0x7fe524114170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:05.108 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.106+0000 7fe52a600640 1 --2- 172.21.15.16:0/4150528869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe524102c60 0x7fe524114170 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe510009900 tx=0x7fe51002a4b0 comp rx=0 tx=0).stop 2024-08-31T06:14:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.106+0000 7fe52a600640 1 -- 172.21.15.16:0/4150528869 shutdown_connections 2024-08-31T06:14:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.106+0000 7fe52a600640 1 --2- 172.21.15.16:0/4150528869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5241146b0 0x7fe524116ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.106+0000 7fe52a600640 1 --2- 172.21.15.16:0/4150528869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe524102c60 0x7fe524114170 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.106+0000 7fe52a600640 1 --2- 172.21.15.16:0/4150528869 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe524102310 0x7fe524102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.106+0000 7fe52a600640 1 -- 172.21.15.16:0/4150528869 >> 172.21.15.16:0/4150528869 conn(0x7fe52407a4e0 msgr2=0x7fe52407a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.106+0000 7fe52a600640 1 -- 172.21.15.16:0/4150528869 shutdown_connections 2024-08-31T06:14:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.107+0000 7fe52a600640 1 -- 172.21.15.16:0/4150528869 wait complete. 2024-08-31T06:14:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.108+0000 7fe52a600640 1 Processor -- start 2024-08-31T06:14:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.109+0000 7fe52a600640 1 -- start start 2024-08-31T06:14:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.109+0000 7fe52a600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe524102310 0x7fe524074c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.109+0000 7fe52a600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe524102c60 0x7fe52406def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.109+0000 7fe52a600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5241146b0 0x7fe52406e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.109+0000 7fe523400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe524102310 0x7fe524074c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.110+0000 7fe523400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe524102310 0x7fe524074c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:46380/0 (socket says 172.21.15.16:46380) 2024-08-31T06:14:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.110+0000 7fe523e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5241146b0 0x7fe52406e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:05.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.110+0000 7fe523e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5241146b0 0x7fe52406e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:52566/0 (socket says 172.21.15.16:52566) 2024-08-31T06:14:05.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.111+0000 7fe522a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe524102c60 0x7fe52406def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:05.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.111+0000 7fe522a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe524102c60 0x7fe52406def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50518/0 (socket says 172.21.15.16:50518) 2024-08-31T06:14:05.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.111+0000 7fe523400640 1 -- 172.21.15.16:0/3169998531 learned_addr learned my addr 172.21.15.16:0/3169998531 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:05.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.111+0000 7fe52a600640 1 -- 172.21.15.16:0/3169998531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe524119e10 con 0x7fe524102c60 2024-08-31T06:14:05.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.111+0000 7fe52a600640 1 -- 172.21.15.16:0/3169998531 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe524119c90 con 0x7fe524102310 2024-08-31T06:14:05.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.111+0000 7fe52a600640 1 -- 172.21.15.16:0/3169998531 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe524119f90 con 0x7fe5241146b0 2024-08-31T06:14:05.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.111+0000 7fe523e00640 1 -- 172.21.15.16:0/3169998531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe524102310 msgr2=0x7fe524074c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:05.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.111+0000 7fe523e00640 1 --2- 172.21.15.16:0/3169998531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe524102310 0x7fe524074c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:05.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.111+0000 7fe523e00640 1 -- 172.21.15.16:0/3169998531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe524102c60 msgr2=0x7fe52406def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:05.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.111+0000 7fe523e00640 1 --2- 172.21.15.16:0/3169998531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe524102c60 0x7fe52406def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.111+0000 7fe523e00640 1 -- 172.21.15.16:0/3169998531 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe52406ecd0 con 0x7fe5241146b0 2024-08-31T06:14:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.112+0000 7fe523e00640 1 --2- 172.21.15.16:0/3169998531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5241146b0 0x7fe52406e4b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe51c00be20 tx=0x7fe51c00bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.113+0000 7fe51be00640 1 -- 172.21.15.16:0/3169998531 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c019070 con 0x7fe5241146b0 2024-08-31T06:14:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.113+0000 7fe52a600640 1 -- 172.21.15.16:0/3169998531 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe524073950 con 0x7fe5241146b0 2024-08-31T06:14:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.113+0000 7fe52a600640 1 -- 172.21.15.16:0/3169998531 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe524073ef0 con 0x7fe5241146b0 2024-08-31T06:14:05.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.113+0000 7fe51be00640 1 -- 172.21.15.16:0/3169998531 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe51c0148f0 con 0x7fe5241146b0 2024-08-31T06:14:05.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.113+0000 7fe51be00640 1 -- 172.21.15.16:0/3169998531 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c019070 con 0x7fe5241146b0 2024-08-31T06:14:05.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.119+0000 7fe531697740 1 Processor -- start 2024-08-31T06:14:05.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:04 smithi155 ceph-mon[21742]: pgmap v571: 97 pgs: 97 active+clean; 3.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 12 MiB/s wr, 884 op/s 2024-08-31T06:14:05.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:04 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/1143596115' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:14:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.128+0000 7fe51be00640 1 -- 172.21.15.16:0/3169998531 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fe51c01c070 con 0x7fe5241146b0 2024-08-31T06:14:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.129+0000 7fe531697740 1 -- start start 2024-08-31T06:14:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.129+0000 7fe531697740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f34f3b21b0 0x55f34f755690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.129+0000 7fe531697740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f34f3892f0 0x55f34f755bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.129+0000 7fe531697740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f3914d0 0x55f34f756110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.129+0000 7fe531697740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f34f6caf70 con 0x55f34f3892f0 2024-08-31T06:14:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.129+0000 7fe531697740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f34f6ba500 con 0x55f34f3914d0 2024-08-31T06:14:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.129+0000 7fe531697740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f34f6be4b0 con 0x55f34f3b21b0 2024-08-31T06:14:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.130+0000 7fe522a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f3914d0 0x55f34f756110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.130+0000 7fe522a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f3914d0 0x55f34f756110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:46394/0 (socket says 172.21.15.16:46394) 2024-08-31T06:14:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.130+0000 7fe522a00640 1 -- 172.21.15.16:0/1408359110 learned_addr learned my addr 172.21.15.16:0/1408359110 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.130+0000 7fe522a00640 1 --2- 172.21.15.16:0/1408359110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f34f3b21b0 0x55f34f755690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.130+0000 7fe522a00640 1 -- 172.21.15.16:0/1408359110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f34f3b21b0 msgr2=0x55f34f755690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.130+0000 7fe522a00640 1 --2- 172.21.15.16:0/1408359110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f34f3b21b0 0x55f34f755690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.130+0000 7fe522a00640 1 -- 172.21.15.16:0/1408359110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f34f3892f0 msgr2=0x55f34f755bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.130+0000 7fe522a00640 1 --2- 172.21.15.16:0/1408359110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f34f3892f0 0x55f34f755bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.130+0000 7fe522a00640 1 -- 172.21.15.16:0/1408359110 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f34f674c10 con 0x55f34f3914d0 2024-08-31T06:14:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.131+0000 7fe522a00640 1 --2- 172.21.15.16:0/1408359110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f3914d0 0x55f34f756110 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe510009850 tx=0x7fe510005e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=400789875d8f1e4c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.131+0000 7fe518c00640 1 -- 172.21.15.16:0/1408359110 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510035070 con 0x55f34f3914d0 2024-08-31T06:14:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.131+0000 7fe518c00640 1 -- 172.21.15.16:0/1408359110 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe510005560 con 0x55f34f3914d0 2024-08-31T06:14:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.131+0000 7fe51be00640 1 --2- 172.21.15.16:0/3169998531 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe5080b4f50 0x7fe5080b7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.132+0000 7fe51be00640 1 -- 172.21.15.16:0/3169998531 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8470+0+0 (secure 0 0 0) 0x7fe51c0e4650 con 0x7fe5241146b0 2024-08-31T06:14:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.132+0000 7fe518c00640 1 -- 172.21.15.16:0/1408359110 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5100039f0 con 0x55f34f3914d0 2024-08-31T06:14:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.132+0000 7fe523400640 1 --2- 172.21.15.16:0/3169998531 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe5080b4f50 0x7fe5080b7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.132+0000 7fe531697740 1 -- 172.21.15.16:0/1408359110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f3914d0 msgr2=0x55f34f756110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.132+0000 7fe531697740 1 --2- 172.21.15.16:0/1408359110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f3914d0 0x55f34f756110 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe510009850 tx=0x7fe510005e70 comp rx=0 tx=0).stop 2024-08-31T06:14:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.133+0000 7fe531697740 1 -- 172.21.15.16:0/1408359110 shutdown_connections 2024-08-31T06:14:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.133+0000 7fe531697740 1 --2- 172.21.15.16:0/1408359110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f3914d0 0x55f34f756110 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.133+0000 7fe531697740 1 --2- 172.21.15.16:0/1408359110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f34f3892f0 0x55f34f755bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.133+0000 7fe531697740 1 --2- 172.21.15.16:0/1408359110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f34f3b21b0 0x55f34f755690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.133+0000 7fe531697740 1 -- 172.21.15.16:0/1408359110 >> 172.21.15.16:0/1408359110 conn(0x55f34f3a6300 msgr2=0x55f34f753580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.133+0000 7fe531697740 1 -- 172.21.15.16:0/1408359110 shutdown_connections 2024-08-31T06:14:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.134+0000 7fe531697740 1 -- 172.21.15.16:0/1408359110 wait complete. 2024-08-31T06:14:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.134+0000 7fe523400640 1 --2- 172.21.15.16:0/3169998531 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe5080b4f50 0x7fe5080b7460 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe51400b770 tx=0x7fe51400b700 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.134+0000 7fe531697740 1 -- start start 2024-08-31T06:14:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.135+0000 7fe531697740 1 Processor -- start 2024-08-31T06:14:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.135+0000 7fe531697740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f3892f0 0x55f34f760b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:05.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.135+0000 7fe531697740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f34f3914d0 0x55f34f7610d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:05.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.135+0000 7fe531697740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f34f3b21b0 0x55f34f761610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:05.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.135+0000 7fe531697740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f34f5f3ab0 con 0x55f34f3914d0 2024-08-31T06:14:05.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.135+0000 7fe531697740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f34f661320 con 0x55f34f3892f0 2024-08-31T06:14:05.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.135+0000 7fe531697740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f34f763e50 con 0x55f34f3b21b0 2024-08-31T06:14:05.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.135+0000 7fe531697740 10 client.0 _ll_register_callbacks cb 0x55f34f69ae80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:14:05.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.136+0000 7fe523400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f3892f0 0x55f34f760b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:05.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.136+0000 7fe523400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f3892f0 0x55f34f760b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:46410/0 (socket says 172.21.15.16:46410) 2024-08-31T06:14:05.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.136+0000 7fe523400640 1 -- 172.21.15.16:0/1645405623 learned_addr learned my addr 172.21.15.16:0/1645405623 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:05.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.136+0000 7fe523e00640 1 --2- 172.21.15.16:0/1645405623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f34f3b21b0 0x55f34f761610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:05.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.136+0000 7fe522a00640 1 --2- 172.21.15.16:0/1645405623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f34f3914d0 0x55f34f7610d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:05.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.136+0000 7fe523400640 1 -- 172.21.15.16:0/1645405623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f34f3b21b0 msgr2=0x55f34f761610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.136+0000 7fe523400640 1 --2- 172.21.15.16:0/1645405623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f34f3b21b0 0x55f34f761610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.136+0000 7fe523400640 1 -- 172.21.15.16:0/1645405623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f34f3914d0 msgr2=0x55f34f7610d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.136+0000 7fe523400640 1 --2- 172.21.15.16:0/1645405623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f34f3914d0 0x55f34f7610d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.136+0000 7fe523400640 1 -- 172.21.15.16:0/1645405623 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f34f41d790 con 0x55f34f3892f0 2024-08-31T06:14:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.136+0000 7fe523e00640 1 --2- 172.21.15.16:0/1645405623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f34f3b21b0 0x55f34f761610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:14:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.136+0000 7fe523400640 1 --2- 172.21.15.16:0/1645405623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f3892f0 0x55f34f760b90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe5140116a0 tx=0x7fe51400b480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bace390886cea2c3 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.136+0000 7fe50d600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:14:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.136+0000 7fe50d600640 1 -- 172.21.15.16:0/1645405623 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514005020 con 0x55f34f3892f0 2024-08-31T06:14:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.137+0000 7fe531697740 1 -- 172.21.15.16:0/1645405623 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe510005560 con 0x55f34f3892f0 2024-08-31T06:14:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.137+0000 7fe50d600640 1 -- 172.21.15.16:0/1645405623 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8470+0+0 (secure 0 0 0) 0x7fe5140051c0 con 0x55f34f3892f0 2024-08-31T06:14:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.137+0000 7fe50d600640 1 -- 172.21.15.16:0/1645405623 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fe51402e050 con 0x55f34f3892f0 2024-08-31T06:14:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.137+0000 7fe531697740 10 client.27074 fetch_fsmap learned FSMap version 38 2024-08-31T06:14:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.137+0000 7fe531697740 1 -- 172.21.15.16:0/1645405623 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55f34f496620 con 0x55f34f3892f0 2024-08-31T06:14:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.137+0000 7fe50d600640 1 -- 172.21.15.16:0/1645405623 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe510005560 con 0x55f34f3892f0 2024-08-31T06:14:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.138+0000 7fe50d600640 1 -- 172.21.15.16:0/1645405623 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7fe514025070 con 0x55f34f3892f0 2024-08-31T06:14:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.138+0000 7fe531697740 10 client.27074 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T06:14:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.138+0000 7fe531697740 10 client.27074 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:14:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.138+0000 7fe531697740 20 client.27074 populate_metadata read hostname 'smithi016' 2024-08-31T06:14:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.138+0000 7fe531697740 1 --2- 172.21.15.16:0/1645405623 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f34f370d80 0x55f34f772fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.138+0000 7fe531697740 4 client.27074 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:14:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.138+0000 7fe531697740 1 -- 172.21.15.16:0/1645405623 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f34f6ccec0 con 0x55f34f370d80 2024-08-31T06:14:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.138+0000 7fe522a00640 1 --2- 172.21.15.16:0/1645405623 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f34f370d80 0x55f34f772fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.140+0000 7fe50d600640 10 client.27074 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.140+0000 7fe522a00640 1 --2- 172.21.15.16:0/1645405623 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f34f370d80 0x55f34f772fe0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=781c2981a1634907 server_cookie=b9a855f54ea84518 in_seq=0 out_seq=0 2024-08-31T06:14:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.141+0000 7fe50d600640 1 -- 172.21.15.16:0/1645405623 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f34f6ccec0 con 0x55f34f370d80 2024-08-31T06:14:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.141+0000 7fe50d600640 10 client.27074 handle_command_reply: tid=0 2024-08-31T06:14:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.141+0000 7fe531697740 1 client.27074 shutdown 2024-08-31T06:14:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.142+0000 7fe531697740 1 -- 172.21.15.16:0/1645405623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f3892f0 msgr2=0x55f34f760b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.142+0000 7fe531697740 1 --2- 172.21.15.16:0/1645405623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f3892f0 0x55f34f760b90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe5140116a0 tx=0x7fe51400b480 comp rx=0 tx=0).stop 2024-08-31T06:14:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.142+0000 7fe531697740 1 -- 172.21.15.16:0/1645405623 shutdown_connections 2024-08-31T06:14:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.142+0000 7fe531697740 1 --2- 172.21.15.16:0/1645405623 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f34f370d80 0x55f34f772fe0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.142+0000 7fe531697740 1 --2- 172.21.15.16:0/1645405623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f34f3b21b0 0x55f34f761610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.142+0000 7fe50d600640 0 client.27074 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.142+0000 7fe531697740 1 --2- 172.21.15.16:0/1645405623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f34f3914d0 0x55f34f7610d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.142+0000 7fe531697740 1 --2- 172.21.15.16:0/1645405623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f3892f0 0x55f34f760b90 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.142+0000 7fe531697740 1 -- 172.21.15.16:0/1645405623 >> 172.21.15.16:0/1645405623 conn(0x55f34f3a6300 msgr2=0x55f34f7566c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.143+0000 7fe531697740 1 -- 172.21.15.16:0/1645405623 shutdown_connections 2024-08-31T06:14:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.143+0000 7fe531697740 1 -- 172.21.15.16:0/1645405623 wait complete. 2024-08-31T06:14:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.143+0000 7fe531697740 20 client.27074 trim_cache size 0 max 16384 2024-08-31T06:14:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.171+0000 7fe531697740 1 Processor -- start 2024-08-31T06:14:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.171+0000 7fe531697740 1 -- start start 2024-08-31T06:14:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.171+0000 7fe531697740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f3892f0 0x55f34f762a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.171+0000 7fe531697740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f34f3914d0 0x55f34f752b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.171+0000 7fe531697740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f34f3b21b0 0x55f34f753070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.171+0000 7fe531697740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe504011090 con 0x55f34f3914d0 2024-08-31T06:14:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.171+0000 7fe531697740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe50400e8d0 con 0x55f34f3892f0 2024-08-31T06:14:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.171+0000 7fe531697740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe50400c740 con 0x55f34f3b21b0 2024-08-31T06:14:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.172+0000 7fe523400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f3892f0 0x55f34f762a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.173+0000 7fe522a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f34f3914d0 0x55f34f752b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.173+0000 7fe522a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f34f3914d0 0x55f34f752b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50548/0 (socket says 172.21.15.16:50548) 2024-08-31T06:14:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.173+0000 7fe522a00640 1 -- 172.21.15.16:0/804882119 learned_addr learned my addr 172.21.15.16:0/804882119 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.173+0000 7fe523e00640 1 --2- 172.21.15.16:0/804882119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f34f3b21b0 0x55f34f753070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:05.177 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.174+0000 7fe523400640 1 -- 172.21.15.16:0/804882119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f34f3b21b0 msgr2=0x55f34f753070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:05.177 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.174+0000 7fe523400640 1 --2- 172.21.15.16:0/804882119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f34f3b21b0 0x55f34f753070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:05.177 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.174+0000 7fe523400640 1 -- 172.21.15.16:0/804882119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f34f3914d0 msgr2=0x55f34f752b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:05.177 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.174+0000 7fe523400640 1 --2- 172.21.15.16:0/804882119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f34f3914d0 0x55f34f752b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:05.177 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.174+0000 7fe523400640 1 -- 172.21.15.16:0/804882119 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f34f75f070 con 0x55f34f3892f0 2024-08-31T06:14:05.177 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.174+0000 7fe522a00640 1 --2- 172.21.15.16:0/804882119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f34f3914d0 0x55f34f752b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:14:05.178 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.175+0000 7fe523400640 1 --2- 172.21.15.16:0/804882119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f3892f0 0x55f34f762a70 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe514003670 tx=0x7fe514005f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eae3542db9cd60b9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:05.178 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.175+0000 7fe50fe00640 1 -- 172.21.15.16:0/804882119 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514005580 con 0x55f34f3892f0 2024-08-31T06:14:05.178 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.175+0000 7fe50fe00640 1 -- 172.21.15.16:0/804882119 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe514037070 con 0x55f34f3892f0 2024-08-31T06:14:05.178 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.175+0000 7fe50fe00640 1 -- 172.21.15.16:0/804882119 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51401ed60 con 0x55f34f3892f0 2024-08-31T06:14:05.178 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.176+0000 7fe531697740 1 -- 172.21.15.16:0/804882119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f3892f0 msgr2=0x55f34f762a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.176+0000 7fe531697740 1 --2- 172.21.15.16:0/804882119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f3892f0 0x55f34f762a70 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe514003670 tx=0x7fe514005f70 comp rx=0 tx=0).stop 2024-08-31T06:14:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.177+0000 7fe531697740 1 -- 172.21.15.16:0/804882119 shutdown_connections 2024-08-31T06:14:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.177+0000 7fe531697740 1 --2- 172.21.15.16:0/804882119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f34f3b21b0 0x55f34f753070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.177+0000 7fe531697740 1 --2- 172.21.15.16:0/804882119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f34f3914d0 0x55f34f752b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.177+0000 7fe531697740 1 --2- 172.21.15.16:0/804882119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f3892f0 0x55f34f762a70 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.177+0000 7fe531697740 1 -- 172.21.15.16:0/804882119 >> 172.21.15.16:0/804882119 conn(0x55f34f3a6300 msgr2=0x55f34f760060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.177+0000 7fe531697740 1 -- 172.21.15.16:0/804882119 shutdown_connections 2024-08-31T06:14:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.177+0000 7fe531697740 1 -- 172.21.15.16:0/804882119 wait complete. 2024-08-31T06:14:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.177+0000 7fe531697740 1 -- start start 2024-08-31T06:14:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.177+0000 7fe531697740 1 Processor -- start 2024-08-31T06:14:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.178+0000 7fe531697740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f34f3892f0 0x55f34f7d89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.178+0000 7fe531697740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f34f3914d0 0x55f34f7d8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.178+0000 7fe531697740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f3b21b0 0x55f34f7d9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.178+0000 7fe531697740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe50400cd90 con 0x55f34f3892f0 2024-08-31T06:14:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.178+0000 7fe531697740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe50400e200 con 0x55f34f3b21b0 2024-08-31T06:14:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.178+0000 7fe531697740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe50400d460 con 0x55f34f3914d0 2024-08-31T06:14:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.178+0000 7fe531697740 10 client.0 _ll_register_callbacks cb 0x7fe51400bed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:14:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.179+0000 7fe522a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f34f3914d0 0x55f34f7d8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.179+0000 7fe523e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f3b21b0 0x55f34f7d9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.179+0000 7fe523e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f3b21b0 0x55f34f7d9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:46434/0 (socket says 172.21.15.16:46434) 2024-08-31T06:14:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.179+0000 7fe523e00640 1 -- 172.21.15.16:0/167481100 learned_addr learned my addr 172.21.15.16:0/167481100 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.179+0000 7fe523e00640 1 -- 172.21.15.16:0/167481100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f34f3914d0 msgr2=0x55f34f7d8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.179+0000 7fe523e00640 1 --2- 172.21.15.16:0/167481100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f34f3914d0 0x55f34f7d8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.179+0000 7fe523e00640 1 -- 172.21.15.16:0/167481100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f34f3892f0 msgr2=0x55f34f7d89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.180+0000 7fe523400640 1 --2- 172.21.15.16:0/167481100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f34f3892f0 0x55f34f7d89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.180+0000 7fe523e00640 1 --2- 172.21.15.16:0/167481100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f34f3892f0 0x55f34f7d89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.180+0000 7fe523e00640 1 -- 172.21.15.16:0/167481100 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f34f7674e0 con 0x55f34f3b21b0 2024-08-31T06:14:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.181+0000 7fe523e00640 1 --2- 172.21.15.16:0/167481100 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f3b21b0 0x55f34f7d9450 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe51c0a8030 tx=0x7fe51c0aa5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=777de73df7bb54bc server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.181+0000 7fe50d600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:14:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.181+0000 7fe50d600640 1 -- 172.21.15.16:0/167481100 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c0af580 con 0x55f34f3b21b0 2024-08-31T06:14:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.182+0000 7fe50d600640 1 -- 172.21.15.16:0/167481100 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8470+0+0 (secure 0 0 0) 0x7fe51c0af150 con 0x55f34f3b21b0 2024-08-31T06:14:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.182+0000 7fe531697740 1 -- 172.21.15.16:0/167481100 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe514037070 con 0x55f34f3b21b0 2024-08-31T06:14:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.182+0000 7fe50d600640 1 -- 172.21.15.16:0/167481100 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fe51c0af350 con 0x55f34f3b21b0 2024-08-31T06:14:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.182+0000 7fe531697740 10 client.27080 fetch_fsmap learned FSMap version 38 2024-08-31T06:14:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.182+0000 7fe531697740 1 -- 172.21.15.16:0/167481100 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55f34f7eeea0 con 0x55f34f3b21b0 2024-08-31T06:14:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.182+0000 7fe50d600640 1 -- 172.21.15.16:0/167481100 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe514037070 con 0x55f34f3b21b0 2024-08-31T06:14:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.183+0000 7fe50d600640 1 -- 172.21.15.16:0/167481100 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7fe51c090bb0 con 0x55f34f3b21b0 2024-08-31T06:14:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.183+0000 7fe531697740 10 client.27080 fetch_fsmap finished waiting for FSMap version 38 2024-08-31T06:14:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.183+0000 7fe531697740 10 client.27080 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:14:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.183+0000 7fe531697740 20 client.27080 populate_metadata read hostname 'smithi016' 2024-08-31T06:14:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.183+0000 7fe531697740 1 --2- 172.21.15.16:0/167481100 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f34f3461c0 0x55f34f7f16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.183+0000 7fe531697740 4 client.27080 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:14:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.183+0000 7fe531697740 1 -- 172.21.15.16:0/167481100 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f34f6c8960 con 0x55f34f3461c0 2024-08-31T06:14:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.184+0000 7fe523400640 1 --2- 172.21.15.16:0/167481100 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f34f3461c0 0x55f34f7f16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.185+0000 7fe523400640 1 --2- 172.21.15.16:0/167481100 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f34f3461c0 0x55f34f7f16c0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59353092013c224e server_cookie=9554634a593637c1 in_seq=0 out_seq=0 2024-08-31T06:14:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.185+0000 7fe50d600640 10 client.27080 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.206+0000 7fe50d600640 1 -- 172.21.15.16:0/167481100 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f34f6c8960 con 0x55f34f3461c0 2024-08-31T06:14:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.206+0000 7fe50d600640 10 client.27080 handle_command_reply: tid=0 2024-08-31T06:14:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.207+0000 7fe531697740 1 client.27080 shutdown 2024-08-31T06:14:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.207+0000 7fe531697740 1 -- 172.21.15.16:0/167481100 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f3b21b0 msgr2=0x55f34f7d9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.207+0000 7fe531697740 1 --2- 172.21.15.16:0/167481100 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f3b21b0 0x55f34f7d9450 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe51c0a8030 tx=0x7fe51c0aa5f0 comp rx=0 tx=0).stop 2024-08-31T06:14:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.208+0000 7fe531697740 1 -- 172.21.15.16:0/167481100 shutdown_connections 2024-08-31T06:14:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.208+0000 7fe531697740 1 --2- 172.21.15.16:0/167481100 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f34f3461c0 0x55f34f7f16c0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.208+0000 7fe531697740 1 --2- 172.21.15.16:0/167481100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f34f3914d0 0x55f34f7d8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.208+0000 7fe531697740 1 --2- 172.21.15.16:0/167481100 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f34f3b21b0 0x55f34f7d9450 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.208+0000 7fe531697740 1 --2- 172.21.15.16:0/167481100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f34f3892f0 0x55f34f7d89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.208+0000 7fe531697740 1 -- 172.21.15.16:0/167481100 >> 172.21.15.16:0/167481100 conn(0x55f34f3a6300 msgr2=0x55f34f760060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.208+0000 7fe50d600640 0 client.27080 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.208+0000 7fe531697740 1 -- 172.21.15.16:0/167481100 shutdown_connections 2024-08-31T06:14:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.208+0000 7fe531697740 1 -- 172.21.15.16:0/167481100 wait complete. 2024-08-31T06:14:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.209+0000 7fe531697740 20 client.27080 trim_cache size 0 max 16384 2024-08-31T06:14:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:14:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:14:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:14:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:14:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.210+0000 7fe52a600640 1 -- 172.21.15.16:0/3169998531 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe5080b4f50 msgr2=0x7fe5080b7460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.210+0000 7fe52a600640 1 --2- 172.21.15.16:0/3169998531 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe5080b4f50 0x7fe5080b7460 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe51400b770 tx=0x7fe51400b700 comp rx=0 tx=0).stop 2024-08-31T06:14:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.210+0000 7fe52a600640 1 -- 172.21.15.16:0/3169998531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5241146b0 msgr2=0x7fe52406e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.210+0000 7fe52a600640 1 --2- 172.21.15.16:0/3169998531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5241146b0 0x7fe52406e4b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe51c00be20 tx=0x7fe51c00bf10 comp rx=0 tx=0).stop 2024-08-31T06:14:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.210+0000 7fe52a600640 1 -- 172.21.15.16:0/3169998531 shutdown_connections 2024-08-31T06:14:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.210+0000 7fe52a600640 1 --2- 172.21.15.16:0/3169998531 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe5080b4f50 0x7fe5080b7460 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.210+0000 7fe52a600640 1 --2- 172.21.15.16:0/3169998531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5241146b0 0x7fe52406e4b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.210+0000 7fe52a600640 1 --2- 172.21.15.16:0/3169998531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe524102c60 0x7fe52406def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.210+0000 7fe52a600640 1 --2- 172.21.15.16:0/3169998531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe524102310 0x7fe524074c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.210+0000 7fe52a600640 1 -- 172.21.15.16:0/3169998531 >> 172.21.15.16:0/3169998531 conn(0x7fe52407a4e0 msgr2=0x7fe52410a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:05.214 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.210+0000 7fe52a600640 1 -- 172.21.15.16:0/3169998531 shutdown_connections 2024-08-31T06:14:05.214 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:05.211+0000 7fe52a600640 1 -- 172.21.15.16:0/3169998531 wait complete. 2024-08-31T06:14:06.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:05 smithi155 ceph-mon[21742]: pgmap v572: 97 pgs: 97 active+clean; 3.7 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 26 MiB/s wr, 1.77k op/s 2024-08-31T06:14:06.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:05 smithi016 ceph-mon[17194]: pgmap v572: 97 pgs: 97 active+clean; 3.7 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 26 MiB/s wr, 1.77k op/s 2024-08-31T06:14:06.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:05 smithi184 ceph-mon[22331]: pgmap v572: 97 pgs: 97 active+clean; 3.7 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 26 MiB/s wr, 1.77k op/s 2024-08-31T06:14:06.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 400.0 s, 1117.8 tps, lat 95.616 ms stddev 222.218 2024-08-31T06:14:07.204 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24737 quiesce db / --timeout 90 --expiration 148.0 --await-for 45.0' 2024-08-31T06:14:07.204 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24737 quiesce db / --timeout 90 --expiration 148.0 --await-for 45.0 2024-08-31T06:14:07.979 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T06:14:07.979 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:14:08.180 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:14:08.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:08 smithi016 ceph-mon[17194]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:14:08.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:08 smithi016 ceph-mon[17194]: pgmap v573: 97 pgs: 97 active+clean; 3.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 MiB/s wr, 1.72k op/s 2024-08-31T06:14:08.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:08 smithi155 ceph-mon[21742]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:14:08.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:08 smithi155 ceph-mon[21742]: pgmap v573: 97 pgs: 97 active+clean; 3.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 MiB/s wr, 1.72k op/s 2024-08-31T06:14:08.472 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:08 smithi184 ceph-mon[22331]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:14:08.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:08 smithi184 ceph-mon[22331]: pgmap v573: 97 pgs: 97 active+clean; 3.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 MiB/s wr, 1.72k op/s 2024-08-31T06:14:09.070 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:14:09.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.168+0000 7fa1cf200640 1 -- 172.21.15.16:0/38111573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1d010a6a0 msgr2=0x7fa1d010aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:09.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.168+0000 7fa1cf200640 1 --2- 172.21.15.16:0/38111573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1d010a6a0 0x7fa1d010aab0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa1b8009900 tx=0x7fa1b802a420 comp rx=0 tx=0).stop 2024-08-31T06:14:09.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.169+0000 7fa1cf200640 1 -- 172.21.15.16:0/38111573 shutdown_connections 2024-08-31T06:14:09.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.169+0000 7fa1cf200640 1 --2- 172.21.15.16:0/38111573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1d011c940 0x7fa1d011ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:09.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.169+0000 7fa1cf200640 1 --2- 172.21.15.16:0/38111573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1d010a6a0 0x7fa1d010aab0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:09.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.169+0000 7fa1cf200640 1 --2- 172.21.15.16:0/38111573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1d0111b10 0x7fa1d010a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:09.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.169+0000 7fa1cf200640 1 -- 172.21.15.16:0/38111573 >> 172.21.15.16:0/38111573 conn(0x7fa1d006e7b0 msgr2=0x7fa1d006ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:09.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.171+0000 7fa1cf200640 1 -- 172.21.15.16:0/38111573 shutdown_connections 2024-08-31T06:14:09.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.171+0000 7fa1cf200640 1 -- 172.21.15.16:0/38111573 wait complete. 2024-08-31T06:14:09.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.175+0000 7fa1cf200640 1 Processor -- start 2024-08-31T06:14:09.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.176+0000 7fa1cf200640 1 -- start start 2024-08-31T06:14:09.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.177+0000 7fa1cf200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1d010a6a0 0x7fa1d01a8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:09.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.177+0000 7fa1cf200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1d0111b10 0x7fa1d01a9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:09.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.177+0000 7fa1cf200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1d011c940 0x7fa1d01ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:09.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.178+0000 7fa1cde00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1d010a6a0 0x7fa1d01a8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:09.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.178+0000 7fa1cde00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1d010a6a0 0x7fa1d01a8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:38804/0 (socket says 172.21.15.16:38804) 2024-08-31T06:14:09.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.179+0000 7fa1ce800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1d011c940 0x7fa1d01ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:09.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.179+0000 7fa1ce800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1d011c940 0x7fa1d01ae7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:35212/0 (socket says 172.21.15.16:35212) 2024-08-31T06:14:09.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.179+0000 7fa1cde00640 1 -- 172.21.15.16:0/1803824656 learned_addr learned my addr 172.21.15.16:0/1803824656 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:09.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.179+0000 7fa1cf200640 1 -- 172.21.15.16:0/1803824656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d0122550 con 0x7fa1d0111b10 2024-08-31T06:14:09.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.180+0000 7fa1cf200640 1 -- 172.21.15.16:0/1803824656 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d01223d0 con 0x7fa1d011c940 2024-08-31T06:14:09.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.180+0000 7fa1cd400640 1 --2- 172.21.15.16:0/1803824656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1d0111b10 0x7fa1d01a9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:09.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.180+0000 7fa1cf200640 1 -- 172.21.15.16:0/1803824656 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d01226d0 con 0x7fa1d010a6a0 2024-08-31T06:14:09.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.182+0000 7fa1ce800640 1 -- 172.21.15.16:0/1803824656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1d010a6a0 msgr2=0x7fa1d01a8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:09.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.182+0000 7fa1ce800640 1 --2- 172.21.15.16:0/1803824656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1d010a6a0 0x7fa1d01a8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:09.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.182+0000 7fa1ce800640 1 -- 172.21.15.16:0/1803824656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1d0111b10 msgr2=0x7fa1d01a9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:09.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.182+0000 7fa1ce800640 1 --2- 172.21.15.16:0/1803824656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1d0111b10 0x7fa1d01a9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:09.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.182+0000 7fa1ce800640 1 -- 172.21.15.16:0/1803824656 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d01aeef0 con 0x7fa1d011c940 2024-08-31T06:14:09.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.183+0000 7fa1ce800640 1 --2- 172.21.15.16:0/1803824656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1d011c940 0x7fa1d01ae7c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa1c800b3e0 tx=0x7fa1c800b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:09.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.184+0000 7fa1c6a00640 1 -- 172.21.15.16:0/1803824656 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8015020 con 0x7fa1d011c940 2024-08-31T06:14:09.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.184+0000 7fa1c6a00640 1 -- 172.21.15.16:0/1803824656 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1c8011ce0 con 0x7fa1d011c940 2024-08-31T06:14:09.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.184+0000 7fa1c6a00640 1 -- 172.21.15.16:0/1803824656 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8010d80 con 0x7fa1d011c940 2024-08-31T06:14:09.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.184+0000 7fa1cf200640 1 -- 172.21.15.16:0/1803824656 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1d01af1e0 con 0x7fa1d011c940 2024-08-31T06:14:09.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.184+0000 7fa1cf200640 1 -- 172.21.15.16:0/1803824656 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1d01b5940 con 0x7fa1d011c940 2024-08-31T06:14:09.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.189+0000 7fa1db653740 1 Processor -- start 2024-08-31T06:14:09.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.197+0000 7fa1c6a00640 1 -- 172.21.15.16:0/1803824656 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fa1c8018020 con 0x7fa1d011c940 2024-08-31T06:14:09.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.197+0000 7fa1db653740 1 -- start start 2024-08-31T06:14:09.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.198+0000 7fa1db653740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faf33c3840 0x55faf376a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:09.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.198+0000 7fa1db653740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf339a980 0x55faf376a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:09.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.198+0000 7fa1db653740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faf33a2b60 0x55faf376ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:09.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.198+0000 7fa1db653740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55faf36df530 con 0x55faf339a980 2024-08-31T06:14:09.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.198+0000 7fa1db653740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55faf36ceac0 con 0x55faf33c3840 2024-08-31T06:14:09.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.198+0000 7fa1db653740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55faf36d2a70 con 0x55faf33a2b60 2024-08-31T06:14:09.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.199+0000 7fa1c6a00640 1 --2- 172.21.15.16:0/1803824656 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa1b40b4d10 0x7fa1b40b7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:09.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.200+0000 7fa1c6a00640 1 -- 172.21.15.16:0/1803824656 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8514+0+0 (secure 0 0 0) 0x7fa1c80e4c40 con 0x7fa1d011c940 2024-08-31T06:14:09.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.200+0000 7fa1cd400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faf33c3840 0x55faf376a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:09.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.200+0000 7fa1cd400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faf33c3840 0x55faf376a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:35220/0 (socket says 172.21.15.16:35220) 2024-08-31T06:14:09.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.201+0000 7fa1cd400640 1 -- 172.21.15.16:0/1151804165 learned_addr learned my addr 172.21.15.16:0/1151804165 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:09.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.201+0000 7fa1cd400640 1 --2- 172.21.15.16:0/1151804165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faf33a2b60 0x55faf376ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:09.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.202+0000 7fa1cd400640 1 -- 172.21.15.16:0/1151804165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faf33a2b60 msgr2=0x55faf376ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:09.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.202+0000 7fa1cd400640 1 --2- 172.21.15.16:0/1151804165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faf33a2b60 0x55faf376ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:09.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.202+0000 7fa1cd400640 1 -- 172.21.15.16:0/1151804165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf339a980 msgr2=0x55faf376a810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:14:09.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.204+0000 7fa1ce800640 1 --2- 172.21.15.16:0/1803824656 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa1b40b4d10 0x7fa1b40b7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:09.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.204+0000 7fa1cde00640 1 --2- 172.21.15.16:0/1151804165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf339a980 0x55faf376a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:09.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.204+0000 7fa1cd400640 1 --2- 172.21.15.16:0/1151804165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf339a980 0x55faf376a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:09.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.205+0000 7fa1cd400640 1 -- 172.21.15.16:0/1151804165 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faf3681880 con 0x55faf33c3840 2024-08-31T06:14:09.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.207+0000 7fa1ce800640 1 --2- 172.21.15.16:0/1803824656 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa1b40b4d10 0x7fa1b40b7220 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa1c80e2440 tx=0x7fa1c8010380 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:09.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.208+0000 7fa1cd400640 1 --2- 172.21.15.16:0/1151804165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faf33c3840 0x55faf376a2d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa1b8000c00 tx=0x7fa1b8009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0cafc4799caa50 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:09.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.209+0000 7fa1bf000640 1 -- 172.21.15.16:0/1151804165 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b8035030 con 0x55faf33c3840 2024-08-31T06:14:09.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.209+0000 7fa1bf000640 1 -- 172.21.15.16:0/1151804165 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1b8004030 con 0x55faf33c3840 2024-08-31T06:14:09.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.209+0000 7fa1bf000640 1 -- 172.21.15.16:0/1151804165 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b8005540 con 0x55faf33c3840 2024-08-31T06:14:09.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.211+0000 7fa1db653740 1 -- 172.21.15.16:0/1151804165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faf33c3840 msgr2=0x55faf376a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:09.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.211+0000 7fa1db653740 1 --2- 172.21.15.16:0/1151804165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faf33c3840 0x55faf376a2d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa1b8000c00 tx=0x7fa1b8009560 comp rx=0 tx=0).stop 2024-08-31T06:14:09.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.211+0000 7fa1db653740 1 -- 172.21.15.16:0/1151804165 shutdown_connections 2024-08-31T06:14:09.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.211+0000 7fa1db653740 1 --2- 172.21.15.16:0/1151804165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faf33a2b60 0x55faf376ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:09.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.211+0000 7fa1db653740 1 --2- 172.21.15.16:0/1151804165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf339a980 0x55faf376a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:09.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.212+0000 7fa1db653740 1 --2- 172.21.15.16:0/1151804165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faf33c3840 0x55faf376a2d0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:09.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.212+0000 7fa1db653740 1 -- 172.21.15.16:0/1151804165 >> 172.21.15.16:0/1151804165 conn(0x55faf33b7990 msgr2=0x55faf37681c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:09.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.212+0000 7fa1db653740 1 -- 172.21.15.16:0/1151804165 shutdown_connections 2024-08-31T06:14:09.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.213+0000 7fa1db653740 1 -- 172.21.15.16:0/1151804165 wait complete. 2024-08-31T06:14:09.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.213+0000 7fa1db653740 1 -- start start 2024-08-31T06:14:09.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.214+0000 7fa1db653740 1 Processor -- start 2024-08-31T06:14:09.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.217+0000 7fa1db653740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faf339a980 0x55faf3775fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:09.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.218+0000 7fa1db653740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faf33a2b60 0x55faf3776580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:09.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.218+0000 7fa1db653740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf33c3840 0x55faf3781be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:09.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.218+0000 7fa1db653740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55faf35317a0 con 0x55faf33c3840 2024-08-31T06:14:09.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.218+0000 7fa1db653740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55faf36557f0 con 0x55faf339a980 2024-08-31T06:14:09.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.218+0000 7fa1db653740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55faf3778890 con 0x55faf33a2b60 2024-08-31T06:14:09.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.218+0000 7fa1db653740 10 client.0 _ll_register_callbacks cb 0x55faf367ae60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:14:09.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.219+0000 7fa1cde00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faf33a2b60 0x55faf3776580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:09.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.219+0000 7fa1cde00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faf33a2b60 0x55faf3776580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:38816/0 (socket says 172.21.15.16:38816) 2024-08-31T06:14:09.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.219+0000 7fa1cde00640 1 -- 172.21.15.16:0/3006387592 learned_addr learned my addr 172.21.15.16:0/3006387592 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:09.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.220+0000 7fa1cd400640 1 --2- 172.21.15.16:0/3006387592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faf339a980 0x55faf3775fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:09.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.220+0000 7fa1cd400640 1 --2- 172.21.15.16:0/3006387592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf33c3840 0x55faf3781be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:09.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.222+0000 7fa1cd400640 1 -- 172.21.15.16:0/3006387592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faf33a2b60 msgr2=0x55faf3776580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:09.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.223+0000 7fa1cd400640 1 --2- 172.21.15.16:0/3006387592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faf33a2b60 0x55faf3776580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:09.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.223+0000 7fa1cd400640 1 -- 172.21.15.16:0/3006387592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faf339a980 msgr2=0x55faf3775fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:09.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.223+0000 7fa1cd400640 1 --2- 172.21.15.16:0/3006387592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faf339a980 0x55faf3775fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:09.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.224+0000 7fa1cd400640 1 -- 172.21.15.16:0/3006387592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faf343db90 con 0x55faf33c3840 2024-08-31T06:14:09.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.225+0000 7fa1cde00640 1 --2- 172.21.15.16:0/3006387592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faf33a2b60 0x55faf3776580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:14:09.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.226+0000 7fa1cd400640 1 --2- 172.21.15.16:0/3006387592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf33c3840 0x55faf3781be0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa1b8031440 tx=0x7fa1b8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47250ccc86446d3 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:09.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.226+0000 7fa1b3400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:14:09.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.228+0000 7fa1b3400640 1 -- 172.21.15.16:0/3006387592 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b8035070 con 0x55faf33c3840 2024-08-31T06:14:09.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.228+0000 7fa1b3400640 1 -- 172.21.15.16:0/3006387592 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8514+0+0 (secure 0 0 0) 0x7fa1b80035f0 con 0x55faf33c3840 2024-08-31T06:14:09.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.228+0000 7fa1db653740 1 -- 172.21.15.16:0/3006387592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1b8004030 con 0x55faf33c3840 2024-08-31T06:14:09.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.229+0000 7fa1b3400640 1 -- 172.21.15.16:0/3006387592 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa1b80037f0 con 0x55faf33c3840 2024-08-31T06:14:09.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.229+0000 7fa1db653740 10 client.16863 fetch_fsmap learned FSMap version 39 2024-08-31T06:14:09.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.229+0000 7fa1db653740 1 -- 172.21.15.16:0/3006387592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55faf37849b0 con 0x55faf33c3840 2024-08-31T06:14:09.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.233+0000 7fa1b3400640 1 -- 172.21.15.16:0/3006387592 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1b8004030 con 0x55faf33c3840 2024-08-31T06:14:09.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.234+0000 7fa1b3400640 1 -- 172.21.15.16:0/3006387592 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fa1b803d7c0 con 0x55faf33c3840 2024-08-31T06:14:09.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.234+0000 7fa1db653740 10 client.16863 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T06:14:09.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.234+0000 7fa1db653740 10 client.16863 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:14:09.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.234+0000 7fa1db653740 20 client.16863 populate_metadata read hostname 'smithi016' 2024-08-31T06:14:09.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.234+0000 7fa1db653740 1 --2- 172.21.15.16:0/3006387592 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55faf33821c0 0x55faf3787dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:09.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.234+0000 7fa1db653740 4 client.16863 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:14:09.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.234+0000 7fa1db653740 1 -- 172.21.15.16:0/3006387592 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55faf36e1480 con 0x55faf33821c0 2024-08-31T06:14:09.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.235+0000 7fa1cde00640 1 --2- 172.21.15.16:0/3006387592 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55faf33821c0 0x55faf3787dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:09.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.235+0000 7fa1cde00640 1 --2- 172.21.15.16:0/3006387592 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55faf33821c0 0x55faf3787dd0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e9e895cba1f0f6a server_cookie=f4d827de62e62f1a in_seq=0 out_seq=0 2024-08-31T06:14:09.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.236+0000 7fa1b3400640 10 client.16863 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:09.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.236+0000 7fa1b3400640 1 -- 172.21.15.16:0/3006387592 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55faf36e1480 con 0x55faf33821c0 2024-08-31T06:14:09.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.236+0000 7fa1b3400640 10 client.16863 handle_command_reply: tid=0 2024-08-31T06:14:09.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.236+0000 7fa1db653740 1 client.16863 shutdown 2024-08-31T06:14:09.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.237+0000 7fa1db653740 1 -- 172.21.15.16:0/3006387592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf33c3840 msgr2=0x55faf3781be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:09.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.237+0000 7fa1db653740 1 --2- 172.21.15.16:0/3006387592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf33c3840 0x55faf3781be0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa1b8031440 tx=0x7fa1b8005fd0 comp rx=0 tx=0).stop 2024-08-31T06:14:09.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.237+0000 7fa1db653740 1 -- 172.21.15.16:0/3006387592 shutdown_connections 2024-08-31T06:14:09.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.237+0000 7fa1db653740 1 --2- 172.21.15.16:0/3006387592 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55faf33821c0 0x55faf3787dd0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:09.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.237+0000 7fa1db653740 1 --2- 172.21.15.16:0/3006387592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faf33a2b60 0x55faf3776580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:09.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.237+0000 7fa1db653740 1 --2- 172.21.15.16:0/3006387592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf33c3840 0x55faf3781be0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:09.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.237+0000 7fa1db653740 1 --2- 172.21.15.16:0/3006387592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faf339a980 0x55faf3775fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:09.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.237+0000 7fa1db653740 1 -- 172.21.15.16:0/3006387592 >> 172.21.15.16:0/3006387592 conn(0x55faf33b7990 msgr2=0x55faf376d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:09.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.237+0000 7fa1b3400640 0 client.16863 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:09.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.238+0000 7fa1db653740 1 -- 172.21.15.16:0/3006387592 shutdown_connections 2024-08-31T06:14:09.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.239+0000 7fa1db653740 1 -- 172.21.15.16:0/3006387592 wait complete. 2024-08-31T06:14:09.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.239+0000 7fa1db653740 20 client.16863 trim_cache size 0 max 16384 2024-08-31T06:14:09.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.278+0000 7fa1db653740 1 Processor -- start 2024-08-31T06:14:09.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.278+0000 7fa1db653740 1 -- start start 2024-08-31T06:14:09.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.279+0000 7fa1db653740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faf339a980 0x55faf3777c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:09.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.279+0000 7fa1db653740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf33a2b60 0x55faf376fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:09.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.279+0000 7fa1db653740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faf33c3840 0x55faf3788de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:09.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.279+0000 7fa1db653740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d01b82d0 con 0x55faf33a2b60 2024-08-31T06:14:09.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.279+0000 7fa1db653740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d01b5bb0 con 0x55faf33c3840 2024-08-31T06:14:09.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.279+0000 7fa1db653740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d01aa0b0 con 0x55faf339a980 2024-08-31T06:14:09.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.279+0000 7fa1cd400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faf339a980 0x55faf3777c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:09.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.279+0000 7fa1cd400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faf339a980 0x55faf3777c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:38832/0 (socket says 172.21.15.16:38832) 2024-08-31T06:14:09.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.279+0000 7fa1cd400640 1 -- 172.21.15.16:0/3880659390 learned_addr learned my addr 172.21.15.16:0/3880659390 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:09.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.279+0000 7fa1cd400640 1 --2- 172.21.15.16:0/3880659390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faf33c3840 0x55faf3788de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:09.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.280+0000 7fa1cd400640 1 -- 172.21.15.16:0/3880659390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faf33c3840 msgr2=0x55faf3788de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:09.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.280+0000 7fa1cd400640 1 --2- 172.21.15.16:0/3880659390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faf33c3840 0x55faf3788de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:09.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.280+0000 7fa1cd400640 1 -- 172.21.15.16:0/3880659390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf33a2b60 msgr2=0x55faf376fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:09.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.280+0000 7fa1cd400640 1 --2- 172.21.15.16:0/3880659390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf33a2b60 0x55faf376fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:09.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.281+0000 7fa1cd400640 1 -- 172.21.15.16:0/3880659390 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faf37826f0 con 0x55faf339a980 2024-08-31T06:14:09.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.281+0000 7fa1cd400640 1 --2- 172.21.15.16:0/3880659390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faf339a980 0x55faf3777c80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa1b80059d0 tx=0x7fa1b802a420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a5d0996faced068 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:09.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.281+0000 7fa1be600640 1 -- 172.21.15.16:0/3880659390 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b8035040 con 0x55faf339a980 2024-08-31T06:14:09.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.282+0000 7fa1be600640 1 -- 172.21.15.16:0/3880659390 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1b803d930 con 0x55faf339a980 2024-08-31T06:14:09.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.282+0000 7fa1be600640 1 -- 172.21.15.16:0/3880659390 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b8003bf0 con 0x55faf339a980 2024-08-31T06:14:09.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.283+0000 7fa1db653740 1 -- 172.21.15.16:0/3880659390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faf339a980 msgr2=0x55faf3777c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:09.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.283+0000 7fa1db653740 1 --2- 172.21.15.16:0/3880659390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faf339a980 0x55faf3777c80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa1b80059d0 tx=0x7fa1b802a420 comp rx=0 tx=0).stop 2024-08-31T06:14:09.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.284+0000 7fa1db653740 1 -- 172.21.15.16:0/3880659390 shutdown_connections 2024-08-31T06:14:09.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.284+0000 7fa1db653740 1 --2- 172.21.15.16:0/3880659390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faf33c3840 0x55faf3788de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:09.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.284+0000 7fa1db653740 1 --2- 172.21.15.16:0/3880659390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf33a2b60 0x55faf376fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:09.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.284+0000 7fa1db653740 1 --2- 172.21.15.16:0/3880659390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faf339a980 0x55faf3777c80 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:09.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.284+0000 7fa1db653740 1 -- 172.21.15.16:0/3880659390 >> 172.21.15.16:0/3880659390 conn(0x55faf33b7990 msgr2=0x55faf376b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:09.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.284+0000 7fa1db653740 1 -- 172.21.15.16:0/3880659390 shutdown_connections 2024-08-31T06:14:09.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.285+0000 7fa1db653740 1 -- 172.21.15.16:0/3880659390 wait complete. 2024-08-31T06:14:09.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.285+0000 7fa1db653740 1 -- start start 2024-08-31T06:14:09.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.286+0000 7fa1db653740 1 Processor -- start 2024-08-31T06:14:09.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.287+0000 7fa1db653740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faf339a980 0x55faf378bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:09.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.287+0000 7fa1db653740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faf33a2b60 0x55faf378c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:09.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.288+0000 7fa1db653740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf33c3840 0x55faf37f1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:09.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.288+0000 7fa1db653740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d0118ed0 con 0x55faf33c3840 2024-08-31T06:14:09.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.288+0000 7fa1db653740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d011a320 con 0x55faf339a980 2024-08-31T06:14:09.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.288+0000 7fa1db653740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d0119580 con 0x55faf33a2b60 2024-08-31T06:14:09.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.288+0000 7fa1db653740 10 client.0 _ll_register_callbacks cb 0x7fa1b8031b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:14:09.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.288+0000 7fa1cde00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faf33a2b60 0x55faf378c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:09.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.288+0000 7fa1cde00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faf33a2b60 0x55faf378c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:38836/0 (socket says 172.21.15.16:38836) 2024-08-31T06:14:09.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.288+0000 7fa1cde00640 1 -- 172.21.15.16:0/855713309 learned_addr learned my addr 172.21.15.16:0/855713309 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:09.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.289+0000 7fa1cde00640 1 -- 172.21.15.16:0/855713309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faf339a980 msgr2=0x55faf378bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:09.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.289+0000 7fa1cde00640 1 --2- 172.21.15.16:0/855713309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faf339a980 0x55faf378bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:09.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.289+0000 7fa1cde00640 1 -- 172.21.15.16:0/855713309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf33c3840 msgr2=0x55faf37f1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:09.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.289+0000 7fa1cde00640 1 --2- 172.21.15.16:0/855713309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf33c3840 0x55faf37f1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:09.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.289+0000 7fa1cde00640 1 -- 172.21.15.16:0/855713309 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faf378c890 con 0x55faf33a2b60 2024-08-31T06:14:09.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.292+0000 7fa1cde00640 1 --2- 172.21.15.16:0/855713309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faf33a2b60 0x55faf378c060 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x55faf36df660 tx=0x7fa1c0014cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3abf65f16c7ecbde server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:09.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.292+0000 7fa1b3400640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:14:09.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.293+0000 7fa1b3400640 1 -- 172.21.15.16:0/855713309 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0014ea0 con 0x55faf33a2b60 2024-08-31T06:14:09.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.294+0000 7fa1db653740 1 -- 172.21.15.16:0/855713309 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1b803d930 con 0x55faf33a2b60 2024-08-31T06:14:09.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.294+0000 7fa1b3400640 1 -- 172.21.15.16:0/855713309 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8514+0+0 (secure 0 0 0) 0x7fa1c0014320 con 0x55faf33a2b60 2024-08-31T06:14:09.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.295+0000 7fa1b3400640 1 -- 172.21.15.16:0/855713309 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa1c0014520 con 0x55faf33a2b60 2024-08-31T06:14:09.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.295+0000 7fa1db653740 10 client.27088 fetch_fsmap learned FSMap version 39 2024-08-31T06:14:09.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.295+0000 7fa1db653740 1 -- 172.21.15.16:0/855713309 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55faf37f3090 con 0x55faf33a2b60 2024-08-31T06:14:09.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.296+0000 7fa1b3400640 1 -- 172.21.15.16:0/855713309 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1b803d930 con 0x55faf33a2b60 2024-08-31T06:14:09.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.296+0000 7fa1b3400640 1 -- 172.21.15.16:0/855713309 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fa1c000f040 con 0x55faf33a2b60 2024-08-31T06:14:09.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.297+0000 7fa1db653740 10 client.27088 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T06:14:09.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.297+0000 7fa1db653740 10 client.27088 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:14:09.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.297+0000 7fa1db653740 20 client.27088 populate_metadata read hostname 'smithi016' 2024-08-31T06:14:09.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.297+0000 7fa1db653740 1 --2- 172.21.15.16:0/855713309 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55faf3357570 0x55faf37f5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:09.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.297+0000 7fa1db653740 4 client.27088 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 148.0, "await_for": 45.0}] 2024-08-31T06:14:09.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.297+0000 7fa1db653740 1 -- 172.21.15.16:0/855713309 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 148.0, "await_for": 45.0}) -- 0x55faf36dcf20 con 0x55faf3357570 2024-08-31T06:14:09.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.298+0000 7fa1cd400640 1 --2- 172.21.15.16:0/855713309 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55faf3357570 0x55faf37f5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:09.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.299+0000 7fa1cd400640 1 --2- 172.21.15.16:0/855713309 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55faf3357570 0x55faf37f5950 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8946b202dbc17f4 server_cookie=c30475d4a7992574 in_seq=0 out_seq=0 2024-08-31T06:14:09.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.299+0000 7fa1b3400640 10 client.27088 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:09.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.621+0000 7fa1b3400640 1 -- 172.21.15.16:0/855713309 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55faf36dcf20 con 0x55faf3357570 2024-08-31T06:14:09.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.621+0000 7fa1b3400640 10 client.27088 handle_command_reply: tid=0 2024-08-31T06:14:09.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.621+0000 7fa1db653740 1 client.27088 shutdown 2024-08-31T06:14:09.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.622+0000 7fa1db653740 1 -- 172.21.15.16:0/855713309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faf33a2b60 msgr2=0x55faf378c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:09.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.622+0000 7fa1db653740 1 --2- 172.21.15.16:0/855713309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faf33a2b60 0x55faf378c060 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x55faf36df660 tx=0x7fa1c0014cb0 comp rx=0 tx=0).stop 2024-08-31T06:14:09.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.622+0000 7fa1db653740 1 -- 172.21.15.16:0/855713309 shutdown_connections 2024-08-31T06:14:09.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.622+0000 7fa1db653740 1 --2- 172.21.15.16:0/855713309 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55faf3357570 0x55faf37f5950 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:09.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.622+0000 7fa1db653740 1 --2- 172.21.15.16:0/855713309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faf33a2b60 0x55faf378c060 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:09.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.622+0000 7fa1db653740 1 --2- 172.21.15.16:0/855713309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf33c3840 0x55faf37f1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:09.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.622+0000 7fa1db653740 1 --2- 172.21.15.16:0/855713309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55faf339a980 0x55faf378bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:09.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.622+0000 7fa1b3400640 0 client.27088 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:09.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.622+0000 7fa1db653740 1 -- 172.21.15.16:0/855713309 >> 172.21.15.16:0/855713309 conn(0x55faf33b7990 msgr2=0x55faf376fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:09.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.622+0000 7fa1db653740 1 -- 172.21.15.16:0/855713309 shutdown_connections 2024-08-31T06:14:09.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.622+0000 7fa1db653740 1 -- 172.21.15.16:0/855713309 wait complete. 2024-08-31T06:14:09.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.622+0000 7fa1db653740 20 client.27088 trim_cache size 0 max 16384 2024-08-31T06:14:09.626 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:14:09.627 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 39, 2024-08-31T06:14:09.627 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24737, 2024-08-31T06:14:09.627 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 22, 2024-08-31T06:14:09.627 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-08-31T06:14:09.627 INFO:teuthology.orchestra.run.smithi016.stdout: "2403ca55": { 2024-08-31T06:14:09.627 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 22, 2024-08-31T06:14:09.627 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-08-31T06:14:09.627 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:14:09.627 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-08-31T06:14:09.627 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:14:09.627 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:14:09.627 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-08-31T06:14:09.628 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 148.0, 2024-08-31T06:14:09.628 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-08-31T06:14:09.628 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-08-31T06:14:09.628 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-08-31T06:14:09.628 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:14:09.628 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-08-31T06:14:09.628 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:14:09.628 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:14:09.628 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:14:09.628 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:14:09.629 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:14:09.629 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:14:09.629 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:14:09.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.623+0000 7fa1cf200640 1 -- 172.21.15.16:0/1803824656 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa1b40b4d10 msgr2=0x7fa1b40b7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:09.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.623+0000 7fa1cf200640 1 --2- 172.21.15.16:0/1803824656 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa1b40b4d10 0x7fa1b40b7220 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa1c80e2440 tx=0x7fa1c8010380 comp rx=0 tx=0).stop 2024-08-31T06:14:09.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.623+0000 7fa1cf200640 1 -- 172.21.15.16:0/1803824656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1d011c940 msgr2=0x7fa1d01ae7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:09.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.623+0000 7fa1cf200640 1 --2- 172.21.15.16:0/1803824656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1d011c940 0x7fa1d01ae7c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa1c800b3e0 tx=0x7fa1c800b8a0 comp rx=0 tx=0).stop 2024-08-31T06:14:09.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.623+0000 7fa1cf200640 1 -- 172.21.15.16:0/1803824656 shutdown_connections 2024-08-31T06:14:09.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.623+0000 7fa1cf200640 1 --2- 172.21.15.16:0/1803824656 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa1b40b4d10 0x7fa1b40b7220 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:09.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.623+0000 7fa1cf200640 1 --2- 172.21.15.16:0/1803824656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1d011c940 0x7fa1d01ae7c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:09.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.623+0000 7fa1cf200640 1 --2- 172.21.15.16:0/1803824656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1d0111b10 0x7fa1d01a9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:09.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.623+0000 7fa1cf200640 1 --2- 172.21.15.16:0/1803824656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1d010a6a0 0x7fa1d01a8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:09.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.623+0000 7fa1cf200640 1 -- 172.21.15.16:0/1803824656 >> 172.21.15.16:0/1803824656 conn(0x7fa1d006e7b0 msgr2=0x7fa1d011a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:09.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.623+0000 7fa1cf200640 1 -- 172.21.15.16:0/1803824656 shutdown_connections 2024-08-31T06:14:09.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.623+0000 7fa1cf200640 1 -- 172.21.15.16:0/1803824656 wait complete. 2024-08-31T06:14:10.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:10 smithi155 ceph-mon[21742]: osdmap e84: 12 total, 12 up, 12 in 2024-08-31T06:14:10.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:10 smithi155 ceph-mon[21742]: pgmap v575: 97 pgs: 97 active+clean; 3.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 29 MiB/s wr, 2.00k op/s 2024-08-31T06:14:10.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:10 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/1076095102' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:14:10.408 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:10 smithi016 ceph-mon[17194]: osdmap e84: 12 total, 12 up, 12 in 2024-08-31T06:14:10.409 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:10 smithi016 ceph-mon[17194]: pgmap v575: 97 pgs: 97 active+clean; 3.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 29 MiB/s wr, 2.00k op/s 2024-08-31T06:14:10.409 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:10 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/1076095102' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:14:10.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:10 smithi184 ceph-mon[22331]: osdmap e84: 12 total, 12 up, 12 in 2024-08-31T06:14:10.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:10 smithi184 ceph-mon[22331]: pgmap v575: 97 pgs: 97 active+clean; 3.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 29 MiB/s wr, 2.00k op/s 2024-08-31T06:14:10.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:10 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/1076095102' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:14:10.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.844+0000 7f3bbd800640 1 -- 172.21.15.16:0/3092941884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb8103900 msgr2=0x7f3bb8103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:10.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.844+0000 7f3bbd800640 1 --2- 172.21.15.16:0/3092941884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb8103900 0x7f3bb8103d10 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8009900 tx=0x7f3ba802a3f0 comp rx=0 tx=0).stop 2024-08-31T06:14:10.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.845+0000 7f3bbd800640 1 -- 172.21.15.16:0/3092941884 shutdown_connections 2024-08-31T06:14:10.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.845+0000 7f3bbd800640 1 --2- 172.21.15.16:0/3092941884 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3bb8109240 0x7f3bb8109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:10.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.845+0000 7f3bbd800640 1 --2- 172.21.15.16:0/3092941884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bb8109b90 0x7f3bb81103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:10.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.845+0000 7f3bbd800640 1 --2- 172.21.15.16:0/3092941884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb8103900 0x7f3bb8103d10 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:10.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.845+0000 7f3bbd800640 1 -- 172.21.15.16:0/3092941884 >> 172.21.15.16:0/3092941884 conn(0x7f3bb807b5f0 msgr2=0x7f3bb807ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:10.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.845+0000 7f3bbd800640 1 -- 172.21.15.16:0/3092941884 shutdown_connections 2024-08-31T06:14:10.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.845+0000 7f3bbd800640 1 -- 172.21.15.16:0/3092941884 wait complete. 2024-08-31T06:14:10.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.846+0000 7f3bbd800640 1 Processor -- start 2024-08-31T06:14:10.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.846+0000 7f3bbd800640 1 -- start start 2024-08-31T06:14:10.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.846+0000 7f3bbd800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb8103900 0x7f3bb819c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:10.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.846+0000 7f3bbd800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bb8109240 0x7f3bb819cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:10.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.846+0000 7f3bbd800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3bb8109b90 0x7f3bb81a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:10.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.846+0000 7f3bbd800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb81138a0 con 0x7f3bb8103900 2024-08-31T06:14:10.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.846+0000 7f3bbd800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb8113720 con 0x7f3bb8109240 2024-08-31T06:14:10.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.846+0000 7f3bbd800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb8113a20 con 0x7f3bb8109b90 2024-08-31T06:14:10.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.847+0000 7f3bb6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb8103900 0x7f3bb819c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:10.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.847+0000 7f3bb6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb8103900 0x7f3bb819c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36144/0 (socket says 172.21.15.16:36144) 2024-08-31T06:14:10.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.847+0000 7f3bb6a00640 1 -- 172.21.15.16:0/1076095102 learned_addr learned my addr 172.21.15.16:0/1076095102 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:10.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.847+0000 7f3bae000640 1 --2- 172.21.15.16:0/1076095102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bb8109240 0x7f3bb819cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:10.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.847+0000 7f3bb7400640 1 --2- 172.21.15.16:0/1076095102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3bb8109b90 0x7f3bb81a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:10.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.847+0000 7f3bae000640 1 -- 172.21.15.16:0/1076095102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3bb8109b90 msgr2=0x7f3bb81a1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:10.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.847+0000 7f3bae000640 1 --2- 172.21.15.16:0/1076095102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3bb8109b90 0x7f3bb81a1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:10.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.847+0000 7f3bae000640 1 -- 172.21.15.16:0/1076095102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb8103900 msgr2=0x7f3bb819c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:10.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.847+0000 7f3bae000640 1 --2- 172.21.15.16:0/1076095102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb8103900 0x7f3bb819c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:10.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.847+0000 7f3bae000640 1 -- 172.21.15.16:0/1076095102 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb81a2480 con 0x7f3bb8109240 2024-08-31T06:14:10.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.847+0000 7f3bb6a00640 1 --2- 172.21.15.16:0/1076095102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb8103900 0x7f3bb819c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:14:10.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.848+0000 7f3bae000640 1 --2- 172.21.15.16:0/1076095102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bb8109240 0x7f3bb819cab0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3bb806c790 tx=0x7f3ba000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:10.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.848+0000 7f3bafe00640 1 -- 172.21.15.16:0/1076095102 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba000cc40 con 0x7f3bb8109240 2024-08-31T06:14:10.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.848+0000 7f3bafe00640 1 -- 172.21.15.16:0/1076095102 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ba000cde0 con 0x7f3bb8109240 2024-08-31T06:14:10.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.848+0000 7f3bbd800640 1 -- 172.21.15.16:0/1076095102 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bb81a2770 con 0x7f3bb8109240 2024-08-31T06:14:10.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.848+0000 7f3bafe00640 1 -- 172.21.15.16:0/1076095102 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0004a00 con 0x7f3bb8109240 2024-08-31T06:14:10.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.848+0000 7f3bbd800640 1 -- 172.21.15.16:0/1076095102 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bb81a8ed0 con 0x7f3bb8109240 2024-08-31T06:14:10.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.849+0000 7f3bbd800640 1 -- 172.21.15.16:0/1076095102 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bb8104b80 con 0x7f3bb8109240 2024-08-31T06:14:10.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.852+0000 7f3bafe00640 1 -- 172.21.15.16:0/1076095102 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f3ba0004ba0 con 0x7f3bb8109240 2024-08-31T06:14:10.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.853+0000 7f3bafe00640 1 --2- 172.21.15.16:0/1076095102 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3b9c0b4ca0 0x7f3b9c0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:10.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.853+0000 7f3bafe00640 1 -- 172.21.15.16:0/1076095102 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8514+0+0 (secure 0 0 0) 0x7f3ba00e5570 con 0x7f3bb8109240 2024-08-31T06:14:10.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.853+0000 7f3bb6a00640 1 --2- 172.21.15.16:0/1076095102 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3b9c0b4ca0 0x7f3b9c0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:10.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.854+0000 7f3bb6a00640 1 --2- 172.21.15.16:0/1076095102 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3b9c0b4ca0 0x7f3b9c0b71b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3ba80096c0 tx=0x7f3ba8005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:10.514 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:14:10.514 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":39,"btime":"2024-08-31T06:14:07:075654+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:14:07.025476+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24737,24724,24712]},"id":1}]} 2024-08-31T06:14:10.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:09.855+0000 7f3bafe00640 1 -- 172.21.15.16:0/1076095102 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f3ba00ac390 con 0x7f3bb8109240 2024-08-31T06:14:10.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:10.096+0000 7f3bbd800640 1 -- 172.21.15.16:0/1076095102 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3bb8103d10 con 0x7f3bb8109240 2024-08-31T06:14:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:10.097+0000 7f3bafe00640 1 -- 172.21.15.16:0/1076095102 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11920 (secure 0 0 0) 0x7f3bb8103d10 con 0x7f3bb8109240 2024-08-31T06:14:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 39 2024-08-31T06:14:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:10.100+0000 7f3bbd800640 1 -- 172.21.15.16:0/1076095102 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3b9c0b4ca0 msgr2=0x7f3b9c0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:10.100+0000 7f3bbd800640 1 --2- 172.21.15.16:0/1076095102 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3b9c0b4ca0 0x7f3b9c0b71b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3ba80096c0 tx=0x7f3ba8005a20 comp rx=0 tx=0).stop 2024-08-31T06:14:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:10.100+0000 7f3bbd800640 1 -- 172.21.15.16:0/1076095102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bb8109240 msgr2=0x7f3bb819cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:10.100+0000 7f3bbd800640 1 --2- 172.21.15.16:0/1076095102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bb8109240 0x7f3bb819cab0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3bb806c790 tx=0x7f3ba000ee50 comp rx=0 tx=0).stop 2024-08-31T06:14:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:10.100+0000 7f3bbd800640 1 -- 172.21.15.16:0/1076095102 shutdown_connections 2024-08-31T06:14:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:10.100+0000 7f3bbd800640 1 --2- 172.21.15.16:0/1076095102 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3b9c0b4ca0 0x7f3b9c0b71b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:10.100+0000 7f3bbd800640 1 --2- 172.21.15.16:0/1076095102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3bb8109b90 0x7f3bb81a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:10.100+0000 7f3bbd800640 1 --2- 172.21.15.16:0/1076095102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bb8109240 0x7f3bb819cab0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:10.100+0000 7f3bbd800640 1 --2- 172.21.15.16:0/1076095102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb8103900 0x7f3bb819c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:10.100+0000 7f3bbd800640 1 -- 172.21.15.16:0/1076095102 >> 172.21.15.16:0/1076095102 conn(0x7f3bb807b5f0 msgr2=0x7f3bb8107050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:10.100+0000 7f3bbd800640 1 -- 172.21.15.16:0/1076095102 shutdown_connections 2024-08-31T06:14:10.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:10.100+0000 7f3bbd800640 1 -- 172.21.15.16:0/1076095102 wait complete. 2024-08-31T06:14:11.384 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2403ca55', quiesce took 9.6 seconds. Will release after: 18.4 2024-08-31T06:14:11.626 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 405.0 s, 1518.5 tps, lat 38.060 ms stddev 41.709 2024-08-31T06:14:12.049 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T06:14:12.430 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:14:12.791 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:12 smithi016 ceph-mon[17194]: pgmap v576: 97 pgs: 97 active+clean; 3.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 29 MiB/s wr, 2.02k op/s 2024-08-31T06:14:12.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:12 smithi155 ceph-mon[21742]: pgmap v576: 97 pgs: 97 active+clean; 3.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 29 MiB/s wr, 2.02k op/s 2024-08-31T06:14:12.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:12 smithi184 ceph-mon[22331]: pgmap v576: 97 pgs: 97 active+clean; 3.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 29 MiB/s wr, 2.02k op/s 2024-08-31T06:14:13.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.120+0000 7f4a39e00640 1 -- 172.21.15.16:0/1493563807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a34114310 msgr2=0x7f4a34116730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:13.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.120+0000 7f4a39e00640 1 --2- 172.21.15.16:0/1493563807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a34114310 0x7f4a34116730 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c009ca0 tx=0x7f4a2c02a8c0 comp rx=0 tx=0).stop 2024-08-31T06:14:13.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.120+0000 7f4a39e00640 1 -- 172.21.15.16:0/1493563807 shutdown_connections 2024-08-31T06:14:13.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.120+0000 7f4a39e00640 1 --2- 172.21.15.16:0/1493563807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a34114310 0x7f4a34116730 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:13.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.120+0000 7f4a39e00640 1 --2- 172.21.15.16:0/1493563807 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a34078030 0x7f4a34113dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:13.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.120+0000 7f4a39e00640 1 --2- 172.21.15.16:0/1493563807 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a340776e0 0x7f4a34077af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:13.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.120+0000 7f4a39e00640 1 -- 172.21.15.16:0/1493563807 >> 172.21.15.16:0/1493563807 conn(0x7f4a34101420 msgr2=0x7f4a34103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:13.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.120+0000 7f4a39e00640 1 -- 172.21.15.16:0/1493563807 shutdown_connections 2024-08-31T06:14:13.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.120+0000 7f4a39e00640 1 -- 172.21.15.16:0/1493563807 wait complete. 2024-08-31T06:14:13.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.121+0000 7f4a39e00640 1 Processor -- start 2024-08-31T06:14:13.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.121+0000 7f4a39e00640 1 -- start start 2024-08-31T06:14:13.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.122+0000 7f4a39e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a340776e0 0x7f4a341a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:13.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.122+0000 7f4a39e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a34078030 0x7f4a341a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:13.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.122+0000 7f4a39e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a34114310 0x7f4a341a5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:13.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.122+0000 7f4a39e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4a34119c90 con 0x7f4a34078030 2024-08-31T06:14:13.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.122+0000 7f4a39e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4a34119b10 con 0x7f4a34114310 2024-08-31T06:14:13.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.122+0000 7f4a39e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4a34119e10 con 0x7f4a340776e0 2024-08-31T06:14:13.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.122+0000 7f4a33e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a34078030 0x7f4a341a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:13.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.122+0000 7f4a33e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a34078030 0x7f4a341a0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36168/0 (socket says 172.21.15.16:36168) 2024-08-31T06:14:13.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.122+0000 7f4a33e00640 1 -- 172.21.15.16:0/1763996877 learned_addr learned my addr 172.21.15.16:0/1763996877 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:13.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.122+0000 7f4a38a00640 1 --2- 172.21.15.16:0/1763996877 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a340776e0 0x7f4a341a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:13.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.122+0000 7f4a39400640 1 --2- 172.21.15.16:0/1763996877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a34114310 0x7f4a341a5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:13.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.122+0000 7f4a38a00640 1 -- 172.21.15.16:0/1763996877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a34114310 msgr2=0x7f4a341a5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:13.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.122+0000 7f4a38a00640 1 --2- 172.21.15.16:0/1763996877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a34114310 0x7f4a341a5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:13.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.122+0000 7f4a38a00640 1 -- 172.21.15.16:0/1763996877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a34078030 msgr2=0x7f4a341a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:13.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.122+0000 7f4a38a00640 1 --2- 172.21.15.16:0/1763996877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a34078030 0x7f4a341a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:13.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.123+0000 7f4a38a00640 1 -- 172.21.15.16:0/1763996877 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a341a6630 con 0x7f4a340776e0 2024-08-31T06:14:13.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.123+0000 7f4a33e00640 1 --2- 172.21.15.16:0/1763996877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a34078030 0x7f4a341a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:14:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.123+0000 7f4a38a00640 1 --2- 172.21.15.16:0/1763996877 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a340776e0 0x7f4a341a06c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4a2800e9f0 tx=0x7f4a2800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.123+0000 7f4a31600640 1 -- 172.21.15.16:0/1763996877 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2800ccc0 con 0x7f4a340776e0 2024-08-31T06:14:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.123+0000 7f4a39e00640 1 -- 172.21.15.16:0/1763996877 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a341a6920 con 0x7f4a340776e0 2024-08-31T06:14:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.123+0000 7f4a31600640 1 -- 172.21.15.16:0/1763996877 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a28005020 con 0x7f4a340776e0 2024-08-31T06:14:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.123+0000 7f4a31600640 1 -- 172.21.15.16:0/1763996877 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a28004a00 con 0x7f4a340776e0 2024-08-31T06:14:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.123+0000 7f4a39e00640 1 -- 172.21.15.16:0/1763996877 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a341ad080 con 0x7f4a340776e0 2024-08-31T06:14:13.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.127+0000 7f4a4206f740 1 Processor -- start 2024-08-31T06:14:13.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.127+0000 7f4a4206f740 1 -- start start 2024-08-31T06:14:13.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.128+0000 7f4a4206f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cd7ed6070 0x563cd827ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:13.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.128+0000 7f4a4206f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563cd7ead1b0 0x563cd827bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:13.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.128+0000 7f4a4206f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cd7eb5390 0x563cd827c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:13.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.128+0000 7f4a4206f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563cd81f27b0 con 0x563cd7ed6070 2024-08-31T06:14:13.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.128+0000 7f4a4206f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563cd81e1d40 con 0x563cd7eb5390 2024-08-31T06:14:13.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.128+0000 7f4a4206f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563cd81e5cf0 con 0x563cd7ead1b0 2024-08-31T06:14:13.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.128+0000 7f4a39400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cd7ed6070 0x563cd827ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:13.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.128+0000 7f4a39400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cd7ed6070 0x563cd827ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36176/0 (socket says 172.21.15.16:36176) 2024-08-31T06:14:13.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.128+0000 7f4a39400640 1 -- 172.21.15.16:0/3846008208 learned_addr learned my addr 172.21.15.16:0/3846008208 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:13.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.128+0000 7f4a33e00640 1 --2- 172.21.15.16:0/3846008208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cd7eb5390 0x563cd827c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:13.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.129+0000 7f4a39400640 1 -- 172.21.15.16:0/3846008208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563cd7ead1b0 msgr2=0x563cd827bfb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:14:13.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.129+0000 7f4a39400640 1 --2- 172.21.15.16:0/3846008208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563cd7ead1b0 0x563cd827bfb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:13.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.129+0000 7f4a39400640 1 -- 172.21.15.16:0/3846008208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cd7eb5390 msgr2=0x563cd827c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:13.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.129+0000 7f4a39400640 1 --2- 172.21.15.16:0/3846008208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cd7eb5390 0x563cd827c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:13.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.129+0000 7f4a39400640 1 -- 172.21.15.16:0/3846008208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cd7f41650 con 0x563cd7ed6070 2024-08-31T06:14:13.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.129+0000 7f4a31600640 1 -- 172.21.15.16:0/1763996877 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f4a28004ba0 con 0x7f4a340776e0 2024-08-31T06:14:13.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.129+0000 7f4a39400640 1 --2- 172.21.15.16:0/3846008208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cd7ed6070 0x563cd827ba70 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c02f410 tx=0x7f4a2c02f440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f9082c152b09b87 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:13.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.129+0000 7f4a25c00640 1 -- 172.21.15.16:0/3846008208 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c031cf0 con 0x563cd7ed6070 2024-08-31T06:14:13.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.129+0000 7f4a25c00640 1 -- 172.21.15.16:0/3846008208 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a2c036030 con 0x563cd7ed6070 2024-08-31T06:14:13.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.129+0000 7f4a25c00640 1 -- 172.21.15.16:0/3846008208 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c004040 con 0x563cd7ed6070 2024-08-31T06:14:13.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.130+0000 7f4a4206f740 1 -- 172.21.15.16:0/3846008208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cd7ed6070 msgr2=0x563cd827ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:13.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.130+0000 7f4a4206f740 1 --2- 172.21.15.16:0/3846008208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cd7ed6070 0x563cd827ba70 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c02f410 tx=0x7f4a2c02f440 comp rx=0 tx=0).stop 2024-08-31T06:14:13.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.130+0000 7f4a4206f740 1 -- 172.21.15.16:0/3846008208 shutdown_connections 2024-08-31T06:14:13.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.130+0000 7f4a4206f740 1 --2- 172.21.15.16:0/3846008208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563cd7ead1b0 0x563cd827bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:13.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.130+0000 7f4a4206f740 1 --2- 172.21.15.16:0/3846008208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cd7eb5390 0x563cd827c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:13.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.130+0000 7f4a4206f740 1 --2- 172.21.15.16:0/3846008208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cd7ed6070 0x563cd827ba70 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:13.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.130+0000 7f4a4206f740 1 -- 172.21.15.16:0/3846008208 >> 172.21.15.16:0/3846008208 conn(0x563cd7eca460 msgr2=0x563cd8279960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:13.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.130+0000 7f4a4206f740 1 -- 172.21.15.16:0/3846008208 shutdown_connections 2024-08-31T06:14:13.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.130+0000 7f4a4206f740 1 -- 172.21.15.16:0/3846008208 wait complete. 2024-08-31T06:14:13.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.131+0000 7f4a31600640 1 --2- 172.21.15.16:0/1763996877 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f4a1c0b4bb0 0x7f4a1c0b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:13.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.131+0000 7f4a31600640 1 -- 172.21.15.16:0/1763996877 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8514+0+0 (secure 0 0 0) 0x7f4a280e5500 con 0x7f4a340776e0 2024-08-31T06:14:13.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.131+0000 7f4a4206f740 1 -- start start 2024-08-31T06:14:13.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.131+0000 7f4a33e00640 1 --2- 172.21.15.16:0/1763996877 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f4a1c0b4bb0 0x7f4a1c0b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:13.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.131+0000 7f4a4206f740 1 Processor -- start 2024-08-31T06:14:13.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.132+0000 7f4a33e00640 1 --2- 172.21.15.16:0/1763996877 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f4a1c0b4bb0 0x7f4a1c0b70c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4a34079e60 tx=0x7f4a20005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:13.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.132+0000 7f4a4206f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cd7ead1b0 0x563cd8287700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:13.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.132+0000 7f4a4206f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cd7eb5390 0x563cd8287ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:13.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.132+0000 7f4a4206f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563cd7ed6070 0x563cd828d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:13.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.132+0000 7f4a4206f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563cd8044700 con 0x563cd7eb5390 2024-08-31T06:14:13.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.132+0000 7f4a4206f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563cd8168cf0 con 0x563cd7ead1b0 2024-08-31T06:14:13.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.132+0000 7f4a4206f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563cd828a070 con 0x563cd7ed6070 2024-08-31T06:14:13.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.132+0000 7f4a4206f740 10 client.0 _ll_register_callbacks cb 0x563cd818e470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:14:13.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.132+0000 7f4a33e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cd7eb5390 0x563cd8287ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:13.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.132+0000 7f4a38a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cd7ead1b0 0x563cd8287700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:13.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.132+0000 7f4a38a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cd7ead1b0 0x563cd8287700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:35316/0 (socket says 172.21.15.16:35316) 2024-08-31T06:14:13.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.132+0000 7f4a38a00640 1 -- 172.21.15.16:0/173725448 learned_addr learned my addr 172.21.15.16:0/173725448 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:13.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.132+0000 7f4a39400640 1 --2- 172.21.15.16:0/173725448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563cd7ed6070 0x563cd828d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:13.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.133+0000 7f4a38a00640 1 -- 172.21.15.16:0/173725448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563cd7ed6070 msgr2=0x563cd828d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:13.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.133+0000 7f4a38a00640 1 --2- 172.21.15.16:0/173725448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563cd7ed6070 0x563cd828d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:13.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.133+0000 7f4a38a00640 1 -- 172.21.15.16:0/173725448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cd7eb5390 msgr2=0x563cd8287ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:13.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.133+0000 7f4a38a00640 1 --2- 172.21.15.16:0/173725448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cd7eb5390 0x563cd8287ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:13.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.133+0000 7f4a38a00640 1 -- 172.21.15.16:0/173725448 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cd81a98b0 con 0x563cd7ead1b0 2024-08-31T06:14:13.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.133+0000 7f4a38a00640 1 --2- 172.21.15.16:0/173725448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cd7ead1b0 0x563cd8287700 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4a2800e9c0 tx=0x7f4a280ab430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca62dc62975d7c0f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:13.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.133+0000 7f4a1a000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:14:13.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.133+0000 7f4a1a000640 1 -- 172.21.15.16:0/173725448 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2801c040 con 0x563cd7ead1b0 2024-08-31T06:14:13.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.133+0000 7f4a4206f740 1 -- 172.21.15.16:0/173725448 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a2c036030 con 0x563cd7ead1b0 2024-08-31T06:14:13.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.133+0000 7f4a1a000640 1 -- 172.21.15.16:0/173725448 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8514+0+0 (secure 0 0 0) 0x7f4a280a7e20 con 0x563cd7ead1b0 2024-08-31T06:14:13.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.134+0000 7f4a1a000640 1 -- 172.21.15.16:0/173725448 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4a280abb90 con 0x563cd7ead1b0 2024-08-31T06:14:13.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.134+0000 7f4a4206f740 10 client.27113 fetch_fsmap learned FSMap version 39 2024-08-31T06:14:13.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.134+0000 7f4a4206f740 1 -- 172.21.15.16:0/173725448 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563cd8296220 con 0x563cd7ead1b0 2024-08-31T06:14:13.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.134+0000 7f4a1a000640 1 -- 172.21.15.16:0/173725448 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a2c036030 con 0x563cd7ead1b0 2024-08-31T06:14:13.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.134+0000 7f4a1a000640 1 -- 172.21.15.16:0/173725448 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f4a28094020 con 0x563cd7ead1b0 2024-08-31T06:14:13.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.134+0000 7f4a4206f740 10 client.27113 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T06:14:13.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.134+0000 7f4a4206f740 10 client.27113 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T06:14:13.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.134+0000 7f4a4206f740 20 client.27113 populate_metadata read hostname 'smithi016' 2024-08-31T06:14:13.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.134+0000 7f4a4206f740 1 --2- 172.21.15.16:0/173725448 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x563cd7e949f0 0x563cd82996f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:13.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.134+0000 7f4a4206f740 4 client.27113 mds_command: new command op to 24712 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:14:13.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.134+0000 7f4a4206f740 1 -- 172.21.15.16:0/173725448 --> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563cd81f4700 con 0x563cd7e949f0 2024-08-31T06:14:13.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.135+0000 7f4a33e00640 1 --2- 172.21.15.16:0/173725448 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x563cd7e949f0 0x563cd82996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:13.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.135+0000 7f4a33e00640 1 --2- 172.21.15.16:0/173725448 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x563cd7e949f0 0x563cd82996f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=18a5e6d88515df04 server_cookie=d88c838e802b08 in_seq=0 out_seq=0 2024-08-31T06:14:13.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.135+0000 7f4a1a000640 10 client.27113 ms_handle_connect on v2:172.21.15.184:6838/4011051479 2024-08-31T06:14:13.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.136+0000 7f4a1a000640 1 -- 172.21.15.16:0/173725448 <== mds.2 v2:172.21.15.184:6838/4011051479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563cd81f4700 con 0x563cd7e949f0 2024-08-31T06:14:13.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.136+0000 7f4a1a000640 10 client.27113 handle_command_reply: tid=0 2024-08-31T06:14:13.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.136+0000 7f4a4206f740 1 client.27113 shutdown 2024-08-31T06:14:13.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.136+0000 7f4a4206f740 1 -- 172.21.15.16:0/173725448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cd7ead1b0 msgr2=0x563cd8287700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:13.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.136+0000 7f4a4206f740 1 --2- 172.21.15.16:0/173725448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cd7ead1b0 0x563cd8287700 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4a2800e9c0 tx=0x7f4a280ab430 comp rx=0 tx=0).stop 2024-08-31T06:14:13.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.136+0000 7f4a4206f740 1 -- 172.21.15.16:0/173725448 shutdown_connections 2024-08-31T06:14:13.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.136+0000 7f4a4206f740 1 --2- 172.21.15.16:0/173725448 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x563cd7e949f0 0x563cd82996f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:13.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.136+0000 7f4a4206f740 1 --2- 172.21.15.16:0/173725448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563cd7ed6070 0x563cd828d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:13.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.136+0000 7f4a4206f740 1 --2- 172.21.15.16:0/173725448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cd7eb5390 0x563cd8287ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:13.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.136+0000 7f4a4206f740 1 --2- 172.21.15.16:0/173725448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cd7ead1b0 0x563cd8287700 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:13.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.136+0000 7f4a1a000640 0 client.27113 ms_handle_reset on v2:172.21.15.184:6838/4011051479 2024-08-31T06:14:13.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.136+0000 7f4a4206f740 1 -- 172.21.15.16:0/173725448 >> 172.21.15.16:0/173725448 conn(0x563cd7eca460 msgr2=0x563cd827ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:13.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.137+0000 7f4a4206f740 1 -- 172.21.15.16:0/173725448 shutdown_connections 2024-08-31T06:14:13.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.137+0000 7f4a4206f740 1 -- 172.21.15.16:0/173725448 wait complete. 2024-08-31T06:14:13.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.137+0000 7f4a4206f740 20 client.27113 trim_cache size 0 max 16384 2024-08-31T06:14:13.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.151+0000 7f4a4206f740 1 Processor -- start 2024-08-31T06:14:13.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.151+0000 7f4a4206f740 1 -- start start 2024-08-31T06:14:13.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.151+0000 7f4a4206f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563cd7ead1b0 0x563cd8279190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:13.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.151+0000 7f4a4206f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cd7eb5390 0x563cd82796d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:13.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.151+0000 7f4a4206f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cd7ed6070 0x563cd8284710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:13.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.151+0000 7f4a4206f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4a341b8170 con 0x563cd7ed6070 2024-08-31T06:14:13.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.151+0000 7f4a4206f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4a341b59e0 con 0x563cd7eb5390 2024-08-31T06:14:13.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.151+0000 7f4a4206f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4a341b3850 con 0x563cd7ead1b0 2024-08-31T06:14:13.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.152+0000 7f4a38a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563cd7ead1b0 0x563cd8279190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:13.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.152+0000 7f4a38a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563cd7ead1b0 0x563cd8279190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:38878/0 (socket says 172.21.15.16:38878) 2024-08-31T06:14:13.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.152+0000 7f4a38a00640 1 -- 172.21.15.16:0/711869110 learned_addr learned my addr 172.21.15.16:0/711869110 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:13.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.152+0000 7f4a33e00640 1 --2- 172.21.15.16:0/711869110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cd7eb5390 0x563cd82796d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:13.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.152+0000 7f4a39400640 1 --2- 172.21.15.16:0/711869110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cd7ed6070 0x563cd8284710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:13.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.152+0000 7f4a38a00640 1 -- 172.21.15.16:0/711869110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cd7eb5390 msgr2=0x563cd82796d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:13.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.152+0000 7f4a38a00640 1 --2- 172.21.15.16:0/711869110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cd7eb5390 0x563cd82796d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:13.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.152+0000 7f4a38a00640 1 -- 172.21.15.16:0/711869110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cd7ed6070 msgr2=0x563cd8284710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:13.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.152+0000 7f4a38a00640 1 --2- 172.21.15.16:0/711869110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cd7ed6070 0x563cd8284710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:13.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.152+0000 7f4a38a00640 1 -- 172.21.15.16:0/711869110 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cd827e8d0 con 0x563cd7ead1b0 2024-08-31T06:14:13.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.152+0000 7f4a33e00640 1 --2- 172.21.15.16:0/711869110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cd7eb5390 0x563cd82796d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:14:13.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.152+0000 7f4a39400640 1 --2- 172.21.15.16:0/711869110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cd7ed6070 0x563cd8284710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:14:13.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.152+0000 7f4a38a00640 1 --2- 172.21.15.16:0/711869110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563cd7ead1b0 0x563cd8279190 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4a280e5470 tx=0x7f4a280b02e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41f7c3e05adb8903 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:13.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.152+0000 7f4a25200640 1 -- 172.21.15.16:0/711869110 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2801c070 con 0x563cd7ead1b0 2024-08-31T06:14:13.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.152+0000 7f4a25200640 1 -- 172.21.15.16:0/711869110 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a280a7500 con 0x563cd7ead1b0 2024-08-31T06:14:13.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.153+0000 7f4a25200640 1 -- 172.21.15.16:0/711869110 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a280a7e20 con 0x563cd7ead1b0 2024-08-31T06:14:13.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.153+0000 7f4a4206f740 1 -- 172.21.15.16:0/711869110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563cd7ead1b0 msgr2=0x563cd8279190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:13.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.153+0000 7f4a4206f740 1 --2- 172.21.15.16:0/711869110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563cd7ead1b0 0x563cd8279190 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4a280e5470 tx=0x7f4a280b02e0 comp rx=0 tx=0).stop 2024-08-31T06:14:13.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.153+0000 7f4a4206f740 1 -- 172.21.15.16:0/711869110 shutdown_connections 2024-08-31T06:14:13.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.153+0000 7f4a4206f740 1 --2- 172.21.15.16:0/711869110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cd7ed6070 0x563cd8284710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:13.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.153+0000 7f4a4206f740 1 --2- 172.21.15.16:0/711869110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cd7eb5390 0x563cd82796d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:13.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.153+0000 7f4a4206f740 1 --2- 172.21.15.16:0/711869110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563cd7ead1b0 0x563cd8279190 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:13.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.153+0000 7f4a4206f740 1 -- 172.21.15.16:0/711869110 >> 172.21.15.16:0/711869110 conn(0x563cd7eca460 msgr2=0x563cd8285d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:13.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.153+0000 7f4a4206f740 1 -- 172.21.15.16:0/711869110 shutdown_connections 2024-08-31T06:14:13.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.154+0000 7f4a4206f740 1 -- 172.21.15.16:0/711869110 wait complete. 2024-08-31T06:14:13.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.154+0000 7f4a4206f740 1 -- start start 2024-08-31T06:14:13.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.154+0000 7f4a4206f740 1 Processor -- start 2024-08-31T06:14:13.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.155+0000 7f4a4206f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563cd7ead1b0 0x563cd828e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:13.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.155+0000 7f4a4206f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cd7eb5390 0x563cd828ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:13.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.155+0000 7f4a4206f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cd7ed6070 0x563cd828f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:13.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.155+0000 7f4a4206f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4a341b3ea0 con 0x563cd7ed6070 2024-08-31T06:14:13.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.155+0000 7f4a4206f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4a341b5310 con 0x563cd7eb5390 2024-08-31T06:14:13.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.155+0000 7f4a4206f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4a341b4570 con 0x563cd7ead1b0 2024-08-31T06:14:13.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.155+0000 7f4a4206f740 10 client.0 _ll_register_callbacks cb 0x7f4a280ab550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:14:13.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.155+0000 7f4a38a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563cd7ead1b0 0x563cd828e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:13.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.155+0000 7f4a38a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563cd7ead1b0 0x563cd828e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:38888/0 (socket says 172.21.15.16:38888) 2024-08-31T06:14:13.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.155+0000 7f4a38a00640 1 -- 172.21.15.16:0/727907264 learned_addr learned my addr 172.21.15.16:0/727907264 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:13.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.155+0000 7f4a33e00640 1 --2- 172.21.15.16:0/727907264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cd7eb5390 0x563cd828ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:13.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.155+0000 7f4a39400640 1 --2- 172.21.15.16:0/727907264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cd7ed6070 0x563cd828f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:13.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.155+0000 7f4a33e00640 1 -- 172.21.15.16:0/727907264 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563cd7ead1b0 msgr2=0x563cd828e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:13.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.155+0000 7f4a33e00640 1 --2- 172.21.15.16:0/727907264 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563cd7ead1b0 0x563cd828e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:13.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.155+0000 7f4a33e00640 1 -- 172.21.15.16:0/727907264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cd7ed6070 msgr2=0x563cd828f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:13.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.155+0000 7f4a33e00640 1 --2- 172.21.15.16:0/727907264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cd7ed6070 0x563cd828f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:13.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.155+0000 7f4a33e00640 1 -- 172.21.15.16:0/727907264 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cd83040e0 con 0x563cd7eb5390 2024-08-31T06:14:13.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.155+0000 7f4a38a00640 1 --2- 172.21.15.16:0/727907264 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563cd7ead1b0 0x563cd828e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:14:13.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.156+0000 7f4a33e00640 1 --2- 172.21.15.16:0/727907264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cd7eb5390 0x563cd828ec60 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x563cd8299e00 tx=0x7f4a200046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0913e1d7cc82165 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:13.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.156+0000 7f4a1a000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:14:13.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.156+0000 7f4a1a000640 1 -- 172.21.15.16:0/727907264 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a20003de0 con 0x563cd7eb5390 2024-08-31T06:14:13.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.156+0000 7f4a1a000640 1 -- 172.21.15.16:0/727907264 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8514+0+0 (secure 0 0 0) 0x7f4a2000fa10 con 0x563cd7eb5390 2024-08-31T06:14:13.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.156+0000 7f4a4206f740 1 -- 172.21.15.16:0/727907264 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a280a7500 con 0x563cd7eb5390 2024-08-31T06:14:13.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.156+0000 7f4a1a000640 1 -- 172.21.15.16:0/727907264 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4a2000fc10 con 0x563cd7eb5390 2024-08-31T06:14:13.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.156+0000 7f4a4206f740 10 client.27119 fetch_fsmap learned FSMap version 39 2024-08-31T06:14:13.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.156+0000 7f4a4206f740 1 -- 172.21.15.16:0/727907264 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563cd8305360 con 0x563cd7eb5390 2024-08-31T06:14:13.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.157+0000 7f4a1a000640 1 -- 172.21.15.16:0/727907264 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a280a7500 con 0x563cd7eb5390 2024-08-31T06:14:13.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.157+0000 7f4a1a000640 1 -- 172.21.15.16:0/727907264 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f4a20039400 con 0x563cd7eb5390 2024-08-31T06:14:13.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.157+0000 7f4a4206f740 10 client.27119 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T06:14:13.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.157+0000 7f4a4206f740 10 client.27119 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T06:14:13.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.157+0000 7f4a4206f740 20 client.27119 populate_metadata read hostname 'smithi016' 2024-08-31T06:14:13.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.157+0000 7f4a4206f740 1 --2- 172.21.15.16:0/727907264 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x563cd7e69da0 0x563cd8307ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:13.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.157+0000 7f4a4206f740 4 client.27119 mds_command: new command op to 24712 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:14:13.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.157+0000 7f4a4206f740 1 -- 172.21.15.16:0/727907264 --> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563cd81f01a0 con 0x563cd7e69da0 2024-08-31T06:14:13.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.157+0000 7f4a38a00640 1 --2- 172.21.15.16:0/727907264 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x563cd7e69da0 0x563cd8307ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:13.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.158+0000 7f4a38a00640 1 --2- 172.21.15.16:0/727907264 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x563cd7e69da0 0x563cd8307ac0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=241d37c6e3fd5b38 server_cookie=68f76c00f3cb6f34 in_seq=0 out_seq=0 2024-08-31T06:14:13.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.158+0000 7f4a1a000640 10 client.27119 ms_handle_connect on v2:172.21.15.184:6838/4011051479 2024-08-31T06:14:13.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.158+0000 7f4a1a000640 1 -- 172.21.15.16:0/727907264 <== mds.2 v2:172.21.15.184:6838/4011051479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563cd81f01a0 con 0x563cd7e69da0 2024-08-31T06:14:13.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.158+0000 7f4a1a000640 10 client.27119 handle_command_reply: tid=0 2024-08-31T06:14:13.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.158+0000 7f4a4206f740 1 client.27119 shutdown 2024-08-31T06:14:13.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.158+0000 7f4a4206f740 1 -- 172.21.15.16:0/727907264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cd7eb5390 msgr2=0x563cd828ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:13.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.158+0000 7f4a4206f740 1 --2- 172.21.15.16:0/727907264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cd7eb5390 0x563cd828ec60 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x563cd8299e00 tx=0x7f4a200046c0 comp rx=0 tx=0).stop 2024-08-31T06:14:13.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.158+0000 7f4a4206f740 1 -- 172.21.15.16:0/727907264 shutdown_connections 2024-08-31T06:14:13.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.158+0000 7f4a4206f740 1 --2- 172.21.15.16:0/727907264 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x563cd7e69da0 0x563cd8307ac0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:13.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.158+0000 7f4a4206f740 1 --2- 172.21.15.16:0/727907264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cd7ed6070 0x563cd828f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:13.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.158+0000 7f4a4206f740 1 --2- 172.21.15.16:0/727907264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cd7eb5390 0x563cd828ec60 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:13.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.158+0000 7f4a4206f740 1 --2- 172.21.15.16:0/727907264 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563cd7ead1b0 0x563cd828e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:13.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.158+0000 7f4a1a000640 0 client.27119 ms_handle_reset on v2:172.21.15.184:6838/4011051479 2024-08-31T06:14:13.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.158+0000 7f4a4206f740 1 -- 172.21.15.16:0/727907264 >> 172.21.15.16:0/727907264 conn(0x563cd7eca460 msgr2=0x563cd8279190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:13.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.159+0000 7f4a4206f740 1 -- 172.21.15.16:0/727907264 shutdown_connections 2024-08-31T06:14:13.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.159+0000 7f4a4206f740 1 -- 172.21.15.16:0/727907264 wait complete. 2024-08-31T06:14:13.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.159+0000 7f4a4206f740 20 client.27119 trim_cache size 0 max 16384 2024-08-31T06:14:13.164 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:14:13.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.159+0000 7f4a39e00640 1 -- 172.21.15.16:0/1763996877 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f4a1c0b4bb0 msgr2=0x7f4a1c0b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:13.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.159+0000 7f4a39e00640 1 --2- 172.21.15.16:0/1763996877 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f4a1c0b4bb0 0x7f4a1c0b70c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4a34079e60 tx=0x7f4a20005f90 comp rx=0 tx=0).stop 2024-08-31T06:14:13.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.159+0000 7f4a39e00640 1 -- 172.21.15.16:0/1763996877 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a340776e0 msgr2=0x7f4a341a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:13.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.159+0000 7f4a39e00640 1 --2- 172.21.15.16:0/1763996877 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a340776e0 0x7f4a341a06c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4a2800e9f0 tx=0x7f4a2800eeb0 comp rx=0 tx=0).stop 2024-08-31T06:14:13.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.160+0000 7f4a39e00640 1 -- 172.21.15.16:0/1763996877 shutdown_connections 2024-08-31T06:14:13.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.160+0000 7f4a39e00640 1 --2- 172.21.15.16:0/1763996877 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f4a1c0b4bb0 0x7f4a1c0b70c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:13.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.160+0000 7f4a39e00640 1 --2- 172.21.15.16:0/1763996877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a34114310 0x7f4a341a5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:13.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.160+0000 7f4a39e00640 1 --2- 172.21.15.16:0/1763996877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a34078030 0x7f4a341a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:13.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.160+0000 7f4a39e00640 1 --2- 172.21.15.16:0/1763996877 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a340776e0 0x7f4a341a06c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:13.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.160+0000 7f4a39e00640 1 -- 172.21.15.16:0/1763996877 >> 172.21.15.16:0/1763996877 conn(0x7f4a34101420 msgr2=0x7f4a34116360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:13.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.160+0000 7f4a39e00640 1 -- 172.21.15.16:0/1763996877 shutdown_connections 2024-08-31T06:14:13.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:13.160+0000 7f4a39e00640 1 -- 172.21.15.16:0/1763996877 wait complete. 2024-08-31T06:14:14.219 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T06:14:14.598 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:14:14.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:14 smithi155 ceph-mon[21742]: pgmap v577: 97 pgs: 97 active+clean; 3.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 25 MiB/s wr, 1.78k op/s 2024-08-31T06:14:14.895 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:14 smithi016 ceph-mon[17194]: pgmap v577: 97 pgs: 97 active+clean; 3.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 25 MiB/s wr, 1.78k op/s 2024-08-31T06:14:14.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:14 smithi184 ceph-mon[22331]: pgmap v577: 97 pgs: 97 active+clean; 3.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 25 MiB/s wr, 1.78k op/s 2024-08-31T06:14:15.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.246+0000 7f0e53a00640 1 -- 172.21.15.16:0/3291621244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e4c077120 msgr2=0x7f0e4c075550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:15.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.246+0000 7f0e53a00640 1 --2- 172.21.15.16:0/3291621244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e4c077120 0x7f0e4c075550 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c009900 tx=0x7f0e3c02a3f0 comp rx=0 tx=0).stop 2024-08-31T06:14:15.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.246+0000 7f0e53a00640 1 -- 172.21.15.16:0/3291621244 shutdown_connections 2024-08-31T06:14:15.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.246+0000 7f0e53a00640 1 --2- 172.21.15.16:0/3291621244 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e4c075b70 0x7f0e4c075f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:15.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.246+0000 7f0e53a00640 1 --2- 172.21.15.16:0/3291621244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e4c1141d0 0x7f0e4c1166a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:15.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.246+0000 7f0e53a00640 1 --2- 172.21.15.16:0/3291621244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e4c077120 0x7f0e4c075550 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:15.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.246+0000 7f0e53a00640 1 -- 172.21.15.16:0/3291621244 >> 172.21.15.16:0/3291621244 conn(0x7f0e4c07b5f0 msgr2=0x7f0e4c07ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:15.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.246+0000 7f0e53a00640 1 -- 172.21.15.16:0/3291621244 shutdown_connections 2024-08-31T06:14:15.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.247+0000 7f0e53a00640 1 -- 172.21.15.16:0/3291621244 wait complete. 2024-08-31T06:14:15.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.248+0000 7f0e53a00640 1 Processor -- start 2024-08-31T06:14:15.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.248+0000 7f0e53a00640 1 -- start start 2024-08-31T06:14:15.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.248+0000 7f0e53a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e4c075b70 0x7f0e4c1a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:15.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.248+0000 7f0e53a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e4c077120 0x7f0e4c1a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:15.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.248+0000 7f0e53a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e4c1141d0 0x7f0e4c1a6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:15.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.248+0000 7f0e53a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4c119e10 con 0x7f0e4c075b70 2024-08-31T06:14:15.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.248+0000 7f0e53a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4c119c90 con 0x7f0e4c077120 2024-08-31T06:14:15.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.248+0000 7f0e53a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4c119f90 con 0x7f0e4c1141d0 2024-08-31T06:14:15.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.248+0000 7f0e51000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e4c075b70 0x7f0e4c1a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:15.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.249+0000 7f0e51000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e4c075b70 0x7f0e4c1a0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36228/0 (socket says 172.21.15.16:36228) 2024-08-31T06:14:15.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.249+0000 7f0e51000640 1 -- 172.21.15.16:0/654934282 learned_addr learned my addr 172.21.15.16:0/654934282 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:15.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.249+0000 7f0e43e00640 1 --2- 172.21.15.16:0/654934282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e4c077120 0x7f0e4c1a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:15.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.249+0000 7f0e51a00640 1 --2- 172.21.15.16:0/654934282 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e4c1141d0 0x7f0e4c1a6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:15.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.249+0000 7f0e51a00640 1 -- 172.21.15.16:0/654934282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e4c077120 msgr2=0x7f0e4c1a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:15.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.249+0000 7f0e51a00640 1 --2- 172.21.15.16:0/654934282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e4c077120 0x7f0e4c1a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:15.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.249+0000 7f0e51a00640 1 -- 172.21.15.16:0/654934282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e4c075b70 msgr2=0x7f0e4c1a0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:15.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.249+0000 7f0e51a00640 1 --2- 172.21.15.16:0/654934282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e4c075b70 0x7f0e4c1a0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:15.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.249+0000 7f0e51a00640 1 -- 172.21.15.16:0/654934282 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e4c1a6770 con 0x7f0e4c1141d0 2024-08-31T06:14:15.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.249+0000 7f0e51000640 1 --2- 172.21.15.16:0/654934282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e4c075b70 0x7f0e4c1a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:14:15.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.249+0000 7f0e51a00640 1 --2- 172.21.15.16:0/654934282 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e4c1141d0 0x7f0e4c1a6040 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0e4400da10 tx=0x7f0e4400ded0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:15.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.250+0000 7f0e4a000640 1 -- 172.21.15.16:0/654934282 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4400bd70 con 0x7f0e4c1141d0 2024-08-31T06:14:15.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.250+0000 7f0e53a00640 1 -- 172.21.15.16:0/654934282 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e4c1a6a60 con 0x7f0e4c1141d0 2024-08-31T06:14:15.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.250+0000 7f0e4a000640 1 -- 172.21.15.16:0/654934282 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e44003e20 con 0x7f0e4c1141d0 2024-08-31T06:14:15.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.250+0000 7f0e4a000640 1 -- 172.21.15.16:0/654934282 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e440106c0 con 0x7f0e4c1141d0 2024-08-31T06:14:15.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.250+0000 7f0e53a00640 1 -- 172.21.15.16:0/654934282 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e4c1ad1c0 con 0x7f0e4c1141d0 2024-08-31T06:14:15.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.253+0000 7f0e5a652740 1 Processor -- start 2024-08-31T06:14:15.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.254+0000 7f0e4a000640 1 -- 172.21.15.16:0/654934282 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f0e44004270 con 0x7f0e4c1141d0 2024-08-31T06:14:15.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.254+0000 7f0e5a652740 1 -- start start 2024-08-31T06:14:15.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.254+0000 7f0e5a652740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c47b8f070 0x558c47f34990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:15.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.254+0000 7f0e5a652740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c47b661b0 0x558c47f34ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:15.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.254+0000 7f0e5a652740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c47b6e390 0x558c47f35410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:15.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.254+0000 7f0e5a652740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558c47eab7b0 con 0x558c47b661b0 2024-08-31T06:14:15.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.254+0000 7f0e5a652740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x558c47e9ad40 con 0x558c47b6e390 2024-08-31T06:14:15.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.254+0000 7f0e5a652740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558c47e9ecf0 con 0x558c47b8f070 2024-08-31T06:14:15.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.255+0000 7f0e51000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c47b661b0 0x558c47f34ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:15.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.255+0000 7f0e43e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c47b8f070 0x558c47f34990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:15.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.255+0000 7f0e51000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c47b661b0 0x558c47f34ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36238/0 (socket says 172.21.15.16:36238) 2024-08-31T06:14:15.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.255+0000 7f0e51000640 1 -- 172.21.15.16:0/2079822064 learned_addr learned my addr 172.21.15.16:0/2079822064 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:15.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.255+0000 7f0e43e00640 1 --2- 172.21.15.16:0/2079822064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c47b6e390 0x558c47f35410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:15.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.255+0000 7f0e51000640 1 -- 172.21.15.16:0/2079822064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c47b8f070 msgr2=0x558c47f34990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:15.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.255+0000 7f0e51000640 1 --2- 172.21.15.16:0/2079822064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c47b8f070 0x558c47f34990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:15.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.255+0000 7f0e51000640 1 -- 172.21.15.16:0/2079822064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c47b6e390 msgr2=0x558c47f35410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:15.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.255+0000 7f0e51000640 1 --2- 172.21.15.16:0/2079822064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c47b6e390 0x558c47f35410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:15.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.255+0000 7f0e51000640 1 -- 172.21.15.16:0/2079822064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c47bfa650 con 0x558c47b661b0 2024-08-31T06:14:15.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.256+0000 7f0e51000640 1 --2- 172.21.15.16:0/2079822064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c47b661b0 0x558c47f34ed0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c000c00 tx=0x7f0e3c0314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c0f64d15d0cb445 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:15.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.256+0000 7f0e4a000640 1 --2- 172.21.15.16:0/654934282 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f0e300b4d20 0x7f0e300b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:15.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.256+0000 7f0e42000640 1 -- 172.21.15.16:0/2079822064 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c035040 con 0x558c47b661b0 2024-08-31T06:14:15.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.256+0000 7f0e42000640 1 -- 172.21.15.16:0/2079822064 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e3c031890 con 0x558c47b661b0 2024-08-31T06:14:15.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.256+0000 7f0e4a000640 1 -- 172.21.15.16:0/654934282 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8514+0+0 (secure 0 0 0) 0x7f0e440e58f0 con 0x7f0e4c1141d0 2024-08-31T06:14:15.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.256+0000 7f0e42000640 1 -- 172.21.15.16:0/2079822064 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c004a00 con 0x558c47b661b0 2024-08-31T06:14:15.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.256+0000 7f0e43e00640 1 --2- 172.21.15.16:0/654934282 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f0e300b4d20 0x7f0e300b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:15.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.257+0000 7f0e5a652740 1 -- 172.21.15.16:0/2079822064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c47b661b0 msgr2=0x558c47f34ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:15.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.257+0000 7f0e43e00640 1 --2- 172.21.15.16:0/654934282 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f0e300b4d20 0x7f0e300b7230 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0e34009fd0 tx=0x7f0e34005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:15.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.257+0000 7f0e5a652740 1 --2- 172.21.15.16:0/2079822064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c47b661b0 0x558c47f34ed0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c000c00 tx=0x7f0e3c0314a0 comp rx=0 tx=0).stop 2024-08-31T06:14:15.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.257+0000 7f0e5a652740 1 -- 172.21.15.16:0/2079822064 shutdown_connections 2024-08-31T06:14:15.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.257+0000 7f0e5a652740 1 --2- 172.21.15.16:0/2079822064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c47b6e390 0x558c47f35410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:15.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.257+0000 7f0e5a652740 1 --2- 172.21.15.16:0/2079822064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c47b661b0 0x558c47f34ed0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:15.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.257+0000 7f0e5a652740 1 --2- 172.21.15.16:0/2079822064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c47b8f070 0x558c47f34990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:15.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.257+0000 7f0e5a652740 1 -- 172.21.15.16:0/2079822064 >> 172.21.15.16:0/2079822064 conn(0x558c47b83460 msgr2=0x558c47f32880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:15.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.257+0000 7f0e5a652740 1 -- 172.21.15.16:0/2079822064 shutdown_connections 2024-08-31T06:14:15.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.257+0000 7f0e5a652740 1 -- 172.21.15.16:0/2079822064 wait complete. 2024-08-31T06:14:15.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.258+0000 7f0e5a652740 1 -- start start 2024-08-31T06:14:15.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.258+0000 7f0e5a652740 1 Processor -- start 2024-08-31T06:14:15.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.259+0000 7f0e5a652740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c47b661b0 0x558c47f40540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:15.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.259+0000 7f0e5a652740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c47b6e390 0x558c47f40a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:15.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.259+0000 7f0e5a652740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c47b8f070 0x558c47f4c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:15.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.259+0000 7f0e5a652740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558c47cfd700 con 0x558c47b661b0 2024-08-31T06:14:15.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.259+0000 7f0e5a652740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x558c47e21cf0 con 0x558c47b8f070 2024-08-31T06:14:15.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.259+0000 7f0e5a652740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558c47f43110 con 0x558c47b6e390 2024-08-31T06:14:15.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.259+0000 7f0e5a652740 10 client.0 _ll_register_callbacks cb 0x558c47e47470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:14:15.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.259+0000 7f0e51000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c47b661b0 0x558c47f40540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:15.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.259+0000 7f0e51a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c47b8f070 0x558c47f4c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:15.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.259+0000 7f0e51000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c47b661b0 0x558c47f40540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36248/0 (socket says 172.21.15.16:36248) 2024-08-31T06:14:15.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.259+0000 7f0e51000640 1 -- 172.21.15.16:0/3195072905 learned_addr learned my addr 172.21.15.16:0/3195072905 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:15.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.259+0000 7f0e43e00640 1 --2- 172.21.15.16:0/3195072905 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c47b6e390 0x558c47f40a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:15.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.259+0000 7f0e51a00640 1 -- 172.21.15.16:0/3195072905 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c47b6e390 msgr2=0x558c47f40a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:15.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.259+0000 7f0e51a00640 1 --2- 172.21.15.16:0/3195072905 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c47b6e390 0x558c47f40a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:15.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.259+0000 7f0e51a00640 1 -- 172.21.15.16:0/3195072905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c47b661b0 msgr2=0x558c47f40540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:15.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.259+0000 7f0e51a00640 1 --2- 172.21.15.16:0/3195072905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c47b661b0 0x558c47f40540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:15.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.259+0000 7f0e51a00640 1 -- 172.21.15.16:0/3195072905 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c47e628b0 con 0x558c47b8f070 2024-08-31T06:14:15.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.259+0000 7f0e51000640 1 --2- 172.21.15.16:0/3195072905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c47b661b0 0x558c47f40540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:14:15.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.260+0000 7f0e51a00640 1 --2- 172.21.15.16:0/3195072905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c47b8f070 0x558c47f4c400 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0e440e2fe0 tx=0x7f0e440ac680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cbbb7aedd430b4e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:15.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.260+0000 7f0e3aa00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:14:15.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.260+0000 7f0e3aa00640 1 -- 172.21.15.16:0/3195072905 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e440b52a0 con 0x558c47b8f070 2024-08-31T06:14:15.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.260+0000 7f0e5a652740 1 -- 172.21.15.16:0/3195072905 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e3c031890 con 0x558c47b8f070 2024-08-31T06:14:15.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.260+0000 7f0e3aa00640 1 -- 172.21.15.16:0/3195072905 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8514+0+0 (secure 0 0 0) 0x7f0e440a8210 con 0x558c47b8f070 2024-08-31T06:14:15.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.260+0000 7f0e3aa00640 1 -- 172.21.15.16:0/3195072905 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0e4405a050 con 0x558c47b8f070 2024-08-31T06:14:15.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.260+0000 7f0e5a652740 10 client.27131 fetch_fsmap learned FSMap version 39 2024-08-31T06:14:15.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.260+0000 7f0e5a652740 1 -- 172.21.15.16:0/3195072905 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558c47f4f1d0 con 0x558c47b8f070 2024-08-31T06:14:15.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.261+0000 7f0e3aa00640 1 -- 172.21.15.16:0/3195072905 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e3c031890 con 0x558c47b8f070 2024-08-31T06:14:15.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.261+0000 7f0e3aa00640 1 -- 172.21.15.16:0/3195072905 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f0e44024060 con 0x558c47b8f070 2024-08-31T06:14:15.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.261+0000 7f0e5a652740 10 client.27131 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T06:14:15.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.261+0000 7f0e5a652740 10 client.27131 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-31T06:14:15.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.261+0000 7f0e5a652740 20 client.27131 populate_metadata read hostname 'smithi016' 2024-08-31T06:14:15.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.261+0000 7f0e5a652740 1 --2- 172.21.15.16:0/3195072905 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x558c47b4d9f0 0x558c47f52720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:15.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.261+0000 7f0e5a652740 4 client.27131 mds_command: new command op to 24724 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:14:15.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.261+0000 7f0e5a652740 1 -- 172.21.15.16:0/3195072905 --> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c47ead700 con 0x558c47b4d9f0 2024-08-31T06:14:15.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.261+0000 7f0e51000640 1 --2- 172.21.15.16:0/3195072905 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x558c47b4d9f0 0x558c47f52720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:15.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.262+0000 7f0e3aa00640 10 client.27131 ms_handle_connect on v2:172.21.15.16:6838/219664460 2024-08-31T06:14:15.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.262+0000 7f0e51000640 1 --2- 172.21.15.16:0/3195072905 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x558c47b4d9f0 0x558c47f52720 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f8d06942921f462 server_cookie=5b378ff0ccb952d4 in_seq=0 out_seq=0 2024-08-31T06:14:15.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.263+0000 7f0e3aa00640 1 -- 172.21.15.16:0/3195072905 <== mds.1 v2:172.21.15.16:6838/219664460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558c47ead700 con 0x558c47b4d9f0 2024-08-31T06:14:15.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.263+0000 7f0e3aa00640 10 client.27131 handle_command_reply: tid=0 2024-08-31T06:14:15.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.263+0000 7f0e5a652740 1 client.27131 shutdown 2024-08-31T06:14:15.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.263+0000 7f0e5a652740 1 -- 172.21.15.16:0/3195072905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c47b8f070 msgr2=0x558c47f4c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:15.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.263+0000 7f0e5a652740 1 --2- 172.21.15.16:0/3195072905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c47b8f070 0x558c47f4c400 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0e440e2fe0 tx=0x7f0e440ac680 comp rx=0 tx=0).stop 2024-08-31T06:14:15.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.263+0000 7f0e5a652740 1 -- 172.21.15.16:0/3195072905 shutdown_connections 2024-08-31T06:14:15.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.263+0000 7f0e5a652740 1 --2- 172.21.15.16:0/3195072905 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c47b6e390 0x558c47f40a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:15.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.263+0000 7f0e5a652740 1 --2- 172.21.15.16:0/3195072905 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x558c47b4d9f0 0x558c47f52720 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:15.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.263+0000 7f0e5a652740 1 --2- 172.21.15.16:0/3195072905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c47b8f070 0x558c47f4c400 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:15.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.263+0000 7f0e5a652740 1 --2- 172.21.15.16:0/3195072905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c47b661b0 0x558c47f40540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:15.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.263+0000 7f0e3aa00640 0 client.27131 ms_handle_reset on v2:172.21.15.16:6838/219664460 2024-08-31T06:14:15.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.263+0000 7f0e5a652740 1 -- 172.21.15.16:0/3195072905 >> 172.21.15.16:0/3195072905 conn(0x558c47b83460 msgr2=0x558c47f37b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:15.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.263+0000 7f0e5a652740 1 -- 172.21.15.16:0/3195072905 shutdown_connections 2024-08-31T06:14:15.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.264+0000 7f0e5a652740 1 -- 172.21.15.16:0/3195072905 wait complete. 2024-08-31T06:14:15.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.264+0000 7f0e5a652740 20 client.27131 trim_cache size 0 max 16384 2024-08-31T06:14:15.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.278+0000 7f0e5a652740 1 Processor -- start 2024-08-31T06:14:15.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.278+0000 7f0e5a652740 1 -- start start 2024-08-31T06:14:15.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.278+0000 7f0e5a652740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c47b661b0 0x558c47f3baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:15.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.278+0000 7f0e5a652740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c47b6e390 0x558c47f41f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:15.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.278+0000 7f0e5a652740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c47b8f070 0x558c47f31e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:15.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.278+0000 7f0e5a652740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4c1b8370 con 0x558c47b6e390 2024-08-31T06:14:15.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.278+0000 7f0e5a652740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4c1b5be0 con 0x558c47b8f070 2024-08-31T06:14:15.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.278+0000 7f0e5a652740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4c1b3a50 con 0x558c47b661b0 2024-08-31T06:14:15.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.278+0000 7f0e43e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c47b6e390 0x558c47f41f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:15.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.278+0000 7f0e43e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c47b6e390 0x558c47f41f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36250/0 (socket says 172.21.15.16:36250) 2024-08-31T06:14:15.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.278+0000 7f0e43e00640 1 -- 172.21.15.16:0/661285699 learned_addr learned my addr 172.21.15.16:0/661285699 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:15.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.279+0000 7f0e51000640 1 --2- 172.21.15.16:0/661285699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c47b661b0 0x558c47f3baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:15.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.279+0000 7f0e51a00640 1 --2- 172.21.15.16:0/661285699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c47b8f070 0x558c47f31e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:15.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.279+0000 7f0e43e00640 1 -- 172.21.15.16:0/661285699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c47b661b0 msgr2=0x558c47f3baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:15.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.279+0000 7f0e43e00640 1 --2- 172.21.15.16:0/661285699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c47b661b0 0x558c47f3baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:15.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.279+0000 7f0e43e00640 1 -- 172.21.15.16:0/661285699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c47b8f070 msgr2=0x558c47f31e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:15.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.279+0000 7f0e43e00640 1 --2- 172.21.15.16:0/661285699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c47b8f070 0x558c47f31e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:15.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.279+0000 7f0e43e00640 1 -- 172.21.15.16:0/661285699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c47f3af50 con 0x558c47b6e390 2024-08-31T06:14:15.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.279+0000 7f0e51a00640 1 --2- 172.21.15.16:0/661285699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c47b8f070 0x558c47f31e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:14:15.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.279+0000 7f0e43e00640 1 --2- 172.21.15.16:0/661285699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c47b6e390 0x558c47f41f70 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0e340115a0 tx=0x7f0e34004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d5c6e8c8992ee server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:15.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.279+0000 7f0e41600640 1 -- 172.21.15.16:0/661285699 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e34011d10 con 0x558c47b6e390 2024-08-31T06:14:15.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.280+0000 7f0e41600640 1 -- 172.21.15.16:0/661285699 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e340054d0 con 0x558c47b6e390 2024-08-31T06:14:15.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.280+0000 7f0e41600640 1 -- 172.21.15.16:0/661285699 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3402c460 con 0x558c47b6e390 2024-08-31T06:14:15.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.280+0000 7f0e5a652740 1 -- 172.21.15.16:0/661285699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c47b6e390 msgr2=0x558c47f41f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:15.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.280+0000 7f0e5a652740 1 --2- 172.21.15.16:0/661285699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c47b6e390 0x558c47f41f70 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0e340115a0 tx=0x7f0e34004730 comp rx=0 tx=0).stop 2024-08-31T06:14:15.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.280+0000 7f0e5a652740 1 -- 172.21.15.16:0/661285699 shutdown_connections 2024-08-31T06:14:15.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.280+0000 7f0e5a652740 1 --2- 172.21.15.16:0/661285699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c47b8f070 0x558c47f31e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:15.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.280+0000 7f0e5a652740 1 --2- 172.21.15.16:0/661285699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c47b6e390 0x558c47f41f70 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:15.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.280+0000 7f0e5a652740 1 --2- 172.21.15.16:0/661285699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c47b661b0 0x558c47f3baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:15.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.280+0000 7f0e5a652740 1 -- 172.21.15.16:0/661285699 >> 172.21.15.16:0/661285699 conn(0x558c47b83460 msgr2=0x558c47f41860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:15.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.281+0000 7f0e5a652740 1 -- 172.21.15.16:0/661285699 shutdown_connections 2024-08-31T06:14:15.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.281+0000 7f0e5a652740 1 -- 172.21.15.16:0/661285699 wait complete. 2024-08-31T06:14:15.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.281+0000 7f0e5a652740 1 -- start start 2024-08-31T06:14:15.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.281+0000 7f0e5a652740 1 Processor -- start 2024-08-31T06:14:15.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.282+0000 7f0e5a652740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c47b661b0 0x558c47f44300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:15.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.282+0000 7f0e5a652740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c47b8f070 0x558c47f44840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:15.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.282+0000 7f0e5a652740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c47b22da0 0x558c47f44d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:15.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.282+0000 7f0e5a652740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4c1b40a0 con 0x558c47b661b0 2024-08-31T06:14:15.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.282+0000 7f0e5a652740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4c1b5510 con 0x558c47b22da0 2024-08-31T06:14:15.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.282+0000 7f0e5a652740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4c1b4770 con 0x558c47b8f070 2024-08-31T06:14:15.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.282+0000 7f0e5a652740 10 client.0 _ll_register_callbacks cb 0x7f0e440ab710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:14:15.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.282+0000 7f0e51000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c47b661b0 0x558c47f44300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:15.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.282+0000 7f0e51000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c47b661b0 0x558c47f44300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36254/0 (socket says 172.21.15.16:36254) 2024-08-31T06:14:15.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.282+0000 7f0e51000640 1 -- 172.21.15.16:0/3583576625 learned_addr learned my addr 172.21.15.16:0/3583576625 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:15.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.282+0000 7f0e43e00640 1 --2- 172.21.15.16:0/3583576625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c47b8f070 0x558c47f44840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:15.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.282+0000 7f0e51a00640 1 --2- 172.21.15.16:0/3583576625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c47b22da0 0x558c47f44d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:15.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.283+0000 7f0e51000640 1 -- 172.21.15.16:0/3583576625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c47b8f070 msgr2=0x558c47f44840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:15.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.283+0000 7f0e51000640 1 --2- 172.21.15.16:0/3583576625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c47b8f070 0x558c47f44840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:15.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.283+0000 7f0e51000640 1 -- 172.21.15.16:0/3583576625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c47b22da0 msgr2=0x558c47f44d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:15.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.283+0000 7f0e51000640 1 --2- 172.21.15.16:0/3583576625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c47b22da0 0x558c47f44d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:15.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.283+0000 7f0e51000640 1 -- 172.21.15.16:0/3583576625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c47fbc0e0 con 0x558c47b661b0 2024-08-31T06:14:15.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.283+0000 7f0e51000640 1 --2- 172.21.15.16:0/3583576625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c47b661b0 0x558c47f44300 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c009a20 tx=0x7f0e3c005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faddcf7085485b7b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:15.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.283+0000 7f0e3aa00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:14:15.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.283+0000 7f0e3aa00640 1 -- 172.21.15.16:0/3583576625 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c035040 con 0x558c47b661b0 2024-08-31T06:14:15.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.283+0000 7f0e3aa00640 1 -- 172.21.15.16:0/3583576625 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8514+0+0 (secure 0 0 0) 0x7f0e3c004450 con 0x558c47b661b0 2024-08-31T06:14:15.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.283+0000 7f0e5a652740 1 -- 172.21.15.16:0/3583576625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e340054d0 con 0x558c47b661b0 2024-08-31T06:14:15.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.284+0000 7f0e3aa00640 1 -- 172.21.15.16:0/3583576625 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0e3c009520 con 0x558c47b661b0 2024-08-31T06:14:15.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.284+0000 7f0e5a652740 10 client.16905 fetch_fsmap learned FSMap version 39 2024-08-31T06:14:15.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.284+0000 7f0e5a652740 1 -- 172.21.15.16:0/3583576625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558c47fbd330 con 0x558c47b661b0 2024-08-31T06:14:15.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.284+0000 7f0e3aa00640 1 -- 172.21.15.16:0/3583576625 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e340054d0 con 0x558c47b661b0 2024-08-31T06:14:15.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.284+0000 7f0e3aa00640 1 -- 172.21.15.16:0/3583576625 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f0e3c0526b0 con 0x558c47b661b0 2024-08-31T06:14:15.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.284+0000 7f0e5a652740 10 client.16905 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T06:14:15.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.284+0000 7f0e5a652740 10 client.16905 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-31T06:14:15.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.284+0000 7f0e5a652740 20 client.16905 populate_metadata read hostname 'smithi016' 2024-08-31T06:14:15.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.284+0000 7f0e5a652740 1 --2- 172.21.15.16:0/3583576625 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x558c47fbd690 0x558c47fbfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:15.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.284+0000 7f0e5a652740 4 client.16905 mds_command: new command op to 24724 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:14:15.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.284+0000 7f0e5a652740 1 -- 172.21.15.16:0/3583576625 --> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558c47ea91a0 con 0x558c47fbd690 2024-08-31T06:14:15.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.285+0000 7f0e43e00640 1 --2- 172.21.15.16:0/3583576625 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x558c47fbd690 0x558c47fbfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:15.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.285+0000 7f0e43e00640 1 --2- 172.21.15.16:0/3583576625 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x558c47fbd690 0x558c47fbfab0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dc34e1d836de8473 server_cookie=3271d98d573d0272 in_seq=0 out_seq=0 2024-08-31T06:14:15.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.285+0000 7f0e3aa00640 10 client.16905 ms_handle_connect on v2:172.21.15.16:6838/219664460 2024-08-31T06:14:15.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.286+0000 7f0e3aa00640 1 -- 172.21.15.16:0/3583576625 <== mds.1 v2:172.21.15.16:6838/219664460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558c47ea91a0 con 0x558c47fbd690 2024-08-31T06:14:15.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.286+0000 7f0e3aa00640 10 client.16905 handle_command_reply: tid=0 2024-08-31T06:14:15.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.286+0000 7f0e5a652740 1 client.16905 shutdown 2024-08-31T06:14:15.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.286+0000 7f0e5a652740 1 -- 172.21.15.16:0/3583576625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c47b661b0 msgr2=0x558c47f44300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:15.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.286+0000 7f0e5a652740 1 --2- 172.21.15.16:0/3583576625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c47b661b0 0x558c47f44300 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c009a20 tx=0x7f0e3c005f00 comp rx=0 tx=0).stop 2024-08-31T06:14:15.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.286+0000 7f0e5a652740 1 -- 172.21.15.16:0/3583576625 shutdown_connections 2024-08-31T06:14:15.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.286+0000 7f0e5a652740 1 --2- 172.21.15.16:0/3583576625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c47b8f070 0x558c47f44840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:15.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.286+0000 7f0e5a652740 1 --2- 172.21.15.16:0/3583576625 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x558c47fbd690 0x558c47fbfab0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:15.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.286+0000 7f0e5a652740 1 --2- 172.21.15.16:0/3583576625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c47b22da0 0x558c47f44d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:15.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.286+0000 7f0e5a652740 1 --2- 172.21.15.16:0/3583576625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c47b661b0 0x558c47f44300 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:15.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.286+0000 7f0e3aa00640 0 client.16905 ms_handle_reset on v2:172.21.15.16:6838/219664460 2024-08-31T06:14:15.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.286+0000 7f0e5a652740 1 -- 172.21.15.16:0/3583576625 >> 172.21.15.16:0/3583576625 conn(0x558c47b83460 msgr2=0x558c47f36380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:15.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.286+0000 7f0e5a652740 1 -- 172.21.15.16:0/3583576625 shutdown_connections 2024-08-31T06:14:15.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.286+0000 7f0e5a652740 1 -- 172.21.15.16:0/3583576625 wait complete. 2024-08-31T06:14:15.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.286+0000 7f0e5a652740 20 client.16905 trim_cache size 0 max 16384 2024-08-31T06:14:15.292 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:14:15.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.287+0000 7f0e53a00640 1 -- 172.21.15.16:0/654934282 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f0e300b4d20 msgr2=0x7f0e300b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:15.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.287+0000 7f0e53a00640 1 --2- 172.21.15.16:0/654934282 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f0e300b4d20 0x7f0e300b7230 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0e34009fd0 tx=0x7f0e34005f90 comp rx=0 tx=0).stop 2024-08-31T06:14:15.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.287+0000 7f0e53a00640 1 -- 172.21.15.16:0/654934282 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e4c1141d0 msgr2=0x7f0e4c1a6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:15.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.287+0000 7f0e53a00640 1 --2- 172.21.15.16:0/654934282 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e4c1141d0 0x7f0e4c1a6040 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0e4400da10 tx=0x7f0e4400ded0 comp rx=0 tx=0).stop 2024-08-31T06:14:15.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.287+0000 7f0e53a00640 1 -- 172.21.15.16:0/654934282 shutdown_connections 2024-08-31T06:14:15.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.287+0000 7f0e53a00640 1 --2- 172.21.15.16:0/654934282 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f0e300b4d20 0x7f0e300b7230 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:15.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.287+0000 7f0e53a00640 1 --2- 172.21.15.16:0/654934282 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e4c1141d0 0x7f0e4c1a6040 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:15.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.287+0000 7f0e53a00640 1 --2- 172.21.15.16:0/654934282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e4c077120 0x7f0e4c1a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:15.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.287+0000 7f0e53a00640 1 --2- 172.21.15.16:0/654934282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e4c075b70 0x7f0e4c1a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:15.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.287+0000 7f0e53a00640 1 -- 172.21.15.16:0/654934282 >> 172.21.15.16:0/654934282 conn(0x7f0e4c07b5f0 msgr2=0x7f0e4c112e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:15.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.287+0000 7f0e53a00640 1 -- 172.21.15.16:0/654934282 shutdown_connections 2024-08-31T06:14:15.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:15.287+0000 7f0e53a00640 1 -- 172.21.15.16:0/654934282 wait complete. 2024-08-31T06:14:16.105 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T06:14:16.491 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:14:16.629 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 410.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-31T06:14:16.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:16 smithi016 ceph-mon[17194]: pgmap v578: 97 pgs: 97 active+clean; 3.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.7 MiB/s wr, 642 op/s 2024-08-31T06:14:16.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:16 smithi016 ceph-mon[17194]: osdmap e85: 12 total, 12 up, 12 in 2024-08-31T06:14:16.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:16 smithi184 ceph-mon[22331]: pgmap v578: 97 pgs: 97 active+clean; 3.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.7 MiB/s wr, 642 op/s 2024-08-31T06:14:16.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:16 smithi184 ceph-mon[22331]: osdmap e85: 12 total, 12 up, 12 in 2024-08-31T06:14:17.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:16 smithi155 ceph-mon[21742]: pgmap v578: 97 pgs: 97 active+clean; 3.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.7 MiB/s wr, 642 op/s 2024-08-31T06:14:17.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:16 smithi155 ceph-mon[21742]: osdmap e85: 12 total, 12 up, 12 in 2024-08-31T06:14:17.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.263+0000 7f043b400640 1 -- 172.21.15.16:0/339739880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0434103900 msgr2=0x7f0434103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:17.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.263+0000 7f043b400640 1 --2- 172.21.15.16:0/339739880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0434103900 0x7f0434103d10 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0424009900 tx=0x7f042402a410 comp rx=0 tx=0).stop 2024-08-31T06:14:17.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.263+0000 7f043b400640 1 -- 172.21.15.16:0/339739880 shutdown_connections 2024-08-31T06:14:17.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.263+0000 7f043b400640 1 --2- 172.21.15.16:0/339739880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0434109b90 0x7f04341103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:17.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.263+0000 7f043b400640 1 --2- 172.21.15.16:0/339739880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0434109240 0x7f0434109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:17.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.263+0000 7f043b400640 1 --2- 172.21.15.16:0/339739880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0434103900 0x7f0434103d10 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:17.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.263+0000 7f043b400640 1 -- 172.21.15.16:0/339739880 >> 172.21.15.16:0/339739880 conn(0x7f043407b5f0 msgr2=0x7f043407ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:17.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.263+0000 7f043b400640 1 -- 172.21.15.16:0/339739880 shutdown_connections 2024-08-31T06:14:17.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.263+0000 7f043b400640 1 -- 172.21.15.16:0/339739880 wait complete. 2024-08-31T06:14:17.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.264+0000 7f043b400640 1 Processor -- start 2024-08-31T06:14:17.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.264+0000 7f043b400640 1 -- start start 2024-08-31T06:14:17.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.265+0000 7f043b400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0434103900 0x7f043419c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:17.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.265+0000 7f043b400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0434109240 0x7f043419c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:17.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.265+0000 7f043b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0434109b90 0x7f04341a1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:17.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.265+0000 7f043b400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0434113850 con 0x7f0434109b90 2024-08-31T06:14:17.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.265+0000 7f043b400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f04341136d0 con 0x7f0434109240 2024-08-31T06:14:17.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.265+0000 7f043b400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f04341139d0 con 0x7f0434103900 2024-08-31T06:14:17.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.265+0000 7f0438a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0434103900 0x7f043419c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:17.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.265+0000 7f0438a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0434103900 0x7f043419c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:44620/0 (socket says 172.21.15.16:44620) 2024-08-31T06:14:17.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.265+0000 7f0439400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0434109b90 0x7f04341a1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:17.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.265+0000 7f0438a00640 1 -- 172.21.15.16:0/4249207176 learned_addr learned my addr 172.21.15.16:0/4249207176 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:17.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.265+0000 7f042be00640 1 --2- 172.21.15.16:0/4249207176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0434109240 0x7f043419c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:17.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.266+0000 7f0438a00640 1 -- 172.21.15.16:0/4249207176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0434109240 msgr2=0x7f043419c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:17.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.266+0000 7f0438a00640 1 --2- 172.21.15.16:0/4249207176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0434109240 0x7f043419c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:17.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.266+0000 7f0438a00640 1 -- 172.21.15.16:0/4249207176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0434109b90 msgr2=0x7f04341a1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:17.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.266+0000 7f0438a00640 1 --2- 172.21.15.16:0/4249207176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0434109b90 0x7f04341a1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:17.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.266+0000 7f0438a00640 1 -- 172.21.15.16:0/4249207176 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04341a2390 con 0x7f0434103900 2024-08-31T06:14:17.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.266+0000 7f0439400640 1 --2- 172.21.15.16:0/4249207176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0434109b90 0x7f04341a1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:14:17.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.266+0000 7f0438a00640 1 --2- 172.21.15.16:0/4249207176 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0434103900 0x7f043419c420 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0424005940 tx=0x7f04240040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:17.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.266+0000 7f0432000640 1 -- 172.21.15.16:0/4249207176 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f042403f030 con 0x7f0434103900 2024-08-31T06:14:17.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.267+0000 7f043b400640 1 -- 172.21.15.16:0/4249207176 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04341a2620 con 0x7f0434103900 2024-08-31T06:14:17.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.267+0000 7f0432000640 1 -- 172.21.15.16:0/4249207176 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04240044c0 con 0x7f0434103900 2024-08-31T06:14:17.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.267+0000 7f0432000640 1 -- 172.21.15.16:0/4249207176 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0424035070 con 0x7f0434103900 2024-08-31T06:14:17.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.267+0000 7f043b400640 1 -- 172.21.15.16:0/4249207176 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04341a8de0 con 0x7f0434103900 2024-08-31T06:14:17.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.270+0000 7f044208f740 1 Processor -- start 2024-08-31T06:14:17.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.270+0000 7f044208f740 1 -- start start 2024-08-31T06:14:17.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.270+0000 7f044208f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562348c78070 0x56234901d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:17.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.270+0000 7f044208f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562348c4f1b0 0x56234901ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:17.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.270+0000 7f044208f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562348c57390 0x56234901e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:17.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.270+0000 7f044208f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562348f947b0 con 0x562348c78070 2024-08-31T06:14:17.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.270+0000 7f044208f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562348f83d40 con 0x562348c57390 2024-08-31T06:14:17.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.270+0000 7f044208f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562348f87cf0 con 0x562348c4f1b0 2024-08-31T06:14:17.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.270+0000 7f0439400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562348c78070 0x56234901d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:17.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.270+0000 7f0439400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562348c78070 0x56234901d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56334/0 (socket says 172.21.15.16:56334) 2024-08-31T06:14:17.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.270+0000 7f0439400640 1 -- 172.21.15.16:0/2131839138 learned_addr learned my addr 172.21.15.16:0/2131839138 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:17.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.270+0000 7f042be00640 1 --2- 172.21.15.16:0/2131839138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562348c57390 0x56234901e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:17.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.271+0000 7f0432000640 1 -- 172.21.15.16:0/4249207176 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f0424042030 con 0x7f0434103900 2024-08-31T06:14:17.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.271+0000 7f0439400640 1 -- 172.21.15.16:0/2131839138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562348c4f1b0 msgr2=0x56234901ddf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T06:14:17.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.271+0000 7f0439400640 1 --2- 172.21.15.16:0/2131839138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562348c4f1b0 0x56234901ddf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:17.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.271+0000 7f0439400640 1 -- 172.21.15.16:0/2131839138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562348c57390 msgr2=0x56234901e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:17.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.271+0000 7f0439400640 1 --2- 172.21.15.16:0/2131839138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562348c57390 0x56234901e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:17.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.271+0000 7f0439400640 1 -- 172.21.15.16:0/2131839138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562348ce3650 con 0x562348c78070 2024-08-31T06:14:17.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.271+0000 7f042be00640 1 --2- 172.21.15.16:0/2131839138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562348c57390 0x56234901e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:14:17.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.271+0000 7f0439400640 1 --2- 172.21.15.16:0/2131839138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562348c78070 0x56234901d8b0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f042c0117e0 tx=0x7f042c011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf275ff3677e2827 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:17.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.272+0000 7f0429c00640 1 -- 172.21.15.16:0/2131839138 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f042c011310 con 0x562348c78070 2024-08-31T06:14:17.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.272+0000 7f0429c00640 1 -- 172.21.15.16:0/2131839138 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f042c003e20 con 0x562348c78070 2024-08-31T06:14:17.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.272+0000 7f0429c00640 1 -- 172.21.15.16:0/2131839138 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f042c0044f0 con 0x562348c78070 2024-08-31T06:14:17.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.272+0000 7f0432000640 1 --2- 172.21.15.16:0/4249207176 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f04180b4ca0 0x7f04180b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:17.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.272+0000 7f044208f740 1 -- 172.21.15.16:0/2131839138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562348c78070 msgr2=0x56234901d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:17.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.272+0000 7f044208f740 1 --2- 172.21.15.16:0/2131839138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562348c78070 0x56234901d8b0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f042c0117e0 tx=0x7f042c011ca0 comp rx=0 tx=0).stop 2024-08-31T06:14:17.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.273+0000 7f044208f740 1 -- 172.21.15.16:0/2131839138 shutdown_connections 2024-08-31T06:14:17.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.273+0000 7f044208f740 1 --2- 172.21.15.16:0/2131839138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562348c4f1b0 0x56234901ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:17.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.273+0000 7f044208f740 1 --2- 172.21.15.16:0/2131839138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562348c57390 0x56234901e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:17.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.273+0000 7f044208f740 1 --2- 172.21.15.16:0/2131839138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562348c78070 0x56234901d8b0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:17.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.273+0000 7f044208f740 1 -- 172.21.15.16:0/2131839138 >> 172.21.15.16:0/2131839138 conn(0x562348c6c460 msgr2=0x56234901b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:17.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.273+0000 7f0438a00640 1 --2- 172.21.15.16:0/4249207176 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f04180b4ca0 0x7f04180b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:17.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.273+0000 7f044208f740 1 -- 172.21.15.16:0/2131839138 shutdown_connections 2024-08-31T06:14:17.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.273+0000 7f0432000640 1 -- 172.21.15.16:0/4249207176 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7f0424105880 con 0x7f0434103900 2024-08-31T06:14:17.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.273+0000 7f044208f740 1 -- 172.21.15.16:0/2131839138 wait complete. 2024-08-31T06:14:17.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.273+0000 7f0438a00640 1 --2- 172.21.15.16:0/4249207176 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f04180b4ca0 0x7f04180b71b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x56234901c530 tx=0x7f042403b460 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:17.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.274+0000 7f044208f740 1 -- start start 2024-08-31T06:14:17.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.274+0000 7f044208f740 1 Processor -- start 2024-08-31T06:14:17.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.274+0000 7f044208f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562348c4f1b0 0x562349032430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:17.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.274+0000 7f044208f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562348c57390 0x562349032a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:17.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.274+0000 7f044208f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562348c78070 0x562349037a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:17.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.274+0000 7f044208f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562348de6700 con 0x562348c4f1b0 2024-08-31T06:14:17.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.274+0000 7f044208f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562348f0acf0 con 0x562348c57390 2024-08-31T06:14:17.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.274+0000 7f044208f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56234902c090 con 0x562348c78070 2024-08-31T06:14:17.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.275+0000 7f044208f740 10 client.0 _ll_register_callbacks cb 0x562348f30470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:14:17.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.275+0000 7f042be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562348c57390 0x562349032a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:17.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.275+0000 7f0439400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562348c4f1b0 0x562349032430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:17.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.275+0000 7f042be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562348c57390 0x562349032a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:33620/0 (socket says 172.21.15.16:33620) 2024-08-31T06:14:17.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.275+0000 7f042be00640 1 -- 172.21.15.16:0/3107912749 learned_addr learned my addr 172.21.15.16:0/3107912749 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:17.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.275+0000 7f042be00640 1 -- 172.21.15.16:0/3107912749 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562348c78070 msgr2=0x562349037a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:17.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.275+0000 7f042be00640 1 --2- 172.21.15.16:0/3107912749 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562348c78070 0x562349037a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:17.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.275+0000 7f042be00640 1 -- 172.21.15.16:0/3107912749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562348c4f1b0 msgr2=0x562349032430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:17.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.275+0000 7f042be00640 1 --2- 172.21.15.16:0/3107912749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562348c4f1b0 0x562349032430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:17.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.275+0000 7f042be00640 1 -- 172.21.15.16:0/3107912749 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562348f4b8b0 con 0x562348c57390 2024-08-31T06:14:17.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.275+0000 7f0439400640 1 --2- 172.21.15.16:0/3107912749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562348c4f1b0 0x562349032430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:14:17.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.276+0000 7f042be00640 1 --2- 172.21.15.16:0/3107912749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562348c57390 0x562349032a10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f041c012700 tx=0x7f041c012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf9789f68d6a6a2e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:17.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.276+0000 7f0422000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:14:17.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.276+0000 7f0422000640 1 -- 172.21.15.16:0/3107912749 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c00f920 con 0x562348c57390 2024-08-31T06:14:17.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.276+0000 7f044208f740 1 -- 172.21.15.16:0/3107912749 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f042c003e20 con 0x562348c57390 2024-08-31T06:14:17.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.276+0000 7f0422000640 1 -- 172.21.15.16:0/3107912749 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7f041c00fac0 con 0x562348c57390 2024-08-31T06:14:17.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.276+0000 7f0422000640 1 -- 172.21.15.16:0/3107912749 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f041c00fcc0 con 0x562348c57390 2024-08-31T06:14:17.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.277+0000 7f044208f740 10 client.27149 fetch_fsmap learned FSMap version 39 2024-08-31T06:14:17.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.277+0000 7f044208f740 1 -- 172.21.15.16:0/3107912749 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562349040a30 con 0x562348c57390 2024-08-31T06:14:17.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.277+0000 7f0422000640 1 -- 172.21.15.16:0/3107912749 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f042c003e20 con 0x562348c57390 2024-08-31T06:14:17.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.277+0000 7f0422000640 1 -- 172.21.15.16:0/3107912749 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f041c01c420 con 0x562348c57390 2024-08-31T06:14:17.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.277+0000 7f044208f740 10 client.27149 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T06:14:17.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.277+0000 7f044208f740 10 client.27149 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:14:17.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.277+0000 7f044208f740 20 client.27149 populate_metadata read hostname 'smithi016' 2024-08-31T06:14:17.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.277+0000 7f044208f740 1 --2- 172.21.15.16:0/3107912749 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x562348c369f0 0x562349043ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:17.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.277+0000 7f044208f740 4 client.27149 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:14:17.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.277+0000 7f044208f740 1 -- 172.21.15.16:0/3107912749 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562348f96700 con 0x562348c369f0 2024-08-31T06:14:17.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.278+0000 7f0439400640 1 --2- 172.21.15.16:0/3107912749 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x562348c369f0 0x562349043ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:17.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.278+0000 7f0439400640 1 --2- 172.21.15.16:0/3107912749 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x562348c369f0 0x562349043ff0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a9aeae756fdb08b server_cookie=6fcbb7778fc6e9f6 in_seq=0 out_seq=0 2024-08-31T06:14:17.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.278+0000 7f0422000640 10 client.27149 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:17.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.279+0000 7f0422000640 1 -- 172.21.15.16:0/3107912749 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562348f96700 con 0x562348c369f0 2024-08-31T06:14:17.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.279+0000 7f0422000640 10 client.27149 handle_command_reply: tid=0 2024-08-31T06:14:17.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.279+0000 7f044208f740 1 client.27149 shutdown 2024-08-31T06:14:17.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.279+0000 7f044208f740 1 -- 172.21.15.16:0/3107912749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562348c57390 msgr2=0x562349032a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:17.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.279+0000 7f044208f740 1 --2- 172.21.15.16:0/3107912749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562348c57390 0x562349032a10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f041c012700 tx=0x7f041c012bc0 comp rx=0 tx=0).stop 2024-08-31T06:14:17.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.279+0000 7f044208f740 1 -- 172.21.15.16:0/3107912749 shutdown_connections 2024-08-31T06:14:17.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.279+0000 7f044208f740 1 --2- 172.21.15.16:0/3107912749 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x562348c369f0 0x562349043ff0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:17.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.279+0000 7f044208f740 1 --2- 172.21.15.16:0/3107912749 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562348c78070 0x562349037a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:17.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.279+0000 7f044208f740 1 --2- 172.21.15.16:0/3107912749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562348c57390 0x562349032a10 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:17.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.279+0000 7f044208f740 1 --2- 172.21.15.16:0/3107912749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562348c4f1b0 0x562349032430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:17.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.279+0000 7f044208f740 1 -- 172.21.15.16:0/3107912749 >> 172.21.15.16:0/3107912749 conn(0x562348c6c460 msgr2=0x562349020a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:17.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.279+0000 7f0422000640 0 client.27149 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:17.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.279+0000 7f044208f740 1 -- 172.21.15.16:0/3107912749 shutdown_connections 2024-08-31T06:14:17.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.280+0000 7f044208f740 1 -- 172.21.15.16:0/3107912749 wait complete. 2024-08-31T06:14:17.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.280+0000 7f044208f740 20 client.27149 trim_cache size 0 max 16384 2024-08-31T06:14:17.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.302+0000 7f044208f740 1 Processor -- start 2024-08-31T06:14:17.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.302+0000 7f044208f740 1 -- start start 2024-08-31T06:14:17.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.302+0000 7f044208f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562348c4f1b0 0x5623490378a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:17.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.302+0000 7f044208f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562348c57390 0x56234901b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:17.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.302+0000 7f044208f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562348c78070 0x5623490347f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:17.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.302+0000 7f044208f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f04341afb60 con 0x562348c57390 2024-08-31T06:14:17.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.302+0000 7f044208f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f04341ad3d0 con 0x562348c78070 2024-08-31T06:14:17.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.302+0000 7f044208f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f04341ab280 con 0x562348c4f1b0 2024-08-31T06:14:17.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.302+0000 7f042be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562348c57390 0x56234901b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:17.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.303+0000 7f042be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562348c57390 0x56234901b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56354/0 (socket says 172.21.15.16:56354) 2024-08-31T06:14:17.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.303+0000 7f042be00640 1 -- 172.21.15.16:0/3234417927 learned_addr learned my addr 172.21.15.16:0/3234417927 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:17.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.303+0000 7f0439400640 1 --2- 172.21.15.16:0/3234417927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562348c78070 0x5623490347f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:17.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.303+0000 7f0439400640 1 --2- 172.21.15.16:0/3234417927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562348c4f1b0 0x5623490378a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:17.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.303+0000 7f042be00640 1 -- 172.21.15.16:0/3234417927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562348c4f1b0 msgr2=0x5623490378a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:17.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.303+0000 7f042be00640 1 --2- 172.21.15.16:0/3234417927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562348c4f1b0 0x5623490378a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:17.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.303+0000 7f042be00640 1 -- 172.21.15.16:0/3234417927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562348c78070 msgr2=0x5623490347f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:17.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.303+0000 7f042be00640 1 --2- 172.21.15.16:0/3234417927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562348c78070 0x5623490347f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:17.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.303+0000 7f042be00640 1 -- 172.21.15.16:0/3234417927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562349040840 con 0x562348c57390 2024-08-31T06:14:17.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.303+0000 7f0439400640 1 --2- 172.21.15.16:0/3234417927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562348c4f1b0 0x5623490378a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:14:17.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.303+0000 7f042be00640 1 --2- 172.21.15.16:0/3234417927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562348c57390 0x56234901b740 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f041c0059f0 tx=0x7f041c00f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53d3b931932282d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:17.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.304+0000 7f0429200640 1 -- 172.21.15.16:0/3234417927 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c029030 con 0x562348c57390 2024-08-31T06:14:17.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.304+0000 7f0429200640 1 -- 172.21.15.16:0/3234417927 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f041c0055a0 con 0x562348c57390 2024-08-31T06:14:17.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.304+0000 7f0429200640 1 -- 172.21.15.16:0/3234417927 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c01c640 con 0x562348c57390 2024-08-31T06:14:17.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.304+0000 7f044208f740 1 -- 172.21.15.16:0/3234417927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562348c57390 msgr2=0x56234901b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:17.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.304+0000 7f044208f740 1 --2- 172.21.15.16:0/3234417927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562348c57390 0x56234901b740 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f041c0059f0 tx=0x7f041c00f810 comp rx=0 tx=0).stop 2024-08-31T06:14:17.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.305+0000 7f044208f740 1 -- 172.21.15.16:0/3234417927 shutdown_connections 2024-08-31T06:14:17.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.305+0000 7f044208f740 1 --2- 172.21.15.16:0/3234417927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562348c78070 0x5623490347f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:17.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.305+0000 7f044208f740 1 --2- 172.21.15.16:0/3234417927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562348c57390 0x56234901b740 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:17.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.305+0000 7f044208f740 1 --2- 172.21.15.16:0/3234417927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562348c4f1b0 0x5623490378a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:17.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.305+0000 7f044208f740 1 -- 172.21.15.16:0/3234417927 >> 172.21.15.16:0/3234417927 conn(0x562348c6c460 msgr2=0x56234901b030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:17.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.305+0000 7f044208f740 1 -- 172.21.15.16:0/3234417927 shutdown_connections 2024-08-31T06:14:17.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.305+0000 7f044208f740 1 -- 172.21.15.16:0/3234417927 wait complete. 2024-08-31T06:14:17.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.305+0000 7f044208f740 1 -- start start 2024-08-31T06:14:17.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.305+0000 7f044208f740 1 Processor -- start 2024-08-31T06:14:17.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.306+0000 7f044208f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562348c4f1b0 0x562349049bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:17.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.306+0000 7f044208f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562348c57390 0x56234904a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:17.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.306+0000 7f044208f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562348c78070 0x5623490487a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:17.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.306+0000 7f044208f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f04341ab8d0 con 0x562348c57390 2024-08-31T06:14:17.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.306+0000 7f044208f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f04341acd00 con 0x562348c4f1b0 2024-08-31T06:14:17.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.306+0000 7f044208f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f04341abf60 con 0x562348c78070 2024-08-31T06:14:17.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.306+0000 7f044208f740 10 client.0 _ll_register_callbacks cb 0x7f041c009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:14:17.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.306+0000 7f042be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562348c57390 0x56234904a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:17.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.306+0000 7f042be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562348c57390 0x56234904a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56370/0 (socket says 172.21.15.16:56370) 2024-08-31T06:14:17.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.306+0000 7f042be00640 1 -- 172.21.15.16:0/3243015202 learned_addr learned my addr 172.21.15.16:0/3243015202 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:17.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.306+0000 7f0439400640 1 --2- 172.21.15.16:0/3243015202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562348c4f1b0 0x562349049bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:17.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.307+0000 7f0439400640 1 --2- 172.21.15.16:0/3243015202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562348c78070 0x5623490487a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:17.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.307+0000 7f042be00640 1 -- 172.21.15.16:0/3243015202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562348c78070 msgr2=0x5623490487a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:17.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.307+0000 7f042be00640 1 --2- 172.21.15.16:0/3243015202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562348c78070 0x5623490487a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:17.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.307+0000 7f042be00640 1 -- 172.21.15.16:0/3243015202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562348c4f1b0 msgr2=0x562349049bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:17.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.307+0000 7f042be00640 1 --2- 172.21.15.16:0/3243015202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562348c4f1b0 0x562349049bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:17.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.307+0000 7f042be00640 1 -- 172.21.15.16:0/3243015202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562349048d60 con 0x562348c57390 2024-08-31T06:14:17.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.307+0000 7f0439400640 1 --2- 172.21.15.16:0/3243015202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562348c78070 0x5623490487a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:14:17.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.307+0000 7f042be00640 1 --2- 172.21.15.16:0/3243015202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562348c57390 0x56234904a100 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f041c009730 tx=0x7f041c005f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d624e13aca0c4 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:17.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.307+0000 7f0422000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:14:17.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.307+0000 7f0422000640 1 -- 172.21.15.16:0/3243015202 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c029070 con 0x562348c57390 2024-08-31T06:14:17.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.307+0000 7f0422000640 1 -- 172.21.15.16:0/3243015202 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7f041c0050f0 con 0x562348c57390 2024-08-31T06:14:17.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.307+0000 7f044208f740 1 -- 172.21.15.16:0/3243015202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f041c0055a0 con 0x562348c57390 2024-08-31T06:14:17.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.308+0000 7f0422000640 1 -- 172.21.15.16:0/3243015202 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f041c01cc90 con 0x562348c57390 2024-08-31T06:14:17.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.308+0000 7f044208f740 10 client.16923 fetch_fsmap learned FSMap version 39 2024-08-31T06:14:17.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.308+0000 7f044208f740 1 -- 172.21.15.16:0/3243015202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5623490af1b0 con 0x562348c57390 2024-08-31T06:14:17.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.308+0000 7f0422000640 1 -- 172.21.15.16:0/3243015202 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f041c0055a0 con 0x562348c57390 2024-08-31T06:14:17.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.308+0000 7f0422000640 1 -- 172.21.15.16:0/3243015202 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f041c025450 con 0x562348c57390 2024-08-31T06:14:17.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.308+0000 7f044208f740 10 client.16923 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T06:14:17.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.308+0000 7f044208f740 10 client.16923 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:14:17.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.308+0000 7f044208f740 20 client.16923 populate_metadata read hostname 'smithi016' 2024-08-31T06:14:17.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.308+0000 7f044208f740 1 --2- 172.21.15.16:0/3243015202 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x562348c0bda0 0x5623490b1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:17.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.308+0000 7f044208f740 4 client.16923 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:14:17.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.308+0000 7f044208f740 1 -- 172.21.15.16:0/3243015202 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562348f921a0 con 0x562348c0bda0 2024-08-31T06:14:17.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.309+0000 7f0439400640 1 --2- 172.21.15.16:0/3243015202 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x562348c0bda0 0x5623490b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:17.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.309+0000 7f0439400640 1 --2- 172.21.15.16:0/3243015202 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x562348c0bda0 0x5623490b1920 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10e13f9624a67653 server_cookie=4a4ce23f5200f605 in_seq=0 out_seq=0 2024-08-31T06:14:17.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.309+0000 7f0422000640 10 client.16923 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:17.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.310+0000 7f0422000640 1 -- 172.21.15.16:0/3243015202 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562348f921a0 con 0x562348c0bda0 2024-08-31T06:14:17.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.310+0000 7f0422000640 10 client.16923 handle_command_reply: tid=0 2024-08-31T06:14:17.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.310+0000 7f044208f740 1 client.16923 shutdown 2024-08-31T06:14:17.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.310+0000 7f044208f740 1 -- 172.21.15.16:0/3243015202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562348c57390 msgr2=0x56234904a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:17.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.310+0000 7f044208f740 1 --2- 172.21.15.16:0/3243015202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562348c57390 0x56234904a100 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f041c009730 tx=0x7f041c005f50 comp rx=0 tx=0).stop 2024-08-31T06:14:17.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.310+0000 7f044208f740 1 -- 172.21.15.16:0/3243015202 shutdown_connections 2024-08-31T06:14:17.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.310+0000 7f044208f740 1 --2- 172.21.15.16:0/3243015202 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x562348c0bda0 0x5623490b1920 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:17.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.310+0000 7f044208f740 1 --2- 172.21.15.16:0/3243015202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562348c78070 0x5623490487a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:17.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.310+0000 7f044208f740 1 --2- 172.21.15.16:0/3243015202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562348c57390 0x56234904a100 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:17.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.310+0000 7f044208f740 1 --2- 172.21.15.16:0/3243015202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562348c4f1b0 0x562349049bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:17.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.310+0000 7f044208f740 1 -- 172.21.15.16:0/3243015202 >> 172.21.15.16:0/3243015202 conn(0x562348c6c460 msgr2=0x56234901e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:17.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.310+0000 7f0422000640 0 client.16923 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:17.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.310+0000 7f044208f740 1 -- 172.21.15.16:0/3243015202 shutdown_connections 2024-08-31T06:14:17.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.310+0000 7f044208f740 1 -- 172.21.15.16:0/3243015202 wait complete. 2024-08-31T06:14:17.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.310+0000 7f044208f740 20 client.16923 trim_cache size 0 max 16384 2024-08-31T06:14:17.316 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:14:17.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.311+0000 7f043b400640 1 -- 172.21.15.16:0/4249207176 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f04180b4ca0 msgr2=0x7f04180b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:17.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.311+0000 7f043b400640 1 --2- 172.21.15.16:0/4249207176 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f04180b4ca0 0x7f04180b71b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x56234901c530 tx=0x7f042403b460 comp rx=0 tx=0).stop 2024-08-31T06:14:17.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.311+0000 7f043b400640 1 -- 172.21.15.16:0/4249207176 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0434103900 msgr2=0x7f043419c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:17.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.311+0000 7f043b400640 1 --2- 172.21.15.16:0/4249207176 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0434103900 0x7f043419c420 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0424005940 tx=0x7f04240040c0 comp rx=0 tx=0).stop 2024-08-31T06:14:17.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.311+0000 7f043b400640 1 -- 172.21.15.16:0/4249207176 shutdown_connections 2024-08-31T06:14:17.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.311+0000 7f043b400640 1 --2- 172.21.15.16:0/4249207176 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f04180b4ca0 0x7f04180b71b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:17.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.311+0000 7f043b400640 1 --2- 172.21.15.16:0/4249207176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0434109b90 0x7f04341a1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:17.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.311+0000 7f043b400640 1 --2- 172.21.15.16:0/4249207176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0434109240 0x7f043419c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:17.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.311+0000 7f043b400640 1 --2- 172.21.15.16:0/4249207176 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0434103900 0x7f043419c420 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:17.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.311+0000 7f043b400640 1 -- 172.21.15.16:0/4249207176 >> 172.21.15.16:0/4249207176 conn(0x7f043407b5f0 msgr2=0x7f0434107050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:17.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.311+0000 7f043b400640 1 -- 172.21.15.16:0/4249207176 shutdown_connections 2024-08-31T06:14:17.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:17.311+0000 7f043b400640 1 -- 172.21.15.16:0/4249207176 wait complete. 2024-08-31T06:14:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.422+0000 7f4a0e200640 1 -- 172.21.15.16:0/288709825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a0810b1f0 msgr2=0x7f4a0810b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.422+0000 7f4a0e200640 1 --2- 172.21.15.16:0/288709825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a0810b1f0 0x7f4a0810b600 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f49f4009900 tx=0x7f49f402a410 comp rx=0 tx=0).stop 2024-08-31T06:14:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.422+0000 7f4a0e200640 1 -- 172.21.15.16:0/288709825 shutdown_connections 2024-08-31T06:14:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.422+0000 7f4a0e200640 1 --2- 172.21.15.16:0/288709825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a0810bb40 0x7f4a08112370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.422+0000 7f4a0e200640 1 --2- 172.21.15.16:0/288709825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a0810b1f0 0x7f4a0810b600 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.422+0000 7f4a0e200640 1 --2- 172.21.15.16:0/288709825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a081058c0 0x7f4a08105cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.422+0000 7f4a0e200640 1 -- 172.21.15.16:0/288709825 >> 172.21.15.16:0/288709825 conn(0x7f4a08101300 msgr2=0x7f4a08103750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.422+0000 7f4a0e200640 1 -- 172.21.15.16:0/288709825 shutdown_connections 2024-08-31T06:14:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.422+0000 7f4a0e200640 1 -- 172.21.15.16:0/288709825 wait complete. 2024-08-31T06:14:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.423+0000 7f4a0e200640 1 Processor -- start 2024-08-31T06:14:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.423+0000 7f4a0e200640 1 -- start start 2024-08-31T06:14:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.424+0000 7f4a0e200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a081058c0 0x7f4a0819c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.424+0000 7f4a0e200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a0810b1f0 0x7f4a0819c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.424+0000 7f4a0e200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a0810bb40 0x7f4a081a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.424+0000 7f4a0e200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4a081157c0 con 0x7f4a0810bb40 2024-08-31T06:14:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.424+0000 7f4a0e200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4a08115640 con 0x7f4a0810b1f0 2024-08-31T06:14:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.424+0000 7f4a0e200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4a08115940 con 0x7f4a081058c0 2024-08-31T06:14:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.424+0000 7f4a0d800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a0810bb40 0x7f4a081a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.424+0000 7f4a0ce00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a081058c0 0x7f4a0819c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.424+0000 7f4a07e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a0810b1f0 0x7f4a0819c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.424+0000 7f4a0ce00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a081058c0 0x7f4a0819c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:44680/0 (socket says 172.21.15.16:44680) 2024-08-31T06:14:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.424+0000 7f4a0d800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a0810bb40 0x7f4a081a1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56392/0 (socket says 172.21.15.16:56392) 2024-08-31T06:14:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.424+0000 7f4a0ce00640 1 -- 172.21.15.16:0/1206821495 learned_addr learned my addr 172.21.15.16:0/1206821495 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.425+0000 7f4a0ce00640 1 -- 172.21.15.16:0/1206821495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a0810b1f0 msgr2=0x7f4a0819c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.425+0000 7f4a0ce00640 1 --2- 172.21.15.16:0/1206821495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a0810b1f0 0x7f4a0819c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.425+0000 7f4a0ce00640 1 -- 172.21.15.16:0/1206821495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a0810bb40 msgr2=0x7f4a081a1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.425+0000 7f4a0ce00640 1 --2- 172.21.15.16:0/1206821495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a0810bb40 0x7f4a081a1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.425+0000 7f4a0ce00640 1 -- 172.21.15.16:0/1206821495 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a081a2170 con 0x7f4a081058c0 2024-08-31T06:14:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.425+0000 7f4a0d800640 1 --2- 172.21.15.16:0/1206821495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a0810bb40 0x7f4a081a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:14:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.425+0000 7f4a0ce00640 1 --2- 172.21.15.16:0/1206821495 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a081058c0 0x7f4a0819c240 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f49fc00e9f0 tx=0x7f49fc00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.425+0000 7f4a05600640 1 -- 172.21.15.16:0/1206821495 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc00cd80 con 0x7f4a081058c0 2024-08-31T06:14:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.425+0000 7f4a0e200640 1 -- 172.21.15.16:0/1206821495 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a081a2460 con 0x7f4a081058c0 2024-08-31T06:14:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.425+0000 7f4a05600640 1 -- 172.21.15.16:0/1206821495 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49fc0054d0 con 0x7f4a081058c0 2024-08-31T06:14:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.425+0000 7f4a05600640 1 -- 172.21.15.16:0/1206821495 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc010690 con 0x7f4a081058c0 2024-08-31T06:14:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.425+0000 7f4a0e200640 1 -- 172.21.15.16:0/1206821495 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a081a8bf0 con 0x7f4a081058c0 2024-08-31T06:14:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.428+0000 7f4a16a48740 1 Processor -- start 2024-08-31T06:14:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.429+0000 7f4a16a48740 1 -- start start 2024-08-31T06:14:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.429+0000 7f4a16a48740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56555feed1b0 0x5655602908a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.429+0000 7f4a16a48740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56555fec42f0 0x565560290de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:18.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.429+0000 7f4a16a48740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56555fecc4d0 0x565560291320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:18.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.429+0000 7f4a16a48740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x565560205f70 con 0x56555fec42f0 2024-08-31T06:14:18.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.429+0000 7f4a16a48740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5655601f5500 con 0x56555feed1b0 2024-08-31T06:14:18.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.429+0000 7f4a16a48740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5655601f94b0 con 0x56555fecc4d0 2024-08-31T06:14:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.429+0000 7f4a0d800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56555feed1b0 0x5655602908a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.429+0000 7f4a07e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56555fecc4d0 0x565560291320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.429+0000 7f4a0d800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56555feed1b0 0x5655602908a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:33670/0 (socket says 172.21.15.16:33670) 2024-08-31T06:14:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.429+0000 7f4a05600640 1 -- 172.21.15.16:0/1206821495 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f49fc004e90 con 0x7f4a081058c0 2024-08-31T06:14:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.429+0000 7f4a0d800640 1 -- 172.21.15.16:0/60851698 learned_addr learned my addr 172.21.15.16:0/60851698 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.430+0000 7f4a0ce00640 1 --2- 172.21.15.16:0/60851698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56555fec42f0 0x565560290de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.430+0000 7f4a0d800640 1 -- 172.21.15.16:0/60851698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56555fecc4d0 msgr2=0x565560291320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.430+0000 7f4a0d800640 1 --2- 172.21.15.16:0/60851698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56555fecc4d0 0x565560291320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.430+0000 7f4a0d800640 1 -- 172.21.15.16:0/60851698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56555fec42f0 msgr2=0x565560290de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.430+0000 7f4a0d800640 1 --2- 172.21.15.16:0/60851698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56555fec42f0 0x565560290de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.430+0000 7f4a0d800640 1 -- 172.21.15.16:0/60851698 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5655601afc10 con 0x56555feed1b0 2024-08-31T06:14:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.430+0000 7f4a0ce00640 1 --2- 172.21.15.16:0/60851698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56555fec42f0 0x565560290de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:14:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.430+0000 7f4a07e00640 1 --2- 172.21.15.16:0/60851698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56555fecc4d0 0x565560291320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:14:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.430+0000 7f4a0d800640 1 --2- 172.21.15.16:0/60851698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56555feed1b0 0x5655602908a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4a00011810 tx=0x7f4a00011cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a1492d7711911f9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.430+0000 7f49fa000640 1 -- 172.21.15.16:0/60851698 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a00011310 con 0x56555feed1b0 2024-08-31T06:14:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.431+0000 7f49fa000640 1 -- 172.21.15.16:0/60851698 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a00003e20 con 0x56555feed1b0 2024-08-31T06:14:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.431+0000 7f49fa000640 1 -- 172.21.15.16:0/60851698 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a00004490 con 0x56555feed1b0 2024-08-31T06:14:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.431+0000 7f4a05600640 1 --2- 172.21.15.16:0/1206821495 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f49f00b4d20 0x7f49f00b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.431+0000 7f4a16a48740 1 -- 172.21.15.16:0/60851698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56555feed1b0 msgr2=0x5655602908a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.431+0000 7f4a16a48740 1 --2- 172.21.15.16:0/60851698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56555feed1b0 0x5655602908a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4a00011810 tx=0x7f4a00011cd0 comp rx=0 tx=0).stop 2024-08-31T06:14:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.431+0000 7f4a05600640 1 -- 172.21.15.16:0/1206821495 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7f49fc0e5930 con 0x7f4a081058c0 2024-08-31T06:14:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.431+0000 7f4a16a48740 1 -- 172.21.15.16:0/60851698 shutdown_connections 2024-08-31T06:14:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.431+0000 7f4a16a48740 1 --2- 172.21.15.16:0/60851698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56555fecc4d0 0x565560291320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.431+0000 7f4a16a48740 1 --2- 172.21.15.16:0/60851698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56555fec42f0 0x565560290de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.431+0000 7f4a16a48740 1 --2- 172.21.15.16:0/60851698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56555feed1b0 0x5655602908a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.431+0000 7f4a16a48740 1 -- 172.21.15.16:0/60851698 >> 172.21.15.16:0/60851698 conn(0x56555fee1300 msgr2=0x56556028e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.431+0000 7f4a0ce00640 1 --2- 172.21.15.16:0/1206821495 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f49f00b4d20 0x7f49f00b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.432+0000 7f4a16a48740 1 -- 172.21.15.16:0/60851698 shutdown_connections 2024-08-31T06:14:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.432+0000 7f4a16a48740 1 -- 172.21.15.16:0/60851698 wait complete. 2024-08-31T06:14:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.432+0000 7f4a0ce00640 1 --2- 172.21.15.16:0/1206821495 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f49f00b4d20 0x7f49f00b7230 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x56556028f520 tx=0x7f49fc004a90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.432+0000 7f4a16a48740 1 -- start start 2024-08-31T06:14:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.432+0000 7f4a16a48740 1 Processor -- start 2024-08-31T06:14:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.433+0000 7f4a16a48740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56555fec42f0 0x56556029bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.433+0000 7f4a16a48740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56555fecc4d0 0x56556029c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.433+0000 7f4a16a48740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56555feed1b0 0x5655602a8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.433+0000 7f4a16a48740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56556012eab0 con 0x56555fec42f0 2024-08-31T06:14:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.433+0000 7f4a16a48740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56556019c320 con 0x56555fecc4d0 2024-08-31T06:14:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.433+0000 7f4a16a48740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56556029f020 con 0x56555feed1b0 2024-08-31T06:14:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.433+0000 7f4a16a48740 10 client.0 _ll_register_callbacks cb 0x5655601d5e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:14:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.433+0000 7f4a0d800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56555fec42f0 0x56556029bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.433+0000 7f4a0d800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56555fec42f0 0x56556029bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56404/0 (socket says 172.21.15.16:56404) 2024-08-31T06:14:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.433+0000 7f4a0d800640 1 -- 172.21.15.16:0/2400456237 learned_addr learned my addr 172.21.15.16:0/2400456237 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.433+0000 7f4a07e00640 1 --2- 172.21.15.16:0/2400456237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56555fecc4d0 0x56556029c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.433+0000 7f4a0d800640 1 --2- 172.21.15.16:0/2400456237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56555feed1b0 0x5655602a8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.434+0000 7f4a0d800640 1 -- 172.21.15.16:0/2400456237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56555feed1b0 msgr2=0x5655602a8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.434+0000 7f4a0d800640 1 --2- 172.21.15.16:0/2400456237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56555feed1b0 0x5655602a8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.434+0000 7f4a0d800640 1 -- 172.21.15.16:0/2400456237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56555fecc4d0 msgr2=0x56556029c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.434+0000 7f4a0d800640 1 --2- 172.21.15.16:0/2400456237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56555fecc4d0 0x56556029c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.434+0000 7f4a0d800640 1 -- 172.21.15.16:0/2400456237 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56555ff58790 con 0x56555fec42f0 2024-08-31T06:14:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.434+0000 7f4a07e00640 1 --2- 172.21.15.16:0/2400456237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56555fecc4d0 0x56556029c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:14:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.434+0000 7f4a0d800640 1 --2- 172.21.15.16:0/2400456237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56555fec42f0 0x56556029bf40 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4a00014040 tx=0x7f4a00004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a87f1e2cc1323863 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.434+0000 7f49eea00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:14:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.434+0000 7f49eea00640 1 -- 172.21.15.16:0/2400456237 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a00023040 con 0x56555fec42f0 2024-08-31T06:14:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.434+0000 7f4a16a48740 1 -- 172.21.15.16:0/2400456237 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a00003e20 con 0x56555fec42f0 2024-08-31T06:14:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.434+0000 7f49eea00640 1 -- 172.21.15.16:0/2400456237 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7f4a00004a40 con 0x56555fec42f0 2024-08-31T06:14:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.435+0000 7f49eea00640 1 -- 172.21.15.16:0/2400456237 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4a00004c40 con 0x56555fec42f0 2024-08-31T06:14:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.435+0000 7f4a16a48740 10 client.16932 fetch_fsmap learned FSMap version 39 2024-08-31T06:14:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.435+0000 7f4a16a48740 1 -- 172.21.15.16:0/2400456237 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56555ffd1620 con 0x56555fec42f0 2024-08-31T06:14:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.435+0000 7f49eea00640 1 -- 172.21.15.16:0/2400456237 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a00003e20 con 0x56555fec42f0 2024-08-31T06:14:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.435+0000 7f49eea00640 1 -- 172.21.15.16:0/2400456237 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f4a00018020 con 0x56555fec42f0 2024-08-31T06:14:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.435+0000 7f4a16a48740 10 client.16932 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T06:14:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.435+0000 7f4a16a48740 10 client.16932 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:14:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.435+0000 7f4a16a48740 20 client.16932 populate_metadata read hostname 'smithi016' 2024-08-31T06:14:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.435+0000 7f4a16a48740 1 --2- 172.21.15.16:0/2400456237 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56555feabd80 0x5655602adef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.435+0000 7f4a16a48740 4 client.16932 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:14:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.435+0000 7f4a16a48740 1 -- 172.21.15.16:0/2400456237 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565560207ec0 con 0x56555feabd80 2024-08-31T06:14:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.436+0000 7f4a07e00640 1 --2- 172.21.15.16:0/2400456237 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56555feabd80 0x5655602adef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.436+0000 7f4a07e00640 1 --2- 172.21.15.16:0/2400456237 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56555feabd80 0x5655602adef0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63f009053801d933 server_cookie=b7cabd055f573dce in_seq=0 out_seq=0 2024-08-31T06:14:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.436+0000 7f49eea00640 10 client.16932 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.437+0000 7f49eea00640 1 -- 172.21.15.16:0/2400456237 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x565560207ec0 con 0x56555feabd80 2024-08-31T06:14:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.437+0000 7f49eea00640 10 client.16932 handle_command_reply: tid=0 2024-08-31T06:14:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.437+0000 7f4a16a48740 1 client.16932 shutdown 2024-08-31T06:14:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.437+0000 7f4a16a48740 1 -- 172.21.15.16:0/2400456237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56555fec42f0 msgr2=0x56556029bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.437+0000 7f4a16a48740 1 --2- 172.21.15.16:0/2400456237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56555fec42f0 0x56556029bf40 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4a00014040 tx=0x7f4a00004170 comp rx=0 tx=0).stop 2024-08-31T06:14:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.438+0000 7f4a16a48740 1 -- 172.21.15.16:0/2400456237 shutdown_connections 2024-08-31T06:14:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.438+0000 7f4a16a48740 1 --2- 172.21.15.16:0/2400456237 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56555feabd80 0x5655602adef0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.438+0000 7f4a16a48740 1 --2- 172.21.15.16:0/2400456237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56555feed1b0 0x5655602a8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.438+0000 7f4a16a48740 1 --2- 172.21.15.16:0/2400456237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56555fecc4d0 0x56556029c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.438+0000 7f4a16a48740 1 --2- 172.21.15.16:0/2400456237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56555fec42f0 0x56556029bf40 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.438+0000 7f49eea00640 0 client.16932 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.438+0000 7f4a16a48740 1 -- 172.21.15.16:0/2400456237 >> 172.21.15.16:0/2400456237 conn(0x56555fee1300 msgr2=0x5655602918d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.438+0000 7f4a16a48740 1 -- 172.21.15.16:0/2400456237 shutdown_connections 2024-08-31T06:14:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.438+0000 7f4a16a48740 1 -- 172.21.15.16:0/2400456237 wait complete. 2024-08-31T06:14:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.438+0000 7f4a16a48740 20 client.16932 trim_cache size 0 max 16384 2024-08-31T06:14:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.454+0000 7f4a16a48740 1 Processor -- start 2024-08-31T06:14:18.457 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.454+0000 7f4a16a48740 1 -- start start 2024-08-31T06:14:18.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.455+0000 7f4a16a48740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56555fec42f0 0x56556028d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:18.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.455+0000 7f4a16a48740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56555fecc4d0 0x56556029db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:18.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.455+0000 7f4a16a48740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56555feed1b0 0x5655602a8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:18.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.455+0000 7f4a16a48740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4a081b3da0 con 0x56555fec42f0 2024-08-31T06:14:18.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.455+0000 7f4a16a48740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4a081b1610 con 0x56555feed1b0 2024-08-31T06:14:18.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.455+0000 7f4a16a48740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4a081af480 con 0x56555fecc4d0 2024-08-31T06:14:18.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.455+0000 7f4a0d800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56555fec42f0 0x56556028d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:18.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.455+0000 7f4a0d800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56555fec42f0 0x56556028d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56418/0 (socket says 172.21.15.16:56418) 2024-08-31T06:14:18.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.455+0000 7f4a0d800640 1 -- 172.21.15.16:0/132722000 learned_addr learned my addr 172.21.15.16:0/132722000 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:18.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.455+0000 7f4a07e00640 1 --2- 172.21.15.16:0/132722000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56555fecc4d0 0x56556029db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:18.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.455+0000 7f4a0d800640 1 --2- 172.21.15.16:0/132722000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56555feed1b0 0x5655602a8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:18.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.455+0000 7f4a0d800640 1 -- 172.21.15.16:0/132722000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56555fecc4d0 msgr2=0x56556029db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:18.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.455+0000 7f4a0d800640 1 --2- 172.21.15.16:0/132722000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56555fecc4d0 0x56556029db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:18.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.455+0000 7f4a0d800640 1 -- 172.21.15.16:0/132722000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56555feed1b0 msgr2=0x5655602a8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:18.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.455+0000 7f4a0d800640 1 --2- 172.21.15.16:0/132722000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56555feed1b0 0x5655602a8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:18.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.455+0000 7f4a0d800640 1 -- 172.21.15.16:0/132722000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56556029d200 con 0x56555fec42f0 2024-08-31T06:14:18.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.456+0000 7f4a0d800640 1 --2- 172.21.15.16:0/132722000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56555fec42f0 0x56556028d730 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4a00005d70 tx=0x7f4a00011cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb0cb76db38d6eeb server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:18.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.456+0000 7f49f9600640 1 -- 172.21.15.16:0/132722000 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a00005970 con 0x56555fec42f0 2024-08-31T06:14:18.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.456+0000 7f49f9600640 1 -- 172.21.15.16:0/132722000 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a0000fd70 con 0x56555fec42f0 2024-08-31T06:14:18.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.456+0000 7f49f9600640 1 -- 172.21.15.16:0/132722000 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a000041e0 con 0x56555fec42f0 2024-08-31T06:14:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.457+0000 7f4a16a48740 1 -- 172.21.15.16:0/132722000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56555fec42f0 msgr2=0x56556028d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.457+0000 7f4a16a48740 1 --2- 172.21.15.16:0/132722000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56555fec42f0 0x56556028d730 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4a00005d70 tx=0x7f4a00011cd0 comp rx=0 tx=0).stop 2024-08-31T06:14:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.457+0000 7f4a16a48740 1 -- 172.21.15.16:0/132722000 shutdown_connections 2024-08-31T06:14:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.457+0000 7f4a16a48740 1 --2- 172.21.15.16:0/132722000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56555fecc4d0 0x56556029db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.457+0000 7f4a16a48740 1 --2- 172.21.15.16:0/132722000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56555feed1b0 0x5655602a8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.457+0000 7f4a16a48740 1 --2- 172.21.15.16:0/132722000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56555fec42f0 0x56556028d730 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.457+0000 7f4a16a48740 1 -- 172.21.15.16:0/132722000 >> 172.21.15.16:0/132722000 conn(0x56555fee1300 msgr2=0x56556029d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.457+0000 7f4a16a48740 1 -- 172.21.15.16:0/132722000 shutdown_connections 2024-08-31T06:14:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.457+0000 7f4a16a48740 1 -- 172.21.15.16:0/132722000 wait complete. 2024-08-31T06:14:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.457+0000 7f4a16a48740 1 -- start start 2024-08-31T06:14:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.458+0000 7f4a16a48740 1 Processor -- start 2024-08-31T06:14:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.458+0000 7f4a16a48740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56555fec42f0 0x5655602a0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.458+0000 7f4a16a48740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56555fecc4d0 0x5655602a1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.458+0000 7f4a16a48740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56555feed1b0 0x565560311810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.458+0000 7f4a16a48740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4a081afad0 con 0x56555fecc4d0 2024-08-31T06:14:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.458+0000 7f4a16a48740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4a081b0f40 con 0x56555feed1b0 2024-08-31T06:14:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.458+0000 7f4a16a48740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4a081b01a0 con 0x56555fec42f0 2024-08-31T06:14:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.458+0000 7f4a16a48740 10 client.0 _ll_register_callbacks cb 0x7f49fc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:14:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.458+0000 7f4a07e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56555fecc4d0 0x5655602a1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.458+0000 7f4a07e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56555fecc4d0 0x5655602a1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56422/0 (socket says 172.21.15.16:56422) 2024-08-31T06:14:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.459+0000 7f4a07e00640 1 -- 172.21.15.16:0/2278820660 learned_addr learned my addr 172.21.15.16:0/2278820660 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.459+0000 7f4a0d800640 1 --2- 172.21.15.16:0/2278820660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56555feed1b0 0x565560311810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.459+0000 7f4a0d800640 1 --2- 172.21.15.16:0/2278820660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56555fec42f0 0x5655602a0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.459+0000 7f4a07e00640 1 -- 172.21.15.16:0/2278820660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56555fec42f0 msgr2=0x5655602a0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.459+0000 7f4a07e00640 1 --2- 172.21.15.16:0/2278820660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56555fec42f0 0x5655602a0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.459+0000 7f4a07e00640 1 -- 172.21.15.16:0/2278820660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56555feed1b0 msgr2=0x565560311810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.459+0000 7f4a07e00640 1 --2- 172.21.15.16:0/2278820660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56555feed1b0 0x565560311810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.459+0000 7f4a07e00640 1 -- 172.21.15.16:0/2278820660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565560311eb0 con 0x56555fecc4d0 2024-08-31T06:14:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.459+0000 7f4a0d800640 1 --2- 172.21.15.16:0/2278820660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56555fec42f0 0x5655602a0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:14:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.459+0000 7f4a07e00640 1 --2- 172.21.15.16:0/2278820660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56555fecc4d0 0x5655602a1390 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f49f402a450 tx=0x7f49f4005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85a5847054daa68 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.459+0000 7f49eea00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:14:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.459+0000 7f49eea00640 1 -- 172.21.15.16:0/2278820660 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f4035040 con 0x56555fecc4d0 2024-08-31T06:14:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.460+0000 7f4a16a48740 1 -- 172.21.15.16:0/2278820660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a0000fd70 con 0x56555fecc4d0 2024-08-31T06:14:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.460+0000 7f49eea00640 1 -- 172.21.15.16:0/2278820660 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7f49f4005040 con 0x56555fecc4d0 2024-08-31T06:14:18.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.460+0000 7f49eea00640 1 -- 172.21.15.16:0/2278820660 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f49f4005240 con 0x56555fecc4d0 2024-08-31T06:14:18.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.460+0000 7f4a16a48740 10 client.16938 fetch_fsmap learned FSMap version 39 2024-08-31T06:14:18.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.460+0000 7f4a16a48740 1 -- 172.21.15.16:0/2278820660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x565560319330 con 0x56555fecc4d0 2024-08-31T06:14:18.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.460+0000 7f49eea00640 1 -- 172.21.15.16:0/2278820660 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a0000fd70 con 0x56555fecc4d0 2024-08-31T06:14:18.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.460+0000 7f49eea00640 1 -- 172.21.15.16:0/2278820660 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f49f40035f0 con 0x56555fecc4d0 2024-08-31T06:14:18.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.460+0000 7f4a16a48740 10 client.16938 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T06:14:18.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.460+0000 7f4a16a48740 10 client.16938 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:14:18.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.460+0000 7f4a16a48740 20 client.16938 populate_metadata read hostname 'smithi016' 2024-08-31T06:14:18.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.460+0000 7f4a16a48740 1 --2- 172.21.15.16:0/2278820660 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56555fe811c0 0x56556031bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:18.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.460+0000 7f4a16a48740 4 client.16938 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:14:18.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.460+0000 7f4a16a48740 1 -- 172.21.15.16:0/2278820660 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565560203960 con 0x56555fe811c0 2024-08-31T06:14:18.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.461+0000 7f4a0d800640 1 --2- 172.21.15.16:0/2278820660 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56555fe811c0 0x56556031bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:18.467 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.461+0000 7f4a0d800640 1 --2- 172.21.15.16:0/2278820660 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56555fe811c0 0x56556031bb30 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=954ed600804e8257 server_cookie=fe86e8d81c29ef4d in_seq=0 out_seq=0 2024-08-31T06:14:18.467 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.461+0000 7f49eea00640 10 client.16938 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.480+0000 7f49eea00640 1 -- 172.21.15.16:0/2278820660 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x565560203960 con 0x56555fe811c0 2024-08-31T06:14:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.480+0000 7f49eea00640 10 client.16938 handle_command_reply: tid=0 2024-08-31T06:14:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.480+0000 7f4a16a48740 1 client.16938 shutdown 2024-08-31T06:14:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.480+0000 7f4a16a48740 1 -- 172.21.15.16:0/2278820660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56555fecc4d0 msgr2=0x5655602a1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.480+0000 7f4a16a48740 1 --2- 172.21.15.16:0/2278820660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56555fecc4d0 0x5655602a1390 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f49f402a450 tx=0x7f49f4005db0 comp rx=0 tx=0).stop 2024-08-31T06:14:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.480+0000 7f4a16a48740 1 -- 172.21.15.16:0/2278820660 shutdown_connections 2024-08-31T06:14:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.480+0000 7f4a16a48740 1 --2- 172.21.15.16:0/2278820660 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56555fe811c0 0x56556031bb30 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:18.484 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.480+0000 7f4a16a48740 1 --2- 172.21.15.16:0/2278820660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56555feed1b0 0x565560311810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:18.484 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.480+0000 7f4a16a48740 1 --2- 172.21.15.16:0/2278820660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56555fecc4d0 0x5655602a1390 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:18.484 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.480+0000 7f4a16a48740 1 --2- 172.21.15.16:0/2278820660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56555fec42f0 0x5655602a0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:18.484 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.480+0000 7f49eea00640 0 client.16938 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:18.484 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.480+0000 7f4a16a48740 1 -- 172.21.15.16:0/2278820660 >> 172.21.15.16:0/2278820660 conn(0x56555fee1300 msgr2=0x56556028d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:18.484 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.481+0000 7f4a16a48740 1 -- 172.21.15.16:0/2278820660 shutdown_connections 2024-08-31T06:14:18.484 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.481+0000 7f4a16a48740 1 -- 172.21.15.16:0/2278820660 wait complete. 2024-08-31T06:14:18.484 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.481+0000 7f4a16a48740 20 client.16938 trim_cache size 0 max 16384 2024-08-31T06:14:18.484 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:14:18.485 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:14:18.485 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:14:18.485 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:14:18.485 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.481+0000 7f4a0e200640 1 -- 172.21.15.16:0/1206821495 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f49f00b4d20 msgr2=0x7f49f00b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:18.485 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.481+0000 7f4a0e200640 1 --2- 172.21.15.16:0/1206821495 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f49f00b4d20 0x7f49f00b7230 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x56556028f520 tx=0x7f49fc004a90 comp rx=0 tx=0).stop 2024-08-31T06:14:18.485 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.481+0000 7f4a0e200640 1 -- 172.21.15.16:0/1206821495 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a081058c0 msgr2=0x7f4a0819c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.481+0000 7f4a0e200640 1 --2- 172.21.15.16:0/1206821495 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a081058c0 0x7f4a0819c240 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f49fc00e9f0 tx=0x7f49fc00edf0 comp rx=0 tx=0).stop 2024-08-31T06:14:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.482+0000 7f4a0e200640 1 -- 172.21.15.16:0/1206821495 shutdown_connections 2024-08-31T06:14:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.482+0000 7f4a0e200640 1 --2- 172.21.15.16:0/1206821495 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f49f00b4d20 0x7f49f00b7230 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.482+0000 7f4a0e200640 1 --2- 172.21.15.16:0/1206821495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a0810bb40 0x7f4a081a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.482+0000 7f4a0e200640 1 --2- 172.21.15.16:0/1206821495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a0810b1f0 0x7f4a0819c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.482+0000 7f4a0e200640 1 --2- 172.21.15.16:0/1206821495 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a081058c0 0x7f4a0819c240 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.482+0000 7f4a0e200640 1 -- 172.21.15.16:0/1206821495 >> 172.21.15.16:0/1206821495 conn(0x7f4a08101300 msgr2=0x7f4a08103720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.482+0000 7f4a0e200640 1 -- 172.21.15.16:0/1206821495 shutdown_connections 2024-08-31T06:14:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:18.482+0000 7f4a0e200640 1 -- 172.21.15.16:0/1206821495 wait complete. 2024-08-31T06:14:18.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:18 smithi184 ceph-mon[22331]: pgmap v580: 97 pgs: 97 active+clean; 3.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 3.7 MiB/s wr, 297 op/s 2024-08-31T06:14:19.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:18 smithi155 ceph-mon[21742]: pgmap v580: 97 pgs: 97 active+clean; 3.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 3.7 MiB/s wr, 297 op/s 2024-08-31T06:14:19.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:18 smithi016 ceph-mon[17194]: pgmap v580: 97 pgs: 97 active+clean; 3.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 3.7 MiB/s wr, 297 op/s 2024-08-31T06:14:19.257 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T06:14:19.257 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T06:14:19.642 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:14:20.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.294+0000 7f9388800640 1 -- 172.21.15.16:0/3864670867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9380105b50 msgr2=0x7f9380105f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:20.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.294+0000 7f9388800640 1 --2- 172.21.15.16:0/3864670867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9380105b50 0x7f9380105f60 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9374009900 tx=0x7f937402a410 comp rx=0 tx=0).stop 2024-08-31T06:14:20.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.294+0000 7f9388800640 1 -- 172.21.15.16:0/3864670867 shutdown_connections 2024-08-31T06:14:20.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.294+0000 7f9388800640 1 --2- 172.21.15.16:0/3864670867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f938010b490 0x7f938010b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:20.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.294+0000 7f9388800640 1 --2- 172.21.15.16:0/3864670867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f938010bde0 0x7f9380112610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:20.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.294+0000 7f9388800640 1 --2- 172.21.15.16:0/3864670867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9380105b50 0x7f9380105f60 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:20.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.294+0000 7f9388800640 1 -- 172.21.15.16:0/3864670867 >> 172.21.15.16:0/3864670867 conn(0x7f9380101580 msgr2=0x7f93801039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:20.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.294+0000 7f9388800640 1 -- 172.21.15.16:0/3864670867 shutdown_connections 2024-08-31T06:14:20.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.294+0000 7f9388800640 1 -- 172.21.15.16:0/3864670867 wait complete. 2024-08-31T06:14:20.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.295+0000 7f9388800640 1 Processor -- start 2024-08-31T06:14:20.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.296+0000 7f9388800640 1 -- start start 2024-08-31T06:14:20.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.296+0000 7f9388800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9380105b50 0x7f938019c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:20.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.296+0000 7f9388800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f938010b490 0x7f938019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:20.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.296+0000 7f9388800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f938010bde0 0x7f93801a1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:20.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.296+0000 7f9388800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9380115af0 con 0x7f938010b490 2024-08-31T06:14:20.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.296+0000 7f9388800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9380115970 con 0x7f9380105b50 2024-08-31T06:14:20.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.296+0000 7f9388800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9380115c70 con 0x7f938010bde0 2024-08-31T06:14:20.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.297+0000 7f9385e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9380105b50 0x7f938019c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:20.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.297+0000 7f9385e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9380105b50 0x7f938019c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:33734/0 (socket says 172.21.15.16:33734) 2024-08-31T06:14:20.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.297+0000 7f9385e00640 1 -- 172.21.15.16:0/3131983024 learned_addr learned my addr 172.21.15.16:0/3131983024 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:20.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.297+0000 7f9386800640 1 --2- 172.21.15.16:0/3131983024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f938010bde0 0x7f93801a1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:20.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.297+0000 7f9385400640 1 --2- 172.21.15.16:0/3131983024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f938010b490 0x7f938019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:20.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.297+0000 7f9385e00640 1 -- 172.21.15.16:0/3131983024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f938010bde0 msgr2=0x7f93801a1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:20.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.297+0000 7f9385e00640 1 --2- 172.21.15.16:0/3131983024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f938010bde0 0x7f93801a1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:20.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.297+0000 7f9385e00640 1 -- 172.21.15.16:0/3131983024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f938010b490 msgr2=0x7f938019ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:20.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.297+0000 7f9385e00640 1 --2- 172.21.15.16:0/3131983024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f938010b490 0x7f938019ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:20.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.297+0000 7f9385e00640 1 -- 172.21.15.16:0/3131983024 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93801a2450 con 0x7f9380105b50 2024-08-31T06:14:20.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.297+0000 7f9386800640 1 --2- 172.21.15.16:0/3131983024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f938010bde0 0x7f93801a1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:14:20.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.298+0000 7f9385e00640 1 --2- 172.21.15.16:0/3131983024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9380105b50 0x7f938019c4e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9374005ab0 tx=0x7f9374003c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:20.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.298+0000 7f937aa00640 1 -- 172.21.15.16:0/3131983024 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f937403f070 con 0x7f9380105b50 2024-08-31T06:14:20.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.298+0000 7f9388800640 1 -- 172.21.15.16:0/3131983024 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93801a26e0 con 0x7f9380105b50 2024-08-31T06:14:20.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.298+0000 7f937aa00640 1 -- 172.21.15.16:0/3131983024 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9374031d60 con 0x7f9380105b50 2024-08-31T06:14:20.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.298+0000 7f937aa00640 1 -- 172.21.15.16:0/3131983024 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9374035030 con 0x7f9380105b50 2024-08-31T06:14:20.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.298+0000 7f9388800640 1 -- 172.21.15.16:0/3131983024 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93801a8ea0 con 0x7f9380105b50 2024-08-31T06:14:20.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.302+0000 7f938f3e6740 1 Processor -- start 2024-08-31T06:14:20.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.302+0000 7f938f3e6740 1 -- start start 2024-08-31T06:14:20.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.302+0000 7f937aa00640 1 -- 172.21.15.16:0/3131983024 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f9374042070 con 0x7f9380105b50 2024-08-31T06:14:20.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.302+0000 7f938f3e6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dbca6d350 0x560dbce13600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:20.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.302+0000 7f938f3e6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dbca44490 0x560dbce13b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:20.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.302+0000 7f938f3e6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560dbca4c670 0x560dbce14080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:20.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.302+0000 7f938f3e6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560dbcd89750 con 0x560dbca44490 2024-08-31T06:14:20.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.302+0000 7f938f3e6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560dbcd78ce0 con 0x560dbca4c670 2024-08-31T06:14:20.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.302+0000 7f938f3e6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560dbcd7cc90 con 0x560dbca6d350 2024-08-31T06:14:20.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.303+0000 7f9385e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dbca44490 0x560dbce13b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:20.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.303+0000 7f9386800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dbca6d350 0x560dbce13600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:20.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.303+0000 7f9385e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dbca44490 0x560dbce13b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56454/0 (socket says 172.21.15.16:56454) 2024-08-31T06:14:20.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.303+0000 7f9385e00640 1 -- 172.21.15.16:0/2293220533 learned_addr learned my addr 172.21.15.16:0/2293220533 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:20.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.303+0000 7f9385400640 1 --2- 172.21.15.16:0/2293220533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560dbca4c670 0x560dbce14080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:20.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.303+0000 7f9386800640 1 -- 172.21.15.16:0/2293220533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560dbca4c670 msgr2=0x560dbce14080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:20.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.303+0000 7f9386800640 1 --2- 172.21.15.16:0/2293220533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560dbca4c670 0x560dbce14080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:20.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.303+0000 7f9386800640 1 -- 172.21.15.16:0/2293220533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dbca44490 msgr2=0x560dbce13b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:20.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.303+0000 7f9386800640 1 --2- 172.21.15.16:0/2293220533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dbca44490 0x560dbce13b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:20.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.303+0000 7f9386800640 1 -- 172.21.15.16:0/2293220533 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dbcad8930 con 0x560dbca6d350 2024-08-31T06:14:20.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.303+0000 7f9386800640 1 --2- 172.21.15.16:0/2293220533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dbca6d350 0x560dbce13600 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f937c0106f0 tx=0x7f937c010bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97f6b7ad2d514813 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:20.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.304+0000 7f937aa00640 1 --2- 172.21.15.16:0/3131983024 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f93680b4ca0 0x7f93680b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:20.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.304+0000 7f936f000640 1 -- 172.21.15.16:0/2293220533 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f937c003b70 con 0x560dbca6d350 2024-08-31T06:14:20.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.304+0000 7f936f000640 1 -- 172.21.15.16:0/2293220533 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f937c00ede0 con 0x560dbca6d350 2024-08-31T06:14:20.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.304+0000 7f936f000640 1 -- 172.21.15.16:0/2293220533 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f937c004690 con 0x560dbca6d350 2024-08-31T06:14:20.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.304+0000 7f937aa00640 1 -- 172.21.15.16:0/3131983024 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7f9374105d50 con 0x7f9380105b50 2024-08-31T06:14:20.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.304+0000 7f9385e00640 1 --2- 172.21.15.16:0/3131983024 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f93680b4ca0 0x7f93680b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:20.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.305+0000 7f938f3e6740 1 -- 172.21.15.16:0/2293220533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dbca6d350 msgr2=0x560dbce13600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:20.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.305+0000 7f9385e00640 1 --2- 172.21.15.16:0/3131983024 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f93680b4ca0 0x7f93680b71b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9374105ff0 tx=0x7f9374109800 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:20.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.305+0000 7f938f3e6740 1 --2- 172.21.15.16:0/2293220533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dbca6d350 0x560dbce13600 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f937c0106f0 tx=0x7f937c010bb0 comp rx=0 tx=0).stop 2024-08-31T06:14:20.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.305+0000 7f938f3e6740 1 -- 172.21.15.16:0/2293220533 shutdown_connections 2024-08-31T06:14:20.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.305+0000 7f938f3e6740 1 --2- 172.21.15.16:0/2293220533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560dbca4c670 0x560dbce14080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:20.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.305+0000 7f938f3e6740 1 --2- 172.21.15.16:0/2293220533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dbca44490 0x560dbce13b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:20.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.305+0000 7f938f3e6740 1 --2- 172.21.15.16:0/2293220533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dbca6d350 0x560dbce13600 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:20.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.305+0000 7f938f3e6740 1 -- 172.21.15.16:0/2293220533 >> 172.21.15.16:0/2293220533 conn(0x560dbca61740 msgr2=0x560dbce114f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:20.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.305+0000 7f938f3e6740 1 -- 172.21.15.16:0/2293220533 shutdown_connections 2024-08-31T06:14:20.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.305+0000 7f938f3e6740 1 -- 172.21.15.16:0/2293220533 wait complete. 2024-08-31T06:14:20.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.306+0000 7f938f3e6740 1 -- start start 2024-08-31T06:14:20.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.306+0000 7f938f3e6740 1 Processor -- start 2024-08-31T06:14:20.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.307+0000 7f938f3e6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560dbca44490 0x560dbce1f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:20.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.307+0000 7f938f3e6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dbca4c670 0x560dbce1f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:20.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.307+0000 7f938f3e6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dbca6d350 0x560dbce2b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:20.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.307+0000 7f938f3e6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560dbcbdb890 con 0x560dbca6d350 2024-08-31T06:14:20.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.307+0000 7f938f3e6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560dbccffcd0 con 0x560dbca44490 2024-08-31T06:14:20.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.307+0000 7f938f3e6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560dbce21de0 con 0x560dbca4c670 2024-08-31T06:14:20.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.307+0000 7f938f3e6740 10 client.0 _ll_register_callbacks cb 0x560dbcd73ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:14:20.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.307+0000 7f9386800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560dbca44490 0x560dbce1f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:20.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.307+0000 7f9386800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dbca6d350 0x560dbce2b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:20.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.307+0000 7f9385400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dbca4c670 0x560dbce1f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:20.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.307+0000 7f9385400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dbca4c670 0x560dbce1f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:44750/0 (socket says 172.21.15.16:44750) 2024-08-31T06:14:20.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.307+0000 7f9385400640 1 -- 172.21.15.16:0/2937008156 learned_addr learned my addr 172.21.15.16:0/2937008156 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:20.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.308+0000 7f9386800640 1 -- 172.21.15.16:0/2937008156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dbca4c670 msgr2=0x560dbce1f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:20.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.308+0000 7f9386800640 1 --2- 172.21.15.16:0/2937008156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dbca4c670 0x560dbce1f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:20.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.308+0000 7f9386800640 1 -- 172.21.15.16:0/2937008156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560dbca44490 msgr2=0x560dbce1f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:20.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.308+0000 7f9386800640 1 --2- 172.21.15.16:0/2937008156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560dbca44490 0x560dbce1f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:20.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.308+0000 7f9386800640 1 -- 172.21.15.16:0/2937008156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dbcd491b0 con 0x560dbca6d350 2024-08-31T06:14:20.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.308+0000 7f9386800640 1 --2- 172.21.15.16:0/2937008156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dbca6d350 0x560dbce2b0d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f937c004370 tx=0x7f937c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b890c36f6e00792 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:20.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.308+0000 7f9363400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:14:20.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.308+0000 7f9363400640 1 -- 172.21.15.16:0/2937008156 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f937c024040 con 0x560dbca6d350 2024-08-31T06:14:20.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.308+0000 7f9363400640 1 -- 172.21.15.16:0/2937008156 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7f937c00eb30 con 0x560dbca6d350 2024-08-31T06:14:20.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.308+0000 7f938f3e6740 1 -- 172.21.15.16:0/2937008156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f937c00ede0 con 0x560dbca6d350 2024-08-31T06:14:20.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.309+0000 7f9363400640 1 -- 172.21.15.16:0/2937008156 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f937c02f050 con 0x560dbca6d350 2024-08-31T06:14:20.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.309+0000 7f938f3e6740 10 client.16950 fetch_fsmap learned FSMap version 39 2024-08-31T06:14:20.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.309+0000 7f938f3e6740 1 -- 172.21.15.16:0/2937008156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560dbce2ddf0 con 0x560dbca6d350 2024-08-31T06:14:20.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.309+0000 7f9363400640 1 -- 172.21.15.16:0/2937008156 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f937c00ede0 con 0x560dbca6d350 2024-08-31T06:14:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.309+0000 7f9363400640 1 -- 172.21.15.16:0/2937008156 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f937c005050 con 0x560dbca6d350 2024-08-31T06:14:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.309+0000 7f938f3e6740 10 client.16950 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T06:14:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.309+0000 7f938f3e6740 10 client.16950 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:14:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.309+0000 7f938f3e6740 20 client.16950 populate_metadata read hostname 'smithi016' 2024-08-31T06:14:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.309+0000 7f938f3e6740 1 --2- 172.21.15.16:0/2937008156 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x560dbca2bcd0 0x560dbce31200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.309+0000 7f938f3e6740 4 client.16950 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:14:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.309+0000 7f938f3e6740 1 -- 172.21.15.16:0/2937008156 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560dbcd8b6a0 con 0x560dbca2bcd0 2024-08-31T06:14:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.310+0000 7f9385400640 1 --2- 172.21.15.16:0/2937008156 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x560dbca2bcd0 0x560dbce31200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.310+0000 7f9385400640 1 --2- 172.21.15.16:0/2937008156 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x560dbca2bcd0 0x560dbce31200 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b25e0959ae1c5fd server_cookie=5bd4f769062ba425 in_seq=0 out_seq=0 2024-08-31T06:14:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.310+0000 7f9363400640 10 client.16950 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.311+0000 7f9363400640 1 -- 172.21.15.16:0/2937008156 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560dbcd8b6a0 con 0x560dbca2bcd0 2024-08-31T06:14:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.311+0000 7f9363400640 10 client.16950 handle_command_reply: tid=0 2024-08-31T06:14:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.311+0000 7f938f3e6740 1 client.16950 shutdown 2024-08-31T06:14:20.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.311+0000 7f938f3e6740 1 -- 172.21.15.16:0/2937008156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dbca6d350 msgr2=0x560dbce2b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:20.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.311+0000 7f938f3e6740 1 --2- 172.21.15.16:0/2937008156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dbca6d350 0x560dbce2b0d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f937c004370 tx=0x7f937c000f40 comp rx=0 tx=0).stop 2024-08-31T06:14:20.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.312+0000 7f938f3e6740 1 -- 172.21.15.16:0/2937008156 shutdown_connections 2024-08-31T06:14:20.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.312+0000 7f938f3e6740 1 --2- 172.21.15.16:0/2937008156 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x560dbca2bcd0 0x560dbce31200 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:20.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.312+0000 7f938f3e6740 1 --2- 172.21.15.16:0/2937008156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dbca4c670 0x560dbce1f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:20.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.312+0000 7f938f3e6740 1 --2- 172.21.15.16:0/2937008156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dbca6d350 0x560dbce2b0d0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:20.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.312+0000 7f938f3e6740 1 --2- 172.21.15.16:0/2937008156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560dbca44490 0x560dbce1f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:20.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.312+0000 7f9363400640 0 client.16950 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:20.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.312+0000 7f938f3e6740 1 -- 172.21.15.16:0/2937008156 >> 172.21.15.16:0/2937008156 conn(0x560dbca61740 msgr2=0x560dbce16790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:20.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.312+0000 7f938f3e6740 1 -- 172.21.15.16:0/2937008156 shutdown_connections 2024-08-31T06:14:20.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.312+0000 7f938f3e6740 1 -- 172.21.15.16:0/2937008156 wait complete. 2024-08-31T06:14:20.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.312+0000 7f938f3e6740 20 client.16950 trim_cache size 0 max 16384 2024-08-31T06:14:20.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.331+0000 7f938f3e6740 1 Processor -- start 2024-08-31T06:14:20.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.331+0000 7f938f3e6740 1 -- start start 2024-08-31T06:14:20.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.331+0000 7f938f3e6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560dbca44490 0x560dbce19720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:20.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.331+0000 7f938f3e6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dbca4c670 0x560dbce20bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:20.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.331+0000 7f938f3e6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dbca6d350 0x560dbce1c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:20.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.331+0000 7f938f3e6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f93801b3fb0 con 0x560dbca6d350 2024-08-31T06:14:20.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.331+0000 7f938f3e6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f93801b1820 con 0x560dbca44490 2024-08-31T06:14:20.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.331+0000 7f938f3e6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f93801af690 con 0x560dbca4c670 2024-08-31T06:14:20.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.332+0000 7f9385400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dbca4c670 0x560dbce20bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:20.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.332+0000 7f9386800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560dbca44490 0x560dbce19720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:20.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.332+0000 7f9385400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dbca4c670 0x560dbce20bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:44752/0 (socket says 172.21.15.16:44752) 2024-08-31T06:14:20.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.332+0000 7f9385400640 1 -- 172.21.15.16:0/1486406292 learned_addr learned my addr 172.21.15.16:0/1486406292 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:20.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.332+0000 7f9386800640 1 --2- 172.21.15.16:0/1486406292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dbca6d350 0x560dbce1c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:20.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.332+0000 7f9385400640 1 -- 172.21.15.16:0/1486406292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560dbca44490 msgr2=0x560dbce19720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:20.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.332+0000 7f9385400640 1 --2- 172.21.15.16:0/1486406292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560dbca44490 0x560dbce19720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:20.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.332+0000 7f9385400640 1 -- 172.21.15.16:0/1486406292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dbca6d350 msgr2=0x560dbce1c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:20.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.332+0000 7f9385400640 1 --2- 172.21.15.16:0/1486406292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dbca6d350 0x560dbce1c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:20.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.332+0000 7f9385400640 1 -- 172.21.15.16:0/1486406292 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dbce20260 con 0x560dbca4c670 2024-08-31T06:14:20.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.332+0000 7f9385400640 1 --2- 172.21.15.16:0/1486406292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dbca4c670 0x560dbce20bd0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9370009e10 tx=0x7f9370005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1cda84f6296bd2d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:20.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.333+0000 7f936e600640 1 -- 172.21.15.16:0/1486406292 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9370005220 con 0x560dbca4c670 2024-08-31T06:14:20.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.333+0000 7f936e600640 1 -- 172.21.15.16:0/1486406292 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9370004c50 con 0x560dbca4c670 2024-08-31T06:14:20.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.333+0000 7f936e600640 1 -- 172.21.15.16:0/1486406292 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f937001fa00 con 0x560dbca4c670 2024-08-31T06:14:20.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.333+0000 7f938f3e6740 1 -- 172.21.15.16:0/1486406292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dbca4c670 msgr2=0x560dbce20bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:20.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.333+0000 7f938f3e6740 1 --2- 172.21.15.16:0/1486406292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dbca4c670 0x560dbce20bd0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9370009e10 tx=0x7f9370005f90 comp rx=0 tx=0).stop 2024-08-31T06:14:20.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.333+0000 7f938f3e6740 1 -- 172.21.15.16:0/1486406292 shutdown_connections 2024-08-31T06:14:20.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.334+0000 7f938f3e6740 1 --2- 172.21.15.16:0/1486406292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dbca4c670 0x560dbce20bd0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:20.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.334+0000 7f938f3e6740 1 --2- 172.21.15.16:0/1486406292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dbca6d350 0x560dbce1c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:20.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.334+0000 7f938f3e6740 1 --2- 172.21.15.16:0/1486406292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560dbca44490 0x560dbce19720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:20.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.334+0000 7f938f3e6740 1 -- 172.21.15.16:0/1486406292 >> 172.21.15.16:0/1486406292 conn(0x560dbca61740 msgr2=0x560dbce204c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:20.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.334+0000 7f938f3e6740 1 -- 172.21.15.16:0/1486406292 shutdown_connections 2024-08-31T06:14:20.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.334+0000 7f938f3e6740 1 -- 172.21.15.16:0/1486406292 wait complete. 2024-08-31T06:14:20.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.334+0000 7f938f3e6740 1 -- start start 2024-08-31T06:14:20.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.334+0000 7f938f3e6740 1 Processor -- start 2024-08-31T06:14:20.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.335+0000 7f938f3e6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560dbca44490 0x560dbce238f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:20.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.335+0000 7f938f3e6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dbca4c670 0x560dbce23ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:20.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.335+0000 7f938f3e6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dbca6d350 0x560dbce94a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:20.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.335+0000 7f938f3e6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f93801afce0 con 0x560dbca4c670 2024-08-31T06:14:20.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.335+0000 7f938f3e6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f93801b1150 con 0x560dbca44490 2024-08-31T06:14:20.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.335+0000 7f938f3e6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f93801b03b0 con 0x560dbca6d350 2024-08-31T06:14:20.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.335+0000 7f938f3e6740 10 client.0 _ll_register_callbacks cb 0x7f93741096d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:14:20.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.335+0000 7f9385400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dbca4c670 0x560dbce23ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:20.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.335+0000 7f9385400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dbca4c670 0x560dbce23ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56470/0 (socket says 172.21.15.16:56470) 2024-08-31T06:14:20.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.335+0000 7f9386800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dbca6d350 0x560dbce94a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:20.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.335+0000 7f9385400640 1 -- 172.21.15.16:0/1434892742 learned_addr learned my addr 172.21.15.16:0/1434892742 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:20.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.335+0000 7f9386800640 1 --2- 172.21.15.16:0/1434892742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560dbca44490 0x560dbce238f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:20.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.336+0000 7f9385400640 1 -- 172.21.15.16:0/1434892742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dbca6d350 msgr2=0x560dbce94a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:20.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.336+0000 7f9385400640 1 --2- 172.21.15.16:0/1434892742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dbca6d350 0x560dbce94a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:20.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.336+0000 7f9385400640 1 -- 172.21.15.16:0/1434892742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560dbca44490 msgr2=0x560dbce238f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:20.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.336+0000 7f9385400640 1 --2- 172.21.15.16:0/1434892742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560dbca44490 0x560dbce238f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:20.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.336+0000 7f9385400640 1 -- 172.21.15.16:0/1434892742 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dbce950e0 con 0x560dbca4c670 2024-08-31T06:14:20.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.336+0000 7f9385400640 1 --2- 172.21.15.16:0/1434892742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dbca4c670 0x560dbce23ed0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f937001f640 tx=0x7f9370004f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9fc13b9a0f96244 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:20.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.336+0000 7f9363400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:14:20.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.336+0000 7f9363400640 1 -- 172.21.15.16:0/1434892742 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9370034060 con 0x560dbca4c670 2024-08-31T06:14:20.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.336+0000 7f9363400640 1 -- 172.21.15.16:0/1434892742 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7f9370004030 con 0x560dbca4c670 2024-08-31T06:14:20.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.336+0000 7f938f3e6740 1 -- 172.21.15.16:0/1434892742 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9370004c50 con 0x560dbca4c670 2024-08-31T06:14:20.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.337+0000 7f9363400640 1 -- 172.21.15.16:0/1434892742 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9370029040 con 0x560dbca4c670 2024-08-31T06:14:20.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.337+0000 7f938f3e6740 10 client.16956 fetch_fsmap learned FSMap version 39 2024-08-31T06:14:20.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.337+0000 7f938f3e6740 1 -- 172.21.15.16:0/1434892742 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560dbce9c3b0 con 0x560dbca4c670 2024-08-31T06:14:20.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.337+0000 7f9363400640 1 -- 172.21.15.16:0/1434892742 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9370004c50 con 0x560dbca4c670 2024-08-31T06:14:20.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.337+0000 7f9363400640 1 -- 172.21.15.16:0/1434892742 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f937001f730 con 0x560dbca4c670 2024-08-31T06:14:20.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.337+0000 7f938f3e6740 10 client.16956 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T06:14:20.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.337+0000 7f938f3e6740 10 client.16956 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:14:20.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.337+0000 7f938f3e6740 20 client.16956 populate_metadata read hostname 'smithi016' 2024-08-31T06:14:20.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.337+0000 7f938f3e6740 1 --2- 172.21.15.16:0/1434892742 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x560dbca01080 0x560dbce9eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:20.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.337+0000 7f938f3e6740 4 client.16956 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T06:14:20.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.337+0000 7f938f3e6740 1 -- 172.21.15.16:0/1434892742 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560dbcd87140 con 0x560dbca01080 2024-08-31T06:14:20.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.338+0000 7f9386800640 1 --2- 172.21.15.16:0/1434892742 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x560dbca01080 0x560dbce9eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:20.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.338+0000 7f9386800640 1 --2- 172.21.15.16:0/1434892742 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x560dbca01080 0x560dbce9eb20 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43f3cad9e5473bab server_cookie=30a21dbe0415881b in_seq=0 out_seq=0 2024-08-31T06:14:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.338+0000 7f9363400640 10 client.16956 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.340+0000 7f9363400640 1 -- 172.21.15.16:0/1434892742 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560dbcd87140 con 0x560dbca01080 2024-08-31T06:14:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.340+0000 7f9363400640 10 client.16956 handle_command_reply: tid=0 2024-08-31T06:14:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.340+0000 7f938f3e6740 1 client.16956 shutdown 2024-08-31T06:14:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.340+0000 7f938f3e6740 1 -- 172.21.15.16:0/1434892742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dbca4c670 msgr2=0x560dbce23ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.340+0000 7f938f3e6740 1 --2- 172.21.15.16:0/1434892742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dbca4c670 0x560dbce23ed0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f937001f640 tx=0x7f9370004f60 comp rx=0 tx=0).stop 2024-08-31T06:14:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.340+0000 7f938f3e6740 1 -- 172.21.15.16:0/1434892742 shutdown_connections 2024-08-31T06:14:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.340+0000 7f938f3e6740 1 --2- 172.21.15.16:0/1434892742 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x560dbca01080 0x560dbce9eb20 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.340+0000 7f938f3e6740 1 --2- 172.21.15.16:0/1434892742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dbca6d350 0x560dbce94a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.340+0000 7f938f3e6740 1 --2- 172.21.15.16:0/1434892742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dbca4c670 0x560dbce23ed0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.340+0000 7f938f3e6740 1 --2- 172.21.15.16:0/1434892742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560dbca44490 0x560dbce238f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.340+0000 7f938f3e6740 1 -- 172.21.15.16:0/1434892742 >> 172.21.15.16:0/1434892742 conn(0x560dbca61740 msgr2=0x560dbce1d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.340+0000 7f9363400640 0 client.16956 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:20.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.340+0000 7f938f3e6740 1 -- 172.21.15.16:0/1434892742 shutdown_connections 2024-08-31T06:14:20.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.340+0000 7f938f3e6740 1 -- 172.21.15.16:0/1434892742 wait complete. 2024-08-31T06:14:20.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.341+0000 7f938f3e6740 20 client.16956 trim_cache size 0 max 16384 2024-08-31T06:14:20.345 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:14:20.345 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-08-31T06:14:20.345 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "a64d5301-8558-4e9c-ac9b-8d0ac7057e74", 2024-08-31T06:14:20.345 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-08-31T06:14:20.345 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:14:20.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.341+0000 7f9388800640 1 -- 172.21.15.16:0/3131983024 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f93680b4ca0 msgr2=0x7f93680b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:20.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.341+0000 7f9388800640 1 --2- 172.21.15.16:0/3131983024 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f93680b4ca0 0x7f93680b71b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9374105ff0 tx=0x7f9374109800 comp rx=0 tx=0).stop 2024-08-31T06:14:20.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.341+0000 7f9388800640 1 -- 172.21.15.16:0/3131983024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9380105b50 msgr2=0x7f938019c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:20.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.341+0000 7f9388800640 1 --2- 172.21.15.16:0/3131983024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9380105b50 0x7f938019c4e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9374005ab0 tx=0x7f9374003c80 comp rx=0 tx=0).stop 2024-08-31T06:14:20.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.342+0000 7f9388800640 1 -- 172.21.15.16:0/3131983024 shutdown_connections 2024-08-31T06:14:20.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.342+0000 7f9388800640 1 --2- 172.21.15.16:0/3131983024 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f93680b4ca0 0x7f93680b71b0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:20.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.342+0000 7f9388800640 1 --2- 172.21.15.16:0/3131983024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f938010bde0 0x7f93801a1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:20.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.342+0000 7f9388800640 1 --2- 172.21.15.16:0/3131983024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f938010b490 0x7f938019ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:20.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.342+0000 7f9388800640 1 --2- 172.21.15.16:0/3131983024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9380105b50 0x7f938019c4e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:20.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.342+0000 7f9388800640 1 -- 172.21.15.16:0/3131983024 >> 172.21.15.16:0/3131983024 conn(0x7f9380101580 msgr2=0x7f938010ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:20.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.342+0000 7f9388800640 1 -- 172.21.15.16:0/3131983024 shutdown_connections 2024-08-31T06:14:20.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:20.342+0000 7f9388800640 1 -- 172.21.15.16:0/3131983024 wait complete. 2024-08-31T06:14:20.684 INFO:journalctl@ceph.mds.b.smithi155.stdout:Aug 31 06:14:20 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-b[53353]: 2024-08-31T06:14:20.382+0000 7f6c22b03640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000000f [...c,head] /volumes/qa/sv_1/ba5423bc-5c83-4d64-ae1c-ad8850110ce4/postgres/data/pg_subtrans/ auth{1=1,2=1} v3065 ap=3 f(v9 m2024-08-31T06:13:59.178758+0000 2=2+0) n(v21 rc2024-08-31T06:14:08.790611+0000 b327680 3=2+1) old_inodes=6 (inest sync->mix g=1,2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:5472 nref=3)) mcw={1=AsLsXsFs} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x558498a41600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000000f:[<0x10000000006/pg_subtrans v252>,<0x10000000005/data v106>,<0x100000001fb/postgres v19>,<0x100000001fa/ba5423bc-5c83-4d64-ae1c-ad8850110ce4 v35>,<0x10000000001/sv_1 v56>,<0x10000000000/qa v44>,<0x1/volumes v40>]//[]","memoryvalue":"(2)0x1000000000f:[<0x10000000006/pg_subtrans v3065>,<0x10000000005/data v694>,<0x100000001fb/postgres v404>,<0x100000001fa/ba5423bc-5c83-4d64-ae1c-ad8850110ce4 v423>,<0x10000000001/sv_1 v429>,<0x10000000000/qa v426>,<0x1/volumes v388>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-08-31T06:13:25.362274+0000 1=1+0)","ondisk_value.rstat":"n(v0 rc2024-08-31T06:14:08.790611+0000 b327680 3=2+1)","memory_value.dirstat":"f(v9 m2024-08-31T06:13:59.178758+0000 2=2+0)","memory_value.rstat":"n(v21 rc2024-08-31T06:14:08.790611+0000 b327680 3=2+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-08-31T06:14:20.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:20 smithi184 ceph-mon[22331]: pgmap v581: 97 pgs: 97 active+clean; 3.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 3.2 MiB/s wr, 267 op/s 2024-08-31T06:14:20.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:20 smithi184 ceph-mon[22331]: scrub queued for path: / 2024-08-31T06:14:20.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:20 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:14:20.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:20 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:14:20.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:20 smithi184 ceph-mon[22331]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_1/ba5423bc-5c83-4d64-ae1c-ad8850110ce4/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-08-31T06:14:20.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:20 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:14:21.120 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:20 smithi016 ceph-mon[17194]: pgmap v581: 97 pgs: 97 active+clean; 3.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 3.2 MiB/s wr, 267 op/s 2024-08-31T06:14:21.120 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:20 smithi016 ceph-mon[17194]: scrub queued for path: / 2024-08-31T06:14:21.120 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:20 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:14:21.120 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:20 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:14:21.120 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:20 smithi016 ceph-mon[17194]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_1/ba5423bc-5c83-4d64-ae1c-ad8850110ce4/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-08-31T06:14:21.120 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:20 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:14:21.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:20 smithi155 ceph-mon[21742]: pgmap v581: 97 pgs: 97 active+clean; 3.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 3.2 MiB/s wr, 267 op/s 2024-08-31T06:14:21.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:20 smithi155 ceph-mon[21742]: scrub queued for path: / 2024-08-31T06:14:21.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:20 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:14:21.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:20 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:14:21.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:20 smithi155 ceph-mon[21742]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_1/ba5423bc-5c83-4d64-ae1c-ad8850110ce4/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-08-31T06:14:21.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:20 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:14:21.159 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T06:14:21.536 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:14:21.629 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 415.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-31T06:14:21.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:21 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:14:21.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:21 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:14:21.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:21 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:14:21.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:21 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:14:21.998 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:21 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:14:21.998 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:21 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:14:21.998 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:21 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:14:21.998 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:21 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:14:22.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:21 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:14:22.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:21 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:14:22.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:21 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:14:22.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:21 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:14:22.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.178+0000 7f17abe00640 1 -- 172.21.15.16:0/3306811666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17ac078030 msgr2=0x7f17ac113dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:22.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.178+0000 7f17abe00640 1 --2- 172.21.15.16:0/3306811666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17ac078030 0x7f17ac113dd0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f1798009900 tx=0x7f179802a3f0 comp rx=0 tx=0).stop 2024-08-31T06:14:22.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.178+0000 7f17abe00640 1 -- 172.21.15.16:0/3306811666 shutdown_connections 2024-08-31T06:14:22.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.178+0000 7f17abe00640 1 --2- 172.21.15.16:0/3306811666 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17ac114310 0x7f17ac116730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:22.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.178+0000 7f17abe00640 1 --2- 172.21.15.16:0/3306811666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17ac078030 0x7f17ac113dd0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:22.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.178+0000 7f17abe00640 1 --2- 172.21.15.16:0/3306811666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17ac0776e0 0x7f17ac077af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:22.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.178+0000 7f17abe00640 1 -- 172.21.15.16:0/3306811666 >> 172.21.15.16:0/3306811666 conn(0x7f17ac101420 msgr2=0x7f17ac103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:22.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.178+0000 7f17abe00640 1 -- 172.21.15.16:0/3306811666 shutdown_connections 2024-08-31T06:14:22.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.178+0000 7f17abe00640 1 -- 172.21.15.16:0/3306811666 wait complete. 2024-08-31T06:14:22.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.179+0000 7f17abe00640 1 Processor -- start 2024-08-31T06:14:22.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.179+0000 7f17abe00640 1 -- start start 2024-08-31T06:14:22.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.180+0000 7f17abe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17ac0776e0 0x7f17ac1a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:22.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.180+0000 7f17abe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17ac078030 0x7f17ac1a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:22.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.180+0000 7f17abe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17ac114310 0x7f17ac1a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:22.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.180+0000 7f17abe00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac119c70 con 0x7f17ac078030 2024-08-31T06:14:22.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.180+0000 7f17abe00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac119af0 con 0x7f17ac0776e0 2024-08-31T06:14:22.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.180+0000 7f17abe00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac119df0 con 0x7f17ac114310 2024-08-31T06:14:22.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.180+0000 7f17aaa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17ac0776e0 0x7f17ac1a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:22.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.180+0000 7f17aa000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17ac078030 0x7f17ac1a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:22.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.180+0000 7f17aaa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17ac0776e0 0x7f17ac1a0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:33776/0 (socket says 172.21.15.16:33776) 2024-08-31T06:14:22.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.180+0000 7f17aaa00640 1 -- 172.21.15.16:0/604263305 learned_addr learned my addr 172.21.15.16:0/604263305 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:22.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.180+0000 7f17ab400640 1 --2- 172.21.15.16:0/604263305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17ac114310 0x7f17ac1a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:22.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.180+0000 7f17aaa00640 1 -- 172.21.15.16:0/604263305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17ac114310 msgr2=0x7f17ac1a5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:22.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.180+0000 7f17aaa00640 1 --2- 172.21.15.16:0/604263305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17ac114310 0x7f17ac1a5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:22.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.180+0000 7f17aaa00640 1 -- 172.21.15.16:0/604263305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17ac078030 msgr2=0x7f17ac1a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:22.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.180+0000 7f17aaa00640 1 --2- 172.21.15.16:0/604263305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17ac078030 0x7f17ac1a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:22.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.180+0000 7f17aaa00640 1 -- 172.21.15.16:0/604263305 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17ac1a66d0 con 0x7f17ac0776e0 2024-08-31T06:14:22.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.181+0000 7f17aaa00640 1 --2- 172.21.15.16:0/604263305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17ac0776e0 0x7f17ac1a0760 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f179c00e9f0 tx=0x7f179c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:22.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.181+0000 7f17a3400640 1 -- 172.21.15.16:0/604263305 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179c00cd80 con 0x7f17ac0776e0 2024-08-31T06:14:22.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.181+0000 7f17abe00640 1 -- 172.21.15.16:0/604263305 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17ac1a69c0 con 0x7f17ac0776e0 2024-08-31T06:14:22.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.181+0000 7f17a3400640 1 -- 172.21.15.16:0/604263305 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f179c0054d0 con 0x7f17ac0776e0 2024-08-31T06:14:22.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.181+0000 7f17a3400640 1 -- 172.21.15.16:0/604263305 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179c010690 con 0x7f17ac0776e0 2024-08-31T06:14:22.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.181+0000 7f17abe00640 1 -- 172.21.15.16:0/604263305 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17ac1ad120 con 0x7f17ac0776e0 2024-08-31T06:14:22.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.184+0000 7f17b85ce740 1 Processor -- start 2024-08-31T06:14:22.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.184+0000 7f17b85ce740 1 -- start start 2024-08-31T06:14:22.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.185+0000 7f17b85ce740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b1c76a070 0x564b1cb0fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:22.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.185+0000 7f17b85ce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b1c7411b0 0x564b1cb10060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:22.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.185+0000 7f17b85ce740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b1c749390 0x564b1cb105a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:22.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.185+0000 7f17b85ce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564b1ca86770 con 0x564b1c7411b0 2024-08-31T06:14:22.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.185+0000 7f17b85ce740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x564b1ca75d00 con 0x564b1c749390 2024-08-31T06:14:22.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.185+0000 7f17b85ce740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564b1ca79cb0 con 0x564b1c76a070 2024-08-31T06:14:22.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.185+0000 7f17a3400640 1 -- 172.21.15.16:0/604263305 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f179c004e90 con 0x7f17ac0776e0 2024-08-31T06:14:22.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.185+0000 7f17ab400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b1c76a070 0x564b1cb0fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:22.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.185+0000 7f17ab400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b1c76a070 0x564b1cb0fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:44806/0 (socket says 172.21.15.16:44806) 2024-08-31T06:14:22.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.185+0000 7f17ab400640 1 -- 172.21.15.16:0/3611571875 learned_addr learned my addr 172.21.15.16:0/3611571875 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:22.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.185+0000 7f17aa000640 1 --2- 172.21.15.16:0/3611571875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b1c749390 0x564b1cb105a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:22.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.186+0000 7f17aaa00640 1 --2- 172.21.15.16:0/3611571875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b1c7411b0 0x564b1cb10060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:22.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.186+0000 7f17ab400640 1 -- 172.21.15.16:0/3611571875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b1c749390 msgr2=0x564b1cb105a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:22.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.186+0000 7f17ab400640 1 --2- 172.21.15.16:0/3611571875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b1c749390 0x564b1cb105a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:22.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.186+0000 7f17ab400640 1 -- 172.21.15.16:0/3611571875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b1c7411b0 msgr2=0x564b1cb10060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:22.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.186+0000 7f17ab400640 1 --2- 172.21.15.16:0/3611571875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b1c7411b0 0x564b1cb10060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:22.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.186+0000 7f17ab400640 1 -- 172.21.15.16:0/3611571875 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b1c7d5650 con 0x564b1c76a070 2024-08-31T06:14:22.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.186+0000 7f17ab400640 1 --2- 172.21.15.16:0/3611571875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b1c76a070 0x564b1cb0fb20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f17a4009f50 tx=0x7f17a40118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5d5d1925a100817 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:22.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.186+0000 7f1797e00640 1 -- 172.21.15.16:0/3611571875 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a4004d20 con 0x564b1c76a070 2024-08-31T06:14:22.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.187+0000 7f1797e00640 1 -- 172.21.15.16:0/3611571875 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17a4004050 con 0x564b1c76a070 2024-08-31T06:14:22.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.187+0000 7f1797e00640 1 -- 172.21.15.16:0/3611571875 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a4005280 con 0x564b1c76a070 2024-08-31T06:14:22.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.187+0000 7f17a3400640 1 --2- 172.21.15.16:0/604263305 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f17900b4ca0 0x7f17900b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:22.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.187+0000 7f17a3400640 1 -- 172.21.15.16:0/604263305 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7f179c0e5960 con 0x7f17ac0776e0 2024-08-31T06:14:22.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.187+0000 7f17aaa00640 1 --2- 172.21.15.16:0/604263305 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f17900b4ca0 0x7f17900b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:22.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.188+0000 7f17b85ce740 1 -- 172.21.15.16:0/3611571875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b1c76a070 msgr2=0x564b1cb0fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:22.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.188+0000 7f17b85ce740 1 --2- 172.21.15.16:0/3611571875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b1c76a070 0x564b1cb0fb20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f17a4009f50 tx=0x7f17a40118d0 comp rx=0 tx=0).stop 2024-08-31T06:14:22.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.188+0000 7f17b85ce740 1 -- 172.21.15.16:0/3611571875 shutdown_connections 2024-08-31T06:14:22.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.188+0000 7f17b85ce740 1 --2- 172.21.15.16:0/3611571875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b1c749390 0x564b1cb105a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:22.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.188+0000 7f17b85ce740 1 --2- 172.21.15.16:0/3611571875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b1c7411b0 0x564b1cb10060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:22.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.188+0000 7f17b85ce740 1 --2- 172.21.15.16:0/3611571875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b1c76a070 0x564b1cb0fb20 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:22.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.188+0000 7f17b85ce740 1 -- 172.21.15.16:0/3611571875 >> 172.21.15.16:0/3611571875 conn(0x564b1c75e460 msgr2=0x564b1cb0da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:22.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.188+0000 7f17aaa00640 1 --2- 172.21.15.16:0/604263305 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f17900b4ca0 0x7f17900b71b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f179c0e3190 tx=0x7f179c004a90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:22.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.188+0000 7f17b85ce740 1 -- 172.21.15.16:0/3611571875 shutdown_connections 2024-08-31T06:14:22.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.188+0000 7f17b85ce740 1 -- 172.21.15.16:0/3611571875 wait complete. 2024-08-31T06:14:22.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.188+0000 7f17b85ce740 1 -- start start 2024-08-31T06:14:22.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.189+0000 7f17b85ce740 1 Processor -- start 2024-08-31T06:14:22.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.189+0000 7f17b85ce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b1c7411b0 0x564b1cb1b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:22.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.189+0000 7f17b85ce740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b1c749390 0x564b1cb1bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:22.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.189+0000 7f17b85ce740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b1c76a070 0x564b1cb275f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:22.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.189+0000 7f17b85ce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564b1c8d8700 con 0x564b1c7411b0 2024-08-31T06:14:22.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.189+0000 7f17b85ce740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x564b1c9fccb0 con 0x564b1c76a070 2024-08-31T06:14:22.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.189+0000 7f17b85ce740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564b1cb1e300 con 0x564b1c749390 2024-08-31T06:14:22.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.189+0000 7f17b85ce740 10 client.0 _ll_register_callbacks cb 0x564b1ca22430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:14:22.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.190+0000 7f17ab400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b1c7411b0 0x564b1cb1b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:22.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.190+0000 7f17ab400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b1c7411b0 0x564b1cb1b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56518/0 (socket says 172.21.15.16:56518) 2024-08-31T06:14:22.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.190+0000 7f17ab400640 1 -- 172.21.15.16:0/2089671890 learned_addr learned my addr 172.21.15.16:0/2089671890 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:22.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.190+0000 7f17aa000640 1 --2- 172.21.15.16:0/2089671890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b1c749390 0x564b1cb1bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:22.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.190+0000 7f17ab400640 1 --2- 172.21.15.16:0/2089671890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b1c76a070 0x564b1cb275f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:22.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.190+0000 7f17aa000640 1 -- 172.21.15.16:0/2089671890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b1c76a070 msgr2=0x564b1cb275f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:22.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.190+0000 7f17aa000640 1 --2- 172.21.15.16:0/2089671890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b1c76a070 0x564b1cb275f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:22.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.190+0000 7f17aa000640 1 -- 172.21.15.16:0/2089671890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b1c7411b0 msgr2=0x564b1cb1b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:22.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.190+0000 7f17aa000640 1 --2- 172.21.15.16:0/2089671890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b1c7411b0 0x564b1cb1b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:22.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.190+0000 7f17aa000640 1 -- 172.21.15.16:0/2089671890 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b1ca3d870 con 0x564b1c749390 2024-08-31T06:14:22.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.190+0000 7f17ab400640 1 --2- 172.21.15.16:0/2089671890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b1c76a070 0x564b1cb275f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:14:22.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.190+0000 7f17aa000640 1 --2- 172.21.15.16:0/2089671890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b1c749390 0x564b1cb1bc10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f179802a910 tx=0x7f1798032e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9437296212217fe7 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:22.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.190+0000 7f1794c00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:14:22.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.191+0000 7f1794c00640 1 -- 172.21.15.16:0/2089671890 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1798035040 con 0x564b1c749390 2024-08-31T06:14:22.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.191+0000 7f17b85ce740 1 -- 172.21.15.16:0/2089671890 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17a4004050 con 0x564b1c749390 2024-08-31T06:14:22.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.191+0000 7f1794c00640 1 -- 172.21.15.16:0/2089671890 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7f1798004050 con 0x564b1c749390 2024-08-31T06:14:22.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.191+0000 7f1794c00640 1 -- 172.21.15.16:0/2089671890 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1798004250 con 0x564b1c749390 2024-08-31T06:14:22.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.191+0000 7f17b85ce740 10 client.27181 fetch_fsmap learned FSMap version 39 2024-08-31T06:14:22.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.191+0000 7f17b85ce740 1 -- 172.21.15.16:0/2089671890 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564b1cb2a360 con 0x564b1c749390 2024-08-31T06:14:22.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.191+0000 7f1794c00640 1 -- 172.21.15.16:0/2089671890 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17a4004050 con 0x564b1c749390 2024-08-31T06:14:22.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.192+0000 7f1794c00640 1 -- 172.21.15.16:0/2089671890 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f1798004850 con 0x564b1c749390 2024-08-31T06:14:22.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.192+0000 7f17b85ce740 10 client.27181 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T06:14:22.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.192+0000 7f17b85ce740 10 client.27181 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:14:22.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.192+0000 7f17b85ce740 20 client.27181 populate_metadata read hostname 'smithi016' 2024-08-31T06:14:22.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.192+0000 7f17b85ce740 1 --2- 172.21.15.16:0/2089671890 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564b1c7289f0 0x564b1cb2d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:22.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.192+0000 7f17b85ce740 4 client.27181 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:14:22.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.192+0000 7f17b85ce740 1 -- 172.21.15.16:0/2089671890 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b1ca886c0 con 0x564b1c7289f0 2024-08-31T06:14:22.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.192+0000 7f17ab400640 1 --2- 172.21.15.16:0/2089671890 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564b1c7289f0 0x564b1cb2d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:22.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.193+0000 7f17ab400640 1 --2- 172.21.15.16:0/2089671890 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564b1c7289f0 0x564b1cb2d850 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9166d60e27456abc server_cookie=6dafeaa639ee7ebd in_seq=0 out_seq=0 2024-08-31T06:14:22.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.193+0000 7f1794c00640 10 client.27181 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:22.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.194+0000 7f1794c00640 1 -- 172.21.15.16:0/2089671890 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564b1ca886c0 con 0x564b1c7289f0 2024-08-31T06:14:22.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.194+0000 7f1794c00640 10 client.27181 handle_command_reply: tid=0 2024-08-31T06:14:22.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.194+0000 7f17b85ce740 1 client.27181 shutdown 2024-08-31T06:14:22.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.194+0000 7f17b85ce740 1 -- 172.21.15.16:0/2089671890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b1c749390 msgr2=0x564b1cb1bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:22.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.194+0000 7f17b85ce740 1 --2- 172.21.15.16:0/2089671890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b1c749390 0x564b1cb1bc10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f179802a910 tx=0x7f1798032e90 comp rx=0 tx=0).stop 2024-08-31T06:14:22.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.194+0000 7f17b85ce740 1 -- 172.21.15.16:0/2089671890 shutdown_connections 2024-08-31T06:14:22.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.194+0000 7f17b85ce740 1 --2- 172.21.15.16:0/2089671890 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564b1c7289f0 0x564b1cb2d850 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:22.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.194+0000 7f17b85ce740 1 --2- 172.21.15.16:0/2089671890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b1c749390 0x564b1cb1bc10 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:22.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.194+0000 7f17b85ce740 1 --2- 172.21.15.16:0/2089671890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b1c76a070 0x564b1cb275f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:22.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.194+0000 7f17b85ce740 1 --2- 172.21.15.16:0/2089671890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b1c7411b0 0x564b1cb1b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:22.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.194+0000 7f17b85ce740 1 -- 172.21.15.16:0/2089671890 >> 172.21.15.16:0/2089671890 conn(0x564b1c75e460 msgr2=0x564b1cb12d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:22.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.194+0000 7f1794c00640 0 client.27181 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:22.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.194+0000 7f17b85ce740 1 -- 172.21.15.16:0/2089671890 shutdown_connections 2024-08-31T06:14:22.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.194+0000 7f17b85ce740 1 -- 172.21.15.16:0/2089671890 wait complete. 2024-08-31T06:14:22.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.195+0000 7f17b85ce740 20 client.27181 trim_cache size 0 max 16384 2024-08-31T06:14:22.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.213+0000 7f17b85ce740 1 Processor -- start 2024-08-31T06:14:22.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.213+0000 7f17b85ce740 1 -- start start 2024-08-31T06:14:22.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.214+0000 7f17b85ce740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b1c7411b0 0x564b1cb0d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:22.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.214+0000 7f17b85ce740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b1c749390 0x564b1cb274f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:22.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.214+0000 7f17b85ce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b1c76a070 0x564b1cb27a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:22.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.214+0000 7f17b85ce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac1b8200 con 0x564b1c76a070 2024-08-31T06:14:22.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.214+0000 7f17b85ce740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac1b5a70 con 0x564b1c7411b0 2024-08-31T06:14:22.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.214+0000 7f17b85ce740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac1b38e0 con 0x564b1c749390 2024-08-31T06:14:22.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.214+0000 7f17ab400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b1c7411b0 0x564b1cb0d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:22.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.214+0000 7f17aa000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b1c749390 0x564b1cb274f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:22.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.214+0000 7f17ab400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b1c7411b0 0x564b1cb0d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:33810/0 (socket says 172.21.15.16:33810) 2024-08-31T06:14:22.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.214+0000 7f17ab400640 1 -- 172.21.15.16:0/3132186652 learned_addr learned my addr 172.21.15.16:0/3132186652 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:22.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.214+0000 7f17ab400640 1 --2- 172.21.15.16:0/3132186652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b1c76a070 0x564b1cb27a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:22.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.215+0000 7f17ab400640 1 -- 172.21.15.16:0/3132186652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b1c749390 msgr2=0x564b1cb274f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:22.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.215+0000 7f17ab400640 1 --2- 172.21.15.16:0/3132186652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b1c749390 0x564b1cb274f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:22.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.215+0000 7f17ab400640 1 -- 172.21.15.16:0/3132186652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b1c76a070 msgr2=0x564b1cb27a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:22.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.215+0000 7f17ab400640 1 --2- 172.21.15.16:0/3132186652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b1c76a070 0x564b1cb27a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:22.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.215+0000 7f17ab400640 1 -- 172.21.15.16:0/3132186652 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b1cb16d60 con 0x564b1c7411b0 2024-08-31T06:14:22.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.215+0000 7f17aa000640 1 --2- 172.21.15.16:0/3132186652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b1c749390 0x564b1cb274f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:14:22.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.215+0000 7f17ab400640 1 --2- 172.21.15.16:0/3132186652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b1c7411b0 0x564b1cb0d400 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f17a4009f50 tx=0x7f17a4005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a59913ce3364da99 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:22.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.215+0000 7f1797400640 1 -- 172.21.15.16:0/3132186652 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a4003cb0 con 0x564b1c7411b0 2024-08-31T06:14:22.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.216+0000 7f1797400640 1 -- 172.21.15.16:0/3132186652 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17a4004c20 con 0x564b1c7411b0 2024-08-31T06:14:22.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.216+0000 7f1797400640 1 -- 172.21.15.16:0/3132186652 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a40056d0 con 0x564b1c7411b0 2024-08-31T06:14:22.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.216+0000 7f17b85ce740 1 -- 172.21.15.16:0/3132186652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b1c7411b0 msgr2=0x564b1cb0d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:22.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.216+0000 7f17b85ce740 1 --2- 172.21.15.16:0/3132186652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b1c7411b0 0x564b1cb0d400 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f17a4009f50 tx=0x7f17a4005bd0 comp rx=0 tx=0).stop 2024-08-31T06:14:22.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.216+0000 7f17b85ce740 1 -- 172.21.15.16:0/3132186652 shutdown_connections 2024-08-31T06:14:22.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.216+0000 7f17b85ce740 1 --2- 172.21.15.16:0/3132186652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b1c749390 0x564b1cb274f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:22.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.216+0000 7f17b85ce740 1 --2- 172.21.15.16:0/3132186652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b1c76a070 0x564b1cb27a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:22.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.216+0000 7f17b85ce740 1 --2- 172.21.15.16:0/3132186652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b1c7411b0 0x564b1cb0d400 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:22.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.216+0000 7f17b85ce740 1 -- 172.21.15.16:0/3132186652 >> 172.21.15.16:0/3132186652 conn(0x564b1c75e460 msgr2=0x564b1cb19d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:22.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.216+0000 7f17b85ce740 1 -- 172.21.15.16:0/3132186652 shutdown_connections 2024-08-31T06:14:22.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.217+0000 7f17b85ce740 1 -- 172.21.15.16:0/3132186652 wait complete. 2024-08-31T06:14:22.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.217+0000 7f17b85ce740 1 -- start start 2024-08-31T06:14:22.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.217+0000 7f17b85ce740 1 Processor -- start 2024-08-31T06:14:22.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.218+0000 7f17b85ce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b1c7411b0 0x564b1cb1fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:22.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.218+0000 7f17b85ce740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b1c749390 0x564b1cb20420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:22.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.218+0000 7f17b85ce740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b1c76a070 0x564b1cb91290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:22.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.218+0000 7f17b85ce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac1b3f30 con 0x564b1c7411b0 2024-08-31T06:14:22.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.218+0000 7f17b85ce740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac1b53a0 con 0x564b1c76a070 2024-08-31T06:14:22.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.218+0000 7f17b85ce740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac1b4600 con 0x564b1c749390 2024-08-31T06:14:22.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.218+0000 7f17b85ce740 10 client.0 _ll_register_callbacks cb 0x7f1798032b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:14:22.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.218+0000 7f17ab400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b1c7411b0 0x564b1cb1fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:22.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.218+0000 7f17ab400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b1c7411b0 0x564b1cb1fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56530/0 (socket says 172.21.15.16:56530) 2024-08-31T06:14:22.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.218+0000 7f17ab400640 1 -- 172.21.15.16:0/150764434 learned_addr learned my addr 172.21.15.16:0/150764434 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:22.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.218+0000 7f17aa000640 1 --2- 172.21.15.16:0/150764434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b1c749390 0x564b1cb20420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:22.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.218+0000 7f17ab400640 1 --2- 172.21.15.16:0/150764434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b1c76a070 0x564b1cb91290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:22.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.219+0000 7f17ab400640 1 -- 172.21.15.16:0/150764434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b1c749390 msgr2=0x564b1cb20420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:22.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.219+0000 7f17ab400640 1 --2- 172.21.15.16:0/150764434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b1c749390 0x564b1cb20420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:22.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.219+0000 7f17ab400640 1 -- 172.21.15.16:0/150764434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b1c76a070 msgr2=0x564b1cb91290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:22.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.219+0000 7f17ab400640 1 --2- 172.21.15.16:0/150764434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b1c76a070 0x564b1cb91290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:22.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.219+0000 7f17ab400640 1 -- 172.21.15.16:0/150764434 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b1cb91900 con 0x564b1c7411b0 2024-08-31T06:14:22.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.219+0000 7f17aa000640 1 --2- 172.21.15.16:0/150764434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b1c749390 0x564b1cb20420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:14:22.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.219+0000 7f17ab400640 1 --2- 172.21.15.16:0/150764434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b1c7411b0 0x564b1cb1fe60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f17a4005cc0 tx=0x7f17a4005370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe873500c338ac82 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:22.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.219+0000 7f1794c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:14:22.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.219+0000 7f1794c00640 1 -- 172.21.15.16:0/150764434 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a4004fd0 con 0x564b1c7411b0 2024-08-31T06:14:22.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.219+0000 7f17b85ce740 1 -- 172.21.15.16:0/150764434 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17a4004c20 con 0x564b1c7411b0 2024-08-31T06:14:22.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.219+0000 7f1794c00640 1 -- 172.21.15.16:0/150764434 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7f17a4003a30 con 0x564b1c7411b0 2024-08-31T06:14:22.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.220+0000 7f1794c00640 1 -- 172.21.15.16:0/150764434 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f17a4003c30 con 0x564b1c7411b0 2024-08-31T06:14:22.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.220+0000 7f17b85ce740 10 client.16974 fetch_fsmap learned FSMap version 39 2024-08-31T06:14:22.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.220+0000 7f17b85ce740 1 -- 172.21.15.16:0/150764434 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564b1cb98c10 con 0x564b1c7411b0 2024-08-31T06:14:22.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.220+0000 7f1794c00640 1 -- 172.21.15.16:0/150764434 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17a4004c20 con 0x564b1c7411b0 2024-08-31T06:14:22.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.220+0000 7f1794c00640 1 -- 172.21.15.16:0/150764434 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f17a40276d0 con 0x564b1c7411b0 2024-08-31T06:14:22.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.220+0000 7f17b85ce740 10 client.16974 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T06:14:22.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.220+0000 7f17b85ce740 10 client.16974 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:14:22.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.220+0000 7f17b85ce740 20 client.16974 populate_metadata read hostname 'smithi016' 2024-08-31T06:14:22.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.220+0000 7f17b85ce740 1 --2- 172.21.15.16:0/150764434 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564b1c6fdda0 0x564b1cb9b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:22.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.220+0000 7f17b85ce740 4 client.16974 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T06:14:22.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.220+0000 7f17b85ce740 1 -- 172.21.15.16:0/150764434 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564b1ca84160 con 0x564b1c6fdda0 2024-08-31T06:14:22.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.221+0000 7f17aa000640 1 --2- 172.21.15.16:0/150764434 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564b1c6fdda0 0x564b1cb9b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:22.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.221+0000 7f17aa000640 1 --2- 172.21.15.16:0/150764434 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564b1c6fdda0 0x564b1cb9b050 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34109f528e55fe0d server_cookie=ad1bf46dd82bb08e in_seq=0 out_seq=0 2024-08-31T06:14:22.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.221+0000 7f1794c00640 10 client.16974 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:22.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.221+0000 7f1794c00640 1 -- 172.21.15.16:0/150764434 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3126 (crc 0 0 0) 0x564b1ca84160 con 0x564b1c6fdda0 2024-08-31T06:14:22.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.221+0000 7f1794c00640 10 client.16974 handle_command_reply: tid=0 2024-08-31T06:14:22.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.222+0000 7f17b85ce740 1 client.16974 shutdown 2024-08-31T06:14:22.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.222+0000 7f17b85ce740 1 -- 172.21.15.16:0/150764434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b1c7411b0 msgr2=0x564b1cb1fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:22.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.222+0000 7f17b85ce740 1 --2- 172.21.15.16:0/150764434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b1c7411b0 0x564b1cb1fe60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f17a4005cc0 tx=0x7f17a4005370 comp rx=0 tx=0).stop 2024-08-31T06:14:22.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.222+0000 7f17b85ce740 1 -- 172.21.15.16:0/150764434 shutdown_connections 2024-08-31T06:14:22.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.222+0000 7f17b85ce740 1 --2- 172.21.15.16:0/150764434 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564b1c6fdda0 0x564b1cb9b050 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:22.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.222+0000 7f17b85ce740 1 --2- 172.21.15.16:0/150764434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b1c749390 0x564b1cb20420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:22.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.222+0000 7f17b85ce740 1 --2- 172.21.15.16:0/150764434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b1c76a070 0x564b1cb91290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:22.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.222+0000 7f17b85ce740 1 --2- 172.21.15.16:0/150764434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b1c7411b0 0x564b1cb1fe60 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:22.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.222+0000 7f1794c00640 0 client.16974 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:22.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.222+0000 7f17b85ce740 1 -- 172.21.15.16:0/150764434 >> 172.21.15.16:0/150764434 conn(0x564b1c75e460 msgr2=0x564b1cb14c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:22.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.222+0000 7f17b85ce740 1 -- 172.21.15.16:0/150764434 shutdown_connections 2024-08-31T06:14:22.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.222+0000 7f17b85ce740 1 -- 172.21.15.16:0/150764434 wait complete. 2024-08-31T06:14:22.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.222+0000 7f17b85ce740 20 client.16974 trim_cache size 0 max 16384 2024-08-31T06:14:22.229 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:14:22.229 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T06:14:22.229 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-08-31T06:14:22.229 INFO:teuthology.orchestra.run.smithi016.stdout: "a64d5301-8558-4e9c-ac9b-8d0ac7057e74": { 2024-08-31T06:14:22.229 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:14:22.229 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "a64d5301-8558-4e9c-ac9b-8d0ac7057e74", 2024-08-31T06:14:22.229 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-08-31T06:14:22.229 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:14:22.229 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:14:22.230 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-08-31T06:14:22.230 INFO:teuthology.orchestra.run.smithi016.stdout: "a64d5301-8558-4e9c-ac9b-8d0ac7057e74": { 2024-08-31T06:14:22.230 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:14:20.338104+0000", 2024-08-31T06:14:22.230 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:14:22.230 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:14:22.230 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:14:22.230 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:14:22.230 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:14:22.230 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:14:22.230 INFO:teuthology.orchestra.run.smithi016.stdout: "e739a16e-8ace-43a8-a1ce-ac27d39ef8a3": { 2024-08-31T06:14:22.230 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:54.084646+0000", 2024-08-31T06:14:22.230 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:14:22.230 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:14:22.230 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:14:22.231 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:14:22.231 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:14:22.231 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:14:22.231 INFO:teuthology.orchestra.run.smithi016.stdout: "91b72a34-4ba9-4126-a414-8b862f8c5f78": { 2024-08-31T06:14:22.231 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:13:28.663237+0000", 2024-08-31T06:14:22.231 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:14:22.231 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:14:22.231 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:14:22.231 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:14:22.231 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:14:22.232 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:14:22.232 INFO:teuthology.orchestra.run.smithi016.stdout: "a10cce76-99ab-46b5-b3b2-07b82899b8f9": { 2024-08-31T06:14:22.232 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:11:43.967546+0000", 2024-08-31T06:14:22.232 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:14:22.232 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:14:22.232 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:14:22.232 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:14:22.232 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:14:22.232 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:14:22.233 INFO:teuthology.orchestra.run.smithi016.stdout: "c1fd6904-2e02-47fa-939a-47349b9fe36b": { 2024-08-31T06:14:22.233 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:06.678691+0000", 2024-08-31T06:14:22.233 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:14:22.233 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:14:22.233 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:14:22.233 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:14:22.233 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:14:22.233 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:14:22.233 INFO:teuthology.orchestra.run.smithi016.stdout: "78d6ef02-8551-45e8-ad60-9627eb573fc1": { 2024-08-31T06:14:22.233 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:09:13.547865+0000", 2024-08-31T06:14:22.234 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:14:22.234 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:14:22.234 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:14:22.234 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:14:22.234 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:14:22.234 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:14:22.234 INFO:teuthology.orchestra.run.smithi016.stdout: "c8455d74-2fe7-4d93-bcf2-19b419d16016": { 2024-08-31T06:14:22.234 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:08:24.035960+0000", 2024-08-31T06:14:22.234 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:14:22.234 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:14:22.234 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:14:22.234 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:14:22.234 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:14:22.234 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:14:22.235 INFO:teuthology.orchestra.run.smithi016.stdout: "88d7bdb8-cb1c-45e5-ad29-4cd8da6c6003": { 2024-08-31T06:14:22.235 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:07:39.648942+0000", 2024-08-31T06:14:22.235 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:14:22.235 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:14:22.235 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:14:22.235 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:14:22.235 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:14:22.235 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:14:22.235 INFO:teuthology.orchestra.run.smithi016.stdout: "6a03046e-cdf4-4a14-b9bd-b560b8968467": { 2024-08-31T06:14:22.235 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:12:38.002354+0000", 2024-08-31T06:14:22.235 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:14:22.235 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:14:22.236 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:14:22.236 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:14:22.236 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:14:22.236 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:14:22.236 INFO:teuthology.orchestra.run.smithi016.stdout: "c300b7e8-f3fd-482c-a909-9329bc61e570": { 2024-08-31T06:14:22.236 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:06:49.868046+0000", 2024-08-31T06:14:22.236 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:14:22.236 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:14:22.236 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:14:22.236 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:14:22.236 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:14:22.236 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:14:22.236 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:14:22.237 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:14:22.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.223+0000 7f17abe00640 1 -- 172.21.15.16:0/604263305 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f17900b4ca0 msgr2=0x7f17900b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:22.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.223+0000 7f17abe00640 1 --2- 172.21.15.16:0/604263305 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f17900b4ca0 0x7f17900b71b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f179c0e3190 tx=0x7f179c004a90 comp rx=0 tx=0).stop 2024-08-31T06:14:22.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.223+0000 7f17abe00640 1 -- 172.21.15.16:0/604263305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17ac0776e0 msgr2=0x7f17ac1a0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:22.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.223+0000 7f17abe00640 1 --2- 172.21.15.16:0/604263305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17ac0776e0 0x7f17ac1a0760 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f179c00e9f0 tx=0x7f179c00edf0 comp rx=0 tx=0).stop 2024-08-31T06:14:22.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.223+0000 7f17abe00640 1 -- 172.21.15.16:0/604263305 shutdown_connections 2024-08-31T06:14:22.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.223+0000 7f17abe00640 1 --2- 172.21.15.16:0/604263305 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f17900b4ca0 0x7f17900b71b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:22.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.223+0000 7f17abe00640 1 --2- 172.21.15.16:0/604263305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17ac114310 0x7f17ac1a5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:22.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.223+0000 7f17abe00640 1 --2- 172.21.15.16:0/604263305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17ac078030 0x7f17ac1a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:22.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.223+0000 7f17abe00640 1 --2- 172.21.15.16:0/604263305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17ac0776e0 0x7f17ac1a0760 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:22.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.223+0000 7f17abe00640 1 -- 172.21.15.16:0/604263305 >> 172.21.15.16:0/604263305 conn(0x7f17ac101420 msgr2=0x7f17ac116420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:22.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.223+0000 7f17abe00640 1 -- 172.21.15.16:0/604263305 shutdown_connections 2024-08-31T06:14:22.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:22.223+0000 7f17abe00640 1 -- 172.21.15.16:0/604263305 wait complete. 2024-08-31T06:14:22.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:22 smithi184 ceph-mon[22331]: pgmap v582: 97 pgs: 97 active+clean; 3.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 2.6 MiB/s wr, 189 op/s 2024-08-31T06:14:23.084 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:22 smithi016 ceph-mon[17194]: pgmap v582: 97 pgs: 97 active+clean; 3.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 2.6 MiB/s wr, 189 op/s 2024-08-31T06:14:23.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:22 smithi155 ceph-mon[21742]: pgmap v582: 97 pgs: 97 active+clean; 3.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 2.6 MiB/s wr, 189 op/s 2024-08-31T06:14:23.162 INFO:tasks.cephfs.filesystem:scrub status for tag:a64d5301-8558-4e9c-ac9b-8d0ac7057e74 - {'path': '/', 'tag': 'a64d5301-8558-4e9c-ac9b-8d0ac7057e74', 'options': 'recursive,force'} 2024-08-31T06:14:24.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:24 smithi184 ceph-mon[22331]: pgmap v583: 97 pgs: 97 active+clean; 3.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 1.5 MiB/s wr, 155 op/s 2024-08-31T06:14:25.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:24 smithi155 ceph-mon[21742]: pgmap v583: 97 pgs: 97 active+clean; 3.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 1.5 MiB/s wr, 155 op/s 2024-08-31T06:14:25.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:24 smithi016 ceph-mon[17194]: pgmap v583: 97 pgs: 97 active+clean; 3.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 1.5 MiB/s wr, 155 op/s 2024-08-31T06:14:26.629 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 420.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-31T06:14:26.830 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:26 smithi016 ceph-mon[17194]: pgmap v584: 97 pgs: 97 active+clean; 3.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 5.2 MiB/s wr, 649 op/s 2024-08-31T06:14:26.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:26 smithi184 ceph-mon[22331]: pgmap v584: 97 pgs: 97 active+clean; 3.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 5.2 MiB/s wr, 649 op/s 2024-08-31T06:14:27.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:26 smithi155 ceph-mon[21742]: pgmap v584: 97 pgs: 97 active+clean; 3.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 5.2 MiB/s wr, 649 op/s 2024-08-31T06:14:28.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:28 smithi184 ceph-mon[22331]: pgmap v585: 97 pgs: 97 active+clean; 3.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 5.3 MiB/s wr, 652 op/s 2024-08-31T06:14:29.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:28 smithi155 ceph-mon[21742]: pgmap v585: 97 pgs: 97 active+clean; 3.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 5.3 MiB/s wr, 652 op/s 2024-08-31T06:14:29.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:28 smithi016 ceph-mon[17194]: pgmap v585: 97 pgs: 97 active+clean; 3.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 5.3 MiB/s wr, 652 op/s 2024-08-31T06:14:29.785 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:14:30.179 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:14:30.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:30.829+0000 7f16bea00640 1 -- 172.21.15.16:0/4178825864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b8103900 msgr2=0x7f16b8103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:30.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:30.829+0000 7f16bea00640 1 --2- 172.21.15.16:0/4178825864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b8103900 0x7f16b8103d10 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f16a8009ca0 tx=0x7f16a802a8c0 comp rx=0 tx=0).stop 2024-08-31T06:14:30.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:30.829+0000 7f16bea00640 1 -- 172.21.15.16:0/4178825864 shutdown_connections 2024-08-31T06:14:30.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:30.829+0000 7f16bea00640 1 --2- 172.21.15.16:0/4178825864 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16b8109240 0x7f16b8109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:30.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:30.829+0000 7f16bea00640 1 --2- 172.21.15.16:0/4178825864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16b8109b90 0x7f16b81103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:30.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:30.829+0000 7f16bea00640 1 --2- 172.21.15.16:0/4178825864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b8103900 0x7f16b8103d10 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:30.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:30.829+0000 7f16bea00640 1 -- 172.21.15.16:0/4178825864 >> 172.21.15.16:0/4178825864 conn(0x7f16b807b5f0 msgr2=0x7f16b807ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:30.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:30.829+0000 7f16bea00640 1 -- 172.21.15.16:0/4178825864 shutdown_connections 2024-08-31T06:14:30.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:30.829+0000 7f16bea00640 1 -- 172.21.15.16:0/4178825864 wait complete. 2024-08-31T06:14:30.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:30.831+0000 7f16bea00640 1 Processor -- start 2024-08-31T06:14:30.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:30.831+0000 7f16bea00640 1 -- start start 2024-08-31T06:14:30.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:30.831+0000 7f16bea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16b8103900 0x7f16b819a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:30.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:30.831+0000 7f16bea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b8109240 0x7f16b819a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:30.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:30.832+0000 7f16bea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16b8109b90 0x7f16b819fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:30.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:30.832+0000 7f16bea00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f16b8113850 con 0x7f16b8109240 2024-08-31T06:14:30.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:30.832+0000 7f16bea00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f16b81136d0 con 0x7f16b8103900 2024-08-31T06:14:30.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:30.832+0000 7f16bea00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f16b81139d0 con 0x7f16b8109b90 2024-08-31T06:14:30.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:30.832+0000 7f16af400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b8109240 0x7f16b819a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:30.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:30.832+0000 7f16b7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16b8103900 0x7f16b819a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:30.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:30.832+0000 7f16b7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16b8103900 0x7f16b819a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:41702/0 (socket says 172.21.15.16:41702) 2024-08-31T06:14:30.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:30.832+0000 7f16b7e00640 1 -- 172.21.15.16:0/2357598035 learned_addr learned my addr 172.21.15.16:0/2357598035 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:30.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:30.832+0000 7f16bca00640 1 --2- 172.21.15.16:0/2357598035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16b8109b90 0x7f16b819fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:30.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:30.832+0000 7f16af400640 1 -- 172.21.15.16:0/2357598035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16b8109b90 msgr2=0x7f16b819fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:30.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:30.832+0000 7f16af400640 1 --2- 172.21.15.16:0/2357598035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16b8109b90 0x7f16b819fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:30.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:30.832+0000 7f16af400640 1 -- 172.21.15.16:0/2357598035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16b8103900 msgr2=0x7f16b819a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:30.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:30.832+0000 7f16af400640 1 --2- 172.21.15.16:0/2357598035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16b8103900 0x7f16b819a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:30.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:30.832+0000 7f16af400640 1 -- 172.21.15.16:0/2357598035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16b81a0170 con 0x7f16b8109240 2024-08-31T06:14:30.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:30.832+0000 7f16bca00640 1 --2- 172.21.15.16:0/2357598035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16b8109b90 0x7f16b819fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:14:30.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:30.833+0000 7f16b7e00640 1 --2- 172.21.15.16:0/2357598035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16b8103900 0x7f16b819a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:14:30.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:30.833+0000 7f16af400640 1 --2- 172.21.15.16:0/2357598035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b8109240 0x7f16b819a7a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f16b806c790 tx=0x7f16a000ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:30.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:30.833+0000 7f16b5600640 1 -- 172.21.15.16:0/2357598035 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a0009e60 con 0x7f16b8109240 2024-08-31T06:14:30.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:30.833+0000 7f16b5600640 1 -- 172.21.15.16:0/2357598035 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16a000ccc0 con 0x7f16b8109240 2024-08-31T06:14:30.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:30.833+0000 7f16b5600640 1 -- 172.21.15.16:0/2357598035 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a00187a0 con 0x7f16b8109240 2024-08-31T06:14:30.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:30.833+0000 7f16bea00640 1 -- 172.21.15.16:0/2357598035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16b81a0460 con 0x7f16b8109240 2024-08-31T06:14:30.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:30.833+0000 7f16bea00640 1 -- 172.21.15.16:0/2357598035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16b81a6bc0 con 0x7f16b8109240 2024-08-31T06:14:30.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:30.835+0000 7f16bea00640 1 -- 172.21.15.16:0/2357598035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16b8103d10 con 0x7f16b8109240 2024-08-31T06:14:30.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:30.837+0000 7f16b5600640 1 -- 172.21.15.16:0/2357598035 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f16a0010430 con 0x7f16b8109240 2024-08-31T06:14:30.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:30.838+0000 7f16b5600640 1 --2- 172.21.15.16:0/2357598035 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f169c0b4ca0 0x7f169c0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:30.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:30.839+0000 7f16b5600640 1 -- 172.21.15.16:0/2357598035 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7f16a00e5c30 con 0x7f16b8109240 2024-08-31T06:14:30.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:30.839+0000 7f16b7e00640 1 --2- 172.21.15.16:0/2357598035 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f169c0b4ca0 0x7f169c0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:30.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:30.840+0000 7f16b7e00640 1 --2- 172.21.15.16:0/2357598035 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f169c0b4ca0 0x7f169c0b71b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f16a8033040 tx=0x7f16a8032eb0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:30.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:30.841+0000 7f16b5600640 1 -- 172.21.15.16:0/2357598035 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f16a00ac9f0 con 0x7f16b8109240 2024-08-31T06:14:30.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:30 smithi016 ceph-mon[17194]: pgmap v586: 97 pgs: 97 active+clean; 3.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 5.0 MiB/s wr, 640 op/s 2024-08-31T06:14:30.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:30 smithi184 ceph-mon[22331]: pgmap v586: 97 pgs: 97 active+clean; 3.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 5.0 MiB/s wr, 640 op/s 2024-08-31T06:14:31.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:31.082+0000 7f16bea00640 1 -- 172.21.15.16:0/2357598035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16b819b330 con 0x7f16b8109240 2024-08-31T06:14:31.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:31.083+0000 7f16b5600640 1 -- 172.21.15.16:0/2357598035 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11920 (secure 0 0 0) 0x7f16b819b330 con 0x7f16b8109240 2024-08-31T06:14:31.088 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:14:31.089 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":39,"btime":"2024-08-31T06:14:07:075654+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:14:07.025476+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24737,24724,24712]},"id":1}]} 2024-08-31T06:14:31.089 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 39 2024-08-31T06:14:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:31.086+0000 7f16bea00640 1 -- 172.21.15.16:0/2357598035 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f169c0b4ca0 msgr2=0x7f169c0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:31.086+0000 7f16bea00640 1 --2- 172.21.15.16:0/2357598035 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f169c0b4ca0 0x7f169c0b71b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f16a8033040 tx=0x7f16a8032eb0 comp rx=0 tx=0).stop 2024-08-31T06:14:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:31.086+0000 7f16bea00640 1 -- 172.21.15.16:0/2357598035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b8109240 msgr2=0x7f16b819a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:31.086+0000 7f16bea00640 1 --2- 172.21.15.16:0/2357598035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b8109240 0x7f16b819a7a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f16b806c790 tx=0x7f16a000ed90 comp rx=0 tx=0).stop 2024-08-31T06:14:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:31.086+0000 7f16bea00640 1 -- 172.21.15.16:0/2357598035 shutdown_connections 2024-08-31T06:14:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:31.086+0000 7f16bea00640 1 --2- 172.21.15.16:0/2357598035 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f169c0b4ca0 0x7f169c0b71b0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:31.086+0000 7f16bea00640 1 --2- 172.21.15.16:0/2357598035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16b8109b90 0x7f16b819fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:31.086+0000 7f16bea00640 1 --2- 172.21.15.16:0/2357598035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b8109240 0x7f16b819a7a0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:31.086+0000 7f16bea00640 1 --2- 172.21.15.16:0/2357598035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16b8103900 0x7f16b819a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:31.086+0000 7f16bea00640 1 -- 172.21.15.16:0/2357598035 >> 172.21.15.16:0/2357598035 conn(0x7f16b807b5f0 msgr2=0x7f16b81072d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:31.086+0000 7f16bea00640 1 -- 172.21.15.16:0/2357598035 shutdown_connections 2024-08-31T06:14:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:31.086+0000 7f16bea00640 1 -- 172.21.15.16:0/2357598035 wait complete. 2024-08-31T06:14:31.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:30 smithi155 ceph-mon[21742]: pgmap v586: 97 pgs: 97 active+clean; 3.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 5.0 MiB/s wr, 640 op/s 2024-08-31T06:14:31.629 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 425.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-31T06:14:31.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.671+0000 7f4e50a00640 1 -- 172.21.15.16:0/3706496356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e4c10d960 msgr2=0x7f4e4c10fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:31.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.671+0000 7f4e50a00640 1 --2- 172.21.15.16:0/3706496356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e4c10d960 0x7f4e4c10fe50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4e44009860 tx=0x7f4e4402a340 comp rx=0 tx=0).stop 2024-08-31T06:14:31.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.672+0000 7f4e50a00640 1 -- 172.21.15.16:0/3706496356 shutdown_connections 2024-08-31T06:14:31.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.672+0000 7f4e50a00640 1 --2- 172.21.15.16:0/3706496356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e4c104e50 0x7f4e4c10d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:31.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.672+0000 7f4e50a00640 1 --2- 172.21.15.16:0/3706496356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e4c10d960 0x7f4e4c10fe50 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:31.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.672+0000 7f4e50a00640 1 --2- 172.21.15.16:0/3706496356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e4c104500 0x7f4e4c104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:31.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.672+0000 7f4e50a00640 1 -- 172.21.15.16:0/3706496356 >> 172.21.15.16:0/3706496356 conn(0x7f4e4c0fdfd0 msgr2=0x7f4e4c100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:31.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.672+0000 7f4e50a00640 1 -- 172.21.15.16:0/3706496356 shutdown_connections 2024-08-31T06:14:31.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.672+0000 7f4e50a00640 1 -- 172.21.15.16:0/3706496356 wait complete. 2024-08-31T06:14:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.673+0000 7f4e50a00640 1 Processor -- start 2024-08-31T06:14:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.673+0000 7f4e50a00640 1 -- start start 2024-08-31T06:14:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.673+0000 7f4e50a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e4c104500 0x7f4e4c19c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.673+0000 7f4e50a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e4c104e50 0x7f4e4c19c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.673+0000 7f4e50a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e4c10d960 0x7f4e4c1a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.673+0000 7f4e50a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4c1135d0 con 0x7f4e4c10d960 2024-08-31T06:14:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.673+0000 7f4e50a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4c113450 con 0x7f4e4c104500 2024-08-31T06:14:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.673+0000 7f4e50a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4c113750 con 0x7f4e4c104e50 2024-08-31T06:14:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.674+0000 7f4e4b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e4c10d960 0x7f4e4c1a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.674+0000 7f4e4b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e4c10d960 0x7f4e4c1a1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55498/0 (socket says 172.21.15.16:55498) 2024-08-31T06:14:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.674+0000 7f4e4b200640 1 -- 172.21.15.16:0/1624495457 learned_addr learned my addr 172.21.15.16:0/1624495457 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.674+0000 7f4e41e00640 1 --2- 172.21.15.16:0/1624495457 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e4c104e50 0x7f4e4c19c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.674+0000 7f4e4a800640 1 --2- 172.21.15.16:0/1624495457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e4c104500 0x7f4e4c19c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.674+0000 7f4e4b200640 1 -- 172.21.15.16:0/1624495457 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e4c104e50 msgr2=0x7f4e4c19c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.674+0000 7f4e4b200640 1 --2- 172.21.15.16:0/1624495457 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e4c104e50 0x7f4e4c19c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.674+0000 7f4e4b200640 1 -- 172.21.15.16:0/1624495457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e4c104500 msgr2=0x7f4e4c19c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.674+0000 7f4e4b200640 1 --2- 172.21.15.16:0/1624495457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e4c104500 0x7f4e4c19c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.674+0000 7f4e4b200640 1 -- 172.21.15.16:0/1624495457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e4c1a2120 con 0x7f4e4c10d960 2024-08-31T06:14:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.674+0000 7f4e4b200640 1 --2- 172.21.15.16:0/1624495457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e4c10d960 0x7f4e4c1a1a20 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f4e4402a930 tx=0x7f4e44032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.675+0000 7f4e43e00640 1 -- 172.21.15.16:0/1624495457 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e44040020 con 0x7f4e4c10d960 2024-08-31T06:14:31.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.675+0000 7f4e43e00640 1 -- 172.21.15.16:0/1624495457 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e44030d50 con 0x7f4e4c10d960 2024-08-31T06:14:31.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.675+0000 7f4e50a00640 1 -- 172.21.15.16:0/1624495457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e4c1a23b0 con 0x7f4e4c10d960 2024-08-31T06:14:31.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.675+0000 7f4e43e00640 1 -- 172.21.15.16:0/1624495457 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e44037040 con 0x7f4e4c10d960 2024-08-31T06:14:31.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.675+0000 7f4e50a00640 1 -- 172.21.15.16:0/1624495457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e4c1a8ba0 con 0x7f4e4c10d960 2024-08-31T06:14:31.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.678+0000 7f4e585bb740 1 Processor -- start 2024-08-31T06:14:31.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.679+0000 7f4e43e00640 1 -- 172.21.15.16:0/1624495457 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f4e44043020 con 0x7f4e4c10d960 2024-08-31T06:14:31.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.679+0000 7f4e585bb740 1 -- start start 2024-08-31T06:14:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.679+0000 7f4e585bb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4d33ad1b0 0x55d4d37508a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.679+0000 7f4e585bb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d33842f0 0x55d4d3750de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.679+0000 7f4e585bb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4d338c4d0 0x55d4d3751320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.679+0000 7f4e585bb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d4d36c5f70 con 0x55d4d33ad1b0 2024-08-31T06:14:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.679+0000 7f4e585bb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d4d36b5500 con 0x55d4d33842f0 2024-08-31T06:14:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.679+0000 7f4e585bb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d4d36b94b0 con 0x55d4d338c4d0 2024-08-31T06:14:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.680+0000 7f4e41e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4d33ad1b0 0x55d4d37508a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.680+0000 7f4e41e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4d33ad1b0 0x55d4d37508a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55512/0 (socket says 172.21.15.16:55512) 2024-08-31T06:14:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.680+0000 7f4e41e00640 1 -- 172.21.15.16:0/760823373 learned_addr learned my addr 172.21.15.16:0/760823373 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.680+0000 7f4e4a800640 1 --2- 172.21.15.16:0/760823373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d33842f0 0x55d4d3750de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.680+0000 7f4e41e00640 1 --2- 172.21.15.16:0/760823373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4d338c4d0 0x55d4d3751320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.680+0000 7f4e41e00640 1 -- 172.21.15.16:0/760823373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4d338c4d0 msgr2=0x55d4d3751320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.680+0000 7f4e41e00640 1 --2- 172.21.15.16:0/760823373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4d338c4d0 0x55d4d3751320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.680+0000 7f4e41e00640 1 -- 172.21.15.16:0/760823373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d33842f0 msgr2=0x55d4d3750de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.680+0000 7f4e41e00640 1 --2- 172.21.15.16:0/760823373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d33842f0 0x55d4d3750de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.680+0000 7f4e41e00640 1 -- 172.21.15.16:0/760823373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4d366fc10 con 0x55d4d33ad1b0 2024-08-31T06:14:31.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.680+0000 7f4e4a800640 1 --2- 172.21.15.16:0/760823373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d33842f0 0x55d4d3750de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:14:31.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.680+0000 7f4e43e00640 1 --2- 172.21.15.16:0/1624495457 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f4e300b4ca0 0x7f4e300b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:31.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.681+0000 7f4e43e00640 1 -- 172.21.15.16:0/1624495457 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7f4e44105170 con 0x7f4e4c10d960 2024-08-31T06:14:31.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.681+0000 7f4e41e00640 1 --2- 172.21.15.16:0/760823373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4d33ad1b0 0x55d4d37508a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4e340093e0 tx=0x7f4e34009370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d07f2b6d590c82 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:31.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.681+0000 7f4e4a800640 1 --2- 172.21.15.16:0/1624495457 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f4e300b4ca0 0x7f4e300b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:31.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.681+0000 7f4e3b400640 1 -- 172.21.15.16:0/760823373 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e34004030 con 0x55d4d33ad1b0 2024-08-31T06:14:31.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.681+0000 7f4e3b400640 1 -- 172.21.15.16:0/760823373 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e3400f790 con 0x55d4d33ad1b0 2024-08-31T06:14:31.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.681+0000 7f4e3b400640 1 -- 172.21.15.16:0/760823373 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e34003850 con 0x55d4d33ad1b0 2024-08-31T06:14:31.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.682+0000 7f4e585bb740 1 -- 172.21.15.16:0/760823373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4d33ad1b0 msgr2=0x55d4d37508a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:31.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.682+0000 7f4e585bb740 1 --2- 172.21.15.16:0/760823373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4d33ad1b0 0x55d4d37508a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4e340093e0 tx=0x7f4e34009370 comp rx=0 tx=0).stop 2024-08-31T06:14:31.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.682+0000 7f4e4a800640 1 --2- 172.21.15.16:0/1624495457 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f4e300b4ca0 0x7f4e300b71b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c009fd0 tx=0x7f4e3c010040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:31.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.682+0000 7f4e585bb740 1 -- 172.21.15.16:0/760823373 shutdown_connections 2024-08-31T06:14:31.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.682+0000 7f4e585bb740 1 --2- 172.21.15.16:0/760823373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4d338c4d0 0x55d4d3751320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:31.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.682+0000 7f4e585bb740 1 --2- 172.21.15.16:0/760823373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d33842f0 0x55d4d3750de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.682+0000 7f4e585bb740 1 --2- 172.21.15.16:0/760823373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4d33ad1b0 0x55d4d37508a0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.682+0000 7f4e585bb740 1 -- 172.21.15.16:0/760823373 >> 172.21.15.16:0/760823373 conn(0x55d4d33a1300 msgr2=0x55d4d374e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.682+0000 7f4e585bb740 1 -- 172.21.15.16:0/760823373 shutdown_connections 2024-08-31T06:14:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.682+0000 7f4e585bb740 1 -- 172.21.15.16:0/760823373 wait complete. 2024-08-31T06:14:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.683+0000 7f4e585bb740 1 -- start start 2024-08-31T06:14:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.683+0000 7f4e585bb740 1 Processor -- start 2024-08-31T06:14:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.683+0000 7f4e585bb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4d33842f0 0x55d4d375be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.683+0000 7f4e585bb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4d338c4d0 0x55d4d375c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.683+0000 7f4e585bb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d33ad1b0 0x55d4d375c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.683+0000 7f4e585bb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d4d35eeab0 con 0x55d4d338c4d0 2024-08-31T06:14:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.683+0000 7f4e585bb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d4d365c320 con 0x55d4d33ad1b0 2024-08-31T06:14:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.683+0000 7f4e585bb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d4d375efc0 con 0x55d4d33842f0 2024-08-31T06:14:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.684+0000 7f4e585bb740 10 client.0 _ll_register_callbacks cb 0x55d4d3695e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:14:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.684+0000 7f4e41e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4d338c4d0 0x55d4d375c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.684+0000 7f4e41e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4d338c4d0 0x55d4d375c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55514/0 (socket says 172.21.15.16:55514) 2024-08-31T06:14:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.684+0000 7f4e41e00640 1 -- 172.21.15.16:0/286842886 learned_addr learned my addr 172.21.15.16:0/286842886 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.684+0000 7f4e4a800640 1 --2- 172.21.15.16:0/286842886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4d33842f0 0x55d4d375be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.684+0000 7f4e4b200640 1 --2- 172.21.15.16:0/286842886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d33ad1b0 0x55d4d375c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.684+0000 7f4e41e00640 1 -- 172.21.15.16:0/286842886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4d33842f0 msgr2=0x55d4d375be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.684+0000 7f4e41e00640 1 --2- 172.21.15.16:0/286842886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4d33842f0 0x55d4d375be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.684+0000 7f4e41e00640 1 -- 172.21.15.16:0/286842886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d33ad1b0 msgr2=0x55d4d375c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.684+0000 7f4e41e00640 1 --2- 172.21.15.16:0/286842886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d33ad1b0 0x55d4d375c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.684+0000 7f4e41e00640 1 -- 172.21.15.16:0/286842886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4d3418790 con 0x55d4d338c4d0 2024-08-31T06:14:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.684+0000 7f4e41e00640 1 --2- 172.21.15.16:0/286842886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4d338c4d0 0x55d4d375c3c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4e3400ffe0 tx=0x7f4e34005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a46ec183350caeb9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.685+0000 7f4e2fe00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:14:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.685+0000 7f4e2fe00640 1 -- 172.21.15.16:0/286842886 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e34005230 con 0x55d4d338c4d0 2024-08-31T06:14:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.685+0000 7f4e585bb740 1 -- 172.21.15.16:0/286842886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e3400f790 con 0x55d4d338c4d0 2024-08-31T06:14:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.685+0000 7f4e2fe00640 1 -- 172.21.15.16:0/286842886 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7f4e340053d0 con 0x55d4d338c4d0 2024-08-31T06:14:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.685+0000 7f4e2fe00640 1 -- 172.21.15.16:0/286842886 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4e34005610 con 0x55d4d338c4d0 2024-08-31T06:14:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.685+0000 7f4e585bb740 10 client.16992 fetch_fsmap learned FSMap version 39 2024-08-31T06:14:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.685+0000 7f4e585bb740 1 -- 172.21.15.16:0/286842886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d4d3491620 con 0x55d4d338c4d0 2024-08-31T06:14:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.685+0000 7f4e2fe00640 1 -- 172.21.15.16:0/286842886 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e3400f790 con 0x55d4d338c4d0 2024-08-31T06:14:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.686+0000 7f4e2fe00640 1 -- 172.21.15.16:0/286842886 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f4e34028760 con 0x55d4d338c4d0 2024-08-31T06:14:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.686+0000 7f4e585bb740 10 client.16992 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T06:14:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.686+0000 7f4e585bb740 10 client.16992 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:14:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.686+0000 7f4e585bb740 20 client.16992 populate_metadata read hostname 'smithi016' 2024-08-31T06:14:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.686+0000 7f4e585bb740 1 --2- 172.21.15.16:0/286842886 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d4d336bd80 0x55d4d376de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.686+0000 7f4e585bb740 4 client.16992 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:14:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.686+0000 7f4e585bb740 1 -- 172.21.15.16:0/286842886 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4d36c7ec0 con 0x55d4d336bd80 2024-08-31T06:14:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.686+0000 7f4e4a800640 1 --2- 172.21.15.16:0/286842886 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d4d336bd80 0x55d4d376de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.687+0000 7f4e4a800640 1 --2- 172.21.15.16:0/286842886 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d4d336bd80 0x55d4d376de60 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f796bcb33426367d server_cookie=853a2baec72d4024 in_seq=0 out_seq=0 2024-08-31T06:14:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.687+0000 7f4e2fe00640 10 client.16992 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.688+0000 7f4e2fe00640 1 -- 172.21.15.16:0/286842886 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d4d36c7ec0 con 0x55d4d336bd80 2024-08-31T06:14:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.688+0000 7f4e2fe00640 10 client.16992 handle_command_reply: tid=0 2024-08-31T06:14:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.688+0000 7f4e585bb740 1 client.16992 shutdown 2024-08-31T06:14:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.688+0000 7f4e585bb740 1 -- 172.21.15.16:0/286842886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4d338c4d0 msgr2=0x55d4d375c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.688+0000 7f4e585bb740 1 --2- 172.21.15.16:0/286842886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4d338c4d0 0x55d4d375c3c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4e3400ffe0 tx=0x7f4e34005040 comp rx=0 tx=0).stop 2024-08-31T06:14:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.688+0000 7f4e585bb740 1 -- 172.21.15.16:0/286842886 shutdown_connections 2024-08-31T06:14:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.688+0000 7f4e585bb740 1 --2- 172.21.15.16:0/286842886 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d4d336bd80 0x55d4d376de60 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.688+0000 7f4e585bb740 1 --2- 172.21.15.16:0/286842886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d33ad1b0 0x55d4d375c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.688+0000 7f4e585bb740 1 --2- 172.21.15.16:0/286842886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4d338c4d0 0x55d4d375c3c0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.688+0000 7f4e585bb740 1 --2- 172.21.15.16:0/286842886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4d33842f0 0x55d4d375be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:31.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.688+0000 7f4e2fe00640 0 client.16992 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:31.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.688+0000 7f4e585bb740 1 -- 172.21.15.16:0/286842886 >> 172.21.15.16:0/286842886 conn(0x55d4d33a1300 msgr2=0x55d4d37518d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:31.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.688+0000 7f4e585bb740 1 -- 172.21.15.16:0/286842886 shutdown_connections 2024-08-31T06:14:31.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.688+0000 7f4e585bb740 1 -- 172.21.15.16:0/286842886 wait complete. 2024-08-31T06:14:31.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.688+0000 7f4e585bb740 20 client.16992 trim_cache size 0 max 16384 2024-08-31T06:14:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.705+0000 7f4e585bb740 1 Processor -- start 2024-08-31T06:14:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.705+0000 7f4e585bb740 1 -- start start 2024-08-31T06:14:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.705+0000 7f4e585bb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4d33842f0 0x55d4d3751fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.705+0000 7f4e585bb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4d338c4d0 0x55d4d3759050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.705+0000 7f4e585bb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d33ad1b0 0x55d4d3759590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.705+0000 7f4e585bb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4c1b3cd0 con 0x55d4d338c4d0 2024-08-31T06:14:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.705+0000 7f4e585bb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4c1b1540 con 0x55d4d33ad1b0 2024-08-31T06:14:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.705+0000 7f4e585bb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4c1af3b0 con 0x55d4d33842f0 2024-08-31T06:14:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.706+0000 7f4e4a800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4d33842f0 0x55d4d3751fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.706+0000 7f4e41e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4d338c4d0 0x55d4d3759050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.706+0000 7f4e4a800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4d33842f0 0x55d4d3751fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:51532/0 (socket says 172.21.15.16:51532) 2024-08-31T06:14:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.706+0000 7f4e4a800640 1 -- 172.21.15.16:0/1014716091 learned_addr learned my addr 172.21.15.16:0/1014716091 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.706+0000 7f4e4b200640 1 --2- 172.21.15.16:0/1014716091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d33ad1b0 0x55d4d3759590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.706+0000 7f4e41e00640 1 -- 172.21.15.16:0/1014716091 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4d33842f0 msgr2=0x55d4d3751fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.706+0000 7f4e41e00640 1 --2- 172.21.15.16:0/1014716091 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4d33842f0 0x55d4d3751fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.706+0000 7f4e41e00640 1 -- 172.21.15.16:0/1014716091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d33ad1b0 msgr2=0x55d4d3759590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.706+0000 7f4e41e00640 1 --2- 172.21.15.16:0/1014716091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d33ad1b0 0x55d4d3759590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.706+0000 7f4e41e00640 1 -- 172.21.15.16:0/1014716091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4d375e9c0 con 0x55d4d338c4d0 2024-08-31T06:14:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.706+0000 7f4e4a800640 1 --2- 172.21.15.16:0/1014716091 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4d33842f0 0x55d4d3751fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:14:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.706+0000 7f4e41e00640 1 --2- 172.21.15.16:0/1014716091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4d338c4d0 0x55d4d3759050 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4e34028d70 tx=0x7f4e34011760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f87f3004ea5a417a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.707+0000 7f4e3aa00640 1 -- 172.21.15.16:0/1014716091 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e34035030 con 0x55d4d338c4d0 2024-08-31T06:14:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.707+0000 7f4e3aa00640 1 -- 172.21.15.16:0/1014716091 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e3400fc50 con 0x55d4d338c4d0 2024-08-31T06:14:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.707+0000 7f4e3aa00640 1 -- 172.21.15.16:0/1014716091 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e340288c0 con 0x55d4d338c4d0 2024-08-31T06:14:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.708+0000 7f4e585bb740 1 -- 172.21.15.16:0/1014716091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4d338c4d0 msgr2=0x55d4d3759050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.708+0000 7f4e585bb740 1 --2- 172.21.15.16:0/1014716091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4d338c4d0 0x55d4d3759050 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4e34028d70 tx=0x7f4e34011760 comp rx=0 tx=0).stop 2024-08-31T06:14:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.708+0000 7f4e585bb740 1 -- 172.21.15.16:0/1014716091 shutdown_connections 2024-08-31T06:14:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.708+0000 7f4e585bb740 1 --2- 172.21.15.16:0/1014716091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d33ad1b0 0x55d4d3759590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.708+0000 7f4e585bb740 1 --2- 172.21.15.16:0/1014716091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4d338c4d0 0x55d4d3759050 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.708+0000 7f4e585bb740 1 --2- 172.21.15.16:0/1014716091 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4d33842f0 0x55d4d3751fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.708+0000 7f4e585bb740 1 -- 172.21.15.16:0/1014716091 >> 172.21.15.16:0/1014716091 conn(0x55d4d33a1300 msgr2=0x55d4d37518d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.708+0000 7f4e585bb740 1 -- 172.21.15.16:0/1014716091 shutdown_connections 2024-08-31T06:14:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.708+0000 7f4e585bb740 1 -- 172.21.15.16:0/1014716091 wait complete. 2024-08-31T06:14:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.709+0000 7f4e585bb740 1 -- start start 2024-08-31T06:14:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.709+0000 7f4e585bb740 1 Processor -- start 2024-08-31T06:14:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.709+0000 7f4e585bb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4d33842f0 0x55d4d3761c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.709+0000 7f4e585bb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4d338c4d0 0x55d4d3762210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.709+0000 7f4e585bb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d33ad1b0 0x55d4d37d0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.709+0000 7f4e585bb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4c1afa00 con 0x55d4d338c4d0 2024-08-31T06:14:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.709+0000 7f4e585bb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4c1b0e70 con 0x55d4d33ad1b0 2024-08-31T06:14:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.709+0000 7f4e585bb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4c1b00d0 con 0x55d4d33842f0 2024-08-31T06:14:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.709+0000 7f4e585bb740 10 client.0 _ll_register_callbacks cb 0x7f4e34009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:14:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.710+0000 7f4e41e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4d338c4d0 0x55d4d3762210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.710+0000 7f4e4a800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4d33842f0 0x55d4d3761c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.710+0000 7f4e4b200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d33ad1b0 0x55d4d37d0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.710+0000 7f4e4a800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4d33842f0 0x55d4d3761c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:51538/0 (socket says 172.21.15.16:51538) 2024-08-31T06:14:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.710+0000 7f4e4a800640 1 -- 172.21.15.16:0/2310889459 learned_addr learned my addr 172.21.15.16:0/2310889459 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.710+0000 7f4e41e00640 1 -- 172.21.15.16:0/2310889459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4d33842f0 msgr2=0x55d4d3761c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.710+0000 7f4e41e00640 1 --2- 172.21.15.16:0/2310889459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4d33842f0 0x55d4d3761c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.710+0000 7f4e41e00640 1 -- 172.21.15.16:0/2310889459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d33ad1b0 msgr2=0x55d4d37d0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.710+0000 7f4e41e00640 1 --2- 172.21.15.16:0/2310889459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d33ad1b0 0x55d4d37d0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.710+0000 7f4e41e00640 1 -- 172.21.15.16:0/2310889459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4d37536b0 con 0x55d4d338c4d0 2024-08-31T06:14:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.710+0000 7f4e4b200640 1 --2- 172.21.15.16:0/2310889459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d33ad1b0 0x55d4d37d0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:14:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.710+0000 7f4e4a800640 1 --2- 172.21.15.16:0/2310889459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4d33842f0 0x55d4d3761c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:14:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.711+0000 7f4e41e00640 1 --2- 172.21.15.16:0/2310889459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4d338c4d0 0x55d4d3762210 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4e3400ff30 tx=0x7f4e34009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeee9a64a9fccf2d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.711+0000 7f4e2fe00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:14:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.712+0000 7f4e2fe00640 1 -- 172.21.15.16:0/2310889459 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3403e070 con 0x55d4d338c4d0 2024-08-31T06:14:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.712+0000 7f4e2fe00640 1 -- 172.21.15.16:0/2310889459 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7f4e34011240 con 0x55d4d338c4d0 2024-08-31T06:14:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.712+0000 7f4e585bb740 1 -- 172.21.15.16:0/2310889459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e3400fc50 con 0x55d4d338c4d0 2024-08-31T06:14:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.712+0000 7f4e2fe00640 1 -- 172.21.15.16:0/2310889459 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4e34049050 con 0x55d4d338c4d0 2024-08-31T06:14:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.712+0000 7f4e585bb740 10 client.16998 fetch_fsmap learned FSMap version 39 2024-08-31T06:14:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.712+0000 7f4e585bb740 1 -- 172.21.15.16:0/2310889459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d4d37d8000 con 0x55d4d338c4d0 2024-08-31T06:14:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.713+0000 7f4e2fe00640 1 -- 172.21.15.16:0/2310889459 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e3400fc50 con 0x55d4d338c4d0 2024-08-31T06:14:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.713+0000 7f4e2fe00640 1 -- 172.21.15.16:0/2310889459 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f4e340036f0 con 0x55d4d338c4d0 2024-08-31T06:14:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.713+0000 7f4e585bb740 10 client.16998 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T06:14:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.713+0000 7f4e585bb740 10 client.16998 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:14:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.713+0000 7f4e585bb740 20 client.16998 populate_metadata read hostname 'smithi016' 2024-08-31T06:14:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.713+0000 7f4e585bb740 1 --2- 172.21.15.16:0/2310889459 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d4d33411c0 0x55d4d37da650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.713+0000 7f4e585bb740 4 client.16998 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:14:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.713+0000 7f4e585bb740 1 -- 172.21.15.16:0/2310889459 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d4d36c3960 con 0x55d4d33411c0 2024-08-31T06:14:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.713+0000 7f4e4a800640 1 --2- 172.21.15.16:0/2310889459 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d4d33411c0 0x55d4d37da650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.714+0000 7f4e4a800640 1 --2- 172.21.15.16:0/2310889459 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d4d33411c0 0x55d4d37da650 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ae405ca94e201ec server_cookie=64936fb784cae0f4 in_seq=0 out_seq=0 2024-08-31T06:14:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.714+0000 7f4e2fe00640 10 client.16998 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:31.730 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.728+0000 7f4e2fe00640 1 -- 172.21.15.16:0/2310889459 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d4d36c3960 con 0x55d4d33411c0 2024-08-31T06:14:31.730 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.728+0000 7f4e2fe00640 10 client.16998 handle_command_reply: tid=0 2024-08-31T06:14:31.730 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.728+0000 7f4e585bb740 1 client.16998 shutdown 2024-08-31T06:14:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.728+0000 7f4e585bb740 1 -- 172.21.15.16:0/2310889459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4d338c4d0 msgr2=0x55d4d3762210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.728+0000 7f4e585bb740 1 --2- 172.21.15.16:0/2310889459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4d338c4d0 0x55d4d3762210 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4e3400ff30 tx=0x7f4e34009cd0 comp rx=0 tx=0).stop 2024-08-31T06:14:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.728+0000 7f4e585bb740 1 -- 172.21.15.16:0/2310889459 shutdown_connections 2024-08-31T06:14:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.728+0000 7f4e585bb740 1 --2- 172.21.15.16:0/2310889459 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d4d33411c0 0x55d4d37da650 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.728+0000 7f4e585bb740 1 --2- 172.21.15.16:0/2310889459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d33ad1b0 0x55d4d37d0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.728+0000 7f4e585bb740 1 --2- 172.21.15.16:0/2310889459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4d338c4d0 0x55d4d3762210 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.728+0000 7f4e2fe00640 0 client.16998 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.728+0000 7f4e585bb740 1 --2- 172.21.15.16:0/2310889459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4d33842f0 0x55d4d3761c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.728+0000 7f4e585bb740 1 -- 172.21.15.16:0/2310889459 >> 172.21.15.16:0/2310889459 conn(0x55d4d33a1300 msgr2=0x55d4d374e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.728+0000 7f4e585bb740 1 -- 172.21.15.16:0/2310889459 shutdown_connections 2024-08-31T06:14:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.728+0000 7f4e585bb740 1 -- 172.21.15.16:0/2310889459 wait complete. 2024-08-31T06:14:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.729+0000 7f4e585bb740 20 client.16998 trim_cache size 0 max 16384 2024-08-31T06:14:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:14:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:14:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:14:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:14:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.729+0000 7f4e50a00640 1 -- 172.21.15.16:0/1624495457 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f4e300b4ca0 msgr2=0x7f4e300b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.729+0000 7f4e50a00640 1 --2- 172.21.15.16:0/1624495457 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f4e300b4ca0 0x7f4e300b71b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c009fd0 tx=0x7f4e3c010040 comp rx=0 tx=0).stop 2024-08-31T06:14:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.729+0000 7f4e50a00640 1 -- 172.21.15.16:0/1624495457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e4c10d960 msgr2=0x7f4e4c1a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.729+0000 7f4e50a00640 1 --2- 172.21.15.16:0/1624495457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e4c10d960 0x7f4e4c1a1a20 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f4e4402a930 tx=0x7f4e44032d50 comp rx=0 tx=0).stop 2024-08-31T06:14:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.729+0000 7f4e50a00640 1 -- 172.21.15.16:0/1624495457 shutdown_connections 2024-08-31T06:14:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.729+0000 7f4e50a00640 1 --2- 172.21.15.16:0/1624495457 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f4e300b4ca0 0x7f4e300b71b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.729+0000 7f4e50a00640 1 --2- 172.21.15.16:0/1624495457 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e4c104e50 0x7f4e4c19c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.729+0000 7f4e50a00640 1 --2- 172.21.15.16:0/1624495457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e4c10d960 0x7f4e4c1a1a20 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.729+0000 7f4e50a00640 1 --2- 172.21.15.16:0/1624495457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e4c104500 0x7f4e4c19c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.729+0000 7f4e50a00640 1 -- 172.21.15.16:0/1624495457 >> 172.21.15.16:0/1624495457 conn(0x7f4e4c0fdfd0 msgr2=0x7f4e4c1020f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.730+0000 7f4e50a00640 1 -- 172.21.15.16:0/1624495457 shutdown_connections 2024-08-31T06:14:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:31.730+0000 7f4e50a00640 1 -- 172.21.15.16:0/1624495457 wait complete. 2024-08-31T06:14:31.876 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24737 quiesce db --set-id 2403ca55 --release --await' 2024-08-31T06:14:31.877 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24737 quiesce db --set-id 2403ca55 --release --await 2024-08-31T06:14:31.966 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:31 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/2357598035' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:14:31.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:31 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/2357598035' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:14:32.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:31 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/2357598035' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:14:32.298 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:14:32.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:32 smithi184 ceph-mon[22331]: pgmap v587: 97 pgs: 97 active+clean; 3.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 4.9 MiB/s wr, 628 op/s 2024-08-31T06:14:32.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.972+0000 7f01ef400640 1 -- 172.21.15.16:0/1775124264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01e806be80 msgr2=0x7f01e806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:32.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.972+0000 7f01ef400640 1 --2- 172.21.15.16:0/1775124264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01e806be80 0x7f01e806c290 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f01d4009900 tx=0x7f01d402a3f0 comp rx=0 tx=0).stop 2024-08-31T06:14:32.975 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:32 smithi016 ceph-mon[17194]: pgmap v587: 97 pgs: 97 active+clean; 3.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 4.9 MiB/s wr, 628 op/s 2024-08-31T06:14:32.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.972+0000 7f01ef400640 1 -- 172.21.15.16:0/1775124264 shutdown_connections 2024-08-31T06:14:32.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.972+0000 7f01ef400640 1 --2- 172.21.15.16:0/1775124264 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01e810fde0 0x7f01e81122f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:32.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.972+0000 7f01ef400640 1 --2- 172.21.15.16:0/1775124264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01e806be80 0x7f01e806c290 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:32.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.972+0000 7f01ef400640 1 --2- 172.21.15.16:0/1775124264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01e8104440 0x7f01e806b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:32.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.972+0000 7f01ef400640 1 -- 172.21.15.16:0/1775124264 >> 172.21.15.16:0/1775124264 conn(0x7f01e8079740 msgr2=0x7f01e8079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:32.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.972+0000 7f01ef400640 1 -- 172.21.15.16:0/1775124264 shutdown_connections 2024-08-31T06:14:32.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.972+0000 7f01ef400640 1 -- 172.21.15.16:0/1775124264 wait complete. 2024-08-31T06:14:32.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.973+0000 7f01ef400640 1 Processor -- start 2024-08-31T06:14:32.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.974+0000 7f01ef400640 1 -- start start 2024-08-31T06:14:32.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.974+0000 7f01ef400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01e806be80 0x7f01e819c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:32.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.974+0000 7f01ef400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01e8104440 0x7f01e819c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:32.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.974+0000 7f01ef400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01e810fde0 0x7f01e81a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:32.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.974+0000 7f01ef400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f01e8115ab0 con 0x7f01e806be80 2024-08-31T06:14:32.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.974+0000 7f01ef400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f01e8115930 con 0x7f01e810fde0 2024-08-31T06:14:32.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.974+0000 7f01ef400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f01e8115c30 con 0x7f01e8104440 2024-08-31T06:14:32.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.974+0000 7f01eca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01e806be80 0x7f01e819c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:32.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.974+0000 7f01eca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01e806be80 0x7f01e819c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55554/0 (socket says 172.21.15.16:55554) 2024-08-31T06:14:32.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.974+0000 7f01eca00640 1 -- 172.21.15.16:0/315044382 learned_addr learned my addr 172.21.15.16:0/315044382 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:32.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.974+0000 7f01e7e00640 1 --2- 172.21.15.16:0/315044382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01e8104440 0x7f01e819c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:32.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.974+0000 7f01ed400640 1 --2- 172.21.15.16:0/315044382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01e810fde0 0x7f01e81a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:32.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.975+0000 7f01eca00640 1 -- 172.21.15.16:0/315044382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01e8104440 msgr2=0x7f01e819c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:32.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.975+0000 7f01eca00640 1 --2- 172.21.15.16:0/315044382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01e8104440 0x7f01e819c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:32.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.975+0000 7f01eca00640 1 -- 172.21.15.16:0/315044382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01e810fde0 msgr2=0x7f01e81a1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:32.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.975+0000 7f01eca00640 1 --2- 172.21.15.16:0/315044382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01e810fde0 0x7f01e81a1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:32.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.975+0000 7f01eca00640 1 -- 172.21.15.16:0/315044382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01e81a23b0 con 0x7f01e806be80 2024-08-31T06:14:32.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.975+0000 7f01e7e00640 1 --2- 172.21.15.16:0/315044382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01e8104440 0x7f01e819c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:14:32.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.975+0000 7f01ed400640 1 --2- 172.21.15.16:0/315044382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01e810fde0 0x7f01e81a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:14:32.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.975+0000 7f01eca00640 1 --2- 172.21.15.16:0/315044382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01e806be80 0x7f01e819c440 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f01dc00ea60 tx=0x7f01dc00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:32.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.976+0000 7f01e5600640 1 -- 172.21.15.16:0/315044382 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01dc00cde0 con 0x7f01e806be80 2024-08-31T06:14:32.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.976+0000 7f01e5600640 1 -- 172.21.15.16:0/315044382 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01dc0035f0 con 0x7f01e806be80 2024-08-31T06:14:32.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.976+0000 7f01e5600640 1 -- 172.21.15.16:0/315044382 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01dc010690 con 0x7f01e806be80 2024-08-31T06:14:32.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.976+0000 7f01ef400640 1 -- 172.21.15.16:0/315044382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01e81a26a0 con 0x7f01e806be80 2024-08-31T06:14:32.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.976+0000 7f01ef400640 1 -- 172.21.15.16:0/315044382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01e81a8e00 con 0x7f01e806be80 2024-08-31T06:14:32.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.979+0000 7f01f5fce740 1 Processor -- start 2024-08-31T06:14:32.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.979+0000 7f01f5fce740 1 -- start start 2024-08-31T06:14:32.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.979+0000 7f01e5600640 1 -- 172.21.15.16:0/315044382 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f01dc0037a0 con 0x7f01e806be80 2024-08-31T06:14:32.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.980+0000 7f01f5fce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aac4f2520 0x556aac897ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:32.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.980+0000 7f01f5fce740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556aac4c9660 0x556aac898430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:32.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.980+0000 7f01f5fce740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556aac4d1840 0x556aac898970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:32.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.980+0000 7f01f5fce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556aac80ea90 con 0x556aac4f2520 2024-08-31T06:14:32.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.980+0000 7f01f5fce740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556aac7fe020 con 0x556aac4c9660 2024-08-31T06:14:32.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.980+0000 7f01f5fce740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556aac801fd0 con 0x556aac4d1840 2024-08-31T06:14:32.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.980+0000 7f01ed400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aac4f2520 0x556aac897ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:32.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.980+0000 7f01ed400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aac4f2520 0x556aac897ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55556/0 (socket says 172.21.15.16:55556) 2024-08-31T06:14:32.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.980+0000 7f01ed400640 1 -- 172.21.15.16:0/2063612713 learned_addr learned my addr 172.21.15.16:0/2063612713 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:32.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.980+0000 7f01eca00640 1 --2- 172.21.15.16:0/2063612713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556aac4c9660 0x556aac898430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:32.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.980+0000 7f01e7e00640 1 --2- 172.21.15.16:0/2063612713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556aac4d1840 0x556aac898970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:32.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.981+0000 7f01ed400640 1 -- 172.21.15.16:0/2063612713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556aac4d1840 msgr2=0x556aac898970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:32.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.981+0000 7f01ed400640 1 --2- 172.21.15.16:0/2063612713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556aac4d1840 0x556aac898970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:32.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.981+0000 7f01ed400640 1 -- 172.21.15.16:0/2063612713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556aac4c9660 msgr2=0x556aac898430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:32.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.981+0000 7f01ed400640 1 --2- 172.21.15.16:0/2063612713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556aac4c9660 0x556aac898430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:32.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.981+0000 7f01ed400640 1 -- 172.21.15.16:0/2063612713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556aac55db00 con 0x556aac4f2520 2024-08-31T06:14:32.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.981+0000 7f01ed400640 1 --2- 172.21.15.16:0/2063612713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aac4f2520 0x556aac897ef0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f01e00117e0 tx=0x7f01e0011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07ee9bc364e2ab2 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:32.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.981+0000 7f01d9c00640 1 -- 172.21.15.16:0/2063612713 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e0011310 con 0x556aac4f2520 2024-08-31T06:14:32.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.981+0000 7f01d9c00640 1 -- 172.21.15.16:0/2063612713 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01e0003e20 con 0x556aac4f2520 2024-08-31T06:14:32.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.981+0000 7f01d9c00640 1 -- 172.21.15.16:0/2063612713 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e00044f0 con 0x556aac4f2520 2024-08-31T06:14:32.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.982+0000 7f01e5600640 1 --2- 172.21.15.16:0/315044382 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f01d00b4d20 0x7f01d00b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:32.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.982+0000 7f01e5600640 1 -- 172.21.15.16:0/315044382 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7f01dc0e5790 con 0x7f01e806be80 2024-08-31T06:14:32.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.982+0000 7f01eca00640 1 --2- 172.21.15.16:0/315044382 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f01d00b4d20 0x7f01d00b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:32.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.982+0000 7f01f5fce740 1 -- 172.21.15.16:0/2063612713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aac4f2520 msgr2=0x556aac897ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:32.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.982+0000 7f01f5fce740 1 --2- 172.21.15.16:0/2063612713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aac4f2520 0x556aac897ef0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f01e00117e0 tx=0x7f01e0011ca0 comp rx=0 tx=0).stop 2024-08-31T06:14:32.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.982+0000 7f01f5fce740 1 -- 172.21.15.16:0/2063612713 shutdown_connections 2024-08-31T06:14:32.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.982+0000 7f01f5fce740 1 --2- 172.21.15.16:0/2063612713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556aac4d1840 0x556aac898970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:32.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.982+0000 7f01f5fce740 1 --2- 172.21.15.16:0/2063612713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556aac4c9660 0x556aac898430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:32.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.982+0000 7f01f5fce740 1 --2- 172.21.15.16:0/2063612713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aac4f2520 0x556aac897ef0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:32.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.982+0000 7f01f5fce740 1 -- 172.21.15.16:0/2063612713 >> 172.21.15.16:0/2063612713 conn(0x556aac4e6910 msgr2=0x556aac895de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:32.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.983+0000 7f01f5fce740 1 -- 172.21.15.16:0/2063612713 shutdown_connections 2024-08-31T06:14:32.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.983+0000 7f01eca00640 1 --2- 172.21.15.16:0/315044382 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f01d00b4d20 0x7f01d00b7230 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x556aac896b70 tx=0x7f01dc0eb530 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:32.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.983+0000 7f01f5fce740 1 -- 172.21.15.16:0/2063612713 wait complete. 2024-08-31T06:14:32.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.983+0000 7f01f5fce740 1 -- start start 2024-08-31T06:14:32.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.984+0000 7f01f5fce740 1 Processor -- start 2024-08-31T06:14:32.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.984+0000 7f01f5fce740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556aac4c9660 0x556aac8a32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:32.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.984+0000 7f01f5fce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aac4d1840 0x556aac8a3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:32.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.984+0000 7f01f5fce740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556aac4f2520 0x556aac8a3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:32.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.984+0000 7f01f5fce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556aac660bb0 con 0x556aac4d1840 2024-08-31T06:14:32.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.984+0000 7f01f5fce740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556aac784f90 con 0x556aac4f2520 2024-08-31T06:14:32.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.984+0000 7f01f5fce740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556aac8a66d0 con 0x556aac4c9660 2024-08-31T06:14:32.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.984+0000 7f01f5fce740 10 client.0 _ll_register_callbacks cb 0x556aac7aa710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:14:32.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.985+0000 7f01ed400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556aac4c9660 0x556aac8a32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:32.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.985+0000 7f01e7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aac4d1840 0x556aac8a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:32.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.985+0000 7f01ed400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556aac4c9660 0x556aac8a32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:51584/0 (socket says 172.21.15.16:51584) 2024-08-31T06:14:32.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.985+0000 7f01e7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aac4d1840 0x556aac8a3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55568/0 (socket says 172.21.15.16:55568) 2024-08-31T06:14:32.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.985+0000 7f01ed400640 1 -- 172.21.15.16:0/1294705831 learned_addr learned my addr 172.21.15.16:0/1294705831 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:32.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.985+0000 7f01ed400640 1 --2- 172.21.15.16:0/1294705831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556aac4f2520 0x556aac8a3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:32.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.985+0000 7f01e7e00640 1 -- 172.21.15.16:0/1294705831 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556aac4c9660 msgr2=0x556aac8a32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:32.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.985+0000 7f01e7e00640 1 --2- 172.21.15.16:0/1294705831 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556aac4c9660 0x556aac8a32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:32.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.985+0000 7f01e7e00640 1 -- 172.21.15.16:0/1294705831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556aac4f2520 msgr2=0x556aac8a3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:32.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.985+0000 7f01e7e00640 1 --2- 172.21.15.16:0/1294705831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556aac4f2520 0x556aac8a3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:32.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.985+0000 7f01e7e00640 1 -- 172.21.15.16:0/1294705831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556aac5d4960 con 0x556aac4d1840 2024-08-31T06:14:32.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.985+0000 7f01e7e00640 1 --2- 172.21.15.16:0/1294705831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aac4d1840 0x556aac8a3800 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f01d4009230 tx=0x7f01d4005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13402d9ce7e4dbc8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:32.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.985+0000 7f01ce000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:14:32.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.986+0000 7f01ce000640 1 -- 172.21.15.16:0/1294705831 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d4035040 con 0x556aac4d1840 2024-08-31T06:14:32.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.986+0000 7f01ce000640 1 -- 172.21.15.16:0/1294705831 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7f01d4004970 con 0x556aac4d1840 2024-08-31T06:14:32.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.986+0000 7f01f5fce740 1 -- 172.21.15.16:0/1294705831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556aac79ce30 con 0x556aac4d1840 2024-08-31T06:14:32.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.986+0000 7f01ce000640 1 -- 172.21.15.16:0/1294705831 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f01d404d050 con 0x556aac4d1840 2024-08-31T06:14:32.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.986+0000 7f01f5fce740 10 client.17010 fetch_fsmap learned FSMap version 39 2024-08-31T06:14:32.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.986+0000 7f01f5fce740 1 -- 172.21.15.16:0/1294705831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556aac8b20c0 con 0x556aac4d1840 2024-08-31T06:14:32.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.986+0000 7f01ce000640 1 -- 172.21.15.16:0/1294705831 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556aac79ce30 con 0x556aac4d1840 2024-08-31T06:14:32.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.987+0000 7f01ce000640 1 -- 172.21.15.16:0/1294705831 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f01d4044070 con 0x556aac4d1840 2024-08-31T06:14:32.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.987+0000 7f01f5fce740 10 client.17010 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T06:14:32.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.987+0000 7f01f5fce740 10 client.17010 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:14:32.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.987+0000 7f01f5fce740 20 client.17010 populate_metadata read hostname 'smithi016' 2024-08-31T06:14:32.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.987+0000 7f01f5fce740 1 --2- 172.21.15.16:0/1294705831 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556aac4b0ea0 0x556aac8b55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:32.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.987+0000 7f01f5fce740 4 client.17010 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:14:32.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.987+0000 7f01f5fce740 1 -- 172.21.15.16:0/1294705831 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556aac8109e0 con 0x556aac4b0ea0 2024-08-31T06:14:32.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.987+0000 7f01ed400640 1 --2- 172.21.15.16:0/1294705831 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556aac4b0ea0 0x556aac8b55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:32.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.988+0000 7f01ed400640 1 --2- 172.21.15.16:0/1294705831 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556aac4b0ea0 0x556aac8b55a0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63f5ff7a46d8f60b server_cookie=f9ff93090c21ee2c in_seq=0 out_seq=0 2024-08-31T06:14:32.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.988+0000 7f01ce000640 10 client.17010 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:32.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.989+0000 7f01ce000640 1 -- 172.21.15.16:0/1294705831 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556aac8109e0 con 0x556aac4b0ea0 2024-08-31T06:14:32.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.989+0000 7f01ce000640 10 client.17010 handle_command_reply: tid=0 2024-08-31T06:14:32.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.989+0000 7f01f5fce740 1 client.17010 shutdown 2024-08-31T06:14:32.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.989+0000 7f01f5fce740 1 -- 172.21.15.16:0/1294705831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aac4d1840 msgr2=0x556aac8a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:32.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.989+0000 7f01f5fce740 1 --2- 172.21.15.16:0/1294705831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aac4d1840 0x556aac8a3800 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f01d4009230 tx=0x7f01d4005270 comp rx=0 tx=0).stop 2024-08-31T06:14:32.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.989+0000 7f01f5fce740 1 -- 172.21.15.16:0/1294705831 shutdown_connections 2024-08-31T06:14:32.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.989+0000 7f01f5fce740 1 --2- 172.21.15.16:0/1294705831 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556aac4b0ea0 0x556aac8b55a0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:32.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.989+0000 7f01f5fce740 1 --2- 172.21.15.16:0/1294705831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556aac4f2520 0x556aac8a3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:32.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.989+0000 7f01f5fce740 1 --2- 172.21.15.16:0/1294705831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aac4d1840 0x556aac8a3800 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:32.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.989+0000 7f01f5fce740 1 --2- 172.21.15.16:0/1294705831 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556aac4c9660 0x556aac8a32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:32.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.989+0000 7f01f5fce740 1 -- 172.21.15.16:0/1294705831 >> 172.21.15.16:0/1294705831 conn(0x556aac4e6910 msgr2=0x556aac89b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:32.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.989+0000 7f01ce000640 0 client.17010 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:32.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.990+0000 7f01f5fce740 1 -- 172.21.15.16:0/1294705831 shutdown_connections 2024-08-31T06:14:32.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.990+0000 7f01f5fce740 1 -- 172.21.15.16:0/1294705831 wait complete. 2024-08-31T06:14:32.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:32.990+0000 7f01f5fce740 20 client.17010 trim_cache size 0 max 16384 2024-08-31T06:14:33.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.008+0000 7f01f5fce740 1 Processor -- start 2024-08-31T06:14:33.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.008+0000 7f01f5fce740 1 -- start start 2024-08-31T06:14:33.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.008+0000 7f01f5fce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aac4c9660 0x556aac8af8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:33.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.008+0000 7f01f5fce740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556aac4d1840 0x556aac8a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:33.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.008+0000 7f01f5fce740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556aac4f2520 0x556aac899630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:33.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.008+0000 7f01f5fce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f01e81b3f80 con 0x556aac4c9660 2024-08-31T06:14:33.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.008+0000 7f01f5fce740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f01e81b17f0 con 0x556aac4f2520 2024-08-31T06:14:33.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.008+0000 7f01f5fce740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f01e81af660 con 0x556aac4d1840 2024-08-31T06:14:33.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.008+0000 7f01ed400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aac4c9660 0x556aac8af8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:33.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.008+0000 7f01ed400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aac4c9660 0x556aac8af8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55570/0 (socket says 172.21.15.16:55570) 2024-08-31T06:14:33.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.008+0000 7f01ed400640 1 -- 172.21.15.16:0/2151926217 learned_addr learned my addr 172.21.15.16:0/2151926217 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:33.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.009+0000 7f01e7e00640 1 --2- 172.21.15.16:0/2151926217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556aac4d1840 0x556aac8a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:33.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.009+0000 7f01ed400640 1 --2- 172.21.15.16:0/2151926217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556aac4f2520 0x556aac899630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:33.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.009+0000 7f01ed400640 1 -- 172.21.15.16:0/2151926217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556aac4d1840 msgr2=0x556aac8a51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:33.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.009+0000 7f01ed400640 1 --2- 172.21.15.16:0/2151926217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556aac4d1840 0x556aac8a51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:33.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.009+0000 7f01ed400640 1 -- 172.21.15.16:0/2151926217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556aac4f2520 msgr2=0x556aac899630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:33.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.009+0000 7f01ed400640 1 --2- 172.21.15.16:0/2151926217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556aac4f2520 0x556aac899630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:33.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.009+0000 7f01ed400640 1 -- 172.21.15.16:0/2151926217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556aac89ad50 con 0x556aac4c9660 2024-08-31T06:14:33.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.009+0000 7f01e7e00640 1 --2- 172.21.15.16:0/2151926217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556aac4d1840 0x556aac8a51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:14:33.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.009+0000 7f01ed400640 1 --2- 172.21.15.16:0/2151926217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aac4c9660 0x556aac8af8f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f01e000f910 tx=0x7f01e0011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6c95fdc424601dd server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:33.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.010+0000 7f01d9200640 1 -- 172.21.15.16:0/2151926217 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e000fa60 con 0x556aac4c9660 2024-08-31T06:14:33.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.010+0000 7f01d9200640 1 -- 172.21.15.16:0/2151926217 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01e000fc00 con 0x556aac4c9660 2024-08-31T06:14:33.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.010+0000 7f01d9200640 1 -- 172.21.15.16:0/2151926217 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e002b6b0 con 0x556aac4c9660 2024-08-31T06:14:33.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.010+0000 7f01f5fce740 1 -- 172.21.15.16:0/2151926217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aac4c9660 msgr2=0x556aac8af8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:33.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.010+0000 7f01f5fce740 1 --2- 172.21.15.16:0/2151926217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aac4c9660 0x556aac8af8f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f01e000f910 tx=0x7f01e0011ca0 comp rx=0 tx=0).stop 2024-08-31T06:14:33.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.011+0000 7f01f5fce740 1 -- 172.21.15.16:0/2151926217 shutdown_connections 2024-08-31T06:14:33.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.011+0000 7f01f5fce740 1 --2- 172.21.15.16:0/2151926217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556aac4d1840 0x556aac8a51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:33.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.011+0000 7f01f5fce740 1 --2- 172.21.15.16:0/2151926217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556aac4f2520 0x556aac899630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:33.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.011+0000 7f01f5fce740 1 --2- 172.21.15.16:0/2151926217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aac4c9660 0x556aac8af8f0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:33.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.011+0000 7f01f5fce740 1 -- 172.21.15.16:0/2151926217 >> 172.21.15.16:0/2151926217 conn(0x556aac4e6910 msgr2=0x556aac898f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:33.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.011+0000 7f01f5fce740 1 -- 172.21.15.16:0/2151926217 shutdown_connections 2024-08-31T06:14:33.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.011+0000 7f01f5fce740 1 -- 172.21.15.16:0/2151926217 wait complete. 2024-08-31T06:14:33.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.011+0000 7f01f5fce740 1 -- start start 2024-08-31T06:14:33.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.012+0000 7f01f5fce740 1 Processor -- start 2024-08-31T06:14:33.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.012+0000 7f01f5fce740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556aac4c9660 0x556aac8a7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:33.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.012+0000 7f01f5fce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aac4d1840 0x556aac8a7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:33.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.012+0000 7f01f5fce740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556aac4f2520 0x556aac8a84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:33.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.012+0000 7f01f5fce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f01e81afcb0 con 0x556aac4d1840 2024-08-31T06:14:33.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.012+0000 7f01f5fce740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f01e81b1120 con 0x556aac4f2520 2024-08-31T06:14:33.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.012+0000 7f01f5fce740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f01e81b0380 con 0x556aac4c9660 2024-08-31T06:14:33.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.012+0000 7f01f5fce740 10 client.0 _ll_register_callbacks cb 0x7f01d4004480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:14:33.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.012+0000 7f01e7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aac4d1840 0x556aac8a7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:33.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.012+0000 7f01ed400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556aac4c9660 0x556aac8a7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:33.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.012+0000 7f01e7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aac4d1840 0x556aac8a7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55576/0 (socket says 172.21.15.16:55576) 2024-08-31T06:14:33.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.013+0000 7f01e7e00640 1 -- 172.21.15.16:0/2629734277 learned_addr learned my addr 172.21.15.16:0/2629734277 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:33.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.013+0000 7f01ed400640 1 --2- 172.21.15.16:0/2629734277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556aac4f2520 0x556aac8a84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:33.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.013+0000 7f01e7e00640 1 -- 172.21.15.16:0/2629734277 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556aac4c9660 msgr2=0x556aac8a7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:33.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.013+0000 7f01e7e00640 1 --2- 172.21.15.16:0/2629734277 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556aac4c9660 0x556aac8a7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:33.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.013+0000 7f01e7e00640 1 -- 172.21.15.16:0/2629734277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556aac4f2520 msgr2=0x556aac8a84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:33.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.013+0000 7f01e7e00640 1 --2- 172.21.15.16:0/2629734277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556aac4f2520 0x556aac8a84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:33.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.013+0000 7f01e7e00640 1 -- 172.21.15.16:0/2629734277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556aac91f000 con 0x556aac4d1840 2024-08-31T06:14:33.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.013+0000 7f01e7e00640 1 --2- 172.21.15.16:0/2629734277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aac4d1840 0x556aac8a7f70 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f01d4009850 tx=0x7f01d4009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c00c4bc5d0d618 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:33.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.013+0000 7f01ce000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:14:33.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.014+0000 7f01ce000640 1 -- 172.21.15.16:0/2629734277 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d4035070 con 0x556aac4d1840 2024-08-31T06:14:33.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.014+0000 7f01ce000640 1 -- 172.21.15.16:0/2629734277 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7f01d4003a00 con 0x556aac4d1840 2024-08-31T06:14:33.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.014+0000 7f01f5fce740 1 -- 172.21.15.16:0/2629734277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01e000fc00 con 0x556aac4d1840 2024-08-31T06:14:33.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.014+0000 7f01ce000640 1 -- 172.21.15.16:0/2629734277 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f01d4004770 con 0x556aac4d1840 2024-08-31T06:14:33.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.014+0000 7f01f5fce740 10 client.17016 fetch_fsmap learned FSMap version 39 2024-08-31T06:14:33.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.014+0000 7f01f5fce740 1 -- 172.21.15.16:0/2629734277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556aac9202f0 con 0x556aac4d1840 2024-08-31T06:14:33.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.014+0000 7f01ce000640 1 -- 172.21.15.16:0/2629734277 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01e000fc00 con 0x556aac4d1840 2024-08-31T06:14:33.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.014+0000 7f01ce000640 1 -- 172.21.15.16:0/2629734277 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f01d4044040 con 0x556aac4d1840 2024-08-31T06:14:33.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.014+0000 7f01f5fce740 10 client.17016 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T06:14:33.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.014+0000 7f01f5fce740 10 client.17016 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:14:33.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.014+0000 7f01f5fce740 20 client.17016 populate_metadata read hostname 'smithi016' 2024-08-31T06:14:33.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.014+0000 7f01f5fce740 1 --2- 172.21.15.16:0/2629734277 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556aac486250 0x556aac922b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:33.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.014+0000 7f01f5fce740 4 client.17016 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2403ca55", "release": true, "await": true}] 2024-08-31T06:14:33.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.014+0000 7f01f5fce740 1 -- 172.21.15.16:0/2629734277 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2403ca55", "release": true, "await": true}) -- 0x556aac80c480 con 0x556aac486250 2024-08-31T06:14:33.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.015+0000 7f01ed400640 1 --2- 172.21.15.16:0/2629734277 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556aac486250 0x556aac922b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:33.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.015+0000 7f01ed400640 1 --2- 172.21.15.16:0/2629734277 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556aac486250 0x556aac922b90 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=123a63ec2602fa91 server_cookie=f541751263d81c8e in_seq=0 out_seq=0 2024-08-31T06:14:33.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.015+0000 7f01ce000640 10 client.17016 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:33.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:32 smithi155 ceph-mon[21742]: pgmap v587: 97 pgs: 97 active+clean; 3.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 4.9 MiB/s wr, 628 op/s 2024-08-31T06:14:33.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.185+0000 7f01ce000640 1 -- 172.21.15.16:0/2629734277 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556aac80c480 con 0x556aac486250 2024-08-31T06:14:33.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.185+0000 7f01ce000640 10 client.17016 handle_command_reply: tid=0 2024-08-31T06:14:33.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.185+0000 7f01f5fce740 1 client.17016 shutdown 2024-08-31T06:14:33.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.186+0000 7f01f5fce740 1 -- 172.21.15.16:0/2629734277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aac4d1840 msgr2=0x556aac8a7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:33.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.186+0000 7f01f5fce740 1 --2- 172.21.15.16:0/2629734277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aac4d1840 0x556aac8a7f70 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f01d4009850 tx=0x7f01d4009680 comp rx=0 tx=0).stop 2024-08-31T06:14:33.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.186+0000 7f01f5fce740 1 -- 172.21.15.16:0/2629734277 shutdown_connections 2024-08-31T06:14:33.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.186+0000 7f01f5fce740 1 --2- 172.21.15.16:0/2629734277 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556aac486250 0x556aac922b90 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:33.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.186+0000 7f01f5fce740 1 --2- 172.21.15.16:0/2629734277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556aac4f2520 0x556aac8a84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:33.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.186+0000 7f01f5fce740 1 --2- 172.21.15.16:0/2629734277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556aac4d1840 0x556aac8a7f70 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:33.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.186+0000 7f01f5fce740 1 --2- 172.21.15.16:0/2629734277 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556aac4c9660 0x556aac8a7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:33.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.186+0000 7f01f5fce740 1 -- 172.21.15.16:0/2629734277 >> 172.21.15.16:0/2629734277 conn(0x556aac4e6910 msgr2=0x556aac8996b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:33.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.186+0000 7f01ce000640 0 client.17016 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:33.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.186+0000 7f01f5fce740 1 -- 172.21.15.16:0/2629734277 shutdown_connections 2024-08-31T06:14:33.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.186+0000 7f01f5fce740 1 -- 172.21.15.16:0/2629734277 wait complete. 2024-08-31T06:14:33.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.187+0000 7f01f5fce740 20 client.17016 trim_cache size 0 max 16384 2024-08-31T06:14:33.192 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:14:33.192 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 39, 2024-08-31T06:14:33.192 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24737, 2024-08-31T06:14:33.192 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 24, 2024-08-31T06:14:33.192 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-08-31T06:14:33.193 INFO:teuthology.orchestra.run.smithi016.stdout: "2403ca55": { 2024-08-31T06:14:33.193 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 24, 2024-08-31T06:14:33.193 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-08-31T06:14:33.193 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:14:33.193 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-08-31T06:14:33.193 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:14:33.194 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:14:33.194 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-08-31T06:14:33.194 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 148.0, 2024-08-31T06:14:33.194 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-08-31T06:14:33.194 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-08-31T06:14:33.194 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-08-31T06:14:33.195 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:14:33.195 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-08-31T06:14:33.195 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:14:33.195 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:14:33.195 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:14:33.195 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:14:33.195 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:14:33.195 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:14:33.195 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:14:33.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.187+0000 7f01ef400640 1 -- 172.21.15.16:0/315044382 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f01d00b4d20 msgr2=0x7f01d00b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:33.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.188+0000 7f01ef400640 1 --2- 172.21.15.16:0/315044382 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f01d00b4d20 0x7f01d00b7230 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x556aac896b70 tx=0x7f01dc0eb530 comp rx=0 tx=0).stop 2024-08-31T06:14:33.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.188+0000 7f01ef400640 1 -- 172.21.15.16:0/315044382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01e806be80 msgr2=0x7f01e819c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:33.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.188+0000 7f01ef400640 1 --2- 172.21.15.16:0/315044382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01e806be80 0x7f01e819c440 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f01dc00ea60 tx=0x7f01dc00ef20 comp rx=0 tx=0).stop 2024-08-31T06:14:33.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.188+0000 7f01ef400640 1 -- 172.21.15.16:0/315044382 shutdown_connections 2024-08-31T06:14:33.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.188+0000 7f01ef400640 1 --2- 172.21.15.16:0/315044382 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f01d00b4d20 0x7f01d00b7230 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:33.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.188+0000 7f01ef400640 1 --2- 172.21.15.16:0/315044382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01e8104440 0x7f01e819c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:33.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.188+0000 7f01ef400640 1 --2- 172.21.15.16:0/315044382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01e810fde0 0x7f01e81a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:33.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.188+0000 7f01ef400640 1 --2- 172.21.15.16:0/315044382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01e806be80 0x7f01e819c440 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:33.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.188+0000 7f01ef400640 1 -- 172.21.15.16:0/315044382 >> 172.21.15.16:0/315044382 conn(0x7f01e8079740 msgr2=0x7f01e8101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:33.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.188+0000 7f01ef400640 1 -- 172.21.15.16:0/315044382 shutdown_connections 2024-08-31T06:14:33.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:33.188+0000 7f01ef400640 1 -- 172.21.15.16:0/315044382 wait complete. 2024-08-31T06:14:34.484 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2403ca55', total seconds elapsed: 32.7 2024-08-31T06:14:34.484 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 28.0 2024-08-31T06:14:34.484 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:14:34.723 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:34 smithi016 ceph-mon[17194]: pgmap v588: 97 pgs: 97 active+clean; 3.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 4.8 MiB/s wr, 615 op/s 2024-08-31T06:14:34.976 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:34 smithi184 ceph-mon[22331]: pgmap v588: 97 pgs: 97 active+clean; 3.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 4.8 MiB/s wr, 615 op/s 2024-08-31T06:14:35.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:34 smithi155 ceph-mon[21742]: pgmap v588: 97 pgs: 97 active+clean; 3.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 4.8 MiB/s wr, 615 op/s 2024-08-31T06:14:35.168 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:14:36.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.098+0000 7fd333600640 1 -- 172.21.15.16:0/3726457937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd32c102370 msgr2=0x7fd32c102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:36.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.098+0000 7fd333600640 1 --2- 172.21.15.16:0/3726457937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd32c102370 0x7fd32c102780 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd320009900 tx=0x7fd32002a480 comp rx=0 tx=0).stop 2024-08-31T06:14:36.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.098+0000 7fd333600640 1 -- 172.21.15.16:0/3726457937 shutdown_connections 2024-08-31T06:14:36.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.098+0000 7fd333600640 1 --2- 172.21.15.16:0/3726457937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd32c10ff20 0x7fd32c112340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:36.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.098+0000 7fd333600640 1 --2- 172.21.15.16:0/3726457937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd32c102cc0 0x7fd32c10f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:36.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.098+0000 7fd333600640 1 --2- 172.21.15.16:0/3726457937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd32c102370 0x7fd32c102780 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:36.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.098+0000 7fd333600640 1 -- 172.21.15.16:0/3726457937 >> 172.21.15.16:0/3726457937 conn(0x7fd32c0794d0 msgr2=0x7fd32c0798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:36.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.099+0000 7fd333600640 1 -- 172.21.15.16:0/3726457937 shutdown_connections 2024-08-31T06:14:36.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.099+0000 7fd333600640 1 -- 172.21.15.16:0/3726457937 wait complete. 2024-08-31T06:14:36.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.101+0000 7fd333600640 1 Processor -- start 2024-08-31T06:14:36.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.101+0000 7fd333600640 1 -- start start 2024-08-31T06:14:36.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.102+0000 7fd333600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd32c102370 0x7fd32c19c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:36.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.102+0000 7fd333600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd32c102cc0 0x7fd32c19c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:36.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.102+0000 7fd333600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd32c10ff20 0x7fd32c1a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:36.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.102+0000 7fd333600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c115a60 con 0x7fd32c102cc0 2024-08-31T06:14:36.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.102+0000 7fd333600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c1158e0 con 0x7fd32c10ff20 2024-08-31T06:14:36.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.102+0000 7fd333600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c115be0 con 0x7fd32c102370 2024-08-31T06:14:36.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.103+0000 7fd330c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd32c102370 0x7fd32c19c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:36.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.103+0000 7fd330c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd32c102370 0x7fd32c19c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:37870/0 (socket says 172.21.15.16:37870) 2024-08-31T06:14:36.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.103+0000 7fd32be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd32c102cc0 0x7fd32c19c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:36.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.103+0000 7fd330c00640 1 -- 172.21.15.16:0/3386733359 learned_addr learned my addr 172.21.15.16:0/3386733359 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:36.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.103+0000 7fd331600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd32c10ff20 0x7fd32c1a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:36.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.104+0000 7fd330c00640 1 -- 172.21.15.16:0/3386733359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd32c10ff20 msgr2=0x7fd32c1a1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:36.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.104+0000 7fd330c00640 1 --2- 172.21.15.16:0/3386733359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd32c10ff20 0x7fd32c1a1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:36.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.104+0000 7fd330c00640 1 -- 172.21.15.16:0/3386733359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd32c102cc0 msgr2=0x7fd32c19c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:36.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.104+0000 7fd330c00640 1 --2- 172.21.15.16:0/3386733359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd32c102cc0 0x7fd32c19c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:36.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.104+0000 7fd330c00640 1 -- 172.21.15.16:0/3386733359 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd32c1a2390 con 0x7fd32c102370 2024-08-31T06:14:36.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.104+0000 7fd32be00640 1 --2- 172.21.15.16:0/3386733359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd32c102cc0 0x7fd32c19c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:14:36.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.104+0000 7fd331600640 1 --2- 172.21.15.16:0/3386733359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd32c10ff20 0x7fd32c1a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:14:36.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.105+0000 7fd330c00640 1 --2- 172.21.15.16:0/3386733359 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd32c102370 0x7fd32c19c450 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd32002aa70 tx=0x7fd320003cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:36.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.105+0000 7fd329600640 1 -- 172.21.15.16:0/3386733359 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd32003f070 con 0x7fd32c102370 2024-08-31T06:14:36.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.106+0000 7fd329600640 1 -- 172.21.15.16:0/3386733359 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd320003ec0 con 0x7fd32c102370 2024-08-31T06:14:36.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.106+0000 7fd333600640 1 -- 172.21.15.16:0/3386733359 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd32c1a2620 con 0x7fd32c102370 2024-08-31T06:14:36.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.106+0000 7fd333600640 1 -- 172.21.15.16:0/3386733359 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd32c06cb80 con 0x7fd32c102370 2024-08-31T06:14:36.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.106+0000 7fd329600640 1 -- 172.21.15.16:0/3386733359 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd320035030 con 0x7fd32c102370 2024-08-31T06:14:36.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.110+0000 7fd333600640 1 -- 172.21.15.16:0/3386733359 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd30c0052c0 con 0x7fd32c102370 2024-08-31T06:14:36.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.113+0000 7fd329600640 1 -- 172.21.15.16:0/3386733359 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fd320042070 con 0x7fd32c102370 2024-08-31T06:14:36.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.115+0000 7fd329600640 1 --2- 172.21.15.16:0/3386733359 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd3140b4e70 0x7fd3140b7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:36.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.115+0000 7fd329600640 1 -- 172.21.15.16:0/3386733359 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7fd320105b70 con 0x7fd32c102370 2024-08-31T06:14:36.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.115+0000 7fd32be00640 1 --2- 172.21.15.16:0/3386733359 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd3140b4e70 0x7fd3140b7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:36.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.118+0000 7fd32be00640 1 --2- 172.21.15.16:0/3386733359 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd3140b4e70 0x7fd3140b7380 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd3180058c0 tx=0x7fd318005850 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:36.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.119+0000 7fd329600640 1 -- 172.21.15.16:0/3386733359 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fd320049030 con 0x7fd32c102370 2024-08-31T06:14:36.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.536+0000 7fd333600640 1 -- 172.21.15.16:0/3386733359 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd30c005880 con 0x7fd32c102370 2024-08-31T06:14:36.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.537+0000 7fd329600640 1 -- 172.21.15.16:0/3386733359 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11920 (secure 0 0 0) 0x7fd3200d3e20 con 0x7fd32c102370 2024-08-31T06:14:36.540 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:14:36.540 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":39,"btime":"2024-08-31T06:14:07:075654+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:14:07.025476+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24737,24724,24712]},"id":1}]} 2024-08-31T06:14:36.540 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 39 2024-08-31T06:14:36.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.540+0000 7fd333600640 1 -- 172.21.15.16:0/3386733359 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd3140b4e70 msgr2=0x7fd3140b7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:36.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.540+0000 7fd333600640 1 --2- 172.21.15.16:0/3386733359 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd3140b4e70 0x7fd3140b7380 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd3180058c0 tx=0x7fd318005850 comp rx=0 tx=0).stop 2024-08-31T06:14:36.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.540+0000 7fd333600640 1 -- 172.21.15.16:0/3386733359 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd32c102370 msgr2=0x7fd32c19c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:36.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.540+0000 7fd333600640 1 --2- 172.21.15.16:0/3386733359 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd32c102370 0x7fd32c19c450 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd32002aa70 tx=0x7fd320003cd0 comp rx=0 tx=0).stop 2024-08-31T06:14:36.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.541+0000 7fd333600640 1 -- 172.21.15.16:0/3386733359 shutdown_connections 2024-08-31T06:14:36.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.541+0000 7fd333600640 1 --2- 172.21.15.16:0/3386733359 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd3140b4e70 0x7fd3140b7380 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:36.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.541+0000 7fd333600640 1 --2- 172.21.15.16:0/3386733359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd32c10ff20 0x7fd32c1a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:36.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.541+0000 7fd333600640 1 --2- 172.21.15.16:0/3386733359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd32c102cc0 0x7fd32c19c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:36.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.541+0000 7fd333600640 1 --2- 172.21.15.16:0/3386733359 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd32c102370 0x7fd32c19c450 secure :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd32002aa70 tx=0x7fd320003cd0 comp rx=0 tx=0).stop 2024-08-31T06:14:36.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.541+0000 7fd333600640 1 -- 172.21.15.16:0/3386733359 >> 172.21.15.16:0/3386733359 conn(0x7fd32c0794d0 msgr2=0x7fd32c109ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:36.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.542+0000 7fd333600640 1 -- 172.21.15.16:0/3386733359 shutdown_connections 2024-08-31T06:14:36.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:36.542+0000 7fd333600640 1 -- 172.21.15.16:0/3386733359 wait complete. 2024-08-31T06:14:36.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 430.0 s, 1924.7 tps, lat 281.839 ms stddev 2410.975 2024-08-31T06:14:36.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:36 smithi016 ceph-mon[17194]: pgmap v589: 97 pgs: 97 active+clean; 3.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 4.6 MiB/s wr, 575 op/s 2024-08-31T06:14:36.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:36 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/3386733359' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:14:36.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:36 smithi184 ceph-mon[22331]: pgmap v589: 97 pgs: 97 active+clean; 3.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 4.6 MiB/s wr, 575 op/s 2024-08-31T06:14:36.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:36 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/3386733359' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:14:37.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:36 smithi155 ceph-mon[21742]: pgmap v589: 97 pgs: 97 active+clean; 3.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 4.6 MiB/s wr, 575 op/s 2024-08-31T06:14:37.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:36 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/3386733359' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:14:38.253 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24737 quiesce db / --timeout 90 --expiration 148.0 --await-for 45.0' 2024-08-31T06:14:38.253 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24737 quiesce db / --timeout 90 --expiration 148.0 --await-for 45.0 2024-08-31T06:14:38.911 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:14:39.075 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:38 smithi016 ceph-mon[17194]: pgmap v590: 97 pgs: 97 active+clean; 3.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 2.6 MiB/s wr, 248 op/s 2024-08-31T06:14:39.076 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:38 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:14:39.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:38 smithi155 ceph-mon[21742]: pgmap v590: 97 pgs: 97 active+clean; 3.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 2.6 MiB/s wr, 248 op/s 2024-08-31T06:14:39.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:38 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:14:39.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:38 smithi184 ceph-mon[22331]: pgmap v590: 97 pgs: 97 active+clean; 3.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 2.6 MiB/s wr, 248 op/s 2024-08-31T06:14:39.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:38 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:14:39.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.776+0000 7fc026000640 1 -- 172.21.15.16:0/2341940814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc020109bb0 msgr2=0x7fc0201103e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:39.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.776+0000 7fc026000640 1 --2- 172.21.15.16:0/2341940814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc020109bb0 0x7fc0201103e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fc018009be0 tx=0x7fc01802a6d0 comp rx=0 tx=0).stop 2024-08-31T06:14:39.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.776+0000 7fc026000640 1 -- 172.21.15.16:0/2341940814 shutdown_connections 2024-08-31T06:14:39.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.777+0000 7fc026000640 1 --2- 172.21.15.16:0/2341940814 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc020109260 0x7fc020109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:39.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.777+0000 7fc026000640 1 --2- 172.21.15.16:0/2341940814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc020109bb0 0x7fc0201103e0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:39.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.777+0000 7fc026000640 1 --2- 172.21.15.16:0/2341940814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc020103920 0x7fc020103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:39.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.777+0000 7fc026000640 1 -- 172.21.15.16:0/2341940814 >> 172.21.15.16:0/2341940814 conn(0x7fc0200ff370 msgr2=0x7fc0201017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:39.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.777+0000 7fc026000640 1 -- 172.21.15.16:0/2341940814 shutdown_connections 2024-08-31T06:14:39.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.777+0000 7fc026000640 1 -- 172.21.15.16:0/2341940814 wait complete. 2024-08-31T06:14:39.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.779+0000 7fc026000640 1 Processor -- start 2024-08-31T06:14:39.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.780+0000 7fc026000640 1 -- start start 2024-08-31T06:14:39.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.781+0000 7fc026000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc020103920 0x7fc02019c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:39.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.781+0000 7fc026000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc020109260 0x7fc02019c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:39.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.781+0000 7fc026000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc020109bb0 0x7fc0201a1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:39.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.781+0000 7fc026000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc020113870 con 0x7fc020109260 2024-08-31T06:14:39.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.781+0000 7fc026000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc0201136f0 con 0x7fc020103920 2024-08-31T06:14:39.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.781+0000 7fc026000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc0201139f0 con 0x7fc020109bb0 2024-08-31T06:14:39.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.781+0000 7fc01ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc020109260 0x7fc02019c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:39.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.781+0000 7fc01ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc020109260 0x7fc02019c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55574/0 (socket says 172.21.15.16:55574) 2024-08-31T06:14:39.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.781+0000 7fc01ea00640 1 -- 172.21.15.16:0/2126198980 learned_addr learned my addr 172.21.15.16:0/2126198980 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:39.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.782+0000 7fc01f400640 1 --2- 172.21.15.16:0/2126198980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc020103920 0x7fc02019c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:39.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.782+0000 7fc01fe00640 1 --2- 172.21.15.16:0/2126198980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc020109bb0 0x7fc0201a1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:39.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.782+0000 7fc01ea00640 1 -- 172.21.15.16:0/2126198980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc020109bb0 msgr2=0x7fc0201a1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:39.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.782+0000 7fc01ea00640 1 --2- 172.21.15.16:0/2126198980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc020109bb0 0x7fc0201a1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:39.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.782+0000 7fc01ea00640 1 -- 172.21.15.16:0/2126198980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc020103920 msgr2=0x7fc02019c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:39.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.782+0000 7fc01ea00640 1 --2- 172.21.15.16:0/2126198980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc020103920 0x7fc02019c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:39.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.782+0000 7fc01ea00640 1 -- 172.21.15.16:0/2126198980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0201a2250 con 0x7fc020109260 2024-08-31T06:14:39.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.782+0000 7fc01f400640 1 --2- 172.21.15.16:0/2126198980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc020103920 0x7fc02019c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:14:39.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.783+0000 7fc01ea00640 1 --2- 172.21.15.16:0/2126198980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc020109260 0x7fc02019c940 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc01000c8c0 tx=0x7fc01000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:39.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.783+0000 7fc017e00640 1 -- 172.21.15.16:0/2126198980 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc010007bb0 con 0x7fc020109260 2024-08-31T06:14:39.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.783+0000 7fc026000640 1 -- 172.21.15.16:0/2126198980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0201a2540 con 0x7fc020109260 2024-08-31T06:14:39.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.783+0000 7fc017e00640 1 -- 172.21.15.16:0/2126198980 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc010007d50 con 0x7fc020109260 2024-08-31T06:14:39.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.784+0000 7fc026000640 1 -- 172.21.15.16:0/2126198980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0201a8d60 con 0x7fc020109260 2024-08-31T06:14:39.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.784+0000 7fc017e00640 1 -- 172.21.15.16:0/2126198980 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0100187c0 con 0x7fc020109260 2024-08-31T06:14:39.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.792+0000 7fc02cdc9740 1 Processor -- start 2024-08-31T06:14:39.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.792+0000 7fc02cdc9740 1 -- start start 2024-08-31T06:14:39.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.793+0000 7fc02cdc9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647e40c3840 0x5647e446a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:39.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.793+0000 7fc017e00640 1 -- 172.21.15.16:0/2126198980 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fc01000fa20 con 0x7fc020109260 2024-08-31T06:14:39.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.793+0000 7fc02cdc9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647e409a980 0x5647e446a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:39.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.793+0000 7fc02cdc9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647e40a2b60 0x5647e446ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:39.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.793+0000 7fc02cdc9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5647e43df530 con 0x5647e409a980 2024-08-31T06:14:39.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.793+0000 7fc02cdc9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5647e43ceac0 con 0x5647e40c3840 2024-08-31T06:14:39.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.793+0000 7fc02cdc9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5647e43d2a70 con 0x5647e40a2b60 2024-08-31T06:14:39.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.793+0000 7fc01fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647e40c3840 0x5647e446a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:39.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.793+0000 7fc01f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647e409a980 0x5647e446a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:39.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.793+0000 7fc01fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647e40c3840 0x5647e446a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:37872/0 (socket says 172.21.15.16:37872) 2024-08-31T06:14:39.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.793+0000 7fc01fe00640 1 -- 172.21.15.16:0/2543073863 learned_addr learned my addr 172.21.15.16:0/2543073863 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:39.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.794+0000 7fc01ea00640 1 --2- 172.21.15.16:0/2543073863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647e40a2b60 0x5647e446ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:39.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.794+0000 7fc01ea00640 1 -- 172.21.15.16:0/2543073863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647e40c3840 msgr2=0x5647e446a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:39.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.794+0000 7fc01ea00640 1 --2- 172.21.15.16:0/2543073863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647e40c3840 0x5647e446a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:39.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.794+0000 7fc01ea00640 1 -- 172.21.15.16:0/2543073863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647e409a980 msgr2=0x5647e446a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:39.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.794+0000 7fc01ea00640 1 --2- 172.21.15.16:0/2543073863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647e409a980 0x5647e446a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:39.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.794+0000 7fc01ea00640 1 -- 172.21.15.16:0/2543073863 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647e4381880 con 0x5647e40a2b60 2024-08-31T06:14:39.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.795+0000 7fc01ea00640 1 --2- 172.21.15.16:0/2543073863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647e40a2b60 0x5647e446ab90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc0100e2d50 tx=0x7fc0100eb7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6297daba34d41524 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:39.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.795+0000 7fc00be00640 1 -- 172.21.15.16:0/2543073863 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0100f9070 con 0x5647e40a2b60 2024-08-31T06:14:39.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.796+0000 7fc00be00640 1 -- 172.21.15.16:0/2543073863 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0100e5df0 con 0x5647e40a2b60 2024-08-31T06:14:39.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.796+0000 7fc00be00640 1 -- 172.21.15.16:0/2543073863 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0100f19b0 con 0x5647e40a2b60 2024-08-31T06:14:39.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.796+0000 7fc017e00640 1 --2- 172.21.15.16:0/2126198980 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc0040b4d90 0x7fc0040b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:39.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.796+0000 7fc017e00640 1 -- 172.21.15.16:0/2126198980 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7fc0100e5630 con 0x7fc020109260 2024-08-31T06:14:39.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.797+0000 7fc02cdc9740 1 -- 172.21.15.16:0/2543073863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647e40a2b60 msgr2=0x5647e446ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:39.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.797+0000 7fc02cdc9740 1 --2- 172.21.15.16:0/2543073863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647e40a2b60 0x5647e446ab90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc0100e2d50 tx=0x7fc0100eb7b0 comp rx=0 tx=0).stop 2024-08-31T06:14:39.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.797+0000 7fc01fe00640 1 --2- 172.21.15.16:0/2126198980 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc0040b4d90 0x7fc0040b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:39.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.799+0000 7fc02cdc9740 1 -- 172.21.15.16:0/2543073863 shutdown_connections 2024-08-31T06:14:39.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.800+0000 7fc02cdc9740 1 --2- 172.21.15.16:0/2543073863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647e40a2b60 0x5647e446ab90 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:39.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.800+0000 7fc02cdc9740 1 --2- 172.21.15.16:0/2543073863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647e409a980 0x5647e446a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:39.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.800+0000 7fc02cdc9740 1 --2- 172.21.15.16:0/2543073863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647e40c3840 0x5647e446a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:39.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.800+0000 7fc02cdc9740 1 -- 172.21.15.16:0/2543073863 >> 172.21.15.16:0/2543073863 conn(0x5647e40b7990 msgr2=0x5647e4468000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:39.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.802+0000 7fc02cdc9740 1 -- 172.21.15.16:0/2543073863 shutdown_connections 2024-08-31T06:14:39.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.802+0000 7fc01fe00640 1 --2- 172.21.15.16:0/2126198980 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc0040b4d90 0x7fc0040b72a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc01802ac40 tx=0x7fc01802abd0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:39.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.802+0000 7fc02cdc9740 1 -- 172.21.15.16:0/2543073863 wait complete. 2024-08-31T06:14:39.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.804+0000 7fc02cdc9740 1 -- start start 2024-08-31T06:14:39.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.806+0000 7fc02cdc9740 1 Processor -- start 2024-08-31T06:14:39.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.808+0000 7fc02cdc9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647e409a980 0x5647e4475ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:39.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.808+0000 7fc02cdc9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647e40a2b60 0x5647e4476220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:39.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.808+0000 7fc01f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647e409a980 0x5647e4475ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:39.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.809+0000 7fc02cdc9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647e40c3840 0x5647e448a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:39.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.809+0000 7fc02cdc9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5647e42317a0 con 0x5647e409a980 2024-08-31T06:14:39.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.810+0000 7fc01f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647e409a980 0x5647e4475ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55604/0 (socket says 172.21.15.16:55604) 2024-08-31T06:14:39.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.810+0000 7fc01f400640 1 -- 172.21.15.16:0/229076698 learned_addr learned my addr 172.21.15.16:0/229076698 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:39.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.810+0000 7fc02cdc9740 1 -- 172.21.15.16:0/229076698 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5647e43557f0 con 0x5647e40c3840 2024-08-31T06:14:39.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.810+0000 7fc02cdc9740 1 -- 172.21.15.16:0/229076698 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5647e4481100 con 0x5647e40a2b60 2024-08-31T06:14:39.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.810+0000 7fc02cdc9740 10 client.0 _ll_register_callbacks cb 0x5647e437ae60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:14:39.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.810+0000 7fc01ea00640 1 --2- 172.21.15.16:0/229076698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647e40a2b60 0x5647e4476220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:39.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.810+0000 7fc01fe00640 1 --2- 172.21.15.16:0/229076698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647e40c3840 0x5647e448a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:39.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.810+0000 7fc01f400640 1 -- 172.21.15.16:0/229076698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647e40a2b60 msgr2=0x5647e4476220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:39.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.811+0000 7fc01f400640 1 --2- 172.21.15.16:0/229076698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647e40a2b60 0x5647e4476220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:39.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.811+0000 7fc01f400640 1 -- 172.21.15.16:0/229076698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647e40c3840 msgr2=0x5647e448a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:39.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.811+0000 7fc01f400640 1 --2- 172.21.15.16:0/229076698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647e40c3840 0x5647e448a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:39.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.811+0000 7fc01f400640 1 -- 172.21.15.16:0/229076698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647e413db90 con 0x5647e409a980 2024-08-31T06:14:39.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.811+0000 7fc01ea00640 1 --2- 172.21.15.16:0/229076698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647e40a2b60 0x5647e4476220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:14:39.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.812+0000 7fc01f400640 1 --2- 172.21.15.16:0/229076698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647e409a980 0x5647e4475ce0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc00c010ee0 tx=0x7fc00c00f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a018e66a051d8a0d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:39.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.812+0000 7fc008c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:14:39.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.815+0000 7fc008c00640 1 -- 172.21.15.16:0/229076698 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c005020 con 0x5647e409a980 2024-08-31T06:14:39.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.815+0000 7fc008c00640 1 -- 172.21.15.16:0/229076698 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7fc00c0051c0 con 0x5647e409a980 2024-08-31T06:14:39.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.815+0000 7fc02cdc9740 1 -- 172.21.15.16:0/229076698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0100e5df0 con 0x5647e409a980 2024-08-31T06:14:39.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.816+0000 7fc008c00640 1 -- 172.21.15.16:0/229076698 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc00c0053c0 con 0x5647e409a980 2024-08-31T06:14:39.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.816+0000 7fc02cdc9740 10 client.17034 fetch_fsmap learned FSMap version 39 2024-08-31T06:14:39.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.816+0000 7fc02cdc9740 1 -- 172.21.15.16:0/229076698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5647e448d140 con 0x5647e409a980 2024-08-31T06:14:39.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.821+0000 7fc008c00640 1 -- 172.21.15.16:0/229076698 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0100e5df0 con 0x5647e409a980 2024-08-31T06:14:39.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.821+0000 7fc008c00640 1 -- 172.21.15.16:0/229076698 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fc00c019610 con 0x5647e409a980 2024-08-31T06:14:39.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.821+0000 7fc02cdc9740 10 client.17034 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T06:14:39.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.822+0000 7fc02cdc9740 10 client.17034 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:14:39.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.822+0000 7fc02cdc9740 20 client.17034 populate_metadata read hostname 'smithi016' 2024-08-31T06:14:39.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.822+0000 7fc02cdc9740 1 --2- 172.21.15.16:0/229076698 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5647e40821c0 0x5647e4490750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:39.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.822+0000 7fc02cdc9740 4 client.17034 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:14:39.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.822+0000 7fc02cdc9740 1 -- 172.21.15.16:0/229076698 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647e43e1480 con 0x5647e40821c0 2024-08-31T06:14:39.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.825+0000 7fc01ea00640 1 --2- 172.21.15.16:0/229076698 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5647e40821c0 0x5647e4490750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:39.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.826+0000 7fc01ea00640 1 --2- 172.21.15.16:0/229076698 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5647e40821c0 0x5647e4490750 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ae8e7bf15ea6986 server_cookie=e32fd3f7254e6592 in_seq=0 out_seq=0 2024-08-31T06:14:39.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.826+0000 7fc008c00640 10 client.17034 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:39.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.827+0000 7fc008c00640 1 -- 172.21.15.16:0/229076698 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5647e43e1480 con 0x5647e40821c0 2024-08-31T06:14:39.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.827+0000 7fc008c00640 10 client.17034 handle_command_reply: tid=0 2024-08-31T06:14:39.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.827+0000 7fc02cdc9740 1 client.17034 shutdown 2024-08-31T06:14:39.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.828+0000 7fc02cdc9740 1 -- 172.21.15.16:0/229076698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647e409a980 msgr2=0x5647e4475ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:39.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.828+0000 7fc02cdc9740 1 --2- 172.21.15.16:0/229076698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647e409a980 0x5647e4475ce0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc00c010ee0 tx=0x7fc00c00f7f0 comp rx=0 tx=0).stop 2024-08-31T06:14:39.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.828+0000 7fc02cdc9740 1 -- 172.21.15.16:0/229076698 shutdown_connections 2024-08-31T06:14:39.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.828+0000 7fc02cdc9740 1 --2- 172.21.15.16:0/229076698 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5647e40821c0 0x5647e4490750 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:39.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.828+0000 7fc008c00640 0 client.17034 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:39.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.828+0000 7fc02cdc9740 1 --2- 172.21.15.16:0/229076698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647e40a2b60 0x5647e4476220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:39.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.828+0000 7fc02cdc9740 1 --2- 172.21.15.16:0/229076698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647e40c3840 0x5647e448a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:39.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.828+0000 7fc02cdc9740 1 --2- 172.21.15.16:0/229076698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647e409a980 0x5647e4475ce0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:39.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.828+0000 7fc02cdc9740 1 -- 172.21.15.16:0/229076698 >> 172.21.15.16:0/229076698 conn(0x5647e40b7990 msgr2=0x5647e44667e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:39.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.829+0000 7fc02cdc9740 1 -- 172.21.15.16:0/229076698 shutdown_connections 2024-08-31T06:14:39.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.829+0000 7fc02cdc9740 1 -- 172.21.15.16:0/229076698 wait complete. 2024-08-31T06:14:39.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.831+0000 7fc02cdc9740 20 client.17034 trim_cache size 0 max 16384 2024-08-31T06:14:39.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.871+0000 7fc02cdc9740 1 Processor -- start 2024-08-31T06:14:39.874 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:39 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:14:39.874 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:39 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:14:39.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.871+0000 7fc02cdc9740 1 -- start start 2024-08-31T06:14:39.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.873+0000 7fc02cdc9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647e409a980 0x5647e4474360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:39.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.873+0000 7fc02cdc9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647e40a2b60 0x5647e446f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:39.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.873+0000 7fc02cdc9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647e40c3840 0x5647e446f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:39.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.873+0000 7fc02cdc9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc0201afc50 con 0x5647e409a980 2024-08-31T06:14:39.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.873+0000 7fc02cdc9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc0201ad4c0 con 0x5647e40c3840 2024-08-31T06:14:39.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.873+0000 7fc02cdc9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc0201ab350 con 0x5647e40a2b60 2024-08-31T06:14:39.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.874+0000 7fc01f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647e409a980 0x5647e4474360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:39.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.874+0000 7fc01ea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647e40a2b60 0x5647e446f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:39.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.874+0000 7fc01ea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647e40a2b60 0x5647e446f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:37908/0 (socket says 172.21.15.16:37908) 2024-08-31T06:14:39.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.874+0000 7fc01ea00640 1 -- 172.21.15.16:0/2956645041 learned_addr learned my addr 172.21.15.16:0/2956645041 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:39.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.874+0000 7fc01fe00640 1 --2- 172.21.15.16:0/2956645041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647e40c3840 0x5647e446f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:39.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.874+0000 7fc01fe00640 1 -- 172.21.15.16:0/2956645041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647e40a2b60 msgr2=0x5647e446f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:39.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.874+0000 7fc01fe00640 1 --2- 172.21.15.16:0/2956645041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647e40a2b60 0x5647e446f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:39.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.874+0000 7fc01fe00640 1 -- 172.21.15.16:0/2956645041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647e409a980 msgr2=0x5647e4474360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:39.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.874+0000 7fc01fe00640 1 --2- 172.21.15.16:0/2956645041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647e409a980 0x5647e4474360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:39.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.875+0000 7fc01fe00640 1 -- 172.21.15.16:0/2956645041 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647e4478220 con 0x5647e40c3840 2024-08-31T06:14:39.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.875+0000 7fc01ea00640 1 --2- 172.21.15.16:0/2956645041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647e40a2b60 0x5647e446f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:14:39.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.875+0000 7fc01fe00640 1 --2- 172.21.15.16:0/2956645041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647e40c3840 0x5647e446f760 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc018006010 tx=0x7fc01803f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8112a280d88d0a95 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:39.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.875+0000 7fc00b400640 1 -- 172.21.15.16:0/2956645041 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc018050070 con 0x5647e40c3840 2024-08-31T06:14:39.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.876+0000 7fc00b400640 1 -- 172.21.15.16:0/2956645041 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc01804cb10 con 0x5647e40c3840 2024-08-31T06:14:39.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.876+0000 7fc00b400640 1 -- 172.21.15.16:0/2956645041 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc018050070 con 0x5647e40c3840 2024-08-31T06:14:39.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.877+0000 7fc02cdc9740 1 -- 172.21.15.16:0/2956645041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647e40c3840 msgr2=0x5647e446f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:39.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.877+0000 7fc02cdc9740 1 --2- 172.21.15.16:0/2956645041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647e40c3840 0x5647e446f760 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc018006010 tx=0x7fc01803f000 comp rx=0 tx=0).stop 2024-08-31T06:14:39.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.881+0000 7fc02cdc9740 1 -- 172.21.15.16:0/2956645041 shutdown_connections 2024-08-31T06:14:39.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.881+0000 7fc02cdc9740 1 --2- 172.21.15.16:0/2956645041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647e40a2b60 0x5647e446f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:39.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.881+0000 7fc02cdc9740 1 --2- 172.21.15.16:0/2956645041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647e40c3840 0x5647e446f760 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:39.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.881+0000 7fc02cdc9740 1 --2- 172.21.15.16:0/2956645041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647e409a980 0x5647e4474360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:39.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.881+0000 7fc02cdc9740 1 -- 172.21.15.16:0/2956645041 >> 172.21.15.16:0/2956645041 conn(0x5647e40b7990 msgr2=0x5647e4470c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:39.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.881+0000 7fc02cdc9740 1 -- 172.21.15.16:0/2956645041 shutdown_connections 2024-08-31T06:14:39.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.882+0000 7fc02cdc9740 1 -- 172.21.15.16:0/2956645041 wait complete. 2024-08-31T06:14:39.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.882+0000 7fc02cdc9740 1 -- start start 2024-08-31T06:14:39.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.883+0000 7fc02cdc9740 1 Processor -- start 2024-08-31T06:14:39.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.884+0000 7fc02cdc9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647e409a980 0x5647e4486a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:39.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.885+0000 7fc02cdc9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647e40a2b60 0x5647e4482ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:39.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.885+0000 7fc01ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647e40a2b60 0x5647e4482ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:39.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.885+0000 7fc01ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647e40a2b60 0x5647e4482ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:37904/0 (socket says 172.21.15.16:37904) 2024-08-31T06:14:39.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.885+0000 7fc02cdc9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647e40c3840 0x5647e4483430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:39.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.885+0000 7fc01ea00640 1 -- 172.21.15.16:0/4088074618 learned_addr learned my addr 172.21.15.16:0/4088074618 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:39.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.886+0000 7fc01f400640 1 --2- 172.21.15.16:0/4088074618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647e409a980 0x5647e4486a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:39.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.886+0000 7fc01fe00640 1 --2- 172.21.15.16:0/4088074618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647e40c3840 0x5647e4483430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:39.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.886+0000 7fc02cdc9740 1 -- 172.21.15.16:0/4088074618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc0201ab9a0 con 0x5647e409a980 2024-08-31T06:14:39.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.886+0000 7fc02cdc9740 1 -- 172.21.15.16:0/4088074618 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc0201acdf0 con 0x5647e40a2b60 2024-08-31T06:14:39.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.886+0000 7fc02cdc9740 1 -- 172.21.15.16:0/4088074618 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc0201ac050 con 0x5647e40c3840 2024-08-31T06:14:39.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.887+0000 7fc02cdc9740 10 client.0 _ll_register_callbacks cb 0x7fc00c010b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:14:39.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.887+0000 7fc01ea00640 1 -- 172.21.15.16:0/4088074618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647e40c3840 msgr2=0x5647e4483430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.887+0000 7fc01ea00640 1 --2- 172.21.15.16:0/4088074618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647e40c3840 0x5647e4483430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.887+0000 7fc01ea00640 1 -- 172.21.15.16:0/4088074618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647e409a980 msgr2=0x5647e4486a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.887+0000 7fc01ea00640 1 --2- 172.21.15.16:0/4088074618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647e409a980 0x5647e4486a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.887+0000 7fc01ea00640 1 -- 172.21.15.16:0/4088074618 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647e4483aa0 con 0x5647e40a2b60 2024-08-31T06:14:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.888+0000 7fc01fe00640 1 --2- 172.21.15.16:0/4088074618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647e40c3840 0x5647e4483430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:14:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.888+0000 7fc01f400640 1 --2- 172.21.15.16:0/4088074618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647e409a980 0x5647e4486a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:14:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.888+0000 7fc008c00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:14:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.889+0000 7fc01ea00640 1 --2- 172.21.15.16:0/4088074618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647e40a2b60 0x5647e4482ef0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc0100a7d70 tx=0x7fc0100f1510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef1cf435ef5a0104 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:39.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.889+0000 7fc008c00640 1 -- 172.21.15.16:0/4088074618 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0100a7bb0 con 0x5647e40a2b60 2024-08-31T06:14:39.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.889+0000 7fc02cdc9740 1 -- 172.21.15.16:0/4088074618 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc01804cb10 con 0x5647e40a2b60 2024-08-31T06:14:39.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.889+0000 7fc008c00640 1 -- 172.21.15.16:0/4088074618 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7fc0100930b0 con 0x5647e40a2b60 2024-08-31T06:14:39.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.890+0000 7fc008c00640 1 -- 172.21.15.16:0/4088074618 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc0100ab460 con 0x5647e40a2b60 2024-08-31T06:14:39.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.890+0000 7fc02cdc9740 10 client.27269 fetch_fsmap learned FSMap version 39 2024-08-31T06:14:39.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.890+0000 7fc02cdc9740 1 -- 172.21.15.16:0/4088074618 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5647e44fbf00 con 0x5647e40a2b60 2024-08-31T06:14:39.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.891+0000 7fc008c00640 1 -- 172.21.15.16:0/4088074618 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc01804cb10 con 0x5647e40a2b60 2024-08-31T06:14:39.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.891+0000 7fc008c00640 1 -- 172.21.15.16:0/4088074618 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fc0100eb150 con 0x5647e40a2b60 2024-08-31T06:14:39.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.893+0000 7fc02cdc9740 10 client.27269 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T06:14:39.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.893+0000 7fc02cdc9740 10 client.27269 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:14:39.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.893+0000 7fc02cdc9740 20 client.27269 populate_metadata read hostname 'smithi016' 2024-08-31T06:14:39.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.893+0000 7fc02cdc9740 1 --2- 172.21.15.16:0/4088074618 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5647e4057570 0x5647e44fe140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:39.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.893+0000 7fc02cdc9740 4 client.27269 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 148.0, "await_for": 45.0}] 2024-08-31T06:14:39.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.893+0000 7fc02cdc9740 1 -- 172.21.15.16:0/4088074618 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 148.0, "await_for": 45.0}) -- 0x5647e43dcf20 con 0x5647e4057570 2024-08-31T06:14:39.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.893+0000 7fc01f400640 1 --2- 172.21.15.16:0/4088074618 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5647e4057570 0x5647e44fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:39.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.894+0000 7fc01f400640 1 --2- 172.21.15.16:0/4088074618 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5647e4057570 0x5647e44fe140 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e187a3d49a0ed266 server_cookie=5b21b72e987d64aa in_seq=0 out_seq=0 2024-08-31T06:14:39.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:39.894+0000 7fc008c00640 10 client.27269 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:40.022 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:39 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:14:40.022 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:39 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:14:40.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:40.113+0000 7fc008c00640 1 -- 172.21.15.16:0/4088074618 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5647e43dcf20 con 0x5647e4057570 2024-08-31T06:14:40.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:40.113+0000 7fc008c00640 10 client.27269 handle_command_reply: tid=0 2024-08-31T06:14:40.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:40.113+0000 7fc02cdc9740 1 client.27269 shutdown 2024-08-31T06:14:40.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:40.114+0000 7fc02cdc9740 1 -- 172.21.15.16:0/4088074618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647e40a2b60 msgr2=0x5647e4482ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:40.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:40.114+0000 7fc02cdc9740 1 --2- 172.21.15.16:0/4088074618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647e40a2b60 0x5647e4482ef0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc0100a7d70 tx=0x7fc0100f1510 comp rx=0 tx=0).stop 2024-08-31T06:14:40.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:40.114+0000 7fc02cdc9740 1 -- 172.21.15.16:0/4088074618 shutdown_connections 2024-08-31T06:14:40.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:40.114+0000 7fc02cdc9740 1 --2- 172.21.15.16:0/4088074618 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5647e4057570 0x5647e44fe140 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:40.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:40.114+0000 7fc02cdc9740 1 --2- 172.21.15.16:0/4088074618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647e40c3840 0x5647e4483430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:40.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:40.114+0000 7fc02cdc9740 1 --2- 172.21.15.16:0/4088074618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647e40a2b60 0x5647e4482ef0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:40.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:40.114+0000 7fc02cdc9740 1 --2- 172.21.15.16:0/4088074618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647e409a980 0x5647e4486a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:40.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:40.114+0000 7fc008c00640 0 client.27269 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:40.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:40.114+0000 7fc02cdc9740 1 -- 172.21.15.16:0/4088074618 >> 172.21.15.16:0/4088074618 conn(0x5647e40b7990 msgr2=0x5647e4470c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:40.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:40.114+0000 7fc02cdc9740 1 -- 172.21.15.16:0/4088074618 shutdown_connections 2024-08-31T06:14:40.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:40.114+0000 7fc02cdc9740 1 -- 172.21.15.16:0/4088074618 wait complete. 2024-08-31T06:14:40.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:40.114+0000 7fc02cdc9740 20 client.27269 trim_cache size 0 max 16384 2024-08-31T06:14:40.118 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:14:40.118 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 39, 2024-08-31T06:14:40.118 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24737, 2024-08-31T06:14:40.118 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 26, 2024-08-31T06:14:40.118 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-08-31T06:14:40.118 INFO:teuthology.orchestra.run.smithi016.stdout: "cf5ba332": { 2024-08-31T06:14:40.118 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 26, 2024-08-31T06:14:40.118 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-08-31T06:14:40.118 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:14:40.118 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-08-31T06:14:40.120 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:14:40.120 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:14:40.120 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-08-31T06:14:40.120 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 148.0, 2024-08-31T06:14:40.120 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-08-31T06:14:40.120 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-08-31T06:14:40.120 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-08-31T06:14:40.120 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:14:40.121 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-08-31T06:14:40.121 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:14:40.121 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:14:40.121 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:14:40.121 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:14:40.121 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:14:40.121 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:14:40.121 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:14:40.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:40.115+0000 7fc026000640 1 -- 172.21.15.16:0/2126198980 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc0040b4d90 msgr2=0x7fc0040b72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:40.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:40.115+0000 7fc026000640 1 --2- 172.21.15.16:0/2126198980 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc0040b4d90 0x7fc0040b72a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc01802ac40 tx=0x7fc01802abd0 comp rx=0 tx=0).stop 2024-08-31T06:14:40.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:40.115+0000 7fc026000640 1 -- 172.21.15.16:0/2126198980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc020109260 msgr2=0x7fc02019c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:40.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:40.115+0000 7fc026000640 1 --2- 172.21.15.16:0/2126198980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc020109260 0x7fc02019c940 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc01000c8c0 tx=0x7fc01000cd80 comp rx=0 tx=0).stop 2024-08-31T06:14:40.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:40.115+0000 7fc026000640 1 -- 172.21.15.16:0/2126198980 shutdown_connections 2024-08-31T06:14:40.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:40.115+0000 7fc026000640 1 --2- 172.21.15.16:0/2126198980 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc0040b4d90 0x7fc0040b72a0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:40.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:40.115+0000 7fc026000640 1 --2- 172.21.15.16:0/2126198980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc020109bb0 0x7fc0201a1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:40.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:40.115+0000 7fc026000640 1 --2- 172.21.15.16:0/2126198980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc020109260 0x7fc02019c940 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:40.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:40.115+0000 7fc026000640 1 --2- 172.21.15.16:0/2126198980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc020103920 0x7fc02019c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:40.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:40.115+0000 7fc026000640 1 -- 172.21.15.16:0/2126198980 >> 172.21.15.16:0/2126198980 conn(0x7fc0200ff370 msgr2=0x7fc0201067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:40.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:40.115+0000 7fc026000640 1 -- 172.21.15.16:0/2126198980 shutdown_connections 2024-08-31T06:14:40.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:40.115+0000 7fc026000640 1 -- 172.21.15.16:0/2126198980 wait complete. 2024-08-31T06:14:40.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:39 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:14:40.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:39 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:14:40.899 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:40 smithi016 ceph-mon[17194]: pgmap v591: 97 pgs: 97 active+clean; 3.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 3.2 MiB/s wr, 287 op/s 2024-08-31T06:14:40.939 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'cf5ba332', quiesce took 6.5 seconds. Will release after: 21.5 2024-08-31T06:14:41.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:40 smithi155 ceph-mon[21742]: pgmap v591: 97 pgs: 97 active+clean; 3.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 3.2 MiB/s wr, 287 op/s 2024-08-31T06:14:41.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:40 smithi184 ceph-mon[22331]: pgmap v591: 97 pgs: 97 active+clean; 3.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 3.2 MiB/s wr, 287 op/s 2024-08-31T06:14:41.626 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 435.0 s, 1336.8 tps, lat 49.651 ms stddev 53.368 2024-08-31T06:14:42.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:41 smithi155 ceph-mon[21742]: pgmap v592: 97 pgs: 97 active+clean; 3.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 3.8 MiB/s wr, 359 op/s 2024-08-31T06:14:42.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:41 smithi016 ceph-mon[17194]: pgmap v592: 97 pgs: 97 active+clean; 3.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 3.8 MiB/s wr, 359 op/s 2024-08-31T06:14:42.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:41 smithi184 ceph-mon[22331]: pgmap v592: 97 pgs: 97 active+clean; 3.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 3.8 MiB/s wr, 359 op/s 2024-08-31T06:14:44.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:44 smithi155 ceph-mon[21742]: pgmap v593: 97 pgs: 97 active+clean; 3.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 4.2 MiB/s wr, 407 op/s 2024-08-31T06:14:44.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:44 smithi016 ceph-mon[17194]: pgmap v593: 97 pgs: 97 active+clean; 3.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 4.2 MiB/s wr, 407 op/s 2024-08-31T06:14:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.930+0000 7fc77a800640 1 -- 172.21.15.16:0/3942726914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7741036b0 msgr2=0x7fc774103ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.930+0000 7fc77a800640 1 --2- 172.21.15.16:0/3942726914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7741036b0 0x7fc774103ac0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc768009900 tx=0x7fc76802a410 comp rx=0 tx=0).stop 2024-08-31T06:14:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.931+0000 7fc77a800640 1 -- 172.21.15.16:0/3942726914 shutdown_connections 2024-08-31T06:14:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.931+0000 7fc77a800640 1 --2- 172.21.15.16:0/3942726914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc774108fe0 0x7fc7741093f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.931+0000 7fc77a800640 1 --2- 172.21.15.16:0/3942726914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc774109930 0x7fc774110160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.931+0000 7fc77a800640 1 --2- 172.21.15.16:0/3942726914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7741036b0 0x7fc774103ac0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.931+0000 7fc77a800640 1 -- 172.21.15.16:0/3942726914 >> 172.21.15.16:0/3942726914 conn(0x7fc7740ff0d0 msgr2=0x7fc774101520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.931+0000 7fc77a800640 1 -- 172.21.15.16:0/3942726914 shutdown_connections 2024-08-31T06:14:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.931+0000 7fc77a800640 1 -- 172.21.15.16:0/3942726914 wait complete. 2024-08-31T06:14:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.932+0000 7fc77a800640 1 Processor -- start 2024-08-31T06:14:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.932+0000 7fc77a800640 1 -- start start 2024-08-31T06:14:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.933+0000 7fc77a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7741036b0 0x7fc77419c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.933+0000 7fc77a800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc774108fe0 0x7fc77419c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.933+0000 7fc77a800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc774109930 0x7fc7741a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.933+0000 7fc77a800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc774113650 con 0x7fc7741036b0 2024-08-31T06:14:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.933+0000 7fc77a800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc7741134d0 con 0x7fc774109930 2024-08-31T06:14:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.933+0000 7fc77a800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc7741137d0 con 0x7fc774108fe0 2024-08-31T06:14:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.933+0000 7fc779400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7741036b0 0x7fc77419c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.933+0000 7fc778a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc774108fe0 0x7fc77419c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.933+0000 7fc779400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7741036b0 0x7fc77419c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55646/0 (socket says 172.21.15.16:55646) 2024-08-31T06:14:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.933+0000 7fc779400640 1 -- 172.21.15.16:0/1291764140 learned_addr learned my addr 172.21.15.16:0/1291764140 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.933+0000 7fc779e00640 1 --2- 172.21.15.16:0/1291764140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc774109930 0x7fc7741a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.933+0000 7fc779400640 1 -- 172.21.15.16:0/1291764140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc774108fe0 msgr2=0x7fc77419c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.933+0000 7fc779400640 1 --2- 172.21.15.16:0/1291764140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc774108fe0 0x7fc77419c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.933+0000 7fc779400640 1 -- 172.21.15.16:0/1291764140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc774109930 msgr2=0x7fc7741a1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.933+0000 7fc779400640 1 --2- 172.21.15.16:0/1291764140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc774109930 0x7fc7741a1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.933+0000 7fc779400640 1 -- 172.21.15.16:0/1291764140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7741a21e0 con 0x7fc7741036b0 2024-08-31T06:14:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.933+0000 7fc778a00640 1 --2- 172.21.15.16:0/1291764140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc774108fe0 0x7fc77419c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:14:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.934+0000 7fc779400640 1 --2- 172.21.15.16:0/1291764140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7741036b0 0x7fc77419c2b0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc768005ab0 tx=0x7fc768003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.934+0000 7fc772000640 1 -- 172.21.15.16:0/1291764140 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76803f070 con 0x7fc7741036b0 2024-08-31T06:14:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.934+0000 7fc77a800640 1 -- 172.21.15.16:0/1291764140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7741a2470 con 0x7fc7741036b0 2024-08-31T06:14:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.934+0000 7fc772000640 1 -- 172.21.15.16:0/1291764140 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc768031b60 con 0x7fc7741036b0 2024-08-31T06:14:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.934+0000 7fc772000640 1 -- 172.21.15.16:0/1291764140 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc768035030 con 0x7fc7741036b0 2024-08-31T06:14:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.934+0000 7fc77a800640 1 -- 172.21.15.16:0/1291764140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7741a8c60 con 0x7fc7741036b0 2024-08-31T06:14:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.937+0000 7fc782e42740 1 Processor -- start 2024-08-31T06:14:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.937+0000 7fc782e42740 1 -- start start 2024-08-31T06:14:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.938+0000 7fc782e42740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eace0851b0 0x55eace4288a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.938+0000 7fc782e42740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eace05c2f0 0x55eace428de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.938+0000 7fc772000640 1 -- 172.21.15.16:0/1291764140 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fc768042070 con 0x7fc7741036b0 2024-08-31T06:14:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.938+0000 7fc782e42740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eace0644d0 0x55eace429320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.938+0000 7fc782e42740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55eace39df70 con 0x55eace0851b0 2024-08-31T06:14:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.938+0000 7fc782e42740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55eace38d500 con 0x55eace05c2f0 2024-08-31T06:14:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.938+0000 7fc782e42740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55eace3914b0 con 0x55eace0644d0 2024-08-31T06:14:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.938+0000 7fc779e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eace0851b0 0x55eace4288a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.938+0000 7fc779e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eace0851b0 0x55eace4288a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55656/0 (socket says 172.21.15.16:55656) 2024-08-31T06:14:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.938+0000 7fc779e00640 1 -- 172.21.15.16:0/2877258421 learned_addr learned my addr 172.21.15.16:0/2877258421 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.938+0000 7fc779400640 1 --2- 172.21.15.16:0/2877258421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eace05c2f0 0x55eace428de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.938+0000 7fc778a00640 1 --2- 172.21.15.16:0/2877258421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eace0644d0 0x55eace429320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.938+0000 7fc779e00640 1 -- 172.21.15.16:0/2877258421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eace0644d0 msgr2=0x55eace429320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.938+0000 7fc779e00640 1 --2- 172.21.15.16:0/2877258421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eace0644d0 0x55eace429320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.938+0000 7fc779e00640 1 -- 172.21.15.16:0/2877258421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eace05c2f0 msgr2=0x55eace428de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.938+0000 7fc779e00640 1 --2- 172.21.15.16:0/2877258421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eace05c2f0 0x55eace428de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.938+0000 7fc779e00640 1 -- 172.21.15.16:0/2877258421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eace347c10 con 0x55eace0851b0 2024-08-31T06:14:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.938+0000 7fc779400640 1 --2- 172.21.15.16:0/2877258421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eace05c2f0 0x55eace428de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:14:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.939+0000 7fc779e00640 1 --2- 172.21.15.16:0/2877258421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eace0851b0 0x55eace4288a0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc76c009fd0 tx=0x7fc76c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=125ca79fb88fa8bd server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.939+0000 7fc766600640 1 -- 172.21.15.16:0/2877258421 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76c004d20 con 0x55eace0851b0 2024-08-31T06:14:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.939+0000 7fc766600640 1 -- 172.21.15.16:0/2877258421 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc76c004050 con 0x55eace0851b0 2024-08-31T06:14:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.939+0000 7fc766600640 1 -- 172.21.15.16:0/2877258421 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76c005220 con 0x55eace0851b0 2024-08-31T06:14:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.939+0000 7fc772000640 1 --2- 172.21.15.16:0/1291764140 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc75c0b4ca0 0x7fc75c0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.940+0000 7fc772000640 1 -- 172.21.15.16:0/1291764140 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7fc768105940 con 0x7fc7741036b0 2024-08-31T06:14:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.940+0000 7fc779400640 1 --2- 172.21.15.16:0/1291764140 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc75c0b4ca0 0x7fc75c0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.940+0000 7fc782e42740 1 -- 172.21.15.16:0/2877258421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eace0851b0 msgr2=0x55eace4288a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.940+0000 7fc782e42740 1 --2- 172.21.15.16:0/2877258421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eace0851b0 0x55eace4288a0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc76c009fd0 tx=0x7fc76c0118d0 comp rx=0 tx=0).stop 2024-08-31T06:14:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.940+0000 7fc782e42740 1 -- 172.21.15.16:0/2877258421 shutdown_connections 2024-08-31T06:14:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.940+0000 7fc782e42740 1 --2- 172.21.15.16:0/2877258421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eace0644d0 0x55eace429320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.940+0000 7fc782e42740 1 --2- 172.21.15.16:0/2877258421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eace05c2f0 0x55eace428de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.940+0000 7fc782e42740 1 --2- 172.21.15.16:0/2877258421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eace0851b0 0x55eace4288a0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.940+0000 7fc782e42740 1 -- 172.21.15.16:0/2877258421 >> 172.21.15.16:0/2877258421 conn(0x55eace079300 msgr2=0x55eace426790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.940+0000 7fc782e42740 1 -- 172.21.15.16:0/2877258421 shutdown_connections 2024-08-31T06:14:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.940+0000 7fc782e42740 1 -- 172.21.15.16:0/2877258421 wait complete. 2024-08-31T06:14:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.940+0000 7fc779400640 1 --2- 172.21.15.16:0/1291764140 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc75c0b4ca0 0x7fc75c0b71b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x55eace427520 tx=0x7fc768031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.941+0000 7fc782e42740 1 -- start start 2024-08-31T06:14:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.941+0000 7fc782e42740 1 Processor -- start 2024-08-31T06:14:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.942+0000 7fc782e42740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eace05c2f0 0x55eace433e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.942+0000 7fc782e42740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eace0644d0 0x55eace434360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.942+0000 7fc782e42740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eace0851b0 0x55eace4348a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.942+0000 7fc782e42740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55eace2c6ab0 con 0x55eace05c2f0 2024-08-31T06:14:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.942+0000 7fc782e42740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55eace334320 con 0x55eace0644d0 2024-08-31T06:14:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.942+0000 7fc782e42740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55eace437020 con 0x55eace0851b0 2024-08-31T06:14:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.942+0000 7fc782e42740 10 client.0 _ll_register_callbacks cb 0x55eace36de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:14:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.942+0000 7fc779e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eace05c2f0 0x55eace433e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.942+0000 7fc779e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eace05c2f0 0x55eace433e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55662/0 (socket says 172.21.15.16:55662) 2024-08-31T06:14:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.942+0000 7fc779e00640 1 -- 172.21.15.16:0/414968961 learned_addr learned my addr 172.21.15.16:0/414968961 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.942+0000 7fc778a00640 1 --2- 172.21.15.16:0/414968961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eace0644d0 0x55eace434360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.942+0000 7fc779e00640 1 --2- 172.21.15.16:0/414968961 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eace0851b0 0x55eace4348a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.942+0000 7fc779e00640 1 -- 172.21.15.16:0/414968961 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eace0851b0 msgr2=0x55eace4348a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.942+0000 7fc779e00640 1 --2- 172.21.15.16:0/414968961 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eace0851b0 0x55eace4348a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.942+0000 7fc779e00640 1 -- 172.21.15.16:0/414968961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eace0644d0 msgr2=0x55eace434360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.942+0000 7fc779e00640 1 --2- 172.21.15.16:0/414968961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eace0644d0 0x55eace434360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.942+0000 7fc779e00640 1 -- 172.21.15.16:0/414968961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eace0f0790 con 0x55eace05c2f0 2024-08-31T06:14:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.943+0000 7fc779e00640 1 --2- 172.21.15.16:0/414968961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eace05c2f0 0x55eace433e20 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc76c000f80 tx=0x7fc76c005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ee9e2c5dea8a9a9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.943+0000 7fc75aa00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:14:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.943+0000 7fc75aa00640 1 -- 172.21.15.16:0/414968961 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76c004e50 con 0x55eace05c2f0 2024-08-31T06:14:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.943+0000 7fc782e42740 1 -- 172.21.15.16:0/414968961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc76c004050 con 0x55eace05c2f0 2024-08-31T06:14:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.943+0000 7fc75aa00640 1 -- 172.21.15.16:0/414968961 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7fc76c005040 con 0x55eace05c2f0 2024-08-31T06:14:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.943+0000 7fc75aa00640 1 -- 172.21.15.16:0/414968961 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc76c035050 con 0x55eace05c2f0 2024-08-31T06:14:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.943+0000 7fc782e42740 10 client.17052 fetch_fsmap learned FSMap version 39 2024-08-31T06:14:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.943+0000 7fc782e42740 1 -- 172.21.15.16:0/414968961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55eace169620 con 0x55eace05c2f0 2024-08-31T06:14:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.943+0000 7fc75aa00640 1 -- 172.21.15.16:0/414968961 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc76c004050 con 0x55eace05c2f0 2024-08-31T06:14:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.944+0000 7fc75aa00640 1 -- 172.21.15.16:0/414968961 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fc76c00f850 con 0x55eace05c2f0 2024-08-31T06:14:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.944+0000 7fc782e42740 10 client.17052 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T06:14:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.944+0000 7fc782e42740 10 client.17052 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:14:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.944+0000 7fc782e42740 20 client.17052 populate_metadata read hostname 'smithi016' 2024-08-31T06:14:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.944+0000 7fc782e42740 1 --2- 172.21.15.16:0/414968961 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55eace043d80 0x55eace445f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.944+0000 7fc782e42740 4 client.17052 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:14:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.944+0000 7fc782e42740 1 -- 172.21.15.16:0/414968961 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eace39fec0 con 0x55eace043d80 2024-08-31T06:14:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.944+0000 7fc778a00640 1 --2- 172.21.15.16:0/414968961 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55eace043d80 0x55eace445f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.945+0000 7fc778a00640 1 --2- 172.21.15.16:0/414968961 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55eace043d80 0x55eace445f00 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42ce95ee06716bec server_cookie=ff8c432a24ee020d in_seq=0 out_seq=0 2024-08-31T06:14:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.945+0000 7fc75aa00640 10 client.17052 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.946+0000 7fc75aa00640 1 -- 172.21.15.16:0/414968961 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55eace39fec0 con 0x55eace043d80 2024-08-31T06:14:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.946+0000 7fc75aa00640 10 client.17052 handle_command_reply: tid=0 2024-08-31T06:14:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.946+0000 7fc782e42740 1 client.17052 shutdown 2024-08-31T06:14:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.946+0000 7fc782e42740 1 -- 172.21.15.16:0/414968961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eace05c2f0 msgr2=0x55eace433e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.946+0000 7fc782e42740 1 --2- 172.21.15.16:0/414968961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eace05c2f0 0x55eace433e20 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc76c000f80 tx=0x7fc76c005bd0 comp rx=0 tx=0).stop 2024-08-31T06:14:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.946+0000 7fc782e42740 1 -- 172.21.15.16:0/414968961 shutdown_connections 2024-08-31T06:14:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.946+0000 7fc782e42740 1 --2- 172.21.15.16:0/414968961 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55eace043d80 0x55eace445f00 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.946+0000 7fc782e42740 1 --2- 172.21.15.16:0/414968961 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eace0851b0 0x55eace4348a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.946+0000 7fc782e42740 1 --2- 172.21.15.16:0/414968961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eace0644d0 0x55eace434360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.946+0000 7fc782e42740 1 --2- 172.21.15.16:0/414968961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eace05c2f0 0x55eace433e20 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.946+0000 7fc75aa00640 0 client.17052 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.946+0000 7fc782e42740 1 -- 172.21.15.16:0/414968961 >> 172.21.15.16:0/414968961 conn(0x55eace079300 msgr2=0x55eace4298d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.946+0000 7fc782e42740 1 -- 172.21.15.16:0/414968961 shutdown_connections 2024-08-31T06:14:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.947+0000 7fc782e42740 1 -- 172.21.15.16:0/414968961 wait complete. 2024-08-31T06:14:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.947+0000 7fc782e42740 20 client.17052 trim_cache size 0 max 16384 2024-08-31T06:14:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.962+0000 7fc782e42740 1 Processor -- start 2024-08-31T06:14:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.962+0000 7fc782e42740 1 -- start start 2024-08-31T06:14:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.962+0000 7fc782e42740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eace05c2f0 0x55eace435a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.962+0000 7fc782e42740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eace0644d0 0x55eace430ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.962+0000 7fc782e42740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eace0851b0 0x55eace431530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.962+0000 7fc782e42740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc7741b3dd0 con 0x55eace0851b0 2024-08-31T06:14:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.962+0000 7fc782e42740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc7741b1640 con 0x55eace0644d0 2024-08-31T06:14:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.962+0000 7fc782e42740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc7741af4b0 con 0x55eace05c2f0 2024-08-31T06:14:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.963+0000 7fc779e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eace0851b0 0x55eace431530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.963+0000 7fc778a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eace0644d0 0x55eace430ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.963+0000 7fc779e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eace0851b0 0x55eace431530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55666/0 (socket says 172.21.15.16:55666) 2024-08-31T06:14:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.963+0000 7fc779e00640 1 -- 172.21.15.16:0/4222237474 learned_addr learned my addr 172.21.15.16:0/4222237474 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.963+0000 7fc779e00640 1 --2- 172.21.15.16:0/4222237474 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eace05c2f0 0x55eace435a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.963+0000 7fc778a00640 1 -- 172.21.15.16:0/4222237474 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eace05c2f0 msgr2=0x55eace435a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.963+0000 7fc778a00640 1 --2- 172.21.15.16:0/4222237474 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eace05c2f0 0x55eace435a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.963+0000 7fc778a00640 1 -- 172.21.15.16:0/4222237474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eace0851b0 msgr2=0x55eace431530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.963+0000 7fc778a00640 1 --2- 172.21.15.16:0/4222237474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eace0851b0 0x55eace431530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.963+0000 7fc778a00640 1 -- 172.21.15.16:0/4222237474 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eace427350 con 0x55eace0644d0 2024-08-31T06:14:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.963+0000 7fc779e00640 1 --2- 172.21.15.16:0/4222237474 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eace05c2f0 0x55eace435a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:14:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.963+0000 7fc778a00640 1 --2- 172.21.15.16:0/4222237474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eace0644d0 0x55eace430ff0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x55eace446550 tx=0x7fc760005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef5448244f4b4fae server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.963+0000 7fc765c00640 1 -- 172.21.15.16:0/4222237474 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc760005220 con 0x55eace0644d0 2024-08-31T06:14:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.964+0000 7fc765c00640 1 -- 172.21.15.16:0/4222237474 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc760004c50 con 0x55eace0644d0 2024-08-31T06:14:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.964+0000 7fc765c00640 1 -- 172.21.15.16:0/4222237474 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc760010a50 con 0x55eace0644d0 2024-08-31T06:14:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.964+0000 7fc782e42740 1 -- 172.21.15.16:0/4222237474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eace0644d0 msgr2=0x55eace430ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.964+0000 7fc782e42740 1 --2- 172.21.15.16:0/4222237474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eace0644d0 0x55eace430ff0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x55eace446550 tx=0x7fc760005f90 comp rx=0 tx=0).stop 2024-08-31T06:14:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.964+0000 7fc782e42740 1 -- 172.21.15.16:0/4222237474 shutdown_connections 2024-08-31T06:14:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.964+0000 7fc782e42740 1 --2- 172.21.15.16:0/4222237474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eace0851b0 0x55eace431530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.964+0000 7fc782e42740 1 --2- 172.21.15.16:0/4222237474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eace0644d0 0x55eace430ff0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.964+0000 7fc782e42740 1 --2- 172.21.15.16:0/4222237474 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eace05c2f0 0x55eace435a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.964+0000 7fc782e42740 1 -- 172.21.15.16:0/4222237474 >> 172.21.15.16:0/4222237474 conn(0x55eace079300 msgr2=0x55eace435340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.964+0000 7fc782e42740 1 -- 172.21.15.16:0/4222237474 shutdown_connections 2024-08-31T06:14:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.964+0000 7fc782e42740 1 -- 172.21.15.16:0/4222237474 wait complete. 2024-08-31T06:14:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.965+0000 7fc782e42740 1 -- start start 2024-08-31T06:14:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.965+0000 7fc782e42740 1 Processor -- start 2024-08-31T06:14:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.965+0000 7fc782e42740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eace05c2f0 0x55eace4a8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.965+0000 7fc782e42740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eace0644d0 0x55eace4a9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.965+0000 7fc782e42740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eace0851b0 0x55eace437ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.965+0000 7fc782e42740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc7741afb00 con 0x55eace0851b0 2024-08-31T06:14:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.965+0000 7fc782e42740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc7741b0f70 con 0x55eace05c2f0 2024-08-31T06:14:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.966+0000 7fc782e42740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc7741b01d0 con 0x55eace0644d0 2024-08-31T06:14:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.966+0000 7fc782e42740 10 client.0 _ll_register_callbacks cb 0x7fc760009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:14:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.966+0000 7fc779e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eace0851b0 0x55eace437ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.966+0000 7fc778a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eace0644d0 0x55eace4a9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.966+0000 7fc779e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eace0851b0 0x55eace437ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55674/0 (socket says 172.21.15.16:55674) 2024-08-31T06:14:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.966+0000 7fc779e00640 1 -- 172.21.15.16:0/3466156048 learned_addr learned my addr 172.21.15.16:0/3466156048 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.966+0000 7fc779e00640 1 --2- 172.21.15.16:0/3466156048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eace05c2f0 0x55eace4a8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.966+0000 7fc778a00640 1 -- 172.21.15.16:0/3466156048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eace05c2f0 msgr2=0x55eace4a8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.966+0000 7fc778a00640 1 --2- 172.21.15.16:0/3466156048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eace05c2f0 0x55eace4a8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.966+0000 7fc778a00640 1 -- 172.21.15.16:0/3466156048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eace0851b0 msgr2=0x55eace437ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.966+0000 7fc778a00640 1 --2- 172.21.15.16:0/3466156048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eace0851b0 0x55eace437ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.966+0000 7fc778a00640 1 -- 172.21.15.16:0/3466156048 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eace42f970 con 0x55eace0644d0 2024-08-31T06:14:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.967+0000 7fc778a00640 1 --2- 172.21.15.16:0/3466156048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eace0644d0 0x55eace4a9120 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc760012620 tx=0x7fc760004e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b833f39bd4b0093b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.967+0000 7fc75aa00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:14:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.967+0000 7fc75aa00640 1 -- 172.21.15.16:0/3466156048 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc760034060 con 0x55eace0644d0 2024-08-31T06:14:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.967+0000 7fc782e42740 1 -- 172.21.15.16:0/3466156048 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc760004c50 con 0x55eace0644d0 2024-08-31T06:14:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.967+0000 7fc75aa00640 1 -- 172.21.15.16:0/3466156048 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7fc7600048c0 con 0x55eace0644d0 2024-08-31T06:14:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.967+0000 7fc75aa00640 1 -- 172.21.15.16:0/3466156048 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc760005020 con 0x55eace0644d0 2024-08-31T06:14:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.967+0000 7fc782e42740 10 client.27268 fetch_fsmap learned FSMap version 39 2024-08-31T06:14:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.967+0000 7fc782e42740 1 -- 172.21.15.16:0/3466156048 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55eace4b0170 con 0x55eace0644d0 2024-08-31T06:14:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.968+0000 7fc75aa00640 1 -- 172.21.15.16:0/3466156048 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc760004c50 con 0x55eace0644d0 2024-08-31T06:14:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.968+0000 7fc75aa00640 1 -- 172.21.15.16:0/3466156048 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fc760010bf0 con 0x55eace0644d0 2024-08-31T06:14:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.968+0000 7fc782e42740 10 client.27268 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T06:14:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.968+0000 7fc782e42740 10 client.27268 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:14:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.968+0000 7fc782e42740 20 client.27268 populate_metadata read hostname 'smithi016' 2024-08-31T06:14:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.968+0000 7fc782e42740 1 --2- 172.21.15.16:0/3466156048 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55eace0191c0 0x55eace4b28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.968+0000 7fc782e42740 4 client.27268 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:14:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.968+0000 7fc782e42740 1 -- 172.21.15.16:0/3466156048 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eace39b960 con 0x55eace0191c0 2024-08-31T06:14:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.969+0000 7fc779e00640 1 --2- 172.21.15.16:0/3466156048 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55eace0191c0 0x55eace4b28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.969+0000 7fc779e00640 1 --2- 172.21.15.16:0/3466156048 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55eace0191c0 0x55eace4b28e0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f274f8eb71344169 server_cookie=a37119ae1ed82b23 in_seq=0 out_seq=0 2024-08-31T06:14:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.969+0000 7fc75aa00640 10 client.27268 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:44.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:44 smithi184 ceph-mon[22331]: pgmap v593: 97 pgs: 97 active+clean; 3.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 4.2 MiB/s wr, 407 op/s 2024-08-31T06:14:44.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.985+0000 7fc75aa00640 1 -- 172.21.15.16:0/3466156048 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eace39b960 con 0x55eace0191c0 2024-08-31T06:14:44.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.985+0000 7fc75aa00640 10 client.27268 handle_command_reply: tid=0 2024-08-31T06:14:44.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.985+0000 7fc782e42740 1 client.27268 shutdown 2024-08-31T06:14:44.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.986+0000 7fc782e42740 1 -- 172.21.15.16:0/3466156048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eace0644d0 msgr2=0x55eace4a9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.986+0000 7fc782e42740 1 --2- 172.21.15.16:0/3466156048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eace0644d0 0x55eace4a9120 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc760012620 tx=0x7fc760004e00 comp rx=0 tx=0).stop 2024-08-31T06:14:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.986+0000 7fc782e42740 1 -- 172.21.15.16:0/3466156048 shutdown_connections 2024-08-31T06:14:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.986+0000 7fc782e42740 1 --2- 172.21.15.16:0/3466156048 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55eace0191c0 0x55eace4b28e0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.986+0000 7fc782e42740 1 --2- 172.21.15.16:0/3466156048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eace0644d0 0x55eace4a9120 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.986+0000 7fc782e42740 1 --2- 172.21.15.16:0/3466156048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eace0851b0 0x55eace437ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.986+0000 7fc782e42740 1 --2- 172.21.15.16:0/3466156048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eace05c2f0 0x55eace4a8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.986+0000 7fc75aa00640 0 client.27268 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.986+0000 7fc782e42740 1 -- 172.21.15.16:0/3466156048 >> 172.21.15.16:0/3466156048 conn(0x55eace079300 msgr2=0x55eace433d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.986+0000 7fc782e42740 1 -- 172.21.15.16:0/3466156048 shutdown_connections 2024-08-31T06:14:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.986+0000 7fc782e42740 1 -- 172.21.15.16:0/3466156048 wait complete. 2024-08-31T06:14:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.986+0000 7fc782e42740 20 client.27268 trim_cache size 0 max 16384 2024-08-31T06:14:44.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:14:44.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:14:44.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:14:44.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:14:44.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.987+0000 7fc77a800640 1 -- 172.21.15.16:0/1291764140 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc75c0b4ca0 msgr2=0x7fc75c0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:44.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.987+0000 7fc77a800640 1 --2- 172.21.15.16:0/1291764140 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc75c0b4ca0 0x7fc75c0b71b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x55eace427520 tx=0x7fc768031400 comp rx=0 tx=0).stop 2024-08-31T06:14:44.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.987+0000 7fc77a800640 1 -- 172.21.15.16:0/1291764140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7741036b0 msgr2=0x7fc77419c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:44.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.987+0000 7fc77a800640 1 --2- 172.21.15.16:0/1291764140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7741036b0 0x7fc77419c2b0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc768005ab0 tx=0x7fc768003c10 comp rx=0 tx=0).stop 2024-08-31T06:14:44.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.987+0000 7fc77a800640 1 -- 172.21.15.16:0/1291764140 shutdown_connections 2024-08-31T06:14:44.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.987+0000 7fc77a800640 1 --2- 172.21.15.16:0/1291764140 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc75c0b4ca0 0x7fc75c0b71b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:44.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.987+0000 7fc77a800640 1 --2- 172.21.15.16:0/1291764140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc774108fe0 0x7fc77419c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:44.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.987+0000 7fc77a800640 1 --2- 172.21.15.16:0/1291764140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc774109930 0x7fc7741a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:44.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.987+0000 7fc77a800640 1 --2- 172.21.15.16:0/1291764140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7741036b0 0x7fc77419c2b0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:44.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.987+0000 7fc77a800640 1 -- 172.21.15.16:0/1291764140 >> 172.21.15.16:0/1291764140 conn(0x7fc7740ff0d0 msgr2=0x7fc77410e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:44.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.987+0000 7fc77a800640 1 -- 172.21.15.16:0/1291764140 shutdown_connections 2024-08-31T06:14:44.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:44.987+0000 7fc77a800640 1 -- 172.21.15.16:0/1291764140 wait complete. 2024-08-31T06:14:46.629 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 440.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-31T06:14:46.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:46 smithi155 ceph-mon[21742]: pgmap v594: 97 pgs: 97 active+clean; 3.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 4.9 MiB/s wr, 480 op/s 2024-08-31T06:14:46.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:46 smithi016 ceph-mon[17194]: pgmap v594: 97 pgs: 97 active+clean; 3.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 4.9 MiB/s wr, 480 op/s 2024-08-31T06:14:46.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:46 smithi184 ceph-mon[22331]: pgmap v594: 97 pgs: 97 active+clean; 3.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 4.9 MiB/s wr, 480 op/s 2024-08-31T06:14:48.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:48 smithi155 ceph-mon[21742]: pgmap v595: 97 pgs: 97 active+clean; 3.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 4.6 MiB/s wr, 437 op/s 2024-08-31T06:14:48.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:48 smithi016 ceph-mon[17194]: pgmap v595: 97 pgs: 97 active+clean; 3.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 4.6 MiB/s wr, 437 op/s 2024-08-31T06:14:48.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:48 smithi184 ceph-mon[22331]: pgmap v595: 97 pgs: 97 active+clean; 3.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 4.6 MiB/s wr, 437 op/s 2024-08-31T06:14:50.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:50 smithi155 ceph-mon[21742]: pgmap v596: 97 pgs: 97 active+clean; 3.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 3.2 MiB/s wr, 338 op/s 2024-08-31T06:14:50.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:50 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:14:50.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:50 smithi016 ceph-mon[17194]: pgmap v596: 97 pgs: 97 active+clean; 3.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 3.2 MiB/s wr, 338 op/s 2024-08-31T06:14:50.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:50 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:14:50.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:50 smithi184 ceph-mon[22331]: pgmap v596: 97 pgs: 97 active+clean; 3.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 3.2 MiB/s wr, 338 op/s 2024-08-31T06:14:50.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:50 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:14:51.629 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 445.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-31T06:14:51.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:51 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:14:51.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:51 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:14:51.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:51 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:14:51.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:51 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:14:51.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:51 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:14:51.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:51 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:14:51.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:51 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:14:51.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:51 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:14:51.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:51 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:14:52.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:52 smithi155 ceph-mon[21742]: pgmap v597: 97 pgs: 97 active+clean; 3.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 1.8 MiB/s wr, 203 op/s 2024-08-31T06:14:52.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:52 smithi016 ceph-mon[17194]: pgmap v597: 97 pgs: 97 active+clean; 3.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 1.8 MiB/s wr, 203 op/s 2024-08-31T06:14:52.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:52 smithi184 ceph-mon[22331]: pgmap v597: 97 pgs: 97 active+clean; 3.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 1.8 MiB/s wr, 203 op/s 2024-08-31T06:14:53.163 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T06:14:53.557 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:14:54.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.217+0000 7f2837200640 1 -- 172.21.15.16:0/3024365776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f283810b2a0 msgr2=0x7f283810b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:54.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.217+0000 7f2837200640 1 --2- 172.21.15.16:0/3024365776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f283810b2a0 0x7f283810b6b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f2824009900 tx=0x7f282402a410 comp rx=0 tx=0).stop 2024-08-31T06:14:54.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.217+0000 7f2837200640 1 -- 172.21.15.16:0/3024365776 shutdown_connections 2024-08-31T06:14:54.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.217+0000 7f2837200640 1 --2- 172.21.15.16:0/3024365776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f283810bbf0 0x7f2838112420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:54.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.217+0000 7f2837200640 1 --2- 172.21.15.16:0/3024365776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f283810b2a0 0x7f283810b6b0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:54.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.217+0000 7f2837200640 1 --2- 172.21.15.16:0/3024365776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2838105960 0x7f2838105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:54.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.217+0000 7f2837200640 1 -- 172.21.15.16:0/3024365776 >> 172.21.15.16:0/3024365776 conn(0x7f2838101410 msgr2=0x7f2838103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:54.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.217+0000 7f2837200640 1 -- 172.21.15.16:0/3024365776 shutdown_connections 2024-08-31T06:14:54.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.217+0000 7f2837200640 1 -- 172.21.15.16:0/3024365776 wait complete. 2024-08-31T06:14:54.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.218+0000 7f2837200640 1 Processor -- start 2024-08-31T06:14:54.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.218+0000 7f2837200640 1 -- start start 2024-08-31T06:14:54.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.219+0000 7f2837200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2838105960 0x7f283819c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:54.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.219+0000 7f2837200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f283810b2a0 0x7f283819c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:54.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.219+0000 7f2837200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f283810bbf0 0x7f28381a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:54.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.219+0000 7f2837200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2838115900 con 0x7f283810b2a0 2024-08-31T06:14:54.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.219+0000 7f2837200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2838115780 con 0x7f2838105960 2024-08-31T06:14:54.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.219+0000 7f2837200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2838115a80 con 0x7f283810bbf0 2024-08-31T06:14:54.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.219+0000 7f2835400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f283810b2a0 0x7f283819c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:54.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.219+0000 7f2835400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f283810b2a0 0x7f283819c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47140/0 (socket says 172.21.15.16:47140) 2024-08-31T06:14:54.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.219+0000 7f2835400640 1 -- 172.21.15.16:0/2169440710 learned_addr learned my addr 172.21.15.16:0/2169440710 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:54.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.220+0000 7f2835e00640 1 --2- 172.21.15.16:0/2169440710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2838105960 0x7f283819c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:54.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.220+0000 7f2836800640 1 --2- 172.21.15.16:0/2169440710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f283810bbf0 0x7f28381a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:54.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.220+0000 7f2835e00640 1 -- 172.21.15.16:0/2169440710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f283810bbf0 msgr2=0x7f28381a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:54.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.220+0000 7f2835e00640 1 --2- 172.21.15.16:0/2169440710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f283810bbf0 0x7f28381a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:54.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.220+0000 7f2835e00640 1 -- 172.21.15.16:0/2169440710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f283810b2a0 msgr2=0x7f283819c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:54.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.220+0000 7f2835e00640 1 --2- 172.21.15.16:0/2169440710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f283810b2a0 0x7f283819c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:54.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.220+0000 7f2835e00640 1 -- 172.21.15.16:0/2169440710 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28381a2220 con 0x7f2838105960 2024-08-31T06:14:54.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.220+0000 7f2835400640 1 --2- 172.21.15.16:0/2169440710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f283810b2a0 0x7f283819c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:14:54.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.220+0000 7f2836800640 1 --2- 172.21.15.16:0/2169440710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f283810bbf0 0x7f28381a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:14:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.220+0000 7f2835e00640 1 --2- 172.21.15.16:0/2169440710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2838105960 0x7f283819c2b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f282800e9f0 tx=0x7f282800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.221+0000 7f282ea00640 1 -- 172.21.15.16:0/2169440710 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282800ccc0 con 0x7f2838105960 2024-08-31T06:14:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.221+0000 7f2837200640 1 -- 172.21.15.16:0/2169440710 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28381a2510 con 0x7f2838105960 2024-08-31T06:14:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.221+0000 7f282ea00640 1 -- 172.21.15.16:0/2169440710 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2828005020 con 0x7f2838105960 2024-08-31T06:14:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.221+0000 7f282ea00640 1 -- 172.21.15.16:0/2169440710 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2828004a00 con 0x7f2838105960 2024-08-31T06:14:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.221+0000 7f2837200640 1 -- 172.21.15.16:0/2169440710 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28381a8c70 con 0x7f2838105960 2024-08-31T06:14:54.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.224+0000 7f284384f740 1 Processor -- start 2024-08-31T06:14:54.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.224+0000 7f284384f740 1 -- start start 2024-08-31T06:14:54.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.225+0000 7f284384f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a09c881070 0x55a09cc26b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:54.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.225+0000 7f284384f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a09c8581b0 0x55a09cc27060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:54.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.225+0000 7f284384f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a09c860390 0x55a09cc275a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:54.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.225+0000 7f284384f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a09cb9d770 con 0x55a09c881070 2024-08-31T06:14:54.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.225+0000 7f284384f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a09cb8cd00 con 0x55a09c8581b0 2024-08-31T06:14:54.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.225+0000 7f284384f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a09cb90cb0 con 0x55a09c860390 2024-08-31T06:14:54.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.225+0000 7f2836800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a09c881070 0x55a09cc26b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:54.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.225+0000 7f282ea00640 1 -- 172.21.15.16:0/2169440710 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f2828004ba0 con 0x7f2838105960 2024-08-31T06:14:54.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.225+0000 7f2836800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a09c881070 0x55a09cc26b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47148/0 (socket says 172.21.15.16:47148) 2024-08-31T06:14:54.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.225+0000 7f2836800640 1 -- 172.21.15.16:0/301148891 learned_addr learned my addr 172.21.15.16:0/301148891 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:54.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.225+0000 7f2835400640 1 --2- 172.21.15.16:0/301148891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a09c860390 0x55a09cc275a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:54.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.225+0000 7f2835e00640 1 --2- 172.21.15.16:0/301148891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a09c8581b0 0x55a09cc27060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:54.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.226+0000 7f2836800640 1 -- 172.21.15.16:0/301148891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a09c860390 msgr2=0x55a09cc275a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:54.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.226+0000 7f2836800640 1 --2- 172.21.15.16:0/301148891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a09c860390 0x55a09cc275a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:54.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.226+0000 7f2836800640 1 -- 172.21.15.16:0/301148891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a09c8581b0 msgr2=0x55a09cc27060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:54.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.226+0000 7f2836800640 1 --2- 172.21.15.16:0/301148891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a09c8581b0 0x55a09cc27060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:54.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.226+0000 7f2836800640 1 -- 172.21.15.16:0/301148891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a09c8ec650 con 0x55a09c881070 2024-08-31T06:14:54.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.226+0000 7f2835400640 1 --2- 172.21.15.16:0/301148891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a09c860390 0x55a09cc275a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:14:54.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.226+0000 7f2835e00640 1 --2- 172.21.15.16:0/301148891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a09c8581b0 0x55a09cc27060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:14:54.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.226+0000 7f2836800640 1 --2- 172.21.15.16:0/301148891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a09c881070 0x55a09cc26b20 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f28300117e0 tx=0x7f2830011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b83fd3adebebeb5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:54.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.226+0000 7f2823000640 1 -- 172.21.15.16:0/301148891 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2830011310 con 0x55a09c881070 2024-08-31T06:14:54.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.226+0000 7f2823000640 1 -- 172.21.15.16:0/301148891 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2830003e20 con 0x55a09c881070 2024-08-31T06:14:54.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.226+0000 7f2823000640 1 -- 172.21.15.16:0/301148891 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2830004490 con 0x55a09c881070 2024-08-31T06:14:54.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.227+0000 7f282ea00640 1 --2- 172.21.15.16:0/2169440710 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f281c0b4ca0 0x7f281c0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:54.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.227+0000 7f282ea00640 1 -- 172.21.15.16:0/2169440710 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7f28280e54d0 con 0x7f2838105960 2024-08-31T06:14:54.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.227+0000 7f2835e00640 1 --2- 172.21.15.16:0/2169440710 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f281c0b4ca0 0x7f281c0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:54.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.227+0000 7f284384f740 1 -- 172.21.15.16:0/301148891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a09c881070 msgr2=0x55a09cc26b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:54.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.227+0000 7f284384f740 1 --2- 172.21.15.16:0/301148891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a09c881070 0x55a09cc26b20 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f28300117e0 tx=0x7f2830011ca0 comp rx=0 tx=0).stop 2024-08-31T06:14:54.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.228+0000 7f284384f740 1 -- 172.21.15.16:0/301148891 shutdown_connections 2024-08-31T06:14:54.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.228+0000 7f284384f740 1 --2- 172.21.15.16:0/301148891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a09c860390 0x55a09cc275a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:54.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.228+0000 7f284384f740 1 --2- 172.21.15.16:0/301148891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a09c8581b0 0x55a09cc27060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:54.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.228+0000 7f284384f740 1 --2- 172.21.15.16:0/301148891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a09c881070 0x55a09cc26b20 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:54.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.228+0000 7f284384f740 1 -- 172.21.15.16:0/301148891 >> 172.21.15.16:0/301148891 conn(0x55a09c875460 msgr2=0x55a09cc24a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:54.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.228+0000 7f2835e00640 1 --2- 172.21.15.16:0/2169440710 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f281c0b4ca0 0x7f281c0b71b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x55a09cc257a0 tx=0x7f2828005460 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:54.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.228+0000 7f284384f740 1 -- 172.21.15.16:0/301148891 shutdown_connections 2024-08-31T06:14:54.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.228+0000 7f284384f740 1 -- 172.21.15.16:0/301148891 wait complete. 2024-08-31T06:14:54.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.228+0000 7f284384f740 1 -- start start 2024-08-31T06:14:54.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.229+0000 7f284384f740 1 Processor -- start 2024-08-31T06:14:54.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.229+0000 7f284384f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a09c8581b0 0x55a09cc32770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:54.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.229+0000 7f284384f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a09c860390 0x55a09cc32cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:54.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.229+0000 7f284384f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a09c881070 0x55a09cc3e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:54.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.229+0000 7f284384f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a09c9ef700 con 0x55a09c8581b0 2024-08-31T06:14:54.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.229+0000 7f284384f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a09cb13cb0 con 0x55a09c860390 2024-08-31T06:14:54.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.229+0000 7f284384f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a09cc35300 con 0x55a09c881070 2024-08-31T06:14:54.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.229+0000 7f284384f740 10 client.0 _ll_register_callbacks cb 0x55a09cb39430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:14:54.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.229+0000 7f2836800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a09c8581b0 0x55a09cc32770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:54.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.230+0000 7f2836800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a09c8581b0 0x55a09cc32770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47158/0 (socket says 172.21.15.16:47158) 2024-08-31T06:14:54.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.230+0000 7f2836800640 1 -- 172.21.15.16:0/2746846359 learned_addr learned my addr 172.21.15.16:0/2746846359 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:54.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.230+0000 7f2835400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a09c860390 0x55a09cc32cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:54.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.230+0000 7f2836800640 1 --2- 172.21.15.16:0/2746846359 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a09c881070 0x55a09cc3e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:54.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.230+0000 7f2835400640 1 -- 172.21.15.16:0/2746846359 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a09c881070 msgr2=0x55a09cc3e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:54.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.230+0000 7f2835400640 1 --2- 172.21.15.16:0/2746846359 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a09c881070 0x55a09cc3e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:54.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.230+0000 7f2835400640 1 -- 172.21.15.16:0/2746846359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a09c8581b0 msgr2=0x55a09cc32770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:54.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.230+0000 7f2835400640 1 --2- 172.21.15.16:0/2746846359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a09c8581b0 0x55a09cc32770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:54.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.230+0000 7f2835400640 1 -- 172.21.15.16:0/2746846359 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a09cb54870 con 0x55a09c860390 2024-08-31T06:14:54.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.230+0000 7f2835400640 1 --2- 172.21.15.16:0/2746846359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a09c860390 0x55a09cc32cf0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2824005ab0 tx=0x7f2824003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54a01c47e4c26237 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:54.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.230+0000 7f2817400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:14:54.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.231+0000 7f2817400640 1 -- 172.21.15.16:0/2746846359 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2824035040 con 0x55a09c860390 2024-08-31T06:14:54.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.231+0000 7f284384f740 1 -- 172.21.15.16:0/2746846359 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2830003e20 con 0x55a09c860390 2024-08-31T06:14:54.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.231+0000 7f2817400640 1 -- 172.21.15.16:0/2746846359 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7f2824003e20 con 0x55a09c860390 2024-08-31T06:14:54.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.231+0000 7f2817400640 1 -- 172.21.15.16:0/2746846359 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2824005020 con 0x55a09c860390 2024-08-31T06:14:54.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.231+0000 7f284384f740 10 client.27299 fetch_fsmap learned FSMap version 39 2024-08-31T06:14:54.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.231+0000 7f284384f740 1 -- 172.21.15.16:0/2746846359 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a09cc413b0 con 0x55a09c860390 2024-08-31T06:14:54.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.231+0000 7f2817400640 1 -- 172.21.15.16:0/2746846359 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2830003e20 con 0x55a09c860390 2024-08-31T06:14:54.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.232+0000 7f2817400640 1 -- 172.21.15.16:0/2746846359 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f28240314a0 con 0x55a09c860390 2024-08-31T06:14:54.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.232+0000 7f284384f740 10 client.27299 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T06:14:54.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.232+0000 7f284384f740 10 client.27299 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:14:54.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.232+0000 7f284384f740 20 client.27299 populate_metadata read hostname 'smithi016' 2024-08-31T06:14:54.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.232+0000 7f284384f740 1 --2- 172.21.15.16:0/2746846359 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a09c83f9f0 0x55a09cc44880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:54.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.232+0000 7f284384f740 4 client.27299 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:14:54.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.232+0000 7f284384f740 1 -- 172.21.15.16:0/2746846359 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a09cb9f6c0 con 0x55a09c83f9f0 2024-08-31T06:14:54.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.232+0000 7f2836800640 1 --2- 172.21.15.16:0/2746846359 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a09c83f9f0 0x55a09cc44880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:54.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.233+0000 7f2836800640 1 --2- 172.21.15.16:0/2746846359 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a09c83f9f0 0x55a09cc44880 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48c869de181c856d server_cookie=e1dfbb8947a70a6d in_seq=0 out_seq=0 2024-08-31T06:14:54.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.233+0000 7f2817400640 10 client.27299 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:54.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.234+0000 7f2817400640 1 -- 172.21.15.16:0/2746846359 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a09cb9f6c0 con 0x55a09c83f9f0 2024-08-31T06:14:54.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.234+0000 7f2817400640 10 client.27299 handle_command_reply: tid=0 2024-08-31T06:14:54.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.234+0000 7f284384f740 1 client.27299 shutdown 2024-08-31T06:14:54.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.234+0000 7f284384f740 1 -- 172.21.15.16:0/2746846359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a09c860390 msgr2=0x55a09cc32cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:54.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.234+0000 7f284384f740 1 --2- 172.21.15.16:0/2746846359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a09c860390 0x55a09cc32cf0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2824005ab0 tx=0x7f2824003820 comp rx=0 tx=0).stop 2024-08-31T06:14:54.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.234+0000 7f284384f740 1 -- 172.21.15.16:0/2746846359 shutdown_connections 2024-08-31T06:14:54.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.234+0000 7f284384f740 1 --2- 172.21.15.16:0/2746846359 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a09c83f9f0 0x55a09cc44880 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:54.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.234+0000 7f284384f740 1 --2- 172.21.15.16:0/2746846359 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a09c881070 0x55a09cc3e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:54.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.234+0000 7f284384f740 1 --2- 172.21.15.16:0/2746846359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a09c860390 0x55a09cc32cf0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:54.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.234+0000 7f2817400640 0 client.27299 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:54.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.234+0000 7f284384f740 1 --2- 172.21.15.16:0/2746846359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a09c8581b0 0x55a09cc32770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:54.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.234+0000 7f284384f740 1 -- 172.21.15.16:0/2746846359 >> 172.21.15.16:0/2746846359 conn(0x55a09c875460 msgr2=0x55a09cc29d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:54.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.234+0000 7f284384f740 1 -- 172.21.15.16:0/2746846359 shutdown_connections 2024-08-31T06:14:54.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.234+0000 7f284384f740 1 -- 172.21.15.16:0/2746846359 wait complete. 2024-08-31T06:14:54.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.234+0000 7f284384f740 20 client.27299 trim_cache size 0 max 16384 2024-08-31T06:14:54.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.253+0000 7f284384f740 1 Processor -- start 2024-08-31T06:14:54.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.253+0000 7f284384f740 1 -- start start 2024-08-31T06:14:54.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.254+0000 7f284384f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a09c8581b0 0x55a09cc2c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:54.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.254+0000 7f284384f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a09c860390 0x55a09cc23fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:54.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.254+0000 7f284384f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a09c881070 0x55a09cc24500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:54.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.254+0000 7f284384f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f28381b3d50 con 0x55a09c860390 2024-08-31T06:14:54.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.254+0000 7f284384f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f28381b15c0 con 0x55a09c8581b0 2024-08-31T06:14:54.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.254+0000 7f284384f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f28381af430 con 0x55a09c881070 2024-08-31T06:14:54.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.254+0000 7f2835400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a09c860390 0x55a09cc23fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:54.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.254+0000 7f2836800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a09c8581b0 0x55a09cc2c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:54.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.254+0000 7f2836800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a09c8581b0 0x55a09cc2c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:45522/0 (socket says 172.21.15.16:45522) 2024-08-31T06:14:54.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.254+0000 7f2835400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a09c860390 0x55a09cc23fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47172/0 (socket says 172.21.15.16:47172) 2024-08-31T06:14:54.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.254+0000 7f2836800640 1 -- 172.21.15.16:0/1767833568 learned_addr learned my addr 172.21.15.16:0/1767833568 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:54.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.254+0000 7f2836800640 1 --2- 172.21.15.16:0/1767833568 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a09c881070 0x55a09cc24500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:54.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.254+0000 7f2836800640 1 -- 172.21.15.16:0/1767833568 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a09c881070 msgr2=0x55a09cc24500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:54.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.254+0000 7f2836800640 1 --2- 172.21.15.16:0/1767833568 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a09c881070 0x55a09cc24500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:54.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.254+0000 7f2836800640 1 -- 172.21.15.16:0/1767833568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a09c860390 msgr2=0x55a09cc23fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:54.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.254+0000 7f2836800640 1 --2- 172.21.15.16:0/1767833568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a09c860390 0x55a09cc23fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:54.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.254+0000 7f2836800640 1 -- 172.21.15.16:0/1767833568 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a09cc33830 con 0x55a09c8581b0 2024-08-31T06:14:54.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.255+0000 7f2836800640 1 --2- 172.21.15.16:0/1767833568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a09c8581b0 0x55a09cc2c340 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f283000f790 tx=0x7f2830003a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e95acf976c7e49cb server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:54.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.255+0000 7f2822600640 1 -- 172.21.15.16:0/1767833568 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2830023040 con 0x55a09c8581b0 2024-08-31T06:14:54.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.255+0000 7f2822600640 1 -- 172.21.15.16:0/1767833568 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2830003f50 con 0x55a09c8581b0 2024-08-31T06:14:54.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.255+0000 7f2822600640 1 -- 172.21.15.16:0/1767833568 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28300058f0 con 0x55a09c8581b0 2024-08-31T06:14:54.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.256+0000 7f284384f740 1 -- 172.21.15.16:0/1767833568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a09c8581b0 msgr2=0x55a09cc2c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:54.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.256+0000 7f284384f740 1 --2- 172.21.15.16:0/1767833568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a09c8581b0 0x55a09cc2c340 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f283000f790 tx=0x7f2830003a70 comp rx=0 tx=0).stop 2024-08-31T06:14:54.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.256+0000 7f284384f740 1 -- 172.21.15.16:0/1767833568 shutdown_connections 2024-08-31T06:14:54.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.256+0000 7f284384f740 1 --2- 172.21.15.16:0/1767833568 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a09c881070 0x55a09cc24500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:54.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.256+0000 7f284384f740 1 --2- 172.21.15.16:0/1767833568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a09c860390 0x55a09cc23fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:54.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.256+0000 7f284384f740 1 --2- 172.21.15.16:0/1767833568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a09c8581b0 0x55a09cc2c340 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:54.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.256+0000 7f284384f740 1 -- 172.21.15.16:0/1767833568 >> 172.21.15.16:0/1767833568 conn(0x55a09c875460 msgr2=0x55a09cc2bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:54.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.256+0000 7f284384f740 1 -- 172.21.15.16:0/1767833568 shutdown_connections 2024-08-31T06:14:54.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.256+0000 7f284384f740 1 -- 172.21.15.16:0/1767833568 wait complete. 2024-08-31T06:14:54.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.256+0000 7f284384f740 1 -- start start 2024-08-31T06:14:54.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.257+0000 7f284384f740 1 Processor -- start 2024-08-31T06:14:54.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.257+0000 7f284384f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a09c8581b0 0x55a09cc376e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:54.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.257+0000 7f284384f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a09c860390 0x55a09cc37c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:54.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.257+0000 7f284384f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a09c881070 0x55a09cc38160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:54.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.257+0000 7f284384f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f28381afa80 con 0x55a09c881070 2024-08-31T06:14:54.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.257+0000 7f284384f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f28381b0ef0 con 0x55a09c860390 2024-08-31T06:14:54.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.257+0000 7f284384f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f28381b0150 con 0x55a09c8581b0 2024-08-31T06:14:54.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.257+0000 7f284384f740 10 client.0 _ll_register_callbacks cb 0x7f2824004480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:14:54.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.257+0000 7f2836800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a09c8581b0 0x55a09cc376e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:54.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.257+0000 7f2836800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a09c881070 0x55a09cc38160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:54.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.257+0000 7f2835400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a09c860390 0x55a09cc37c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:54.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.257+0000 7f2836800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a09c881070 0x55a09cc38160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47186/0 (socket says 172.21.15.16:47186) 2024-08-31T06:14:54.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.258+0000 7f2836800640 1 -- 172.21.15.16:0/434228172 learned_addr learned my addr 172.21.15.16:0/434228172 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:54.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.258+0000 7f2835400640 1 -- 172.21.15.16:0/434228172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a09c8581b0 msgr2=0x55a09cc376e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:54.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.258+0000 7f2835400640 1 --2- 172.21.15.16:0/434228172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a09c8581b0 0x55a09cc376e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:54.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.258+0000 7f2835400640 1 -- 172.21.15.16:0/434228172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a09c881070 msgr2=0x55a09cc38160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:54.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.258+0000 7f2835400640 1 --2- 172.21.15.16:0/434228172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a09c881070 0x55a09cc38160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:54.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.258+0000 7f2835400640 1 -- 172.21.15.16:0/434228172 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a09ccaf440 con 0x55a09c860390 2024-08-31T06:14:54.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.258+0000 7f2836800640 1 --2- 172.21.15.16:0/434228172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a09c881070 0x55a09cc38160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:14:54.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.258+0000 7f2835400640 1 --2- 172.21.15.16:0/434228172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a09c860390 0x55a09cc37c20 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f28240318f0 tx=0x7f2824005f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8acf72b847affa0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:54.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.258+0000 7f2817400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:14:54.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.258+0000 7f2817400640 1 -- 172.21.15.16:0/434228172 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2824052040 con 0x55a09c860390 2024-08-31T06:14:54.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.259+0000 7f284384f740 1 -- 172.21.15.16:0/434228172 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a09cb9b160 con 0x55a09c860390 2024-08-31T06:14:54.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.259+0000 7f2817400640 1 -- 172.21.15.16:0/434228172 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7f2824003c20 con 0x55a09c860390 2024-08-31T06:14:54.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.259+0000 7f2817400640 1 -- 172.21.15.16:0/434228172 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2824035040 con 0x55a09c860390 2024-08-31T06:14:54.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.259+0000 7f284384f740 10 client.27305 fetch_fsmap learned FSMap version 39 2024-08-31T06:14:54.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.259+0000 7f284384f740 1 -- 172.21.15.16:0/434228172 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a09ccb06d0 con 0x55a09c860390 2024-08-31T06:14:54.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.259+0000 7f2817400640 1 -- 172.21.15.16:0/434228172 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a09cb9b160 con 0x55a09c860390 2024-08-31T06:14:54.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.259+0000 7f2817400640 1 -- 172.21.15.16:0/434228172 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f28240049e0 con 0x55a09c860390 2024-08-31T06:14:54.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.259+0000 7f284384f740 10 client.27305 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T06:14:54.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.259+0000 7f284384f740 10 client.27305 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:14:54.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.259+0000 7f284384f740 20 client.27305 populate_metadata read hostname 'smithi016' 2024-08-31T06:14:54.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.259+0000 7f284384f740 1 --2- 172.21.15.16:0/434228172 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a09c814da0 0x55a09ccb2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:54.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.260+0000 7f284384f740 4 client.27305 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T06:14:54.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.260+0000 7f284384f740 1 -- 172.21.15.16:0/434228172 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a09cb988a0 con 0x55a09c814da0 2024-08-31T06:14:54.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.260+0000 7f2836800640 1 --2- 172.21.15.16:0/434228172 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a09c814da0 0x55a09ccb2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:54.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.260+0000 7f2836800640 1 --2- 172.21.15.16:0/434228172 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a09c814da0 0x55a09ccb2ef0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4214f325d2b67e server_cookie=b6332552d6e50b13 in_seq=0 out_seq=0 2024-08-31T06:14:54.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.260+0000 7f2817400640 10 client.27305 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:54.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.261+0000 7f2817400640 1 -- 172.21.15.16:0/434228172 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2930 (crc 0 0 0) 0x55a09cb988a0 con 0x55a09c814da0 2024-08-31T06:14:54.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.261+0000 7f2817400640 10 client.27305 handle_command_reply: tid=0 2024-08-31T06:14:54.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.261+0000 7f284384f740 1 client.27305 shutdown 2024-08-31T06:14:54.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.261+0000 7f284384f740 1 -- 172.21.15.16:0/434228172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a09c860390 msgr2=0x55a09cc37c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:54.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.261+0000 7f284384f740 1 --2- 172.21.15.16:0/434228172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a09c860390 0x55a09cc37c20 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f28240318f0 tx=0x7f2824005f30 comp rx=0 tx=0).stop 2024-08-31T06:14:54.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.261+0000 7f284384f740 1 -- 172.21.15.16:0/434228172 shutdown_connections 2024-08-31T06:14:54.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.261+0000 7f284384f740 1 --2- 172.21.15.16:0/434228172 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a09c814da0 0x55a09ccb2ef0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:54.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.261+0000 7f284384f740 1 --2- 172.21.15.16:0/434228172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a09c881070 0x55a09cc38160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:54.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.261+0000 7f284384f740 1 --2- 172.21.15.16:0/434228172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a09c860390 0x55a09cc37c20 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:54.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.261+0000 7f284384f740 1 --2- 172.21.15.16:0/434228172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a09c8581b0 0x55a09cc376e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:54.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.261+0000 7f2817400640 0 client.27305 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:54.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.261+0000 7f284384f740 1 -- 172.21.15.16:0/434228172 >> 172.21.15.16:0/434228172 conn(0x55a09c875460 msgr2=0x55a09cc27b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:54.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.261+0000 7f284384f740 1 -- 172.21.15.16:0/434228172 shutdown_connections 2024-08-31T06:14:54.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.261+0000 7f284384f740 1 -- 172.21.15.16:0/434228172 wait complete. 2024-08-31T06:14:54.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.261+0000 7f284384f740 20 client.27305 trim_cache size 0 max 16384 2024-08-31T06:14:54.267 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:14:54.267 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-08-31T06:14:54.267 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {}, 2024-08-31T06:14:54.267 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-08-31T06:14:54.268 INFO:teuthology.orchestra.run.smithi016.stdout: "a64d5301-8558-4e9c-ac9b-8d0ac7057e74": { 2024-08-31T06:14:54.268 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:14:20.338104+0000", 2024-08-31T06:14:54.268 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:14:54.268 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:14:54.268 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:14:54.268 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:14:54.268 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:14:54.268 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:14:54.268 INFO:teuthology.orchestra.run.smithi016.stdout: "e739a16e-8ace-43a8-a1ce-ac27d39ef8a3": { 2024-08-31T06:14:54.268 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:54.084646+0000", 2024-08-31T06:14:54.268 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:14:54.268 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:14:54.268 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:14:54.269 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:14:54.269 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:14:54.269 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:14:54.269 INFO:teuthology.orchestra.run.smithi016.stdout: "91b72a34-4ba9-4126-a414-8b862f8c5f78": { 2024-08-31T06:14:54.269 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:13:28.663237+0000", 2024-08-31T06:14:54.269 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:14:54.269 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:14:54.269 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:14:54.269 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:14:54.269 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:14:54.269 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:14:54.269 INFO:teuthology.orchestra.run.smithi016.stdout: "a10cce76-99ab-46b5-b3b2-07b82899b8f9": { 2024-08-31T06:14:54.269 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:11:43.967546+0000", 2024-08-31T06:14:54.269 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:14:54.270 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:14:54.270 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:14:54.270 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:14:54.270 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:14:54.270 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:14:54.270 INFO:teuthology.orchestra.run.smithi016.stdout: "c1fd6904-2e02-47fa-939a-47349b9fe36b": { 2024-08-31T06:14:54.270 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:06.678691+0000", 2024-08-31T06:14:54.270 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:14:54.270 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:14:54.270 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:14:54.270 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:14:54.270 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:14:54.270 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:14:54.271 INFO:teuthology.orchestra.run.smithi016.stdout: "78d6ef02-8551-45e8-ad60-9627eb573fc1": { 2024-08-31T06:14:54.271 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:09:13.547865+0000", 2024-08-31T06:14:54.271 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:14:54.271 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:14:54.271 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:14:54.271 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:14:54.271 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:14:54.271 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:14:54.271 INFO:teuthology.orchestra.run.smithi016.stdout: "c8455d74-2fe7-4d93-bcf2-19b419d16016": { 2024-08-31T06:14:54.271 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:08:24.035960+0000", 2024-08-31T06:14:54.271 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:14:54.271 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:14:54.271 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:14:54.271 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:14:54.272 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:14:54.272 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:14:54.272 INFO:teuthology.orchestra.run.smithi016.stdout: "88d7bdb8-cb1c-45e5-ad29-4cd8da6c6003": { 2024-08-31T06:14:54.272 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:07:39.648942+0000", 2024-08-31T06:14:54.272 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:14:54.272 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:14:54.272 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:14:54.272 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:14:54.272 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:14:54.272 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:14:54.272 INFO:teuthology.orchestra.run.smithi016.stdout: "6a03046e-cdf4-4a14-b9bd-b560b8968467": { 2024-08-31T06:14:54.272 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:12:38.002354+0000", 2024-08-31T06:14:54.272 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:14:54.272 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:14:54.273 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:14:54.273 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:14:54.273 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:14:54.273 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:14:54.273 INFO:teuthology.orchestra.run.smithi016.stdout: "c300b7e8-f3fd-482c-a909-9329bc61e570": { 2024-08-31T06:14:54.273 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:06:49.868046+0000", 2024-08-31T06:14:54.273 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:14:54.273 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:14:54.273 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:14:54.273 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:14:54.273 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:14:54.273 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:14:54.274 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:14:54.274 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:14:54.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.262+0000 7f2837200640 1 -- 172.21.15.16:0/2169440710 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f281c0b4ca0 msgr2=0x7f281c0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:54.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.262+0000 7f2837200640 1 --2- 172.21.15.16:0/2169440710 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f281c0b4ca0 0x7f281c0b71b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x55a09cc257a0 tx=0x7f2828005460 comp rx=0 tx=0).stop 2024-08-31T06:14:54.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.262+0000 7f2837200640 1 -- 172.21.15.16:0/2169440710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2838105960 msgr2=0x7f283819c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:54.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.262+0000 7f2837200640 1 --2- 172.21.15.16:0/2169440710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2838105960 0x7f283819c2b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f282800e9f0 tx=0x7f282800eeb0 comp rx=0 tx=0).stop 2024-08-31T06:14:54.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.262+0000 7f2837200640 1 -- 172.21.15.16:0/2169440710 shutdown_connections 2024-08-31T06:14:54.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.262+0000 7f2837200640 1 --2- 172.21.15.16:0/2169440710 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f281c0b4ca0 0x7f281c0b71b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:54.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.262+0000 7f2837200640 1 --2- 172.21.15.16:0/2169440710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f283810bbf0 0x7f28381a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:54.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.262+0000 7f2837200640 1 --2- 172.21.15.16:0/2169440710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f283810b2a0 0x7f283819c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:54.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.262+0000 7f2837200640 1 --2- 172.21.15.16:0/2169440710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2838105960 0x7f283819c2b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:54.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.262+0000 7f2837200640 1 -- 172.21.15.16:0/2169440710 >> 172.21.15.16:0/2169440710 conn(0x7f2838101410 msgr2=0x7f28381088d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:54.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.262+0000 7f2837200640 1 -- 172.21.15.16:0/2169440710 shutdown_connections 2024-08-31T06:14:54.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:54.262+0000 7f2837200640 1 -- 172.21.15.16:0/2169440710 wait complete. 2024-08-31T06:14:54.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:54 smithi155 ceph-mon[21742]: pgmap v598: 97 pgs: 97 active+clean; 3.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-08-31T06:14:54.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:54 smithi016 ceph-mon[17194]: pgmap v598: 97 pgs: 97 active+clean; 3.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-08-31T06:14:54.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:54 smithi184 ceph-mon[22331]: pgmap v598: 97 pgs: 97 active+clean; 3.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-08-31T06:14:55.199 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T06:14:55.199 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:14:55.580 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:14:56.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.392+0000 7f6051a00640 1 -- 172.21.15.16:0/1110794474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f604c103920 msgr2=0x7f604c103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:56.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.392+0000 7f6051a00640 1 --2- 172.21.15.16:0/1110794474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f604c103920 0x7f604c103d30 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f603c009900 tx=0x7f603c02a400 comp rx=0 tx=0).stop 2024-08-31T06:14:56.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.392+0000 7f6051a00640 1 -- 172.21.15.16:0/1110794474 shutdown_connections 2024-08-31T06:14:56.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.392+0000 7f6051a00640 1 --2- 172.21.15.16:0/1110794474 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f604c109bb0 0x7f604c1103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:56.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.392+0000 7f6051a00640 1 --2- 172.21.15.16:0/1110794474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f604c109260 0x7f604c109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:56.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.392+0000 7f6051a00640 1 --2- 172.21.15.16:0/1110794474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f604c103920 0x7f604c103d30 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:56.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.392+0000 7f6051a00640 1 -- 172.21.15.16:0/1110794474 >> 172.21.15.16:0/1110794474 conn(0x7f604c0ff370 msgr2=0x7f604c1017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:56.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.392+0000 7f6051a00640 1 -- 172.21.15.16:0/1110794474 shutdown_connections 2024-08-31T06:14:56.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.392+0000 7f6051a00640 1 -- 172.21.15.16:0/1110794474 wait complete. 2024-08-31T06:14:56.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.394+0000 7f6051a00640 1 Processor -- start 2024-08-31T06:14:56.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.394+0000 7f6051a00640 1 -- start start 2024-08-31T06:14:56.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.394+0000 7f6051a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f604c103920 0x7f604c19a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:56.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.394+0000 7f6051a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f604c109260 0x7f604c19a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:56.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.394+0000 7f6051a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f604c109bb0 0x7f604c19fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:56.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.394+0000 7f6051a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f604c1138c0 con 0x7f604c103920 2024-08-31T06:14:56.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.394+0000 7f6051a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f604c113740 con 0x7f604c109260 2024-08-31T06:14:56.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.394+0000 7f6051a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f604c113a40 con 0x7f604c109bb0 2024-08-31T06:14:56.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.395+0000 7f604aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f604c103920 0x7f604c19a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:56.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.395+0000 7f604aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f604c103920 0x7f604c19a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53212/0 (socket says 172.21.15.16:53212) 2024-08-31T06:14:56.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.395+0000 7f604aa00640 1 -- 172.21.15.16:0/1376979078 learned_addr learned my addr 172.21.15.16:0/1376979078 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:56.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.395+0000 7f604b400640 1 --2- 172.21.15.16:0/1376979078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f604c109bb0 0x7f604c19fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:56.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.395+0000 7f604a000640 1 --2- 172.21.15.16:0/1376979078 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f604c109260 0x7f604c19a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:56.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.395+0000 7f604aa00640 1 -- 172.21.15.16:0/1376979078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f604c109bb0 msgr2=0x7f604c19fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:56.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.395+0000 7f604aa00640 1 --2- 172.21.15.16:0/1376979078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f604c109bb0 0x7f604c19fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:56.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.395+0000 7f604aa00640 1 -- 172.21.15.16:0/1376979078 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f604c109260 msgr2=0x7f604c19a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:56.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.395+0000 7f604aa00640 1 --2- 172.21.15.16:0/1376979078 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f604c109260 0x7f604c19a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:56.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.395+0000 7f604aa00640 1 -- 172.21.15.16:0/1376979078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f604c1a0180 con 0x7f604c103920 2024-08-31T06:14:56.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.395+0000 7f604a000640 1 --2- 172.21.15.16:0/1376979078 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f604c109260 0x7f604c19a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:14:56.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.396+0000 7f604aa00640 1 --2- 172.21.15.16:0/1376979078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f604c103920 0x7f604c19a210 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f603c005ab0 tx=0x7f603c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:56.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.396+0000 7f6043400640 1 -- 172.21.15.16:0/1376979078 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603c03f070 con 0x7f604c103920 2024-08-31T06:14:56.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.396+0000 7f6043400640 1 -- 172.21.15.16:0/1376979078 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f603c031b60 con 0x7f604c103920 2024-08-31T06:14:56.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.396+0000 7f6051a00640 1 -- 172.21.15.16:0/1376979078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f604c1a0410 con 0x7f604c103920 2024-08-31T06:14:56.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.396+0000 7f6043400640 1 -- 172.21.15.16:0/1376979078 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603c035030 con 0x7f604c103920 2024-08-31T06:14:56.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.396+0000 7f6051a00640 1 -- 172.21.15.16:0/1376979078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f604c1a6bd0 con 0x7f604c103920 2024-08-31T06:14:56.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.397+0000 7f6051a00640 1 -- 172.21.15.16:0/1376979078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f604c103d30 con 0x7f604c103920 2024-08-31T06:14:56.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.400+0000 7f6043400640 1 -- 172.21.15.16:0/1376979078 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f603c042070 con 0x7f604c103920 2024-08-31T06:14:56.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.401+0000 7f6043400640 1 --2- 172.21.15.16:0/1376979078 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f60300b4d20 0x7f60300b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:56.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.401+0000 7f6043400640 1 -- 172.21.15.16:0/1376979078 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7f603c1058b0 con 0x7f604c103920 2024-08-31T06:14:56.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.401+0000 7f604a000640 1 --2- 172.21.15.16:0/1376979078 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f60300b4d20 0x7f60300b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:56.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.402+0000 7f604a000640 1 --2- 172.21.15.16:0/1376979078 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f60300b4d20 0x7f60300b7230 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f604c108bb0 tx=0x7f6038005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:56.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.406+0000 7f6043400640 1 -- 172.21.15.16:0/1376979078 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f603c0cc020 con 0x7f604c103920 2024-08-31T06:14:56.629 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 450.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-31T06:14:56.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.640+0000 7f6051a00640 1 -- 172.21.15.16:0/1376979078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f604c02e430 con 0x7f604c103920 2024-08-31T06:14:56.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.641+0000 7f6043400640 1 -- 172.21.15.16:0/1376979078 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11920 (secure 0 0 0) 0x7f603c0d07e0 con 0x7f604c103920 2024-08-31T06:14:56.645 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:14:56.645 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":39,"btime":"2024-08-31T06:14:07:075654+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:14:07.025476+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24737,24724,24712]},"id":1}]} 2024-08-31T06:14:56.645 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 39 2024-08-31T06:14:56.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.644+0000 7f6051a00640 1 -- 172.21.15.16:0/1376979078 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f60300b4d20 msgr2=0x7f60300b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:56.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.644+0000 7f6051a00640 1 --2- 172.21.15.16:0/1376979078 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f60300b4d20 0x7f60300b7230 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f604c108bb0 tx=0x7f6038005f90 comp rx=0 tx=0).stop 2024-08-31T06:14:56.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.644+0000 7f6051a00640 1 -- 172.21.15.16:0/1376979078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f604c103920 msgr2=0x7f604c19a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:56.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.644+0000 7f6051a00640 1 --2- 172.21.15.16:0/1376979078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f604c103920 0x7f604c19a210 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f603c005ab0 tx=0x7f603c003c10 comp rx=0 tx=0).stop 2024-08-31T06:14:56.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.644+0000 7f6051a00640 1 -- 172.21.15.16:0/1376979078 shutdown_connections 2024-08-31T06:14:56.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.644+0000 7f6051a00640 1 --2- 172.21.15.16:0/1376979078 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f60300b4d20 0x7f60300b7230 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:56.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.644+0000 7f6051a00640 1 --2- 172.21.15.16:0/1376979078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f604c109bb0 0x7f604c19fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:56.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.644+0000 7f6051a00640 1 --2- 172.21.15.16:0/1376979078 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f604c109260 0x7f604c19a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:56.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.644+0000 7f6051a00640 1 --2- 172.21.15.16:0/1376979078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f604c103920 0x7f604c19a210 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:56.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.644+0000 7f6051a00640 1 -- 172.21.15.16:0/1376979078 >> 172.21.15.16:0/1376979078 conn(0x7f604c0ff370 msgr2=0x7f604c101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:56.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.644+0000 7f6051a00640 1 -- 172.21.15.16:0/1376979078 shutdown_connections 2024-08-31T06:14:56.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:56.644+0000 7f6051a00640 1 -- 172.21.15.16:0/1376979078 wait complete. 2024-08-31T06:14:56.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:56 smithi155 ceph-mon[21742]: pgmap v599: 97 pgs: 97 active+clean; 3.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 779 KiB/s wr, 84 op/s 2024-08-31T06:14:56.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:56 smithi016 ceph-mon[17194]: pgmap v599: 97 pgs: 97 active+clean; 3.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 779 KiB/s wr, 84 op/s 2024-08-31T06:14:56.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:56 smithi184 ceph-mon[22331]: pgmap v599: 97 pgs: 97 active+clean; 3.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 779 KiB/s wr, 84 op/s 2024-08-31T06:14:57.547 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T06:14:57.601 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:57 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/1376979078' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:14:57.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:57 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/1376979078' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:14:57.942 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:14:57.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:57 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/1376979078' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:14:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.180+0000 7f5045c00640 1 -- 172.21.15.16:0/2034659065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50401036b0 msgr2=0x7f5040103ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.180+0000 7f5045c00640 1 --2- 172.21.15.16:0/2034659065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50401036b0 0x7f5040103ac0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f5030009900 tx=0x7f503002a410 comp rx=0 tx=0).stop 2024-08-31T06:14:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.180+0000 7f5045c00640 1 -- 172.21.15.16:0/2034659065 shutdown_connections 2024-08-31T06:14:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.180+0000 7f5045c00640 1 --2- 172.21.15.16:0/2034659065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5040108fe0 0x7f50401093f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.180+0000 7f5045c00640 1 --2- 172.21.15.16:0/2034659065 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5040109930 0x7f5040110160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.180+0000 7f5045c00640 1 --2- 172.21.15.16:0/2034659065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50401036b0 0x7f5040103ac0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.184 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.180+0000 7f5045c00640 1 -- 172.21.15.16:0/2034659065 >> 172.21.15.16:0/2034659065 conn(0x7f50400ff0d0 msgr2=0x7f5040101520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:58.184 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.180+0000 7f5045c00640 1 -- 172.21.15.16:0/2034659065 shutdown_connections 2024-08-31T06:14:58.184 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.181+0000 7f5045c00640 1 -- 172.21.15.16:0/2034659065 wait complete. 2024-08-31T06:14:58.184 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.182+0000 7f5045c00640 1 Processor -- start 2024-08-31T06:14:58.184 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.182+0000 7f5045c00640 1 -- start start 2024-08-31T06:14:58.186 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.183+0000 7f5045c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50401036b0 0x7f504019c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:58.186 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.183+0000 7f5045c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5040108fe0 0x7f504019c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:58.186 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.183+0000 7f5045c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5040109930 0x7f50401a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:58.186 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.183+0000 7f5045c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5040113650 con 0x7f5040109930 2024-08-31T06:14:58.186 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.183+0000 7f5045c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f50401134d0 con 0x7f5040108fe0 2024-08-31T06:14:58.186 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.183+0000 7f5045c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f50401137d0 con 0x7f50401036b0 2024-08-31T06:14:58.186 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.183+0000 7f5045200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5040109930 0x7f50401a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:58.186 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.183+0000 7f503f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5040108fe0 0x7f504019c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:58.186 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.183+0000 7f5045200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5040109930 0x7f50401a1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53228/0 (socket says 172.21.15.16:53228) 2024-08-31T06:14:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.183+0000 7f5045200640 1 -- 172.21.15.16:0/3983924509 learned_addr learned my addr 172.21.15.16:0/3983924509 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.183+0000 7f503f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5040108fe0 0x7f504019c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:35168/0 (socket says 172.21.15.16:35168) 2024-08-31T06:14:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.183+0000 7f503fe00640 1 --2- 172.21.15.16:0/3983924509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50401036b0 0x7f504019c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.184+0000 7f503f400640 1 -- 172.21.15.16:0/3983924509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50401036b0 msgr2=0x7f504019c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.184+0000 7f503f400640 1 --2- 172.21.15.16:0/3983924509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50401036b0 0x7f504019c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.184+0000 7f503f400640 1 -- 172.21.15.16:0/3983924509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5040109930 msgr2=0x7f50401a1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.184+0000 7f503f400640 1 --2- 172.21.15.16:0/3983924509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5040109930 0x7f50401a1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.184+0000 7f503f400640 1 -- 172.21.15.16:0/3983924509 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50401a21e0 con 0x7f5040108fe0 2024-08-31T06:14:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.184+0000 7f503f400640 1 --2- 172.21.15.16:0/3983924509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5040108fe0 0x7f504019c850 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f504006c430 tx=0x7f502c00ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.184+0000 7f503cc00640 1 -- 172.21.15.16:0/3983924509 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502c00cc40 con 0x7f5040108fe0 2024-08-31T06:14:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.184+0000 7f5045c00640 1 -- 172.21.15.16:0/3983924509 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50401a24d0 con 0x7f5040108fe0 2024-08-31T06:14:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.184+0000 7f503cc00640 1 -- 172.21.15.16:0/3983924509 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f502c00cde0 con 0x7f5040108fe0 2024-08-31T06:14:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.184+0000 7f503cc00640 1 -- 172.21.15.16:0/3983924509 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502c004a00 con 0x7f5040108fe0 2024-08-31T06:14:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.185+0000 7f5045c00640 1 -- 172.21.15.16:0/3983924509 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50401a8c60 con 0x7f5040108fe0 2024-08-31T06:14:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.188+0000 7f504e267740 1 Processor -- start 2024-08-31T06:14:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.188+0000 7f504e267740 1 -- start start 2024-08-31T06:14:58.191 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.188+0000 7f504e267740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a745fba1b0 0x55a74635db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:58.192 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.188+0000 7f504e267740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a745f912f0 0x55a74635e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:58.192 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.188+0000 7f504e267740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a745f994d0 0x55a74635e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:58.192 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.188+0000 7f504e267740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a7462d2f70 con 0x55a745f994d0 2024-08-31T06:14:58.192 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.188+0000 7f504e267740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a7462c2500 con 0x55a745fba1b0 2024-08-31T06:14:58.192 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.188+0000 7f504e267740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a7462c64b0 con 0x55a745f912f0 2024-08-31T06:14:58.192 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.188+0000 7f503cc00640 1 -- 172.21.15.16:0/3983924509 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f502c004ba0 con 0x7f5040108fe0 2024-08-31T06:14:58.192 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.189+0000 7f5045200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a745fba1b0 0x55a74635db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:58.192 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.189+0000 7f503fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a745f912f0 0x55a74635e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:58.192 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.189+0000 7f5045200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a745fba1b0 0x55a74635db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:35180/0 (socket says 172.21.15.16:35180) 2024-08-31T06:14:58.192 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.189+0000 7f5045200640 1 -- 172.21.15.16:0/2839283364 learned_addr learned my addr 172.21.15.16:0/2839283364 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:58.192 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.189+0000 7f503f400640 1 --2- 172.21.15.16:0/2839283364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a745f994d0 0x55a74635e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:58.192 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.189+0000 7f5045200640 1 -- 172.21.15.16:0/2839283364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a745f912f0 msgr2=0x55a74635e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:58.192 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.189+0000 7f5045200640 1 --2- 172.21.15.16:0/2839283364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a745f912f0 0x55a74635e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.193 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.189+0000 7f5045200640 1 -- 172.21.15.16:0/2839283364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a745f994d0 msgr2=0x55a74635e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:58.193 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.189+0000 7f5045200640 1 --2- 172.21.15.16:0/2839283364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a745f994d0 0x55a74635e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.193 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.189+0000 7f5045200640 1 -- 172.21.15.16:0/2839283364 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a74627cc10 con 0x55a745fba1b0 2024-08-31T06:14:58.193 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.189+0000 7f503f400640 1 --2- 172.21.15.16:0/2839283364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a745f994d0 0x55a74635e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:14:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.190+0000 7f5045200640 1 --2- 172.21.15.16:0/2839283364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a745fba1b0 0x55a74635db00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f50380115a0 tx=0x7f5038011a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54645a7dbf51ff96 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.190+0000 7f5035600640 1 -- 172.21.15.16:0/2839283364 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503801e070 con 0x55a745fba1b0 2024-08-31T06:14:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.190+0000 7f5035600640 1 -- 172.21.15.16:0/2839283364 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f503800fa90 con 0x55a745fba1b0 2024-08-31T06:14:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.190+0000 7f5035600640 1 -- 172.21.15.16:0/2839283364 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038019c10 con 0x55a745fba1b0 2024-08-31T06:14:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.190+0000 7f503cc00640 1 --2- 172.21.15.16:0/3983924509 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f50240b4ca0 0x7f50240b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.190+0000 7f503cc00640 1 -- 172.21.15.16:0/3983924509 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7f502c0e54b0 con 0x7f5040108fe0 2024-08-31T06:14:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.190+0000 7f503fe00640 1 --2- 172.21.15.16:0/3983924509 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f50240b4ca0 0x7f50240b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.191+0000 7f504e267740 1 -- 172.21.15.16:0/2839283364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a745fba1b0 msgr2=0x55a74635db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.191+0000 7f504e267740 1 --2- 172.21.15.16:0/2839283364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a745fba1b0 0x55a74635db00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f50380115a0 tx=0x7f5038011a60 comp rx=0 tx=0).stop 2024-08-31T06:14:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.191+0000 7f504e267740 1 -- 172.21.15.16:0/2839283364 shutdown_connections 2024-08-31T06:14:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.191+0000 7f504e267740 1 --2- 172.21.15.16:0/2839283364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a745f912f0 0x55a74635e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.191+0000 7f504e267740 1 --2- 172.21.15.16:0/2839283364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a745f994d0 0x55a74635e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.191+0000 7f504e267740 1 --2- 172.21.15.16:0/2839283364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a745fba1b0 0x55a74635db00 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.191+0000 7f504e267740 1 -- 172.21.15.16:0/2839283364 >> 172.21.15.16:0/2839283364 conn(0x55a745fae300 msgr2=0x55a74635b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.191+0000 7f504e267740 1 -- 172.21.15.16:0/2839283364 shutdown_connections 2024-08-31T06:14:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.191+0000 7f504e267740 1 -- 172.21.15.16:0/2839283364 wait complete. 2024-08-31T06:14:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.191+0000 7f503fe00640 1 --2- 172.21.15.16:0/3983924509 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f50240b4ca0 0x7f50240b71b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5030005ab0 tx=0x7f5030005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.192+0000 7f504e267740 1 -- start start 2024-08-31T06:14:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.192+0000 7f504e267740 1 Processor -- start 2024-08-31T06:14:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.192+0000 7f504e267740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a745f912f0 0x55a746369060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.192+0000 7f504e267740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a745f994d0 0x55a7463695a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.192+0000 7f504e267740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a745fba1b0 0x55a746369ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.192+0000 7f504e267740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a7461fbab0 con 0x55a745fba1b0 2024-08-31T06:14:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.192+0000 7f504e267740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a746269320 con 0x55a745f912f0 2024-08-31T06:14:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.192+0000 7f504e267740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a74636c280 con 0x55a745f994d0 2024-08-31T06:14:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.193+0000 7f504e267740 10 client.0 _ll_register_callbacks cb 0x55a74626f6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:14:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.193+0000 7f503f400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a745f994d0 0x55a7463695a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.193+0000 7f5045200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a745fba1b0 0x55a746369ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.193+0000 7f503fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a745f912f0 0x55a746369060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.193+0000 7f503f400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a745f994d0 0x55a7463695a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:39558/0 (socket says 172.21.15.16:39558) 2024-08-31T06:14:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.193+0000 7f503f400640 1 -- 172.21.15.16:0/3750707468 learned_addr learned my addr 172.21.15.16:0/3750707468 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.193+0000 7f503f400640 1 -- 172.21.15.16:0/3750707468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a745f912f0 msgr2=0x55a746369060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.193+0000 7f503f400640 1 --2- 172.21.15.16:0/3750707468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a745f912f0 0x55a746369060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.193+0000 7f503f400640 1 -- 172.21.15.16:0/3750707468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a745fba1b0 msgr2=0x55a746369ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.193+0000 7f503f400640 1 --2- 172.21.15.16:0/3750707468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a745fba1b0 0x55a746369ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.193+0000 7f503f400640 1 -- 172.21.15.16:0/3750707468 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a746025790 con 0x55a745f994d0 2024-08-31T06:14:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.194+0000 7f503f400640 1 --2- 172.21.15.16:0/3750707468 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a745f994d0 0x55a7463695a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f502c0abb00 tx=0x7f502c0a7a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34f8b0a392477365 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.194+0000 7f502a000640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:14:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.194+0000 7f502a000640 1 -- 172.21.15.16:0/3750707468 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502c01c070 con 0x55a745f994d0 2024-08-31T06:14:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.194+0000 7f504e267740 1 -- 172.21.15.16:0/3750707468 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f503800fa90 con 0x55a745f994d0 2024-08-31T06:14:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.194+0000 7f502a000640 1 -- 172.21.15.16:0/3750707468 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7f502c0b0170 con 0x55a745f994d0 2024-08-31T06:14:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.194+0000 7f502a000640 1 -- 172.21.15.16:0/3750707468 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f502c0ab2e0 con 0x55a745f994d0 2024-08-31T06:14:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.194+0000 7f504e267740 10 client.27304 fetch_fsmap learned FSMap version 39 2024-08-31T06:14:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.194+0000 7f504e267740 1 -- 172.21.15.16:0/3750707468 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a74609e620 con 0x55a745f994d0 2024-08-31T06:14:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.194+0000 7f502a000640 1 -- 172.21.15.16:0/3750707468 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f503800fa90 con 0x55a745f994d0 2024-08-31T06:14:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.194+0000 7f502a000640 1 -- 172.21.15.16:0/3750707468 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f502c0e2e50 con 0x55a745f994d0 2024-08-31T06:14:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.195+0000 7f504e267740 10 client.27304 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T06:14:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.195+0000 7f504e267740 10 client.27304 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:14:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.195+0000 7f504e267740 20 client.27304 populate_metadata read hostname 'smithi016' 2024-08-31T06:14:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.195+0000 7f504e267740 1 --2- 172.21.15.16:0/3750707468 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a745f78d80 0x55a74637b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.195+0000 7f504e267740 4 client.27304 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:14:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.195+0000 7f504e267740 1 -- 172.21.15.16:0/3750707468 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7462d4ec0 con 0x55a745f78d80 2024-08-31T06:14:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.195+0000 7f503fe00640 1 --2- 172.21.15.16:0/3750707468 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a745f78d80 0x55a74637b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.196+0000 7f503fe00640 1 --2- 172.21.15.16:0/3750707468 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a745f78d80 0x55a74637b1f0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24b36c0f6c299247 server_cookie=a94de9e54f38860c in_seq=0 out_seq=0 2024-08-31T06:14:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.196+0000 7f502a000640 10 client.27304 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.196+0000 7f502a000640 1 -- 172.21.15.16:0/3750707468 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a7462d4ec0 con 0x55a745f78d80 2024-08-31T06:14:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.196+0000 7f502a000640 10 client.27304 handle_command_reply: tid=0 2024-08-31T06:14:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.196+0000 7f504e267740 1 client.27304 shutdown 2024-08-31T06:14:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.197+0000 7f504e267740 1 -- 172.21.15.16:0/3750707468 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a745f994d0 msgr2=0x55a7463695a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.197+0000 7f504e267740 1 --2- 172.21.15.16:0/3750707468 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a745f994d0 0x55a7463695a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f502c0abb00 tx=0x7f502c0a7a30 comp rx=0 tx=0).stop 2024-08-31T06:14:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.197+0000 7f504e267740 1 -- 172.21.15.16:0/3750707468 shutdown_connections 2024-08-31T06:14:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.197+0000 7f504e267740 1 --2- 172.21.15.16:0/3750707468 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a745f78d80 0x55a74637b1f0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.197+0000 7f504e267740 1 --2- 172.21.15.16:0/3750707468 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a745f994d0 0x55a7463695a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.197+0000 7f504e267740 1 --2- 172.21.15.16:0/3750707468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a745fba1b0 0x55a746369ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.197+0000 7f502a000640 0 client.27304 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.197+0000 7f504e267740 1 --2- 172.21.15.16:0/3750707468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a745f912f0 0x55a746369060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.197+0000 7f504e267740 1 -- 172.21.15.16:0/3750707468 >> 172.21.15.16:0/3750707468 conn(0x55a745fae300 msgr2=0x55a74635eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.197+0000 7f504e267740 1 -- 172.21.15.16:0/3750707468 shutdown_connections 2024-08-31T06:14:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.197+0000 7f504e267740 1 -- 172.21.15.16:0/3750707468 wait complete. 2024-08-31T06:14:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.197+0000 7f504e267740 20 client.27304 trim_cache size 0 max 16384 2024-08-31T06:14:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.213+0000 7f504e267740 1 Processor -- start 2024-08-31T06:14:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.213+0000 7f504e267740 1 -- start start 2024-08-31T06:14:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.213+0000 7f504e267740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a745f912f0 0x55a74635f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.213+0000 7f504e267740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a745f994d0 0x55a74636ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.213+0000 7f504e267740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a745fba1b0 0x55a746366230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.213+0000 7f504e267740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f50401b3d80 con 0x55a745f912f0 2024-08-31T06:14:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.213+0000 7f504e267740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f50401b15f0 con 0x55a745fba1b0 2024-08-31T06:14:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.213+0000 7f504e267740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f50401af460 con 0x55a745f994d0 2024-08-31T06:14:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.213+0000 7f503fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a745f912f0 0x55a74635f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.213+0000 7f503fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a745f912f0 0x55a74635f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53246/0 (socket says 172.21.15.16:53246) 2024-08-31T06:14:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.213+0000 7f503fe00640 1 -- 172.21.15.16:0/3710362390 learned_addr learned my addr 172.21.15.16:0/3710362390 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.214+0000 7f503f400640 1 --2- 172.21.15.16:0/3710362390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a745f994d0 0x55a74636ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.214+0000 7f5045200640 1 --2- 172.21.15.16:0/3710362390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a745fba1b0 0x55a746366230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.214+0000 7f503f400640 1 -- 172.21.15.16:0/3710362390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a745fba1b0 msgr2=0x55a746366230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.214+0000 7f503f400640 1 --2- 172.21.15.16:0/3710362390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a745fba1b0 0x55a746366230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.214+0000 7f503f400640 1 -- 172.21.15.16:0/3710362390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a745f912f0 msgr2=0x55a74635f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.214+0000 7f503f400640 1 --2- 172.21.15.16:0/3710362390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a745f912f0 0x55a74635f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.214+0000 7f503f400640 1 -- 172.21.15.16:0/3710362390 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a746360910 con 0x55a745f994d0 2024-08-31T06:14:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.214+0000 7f503fe00640 1 --2- 172.21.15.16:0/3710362390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a745f912f0 0x55a74635f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:14:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.214+0000 7f503f400640 1 --2- 172.21.15.16:0/3710362390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a745f994d0 0x55a74636ac90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f502c00c6a0 tx=0x7f502c0a7b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=162a525a10603467 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.214+0000 7f5034c00640 1 -- 172.21.15.16:0/3710362390 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502c0c5070 con 0x55a745f994d0 2024-08-31T06:14:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.215+0000 7f5034c00640 1 -- 172.21.15.16:0/3710362390 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f502c0ac270 con 0x55a745f994d0 2024-08-31T06:14:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.215+0000 7f5034c00640 1 -- 172.21.15.16:0/3710362390 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502c01c070 con 0x55a745f994d0 2024-08-31T06:14:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.215+0000 7f504e267740 1 -- 172.21.15.16:0/3710362390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a745f994d0 msgr2=0x55a74636ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.215+0000 7f504e267740 1 --2- 172.21.15.16:0/3710362390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a745f994d0 0x55a74636ac90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f502c00c6a0 tx=0x7f502c0a7b60 comp rx=0 tx=0).stop 2024-08-31T06:14:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.215+0000 7f504e267740 1 -- 172.21.15.16:0/3710362390 shutdown_connections 2024-08-31T06:14:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.215+0000 7f504e267740 1 --2- 172.21.15.16:0/3710362390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a745f994d0 0x55a74636ac90 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.215+0000 7f504e267740 1 --2- 172.21.15.16:0/3710362390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a745fba1b0 0x55a746366230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.215+0000 7f504e267740 1 --2- 172.21.15.16:0/3710362390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a745f912f0 0x55a74635f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.215+0000 7f504e267740 1 -- 172.21.15.16:0/3710362390 >> 172.21.15.16:0/3710362390 conn(0x55a745fae300 msgr2=0x55a74636a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.216+0000 7f504e267740 1 -- 172.21.15.16:0/3710362390 shutdown_connections 2024-08-31T06:14:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.216+0000 7f504e267740 1 -- 172.21.15.16:0/3710362390 wait complete. 2024-08-31T06:14:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.216+0000 7f504e267740 1 -- start start 2024-08-31T06:14:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.216+0000 7f504e267740 1 Processor -- start 2024-08-31T06:14:58.222 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.217+0000 7f504e267740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a745f912f0 0x55a746371bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.217+0000 7f504e267740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a745f994d0 0x55a74636f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.217+0000 7f504e267740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a745fba1b0 0x55a74636f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.217+0000 7f504e267740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f50401afab0 con 0x55a745fba1b0 2024-08-31T06:14:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.217+0000 7f504e267740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f50401b0f20 con 0x55a745f912f0 2024-08-31T06:14:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.217+0000 7f504e267740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f50401b0180 con 0x55a745f994d0 2024-08-31T06:14:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.217+0000 7f504e267740 10 client.0 _ll_register_callbacks cb 0x7f502c093050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:14:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.217+0000 7f503fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a745f912f0 0x55a746371bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.217+0000 7f5045200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a745fba1b0 0x55a74636f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.217+0000 7f503fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a745f912f0 0x55a746371bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:35216/0 (socket says 172.21.15.16:35216) 2024-08-31T06:14:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.217+0000 7f503fe00640 1 -- 172.21.15.16:0/1813818360 learned_addr learned my addr 172.21.15.16:0/1813818360 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.217+0000 7f503f400640 1 --2- 172.21.15.16:0/1813818360 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a745f994d0 0x55a74636f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:58.224 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.217+0000 7f503fe00640 1 -- 172.21.15.16:0/1813818360 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a745f994d0 msgr2=0x55a74636f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:58.224 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.217+0000 7f503fe00640 1 --2- 172.21.15.16:0/1813818360 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a745f994d0 0x55a74636f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.224 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.217+0000 7f503fe00640 1 -- 172.21.15.16:0/1813818360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a745fba1b0 msgr2=0x55a74636f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:58.224 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.217+0000 7f503fe00640 1 --2- 172.21.15.16:0/1813818360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a745fba1b0 0x55a74636f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.224 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.217+0000 7f503fe00640 1 -- 172.21.15.16:0/1813818360 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a74636fcb0 con 0x55a745f912f0 2024-08-31T06:14:58.224 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.217+0000 7f5045200640 1 --2- 172.21.15.16:0/1813818360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a745fba1b0 0x55a74636f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:14:58.224 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.218+0000 7f503fe00640 1 --2- 172.21.15.16:0/1813818360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a745f912f0 0x55a746371bf0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f503002aa00 tx=0x7f5030009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97ea7ef268c130f8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:58.224 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.218+0000 7f502a000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:14:58.224 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.218+0000 7f502a000640 1 -- 172.21.15.16:0/1813818360 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5030060060 con 0x55a745f912f0 2024-08-31T06:14:58.224 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.218+0000 7f502a000640 1 -- 172.21.15.16:0/1813818360 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7f5030004c40 con 0x55a745f912f0 2024-08-31T06:14:58.224 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.218+0000 7f504e267740 1 -- 172.21.15.16:0/1813818360 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f502c0ac270 con 0x55a745f912f0 2024-08-31T06:14:58.224 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.218+0000 7f502a000640 1 -- 172.21.15.16:0/1813818360 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5030035040 con 0x55a745f912f0 2024-08-31T06:14:58.224 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.218+0000 7f504e267740 10 client.27329 fetch_fsmap learned FSMap version 39 2024-08-31T06:14:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.218+0000 7f504e267740 1 -- 172.21.15.16:0/1813818360 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a7463e5a80 con 0x55a745f912f0 2024-08-31T06:14:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.219+0000 7f502a000640 1 -- 172.21.15.16:0/1813818360 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f502c0ac270 con 0x55a745f912f0 2024-08-31T06:14:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.219+0000 7f502a000640 1 -- 172.21.15.16:0/1813818360 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f50300325d0 con 0x55a745f912f0 2024-08-31T06:14:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.219+0000 7f504e267740 10 client.27329 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T06:14:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.219+0000 7f504e267740 10 client.27329 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:14:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.219+0000 7f504e267740 20 client.27329 populate_metadata read hostname 'smithi016' 2024-08-31T06:14:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.219+0000 7f504e267740 1 --2- 172.21.15.16:0/1813818360 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a745f4e1c0 0x55a7463e8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.219+0000 7f504e267740 4 client.27329 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:14:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.219+0000 7f504e267740 1 -- 172.21.15.16:0/1813818360 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a7462d0960 con 0x55a745f4e1c0 2024-08-31T06:14:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.220+0000 7f503f400640 1 --2- 172.21.15.16:0/1813818360 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a745f4e1c0 0x55a7463e8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.220+0000 7f503f400640 1 --2- 172.21.15.16:0/1813818360 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a745f4e1c0 0x55a7463e8280 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbe37d0b78c279dd server_cookie=ef927e1c36e48ff8 in_seq=0 out_seq=0 2024-08-31T06:14:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.220+0000 7f502a000640 10 client.27329 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.234+0000 7f502a000640 1 -- 172.21.15.16:0/1813818360 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a7462d0960 con 0x55a745f4e1c0 2024-08-31T06:14:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.234+0000 7f502a000640 10 client.27329 handle_command_reply: tid=0 2024-08-31T06:14:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.234+0000 7f504e267740 1 client.27329 shutdown 2024-08-31T06:14:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.235+0000 7f504e267740 1 -- 172.21.15.16:0/1813818360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a745f912f0 msgr2=0x55a746371bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.235+0000 7f504e267740 1 --2- 172.21.15.16:0/1813818360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a745f912f0 0x55a746371bf0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f503002aa00 tx=0x7f5030009260 comp rx=0 tx=0).stop 2024-08-31T06:14:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.235+0000 7f504e267740 1 -- 172.21.15.16:0/1813818360 shutdown_connections 2024-08-31T06:14:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.235+0000 7f504e267740 1 --2- 172.21.15.16:0/1813818360 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a745f4e1c0 0x55a7463e8280 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.235+0000 7f504e267740 1 --2- 172.21.15.16:0/1813818360 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a745f994d0 0x55a74636f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.235+0000 7f504e267740 1 --2- 172.21.15.16:0/1813818360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a745fba1b0 0x55a74636f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.235+0000 7f504e267740 1 --2- 172.21.15.16:0/1813818360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a745f912f0 0x55a746371bf0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.235+0000 7f502a000640 0 client.27329 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:14:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.235+0000 7f504e267740 1 -- 172.21.15.16:0/1813818360 >> 172.21.15.16:0/1813818360 conn(0x55a745fae300 msgr2=0x55a74636ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.235+0000 7f504e267740 1 -- 172.21.15.16:0/1813818360 shutdown_connections 2024-08-31T06:14:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.235+0000 7f504e267740 1 -- 172.21.15.16:0/1813818360 wait complete. 2024-08-31T06:14:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.235+0000 7f504e267740 20 client.27329 trim_cache size 0 max 16384 2024-08-31T06:14:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:14:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:14:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:14:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:14:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.236+0000 7f5045c00640 1 -- 172.21.15.16:0/3983924509 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f50240b4ca0 msgr2=0x7f50240b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.236+0000 7f5045c00640 1 --2- 172.21.15.16:0/3983924509 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f50240b4ca0 0x7f50240b71b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5030005ab0 tx=0x7f5030005a20 comp rx=0 tx=0).stop 2024-08-31T06:14:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.236+0000 7f5045c00640 1 -- 172.21.15.16:0/3983924509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5040108fe0 msgr2=0x7f504019c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.236+0000 7f5045c00640 1 --2- 172.21.15.16:0/3983924509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5040108fe0 0x7f504019c850 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f504006c430 tx=0x7f502c00ee50 comp rx=0 tx=0).stop 2024-08-31T06:14:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.236+0000 7f5045c00640 1 -- 172.21.15.16:0/3983924509 shutdown_connections 2024-08-31T06:14:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.236+0000 7f5045c00640 1 --2- 172.21.15.16:0/3983924509 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f50240b4ca0 0x7f50240b71b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.236+0000 7f5045c00640 1 --2- 172.21.15.16:0/3983924509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5040109930 0x7f50401a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.236+0000 7f5045c00640 1 --2- 172.21.15.16:0/3983924509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5040108fe0 0x7f504019c850 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.236+0000 7f5045c00640 1 --2- 172.21.15.16:0/3983924509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50401036b0 0x7f504019c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.236+0000 7f5045c00640 1 -- 172.21.15.16:0/3983924509 >> 172.21.15.16:0/3983924509 conn(0x7f50400ff0d0 msgr2=0x7f50401086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.236+0000 7f5045c00640 1 -- 172.21.15.16:0/3983924509 shutdown_connections 2024-08-31T06:14:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:14:58.236+0000 7f5045c00640 1 -- 172.21.15.16:0/3983924509 wait complete. 2024-08-31T06:14:58.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.640+0000 7fcfbd800640 1 -- 172.21.15.16:0/1865638288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfb8103750 msgr2=0x7fcfb8103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:58.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.640+0000 7fcfbd800640 1 --2- 172.21.15.16:0/1865638288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfb8103750 0x7fcfb8103b60 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8009900 tx=0x7fcfa802a410 comp rx=0 tx=0).stop 2024-08-31T06:14:58.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.640+0000 7fcfbd800640 1 -- 172.21.15.16:0/1865638288 shutdown_connections 2024-08-31T06:14:58.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.640+0000 7fcfbd800640 1 --2- 172.21.15.16:0/1865638288 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfb8109090 0x7fcfb81094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.640+0000 7fcfbd800640 1 --2- 172.21.15.16:0/1865638288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfb81099e0 0x7fcfb8110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.640+0000 7fcfbd800640 1 --2- 172.21.15.16:0/1865638288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfb8103750 0x7fcfb8103b60 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.640+0000 7fcfbd800640 1 -- 172.21.15.16:0/1865638288 >> 172.21.15.16:0/1865638288 conn(0x7fcfb80ff1a0 msgr2=0x7fcfb81015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:58.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.640+0000 7fcfbd800640 1 -- 172.21.15.16:0/1865638288 shutdown_connections 2024-08-31T06:14:58.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.640+0000 7fcfbd800640 1 -- 172.21.15.16:0/1865638288 wait complete. 2024-08-31T06:14:58.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.641+0000 7fcfbd800640 1 Processor -- start 2024-08-31T06:14:58.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.641+0000 7fcfbd800640 1 -- start start 2024-08-31T06:14:58.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.641+0000 7fcfbd800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfb8103750 0x7fcfb80754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:58.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.642+0000 7fcfbd800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfb8109090 0x7fcfb80759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:58.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.642+0000 7fcfbd800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfb81099e0 0x7fcfb8075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:58.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.642+0000 7fcfbd800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb8113700 con 0x7fcfb8109090 2024-08-31T06:14:58.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.642+0000 7fcfbd800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb8113580 con 0x7fcfb8103750 2024-08-31T06:14:58.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.642+0000 7fcfbd800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb8113880 con 0x7fcfb81099e0 2024-08-31T06:14:58.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.642+0000 7fcfaf400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfb8109090 0x7fcfb80759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:58.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.642+0000 7fcfaf400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfb8109090 0x7fcfb80759f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53284/0 (socket says 172.21.15.16:53284) 2024-08-31T06:14:58.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.642+0000 7fcfaf400640 1 -- 172.21.15.16:0/793603367 learned_addr learned my addr 172.21.15.16:0/793603367 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:58.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.642+0000 7fcfb7e00640 1 --2- 172.21.15.16:0/793603367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfb8103750 0x7fcfb80754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:58.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.642+0000 7fcfbce00640 1 --2- 172.21.15.16:0/793603367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfb81099e0 0x7fcfb8075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:58.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.642+0000 7fcfaf400640 1 -- 172.21.15.16:0/793603367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfb81099e0 msgr2=0x7fcfb8075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:58.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.642+0000 7fcfaf400640 1 --2- 172.21.15.16:0/793603367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfb81099e0 0x7fcfb8075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.642+0000 7fcfaf400640 1 -- 172.21.15.16:0/793603367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfb8103750 msgr2=0x7fcfb80754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:58.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.642+0000 7fcfaf400640 1 --2- 172.21.15.16:0/793603367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfb8103750 0x7fcfb80754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.642+0000 7fcfaf400640 1 -- 172.21.15.16:0/793603367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfb81a8370 con 0x7fcfb8109090 2024-08-31T06:14:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.643+0000 7fcfaf400640 1 --2- 172.21.15.16:0/793603367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfb8109090 0x7fcfb80759f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000e9c0 tx=0x7fcfa000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.643+0000 7fcfb5600640 1 -- 172.21.15.16:0/793603367 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa000ccc0 con 0x7fcfb8109090 2024-08-31T06:14:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.643+0000 7fcfb5600640 1 -- 172.21.15.16:0/793603367 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfa00054d0 con 0x7fcfb8109090 2024-08-31T06:14:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.643+0000 7fcfb5600640 1 -- 172.21.15.16:0/793603367 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa0010430 con 0x7fcfb8109090 2024-08-31T06:14:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.643+0000 7fcfbd800640 1 -- 172.21.15.16:0/793603367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfb81a8660 con 0x7fcfb8109090 2024-08-31T06:14:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.643+0000 7fcfbd800640 1 -- 172.21.15.16:0/793603367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfb81a8b20 con 0x7fcfb8109090 2024-08-31T06:14:58.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.647+0000 7fcfc5aba740 1 Processor -- start 2024-08-31T06:14:58.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.647+0000 7fcfc5aba740 1 -- start start 2024-08-31T06:14:58.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.647+0000 7fcfb5600640 1 -- 172.21.15.16:0/793603367 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fcfa0005020 con 0x7fcfb8109090 2024-08-31T06:14:58.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.647+0000 7fcfc5aba740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5655143a0070 0x565514745a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:58.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.647+0000 7fcfc5aba740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5655143771b0 0x565514745fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:58.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.647+0000 7fcfc5aba740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56551437f390 0x5655147464f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:58.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.647+0000 7fcfc5aba740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5655146bc7b0 con 0x56551437f390 2024-08-31T06:14:58.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.647+0000 7fcfc5aba740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5655146abd40 con 0x5655143a0070 2024-08-31T06:14:58.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.647+0000 7fcfc5aba740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5655146afcf0 con 0x5655143771b0 2024-08-31T06:14:58.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.647+0000 7fcfbce00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5655143a0070 0x565514745a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:58.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.647+0000 7fcfbce00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5655143a0070 0x565514745a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:35232/0 (socket says 172.21.15.16:35232) 2024-08-31T06:14:58.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.647+0000 7fcfbce00640 1 -- 172.21.15.16:0/981500754 learned_addr learned my addr 172.21.15.16:0/981500754 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:58.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.648+0000 7fcfb7e00640 1 --2- 172.21.15.16:0/981500754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5655143771b0 0x565514745fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:58.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.648+0000 7fcfaf400640 1 --2- 172.21.15.16:0/981500754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56551437f390 0x5655147464f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:58.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.648+0000 7fcfbce00640 1 -- 172.21.15.16:0/981500754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5655143771b0 msgr2=0x565514745fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:58.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.648+0000 7fcfbce00640 1 --2- 172.21.15.16:0/981500754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5655143771b0 0x565514745fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.648+0000 7fcfbce00640 1 -- 172.21.15.16:0/981500754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56551437f390 msgr2=0x5655147464f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:58.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.648+0000 7fcfbce00640 1 --2- 172.21.15.16:0/981500754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56551437f390 0x5655147464f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.648+0000 7fcfbce00640 1 -- 172.21.15.16:0/981500754 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56551440b650 con 0x5655143a0070 2024-08-31T06:14:58.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.648+0000 7fcfb7e00640 1 --2- 172.21.15.16:0/981500754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5655143771b0 0x565514745fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:14:58.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.648+0000 7fcfbce00640 1 --2- 172.21.15.16:0/981500754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5655143a0070 0x565514745a70 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0009fd0 tx=0x7fcfb00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47c2623b3675e642 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:58.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.649+0000 7fcfad600640 1 -- 172.21.15.16:0/981500754 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0004d20 con 0x5655143a0070 2024-08-31T06:14:58.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.649+0000 7fcfad600640 1 -- 172.21.15.16:0/981500754 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfb0004050 con 0x5655143a0070 2024-08-31T06:14:58.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.649+0000 7fcfad600640 1 -- 172.21.15.16:0/981500754 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0005280 con 0x5655143a0070 2024-08-31T06:14:58.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.649+0000 7fcfb5600640 1 --2- 172.21.15.16:0/793603367 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fcf9c0b4ca0 0x7fcf9c0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:58.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.649+0000 7fcfb5600640 1 -- 172.21.15.16:0/793603367 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7fcfa00e58d0 con 0x7fcfb8109090 2024-08-31T06:14:58.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.649+0000 7fcfb7e00640 1 --2- 172.21.15.16:0/793603367 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fcf9c0b4ca0 0x7fcf9c0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:58.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.650+0000 7fcfc5aba740 1 -- 172.21.15.16:0/981500754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5655143a0070 msgr2=0x565514745a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:58.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.650+0000 7fcfc5aba740 1 --2- 172.21.15.16:0/981500754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5655143a0070 0x565514745a70 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0009fd0 tx=0x7fcfb00118d0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.650+0000 7fcfc5aba740 1 -- 172.21.15.16:0/981500754 shutdown_connections 2024-08-31T06:14:58.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.650+0000 7fcfc5aba740 1 --2- 172.21.15.16:0/981500754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5655143771b0 0x565514745fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.650+0000 7fcfc5aba740 1 --2- 172.21.15.16:0/981500754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56551437f390 0x5655147464f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.650+0000 7fcfc5aba740 1 --2- 172.21.15.16:0/981500754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5655143a0070 0x565514745a70 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.650+0000 7fcfc5aba740 1 -- 172.21.15.16:0/981500754 >> 172.21.15.16:0/981500754 conn(0x565514394460 msgr2=0x565514743960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:58.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.650+0000 7fcfc5aba740 1 -- 172.21.15.16:0/981500754 shutdown_connections 2024-08-31T06:14:58.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.650+0000 7fcfc5aba740 1 -- 172.21.15.16:0/981500754 wait complete. 2024-08-31T06:14:58.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.650+0000 7fcfb7e00640 1 --2- 172.21.15.16:0/793603367 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fcf9c0b4ca0 0x7fcf9c0b71b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8005ab0 tx=0x7fcfa8005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:58.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.651+0000 7fcfc5aba740 1 -- start start 2024-08-31T06:14:58.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.651+0000 7fcfc5aba740 1 Processor -- start 2024-08-31T06:14:58.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.651+0000 7fcfc5aba740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5655143771b0 0x565514751580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:58.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.651+0000 7fcfc5aba740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56551437f390 0x565514751ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:58.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.652+0000 7fcfc5aba740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5655143a0070 0x56551475d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:58.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.652+0000 7fcfc5aba740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56551450e700 con 0x56551437f390 2024-08-31T06:14:58.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.652+0000 7fcfc5aba740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x565514632cf0 con 0x5655143a0070 2024-08-31T06:14:58.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.652+0000 7fcfc5aba740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x565514754250 con 0x5655143771b0 2024-08-31T06:14:58.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.652+0000 7fcfc5aba740 10 client.0 _ll_register_callbacks cb 0x565514658470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:14:58.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.652+0000 7fcfbce00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5655143a0070 0x56551475d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:58.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.652+0000 7fcfaf400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56551437f390 0x565514751ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:58.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.652+0000 7fcfbce00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5655143a0070 0x56551475d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:35248/0 (socket says 172.21.15.16:35248) 2024-08-31T06:14:58.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.652+0000 7fcfbce00640 1 -- 172.21.15.16:0/1067665397 learned_addr learned my addr 172.21.15.16:0/1067665397 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:58.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.652+0000 7fcfb7e00640 1 --2- 172.21.15.16:0/1067665397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5655143771b0 0x565514751580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:58.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.652+0000 7fcfbce00640 1 -- 172.21.15.16:0/1067665397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5655143771b0 msgr2=0x565514751580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:58.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.652+0000 7fcfbce00640 1 --2- 172.21.15.16:0/1067665397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5655143771b0 0x565514751580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.652+0000 7fcfbce00640 1 -- 172.21.15.16:0/1067665397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56551437f390 msgr2=0x565514751ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:58.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.652+0000 7fcfbce00640 1 --2- 172.21.15.16:0/1067665397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56551437f390 0x565514751ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.652+0000 7fcfbce00640 1 -- 172.21.15.16:0/1067665397 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5655146738b0 con 0x5655143a0070 2024-08-31T06:14:58.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.652+0000 7fcfaf400640 1 --2- 172.21.15.16:0/1067665397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56551437f390 0x565514751ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:14:58.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.653+0000 7fcfbce00640 1 --2- 172.21.15.16:0/1067665397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5655143a0070 0x56551475d570 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fcfb001b040 tx=0x7fcfb0004430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60a90ee6737fabe0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:58.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.653+0000 7fcfa6000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:14:58.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.653+0000 7fcfa6000640 1 -- 172.21.15.16:0/1067665397 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0028020 con 0x5655143a0070 2024-08-31T06:14:58.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.653+0000 7fcfa6000640 1 -- 172.21.15.16:0/1067665397 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7fcfb0004c10 con 0x5655143a0070 2024-08-31T06:14:58.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.653+0000 7fcfc5aba740 1 -- 172.21.15.16:0/1067665397 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfb0004050 con 0x5655143a0070 2024-08-31T06:14:58.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.654+0000 7fcfa6000640 1 -- 172.21.15.16:0/1067665397 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcfb0033050 con 0x5655143a0070 2024-08-31T06:14:58.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.654+0000 7fcfc5aba740 10 client.27341 fetch_fsmap learned FSMap version 39 2024-08-31T06:14:58.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.654+0000 7fcfc5aba740 1 -- 172.21.15.16:0/1067665397 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5655147601e0 con 0x5655143a0070 2024-08-31T06:14:58.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.654+0000 7fcfa6000640 1 -- 172.21.15.16:0/1067665397 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfb0004050 con 0x5655143a0070 2024-08-31T06:14:58.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.654+0000 7fcfa6000640 1 -- 172.21.15.16:0/1067665397 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fcfb000f4e0 con 0x5655143a0070 2024-08-31T06:14:58.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.654+0000 7fcfc5aba740 10 client.27341 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T06:14:58.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.654+0000 7fcfc5aba740 10 client.27341 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T06:14:58.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.654+0000 7fcfc5aba740 20 client.27341 populate_metadata read hostname 'smithi016' 2024-08-31T06:14:58.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.654+0000 7fcfc5aba740 1 --2- 172.21.15.16:0/1067665397 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x56551435e9f0 0x565514763650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:58.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.654+0000 7fcfc5aba740 4 client.27341 mds_command: new command op to 24712 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:14:58.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.654+0000 7fcfc5aba740 1 -- 172.21.15.16:0/1067665397 --> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5655146be700 con 0x56551435e9f0 2024-08-31T06:14:58.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.655+0000 7fcfb7e00640 1 --2- 172.21.15.16:0/1067665397 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x56551435e9f0 0x565514763650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:58.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.655+0000 7fcfb7e00640 1 --2- 172.21.15.16:0/1067665397 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x56551435e9f0 0x565514763650 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b902b036daf5dfdd server_cookie=cc1d996f261588f2 in_seq=0 out_seq=0 2024-08-31T06:14:58.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.655+0000 7fcfa6000640 10 client.27341 ms_handle_connect on v2:172.21.15.184:6838/4011051479 2024-08-31T06:14:58.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.656+0000 7fcfa6000640 1 -- 172.21.15.16:0/1067665397 <== mds.2 v2:172.21.15.184:6838/4011051479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5655146be700 con 0x56551435e9f0 2024-08-31T06:14:58.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.656+0000 7fcfa6000640 10 client.27341 handle_command_reply: tid=0 2024-08-31T06:14:58.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.656+0000 7fcfc5aba740 1 client.27341 shutdown 2024-08-31T06:14:58.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.657+0000 7fcfc5aba740 1 -- 172.21.15.16:0/1067665397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5655143a0070 msgr2=0x56551475d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:58.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.657+0000 7fcfc5aba740 1 --2- 172.21.15.16:0/1067665397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5655143a0070 0x56551475d570 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fcfb001b040 tx=0x7fcfb0004430 comp rx=0 tx=0).stop 2024-08-31T06:14:58.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.657+0000 7fcfc5aba740 1 -- 172.21.15.16:0/1067665397 shutdown_connections 2024-08-31T06:14:58.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.657+0000 7fcfc5aba740 1 --2- 172.21.15.16:0/1067665397 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x56551435e9f0 0x565514763650 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.657+0000 7fcfc5aba740 1 --2- 172.21.15.16:0/1067665397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5655143a0070 0x56551475d570 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.657+0000 7fcfc5aba740 1 --2- 172.21.15.16:0/1067665397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56551437f390 0x565514751ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.657+0000 7fcfa6000640 0 client.27341 ms_handle_reset on v2:172.21.15.184:6838/4011051479 2024-08-31T06:14:58.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.657+0000 7fcfc5aba740 1 --2- 172.21.15.16:0/1067665397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5655143771b0 0x565514751580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.657+0000 7fcfc5aba740 1 -- 172.21.15.16:0/1067665397 >> 172.21.15.16:0/1067665397 conn(0x565514394460 msgr2=0x565514748c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:58.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.657+0000 7fcfc5aba740 1 -- 172.21.15.16:0/1067665397 shutdown_connections 2024-08-31T06:14:58.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.657+0000 7fcfc5aba740 1 -- 172.21.15.16:0/1067665397 wait complete. 2024-08-31T06:14:58.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.657+0000 7fcfc5aba740 20 client.27341 trim_cache size 0 max 16384 2024-08-31T06:14:58.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.671+0000 7fcfc5aba740 1 Processor -- start 2024-08-31T06:14:58.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.671+0000 7fcfc5aba740 1 -- start start 2024-08-31T06:14:58.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.672+0000 7fcfc5aba740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5655143771b0 0x5655147471b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:58.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.672+0000 7fcfc5aba740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56551437f390 0x56551475d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:58.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.672+0000 7fcfc5aba740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5655143a0070 0x56551475d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:58.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.672+0000 7fcfc5aba740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb81b3da0 con 0x56551437f390 2024-08-31T06:14:58.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.672+0000 7fcfc5aba740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb81b1610 con 0x5655143a0070 2024-08-31T06:14:58.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.672+0000 7fcfc5aba740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb81af480 con 0x5655143771b0 2024-08-31T06:14:58.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.672+0000 7fcfb7e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5655143771b0 0x5655147471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:58.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.672+0000 7fcfb7e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5655143771b0 0x5655147471b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:39600/0 (socket says 172.21.15.16:39600) 2024-08-31T06:14:58.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.672+0000 7fcfb7e00640 1 -- 172.21.15.16:0/3689803303 learned_addr learned my addr 172.21.15.16:0/3689803303 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:58.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.672+0000 7fcfaf400640 1 --2- 172.21.15.16:0/3689803303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56551437f390 0x56551475d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:58.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.672+0000 7fcfbce00640 1 --2- 172.21.15.16:0/3689803303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5655143a0070 0x56551475d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:58.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.672+0000 7fcfb7e00640 1 -- 172.21.15.16:0/3689803303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5655143a0070 msgr2=0x56551475d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:58.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.672+0000 7fcfb7e00640 1 --2- 172.21.15.16:0/3689803303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5655143a0070 0x56551475d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.672+0000 7fcfb7e00640 1 -- 172.21.15.16:0/3689803303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56551437f390 msgr2=0x56551475d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:58.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.672+0000 7fcfb7e00640 1 --2- 172.21.15.16:0/3689803303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56551437f390 0x56551475d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.672+0000 7fcfb7e00640 1 -- 172.21.15.16:0/3689803303 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565514748280 con 0x5655143771b0 2024-08-31T06:14:58.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.673+0000 7fcfb7e00640 1 --2- 172.21.15.16:0/3689803303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5655143771b0 0x5655147471b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8006010 tx=0x7fcfa803f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2106af89f7f7d4b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:58.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.673+0000 7fcfacc00640 1 -- 172.21.15.16:0/3689803303 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8035040 con 0x5655143771b0 2024-08-31T06:14:58.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.673+0000 7fcfacc00640 1 -- 172.21.15.16:0/3689803303 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfa8004e20 con 0x5655143771b0 2024-08-31T06:14:58.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.673+0000 7fcfacc00640 1 -- 172.21.15.16:0/3689803303 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa80324b0 con 0x5655143771b0 2024-08-31T06:14:58.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.674+0000 7fcfc5aba740 1 -- 172.21.15.16:0/3689803303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5655143771b0 msgr2=0x5655147471b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:58.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.674+0000 7fcfc5aba740 1 --2- 172.21.15.16:0/3689803303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5655143771b0 0x5655147471b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8006010 tx=0x7fcfa803f000 comp rx=0 tx=0).stop 2024-08-31T06:14:58.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.674+0000 7fcfc5aba740 1 -- 172.21.15.16:0/3689803303 shutdown_connections 2024-08-31T06:14:58.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.674+0000 7fcfc5aba740 1 --2- 172.21.15.16:0/3689803303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5655143a0070 0x56551475d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.674+0000 7fcfc5aba740 1 --2- 172.21.15.16:0/3689803303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56551437f390 0x56551475d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.674+0000 7fcfc5aba740 1 --2- 172.21.15.16:0/3689803303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5655143771b0 0x5655147471b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.674+0000 7fcfc5aba740 1 -- 172.21.15.16:0/3689803303 >> 172.21.15.16:0/3689803303 conn(0x565514394460 msgr2=0x565514746aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:58.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.674+0000 7fcfc5aba740 1 -- 172.21.15.16:0/3689803303 shutdown_connections 2024-08-31T06:14:58.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.674+0000 7fcfc5aba740 1 -- 172.21.15.16:0/3689803303 wait complete. 2024-08-31T06:14:58.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.674+0000 7fcfc5aba740 1 -- start start 2024-08-31T06:14:58.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.675+0000 7fcfc5aba740 1 Processor -- start 2024-08-31T06:14:58.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.675+0000 7fcfc5aba740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5655143771b0 0x565514755820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:58.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.675+0000 7fcfc5aba740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56551437f390 0x565514755d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:58.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.675+0000 7fcfc5aba740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5655143a0070 0x5655147ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:58.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.675+0000 7fcfc5aba740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb81afad0 con 0x5655143a0070 2024-08-31T06:14:58.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.675+0000 7fcfc5aba740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb81b0f40 con 0x56551437f390 2024-08-31T06:14:58.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.675+0000 7fcfc5aba740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb81b01a0 con 0x5655143771b0 2024-08-31T06:14:58.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.675+0000 7fcfc5aba740 10 client.0 _ll_register_callbacks cb 0x7fcfb0009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:14:58.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.675+0000 7fcfb7e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5655143771b0 0x565514755820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:58.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.675+0000 7fcfb7e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5655143771b0 0x565514755820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:39612/0 (socket says 172.21.15.16:39612) 2024-08-31T06:14:58.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.675+0000 7fcfb7e00640 1 -- 172.21.15.16:0/4166977953 learned_addr learned my addr 172.21.15.16:0/4166977953 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:14:58.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.675+0000 7fcfaf400640 1 --2- 172.21.15.16:0/4166977953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56551437f390 0x565514755d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:58.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.675+0000 7fcfbce00640 1 --2- 172.21.15.16:0/4166977953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5655143a0070 0x5655147ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:58.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.676+0000 7fcfb7e00640 1 -- 172.21.15.16:0/4166977953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56551437f390 msgr2=0x565514755d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:58.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.676+0000 7fcfb7e00640 1 --2- 172.21.15.16:0/4166977953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56551437f390 0x565514755d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.676+0000 7fcfb7e00640 1 -- 172.21.15.16:0/4166977953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5655143a0070 msgr2=0x5655147ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:14:58.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.676+0000 7fcfb7e00640 1 --2- 172.21.15.16:0/4166977953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5655143a0070 0x5655147ccf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.676+0000 7fcfb7e00640 1 -- 172.21.15.16:0/4166977953 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5655147cd440 con 0x5655143771b0 2024-08-31T06:14:58.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.676+0000 7fcfb7e00640 1 --2- 172.21.15.16:0/4166977953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5655143771b0 0x565514755820 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8004600 tx=0x7fcfa8004370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1df4c3ece935ceb server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:14:58.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.676+0000 7fcfa6000640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:14:58.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.676+0000 7fcfa6000640 1 -- 172.21.15.16:0/4166977953 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa80035f0 con 0x5655143771b0 2024-08-31T06:14:58.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.677+0000 7fcfa6000640 1 -- 172.21.15.16:0/4166977953 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7fcfa8003790 con 0x5655143771b0 2024-08-31T06:14:58.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.677+0000 7fcfc5aba740 1 -- 172.21.15.16:0/4166977953 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfa8004e20 con 0x5655143771b0 2024-08-31T06:14:58.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.677+0000 7fcfa6000640 1 -- 172.21.15.16:0/4166977953 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcfa8003990 con 0x5655143771b0 2024-08-31T06:14:58.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.677+0000 7fcfc5aba740 10 client.27328 fetch_fsmap learned FSMap version 39 2024-08-31T06:14:58.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.677+0000 7fcfc5aba740 1 -- 172.21.15.16:0/4166977953 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5655147ce500 con 0x5655143771b0 2024-08-31T06:14:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.677+0000 7fcfa6000640 1 -- 172.21.15.16:0/4166977953 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfa8004e20 con 0x5655143771b0 2024-08-31T06:14:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.677+0000 7fcfa6000640 1 -- 172.21.15.16:0/4166977953 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fcfa805a020 con 0x5655143771b0 2024-08-31T06:14:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.677+0000 7fcfc5aba740 10 client.27328 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T06:14:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.677+0000 7fcfc5aba740 10 client.27328 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T06:14:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.677+0000 7fcfc5aba740 20 client.27328 populate_metadata read hostname 'smithi016' 2024-08-31T06:14:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.677+0000 7fcfc5aba740 1 --2- 172.21.15.16:0/4166977953 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x565514333da0 0x5655147d0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:14:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.677+0000 7fcfc5aba740 4 client.27328 mds_command: new command op to 24712 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:14:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.677+0000 7fcfc5aba740 1 -- 172.21.15.16:0/4166977953 --> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5655146ba1a0 con 0x565514333da0 2024-08-31T06:14:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.678+0000 7fcfaf400640 1 --2- 172.21.15.16:0/4166977953 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x565514333da0 0x5655147d0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:14:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.678+0000 7fcfaf400640 1 --2- 172.21.15.16:0/4166977953 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x565514333da0 0x5655147d0c30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2146310889622b63 server_cookie=2bacda6393b8bb5c in_seq=0 out_seq=0 2024-08-31T06:14:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.678+0000 7fcfa6000640 10 client.27328 ms_handle_connect on v2:172.21.15.184:6838/4011051479 2024-08-31T06:14:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.679+0000 7fcfa6000640 1 -- 172.21.15.16:0/4166977953 <== mds.2 v2:172.21.15.184:6838/4011051479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5655146ba1a0 con 0x565514333da0 2024-08-31T06:14:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.679+0000 7fcfa6000640 10 client.27328 handle_command_reply: tid=0 2024-08-31T06:14:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.679+0000 7fcfc5aba740 1 client.27328 shutdown 2024-08-31T06:14:58.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.679+0000 7fcfc5aba740 1 -- 172.21.15.16:0/4166977953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5655143771b0 msgr2=0x565514755820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:58.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.679+0000 7fcfc5aba740 1 --2- 172.21.15.16:0/4166977953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5655143771b0 0x565514755820 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8004600 tx=0x7fcfa8004370 comp rx=0 tx=0).stop 2024-08-31T06:14:58.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.679+0000 7fcfc5aba740 1 -- 172.21.15.16:0/4166977953 shutdown_connections 2024-08-31T06:14:58.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.679+0000 7fcfc5aba740 1 --2- 172.21.15.16:0/4166977953 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x565514333da0 0x5655147d0c30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.679+0000 7fcfc5aba740 1 --2- 172.21.15.16:0/4166977953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5655143a0070 0x5655147ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.679+0000 7fcfc5aba740 1 --2- 172.21.15.16:0/4166977953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56551437f390 0x565514755d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.679+0000 7fcfc5aba740 1 --2- 172.21.15.16:0/4166977953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5655143771b0 0x565514755820 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.679+0000 7fcfc5aba740 1 -- 172.21.15.16:0/4166977953 >> 172.21.15.16:0/4166977953 conn(0x565514394460 msgr2=0x565514752640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:58.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.679+0000 7fcfa6000640 0 client.27328 ms_handle_reset on v2:172.21.15.184:6838/4011051479 2024-08-31T06:14:58.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.679+0000 7fcfc5aba740 1 -- 172.21.15.16:0/4166977953 shutdown_connections 2024-08-31T06:14:58.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.679+0000 7fcfc5aba740 1 -- 172.21.15.16:0/4166977953 wait complete. 2024-08-31T06:14:58.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.679+0000 7fcfc5aba740 20 client.27328 trim_cache size 0 max 16384 2024-08-31T06:14:58.685 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:14:58.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.680+0000 7fcfbd800640 1 -- 172.21.15.16:0/793603367 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fcf9c0b4ca0 msgr2=0x7fcf9c0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:58.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.680+0000 7fcfbd800640 1 --2- 172.21.15.16:0/793603367 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fcf9c0b4ca0 0x7fcf9c0b71b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8005ab0 tx=0x7fcfa8005a20 comp rx=0 tx=0).stop 2024-08-31T06:14:58.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.680+0000 7fcfbd800640 1 -- 172.21.15.16:0/793603367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfb8109090 msgr2=0x7fcfb80759f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:14:58.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.680+0000 7fcfbd800640 1 --2- 172.21.15.16:0/793603367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfb8109090 0x7fcfb80759f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000e9c0 tx=0x7fcfa000ee80 comp rx=0 tx=0).stop 2024-08-31T06:14:58.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.680+0000 7fcfbd800640 1 -- 172.21.15.16:0/793603367 shutdown_connections 2024-08-31T06:14:58.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.680+0000 7fcfbd800640 1 --2- 172.21.15.16:0/793603367 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fcf9c0b4ca0 0x7fcf9c0b71b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.680+0000 7fcfbd800640 1 --2- 172.21.15.16:0/793603367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfb81099e0 0x7fcfb8075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.680+0000 7fcfbd800640 1 --2- 172.21.15.16:0/793603367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfb8109090 0x7fcfb80759f0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.680+0000 7fcfbd800640 1 --2- 172.21.15.16:0/793603367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfb8103750 0x7fcfb80754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:14:58.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.680+0000 7fcfbd800640 1 -- 172.21.15.16:0/793603367 >> 172.21.15.16:0/793603367 conn(0x7fcfb80ff1a0 msgr2=0x7fcfb81015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:14:58.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.680+0000 7fcfbd800640 1 -- 172.21.15.16:0/793603367 shutdown_connections 2024-08-31T06:14:58.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:14:58.680+0000 7fcfbd800640 1 -- 172.21.15.16:0/793603367 wait complete. 2024-08-31T06:14:58.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:14:58 smithi155 ceph-mon[21742]: pgmap v600: 97 pgs: 97 active+clean; 3.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 9 op/s 2024-08-31T06:14:58.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:14:58 smithi016 ceph-mon[17194]: pgmap v600: 97 pgs: 97 active+clean; 3.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 9 op/s 2024-08-31T06:14:58.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:14:58 smithi184 ceph-mon[22331]: pgmap v600: 97 pgs: 97 active+clean; 3.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 9 op/s 2024-08-31T06:14:59.554 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T06:14:59.936 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:15:00.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.615+0000 7fa918400640 1 -- 172.21.15.16:0/4188267975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa910109b90 msgr2=0x7fa9101103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:00.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.615+0000 7fa918400640 1 --2- 172.21.15.16:0/4188267975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa910109b90 0x7fa9101103c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa908009ca0 tx=0x7fa90802a8c0 comp rx=0 tx=0).stop 2024-08-31T06:15:00.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.615+0000 7fa918400640 1 -- 172.21.15.16:0/4188267975 shutdown_connections 2024-08-31T06:15:00.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.615+0000 7fa918400640 1 --2- 172.21.15.16:0/4188267975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa910109b90 0x7fa9101103c0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:00.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.615+0000 7fa918400640 1 --2- 172.21.15.16:0/4188267975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa910109240 0x7fa910109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:00.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.615+0000 7fa918400640 1 --2- 172.21.15.16:0/4188267975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa910103900 0x7fa910103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:00.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.615+0000 7fa918400640 1 -- 172.21.15.16:0/4188267975 >> 172.21.15.16:0/4188267975 conn(0x7fa91007b5f0 msgr2=0x7fa91007ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:00.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.615+0000 7fa918400640 1 -- 172.21.15.16:0/4188267975 shutdown_connections 2024-08-31T06:15:00.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.615+0000 7fa918400640 1 -- 172.21.15.16:0/4188267975 wait complete. 2024-08-31T06:15:00.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.616+0000 7fa918400640 1 Processor -- start 2024-08-31T06:15:00.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.616+0000 7fa918400640 1 -- start start 2024-08-31T06:15:00.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.617+0000 7fa918400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa910103900 0x7fa91019a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:00.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.617+0000 7fa918400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa910109240 0x7fa91019a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:00.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.617+0000 7fa918400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa910109b90 0x7fa91019fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:00.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.617+0000 7fa918400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa9101138a0 con 0x7fa910109240 2024-08-31T06:15:00.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.617+0000 7fa918400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa910113720 con 0x7fa910103900 2024-08-31T06:15:00.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.617+0000 7fa918400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa910113a20 con 0x7fa910109b90 2024-08-31T06:15:00.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.617+0000 7fa915000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa910109240 0x7fa91019a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:00.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.617+0000 7fa915a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa910103900 0x7fa91019a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:00.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.617+0000 7fa915000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa910109240 0x7fa91019a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53336/0 (socket says 172.21.15.16:53336) 2024-08-31T06:15:00.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.617+0000 7fa915000640 1 -- 172.21.15.16:0/3005998473 learned_addr learned my addr 172.21.15.16:0/3005998473 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:00.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.617+0000 7fa916400640 1 --2- 172.21.15.16:0/3005998473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa910109b90 0x7fa91019fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:00.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.617+0000 7fa915000640 1 -- 172.21.15.16:0/3005998473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa910109b90 msgr2=0x7fa91019fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:00.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.617+0000 7fa915000640 1 --2- 172.21.15.16:0/3005998473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa910109b90 0x7fa91019fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:00.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.617+0000 7fa915000640 1 -- 172.21.15.16:0/3005998473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa910103900 msgr2=0x7fa91019a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:00.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.617+0000 7fa915000640 1 --2- 172.21.15.16:0/3005998473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa910103900 0x7fa91019a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:00.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.618+0000 7fa915000640 1 -- 172.21.15.16:0/3005998473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9101a0160 con 0x7fa910109240 2024-08-31T06:15:00.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.618+0000 7fa916400640 1 --2- 172.21.15.16:0/3005998473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa910109b90 0x7fa91019fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:15:00.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.618+0000 7fa915000640 1 --2- 172.21.15.16:0/3005998473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa910109240 0x7fa91019a790 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc00e9f0 tx=0x7fa8fc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:00.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.618+0000 7fa90e000640 1 -- 172.21.15.16:0/3005998473 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc00ccc0 con 0x7fa910109240 2024-08-31T06:15:00.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.618+0000 7fa918400640 1 -- 172.21.15.16:0/3005998473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9101a0450 con 0x7fa910109240 2024-08-31T06:15:00.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.618+0000 7fa90e000640 1 -- 172.21.15.16:0/3005998473 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8fc0108e0 con 0x7fa910109240 2024-08-31T06:15:00.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.618+0000 7fa90e000640 1 -- 172.21.15.16:0/3005998473 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc018860 con 0x7fa910109240 2024-08-31T06:15:00.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.618+0000 7fa918400640 1 -- 172.21.15.16:0/3005998473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa910075390 con 0x7fa910109240 2024-08-31T06:15:00.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.622+0000 7fa91f01d740 1 Processor -- start 2024-08-31T06:15:00.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.622+0000 7fa91f01d740 1 -- start start 2024-08-31T06:15:00.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.622+0000 7fa91f01d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e5bfbc070 0x563e5c361990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:00.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.622+0000 7fa91f01d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e5bf931b0 0x563e5c361ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:00.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.622+0000 7fa91f01d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e5bf9b390 0x563e5c362410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:00.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.622+0000 7fa91f01d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563e5c2d87b0 con 0x563e5bf931b0 2024-08-31T06:15:00.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.622+0000 7fa91f01d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563e5c2c7d40 con 0x563e5bf9b390 2024-08-31T06:15:00.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.622+0000 7fa91f01d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563e5c2cbcf0 con 0x563e5bfbc070 2024-08-31T06:15:00.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.622+0000 7fa915a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e5bf931b0 0x563e5c361ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:00.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.622+0000 7fa915a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e5bf931b0 0x563e5c361ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53352/0 (socket says 172.21.15.16:53352) 2024-08-31T06:15:00.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.623+0000 7fa915a00640 1 -- 172.21.15.16:0/3378098340 learned_addr learned my addr 172.21.15.16:0/3378098340 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:00.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.623+0000 7fa916400640 1 --2- 172.21.15.16:0/3378098340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e5bfbc070 0x563e5c361990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:00.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.623+0000 7fa90e000640 1 -- 172.21.15.16:0/3005998473 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fa8fc010430 con 0x7fa910109240 2024-08-31T06:15:00.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.623+0000 7fa915a00640 1 -- 172.21.15.16:0/3378098340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e5bfbc070 msgr2=0x563e5c361990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:00.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.623+0000 7fa915a00640 1 --2- 172.21.15.16:0/3378098340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e5bfbc070 0x563e5c361990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:00.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.623+0000 7fa915a00640 1 -- 172.21.15.16:0/3378098340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e5bf9b390 msgr2=0x563e5c362410 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T06:15:00.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.623+0000 7fa915a00640 1 --2- 172.21.15.16:0/3378098340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e5bf9b390 0x563e5c362410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:00.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.623+0000 7fa915a00640 1 -- 172.21.15.16:0/3378098340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e5c027650 con 0x563e5bf931b0 2024-08-31T06:15:00.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.623+0000 7fa915a00640 1 --2- 172.21.15.16:0/3378098340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e5bf931b0 0x563e5c361ed0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fa904010e50 tx=0x7fa904005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae4fd3b93c6d9e77 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:00.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.624+0000 7fa902a00640 1 -- 172.21.15.16:0/3378098340 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90401e020 con 0x563e5bf931b0 2024-08-31T06:15:00.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.624+0000 7fa902a00640 1 -- 172.21.15.16:0/3378098340 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa90401aab0 con 0x563e5bf931b0 2024-08-31T06:15:00.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.624+0000 7fa902a00640 1 -- 172.21.15.16:0/3378098340 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa904019b50 con 0x563e5bf931b0 2024-08-31T06:15:00.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.625+0000 7fa91f01d740 1 -- 172.21.15.16:0/3378098340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e5bf931b0 msgr2=0x563e5c361ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:00.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.625+0000 7fa91f01d740 1 --2- 172.21.15.16:0/3378098340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e5bf931b0 0x563e5c361ed0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fa904010e50 tx=0x7fa904005f90 comp rx=0 tx=0).stop 2024-08-31T06:15:00.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.625+0000 7fa91f01d740 1 -- 172.21.15.16:0/3378098340 shutdown_connections 2024-08-31T06:15:00.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.625+0000 7fa91f01d740 1 --2- 172.21.15.16:0/3378098340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e5bf9b390 0x563e5c362410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:00.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.625+0000 7fa91f01d740 1 --2- 172.21.15.16:0/3378098340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e5bf931b0 0x563e5c361ed0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:00.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.625+0000 7fa91f01d740 1 --2- 172.21.15.16:0/3378098340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e5bfbc070 0x563e5c361990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:00.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.625+0000 7fa91f01d740 1 -- 172.21.15.16:0/3378098340 >> 172.21.15.16:0/3378098340 conn(0x563e5bfb0460 msgr2=0x563e5c35f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:00.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.625+0000 7fa90e000640 1 --2- 172.21.15.16:0/3005998473 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa8f80b4d20 0x7fa8f80b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:00.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.625+0000 7fa91f01d740 1 -- 172.21.15.16:0/3378098340 shutdown_connections 2024-08-31T06:15:00.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.625+0000 7fa90e000640 1 -- 172.21.15.16:0/3005998473 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7fa8fc0e5840 con 0x7fa910109240 2024-08-31T06:15:00.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.625+0000 7fa916400640 1 --2- 172.21.15.16:0/3005998473 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa8f80b4d20 0x7fa8f80b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:00.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.626+0000 7fa91f01d740 1 -- 172.21.15.16:0/3378098340 wait complete. 2024-08-31T06:15:00.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.626+0000 7fa91f01d740 1 -- start start 2024-08-31T06:15:00.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.626+0000 7fa916400640 1 --2- 172.21.15.16:0/3005998473 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa8f80b4d20 0x7fa8f80b7230 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa90802f540 tx=0x7fa90802f330 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:00.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.626+0000 7fa91f01d740 1 Processor -- start 2024-08-31T06:15:00.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.627+0000 7fa91f01d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e5bf931b0 0x563e5c36d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:00.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.627+0000 7fa91f01d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e5bf9b390 0x563e5c36db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:00.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.627+0000 7fa91f01d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e5bfbc070 0x563e5c3794c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:00.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.627+0000 7fa91f01d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563e5c12a700 con 0x563e5bfbc070 2024-08-31T06:15:00.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.627+0000 7fa91f01d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563e5c24ecf0 con 0x563e5bf931b0 2024-08-31T06:15:00.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.627+0000 7fa91f01d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563e5c370170 con 0x563e5bf9b390 2024-08-31T06:15:00.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.627+0000 7fa91f01d740 10 client.0 _ll_register_callbacks cb 0x563e5c274470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:15:00.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.627+0000 7fa915a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e5bf931b0 0x563e5c36d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:00.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.627+0000 7fa915a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e5bf931b0 0x563e5c36d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:35302/0 (socket says 172.21.15.16:35302) 2024-08-31T06:15:00.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.627+0000 7fa915a00640 1 -- 172.21.15.16:0/720246552 learned_addr learned my addr 172.21.15.16:0/720246552 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:00.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.627+0000 7fa916400640 1 --2- 172.21.15.16:0/720246552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e5bfbc070 0x563e5c3794c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:00.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.627+0000 7fa915000640 1 --2- 172.21.15.16:0/720246552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e5bf9b390 0x563e5c36db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:00.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.628+0000 7fa915a00640 1 -- 172.21.15.16:0/720246552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e5bf9b390 msgr2=0x563e5c36db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:00.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.628+0000 7fa915a00640 1 --2- 172.21.15.16:0/720246552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e5bf9b390 0x563e5c36db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:00.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.628+0000 7fa915a00640 1 -- 172.21.15.16:0/720246552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e5bfbc070 msgr2=0x563e5c3794c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:00.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.628+0000 7fa915a00640 1 --2- 172.21.15.16:0/720246552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e5bfbc070 0x563e5c3794c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:00.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.628+0000 7fa915a00640 1 -- 172.21.15.16:0/720246552 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e5c28f8b0 con 0x563e5bf931b0 2024-08-31T06:15:00.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.628+0000 7fa915000640 1 --2- 172.21.15.16:0/720246552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e5bf9b390 0x563e5c36db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:15:00.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.628+0000 7fa915a00640 1 --2- 172.21.15.16:0/720246552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e5bf931b0 0x563e5c36d5c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa904019830 tx=0x7fa9040195f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f14529a0a24ef383 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:00.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.628+0000 7fa8f7400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:15:00.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.628+0000 7fa8f7400640 1 -- 172.21.15.16:0/720246552 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90401e070 con 0x563e5bf931b0 2024-08-31T06:15:00.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.628+0000 7fa8f7400640 1 -- 172.21.15.16:0/720246552 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7fa90401a400 con 0x563e5bf931b0 2024-08-31T06:15:00.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.628+0000 7fa91f01d740 1 -- 172.21.15.16:0/720246552 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa90401aab0 con 0x563e5bf931b0 2024-08-31T06:15:00.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.629+0000 7fa8f7400640 1 -- 172.21.15.16:0/720246552 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa90401a600 con 0x563e5bf931b0 2024-08-31T06:15:00.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.629+0000 7fa91f01d740 10 client.27356 fetch_fsmap learned FSMap version 39 2024-08-31T06:15:00.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.629+0000 7fa91f01d740 1 -- 172.21.15.16:0/720246552 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563e5c37c1f0 con 0x563e5bf931b0 2024-08-31T06:15:00.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.629+0000 7fa8f7400640 1 -- 172.21.15.16:0/720246552 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa90401aab0 con 0x563e5bf931b0 2024-08-31T06:15:00.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.629+0000 7fa8f7400640 1 -- 172.21.15.16:0/720246552 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fa90402b020 con 0x563e5bf931b0 2024-08-31T06:15:00.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.629+0000 7fa91f01d740 10 client.27356 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T06:15:00.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.629+0000 7fa91f01d740 10 client.27356 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-31T06:15:00.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.629+0000 7fa91f01d740 20 client.27356 populate_metadata read hostname 'smithi016' 2024-08-31T06:15:00.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.629+0000 7fa91f01d740 1 --2- 172.21.15.16:0/720246552 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x563e5bf7a9f0 0x563e5c37f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:00.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.629+0000 7fa91f01d740 4 client.27356 mds_command: new command op to 24724 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:15:00.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.629+0000 7fa91f01d740 1 -- 172.21.15.16:0/720246552 --> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e5c2da700 con 0x563e5bf7a9f0 2024-08-31T06:15:00.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.630+0000 7fa915000640 1 --2- 172.21.15.16:0/720246552 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x563e5bf7a9f0 0x563e5c37f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:00.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.630+0000 7fa915000640 1 --2- 172.21.15.16:0/720246552 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x563e5bf7a9f0 0x563e5c37f600 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d42f2930f7b9654d server_cookie=5ba86dd62ee5e81f in_seq=0 out_seq=0 2024-08-31T06:15:00.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.630+0000 7fa8f7400640 10 client.27356 ms_handle_connect on v2:172.21.15.16:6838/219664460 2024-08-31T06:15:00.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.631+0000 7fa8f7400640 1 -- 172.21.15.16:0/720246552 <== mds.1 v2:172.21.15.16:6838/219664460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563e5c2da700 con 0x563e5bf7a9f0 2024-08-31T06:15:00.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.631+0000 7fa8f7400640 10 client.27356 handle_command_reply: tid=0 2024-08-31T06:15:00.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.631+0000 7fa91f01d740 1 client.27356 shutdown 2024-08-31T06:15:00.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.631+0000 7fa91f01d740 1 -- 172.21.15.16:0/720246552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e5bf931b0 msgr2=0x563e5c36d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:00.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.631+0000 7fa91f01d740 1 --2- 172.21.15.16:0/720246552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e5bf931b0 0x563e5c36d5c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa904019830 tx=0x7fa9040195f0 comp rx=0 tx=0).stop 2024-08-31T06:15:00.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.631+0000 7fa91f01d740 1 -- 172.21.15.16:0/720246552 shutdown_connections 2024-08-31T06:15:00.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.631+0000 7fa91f01d740 1 --2- 172.21.15.16:0/720246552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e5bf9b390 0x563e5c36db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:00.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.631+0000 7fa91f01d740 1 --2- 172.21.15.16:0/720246552 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x563e5bf7a9f0 0x563e5c37f600 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:00.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.631+0000 7fa91f01d740 1 --2- 172.21.15.16:0/720246552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e5bfbc070 0x563e5c3794c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:00.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.631+0000 7fa91f01d740 1 --2- 172.21.15.16:0/720246552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e5bf931b0 0x563e5c36d5c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:00.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.631+0000 7fa8f7400640 0 client.27356 ms_handle_reset on v2:172.21.15.16:6838/219664460 2024-08-31T06:15:00.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.631+0000 7fa91f01d740 1 -- 172.21.15.16:0/720246552 >> 172.21.15.16:0/720246552 conn(0x563e5bfb0460 msgr2=0x563e5c364b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:00.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.632+0000 7fa91f01d740 1 -- 172.21.15.16:0/720246552 shutdown_connections 2024-08-31T06:15:00.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.632+0000 7fa91f01d740 1 -- 172.21.15.16:0/720246552 wait complete. 2024-08-31T06:15:00.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.632+0000 7fa91f01d740 20 client.27356 trim_cache size 0 max 16384 2024-08-31T06:15:00.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.646+0000 7fa91f01d740 1 Processor -- start 2024-08-31T06:15:00.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.646+0000 7fa91f01d740 1 -- start start 2024-08-31T06:15:00.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.646+0000 7fa91f01d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e5bf931b0 0x563e5c3793c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:00.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.646+0000 7fa91f01d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e5bf9b390 0x563e5c366bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:00.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.646+0000 7fa91f01d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e5bfbc070 0x563e5c367130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:00.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.646+0000 7fa91f01d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa9101afbc0 con 0x563e5bfbc070 2024-08-31T06:15:00.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.646+0000 7fa91f01d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa9101ad430 con 0x563e5bf931b0 2024-08-31T06:15:00.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.646+0000 7fa91f01d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa9101ab2a0 con 0x563e5bf9b390 2024-08-31T06:15:00.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.647+0000 7fa916400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e5bfbc070 0x563e5c367130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:00.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.647+0000 7fa916400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e5bfbc070 0x563e5c367130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53356/0 (socket says 172.21.15.16:53356) 2024-08-31T06:15:00.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.647+0000 7fa916400640 1 -- 172.21.15.16:0/2589425354 learned_addr learned my addr 172.21.15.16:0/2589425354 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:00.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.647+0000 7fa915000640 1 --2- 172.21.15.16:0/2589425354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e5bf9b390 0x563e5c366bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:00.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.647+0000 7fa915a00640 1 --2- 172.21.15.16:0/2589425354 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e5bf931b0 0x563e5c3793c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:00.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.647+0000 7fa916400640 1 -- 172.21.15.16:0/2589425354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e5bf9b390 msgr2=0x563e5c366bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:00.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.647+0000 7fa916400640 1 --2- 172.21.15.16:0/2589425354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e5bf9b390 0x563e5c366bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:00.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.647+0000 7fa916400640 1 -- 172.21.15.16:0/2589425354 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e5bf931b0 msgr2=0x563e5c3793c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:00.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.647+0000 7fa916400640 1 --2- 172.21.15.16:0/2589425354 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e5bf931b0 0x563e5c3793c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:00.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.647+0000 7fa916400640 1 -- 172.21.15.16:0/2589425354 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e5c363c80 con 0x563e5bfbc070 2024-08-31T06:15:00.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.647+0000 7fa915000640 1 --2- 172.21.15.16:0/2589425354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e5bf9b390 0x563e5c366bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:15:00.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.647+0000 7fa916400640 1 --2- 172.21.15.16:0/2589425354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e5bfbc070 0x563e5c367130 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa908031e10 tx=0x7fa908041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c5a7710dd1572d5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:00.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.648+0000 7fa902000640 1 -- 172.21.15.16:0/2589425354 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa908031650 con 0x563e5bfbc070 2024-08-31T06:15:00.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.648+0000 7fa902000640 1 -- 172.21.15.16:0/2589425354 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa908036040 con 0x563e5bfbc070 2024-08-31T06:15:00.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.648+0000 7fa902000640 1 -- 172.21.15.16:0/2589425354 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa908004440 con 0x563e5bfbc070 2024-08-31T06:15:00.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.648+0000 7fa91f01d740 1 -- 172.21.15.16:0/2589425354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e5bfbc070 msgr2=0x563e5c367130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:00.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.648+0000 7fa91f01d740 1 --2- 172.21.15.16:0/2589425354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e5bfbc070 0x563e5c367130 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa908031e10 tx=0x7fa908041000 comp rx=0 tx=0).stop 2024-08-31T06:15:00.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.649+0000 7fa91f01d740 1 -- 172.21.15.16:0/2589425354 shutdown_connections 2024-08-31T06:15:00.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.649+0000 7fa91f01d740 1 --2- 172.21.15.16:0/2589425354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e5bf9b390 0x563e5c366bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:00.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.649+0000 7fa91f01d740 1 --2- 172.21.15.16:0/2589425354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e5bfbc070 0x563e5c367130 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:00.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.649+0000 7fa91f01d740 1 --2- 172.21.15.16:0/2589425354 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e5bf931b0 0x563e5c3793c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:00.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.649+0000 7fa91f01d740 1 -- 172.21.15.16:0/2589425354 >> 172.21.15.16:0/2589425354 conn(0x563e5bfb0460 msgr2=0x563e5c368500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:00.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.649+0000 7fa91f01d740 1 -- 172.21.15.16:0/2589425354 shutdown_connections 2024-08-31T06:15:00.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.649+0000 7fa91f01d740 1 -- 172.21.15.16:0/2589425354 wait complete. 2024-08-31T06:15:00.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.649+0000 7fa91f01d740 1 -- start start 2024-08-31T06:15:00.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.649+0000 7fa91f01d740 1 Processor -- start 2024-08-31T06:15:00.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.650+0000 7fa91f01d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e5bf931b0 0x563e5c375430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:00.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.650+0000 7fa91f01d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e5bf9b390 0x563e5c375970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:00.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.650+0000 7fa91f01d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e5bfbc070 0x563e5c3703e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:00.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.650+0000 7fa91f01d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa9101ab8f0 con 0x563e5bf9b390 2024-08-31T06:15:00.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.650+0000 7fa91f01d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa9101acd60 con 0x563e5bf931b0 2024-08-31T06:15:00.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.650+0000 7fa91f01d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa9101abfc0 con 0x563e5bfbc070 2024-08-31T06:15:00.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.650+0000 7fa91f01d740 10 client.0 _ll_register_callbacks cb 0x7fa904010b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:15:00.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.650+0000 7fa915000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e5bf9b390 0x563e5c375970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:00.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.650+0000 7fa915a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e5bf931b0 0x563e5c375430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:00.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.650+0000 7fa915a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e5bf931b0 0x563e5c375430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:35318/0 (socket says 172.21.15.16:35318) 2024-08-31T06:15:00.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.650+0000 7fa915a00640 1 -- 172.21.15.16:0/4051736175 learned_addr learned my addr 172.21.15.16:0/4051736175 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:00.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.650+0000 7fa916400640 1 --2- 172.21.15.16:0/4051736175 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e5bfbc070 0x563e5c3703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:00.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.651+0000 7fa915a00640 1 -- 172.21.15.16:0/4051736175 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e5bfbc070 msgr2=0x563e5c3703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:00.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.651+0000 7fa915a00640 1 --2- 172.21.15.16:0/4051736175 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e5bfbc070 0x563e5c3703e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:00.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.651+0000 7fa915a00640 1 -- 172.21.15.16:0/4051736175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e5bf9b390 msgr2=0x563e5c375970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:00.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.651+0000 7fa915a00640 1 --2- 172.21.15.16:0/4051736175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e5bf9b390 0x563e5c375970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:00.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.651+0000 7fa915a00640 1 -- 172.21.15.16:0/4051736175 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e5c375eb0 con 0x563e5bf931b0 2024-08-31T06:15:00.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.651+0000 7fa915000640 1 --2- 172.21.15.16:0/4051736175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e5bf9b390 0x563e5c375970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:15:00.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.651+0000 7fa915a00640 1 --2- 172.21.15.16:0/4051736175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e5bf931b0 0x563e5c375430 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa904019ef0 tx=0x7fa904019ff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88965c016c8db495 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:00.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.651+0000 7fa8f7400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:15:00.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.651+0000 7fa8f7400640 1 -- 172.21.15.16:0/4051736175 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90403a070 con 0x563e5bf931b0 2024-08-31T06:15:00.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.651+0000 7fa8f7400640 1 -- 172.21.15.16:0/4051736175 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7fa90401a9b0 con 0x563e5bf931b0 2024-08-31T06:15:00.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.651+0000 7fa91f01d740 1 -- 172.21.15.16:0/4051736175 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa908036040 con 0x563e5bf931b0 2024-08-31T06:15:00.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.652+0000 7fa8f7400640 1 -- 172.21.15.16:0/4051736175 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa904046050 con 0x563e5bf931b0 2024-08-31T06:15:00.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.652+0000 7fa91f01d740 10 client.27362 fetch_fsmap learned FSMap version 39 2024-08-31T06:15:00.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.652+0000 7fa91f01d740 1 -- 172.21.15.16:0/4051736175 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563e5c3ea0e0 con 0x563e5bf931b0 2024-08-31T06:15:00.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.652+0000 7fa8f7400640 1 -- 172.21.15.16:0/4051736175 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa908036040 con 0x563e5bf931b0 2024-08-31T06:15:00.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.652+0000 7fa8f7400640 1 -- 172.21.15.16:0/4051736175 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fa90402b040 con 0x563e5bf931b0 2024-08-31T06:15:00.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.653+0000 7fa91f01d740 10 client.27362 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T06:15:00.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.653+0000 7fa91f01d740 10 client.27362 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-31T06:15:00.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.653+0000 7fa91f01d740 20 client.27362 populate_metadata read hostname 'smithi016' 2024-08-31T06:15:00.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.653+0000 7fa91f01d740 1 --2- 172.21.15.16:0/4051736175 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x563e5bf4fda0 0x563e5c3ec7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:00.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.653+0000 7fa91f01d740 4 client.27362 mds_command: new command op to 24724 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:15:00.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.653+0000 7fa91f01d740 1 -- 172.21.15.16:0/4051736175 --> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563e5c2d61a0 con 0x563e5bf4fda0 2024-08-31T06:15:00.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.653+0000 7fa915000640 1 --2- 172.21.15.16:0/4051736175 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x563e5bf4fda0 0x563e5c3ec7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:00.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.653+0000 7fa8f7400640 10 client.27362 ms_handle_connect on v2:172.21.15.16:6838/219664460 2024-08-31T06:15:00.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.653+0000 7fa915000640 1 --2- 172.21.15.16:0/4051736175 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x563e5bf4fda0 0x563e5c3ec7e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=34d82b39e49e76d server_cookie=5697eb7745699677 in_seq=0 out_seq=0 2024-08-31T06:15:00.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.654+0000 7fa8f7400640 1 -- 172.21.15.16:0/4051736175 <== mds.1 v2:172.21.15.16:6838/219664460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563e5c2d61a0 con 0x563e5bf4fda0 2024-08-31T06:15:00.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.654+0000 7fa8f7400640 10 client.27362 handle_command_reply: tid=0 2024-08-31T06:15:00.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.654+0000 7fa91f01d740 1 client.27362 shutdown 2024-08-31T06:15:00.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.654+0000 7fa91f01d740 1 -- 172.21.15.16:0/4051736175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e5bf931b0 msgr2=0x563e5c375430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:00.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.654+0000 7fa91f01d740 1 --2- 172.21.15.16:0/4051736175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e5bf931b0 0x563e5c375430 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa904019ef0 tx=0x7fa904019ff0 comp rx=0 tx=0).stop 2024-08-31T06:15:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.654+0000 7fa91f01d740 1 -- 172.21.15.16:0/4051736175 shutdown_connections 2024-08-31T06:15:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.654+0000 7fa91f01d740 1 --2- 172.21.15.16:0/4051736175 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e5bfbc070 0x563e5c3703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.654+0000 7fa91f01d740 1 --2- 172.21.15.16:0/4051736175 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x563e5bf4fda0 0x563e5c3ec7e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.654+0000 7fa91f01d740 1 --2- 172.21.15.16:0/4051736175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e5bf9b390 0x563e5c375970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.654+0000 7fa91f01d740 1 --2- 172.21.15.16:0/4051736175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e5bf931b0 0x563e5c375430 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.654+0000 7fa8f7400640 0 client.27362 ms_handle_reset on v2:172.21.15.16:6838/219664460 2024-08-31T06:15:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.654+0000 7fa91f01d740 1 -- 172.21.15.16:0/4051736175 >> 172.21.15.16:0/4051736175 conn(0x563e5bfb0460 msgr2=0x563e5c36d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.654+0000 7fa91f01d740 1 -- 172.21.15.16:0/4051736175 shutdown_connections 2024-08-31T06:15:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.654+0000 7fa91f01d740 1 -- 172.21.15.16:0/4051736175 wait complete. 2024-08-31T06:15:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.655+0000 7fa91f01d740 20 client.27362 trim_cache size 0 max 16384 2024-08-31T06:15:00.660 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:15:00.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.655+0000 7fa918400640 1 -- 172.21.15.16:0/3005998473 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa8f80b4d20 msgr2=0x7fa8f80b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:00.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.655+0000 7fa918400640 1 --2- 172.21.15.16:0/3005998473 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa8f80b4d20 0x7fa8f80b7230 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa90802f540 tx=0x7fa90802f330 comp rx=0 tx=0).stop 2024-08-31T06:15:00.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.655+0000 7fa918400640 1 -- 172.21.15.16:0/3005998473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa910109240 msgr2=0x7fa91019a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:00.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.655+0000 7fa918400640 1 --2- 172.21.15.16:0/3005998473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa910109240 0x7fa91019a790 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc00e9f0 tx=0x7fa8fc00eeb0 comp rx=0 tx=0).stop 2024-08-31T06:15:00.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.655+0000 7fa918400640 1 -- 172.21.15.16:0/3005998473 shutdown_connections 2024-08-31T06:15:00.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.655+0000 7fa918400640 1 --2- 172.21.15.16:0/3005998473 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa8f80b4d20 0x7fa8f80b7230 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:00.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.655+0000 7fa918400640 1 --2- 172.21.15.16:0/3005998473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa910109b90 0x7fa91019fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:00.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.655+0000 7fa918400640 1 --2- 172.21.15.16:0/3005998473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa910109240 0x7fa91019a790 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:00.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.655+0000 7fa918400640 1 --2- 172.21.15.16:0/3005998473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa910103900 0x7fa91019a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:00.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.655+0000 7fa918400640 1 -- 172.21.15.16:0/3005998473 >> 172.21.15.16:0/3005998473 conn(0x7fa91007b5f0 msgr2=0x7fa910105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:00.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.656+0000 7fa918400640 1 -- 172.21.15.16:0/3005998473 shutdown_connections 2024-08-31T06:15:00.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:00.656+0000 7fa918400640 1 -- 172.21.15.16:0/3005998473 wait complete. 2024-08-31T06:15:00.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:00 smithi155 ceph-mon[21742]: pgmap v601: 97 pgs: 97 active+clean; 3.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-08-31T06:15:00.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:00 smithi016 ceph-mon[17194]: pgmap v601: 97 pgs: 97 active+clean; 3.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-08-31T06:15:00.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:00 smithi184 ceph-mon[22331]: pgmap v601: 97 pgs: 97 active+clean; 3.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-08-31T06:15:01.480 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T06:15:01.626 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 455.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-31T06:15:01.864 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:15:02.441 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:15:02.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.510+0000 7f3103400640 1 -- 172.21.15.16:0/973131960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30fc103920 msgr2=0x7f30fc103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:02.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.510+0000 7f3103400640 1 --2- 172.21.15.16:0/973131960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30fc103920 0x7f30fc103d30 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f30f0009900 tx=0x7f30f002a400 comp rx=0 tx=0).stop 2024-08-31T06:15:02.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.511+0000 7f3103400640 1 -- 172.21.15.16:0/973131960 shutdown_connections 2024-08-31T06:15:02.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.511+0000 7f3103400640 1 --2- 172.21.15.16:0/973131960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30fc109bb0 0x7f30fc1103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:02.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.511+0000 7f3103400640 1 --2- 172.21.15.16:0/973131960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30fc109260 0x7f30fc109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:02.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.511+0000 7f3103400640 1 --2- 172.21.15.16:0/973131960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30fc103920 0x7f30fc103d30 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:02.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.511+0000 7f3103400640 1 -- 172.21.15.16:0/973131960 >> 172.21.15.16:0/973131960 conn(0x7f30fc0ff370 msgr2=0x7f30fc1017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:02.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.511+0000 7f3103400640 1 -- 172.21.15.16:0/973131960 shutdown_connections 2024-08-31T06:15:02.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.511+0000 7f3103400640 1 -- 172.21.15.16:0/973131960 wait complete. 2024-08-31T06:15:02.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.512+0000 7f3103400640 1 Processor -- start 2024-08-31T06:15:02.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.512+0000 7f3103400640 1 -- start start 2024-08-31T06:15:02.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.513+0000 7f3103400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30fc103920 0x7f30fc19a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:02.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.513+0000 7f3103400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30fc109260 0x7f30fc19a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:02.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.513+0000 7f3103400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30fc109bb0 0x7f30fc19fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:02.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.513+0000 7f3103400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc1138c0 con 0x7f30fc109bb0 2024-08-31T06:15:02.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.513+0000 7f3103400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc113740 con 0x7f30fc109260 2024-08-31T06:15:02.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.513+0000 7f3103400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc113a40 con 0x7f30fc103920 2024-08-31T06:15:02.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.513+0000 7f30fbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30fc109260 0x7f30fc19a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:02.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.513+0000 7f30fbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30fc109260 0x7f30fc19a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:35342/0 (socket says 172.21.15.16:35342) 2024-08-31T06:15:02.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.513+0000 7f30fbe00640 1 -- 172.21.15.16:0/3570993742 learned_addr learned my addr 172.21.15.16:0/3570993742 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:02.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.513+0000 7f3101400640 1 --2- 172.21.15.16:0/3570993742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30fc109bb0 0x7f30fc19fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:02.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.513+0000 7f3100a00640 1 --2- 172.21.15.16:0/3570993742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30fc103920 0x7f30fc19a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:02.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.514+0000 7f30fbe00640 1 -- 172.21.15.16:0/3570993742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30fc103920 msgr2=0x7f30fc19a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:02.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.514+0000 7f30fbe00640 1 --2- 172.21.15.16:0/3570993742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30fc103920 0x7f30fc19a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:02.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.514+0000 7f30fbe00640 1 -- 172.21.15.16:0/3570993742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30fc109bb0 msgr2=0x7f30fc19fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:02.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.514+0000 7f30fbe00640 1 --2- 172.21.15.16:0/3570993742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30fc109bb0 0x7f30fc19fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:02.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.514+0000 7f30fbe00640 1 -- 172.21.15.16:0/3570993742 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30fc1a01e0 con 0x7f30fc109260 2024-08-31T06:15:02.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.514+0000 7f3101400640 1 --2- 172.21.15.16:0/3570993742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30fc109bb0 0x7f30fc19fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:15:02.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.514+0000 7f30fbe00640 1 --2- 172.21.15.16:0/3570993742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30fc109260 0x7f30fc19a810 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f30e800e9f0 tx=0x7f30e800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:02.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.515+0000 7f30f9600640 1 -- 172.21.15.16:0/3570993742 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e800ccc0 con 0x7f30fc109260 2024-08-31T06:15:02.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.515+0000 7f3103400640 1 -- 172.21.15.16:0/3570993742 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30fc1a04d0 con 0x7f30fc109260 2024-08-31T06:15:02.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.515+0000 7f3103400640 1 -- 172.21.15.16:0/3570993742 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30fc1a6c30 con 0x7f30fc109260 2024-08-31T06:15:02.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.515+0000 7f30f9600640 1 -- 172.21.15.16:0/3570993742 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30e80108e0 con 0x7f30fc109260 2024-08-31T06:15:02.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.515+0000 7f30f9600640 1 -- 172.21.15.16:0/3570993742 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e80187a0 con 0x7f30fc109260 2024-08-31T06:15:02.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.518+0000 7f310a21b740 1 Processor -- start 2024-08-31T06:15:02.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.518+0000 7f310a21b740 1 -- start start 2024-08-31T06:15:02.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.518+0000 7f310a21b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56121ac8f070 0x56121b038da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:02.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.518+0000 7f310a21b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56121ac661b0 0x56121b0392e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:02.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.518+0000 7f310a21b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56121ac6e390 0x56121b039820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:02.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.518+0000 7f310a21b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56121afab7b0 con 0x56121ac661b0 2024-08-31T06:15:02.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.518+0000 7f310a21b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56121af9ad40 con 0x56121ac8f070 2024-08-31T06:15:02.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.518+0000 7f310a21b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56121af9ecf0 con 0x56121ac6e390 2024-08-31T06:15:02.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.519+0000 7f3101400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56121ac8f070 0x56121b038da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:02.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.519+0000 7f3100a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56121ac661b0 0x56121b0392e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:02.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.519+0000 7f3101400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56121ac8f070 0x56121b038da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:35344/0 (socket says 172.21.15.16:35344) 2024-08-31T06:15:02.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.519+0000 7f3101400640 1 -- 172.21.15.16:0/660711170 learned_addr learned my addr 172.21.15.16:0/660711170 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:02.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.519+0000 7f30f9600640 1 -- 172.21.15.16:0/3570993742 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f30e8018940 con 0x7f30fc109260 2024-08-31T06:15:02.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.519+0000 7f3101400640 1 -- 172.21.15.16:0/660711170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56121ac6e390 msgr2=0x56121b039820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:15:02.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.519+0000 7f3101400640 1 --2- 172.21.15.16:0/660711170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56121ac6e390 0x56121b039820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:02.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.519+0000 7f3101400640 1 -- 172.21.15.16:0/660711170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56121ac661b0 msgr2=0x56121b0392e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:02.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.519+0000 7f3101400640 1 --2- 172.21.15.16:0/660711170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56121ac661b0 0x56121b0392e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:02.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.519+0000 7f3101400640 1 -- 172.21.15.16:0/660711170 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56121acfa650 con 0x56121ac8f070 2024-08-31T06:15:02.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.520+0000 7f3101400640 1 --2- 172.21.15.16:0/660711170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56121ac8f070 0x56121b038da0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f30f4011810 tx=0x7f30f4011cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5084ff1e54ffb382 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:02.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.520+0000 7f30edc00640 1 -- 172.21.15.16:0/660711170 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f4011310 con 0x56121ac8f070 2024-08-31T06:15:02.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.520+0000 7f30edc00640 1 -- 172.21.15.16:0/660711170 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30f400f9a0 con 0x56121ac8f070 2024-08-31T06:15:02.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.520+0000 7f30edc00640 1 -- 172.21.15.16:0/660711170 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f4005650 con 0x56121ac8f070 2024-08-31T06:15:02.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.521+0000 7f310a21b740 1 -- 172.21.15.16:0/660711170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56121ac8f070 msgr2=0x56121b038da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:02.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.521+0000 7f310a21b740 1 --2- 172.21.15.16:0/660711170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56121ac8f070 0x56121b038da0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f30f4011810 tx=0x7f30f4011cd0 comp rx=0 tx=0).stop 2024-08-31T06:15:02.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.521+0000 7f310a21b740 1 -- 172.21.15.16:0/660711170 shutdown_connections 2024-08-31T06:15:02.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.521+0000 7f310a21b740 1 --2- 172.21.15.16:0/660711170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56121ac6e390 0x56121b039820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:02.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.521+0000 7f310a21b740 1 --2- 172.21.15.16:0/660711170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56121ac661b0 0x56121b0392e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:02.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.521+0000 7f310a21b740 1 --2- 172.21.15.16:0/660711170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56121ac8f070 0x56121b038da0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:02.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.521+0000 7f310a21b740 1 -- 172.21.15.16:0/660711170 >> 172.21.15.16:0/660711170 conn(0x56121ac83460 msgr2=0x56121b036c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:02.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.521+0000 7f310a21b740 1 -- 172.21.15.16:0/660711170 shutdown_connections 2024-08-31T06:15:02.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.521+0000 7f310a21b740 1 -- 172.21.15.16:0/660711170 wait complete. 2024-08-31T06:15:02.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.522+0000 7f30f9600640 1 --2- 172.21.15.16:0/3570993742 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f30e40b4c30 0x7f30e40b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:02.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.522+0000 7f310a21b740 1 -- start start 2024-08-31T06:15:02.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.522+0000 7f30f9600640 1 -- 172.21.15.16:0/3570993742 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7f30e80e56a0 con 0x7f30fc109260 2024-08-31T06:15:02.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.522+0000 7f310a21b740 1 Processor -- start 2024-08-31T06:15:02.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.522+0000 7f3101400640 1 --2- 172.21.15.16:0/3570993742 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f30e40b4c30 0x7f30e40b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:02.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.523+0000 7f310a21b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56121ac661b0 0x56121b0449d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:02.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.523+0000 7f310a21b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56121ac6e390 0x56121b044f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:02.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.523+0000 7f310a21b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56121ac8f070 0x56121b0508d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:02.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.523+0000 7f310a21b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56121adfd700 con 0x56121ac6e390 2024-08-31T06:15:02.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.523+0000 7f310a21b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56121af21cf0 con 0x56121ac8f070 2024-08-31T06:15:02.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.523+0000 7f310a21b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56121b047580 con 0x56121ac661b0 2024-08-31T06:15:02.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.523+0000 7f310a21b740 10 client.0 _ll_register_callbacks cb 0x56121af47470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:15:02.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.523+0000 7f3101400640 1 --2- 172.21.15.16:0/3570993742 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f30e40b4c30 0x7f30e40b7140 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f30f4006010 tx=0x7f30f4000f40 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:02.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.523+0000 7f3100a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56121ac661b0 0x56121b0449d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:02.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.524+0000 7f30fbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56121ac6e390 0x56121b044f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:02.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.524+0000 7f3100a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56121ac661b0 0x56121b0449d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:39700/0 (socket says 172.21.15.16:39700) 2024-08-31T06:15:02.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.524+0000 7f3100a00640 1 -- 172.21.15.16:0/4192050085 learned_addr learned my addr 172.21.15.16:0/4192050085 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:02.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.524+0000 7f3101400640 1 --2- 172.21.15.16:0/4192050085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56121ac8f070 0x56121b0508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:02.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.524+0000 7f3100a00640 1 -- 172.21.15.16:0/4192050085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56121ac8f070 msgr2=0x56121b0508d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:02.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.524+0000 7f3100a00640 1 --2- 172.21.15.16:0/4192050085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56121ac8f070 0x56121b0508d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:02.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.524+0000 7f3100a00640 1 -- 172.21.15.16:0/4192050085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56121ac6e390 msgr2=0x56121b044f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:02.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.524+0000 7f3100a00640 1 --2- 172.21.15.16:0/4192050085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56121ac6e390 0x56121b044f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:02.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.524+0000 7f3100a00640 1 -- 172.21.15.16:0/4192050085 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56121af628b0 con 0x56121ac661b0 2024-08-31T06:15:02.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.524+0000 7f3101400640 1 --2- 172.21.15.16:0/4192050085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56121ac8f070 0x56121b0508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:15:02.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.524+0000 7f30fbe00640 1 --2- 172.21.15.16:0/4192050085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56121ac6e390 0x56121b044f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:15:02.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.525+0000 7f3100a00640 1 --2- 172.21.15.16:0/4192050085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56121ac661b0 0x56121b0449d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f30f0009230 tx=0x7f30f0004440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac73a7410808e3d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:02.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.525+0000 7f30e2000640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:15:02.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.525+0000 7f30e2000640 1 -- 172.21.15.16:0/4192050085 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f0035040 con 0x56121ac661b0 2024-08-31T06:15:02.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.525+0000 7f30e2000640 1 -- 172.21.15.16:0/4192050085 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7f30f0005510 con 0x56121ac661b0 2024-08-31T06:15:02.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.525+0000 7f310a21b740 1 -- 172.21.15.16:0/4192050085 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30f400f9a0 con 0x56121ac661b0 2024-08-31T06:15:02.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.525+0000 7f30e2000640 1 -- 172.21.15.16:0/4192050085 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f30f0004770 con 0x56121ac661b0 2024-08-31T06:15:02.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.525+0000 7f310a21b740 10 client.27355 fetch_fsmap learned FSMap version 39 2024-08-31T06:15:02.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.525+0000 7f310a21b740 1 -- 172.21.15.16:0/4192050085 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56121b053620 con 0x56121ac661b0 2024-08-31T06:15:02.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.526+0000 7f30e2000640 1 -- 172.21.15.16:0/4192050085 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30f400f9a0 con 0x56121ac661b0 2024-08-31T06:15:02.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.526+0000 7f30e2000640 1 -- 172.21.15.16:0/4192050085 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f30f0003b90 con 0x56121ac661b0 2024-08-31T06:15:02.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.526+0000 7f310a21b740 10 client.27355 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T06:15:02.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.526+0000 7f310a21b740 10 client.27355 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:15:02.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.526+0000 7f310a21b740 20 client.27355 populate_metadata read hostname 'smithi016' 2024-08-31T06:15:02.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.526+0000 7f310a21b740 1 --2- 172.21.15.16:0/4192050085 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56121ac4d9f0 0x56121b056b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:02.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.526+0000 7f310a21b740 4 client.27355 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:15:02.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.526+0000 7f310a21b740 1 -- 172.21.15.16:0/4192050085 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56121afad700 con 0x56121ac4d9f0 2024-08-31T06:15:02.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.527+0000 7f30fbe00640 1 --2- 172.21.15.16:0/4192050085 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56121ac4d9f0 0x56121b056b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:02.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.527+0000 7f30fbe00640 1 --2- 172.21.15.16:0/4192050085 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56121ac4d9f0 0x56121b056b80 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80f1fc1319f2e64d server_cookie=aee391ee0177b961 in_seq=0 out_seq=0 2024-08-31T06:15:02.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.527+0000 7f30e2000640 10 client.27355 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:15:02.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.528+0000 7f30e2000640 1 -- 172.21.15.16:0/4192050085 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56121afad700 con 0x56121ac4d9f0 2024-08-31T06:15:02.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.528+0000 7f30e2000640 10 client.27355 handle_command_reply: tid=0 2024-08-31T06:15:02.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.528+0000 7f310a21b740 1 client.27355 shutdown 2024-08-31T06:15:02.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.528+0000 7f310a21b740 1 -- 172.21.15.16:0/4192050085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56121ac661b0 msgr2=0x56121b0449d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:02.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.528+0000 7f310a21b740 1 --2- 172.21.15.16:0/4192050085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56121ac661b0 0x56121b0449d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f30f0009230 tx=0x7f30f0004440 comp rx=0 tx=0).stop 2024-08-31T06:15:02.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.529+0000 7f310a21b740 1 -- 172.21.15.16:0/4192050085 shutdown_connections 2024-08-31T06:15:02.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.529+0000 7f310a21b740 1 --2- 172.21.15.16:0/4192050085 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56121ac4d9f0 0x56121b056b80 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:02.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.529+0000 7f310a21b740 1 --2- 172.21.15.16:0/4192050085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56121ac8f070 0x56121b0508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:02.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.529+0000 7f310a21b740 1 --2- 172.21.15.16:0/4192050085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56121ac6e390 0x56121b044f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:02.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.529+0000 7f310a21b740 1 --2- 172.21.15.16:0/4192050085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56121ac661b0 0x56121b0449d0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:02.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.529+0000 7f30e2000640 0 client.27355 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:15:02.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.529+0000 7f310a21b740 1 -- 172.21.15.16:0/4192050085 >> 172.21.15.16:0/4192050085 conn(0x56121ac83460 msgr2=0x56121b030e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:02.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.529+0000 7f310a21b740 1 -- 172.21.15.16:0/4192050085 shutdown_connections 2024-08-31T06:15:02.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.529+0000 7f310a21b740 1 -- 172.21.15.16:0/4192050085 wait complete. 2024-08-31T06:15:02.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.529+0000 7f310a21b740 20 client.27355 trim_cache size 0 max 16384 2024-08-31T06:15:02.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.543+0000 7f310a21b740 1 Processor -- start 2024-08-31T06:15:02.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.543+0000 7f310a21b740 1 -- start start 2024-08-31T06:15:02.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.544+0000 7f310a21b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56121ac661b0 0x56121b031530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:02.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.544+0000 7f310a21b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56121ac6e390 0x56121b03deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:02.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.544+0000 7f310a21b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56121ac8f070 0x56121b03e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:02.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.544+0000 7f310a21b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f30dc011060 con 0x56121ac6e390 2024-08-31T06:15:02.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.544+0000 7f310a21b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f30dc00e8d0 con 0x56121ac8f070 2024-08-31T06:15:02.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.544+0000 7f310a21b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f30dc00c740 con 0x56121ac661b0 2024-08-31T06:15:02.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.544+0000 7f30fbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56121ac6e390 0x56121b03deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:02.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.544+0000 7f3100a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56121ac661b0 0x56121b031530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:02.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.544+0000 7f3100a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56121ac661b0 0x56121b031530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:39704/0 (socket says 172.21.15.16:39704) 2024-08-31T06:15:02.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.544+0000 7f30fbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56121ac6e390 0x56121b03deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53416/0 (socket says 172.21.15.16:53416) 2024-08-31T06:15:02.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.544+0000 7f3100a00640 1 -- 172.21.15.16:0/1862567248 learned_addr learned my addr 172.21.15.16:0/1862567248 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:02.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.544+0000 7f3101400640 1 --2- 172.21.15.16:0/1862567248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56121ac8f070 0x56121b03e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:02.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.545+0000 7f30fbe00640 1 -- 172.21.15.16:0/1862567248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56121ac661b0 msgr2=0x56121b031530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:02.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.545+0000 7f30fbe00640 1 --2- 172.21.15.16:0/1862567248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56121ac661b0 0x56121b031530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:02.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.545+0000 7f30fbe00640 1 -- 172.21.15.16:0/1862567248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56121ac8f070 msgr2=0x56121b03e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:02.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.545+0000 7f30fbe00640 1 --2- 172.21.15.16:0/1862567248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56121ac8f070 0x56121b03e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:02.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.545+0000 7f30fbe00640 1 -- 172.21.15.16:0/1862567248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56121b039be0 con 0x56121ac6e390 2024-08-31T06:15:02.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.545+0000 7f30fbe00640 1 --2- 172.21.15.16:0/1862567248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56121ac6e390 0x56121b03deb0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f30e80e2dc0 tx=0x7f30e8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd295f7cd9d5605b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:02.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.545+0000 7f30ed200640 1 -- 172.21.15.16:0/1862567248 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e80b05d0 con 0x56121ac6e390 2024-08-31T06:15:02.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.545+0000 7f30ed200640 1 -- 172.21.15.16:0/1862567248 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30e80b5080 con 0x56121ac6e390 2024-08-31T06:15:02.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.545+0000 7f30ed200640 1 -- 172.21.15.16:0/1862567248 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e80b4e50 con 0x56121ac6e390 2024-08-31T06:15:02.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.546+0000 7f310a21b740 1 -- 172.21.15.16:0/1862567248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56121ac6e390 msgr2=0x56121b03deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:02.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.546+0000 7f310a21b740 1 --2- 172.21.15.16:0/1862567248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56121ac6e390 0x56121b03deb0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f30e80e2dc0 tx=0x7f30e8014000 comp rx=0 tx=0).stop 2024-08-31T06:15:02.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.546+0000 7f310a21b740 1 -- 172.21.15.16:0/1862567248 shutdown_connections 2024-08-31T06:15:02.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.546+0000 7f310a21b740 1 --2- 172.21.15.16:0/1862567248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56121ac8f070 0x56121b03e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:02.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.546+0000 7f310a21b740 1 --2- 172.21.15.16:0/1862567248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56121ac6e390 0x56121b03deb0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:02.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.546+0000 7f310a21b740 1 --2- 172.21.15.16:0/1862567248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56121ac661b0 0x56121b031530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:02.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.546+0000 7f310a21b740 1 -- 172.21.15.16:0/1862567248 >> 172.21.15.16:0/1862567248 conn(0x56121ac83460 msgr2=0x56121b030e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:02.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.547+0000 7f310a21b740 1 -- 172.21.15.16:0/1862567248 shutdown_connections 2024-08-31T06:15:02.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.547+0000 7f310a21b740 1 -- 172.21.15.16:0/1862567248 wait complete. 2024-08-31T06:15:02.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.547+0000 7f310a21b740 1 -- start start 2024-08-31T06:15:02.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.548+0000 7f310a21b740 1 Processor -- start 2024-08-31T06:15:02.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.548+0000 7f310a21b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56121ac661b0 0x56121b048d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:02.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.548+0000 7f310a21b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56121ac6e390 0x56121b0492c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:02.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.548+0000 7f310a21b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56121ac8f070 0x56121b0c0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:02.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.548+0000 7f310a21b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f30dc00cd90 con 0x56121ac661b0 2024-08-31T06:15:02.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.548+0000 7f310a21b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f30dc00e200 con 0x56121ac6e390 2024-08-31T06:15:02.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.548+0000 7f310a21b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f30dc00d460 con 0x56121ac8f070 2024-08-31T06:15:02.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.548+0000 7f310a21b740 10 client.0 _ll_register_callbacks cb 0x7f30f00040d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:15:02.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.549+0000 7f3100a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56121ac661b0 0x56121b048d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:02.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.549+0000 7f3100a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56121ac661b0 0x56121b048d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53428/0 (socket says 172.21.15.16:53428) 2024-08-31T06:15:02.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.549+0000 7f3100a00640 1 -- 172.21.15.16:0/2150327832 learned_addr learned my addr 172.21.15.16:0/2150327832 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:02.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.549+0000 7f3101400640 1 --2- 172.21.15.16:0/2150327832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56121ac8f070 0x56121b0c0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:02.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.549+0000 7f30fbe00640 1 --2- 172.21.15.16:0/2150327832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56121ac6e390 0x56121b0492c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:02.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.549+0000 7f3100a00640 1 -- 172.21.15.16:0/2150327832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56121ac8f070 msgr2=0x56121b0c0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:02.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.549+0000 7f3100a00640 1 --2- 172.21.15.16:0/2150327832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56121ac8f070 0x56121b0c0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:02.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.549+0000 7f3100a00640 1 -- 172.21.15.16:0/2150327832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56121ac6e390 msgr2=0x56121b0492c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:02.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.549+0000 7f3100a00640 1 --2- 172.21.15.16:0/2150327832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56121ac6e390 0x56121b0492c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:02.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.549+0000 7f3100a00640 1 -- 172.21.15.16:0/2150327832 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56121b0c0a80 con 0x56121ac661b0 2024-08-31T06:15:02.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.549+0000 7f30fbe00640 1 --2- 172.21.15.16:0/2150327832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56121ac6e390 0x56121b0492c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:15:02.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.549+0000 7f3100a00640 1 --2- 172.21.15.16:0/2150327832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56121ac661b0 0x56121b048d00 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f30f0009900 tx=0x7f30f0003fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc785714af8c834 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:02.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.549+0000 7f30e2000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:15:02.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.550+0000 7f30e2000640 1 -- 172.21.15.16:0/2150327832 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f005c060 con 0x56121ac661b0 2024-08-31T06:15:02.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.550+0000 7f30e2000640 1 -- 172.21.15.16:0/2150327832 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7f30f0005510 con 0x56121ac661b0 2024-08-31T06:15:02.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.550+0000 7f310a21b740 1 -- 172.21.15.16:0/2150327832 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30e80b5080 con 0x56121ac661b0 2024-08-31T06:15:02.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.550+0000 7f30e2000640 1 -- 172.21.15.16:0/2150327832 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f30f0035050 con 0x56121ac661b0 2024-08-31T06:15:02.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.550+0000 7f310a21b740 10 client.17154 fetch_fsmap learned FSMap version 39 2024-08-31T06:15:02.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.550+0000 7f310a21b740 1 -- 172.21.15.16:0/2150327832 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56121b0c1c10 con 0x56121ac661b0 2024-08-31T06:15:02.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.550+0000 7f30e2000640 1 -- 172.21.15.16:0/2150327832 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30e80b5080 con 0x56121ac661b0 2024-08-31T06:15:02.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.551+0000 7f30e2000640 1 -- 172.21.15.16:0/2150327832 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f30f0056070 con 0x56121ac661b0 2024-08-31T06:15:02.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.551+0000 7f310a21b740 10 client.17154 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T06:15:02.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.551+0000 7f310a21b740 10 client.17154 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:15:02.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.551+0000 7f310a21b740 20 client.17154 populate_metadata read hostname 'smithi016' 2024-08-31T06:15:02.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.551+0000 7f310a21b740 1 --2- 172.21.15.16:0/2150327832 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56121ac22da0 0x56121b0c4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:02.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.551+0000 7f310a21b740 4 client.17154 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:15:02.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.551+0000 7f310a21b740 1 -- 172.21.15.16:0/2150327832 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56121afa91a0 con 0x56121ac22da0 2024-08-31T06:15:02.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.551+0000 7f30fbe00640 1 --2- 172.21.15.16:0/2150327832 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56121ac22da0 0x56121b0c4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:02.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.552+0000 7f30fbe00640 1 --2- 172.21.15.16:0/2150327832 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56121ac22da0 0x56121b0c4370 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d862188abeeda7a0 server_cookie=fd18f115ec97c81c in_seq=0 out_seq=0 2024-08-31T06:15:02.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.552+0000 7f30e2000640 10 client.17154 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:15:02.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.552+0000 7f30e2000640 1 -- 172.21.15.16:0/2150327832 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56121afa91a0 con 0x56121ac22da0 2024-08-31T06:15:02.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.552+0000 7f30e2000640 10 client.17154 handle_command_reply: tid=0 2024-08-31T06:15:02.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.552+0000 7f310a21b740 1 client.17154 shutdown 2024-08-31T06:15:02.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.552+0000 7f310a21b740 1 -- 172.21.15.16:0/2150327832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56121ac661b0 msgr2=0x56121b048d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:02.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.552+0000 7f310a21b740 1 --2- 172.21.15.16:0/2150327832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56121ac661b0 0x56121b048d00 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f30f0009900 tx=0x7f30f0003fd0 comp rx=0 tx=0).stop 2024-08-31T06:15:02.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.552+0000 7f310a21b740 1 -- 172.21.15.16:0/2150327832 shutdown_connections 2024-08-31T06:15:02.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.552+0000 7f310a21b740 1 --2- 172.21.15.16:0/2150327832 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56121ac22da0 0x56121b0c4370 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:02.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.552+0000 7f310a21b740 1 --2- 172.21.15.16:0/2150327832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56121ac8f070 0x56121b0c0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:02.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.552+0000 7f310a21b740 1 --2- 172.21.15.16:0/2150327832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56121ac6e390 0x56121b0492c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:02.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.552+0000 7f310a21b740 1 --2- 172.21.15.16:0/2150327832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56121ac661b0 0x56121b048d00 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:02.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.552+0000 7f310a21b740 1 -- 172.21.15.16:0/2150327832 >> 172.21.15.16:0/2150327832 conn(0x56121ac83460 msgr2=0x56121b03deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:02.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.552+0000 7f30e2000640 0 client.17154 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:15:02.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.553+0000 7f310a21b740 1 -- 172.21.15.16:0/2150327832 shutdown_connections 2024-08-31T06:15:02.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.553+0000 7f310a21b740 1 -- 172.21.15.16:0/2150327832 wait complete. 2024-08-31T06:15:02.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.553+0000 7f310a21b740 20 client.17154 trim_cache size 0 max 16384 2024-08-31T06:15:02.558 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:15:02.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.553+0000 7f3103400640 1 -- 172.21.15.16:0/3570993742 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f30e40b4c30 msgr2=0x7f30e40b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:02.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.553+0000 7f3103400640 1 --2- 172.21.15.16:0/3570993742 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f30e40b4c30 0x7f30e40b7140 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f30f4006010 tx=0x7f30f4000f40 comp rx=0 tx=0).stop 2024-08-31T06:15:02.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.553+0000 7f3103400640 1 -- 172.21.15.16:0/3570993742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30fc109260 msgr2=0x7f30fc19a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:02.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.553+0000 7f3103400640 1 --2- 172.21.15.16:0/3570993742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30fc109260 0x7f30fc19a810 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f30e800e9f0 tx=0x7f30e800edf0 comp rx=0 tx=0).stop 2024-08-31T06:15:02.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.554+0000 7f3103400640 1 -- 172.21.15.16:0/3570993742 shutdown_connections 2024-08-31T06:15:02.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.554+0000 7f3103400640 1 --2- 172.21.15.16:0/3570993742 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f30e40b4c30 0x7f30e40b7140 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:02.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.554+0000 7f3103400640 1 --2- 172.21.15.16:0/3570993742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30fc109bb0 0x7f30fc19fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:02.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.554+0000 7f3103400640 1 --2- 172.21.15.16:0/3570993742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30fc109260 0x7f30fc19a810 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:02.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.554+0000 7f3103400640 1 --2- 172.21.15.16:0/3570993742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30fc103920 0x7f30fc19a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:02.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.554+0000 7f3103400640 1 -- 172.21.15.16:0/3570993742 >> 172.21.15.16:0/3570993742 conn(0x7f30fc0ff370 msgr2=0x7f30fc101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:02.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.554+0000 7f3103400640 1 -- 172.21.15.16:0/3570993742 shutdown_connections 2024-08-31T06:15:02.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:02.554+0000 7f3103400640 1 -- 172.21.15.16:0/3570993742 wait complete. 2024-08-31T06:15:02.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:02 smithi016 ceph-mon[17194]: pgmap v602: 97 pgs: 97 active+clean; 3.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 20 KiB/s wr, 11 op/s 2024-08-31T06:15:02.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:02 smithi155 ceph-mon[21742]: pgmap v602: 97 pgs: 97 active+clean; 3.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 20 KiB/s wr, 11 op/s 2024-08-31T06:15:02.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:02 smithi184 ceph-mon[22331]: pgmap v602: 97 pgs: 97 active+clean; 3.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 20 KiB/s wr, 11 op/s 2024-08-31T06:15:03.607 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:15:04.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.334+0000 7fa142c00640 1 -- 172.21.15.16:0/3480320458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa13c075e20 msgr2=0x7fa13c10f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:04.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.334+0000 7fa142c00640 1 --2- 172.21.15.16:0/3480320458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa13c075e20 0x7fa13c10f9c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa128009900 tx=0x7fa12802a410 comp rx=0 tx=0).stop 2024-08-31T06:15:04.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.334+0000 7fa142c00640 1 -- 172.21.15.16:0/3480320458 shutdown_connections 2024-08-31T06:15:04.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.334+0000 7fa142c00640 1 --2- 172.21.15.16:0/3480320458 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa13c10ff00 0x7fa13c112320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:04.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.334+0000 7fa142c00640 1 --2- 172.21.15.16:0/3480320458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa13c075e20 0x7fa13c10f9c0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:04.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.334+0000 7fa142c00640 1 --2- 172.21.15.16:0/3480320458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa13c0754d0 0x7fa13c0758e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:04.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.334+0000 7fa142c00640 1 -- 172.21.15.16:0/3480320458 >> 172.21.15.16:0/3480320458 conn(0x7fa13c0ff370 msgr2=0x7fa13c1017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:04.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.334+0000 7fa142c00640 1 -- 172.21.15.16:0/3480320458 shutdown_connections 2024-08-31T06:15:04.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.334+0000 7fa142c00640 1 -- 172.21.15.16:0/3480320458 wait complete. 2024-08-31T06:15:04.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.335+0000 7fa142c00640 1 Processor -- start 2024-08-31T06:15:04.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.336+0000 7fa142c00640 1 -- start start 2024-08-31T06:15:04.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.336+0000 7fa142c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa13c0754d0 0x7fa13c19c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:04.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.336+0000 7fa142c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa13c075e20 0x7fa13c19ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:04.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.336+0000 7fa142c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa13c10ff00 0x7fa13c1a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:04.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.336+0000 7fa142c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa13c115a40 con 0x7fa13c075e20 2024-08-31T06:15:04.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.336+0000 7fa142c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa13c1158c0 con 0x7fa13c0754d0 2024-08-31T06:15:04.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.336+0000 7fa142c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa13c115bc0 con 0x7fa13c10ff00 2024-08-31T06:15:04.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.336+0000 7fa13be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa13c0754d0 0x7fa13c19c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:04.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.336+0000 7fa13b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa13c075e20 0x7fa13c19ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:04.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.336+0000 7fa13be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa13c0754d0 0x7fa13c19c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:35406/0 (socket says 172.21.15.16:35406) 2024-08-31T06:15:04.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.336+0000 7fa13be00640 1 -- 172.21.15.16:0/4218115374 learned_addr learned my addr 172.21.15.16:0/4218115374 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:04.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.337+0000 7fa140c00640 1 --2- 172.21.15.16:0/4218115374 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa13c10ff00 0x7fa13c1a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:04.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.337+0000 7fa13be00640 1 -- 172.21.15.16:0/4218115374 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa13c10ff00 msgr2=0x7fa13c1a1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:04.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.337+0000 7fa13be00640 1 --2- 172.21.15.16:0/4218115374 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa13c10ff00 0x7fa13c1a1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:04.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.337+0000 7fa13be00640 1 -- 172.21.15.16:0/4218115374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa13c075e20 msgr2=0x7fa13c19ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:04.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.337+0000 7fa13be00640 1 --2- 172.21.15.16:0/4218115374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa13c075e20 0x7fa13c19ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:04.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.337+0000 7fa13be00640 1 -- 172.21.15.16:0/4218115374 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa13c1a2400 con 0x7fa13c0754d0 2024-08-31T06:15:04.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.337+0000 7fa13b400640 1 --2- 172.21.15.16:0/4218115374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa13c075e20 0x7fa13c19ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:15:04.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.337+0000 7fa13be00640 1 --2- 172.21.15.16:0/4218115374 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa13c0754d0 0x7fa13c19c490 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa12c00e9f0 tx=0x7fa12c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:04.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.338+0000 7fa138c00640 1 -- 172.21.15.16:0/4218115374 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12c00cd80 con 0x7fa13c0754d0 2024-08-31T06:15:04.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.338+0000 7fa142c00640 1 -- 172.21.15.16:0/4218115374 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa13c1a26f0 con 0x7fa13c0754d0 2024-08-31T06:15:04.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.338+0000 7fa138c00640 1 -- 172.21.15.16:0/4218115374 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa12c0054d0 con 0x7fa13c0754d0 2024-08-31T06:15:04.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.338+0000 7fa138c00640 1 -- 172.21.15.16:0/4218115374 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12c010690 con 0x7fa13c0754d0 2024-08-31T06:15:04.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.338+0000 7fa142c00640 1 -- 172.21.15.16:0/4218115374 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa13c1a8e50 con 0x7fa13c0754d0 2024-08-31T06:15:04.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.339+0000 7fa142c00640 1 -- 172.21.15.16:0/4218115374 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa13c0758e0 con 0x7fa13c0754d0 2024-08-31T06:15:04.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.342+0000 7fa138c00640 1 -- 172.21.15.16:0/4218115374 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fa12c004e30 con 0x7fa13c0754d0 2024-08-31T06:15:04.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.343+0000 7fa138c00640 1 --2- 172.21.15.16:0/4218115374 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa1200b4ca0 0x7fa1200b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:04.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.343+0000 7fa138c00640 1 -- 172.21.15.16:0/4218115374 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7fa12c0e5b40 con 0x7fa13c0754d0 2024-08-31T06:15:04.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.343+0000 7fa13b400640 1 --2- 172.21.15.16:0/4218115374 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa1200b4ca0 0x7fa1200b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:04.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.344+0000 7fa13b400640 1 --2- 172.21.15.16:0/4218115374 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa1200b4ca0 0x7fa1200b71b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa128009230 tx=0x7fa128031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:04.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.345+0000 7fa138c00640 1 -- 172.21.15.16:0/4218115374 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fa12c0ac900 con 0x7fa13c0754d0 2024-08-31T06:15:04.582 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T06:15:04.582 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T06:15:04.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.581+0000 7fa142c00640 1 -- 172.21.15.16:0/4218115374 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa13c19d5c0 con 0x7fa13c0754d0 2024-08-31T06:15:04.587 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:15:04.587 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":39,"btime":"2024-08-31T06:14:07:075654+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:14:07.025476+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24737,24724,24712]},"id":1}]} 2024-08-31T06:15:04.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.582+0000 7fa138c00640 1 -- 172.21.15.16:0/4218115374 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11920 (secure 0 0 0) 0x7fa13c19d5c0 con 0x7fa13c0754d0 2024-08-31T06:15:04.588 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 39 2024-08-31T06:15:04.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.584+0000 7fa142c00640 1 -- 172.21.15.16:0/4218115374 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa1200b4ca0 msgr2=0x7fa1200b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:04.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.584+0000 7fa142c00640 1 --2- 172.21.15.16:0/4218115374 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa1200b4ca0 0x7fa1200b71b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa128009230 tx=0x7fa128031400 comp rx=0 tx=0).stop 2024-08-31T06:15:04.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.584+0000 7fa142c00640 1 -- 172.21.15.16:0/4218115374 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa13c0754d0 msgr2=0x7fa13c19c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:04.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.584+0000 7fa142c00640 1 --2- 172.21.15.16:0/4218115374 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa13c0754d0 0x7fa13c19c490 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa12c00e9f0 tx=0x7fa12c00edf0 comp rx=0 tx=0).stop 2024-08-31T06:15:04.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.585+0000 7fa142c00640 1 -- 172.21.15.16:0/4218115374 shutdown_connections 2024-08-31T06:15:04.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.585+0000 7fa142c00640 1 --2- 172.21.15.16:0/4218115374 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa1200b4ca0 0x7fa1200b71b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:04.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.585+0000 7fa142c00640 1 --2- 172.21.15.16:0/4218115374 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa13c10ff00 0x7fa13c1a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:04.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.585+0000 7fa142c00640 1 --2- 172.21.15.16:0/4218115374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa13c075e20 0x7fa13c19ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:04.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.585+0000 7fa142c00640 1 --2- 172.21.15.16:0/4218115374 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa13c0754d0 0x7fa13c19c490 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:04.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.585+0000 7fa142c00640 1 -- 172.21.15.16:0/4218115374 >> 172.21.15.16:0/4218115374 conn(0x7fa13c0ff370 msgr2=0x7fa13c109ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:04.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.585+0000 7fa142c00640 1 -- 172.21.15.16:0/4218115374 shutdown_connections 2024-08-31T06:15:04.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:04.585+0000 7fa142c00640 1 -- 172.21.15.16:0/4218115374 wait complete. 2024-08-31T06:15:04.833 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:04 smithi016 ceph-mon[17194]: pgmap v603: 97 pgs: 97 active+clean; 3.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-08-31T06:15:04.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:04 smithi155 ceph-mon[21742]: pgmap v603: 97 pgs: 97 active+clean; 3.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-08-31T06:15:04.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:04 smithi184 ceph-mon[22331]: pgmap v603: 97 pgs: 97 active+clean; 3.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-08-31T06:15:05.501 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:15:05.530 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24737 quiesce db --set-id cf5ba332 --release --await' 2024-08-31T06:15:05.530 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24737 quiesce db --set-id cf5ba332 --release --await 2024-08-31T06:15:05.783 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:05 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/4218115374' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:15:05.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:05 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/4218115374' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:15:05.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:05 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/4218115374' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:15:06.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.120+0000 7fce09a00640 1 -- 172.21.15.16:0/734581244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce04109090 msgr2=0x7fce041094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:06.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.120+0000 7fce09a00640 1 --2- 172.21.15.16:0/734581244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce04109090 0x7fce041094a0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0009900 tx=0x7fcdf002a410 comp rx=0 tx=0).stop 2024-08-31T06:15:06.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.120+0000 7fce09a00640 1 -- 172.21.15.16:0/734581244 shutdown_connections 2024-08-31T06:15:06.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.120+0000 7fce09a00640 1 --2- 172.21.15.16:0/734581244 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce041099e0 0x7fce04110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:06.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.120+0000 7fce09a00640 1 --2- 172.21.15.16:0/734581244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce04109090 0x7fce041094a0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:06.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.121+0000 7fce09a00640 1 --2- 172.21.15.16:0/734581244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce04103750 0x7fce04103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:06.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.121+0000 7fce09a00640 1 -- 172.21.15.16:0/734581244 >> 172.21.15.16:0/734581244 conn(0x7fce040ff1e0 msgr2=0x7fce04101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:06.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.121+0000 7fce09a00640 1 -- 172.21.15.16:0/734581244 shutdown_connections 2024-08-31T06:15:06.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.121+0000 7fce09a00640 1 -- 172.21.15.16:0/734581244 wait complete. 2024-08-31T06:15:06.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.122+0000 7fce09a00640 1 Processor -- start 2024-08-31T06:15:06.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.122+0000 7fce09a00640 1 -- start start 2024-08-31T06:15:06.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.123+0000 7fce09a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce04103750 0x7fce0419c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:06.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.123+0000 7fce09a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce04109090 0x7fce0419c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:06.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.123+0000 7fce09a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce041099e0 0x7fce041a1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:06.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.123+0000 7fce09a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fce041136a0 con 0x7fce041099e0 2024-08-31T06:15:06.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.123+0000 7fce09a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fce04113520 con 0x7fce04109090 2024-08-31T06:15:06.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.123+0000 7fce09a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fce04113820 con 0x7fce04103750 2024-08-31T06:15:06.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.123+0000 7fce03e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce04103750 0x7fce0419c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:06.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.123+0000 7fce03e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce04103750 0x7fce0419c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:53884/0 (socket says 172.21.15.16:53884) 2024-08-31T06:15:06.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.123+0000 7fce03e00640 1 -- 172.21.15.16:0/484836251 learned_addr learned my addr 172.21.15.16:0/484836251 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:06.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.123+0000 7fce09000640 1 --2- 172.21.15.16:0/484836251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce041099e0 0x7fce041a1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:06.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.123+0000 7fce03400640 1 --2- 172.21.15.16:0/484836251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce04109090 0x7fce0419c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:06.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.123+0000 7fce03e00640 1 -- 172.21.15.16:0/484836251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce04109090 msgr2=0x7fce0419c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:06.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.123+0000 7fce03e00640 1 --2- 172.21.15.16:0/484836251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce04109090 0x7fce0419c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:06.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.123+0000 7fce03e00640 1 -- 172.21.15.16:0/484836251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce041099e0 msgr2=0x7fce041a1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:06.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.123+0000 7fce03e00640 1 --2- 172.21.15.16:0/484836251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce041099e0 0x7fce041a1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:06.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.123+0000 7fce03e00640 1 -- 172.21.15.16:0/484836251 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce041a2260 con 0x7fce04103750 2024-08-31T06:15:06.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.123+0000 7fce03400640 1 --2- 172.21.15.16:0/484836251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce04109090 0x7fce0419c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:15:06.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.124+0000 7fce03e00640 1 --2- 172.21.15.16:0/484836251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce04103750 0x7fce0419c2f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fcdf400e9f0 tx=0x7fcdf400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:06.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.124+0000 7fce00c00640 1 -- 172.21.15.16:0/484836251 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf400cd80 con 0x7fce04103750 2024-08-31T06:15:06.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.124+0000 7fce09a00640 1 -- 172.21.15.16:0/484836251 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce041a2550 con 0x7fce04103750 2024-08-31T06:15:06.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.124+0000 7fce00c00640 1 -- 172.21.15.16:0/484836251 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdf40108e0 con 0x7fce04103750 2024-08-31T06:15:06.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.124+0000 7fce00c00640 1 -- 172.21.15.16:0/484836251 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4018980 con 0x7fce04103750 2024-08-31T06:15:06.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.124+0000 7fce09a00640 1 -- 172.21.15.16:0/484836251 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce041a8cb0 con 0x7fce04103750 2024-08-31T06:15:06.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.127+0000 7fce11c74740 1 Processor -- start 2024-08-31T06:15:06.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.127+0000 7fce11c74740 1 -- start start 2024-08-31T06:15:06.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.128+0000 7fce11c74740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf9e7ed350 0x55cf9eb937d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:06.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.128+0000 7fce11c74740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf9e7c4490 0x55cf9eb93d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:06.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.128+0000 7fce11c74740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf9e7cc670 0x55cf9eb94250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:06.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.128+0000 7fce11c74740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cf9eb09750 con 0x55cf9e7c4490 2024-08-31T06:15:06.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.128+0000 7fce11c74740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55cf9eaf8ce0 con 0x55cf9e7ed350 2024-08-31T06:15:06.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.128+0000 7fce11c74740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55cf9eafcc90 con 0x55cf9e7cc670 2024-08-31T06:15:06.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.128+0000 7fce09000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf9e7ed350 0x55cf9eb937d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:06.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.128+0000 7fce09000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf9e7ed350 0x55cf9eb937d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:34554/0 (socket says 172.21.15.16:34554) 2024-08-31T06:15:06.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.128+0000 7fce09000640 1 -- 172.21.15.16:0/2274628619 learned_addr learned my addr 172.21.15.16:0/2274628619 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:06.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.128+0000 7fce03400640 1 --2- 172.21.15.16:0/2274628619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf9e7cc670 0x55cf9eb94250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:06.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.129+0000 7fce00c00640 1 -- 172.21.15.16:0/484836251 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fcdf4010430 con 0x7fce04103750 2024-08-31T06:15:06.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.129+0000 7fce09000640 1 -- 172.21.15.16:0/2274628619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf9e7cc670 msgr2=0x55cf9eb94250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:06.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.129+0000 7fce03e00640 1 --2- 172.21.15.16:0/2274628619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf9e7c4490 0x55cf9eb93d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:06.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.129+0000 7fce09000640 1 --2- 172.21.15.16:0/2274628619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf9e7cc670 0x55cf9eb94250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:06.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.129+0000 7fce09000640 1 -- 172.21.15.16:0/2274628619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf9e7c4490 msgr2=0x55cf9eb93d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:06.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.129+0000 7fce09000640 1 --2- 172.21.15.16:0/2274628619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf9e7c4490 0x55cf9eb93d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:06.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.129+0000 7fce09000640 1 -- 172.21.15.16:0/2274628619 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf9e858930 con 0x55cf9e7ed350 2024-08-31T06:15:06.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.129+0000 7fce03e00640 1 --2- 172.21.15.16:0/2274628619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf9e7c4490 0x55cf9eb93d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:15:06.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.129+0000 7fce03400640 1 --2- 172.21.15.16:0/2274628619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf9e7cc670 0x55cf9eb94250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:15:06.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.130+0000 7fce09000640 1 --2- 172.21.15.16:0/2274628619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf9e7ed350 0x55cf9eb937d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009fd0 tx=0x7fcdfc0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd1e640937fc966d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:06.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.130+0000 7fcdf9600640 1 -- 172.21.15.16:0/2274628619 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc004d20 con 0x55cf9e7ed350 2024-08-31T06:15:06.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.130+0000 7fcdf9600640 1 -- 172.21.15.16:0/2274628619 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdfc004050 con 0x55cf9e7ed350 2024-08-31T06:15:06.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.130+0000 7fcdf9600640 1 -- 172.21.15.16:0/2274628619 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc005280 con 0x55cf9e7ed350 2024-08-31T06:15:06.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.131+0000 7fce00c00640 1 --2- 172.21.15.16:0/484836251 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fcde80bd530 0x7fcde80bfa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:06.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.131+0000 7fce00c00640 1 -- 172.21.15.16:0/484836251 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7fcdf40e5930 con 0x7fce04103750 2024-08-31T06:15:06.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.131+0000 7fce11c74740 1 -- 172.21.15.16:0/2274628619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf9e7ed350 msgr2=0x55cf9eb937d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:06.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.131+0000 7fce11c74740 1 --2- 172.21.15.16:0/2274628619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf9e7ed350 0x55cf9eb937d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009fd0 tx=0x7fcdfc0118d0 comp rx=0 tx=0).stop 2024-08-31T06:15:06.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.131+0000 7fce03e00640 1 --2- 172.21.15.16:0/484836251 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fcde80bd530 0x7fcde80bfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:06.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.131+0000 7fce11c74740 1 -- 172.21.15.16:0/2274628619 shutdown_connections 2024-08-31T06:15:06.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.131+0000 7fce11c74740 1 --2- 172.21.15.16:0/2274628619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf9e7cc670 0x55cf9eb94250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:06.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.131+0000 7fce11c74740 1 --2- 172.21.15.16:0/2274628619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf9e7c4490 0x55cf9eb93d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:06.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.131+0000 7fce11c74740 1 --2- 172.21.15.16:0/2274628619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf9e7ed350 0x55cf9eb937d0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:06.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.131+0000 7fce11c74740 1 -- 172.21.15.16:0/2274628619 >> 172.21.15.16:0/2274628619 conn(0x55cf9e7e1740 msgr2=0x55cf9eb916c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:06.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.131+0000 7fce11c74740 1 -- 172.21.15.16:0/2274628619 shutdown_connections 2024-08-31T06:15:06.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.132+0000 7fce11c74740 1 -- 172.21.15.16:0/2274628619 wait complete. 2024-08-31T06:15:06.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.132+0000 7fce03e00640 1 --2- 172.21.15.16:0/484836251 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fcde80bd530 0x7fcde80bfa40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x55cf9eb92450 tx=0x7fcdf4018420 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:06.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.132+0000 7fce11c74740 1 -- start start 2024-08-31T06:15:06.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.132+0000 7fce11c74740 1 Processor -- start 2024-08-31T06:15:06.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.133+0000 7fce11c74740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf9e7c4490 0x55cf9eb9f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:06.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.133+0000 7fce11c74740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf9e7cc670 0x55cf9eb9f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:06.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.133+0000 7fce11c74740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf9e7ed350 0x55cf9ebab2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:06.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.133+0000 7fce11c74740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cf9e95b890 con 0x55cf9e7ed350 2024-08-31T06:15:06.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.133+0000 7fce11c74740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55cf9ea7fcd0 con 0x55cf9e7cc670 2024-08-31T06:15:06.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.133+0000 7fce11c74740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55cf9eba1fb0 con 0x55cf9e7c4490 2024-08-31T06:15:06.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.133+0000 7fce11c74740 10 client.0 _ll_register_callbacks cb 0x55cf9eaf3ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:15:06.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.133+0000 7fce09000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf9e7ed350 0x55cf9ebab2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:06.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.133+0000 7fce09000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf9e7c4490 0x55cf9eb9f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:06.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.133+0000 7fce09000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf9e7c4490 0x55cf9eb9f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:53890/0 (socket says 172.21.15.16:53890) 2024-08-31T06:15:06.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.133+0000 7fce09000640 1 -- 172.21.15.16:0/2623967183 learned_addr learned my addr 172.21.15.16:0/2623967183 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:06.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.134+0000 7fce03400640 1 --2- 172.21.15.16:0/2623967183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf9e7cc670 0x55cf9eb9f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:06.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.134+0000 7fce09000640 1 -- 172.21.15.16:0/2623967183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf9e7cc670 msgr2=0x55cf9eb9f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:06.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.134+0000 7fce09000640 1 --2- 172.21.15.16:0/2623967183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf9e7cc670 0x55cf9eb9f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:06.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.134+0000 7fce09000640 1 -- 172.21.15.16:0/2623967183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf9e7ed350 msgr2=0x55cf9ebab2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:06.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.134+0000 7fce09000640 1 --2- 172.21.15.16:0/2623967183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf9e7ed350 0x55cf9ebab2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:06.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.134+0000 7fce09000640 1 -- 172.21.15.16:0/2623967183 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf9eac91b0 con 0x55cf9e7c4490 2024-08-31T06:15:06.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.134+0000 7fce09000640 1 --2- 172.21.15.16:0/2623967183 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf9e7c4490 0x55cf9eb9f430 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009fa0 tx=0x7fcdfc000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cca370f94a11755f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:06.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.134+0000 7fcdee000640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:15:06.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.135+0000 7fcdee000640 1 -- 172.21.15.16:0/2623967183 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc004c40 con 0x55cf9e7c4490 2024-08-31T06:15:06.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.135+0000 7fcdee000640 1 -- 172.21.15.16:0/2623967183 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7fcdfc004de0 con 0x55cf9e7c4490 2024-08-31T06:15:06.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.135+0000 7fce11c74740 1 -- 172.21.15.16:0/2623967183 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdfc004050 con 0x55cf9e7c4490 2024-08-31T06:15:06.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.135+0000 7fcdee000640 1 -- 172.21.15.16:0/2623967183 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcdfc036050 con 0x55cf9e7c4490 2024-08-31T06:15:06.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.135+0000 7fce11c74740 10 client.27379 fetch_fsmap learned FSMap version 39 2024-08-31T06:15:06.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.135+0000 7fce11c74740 1 -- 172.21.15.16:0/2623967183 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55cf9ebadfb0 con 0x55cf9e7c4490 2024-08-31T06:15:06.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.135+0000 7fcdee000640 1 -- 172.21.15.16:0/2623967183 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdfc004050 con 0x55cf9e7c4490 2024-08-31T06:15:06.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.136+0000 7fcdee000640 1 -- 172.21.15.16:0/2623967183 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fcdfc00f4e0 con 0x55cf9e7c4490 2024-08-31T06:15:06.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.136+0000 7fce11c74740 10 client.27379 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T06:15:06.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.136+0000 7fce11c74740 10 client.27379 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:15:06.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.136+0000 7fce11c74740 20 client.27379 populate_metadata read hostname 'smithi016' 2024-08-31T06:15:06.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.136+0000 7fce11c74740 1 --2- 172.21.15.16:0/2623967183 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55cf9e7abcd0 0x55cf9ebb13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:06.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.136+0000 7fce11c74740 4 client.27379 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:15:06.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.136+0000 7fce11c74740 1 -- 172.21.15.16:0/2623967183 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf9eb0b6a0 con 0x55cf9e7abcd0 2024-08-31T06:15:06.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.136+0000 7fce03400640 1 --2- 172.21.15.16:0/2623967183 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55cf9e7abcd0 0x55cf9ebb13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:06.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.137+0000 7fce03400640 1 --2- 172.21.15.16:0/2623967183 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55cf9e7abcd0 0x55cf9ebb13c0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dee118a79cc587b server_cookie=76093e9159b24be1 in_seq=0 out_seq=0 2024-08-31T06:15:06.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.137+0000 7fcdee000640 10 client.27379 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:15:06.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.138+0000 7fcdee000640 1 -- 172.21.15.16:0/2623967183 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cf9eb0b6a0 con 0x55cf9e7abcd0 2024-08-31T06:15:06.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.138+0000 7fcdee000640 10 client.27379 handle_command_reply: tid=0 2024-08-31T06:15:06.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.138+0000 7fce11c74740 1 client.27379 shutdown 2024-08-31T06:15:06.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.138+0000 7fce11c74740 1 -- 172.21.15.16:0/2623967183 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf9e7c4490 msgr2=0x55cf9eb9f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:06.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.138+0000 7fce11c74740 1 --2- 172.21.15.16:0/2623967183 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf9e7c4490 0x55cf9eb9f430 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009fa0 tx=0x7fcdfc000f40 comp rx=0 tx=0).stop 2024-08-31T06:15:06.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.138+0000 7fce11c74740 1 -- 172.21.15.16:0/2623967183 shutdown_connections 2024-08-31T06:15:06.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.138+0000 7fce11c74740 1 --2- 172.21.15.16:0/2623967183 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55cf9e7abcd0 0x55cf9ebb13c0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:06.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.138+0000 7fce11c74740 1 --2- 172.21.15.16:0/2623967183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf9e7ed350 0x55cf9ebab2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:06.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.138+0000 7fce11c74740 1 --2- 172.21.15.16:0/2623967183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf9e7cc670 0x55cf9eb9f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:06.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.138+0000 7fce11c74740 1 --2- 172.21.15.16:0/2623967183 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf9e7c4490 0x55cf9eb9f430 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:06.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.138+0000 7fcdee000640 0 client.27379 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:15:06.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.138+0000 7fce11c74740 1 -- 172.21.15.16:0/2623967183 >> 172.21.15.16:0/2623967183 conn(0x55cf9e7e1740 msgr2=0x55cf9eb96960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:06.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.138+0000 7fce11c74740 1 -- 172.21.15.16:0/2623967183 shutdown_connections 2024-08-31T06:15:06.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.138+0000 7fce11c74740 1 -- 172.21.15.16:0/2623967183 wait complete. 2024-08-31T06:15:06.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.139+0000 7fce11c74740 20 client.27379 trim_cache size 0 max 16384 2024-08-31T06:15:06.152 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:15:06.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.161+0000 7fce11c74740 1 Processor -- start 2024-08-31T06:15:06.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.161+0000 7fce11c74740 1 -- start start 2024-08-31T06:15:06.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.161+0000 7fce11c74740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf9e7c4490 0x55cf9eb96f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:06.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.161+0000 7fce11c74740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf9e7cc670 0x55cf9eba0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:06.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.161+0000 7fce11c74740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf9e7ed350 0x55cf9ebab1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:06.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.161+0000 7fce11c74740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fce041afa50 con 0x55cf9e7c4490 2024-08-31T06:15:06.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.161+0000 7fce11c74740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fce041ad2c0 con 0x55cf9e7cc670 2024-08-31T06:15:06.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.161+0000 7fce11c74740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fce041ab1b0 con 0x55cf9e7ed350 2024-08-31T06:15:06.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.162+0000 7fce09000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf9e7c4490 0x55cf9eb96f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:06.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.162+0000 7fce03400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf9e7cc670 0x55cf9eba0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:06.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.162+0000 7fce03400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf9e7cc670 0x55cf9eba0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:34578/0 (socket says 172.21.15.16:34578) 2024-08-31T06:15:06.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.162+0000 7fce03400640 1 -- 172.21.15.16:0/2590459663 learned_addr learned my addr 172.21.15.16:0/2590459663 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:06.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.162+0000 7fce09000640 1 --2- 172.21.15.16:0/2590459663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf9e7ed350 0x55cf9ebab1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:06.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.162+0000 7fce03400640 1 -- 172.21.15.16:0/2590459663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf9e7ed350 msgr2=0x55cf9ebab1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:06.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.162+0000 7fce03400640 1 --2- 172.21.15.16:0/2590459663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf9e7ed350 0x55cf9ebab1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:06.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.162+0000 7fce03400640 1 -- 172.21.15.16:0/2590459663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf9e7c4490 msgr2=0x55cf9eb96f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:06.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.162+0000 7fce03400640 1 --2- 172.21.15.16:0/2590459663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf9e7c4490 0x55cf9eb96f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:06.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.162+0000 7fce03400640 1 -- 172.21.15.16:0/2590459663 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf9eb9fe10 con 0x55cf9e7cc670 2024-08-31T06:15:06.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.162+0000 7fce03400640 1 --2- 172.21.15.16:0/2590459663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf9e7cc670 0x55cf9eba0e60 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0005ab0 tx=0x7fcdf0009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38cb764e885b1bbf server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:06.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.163+0000 7fcdf8c00640 1 -- 172.21.15.16:0/2590459663 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0035040 con 0x55cf9e7cc670 2024-08-31T06:15:06.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.163+0000 7fcdf8c00640 1 -- 172.21.15.16:0/2590459663 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdf00054f0 con 0x55cf9e7cc670 2024-08-31T06:15:06.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.163+0000 7fcdf8c00640 1 -- 172.21.15.16:0/2590459663 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0004d90 con 0x55cf9e7cc670 2024-08-31T06:15:06.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.164+0000 7fce11c74740 1 -- 172.21.15.16:0/2590459663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf9e7cc670 msgr2=0x55cf9eba0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:06.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.164+0000 7fce11c74740 1 --2- 172.21.15.16:0/2590459663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf9e7cc670 0x55cf9eba0e60 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0005ab0 tx=0x7fcdf0009560 comp rx=0 tx=0).stop 2024-08-31T06:15:06.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.164+0000 7fce11c74740 1 -- 172.21.15.16:0/2590459663 shutdown_connections 2024-08-31T06:15:06.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.164+0000 7fce11c74740 1 --2- 172.21.15.16:0/2590459663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf9e7ed350 0x55cf9ebab1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:06.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.164+0000 7fce11c74740 1 --2- 172.21.15.16:0/2590459663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf9e7cc670 0x55cf9eba0e60 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:06.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.164+0000 7fce11c74740 1 --2- 172.21.15.16:0/2590459663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf9e7c4490 0x55cf9eb96f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:06.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.164+0000 7fce11c74740 1 -- 172.21.15.16:0/2590459663 >> 172.21.15.16:0/2590459663 conn(0x55cf9e7e1740 msgr2=0x55cf9eba0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:06.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.164+0000 7fce11c74740 1 -- 172.21.15.16:0/2590459663 shutdown_connections 2024-08-31T06:15:06.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.164+0000 7fce11c74740 1 -- 172.21.15.16:0/2590459663 wait complete. 2024-08-31T06:15:06.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.164+0000 7fce11c74740 1 -- start start 2024-08-31T06:15:06.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.165+0000 7fce11c74740 1 Processor -- start 2024-08-31T06:15:06.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.165+0000 7fce11c74740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf9e7c4490 0x55cf9eba7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:06.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.165+0000 7fce11c74740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf9e7cc670 0x55cf9eba7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:06.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.165+0000 7fce11c74740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf9e7ed350 0x55cf9eba21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:06.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.165+0000 7fce11c74740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fce041ab800 con 0x55cf9e7c4490 2024-08-31T06:15:06.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.165+0000 7fce11c74740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fce041acbf0 con 0x55cf9e7cc670 2024-08-31T06:15:06.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.165+0000 7fce11c74740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fce041abe50 con 0x55cf9e7ed350 2024-08-31T06:15:06.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.165+0000 7fce11c74740 10 client.0 _ll_register_callbacks cb 0x7fcdf4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:15:06.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.165+0000 7fce09000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf9e7c4490 0x55cf9eba7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:06.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.166+0000 7fce09000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf9e7c4490 0x55cf9eba7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47220/0 (socket says 172.21.15.16:47220) 2024-08-31T06:15:06.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.166+0000 7fce09000640 1 -- 172.21.15.16:0/3620372162 learned_addr learned my addr 172.21.15.16:0/3620372162 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:06.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.166+0000 7fce03400640 1 --2- 172.21.15.16:0/3620372162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf9e7cc670 0x55cf9eba7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:06.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.166+0000 7fce09000640 1 --2- 172.21.15.16:0/3620372162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf9e7ed350 0x55cf9eba21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:06.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.166+0000 7fce09000640 1 -- 172.21.15.16:0/3620372162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf9e7ed350 msgr2=0x55cf9eba21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:06.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.166+0000 7fce09000640 1 --2- 172.21.15.16:0/3620372162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf9e7ed350 0x55cf9eba21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:06.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.166+0000 7fce09000640 1 -- 172.21.15.16:0/3620372162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf9e7cc670 msgr2=0x55cf9eba7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:06.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.166+0000 7fce09000640 1 --2- 172.21.15.16:0/3620372162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf9e7cc670 0x55cf9eba7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:06.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.166+0000 7fce09000640 1 -- 172.21.15.16:0/3620372162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf9eba2730 con 0x55cf9e7c4490 2024-08-31T06:15:06.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.166+0000 7fce03400640 1 --2- 172.21.15.16:0/3620372162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf9e7cc670 0x55cf9eba7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:15:06.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.166+0000 7fce09000640 1 --2- 172.21.15.16:0/3620372162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf9e7c4490 0x55cf9eba7240 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc003d00 tx=0x7fcdfc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=242e2f3317816627 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:06.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.166+0000 7fcdee000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:15:06.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.167+0000 7fcdee000640 1 -- 172.21.15.16:0/3620372162 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc004640 con 0x55cf9e7c4490 2024-08-31T06:15:06.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.167+0000 7fcdee000640 1 -- 172.21.15.16:0/3620372162 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7fcdfc0047e0 con 0x55cf9e7c4490 2024-08-31T06:15:06.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.167+0000 7fce11c74740 1 -- 172.21.15.16:0/3620372162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdf00054f0 con 0x55cf9e7c4490 2024-08-31T06:15:06.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.167+0000 7fcdee000640 1 -- 172.21.15.16:0/3620372162 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcdfc0049e0 con 0x55cf9e7c4490 2024-08-31T06:15:06.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.167+0000 7fce11c74740 10 client.17175 fetch_fsmap learned FSMap version 39 2024-08-31T06:15:06.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.167+0000 7fce11c74740 1 -- 172.21.15.16:0/3620372162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55cf9ec1c360 con 0x55cf9e7c4490 2024-08-31T06:15:06.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.167+0000 7fcdee000640 1 -- 172.21.15.16:0/3620372162 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdf00054f0 con 0x55cf9e7c4490 2024-08-31T06:15:06.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.168+0000 7fcdee000640 1 -- 172.21.15.16:0/3620372162 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fcdfc028b70 con 0x55cf9e7c4490 2024-08-31T06:15:06.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.168+0000 7fce11c74740 10 client.17175 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T06:15:06.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.168+0000 7fce11c74740 10 client.17175 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:15:06.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.168+0000 7fce11c74740 20 client.17175 populate_metadata read hostname 'smithi016' 2024-08-31T06:15:06.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.168+0000 7fce11c74740 1 --2- 172.21.15.16:0/3620372162 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55cf9e781080 0x55cf9ec1ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.168+0000 7fce11c74740 4 client.17175 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T06:15:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.168+0000 7fce11c74740 1 -- 172.21.15.16:0/3620372162 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cf9eb07140 con 0x55cf9e781080 2024-08-31T06:15:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.168+0000 7fce03400640 1 --2- 172.21.15.16:0/3620372162 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55cf9e781080 0x55cf9ec1ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.169+0000 7fce03400640 1 --2- 172.21.15.16:0/3620372162 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55cf9e781080 0x55cf9ec1ea60 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b78fc27aa015b063 server_cookie=58ddcafcaefd85e3 in_seq=0 out_seq=0 2024-08-31T06:15:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.169+0000 7fcdee000640 10 client.17175 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:15:06.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.171+0000 7fcdee000640 1 -- 172.21.15.16:0/3620372162 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cf9eb07140 con 0x55cf9e781080 2024-08-31T06:15:06.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.171+0000 7fcdee000640 10 client.17175 handle_command_reply: tid=0 2024-08-31T06:15:06.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.171+0000 7fce11c74740 1 client.17175 shutdown 2024-08-31T06:15:06.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.171+0000 7fce11c74740 1 -- 172.21.15.16:0/3620372162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf9e7c4490 msgr2=0x55cf9eba7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:06.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.171+0000 7fce11c74740 1 --2- 172.21.15.16:0/3620372162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf9e7c4490 0x55cf9eba7240 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc003d00 tx=0x7fcdfc005fd0 comp rx=0 tx=0).stop 2024-08-31T06:15:06.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.171+0000 7fce11c74740 1 -- 172.21.15.16:0/3620372162 shutdown_connections 2024-08-31T06:15:06.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.171+0000 7fce11c74740 1 --2- 172.21.15.16:0/3620372162 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55cf9e781080 0x55cf9ec1ea60 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:06.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.171+0000 7fce11c74740 1 --2- 172.21.15.16:0/3620372162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf9e7ed350 0x55cf9eba21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:06.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.171+0000 7fce11c74740 1 --2- 172.21.15.16:0/3620372162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf9e7cc670 0x55cf9eba7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:06.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.171+0000 7fce11c74740 1 --2- 172.21.15.16:0/3620372162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf9e7c4490 0x55cf9eba7240 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:06.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.171+0000 7fcdee000640 0 client.17175 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:15:06.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.171+0000 7fce11c74740 1 -- 172.21.15.16:0/3620372162 >> 172.21.15.16:0/3620372162 conn(0x55cf9e7e1740 msgr2=0x55cf9eba04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:06.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.171+0000 7fce11c74740 1 -- 172.21.15.16:0/3620372162 shutdown_connections 2024-08-31T06:15:06.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.172+0000 7fce11c74740 1 -- 172.21.15.16:0/3620372162 wait complete. 2024-08-31T06:15:06.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.172+0000 7fce11c74740 20 client.17175 trim_cache size 0 max 16384 2024-08-31T06:15:06.175 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:15:06.176 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-08-31T06:15:06.176 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "0fdcd762-533d-42be-9289-b71c8e3b81b1", 2024-08-31T06:15:06.176 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-08-31T06:15:06.176 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:15:06.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.173+0000 7fce09a00640 1 -- 172.21.15.16:0/484836251 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fcde80bd530 msgr2=0x7fcde80bfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:06.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.173+0000 7fce09a00640 1 --2- 172.21.15.16:0/484836251 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fcde80bd530 0x7fcde80bfa40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x55cf9eb92450 tx=0x7fcdf4018420 comp rx=0 tx=0).stop 2024-08-31T06:15:06.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.173+0000 7fce09a00640 1 -- 172.21.15.16:0/484836251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce04103750 msgr2=0x7fce0419c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:06.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.173+0000 7fce09a00640 1 --2- 172.21.15.16:0/484836251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce04103750 0x7fce0419c2f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fcdf400e9f0 tx=0x7fcdf400eeb0 comp rx=0 tx=0).stop 2024-08-31T06:15:06.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.173+0000 7fce09a00640 1 -- 172.21.15.16:0/484836251 shutdown_connections 2024-08-31T06:15:06.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.173+0000 7fce09a00640 1 --2- 172.21.15.16:0/484836251 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fcde80bd530 0x7fcde80bfa40 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:06.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.173+0000 7fce09a00640 1 --2- 172.21.15.16:0/484836251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce041099e0 0x7fce041a1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:06.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.173+0000 7fce09a00640 1 --2- 172.21.15.16:0/484836251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce04109090 0x7fce0419c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:06.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.173+0000 7fce09a00640 1 --2- 172.21.15.16:0/484836251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce04103750 0x7fce0419c2f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:06.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.173+0000 7fce09a00640 1 -- 172.21.15.16:0/484836251 >> 172.21.15.16:0/484836251 conn(0x7fce040ff1e0 msgr2=0x7fce0410e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:06.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.173+0000 7fce09a00640 1 -- 172.21.15.16:0/484836251 shutdown_connections 2024-08-31T06:15:06.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:06.173+0000 7fce09a00640 1 -- 172.21.15.16:0/484836251 wait complete. 2024-08-31T06:15:06.628 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 460.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-31T06:15:06.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:06 smithi155 ceph-mon[21742]: pgmap v604: 97 pgs: 97 active+clean; 3.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 327 KiB/s wr, 57 op/s 2024-08-31T06:15:06.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:06 smithi155 ceph-mon[21742]: scrub queued for path: / 2024-08-31T06:15:06.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:06 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:15:06.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:06 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:15:06.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:06 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:15:06.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:06 smithi016 ceph-mon[17194]: pgmap v604: 97 pgs: 97 active+clean; 3.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 327 KiB/s wr, 57 op/s 2024-08-31T06:15:06.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:06 smithi016 ceph-mon[17194]: scrub queued for path: / 2024-08-31T06:15:06.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:06 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:15:06.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:06 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:15:06.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:06 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:15:06.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:06 smithi184 ceph-mon[22331]: pgmap v604: 97 pgs: 97 active+clean; 3.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 327 KiB/s wr, 57 op/s 2024-08-31T06:15:06.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:06 smithi184 ceph-mon[22331]: scrub queued for path: / 2024-08-31T06:15:06.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:06 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:15:06.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:06 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:15:06.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:06 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:15:07.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.190+0000 7fa5d4a00640 1 -- 172.21.15.16:0/3851834709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5d0105950 msgr2=0x7fa5d0105d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:07.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.190+0000 7fa5d4a00640 1 --2- 172.21.15.16:0/3851834709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5d0105950 0x7fa5d0105d60 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0009900 tx=0x7fa5c002a410 comp rx=0 tx=0).stop 2024-08-31T06:15:07.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.190+0000 7fa5d4a00640 1 -- 172.21.15.16:0/3851834709 shutdown_connections 2024-08-31T06:15:07.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.190+0000 7fa5d4a00640 1 --2- 172.21.15.16:0/3851834709 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5d010b290 0x7fa5d010b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:07.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.190+0000 7fa5d4a00640 1 --2- 172.21.15.16:0/3851834709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5d010bbe0 0x7fa5d0112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:07.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.190+0000 7fa5d4a00640 1 --2- 172.21.15.16:0/3851834709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5d0105950 0x7fa5d0105d60 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:07.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.190+0000 7fa5d4a00640 1 -- 172.21.15.16:0/3851834709 >> 172.21.15.16:0/3851834709 conn(0x7fa5d0101420 msgr2=0x7fa5d0103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:07.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.190+0000 7fa5d4a00640 1 -- 172.21.15.16:0/3851834709 shutdown_connections 2024-08-31T06:15:07.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.190+0000 7fa5d4a00640 1 -- 172.21.15.16:0/3851834709 wait complete. 2024-08-31T06:15:07.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.191+0000 7fa5d4a00640 1 Processor -- start 2024-08-31T06:15:07.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.191+0000 7fa5d4a00640 1 -- start start 2024-08-31T06:15:07.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.192+0000 7fa5d4a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5d0105950 0x7fa5d019c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:07.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.192+0000 7fa5d4a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5d010b290 0x7fa5d019c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:07.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.192+0000 7fa5d4a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5d010bbe0 0x7fa5d01a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:07.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.192+0000 7fa5d4a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d01158f0 con 0x7fa5d010bbe0 2024-08-31T06:15:07.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.192+0000 7fa5d4a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d0115770 con 0x7fa5d0105950 2024-08-31T06:15:07.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.192+0000 7fa5d4a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d0115a70 con 0x7fa5d010b290 2024-08-31T06:15:07.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.192+0000 7fa5ce800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5d0105950 0x7fa5d019c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:07.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.192+0000 7fa5ce800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5d0105950 0x7fa5d019c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:34598/0 (socket says 172.21.15.16:34598) 2024-08-31T06:15:07.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.192+0000 7fa5ce800640 1 -- 172.21.15.16:0/2295963000 learned_addr learned my addr 172.21.15.16:0/2295963000 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:07.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.192+0000 7fa5cf200640 1 --2- 172.21.15.16:0/2295963000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5d010bbe0 0x7fa5d01a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:07.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.192+0000 7fa5cde00640 1 --2- 172.21.15.16:0/2295963000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5d010b290 0x7fa5d019c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:07.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.193+0000 7fa5ce800640 1 -- 172.21.15.16:0/2295963000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5d010b290 msgr2=0x7fa5d019c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:07.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.193+0000 7fa5ce800640 1 --2- 172.21.15.16:0/2295963000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5d010b290 0x7fa5d019c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:07.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.193+0000 7fa5ce800640 1 -- 172.21.15.16:0/2295963000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5d010bbe0 msgr2=0x7fa5d01a1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:07.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.193+0000 7fa5ce800640 1 --2- 172.21.15.16:0/2295963000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5d010bbe0 0x7fa5d01a1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:07.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.193+0000 7fa5ce800640 1 -- 172.21.15.16:0/2295963000 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5d01a2210 con 0x7fa5d0105950 2024-08-31T06:15:07.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.193+0000 7fa5ce800640 1 --2- 172.21.15.16:0/2295963000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5d0105950 0x7fa5d019c2a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0005ab0 tx=0x7fa5c0003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:07.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.193+0000 7fa5c7400640 1 -- 172.21.15.16:0/2295963000 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c003f070 con 0x7fa5d0105950 2024-08-31T06:15:07.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.194+0000 7fa5c7400640 1 -- 172.21.15.16:0/2295963000 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5c0031b60 con 0x7fa5d0105950 2024-08-31T06:15:07.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.194+0000 7fa5c7400640 1 -- 172.21.15.16:0/2295963000 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0035030 con 0x7fa5d0105950 2024-08-31T06:15:07.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.194+0000 7fa5d4a00640 1 -- 172.21.15.16:0/2295963000 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5d01a24a0 con 0x7fa5d0105950 2024-08-31T06:15:07.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.194+0000 7fa5d4a00640 1 -- 172.21.15.16:0/2295963000 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5d01a8c60 con 0x7fa5d0105950 2024-08-31T06:15:07.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.197+0000 7fa5dc096740 1 Processor -- start 2024-08-31T06:15:07.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.197+0000 7fa5dc096740 1 -- start start 2024-08-31T06:15:07.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.198+0000 7fa5dc096740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d2675f520 0x560d26b05090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:07.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.198+0000 7fa5dc096740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d26736660 0x560d26b055d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:07.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.198+0000 7fa5dc096740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d2673e840 0x560d26b05b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:07.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.198+0000 7fa5dc096740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560d26a7ba90 con 0x560d2675f520 2024-08-31T06:15:07.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.198+0000 7fa5dc096740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560d26a6b020 con 0x560d26736660 2024-08-31T06:15:07.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.198+0000 7fa5dc096740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560d26a6efd0 con 0x560d2673e840 2024-08-31T06:15:07.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.198+0000 7fa5cf200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d2675f520 0x560d26b05090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:07.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.198+0000 7fa5cf200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d2675f520 0x560d26b05090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47234/0 (socket says 172.21.15.16:47234) 2024-08-31T06:15:07.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.198+0000 7fa5cf200640 1 -- 172.21.15.16:0/3138337022 learned_addr learned my addr 172.21.15.16:0/3138337022 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:07.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.198+0000 7fa5cde00640 1 --2- 172.21.15.16:0/3138337022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d2673e840 0x560d26b05b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:07.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.198+0000 7fa5cf200640 1 -- 172.21.15.16:0/3138337022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d2673e840 msgr2=0x560d26b05b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:07.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.199+0000 7fa5cf200640 1 --2- 172.21.15.16:0/3138337022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d2673e840 0x560d26b05b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:07.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.199+0000 7fa5cf200640 1 -- 172.21.15.16:0/3138337022 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d26736660 msgr2=0x560d26b055d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:15:07.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.199+0000 7fa5cf200640 1 --2- 172.21.15.16:0/3138337022 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d26736660 0x560d26b055d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:07.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.199+0000 7fa5cf200640 1 -- 172.21.15.16:0/3138337022 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d267cab00 con 0x560d2675f520 2024-08-31T06:15:07.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.199+0000 7fa5cde00640 1 --2- 172.21.15.16:0/3138337022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d2673e840 0x560d26b05b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:15:07.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.199+0000 7fa5cf200640 1 --2- 172.21.15.16:0/3138337022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d2675f520 0x560d26b05090 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8009fd0 tx=0x7fa5c80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d9dc238e9fe0191 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:07.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.199+0000 7fa5c7400640 1 -- 172.21.15.16:0/2295963000 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fa5c0042070 con 0x7fa5d0105950 2024-08-31T06:15:07.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.199+0000 7fa5bb400640 1 -- 172.21.15.16:0/3138337022 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c8004d20 con 0x560d2675f520 2024-08-31T06:15:07.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.199+0000 7fa5bb400640 1 -- 172.21.15.16:0/3138337022 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5c8004050 con 0x560d2675f520 2024-08-31T06:15:07.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.199+0000 7fa5bb400640 1 -- 172.21.15.16:0/3138337022 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c8005220 con 0x560d2675f520 2024-08-31T06:15:07.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.200+0000 7fa5dc096740 1 -- 172.21.15.16:0/3138337022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d2675f520 msgr2=0x560d26b05090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:07.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.200+0000 7fa5dc096740 1 --2- 172.21.15.16:0/3138337022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d2675f520 0x560d26b05090 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8009fd0 tx=0x7fa5c80118d0 comp rx=0 tx=0).stop 2024-08-31T06:15:07.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.200+0000 7fa5dc096740 1 -- 172.21.15.16:0/3138337022 shutdown_connections 2024-08-31T06:15:07.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.200+0000 7fa5dc096740 1 --2- 172.21.15.16:0/3138337022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d2673e840 0x560d26b05b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:07.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.200+0000 7fa5dc096740 1 --2- 172.21.15.16:0/3138337022 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d26736660 0x560d26b055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:07.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.200+0000 7fa5dc096740 1 --2- 172.21.15.16:0/3138337022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d2675f520 0x560d26b05090 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:07.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.200+0000 7fa5dc096740 1 -- 172.21.15.16:0/3138337022 >> 172.21.15.16:0/3138337022 conn(0x560d26753910 msgr2=0x560d26b02f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:07.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.201+0000 7fa5dc096740 1 -- 172.21.15.16:0/3138337022 shutdown_connections 2024-08-31T06:15:07.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.201+0000 7fa5dc096740 1 -- 172.21.15.16:0/3138337022 wait complete. 2024-08-31T06:15:07.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.201+0000 7fa5c7400640 1 --2- 172.21.15.16:0/2295963000 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa5b40b4c30 0x7fa5b40b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:07.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.201+0000 7fa5dc096740 1 -- start start 2024-08-31T06:15:07.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.201+0000 7fa5c7400640 1 -- 172.21.15.16:0/2295963000 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7fa5c0105b70 con 0x7fa5d0105950 2024-08-31T06:15:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.201+0000 7fa5dc096740 1 Processor -- start 2024-08-31T06:15:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.201+0000 7fa5cf200640 1 --2- 172.21.15.16:0/2295963000 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa5b40b4c30 0x7fa5b40b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.202+0000 7fa5cf200640 1 --2- 172.21.15.16:0/2295963000 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa5b40b4c30 0x7fa5b40b7140 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8004930 tx=0x7fa5c8000f40 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.202+0000 7fa5dc096740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d26736660 0x560d26b10490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.202+0000 7fa5dc096740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d2673e840 0x560d26b109d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.202+0000 7fa5dc096740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d2675f520 0x560d26b10f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.202+0000 7fa5dc096740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560d268cdbb0 con 0x560d2675f520 2024-08-31T06:15:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.202+0000 7fa5dc096740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560d269f1f90 con 0x560d2673e840 2024-08-31T06:15:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.202+0000 7fa5dc096740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560d26b13870 con 0x560d26736660 2024-08-31T06:15:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.203+0000 7fa5dc096740 10 client.0 _ll_register_callbacks cb 0x560d26a17710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:15:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.203+0000 7fa5ce800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d26736660 0x560d26b10490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.203+0000 7fa5ce800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d26736660 0x560d26b10490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:53948/0 (socket says 172.21.15.16:53948) 2024-08-31T06:15:07.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.203+0000 7fa5ce800640 1 -- 172.21.15.16:0/2429403474 learned_addr learned my addr 172.21.15.16:0/2429403474 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:07.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.203+0000 7fa5cf200640 1 --2- 172.21.15.16:0/2429403474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d2675f520 0x560d26b10f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:07.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.203+0000 7fa5cde00640 1 --2- 172.21.15.16:0/2429403474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d2673e840 0x560d26b109d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:07.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.203+0000 7fa5ce800640 1 -- 172.21.15.16:0/2429403474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d2673e840 msgr2=0x560d26b109d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:07.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.203+0000 7fa5ce800640 1 --2- 172.21.15.16:0/2429403474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d2673e840 0x560d26b109d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:07.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.203+0000 7fa5ce800640 1 -- 172.21.15.16:0/2429403474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d2675f520 msgr2=0x560d26b10f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:07.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.203+0000 7fa5ce800640 1 --2- 172.21.15.16:0/2429403474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d2675f520 0x560d26b10f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:07.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.203+0000 7fa5ce800640 1 -- 172.21.15.16:0/2429403474 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d26841960 con 0x560d26736660 2024-08-31T06:15:07.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.203+0000 7fa5cf200640 1 --2- 172.21.15.16:0/2429403474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d2675f520 0x560d26b10f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:15:07.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.204+0000 7fa5ce800640 1 --2- 172.21.15.16:0/2429403474 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d26736660 0x560d26b10490 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0031440 tx=0x7fa5c00cbcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1df3589eaeb15b56 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:07.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.204+0000 7fa5afe00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:15:07.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.204+0000 7fa5afe00640 1 -- 172.21.15.16:0/2429403474 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c00cc930 con 0x560d26736660 2024-08-31T06:15:07.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.204+0000 7fa5afe00640 1 -- 172.21.15.16:0/2429403474 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7fa5c003b460 con 0x560d26736660 2024-08-31T06:15:07.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.204+0000 7fa5dc096740 1 -- 172.21.15.16:0/2429403474 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560d26a09e30 con 0x560d26736660 2024-08-31T06:15:07.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.204+0000 7fa5afe00640 1 -- 172.21.15.16:0/2429403474 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa5c00cc230 con 0x560d26736660 2024-08-31T06:15:07.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.204+0000 7fa5dc096740 10 client.27397 fetch_fsmap learned FSMap version 39 2024-08-31T06:15:07.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.204+0000 7fa5dc096740 1 -- 172.21.15.16:0/2429403474 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560d26b1f2b0 con 0x560d26736660 2024-08-31T06:15:07.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.204+0000 7fa5afe00640 1 -- 172.21.15.16:0/2429403474 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560d26a09e30 con 0x560d26736660 2024-08-31T06:15:07.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.205+0000 7fa5afe00640 1 -- 172.21.15.16:0/2429403474 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fa5c00d50c0 con 0x560d26736660 2024-08-31T06:15:07.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.205+0000 7fa5dc096740 10 client.27397 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T06:15:07.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.205+0000 7fa5dc096740 10 client.27397 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:15:07.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.205+0000 7fa5dc096740 20 client.27397 populate_metadata read hostname 'smithi016' 2024-08-31T06:15:07.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.205+0000 7fa5dc096740 1 --2- 172.21.15.16:0/2429403474 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x560d2671dea0 0x560d26b226b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:07.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.205+0000 7fa5dc096740 4 client.27397 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:15:07.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.205+0000 7fa5dc096740 1 -- 172.21.15.16:0/2429403474 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d26a7d9e0 con 0x560d2671dea0 2024-08-31T06:15:07.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.205+0000 7fa5cde00640 1 --2- 172.21.15.16:0/2429403474 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x560d2671dea0 0x560d26b226b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:07.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.206+0000 7fa5cde00640 1 --2- 172.21.15.16:0/2429403474 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x560d2671dea0 0x560d26b226b0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a834b1459619b76 server_cookie=57d3437cfe9bd9d1 in_seq=0 out_seq=0 2024-08-31T06:15:07.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.206+0000 7fa5afe00640 10 client.27397 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:15:07.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.207+0000 7fa5afe00640 1 -- 172.21.15.16:0/2429403474 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560d26a7d9e0 con 0x560d2671dea0 2024-08-31T06:15:07.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.207+0000 7fa5afe00640 10 client.27397 handle_command_reply: tid=0 2024-08-31T06:15:07.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.207+0000 7fa5dc096740 1 client.27397 shutdown 2024-08-31T06:15:07.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.207+0000 7fa5dc096740 1 -- 172.21.15.16:0/2429403474 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d26736660 msgr2=0x560d26b10490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:07.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.207+0000 7fa5dc096740 1 --2- 172.21.15.16:0/2429403474 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d26736660 0x560d26b10490 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0031440 tx=0x7fa5c00cbcb0 comp rx=0 tx=0).stop 2024-08-31T06:15:07.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.207+0000 7fa5dc096740 1 -- 172.21.15.16:0/2429403474 shutdown_connections 2024-08-31T06:15:07.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.207+0000 7fa5dc096740 1 --2- 172.21.15.16:0/2429403474 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x560d2671dea0 0x560d26b226b0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:07.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.207+0000 7fa5dc096740 1 --2- 172.21.15.16:0/2429403474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d2675f520 0x560d26b10f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:07.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.207+0000 7fa5dc096740 1 --2- 172.21.15.16:0/2429403474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d2673e840 0x560d26b109d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:07.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.207+0000 7fa5afe00640 0 client.27397 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:15:07.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.207+0000 7fa5dc096740 1 --2- 172.21.15.16:0/2429403474 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d26736660 0x560d26b10490 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:07.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.207+0000 7fa5dc096740 1 -- 172.21.15.16:0/2429403474 >> 172.21.15.16:0/2429403474 conn(0x560d26753910 msgr2=0x560d26b08270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:07.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.207+0000 7fa5dc096740 1 -- 172.21.15.16:0/2429403474 shutdown_connections 2024-08-31T06:15:07.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.207+0000 7fa5dc096740 1 -- 172.21.15.16:0/2429403474 wait complete. 2024-08-31T06:15:07.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.207+0000 7fa5dc096740 20 client.27397 trim_cache size 0 max 16384 2024-08-31T06:15:07.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.236+0000 7fa5dc096740 1 Processor -- start 2024-08-31T06:15:07.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.236+0000 7fa5dc096740 1 -- start start 2024-08-31T06:15:07.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.236+0000 7fa5dc096740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d26736660 0x560d26b1cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:07.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.236+0000 7fa5dc096740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d2673e840 0x560d26b06780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:07.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.236+0000 7fa5dc096740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d2675f520 0x560d26b0d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:07.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.236+0000 7fa5dc096740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d01b3cd0 con 0x560d2675f520 2024-08-31T06:15:07.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.236+0000 7fa5dc096740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d01b1540 con 0x560d2673e840 2024-08-31T06:15:07.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.236+0000 7fa5dc096740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d01af3b0 con 0x560d26736660 2024-08-31T06:15:07.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.236+0000 7fa5cf200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d2675f520 0x560d26b0d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:07.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.237+0000 7fa5cf200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d2675f520 0x560d26b0d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47262/0 (socket says 172.21.15.16:47262) 2024-08-31T06:15:07.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.237+0000 7fa5cf200640 1 -- 172.21.15.16:0/3874500659 learned_addr learned my addr 172.21.15.16:0/3874500659 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:07.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.237+0000 7fa5cde00640 1 --2- 172.21.15.16:0/3874500659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d2673e840 0x560d26b06780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:07.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.237+0000 7fa5ce800640 1 --2- 172.21.15.16:0/3874500659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d26736660 0x560d26b1cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:07.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.237+0000 7fa5ce800640 1 -- 172.21.15.16:0/3874500659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d2673e840 msgr2=0x560d26b06780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:07.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.237+0000 7fa5ce800640 1 --2- 172.21.15.16:0/3874500659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d2673e840 0x560d26b06780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:07.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.237+0000 7fa5ce800640 1 -- 172.21.15.16:0/3874500659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d2675f520 msgr2=0x560d26b0d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:07.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.237+0000 7fa5ce800640 1 --2- 172.21.15.16:0/3874500659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d2675f520 0x560d26b0d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:07.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.237+0000 7fa5ce800640 1 -- 172.21.15.16:0/3874500659 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d26b07ef0 con 0x560d26736660 2024-08-31T06:15:07.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.237+0000 7fa5ce800640 1 --2- 172.21.15.16:0/3874500659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d26736660 0x560d26b1cac0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa5c00cbf30 tx=0x7fa5c003b800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6be0e595447f52f8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:07.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.237+0000 7fa5baa00640 1 -- 172.21.15.16:0/3874500659 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c00d0aa0 con 0x560d26736660 2024-08-31T06:15:07.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.238+0000 7fa5baa00640 1 -- 172.21.15.16:0/3874500659 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5c00d5550 con 0x560d26736660 2024-08-31T06:15:07.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.238+0000 7fa5baa00640 1 -- 172.21.15.16:0/3874500659 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c00cc490 con 0x560d26736660 2024-08-31T06:15:07.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.238+0000 7fa5dc096740 1 -- 172.21.15.16:0/3874500659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d26736660 msgr2=0x560d26b1cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:07.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.238+0000 7fa5dc096740 1 --2- 172.21.15.16:0/3874500659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d26736660 0x560d26b1cac0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa5c00cbf30 tx=0x7fa5c003b800 comp rx=0 tx=0).stop 2024-08-31T06:15:07.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.238+0000 7fa5dc096740 1 -- 172.21.15.16:0/3874500659 shutdown_connections 2024-08-31T06:15:07.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.238+0000 7fa5dc096740 1 --2- 172.21.15.16:0/3874500659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d2675f520 0x560d26b0d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:07.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.238+0000 7fa5dc096740 1 --2- 172.21.15.16:0/3874500659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d2673e840 0x560d26b06780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:07.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.238+0000 7fa5dc096740 1 --2- 172.21.15.16:0/3874500659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d26736660 0x560d26b1cac0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:07.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.238+0000 7fa5dc096740 1 -- 172.21.15.16:0/3874500659 >> 172.21.15.16:0/3874500659 conn(0x560d26753910 msgr2=0x560d26b06070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:07.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.239+0000 7fa5dc096740 1 -- 172.21.15.16:0/3874500659 shutdown_connections 2024-08-31T06:15:07.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.239+0000 7fa5dc096740 1 -- 172.21.15.16:0/3874500659 wait complete. 2024-08-31T06:15:07.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.239+0000 7fa5dc096740 1 -- start start 2024-08-31T06:15:07.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.239+0000 7fa5dc096740 1 Processor -- start 2024-08-31T06:15:07.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.240+0000 7fa5dc096740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d26736660 0x560d26b19500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:07.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.240+0000 7fa5dc096740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d2673e840 0x560d26b15a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:07.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.240+0000 7fa5dc096740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d2675f520 0x560d26b16110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:07.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.240+0000 7fa5dc096740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d01afa00 con 0x560d2675f520 2024-08-31T06:15:07.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.240+0000 7fa5dc096740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d01b0e70 con 0x560d2673e840 2024-08-31T06:15:07.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.240+0000 7fa5dc096740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d01b00d0 con 0x560d26736660 2024-08-31T06:15:07.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.240+0000 7fa5dc096740 10 client.0 _ll_register_callbacks cb 0x7fa5c00cc070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:15:07.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.240+0000 7fa5ce800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d26736660 0x560d26b19500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:07.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.240+0000 7fa5ce800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d26736660 0x560d26b19500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:53966/0 (socket says 172.21.15.16:53966) 2024-08-31T06:15:07.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.240+0000 7fa5ce800640 1 -- 172.21.15.16:0/3219778744 learned_addr learned my addr 172.21.15.16:0/3219778744 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:07.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.240+0000 7fa5cde00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d2673e840 0x560d26b15a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:07.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.240+0000 7fa5cf200640 1 --2- 172.21.15.16:0/3219778744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d2675f520 0x560d26b16110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:07.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.241+0000 7fa5ce800640 1 -- 172.21.15.16:0/3219778744 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d2673e840 msgr2=0x560d26b15a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:07.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.241+0000 7fa5ce800640 1 --2- 172.21.15.16:0/3219778744 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d2673e840 0x560d26b15a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:07.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.241+0000 7fa5ce800640 1 -- 172.21.15.16:0/3219778744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d2675f520 msgr2=0x560d26b16110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:07.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.241+0000 7fa5ce800640 1 --2- 172.21.15.16:0/3219778744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d2675f520 0x560d26b16110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:07.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.241+0000 7fa5ce800640 1 -- 172.21.15.16:0/3219778744 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d26b0c040 con 0x560d26736660 2024-08-31T06:15:07.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.241+0000 7fa5cf200640 1 --2- 172.21.15.16:0/3219778744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d2675f520 0x560d26b16110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:15:07.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.241+0000 7fa5ce800640 1 --2- 172.21.15.16:0/3219778744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d26736660 0x560d26b19500 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa5c00ef1c0 tx=0x7fa5c002a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ff7c2d64653e103 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:07.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.241+0000 7fa5afe00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:15:07.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.241+0000 7fa5afe00640 1 -- 172.21.15.16:0/3219778744 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c00cc930 con 0x560d26736660 2024-08-31T06:15:07.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.241+0000 7fa5afe00640 1 -- 172.21.15.16:0/3219778744 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8470+0+0 (secure 0 0 0) 0x7fa5c00cc490 con 0x560d26736660 2024-08-31T06:15:07.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.241+0000 7fa5dc096740 1 -- 172.21.15.16:0/3219778744 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5c00d5550 con 0x560d26736660 2024-08-31T06:15:07.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.242+0000 7fa5afe00640 1 -- 172.21.15.16:0/3219778744 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa5c00d0aa0 con 0x560d26736660 2024-08-31T06:15:07.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.242+0000 7fa5dc096740 10 client.27403 fetch_fsmap learned FSMap version 39 2024-08-31T06:15:07.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.242+0000 7fa5dc096740 1 -- 172.21.15.16:0/3219778744 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560d26b0c040 con 0x560d26736660 2024-08-31T06:15:07.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.242+0000 7fa5afe00640 1 -- 172.21.15.16:0/3219778744 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5c00d5550 con 0x560d26736660 2024-08-31T06:15:07.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.242+0000 7fa5afe00640 1 -- 172.21.15.16:0/3219778744 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fa5c00cb070 con 0x560d26736660 2024-08-31T06:15:07.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.242+0000 7fa5dc096740 10 client.27403 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T06:15:07.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.242+0000 7fa5dc096740 10 client.27403 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:15:07.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.242+0000 7fa5dc096740 20 client.27403 populate_metadata read hostname 'smithi016' 2024-08-31T06:15:07.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.242+0000 7fa5dc096740 1 --2- 172.21.15.16:0/3219778744 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x560d266f3250 0x560d26b903f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:07.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.242+0000 7fa5dc096740 4 client.27403 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "cf5ba332", "release": true, "await": true}] 2024-08-31T06:15:07.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.242+0000 7fa5dc096740 1 -- 172.21.15.16:0/3219778744 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "quiesce db", "set_id": "cf5ba332", "release": true, "await": true}) -- 0x560d26a79480 con 0x560d266f3250 2024-08-31T06:15:07.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.243+0000 7fa5cde00640 1 --2- 172.21.15.16:0/3219778744 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x560d266f3250 0x560d26b903f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:07.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.243+0000 7fa5cde00640 1 --2- 172.21.15.16:0/3219778744 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x560d266f3250 0x560d26b903f0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4be3d408055cb6c1 server_cookie=c59303e3d77fe569 in_seq=0 out_seq=0 2024-08-31T06:15:07.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.243+0000 7fa5afe00640 10 client.27403 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:15:07.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.406+0000 7fa5afe00640 1 -- 172.21.15.16:0/3219778744 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560d26a79480 con 0x560d266f3250 2024-08-31T06:15:07.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.406+0000 7fa5afe00640 10 client.27403 handle_command_reply: tid=0 2024-08-31T06:15:07.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.406+0000 7fa5dc096740 1 client.27403 shutdown 2024-08-31T06:15:07.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.406+0000 7fa5dc096740 1 -- 172.21.15.16:0/3219778744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d26736660 msgr2=0x560d26b19500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:07.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.406+0000 7fa5dc096740 1 --2- 172.21.15.16:0/3219778744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d26736660 0x560d26b19500 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa5c00ef1c0 tx=0x7fa5c002a410 comp rx=0 tx=0).stop 2024-08-31T06:15:07.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.407+0000 7fa5dc096740 1 -- 172.21.15.16:0/3219778744 shutdown_connections 2024-08-31T06:15:07.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.407+0000 7fa5dc096740 1 --2- 172.21.15.16:0/3219778744 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x560d266f3250 0x560d26b903f0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:07.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.407+0000 7fa5dc096740 1 --2- 172.21.15.16:0/3219778744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d2675f520 0x560d26b16110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:07.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.407+0000 7fa5dc096740 1 --2- 172.21.15.16:0/3219778744 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560d2673e840 0x560d26b15a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:07.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.407+0000 7fa5dc096740 1 --2- 172.21.15.16:0/3219778744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560d26736660 0x560d26b19500 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:07.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.407+0000 7fa5dc096740 1 -- 172.21.15.16:0/3219778744 >> 172.21.15.16:0/3219778744 conn(0x560d26753910 msgr2=0x560d26b05fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:07.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.407+0000 7fa5afe00640 0 client.27403 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:15:07.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.407+0000 7fa5dc096740 1 -- 172.21.15.16:0/3219778744 shutdown_connections 2024-08-31T06:15:07.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.407+0000 7fa5dc096740 1 -- 172.21.15.16:0/3219778744 wait complete. 2024-08-31T06:15:07.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.407+0000 7fa5dc096740 20 client.27403 trim_cache size 0 max 16384 2024-08-31T06:15:07.411 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:15:07.411 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 39, 2024-08-31T06:15:07.411 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24737, 2024-08-31T06:15:07.411 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 28, 2024-08-31T06:15:07.411 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-08-31T06:15:07.411 INFO:teuthology.orchestra.run.smithi016.stdout: "cf5ba332": { 2024-08-31T06:15:07.411 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 28, 2024-08-31T06:15:07.411 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-08-31T06:15:07.411 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:15:07.411 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-08-31T06:15:07.412 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:15:07.412 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:15:07.412 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-08-31T06:15:07.412 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 148.0, 2024-08-31T06:15:07.412 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-08-31T06:15:07.412 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-08-31T06:15:07.412 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-08-31T06:15:07.412 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:15:07.412 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-08-31T06:15:07.412 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:15:07.412 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:15:07.412 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:15:07.412 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:15:07.412 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:15:07.413 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:15:07.413 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:15:07.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.408+0000 7fa5d4a00640 1 -- 172.21.15.16:0/2295963000 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa5b40b4c30 msgr2=0x7fa5b40b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:07.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.408+0000 7fa5d4a00640 1 --2- 172.21.15.16:0/2295963000 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa5b40b4c30 0x7fa5b40b7140 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8004930 tx=0x7fa5c8000f40 comp rx=0 tx=0).stop 2024-08-31T06:15:07.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.408+0000 7fa5d4a00640 1 -- 172.21.15.16:0/2295963000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5d0105950 msgr2=0x7fa5d019c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:07.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.408+0000 7fa5d4a00640 1 --2- 172.21.15.16:0/2295963000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5d0105950 0x7fa5d019c2a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0005ab0 tx=0x7fa5c0003c10 comp rx=0 tx=0).stop 2024-08-31T06:15:07.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.408+0000 7fa5d4a00640 1 -- 172.21.15.16:0/2295963000 shutdown_connections 2024-08-31T06:15:07.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.408+0000 7fa5d4a00640 1 --2- 172.21.15.16:0/2295963000 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa5b40b4c30 0x7fa5b40b7140 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:07.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.408+0000 7fa5d4a00640 1 --2- 172.21.15.16:0/2295963000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5d010b290 0x7fa5d019c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:07.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.408+0000 7fa5d4a00640 1 --2- 172.21.15.16:0/2295963000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5d010bbe0 0x7fa5d01a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:07.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.408+0000 7fa5d4a00640 1 --2- 172.21.15.16:0/2295963000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5d0105950 0x7fa5d019c2a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:07.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.408+0000 7fa5d4a00640 1 -- 172.21.15.16:0/2295963000 >> 172.21.15.16:0/2295963000 conn(0x7fa5d0101420 msgr2=0x7fa5d010a950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:07.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.409+0000 7fa5d4a00640 1 -- 172.21.15.16:0/2295963000 shutdown_connections 2024-08-31T06:15:07.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:07.409+0000 7fa5d4a00640 1 -- 172.21.15.16:0/2295963000 wait complete. 2024-08-31T06:15:07.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:07 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:15:07.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:07 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:15:07.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:07 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:15:07.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:07 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:15:07.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:07 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:15:07.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:07 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:15:07.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:07 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:15:07.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:07 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:15:07.920 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T06:15:07.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:07 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:15:07.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:07 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:15:07.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:07 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:15:07.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:07 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:15:08.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:08 smithi155 ceph-mon[21742]: pgmap v605: 97 pgs: 97 active+clean; 3.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 721 KiB/s wr, 98 op/s 2024-08-31T06:15:08.873 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:08 smithi184 ceph-mon[22331]: pgmap v605: 97 pgs: 97 active+clean; 3.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 721 KiB/s wr, 98 op/s 2024-08-31T06:15:08.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:08 smithi016 ceph-mon[17194]: pgmap v605: 97 pgs: 97 active+clean; 3.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 721 KiB/s wr, 98 op/s 2024-08-31T06:15:08.913 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'cf5ba332', total seconds elapsed: 34.4 2024-08-31T06:15:08.996 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:15:09.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.839+0000 7fbae1a00640 1 -- 172.21.15.16:0/2570727510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbadc103910 msgr2=0x7fbadc103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:09.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.839+0000 7fbae1a00640 1 --2- 172.21.15.16:0/2570727510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbadc103910 0x7fbadc103d20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fbacc009ca0 tx=0x7fbacc02a8f0 comp rx=0 tx=0).stop 2024-08-31T06:15:09.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.839+0000 7fbae1a00640 1 -- 172.21.15.16:0/2570727510 shutdown_connections 2024-08-31T06:15:09.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.839+0000 7fbae1a00640 1 --2- 172.21.15.16:0/2570727510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbadc109ba0 0x7fbadc1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:09.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.839+0000 7fbae1a00640 1 --2- 172.21.15.16:0/2570727510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbadc109250 0x7fbadc109660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:09.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.839+0000 7fbae1a00640 1 --2- 172.21.15.16:0/2570727510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbadc103910 0x7fbadc103d20 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:09.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.839+0000 7fbae1a00640 1 -- 172.21.15.16:0/2570727510 >> 172.21.15.16:0/2570727510 conn(0x7fbadc0ff380 msgr2=0x7fbadc1017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:09.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.839+0000 7fbae1a00640 1 -- 172.21.15.16:0/2570727510 shutdown_connections 2024-08-31T06:15:09.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.840+0000 7fbae1a00640 1 -- 172.21.15.16:0/2570727510 wait complete. 2024-08-31T06:15:09.845 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:09 smithi016 ceph-mon[17194]: osdmap e86: 12 total, 12 up, 12 in 2024-08-31T06:15:09.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.843+0000 7fbae1a00640 1 Processor -- start 2024-08-31T06:15:09.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.843+0000 7fbae1a00640 1 -- start start 2024-08-31T06:15:09.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.844+0000 7fbae1a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbadc103910 0x7fbadc06f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:09.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.844+0000 7fbae1a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbadc109250 0x7fbadc06f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:09.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.845+0000 7fbae1a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbadc109ba0 0x7fbadc074980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:09.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.845+0000 7fbae1a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbadc06d440 con 0x7fbadc109250 2024-08-31T06:15:09.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.845+0000 7fbadb400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbadc109ba0 0x7fbadc074980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:09.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.845+0000 7fbadb400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbadc109ba0 0x7fbadc074980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:53990/0 (socket says 172.21.15.16:53990) 2024-08-31T06:15:09.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.845+0000 7fbadb400640 1 -- 172.21.15.16:0/2438122861 learned_addr learned my addr 172.21.15.16:0/2438122861 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:09.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.846+0000 7fbae1a00640 1 -- 172.21.15.16:0/2438122861 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbadc06d2c0 con 0x7fbadc103910 2024-08-31T06:15:09.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.846+0000 7fbae1a00640 1 -- 172.21.15.16:0/2438122861 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbadc06d5c0 con 0x7fbadc109ba0 2024-08-31T06:15:09.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.845+0000 7fbadaa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbadc103910 0x7fbadc06f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:09.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.847+0000 7fbada000640 1 --2- 172.21.15.16:0/2438122861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbadc109250 0x7fbadc06f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:09.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.848+0000 7fbada000640 1 -- 172.21.15.16:0/2438122861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbadc109ba0 msgr2=0x7fbadc074980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:09.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.848+0000 7fbada000640 1 --2- 172.21.15.16:0/2438122861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbadc109ba0 0x7fbadc074980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:09.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.848+0000 7fbada000640 1 -- 172.21.15.16:0/2438122861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbadc103910 msgr2=0x7fbadc06f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:09.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.848+0000 7fbada000640 1 --2- 172.21.15.16:0/2438122861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbadc103910 0x7fbadc06f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:09.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.849+0000 7fbada000640 1 -- 172.21.15.16:0/2438122861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbadc0750b0 con 0x7fbadc109250 2024-08-31T06:15:09.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.849+0000 7fbadb400640 1 --2- 172.21.15.16:0/2438122861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbadc109ba0 0x7fbadc074980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:15:09.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.850+0000 7fbadaa00640 1 --2- 172.21.15.16:0/2438122861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbadc103910 0x7fbadc06f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:15:09.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.850+0000 7fbada000640 1 --2- 172.21.15.16:0/2438122861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbadc109250 0x7fbadc06f6c0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fbadc06bae0 tx=0x7fbac400db30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:09.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.851+0000 7fbad3400640 1 -- 172.21.15.16:0/2438122861 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac4017070 con 0x7fbadc109250 2024-08-31T06:15:09.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.851+0000 7fbae1a00640 1 -- 172.21.15.16:0/2438122861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbadc075340 con 0x7fbadc109250 2024-08-31T06:15:09.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.851+0000 7fbad3400640 1 -- 172.21.15.16:0/2438122861 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbac4009820 con 0x7fbadc109250 2024-08-31T06:15:09.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.852+0000 7fbad3400640 1 -- 172.21.15.16:0/2438122861 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac4004b90 con 0x7fbadc109250 2024-08-31T06:15:09.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.854+0000 7fbae1a00640 1 -- 172.21.15.16:0/2438122861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbadc075af0 con 0x7fbadc109250 2024-08-31T06:15:09.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.861+0000 7fbad3400640 1 -- 172.21.15.16:0/2438122861 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fbac4004d30 con 0x7fbadc109250 2024-08-31T06:15:09.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.865+0000 7fbad3400640 1 --2- 172.21.15.16:0/2438122861 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbac00b4e70 0x7fbac00b7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:09.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.866+0000 7fbad3400640 1 -- 172.21.15.16:0/2438122861 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8514+0+0 (secure 0 0 0) 0x7fbac40e5040 con 0x7fbadc109250 2024-08-31T06:15:09.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.867+0000 7fbadaa00640 1 --2- 172.21.15.16:0/2438122861 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbac00b4e70 0x7fbac00b7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:09.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.867+0000 7fbae8656740 1 Processor -- start 2024-08-31T06:15:09.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.867+0000 7fbae8656740 1 -- start start 2024-08-31T06:15:09.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.868+0000 7fbae8656740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a91bc070 0x55e5a9561950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:09.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.868+0000 7fbae8656740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5a91931b0 0x55e5a9561e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:09.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.868+0000 7fbae8656740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5a919b390 0x55e5a95623d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:09.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.868+0000 7fbae8656740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e5a94d8770 con 0x55e5a91931b0 2024-08-31T06:15:09.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.868+0000 7fbae8656740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e5a94c7d00 con 0x55e5a91bc070 2024-08-31T06:15:09.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.868+0000 7fbae8656740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e5a94cbcb0 con 0x55e5a919b390 2024-08-31T06:15:09.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.869+0000 7fbadaa00640 1 --2- 172.21.15.16:0/2438122861 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbac00b4e70 0x7fbac00b7380 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbadc075a10 tx=0x7fbacc005d80 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:09.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.869+0000 7fbadaa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a91bc070 0x55e5a9561950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:09.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.869+0000 7fbadaa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a91bc070 0x55e5a9561950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:34652/0 (socket says 172.21.15.16:34652) 2024-08-31T06:15:09.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.869+0000 7fbadaa00640 1 -- 172.21.15.16:0/1278696578 learned_addr learned my addr 172.21.15.16:0/1278696578 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:09.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.869+0000 7fbadb400640 1 --2- 172.21.15.16:0/1278696578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5a919b390 0x55e5a95623d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:09.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.869+0000 7fbada000640 1 --2- 172.21.15.16:0/1278696578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5a91931b0 0x55e5a9561e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:09.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:09 smithi155 ceph-mon[21742]: osdmap e86: 12 total, 12 up, 12 in 2024-08-31T06:15:09.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.869+0000 7fbadaa00640 1 -- 172.21.15.16:0/1278696578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5a919b390 msgr2=0x55e5a95623d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:09.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.869+0000 7fbadaa00640 1 --2- 172.21.15.16:0/1278696578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5a919b390 0x55e5a95623d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:09.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.869+0000 7fbadaa00640 1 -- 172.21.15.16:0/1278696578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5a91931b0 msgr2=0x55e5a9561e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:09.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.869+0000 7fbadaa00640 1 --2- 172.21.15.16:0/1278696578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5a91931b0 0x55e5a9561e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:09.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.870+0000 7fbadaa00640 1 -- 172.21.15.16:0/1278696578 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5a9227650 con 0x55e5a91bc070 2024-08-31T06:15:09.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.870+0000 7fbadaa00640 1 --2- 172.21.15.16:0/1278696578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a91bc070 0x55e5a9561950 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fbacc006010 tx=0x7fbacc03f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e89f9e046c03bb8d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:09.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.871+0000 7fbacbe00640 1 -- 172.21.15.16:0/1278696578 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbacc004300 con 0x55e5a91bc070 2024-08-31T06:15:09.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.871+0000 7fbacbe00640 1 -- 172.21.15.16:0/1278696578 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbacc02d070 con 0x55e5a91bc070 2024-08-31T06:15:09.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.871+0000 7fbacbe00640 1 -- 172.21.15.16:0/1278696578 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbacc045480 con 0x55e5a91bc070 2024-08-31T06:15:09.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.872+0000 7fbae8656740 1 -- 172.21.15.16:0/1278696578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a91bc070 msgr2=0x55e5a9561950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:09.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.872+0000 7fbae8656740 1 --2- 172.21.15.16:0/1278696578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a91bc070 0x55e5a9561950 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fbacc006010 tx=0x7fbacc03f000 comp rx=0 tx=0).stop 2024-08-31T06:15:09.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.873+0000 7fbae8656740 1 -- 172.21.15.16:0/1278696578 shutdown_connections 2024-08-31T06:15:09.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.873+0000 7fbae8656740 1 --2- 172.21.15.16:0/1278696578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5a919b390 0x55e5a95623d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:09.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.873+0000 7fbae8656740 1 --2- 172.21.15.16:0/1278696578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5a91931b0 0x55e5a9561e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:09.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.873+0000 7fbae8656740 1 --2- 172.21.15.16:0/1278696578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a91bc070 0x55e5a9561950 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:09.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.873+0000 7fbae8656740 1 -- 172.21.15.16:0/1278696578 >> 172.21.15.16:0/1278696578 conn(0x55e5a91b0460 msgr2=0x55e5a955f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:09.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.874+0000 7fbae8656740 1 -- 172.21.15.16:0/1278696578 shutdown_connections 2024-08-31T06:15:09.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.874+0000 7fbae8656740 1 -- 172.21.15.16:0/1278696578 wait complete. 2024-08-31T06:15:09.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.875+0000 7fbae8656740 1 -- start start 2024-08-31T06:15:09.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.876+0000 7fbae8656740 1 Processor -- start 2024-08-31T06:15:09.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.878+0000 7fbae8656740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5a91931b0 0x55e5a956d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:09.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.878+0000 7fbae8656740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5a919b390 0x55e5a956da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:09.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.878+0000 7fbadaa00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5a91931b0 0x55e5a956d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:09.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.878+0000 7fbadaa00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5a91931b0 0x55e5a956d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:54008/0 (socket says 172.21.15.16:54008) 2024-08-31T06:15:09.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.879+0000 7fbae8656740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a91bc070 0x55e5a9579240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:09.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.879+0000 7fbae8656740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e5a932a700 con 0x55e5a919b390 2024-08-31T06:15:09.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.879+0000 7fbae8656740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e5a944ecb0 con 0x55e5a91bc070 2024-08-31T06:15:09.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.879+0000 7fbae8656740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e5a956ff50 con 0x55e5a91931b0 2024-08-31T06:15:09.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.880+0000 7fbadaa00640 1 -- 172.21.15.16:0/328244890 learned_addr learned my addr 172.21.15.16:0/328244890 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:09.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.880+0000 7fbae8656740 10 client.0 _ll_register_callbacks cb 0x55e5a9474430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:15:09.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.880+0000 7fbadb400640 1 --2- 172.21.15.16:0/328244890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a91bc070 0x55e5a9579240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:09.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.880+0000 7fbadb400640 1 -- 172.21.15.16:0/328244890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5a91931b0 msgr2=0x55e5a956d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:09.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.880+0000 7fbadb400640 1 --2- 172.21.15.16:0/328244890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5a91931b0 0x55e5a956d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:09.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.880+0000 7fbadb400640 1 -- 172.21.15.16:0/328244890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5a919b390 msgr2=0x55e5a956da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:09.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.880+0000 7fbadb400640 1 --2- 172.21.15.16:0/328244890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5a919b390 0x55e5a956da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:09.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.880+0000 7fbadb400640 1 -- 172.21.15.16:0/328244890 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5a948f870 con 0x55e5a91bc070 2024-08-31T06:15:09.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.881+0000 7fbadb400640 1 --2- 172.21.15.16:0/328244890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a91bc070 0x55e5a9579240 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fbad40129f0 tx=0x7fbad4012eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ebe0203fdf6ff1e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:09.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.881+0000 7fbac8c00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:15:09.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.881+0000 7fbac8c00640 1 -- 172.21.15.16:0/328244890 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad400f9c0 con 0x55e5a91bc070 2024-08-31T06:15:09.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.881+0000 7fbae8656740 1 -- 172.21.15.16:0/328244890 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbacc02d070 con 0x55e5a91bc070 2024-08-31T06:15:09.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.882+0000 7fbac8c00640 1 -- 172.21.15.16:0/328244890 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8514+0+0 (secure 0 0 0) 0x7fbad4018070 con 0x55e5a91bc070 2024-08-31T06:15:09.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.882+0000 7fbac8c00640 1 -- 172.21.15.16:0/328244890 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fbad400f9c0 con 0x55e5a91bc070 2024-08-31T06:15:09.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.882+0000 7fbae8656740 10 client.27428 fetch_fsmap learned FSMap version 39 2024-08-31T06:15:09.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.882+0000 7fbae8656740 1 -- 172.21.15.16:0/328244890 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e5a957bfc0 con 0x55e5a91bc070 2024-08-31T06:15:09.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.884+0000 7fbac8c00640 1 -- 172.21.15.16:0/328244890 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbacc02d070 con 0x55e5a91bc070 2024-08-31T06:15:09.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.884+0000 7fbac8c00640 1 -- 172.21.15.16:0/328244890 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fbad40047e0 con 0x55e5a91bc070 2024-08-31T06:15:09.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.885+0000 7fbae8656740 10 client.27428 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T06:15:09.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.885+0000 7fbae8656740 10 client.27428 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:15:09.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.885+0000 7fbae8656740 20 client.27428 populate_metadata read hostname 'smithi016' 2024-08-31T06:15:09.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.885+0000 7fbae8656740 1 --2- 172.21.15.16:0/328244890 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55e5a917a9f0 0x55e5a957f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:09.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.887+0000 7fbae8656740 4 client.27428 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:15:09.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.887+0000 7fbae8656740 1 -- 172.21.15.16:0/328244890 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5a94da6c0 con 0x55e5a917a9f0 2024-08-31T06:15:09.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.887+0000 7fbadaa00640 1 --2- 172.21.15.16:0/328244890 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55e5a917a9f0 0x55e5a957f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:09.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.890+0000 7fbac8c00640 10 client.27428 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:15:09.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.890+0000 7fbadaa00640 1 --2- 172.21.15.16:0/328244890 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55e5a917a9f0 0x55e5a957f4f0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8735b7439278582 server_cookie=5736872b1cef737c in_seq=0 out_seq=0 2024-08-31T06:15:09.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.891+0000 7fbac8c00640 1 -- 172.21.15.16:0/328244890 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e5a94da6c0 con 0x55e5a917a9f0 2024-08-31T06:15:09.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.892+0000 7fbac8c00640 10 client.27428 handle_command_reply: tid=0 2024-08-31T06:15:09.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.892+0000 7fbae8656740 1 client.27428 shutdown 2024-08-31T06:15:09.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.893+0000 7fbae8656740 1 -- 172.21.15.16:0/328244890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a91bc070 msgr2=0x55e5a9579240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:09.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.893+0000 7fbae8656740 1 --2- 172.21.15.16:0/328244890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a91bc070 0x55e5a9579240 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fbad40129f0 tx=0x7fbad4012eb0 comp rx=0 tx=0).stop 2024-08-31T06:15:09.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.894+0000 7fbae8656740 1 -- 172.21.15.16:0/328244890 shutdown_connections 2024-08-31T06:15:09.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.894+0000 7fbae8656740 1 --2- 172.21.15.16:0/328244890 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55e5a917a9f0 0x55e5a957f4f0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:09.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.894+0000 7fbac8c00640 0 client.27428 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:15:09.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.894+0000 7fbae8656740 1 --2- 172.21.15.16:0/328244890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a91bc070 0x55e5a9579240 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:09.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.894+0000 7fbae8656740 1 --2- 172.21.15.16:0/328244890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5a919b390 0x55e5a956da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:09.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.894+0000 7fbae8656740 1 --2- 172.21.15.16:0/328244890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5a91931b0 0x55e5a956d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:09.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.894+0000 7fbae8656740 1 -- 172.21.15.16:0/328244890 >> 172.21.15.16:0/328244890 conn(0x55e5a91b0460 msgr2=0x55e5a9564b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:09.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.894+0000 7fbae8656740 1 -- 172.21.15.16:0/328244890 shutdown_connections 2024-08-31T06:15:09.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.894+0000 7fbae8656740 1 -- 172.21.15.16:0/328244890 wait complete. 2024-08-31T06:15:09.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.895+0000 7fbae8656740 20 client.27428 trim_cache size 0 max 16384 2024-08-31T06:15:09.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.937+0000 7fbae8656740 1 Processor -- start 2024-08-31T06:15:09.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.937+0000 7fbae8656740 1 -- start start 2024-08-31T06:15:09.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.937+0000 7fbae8656740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a91931b0 0x55e5a9579140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:09.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.938+0000 7fbae8656740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5a919b390 0x55e5a9563090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:09.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.938+0000 7fbae8656740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5a91bc070 0x55e5a9566bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:09.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.938+0000 7fbae8656740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbadc1c9440 con 0x55e5a91bc070 2024-08-31T06:15:09.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.938+0000 7fbae8656740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbadc1c6cb0 con 0x55e5a91931b0 2024-08-31T06:15:09.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.938+0000 7fbae8656740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbadc1c4b20 con 0x55e5a919b390 2024-08-31T06:15:09.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.939+0000 7fbadaa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a91931b0 0x55e5a9579140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:09.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.939+0000 7fbada000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5a919b390 0x55e5a9563090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:09.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.939+0000 7fbadb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5a91bc070 0x55e5a9566bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:09.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.939+0000 7fbadaa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a91931b0 0x55e5a9579140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:34676/0 (socket says 172.21.15.16:34676) 2024-08-31T06:15:09.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.939+0000 7fbadaa00640 1 -- 172.21.15.16:0/2815930924 learned_addr learned my addr 172.21.15.16:0/2815930924 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:09.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.940+0000 7fbadaa00640 1 -- 172.21.15.16:0/2815930924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5a919b390 msgr2=0x55e5a9563090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:09.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.940+0000 7fbadaa00640 1 --2- 172.21.15.16:0/2815930924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5a919b390 0x55e5a9563090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:09.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.941+0000 7fbadaa00640 1 -- 172.21.15.16:0/2815930924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5a91bc070 msgr2=0x55e5a9566bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:09.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.941+0000 7fbadaa00640 1 --2- 172.21.15.16:0/2815930924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5a91bc070 0x55e5a9566bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:09.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.941+0000 7fbadaa00640 1 -- 172.21.15.16:0/2815930924 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5a956fd80 con 0x55e5a91931b0 2024-08-31T06:15:09.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.941+0000 7fbada000640 1 --2- 172.21.15.16:0/2815930924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5a919b390 0x55e5a9563090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:15:09.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.941+0000 7fbadb400640 1 --2- 172.21.15.16:0/2815930924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5a91bc070 0x55e5a9566bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:15:09.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.941+0000 7fbadaa00640 1 --2- 172.21.15.16:0/2815930924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a91931b0 0x55e5a9579140 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fbacc006010 tx=0x7fbacc0095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76737b9555a0fb76 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:09.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.942+0000 7fbacb400640 1 -- 172.21.15.16:0/2815930924 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbacc0307f0 con 0x55e5a91931b0 2024-08-31T06:15:09.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.942+0000 7fbacb400640 1 -- 172.21.15.16:0/2815930924 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbacc003e40 con 0x55e5a91931b0 2024-08-31T06:15:09.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.942+0000 7fbacb400640 1 -- 172.21.15.16:0/2815930924 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbacc045900 con 0x55e5a91931b0 2024-08-31T06:15:09.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.945+0000 7fbae8656740 1 -- 172.21.15.16:0/2815930924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a91931b0 msgr2=0x55e5a9579140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:09.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.945+0000 7fbae8656740 1 --2- 172.21.15.16:0/2815930924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a91931b0 0x55e5a9579140 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fbacc006010 tx=0x7fbacc0095c0 comp rx=0 tx=0).stop 2024-08-31T06:15:09.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.945+0000 7fbae8656740 1 -- 172.21.15.16:0/2815930924 shutdown_connections 2024-08-31T06:15:09.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.945+0000 7fbae8656740 1 --2- 172.21.15.16:0/2815930924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5a919b390 0x55e5a9563090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:09.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.945+0000 7fbae8656740 1 --2- 172.21.15.16:0/2815930924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5a91bc070 0x55e5a9566bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:09.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.945+0000 7fbae8656740 1 --2- 172.21.15.16:0/2815930924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a91931b0 0x55e5a9579140 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:09.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.945+0000 7fbae8656740 1 -- 172.21.15.16:0/2815930924 >> 172.21.15.16:0/2815930924 conn(0x55e5a91b0460 msgr2=0x55e5a9562980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:09.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.946+0000 7fbae8656740 1 -- 172.21.15.16:0/2815930924 shutdown_connections 2024-08-31T06:15:09.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.946+0000 7fbae8656740 1 -- 172.21.15.16:0/2815930924 wait complete. 2024-08-31T06:15:09.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.947+0000 7fbae8656740 1 -- start start 2024-08-31T06:15:09.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.948+0000 7fbae8656740 1 Processor -- start 2024-08-31T06:15:09.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.949+0000 7fbae8656740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a91931b0 0x55e5a95756a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:09.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.949+0000 7fbae8656740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5a919b390 0x55e5a9570160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:09.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.949+0000 7fbae8656740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5a91bc070 0x55e5a95706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:09.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.950+0000 7fbae8656740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbadc1c5170 con 0x55e5a919b390 2024-08-31T06:15:09.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.950+0000 7fbae8656740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbadc1c65e0 con 0x55e5a91931b0 2024-08-31T06:15:09.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.950+0000 7fbae8656740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbadc1c5840 con 0x55e5a91bc070 2024-08-31T06:15:09.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.950+0000 7fbae8656740 10 client.0 _ll_register_callbacks cb 0x7fbad4012600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:15:09.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.950+0000 7fbadaa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a91931b0 0x55e5a95756a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:09.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.951+0000 7fbadaa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a91931b0 0x55e5a95756a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:34680/0 (socket says 172.21.15.16:34680) 2024-08-31T06:15:09.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.951+0000 7fbadaa00640 1 -- 172.21.15.16:0/3953864562 learned_addr learned my addr 172.21.15.16:0/3953864562 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:09.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.952+0000 7fbadb400640 1 --2- 172.21.15.16:0/3953864562 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5a91bc070 0x55e5a95706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:09.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.952+0000 7fbadaa00640 1 -- 172.21.15.16:0/3953864562 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5a91bc070 msgr2=0x55e5a95706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:09.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.952+0000 7fbada000640 1 --2- 172.21.15.16:0/3953864562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5a919b390 0x55e5a9570160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:09.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.952+0000 7fbadaa00640 1 --2- 172.21.15.16:0/3953864562 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5a91bc070 0x55e5a95706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:09.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.952+0000 7fbadaa00640 1 -- 172.21.15.16:0/3953864562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5a919b390 msgr2=0x55e5a9570160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:09.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.952+0000 7fbadaa00640 1 --2- 172.21.15.16:0/3953864562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5a919b390 0x55e5a9570160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:09.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.952+0000 7fbadaa00640 1 -- 172.21.15.16:0/3953864562 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5a9570be0 con 0x55e5a91931b0 2024-08-31T06:15:09.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.952+0000 7fbadb400640 1 --2- 172.21.15.16:0/3953864562 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5a91bc070 0x55e5a95706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:15:09.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.953+0000 7fbada000640 1 --2- 172.21.15.16:0/3953864562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5a919b390 0x55e5a9570160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:15:09.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.953+0000 7fbadaa00640 1 --2- 172.21.15.16:0/3953864562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a91931b0 0x55e5a95756a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fbacc031040 tx=0x7fbacc030960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2c1b18bc9dabb05 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:09.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.953+0000 7fbac8c00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:15:09.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.953+0000 7fbac8c00640 1 -- 172.21.15.16:0/3953864562 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbacc004510 con 0x55e5a91931b0 2024-08-31T06:15:09.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.954+0000 7fbae8656740 1 -- 172.21.15.16:0/3953864562 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e5a94d6160 con 0x55e5a91931b0 2024-08-31T06:15:09.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.954+0000 7fbac8c00640 1 -- 172.21.15.16:0/3953864562 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8514+0+0 (secure 0 0 0) 0x7fbacc0046b0 con 0x55e5a91931b0 2024-08-31T06:15:09.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.954+0000 7fbac8c00640 1 -- 172.21.15.16:0/3953864562 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fbacc0456c0 con 0x55e5a91931b0 2024-08-31T06:15:09.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.954+0000 7fbae8656740 10 client.27434 fetch_fsmap learned FSMap version 39 2024-08-31T06:15:09.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.954+0000 7fbae8656740 1 -- 172.21.15.16:0/3953864562 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e5a95eab50 con 0x55e5a91931b0 2024-08-31T06:15:09.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.956+0000 7fbac8c00640 1 -- 172.21.15.16:0/3953864562 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e5a94d6160 con 0x55e5a91931b0 2024-08-31T06:15:09.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.956+0000 7fbac8c00640 1 -- 172.21.15.16:0/3953864562 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fbad4009ce0 con 0x55e5a91931b0 2024-08-31T06:15:09.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.956+0000 7fbae8656740 10 client.27434 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T06:15:09.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.956+0000 7fbae8656740 10 client.27434 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:15:09.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.956+0000 7fbae8656740 20 client.27434 populate_metadata read hostname 'smithi016' 2024-08-31T06:15:09.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.956+0000 7fbae8656740 1 --2- 172.21.15.16:0/3953864562 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55e5a914fda0 0x55e5a95ed010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:09.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.956+0000 7fbae8656740 4 client.27434 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T06:15:09.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.956+0000 7fbae8656740 1 -- 172.21.15.16:0/3953864562 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e5a94d38a0 con 0x55e5a914fda0 2024-08-31T06:15:09.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.956+0000 7fbada000640 1 --2- 172.21.15.16:0/3953864562 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55e5a914fda0 0x55e5a95ed010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:09.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.957+0000 7fbac8c00640 10 client.27434 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:15:09.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.957+0000 7fbada000640 1 --2- 172.21.15.16:0/3953864562 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55e5a914fda0 0x55e5a95ed010 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60548ee31d9121ac server_cookie=531bdc3ec7c3430f in_seq=0 out_seq=0 2024-08-31T06:15:09.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.958+0000 7fbac8c00640 1 -- 172.21.15.16:0/3953864562 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3410 (crc 0 0 0) 0x55e5a94d38a0 con 0x55e5a914fda0 2024-08-31T06:15:09.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.958+0000 7fbac8c00640 10 client.27434 handle_command_reply: tid=0 2024-08-31T06:15:09.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.958+0000 7fbae8656740 1 client.27434 shutdown 2024-08-31T06:15:09.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.958+0000 7fbae8656740 1 -- 172.21.15.16:0/3953864562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a91931b0 msgr2=0x55e5a95756a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:09.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.958+0000 7fbae8656740 1 --2- 172.21.15.16:0/3953864562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a91931b0 0x55e5a95756a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fbacc031040 tx=0x7fbacc030960 comp rx=0 tx=0).stop 2024-08-31T06:15:09.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.959+0000 7fbae8656740 1 -- 172.21.15.16:0/3953864562 shutdown_connections 2024-08-31T06:15:09.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.959+0000 7fbae8656740 1 --2- 172.21.15.16:0/3953864562 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55e5a914fda0 0x55e5a95ed010 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:09.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.959+0000 7fbae8656740 1 --2- 172.21.15.16:0/3953864562 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e5a91bc070 0x55e5a95706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:09.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.959+0000 7fbae8656740 1 --2- 172.21.15.16:0/3953864562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5a919b390 0x55e5a9570160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:09.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.959+0000 7fbae8656740 1 --2- 172.21.15.16:0/3953864562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5a91931b0 0x55e5a95756a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:09.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.959+0000 7fbae8656740 1 -- 172.21.15.16:0/3953864562 >> 172.21.15.16:0/3953864562 conn(0x55e5a91b0460 msgr2=0x55e5a9564a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:09.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.959+0000 7fbac8c00640 0 client.27434 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:15:09.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.959+0000 7fbae8656740 1 -- 172.21.15.16:0/3953864562 shutdown_connections 2024-08-31T06:15:09.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.959+0000 7fbae8656740 1 -- 172.21.15.16:0/3953864562 wait complete. 2024-08-31T06:15:09.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.959+0000 7fbae8656740 20 client.27434 trim_cache size 0 max 16384 2024-08-31T06:15:09.964 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:15:09.964 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T06:15:09.964 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-08-31T06:15:09.964 INFO:teuthology.orchestra.run.smithi016.stdout: "0fdcd762-533d-42be-9289-b71c8e3b81b1": { 2024-08-31T06:15:09.964 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:15:09.964 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "0fdcd762-533d-42be-9289-b71c8e3b81b1", 2024-08-31T06:15:09.964 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-08-31T06:15:09.964 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:15:09.964 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:15:09.964 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-08-31T06:15:09.964 INFO:teuthology.orchestra.run.smithi016.stdout: "a64d5301-8558-4e9c-ac9b-8d0ac7057e74": { 2024-08-31T06:15:09.964 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:14:20.338104+0000", 2024-08-31T06:15:09.965 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:15:09.965 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:15:09.965 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:15:09.965 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:15:09.965 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:15:09.965 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:15:09.965 INFO:teuthology.orchestra.run.smithi016.stdout: "e739a16e-8ace-43a8-a1ce-ac27d39ef8a3": { 2024-08-31T06:15:09.965 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:54.084646+0000", 2024-08-31T06:15:09.965 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:15:09.965 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:15:09.965 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:15:09.965 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:15:09.965 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:15:09.965 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:15:09.966 INFO:teuthology.orchestra.run.smithi016.stdout: "91b72a34-4ba9-4126-a414-8b862f8c5f78": { 2024-08-31T06:15:09.966 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:13:28.663237+0000", 2024-08-31T06:15:09.966 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:15:09.966 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:15:09.966 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:15:09.966 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:15:09.966 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:15:09.966 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:15:09.966 INFO:teuthology.orchestra.run.smithi016.stdout: "a10cce76-99ab-46b5-b3b2-07b82899b8f9": { 2024-08-31T06:15:09.966 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:11:43.967546+0000", 2024-08-31T06:15:09.966 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:15:09.966 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:15:09.966 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:15:09.966 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:15:09.966 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:15:09.966 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:15:09.967 INFO:teuthology.orchestra.run.smithi016.stdout: "c1fd6904-2e02-47fa-939a-47349b9fe36b": { 2024-08-31T06:15:09.967 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:06.678691+0000", 2024-08-31T06:15:09.967 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:15:09.967 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:15:09.967 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:15:09.967 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:15:09.967 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:15:09.967 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:15:09.967 INFO:teuthology.orchestra.run.smithi016.stdout: "78d6ef02-8551-45e8-ad60-9627eb573fc1": { 2024-08-31T06:15:09.967 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:09:13.547865+0000", 2024-08-31T06:15:09.967 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:15:09.967 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:15:09.967 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:15:09.967 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:15:09.967 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:15:09.967 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:15:09.968 INFO:teuthology.orchestra.run.smithi016.stdout: "c8455d74-2fe7-4d93-bcf2-19b419d16016": { 2024-08-31T06:15:09.968 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:08:24.035960+0000", 2024-08-31T06:15:09.968 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:15:09.968 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:15:09.968 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:15:09.968 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:15:09.968 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:15:09.968 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:15:09.968 INFO:teuthology.orchestra.run.smithi016.stdout: "0fdcd762-533d-42be-9289-b71c8e3b81b1": { 2024-08-31T06:15:09.968 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:15:06.169112+0000", 2024-08-31T06:15:09.968 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:15:09.968 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:15:09.968 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:15:09.968 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:15:09.968 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:15:09.969 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:15:09.969 INFO:teuthology.orchestra.run.smithi016.stdout: "88d7bdb8-cb1c-45e5-ad29-4cd8da6c6003": { 2024-08-31T06:15:09.969 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:07:39.648942+0000", 2024-08-31T06:15:09.969 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:15:09.969 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:15:09.969 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:15:09.969 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:15:09.969 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:15:09.969 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:15:09.969 INFO:teuthology.orchestra.run.smithi016.stdout: "6a03046e-cdf4-4a14-b9bd-b560b8968467": { 2024-08-31T06:15:09.969 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:12:38.002354+0000", 2024-08-31T06:15:09.969 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:15:09.969 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:15:09.969 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:15:09.969 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:15:09.970 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:15:09.970 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:15:09.970 INFO:teuthology.orchestra.run.smithi016.stdout: "c300b7e8-f3fd-482c-a909-9329bc61e570": { 2024-08-31T06:15:09.970 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:06:49.868046+0000", 2024-08-31T06:15:09.970 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:15:09.970 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:15:09.970 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:15:09.970 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:15:09.970 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:15:09.970 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:15:09.970 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:15:09.970 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:15:09.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.960+0000 7fbae1a00640 1 -- 172.21.15.16:0/2438122861 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbac00b4e70 msgr2=0x7fbac00b7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:09.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.960+0000 7fbae1a00640 1 --2- 172.21.15.16:0/2438122861 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbac00b4e70 0x7fbac00b7380 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbadc075a10 tx=0x7fbacc005d80 comp rx=0 tx=0).stop 2024-08-31T06:15:09.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.960+0000 7fbae1a00640 1 -- 172.21.15.16:0/2438122861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbadc109250 msgr2=0x7fbadc06f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:09.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.960+0000 7fbae1a00640 1 --2- 172.21.15.16:0/2438122861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbadc109250 0x7fbadc06f6c0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fbadc06bae0 tx=0x7fbac400db30 comp rx=0 tx=0).stop 2024-08-31T06:15:09.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.961+0000 7fbae1a00640 1 -- 172.21.15.16:0/2438122861 shutdown_connections 2024-08-31T06:15:09.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.961+0000 7fbae1a00640 1 --2- 172.21.15.16:0/2438122861 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbac00b4e70 0x7fbac00b7380 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:09.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.961+0000 7fbae1a00640 1 --2- 172.21.15.16:0/2438122861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbadc109ba0 0x7fbadc074980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:09.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.961+0000 7fbae1a00640 1 --2- 172.21.15.16:0/2438122861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbadc109250 0x7fbadc06f6c0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:09.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.961+0000 7fbae1a00640 1 --2- 172.21.15.16:0/2438122861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbadc103910 0x7fbadc06f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:09.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.961+0000 7fbae1a00640 1 -- 172.21.15.16:0/2438122861 >> 172.21.15.16:0/2438122861 conn(0x7fbadc0ff380 msgr2=0x7fbadc102850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:09.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.961+0000 7fbae1a00640 1 -- 172.21.15.16:0/2438122861 shutdown_connections 2024-08-31T06:15:09.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:09.961+0000 7fbae1a00640 1 -- 172.21.15.16:0/2438122861 wait complete. 2024-08-31T06:15:09.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:09 smithi184 ceph-mon[22331]: osdmap e86: 12 total, 12 up, 12 in 2024-08-31T06:15:10.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:10 smithi155 ceph-mon[21742]: pgmap v607: 97 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 87 active+clean; 3.4 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 5.1 MiB/s wr, 568 op/s 2024-08-31T06:15:10.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:10 smithi016 ceph-mon[17194]: pgmap v607: 97 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 87 active+clean; 3.4 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 5.1 MiB/s wr, 568 op/s 2024-08-31T06:15:10.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:10 smithi184 ceph-mon[22331]: pgmap v607: 97 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 87 active+clean; 3.4 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 5.1 MiB/s wr, 568 op/s 2024-08-31T06:15:11.012 INFO:tasks.cephfs.filesystem:scrub status for tag:0fdcd762-533d-42be-9289-b71c8e3b81b1 - {'path': '/', 'tag': '0fdcd762-533d-42be-9289-b71c8e3b81b1', 'options': 'recursive,force'} 2024-08-31T06:15:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.516+0000 7f919b200640 1 -- 172.21.15.16:0/523760735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f919c104e30 msgr2=0x7f919c10d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.516+0000 7f919b200640 1 --2- 172.21.15.16:0/523760735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f919c104e30 0x7f919c10d370 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9188009900 tx=0x7f918802a3f0 comp rx=0 tx=0).stop 2024-08-31T06:15:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.516+0000 7f919b200640 1 -- 172.21.15.16:0/523760735 shutdown_connections 2024-08-31T06:15:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.516+0000 7f919b200640 1 --2- 172.21.15.16:0/523760735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f919c10d940 0x7f919c10fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.516+0000 7f919b200640 1 --2- 172.21.15.16:0/523760735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f919c104e30 0x7f919c10d370 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.516+0000 7f919b200640 1 --2- 172.21.15.16:0/523760735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f919c1044e0 0x7f919c1048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.516+0000 7f919b200640 1 -- 172.21.15.16:0/523760735 >> 172.21.15.16:0/523760735 conn(0x7f919c0fdfd0 msgr2=0x7f919c100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.517+0000 7f919b200640 1 -- 172.21.15.16:0/523760735 shutdown_connections 2024-08-31T06:15:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.517+0000 7f919b200640 1 -- 172.21.15.16:0/523760735 wait complete. 2024-08-31T06:15:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.518+0000 7f919b200640 1 Processor -- start 2024-08-31T06:15:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.518+0000 7f919b200640 1 -- start start 2024-08-31T06:15:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.518+0000 7f919b200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f919c1044e0 0x7f919c1a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.518+0000 7f919b200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f919c104e30 0x7f919c1a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.518+0000 7f919b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f919c10d940 0x7f919c1ac320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.518+0000 7f919b200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f919c1135c0 con 0x7f919c10d940 2024-08-31T06:15:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.518+0000 7f919b200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f919c113440 con 0x7f919c104e30 2024-08-31T06:15:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.518+0000 7f919b200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f919c113740 con 0x7f919c1044e0 2024-08-31T06:15:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.519+0000 7f9199e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f919c1044e0 0x7f919c1a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.519+0000 7f9199e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f919c1044e0 0x7f919c1a12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:54048/0 (socket says 172.21.15.16:54048) 2024-08-31T06:15:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.519+0000 7f9199e00640 1 -- 172.21.15.16:0/787544957 learned_addr learned my addr 172.21.15.16:0/787544957 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.519+0000 7f9199400640 1 --2- 172.21.15.16:0/787544957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f919c104e30 0x7f919c1a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.519+0000 7f919a800640 1 --2- 172.21.15.16:0/787544957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f919c10d940 0x7f919c1ac320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.519+0000 7f9199e00640 1 -- 172.21.15.16:0/787544957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f919c104e30 msgr2=0x7f919c1a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.519+0000 7f9199e00640 1 --2- 172.21.15.16:0/787544957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f919c104e30 0x7f919c1a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.519+0000 7f9199e00640 1 -- 172.21.15.16:0/787544957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f919c10d940 msgr2=0x7f919c1ac320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.519+0000 7f9199e00640 1 --2- 172.21.15.16:0/787544957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f919c10d940 0x7f919c1ac320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.519+0000 7f9199e00640 1 -- 172.21.15.16:0/787544957 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f919c1a2060 con 0x7f919c1044e0 2024-08-31T06:15:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.519+0000 7f9199400640 1 --2- 172.21.15.16:0/787544957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f919c104e30 0x7f919c1a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:15:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.520+0000 7f9199e00640 1 --2- 172.21.15.16:0/787544957 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f919c1044e0 0x7f919c1a12a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f918c00c920 tx=0x7f918c00cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.520+0000 7f9192a00640 1 -- 172.21.15.16:0/787544957 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c007c70 con 0x7f919c1044e0 2024-08-31T06:15:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.520+0000 7f9192a00640 1 -- 172.21.15.16:0/787544957 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f918c007e10 con 0x7f919c1044e0 2024-08-31T06:15:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.520+0000 7f919b200640 1 -- 172.21.15.16:0/787544957 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f919c1ac920 con 0x7f919c1044e0 2024-08-31T06:15:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.520+0000 7f9192a00640 1 -- 172.21.15.16:0/787544957 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c018820 con 0x7f919c1044e0 2024-08-31T06:15:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.520+0000 7f919b200640 1 -- 172.21.15.16:0/787544957 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f919c1ace90 con 0x7f919c1044e0 2024-08-31T06:15:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.526+0000 7f91a7eaa740 1 Processor -- start 2024-08-31T06:15:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.526+0000 7f91a7eaa740 1 -- start start 2024-08-31T06:15:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.526+0000 7f91a7eaa740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572e54731b0 0x5572e5816b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.526+0000 7f91a7eaa740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572e544a2f0 0x5572e5817040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.526+0000 7f91a7eaa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e54524d0 0x5572e5817580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.526+0000 7f91a7eaa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5572e578bf70 con 0x5572e54524d0 2024-08-31T06:15:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.526+0000 7f91a7eaa740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5572e577b500 con 0x5572e544a2f0 2024-08-31T06:15:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.526+0000 7f91a7eaa740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5572e577f4b0 con 0x5572e54731b0 2024-08-31T06:15:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.527+0000 7f919a800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572e54731b0 0x5572e5816b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.527+0000 7f919a800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572e54731b0 0x5572e5816b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:54056/0 (socket says 172.21.15.16:54056) 2024-08-31T06:15:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.527+0000 7f919a800640 1 -- 172.21.15.16:0/3794134083 learned_addr learned my addr 172.21.15.16:0/3794134083 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.527+0000 7f9199400640 1 --2- 172.21.15.16:0/3794134083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e54524d0 0x5572e5817580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.527+0000 7f9199400640 1 -- 172.21.15.16:0/3794134083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572e54731b0 msgr2=0x5572e5816b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.527+0000 7f9199400640 1 --2- 172.21.15.16:0/3794134083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572e54731b0 0x5572e5816b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.527+0000 7f9199400640 1 -- 172.21.15.16:0/3794134083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572e544a2f0 msgr2=0x5572e5817040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:15:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.527+0000 7f9199400640 1 --2- 172.21.15.16:0/3794134083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572e544a2f0 0x5572e5817040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.527+0000 7f9199400640 1 -- 172.21.15.16:0/3794134083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572e5735c10 con 0x5572e54524d0 2024-08-31T06:15:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.527+0000 7f919a800640 1 --2- 172.21.15.16:0/3794134083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572e54731b0 0x5572e5816b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:15:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.528+0000 7f9199400640 1 --2- 172.21.15.16:0/3794134083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e54524d0 0x5572e5817580 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f9188009230 tx=0x7f9188005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77adf8d69ea17482 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.528+0000 7f9192a00640 1 -- 172.21.15.16:0/787544957 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f918c00f450 con 0x7f919c1044e0 2024-08-31T06:15:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.529+0000 7f9187400640 1 -- 172.21.15.16:0/3794134083 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9188035040 con 0x5572e54524d0 2024-08-31T06:15:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.529+0000 7f9187400640 1 -- 172.21.15.16:0/3794134083 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91880054f0 con 0x5572e54524d0 2024-08-31T06:15:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.529+0000 7f9187400640 1 -- 172.21.15.16:0/3794134083 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9188003b80 con 0x5572e54524d0 2024-08-31T06:15:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.530+0000 7f91a7eaa740 1 -- 172.21.15.16:0/3794134083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e54524d0 msgr2=0x5572e5817580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.530+0000 7f91a7eaa740 1 --2- 172.21.15.16:0/3794134083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e54524d0 0x5572e5817580 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f9188009230 tx=0x7f9188005040 comp rx=0 tx=0).stop 2024-08-31T06:15:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.530+0000 7f91a7eaa740 1 -- 172.21.15.16:0/3794134083 shutdown_connections 2024-08-31T06:15:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.530+0000 7f91a7eaa740 1 --2- 172.21.15.16:0/3794134083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e54524d0 0x5572e5817580 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.530+0000 7f91a7eaa740 1 --2- 172.21.15.16:0/3794134083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572e544a2f0 0x5572e5817040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.530+0000 7f91a7eaa740 1 --2- 172.21.15.16:0/3794134083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572e54731b0 0x5572e5816b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.530+0000 7f91a7eaa740 1 -- 172.21.15.16:0/3794134083 >> 172.21.15.16:0/3794134083 conn(0x5572e5467300 msgr2=0x5572e58149f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.530+0000 7f91a7eaa740 1 -- 172.21.15.16:0/3794134083 shutdown_connections 2024-08-31T06:15:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.530+0000 7f91a7eaa740 1 -- 172.21.15.16:0/3794134083 wait complete. 2024-08-31T06:15:11.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.531+0000 7f9192a00640 1 --2- 172.21.15.16:0/787544957 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f91800b4c30 0x7f91800b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:11.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.531+0000 7f91a7eaa740 1 -- start start 2024-08-31T06:15:11.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.531+0000 7f91a7eaa740 1 Processor -- start 2024-08-31T06:15:11.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.532+0000 7f9192a00640 1 -- 172.21.15.16:0/787544957 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8514+0+0 (secure 0 0 0) 0x7f918c0e5620 con 0x7f919c1044e0 2024-08-31T06:15:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.533+0000 7f919a800640 1 --2- 172.21.15.16:0/787544957 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f91800b4c30 0x7f91800b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.534+0000 7f91a7eaa740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572e544a2f0 0x5572e5822190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.534+0000 7f91a7eaa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e54524d0 0x5572e58226d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.534+0000 7f91a7eaa740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572e54731b0 0x5572e582e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.534+0000 7f91a7eaa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5572e56b4ab0 con 0x5572e54524d0 2024-08-31T06:15:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.534+0000 7f91a7eaa740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5572e5722320 con 0x5572e544a2f0 2024-08-31T06:15:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.534+0000 7f91a7eaa740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5572e5825220 con 0x5572e54731b0 2024-08-31T06:15:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.534+0000 7f91a7eaa740 10 client.0 _ll_register_callbacks cb 0x5572e57286a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:15:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.535+0000 7f919a800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572e54731b0 0x5572e582e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.535+0000 7f919a800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572e54731b0 0x5572e582e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:54066/0 (socket says 172.21.15.16:54066) 2024-08-31T06:15:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.535+0000 7f919a800640 1 -- 172.21.15.16:0/371433605 learned_addr learned my addr 172.21.15.16:0/371433605 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.535+0000 7f9199e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572e544a2f0 0x5572e5822190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.535+0000 7f919a800640 1 --2- 172.21.15.16:0/787544957 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f91800b4c30 0x7f91800b7140 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9194005c10 tx=0x7f91940097e0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.535+0000 7f919a800640 1 -- 172.21.15.16:0/371433605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572e544a2f0 msgr2=0x5572e5822190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.535+0000 7f919a800640 1 --2- 172.21.15.16:0/371433605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572e544a2f0 0x5572e5822190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.535+0000 7f919a800640 1 -- 172.21.15.16:0/371433605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e54524d0 msgr2=0x5572e58226d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:15:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.536+0000 7f9199400640 1 --2- 172.21.15.16:0/371433605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e54524d0 0x5572e58226d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.536+0000 7f919a800640 1 --2- 172.21.15.16:0/371433605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e54524d0 0x5572e58226d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.536+0000 7f919a800640 1 -- 172.21.15.16:0/371433605 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572e54de790 con 0x5572e54731b0 2024-08-31T06:15:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.536+0000 7f9199400640 1 --2- 172.21.15.16:0/371433605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e54524d0 0x5572e58226d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:15:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.536+0000 7f919a800640 1 --2- 172.21.15.16:0/371433605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572e54731b0 0x5572e582e540 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f91940109e0 tx=0x7f9194004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adeaed517d5c0da5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.536+0000 7f917be00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:15:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.537+0000 7f917be00640 1 -- 172.21.15.16:0/371433605 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919401ed40 con 0x5572e54731b0 2024-08-31T06:15:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.537+0000 7f917be00640 1 -- 172.21.15.16:0/371433605 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8514+0+0 (secure 0 0 0) 0x7f919401f420 con 0x5572e54731b0 2024-08-31T06:15:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.537+0000 7f91a7eaa740 1 -- 172.21.15.16:0/371433605 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91880054f0 con 0x5572e54731b0 2024-08-31T06:15:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.537+0000 7f917be00640 1 -- 172.21.15.16:0/371433605 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f919401f620 con 0x5572e54731b0 2024-08-31T06:15:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.537+0000 7f91a7eaa740 10 client.27430 fetch_fsmap learned FSMap version 39 2024-08-31T06:15:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.538+0000 7f91a7eaa740 1 -- 172.21.15.16:0/371433605 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5572e5557620 con 0x5572e54731b0 2024-08-31T06:15:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.538+0000 7f917be00640 1 -- 172.21.15.16:0/371433605 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91880054f0 con 0x5572e54731b0 2024-08-31T06:15:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.538+0000 7f917be00640 1 -- 172.21.15.16:0/371433605 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f919400f040 con 0x5572e54731b0 2024-08-31T06:15:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.538+0000 7f91a7eaa740 10 client.27430 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T06:15:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.538+0000 7f91a7eaa740 10 client.27430 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:15:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.538+0000 7f91a7eaa740 20 client.27430 populate_metadata read hostname 'smithi016' 2024-08-31T06:15:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.538+0000 7f91a7eaa740 1 --2- 172.21.15.16:0/371433605 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5572e5431d80 0x5572e5834330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.538+0000 7f91a7eaa740 4 client.27430 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:15:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.538+0000 7f91a7eaa740 1 -- 172.21.15.16:0/371433605 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572e578dec0 con 0x5572e5431d80 2024-08-31T06:15:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.539+0000 7f9199e00640 1 --2- 172.21.15.16:0/371433605 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5572e5431d80 0x5572e5834330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.539+0000 7f9199e00640 1 --2- 172.21.15.16:0/371433605 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5572e5431d80 0x5572e5834330 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cd19873759a8148 server_cookie=3bdc8657a67e9e24 in_seq=0 out_seq=0 2024-08-31T06:15:11.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.539+0000 7f917be00640 10 client.27430 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:15:11.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.540+0000 7f917be00640 1 -- 172.21.15.16:0/371433605 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5572e578dec0 con 0x5572e5431d80 2024-08-31T06:15:11.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.540+0000 7f917be00640 10 client.27430 handle_command_reply: tid=0 2024-08-31T06:15:11.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.540+0000 7f91a7eaa740 1 client.27430 shutdown 2024-08-31T06:15:11.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.541+0000 7f91a7eaa740 1 -- 172.21.15.16:0/371433605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572e54731b0 msgr2=0x5572e582e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:11.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.541+0000 7f91a7eaa740 1 --2- 172.21.15.16:0/371433605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572e54731b0 0x5572e582e540 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f91940109e0 tx=0x7f9194004a40 comp rx=0 tx=0).stop 2024-08-31T06:15:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.541+0000 7f91a7eaa740 1 -- 172.21.15.16:0/371433605 shutdown_connections 2024-08-31T06:15:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.541+0000 7f91a7eaa740 1 --2- 172.21.15.16:0/371433605 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5572e5431d80 0x5572e5834330 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.542+0000 7f91a7eaa740 1 --2- 172.21.15.16:0/371433605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572e54731b0 0x5572e582e540 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.542+0000 7f91a7eaa740 1 --2- 172.21.15.16:0/371433605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e54524d0 0x5572e58226d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.542+0000 7f91a7eaa740 1 --2- 172.21.15.16:0/371433605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572e544a2f0 0x5572e5822190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.542+0000 7f91a7eaa740 1 -- 172.21.15.16:0/371433605 >> 172.21.15.16:0/371433605 conn(0x5572e5467300 msgr2=0x5572e5817b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.542+0000 7f917be00640 0 client.27430 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:15:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.543+0000 7f91a7eaa740 1 -- 172.21.15.16:0/371433605 shutdown_connections 2024-08-31T06:15:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.543+0000 7f91a7eaa740 1 -- 172.21.15.16:0/371433605 wait complete. 2024-08-31T06:15:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.543+0000 7f91a7eaa740 20 client.27430 trim_cache size 0 max 16384 2024-08-31T06:15:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.570+0000 7f91a7eaa740 1 Processor -- start 2024-08-31T06:15:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.570+0000 7f91a7eaa740 1 -- start start 2024-08-31T06:15:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.571+0000 7f91a7eaa740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572e544a2f0 0x5572e5824070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.571+0000 7f91a7eaa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e54524d0 0x5572e582e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.571+0000 7f91a7eaa740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572e54731b0 0x5572e5821d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.571+0000 7f91a7eaa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f919c1b8150 con 0x5572e54524d0 2024-08-31T06:15:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.571+0000 7f91a7eaa740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f919c1b59c0 con 0x5572e54731b0 2024-08-31T06:15:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.571+0000 7f91a7eaa740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f919c1b3830 con 0x5572e544a2f0 2024-08-31T06:15:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.572+0000 7f9199e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572e544a2f0 0x5572e5824070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.572+0000 7f9199400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e54524d0 0x5572e582e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.572+0000 7f9199e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572e544a2f0 0x5572e5824070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:54080/0 (socket says 172.21.15.16:54080) 2024-08-31T06:15:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.572+0000 7f9199400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e54524d0 0x5572e582e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47358/0 (socket says 172.21.15.16:47358) 2024-08-31T06:15:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.572+0000 7f9199400640 1 -- 172.21.15.16:0/4251244120 learned_addr learned my addr 172.21.15.16:0/4251244120 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.572+0000 7f919a800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572e54731b0 0x5572e5821d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.572+0000 7f9199400640 1 -- 172.21.15.16:0/4251244120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572e544a2f0 msgr2=0x5572e5824070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.572+0000 7f9199400640 1 --2- 172.21.15.16:0/4251244120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572e544a2f0 0x5572e5824070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.572+0000 7f9199400640 1 -- 172.21.15.16:0/4251244120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572e54731b0 msgr2=0x5572e5821d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.572+0000 7f9199400640 1 --2- 172.21.15.16:0/4251244120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572e54731b0 0x5572e5821d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.572+0000 7f9199400640 1 -- 172.21.15.16:0/4251244120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572e5431fb0 con 0x5572e54524d0 2024-08-31T06:15:11.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.574+0000 7f9199400640 1 --2- 172.21.15.16:0/4251244120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e54524d0 0x5572e582e440 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f918802abd0 tx=0x7f9188004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edb892102a1c2e9a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:11.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.575+0000 7f9186a00640 1 -- 172.21.15.16:0/4251244120 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9188052070 con 0x5572e54524d0 2024-08-31T06:15:11.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.576+0000 7f9186a00640 1 -- 172.21.15.16:0/4251244120 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9188005630 con 0x5572e54524d0 2024-08-31T06:15:11.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.576+0000 7f9186a00640 1 -- 172.21.15.16:0/4251244120 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9188035040 con 0x5572e54524d0 2024-08-31T06:15:11.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.577+0000 7f91a7eaa740 1 -- 172.21.15.16:0/4251244120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e54524d0 msgr2=0x5572e582e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:11.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.577+0000 7f91a7eaa740 1 --2- 172.21.15.16:0/4251244120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e54524d0 0x5572e582e440 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f918802abd0 tx=0x7f9188004610 comp rx=0 tx=0).stop 2024-08-31T06:15:11.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.577+0000 7f91a7eaa740 1 -- 172.21.15.16:0/4251244120 shutdown_connections 2024-08-31T06:15:11.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.577+0000 7f91a7eaa740 1 --2- 172.21.15.16:0/4251244120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572e54731b0 0x5572e5821d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:11.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.577+0000 7f91a7eaa740 1 --2- 172.21.15.16:0/4251244120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e54524d0 0x5572e582e440 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:11.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.577+0000 7f91a7eaa740 1 --2- 172.21.15.16:0/4251244120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572e544a2f0 0x5572e5824070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:11.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.577+0000 7f91a7eaa740 1 -- 172.21.15.16:0/4251244120 >> 172.21.15.16:0/4251244120 conn(0x5572e5467300 msgr2=0x5572e5821680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:11.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.577+0000 7f91a7eaa740 1 -- 172.21.15.16:0/4251244120 shutdown_connections 2024-08-31T06:15:11.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.577+0000 7f91a7eaa740 1 -- 172.21.15.16:0/4251244120 wait complete. 2024-08-31T06:15:11.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.578+0000 7f91a7eaa740 1 -- start start 2024-08-31T06:15:11.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.578+0000 7f91a7eaa740 1 Processor -- start 2024-08-31T06:15:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.579+0000 7f91a7eaa740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572e544a2f0 0x5572e58285a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.580+0000 7f91a7eaa740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572e54524d0 0x5572e5828ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.580+0000 7f91a7eaa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e54731b0 0x5572e5829020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.580+0000 7f91a7eaa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f919c1b3e80 con 0x5572e54731b0 2024-08-31T06:15:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.580+0000 7f91a7eaa740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f919c1b52f0 con 0x5572e544a2f0 2024-08-31T06:15:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.580+0000 7f91a7eaa740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f919c1b4550 con 0x5572e54524d0 2024-08-31T06:15:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.580+0000 7f91a7eaa740 10 client.0 _ll_register_callbacks cb 0x7f9194010690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:15:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.580+0000 7f919a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e54731b0 0x5572e5829020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.580+0000 7f919a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e54731b0 0x5572e5829020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47364/0 (socket says 172.21.15.16:47364) 2024-08-31T06:15:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.580+0000 7f919a800640 1 -- 172.21.15.16:0/2051011467 learned_addr learned my addr 172.21.15.16:0/2051011467 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.580+0000 7f9199400640 1 --2- 172.21.15.16:0/2051011467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572e54524d0 0x5572e5828ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.580+0000 7f9199e00640 1 --2- 172.21.15.16:0/2051011467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572e544a2f0 0x5572e58285a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.580+0000 7f919a800640 1 -- 172.21.15.16:0/2051011467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572e54524d0 msgr2=0x5572e5828ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.580+0000 7f919a800640 1 --2- 172.21.15.16:0/2051011467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572e54524d0 0x5572e5828ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.580+0000 7f919a800640 1 -- 172.21.15.16:0/2051011467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572e544a2f0 msgr2=0x5572e58285a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.580+0000 7f919a800640 1 --2- 172.21.15.16:0/2051011467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572e544a2f0 0x5572e58285a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.580+0000 7f919a800640 1 -- 172.21.15.16:0/2051011467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572e589de60 con 0x5572e54731b0 2024-08-31T06:15:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.581+0000 7f9199e00640 1 --2- 172.21.15.16:0/2051011467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572e544a2f0 0x5572e58285a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:15:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.581+0000 7f919a800640 1 --2- 172.21.15.16:0/2051011467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e54731b0 0x5572e5829020 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f91940109e0 tx=0x7f9194000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d5b9389d8f9aa40 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.581+0000 7f917be00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:15:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.581+0000 7f917be00640 1 -- 172.21.15.16:0/2051011467 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919401cd50 con 0x5572e54731b0 2024-08-31T06:15:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.581+0000 7f917be00640 1 -- 172.21.15.16:0/2051011467 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8514+0+0 (secure 0 0 0) 0x7f919401f7c0 con 0x5572e54731b0 2024-08-31T06:15:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.581+0000 7f91a7eaa740 1 -- 172.21.15.16:0/2051011467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5572e5789960 con 0x5572e54731b0 2024-08-31T06:15:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.582+0000 7f917be00640 1 -- 172.21.15.16:0/2051011467 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f919401f9c0 con 0x5572e54731b0 2024-08-31T06:15:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.582+0000 7f91a7eaa740 10 client.17217 fetch_fsmap learned FSMap version 39 2024-08-31T06:15:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.582+0000 7f91a7eaa740 1 -- 172.21.15.16:0/2051011467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5572e589f0f0 con 0x5572e54731b0 2024-08-31T06:15:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.582+0000 7f917be00640 1 -- 172.21.15.16:0/2051011467 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5572e5789960 con 0x5572e54731b0 2024-08-31T06:15:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.582+0000 7f917be00640 1 -- 172.21.15.16:0/2051011467 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f91940409b0 con 0x5572e54731b0 2024-08-31T06:15:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.582+0000 7f91a7eaa740 10 client.17217 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T06:15:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.582+0000 7f91a7eaa740 10 client.17217 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:15:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.582+0000 7f91a7eaa740 20 client.17217 populate_metadata read hostname 'smithi016' 2024-08-31T06:15:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.583+0000 7f91a7eaa740 1 --2- 172.21.15.16:0/2051011467 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5572e54071c0 0x5572e58a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.583+0000 7f91a7eaa740 4 client.17217 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:15:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.583+0000 7f91a7eaa740 1 -- 172.21.15.16:0/2051011467 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5572e57870a0 con 0x5572e54071c0 2024-08-31T06:15:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.584+0000 7f9199e00640 1 --2- 172.21.15.16:0/2051011467 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5572e54071c0 0x5572e58a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.585+0000 7f9199e00640 1 --2- 172.21.15.16:0/2051011467 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5572e54071c0 0x5572e58a18f0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd262140300c6d5b server_cookie=c4f33fc8e7f763f7 in_seq=0 out_seq=0 2024-08-31T06:15:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.585+0000 7f917be00640 10 client.17217 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:15:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.602+0000 7f917be00640 1 -- 172.21.15.16:0/2051011467 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5572e57870a0 con 0x5572e54071c0 2024-08-31T06:15:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.602+0000 7f917be00640 10 client.17217 handle_command_reply: tid=0 2024-08-31T06:15:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.602+0000 7f91a7eaa740 1 client.17217 shutdown 2024-08-31T06:15:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.603+0000 7f91a7eaa740 1 -- 172.21.15.16:0/2051011467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e54731b0 msgr2=0x5572e5829020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.603+0000 7f91a7eaa740 1 --2- 172.21.15.16:0/2051011467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e54731b0 0x5572e5829020 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f91940109e0 tx=0x7f9194000f40 comp rx=0 tx=0).stop 2024-08-31T06:15:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.603+0000 7f91a7eaa740 1 -- 172.21.15.16:0/2051011467 shutdown_connections 2024-08-31T06:15:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.603+0000 7f91a7eaa740 1 --2- 172.21.15.16:0/2051011467 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5572e54071c0 0x5572e58a18f0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.603+0000 7f91a7eaa740 1 --2- 172.21.15.16:0/2051011467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572e54524d0 0x5572e5828ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.603+0000 7f91a7eaa740 1 --2- 172.21.15.16:0/2051011467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5572e54731b0 0x5572e5829020 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.603+0000 7f917be00640 0 client.17217 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:15:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.603+0000 7f91a7eaa740 1 --2- 172.21.15.16:0/2051011467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572e544a2f0 0x5572e58285a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.603+0000 7f91a7eaa740 1 -- 172.21.15.16:0/2051011467 >> 172.21.15.16:0/2051011467 conn(0x5572e5467300 msgr2=0x5572e5821680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.604+0000 7f91a7eaa740 1 -- 172.21.15.16:0/2051011467 shutdown_connections 2024-08-31T06:15:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.604+0000 7f91a7eaa740 1 -- 172.21.15.16:0/2051011467 wait complete. 2024-08-31T06:15:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.604+0000 7f91a7eaa740 20 client.17217 trim_cache size 0 max 16384 2024-08-31T06:15:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:15:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:15:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:15:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:15:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.605+0000 7f919b200640 1 -- 172.21.15.16:0/787544957 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f91800b4c30 msgr2=0x7f91800b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.605+0000 7f919b200640 1 --2- 172.21.15.16:0/787544957 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f91800b4c30 0x7f91800b7140 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9194005c10 tx=0x7f91940097e0 comp rx=0 tx=0).stop 2024-08-31T06:15:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.605+0000 7f919b200640 1 -- 172.21.15.16:0/787544957 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f919c1044e0 msgr2=0x7f919c1a12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.605+0000 7f919b200640 1 --2- 172.21.15.16:0/787544957 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f919c1044e0 0x7f919c1a12a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f918c00c920 tx=0x7f918c00cde0 comp rx=0 tx=0).stop 2024-08-31T06:15:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.605+0000 7f919b200640 1 -- 172.21.15.16:0/787544957 shutdown_connections 2024-08-31T06:15:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.605+0000 7f919b200640 1 --2- 172.21.15.16:0/787544957 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f91800b4c30 0x7f91800b7140 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.605+0000 7f919b200640 1 --2- 172.21.15.16:0/787544957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f919c10d940 0x7f919c1ac320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.605+0000 7f919b200640 1 --2- 172.21.15.16:0/787544957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f919c104e30 0x7f919c1a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.605+0000 7f919b200640 1 --2- 172.21.15.16:0/787544957 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f919c1044e0 0x7f919c1a12a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.605+0000 7f919b200640 1 -- 172.21.15.16:0/787544957 >> 172.21.15.16:0/787544957 conn(0x7f919c0fdfd0 msgr2=0x7f919c1020f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:11.610 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.606+0000 7f919b200640 1 -- 172.21.15.16:0/787544957 shutdown_connections 2024-08-31T06:15:11.610 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:11.606+0000 7f919b200640 1 -- 172.21.15.16:0/787544957 wait complete. 2024-08-31T06:15:11.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 465.0 s, 1171.3 tps, lat 541.489 ms stddev 3581.689 2024-08-31T06:15:12.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:12 smithi184 ceph-mon[22331]: pgmap v608: 97 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 87 active+clean; 3.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 7.5 MiB/s wr, 853 op/s 2024-08-31T06:15:13.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:12 smithi155 ceph-mon[21742]: pgmap v608: 97 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 87 active+clean; 3.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 7.5 MiB/s wr, 853 op/s 2024-08-31T06:15:13.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:12 smithi016 ceph-mon[17194]: pgmap v608: 97 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 87 active+clean; 3.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 7.5 MiB/s wr, 853 op/s 2024-08-31T06:15:14.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:14 smithi184 ceph-mon[22331]: pgmap v609: 97 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 87 active+clean; 3.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 10 MiB/s wr, 1.16k op/s 2024-08-31T06:15:15.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:14 smithi155 ceph-mon[21742]: pgmap v609: 97 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 87 active+clean; 3.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 10 MiB/s wr, 1.16k op/s 2024-08-31T06:15:15.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:14 smithi016 ceph-mon[17194]: pgmap v609: 97 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 87 active+clean; 3.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 10 MiB/s wr, 1.16k op/s 2024-08-31T06:15:16.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 470.0 s, 2090.4 tps, lat 47.774 ms stddev 61.539 2024-08-31T06:15:16.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:16 smithi184 ceph-mon[22331]: pgmap v610: 97 pgs: 97 active+clean; 3.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 12 MiB/s wr, 1.35k op/s 2024-08-31T06:15:17.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:16 smithi155 ceph-mon[21742]: pgmap v610: 97 pgs: 97 active+clean; 3.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 12 MiB/s wr, 1.35k op/s 2024-08-31T06:15:17.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:16 smithi016 ceph-mon[17194]: pgmap v610: 97 pgs: 97 active+clean; 3.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 12 MiB/s wr, 1.35k op/s 2024-08-31T06:15:19.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:18 smithi155 ceph-mon[21742]: pgmap v611: 97 pgs: 97 active+clean; 3.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 12 MiB/s wr, 1.35k op/s 2024-08-31T06:15:19.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:18 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:15:19.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:18 smithi016 ceph-mon[17194]: pgmap v611: 97 pgs: 97 active+clean; 3.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 12 MiB/s wr, 1.35k op/s 2024-08-31T06:15:19.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:18 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:15:19.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:18 smithi184 ceph-mon[22331]: pgmap v611: 97 pgs: 97 active+clean; 3.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 12 MiB/s wr, 1.35k op/s 2024-08-31T06:15:19.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:18 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:15:20.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:19 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:15:20.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:19 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:15:20.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:19 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:15:20.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:19 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:15:20.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:19 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:15:20.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:19 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:15:21.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:20 smithi155 ceph-mon[21742]: pgmap v612: 97 pgs: 97 active+clean; 3.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 12 MiB/s wr, 1.25k op/s 2024-08-31T06:15:21.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:20 smithi155 ceph-mon[21742]: osdmap e87: 12 total, 12 up, 12 in 2024-08-31T06:15:21.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:20 smithi016 ceph-mon[17194]: pgmap v612: 97 pgs: 97 active+clean; 3.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 12 MiB/s wr, 1.25k op/s 2024-08-31T06:15:21.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:20 smithi016 ceph-mon[17194]: osdmap e87: 12 total, 12 up, 12 in 2024-08-31T06:15:21.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:20 smithi184 ceph-mon[22331]: pgmap v612: 97 pgs: 97 active+clean; 3.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 12 MiB/s wr, 1.25k op/s 2024-08-31T06:15:21.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:20 smithi184 ceph-mon[22331]: osdmap e87: 12 total, 12 up, 12 in 2024-08-31T06:15:21.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 475.0 s, 2477.0 tps, lat 40.227 ms stddev 44.522 2024-08-31T06:15:23.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:22 smithi155 ceph-mon[21742]: pgmap v614: 97 pgs: 97 active+clean; 3.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.8 MiB/s wr, 886 op/s 2024-08-31T06:15:23.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:22 smithi016 ceph-mon[17194]: pgmap v614: 97 pgs: 97 active+clean; 3.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.8 MiB/s wr, 886 op/s 2024-08-31T06:15:23.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:22 smithi184 ceph-mon[22331]: pgmap v614: 97 pgs: 97 active+clean; 3.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.8 MiB/s wr, 886 op/s 2024-08-31T06:15:24.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:23 smithi155 ceph-mon[21742]: pgmap v615: 97 pgs: 97 active+clean; 3.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.9 MiB/s wr, 911 op/s 2024-08-31T06:15:24.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:23 smithi016 ceph-mon[17194]: pgmap v615: 97 pgs: 97 active+clean; 3.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.9 MiB/s wr, 911 op/s 2024-08-31T06:15:24.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:23 smithi184 ceph-mon[22331]: pgmap v615: 97 pgs: 97 active+clean; 3.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.9 MiB/s wr, 911 op/s 2024-08-31T06:15:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.893+0000 7f3713c00640 1 -- 172.21.15.16:0/764839443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f370c105a90 msgr2=0x7f370c105ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.893+0000 7f3713c00640 1 --2- 172.21.15.16:0/764839443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f370c105a90 0x7f370c105ea0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3700009900 tx=0x7f370002a400 comp rx=0 tx=0).stop 2024-08-31T06:15:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.893+0000 7f3713c00640 1 -- 172.21.15.16:0/764839443 shutdown_connections 2024-08-31T06:15:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.893+0000 7f3713c00640 1 --2- 172.21.15.16:0/764839443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f370c10b3c0 0x7f370c10b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.893+0000 7f3713c00640 1 --2- 172.21.15.16:0/764839443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f370c10bd10 0x7f370c112540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.893+0000 7f3713c00640 1 --2- 172.21.15.16:0/764839443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f370c105a90 0x7f370c105ea0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.893+0000 7f3713c00640 1 -- 172.21.15.16:0/764839443 >> 172.21.15.16:0/764839443 conn(0x7f370c101490 msgr2=0x7f370c1038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.894+0000 7f3713c00640 1 -- 172.21.15.16:0/764839443 shutdown_connections 2024-08-31T06:15:24.898 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.894+0000 7f3713c00640 1 -- 172.21.15.16:0/764839443 wait complete. 2024-08-31T06:15:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.897+0000 7f3713c00640 1 Processor -- start 2024-08-31T06:15:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.897+0000 7f3713c00640 1 -- start start 2024-08-31T06:15:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.898+0000 7f3713c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f370c105a90 0x7f370c10fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.898+0000 7f3713c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f370c10b3c0 0x7f370c10ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.898+0000 7f3711200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f370c105a90 0x7f370c10fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.899+0000 7f3711200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f370c105a90 0x7f370c10fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60016/0 (socket says 172.21.15.16:60016) 2024-08-31T06:15:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.899+0000 7f370be00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f370c10b3c0 0x7f370c10ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.899+0000 7f370be00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f370c10b3c0 0x7f370c10ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:52880/0 (socket says 172.21.15.16:52880) 2024-08-31T06:15:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.899+0000 7f3711200640 1 -- 172.21.15.16:0/2575759609 learned_addr learned my addr 172.21.15.16:0/2575759609 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.899+0000 7f3713c00640 1 --2- 172.21.15.16:0/2575759609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f370c10bd10 0x7f370c1104a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.899+0000 7f3713c00640 1 -- 172.21.15.16:0/2575759609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f370c115a20 con 0x7f370c105a90 2024-08-31T06:15:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.899+0000 7f3713c00640 1 -- 172.21.15.16:0/2575759609 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f370c1158a0 con 0x7f370c10bd10 2024-08-31T06:15:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.900+0000 7f3713c00640 1 -- 172.21.15.16:0/2575759609 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f370c115ba0 con 0x7f370c10b3c0 2024-08-31T06:15:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.900+0000 7f3711c00640 1 --2- 172.21.15.16:0/2575759609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f370c10bd10 0x7f370c1104a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.900+0000 7f3711200640 1 -- 172.21.15.16:0/2575759609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f370c10b3c0 msgr2=0x7f370c10ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.900+0000 7f3711200640 1 --2- 172.21.15.16:0/2575759609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f370c10b3c0 0x7f370c10ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.900+0000 7f3711200640 1 -- 172.21.15.16:0/2575759609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f370c10bd10 msgr2=0x7f370c1104a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.900+0000 7f3711200640 1 --2- 172.21.15.16:0/2575759609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f370c10bd10 0x7f370c1104a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.900+0000 7f3711200640 1 -- 172.21.15.16:0/2575759609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f370c1a20e0 con 0x7f370c105a90 2024-08-31T06:15:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.900+0000 7f3711c00640 1 --2- 172.21.15.16:0/2575759609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f370c10bd10 0x7f370c1104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:15:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.901+0000 7f3711200640 1 --2- 172.21.15.16:0/2575759609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f370c105a90 0x7f370c10fa20 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f3700005ab0 tx=0x7f3700003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:24.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.902+0000 7f3709600640 1 -- 172.21.15.16:0/2575759609 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370003f070 con 0x7f370c105a90 2024-08-31T06:15:24.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.902+0000 7f3713c00640 1 -- 172.21.15.16:0/2575759609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f370c1a2370 con 0x7f370c105a90 2024-08-31T06:15:24.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.902+0000 7f3709600640 1 -- 172.21.15.16:0/2575759609 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3700031b50 con 0x7f370c105a90 2024-08-31T06:15:24.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.902+0000 7f3709600640 1 -- 172.21.15.16:0/2575759609 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3700035030 con 0x7f370c105a90 2024-08-31T06:15:24.905 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.902+0000 7f3713c00640 1 -- 172.21.15.16:0/2575759609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f370c1a28b0 con 0x7f370c105a90 2024-08-31T06:15:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.908+0000 7f3709600640 1 -- 172.21.15.16:0/2575759609 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f3700042070 con 0x7f370c105a90 2024-08-31T06:15:24.911 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.909+0000 7f371ac86740 1 Processor -- start 2024-08-31T06:15:24.912 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.909+0000 7f371ac86740 1 -- start start 2024-08-31T06:15:24.912 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.910+0000 7f371ac86740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634da9791b0 0x5634dad1c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:24.912 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.910+0000 7f371ac86740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634da9502f0 0x5634dad1ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:24.912 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.910+0000 7f371ac86740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634da9584d0 0x5634dad1d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.910+0000 7f371ac86740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5634dac91f70 con 0x5634da9502f0 2024-08-31T06:15:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.910+0000 7f371ac86740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5634dac81500 con 0x5634da9791b0 2024-08-31T06:15:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.910+0000 7f371ac86740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5634dac854b0 con 0x5634da9584d0 2024-08-31T06:15:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.910+0000 7f3709600640 1 --2- 172.21.15.16:0/2575759609 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f36f40b4d50 0x7f36f40b7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:24.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.911+0000 7f3711c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634da9791b0 0x5634dad1c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.911+0000 7f3711200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634da9502f0 0x5634dad1ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.911+0000 7f3711c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634da9791b0 0x5634dad1c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:43908/0 (socket says 172.21.15.16:43908) 2024-08-31T06:15:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.911+0000 7f3711c00640 1 -- 172.21.15.16:0/3386983863 learned_addr learned my addr 172.21.15.16:0/3386983863 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.911+0000 7f370be00640 1 --2- 172.21.15.16:0/3386983863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634da9584d0 0x5634dad1d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.911+0000 7f3711c00640 1 --2- 172.21.15.16:0/2575759609 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f36f40b4d50 0x7f36f40b7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.912+0000 7f370be00640 1 -- 172.21.15.16:0/3386983863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634da9791b0 msgr2=0x5634dad1c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.912+0000 7f370be00640 1 --2- 172.21.15.16:0/3386983863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634da9791b0 0x5634dad1c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.912+0000 7f370be00640 1 -- 172.21.15.16:0/3386983863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634da9502f0 msgr2=0x5634dad1ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.912+0000 7f370be00640 1 --2- 172.21.15.16:0/3386983863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634da9502f0 0x5634dad1ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.912+0000 7f370be00640 1 -- 172.21.15.16:0/3386983863 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634dac3bc10 con 0x5634da9584d0 2024-08-31T06:15:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.912+0000 7f3711200640 1 --2- 172.21.15.16:0/3386983863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634da9502f0 0x5634dad1ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:15:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.913+0000 7f3709600640 1 -- 172.21.15.16:0/2575759609 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8470+0+0 (secure 0 0 0) 0x7f37001059b0 con 0x7f370c105a90 2024-08-31T06:15:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.913+0000 7f3711c00640 1 --2- 172.21.15.16:0/2575759609 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f36f40b4d50 0x7f36f40b7260 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3704015a80 tx=0x7f3704009b00 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.913+0000 7f370be00640 1 --2- 172.21.15.16:0/3386983863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634da9584d0 0x5634dad1d3a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f370c06c650 tx=0x7f36f8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1912aa47ccb33f8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.915+0000 7f36fe000640 1 -- 172.21.15.16:0/3386983863 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f8009d70 con 0x5634da9584d0 2024-08-31T06:15:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.915+0000 7f36fe000640 1 -- 172.21.15.16:0/3386983863 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36f8005020 con 0x5634da9584d0 2024-08-31T06:15:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.915+0000 7f36fe000640 1 -- 172.21.15.16:0/3386983863 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f8004a00 con 0x5634da9584d0 2024-08-31T06:15:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.917+0000 7f371ac86740 1 -- 172.21.15.16:0/3386983863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634da9584d0 msgr2=0x5634dad1d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.917+0000 7f371ac86740 1 --2- 172.21.15.16:0/3386983863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634da9584d0 0x5634dad1d3a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f370c06c650 tx=0x7f36f8011810 comp rx=0 tx=0).stop 2024-08-31T06:15:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.919+0000 7f371ac86740 1 -- 172.21.15.16:0/3386983863 shutdown_connections 2024-08-31T06:15:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.919+0000 7f371ac86740 1 --2- 172.21.15.16:0/3386983863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634da9584d0 0x5634dad1d3a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.919+0000 7f371ac86740 1 --2- 172.21.15.16:0/3386983863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634da9502f0 0x5634dad1ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.919+0000 7f371ac86740 1 --2- 172.21.15.16:0/3386983863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634da9791b0 0x5634dad1c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.919+0000 7f371ac86740 1 -- 172.21.15.16:0/3386983863 >> 172.21.15.16:0/3386983863 conn(0x5634da96d300 msgr2=0x5634dad1a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.920+0000 7f371ac86740 1 -- 172.21.15.16:0/3386983863 shutdown_connections 2024-08-31T06:15:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.920+0000 7f371ac86740 1 -- 172.21.15.16:0/3386983863 wait complete. 2024-08-31T06:15:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.921+0000 7f371ac86740 1 -- start start 2024-08-31T06:15:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.922+0000 7f371ac86740 1 Processor -- start 2024-08-31T06:15:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.923+0000 7f371ac86740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634da9502f0 0x5634dad27f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.923+0000 7f3711200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634da9502f0 0x5634dad27f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.923+0000 7f3711200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634da9502f0 0x5634dad27f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:43912/0 (socket says 172.21.15.16:43912) 2024-08-31T06:15:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.923+0000 7f3711200640 1 -- 172.21.15.16:0/3254382747 learned_addr learned my addr 172.21.15.16:0/3254382747 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.924+0000 7f371ac86740 1 --2- 172.21.15.16:0/3254382747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634da9584d0 0x5634dad28440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.924+0000 7f371ac86740 1 --2- 172.21.15.16:0/3254382747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634da9791b0 0x5634dad28980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.924+0000 7f370be00640 1 --2- 172.21.15.16:0/3254382747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634da9584d0 0x5634dad28440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.926+0000 7f3711c00640 1 --2- 172.21.15.16:0/3254382747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634da9791b0 0x5634dad28980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.926+0000 7f371ac86740 1 -- 172.21.15.16:0/3254382747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5634dabbaab0 con 0x5634da9791b0 2024-08-31T06:15:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.926+0000 7f371ac86740 1 -- 172.21.15.16:0/3254382747 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5634dac28320 con 0x5634da9502f0 2024-08-31T06:15:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.926+0000 7f371ac86740 1 -- 172.21.15.16:0/3254382747 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5634dad2b0a0 con 0x5634da9584d0 2024-08-31T06:15:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.926+0000 7f371ac86740 10 client.0 _ll_register_callbacks cb 0x5634dac2e6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:15:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.927+0000 7f370be00640 1 -- 172.21.15.16:0/3254382747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634da9502f0 msgr2=0x5634dad27f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.927+0000 7f370be00640 1 --2- 172.21.15.16:0/3254382747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634da9502f0 0x5634dad27f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.927+0000 7f370be00640 1 -- 172.21.15.16:0/3254382747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634da9791b0 msgr2=0x5634dad28980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.928+0000 7f370be00640 1 --2- 172.21.15.16:0/3254382747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634da9791b0 0x5634dad28980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.929+0000 7f370be00640 1 -- 172.21.15.16:0/3254382747 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634da9e4790 con 0x5634da9584d0 2024-08-31T06:15:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.929+0000 7f3711c00640 1 --2- 172.21.15.16:0/3254382747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634da9791b0 0x5634dad28980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:15:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.931+0000 7f370be00640 1 --2- 172.21.15.16:0/3254382747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634da9584d0 0x5634dad28440 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f36f8000c00 tx=0x7f36f80056d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=691038ebabd2aa96 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.931+0000 7f36f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:15:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.931+0000 7f36f2a00640 1 -- 172.21.15.16:0/3254382747 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f8027020 con 0x5634da9584d0 2024-08-31T06:15:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.932+0000 7f371ac86740 1 -- 172.21.15.16:0/3254382747 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36f8005020 con 0x5634da9584d0 2024-08-31T06:15:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.932+0000 7f36f2a00640 1 -- 172.21.15.16:0/3254382747 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8470+0+0 (secure 0 0 0) 0x7f36f8004a10 con 0x5634da9584d0 2024-08-31T06:15:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.935+0000 7f36f2a00640 1 -- 172.21.15.16:0/3254382747 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f36f8004c10 con 0x5634da9584d0 2024-08-31T06:15:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.935+0000 7f371ac86740 10 client.27448 fetch_fsmap learned FSMap version 39 2024-08-31T06:15:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.935+0000 7f371ac86740 1 -- 172.21.15.16:0/3254382747 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5634daa5d620 con 0x5634da9584d0 2024-08-31T06:15:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.935+0000 7f36f2a00640 1 -- 172.21.15.16:0/3254382747 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36f8005020 con 0x5634da9584d0 2024-08-31T06:15:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.935+0000 7f36f2a00640 1 -- 172.21.15.16:0/3254382747 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f36f8019410 con 0x5634da9584d0 2024-08-31T06:15:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.935+0000 7f371ac86740 10 client.27448 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T06:15:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.935+0000 7f371ac86740 10 client.27448 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:15:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.936+0000 7f371ac86740 20 client.27448 populate_metadata read hostname 'smithi016' 2024-08-31T06:15:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.936+0000 7f371ac86740 1 --2- 172.21.15.16:0/3254382747 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5634da937d80 0x5634dad39f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.936+0000 7f371ac86740 4 client.27448 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:15:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.936+0000 7f371ac86740 1 -- 172.21.15.16:0/3254382747 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634dac93ec0 con 0x5634da937d80 2024-08-31T06:15:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.938+0000 7f3711200640 1 --2- 172.21.15.16:0/3254382747 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5634da937d80 0x5634dad39f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.939+0000 7f3711200640 1 --2- 172.21.15.16:0/3254382747 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5634da937d80 0x5634dad39f90 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7aa88dc5b547134c server_cookie=61c6b4c3f1f1f78c in_seq=0 out_seq=0 2024-08-31T06:15:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.939+0000 7f36f2a00640 10 client.27448 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:15:24.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.942+0000 7f36f2a00640 1 -- 172.21.15.16:0/3254382747 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5634dac93ec0 con 0x5634da937d80 2024-08-31T06:15:24.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.942+0000 7f36f2a00640 10 client.27448 handle_command_reply: tid=0 2024-08-31T06:15:24.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.942+0000 7f371ac86740 1 client.27448 shutdown 2024-08-31T06:15:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.946+0000 7f371ac86740 1 -- 172.21.15.16:0/3254382747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634da9584d0 msgr2=0x5634dad28440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.946+0000 7f371ac86740 1 --2- 172.21.15.16:0/3254382747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634da9584d0 0x5634dad28440 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f36f8000c00 tx=0x7f36f80056d0 comp rx=0 tx=0).stop 2024-08-31T06:15:24.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.947+0000 7f371ac86740 1 -- 172.21.15.16:0/3254382747 shutdown_connections 2024-08-31T06:15:24.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.947+0000 7f371ac86740 1 --2- 172.21.15.16:0/3254382747 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5634da937d80 0x5634dad39f90 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:24.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.947+0000 7f371ac86740 1 --2- 172.21.15.16:0/3254382747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634da9584d0 0x5634dad28440 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.947+0000 7f371ac86740 1 --2- 172.21.15.16:0/3254382747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634da9791b0 0x5634dad28980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.947+0000 7f371ac86740 1 --2- 172.21.15.16:0/3254382747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634da9502f0 0x5634dad27f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.947+0000 7f371ac86740 1 -- 172.21.15.16:0/3254382747 >> 172.21.15.16:0/3254382747 conn(0x5634da96d300 msgr2=0x5634dad1d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.947+0000 7f36f2a00640 0 client.27448 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:15:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.947+0000 7f371ac86740 1 -- 172.21.15.16:0/3254382747 shutdown_connections 2024-08-31T06:15:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.948+0000 7f371ac86740 1 -- 172.21.15.16:0/3254382747 wait complete. 2024-08-31T06:15:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.948+0000 7f371ac86740 20 client.27448 trim_cache size 0 max 16384 2024-08-31T06:15:24.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.984+0000 7f371ac86740 1 Processor -- start 2024-08-31T06:15:24.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.985+0000 7f371ac86740 1 -- start start 2024-08-31T06:15:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.985+0000 7f371ac86740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634da9502f0 0x5634dad29bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.985+0000 7f371ac86740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634da9584d0 0x5634dad3f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.985+0000 7f371ac86740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634da9791b0 0x5634dad3fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.985+0000 7f371ac86740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f370c1c0b60 con 0x5634da9584d0 2024-08-31T06:15:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.985+0000 7f371ac86740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f370c1be3d0 con 0x5634da9791b0 2024-08-31T06:15:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.985+0000 7f371ac86740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f370c1bc240 con 0x5634da9502f0 2024-08-31T06:15:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.986+0000 7f3711200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634da9502f0 0x5634dad29bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.986+0000 7f3711c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634da9791b0 0x5634dad3fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.986+0000 7f370be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634da9584d0 0x5634dad3f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.986+0000 7f3711200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634da9502f0 0x5634dad29bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:52904/0 (socket says 172.21.15.16:52904) 2024-08-31T06:15:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.986+0000 7f3711200640 1 -- 172.21.15.16:0/3348486155 learned_addr learned my addr 172.21.15.16:0/3348486155 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:24.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.987+0000 7f3711c00640 1 -- 172.21.15.16:0/3348486155 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634da9502f0 msgr2=0x5634dad29bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:24.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.987+0000 7f3711c00640 1 --2- 172.21.15.16:0/3348486155 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634da9502f0 0x5634dad29bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:24.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.987+0000 7f3711c00640 1 -- 172.21.15.16:0/3348486155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634da9584d0 msgr2=0x5634dad3f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:24.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.987+0000 7f3711c00640 1 --2- 172.21.15.16:0/3348486155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634da9584d0 0x5634dad3f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:24.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.987+0000 7f3711c00640 1 -- 172.21.15.16:0/3348486155 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634dad1ec20 con 0x5634da9791b0 2024-08-31T06:15:24.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.987+0000 7f3711200640 1 --2- 172.21.15.16:0/3348486155 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634da9502f0 0x5634dad29bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:15:24.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.987+0000 7f3711c00640 1 --2- 172.21.15.16:0/3348486155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634da9791b0 0x5634dad3fbb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x5634dad2ac70 tx=0x7f370401c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a716c1d6f1d1846 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:24.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.987+0000 7f36fd600640 1 -- 172.21.15.16:0/3348486155 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3704004cf0 con 0x5634da9791b0 2024-08-31T06:15:24.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.988+0000 7f36fd600640 1 -- 172.21.15.16:0/3348486155 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3704004e90 con 0x5634da9791b0 2024-08-31T06:15:24.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.988+0000 7f36fd600640 1 -- 172.21.15.16:0/3348486155 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370401e9f0 con 0x5634da9791b0 2024-08-31T06:15:24.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.989+0000 7f371ac86740 1 -- 172.21.15.16:0/3348486155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634da9791b0 msgr2=0x5634dad3fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:24.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.989+0000 7f371ac86740 1 --2- 172.21.15.16:0/3348486155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634da9791b0 0x5634dad3fbb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x5634dad2ac70 tx=0x7f370401c000 comp rx=0 tx=0).stop 2024-08-31T06:15:24.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.989+0000 7f371ac86740 1 -- 172.21.15.16:0/3348486155 shutdown_connections 2024-08-31T06:15:24.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.989+0000 7f371ac86740 1 --2- 172.21.15.16:0/3348486155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634da9791b0 0x5634dad3fbb0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:24.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.989+0000 7f371ac86740 1 --2- 172.21.15.16:0/3348486155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634da9584d0 0x5634dad3f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:24.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.989+0000 7f371ac86740 1 --2- 172.21.15.16:0/3348486155 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634da9502f0 0x5634dad29bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:24.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.989+0000 7f371ac86740 1 -- 172.21.15.16:0/3348486155 >> 172.21.15.16:0/3348486155 conn(0x5634da96d300 msgr2=0x5634dad1e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:24.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.990+0000 7f371ac86740 1 -- 172.21.15.16:0/3348486155 shutdown_connections 2024-08-31T06:15:24.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.990+0000 7f371ac86740 1 -- 172.21.15.16:0/3348486155 wait complete. 2024-08-31T06:15:24.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.991+0000 7f371ac86740 1 -- start start 2024-08-31T06:15:24.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.991+0000 7f371ac86740 1 Processor -- start 2024-08-31T06:15:24.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.992+0000 7f371ac86740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634da9502f0 0x5634dad300c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:24.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.992+0000 7f371ac86740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634da9584d0 0x5634dad306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:24.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.992+0000 7f371ac86740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634da9791b0 0x5634dad999e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:24.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.993+0000 7f371ac86740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f370c1bc890 con 0x5634da9791b0 2024-08-31T06:15:24.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.993+0000 7f371ac86740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f370c1bdd00 con 0x5634da9502f0 2024-08-31T06:15:24.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.993+0000 7f371ac86740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f370c1bcf60 con 0x5634da9584d0 2024-08-31T06:15:24.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.993+0000 7f371ac86740 10 client.0 _ll_register_callbacks cb 0x7f36f8005570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:15:24.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.993+0000 7f3711c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634da9791b0 0x5634dad999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:24.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.993+0000 7f3711200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634da9502f0 0x5634dad300c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:24.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.993+0000 7f3711c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634da9791b0 0x5634dad999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60040/0 (socket says 172.21.15.16:60040) 2024-08-31T06:15:24.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.993+0000 7f3711c00640 1 -- 172.21.15.16:0/4229555798 learned_addr learned my addr 172.21.15.16:0/4229555798 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:24.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.993+0000 7f370be00640 1 --2- 172.21.15.16:0/4229555798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634da9584d0 0x5634dad306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.994+0000 7f3711200640 1 -- 172.21.15.16:0/4229555798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634da9584d0 msgr2=0x5634dad306a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.994+0000 7f3711200640 1 --2- 172.21.15.16:0/4229555798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634da9584d0 0x5634dad306a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.994+0000 7f3711200640 1 -- 172.21.15.16:0/4229555798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634da9791b0 msgr2=0x5634dad999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.994+0000 7f3711200640 1 --2- 172.21.15.16:0/4229555798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634da9791b0 0x5634dad999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.994+0000 7f3711200640 1 -- 172.21.15.16:0/4229555798 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634dad9a080 con 0x5634da9502f0 2024-08-31T06:15:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.994+0000 7f370be00640 1 --2- 172.21.15.16:0/4229555798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634da9584d0 0x5634dad306a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:15:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.995+0000 7f3711200640 1 --2- 172.21.15.16:0/4229555798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634da9502f0 0x5634dad300c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f37000c9340 tx=0x7f37000b4040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fc8b37f43e08063 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.995+0000 7f36f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:15:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.995+0000 7f36f2a00640 1 -- 172.21.15.16:0/4229555798 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3700049040 con 0x5634da9502f0 2024-08-31T06:15:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.995+0000 7f36f2a00640 1 -- 172.21.15.16:0/4229555798 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8470+0+0 (secure 0 0 0) 0x7f37000a2300 con 0x5634da9502f0 2024-08-31T06:15:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.995+0000 7f371ac86740 1 -- 172.21.15.16:0/4229555798 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3704004e90 con 0x5634da9502f0 2024-08-31T06:15:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.996+0000 7f36f2a00640 1 -- 172.21.15.16:0/4229555798 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3700103550 con 0x5634da9502f0 2024-08-31T06:15:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.996+0000 7f371ac86740 10 client.27464 fetch_fsmap learned FSMap version 39 2024-08-31T06:15:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.996+0000 7f371ac86740 1 -- 172.21.15.16:0/4229555798 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5634dada46f0 con 0x5634da9502f0 2024-08-31T06:15:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.996+0000 7f36f2a00640 1 -- 172.21.15.16:0/4229555798 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3704004e90 con 0x5634da9502f0 2024-08-31T06:15:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.996+0000 7f36f2a00640 1 -- 172.21.15.16:0/4229555798 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f370004c060 con 0x5634da9502f0 2024-08-31T06:15:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.996+0000 7f371ac86740 10 client.27464 fetch_fsmap finished waiting for FSMap version 39 2024-08-31T06:15:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.996+0000 7f371ac86740 10 client.27464 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:15:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.996+0000 7f371ac86740 20 client.27464 populate_metadata read hostname 'smithi016' 2024-08-31T06:15:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.996+0000 7f371ac86740 1 --2- 172.21.15.16:0/4229555798 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5634da90d1c0 0x5634dada6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.996+0000 7f371ac86740 4 client.27464 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:15:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.996+0000 7f371ac86740 1 -- 172.21.15.16:0/4229555798 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5634dac8f960 con 0x5634da90d1c0 2024-08-31T06:15:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.997+0000 7f370be00640 1 --2- 172.21.15.16:0/4229555798 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5634da90d1c0 0x5634dada6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.998+0000 7f36f2a00640 10 client.27464 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:15:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:24.998+0000 7f370be00640 1 --2- 172.21.15.16:0/4229555798 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5634da90d1c0 0x5634dada6f60 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e2ddf2232728ead server_cookie=a64f99854e94273a in_seq=0 out_seq=0 2024-08-31T06:15:25.014 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:25.012+0000 7f36f2a00640 1 -- 172.21.15.16:0/4229555798 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5634dac8f960 con 0x5634da90d1c0 2024-08-31T06:15:25.014 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:25.012+0000 7f36f2a00640 10 client.27464 handle_command_reply: tid=0 2024-08-31T06:15:25.014 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:25.012+0000 7f371ac86740 1 client.27464 shutdown 2024-08-31T06:15:25.015 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:25.013+0000 7f371ac86740 1 -- 172.21.15.16:0/4229555798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634da9502f0 msgr2=0x5634dad300c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:25.015 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:25.013+0000 7f371ac86740 1 --2- 172.21.15.16:0/4229555798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634da9502f0 0x5634dad300c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f37000c9340 tx=0x7f37000b4040 comp rx=0 tx=0).stop 2024-08-31T06:15:25.015 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:25.013+0000 7f371ac86740 1 -- 172.21.15.16:0/4229555798 shutdown_connections 2024-08-31T06:15:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:25.013+0000 7f371ac86740 1 --2- 172.21.15.16:0/4229555798 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5634da90d1c0 0x5634dada6f60 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:25.013+0000 7f371ac86740 1 --2- 172.21.15.16:0/4229555798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634da9584d0 0x5634dad306a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:25.013+0000 7f371ac86740 1 --2- 172.21.15.16:0/4229555798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634da9791b0 0x5634dad999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:25.013+0000 7f371ac86740 1 --2- 172.21.15.16:0/4229555798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634da9502f0 0x5634dad300c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:25.013+0000 7f371ac86740 1 -- 172.21.15.16:0/4229555798 >> 172.21.15.16:0/4229555798 conn(0x5634da96d300 msgr2=0x5634dad1d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:25.013+0000 7f36f2a00640 0 client.27464 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:15:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:25.013+0000 7f371ac86740 1 -- 172.21.15.16:0/4229555798 shutdown_connections 2024-08-31T06:15:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:25.013+0000 7f371ac86740 1 -- 172.21.15.16:0/4229555798 wait complete. 2024-08-31T06:15:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:25.013+0000 7f371ac86740 20 client.27464 trim_cache size 0 max 16384 2024-08-31T06:15:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:15:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:15:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:15:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:15:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:25.014+0000 7f3713c00640 1 -- 172.21.15.16:0/2575759609 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f36f40b4d50 msgr2=0x7f36f40b7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:25.014+0000 7f3713c00640 1 --2- 172.21.15.16:0/2575759609 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f36f40b4d50 0x7f36f40b7260 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3704015a80 tx=0x7f3704009b00 comp rx=0 tx=0).stop 2024-08-31T06:15:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:25.014+0000 7f3713c00640 1 -- 172.21.15.16:0/2575759609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f370c105a90 msgr2=0x7f370c10fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:25.014+0000 7f3713c00640 1 --2- 172.21.15.16:0/2575759609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f370c105a90 0x7f370c10fa20 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f3700005ab0 tx=0x7f3700003c20 comp rx=0 tx=0).stop 2024-08-31T06:15:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:25.014+0000 7f3713c00640 1 -- 172.21.15.16:0/2575759609 shutdown_connections 2024-08-31T06:15:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:25.014+0000 7f3713c00640 1 --2- 172.21.15.16:0/2575759609 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f36f40b4d50 0x7f36f40b7260 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:25.014+0000 7f3713c00640 1 --2- 172.21.15.16:0/2575759609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f370c10b3c0 0x7f370c10ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:25.014+0000 7f3713c00640 1 --2- 172.21.15.16:0/2575759609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f370c10bd10 0x7f370c1104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:25.014+0000 7f3713c00640 1 --2- 172.21.15.16:0/2575759609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f370c105a90 0x7f370c10fa20 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:25.014+0000 7f3713c00640 1 -- 172.21.15.16:0/2575759609 >> 172.21.15.16:0/2575759609 conn(0x7f370c101490 msgr2=0x7f370c10aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:25.015+0000 7f3713c00640 1 -- 172.21.15.16:0/2575759609 shutdown_connections 2024-08-31T06:15:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:25.015+0000 7f3713c00640 1 -- 172.21.15.16:0/2575759609 wait complete. 2024-08-31T06:15:26.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 480.0 s, 1751.7 tps, lat 56.893 ms stddev 99.685 2024-08-31T06:15:26.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:26 smithi016 ceph-mon[17194]: pgmap v616: 97 pgs: 97 active+clean; 3.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 1.42k op/s 2024-08-31T06:15:26.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:26 smithi155 ceph-mon[21742]: pgmap v616: 97 pgs: 97 active+clean; 3.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 1.42k op/s 2024-08-31T06:15:26.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:26 smithi184 ceph-mon[22331]: pgmap v616: 97 pgs: 97 active+clean; 3.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 1.42k op/s 2024-08-31T06:15:28.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:28 smithi155 ceph-mon[21742]: pgmap v617: 97 pgs: 97 active+clean; 3.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 MiB/s wr, 1.67k op/s 2024-08-31T06:15:28.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:28 smithi016 ceph-mon[17194]: pgmap v617: 97 pgs: 97 active+clean; 3.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 MiB/s wr, 1.67k op/s 2024-08-31T06:15:28.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:28 smithi184 ceph-mon[22331]: pgmap v617: 97 pgs: 97 active+clean; 3.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 MiB/s wr, 1.67k op/s 2024-08-31T06:15:30.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:30 smithi155 ceph-mon[21742]: pgmap v618: 97 pgs: 97 active+clean; 3.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 19 MiB/s wr, 1.74k op/s 2024-08-31T06:15:30.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:30 smithi016 ceph-mon[17194]: pgmap v618: 97 pgs: 97 active+clean; 3.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 19 MiB/s wr, 1.74k op/s 2024-08-31T06:15:30.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:30 smithi184 ceph-mon[22331]: pgmap v618: 97 pgs: 97 active+clean; 3.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 19 MiB/s wr, 1.74k op/s 2024-08-31T06:15:31.625 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 485.0 s, 2169.7 tps, lat 46.058 ms stddev 57.021 2024-08-31T06:15:32.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:32 smithi155 ceph-mon[21742]: pgmap v619: 97 pgs: 97 active+clean; 3.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 1.64k op/s 2024-08-31T06:15:32.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:32 smithi016 ceph-mon[17194]: pgmap v619: 97 pgs: 97 active+clean; 3.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 1.64k op/s 2024-08-31T06:15:32.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:32 smithi184 ceph-mon[22331]: pgmap v619: 97 pgs: 97 active+clean; 3.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 1.64k op/s 2024-08-31T06:15:34.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:34 smithi155 ceph-mon[21742]: pgmap v620: 97 pgs: 97 active+clean; 3.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 17 MiB/s wr, 1.56k op/s 2024-08-31T06:15:34.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:34 smithi016 ceph-mon[17194]: pgmap v620: 97 pgs: 97 active+clean; 3.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 17 MiB/s wr, 1.56k op/s 2024-08-31T06:15:34.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:34 smithi184 ceph-mon[22331]: pgmap v620: 97 pgs: 97 active+clean; 3.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 17 MiB/s wr, 1.56k op/s 2024-08-31T06:15:36.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 490.0 s, 2602.0 tps, lat 38.417 ms stddev 46.400 2024-08-31T06:15:36.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:36 smithi155 ceph-mon[21742]: pgmap v621: 97 pgs: 97 active+clean; 3.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 15 MiB/s wr, 1.46k op/s 2024-08-31T06:15:36.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:36 smithi016 ceph-mon[17194]: pgmap v621: 97 pgs: 97 active+clean; 3.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 15 MiB/s wr, 1.46k op/s 2024-08-31T06:15:36.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:36 smithi184 ceph-mon[22331]: pgmap v621: 97 pgs: 97 active+clean; 3.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 15 MiB/s wr, 1.46k op/s 2024-08-31T06:15:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.307+0000 7fa112200640 1 -- 172.21.15.16:0/3270384515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa10c10d950 msgr2=0x7fa10c10fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.307+0000 7fa112200640 1 --2- 172.21.15.16:0/3270384515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa10c10d950 0x7fa10c10fe40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa104009ca0 tx=0x7fa10402e9a0 comp rx=0 tx=0).stop 2024-08-31T06:15:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.307+0000 7fa112200640 1 -- 172.21.15.16:0/3270384515 shutdown_connections 2024-08-31T06:15:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.307+0000 7fa112200640 1 --2- 172.21.15.16:0/3270384515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa10c10d950 0x7fa10c10fe40 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.307+0000 7fa112200640 1 --2- 172.21.15.16:0/3270384515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa10c104dd0 0x7fa10c10d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.307+0000 7fa112200640 1 --2- 172.21.15.16:0/3270384515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa10c1043a0 0x7fa10c1047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.307+0000 7fa112200640 1 -- 172.21.15.16:0/3270384515 >> 172.21.15.16:0/3270384515 conn(0x7fa10c0fdfd0 msgr2=0x7fa10c100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.308+0000 7fa112200640 1 -- 172.21.15.16:0/3270384515 shutdown_connections 2024-08-31T06:15:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.308+0000 7fa112200640 1 -- 172.21.15.16:0/3270384515 wait complete. 2024-08-31T06:15:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.309+0000 7fa112200640 1 Processor -- start 2024-08-31T06:15:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.310+0000 7fa112200640 1 -- start start 2024-08-31T06:15:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.311+0000 7fa112200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa10c1043a0 0x7fa10c074c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.311+0000 7fa112200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa10c104dd0 0x7fa10c06dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:38.314 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.311+0000 7fa110e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa10c1043a0 0x7fa10c074c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:38.314 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.312+0000 7fa110e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa10c1043a0 0x7fa10c074c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:33104/0 (socket says 172.21.15.16:33104) 2024-08-31T06:15:38.314 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.312+0000 7fa110e00640 1 -- 172.21.15.16:0/1914280909 learned_addr learned my addr 172.21.15.16:0/1914280909 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:38.315 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.312+0000 7fa10be00640 1 --2- 172.21.15.16:0/1914280909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa10c104dd0 0x7fa10c06dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:38.315 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.312+0000 7fa112200640 1 --2- 172.21.15.16:0/1914280909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa10c10d950 0x7fa10c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:38.315 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.312+0000 7fa112200640 1 -- 172.21.15.16:0/1914280909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c1113c0 con 0x7fa10c104dd0 2024-08-31T06:15:38.315 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.312+0000 7fa112200640 1 -- 172.21.15.16:0/1914280909 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c111240 con 0x7fa10c10d950 2024-08-31T06:15:38.315 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.312+0000 7fa112200640 1 -- 172.21.15.16:0/1914280909 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c111540 con 0x7fa10c1043a0 2024-08-31T06:15:38.315 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.312+0000 7fa111800640 1 --2- 172.21.15.16:0/1914280909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa10c10d950 0x7fa10c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:38.315 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.312+0000 7fa110e00640 1 -- 172.21.15.16:0/1914280909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa10c10d950 msgr2=0x7fa10c06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:38.315 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.312+0000 7fa110e00640 1 --2- 172.21.15.16:0/1914280909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa10c10d950 0x7fa10c06e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:38.315 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.312+0000 7fa110e00640 1 -- 172.21.15.16:0/1914280909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa10c104dd0 msgr2=0x7fa10c06dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.312+0000 7fa110e00640 1 --2- 172.21.15.16:0/1914280909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa10c104dd0 0x7fa10c06dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.312+0000 7fa110e00640 1 -- 172.21.15.16:0/1914280909 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa10c06ec60 con 0x7fa10c1043a0 2024-08-31T06:15:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.313+0000 7fa110e00640 1 --2- 172.21.15.16:0/1914280909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa10c1043a0 0x7fa10c074c00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa10000e970 tx=0x7fa10000ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.313+0000 7fa109600640 1 -- 172.21.15.16:0/1914280909 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10000ccc0 con 0x7fa10c1043a0 2024-08-31T06:15:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.313+0000 7fa109600640 1 -- 172.21.15.16:0/1914280909 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1000108e0 con 0x7fa10c1043a0 2024-08-31T06:15:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.313+0000 7fa109600640 1 -- 172.21.15.16:0/1914280909 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1000187c0 con 0x7fa10c1043a0 2024-08-31T06:15:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.313+0000 7fa112200640 1 -- 172.21.15.16:0/1914280909 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa10c073920 con 0x7fa10c1043a0 2024-08-31T06:15:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.314+0000 7fa112200640 1 -- 172.21.15.16:0/1914280909 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa10c073e20 con 0x7fa10c1043a0 2024-08-31T06:15:38.322 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.320+0000 7fa11aa34740 1 Processor -- start 2024-08-31T06:15:38.323 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.320+0000 7fa11aa34740 1 -- start start 2024-08-31T06:15:38.323 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.321+0000 7fa11aa34740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd0dc091b0 0x55dd0dfacb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:38.323 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.321+0000 7fa11aa34740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd0dbe02f0 0x55dd0dfad040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:38.323 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.321+0000 7fa11aa34740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd0dbe84d0 0x55dd0dfad580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:38.324 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.321+0000 7fa11aa34740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dd0df21f70 con 0x55dd0dbe02f0 2024-08-31T06:15:38.324 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.321+0000 7fa11aa34740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55dd0df11500 con 0x55dd0dbe84d0 2024-08-31T06:15:38.324 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.321+0000 7fa11aa34740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55dd0df154b0 con 0x55dd0dc091b0 2024-08-31T06:15:38.324 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.321+0000 7fa111800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd0dc091b0 0x55dd0dfacb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:38.324 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.321+0000 7fa10be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd0dbe84d0 0x55dd0dfad580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:38.325 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.322+0000 7fa111800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd0dc091b0 0x55dd0dfacb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:33106/0 (socket says 172.21.15.16:33106) 2024-08-31T06:15:38.325 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.322+0000 7fa111800640 1 -- 172.21.15.16:0/2284759774 learned_addr learned my addr 172.21.15.16:0/2284759774 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:38.325 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.322+0000 7fa109600640 1 -- 172.21.15.16:0/1914280909 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fa100010430 con 0x7fa10c1043a0 2024-08-31T06:15:38.325 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.322+0000 7fa111800640 1 -- 172.21.15.16:0/2284759774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd0dbe84d0 msgr2=0x55dd0dfad580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:38.325 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.322+0000 7fa111800640 1 --2- 172.21.15.16:0/2284759774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd0dbe84d0 0x55dd0dfad580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:38.325 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.322+0000 7fa111800640 1 -- 172.21.15.16:0/2284759774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd0dbe02f0 msgr2=0x55dd0dfad040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:38.325 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.322+0000 7fa111800640 1 --2- 172.21.15.16:0/2284759774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd0dbe02f0 0x55dd0dfad040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:38.325 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.322+0000 7fa111800640 1 -- 172.21.15.16:0/2284759774 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd0decbc10 con 0x55dd0dc091b0 2024-08-31T06:15:38.326 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.323+0000 7fa10be00640 1 --2- 172.21.15.16:0/2284759774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd0dbe84d0 0x55dd0dfad580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:15:38.326 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.323+0000 7fa111800640 1 --2- 172.21.15.16:0/2284759774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd0dc091b0 0x55dd0dfacb00 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa104009490 tx=0x7fa104009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60d798a3d16f7557 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:38.326 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.324+0000 7fa0fe000640 1 -- 172.21.15.16:0/2284759774 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa104037070 con 0x55dd0dc091b0 2024-08-31T06:15:38.326 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.324+0000 7fa0fe000640 1 -- 172.21.15.16:0/2284759774 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa10403a810 con 0x55dd0dc091b0 2024-08-31T06:15:38.326 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.324+0000 7fa0fe000640 1 -- 172.21.15.16:0/2284759774 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa104004280 con 0x55dd0dc091b0 2024-08-31T06:15:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.325+0000 7fa109600640 1 --2- 172.21.15.16:0/1914280909 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa0f40b4dd0 0x7fa0f40b72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:38.328 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.325+0000 7fa11aa34740 1 -- 172.21.15.16:0/2284759774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd0dc091b0 msgr2=0x55dd0dfacb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:38.328 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.325+0000 7fa11aa34740 1 --2- 172.21.15.16:0/2284759774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd0dc091b0 0x55dd0dfacb00 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa104009490 tx=0x7fa104009a20 comp rx=0 tx=0).stop 2024-08-31T06:15:38.328 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.325+0000 7fa109600640 1 -- 172.21.15.16:0/1914280909 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8470+0+0 (secure 0 0 0) 0x7fa1000e5850 con 0x7fa10c1043a0 2024-08-31T06:15:38.328 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.325+0000 7fa110e00640 1 --2- 172.21.15.16:0/1914280909 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa0f40b4dd0 0x7fa0f40b72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:38.328 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.326+0000 7fa11aa34740 1 -- 172.21.15.16:0/2284759774 shutdown_connections 2024-08-31T06:15:38.328 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.326+0000 7fa11aa34740 1 --2- 172.21.15.16:0/2284759774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd0dbe84d0 0x55dd0dfad580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.326+0000 7fa11aa34740 1 --2- 172.21.15.16:0/2284759774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd0dbe02f0 0x55dd0dfad040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.326+0000 7fa11aa34740 1 --2- 172.21.15.16:0/2284759774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd0dc091b0 0x55dd0dfacb00 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.326+0000 7fa11aa34740 1 -- 172.21.15.16:0/2284759774 >> 172.21.15.16:0/2284759774 conn(0x55dd0dbfd300 msgr2=0x55dd0dfaa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.326+0000 7fa11aa34740 1 -- 172.21.15.16:0/2284759774 shutdown_connections 2024-08-31T06:15:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.326+0000 7fa110e00640 1 --2- 172.21.15.16:0/1914280909 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa0f40b4dd0 0x7fa0f40b72e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x55dd0dfab780 tx=0x7fa100010770 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.326+0000 7fa11aa34740 1 -- 172.21.15.16:0/2284759774 wait complete. 2024-08-31T06:15:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.327+0000 7fa11aa34740 1 -- start start 2024-08-31T06:15:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.328+0000 7fa11aa34740 1 Processor -- start 2024-08-31T06:15:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.330+0000 7fa11aa34740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd0dbe02f0 0x55dd0dfb80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.331+0000 7fa11aa34740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd0dbe84d0 0x55dd0dfb85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.331+0000 7fa11aa34740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd0dc091b0 0x55dd0dfb8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.331+0000 7fa11aa34740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dd0de4aab0 con 0x55dd0dbe02f0 2024-08-31T06:15:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.331+0000 7fa11aa34740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55dd0deb8320 con 0x55dd0dbe84d0 2024-08-31T06:15:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.331+0000 7fa11aa34740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55dd0dfbb220 con 0x55dd0dc091b0 2024-08-31T06:15:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.331+0000 7fa11aa34740 10 client.0 _ll_register_callbacks cb 0x55dd0debe6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:15:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.331+0000 7fa10be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd0dbe84d0 0x55dd0dfb85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.331+0000 7fa10be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd0dbe84d0 0x55dd0dfb85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:58874/0 (socket says 172.21.15.16:58874) 2024-08-31T06:15:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.331+0000 7fa10be00640 1 -- 172.21.15.16:0/3565489992 learned_addr learned my addr 172.21.15.16:0/3565489992 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.332+0000 7fa10be00640 1 -- 172.21.15.16:0/3565489992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd0dc091b0 msgr2=0x55dd0dfb8b30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T06:15:38.335 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.332+0000 7fa10be00640 1 --2- 172.21.15.16:0/3565489992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd0dc091b0 0x55dd0dfb8b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:38.335 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.332+0000 7fa10be00640 1 -- 172.21.15.16:0/3565489992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd0dbe02f0 msgr2=0x55dd0dfb80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:38.335 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.332+0000 7fa10be00640 1 --2- 172.21.15.16:0/3565489992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd0dbe02f0 0x55dd0dfb80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:38.335 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.332+0000 7fa10be00640 1 -- 172.21.15.16:0/3565489992 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd0dc74790 con 0x55dd0dbe84d0 2024-08-31T06:15:38.335 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.332+0000 7fa10be00640 1 --2- 172.21.15.16:0/3565489992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd0dbe84d0 0x55dd0dfb85f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa0f800bf20 tx=0x7fa0f80128e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9befdab8d077581 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:38.335 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.333+0000 7fa0f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:15:38.337 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.334+0000 7fa0f2a00640 1 -- 172.21.15.16:0/3565489992 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f8005220 con 0x55dd0dbe84d0 2024-08-31T06:15:38.337 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.335+0000 7fa11aa34740 1 -- 172.21.15.16:0/3565489992 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55dd0df23ec0 con 0x55dd0dbe84d0 2024-08-31T06:15:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.336+0000 7fa0f2a00640 1 -- 172.21.15.16:0/3565489992 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8470+0+0 (secure 0 0 0) 0x7fa0f80053c0 con 0x55dd0dbe84d0 2024-08-31T06:15:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.336+0000 7fa0f2a00640 1 -- 172.21.15.16:0/3565489992 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa0f8005600 con 0x55dd0dbe84d0 2024-08-31T06:15:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.336+0000 7fa11aa34740 10 client.27476 fetch_fsmap learned FSMap version 40 2024-08-31T06:15:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.336+0000 7fa11aa34740 1 -- 172.21.15.16:0/3565489992 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55dd0dced620 con 0x55dd0dbe84d0 2024-08-31T06:15:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.337+0000 7fa0f2a00640 1 -- 172.21.15.16:0/3565489992 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55dd0df23ec0 con 0x55dd0dbe84d0 2024-08-31T06:15:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.337+0000 7fa0f2a00640 1 -- 172.21.15.16:0/3565489992 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 40) ==== 8358+0+0 (secure 0 0 0) 0x7fa0f8010a80 con 0x55dd0dbe84d0 2024-08-31T06:15:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.337+0000 7fa11aa34740 10 client.27476 fetch_fsmap finished waiting for FSMap version 40 2024-08-31T06:15:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.337+0000 7fa11aa34740 10 client.27476 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:15:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.337+0000 7fa11aa34740 20 client.27476 populate_metadata read hostname 'smithi016' 2024-08-31T06:15:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.337+0000 7fa11aa34740 1 --2- 172.21.15.16:0/3565489992 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55dd0dbc7d80 0x55dd0dfca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.338+0000 7fa11aa34740 4 client.27476 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:15:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.338+0000 7fa11aa34740 1 -- 172.21.15.16:0/3565489992 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd0df1f960 con 0x55dd0dbc7d80 2024-08-31T06:15:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.338+0000 7fa111800640 1 --2- 172.21.15.16:0/3565489992 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55dd0dbc7d80 0x55dd0dfca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.339+0000 7fa111800640 1 --2- 172.21.15.16:0/3565489992 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55dd0dbc7d80 0x55dd0dfca3a0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=192696348404a420 server_cookie=533a8aa883e1aa2d in_seq=0 out_seq=0 2024-08-31T06:15:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.339+0000 7fa0f2a00640 10 client.27476 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:15:38.342 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.340+0000 7fa0f2a00640 1 -- 172.21.15.16:0/3565489992 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dd0df1f960 con 0x55dd0dbc7d80 2024-08-31T06:15:38.342 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.340+0000 7fa0f2a00640 10 client.27476 handle_command_reply: tid=0 2024-08-31T06:15:38.343 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.341+0000 7fa11aa34740 1 client.27476 shutdown 2024-08-31T06:15:38.343 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.341+0000 7fa11aa34740 1 -- 172.21.15.16:0/3565489992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd0dbe84d0 msgr2=0x55dd0dfb85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:38.343 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.341+0000 7fa11aa34740 1 --2- 172.21.15.16:0/3565489992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd0dbe84d0 0x55dd0dfb85f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa0f800bf20 tx=0x7fa0f80128e0 comp rx=0 tx=0).stop 2024-08-31T06:15:38.344 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.341+0000 7fa11aa34740 1 -- 172.21.15.16:0/3565489992 shutdown_connections 2024-08-31T06:15:38.344 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.341+0000 7fa11aa34740 1 --2- 172.21.15.16:0/3565489992 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55dd0dbc7d80 0x55dd0dfca3a0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:38.344 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.341+0000 7fa11aa34740 1 --2- 172.21.15.16:0/3565489992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd0dc091b0 0x55dd0dfb8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:38.344 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.341+0000 7fa11aa34740 1 --2- 172.21.15.16:0/3565489992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd0dbe84d0 0x55dd0dfb85f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:38.344 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.341+0000 7fa11aa34740 1 --2- 172.21.15.16:0/3565489992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd0dbe02f0 0x55dd0dfb80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:38.344 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.341+0000 7fa11aa34740 1 -- 172.21.15.16:0/3565489992 >> 172.21.15.16:0/3565489992 conn(0x55dd0dbfd300 msgr2=0x55dd0dfadb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:38.344 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.341+0000 7fa0f2a00640 0 client.27476 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:15:38.345 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.342+0000 7fa11aa34740 1 -- 172.21.15.16:0/3565489992 shutdown_connections 2024-08-31T06:15:38.345 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.343+0000 7fa11aa34740 1 -- 172.21.15.16:0/3565489992 wait complete. 2024-08-31T06:15:38.345 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.343+0000 7fa11aa34740 20 client.27476 trim_cache size 0 max 16384 2024-08-31T06:15:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.368+0000 7fa11aa34740 1 Processor -- start 2024-08-31T06:15:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.368+0000 7fa11aa34740 1 -- start start 2024-08-31T06:15:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.369+0000 7fa11aa34740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd0dbe02f0 0x55dd0dfae240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.369+0000 7fa11aa34740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd0dbe84d0 0x55dd0dfa9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.369+0000 7fa11aa34740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd0dc091b0 0x55dd0dfaa490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.369+0000 7fa11aa34740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c1b8150 con 0x55dd0dbe84d0 2024-08-31T06:15:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.369+0000 7fa11aa34740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c1b5990 con 0x55dd0dc091b0 2024-08-31T06:15:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.369+0000 7fa11aa34740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c1b3800 con 0x55dd0dbe02f0 2024-08-31T06:15:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.370+0000 7fa111800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd0dbe02f0 0x55dd0dfae240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.370+0000 7fa111800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd0dbe02f0 0x55dd0dfae240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:33116/0 (socket says 172.21.15.16:33116) 2024-08-31T06:15:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.370+0000 7fa111800640 1 -- 172.21.15.16:0/1357796410 learned_addr learned my addr 172.21.15.16:0/1357796410 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.370+0000 7fa111800640 1 --2- 172.21.15.16:0/1357796410 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd0dc091b0 0x55dd0dfaa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.370+0000 7fa10be00640 1 --2- 172.21.15.16:0/1357796410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd0dbe84d0 0x55dd0dfa9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.371+0000 7fa111800640 1 -- 172.21.15.16:0/1357796410 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd0dc091b0 msgr2=0x55dd0dfaa490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.371+0000 7fa111800640 1 --2- 172.21.15.16:0/1357796410 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd0dc091b0 0x55dd0dfaa490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.371+0000 7fa111800640 1 -- 172.21.15.16:0/1357796410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd0dbe84d0 msgr2=0x55dd0dfa9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.371+0000 7fa111800640 1 --2- 172.21.15.16:0/1357796410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd0dbe84d0 0x55dd0dfa9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.371+0000 7fa111800640 1 -- 172.21.15.16:0/1357796410 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd0dfaf360 con 0x55dd0dbe02f0 2024-08-31T06:15:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.371+0000 7fa10be00640 1 --2- 172.21.15.16:0/1357796410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd0dbe84d0 0x55dd0dfa9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:15:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.371+0000 7fa111800640 1 --2- 172.21.15.16:0/1357796410 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd0dbe02f0 0x55dd0dfae240 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa1040094e0 tx=0x7fa104003e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af4bdd907e51be45 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.371+0000 7fa0fd600640 1 -- 172.21.15.16:0/1357796410 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa104037040 con 0x55dd0dbe02f0 2024-08-31T06:15:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.371+0000 7fa0fd600640 1 -- 172.21.15.16:0/1357796410 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa104003b70 con 0x55dd0dbe02f0 2024-08-31T06:15:38.375 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.372+0000 7fa0fd600640 1 -- 172.21.15.16:0/1357796410 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa104033e40 con 0x55dd0dbe02f0 2024-08-31T06:15:38.377 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.373+0000 7fa11aa34740 1 -- 172.21.15.16:0/1357796410 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd0dbe02f0 msgr2=0x55dd0dfae240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.373+0000 7fa11aa34740 1 --2- 172.21.15.16:0/1357796410 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd0dbe02f0 0x55dd0dfae240 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa1040094e0 tx=0x7fa104003e20 comp rx=0 tx=0).stop 2024-08-31T06:15:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.375+0000 7fa11aa34740 1 -- 172.21.15.16:0/1357796410 shutdown_connections 2024-08-31T06:15:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.375+0000 7fa11aa34740 1 --2- 172.21.15.16:0/1357796410 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd0dc091b0 0x55dd0dfaa490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.375+0000 7fa11aa34740 1 --2- 172.21.15.16:0/1357796410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd0dbe84d0 0x55dd0dfa9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.375+0000 7fa11aa34740 1 --2- 172.21.15.16:0/1357796410 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd0dbe02f0 0x55dd0dfae240 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.375+0000 7fa11aa34740 1 -- 172.21.15.16:0/1357796410 >> 172.21.15.16:0/1357796410 conn(0x55dd0dbfd300 msgr2=0x55dd0dfadb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.375+0000 7fa11aa34740 1 -- 172.21.15.16:0/1357796410 shutdown_connections 2024-08-31T06:15:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.375+0000 7fa11aa34740 1 -- 172.21.15.16:0/1357796410 wait complete. 2024-08-31T06:15:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.376+0000 7fa11aa34740 1 -- start start 2024-08-31T06:15:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.376+0000 7fa11aa34740 1 Processor -- start 2024-08-31T06:15:38.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.377+0000 7fa11aa34740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd0dbe02f0 0x55dd0e035890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:38.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.377+0000 7fa11aa34740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd0dbe84d0 0x55dd0e035e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:38.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.377+0000 7fa11aa34740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd0dc091b0 0x55dd0dfbdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:38.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.377+0000 7fa11aa34740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c1b3e50 con 0x55dd0dbe02f0 2024-08-31T06:15:38.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.377+0000 7fa11aa34740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c1b52c0 con 0x55dd0dbe84d0 2024-08-31T06:15:38.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.377+0000 7fa11aa34740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c1b4520 con 0x55dd0dc091b0 2024-08-31T06:15:38.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.377+0000 7fa11aa34740 10 client.0 _ll_register_callbacks cb 0x7fa0f800bab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:15:38.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.377+0000 7fa10be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd0dbe84d0 0x55dd0e035e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:38.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.377+0000 7fa10be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd0dbe84d0 0x55dd0e035e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:58888/0 (socket says 172.21.15.16:58888) 2024-08-31T06:15:38.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.377+0000 7fa10be00640 1 -- 172.21.15.16:0/686502215 learned_addr learned my addr 172.21.15.16:0/686502215 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:38.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.377+0000 7fa111800640 1 --2- 172.21.15.16:0/686502215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd0dbe02f0 0x55dd0e035890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.377+0000 7fa111800640 1 --2- 172.21.15.16:0/686502215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd0dc091b0 0x55dd0dfbdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.378+0000 7fa10be00640 1 -- 172.21.15.16:0/686502215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd0dc091b0 msgr2=0x55dd0dfbdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.378+0000 7fa10be00640 1 --2- 172.21.15.16:0/686502215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd0dc091b0 0x55dd0dfbdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.378+0000 7fa10be00640 1 -- 172.21.15.16:0/686502215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd0dbe02f0 msgr2=0x55dd0e035890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.378+0000 7fa10be00640 1 --2- 172.21.15.16:0/686502215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd0dbe02f0 0x55dd0e035890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.378+0000 7fa10be00640 1 -- 172.21.15.16:0/686502215 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd0dfbe2c0 con 0x55dd0dbe84d0 2024-08-31T06:15:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.378+0000 7fa111800640 1 --2- 172.21.15.16:0/686502215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd0dc091b0 0x55dd0dfbdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:15:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.378+0000 7fa10be00640 1 --2- 172.21.15.16:0/686502215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd0dbe84d0 0x55dd0e035e50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8000c00 tx=0x7fa0f8025820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d8e1427df64042f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.378+0000 7fa0f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:15:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.378+0000 7fa0f2a00640 1 -- 172.21.15.16:0/686502215 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f8032060 con 0x55dd0dbe84d0 2024-08-31T06:15:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.379+0000 7fa0f2a00640 1 -- 172.21.15.16:0/686502215 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8470+0+0 (secure 0 0 0) 0x7fa0f8025a50 con 0x55dd0dbe84d0 2024-08-31T06:15:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.379+0000 7fa11aa34740 1 -- 172.21.15.16:0/686502215 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa104003b70 con 0x55dd0dbe84d0 2024-08-31T06:15:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.379+0000 7fa0f2a00640 1 -- 172.21.15.16:0/686502215 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa0f8025c50 con 0x55dd0dbe84d0 2024-08-31T06:15:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.379+0000 7fa11aa34740 10 client.27482 fetch_fsmap learned FSMap version 40 2024-08-31T06:15:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.379+0000 7fa11aa34740 1 -- 172.21.15.16:0/686502215 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55dd0e03cfd0 con 0x55dd0dbe84d0 2024-08-31T06:15:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.380+0000 7fa0f2a00640 1 -- 172.21.15.16:0/686502215 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa104003b70 con 0x55dd0dbe84d0 2024-08-31T06:15:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.381+0000 7fa0f2a00640 1 -- 172.21.15.16:0/686502215 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 40) ==== 8358+0+0 (secure 0 0 0) 0x7fa0f8003830 con 0x55dd0dbe84d0 2024-08-31T06:15:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.381+0000 7fa11aa34740 10 client.27482 fetch_fsmap finished waiting for FSMap version 40 2024-08-31T06:15:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.381+0000 7fa11aa34740 10 client.27482 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:15:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.381+0000 7fa11aa34740 20 client.27482 populate_metadata read hostname 'smithi016' 2024-08-31T06:15:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.381+0000 7fa11aa34740 1 --2- 172.21.15.16:0/686502215 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55dd0db9d1c0 0x55dd0e03f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.381+0000 7fa11aa34740 4 client.27482 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:15:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.381+0000 7fa11aa34740 1 -- 172.21.15.16:0/686502215 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dd0df1d0a0 con 0x55dd0db9d1c0 2024-08-31T06:15:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.381+0000 7fa111800640 1 --2- 172.21.15.16:0/686502215 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55dd0db9d1c0 0x55dd0e03f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.382+0000 7fa0f2a00640 10 client.27482 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:15:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.382+0000 7fa111800640 1 --2- 172.21.15.16:0/686502215 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55dd0db9d1c0 0x55dd0e03f7d0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b18b902fbd26150 server_cookie=6a8e6e83afbaf20e in_seq=0 out_seq=0 2024-08-31T06:15:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.404+0000 7fa0f2a00640 1 -- 172.21.15.16:0/686502215 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dd0df1d0a0 con 0x55dd0db9d1c0 2024-08-31T06:15:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.404+0000 7fa0f2a00640 10 client.27482 handle_command_reply: tid=0 2024-08-31T06:15:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.404+0000 7fa11aa34740 1 client.27482 shutdown 2024-08-31T06:15:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.404+0000 7fa11aa34740 1 -- 172.21.15.16:0/686502215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd0dbe84d0 msgr2=0x55dd0e035e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.404+0000 7fa11aa34740 1 --2- 172.21.15.16:0/686502215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd0dbe84d0 0x55dd0e035e50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8000c00 tx=0x7fa0f8025820 comp rx=0 tx=0).stop 2024-08-31T06:15:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.404+0000 7fa11aa34740 1 -- 172.21.15.16:0/686502215 shutdown_connections 2024-08-31T06:15:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.404+0000 7fa11aa34740 1 --2- 172.21.15.16:0/686502215 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55dd0db9d1c0 0x55dd0e03f7d0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.404+0000 7fa11aa34740 1 --2- 172.21.15.16:0/686502215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd0dc091b0 0x55dd0dfbdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.404+0000 7fa11aa34740 1 --2- 172.21.15.16:0/686502215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd0dbe84d0 0x55dd0e035e50 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.404+0000 7fa11aa34740 1 --2- 172.21.15.16:0/686502215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd0dbe02f0 0x55dd0e035890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.404+0000 7fa11aa34740 1 -- 172.21.15.16:0/686502215 >> 172.21.15.16:0/686502215 conn(0x55dd0dbfd300 msgr2=0x55dd0dfadb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.404+0000 7fa0f2a00640 0 client.27482 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:15:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.405+0000 7fa11aa34740 1 -- 172.21.15.16:0/686502215 shutdown_connections 2024-08-31T06:15:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.406+0000 7fa11aa34740 1 -- 172.21.15.16:0/686502215 wait complete. 2024-08-31T06:15:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.406+0000 7fa11aa34740 20 client.27482 trim_cache size 0 max 16384 2024-08-31T06:15:38.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:15:38.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:15:38.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:15:38.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:15:38.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.412+0000 7fa112200640 1 -- 172.21.15.16:0/1914280909 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa0f40b4dd0 msgr2=0x7fa0f40b72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:38.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.412+0000 7fa112200640 1 --2- 172.21.15.16:0/1914280909 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa0f40b4dd0 0x7fa0f40b72e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x55dd0dfab780 tx=0x7fa100010770 comp rx=0 tx=0).stop 2024-08-31T06:15:38.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.412+0000 7fa112200640 1 -- 172.21.15.16:0/1914280909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa10c1043a0 msgr2=0x7fa10c074c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:38.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.412+0000 7fa112200640 1 --2- 172.21.15.16:0/1914280909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa10c1043a0 0x7fa10c074c00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa10000e970 tx=0x7fa10000ee30 comp rx=0 tx=0).stop 2024-08-31T06:15:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.413+0000 7fa112200640 1 -- 172.21.15.16:0/1914280909 shutdown_connections 2024-08-31T06:15:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.413+0000 7fa112200640 1 --2- 172.21.15.16:0/1914280909 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa0f40b4dd0 0x7fa0f40b72e0 secure :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0x55dd0dfab780 tx=0x7fa100010770 comp rx=0 tx=0).stop 2024-08-31T06:15:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.413+0000 7fa112200640 1 --2- 172.21.15.16:0/1914280909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa10c10d950 0x7fa10c06e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.413+0000 7fa112200640 1 --2- 172.21.15.16:0/1914280909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa10c104dd0 0x7fa10c06dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.413+0000 7fa112200640 1 --2- 172.21.15.16:0/1914280909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa10c1043a0 0x7fa10c074c00 secure :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa10000e970 tx=0x7fa10000ee30 comp rx=0 tx=0).stop 2024-08-31T06:15:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.413+0000 7fa112200640 1 -- 172.21.15.16:0/1914280909 >> 172.21.15.16:0/1914280909 conn(0x7fa10c0fdfd0 msgr2=0x7fa10c1074d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.416+0000 7fa112200640 1 -- 172.21.15.16:0/1914280909 shutdown_connections 2024-08-31T06:15:38.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:38.418+0000 7fa112200640 1 -- 172.21.15.16:0/1914280909 wait complete. 2024-08-31T06:15:38.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:38 smithi016 ceph-mon[17194]: pgmap v622: 97 pgs: 97 active+clean; 3.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 9.6 MiB/s wr, 915 op/s 2024-08-31T06:15:38.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:38 smithi016 ceph-mon[17194]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:15:38.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:38 smithi155 ceph-mon[21742]: pgmap v622: 97 pgs: 97 active+clean; 3.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 9.6 MiB/s wr, 915 op/s 2024-08-31T06:15:38.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:38 smithi155 ceph-mon[21742]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:15:38.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:38 smithi184 ceph-mon[22331]: pgmap v622: 97 pgs: 97 active+clean; 3.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 9.6 MiB/s wr, 915 op/s 2024-08-31T06:15:38.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:38 smithi184 ceph-mon[22331]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:15:40.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:40 smithi155 ceph-mon[21742]: pgmap v623: 97 pgs: 97 active+clean; 3.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 8.4 MiB/s wr, 815 op/s 2024-08-31T06:15:40.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:40 smithi016 ceph-mon[17194]: pgmap v623: 97 pgs: 97 active+clean; 3.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 8.4 MiB/s wr, 815 op/s 2024-08-31T06:15:40.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:40 smithi184 ceph-mon[22331]: pgmap v623: 97 pgs: 97 active+clean; 3.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 8.4 MiB/s wr, 815 op/s 2024-08-31T06:15:41.013 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T06:15:41.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 495.0 s, 1935.6 tps, lat 51.631 ms stddev 60.981 2024-08-31T06:15:41.651 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:15:42.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.509+0000 7f76b5200640 1 -- 172.21.15.16:0/3459298543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76b006f7c0 msgr2=0x7f76b011c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:42.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.509+0000 7f76b5200640 1 --2- 172.21.15.16:0/3459298543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76b006f7c0 0x7f76b011c7e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f769c009900 tx=0x7f769c02a3f0 comp rx=0 tx=0).stop 2024-08-31T06:15:42.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.510+0000 7f76b5200640 1 -- 172.21.15.16:0/3459298543 shutdown_connections 2024-08-31T06:15:42.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.510+0000 7f76b5200640 1 --2- 172.21.15.16:0/3459298543 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76b011cd20 0x7f76b011f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:42.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.510+0000 7f76b5200640 1 --2- 172.21.15.16:0/3459298543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76b006f7c0 0x7f76b011c7e0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:42.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.510+0000 7f76b5200640 1 --2- 172.21.15.16:0/3459298543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76b006ee70 0x7f76b006f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:42.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.510+0000 7f76b5200640 1 -- 172.21.15.16:0/3459298543 >> 172.21.15.16:0/3459298543 conn(0x7f76b010a160 msgr2=0x7f76b010c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:42.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.510+0000 7f76b5200640 1 -- 172.21.15.16:0/3459298543 shutdown_connections 2024-08-31T06:15:42.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.510+0000 7f76b5200640 1 -- 172.21.15.16:0/3459298543 wait complete. 2024-08-31T06:15:42.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.511+0000 7f76b5200640 1 Processor -- start 2024-08-31T06:15:42.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.512+0000 7f76b5200640 1 -- start start 2024-08-31T06:15:42.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.512+0000 7f76b5200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76b006ee70 0x7f76b01a9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:42.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.512+0000 7f76b5200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76b006f7c0 0x7f76b01a9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:42.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.512+0000 7f76b5200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76b011cd20 0x7f76b01ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:42.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.512+0000 7f76b5200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f76b01225f0 con 0x7f76b006f7c0 2024-08-31T06:15:42.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.512+0000 7f76b5200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f76b0122470 con 0x7f76b011cd20 2024-08-31T06:15:42.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.512+0000 7f76b5200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f76b0122770 con 0x7f76b006ee70 2024-08-31T06:15:42.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.513+0000 7f76ae800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76b011cd20 0x7f76b01ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:42.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.513+0000 7f76ad400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76b006f7c0 0x7f76b01a9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:42.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.513+0000 7f76ade00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76b006ee70 0x7f76b01a9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:42.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.513+0000 7f76ade00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76b006ee70 0x7f76b01a9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:33150/0 (socket says 172.21.15.16:33150) 2024-08-31T06:15:42.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.513+0000 7f76ade00640 1 -- 172.21.15.16:0/3952798982 learned_addr learned my addr 172.21.15.16:0/3952798982 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:42.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.514+0000 7f76ade00640 1 -- 172.21.15.16:0/3952798982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76b011cd20 msgr2=0x7f76b01ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:42.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.514+0000 7f76ade00640 1 --2- 172.21.15.16:0/3952798982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76b011cd20 0x7f76b01ae8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:42.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.514+0000 7f76ade00640 1 -- 172.21.15.16:0/3952798982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76b006f7c0 msgr2=0x7f76b01a9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:42.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.514+0000 7f76ade00640 1 --2- 172.21.15.16:0/3952798982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76b006f7c0 0x7f76b01a9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:42.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.514+0000 7f76ade00640 1 -- 172.21.15.16:0/3952798982 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b01aefd0 con 0x7f76b006ee70 2024-08-31T06:15:42.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.514+0000 7f76ade00640 1 --2- 172.21.15.16:0/3952798982 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76b006ee70 0x7f76b01a9060 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f76a000e9f0 tx=0x7f76a000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:42.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.514+0000 7f76a6a00640 1 -- 172.21.15.16:0/3952798982 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a000cd80 con 0x7f76b006ee70 2024-08-31T06:15:42.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.514+0000 7f76a6a00640 1 -- 172.21.15.16:0/3952798982 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76a00108e0 con 0x7f76b006ee70 2024-08-31T06:15:42.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.514+0000 7f76a6a00640 1 -- 172.21.15.16:0/3952798982 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a0018980 con 0x7f76b006ee70 2024-08-31T06:15:42.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.514+0000 7f76b5200640 1 -- 172.21.15.16:0/3952798982 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76b01af2c0 con 0x7f76b006ee70 2024-08-31T06:15:42.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.515+0000 7f76b5200640 1 -- 172.21.15.16:0/3952798982 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76b01af6c0 con 0x7f76b006ee70 2024-08-31T06:15:42.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.520+0000 7f76bbe7d740 1 Processor -- start 2024-08-31T06:15:42.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.521+0000 7f76bbe7d740 1 -- start start 2024-08-31T06:15:42.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.521+0000 7f76bbe7d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632515ec070 0x563251991950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:42.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.521+0000 7f76bbe7d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632515c31b0 0x563251991e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:42.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.521+0000 7f76bbe7d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632515cb390 0x5632519923d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:42.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.521+0000 7f76bbe7d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563251908770 con 0x5632515ec070 2024-08-31T06:15:42.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.521+0000 7f76bbe7d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5632518f7d00 con 0x5632515c31b0 2024-08-31T06:15:42.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.521+0000 7f76bbe7d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5632518fbcb0 con 0x5632515cb390 2024-08-31T06:15:42.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.522+0000 7f76a6a00640 1 -- 172.21.15.16:0/3952798982 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f76a0010430 con 0x7f76b006ee70 2024-08-31T06:15:42.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.522+0000 7f76ad400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632515cb390 0x5632519923d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:42.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.523+0000 7f76ad400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632515cb390 0x5632519923d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:33162/0 (socket says 172.21.15.16:33162) 2024-08-31T06:15:42.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.522+0000 7f76ae800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632515ec070 0x563251991950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:42.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.522+0000 7f76ade00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632515c31b0 0x563251991e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:42.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.523+0000 7f76ad400640 1 -- 172.21.15.16:0/2049072790 learned_addr learned my addr 172.21.15.16:0/2049072790 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:42.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.524+0000 7f76ade00640 1 -- 172.21.15.16:0/2049072790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632515cb390 msgr2=0x5632519923d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:42.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.524+0000 7f76ade00640 1 --2- 172.21.15.16:0/2049072790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632515cb390 0x5632519923d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:42.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.524+0000 7f76ade00640 1 -- 172.21.15.16:0/2049072790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632515ec070 msgr2=0x563251991950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:42.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.524+0000 7f76ade00640 1 --2- 172.21.15.16:0/2049072790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632515ec070 0x563251991950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:42.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.524+0000 7f76ade00640 1 -- 172.21.15.16:0/2049072790 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563251657650 con 0x5632515c31b0 2024-08-31T06:15:42.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.524+0000 7f76a6a00640 1 --2- 172.21.15.16:0/3952798982 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f76940b4da0 0x7f76940b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.526+0000 7f76a6a00640 1 -- 172.21.15.16:0/3952798982 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8470+0+0 (secure 0 0 0) 0x7f76a00e5980 con 0x7f76b006ee70 2024-08-31T06:15:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.526+0000 7f76ade00640 1 --2- 172.21.15.16:0/2049072790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632515c31b0 0x563251991e90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f76a00e30a0 tx=0x7f76a00ed670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f24161a87be33ed6 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.526+0000 7f76ae800640 1 --2- 172.21.15.16:0/3952798982 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f76940b4da0 0x7f76940b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.526+0000 7f769b400640 1 -- 172.21.15.16:0/2049072790 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a00a7f00 con 0x5632515c31b0 2024-08-31T06:15:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.526+0000 7f769b400640 1 -- 172.21.15.16:0/2049072790 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76a0010720 con 0x5632515c31b0 2024-08-31T06:15:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.526+0000 7f769b400640 1 -- 172.21.15.16:0/2049072790 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a00ab7b0 con 0x5632515c31b0 2024-08-31T06:15:42.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.527+0000 7f76bbe7d740 1 -- 172.21.15.16:0/2049072790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632515c31b0 msgr2=0x563251991e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:42.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.527+0000 7f76bbe7d740 1 --2- 172.21.15.16:0/2049072790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632515c31b0 0x563251991e90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f76a00e30a0 tx=0x7f76a00ed670 comp rx=0 tx=0).stop 2024-08-31T06:15:42.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.529+0000 7f76ae800640 1 --2- 172.21.15.16:0/3952798982 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f76940b4da0 0x7f76940b72b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f76a8005fd0 tx=0x7f76a8009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:42.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.529+0000 7f76bbe7d740 1 -- 172.21.15.16:0/2049072790 shutdown_connections 2024-08-31T06:15:42.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.529+0000 7f76bbe7d740 1 --2- 172.21.15.16:0/2049072790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632515cb390 0x5632519923d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:42.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.530+0000 7f76bbe7d740 1 --2- 172.21.15.16:0/2049072790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632515c31b0 0x563251991e90 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:42.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.530+0000 7f76bbe7d740 1 --2- 172.21.15.16:0/2049072790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632515ec070 0x563251991950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:42.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.530+0000 7f76bbe7d740 1 -- 172.21.15.16:0/2049072790 >> 172.21.15.16:0/2049072790 conn(0x5632515e0460 msgr2=0x56325198f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:42.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.530+0000 7f76bbe7d740 1 -- 172.21.15.16:0/2049072790 shutdown_connections 2024-08-31T06:15:42.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.530+0000 7f76bbe7d740 1 -- 172.21.15.16:0/2049072790 wait complete. 2024-08-31T06:15:42.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.531+0000 7f76bbe7d740 1 -- start start 2024-08-31T06:15:42.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.531+0000 7f76bbe7d740 1 Processor -- start 2024-08-31T06:15:42.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.532+0000 7f76bbe7d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632515c31b0 0x56325199d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:42.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.532+0000 7f76bbe7d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632515cb390 0x56325199dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:42.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.532+0000 7f76bbe7d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632515ec070 0x5632519a33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:42.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.532+0000 7f76bbe7d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56325175a700 con 0x5632515ec070 2024-08-31T06:15:42.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.532+0000 7f76bbe7d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56325187ecb0 con 0x5632515c31b0 2024-08-31T06:15:42.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.532+0000 7f76bbe7d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5632519a0130 con 0x5632515cb390 2024-08-31T06:15:42.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.532+0000 7f76bbe7d740 10 client.0 _ll_register_callbacks cb 0x5632518a4430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:15:42.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.533+0000 7f76ade00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632515c31b0 0x56325199d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:42.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.533+0000 7f76ae800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632515ec070 0x5632519a33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:42.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.533+0000 7f76ade00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632515c31b0 0x56325199d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:58918/0 (socket says 172.21.15.16:58918) 2024-08-31T06:15:42.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.533+0000 7f76ade00640 1 -- 172.21.15.16:0/270592920 learned_addr learned my addr 172.21.15.16:0/270592920 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:42.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.533+0000 7f76ad400640 1 --2- 172.21.15.16:0/270592920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632515cb390 0x56325199dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:42.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.533+0000 7f76ade00640 1 -- 172.21.15.16:0/270592920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632515cb390 msgr2=0x56325199dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:42.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.533+0000 7f76ade00640 1 --2- 172.21.15.16:0/270592920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632515cb390 0x56325199dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:42.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.533+0000 7f76ade00640 1 -- 172.21.15.16:0/270592920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632515ec070 msgr2=0x5632519a33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:42.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.533+0000 7f76ade00640 1 --2- 172.21.15.16:0/270592920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632515ec070 0x5632519a33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:42.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.533+0000 7f76ade00640 1 -- 172.21.15.16:0/270592920 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632518bf870 con 0x5632515c31b0 2024-08-31T06:15:42.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.534+0000 7f76ade00640 1 --2- 172.21.15.16:0/270592920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632515c31b0 0x56325199d640 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f76a00a92b0 tx=0x7f76a0018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f761c26988c1fee2 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:42.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.534+0000 7f768fe00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:15:42.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.534+0000 7f768fe00640 1 -- 172.21.15.16:0/270592920 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a00ab990 con 0x5632515c31b0 2024-08-31T06:15:42.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.534+0000 7f768fe00640 1 -- 172.21.15.16:0/270592920 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8470+0+0 (secure 0 0 0) 0x7f76a00ebcb0 con 0x5632515c31b0 2024-08-31T06:15:42.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.534+0000 7f76bbe7d740 1 -- 172.21.15.16:0/270592920 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56325190a6c0 con 0x5632515c31b0 2024-08-31T06:15:42.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.535+0000 7f768fe00640 1 -- 172.21.15.16:0/270592920 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f76a00abe10 con 0x5632515c31b0 2024-08-31T06:15:42.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.535+0000 7f76bbe7d740 10 client.27494 fetch_fsmap learned FSMap version 40 2024-08-31T06:15:42.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.535+0000 7f76bbe7d740 1 -- 172.21.15.16:0/270592920 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5632519ac230 con 0x5632515c31b0 2024-08-31T06:15:42.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.535+0000 7f768fe00640 1 -- 172.21.15.16:0/270592920 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56325190a6c0 con 0x5632515c31b0 2024-08-31T06:15:42.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.536+0000 7f768fe00640 1 -- 172.21.15.16:0/270592920 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 40) ==== 8358+0+0 (secure 0 0 0) 0x7f76a00ac2a0 con 0x5632515c31b0 2024-08-31T06:15:42.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.536+0000 7f76bbe7d740 10 client.27494 fetch_fsmap finished waiting for FSMap version 40 2024-08-31T06:15:42.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.536+0000 7f76bbe7d740 10 client.27494 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:15:42.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.536+0000 7f76bbe7d740 20 client.27494 populate_metadata read hostname 'smithi016' 2024-08-31T06:15:42.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.536+0000 7f76bbe7d740 1 --2- 172.21.15.16:0/270592920 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5632515aa9f0 0x5632519af590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:42.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.536+0000 7f76bbe7d740 4 client.27494 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:15:42.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.536+0000 7f76bbe7d740 1 -- 172.21.15.16:0/270592920 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563251906160 con 0x5632515aa9f0 2024-08-31T06:15:42.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.538+0000 7f76ad400640 1 --2- 172.21.15.16:0/270592920 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5632515aa9f0 0x5632519af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:42.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.539+0000 7f768fe00640 10 client.27494 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:15:42.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.539+0000 7f76ad400640 1 --2- 172.21.15.16:0/270592920 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5632515aa9f0 0x5632519af590 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfc1e1b5e48ed69e server_cookie=d74b71fa57258f3d in_seq=0 out_seq=0 2024-08-31T06:15:42.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.541+0000 7f768fe00640 1 -- 172.21.15.16:0/270592920 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563251906160 con 0x5632515aa9f0 2024-08-31T06:15:42.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.541+0000 7f768fe00640 10 client.27494 handle_command_reply: tid=0 2024-08-31T06:15:42.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.541+0000 7f76bbe7d740 1 client.27494 shutdown 2024-08-31T06:15:42.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.541+0000 7f76bbe7d740 1 -- 172.21.15.16:0/270592920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632515c31b0 msgr2=0x56325199d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:42.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.541+0000 7f76bbe7d740 1 --2- 172.21.15.16:0/270592920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632515c31b0 0x56325199d640 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f76a00a92b0 tx=0x7f76a0018420 comp rx=0 tx=0).stop 2024-08-31T06:15:42.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.542+0000 7f76bbe7d740 1 -- 172.21.15.16:0/270592920 shutdown_connections 2024-08-31T06:15:42.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.542+0000 7f76bbe7d740 1 --2- 172.21.15.16:0/270592920 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5632515aa9f0 0x5632519af590 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:42.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.542+0000 7f76bbe7d740 1 --2- 172.21.15.16:0/270592920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632515cb390 0x56325199dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:42.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.542+0000 7f76bbe7d740 1 --2- 172.21.15.16:0/270592920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632515ec070 0x5632519a33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:42.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.542+0000 7f76bbe7d740 1 --2- 172.21.15.16:0/270592920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632515c31b0 0x56325199d640 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:42.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.542+0000 7f768fe00640 0 client.27494 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:15:42.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.542+0000 7f76bbe7d740 1 -- 172.21.15.16:0/270592920 >> 172.21.15.16:0/270592920 conn(0x5632515e0460 msgr2=0x563251994b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:42.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.542+0000 7f76bbe7d740 1 -- 172.21.15.16:0/270592920 shutdown_connections 2024-08-31T06:15:42.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.542+0000 7f76bbe7d740 1 -- 172.21.15.16:0/270592920 wait complete. 2024-08-31T06:15:42.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.542+0000 7f76bbe7d740 20 client.27494 trim_cache size 0 max 16384 2024-08-31T06:15:42.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.570+0000 7f76bbe7d740 1 Processor -- start 2024-08-31T06:15:42.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.571+0000 7f76bbe7d740 1 -- start start 2024-08-31T06:15:42.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.571+0000 7f76bbe7d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632515c31b0 0x5632519aa1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:42.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.571+0000 7f76bbe7d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632515cb390 0x5632519a3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:42.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.571+0000 7f76bbe7d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632515ec070 0x563251993090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:42.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.571+0000 7f76bbe7d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f76b01c0cc0 con 0x5632515c31b0 2024-08-31T06:15:42.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.571+0000 7f76bbe7d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f76b01be500 con 0x5632515cb390 2024-08-31T06:15:42.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.571+0000 7f76bbe7d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f76b01bc370 con 0x5632515ec070 2024-08-31T06:15:42.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.572+0000 7f76ad400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632515cb390 0x5632519a3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:42.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.572+0000 7f76ad400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632515cb390 0x5632519a3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:58920/0 (socket says 172.21.15.16:58920) 2024-08-31T06:15:42.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.572+0000 7f76ade00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632515c31b0 0x5632519aa1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:42.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.572+0000 7f76ad400640 1 -- 172.21.15.16:0/4127077763 learned_addr learned my addr 172.21.15.16:0/4127077763 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:42.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.572+0000 7f76ae800640 1 --2- 172.21.15.16:0/4127077763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632515ec070 0x563251993090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:42.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.573+0000 7f76ade00640 1 -- 172.21.15.16:0/4127077763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632515ec070 msgr2=0x563251993090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:42.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.573+0000 7f76ade00640 1 --2- 172.21.15.16:0/4127077763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632515ec070 0x563251993090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:42.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.573+0000 7f76ade00640 1 -- 172.21.15.16:0/4127077763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632515cb390 msgr2=0x5632519a3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:42.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.573+0000 7f76ade00640 1 --2- 172.21.15.16:0/4127077763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632515cb390 0x5632519a3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:42.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.574+0000 7f76ade00640 1 -- 172.21.15.16:0/4127077763 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56325199e700 con 0x5632515c31b0 2024-08-31T06:15:42.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.575+0000 7f76ade00640 1 --2- 172.21.15.16:0/4127077763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632515c31b0 0x5632519aa1c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f76a000c6a0 tx=0x7f76a00ed670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=821027f75d21435a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:42.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.576+0000 7f769aa00640 1 -- 172.21.15.16:0/4127077763 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a00c5070 con 0x5632515c31b0 2024-08-31T06:15:42.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.576+0000 7f769aa00640 1 -- 172.21.15.16:0/4127077763 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76a00eb300 con 0x5632515c31b0 2024-08-31T06:15:42.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.576+0000 7f769aa00640 1 -- 172.21.15.16:0/4127077763 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a00a7f00 con 0x5632515c31b0 2024-08-31T06:15:42.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.578+0000 7f76bbe7d740 1 -- 172.21.15.16:0/4127077763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632515c31b0 msgr2=0x5632519aa1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:42.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.578+0000 7f76bbe7d740 1 --2- 172.21.15.16:0/4127077763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632515c31b0 0x5632519aa1c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f76a000c6a0 tx=0x7f76a00ed670 comp rx=0 tx=0).stop 2024-08-31T06:15:42.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.578+0000 7f76bbe7d740 1 -- 172.21.15.16:0/4127077763 shutdown_connections 2024-08-31T06:15:42.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.578+0000 7f76bbe7d740 1 --2- 172.21.15.16:0/4127077763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632515ec070 0x563251993090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:42.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.578+0000 7f76bbe7d740 1 --2- 172.21.15.16:0/4127077763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632515cb390 0x5632519a3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:42.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.578+0000 7f76bbe7d740 1 --2- 172.21.15.16:0/4127077763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632515c31b0 0x5632519aa1c0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:42.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.578+0000 7f76bbe7d740 1 -- 172.21.15.16:0/4127077763 >> 172.21.15.16:0/4127077763 conn(0x5632515e0460 msgr2=0x563251992980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:42.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.579+0000 7f76bbe7d740 1 -- 172.21.15.16:0/4127077763 shutdown_connections 2024-08-31T06:15:42.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.579+0000 7f76bbe7d740 1 -- 172.21.15.16:0/4127077763 wait complete. 2024-08-31T06:15:42.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.579+0000 7f76bbe7d740 1 -- start start 2024-08-31T06:15:42.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.580+0000 7f76bbe7d740 1 Processor -- start 2024-08-31T06:15:42.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.581+0000 7f76bbe7d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632515c31b0 0x5632519a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:42.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.581+0000 7f76bbe7d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632515cb390 0x5632519a82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:42.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.581+0000 7f76bbe7d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632515ec070 0x5632519a8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:42.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.581+0000 7f76bbe7d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f76b01bc9c0 con 0x5632515c31b0 2024-08-31T06:15:42.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.581+0000 7f76bbe7d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f76b01bde30 con 0x5632515ec070 2024-08-31T06:15:42.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.581+0000 7f76bbe7d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f76b01bd090 con 0x5632515cb390 2024-08-31T06:15:42.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.581+0000 7f76ad400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632515cb390 0x5632519a82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:42.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.581+0000 7f76bbe7d740 10 client.0 _ll_register_callbacks cb 0x7f76a00eba70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:15:42.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.581+0000 7f76ad400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632515cb390 0x5632519a82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:33200/0 (socket says 172.21.15.16:33200) 2024-08-31T06:15:42.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.581+0000 7f76ad400640 1 -- 172.21.15.16:0/3272579195 learned_addr learned my addr 172.21.15.16:0/3272579195 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:42.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.581+0000 7f76ae800640 1 --2- 172.21.15.16:0/3272579195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632515ec070 0x5632519a8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:42.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.581+0000 7f76ade00640 1 --2- 172.21.15.16:0/3272579195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632515c31b0 0x5632519a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:42.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.582+0000 7f76ad400640 1 -- 172.21.15.16:0/3272579195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632515ec070 msgr2=0x5632519a8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:42.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.582+0000 7f76ad400640 1 --2- 172.21.15.16:0/3272579195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632515ec070 0x5632519a8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:42.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.582+0000 7f76ad400640 1 -- 172.21.15.16:0/3272579195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632515c31b0 msgr2=0x5632519a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:42.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.582+0000 7f76ad400640 1 --2- 172.21.15.16:0/3272579195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632515c31b0 0x5632519a9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:42.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.582+0000 7f76ad400640 1 -- 172.21.15.16:0/3272579195 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632519a8df0 con 0x5632515cb390 2024-08-31T06:15:42.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.582+0000 7f76ae800640 1 --2- 172.21.15.16:0/3272579195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632515ec070 0x5632519a8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:15:42.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.582+0000 7f76ad400640 1 --2- 172.21.15.16:0/3272579195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632515cb390 0x5632519a82c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f769c005ab0 tx=0x7f769c004130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6afc94602d26cf0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:42.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.583+0000 7f768fe00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:15:42.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.583+0000 7f768fe00640 1 -- 172.21.15.16:0/3272579195 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769c035040 con 0x5632515cb390 2024-08-31T06:15:42.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.583+0000 7f768fe00640 1 -- 172.21.15.16:0/3272579195 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8470+0+0 (secure 0 0 0) 0x7f769c005240 con 0x5632515cb390 2024-08-31T06:15:42.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.583+0000 7f76bbe7d740 1 -- 172.21.15.16:0/3272579195 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76a00eb300 con 0x5632515cb390 2024-08-31T06:15:42.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.583+0000 7f768fe00640 1 -- 172.21.15.16:0/3272579195 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f769c005440 con 0x5632515cb390 2024-08-31T06:15:42.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.584+0000 7f76bbe7d740 10 client.27487 fetch_fsmap learned FSMap version 40 2024-08-31T06:15:42.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.584+0000 7f76bbe7d740 1 -- 172.21.15.16:0/3272579195 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x563251a1aad0 con 0x5632515cb390 2024-08-31T06:15:42.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.584+0000 7f768fe00640 1 -- 172.21.15.16:0/3272579195 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76a00eb300 con 0x5632515cb390 2024-08-31T06:15:42.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.587+0000 7f768fe00640 1 -- 172.21.15.16:0/3272579195 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 40) ==== 8358+0+0 (secure 0 0 0) 0x7f769c0036d0 con 0x5632515cb390 2024-08-31T06:15:42.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.588+0000 7f76bbe7d740 10 client.27487 fetch_fsmap finished waiting for FSMap version 40 2024-08-31T06:15:42.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.588+0000 7f76bbe7d740 10 client.27487 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:15:42.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.588+0000 7f76bbe7d740 20 client.27487 populate_metadata read hostname 'smithi016' 2024-08-31T06:15:42.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.588+0000 7f76bbe7d740 1 --2- 172.21.15.16:0/3272579195 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56325157fda0 0x563251a1d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:42.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.588+0000 7f76bbe7d740 4 client.27487 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T06:15:42.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.588+0000 7f76bbe7d740 1 -- 172.21.15.16:0/3272579195 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5632519038a0 con 0x56325157fda0 2024-08-31T06:15:42.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.589+0000 7f76ade00640 1 --2- 172.21.15.16:0/3272579195 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56325157fda0 0x563251a1d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:42.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.592+0000 7f768fe00640 10 client.27487 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:15:42.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.592+0000 7f76ade00640 1 --2- 172.21.15.16:0/3272579195 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56325157fda0 0x563251a1d2a0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f05f89665f036a9 server_cookie=9fdd51a6f6c47ecc in_seq=0 out_seq=0 2024-08-31T06:15:42.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.592+0000 7f768fe00640 1 -- 172.21.15.16:0/3272579195 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3214 (crc 0 0 0) 0x5632519038a0 con 0x56325157fda0 2024-08-31T06:15:42.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.592+0000 7f768fe00640 10 client.27487 handle_command_reply: tid=0 2024-08-31T06:15:42.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.592+0000 7f76bbe7d740 1 client.27487 shutdown 2024-08-31T06:15:42.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.593+0000 7f76bbe7d740 1 -- 172.21.15.16:0/3272579195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632515cb390 msgr2=0x5632519a82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:42.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.593+0000 7f76bbe7d740 1 --2- 172.21.15.16:0/3272579195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632515cb390 0x5632519a82c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f769c005ab0 tx=0x7f769c004130 comp rx=0 tx=0).stop 2024-08-31T06:15:42.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.593+0000 7f76bbe7d740 1 -- 172.21.15.16:0/3272579195 shutdown_connections 2024-08-31T06:15:42.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.593+0000 7f76bbe7d740 1 --2- 172.21.15.16:0/3272579195 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56325157fda0 0x563251a1d2a0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:42.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.593+0000 7f768fe00640 0 client.27487 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:15:42.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.593+0000 7f76bbe7d740 1 --2- 172.21.15.16:0/3272579195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632515cb390 0x5632519a82c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:42.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.593+0000 7f76bbe7d740 1 --2- 172.21.15.16:0/3272579195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632515ec070 0x5632519a8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:42.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.593+0000 7f76bbe7d740 1 --2- 172.21.15.16:0/3272579195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632515c31b0 0x5632519a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:42.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.593+0000 7f76bbe7d740 1 -- 172.21.15.16:0/3272579195 >> 172.21.15.16:0/3272579195 conn(0x5632515e0460 msgr2=0x563251992960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:42.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.595+0000 7f76bbe7d740 1 -- 172.21.15.16:0/3272579195 shutdown_connections 2024-08-31T06:15:42.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.595+0000 7f76bbe7d740 1 -- 172.21.15.16:0/3272579195 wait complete. 2024-08-31T06:15:42.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.595+0000 7f76bbe7d740 20 client.27487 trim_cache size 0 max 16384 2024-08-31T06:15:42.600 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:15:42.601 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-08-31T06:15:42.601 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {}, 2024-08-31T06:15:42.601 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-08-31T06:15:42.601 INFO:teuthology.orchestra.run.smithi016.stdout: "a64d5301-8558-4e9c-ac9b-8d0ac7057e74": { 2024-08-31T06:15:42.601 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:14:20.338104+0000", 2024-08-31T06:15:42.601 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:15:42.601 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:15:42.601 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:15:42.601 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:15:42.601 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:15:42.601 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:15:42.601 INFO:teuthology.orchestra.run.smithi016.stdout: "e739a16e-8ace-43a8-a1ce-ac27d39ef8a3": { 2024-08-31T06:15:42.601 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:54.084646+0000", 2024-08-31T06:15:42.602 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:15:42.602 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:15:42.602 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:15:42.602 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:15:42.602 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:15:42.602 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:15:42.602 INFO:teuthology.orchestra.run.smithi016.stdout: "91b72a34-4ba9-4126-a414-8b862f8c5f78": { 2024-08-31T06:15:42.602 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:13:28.663237+0000", 2024-08-31T06:15:42.602 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:15:42.602 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:15:42.602 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:15:42.602 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:15:42.602 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:15:42.603 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:15:42.603 INFO:teuthology.orchestra.run.smithi016.stdout: "a10cce76-99ab-46b5-b3b2-07b82899b8f9": { 2024-08-31T06:15:42.603 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:11:43.967546+0000", 2024-08-31T06:15:42.603 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:15:42.603 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:15:42.603 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:15:42.603 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:15:42.603 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:15:42.603 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:15:42.603 INFO:teuthology.orchestra.run.smithi016.stdout: "c1fd6904-2e02-47fa-939a-47349b9fe36b": { 2024-08-31T06:15:42.603 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:06.678691+0000", 2024-08-31T06:15:42.603 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:15:42.603 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:15:42.603 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:15:42.604 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:15:42.604 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:15:42.604 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:15:42.604 INFO:teuthology.orchestra.run.smithi016.stdout: "78d6ef02-8551-45e8-ad60-9627eb573fc1": { 2024-08-31T06:15:42.604 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:09:13.547865+0000", 2024-08-31T06:15:42.604 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:15:42.604 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:15:42.604 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:15:42.604 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:15:42.604 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:15:42.604 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:15:42.604 INFO:teuthology.orchestra.run.smithi016.stdout: "c8455d74-2fe7-4d93-bcf2-19b419d16016": { 2024-08-31T06:15:42.604 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:08:24.035960+0000", 2024-08-31T06:15:42.604 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:15:42.605 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:15:42.605 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:15:42.605 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:15:42.605 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:15:42.605 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:15:42.605 INFO:teuthology.orchestra.run.smithi016.stdout: "0fdcd762-533d-42be-9289-b71c8e3b81b1": { 2024-08-31T06:15:42.605 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:15:06.169112+0000", 2024-08-31T06:15:42.605 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:15:42.605 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:15:42.605 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:15:42.605 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:15:42.605 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:15:42.605 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:15:42.605 INFO:teuthology.orchestra.run.smithi016.stdout: "88d7bdb8-cb1c-45e5-ad29-4cd8da6c6003": { 2024-08-31T06:15:42.606 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:07:39.648942+0000", 2024-08-31T06:15:42.606 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:15:42.606 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:15:42.606 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:15:42.606 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:15:42.606 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:15:42.606 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:15:42.606 INFO:teuthology.orchestra.run.smithi016.stdout: "6a03046e-cdf4-4a14-b9bd-b560b8968467": { 2024-08-31T06:15:42.606 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:12:38.002354+0000", 2024-08-31T06:15:42.606 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:15:42.606 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:15:42.606 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:15:42.606 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:15:42.607 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:15:42.607 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:15:42.607 INFO:teuthology.orchestra.run.smithi016.stdout: "c300b7e8-f3fd-482c-a909-9329bc61e570": { 2024-08-31T06:15:42.607 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:06:49.868046+0000", 2024-08-31T06:15:42.607 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:15:42.607 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:15:42.607 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:15:42.607 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:15:42.607 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:15:42.607 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:15:42.607 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:15:42.607 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:15:42.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.598+0000 7f76b5200640 1 -- 172.21.15.16:0/3952798982 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f76940b4da0 msgr2=0x7f76940b72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:42.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.598+0000 7f76b5200640 1 --2- 172.21.15.16:0/3952798982 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f76940b4da0 0x7f76940b72b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f76a8005fd0 tx=0x7f76a8009500 comp rx=0 tx=0).stop 2024-08-31T06:15:42.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.598+0000 7f76b5200640 1 -- 172.21.15.16:0/3952798982 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76b006ee70 msgr2=0x7f76b01a9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:42.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.598+0000 7f76b5200640 1 --2- 172.21.15.16:0/3952798982 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76b006ee70 0x7f76b01a9060 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f76a000e9f0 tx=0x7f76a000eeb0 comp rx=0 tx=0).stop 2024-08-31T06:15:42.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.599+0000 7f76b5200640 1 -- 172.21.15.16:0/3952798982 shutdown_connections 2024-08-31T06:15:42.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.599+0000 7f76b5200640 1 --2- 172.21.15.16:0/3952798982 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f76940b4da0 0x7f76940b72b0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:42.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.599+0000 7f76b5200640 1 --2- 172.21.15.16:0/3952798982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76b011cd20 0x7f76b01ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:42.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.599+0000 7f76b5200640 1 --2- 172.21.15.16:0/3952798982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76b006f7c0 0x7f76b01a9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:42.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.599+0000 7f76b5200640 1 --2- 172.21.15.16:0/3952798982 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76b006ee70 0x7f76b01a9060 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:42.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.599+0000 7f76b5200640 1 -- 172.21.15.16:0/3952798982 >> 172.21.15.16:0/3952798982 conn(0x7f76b010a160 msgr2=0x7f76b011ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:42.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.599+0000 7f76b5200640 1 -- 172.21.15.16:0/3952798982 shutdown_connections 2024-08-31T06:15:42.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:42.599+0000 7f76b5200640 1 -- 172.21.15.16:0/3952798982 wait complete. 2024-08-31T06:15:42.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:42 smithi016 ceph-mon[17194]: pgmap v624: 97 pgs: 97 active+clean; 3.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 6.3 MiB/s wr, 644 op/s 2024-08-31T06:15:42.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:42 smithi184 ceph-mon[22331]: pgmap v624: 97 pgs: 97 active+clean; 3.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 6.3 MiB/s wr, 644 op/s 2024-08-31T06:15:43.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:42 smithi155 ceph-mon[21742]: pgmap v624: 97 pgs: 97 active+clean; 3.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 6.3 MiB/s wr, 644 op/s 2024-08-31T06:15:43.620 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T06:15:43.620 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:15:43.713 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:43 smithi016 ceph-mon[17194]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:15:43.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:43 smithi184 ceph-mon[22331]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:15:44.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:43 smithi155 ceph-mon[21742]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:15:44.310 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:15:44.761 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:44 smithi016 ceph-mon[17194]: pgmap v625: 97 pgs: 97 active+clean; 3.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 7.3 MiB/s wr, 738 op/s 2024-08-31T06:15:44.761 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:44 smithi016 ceph-mon[17194]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:15:44.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:44 smithi184 ceph-mon[22331]: pgmap v625: 97 pgs: 97 active+clean; 3.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 7.3 MiB/s wr, 738 op/s 2024-08-31T06:15:44.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:44 smithi184 ceph-mon[22331]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:15:45.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:44 smithi155 ceph-mon[21742]: pgmap v625: 97 pgs: 97 active+clean; 3.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 7.3 MiB/s wr, 738 op/s 2024-08-31T06:15:45.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:44 smithi155 ceph-mon[21742]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:15:45.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.351+0000 7fa33ec00640 1 -- 172.21.15.16:0/2151457680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa33810a160 msgr2=0x7fa33810a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:45.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.351+0000 7fa33ec00640 1 --2- 172.21.15.16:0/2151457680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa33810a160 0x7fa33810a640 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa330009ca0 tx=0x7fa33002a8b0 comp rx=0 tx=0).stop 2024-08-31T06:15:45.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.352+0000 7fa33ec00640 1 -- 172.21.15.16:0/2151457680 shutdown_connections 2024-08-31T06:15:45.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.353+0000 7fa33ec00640 1 --2- 172.21.15.16:0/2151457680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa33810bf50 0x7fa3381109c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:45.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.353+0000 7fa33ec00640 1 --2- 172.21.15.16:0/2151457680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa33810b600 0x7fa33810ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:45.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.353+0000 7fa33ec00640 1 --2- 172.21.15.16:0/2151457680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa33810a160 0x7fa33810a640 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:45.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.354+0000 7fa33ec00640 1 -- 172.21.15.16:0/2151457680 >> 172.21.15.16:0/2151457680 conn(0x7fa338070c60 msgr2=0x7fa338071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:45.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.355+0000 7fa33ec00640 1 -- 172.21.15.16:0/2151457680 shutdown_connections 2024-08-31T06:15:45.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.355+0000 7fa33ec00640 1 -- 172.21.15.16:0/2151457680 wait complete. 2024-08-31T06:15:45.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.360+0000 7fa33ec00640 1 Processor -- start 2024-08-31T06:15:45.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.362+0000 7fa33ec00640 1 -- start start 2024-08-31T06:15:45.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.363+0000 7fa33ec00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa33810a160 0x7fa3381a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:45.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.363+0000 7fa33ec00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa33810b600 0x7fa3381a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:45.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.363+0000 7fa33ec00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa33810bf50 0x7fa3381aa620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:45.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.363+0000 7fa33ec00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa33811e3f0 con 0x7fa33810b600 2024-08-31T06:15:45.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.363+0000 7fa33ec00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa33811e270 con 0x7fa33810bf50 2024-08-31T06:15:45.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.363+0000 7fa33ec00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa33811e570 con 0x7fa33810a160 2024-08-31T06:15:45.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.364+0000 7fa337e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa33810a160 0x7fa3381a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:45.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.364+0000 7fa337e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa33810a160 0x7fa3381a4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:33212/0 (socket says 172.21.15.16:33212) 2024-08-31T06:15:45.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.364+0000 7fa337e00640 1 -- 172.21.15.16:0/2568694184 learned_addr learned my addr 172.21.15.16:0/2568694184 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:45.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.365+0000 7fa337e00640 1 -- 172.21.15.16:0/2568694184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa33810bf50 msgr2=0x7fa3381aa620 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-31T06:15:45.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.365+0000 7fa337e00640 1 --2- 172.21.15.16:0/2568694184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa33810bf50 0x7fa3381aa620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:45.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.365+0000 7fa337e00640 1 -- 172.21.15.16:0/2568694184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa33810b600 msgr2=0x7fa3381a5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:45.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.365+0000 7fa337e00640 1 --2- 172.21.15.16:0/2568694184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa33810b600 0x7fa3381a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:45.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.365+0000 7fa337e00640 1 -- 172.21.15.16:0/2568694184 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3381aad50 con 0x7fa33810a160 2024-08-31T06:15:45.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.366+0000 7fa337e00640 1 --2- 172.21.15.16:0/2568694184 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa33810a160 0x7fa3381a4de0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa330005e30 tx=0x7fa330003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:45.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.367+0000 7fa334c00640 1 -- 172.21.15.16:0/2568694184 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa330004240 con 0x7fa33810a160 2024-08-31T06:15:45.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.367+0000 7fa334c00640 1 -- 172.21.15.16:0/2568694184 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa330040070 con 0x7fa33810a160 2024-08-31T06:15:45.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.367+0000 7fa334c00640 1 -- 172.21.15.16:0/2568694184 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa330045bf0 con 0x7fa33810a160 2024-08-31T06:15:45.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.367+0000 7fa33ec00640 1 -- 172.21.15.16:0/2568694184 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3381aafe0 con 0x7fa33810a160 2024-08-31T06:15:45.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.368+0000 7fa33ec00640 1 -- 172.21.15.16:0/2568694184 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3381b17a0 con 0x7fa33810a160 2024-08-31T06:15:45.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.374+0000 7fa33ec00640 1 -- 172.21.15.16:0/2568694184 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa33810ba10 con 0x7fa33810a160 2024-08-31T06:15:45.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.377+0000 7fa334c00640 1 -- 172.21.15.16:0/2568694184 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fa330031440 con 0x7fa33810a160 2024-08-31T06:15:45.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.381+0000 7fa334c00640 1 --2- 172.21.15.16:0/2568694184 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa31c0b4da0 0x7fa31c0b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:45.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.381+0000 7fa334c00640 1 -- 172.21.15.16:0/2568694184 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8470+0+0 (secure 0 0 0) 0x7fa330106d70 con 0x7fa33810a160 2024-08-31T06:15:45.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.381+0000 7fa337400640 1 --2- 172.21.15.16:0/2568694184 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa31c0b4da0 0x7fa31c0b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:45.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.383+0000 7fa337400640 1 --2- 172.21.15.16:0/2568694184 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa31c0b4da0 0x7fa31c0b72b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa3381107b0 tx=0x7fa324006c90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:45.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.387+0000 7fa334c00640 1 -- 172.21.15.16:0/2568694184 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fa33010b070 con 0x7fa33810a160 2024-08-31T06:15:45.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.874+0000 7fa33ec00640 1 -- 172.21.15.16:0/2568694184 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3381b1970 con 0x7fa33810a160 2024-08-31T06:15:45.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.875+0000 7fa334c00640 1 -- 172.21.15.16:0/2568694184 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11921 (secure 0 0 0) 0x7fa33004a060 con 0x7fa33810a160 2024-08-31T06:15:45.878 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:15:45.879 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":42,"btime":"2024-08-31T06:15:43:639920+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:15:43.339970+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24712,24724,24737]},"id":1}]} 2024-08-31T06:15:45.879 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 42 2024-08-31T06:15:45.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.879+0000 7fa33ec00640 1 -- 172.21.15.16:0/2568694184 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa31c0b4da0 msgr2=0x7fa31c0b72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:45.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.879+0000 7fa33ec00640 1 --2- 172.21.15.16:0/2568694184 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa31c0b4da0 0x7fa31c0b72b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa3381107b0 tx=0x7fa324006c90 comp rx=0 tx=0).stop 2024-08-31T06:15:45.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.879+0000 7fa33ec00640 1 -- 172.21.15.16:0/2568694184 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa33810a160 msgr2=0x7fa3381a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:45.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.879+0000 7fa33ec00640 1 --2- 172.21.15.16:0/2568694184 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa33810a160 0x7fa3381a4de0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa330005e30 tx=0x7fa330003ba0 comp rx=0 tx=0).stop 2024-08-31T06:15:45.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.879+0000 7fa33ec00640 1 -- 172.21.15.16:0/2568694184 shutdown_connections 2024-08-31T06:15:45.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.879+0000 7fa33ec00640 1 --2- 172.21.15.16:0/2568694184 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa31c0b4da0 0x7fa31c0b72b0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:45.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.879+0000 7fa33ec00640 1 --2- 172.21.15.16:0/2568694184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa33810bf50 0x7fa3381aa620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:45.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.879+0000 7fa33ec00640 1 --2- 172.21.15.16:0/2568694184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa33810b600 0x7fa3381a5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:45.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.879+0000 7fa33ec00640 1 --2- 172.21.15.16:0/2568694184 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa33810a160 0x7fa3381a4de0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:45.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.879+0000 7fa33ec00640 1 -- 172.21.15.16:0/2568694184 >> 172.21.15.16:0/2568694184 conn(0x7fa338070c60 msgr2=0x7fa338073bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:45.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.880+0000 7fa33ec00640 1 -- 172.21.15.16:0/2568694184 shutdown_connections 2024-08-31T06:15:45.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:45.880+0000 7fa33ec00640 1 -- 172.21.15.16:0/2568694184 wait complete. 2024-08-31T06:15:46.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 500.0 s, 2668.3 tps, lat 37.761 ms stddev 41.005 2024-08-31T06:15:46.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:46 smithi016 ceph-mon[17194]: pgmap v626: 97 pgs: 97 active+clean; 3.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 8.4 MiB/s wr, 838 op/s 2024-08-31T06:15:46.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:46 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/2568694184' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:15:46.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:46 smithi184 ceph-mon[22331]: pgmap v626: 97 pgs: 97 active+clean; 3.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 8.4 MiB/s wr, 838 op/s 2024-08-31T06:15:46.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:46 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/2568694184' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:15:46.987 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T06:15:47.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:46 smithi155 ceph-mon[21742]: pgmap v626: 97 pgs: 97 active+clean; 3.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 8.4 MiB/s wr, 838 op/s 2024-08-31T06:15:47.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:46 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/2568694184' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:15:47.621 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:15:48.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.443+0000 7f96fa600640 1 -- 172.21.15.16:0/2133236058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96ec0a26c0 msgr2=0x7f96ec0a4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:48.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.443+0000 7f96fa600640 1 --2- 172.21.15.16:0/2133236058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96ec0a26c0 0x7f96ec0a4ae0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f96f4066220 tx=0x7f96f4097ea0 comp rx=0 tx=0).stop 2024-08-31T06:15:48.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.444+0000 7f96fa600640 1 -- 172.21.15.16:0/2133236058 shutdown_connections 2024-08-31T06:15:48.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.444+0000 7f96fa600640 1 --2- 172.21.15.16:0/2133236058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96ec0075f0 0x7f96ec0a2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:48.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.444+0000 7f96fa600640 1 --2- 172.21.15.16:0/2133236058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96ec0a26c0 0x7f96ec0a4ae0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:48.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.444+0000 7f96fa600640 1 --2- 172.21.15.16:0/2133236058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96ec006ca0 0x7f96ec0070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:48.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.444+0000 7f96fa600640 1 -- 172.21.15.16:0/2133236058 >> 172.21.15.16:0/2133236058 conn(0x7f96ec091b50 msgr2=0x7f96ec093fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:48.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.444+0000 7f96fa600640 1 -- 172.21.15.16:0/2133236058 shutdown_connections 2024-08-31T06:15:48.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.444+0000 7f96fa600640 1 -- 172.21.15.16:0/2133236058 wait complete. 2024-08-31T06:15:48.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.445+0000 7f96fa600640 1 Processor -- start 2024-08-31T06:15:48.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.446+0000 7f96fa600640 1 -- start start 2024-08-31T06:15:48.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.446+0000 7f96fa600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96ec006ca0 0x7f96ec09a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:48.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.446+0000 7f96fa600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96ec0075f0 0x7f96ec09a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:48.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.446+0000 7f96fa600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96ec0a26c0 0x7f96ec09ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:48.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.446+0000 7f96fa600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec0a81a0 con 0x7f96ec0a26c0 2024-08-31T06:15:48.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.446+0000 7f96fa600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec0a8020 con 0x7f96ec006ca0 2024-08-31T06:15:48.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.446+0000 7f96fa600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec0a8320 con 0x7f96ec0075f0 2024-08-31T06:15:48.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.447+0000 7f96f3e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96ec0075f0 0x7f96ec09a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:48.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.447+0000 7f96f3e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96ec0075f0 0x7f96ec09a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:48812/0 (socket says 172.21.15.16:48812) 2024-08-31T06:15:48.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.447+0000 7f96f3e00640 1 -- 172.21.15.16:0/3556955826 learned_addr learned my addr 172.21.15.16:0/3556955826 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:48.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.447+0000 7f96f9200640 1 --2- 172.21.15.16:0/3556955826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96ec006ca0 0x7f96ec09a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:48.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.447+0000 7f96f3e00640 1 -- 172.21.15.16:0/3556955826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96ec006ca0 msgr2=0x7f96ec09a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:48.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.448+0000 7f96f3e00640 1 --2- 172.21.15.16:0/3556955826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96ec006ca0 0x7f96ec09a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:48.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.448+0000 7f96f3e00640 1 -- 172.21.15.16:0/3556955826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96ec0a26c0 msgr2=0x7f96ec09ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:48.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.448+0000 7f96f3e00640 1 --2- 172.21.15.16:0/3556955826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96ec0a26c0 0x7f96ec09ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:48.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.448+0000 7f96f3e00640 1 -- 172.21.15.16:0/3556955826 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96ec13efe0 con 0x7f96ec0075f0 2024-08-31T06:15:48.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.448+0000 7f96f3e00640 1 --2- 172.21.15.16:0/3556955826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96ec0075f0 0x7f96ec09a5f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f96e000e9f0 tx=0x7f96e000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:48.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.449+0000 7f96f1600640 1 -- 172.21.15.16:0/3556955826 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e000ccc0 con 0x7f96ec0075f0 2024-08-31T06:15:48.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.449+0000 7f96f1600640 1 -- 172.21.15.16:0/3556955826 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96e00108e0 con 0x7f96ec0075f0 2024-08-31T06:15:48.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.449+0000 7f96f1600640 1 -- 172.21.15.16:0/3556955826 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e0018860 con 0x7f96ec0075f0 2024-08-31T06:15:48.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.449+0000 7f96fa600640 1 -- 172.21.15.16:0/3556955826 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96ec13f2d0 con 0x7f96ec0075f0 2024-08-31T06:15:48.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.449+0000 7f96fa600640 1 -- 172.21.15.16:0/3556955826 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96ec13f7c0 con 0x7f96ec0075f0 2024-08-31T06:15:48.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.454+0000 7f9702877740 1 Processor -- start 2024-08-31T06:15:48.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.454+0000 7f9702877740 1 -- start start 2024-08-31T06:15:48.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.455+0000 7f9702877740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4baa8070 0x564b4be4db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:48.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.455+0000 7f9702877740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b4ba7f1b0 0x564b4be4e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:48.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.455+0000 7f96f9c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4baa8070 0x564b4be4db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:48.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.455+0000 7f96f9c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4baa8070 0x564b4be4db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:41220/0 (socket says 172.21.15.16:41220) 2024-08-31T06:15:48.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.456+0000 7f9702877740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4ba87390 0x564b4be4e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:48.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.456+0000 7f9702877740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564b4bdc47b0 con 0x564b4ba87390 2024-08-31T06:15:48.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.456+0000 7f9702877740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x564b4bdb3d40 con 0x564b4baa8070 2024-08-31T06:15:48.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.456+0000 7f9702877740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564b4bdb7cf0 con 0x564b4ba7f1b0 2024-08-31T06:15:48.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.456+0000 7f96f9c00640 1 -- 172.21.15.16:0/2621855768 learned_addr learned my addr 172.21.15.16:0/2621855768 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.456+0000 7f96f1600640 1 -- 172.21.15.16:0/3556955826 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f96e0018a00 con 0x7f96ec0075f0 2024-08-31T06:15:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.456+0000 7f96f9c00640 1 -- 172.21.15.16:0/2621855768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b4ba7f1b0 msgr2=0x564b4be4e0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T06:15:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.456+0000 7f96f9c00640 1 --2- 172.21.15.16:0/2621855768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b4ba7f1b0 0x564b4be4e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.456+0000 7f96f9c00640 1 -- 172.21.15.16:0/2621855768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4ba87390 msgr2=0x564b4be4e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.456+0000 7f96f9c00640 1 --2- 172.21.15.16:0/2621855768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4ba87390 0x564b4be4e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.456+0000 7f96f9c00640 1 -- 172.21.15.16:0/2621855768 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b4bb13650 con 0x564b4baa8070 2024-08-31T06:15:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.457+0000 7f96f9c00640 1 --2- 172.21.15.16:0/2621855768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4baa8070 0x564b4be4db60 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f96f40526b0 tx=0x7f96f4069140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85a878c3d881cb0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.457+0000 7f96e6000640 1 -- 172.21.15.16:0/2621855768 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f409f480 con 0x564b4baa8070 2024-08-31T06:15:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.457+0000 7f96e6000640 1 -- 172.21.15.16:0/2621855768 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96f409c040 con 0x564b4baa8070 2024-08-31T06:15:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.457+0000 7f96e6000640 1 -- 172.21.15.16:0/2621855768 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f40642b0 con 0x564b4baa8070 2024-08-31T06:15:48.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.458+0000 7f96f1600640 1 --2- 172.21.15.16:0/3556955826 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f96dc0b4d20 0x7f96dc0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:48.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.458+0000 7f96f1600640 1 -- 172.21.15.16:0/3556955826 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8470+0+0 (secure 0 0 0) 0x7f96e00e5970 con 0x7f96ec0075f0 2024-08-31T06:15:48.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.459+0000 7f96f9200640 1 --2- 172.21.15.16:0/3556955826 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f96dc0b4d20 0x7f96dc0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:48.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.459+0000 7f9702877740 1 -- 172.21.15.16:0/2621855768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4baa8070 msgr2=0x564b4be4db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.459+0000 7f9702877740 1 --2- 172.21.15.16:0/2621855768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4baa8070 0x564b4be4db60 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f96f40526b0 tx=0x7f96f4069140 comp rx=0 tx=0).stop 2024-08-31T06:15:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.459+0000 7f9702877740 1 -- 172.21.15.16:0/2621855768 shutdown_connections 2024-08-31T06:15:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.459+0000 7f9702877740 1 --2- 172.21.15.16:0/2621855768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b4ba7f1b0 0x564b4be4e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.459+0000 7f9702877740 1 --2- 172.21.15.16:0/2621855768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4ba87390 0x564b4be4e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.459+0000 7f9702877740 1 --2- 172.21.15.16:0/2621855768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4baa8070 0x564b4be4db60 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.459+0000 7f9702877740 1 -- 172.21.15.16:0/2621855768 >> 172.21.15.16:0/2621855768 conn(0x564b4ba9c460 msgr2=0x564b4be4ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.459+0000 7f96f9200640 1 --2- 172.21.15.16:0/3556955826 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f96dc0b4d20 0x7f96dc0b7230 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x564b4be4c7e0 tx=0x7f96e8005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:48.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.460+0000 7f9702877740 1 -- 172.21.15.16:0/2621855768 shutdown_connections 2024-08-31T06:15:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.460+0000 7f9702877740 1 -- 172.21.15.16:0/2621855768 wait complete. 2024-08-31T06:15:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.461+0000 7f9702877740 1 -- start start 2024-08-31T06:15:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.461+0000 7f9702877740 1 Processor -- start 2024-08-31T06:15:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.463+0000 7f9702877740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4ba7f1b0 0x564b4be59830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.463+0000 7f9702877740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4ba87390 0x564b4be59e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.463+0000 7f9702877740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b4baa8070 0x564b4be65630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.463+0000 7f9702877740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564b4bc16700 con 0x564b4ba7f1b0 2024-08-31T06:15:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.463+0000 7f9702877740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x564b4bd3acf0 con 0x564b4ba87390 2024-08-31T06:15:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.463+0000 7f9702877740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564b4be5c2e0 con 0x564b4baa8070 2024-08-31T06:15:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.464+0000 7f9702877740 10 client.0 _ll_register_callbacks cb 0x564b4bd60470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:15:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.464+0000 7f96f3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4ba87390 0x564b4be59e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.464+0000 7f96f3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4ba87390 0x564b4be59e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:41222/0 (socket says 172.21.15.16:41222) 2024-08-31T06:15:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.464+0000 7f96f3e00640 1 -- 172.21.15.16:0/3758250529 learned_addr learned my addr 172.21.15.16:0/3758250529 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.464+0000 7f96f9c00640 1 --2- 172.21.15.16:0/3758250529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b4baa8070 0x564b4be65630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.464+0000 7f96f9200640 1 --2- 172.21.15.16:0/3758250529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4ba7f1b0 0x564b4be59830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.464+0000 7f96f3e00640 1 -- 172.21.15.16:0/3758250529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b4baa8070 msgr2=0x564b4be65630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.464+0000 7f96f3e00640 1 --2- 172.21.15.16:0/3758250529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b4baa8070 0x564b4be65630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.464+0000 7f96f3e00640 1 -- 172.21.15.16:0/3758250529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4ba7f1b0 msgr2=0x564b4be59830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.464+0000 7f96f3e00640 1 --2- 172.21.15.16:0/3758250529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4ba7f1b0 0x564b4be59830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.464+0000 7f96f3e00640 1 -- 172.21.15.16:0/3758250529 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b4bd7b8b0 con 0x564b4ba87390 2024-08-31T06:15:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.465+0000 7f96f3e00640 1 --2- 172.21.15.16:0/3758250529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4ba87390 0x564b4be59e10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f96e00abfc0 tx=0x7f96e00abb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7080ed697ec358a4 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.465+0000 7f96daa00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:15:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.465+0000 7f96daa00640 1 -- 172.21.15.16:0/3758250529 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e00ac030 con 0x564b4ba87390 2024-08-31T06:15:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.465+0000 7f96daa00640 1 -- 172.21.15.16:0/3758250529 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8470+0+0 (secure 0 0 0) 0x7f96e0010430 con 0x564b4ba87390 2024-08-31T06:15:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.466+0000 7f9702877740 1 -- 172.21.15.16:0/3758250529 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96f409c040 con 0x564b4ba87390 2024-08-31T06:15:48.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.467+0000 7f96daa00640 1 -- 172.21.15.16:0/3758250529 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f96e00b08a0 con 0x564b4ba87390 2024-08-31T06:15:48.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.468+0000 7f9702877740 10 client.27515 fetch_fsmap learned FSMap version 42 2024-08-31T06:15:48.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.468+0000 7f9702877740 1 -- 172.21.15.16:0/3758250529 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x564b4be68400 con 0x564b4ba87390 2024-08-31T06:15:48.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.468+0000 7f96daa00640 1 -- 172.21.15.16:0/3758250529 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96f409c040 con 0x564b4ba87390 2024-08-31T06:15:48.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.468+0000 7f96daa00640 1 -- 172.21.15.16:0/3758250529 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7f96e00a79a0 con 0x564b4ba87390 2024-08-31T06:15:48.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.468+0000 7f9702877740 10 client.27515 fetch_fsmap finished waiting for FSMap version 42 2024-08-31T06:15:48.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.468+0000 7f9702877740 10 client.27515 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T06:15:48.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.468+0000 7f9702877740 20 client.27515 populate_metadata read hostname 'smithi016' 2024-08-31T06:15:48.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.468+0000 7f9702877740 1 --2- 172.21.15.16:0/3758250529 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x564b4ba669f0 0x564b4be6b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:48.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.468+0000 7f9702877740 4 client.27515 mds_command: new command op to 24712 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:15:48.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.468+0000 7f9702877740 1 -- 172.21.15.16:0/3758250529 --> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b4bdc6700 con 0x564b4ba669f0 2024-08-31T06:15:48.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.469+0000 7f96f9200640 1 --2- 172.21.15.16:0/3758250529 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x564b4ba669f0 0x564b4be6b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:48.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.470+0000 7f96f9200640 1 --2- 172.21.15.16:0/3758250529 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x564b4ba669f0 0x564b4be6b8d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9fc52abb13debc21 server_cookie=97187511b0fc73bc in_seq=0 out_seq=0 2024-08-31T06:15:48.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.470+0000 7f96daa00640 10 client.27515 ms_handle_connect on v2:172.21.15.184:6838/4011051479 2024-08-31T06:15:48.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.470+0000 7f96daa00640 1 -- 172.21.15.16:0/3758250529 <== mds.2 v2:172.21.15.184:6838/4011051479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564b4bdc6700 con 0x564b4ba669f0 2024-08-31T06:15:48.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.470+0000 7f96daa00640 10 client.27515 handle_command_reply: tid=0 2024-08-31T06:15:48.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.471+0000 7f9702877740 1 client.27515 shutdown 2024-08-31T06:15:48.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.471+0000 7f9702877740 1 -- 172.21.15.16:0/3758250529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4ba87390 msgr2=0x564b4be59e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:48.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.471+0000 7f9702877740 1 --2- 172.21.15.16:0/3758250529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4ba87390 0x564b4be59e10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f96e00abfc0 tx=0x7f96e00abb40 comp rx=0 tx=0).stop 2024-08-31T06:15:48.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.471+0000 7f9702877740 1 -- 172.21.15.16:0/3758250529 shutdown_connections 2024-08-31T06:15:48.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.471+0000 7f9702877740 1 --2- 172.21.15.16:0/3758250529 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x564b4ba669f0 0x564b4be6b8d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:48.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.471+0000 7f9702877740 1 --2- 172.21.15.16:0/3758250529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b4baa8070 0x564b4be65630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:48.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.471+0000 7f9702877740 1 --2- 172.21.15.16:0/3758250529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4ba87390 0x564b4be59e10 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:48.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.471+0000 7f96daa00640 0 client.27515 ms_handle_reset on v2:172.21.15.184:6838/4011051479 2024-08-31T06:15:48.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.471+0000 7f9702877740 1 --2- 172.21.15.16:0/3758250529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4ba7f1b0 0x564b4be59830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:48.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.471+0000 7f9702877740 1 -- 172.21.15.16:0/3758250529 >> 172.21.15.16:0/3758250529 conn(0x564b4ba9c460 msgr2=0x564b4be4eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:48.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.472+0000 7f9702877740 1 -- 172.21.15.16:0/3758250529 shutdown_connections 2024-08-31T06:15:48.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.472+0000 7f9702877740 1 -- 172.21.15.16:0/3758250529 wait complete. 2024-08-31T06:15:48.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.472+0000 7f9702877740 20 client.27515 trim_cache size 0 max 16384 2024-08-31T06:15:48.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.493+0000 7f9702877740 1 Processor -- start 2024-08-31T06:15:48.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.493+0000 7f9702877740 1 -- start start 2024-08-31T06:15:48.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.493+0000 7f9702877740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4ba7f1b0 0x564b4be4a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:48.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.493+0000 7f9702877740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4ba87390 0x564b4be4ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:48.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.493+0000 7f9702877740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b4baa8070 0x564b4be57eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:48.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.493+0000 7f9702877740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec14ac00 con 0x564b4ba7f1b0 2024-08-31T06:15:48.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.493+0000 7f9702877740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec148440 con 0x564b4ba87390 2024-08-31T06:15:48.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.493+0000 7f9702877740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec1462b0 con 0x564b4baa8070 2024-08-31T06:15:48.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.494+0000 7f96f3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4ba87390 0x564b4be4ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:48.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.494+0000 7f96f3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4ba87390 0x564b4be4ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:41232/0 (socket says 172.21.15.16:41232) 2024-08-31T06:15:48.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.494+0000 7f96f3e00640 1 -- 172.21.15.16:0/3361803648 learned_addr learned my addr 172.21.15.16:0/3361803648 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:48.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.494+0000 7f96f9200640 1 --2- 172.21.15.16:0/3361803648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4ba7f1b0 0x564b4be4a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:48.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.494+0000 7f96f9c00640 1 --2- 172.21.15.16:0/3361803648 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b4baa8070 0x564b4be57eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:48.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.494+0000 7f96f3e00640 1 -- 172.21.15.16:0/3361803648 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b4baa8070 msgr2=0x564b4be57eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:48.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.494+0000 7f96f3e00640 1 --2- 172.21.15.16:0/3361803648 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b4baa8070 0x564b4be57eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:48.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.494+0000 7f96f3e00640 1 -- 172.21.15.16:0/3361803648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4ba7f1b0 msgr2=0x564b4be4a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:48.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.494+0000 7f96f3e00640 1 --2- 172.21.15.16:0/3361803648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4ba7f1b0 0x564b4be4a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:48.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.494+0000 7f96f3e00640 1 -- 172.21.15.16:0/3361803648 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b4be5a8f0 con 0x564b4ba87390 2024-08-31T06:15:48.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.495+0000 7f96f3e00640 1 --2- 172.21.15.16:0/3361803648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4ba87390 0x564b4be4ac30 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f96e000c6a0 tx=0x7f96e00096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e811ae9f217b9724 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:48.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.495+0000 7f96e5600640 1 -- 172.21.15.16:0/3361803648 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e00c5070 con 0x564b4ba87390 2024-08-31T06:15:48.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.495+0000 7f96e5600640 1 -- 172.21.15.16:0/3361803648 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96e00a79a0 con 0x564b4ba87390 2024-08-31T06:15:48.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.495+0000 7f96e5600640 1 -- 172.21.15.16:0/3361803648 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e00aad00 con 0x564b4ba87390 2024-08-31T06:15:48.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.495+0000 7f96f9c00640 1 --2- 172.21.15.16:0/3361803648 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b4baa8070 0x564b4be57eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:15:48.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.496+0000 7f9702877740 1 -- 172.21.15.16:0/3361803648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4ba87390 msgr2=0x564b4be4ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:48.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.496+0000 7f9702877740 1 --2- 172.21.15.16:0/3361803648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4ba87390 0x564b4be4ac30 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f96e000c6a0 tx=0x7f96e00096a0 comp rx=0 tx=0).stop 2024-08-31T06:15:48.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.497+0000 7f9702877740 1 -- 172.21.15.16:0/3361803648 shutdown_connections 2024-08-31T06:15:48.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.497+0000 7f9702877740 1 --2- 172.21.15.16:0/3361803648 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b4baa8070 0x564b4be57eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:48.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.497+0000 7f9702877740 1 --2- 172.21.15.16:0/3361803648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4ba87390 0x564b4be4ac30 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:48.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.497+0000 7f9702877740 1 --2- 172.21.15.16:0/3361803648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4ba7f1b0 0x564b4be4a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:48.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.497+0000 7f9702877740 1 -- 172.21.15.16:0/3361803648 >> 172.21.15.16:0/3361803648 conn(0x564b4ba9c460 msgr2=0x564b4be4b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:48.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.498+0000 7f9702877740 1 -- 172.21.15.16:0/3361803648 shutdown_connections 2024-08-31T06:15:48.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.498+0000 7f9702877740 1 -- 172.21.15.16:0/3361803648 wait complete. 2024-08-31T06:15:48.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.499+0000 7f9702877740 1 -- start start 2024-08-31T06:15:48.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.499+0000 7f9702877740 1 Processor -- start 2024-08-31T06:15:48.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.500+0000 7f9702877740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4ba7f1b0 0x564b4be615a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:48.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.500+0000 7f9702877740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b4ba87390 0x564b4be61ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:48.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.500+0000 7f9702877740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4baa8070 0x564b4be5c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:48.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.500+0000 7f9702877740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec146900 con 0x564b4baa8070 2024-08-31T06:15:48.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.500+0000 7f9702877740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec147d70 con 0x564b4ba7f1b0 2024-08-31T06:15:48.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.500+0000 7f9702877740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec146fd0 con 0x564b4ba87390 2024-08-31T06:15:48.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.500+0000 7f9702877740 10 client.0 _ll_register_callbacks cb 0x7f96e00ab980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:15:48.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.500+0000 7f96f9200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4ba7f1b0 0x564b4be615a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:48.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.500+0000 7f96f9200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4ba7f1b0 0x564b4be615a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:41234/0 (socket says 172.21.15.16:41234) 2024-08-31T06:15:48.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.500+0000 7f96f9200640 1 -- 172.21.15.16:0/674827021 learned_addr learned my addr 172.21.15.16:0/674827021 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:48.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.500+0000 7f96f9c00640 1 --2- 172.21.15.16:0/674827021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4baa8070 0x564b4be5c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:48.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.500+0000 7f96f3e00640 1 --2- 172.21.15.16:0/674827021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b4ba87390 0x564b4be61ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:48.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.501+0000 7f96f3e00640 1 -- 172.21.15.16:0/674827021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4ba7f1b0 msgr2=0x564b4be615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:48.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.501+0000 7f96f3e00640 1 --2- 172.21.15.16:0/674827021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4ba7f1b0 0x564b4be615a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:48.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.501+0000 7f96f3e00640 1 -- 172.21.15.16:0/674827021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4baa8070 msgr2=0x564b4be5c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:48.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.501+0000 7f96f3e00640 1 --2- 172.21.15.16:0/674827021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4baa8070 0x564b4be5c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:48.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.501+0000 7f96f3e00640 1 -- 172.21.15.16:0/674827021 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b4be62150 con 0x564b4ba87390 2024-08-31T06:15:48.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.501+0000 7f96f9200640 1 --2- 172.21.15.16:0/674827021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4ba7f1b0 0x564b4be615a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:15:48.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.501+0000 7f96f3e00640 1 --2- 172.21.15.16:0/674827021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b4ba87390 0x564b4be61ae0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f96e00abf70 tx=0x7f96e0009740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6533d1324508be81 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:48.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.501+0000 7f96daa00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:15:48.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.502+0000 7f96daa00640 1 -- 172.21.15.16:0/674827021 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e00c5020 con 0x564b4ba87390 2024-08-31T06:15:48.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.502+0000 7f96daa00640 1 -- 172.21.15.16:0/674827021 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8470+0+0 (secure 0 0 0) 0x7f96e0010640 con 0x564b4ba87390 2024-08-31T06:15:48.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.502+0000 7f9702877740 1 -- 172.21.15.16:0/674827021 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96e00a79a0 con 0x564b4ba87390 2024-08-31T06:15:48.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.502+0000 7f96daa00640 1 -- 172.21.15.16:0/674827021 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f96e00ab290 con 0x564b4ba87390 2024-08-31T06:15:48.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.503+0000 7f9702877740 10 client.27505 fetch_fsmap learned FSMap version 42 2024-08-31T06:15:48.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.503+0000 7f9702877740 1 -- 172.21.15.16:0/674827021 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x564b4bed6580 con 0x564b4ba87390 2024-08-31T06:15:48.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.503+0000 7f96daa00640 1 -- 172.21.15.16:0/674827021 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96e00a79a0 con 0x564b4ba87390 2024-08-31T06:15:48.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.503+0000 7f96daa00640 1 -- 172.21.15.16:0/674827021 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7f96e00930f0 con 0x564b4ba87390 2024-08-31T06:15:48.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.503+0000 7f9702877740 10 client.27505 fetch_fsmap finished waiting for FSMap version 42 2024-08-31T06:15:48.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.504+0000 7f9702877740 10 client.27505 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T06:15:48.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.504+0000 7f9702877740 20 client.27505 populate_metadata read hostname 'smithi016' 2024-08-31T06:15:48.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.504+0000 7f9702877740 1 --2- 172.21.15.16:0/674827021 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x564b4ba3bda0 0x564b4bed88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:48.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.504+0000 7f9702877740 4 client.27505 mds_command: new command op to 24712 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:15:48.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.504+0000 7f9702877740 1 -- 172.21.15.16:0/674827021 --> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564b4bdc21a0 con 0x564b4ba3bda0 2024-08-31T06:15:48.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.504+0000 7f96f9200640 1 --2- 172.21.15.16:0/674827021 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x564b4ba3bda0 0x564b4bed88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:48.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.505+0000 7f96f9200640 1 --2- 172.21.15.16:0/674827021 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x564b4ba3bda0 0x564b4bed88b0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7fe5bae1870b76c4 server_cookie=76ef5a075c439ee9 in_seq=0 out_seq=0 2024-08-31T06:15:48.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.505+0000 7f96daa00640 10 client.27505 ms_handle_connect on v2:172.21.15.184:6838/4011051479 2024-08-31T06:15:48.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.505+0000 7f96daa00640 1 -- 172.21.15.16:0/674827021 <== mds.2 v2:172.21.15.184:6838/4011051479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564b4bdc21a0 con 0x564b4ba3bda0 2024-08-31T06:15:48.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.505+0000 7f96daa00640 10 client.27505 handle_command_reply: tid=0 2024-08-31T06:15:48.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.505+0000 7f9702877740 1 client.27505 shutdown 2024-08-31T06:15:48.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.506+0000 7f9702877740 1 -- 172.21.15.16:0/674827021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b4ba87390 msgr2=0x564b4be61ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:48.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.506+0000 7f9702877740 1 --2- 172.21.15.16:0/674827021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b4ba87390 0x564b4be61ae0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f96e00abf70 tx=0x7f96e0009740 comp rx=0 tx=0).stop 2024-08-31T06:15:48.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.506+0000 7f9702877740 1 -- 172.21.15.16:0/674827021 shutdown_connections 2024-08-31T06:15:48.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.506+0000 7f9702877740 1 --2- 172.21.15.16:0/674827021 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x564b4ba3bda0 0x564b4bed88b0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:48.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.506+0000 7f9702877740 1 --2- 172.21.15.16:0/674827021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b4ba87390 0x564b4be61ae0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:48.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.506+0000 7f9702877740 1 --2- 172.21.15.16:0/674827021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4baa8070 0x564b4be5c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:48.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.506+0000 7f9702877740 1 --2- 172.21.15.16:0/674827021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4ba7f1b0 0x564b4be615a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:48.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.506+0000 7f96daa00640 0 client.27505 ms_handle_reset on v2:172.21.15.184:6838/4011051479 2024-08-31T06:15:48.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.506+0000 7f9702877740 1 -- 172.21.15.16:0/674827021 >> 172.21.15.16:0/674827021 conn(0x564b4ba9c460 msgr2=0x564b4be59830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:48.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.506+0000 7f9702877740 1 -- 172.21.15.16:0/674827021 shutdown_connections 2024-08-31T06:15:48.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.506+0000 7f9702877740 1 -- 172.21.15.16:0/674827021 wait complete. 2024-08-31T06:15:48.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.506+0000 7f9702877740 20 client.27505 trim_cache size 0 max 16384 2024-08-31T06:15:48.509 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:15:48.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.510+0000 7f96fa600640 1 -- 172.21.15.16:0/3556955826 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f96dc0b4d20 msgr2=0x7f96dc0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:48.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.510+0000 7f96fa600640 1 --2- 172.21.15.16:0/3556955826 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f96dc0b4d20 0x7f96dc0b7230 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x564b4be4c7e0 tx=0x7f96e8005f90 comp rx=0 tx=0).stop 2024-08-31T06:15:48.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.510+0000 7f96fa600640 1 -- 172.21.15.16:0/3556955826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96ec0075f0 msgr2=0x7f96ec09a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:48.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.510+0000 7f96fa600640 1 --2- 172.21.15.16:0/3556955826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96ec0075f0 0x7f96ec09a5f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f96e000e9f0 tx=0x7f96e000eeb0 comp rx=0 tx=0).stop 2024-08-31T06:15:48.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.510+0000 7f96fa600640 1 -- 172.21.15.16:0/3556955826 shutdown_connections 2024-08-31T06:15:48.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.510+0000 7f96fa600640 1 --2- 172.21.15.16:0/3556955826 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f96dc0b4d20 0x7f96dc0b7230 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:48.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.510+0000 7f96fa600640 1 --2- 172.21.15.16:0/3556955826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96ec0075f0 0x7f96ec09a5f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:48.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.510+0000 7f96fa600640 1 --2- 172.21.15.16:0/3556955826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96ec0a26c0 0x7f96ec09ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:48.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.510+0000 7f96fa600640 1 --2- 172.21.15.16:0/3556955826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96ec006ca0 0x7f96ec09a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:48.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.510+0000 7f96fa600640 1 -- 172.21.15.16:0/3556955826 >> 172.21.15.16:0/3556955826 conn(0x7f96ec091b50 msgr2=0x7f96ec09d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:48.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.511+0000 7f96fa600640 1 -- 172.21.15.16:0/3556955826 shutdown_connections 2024-08-31T06:15:48.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:48.511+0000 7f96fa600640 1 -- 172.21.15.16:0/3556955826 wait complete. 2024-08-31T06:15:48.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:48 smithi016 ceph-mon[17194]: pgmap v627: 97 pgs: 97 active+clean; 3.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 7.2 MiB/s wr, 712 op/s 2024-08-31T06:15:48.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:48 smithi184 ceph-mon[22331]: pgmap v627: 97 pgs: 97 active+clean; 3.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 7.2 MiB/s wr, 712 op/s 2024-08-31T06:15:49.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:48 smithi155 ceph-mon[21742]: pgmap v627: 97 pgs: 97 active+clean; 3.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 7.2 MiB/s wr, 712 op/s 2024-08-31T06:15:49.637 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T06:15:50.299 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:15:50.714 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:50 smithi016 ceph-mon[17194]: pgmap v628: 97 pgs: 97 active+clean; 3.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 7.3 MiB/s wr, 728 op/s 2024-08-31T06:15:50.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:50 smithi184 ceph-mon[22331]: pgmap v628: 97 pgs: 97 active+clean; 3.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 7.3 MiB/s wr, 728 op/s 2024-08-31T06:15:51.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.106+0000 7f9c15400640 1 -- 172.21.15.16:0/566627460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c10102610 msgr2=0x7f9c10102a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:51.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.106+0000 7f9c15400640 1 --2- 172.21.15.16:0/566627460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c10102610 0x7f9c10102a20 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9c00009860 tx=0x7f9c0002a340 comp rx=0 tx=0).stop 2024-08-31T06:15:51.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.106+0000 7f9c15400640 1 -- 172.21.15.16:0/566627460 shutdown_connections 2024-08-31T06:15:51.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.106+0000 7f9c15400640 1 --2- 172.21.15.16:0/566627460 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c101088a0 0x7f9c1010f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.106+0000 7f9c15400640 1 --2- 172.21.15.16:0/566627460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c10107f50 0x7f9c10108360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.106+0000 7f9c15400640 1 --2- 172.21.15.16:0/566627460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c10102610 0x7f9c10102a20 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.106+0000 7f9c15400640 1 -- 172.21.15.16:0/566627460 >> 172.21.15.16:0/566627460 conn(0x7f9c100fe060 msgr2=0x7f9c101004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:51.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.107+0000 7f9c15400640 1 -- 172.21.15.16:0/566627460 shutdown_connections 2024-08-31T06:15:51.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.107+0000 7f9c15400640 1 -- 172.21.15.16:0/566627460 wait complete. 2024-08-31T06:15:51.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.109+0000 7f9c15400640 1 Processor -- start 2024-08-31T06:15:51.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.109+0000 7f9c15400640 1 -- start start 2024-08-31T06:15:51.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.109+0000 7f9c15400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c10102610 0x7f9c1019c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:51.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.109+0000 7f9c15400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c10107f50 0x7f9c1019c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:51.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.109+0000 7f9c15400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c101088a0 0x7f9c101a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:51.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.109+0000 7f9c15400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9c10112550 con 0x7f9c101088a0 2024-08-31T06:15:51.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.110+0000 7f9c15400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9c101123d0 con 0x7f9c10107f50 2024-08-31T06:15:51.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.110+0000 7f9c15400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9c101126d0 con 0x7f9c10102610 2024-08-31T06:15:51.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.111+0000 7f9c14a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c101088a0 0x7f9c101a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:51.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.111+0000 7f9c0f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c10107f50 0x7f9c1019c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:51.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.111+0000 7f9c0fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c10102610 0x7f9c1019c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:51.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.111+0000 7f9c0f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c10107f50 0x7f9c1019c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:41242/0 (socket says 172.21.15.16:41242) 2024-08-31T06:15:51.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.111+0000 7f9c0f400640 1 -- 172.21.15.16:0/2812067557 learned_addr learned my addr 172.21.15.16:0/2812067557 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:51.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.111+0000 7f9c0fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c10102610 0x7f9c1019c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:48876/0 (socket says 172.21.15.16:48876) 2024-08-31T06:15:51.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.113+0000 7f9c14a00640 1 -- 172.21.15.16:0/2812067557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c10102610 msgr2=0x7f9c1019c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:51.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.114+0000 7f9c14a00640 1 --2- 172.21.15.16:0/2812067557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c10102610 0x7f9c1019c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.114+0000 7f9c14a00640 1 -- 172.21.15.16:0/2812067557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c10107f50 msgr2=0x7f9c1019c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:51.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.114+0000 7f9c14a00640 1 --2- 172.21.15.16:0/2812067557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c10107f50 0x7f9c1019c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.114+0000 7f9c14a00640 1 -- 172.21.15.16:0/2812067557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c101a21b0 con 0x7f9c101088a0 2024-08-31T06:15:51.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.115+0000 7f9c0fe00640 1 --2- 172.21.15.16:0/2812067557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c10102610 0x7f9c1019c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:15:51.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.115+0000 7f9c0f400640 1 --2- 172.21.15.16:0/2812067557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c10107f50 0x7f9c1019c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:15:51.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.115+0000 7f9c14a00640 1 --2- 172.21.15.16:0/2812067557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c101088a0 0x7f9c101a1ab0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800ef00 tx=0x7f9c0800c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:51.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.116+0000 7f9c0cc00640 1 -- 172.21.15.16:0/2812067557 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c08004b20 con 0x7f9c101088a0 2024-08-31T06:15:51.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.116+0000 7f9c0cc00640 1 -- 172.21.15.16:0/2812067557 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c08003e20 con 0x7f9c101088a0 2024-08-31T06:15:51.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.116+0000 7f9c0cc00640 1 -- 172.21.15.16:0/2812067557 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0800b570 con 0x7f9c101088a0 2024-08-31T06:15:51.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.116+0000 7f9c15400640 1 -- 172.21.15.16:0/2812067557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c101a24a0 con 0x7f9c101088a0 2024-08-31T06:15:51.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.117+0000 7f9c15400640 1 -- 172.21.15.16:0/2812067557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c1006cb80 con 0x7f9c101088a0 2024-08-31T06:15:51.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:50 smithi155 ceph-mon[21742]: pgmap v628: 97 pgs: 97 active+clean; 3.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 7.3 MiB/s wr, 728 op/s 2024-08-31T06:15:51.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.122+0000 7f9c1d5c5740 1 Processor -- start 2024-08-31T06:15:51.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.123+0000 7f9c0cc00640 1 -- 172.21.15.16:0/2812067557 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f9c0801d020 con 0x7f9c101088a0 2024-08-31T06:15:51.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.123+0000 7f9c1d5c5740 1 -- start start 2024-08-31T06:15:51.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.123+0000 7f9c1d5c5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958eecd070 0x55958f272b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:51.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.123+0000 7f9c1d5c5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958eea41b0 0x55958f2730a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:51.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.123+0000 7f9c1d5c5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958eeac390 0x55958f2735e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:51.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.123+0000 7f9c1d5c5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55958f1e97b0 con 0x55958eea41b0 2024-08-31T06:15:51.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.123+0000 7f9c1d5c5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55958f1d8d40 con 0x55958eeac390 2024-08-31T06:15:51.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.123+0000 7f9c1d5c5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55958f1dccf0 con 0x55958eecd070 2024-08-31T06:15:51.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.124+0000 7f9c0f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958eeac390 0x55958f2735e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:51.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.124+0000 7f9c0f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958eeac390 0x55958f2735e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:41254/0 (socket says 172.21.15.16:41254) 2024-08-31T06:15:51.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.124+0000 7f9c0f400640 1 -- 172.21.15.16:0/815111677 learned_addr learned my addr 172.21.15.16:0/815111677 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:51.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.124+0000 7f9c0f400640 1 --2- 172.21.15.16:0/815111677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958eecd070 0x55958f272b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:51.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.124+0000 7f9c0fe00640 1 --2- 172.21.15.16:0/815111677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958eea41b0 0x55958f2730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:51.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.125+0000 7f9c0f400640 1 -- 172.21.15.16:0/815111677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958eecd070 msgr2=0x55958f272b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:51.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.126+0000 7f9c0f400640 1 --2- 172.21.15.16:0/815111677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958eecd070 0x55958f272b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.126+0000 7f9c0f400640 1 -- 172.21.15.16:0/815111677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958eea41b0 msgr2=0x55958f2730a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:51.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.126+0000 7f9c0f400640 1 --2- 172.21.15.16:0/815111677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958eea41b0 0x55958f2730a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.126+0000 7f9c0f400640 1 -- 172.21.15.16:0/815111677 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55958ef38650 con 0x55958eeac390 2024-08-31T06:15:51.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.126+0000 7f9c0fe00640 1 --2- 172.21.15.16:0/815111677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958eea41b0 0x55958f2730a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:15:51.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.126+0000 7f9c0cc00640 1 --2- 172.21.15.16:0/2812067557 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9bf40b4da0 0x7f9bf40b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:51.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.126+0000 7f9c0cc00640 1 -- 172.21.15.16:0/2812067557 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8470+0+0 (secure 0 0 0) 0x7f9c080e5350 con 0x7f9c101088a0 2024-08-31T06:15:51.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.127+0000 7f9c0fe00640 1 --2- 172.21.15.16:0/2812067557 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9bf40b4da0 0x7f9bf40b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:51.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.127+0000 7f9c0f400640 1 --2- 172.21.15.16:0/815111677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958eeac390 0x55958f2735e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc0095f0 tx=0x7f9bfc009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=969780c10eb060cd server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:51.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.127+0000 7f9c05200640 1 -- 172.21.15.16:0/815111677 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc00fc50 con 0x55958eeac390 2024-08-31T06:15:51.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.127+0000 7f9c05200640 1 -- 172.21.15.16:0/815111677 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bfc00fdf0 con 0x55958eeac390 2024-08-31T06:15:51.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.128+0000 7f9c05200640 1 -- 172.21.15.16:0/815111677 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc0047a0 con 0x55958eeac390 2024-08-31T06:15:51.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.128+0000 7f9c1d5c5740 1 -- 172.21.15.16:0/815111677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958eeac390 msgr2=0x55958f2735e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:51.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.128+0000 7f9c1d5c5740 1 --2- 172.21.15.16:0/815111677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958eeac390 0x55958f2735e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc0095f0 tx=0x7f9bfc009560 comp rx=0 tx=0).stop 2024-08-31T06:15:51.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.129+0000 7f9c0fe00640 1 --2- 172.21.15.16:0/2812067557 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9bf40b4da0 0x7f9bf40b72b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9c00009830 tx=0x7f9c0002c040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:51.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.129+0000 7f9c1d5c5740 1 -- 172.21.15.16:0/815111677 shutdown_connections 2024-08-31T06:15:51.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.129+0000 7f9c1d5c5740 1 --2- 172.21.15.16:0/815111677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958eeac390 0x55958f2735e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.129+0000 7f9c1d5c5740 1 --2- 172.21.15.16:0/815111677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958eea41b0 0x55958f2730a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.129+0000 7f9c1d5c5740 1 --2- 172.21.15.16:0/815111677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958eecd070 0x55958f272b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.129+0000 7f9c1d5c5740 1 -- 172.21.15.16:0/815111677 >> 172.21.15.16:0/815111677 conn(0x55958eec1460 msgr2=0x55958f270a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:51.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.129+0000 7f9c1d5c5740 1 -- 172.21.15.16:0/815111677 shutdown_connections 2024-08-31T06:15:51.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.129+0000 7f9c1d5c5740 1 -- 172.21.15.16:0/815111677 wait complete. 2024-08-31T06:15:51.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.130+0000 7f9c1d5c5740 1 -- start start 2024-08-31T06:15:51.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.130+0000 7f9c1d5c5740 1 Processor -- start 2024-08-31T06:15:51.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.131+0000 7f9c1d5c5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958eea41b0 0x55958f27e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:51.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.131+0000 7f9c1d5c5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958eeac390 0x55958f27ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:51.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.131+0000 7f9c1d5c5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958eecd070 0x55958f292e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:51.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.131+0000 7f9c1d5c5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55958f03b700 con 0x55958eecd070 2024-08-31T06:15:51.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.131+0000 7f9c1d5c5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55958f15fcf0 con 0x55958eea41b0 2024-08-31T06:15:51.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.131+0000 7f9c1d5c5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55958f289ba0 con 0x55958eeac390 2024-08-31T06:15:51.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.131+0000 7f9c1d5c5740 10 client.0 _ll_register_callbacks cb 0x55958f185470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:15:51.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.131+0000 7f9c0fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958eea41b0 0x55958f27e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:51.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.132+0000 7f9c0f400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958eeac390 0x55958f27ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:51.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.132+0000 7f9c0f400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958eeac390 0x55958f27ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:48900/0 (socket says 172.21.15.16:48900) 2024-08-31T06:15:51.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.132+0000 7f9c0f400640 1 -- 172.21.15.16:0/75766671 learned_addr learned my addr 172.21.15.16:0/75766671 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:51.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.132+0000 7f9c14a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958eecd070 0x55958f292e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:51.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.133+0000 7f9c14a00640 1 -- 172.21.15.16:0/75766671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958eeac390 msgr2=0x55958f27ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:51.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.133+0000 7f9c14a00640 1 --2- 172.21.15.16:0/75766671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958eeac390 0x55958f27ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.133+0000 7f9c14a00640 1 -- 172.21.15.16:0/75766671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958eea41b0 msgr2=0x55958f27e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:51.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.133+0000 7f9c14a00640 1 --2- 172.21.15.16:0/75766671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958eea41b0 0x55958f27e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.133+0000 7f9c14a00640 1 -- 172.21.15.16:0/75766671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55958f1a08b0 con 0x55958eecd070 2024-08-31T06:15:51.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.133+0000 7f9c0fe00640 1 --2- 172.21.15.16:0/75766671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958eea41b0 0x55958f27e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:15:51.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.134+0000 7f9c14a00640 1 --2- 172.21.15.16:0/75766671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958eecd070 0x55958f292e90 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9c080e2a70 tx=0x7f9c080a7260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27375c4c2a3c3d7e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:51.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.134+0000 7f9bf9600640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:15:51.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.134+0000 7f9bf9600640 1 -- 172.21.15.16:0/75766671 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c08094020 con 0x55958eecd070 2024-08-31T06:15:51.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.134+0000 7f9c1d5c5740 1 -- 172.21.15.16:0/75766671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bfc00fdf0 con 0x55958eecd070 2024-08-31T06:15:51.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.135+0000 7f9bf9600640 1 -- 172.21.15.16:0/75766671 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8470+0+0 (secure 0 0 0) 0x7f9c080b48a0 con 0x55958eecd070 2024-08-31T06:15:51.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.136+0000 7f9bf9600640 1 -- 172.21.15.16:0/75766671 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f9c080e2f00 con 0x55958eecd070 2024-08-31T06:15:51.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.136+0000 7f9c1d5c5740 10 client.17280 fetch_fsmap learned FSMap version 42 2024-08-31T06:15:51.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.136+0000 7f9c1d5c5740 1 -- 172.21.15.16:0/75766671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55958f295bb0 con 0x55958eecd070 2024-08-31T06:15:51.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.136+0000 7f9bf9600640 1 -- 172.21.15.16:0/75766671 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bfc00fdf0 con 0x55958eecd070 2024-08-31T06:15:51.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.137+0000 7f9bf9600640 1 -- 172.21.15.16:0/75766671 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7f9c0800b040 con 0x55958eecd070 2024-08-31T06:15:51.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.137+0000 7f9c1d5c5740 10 client.17280 fetch_fsmap finished waiting for FSMap version 42 2024-08-31T06:15:51.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.137+0000 7f9c1d5c5740 10 client.17280 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-31T06:15:51.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.137+0000 7f9c1d5c5740 20 client.17280 populate_metadata read hostname 'smithi016' 2024-08-31T06:15:51.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.137+0000 7f9c1d5c5740 1 --2- 172.21.15.16:0/75766671 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55958ee8b9f0 0x55958f299080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:51.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.137+0000 7f9c1d5c5740 4 client.17280 mds_command: new command op to 24724 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:15:51.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.137+0000 7f9c1d5c5740 1 -- 172.21.15.16:0/75766671 --> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55958f1eb700 con 0x55958ee8b9f0 2024-08-31T06:15:51.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.137+0000 7f9c0fe00640 1 --2- 172.21.15.16:0/75766671 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55958ee8b9f0 0x55958f299080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:51.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.138+0000 7f9c0fe00640 1 --2- 172.21.15.16:0/75766671 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55958ee8b9f0 0x55958f299080 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ad39c8f4248bb168 server_cookie=de44eeb406169083 in_seq=0 out_seq=0 2024-08-31T06:15:51.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.138+0000 7f9bf9600640 10 client.17280 ms_handle_connect on v2:172.21.15.16:6838/219664460 2024-08-31T06:15:51.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.139+0000 7f9bf9600640 1 -- 172.21.15.16:0/75766671 <== mds.1 v2:172.21.15.16:6838/219664460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55958f1eb700 con 0x55958ee8b9f0 2024-08-31T06:15:51.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.140+0000 7f9bf9600640 10 client.17280 handle_command_reply: tid=0 2024-08-31T06:15:51.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.140+0000 7f9c1d5c5740 1 client.17280 shutdown 2024-08-31T06:15:51.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.140+0000 7f9c1d5c5740 1 -- 172.21.15.16:0/75766671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958eecd070 msgr2=0x55958f292e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:51.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.140+0000 7f9c1d5c5740 1 --2- 172.21.15.16:0/75766671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958eecd070 0x55958f292e90 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9c080e2a70 tx=0x7f9c080a7260 comp rx=0 tx=0).stop 2024-08-31T06:15:51.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.141+0000 7f9c1d5c5740 1 -- 172.21.15.16:0/75766671 shutdown_connections 2024-08-31T06:15:51.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.141+0000 7f9c1d5c5740 1 --2- 172.21.15.16:0/75766671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958eeac390 0x55958f27ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.141+0000 7f9c1d5c5740 1 --2- 172.21.15.16:0/75766671 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55958ee8b9f0 0x55958f299080 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.141+0000 7f9c1d5c5740 1 --2- 172.21.15.16:0/75766671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958eecd070 0x55958f292e90 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.141+0000 7f9c1d5c5740 1 --2- 172.21.15.16:0/75766671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958eea41b0 0x55958f27e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.141+0000 7f9c1d5c5740 1 -- 172.21.15.16:0/75766671 >> 172.21.15.16:0/75766671 conn(0x55958eec1460 msgr2=0x55958f274550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:51.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.141+0000 7f9bf9600640 0 client.17280 ms_handle_reset on v2:172.21.15.16:6838/219664460 2024-08-31T06:15:51.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.142+0000 7f9c1d5c5740 1 -- 172.21.15.16:0/75766671 shutdown_connections 2024-08-31T06:15:51.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.142+0000 7f9c1d5c5740 1 -- 172.21.15.16:0/75766671 wait complete. 2024-08-31T06:15:51.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.142+0000 7f9c1d5c5740 20 client.17280 trim_cache size 0 max 16384 2024-08-31T06:15:51.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.162+0000 7f9c1d5c5740 1 Processor -- start 2024-08-31T06:15:51.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.162+0000 7f9c1d5c5740 1 -- start start 2024-08-31T06:15:51.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.162+0000 7f9c1d5c5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958eea41b0 0x55958f292d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:51.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.162+0000 7f9c1d5c5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958eeac390 0x55958f2932d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:51.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.163+0000 7f9c1d5c5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958eecd070 0x55958f27ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:51.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.163+0000 7f9c1d5c5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9c100738c0 con 0x55958eea41b0 2024-08-31T06:15:51.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.163+0000 7f9c1d5c5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9c10071100 con 0x55958eecd070 2024-08-31T06:15:51.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.163+0000 7f9c1d5c5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1006ef70 con 0x55958eeac390 2024-08-31T06:15:51.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.163+0000 7f9c14a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958eecd070 0x55958f27ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:51.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.163+0000 7f9c14a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958eecd070 0x55958f27ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:41278/0 (socket says 172.21.15.16:41278) 2024-08-31T06:15:51.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.163+0000 7f9c14a00640 1 -- 172.21.15.16:0/151639483 learned_addr learned my addr 172.21.15.16:0/151639483 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:51.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.163+0000 7f9c0f400640 1 --2- 172.21.15.16:0/151639483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958eeac390 0x55958f2932d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:51.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.164+0000 7f9c14a00640 1 -- 172.21.15.16:0/151639483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958eeac390 msgr2=0x55958f2932d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:51.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.164+0000 7f9c0fe00640 1 --2- 172.21.15.16:0/151639483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958eea41b0 0x55958f292d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:51.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.164+0000 7f9c14a00640 1 --2- 172.21.15.16:0/151639483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958eeac390 0x55958f2932d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.164+0000 7f9c14a00640 1 -- 172.21.15.16:0/151639483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958eea41b0 msgr2=0x55958f292d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:51.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.164+0000 7f9c14a00640 1 --2- 172.21.15.16:0/151639483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958eea41b0 0x55958f292d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.164+0000 7f9c14a00640 1 -- 172.21.15.16:0/151639483 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55958f278630 con 0x55958eecd070 2024-08-31T06:15:51.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.164+0000 7f9c0fe00640 1 --2- 172.21.15.16:0/151639483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958eea41b0 0x55958f292d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:15:51.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.165+0000 7f9c14a00640 1 --2- 172.21.15.16:0/151639483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958eecd070 0x55958f27ccf0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9c080ab9a0 tx=0x7f9c080a72d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4aca77d73a15f7a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:51.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.165+0000 7f9bfbe00640 1 -- 172.21.15.16:0/151639483 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c08094070 con 0x55958eecd070 2024-08-31T06:15:51.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.165+0000 7f9bfbe00640 1 -- 172.21.15.16:0/151639483 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c0805c040 con 0x55958eecd070 2024-08-31T06:15:51.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.165+0000 7f9bfbe00640 1 -- 172.21.15.16:0/151639483 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0800e0b0 con 0x55958eecd070 2024-08-31T06:15:51.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.166+0000 7f9c1d5c5740 1 -- 172.21.15.16:0/151639483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958eecd070 msgr2=0x55958f27ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:51.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.166+0000 7f9c1d5c5740 1 --2- 172.21.15.16:0/151639483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958eecd070 0x55958f27ccf0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9c080ab9a0 tx=0x7f9c080a72d0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.166+0000 7f9c1d5c5740 1 -- 172.21.15.16:0/151639483 shutdown_connections 2024-08-31T06:15:51.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.166+0000 7f9c1d5c5740 1 --2- 172.21.15.16:0/151639483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958eeac390 0x55958f2932d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.166+0000 7f9c1d5c5740 1 --2- 172.21.15.16:0/151639483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958eecd070 0x55958f27ccf0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.166+0000 7f9c1d5c5740 1 --2- 172.21.15.16:0/151639483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958eea41b0 0x55958f292d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.166+0000 7f9c1d5c5740 1 -- 172.21.15.16:0/151639483 >> 172.21.15.16:0/151639483 conn(0x55958eec1460 msgr2=0x55958f279620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:51.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.167+0000 7f9c1d5c5740 1 -- 172.21.15.16:0/151639483 shutdown_connections 2024-08-31T06:15:51.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.167+0000 7f9c1d5c5740 1 -- 172.21.15.16:0/151639483 wait complete. 2024-08-31T06:15:51.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.167+0000 7f9c1d5c5740 1 -- start start 2024-08-31T06:15:51.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.167+0000 7f9c1d5c5740 1 Processor -- start 2024-08-31T06:15:51.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.168+0000 7f9c1d5c5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958eea41b0 0x55958f28f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:51.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.168+0000 7f9c1d5c5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958eeac390 0x55958f28ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:51.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.168+0000 7f9c1d5c5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958eecd070 0x55958f28bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:51.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.168+0000 7f9c1d5c5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1006f5c0 con 0x55958eeac390 2024-08-31T06:15:51.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.168+0000 7f9c1d5c5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9c10070a30 con 0x55958eecd070 2024-08-31T06:15:51.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.168+0000 7f9c1d5c5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1006fc90 con 0x55958eea41b0 2024-08-31T06:15:51.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.168+0000 7f9c1d5c5740 10 client.0 _ll_register_callbacks cb 0x7f9c080ab840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:15:51.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.168+0000 7f9c0fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958eea41b0 0x55958f28f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:51.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.169+0000 7f9c0fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958eea41b0 0x55958f28f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:48918/0 (socket says 172.21.15.16:48918) 2024-08-31T06:15:51.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.169+0000 7f9c0fe00640 1 -- 172.21.15.16:0/1785050164 learned_addr learned my addr 172.21.15.16:0/1785050164 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:51.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.169+0000 7f9c14a00640 1 --2- 172.21.15.16:0/1785050164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958eecd070 0x55958f28bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:51.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.169+0000 7f9c0f400640 1 --2- 172.21.15.16:0/1785050164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958eeac390 0x55958f28ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:51.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.169+0000 7f9c14a00640 1 -- 172.21.15.16:0/1785050164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958eea41b0 msgr2=0x55958f28f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:51.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.169+0000 7f9c14a00640 1 --2- 172.21.15.16:0/1785050164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958eea41b0 0x55958f28f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.169+0000 7f9c14a00640 1 -- 172.21.15.16:0/1785050164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958eeac390 msgr2=0x55958f28ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:51.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.169+0000 7f9c14a00640 1 --2- 172.21.15.16:0/1785050164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958eeac390 0x55958f28ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.169+0000 7f9c14a00640 1 -- 172.21.15.16:0/1785050164 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55958f28c660 con 0x55958eecd070 2024-08-31T06:15:51.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.169+0000 7f9c0fe00640 1 --2- 172.21.15.16:0/1785050164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958eea41b0 0x55958f28f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:15:51.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.170+0000 7f9c14a00640 1 --2- 172.21.15.16:0/1785050164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958eecd070 0x55958f28bf70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9c080a7410 tx=0x7f9c08009b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a9dfe41aac8b90a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:51.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.170+0000 7f9bf9600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:15:51.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.170+0000 7f9bf9600640 1 -- 172.21.15.16:0/1785050164 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c08094070 con 0x55958eecd070 2024-08-31T06:15:51.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.170+0000 7f9bf9600640 1 -- 172.21.15.16:0/1785050164 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8470+0+0 (secure 0 0 0) 0x7f9c080a7490 con 0x55958eecd070 2024-08-31T06:15:51.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.170+0000 7f9c1d5c5740 1 -- 172.21.15.16:0/1785050164 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c0805c040 con 0x55958eecd070 2024-08-31T06:15:51.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.171+0000 7f9bf9600640 1 -- 172.21.15.16:0/1785050164 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f9c080aba10 con 0x55958eecd070 2024-08-31T06:15:51.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.171+0000 7f9c1d5c5740 10 client.27536 fetch_fsmap learned FSMap version 42 2024-08-31T06:15:51.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.171+0000 7f9c1d5c5740 1 -- 172.21.15.16:0/1785050164 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55958f304890 con 0x55958eecd070 2024-08-31T06:15:51.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.171+0000 7f9bf9600640 1 -- 172.21.15.16:0/1785050164 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c0805c040 con 0x55958eecd070 2024-08-31T06:15:51.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.171+0000 7f9bf9600640 1 -- 172.21.15.16:0/1785050164 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7f9c08058b00 con 0x55958eecd070 2024-08-31T06:15:51.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.171+0000 7f9c1d5c5740 10 client.27536 fetch_fsmap finished waiting for FSMap version 42 2024-08-31T06:15:51.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.172+0000 7f9c1d5c5740 10 client.27536 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-31T06:15:51.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.172+0000 7f9c1d5c5740 20 client.27536 populate_metadata read hostname 'smithi016' 2024-08-31T06:15:51.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.172+0000 7f9c1d5c5740 1 --2- 172.21.15.16:0/1785050164 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55958ee60da0 0x55958f306d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:51.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.172+0000 7f9c1d5c5740 4 client.27536 mds_command: new command op to 24724 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:15:51.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.172+0000 7f9c1d5c5740 1 -- 172.21.15.16:0/1785050164 --> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55958f1e71a0 con 0x55958ee60da0 2024-08-31T06:15:51.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.172+0000 7f9c0fe00640 1 --2- 172.21.15.16:0/1785050164 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55958ee60da0 0x55958f306d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:51.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.174+0000 7f9c0fe00640 1 --2- 172.21.15.16:0/1785050164 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55958ee60da0 0x55958f306d10 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f163c791d5fb2ff2 server_cookie=67044c8c7a6bb807 in_seq=0 out_seq=0 2024-08-31T06:15:51.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.174+0000 7f9bf9600640 10 client.27536 ms_handle_connect on v2:172.21.15.16:6838/219664460 2024-08-31T06:15:51.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.175+0000 7f9bf9600640 1 -- 172.21.15.16:0/1785050164 <== mds.1 v2:172.21.15.16:6838/219664460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55958f1e71a0 con 0x55958ee60da0 2024-08-31T06:15:51.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.175+0000 7f9bf9600640 10 client.27536 handle_command_reply: tid=0 2024-08-31T06:15:51.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.175+0000 7f9c1d5c5740 1 client.27536 shutdown 2024-08-31T06:15:51.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.175+0000 7f9c1d5c5740 1 -- 172.21.15.16:0/1785050164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958eecd070 msgr2=0x55958f28bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:51.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.175+0000 7f9c1d5c5740 1 --2- 172.21.15.16:0/1785050164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958eecd070 0x55958f28bf70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9c080a7410 tx=0x7f9c08009b70 comp rx=0 tx=0).stop 2024-08-31T06:15:51.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.175+0000 7f9c1d5c5740 1 -- 172.21.15.16:0/1785050164 shutdown_connections 2024-08-31T06:15:51.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.175+0000 7f9c1d5c5740 1 --2- 172.21.15.16:0/1785050164 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55958ee60da0 0x55958f306d10 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.175+0000 7f9c1d5c5740 1 --2- 172.21.15.16:0/1785050164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55958eecd070 0x55958f28bf70 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.175+0000 7f9c1d5c5740 1 --2- 172.21.15.16:0/1785050164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55958eeac390 0x55958f28ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.175+0000 7f9bf9600640 0 client.27536 ms_handle_reset on v2:172.21.15.16:6838/219664460 2024-08-31T06:15:51.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.175+0000 7f9c1d5c5740 1 --2- 172.21.15.16:0/1785050164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55958eea41b0 0x55958f28f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.175+0000 7f9c1d5c5740 1 -- 172.21.15.16:0/1785050164 >> 172.21.15.16:0/1785050164 conn(0x55958eec1460 msgr2=0x55958f279620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:51.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.176+0000 7f9c1d5c5740 1 -- 172.21.15.16:0/1785050164 shutdown_connections 2024-08-31T06:15:51.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.176+0000 7f9c1d5c5740 1 -- 172.21.15.16:0/1785050164 wait complete. 2024-08-31T06:15:51.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.176+0000 7f9c1d5c5740 20 client.27536 trim_cache size 0 max 16384 2024-08-31T06:15:51.179 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:15:51.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.177+0000 7f9c15400640 1 -- 172.21.15.16:0/2812067557 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9bf40b4da0 msgr2=0x7f9bf40b72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:51.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.177+0000 7f9c15400640 1 --2- 172.21.15.16:0/2812067557 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9bf40b4da0 0x7f9bf40b72b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9c00009830 tx=0x7f9c0002c040 comp rx=0 tx=0).stop 2024-08-31T06:15:51.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.177+0000 7f9c15400640 1 -- 172.21.15.16:0/2812067557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c101088a0 msgr2=0x7f9c101a1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:51.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.177+0000 7f9c15400640 1 --2- 172.21.15.16:0/2812067557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c101088a0 0x7f9c101a1ab0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800ef00 tx=0x7f9c0800c740 comp rx=0 tx=0).stop 2024-08-31T06:15:51.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.177+0000 7f9c15400640 1 -- 172.21.15.16:0/2812067557 shutdown_connections 2024-08-31T06:15:51.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.177+0000 7f9c15400640 1 --2- 172.21.15.16:0/2812067557 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9bf40b4da0 0x7f9bf40b72b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.177+0000 7f9c15400640 1 --2- 172.21.15.16:0/2812067557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c101088a0 0x7f9c101a1ab0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.177+0000 7f9c15400640 1 --2- 172.21.15.16:0/2812067557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c10107f50 0x7f9c1019c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.177+0000 7f9c15400640 1 --2- 172.21.15.16:0/2812067557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c10102610 0x7f9c1019c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.177+0000 7f9c15400640 1 -- 172.21.15.16:0/2812067557 >> 172.21.15.16:0/2812067557 conn(0x7f9c100fe060 msgr2=0x7f9c10107620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:51.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.177+0000 7f9c15400640 1 -- 172.21.15.16:0/2812067557 shutdown_connections 2024-08-31T06:15:51.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:51.177+0000 7f9c15400640 1 -- 172.21.15.16:0/2812067557 wait complete. 2024-08-31T06:15:51.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 505.0 s, 1644.6 tps, lat 59.954 ms stddev 73.632 2024-08-31T06:15:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.773+0000 7f7d43000640 1 -- 172.21.15.16:0/1255087327 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d3c078030 msgr2=0x7f7d3c113d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.773+0000 7f7d43000640 1 --2- 172.21.15.16:0/1255087327 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d3c078030 0x7f7d3c113d20 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7d28009900 tx=0x7f7d2802a440 comp rx=0 tx=0).stop 2024-08-31T06:15:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.774+0000 7f7d43000640 1 -- 172.21.15.16:0/1255087327 shutdown_connections 2024-08-31T06:15:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.774+0000 7f7d43000640 1 --2- 172.21.15.16:0/1255087327 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d3c078030 0x7f7d3c113d20 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.774+0000 7f7d43000640 1 --2- 172.21.15.16:0/1255087327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d3c114260 0x7f7d3c116680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.774+0000 7f7d43000640 1 --2- 172.21.15.16:0/1255087327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d3c0776e0 0x7f7d3c077af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.774+0000 7f7d43000640 1 -- 172.21.15.16:0/1255087327 >> 172.21.15.16:0/1255087327 conn(0x7f7d3c101290 msgr2=0x7f7d3c1036e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.775+0000 7f7d43000640 1 -- 172.21.15.16:0/1255087327 shutdown_connections 2024-08-31T06:15:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.775+0000 7f7d43000640 1 -- 172.21.15.16:0/1255087327 wait complete. 2024-08-31T06:15:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.777+0000 7f7d43000640 1 Processor -- start 2024-08-31T06:15:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.778+0000 7f7d43000640 1 -- start start 2024-08-31T06:15:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.779+0000 7f7d43000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d3c0776e0 0x7f7d3c1a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.779+0000 7f7d43000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d3c078030 0x7f7d3c1a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.779+0000 7f7d43000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d3c114260 0x7f7d3c1a5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.779+0000 7f7d43000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c119c10 con 0x7f7d3c078030 2024-08-31T06:15:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.779+0000 7f7d43000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c119a90 con 0x7f7d3c0776e0 2024-08-31T06:15:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.779+0000 7f7d43000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c119d90 con 0x7f7d3c114260 2024-08-31T06:15:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.780+0000 7f7d41c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d3c0776e0 0x7f7d3c1a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:51.782 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.781+0000 7f7d42600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d3c114260 0x7f7d3c1a5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:51.782 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.781+0000 7f7d42600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d3c114260 0x7f7d3c1a5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:48942/0 (socket says 172.21.15.16:48942) 2024-08-31T06:15:51.782 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.781+0000 7f7d42600640 1 -- 172.21.15.16:0/1217851074 learned_addr learned my addr 172.21.15.16:0/1217851074 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.782+0000 7f7d41200640 1 --2- 172.21.15.16:0/1217851074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d3c078030 0x7f7d3c1a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.782+0000 7f7d41c00640 1 -- 172.21.15.16:0/1217851074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d3c114260 msgr2=0x7f7d3c1a5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.782+0000 7f7d41c00640 1 --2- 172.21.15.16:0/1217851074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d3c114260 0x7f7d3c1a5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.782+0000 7f7d41c00640 1 -- 172.21.15.16:0/1217851074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d3c078030 msgr2=0x7f7d3c1a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.782+0000 7f7d41c00640 1 --2- 172.21.15.16:0/1217851074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d3c078030 0x7f7d3c1a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.782+0000 7f7d41c00640 1 -- 172.21.15.16:0/1217851074 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d3c1a6540 con 0x7f7d3c0776e0 2024-08-31T06:15:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.784+0000 7f7d41c00640 1 --2- 172.21.15.16:0/1217851074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d3c0776e0 0x7f7d3c1a0610 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7d3000d6d0 tx=0x7f7d3000db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.784+0000 7f7d3a000640 1 -- 172.21.15.16:0/1217851074 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d30017070 con 0x7f7d3c0776e0 2024-08-31T06:15:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.784+0000 7f7d3a000640 1 -- 172.21.15.16:0/1217851074 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d30009820 con 0x7f7d3c0776e0 2024-08-31T06:15:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.784+0000 7f7d3a000640 1 -- 172.21.15.16:0/1217851074 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d30004c40 con 0x7f7d3c0776e0 2024-08-31T06:15:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.784+0000 7f7d43000640 1 -- 172.21.15.16:0/1217851074 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d3c1a6830 con 0x7f7d3c0776e0 2024-08-31T06:15:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.785+0000 7f7d43000640 1 -- 172.21.15.16:0/1217851074 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d3c1acfc0 con 0x7f7d3c0776e0 2024-08-31T06:15:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.798+0000 7f7d3a000640 1 -- 172.21.15.16:0/1217851074 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f7d30005220 con 0x7f7d3c0776e0 2024-08-31T06:15:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.799+0000 7f7d4b7c4740 1 Processor -- start 2024-08-31T06:15:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.799+0000 7f7d4b7c4740 1 -- start start 2024-08-31T06:15:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.800+0000 7f7d4b7c4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd3a2cb1b0 0x55fd3a66eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.800+0000 7f7d4b7c4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd3a2a22f0 0x55fd3a66f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.800+0000 7f7d4b7c4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd3a2aa4d0 0x55fd3a66f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.800+0000 7f7d4b7c4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fd3a5e3f70 con 0x55fd3a2a22f0 2024-08-31T06:15:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.800+0000 7f7d4b7c4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55fd3a5d3500 con 0x55fd3a2aa4d0 2024-08-31T06:15:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.800+0000 7f7d4b7c4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55fd3a5d74b0 con 0x55fd3a2cb1b0 2024-08-31T06:15:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.800+0000 7f7d41c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd3a2a22f0 0x55fd3a66f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.801+0000 7f7d41c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd3a2a22f0 0x55fd3a66f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57098/0 (socket says 172.21.15.16:57098) 2024-08-31T06:15:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.801+0000 7f7d41c00640 1 -- 172.21.15.16:0/1092675457 learned_addr learned my addr 172.21.15.16:0/1092675457 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.801+0000 7f7d42600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd3a2cb1b0 0x55fd3a66eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.801+0000 7f7d41200640 1 --2- 172.21.15.16:0/1092675457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd3a2aa4d0 0x55fd3a66f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.801+0000 7f7d41c00640 1 -- 172.21.15.16:0/1092675457 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd3a2cb1b0 msgr2=0x55fd3a66eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.801+0000 7f7d41c00640 1 --2- 172.21.15.16:0/1092675457 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd3a2cb1b0 0x55fd3a66eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.801+0000 7f7d41c00640 1 -- 172.21.15.16:0/1092675457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd3a2aa4d0 msgr2=0x55fd3a66f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.801+0000 7f7d41c00640 1 --2- 172.21.15.16:0/1092675457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd3a2aa4d0 0x55fd3a66f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.801+0000 7f7d41c00640 1 -- 172.21.15.16:0/1092675457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd3a58dc10 con 0x55fd3a2a22f0 2024-08-31T06:15:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.801+0000 7f7d42600640 1 --2- 172.21.15.16:0/1092675457 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd3a2cb1b0 0x55fd3a66eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:15:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.802+0000 7f7d3a000640 1 --2- 172.21.15.16:0/1217851074 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7d240b4d50 0x7f7d240b7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.802+0000 7f7d41c00640 1 --2- 172.21.15.16:0/1092675457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd3a2a22f0 0x55fd3a66f040 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f7d300e24b0 tx=0x7f7d300ed6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83badfc1b4870519 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.803+0000 7f7d3a000640 1 -- 172.21.15.16:0/1217851074 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8470+0+0 (secure 0 0 0) 0x7f7d300e4dc0 con 0x7f7d3c0776e0 2024-08-31T06:15:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.803+0000 7f7d42600640 1 --2- 172.21.15.16:0/1217851074 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7d240b4d50 0x7f7d240b7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.803+0000 7f7d2ea00640 1 -- 172.21.15.16:0/1092675457 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d300a73a0 con 0x55fd3a2a22f0 2024-08-31T06:15:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.803+0000 7f7d2ea00640 1 -- 172.21.15.16:0/1092675457 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d300aaff0 con 0x55fd3a2a22f0 2024-08-31T06:15:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.804+0000 7f7d2ea00640 1 -- 172.21.15.16:0/1092675457 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d300aac50 con 0x55fd3a2a22f0 2024-08-31T06:15:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.805+0000 7f7d4b7c4740 1 -- 172.21.15.16:0/1092675457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd3a2a22f0 msgr2=0x55fd3a66f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.805+0000 7f7d4b7c4740 1 --2- 172.21.15.16:0/1092675457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd3a2a22f0 0x55fd3a66f040 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f7d300e24b0 tx=0x7f7d300ed6f0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.805+0000 7f7d42600640 1 --2- 172.21.15.16:0/1217851074 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7d240b4d50 0x7f7d240b7260 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7d34009270 tx=0x7f7d34009680 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.806+0000 7f7d4b7c4740 1 -- 172.21.15.16:0/1092675457 shutdown_connections 2024-08-31T06:15:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.806+0000 7f7d4b7c4740 1 --2- 172.21.15.16:0/1092675457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd3a2aa4d0 0x55fd3a66f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.806+0000 7f7d4b7c4740 1 --2- 172.21.15.16:0/1092675457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd3a2a22f0 0x55fd3a66f040 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.806+0000 7f7d4b7c4740 1 --2- 172.21.15.16:0/1092675457 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd3a2cb1b0 0x55fd3a66eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.806+0000 7f7d4b7c4740 1 -- 172.21.15.16:0/1092675457 >> 172.21.15.16:0/1092675457 conn(0x55fd3a2bf300 msgr2=0x55fd3a66c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.806+0000 7f7d4b7c4740 1 -- 172.21.15.16:0/1092675457 shutdown_connections 2024-08-31T06:15:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.806+0000 7f7d4b7c4740 1 -- 172.21.15.16:0/1092675457 wait complete. 2024-08-31T06:15:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.806+0000 7f7d4b7c4740 1 -- start start 2024-08-31T06:15:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.807+0000 7f7d4b7c4740 1 Processor -- start 2024-08-31T06:15:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.807+0000 7f7d4b7c4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd3a2a22f0 0x55fd3a67a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.807+0000 7f7d4b7c4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd3a2aa4d0 0x55fd3a67a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.807+0000 7f7d4b7c4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd3a2cb1b0 0x55fd3a67abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.807+0000 7f7d4b7c4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fd3a50cab0 con 0x55fd3a2aa4d0 2024-08-31T06:15:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.807+0000 7f7d4b7c4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55fd3a57a320 con 0x55fd3a2cb1b0 2024-08-31T06:15:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.807+0000 7f7d4b7c4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55fd3a67d280 con 0x55fd3a2a22f0 2024-08-31T06:15:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.808+0000 7f7d4b7c4740 10 client.0 _ll_register_callbacks cb 0x55fd3a5806a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:15:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.808+0000 7f7d42600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd3a2cb1b0 0x55fd3a67abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.808+0000 7f7d41c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd3a2a22f0 0x55fd3a67a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.808+0000 7f7d41c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd3a2a22f0 0x55fd3a67a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:48974/0 (socket says 172.21.15.16:48974) 2024-08-31T06:15:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.808+0000 7f7d41c00640 1 -- 172.21.15.16:0/970346758 learned_addr learned my addr 172.21.15.16:0/970346758 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.808+0000 7f7d42600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd3a2cb1b0 0x55fd3a67abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:41336/0 (socket says 172.21.15.16:41336) 2024-08-31T06:15:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.808+0000 7f7d41200640 1 --2- 172.21.15.16:0/970346758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd3a2aa4d0 0x55fd3a67a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.808+0000 7f7d42600640 1 -- 172.21.15.16:0/970346758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd3a2a22f0 msgr2=0x55fd3a67a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.808+0000 7f7d42600640 1 --2- 172.21.15.16:0/970346758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd3a2a22f0 0x55fd3a67a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.809+0000 7f7d42600640 1 -- 172.21.15.16:0/970346758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd3a2aa4d0 msgr2=0x55fd3a67a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.809+0000 7f7d42600640 1 --2- 172.21.15.16:0/970346758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd3a2aa4d0 0x55fd3a67a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.809+0000 7f7d42600640 1 -- 172.21.15.16:0/970346758 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd3a336790 con 0x55fd3a2cb1b0 2024-08-31T06:15:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.809+0000 7f7d41200640 1 --2- 172.21.15.16:0/970346758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd3a2aa4d0 0x55fd3a67a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:15:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.809+0000 7f7d42600640 1 --2- 172.21.15.16:0/970346758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd3a2cb1b0 0x55fd3a67abb0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7d34012f80 tx=0x7f7d34004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5234435c81cd523 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.809+0000 7f7d23400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:15:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.809+0000 7f7d23400640 1 -- 172.21.15.16:0/970346758 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3402d060 con 0x55fd3a2cb1b0 2024-08-31T06:15:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.810+0000 7f7d4b7c4740 1 -- 172.21.15.16:0/970346758 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d300aaff0 con 0x55fd3a2cb1b0 2024-08-31T06:15:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.809+0000 7f7d23400640 1 -- 172.21.15.16:0/970346758 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8470+0+0 (secure 0 0 0) 0x7f7d34018030 con 0x55fd3a2cb1b0 2024-08-31T06:15:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.810+0000 7f7d23400640 1 -- 172.21.15.16:0/970346758 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f7d34011e60 con 0x55fd3a2cb1b0 2024-08-31T06:15:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.811+0000 7f7d4b7c4740 10 client.27548 fetch_fsmap learned FSMap version 42 2024-08-31T06:15:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.811+0000 7f7d4b7c4740 1 -- 172.21.15.16:0/970346758 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55fd3a3af620 con 0x55fd3a2cb1b0 2024-08-31T06:15:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.812+0000 7f7d23400640 1 -- 172.21.15.16:0/970346758 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d300aaff0 con 0x55fd3a2cb1b0 2024-08-31T06:15:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.812+0000 7f7d23400640 1 -- 172.21.15.16:0/970346758 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7f7d3400f040 con 0x55fd3a2cb1b0 2024-08-31T06:15:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.812+0000 7f7d4b7c4740 10 client.27548 fetch_fsmap finished waiting for FSMap version 42 2024-08-31T06:15:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.812+0000 7f7d4b7c4740 10 client.27548 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:15:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.812+0000 7f7d4b7c4740 20 client.27548 populate_metadata read hostname 'smithi016' 2024-08-31T06:15:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.812+0000 7f7d4b7c4740 1 --2- 172.21.15.16:0/970346758 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fd3a289d80 0x55fd3a68c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.812+0000 7f7d4b7c4740 4 client.27548 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:15:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.812+0000 7f7d4b7c4740 1 -- 172.21.15.16:0/970346758 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd3a5e5ec0 con 0x55fd3a289d80 2024-08-31T06:15:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.813+0000 7f7d41c00640 1 --2- 172.21.15.16:0/970346758 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fd3a289d80 0x55fd3a68c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.814+0000 7f7d23400640 10 client.27548 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:15:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.815+0000 7f7d41c00640 1 --2- 172.21.15.16:0/970346758 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fd3a289d80 0x55fd3a68c3d0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f40282be51a512ff server_cookie=bf25ab0c876dea05 in_seq=0 out_seq=0 2024-08-31T06:15:51.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.816+0000 7f7d23400640 1 -- 172.21.15.16:0/970346758 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fd3a5e5ec0 con 0x55fd3a289d80 2024-08-31T06:15:51.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.816+0000 7f7d23400640 10 client.27548 handle_command_reply: tid=0 2024-08-31T06:15:51.817 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.816+0000 7f7d4b7c4740 1 client.27548 shutdown 2024-08-31T06:15:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.817+0000 7f7d4b7c4740 1 -- 172.21.15.16:0/970346758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd3a2cb1b0 msgr2=0x55fd3a67abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.817+0000 7f7d4b7c4740 1 --2- 172.21.15.16:0/970346758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd3a2cb1b0 0x55fd3a67abb0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7d34012f80 tx=0x7f7d34004ab0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.817+0000 7f7d4b7c4740 1 -- 172.21.15.16:0/970346758 shutdown_connections 2024-08-31T06:15:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.817+0000 7f7d4b7c4740 1 --2- 172.21.15.16:0/970346758 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fd3a289d80 0x55fd3a68c3d0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.817+0000 7f7d4b7c4740 1 --2- 172.21.15.16:0/970346758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd3a2cb1b0 0x55fd3a67abb0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.817+0000 7f7d4b7c4740 1 --2- 172.21.15.16:0/970346758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd3a2aa4d0 0x55fd3a67a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.817+0000 7f7d4b7c4740 1 --2- 172.21.15.16:0/970346758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd3a2a22f0 0x55fd3a67a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.817+0000 7f7d4b7c4740 1 -- 172.21.15.16:0/970346758 >> 172.21.15.16:0/970346758 conn(0x55fd3a2bf300 msgr2=0x55fd3a66fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.817+0000 7f7d23400640 0 client.27548 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:15:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.818+0000 7f7d4b7c4740 1 -- 172.21.15.16:0/970346758 shutdown_connections 2024-08-31T06:15:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.818+0000 7f7d4b7c4740 1 -- 172.21.15.16:0/970346758 wait complete. 2024-08-31T06:15:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.821+0000 7f7d4b7c4740 20 client.27548 trim_cache size 0 max 16384 2024-08-31T06:15:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.846+0000 7f7d4b7c4740 1 Processor -- start 2024-08-31T06:15:51.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.846+0000 7f7d4b7c4740 1 -- start start 2024-08-31T06:15:51.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.847+0000 7f7d4b7c4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd3a2a22f0 0x55fd3a6704f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.847+0000 7f7d4b7c4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd3a2aa4d0 0x55fd3a67bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.847+0000 7f7d4b7c4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd3a2cb1b0 0x55fd3a6772e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:51.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.849+0000 7f7d4b7c4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c1b82c0 con 0x55fd3a2cb1b0 2024-08-31T06:15:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.849+0000 7f7d4b7c4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c1b5b00 con 0x55fd3a2aa4d0 2024-08-31T06:15:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.849+0000 7f7d4b7c4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c1b3970 con 0x55fd3a2a22f0 2024-08-31T06:15:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.849+0000 7f7d41200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd3a2aa4d0 0x55fd3a67bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.848+0000 7f7d41c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd3a2a22f0 0x55fd3a6704f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.849+0000 7f7d41c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd3a2a22f0 0x55fd3a6704f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:48990/0 (socket says 172.21.15.16:48990) 2024-08-31T06:15:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.849+0000 7f7d41c00640 1 -- 172.21.15.16:0/4280392340 learned_addr learned my addr 172.21.15.16:0/4280392340 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.849+0000 7f7d41200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd3a2aa4d0 0x55fd3a67bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:41348/0 (socket says 172.21.15.16:41348) 2024-08-31T06:15:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.849+0000 7f7d42600640 1 --2- 172.21.15.16:0/4280392340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd3a2cb1b0 0x55fd3a6772e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.849+0000 7f7d41c00640 1 -- 172.21.15.16:0/4280392340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd3a2aa4d0 msgr2=0x55fd3a67bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.849+0000 7f7d41c00640 1 --2- 172.21.15.16:0/4280392340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd3a2aa4d0 0x55fd3a67bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.849+0000 7f7d41c00640 1 -- 172.21.15.16:0/4280392340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd3a2cb1b0 msgr2=0x55fd3a6772e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.849+0000 7f7d41c00640 1 --2- 172.21.15.16:0/4280392340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd3a2cb1b0 0x55fd3a6772e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.850+0000 7f7d41c00640 1 -- 172.21.15.16:0/4280392340 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd3a6864f0 con 0x55fd3a2a22f0 2024-08-31T06:15:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.850+0000 7f7d41c00640 1 --2- 172.21.15.16:0/4280392340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd3a2a22f0 0x55fd3a6704f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7d300a7560 tx=0x7f7d300e8b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f46f65eb295e83cb server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.850+0000 7f7d2e000640 1 -- 172.21.15.16:0/4280392340 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3005b030 con 0x55fd3a2a22f0 2024-08-31T06:15:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.850+0000 7f7d2e000640 1 -- 172.21.15.16:0/4280392340 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d300aaff0 con 0x55fd3a2a22f0 2024-08-31T06:15:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.850+0000 7f7d2e000640 1 -- 172.21.15.16:0/4280392340 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d30020040 con 0x55fd3a2a22f0 2024-08-31T06:15:51.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.851+0000 7f7d4b7c4740 1 -- 172.21.15.16:0/4280392340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd3a2a22f0 msgr2=0x55fd3a6704f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:51.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.851+0000 7f7d4b7c4740 1 --2- 172.21.15.16:0/4280392340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd3a2a22f0 0x55fd3a6704f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7d300a7560 tx=0x7f7d300e8b20 comp rx=0 tx=0).stop 2024-08-31T06:15:51.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.852+0000 7f7d4b7c4740 1 -- 172.21.15.16:0/4280392340 shutdown_connections 2024-08-31T06:15:51.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.852+0000 7f7d4b7c4740 1 --2- 172.21.15.16:0/4280392340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd3a2cb1b0 0x55fd3a6772e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.852+0000 7f7d4b7c4740 1 --2- 172.21.15.16:0/4280392340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd3a2aa4d0 0x55fd3a67bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.852+0000 7f7d4b7c4740 1 --2- 172.21.15.16:0/4280392340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd3a2a22f0 0x55fd3a6704f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.852+0000 7f7d4b7c4740 1 -- 172.21.15.16:0/4280392340 >> 172.21.15.16:0/4280392340 conn(0x55fd3a2bf300 msgr2=0x55fd3a67b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:51.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.852+0000 7f7d4b7c4740 1 -- 172.21.15.16:0/4280392340 shutdown_connections 2024-08-31T06:15:51.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.852+0000 7f7d4b7c4740 1 -- 172.21.15.16:0/4280392340 wait complete. 2024-08-31T06:15:51.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.853+0000 7f7d4b7c4740 1 -- start start 2024-08-31T06:15:51.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.853+0000 7f7d4b7c4740 1 Processor -- start 2024-08-31T06:15:51.856 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.854+0000 7f7d4b7c4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd3a2a22f0 0x55fd3a682d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:51.856 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.854+0000 7f7d4b7c4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd3a2aa4d0 0x55fd3a680220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:51.856 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.854+0000 7f7d4b7c4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd3a2cb1b0 0x55fd3a680780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:51.856 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.854+0000 7f7d4b7c4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c1b3fc0 con 0x55fd3a2aa4d0 2024-08-31T06:15:51.856 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.854+0000 7f7d4b7c4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c1b5430 con 0x55fd3a2cb1b0 2024-08-31T06:15:51.856 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.854+0000 7f7d4b7c4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c1b4690 con 0x55fd3a2a22f0 2024-08-31T06:15:51.856 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.854+0000 7f7d4b7c4740 10 client.0 _ll_register_callbacks cb 0x7f7d34011710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:15:51.856 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.855+0000 7f7d41c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd3a2a22f0 0x55fd3a682d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:51.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.855+0000 7f7d42600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd3a2cb1b0 0x55fd3a680780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:51.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.855+0000 7f7d42600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd3a2cb1b0 0x55fd3a680780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:41352/0 (socket says 172.21.15.16:41352) 2024-08-31T06:15:51.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.855+0000 7f7d42600640 1 -- 172.21.15.16:0/208664667 learned_addr learned my addr 172.21.15.16:0/208664667 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:51.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.855+0000 7f7d41200640 1 --2- 172.21.15.16:0/208664667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd3a2aa4d0 0x55fd3a680220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:51.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.857+0000 7f7d41c00640 1 -- 172.21.15.16:0/208664667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd3a2cb1b0 msgr2=0x55fd3a680780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:51.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.857+0000 7f7d41c00640 1 --2- 172.21.15.16:0/208664667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd3a2cb1b0 0x55fd3a680780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.857+0000 7f7d41c00640 1 -- 172.21.15.16:0/208664667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd3a2aa4d0 msgr2=0x55fd3a680220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:51.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.857+0000 7f7d41c00640 1 --2- 172.21.15.16:0/208664667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd3a2aa4d0 0x55fd3a680220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.857+0000 7f7d41c00640 1 -- 172.21.15.16:0/208664667 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd3a680f10 con 0x55fd3a2a22f0 2024-08-31T06:15:51.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.857+0000 7f7d42600640 1 --2- 172.21.15.16:0/208664667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd3a2cb1b0 0x55fd3a680780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:15:51.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.857+0000 7f7d41200640 1 --2- 172.21.15.16:0/208664667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd3a2aa4d0 0x55fd3a680220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:15:51.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.858+0000 7f7d41c00640 1 --2- 172.21.15.16:0/208664667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd3a2a22f0 0x55fd3a682d60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7d300ed1b0 tx=0x7f7d300ed6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8124330ada97708b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:51.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.858+0000 7f7d23400640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:15:51.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.858+0000 7f7d23400640 1 -- 172.21.15.16:0/208664667 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d30020050 con 0x55fd3a2a22f0 2024-08-31T06:15:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.858+0000 7f7d23400640 1 -- 172.21.15.16:0/208664667 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8470+0+0 (secure 0 0 0) 0x7f7d300a7ad0 con 0x55fd3a2a22f0 2024-08-31T06:15:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.858+0000 7f7d4b7c4740 1 -- 172.21.15.16:0/208664667 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d300aaff0 con 0x55fd3a2a22f0 2024-08-31T06:15:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.859+0000 7f7d23400640 1 -- 172.21.15.16:0/208664667 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f7d3005b040 con 0x55fd3a2a22f0 2024-08-31T06:15:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.859+0000 7f7d4b7c4740 10 client.27541 fetch_fsmap learned FSMap version 42 2024-08-31T06:15:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.859+0000 7f7d4b7c4740 1 -- 172.21.15.16:0/208664667 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55fd3a707d60 con 0x55fd3a2a22f0 2024-08-31T06:15:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.859+0000 7f7d23400640 1 -- 172.21.15.16:0/208664667 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d300aaff0 con 0x55fd3a2a22f0 2024-08-31T06:15:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.859+0000 7f7d23400640 1 -- 172.21.15.16:0/208664667 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7f7d300e27c0 con 0x55fd3a2a22f0 2024-08-31T06:15:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.859+0000 7f7d4b7c4740 10 client.27541 fetch_fsmap finished waiting for FSMap version 42 2024-08-31T06:15:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.859+0000 7f7d4b7c4740 10 client.27541 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:15:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.859+0000 7f7d4b7c4740 20 client.27541 populate_metadata read hostname 'smithi016' 2024-08-31T06:15:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.859+0000 7f7d4b7c4740 1 --2- 172.21.15.16:0/208664667 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fd3a25f1c0 0x55fd3a709f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:51.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.859+0000 7f7d4b7c4740 4 client.27541 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:15:51.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.859+0000 7f7d4b7c4740 1 -- 172.21.15.16:0/208664667 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fd3a5e1960 con 0x55fd3a25f1c0 2024-08-31T06:15:51.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.860+0000 7f7d41200640 1 --2- 172.21.15.16:0/208664667 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fd3a25f1c0 0x55fd3a709f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:51.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.861+0000 7f7d23400640 10 client.27541 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:15:51.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.861+0000 7f7d41200640 1 --2- 172.21.15.16:0/208664667 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fd3a25f1c0 0x55fd3a709f40 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=870bbbd18a82d520 server_cookie=8f72f7ab4bfab8c3 in_seq=0 out_seq=0 2024-08-31T06:15:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.880+0000 7f7d23400640 1 -- 172.21.15.16:0/208664667 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fd3a5e1960 con 0x55fd3a25f1c0 2024-08-31T06:15:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.880+0000 7f7d23400640 10 client.27541 handle_command_reply: tid=0 2024-08-31T06:15:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.880+0000 7f7d4b7c4740 1 client.27541 shutdown 2024-08-31T06:15:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.880+0000 7f7d4b7c4740 1 -- 172.21.15.16:0/208664667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd3a2a22f0 msgr2=0x55fd3a682d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.880+0000 7f7d4b7c4740 1 --2- 172.21.15.16:0/208664667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd3a2a22f0 0x55fd3a682d60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7d300ed1b0 tx=0x7f7d300ed6f0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.880+0000 7f7d4b7c4740 1 -- 172.21.15.16:0/208664667 shutdown_connections 2024-08-31T06:15:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.880+0000 7f7d4b7c4740 1 --2- 172.21.15.16:0/208664667 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fd3a25f1c0 0x55fd3a709f40 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.880+0000 7f7d4b7c4740 1 --2- 172.21.15.16:0/208664667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd3a2cb1b0 0x55fd3a680780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.880+0000 7f7d4b7c4740 1 --2- 172.21.15.16:0/208664667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fd3a2aa4d0 0x55fd3a680220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.880+0000 7f7d4b7c4740 1 --2- 172.21.15.16:0/208664667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fd3a2a22f0 0x55fd3a682d60 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.880+0000 7f7d4b7c4740 1 -- 172.21.15.16:0/208664667 >> 172.21.15.16:0/208664667 conn(0x55fd3a2bf300 msgr2=0x55fd3a6702c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.881+0000 7f7d23400640 0 client.27541 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:15:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.881+0000 7f7d4b7c4740 1 -- 172.21.15.16:0/208664667 shutdown_connections 2024-08-31T06:15:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.881+0000 7f7d4b7c4740 1 -- 172.21.15.16:0/208664667 wait complete. 2024-08-31T06:15:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.881+0000 7f7d4b7c4740 20 client.27541 trim_cache size 0 max 16384 2024-08-31T06:15:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:15:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:15:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:15:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:15:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.882+0000 7f7d43000640 1 -- 172.21.15.16:0/1217851074 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7d240b4d50 msgr2=0x7f7d240b7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.882+0000 7f7d43000640 1 --2- 172.21.15.16:0/1217851074 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7d240b4d50 0x7f7d240b7260 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7d34009270 tx=0x7f7d34009680 comp rx=0 tx=0).stop 2024-08-31T06:15:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.882+0000 7f7d43000640 1 -- 172.21.15.16:0/1217851074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d3c0776e0 msgr2=0x7f7d3c1a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.882+0000 7f7d43000640 1 --2- 172.21.15.16:0/1217851074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d3c0776e0 0x7f7d3c1a0610 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7d3000d6d0 tx=0x7f7d3000db90 comp rx=0 tx=0).stop 2024-08-31T06:15:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.882+0000 7f7d43000640 1 -- 172.21.15.16:0/1217851074 shutdown_connections 2024-08-31T06:15:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.882+0000 7f7d43000640 1 --2- 172.21.15.16:0/1217851074 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7d240b4d50 0x7f7d240b7260 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.882+0000 7f7d43000640 1 --2- 172.21.15.16:0/1217851074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d3c114260 0x7f7d3c1a5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.882+0000 7f7d43000640 1 --2- 172.21.15.16:0/1217851074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d3c078030 0x7f7d3c1a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.882+0000 7f7d43000640 1 --2- 172.21.15.16:0/1217851074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d3c0776e0 0x7f7d3c1a0610 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.882+0000 7f7d43000640 1 -- 172.21.15.16:0/1217851074 >> 172.21.15.16:0/1217851074 conn(0x7f7d3c101290 msgr2=0x7f7d3c1036b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.883+0000 7f7d43000640 1 -- 172.21.15.16:0/1217851074 shutdown_connections 2024-08-31T06:15:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:15:51.883+0000 7f7d43000640 1 -- 172.21.15.16:0/1217851074 wait complete. 2024-08-31T06:15:51.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:51 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:15:51.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:51 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:15:52.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:51 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:15:52.208 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T06:15:52.927 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:15:52.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:52 smithi184 ceph-mon[22331]: pgmap v629: 97 pgs: 97 active+clean; 3.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 7.5 MiB/s wr, 722 op/s 2024-08-31T06:15:53.097 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:52 smithi016 ceph-mon[17194]: pgmap v629: 97 pgs: 97 active+clean; 3.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 7.5 MiB/s wr, 722 op/s 2024-08-31T06:15:53.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:52 smithi155 ceph-mon[21742]: pgmap v629: 97 pgs: 97 active+clean; 3.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 7.5 MiB/s wr, 722 op/s 2024-08-31T06:15:53.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.772+0000 7f408fe00640 1 -- 172.21.15.16:0/2624156016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4090105960 msgr2=0x7f4090105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:53.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.772+0000 7f408fe00640 1 --2- 172.21.15.16:0/2624156016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4090105960 0x7f4090105d70 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f4080009900 tx=0x7f408002a3f0 comp rx=0 tx=0).stop 2024-08-31T06:15:53.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.773+0000 7f408fe00640 1 -- 172.21.15.16:0/2624156016 shutdown_connections 2024-08-31T06:15:53.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.773+0000 7f408fe00640 1 --2- 172.21.15.16:0/2624156016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f409010bbf0 0x7f4090112420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:53.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.773+0000 7f408fe00640 1 --2- 172.21.15.16:0/2624156016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f409010b2a0 0x7f409010b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:53.777 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:53 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:15:53.777 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:53 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:15:53.777 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:53 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:15:53.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.773+0000 7f408fe00640 1 --2- 172.21.15.16:0/2624156016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4090105960 0x7f4090105d70 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:53.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.773+0000 7f408fe00640 1 -- 172.21.15.16:0/2624156016 >> 172.21.15.16:0/2624156016 conn(0x7f40901013d0 msgr2=0x7f4090103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:53.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.774+0000 7f408fe00640 1 -- 172.21.15.16:0/2624156016 shutdown_connections 2024-08-31T06:15:53.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.774+0000 7f408fe00640 1 -- 172.21.15.16:0/2624156016 wait complete. 2024-08-31T06:15:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.778+0000 7f408fe00640 1 Processor -- start 2024-08-31T06:15:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.778+0000 7f408fe00640 1 -- start start 2024-08-31T06:15:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.779+0000 7f408fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4090105960 0x7f409019c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:53.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.779+0000 7f408fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f409010b2a0 0x7f409019c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:53.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.779+0000 7f408fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f409010bbf0 0x7f40901a1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:53.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.780+0000 7f408e000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f409010b2a0 0x7f409019c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:53.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.780+0000 7f408e000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f409010b2a0 0x7f409019c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:41368/0 (socket says 172.21.15.16:41368) 2024-08-31T06:15:53.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.780+0000 7f408f400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f409010bbf0 0x7f40901a1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:53.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.780+0000 7f408f400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f409010bbf0 0x7f40901a1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:49020/0 (socket says 172.21.15.16:49020) 2024-08-31T06:15:53.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.781+0000 7f408fe00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4090115900 con 0x7f4090105960 2024-08-31T06:15:53.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.781+0000 7f408fe00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4090115780 con 0x7f409010b2a0 2024-08-31T06:15:53.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.781+0000 7f408fe00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4090115a80 con 0x7f409010bbf0 2024-08-31T06:15:53.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.781+0000 7f408e000640 1 -- 172.21.15.16:0/2966190877 learned_addr learned my addr 172.21.15.16:0/2966190877 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:53.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.782+0000 7f408f400640 1 -- 172.21.15.16:0/2966190877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f409010b2a0 msgr2=0x7f409019c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:53.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.782+0000 7f408f400640 1 --2- 172.21.15.16:0/2966190877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f409010b2a0 0x7f409019c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:53.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.782+0000 7f408f400640 1 -- 172.21.15.16:0/2966190877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4090105960 msgr2=0x7f409019c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:53.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.782+0000 7f408f400640 1 --2- 172.21.15.16:0/2966190877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4090105960 0x7f409019c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:53.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.782+0000 7f408f400640 1 -- 172.21.15.16:0/2966190877 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40901a2280 con 0x7f409010bbf0 2024-08-31T06:15:53.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.783+0000 7f408e000640 1 --2- 172.21.15.16:0/2966190877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f409010b2a0 0x7f409019c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:15:53.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.784+0000 7f408f400640 1 --2- 172.21.15.16:0/2966190877 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f409010bbf0 0x7f40901a1b50 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4088009fd0 tx=0x7f408800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:53.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.785+0000 7f4087400640 1 -- 172.21.15.16:0/2966190877 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408800ce20 con 0x7f409010bbf0 2024-08-31T06:15:53.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.786+0000 7f4087400640 1 -- 172.21.15.16:0/2966190877 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4088005020 con 0x7f409010bbf0 2024-08-31T06:15:53.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.786+0000 7f4087400640 1 -- 172.21.15.16:0/2966190877 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4088010690 con 0x7f409010bbf0 2024-08-31T06:15:53.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.786+0000 7f408fe00640 1 -- 172.21.15.16:0/2966190877 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40901a2570 con 0x7f409010bbf0 2024-08-31T06:15:53.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.786+0000 7f408fe00640 1 -- 172.21.15.16:0/2966190877 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40901a8cd0 con 0x7f409010bbf0 2024-08-31T06:15:53.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.793+0000 7f409c823740 1 Processor -- start 2024-08-31T06:15:53.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.794+0000 7f4087400640 1 -- 172.21.15.16:0/2966190877 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f40880054d0 con 0x7f409010bbf0 2024-08-31T06:15:53.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.795+0000 7f409c823740 1 -- start start 2024-08-31T06:15:53.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.795+0000 7f409c823740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e618660070 0x55e618a0e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:53.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.795+0000 7f409c823740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6186371b0 0x55e618a0e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:53.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.795+0000 7f409c823740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e61863f390 0x55e618a0edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:53.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.795+0000 7f409c823740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e61897c7b0 con 0x55e6186371b0 2024-08-31T06:15:53.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.795+0000 7f409c823740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e61896bd40 con 0x55e618660070 2024-08-31T06:15:53.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.795+0000 7f409c823740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e61896fcf0 con 0x55e61863f390 2024-08-31T06:15:53.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.796+0000 7f408e000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e618660070 0x55e618a0e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:53.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.796+0000 7f408e000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e618660070 0x55e618a0e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:41376/0 (socket says 172.21.15.16:41376) 2024-08-31T06:15:53.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.796+0000 7f408e000640 1 -- 172.21.15.16:0/920282826 learned_addr learned my addr 172.21.15.16:0/920282826 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:53.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.796+0000 7f408e000640 1 --2- 172.21.15.16:0/920282826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e61863f390 0x55e618a0edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:53.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.797+0000 7f4087400640 1 --2- 172.21.15.16:0/2966190877 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f40740b4eb0 0x7f40740b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:53.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.797+0000 7f4087400640 1 -- 172.21.15.16:0/2966190877 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8470+0+0 (secure 0 0 0) 0x7f40880e5bd0 con 0x7f409010bbf0 2024-08-31T06:15:53.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.797+0000 7f408ea00640 1 --2- 172.21.15.16:0/920282826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6186371b0 0x55e618a0e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:53.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.797+0000 7f408e000640 1 -- 172.21.15.16:0/920282826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e61863f390 msgr2=0x55e618a0edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:53.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.797+0000 7f408e000640 1 --2- 172.21.15.16:0/920282826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e61863f390 0x55e618a0edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:53.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.797+0000 7f408e000640 1 -- 172.21.15.16:0/920282826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6186371b0 msgr2=0x55e618a0e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:53.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.797+0000 7f408e000640 1 --2- 172.21.15.16:0/920282826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6186371b0 0x55e618a0e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:53.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.798+0000 7f408e000640 1 -- 172.21.15.16:0/920282826 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6186cb650 con 0x55e618660070 2024-08-31T06:15:53.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.799+0000 7f408f400640 1 --2- 172.21.15.16:0/2966190877 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f40740b4eb0 0x7f40740b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:53.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.799+0000 7f408e000640 1 --2- 172.21.15.16:0/920282826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e618660070 0x55e618a0e370 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f407c013860 tx=0x7f407c009740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb678573a539bcf4 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:53.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.799+0000 7f407be00640 1 -- 172.21.15.16:0/920282826 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f407c0130d0 con 0x55e618660070 2024-08-31T06:15:53.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.800+0000 7f407be00640 1 -- 172.21.15.16:0/920282826 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f407c00fc10 con 0x55e618660070 2024-08-31T06:15:53.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.801+0000 7f407be00640 1 -- 172.21.15.16:0/920282826 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f407c011a70 con 0x55e618660070 2024-08-31T06:15:53.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.801+0000 7f409c823740 1 -- 172.21.15.16:0/920282826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e618660070 msgr2=0x55e618a0e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:53.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.801+0000 7f409c823740 1 --2- 172.21.15.16:0/920282826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e618660070 0x55e618a0e370 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f407c013860 tx=0x7f407c009740 comp rx=0 tx=0).stop 2024-08-31T06:15:53.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.801+0000 7f408f400640 1 --2- 172.21.15.16:0/2966190877 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f40740b4eb0 0x7f40740b73c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f408800c760 tx=0x7f4088005f70 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:53.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.801+0000 7f409c823740 1 -- 172.21.15.16:0/920282826 shutdown_connections 2024-08-31T06:15:53.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.801+0000 7f409c823740 1 --2- 172.21.15.16:0/920282826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e61863f390 0x55e618a0edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:53.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.801+0000 7f409c823740 1 --2- 172.21.15.16:0/920282826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6186371b0 0x55e618a0e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:53.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.801+0000 7f409c823740 1 --2- 172.21.15.16:0/920282826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e618660070 0x55e618a0e370 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:53.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.801+0000 7f409c823740 1 -- 172.21.15.16:0/920282826 >> 172.21.15.16:0/920282826 conn(0x55e618654460 msgr2=0x55e618a0c260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:53.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.802+0000 7f409c823740 1 -- 172.21.15.16:0/920282826 shutdown_connections 2024-08-31T06:15:53.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.802+0000 7f409c823740 1 -- 172.21.15.16:0/920282826 wait complete. 2024-08-31T06:15:53.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.803+0000 7f409c823740 1 -- start start 2024-08-31T06:15:53.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.803+0000 7f409c823740 1 Processor -- start 2024-08-31T06:15:53.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.804+0000 7f409c823740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6186371b0 0x55e618a19f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:53.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.804+0000 7f409c823740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e61863f390 0x55e618a1a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:53.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.804+0000 7f409c823740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e618660070 0x55e618a25cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:53.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.804+0000 7f409c823740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e6187ce700 con 0x55e61863f390 2024-08-31T06:15:53.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.804+0000 7f409c823740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e6188f2cf0 con 0x55e6186371b0 2024-08-31T06:15:53.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.804+0000 7f409c823740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e618a1c980 con 0x55e618660070 2024-08-31T06:15:53.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.805+0000 7f408ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e61863f390 0x55e618a1a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:53.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.806+0000 7f408ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e61863f390 0x55e618a1a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57158/0 (socket says 172.21.15.16:57158) 2024-08-31T06:15:53.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.806+0000 7f408ea00640 1 -- 172.21.15.16:0/4163170121 learned_addr learned my addr 172.21.15.16:0/4163170121 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:53.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.806+0000 7f409c823740 10 client.0 _ll_register_callbacks cb 0x55e618918470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:15:53.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.806+0000 7f408e000640 1 --2- 172.21.15.16:0/4163170121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e618660070 0x55e618a25cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:53.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.806+0000 7f408e000640 1 --2- 172.21.15.16:0/4163170121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6186371b0 0x55e618a19f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:53.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.807+0000 7f408ea00640 1 -- 172.21.15.16:0/4163170121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e618660070 msgr2=0x55e618a25cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:53.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.807+0000 7f408ea00640 1 --2- 172.21.15.16:0/4163170121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e618660070 0x55e618a25cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:53.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.807+0000 7f408ea00640 1 -- 172.21.15.16:0/4163170121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6186371b0 msgr2=0x55e618a19f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:53.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.807+0000 7f408ea00640 1 --2- 172.21.15.16:0/4163170121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6186371b0 0x55e618a19f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:53.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.807+0000 7f408ea00640 1 -- 172.21.15.16:0/4163170121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6189338b0 con 0x55e61863f390 2024-08-31T06:15:53.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.808+0000 7f408e000640 1 --2- 172.21.15.16:0/4163170121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6186371b0 0x55e618a19f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:15:53.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.808+0000 7f408ea00640 1 --2- 172.21.15.16:0/4163170121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e61863f390 0x55e618a1a460 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f4080009230 tx=0x7f4080004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=777ef381014f291b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:53.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.808+0000 7f4078c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:15:53.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.809+0000 7f4078c00640 1 -- 172.21.15.16:0/4163170121 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408003f070 con 0x55e61863f390 2024-08-31T06:15:53.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.809+0000 7f4078c00640 1 -- 172.21.15.16:0/4163170121 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8470+0+0 (secure 0 0 0) 0x7f4080005240 con 0x55e61863f390 2024-08-31T06:15:53.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.809+0000 7f409c823740 1 -- 172.21.15.16:0/4163170121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f407c00fc10 con 0x55e61863f390 2024-08-31T06:15:53.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.809+0000 7f4078c00640 1 -- 172.21.15.16:0/4163170121 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f408004b050 con 0x55e61863f390 2024-08-31T06:15:53.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.809+0000 7f409c823740 10 client.17304 fetch_fsmap learned FSMap version 42 2024-08-31T06:15:53.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.809+0000 7f409c823740 1 -- 172.21.15.16:0/4163170121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55e618a28990 con 0x55e61863f390 2024-08-31T06:15:53.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.810+0000 7f4078c00640 1 -- 172.21.15.16:0/4163170121 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f407c00fc10 con 0x55e61863f390 2024-08-31T06:15:53.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.810+0000 7f4078c00640 1 -- 172.21.15.16:0/4163170121 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7f40800035f0 con 0x55e61863f390 2024-08-31T06:15:53.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.810+0000 7f409c823740 10 client.17304 fetch_fsmap finished waiting for FSMap version 42 2024-08-31T06:15:53.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.810+0000 7f409c823740 10 client.17304 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:15:53.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.810+0000 7f409c823740 20 client.17304 populate_metadata read hostname 'smithi016' 2024-08-31T06:15:53.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.810+0000 7f409c823740 1 --2- 172.21.15.16:0/4163170121 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55e61861e9f0 0x55e618a2beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:53.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.810+0000 7f409c823740 4 client.17304 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:15:53.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.810+0000 7f409c823740 1 -- 172.21.15.16:0/4163170121 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e61897e700 con 0x55e61861e9f0 2024-08-31T06:15:53.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.811+0000 7f408e000640 1 --2- 172.21.15.16:0/4163170121 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55e61861e9f0 0x55e618a2beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:53.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.811+0000 7f408e000640 1 --2- 172.21.15.16:0/4163170121 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55e61861e9f0 0x55e618a2beb0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad8b739d067b8196 server_cookie=f15a9927a093459f in_seq=0 out_seq=0 2024-08-31T06:15:53.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.811+0000 7f4078c00640 10 client.17304 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:15:53.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.812+0000 7f4078c00640 1 -- 172.21.15.16:0/4163170121 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e61897e700 con 0x55e61861e9f0 2024-08-31T06:15:53.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.812+0000 7f4078c00640 10 client.17304 handle_command_reply: tid=0 2024-08-31T06:15:53.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.813+0000 7f409c823740 1 client.17304 shutdown 2024-08-31T06:15:53.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.814+0000 7f409c823740 1 -- 172.21.15.16:0/4163170121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e61863f390 msgr2=0x55e618a1a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:53.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.814+0000 7f409c823740 1 --2- 172.21.15.16:0/4163170121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e61863f390 0x55e618a1a460 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f4080009230 tx=0x7f4080004270 comp rx=0 tx=0).stop 2024-08-31T06:15:53.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.814+0000 7f409c823740 1 -- 172.21.15.16:0/4163170121 shutdown_connections 2024-08-31T06:15:53.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.814+0000 7f409c823740 1 --2- 172.21.15.16:0/4163170121 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55e61861e9f0 0x55e618a2beb0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:53.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.814+0000 7f409c823740 1 --2- 172.21.15.16:0/4163170121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e618660070 0x55e618a25cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:53.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.814+0000 7f409c823740 1 --2- 172.21.15.16:0/4163170121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e61863f390 0x55e618a1a460 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:53.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.814+0000 7f409c823740 1 --2- 172.21.15.16:0/4163170121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6186371b0 0x55e618a19f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:53.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.814+0000 7f409c823740 1 -- 172.21.15.16:0/4163170121 >> 172.21.15.16:0/4163170121 conn(0x55e618654460 msgr2=0x55e618a11550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:53.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.814+0000 7f4078c00640 0 client.17304 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:15:53.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.814+0000 7f409c823740 1 -- 172.21.15.16:0/4163170121 shutdown_connections 2024-08-31T06:15:53.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.815+0000 7f409c823740 1 -- 172.21.15.16:0/4163170121 wait complete. 2024-08-31T06:15:53.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.815+0000 7f409c823740 20 client.17304 trim_cache size 0 max 16384 2024-08-31T06:15:53.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.833+0000 7f409c823740 1 Processor -- start 2024-08-31T06:15:53.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.833+0000 7f409c823740 1 -- start start 2024-08-31T06:15:53.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.834+0000 7f409c823740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e6186371b0 0x55e618a154d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:53.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.834+0000 7f409c823740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e618660070 0x55e618a0fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:53.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.834+0000 7f409c823740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6185f3da0 0x55e618a25c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:53.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.834+0000 7f409c823740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f40900736e0 con 0x55e6185f3da0 2024-08-31T06:15:53.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.834+0000 7f409c823740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4090070f20 con 0x55e618660070 2024-08-31T06:15:53.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.834+0000 7f409c823740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f409006ed90 con 0x55e6186371b0 2024-08-31T06:15:53.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.834+0000 7f408ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e618660070 0x55e618a0fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:53.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.834+0000 7f408ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e618660070 0x55e618a0fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:41392/0 (socket says 172.21.15.16:41392) 2024-08-31T06:15:53.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.834+0000 7f408ea00640 1 -- 172.21.15.16:0/3529936028 learned_addr learned my addr 172.21.15.16:0/3529936028 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:53.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.834+0000 7f408e000640 1 --2- 172.21.15.16:0/3529936028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e6186371b0 0x55e618a154d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:53.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.835+0000 7f408e000640 1 --2- 172.21.15.16:0/3529936028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6185f3da0 0x55e618a25c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:53.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.835+0000 7f408ea00640 1 -- 172.21.15.16:0/3529936028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e6186371b0 msgr2=0x55e618a154d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:53.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.835+0000 7f408ea00640 1 --2- 172.21.15.16:0/3529936028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e6186371b0 0x55e618a154d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:53.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.835+0000 7f408ea00640 1 -- 172.21.15.16:0/3529936028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6185f3da0 msgr2=0x55e618a25c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:53.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.835+0000 7f408ea00640 1 --2- 172.21.15.16:0/3529936028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6185f3da0 0x55e618a25c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:53.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.835+0000 7f408ea00640 1 -- 172.21.15.16:0/3529936028 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e61861ec20 con 0x55e618660070 2024-08-31T06:15:53.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.835+0000 7f408e000640 1 --2- 172.21.15.16:0/3529936028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6185f3da0 0x55e618a25c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:15:53.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.836+0000 7f408ea00640 1 --2- 172.21.15.16:0/3529936028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e618660070 0x55e618a0fab0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4080003a90 tx=0x7f4080005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9945da050cc3fd35 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:53.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.836+0000 7f407b400640 1 -- 172.21.15.16:0/3529936028 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4080053020 con 0x55e618660070 2024-08-31T06:15:53.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.836+0000 7f407b400640 1 -- 172.21.15.16:0/3529936028 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40800037f0 con 0x55e618660070 2024-08-31T06:15:53.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.836+0000 7f407b400640 1 -- 172.21.15.16:0/3529936028 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408003f070 con 0x55e618660070 2024-08-31T06:15:53.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.838+0000 7f409c823740 1 -- 172.21.15.16:0/3529936028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e618660070 msgr2=0x55e618a0fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:53.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.838+0000 7f409c823740 1 --2- 172.21.15.16:0/3529936028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e618660070 0x55e618a0fab0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4080003a90 tx=0x7f4080005fd0 comp rx=0 tx=0).stop 2024-08-31T06:15:53.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.838+0000 7f409c823740 1 -- 172.21.15.16:0/3529936028 shutdown_connections 2024-08-31T06:15:53.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.838+0000 7f409c823740 1 --2- 172.21.15.16:0/3529936028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6185f3da0 0x55e618a25c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:53.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.838+0000 7f409c823740 1 --2- 172.21.15.16:0/3529936028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e618660070 0x55e618a0fab0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:53.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.838+0000 7f409c823740 1 --2- 172.21.15.16:0/3529936028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e6186371b0 0x55e618a154d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:53.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.838+0000 7f409c823740 1 -- 172.21.15.16:0/3529936028 >> 172.21.15.16:0/3529936028 conn(0x55e618654460 msgr2=0x55e618a0f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:53.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.838+0000 7f409c823740 1 -- 172.21.15.16:0/3529936028 shutdown_connections 2024-08-31T06:15:53.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.838+0000 7f409c823740 1 -- 172.21.15.16:0/3529936028 wait complete. 2024-08-31T06:15:53.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.839+0000 7f409c823740 1 -- start start 2024-08-31T06:15:53.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.839+0000 7f409c823740 1 Processor -- start 2024-08-31T06:15:53.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.840+0000 7f409c823740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e6185f3da0 0x55e618a32fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:53.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.840+0000 7f409c823740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6186371b0 0x55e618a2f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:53.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.840+0000 7f409c823740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e618660070 0x55e618a2f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:53.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.840+0000 7f409c823740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f409006f3e0 con 0x55e618660070 2024-08-31T06:15:53.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.840+0000 7f409c823740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4090070850 con 0x55e6186371b0 2024-08-31T06:15:53.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.840+0000 7f409c823740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f409006fab0 con 0x55e6185f3da0 2024-08-31T06:15:53.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.840+0000 7f409c823740 10 client.0 _ll_register_callbacks cb 0x7f4080009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:15:53.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.840+0000 7f408ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6186371b0 0x55e618a2f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:53.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.840+0000 7f408ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6186371b0 0x55e618a2f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:41394/0 (socket says 172.21.15.16:41394) 2024-08-31T06:15:53.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.840+0000 7f408ea00640 1 -- 172.21.15.16:0/3072439221 learned_addr learned my addr 172.21.15.16:0/3072439221 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:53.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.840+0000 7f408e000640 1 --2- 172.21.15.16:0/3072439221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e618660070 0x55e618a2f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:53.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.840+0000 7f408ea00640 1 -- 172.21.15.16:0/3072439221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e6185f3da0 msgr2=0x55e618a32fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T06:15:53.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.841+0000 7f408ea00640 1 --2- 172.21.15.16:0/3072439221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e6185f3da0 0x55e618a32fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:53.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.841+0000 7f408ea00640 1 -- 172.21.15.16:0/3072439221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e618660070 msgr2=0x55e618a2f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:53.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.841+0000 7f408ea00640 1 --2- 172.21.15.16:0/3072439221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e618660070 0x55e618a2f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:53.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.841+0000 7f408ea00640 1 -- 172.21.15.16:0/3072439221 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e618a2fff0 con 0x55e6186371b0 2024-08-31T06:15:53.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.841+0000 7f408e000640 1 --2- 172.21.15.16:0/3072439221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e618660070 0x55e618a2f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:15:53.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.841+0000 7f408ea00640 1 --2- 172.21.15.16:0/3072439221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6186371b0 0x55e618a2f470 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4080003fb0 tx=0x7f40800316d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54a0525a007d396b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:53.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.841+0000 7f4078c00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:15:53.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.841+0000 7f4078c00640 1 -- 172.21.15.16:0/3072439221 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4080053040 con 0x55e6186371b0 2024-08-31T06:15:53.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.842+0000 7f4078c00640 1 -- 172.21.15.16:0/3072439221 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8470+0+0 (secure 0 0 0) 0x7f4080005240 con 0x55e6186371b0 2024-08-31T06:15:53.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.842+0000 7f409c823740 1 -- 172.21.15.16:0/3072439221 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40800037f0 con 0x55e6186371b0 2024-08-31T06:15:53.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.842+0000 7f4078c00640 1 -- 172.21.15.16:0/3072439221 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f408003f040 con 0x55e6186371b0 2024-08-31T06:15:53.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.842+0000 7f409c823740 10 client.27572 fetch_fsmap learned FSMap version 42 2024-08-31T06:15:53.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.842+0000 7f409c823740 1 -- 172.21.15.16:0/3072439221 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55e618a96920 con 0x55e6186371b0 2024-08-31T06:15:53.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.843+0000 7f4078c00640 1 -- 172.21.15.16:0/3072439221 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40800037f0 con 0x55e6186371b0 2024-08-31T06:15:53.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.843+0000 7f4078c00640 1 -- 172.21.15.16:0/3072439221 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7f40800046c0 con 0x55e6186371b0 2024-08-31T06:15:53.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.843+0000 7f409c823740 10 client.27572 fetch_fsmap finished waiting for FSMap version 42 2024-08-31T06:15:53.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.843+0000 7f409c823740 10 client.27572 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:15:53.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.843+0000 7f409c823740 20 client.27572 populate_metadata read hostname 'smithi016' 2024-08-31T06:15:53.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.843+0000 7f409c823740 1 --2- 172.21.15.16:0/3072439221 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55e618a96fd0 0x55e618a993f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:53.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.843+0000 7f409c823740 4 client.27572 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:15:53.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.843+0000 7f409c823740 1 -- 172.21.15.16:0/3072439221 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e61897a1a0 con 0x55e618a96fd0 2024-08-31T06:15:53.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.844+0000 7f408e000640 1 --2- 172.21.15.16:0/3072439221 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55e618a96fd0 0x55e618a993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:53.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.844+0000 7f4078c00640 10 client.27572 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:15:53.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.844+0000 7f408e000640 1 --2- 172.21.15.16:0/3072439221 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55e618a96fd0 0x55e618a993f0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d2f3c3afb7aefd3 server_cookie=19b3d3ed736e272b in_seq=0 out_seq=0 2024-08-31T06:15:53.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.844+0000 7f4078c00640 1 -- 172.21.15.16:0/3072439221 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e61897a1a0 con 0x55e618a96fd0 2024-08-31T06:15:53.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.844+0000 7f4078c00640 10 client.27572 handle_command_reply: tid=0 2024-08-31T06:15:53.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.844+0000 7f409c823740 1 client.27572 shutdown 2024-08-31T06:15:53.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.845+0000 7f409c823740 1 -- 172.21.15.16:0/3072439221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6186371b0 msgr2=0x55e618a2f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:53.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.845+0000 7f409c823740 1 --2- 172.21.15.16:0/3072439221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6186371b0 0x55e618a2f470 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4080003fb0 tx=0x7f40800316d0 comp rx=0 tx=0).stop 2024-08-31T06:15:53.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.845+0000 7f409c823740 1 -- 172.21.15.16:0/3072439221 shutdown_connections 2024-08-31T06:15:53.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.845+0000 7f409c823740 1 --2- 172.21.15.16:0/3072439221 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55e618a96fd0 0x55e618a993f0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:53.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.845+0000 7f409c823740 1 --2- 172.21.15.16:0/3072439221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e618660070 0x55e618a2f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:53.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.845+0000 7f409c823740 1 --2- 172.21.15.16:0/3072439221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6186371b0 0x55e618a2f470 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:53.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.845+0000 7f409c823740 1 --2- 172.21.15.16:0/3072439221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e6185f3da0 0x55e618a32fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:53.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.845+0000 7f409c823740 1 -- 172.21.15.16:0/3072439221 >> 172.21.15.16:0/3072439221 conn(0x55e618654460 msgr2=0x55e618a2c640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:53.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.845+0000 7f4078c00640 0 client.27572 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:15:53.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.845+0000 7f409c823740 1 -- 172.21.15.16:0/3072439221 shutdown_connections 2024-08-31T06:15:53.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.845+0000 7f409c823740 1 -- 172.21.15.16:0/3072439221 wait complete. 2024-08-31T06:15:53.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.846+0000 7f409c823740 20 client.27572 trim_cache size 0 max 16384 2024-08-31T06:15:53.850 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:15:53.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.847+0000 7f408fe00640 1 -- 172.21.15.16:0/2966190877 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f40740b4eb0 msgr2=0x7f40740b73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:53.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.847+0000 7f408fe00640 1 --2- 172.21.15.16:0/2966190877 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f40740b4eb0 0x7f40740b73c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f408800c760 tx=0x7f4088005f70 comp rx=0 tx=0).stop 2024-08-31T06:15:53.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.847+0000 7f408fe00640 1 -- 172.21.15.16:0/2966190877 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f409010bbf0 msgr2=0x7f40901a1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:53.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.847+0000 7f408fe00640 1 --2- 172.21.15.16:0/2966190877 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f409010bbf0 0x7f40901a1b50 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4088009fd0 tx=0x7f408800ef10 comp rx=0 tx=0).stop 2024-08-31T06:15:53.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.847+0000 7f408fe00640 1 -- 172.21.15.16:0/2966190877 shutdown_connections 2024-08-31T06:15:53.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.847+0000 7f408fe00640 1 --2- 172.21.15.16:0/2966190877 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f40740b4eb0 0x7f40740b73c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:53.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.847+0000 7f408fe00640 1 --2- 172.21.15.16:0/2966190877 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f409010bbf0 0x7f40901a1b50 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:53.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.847+0000 7f408fe00640 1 --2- 172.21.15.16:0/2966190877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f409010b2a0 0x7f409019c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:53.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.847+0000 7f408fe00640 1 --2- 172.21.15.16:0/2966190877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4090105960 0x7f409019c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:53.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.847+0000 7f408fe00640 1 -- 172.21.15.16:0/2966190877 >> 172.21.15.16:0/2966190877 conn(0x7f40901013d0 msgr2=0x7f40901037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:53.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.848+0000 7f408fe00640 1 -- 172.21.15.16:0/2966190877 shutdown_connections 2024-08-31T06:15:53.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:53.848+0000 7f408fe00640 1 -- 172.21.15.16:0/2966190877 wait complete. 2024-08-31T06:15:53.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:53 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:15:53.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:53 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:15:53.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:53 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:15:54.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:53 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:15:54.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:53 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:15:54.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:53 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:15:54.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:54 smithi184 ceph-mon[22331]: pgmap v630: 97 pgs: 97 active+clean; 3.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.9 MiB/s wr, 771 op/s 2024-08-31T06:15:55.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:54 smithi155 ceph-mon[21742]: pgmap v630: 97 pgs: 97 active+clean; 3.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.9 MiB/s wr, 771 op/s 2024-08-31T06:15:55.165 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:54 smithi016 ceph-mon[17194]: pgmap v630: 97 pgs: 97 active+clean; 3.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.9 MiB/s wr, 771 op/s 2024-08-31T06:15:55.819 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T06:15:55.820 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T06:15:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:15:56.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 510.0 s, 1603.0 tps, lat 63.163 ms stddev 77.616 2024-08-31T06:15:56.721 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:56 smithi016 ceph-mon[17194]: pgmap v631: 97 pgs: 97 active+clean; 3.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.8 MiB/s wr, 784 op/s 2024-08-31T06:15:56.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:56 smithi184 ceph-mon[22331]: pgmap v631: 97 pgs: 97 active+clean; 3.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.8 MiB/s wr, 784 op/s 2024-08-31T06:15:57.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:56 smithi155 ceph-mon[21742]: pgmap v631: 97 pgs: 97 active+clean; 3.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.8 MiB/s wr, 784 op/s 2024-08-31T06:15:57.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.406+0000 7f401d200640 1 -- 172.21.15.16:0/626742260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4018105b50 msgr2=0x7f4018105f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:57.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.406+0000 7f401d200640 1 --2- 172.21.15.16:0/626742260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4018105b50 0x7f4018105f60 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f4008009900 tx=0x7f400802a3f0 comp rx=0 tx=0).stop 2024-08-31T06:15:57.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.406+0000 7f401d200640 1 -- 172.21.15.16:0/626742260 shutdown_connections 2024-08-31T06:15:57.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.406+0000 7f401d200640 1 --2- 172.21.15.16:0/626742260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f401810bde0 0x7f4018112610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:57.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.406+0000 7f401d200640 1 --2- 172.21.15.16:0/626742260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f401810b490 0x7f401810b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:57.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.406+0000 7f401d200640 1 --2- 172.21.15.16:0/626742260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4018105b50 0x7f4018105f60 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:57.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.406+0000 7f401d200640 1 -- 172.21.15.16:0/626742260 >> 172.21.15.16:0/626742260 conn(0x7f4018101580 msgr2=0x7f40181039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:57.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.406+0000 7f401d200640 1 -- 172.21.15.16:0/626742260 shutdown_connections 2024-08-31T06:15:57.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.407+0000 7f401d200640 1 -- 172.21.15.16:0/626742260 wait complete. 2024-08-31T06:15:57.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.411+0000 7f401d200640 1 Processor -- start 2024-08-31T06:15:57.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.411+0000 7f401d200640 1 -- start start 2024-08-31T06:15:57.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.412+0000 7f401d200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4018105b50 0x7f401819c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:57.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.412+0000 7f401d200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f401810b490 0x7f401819cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:57.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.412+0000 7f401d200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f401810bde0 0x7f40181a1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:57.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.412+0000 7f401d200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4018115af0 con 0x7f4018105b50 2024-08-31T06:15:57.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.412+0000 7f401d200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4018115970 con 0x7f401810bde0 2024-08-31T06:15:57.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.412+0000 7f401d200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4018115c70 con 0x7f401810b490 2024-08-31T06:15:57.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.412+0000 7f4015e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4018105b50 0x7f401819c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:57.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.413+0000 7f4015e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4018105b50 0x7f401819c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47556/0 (socket says 172.21.15.16:47556) 2024-08-31T06:15:57.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.413+0000 7f4015e00640 1 -- 172.21.15.16:0/2366499324 learned_addr learned my addr 172.21.15.16:0/2366499324 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:57.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.413+0000 7f4015400640 1 --2- 172.21.15.16:0/2366499324 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f401810b490 0x7f401819cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:57.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.413+0000 7f4016800640 1 --2- 172.21.15.16:0/2366499324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f401810bde0 0x7f40181a1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:57.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.413+0000 7f4015e00640 1 -- 172.21.15.16:0/2366499324 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f401810b490 msgr2=0x7f401819cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:57.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.413+0000 7f4015e00640 1 --2- 172.21.15.16:0/2366499324 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f401810b490 0x7f401819cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:57.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.414+0000 7f4015e00640 1 -- 172.21.15.16:0/2366499324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f401810bde0 msgr2=0x7f40181a1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:57.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.414+0000 7f4015e00640 1 --2- 172.21.15.16:0/2366499324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f401810bde0 0x7f40181a1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:57.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.414+0000 7f4015e00640 1 -- 172.21.15.16:0/2366499324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40181a2410 con 0x7f4018105b50 2024-08-31T06:15:57.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.414+0000 7f4016800640 1 --2- 172.21.15.16:0/2366499324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f401810bde0 0x7f40181a1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:15:57.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.415+0000 7f4015e00640 1 --2- 172.21.15.16:0/2366499324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4018105b50 0x7f401819c560 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4008005ab0 tx=0x7f4008003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:57.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.415+0000 7f400ea00640 1 -- 172.21.15.16:0/2366499324 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400803f070 con 0x7f4018105b50 2024-08-31T06:15:57.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.416+0000 7f400ea00640 1 -- 172.21.15.16:0/2366499324 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4008031d60 con 0x7f4018105b50 2024-08-31T06:15:57.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.416+0000 7f400ea00640 1 -- 172.21.15.16:0/2366499324 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4008035030 con 0x7f4018105b50 2024-08-31T06:15:57.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.416+0000 7f401d200640 1 -- 172.21.15.16:0/2366499324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40181a26a0 con 0x7f4018105b50 2024-08-31T06:15:57.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.417+0000 7f401d200640 1 -- 172.21.15.16:0/2366499324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40181a2b30 con 0x7f4018105b50 2024-08-31T06:15:57.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.423+0000 7f400ea00640 1 -- 172.21.15.16:0/2366499324 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f4008042070 con 0x7f4018105b50 2024-08-31T06:15:57.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.424+0000 7f4023e67740 1 Processor -- start 2024-08-31T06:15:57.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.424+0000 7f4023e67740 1 -- start start 2024-08-31T06:15:57.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.425+0000 7f400ea00640 1 --2- 172.21.15.16:0/2366499324 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3ffc0b4dd0 0x7f3ffc0b72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:57.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.425+0000 7f4023e67740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e759ff350 0x564e75da5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:57.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.425+0000 7f400ea00640 1 -- 172.21.15.16:0/2366499324 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8470+0+0 (secure 0 0 0) 0x7f4008105d00 con 0x7f4018105b50 2024-08-31T06:15:57.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.425+0000 7f4023e67740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e759d6490 0x564e75da5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:57.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.425+0000 7f4023e67740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e759de670 0x564e75da6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:57.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.425+0000 7f4023e67740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564e75d1b750 con 0x564e759d6490 2024-08-31T06:15:57.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.425+0000 7f4023e67740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x564e75d0ace0 con 0x564e759de670 2024-08-31T06:15:57.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.425+0000 7f4023e67740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564e75d0ec90 con 0x564e759ff350 2024-08-31T06:15:57.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.425+0000 7f4015e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e759ff350 0x564e75da5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:57.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.425+0000 7f4015e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e759ff350 0x564e75da5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:33332/0 (socket says 172.21.15.16:33332) 2024-08-31T06:15:57.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.425+0000 7f4015e00640 1 -- 172.21.15.16:0/2895872757 learned_addr learned my addr 172.21.15.16:0/2895872757 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:57.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.426+0000 7f4015e00640 1 -- 172.21.15.16:0/2895872757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e759de670 msgr2=0x564e75da6080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:15:57.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.426+0000 7f4015e00640 1 --2- 172.21.15.16:0/2895872757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e759de670 0x564e75da6080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:57.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.426+0000 7f4015e00640 1 -- 172.21.15.16:0/2895872757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e759d6490 msgr2=0x564e75da5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:57.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.426+0000 7f4015400640 1 --2- 172.21.15.16:0/2895872757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e759d6490 0x564e75da5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:57.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.426+0000 7f4015e00640 1 --2- 172.21.15.16:0/2895872757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e759d6490 0x564e75da5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:57.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.426+0000 7f4015e00640 1 -- 172.21.15.16:0/2895872757 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e75a6a930 con 0x564e759ff350 2024-08-31T06:15:57.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.426+0000 7f4015400640 1 --2- 172.21.15.16:0/2895872757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e759d6490 0x564e75da5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:15:57.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.426+0000 7f4016800640 1 --2- 172.21.15.16:0/2366499324 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3ffc0b4dd0 0x7f3ffc0b72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:57.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.427+0000 7f4015e00640 1 --2- 172.21.15.16:0/2895872757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e759ff350 0x564e75da5600 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f40080c8440 tx=0x7f400803b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f366946ad99d73bc server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:57.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.427+0000 7f4003400640 1 -- 172.21.15.16:0/2895872757 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40080cbed0 con 0x564e759ff350 2024-08-31T06:15:57.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.428+0000 7f4003400640 1 -- 172.21.15.16:0/2895872757 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40080ccac0 con 0x564e759ff350 2024-08-31T06:15:57.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.428+0000 7f4003400640 1 -- 172.21.15.16:0/2895872757 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40081036a0 con 0x564e759ff350 2024-08-31T06:15:57.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.429+0000 7f4023e67740 1 -- 172.21.15.16:0/2895872757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e759ff350 msgr2=0x564e75da5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:57.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.429+0000 7f4016800640 1 --2- 172.21.15.16:0/2366499324 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3ffc0b4dd0 0x7f3ffc0b72e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f40100092f0 tx=0x7f4010009280 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:57.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.429+0000 7f4023e67740 1 --2- 172.21.15.16:0/2895872757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e759ff350 0x564e75da5600 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f40080c8440 tx=0x7f400803b870 comp rx=0 tx=0).stop 2024-08-31T06:15:57.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.429+0000 7f4023e67740 1 -- 172.21.15.16:0/2895872757 shutdown_connections 2024-08-31T06:15:57.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.429+0000 7f4023e67740 1 --2- 172.21.15.16:0/2895872757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e759de670 0x564e75da6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:57.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.429+0000 7f4023e67740 1 --2- 172.21.15.16:0/2895872757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e759d6490 0x564e75da5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:57.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.429+0000 7f4023e67740 1 --2- 172.21.15.16:0/2895872757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e759ff350 0x564e75da5600 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:57.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.429+0000 7f4023e67740 1 -- 172.21.15.16:0/2895872757 >> 172.21.15.16:0/2895872757 conn(0x564e759f3740 msgr2=0x564e75da34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:57.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.430+0000 7f4023e67740 1 -- 172.21.15.16:0/2895872757 shutdown_connections 2024-08-31T06:15:57.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.431+0000 7f4023e67740 1 -- 172.21.15.16:0/2895872757 wait complete. 2024-08-31T06:15:57.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.431+0000 7f4023e67740 1 -- start start 2024-08-31T06:15:57.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.432+0000 7f4023e67740 1 Processor -- start 2024-08-31T06:15:57.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.433+0000 7f4023e67740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e759d6490 0x564e75db9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:57.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.433+0000 7f4023e67740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e759de670 0x564e75dba300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:57.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.433+0000 7f4023e67740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e759ff350 0x564e75dbf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:57.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.433+0000 7f4023e67740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564e75b6d890 con 0x564e759de670 2024-08-31T06:15:57.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.433+0000 7f4023e67740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x564e75c91cd0 con 0x564e759d6490 2024-08-31T06:15:57.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.433+0000 7f4023e67740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564e75db3ba0 con 0x564e759ff350 2024-08-31T06:15:57.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.433+0000 7f4023e67740 10 client.0 _ll_register_callbacks cb 0x564e75d05ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:15:57.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.433+0000 7f4016800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e759ff350 0x564e75dbf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:57.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.434+0000 7f4016800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e759ff350 0x564e75dbf320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:33334/0 (socket says 172.21.15.16:33334) 2024-08-31T06:15:57.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.434+0000 7f4016800640 1 -- 172.21.15.16:0/2217815772 learned_addr learned my addr 172.21.15.16:0/2217815772 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:57.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.434+0000 7f4015e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e759d6490 0x564e75db9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:57.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.434+0000 7f4015400640 1 --2- 172.21.15.16:0/2217815772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e759de670 0x564e75dba300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:57.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.434+0000 7f4016800640 1 -- 172.21.15.16:0/2217815772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e759d6490 msgr2=0x564e75db9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:57.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.434+0000 7f4016800640 1 --2- 172.21.15.16:0/2217815772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e759d6490 0x564e75db9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:57.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.434+0000 7f4016800640 1 -- 172.21.15.16:0/2217815772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e759de670 msgr2=0x564e75dba300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:57.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.434+0000 7f4016800640 1 --2- 172.21.15.16:0/2217815772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e759de670 0x564e75dba300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:57.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.434+0000 7f4016800640 1 -- 172.21.15.16:0/2217815772 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e75cdb1b0 con 0x564e759ff350 2024-08-31T06:15:57.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.435+0000 7f4016800640 1 --2- 172.21.15.16:0/2217815772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e759ff350 0x564e75dbf320 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4010005f40 tx=0x7f4010004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a38e796a2f530b99 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:57.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.435+0000 7f3ff7e00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:15:57.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.435+0000 7f3ff7e00640 1 -- 172.21.15.16:0/2217815772 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4010023070 con 0x564e759ff350 2024-08-31T06:15:57.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.436+0000 7f3ff7e00640 1 -- 172.21.15.16:0/2217815772 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8470+0+0 (secure 0 0 0) 0x7f40100119b0 con 0x564e759ff350 2024-08-31T06:15:57.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.436+0000 7f4023e67740 1 -- 172.21.15.16:0/2217815772 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40080ccac0 con 0x564e759ff350 2024-08-31T06:15:57.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.436+0000 7f3ff7e00640 1 -- 172.21.15.16:0/2217815772 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f401002f050 con 0x564e759ff350 2024-08-31T06:15:57.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.437+0000 7f4023e67740 10 client.27568 fetch_fsmap learned FSMap version 42 2024-08-31T06:15:57.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.437+0000 7f4023e67740 1 -- 172.21.15.16:0/2217815772 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x564e75dc8290 con 0x564e759ff350 2024-08-31T06:15:57.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.438+0000 7f3ff7e00640 1 -- 172.21.15.16:0/2217815772 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40080ccac0 con 0x564e759ff350 2024-08-31T06:15:57.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.438+0000 7f3ff7e00640 1 -- 172.21.15.16:0/2217815772 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7f401000f040 con 0x564e759ff350 2024-08-31T06:15:57.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.438+0000 7f4023e67740 10 client.27568 fetch_fsmap finished waiting for FSMap version 42 2024-08-31T06:15:57.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.438+0000 7f4023e67740 10 client.27568 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:15:57.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.438+0000 7f4023e67740 20 client.27568 populate_metadata read hostname 'smithi016' 2024-08-31T06:15:57.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.438+0000 7f4023e67740 1 --2- 172.21.15.16:0/2217815772 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564e759bdcd0 0x564e75dcb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:57.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.438+0000 7f4023e67740 4 client.27568 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:15:57.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.438+0000 7f4023e67740 1 -- 172.21.15.16:0/2217815772 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e75d1d6a0 con 0x564e759bdcd0 2024-08-31T06:15:57.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.439+0000 7f4015e00640 1 --2- 172.21.15.16:0/2217815772 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564e759bdcd0 0x564e75dcb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:57.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.440+0000 7f4015e00640 1 --2- 172.21.15.16:0/2217815772 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564e759bdcd0 0x564e75dcb740 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53de305ef0fbf516 server_cookie=ee58d0885865e794 in_seq=0 out_seq=0 2024-08-31T06:15:57.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.440+0000 7f3ff7e00640 10 client.27568 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:15:57.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.440+0000 7f3ff7e00640 1 -- 172.21.15.16:0/2217815772 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564e75d1d6a0 con 0x564e759bdcd0 2024-08-31T06:15:57.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.440+0000 7f3ff7e00640 10 client.27568 handle_command_reply: tid=0 2024-08-31T06:15:57.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.441+0000 7f4023e67740 1 client.27568 shutdown 2024-08-31T06:15:57.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.441+0000 7f4023e67740 1 -- 172.21.15.16:0/2217815772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e759ff350 msgr2=0x564e75dbf320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:57.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.441+0000 7f4023e67740 1 --2- 172.21.15.16:0/2217815772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e759ff350 0x564e75dbf320 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4010005f40 tx=0x7f4010004a40 comp rx=0 tx=0).stop 2024-08-31T06:15:57.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.441+0000 7f4023e67740 1 -- 172.21.15.16:0/2217815772 shutdown_connections 2024-08-31T06:15:57.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.441+0000 7f4023e67740 1 --2- 172.21.15.16:0/2217815772 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564e759bdcd0 0x564e75dcb740 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:57.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.441+0000 7f4023e67740 1 --2- 172.21.15.16:0/2217815772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e759ff350 0x564e75dbf320 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:57.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.441+0000 7f4023e67740 1 --2- 172.21.15.16:0/2217815772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e759de670 0x564e75dba300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:57.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.441+0000 7f4023e67740 1 --2- 172.21.15.16:0/2217815772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e759d6490 0x564e75db9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:57.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.441+0000 7f4023e67740 1 -- 172.21.15.16:0/2217815772 >> 172.21.15.16:0/2217815772 conn(0x564e759f3740 msgr2=0x564e75da6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:57.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.441+0000 7f3ff7e00640 0 client.27568 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:15:57.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.442+0000 7f4023e67740 1 -- 172.21.15.16:0/2217815772 shutdown_connections 2024-08-31T06:15:57.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.442+0000 7f4023e67740 1 -- 172.21.15.16:0/2217815772 wait complete. 2024-08-31T06:15:57.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.442+0000 7f4023e67740 20 client.27568 trim_cache size 0 max 16384 2024-08-31T06:15:57.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.478+0000 7f4023e67740 1 Processor -- start 2024-08-31T06:15:57.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.478+0000 7f4023e67740 1 -- start start 2024-08-31T06:15:57.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.479+0000 7f4023e67740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e759d6490 0x564e75da3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:57.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.479+0000 7f4023e67740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e759de670 0x564e75dbd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:57.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.479+0000 7f4023e67740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e759ff350 0x564e75dbd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:57.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.479+0000 7f4023e67740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f40181afd30 con 0x564e759ff350 2024-08-31T06:15:57.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.479+0000 7f4023e67740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f40181ad570 con 0x564e759d6490 2024-08-31T06:15:57.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.479+0000 7f4023e67740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f40181ab3e0 con 0x564e759de670 2024-08-31T06:15:57.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.480+0000 7f4015e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e759d6490 0x564e75da3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:57.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.480+0000 7f4015e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e759d6490 0x564e75da3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:50058/0 (socket says 172.21.15.16:50058) 2024-08-31T06:15:57.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.480+0000 7f4015e00640 1 -- 172.21.15.16:0/2991040661 learned_addr learned my addr 172.21.15.16:0/2991040661 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:57.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.480+0000 7f4015400640 1 --2- 172.21.15.16:0/2991040661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e759de670 0x564e75dbd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:57.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.481+0000 7f4015e00640 1 -- 172.21.15.16:0/2991040661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e759de670 msgr2=0x564e75dbd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:57.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.481+0000 7f4015e00640 1 --2- 172.21.15.16:0/2991040661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e759de670 0x564e75dbd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:57.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.481+0000 7f4015e00640 1 -- 172.21.15.16:0/2991040661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e759ff350 msgr2=0x564e75dbd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:57.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.481+0000 7f4016800640 1 --2- 172.21.15.16:0/2991040661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e759ff350 0x564e75dbd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:57.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.481+0000 7f4015e00640 1 --2- 172.21.15.16:0/2991040661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e759ff350 0x564e75dbd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:57.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.481+0000 7f4015e00640 1 -- 172.21.15.16:0/2991040661 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e75db8a80 con 0x564e759d6490 2024-08-31T06:15:57.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.481+0000 7f4016800640 1 --2- 172.21.15.16:0/2991040661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e759ff350 0x564e75dbd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:15:57.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.481+0000 7f4015e00640 1 --2- 172.21.15.16:0/2991040661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e759d6490 0x564e75da3540 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f40080cc0c0 tx=0x7f40080ef040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b6b7e1cce9d644a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:57.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.482+0000 7f4002a00640 1 -- 172.21.15.16:0/2991040661 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4008049040 con 0x564e759d6490 2024-08-31T06:15:57.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.482+0000 7f4002a00640 1 -- 172.21.15.16:0/2991040661 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40080d0c30 con 0x564e759d6490 2024-08-31T06:15:57.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.482+0000 7f4002a00640 1 -- 172.21.15.16:0/2991040661 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40080cc620 con 0x564e759d6490 2024-08-31T06:15:57.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.484+0000 7f4023e67740 1 -- 172.21.15.16:0/2991040661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e759d6490 msgr2=0x564e75da3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:57.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.484+0000 7f4023e67740 1 --2- 172.21.15.16:0/2991040661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e759d6490 0x564e75da3540 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f40080cc0c0 tx=0x7f40080ef040 comp rx=0 tx=0).stop 2024-08-31T06:15:57.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.484+0000 7f4023e67740 1 -- 172.21.15.16:0/2991040661 shutdown_connections 2024-08-31T06:15:57.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.484+0000 7f4023e67740 1 --2- 172.21.15.16:0/2991040661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e759de670 0x564e75dbd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:57.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.484+0000 7f4023e67740 1 --2- 172.21.15.16:0/2991040661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e759ff350 0x564e75dbd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:57.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.484+0000 7f4023e67740 1 --2- 172.21.15.16:0/2991040661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e759d6490 0x564e75da3540 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:57.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.484+0000 7f4023e67740 1 -- 172.21.15.16:0/2991040661 >> 172.21.15.16:0/2991040661 conn(0x564e759f3740 msgr2=0x564e75da2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:57.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.485+0000 7f4023e67740 1 -- 172.21.15.16:0/2991040661 shutdown_connections 2024-08-31T06:15:57.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.485+0000 7f4023e67740 1 -- 172.21.15.16:0/2991040661 wait complete. 2024-08-31T06:15:57.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.485+0000 7f4023e67740 1 -- start start 2024-08-31T06:15:57.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.486+0000 7f4023e67740 1 Processor -- start 2024-08-31T06:15:57.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.487+0000 7f4023e67740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e759d6490 0x564e75dbade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:57.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.487+0000 7f4023e67740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e759de670 0x564e75dbb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:57.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.487+0000 7f4023e67740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e759ff350 0x564e75dbb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:57.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.487+0000 7f4015e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e759d6490 0x564e75dbade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:57.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.488+0000 7f4023e67740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f40181aba30 con 0x564e759ff350 2024-08-31T06:15:57.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.488+0000 7f4023e67740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f40181acea0 con 0x564e759de670 2024-08-31T06:15:57.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.488+0000 7f4015400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e759de670 0x564e75dbb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:57.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.488+0000 7f4023e67740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f40181ac100 con 0x564e759d6490 2024-08-31T06:15:57.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.488+0000 7f4015400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e759de670 0x564e75dbb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:50064/0 (socket says 172.21.15.16:50064) 2024-08-31T06:15:57.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.488+0000 7f4015400640 1 -- 172.21.15.16:0/3859665783 learned_addr learned my addr 172.21.15.16:0/3859665783 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:15:57.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.488+0000 7f4023e67740 10 client.0 _ll_register_callbacks cb 0x7f4010010f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:15:57.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.488+0000 7f4016800640 1 --2- 172.21.15.16:0/3859665783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e759ff350 0x564e75dbb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:57.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.489+0000 7f4015e00640 1 -- 172.21.15.16:0/3859665783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e759de670 msgr2=0x564e75dbb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:57.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.489+0000 7f4015e00640 1 --2- 172.21.15.16:0/3859665783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e759de670 0x564e75dbb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:57.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.489+0000 7f4015e00640 1 -- 172.21.15.16:0/3859665783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e759ff350 msgr2=0x564e75dbb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:15:57.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.489+0000 7f4015e00640 1 --2- 172.21.15.16:0/3859665783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e759ff350 0x564e75dbb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:57.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.489+0000 7f4015e00640 1 -- 172.21.15.16:0/3859665783 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e75dba670 con 0x564e759d6490 2024-08-31T06:15:57.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.489+0000 7f4016800640 1 --2- 172.21.15.16:0/3859665783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e759ff350 0x564e75dbb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:15:57.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.489+0000 7f4015e00640 1 --2- 172.21.15.16:0/3859665783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e759d6490 0x564e75dbade0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x564e75dbc090 tx=0x7f4008031f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48a8c3ebdd0b75b3 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:15:57.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.489+0000 7f3ff7e00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:15:57.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.490+0000 7f3ff7e00640 1 -- 172.21.15.16:0/3859665783 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4008049070 con 0x564e759d6490 2024-08-31T06:15:57.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.490+0000 7f3ff7e00640 1 -- 172.21.15.16:0/3859665783 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8470+0+0 (secure 0 0 0) 0x7f40080c8650 con 0x564e759d6490 2024-08-31T06:15:57.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.490+0000 7f4023e67740 1 -- 172.21.15.16:0/3859665783 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564e75d19140 con 0x564e759d6490 2024-08-31T06:15:57.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.491+0000 7f3ff7e00640 1 -- 172.21.15.16:0/3859665783 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f40080cc620 con 0x564e759d6490 2024-08-31T06:15:57.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.491+0000 7f4023e67740 10 client.27574 fetch_fsmap learned FSMap version 42 2024-08-31T06:15:57.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.491+0000 7f4023e67740 1 -- 172.21.15.16:0/3859665783 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x564e75e36f70 con 0x564e759d6490 2024-08-31T06:15:57.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.492+0000 7f3ff7e00640 1 -- 172.21.15.16:0/3859665783 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x564e75d19140 con 0x564e759d6490 2024-08-31T06:15:57.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.492+0000 7f3ff7e00640 1 -- 172.21.15.16:0/3859665783 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7f4008103b70 con 0x564e759d6490 2024-08-31T06:15:57.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.493+0000 7f4023e67740 10 client.27574 fetch_fsmap finished waiting for FSMap version 42 2024-08-31T06:15:57.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.493+0000 7f4023e67740 10 client.27574 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:15:57.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.494+0000 7f4023e67740 20 client.27574 populate_metadata read hostname 'smithi016' 2024-08-31T06:15:57.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.494+0000 7f4023e67740 1 --2- 172.21.15.16:0/3859665783 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564e75993080 0x564e75e392c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:15:57.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.494+0000 7f4023e67740 4 client.27574 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T06:15:57.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.494+0000 7f4023e67740 1 -- 172.21.15.16:0/3859665783 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564e75d16880 con 0x564e75993080 2024-08-31T06:15:57.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.494+0000 7f4015400640 1 --2- 172.21.15.16:0/3859665783 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564e75993080 0x564e75e392c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:15:57.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.495+0000 7f4015400640 1 --2- 172.21.15.16:0/3859665783 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564e75993080 0x564e75e392c0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=935ad30f924453ac server_cookie=7f02d19ba815359f in_seq=0 out_seq=0 2024-08-31T06:15:57.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.495+0000 7f3ff7e00640 10 client.27574 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:15:57.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.497+0000 7f3ff7e00640 1 -- 172.21.15.16:0/3859665783 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564e75d16880 con 0x564e75993080 2024-08-31T06:15:57.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.497+0000 7f3ff7e00640 10 client.27574 handle_command_reply: tid=0 2024-08-31T06:15:57.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.498+0000 7f4023e67740 1 client.27574 shutdown 2024-08-31T06:15:57.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.500+0000 7f4023e67740 1 -- 172.21.15.16:0/3859665783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e759d6490 msgr2=0x564e75dbade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:57.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.500+0000 7f4023e67740 1 --2- 172.21.15.16:0/3859665783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e759d6490 0x564e75dbade0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x564e75dbc090 tx=0x7f4008031f10 comp rx=0 tx=0).stop 2024-08-31T06:15:57.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.500+0000 7f4023e67740 1 -- 172.21.15.16:0/3859665783 shutdown_connections 2024-08-31T06:15:57.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.500+0000 7f4023e67740 1 --2- 172.21.15.16:0/3859665783 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564e75993080 0x564e75e392c0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:57.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.500+0000 7f4023e67740 1 --2- 172.21.15.16:0/3859665783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e759ff350 0x564e75dbb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:57.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.500+0000 7f4023e67740 1 --2- 172.21.15.16:0/3859665783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e759de670 0x564e75dbb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:57.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.500+0000 7f4023e67740 1 --2- 172.21.15.16:0/3859665783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e759d6490 0x564e75dbade0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:57.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.500+0000 7f4023e67740 1 -- 172.21.15.16:0/3859665783 >> 172.21.15.16:0/3859665783 conn(0x564e759f3740 msgr2=0x564e75dbd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:57.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.500+0000 7f3ff7e00640 0 client.27574 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:15:57.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.500+0000 7f4023e67740 1 -- 172.21.15.16:0/3859665783 shutdown_connections 2024-08-31T06:15:57.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.500+0000 7f4023e67740 1 -- 172.21.15.16:0/3859665783 wait complete. 2024-08-31T06:15:57.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.501+0000 7f4023e67740 20 client.27574 trim_cache size 0 max 16384 2024-08-31T06:15:57.504 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:15:57.504 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-08-31T06:15:57.504 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "f9c4a22e-0c8a-4928-9b0d-e2ce1db5a86b", 2024-08-31T06:15:57.504 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-08-31T06:15:57.504 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:15:57.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.502+0000 7f401d200640 1 -- 172.21.15.16:0/2366499324 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3ffc0b4dd0 msgr2=0x7f3ffc0b72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:57.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.502+0000 7f401d200640 1 --2- 172.21.15.16:0/2366499324 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3ffc0b4dd0 0x7f3ffc0b72e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f40100092f0 tx=0x7f4010009280 comp rx=0 tx=0).stop 2024-08-31T06:15:57.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.502+0000 7f401d200640 1 -- 172.21.15.16:0/2366499324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4018105b50 msgr2=0x7f401819c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:15:57.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.502+0000 7f401d200640 1 --2- 172.21.15.16:0/2366499324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4018105b50 0x7f401819c560 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4008005ab0 tx=0x7f4008003cd0 comp rx=0 tx=0).stop 2024-08-31T06:15:57.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.505+0000 7f401d200640 1 -- 172.21.15.16:0/2366499324 shutdown_connections 2024-08-31T06:15:57.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.505+0000 7f401d200640 1 --2- 172.21.15.16:0/2366499324 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3ffc0b4dd0 0x7f3ffc0b72e0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:57.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.505+0000 7f401d200640 1 --2- 172.21.15.16:0/2366499324 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f401810b490 0x7f401819cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:57.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.505+0000 7f401d200640 1 --2- 172.21.15.16:0/2366499324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f401810bde0 0x7f40181a1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:57.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.505+0000 7f401d200640 1 --2- 172.21.15.16:0/2366499324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4018105b50 0x7f401819c560 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:15:57.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.505+0000 7f401d200640 1 -- 172.21.15.16:0/2366499324 >> 172.21.15.16:0/2366499324 conn(0x7f4018101580 msgr2=0x7f401810aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:15:57.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.506+0000 7f401d200640 1 -- 172.21.15.16:0/2366499324 shutdown_connections 2024-08-31T06:15:57.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:15:57.506+0000 7f401d200640 1 -- 172.21.15.16:0/2366499324 wait complete. 2024-08-31T06:15:58.628 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T06:15:58.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:58 smithi016 ceph-mon[17194]: scrub queued for path: / 2024-08-31T06:15:58.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:58 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:15:58.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:58 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:15:58.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:58 smithi016 ceph-mon[17194]: pgmap v632: 97 pgs: 97 active+clean; 3.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.5 MiB/s wr, 660 op/s 2024-08-31T06:15:58.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:58 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:15:58.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:58 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:15:58.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:58 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:15:58.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:58 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:15:58.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:58 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:15:58.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:58 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:15:58.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:58 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:15:58.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:58 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:15:58.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:58 smithi184 ceph-mon[22331]: scrub queued for path: / 2024-08-31T06:15:58.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:58 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:15:58.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:58 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:15:58.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:58 smithi184 ceph-mon[22331]: pgmap v632: 97 pgs: 97 active+clean; 3.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.5 MiB/s wr, 660 op/s 2024-08-31T06:15:58.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:58 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:15:58.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:58 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:15:58.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:58 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:15:58.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:58 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:15:58.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:58 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:15:58.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:58 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:15:58.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:58 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:15:58.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:58 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:15:59.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:58 smithi155 ceph-mon[21742]: scrub queued for path: / 2024-08-31T06:15:59.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:58 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:15:59.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:58 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:15:59.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:58 smithi155 ceph-mon[21742]: pgmap v632: 97 pgs: 97 active+clean; 3.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.5 MiB/s wr, 660 op/s 2024-08-31T06:15:59.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:58 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:15:59.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:58 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:15:59.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:58 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:15:59.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:58 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:15:59.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:58 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:15:59.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:58 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:15:59.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:58 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:15:59.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:58 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:15:59.325 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:15:59.842 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:15:59 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:15:59.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:15:59 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:16:00.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:15:59 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:16:00.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.452+0000 7f9caa400640 1 -- 172.21.15.16:0/188818421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ca4109bc0 msgr2=0x7f9ca41103f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:00.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.452+0000 7f9caa400640 1 --2- 172.21.15.16:0/188818421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ca4109bc0 0x7f9ca41103f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c009be0 tx=0x7f9c9c02a6c0 comp rx=0 tx=0).stop 2024-08-31T06:16:00.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.452+0000 7f9caa400640 1 -- 172.21.15.16:0/188818421 shutdown_connections 2024-08-31T06:16:00.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.452+0000 7f9caa400640 1 --2- 172.21.15.16:0/188818421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ca4109bc0 0x7f9ca41103f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:00.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.452+0000 7f9caa400640 1 --2- 172.21.15.16:0/188818421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ca4109270 0x7f9ca4109680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:00.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.452+0000 7f9caa400640 1 --2- 172.21.15.16:0/188818421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ca4103930 0x7f9ca4103d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:00.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.452+0000 7f9caa400640 1 -- 172.21.15.16:0/188818421 >> 172.21.15.16:0/188818421 conn(0x7f9ca40ff380 msgr2=0x7f9ca41017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:00.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.453+0000 7f9caa400640 1 -- 172.21.15.16:0/188818421 shutdown_connections 2024-08-31T06:16:00.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.453+0000 7f9caa400640 1 -- 172.21.15.16:0/188818421 wait complete. 2024-08-31T06:16:00.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.455+0000 7f9caa400640 1 Processor -- start 2024-08-31T06:16:00.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.455+0000 7f9caa400640 1 -- start start 2024-08-31T06:16:00.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.455+0000 7f9caa400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ca4103930 0x7f9ca41132d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:00.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.455+0000 7f9caa400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ca4109270 0x7f9ca4113850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:00.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.455+0000 7f9caa400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ca4109bc0 0x7f9ca4118b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:00.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.455+0000 7f9caa400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca4111650 con 0x7f9ca4109bc0 2024-08-31T06:16:00.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.455+0000 7f9caa400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca41114d0 con 0x7f9ca4109270 2024-08-31T06:16:00.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.455+0000 7f9caa400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca41117d0 con 0x7f9ca4103930 2024-08-31T06:16:00.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.456+0000 7f9ca3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ca4109bc0 0x7f9ca4118b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:00.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.456+0000 7f9ca2a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ca4109270 0x7f9ca4113850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:00.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.456+0000 7f9ca3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ca4109bc0 0x7f9ca4118b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47632/0 (socket says 172.21.15.16:47632) 2024-08-31T06:16:00.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.456+0000 7f9ca2a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ca4109270 0x7f9ca4113850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:50070/0 (socket says 172.21.15.16:50070) 2024-08-31T06:16:00.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.456+0000 7f9ca3e00640 1 -- 172.21.15.16:0/28332279 learned_addr learned my addr 172.21.15.16:0/28332279 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:00.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.456+0000 7f9ca3400640 1 --2- 172.21.15.16:0/28332279 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ca4103930 0x7f9ca41132d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:00.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.457+0000 7f9ca2a00640 1 -- 172.21.15.16:0/28332279 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ca4103930 msgr2=0x7f9ca41132d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:00.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.457+0000 7f9ca2a00640 1 --2- 172.21.15.16:0/28332279 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ca4103930 0x7f9ca41132d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:00.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.457+0000 7f9ca2a00640 1 -- 172.21.15.16:0/28332279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ca4109bc0 msgr2=0x7f9ca4118b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:00.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.457+0000 7f9ca2a00640 1 --2- 172.21.15.16:0/28332279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ca4109bc0 0x7f9ca4118b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:00.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.457+0000 7f9ca2a00640 1 -- 172.21.15.16:0/28332279 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca4119240 con 0x7f9ca4109270 2024-08-31T06:16:00.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.457+0000 7f9ca3e00640 1 --2- 172.21.15.16:0/28332279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ca4109bc0 0x7f9ca4118b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:16:00.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.457+0000 7f9ca2a00640 1 --2- 172.21.15.16:0/28332279 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ca4109270 0x7f9ca4113850 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9ca406b640 tx=0x7f9c9000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:00.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.458+0000 7f9c9be00640 1 -- 172.21.15.16:0/28332279 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9000ccd0 con 0x7f9ca4109270 2024-08-31T06:16:00.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.458+0000 7f9c9be00640 1 -- 172.21.15.16:0/28332279 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c900035f0 con 0x7f9ca4109270 2024-08-31T06:16:00.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.458+0000 7f9c9be00640 1 -- 172.21.15.16:0/28332279 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c90010690 con 0x7f9ca4109270 2024-08-31T06:16:00.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.458+0000 7f9caa400640 1 -- 172.21.15.16:0/28332279 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ca41194d0 con 0x7f9ca4109270 2024-08-31T06:16:00.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.458+0000 7f9caa400640 1 -- 172.21.15.16:0/28332279 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ca411bc90 con 0x7f9ca4109270 2024-08-31T06:16:00.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.462+0000 7f9cb10d5740 1 Processor -- start 2024-08-31T06:16:00.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.463+0000 7f9cb10d5740 1 -- start start 2024-08-31T06:16:00.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.464+0000 7f9cb10d5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9b5003070 0x55f9b53a8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:00.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.464+0000 7f9ca3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9b5003070 0x55f9b53a8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:00.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.464+0000 7f9cb10d5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9b4fda1b0 0x55f9b53a8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:00.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.464+0000 7f9cb10d5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9b4fe2390 0x55f9b53a93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:00.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.464+0000 7f9cb10d5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f9b531f770 con 0x55f9b5003070 2024-08-31T06:16:00.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.465+0000 7f9ca3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9b5003070 0x55f9b53a8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47648/0 (socket says 172.21.15.16:47648) 2024-08-31T06:16:00.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.465+0000 7f9c9be00640 1 -- 172.21.15.16:0/28332279 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f9c900037a0 con 0x7f9ca4109270 2024-08-31T06:16:00.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.465+0000 7f9ca3e00640 1 -- 172.21.15.16:0/743794027 learned_addr learned my addr 172.21.15.16:0/743794027 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:00.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.465+0000 7f9cb10d5740 1 -- 172.21.15.16:0/743794027 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f9b530ed00 con 0x55f9b4fda1b0 2024-08-31T06:16:00.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.465+0000 7f9cb10d5740 1 -- 172.21.15.16:0/743794027 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f9b5312cb0 con 0x55f9b4fe2390 2024-08-31T06:16:00.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.466+0000 7f9ca3400640 1 --2- 172.21.15.16:0/743794027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9b4fda1b0 0x55f9b53a8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:00.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.466+0000 7f9ca3e00640 1 -- 172.21.15.16:0/743794027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9b4fe2390 msgr2=0x55f9b53a93d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T06:16:00.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.466+0000 7f9ca3e00640 1 --2- 172.21.15.16:0/743794027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9b4fe2390 0x55f9b53a93d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:00.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.466+0000 7f9ca3e00640 1 -- 172.21.15.16:0/743794027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9b4fda1b0 msgr2=0x55f9b53a8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:00.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.466+0000 7f9ca3e00640 1 --2- 172.21.15.16:0/743794027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9b4fda1b0 0x55f9b53a8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:00.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.466+0000 7f9ca3e00640 1 -- 172.21.15.16:0/743794027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9b506e650 con 0x55f9b5003070 2024-08-31T06:16:00.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.467+0000 7f9ca3e00640 1 --2- 172.21.15.16:0/743794027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9b5003070 0x55f9b53a8950 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c030890 tx=0x7f9c9c0308c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf01159e248def5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:00.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.467+0000 7f9c9be00640 1 --2- 172.21.15.16:0/28332279 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9c880b4da0 0x7f9c880b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:00.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.468+0000 7f9c9be00640 1 -- 172.21.15.16:0/28332279 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8470+0+0 (secure 0 0 0) 0x7f9c900e5680 con 0x7f9ca4109270 2024-08-31T06:16:00.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.468+0000 7f9c98c00640 1 -- 172.21.15.16:0/743794027 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c041030 con 0x55f9b5003070 2024-08-31T06:16:00.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.468+0000 7f9ca3400640 1 --2- 172.21.15.16:0/28332279 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9c880b4da0 0x7f9c880b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:00.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.469+0000 7f9c98c00640 1 -- 172.21.15.16:0/743794027 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c9c030e20 con 0x55f9b5003070 2024-08-31T06:16:00.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.470+0000 7f9cb10d5740 1 -- 172.21.15.16:0/743794027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9b5003070 msgr2=0x55f9b53a8950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:00.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.470+0000 7f9cb10d5740 1 --2- 172.21.15.16:0/743794027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9b5003070 0x55f9b53a8950 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c030890 tx=0x7f9c9c0308c0 comp rx=0 tx=0).stop 2024-08-31T06:16:00.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.471+0000 7f9cb10d5740 1 -- 172.21.15.16:0/743794027 shutdown_connections 2024-08-31T06:16:00.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.471+0000 7f9cb10d5740 1 --2- 172.21.15.16:0/743794027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9b4fe2390 0x55f9b53a93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:00.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.471+0000 7f9cb10d5740 1 --2- 172.21.15.16:0/743794027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9b4fda1b0 0x55f9b53a8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:00.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.471+0000 7f9cb10d5740 1 --2- 172.21.15.16:0/743794027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9b5003070 0x55f9b53a8950 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:00.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.471+0000 7f9cb10d5740 1 -- 172.21.15.16:0/743794027 >> 172.21.15.16:0/743794027 conn(0x55f9b4ff7460 msgr2=0x55f9b53a6840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:00.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.471+0000 7f9ca3400640 1 --2- 172.21.15.16:0/28332279 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9c880b4da0 0x7f9c880b72b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x55f9b53a75d0 tx=0x7f9c94005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:00.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.471+0000 7f9cb10d5740 1 -- 172.21.15.16:0/743794027 shutdown_connections 2024-08-31T06:16:00.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.472+0000 7f9cb10d5740 1 -- 172.21.15.16:0/743794027 wait complete. 2024-08-31T06:16:00.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.472+0000 7f9cb10d5740 1 -- start start 2024-08-31T06:16:00.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.473+0000 7f9cb10d5740 1 Processor -- start 2024-08-31T06:16:00.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.473+0000 7f9cb10d5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9b4fda1b0 0x55f9b53b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:00.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.473+0000 7f9cb10d5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9b4fe2390 0x55f9b53b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:00.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.473+0000 7f9cb10d5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9b5003070 0x55f9b53c04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:00.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.473+0000 7f9cb10d5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f9b5171700 con 0x55f9b5003070 2024-08-31T06:16:00.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.473+0000 7f9cb10d5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f9b5295cb0 con 0x55f9b4fe2390 2024-08-31T06:16:00.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.473+0000 7f9cb10d5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f9b53b7170 con 0x55f9b4fda1b0 2024-08-31T06:16:00.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.473+0000 7f9ca3400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9b4fda1b0 0x55f9b53b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:00.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.473+0000 7f9ca2a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9b4fe2390 0x55f9b53b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:00.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.473+0000 7f9ca3400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9b4fda1b0 0x55f9b53b44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:33396/0 (socket says 172.21.15.16:33396) 2024-08-31T06:16:00.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.473+0000 7f9ca2a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9b4fe2390 0x55f9b53b4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:50076/0 (socket says 172.21.15.16:50076) 2024-08-31T06:16:00.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.474+0000 7f9ca3400640 1 -- 172.21.15.16:0/2942397681 learned_addr learned my addr 172.21.15.16:0/2942397681 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:00.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.474+0000 7f9cb10d5740 10 client.0 _ll_register_callbacks cb 0x55f9b52bb430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:16:00.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.474+0000 7f9ca2a00640 1 -- 172.21.15.16:0/2942397681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9b4fda1b0 msgr2=0x55f9b53b44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:00.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.474+0000 7f9ca2a00640 1 --2- 172.21.15.16:0/2942397681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9b4fda1b0 0x55f9b53b44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:00.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.474+0000 7f9ca2a00640 1 -- 172.21.15.16:0/2942397681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9b5003070 msgr2=0x55f9b53c04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:00.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.474+0000 7f9ca2a00640 1 --2- 172.21.15.16:0/2942397681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9b5003070 0x55f9b53c04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:00.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.474+0000 7f9ca2a00640 1 -- 172.21.15.16:0/2942397681 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9b52d6870 con 0x55f9b4fe2390 2024-08-31T06:16:00.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.474+0000 7f9ca3400640 1 --2- 172.21.15.16:0/2942397681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9b4fda1b0 0x55f9b53b44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:16:00.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.474+0000 7f9ca2a00640 1 --2- 172.21.15.16:0/2942397681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9b4fe2390 0x55f9b53b4a20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9c900e2da0 tx=0x7f9c900ab570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=765f74959041f561 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:00.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.474+0000 7f9c8d600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:16:00.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.475+0000 7f9c8d600640 1 -- 172.21.15.16:0/2942397681 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c900abd40 con 0x55f9b4fe2390 2024-08-31T06:16:00.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.475+0000 7f9c8d600640 1 -- 172.21.15.16:0/2942397681 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8470+0+0 (secure 0 0 0) 0x7f9c900e3020 con 0x55f9b4fe2390 2024-08-31T06:16:00.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.475+0000 7f9cb10d5740 1 -- 172.21.15.16:0/2942397681 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c9c030e20 con 0x55f9b4fe2390 2024-08-31T06:16:00.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.475+0000 7f9c8d600640 1 -- 172.21.15.16:0/2942397681 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f9c900e3220 con 0x55f9b4fe2390 2024-08-31T06:16:00.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.475+0000 7f9cb10d5740 10 client.27596 fetch_fsmap learned FSMap version 42 2024-08-31T06:16:00.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.475+0000 7f9cb10d5740 1 -- 172.21.15.16:0/2942397681 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55f9b53c3180 con 0x55f9b4fe2390 2024-08-31T06:16:00.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.476+0000 7f9c8d600640 1 -- 172.21.15.16:0/2942397681 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c9c030e20 con 0x55f9b4fe2390 2024-08-31T06:16:00.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.476+0000 7f9c8d600640 1 -- 172.21.15.16:0/2942397681 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7f9c900b0100 con 0x55f9b4fe2390 2024-08-31T06:16:00.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.476+0000 7f9cb10d5740 10 client.27596 fetch_fsmap finished waiting for FSMap version 42 2024-08-31T06:16:00.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.476+0000 7f9cb10d5740 10 client.27596 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:16:00.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.476+0000 7f9cb10d5740 20 client.27596 populate_metadata read hostname 'smithi016' 2024-08-31T06:16:00.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.476+0000 7f9cb10d5740 1 --2- 172.21.15.16:0/2942397681 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f9b4fc19f0 0x55f9b53c6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:00.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.476+0000 7f9cb10d5740 4 client.27596 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:16:00.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.476+0000 7f9cb10d5740 1 -- 172.21.15.16:0/2942397681 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9b53216c0 con 0x55f9b4fc19f0 2024-08-31T06:16:00.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.477+0000 7f9ca3400640 1 --2- 172.21.15.16:0/2942397681 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f9b4fc19f0 0x55f9b53c6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:00.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.477+0000 7f9ca3400640 1 --2- 172.21.15.16:0/2942397681 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f9b4fc19f0 0x55f9b53c6650 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6acd5f787739f821 server_cookie=f6800c1547364db3 in_seq=0 out_seq=0 2024-08-31T06:16:00.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.477+0000 7f9c8d600640 10 client.27596 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:16:00.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.478+0000 7f9c8d600640 1 -- 172.21.15.16:0/2942397681 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f9b53216c0 con 0x55f9b4fc19f0 2024-08-31T06:16:00.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.478+0000 7f9c8d600640 10 client.27596 handle_command_reply: tid=0 2024-08-31T06:16:00.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.479+0000 7f9cb10d5740 1 client.27596 shutdown 2024-08-31T06:16:00.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.479+0000 7f9cb10d5740 1 -- 172.21.15.16:0/2942397681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9b4fe2390 msgr2=0x55f9b53b4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:00.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.479+0000 7f9cb10d5740 1 --2- 172.21.15.16:0/2942397681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9b4fe2390 0x55f9b53b4a20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9c900e2da0 tx=0x7f9c900ab570 comp rx=0 tx=0).stop 2024-08-31T06:16:00.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.479+0000 7f9cb10d5740 1 -- 172.21.15.16:0/2942397681 shutdown_connections 2024-08-31T06:16:00.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.479+0000 7f9cb10d5740 1 --2- 172.21.15.16:0/2942397681 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f9b4fc19f0 0x55f9b53c6650 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:00.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.479+0000 7f9cb10d5740 1 --2- 172.21.15.16:0/2942397681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9b5003070 0x55f9b53c04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:00.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.479+0000 7f9cb10d5740 1 --2- 172.21.15.16:0/2942397681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9b4fe2390 0x55f9b53b4a20 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:00.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.479+0000 7f9cb10d5740 1 --2- 172.21.15.16:0/2942397681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9b4fda1b0 0x55f9b53b44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:00.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.479+0000 7f9cb10d5740 1 -- 172.21.15.16:0/2942397681 >> 172.21.15.16:0/2942397681 conn(0x55f9b4ff7460 msgr2=0x55f9b53abb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:00.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.479+0000 7f9c8d600640 0 client.27596 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:16:00.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.480+0000 7f9cb10d5740 1 -- 172.21.15.16:0/2942397681 shutdown_connections 2024-08-31T06:16:00.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.480+0000 7f9cb10d5740 1 -- 172.21.15.16:0/2942397681 wait complete. 2024-08-31T06:16:00.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.480+0000 7f9cb10d5740 20 client.27596 trim_cache size 0 max 16384 2024-08-31T06:16:00.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.514+0000 7f9cb10d5740 1 Processor -- start 2024-08-31T06:16:00.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.515+0000 7f9cb10d5740 1 -- start start 2024-08-31T06:16:00.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.515+0000 7f9cb10d5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9b4fda1b0 0x55f9b53aa340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:00.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.515+0000 7f9cb10d5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9b4fe2390 0x55f9b53af490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:00.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.515+0000 7f9cb10d5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9b5003070 0x55f9b53c03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:00.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.515+0000 7f9cb10d5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca41c0c30 con 0x55f9b5003070 2024-08-31T06:16:00.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.515+0000 7f9cb10d5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca41be470 con 0x55f9b4fe2390 2024-08-31T06:16:00.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.515+0000 7f9cb10d5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca41bc300 con 0x55f9b4fda1b0 2024-08-31T06:16:00.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.516+0000 7f9ca3400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9b4fda1b0 0x55f9b53aa340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:00.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.516+0000 7f9ca2a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9b4fe2390 0x55f9b53af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:00.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.516+0000 7f9ca2a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9b4fe2390 0x55f9b53af490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:50086/0 (socket says 172.21.15.16:50086) 2024-08-31T06:16:00.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.516+0000 7f9ca3400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9b4fda1b0 0x55f9b53aa340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:33400/0 (socket says 172.21.15.16:33400) 2024-08-31T06:16:00.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.516+0000 7f9ca2a00640 1 -- 172.21.15.16:0/3245078930 learned_addr learned my addr 172.21.15.16:0/3245078930 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:00.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.516+0000 7f9ca3e00640 1 --2- 172.21.15.16:0/3245078930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9b5003070 0x55f9b53c03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:00.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.516+0000 7f9ca2a00640 1 -- 172.21.15.16:0/3245078930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9b4fda1b0 msgr2=0x55f9b53aa340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:00.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.516+0000 7f9ca2a00640 1 --2- 172.21.15.16:0/3245078930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9b4fda1b0 0x55f9b53aa340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:00.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.516+0000 7f9ca2a00640 1 -- 172.21.15.16:0/3245078930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9b5003070 msgr2=0x55f9b53c03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:00.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.516+0000 7f9ca2a00640 1 --2- 172.21.15.16:0/3245078930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9b5003070 0x55f9b53c03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:00.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.516+0000 7f9ca2a00640 1 -- 172.21.15.16:0/3245078930 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9b53aac40 con 0x55f9b4fe2390 2024-08-31T06:16:00.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.517+0000 7f9ca2a00640 1 --2- 172.21.15.16:0/3245078930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9b4fe2390 0x55f9b53af490 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f9c900abf00 tx=0x7f9c90000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3957bf0cfa219ee0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:00.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.517+0000 7f9c8fe00640 1 -- 172.21.15.16:0/3245078930 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c900b4bd0 con 0x55f9b4fe2390 2024-08-31T06:16:00.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.517+0000 7f9c8fe00640 1 -- 172.21.15.16:0/3245078930 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c900b05b0 con 0x55f9b4fe2390 2024-08-31T06:16:00.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.517+0000 7f9c8fe00640 1 -- 172.21.15.16:0/3245078930 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c900b0100 con 0x55f9b4fe2390 2024-08-31T06:16:00.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.518+0000 7f9cb10d5740 1 -- 172.21.15.16:0/3245078930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9b4fe2390 msgr2=0x55f9b53af490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:00.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.518+0000 7f9cb10d5740 1 --2- 172.21.15.16:0/3245078930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9b4fe2390 0x55f9b53af490 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f9c900abf00 tx=0x7f9c90000bc0 comp rx=0 tx=0).stop 2024-08-31T06:16:00.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.518+0000 7f9cb10d5740 1 -- 172.21.15.16:0/3245078930 shutdown_connections 2024-08-31T06:16:00.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.518+0000 7f9cb10d5740 1 --2- 172.21.15.16:0/3245078930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9b5003070 0x55f9b53c03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:00.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.518+0000 7f9cb10d5740 1 --2- 172.21.15.16:0/3245078930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9b4fe2390 0x55f9b53af490 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:00.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.518+0000 7f9cb10d5740 1 --2- 172.21.15.16:0/3245078930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9b4fda1b0 0x55f9b53aa340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:00.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.518+0000 7f9cb10d5740 1 -- 172.21.15.16:0/3245078930 >> 172.21.15.16:0/3245078930 conn(0x55f9b4ff7460 msgr2=0x55f9b53b2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:00.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.519+0000 7f9cb10d5740 1 -- 172.21.15.16:0/3245078930 shutdown_connections 2024-08-31T06:16:00.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.519+0000 7f9cb10d5740 1 -- 172.21.15.16:0/3245078930 wait complete. 2024-08-31T06:16:00.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.520+0000 7f9cb10d5740 1 -- start start 2024-08-31T06:16:00.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.520+0000 7f9cb10d5740 1 Processor -- start 2024-08-31T06:16:00.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.521+0000 7f9cb10d5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9b4fda1b0 0x55f9b53bcba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:00.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.521+0000 7f9cb10d5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9b4fe2390 0x55f9b53b9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:00.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.521+0000 7f9cb10d5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9b5003070 0x55f9b53b95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:00.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.521+0000 7f9cb10d5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca41bc950 con 0x55f9b4fda1b0 2024-08-31T06:16:00.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.521+0000 7f9cb10d5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca41bdda0 con 0x55f9b5003070 2024-08-31T06:16:00.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.521+0000 7f9cb10d5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca41bd000 con 0x55f9b4fe2390 2024-08-31T06:16:00.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.521+0000 7f9cb10d5740 10 client.0 _ll_register_callbacks cb 0x7f9c900abb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:16:00.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.522+0000 7f9ca3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9b4fda1b0 0x55f9b53bcba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:00.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.521+0000 7f9ca3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9b5003070 0x55f9b53b95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:00.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.522+0000 7f9ca3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9b5003070 0x55f9b53b95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:50088/0 (socket says 172.21.15.16:50088) 2024-08-31T06:16:00.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.522+0000 7f9ca3e00640 1 -- 172.21.15.16:0/3067771424 learned_addr learned my addr 172.21.15.16:0/3067771424 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:00.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.522+0000 7f9ca2a00640 1 --2- 172.21.15.16:0/3067771424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9b4fe2390 0x55f9b53b9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:00.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.522+0000 7f9ca3e00640 1 -- 172.21.15.16:0/3067771424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9b4fe2390 msgr2=0x55f9b53b9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:00.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.522+0000 7f9ca3e00640 1 --2- 172.21.15.16:0/3067771424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9b4fe2390 0x55f9b53b9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:00.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.522+0000 7f9ca3e00640 1 -- 172.21.15.16:0/3067771424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9b4fda1b0 msgr2=0x55f9b53bcba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:00.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.522+0000 7f9ca3e00640 1 --2- 172.21.15.16:0/3067771424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9b4fda1b0 0x55f9b53bcba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:00.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.522+0000 7f9ca3e00640 1 -- 172.21.15.16:0/3067771424 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9b53b9ce0 con 0x55f9b5003070 2024-08-31T06:16:00.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.522+0000 7f9ca3400640 1 --2- 172.21.15.16:0/3067771424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9b4fda1b0 0x55f9b53bcba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:16:00.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.523+0000 7f9ca3e00640 1 --2- 172.21.15.16:0/3067771424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9b5003070 0x55f9b53b95a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c02ac80 tx=0x7f9c9c009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b6d6b472b886005 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:00.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.523+0000 7f9c8d600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:16:00.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.523+0000 7f9c8d600640 1 -- 172.21.15.16:0/3067771424 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c036040 con 0x55f9b5003070 2024-08-31T06:16:00.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.523+0000 7f9c8d600640 1 -- 172.21.15.16:0/3067771424 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8470+0+0 (secure 0 0 0) 0x7f9c9c030b80 con 0x55f9b5003070 2024-08-31T06:16:00.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.523+0000 7f9cb10d5740 1 -- 172.21.15.16:0/3067771424 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c900b05b0 con 0x55f9b5003070 2024-08-31T06:16:00.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.523+0000 7f9c8d600640 1 -- 172.21.15.16:0/3067771424 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f9c9c030d80 con 0x55f9b5003070 2024-08-31T06:16:00.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.524+0000 7f9cb10d5740 10 client.27602 fetch_fsmap learned FSMap version 42 2024-08-31T06:16:00.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.524+0000 7f9cb10d5740 1 -- 172.21.15.16:0/3067771424 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55f9b5431ee0 con 0x55f9b5003070 2024-08-31T06:16:00.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.524+0000 7f9c8d600640 1 -- 172.21.15.16:0/3067771424 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c900b05b0 con 0x55f9b5003070 2024-08-31T06:16:00.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.524+0000 7f9c8d600640 1 -- 172.21.15.16:0/3067771424 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7f9c9c057030 con 0x55f9b5003070 2024-08-31T06:16:00.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.524+0000 7f9cb10d5740 10 client.27602 fetch_fsmap finished waiting for FSMap version 42 2024-08-31T06:16:00.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.524+0000 7f9cb10d5740 10 client.27602 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:16:00.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.524+0000 7f9cb10d5740 20 client.27602 populate_metadata read hostname 'smithi016' 2024-08-31T06:16:00.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.524+0000 7f9cb10d5740 1 --2- 172.21.15.16:0/3067771424 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f9b4f96da0 0x55f9b54341c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:00.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.525+0000 7f9cb10d5740 4 client.27602 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T06:16:00.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.525+0000 7f9cb10d5740 1 -- 172.21.15.16:0/3067771424 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f9b53a6ab0 con 0x55f9b4f96da0 2024-08-31T06:16:00.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.525+0000 7f9ca3400640 1 --2- 172.21.15.16:0/3067771424 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f9b4f96da0 0x55f9b54341c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:00.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.526+0000 7f9c8d600640 10 client.27602 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:16:00.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.526+0000 7f9ca3400640 1 --2- 172.21.15.16:0/3067771424 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f9b4f96da0 0x55f9b54341c0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43293231fe9963f1 server_cookie=3c3b27e62e853d36 in_seq=0 out_seq=0 2024-08-31T06:16:00.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.527+0000 7f9c8d600640 1 -- 172.21.15.16:0/3067771424 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3694 (crc 0 0 0) 0x55f9b53a6ab0 con 0x55f9b4f96da0 2024-08-31T06:16:00.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.527+0000 7f9c8d600640 10 client.27602 handle_command_reply: tid=0 2024-08-31T06:16:00.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.527+0000 7f9cb10d5740 1 client.27602 shutdown 2024-08-31T06:16:00.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.528+0000 7f9cb10d5740 1 -- 172.21.15.16:0/3067771424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9b5003070 msgr2=0x55f9b53b95a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:00.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.528+0000 7f9cb10d5740 1 --2- 172.21.15.16:0/3067771424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9b5003070 0x55f9b53b95a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c02ac80 tx=0x7f9c9c009800 comp rx=0 tx=0).stop 2024-08-31T06:16:00.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.528+0000 7f9cb10d5740 1 -- 172.21.15.16:0/3067771424 shutdown_connections 2024-08-31T06:16:00.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.528+0000 7f9cb10d5740 1 --2- 172.21.15.16:0/3067771424 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f9b4f96da0 0x55f9b54341c0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:00.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.528+0000 7f9cb10d5740 1 --2- 172.21.15.16:0/3067771424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9b4fe2390 0x55f9b53b9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:00.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.528+0000 7f9cb10d5740 1 --2- 172.21.15.16:0/3067771424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9b5003070 0x55f9b53b95a0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:00.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.528+0000 7f9cb10d5740 1 --2- 172.21.15.16:0/3067771424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9b4fda1b0 0x55f9b53bcba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:00.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.528+0000 7f9cb10d5740 1 -- 172.21.15.16:0/3067771424 >> 172.21.15.16:0/3067771424 conn(0x55f9b4ff7460 msgr2=0x55f9b53aba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:00.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.528+0000 7f9c8d600640 0 client.27602 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:16:00.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.529+0000 7f9cb10d5740 1 -- 172.21.15.16:0/3067771424 shutdown_connections 2024-08-31T06:16:00.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.529+0000 7f9cb10d5740 1 -- 172.21.15.16:0/3067771424 wait complete. 2024-08-31T06:16:00.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.529+0000 7f9cb10d5740 20 client.27602 trim_cache size 0 max 16384 2024-08-31T06:16:00.533 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:16:00.533 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T06:16:00.533 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-08-31T06:16:00.534 INFO:teuthology.orchestra.run.smithi016.stdout: "f9c4a22e-0c8a-4928-9b0d-e2ce1db5a86b": { 2024-08-31T06:16:00.534 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:16:00.534 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "f9c4a22e-0c8a-4928-9b0d-e2ce1db5a86b", 2024-08-31T06:16:00.534 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-08-31T06:16:00.534 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:16:00.534 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:16:00.534 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-08-31T06:16:00.534 INFO:teuthology.orchestra.run.smithi016.stdout: "f9c4a22e-0c8a-4928-9b0d-e2ce1db5a86b": { 2024-08-31T06:16:00.534 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:15:57.496015+0000", 2024-08-31T06:16:00.534 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:16:00.534 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:16:00.534 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:16:00.534 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:16:00.534 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:16:00.535 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:16:00.535 INFO:teuthology.orchestra.run.smithi016.stdout: "a64d5301-8558-4e9c-ac9b-8d0ac7057e74": { 2024-08-31T06:16:00.535 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:14:20.338104+0000", 2024-08-31T06:16:00.535 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:16:00.535 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:16:00.535 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:16:00.535 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:16:00.535 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:16:00.535 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:16:00.535 INFO:teuthology.orchestra.run.smithi016.stdout: "e739a16e-8ace-43a8-a1ce-ac27d39ef8a3": { 2024-08-31T06:16:00.535 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:54.084646+0000", 2024-08-31T06:16:00.535 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:16:00.535 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:16:00.536 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:16:00.536 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:16:00.536 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:16:00.536 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:16:00.536 INFO:teuthology.orchestra.run.smithi016.stdout: "91b72a34-4ba9-4126-a414-8b862f8c5f78": { 2024-08-31T06:16:00.536 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:13:28.663237+0000", 2024-08-31T06:16:00.536 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:16:00.536 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:16:00.536 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:16:00.536 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:16:00.536 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:16:00.536 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:16:00.536 INFO:teuthology.orchestra.run.smithi016.stdout: "a10cce76-99ab-46b5-b3b2-07b82899b8f9": { 2024-08-31T06:16:00.536 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:11:43.967546+0000", 2024-08-31T06:16:00.537 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:16:00.537 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:16:00.537 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:16:00.537 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:16:00.537 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:16:00.537 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:16:00.537 INFO:teuthology.orchestra.run.smithi016.stdout: "c1fd6904-2e02-47fa-939a-47349b9fe36b": { 2024-08-31T06:16:00.537 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:06.678691+0000", 2024-08-31T06:16:00.537 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:16:00.537 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:16:00.537 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:16:00.537 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:16:00.537 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:16:00.537 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:16:00.538 INFO:teuthology.orchestra.run.smithi016.stdout: "78d6ef02-8551-45e8-ad60-9627eb573fc1": { 2024-08-31T06:16:00.538 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:09:13.547865+0000", 2024-08-31T06:16:00.538 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:16:00.538 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:16:00.538 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:16:00.538 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:16:00.538 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:16:00.538 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:16:00.538 INFO:teuthology.orchestra.run.smithi016.stdout: "c8455d74-2fe7-4d93-bcf2-19b419d16016": { 2024-08-31T06:16:00.538 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:08:24.035960+0000", 2024-08-31T06:16:00.538 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:16:00.538 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:16:00.538 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:16:00.539 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:16:00.539 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:16:00.539 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:16:00.539 INFO:teuthology.orchestra.run.smithi016.stdout: "0fdcd762-533d-42be-9289-b71c8e3b81b1": { 2024-08-31T06:16:00.539 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:15:06.169112+0000", 2024-08-31T06:16:00.539 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:16:00.539 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:16:00.539 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:16:00.539 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:16:00.539 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:16:00.539 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:16:00.539 INFO:teuthology.orchestra.run.smithi016.stdout: "88d7bdb8-cb1c-45e5-ad29-4cd8da6c6003": { 2024-08-31T06:16:00.539 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:07:39.648942+0000", 2024-08-31T06:16:00.539 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:16:00.540 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:16:00.540 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:16:00.540 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:16:00.540 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:16:00.540 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:16:00.540 INFO:teuthology.orchestra.run.smithi016.stdout: "6a03046e-cdf4-4a14-b9bd-b560b8968467": { 2024-08-31T06:16:00.540 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:12:38.002354+0000", 2024-08-31T06:16:00.540 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:16:00.540 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:16:00.540 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:16:00.540 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:16:00.540 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:16:00.540 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:16:00.540 INFO:teuthology.orchestra.run.smithi016.stdout: "c300b7e8-f3fd-482c-a909-9329bc61e570": { 2024-08-31T06:16:00.541 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:06:49.868046+0000", 2024-08-31T06:16:00.541 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:16:00.541 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:16:00.541 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:16:00.541 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:16:00.541 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:16:00.541 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:16:00.541 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:16:00.541 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:16:00.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.532+0000 7f9caa400640 1 -- 172.21.15.16:0/28332279 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9c880b4da0 msgr2=0x7f9c880b72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:00.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.532+0000 7f9caa400640 1 --2- 172.21.15.16:0/28332279 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9c880b4da0 0x7f9c880b72b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x55f9b53a75d0 tx=0x7f9c94005f90 comp rx=0 tx=0).stop 2024-08-31T06:16:00.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.532+0000 7f9caa400640 1 -- 172.21.15.16:0/28332279 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ca4109270 msgr2=0x7f9ca4113850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:00.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.532+0000 7f9caa400640 1 --2- 172.21.15.16:0/28332279 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ca4109270 0x7f9ca4113850 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9ca406b640 tx=0x7f9c9000ee50 comp rx=0 tx=0).stop 2024-08-31T06:16:00.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.532+0000 7f9caa400640 1 -- 172.21.15.16:0/28332279 shutdown_connections 2024-08-31T06:16:00.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.532+0000 7f9caa400640 1 --2- 172.21.15.16:0/28332279 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9c880b4da0 0x7f9c880b72b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:00.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.532+0000 7f9caa400640 1 --2- 172.21.15.16:0/28332279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ca4109bc0 0x7f9ca4118b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:00.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.532+0000 7f9caa400640 1 --2- 172.21.15.16:0/28332279 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ca4109270 0x7f9ca4113850 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:00.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.532+0000 7f9caa400640 1 --2- 172.21.15.16:0/28332279 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ca4103930 0x7f9ca41132d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:00.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.532+0000 7f9caa400640 1 -- 172.21.15.16:0/28332279 >> 172.21.15.16:0/28332279 conn(0x7f9ca40ff380 msgr2=0x7f9ca41017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:00.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.534+0000 7f9caa400640 1 -- 172.21.15.16:0/28332279 shutdown_connections 2024-08-31T06:16:00.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:00.534+0000 7f9caa400640 1 -- 172.21.15.16:0/28332279 wait complete. 2024-08-31T06:16:00.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:00 smithi184 ceph-mon[22331]: pgmap v633: 97 pgs: 97 active+clean; 3.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 7.9 MiB/s wr, 831 op/s 2024-08-31T06:16:01.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:00 smithi155 ceph-mon[21742]: pgmap v633: 97 pgs: 97 active+clean; 3.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 7.9 MiB/s wr, 831 op/s 2024-08-31T06:16:01.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:00 smithi016 ceph-mon[17194]: pgmap v633: 97 pgs: 97 active+clean; 3.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 7.9 MiB/s wr, 831 op/s 2024-08-31T06:16:01.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 515.0 s, 1553.6 tps, lat 61.060 ms stddev 84.412 2024-08-31T06:16:02.084 INFO:tasks.cephfs.filesystem:scrub status for tag:f9c4a22e-0c8a-4928-9b0d-e2ce1db5a86b - {'path': '/', 'tag': 'f9c4a22e-0c8a-4928-9b0d-e2ce1db5a86b', 'options': 'recursive,force'} 2024-08-31T06:16:02.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:02 smithi184 ceph-mon[22331]: pgmap v634: 97 pgs: 97 active+clean; 3.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 7.3 MiB/s wr, 815 op/s 2024-08-31T06:16:03.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:02 smithi155 ceph-mon[21742]: pgmap v634: 97 pgs: 97 active+clean; 3.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 7.3 MiB/s wr, 815 op/s 2024-08-31T06:16:03.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:02 smithi016 ceph-mon[17194]: pgmap v634: 97 pgs: 97 active+clean; 3.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 7.3 MiB/s wr, 815 op/s 2024-08-31T06:16:03.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:03 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:16:04.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:03 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:16:04.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:03 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:16:04.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:04 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:16:04.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:04 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:16:04.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:04 smithi184 ceph-mon[22331]: pgmap v635: 97 pgs: 97 active+clean; 3.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 6.1 MiB/s wr, 761 op/s 2024-08-31T06:16:04.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:04 smithi184 ceph-mon[22331]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:16:05.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:04 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:16:05.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:04 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:16:05.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:04 smithi155 ceph-mon[21742]: pgmap v635: 97 pgs: 97 active+clean; 3.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 6.1 MiB/s wr, 761 op/s 2024-08-31T06:16:05.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:04 smithi155 ceph-mon[21742]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:16:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.139+0000 7f5e82c00640 1 -- 172.21.15.16:0/2515593847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e7c1038a0 msgr2=0x7f5e7c103cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.139+0000 7f5e82c00640 1 --2- 172.21.15.16:0/2515593847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e7c1038a0 0x7f5e7c103cb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c009900 tx=0x7f5e6c02a3f0 comp rx=0 tx=0).stop 2024-08-31T06:16:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.139+0000 7f5e82c00640 1 -- 172.21.15.16:0/2515593847 shutdown_connections 2024-08-31T06:16:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.139+0000 7f5e82c00640 1 --2- 172.21.15.16:0/2515593847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e7c109b20 0x7f5e7c110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.139+0000 7f5e82c00640 1 --2- 172.21.15.16:0/2515593847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e7c1091d0 0x7f5e7c1095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.139+0000 7f5e82c00640 1 --2- 172.21.15.16:0/2515593847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e7c1038a0 0x7f5e7c103cb0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.139+0000 7f5e82c00640 1 -- 172.21.15.16:0/2515593847 >> 172.21.15.16:0/2515593847 conn(0x7f5e7c0ff2a0 msgr2=0x7f5e7c1016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.139+0000 7f5e82c00640 1 -- 172.21.15.16:0/2515593847 shutdown_connections 2024-08-31T06:16:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.140+0000 7f5e82c00640 1 -- 172.21.15.16:0/2515593847 wait complete. 2024-08-31T06:16:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.141+0000 7f5e82c00640 1 Processor -- start 2024-08-31T06:16:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.141+0000 7f5e82c00640 1 -- start start 2024-08-31T06:16:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.141+0000 7f5e82c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e7c1038a0 0x7f5e7c19a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.141+0000 7f5e82c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e7c1091d0 0x7f5e7c19a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.141+0000 7f5e82c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e7c109b20 0x7f5e7c19fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.142+0000 7f5e82c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5e7c113810 con 0x7f5e7c1091d0 2024-08-31T06:16:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.142+0000 7f5e82c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5e7c113690 con 0x7f5e7c1038a0 2024-08-31T06:16:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.142+0000 7f5e82c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5e7c113990 con 0x7f5e7c109b20 2024-08-31T06:16:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.142+0000 7f5e7be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e7c1038a0 0x7f5e7c19a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.142+0000 7f5e80c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e7c109b20 0x7f5e7c19fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.142+0000 7f5e7b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e7c1091d0 0x7f5e7c19a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.142+0000 7f5e80c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e7c109b20 0x7f5e7c19fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:33420/0 (socket says 172.21.15.16:33420) 2024-08-31T06:16:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.142+0000 7f5e7be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e7c1038a0 0x7f5e7c19a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:50106/0 (socket says 172.21.15.16:50106) 2024-08-31T06:16:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.142+0000 7f5e80c00640 1 -- 172.21.15.16:0/4262842811 learned_addr learned my addr 172.21.15.16:0/4262842811 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.142+0000 7f5e80c00640 1 -- 172.21.15.16:0/4262842811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e7c1038a0 msgr2=0x7f5e7c19a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.142+0000 7f5e80c00640 1 --2- 172.21.15.16:0/4262842811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e7c1038a0 0x7f5e7c19a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.142+0000 7f5e80c00640 1 -- 172.21.15.16:0/4262842811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e7c1091d0 msgr2=0x7f5e7c19a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.142+0000 7f5e80c00640 1 --2- 172.21.15.16:0/4262842811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e7c1091d0 0x7f5e7c19a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.142+0000 7f5e80c00640 1 -- 172.21.15.16:0/4262842811 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e7c1a0140 con 0x7f5e7c109b20 2024-08-31T06:16:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.143+0000 7f5e80c00640 1 --2- 172.21.15.16:0/4262842811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e7c109b20 0x7f5e7c19fa40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5e7400eb10 tx=0x7f5e7400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.143+0000 7f5e78c00640 1 -- 172.21.15.16:0/4262842811 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e74004b20 con 0x7f5e7c109b20 2024-08-31T06:16:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.143+0000 7f5e82c00640 1 -- 172.21.15.16:0/4262842811 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e7c1a0430 con 0x7f5e7c109b20 2024-08-31T06:16:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.143+0000 7f5e82c00640 1 -- 172.21.15.16:0/4262842811 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e7c1a6bc0 con 0x7f5e7c109b20 2024-08-31T06:16:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.143+0000 7f5e78c00640 1 -- 172.21.15.16:0/4262842811 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e74004cc0 con 0x7f5e7c109b20 2024-08-31T06:16:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.143+0000 7f5e78c00640 1 -- 172.21.15.16:0/4262842811 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e74010690 con 0x7f5e7c109b20 2024-08-31T06:16:05.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.146+0000 7f5e89c45740 1 Processor -- start 2024-08-31T06:16:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.147+0000 7f5e78c00640 1 -- 172.21.15.16:0/4262842811 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f5e74010830 con 0x7f5e7c109b20 2024-08-31T06:16:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.147+0000 7f5e89c45740 1 -- start start 2024-08-31T06:16:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.147+0000 7f5e89c45740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3177e31b0 0x55a317b8ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.147+0000 7f5e89c45740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3177ba2f0 0x55a317b8b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.147+0000 7f5e89c45740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3177c24d0 0x55a317b8b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.147+0000 7f5e89c45740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a317afbf70 con 0x55a3177c24d0 2024-08-31T06:16:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.147+0000 7f5e89c45740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a317aeb500 con 0x55a3177ba2f0 2024-08-31T06:16:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.147+0000 7f5e89c45740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a317aef4b0 con 0x55a3177e31b0 2024-08-31T06:16:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.148+0000 7f5e7be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3177ba2f0 0x55a317b8b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.148+0000 7f5e7b400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3177e31b0 0x55a317b8ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.148+0000 7f5e7be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3177ba2f0 0x55a317b8b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:50116/0 (socket says 172.21.15.16:50116) 2024-08-31T06:16:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.148+0000 7f5e7be00640 1 -- 172.21.15.16:0/3226093498 learned_addr learned my addr 172.21.15.16:0/3226093498 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.148+0000 7f5e7b400640 1 --2- 172.21.15.16:0/3226093498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3177c24d0 0x55a317b8b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.148+0000 7f5e7be00640 1 -- 172.21.15.16:0/3226093498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3177e31b0 msgr2=0x55a317b8ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.148+0000 7f5e7be00640 1 --2- 172.21.15.16:0/3226093498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3177e31b0 0x55a317b8ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.148+0000 7f5e7be00640 1 -- 172.21.15.16:0/3226093498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3177c24d0 msgr2=0x55a317b8b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.148+0000 7f5e7be00640 1 --2- 172.21.15.16:0/3226093498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3177c24d0 0x55a317b8b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.148+0000 7f5e7be00640 1 -- 172.21.15.16:0/3226093498 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a317aa5c10 con 0x55a3177ba2f0 2024-08-31T06:16:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.149+0000 7f5e7be00640 1 --2- 172.21.15.16:0/3226093498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3177ba2f0 0x55a317b8b270 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c009900 tx=0x7f5e6c004170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35a5ef98eb93478e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.149+0000 7f5e78c00640 1 --2- 172.21.15.16:0/4262842811 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f5e600b4d20 0x7f5e600b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.149+0000 7f5e78c00640 1 -- 172.21.15.16:0/4262842811 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8470+0+0 (secure 0 0 0) 0x7f5e740e5a40 con 0x7f5e7c109b20 2024-08-31T06:16:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.149+0000 7f5e7b400640 1 --2- 172.21.15.16:0/4262842811 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f5e600b4d20 0x7f5e600b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.149+0000 7f5e71200640 1 -- 172.21.15.16:0/3226093498 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c040070 con 0x55a3177ba2f0 2024-08-31T06:16:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.149+0000 7f5e71200640 1 -- 172.21.15.16:0/3226093498 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e6c005500 con 0x55a3177ba2f0 2024-08-31T06:16:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.149+0000 7f5e71200640 1 -- 172.21.15.16:0/3226093498 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c035040 con 0x55a3177ba2f0 2024-08-31T06:16:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.150+0000 7f5e89c45740 1 -- 172.21.15.16:0/3226093498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3177ba2f0 msgr2=0x55a317b8b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.150+0000 7f5e7b400640 1 --2- 172.21.15.16:0/4262842811 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f5e600b4d20 0x7f5e600b7230 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5e6800b6c0 tx=0x7f5e6800b410 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.150+0000 7f5e89c45740 1 --2- 172.21.15.16:0/3226093498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3177ba2f0 0x55a317b8b270 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c009900 tx=0x7f5e6c004170 comp rx=0 tx=0).stop 2024-08-31T06:16:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.150+0000 7f5e89c45740 1 -- 172.21.15.16:0/3226093498 shutdown_connections 2024-08-31T06:16:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.150+0000 7f5e89c45740 1 --2- 172.21.15.16:0/3226093498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3177c24d0 0x55a317b8b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.150+0000 7f5e89c45740 1 --2- 172.21.15.16:0/3226093498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3177ba2f0 0x55a317b8b270 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.150+0000 7f5e89c45740 1 --2- 172.21.15.16:0/3226093498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3177e31b0 0x55a317b8ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.150+0000 7f5e89c45740 1 -- 172.21.15.16:0/3226093498 >> 172.21.15.16:0/3226093498 conn(0x55a3177d7300 msgr2=0x55a317b88c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.150+0000 7f5e89c45740 1 -- 172.21.15.16:0/3226093498 shutdown_connections 2024-08-31T06:16:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.150+0000 7f5e89c45740 1 -- 172.21.15.16:0/3226093498 wait complete. 2024-08-31T06:16:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.151+0000 7f5e89c45740 1 -- start start 2024-08-31T06:16:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.151+0000 7f5e89c45740 1 Processor -- start 2024-08-31T06:16:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.152+0000 7f5e89c45740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3177ba2f0 0x55a317b923e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.152+0000 7f5e89c45740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3177c24d0 0x55a317b92920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.152+0000 7f5e89c45740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3177e31b0 0x55a317b97b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.152+0000 7f5e89c45740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a317a24ab0 con 0x55a3177ba2f0 2024-08-31T06:16:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.152+0000 7f5e89c45740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a317a92320 con 0x55a3177e31b0 2024-08-31T06:16:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.152+0000 7f5e89c45740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a317b84260 con 0x55a3177c24d0 2024-08-31T06:16:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.152+0000 7f5e89c45740 10 client.0 _ll_register_callbacks cb 0x55a317a986a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:16:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.152+0000 7f5e80c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3177e31b0 0x55a317b97b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.152+0000 7f5e7be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3177ba2f0 0x55a317b923e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.152+0000 7f5e7b400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3177c24d0 0x55a317b92920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.152+0000 7f5e80c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3177e31b0 0x55a317b97b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:50132/0 (socket says 172.21.15.16:50132) 2024-08-31T06:16:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.152+0000 7f5e80c00640 1 -- 172.21.15.16:0/3695337936 learned_addr learned my addr 172.21.15.16:0/3695337936 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.152+0000 7f5e80c00640 1 -- 172.21.15.16:0/3695337936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3177c24d0 msgr2=0x55a317b92920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.152+0000 7f5e80c00640 1 --2- 172.21.15.16:0/3695337936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3177c24d0 0x55a317b92920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.152+0000 7f5e80c00640 1 -- 172.21.15.16:0/3695337936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3177ba2f0 msgr2=0x55a317b923e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.152+0000 7f5e80c00640 1 --2- 172.21.15.16:0/3695337936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3177ba2f0 0x55a317b923e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.152+0000 7f5e80c00640 1 -- 172.21.15.16:0/3695337936 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a31784e790 con 0x55a3177e31b0 2024-08-31T06:16:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.153+0000 7f5e80c00640 1 --2- 172.21.15.16:0/3695337936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3177e31b0 0x55a317b97b70 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5e740a8180 tx=0x7f5e740abb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=140452edffb018fb server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.153+0000 7f5e65600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:16:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.153+0000 7f5e65600640 1 -- 172.21.15.16:0/3695337936 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e740ac100 con 0x55a3177e31b0 2024-08-31T06:16:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.153+0000 7f5e65600640 1 -- 172.21.15.16:0/3695337936 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8470+0+0 (secure 0 0 0) 0x7f5e740b51a0 con 0x55a3177e31b0 2024-08-31T06:16:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.153+0000 7f5e89c45740 1 -- 172.21.15.16:0/3695337936 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e6c005500 con 0x55a3177e31b0 2024-08-31T06:16:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.153+0000 7f5e65600640 1 -- 172.21.15.16:0/3695337936 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f5e740b0970 con 0x55a3177e31b0 2024-08-31T06:16:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.153+0000 7f5e89c45740 10 client.27614 fetch_fsmap learned FSMap version 43 2024-08-31T06:16:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.153+0000 7f5e89c45740 1 -- 172.21.15.16:0/3695337936 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55a3178c7620 con 0x55a3177e31b0 2024-08-31T06:16:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.153+0000 7f5e65600640 1 -- 172.21.15.16:0/3695337936 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e6c005500 con 0x55a3177e31b0 2024-08-31T06:16:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.154+0000 7f5e65600640 1 -- 172.21.15.16:0/3695337936 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f5e740a7a70 con 0x55a3177e31b0 2024-08-31T06:16:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.154+0000 7f5e89c45740 10 client.27614 fetch_fsmap finished waiting for FSMap version 43 2024-08-31T06:16:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.154+0000 7f5e89c45740 10 client.27614 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:16:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.154+0000 7f5e89c45740 20 client.27614 populate_metadata read hostname 'smithi016' 2024-08-31T06:16:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.154+0000 7f5e89c45740 1 --2- 172.21.15.16:0/3695337936 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a3177a1d80 0x55a317ba3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.154+0000 7f5e89c45740 4 client.27614 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:16:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.154+0000 7f5e89c45740 1 -- 172.21.15.16:0/3695337936 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a317afdec0 con 0x55a3177a1d80 2024-08-31T06:16:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.154+0000 7f5e7be00640 1 --2- 172.21.15.16:0/3695337936 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a3177a1d80 0x55a317ba3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.154+0000 7f5e7be00640 1 --2- 172.21.15.16:0/3695337936 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a3177a1d80 0x55a317ba3f90 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=853da0d20d1e46eb server_cookie=e0f7123a75c2aba8 in_seq=0 out_seq=0 2024-08-31T06:16:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.154+0000 7f5e65600640 10 client.27614 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:16:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.155+0000 7f5e65600640 1 -- 172.21.15.16:0/3695337936 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a317afdec0 con 0x55a3177a1d80 2024-08-31T06:16:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.155+0000 7f5e65600640 10 client.27614 handle_command_reply: tid=0 2024-08-31T06:16:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.155+0000 7f5e89c45740 1 client.27614 shutdown 2024-08-31T06:16:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.156+0000 7f5e89c45740 1 -- 172.21.15.16:0/3695337936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3177e31b0 msgr2=0x55a317b97b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.156+0000 7f5e89c45740 1 --2- 172.21.15.16:0/3695337936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3177e31b0 0x55a317b97b70 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5e740a8180 tx=0x7f5e740abb80 comp rx=0 tx=0).stop 2024-08-31T06:16:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.156+0000 7f5e89c45740 1 -- 172.21.15.16:0/3695337936 shutdown_connections 2024-08-31T06:16:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.156+0000 7f5e89c45740 1 --2- 172.21.15.16:0/3695337936 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a3177a1d80 0x55a317ba3f90 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.156+0000 7f5e89c45740 1 --2- 172.21.15.16:0/3695337936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3177c24d0 0x55a317b92920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.156+0000 7f5e89c45740 1 --2- 172.21.15.16:0/3695337936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3177e31b0 0x55a317b97b70 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.156+0000 7f5e89c45740 1 --2- 172.21.15.16:0/3695337936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3177ba2f0 0x55a317b923e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.156+0000 7f5e65600640 0 client.27614 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:16:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.156+0000 7f5e89c45740 1 -- 172.21.15.16:0/3695337936 >> 172.21.15.16:0/3695337936 conn(0x55a3177d7300 msgr2=0x55a317b8bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.156+0000 7f5e89c45740 1 -- 172.21.15.16:0/3695337936 shutdown_connections 2024-08-31T06:16:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.156+0000 7f5e89c45740 1 -- 172.21.15.16:0/3695337936 wait complete. 2024-08-31T06:16:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.156+0000 7f5e89c45740 20 client.27614 trim_cache size 0 max 16384 2024-08-31T06:16:05.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:04 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:16:05.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:04 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:16:05.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:04 smithi016 ceph-mon[17194]: pgmap v635: 97 pgs: 97 active+clean; 3.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 6.1 MiB/s wr, 761 op/s 2024-08-31T06:16:05.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:04 smithi016 ceph-mon[17194]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:16:05.176 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.174+0000 7f5e89c45740 1 Processor -- start 2024-08-31T06:16:05.177 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.174+0000 7f5e89c45740 1 -- start start 2024-08-31T06:16:05.177 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.174+0000 7f5e89c45740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3177ba2f0 0x55a317b8bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:05.177 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.174+0000 7f5e89c45740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3177c24d0 0x55a317b8c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:05.177 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.174+0000 7f5e89c45740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3177e31b0 0x55a317b8c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:05.177 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.174+0000 7f5e89c45740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5e7c1afa90 con 0x55a3177c24d0 2024-08-31T06:16:05.177 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.174+0000 7f5e89c45740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5e7c1ad300 con 0x55a3177e31b0 2024-08-31T06:16:05.177 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.174+0000 7f5e89c45740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5e7c1ab170 con 0x55a3177ba2f0 2024-08-31T06:16:05.178 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.174+0000 7f5e7b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3177c24d0 0x55a317b8c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:05.178 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.175+0000 7f5e80c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3177e31b0 0x55a317b8c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:05.178 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.175+0000 7f5e7be00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3177ba2f0 0x55a317b8bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:05.178 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.175+0000 7f5e7be00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3177ba2f0 0x55a317b8bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:33428/0 (socket says 172.21.15.16:33428) 2024-08-31T06:16:05.178 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.175+0000 7f5e80c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3177e31b0 0x55a317b8c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:50138/0 (socket says 172.21.15.16:50138) 2024-08-31T06:16:05.178 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.175+0000 7f5e7be00640 1 -- 172.21.15.16:0/1467902172 learned_addr learned my addr 172.21.15.16:0/1467902172 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.175+0000 7f5e7be00640 1 -- 172.21.15.16:0/1467902172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3177e31b0 msgr2=0x55a317b8c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.175+0000 7f5e7be00640 1 --2- 172.21.15.16:0/1467902172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3177e31b0 0x55a317b8c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.175+0000 7f5e7be00640 1 -- 172.21.15.16:0/1467902172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3177c24d0 msgr2=0x55a317b8c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.175+0000 7f5e7be00640 1 --2- 172.21.15.16:0/1467902172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3177c24d0 0x55a317b8c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.175+0000 7f5e7be00640 1 -- 172.21.15.16:0/1467902172 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a317b92d10 con 0x55a3177ba2f0 2024-08-31T06:16:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.175+0000 7f5e7be00640 1 --2- 172.21.15.16:0/1467902172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3177ba2f0 0x55a317b8bd60 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c005f70 tx=0x7f5e6c048040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=103c680d1ab50d10 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.176+0000 7f5e67e00640 1 -- 172.21.15.16:0/1467902172 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c040040 con 0x55a3177ba2f0 2024-08-31T06:16:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.176+0000 7f5e67e00640 1 -- 172.21.15.16:0/1467902172 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e6c004970 con 0x55a3177ba2f0 2024-08-31T06:16:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.176+0000 7f5e67e00640 1 -- 172.21.15.16:0/1467902172 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c035040 con 0x55a3177ba2f0 2024-08-31T06:16:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.177+0000 7f5e89c45740 1 -- 172.21.15.16:0/1467902172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3177ba2f0 msgr2=0x55a317b8bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.177+0000 7f5e89c45740 1 --2- 172.21.15.16:0/1467902172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3177ba2f0 0x55a317b8bd60 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c005f70 tx=0x7f5e6c048040 comp rx=0 tx=0).stop 2024-08-31T06:16:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.177+0000 7f5e89c45740 1 -- 172.21.15.16:0/1467902172 shutdown_connections 2024-08-31T06:16:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.177+0000 7f5e89c45740 1 --2- 172.21.15.16:0/1467902172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3177e31b0 0x55a317b8c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.177+0000 7f5e89c45740 1 --2- 172.21.15.16:0/1467902172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3177c24d0 0x55a317b8c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.177+0000 7f5e89c45740 1 --2- 172.21.15.16:0/1467902172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3177ba2f0 0x55a317b8bd60 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.177+0000 7f5e89c45740 1 -- 172.21.15.16:0/1467902172 >> 172.21.15.16:0/1467902172 conn(0x55a3177d7300 msgr2=0x55a317b90c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.177+0000 7f5e89c45740 1 -- 172.21.15.16:0/1467902172 shutdown_connections 2024-08-31T06:16:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.177+0000 7f5e89c45740 1 -- 172.21.15.16:0/1467902172 wait complete. 2024-08-31T06:16:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.178+0000 7f5e89c45740 1 -- start start 2024-08-31T06:16:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.178+0000 7f5e89c45740 1 Processor -- start 2024-08-31T06:16:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.179+0000 7f5e89c45740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3177ba2f0 0x55a317b99180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.179+0000 7f5e89c45740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3177c24d0 0x55a317b99740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.179+0000 7f5e89c45740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3177e31b0 0x55a317b9e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.179+0000 7f5e89c45740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5e7c1ab7c0 con 0x55a3177c24d0 2024-08-31T06:16:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.179+0000 7f5e89c45740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5e7c1acc30 con 0x55a3177e31b0 2024-08-31T06:16:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.179+0000 7f5e89c45740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5e7c1abe90 con 0x55a3177ba2f0 2024-08-31T06:16:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.179+0000 7f5e89c45740 10 client.0 _ll_register_callbacks cb 0x7f5e740abf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:16:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.179+0000 7f5e7be00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3177ba2f0 0x55a317b99180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.179+0000 7f5e7be00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3177ba2f0 0x55a317b99180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:33436/0 (socket says 172.21.15.16:33436) 2024-08-31T06:16:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.179+0000 7f5e7be00640 1 -- 172.21.15.16:0/2247072287 learned_addr learned my addr 172.21.15.16:0/2247072287 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.179+0000 7f5e80c00640 1 --2- 172.21.15.16:0/2247072287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3177e31b0 0x55a317b9e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.179+0000 7f5e7b400640 1 --2- 172.21.15.16:0/2247072287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3177c24d0 0x55a317b99740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.179+0000 7f5e7be00640 1 -- 172.21.15.16:0/2247072287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3177e31b0 msgr2=0x55a317b9e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.179+0000 7f5e7be00640 1 --2- 172.21.15.16:0/2247072287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3177e31b0 0x55a317b9e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.179+0000 7f5e7be00640 1 -- 172.21.15.16:0/2247072287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3177c24d0 msgr2=0x55a317b99740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.180+0000 7f5e7be00640 1 --2- 172.21.15.16:0/2247072287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3177c24d0 0x55a317b99740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.180+0000 7f5e7be00640 1 -- 172.21.15.16:0/2247072287 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a317b9e710 con 0x55a3177ba2f0 2024-08-31T06:16:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.180+0000 7f5e80c00640 1 --2- 172.21.15.16:0/2247072287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3177e31b0 0x55a317b9e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:16:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.180+0000 7f5e7b400640 1 --2- 172.21.15.16:0/2247072287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3177c24d0 0x55a317b99740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:16:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.180+0000 7f5e7be00640 1 --2- 172.21.15.16:0/2247072287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3177ba2f0 0x55a317b99180 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c009900 tx=0x7f5e6c005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63b0da21d0bfe0af server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.180+0000 7f5e65600640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:16:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.180+0000 7f5e65600640 1 -- 172.21.15.16:0/2247072287 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c040030 con 0x55a3177ba2f0 2024-08-31T06:16:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.181+0000 7f5e65600640 1 -- 172.21.15.16:0/2247072287 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8470+0+0 (secure 0 0 0) 0x7f5e6c005230 con 0x55a3177ba2f0 2024-08-31T06:16:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.181+0000 7f5e89c45740 1 -- 172.21.15.16:0/2247072287 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e6c004970 con 0x55a3177ba2f0 2024-08-31T06:16:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.181+0000 7f5e65600640 1 -- 172.21.15.16:0/2247072287 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f5e6c035040 con 0x55a3177ba2f0 2024-08-31T06:16:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.181+0000 7f5e89c45740 10 client.27607 fetch_fsmap learned FSMap version 43 2024-08-31T06:16:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.181+0000 7f5e89c45740 1 -- 172.21.15.16:0/2247072287 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55a317c0f080 con 0x55a3177ba2f0 2024-08-31T06:16:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.181+0000 7f5e65600640 1 -- 172.21.15.16:0/2247072287 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e6c004970 con 0x55a3177ba2f0 2024-08-31T06:16:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.182+0000 7f5e65600640 1 -- 172.21.15.16:0/2247072287 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f5e6c051470 con 0x55a3177ba2f0 2024-08-31T06:16:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.182+0000 7f5e89c45740 10 client.27607 fetch_fsmap finished waiting for FSMap version 43 2024-08-31T06:16:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.182+0000 7f5e89c45740 10 client.27607 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:16:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.182+0000 7f5e89c45740 20 client.27607 populate_metadata read hostname 'smithi016' 2024-08-31T06:16:05.188 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.182+0000 7f5e89c45740 1 --2- 172.21.15.16:0/2247072287 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a3177771c0 0x55a317c11360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:05.188 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.182+0000 7f5e89c45740 4 client.27607 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:16:05.188 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.182+0000 7f5e89c45740 1 -- 172.21.15.16:0/2247072287 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a317af9960 con 0x55a3177771c0 2024-08-31T06:16:05.188 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.182+0000 7f5e7b400640 1 --2- 172.21.15.16:0/2247072287 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a3177771c0 0x55a317c11360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:05.188 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.183+0000 7f5e65600640 10 client.27607 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:16:05.188 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.183+0000 7f5e7b400640 1 --2- 172.21.15.16:0/2247072287 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a3177771c0 0x55a317c11360 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa0b9b3597c9f994 server_cookie=7711c08680a83cc2 in_seq=0 out_seq=0 2024-08-31T06:16:05.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.200+0000 7f5e65600640 1 -- 172.21.15.16:0/2247072287 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a317af9960 con 0x55a3177771c0 2024-08-31T06:16:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.200+0000 7f5e65600640 10 client.27607 handle_command_reply: tid=0 2024-08-31T06:16:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.200+0000 7f5e89c45740 1 client.27607 shutdown 2024-08-31T06:16:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.201+0000 7f5e89c45740 1 -- 172.21.15.16:0/2247072287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3177ba2f0 msgr2=0x55a317b99180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.201+0000 7f5e89c45740 1 --2- 172.21.15.16:0/2247072287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3177ba2f0 0x55a317b99180 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c009900 tx=0x7f5e6c005f00 comp rx=0 tx=0).stop 2024-08-31T06:16:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.201+0000 7f5e89c45740 1 -- 172.21.15.16:0/2247072287 shutdown_connections 2024-08-31T06:16:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.201+0000 7f5e89c45740 1 --2- 172.21.15.16:0/2247072287 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a3177771c0 0x55a317c11360 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.201+0000 7f5e89c45740 1 --2- 172.21.15.16:0/2247072287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3177e31b0 0x55a317b9e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.201+0000 7f5e89c45740 1 --2- 172.21.15.16:0/2247072287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3177c24d0 0x55a317b99740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.201+0000 7f5e65600640 0 client.27607 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:16:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.201+0000 7f5e89c45740 1 --2- 172.21.15.16:0/2247072287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3177ba2f0 0x55a317b99180 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.201+0000 7f5e89c45740 1 -- 172.21.15.16:0/2247072287 >> 172.21.15.16:0/2247072287 conn(0x55a3177d7300 msgr2=0x55a317b959d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.201+0000 7f5e89c45740 1 -- 172.21.15.16:0/2247072287 shutdown_connections 2024-08-31T06:16:05.205 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.201+0000 7f5e89c45740 1 -- 172.21.15.16:0/2247072287 wait complete. 2024-08-31T06:16:05.205 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.201+0000 7f5e89c45740 20 client.27607 trim_cache size 0 max 16384 2024-08-31T06:16:05.205 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:16:05.205 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:16:05.205 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:16:05.205 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:16:05.206 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.202+0000 7f5e82c00640 1 -- 172.21.15.16:0/4262842811 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f5e600b4d20 msgr2=0x7f5e600b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:05.206 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.202+0000 7f5e82c00640 1 --2- 172.21.15.16:0/4262842811 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f5e600b4d20 0x7f5e600b7230 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5e6800b6c0 tx=0x7f5e6800b410 comp rx=0 tx=0).stop 2024-08-31T06:16:05.206 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.202+0000 7f5e82c00640 1 -- 172.21.15.16:0/4262842811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e7c109b20 msgr2=0x7f5e7c19fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:05.206 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.202+0000 7f5e82c00640 1 --2- 172.21.15.16:0/4262842811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e7c109b20 0x7f5e7c19fa40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5e7400eb10 tx=0x7f5e7400c2d0 comp rx=0 tx=0).stop 2024-08-31T06:16:05.206 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.202+0000 7f5e82c00640 1 -- 172.21.15.16:0/4262842811 shutdown_connections 2024-08-31T06:16:05.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.202+0000 7f5e82c00640 1 --2- 172.21.15.16:0/4262842811 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f5e600b4d20 0x7f5e600b7230 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:05.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.202+0000 7f5e82c00640 1 --2- 172.21.15.16:0/4262842811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e7c109b20 0x7f5e7c19fa40 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:05.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.202+0000 7f5e82c00640 1 --2- 172.21.15.16:0/4262842811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e7c1091d0 0x7f5e7c19a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:05.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.202+0000 7f5e82c00640 1 --2- 172.21.15.16:0/4262842811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e7c1038a0 0x7f5e7c19a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:05.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.202+0000 7f5e82c00640 1 -- 172.21.15.16:0/4262842811 >> 172.21.15.16:0/4262842811 conn(0x7f5e7c0ff2a0 msgr2=0x7f5e7c1088b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:05.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.202+0000 7f5e82c00640 1 -- 172.21.15.16:0/4262842811 shutdown_connections 2024-08-31T06:16:05.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:05.202+0000 7f5e82c00640 1 -- 172.21.15.16:0/4262842811 wait complete. 2024-08-31T06:16:06.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 520.0 s, 298.8 tps, lat 342.729 ms stddev 196.369 2024-08-31T06:16:06.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:06 smithi184 ceph-mon[22331]: pgmap v636: 97 pgs: 97 active+clean; 3.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 6.1 MiB/s wr, 782 op/s 2024-08-31T06:16:06.998 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:06 smithi016 ceph-mon[17194]: pgmap v636: 97 pgs: 97 active+clean; 3.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 6.1 MiB/s wr, 782 op/s 2024-08-31T06:16:07.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:06 smithi155 ceph-mon[21742]: pgmap v636: 97 pgs: 97 active+clean; 3.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 6.1 MiB/s wr, 782 op/s 2024-08-31T06:16:09.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:08 smithi155 ceph-mon[21742]: pgmap v637: 97 pgs: 97 active+clean; 3.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 4.4 MiB/s wr, 612 op/s 2024-08-31T06:16:09.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:08 smithi016 ceph-mon[17194]: pgmap v637: 97 pgs: 97 active+clean; 3.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 4.4 MiB/s wr, 612 op/s 2024-08-31T06:16:09.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:08 smithi184 ceph-mon[22331]: pgmap v637: 97 pgs: 97 active+clean; 3.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 4.4 MiB/s wr, 612 op/s 2024-08-31T06:16:10.023 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:09 smithi155 ceph-mon[21742]: osdmap e88: 12 total, 12 up, 12 in 2024-08-31T06:16:10.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:09 smithi016 ceph-mon[17194]: osdmap e88: 12 total, 12 up, 12 in 2024-08-31T06:16:10.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:09 smithi184 ceph-mon[22331]: osdmap e88: 12 total, 12 up, 12 in 2024-08-31T06:16:11.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:10 smithi155 ceph-mon[21742]: pgmap v639: 97 pgs: 6 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 74 active+clean; 3.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 12 MiB/s wr, 1.06k op/s 2024-08-31T06:16:11.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:10 smithi016 ceph-mon[17194]: pgmap v639: 97 pgs: 6 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 74 active+clean; 3.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 12 MiB/s wr, 1.06k op/s 2024-08-31T06:16:11.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:10 smithi184 ceph-mon[22331]: pgmap v639: 97 pgs: 6 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 74 active+clean; 3.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 12 MiB/s wr, 1.06k op/s 2024-08-31T06:16:11.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 525.0 s, 692.0 tps, lat 147.411 ms stddev 300.586 2024-08-31T06:16:13.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:12 smithi155 ceph-mon[21742]: pgmap v640: 97 pgs: 6 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 74 active+clean; 3.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 18 MiB/s wr, 1.39k op/s 2024-08-31T06:16:13.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:12 smithi016 ceph-mon[17194]: pgmap v640: 97 pgs: 6 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 74 active+clean; 3.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 18 MiB/s wr, 1.39k op/s 2024-08-31T06:16:13.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:12 smithi184 ceph-mon[22331]: pgmap v640: 97 pgs: 6 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 74 active+clean; 3.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 18 MiB/s wr, 1.39k op/s 2024-08-31T06:16:14.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:13 smithi155 ceph-mon[21742]: pgmap v641: 97 pgs: 6 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 74 active+clean; 3.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 24 MiB/s wr, 1.71k op/s 2024-08-31T06:16:14.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:13 smithi016 ceph-mon[17194]: pgmap v641: 97 pgs: 6 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 74 active+clean; 3.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 24 MiB/s wr, 1.71k op/s 2024-08-31T06:16:14.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:13 smithi184 ceph-mon[22331]: pgmap v641: 97 pgs: 6 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 74 active+clean; 3.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 24 MiB/s wr, 1.71k op/s 2024-08-31T06:16:16.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 530.0 s, 1878.2 tps, lat 53.342 ms stddev 83.870 2024-08-31T06:16:17.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:16 smithi155 ceph-mon[21742]: pgmap v642: 97 pgs: 97 active+clean; 3.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 26 MiB/s wr, 1.87k op/s 2024-08-31T06:16:17.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:16 smithi155 ceph-mon[21742]: osdmap e89: 12 total, 12 up, 12 in 2024-08-31T06:16:17.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:16 smithi016 ceph-mon[17194]: pgmap v642: 97 pgs: 97 active+clean; 3.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 26 MiB/s wr, 1.87k op/s 2024-08-31T06:16:17.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:16 smithi016 ceph-mon[17194]: osdmap e89: 12 total, 12 up, 12 in 2024-08-31T06:16:17.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:16 smithi184 ceph-mon[22331]: pgmap v642: 97 pgs: 97 active+clean; 3.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 26 MiB/s wr, 1.87k op/s 2024-08-31T06:16:17.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:16 smithi184 ceph-mon[22331]: osdmap e89: 12 total, 12 up, 12 in 2024-08-31T06:16:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.492+0000 7f85f2400640 1 -- 172.21.15.16:0/2148041461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85ec10b3a0 msgr2=0x7f85ec10b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.492+0000 7f85f2400640 1 --2- 172.21.15.16:0/2148041461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85ec10b3a0 0x7f85ec10b7b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f85d8009900 tx=0x7f85d802a410 comp rx=0 tx=0).stop 2024-08-31T06:16:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.492+0000 7f85f2400640 1 -- 172.21.15.16:0/2148041461 shutdown_connections 2024-08-31T06:16:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.492+0000 7f85f2400640 1 --2- 172.21.15.16:0/2148041461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85ec10bcf0 0x7f85ec112520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.492+0000 7f85f2400640 1 --2- 172.21.15.16:0/2148041461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85ec10b3a0 0x7f85ec10b7b0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.492+0000 7f85f2400640 1 --2- 172.21.15.16:0/2148041461 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85ec105a70 0x7f85ec105e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.492+0000 7f85f2400640 1 -- 172.21.15.16:0/2148041461 >> 172.21.15.16:0/2148041461 conn(0x7f85ec101490 msgr2=0x7f85ec1038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:18.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.492+0000 7f85f2400640 1 -- 172.21.15.16:0/2148041461 shutdown_connections 2024-08-31T06:16:18.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.493+0000 7f85f2400640 1 -- 172.21.15.16:0/2148041461 wait complete. 2024-08-31T06:16:18.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.494+0000 7f85f2400640 1 Processor -- start 2024-08-31T06:16:18.496 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.494+0000 7f85f2400640 1 -- start start 2024-08-31T06:16:18.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.495+0000 7f85f2400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85ec105a70 0x7f85ec19c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:18.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.495+0000 7f85f2400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85ec10b3a0 0x7f85ec19c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:18.498 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.495+0000 7f85f2400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85ec10bcf0 0x7f85ec1a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:18.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.495+0000 7f85f2400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec115a00 con 0x7f85ec105a70 2024-08-31T06:16:18.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.495+0000 7f85f2400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec115880 con 0x7f85ec10bcf0 2024-08-31T06:16:18.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.495+0000 7f85f2400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec115b80 con 0x7f85ec10b3a0 2024-08-31T06:16:18.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.495+0000 7f85eb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85ec105a70 0x7f85ec19c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:18.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.495+0000 7f85ebe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85ec10bcf0 0x7f85ec1a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:18.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.495+0000 7f85eb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85ec105a70 0x7f85ec19c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58104/0 (socket says 172.21.15.16:58104) 2024-08-31T06:16:18.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.495+0000 7f85eb400640 1 -- 172.21.15.16:0/3636898780 learned_addr learned my addr 172.21.15.16:0/3636898780 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:18.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.495+0000 7f85eaa00640 1 --2- 172.21.15.16:0/3636898780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85ec10b3a0 0x7f85ec19c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:18.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.496+0000 7f85ebe00640 1 -- 172.21.15.16:0/3636898780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85ec10b3a0 msgr2=0x7f85ec19c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:18.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.496+0000 7f85ebe00640 1 --2- 172.21.15.16:0/3636898780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85ec10b3a0 0x7f85ec19c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:18.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.496+0000 7f85ebe00640 1 -- 172.21.15.16:0/3636898780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85ec105a70 msgr2=0x7f85ec19c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:18.499 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.496+0000 7f85ebe00640 1 --2- 172.21.15.16:0/3636898780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85ec105a70 0x7f85ec19c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:18.500 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.496+0000 7f85ebe00640 1 -- 172.21.15.16:0/3636898780 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85ec1a2350 con 0x7f85ec10bcf0 2024-08-31T06:16:18.500 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.496+0000 7f85ebe00640 1 --2- 172.21.15.16:0/3636898780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85ec10bcf0 0x7f85ec1a1c50 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f85e400da10 tx=0x7f85e400ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:18.500 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.496+0000 7f85e3e00640 1 -- 172.21.15.16:0/3636898780 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e400bd70 con 0x7f85ec10bcf0 2024-08-31T06:16:18.501 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.496+0000 7f85e3e00640 1 -- 172.21.15.16:0/3636898780 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85e4003e20 con 0x7f85ec10bcf0 2024-08-31T06:16:18.501 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.497+0000 7f85f2400640 1 -- 172.21.15.16:0/3636898780 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85ec1a2640 con 0x7f85ec10bcf0 2024-08-31T06:16:18.501 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.497+0000 7f85e3e00640 1 -- 172.21.15.16:0/3636898780 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e40106c0 con 0x7f85ec10bcf0 2024-08-31T06:16:18.501 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.497+0000 7f85f2400640 1 -- 172.21.15.16:0/3636898780 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85ec1a8dd0 con 0x7f85ec10bcf0 2024-08-31T06:16:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.502+0000 7f85f9602740 1 Processor -- start 2024-08-31T06:16:18.508 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.505+0000 7f85e3e00640 1 -- 172.21.15.16:0/3636898780 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f85e4010860 con 0x7f85ec10bcf0 2024-08-31T06:16:18.508 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.505+0000 7f85f9602740 1 -- start start 2024-08-31T06:16:18.508 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.505+0000 7f85f9602740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56312c0e81b0 0x56312c494130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:18.508 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.505+0000 7f85f9602740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c0bf2f0 0x56312c494670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:18.508 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.505+0000 7f85f9602740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56312c0c74d0 0x56312c494bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:18.508 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.505+0000 7f85f9602740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56312c400f70 con 0x56312c0bf2f0 2024-08-31T06:16:18.508 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.505+0000 7f85f9602740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56312c3f0500 con 0x56312c0c74d0 2024-08-31T06:16:18.508 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.505+0000 7f85f9602740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56312c3f44b0 con 0x56312c0e81b0 2024-08-31T06:16:18.508 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.505+0000 7f85eaa00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56312c0e81b0 0x56312c494130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.505+0000 7f85eaa00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56312c0e81b0 0x56312c494130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:33040/0 (socket says 172.21.15.16:33040) 2024-08-31T06:16:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.505+0000 7f85eaa00640 1 -- 172.21.15.16:0/1402825700 learned_addr learned my addr 172.21.15.16:0/1402825700 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.506+0000 7f85eaa00640 1 --2- 172.21.15.16:0/1402825700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56312c0c74d0 0x56312c494bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.506+0000 7f85eaa00640 1 -- 172.21.15.16:0/1402825700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56312c0c74d0 msgr2=0x56312c494bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.506+0000 7f85eaa00640 1 --2- 172.21.15.16:0/1402825700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56312c0c74d0 0x56312c494bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.506+0000 7f85eaa00640 1 -- 172.21.15.16:0/1402825700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c0bf2f0 msgr2=0x56312c494670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.506+0000 7f85eaa00640 1 --2- 172.21.15.16:0/1402825700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c0bf2f0 0x56312c494670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.506+0000 7f85eaa00640 1 -- 172.21.15.16:0/1402825700 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56312c3aac10 con 0x56312c0e81b0 2024-08-31T06:16:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.507+0000 7f85eaa00640 1 --2- 172.21.15.16:0/1402825700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56312c0e81b0 0x56312c494130 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f85ec06c7a0 tx=0x7f85d8000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2ef29749b5c6480 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.507+0000 7f85e3e00640 1 --2- 172.21.15.16:0/3636898780 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f85d00b4df0 0x7f85d00b7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.507+0000 7f85e0c00640 1 -- 172.21.15.16:0/1402825700 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d8035040 con 0x56312c0e81b0 2024-08-31T06:16:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.507+0000 7f85e0c00640 1 -- 172.21.15.16:0/1402825700 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85d80046c0 con 0x56312c0e81b0 2024-08-31T06:16:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.507+0000 7f85e0c00640 1 -- 172.21.15.16:0/1402825700 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d8003900 con 0x56312c0e81b0 2024-08-31T06:16:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.507+0000 7f85e3e00640 1 -- 172.21.15.16:0/3636898780 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8470+0+0 (secure 0 0 0) 0x7f85e40e5890 con 0x7f85ec10bcf0 2024-08-31T06:16:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.509+0000 7f85eb400640 1 --2- 172.21.15.16:0/3636898780 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f85d00b4df0 0x7f85d00b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.509+0000 7f85f9602740 1 -- 172.21.15.16:0/1402825700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56312c0e81b0 msgr2=0x56312c494130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.509+0000 7f85f9602740 1 --2- 172.21.15.16:0/1402825700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56312c0e81b0 0x56312c494130 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f85ec06c7a0 tx=0x7f85d8000bc0 comp rx=0 tx=0).stop 2024-08-31T06:16:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.509+0000 7f85f9602740 1 -- 172.21.15.16:0/1402825700 shutdown_connections 2024-08-31T06:16:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.509+0000 7f85f9602740 1 --2- 172.21.15.16:0/1402825700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56312c0c74d0 0x56312c494bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.509+0000 7f85f9602740 1 --2- 172.21.15.16:0/1402825700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c0bf2f0 0x56312c494670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.509+0000 7f85f9602740 1 --2- 172.21.15.16:0/1402825700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56312c0e81b0 0x56312c494130 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.509+0000 7f85f9602740 1 -- 172.21.15.16:0/1402825700 >> 172.21.15.16:0/1402825700 conn(0x56312c0dc300 msgr2=0x56312c492020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:18.513 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.511+0000 7f85f9602740 1 -- 172.21.15.16:0/1402825700 shutdown_connections 2024-08-31T06:16:18.513 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.511+0000 7f85eb400640 1 --2- 172.21.15.16:0/3636898780 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f85d00b4df0 0x7f85d00b7300 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f85dc009810 tx=0x7f85dc0097a0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:18.513 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.511+0000 7f85f9602740 1 -- 172.21.15.16:0/1402825700 wait complete. 2024-08-31T06:16:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.512+0000 7f85f9602740 1 -- start start 2024-08-31T06:16:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.512+0000 7f85f9602740 1 Processor -- start 2024-08-31T06:16:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.513+0000 7f85f9602740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c0bf2f0 0x56312c49f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:18.516 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.513+0000 7f85f9602740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56312c0c74d0 0x56312c49fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:18.516 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.513+0000 7f85f9602740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56312c0e81b0 0x56312c4a0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:18.516 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.513+0000 7f85f9602740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56312c329ab0 con 0x56312c0bf2f0 2024-08-31T06:16:18.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.513+0000 7f85f9602740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56312c397320 con 0x56312c0e81b0 2024-08-31T06:16:18.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.513+0000 7f85f9602740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56312c4a2850 con 0x56312c0c74d0 2024-08-31T06:16:18.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.514+0000 7f85ebe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56312c0e81b0 0x56312c4a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:18.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.514+0000 7f85f9602740 10 client.0 _ll_register_callbacks cb 0x56312c39d6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:16:18.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.514+0000 7f85eb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c0bf2f0 0x56312c49f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:18.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.514+0000 7f85eb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c0bf2f0 0x56312c49f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58108/0 (socket says 172.21.15.16:58108) 2024-08-31T06:16:18.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.514+0000 7f85eb400640 1 -- 172.21.15.16:0/4223345726 learned_addr learned my addr 172.21.15.16:0/4223345726 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:18.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.514+0000 7f85eaa00640 1 --2- 172.21.15.16:0/4223345726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56312c0c74d0 0x56312c49fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:18.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.514+0000 7f85eaa00640 1 -- 172.21.15.16:0/4223345726 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56312c0e81b0 msgr2=0x56312c4a0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:18.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.514+0000 7f85eaa00640 1 --2- 172.21.15.16:0/4223345726 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56312c0e81b0 0x56312c4a0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:18.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.515+0000 7f85eaa00640 1 -- 172.21.15.16:0/4223345726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c0bf2f0 msgr2=0x56312c49f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:18.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.515+0000 7f85eaa00640 1 --2- 172.21.15.16:0/4223345726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c0bf2f0 0x56312c49f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:18.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.515+0000 7f85eaa00640 1 -- 172.21.15.16:0/4223345726 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56312c153790 con 0x56312c0c74d0 2024-08-31T06:16:18.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.515+0000 7f85eaa00640 1 --2- 172.21.15.16:0/4223345726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56312c0c74d0 0x56312c49fc10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f85d80096c0 tx=0x7f85d80044a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa379ecc4a5a5f69 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:18.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.515+0000 7f85d5600640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:16:18.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.515+0000 7f85d5600640 1 -- 172.21.15.16:0/4223345726 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d804d070 con 0x56312c0c74d0 2024-08-31T06:16:18.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.516+0000 7f85d5600640 1 -- 172.21.15.16:0/4223345726 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8470+0+0 (secure 0 0 0) 0x7f85d8004870 con 0x56312c0c74d0 2024-08-31T06:16:18.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.516+0000 7f85f9602740 1 -- 172.21.15.16:0/4223345726 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85d80046c0 con 0x56312c0c74d0 2024-08-31T06:16:18.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.516+0000 7f85d5600640 1 -- 172.21.15.16:0/4223345726 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f85d8059050 con 0x56312c0c74d0 2024-08-31T06:16:18.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.516+0000 7f85f9602740 10 client.27619 fetch_fsmap learned FSMap version 43 2024-08-31T06:16:18.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.516+0000 7f85f9602740 1 -- 172.21.15.16:0/4223345726 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56312c1cc620 con 0x56312c0c74d0 2024-08-31T06:16:18.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.516+0000 7f85d5600640 1 -- 172.21.15.16:0/4223345726 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85d80046c0 con 0x56312c0c74d0 2024-08-31T06:16:18.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.516+0000 7f85d5600640 1 -- 172.21.15.16:0/4223345726 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f85d803a040 con 0x56312c0c74d0 2024-08-31T06:16:18.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.516+0000 7f85f9602740 10 client.27619 fetch_fsmap finished waiting for FSMap version 43 2024-08-31T06:16:18.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.516+0000 7f85f9602740 10 client.27619 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:16:18.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.516+0000 7f85f9602740 20 client.27619 populate_metadata read hostname 'smithi016' 2024-08-31T06:16:18.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.516+0000 7f85f9602740 1 --2- 172.21.15.16:0/4223345726 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56312c0a6d80 0x56312c4b1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:18.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.516+0000 7f85f9602740 4 client.27619 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:16:18.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.516+0000 7f85f9602740 1 -- 172.21.15.16:0/4223345726 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56312c402ec0 con 0x56312c0a6d80 2024-08-31T06:16:18.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.517+0000 7f85eb400640 1 --2- 172.21.15.16:0/4223345726 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56312c0a6d80 0x56312c4b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:18.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.517+0000 7f85d5600640 10 client.27619 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:16:18.522 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.517+0000 7f85eb400640 1 --2- 172.21.15.16:0/4223345726 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56312c0a6d80 0x56312c4b1640 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d7f1e4a933f8297 server_cookie=f74f6cac21953d88 in_seq=0 out_seq=0 2024-08-31T06:16:18.522 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.518+0000 7f85d5600640 1 -- 172.21.15.16:0/4223345726 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56312c402ec0 con 0x56312c0a6d80 2024-08-31T06:16:18.522 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.518+0000 7f85d5600640 10 client.27619 handle_command_reply: tid=0 2024-08-31T06:16:18.522 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.518+0000 7f85f9602740 1 client.27619 shutdown 2024-08-31T06:16:18.522 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.519+0000 7f85f9602740 1 -- 172.21.15.16:0/4223345726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56312c0c74d0 msgr2=0x56312c49fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:18.522 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.519+0000 7f85f9602740 1 --2- 172.21.15.16:0/4223345726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56312c0c74d0 0x56312c49fc10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f85d80096c0 tx=0x7f85d80044a0 comp rx=0 tx=0).stop 2024-08-31T06:16:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.520+0000 7f85f9602740 1 -- 172.21.15.16:0/4223345726 shutdown_connections 2024-08-31T06:16:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.520+0000 7f85f9602740 1 --2- 172.21.15.16:0/4223345726 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56312c0a6d80 0x56312c4b1640 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.520+0000 7f85f9602740 1 --2- 172.21.15.16:0/4223345726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56312c0c74d0 0x56312c49fc10 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.520+0000 7f85f9602740 1 --2- 172.21.15.16:0/4223345726 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56312c0e81b0 0x56312c4a0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.520+0000 7f85f9602740 1 --2- 172.21.15.16:0/4223345726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c0bf2f0 0x56312c49f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.520+0000 7f85f9602740 1 -- 172.21.15.16:0/4223345726 >> 172.21.15.16:0/4223345726 conn(0x56312c0dc300 msgr2=0x56312c495160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.520+0000 7f85d5600640 0 client.27619 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:16:18.524 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.521+0000 7f85f9602740 1 -- 172.21.15.16:0/4223345726 shutdown_connections 2024-08-31T06:16:18.524 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.521+0000 7f85f9602740 1 -- 172.21.15.16:0/4223345726 wait complete. 2024-08-31T06:16:18.524 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.521+0000 7f85f9602740 20 client.27619 trim_cache size 0 max 16384 2024-08-31T06:16:18.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.548+0000 7f85f9602740 1 Processor -- start 2024-08-31T06:16:18.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.548+0000 7f85f9602740 1 -- start start 2024-08-31T06:16:18.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.549+0000 7f85f9602740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c0bf2f0 0x56312c495b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:18.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.549+0000 7f85f9602740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56312c0c74d0 0x56312c4ab210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.549+0000 7f85f9602740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56312c0e81b0 0x56312c49c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.549+0000 7f85f9602740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec1afa30 con 0x56312c0bf2f0 2024-08-31T06:16:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.549+0000 7f85f9602740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec1ad2a0 con 0x56312c0c74d0 2024-08-31T06:16:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.549+0000 7f85f9602740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec1ab110 con 0x56312c0e81b0 2024-08-31T06:16:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.549+0000 7f85ebe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56312c0e81b0 0x56312c49c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.549+0000 7f85ebe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56312c0e81b0 0x56312c49c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:33060/0 (socket says 172.21.15.16:33060) 2024-08-31T06:16:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.549+0000 7f85ebe00640 1 -- 172.21.15.16:0/2175025632 learned_addr learned my addr 172.21.15.16:0/2175025632 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.550+0000 7f85ebe00640 1 -- 172.21.15.16:0/2175025632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56312c0c74d0 msgr2=0x56312c4ab210 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T06:16:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.550+0000 7f85eaa00640 1 --2- 172.21.15.16:0/2175025632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56312c0c74d0 0x56312c4ab210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.550+0000 7f85eb400640 1 --2- 172.21.15.16:0/2175025632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c0bf2f0 0x56312c495b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.551+0000 7f85ebe00640 1 --2- 172.21.15.16:0/2175025632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56312c0c74d0 0x56312c4ab210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.551+0000 7f85ebe00640 1 -- 172.21.15.16:0/2175025632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c0bf2f0 msgr2=0x56312c495b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.551+0000 7f85ebe00640 1 --2- 172.21.15.16:0/2175025632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c0bf2f0 0x56312c495b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.551+0000 7f85ebe00640 1 -- 172.21.15.16:0/2175025632 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56312c4a2210 con 0x56312c0e81b0 2024-08-31T06:16:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.551+0000 7f85eaa00640 1 --2- 172.21.15.16:0/2175025632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56312c0c74d0 0x56312c4ab210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:16:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.551+0000 7f85eb400640 1 --2- 172.21.15.16:0/2175025632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c0bf2f0 0x56312c495b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:16:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.554+0000 7f85ebe00640 1 --2- 172.21.15.16:0/2175025632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56312c0e81b0 0x56312c49c8a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f85e40a9220 tx=0x7f85e40ab810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38fdd3ca5cc095cb server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:18.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.556+0000 7f85d7e00640 1 -- 172.21.15.16:0/2175025632 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e40b0820 con 0x56312c0e81b0 2024-08-31T06:16:18.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.557+0000 7f85d7e00640 1 -- 172.21.15.16:0/2175025632 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85e40b52d0 con 0x56312c0e81b0 2024-08-31T06:16:18.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.557+0000 7f85d7e00640 1 -- 172.21.15.16:0/2175025632 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e40b50a0 con 0x56312c0e81b0 2024-08-31T06:16:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.559+0000 7f85f9602740 1 -- 172.21.15.16:0/2175025632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56312c0e81b0 msgr2=0x56312c49c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.559+0000 7f85f9602740 1 --2- 172.21.15.16:0/2175025632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56312c0e81b0 0x56312c49c8a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f85e40a9220 tx=0x7f85e40ab810 comp rx=0 tx=0).stop 2024-08-31T06:16:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.559+0000 7f85f9602740 1 -- 172.21.15.16:0/2175025632 shutdown_connections 2024-08-31T06:16:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.559+0000 7f85f9602740 1 --2- 172.21.15.16:0/2175025632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56312c0e81b0 0x56312c49c8a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.559+0000 7f85f9602740 1 --2- 172.21.15.16:0/2175025632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56312c0c74d0 0x56312c4ab210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.559+0000 7f85f9602740 1 --2- 172.21.15.16:0/2175025632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c0bf2f0 0x56312c495b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.559+0000 7f85f9602740 1 -- 172.21.15.16:0/2175025632 >> 172.21.15.16:0/2175025632 conn(0x56312c0dc300 msgr2=0x56312c4aab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.560+0000 7f85f9602740 1 -- 172.21.15.16:0/2175025632 shutdown_connections 2024-08-31T06:16:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.560+0000 7f85f9602740 1 -- 172.21.15.16:0/2175025632 wait complete. 2024-08-31T06:16:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.561+0000 7f85f9602740 1 -- start start 2024-08-31T06:16:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.562+0000 7f85f9602740 1 Processor -- start 2024-08-31T06:16:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.562+0000 7f85f9602740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c0bf2f0 0x56312c50fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.562+0000 7f85f9602740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56312c0c74d0 0x56312c510290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.562+0000 7f85f9602740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56312c0e81b0 0x56312c5107d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.562+0000 7f85f9602740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec1ab760 con 0x56312c0bf2f0 2024-08-31T06:16:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.562+0000 7f85f9602740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec1acbd0 con 0x56312c0c74d0 2024-08-31T06:16:18.566 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.562+0000 7f85f9602740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec1abe30 con 0x56312c0e81b0 2024-08-31T06:16:18.566 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.562+0000 7f85f9602740 10 client.0 _ll_register_callbacks cb 0x7f85d8004340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:16:18.566 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.563+0000 7f85ebe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56312c0e81b0 0x56312c5107d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.563+0000 7f85eb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c0bf2f0 0x56312c50fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.563+0000 7f85ebe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56312c0e81b0 0x56312c5107d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:33074/0 (socket says 172.21.15.16:33074) 2024-08-31T06:16:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.563+0000 7f85ebe00640 1 -- 172.21.15.16:0/1871191117 learned_addr learned my addr 172.21.15.16:0/1871191117 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.563+0000 7f85eaa00640 1 --2- 172.21.15.16:0/1871191117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56312c0c74d0 0x56312c510290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.564+0000 7f85eb400640 1 -- 172.21.15.16:0/1871191117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56312c0e81b0 msgr2=0x56312c5107d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.564+0000 7f85eb400640 1 --2- 172.21.15.16:0/1871191117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56312c0e81b0 0x56312c5107d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.564+0000 7f85eb400640 1 -- 172.21.15.16:0/1871191117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56312c0c74d0 msgr2=0x56312c510290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.564+0000 7f85eb400640 1 --2- 172.21.15.16:0/1871191117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56312c0c74d0 0x56312c510290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.564+0000 7f85eb400640 1 -- 172.21.15.16:0/1871191117 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56312c51af30 con 0x56312c0bf2f0 2024-08-31T06:16:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.570+0000 7f85eb400640 1 --2- 172.21.15.16:0/1871191117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c0bf2f0 0x56312c50fd50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x56312c4a0740 tx=0x7f85dc005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95658fbe27ef813 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.570+0000 7f85d5600640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:16:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.571+0000 7f85d5600640 1 -- 172.21.15.16:0/1871191117 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85dc004c20 con 0x56312c0bf2f0 2024-08-31T06:16:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.571+0000 7f85d5600640 1 -- 172.21.15.16:0/1871191117 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8470+0+0 (secure 0 0 0) 0x7f85dc004dc0 con 0x56312c0bf2f0 2024-08-31T06:16:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.571+0000 7f85f9602740 1 -- 172.21.15.16:0/1871191117 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85e40b52d0 con 0x56312c0bf2f0 2024-08-31T06:16:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.572+0000 7f85d5600640 1 -- 172.21.15.16:0/1871191117 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f85dc035050 con 0x56312c0bf2f0 2024-08-31T06:16:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.574+0000 7f85f9602740 10 client.17361 fetch_fsmap learned FSMap version 43 2024-08-31T06:16:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.574+0000 7f85f9602740 1 -- 172.21.15.16:0/1871191117 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56312c51c0e0 con 0x56312c0bf2f0 2024-08-31T06:16:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.575+0000 7f85d5600640 1 -- 172.21.15.16:0/1871191117 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85e40b52d0 con 0x56312c0bf2f0 2024-08-31T06:16:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.575+0000 7f85d5600640 1 -- 172.21.15.16:0/1871191117 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f85dc028970 con 0x56312c0bf2f0 2024-08-31T06:16:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.575+0000 7f85f9602740 10 client.17361 fetch_fsmap finished waiting for FSMap version 43 2024-08-31T06:16:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.575+0000 7f85f9602740 10 client.17361 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:16:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.575+0000 7f85f9602740 20 client.17361 populate_metadata read hostname 'smithi016' 2024-08-31T06:16:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.575+0000 7f85f9602740 1 --2- 172.21.15.16:0/1871191117 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56312c07c1c0 0x56312c51e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.575+0000 7f85f9602740 4 client.17361 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:16:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.575+0000 7f85f9602740 1 -- 172.21.15.16:0/1871191117 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56312c3fe960 con 0x56312c07c1c0 2024-08-31T06:16:18.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.576+0000 7f85eaa00640 1 --2- 172.21.15.16:0/1871191117 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56312c07c1c0 0x56312c51e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:18.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.578+0000 7f85eaa00640 1 --2- 172.21.15.16:0/1871191117 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56312c07c1c0 0x56312c51e870 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=687917142a07b8 server_cookie=bd1b5779a7ad866f in_seq=0 out_seq=0 2024-08-31T06:16:18.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.578+0000 7f85d5600640 10 client.17361 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:16:18.603 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.601+0000 7f85d5600640 1 -- 172.21.15.16:0/1871191117 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56312c3fe960 con 0x56312c07c1c0 2024-08-31T06:16:18.604 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.601+0000 7f85d5600640 10 client.17361 handle_command_reply: tid=0 2024-08-31T06:16:18.604 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.601+0000 7f85f9602740 1 client.17361 shutdown 2024-08-31T06:16:18.605 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.602+0000 7f85f9602740 1 -- 172.21.15.16:0/1871191117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c0bf2f0 msgr2=0x56312c50fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:18.605 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.602+0000 7f85f9602740 1 --2- 172.21.15.16:0/1871191117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c0bf2f0 0x56312c50fd50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x56312c4a0740 tx=0x7f85dc005f90 comp rx=0 tx=0).stop 2024-08-31T06:16:18.605 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.602+0000 7f85f9602740 1 -- 172.21.15.16:0/1871191117 shutdown_connections 2024-08-31T06:16:18.605 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.602+0000 7f85f9602740 1 --2- 172.21.15.16:0/1871191117 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56312c07c1c0 0x56312c51e870 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:18.605 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.602+0000 7f85f9602740 1 --2- 172.21.15.16:0/1871191117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56312c0e81b0 0x56312c5107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:18.605 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.602+0000 7f85f9602740 1 --2- 172.21.15.16:0/1871191117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56312c0c74d0 0x56312c510290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:18.605 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.602+0000 7f85f9602740 1 --2- 172.21.15.16:0/1871191117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c0bf2f0 0x56312c50fd50 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:18.605 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.602+0000 7f85d5600640 0 client.17361 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:16:18.605 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.602+0000 7f85f9602740 1 -- 172.21.15.16:0/1871191117 >> 172.21.15.16:0/1871191117 conn(0x56312c0dc300 msgr2=0x56312c4958d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:18.605 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.603+0000 7f85f9602740 1 -- 172.21.15.16:0/1871191117 shutdown_connections 2024-08-31T06:16:18.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.603+0000 7f85f9602740 1 -- 172.21.15.16:0/1871191117 wait complete. 2024-08-31T06:16:18.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.603+0000 7f85f9602740 20 client.17361 trim_cache size 0 max 16384 2024-08-31T06:16:18.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:16:18.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:16:18.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:16:18.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:16:18.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.604+0000 7f85f2400640 1 -- 172.21.15.16:0/3636898780 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f85d00b4df0 msgr2=0x7f85d00b7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:18.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.604+0000 7f85f2400640 1 --2- 172.21.15.16:0/3636898780 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f85d00b4df0 0x7f85d00b7300 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f85dc009810 tx=0x7f85dc0097a0 comp rx=0 tx=0).stop 2024-08-31T06:16:18.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.604+0000 7f85f2400640 1 -- 172.21.15.16:0/3636898780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85ec10bcf0 msgr2=0x7f85ec1a1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:18.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.604+0000 7f85f2400640 1 --2- 172.21.15.16:0/3636898780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85ec10bcf0 0x7f85ec1a1c50 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f85e400da10 tx=0x7f85e400ded0 comp rx=0 tx=0).stop 2024-08-31T06:16:18.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.605+0000 7f85f2400640 1 -- 172.21.15.16:0/3636898780 shutdown_connections 2024-08-31T06:16:18.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.605+0000 7f85f2400640 1 --2- 172.21.15.16:0/3636898780 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f85d00b4df0 0x7f85d00b7300 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:18.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.605+0000 7f85f2400640 1 --2- 172.21.15.16:0/3636898780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85ec10b3a0 0x7f85ec19c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:18.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.605+0000 7f85f2400640 1 --2- 172.21.15.16:0/3636898780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85ec10bcf0 0x7f85ec1a1c50 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:18.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.605+0000 7f85f2400640 1 --2- 172.21.15.16:0/3636898780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85ec105a70 0x7f85ec19c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:18.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.605+0000 7f85f2400640 1 -- 172.21.15.16:0/3636898780 >> 172.21.15.16:0/3636898780 conn(0x7f85ec101490 msgr2=0x7f85ec1038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:18.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.605+0000 7f85f2400640 1 -- 172.21.15.16:0/3636898780 shutdown_connections 2024-08-31T06:16:18.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:18.606+0000 7f85f2400640 1 -- 172.21.15.16:0/3636898780 wait complete. 2024-08-31T06:16:19.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:18 smithi155 ceph-mon[21742]: pgmap v644: 97 pgs: 97 active+clean; 3.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 30 MiB/s wr, 2.13k op/s 2024-08-31T06:16:19.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:18 smithi016 ceph-mon[17194]: pgmap v644: 97 pgs: 97 active+clean; 3.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 30 MiB/s wr, 2.13k op/s 2024-08-31T06:16:19.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:18 smithi184 ceph-mon[22331]: pgmap v644: 97 pgs: 97 active+clean; 3.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 30 MiB/s wr, 2.13k op/s 2024-08-31T06:16:21.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:20 smithi155 ceph-mon[21742]: pgmap v645: 97 pgs: 97 active+clean; 3.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 19 MiB/s wr, 1.46k op/s 2024-08-31T06:16:21.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:20 smithi016 ceph-mon[17194]: pgmap v645: 97 pgs: 97 active+clean; 3.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 19 MiB/s wr, 1.46k op/s 2024-08-31T06:16:21.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:20 smithi184 ceph-mon[22331]: pgmap v645: 97 pgs: 97 active+clean; 3.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 19 MiB/s wr, 1.46k op/s 2024-08-31T06:16:21.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 535.0 s, 2178.4 tps, lat 46.104 ms stddev 57.144 2024-08-31T06:16:23.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:22 smithi155 ceph-mon[21742]: pgmap v646: 97 pgs: 97 active+clean; 3.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 1.10k op/s 2024-08-31T06:16:23.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:22 smithi016 ceph-mon[17194]: pgmap v646: 97 pgs: 97 active+clean; 3.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 1.10k op/s 2024-08-31T06:16:23.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:22 smithi184 ceph-mon[22331]: pgmap v646: 97 pgs: 97 active+clean; 3.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 1.10k op/s 2024-08-31T06:16:25.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:24 smithi155 ceph-mon[21742]: pgmap v647: 97 pgs: 97 active+clean; 3.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.2 MiB/s wr, 888 op/s 2024-08-31T06:16:25.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:24 smithi016 ceph-mon[17194]: pgmap v647: 97 pgs: 97 active+clean; 3.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.2 MiB/s wr, 888 op/s 2024-08-31T06:16:25.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:24 smithi184 ceph-mon[22331]: pgmap v647: 97 pgs: 97 active+clean; 3.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.2 MiB/s wr, 888 op/s 2024-08-31T06:16:26.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 540.0 s, 1396.4 tps, lat 70.993 ms stddev 105.112 2024-08-31T06:16:27.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:26 smithi155 ceph-mon[21742]: pgmap v648: 97 pgs: 97 active+clean; 3.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 1.52k op/s 2024-08-31T06:16:27.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:26 smithi016 ceph-mon[17194]: pgmap v648: 97 pgs: 97 active+clean; 3.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 1.52k op/s 2024-08-31T06:16:27.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:26 smithi184 ceph-mon[22331]: pgmap v648: 97 pgs: 97 active+clean; 3.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 1.52k op/s 2024-08-31T06:16:29.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:28 smithi155 ceph-mon[21742]: pgmap v649: 97 pgs: 97 active+clean; 3.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 1.55k op/s 2024-08-31T06:16:29.165 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:28 smithi016 ceph-mon[17194]: pgmap v649: 97 pgs: 97 active+clean; 3.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 1.55k op/s 2024-08-31T06:16:29.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:28 smithi184 ceph-mon[22331]: pgmap v649: 97 pgs: 97 active+clean; 3.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 1.55k op/s 2024-08-31T06:16:31.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:30 smithi155 ceph-mon[21742]: pgmap v650: 97 pgs: 97 active+clean; 3.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 1.53k op/s 2024-08-31T06:16:31.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:30 smithi016 ceph-mon[17194]: pgmap v650: 97 pgs: 97 active+clean; 3.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 1.53k op/s 2024-08-31T06:16:31.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:30 smithi184 ceph-mon[22331]: pgmap v650: 97 pgs: 97 active+clean; 3.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 1.53k op/s 2024-08-31T06:16:31.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 545.0 s, 1439.2 tps, lat 69.034 ms stddev 86.206 2024-08-31T06:16:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.868+0000 7f1744c00640 1 -- 172.21.15.16:0/846217988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17401036b0 msgr2=0x7f1740103ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.868+0000 7f1744c00640 1 --2- 172.21.15.16:0/846217988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17401036b0 0x7f1740103ac0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1730009900 tx=0x7f173002a3f0 comp rx=0 tx=0).stop 2024-08-31T06:16:31.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.869+0000 7f1744c00640 1 -- 172.21.15.16:0/846217988 shutdown_connections 2024-08-31T06:16:31.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.869+0000 7f1744c00640 1 --2- 172.21.15.16:0/846217988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1740109930 0x7f1740110160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:31.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.869+0000 7f1744c00640 1 --2- 172.21.15.16:0/846217988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1740108fe0 0x7f17401093f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:31.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.869+0000 7f1744c00640 1 --2- 172.21.15.16:0/846217988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17401036b0 0x7f1740103ac0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:31.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.869+0000 7f1744c00640 1 -- 172.21.15.16:0/846217988 >> 172.21.15.16:0/846217988 conn(0x7f17400ff0d0 msgr2=0x7f1740101520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:31.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.869+0000 7f1744c00640 1 -- 172.21.15.16:0/846217988 shutdown_connections 2024-08-31T06:16:31.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.869+0000 7f1744c00640 1 -- 172.21.15.16:0/846217988 wait complete. 2024-08-31T06:16:31.873 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.871+0000 7f1744c00640 1 Processor -- start 2024-08-31T06:16:31.873 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.871+0000 7f1744c00640 1 -- start start 2024-08-31T06:16:31.874 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.871+0000 7f1744c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17401036b0 0x7f174019a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:31.874 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.871+0000 7f1744c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1740108fe0 0x7f174019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:31.874 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.871+0000 7f1744c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1740109930 0x7f174019f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:31.874 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.871+0000 7f1744c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1740113630 con 0x7f17401036b0 2024-08-31T06:16:31.874 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.871+0000 7f1744c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f17401134b0 con 0x7f1740108fe0 2024-08-31T06:16:31.874 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.871+0000 7f1744c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f17401137b0 con 0x7f1740109930 2024-08-31T06:16:31.874 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.872+0000 7f173ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1740108fe0 0x7f174019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:31.875 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.872+0000 7f173ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1740108fe0 0x7f174019a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:36914/0 (socket says 172.21.15.16:36914) 2024-08-31T06:16:31.875 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.872+0000 7f173fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1740109930 0x7f174019f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:31.875 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.872+0000 7f173ea00640 1 -- 172.21.15.16:0/1389644343 learned_addr learned my addr 172.21.15.16:0/1389644343 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:31.875 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.872+0000 7f173f400640 1 --2- 172.21.15.16:0/1389644343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17401036b0 0x7f174019a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:31.875 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.872+0000 7f173ea00640 1 -- 172.21.15.16:0/1389644343 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1740109930 msgr2=0x7f174019f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:31.875 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.872+0000 7f173ea00640 1 --2- 172.21.15.16:0/1389644343 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1740109930 0x7f174019f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:31.875 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.872+0000 7f173ea00640 1 -- 172.21.15.16:0/1389644343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17401036b0 msgr2=0x7f174019a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:31.876 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.872+0000 7f173ea00640 1 --2- 172.21.15.16:0/1389644343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17401036b0 0x7f174019a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:31.876 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.872+0000 7f173ea00640 1 -- 172.21.15.16:0/1389644343 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f174019ff80 con 0x7f1740108fe0 2024-08-31T06:16:31.876 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.872+0000 7f173f400640 1 --2- 172.21.15.16:0/1389644343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17401036b0 0x7f174019a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:16:31.876 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.873+0000 7f173ea00640 1 --2- 172.21.15.16:0/1389644343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1740108fe0 0x7f174019a5f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f172c00e970 tx=0x7f172c00ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:31.876 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.873+0000 7f1737e00640 1 -- 172.21.15.16:0/1389644343 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c00cd80 con 0x7f1740108fe0 2024-08-31T06:16:31.876 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.873+0000 7f1737e00640 1 -- 172.21.15.16:0/1389644343 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f172c0035f0 con 0x7f1740108fe0 2024-08-31T06:16:31.876 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.873+0000 7f1737e00640 1 -- 172.21.15.16:0/1389644343 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c010690 con 0x7f1740108fe0 2024-08-31T06:16:31.876 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.873+0000 7f1744c00640 1 -- 172.21.15.16:0/1389644343 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17401a0270 con 0x7f1740108fe0 2024-08-31T06:16:31.877 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.873+0000 7f1744c00640 1 -- 172.21.15.16:0/1389644343 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1740075390 con 0x7f1740108fe0 2024-08-31T06:16:31.879 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.877+0000 7f174d2b6740 1 Processor -- start 2024-08-31T06:16:31.879 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.877+0000 7f174d2b6740 1 -- start start 2024-08-31T06:16:31.879 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.877+0000 7f1737e00640 1 -- 172.21.15.16:0/1389644343 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f172c003aa0 con 0x7f1740108fe0 2024-08-31T06:16:31.880 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.877+0000 7f174d2b6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db6cfb01b0 0x55db6d3538a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:31.881 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.877+0000 7f174d2b6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6cf872f0 0x55db6d353de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:31.881 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.877+0000 7f174d2b6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db6cf8f4d0 0x55db6d354320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:31.881 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.877+0000 7f174d2b6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55db6d2c8f70 con 0x55db6cfb01b0 2024-08-31T06:16:31.881 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.877+0000 7f174d2b6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55db6d2b8500 con 0x55db6cf8f4d0 2024-08-31T06:16:31.881 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.877+0000 7f174d2b6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55db6d2bc4b0 con 0x55db6cf872f0 2024-08-31T06:16:31.882 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.878+0000 7f173f400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6cf872f0 0x55db6d353de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:31.882 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.878+0000 7f173f400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6cf872f0 0x55db6d353de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:43960/0 (socket says 172.21.15.16:43960) 2024-08-31T06:16:31.882 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.878+0000 7f173f400640 1 -- 172.21.15.16:0/1380253470 learned_addr learned my addr 172.21.15.16:0/1380253470 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:31.882 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.878+0000 7f173ea00640 1 --2- 172.21.15.16:0/1380253470 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db6cf8f4d0 0x55db6d354320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:31.883 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.878+0000 7f173f400640 1 -- 172.21.15.16:0/1380253470 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db6cf8f4d0 msgr2=0x55db6d354320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:31.883 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.878+0000 7f173f400640 1 --2- 172.21.15.16:0/1380253470 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db6cf8f4d0 0x55db6d354320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:31.883 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.878+0000 7f173fe00640 1 --2- 172.21.15.16:0/1380253470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db6cfb01b0 0x55db6d3538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:31.883 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.879+0000 7f173f400640 1 -- 172.21.15.16:0/1380253470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db6cfb01b0 msgr2=0x55db6d3538a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:31.883 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.879+0000 7f173f400640 1 --2- 172.21.15.16:0/1380253470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db6cfb01b0 0x55db6d3538a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:31.883 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.879+0000 7f173f400640 1 -- 172.21.15.16:0/1380253470 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db6d272c10 con 0x55db6cf872f0 2024-08-31T06:16:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.879+0000 7f173fe00640 1 --2- 172.21.15.16:0/1380253470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db6cfb01b0 0x55db6d3538a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:16:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.880+0000 7f173f400640 1 --2- 172.21.15.16:0/1380253470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6cf872f0 0x55db6d353de0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1730009900 tx=0x7f1730004520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75963faa600467a5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.881+0000 7f1737e00640 1 --2- 172.21.15.16:0/1389644343 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f17240b4d20 0x7f17240b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.881+0000 7f172be00640 1 -- 172.21.15.16:0/1380253470 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1730035040 con 0x55db6cf872f0 2024-08-31T06:16:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.881+0000 7f172be00640 1 -- 172.21.15.16:0/1380253470 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1730005400 con 0x55db6cf872f0 2024-08-31T06:16:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.881+0000 7f1737e00640 1 -- 172.21.15.16:0/1389644343 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8470+0+0 (secure 0 0 0) 0x7f172c0e57a0 con 0x7f1740108fe0 2024-08-31T06:16:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.882+0000 7f173fe00640 1 --2- 172.21.15.16:0/1389644343 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f17240b4d20 0x7f17240b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:31.886 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.882+0000 7f174d2b6740 1 -- 172.21.15.16:0/1380253470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6cf872f0 msgr2=0x55db6d353de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:31.886 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.883+0000 7f172be00640 1 -- 172.21.15.16:0/1380253470 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1730035040 con 0x55db6cf872f0 2024-08-31T06:16:31.886 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.883+0000 7f174d2b6740 1 --2- 172.21.15.16:0/1380253470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6cf872f0 0x55db6d353de0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1730009900 tx=0x7f1730004520 comp rx=0 tx=0).stop 2024-08-31T06:16:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.883+0000 7f173fe00640 1 --2- 172.21.15.16:0/1389644343 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f17240b4d20 0x7f17240b7230 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1738005fd0 tx=0x7f1738009600 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.883+0000 7f174d2b6740 1 -- 172.21.15.16:0/1380253470 shutdown_connections 2024-08-31T06:16:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.883+0000 7f174d2b6740 1 --2- 172.21.15.16:0/1380253470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6cf872f0 0x55db6d353de0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.883+0000 7f174d2b6740 1 --2- 172.21.15.16:0/1380253470 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db6cf8f4d0 0x55db6d354320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.883+0000 7f174d2b6740 1 --2- 172.21.15.16:0/1380253470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db6cfb01b0 0x55db6d3538a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.883+0000 7f174d2b6740 1 -- 172.21.15.16:0/1380253470 >> 172.21.15.16:0/1380253470 conn(0x55db6cfa4300 msgr2=0x55db6d351790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.883+0000 7f174d2b6740 1 -- 172.21.15.16:0/1380253470 shutdown_connections 2024-08-31T06:16:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.883+0000 7f174d2b6740 1 -- 172.21.15.16:0/1380253470 wait complete. 2024-08-31T06:16:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.884+0000 7f174d2b6740 1 -- start start 2024-08-31T06:16:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.884+0000 7f174d2b6740 1 Processor -- start 2024-08-31T06:16:31.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.885+0000 7f174d2b6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db6cf872f0 0x55db6d35ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:31.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.885+0000 7f174d2b6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6cf8f4d0 0x55db6d35f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:31.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.885+0000 7f174d2b6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db6cfb01b0 0x55db6d35f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:31.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.885+0000 7f173f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db6cf872f0 0x55db6d35ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:31.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.885+0000 7f173f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db6cf872f0 0x55db6d35ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57998/0 (socket says 172.21.15.16:57998) 2024-08-31T06:16:31.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.885+0000 7f173f400640 1 -- 172.21.15.16:0/1333427096 learned_addr learned my addr 172.21.15.16:0/1333427096 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:31.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.885+0000 7f174d2b6740 1 -- 172.21.15.16:0/1333427096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55db6d1f1ab0 con 0x55db6cf872f0 2024-08-31T06:16:31.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.885+0000 7f174d2b6740 1 -- 172.21.15.16:0/1333427096 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55db6d25f320 con 0x55db6cfb01b0 2024-08-31T06:16:31.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.885+0000 7f174d2b6740 1 -- 172.21.15.16:0/1333427096 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55db6d362010 con 0x55db6cf8f4d0 2024-08-31T06:16:31.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.885+0000 7f173ea00640 1 --2- 172.21.15.16:0/1333427096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6cf8f4d0 0x55db6d35f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:31.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.885+0000 7f174d2b6740 10 client.0 _ll_register_callbacks cb 0x55db6d298e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:16:31.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.886+0000 7f173f400640 1 -- 172.21.15.16:0/1333427096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6cf8f4d0 msgr2=0x55db6d35f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:31.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.886+0000 7f173fe00640 1 --2- 172.21.15.16:0/1333427096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db6cfb01b0 0x55db6d35f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:31.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.886+0000 7f173f400640 1 --2- 172.21.15.16:0/1333427096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6cf8f4d0 0x55db6d35f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:31.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.886+0000 7f173f400640 1 -- 172.21.15.16:0/1333427096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db6cfb01b0 msgr2=0x55db6d35f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:31.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.886+0000 7f173f400640 1 --2- 172.21.15.16:0/1333427096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db6cfb01b0 0x55db6d35f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:31.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.886+0000 7f173f400640 1 -- 172.21.15.16:0/1333427096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db6d01b790 con 0x55db6cf872f0 2024-08-31T06:16:31.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.886+0000 7f173fe00640 1 --2- 172.21.15.16:0/1333427096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db6cfb01b0 0x55db6d35f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:16:31.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.886+0000 7f173f400640 1 --2- 172.21.15.16:0/1333427096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db6cf872f0 0x55db6d35ed70 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1730004d20 tx=0x7f173002a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13683b1bf260bfa7 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:31.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.886+0000 7f1728c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:16:31.892 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.887+0000 7f1728c00640 1 -- 172.21.15.16:0/1333427096 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1730035040 con 0x55db6cf872f0 2024-08-31T06:16:31.892 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.887+0000 7f1728c00640 1 -- 172.21.15.16:0/1333427096 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8470+0+0 (secure 0 0 0) 0x7f17300038a0 con 0x55db6cf872f0 2024-08-31T06:16:31.892 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.887+0000 7f174d2b6740 1 -- 172.21.15.16:0/1333427096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1730005400 con 0x55db6cf872f0 2024-08-31T06:16:31.892 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.887+0000 7f1728c00640 1 -- 172.21.15.16:0/1333427096 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f17300055b0 con 0x55db6cf872f0 2024-08-31T06:16:31.893 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.887+0000 7f174d2b6740 10 client.17370 fetch_fsmap learned FSMap version 43 2024-08-31T06:16:31.893 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.887+0000 7f174d2b6740 1 -- 172.21.15.16:0/1333427096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55db6d094620 con 0x55db6cf872f0 2024-08-31T06:16:31.893 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.888+0000 7f1728c00640 1 -- 172.21.15.16:0/1333427096 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1730005400 con 0x55db6cf872f0 2024-08-31T06:16:31.893 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.888+0000 7f1728c00640 1 -- 172.21.15.16:0/1333427096 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f173004c070 con 0x55db6cf872f0 2024-08-31T06:16:31.893 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.888+0000 7f174d2b6740 10 client.17370 fetch_fsmap finished waiting for FSMap version 43 2024-08-31T06:16:31.893 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.888+0000 7f174d2b6740 10 client.17370 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:16:31.893 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.888+0000 7f174d2b6740 20 client.17370 populate_metadata read hostname 'smithi016' 2024-08-31T06:16:31.893 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.888+0000 7f174d2b6740 1 --2- 172.21.15.16:0/1333427096 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55db6cf6ed80 0x55db6d370cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:31.893 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.888+0000 7f174d2b6740 4 client.17370 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:16:31.893 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.888+0000 7f174d2b6740 1 -- 172.21.15.16:0/1333427096 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db6d2caec0 con 0x55db6cf6ed80 2024-08-31T06:16:31.893 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.888+0000 7f173ea00640 1 --2- 172.21.15.16:0/1333427096 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55db6cf6ed80 0x55db6d370cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:31.893 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.889+0000 7f1728c00640 10 client.17370 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:16:31.894 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.889+0000 7f173ea00640 1 --2- 172.21.15.16:0/1333427096 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55db6cf6ed80 0x55db6d370cd0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab592289a4731532 server_cookie=44ebafdf87218266 in_seq=0 out_seq=0 2024-08-31T06:16:31.894 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.889+0000 7f1728c00640 1 -- 172.21.15.16:0/1333427096 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55db6d2caec0 con 0x55db6cf6ed80 2024-08-31T06:16:31.894 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.889+0000 7f1728c00640 10 client.17370 handle_command_reply: tid=0 2024-08-31T06:16:31.894 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.890+0000 7f174d2b6740 1 client.17370 shutdown 2024-08-31T06:16:31.894 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.890+0000 7f174d2b6740 1 -- 172.21.15.16:0/1333427096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db6cf872f0 msgr2=0x55db6d35ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:31.894 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.890+0000 7f174d2b6740 1 --2- 172.21.15.16:0/1333427096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db6cf872f0 0x55db6d35ed70 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1730004d20 tx=0x7f173002a3f0 comp rx=0 tx=0).stop 2024-08-31T06:16:31.894 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.890+0000 7f174d2b6740 1 -- 172.21.15.16:0/1333427096 shutdown_connections 2024-08-31T06:16:31.894 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.890+0000 7f174d2b6740 1 --2- 172.21.15.16:0/1333427096 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55db6cf6ed80 0x55db6d370cd0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:31.894 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.890+0000 7f174d2b6740 1 --2- 172.21.15.16:0/1333427096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6cf8f4d0 0x55db6d35f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:31.894 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.890+0000 7f174d2b6740 1 --2- 172.21.15.16:0/1333427096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db6cfb01b0 0x55db6d35f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:31.894 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.890+0000 7f174d2b6740 1 --2- 172.21.15.16:0/1333427096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db6cf872f0 0x55db6d35ed70 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:31.894 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.890+0000 7f174d2b6740 1 -- 172.21.15.16:0/1333427096 >> 172.21.15.16:0/1333427096 conn(0x55db6cfa4300 msgr2=0x55db6d3548d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:31.894 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.890+0000 7f1728c00640 0 client.17370 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:16:31.894 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.890+0000 7f174d2b6740 1 -- 172.21.15.16:0/1333427096 shutdown_connections 2024-08-31T06:16:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.890+0000 7f174d2b6740 1 -- 172.21.15.16:0/1333427096 wait complete. 2024-08-31T06:16:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.891+0000 7f174d2b6740 20 client.17370 trim_cache size 0 max 16384 2024-08-31T06:16:31.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.916+0000 7f174d2b6740 1 Processor -- start 2024-08-31T06:16:31.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.916+0000 7f174d2b6740 1 -- start start 2024-08-31T06:16:31.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.917+0000 7f174d2b6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db6cf872f0 0x55db6d350730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:31.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.917+0000 7f174d2b6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db6cf8f4d0 0x55db6d354fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:31.920 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.917+0000 7f174d2b6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6cfb01b0 0x55db6d35bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:31.920 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.917+0000 7f174d2b6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f17401b8150 con 0x55db6cf872f0 2024-08-31T06:16:31.920 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.917+0000 7f174d2b6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f17401b59c0 con 0x55db6cf8f4d0 2024-08-31T06:16:31.920 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.917+0000 7f174d2b6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f17401b3830 con 0x55db6cfb01b0 2024-08-31T06:16:31.920 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.917+0000 7f173f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db6cf872f0 0x55db6d350730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:31.920 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.917+0000 7f173ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db6cf8f4d0 0x55db6d354fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:31.920 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.917+0000 7f173f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db6cf872f0 0x55db6d350730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58012/0 (socket says 172.21.15.16:58012) 2024-08-31T06:16:31.920 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.917+0000 7f173f400640 1 -- 172.21.15.16:0/397303713 learned_addr learned my addr 172.21.15.16:0/397303713 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:31.920 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.917+0000 7f173fe00640 1 --2- 172.21.15.16:0/397303713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6cfb01b0 0x55db6d35bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:31.921 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.918+0000 7f173f400640 1 -- 172.21.15.16:0/397303713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6cfb01b0 msgr2=0x55db6d35bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:31.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.918+0000 7f173f400640 1 --2- 172.21.15.16:0/397303713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6cfb01b0 0x55db6d35bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:31.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.918+0000 7f173f400640 1 -- 172.21.15.16:0/397303713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db6cf8f4d0 msgr2=0x55db6d354fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:31.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.918+0000 7f173f400640 1 --2- 172.21.15.16:0/397303713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db6cf8f4d0 0x55db6d354fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:31.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.918+0000 7f173f400640 1 -- 172.21.15.16:0/397303713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db6d355dd0 con 0x55db6cf872f0 2024-08-31T06:16:31.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.918+0000 7f173ea00640 1 --2- 172.21.15.16:0/397303713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db6cf8f4d0 0x55db6d354fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:16:31.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.918+0000 7f173f400640 1 --2- 172.21.15.16:0/397303713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db6cf872f0 0x55db6d350730 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f1730031a70 tx=0x7f1730005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30af4a3f45aaaa18 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:31.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.918+0000 7f172b400640 1 -- 172.21.15.16:0/397303713 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1730035040 con 0x55db6cf872f0 2024-08-31T06:16:31.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.918+0000 7f172b400640 1 -- 172.21.15.16:0/397303713 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1730005200 con 0x55db6cf872f0 2024-08-31T06:16:31.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.918+0000 7f172b400640 1 -- 172.21.15.16:0/397303713 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1730003e10 con 0x55db6cf872f0 2024-08-31T06:16:31.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.919+0000 7f174d2b6740 1 -- 172.21.15.16:0/397303713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db6cf872f0 msgr2=0x55db6d350730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:31.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.919+0000 7f174d2b6740 1 --2- 172.21.15.16:0/397303713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db6cf872f0 0x55db6d350730 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f1730031a70 tx=0x7f1730005fd0 comp rx=0 tx=0).stop 2024-08-31T06:16:31.923 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.920+0000 7f174d2b6740 1 -- 172.21.15.16:0/397303713 shutdown_connections 2024-08-31T06:16:31.923 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.920+0000 7f174d2b6740 1 --2- 172.21.15.16:0/397303713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6cfb01b0 0x55db6d35bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:31.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.920+0000 7f174d2b6740 1 --2- 172.21.15.16:0/397303713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db6cf8f4d0 0x55db6d354fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:31.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.920+0000 7f174d2b6740 1 --2- 172.21.15.16:0/397303713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db6cf872f0 0x55db6d350730 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:31.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.920+0000 7f174d2b6740 1 -- 172.21.15.16:0/397303713 >> 172.21.15.16:0/397303713 conn(0x55db6cfa4300 msgr2=0x55db6d3548d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:31.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.920+0000 7f174d2b6740 1 -- 172.21.15.16:0/397303713 shutdown_connections 2024-08-31T06:16:31.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.920+0000 7f174d2b6740 1 -- 172.21.15.16:0/397303713 wait complete. 2024-08-31T06:16:31.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.921+0000 7f174d2b6740 1 -- start start 2024-08-31T06:16:31.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.921+0000 7f174d2b6740 1 Processor -- start 2024-08-31T06:16:31.926 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.922+0000 7f174d2b6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db6cf872f0 0x55db6d373890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:31.926 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.922+0000 7f174d2b6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db6cf8f4d0 0x55db6d373e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:31.926 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.922+0000 7f174d2b6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6cfb01b0 0x55db6d3d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:31.926 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.922+0000 7f174d2b6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f17401b3e80 con 0x55db6cf872f0 2024-08-31T06:16:31.926 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.922+0000 7f174d2b6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f17401b52f0 con 0x55db6cf8f4d0 2024-08-31T06:16:31.926 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.922+0000 7f174d2b6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f17401b4550 con 0x55db6cfb01b0 2024-08-31T06:16:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.922+0000 7f174d2b6740 10 client.0 _ll_register_callbacks cb 0x7f172c0eb400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:16:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.922+0000 7f173f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db6cf872f0 0x55db6d373890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.922+0000 7f173ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db6cf8f4d0 0x55db6d373e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.922+0000 7f173fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6cfb01b0 0x55db6d3d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.922+0000 7f173ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db6cf8f4d0 0x55db6d373e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:36938/0 (socket says 172.21.15.16:36938) 2024-08-31T06:16:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.922+0000 7f173f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db6cf872f0 0x55db6d373890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58026/0 (socket says 172.21.15.16:58026) 2024-08-31T06:16:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.922+0000 7f173ea00640 1 -- 172.21.15.16:0/3348866259 learned_addr learned my addr 172.21.15.16:0/3348866259 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.922+0000 7f173f400640 1 -- 172.21.15.16:0/3348866259 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6cfb01b0 msgr2=0x55db6d3d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.922+0000 7f173f400640 1 --2- 172.21.15.16:0/3348866259 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6cfb01b0 0x55db6d3d1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.922+0000 7f173f400640 1 -- 172.21.15.16:0/3348866259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db6cf8f4d0 msgr2=0x55db6d373e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.922+0000 7f173f400640 1 --2- 172.21.15.16:0/3348866259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db6cf8f4d0 0x55db6d373e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.922+0000 7f173f400640 1 -- 172.21.15.16:0/3348866259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db6d3d1a70 con 0x55db6cf872f0 2024-08-31T06:16:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.922+0000 7f173ea00640 1 --2- 172.21.15.16:0/3348866259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db6cf8f4d0 0x55db6d373e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:16:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.922+0000 7f173fe00640 1 --2- 172.21.15.16:0/3348866259 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6cfb01b0 0x55db6d3d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:16:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.923+0000 7f173f400640 1 --2- 172.21.15.16:0/3348866259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db6cf872f0 0x55db6d373890 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f1730031af0 tx=0x7f1730031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4753c92a7ec6e5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.923+0000 7f1728c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:16:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.923+0000 7f1728c00640 1 -- 172.21.15.16:0/3348866259 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17300038a0 con 0x55db6cf872f0 2024-08-31T06:16:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.923+0000 7f1728c00640 1 -- 172.21.15.16:0/3348866259 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8470+0+0 (secure 0 0 0) 0x7f17300053b0 con 0x55db6cf872f0 2024-08-31T06:16:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.923+0000 7f174d2b6740 1 -- 172.21.15.16:0/3348866259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1730005200 con 0x55db6cf872f0 2024-08-31T06:16:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.923+0000 7f1728c00640 1 -- 172.21.15.16:0/3348866259 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f1730009520 con 0x55db6cf872f0 2024-08-31T06:16:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.923+0000 7f174d2b6740 10 client.17376 fetch_fsmap learned FSMap version 43 2024-08-31T06:16:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.923+0000 7f174d2b6740 1 -- 172.21.15.16:0/3348866259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55db6d3db060 con 0x55db6cf872f0 2024-08-31T06:16:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.924+0000 7f1728c00640 1 -- 172.21.15.16:0/3348866259 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1730005200 con 0x55db6cf872f0 2024-08-31T06:16:31.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.925+0000 7f1728c00640 1 -- 172.21.15.16:0/3348866259 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f173005d070 con 0x55db6cf872f0 2024-08-31T06:16:31.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.925+0000 7f174d2b6740 10 client.17376 fetch_fsmap finished waiting for FSMap version 43 2024-08-31T06:16:31.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.925+0000 7f174d2b6740 10 client.17376 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:16:31.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.925+0000 7f174d2b6740 20 client.17376 populate_metadata read hostname 'smithi016' 2024-08-31T06:16:31.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.925+0000 7f174d2b6740 1 --2- 172.21.15.16:0/3348866259 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55db6cf441c0 0x55db6d3dd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:31.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.925+0000 7f174d2b6740 4 client.17376 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:16:31.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.925+0000 7f174d2b6740 1 -- 172.21.15.16:0/3348866259 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55db6d2c6960 con 0x55db6cf441c0 2024-08-31T06:16:31.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.926+0000 7f173ea00640 1 --2- 172.21.15.16:0/3348866259 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55db6cf441c0 0x55db6d3dd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:31.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.927+0000 7f173ea00640 1 --2- 172.21.15.16:0/3348866259 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55db6cf441c0 0x55db6d3dd540 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f61f33af1c8445e server_cookie=c439c20fc70360bf in_seq=0 out_seq=0 2024-08-31T06:16:31.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.927+0000 7f1728c00640 10 client.17376 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:16:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.942+0000 7f1728c00640 1 -- 172.21.15.16:0/3348866259 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55db6d2c6960 con 0x55db6cf441c0 2024-08-31T06:16:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.942+0000 7f1728c00640 10 client.17376 handle_command_reply: tid=0 2024-08-31T06:16:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.942+0000 7f174d2b6740 1 client.17376 shutdown 2024-08-31T06:16:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.942+0000 7f174d2b6740 1 -- 172.21.15.16:0/3348866259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db6cf872f0 msgr2=0x55db6d373890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.942+0000 7f174d2b6740 1 --2- 172.21.15.16:0/3348866259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db6cf872f0 0x55db6d373890 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f1730031af0 tx=0x7f1730031400 comp rx=0 tx=0).stop 2024-08-31T06:16:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.943+0000 7f174d2b6740 1 -- 172.21.15.16:0/3348866259 shutdown_connections 2024-08-31T06:16:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.943+0000 7f174d2b6740 1 --2- 172.21.15.16:0/3348866259 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55db6cf441c0 0x55db6d3dd540 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.943+0000 7f174d2b6740 1 --2- 172.21.15.16:0/3348866259 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55db6cfb01b0 0x55db6d3d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.943+0000 7f174d2b6740 1 --2- 172.21.15.16:0/3348866259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55db6cf8f4d0 0x55db6d373e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.943+0000 7f1728c00640 0 client.17376 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:16:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.943+0000 7f174d2b6740 1 --2- 172.21.15.16:0/3348866259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db6cf872f0 0x55db6d373890 secure :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f1730031af0 tx=0x7f1730031400 comp rx=0 tx=0).stop 2024-08-31T06:16:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.943+0000 7f174d2b6740 1 -- 172.21.15.16:0/3348866259 >> 172.21.15.16:0/3348866259 conn(0x55db6cfa4300 msgr2=0x55db6d355290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.943+0000 7f174d2b6740 1 -- 172.21.15.16:0/3348866259 shutdown_connections 2024-08-31T06:16:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.943+0000 7f174d2b6740 1 -- 172.21.15.16:0/3348866259 wait complete. 2024-08-31T06:16:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.943+0000 7f174d2b6740 20 client.17376 trim_cache size 0 max 16384 2024-08-31T06:16:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:16:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:16:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:16:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:16:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.944+0000 7f1744c00640 1 -- 172.21.15.16:0/1389644343 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f17240b4d20 msgr2=0x7f17240b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.944+0000 7f1744c00640 1 --2- 172.21.15.16:0/1389644343 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f17240b4d20 0x7f17240b7230 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1738005fd0 tx=0x7f1738009600 comp rx=0 tx=0).stop 2024-08-31T06:16:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.944+0000 7f1744c00640 1 -- 172.21.15.16:0/1389644343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1740108fe0 msgr2=0x7f174019a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.944+0000 7f1744c00640 1 --2- 172.21.15.16:0/1389644343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1740108fe0 0x7f174019a5f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f172c00e970 tx=0x7f172c00ee30 comp rx=0 tx=0).stop 2024-08-31T06:16:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.944+0000 7f1744c00640 1 -- 172.21.15.16:0/1389644343 shutdown_connections 2024-08-31T06:16:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.944+0000 7f1744c00640 1 --2- 172.21.15.16:0/1389644343 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f17240b4d20 0x7f17240b7230 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.944+0000 7f1744c00640 1 --2- 172.21.15.16:0/1389644343 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1740109930 0x7f174019f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.944+0000 7f1744c00640 1 --2- 172.21.15.16:0/1389644343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1740108fe0 0x7f174019a5f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.944+0000 7f1744c00640 1 --2- 172.21.15.16:0/1389644343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17401036b0 0x7f174019a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.944+0000 7f1744c00640 1 -- 172.21.15.16:0/1389644343 >> 172.21.15.16:0/1389644343 conn(0x7f17400ff0d0 msgr2=0x7f17401014f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.945+0000 7f1744c00640 1 -- 172.21.15.16:0/1389644343 shutdown_connections 2024-08-31T06:16:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:31.945+0000 7f1744c00640 1 -- 172.21.15.16:0/1389644343 wait complete. 2024-08-31T06:16:32.085 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T06:16:32.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:31 smithi155 ceph-mon[21742]: pgmap v651: 97 pgs: 97 active+clean; 3.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 1.49k op/s 2024-08-31T06:16:32.151 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:31 smithi016 ceph-mon[17194]: pgmap v651: 97 pgs: 97 active+clean; 3.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 1.49k op/s 2024-08-31T06:16:32.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:31 smithi184 ceph-mon[22331]: pgmap v651: 97 pgs: 97 active+clean; 3.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 1.49k op/s 2024-08-31T06:16:32.671 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:16:33.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.458+0000 7fc796400640 1 -- 172.21.15.16:0/1097064591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7901099e0 msgr2=0x7fc790110210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:33.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.458+0000 7fc796400640 1 --2- 172.21.15.16:0/1097064591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7901099e0 0x7fc790110210 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc788009ca0 tx=0x7fc78802a8b0 comp rx=0 tx=0).stop 2024-08-31T06:16:33.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.458+0000 7fc796400640 1 -- 172.21.15.16:0/1097064591 shutdown_connections 2024-08-31T06:16:33.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.458+0000 7fc796400640 1 --2- 172.21.15.16:0/1097064591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc790109090 0x7fc7901094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:33.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.458+0000 7fc796400640 1 --2- 172.21.15.16:0/1097064591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7901099e0 0x7fc790110210 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:33.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.458+0000 7fc796400640 1 --2- 172.21.15.16:0/1097064591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc790103750 0x7fc790103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:33.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.458+0000 7fc796400640 1 -- 172.21.15.16:0/1097064591 >> 172.21.15.16:0/1097064591 conn(0x7fc7900ff1a0 msgr2=0x7fc7901015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:33.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.458+0000 7fc796400640 1 -- 172.21.15.16:0/1097064591 shutdown_connections 2024-08-31T06:16:33.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.458+0000 7fc796400640 1 -- 172.21.15.16:0/1097064591 wait complete. 2024-08-31T06:16:33.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.459+0000 7fc796400640 1 Processor -- start 2024-08-31T06:16:33.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.459+0000 7fc796400640 1 -- start start 2024-08-31T06:16:33.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.459+0000 7fc796400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc790103750 0x7fc79019c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:33.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.460+0000 7fc796400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc790109090 0x7fc79019c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:33.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.460+0000 7fc796400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7901099e0 0x7fc7901a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:33.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.460+0000 7fc796400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc7901136e0 con 0x7fc7901099e0 2024-08-31T06:16:33.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.460+0000 7fc796400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc790113560 con 0x7fc790109090 2024-08-31T06:16:33.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.460+0000 7fc796400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc790113860 con 0x7fc790103750 2024-08-31T06:16:33.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.460+0000 7fc795000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc790103750 0x7fc79019c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:33.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.460+0000 7fc787e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc790109090 0x7fc79019c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:33.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.460+0000 7fc795a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7901099e0 0x7fc7901a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:33.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.460+0000 7fc795000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc790103750 0x7fc79019c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:44010/0 (socket says 172.21.15.16:44010) 2024-08-31T06:16:33.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.460+0000 7fc795000640 1 -- 172.21.15.16:0/944160097 learned_addr learned my addr 172.21.15.16:0/944160097 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:33.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.460+0000 7fc795000640 1 -- 172.21.15.16:0/944160097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc790109090 msgr2=0x7fc79019c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:33.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.460+0000 7fc795000640 1 --2- 172.21.15.16:0/944160097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc790109090 0x7fc79019c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:33.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.460+0000 7fc795000640 1 -- 172.21.15.16:0/944160097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7901099e0 msgr2=0x7fc7901a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:33.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.460+0000 7fc795000640 1 --2- 172.21.15.16:0/944160097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7901099e0 0x7fc7901a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:33.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.460+0000 7fc795000640 1 -- 172.21.15.16:0/944160097 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7901a2180 con 0x7fc790103750 2024-08-31T06:16:33.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.460+0000 7fc787e00640 1 --2- 172.21.15.16:0/944160097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc790109090 0x7fc79019c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:16:33.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.461+0000 7fc795000640 1 --2- 172.21.15.16:0/944160097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc790103750 0x7fc79019c210 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc78000e9f0 tx=0x7fc78000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:33.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.461+0000 7fc78e000640 1 -- 172.21.15.16:0/944160097 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78000cd80 con 0x7fc790103750 2024-08-31T06:16:33.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.461+0000 7fc796400640 1 -- 172.21.15.16:0/944160097 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7901a2470 con 0x7fc790103750 2024-08-31T06:16:33.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.461+0000 7fc78e000640 1 -- 172.21.15.16:0/944160097 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7800108e0 con 0x7fc790103750 2024-08-31T06:16:33.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.461+0000 7fc78e000640 1 -- 172.21.15.16:0/944160097 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc780018980 con 0x7fc790103750 2024-08-31T06:16:33.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.461+0000 7fc796400640 1 -- 172.21.15.16:0/944160097 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7901a8bd0 con 0x7fc790103750 2024-08-31T06:16:33.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.464+0000 7fc79e5e5740 1 Processor -- start 2024-08-31T06:16:33.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.465+0000 7fc79e5e5740 1 -- start start 2024-08-31T06:16:33.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.465+0000 7fc79e5e5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56083cd63070 0x56083d108b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:33.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.465+0000 7fc79e5e5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56083cd3a1b0 0x56083d109060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:33.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.465+0000 7fc79e5e5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56083cd42390 0x56083d1095a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:33.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.465+0000 7fc79e5e5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56083d07f770 con 0x56083cd63070 2024-08-31T06:16:33.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.465+0000 7fc79e5e5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56083d06ed00 con 0x56083cd3a1b0 2024-08-31T06:16:33.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.465+0000 7fc79e5e5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56083d072cb0 con 0x56083cd42390 2024-08-31T06:16:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.465+0000 7fc78e000640 1 -- 172.21.15.16:0/944160097 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fc780010430 con 0x7fc790103750 2024-08-31T06:16:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.465+0000 7fc795a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56083cd63070 0x56083d108b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.465+0000 7fc787e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56083cd42390 0x56083d1095a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.465+0000 7fc795a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56083cd63070 0x56083d108b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58056/0 (socket says 172.21.15.16:58056) 2024-08-31T06:16:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.465+0000 7fc787e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56083cd42390 0x56083d1095a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:44020/0 (socket says 172.21.15.16:44020) 2024-08-31T06:16:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.465+0000 7fc795a00640 1 -- 172.21.15.16:0/4040110421 learned_addr learned my addr 172.21.15.16:0/4040110421 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.466+0000 7fc795000640 1 --2- 172.21.15.16:0/4040110421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56083cd3a1b0 0x56083d109060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.466+0000 7fc787e00640 1 -- 172.21.15.16:0/4040110421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56083cd3a1b0 msgr2=0x56083d109060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.466+0000 7fc787e00640 1 --2- 172.21.15.16:0/4040110421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56083cd3a1b0 0x56083d109060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.466+0000 7fc787e00640 1 -- 172.21.15.16:0/4040110421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56083cd63070 msgr2=0x56083d108b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.466+0000 7fc787e00640 1 --2- 172.21.15.16:0/4040110421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56083cd63070 0x56083d108b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.466+0000 7fc787e00640 1 -- 172.21.15.16:0/4040110421 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56083cdce650 con 0x56083cd42390 2024-08-31T06:16:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.466+0000 7fc787e00640 1 --2- 172.21.15.16:0/4040110421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56083cd42390 0x56083d1095a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc79006b4a0 tx=0x7fc7780118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b1437484c22824c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.466+0000 7fc786000640 1 -- 172.21.15.16:0/4040110421 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc778004020 con 0x56083cd42390 2024-08-31T06:16:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.466+0000 7fc786000640 1 -- 172.21.15.16:0/4040110421 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7780041c0 con 0x56083cd42390 2024-08-31T06:16:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.466+0000 7fc786000640 1 -- 172.21.15.16:0/4040110421 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77800f780 con 0x56083cd42390 2024-08-31T06:16:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.467+0000 7fc79e5e5740 1 -- 172.21.15.16:0/4040110421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56083cd42390 msgr2=0x56083d1095a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.467+0000 7fc79e5e5740 1 --2- 172.21.15.16:0/4040110421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56083cd42390 0x56083d1095a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc79006b4a0 tx=0x7fc7780118d0 comp rx=0 tx=0).stop 2024-08-31T06:16:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.467+0000 7fc79e5e5740 1 -- 172.21.15.16:0/4040110421 shutdown_connections 2024-08-31T06:16:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.467+0000 7fc79e5e5740 1 --2- 172.21.15.16:0/4040110421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56083cd42390 0x56083d1095a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.467+0000 7fc79e5e5740 1 --2- 172.21.15.16:0/4040110421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56083cd3a1b0 0x56083d109060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.467+0000 7fc79e5e5740 1 --2- 172.21.15.16:0/4040110421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56083cd63070 0x56083d108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.467+0000 7fc79e5e5740 1 -- 172.21.15.16:0/4040110421 >> 172.21.15.16:0/4040110421 conn(0x56083cd57460 msgr2=0x56083d106a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.467+0000 7fc79e5e5740 1 -- 172.21.15.16:0/4040110421 shutdown_connections 2024-08-31T06:16:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.467+0000 7fc79e5e5740 1 -- 172.21.15.16:0/4040110421 wait complete. 2024-08-31T06:16:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.468+0000 7fc78e000640 1 --2- 172.21.15.16:0/944160097 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc7740b4bb0 0x7fc7740b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.468+0000 7fc79e5e5740 1 -- start start 2024-08-31T06:16:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.468+0000 7fc78e000640 1 -- 172.21.15.16:0/944160097 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8470+0+0 (secure 0 0 0) 0x7fc7800e5880 con 0x7fc790103750 2024-08-31T06:16:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.468+0000 7fc795a00640 1 --2- 172.21.15.16:0/944160097 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc7740b4bb0 0x7fc7740b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.468+0000 7fc79e5e5740 1 Processor -- start 2024-08-31T06:16:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.469+0000 7fc795a00640 1 --2- 172.21.15.16:0/944160097 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc7740b4bb0 0x7fc7740b70c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc788031040 tx=0x7fc788005d80 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.469+0000 7fc79e5e5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56083cd3a1b0 0x56083d114740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.469+0000 7fc79e5e5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56083cd42390 0x56083d114c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.469+0000 7fc79e5e5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56083cd63070 0x56083d1205f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.469+0000 7fc79e5e5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56083ced1700 con 0x56083cd42390 2024-08-31T06:16:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.469+0000 7fc79e5e5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56083cff5cb0 con 0x56083cd63070 2024-08-31T06:16:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.469+0000 7fc79e5e5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56083d117300 con 0x56083cd3a1b0 2024-08-31T06:16:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.469+0000 7fc79e5e5740 10 client.0 _ll_register_callbacks cb 0x56083d01b430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:16:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.469+0000 7fc795000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56083cd3a1b0 0x56083d114740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.469+0000 7fc795000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56083cd3a1b0 0x56083d114740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:44028/0 (socket says 172.21.15.16:44028) 2024-08-31T06:16:33.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.469+0000 7fc795000640 1 -- 172.21.15.16:0/110719515 learned_addr learned my addr 172.21.15.16:0/110719515 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:33.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.469+0000 7fc787e00640 1 --2- 172.21.15.16:0/110719515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56083cd42390 0x56083d114c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:33.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.469+0000 7fc795a00640 1 --2- 172.21.15.16:0/110719515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56083cd63070 0x56083d1205f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:33.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.469+0000 7fc795000640 1 -- 172.21.15.16:0/110719515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56083cd63070 msgr2=0x56083d1205f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:33.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.469+0000 7fc795000640 1 --2- 172.21.15.16:0/110719515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56083cd63070 0x56083d1205f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:33.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.469+0000 7fc795000640 1 -- 172.21.15.16:0/110719515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56083cd42390 msgr2=0x56083d114c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:33.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.469+0000 7fc795000640 1 --2- 172.21.15.16:0/110719515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56083cd42390 0x56083d114c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:33.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.470+0000 7fc795000640 1 -- 172.21.15.16:0/110719515 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56083d036870 con 0x56083cd3a1b0 2024-08-31T06:16:33.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.470+0000 7fc795000640 1 --2- 172.21.15.16:0/110719515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56083cd3a1b0 0x56083d114740 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc780000c00 tx=0x7fc7800ab820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c74b18da89705b93 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:33.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.470+0000 7fc77ea00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:16:33.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.470+0000 7fc77ea00640 1 -- 172.21.15.16:0/110719515 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7800e8e60 con 0x56083cd3a1b0 2024-08-31T06:16:33.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.470+0000 7fc77ea00640 1 -- 172.21.15.16:0/110719515 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8470+0+0 (secure 0 0 0) 0x7fc7800b3b30 con 0x56083cd3a1b0 2024-08-31T06:16:33.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.470+0000 7fc79e5e5740 1 -- 172.21.15.16:0/110719515 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7780041c0 con 0x56083cd3a1b0 2024-08-31T06:16:33.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.470+0000 7fc77ea00640 1 -- 172.21.15.16:0/110719515 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc7800e3430 con 0x56083cd3a1b0 2024-08-31T06:16:33.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.470+0000 7fc79e5e5740 10 client.27655 fetch_fsmap learned FSMap version 43 2024-08-31T06:16:33.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.470+0000 7fc79e5e5740 1 -- 172.21.15.16:0/110719515 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56083d123320 con 0x56083cd3a1b0 2024-08-31T06:16:33.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.470+0000 7fc77ea00640 1 -- 172.21.15.16:0/110719515 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7780041c0 con 0x56083cd3a1b0 2024-08-31T06:16:33.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.471+0000 7fc77ea00640 1 -- 172.21.15.16:0/110719515 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fc7800a7960 con 0x56083cd3a1b0 2024-08-31T06:16:33.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.471+0000 7fc79e5e5740 10 client.27655 fetch_fsmap finished waiting for FSMap version 43 2024-08-31T06:16:33.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.471+0000 7fc79e5e5740 10 client.27655 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:16:33.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.471+0000 7fc79e5e5740 20 client.27655 populate_metadata read hostname 'smithi016' 2024-08-31T06:16:33.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.471+0000 7fc79e5e5740 1 --2- 172.21.15.16:0/110719515 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56083cd219f0 0x56083d1268d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:33.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.471+0000 7fc79e5e5740 4 client.27655 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:16:33.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.471+0000 7fc79e5e5740 1 -- 172.21.15.16:0/110719515 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56083d0816c0 con 0x56083cd219f0 2024-08-31T06:16:33.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.471+0000 7fc787e00640 1 --2- 172.21.15.16:0/110719515 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56083cd219f0 0x56083d1268d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:33.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.472+0000 7fc787e00640 1 --2- 172.21.15.16:0/110719515 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56083cd219f0 0x56083d1268d0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b00415aef8499353 server_cookie=e447ae77ab035156 in_seq=0 out_seq=0 2024-08-31T06:16:33.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.472+0000 7fc77ea00640 10 client.27655 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:16:33.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.472+0000 7fc77ea00640 1 -- 172.21.15.16:0/110719515 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56083d0816c0 con 0x56083cd219f0 2024-08-31T06:16:33.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.472+0000 7fc77ea00640 10 client.27655 handle_command_reply: tid=0 2024-08-31T06:16:33.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.473+0000 7fc79e5e5740 1 client.27655 shutdown 2024-08-31T06:16:33.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.473+0000 7fc79e5e5740 1 -- 172.21.15.16:0/110719515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56083cd3a1b0 msgr2=0x56083d114740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:33.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.473+0000 7fc79e5e5740 1 --2- 172.21.15.16:0/110719515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56083cd3a1b0 0x56083d114740 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc780000c00 tx=0x7fc7800ab820 comp rx=0 tx=0).stop 2024-08-31T06:16:33.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.473+0000 7fc79e5e5740 1 -- 172.21.15.16:0/110719515 shutdown_connections 2024-08-31T06:16:33.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.473+0000 7fc79e5e5740 1 --2- 172.21.15.16:0/110719515 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56083cd219f0 0x56083d1268d0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:33.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.473+0000 7fc79e5e5740 1 --2- 172.21.15.16:0/110719515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56083cd63070 0x56083d1205f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:33.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.473+0000 7fc79e5e5740 1 --2- 172.21.15.16:0/110719515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56083cd42390 0x56083d114c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:33.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.473+0000 7fc77ea00640 0 client.27655 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:16:33.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.473+0000 7fc79e5e5740 1 --2- 172.21.15.16:0/110719515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56083cd3a1b0 0x56083d114740 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:33.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.473+0000 7fc79e5e5740 1 -- 172.21.15.16:0/110719515 >> 172.21.15.16:0/110719515 conn(0x56083cd57460 msgr2=0x56083d10bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:33.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.473+0000 7fc79e5e5740 1 -- 172.21.15.16:0/110719515 shutdown_connections 2024-08-31T06:16:33.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.473+0000 7fc79e5e5740 1 -- 172.21.15.16:0/110719515 wait complete. 2024-08-31T06:16:33.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.473+0000 7fc79e5e5740 20 client.27655 trim_cache size 0 max 16384 2024-08-31T06:16:33.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.492+0000 7fc79e5e5740 1 Processor -- start 2024-08-31T06:16:33.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.492+0000 7fc79e5e5740 1 -- start start 2024-08-31T06:16:33.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.492+0000 7fc79e5e5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56083cd3a1b0 0x56083d10fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:33.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.492+0000 7fc79e5e5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56083cd42390 0x56083d116170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:33.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.492+0000 7fc79e5e5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56083cd63070 0x56083d105fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:33.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.492+0000 7fc79e5e5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc7901b3e40 con 0x56083cd3a1b0 2024-08-31T06:16:33.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.492+0000 7fc79e5e5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc7901b16b0 con 0x56083cd42390 2024-08-31T06:16:33.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.492+0000 7fc79e5e5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc7901af520 con 0x56083cd63070 2024-08-31T06:16:33.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.493+0000 7fc795000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56083cd3a1b0 0x56083d10fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:33.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.493+0000 7fc795000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56083cd3a1b0 0x56083d10fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58084/0 (socket says 172.21.15.16:58084) 2024-08-31T06:16:33.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.493+0000 7fc795000640 1 -- 172.21.15.16:0/1291492370 learned_addr learned my addr 172.21.15.16:0/1291492370 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:33.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.493+0000 7fc795a00640 1 --2- 172.21.15.16:0/1291492370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56083cd63070 0x56083d105fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:33.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.493+0000 7fc787e00640 1 --2- 172.21.15.16:0/1291492370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56083cd42390 0x56083d116170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:33.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.493+0000 7fc795000640 1 -- 172.21.15.16:0/1291492370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56083cd63070 msgr2=0x56083d105fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:33.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.493+0000 7fc795000640 1 --2- 172.21.15.16:0/1291492370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56083cd63070 0x56083d105fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:33.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.493+0000 7fc795000640 1 -- 172.21.15.16:0/1291492370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56083cd42390 msgr2=0x56083d116170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:33.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.493+0000 7fc795000640 1 --2- 172.21.15.16:0/1291492370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56083cd42390 0x56083d116170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:33.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.493+0000 7fc795000640 1 -- 172.21.15.16:0/1291492370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56083d115800 con 0x56083cd3a1b0 2024-08-31T06:16:33.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.494+0000 7fc795000640 1 --2- 172.21.15.16:0/1291492370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56083cd3a1b0 0x56083d10fd10 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc78000e9c0 tx=0x7fc7800a7fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c9b19dde313dc3b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:33.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.494+0000 7fc785600640 1 -- 172.21.15.16:0/1291492370 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7800e8e60 con 0x56083cd3a1b0 2024-08-31T06:16:33.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.494+0000 7fc785600640 1 -- 172.21.15.16:0/1291492370 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7800b07b0 con 0x56083cd3a1b0 2024-08-31T06:16:33.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.494+0000 7fc785600640 1 -- 172.21.15.16:0/1291492370 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc780093480 con 0x56083cd3a1b0 2024-08-31T06:16:33.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.495+0000 7fc79e5e5740 1 -- 172.21.15.16:0/1291492370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56083cd3a1b0 msgr2=0x56083d10fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:33.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.495+0000 7fc79e5e5740 1 --2- 172.21.15.16:0/1291492370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56083cd3a1b0 0x56083d10fd10 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc78000e9c0 tx=0x7fc7800a7fe0 comp rx=0 tx=0).stop 2024-08-31T06:16:33.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.495+0000 7fc79e5e5740 1 -- 172.21.15.16:0/1291492370 shutdown_connections 2024-08-31T06:16:33.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.495+0000 7fc79e5e5740 1 --2- 172.21.15.16:0/1291492370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56083cd63070 0x56083d105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:33.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.495+0000 7fc79e5e5740 1 --2- 172.21.15.16:0/1291492370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56083cd42390 0x56083d116170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:33.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.495+0000 7fc79e5e5740 1 --2- 172.21.15.16:0/1291492370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56083cd3a1b0 0x56083d10fd10 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:33.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.495+0000 7fc79e5e5740 1 -- 172.21.15.16:0/1291492370 >> 172.21.15.16:0/1291492370 conn(0x56083cd57460 msgr2=0x56083d115a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:33.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.495+0000 7fc79e5e5740 1 -- 172.21.15.16:0/1291492370 shutdown_connections 2024-08-31T06:16:33.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.495+0000 7fc79e5e5740 1 -- 172.21.15.16:0/1291492370 wait complete. 2024-08-31T06:16:33.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.495+0000 7fc79e5e5740 1 -- start start 2024-08-31T06:16:33.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.495+0000 7fc79e5e5740 1 Processor -- start 2024-08-31T06:16:33.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.496+0000 7fc79e5e5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56083cd3a1b0 0x56083d1186d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:33.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.496+0000 7fc79e5e5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56083cd42390 0x56083d118c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:33.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.496+0000 7fc79e5e5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56083cd63070 0x56083d119150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:33.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.496+0000 7fc79e5e5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc7901afb70 con 0x56083cd63070 2024-08-31T06:16:33.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.496+0000 7fc79e5e5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc7901b0fe0 con 0x56083cd42390 2024-08-31T06:16:33.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.496+0000 7fc79e5e5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc7901b0240 con 0x56083cd3a1b0 2024-08-31T06:16:33.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.496+0000 7fc79e5e5740 10 client.0 _ll_register_callbacks cb 0x7fc7800aba10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:16:33.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.496+0000 7fc795a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56083cd63070 0x56083d119150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:33.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.496+0000 7fc795000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56083cd3a1b0 0x56083d1186d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:33.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.496+0000 7fc795a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56083cd63070 0x56083d119150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58086/0 (socket says 172.21.15.16:58086) 2024-08-31T06:16:33.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.496+0000 7fc795a00640 1 -- 172.21.15.16:0/381867715 learned_addr learned my addr 172.21.15.16:0/381867715 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:33.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.496+0000 7fc787e00640 1 --2- 172.21.15.16:0/381867715 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56083cd42390 0x56083d118c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:33.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.497+0000 7fc795a00640 1 -- 172.21.15.16:0/381867715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56083cd3a1b0 msgr2=0x56083d1186d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:33.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.497+0000 7fc795a00640 1 --2- 172.21.15.16:0/381867715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56083cd3a1b0 0x56083d1186d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:33.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.497+0000 7fc795a00640 1 -- 172.21.15.16:0/381867715 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56083cd42390 msgr2=0x56083d118c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:33.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.497+0000 7fc795a00640 1 --2- 172.21.15.16:0/381867715 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56083cd42390 0x56083d118c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:33.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.497+0000 7fc795a00640 1 -- 172.21.15.16:0/381867715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56083d190440 con 0x56083cd63070 2024-08-31T06:16:33.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.497+0000 7fc787e00640 1 --2- 172.21.15.16:0/381867715 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56083cd42390 0x56083d118c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:16:33.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.497+0000 7fc795a00640 1 --2- 172.21.15.16:0/381867715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56083cd63070 0x56083d119150 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc788009ca0 tx=0x7fc78802a8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=684c3db1a99d5d09 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:33.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.497+0000 7fc77ea00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:16:33.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.497+0000 7fc77ea00640 1 -- 172.21.15.16:0/381867715 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc788055420 con 0x56083cd63070 2024-08-31T06:16:33.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.497+0000 7fc77ea00640 1 -- 172.21.15.16:0/381867715 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8470+0+0 (secure 0 0 0) 0x7fc788042030 con 0x56083cd63070 2024-08-31T06:16:33.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.497+0000 7fc79e5e5740 1 -- 172.21.15.16:0/381867715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7800b07b0 con 0x56083cd63070 2024-08-31T06:16:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.498+0000 7fc77ea00640 1 -- 172.21.15.16:0/381867715 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc7880555c0 con 0x56083cd63070 2024-08-31T06:16:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.498+0000 7fc79e5e5740 10 client.17394 fetch_fsmap learned FSMap version 43 2024-08-31T06:16:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.498+0000 7fc79e5e5740 1 -- 172.21.15.16:0/381867715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56083d1916d0 con 0x56083cd63070 2024-08-31T06:16:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.498+0000 7fc77ea00640 1 -- 172.21.15.16:0/381867715 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7800b07b0 con 0x56083cd63070 2024-08-31T06:16:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.498+0000 7fc77ea00640 1 -- 172.21.15.16:0/381867715 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fc788048610 con 0x56083cd63070 2024-08-31T06:16:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.498+0000 7fc79e5e5740 10 client.17394 fetch_fsmap finished waiting for FSMap version 43 2024-08-31T06:16:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.498+0000 7fc79e5e5740 10 client.17394 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:16:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.498+0000 7fc79e5e5740 20 client.17394 populate_metadata read hostname 'smithi016' 2024-08-31T06:16:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.498+0000 7fc79e5e5740 1 --2- 172.21.15.16:0/381867715 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56083ccf6da0 0x56083d193f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.498+0000 7fc79e5e5740 4 client.17394 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T06:16:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.498+0000 7fc79e5e5740 1 -- 172.21.15.16:0/381867715 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56083d07d160 con 0x56083ccf6da0 2024-08-31T06:16:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.498+0000 7fc795000640 1 --2- 172.21.15.16:0/381867715 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56083ccf6da0 0x56083d193f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.499+0000 7fc795000640 1 --2- 172.21.15.16:0/381867715 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56083ccf6da0 0x56083d193f40 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ad8277abeb172df server_cookie=f1d7398b8aaccf07 in_seq=0 out_seq=0 2024-08-31T06:16:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.499+0000 7fc77ea00640 10 client.17394 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:16:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.499+0000 7fc77ea00640 1 -- 172.21.15.16:0/381867715 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3498 (crc 0 0 0) 0x56083d07d160 con 0x56083ccf6da0 2024-08-31T06:16:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.499+0000 7fc77ea00640 10 client.17394 handle_command_reply: tid=0 2024-08-31T06:16:33.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.499+0000 7fc79e5e5740 1 client.17394 shutdown 2024-08-31T06:16:33.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.499+0000 7fc79e5e5740 1 -- 172.21.15.16:0/381867715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56083cd63070 msgr2=0x56083d119150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:33.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.499+0000 7fc79e5e5740 1 --2- 172.21.15.16:0/381867715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56083cd63070 0x56083d119150 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc788009ca0 tx=0x7fc78802a8b0 comp rx=0 tx=0).stop 2024-08-31T06:16:33.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.500+0000 7fc79e5e5740 1 -- 172.21.15.16:0/381867715 shutdown_connections 2024-08-31T06:16:33.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.500+0000 7fc79e5e5740 1 --2- 172.21.15.16:0/381867715 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56083ccf6da0 0x56083d193f40 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:33.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.500+0000 7fc79e5e5740 1 --2- 172.21.15.16:0/381867715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56083cd63070 0x56083d119150 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:33.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.500+0000 7fc79e5e5740 1 --2- 172.21.15.16:0/381867715 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56083cd42390 0x56083d118c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:33.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.500+0000 7fc79e5e5740 1 --2- 172.21.15.16:0/381867715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56083cd3a1b0 0x56083d1186d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:33.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.500+0000 7fc79e5e5740 1 -- 172.21.15.16:0/381867715 >> 172.21.15.16:0/381867715 conn(0x56083cd57460 msgr2=0x56083d1159d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:33.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.500+0000 7fc77ea00640 0 client.17394 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:16:33.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.500+0000 7fc79e5e5740 1 -- 172.21.15.16:0/381867715 shutdown_connections 2024-08-31T06:16:33.953 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:16:33.953 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-08-31T06:16:33.953 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {}, 2024-08-31T06:16:33.953 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-08-31T06:16:33.953 INFO:teuthology.orchestra.run.smithi016.stdout: "f9c4a22e-0c8a-4928-9b0d-e2ce1db5a86b": { 2024-08-31T06:16:33.953 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:15:57.496015+0000", 2024-08-31T06:16:33.953 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:16:33.953 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:16:33.953 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:16:33.953 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:16:33.953 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:16:33.953 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:16:33.953 INFO:teuthology.orchestra.run.smithi016.stdout: "a64d5301-8558-4e9c-ac9b-8d0ac7057e74": { 2024-08-31T06:16:33.954 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:14:20.338104+0000", 2024-08-31T06:16:33.954 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:16:33.954 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:16:33.954 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:16:33.954 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:16:33.954 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:16:33.954 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:16:33.954 INFO:teuthology.orchestra.run.smithi016.stdout: "e739a16e-8ace-43a8-a1ce-ac27d39ef8a3": { 2024-08-31T06:16:33.954 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:54.084646+0000", 2024-08-31T06:16:33.954 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:16:33.954 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:16:33.954 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:16:33.954 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:16:33.954 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:16:33.954 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:16:33.955 INFO:teuthology.orchestra.run.smithi016.stdout: "91b72a34-4ba9-4126-a414-8b862f8c5f78": { 2024-08-31T06:16:33.955 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:13:28.663237+0000", 2024-08-31T06:16:33.955 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:16:33.955 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:16:33.955 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:16:33.955 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:16:33.955 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:16:33.955 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:16:33.955 INFO:teuthology.orchestra.run.smithi016.stdout: "a10cce76-99ab-46b5-b3b2-07b82899b8f9": { 2024-08-31T06:16:33.955 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:11:43.967546+0000", 2024-08-31T06:16:33.955 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:16:33.955 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:16:33.955 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:16:33.955 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:16:33.956 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:16:33.956 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:16:33.956 INFO:teuthology.orchestra.run.smithi016.stdout: "c1fd6904-2e02-47fa-939a-47349b9fe36b": { 2024-08-31T06:16:33.956 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:06.678691+0000", 2024-08-31T06:16:33.956 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:16:33.956 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:16:33.956 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:16:33.956 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:16:33.956 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:16:33.956 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:16:33.957 INFO:teuthology.orchestra.run.smithi016.stdout: "78d6ef02-8551-45e8-ad60-9627eb573fc1": { 2024-08-31T06:16:33.957 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:09:13.547865+0000", 2024-08-31T06:16:33.957 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:16:33.957 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:16:33.957 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:16:33.957 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:16:33.957 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:16:33.957 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:16:33.957 INFO:teuthology.orchestra.run.smithi016.stdout: "c8455d74-2fe7-4d93-bcf2-19b419d16016": { 2024-08-31T06:16:33.957 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:08:24.035960+0000", 2024-08-31T06:16:33.957 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:16:33.957 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:16:33.957 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:16:33.957 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:16:33.958 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:16:33.958 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:16:33.958 INFO:teuthology.orchestra.run.smithi016.stdout: "0fdcd762-533d-42be-9289-b71c8e3b81b1": { 2024-08-31T06:16:33.958 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:15:06.169112+0000", 2024-08-31T06:16:33.958 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:16:33.958 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:16:33.958 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:16:33.958 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:16:33.958 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:16:33.958 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:16:33.958 INFO:teuthology.orchestra.run.smithi016.stdout: "88d7bdb8-cb1c-45e5-ad29-4cd8da6c6003": { 2024-08-31T06:16:33.958 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:07:39.648942+0000", 2024-08-31T06:16:33.958 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:16:33.958 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:16:33.958 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:16:33.959 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:16:33.959 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:16:33.959 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:16:33.959 INFO:teuthology.orchestra.run.smithi016.stdout: "6a03046e-cdf4-4a14-b9bd-b560b8968467": { 2024-08-31T06:16:33.959 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:12:38.002354+0000", 2024-08-31T06:16:33.959 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:16:33.959 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:16:33.959 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:16:33.959 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:16:33.959 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:16:33.959 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:16:33.959 INFO:teuthology.orchestra.run.smithi016.stdout: "c300b7e8-f3fd-482c-a909-9329bc61e570": { 2024-08-31T06:16:33.959 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:06:49.868046+0000", 2024-08-31T06:16:33.959 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:16:33.959 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:16:33.959 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:16:33.960 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:16:33.960 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:16:33.960 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:16:33.960 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:16:33.960 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:16:33.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.500+0000 7fc79e5e5740 1 -- 172.21.15.16:0/381867715 wait complete. 2024-08-31T06:16:33.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.500+0000 7fc79e5e5740 20 client.17394 trim_cache size 0 max 16384 2024-08-31T06:16:33.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.500+0000 7fc796400640 1 -- 172.21.15.16:0/944160097 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc7740b4bb0 msgr2=0x7fc7740b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:33.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.500+0000 7fc796400640 1 --2- 172.21.15.16:0/944160097 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc7740b4bb0 0x7fc7740b70c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc788031040 tx=0x7fc788005d80 comp rx=0 tx=0).stop 2024-08-31T06:16:33.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.501+0000 7fc796400640 1 -- 172.21.15.16:0/944160097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc790103750 msgr2=0x7fc79019c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:33.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.501+0000 7fc796400640 1 --2- 172.21.15.16:0/944160097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc790103750 0x7fc79019c210 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc78000e9f0 tx=0x7fc78000eeb0 comp rx=0 tx=0).stop 2024-08-31T06:16:33.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.501+0000 7fc796400640 1 -- 172.21.15.16:0/944160097 shutdown_connections 2024-08-31T06:16:33.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.501+0000 7fc796400640 1 --2- 172.21.15.16:0/944160097 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc7740b4bb0 0x7fc7740b70c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:33.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.501+0000 7fc796400640 1 --2- 172.21.15.16:0/944160097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7901099e0 0x7fc7901a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:33.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.501+0000 7fc796400640 1 --2- 172.21.15.16:0/944160097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc790109090 0x7fc79019c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:33.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.501+0000 7fc796400640 1 --2- 172.21.15.16:0/944160097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc790103750 0x7fc79019c210 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:33.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.501+0000 7fc796400640 1 -- 172.21.15.16:0/944160097 >> 172.21.15.16:0/944160097 conn(0x7fc7900ff1a0 msgr2=0x7fc7901086c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:33.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.501+0000 7fc796400640 1 -- 172.21.15.16:0/944160097 shutdown_connections 2024-08-31T06:16:33.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:33.501+0000 7fc796400640 1 -- 172.21.15.16:0/944160097 wait complete. 2024-08-31T06:16:34.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:34 smithi155 ceph-mon[21742]: pgmap v652: 97 pgs: 97 active+clean; 3.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 1.49k op/s 2024-08-31T06:16:34.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:34 smithi016 ceph-mon[17194]: pgmap v652: 97 pgs: 97 active+clean; 3.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 1.49k op/s 2024-08-31T06:16:34.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:34 smithi184 ceph-mon[22331]: pgmap v652: 97 pgs: 97 active+clean; 3.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 1.49k op/s 2024-08-31T06:16:35.073 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T06:16:35.074 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:16:35.691 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:16:36.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:36.555+0000 7fcc0aa00640 1 -- 172.21.15.16:0/149832204 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc00035020 con 0x7fcc04096100 2024-08-31T06:16:36.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:36.555+0000 7fcc11600640 1 -- 172.21.15.16:0/149832204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc04096100 msgr2=0x7fcc04096510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:36.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:36.555+0000 7fcc11600640 1 --2- 172.21.15.16:0/149832204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc04096100 0x7fcc04096510 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fcc00009900 tx=0x7fcc0002a3f0 comp rx=0 tx=0).stop 2024-08-31T06:16:36.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:36.557+0000 7fcc11600640 1 -- 172.21.15.16:0/149832204 shutdown_connections 2024-08-31T06:16:36.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:36.557+0000 7fcc11600640 1 --2- 172.21.15.16:0/149832204 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc0409c390 0x7fcc040a2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:36.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:36.557+0000 7fcc11600640 1 --2- 172.21.15.16:0/149832204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc0409ba40 0x7fcc0409be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:36.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:36.557+0000 7fcc11600640 1 --2- 172.21.15.16:0/149832204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc04096100 0x7fcc04096510 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:36.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:36.557+0000 7fcc11600640 1 -- 172.21.15.16:0/149832204 >> 172.21.15.16:0/149832204 conn(0x7fcc0400bb80 msgr2=0x7fcc0400bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:36.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:36.558+0000 7fcc11600640 1 -- 172.21.15.16:0/149832204 shutdown_connections 2024-08-31T06:16:36.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:36.558+0000 7fcc11600640 1 -- 172.21.15.16:0/149832204 wait complete. 2024-08-31T06:16:36.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:36.560+0000 7fcc11600640 1 Processor -- start 2024-08-31T06:16:36.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:36.560+0000 7fcc11600640 1 -- start start 2024-08-31T06:16:36.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:36.561+0000 7fcc11600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc04096100 0x7fcc0412cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:36.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:36.561+0000 7fcc11600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc0409ba40 0x7fcc0412d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:36.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:36.561+0000 7fcc11600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc0409c390 0x7fcc04132400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:36.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:36.561+0000 7fcc11600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcc040a6090 con 0x7fcc04096100 2024-08-31T06:16:36.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:36.561+0000 7fcc11600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcc040a5f10 con 0x7fcc0409c390 2024-08-31T06:16:36.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:36.561+0000 7fcc11600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcc040a6210 con 0x7fcc0409ba40 2024-08-31T06:16:36.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:36.561+0000 7fcc0b400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc0409ba40 0x7fcc0412d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:36.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:36.562+0000 7fcc0b400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc0409ba40 0x7fcc0412d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:52644/0 (socket says 172.21.15.16:52644) 2024-08-31T06:16:36.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:36.562+0000 7fcc0b400640 1 -- 172.21.15.16:0/477941231 learned_addr learned my addr 172.21.15.16:0/477941231 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:36.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:36.562+0000 7fcc10c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc0409c390 0x7fcc04132400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:36.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:36.562+0000 7fcc0be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc04096100 0x7fcc0412cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:36.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:36.563+0000 7fcc0b400640 1 -- 172.21.15.16:0/477941231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc0409c390 msgr2=0x7fcc04132400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:36.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:36.563+0000 7fcc0b400640 1 --2- 172.21.15.16:0/477941231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc0409c390 0x7fcc04132400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:36.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:36.563+0000 7fcc0b400640 1 -- 172.21.15.16:0/477941231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc04096100 msgr2=0x7fcc0412cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:36.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:36.563+0000 7fcc0b400640 1 --2- 172.21.15.16:0/477941231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc04096100 0x7fcc0412cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:36.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:36.563+0000 7fcc0b400640 1 -- 172.21.15.16:0/477941231 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc04132940 con 0x7fcc0409ba40 2024-08-31T06:16:36.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:36.564+0000 7fcc0be00640 1 --2- 172.21.15.16:0/477941231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc04096100 0x7fcc0412cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:16:36.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:36.564+0000 7fcc0b400640 1 --2- 172.21.15.16:0/477941231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc0409ba40 0x7fcc0412d160 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fcbf800d950 tx=0x7fcbf800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:36.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:36.564+0000 7fcc08c00640 1 -- 172.21.15.16:0/477941231 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8017070 con 0x7fcc0409ba40 2024-08-31T06:16:36.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:36.565+0000 7fcc08c00640 1 -- 172.21.15.16:0/477941231 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbf8009c30 con 0x7fcc0409ba40 2024-08-31T06:16:36.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:36.565+0000 7fcc08c00640 1 -- 172.21.15.16:0/477941231 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8003d80 con 0x7fcc0409ba40 2024-08-31T06:16:36.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:36.565+0000 7fcc11600640 1 -- 172.21.15.16:0/477941231 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc04132c30 con 0x7fcc0409ba40 2024-08-31T06:16:36.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:36.565+0000 7fcc11600640 1 -- 172.21.15.16:0/477941231 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc04133060 con 0x7fcc0409ba40 2024-08-31T06:16:36.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:36.567+0000 7fcc11600640 1 -- 172.21.15.16:0/477941231 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc04096510 con 0x7fcc0409ba40 2024-08-31T06:16:36.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:36.574+0000 7fcc08c00640 1 -- 172.21.15.16:0/477941231 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fcbf8004020 con 0x7fcc0409ba40 2024-08-31T06:16:36.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:36.580+0000 7fcc08c00640 1 --2- 172.21.15.16:0/477941231 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fcbf40b4d20 0x7fcbf40b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:36.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:36.581+0000 7fcc08c00640 1 -- 172.21.15.16:0/477941231 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8470+0+0 (secure 0 0 0) 0x7fcbf80e5250 con 0x7fcc0409ba40 2024-08-31T06:16:36.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:36.582+0000 7fcc08c00640 1 -- 172.21.15.16:0/477941231 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fcbf80e5770 con 0x7fcc0409ba40 2024-08-31T06:16:36.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:36.583+0000 7fcc0be00640 1 --2- 172.21.15.16:0/477941231 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fcbf40b4d20 0x7fcbf40b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:36.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:36.585+0000 7fcc0be00640 1 --2- 172.21.15.16:0/477941231 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fcbf40b4d20 0x7fcbf40b7230 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fcc00005ab0 tx=0x7fcc00004040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:36.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 550.0 s, 1531.6 tps, lat 65.974 ms stddev 90.952 2024-08-31T06:16:36.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:36 smithi155 ceph-mon[21742]: pgmap v653: 97 pgs: 97 active+clean; 3.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 15 MiB/s wr, 1.51k op/s 2024-08-31T06:16:36.916 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:36 smithi016 ceph-mon[17194]: pgmap v653: 97 pgs: 97 active+clean; 3.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 15 MiB/s wr, 1.51k op/s 2024-08-31T06:16:36.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:36 smithi184 ceph-mon[22331]: pgmap v653: 97 pgs: 97 active+clean; 3.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 15 MiB/s wr, 1.51k op/s 2024-08-31T06:16:37.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:37.017+0000 7fcc11600640 1 -- 172.21.15.16:0/477941231 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc0412dcf0 con 0x7fcc0409ba40 2024-08-31T06:16:37.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:37.018+0000 7fcc08c00640 1 -- 172.21.15.16:0/477941231 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11920 (secure 0 0 0) 0x7fcbf80ac0f0 con 0x7fcc0409ba40 2024-08-31T06:16:37.021 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:16:37.021 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":43,"btime":"2024-08-31T06:16:03:711285+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:16:03.340730+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24737,24724,24712]},"id":1}]} 2024-08-31T06:16:37.022 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 43 2024-08-31T06:16:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:37.023+0000 7fcc11600640 1 -- 172.21.15.16:0/477941231 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fcbf40b4d20 msgr2=0x7fcbf40b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:37.023+0000 7fcc11600640 1 --2- 172.21.15.16:0/477941231 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fcbf40b4d20 0x7fcbf40b7230 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fcc00005ab0 tx=0x7fcc00004040 comp rx=0 tx=0).stop 2024-08-31T06:16:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:37.023+0000 7fcc11600640 1 -- 172.21.15.16:0/477941231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc0409ba40 msgr2=0x7fcc0412d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:37.023+0000 7fcc11600640 1 --2- 172.21.15.16:0/477941231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc0409ba40 0x7fcc0412d160 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fcbf800d950 tx=0x7fcbf800de10 comp rx=0 tx=0).stop 2024-08-31T06:16:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:37.023+0000 7fcc11600640 1 -- 172.21.15.16:0/477941231 shutdown_connections 2024-08-31T06:16:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:37.023+0000 7fcc11600640 1 --2- 172.21.15.16:0/477941231 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fcbf40b4d20 0x7fcbf40b7230 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:37.023+0000 7fcc11600640 1 --2- 172.21.15.16:0/477941231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc0409ba40 0x7fcc0412d160 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:37.023+0000 7fcc11600640 1 --2- 172.21.15.16:0/477941231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc0409c390 0x7fcc04132400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:37.023+0000 7fcc11600640 1 --2- 172.21.15.16:0/477941231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc04096100 0x7fcc0412cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:37.023+0000 7fcc11600640 1 -- 172.21.15.16:0/477941231 >> 172.21.15.16:0/477941231 conn(0x7fcc0400bb80 msgr2=0x7fcc04099c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:37.023+0000 7fcc11600640 1 -- 172.21.15.16:0/477941231 shutdown_connections 2024-08-31T06:16:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:37.023+0000 7fcc11600640 1 -- 172.21.15.16:0/477941231 wait complete. 2024-08-31T06:16:37.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:37 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/477941231' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:16:37.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:37 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/477941231' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:16:37.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:37 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/477941231' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:16:38.013 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T06:16:38.659 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:16:38.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:38 smithi016 ceph-mon[17194]: pgmap v654: 97 pgs: 97 active+clean; 3.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.3 MiB/s wr, 756 op/s 2024-08-31T06:16:38.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:38 smithi155 ceph-mon[21742]: pgmap v654: 97 pgs: 97 active+clean; 3.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.3 MiB/s wr, 756 op/s 2024-08-31T06:16:38.873 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:38 smithi184 ceph-mon[22331]: pgmap v654: 97 pgs: 97 active+clean; 3.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.3 MiB/s wr, 756 op/s 2024-08-31T06:16:39.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.982+0000 7fe4e6400640 1 -- 172.21.15.16:0/3025878462 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4e010e660 msgr2=0x7fe4e0117080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:39.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.982+0000 7fe4e6400640 1 --2- 172.21.15.16:0/3025878462 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4e010e660 0x7fe4e0117080 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8009ca0 tx=0x7fe4d802a8b0 comp rx=0 tx=0).stop 2024-08-31T06:16:39.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.982+0000 7fe4e6400640 1 -- 172.21.15.16:0/3025878462 shutdown_connections 2024-08-31T06:16:39.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.982+0000 7fe4e6400640 1 --2- 172.21.15.16:0/3025878462 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4e010e660 0x7fe4e0117080 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:39.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.982+0000 7fe4e6400640 1 --2- 172.21.15.16:0/3025878462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4e010dd10 0x7fe4e010e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:39.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.982+0000 7fe4e6400640 1 --2- 172.21.15.16:0/3025878462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4e010ac80 0x7fe4e010b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:39.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.982+0000 7fe4e6400640 1 -- 172.21.15.16:0/3025878462 >> 172.21.15.16:0/3025878462 conn(0x7fe4e006ebf0 msgr2=0x7fe4e006f030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:39.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.983+0000 7fe4e6400640 1 -- 172.21.15.16:0/3025878462 shutdown_connections 2024-08-31T06:16:39.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.984+0000 7fe4e6400640 1 -- 172.21.15.16:0/3025878462 wait complete. 2024-08-31T06:16:39.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.986+0000 7fe4e6400640 1 Processor -- start 2024-08-31T06:16:39.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.986+0000 7fe4e6400640 1 -- start start 2024-08-31T06:16:39.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.986+0000 7fe4e6400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4e010ac80 0x7fe4e01a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:39.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.986+0000 7fe4e6400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4e010dd10 0x7fe4e01a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:39.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.987+0000 7fe4e6400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4e010e660 0x7fe4e01a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:39.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.987+0000 7fe4e6400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e011bef0 con 0x7fe4e010dd10 2024-08-31T06:16:39.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.987+0000 7fe4e6400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e011bd70 con 0x7fe4e010ac80 2024-08-31T06:16:39.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.987+0000 7fe4e6400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e011c070 con 0x7fe4e010e660 2024-08-31T06:16:39.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.987+0000 7fe4e5a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4e010e660 0x7fe4e01a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:39.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.987+0000 7fe4e5a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4e010e660 0x7fe4e01a1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:52658/0 (socket says 172.21.15.16:52658) 2024-08-31T06:16:39.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.987+0000 7fe4e5a00640 1 -- 172.21.15.16:0/242314438 learned_addr learned my addr 172.21.15.16:0/242314438 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:39.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.987+0000 7fe4e5000640 1 --2- 172.21.15.16:0/242314438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4e010ac80 0x7fe4e01a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:39.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.988+0000 7fe4e5000640 1 -- 172.21.15.16:0/242314438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4e010e660 msgr2=0x7fe4e01a1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:39.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.988+0000 7fe4e5000640 1 --2- 172.21.15.16:0/242314438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4e010e660 0x7fe4e01a1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:39.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.988+0000 7fe4e5000640 1 -- 172.21.15.16:0/242314438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4e010dd10 msgr2=0x7fe4e01a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:39.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.988+0000 7fe4dfe00640 1 --2- 172.21.15.16:0/242314438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4e010dd10 0x7fe4e01a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:39.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.988+0000 7fe4e5000640 1 --2- 172.21.15.16:0/242314438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4e010dd10 0x7fe4e01a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:39.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.988+0000 7fe4e5000640 1 -- 172.21.15.16:0/242314438 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4e01a86e0 con 0x7fe4e010ac80 2024-08-31T06:16:39.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.988+0000 7fe4dfe00640 1 --2- 172.21.15.16:0/242314438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4e010dd10 0x7fe4e01a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:16:39.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.989+0000 7fe4e5000640 1 --2- 172.21.15.16:0/242314438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4e010ac80 0x7fe4e01a13a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe4d400ea90 tx=0x7fe4d400ef50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:39.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.990+0000 7fe4dd600640 1 -- 172.21.15.16:0/242314438 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d401b070 con 0x7fe4e010ac80 2024-08-31T06:16:39.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.990+0000 7fe4dd600640 1 -- 172.21.15.16:0/242314438 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4d400cc80 con 0x7fe4e010ac80 2024-08-31T06:16:39.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.990+0000 7fe4dd600640 1 -- 172.21.15.16:0/242314438 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d400b5e0 con 0x7fe4e010ac80 2024-08-31T06:16:39.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.990+0000 7fe4e6400640 1 -- 172.21.15.16:0/242314438 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4e01a89d0 con 0x7fe4e010ac80 2024-08-31T06:16:39.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.990+0000 7fe4e6400640 1 -- 172.21.15.16:0/242314438 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4e01a8ec0 con 0x7fe4e010ac80 2024-08-31T06:16:39.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.997+0000 7fe4ee5fb740 1 Processor -- start 2024-08-31T06:16:40.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.997+0000 7fe4ee5fb740 1 -- start start 2024-08-31T06:16:40.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.998+0000 7fe4ee5fb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcf07c2070 0x55dcf0b67b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:40.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.998+0000 7fe4ee5fb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dcf07991b0 0x55dcf0b680a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:40.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.998+0000 7fe4ee5fb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf07a1390 0x55dcf0b685e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:40.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.998+0000 7fe4ee5fb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dcf0ade7b0 con 0x55dcf07c2070 2024-08-31T06:16:40.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.998+0000 7fe4ee5fb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55dcf0acdd40 con 0x55dcf07a1390 2024-08-31T06:16:40.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.998+0000 7fe4ee5fb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55dcf0ad1cf0 con 0x55dcf07991b0 2024-08-31T06:16:40.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.998+0000 7fe4dd600640 1 -- 172.21.15.16:0/242314438 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fe4d401e070 con 0x7fe4e010ac80 2024-08-31T06:16:40.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.999+0000 7fe4e5000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dcf07991b0 0x55dcf0b680a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:40.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.999+0000 7fe4e5000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dcf07991b0 0x55dcf0b680a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:52670/0 (socket says 172.21.15.16:52670) 2024-08-31T06:16:40.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.999+0000 7fe4e5000640 1 -- 172.21.15.16:0/2757115050 learned_addr learned my addr 172.21.15.16:0/2757115050 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:40.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:39.999+0000 7fe4e5a00640 1 --2- 172.21.15.16:0/2757115050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcf07c2070 0x55dcf0b67b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:40.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.000+0000 7fe4dfe00640 1 --2- 172.21.15.16:0/2757115050 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf07a1390 0x55dcf0b685e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:40.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.000+0000 7fe4e5a00640 1 -- 172.21.15.16:0/2757115050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dcf07991b0 msgr2=0x55dcf0b680a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:40.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.000+0000 7fe4e5a00640 1 --2- 172.21.15.16:0/2757115050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dcf07991b0 0x55dcf0b680a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:40.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.000+0000 7fe4e5a00640 1 -- 172.21.15.16:0/2757115050 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf07a1390 msgr2=0x55dcf0b685e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:40.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.000+0000 7fe4e5a00640 1 --2- 172.21.15.16:0/2757115050 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf07a1390 0x55dcf0b685e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:40.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.000+0000 7fe4e5a00640 1 -- 172.21.15.16:0/2757115050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcf082d650 con 0x55dcf07c2070 2024-08-31T06:16:40.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.000+0000 7fe4dfe00640 1 --2- 172.21.15.16:0/2757115050 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf07a1390 0x55dcf0b685e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:16:40.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.002+0000 7fe4e5a00640 1 --2- 172.21.15.16:0/2757115050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcf07c2070 0x55dcf0b67b60 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8006010 tx=0x7fe4d8030ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f64a45af36d1e4 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:40.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.002+0000 7fe4dd600640 1 --2- 172.21.15.16:0/242314438 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe4c80b4e70 0x7fe4c80b7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:40.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.003+0000 7fe4dd600640 1 -- 172.21.15.16:0/242314438 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8470+0+0 (secure 0 0 0) 0x7fe4d40e4ff0 con 0x7fe4e010ac80 2024-08-31T06:16:40.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.003+0000 7fe4d2000640 1 -- 172.21.15.16:0/2757115050 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8030800 con 0x55dcf07c2070 2024-08-31T06:16:40.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.003+0000 7fe4d2000640 1 -- 172.21.15.16:0/2757115050 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4d8036030 con 0x55dcf07c2070 2024-08-31T06:16:40.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.003+0000 7fe4d2000640 1 -- 172.21.15.16:0/2757115050 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8030800 con 0x55dcf07c2070 2024-08-31T06:16:40.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.003+0000 7fe4e5000640 1 --2- 172.21.15.16:0/242314438 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe4c80b4e70 0x7fe4c80b7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:40.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.004+0000 7fe4ee5fb740 1 -- 172.21.15.16:0/2757115050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcf07c2070 msgr2=0x55dcf0b67b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:40.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.004+0000 7fe4ee5fb740 1 --2- 172.21.15.16:0/2757115050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcf07c2070 0x55dcf0b67b60 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8006010 tx=0x7fe4d8030ff0 comp rx=0 tx=0).stop 2024-08-31T06:16:40.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.004+0000 7fe4e5000640 1 --2- 172.21.15.16:0/242314438 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe4c80b4e70 0x7fe4c80b7380 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40a77e0 tx=0x7fe4d400af90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:40.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.005+0000 7fe4ee5fb740 1 -- 172.21.15.16:0/2757115050 shutdown_connections 2024-08-31T06:16:40.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.005+0000 7fe4ee5fb740 1 --2- 172.21.15.16:0/2757115050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dcf07991b0 0x55dcf0b680a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:40.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.005+0000 7fe4ee5fb740 1 --2- 172.21.15.16:0/2757115050 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf07a1390 0x55dcf0b685e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:40.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.005+0000 7fe4ee5fb740 1 --2- 172.21.15.16:0/2757115050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcf07c2070 0x55dcf0b67b60 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:40.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.005+0000 7fe4ee5fb740 1 -- 172.21.15.16:0/2757115050 >> 172.21.15.16:0/2757115050 conn(0x55dcf07b6460 msgr2=0x55dcf0b65a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:40.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.005+0000 7fe4ee5fb740 1 -- 172.21.15.16:0/2757115050 shutdown_connections 2024-08-31T06:16:40.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.005+0000 7fe4ee5fb740 1 -- 172.21.15.16:0/2757115050 wait complete. 2024-08-31T06:16:40.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.006+0000 7fe4ee5fb740 1 -- start start 2024-08-31T06:16:40.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.006+0000 7fe4ee5fb740 1 Processor -- start 2024-08-31T06:16:40.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.007+0000 7fe4ee5fb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf07991b0 0x55dcf0b736a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:40.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.007+0000 7fe4ee5fb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dcf07a1390 0x55dcf0b73be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:40.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.007+0000 7fe4ee5fb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcf07c2070 0x55dcf0b7f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:40.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.007+0000 7fe4ee5fb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dcf0930700 con 0x55dcf07c2070 2024-08-31T06:16:40.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.007+0000 7fe4ee5fb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55dcf0a54cf0 con 0x55dcf07991b0 2024-08-31T06:16:40.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.007+0000 7fe4ee5fb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55dcf0b762e0 con 0x55dcf07a1390 2024-08-31T06:16:40.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.007+0000 7fe4ee5fb740 10 client.0 _ll_register_callbacks cb 0x55dcf0a7a470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:16:40.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.007+0000 7fe4e5a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf07991b0 0x55dcf0b736a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:40.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.007+0000 7fe4dfe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dcf07a1390 0x55dcf0b73be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:40.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.007+0000 7fe4e5a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf07991b0 0x55dcf0b736a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:48212/0 (socket says 172.21.15.16:48212) 2024-08-31T06:16:40.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.007+0000 7fe4e5a00640 1 -- 172.21.15.16:0/1324484768 learned_addr learned my addr 172.21.15.16:0/1324484768 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:40.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.007+0000 7fe4e5a00640 1 --2- 172.21.15.16:0/1324484768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcf07c2070 0x55dcf0b7f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:40.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.008+0000 7fe4dfe00640 1 -- 172.21.15.16:0/1324484768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf07991b0 msgr2=0x55dcf0b736a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:40.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.008+0000 7fe4dfe00640 1 --2- 172.21.15.16:0/1324484768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf07991b0 0x55dcf0b736a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:40.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.008+0000 7fe4dfe00640 1 -- 172.21.15.16:0/1324484768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcf07c2070 msgr2=0x55dcf0b7f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:40.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.008+0000 7fe4dfe00640 1 --2- 172.21.15.16:0/1324484768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcf07c2070 0x55dcf0b7f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:40.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.008+0000 7fe4dfe00640 1 -- 172.21.15.16:0/1324484768 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcf0a958b0 con 0x55dcf07a1390 2024-08-31T06:16:40.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.008+0000 7fe4dfe00640 1 --2- 172.21.15.16:0/1324484768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dcf07a1390 0x55dcf0b73be0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe4e006c420 tx=0x7fe4cc0118c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c9a79c350f8a94 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:40.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.008+0000 7fe4c6a00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:16:40.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.008+0000 7fe4c6a00640 1 -- 172.21.15.16:0/1324484768 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc004220 con 0x55dcf07a1390 2024-08-31T06:16:40.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.008+0000 7fe4c6a00640 1 -- 172.21.15.16:0/1324484768 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8470+0+0 (secure 0 0 0) 0x7fe4cc0043c0 con 0x55dcf07a1390 2024-08-31T06:16:40.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.008+0000 7fe4ee5fb740 1 -- 172.21.15.16:0/1324484768 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4d8036030 con 0x55dcf07a1390 2024-08-31T06:16:40.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.009+0000 7fe4c6a00640 1 -- 172.21.15.16:0/1324484768 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe4cc012720 con 0x55dcf07a1390 2024-08-31T06:16:40.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.009+0000 7fe4ee5fb740 10 client.27679 fetch_fsmap learned FSMap version 43 2024-08-31T06:16:40.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.009+0000 7fe4ee5fb740 1 -- 172.21.15.16:0/1324484768 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55dcf0b82270 con 0x55dcf07a1390 2024-08-31T06:16:40.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.009+0000 7fe4c6a00640 1 -- 172.21.15.16:0/1324484768 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4d8036030 con 0x55dcf07a1390 2024-08-31T06:16:40.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.010+0000 7fe4c6a00640 1 -- 172.21.15.16:0/1324484768 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fe4cc003c50 con 0x55dcf07a1390 2024-08-31T06:16:40.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.010+0000 7fe4ee5fb740 10 client.27679 fetch_fsmap finished waiting for FSMap version 43 2024-08-31T06:16:40.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.010+0000 7fe4ee5fb740 10 client.27679 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T06:16:40.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.010+0000 7fe4ee5fb740 20 client.27679 populate_metadata read hostname 'smithi016' 2024-08-31T06:16:40.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.010+0000 7fe4ee5fb740 1 --2- 172.21.15.16:0/1324484768 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55dcf07809f0 0x55dcf0b857a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:40.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.010+0000 7fe4ee5fb740 4 client.27679 mds_command: new command op to 24712 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:16:40.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.010+0000 7fe4ee5fb740 1 -- 172.21.15.16:0/1324484768 --> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcf0ae0700 con 0x55dcf07809f0 2024-08-31T06:16:40.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.010+0000 7fe4e5a00640 1 --2- 172.21.15.16:0/1324484768 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55dcf07809f0 0x55dcf0b857a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:40.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.011+0000 7fe4e5a00640 1 --2- 172.21.15.16:0/1324484768 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55dcf07809f0 0x55dcf0b857a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=22f97d1547208e3e server_cookie=986638cb5c9ef531 in_seq=0 out_seq=0 2024-08-31T06:16:40.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.011+0000 7fe4c6a00640 10 client.27679 ms_handle_connect on v2:172.21.15.184:6838/4011051479 2024-08-31T06:16:40.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.011+0000 7fe4c6a00640 1 -- 172.21.15.16:0/1324484768 <== mds.2 v2:172.21.15.184:6838/4011051479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dcf0ae0700 con 0x55dcf07809f0 2024-08-31T06:16:40.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.011+0000 7fe4c6a00640 10 client.27679 handle_command_reply: tid=0 2024-08-31T06:16:40.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.012+0000 7fe4ee5fb740 1 client.27679 shutdown 2024-08-31T06:16:40.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.012+0000 7fe4ee5fb740 1 -- 172.21.15.16:0/1324484768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dcf07a1390 msgr2=0x55dcf0b73be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:40.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.012+0000 7fe4ee5fb740 1 --2- 172.21.15.16:0/1324484768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dcf07a1390 0x55dcf0b73be0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe4e006c420 tx=0x7fe4cc0118c0 comp rx=0 tx=0).stop 2024-08-31T06:16:40.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.012+0000 7fe4ee5fb740 1 -- 172.21.15.16:0/1324484768 shutdown_connections 2024-08-31T06:16:40.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.012+0000 7fe4ee5fb740 1 --2- 172.21.15.16:0/1324484768 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55dcf07809f0 0x55dcf0b857a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:40.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.012+0000 7fe4ee5fb740 1 --2- 172.21.15.16:0/1324484768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dcf07a1390 0x55dcf0b73be0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:40.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.012+0000 7fe4ee5fb740 1 --2- 172.21.15.16:0/1324484768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcf07c2070 0x55dcf0b7f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:40.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.012+0000 7fe4ee5fb740 1 --2- 172.21.15.16:0/1324484768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf07991b0 0x55dcf0b736a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:40.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.012+0000 7fe4c6a00640 0 client.27679 ms_handle_reset on v2:172.21.15.184:6838/4011051479 2024-08-31T06:16:40.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.012+0000 7fe4ee5fb740 1 -- 172.21.15.16:0/1324484768 >> 172.21.15.16:0/1324484768 conn(0x55dcf07b6460 msgr2=0x55dcf0b6ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:40.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.012+0000 7fe4ee5fb740 1 -- 172.21.15.16:0/1324484768 shutdown_connections 2024-08-31T06:16:40.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.012+0000 7fe4ee5fb740 1 -- 172.21.15.16:0/1324484768 wait complete. 2024-08-31T06:16:40.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.012+0000 7fe4ee5fb740 20 client.27679 trim_cache size 0 max 16384 2024-08-31T06:16:40.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.041+0000 7fe4ee5fb740 1 Processor -- start 2024-08-31T06:16:40.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.042+0000 7fe4ee5fb740 1 -- start start 2024-08-31T06:16:40.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.042+0000 7fe4ee5fb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcf07991b0 0x55dcf0b7f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:40.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.042+0000 7fe4ee5fb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dcf07a1390 0x55dcf0b7fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:40.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.042+0000 7fe4ee5fb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf07c2070 0x55dcf0b70690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:40.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.042+0000 7fe4ee5fb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e01c9030 con 0x55dcf07991b0 2024-08-31T06:16:40.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.043+0000 7fe4ee5fb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e01c68a0 con 0x55dcf07c2070 2024-08-31T06:16:40.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.043+0000 7fe4ee5fb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e01c4710 con 0x55dcf07a1390 2024-08-31T06:16:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.043+0000 7fe4dfe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dcf07a1390 0x55dcf0b7fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.043+0000 7fe4e5a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf07c2070 0x55dcf0b70690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.043+0000 7fe4e5a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcf07991b0 0x55dcf0b7f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.043+0000 7fe4dfe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dcf07a1390 0x55dcf0b7fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:52690/0 (socket says 172.21.15.16:52690) 2024-08-31T06:16:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.044+0000 7fe4dfe00640 1 -- 172.21.15.16:0/3533157025 learned_addr learned my addr 172.21.15.16:0/3533157025 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.044+0000 7fe4dfe00640 1 -- 172.21.15.16:0/3533157025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf07c2070 msgr2=0x55dcf0b70690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.044+0000 7fe4dfe00640 1 --2- 172.21.15.16:0/3533157025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf07c2070 0x55dcf0b70690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.044+0000 7fe4dfe00640 1 -- 172.21.15.16:0/3533157025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcf07991b0 msgr2=0x55dcf0b7f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.044+0000 7fe4dfe00640 1 --2- 172.21.15.16:0/3533157025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcf07991b0 0x55dcf0b7f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.044+0000 7fe4dfe00640 1 -- 172.21.15.16:0/3533157025 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcf0b6a080 con 0x55dcf07a1390 2024-08-31T06:16:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.044+0000 7fe4e5a00640 1 --2- 172.21.15.16:0/3533157025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcf07991b0 0x55dcf0b7f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:16:40.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.046+0000 7fe4dfe00640 1 --2- 172.21.15.16:0/3533157025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dcf07a1390 0x55dcf0b7fa40 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc001eb0 tx=0x7fe4cc009660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=602dfd4ad5ea01ef server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.046+0000 7fe4d1600640 1 -- 172.21.15.16:0/3533157025 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc0128c0 con 0x55dcf07a1390 2024-08-31T06:16:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.046+0000 7fe4d1600640 1 -- 172.21.15.16:0/3533157025 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4cc0237e0 con 0x55dcf07a1390 2024-08-31T06:16:40.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.047+0000 7fe4d1600640 1 -- 172.21.15.16:0/3533157025 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc01a7d0 con 0x55dcf07a1390 2024-08-31T06:16:40.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.048+0000 7fe4ee5fb740 1 -- 172.21.15.16:0/3533157025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dcf07a1390 msgr2=0x55dcf0b7fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:40.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.048+0000 7fe4ee5fb740 1 --2- 172.21.15.16:0/3533157025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dcf07a1390 0x55dcf0b7fa40 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc001eb0 tx=0x7fe4cc009660 comp rx=0 tx=0).stop 2024-08-31T06:16:40.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.049+0000 7fe4ee5fb740 1 -- 172.21.15.16:0/3533157025 shutdown_connections 2024-08-31T06:16:40.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.049+0000 7fe4ee5fb740 1 --2- 172.21.15.16:0/3533157025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dcf07a1390 0x55dcf0b7fa40 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:40.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.049+0000 7fe4ee5fb740 1 --2- 172.21.15.16:0/3533157025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf07c2070 0x55dcf0b70690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:40.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.049+0000 7fe4ee5fb740 1 --2- 172.21.15.16:0/3533157025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcf07991b0 0x55dcf0b7f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:40.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.049+0000 7fe4ee5fb740 1 -- 172.21.15.16:0/3533157025 >> 172.21.15.16:0/3533157025 conn(0x55dcf07b6460 msgr2=0x55dcf0b71d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:40.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.050+0000 7fe4ee5fb740 1 -- 172.21.15.16:0/3533157025 shutdown_connections 2024-08-31T06:16:40.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.050+0000 7fe4ee5fb740 1 -- 172.21.15.16:0/3533157025 wait complete. 2024-08-31T06:16:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.051+0000 7fe4ee5fb740 1 -- start start 2024-08-31T06:16:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.052+0000 7fe4ee5fb740 1 Processor -- start 2024-08-31T06:16:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.053+0000 7fe4ee5fb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dcf07991b0 0x55dcf0b78070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.053+0000 7fe4ee5fb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcf07c2070 0x55dcf0b78650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.053+0000 7fe4ee5fb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf0755da0 0x55dcf0be9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.053+0000 7fe4ee5fb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e01c4d60 con 0x55dcf07c2070 2024-08-31T06:16:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.053+0000 7fe4ee5fb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e01c61d0 con 0x55dcf0755da0 2024-08-31T06:16:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.053+0000 7fe4ee5fb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e01c5430 con 0x55dcf07991b0 2024-08-31T06:16:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.053+0000 7fe4ee5fb740 10 client.0 _ll_register_callbacks cb 0x7fe4cc009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:16:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.054+0000 7fe4e5a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dcf07991b0 0x55dcf0b78070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:40.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.055+0000 7fe4e5a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dcf07991b0 0x55dcf0b78070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:52704/0 (socket says 172.21.15.16:52704) 2024-08-31T06:16:40.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.055+0000 7fe4e5a00640 1 -- 172.21.15.16:0/4190884802 learned_addr learned my addr 172.21.15.16:0/4190884802 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:40.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.056+0000 7fe4dfe00640 1 --2- 172.21.15.16:0/4190884802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcf07c2070 0x55dcf0b78650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:40.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.056+0000 7fe4e5a00640 1 --2- 172.21.15.16:0/4190884802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf0755da0 0x55dcf0be9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:40.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.057+0000 7fe4e5a00640 1 -- 172.21.15.16:0/4190884802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf0755da0 msgr2=0x55dcf0be9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:40.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.057+0000 7fe4e5a00640 1 --2- 172.21.15.16:0/4190884802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf0755da0 0x55dcf0be9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:40.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.057+0000 7fe4e5a00640 1 -- 172.21.15.16:0/4190884802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcf07c2070 msgr2=0x55dcf0b78650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:40.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.057+0000 7fe4e5a00640 1 --2- 172.21.15.16:0/4190884802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcf07c2070 0x55dcf0b78650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:40.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.057+0000 7fe4e5a00640 1 -- 172.21.15.16:0/4190884802 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcf0be98f0 con 0x55dcf07991b0 2024-08-31T06:16:40.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.058+0000 7fe4dfe00640 1 --2- 172.21.15.16:0/4190884802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcf07c2070 0x55dcf0b78650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:16:40.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.059+0000 7fe4e5a00640 1 --2- 172.21.15.16:0/4190884802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dcf07991b0 0x55dcf0b78070 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8005e30 tx=0x7fe4d8004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=995c16eef45d56f0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:40.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.060+0000 7fe4c6a00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:16:40.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.061+0000 7fe4c6a00640 1 -- 172.21.15.16:0/4190884802 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8030800 con 0x55dcf07991b0 2024-08-31T06:16:40.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.061+0000 7fe4c6a00640 1 -- 172.21.15.16:0/4190884802 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8470+0+0 (secure 0 0 0) 0x7fe4d804a040 con 0x55dcf07991b0 2024-08-31T06:16:40.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.061+0000 7fe4ee5fb740 1 -- 172.21.15.16:0/4190884802 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4cc0237e0 con 0x55dcf07991b0 2024-08-31T06:16:40.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.061+0000 7fe4c6a00640 1 -- 172.21.15.16:0/4190884802 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe4d8030bb0 con 0x55dcf07991b0 2024-08-31T06:16:40.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.062+0000 7fe4ee5fb740 10 client.27685 fetch_fsmap learned FSMap version 43 2024-08-31T06:16:40.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.062+0000 7fe4ee5fb740 1 -- 172.21.15.16:0/4190884802 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55dcf0bf0c90 con 0x55dcf07991b0 2024-08-31T06:16:40.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.062+0000 7fe4c6a00640 1 -- 172.21.15.16:0/4190884802 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4cc0237e0 con 0x55dcf07991b0 2024-08-31T06:16:40.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.063+0000 7fe4c6a00640 1 -- 172.21.15.16:0/4190884802 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fe4d8043070 con 0x55dcf07991b0 2024-08-31T06:16:40.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.063+0000 7fe4ee5fb740 10 client.27685 fetch_fsmap finished waiting for FSMap version 43 2024-08-31T06:16:40.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.063+0000 7fe4ee5fb740 10 client.27685 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T06:16:40.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.063+0000 7fe4ee5fb740 20 client.27685 populate_metadata read hostname 'smithi016' 2024-08-31T06:16:40.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.063+0000 7fe4ee5fb740 1 --2- 172.21.15.16:0/4190884802 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55dcf0bf0fd0 0x55dcf0bf33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:40.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.063+0000 7fe4ee5fb740 4 client.27685 mds_command: new command op to 24712 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:16:40.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.063+0000 7fe4ee5fb740 1 -- 172.21.15.16:0/4190884802 --> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dcf0adc1a0 con 0x55dcf0bf0fd0 2024-08-31T06:16:40.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.063+0000 7fe4dfe00640 1 --2- 172.21.15.16:0/4190884802 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55dcf0bf0fd0 0x55dcf0bf33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:40.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.064+0000 7fe4dfe00640 1 --2- 172.21.15.16:0/4190884802 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55dcf0bf0fd0 0x55dcf0bf33f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=67e2f3dd2a6647b0 server_cookie=8bf195ff93997572 in_seq=0 out_seq=0 2024-08-31T06:16:40.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.064+0000 7fe4c6a00640 10 client.27685 ms_handle_connect on v2:172.21.15.184:6838/4011051479 2024-08-31T06:16:40.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.064+0000 7fe4c6a00640 1 -- 172.21.15.16:0/4190884802 <== mds.2 v2:172.21.15.184:6838/4011051479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dcf0adc1a0 con 0x55dcf0bf0fd0 2024-08-31T06:16:40.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.064+0000 7fe4c6a00640 10 client.27685 handle_command_reply: tid=0 2024-08-31T06:16:40.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.065+0000 7fe4ee5fb740 1 client.27685 shutdown 2024-08-31T06:16:40.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.065+0000 7fe4ee5fb740 1 -- 172.21.15.16:0/4190884802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dcf07991b0 msgr2=0x55dcf0b78070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:40.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.065+0000 7fe4ee5fb740 1 --2- 172.21.15.16:0/4190884802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dcf07991b0 0x55dcf0b78070 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8005e30 tx=0x7fe4d8004040 comp rx=0 tx=0).stop 2024-08-31T06:16:40.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.065+0000 7fe4ee5fb740 1 -- 172.21.15.16:0/4190884802 shutdown_connections 2024-08-31T06:16:40.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.065+0000 7fe4ee5fb740 1 --2- 172.21.15.16:0/4190884802 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55dcf0bf0fd0 0x55dcf0bf33f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:40.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.065+0000 7fe4ee5fb740 1 --2- 172.21.15.16:0/4190884802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf0755da0 0x55dcf0be9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:40.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.065+0000 7fe4ee5fb740 1 --2- 172.21.15.16:0/4190884802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcf07c2070 0x55dcf0b78650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:40.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.065+0000 7fe4ee5fb740 1 --2- 172.21.15.16:0/4190884802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dcf07991b0 0x55dcf0b78070 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:40.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.065+0000 7fe4ee5fb740 1 -- 172.21.15.16:0/4190884802 >> 172.21.15.16:0/4190884802 conn(0x55dcf07b6460 msgr2=0x55dcf0b71ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:40.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.065+0000 7fe4c6a00640 0 client.27685 ms_handle_reset on v2:172.21.15.184:6838/4011051479 2024-08-31T06:16:40.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.066+0000 7fe4ee5fb740 1 -- 172.21.15.16:0/4190884802 shutdown_connections 2024-08-31T06:16:40.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.066+0000 7fe4ee5fb740 1 -- 172.21.15.16:0/4190884802 wait complete. 2024-08-31T06:16:40.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.069+0000 7fe4ee5fb740 20 client.27685 trim_cache size 0 max 16384 2024-08-31T06:16:40.072 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:16:40.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.070+0000 7fe4e6400640 1 -- 172.21.15.16:0/242314438 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe4c80b4e70 msgr2=0x7fe4c80b7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:40.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.070+0000 7fe4e6400640 1 --2- 172.21.15.16:0/242314438 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe4c80b4e70 0x7fe4c80b7380 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40a77e0 tx=0x7fe4d400af90 comp rx=0 tx=0).stop 2024-08-31T06:16:40.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.070+0000 7fe4e6400640 1 -- 172.21.15.16:0/242314438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4e010ac80 msgr2=0x7fe4e01a13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:40.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.070+0000 7fe4e6400640 1 --2- 172.21.15.16:0/242314438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4e010ac80 0x7fe4e01a13a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe4d400ea90 tx=0x7fe4d400ef50 comp rx=0 tx=0).stop 2024-08-31T06:16:40.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.070+0000 7fe4e6400640 1 -- 172.21.15.16:0/242314438 shutdown_connections 2024-08-31T06:16:40.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.070+0000 7fe4e6400640 1 --2- 172.21.15.16:0/242314438 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe4c80b4e70 0x7fe4c80b7380 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:40.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.070+0000 7fe4e6400640 1 --2- 172.21.15.16:0/242314438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4e010e660 0x7fe4e01a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:40.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.070+0000 7fe4e6400640 1 --2- 172.21.15.16:0/242314438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4e010dd10 0x7fe4e01a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:40.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.070+0000 7fe4e6400640 1 --2- 172.21.15.16:0/242314438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4e010ac80 0x7fe4e01a13a0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:40.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.070+0000 7fe4e6400640 1 -- 172.21.15.16:0/242314438 >> 172.21.15.16:0/242314438 conn(0x7fe4e006ebf0 msgr2=0x7fe4e0118080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:40.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.070+0000 7fe4e6400640 1 -- 172.21.15.16:0/242314438 shutdown_connections 2024-08-31T06:16:40.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:40.071+0000 7fe4e6400640 1 -- 172.21.15.16:0/242314438 wait complete. 2024-08-31T06:16:40.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:40 smithi155 ceph-mon[21742]: pgmap v655: 97 pgs: 97 active+clean; 3.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.0 MiB/s wr, 628 op/s 2024-08-31T06:16:40.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:40 smithi016 ceph-mon[17194]: pgmap v655: 97 pgs: 97 active+clean; 3.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.0 MiB/s wr, 628 op/s 2024-08-31T06:16:40.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:40 smithi184 ceph-mon[22331]: pgmap v655: 97 pgs: 97 active+clean; 3.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.0 MiB/s wr, 628 op/s 2024-08-31T06:16:41.109 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T06:16:41.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 555.0 s, 1785.8 tps, lat 56.316 ms stddev 64.569 2024-08-31T06:16:41.799 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:16:42.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.709+0000 7f60be600640 1 -- 172.21.15.16:0/3869269611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60b81018a0 msgr2=0x7f60b8101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:42.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.709+0000 7f60be600640 1 --2- 172.21.15.16:0/3869269611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60b81018a0 0x7f60b8101cb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f60a8009900 tx=0x7f60a802a400 comp rx=0 tx=0).stop 2024-08-31T06:16:42.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.710+0000 7f60be600640 1 -- 172.21.15.16:0/3869269611 shutdown_connections 2024-08-31T06:16:42.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.710+0000 7f60be600640 1 --2- 172.21.15.16:0/3869269611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60b810fd90 0x7f60b81122a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:42.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.710+0000 7f60be600640 1 --2- 172.21.15.16:0/3869269611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60b8102280 0x7f60b810f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:42.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.710+0000 7f60be600640 1 --2- 172.21.15.16:0/3869269611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60b81018a0 0x7f60b8101cb0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:42.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.710+0000 7f60be600640 1 -- 172.21.15.16:0/3869269611 >> 172.21.15.16:0/3869269611 conn(0x7f60b80775a0 msgr2=0x7f60b80779e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:42.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.710+0000 7f60be600640 1 -- 172.21.15.16:0/3869269611 shutdown_connections 2024-08-31T06:16:42.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.710+0000 7f60be600640 1 -- 172.21.15.16:0/3869269611 wait complete. 2024-08-31T06:16:42.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.711+0000 7f60be600640 1 Processor -- start 2024-08-31T06:16:42.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.712+0000 7f60be600640 1 -- start start 2024-08-31T06:16:42.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.712+0000 7f60be600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60b81018a0 0x7f60b819c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:42.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.712+0000 7f60be600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60b8102280 0x7f60b819ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:42.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.712+0000 7f60be600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60b810fd90 0x7f60b81a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:42.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.712+0000 7f60be600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f60b8115a60 con 0x7f60b81018a0 2024-08-31T06:16:42.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.712+0000 7f60be600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f60b81158e0 con 0x7f60b810fd90 2024-08-31T06:16:42.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.712+0000 7f60be600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f60b8115be0 con 0x7f60b8102280 2024-08-31T06:16:42.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.713+0000 7f60b7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60b81018a0 0x7f60b819c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:42.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.713+0000 7f60b6a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60b8102280 0x7f60b819ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:42.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.713+0000 7f60b7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60b81018a0 0x7f60b819c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51350/0 (socket says 172.21.15.16:51350) 2024-08-31T06:16:42.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.713+0000 7f60b7400640 1 -- 172.21.15.16:0/677173816 learned_addr learned my addr 172.21.15.16:0/677173816 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:42.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.713+0000 7f60b7e00640 1 --2- 172.21.15.16:0/677173816 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60b810fd90 0x7f60b81a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:42.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.713+0000 7f60b6a00640 1 -- 172.21.15.16:0/677173816 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60b810fd90 msgr2=0x7f60b81a1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:42.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.713+0000 7f60b6a00640 1 --2- 172.21.15.16:0/677173816 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60b810fd90 0x7f60b81a1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:42.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.713+0000 7f60b6a00640 1 -- 172.21.15.16:0/677173816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60b81018a0 msgr2=0x7f60b819c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:42.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.713+0000 7f60b6a00640 1 --2- 172.21.15.16:0/677173816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60b81018a0 0x7f60b819c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:42.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.713+0000 7f60b6a00640 1 -- 172.21.15.16:0/677173816 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b81a23e0 con 0x7f60b8102280 2024-08-31T06:16:42.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.714+0000 7f60b6a00640 1 --2- 172.21.15.16:0/677173816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60b8102280 0x7f60b819ca10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f60a4005fd0 tx=0x7f60a400dd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:42.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.714+0000 7f60afe00640 1 -- 172.21.15.16:0/677173816 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a401c070 con 0x7f60b8102280 2024-08-31T06:16:42.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.714+0000 7f60afe00640 1 -- 172.21.15.16:0/677173816 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60a40097f0 con 0x7f60b8102280 2024-08-31T06:16:42.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.714+0000 7f60afe00640 1 -- 172.21.15.16:0/677173816 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a4003850 con 0x7f60b8102280 2024-08-31T06:16:42.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.714+0000 7f60be600640 1 -- 172.21.15.16:0/677173816 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60b81a26d0 con 0x7f60b8102280 2024-08-31T06:16:42.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.715+0000 7f60be600640 1 -- 172.21.15.16:0/677173816 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60b8105280 con 0x7f60b8102280 2024-08-31T06:16:42.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.720+0000 7f60c5263740 1 Processor -- start 2024-08-31T06:16:42.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.720+0000 7f60c5263740 1 -- start start 2024-08-31T06:16:42.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.721+0000 7f60c5263740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d5723a070 0x563d575df8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:42.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.721+0000 7f60c5263740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d572111b0 0x563d575dfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:42.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.721+0000 7f60c5263740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d57219390 0x563d575e0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:42.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.721+0000 7f60c5263740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563d575567b0 con 0x563d572111b0 2024-08-31T06:16:42.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.721+0000 7f60c5263740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563d57545d40 con 0x563d57219390 2024-08-31T06:16:42.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.721+0000 7f60c5263740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563d57549cf0 con 0x563d5723a070 2024-08-31T06:16:42.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.721+0000 7f60b7e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d5723a070 0x563d575df8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:42.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.721+0000 7f60b7e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d5723a070 0x563d575df8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:52732/0 (socket says 172.21.15.16:52732) 2024-08-31T06:16:42.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.721+0000 7f60b7e00640 1 -- 172.21.15.16:0/1342463441 learned_addr learned my addr 172.21.15.16:0/1342463441 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:42.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.722+0000 7f60b7400640 1 --2- 172.21.15.16:0/1342463441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d572111b0 0x563d575dfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:42.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.722+0000 7f60afe00640 1 -- 172.21.15.16:0/677173816 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f60a4009c80 con 0x7f60b8102280 2024-08-31T06:16:42.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.722+0000 7f60b7e00640 1 -- 172.21.15.16:0/1342463441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d57219390 msgr2=0x563d575e0330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T06:16:42.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.722+0000 7f60b7e00640 1 --2- 172.21.15.16:0/1342463441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d57219390 0x563d575e0330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:42.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.722+0000 7f60b7e00640 1 -- 172.21.15.16:0/1342463441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d572111b0 msgr2=0x563d575dfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:42.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.722+0000 7f60b7e00640 1 --2- 172.21.15.16:0/1342463441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d572111b0 0x563d575dfdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:42.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.722+0000 7f60b7e00640 1 -- 172.21.15.16:0/1342463441 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d572a5650 con 0x563d5723a070 2024-08-31T06:16:42.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.723+0000 7f60b7e00640 1 --2- 172.21.15.16:0/1342463441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d5723a070 0x563d575df8b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f60b000ae10 tx=0x7f60b000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33032ab45dc56d4d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:42.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.723+0000 7f60acc00640 1 -- 172.21.15.16:0/1342463441 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b0003b70 con 0x563d5723a070 2024-08-31T06:16:42.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.723+0000 7f60acc00640 1 -- 172.21.15.16:0/1342463441 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60b0003d10 con 0x563d5723a070 2024-08-31T06:16:42.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.723+0000 7f60acc00640 1 -- 172.21.15.16:0/1342463441 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b0010a00 con 0x563d5723a070 2024-08-31T06:16:42.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.725+0000 7f60c5263740 1 -- 172.21.15.16:0/1342463441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d5723a070 msgr2=0x563d575df8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:42.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.725+0000 7f60c5263740 1 --2- 172.21.15.16:0/1342463441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d5723a070 0x563d575df8b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f60b000ae10 tx=0x7f60b000f790 comp rx=0 tx=0).stop 2024-08-31T06:16:42.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.725+0000 7f60c5263740 1 -- 172.21.15.16:0/1342463441 shutdown_connections 2024-08-31T06:16:42.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.725+0000 7f60c5263740 1 --2- 172.21.15.16:0/1342463441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d57219390 0x563d575e0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:42.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.725+0000 7f60c5263740 1 --2- 172.21.15.16:0/1342463441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d572111b0 0x563d575dfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:42.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.725+0000 7f60c5263740 1 --2- 172.21.15.16:0/1342463441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d5723a070 0x563d575df8b0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:42.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.725+0000 7f60c5263740 1 -- 172.21.15.16:0/1342463441 >> 172.21.15.16:0/1342463441 conn(0x563d5722e460 msgr2=0x563d575dd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:42.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.725+0000 7f60afe00640 1 --2- 172.21.15.16:0/677173816 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f609c0b4e30 0x7f609c0b7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:42.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.725+0000 7f60c5263740 1 -- 172.21.15.16:0/1342463441 shutdown_connections 2024-08-31T06:16:42.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.725+0000 7f60afe00640 1 -- 172.21.15.16:0/677173816 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8470+0+0 (secure 0 0 0) 0x7f60a40ed3c0 con 0x7f60b8102280 2024-08-31T06:16:42.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.726+0000 7f60c5263740 1 -- 172.21.15.16:0/1342463441 wait complete. 2024-08-31T06:16:42.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.726+0000 7f60b7400640 1 --2- 172.21.15.16:0/677173816 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f609c0b4e30 0x7f609c0b7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:42.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.727+0000 7f60c5263740 1 -- start start 2024-08-31T06:16:42.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.727+0000 7f60c5263740 1 Processor -- start 2024-08-31T06:16:42.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.728+0000 7f60b7400640 1 --2- 172.21.15.16:0/677173816 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f609c0b4e30 0x7f609c0b7340 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f60a8005f70 tx=0x7f60a802a900 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:42.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.728+0000 7f60c5263740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d572111b0 0x563d575eb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:42.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.728+0000 7f60c5263740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d57219390 0x563d575ebaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:42.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.728+0000 7f60c5263740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d5723a070 0x563d575f1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:42.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.728+0000 7f60c5263740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563d573a8700 con 0x563d57219390 2024-08-31T06:16:42.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.728+0000 7f60c5263740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563d574cccf0 con 0x563d5723a070 2024-08-31T06:16:42.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.728+0000 7f60c5263740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563d575ee090 con 0x563d572111b0 2024-08-31T06:16:42.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.728+0000 7f60c5263740 10 client.0 _ll_register_callbacks cb 0x563d574f2470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:16:42.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.729+0000 7f60b7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d5723a070 0x563d575f1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:42.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.729+0000 7f60b7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d5723a070 0x563d575f1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:48278/0 (socket says 172.21.15.16:48278) 2024-08-31T06:16:42.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.729+0000 7f60b7e00640 1 -- 172.21.15.16:0/989414992 learned_addr learned my addr 172.21.15.16:0/989414992 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:42.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.729+0000 7f60b6a00640 1 --2- 172.21.15.16:0/989414992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d57219390 0x563d575ebaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:42.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.729+0000 7f60b7400640 1 --2- 172.21.15.16:0/989414992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d572111b0 0x563d575eb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:42.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.730+0000 7f60b6a00640 1 -- 172.21.15.16:0/989414992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d572111b0 msgr2=0x563d575eb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:42.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.730+0000 7f60b6a00640 1 --2- 172.21.15.16:0/989414992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d572111b0 0x563d575eb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:42.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.731+0000 7f60b6a00640 1 -- 172.21.15.16:0/989414992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d5723a070 msgr2=0x563d575f1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:42.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.731+0000 7f60b6a00640 1 --2- 172.21.15.16:0/989414992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d5723a070 0x563d575f1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:42.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.731+0000 7f60b6a00640 1 -- 172.21.15.16:0/989414992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d5750d8b0 con 0x563d57219390 2024-08-31T06:16:42.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.731+0000 7f60b7e00640 1 --2- 172.21.15.16:0/989414992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d5723a070 0x563d575f1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:16:42.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.731+0000 7f60b7400640 1 --2- 172.21.15.16:0/989414992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d572111b0 0x563d575eb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:16:42.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.732+0000 7f60b6a00640 1 --2- 172.21.15.16:0/989414992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d57219390 0x563d575ebaf0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f60a40eab40 tx=0x7f60a40b36b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc7a612f1b1e34c1 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:42.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.732+0000 7f60a1600640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:16:42.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.733+0000 7f60a1600640 1 -- 172.21.15.16:0/989414992 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a40b8070 con 0x563d57219390 2024-08-31T06:16:42.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.733+0000 7f60a1600640 1 -- 172.21.15.16:0/989414992 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8470+0+0 (secure 0 0 0) 0x7f60a40b3d40 con 0x563d57219390 2024-08-31T06:16:42.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.734+0000 7f60c5263740 1 -- 172.21.15.16:0/989414992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60b0003d10 con 0x563d57219390 2024-08-31T06:16:42.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.734+0000 7f60a1600640 1 -- 172.21.15.16:0/989414992 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f60a40ce050 con 0x563d57219390 2024-08-31T06:16:42.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.734+0000 7f60c5263740 10 client.17421 fetch_fsmap learned FSMap version 43 2024-08-31T06:16:42.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.734+0000 7f60c5263740 1 -- 172.21.15.16:0/989414992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x563d575fa230 con 0x563d57219390 2024-08-31T06:16:42.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.735+0000 7f60a1600640 1 -- 172.21.15.16:0/989414992 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60b0003d10 con 0x563d57219390 2024-08-31T06:16:42.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.736+0000 7f60a1600640 1 -- 172.21.15.16:0/989414992 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f60a40bc970 con 0x563d57219390 2024-08-31T06:16:42.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.736+0000 7f60c5263740 10 client.17421 fetch_fsmap finished waiting for FSMap version 43 2024-08-31T06:16:42.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.736+0000 7f60c5263740 10 client.17421 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-31T06:16:42.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.736+0000 7f60c5263740 20 client.17421 populate_metadata read hostname 'smithi016' 2024-08-31T06:16:42.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.736+0000 7f60c5263740 1 --2- 172.21.15.16:0/989414992 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x563d571f89f0 0x563d575fdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:42.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.736+0000 7f60c5263740 4 client.17421 mds_command: new command op to 24724 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:16:42.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.736+0000 7f60c5263740 1 -- 172.21.15.16:0/989414992 --> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d57558700 con 0x563d571f89f0 2024-08-31T06:16:42.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.737+0000 7f60b7400640 1 --2- 172.21.15.16:0/989414992 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x563d571f89f0 0x563d575fdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:42.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.739+0000 7f60a1600640 10 client.17421 ms_handle_connect on v2:172.21.15.16:6838/219664460 2024-08-31T06:16:42.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.739+0000 7f60b7400640 1 --2- 172.21.15.16:0/989414992 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x563d571f89f0 0x563d575fdbc0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=96bdf4bb472716de server_cookie=de2505dda8b7c087 in_seq=0 out_seq=0 2024-08-31T06:16:42.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.741+0000 7f60a1600640 1 -- 172.21.15.16:0/989414992 <== mds.1 v2:172.21.15.16:6838/219664460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563d57558700 con 0x563d571f89f0 2024-08-31T06:16:42.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.741+0000 7f60a1600640 10 client.17421 handle_command_reply: tid=0 2024-08-31T06:16:42.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.741+0000 7f60c5263740 1 client.17421 shutdown 2024-08-31T06:16:42.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.741+0000 7f60c5263740 1 -- 172.21.15.16:0/989414992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d57219390 msgr2=0x563d575ebaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:42.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.741+0000 7f60c5263740 1 --2- 172.21.15.16:0/989414992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d57219390 0x563d575ebaf0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f60a40eab40 tx=0x7f60a40b36b0 comp rx=0 tx=0).stop 2024-08-31T06:16:42.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.742+0000 7f60c5263740 1 -- 172.21.15.16:0/989414992 shutdown_connections 2024-08-31T06:16:42.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.742+0000 7f60c5263740 1 --2- 172.21.15.16:0/989414992 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x563d571f89f0 0x563d575fdbc0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:42.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.742+0000 7f60c5263740 1 --2- 172.21.15.16:0/989414992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d5723a070 0x563d575f1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:42.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.742+0000 7f60c5263740 1 --2- 172.21.15.16:0/989414992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d57219390 0x563d575ebaf0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:42.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.742+0000 7f60c5263740 1 --2- 172.21.15.16:0/989414992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d572111b0 0x563d575eb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:42.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.742+0000 7f60c5263740 1 -- 172.21.15.16:0/989414992 >> 172.21.15.16:0/989414992 conn(0x563d5722e460 msgr2=0x563d575e2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:42.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.742+0000 7f60a1600640 0 client.17421 ms_handle_reset on v2:172.21.15.16:6838/219664460 2024-08-31T06:16:42.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.742+0000 7f60c5263740 1 -- 172.21.15.16:0/989414992 shutdown_connections 2024-08-31T06:16:42.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.743+0000 7f60c5263740 1 -- 172.21.15.16:0/989414992 wait complete. 2024-08-31T06:16:42.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.743+0000 7f60c5263740 20 client.17421 trim_cache size 0 max 16384 2024-08-31T06:16:42.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.770+0000 7f60c5263740 1 Processor -- start 2024-08-31T06:16:42.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.770+0000 7f60c5263740 1 -- start start 2024-08-31T06:16:42.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.771+0000 7f60c5263740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d572111b0 0x563d575f12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:42.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.771+0000 7f60c5263740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d57219390 0x563d575e9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:42.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.771+0000 7f60c5263740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d5723a070 0x563d575ea0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:42.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.771+0000 7f60c5263740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f60b81afc90 con 0x563d5723a070 2024-08-31T06:16:42.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.771+0000 7f60c5263740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f60b81ad500 con 0x563d572111b0 2024-08-31T06:16:42.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.771+0000 7f60c5263740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f60b81ab390 con 0x563d57219390 2024-08-31T06:16:42.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.771+0000 7f60b7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d572111b0 0x563d575f12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:42.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.771+0000 7f60b7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d572111b0 0x563d575f12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:48292/0 (socket says 172.21.15.16:48292) 2024-08-31T06:16:42.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.771+0000 7f60b7400640 1 -- 172.21.15.16:0/34465880 learned_addr learned my addr 172.21.15.16:0/34465880 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:42.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.772+0000 7f60b7400640 1 -- 172.21.15.16:0/34465880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d57219390 msgr2=0x563d575e9b90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:16:42.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.772+0000 7f60b7400640 1 --2- 172.21.15.16:0/34465880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d57219390 0x563d575e9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:42.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.772+0000 7f60b7400640 1 -- 172.21.15.16:0/34465880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d5723a070 msgr2=0x563d575ea0d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:16:42.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.773+0000 7f60b7e00640 1 --2- 172.21.15.16:0/34465880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d5723a070 0x563d575ea0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:42.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.773+0000 7f60b7400640 1 --2- 172.21.15.16:0/34465880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d5723a070 0x563d575ea0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:42.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.773+0000 7f60b7400640 1 -- 172.21.15.16:0/34465880 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d575e1ba0 con 0x563d572111b0 2024-08-31T06:16:42.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.773+0000 7f60b7e00640 1 --2- 172.21.15.16:0/34465880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d5723a070 0x563d575ea0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:16:42.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.773+0000 7f60b7400640 1 --2- 172.21.15.16:0/34465880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d572111b0 0x563d575f12c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f60a8000c00 tx=0x7f60a803e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d43e88bf5a2a534 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:42.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.774+0000 7f60a3e00640 1 -- 172.21.15.16:0/34465880 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a8035040 con 0x563d572111b0 2024-08-31T06:16:42.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.774+0000 7f60a3e00640 1 -- 172.21.15.16:0/34465880 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60a8032b60 con 0x563d572111b0 2024-08-31T06:16:42.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.774+0000 7f60a3e00640 1 -- 172.21.15.16:0/34465880 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a805baf0 con 0x563d572111b0 2024-08-31T06:16:42.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.775+0000 7f60c5263740 1 -- 172.21.15.16:0/34465880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d572111b0 msgr2=0x563d575f12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:42.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.775+0000 7f60c5263740 1 --2- 172.21.15.16:0/34465880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d572111b0 0x563d575f12c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f60a8000c00 tx=0x7f60a803e000 comp rx=0 tx=0).stop 2024-08-31T06:16:42.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.775+0000 7f60c5263740 1 -- 172.21.15.16:0/34465880 shutdown_connections 2024-08-31T06:16:42.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.775+0000 7f60c5263740 1 --2- 172.21.15.16:0/34465880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d57219390 0x563d575e9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:42.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.775+0000 7f60c5263740 1 --2- 172.21.15.16:0/34465880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d5723a070 0x563d575ea0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:42.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.775+0000 7f60c5263740 1 --2- 172.21.15.16:0/34465880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d572111b0 0x563d575f12c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:42.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.775+0000 7f60c5263740 1 -- 172.21.15.16:0/34465880 >> 172.21.15.16:0/34465880 conn(0x563d5722e460 msgr2=0x563d575e64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:42.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.775+0000 7f60c5263740 1 -- 172.21.15.16:0/34465880 shutdown_connections 2024-08-31T06:16:42.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.775+0000 7f60c5263740 1 -- 172.21.15.16:0/34465880 wait complete. 2024-08-31T06:16:42.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.776+0000 7f60c5263740 1 -- start start 2024-08-31T06:16:42.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.776+0000 7f60c5263740 1 Processor -- start 2024-08-31T06:16:42.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.778+0000 7f60c5263740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d572111b0 0x563d57663450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:42.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.778+0000 7f60c5263740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d57219390 0x563d57663990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:42.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.778+0000 7f60c5263740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d5723a070 0x563d57601540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:42.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.778+0000 7f60b7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d572111b0 0x563d57663450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:42.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.779+0000 7f60b7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d572111b0 0x563d57663450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51386/0 (socket says 172.21.15.16:51386) 2024-08-31T06:16:42.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.779+0000 7f60b6a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d57219390 0x563d57663990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:42.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.779+0000 7f60b6a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d57219390 0x563d57663990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:52770/0 (socket says 172.21.15.16:52770) 2024-08-31T06:16:42.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.779+0000 7f60b7400640 1 -- 172.21.15.16:0/3401847618 learned_addr learned my addr 172.21.15.16:0/3401847618 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:42.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.779+0000 7f60c5263740 1 -- 172.21.15.16:0/3401847618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f60b81ab9e0 con 0x563d572111b0 2024-08-31T06:16:42.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.780+0000 7f60c5263740 1 -- 172.21.15.16:0/3401847618 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f60b81ace30 con 0x563d5723a070 2024-08-31T06:16:42.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.780+0000 7f60c5263740 1 -- 172.21.15.16:0/3401847618 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f60b81ac090 con 0x563d57219390 2024-08-31T06:16:42.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.780+0000 7f60b7e00640 1 --2- 172.21.15.16:0/3401847618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d5723a070 0x563d57601540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:42.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.780+0000 7f60b7400640 1 -- 172.21.15.16:0/3401847618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d57219390 msgr2=0x563d57663990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:42.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.780+0000 7f60c5263740 10 client.0 _ll_register_callbacks cb 0x7f60a40b3250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:16:42.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.780+0000 7f60b7400640 1 --2- 172.21.15.16:0/3401847618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d57219390 0x563d57663990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:42.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.780+0000 7f60b7400640 1 -- 172.21.15.16:0/3401847618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d5723a070 msgr2=0x563d57601540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:42.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.780+0000 7f60b7400640 1 --2- 172.21.15.16:0/3401847618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d5723a070 0x563d57601540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:42.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.780+0000 7f60b7400640 1 -- 172.21.15.16:0/3401847618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d57664230 con 0x563d572111b0 2024-08-31T06:16:42.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.780+0000 7f60b6a00640 1 --2- 172.21.15.16:0/3401847618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d57219390 0x563d57663990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:16:42.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.781+0000 7f60c5263740 1 -- 172.21.15.16:0/3401847618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60a8032b60 con 0x563d572111b0 2024-08-31T06:16:42.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.781+0000 7f60b7400640 1 --2- 172.21.15.16:0/3401847618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d572111b0 0x563d57663450 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f60a802a970 tx=0x7f60a8032e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15db987cb0826335 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:42.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.781+0000 7f60a1600640 10 client.17424 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:16:42.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.781+0000 7f60a1600640 1 -- 172.21.15.16:0/3401847618 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a8035040 con 0x563d572111b0 2024-08-31T06:16:42.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.781+0000 7f60a1600640 1 -- 172.21.15.16:0/3401847618 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8470+0+0 (secure 0 0 0) 0x7f60a8032800 con 0x563d572111b0 2024-08-31T06:16:42.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.781+0000 7f60a1600640 1 -- 172.21.15.16:0/3401847618 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f60a805b440 con 0x563d572111b0 2024-08-31T06:16:42.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.781+0000 7f60c5263740 10 client.17424 fetch_fsmap learned FSMap version 43 2024-08-31T06:16:42.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.781+0000 7f60c5263740 1 -- 172.21.15.16:0/3401847618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=43}) -- 0x563d576797c0 con 0x563d572111b0 2024-08-31T06:16:42.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.782+0000 7f60a1600640 1 -- 172.21.15.16:0/3401847618 <== mon.0 v2:172.21.15.16:3300/0 4 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f60a8059020 con 0x563d572111b0 2024-08-31T06:16:42.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.782+0000 7f60c5263740 10 client.17424 fetch_fsmap finished waiting for FSMap version 43 2024-08-31T06:16:42.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.782+0000 7f60c5263740 10 client.17424 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-31T06:16:42.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.782+0000 7f60c5263740 20 client.17424 populate_metadata read hostname 'smithi016' 2024-08-31T06:16:42.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.782+0000 7f60c5263740 1 --2- 172.21.15.16:0/3401847618 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x563d571cdda0 0x563d5767bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:42.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.782+0000 7f60c5263740 4 client.17424 mds_command: new command op to 24724 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:16:42.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.782+0000 7f60c5263740 1 -- 172.21.15.16:0/3401847618 --> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563d575541a0 con 0x563d571cdda0 2024-08-31T06:16:42.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.783+0000 7f60b6a00640 1 --2- 172.21.15.16:0/3401847618 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x563d571cdda0 0x563d5767bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:42.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.784+0000 7f60b6a00640 1 --2- 172.21.15.16:0/3401847618 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x563d571cdda0 0x563d5767bdf0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4edc3b8e7e247a2e server_cookie=e8cf3607548ef3f6 in_seq=0 out_seq=0 2024-08-31T06:16:42.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.784+0000 7f60a1600640 10 client.17424 ms_handle_connect on v2:172.21.15.16:6838/219664460 2024-08-31T06:16:42.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.785+0000 7f60a1600640 1 -- 172.21.15.16:0/3401847618 <== mds.1 v2:172.21.15.16:6838/219664460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563d575541a0 con 0x563d571cdda0 2024-08-31T06:16:42.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.785+0000 7f60a1600640 10 client.17424 handle_command_reply: tid=0 2024-08-31T06:16:42.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.785+0000 7f60c5263740 1 client.17424 shutdown 2024-08-31T06:16:42.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.786+0000 7f60c5263740 1 -- 172.21.15.16:0/3401847618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d572111b0 msgr2=0x563d57663450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:42.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.786+0000 7f60c5263740 1 --2- 172.21.15.16:0/3401847618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d572111b0 0x563d57663450 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f60a802a970 tx=0x7f60a8032e70 comp rx=0 tx=0).stop 2024-08-31T06:16:42.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.786+0000 7f60c5263740 1 -- 172.21.15.16:0/3401847618 shutdown_connections 2024-08-31T06:16:42.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.786+0000 7f60c5263740 1 --2- 172.21.15.16:0/3401847618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d57219390 0x563d57663990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:42.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.786+0000 7f60c5263740 1 --2- 172.21.15.16:0/3401847618 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x563d571cdda0 0x563d5767bdf0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:42.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.786+0000 7f60c5263740 1 --2- 172.21.15.16:0/3401847618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d5723a070 0x563d57601540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:42.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.786+0000 7f60c5263740 1 --2- 172.21.15.16:0/3401847618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d572111b0 0x563d57663450 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:42.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.786+0000 7f60c5263740 1 -- 172.21.15.16:0/3401847618 >> 172.21.15.16:0/3401847618 conn(0x563d5722e460 msgr2=0x563d575e9ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:42.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.786+0000 7f60a1600640 0 client.17424 ms_handle_reset on v2:172.21.15.16:6838/219664460 2024-08-31T06:16:42.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.787+0000 7f60c5263740 1 -- 172.21.15.16:0/3401847618 shutdown_connections 2024-08-31T06:16:42.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.787+0000 7f60c5263740 1 -- 172.21.15.16:0/3401847618 wait complete. 2024-08-31T06:16:42.790 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:16:42.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.787+0000 7f60c5263740 20 client.17424 trim_cache size 0 max 16384 2024-08-31T06:16:42.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.788+0000 7f60be600640 1 -- 172.21.15.16:0/677173816 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f609c0b4e30 msgr2=0x7f609c0b7340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:42.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.788+0000 7f60be600640 1 --2- 172.21.15.16:0/677173816 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f609c0b4e30 0x7f609c0b7340 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f60a8005f70 tx=0x7f60a802a900 comp rx=0 tx=0).stop 2024-08-31T06:16:42.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.788+0000 7f60be600640 1 -- 172.21.15.16:0/677173816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60b8102280 msgr2=0x7f60b819ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:42.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.788+0000 7f60be600640 1 --2- 172.21.15.16:0/677173816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60b8102280 0x7f60b819ca10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f60a4005fd0 tx=0x7f60a400dd20 comp rx=0 tx=0).stop 2024-08-31T06:16:42.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.788+0000 7f60be600640 1 -- 172.21.15.16:0/677173816 shutdown_connections 2024-08-31T06:16:42.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.788+0000 7f60be600640 1 --2- 172.21.15.16:0/677173816 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f609c0b4e30 0x7f609c0b7340 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:42.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.788+0000 7f60be600640 1 --2- 172.21.15.16:0/677173816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60b8102280 0x7f60b819ca10 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:42.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.788+0000 7f60be600640 1 --2- 172.21.15.16:0/677173816 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60b810fd90 0x7f60b81a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:42.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.788+0000 7f60be600640 1 --2- 172.21.15.16:0/677173816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60b81018a0 0x7f60b819c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:42.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.788+0000 7f60be600640 1 -- 172.21.15.16:0/677173816 >> 172.21.15.16:0/677173816 conn(0x7f60b80775a0 msgr2=0x7f60b81104d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:42.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.789+0000 7f60be600640 1 -- 172.21.15.16:0/677173816 shutdown_connections 2024-08-31T06:16:42.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:42.789+0000 7f60be600640 1 -- 172.21.15.16:0/677173816 wait complete. 2024-08-31T06:16:42.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:42 smithi155 ceph-mon[21742]: pgmap v656: 97 pgs: 97 active+clean; 3.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.9 MiB/s wr, 610 op/s 2024-08-31T06:16:42.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:42 smithi016 ceph-mon[17194]: pgmap v656: 97 pgs: 97 active+clean; 3.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.9 MiB/s wr, 610 op/s 2024-08-31T06:16:42.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:42 smithi184 ceph-mon[22331]: pgmap v656: 97 pgs: 97 active+clean; 3.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.9 MiB/s wr, 610 op/s 2024-08-31T06:16:43.727 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T06:16:44.373 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:16:44.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:44 smithi155 ceph-mon[21742]: pgmap v657: 97 pgs: 97 active+clean; 3.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.6 MiB/s wr, 592 op/s 2024-08-31T06:16:44.905 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:44 smithi016 ceph-mon[17194]: pgmap v657: 97 pgs: 97 active+clean; 3.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.6 MiB/s wr, 592 op/s 2024-08-31T06:16:44.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:44 smithi184 ceph-mon[22331]: pgmap v657: 97 pgs: 97 active+clean; 3.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.6 MiB/s wr, 592 op/s 2024-08-31T06:16:45.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.237+0000 7fce25400640 1 -- 172.21.15.16:0/4097041421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce20109250 msgr2=0x7fce20109660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:45.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.237+0000 7fce25400640 1 --2- 172.21.15.16:0/4097041421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce20109250 0x7fce20109660 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fce18009ca0 tx=0x7fce1802a8d0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.242+0000 7fce25400640 1 -- 172.21.15.16:0/4097041421 shutdown_connections 2024-08-31T06:16:45.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.242+0000 7fce25400640 1 --2- 172.21.15.16:0/4097041421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce20109ba0 0x7fce201103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.242+0000 7fce25400640 1 --2- 172.21.15.16:0/4097041421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce20109250 0x7fce20109660 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.242+0000 7fce25400640 1 --2- 172.21.15.16:0/4097041421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce20103910 0x7fce20103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.242+0000 7fce25400640 1 -- 172.21.15.16:0/4097041421 >> 172.21.15.16:0/4097041421 conn(0x7fce200ff380 msgr2=0x7fce201017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:45.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.242+0000 7fce25400640 1 -- 172.21.15.16:0/4097041421 shutdown_connections 2024-08-31T06:16:45.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.242+0000 7fce25400640 1 -- 172.21.15.16:0/4097041421 wait complete. 2024-08-31T06:16:45.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.244+0000 7fce25400640 1 Processor -- start 2024-08-31T06:16:45.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.244+0000 7fce25400640 1 -- start start 2024-08-31T06:16:45.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.245+0000 7fce25400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce20103910 0x7fce20198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:45.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.245+0000 7fce25400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce20109250 0x7fce20199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:45.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.245+0000 7fce25400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce20109ba0 0x7fce2019fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:45.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.245+0000 7fce25400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fce20113860 con 0x7fce20109ba0 2024-08-31T06:16:45.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.245+0000 7fce25400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fce201136e0 con 0x7fce20103910 2024-08-31T06:16:45.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.245+0000 7fce25400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fce201139e0 con 0x7fce20109250 2024-08-31T06:16:45.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.245+0000 7fce1e800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce20103910 0x7fce20198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:45.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.245+0000 7fce1de00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce20109250 0x7fce20199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:45.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.245+0000 7fce1e800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce20103910 0x7fce20198cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:48302/0 (socket says 172.21.15.16:48302) 2024-08-31T06:16:45.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.245+0000 7fce1e800640 1 -- 172.21.15.16:0/4163284638 learned_addr learned my addr 172.21.15.16:0/4163284638 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:45.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.246+0000 7fce1f200640 1 --2- 172.21.15.16:0/4163284638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce20109ba0 0x7fce2019fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:45.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.246+0000 7fce1e800640 1 -- 172.21.15.16:0/4163284638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce20109250 msgr2=0x7fce20199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:45.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.246+0000 7fce1e800640 1 --2- 172.21.15.16:0/4163284638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce20109250 0x7fce20199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.246+0000 7fce1e800640 1 -- 172.21.15.16:0/4163284638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce20109ba0 msgr2=0x7fce2019fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:45.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.246+0000 7fce1e800640 1 --2- 172.21.15.16:0/4163284638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce20109ba0 0x7fce2019fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.246+0000 7fce1e800640 1 -- 172.21.15.16:0/4163284638 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce201a0270 con 0x7fce20103910 2024-08-31T06:16:45.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.247+0000 7fce1e800640 1 --2- 172.21.15.16:0/4163284638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce20103910 0x7fce20198cd0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fce0c00c890 tx=0x7fce0c00cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:45.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.247+0000 7fce17400640 1 -- 172.21.15.16:0/4163284638 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0c004220 con 0x7fce20103910 2024-08-31T06:16:45.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.248+0000 7fce25400640 1 -- 172.21.15.16:0/4163284638 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce201a0560 con 0x7fce20103910 2024-08-31T06:16:45.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.248+0000 7fce17400640 1 -- 172.21.15.16:0/4163284638 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce0c0043c0 con 0x7fce20103910 2024-08-31T06:16:45.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.248+0000 7fce25400640 1 -- 172.21.15.16:0/4163284638 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce201a0a60 con 0x7fce20103910 2024-08-31T06:16:45.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.248+0000 7fce17400640 1 -- 172.21.15.16:0/4163284638 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0c004a00 con 0x7fce20103910 2024-08-31T06:16:45.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.254+0000 7fce17400640 1 -- 172.21.15.16:0/4163284638 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fce0c004ba0 con 0x7fce20103910 2024-08-31T06:16:45.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.257+0000 7fce17400640 1 --2- 172.21.15.16:0/4163284638 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fce040b4d20 0x7fce040b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:45.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.258+0000 7fce2c05b740 1 Processor -- start 2024-08-31T06:16:45.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.258+0000 7fce17400640 1 -- 172.21.15.16:0/4163284638 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8470+0+0 (secure 0 0 0) 0x7fce0c0e5820 con 0x7fce20103910 2024-08-31T06:16:45.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.258+0000 7fce1de00640 1 --2- 172.21.15.16:0/4163284638 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fce040b4d20 0x7fce040b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:45.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.259+0000 7fce2c05b740 1 -- start start 2024-08-31T06:16:45.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.260+0000 7fce1de00640 1 --2- 172.21.15.16:0/4163284638 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fce040b4d20 0x7fce040b7230 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fce18005e30 tx=0x7fce18005fd0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:45.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.262+0000 7fce2c05b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653bcf52070 0x5653bd2f7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:45.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.262+0000 7fce2c05b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653bcf291b0 0x5653bd2f7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:45.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.262+0000 7fce2c05b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653bcf31390 0x5653bd2f8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:45.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.262+0000 7fce2c05b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5653bd26e7b0 con 0x5653bcf291b0 2024-08-31T06:16:45.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.262+0000 7fce2c05b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5653bd25dd40 con 0x5653bcf52070 2024-08-31T06:16:45.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.262+0000 7fce2c05b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5653bd261cf0 con 0x5653bcf31390 2024-08-31T06:16:45.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.263+0000 7fce1e800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653bcf52070 0x5653bd2f7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:45.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.263+0000 7fce1e800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653bcf52070 0x5653bd2f7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:48310/0 (socket says 172.21.15.16:48310) 2024-08-31T06:16:45.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.263+0000 7fce1e800640 1 -- 172.21.15.16:0/1061372774 learned_addr learned my addr 172.21.15.16:0/1061372774 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:45.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.263+0000 7fce1f200640 1 --2- 172.21.15.16:0/1061372774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653bcf31390 0x5653bd2f8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:45.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.264+0000 7fce1de00640 1 --2- 172.21.15.16:0/1061372774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653bcf291b0 0x5653bd2f7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:45.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.264+0000 7fce1e800640 1 -- 172.21.15.16:0/1061372774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653bcf31390 msgr2=0x5653bd2f8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:45.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.265+0000 7fce1e800640 1 --2- 172.21.15.16:0/1061372774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653bcf31390 0x5653bd2f8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.265+0000 7fce1e800640 1 -- 172.21.15.16:0/1061372774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653bcf291b0 msgr2=0x5653bd2f7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:45.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.265+0000 7fce1e800640 1 --2- 172.21.15.16:0/1061372774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653bcf291b0 0x5653bd2f7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.265+0000 7fce1e800640 1 -- 172.21.15.16:0/1061372774 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653bcfbd650 con 0x5653bcf52070 2024-08-31T06:16:45.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.265+0000 7fce1f200640 1 --2- 172.21.15.16:0/1061372774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653bcf31390 0x5653bd2f8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:16:45.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.266+0000 7fce1e800640 1 --2- 172.21.15.16:0/1061372774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653bcf52070 0x5653bd2f7990 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fce0c005fd0 tx=0x7fce0c0ab990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c551ece775ec4c4f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:45.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.266+0000 7fce0b400640 1 -- 172.21.15.16:0/1061372774 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0c0abf40 con 0x5653bcf52070 2024-08-31T06:16:45.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.268+0000 7fce0b400640 1 -- 172.21.15.16:0/1061372774 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce0c0b07b0 con 0x5653bcf52070 2024-08-31T06:16:45.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.268+0000 7fce0b400640 1 -- 172.21.15.16:0/1061372774 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0c0e33b0 con 0x5653bcf52070 2024-08-31T06:16:45.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.273+0000 7fce2c05b740 1 -- 172.21.15.16:0/1061372774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653bcf52070 msgr2=0x5653bd2f7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:45.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.273+0000 7fce2c05b740 1 --2- 172.21.15.16:0/1061372774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653bcf52070 0x5653bd2f7990 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fce0c005fd0 tx=0x7fce0c0ab990 comp rx=0 tx=0).stop 2024-08-31T06:16:45.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.274+0000 7fce2c05b740 1 -- 172.21.15.16:0/1061372774 shutdown_connections 2024-08-31T06:16:45.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.274+0000 7fce2c05b740 1 --2- 172.21.15.16:0/1061372774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653bcf31390 0x5653bd2f8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.274+0000 7fce2c05b740 1 --2- 172.21.15.16:0/1061372774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653bcf291b0 0x5653bd2f7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.274+0000 7fce2c05b740 1 --2- 172.21.15.16:0/1061372774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653bcf52070 0x5653bd2f7990 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.274+0000 7fce2c05b740 1 -- 172.21.15.16:0/1061372774 >> 172.21.15.16:0/1061372774 conn(0x5653bcf46460 msgr2=0x5653bd2f5880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:45.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.274+0000 7fce2c05b740 1 -- 172.21.15.16:0/1061372774 shutdown_connections 2024-08-31T06:16:45.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.275+0000 7fce2c05b740 1 -- 172.21.15.16:0/1061372774 wait complete. 2024-08-31T06:16:45.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.275+0000 7fce2c05b740 1 -- start start 2024-08-31T06:16:45.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.276+0000 7fce2c05b740 1 Processor -- start 2024-08-31T06:16:45.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.278+0000 7fce2c05b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653bcf291b0 0x5653bd303690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:45.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.279+0000 7fce2c05b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653bcf31390 0x5653bd303c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:45.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.279+0000 7fce2c05b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653bcf52070 0x5653bd309430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:45.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.279+0000 7fce1e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653bcf291b0 0x5653bd303690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:45.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.279+0000 7fce1e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653bcf291b0 0x5653bd303690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51440/0 (socket says 172.21.15.16:51440) 2024-08-31T06:16:45.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.280+0000 7fce1de00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653bcf31390 0x5653bd303c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:45.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.280+0000 7fce1de00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653bcf31390 0x5653bd303c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:48332/0 (socket says 172.21.15.16:48332) 2024-08-31T06:16:45.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.280+0000 7fce1f200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653bcf52070 0x5653bd309430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:45.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.280+0000 7fce1f200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653bcf52070 0x5653bd309430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:52812/0 (socket says 172.21.15.16:52812) 2024-08-31T06:16:45.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.280+0000 7fce2c05b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5653bd0c0700 con 0x5653bcf291b0 2024-08-31T06:16:45.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.281+0000 7f0943e00640 1 -- 172.21.15.16:0/2200641193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f093c10fc00 msgr2=0x7f093c110010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:45.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.280+0000 7fce1e800640 1 -- 172.21.15.16:0/1289542928 learned_addr learned my addr 172.21.15.16:0/1289542928 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:45.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.280+0000 7fce2c05b740 1 -- 172.21.15.16:0/1289542928 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5653bd1e4cf0 con 0x5653bcf31390 2024-08-31T06:16:45.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.280+0000 7fce2c05b740 1 -- 172.21.15.16:0/1289542928 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5653bd306110 con 0x5653bcf52070 2024-08-31T06:16:45.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.281+0000 7f0943e00640 1 --2- 172.21.15.16:0/2200641193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f093c10fc00 0x7f093c110010 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0928009370 tx=0x7f092802a500 comp rx=0 tx=0).stop 2024-08-31T06:16:45.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.281+0000 7fce2c05b740 10 client.0 _ll_register_callbacks cb 0x5653bd20a470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:16:45.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.282+0000 7f0943e00640 1 -- 172.21.15.16:0/2200641193 shutdown_connections 2024-08-31T06:16:45.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.282+0000 7f0943e00640 1 --2- 172.21.15.16:0/2200641193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f093c110550 0x7f093c116d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.286 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.282+0000 7f0943e00640 1 --2- 172.21.15.16:0/2200641193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f093c10fc00 0x7f093c110010 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.286 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.282+0000 7f0943e00640 1 --2- 172.21.15.16:0/2200641193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f093c10a400 0x7f093c10a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.286 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.282+0000 7f0943e00640 1 -- 172.21.15.16:0/2200641193 >> 172.21.15.16:0/2200641193 conn(0x7f093c070fc0 msgr2=0x7f093c073af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:45.286 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.282+0000 7f0943e00640 1 -- 172.21.15.16:0/2200641193 shutdown_connections 2024-08-31T06:16:45.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.281+0000 7fce1f200640 1 -- 172.21.15.16:0/1289542928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653bcf31390 msgr2=0x5653bd303c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:45.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.281+0000 7fce1f200640 1 --2- 172.21.15.16:0/1289542928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653bcf31390 0x5653bd303c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.281+0000 7fce1f200640 1 -- 172.21.15.16:0/1289542928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653bcf291b0 msgr2=0x5653bd303690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:45.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.281+0000 7fce1f200640 1 --2- 172.21.15.16:0/1289542928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653bcf291b0 0x5653bd303690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.282+0000 7fce1f200640 1 -- 172.21.15.16:0/1289542928 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653bd2258b0 con 0x5653bcf52070 2024-08-31T06:16:45.287 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.282+0000 7f0943e00640 1 -- 172.21.15.16:0/2200641193 wait complete. 2024-08-31T06:16:45.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.282+0000 7fce1de00640 1 --2- 172.21.15.16:0/1289542928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653bcf31390 0x5653bd303c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:16:45.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.282+0000 7fce1f200640 1 --2- 172.21.15.16:0/1289542928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653bcf52070 0x5653bd309430 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fce10012700 tx=0x7fce10012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d5b98543b6e5a3a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:45.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.282+0000 7fcdffe00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:16:45.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.283+0000 7fcdffe00640 1 -- 172.21.15.16:0/1289542928 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce10011bf0 con 0x5653bcf52070 2024-08-31T06:16:45.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.283+0000 7fcdffe00640 1 -- 172.21.15.16:0/1289542928 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8470+0+0 (secure 0 0 0) 0x7fce10011d90 con 0x5653bcf52070 2024-08-31T06:16:45.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.284+0000 7fce2c05b740 1 -- 172.21.15.16:0/1289542928 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce0c0b07b0 con 0x5653bcf52070 2024-08-31T06:16:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.285+0000 7f0943e00640 1 Processor -- start 2024-08-31T06:16:45.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.284+0000 7fcdffe00640 1 -- 172.21.15.16:0/1289542928 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fce10004030 con 0x5653bcf52070 2024-08-31T06:16:45.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.285+0000 7fce2c05b740 10 client.27715 fetch_fsmap learned FSMap version 43 2024-08-31T06:16:45.289 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.285+0000 7f0943e00640 1 -- start start 2024-08-31T06:16:45.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.285+0000 7fce2c05b740 1 -- 172.21.15.16:0/1289542928 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5653bd3122d0 con 0x5653bcf52070 2024-08-31T06:16:45.289 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.285+0000 7f0943e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f093c10a400 0x7f093c114a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:45.289 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.285+0000 7f0943e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f093c10fc00 0x7f093c110d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:45.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.285+0000 7fcdffe00640 1 -- 172.21.15.16:0/1289542928 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce0c0b07b0 con 0x5653bcf52070 2024-08-31T06:16:45.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.286+0000 7f0943e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f093c110550 0x7f093c111480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:45.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.286+0000 7f0943e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f093c119e10 con 0x7f093c110550 2024-08-31T06:16:45.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.286+0000 7f0943e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f093c119c90 con 0x7f093c10a400 2024-08-31T06:16:45.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.286+0000 7f0943e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f093c119f90 con 0x7f093c10fc00 2024-08-31T06:16:45.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.286+0000 7f0941400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f093c10a400 0x7f093c114a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:45.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.286+0000 7f0941400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f093c10a400 0x7f093c114a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:48338/0 (socket says 172.21.15.16:48338) 2024-08-31T06:16:45.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.286+0000 7f0941400640 1 -- 172.21.15.16:0/871043720 learned_addr learned my addr 172.21.15.16:0/871043720 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:45.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.287+0000 7f0940a00640 1 --2- 172.21.15.16:0/871043720 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f093c10fc00 0x7f093c110d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:45.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.287+0000 7f0941e00640 1 --2- 172.21.15.16:0/871043720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f093c110550 0x7f093c111480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:45.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.286+0000 7fcdffe00640 1 -- 172.21.15.16:0/1289542928 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fce1001c460 con 0x5653bcf52070 2024-08-31T06:16:45.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.286+0000 7fce2c05b740 10 client.27715 fetch_fsmap finished waiting for FSMap version 43 2024-08-31T06:16:45.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.286+0000 7fce2c05b740 10 client.27715 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:16:45.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.286+0000 7fce2c05b740 20 client.27715 populate_metadata read hostname 'smithi016' 2024-08-31T06:16:45.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.286+0000 7fce2c05b740 1 --2- 172.21.15.16:0/1289542928 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5653bcf109f0 0x5653bd315790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:45.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.286+0000 7fce2c05b740 4 client.27715 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:16:45.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.286+0000 7fce2c05b740 1 -- 172.21.15.16:0/1289542928 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653bd270700 con 0x5653bcf109f0 2024-08-31T06:16:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.287+0000 7f0941400640 1 -- 172.21.15.16:0/871043720 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f093c10fc00 msgr2=0x7f093c110d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.287+0000 7f0941400640 1 --2- 172.21.15.16:0/871043720 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f093c10fc00 0x7f093c110d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.287+0000 7f0941400640 1 -- 172.21.15.16:0/871043720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f093c110550 msgr2=0x7f093c111480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.287+0000 7f0941400640 1 --2- 172.21.15.16:0/871043720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f093c110550 0x7f093c111480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.287+0000 7fce1e800640 1 --2- 172.21.15.16:0/1289542928 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5653bcf109f0 0x5653bd315790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:45.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.287+0000 7f0941400640 1 -- 172.21.15.16:0/871043720 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f093c1b5170 con 0x7f093c10a400 2024-08-31T06:16:45.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.288+0000 7f0941400640 1 --2- 172.21.15.16:0/871043720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f093c10a400 0x7f093c114a80 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f093400dfd0 tx=0x7f093400b630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:45.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.288+0000 7f093a000640 1 -- 172.21.15.16:0/871043720 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093401b020 con 0x7f093c10a400 2024-08-31T06:16:45.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.288+0000 7f093a000640 1 -- 172.21.15.16:0/871043720 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0934003e20 con 0x7f093c10a400 2024-08-31T06:16:45.293 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.288+0000 7f093a000640 1 -- 172.21.15.16:0/871043720 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0934017650 con 0x7f093c10a400 2024-08-31T06:16:45.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.288+0000 7fcdffe00640 10 client.27715 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:16:45.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.288+0000 7f0943e00640 1 -- 172.21.15.16:0/871043720 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f093c1b53a0 con 0x7f093c10a400 2024-08-31T06:16:45.294 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.288+0000 7f0943e00640 1 -- 172.21.15.16:0/871043720 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f093c1b5870 con 0x7f093c10a400 2024-08-31T06:16:45.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.289+0000 7fce1e800640 1 --2- 172.21.15.16:0/1289542928 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5653bcf109f0 0x5653bd315790 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4281fab0bcdc1356 server_cookie=330b2851f6dc555c in_seq=0 out_seq=0 2024-08-31T06:16:45.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.290+0000 7fcdffe00640 1 -- 172.21.15.16:0/1289542928 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5653bd270700 con 0x5653bcf109f0 2024-08-31T06:16:45.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.290+0000 7fcdffe00640 10 client.27715 handle_command_reply: tid=0 2024-08-31T06:16:45.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.290+0000 7fce2c05b740 1 client.27715 shutdown 2024-08-31T06:16:45.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.290+0000 7fce2c05b740 1 -- 172.21.15.16:0/1289542928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653bcf52070 msgr2=0x5653bd309430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:45.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.290+0000 7fce2c05b740 1 --2- 172.21.15.16:0/1289542928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653bcf52070 0x5653bd309430 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fce10012700 tx=0x7fce10012bc0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.291+0000 7fce2c05b740 1 -- 172.21.15.16:0/1289542928 shutdown_connections 2024-08-31T06:16:45.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.291+0000 7fce2c05b740 1 --2- 172.21.15.16:0/1289542928 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5653bcf109f0 0x5653bd315790 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.291+0000 7fce2c05b740 1 --2- 172.21.15.16:0/1289542928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653bcf52070 0x5653bd309430 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.291+0000 7fce2c05b740 1 --2- 172.21.15.16:0/1289542928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653bcf31390 0x5653bd303c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.291+0000 7fce2c05b740 1 --2- 172.21.15.16:0/1289542928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653bcf291b0 0x5653bd303690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.291+0000 7fce2c05b740 1 -- 172.21.15.16:0/1289542928 >> 172.21.15.16:0/1289542928 conn(0x5653bcf46460 msgr2=0x5653bd2fab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:45.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.291+0000 7fcdffe00640 0 client.27715 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:16:45.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.291+0000 7fce2c05b740 1 -- 172.21.15.16:0/1289542928 shutdown_connections 2024-08-31T06:16:45.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.292+0000 7fce2c05b740 1 -- 172.21.15.16:0/1289542928 wait complete. 2024-08-31T06:16:45.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.292+0000 7fce2c05b740 20 client.27715 trim_cache size 0 max 16384 2024-08-31T06:16:45.298 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.296+0000 7f094ae3c740 1 Processor -- start 2024-08-31T06:16:45.300 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.297+0000 7f094ae3c740 1 -- start start 2024-08-31T06:16:45.300 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.298+0000 7f094ae3c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8991ac1b0 0x55e89954f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.298+0000 7f094ae3c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e8991832f0 0x55e89954fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.298+0000 7f094ae3c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e89918b4d0 0x55e8995503a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.298+0000 7f094ae3c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e8994c4f70 con 0x55e8991ac1b0 2024-08-31T06:16:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.298+0000 7f094ae3c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e8994b4500 con 0x55e89918b4d0 2024-08-31T06:16:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.298+0000 7f094ae3c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e8994b84b0 con 0x55e8991832f0 2024-08-31T06:16:45.302 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.299+0000 7f0940a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e89918b4d0 0x55e8995503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.299+0000 7f0940a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e89918b4d0 0x55e8995503a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:48342/0 (socket says 172.21.15.16:48342) 2024-08-31T06:16:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.299+0000 7f0940a00640 1 -- 172.21.15.16:0/2495374645 learned_addr learned my addr 172.21.15.16:0/2495374645 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.301+0000 7f0940a00640 1 -- 172.21.15.16:0/2495374645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e8991832f0 msgr2=0x55e89954fe60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:16:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.301+0000 7f0940a00640 1 --2- 172.21.15.16:0/2495374645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e8991832f0 0x55e89954fe60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.301+0000 7f0940a00640 1 -- 172.21.15.16:0/2495374645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8991ac1b0 msgr2=0x55e89954f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.301+0000 7f0940a00640 1 --2- 172.21.15.16:0/2495374645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8991ac1b0 0x55e89954f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.301+0000 7f0940a00640 1 -- 172.21.15.16:0/2495374645 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e89946ec10 con 0x55e89918b4d0 2024-08-31T06:16:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.302+0000 7f0940a00640 1 --2- 172.21.15.16:0/2495374645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e89918b4d0 0x55e8995503a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0928009370 tx=0x7f0928005140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=635a71c0805c114d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:45.305 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.303+0000 7f093a000640 1 -- 172.21.15.16:0/871043720 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f0934003970 con 0x7f093c10a400 2024-08-31T06:16:45.305 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.303+0000 7f092e600640 1 -- 172.21.15.16:0/2495374645 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0928035030 con 0x55e89918b4d0 2024-08-31T06:16:45.305 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.303+0000 7f092e600640 1 -- 172.21.15.16:0/2495374645 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09280054b0 con 0x55e89918b4d0 2024-08-31T06:16:45.305 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.303+0000 7f092e600640 1 -- 172.21.15.16:0/2495374645 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09280319f0 con 0x55e89918b4d0 2024-08-31T06:16:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.305+0000 7f093a000640 1 --2- 172.21.15.16:0/871043720 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f09240b4eb0 0x7f09240b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:45.310 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.307+0000 7f0941e00640 1 --2- 172.21.15.16:0/871043720 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f09240b4eb0 0x7f09240b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:45.310 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.308+0000 7f094ae3c740 1 -- 172.21.15.16:0/2495374645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e89918b4d0 msgr2=0x55e8995503a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:45.310 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.308+0000 7f094ae3c740 1 --2- 172.21.15.16:0/2495374645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e89918b4d0 0x55e8995503a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0928009370 tx=0x7f0928005140 comp rx=0 tx=0).stop 2024-08-31T06:16:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.308+0000 7f094ae3c740 1 -- 172.21.15.16:0/2495374645 shutdown_connections 2024-08-31T06:16:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.308+0000 7f094ae3c740 1 --2- 172.21.15.16:0/2495374645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e8991832f0 0x55e89954fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.308+0000 7f094ae3c740 1 --2- 172.21.15.16:0/2495374645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e89918b4d0 0x55e8995503a0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.308+0000 7f094ae3c740 1 --2- 172.21.15.16:0/2495374645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8991ac1b0 0x55e89954f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.308+0000 7f094ae3c740 1 -- 172.21.15.16:0/2495374645 >> 172.21.15.16:0/2495374645 conn(0x55e8991a0300 msgr2=0x55e89954d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:45.312 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.310+0000 7f094ae3c740 1 -- 172.21.15.16:0/2495374645 shutdown_connections 2024-08-31T06:16:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.314+0000 7f0941e00640 1 --2- 172.21.15.16:0/871043720 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f09240b4eb0 0x7f09240b73c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f093000b980 tx=0x7f0930005c10 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.315+0000 7f094ae3c740 1 -- 172.21.15.16:0/2495374645 wait complete. 2024-08-31T06:16:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.315+0000 7f094ae3c740 1 -- start start 2024-08-31T06:16:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.316+0000 7f094ae3c740 1 Processor -- start 2024-08-31T06:16:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.314+0000 7f093a000640 1 -- 172.21.15.16:0/871043720 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8470+0+0 (secure 0 0 0) 0x7f09340e5070 con 0x7f093c10a400 2024-08-31T06:16:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.317+0000 7f094ae3c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8991832f0 0x55e89955aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.317+0000 7f094ae3c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e89918b4d0 0x55e89955b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.317+0000 7f094ae3c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8991ac1b0 0x55e8995673c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.318+0000 7f0940a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e89918b4d0 0x55e89955b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.318+0000 7f0940a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e89918b4d0 0x55e89955b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:52830/0 (socket says 172.21.15.16:52830) 2024-08-31T06:16:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.318+0000 7f0941e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8991ac1b0 0x55e8995673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.318+0000 7f0941e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8991ac1b0 0x55e8995673c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:48352/0 (socket says 172.21.15.16:48352) 2024-08-31T06:16:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.318+0000 7f0941400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8991832f0 0x55e89955aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.318+0000 7f0941400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8991832f0 0x55e89955aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51486/0 (socket says 172.21.15.16:51486) 2024-08-31T06:16:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.318+0000 7f0940a00640 1 -- 172.21.15.16:0/2408040021 learned_addr learned my addr 172.21.15.16:0/2408040021 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.318+0000 7f094ae3c740 1 -- 172.21.15.16:0/2408040021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e8993edab0 con 0x55e8991832f0 2024-08-31T06:16:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.318+0000 7f094ae3c740 1 -- 172.21.15.16:0/2408040021 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e89945b320 con 0x55e8991ac1b0 2024-08-31T06:16:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.318+0000 7f094ae3c740 1 -- 172.21.15.16:0/2408040021 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e89955e040 con 0x55e89918b4d0 2024-08-31T06:16:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.318+0000 7f094ae3c740 10 client.0 _ll_register_callbacks cb 0x55e8994616a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:16:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.319+0000 7f0941e00640 1 -- 172.21.15.16:0/2408040021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e89918b4d0 msgr2=0x55e89955b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.319+0000 7f0941e00640 1 --2- 172.21.15.16:0/2408040021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e89918b4d0 0x55e89955b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.319+0000 7f0941e00640 1 -- 172.21.15.16:0/2408040021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8991832f0 msgr2=0x55e89955aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.319+0000 7f0941e00640 1 --2- 172.21.15.16:0/2408040021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8991832f0 0x55e89955aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.319+0000 7f0941e00640 1 -- 172.21.15.16:0/2408040021 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e899217790 con 0x55e8991ac1b0 2024-08-31T06:16:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.320+0000 7f0922a00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:16:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.320+0000 7f0941e00640 1 --2- 172.21.15.16:0/2408040021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8991ac1b0 0x55e8995673c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f093000d710 tx=0x7f0930004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=673f646bc8b7bbb9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.321+0000 7f0922a00640 1 -- 172.21.15.16:0/2408040021 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093001f070 con 0x55e8991ac1b0 2024-08-31T06:16:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.321+0000 7f094ae3c740 1 -- 172.21.15.16:0/2408040021 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09280054b0 con 0x55e8991ac1b0 2024-08-31T06:16:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.321+0000 7f0922a00640 1 -- 172.21.15.16:0/2408040021 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8470+0+0 (secure 0 0 0) 0x7f0930005020 con 0x55e8991ac1b0 2024-08-31T06:16:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.321+0000 7f0922a00640 1 -- 172.21.15.16:0/2408040021 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f093001f070 con 0x55e8991ac1b0 2024-08-31T06:16:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.321+0000 7f094ae3c740 10 client.27722 fetch_fsmap learned FSMap version 43 2024-08-31T06:16:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.321+0000 7f094ae3c740 1 -- 172.21.15.16:0/2408040021 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55e899290620 con 0x55e8991ac1b0 2024-08-31T06:16:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.322+0000 7f0922a00640 1 -- 172.21.15.16:0/2408040021 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09280054b0 con 0x55e8991ac1b0 2024-08-31T06:16:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.322+0000 7f0922a00640 1 -- 172.21.15.16:0/2408040021 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f093000e7c0 con 0x55e8991ac1b0 2024-08-31T06:16:45.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.323+0000 7fce2c05b740 1 Processor -- start 2024-08-31T06:16:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.324+0000 7f094ae3c740 10 client.27722 fetch_fsmap finished waiting for FSMap version 43 2024-08-31T06:16:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.324+0000 7f094ae3c740 10 client.27722 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:16:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.324+0000 7f094ae3c740 20 client.27722 populate_metadata read hostname 'smithi016' 2024-08-31T06:16:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.324+0000 7f094ae3c740 1 --2- 172.21.15.16:0/2408040021 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55e89916ad80 0x55e89956d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.324+0000 7f094ae3c740 4 client.27722 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:16:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.324+0000 7f094ae3c740 1 -- 172.21.15.16:0/2408040021 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8994c6ec0 con 0x55e89916ad80 2024-08-31T06:16:45.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.323+0000 7fce2c05b740 1 -- start start 2024-08-31T06:16:45.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.324+0000 7fce2c05b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653bcf291b0 0x5653bd2f5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:45.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.324+0000 7fce2c05b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653bcf31390 0x5653bd2f90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:45.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.324+0000 7fce1e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653bcf291b0 0x5653bd2f5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:45.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.324+0000 7fce2c05b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653bcf52070 0x5653bd3093a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:45.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.324+0000 7fce2c05b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fce00011130 con 0x5653bcf291b0 2024-08-31T06:16:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.325+0000 7f0941400640 1 --2- 172.21.15.16:0/2408040021 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55e89916ad80 0x55e89956d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:45.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.324+0000 7fce2c05b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fce0000e9a0 con 0x5653bcf31390 2024-08-31T06:16:45.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.324+0000 7fce2c05b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fce0000c810 con 0x5653bcf52070 2024-08-31T06:16:45.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.325+0000 7fce1de00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653bcf31390 0x5653bd2f90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:45.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.325+0000 7fce1de00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653bcf31390 0x5653bd2f90d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:48360/0 (socket says 172.21.15.16:48360) 2024-08-31T06:16:45.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.325+0000 7fce1de00640 1 -- 172.21.15.16:0/706415967 learned_addr learned my addr 172.21.15.16:0/706415967 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:45.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.325+0000 7fce1f200640 1 --2- 172.21.15.16:0/706415967 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653bcf52070 0x5653bd3093a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:45.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.326+0000 7fce1de00640 1 -- 172.21.15.16:0/706415967 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653bcf52070 msgr2=0x5653bd3093a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:45.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.326+0000 7fce1de00640 1 --2- 172.21.15.16:0/706415967 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653bcf52070 0x5653bd3093a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.326+0000 7fce1de00640 1 -- 172.21.15.16:0/706415967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653bcf291b0 msgr2=0x5653bd2f5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:45.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.326+0000 7fce1de00640 1 --2- 172.21.15.16:0/706415967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653bcf291b0 0x5653bd2f5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.326+0000 7fce1de00640 1 -- 172.21.15.16:0/706415967 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653bd301990 con 0x5653bcf31390 2024-08-31T06:16:45.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.327+0000 7fce1de00640 1 --2- 172.21.15.16:0/706415967 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653bcf31390 0x5653bd2f90d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fce18009ca0 tx=0x7fce1803f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7be0829fb1f3784 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.327+0000 7f0941400640 1 --2- 172.21.15.16:0/2408040021 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55e89916ad80 0x55e89956d190 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bf4f15bec5036ec server_cookie=1d738f79a5d34721 in_seq=0 out_seq=0 2024-08-31T06:16:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.328+0000 7f0922a00640 10 client.27722 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:16:45.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.327+0000 7fce0aa00640 1 -- 172.21.15.16:0/706415967 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce18003970 con 0x5653bcf31390 2024-08-31T06:16:45.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.328+0000 7fce0aa00640 1 -- 172.21.15.16:0/706415967 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce18036040 con 0x5653bcf31390 2024-08-31T06:16:45.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.328+0000 7fce0aa00640 1 -- 172.21.15.16:0/706415967 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce18003970 con 0x5653bcf31390 2024-08-31T06:16:45.332 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.328+0000 7f0922a00640 1 -- 172.21.15.16:0/2408040021 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e8994c6ec0 con 0x55e89916ad80 2024-08-31T06:16:45.332 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.328+0000 7f0922a00640 10 client.27722 handle_command_reply: tid=0 2024-08-31T06:16:45.332 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.329+0000 7f094ae3c740 1 client.27722 shutdown 2024-08-31T06:16:45.332 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.329+0000 7f094ae3c740 1 -- 172.21.15.16:0/2408040021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8991ac1b0 msgr2=0x55e8995673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:45.332 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.329+0000 7f094ae3c740 1 --2- 172.21.15.16:0/2408040021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8991ac1b0 0x55e8995673c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f093000d710 tx=0x7f0930004730 comp rx=0 tx=0).stop 2024-08-31T06:16:45.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.329+0000 7fce2c05b740 1 -- 172.21.15.16:0/706415967 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653bcf31390 msgr2=0x5653bd2f90d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:45.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.329+0000 7fce2c05b740 1 --2- 172.21.15.16:0/706415967 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653bcf31390 0x5653bd2f90d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fce18009ca0 tx=0x7fce1803f000 comp rx=0 tx=0).stop 2024-08-31T06:16:45.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.330+0000 7f094ae3c740 1 -- 172.21.15.16:0/2408040021 shutdown_connections 2024-08-31T06:16:45.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.330+0000 7f094ae3c740 1 --2- 172.21.15.16:0/2408040021 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55e89916ad80 0x55e89956d190 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.330+0000 7f094ae3c740 1 --2- 172.21.15.16:0/2408040021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e89918b4d0 0x55e89955b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.330+0000 7f0922a00640 0 client.27722 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:16:45.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.330+0000 7f094ae3c740 1 --2- 172.21.15.16:0/2408040021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8991ac1b0 0x55e8995673c0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.330+0000 7f094ae3c740 1 --2- 172.21.15.16:0/2408040021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8991832f0 0x55e89955aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.330+0000 7f094ae3c740 1 -- 172.21.15.16:0/2408040021 >> 172.21.15.16:0/2408040021 conn(0x55e8991a0300 msgr2=0x55e89954e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:45.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.330+0000 7fce2c05b740 1 -- 172.21.15.16:0/706415967 shutdown_connections 2024-08-31T06:16:45.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.330+0000 7fce2c05b740 1 --2- 172.21.15.16:0/706415967 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653bcf52070 0x5653bd3093a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.330+0000 7fce2c05b740 1 --2- 172.21.15.16:0/706415967 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653bcf31390 0x5653bd2f90d0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.330+0000 7fce2c05b740 1 --2- 172.21.15.16:0/706415967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653bcf291b0 0x5653bd2f5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.330+0000 7fce2c05b740 1 -- 172.21.15.16:0/706415967 >> 172.21.15.16:0/706415967 conn(0x5653bcf46460 msgr2=0x5653bd2f89c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:45.334 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.330+0000 7f094ae3c740 1 -- 172.21.15.16:0/2408040021 shutdown_connections 2024-08-31T06:16:45.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.330+0000 7fce2c05b740 1 -- 172.21.15.16:0/706415967 shutdown_connections 2024-08-31T06:16:45.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.330+0000 7fce2c05b740 1 -- 172.21.15.16:0/706415967 wait complete. 2024-08-31T06:16:45.334 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.331+0000 7f094ae3c740 1 -- 172.21.15.16:0/2408040021 wait complete. 2024-08-31T06:16:45.335 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.331+0000 7f094ae3c740 20 client.27722 trim_cache size 0 max 16384 2024-08-31T06:16:45.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.331+0000 7fce2c05b740 1 -- start start 2024-08-31T06:16:45.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.331+0000 7fce2c05b740 1 Processor -- start 2024-08-31T06:16:45.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.333+0000 7fce2c05b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653bcf291b0 0x5653bd30bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:45.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.333+0000 7fce2c05b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653bcf31390 0x5653bd30c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:45.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.333+0000 7fce2c05b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653bcf52070 0x5653bd390130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:45.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.333+0000 7fce2c05b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fce0000ce60 con 0x5653bcf52070 2024-08-31T06:16:45.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.333+0000 7fce2c05b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fce0000e2d0 con 0x5653bcf31390 2024-08-31T06:16:45.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.333+0000 7fce2c05b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fce0000d530 con 0x5653bcf291b0 2024-08-31T06:16:45.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.333+0000 7fce2c05b740 10 client.0 _ll_register_callbacks cb 0x7fce10009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:16:45.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.333+0000 7fce1e800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653bcf291b0 0x5653bd30bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:45.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.333+0000 7fce1e800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653bcf291b0 0x5653bd30bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:52844/0 (socket says 172.21.15.16:52844) 2024-08-31T06:16:45.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.333+0000 7fce1e800640 1 -- 172.21.15.16:0/3421254219 learned_addr learned my addr 172.21.15.16:0/3421254219 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:45.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.334+0000 7fce1de00640 1 --2- 172.21.15.16:0/3421254219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653bcf31390 0x5653bd30c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:45.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.334+0000 7fce1e800640 1 -- 172.21.15.16:0/3421254219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653bcf31390 msgr2=0x5653bd30c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:45.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.334+0000 7fce1e800640 1 --2- 172.21.15.16:0/3421254219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653bcf31390 0x5653bd30c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.334+0000 7fce1e800640 1 -- 172.21.15.16:0/3421254219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653bcf52070 msgr2=0x5653bd390130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:16:45.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.334+0000 7fce1f200640 1 --2- 172.21.15.16:0/3421254219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653bcf52070 0x5653bd390130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:45.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.335+0000 7fce1e800640 1 --2- 172.21.15.16:0/3421254219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653bcf52070 0x5653bd390130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.335+0000 7fce1e800640 1 -- 172.21.15.16:0/3421254219 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653bd390670 con 0x5653bcf291b0 2024-08-31T06:16:45.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.335+0000 7fce1f200640 1 --2- 172.21.15.16:0/3421254219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653bcf52070 0x5653bd390130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:16:45.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.335+0000 7fce1e800640 1 --2- 172.21.15.16:0/3421254219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653bcf291b0 0x5653bd30bc00 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fce0c005fd0 tx=0x7fce0c005650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b95f558f5e59f551 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:45.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.335+0000 7fcdffe00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:16:45.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.335+0000 7fcdffe00640 1 -- 172.21.15.16:0/3421254219 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0c0c6060 con 0x5653bcf291b0 2024-08-31T06:16:45.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.335+0000 7fcdffe00640 1 -- 172.21.15.16:0/3421254219 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8470+0+0 (secure 0 0 0) 0x7fce0c0e3220 con 0x5653bcf291b0 2024-08-31T06:16:45.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.336+0000 7fce2c05b740 1 -- 172.21.15.16:0/3421254219 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce18036040 con 0x5653bcf291b0 2024-08-31T06:16:45.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.336+0000 7fcdffe00640 1 -- 172.21.15.16:0/3421254219 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fce0c0cd050 con 0x5653bcf291b0 2024-08-31T06:16:45.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.336+0000 7fce2c05b740 10 client.27727 fetch_fsmap learned FSMap version 43 2024-08-31T06:16:45.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.336+0000 7fce2c05b740 1 -- 172.21.15.16:0/3421254219 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5653bd391850 con 0x5653bcf291b0 2024-08-31T06:16:45.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.337+0000 7fcdffe00640 1 -- 172.21.15.16:0/3421254219 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce18036040 con 0x5653bcf291b0 2024-08-31T06:16:45.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.337+0000 7fcdffe00640 1 -- 172.21.15.16:0/3421254219 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fce0c0b0400 con 0x5653bcf291b0 2024-08-31T06:16:45.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.337+0000 7fce2c05b740 10 client.27727 fetch_fsmap finished waiting for FSMap version 43 2024-08-31T06:16:45.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.337+0000 7fce2c05b740 10 client.27727 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:16:45.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.337+0000 7fce2c05b740 20 client.27727 populate_metadata read hostname 'smithi016' 2024-08-31T06:16:45.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.337+0000 7fce2c05b740 1 --2- 172.21.15.16:0/3421254219 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5653bcee5da0 0x5653bd393af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:45.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.337+0000 7fce2c05b740 4 client.27727 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:16:45.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.337+0000 7fce2c05b740 1 -- 172.21.15.16:0/3421254219 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5653bd2ff040 con 0x5653bcee5da0 2024-08-31T06:16:45.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.337+0000 7fce1de00640 1 --2- 172.21.15.16:0/3421254219 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5653bcee5da0 0x5653bd393af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:45.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.338+0000 7fce1de00640 1 --2- 172.21.15.16:0/3421254219 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5653bcee5da0 0x5653bd393af0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e17cfefe24a4be5 server_cookie=322d5b6db6e62ceb in_seq=0 out_seq=0 2024-08-31T06:16:45.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.338+0000 7fcdffe00640 10 client.27727 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:16:45.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.338+0000 7fcdffe00640 1 -- 172.21.15.16:0/3421254219 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5653bd2ff040 con 0x5653bcee5da0 2024-08-31T06:16:45.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.338+0000 7fcdffe00640 10 client.27727 handle_command_reply: tid=0 2024-08-31T06:16:45.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.338+0000 7fce2c05b740 1 client.27727 shutdown 2024-08-31T06:16:45.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.340+0000 7fce2c05b740 1 -- 172.21.15.16:0/3421254219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653bcf291b0 msgr2=0x5653bd30bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:45.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.340+0000 7fce2c05b740 1 --2- 172.21.15.16:0/3421254219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653bcf291b0 0x5653bd30bc00 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fce0c005fd0 tx=0x7fce0c005650 comp rx=0 tx=0).stop 2024-08-31T06:16:45.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.340+0000 7fce2c05b740 1 -- 172.21.15.16:0/3421254219 shutdown_connections 2024-08-31T06:16:45.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.340+0000 7fce2c05b740 1 --2- 172.21.15.16:0/3421254219 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5653bcee5da0 0x5653bd393af0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.340+0000 7fce2c05b740 1 --2- 172.21.15.16:0/3421254219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653bcf52070 0x5653bd390130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.340+0000 7fce2c05b740 1 --2- 172.21.15.16:0/3421254219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5653bcf31390 0x5653bd30c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.340+0000 7fce2c05b740 1 --2- 172.21.15.16:0/3421254219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653bcf291b0 0x5653bd30bc00 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.340+0000 7fce2c05b740 1 -- 172.21.15.16:0/3421254219 >> 172.21.15.16:0/3421254219 conn(0x5653bcf46460 msgr2=0x5653bd2f5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:45.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.340+0000 7fcdffe00640 0 client.27727 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:16:45.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.341+0000 7fce2c05b740 1 -- 172.21.15.16:0/3421254219 shutdown_connections 2024-08-31T06:16:45.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.341+0000 7fce2c05b740 1 -- 172.21.15.16:0/3421254219 wait complete. 2024-08-31T06:16:45.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.341+0000 7fce2c05b740 20 client.27727 trim_cache size 0 max 16384 2024-08-31T06:16:45.345 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:16:45.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.344+0000 7fce25400640 1 -- 172.21.15.16:0/4163284638 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fce040b4d20 msgr2=0x7fce040b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:45.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.344+0000 7fce25400640 1 --2- 172.21.15.16:0/4163284638 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fce040b4d20 0x7fce040b7230 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fce18005e30 tx=0x7fce18005fd0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.344+0000 7fce25400640 1 -- 172.21.15.16:0/4163284638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce20103910 msgr2=0x7fce20198cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:45.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.344+0000 7fce25400640 1 --2- 172.21.15.16:0/4163284638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce20103910 0x7fce20198cd0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fce0c00c890 tx=0x7fce0c00cd50 comp rx=0 tx=0).stop 2024-08-31T06:16:45.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.344+0000 7fce25400640 1 -- 172.21.15.16:0/4163284638 shutdown_connections 2024-08-31T06:16:45.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.344+0000 7fce25400640 1 --2- 172.21.15.16:0/4163284638 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fce040b4d20 0x7fce040b7230 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.344+0000 7fce25400640 1 --2- 172.21.15.16:0/4163284638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce20109250 0x7fce20199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.344+0000 7fce25400640 1 --2- 172.21.15.16:0/4163284638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce20109ba0 0x7fce2019fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.344+0000 7fce25400640 1 --2- 172.21.15.16:0/4163284638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce20103910 0x7fce20198cd0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.344+0000 7fce25400640 1 -- 172.21.15.16:0/4163284638 >> 172.21.15.16:0/4163284638 conn(0x7fce200ff380 msgr2=0x7fce20108920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:45.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.345+0000 7fce25400640 1 -- 172.21.15.16:0/4163284638 shutdown_connections 2024-08-31T06:16:45.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:45.345+0000 7fce25400640 1 -- 172.21.15.16:0/4163284638 wait complete. 2024-08-31T06:16:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.383+0000 7f094ae3c740 1 Processor -- start 2024-08-31T06:16:45.386 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.383+0000 7f094ae3c740 1 -- start start 2024-08-31T06:16:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.385+0000 7f094ae3c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e8991832f0 0x55e89954c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.386+0000 7f094ae3c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e89918b4d0 0x55e899558a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.386+0000 7f094ae3c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8991ac1b0 0x55e89954ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.386+0000 7f0941400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e8991832f0 0x55e89954c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.386+0000 7f094ae3c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f091c011060 con 0x55e8991ac1b0 2024-08-31T06:16:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.386+0000 7f094ae3c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f091c00e8d0 con 0x55e89918b4d0 2024-08-31T06:16:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.386+0000 7f094ae3c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f091c00c740 con 0x55e8991832f0 2024-08-31T06:16:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.386+0000 7f0941400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e8991832f0 0x55e89954c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:52850/0 (socket says 172.21.15.16:52850) 2024-08-31T06:16:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.386+0000 7f0941400640 1 -- 172.21.15.16:0/3222465612 learned_addr learned my addr 172.21.15.16:0/3222465612 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.386+0000 7f0940a00640 1 --2- 172.21.15.16:0/3222465612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e89918b4d0 0x55e899558a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.387+0000 7f0941e00640 1 --2- 172.21.15.16:0/3222465612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8991ac1b0 0x55e89954ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.387+0000 7f0941400640 1 -- 172.21.15.16:0/3222465612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e89918b4d0 msgr2=0x55e899558a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.388+0000 7f0941400640 1 --2- 172.21.15.16:0/3222465612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e89918b4d0 0x55e899558a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.388+0000 7f0941400640 1 -- 172.21.15.16:0/3222465612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8991ac1b0 msgr2=0x55e89954ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.388+0000 7f0941400640 1 --2- 172.21.15.16:0/3222465612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8991ac1b0 0x55e89954ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.388+0000 7f0941400640 1 -- 172.21.15.16:0/3222465612 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e899551c20 con 0x55e8991832f0 2024-08-31T06:16:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.388+0000 7f0941e00640 1 --2- 172.21.15.16:0/3222465612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8991ac1b0 0x55e89954ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:16:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.388+0000 7f0940a00640 1 --2- 172.21.15.16:0/3222465612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e89918b4d0 0x55e899558a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:16:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.391+0000 7f0941400640 1 --2- 172.21.15.16:0/3222465612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e8991832f0 0x55e89954c6b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0934000f80 tx=0x7f09340a7050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b92f694eca9aa8fb server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.391+0000 7f092dc00640 1 -- 172.21.15.16:0/3222465612 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0934096070 con 0x55e8991832f0 2024-08-31T06:16:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.391+0000 7f092dc00640 1 -- 172.21.15.16:0/3222465612 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09340b4390 con 0x55e8991832f0 2024-08-31T06:16:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.391+0000 7f092dc00640 1 -- 172.21.15.16:0/3222465612 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0934096070 con 0x55e8991832f0 2024-08-31T06:16:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.394+0000 7f094ae3c740 1 -- 172.21.15.16:0/3222465612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e8991832f0 msgr2=0x55e89954c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.395+0000 7f094ae3c740 1 --2- 172.21.15.16:0/3222465612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e8991832f0 0x55e89954c6b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0934000f80 tx=0x7f09340a7050 comp rx=0 tx=0).stop 2024-08-31T06:16:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.395+0000 7f094ae3c740 1 -- 172.21.15.16:0/3222465612 shutdown_connections 2024-08-31T06:16:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.395+0000 7f094ae3c740 1 --2- 172.21.15.16:0/3222465612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8991ac1b0 0x55e89954ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.395+0000 7f094ae3c740 1 --2- 172.21.15.16:0/3222465612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e89918b4d0 0x55e899558a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.395+0000 7f094ae3c740 1 --2- 172.21.15.16:0/3222465612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e8991832f0 0x55e89954c6b0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.395+0000 7f094ae3c740 1 -- 172.21.15.16:0/3222465612 >> 172.21.15.16:0/3222465612 conn(0x55e8991a0300 msgr2=0x55e89954e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.395+0000 7f094ae3c740 1 -- 172.21.15.16:0/3222465612 shutdown_connections 2024-08-31T06:16:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.396+0000 7f094ae3c740 1 -- 172.21.15.16:0/3222465612 wait complete. 2024-08-31T06:16:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.396+0000 7f094ae3c740 1 -- start start 2024-08-31T06:16:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.397+0000 7f094ae3c740 1 Processor -- start 2024-08-31T06:16:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.400+0000 7f094ae3c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e8991832f0 0x55e899556710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.400+0000 7f094ae3c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e89918b4d0 0x55e899556c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.400+0000 7f094ae3c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8991ac1b0 0x55e8995d6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.400+0000 7f094ae3c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f091c00cd90 con 0x55e8991ac1b0 2024-08-31T06:16:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.400+0000 7f094ae3c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f091c00e200 con 0x55e89918b4d0 2024-08-31T06:16:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.400+0000 7f094ae3c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f091c00d460 con 0x55e8991832f0 2024-08-31T06:16:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.400+0000 7f094ae3c740 10 client.0 _ll_register_callbacks cb 0x7f093000b610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:16:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.401+0000 7f0941400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e8991832f0 0x55e899556710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.401+0000 7f0941400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e8991832f0 0x55e899556710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:52866/0 (socket says 172.21.15.16:52866) 2024-08-31T06:16:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.401+0000 7f0941400640 1 -- 172.21.15.16:0/3663642305 learned_addr learned my addr 172.21.15.16:0/3663642305 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.402+0000 7f0940a00640 1 --2- 172.21.15.16:0/3663642305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e89918b4d0 0x55e899556c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.402+0000 7f0941e00640 1 --2- 172.21.15.16:0/3663642305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8991ac1b0 0x55e8995d6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.402+0000 7f0941400640 1 -- 172.21.15.16:0/3663642305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e89918b4d0 msgr2=0x55e899556c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.402+0000 7f0941400640 1 --2- 172.21.15.16:0/3663642305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e89918b4d0 0x55e899556c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.402+0000 7f0941400640 1 -- 172.21.15.16:0/3663642305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8991ac1b0 msgr2=0x55e8995d6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.402+0000 7f0941400640 1 --2- 172.21.15.16:0/3663642305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8991ac1b0 0x55e8995d6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.402+0000 7f0941400640 1 -- 172.21.15.16:0/3663642305 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8995d70a0 con 0x55e8991832f0 2024-08-31T06:16:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.403+0000 7f0941400640 1 --2- 172.21.15.16:0/3663642305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e8991832f0 0x55e899556710 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f09340a8a30 tx=0x7f09340a7ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13940a1269f5472 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.403+0000 7f0922a00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:16:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.404+0000 7f0922a00640 1 -- 172.21.15.16:0/3663642305 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0934096070 con 0x55e8991832f0 2024-08-31T06:16:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.404+0000 7f0922a00640 1 -- 172.21.15.16:0/3663642305 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8470+0+0 (secure 0 0 0) 0x7f09340d2170 con 0x55e8991832f0 2024-08-31T06:16:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.405+0000 7f094ae3c740 1 -- 172.21.15.16:0/3663642305 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e8994c2960 con 0x55e8991832f0 2024-08-31T06:16:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.405+0000 7f0922a00640 1 -- 172.21.15.16:0/3663642305 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f09340e2c20 con 0x55e8991832f0 2024-08-31T06:16:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.407+0000 7f094ae3c740 10 client.27733 fetch_fsmap learned FSMap version 43 2024-08-31T06:16:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.407+0000 7f094ae3c740 1 -- 172.21.15.16:0/3663642305 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55e8995d8300 con 0x55e8991832f0 2024-08-31T06:16:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.407+0000 7f0922a00640 1 -- 172.21.15.16:0/3663642305 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09340c6020 con 0x55e8991832f0 2024-08-31T06:16:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.408+0000 7f0922a00640 1 -- 172.21.15.16:0/3663642305 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f0934016070 con 0x55e8991832f0 2024-08-31T06:16:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.409+0000 7f094ae3c740 10 client.27733 fetch_fsmap finished waiting for FSMap version 43 2024-08-31T06:16:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.409+0000 7f094ae3c740 10 client.27733 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:16:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.409+0000 7f094ae3c740 20 client.27733 populate_metadata read hostname 'smithi016' 2024-08-31T06:16:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.409+0000 7f094ae3c740 1 --2- 172.21.15.16:0/3663642305 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55e8991401c0 0x55e8995da980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.409+0000 7f094ae3c740 4 client.27733 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:16:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.409+0000 7f094ae3c740 1 -- 172.21.15.16:0/3663642305 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e8994c00a0 con 0x55e8991401c0 2024-08-31T06:16:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.410+0000 7f0940a00640 1 --2- 172.21.15.16:0/3663642305 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55e8991401c0 0x55e8995da980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.410+0000 7f0940a00640 1 --2- 172.21.15.16:0/3663642305 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55e8991401c0 0x55e8995da980 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7dc0943be5b4371 server_cookie=b3c843d2f29b5998 in_seq=0 out_seq=0 2024-08-31T06:16:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.411+0000 7f0922a00640 10 client.27733 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:16:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.459+0000 7f0922a00640 1 -- 172.21.15.16:0/3663642305 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e8994c00a0 con 0x55e8991401c0 2024-08-31T06:16:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.459+0000 7f0922a00640 10 client.27733 handle_command_reply: tid=0 2024-08-31T06:16:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.459+0000 7f094ae3c740 1 client.27733 shutdown 2024-08-31T06:16:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.459+0000 7f094ae3c740 1 -- 172.21.15.16:0/3663642305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e8991832f0 msgr2=0x55e899556710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.460+0000 7f094ae3c740 1 --2- 172.21.15.16:0/3663642305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e8991832f0 0x55e899556710 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f09340a8a30 tx=0x7f09340a7ce0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.460+0000 7f094ae3c740 1 -- 172.21.15.16:0/3663642305 shutdown_connections 2024-08-31T06:16:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.460+0000 7f094ae3c740 1 --2- 172.21.15.16:0/3663642305 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55e8991401c0 0x55e8995da980 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.460+0000 7f094ae3c740 1 --2- 172.21.15.16:0/3663642305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8991ac1b0 0x55e8995d6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.460+0000 7f094ae3c740 1 --2- 172.21.15.16:0/3663642305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e89918b4d0 0x55e899556c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.460+0000 7f094ae3c740 1 --2- 172.21.15.16:0/3663642305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e8991832f0 0x55e899556710 secure :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f09340a8a30 tx=0x7f09340a7ce0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.460+0000 7f094ae3c740 1 -- 172.21.15.16:0/3663642305 >> 172.21.15.16:0/3663642305 conn(0x55e8991a0300 msgr2=0x55e89954c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.460+0000 7f0922a00640 0 client.27733 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:16:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.460+0000 7f094ae3c740 1 -- 172.21.15.16:0/3663642305 shutdown_connections 2024-08-31T06:16:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.461+0000 7f094ae3c740 1 -- 172.21.15.16:0/3663642305 wait complete. 2024-08-31T06:16:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.461+0000 7f094ae3c740 20 client.27733 trim_cache size 0 max 16384 2024-08-31T06:16:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:16:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:16:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:16:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:16:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.463+0000 7f0943e00640 1 -- 172.21.15.16:0/871043720 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f09240b4eb0 msgr2=0x7f09240b73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.463+0000 7f0943e00640 1 --2- 172.21.15.16:0/871043720 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f09240b4eb0 0x7f09240b73c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f093000b980 tx=0x7f0930005c10 comp rx=0 tx=0).stop 2024-08-31T06:16:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.463+0000 7f0943e00640 1 -- 172.21.15.16:0/871043720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f093c10a400 msgr2=0x7f093c114a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.463+0000 7f0943e00640 1 --2- 172.21.15.16:0/871043720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f093c10a400 0x7f093c114a80 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f093400dfd0 tx=0x7f093400b630 comp rx=0 tx=0).stop 2024-08-31T06:16:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.463+0000 7f0943e00640 1 -- 172.21.15.16:0/871043720 shutdown_connections 2024-08-31T06:16:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.463+0000 7f0943e00640 1 --2- 172.21.15.16:0/871043720 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f09240b4eb0 0x7f09240b73c0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.463+0000 7f0943e00640 1 --2- 172.21.15.16:0/871043720 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f093c10fc00 0x7f093c110d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.463+0000 7f0943e00640 1 --2- 172.21.15.16:0/871043720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f093c110550 0x7f093c111480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.463+0000 7f0943e00640 1 --2- 172.21.15.16:0/871043720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f093c10a400 0x7f093c114a80 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.463+0000 7f0943e00640 1 -- 172.21.15.16:0/871043720 >> 172.21.15.16:0/871043720 conn(0x7f093c070fc0 msgr2=0x7f093c10f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.463+0000 7f0943e00640 1 -- 172.21.15.16:0/871043720 shutdown_connections 2024-08-31T06:16:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:45.464+0000 7f0943e00640 1 -- 172.21.15.16:0/871043720 wait complete. 2024-08-31T06:16:46.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 560.0 s, 2505.4 tps, lat 39.925 ms stddev 41.511 2024-08-31T06:16:46.628 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:46 smithi016 ceph-mon[17194]: pgmap v658: 97 pgs: 97 active+clean; 3.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 7.9 MiB/s wr, 772 op/s 2024-08-31T06:16:46.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:46 smithi184 ceph-mon[22331]: pgmap v658: 97 pgs: 97 active+clean; 3.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 7.9 MiB/s wr, 772 op/s 2024-08-31T06:16:47.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:46 smithi155 ceph-mon[21742]: pgmap v658: 97 pgs: 97 active+clean; 3.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 7.9 MiB/s wr, 772 op/s 2024-08-31T06:16:47.292 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T06:16:47.292 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T06:16:47.939 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:16:48.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.840+0000 7f356d800640 1 -- 172.21.15.16:0/265132080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f356810b2a0 msgr2=0x7f356810b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:48.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.840+0000 7f356d800640 1 --2- 172.21.15.16:0/265132080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f356810b2a0 0x7f356810b6b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3554009900 tx=0x7f355402a3f0 comp rx=0 tx=0).stop 2024-08-31T06:16:48.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.840+0000 7f356d800640 1 -- 172.21.15.16:0/265132080 shutdown_connections 2024-08-31T06:16:48.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.840+0000 7f356d800640 1 --2- 172.21.15.16:0/265132080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f356810bbf0 0x7f3568112420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:48.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.840+0000 7f356d800640 1 --2- 172.21.15.16:0/265132080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f356810b2a0 0x7f356810b6b0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:48.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.840+0000 7f356d800640 1 --2- 172.21.15.16:0/265132080 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3568105960 0x7f3568105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:48.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.840+0000 7f356d800640 1 -- 172.21.15.16:0/265132080 >> 172.21.15.16:0/265132080 conn(0x7f35681013d0 msgr2=0x7f3568103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:48.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.841+0000 7f356d800640 1 -- 172.21.15.16:0/265132080 shutdown_connections 2024-08-31T06:16:48.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.841+0000 7f356d800640 1 -- 172.21.15.16:0/265132080 wait complete. 2024-08-31T06:16:48.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.842+0000 7f356d800640 1 Processor -- start 2024-08-31T06:16:48.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.842+0000 7f356d800640 1 -- start start 2024-08-31T06:16:48.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.843+0000 7f356d800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3568105960 0x7f356819c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:48.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.843+0000 7f356d800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f356810b2a0 0x7f356819c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:48.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.844+0000 7f356d800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f356810bbf0 0x7f35681a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:48.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.844+0000 7f356d800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3568115900 con 0x7f356810b2a0 2024-08-31T06:16:48.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.844+0000 7f356d800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3568115780 con 0x7f356810bbf0 2024-08-31T06:16:48.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.844+0000 7f356d800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3568115a80 con 0x7f3568105960 2024-08-31T06:16:48.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.844+0000 7f3567e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3568105960 0x7f356819c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:48.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.844+0000 7f3567e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3568105960 0x7f356819c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:44122/0 (socket says 172.21.15.16:44122) 2024-08-31T06:16:48.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.844+0000 7f3567e00640 1 -- 172.21.15.16:0/2050015199 learned_addr learned my addr 172.21.15.16:0/2050015199 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:48.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.844+0000 7f356ce00640 1 --2- 172.21.15.16:0/2050015199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f356810bbf0 0x7f35681a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:48.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.844+0000 7f3567400640 1 --2- 172.21.15.16:0/2050015199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f356810b2a0 0x7f356819c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:48.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.845+0000 7f3567400640 1 -- 172.21.15.16:0/2050015199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3568105960 msgr2=0x7f356819c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:48.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.845+0000 7f3567400640 1 --2- 172.21.15.16:0/2050015199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3568105960 0x7f356819c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:48.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.846+0000 7f3567400640 1 -- 172.21.15.16:0/2050015199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f356810bbf0 msgr2=0x7f35681a1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:48.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.846+0000 7f3567400640 1 --2- 172.21.15.16:0/2050015199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f356810bbf0 0x7f35681a1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:48.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.846+0000 7f3567400640 1 -- 172.21.15.16:0/2050015199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35681a21e0 con 0x7f356810b2a0 2024-08-31T06:16:48.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.846+0000 7f3567400640 1 --2- 172.21.15.16:0/2050015199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f356810b2a0 0x7f356819c810 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f3554009230 tx=0x7f3554003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:48.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.847+0000 7f3564c00640 1 -- 172.21.15.16:0/2050015199 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355403f070 con 0x7f356810b2a0 2024-08-31T06:16:48.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.847+0000 7f3564c00640 1 -- 172.21.15.16:0/2050015199 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3554031b50 con 0x7f356810b2a0 2024-08-31T06:16:48.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.847+0000 7f3564c00640 1 -- 172.21.15.16:0/2050015199 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3554035030 con 0x7f356810b2a0 2024-08-31T06:16:48.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.847+0000 7f356d800640 1 -- 172.21.15.16:0/2050015199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35681a2470 con 0x7f356810b2a0 2024-08-31T06:16:48.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.848+0000 7f356d800640 1 -- 172.21.15.16:0/2050015199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35681a28d0 con 0x7f356810b2a0 2024-08-31T06:16:48.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.852+0000 7f3575aa8740 1 Processor -- start 2024-08-31T06:16:48.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.853+0000 7f3575aa8740 1 -- start start 2024-08-31T06:16:48.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.853+0000 7f3575aa8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c34094350 0x555c3443a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:48.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.853+0000 7f3575aa8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c3406b490 0x555c3443ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:48.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.853+0000 7f3575aa8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c34073670 0x555c3443b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:48.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.853+0000 7f3575aa8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555c343b0750 con 0x555c34073670 2024-08-31T06:16:48.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.853+0000 7f3575aa8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555c3439fce0 con 0x555c34094350 2024-08-31T06:16:48.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.853+0000 7f3575aa8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x555c343a3c90 con 0x555c3406b490 2024-08-31T06:16:48.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.853+0000 7f356ce00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c34094350 0x555c3443a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:48.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.854+0000 7f356ce00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c34094350 0x555c3443a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:54340/0 (socket says 172.21.15.16:54340) 2024-08-31T06:16:48.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.854+0000 7f356ce00640 1 -- 172.21.15.16:0/4137241310 learned_addr learned my addr 172.21.15.16:0/4137241310 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:48.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.854+0000 7f3567e00640 1 --2- 172.21.15.16:0/4137241310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c3406b490 0x555c3443ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:48.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.854+0000 7f356ce00640 1 -- 172.21.15.16:0/4137241310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c3406b490 msgr2=0x555c3443ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:48.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.854+0000 7f356ce00640 1 --2- 172.21.15.16:0/4137241310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c3406b490 0x555c3443ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:48.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.854+0000 7f356ce00640 1 -- 172.21.15.16:0/4137241310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c34073670 msgr2=0x555c3443b250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:16:48.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.854+0000 7f356ce00640 1 --2- 172.21.15.16:0/4137241310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c34073670 0x555c3443b250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:48.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.854+0000 7f356ce00640 1 -- 172.21.15.16:0/4137241310 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c340ff930 con 0x555c34094350 2024-08-31T06:16:48.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.854+0000 7f3567e00640 1 --2- 172.21.15.16:0/4137241310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c3406b490 0x555c3443ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:16:48.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.854+0000 7f356ce00640 1 --2- 172.21.15.16:0/4137241310 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c34094350 0x555c3443a7d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3560009fc0 tx=0x7f3560010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec3f6ea83ff9b907 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:48.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.855+0000 7f3564c00640 1 -- 172.21.15.16:0/2050015199 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f3554042070 con 0x7f356810b2a0 2024-08-31T06:16:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.855+0000 7f355d600640 1 -- 172.21.15.16:0/4137241310 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f356001f070 con 0x555c34094350 2024-08-31T06:16:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.855+0000 7f355d600640 1 -- 172.21.15.16:0/4137241310 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3560003b70 con 0x555c34094350 2024-08-31T06:16:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.855+0000 7f355d600640 1 -- 172.21.15.16:0/4137241310 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f356000e780 con 0x555c34094350 2024-08-31T06:16:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.856+0000 7f3575aa8740 1 -- 172.21.15.16:0/4137241310 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c34094350 msgr2=0x555c3443a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.856+0000 7f3575aa8740 1 --2- 172.21.15.16:0/4137241310 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c34094350 0x555c3443a7d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3560009fc0 tx=0x7f3560010860 comp rx=0 tx=0).stop 2024-08-31T06:16:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.856+0000 7f3575aa8740 1 -- 172.21.15.16:0/4137241310 shutdown_connections 2024-08-31T06:16:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.856+0000 7f3575aa8740 1 --2- 172.21.15.16:0/4137241310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c3406b490 0x555c3443ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.856+0000 7f3575aa8740 1 --2- 172.21.15.16:0/4137241310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c34073670 0x555c3443b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.856+0000 7f3575aa8740 1 --2- 172.21.15.16:0/4137241310 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c34094350 0x555c3443a7d0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.856+0000 7f3575aa8740 1 -- 172.21.15.16:0/4137241310 >> 172.21.15.16:0/4137241310 conn(0x555c34088740 msgr2=0x555c344386c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.857+0000 7f3564c00640 1 --2- 172.21.15.16:0/2050015199 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f354c0b4eb0 0x7f354c0b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.858+0000 7f3575aa8740 1 -- 172.21.15.16:0/4137241310 shutdown_connections 2024-08-31T06:16:48.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.858+0000 7f3575aa8740 1 -- 172.21.15.16:0/4137241310 wait complete. 2024-08-31T06:16:48.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.858+0000 7f3567e00640 1 --2- 172.21.15.16:0/2050015199 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f354c0b4eb0 0x7f354c0b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:48.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.859+0000 7f3575aa8740 1 -- start start 2024-08-31T06:16:48.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.859+0000 7f3575aa8740 1 Processor -- start 2024-08-31T06:16:48.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.860+0000 7f3575aa8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c3406b490 0x555c34446490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:48.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.860+0000 7f3575aa8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c34073670 0x555c34446a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:48.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.860+0000 7f3575aa8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c34094350 0x555c3444c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:48.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.860+0000 7f3575aa8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555c34202890 con 0x555c34073670 2024-08-31T06:16:48.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.860+0000 7f3575aa8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555c34326cd0 con 0x555c3406b490 2024-08-31T06:16:48.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.860+0000 7f3575aa8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x555c34448fb0 con 0x555c34094350 2024-08-31T06:16:48.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.860+0000 7f3575aa8740 10 client.0 _ll_register_callbacks cb 0x555c3439aef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:16:48.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.861+0000 7f356ce00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c34094350 0x555c3444c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:48.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.861+0000 7f356ce00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c34094350 0x555c3444c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:44138/0 (socket says 172.21.15.16:44138) 2024-08-31T06:16:48.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.861+0000 7f356ce00640 1 -- 172.21.15.16:0/4238891096 learned_addr learned my addr 172.21.15.16:0/4238891096 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:48.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.861+0000 7f3567400640 1 --2- 172.21.15.16:0/4238891096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c34073670 0x555c34446a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:48.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.861+0000 7f3564c00640 1 -- 172.21.15.16:0/2050015199 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8470+0+0 (secure 0 0 0) 0x7f3554105b20 con 0x7f356810b2a0 2024-08-31T06:16:48.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.862+0000 7f356ce00640 1 -- 172.21.15.16:0/4238891096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c3406b490 msgr2=0x555c34446490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:16:48.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.862+0000 7f356ce00640 1 --2- 172.21.15.16:0/4238891096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c3406b490 0x555c34446490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:48.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.862+0000 7f356ce00640 1 -- 172.21.15.16:0/4238891096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c34073670 msgr2=0x555c34446a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:48.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.862+0000 7f356ce00640 1 --2- 172.21.15.16:0/4238891096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c34073670 0x555c34446a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:48.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.862+0000 7f356ce00640 1 -- 172.21.15.16:0/4238891096 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c343701b0 con 0x555c34094350 2024-08-31T06:16:48.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.862+0000 7f3567e00640 1 --2- 172.21.15.16:0/2050015199 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f354c0b4eb0 0x7f354c0b73c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x555c34439450 tx=0x7f3558005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:48.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.862+0000 7f356ce00640 1 --2- 172.21.15.16:0/4238891096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c34094350 0x555c3444c270 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3560005c00 tx=0x7f3560009320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b99bd85a66dbf4b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:48.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.862+0000 7f3552000640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:16:48.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.863+0000 7f3552000640 1 -- 172.21.15.16:0/4238891096 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f356001f040 con 0x555c34094350 2024-08-31T06:16:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.863+0000 7f3575aa8740 1 -- 172.21.15.16:0/4238891096 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3560003b70 con 0x555c34094350 2024-08-31T06:16:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.863+0000 7f3552000640 1 -- 172.21.15.16:0/4238891096 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8470+0+0 (secure 0 0 0) 0x7f3560004210 con 0x555c34094350 2024-08-31T06:16:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.864+0000 7f3552000640 1 -- 172.21.15.16:0/4238891096 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f3560039050 con 0x555c34094350 2024-08-31T06:16:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.864+0000 7f3575aa8740 10 client.27745 fetch_fsmap learned FSMap version 43 2024-08-31T06:16:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.864+0000 7f3575aa8740 1 -- 172.21.15.16:0/4238891096 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x555c344550d0 con 0x555c34094350 2024-08-31T06:16:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.864+0000 7f3552000640 1 -- 172.21.15.16:0/4238891096 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3560003b70 con 0x555c34094350 2024-08-31T06:16:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.864+0000 7f3552000640 1 -- 172.21.15.16:0/4238891096 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f3560005720 con 0x555c34094350 2024-08-31T06:16:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.865+0000 7f3575aa8740 10 client.27745 fetch_fsmap finished waiting for FSMap version 43 2024-08-31T06:16:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.865+0000 7f3575aa8740 10 client.27745 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:16:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.865+0000 7f3575aa8740 20 client.27745 populate_metadata read hostname 'smithi016' 2024-08-31T06:16:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.865+0000 7f3575aa8740 1 --2- 172.21.15.16:0/4238891096 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x555c34052cd0 0x555c344584e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.865+0000 7f3575aa8740 4 client.27745 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:16:48.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.865+0000 7f3575aa8740 1 -- 172.21.15.16:0/4238891096 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c343b26a0 con 0x555c34052cd0 2024-08-31T06:16:48.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.865+0000 7f3567e00640 1 --2- 172.21.15.16:0/4238891096 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x555c34052cd0 0x555c344584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:48.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.866+0000 7f3552000640 10 client.27745 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:16:48.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.866+0000 7f3567e00640 1 --2- 172.21.15.16:0/4238891096 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x555c34052cd0 0x555c344584e0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83545c8221bb95d5 server_cookie=98df23335f85efbd in_seq=0 out_seq=0 2024-08-31T06:16:48.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.867+0000 7f3552000640 1 -- 172.21.15.16:0/4238891096 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555c343b26a0 con 0x555c34052cd0 2024-08-31T06:16:48.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.867+0000 7f3552000640 10 client.27745 handle_command_reply: tid=0 2024-08-31T06:16:48.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.867+0000 7f3575aa8740 1 client.27745 shutdown 2024-08-31T06:16:48.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.868+0000 7f3575aa8740 1 -- 172.21.15.16:0/4238891096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c34094350 msgr2=0x555c3444c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:48.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.868+0000 7f3575aa8740 1 --2- 172.21.15.16:0/4238891096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c34094350 0x555c3444c270 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3560005c00 tx=0x7f3560009320 comp rx=0 tx=0).stop 2024-08-31T06:16:48.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.868+0000 7f3575aa8740 1 -- 172.21.15.16:0/4238891096 shutdown_connections 2024-08-31T06:16:48.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.868+0000 7f3575aa8740 1 --2- 172.21.15.16:0/4238891096 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x555c34052cd0 0x555c344584e0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:48.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.868+0000 7f3552000640 0 client.27745 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:16:48.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.868+0000 7f3575aa8740 1 --2- 172.21.15.16:0/4238891096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c34094350 0x555c3444c270 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:48.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.868+0000 7f3575aa8740 1 --2- 172.21.15.16:0/4238891096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c34073670 0x555c34446a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:48.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.868+0000 7f3575aa8740 1 --2- 172.21.15.16:0/4238891096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c3406b490 0x555c34446490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:48.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.868+0000 7f3575aa8740 1 -- 172.21.15.16:0/4238891096 >> 172.21.15.16:0/4238891096 conn(0x555c34088740 msgr2=0x555c3443d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:48.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.868+0000 7f3575aa8740 1 -- 172.21.15.16:0/4238891096 shutdown_connections 2024-08-31T06:16:48.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.869+0000 7f3575aa8740 1 -- 172.21.15.16:0/4238891096 wait complete. 2024-08-31T06:16:48.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.869+0000 7f3575aa8740 20 client.27745 trim_cache size 0 max 16384 2024-08-31T06:16:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.899+0000 7f3575aa8740 1 Processor -- start 2024-08-31T06:16:48.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.899+0000 7f3575aa8740 1 -- start start 2024-08-31T06:16:48.902 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:48 smithi016 ceph-mon[17194]: pgmap v659: 97 pgs: 97 active+clean; 3.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.3 MiB/s wr, 621 op/s 2024-08-31T06:16:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.900+0000 7f3575aa8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c3406b490 0x555c3443f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.900+0000 7f3575aa8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c34073670 0x555c3443fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.900+0000 7f3575aa8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c34094350 0x555c344434c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.900+0000 7f3575aa8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f35681af8d0 con 0x555c34094350 2024-08-31T06:16:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.900+0000 7f3575aa8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f35681ad140 con 0x555c3406b490 2024-08-31T06:16:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.900+0000 7f3575aa8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f35681aafb0 con 0x555c34073670 2024-08-31T06:16:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.900+0000 7f3567400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c34073670 0x555c3443fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.900+0000 7f3567400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c34073670 0x555c3443fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:44152/0 (socket says 172.21.15.16:44152) 2024-08-31T06:16:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.900+0000 7f3567400640 1 -- 172.21.15.16:0/543425821 learned_addr learned my addr 172.21.15.16:0/543425821 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.900+0000 7f3567e00640 1 --2- 172.21.15.16:0/543425821 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c3406b490 0x555c3443f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:48.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.900+0000 7f356ce00640 1 --2- 172.21.15.16:0/543425821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c34094350 0x555c344434c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:48.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.901+0000 7f3567400640 1 -- 172.21.15.16:0/543425821 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c3406b490 msgr2=0x555c3443f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:48.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.901+0000 7f3567400640 1 --2- 172.21.15.16:0/543425821 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c3406b490 0x555c3443f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:48.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.901+0000 7f3567400640 1 -- 172.21.15.16:0/543425821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c34094350 msgr2=0x555c344434c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:48.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.901+0000 7f3567400640 1 --2- 172.21.15.16:0/543425821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c34094350 0x555c344434c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:48.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.901+0000 7f3567400640 1 -- 172.21.15.16:0/543425821 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c34446dc0 con 0x555c34073670 2024-08-31T06:16:48.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.901+0000 7f3567400640 1 --2- 172.21.15.16:0/543425821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c34073670 0x555c3443fdf0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f35540c9450 tx=0x7f3554003f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f978b725e2a13504 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:48.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.901+0000 7f355cc00640 1 -- 172.21.15.16:0/543425821 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3554031d00 con 0x555c34073670 2024-08-31T06:16:48.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.902+0000 7f355cc00640 1 -- 172.21.15.16:0/543425821 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f355403b460 con 0x555c34073670 2024-08-31T06:16:48.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.902+0000 7f355cc00640 1 -- 172.21.15.16:0/543425821 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3554049d50 con 0x555c34073670 2024-08-31T06:16:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.903+0000 7f3575aa8740 1 -- 172.21.15.16:0/543425821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c34073670 msgr2=0x555c3443fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.903+0000 7f3575aa8740 1 --2- 172.21.15.16:0/543425821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c34073670 0x555c3443fdf0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f35540c9450 tx=0x7f3554003f10 comp rx=0 tx=0).stop 2024-08-31T06:16:48.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.903+0000 7f3575aa8740 1 -- 172.21.15.16:0/543425821 shutdown_connections 2024-08-31T06:16:48.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.903+0000 7f3575aa8740 1 --2- 172.21.15.16:0/543425821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c34073670 0x555c3443fdf0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:48.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.903+0000 7f3575aa8740 1 --2- 172.21.15.16:0/543425821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c34094350 0x555c344434c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:48.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.903+0000 7f3575aa8740 1 --2- 172.21.15.16:0/543425821 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c3406b490 0x555c3443f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:48.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.903+0000 7f3575aa8740 1 -- 172.21.15.16:0/543425821 >> 172.21.15.16:0/543425821 conn(0x555c34088740 msgr2=0x555c34441480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:48.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.904+0000 7f3575aa8740 1 -- 172.21.15.16:0/543425821 shutdown_connections 2024-08-31T06:16:48.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.904+0000 7f3575aa8740 1 -- 172.21.15.16:0/543425821 wait complete. 2024-08-31T06:16:48.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.905+0000 7f3575aa8740 1 -- start start 2024-08-31T06:16:48.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.906+0000 7f3575aa8740 1 Processor -- start 2024-08-31T06:16:48.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.906+0000 7f3575aa8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c3406b490 0x555c344526f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:48.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.906+0000 7f3575aa8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c34073670 0x555c34450bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:48.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.906+0000 7f3575aa8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c34094350 0x555c34451110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:48.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.907+0000 7f3575aa8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f35681ab600 con 0x555c3406b490 2024-08-31T06:16:48.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.907+0000 7f3575aa8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f35681aca70 con 0x555c34073670 2024-08-31T06:16:48.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.907+0000 7f3575aa8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f35681abcd0 con 0x555c34094350 2024-08-31T06:16:48.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.907+0000 7f3575aa8740 10 client.0 _ll_register_callbacks cb 0x7f3560009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:16:48.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.907+0000 7f3567400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c34073670 0x555c34450bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:48.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.907+0000 7f3567400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c34073670 0x555c34450bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:54364/0 (socket says 172.21.15.16:54364) 2024-08-31T06:16:48.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.907+0000 7f3567400640 1 -- 172.21.15.16:0/1739891548 learned_addr learned my addr 172.21.15.16:0/1739891548 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:48.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.907+0000 7f356ce00640 1 --2- 172.21.15.16:0/1739891548 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c34094350 0x555c34451110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:48.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.907+0000 7f3567e00640 1 --2- 172.21.15.16:0/1739891548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c3406b490 0x555c344526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:48.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.908+0000 7f356ce00640 1 -- 172.21.15.16:0/1739891548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c34073670 msgr2=0x555c34450bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:48.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.908+0000 7f356ce00640 1 --2- 172.21.15.16:0/1739891548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c34073670 0x555c34450bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:48.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.908+0000 7f356ce00640 1 -- 172.21.15.16:0/1739891548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c3406b490 msgr2=0x555c344526f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:48.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.908+0000 7f356ce00640 1 --2- 172.21.15.16:0/1739891548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c3406b490 0x555c344526f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:48.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.908+0000 7f356ce00640 1 -- 172.21.15.16:0/1739891548 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c34451750 con 0x555c34094350 2024-08-31T06:16:48.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.908+0000 7f3567e00640 1 --2- 172.21.15.16:0/1739891548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c3406b490 0x555c344526f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:16:48.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.910+0000 7f3552000640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:16:48.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.910+0000 7f356ce00640 1 --2- 172.21.15.16:0/1739891548 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c34094350 0x555c34451110 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3560006010 tx=0x7f3560010fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20c06b7aee9f65cd server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:48.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.911+0000 7f3552000640 1 -- 172.21.15.16:0/1739891548 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f356001f070 con 0x555c34094350 2024-08-31T06:16:48.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.911+0000 7f3552000640 1 -- 172.21.15.16:0/1739891548 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8470+0+0 (secure 0 0 0) 0x7f3560003a30 con 0x555c34094350 2024-08-31T06:16:48.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.911+0000 7f3575aa8740 1 -- 172.21.15.16:0/1739891548 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555c343ae140 con 0x555c34094350 2024-08-31T06:16:48.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.912+0000 7f3552000640 1 -- 172.21.15.16:0/1739891548 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f356004a050 con 0x555c34094350 2024-08-31T06:16:48.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.912+0000 7f3575aa8740 10 client.27751 fetch_fsmap learned FSMap version 43 2024-08-31T06:16:48.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.912+0000 7f3575aa8740 1 -- 172.21.15.16:0/1739891548 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x555c344c3aa0 con 0x555c34094350 2024-08-31T06:16:48.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.913+0000 7f3552000640 1 -- 172.21.15.16:0/1739891548 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x555c343ae140 con 0x555c34094350 2024-08-31T06:16:48.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.913+0000 7f3552000640 1 -- 172.21.15.16:0/1739891548 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f3560004690 con 0x555c34094350 2024-08-31T06:16:48.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.913+0000 7f3575aa8740 10 client.27751 fetch_fsmap finished waiting for FSMap version 43 2024-08-31T06:16:48.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.913+0000 7f3575aa8740 10 client.27751 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:16:48.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.913+0000 7f3575aa8740 20 client.27751 populate_metadata read hostname 'smithi016' 2024-08-31T06:16:48.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.913+0000 7f3575aa8740 1 --2- 172.21.15.16:0/1739891548 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x555c34028080 0x555c344c61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:48.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.913+0000 7f3575aa8740 4 client.27751 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T06:16:48.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.913+0000 7f3575aa8740 1 -- 172.21.15.16:0/1739891548 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555c343ab880 con 0x555c34028080 2024-08-31T06:16:48.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.914+0000 7f3567e00640 1 --2- 172.21.15.16:0/1739891548 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x555c34028080 0x555c344c61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:48.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.915+0000 7f3567e00640 1 --2- 172.21.15.16:0/1739891548 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x555c34028080 0x555c344c61a0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=845cadbb2fb33c5e server_cookie=c0a1857655ff4c30 in_seq=0 out_seq=0 2024-08-31T06:16:48.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.915+0000 7f3552000640 10 client.27751 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:16:48.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.917+0000 7f3552000640 1 -- 172.21.15.16:0/1739891548 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555c343ab880 con 0x555c34028080 2024-08-31T06:16:48.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.917+0000 7f3552000640 10 client.27751 handle_command_reply: tid=0 2024-08-31T06:16:48.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.917+0000 7f3575aa8740 1 client.27751 shutdown 2024-08-31T06:16:48.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.918+0000 7f3575aa8740 1 -- 172.21.15.16:0/1739891548 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c34094350 msgr2=0x555c34451110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:48.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.918+0000 7f3575aa8740 1 --2- 172.21.15.16:0/1739891548 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c34094350 0x555c34451110 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3560006010 tx=0x7f3560010fd0 comp rx=0 tx=0).stop 2024-08-31T06:16:48.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.920+0000 7f3575aa8740 1 -- 172.21.15.16:0/1739891548 shutdown_connections 2024-08-31T06:16:48.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.920+0000 7f3575aa8740 1 --2- 172.21.15.16:0/1739891548 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x555c34028080 0x555c344c61a0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:48.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.920+0000 7f3575aa8740 1 --2- 172.21.15.16:0/1739891548 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c34094350 0x555c34451110 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:48.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.920+0000 7f3552000640 0 client.27751 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:16:48.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.920+0000 7f3575aa8740 1 --2- 172.21.15.16:0/1739891548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c34073670 0x555c34450bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:48.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.920+0000 7f3575aa8740 1 --2- 172.21.15.16:0/1739891548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c3406b490 0x555c344526f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:48.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.920+0000 7f3575aa8740 1 -- 172.21.15.16:0/1739891548 >> 172.21.15.16:0/1739891548 conn(0x555c34088740 msgr2=0x555c34441480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:48.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.924+0000 7f3575aa8740 1 -- 172.21.15.16:0/1739891548 shutdown_connections 2024-08-31T06:16:48.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.924+0000 7f3575aa8740 1 -- 172.21.15.16:0/1739891548 wait complete. 2024-08-31T06:16:48.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.925+0000 7f3575aa8740 20 client.27751 trim_cache size 0 max 16384 2024-08-31T06:16:48.931 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:16:48.931 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-08-31T06:16:48.931 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "004b47bb-0fe8-4d28-9d9d-78bde6a61199", 2024-08-31T06:16:48.931 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-08-31T06:16:48.931 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:16:48.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.929+0000 7f356d800640 1 -- 172.21.15.16:0/2050015199 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f354c0b4eb0 msgr2=0x7f354c0b73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:48.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.929+0000 7f356d800640 1 --2- 172.21.15.16:0/2050015199 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f354c0b4eb0 0x7f354c0b73c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x555c34439450 tx=0x7f3558005f90 comp rx=0 tx=0).stop 2024-08-31T06:16:48.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.929+0000 7f356d800640 1 -- 172.21.15.16:0/2050015199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f356810b2a0 msgr2=0x7f356819c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:48.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.929+0000 7f356d800640 1 --2- 172.21.15.16:0/2050015199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f356810b2a0 0x7f356819c810 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f3554009230 tx=0x7f3554003b90 comp rx=0 tx=0).stop 2024-08-31T06:16:48.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.929+0000 7f356d800640 1 -- 172.21.15.16:0/2050015199 shutdown_connections 2024-08-31T06:16:48.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.929+0000 7f356d800640 1 --2- 172.21.15.16:0/2050015199 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f354c0b4eb0 0x7f354c0b73c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:48.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.929+0000 7f356d800640 1 --2- 172.21.15.16:0/2050015199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f356810bbf0 0x7f35681a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:48.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.929+0000 7f356d800640 1 --2- 172.21.15.16:0/2050015199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f356810b2a0 0x7f356819c810 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:48.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.929+0000 7f356d800640 1 --2- 172.21.15.16:0/2050015199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3568105960 0x7f356819c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:48.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.929+0000 7f356d800640 1 -- 172.21.15.16:0/2050015199 >> 172.21.15.16:0/2050015199 conn(0x7f35681013d0 msgr2=0x7f35681037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:48.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.932+0000 7f356d800640 1 -- 172.21.15.16:0/2050015199 shutdown_connections 2024-08-31T06:16:48.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:48.932+0000 7f356d800640 1 -- 172.21.15.16:0/2050015199 wait complete. 2024-08-31T06:16:48.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:48 smithi184 ceph-mon[22331]: pgmap v659: 97 pgs: 97 active+clean; 3.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.3 MiB/s wr, 621 op/s 2024-08-31T06:16:49.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:48 smithi155 ceph-mon[21742]: pgmap v659: 97 pgs: 97 active+clean; 3.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.3 MiB/s wr, 621 op/s 2024-08-31T06:16:49.640 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:49 smithi016 ceph-mon[17194]: scrub queued for path: / 2024-08-31T06:16:49.640 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:49 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:16:49.640 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:49 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:16:49.640 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:49 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:16:49.641 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:49 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:16:49.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:49 smithi184 ceph-mon[22331]: scrub queued for path: / 2024-08-31T06:16:49.976 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:49 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:16:49.976 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:49 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:16:49.976 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:49 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:16:49.976 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:49 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:16:50.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:49 smithi155 ceph-mon[21742]: scrub queued for path: / 2024-08-31T06:16:50.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:49 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:16:50.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:49 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:16:50.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:49 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:16:50.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:49 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:16:50.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:50 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:16:50.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:50 smithi016 ceph-mon[17194]: pgmap v660: 97 pgs: 97 active+clean; 3.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.5 MiB/s wr, 745 op/s 2024-08-31T06:16:50.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:50 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:16:50.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:50 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:16:50.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:50 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:16:50.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:50 smithi184 ceph-mon[22331]: pgmap v660: 97 pgs: 97 active+clean; 3.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.5 MiB/s wr, 745 op/s 2024-08-31T06:16:50.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:50 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:16:50.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:50 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:16:51.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:50 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:16:51.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:50 smithi155 ceph-mon[21742]: pgmap v660: 97 pgs: 97 active+clean; 3.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.5 MiB/s wr, 745 op/s 2024-08-31T06:16:51.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:50 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:16:51.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:50 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:16:51.300 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T06:16:51.625 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 565.0 s, 1994.7 tps, lat 49.740 ms stddev 67.008 2024-08-31T06:16:51.947 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:16:52.907 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:52 smithi016 ceph-mon[17194]: pgmap v661: 97 pgs: 97 active+clean; 3.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 7.2 MiB/s wr, 747 op/s 2024-08-31T06:16:52.915 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 15.5 2024-08-31T06:16:52.915 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:16:52.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:52 smithi184 ceph-mon[22331]: pgmap v661: 97 pgs: 97 active+clean; 3.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 7.2 MiB/s wr, 747 op/s 2024-08-31T06:16:53.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:52 smithi155 ceph-mon[21742]: pgmap v661: 97 pgs: 97 active+clean; 3.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 7.2 MiB/s wr, 747 op/s 2024-08-31T06:16:53.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.202+0000 7fa30ca00640 1 -- 172.21.15.16:0/44423240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa30811c940 msgr2=0x7fa30811ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:53.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.202+0000 7fa30ca00640 1 --2- 172.21.15.16:0/44423240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa30811c940 0x7fa30811ed60 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa2f401a910 tx=0x7fa2f403b430 comp rx=0 tx=0).stop 2024-08-31T06:16:53.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.206+0000 7fa30ca00640 1 -- 172.21.15.16:0/44423240 shutdown_connections 2024-08-31T06:16:53.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.206+0000 7fa30ca00640 1 --2- 172.21.15.16:0/44423240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa30811c940 0x7fa30811ed60 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:53.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.206+0000 7fa30ca00640 1 --2- 172.21.15.16:0/44423240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa30810d990 0x7fa30810dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:53.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.206+0000 7fa30ca00640 1 --2- 172.21.15.16:0/44423240 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa308111a70 0x7fa30810d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:53.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.206+0000 7fa30ca00640 1 -- 172.21.15.16:0/44423240 >> 172.21.15.16:0/44423240 conn(0x7fa30806e790 msgr2=0x7fa30806ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:53.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.206+0000 7fa30ca00640 1 -- 172.21.15.16:0/44423240 shutdown_connections 2024-08-31T06:16:53.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.206+0000 7fa30ca00640 1 -- 172.21.15.16:0/44423240 wait complete. 2024-08-31T06:16:53.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.208+0000 7fa30ca00640 1 Processor -- start 2024-08-31T06:16:53.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.208+0000 7fa30ca00640 1 -- start start 2024-08-31T06:16:53.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.209+0000 7fa30ca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa30810d990 0x7fa30811c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:53.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.209+0000 7fa30ca00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa308111a70 0x7fa3081182f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:53.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.210+0000 7fa30ca00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa30811c940 0x7fa308118830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:53.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.210+0000 7fa307400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa30810d990 0x7fa30811c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:53.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.210+0000 7fa30ca00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa308114a30 con 0x7fa30810d990 2024-08-31T06:16:53.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.210+0000 7fa306a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa308111a70 0x7fa3081182f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:53.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.210+0000 7fa307e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa30811c940 0x7fa308118830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:53.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.210+0000 7fa307e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa30811c940 0x7fa308118830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:44182/0 (socket says 172.21.15.16:44182) 2024-08-31T06:16:53.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.210+0000 7fa307e00640 1 -- 172.21.15.16:0/84858591 learned_addr learned my addr 172.21.15.16:0/84858591 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:53.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.211+0000 7fa30ca00640 1 -- 172.21.15.16:0/84858591 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa3081148b0 con 0x7fa308111a70 2024-08-31T06:16:53.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.211+0000 7fa30ca00640 1 -- 172.21.15.16:0/84858591 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa308114bb0 con 0x7fa30811c940 2024-08-31T06:16:53.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.211+0000 7fa307e00640 1 -- 172.21.15.16:0/84858591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa308111a70 msgr2=0x7fa3081182f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:53.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.211+0000 7fa307e00640 1 --2- 172.21.15.16:0/84858591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa308111a70 0x7fa3081182f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:53.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.211+0000 7fa307e00640 1 -- 172.21.15.16:0/84858591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa30810d990 msgr2=0x7fa30811c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:53.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.211+0000 7fa307e00640 1 --2- 172.21.15.16:0/84858591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa30810d990 0x7fa30811c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:53.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.211+0000 7fa307e00640 1 -- 172.21.15.16:0/84858591 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa308118e40 con 0x7fa30811c940 2024-08-31T06:16:53.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.212+0000 7fa307e00640 1 --2- 172.21.15.16:0/84858591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa30811c940 0x7fa308118830 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4005ab0 tx=0x7fa2f40048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:53.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.212+0000 7fa2ffe00640 1 -- 172.21.15.16:0/84858591 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f404f020 con 0x7fa30811c940 2024-08-31T06:16:53.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.212+0000 7fa2ffe00640 1 -- 172.21.15.16:0/84858591 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2f40035f0 con 0x7fa30811c940 2024-08-31T06:16:53.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.212+0000 7fa2ffe00640 1 -- 172.21.15.16:0/84858591 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f4046070 con 0x7fa30811c940 2024-08-31T06:16:53.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.213+0000 7fa30ca00640 1 -- 172.21.15.16:0/84858591 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3081190d0 con 0x7fa30811c940 2024-08-31T06:16:53.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.213+0000 7fa30ca00640 1 -- 172.21.15.16:0/84858591 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3081accb0 con 0x7fa30811c940 2024-08-31T06:16:53.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.230+0000 7fa3147fd740 1 Processor -- start 2024-08-31T06:16:53.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.232+0000 7fa3147fd740 1 -- start start 2024-08-31T06:16:53.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.232+0000 7fa2ffe00640 1 -- 172.21.15.16:0/84858591 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fa2f4052020 con 0x7fa30811c940 2024-08-31T06:16:53.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.233+0000 7fa3147fd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56283b984070 0x56283bd29a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:53.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.234+0000 7fa3147fd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56283b95b1b0 0x56283bd29f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:53.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.234+0000 7fa3147fd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56283b963390 0x56283bd2a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:53.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.234+0000 7fa3147fd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56283bca0770 con 0x56283b984070 2024-08-31T06:16:53.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.234+0000 7fa3147fd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56283bc8fd00 con 0x56283b963390 2024-08-31T06:16:53.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.234+0000 7fa3147fd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56283bc93cb0 con 0x56283b95b1b0 2024-08-31T06:16:53.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.235+0000 7fa306a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56283b984070 0x56283bd29a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:53.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.235+0000 7fa307400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56283b95b1b0 0x56283bd29f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:53.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.235+0000 7fa307400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56283b95b1b0 0x56283bd29f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:44194/0 (socket says 172.21.15.16:44194) 2024-08-31T06:16:53.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.235+0000 7fa306a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56283b984070 0x56283bd29a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58600/0 (socket says 172.21.15.16:58600) 2024-08-31T06:16:53.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.235+0000 7fa306a00640 1 -- 172.21.15.16:0/78602731 learned_addr learned my addr 172.21.15.16:0/78602731 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:53.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.236+0000 7fa306a00640 1 --2- 172.21.15.16:0/78602731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56283b963390 0x56283bd2a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:53.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.236+0000 7fa307400640 1 -- 172.21.15.16:0/78602731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56283b963390 msgr2=0x56283bd2a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:53.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.236+0000 7fa307400640 1 --2- 172.21.15.16:0/78602731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56283b963390 0x56283bd2a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:53.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.236+0000 7fa307400640 1 -- 172.21.15.16:0/78602731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56283b984070 msgr2=0x56283bd29a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:53.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.236+0000 7fa307400640 1 --2- 172.21.15.16:0/78602731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56283b984070 0x56283bd29a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:53.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.237+0000 7fa307400640 1 -- 172.21.15.16:0/78602731 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56283b9ef650 con 0x56283b95b1b0 2024-08-31T06:16:53.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.237+0000 7fa306a00640 1 --2- 172.21.15.16:0/78602731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56283b984070 0x56283bd29a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:16:53.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.238+0000 7fa2ffe00640 1 --2- 172.21.15.16:0/84858591 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa2ec0b4da0 0x7fa2ec0b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:53.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.238+0000 7fa2ffe00640 1 -- 172.21.15.16:0/84858591 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8470+0+0 (secure 0 0 0) 0x7fa2f41167a0 con 0x7fa30811c940 2024-08-31T06:16:53.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.239+0000 7fa307400640 1 --2- 172.21.15.16:0/78602731 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56283b95b1b0 0x56283bd29f70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa30811bd60 tx=0x7fa2f800ace0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ab7ebb016c6320d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:53.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.239+0000 7fa2f3e00640 1 -- 172.21.15.16:0/78602731 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f8015070 con 0x56283b95b1b0 2024-08-31T06:16:53.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.240+0000 7fa2f3e00640 1 -- 172.21.15.16:0/78602731 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2f80113f0 con 0x56283b95b1b0 2024-08-31T06:16:53.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.240+0000 7fa2f3e00640 1 -- 172.21.15.16:0/78602731 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f8015070 con 0x56283b95b1b0 2024-08-31T06:16:53.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.241+0000 7fa306a00640 1 --2- 172.21.15.16:0/84858591 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa2ec0b4da0 0x7fa2ec0b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:53.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.241+0000 7fa3147fd740 1 -- 172.21.15.16:0/78602731 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56283b95b1b0 msgr2=0x56283bd29f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:53.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.241+0000 7fa3147fd740 1 --2- 172.21.15.16:0/78602731 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56283b95b1b0 0x56283bd29f70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa30811bd60 tx=0x7fa2f800ace0 comp rx=0 tx=0).stop 2024-08-31T06:16:53.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.241+0000 7fa3147fd740 1 -- 172.21.15.16:0/78602731 shutdown_connections 2024-08-31T06:16:53.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.241+0000 7fa3147fd740 1 --2- 172.21.15.16:0/78602731 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56283b95b1b0 0x56283bd29f70 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:53.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.241+0000 7fa3147fd740 1 --2- 172.21.15.16:0/78602731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56283b963390 0x56283bd2a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:53.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.241+0000 7fa3147fd740 1 --2- 172.21.15.16:0/78602731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56283b984070 0x56283bd29a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:53.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.241+0000 7fa3147fd740 1 -- 172.21.15.16:0/78602731 >> 172.21.15.16:0/78602731 conn(0x56283b978460 msgr2=0x56283bd27920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:53.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.242+0000 7fa3147fd740 1 -- 172.21.15.16:0/78602731 shutdown_connections 2024-08-31T06:16:53.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.242+0000 7fa3147fd740 1 -- 172.21.15.16:0/78602731 wait complete. 2024-08-31T06:16:53.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.244+0000 7fa3147fd740 1 -- start start 2024-08-31T06:16:53.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.244+0000 7fa306a00640 1 --2- 172.21.15.16:0/84858591 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa2ec0b4da0 0x7fa2ec0b72b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa300005f70 tx=0x7fa300007980 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:53.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.245+0000 7fa3147fd740 1 Processor -- start 2024-08-31T06:16:53.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.246+0000 7fa3147fd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56283b95b1b0 0x56283bd35620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:53.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.246+0000 7fa3147fd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56283b963390 0x56283bd35b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:53.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.246+0000 7fa3147fd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56283b984070 0x56283bd41530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:53.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.246+0000 7fa3147fd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56283baf2700 con 0x56283b95b1b0 2024-08-31T06:16:53.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.246+0000 7fa3147fd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56283bc16cb0 con 0x56283b963390 2024-08-31T06:16:53.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.246+0000 7fa3147fd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56283bd38210 con 0x56283b984070 2024-08-31T06:16:53.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.246+0000 7fa3147fd740 10 client.0 _ll_register_callbacks cb 0x56283bc3c430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:16:53.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.247+0000 7fa306a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56283b963390 0x56283bd35b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:53.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.247+0000 7fa306a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56283b963390 0x56283bd35b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:54388/0 (socket says 172.21.15.16:54388) 2024-08-31T06:16:53.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.247+0000 7fa306a00640 1 -- 172.21.15.16:0/2275893163 learned_addr learned my addr 172.21.15.16:0/2275893163 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:53.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.247+0000 7fa307e00640 1 --2- 172.21.15.16:0/2275893163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56283b984070 0x56283bd41530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:53.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.248+0000 7fa306a00640 1 -- 172.21.15.16:0/2275893163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56283b984070 msgr2=0x56283bd41530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:53.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.248+0000 7fa306a00640 1 --2- 172.21.15.16:0/2275893163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56283b984070 0x56283bd41530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:53.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.248+0000 7fa306a00640 1 -- 172.21.15.16:0/2275893163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56283b95b1b0 msgr2=0x56283bd35620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:16:53.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.248+0000 7fa307400640 1 --2- 172.21.15.16:0/2275893163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56283b95b1b0 0x56283bd35620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:53.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.248+0000 7fa306a00640 1 --2- 172.21.15.16:0/2275893163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56283b95b1b0 0x56283bd35620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:53.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.249+0000 7fa306a00640 1 -- 172.21.15.16:0/2275893163 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56283bc57870 con 0x56283b963390 2024-08-31T06:16:53.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.249+0000 7fa306a00640 1 --2- 172.21.15.16:0/2275893163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56283b963390 0x56283bd35b60 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa300007430 tx=0x7fa3000075b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c4563b3ee15b5bf server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:53.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.250+0000 7fa2f0c00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:16:53.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.250+0000 7fa2f0c00640 1 -- 172.21.15.16:0/2275893163 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa300021620 con 0x56283b963390 2024-08-31T06:16:53.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.250+0000 7fa2f0c00640 1 -- 172.21.15.16:0/2275893163 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8470+0+0 (secure 0 0 0) 0x7fa3000217c0 con 0x56283b963390 2024-08-31T06:16:53.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.251+0000 7fa3147fd740 1 -- 172.21.15.16:0/2275893163 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2f80113f0 con 0x56283b963390 2024-08-31T06:16:53.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.252+0000 7fa2f0c00640 1 -- 172.21.15.16:0/2275893163 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa3000219c0 con 0x56283b963390 2024-08-31T06:16:53.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.252+0000 7fa3147fd740 10 client.27758 fetch_fsmap learned FSMap version 43 2024-08-31T06:16:53.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.252+0000 7fa3147fd740 1 -- 172.21.15.16:0/2275893163 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56283bd44280 con 0x56283b963390 2024-08-31T06:16:53.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.257+0000 7fa2f0c00640 1 -- 172.21.15.16:0/2275893163 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2f80113f0 con 0x56283b963390 2024-08-31T06:16:53.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.257+0000 7fa2f0c00640 1 -- 172.21.15.16:0/2275893163 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fa300029400 con 0x56283b963390 2024-08-31T06:16:53.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.257+0000 7fa3147fd740 10 client.27758 fetch_fsmap finished waiting for FSMap version 43 2024-08-31T06:16:53.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.257+0000 7fa3147fd740 10 client.27758 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:16:53.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.257+0000 7fa3147fd740 20 client.27758 populate_metadata read hostname 'smithi016' 2024-08-31T06:16:53.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.257+0000 7fa3147fd740 1 --2- 172.21.15.16:0/2275893163 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56283b9429f0 0x56283bd47770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:53.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.258+0000 7fa3147fd740 4 client.27758 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:16:53.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.258+0000 7fa3147fd740 1 -- 172.21.15.16:0/2275893163 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56283bca26c0 con 0x56283b9429f0 2024-08-31T06:16:53.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.258+0000 7fa307400640 1 --2- 172.21.15.16:0/2275893163 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56283b9429f0 0x56283bd47770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:53.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.260+0000 7fa2f0c00640 10 client.27758 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:16:53.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.260+0000 7fa307400640 1 --2- 172.21.15.16:0/2275893163 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56283b9429f0 0x56283bd47770 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5082be762d42c94a server_cookie=6b13586959d74113 in_seq=0 out_seq=0 2024-08-31T06:16:53.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.261+0000 7fa2f0c00640 1 -- 172.21.15.16:0/2275893163 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56283bca26c0 con 0x56283b9429f0 2024-08-31T06:16:53.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.261+0000 7fa2f0c00640 10 client.27758 handle_command_reply: tid=0 2024-08-31T06:16:53.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.261+0000 7fa3147fd740 1 client.27758 shutdown 2024-08-31T06:16:53.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.262+0000 7fa3147fd740 1 -- 172.21.15.16:0/2275893163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56283b963390 msgr2=0x56283bd35b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:53.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.262+0000 7fa3147fd740 1 --2- 172.21.15.16:0/2275893163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56283b963390 0x56283bd35b60 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa300007430 tx=0x7fa3000075b0 comp rx=0 tx=0).stop 2024-08-31T06:16:53.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.262+0000 7fa3147fd740 1 -- 172.21.15.16:0/2275893163 shutdown_connections 2024-08-31T06:16:53.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.262+0000 7fa3147fd740 1 --2- 172.21.15.16:0/2275893163 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56283b9429f0 0x56283bd47770 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:53.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.262+0000 7fa3147fd740 1 --2- 172.21.15.16:0/2275893163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56283b984070 0x56283bd41530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:53.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.262+0000 7fa3147fd740 1 --2- 172.21.15.16:0/2275893163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56283b963390 0x56283bd35b60 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:53.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.262+0000 7fa3147fd740 1 --2- 172.21.15.16:0/2275893163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56283b95b1b0 0x56283bd35620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:53.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.262+0000 7fa3147fd740 1 -- 172.21.15.16:0/2275893163 >> 172.21.15.16:0/2275893163 conn(0x56283b978460 msgr2=0x56283bd2cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:53.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.263+0000 7fa2f0c00640 0 client.27758 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:16:53.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.264+0000 7fa3147fd740 1 -- 172.21.15.16:0/2275893163 shutdown_connections 2024-08-31T06:16:53.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.264+0000 7fa3147fd740 1 -- 172.21.15.16:0/2275893163 wait complete. 2024-08-31T06:16:53.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.265+0000 7fa3147fd740 20 client.27758 trim_cache size 0 max 16384 2024-08-31T06:16:53.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.297+0000 7fa3147fd740 1 Processor -- start 2024-08-31T06:16:53.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.297+0000 7fa3147fd740 1 -- start start 2024-08-31T06:16:53.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.298+0000 7fa3147fd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56283b95b1b0 0x56283bd305d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:53.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.298+0000 7fa3147fd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56283b963390 0x56283bd26f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:53.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.298+0000 7fa3147fd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56283b984070 0x56283bd27450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:53.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.298+0000 7fa3147fd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa3081b8170 con 0x56283b963390 2024-08-31T06:16:53.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.298+0000 7fa3147fd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa3081b59e0 con 0x56283b95b1b0 2024-08-31T06:16:53.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.298+0000 7fa3147fd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa3081b3850 con 0x56283b984070 2024-08-31T06:16:53.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.298+0000 7fa307400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56283b95b1b0 0x56283bd305d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:53.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.298+0000 7fa307400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56283b95b1b0 0x56283bd305d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:54400/0 (socket says 172.21.15.16:54400) 2024-08-31T06:16:53.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.298+0000 7fa307400640 1 -- 172.21.15.16:0/249623981 learned_addr learned my addr 172.21.15.16:0/249623981 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:53.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.299+0000 7fa307e00640 1 --2- 172.21.15.16:0/249623981 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56283b984070 0x56283bd27450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:53.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.300+0000 7fa307400640 1 -- 172.21.15.16:0/249623981 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56283b984070 msgr2=0x56283bd27450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:53.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.300+0000 7fa307400640 1 --2- 172.21.15.16:0/249623981 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56283b984070 0x56283bd27450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:53.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.300+0000 7fa307400640 1 -- 172.21.15.16:0/249623981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56283b963390 msgr2=0x56283bd26f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:53.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.300+0000 7fa306a00640 1 --2- 172.21.15.16:0/249623981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56283b963390 0x56283bd26f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:53.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.301+0000 7fa307400640 1 --2- 172.21.15.16:0/249623981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56283b963390 0x56283bd26f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:53.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.301+0000 7fa307400640 1 -- 172.21.15.16:0/249623981 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56283bd21c20 con 0x56283b95b1b0 2024-08-31T06:16:53.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.301+0000 7fa306a00640 1 --2- 172.21.15.16:0/249623981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56283b963390 0x56283bd26f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:16:53.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.301+0000 7fa307400640 1 --2- 172.21.15.16:0/249623981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56283b95b1b0 0x56283bd305d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8004810 tx=0x7fa2f8008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64c7e7246d122dac server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:53.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.302+0000 7fa2f3400640 1 -- 172.21.15.16:0/249623981 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f8015040 con 0x56283b95b1b0 2024-08-31T06:16:53.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.302+0000 7fa2f3400640 1 -- 172.21.15.16:0/249623981 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2f800cdc0 con 0x56283b95b1b0 2024-08-31T06:16:53.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.302+0000 7fa2f3400640 1 -- 172.21.15.16:0/249623981 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f801a6b0 con 0x56283b95b1b0 2024-08-31T06:16:53.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.303+0000 7fa3147fd740 1 -- 172.21.15.16:0/249623981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56283b95b1b0 msgr2=0x56283bd305d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:53.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.303+0000 7fa3147fd740 1 --2- 172.21.15.16:0/249623981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56283b95b1b0 0x56283bd305d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8004810 tx=0x7fa2f8008040 comp rx=0 tx=0).stop 2024-08-31T06:16:53.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.303+0000 7fa3147fd740 1 -- 172.21.15.16:0/249623981 shutdown_connections 2024-08-31T06:16:53.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.303+0000 7fa3147fd740 1 --2- 172.21.15.16:0/249623981 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56283b984070 0x56283bd27450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:53.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.303+0000 7fa3147fd740 1 --2- 172.21.15.16:0/249623981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56283b963390 0x56283bd26f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:53.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.303+0000 7fa3147fd740 1 --2- 172.21.15.16:0/249623981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56283b95b1b0 0x56283bd305d0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:53.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.303+0000 7fa3147fd740 1 -- 172.21.15.16:0/249623981 >> 172.21.15.16:0/249623981 conn(0x56283b978460 msgr2=0x56283bd33ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:53.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.307+0000 7fa3147fd740 1 -- 172.21.15.16:0/249623981 shutdown_connections 2024-08-31T06:16:53.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.307+0000 7fa3147fd740 1 -- 172.21.15.16:0/249623981 wait complete. 2024-08-31T06:16:53.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.308+0000 7fa3147fd740 1 -- start start 2024-08-31T06:16:53.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.308+0000 7fa3147fd740 1 Processor -- start 2024-08-31T06:16:53.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.309+0000 7fa3147fd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56283b95b1b0 0x56283bd39b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:53.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.309+0000 7fa3147fd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56283b963390 0x56283bd3a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:53.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.309+0000 7fa3147fd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56283b984070 0x56283bdb1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:53.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.309+0000 7fa3147fd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa3081b3ea0 con 0x56283b95b1b0 2024-08-31T06:16:53.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.309+0000 7fa3147fd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa3081b5310 con 0x56283b963390 2024-08-31T06:16:53.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.309+0000 7fa3147fd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa3081b4570 con 0x56283b984070 2024-08-31T06:16:53.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.309+0000 7fa3147fd740 10 client.0 _ll_register_callbacks cb 0x7fa300010a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:16:53.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.309+0000 7fa306a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56283b963390 0x56283bd3a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:53.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.309+0000 7fa306a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56283b963390 0x56283bd3a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:54404/0 (socket says 172.21.15.16:54404) 2024-08-31T06:16:53.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.309+0000 7fa306a00640 1 -- 172.21.15.16:0/3797272209 learned_addr learned my addr 172.21.15.16:0/3797272209 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:53.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.310+0000 7fa307400640 1 --2- 172.21.15.16:0/3797272209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56283b95b1b0 0x56283bd39b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:53.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.310+0000 7fa307e00640 1 --2- 172.21.15.16:0/3797272209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56283b984070 0x56283bdb1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:53.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.311+0000 7fa307400640 1 -- 172.21.15.16:0/3797272209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56283b984070 msgr2=0x56283bdb1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:53.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.311+0000 7fa307400640 1 --2- 172.21.15.16:0/3797272209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56283b984070 0x56283bdb1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:53.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.311+0000 7fa307400640 1 -- 172.21.15.16:0/3797272209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56283b963390 msgr2=0x56283bd3a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:53.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.311+0000 7fa307400640 1 --2- 172.21.15.16:0/3797272209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56283b963390 0x56283bd3a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:53.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.311+0000 7fa307400640 1 -- 172.21.15.16:0/3797272209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56283bdb16b0 con 0x56283b95b1b0 2024-08-31T06:16:53.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.312+0000 7fa307e00640 1 --2- 172.21.15.16:0/3797272209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56283b984070 0x56283bdb1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:16:53.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.313+0000 7fa307400640 1 --2- 172.21.15.16:0/3797272209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56283b95b1b0 0x56283bd39b70 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8011870 tx=0x7fa2f800c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbfeeca3cacb216f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:53.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.313+0000 7fa2f0c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:16:53.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.314+0000 7fa2f0c00640 1 -- 172.21.15.16:0/3797272209 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f8015050 con 0x56283b95b1b0 2024-08-31T06:16:53.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.314+0000 7fa2f0c00640 1 -- 172.21.15.16:0/3797272209 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8470+0+0 (secure 0 0 0) 0x7fa2f80113f0 con 0x56283b95b1b0 2024-08-31T06:16:53.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.315+0000 7fa3147fd740 1 -- 172.21.15.16:0/3797272209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2f800cdc0 con 0x56283b95b1b0 2024-08-31T06:16:53.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.315+0000 7fa2f0c00640 1 -- 172.21.15.16:0/3797272209 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa2f8011610 con 0x56283b95b1b0 2024-08-31T06:16:53.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.315+0000 7fa3147fd740 10 client.17466 fetch_fsmap learned FSMap version 43 2024-08-31T06:16:53.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.316+0000 7fa3147fd740 1 -- 172.21.15.16:0/3797272209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56283bdb2820 con 0x56283b95b1b0 2024-08-31T06:16:53.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.316+0000 7fa2f0c00640 1 -- 172.21.15.16:0/3797272209 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2f800cdc0 con 0x56283b95b1b0 2024-08-31T06:16:53.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.316+0000 7fa2f0c00640 1 -- 172.21.15.16:0/3797272209 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fa2f8026030 con 0x56283b95b1b0 2024-08-31T06:16:53.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.316+0000 7fa3147fd740 10 client.17466 fetch_fsmap finished waiting for FSMap version 43 2024-08-31T06:16:53.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.316+0000 7fa3147fd740 10 client.17466 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:16:53.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.316+0000 7fa3147fd740 20 client.17466 populate_metadata read hostname 'smithi016' 2024-08-31T06:16:53.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.317+0000 7fa3147fd740 1 --2- 172.21.15.16:0/3797272209 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56283b917da0 0x56283bdb4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:53.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.317+0000 7fa3147fd740 4 client.17466 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T06:16:53.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.317+0000 7fa3147fd740 1 -- 172.21.15.16:0/3797272209 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56283bc9e160 con 0x56283b917da0 2024-08-31T06:16:53.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.317+0000 7fa306a00640 1 --2- 172.21.15.16:0/3797272209 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56283b917da0 0x56283bdb4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:53.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.319+0000 7fa2f0c00640 10 client.17466 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:16:53.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.319+0000 7fa306a00640 1 --2- 172.21.15.16:0/3797272209 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56283b917da0 0x56283bdb4c70 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=149c250c5c1871e8 server_cookie=c8eaac1729e8e23a in_seq=0 out_seq=0 2024-08-31T06:16:53.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.320+0000 7fa2f0c00640 1 -- 172.21.15.16:0/3797272209 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3978 (crc 0 0 0) 0x56283bc9e160 con 0x56283b917da0 2024-08-31T06:16:53.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.320+0000 7fa2f0c00640 10 client.17466 handle_command_reply: tid=0 2024-08-31T06:16:53.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.320+0000 7fa3147fd740 1 client.17466 shutdown 2024-08-31T06:16:53.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.322+0000 7fa3147fd740 1 -- 172.21.15.16:0/3797272209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56283b95b1b0 msgr2=0x56283bd39b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:53.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.322+0000 7fa3147fd740 1 --2- 172.21.15.16:0/3797272209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56283b95b1b0 0x56283bd39b70 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8011870 tx=0x7fa2f800c620 comp rx=0 tx=0).stop 2024-08-31T06:16:53.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.322+0000 7fa3147fd740 1 -- 172.21.15.16:0/3797272209 shutdown_connections 2024-08-31T06:16:53.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.322+0000 7fa3147fd740 1 --2- 172.21.15.16:0/3797272209 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56283b917da0 0x56283bdb4c70 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:53.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.322+0000 7fa2f0c00640 0 client.17466 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:16:53.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.322+0000 7fa3147fd740 1 --2- 172.21.15.16:0/3797272209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56283b984070 0x56283bdb1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:53.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.322+0000 7fa3147fd740 1 --2- 172.21.15.16:0/3797272209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56283b963390 0x56283bd3a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:53.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.322+0000 7fa3147fd740 1 --2- 172.21.15.16:0/3797272209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56283b95b1b0 0x56283bd39b70 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:53.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.322+0000 7fa3147fd740 1 -- 172.21.15.16:0/3797272209 >> 172.21.15.16:0/3797272209 conn(0x56283b978460 msgr2=0x56283bd33c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:53.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.323+0000 7fa3147fd740 1 -- 172.21.15.16:0/3797272209 shutdown_connections 2024-08-31T06:16:53.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.323+0000 7fa3147fd740 1 -- 172.21.15.16:0/3797272209 wait complete. 2024-08-31T06:16:53.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.323+0000 7fa3147fd740 20 client.17466 trim_cache size 0 max 16384 2024-08-31T06:16:53.328 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:16:53.328 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T06:16:53.328 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-08-31T06:16:53.329 INFO:teuthology.orchestra.run.smithi016.stdout: "004b47bb-0fe8-4d28-9d9d-78bde6a61199": { 2024-08-31T06:16:53.329 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:16:53.329 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "004b47bb-0fe8-4d28-9d9d-78bde6a61199", 2024-08-31T06:16:53.329 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-08-31T06:16:53.329 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:16:53.329 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:16:53.329 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-08-31T06:16:53.329 INFO:teuthology.orchestra.run.smithi016.stdout: "f9c4a22e-0c8a-4928-9b0d-e2ce1db5a86b": { 2024-08-31T06:16:53.329 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:15:57.496015+0000", 2024-08-31T06:16:53.329 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:16:53.329 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:16:53.329 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:16:53.329 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:16:53.329 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:16:53.330 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:16:53.330 INFO:teuthology.orchestra.run.smithi016.stdout: "a64d5301-8558-4e9c-ac9b-8d0ac7057e74": { 2024-08-31T06:16:53.330 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:14:20.338104+0000", 2024-08-31T06:16:53.330 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:16:53.330 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:16:53.330 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:16:53.330 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:16:53.330 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:16:53.330 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:16:53.330 INFO:teuthology.orchestra.run.smithi016.stdout: "e739a16e-8ace-43a8-a1ce-ac27d39ef8a3": { 2024-08-31T06:16:53.330 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:54.084646+0000", 2024-08-31T06:16:53.330 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:16:53.330 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:16:53.330 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:16:53.330 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:16:53.331 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:16:53.331 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:16:53.331 INFO:teuthology.orchestra.run.smithi016.stdout: "91b72a34-4ba9-4126-a414-8b862f8c5f78": { 2024-08-31T06:16:53.331 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:13:28.663237+0000", 2024-08-31T06:16:53.331 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:16:53.331 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:16:53.331 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:16:53.331 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:16:53.331 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:16:53.331 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:16:53.331 INFO:teuthology.orchestra.run.smithi016.stdout: "a10cce76-99ab-46b5-b3b2-07b82899b8f9": { 2024-08-31T06:16:53.331 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:11:43.967546+0000", 2024-08-31T06:16:53.331 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:16:53.331 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:16:53.331 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:16:53.331 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:16:53.332 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:16:53.332 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:16:53.332 INFO:teuthology.orchestra.run.smithi016.stdout: "c1fd6904-2e02-47fa-939a-47349b9fe36b": { 2024-08-31T06:16:53.332 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:06.678691+0000", 2024-08-31T06:16:53.332 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:16:53.332 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:16:53.332 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:16:53.332 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:16:53.332 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:16:53.332 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:16:53.332 INFO:teuthology.orchestra.run.smithi016.stdout: "78d6ef02-8551-45e8-ad60-9627eb573fc1": { 2024-08-31T06:16:53.332 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:09:13.547865+0000", 2024-08-31T06:16:53.332 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:16:53.332 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:16:53.332 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:16:53.333 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:16:53.333 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:16:53.333 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:16:53.333 INFO:teuthology.orchestra.run.smithi016.stdout: "c8455d74-2fe7-4d93-bcf2-19b419d16016": { 2024-08-31T06:16:53.333 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:08:24.035960+0000", 2024-08-31T06:16:53.333 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:16:53.333 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:16:53.333 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:16:53.333 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:16:53.333 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:16:53.333 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:16:53.333 INFO:teuthology.orchestra.run.smithi016.stdout: "0fdcd762-533d-42be-9289-b71c8e3b81b1": { 2024-08-31T06:16:53.333 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:15:06.169112+0000", 2024-08-31T06:16:53.333 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:16:53.333 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:16:53.334 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:16:53.334 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:16:53.334 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:16:53.334 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:16:53.334 INFO:teuthology.orchestra.run.smithi016.stdout: "88d7bdb8-cb1c-45e5-ad29-4cd8da6c6003": { 2024-08-31T06:16:53.334 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:07:39.648942+0000", 2024-08-31T06:16:53.334 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:16:53.334 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:16:53.334 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:16:53.334 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:16:53.334 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:16:53.334 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:16:53.334 INFO:teuthology.orchestra.run.smithi016.stdout: "004b47bb-0fe8-4d28-9d9d-78bde6a61199": { 2024-08-31T06:16:53.334 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:16:48.915926+0000", 2024-08-31T06:16:53.334 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:16:53.335 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:16:53.335 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:16:53.335 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:16:53.335 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:16:53.335 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:16:53.335 INFO:teuthology.orchestra.run.smithi016.stdout: "6a03046e-cdf4-4a14-b9bd-b560b8968467": { 2024-08-31T06:16:53.335 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:12:38.002354+0000", 2024-08-31T06:16:53.335 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:16:53.335 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:16:53.335 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:16:53.335 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:16:53.335 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:16:53.335 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:16:53.335 INFO:teuthology.orchestra.run.smithi016.stdout: "c300b7e8-f3fd-482c-a909-9329bc61e570": { 2024-08-31T06:16:53.335 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:06:49.868046+0000", 2024-08-31T06:16:53.336 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:16:53.336 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:16:53.336 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:16:53.336 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:16:53.336 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:16:53.336 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:16:53.336 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:16:53.336 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:16:53.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.324+0000 7fa30ca00640 1 -- 172.21.15.16:0/84858591 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa2ec0b4da0 msgr2=0x7fa2ec0b72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:53.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.324+0000 7fa30ca00640 1 --2- 172.21.15.16:0/84858591 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa2ec0b4da0 0x7fa2ec0b72b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa300005f70 tx=0x7fa300007980 comp rx=0 tx=0).stop 2024-08-31T06:16:53.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.324+0000 7fa30ca00640 1 -- 172.21.15.16:0/84858591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa30811c940 msgr2=0x7fa308118830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:53.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.324+0000 7fa30ca00640 1 --2- 172.21.15.16:0/84858591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa30811c940 0x7fa308118830 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4005ab0 tx=0x7fa2f40048f0 comp rx=0 tx=0).stop 2024-08-31T06:16:53.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.325+0000 7fa30ca00640 1 -- 172.21.15.16:0/84858591 shutdown_connections 2024-08-31T06:16:53.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.325+0000 7fa30ca00640 1 --2- 172.21.15.16:0/84858591 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa2ec0b4da0 0x7fa2ec0b72b0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:53.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.325+0000 7fa30ca00640 1 --2- 172.21.15.16:0/84858591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa30811c940 0x7fa308118830 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:53.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.325+0000 7fa30ca00640 1 --2- 172.21.15.16:0/84858591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa308111a70 0x7fa3081182f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:53.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.325+0000 7fa30ca00640 1 --2- 172.21.15.16:0/84858591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa30810d990 0x7fa30811c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:53.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.325+0000 7fa30ca00640 1 -- 172.21.15.16:0/84858591 >> 172.21.15.16:0/84858591 conn(0x7fa30806e790 msgr2=0x7fa30810c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:53.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.325+0000 7fa30ca00640 1 -- 172.21.15.16:0/84858591 shutdown_connections 2024-08-31T06:16:53.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:53.325+0000 7fa30ca00640 1 -- 172.21.15.16:0/84858591 wait complete. 2024-08-31T06:16:53.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:53 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:16:53.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:53 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:16:53.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:53 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:16:53.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:53 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:16:53.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:53 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:16:53.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:53 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:16:54.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:53 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:16:54.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:53 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:16:54.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:53 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:16:54.474 INFO:tasks.cephfs.filesystem:scrub status for tag:004b47bb-0fe8-4d28-9d9d-78bde6a61199 - {'path': '/', 'tag': '004b47bb-0fe8-4d28-9d9d-78bde6a61199', 'options': 'recursive,force'} 2024-08-31T06:16:54.552 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:16:54.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:54 smithi184 ceph-mon[22331]: pgmap v662: 97 pgs: 97 active+clean; 3.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 6.9 MiB/s wr, 747 op/s 2024-08-31T06:16:54.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:54 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:16:54.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:54 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:16:55.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:54 smithi155 ceph-mon[21742]: pgmap v662: 97 pgs: 97 active+clean; 3.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 6.9 MiB/s wr, 747 op/s 2024-08-31T06:16:55.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:54 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:16:55.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:54 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:16:55.129 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:54 smithi016 ceph-mon[17194]: pgmap v662: 97 pgs: 97 active+clean; 3.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 6.9 MiB/s wr, 747 op/s 2024-08-31T06:16:55.129 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:54 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:16:55.129 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:54 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:16:55.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:55.745+0000 7fa3dc600640 1 -- 172.21.15.16:0/1003790196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3d4107980 msgr2=0x7fa3d4107e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:55.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:55.746+0000 7fa3dc600640 1 --2- 172.21.15.16:0/1003790196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3d4107980 0x7fa3d4107e60 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc009be0 tx=0x7fa3cc02a6f0 comp rx=0 tx=0).stop 2024-08-31T06:16:55.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:55.746+0000 7fa3dc600640 1 -- 172.21.15.16:0/1003790196 shutdown_connections 2024-08-31T06:16:55.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:55.746+0000 7fa3dc600640 1 --2- 172.21.15.16:0/1003790196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3d4107980 0x7fa3d4107e60 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:55.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:55.746+0000 7fa3dc600640 1 --2- 172.21.15.16:0/1003790196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3d4102e90 0x7fa3d41073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:55.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:55.746+0000 7fa3dc600640 1 --2- 172.21.15.16:0/1003790196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3d4102540 0x7fa3d4102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:55.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:55.746+0000 7fa3dc600640 1 -- 172.21.15.16:0/1003790196 >> 172.21.15.16:0/1003790196 conn(0x7fa3d40794d0 msgr2=0x7fa3d40798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:55.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:55.746+0000 7fa3dc600640 1 -- 172.21.15.16:0/1003790196 shutdown_connections 2024-08-31T06:16:55.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:55.747+0000 7fa3dc600640 1 -- 172.21.15.16:0/1003790196 wait complete. 2024-08-31T06:16:55.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:55.748+0000 7fa3dc600640 1 Processor -- start 2024-08-31T06:16:55.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:55.748+0000 7fa3dc600640 1 -- start start 2024-08-31T06:16:55.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:55.749+0000 7fa3dc600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3d4102540 0x7fa3d41a0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:55.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:55.749+0000 7fa3dc600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3d4102e90 0x7fa3d41a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:55.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:55.749+0000 7fa3dc600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3d4107980 0x7fa3d41a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:55.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:55.749+0000 7fa3dc600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d4119e20 con 0x7fa3d4107980 2024-08-31T06:16:55.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:55.749+0000 7fa3dc600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d4119ca0 con 0x7fa3d4102e90 2024-08-31T06:16:55.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:55.749+0000 7fa3dc600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d4119fa0 con 0x7fa3d4102540 2024-08-31T06:16:55.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:55.749+0000 7fa3d9c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3d4102540 0x7fa3d41a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:55.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:55.749+0000 7fa3d9c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3d4102540 0x7fa3d41a0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:44232/0 (socket says 172.21.15.16:44232) 2024-08-31T06:16:55.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:55.749+0000 7fa3d9c00640 1 -- 172.21.15.16:0/1951509310 learned_addr learned my addr 172.21.15.16:0/1951509310 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:55.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:55.750+0000 7fa3da600640 1 --2- 172.21.15.16:0/1951509310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3d4107980 0x7fa3d41a60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:55.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:55.750+0000 7fa3d9c00640 1 -- 172.21.15.16:0/1951509310 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3d4102e90 msgr2=0x7fa3d41a0e10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-31T06:16:55.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:55.750+0000 7fa3d9c00640 1 --2- 172.21.15.16:0/1951509310 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3d4102e90 0x7fa3d41a0e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:55.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:55.750+0000 7fa3d9c00640 1 -- 172.21.15.16:0/1951509310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3d4107980 msgr2=0x7fa3d41a60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:55.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:55.750+0000 7fa3d9c00640 1 --2- 172.21.15.16:0/1951509310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3d4107980 0x7fa3d41a60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:55.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:55.750+0000 7fa3d9c00640 1 -- 172.21.15.16:0/1951509310 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d41a67e0 con 0x7fa3d4102540 2024-08-31T06:16:55.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:55.750+0000 7fa3d9c00640 1 --2- 172.21.15.16:0/1951509310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3d4102540 0x7fa3d41a0870 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa3c800e9a0 tx=0x7fa3c800ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:55.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:55.750+0000 7fa3d2000640 1 -- 172.21.15.16:0/1951509310 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c800cd80 con 0x7fa3d4102540 2024-08-31T06:16:55.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:55.750+0000 7fa3d2000640 1 -- 172.21.15.16:0/1951509310 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3c80108e0 con 0x7fa3d4102540 2024-08-31T06:16:55.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:55.750+0000 7fa3d2000640 1 -- 172.21.15.16:0/1951509310 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8018960 con 0x7fa3d4102540 2024-08-31T06:16:55.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:55.751+0000 7fa3dc600640 1 -- 172.21.15.16:0/1951509310 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3d41a6ad0 con 0x7fa3d4102540 2024-08-31T06:16:55.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:55.751+0000 7fa3dc600640 1 -- 172.21.15.16:0/1951509310 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3d41ad230 con 0x7fa3d4102540 2024-08-31T06:16:55.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:55.753+0000 7fa3dc600640 1 -- 172.21.15.16:0/1951509310 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3d41073c0 con 0x7fa3d4102540 2024-08-31T06:16:55.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:55.758+0000 7fa3d2000640 1 -- 172.21.15.16:0/1951509310 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fa3c8010430 con 0x7fa3d4102540 2024-08-31T06:16:55.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:55.761+0000 7fa3d2000640 1 --2- 172.21.15.16:0/1951509310 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa3bc0b4d20 0x7fa3bc0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:55.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:55.761+0000 7fa3d2000640 1 -- 172.21.15.16:0/1951509310 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8470+0+0 (secure 0 0 0) 0x7fa3c80e5af0 con 0x7fa3d4102540 2024-08-31T06:16:55.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:55.762+0000 7fa3d9200640 1 --2- 172.21.15.16:0/1951509310 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa3bc0b4d20 0x7fa3bc0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:55.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:55.763+0000 7fa3d2000640 1 -- 172.21.15.16:0/1951509310 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fa3c80ac8b0 con 0x7fa3d4102540 2024-08-31T06:16:55.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:55.764+0000 7fa3d9200640 1 --2- 172.21.15.16:0/1951509310 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa3bc0b4d20 0x7fa3bc0b7230 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa3d406a190 tx=0x7fa3c0005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:56.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:55 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:16:56.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:55 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:16:56.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:55 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:16:56.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:55 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:16:56.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:55 smithi155 ceph-mon[21742]: pgmap v663: 97 pgs: 97 active+clean; 3.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 8.9 MiB/s wr, 899 op/s 2024-08-31T06:16:56.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:56.158+0000 7fa3dc600640 1 -- 172.21.15.16:0/1951509310 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3d41a19a0 con 0x7fa3d4102540 2024-08-31T06:16:56.160 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:55 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:16:56.160 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:55 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:16:56.160 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:55 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:16:56.160 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:55 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:16:56.160 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:55 smithi016 ceph-mon[17194]: pgmap v663: 97 pgs: 97 active+clean; 3.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 8.9 MiB/s wr, 899 op/s 2024-08-31T06:16:56.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:56.159+0000 7fa3d2000640 1 -- 172.21.15.16:0/1951509310 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11920 (secure 0 0 0) 0x7fa3c80ac1b0 con 0x7fa3d4102540 2024-08-31T06:16:56.161 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:16:56.162 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 43 2024-08-31T06:16:56.162 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":43,"btime":"2024-08-31T06:16:03:711285+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:16:03.340730+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24737,24724,24712]},"id":1}]} 2024-08-31T06:16:56.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:56.164+0000 7fa3dc600640 1 -- 172.21.15.16:0/1951509310 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa3bc0b4d20 msgr2=0x7fa3bc0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:56.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:56.164+0000 7fa3dc600640 1 --2- 172.21.15.16:0/1951509310 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa3bc0b4d20 0x7fa3bc0b7230 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa3d406a190 tx=0x7fa3c0005f90 comp rx=0 tx=0).stop 2024-08-31T06:16:56.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:56.164+0000 7fa3dc600640 1 -- 172.21.15.16:0/1951509310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3d4102540 msgr2=0x7fa3d41a0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:56.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:56.164+0000 7fa3dc600640 1 --2- 172.21.15.16:0/1951509310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3d4102540 0x7fa3d41a0870 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa3c800e9a0 tx=0x7fa3c800ee60 comp rx=0 tx=0).stop 2024-08-31T06:16:56.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:56.165+0000 7fa3dc600640 1 -- 172.21.15.16:0/1951509310 shutdown_connections 2024-08-31T06:16:56.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:56.165+0000 7fa3dc600640 1 --2- 172.21.15.16:0/1951509310 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa3bc0b4d20 0x7fa3bc0b7230 secure :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa3d406a190 tx=0x7fa3c0005f90 comp rx=0 tx=0).stop 2024-08-31T06:16:56.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:56.165+0000 7fa3dc600640 1 --2- 172.21.15.16:0/1951509310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3d4107980 0x7fa3d41a60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:56.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:56.165+0000 7fa3dc600640 1 --2- 172.21.15.16:0/1951509310 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3d4102e90 0x7fa3d41a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:56.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:56.165+0000 7fa3dc600640 1 --2- 172.21.15.16:0/1951509310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3d4102540 0x7fa3d41a0870 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:56.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:56.165+0000 7fa3dc600640 1 -- 172.21.15.16:0/1951509310 >> 172.21.15.16:0/1951509310 conn(0x7fa3d40794d0 msgr2=0x7fa3d4105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:56.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:56.165+0000 7fa3dc600640 1 -- 172.21.15.16:0/1951509310 shutdown_connections 2024-08-31T06:16:56.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:56.165+0000 7fa3dc600640 1 -- 172.21.15.16:0/1951509310 wait complete. 2024-08-31T06:16:56.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:55 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:16:56.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:55 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:16:56.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:55 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:16:56.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:55 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:16:56.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:55 smithi184 ceph-mon[22331]: pgmap v663: 97 pgs: 97 active+clean; 3.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 8.9 MiB/s wr, 899 op/s 2024-08-31T06:16:56.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 570.0 s, 1638.3 tps, lat 60.969 ms stddev 74.171 2024-08-31T06:16:57.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:56 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/1951509310' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:16:57.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:56 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/1951509310' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:16:57.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:56 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/1951509310' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:16:57.860 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24737 quiesce db / --timeout 90 --expiration 135.5 --await-for 45.0' 2024-08-31T06:16:57.861 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24737 quiesce db / --timeout 90 --expiration 135.5 --await-for 45.0 2024-08-31T06:16:58.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:57 smithi155 ceph-mon[21742]: pgmap v664: 97 pgs: 97 active+clean; 3.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 6.8 MiB/s wr, 711 op/s 2024-08-31T06:16:58.167 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:57 smithi016 ceph-mon[17194]: pgmap v664: 97 pgs: 97 active+clean; 3.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 6.8 MiB/s wr, 711 op/s 2024-08-31T06:16:58.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:57 smithi184 ceph-mon[22331]: pgmap v664: 97 pgs: 97 active+clean; 3.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 6.8 MiB/s wr, 711 op/s 2024-08-31T06:16:58.649 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:16:58.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:58 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:16:58.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:58 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:16:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.912+0000 7fc639000640 1 -- 172.21.15.16:0/209765996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc63410fe60 msgr2=0x7fc634112280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.912+0000 7fc639000640 1 --2- 172.21.15.16:0/209765996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc63410fe60 0x7fc634112280 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc62c009ca0 tx=0x7fc62c02a940 comp rx=0 tx=0).stop 2024-08-31T06:16:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.912+0000 7fc639000640 1 -- 172.21.15.16:0/209765996 shutdown_connections 2024-08-31T06:16:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.912+0000 7fc639000640 1 --2- 172.21.15.16:0/209765996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc63410fe60 0x7fc634112280 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.912+0000 7fc639000640 1 --2- 172.21.15.16:0/209765996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc634102c00 0x7fc63410f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.912+0000 7fc639000640 1 --2- 172.21.15.16:0/209765996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6341022b0 0x7fc6341026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.912+0000 7fc639000640 1 -- 172.21.15.16:0/209765996 >> 172.21.15.16:0/209765996 conn(0x7fc63407a4e0 msgr2=0x7fc63407a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:58.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.912+0000 7fc639000640 1 -- 172.21.15.16:0/209765996 shutdown_connections 2024-08-31T06:16:58.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.912+0000 7fc639000640 1 -- 172.21.15.16:0/209765996 wait complete. 2024-08-31T06:16:58.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.914+0000 7fc639000640 1 Processor -- start 2024-08-31T06:16:58.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.914+0000 7fc639000640 1 -- start start 2024-08-31T06:16:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.915+0000 7fc639000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6341022b0 0x7fc63410cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.915+0000 7fc639000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc634102c00 0x7fc63410d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.915+0000 7fc639000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc63410fe60 0x7fc63410d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:58.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.915+0000 7fc639000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc6341159a0 con 0x7fc63410fe60 2024-08-31T06:16:58.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.915+0000 7fc639000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc634115820 con 0x7fc6341022b0 2024-08-31T06:16:58.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.915+0000 7fc639000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc634115b20 con 0x7fc634102c00 2024-08-31T06:16:58.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.915+0000 7fc631e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6341022b0 0x7fc63410cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:58.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.915+0000 7fc631e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6341022b0 0x7fc63410cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51916/0 (socket says 172.21.15.16:51916) 2024-08-31T06:16:58.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.915+0000 7fc631e00640 1 -- 172.21.15.16:0/1034939816 learned_addr learned my addr 172.21.15.16:0/1034939816 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:58.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.915+0000 7fc631400640 1 --2- 172.21.15.16:0/1034939816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc634102c00 0x7fc63410d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.915+0000 7fc631e00640 1 -- 172.21.15.16:0/1034939816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc634102c00 msgr2=0x7fc63410d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.915+0000 7fc631e00640 1 --2- 172.21.15.16:0/1034939816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc634102c00 0x7fc63410d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.915+0000 7fc631e00640 1 -- 172.21.15.16:0/1034939816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc63410fe60 msgr2=0x7fc63410d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.915+0000 7fc631e00640 1 --2- 172.21.15.16:0/1034939816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc63410fe60 0x7fc63410d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.915+0000 7fc631e00640 1 -- 172.21.15.16:0/1034939816 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6341ac7a0 con 0x7fc6341022b0 2024-08-31T06:16:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.916+0000 7fc631e00640 1 --2- 172.21.15.16:0/1034939816 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6341022b0 0x7fc63410cd40 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc62400d950 tx=0x7fc62400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.916+0000 7fc62aa00640 1 -- 172.21.15.16:0/1034939816 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc624017070 con 0x7fc6341022b0 2024-08-31T06:16:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.916+0000 7fc62aa00640 1 -- 172.21.15.16:0/1034939816 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc624009c30 con 0x7fc6341022b0 2024-08-31T06:16:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.916+0000 7fc62aa00640 1 -- 172.21.15.16:0/1034939816 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62400f620 con 0x7fc6341022b0 2024-08-31T06:16:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.916+0000 7fc639000640 1 -- 172.21.15.16:0/1034939816 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6341aca90 con 0x7fc6341022b0 2024-08-31T06:16:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.917+0000 7fc639000640 1 -- 172.21.15.16:0/1034939816 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6341acf90 con 0x7fc6341022b0 2024-08-31T06:16:58.923 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.921+0000 7fc62aa00640 1 -- 172.21.15.16:0/1034939816 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fc62400f7c0 con 0x7fc6341022b0 2024-08-31T06:16:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.922+0000 7fc640087740 1 Processor -- start 2024-08-31T06:16:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.923+0000 7fc62aa00640 1 --2- 172.21.15.16:0/1034939816 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc6180b4e70 0x7fc6180b7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.923+0000 7fc640087740 1 -- start start 2024-08-31T06:16:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.923+0000 7fc62aa00640 1 -- 172.21.15.16:0/1034939816 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8470+0+0 (secure 0 0 0) 0x7fc6240e5130 con 0x7fc6341022b0 2024-08-31T06:16:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.923+0000 7fc640087740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639878431b0 0x563987be6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.923+0000 7fc640087740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56398781a2f0 0x563987be6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.923+0000 7fc640087740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639878224d0 0x563987be7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.923+0000 7fc640087740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563987b5bf70 con 0x5639878224d0 2024-08-31T06:16:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.923+0000 7fc640087740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563987b4b500 con 0x5639878431b0 2024-08-31T06:16:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.923+0000 7fc640087740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563987b4f4b0 con 0x56398781a2f0 2024-08-31T06:16:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.923+0000 7fc631e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639878431b0 0x563987be6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.923+0000 7fc631e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639878431b0 0x563987be6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51930/0 (socket says 172.21.15.16:51930) 2024-08-31T06:16:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.924+0000 7fc631e00640 1 -- 172.21.15.16:0/523589458 learned_addr learned my addr 172.21.15.16:0/523589458 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.924+0000 7fc631e00640 1 -- 172.21.15.16:0/523589458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56398781a2f0 msgr2=0x563987be6bd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:16:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.924+0000 7fc631e00640 1 --2- 172.21.15.16:0/523589458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56398781a2f0 0x563987be6bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.924+0000 7fc631e00640 1 -- 172.21.15.16:0/523589458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639878224d0 msgr2=0x563987be7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.924+0000 7fc631e00640 1 --2- 172.21.15.16:0/523589458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639878224d0 0x563987be7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.924+0000 7fc631e00640 1 -- 172.21.15.16:0/523589458 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563987b05c10 con 0x5639878431b0 2024-08-31T06:16:58.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.924+0000 7fc632800640 1 --2- 172.21.15.16:0/1034939816 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc6180b4e70 0x7fc6180b7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:58.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.924+0000 7fc631e00640 1 --2- 172.21.15.16:0/523589458 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639878431b0 0x563987be6690 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc6240a7870 tx=0x7fc6240ab270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69123cbd345bf9a7 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:58.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.925+0000 7fc61f400640 1 -- 172.21.15.16:0/523589458 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc624020040 con 0x5639878431b0 2024-08-31T06:16:58.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.925+0000 7fc61f400640 1 -- 172.21.15.16:0/523589458 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6240ab7f0 con 0x5639878431b0 2024-08-31T06:16:58.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.925+0000 7fc61f400640 1 -- 172.21.15.16:0/523589458 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6240a7160 con 0x5639878431b0 2024-08-31T06:16:58.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.926+0000 7fc640087740 1 -- 172.21.15.16:0/523589458 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639878431b0 msgr2=0x563987be6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:58.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.926+0000 7fc640087740 1 --2- 172.21.15.16:0/523589458 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639878431b0 0x563987be6690 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc6240a7870 tx=0x7fc6240ab270 comp rx=0 tx=0).stop 2024-08-31T06:16:58.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.926+0000 7fc632800640 1 --2- 172.21.15.16:0/1034939816 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc6180b4e70 0x7fc6180b7380 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc62c02ae60 tx=0x7fc62c005d80 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:58.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.926+0000 7fc640087740 1 -- 172.21.15.16:0/523589458 shutdown_connections 2024-08-31T06:16:58.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.926+0000 7fc640087740 1 --2- 172.21.15.16:0/523589458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56398781a2f0 0x563987be6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:58.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.926+0000 7fc640087740 1 --2- 172.21.15.16:0/523589458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639878224d0 0x563987be7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:58.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.926+0000 7fc640087740 1 --2- 172.21.15.16:0/523589458 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639878431b0 0x563987be6690 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:58.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.926+0000 7fc640087740 1 -- 172.21.15.16:0/523589458 >> 172.21.15.16:0/523589458 conn(0x563987837300 msgr2=0x563987be4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:58.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.926+0000 7fc640087740 1 -- 172.21.15.16:0/523589458 shutdown_connections 2024-08-31T06:16:58.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.926+0000 7fc640087740 1 -- 172.21.15.16:0/523589458 wait complete. 2024-08-31T06:16:58.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.928+0000 7fc640087740 1 -- start start 2024-08-31T06:16:58.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.928+0000 7fc640087740 1 Processor -- start 2024-08-31T06:16:58.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.929+0000 7fc640087740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56398781a2f0 0x563987bf1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:58.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.929+0000 7fc640087740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639878224d0 0x563987bf21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:58.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.929+0000 7fc640087740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639878431b0 0x563987bf2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:58.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.929+0000 7fc640087740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563987a84ab0 con 0x5639878224d0 2024-08-31T06:16:58.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.929+0000 7fc640087740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563987af2320 con 0x56398781a2f0 2024-08-31T06:16:58.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.929+0000 7fc640087740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563987bf4d50 con 0x5639878431b0 2024-08-31T06:16:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.929+0000 7fc640087740 10 client.0 _ll_register_callbacks cb 0x563987b2be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:16:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.929+0000 7fc631e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56398781a2f0 0x563987bf1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.929+0000 7fc631e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56398781a2f0 0x563987bf1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51942/0 (socket says 172.21.15.16:51942) 2024-08-31T06:16:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.929+0000 7fc631e00640 1 -- 172.21.15.16:0/3385252179 learned_addr learned my addr 172.21.15.16:0/3385252179 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.929+0000 7fc631400640 1 --2- 172.21.15.16:0/3385252179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639878224d0 0x563987bf21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.930+0000 7fc632800640 1 --2- 172.21.15.16:0/3385252179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639878431b0 0x563987bf2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.930+0000 7fc631e00640 1 -- 172.21.15.16:0/3385252179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639878431b0 msgr2=0x563987bf2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.930+0000 7fc631e00640 1 --2- 172.21.15.16:0/3385252179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639878431b0 0x563987bf2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.930+0000 7fc631e00640 1 -- 172.21.15.16:0/3385252179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639878224d0 msgr2=0x563987bf21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.930+0000 7fc631e00640 1 --2- 172.21.15.16:0/3385252179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639878224d0 0x563987bf21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.930+0000 7fc631e00640 1 -- 172.21.15.16:0/3385252179 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639878ae790 con 0x56398781a2f0 2024-08-31T06:16:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.930+0000 7fc632800640 1 --2- 172.21.15.16:0/3385252179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639878431b0 0x563987bf2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:16:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.930+0000 7fc631e00640 1 --2- 172.21.15.16:0/3385252179 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56398781a2f0 0x563987bf1ca0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc6240a8a60 tx=0x7fc624005f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=937926539fa43173 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.930+0000 7fc613e00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:16:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.930+0000 7fc613e00640 1 -- 172.21.15.16:0/3385252179 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc624020050 con 0x56398781a2f0 2024-08-31T06:16:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.930+0000 7fc613e00640 1 -- 172.21.15.16:0/3385252179 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8470+0+0 (secure 0 0 0) 0x7fc6240aac60 con 0x56398781a2f0 2024-08-31T06:16:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.930+0000 7fc640087740 1 -- 172.21.15.16:0/3385252179 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6240ab7f0 con 0x56398781a2f0 2024-08-31T06:16:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.930+0000 7fc613e00640 1 -- 172.21.15.16:0/3385252179 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc6240b0060 con 0x56398781a2f0 2024-08-31T06:16:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.930+0000 7fc640087740 10 client.27779 fetch_fsmap learned FSMap version 43 2024-08-31T06:16:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.930+0000 7fc640087740 1 -- 172.21.15.16:0/3385252179 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x563987927620 con 0x56398781a2f0 2024-08-31T06:16:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.931+0000 7fc613e00640 1 -- 172.21.15.16:0/3385252179 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6240ab7f0 con 0x56398781a2f0 2024-08-31T06:16:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.931+0000 7fc613e00640 1 -- 172.21.15.16:0/3385252179 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fc6240a7160 con 0x56398781a2f0 2024-08-31T06:16:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.931+0000 7fc640087740 10 client.27779 fetch_fsmap finished waiting for FSMap version 43 2024-08-31T06:16:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.931+0000 7fc640087740 10 client.27779 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:16:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.931+0000 7fc640087740 20 client.27779 populate_metadata read hostname 'smithi016' 2024-08-31T06:16:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.931+0000 7fc640087740 1 --2- 172.21.15.16:0/3385252179 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x563987801d80 0x563987c03b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.931+0000 7fc640087740 4 client.27779 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:16:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.931+0000 7fc640087740 1 -- 172.21.15.16:0/3385252179 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563987b5dec0 con 0x563987801d80 2024-08-31T06:16:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.931+0000 7fc631400640 1 --2- 172.21.15.16:0/3385252179 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x563987801d80 0x563987c03b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.932+0000 7fc631400640 1 --2- 172.21.15.16:0/3385252179 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x563987801d80 0x563987c03b90 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7206058c0acd1e2 server_cookie=1a6303a34b7c147b in_seq=0 out_seq=0 2024-08-31T06:16:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.932+0000 7fc613e00640 10 client.27779 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:16:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.933+0000 7fc613e00640 1 -- 172.21.15.16:0/3385252179 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563987b5dec0 con 0x563987801d80 2024-08-31T06:16:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.933+0000 7fc613e00640 10 client.27779 handle_command_reply: tid=0 2024-08-31T06:16:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.933+0000 7fc640087740 1 client.27779 shutdown 2024-08-31T06:16:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.934+0000 7fc640087740 1 -- 172.21.15.16:0/3385252179 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56398781a2f0 msgr2=0x563987bf1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.934+0000 7fc640087740 1 --2- 172.21.15.16:0/3385252179 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56398781a2f0 0x563987bf1ca0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc6240a8a60 tx=0x7fc624005f70 comp rx=0 tx=0).stop 2024-08-31T06:16:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.934+0000 7fc640087740 1 -- 172.21.15.16:0/3385252179 shutdown_connections 2024-08-31T06:16:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.934+0000 7fc640087740 1 --2- 172.21.15.16:0/3385252179 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x563987801d80 0x563987c03b90 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.934+0000 7fc640087740 1 --2- 172.21.15.16:0/3385252179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639878431b0 0x563987bf2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.934+0000 7fc640087740 1 --2- 172.21.15.16:0/3385252179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639878224d0 0x563987bf21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.934+0000 7fc640087740 1 --2- 172.21.15.16:0/3385252179 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56398781a2f0 0x563987bf1ca0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.934+0000 7fc613e00640 0 client.27779 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:16:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.934+0000 7fc640087740 1 -- 172.21.15.16:0/3385252179 >> 172.21.15.16:0/3385252179 conn(0x563987837300 msgr2=0x563987be76c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.934+0000 7fc640087740 1 -- 172.21.15.16:0/3385252179 shutdown_connections 2024-08-31T06:16:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.934+0000 7fc640087740 1 -- 172.21.15.16:0/3385252179 wait complete. 2024-08-31T06:16:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.935+0000 7fc640087740 20 client.27779 trim_cache size 0 max 16384 2024-08-31T06:16:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.958+0000 7fc640087740 1 Processor -- start 2024-08-31T06:16:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.958+0000 7fc640087740 1 -- start start 2024-08-31T06:16:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.958+0000 7fc640087740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56398781a2f0 0x563987bf2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.958+0000 7fc640087740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639878224d0 0x563987bf38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.959+0000 7fc640087740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639878431b0 0x563987be3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.959+0000 7fc640087740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc6341b8380 con 0x5639878224d0 2024-08-31T06:16:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.959+0000 7fc640087740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc6341b5bf0 con 0x5639878431b0 2024-08-31T06:16:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.959+0000 7fc640087740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc6341b3a60 con 0x56398781a2f0 2024-08-31T06:16:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.959+0000 7fc631e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56398781a2f0 0x563987bf2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.959+0000 7fc631e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56398781a2f0 0x563987bf2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:51434/0 (socket says 172.21.15.16:51434) 2024-08-31T06:16:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.959+0000 7fc631e00640 1 -- 172.21.15.16:0/815809117 learned_addr learned my addr 172.21.15.16:0/815809117 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.959+0000 7fc632800640 1 --2- 172.21.15.16:0/815809117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639878431b0 0x563987be3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.959+0000 7fc631400640 1 --2- 172.21.15.16:0/815809117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639878224d0 0x563987bf38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.959+0000 7fc631e00640 1 -- 172.21.15.16:0/815809117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639878431b0 msgr2=0x563987be3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.959+0000 7fc631e00640 1 --2- 172.21.15.16:0/815809117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639878431b0 0x563987be3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.959+0000 7fc631e00640 1 -- 172.21.15.16:0/815809117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639878224d0 msgr2=0x563987bf38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.959+0000 7fc631e00640 1 --2- 172.21.15.16:0/815809117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639878224d0 0x563987bf38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.959+0000 7fc631e00640 1 -- 172.21.15.16:0/815809117 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563987bec290 con 0x56398781a2f0 2024-08-31T06:16:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.959+0000 7fc631400640 1 --2- 172.21.15.16:0/815809117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639878224d0 0x563987bf38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:16:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.960+0000 7fc631e00640 1 --2- 172.21.15.16:0/815809117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56398781a2f0 0x563987bf2180 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc62400bfd0 tx=0x7fc6240ab270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe7d614f390da6a9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:58.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.960+0000 7fc61ea00640 1 -- 172.21.15.16:0/815809117 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc624020070 con 0x56398781a2f0 2024-08-31T06:16:58.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.960+0000 7fc61ea00640 1 -- 172.21.15.16:0/815809117 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6240ab7f0 con 0x56398781a2f0 2024-08-31T06:16:58.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.960+0000 7fc61ea00640 1 -- 172.21.15.16:0/815809117 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6240c61a0 con 0x56398781a2f0 2024-08-31T06:16:58.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.961+0000 7fc640087740 1 -- 172.21.15.16:0/815809117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56398781a2f0 msgr2=0x563987bf2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:58.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.961+0000 7fc640087740 1 --2- 172.21.15.16:0/815809117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56398781a2f0 0x563987bf2180 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc62400bfd0 tx=0x7fc6240ab270 comp rx=0 tx=0).stop 2024-08-31T06:16:58.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.961+0000 7fc640087740 1 -- 172.21.15.16:0/815809117 shutdown_connections 2024-08-31T06:16:58.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.961+0000 7fc640087740 1 --2- 172.21.15.16:0/815809117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639878431b0 0x563987be3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:58.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.961+0000 7fc640087740 1 --2- 172.21.15.16:0/815809117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639878224d0 0x563987bf38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:58.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.961+0000 7fc640087740 1 --2- 172.21.15.16:0/815809117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56398781a2f0 0x563987bf2180 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:58.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.961+0000 7fc640087740 1 -- 172.21.15.16:0/815809117 >> 172.21.15.16:0/815809117 conn(0x563987837300 msgr2=0x563987bf31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:58.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.961+0000 7fc640087740 1 -- 172.21.15.16:0/815809117 shutdown_connections 2024-08-31T06:16:58.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.962+0000 7fc640087740 1 -- 172.21.15.16:0/815809117 wait complete. 2024-08-31T06:16:58.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.962+0000 7fc640087740 1 -- start start 2024-08-31T06:16:58.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.962+0000 7fc640087740 1 Processor -- start 2024-08-31T06:16:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.963+0000 7fc640087740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56398781a2f0 0x563987c04440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.963+0000 7fc640087740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639878224d0 0x563987c04a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.963+0000 7fc640087740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639878431b0 0x563987c6c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.963+0000 7fc640087740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc6341b40b0 con 0x5639878224d0 2024-08-31T06:16:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.963+0000 7fc640087740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc6341b5520 con 0x56398781a2f0 2024-08-31T06:16:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.963+0000 7fc640087740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc6341b4780 con 0x5639878431b0 2024-08-31T06:16:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.963+0000 7fc640087740 10 client.0 _ll_register_callbacks cb 0x7fc624092f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:16:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.963+0000 7fc631e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56398781a2f0 0x563987c04440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.963+0000 7fc631e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56398781a2f0 0x563987c04440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51964/0 (socket says 172.21.15.16:51964) 2024-08-31T06:16:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.963+0000 7fc631e00640 1 -- 172.21.15.16:0/1884548702 learned_addr learned my addr 172.21.15.16:0/1884548702 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.963+0000 7fc631400640 1 --2- 172.21.15.16:0/1884548702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639878224d0 0x563987c04a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.964+0000 7fc632800640 1 --2- 172.21.15.16:0/1884548702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639878431b0 0x563987c6c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.964+0000 7fc631e00640 1 -- 172.21.15.16:0/1884548702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639878431b0 msgr2=0x563987c6c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.964+0000 7fc631e00640 1 --2- 172.21.15.16:0/1884548702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639878431b0 0x563987c6c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.964+0000 7fc631e00640 1 -- 172.21.15.16:0/1884548702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639878224d0 msgr2=0x563987c04a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.964+0000 7fc631e00640 1 --2- 172.21.15.16:0/1884548702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639878224d0 0x563987c04a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.964+0000 7fc631e00640 1 -- 172.21.15.16:0/1884548702 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563987c6c9a0 con 0x56398781a2f0 2024-08-31T06:16:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.964+0000 7fc631e00640 1 --2- 172.21.15.16:0/1884548702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56398781a2f0 0x563987c04440 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc6240b0220 tx=0x7fc6240aac60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=628d7a8be29c7c28 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.964+0000 7fc613e00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:16:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.964+0000 7fc613e00640 1 -- 172.21.15.16:0/1884548702 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6240cc060 con 0x56398781a2f0 2024-08-31T06:16:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.964+0000 7fc613e00640 1 -- 172.21.15.16:0/1884548702 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8470+0+0 (secure 0 0 0) 0x7fc6240d21c0 con 0x56398781a2f0 2024-08-31T06:16:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.964+0000 7fc640087740 1 -- 172.21.15.16:0/1884548702 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6240ab7f0 con 0x56398781a2f0 2024-08-31T06:16:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.965+0000 7fc613e00640 1 -- 172.21.15.16:0/1884548702 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc624020070 con 0x56398781a2f0 2024-08-31T06:16:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.965+0000 7fc640087740 10 client.27785 fetch_fsmap learned FSMap version 43 2024-08-31T06:16:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.965+0000 7fc640087740 1 -- 172.21.15.16:0/1884548702 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x563987c6c9a0 con 0x56398781a2f0 2024-08-31T06:16:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.965+0000 7fc613e00640 1 -- 172.21.15.16:0/1884548702 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6240ab7f0 con 0x56398781a2f0 2024-08-31T06:16:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.965+0000 7fc613e00640 1 -- 172.21.15.16:0/1884548702 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fc6240b11f0 con 0x56398781a2f0 2024-08-31T06:16:58.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.966+0000 7fc640087740 10 client.27785 fetch_fsmap finished waiting for FSMap version 43 2024-08-31T06:16:58.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.966+0000 7fc640087740 10 client.27785 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:16:58.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.966+0000 7fc640087740 20 client.27785 populate_metadata read hostname 'smithi016' 2024-08-31T06:16:58.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.966+0000 7fc640087740 1 --2- 172.21.15.16:0/1884548702 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5639877d71c0 0x563987c6ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:58.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.966+0000 7fc640087740 4 client.27785 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:16:58.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.966+0000 7fc640087740 1 -- 172.21.15.16:0/1884548702 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563987b59960 con 0x5639877d71c0 2024-08-31T06:16:58.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.966+0000 7fc631400640 1 --2- 172.21.15.16:0/1884548702 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5639877d71c0 0x563987c6ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:58.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.967+0000 7fc631400640 1 --2- 172.21.15.16:0/1884548702 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5639877d71c0 0x563987c6ff90 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fa39cb87e7f7197 server_cookie=528c65edb4bf5a6c in_seq=0 out_seq=0 2024-08-31T06:16:58.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.967+0000 7fc613e00640 10 client.27785 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:16:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.988+0000 7fc613e00640 1 -- 172.21.15.16:0/1884548702 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563987b59960 con 0x5639877d71c0 2024-08-31T06:16:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.988+0000 7fc613e00640 10 client.27785 handle_command_reply: tid=0 2024-08-31T06:16:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.988+0000 7fc640087740 1 client.27785 shutdown 2024-08-31T06:16:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.989+0000 7fc640087740 1 -- 172.21.15.16:0/1884548702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56398781a2f0 msgr2=0x563987c04440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.989+0000 7fc640087740 1 --2- 172.21.15.16:0/1884548702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56398781a2f0 0x563987c04440 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc6240b0220 tx=0x7fc6240aac60 comp rx=0 tx=0).stop 2024-08-31T06:16:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.989+0000 7fc640087740 1 -- 172.21.15.16:0/1884548702 shutdown_connections 2024-08-31T06:16:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.989+0000 7fc640087740 1 --2- 172.21.15.16:0/1884548702 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5639877d71c0 0x563987c6ff90 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.989+0000 7fc640087740 1 --2- 172.21.15.16:0/1884548702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639878431b0 0x563987c6c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.989+0000 7fc640087740 1 --2- 172.21.15.16:0/1884548702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639878224d0 0x563987c04a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.989+0000 7fc613e00640 0 client.27785 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:16:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.989+0000 7fc640087740 1 --2- 172.21.15.16:0/1884548702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56398781a2f0 0x563987c04440 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.989+0000 7fc640087740 1 -- 172.21.15.16:0/1884548702 >> 172.21.15.16:0/1884548702 conn(0x563987837300 msgr2=0x563987bf30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.989+0000 7fc640087740 1 -- 172.21.15.16:0/1884548702 shutdown_connections 2024-08-31T06:16:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.989+0000 7fc640087740 1 -- 172.21.15.16:0/1884548702 wait complete. 2024-08-31T06:16:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.989+0000 7fc640087740 20 client.27785 trim_cache size 0 max 16384 2024-08-31T06:16:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:16:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:16:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:16:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:16:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.991+0000 7fc639000640 1 -- 172.21.15.16:0/1034939816 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc6180b4e70 msgr2=0x7fc6180b7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.991+0000 7fc639000640 1 --2- 172.21.15.16:0/1034939816 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc6180b4e70 0x7fc6180b7380 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc62c02ae60 tx=0x7fc62c005d80 comp rx=0 tx=0).stop 2024-08-31T06:16:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.991+0000 7fc639000640 1 -- 172.21.15.16:0/1034939816 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6341022b0 msgr2=0x7fc63410cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.991+0000 7fc639000640 1 --2- 172.21.15.16:0/1034939816 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6341022b0 0x7fc63410cd40 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc62400d950 tx=0x7fc62400de10 comp rx=0 tx=0).stop 2024-08-31T06:16:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.992+0000 7fc639000640 1 -- 172.21.15.16:0/1034939816 shutdown_connections 2024-08-31T06:16:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.992+0000 7fc639000640 1 --2- 172.21.15.16:0/1034939816 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc6180b4e70 0x7fc6180b7380 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.992+0000 7fc639000640 1 --2- 172.21.15.16:0/1034939816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc634102c00 0x7fc63410d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.992+0000 7fc639000640 1 --2- 172.21.15.16:0/1034939816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc63410fe60 0x7fc63410d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.992+0000 7fc639000640 1 --2- 172.21.15.16:0/1034939816 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6341022b0 0x7fc63410cd40 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.992+0000 7fc639000640 1 -- 172.21.15.16:0/1034939816 >> 172.21.15.16:0/1034939816 conn(0x7fc63407a4e0 msgr2=0x7fc634111100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:58.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.992+0000 7fc639000640 1 -- 172.21.15.16:0/1034939816 shutdown_connections 2024-08-31T06:16:58.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:16:58.992+0000 7fc639000640 1 -- 172.21.15.16:0/1034939816 wait complete. 2024-08-31T06:16:59.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:58 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:16:59.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:58 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:16:59.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:58 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:16:59.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:58 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:16:59.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.685+0000 7fb3a9800640 1 -- 172.21.15.16:0/2931036565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3a4104ee0 msgr2=0x7fb3a410d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:59.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.685+0000 7fb3a9800640 1 --2- 172.21.15.16:0/2931036565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3a4104ee0 0x7fb3a410d430 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb38c009900 tx=0x7fb38c02a420 comp rx=0 tx=0).stop 2024-08-31T06:16:59.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.686+0000 7fb3a9800640 1 -- 172.21.15.16:0/2931036565 shutdown_connections 2024-08-31T06:16:59.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.686+0000 7fb3a9800640 1 --2- 172.21.15.16:0/2931036565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3a4104ee0 0x7fb3a410d430 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:59.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.686+0000 7fb3a9800640 1 --2- 172.21.15.16:0/2931036565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3a410d9f0 0x7fb3a410fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:59.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.686+0000 7fb3a9800640 1 --2- 172.21.15.16:0/2931036565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3a4104590 0x7fb3a41049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:59.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.686+0000 7fb3a9800640 1 -- 172.21.15.16:0/2931036565 >> 172.21.15.16:0/2931036565 conn(0x7fb3a40fe0d0 msgr2=0x7fb3a4100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:59.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.686+0000 7fb3a9800640 1 -- 172.21.15.16:0/2931036565 shutdown_connections 2024-08-31T06:16:59.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.686+0000 7fb3a9800640 1 -- 172.21.15.16:0/2931036565 wait complete. 2024-08-31T06:16:59.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.688+0000 7fb3a9800640 1 Processor -- start 2024-08-31T06:16:59.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.688+0000 7fb3a9800640 1 -- start start 2024-08-31T06:16:59.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.689+0000 7fb3a9800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3a4104590 0x7fb3a419a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:59.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.689+0000 7fb3a9800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3a4104ee0 0x7fb3a419a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:59.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.689+0000 7fb3a9800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3a410d9f0 0x7fb3a419fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:59.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.689+0000 7fb3a9800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a4113700 con 0x7fb3a4104ee0 2024-08-31T06:16:59.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.689+0000 7fb3a9800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a4113580 con 0x7fb3a4104590 2024-08-31T06:16:59.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.689+0000 7fb3a9800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a4113880 con 0x7fb3a410d9f0 2024-08-31T06:16:59.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.689+0000 7fb3a3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3a4104590 0x7fb3a419a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:59.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.689+0000 7fb3a8e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3a410d9f0 0x7fb3a419fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:59.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.689+0000 7fb3a3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3a4104590 0x7fb3a419a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51988/0 (socket says 172.21.15.16:51988) 2024-08-31T06:16:59.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.689+0000 7fb3a3e00640 1 -- 172.21.15.16:0/1038896630 learned_addr learned my addr 172.21.15.16:0/1038896630 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:59.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.690+0000 7fb39b400640 1 --2- 172.21.15.16:0/1038896630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3a4104ee0 0x7fb3a419a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:59.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.690+0000 7fb3a3e00640 1 -- 172.21.15.16:0/1038896630 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3a410d9f0 msgr2=0x7fb3a419fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:59.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.690+0000 7fb3a3e00640 1 --2- 172.21.15.16:0/1038896630 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3a410d9f0 0x7fb3a419fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:59.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.690+0000 7fb3a3e00640 1 -- 172.21.15.16:0/1038896630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3a4104ee0 msgr2=0x7fb3a419a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:59.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.690+0000 7fb3a3e00640 1 --2- 172.21.15.16:0/1038896630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3a4104ee0 0x7fb3a419a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:59.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.690+0000 7fb3a3e00640 1 -- 172.21.15.16:0/1038896630 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3a41a0180 con 0x7fb3a4104590 2024-08-31T06:16:59.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.690+0000 7fb3a8e00640 1 --2- 172.21.15.16:0/1038896630 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3a410d9f0 0x7fb3a419fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:16:59.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.690+0000 7fb39b400640 1 --2- 172.21.15.16:0/1038896630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3a4104ee0 0x7fb3a419a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:16:59.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.692+0000 7fb3a3e00640 1 --2- 172.21.15.16:0/1038896630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3a4104590 0x7fb3a419a210 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb39400c890 tx=0x7fb39400cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:59.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.692+0000 7fb3a1600640 1 -- 172.21.15.16:0/1038896630 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb394004220 con 0x7fb3a4104590 2024-08-31T06:16:59.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.693+0000 7fb3a9800640 1 -- 172.21.15.16:0/1038896630 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3a41a0470 con 0x7fb3a4104590 2024-08-31T06:16:59.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.693+0000 7fb3a1600640 1 -- 172.21.15.16:0/1038896630 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3940043c0 con 0x7fb3a4104590 2024-08-31T06:16:59.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.693+0000 7fb3a1600640 1 -- 172.21.15.16:0/1038896630 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb394004a00 con 0x7fb3a4104590 2024-08-31T06:16:59.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.693+0000 7fb3a9800640 1 -- 172.21.15.16:0/1038896630 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3a41021b0 con 0x7fb3a4104590 2024-08-31T06:16:59.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.698+0000 7fb3b1a69740 1 Processor -- start 2024-08-31T06:16:59.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.698+0000 7fb3b1a69740 1 -- start start 2024-08-31T06:16:59.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.698+0000 7fb3a1600640 1 -- 172.21.15.16:0/1038896630 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fb3940079b0 con 0x7fb3a4104590 2024-08-31T06:16:59.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.698+0000 7fb3b1a69740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b6c3b840 0x55c7b6fe22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:59.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.698+0000 7fb3b1a69740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b6c12980 0x55c7b6fe2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:59.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.699+0000 7fb3a8e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b6c3b840 0x55c7b6fe22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:59.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.699+0000 7fb3b1a69740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c7b6c1ab60 0x55c7b6fe2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:59.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.699+0000 7fb3a8e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b6c3b840 0x55c7b6fe22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:52000/0 (socket says 172.21.15.16:52000) 2024-08-31T06:16:59.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.699+0000 7fb3b1a69740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c7b6f57530 con 0x55c7b6c12980 2024-08-31T06:16:59.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.699+0000 7fb3b1a69740 1 -- 172.21.15.16:0/2747087280 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c7b6f46ac0 con 0x55c7b6c3b840 2024-08-31T06:16:59.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.699+0000 7fb3a8e00640 1 -- 172.21.15.16:0/2747087280 learned_addr learned my addr 172.21.15.16:0/2747087280 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:59.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.699+0000 7fb3b1a69740 1 -- 172.21.15.16:0/2747087280 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c7b6f4aa70 con 0x55c7b6c1ab60 2024-08-31T06:16:59.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.699+0000 7fb3a3e00640 1 --2- 172.21.15.16:0/2747087280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b6c12980 0x55c7b6fe2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:59.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.700+0000 7fb3a8e00640 1 -- 172.21.15.16:0/2747087280 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c7b6c1ab60 msgr2=0x55c7b6fe2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:59.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.700+0000 7fb39b400640 1 --2- 172.21.15.16:0/2747087280 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c7b6c1ab60 0x55c7b6fe2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:59.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.701+0000 7fb3a1600640 1 --2- 172.21.15.16:0/1038896630 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb3840b4d60 0x7fb3840b7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:59.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.701+0000 7fb3a1600640 1 -- 172.21.15.16:0/1038896630 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8470+0+0 (secure 0 0 0) 0x7fb3940e5630 con 0x7fb3a4104590 2024-08-31T06:16:59.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.702+0000 7fb3a8e00640 1 --2- 172.21.15.16:0/2747087280 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c7b6c1ab60 0x55c7b6fe2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:59.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.702+0000 7fb3a8e00640 1 -- 172.21.15.16:0/2747087280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b6c12980 msgr2=0x55c7b6fe2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:59.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.702+0000 7fb3a8e00640 1 --2- 172.21.15.16:0/2747087280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b6c12980 0x55c7b6fe2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:59.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.702+0000 7fb3a8e00640 1 -- 172.21.15.16:0/2747087280 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7b6ef9880 con 0x55c7b6c3b840 2024-08-31T06:16:59.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.702+0000 7fb3a3e00640 1 --2- 172.21.15.16:0/2747087280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b6c12980 0x55c7b6fe2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:16:59.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.702+0000 7fb39b400640 1 --2- 172.21.15.16:0/2747087280 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c7b6c1ab60 0x55c7b6fe2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:16:59.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.703+0000 7fb3a8e00640 1 --2- 172.21.15.16:0/1038896630 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb3840b4d60 0x7fb3840b7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:59.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.703+0000 7fb3a8e00640 1 --2- 172.21.15.16:0/2747087280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b6c3b840 0x55c7b6fe22d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb39c010840 tx=0x7fb39c010d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d079e9e0a607c2a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:59.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.703+0000 7fb399600640 1 -- 172.21.15.16:0/2747087280 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39c020040 con 0x55c7b6c3b840 2024-08-31T06:16:59.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.704+0000 7fb399600640 1 -- 172.21.15.16:0/2747087280 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb39c021020 con 0x55c7b6c3b840 2024-08-31T06:16:59.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.704+0000 7fb399600640 1 -- 172.21.15.16:0/2747087280 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39c01b860 con 0x55c7b6c3b840 2024-08-31T06:16:59.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.705+0000 7fb3b1a69740 1 -- 172.21.15.16:0/2747087280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b6c3b840 msgr2=0x55c7b6fe22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:59.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.705+0000 7fb3a8e00640 1 --2- 172.21.15.16:0/1038896630 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb3840b4d60 0x7fb3840b7270 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb39c022000 tx=0x7fb39c01bf70 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:59.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.705+0000 7fb3b1a69740 1 --2- 172.21.15.16:0/2747087280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b6c3b840 0x55c7b6fe22d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb39c010840 tx=0x7fb39c010d00 comp rx=0 tx=0).stop 2024-08-31T06:16:59.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.705+0000 7fb3b1a69740 1 -- 172.21.15.16:0/2747087280 shutdown_connections 2024-08-31T06:16:59.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.705+0000 7fb3b1a69740 1 --2- 172.21.15.16:0/2747087280 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c7b6c1ab60 0x55c7b6fe2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:59.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.705+0000 7fb3b1a69740 1 --2- 172.21.15.16:0/2747087280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b6c12980 0x55c7b6fe2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:59.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.705+0000 7fb3b1a69740 1 --2- 172.21.15.16:0/2747087280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b6c3b840 0x55c7b6fe22d0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:59.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.705+0000 7fb3b1a69740 1 -- 172.21.15.16:0/2747087280 >> 172.21.15.16:0/2747087280 conn(0x55c7b6c2f990 msgr2=0x55c7b6fe01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:59.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.705+0000 7fb3b1a69740 1 -- 172.21.15.16:0/2747087280 shutdown_connections 2024-08-31T06:16:59.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.706+0000 7fb3b1a69740 1 -- 172.21.15.16:0/2747087280 wait complete. 2024-08-31T06:16:59.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.707+0000 7fb3b1a69740 1 -- start start 2024-08-31T06:16:59.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.708+0000 7fb3b1a69740 1 Processor -- start 2024-08-31T06:16:59.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.708+0000 7fb3b1a69740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c7b6c12980 0x55c7b6fedec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:59.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.708+0000 7fb3b1a69740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b6c1ab60 0x55c7b6fee400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:59.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.708+0000 7fb3b1a69740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b6c3b840 0x55c7b6ff9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:59.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.708+0000 7fb3b1a69740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c7b6da97a0 con 0x55c7b6c3b840 2024-08-31T06:16:59.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.708+0000 7fb3b1a69740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c7b6ecd7f0 con 0x55c7b6c1ab60 2024-08-31T06:16:59.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.708+0000 7fb3b1a69740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c7b6ff0ab0 con 0x55c7b6c12980 2024-08-31T06:16:59.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.708+0000 7fb3a3e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c7b6c12980 0x55c7b6fedec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:59.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.708+0000 7fb3b1a69740 10 client.0 _ll_register_callbacks cb 0x55c7b6ef2e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:16:59.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.709+0000 7fb3a3e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c7b6c12980 0x55c7b6fedec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:51478/0 (socket says 172.21.15.16:51478) 2024-08-31T06:16:59.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.709+0000 7fb3a3e00640 1 -- 172.21.15.16:0/4274821572 learned_addr learned my addr 172.21.15.16:0/4274821572 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:59.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.709+0000 7fb39b400640 1 --2- 172.21.15.16:0/4274821572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b6c1ab60 0x55c7b6fee400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:59.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.709+0000 7fb3a8e00640 1 --2- 172.21.15.16:0/4274821572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b6c3b840 0x55c7b6ff9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:59.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.709+0000 7fb3a3e00640 1 -- 172.21.15.16:0/4274821572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b6c1ab60 msgr2=0x55c7b6fee400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:59.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.709+0000 7fb3a3e00640 1 --2- 172.21.15.16:0/4274821572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b6c1ab60 0x55c7b6fee400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:59.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.709+0000 7fb3a3e00640 1 -- 172.21.15.16:0/4274821572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b6c3b840 msgr2=0x55c7b6ff9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:59.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.709+0000 7fb3a3e00640 1 --2- 172.21.15.16:0/4274821572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b6c3b840 0x55c7b6ff9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:59.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.709+0000 7fb3a3e00640 1 -- 172.21.15.16:0/4274821572 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7b6cb5b90 con 0x55c7b6c12980 2024-08-31T06:16:59.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.709+0000 7fb3a8e00640 1 --2- 172.21.15.16:0/4274821572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b6c3b840 0x55c7b6ff9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:16:59.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.709+0000 7fb3a3e00640 1 --2- 172.21.15.16:0/4274821572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c7b6c12980 0x55c7b6fedec0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb3940a8f60 tx=0x7fb394005290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd8baaa642b65dd7 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:59.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.709+0000 7fb392000640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:16:59.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.709+0000 7fb392000640 1 -- 172.21.15.16:0/4274821572 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3940abcf0 con 0x55c7b6c12980 2024-08-31T06:16:59.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.710+0000 7fb3b1a69740 1 -- 172.21.15.16:0/4274821572 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb39c021020 con 0x55c7b6c12980 2024-08-31T06:16:59.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.710+0000 7fb392000640 1 -- 172.21.15.16:0/4274821572 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8470+0+0 (secure 0 0 0) 0x7fb3940a7f80 con 0x55c7b6c12980 2024-08-31T06:16:59.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.710+0000 7fb392000640 1 -- 172.21.15.16:0/4274821572 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fb3940b0560 con 0x55c7b6c12980 2024-08-31T06:16:59.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.710+0000 7fb3b1a69740 10 client.27796 fetch_fsmap learned FSMap version 43 2024-08-31T06:16:59.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.710+0000 7fb3b1a69740 1 -- 172.21.15.16:0/4274821572 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c7b6ffca00 con 0x55c7b6c12980 2024-08-31T06:16:59.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.710+0000 7fb392000640 1 -- 172.21.15.16:0/4274821572 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb39c021020 con 0x55c7b6c12980 2024-08-31T06:16:59.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.710+0000 7fb392000640 1 -- 172.21.15.16:0/4274821572 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fb3940aaf50 con 0x55c7b6c12980 2024-08-31T06:16:59.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.710+0000 7fb3b1a69740 10 client.27796 fetch_fsmap finished waiting for FSMap version 43 2024-08-31T06:16:59.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.710+0000 7fb3b1a69740 10 client.27796 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:16:59.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.710+0000 7fb3b1a69740 20 client.27796 populate_metadata read hostname 'smithi016' 2024-08-31T06:16:59.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.710+0000 7fb3b1a69740 1 --2- 172.21.15.16:0/4274821572 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55c7b6bfa1c0 0x55c7b6fffe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:59.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.711+0000 7fb3b1a69740 4 client.27796 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:16:59.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.711+0000 7fb3b1a69740 1 -- 172.21.15.16:0/4274821572 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7b6f59480 con 0x55c7b6bfa1c0 2024-08-31T06:16:59.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.711+0000 7fb39b400640 1 --2- 172.21.15.16:0/4274821572 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55c7b6bfa1c0 0x55c7b6fffe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:59.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.711+0000 7fb39b400640 1 --2- 172.21.15.16:0/4274821572 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55c7b6bfa1c0 0x55c7b6fffe40 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a7ad33b8198de83 server_cookie=94fc1e59946af6bc in_seq=0 out_seq=0 2024-08-31T06:16:59.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.711+0000 7fb392000640 10 client.27796 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:16:59.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.712+0000 7fb392000640 1 -- 172.21.15.16:0/4274821572 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c7b6f59480 con 0x55c7b6bfa1c0 2024-08-31T06:16:59.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.712+0000 7fb392000640 10 client.27796 handle_command_reply: tid=0 2024-08-31T06:16:59.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.713+0000 7fb3b1a69740 1 client.27796 shutdown 2024-08-31T06:16:59.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.713+0000 7fb3b1a69740 1 -- 172.21.15.16:0/4274821572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c7b6c12980 msgr2=0x55c7b6fedec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:59.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.713+0000 7fb3b1a69740 1 --2- 172.21.15.16:0/4274821572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c7b6c12980 0x55c7b6fedec0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb3940a8f60 tx=0x7fb394005290 comp rx=0 tx=0).stop 2024-08-31T06:16:59.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.713+0000 7fb3b1a69740 1 -- 172.21.15.16:0/4274821572 shutdown_connections 2024-08-31T06:16:59.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.713+0000 7fb3b1a69740 1 --2- 172.21.15.16:0/4274821572 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55c7b6bfa1c0 0x55c7b6fffe40 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:59.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.713+0000 7fb392000640 0 client.27796 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:16:59.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.713+0000 7fb3b1a69740 1 --2- 172.21.15.16:0/4274821572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b6c3b840 0x55c7b6ff9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:59.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.714+0000 7fb3b1a69740 1 --2- 172.21.15.16:0/4274821572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b6c1ab60 0x55c7b6fee400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:59.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.714+0000 7fb3b1a69740 1 --2- 172.21.15.16:0/4274821572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c7b6c12980 0x55c7b6fedec0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:59.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.714+0000 7fb3b1a69740 1 -- 172.21.15.16:0/4274821572 >> 172.21.15.16:0/4274821572 conn(0x55c7b6c2f990 msgr2=0x55c7b6fe54b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:59.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.714+0000 7fb3b1a69740 1 -- 172.21.15.16:0/4274821572 shutdown_connections 2024-08-31T06:16:59.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.714+0000 7fb3b1a69740 1 -- 172.21.15.16:0/4274821572 wait complete. 2024-08-31T06:16:59.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.714+0000 7fb3b1a69740 20 client.27796 trim_cache size 0 max 16384 2024-08-31T06:16:59.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.740+0000 7fb3b1a69740 1 Processor -- start 2024-08-31T06:16:59.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.740+0000 7fb3b1a69740 1 -- start start 2024-08-31T06:16:59.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.741+0000 7fb3b1a69740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b6bfa1c0 0x55c7b6ff9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:59.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.741+0000 7fb3b1a69740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b6c12980 0x55c7b6fe73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:59.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.741+0000 7fb3b1a69740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c7b6c1ab60 0x55c7b6fe7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:59.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.741+0000 7fb3b1a69740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a41afb90 con 0x55c7b6bfa1c0 2024-08-31T06:16:59.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.741+0000 7fb3b1a69740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a41ad400 con 0x55c7b6c12980 2024-08-31T06:16:59.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.741+0000 7fb3b1a69740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a41ab270 con 0x55c7b6c1ab60 2024-08-31T06:16:59.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.742+0000 7fb3a8e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c7b6c1ab60 0x55c7b6fe7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:59.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.742+0000 7fb3a8e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c7b6c1ab60 0x55c7b6fe7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:51492/0 (socket says 172.21.15.16:51492) 2024-08-31T06:16:59.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.742+0000 7fb3a8e00640 1 -- 172.21.15.16:0/2560865351 learned_addr learned my addr 172.21.15.16:0/2560865351 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:59.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.742+0000 7fb39b400640 1 --2- 172.21.15.16:0/2560865351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b6c12980 0x55c7b6fe73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:59.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.742+0000 7fb3a3e00640 1 --2- 172.21.15.16:0/2560865351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b6bfa1c0 0x55c7b6ff9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:59.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.743+0000 7fb39b400640 1 -- 172.21.15.16:0/2560865351 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c7b6c1ab60 msgr2=0x55c7b6fe7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:59.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.743+0000 7fb39b400640 1 --2- 172.21.15.16:0/2560865351 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c7b6c1ab60 0x55c7b6fe7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:59.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.743+0000 7fb39b400640 1 -- 172.21.15.16:0/2560865351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b6bfa1c0 msgr2=0x55c7b6ff9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:59.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.743+0000 7fb39b400640 1 --2- 172.21.15.16:0/2560865351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b6bfa1c0 0x55c7b6ff9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:59.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.743+0000 7fb39b400640 1 -- 172.21.15.16:0/2560865351 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7b6fee580 con 0x55c7b6c12980 2024-08-31T06:16:59.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.744+0000 7fb39b400640 1 --2- 172.21.15.16:0/2560865351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b6c12980 0x55c7b6fe73e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fb3a410fee0 tx=0x7fb38c031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d10a6812c0473568 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:59.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.744+0000 7fb398c00640 1 -- 172.21.15.16:0/2560865351 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38c047020 con 0x55c7b6c12980 2024-08-31T06:16:59.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.745+0000 7fb398c00640 1 -- 172.21.15.16:0/2560865351 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb38c005630 con 0x55c7b6c12980 2024-08-31T06:16:59.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.745+0000 7fb398c00640 1 -- 172.21.15.16:0/2560865351 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38c035040 con 0x55c7b6c12980 2024-08-31T06:16:59.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.746+0000 7fb3b1a69740 1 -- 172.21.15.16:0/2560865351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b6c12980 msgr2=0x55c7b6fe73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:16:59.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.746+0000 7fb3b1a69740 1 --2- 172.21.15.16:0/2560865351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b6c12980 0x55c7b6fe73e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fb3a410fee0 tx=0x7fb38c031400 comp rx=0 tx=0).stop 2024-08-31T06:16:59.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.746+0000 7fb3b1a69740 1 -- 172.21.15.16:0/2560865351 shutdown_connections 2024-08-31T06:16:59.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.746+0000 7fb3b1a69740 1 --2- 172.21.15.16:0/2560865351 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c7b6c1ab60 0x55c7b6fe7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:59.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.746+0000 7fb3b1a69740 1 --2- 172.21.15.16:0/2560865351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b6c12980 0x55c7b6fe73e0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:59.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.746+0000 7fb3b1a69740 1 --2- 172.21.15.16:0/2560865351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b6bfa1c0 0x55c7b6ff9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:59.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.746+0000 7fb3b1a69740 1 -- 172.21.15.16:0/2560865351 >> 172.21.15.16:0/2560865351 conn(0x55c7b6c2f990 msgr2=0x55c7b6fe8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:16:59.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.747+0000 7fb3b1a69740 1 -- 172.21.15.16:0/2560865351 shutdown_connections 2024-08-31T06:16:59.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.747+0000 7fb3b1a69740 1 -- 172.21.15.16:0/2560865351 wait complete. 2024-08-31T06:16:59.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.747+0000 7fb3b1a69740 1 -- start start 2024-08-31T06:16:59.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.748+0000 7fb3b1a69740 1 Processor -- start 2024-08-31T06:16:59.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.749+0000 7fb3b1a69740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b6bfa1c0 0x55c7b6ff6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:59.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.749+0000 7fb3b1a69740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b6c12980 0x55c7b6ff66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:59.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.749+0000 7fb3b1a69740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c7b6c1ab60 0x55c7b6fe1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:59.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.749+0000 7fb3b1a69740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a41ab8c0 con 0x55c7b6c12980 2024-08-31T06:16:59.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.749+0000 7fb3b1a69740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a41acd30 con 0x55c7b6bfa1c0 2024-08-31T06:16:59.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.749+0000 7fb3b1a69740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a41abf90 con 0x55c7b6c1ab60 2024-08-31T06:16:59.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.749+0000 7fb3b1a69740 10 client.0 _ll_register_callbacks cb 0x7fb3940ab640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:16:59.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.749+0000 7fb3a8e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c7b6c1ab60 0x55c7b6fe1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:59.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.749+0000 7fb3a3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b6bfa1c0 0x55c7b6ff6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:59.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.749+0000 7fb39b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b6c12980 0x55c7b6ff66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:59.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.749+0000 7fb3a3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b6bfa1c0 0x55c7b6ff6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:52032/0 (socket says 172.21.15.16:52032) 2024-08-31T06:16:59.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.749+0000 7fb3a8e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c7b6c1ab60 0x55c7b6fe1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:51508/0 (socket says 172.21.15.16:51508) 2024-08-31T06:16:59.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.749+0000 7fb3a3e00640 1 -- 172.21.15.16:0/754227625 learned_addr learned my addr 172.21.15.16:0/754227625 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:16:59.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.750+0000 7fb3a8e00640 1 -- 172.21.15.16:0/754227625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b6bfa1c0 msgr2=0x55c7b6ff6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:59.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.750+0000 7fb3a8e00640 1 --2- 172.21.15.16:0/754227625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b6bfa1c0 0x55c7b6ff6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:59.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.750+0000 7fb3a8e00640 1 -- 172.21.15.16:0/754227625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b6c12980 msgr2=0x55c7b6ff66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:16:59.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.750+0000 7fb3a8e00640 1 --2- 172.21.15.16:0/754227625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b6c12980 0x55c7b6ff66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:16:59.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.750+0000 7fb3a8e00640 1 -- 172.21.15.16:0/754227625 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7b6fe24a0 con 0x55c7b6c1ab60 2024-08-31T06:16:59.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.750+0000 7fb3a3e00640 1 --2- 172.21.15.16:0/754227625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b6bfa1c0 0x55c7b6ff6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:16:59.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.750+0000 7fb3a8e00640 1 --2- 172.21.15.16:0/754227625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c7b6c1ab60 0x55c7b6fe1d30 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb39c010d40 tx=0x7fb39c015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccf3785bd4fb4150 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:16:59.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.750+0000 7fb392000640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:16:59.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.751+0000 7fb392000640 1 -- 172.21.15.16:0/754227625 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39c037060 con 0x55c7b6c1ab60 2024-08-31T06:16:59.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.751+0000 7fb392000640 1 -- 172.21.15.16:0/754227625 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8470+0+0 (secure 0 0 0) 0x7fb39c016070 con 0x55c7b6c1ab60 2024-08-31T06:16:59.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.751+0000 7fb3b1a69740 1 -- 172.21.15.16:0/754227625 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c7b6f54f20 con 0x55c7b6c1ab60 2024-08-31T06:16:59.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.751+0000 7fb392000640 1 -- 172.21.15.16:0/754227625 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fb39c020030 con 0x55c7b6c1ab60 2024-08-31T06:16:59.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.752+0000 7fb3b1a69740 10 client.27802 fetch_fsmap learned FSMap version 43 2024-08-31T06:16:59.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.752+0000 7fb3b1a69740 1 -- 172.21.15.16:0/754227625 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c7b7078340 con 0x55c7b6c1ab60 2024-08-31T06:16:59.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.752+0000 7fb392000640 1 -- 172.21.15.16:0/754227625 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c7b6f54f20 con 0x55c7b6c1ab60 2024-08-31T06:16:59.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.752+0000 7fb392000640 1 -- 172.21.15.16:0/754227625 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fb39c004040 con 0x55c7b6c1ab60 2024-08-31T06:16:59.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.753+0000 7fb3b1a69740 10 client.27802 fetch_fsmap finished waiting for FSMap version 43 2024-08-31T06:16:59.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.753+0000 7fb3b1a69740 10 client.27802 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:16:59.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.753+0000 7fb3b1a69740 20 client.27802 populate_metadata read hostname 'smithi016' 2024-08-31T06:16:59.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.753+0000 7fb3b1a69740 1 --2- 172.21.15.16:0/754227625 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55c7b6c3b840 0x55c7b707a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:16:59.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.753+0000 7fb3b1a69740 4 client.27802 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 135.5, "await_for": 45.0}] 2024-08-31T06:16:59.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.753+0000 7fb3b1a69740 1 -- 172.21.15.16:0/754227625 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 135.5, "await_for": 45.0}) -- 0x55c7b6f52660 con 0x55c7b6c3b840 2024-08-31T06:16:59.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.754+0000 7fb3a3e00640 1 --2- 172.21.15.16:0/754227625 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55c7b6c3b840 0x55c7b707a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:16:59.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.754+0000 7fb392000640 10 client.27802 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:16:59.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.756+0000 7fb3a3e00640 1 --2- 172.21.15.16:0/754227625 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55c7b6c3b840 0x55c7b707a910 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=598ee8fc8489f0f3 server_cookie=b32a1938d5163a72 in_seq=0 out_seq=0 2024-08-31T06:16:59.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.996+0000 7fb392000640 1 -- 172.21.15.16:0/754227625 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c7b6f52660 con 0x55c7b6c3b840 2024-08-31T06:16:59.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.996+0000 7fb392000640 10 client.27802 handle_command_reply: tid=0 2024-08-31T06:16:59.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.996+0000 7fb3b1a69740 1 client.27802 shutdown 2024-08-31T06:16:59.999 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:59 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:16:59.999 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:16:59 smithi016 ceph-mon[17194]: pgmap v665: 97 pgs: 97 active+clean; 3.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 7.2 MiB/s wr, 765 op/s 2024-08-31T06:17:00.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.996+0000 7fb3b1a69740 1 -- 172.21.15.16:0/754227625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c7b6c1ab60 msgr2=0x55c7b6fe1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:00.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.996+0000 7fb3b1a69740 1 --2- 172.21.15.16:0/754227625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c7b6c1ab60 0x55c7b6fe1d30 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb39c010d40 tx=0x7fb39c015000 comp rx=0 tx=0).stop 2024-08-31T06:17:00.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.996+0000 7fb3b1a69740 1 -- 172.21.15.16:0/754227625 shutdown_connections 2024-08-31T06:17:00.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.996+0000 7fb3b1a69740 1 --2- 172.21.15.16:0/754227625 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55c7b6c3b840 0x55c7b707a910 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:00.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.996+0000 7fb3b1a69740 1 --2- 172.21.15.16:0/754227625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c7b6c1ab60 0x55c7b6fe1d30 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:00.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.996+0000 7fb3b1a69740 1 --2- 172.21.15.16:0/754227625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b6c12980 0x55c7b6ff66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:00.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.996+0000 7fb3b1a69740 1 --2- 172.21.15.16:0/754227625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b6bfa1c0 0x55c7b6ff6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:00.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.996+0000 7fb392000640 0 client.27802 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:17:00.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.996+0000 7fb3b1a69740 1 -- 172.21.15.16:0/754227625 >> 172.21.15.16:0/754227625 conn(0x55c7b6c2f990 msgr2=0x55c7b6ff9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:00.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.996+0000 7fb3b1a69740 1 -- 172.21.15.16:0/754227625 shutdown_connections 2024-08-31T06:17:00.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.996+0000 7fb3b1a69740 1 -- 172.21.15.16:0/754227625 wait complete. 2024-08-31T06:17:00.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.997+0000 7fb3b1a69740 20 client.27802 trim_cache size 0 max 16384 2024-08-31T06:17:00.001 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:17:00.001 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 43, 2024-08-31T06:17:00.001 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24737, 2024-08-31T06:17:00.001 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 30, 2024-08-31T06:17:00.001 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-08-31T06:17:00.001 INFO:teuthology.orchestra.run.smithi016.stdout: "a57ea9ba": { 2024-08-31T06:17:00.001 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 30, 2024-08-31T06:17:00.001 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-08-31T06:17:00.001 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:17:00.002 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-08-31T06:17:00.002 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:17:00.002 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:17:00.002 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-08-31T06:17:00.002 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 135.5, 2024-08-31T06:17:00.002 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-08-31T06:17:00.002 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-08-31T06:17:00.002 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-08-31T06:17:00.002 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:17:00.002 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-08-31T06:17:00.002 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:17:00.002 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:17:00.002 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:17:00.004 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:17:00.005 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:17:00.005 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:17:00.005 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:17:00.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.997+0000 7fb3a9800640 1 -- 172.21.15.16:0/1038896630 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb3840b4d60 msgr2=0x7fb3840b7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:00.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.997+0000 7fb3a9800640 1 --2- 172.21.15.16:0/1038896630 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb3840b4d60 0x7fb3840b7270 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb39c022000 tx=0x7fb39c01bf70 comp rx=0 tx=0).stop 2024-08-31T06:17:00.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.997+0000 7fb3a9800640 1 -- 172.21.15.16:0/1038896630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3a4104590 msgr2=0x7fb3a419a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:00.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.997+0000 7fb3a9800640 1 --2- 172.21.15.16:0/1038896630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3a4104590 0x7fb3a419a210 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb39400c890 tx=0x7fb39400cd50 comp rx=0 tx=0).stop 2024-08-31T06:17:00.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.998+0000 7fb3a9800640 1 -- 172.21.15.16:0/1038896630 shutdown_connections 2024-08-31T06:17:00.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.998+0000 7fb3a9800640 1 --2- 172.21.15.16:0/1038896630 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb3840b4d60 0x7fb3840b7270 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:00.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.998+0000 7fb3a9800640 1 --2- 172.21.15.16:0/1038896630 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3a410d9f0 0x7fb3a419fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:00.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.998+0000 7fb3a9800640 1 --2- 172.21.15.16:0/1038896630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3a4104ee0 0x7fb3a419a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:00.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.998+0000 7fb3a9800640 1 --2- 172.21.15.16:0/1038896630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3a4104590 0x7fb3a419a210 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:00.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.998+0000 7fb3a9800640 1 -- 172.21.15.16:0/1038896630 >> 172.21.15.16:0/1038896630 conn(0x7fb3a40fe0d0 msgr2=0x7fb3a4107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:00.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.998+0000 7fb3a9800640 1 -- 172.21.15.16:0/1038896630 shutdown_connections 2024-08-31T06:17:00.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:16:59.998+0000 7fb3a9800640 1 -- 172.21.15.16:0/1038896630 wait complete. 2024-08-31T06:17:00.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:59 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:17:00.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:16:59 smithi155 ceph-mon[21742]: pgmap v665: 97 pgs: 97 active+clean; 3.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 7.2 MiB/s wr, 765 op/s 2024-08-31T06:17:00.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:59 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:17:00.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:16:59 smithi184 ceph-mon[22331]: pgmap v665: 97 pgs: 97 active+clean; 3.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 7.2 MiB/s wr, 765 op/s 2024-08-31T06:17:01.028 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a57ea9ba', quiesce took 8.1 seconds. Will release after: 7.4 2024-08-31T06:17:01.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:00 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:17:01.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:00 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:17:01.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:00 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:17:01.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:00 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:17:01.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:00 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:17:01.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:00 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:17:01.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:00 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:17:01.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:00 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:17:01.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:00 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:17:01.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:00 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:17:01.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:00 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:17:01.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:00 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:17:01.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:00 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:17:01.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:00 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:17:01.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:00 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:17:01.626 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 575.0 s, 1200.6 tps, lat 54.517 ms stddev 58.489 2024-08-31T06:17:02.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:01 smithi184 ceph-mon[22331]: pgmap v666: 97 pgs: 97 active+clean; 3.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 6.2 MiB/s wr, 661 op/s 2024-08-31T06:17:02.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:01 smithi155 ceph-mon[21742]: pgmap v666: 97 pgs: 97 active+clean; 3.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 6.2 MiB/s wr, 661 op/s 2024-08-31T06:17:02.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:01 smithi016 ceph-mon[17194]: pgmap v666: 97 pgs: 97 active+clean; 3.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 6.2 MiB/s wr, 661 op/s 2024-08-31T06:17:04.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:04 smithi155 ceph-mon[21742]: pgmap v667: 97 pgs: 97 active+clean; 3.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 5.9 MiB/s wr, 601 op/s 2024-08-31T06:17:04.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:04 smithi016 ceph-mon[17194]: pgmap v667: 97 pgs: 97 active+clean; 3.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 5.9 MiB/s wr, 601 op/s 2024-08-31T06:17:04.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:04 smithi184 ceph-mon[22331]: pgmap v667: 97 pgs: 97 active+clean; 3.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 5.9 MiB/s wr, 601 op/s 2024-08-31T06:17:06.629 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 580.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-31T06:17:06.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:06 smithi155 ceph-mon[21742]: pgmap v668: 97 pgs: 97 active+clean; 3.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 5.5 MiB/s wr, 550 op/s 2024-08-31T06:17:06.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:06 smithi016 ceph-mon[17194]: pgmap v668: 97 pgs: 97 active+clean; 3.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 5.5 MiB/s wr, 550 op/s 2024-08-31T06:17:06.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:06 smithi184 ceph-mon[22331]: pgmap v668: 97 pgs: 97 active+clean; 3.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 5.5 MiB/s wr, 550 op/s 2024-08-31T06:17:08.429 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:17:08.786 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:08 smithi016 ceph-mon[17194]: pgmap v669: 97 pgs: 97 active+clean; 3.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 2.8 MiB/s wr, 322 op/s 2024-08-31T06:17:08.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:08 smithi155 ceph-mon[21742]: pgmap v669: 97 pgs: 97 active+clean; 3.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 2.8 MiB/s wr, 322 op/s 2024-08-31T06:17:08.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:08 smithi184 ceph-mon[22331]: pgmap v669: 97 pgs: 97 active+clean; 3.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 2.8 MiB/s wr, 322 op/s 2024-08-31T06:17:08.980 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:17:09.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.691+0000 7f6ec0c00640 1 -- 172.21.15.16:0/4015321525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ebc1141f0 msgr2=0x7f6ebc1166c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:09.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.691+0000 7f6ec0c00640 1 --2- 172.21.15.16:0/4015321525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ebc1141f0 0x7f6ebc1166c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4009ca0 tx=0x7f6eb402a8b0 comp rx=0 tx=0).stop 2024-08-31T06:17:09.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.691+0000 7f6ec0c00640 1 -- 172.21.15.16:0/4015321525 shutdown_connections 2024-08-31T06:17:09.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.691+0000 7f6ec0c00640 1 --2- 172.21.15.16:0/4015321525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ebc1141f0 0x7f6ebc1166c0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:09.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.691+0000 7f6ec0c00640 1 --2- 172.21.15.16:0/4015321525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ebc075b70 0x7f6ebc075f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:09.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.691+0000 7f6ec0c00640 1 --2- 172.21.15.16:0/4015321525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ebc077120 0x7f6ebc075550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:09.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.691+0000 7f6ec0c00640 1 -- 172.21.15.16:0/4015321525 >> 172.21.15.16:0/4015321525 conn(0x7f6ebc0ff370 msgr2=0x7f6ebc1017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:09.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.692+0000 7f6ec0c00640 1 -- 172.21.15.16:0/4015321525 shutdown_connections 2024-08-31T06:17:09.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.692+0000 7f6ec0c00640 1 -- 172.21.15.16:0/4015321525 wait complete. 2024-08-31T06:17:09.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.693+0000 7f6ec0c00640 1 Processor -- start 2024-08-31T06:17:09.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.693+0000 7f6ec0c00640 1 -- start start 2024-08-31T06:17:09.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.694+0000 7f6ec0c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ebc075b70 0x7f6ebc1a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:09.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.694+0000 7f6ec0c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ebc077120 0x7f6ebc1a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:09.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.694+0000 7f6ec0c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ebc1141f0 0x7f6ebc1a5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:09.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.694+0000 7f6ec0c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6ebc119e30 con 0x7f6ebc075b70 2024-08-31T06:17:09.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.694+0000 7f6ec0c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6ebc119cb0 con 0x7f6ebc1141f0 2024-08-31T06:17:09.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.694+0000 7f6ec0c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6ebc119fb0 con 0x7f6ebc077120 2024-08-31T06:17:09.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.694+0000 7f6eb9e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ebc075b70 0x7f6ebc1a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:09.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.694+0000 7f6eb9e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ebc075b70 0x7f6ebc1a07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47676/0 (socket says 172.21.15.16:47676) 2024-08-31T06:17:09.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.694+0000 7f6eb9e00640 1 -- 172.21.15.16:0/2845501541 learned_addr learned my addr 172.21.15.16:0/2845501541 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:09.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.694+0000 7f6eb9400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ebc077120 0x7f6ebc1a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:09.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.694+0000 7f6eba800640 1 --2- 172.21.15.16:0/2845501541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ebc1141f0 0x7f6ebc1a5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:09.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.695+0000 7f6eb9e00640 1 -- 172.21.15.16:0/2845501541 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ebc077120 msgr2=0x7f6ebc1a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:09.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.695+0000 7f6eb9e00640 1 --2- 172.21.15.16:0/2845501541 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ebc077120 0x7f6ebc1a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:09.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.695+0000 7f6eb9e00640 1 -- 172.21.15.16:0/2845501541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ebc1141f0 msgr2=0x7f6ebc1a5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:09.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.695+0000 7f6eb9e00640 1 --2- 172.21.15.16:0/2845501541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ebc1141f0 0x7f6ebc1a5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:09.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.695+0000 7f6eb9e00640 1 -- 172.21.15.16:0/2845501541 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ebc1a6710 con 0x7f6ebc075b70 2024-08-31T06:17:09.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.695+0000 7f6eb9400640 1 --2- 172.21.15.16:0/2845501541 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ebc077120 0x7f6ebc1a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:17:09.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.695+0000 7f6eb9e00640 1 --2- 172.21.15.16:0/2845501541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ebc075b70 0x7f6ebc1a07a0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6eac00e9f0 tx=0x7f6eac00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:09.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.695+0000 7f6eb2a00640 1 -- 172.21.15.16:0/2845501541 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eac00cd80 con 0x7f6ebc075b70 2024-08-31T06:17:09.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.695+0000 7f6eb2a00640 1 -- 172.21.15.16:0/2845501541 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6eac0054d0 con 0x7f6ebc075b70 2024-08-31T06:17:09.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.695+0000 7f6eb2a00640 1 -- 172.21.15.16:0/2845501541 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eac010690 con 0x7f6ebc075b70 2024-08-31T06:17:09.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.695+0000 7f6ec0c00640 1 -- 172.21.15.16:0/2845501541 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ebc1a6a00 con 0x7f6ebc075b70 2024-08-31T06:17:09.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.696+0000 7f6ec0c00640 1 -- 172.21.15.16:0/2845501541 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ebc1ad160 con 0x7f6ebc075b70 2024-08-31T06:17:09.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.697+0000 7f6ec0c00640 1 -- 172.21.15.16:0/2845501541 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ebc075f80 con 0x7f6ebc075b70 2024-08-31T06:17:09.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.699+0000 7f6eb2a00640 1 -- 172.21.15.16:0/2845501541 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f6eac004e30 con 0x7f6ebc075b70 2024-08-31T06:17:09.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.701+0000 7f6eb2a00640 1 --2- 172.21.15.16:0/2845501541 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6ea00b4d20 0x7f6ea00b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:09.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.701+0000 7f6eb2a00640 1 -- 172.21.15.16:0/2845501541 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8514+0+0 (secure 0 0 0) 0x7f6eac0e5b40 con 0x7f6ebc075b70 2024-08-31T06:17:09.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.701+0000 7f6eb9400640 1 --2- 172.21.15.16:0/2845501541 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6ea00b4d20 0x7f6ea00b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:09.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.702+0000 7f6eb9400640 1 --2- 172.21.15.16:0/2845501541 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6ea00b4d20 0x7f6ea00b7230 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc116160 tx=0x7f6ea8005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:09.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.703+0000 7f6eb2a00640 1 -- 172.21.15.16:0/2845501541 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f6eac056130 con 0x7f6ebc075b70 2024-08-31T06:17:09.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:09 smithi155 ceph-mon[21742]: osdmap e90: 12 total, 12 up, 12 in 2024-08-31T06:17:09.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:09 smithi016 ceph-mon[17194]: osdmap e90: 12 total, 12 up, 12 in 2024-08-31T06:17:09.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.935+0000 7f6ec0c00640 1 -- 172.21.15.16:0/2845501541 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ebc1a18d0 con 0x7f6ebc075b70 2024-08-31T06:17:09.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.935+0000 7f6eb2a00640 1 -- 172.21.15.16:0/2845501541 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11920 (secure 0 0 0) 0x7f6eac0ac8d0 con 0x7f6ebc075b70 2024-08-31T06:17:09.938 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:17:09.938 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":43,"btime":"2024-08-31T06:16:03:711285+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:16:03.340730+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24737,24724,24712]},"id":1}]} 2024-08-31T06:17:09.939 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 43 2024-08-31T06:17:09.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.938+0000 7f6ec0c00640 1 -- 172.21.15.16:0/2845501541 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6ea00b4d20 msgr2=0x7f6ea00b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:09.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.938+0000 7f6ec0c00640 1 --2- 172.21.15.16:0/2845501541 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6ea00b4d20 0x7f6ea00b7230 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc116160 tx=0x7f6ea8005f90 comp rx=0 tx=0).stop 2024-08-31T06:17:09.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.938+0000 7f6ec0c00640 1 -- 172.21.15.16:0/2845501541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ebc075b70 msgr2=0x7f6ebc1a07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:09.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.938+0000 7f6ec0c00640 1 --2- 172.21.15.16:0/2845501541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ebc075b70 0x7f6ebc1a07a0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6eac00e9f0 tx=0x7f6eac00edf0 comp rx=0 tx=0).stop 2024-08-31T06:17:09.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.938+0000 7f6ec0c00640 1 -- 172.21.15.16:0/2845501541 shutdown_connections 2024-08-31T06:17:09.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.938+0000 7f6ec0c00640 1 --2- 172.21.15.16:0/2845501541 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6ea00b4d20 0x7f6ea00b7230 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:09.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.938+0000 7f6ec0c00640 1 --2- 172.21.15.16:0/2845501541 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ebc077120 0x7f6ebc1a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:09.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.938+0000 7f6ec0c00640 1 --2- 172.21.15.16:0/2845501541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ebc1141f0 0x7f6ebc1a5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:09.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.938+0000 7f6ec0c00640 1 --2- 172.21.15.16:0/2845501541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ebc075b70 0x7f6ebc1a07a0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:09.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.938+0000 7f6ec0c00640 1 -- 172.21.15.16:0/2845501541 >> 172.21.15.16:0/2845501541 conn(0x7f6ebc0ff370 msgr2=0x7f6ebc1100f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:09.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.939+0000 7f6ec0c00640 1 -- 172.21.15.16:0/2845501541 shutdown_connections 2024-08-31T06:17:09.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:09.939+0000 7f6ec0c00640 1 -- 172.21.15.16:0/2845501541 wait complete. 2024-08-31T06:17:09.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:09 smithi184 ceph-mon[22331]: osdmap e90: 12 total, 12 up, 12 in 2024-08-31T06:17:10.785 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:10 smithi016 ceph-mon[17194]: pgmap v671: 97 pgs: 97 active+clean; 3.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 1.6 MiB/s wr, 196 op/s 2024-08-31T06:17:10.785 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:10 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/2845501541' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:17:10.824 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24737 quiesce db --set-id a57ea9ba --release --await' 2024-08-31T06:17:10.824 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24737 quiesce db --set-id a57ea9ba --release --await 2024-08-31T06:17:10.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:10 smithi155 ceph-mon[21742]: pgmap v671: 97 pgs: 97 active+clean; 3.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 1.6 MiB/s wr, 196 op/s 2024-08-31T06:17:10.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:10 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/2845501541' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:17:10.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:10 smithi184 ceph-mon[22331]: pgmap v671: 97 pgs: 97 active+clean; 3.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 1.6 MiB/s wr, 196 op/s 2024-08-31T06:17:10.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:10 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/2845501541' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:17:11.205 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:17:11.628 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 585.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-31T06:17:11.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.891+0000 7fb60de00640 1 -- 172.21.15.16:0/4199927194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb608103900 msgr2=0x7fb608103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:11.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.891+0000 7fb60de00640 1 --2- 172.21.15.16:0/4199927194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb608103900 0x7fb608103d10 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8009900 tx=0x7fb5f802a410 comp rx=0 tx=0).stop 2024-08-31T06:17:11.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.891+0000 7fb60de00640 1 -- 172.21.15.16:0/4199927194 shutdown_connections 2024-08-31T06:17:11.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.891+0000 7fb60de00640 1 --2- 172.21.15.16:0/4199927194 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb608109240 0x7fb608109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:11.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.891+0000 7fb60de00640 1 --2- 172.21.15.16:0/4199927194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb608109b90 0x7fb6081103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:11.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.891+0000 7fb60de00640 1 --2- 172.21.15.16:0/4199927194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb608103900 0x7fb608103d10 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:11.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.891+0000 7fb60de00640 1 -- 172.21.15.16:0/4199927194 >> 172.21.15.16:0/4199927194 conn(0x7fb60807b5f0 msgr2=0x7fb60807ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:11.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.892+0000 7fb60de00640 1 -- 172.21.15.16:0/4199927194 shutdown_connections 2024-08-31T06:17:11.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.892+0000 7fb60de00640 1 -- 172.21.15.16:0/4199927194 wait complete. 2024-08-31T06:17:11.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.893+0000 7fb60de00640 1 Processor -- start 2024-08-31T06:17:11.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.893+0000 7fb60de00640 1 -- start start 2024-08-31T06:17:11.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.893+0000 7fb60de00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb608103900 0x7fb6080754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:11.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.893+0000 7fb60de00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb608109240 0x7fb6080759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:11.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.893+0000 7fb60de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb608109b90 0x7fb608075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:11.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.893+0000 7fb60de00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb6081138a0 con 0x7fb608109b90 2024-08-31T06:17:11.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.893+0000 7fb60de00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb608113720 con 0x7fb608109240 2024-08-31T06:17:11.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.893+0000 7fb60de00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb608113a20 con 0x7fb608103900 2024-08-31T06:17:11.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.894+0000 7fb606000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb608109240 0x7fb6080759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:11.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.894+0000 7fb607400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb608109b90 0x7fb608075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:11.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.894+0000 7fb607400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb608109b90 0x7fb608075f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47690/0 (socket says 172.21.15.16:47690) 2024-08-31T06:17:11.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.894+0000 7fb606a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb608103900 0x7fb6080754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:11.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.894+0000 7fb606000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb608109240 0x7fb6080759f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:59906/0 (socket says 172.21.15.16:59906) 2024-08-31T06:17:11.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.894+0000 7fb607400640 1 -- 172.21.15.16:0/2805458209 learned_addr learned my addr 172.21.15.16:0/2805458209 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:11.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.894+0000 7fb606000640 1 -- 172.21.15.16:0/2805458209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb608103900 msgr2=0x7fb6080754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:11.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.894+0000 7fb606000640 1 --2- 172.21.15.16:0/2805458209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb608103900 0x7fb6080754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:11.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.894+0000 7fb606000640 1 -- 172.21.15.16:0/2805458209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb608109b90 msgr2=0x7fb608075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:11.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.894+0000 7fb606000640 1 --2- 172.21.15.16:0/2805458209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb608109b90 0x7fb608075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:11.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.894+0000 7fb606000640 1 -- 172.21.15.16:0/2805458209 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6081a8550 con 0x7fb608109240 2024-08-31T06:17:11.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.894+0000 7fb606000640 1 --2- 172.21.15.16:0/2805458209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb608109240 0x7fb6080759f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fb60806c790 tx=0x7fb5f400ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:11.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.895+0000 7fb60ca00640 1 -- 172.21.15.16:0/2805458209 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f400cc40 con 0x7fb608109240 2024-08-31T06:17:11.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.895+0000 7fb60de00640 1 -- 172.21.15.16:0/2805458209 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6081a8840 con 0x7fb608109240 2024-08-31T06:17:11.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.895+0000 7fb60ca00640 1 -- 172.21.15.16:0/2805458209 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5f400cde0 con 0x7fb608109240 2024-08-31T06:17:11.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.895+0000 7fb60ca00640 1 -- 172.21.15.16:0/2805458209 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4004a00 con 0x7fb608109240 2024-08-31T06:17:11.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.895+0000 7fb60de00640 1 -- 172.21.15.16:0/2805458209 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6081a8d50 con 0x7fb608109240 2024-08-31T06:17:11.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.898+0000 7fb614aa3740 1 Processor -- start 2024-08-31T06:17:11.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.898+0000 7fb614aa3740 1 -- start start 2024-08-31T06:17:11.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.898+0000 7fb614aa3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdffa2f520 0x55fdffdd4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:11.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.899+0000 7fb614aa3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fdffa06660 0x55fdffdd5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:11.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.899+0000 7fb614aa3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdffa0e840 0x55fdffdd5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:11.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.899+0000 7fb614aa3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fdffd4ba90 con 0x55fdffa2f520 2024-08-31T06:17:11.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.899+0000 7fb60ca00640 1 -- 172.21.15.16:0/2805458209 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fb5f4004ba0 con 0x7fb608109240 2024-08-31T06:17:11.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.899+0000 7fb614aa3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55fdffd3b020 con 0x55fdffa06660 2024-08-31T06:17:11.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.899+0000 7fb614aa3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55fdffd3efd0 con 0x55fdffa0e840 2024-08-31T06:17:11.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.899+0000 7fb607400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdffa2f520 0x55fdffdd4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:11.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.899+0000 7fb606000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdffa0e840 0x55fdffdd5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:11.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.899+0000 7fb607400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdffa2f520 0x55fdffdd4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47694/0 (socket says 172.21.15.16:47694) 2024-08-31T06:17:11.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.899+0000 7fb607400640 1 -- 172.21.15.16:0/347053038 learned_addr learned my addr 172.21.15.16:0/347053038 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:11.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.899+0000 7fb606000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdffa0e840 0x55fdffdd5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:39434/0 (socket says 172.21.15.16:39434) 2024-08-31T06:17:11.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.899+0000 7fb606a00640 1 --2- 172.21.15.16:0/347053038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fdffa06660 0x55fdffdd5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:11.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.899+0000 7fb606000640 1 -- 172.21.15.16:0/347053038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fdffa06660 msgr2=0x55fdffdd5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:11.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.899+0000 7fb606000640 1 --2- 172.21.15.16:0/347053038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fdffa06660 0x55fdffdd5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:11.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.899+0000 7fb606000640 1 -- 172.21.15.16:0/347053038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdffa2f520 msgr2=0x55fdffdd4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:11.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.899+0000 7fb606000640 1 --2- 172.21.15.16:0/347053038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdffa2f520 0x55fdffdd4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:11.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.899+0000 7fb606000640 1 -- 172.21.15.16:0/347053038 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdffa9ab00 con 0x55fdffa0e840 2024-08-31T06:17:11.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.900+0000 7fb607400640 1 --2- 172.21.15.16:0/347053038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdffa2f520 0x55fdffdd4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:17:11.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.900+0000 7fb606000640 1 --2- 172.21.15.16:0/347053038 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdffa0e840 0x55fdffdd5970 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40e2b50 tx=0x7fb5f40eb6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad8dfb3c529d5434 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:11.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.900+0000 7fb5f3e00640 1 -- 172.21.15.16:0/347053038 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f40f9070 con 0x55fdffa0e840 2024-08-31T06:17:11.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.900+0000 7fb5f3e00640 1 -- 172.21.15.16:0/347053038 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5f40e7580 con 0x55fdffa0e840 2024-08-31T06:17:11.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.900+0000 7fb5f3e00640 1 -- 172.21.15.16:0/347053038 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f40f1780 con 0x55fdffa0e840 2024-08-31T06:17:11.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.900+0000 7fb60ca00640 1 --2- 172.21.15.16:0/2805458209 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb5ec0b4d20 0x7fb5ec0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:11.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.901+0000 7fb60ca00640 1 -- 172.21.15.16:0/2805458209 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8514+0+0 (secure 0 0 0) 0x7fb5f40e5400 con 0x7fb608109240 2024-08-31T06:17:11.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.901+0000 7fb607400640 1 --2- 172.21.15.16:0/2805458209 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb5ec0b4d20 0x7fb5ec0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:11.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.901+0000 7fb614aa3740 1 -- 172.21.15.16:0/347053038 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdffa0e840 msgr2=0x55fdffdd5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:11.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.901+0000 7fb614aa3740 1 --2- 172.21.15.16:0/347053038 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdffa0e840 0x55fdffdd5970 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40e2b50 tx=0x7fb5f40eb6a0 comp rx=0 tx=0).stop 2024-08-31T06:17:11.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.901+0000 7fb614aa3740 1 -- 172.21.15.16:0/347053038 shutdown_connections 2024-08-31T06:17:11.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.901+0000 7fb614aa3740 1 --2- 172.21.15.16:0/347053038 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdffa0e840 0x55fdffdd5970 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:11.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.901+0000 7fb614aa3740 1 --2- 172.21.15.16:0/347053038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fdffa06660 0x55fdffdd5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:11.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.901+0000 7fb614aa3740 1 --2- 172.21.15.16:0/347053038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdffa2f520 0x55fdffdd4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:11.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.901+0000 7fb614aa3740 1 -- 172.21.15.16:0/347053038 >> 172.21.15.16:0/347053038 conn(0x55fdffa23910 msgr2=0x55fdffdd2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:11.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.901+0000 7fb614aa3740 1 -- 172.21.15.16:0/347053038 shutdown_connections 2024-08-31T06:17:11.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.902+0000 7fb614aa3740 1 -- 172.21.15.16:0/347053038 wait complete. 2024-08-31T06:17:11.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.902+0000 7fb607400640 1 --2- 172.21.15.16:0/2805458209 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb5ec0b4d20 0x7fb5ec0b7230 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb600012960 tx=0x7fb600009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:11.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.902+0000 7fb614aa3740 1 -- start start 2024-08-31T06:17:11.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.902+0000 7fb614aa3740 1 Processor -- start 2024-08-31T06:17:11.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.903+0000 7fb614aa3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdffa06660 0x55fdffde0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:11.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.903+0000 7fb614aa3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fdffa0e840 0x55fdffde0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:11.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.903+0000 7fb614aa3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdffa2f520 0x55fdffde0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:11.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.903+0000 7fb614aa3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fdffb9dbb0 con 0x55fdffa06660 2024-08-31T06:17:11.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.903+0000 7fb614aa3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55fdffcc1f90 con 0x55fdffa0e840 2024-08-31T06:17:11.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.903+0000 7fb614aa3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55fdffde36d0 con 0x55fdffa2f520 2024-08-31T06:17:11.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.903+0000 7fb614aa3740 10 client.0 _ll_register_callbacks cb 0x55fdffce7710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:17:11.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.903+0000 7fb606000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fdffa0e840 0x55fdffde0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:11.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.903+0000 7fb606a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdffa06660 0x55fdffde0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:11.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.903+0000 7fb606000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fdffa0e840 0x55fdffde0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:59922/0 (socket says 172.21.15.16:59922) 2024-08-31T06:17:11.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.903+0000 7fb606000640 1 -- 172.21.15.16:0/1758206070 learned_addr learned my addr 172.21.15.16:0/1758206070 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:11.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.903+0000 7fb607400640 1 --2- 172.21.15.16:0/1758206070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdffa2f520 0x55fdffde0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:11.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.904+0000 7fb606000640 1 -- 172.21.15.16:0/1758206070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdffa2f520 msgr2=0x55fdffde0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:11.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.904+0000 7fb606000640 1 --2- 172.21.15.16:0/1758206070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdffa2f520 0x55fdffde0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:11.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.904+0000 7fb606000640 1 -- 172.21.15.16:0/1758206070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdffa06660 msgr2=0x55fdffde0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:11.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.904+0000 7fb606000640 1 --2- 172.21.15.16:0/1758206070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdffa06660 0x55fdffde0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:11.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.904+0000 7fb606000640 1 -- 172.21.15.16:0/1758206070 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdffb11960 con 0x55fdffa0e840 2024-08-31T06:17:11.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.904+0000 7fb606000640 1 --2- 172.21.15.16:0/1758206070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fdffa0e840 0x55fdffde0850 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40a7b70 tx=0x7fb5f40053f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=923900da6fec7031 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:11.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.904+0000 7fb5f0c00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:17:11.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.904+0000 7fb5f0c00640 1 -- 172.21.15.16:0/1758206070 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f401c070 con 0x55fdffa0e840 2024-08-31T06:17:11.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.904+0000 7fb5f0c00640 1 -- 172.21.15.16:0/1758206070 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8514+0+0 (secure 0 0 0) 0x7fb5f4005020 con 0x55fdffa0e840 2024-08-31T06:17:11.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.904+0000 7fb614aa3740 1 -- 172.21.15.16:0/1758206070 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fdffcd9e30 con 0x55fdffa0e840 2024-08-31T06:17:11.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.904+0000 7fb5f0c00640 1 -- 172.21.15.16:0/1758206070 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fb5f40a79b0 con 0x55fdffa0e840 2024-08-31T06:17:11.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.905+0000 7fb614aa3740 10 client.27818 fetch_fsmap learned FSMap version 43 2024-08-31T06:17:11.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.905+0000 7fb614aa3740 1 -- 172.21.15.16:0/1758206070 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55fdffdef150 con 0x55fdffa0e840 2024-08-31T06:17:11.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.905+0000 7fb5f0c00640 1 -- 172.21.15.16:0/1758206070 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55fdffcd9e30 con 0x55fdffa0e840 2024-08-31T06:17:11.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.905+0000 7fb5f0c00640 1 -- 172.21.15.16:0/1758206070 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fb5f40e2dd0 con 0x55fdffa0e840 2024-08-31T06:17:11.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.905+0000 7fb614aa3740 10 client.27818 fetch_fsmap finished waiting for FSMap version 43 2024-08-31T06:17:11.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.905+0000 7fb614aa3740 10 client.27818 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:17:11.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.905+0000 7fb614aa3740 20 client.27818 populate_metadata read hostname 'smithi016' 2024-08-31T06:17:11.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.905+0000 7fb614aa3740 1 --2- 172.21.15.16:0/1758206070 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fdff9edea0 0x55fdffdf24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:11.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.905+0000 7fb614aa3740 4 client.27818 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:17:11.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.905+0000 7fb614aa3740 1 -- 172.21.15.16:0/1758206070 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fdffd4d9e0 con 0x55fdff9edea0 2024-08-31T06:17:11.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.905+0000 7fb606a00640 1 --2- 172.21.15.16:0/1758206070 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fdff9edea0 0x55fdffdf24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:11.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.906+0000 7fb606a00640 1 --2- 172.21.15.16:0/1758206070 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fdff9edea0 0x55fdffdf24b0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=771dff3cc849c110 server_cookie=bb8c60578be36d5c in_seq=0 out_seq=0 2024-08-31T06:17:11.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.906+0000 7fb5f0c00640 10 client.27818 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:17:11.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.907+0000 7fb5f0c00640 1 -- 172.21.15.16:0/1758206070 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fdffd4d9e0 con 0x55fdff9edea0 2024-08-31T06:17:11.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.907+0000 7fb5f0c00640 10 client.27818 handle_command_reply: tid=0 2024-08-31T06:17:11.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.907+0000 7fb614aa3740 1 client.27818 shutdown 2024-08-31T06:17:11.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.907+0000 7fb614aa3740 1 -- 172.21.15.16:0/1758206070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fdffa0e840 msgr2=0x55fdffde0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:11.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.907+0000 7fb614aa3740 1 --2- 172.21.15.16:0/1758206070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fdffa0e840 0x55fdffde0850 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40a7b70 tx=0x7fb5f40053f0 comp rx=0 tx=0).stop 2024-08-31T06:17:11.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.907+0000 7fb614aa3740 1 -- 172.21.15.16:0/1758206070 shutdown_connections 2024-08-31T06:17:11.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.907+0000 7fb614aa3740 1 --2- 172.21.15.16:0/1758206070 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fdff9edea0 0x55fdffdf24b0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:11.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.907+0000 7fb614aa3740 1 --2- 172.21.15.16:0/1758206070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdffa2f520 0x55fdffde0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.907+0000 7fb614aa3740 1 --2- 172.21.15.16:0/1758206070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fdffa0e840 0x55fdffde0850 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.907+0000 7fb5f0c00640 0 client.27818 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:17:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.907+0000 7fb614aa3740 1 --2- 172.21.15.16:0/1758206070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdffa06660 0x55fdffde0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.907+0000 7fb614aa3740 1 -- 172.21.15.16:0/1758206070 >> 172.21.15.16:0/1758206070 conn(0x55fdffa23910 msgr2=0x55fdffdd80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.907+0000 7fb614aa3740 1 -- 172.21.15.16:0/1758206070 shutdown_connections 2024-08-31T06:17:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.907+0000 7fb614aa3740 1 -- 172.21.15.16:0/1758206070 wait complete. 2024-08-31T06:17:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.908+0000 7fb614aa3740 20 client.27818 trim_cache size 0 max 16384 2024-08-31T06:17:11.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.926+0000 7fb614aa3740 1 Processor -- start 2024-08-31T06:17:11.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.926+0000 7fb614aa3740 1 -- start start 2024-08-31T06:17:11.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.926+0000 7fb614aa3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fdffa06660 0x55fdffde21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:11.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.926+0000 7fb614aa3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdffa0e840 0x55fdffdec930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:11.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.926+0000 7fb614aa3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdffa2f520 0x55fdffdd6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:11.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.926+0000 7fb614aa3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb6081b3e60 con 0x55fdffa2f520 2024-08-31T06:17:11.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.926+0000 7fb614aa3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb6081b16d0 con 0x55fdffa06660 2024-08-31T06:17:11.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.926+0000 7fb614aa3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb6081af540 con 0x55fdffa0e840 2024-08-31T06:17:11.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.926+0000 7fb606a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fdffa06660 0x55fdffde21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:11.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.926+0000 7fb606a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fdffa06660 0x55fdffde21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:59936/0 (socket says 172.21.15.16:59936) 2024-08-31T06:17:11.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.926+0000 7fb606a00640 1 -- 172.21.15.16:0/1542700479 learned_addr learned my addr 172.21.15.16:0/1542700479 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:11.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.926+0000 7fb607400640 1 --2- 172.21.15.16:0/1542700479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdffa2f520 0x55fdffdd6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:11.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.927+0000 7fb606000640 1 --2- 172.21.15.16:0/1542700479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdffa0e840 0x55fdffdec930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:11.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.927+0000 7fb606a00640 1 -- 172.21.15.16:0/1542700479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdffa0e840 msgr2=0x55fdffdec930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:11.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.927+0000 7fb606a00640 1 --2- 172.21.15.16:0/1542700479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdffa0e840 0x55fdffdec930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:11.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.927+0000 7fb606a00640 1 -- 172.21.15.16:0/1542700479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdffa2f520 msgr2=0x55fdffdd6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:11.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.927+0000 7fb606a00640 1 --2- 172.21.15.16:0/1542700479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdffa2f520 0x55fdffdd6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:11.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.927+0000 7fb606a00640 1 -- 172.21.15.16:0/1542700479 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdffddbea0 con 0x55fdffa06660 2024-08-31T06:17:11.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.927+0000 7fb606a00640 1 --2- 172.21.15.16:0/1542700479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fdffa06660 0x55fdffde21f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8005940 tx=0x7fb5f8005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e25d6d6c3e039f8c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:11.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.927+0000 7fb5f3400640 1 -- 172.21.15.16:0/1542700479 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f804a020 con 0x55fdffa06660 2024-08-31T06:17:11.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.928+0000 7fb5f3400640 1 -- 172.21.15.16:0/1542700479 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5f8004770 con 0x55fdffa06660 2024-08-31T06:17:11.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.928+0000 7fb5f3400640 1 -- 172.21.15.16:0/1542700479 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f8035040 con 0x55fdffa06660 2024-08-31T06:17:11.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.928+0000 7fb614aa3740 1 -- 172.21.15.16:0/1542700479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fdffa06660 msgr2=0x55fdffde21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:11.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.928+0000 7fb614aa3740 1 --2- 172.21.15.16:0/1542700479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fdffa06660 0x55fdffde21f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8005940 tx=0x7fb5f8005a20 comp rx=0 tx=0).stop 2024-08-31T06:17:11.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.928+0000 7fb614aa3740 1 -- 172.21.15.16:0/1542700479 shutdown_connections 2024-08-31T06:17:11.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.928+0000 7fb614aa3740 1 --2- 172.21.15.16:0/1542700479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdffa0e840 0x55fdffdec930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:11.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.928+0000 7fb614aa3740 1 --2- 172.21.15.16:0/1542700479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdffa2f520 0x55fdffdd6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:11.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.928+0000 7fb614aa3740 1 --2- 172.21.15.16:0/1542700479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fdffa06660 0x55fdffde21f0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:11.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.928+0000 7fb614aa3740 1 -- 172.21.15.16:0/1542700479 >> 172.21.15.16:0/1542700479 conn(0x55fdffa23910 msgr2=0x55fdffdd5f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:11.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.928+0000 7fb614aa3740 1 -- 172.21.15.16:0/1542700479 shutdown_connections 2024-08-31T06:17:11.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.928+0000 7fb614aa3740 1 -- 172.21.15.16:0/1542700479 wait complete. 2024-08-31T06:17:11.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.929+0000 7fb614aa3740 1 -- start start 2024-08-31T06:17:11.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.929+0000 7fb614aa3740 1 Processor -- start 2024-08-31T06:17:11.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.930+0000 7fb614aa3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdffa06660 0x55fdffdf3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:11.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.930+0000 7fb614aa3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdffa0e840 0x55fdffdf4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:11.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.930+0000 7fb614aa3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fdffa2f520 0x55fdffe5bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:11.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.930+0000 7fb614aa3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb6081afb90 con 0x55fdffa06660 2024-08-31T06:17:11.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.930+0000 7fb614aa3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb6081b1000 con 0x55fdffa2f520 2024-08-31T06:17:11.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.930+0000 7fb614aa3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb6081b0260 con 0x55fdffa0e840 2024-08-31T06:17:11.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.930+0000 7fb614aa3740 10 client.0 _ll_register_callbacks cb 0x7fb5f4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:17:11.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.930+0000 7fb606a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdffa06660 0x55fdffdf3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:11.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.930+0000 7fb607400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fdffa2f520 0x55fdffe5bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:11.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.930+0000 7fb606a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdffa06660 0x55fdffdf3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47712/0 (socket says 172.21.15.16:47712) 2024-08-31T06:17:11.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.930+0000 7fb606a00640 1 -- 172.21.15.16:0/2043535047 learned_addr learned my addr 172.21.15.16:0/2043535047 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:11.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.930+0000 7fb606000640 1 --2- 172.21.15.16:0/2043535047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdffa0e840 0x55fdffdf4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:11.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.930+0000 7fb606a00640 1 -- 172.21.15.16:0/2043535047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdffa0e840 msgr2=0x55fdffdf4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:11.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.930+0000 7fb606a00640 1 --2- 172.21.15.16:0/2043535047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdffa0e840 0x55fdffdf4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:11.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.930+0000 7fb606a00640 1 -- 172.21.15.16:0/2043535047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fdffa2f520 msgr2=0x55fdffe5bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:11.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.930+0000 7fb606a00640 1 --2- 172.21.15.16:0/2043535047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fdffa2f520 0x55fdffe5bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:11.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.930+0000 7fb606a00640 1 -- 172.21.15.16:0/2043535047 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdffe5c170 con 0x55fdffa06660 2024-08-31T06:17:11.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.930+0000 7fb606000640 1 --2- 172.21.15.16:0/2043535047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdffa0e840 0x55fdffdf4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:17:11.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.930+0000 7fb607400640 1 --2- 172.21.15.16:0/2043535047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fdffa2f520 0x55fdffe5bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:17:11.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.931+0000 7fb606a00640 1 --2- 172.21.15.16:0/2043535047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdffa06660 0x55fdffdf3c20 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fb5f80095f0 tx=0x7fb5f802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2bee6717b1e098f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:11.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.931+0000 7fb5f0c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:17:11.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.931+0000 7fb5f0c00640 1 -- 172.21.15.16:0/2043535047 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f804a030 con 0x55fdffa06660 2024-08-31T06:17:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.931+0000 7fb614aa3740 1 -- 172.21.15.16:0/2043535047 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5f8004770 con 0x55fdffa06660 2024-08-31T06:17:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.931+0000 7fb5f0c00640 1 -- 172.21.15.16:0/2043535047 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8514+0+0 (secure 0 0 0) 0x7fb5f8004d60 con 0x55fdffa06660 2024-08-31T06:17:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.931+0000 7fb5f0c00640 1 -- 172.21.15.16:0/2043535047 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fb5f8035040 con 0x55fdffa06660 2024-08-31T06:17:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.931+0000 7fb614aa3740 10 client.17517 fetch_fsmap learned FSMap version 43 2024-08-31T06:17:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.931+0000 7fb614aa3740 1 -- 172.21.15.16:0/2043535047 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55fdffe5d430 con 0x55fdffa06660 2024-08-31T06:17:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.932+0000 7fb5f0c00640 1 -- 172.21.15.16:0/2043535047 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5f8004770 con 0x55fdffa06660 2024-08-31T06:17:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.932+0000 7fb5f0c00640 1 -- 172.21.15.16:0/2043535047 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fb5f8003bd0 con 0x55fdffa06660 2024-08-31T06:17:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.932+0000 7fb614aa3740 10 client.17517 fetch_fsmap finished waiting for FSMap version 43 2024-08-31T06:17:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.932+0000 7fb614aa3740 10 client.17517 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:17:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.932+0000 7fb614aa3740 20 client.17517 populate_metadata read hostname 'smithi016' 2024-08-31T06:17:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.932+0000 7fb614aa3740 1 --2- 172.21.15.16:0/2043535047 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fdff9c3250 0x55fdffe5fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.932+0000 7fb614aa3740 4 client.17517 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a57ea9ba", "release": true, "await": true}] 2024-08-31T06:17:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.932+0000 7fb614aa3740 1 -- 172.21.15.16:0/2043535047 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a57ea9ba", "release": true, "await": true}) -- 0x55fdffdd3340 con 0x55fdff9c3250 2024-08-31T06:17:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.932+0000 7fb606000640 1 --2- 172.21.15.16:0/2043535047 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fdff9c3250 0x55fdffe5fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:11.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.933+0000 7fb606000640 1 --2- 172.21.15.16:0/2043535047 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fdff9c3250 0x55fdffe5fbb0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=280d178746d38e67 server_cookie=399942c51378da53 in_seq=0 out_seq=0 2024-08-31T06:17:11.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.933+0000 7fb5f0c00640 10 client.17517 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:17:11.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.976+0000 7fb5f0c00640 1 -- 172.21.15.16:0/2043535047 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55fdffdd3340 con 0x55fdff9c3250 2024-08-31T06:17:11.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.976+0000 7fb5f0c00640 10 client.17517 handle_command_reply: tid=0 2024-08-31T06:17:11.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.976+0000 7fb614aa3740 1 client.17517 shutdown 2024-08-31T06:17:11.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.980+0000 7fb614aa3740 1 -- 172.21.15.16:0/2043535047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdffa06660 msgr2=0x55fdffdf3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:11.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.980+0000 7fb614aa3740 1 --2- 172.21.15.16:0/2043535047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdffa06660 0x55fdffdf3c20 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fb5f80095f0 tx=0x7fb5f802a410 comp rx=0 tx=0).stop 2024-08-31T06:17:11.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.981+0000 7fb614aa3740 1 -- 172.21.15.16:0/2043535047 shutdown_connections 2024-08-31T06:17:11.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.981+0000 7fb614aa3740 1 --2- 172.21.15.16:0/2043535047 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fdff9c3250 0x55fdffe5fbb0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:11.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.981+0000 7fb614aa3740 1 --2- 172.21.15.16:0/2043535047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fdffa0e840 0x55fdffdf4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:11.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.981+0000 7fb614aa3740 1 --2- 172.21.15.16:0/2043535047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fdffa2f520 0x55fdffe5bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:11.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.981+0000 7fb614aa3740 1 --2- 172.21.15.16:0/2043535047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdffa06660 0x55fdffdf3c20 secure :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fb5f80095f0 tx=0x7fb5f802a410 comp rx=0 tx=0).stop 2024-08-31T06:17:11.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.981+0000 7fb614aa3740 1 -- 172.21.15.16:0/2043535047 >> 172.21.15.16:0/2043535047 conn(0x55fdffa23910 msgr2=0x55fdffdd68e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:11.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.985+0000 7fb5f0c00640 0 client.17517 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:17:11.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.986+0000 7fb614aa3740 1 -- 172.21.15.16:0/2043535047 shutdown_connections 2024-08-31T06:17:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.986+0000 7fb614aa3740 1 -- 172.21.15.16:0/2043535047 wait complete. 2024-08-31T06:17:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.986+0000 7fb614aa3740 20 client.17517 trim_cache size 0 max 16384 2024-08-31T06:17:11.990 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:17:11.990 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 43, 2024-08-31T06:17:11.990 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24737, 2024-08-31T06:17:11.990 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 32, 2024-08-31T06:17:11.990 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-08-31T06:17:11.991 INFO:teuthology.orchestra.run.smithi016.stdout: "a57ea9ba": { 2024-08-31T06:17:11.991 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 32, 2024-08-31T06:17:11.991 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-08-31T06:17:11.991 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:17:11.991 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-08-31T06:17:11.991 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:17:11.991 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:17:11.991 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-08-31T06:17:11.991 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 135.5, 2024-08-31T06:17:11.991 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-08-31T06:17:11.991 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-08-31T06:17:11.991 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-08-31T06:17:11.991 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:17:11.991 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-08-31T06:17:11.991 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:17:11.991 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:17:11.992 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:17:11.992 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:17:11.992 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:17:11.992 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:17:11.992 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:17:11.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.989+0000 7fb60de00640 1 -- 172.21.15.16:0/2805458209 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb5ec0b4d20 msgr2=0x7fb5ec0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:11.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.990+0000 7fb60de00640 1 --2- 172.21.15.16:0/2805458209 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb5ec0b4d20 0x7fb5ec0b7230 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb600012960 tx=0x7fb600009500 comp rx=0 tx=0).stop 2024-08-31T06:17:11.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.990+0000 7fb60de00640 1 -- 172.21.15.16:0/2805458209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb608109240 msgr2=0x7fb6080759f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:11.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.990+0000 7fb60de00640 1 --2- 172.21.15.16:0/2805458209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb608109240 0x7fb6080759f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fb60806c790 tx=0x7fb5f400ee50 comp rx=0 tx=0).stop 2024-08-31T06:17:11.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.992+0000 7fb60de00640 1 -- 172.21.15.16:0/2805458209 shutdown_connections 2024-08-31T06:17:11.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.992+0000 7fb60de00640 1 --2- 172.21.15.16:0/2805458209 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb5ec0b4d20 0x7fb5ec0b7230 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:11.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.992+0000 7fb60de00640 1 --2- 172.21.15.16:0/2805458209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb608109b90 0x7fb608075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:11.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.992+0000 7fb60de00640 1 --2- 172.21.15.16:0/2805458209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb608109240 0x7fb6080759f0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:11.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.992+0000 7fb60de00640 1 --2- 172.21.15.16:0/2805458209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb608103900 0x7fb6080754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:11.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.992+0000 7fb60de00640 1 -- 172.21.15.16:0/2805458209 >> 172.21.15.16:0/2805458209 conn(0x7fb60807b5f0 msgr2=0x7fb6081072d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:11.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.992+0000 7fb60de00640 1 -- 172.21.15.16:0/2805458209 shutdown_connections 2024-08-31T06:17:11.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:11.994+0000 7fb60de00640 1 -- 172.21.15.16:0/2805458209 wait complete. 2024-08-31T06:17:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.343+0000 7f32c7400640 1 -- 172.21.15.16:0/1416473988 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b8035020 con 0x7f32c8102590 2024-08-31T06:17:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.344+0000 7f32cde00640 1 -- 172.21.15.16:0/1416473988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32c8102590 msgr2=0x7f32c81029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.344+0000 7f32cde00640 1 --2- 172.21.15.16:0/1416473988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32c8102590 0x7f32c81029a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f32b8009900 tx=0x7f32b802a410 comp rx=0 tx=0).stop 2024-08-31T06:17:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.345+0000 7f32cde00640 1 -- 172.21.15.16:0/1416473988 shutdown_connections 2024-08-31T06:17:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.345+0000 7f32cde00640 1 --2- 172.21.15.16:0/1416473988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32c8108810 0x7f32c810f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.345+0000 7f32cde00640 1 --2- 172.21.15.16:0/1416473988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32c8107ec0 0x7f32c81082d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.345+0000 7f32cde00640 1 --2- 172.21.15.16:0/1416473988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32c8102590 0x7f32c81029a0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.345+0000 7f32cde00640 1 -- 172.21.15.16:0/1416473988 >> 172.21.15.16:0/1416473988 conn(0x7f32c80fdfd0 msgr2=0x7f32c8100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.345+0000 7f32cde00640 1 -- 172.21.15.16:0/1416473988 shutdown_connections 2024-08-31T06:17:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.345+0000 7f32cde00640 1 -- 172.21.15.16:0/1416473988 wait complete. 2024-08-31T06:17:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.347+0000 7f32cde00640 1 Processor -- start 2024-08-31T06:17:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.347+0000 7f32cde00640 1 -- start start 2024-08-31T06:17:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.348+0000 7f32cde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32c8102590 0x7f32c8074a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:12.351 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.348+0000 7f32cde00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32c8107ec0 0x7f32c806dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.349+0000 7f32cde00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32c8108810 0x7f32c806e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.349+0000 7f32c7e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32c8107ec0 0x7f32c806dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.349+0000 7f32c7e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32c8107ec0 0x7f32c806dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:39474/0 (socket says 172.21.15.16:39474) 2024-08-31T06:17:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.350+0000 7f32c7e00640 1 -- 172.21.15.16:0/3188235832 learned_addr learned my addr 172.21.15.16:0/3188235832 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.350+0000 7f32cde00640 1 -- 172.21.15.16:0/3188235832 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f32c8113660 con 0x7f32c8102590 2024-08-31T06:17:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.350+0000 7f32cde00640 1 -- 172.21.15.16:0/3188235832 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f32c81134e0 con 0x7f32c8108810 2024-08-31T06:17:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.350+0000 7f32cde00640 1 -- 172.21.15.16:0/3188235832 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f32c81137e0 con 0x7f32c8107ec0 2024-08-31T06:17:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.350+0000 7f32cd400640 1 --2- 172.21.15.16:0/3188235832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32c8108810 0x7f32c806e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.351+0000 7f32cd400640 1 -- 172.21.15.16:0/3188235832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32c8107ec0 msgr2=0x7f32c806dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.351+0000 7f32cd400640 1 --2- 172.21.15.16:0/3188235832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32c8107ec0 0x7f32c806dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.351+0000 7f32cd400640 1 -- 172.21.15.16:0/3188235832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32c8102590 msgr2=0x7f32c8074a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.351+0000 7f32cd400640 1 --2- 172.21.15.16:0/3188235832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32c8102590 0x7f32c8074a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.351+0000 7f32cd400640 1 -- 172.21.15.16:0/3188235832 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c806e9c0 con 0x7f32c8108810 2024-08-31T06:17:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.352+0000 7f32c7e00640 1 --2- 172.21.15.16:0/3188235832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32c8107ec0 0x7f32c806dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:17:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.352+0000 7f32cd400640 1 --2- 172.21.15.16:0/3188235832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32c8108810 0x7f32c806e290 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f32c0009fd0 tx=0x7f32c000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.352+0000 7f32c5600640 1 -- 172.21.15.16:0/3188235832 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c000ce20 con 0x7f32c8108810 2024-08-31T06:17:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.353+0000 7f32c5600640 1 -- 172.21.15.16:0/3188235832 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32c0005020 con 0x7f32c8108810 2024-08-31T06:17:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.353+0000 7f32c5600640 1 -- 172.21.15.16:0/3188235832 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c0010690 con 0x7f32c8108810 2024-08-31T06:17:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.353+0000 7f32cde00640 1 -- 172.21.15.16:0/3188235832 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32c80736f0 con 0x7f32c8108810 2024-08-31T06:17:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.353+0000 7f32cde00640 1 -- 172.21.15.16:0/3188235832 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32c8073c70 con 0x7f32c8108810 2024-08-31T06:17:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.359+0000 7f32d6453740 1 Processor -- start 2024-08-31T06:17:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.361+0000 7f32d6453740 1 -- start start 2024-08-31T06:17:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.361+0000 7f32d6453740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567c3c231b0 0x5567c3fcf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.361+0000 7f32d6453740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567c3bfa2f0 0x5567c3fcf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.361+0000 7f32d6453740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567c3c024d0 0x5567c3fcfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.361+0000 7f32d6453740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5567c3f3bf70 con 0x5567c3c231b0 2024-08-31T06:17:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.361+0000 7f32d6453740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5567c3f2b500 con 0x5567c3c024d0 2024-08-31T06:17:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.361+0000 7f32d6453740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5567c3f2f4b0 con 0x5567c3bfa2f0 2024-08-31T06:17:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.363+0000 7f32c7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567c3c231b0 0x5567c3fcf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.363+0000 7f32cca00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567c3bfa2f0 0x5567c3fcf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.364+0000 7f32cca00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567c3bfa2f0 0x5567c3fcf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:39482/0 (socket says 172.21.15.16:39482) 2024-08-31T06:17:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.364+0000 7f32cca00640 1 -- 172.21.15.16:0/1463529203 learned_addr learned my addr 172.21.15.16:0/1463529203 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.364+0000 7f32c5600640 1 -- 172.21.15.16:0/3188235832 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f32c00054d0 con 0x7f32c8108810 2024-08-31T06:17:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.365+0000 7f32cca00640 1 -- 172.21.15.16:0/1463529203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567c3c024d0 msgr2=0x5567c3fcfb30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T06:17:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.365+0000 7f32cca00640 1 --2- 172.21.15.16:0/1463529203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567c3c024d0 0x5567c3fcfb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.365+0000 7f32cca00640 1 -- 172.21.15.16:0/1463529203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567c3c231b0 msgr2=0x5567c3fcf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.365+0000 7f32cca00640 1 --2- 172.21.15.16:0/1463529203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567c3c231b0 0x5567c3fcf0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.365+0000 7f32cca00640 1 -- 172.21.15.16:0/1463529203 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567c3ee5c10 con 0x5567c3bfa2f0 2024-08-31T06:17:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.365+0000 7f32cca00640 1 --2- 172.21.15.16:0/1463529203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567c3bfa2f0 0x5567c3fcf5f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f32b8005f30 tx=0x7f32b800b050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1d11e5c68ffe66b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.366+0000 7f32bdc00640 1 -- 172.21.15.16:0/1463529203 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b803e070 con 0x5567c3bfa2f0 2024-08-31T06:17:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.366+0000 7f32bdc00640 1 -- 172.21.15.16:0/1463529203 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32b80316d0 con 0x5567c3bfa2f0 2024-08-31T06:17:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.366+0000 7f32bdc00640 1 -- 172.21.15.16:0/1463529203 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b8035040 con 0x5567c3bfa2f0 2024-08-31T06:17:12.370 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.367+0000 7f32c5600640 1 --2- 172.21.15.16:0/3188235832 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f32ac0b4da0 0x7f32ac0b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:12.370 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.368+0000 7f32c5600640 1 -- 172.21.15.16:0/3188235832 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8514+0+0 (secure 0 0 0) 0x7f32c00e5cb0 con 0x7f32c8108810 2024-08-31T06:17:12.370 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.368+0000 7f32d6453740 1 -- 172.21.15.16:0/1463529203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567c3bfa2f0 msgr2=0x5567c3fcf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:12.370 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.368+0000 7f32d6453740 1 --2- 172.21.15.16:0/1463529203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567c3bfa2f0 0x5567c3fcf5f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f32b8005f30 tx=0x7f32b800b050 comp rx=0 tx=0).stop 2024-08-31T06:17:12.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.368+0000 7f32c7e00640 1 --2- 172.21.15.16:0/3188235832 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f32ac0b4da0 0x7f32ac0b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.369+0000 7f32d6453740 1 -- 172.21.15.16:0/1463529203 shutdown_connections 2024-08-31T06:17:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.369+0000 7f32d6453740 1 --2- 172.21.15.16:0/1463529203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567c3bfa2f0 0x5567c3fcf5f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.369+0000 7f32d6453740 1 --2- 172.21.15.16:0/1463529203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567c3c024d0 0x5567c3fcfb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.369+0000 7f32d6453740 1 --2- 172.21.15.16:0/1463529203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567c3c231b0 0x5567c3fcf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.369+0000 7f32d6453740 1 -- 172.21.15.16:0/1463529203 >> 172.21.15.16:0/1463529203 conn(0x5567c3c17300 msgr2=0x5567c3fccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.370+0000 7f32d6453740 1 -- 172.21.15.16:0/1463529203 shutdown_connections 2024-08-31T06:17:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.371+0000 7f32c7e00640 1 --2- 172.21.15.16:0/3188235832 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f32ac0b4da0 0x7f32ac0b72b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f32c81017b0 tx=0x7f32b4008040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.371+0000 7f32d6453740 1 -- 172.21.15.16:0/1463529203 wait complete. 2024-08-31T06:17:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.372+0000 7f32d6453740 1 -- start start 2024-08-31T06:17:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.372+0000 7f32d6453740 1 Processor -- start 2024-08-31T06:17:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.373+0000 7f32d6453740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567c3bfa2f0 0x5567c3fda6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.373+0000 7f32d6453740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567c3c024d0 0x5567c3fdabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.373+0000 7f32d6453740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567c3c231b0 0x5567c3fdb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.375+0000 7f32cd400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567c3c231b0 0x5567c3fdb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.375+0000 7f32cd400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567c3c231b0 0x5567c3fdb150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:39496/0 (socket says 172.21.15.16:39496) 2024-08-31T06:17:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.375+0000 7f32cd400640 1 -- 172.21.15.16:0/2998275186 learned_addr learned my addr 172.21.15.16:0/2998275186 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.374+0000 7f32cca00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567c3bfa2f0 0x5567c3fda6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:12.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.375+0000 7f32d6453740 1 -- 172.21.15.16:0/2998275186 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5567c3e64ab0 con 0x5567c3c024d0 2024-08-31T06:17:12.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.375+0000 7f32d6453740 1 -- 172.21.15.16:0/2998275186 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5567c3ed2320 con 0x5567c3bfa2f0 2024-08-31T06:17:12.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.375+0000 7f32d6453740 1 -- 172.21.15.16:0/2998275186 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5567c3fdd7d0 con 0x5567c3c231b0 2024-08-31T06:17:12.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.376+0000 7f32cd400640 1 -- 172.21.15.16:0/2998275186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567c3bfa2f0 msgr2=0x5567c3fda6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:12.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.376+0000 7f32cd400640 1 --2- 172.21.15.16:0/2998275186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567c3bfa2f0 0x5567c3fda6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:12.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.376+0000 7f32cd400640 1 -- 172.21.15.16:0/2998275186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567c3c024d0 msgr2=0x5567c3fdabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:12.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.376+0000 7f32cd400640 1 --2- 172.21.15.16:0/2998275186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567c3c024d0 0x5567c3fdabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:12.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.376+0000 7f32cd400640 1 -- 172.21.15.16:0/2998275186 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567c3c8e790 con 0x5567c3c231b0 2024-08-31T06:17:12.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.376+0000 7f32cd400640 1 --2- 172.21.15.16:0/2998275186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567c3c231b0 0x5567c3fdb150 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f32c0005fd0 tx=0x7f32c00acb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c35b7514455c6ca8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:12.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.376+0000 7f32b2000640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:17:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.376+0000 7f32b2000640 1 -- 172.21.15.16:0/2998275186 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c00b5660 con 0x5567c3c231b0 2024-08-31T06:17:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.377+0000 7f32b2000640 1 -- 172.21.15.16:0/2998275186 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8514+0+0 (secure 0 0 0) 0x7f32c00e3620 con 0x5567c3c231b0 2024-08-31T06:17:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.377+0000 7f32b2000640 10 client.0 inactive, discarding osd_map(90..90 src has 1..90) 2024-08-31T06:17:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.377+0000 7f32d6453740 10 client.0 _ll_register_callbacks cb 0x5567c3f0be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:17:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.377+0000 7f32d6453740 1 -- 172.21.15.16:0/2998275186 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32b80316d0 con 0x5567c3c231b0 2024-08-31T06:17:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.381+0000 7f32b2000640 1 -- 172.21.15.16:0/2998275186 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f32c00e3820 con 0x5567c3c231b0 2024-08-31T06:17:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.381+0000 7f32d6453740 10 client.27838 fetch_fsmap learned FSMap version 43 2024-08-31T06:17:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.381+0000 7f32d6453740 1 -- 172.21.15.16:0/2998275186 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({fsmap=43}) -- 0x5567c3d07620 con 0x5567c3c231b0 2024-08-31T06:17:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.382+0000 7f32b2000640 1 -- 172.21.15.16:0/2998275186 <== mon.1 v2:172.21.15.184:3300/0 4 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f32c00b51d0 con 0x5567c3c231b0 2024-08-31T06:17:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.382+0000 7f32d6453740 10 client.27838 fetch_fsmap finished waiting for FSMap version 43 2024-08-31T06:17:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.382+0000 7f32d6453740 10 client.27838 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:17:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.382+0000 7f32d6453740 20 client.27838 populate_metadata read hostname 'smithi016' 2024-08-31T06:17:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.382+0000 7f32d6453740 1 --2- 172.21.15.16:0/2998275186 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5567c3be1d80 0x5567c3fec600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.383+0000 7f32d6453740 4 client.27838 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:17:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.383+0000 7f32d6453740 1 -- 172.21.15.16:0/2998275186 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567c3f3dec0 con 0x5567c3be1d80 2024-08-31T06:17:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.383+0000 7f32cca00640 1 --2- 172.21.15.16:0/2998275186 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5567c3be1d80 0x5567c3fec600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.384+0000 7f32cca00640 1 --2- 172.21.15.16:0/2998275186 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5567c3be1d80 0x5567c3fec600 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9726f5a0a12ff9c server_cookie=f2d63ac8520562af in_seq=0 out_seq=0 2024-08-31T06:17:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.384+0000 7f32b2000640 10 client.27838 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:17:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.387+0000 7f32b2000640 1 -- 172.21.15.16:0/2998275186 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5567c3f3dec0 con 0x5567c3be1d80 2024-08-31T06:17:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.387+0000 7f32b2000640 10 client.27838 handle_command_reply: tid=0 2024-08-31T06:17:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.387+0000 7f32d6453740 1 client.27838 shutdown 2024-08-31T06:17:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.387+0000 7f32d6453740 1 -- 172.21.15.16:0/2998275186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567c3c231b0 msgr2=0x5567c3fdb150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.387+0000 7f32d6453740 1 --2- 172.21.15.16:0/2998275186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567c3c231b0 0x5567c3fdb150 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f32c0005fd0 tx=0x7f32c00acb80 comp rx=0 tx=0).stop 2024-08-31T06:17:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.388+0000 7f32d6453740 1 -- 172.21.15.16:0/2998275186 shutdown_connections 2024-08-31T06:17:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.388+0000 7f32d6453740 1 --2- 172.21.15.16:0/2998275186 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5567c3be1d80 0x5567c3fec600 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.388+0000 7f32d6453740 1 --2- 172.21.15.16:0/2998275186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567c3c231b0 0x5567c3fdb150 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.388+0000 7f32d6453740 1 --2- 172.21.15.16:0/2998275186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567c3c024d0 0x5567c3fdabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.388+0000 7f32d6453740 1 --2- 172.21.15.16:0/2998275186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567c3bfa2f0 0x5567c3fda6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.388+0000 7f32d6453740 1 -- 172.21.15.16:0/2998275186 >> 172.21.15.16:0/2998275186 conn(0x5567c3c17300 msgr2=0x5567c3fd00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.388+0000 7f32b2000640 0 client.27838 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:17:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.388+0000 7f32d6453740 1 -- 172.21.15.16:0/2998275186 shutdown_connections 2024-08-31T06:17:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.388+0000 7f32d6453740 1 -- 172.21.15.16:0/2998275186 wait complete. 2024-08-31T06:17:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.389+0000 7f32d6453740 20 client.27838 trim_cache size 0 max 16384 2024-08-31T06:17:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.420+0000 7f32d6453740 1 Processor -- start 2024-08-31T06:17:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.420+0000 7f32d6453740 1 -- start start 2024-08-31T06:17:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.421+0000 7f32d6453740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567c3bfa2f0 0x5567c3fdc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.421+0000 7f32d6453740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567c3c024d0 0x5567c3fe6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.421+0000 7f32d6453740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567c3c231b0 0x5567c3fe6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.421+0000 7f32d6453740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f32c81b8190 con 0x5567c3bfa2f0 2024-08-31T06:17:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.421+0000 7f32d6453740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f32c81b5a00 con 0x5567c3c024d0 2024-08-31T06:17:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.421+0000 7f32d6453740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f32c81b3870 con 0x5567c3c231b0 2024-08-31T06:17:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.422+0000 7f32c7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567c3c024d0 0x5567c3fe6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.422+0000 7f32c7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567c3c024d0 0x5567c3fe6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:59992/0 (socket says 172.21.15.16:59992) 2024-08-31T06:17:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.423+0000 7f32cd400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567c3c231b0 0x5567c3fe6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.423+0000 7f32cd400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567c3c231b0 0x5567c3fe6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:39512/0 (socket says 172.21.15.16:39512) 2024-08-31T06:17:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.423+0000 7f32cd400640 1 -- 172.21.15.16:0/2986827546 learned_addr learned my addr 172.21.15.16:0/2986827546 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.423+0000 7f32cd400640 1 -- 172.21.15.16:0/2986827546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567c3c024d0 msgr2=0x5567c3fe6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.423+0000 7f32cd400640 1 --2- 172.21.15.16:0/2986827546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567c3c024d0 0x5567c3fe6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.424+0000 7f32cd400640 1 -- 172.21.15.16:0/2986827546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567c3bfa2f0 msgr2=0x5567c3fdc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.424+0000 7f32cd400640 1 --2- 172.21.15.16:0/2986827546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567c3bfa2f0 0x5567c3fdc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.424+0000 7f32cd400640 1 -- 172.21.15.16:0/2986827546 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567c3fd15e0 con 0x5567c3c231b0 2024-08-31T06:17:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.427+0000 7f32cd400640 1 --2- 172.21.15.16:0/2986827546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567c3c231b0 0x5567c3fe6660 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f32c0005820 tx=0x7f32c0011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26873d57577beaf7 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.430+0000 7f32bd200640 1 -- 172.21.15.16:0/2986827546 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c00ab820 con 0x5567c3c231b0 2024-08-31T06:17:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.430+0000 7f32bd200640 1 -- 172.21.15.16:0/2986827546 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32b80316d0 con 0x5567c3c231b0 2024-08-31T06:17:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.430+0000 7f32bd200640 1 -- 172.21.15.16:0/2986827546 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c00a7f10 con 0x5567c3c231b0 2024-08-31T06:17:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.431+0000 7f32d6453740 1 -- 172.21.15.16:0/2986827546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567c3c231b0 msgr2=0x5567c3fe6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.431+0000 7f32d6453740 1 --2- 172.21.15.16:0/2986827546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567c3c231b0 0x5567c3fe6660 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f32c0005820 tx=0x7f32c0011000 comp rx=0 tx=0).stop 2024-08-31T06:17:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.431+0000 7f32d6453740 1 -- 172.21.15.16:0/2986827546 shutdown_connections 2024-08-31T06:17:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.431+0000 7f32d6453740 1 --2- 172.21.15.16:0/2986827546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567c3c231b0 0x5567c3fe6660 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.432+0000 7f32d6453740 1 --2- 172.21.15.16:0/2986827546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567c3c024d0 0x5567c3fe6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.432+0000 7f32d6453740 1 --2- 172.21.15.16:0/2986827546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567c3bfa2f0 0x5567c3fdc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.432+0000 7f32d6453740 1 -- 172.21.15.16:0/2986827546 >> 172.21.15.16:0/2986827546 conn(0x5567c3c17300 msgr2=0x5567c3fe5a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.432+0000 7f32d6453740 1 -- 172.21.15.16:0/2986827546 shutdown_connections 2024-08-31T06:17:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.432+0000 7f32d6453740 1 -- 172.21.15.16:0/2986827546 wait complete. 2024-08-31T06:17:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.433+0000 7f32d6453740 1 -- start start 2024-08-31T06:17:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.434+0000 7f32d6453740 1 Processor -- start 2024-08-31T06:17:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.435+0000 7f32d6453740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567c3bfa2f0 0x5567c3fe1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.435+0000 7f32d6453740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567c3c024d0 0x5567c3fe1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:12.438 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.435+0000 7f32d6453740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567c3c231b0 0x5567c404ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:12.439 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.436+0000 7f32d6453740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f32c81b3ec0 con 0x5567c3c231b0 2024-08-31T06:17:12.439 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.436+0000 7f32d6453740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f32c81b5330 con 0x5567c3bfa2f0 2024-08-31T06:17:12.439 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.436+0000 7f32d6453740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f32c81b4590 con 0x5567c3c024d0 2024-08-31T06:17:12.439 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.436+0000 7f32d6453740 10 client.0 _ll_register_callbacks cb 0x7f32c00a8200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:17:12.439 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.436+0000 7f32cca00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567c3bfa2f0 0x5567c3fe1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:12.439 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.436+0000 7f32cca00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567c3bfa2f0 0x5567c3fe1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:60004/0 (socket says 172.21.15.16:60004) 2024-08-31T06:17:12.439 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.436+0000 7f32cca00640 1 -- 172.21.15.16:0/963252026 learned_addr learned my addr 172.21.15.16:0/963252026 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:12.439 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.436+0000 7f32c7e00640 1 --2- 172.21.15.16:0/963252026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567c3c024d0 0x5567c3fe1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.437+0000 7f32cca00640 1 -- 172.21.15.16:0/963252026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567c3c024d0 msgr2=0x5567c3fe1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.437+0000 7f32cca00640 1 --2- 172.21.15.16:0/963252026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567c3c024d0 0x5567c3fe1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.437+0000 7f32cca00640 1 -- 172.21.15.16:0/963252026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567c3c231b0 msgr2=0x5567c404ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.437+0000 7f32cca00640 1 --2- 172.21.15.16:0/963252026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567c3c231b0 0x5567c404ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.438+0000 7f32cca00640 1 -- 172.21.15.16:0/963252026 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567c4050620 con 0x5567c3bfa2f0 2024-08-31T06:17:12.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.438+0000 7f32cca00640 1 --2- 172.21.15.16:0/963252026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567c3bfa2f0 0x5567c3fe1360 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f32b800b0c0 tx=0x7f32b8004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1598de5977f86a2d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:12.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.438+0000 7f32b2000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:17:12.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.439+0000 7f32b2000640 1 -- 172.21.15.16:0/963252026 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b803e040 con 0x5567c3bfa2f0 2024-08-31T06:17:12.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.439+0000 7f32b2000640 1 -- 172.21.15.16:0/963252026 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8514+0+0 (secure 0 0 0) 0x7f32b8031880 con 0x5567c3bfa2f0 2024-08-31T06:17:12.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.440+0000 7f32d6453740 1 -- 172.21.15.16:0/963252026 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32b80316d0 con 0x5567c3bfa2f0 2024-08-31T06:17:12.446 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.444+0000 7f32b2000640 1 -- 172.21.15.16:0/963252026 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f32b8035040 con 0x5567c3bfa2f0 2024-08-31T06:17:12.446 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.444+0000 7f32d6453740 10 client.27839 fetch_fsmap learned FSMap version 43 2024-08-31T06:17:12.446 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.444+0000 7f32d6453740 1 -- 172.21.15.16:0/963252026 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5567c40519c0 con 0x5567c3bfa2f0 2024-08-31T06:17:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.445+0000 7f32b2000640 1 -- 172.21.15.16:0/963252026 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32b80316d0 con 0x5567c3bfa2f0 2024-08-31T06:17:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.445+0000 7f32b2000640 1 -- 172.21.15.16:0/963252026 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f32b8004bf0 con 0x5567c3bfa2f0 2024-08-31T06:17:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.445+0000 7f32d6453740 10 client.27839 fetch_fsmap finished waiting for FSMap version 43 2024-08-31T06:17:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.445+0000 7f32d6453740 10 client.27839 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:17:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.445+0000 7f32d6453740 20 client.27839 populate_metadata read hostname 'smithi016' 2024-08-31T06:17:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.446+0000 7f32d6453740 1 --2- 172.21.15.16:0/963252026 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5567c3bb71c0 0x5567c3fcbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.446+0000 7f32d6453740 4 client.27839 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:17:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.446+0000 7f32d6453740 1 -- 172.21.15.16:0/963252026 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5567c3f39960 con 0x5567c3bb71c0 2024-08-31T06:17:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.447+0000 7f32c7e00640 1 --2- 172.21.15.16:0/963252026 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5567c3bb71c0 0x5567c3fcbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.447+0000 7f32c7e00640 1 --2- 172.21.15.16:0/963252026 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5567c3bb71c0 0x5567c3fcbfd0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4a233b4175bcab0 server_cookie=47d962eb0276293a in_seq=0 out_seq=0 2024-08-31T06:17:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:12.448+0000 7f32b2000640 10 client.27839 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:17:13.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:12 smithi155 ceph-mon[21742]: pgmap v672: 97 pgs: 97 active+clean; 3.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 1.0 MiB/s wr, 123 op/s 2024-08-31T06:17:13.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:12 smithi016 ceph-mon[17194]: pgmap v672: 97 pgs: 97 active+clean; 3.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 1.0 MiB/s wr, 123 op/s 2024-08-31T06:17:13.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:12 smithi184 ceph-mon[22331]: pgmap v672: 97 pgs: 97 active+clean; 3.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 1.0 MiB/s wr, 123 op/s 2024-08-31T06:17:13.407 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a57ea9ba', total seconds elapsed: 20.5 2024-08-31T06:17:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:13.507+0000 7f32b2000640 1 -- 172.21.15.16:0/963252026 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5567c3f39960 con 0x5567c3bb71c0 2024-08-31T06:17:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:13.507+0000 7f32b2000640 10 client.27839 handle_command_reply: tid=0 2024-08-31T06:17:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:13.507+0000 7f32d6453740 1 client.27839 shutdown 2024-08-31T06:17:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:13.508+0000 7f32d6453740 1 -- 172.21.15.16:0/963252026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567c3bfa2f0 msgr2=0x5567c3fe1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:13.508+0000 7f32d6453740 1 --2- 172.21.15.16:0/963252026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567c3bfa2f0 0x5567c3fe1360 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f32b800b0c0 tx=0x7f32b8004270 comp rx=0 tx=0).stop 2024-08-31T06:17:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:13.508+0000 7f32d6453740 1 -- 172.21.15.16:0/963252026 shutdown_connections 2024-08-31T06:17:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:13.508+0000 7f32d6453740 1 --2- 172.21.15.16:0/963252026 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5567c3bb71c0 0x5567c3fcbfd0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:13.508+0000 7f32d6453740 1 --2- 172.21.15.16:0/963252026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567c3c024d0 0x5567c3fe1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:13.508+0000 7f32d6453740 1 --2- 172.21.15.16:0/963252026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567c3c231b0 0x5567c404ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:13.508+0000 7f32d6453740 1 --2- 172.21.15.16:0/963252026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5567c3bfa2f0 0x5567c3fe1360 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:13.508+0000 7f32d6453740 1 -- 172.21.15.16:0/963252026 >> 172.21.15.16:0/963252026 conn(0x5567c3c17300 msgr2=0x5567c3fe63d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:13.508+0000 7f32b2000640 0 client.27839 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:17:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:13.508+0000 7f32d6453740 1 -- 172.21.15.16:0/963252026 shutdown_connections 2024-08-31T06:17:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:13.509+0000 7f32d6453740 1 -- 172.21.15.16:0/963252026 wait complete. 2024-08-31T06:17:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:13.509+0000 7f32d6453740 20 client.27839 trim_cache size 0 max 16384 2024-08-31T06:17:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:17:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:17:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:17:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:17:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:13.510+0000 7f32cde00640 1 -- 172.21.15.16:0/3188235832 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f32ac0b4da0 msgr2=0x7f32ac0b72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:13.510+0000 7f32cde00640 1 --2- 172.21.15.16:0/3188235832 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f32ac0b4da0 0x7f32ac0b72b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f32c81017b0 tx=0x7f32b4008040 comp rx=0 tx=0).stop 2024-08-31T06:17:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:13.510+0000 7f32cde00640 1 -- 172.21.15.16:0/3188235832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32c8108810 msgr2=0x7f32c806e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:13.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:13.510+0000 7f32cde00640 1 --2- 172.21.15.16:0/3188235832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32c8108810 0x7f32c806e290 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f32c0009fd0 tx=0x7f32c000ef10 comp rx=0 tx=0).stop 2024-08-31T06:17:13.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:13.510+0000 7f32cde00640 1 -- 172.21.15.16:0/3188235832 shutdown_connections 2024-08-31T06:17:13.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:13.510+0000 7f32cde00640 1 --2- 172.21.15.16:0/3188235832 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f32ac0b4da0 0x7f32ac0b72b0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:13.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:13.510+0000 7f32cde00640 1 --2- 172.21.15.16:0/3188235832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32c8107ec0 0x7f32c806dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:13.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:13.510+0000 7f32cde00640 1 --2- 172.21.15.16:0/3188235832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32c8108810 0x7f32c806e290 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:13.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:13.510+0000 7f32cde00640 1 --2- 172.21.15.16:0/3188235832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32c8102590 0x7f32c8074a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:13.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:13.510+0000 7f32cde00640 1 -- 172.21.15.16:0/3188235832 >> 172.21.15.16:0/3188235832 conn(0x7f32c80fdfd0 msgr2=0x7f32c81073f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:13.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:13.511+0000 7f32cde00640 1 -- 172.21.15.16:0/3188235832 shutdown_connections 2024-08-31T06:17:13.514 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:13.511+0000 7f32cde00640 1 -- 172.21.15.16:0/3188235832 wait complete. 2024-08-31T06:17:15.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:14 smithi155 ceph-mon[21742]: pgmap v673: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 4.0 MiB/s wr, 213 op/s 2024-08-31T06:17:15.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:14 smithi016 ceph-mon[17194]: pgmap v673: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 4.0 MiB/s wr, 213 op/s 2024-08-31T06:17:15.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:14 smithi184 ceph-mon[22331]: pgmap v673: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 4.0 MiB/s wr, 213 op/s 2024-08-31T06:17:16.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 590.0 s, 933.3 tps, lat 358.500 ms stddev 1898.408 2024-08-31T06:17:17.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:16 smithi155 ceph-mon[21742]: pgmap v674: 97 pgs: 97 active+clean; 3.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 25 MiB/s wr, 1.10k op/s 2024-08-31T06:17:17.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:16 smithi155 ceph-mon[21742]: osdmap e91: 12 total, 12 up, 12 in 2024-08-31T06:17:17.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:16 smithi016 ceph-mon[17194]: pgmap v674: 97 pgs: 97 active+clean; 3.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 25 MiB/s wr, 1.10k op/s 2024-08-31T06:17:17.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:16 smithi016 ceph-mon[17194]: osdmap e91: 12 total, 12 up, 12 in 2024-08-31T06:17:17.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:16 smithi184 ceph-mon[22331]: pgmap v674: 97 pgs: 97 active+clean; 3.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 25 MiB/s wr, 1.10k op/s 2024-08-31T06:17:17.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:16 smithi184 ceph-mon[22331]: osdmap e91: 12 total, 12 up, 12 in 2024-08-31T06:17:19.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:18 smithi155 ceph-mon[21742]: pgmap v676: 97 pgs: 97 active+clean; 3.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 36 MiB/s wr, 1.56k op/s 2024-08-31T06:17:19.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:18 smithi016 ceph-mon[17194]: pgmap v676: 97 pgs: 97 active+clean; 3.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 36 MiB/s wr, 1.56k op/s 2024-08-31T06:17:19.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:18 smithi184 ceph-mon[22331]: pgmap v676: 97 pgs: 97 active+clean; 3.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 36 MiB/s wr, 1.56k op/s 2024-08-31T06:17:21.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:20 smithi155 ceph-mon[21742]: pgmap v677: 97 pgs: 97 active+clean; 3.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 36 MiB/s wr, 1.60k op/s 2024-08-31T06:17:21.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:20 smithi016 ceph-mon[17194]: pgmap v677: 97 pgs: 97 active+clean; 3.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 36 MiB/s wr, 1.60k op/s 2024-08-31T06:17:21.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:20 smithi184 ceph-mon[22331]: pgmap v677: 97 pgs: 97 active+clean; 3.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 36 MiB/s wr, 1.60k op/s 2024-08-31T06:17:21.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 595.0 s, 1544.8 tps, lat 64.584 ms stddev 77.327 2024-08-31T06:17:23.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:22 smithi155 ceph-mon[21742]: pgmap v678: 97 pgs: 97 active+clean; 3.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 39 MiB/s wr, 1.71k op/s 2024-08-31T06:17:23.165 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:22 smithi016 ceph-mon[17194]: pgmap v678: 97 pgs: 97 active+clean; 3.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 39 MiB/s wr, 1.71k op/s 2024-08-31T06:17:23.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:22 smithi184 ceph-mon[22331]: pgmap v678: 97 pgs: 97 active+clean; 3.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 39 MiB/s wr, 1.71k op/s 2024-08-31T06:17:24.475 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T06:17:24.915 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:24 smithi016 ceph-mon[17194]: pgmap v679: 97 pgs: 97 active+clean; 3.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 36 MiB/s wr, 1.64k op/s 2024-08-31T06:17:25.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:24 smithi155 ceph-mon[21742]: pgmap v679: 97 pgs: 97 active+clean; 3.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 36 MiB/s wr, 1.64k op/s 2024-08-31T06:17:25.159 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:17:25.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:24 smithi184 ceph-mon[22331]: pgmap v679: 97 pgs: 97 active+clean; 3.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 36 MiB/s wr, 1.64k op/s 2024-08-31T06:17:26.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.007+0000 7fdd7be00640 1 -- 172.21.15.16:0/845495349 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd7c10d990 msgr2=0x7fdd7c10dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:26.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.007+0000 7fdd7be00640 1 --2- 172.21.15.16:0/845495349 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd7c10d990 0x7fdd7c10dda0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c009900 tx=0x7fdd6c02a400 comp rx=0 tx=0).stop 2024-08-31T06:17:26.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.009+0000 7fdd7be00640 1 -- 172.21.15.16:0/845495349 shutdown_connections 2024-08-31T06:17:26.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.009+0000 7fdd7be00640 1 --2- 172.21.15.16:0/845495349 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd7c11c940 0x7fdd7c11ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.009+0000 7fdd7be00640 1 --2- 172.21.15.16:0/845495349 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd7c10d990 0x7fdd7c10dda0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.009+0000 7fdd7be00640 1 --2- 172.21.15.16:0/845495349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd7c111a70 0x7fdd7c10d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.009+0000 7fdd7be00640 1 -- 172.21.15.16:0/845495349 >> 172.21.15.16:0/845495349 conn(0x7fdd7c06e790 msgr2=0x7fdd7c06ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:26.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.010+0000 7fdd7be00640 1 -- 172.21.15.16:0/845495349 shutdown_connections 2024-08-31T06:17:26.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.010+0000 7fdd7be00640 1 -- 172.21.15.16:0/845495349 wait complete. 2024-08-31T06:17:26.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.012+0000 7fdd7be00640 1 Processor -- start 2024-08-31T06:17:26.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.013+0000 7fdd7be00640 1 -- start start 2024-08-31T06:17:26.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.013+0000 7fdd7be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd7c10d990 0x7fdd7c1a8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:26.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.013+0000 7fdd7be00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd7c111a70 0x7fdd7c1a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:26.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.013+0000 7fdd7be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd7c11c940 0x7fdd7c1ae770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:26.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.013+0000 7fdd7be00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdd7c122420 con 0x7fdd7c10d990 2024-08-31T06:17:26.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.013+0000 7fdd7be00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdd7c1222a0 con 0x7fdd7c11c940 2024-08-31T06:17:26.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.013+0000 7fdd7be00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdd7c1225a0 con 0x7fdd7c111a70 2024-08-31T06:17:26.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.015+0000 7fdd7b400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd7c11c940 0x7fdd7c1ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:26.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.015+0000 7fdd7b400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd7c11c940 0x7fdd7c1ae770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:58500/0 (socket says 172.21.15.16:58500) 2024-08-31T06:17:26.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.015+0000 7fdd7b400640 1 -- 172.21.15.16:0/2232296034 learned_addr learned my addr 172.21.15.16:0/2232296034 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:26.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.015+0000 7fdd7a000640 1 --2- 172.21.15.16:0/2232296034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd7c111a70 0x7fdd7c1a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:26.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.016+0000 7fdd7b400640 1 -- 172.21.15.16:0/2232296034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd7c111a70 msgr2=0x7fdd7c1a94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:26.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.016+0000 7fdd7b400640 1 --2- 172.21.15.16:0/2232296034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd7c111a70 0x7fdd7c1a94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.016+0000 7fdd7b400640 1 -- 172.21.15.16:0/2232296034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd7c10d990 msgr2=0x7fdd7c1a8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:26.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.016+0000 7fdd7b400640 1 --2- 172.21.15.16:0/2232296034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd7c10d990 0x7fdd7c1a8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.016+0000 7fdd7b400640 1 -- 172.21.15.16:0/2232296034 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd7c1aede0 con 0x7fdd7c11c940 2024-08-31T06:17:26.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.016+0000 7fdd7a000640 1 --2- 172.21.15.16:0/2232296034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd7c111a70 0x7fdd7c1a94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:17:26.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.017+0000 7fdd7b400640 1 --2- 172.21.15.16:0/2232296034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd7c11c940 0x7fdd7c1ae770 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fdd6800d950 tx=0x7fdd6800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:26.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.018+0000 7fdd73400640 1 -- 172.21.15.16:0/2232296034 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd68017030 con 0x7fdd7c11c940 2024-08-31T06:17:26.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.018+0000 7fdd7be00640 1 -- 172.21.15.16:0/2232296034 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd7c1af0d0 con 0x7fdd7c11c940 2024-08-31T06:17:26.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.018+0000 7fdd7be00640 1 -- 172.21.15.16:0/2232296034 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd7c1b58f0 con 0x7fdd7c11c940 2024-08-31T06:17:26.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.018+0000 7fdd73400640 1 -- 172.21.15.16:0/2232296034 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd68004020 con 0x7fdd7c11c940 2024-08-31T06:17:26.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.018+0000 7fdd73400640 1 -- 172.21.15.16:0/2232296034 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd68003a10 con 0x7fdd7c11c940 2024-08-31T06:17:26.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.027+0000 7fdd88a1c740 1 Processor -- start 2024-08-31T06:17:26.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.029+0000 7fdd73400640 1 -- 172.21.15.16:0/2232296034 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fdd6801a070 con 0x7fdd7c11c940 2024-08-31T06:17:26.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.032+0000 7fdd88a1c740 1 -- start start 2024-08-31T06:17:26.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.032+0000 7fdd88a1c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6d6710070 0x55d6d6ab5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:26.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.032+0000 7fdd88a1c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6d66e71b0 0x55d6d6ab5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:26.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.032+0000 7fdd88a1c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d6d66ef390 0x55d6d6ab64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:26.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.032+0000 7fdd88a1c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d6d6a2c770 con 0x55d6d66e71b0 2024-08-31T06:17:26.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.032+0000 7fdd88a1c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d6d6a1bd00 con 0x55d6d66ef390 2024-08-31T06:17:26.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.032+0000 7fdd88a1c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d6d6a1fcb0 con 0x55d6d6710070 2024-08-31T06:17:26.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.033+0000 7fdd73400640 1 --2- 172.21.15.16:0/2232296034 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdd600b4dd0 0x7fdd600b72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:26.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.033+0000 7fdd7a000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6d6710070 0x55d6d6ab5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:26.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.034+0000 7fdd7a000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6d6710070 0x55d6d6ab5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:45538/0 (socket says 172.21.15.16:45538) 2024-08-31T06:17:26.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.034+0000 7fdd7a000640 1 -- 172.21.15.16:0/282292358 learned_addr learned my addr 172.21.15.16:0/282292358 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:26.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.034+0000 7fdd7a000640 1 --2- 172.21.15.16:0/282292358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d6d66ef390 0x55d6d6ab64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:26.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.034+0000 7fdd7aa00640 1 --2- 172.21.15.16:0/282292358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6d66e71b0 0x55d6d6ab5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:26.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.035+0000 7fdd7b400640 1 --2- 172.21.15.16:0/2232296034 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdd600b4dd0 0x7fdd600b72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:26.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.035+0000 7fdd7a000640 1 -- 172.21.15.16:0/282292358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d6d66ef390 msgr2=0x55d6d6ab64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:26.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.035+0000 7fdd7a000640 1 --2- 172.21.15.16:0/282292358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d6d66ef390 0x55d6d6ab64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.035+0000 7fdd7a000640 1 -- 172.21.15.16:0/282292358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6d66e71b0 msgr2=0x55d6d6ab5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:26.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.035+0000 7fdd7a000640 1 --2- 172.21.15.16:0/282292358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6d66e71b0 0x55d6d6ab5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.035+0000 7fdd73400640 1 -- 172.21.15.16:0/2232296034 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8470+0+0 (secure 0 0 0) 0x7fdd680e4ef0 con 0x7fdd7c11c940 2024-08-31T06:17:26.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.035+0000 7fdd7a000640 1 -- 172.21.15.16:0/282292358 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6d677b650 con 0x55d6d6710070 2024-08-31T06:17:26.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.036+0000 7fdd7a000640 1 --2- 172.21.15.16:0/282292358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6d6710070 0x55d6d6ab5a30 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c000c00 tx=0x7fdd6c009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f9f9bad10ce93cf server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:26.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.037+0000 7fdd67e00640 1 -- 172.21.15.16:0/282292358 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c035030 con 0x55d6d6710070 2024-08-31T06:17:26.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.037+0000 7fdd7b400640 1 --2- 172.21.15.16:0/2232296034 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdd600b4dd0 0x7fdd600b72e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fdd680e51d0 tx=0x7fdd680e5140 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:26.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.037+0000 7fdd67e00640 1 -- 172.21.15.16:0/282292358 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd6c004030 con 0x55d6d6710070 2024-08-31T06:17:26.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.038+0000 7fdd67e00640 1 -- 172.21.15.16:0/282292358 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c005540 con 0x55d6d6710070 2024-08-31T06:17:26.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.041+0000 7fdd88a1c740 1 -- 172.21.15.16:0/282292358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6d6710070 msgr2=0x55d6d6ab5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:26.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.041+0000 7fdd88a1c740 1 --2- 172.21.15.16:0/282292358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6d6710070 0x55d6d6ab5a30 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c000c00 tx=0x7fdd6c009560 comp rx=0 tx=0).stop 2024-08-31T06:17:26.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.041+0000 7fdd88a1c740 1 -- 172.21.15.16:0/282292358 shutdown_connections 2024-08-31T06:17:26.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.041+0000 7fdd88a1c740 1 --2- 172.21.15.16:0/282292358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d6d66ef390 0x55d6d6ab64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.042+0000 7fdd88a1c740 1 --2- 172.21.15.16:0/282292358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6d66e71b0 0x55d6d6ab5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.042+0000 7fdd88a1c740 1 --2- 172.21.15.16:0/282292358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6d6710070 0x55d6d6ab5a30 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.042+0000 7fdd88a1c740 1 -- 172.21.15.16:0/282292358 >> 172.21.15.16:0/282292358 conn(0x55d6d6704460 msgr2=0x55d6d6ab3920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:26.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.042+0000 7fdd88a1c740 1 -- 172.21.15.16:0/282292358 shutdown_connections 2024-08-31T06:17:26.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.042+0000 7fdd88a1c740 1 -- 172.21.15.16:0/282292358 wait complete. 2024-08-31T06:17:26.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.044+0000 7fdd88a1c740 1 -- start start 2024-08-31T06:17:26.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.044+0000 7fdd88a1c740 1 Processor -- start 2024-08-31T06:17:26.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.045+0000 7fdd88a1c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d6d66e71b0 0x55d6d6ac16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:26.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.045+0000 7fdd88a1c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6d66ef390 0x55d6d6ac1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:26.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.045+0000 7fdd88a1c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6d6710070 0x55d6d6acd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:26.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.045+0000 7fdd88a1c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d6d687e700 con 0x55d6d66ef390 2024-08-31T06:17:26.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.045+0000 7fdd88a1c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d6d69a2cb0 con 0x55d6d66e71b0 2024-08-31T06:17:26.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.045+0000 7fdd88a1c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d6d6ac4150 con 0x55d6d6710070 2024-08-31T06:17:26.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.045+0000 7fdd88a1c740 10 client.0 _ll_register_callbacks cb 0x55d6d69c8430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:17:26.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.046+0000 7fdd7a000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6d6710070 0x55d6d6acd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:26.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.047+0000 7fdd7a000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d6d66e71b0 0x55d6d6ac16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:26.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.047+0000 7fdd7aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6d66ef390 0x55d6d6ac1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:26.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.047+0000 7fdd7aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6d66ef390 0x55d6d6ac1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34768/0 (socket says 172.21.15.16:34768) 2024-08-31T06:17:26.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.047+0000 7fdd7aa00640 1 -- 172.21.15.16:0/3406239205 learned_addr learned my addr 172.21.15.16:0/3406239205 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:26.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.048+0000 7fdd7aa00640 1 -- 172.21.15.16:0/3406239205 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6d6710070 msgr2=0x55d6d6acd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:26.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.048+0000 7fdd7aa00640 1 --2- 172.21.15.16:0/3406239205 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6d6710070 0x55d6d6acd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.048+0000 7fdd7aa00640 1 -- 172.21.15.16:0/3406239205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d6d66e71b0 msgr2=0x55d6d6ac16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:26.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.048+0000 7fdd7aa00640 1 --2- 172.21.15.16:0/3406239205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d6d66e71b0 0x55d6d6ac16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.048+0000 7fdd7aa00640 1 -- 172.21.15.16:0/3406239205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6d69e3870 con 0x55d6d66ef390 2024-08-31T06:17:26.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.048+0000 7fdd7a000640 1 --2- 172.21.15.16:0/3406239205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d6d66e71b0 0x55d6d6ac16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:17:26.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.049+0000 7fdd7aa00640 1 --2- 172.21.15.16:0/3406239205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6d66ef390 0x55d6d6ac1c20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fdd7400e9f0 tx=0x7fdd7400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ea51c43d969c37 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:26.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.049+0000 7fdd64c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:17:26.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.050+0000 7fdd64c00640 1 -- 172.21.15.16:0/3406239205 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd7400ccc0 con 0x55d6d66ef390 2024-08-31T06:17:26.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.050+0000 7fdd88a1c740 1 -- 172.21.15.16:0/3406239205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd6c004030 con 0x55d6d66ef390 2024-08-31T06:17:26.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.050+0000 7fdd64c00640 1 -- 172.21.15.16:0/3406239205 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8470+0+0 (secure 0 0 0) 0x7fdd74003970 con 0x55d6d66ef390 2024-08-31T06:17:26.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.051+0000 7fdd64c00640 1 -- 172.21.15.16:0/3406239205 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fdd7400ce60 con 0x55d6d66ef390 2024-08-31T06:17:26.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.051+0000 7fdd88a1c740 10 client.17526 fetch_fsmap learned FSMap version 43 2024-08-31T06:17:26.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.051+0000 7fdd88a1c740 1 -- 172.21.15.16:0/3406239205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d6d6ad02d0 con 0x55d6d66ef390 2024-08-31T06:17:26.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.052+0000 7fdd64c00640 1 -- 172.21.15.16:0/3406239205 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd6c004030 con 0x55d6d66ef390 2024-08-31T06:17:26.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.053+0000 7fdd64c00640 1 -- 172.21.15.16:0/3406239205 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fdd74010610 con 0x55d6d66ef390 2024-08-31T06:17:26.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.053+0000 7fdd88a1c740 10 client.17526 fetch_fsmap finished waiting for FSMap version 43 2024-08-31T06:17:26.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.053+0000 7fdd88a1c740 10 client.17526 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:17:26.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.053+0000 7fdd88a1c740 20 client.17526 populate_metadata read hostname 'smithi016' 2024-08-31T06:17:26.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.053+0000 7fdd88a1c740 1 --2- 172.21.15.16:0/3406239205 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d6d66ce9f0 0x55d6d6ad3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:26.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.053+0000 7fdd88a1c740 4 client.17526 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:17:26.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.053+0000 7fdd88a1c740 1 -- 172.21.15.16:0/3406239205 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6d6a2e6c0 con 0x55d6d66ce9f0 2024-08-31T06:17:26.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.054+0000 7fdd7a000640 1 --2- 172.21.15.16:0/3406239205 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d6d66ce9f0 0x55d6d6ad3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:26.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.055+0000 7fdd7a000640 1 --2- 172.21.15.16:0/3406239205 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d6d66ce9f0 0x55d6d6ad3850 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7e942b686947caf server_cookie=c3713ae26fe0044a in_seq=0 out_seq=0 2024-08-31T06:17:26.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.055+0000 7fdd64c00640 10 client.17526 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:17:26.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.056+0000 7fdd64c00640 1 -- 172.21.15.16:0/3406239205 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d6d6a2e6c0 con 0x55d6d66ce9f0 2024-08-31T06:17:26.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.056+0000 7fdd64c00640 10 client.17526 handle_command_reply: tid=0 2024-08-31T06:17:26.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.056+0000 7fdd88a1c740 1 client.17526 shutdown 2024-08-31T06:17:26.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.057+0000 7fdd88a1c740 1 -- 172.21.15.16:0/3406239205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6d66ef390 msgr2=0x55d6d6ac1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:26.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.057+0000 7fdd88a1c740 1 --2- 172.21.15.16:0/3406239205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6d66ef390 0x55d6d6ac1c20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fdd7400e9f0 tx=0x7fdd7400eeb0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.057+0000 7fdd88a1c740 1 -- 172.21.15.16:0/3406239205 shutdown_connections 2024-08-31T06:17:26.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.057+0000 7fdd88a1c740 1 --2- 172.21.15.16:0/3406239205 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d6d66ce9f0 0x55d6d6ad3850 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.058+0000 7fdd64c00640 0 client.17526 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:17:26.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.058+0000 7fdd88a1c740 1 --2- 172.21.15.16:0/3406239205 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6d6710070 0x55d6d6acd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.058+0000 7fdd88a1c740 1 --2- 172.21.15.16:0/3406239205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6d66ef390 0x55d6d6ac1c20 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.058+0000 7fdd88a1c740 1 --2- 172.21.15.16:0/3406239205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d6d66e71b0 0x55d6d6ac16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.058+0000 7fdd88a1c740 1 -- 172.21.15.16:0/3406239205 >> 172.21.15.16:0/3406239205 conn(0x55d6d6704460 msgr2=0x55d6d6ab8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:26.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.058+0000 7fdd88a1c740 1 -- 172.21.15.16:0/3406239205 shutdown_connections 2024-08-31T06:17:26.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.058+0000 7fdd88a1c740 1 -- 172.21.15.16:0/3406239205 wait complete. 2024-08-31T06:17:26.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.058+0000 7fdd88a1c740 20 client.17526 trim_cache size 0 max 16384 2024-08-31T06:17:26.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.092+0000 7fdd88a1c740 1 Processor -- start 2024-08-31T06:17:26.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.093+0000 7fdd88a1c740 1 -- start start 2024-08-31T06:17:26.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.093+0000 7fdd88a1c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d6d66e71b0 0x55d6d6ac1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:26.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.093+0000 7fdd88a1c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6d66ef390 0x55d6d6abab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:26.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.093+0000 7fdd88a1c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6d6710070 0x55d6d6abb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:26.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.093+0000 7fdd88a1c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5c011040 con 0x55d6d66ef390 2024-08-31T06:17:26.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.093+0000 7fdd88a1c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5c00e8b0 con 0x55d6d66e71b0 2024-08-31T06:17:26.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.093+0000 7fdd88a1c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5c00c720 con 0x55d6d6710070 2024-08-31T06:17:26.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.094+0000 7fdd7a000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6d6710070 0x55d6d6abb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:26.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.094+0000 7fdd7a000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6d6710070 0x55d6d6abb080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:45554/0 (socket says 172.21.15.16:45554) 2024-08-31T06:17:26.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.094+0000 7fdd7a000640 1 -- 172.21.15.16:0/1504447308 learned_addr learned my addr 172.21.15.16:0/1504447308 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:26.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.094+0000 7fdd7a000640 1 --2- 172.21.15.16:0/1504447308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d6d66e71b0 0x55d6d6ac1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:26.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.094+0000 7fdd7aa00640 1 --2- 172.21.15.16:0/1504447308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6d66ef390 0x55d6d6abab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:26.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.097+0000 7fdd7a000640 1 -- 172.21.15.16:0/1504447308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d6d66e71b0 msgr2=0x55d6d6ac1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:26.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.097+0000 7fdd7a000640 1 --2- 172.21.15.16:0/1504447308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d6d66e71b0 0x55d6d6ac1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.097+0000 7fdd7a000640 1 -- 172.21.15.16:0/1504447308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6d66ef390 msgr2=0x55d6d6abab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:26.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.097+0000 7fdd7a000640 1 --2- 172.21.15.16:0/1504447308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6d66ef390 0x55d6d6abab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.097+0000 7fdd7a000640 1 -- 172.21.15.16:0/1504447308 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6d6ac0400 con 0x55d6d6710070 2024-08-31T06:17:26.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.098+0000 7fdd7a000640 1 --2- 172.21.15.16:0/1504447308 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6d6710070 0x55d6d6abb080 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c006010 tx=0x7fdd6c005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de28b4628c1641fd server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:26.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.098+0000 7fdd67400640 1 -- 172.21.15.16:0/1504447308 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c035040 con 0x55d6d6710070 2024-08-31T06:17:26.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.098+0000 7fdd67400640 1 -- 172.21.15.16:0/1504447308 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd6c0317d0 con 0x55d6d6710070 2024-08-31T06:17:26.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.099+0000 7fdd67400640 1 -- 172.21.15.16:0/1504447308 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c0038c0 con 0x55d6d6710070 2024-08-31T06:17:26.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.100+0000 7fdd88a1c740 1 -- 172.21.15.16:0/1504447308 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6d6710070 msgr2=0x55d6d6abb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:26.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.100+0000 7fdd88a1c740 1 --2- 172.21.15.16:0/1504447308 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6d6710070 0x55d6d6abb080 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c006010 tx=0x7fdd6c005d50 comp rx=0 tx=0).stop 2024-08-31T06:17:26.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.100+0000 7fdd88a1c740 1 -- 172.21.15.16:0/1504447308 shutdown_connections 2024-08-31T06:17:26.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.101+0000 7fdd88a1c740 1 --2- 172.21.15.16:0/1504447308 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6d6710070 0x55d6d6abb080 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.101+0000 7fdd88a1c740 1 --2- 172.21.15.16:0/1504447308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6d66ef390 0x55d6d6abab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.101+0000 7fdd88a1c740 1 --2- 172.21.15.16:0/1504447308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d6d66e71b0 0x55d6d6ac1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.101+0000 7fdd88a1c740 1 -- 172.21.15.16:0/1504447308 >> 172.21.15.16:0/1504447308 conn(0x55d6d6704460 msgr2=0x55d6d6ac16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:26.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.101+0000 7fdd88a1c740 1 -- 172.21.15.16:0/1504447308 shutdown_connections 2024-08-31T06:17:26.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.101+0000 7fdd88a1c740 1 -- 172.21.15.16:0/1504447308 wait complete. 2024-08-31T06:17:26.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.102+0000 7fdd88a1c740 1 -- start start 2024-08-31T06:17:26.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.102+0000 7fdd88a1c740 1 Processor -- start 2024-08-31T06:17:26.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.104+0000 7fdd88a1c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d6d66e71b0 0x55d6d6ac5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:26.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.104+0000 7fdd88a1c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6d66ef390 0x55d6d6ac6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:26.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.104+0000 7fdd88a1c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6d6710070 0x55d6d6b3d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:26.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.104+0000 7fdd88a1c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5c00cd70 con 0x55d6d6710070 2024-08-31T06:17:26.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.104+0000 7fdd88a1c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5c00e1e0 con 0x55d6d66e71b0 2024-08-31T06:17:26.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.104+0000 7fdd88a1c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5c00d440 con 0x55d6d66ef390 2024-08-31T06:17:26.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.104+0000 7fdd88a1c740 10 client.0 _ll_register_callbacks cb 0x7fdd7400e580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:17:26.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.104+0000 7fdd7a000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d6d66e71b0 0x55d6d6ac5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:26.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.104+0000 7fdd7a000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d6d66e71b0 0x55d6d6ac5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:58532/0 (socket says 172.21.15.16:58532) 2024-08-31T06:17:26.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.104+0000 7fdd7a000640 1 -- 172.21.15.16:0/2381052086 learned_addr learned my addr 172.21.15.16:0/2381052086 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:26.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.104+0000 7fdd7aa00640 1 --2- 172.21.15.16:0/2381052086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6d66ef390 0x55d6d6ac6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:26.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.104+0000 7fdd7a000640 1 --2- 172.21.15.16:0/2381052086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6d6710070 0x55d6d6b3d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:26.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.104+0000 7fdd7aa00640 1 -- 172.21.15.16:0/2381052086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d6d66e71b0 msgr2=0x55d6d6ac5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:26.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.104+0000 7fdd7aa00640 1 --2- 172.21.15.16:0/2381052086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d6d66e71b0 0x55d6d6ac5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.104+0000 7fdd7aa00640 1 -- 172.21.15.16:0/2381052086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6d6710070 msgr2=0x55d6d6b3d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:26.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.104+0000 7fdd7aa00640 1 --2- 172.21.15.16:0/2381052086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6d6710070 0x55d6d6b3d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.104+0000 7fdd7aa00640 1 -- 172.21.15.16:0/2381052086 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6d6b3d880 con 0x55d6d66ef390 2024-08-31T06:17:26.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.105+0000 7fdd7a000640 1 --2- 172.21.15.16:0/2381052086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6d6710070 0x55d6d6b3d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:17:26.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.105+0000 7fdd7aa00640 1 --2- 172.21.15.16:0/2381052086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6d66ef390 0x55d6d6ac6260 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x55d6d69a33f0 tx=0x7fdd74007900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4532ecf5104dbf server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:26.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.105+0000 7fdd64c00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:17:26.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.106+0000 7fdd64c00640 1 -- 172.21.15.16:0/2381052086 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd74004d20 con 0x55d6d66ef390 2024-08-31T06:17:26.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.106+0000 7fdd64c00640 1 -- 172.21.15.16:0/2381052086 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8470+0+0 (secure 0 0 0) 0x7fdd74010430 con 0x55d6d66ef390 2024-08-31T06:17:26.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.107+0000 7fdd88a1c740 1 -- 172.21.15.16:0/2381052086 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd6c0317d0 con 0x55d6d66ef390 2024-08-31T06:17:26.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.107+0000 7fdd64c00640 1 -- 172.21.15.16:0/2381052086 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fdd74010650 con 0x55d6d66ef390 2024-08-31T06:17:26.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.107+0000 7fdd88a1c740 10 client.27859 fetch_fsmap learned FSMap version 43 2024-08-31T06:17:26.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.107+0000 7fdd88a1c740 1 -- 172.21.15.16:0/2381052086 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d6d6b3eb10 con 0x55d6d66ef390 2024-08-31T06:17:26.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.108+0000 7fdd64c00640 1 -- 172.21.15.16:0/2381052086 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd6c0317d0 con 0x55d6d66ef390 2024-08-31T06:17:26.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.108+0000 7fdd64c00640 1 -- 172.21.15.16:0/2381052086 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fdd7402d070 con 0x55d6d66ef390 2024-08-31T06:17:26.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.108+0000 7fdd88a1c740 10 client.27859 fetch_fsmap finished waiting for FSMap version 43 2024-08-31T06:17:26.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.108+0000 7fdd88a1c740 10 client.27859 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:17:26.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.108+0000 7fdd88a1c740 20 client.27859 populate_metadata read hostname 'smithi016' 2024-08-31T06:17:26.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.108+0000 7fdd88a1c740 1 --2- 172.21.15.16:0/2381052086 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d6d66a3da0 0x55d6d6b412a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:26.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.108+0000 7fdd88a1c740 4 client.27859 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T06:17:26.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.108+0000 7fdd88a1c740 1 -- 172.21.15.16:0/2381052086 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d6d6a2a160 con 0x55d6d66a3da0 2024-08-31T06:17:26.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.109+0000 7fdd7a000640 1 --2- 172.21.15.16:0/2381052086 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d6d66a3da0 0x55d6d6b412a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.110+0000 7fdd7a000640 1 --2- 172.21.15.16:0/2381052086 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d6d66a3da0 0x55d6d6b412a0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4824e577bc1b212 server_cookie=cd5c81c4b3287370 in_seq=0 out_seq=0 2024-08-31T06:17:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.111+0000 7fdd64c00640 10 client.27859 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:17:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.111+0000 7fdd64c00640 1 -- 172.21.15.16:0/2381052086 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3782 (crc 0 0 0) 0x55d6d6a2a160 con 0x55d6d66a3da0 2024-08-31T06:17:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.111+0000 7fdd64c00640 10 client.27859 handle_command_reply: tid=0 2024-08-31T06:17:26.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.111+0000 7fdd88a1c740 1 client.27859 shutdown 2024-08-31T06:17:26.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.111+0000 7fdd88a1c740 1 -- 172.21.15.16:0/2381052086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6d66ef390 msgr2=0x55d6d6ac6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:26.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.111+0000 7fdd88a1c740 1 --2- 172.21.15.16:0/2381052086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6d66ef390 0x55d6d6ac6260 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x55d6d69a33f0 tx=0x7fdd74007900 comp rx=0 tx=0).stop 2024-08-31T06:17:26.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.112+0000 7fdd88a1c740 1 -- 172.21.15.16:0/2381052086 shutdown_connections 2024-08-31T06:17:26.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.112+0000 7fdd88a1c740 1 --2- 172.21.15.16:0/2381052086 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d6d66a3da0 0x55d6d6b412a0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.112+0000 7fdd88a1c740 1 --2- 172.21.15.16:0/2381052086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d6d66ef390 0x55d6d6ac6260 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.112+0000 7fdd88a1c740 1 --2- 172.21.15.16:0/2381052086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d6d6710070 0x55d6d6b3d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.112+0000 7fdd88a1c740 1 --2- 172.21.15.16:0/2381052086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d6d66e71b0 0x55d6d6ac5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.112+0000 7fdd88a1c740 1 -- 172.21.15.16:0/2381052086 >> 172.21.15.16:0/2381052086 conn(0x55d6d6704460 msgr2=0x55d6d6abab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:26.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.112+0000 7fdd64c00640 0 client.27859 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:17:26.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.112+0000 7fdd88a1c740 1 -- 172.21.15.16:0/2381052086 shutdown_connections 2024-08-31T06:17:26.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.112+0000 7fdd88a1c740 1 -- 172.21.15.16:0/2381052086 wait complete. 2024-08-31T06:17:26.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.112+0000 7fdd88a1c740 20 client.27859 trim_cache size 0 max 16384 2024-08-31T06:17:26.116 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:17:26.116 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-08-31T06:17:26.116 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {}, 2024-08-31T06:17:26.116 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-08-31T06:17:26.117 INFO:teuthology.orchestra.run.smithi016.stdout: "f9c4a22e-0c8a-4928-9b0d-e2ce1db5a86b": { 2024-08-31T06:17:26.117 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:15:57.496015+0000", 2024-08-31T06:17:26.117 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:17:26.117 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:17:26.117 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:17:26.117 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:17:26.117 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:17:26.117 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:17:26.117 INFO:teuthology.orchestra.run.smithi016.stdout: "a64d5301-8558-4e9c-ac9b-8d0ac7057e74": { 2024-08-31T06:17:26.117 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:14:20.338104+0000", 2024-08-31T06:17:26.117 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:17:26.117 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:17:26.117 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:17:26.117 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:17:26.117 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:17:26.118 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:17:26.118 INFO:teuthology.orchestra.run.smithi016.stdout: "e739a16e-8ace-43a8-a1ce-ac27d39ef8a3": { 2024-08-31T06:17:26.118 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:54.084646+0000", 2024-08-31T06:17:26.118 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:17:26.118 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:17:26.118 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:17:26.118 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:17:26.118 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:17:26.118 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:17:26.118 INFO:teuthology.orchestra.run.smithi016.stdout: "91b72a34-4ba9-4126-a414-8b862f8c5f78": { 2024-08-31T06:17:26.118 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:13:28.663237+0000", 2024-08-31T06:17:26.118 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:17:26.118 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:17:26.118 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:17:26.118 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:17:26.119 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:17:26.119 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:17:26.119 INFO:teuthology.orchestra.run.smithi016.stdout: "a10cce76-99ab-46b5-b3b2-07b82899b8f9": { 2024-08-31T06:17:26.119 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:11:43.967546+0000", 2024-08-31T06:17:26.119 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:17:26.119 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:17:26.119 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:17:26.119 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:17:26.119 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:17:26.119 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:17:26.119 INFO:teuthology.orchestra.run.smithi016.stdout: "c1fd6904-2e02-47fa-939a-47349b9fe36b": { 2024-08-31T06:17:26.119 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:06.678691+0000", 2024-08-31T06:17:26.119 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:17:26.119 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:17:26.119 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:17:26.119 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:17:26.120 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:17:26.120 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:17:26.120 INFO:teuthology.orchestra.run.smithi016.stdout: "78d6ef02-8551-45e8-ad60-9627eb573fc1": { 2024-08-31T06:17:26.120 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:09:13.547865+0000", 2024-08-31T06:17:26.120 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:17:26.120 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:17:26.120 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:17:26.120 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:17:26.120 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:17:26.120 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:17:26.120 INFO:teuthology.orchestra.run.smithi016.stdout: "c8455d74-2fe7-4d93-bcf2-19b419d16016": { 2024-08-31T06:17:26.120 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:08:24.035960+0000", 2024-08-31T06:17:26.121 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:17:26.121 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:17:26.121 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:17:26.121 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:17:26.121 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:17:26.121 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:17:26.121 INFO:teuthology.orchestra.run.smithi016.stdout: "0fdcd762-533d-42be-9289-b71c8e3b81b1": { 2024-08-31T06:17:26.121 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:15:06.169112+0000", 2024-08-31T06:17:26.121 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:17:26.121 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:17:26.121 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:17:26.121 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:17:26.121 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:17:26.121 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:17:26.121 INFO:teuthology.orchestra.run.smithi016.stdout: "88d7bdb8-cb1c-45e5-ad29-4cd8da6c6003": { 2024-08-31T06:17:26.122 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:07:39.648942+0000", 2024-08-31T06:17:26.122 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:17:26.122 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:17:26.122 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:17:26.122 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:17:26.122 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:17:26.122 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:17:26.122 INFO:teuthology.orchestra.run.smithi016.stdout: "004b47bb-0fe8-4d28-9d9d-78bde6a61199": { 2024-08-31T06:17:26.122 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:16:48.915926+0000", 2024-08-31T06:17:26.122 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:17:26.122 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:17:26.122 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:17:26.122 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:17:26.122 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:17:26.122 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:17:26.122 INFO:teuthology.orchestra.run.smithi016.stdout: "6a03046e-cdf4-4a14-b9bd-b560b8968467": { 2024-08-31T06:17:26.123 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:12:38.002354+0000", 2024-08-31T06:17:26.123 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:17:26.123 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:17:26.123 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:17:26.123 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:17:26.123 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:17:26.123 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:17:26.123 INFO:teuthology.orchestra.run.smithi016.stdout: "c300b7e8-f3fd-482c-a909-9329bc61e570": { 2024-08-31T06:17:26.123 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:06:49.868046+0000", 2024-08-31T06:17:26.123 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:17:26.123 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:17:26.123 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:17:26.123 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:17:26.123 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:17:26.123 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:17:26.123 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:17:26.124 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:17:26.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.113+0000 7fdd7be00640 1 -- 172.21.15.16:0/2232296034 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdd600b4dd0 msgr2=0x7fdd600b72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:26.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.113+0000 7fdd7be00640 1 --2- 172.21.15.16:0/2232296034 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdd600b4dd0 0x7fdd600b72e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fdd680e51d0 tx=0x7fdd680e5140 comp rx=0 tx=0).stop 2024-08-31T06:17:26.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.113+0000 7fdd7be00640 1 -- 172.21.15.16:0/2232296034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd7c11c940 msgr2=0x7fdd7c1ae770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:26.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.113+0000 7fdd7be00640 1 --2- 172.21.15.16:0/2232296034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd7c11c940 0x7fdd7c1ae770 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fdd6800d950 tx=0x7fdd6800de10 comp rx=0 tx=0).stop 2024-08-31T06:17:26.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.113+0000 7fdd7be00640 1 -- 172.21.15.16:0/2232296034 shutdown_connections 2024-08-31T06:17:26.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.113+0000 7fdd7be00640 1 --2- 172.21.15.16:0/2232296034 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdd600b4dd0 0x7fdd600b72e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.113+0000 7fdd7be00640 1 --2- 172.21.15.16:0/2232296034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd7c111a70 0x7fdd7c1a94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.113+0000 7fdd7be00640 1 --2- 172.21.15.16:0/2232296034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd7c11c940 0x7fdd7c1ae770 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.113+0000 7fdd7be00640 1 --2- 172.21.15.16:0/2232296034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd7c10d990 0x7fdd7c1a8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.113+0000 7fdd7be00640 1 -- 172.21.15.16:0/2232296034 >> 172.21.15.16:0/2232296034 conn(0x7fdd7c06e790 msgr2=0x7fdd7c110720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:26.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.114+0000 7fdd7be00640 1 -- 172.21.15.16:0/2232296034 shutdown_connections 2024-08-31T06:17:26.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:26.114+0000 7fdd7be00640 1 -- 172.21.15.16:0/2232296034 wait complete. 2024-08-31T06:17:26.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 600.0 s, 1542.0 tps, lat 64.963 ms stddev 78.999 2024-08-31T06:17:26.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.817+0000 7fa54d600640 1 -- 172.21.15.16:0/28047204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa548102c20 msgr2=0x7fa548107140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:26.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.817+0000 7fa54d600640 1 --2- 172.21.15.16:0/28047204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa548102c20 0x7fa548107140 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa534009860 tx=0x7fa53402a340 comp rx=0 tx=0).stop 2024-08-31T06:17:26.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.818+0000 7fa54d600640 1 -- 172.21.15.16:0/28047204 shutdown_connections 2024-08-31T06:17:26.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.818+0000 7fa54d600640 1 --2- 172.21.15.16:0/28047204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa548107790 0x7fa548107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.818+0000 7fa54d600640 1 --2- 172.21.15.16:0/28047204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa548102c20 0x7fa548107140 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.818+0000 7fa54d600640 1 --2- 172.21.15.16:0/28047204 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5481022d0 0x7fa5481026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.819+0000 7fa54d600640 1 -- 172.21.15.16:0/28047204 >> 172.21.15.16:0/28047204 conn(0x7fa5480fdfd0 msgr2=0x7fa548100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:26.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.819+0000 7fa54d600640 1 -- 172.21.15.16:0/28047204 shutdown_connections 2024-08-31T06:17:26.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.819+0000 7fa54d600640 1 -- 172.21.15.16:0/28047204 wait complete. 2024-08-31T06:17:26.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.821+0000 7fa54d600640 1 Processor -- start 2024-08-31T06:17:26.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.821+0000 7fa54d600640 1 -- start start 2024-08-31T06:17:26.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.822+0000 7fa54d600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5481022d0 0x7fa548116840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:26.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.822+0000 7fa54d600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa548102c20 0x7fa548112b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:26.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.822+0000 7fa54d600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa548107790 0x7fa548113100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:26.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.822+0000 7fa54d600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa54810af30 con 0x7fa548107790 2024-08-31T06:17:26.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.822+0000 7fa54d600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa54810adb0 con 0x7fa5481022d0 2024-08-31T06:17:26.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.822+0000 7fa54d600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa54810b0b0 con 0x7fa548102c20 2024-08-31T06:17:26.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.822+0000 7fa547400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa548102c20 0x7fa548112b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:26.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.823+0000 7fa547400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa548102c20 0x7fa548112b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:45574/0 (socket says 172.21.15.16:45574) 2024-08-31T06:17:26.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.823+0000 7fa547400640 1 -- 172.21.15.16:0/1845728916 learned_addr learned my addr 172.21.15.16:0/1845728916 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:26.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.822+0000 7fa547e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5481022d0 0x7fa548116840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:26.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.823+0000 7fa54cc00640 1 --2- 172.21.15.16:0/1845728916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa548107790 0x7fa548113100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:26.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.823+0000 7fa547e00640 1 -- 172.21.15.16:0/1845728916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa548102c20 msgr2=0x7fa548112b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:26.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.823+0000 7fa547e00640 1 --2- 172.21.15.16:0/1845728916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa548102c20 0x7fa548112b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.823+0000 7fa547e00640 1 -- 172.21.15.16:0/1845728916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa548107790 msgr2=0x7fa548113100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:26.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.823+0000 7fa547e00640 1 --2- 172.21.15.16:0/1845728916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa548107790 0x7fa548113100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.823+0000 7fa547e00640 1 -- 172.21.15.16:0/1845728916 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa548113920 con 0x7fa5481022d0 2024-08-31T06:17:26.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.824+0000 7fa547e00640 1 --2- 172.21.15.16:0/1845728916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5481022d0 0x7fa548116840 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa53801f9b0 tx=0x7fa53801fe70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:26.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.825+0000 7fa54cc00640 1 --2- 172.21.15.16:0/1845728916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa548107790 0x7fa548113100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:17:26.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.828+0000 7fa544c00640 1 -- 172.21.15.16:0/1845728916 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53801db90 con 0x7fa5481022d0 2024-08-31T06:17:26.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.828+0000 7fa54d600640 1 -- 172.21.15.16:0/1845728916 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5481a4210 con 0x7fa5481022d0 2024-08-31T06:17:26.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.829+0000 7fa54d600640 1 -- 172.21.15.16:0/1845728916 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5481a4780 con 0x7fa5481022d0 2024-08-31T06:17:26.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.830+0000 7fa544c00640 1 -- 172.21.15.16:0/1845728916 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa53801dd30 con 0x7fa5481022d0 2024-08-31T06:17:26.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.830+0000 7fa544c00640 1 -- 172.21.15.16:0/1845728916 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5380297a0 con 0x7fa5481022d0 2024-08-31T06:17:26.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.833+0000 7fa555c58740 1 Processor -- start 2024-08-31T06:17:26.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.835+0000 7fa555c58740 1 -- start start 2024-08-31T06:17:26.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.835+0000 7fa555c58740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559340a5e1b0 0x559340e018a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:26.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.835+0000 7fa555c58740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559340a352f0 0x559340e01de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:26.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.835+0000 7fa555c58740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559340a3d4d0 0x559340e02320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:26.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.835+0000 7fa555c58740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559340d76f70 con 0x559340a3d4d0 2024-08-31T06:17:26.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.835+0000 7fa555c58740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559340d66500 con 0x559340a5e1b0 2024-08-31T06:17:26.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.835+0000 7fa555c58740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559340d6a4b0 con 0x559340a352f0 2024-08-31T06:17:26.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.836+0000 7fa547400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559340a3d4d0 0x559340e02320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:26.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.836+0000 7fa547400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559340a3d4d0 0x559340e02320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34812/0 (socket says 172.21.15.16:34812) 2024-08-31T06:17:26.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.836+0000 7fa547400640 1 -- 172.21.15.16:0/3448010438 learned_addr learned my addr 172.21.15.16:0/3448010438 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:26.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.837+0000 7fa544c00640 1 -- 172.21.15.16:0/1845728916 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fa538021440 con 0x7fa5481022d0 2024-08-31T06:17:26.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.838+0000 7fa54cc00640 1 --2- 172.21.15.16:0/3448010438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559340a5e1b0 0x559340e018a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:26.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.838+0000 7fa547400640 1 -- 172.21.15.16:0/3448010438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559340a352f0 msgr2=0x559340e01de0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T06:17:26.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.838+0000 7fa547400640 1 --2- 172.21.15.16:0/3448010438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559340a352f0 0x559340e01de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.838+0000 7fa547400640 1 -- 172.21.15.16:0/3448010438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559340a5e1b0 msgr2=0x559340e018a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:26.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.838+0000 7fa547400640 1 --2- 172.21.15.16:0/3448010438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559340a5e1b0 0x559340e018a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.838+0000 7fa547400640 1 -- 172.21.15.16:0/3448010438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559340d20c10 con 0x559340a3d4d0 2024-08-31T06:17:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.840+0000 7fa547400640 1 --2- 172.21.15.16:0/3448010438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559340a3d4d0 0x559340e02320 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa534035040 tx=0x7fa534005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d4f4edcae49599b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.840+0000 7fa544c00640 1 --2- 172.21.15.16:0/1845728916 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa52c0b4da0 0x7fa52c0b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.841+0000 7fa53d200640 1 -- 172.21.15.16:0/3448010438 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa534037040 con 0x559340a3d4d0 2024-08-31T06:17:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.841+0000 7fa53d200640 1 -- 172.21.15.16:0/3448010438 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa534030450 con 0x559340a3d4d0 2024-08-31T06:17:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.841+0000 7fa53d200640 1 -- 172.21.15.16:0/3448010438 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5340039f0 con 0x559340a3d4d0 2024-08-31T06:17:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.841+0000 7fa544c00640 1 -- 172.21.15.16:0/1845728916 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8470+0+0 (secure 0 0 0) 0x7fa5380f66e0 con 0x7fa5481022d0 2024-08-31T06:17:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.842+0000 7fa54cc00640 1 --2- 172.21.15.16:0/1845728916 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa52c0b4da0 0x7fa52c0b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.845+0000 7fa555c58740 1 -- 172.21.15.16:0/3448010438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559340a3d4d0 msgr2=0x559340e02320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.845+0000 7fa555c58740 1 --2- 172.21.15.16:0/3448010438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559340a3d4d0 0x559340e02320 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa534035040 tx=0x7fa534005d30 comp rx=0 tx=0).stop 2024-08-31T06:17:26.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.847+0000 7fa555c58740 1 -- 172.21.15.16:0/3448010438 shutdown_connections 2024-08-31T06:17:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.847+0000 7fa555c58740 1 --2- 172.21.15.16:0/3448010438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559340a352f0 0x559340e01de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.847+0000 7fa555c58740 1 --2- 172.21.15.16:0/3448010438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559340a3d4d0 0x559340e02320 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.847+0000 7fa555c58740 1 --2- 172.21.15.16:0/3448010438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559340a5e1b0 0x559340e018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.847+0000 7fa555c58740 1 -- 172.21.15.16:0/3448010438 >> 172.21.15.16:0/3448010438 conn(0x559340a52300 msgr2=0x559340dff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.848+0000 7fa54cc00640 1 --2- 172.21.15.16:0/1845728916 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa52c0b4da0 0x7fa52c0b72b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa5481030d0 tx=0x7fa540009580 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.849+0000 7fa555c58740 1 -- 172.21.15.16:0/3448010438 shutdown_connections 2024-08-31T06:17:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.849+0000 7fa555c58740 1 -- 172.21.15.16:0/3448010438 wait complete. 2024-08-31T06:17:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.850+0000 7fa555c58740 1 -- start start 2024-08-31T06:17:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.851+0000 7fa555c58740 1 Processor -- start 2024-08-31T06:17:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.854+0000 7fa555c58740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559340a352f0 0x559340e0ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.854+0000 7fa555c58740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559340a3d4d0 0x559340e0d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.856+0000 7fa547e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559340a352f0 0x559340e0ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.856+0000 7fa547e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559340a352f0 0x559340e0ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:45590/0 (socket says 172.21.15.16:45590) 2024-08-31T06:17:26.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.857+0000 7fa555c58740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559340a5e1b0 0x559340e0d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:26.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.858+0000 7fa555c58740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559340c9fab0 con 0x559340a5e1b0 2024-08-31T06:17:26.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.858+0000 7fa555c58740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559340d0d320 con 0x559340a3d4d0 2024-08-31T06:17:26.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.858+0000 7fa555c58740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559340e0ffc0 con 0x559340a352f0 2024-08-31T06:17:26.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.858+0000 7fa555c58740 10 client.0 _ll_register_callbacks cb 0x559340d46e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:17:26.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.859+0000 7fa547e00640 1 -- 172.21.15.16:0/49997437 learned_addr learned my addr 172.21.15.16:0/49997437 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:26.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.857+0000 7fa547400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559340a3d4d0 0x559340e0d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.860+0000 7fa54cc00640 1 --2- 172.21.15.16:0/49997437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559340a5e1b0 0x559340e0d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.861+0000 7fa547400640 1 -- 172.21.15.16:0/49997437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559340a352f0 msgr2=0x559340e0ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.861+0000 7fa547400640 1 --2- 172.21.15.16:0/49997437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559340a352f0 0x559340e0ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.861+0000 7fa547400640 1 -- 172.21.15.16:0/49997437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559340a5e1b0 msgr2=0x559340e0d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.861+0000 7fa547400640 1 --2- 172.21.15.16:0/49997437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559340a5e1b0 0x559340e0d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.861+0000 7fa547400640 1 -- 172.21.15.16:0/49997437 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559340ac9790 con 0x559340a3d4d0 2024-08-31T06:17:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.862+0000 7fa547400640 1 --2- 172.21.15.16:0/49997437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559340a3d4d0 0x559340e0d410 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa534009830 tx=0x7fa534030a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=448b805f33089c8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.862+0000 7fa531600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:17:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.863+0000 7fa531600640 1 -- 172.21.15.16:0/49997437 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa534037030 con 0x559340a3d4d0 2024-08-31T06:17:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.863+0000 7fa531600640 1 -- 172.21.15.16:0/49997437 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8470+0+0 (secure 0 0 0) 0x7fa534004230 con 0x559340a3d4d0 2024-08-31T06:17:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.863+0000 7fa555c58740 1 -- 172.21.15.16:0/49997437 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa534030450 con 0x559340a3d4d0 2024-08-31T06:17:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.863+0000 7fa531600640 1 -- 172.21.15.16:0/49997437 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa534005030 con 0x559340a3d4d0 2024-08-31T06:17:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.863+0000 7fa555c58740 10 client.27869 fetch_fsmap learned FSMap version 43 2024-08-31T06:17:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.863+0000 7fa555c58740 1 -- 172.21.15.16:0/49997437 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x559340b42620 con 0x559340a3d4d0 2024-08-31T06:17:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.864+0000 7fa531600640 1 -- 172.21.15.16:0/49997437 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa534030450 con 0x559340a3d4d0 2024-08-31T06:17:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.864+0000 7fa531600640 1 -- 172.21.15.16:0/49997437 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fa534003ce0 con 0x559340a3d4d0 2024-08-31T06:17:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.864+0000 7fa555c58740 10 client.27869 fetch_fsmap finished waiting for FSMap version 43 2024-08-31T06:17:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.864+0000 7fa555c58740 10 client.27869 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:17:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.864+0000 7fa555c58740 20 client.27869 populate_metadata read hostname 'smithi016' 2024-08-31T06:17:26.868 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.864+0000 7fa555c58740 1 --2- 172.21.15.16:0/49997437 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559340a1cd80 0x559340e1ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:26.868 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.864+0000 7fa555c58740 4 client.27869 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:17:26.868 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.864+0000 7fa555c58740 1 -- 172.21.15.16:0/49997437 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559340d78ec0 con 0x559340a1cd80 2024-08-31T06:17:26.868 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.865+0000 7fa547e00640 1 --2- 172.21.15.16:0/49997437 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559340a1cd80 0x559340e1ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:26.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.866+0000 7fa531600640 10 client.27869 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:17:26.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.868+0000 7fa547e00640 1 --2- 172.21.15.16:0/49997437 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559340a1cd80 0x559340e1ee80 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dfc8cac46ff0ae7 server_cookie=74bb5b981e0901eb in_seq=0 out_seq=0 2024-08-31T06:17:26.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.869+0000 7fa531600640 1 -- 172.21.15.16:0/49997437 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559340d78ec0 con 0x559340a1cd80 2024-08-31T06:17:26.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.869+0000 7fa531600640 10 client.27869 handle_command_reply: tid=0 2024-08-31T06:17:26.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.869+0000 7fa555c58740 1 client.27869 shutdown 2024-08-31T06:17:26.873 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.870+0000 7fa555c58740 1 -- 172.21.15.16:0/49997437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559340a3d4d0 msgr2=0x559340e0d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:26.873 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.870+0000 7fa555c58740 1 --2- 172.21.15.16:0/49997437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559340a3d4d0 0x559340e0d410 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa534009830 tx=0x7fa534030a40 comp rx=0 tx=0).stop 2024-08-31T06:17:26.873 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.870+0000 7fa555c58740 1 -- 172.21.15.16:0/49997437 shutdown_connections 2024-08-31T06:17:26.873 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.870+0000 7fa555c58740 1 --2- 172.21.15.16:0/49997437 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559340a1cd80 0x559340e1ee80 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.873 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.870+0000 7fa555c58740 1 --2- 172.21.15.16:0/49997437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559340a5e1b0 0x559340e0d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.873 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.870+0000 7fa555c58740 1 --2- 172.21.15.16:0/49997437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559340a3d4d0 0x559340e0d410 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.874 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.870+0000 7fa555c58740 1 --2- 172.21.15.16:0/49997437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559340a352f0 0x559340e0ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.874 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.870+0000 7fa555c58740 1 -- 172.21.15.16:0/49997437 >> 172.21.15.16:0/49997437 conn(0x559340a52300 msgr2=0x559340e028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:26.874 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.870+0000 7fa531600640 0 client.27869 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:17:26.874 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.871+0000 7fa555c58740 1 -- 172.21.15.16:0/49997437 shutdown_connections 2024-08-31T06:17:26.874 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.871+0000 7fa555c58740 1 -- 172.21.15.16:0/49997437 wait complete. 2024-08-31T06:17:26.874 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.871+0000 7fa555c58740 20 client.27869 trim_cache size 0 max 16384 2024-08-31T06:17:26.906 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.903+0000 7fa555c58740 1 Processor -- start 2024-08-31T06:17:26.906 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.903+0000 7fa555c58740 1 -- start start 2024-08-31T06:17:26.907 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.904+0000 7fa555c58740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559340a352f0 0x559340e02fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:26.907 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.904+0000 7fa555c58740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559340a3d4d0 0x559340e0a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:26.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.904+0000 7fa555c58740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559340a5e1b0 0x559340e0a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:26.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.904+0000 7fa555c58740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa528011130 con 0x559340a352f0 2024-08-31T06:17:26.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.904+0000 7fa555c58740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa52800e9a0 con 0x559340a3d4d0 2024-08-31T06:17:26.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.904+0000 7fa555c58740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa52800c810 con 0x559340a5e1b0 2024-08-31T06:17:26.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.904+0000 7fa547e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559340a352f0 0x559340e02fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:26.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.904+0000 7fa547400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559340a3d4d0 0x559340e0a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:26.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.904+0000 7fa547e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559340a352f0 0x559340e02fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34832/0 (socket says 172.21.15.16:34832) 2024-08-31T06:17:26.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.904+0000 7fa547e00640 1 -- 172.21.15.16:0/1555514300 learned_addr learned my addr 172.21.15.16:0/1555514300 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:26.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.904+0000 7fa54cc00640 1 --2- 172.21.15.16:0/1555514300 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559340a5e1b0 0x559340e0a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:26.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.905+0000 7fa547e00640 1 -- 172.21.15.16:0/1555514300 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559340a5e1b0 msgr2=0x559340e0a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:26.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.905+0000 7fa547e00640 1 --2- 172.21.15.16:0/1555514300 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559340a5e1b0 0x559340e0a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.905+0000 7fa547e00640 1 -- 172.21.15.16:0/1555514300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559340a3d4d0 msgr2=0x559340e0a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:26.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.905+0000 7fa547e00640 1 --2- 172.21.15.16:0/1555514300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559340a3d4d0 0x559340e0a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.905+0000 7fa547e00640 1 -- 172.21.15.16:0/1555514300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559340e03dd0 con 0x559340a352f0 2024-08-31T06:17:26.909 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.905+0000 7fa547e00640 1 --2- 172.21.15.16:0/1555514300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559340a352f0 0x559340e02fe0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa5380ba010 tx=0x7fa5380bc7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fe7010f8e51f16a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:26.909 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.906+0000 7fa533e00640 1 -- 172.21.15.16:0/1555514300 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5380c0020 con 0x559340a352f0 2024-08-31T06:17:26.910 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.907+0000 7fa533e00640 1 -- 172.21.15.16:0/1555514300 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5380bcda0 con 0x559340a352f0 2024-08-31T06:17:26.910 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.907+0000 7fa533e00640 1 -- 172.21.15.16:0/1555514300 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5380c1220 con 0x559340a352f0 2024-08-31T06:17:26.911 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.908+0000 7fa555c58740 1 -- 172.21.15.16:0/1555514300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559340a352f0 msgr2=0x559340e02fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:26.911 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.908+0000 7fa555c58740 1 --2- 172.21.15.16:0/1555514300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559340a352f0 0x559340e02fe0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa5380ba010 tx=0x7fa5380bc7b0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.911 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.908+0000 7fa555c58740 1 -- 172.21.15.16:0/1555514300 shutdown_connections 2024-08-31T06:17:26.911 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.908+0000 7fa555c58740 1 --2- 172.21.15.16:0/1555514300 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559340a5e1b0 0x559340e0a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.911 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.909+0000 7fa555c58740 1 --2- 172.21.15.16:0/1555514300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559340a3d4d0 0x559340e0a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.911 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.909+0000 7fa555c58740 1 --2- 172.21.15.16:0/1555514300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559340a352f0 0x559340e02fe0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.911 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.909+0000 7fa555c58740 1 -- 172.21.15.16:0/1555514300 >> 172.21.15.16:0/1555514300 conn(0x559340a52300 msgr2=0x559340e028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:26.912 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.909+0000 7fa555c58740 1 -- 172.21.15.16:0/1555514300 shutdown_connections 2024-08-31T06:17:26.912 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.909+0000 7fa555c58740 1 -- 172.21.15.16:0/1555514300 wait complete. 2024-08-31T06:17:26.912 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.909+0000 7fa555c58740 1 -- start start 2024-08-31T06:17:26.912 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.909+0000 7fa555c58740 1 Processor -- start 2024-08-31T06:17:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.910+0000 7fa555c58740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559340a352f0 0x559340e217a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.910+0000 7fa555c58740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559340a3d4d0 0x559340e21d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.910+0000 7fa555c58740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559340a5e1b0 0x559340e87600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.910+0000 7fa555c58740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa52800ce60 con 0x559340a352f0 2024-08-31T06:17:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.910+0000 7fa555c58740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa52800e2d0 con 0x559340a3d4d0 2024-08-31T06:17:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.910+0000 7fa555c58740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa52800d530 con 0x559340a5e1b0 2024-08-31T06:17:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.910+0000 7fa555c58740 10 client.0 _ll_register_callbacks cb 0x7fa5340096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:17:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.910+0000 7fa547e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559340a352f0 0x559340e217a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.910+0000 7fa547e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559340a352f0 0x559340e217a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34842/0 (socket says 172.21.15.16:34842) 2024-08-31T06:17:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.910+0000 7fa547e00640 1 -- 172.21.15.16:0/1114163981 learned_addr learned my addr 172.21.15.16:0/1114163981 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.910+0000 7fa547400640 1 --2- 172.21.15.16:0/1114163981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559340a3d4d0 0x559340e21d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.910+0000 7fa54cc00640 1 --2- 172.21.15.16:0/1114163981 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559340a5e1b0 0x559340e87600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.911+0000 7fa547e00640 1 -- 172.21.15.16:0/1114163981 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559340a5e1b0 msgr2=0x559340e87600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.911+0000 7fa547e00640 1 --2- 172.21.15.16:0/1114163981 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559340a5e1b0 0x559340e87600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.911+0000 7fa547e00640 1 -- 172.21.15.16:0/1114163981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559340a3d4d0 msgr2=0x559340e21d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.911+0000 7fa547e00640 1 --2- 172.21.15.16:0/1114163981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559340a3d4d0 0x559340e21d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.911+0000 7fa547e00640 1 -- 172.21.15.16:0/1114163981 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559340e87ca0 con 0x559340a352f0 2024-08-31T06:17:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.911+0000 7fa547400640 1 --2- 172.21.15.16:0/1114163981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559340a3d4d0 0x559340e21d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:17:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.911+0000 7fa547e00640 1 --2- 172.21.15.16:0/1114163981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559340a352f0 0x559340e217a0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fa5380bc860 tx=0x7fa5380b9260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8240facfe722ffa1 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.911+0000 7fa531600640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:17:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.911+0000 7fa531600640 1 -- 172.21.15.16:0/1114163981 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5380c0070 con 0x559340a352f0 2024-08-31T06:17:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.911+0000 7fa555c58740 1 -- 172.21.15.16:0/1114163981 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5380bcda0 con 0x559340a352f0 2024-08-31T06:17:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.911+0000 7fa531600640 1 -- 172.21.15.16:0/1114163981 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8470+0+0 (secure 0 0 0) 0x7fa5380c1220 con 0x559340a352f0 2024-08-31T06:17:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.912+0000 7fa531600640 1 -- 172.21.15.16:0/1114163981 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa5380bc8b0 con 0x559340a352f0 2024-08-31T06:17:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.912+0000 7fa555c58740 10 client.17547 fetch_fsmap learned FSMap version 43 2024-08-31T06:17:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.912+0000 7fa555c58740 1 -- 172.21.15.16:0/1114163981 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x559340e88f10 con 0x559340a352f0 2024-08-31T06:17:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.914+0000 7fa531600640 1 -- 172.21.15.16:0/1114163981 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5380bcda0 con 0x559340a352f0 2024-08-31T06:17:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.914+0000 7fa531600640 1 -- 172.21.15.16:0/1114163981 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fa5380b8710 con 0x559340a352f0 2024-08-31T06:17:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.914+0000 7fa555c58740 10 client.17547 fetch_fsmap finished waiting for FSMap version 43 2024-08-31T06:17:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.914+0000 7fa555c58740 10 client.17547 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:17:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.914+0000 7fa555c58740 20 client.17547 populate_metadata read hostname 'smithi016' 2024-08-31T06:17:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.914+0000 7fa555c58740 1 --2- 172.21.15.16:0/1114163981 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5593409f21c0 0x559340e8b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.915+0000 7fa555c58740 4 client.17547 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:17:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.915+0000 7fa555c58740 1 -- 172.21.15.16:0/1114163981 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559340d74960 con 0x5593409f21c0 2024-08-31T06:17:26.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.915+0000 7fa547400640 1 --2- 172.21.15.16:0/1114163981 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5593409f21c0 0x559340e8b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:26.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.916+0000 7fa531600640 10 client.17547 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:17:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.916+0000 7fa547400640 1 --2- 172.21.15.16:0/1114163981 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5593409f21c0 0x559340e8b600 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98e5da75513daed8 server_cookie=9801fa24f395a529 in_seq=0 out_seq=0 2024-08-31T06:17:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.941+0000 7fa531600640 1 -- 172.21.15.16:0/1114163981 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559340d74960 con 0x5593409f21c0 2024-08-31T06:17:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.941+0000 7fa531600640 10 client.17547 handle_command_reply: tid=0 2024-08-31T06:17:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.941+0000 7fa555c58740 1 client.17547 shutdown 2024-08-31T06:17:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.941+0000 7fa555c58740 1 -- 172.21.15.16:0/1114163981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559340a352f0 msgr2=0x559340e217a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.941+0000 7fa555c58740 1 --2- 172.21.15.16:0/1114163981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559340a352f0 0x559340e217a0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fa5380bc860 tx=0x7fa5380b9260 comp rx=0 tx=0).stop 2024-08-31T06:17:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.941+0000 7fa555c58740 1 -- 172.21.15.16:0/1114163981 shutdown_connections 2024-08-31T06:17:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.941+0000 7fa555c58740 1 --2- 172.21.15.16:0/1114163981 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5593409f21c0 0x559340e8b600 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.941+0000 7fa555c58740 1 --2- 172.21.15.16:0/1114163981 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559340a5e1b0 0x559340e87600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.941+0000 7fa555c58740 1 --2- 172.21.15.16:0/1114163981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559340a3d4d0 0x559340e21d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.941+0000 7fa555c58740 1 --2- 172.21.15.16:0/1114163981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559340a352f0 0x559340e217a0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.941+0000 7fa555c58740 1 -- 172.21.15.16:0/1114163981 >> 172.21.15.16:0/1114163981 conn(0x559340a52300 msgr2=0x559340dff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.941+0000 7fa531600640 0 client.17547 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:17:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.942+0000 7fa555c58740 1 -- 172.21.15.16:0/1114163981 shutdown_connections 2024-08-31T06:17:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.942+0000 7fa555c58740 1 -- 172.21.15.16:0/1114163981 wait complete. 2024-08-31T06:17:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.942+0000 7fa555c58740 20 client.17547 trim_cache size 0 max 16384 2024-08-31T06:17:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:17:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:17:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:17:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:17:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.943+0000 7fa54d600640 1 -- 172.21.15.16:0/1845728916 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa52c0b4da0 msgr2=0x7fa52c0b72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.943+0000 7fa54d600640 1 --2- 172.21.15.16:0/1845728916 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa52c0b4da0 0x7fa52c0b72b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa5481030d0 tx=0x7fa540009580 comp rx=0 tx=0).stop 2024-08-31T06:17:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.943+0000 7fa54d600640 1 -- 172.21.15.16:0/1845728916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5481022d0 msgr2=0x7fa548116840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.943+0000 7fa54d600640 1 --2- 172.21.15.16:0/1845728916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5481022d0 0x7fa548116840 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa53801f9b0 tx=0x7fa53801fe70 comp rx=0 tx=0).stop 2024-08-31T06:17:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.944+0000 7fa54d600640 1 -- 172.21.15.16:0/1845728916 shutdown_connections 2024-08-31T06:17:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.944+0000 7fa54d600640 1 --2- 172.21.15.16:0/1845728916 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa52c0b4da0 0x7fa52c0b72b0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.944+0000 7fa54d600640 1 --2- 172.21.15.16:0/1845728916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa548102c20 0x7fa548112b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.944+0000 7fa54d600640 1 --2- 172.21.15.16:0/1845728916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa548107790 0x7fa548113100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.944+0000 7fa54d600640 1 --2- 172.21.15.16:0/1845728916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5481022d0 0x7fa548116840 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.944+0000 7fa54d600640 1 -- 172.21.15.16:0/1845728916 >> 172.21.15.16:0/1845728916 conn(0x7fa5480fdfd0 msgr2=0x7fa548105110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.944+0000 7fa54d600640 1 -- 172.21.15.16:0/1845728916 shutdown_connections 2024-08-31T06:17:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:26.944+0000 7fa54d600640 1 -- 172.21.15.16:0/1845728916 wait complete. 2024-08-31T06:17:27.048 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:26 smithi016 ceph-mon[17194]: pgmap v680: 97 pgs: 97 active+clean; 3.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 21 MiB/s wr, 962 op/s 2024-08-31T06:17:27.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:26 smithi155 ceph-mon[21742]: pgmap v680: 97 pgs: 97 active+clean; 3.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 21 MiB/s wr, 962 op/s 2024-08-31T06:17:27.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:26 smithi184 ceph-mon[22331]: pgmap v680: 97 pgs: 97 active+clean; 3.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 21 MiB/s wr, 962 op/s 2024-08-31T06:17:27.683 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T06:17:27.683 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:17:28.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:27 smithi016 ceph-mon[17194]: pgmap v681: 97 pgs: 97 active+clean; 3.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 19 MiB/s wr, 872 op/s 2024-08-31T06:17:28.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:27 smithi184 ceph-mon[22331]: pgmap v681: 97 pgs: 97 active+clean; 3.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 19 MiB/s wr, 872 op/s 2024-08-31T06:17:28.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:27 smithi155 ceph-mon[21742]: pgmap v681: 97 pgs: 97 active+clean; 3.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 19 MiB/s wr, 872 op/s 2024-08-31T06:17:28.401 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:17:29.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:29.739+0000 7fbf09e00640 1 -- 172.21.15.16:0/3167579851 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf041099e0 msgr2=0x7fbf04110210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:29.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:29.739+0000 7fbf09e00640 1 --2- 172.21.15.16:0/3167579851 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf041099e0 0x7fbf04110210 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fbefc009be0 tx=0x7fbefc02a6c0 comp rx=0 tx=0).stop 2024-08-31T06:17:29.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:29.740+0000 7fbf09e00640 1 -- 172.21.15.16:0/3167579851 shutdown_connections 2024-08-31T06:17:29.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:29.740+0000 7fbf09e00640 1 --2- 172.21.15.16:0/3167579851 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf041099e0 0x7fbf04110210 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:29.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:29.740+0000 7fbf09e00640 1 --2- 172.21.15.16:0/3167579851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf04109090 0x7fbf041094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:29.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:29.740+0000 7fbf09e00640 1 --2- 172.21.15.16:0/3167579851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf04103750 0x7fbf04103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:29.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:29.740+0000 7fbf09e00640 1 -- 172.21.15.16:0/3167579851 >> 172.21.15.16:0/3167579851 conn(0x7fbf040ff1a0 msgr2=0x7fbf041015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:29.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:29.740+0000 7fbf09e00640 1 -- 172.21.15.16:0/3167579851 shutdown_connections 2024-08-31T06:17:29.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:29.740+0000 7fbf09e00640 1 -- 172.21.15.16:0/3167579851 wait complete. 2024-08-31T06:17:29.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:29.742+0000 7fbf09e00640 1 Processor -- start 2024-08-31T06:17:29.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:29.742+0000 7fbf09e00640 1 -- start start 2024-08-31T06:17:29.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:29.742+0000 7fbf09e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf04103750 0x7fbf0410d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:29.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:29.742+0000 7fbf09e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf04109090 0x7fbf0410dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:29.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:29.742+0000 7fbf09e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf041099e0 0x7fbf0410e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:29.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:29.742+0000 7fbf09e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbf04076f20 con 0x7fbf041099e0 2024-08-31T06:17:29.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:29.742+0000 7fbf09e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbf04076da0 con 0x7fbf04103750 2024-08-31T06:17:29.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:29.742+0000 7fbf09e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbf040770a0 con 0x7fbf04109090 2024-08-31T06:17:29.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:29.743+0000 7fbf08a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf04103750 0x7fbf0410d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:29.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:29.743+0000 7fbf08a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf04103750 0x7fbf0410d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:58618/0 (socket says 172.21.15.16:58618) 2024-08-31T06:17:29.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:29.743+0000 7fbf08a00640 1 -- 172.21.15.16:0/2871823705 learned_addr learned my addr 172.21.15.16:0/2871823705 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:29.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:29.743+0000 7fbf03e00640 1 --2- 172.21.15.16:0/2871823705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf04109090 0x7fbf0410dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:29.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:29.743+0000 7fbf08a00640 1 -- 172.21.15.16:0/2871823705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf04109090 msgr2=0x7fbf0410dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:29.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:29.743+0000 7fbf08a00640 1 --2- 172.21.15.16:0/2871823705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf04109090 0x7fbf0410dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:29.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:29.743+0000 7fbf08a00640 1 -- 172.21.15.16:0/2871823705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf041099e0 msgr2=0x7fbf0410e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:29.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:29.743+0000 7fbf08a00640 1 --2- 172.21.15.16:0/2871823705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf041099e0 0x7fbf0410e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:29.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:29.743+0000 7fbf08a00640 1 -- 172.21.15.16:0/2871823705 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf041ac6d0 con 0x7fbf04103750 2024-08-31T06:17:29.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:29.745+0000 7fbf08a00640 1 --2- 172.21.15.16:0/2871823705 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf04103750 0x7fbf0410d770 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fbef800e9c0 tx=0x7fbef800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:29.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:29.746+0000 7fbf01600640 1 -- 172.21.15.16:0/2871823705 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef800cde0 con 0x7fbf04103750 2024-08-31T06:17:29.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:29.746+0000 7fbf01600640 1 -- 172.21.15.16:0/2871823705 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbef80035f0 con 0x7fbf04103750 2024-08-31T06:17:29.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:29.746+0000 7fbf01600640 1 -- 172.21.15.16:0/2871823705 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef8010690 con 0x7fbf04103750 2024-08-31T06:17:29.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:29.746+0000 7fbf09e00640 1 -- 172.21.15.16:0/2871823705 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf041ac9c0 con 0x7fbf04103750 2024-08-31T06:17:29.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:29.747+0000 7fbf09e00640 1 -- 172.21.15.16:0/2871823705 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf041aceb0 con 0x7fbf04103750 2024-08-31T06:17:29.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:29.748+0000 7fbf09e00640 1 -- 172.21.15.16:0/2871823705 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf04103b60 con 0x7fbf04103750 2024-08-31T06:17:29.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:29.753+0000 7fbf01600640 1 -- 172.21.15.16:0/2871823705 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fbef8004020 con 0x7fbf04103750 2024-08-31T06:17:29.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:29.755+0000 7fbf01600640 1 --2- 172.21.15.16:0/2871823705 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbeec0b4d20 0x7fbeec0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:29.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:29.756+0000 7fbf01600640 1 -- 172.21.15.16:0/2871823705 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8470+0+0 (secure 0 0 0) 0x7fbef80e58e0 con 0x7fbf04103750 2024-08-31T06:17:29.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:29.757+0000 7fbf03e00640 1 --2- 172.21.15.16:0/2871823705 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbeec0b4d20 0x7fbeec0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:29.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:29.758+0000 7fbf03e00640 1 --2- 172.21.15.16:0/2871823705 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbeec0b4d20 0x7fbeec0b7230 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fbf041029f0 tx=0x7fbef0005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:29.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:29.759+0000 7fbf01600640 1 -- 172.21.15.16:0/2871823705 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fbef8009510 con 0x7fbf04103750 2024-08-31T06:17:30.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:30.172+0000 7fbf09e00640 1 -- 172.21.15.16:0/2871823705 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf0410ea70 con 0x7fbf04103750 2024-08-31T06:17:30.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:30.173+0000 7fbf01600640 1 -- 172.21.15.16:0/2871823705 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11920 (secure 0 0 0) 0x7fbef80ac700 con 0x7fbf04103750 2024-08-31T06:17:30.176 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:17:30.176 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":43,"btime":"2024-08-31T06:16:03:711285+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:16:03.340730+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24737,24724,24712]},"id":1}]} 2024-08-31T06:17:30.176 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 43 2024-08-31T06:17:30.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:30.178+0000 7fbf09e00640 1 -- 172.21.15.16:0/2871823705 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbeec0b4d20 msgr2=0x7fbeec0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:30.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:30.178+0000 7fbf09e00640 1 --2- 172.21.15.16:0/2871823705 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbeec0b4d20 0x7fbeec0b7230 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fbf041029f0 tx=0x7fbef0005f90 comp rx=0 tx=0).stop 2024-08-31T06:17:30.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:30.178+0000 7fbf09e00640 1 -- 172.21.15.16:0/2871823705 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf04103750 msgr2=0x7fbf0410d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:30.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:30.178+0000 7fbf09e00640 1 --2- 172.21.15.16:0/2871823705 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf04103750 0x7fbf0410d770 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fbef800e9c0 tx=0x7fbef800ee80 comp rx=0 tx=0).stop 2024-08-31T06:17:30.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:30.178+0000 7fbf09e00640 1 -- 172.21.15.16:0/2871823705 shutdown_connections 2024-08-31T06:17:30.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:30.178+0000 7fbf09e00640 1 --2- 172.21.15.16:0/2871823705 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbeec0b4d20 0x7fbeec0b7230 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:30.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:30.178+0000 7fbf09e00640 1 --2- 172.21.15.16:0/2871823705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf04109090 0x7fbf0410dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:30.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:30.178+0000 7fbf09e00640 1 --2- 172.21.15.16:0/2871823705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf041099e0 0x7fbf0410e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:30.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:30.178+0000 7fbf09e00640 1 --2- 172.21.15.16:0/2871823705 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf04103750 0x7fbf0410d770 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:30.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:30.178+0000 7fbf09e00640 1 -- 172.21.15.16:0/2871823705 >> 172.21.15.16:0/2871823705 conn(0x7fbf040ff1a0 msgr2=0x7fbf041086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:30.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:30.178+0000 7fbf09e00640 1 -- 172.21.15.16:0/2871823705 shutdown_connections 2024-08-31T06:17:30.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:30.178+0000 7fbf09e00640 1 -- 172.21.15.16:0/2871823705 wait complete. 2024-08-31T06:17:30.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:30 smithi155 ceph-mon[21742]: pgmap v682: 97 pgs: 97 active+clean; 3.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 20 MiB/s wr, 1.09k op/s 2024-08-31T06:17:30.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:30 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/2871823705' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:17:30.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:30 smithi016 ceph-mon[17194]: pgmap v682: 97 pgs: 97 active+clean; 3.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 20 MiB/s wr, 1.09k op/s 2024-08-31T06:17:30.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:30 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/2871823705' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:17:30.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:30 smithi184 ceph-mon[22331]: pgmap v682: 97 pgs: 97 active+clean; 3.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 20 MiB/s wr, 1.09k op/s 2024-08-31T06:17:30.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:30 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/2871823705' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:17:31.343 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T06:17:31.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 605.0 s, 1504.6 tps, lat 66.579 ms stddev 97.244 2024-08-31T06:17:31.985 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:17:32.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:32 smithi016 ceph-mon[17194]: pgmap v683: 97 pgs: 97 active+clean; 3.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 20 MiB/s wr, 1.20k op/s 2024-08-31T06:17:32.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:32 smithi184 ceph-mon[22331]: pgmap v683: 97 pgs: 97 active+clean; 3.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 20 MiB/s wr, 1.20k op/s 2024-08-31T06:17:33.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:32 smithi155 ceph-mon[21742]: pgmap v683: 97 pgs: 97 active+clean; 3.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 20 MiB/s wr, 1.20k op/s 2024-08-31T06:17:33.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.245+0000 7fc35e000640 1 -- 172.21.15.16:0/2001765697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3580776e0 msgr2=0x7fc358077af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:33.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.245+0000 7fc35e000640 1 --2- 172.21.15.16:0/2001765697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3580776e0 0x7fc358077af0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc348009ca0 tx=0x7fc34802a8c0 comp rx=0 tx=0).stop 2024-08-31T06:17:33.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.245+0000 7fc35e000640 1 -- 172.21.15.16:0/2001765697 shutdown_connections 2024-08-31T06:17:33.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.245+0000 7fc35e000640 1 --2- 172.21.15.16:0/2001765697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc358078030 0x7fc358113de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:33.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.245+0000 7fc35e000640 1 --2- 172.21.15.16:0/2001765697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc358114320 0x7fc358116740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:33.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.245+0000 7fc35e000640 1 --2- 172.21.15.16:0/2001765697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3580776e0 0x7fc358077af0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:33.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.245+0000 7fc35e000640 1 -- 172.21.15.16:0/2001765697 >> 172.21.15.16:0/2001765697 conn(0x7fc3581013d0 msgr2=0x7fc358103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:33.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.245+0000 7fc35e000640 1 -- 172.21.15.16:0/2001765697 shutdown_connections 2024-08-31T06:17:33.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.245+0000 7fc35e000640 1 -- 172.21.15.16:0/2001765697 wait complete. 2024-08-31T06:17:33.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.248+0000 7fc35e000640 1 Processor -- start 2024-08-31T06:17:33.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.248+0000 7fc35e000640 1 -- start start 2024-08-31T06:17:33.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.249+0000 7fc35e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3580776e0 0x7fc3581a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:33.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.249+0000 7fc35e000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc358078030 0x7fc3581a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:33.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.249+0000 7fc357e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc358078030 0x7fc3581a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:33.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.249+0000 7fc357e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc358078030 0x7fc3581a0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:45650/0 (socket says 172.21.15.16:45650) 2024-08-31T06:17:33.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.249+0000 7fc35e000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc358114320 0x7fc3581a5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:33.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.249+0000 7fc35e000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc358119cd0 con 0x7fc3580776e0 2024-08-31T06:17:33.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.249+0000 7fc35e000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc358119b50 con 0x7fc358114320 2024-08-31T06:17:33.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.249+0000 7fc35e000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc358119e50 con 0x7fc358078030 2024-08-31T06:17:33.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.249+0000 7fc357e00640 1 -- 172.21.15.16:0/277007598 learned_addr learned my addr 172.21.15.16:0/277007598 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:33.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.250+0000 7fc35cc00640 1 --2- 172.21.15.16:0/277007598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3580776e0 0x7fc3581a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:33.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.250+0000 7fc357e00640 1 -- 172.21.15.16:0/277007598 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc358114320 msgr2=0x7fc3581a5ee0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-31T06:17:33.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.250+0000 7fc357e00640 1 --2- 172.21.15.16:0/277007598 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc358114320 0x7fc3581a5ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:33.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.250+0000 7fc357e00640 1 -- 172.21.15.16:0/277007598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3580776e0 msgr2=0x7fc3581a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:33.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.250+0000 7fc357e00640 1 --2- 172.21.15.16:0/277007598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3580776e0 0x7fc3581a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:33.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.250+0000 7fc357e00640 1 -- 172.21.15.16:0/277007598 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3581a6610 con 0x7fc358078030 2024-08-31T06:17:33.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.250+0000 7fc35cc00640 1 --2- 172.21.15.16:0/277007598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3580776e0 0x7fc3581a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:17:33.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.250+0000 7fc357e00640 1 --2- 172.21.15.16:0/277007598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc358078030 0x7fc3581a0c40 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc34000d950 tx=0x7fc34000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:33.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.251+0000 7fc355600640 1 -- 172.21.15.16:0/277007598 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc340017070 con 0x7fc358078030 2024-08-31T06:17:33.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.251+0000 7fc355600640 1 -- 172.21.15.16:0/277007598 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc340009c30 con 0x7fc358078030 2024-08-31T06:17:33.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.251+0000 7fc355600640 1 -- 172.21.15.16:0/277007598 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc340003d00 con 0x7fc358078030 2024-08-31T06:17:33.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.251+0000 7fc35e000640 1 -- 172.21.15.16:0/277007598 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3581a6900 con 0x7fc358078030 2024-08-31T06:17:33.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.252+0000 7fc35e000640 1 -- 172.21.15.16:0/277007598 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3581ad060 con 0x7fc358078030 2024-08-31T06:17:33.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.257+0000 7fc36640d740 1 Processor -- start 2024-08-31T06:17:33.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.257+0000 7fc36640d740 1 -- start start 2024-08-31T06:17:33.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.258+0000 7fc36640d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560868ad7070 0x560868e7cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:33.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.258+0000 7fc36640d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560868aae1b0 0x560868e7d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:33.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.258+0000 7fc36640d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560868ab6390 0x560868e7d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:33.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.258+0000 7fc36640d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560868df37b0 con 0x560868aae1b0 2024-08-31T06:17:33.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.258+0000 7fc36640d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560868de2d40 con 0x560868ad7070 2024-08-31T06:17:33.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.258+0000 7fc36640d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560868de6cf0 con 0x560868ab6390 2024-08-31T06:17:33.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.259+0000 7fc35d600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560868ad7070 0x560868e7cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:33.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.259+0000 7fc35d600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560868ad7070 0x560868e7cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:58648/0 (socket says 172.21.15.16:58648) 2024-08-31T06:17:33.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.259+0000 7fc35d600640 1 -- 172.21.15.16:0/3095580649 learned_addr learned my addr 172.21.15.16:0/3095580649 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:33.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.259+0000 7fc355600640 1 -- 172.21.15.16:0/277007598 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fc340003ea0 con 0x7fc358078030 2024-08-31T06:17:33.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.260+0000 7fc357e00640 1 --2- 172.21.15.16:0/3095580649 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560868ab6390 0x560868e7d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:33.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.260+0000 7fc35cc00640 1 --2- 172.21.15.16:0/3095580649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560868aae1b0 0x560868e7d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:33.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.260+0000 7fc35d600640 1 -- 172.21.15.16:0/3095580649 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560868ab6390 msgr2=0x560868e7d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:33.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.260+0000 7fc35d600640 1 --2- 172.21.15.16:0/3095580649 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560868ab6390 0x560868e7d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:33.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.260+0000 7fc35d600640 1 -- 172.21.15.16:0/3095580649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560868aae1b0 msgr2=0x560868e7d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:33.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.260+0000 7fc35d600640 1 --2- 172.21.15.16:0/3095580649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560868aae1b0 0x560868e7d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:33.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.260+0000 7fc35d600640 1 -- 172.21.15.16:0/3095580649 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560868b42650 con 0x560868ad7070 2024-08-31T06:17:33.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.261+0000 7fc35cc00640 1 --2- 172.21.15.16:0/3095580649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560868aae1b0 0x560868e7d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:17:33.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.261+0000 7fc35d600640 1 --2- 172.21.15.16:0/3095580649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560868ad7070 0x560868e7cb60 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc35000b980 tx=0x7fc35000be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf4ec2e29a81500b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:33.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.262+0000 7fc34dc00640 1 -- 172.21.15.16:0/3095580649 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc350016070 con 0x560868ad7070 2024-08-31T06:17:33.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.262+0000 7fc34dc00640 1 -- 172.21.15.16:0/3095580649 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3500054d0 con 0x560868ad7070 2024-08-31T06:17:33.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.262+0000 7fc34dc00640 1 -- 172.21.15.16:0/3095580649 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc350004c60 con 0x560868ad7070 2024-08-31T06:17:33.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.262+0000 7fc355600640 1 --2- 172.21.15.16:0/277007598 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc33c0b4d20 0x7fc33c0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:33.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.263+0000 7fc355600640 1 -- 172.21.15.16:0/277007598 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8470+0+0 (secure 0 0 0) 0x7fc3400e52b0 con 0x7fc358078030 2024-08-31T06:17:33.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.263+0000 7fc35cc00640 1 --2- 172.21.15.16:0/277007598 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc33c0b4d20 0x7fc33c0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:33.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.263+0000 7fc36640d740 1 -- 172.21.15.16:0/3095580649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560868ad7070 msgr2=0x560868e7cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:33.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.263+0000 7fc36640d740 1 --2- 172.21.15.16:0/3095580649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560868ad7070 0x560868e7cb60 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc35000b980 tx=0x7fc35000be40 comp rx=0 tx=0).stop 2024-08-31T06:17:33.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.263+0000 7fc36640d740 1 -- 172.21.15.16:0/3095580649 shutdown_connections 2024-08-31T06:17:33.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.263+0000 7fc36640d740 1 --2- 172.21.15.16:0/3095580649 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560868ab6390 0x560868e7d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:33.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.263+0000 7fc36640d740 1 --2- 172.21.15.16:0/3095580649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560868aae1b0 0x560868e7d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:33.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.263+0000 7fc36640d740 1 --2- 172.21.15.16:0/3095580649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560868ad7070 0x560868e7cb60 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:33.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.263+0000 7fc36640d740 1 -- 172.21.15.16:0/3095580649 >> 172.21.15.16:0/3095580649 conn(0x560868acb460 msgr2=0x560868e7aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:33.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.264+0000 7fc36640d740 1 -- 172.21.15.16:0/3095580649 shutdown_connections 2024-08-31T06:17:33.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.265+0000 7fc36640d740 1 -- 172.21.15.16:0/3095580649 wait complete. 2024-08-31T06:17:33.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.265+0000 7fc35cc00640 1 --2- 172.21.15.16:0/277007598 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc33c0b4d20 0x7fc33c0b7230 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc348004af0 tx=0x7fc348005fd0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:33.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.266+0000 7fc36640d740 1 -- start start 2024-08-31T06:17:33.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.266+0000 7fc36640d740 1 Processor -- start 2024-08-31T06:17:33.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.267+0000 7fc36640d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560868aae1b0 0x560868e887b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:33.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.267+0000 7fc36640d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560868ab6390 0x560868e88d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:33.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.267+0000 7fc36640d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560868ad7070 0x560868e94680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:33.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.267+0000 7fc36640d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560868c45700 con 0x560868ab6390 2024-08-31T06:17:33.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.267+0000 7fc36640d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560868d69cf0 con 0x560868ad7070 2024-08-31T06:17:33.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.267+0000 7fc36640d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560868e8b340 con 0x560868aae1b0 2024-08-31T06:17:33.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.267+0000 7fc36640d740 10 client.0 _ll_register_callbacks cb 0x560868d8f470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:17:33.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.268+0000 7fc35cc00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560868aae1b0 0x560868e887b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:33.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.268+0000 7fc35cc00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560868aae1b0 0x560868e887b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:45670/0 (socket says 172.21.15.16:45670) 2024-08-31T06:17:33.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.268+0000 7fc35cc00640 1 -- 172.21.15.16:0/2436799375 learned_addr learned my addr 172.21.15.16:0/2436799375 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:33.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.269+0000 7fc35cc00640 1 -- 172.21.15.16:0/2436799375 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560868ad7070 msgr2=0x560868e94680 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T06:17:33.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.269+0000 7fc35cc00640 1 --2- 172.21.15.16:0/2436799375 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560868ad7070 0x560868e94680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:33.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.269+0000 7fc35cc00640 1 -- 172.21.15.16:0/2436799375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560868ab6390 msgr2=0x560868e88d30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:17:33.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.269+0000 7fc357e00640 1 --2- 172.21.15.16:0/2436799375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560868ab6390 0x560868e88d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:33.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.269+0000 7fc35cc00640 1 --2- 172.21.15.16:0/2436799375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560868ab6390 0x560868e88d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:33.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.269+0000 7fc35cc00640 1 -- 172.21.15.16:0/2436799375 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560868daa8b0 con 0x560868aae1b0 2024-08-31T06:17:33.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.270+0000 7fc35cc00640 1 --2- 172.21.15.16:0/2436799375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560868aae1b0 0x560868e887b0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc348009ca0 tx=0x7fc34802a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb354ff7c5588771 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:33.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.270+0000 7fc357e00640 1 --2- 172.21.15.16:0/2436799375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560868ab6390 0x560868e88d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:17:33.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.270+0000 7fc346000640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:17:33.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.271+0000 7fc346000640 1 -- 172.21.15.16:0/2436799375 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc348031920 con 0x560868aae1b0 2024-08-31T06:17:33.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.271+0000 7fc346000640 1 -- 172.21.15.16:0/2436799375 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8470+0+0 (secure 0 0 0) 0x7fc348004450 con 0x560868aae1b0 2024-08-31T06:17:33.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.271+0000 7fc36640d740 1 -- 172.21.15.16:0/2436799375 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3500054d0 con 0x560868aae1b0 2024-08-31T06:17:33.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.272+0000 7fc346000640 1 -- 172.21.15.16:0/2436799375 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc348004650 con 0x560868aae1b0 2024-08-31T06:17:33.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.272+0000 7fc36640d740 10 client.27892 fetch_fsmap learned FSMap version 44 2024-08-31T06:17:33.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.272+0000 7fc36640d740 1 -- 172.21.15.16:0/2436799375 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x560868e973b0 con 0x560868aae1b0 2024-08-31T06:17:33.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.272+0000 7fc346000640 1 -- 172.21.15.16:0/2436799375 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3500054d0 con 0x560868aae1b0 2024-08-31T06:17:33.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.273+0000 7fc346000640 1 -- 172.21.15.16:0/2436799375 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 44) ==== 8350+0+0 (secure 0 0 0) 0x7fc348030600 con 0x560868aae1b0 2024-08-31T06:17:33.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.274+0000 7fc36640d740 10 client.27892 fetch_fsmap finished waiting for FSMap version 44 2024-08-31T06:17:33.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.274+0000 7fc36640d740 10 client.27892 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T06:17:33.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.274+0000 7fc36640d740 20 client.27892 populate_metadata read hostname 'smithi016' 2024-08-31T06:17:33.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.274+0000 7fc36640d740 1 --2- 172.21.15.16:0/2436799375 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x560868a959f0 0x560868e9a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:33.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.274+0000 7fc36640d740 4 client.27892 mds_command: new command op to 24712 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:17:33.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.274+0000 7fc36640d740 1 -- 172.21.15.16:0/2436799375 --> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560868df5700 con 0x560868a959f0 2024-08-31T06:17:33.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.274+0000 7fc357e00640 1 --2- 172.21.15.16:0/2436799375 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x560868a959f0 0x560868e9a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:33.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.275+0000 7fc357e00640 1 --2- 172.21.15.16:0/2436799375 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x560868a959f0 0x560868e9a950 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=af353dcae40601f server_cookie=b6752d66b41d9d1f in_seq=0 out_seq=0 2024-08-31T06:17:33.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.275+0000 7fc346000640 10 client.27892 ms_handle_connect on v2:172.21.15.184:6838/4011051479 2024-08-31T06:17:33.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.276+0000 7fc346000640 1 -- 172.21.15.16:0/2436799375 <== mds.2 v2:172.21.15.184:6838/4011051479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560868df5700 con 0x560868a959f0 2024-08-31T06:17:33.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.276+0000 7fc346000640 10 client.27892 handle_command_reply: tid=0 2024-08-31T06:17:33.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.276+0000 7fc36640d740 1 client.27892 shutdown 2024-08-31T06:17:33.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.277+0000 7fc36640d740 1 -- 172.21.15.16:0/2436799375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560868aae1b0 msgr2=0x560868e887b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:33.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.277+0000 7fc36640d740 1 --2- 172.21.15.16:0/2436799375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560868aae1b0 0x560868e887b0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc348009ca0 tx=0x7fc34802a8c0 comp rx=0 tx=0).stop 2024-08-31T06:17:33.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.277+0000 7fc36640d740 1 -- 172.21.15.16:0/2436799375 shutdown_connections 2024-08-31T06:17:33.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.277+0000 7fc36640d740 1 --2- 172.21.15.16:0/2436799375 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x560868a959f0 0x560868e9a950 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:33.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.277+0000 7fc36640d740 1 --2- 172.21.15.16:0/2436799375 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560868ad7070 0x560868e94680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:33.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.277+0000 7fc36640d740 1 --2- 172.21.15.16:0/2436799375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560868ab6390 0x560868e88d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:33.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.277+0000 7fc36640d740 1 --2- 172.21.15.16:0/2436799375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560868aae1b0 0x560868e887b0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:33.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.277+0000 7fc36640d740 1 -- 172.21.15.16:0/2436799375 >> 172.21.15.16:0/2436799375 conn(0x560868acb460 msgr2=0x560868e7fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:33.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.278+0000 7fc346000640 0 client.27892 ms_handle_reset on v2:172.21.15.184:6838/4011051479 2024-08-31T06:17:33.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.278+0000 7fc36640d740 1 -- 172.21.15.16:0/2436799375 shutdown_connections 2024-08-31T06:17:33.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.278+0000 7fc36640d740 1 -- 172.21.15.16:0/2436799375 wait complete. 2024-08-31T06:17:33.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.278+0000 7fc36640d740 20 client.27892 trim_cache size 0 max 16384 2024-08-31T06:17:33.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.307+0000 7fc36640d740 1 Processor -- start 2024-08-31T06:17:33.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.310+0000 7fc36640d740 1 -- start start 2024-08-31T06:17:33.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.311+0000 7fc36640d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560868aae1b0 0x560868e80370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:33.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.311+0000 7fc36640d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560868ab6390 0x560868e81c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:33.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.311+0000 7fc36640d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560868ad7070 0x560868e82180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:33.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.311+0000 7fc36640d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc3581b8270 con 0x560868ad7070 2024-08-31T06:17:33.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.311+0000 7fc36640d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc3581b5b10 con 0x560868aae1b0 2024-08-31T06:17:33.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.311+0000 7fc36640d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc3581b3980 con 0x560868ab6390 2024-08-31T06:17:33.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.313+0000 7fc35cc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560868aae1b0 0x560868e80370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:33.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.313+0000 7fc357e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560868ab6390 0x560868e81c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:33.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.313+0000 7fc357e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560868ab6390 0x560868e81c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:45686/0 (socket says 172.21.15.16:45686) 2024-08-31T06:17:33.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.313+0000 7fc357e00640 1 -- 172.21.15.16:0/1727149660 learned_addr learned my addr 172.21.15.16:0/1727149660 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:33.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.314+0000 7fc35d600640 1 --2- 172.21.15.16:0/1727149660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560868ad7070 0x560868e82180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:33.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.314+0000 7fc357e00640 1 -- 172.21.15.16:0/1727149660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560868aae1b0 msgr2=0x560868e80370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:33.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.314+0000 7fc357e00640 1 --2- 172.21.15.16:0/1727149660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560868aae1b0 0x560868e80370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:33.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.314+0000 7fc357e00640 1 -- 172.21.15.16:0/1727149660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560868ad7070 msgr2=0x560868e82180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:33.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.314+0000 7fc357e00640 1 --2- 172.21.15.16:0/1727149660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560868ad7070 0x560868e82180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:33.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.314+0000 7fc357e00640 1 -- 172.21.15.16:0/1727149660 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560868e89870 con 0x560868ab6390 2024-08-31T06:17:33.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.314+0000 7fc357e00640 1 --2- 172.21.15.16:0/1727149660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560868ab6390 0x560868e81c40 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc3400a8c40 tx=0x7fc340003760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2917871e6af257c4 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:33.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.315+0000 7fc34d200640 1 -- 172.21.15.16:0/1727149660 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc340020040 con 0x560868ab6390 2024-08-31T06:17:33.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.315+0000 7fc34d200640 1 -- 172.21.15.16:0/1727149660 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3400e4030 con 0x560868ab6390 2024-08-31T06:17:33.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.315+0000 7fc34d200640 1 -- 172.21.15.16:0/1727149660 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3400b4cf0 con 0x560868ab6390 2024-08-31T06:17:33.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.318+0000 7fc36640d740 1 -- 172.21.15.16:0/1727149660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560868ab6390 msgr2=0x560868e81c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:33.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.318+0000 7fc36640d740 1 --2- 172.21.15.16:0/1727149660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560868ab6390 0x560868e81c40 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc3400a8c40 tx=0x7fc340003760 comp rx=0 tx=0).stop 2024-08-31T06:17:33.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.319+0000 7fc36640d740 1 -- 172.21.15.16:0/1727149660 shutdown_connections 2024-08-31T06:17:33.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.319+0000 7fc36640d740 1 --2- 172.21.15.16:0/1727149660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560868ab6390 0x560868e81c40 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:33.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.319+0000 7fc36640d740 1 --2- 172.21.15.16:0/1727149660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560868ad7070 0x560868e82180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:33.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.319+0000 7fc36640d740 1 --2- 172.21.15.16:0/1727149660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560868aae1b0 0x560868e80370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:33.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.320+0000 7fc36640d740 1 -- 172.21.15.16:0/1727149660 >> 172.21.15.16:0/1727149660 conn(0x560868acb460 msgr2=0x560868e7fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:33.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.322+0000 7fc36640d740 1 -- 172.21.15.16:0/1727149660 shutdown_connections 2024-08-31T06:17:33.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.322+0000 7fc36640d740 1 -- 172.21.15.16:0/1727149660 wait complete. 2024-08-31T06:17:33.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.323+0000 7fc36640d740 1 -- start start 2024-08-31T06:17:33.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.323+0000 7fc36640d740 1 Processor -- start 2024-08-31T06:17:33.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.324+0000 7fc36640d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560868aae1b0 0x560868e8d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:33.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.324+0000 7fc36640d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560868ab6390 0x560868e8dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:33.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.324+0000 7fc36640d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560868ad7070 0x560868e8e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:33.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.324+0000 7fc36640d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc3581b3fd0 con 0x560868ad7070 2024-08-31T06:17:33.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.324+0000 7fc36640d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc3581b5440 con 0x560868ab6390 2024-08-31T06:17:33.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.324+0000 7fc36640d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc3581b46a0 con 0x560868aae1b0 2024-08-31T06:17:33.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.324+0000 7fc36640d740 10 client.0 _ll_register_callbacks cb 0x7fc348031450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:17:33.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.325+0000 7fc357e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560868ab6390 0x560868e8dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:33.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.325+0000 7fc35d600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560868ad7070 0x560868e8e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:33.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.325+0000 7fc35d600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560868ad7070 0x560868e8e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34932/0 (socket says 172.21.15.16:34932) 2024-08-31T06:17:33.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.325+0000 7fc35d600640 1 -- 172.21.15.16:0/1647740315 learned_addr learned my addr 172.21.15.16:0/1647740315 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:33.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.326+0000 7fc35d600640 1 -- 172.21.15.16:0/1647740315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560868aae1b0 msgr2=0x560868e8d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:33.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.326+0000 7fc35d600640 1 --2- 172.21.15.16:0/1647740315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560868aae1b0 0x560868e8d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:33.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.326+0000 7fc35d600640 1 -- 172.21.15.16:0/1647740315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560868ab6390 msgr2=0x560868e8dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:33.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.326+0000 7fc35d600640 1 --2- 172.21.15.16:0/1647740315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560868ab6390 0x560868e8dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:33.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.326+0000 7fc35d600640 1 -- 172.21.15.16:0/1647740315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560868f16530 con 0x560868ad7070 2024-08-31T06:17:33.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.330+0000 7fc35d600640 1 --2- 172.21.15.16:0/1647740315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560868ad7070 0x560868e8e210 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fc350000c00 tx=0x7fc35000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ce73c89ced5a17 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:33.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.330+0000 7fc346000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:17:33.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.330+0000 7fc346000640 1 -- 172.21.15.16:0/1647740315 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc350016050 con 0x560868ad7070 2024-08-31T06:17:33.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.331+0000 7fc36640d740 1 -- 172.21.15.16:0/1647740315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3400e4030 con 0x560868ad7070 2024-08-31T06:17:33.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.331+0000 7fc346000640 1 -- 172.21.15.16:0/1647740315 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8470+0+0 (secure 0 0 0) 0x7fc350005230 con 0x560868ad7070 2024-08-31T06:17:33.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.331+0000 7fc346000640 1 -- 172.21.15.16:0/1647740315 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc350005430 con 0x560868ad7070 2024-08-31T06:17:33.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.332+0000 7fc36640d740 10 client.17556 fetch_fsmap learned FSMap version 44 2024-08-31T06:17:33.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.332+0000 7fc36640d740 1 -- 172.21.15.16:0/1647740315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x560868f17770 con 0x560868ad7070 2024-08-31T06:17:33.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.336+0000 7fc346000640 1 -- 172.21.15.16:0/1647740315 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3400e4030 con 0x560868ad7070 2024-08-31T06:17:33.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.336+0000 7fc346000640 1 -- 172.21.15.16:0/1647740315 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 44) ==== 8350+0+0 (secure 0 0 0) 0x7fc350024620 con 0x560868ad7070 2024-08-31T06:17:33.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.336+0000 7fc36640d740 10 client.17556 fetch_fsmap finished waiting for FSMap version 44 2024-08-31T06:17:33.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.336+0000 7fc36640d740 10 client.17556 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T06:17:33.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.336+0000 7fc36640d740 20 client.17556 populate_metadata read hostname 'smithi016' 2024-08-31T06:17:33.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.336+0000 7fc36640d740 1 --2- 172.21.15.16:0/1647740315 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x560868a6ada0 0x560868f19d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:33.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.336+0000 7fc36640d740 4 client.17556 mds_command: new command op to 24712 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:17:33.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.336+0000 7fc36640d740 1 -- 172.21.15.16:0/1647740315 --> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560868df11a0 con 0x560868a6ada0 2024-08-31T06:17:33.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.337+0000 7fc35cc00640 1 --2- 172.21.15.16:0/1647740315 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x560868a6ada0 0x560868f19d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:33.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.338+0000 7fc35cc00640 1 --2- 172.21.15.16:0/1647740315 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x560868a6ada0 0x560868f19d50 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=718349477e7481d9 server_cookie=453217aba298e7a6 in_seq=0 out_seq=0 2024-08-31T06:17:33.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.338+0000 7fc346000640 10 client.17556 ms_handle_connect on v2:172.21.15.184:6838/4011051479 2024-08-31T06:17:33.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.339+0000 7fc346000640 1 -- 172.21.15.16:0/1647740315 <== mds.2 v2:172.21.15.184:6838/4011051479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560868df11a0 con 0x560868a6ada0 2024-08-31T06:17:33.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.339+0000 7fc346000640 10 client.17556 handle_command_reply: tid=0 2024-08-31T06:17:33.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.339+0000 7fc36640d740 1 client.17556 shutdown 2024-08-31T06:17:33.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.339+0000 7fc36640d740 1 -- 172.21.15.16:0/1647740315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560868ad7070 msgr2=0x560868e8e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:33.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.339+0000 7fc36640d740 1 --2- 172.21.15.16:0/1647740315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560868ad7070 0x560868e8e210 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fc350000c00 tx=0x7fc35000dbb0 comp rx=0 tx=0).stop 2024-08-31T06:17:33.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.340+0000 7fc36640d740 1 -- 172.21.15.16:0/1647740315 shutdown_connections 2024-08-31T06:17:33.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.340+0000 7fc36640d740 1 --2- 172.21.15.16:0/1647740315 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x560868a6ada0 0x560868f19d50 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:33.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.340+0000 7fc36640d740 1 --2- 172.21.15.16:0/1647740315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560868ad7070 0x560868e8e210 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:33.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.340+0000 7fc346000640 0 client.17556 ms_handle_reset on v2:172.21.15.184:6838/4011051479 2024-08-31T06:17:33.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.340+0000 7fc36640d740 1 --2- 172.21.15.16:0/1647740315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560868ab6390 0x560868e8dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:33.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.340+0000 7fc36640d740 1 --2- 172.21.15.16:0/1647740315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560868aae1b0 0x560868e8d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:33.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.340+0000 7fc36640d740 1 -- 172.21.15.16:0/1647740315 >> 172.21.15.16:0/1647740315 conn(0x560868acb460 msgr2=0x560868e7db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:33.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.341+0000 7fc36640d740 1 -- 172.21.15.16:0/1647740315 shutdown_connections 2024-08-31T06:17:33.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.344+0000 7fc36640d740 1 -- 172.21.15.16:0/1647740315 wait complete. 2024-08-31T06:17:33.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.344+0000 7fc36640d740 20 client.17556 trim_cache size 0 max 16384 2024-08-31T06:17:33.347 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:17:33.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.345+0000 7fc35e000640 1 -- 172.21.15.16:0/277007598 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc33c0b4d20 msgr2=0x7fc33c0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:33.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.345+0000 7fc35e000640 1 --2- 172.21.15.16:0/277007598 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc33c0b4d20 0x7fc33c0b7230 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc348004af0 tx=0x7fc348005fd0 comp rx=0 tx=0).stop 2024-08-31T06:17:33.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.345+0000 7fc35e000640 1 -- 172.21.15.16:0/277007598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc358078030 msgr2=0x7fc3581a0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:33.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.345+0000 7fc35e000640 1 --2- 172.21.15.16:0/277007598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc358078030 0x7fc3581a0c40 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc34000d950 tx=0x7fc34000de10 comp rx=0 tx=0).stop 2024-08-31T06:17:33.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.346+0000 7fc35e000640 1 -- 172.21.15.16:0/277007598 shutdown_connections 2024-08-31T06:17:33.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.346+0000 7fc35e000640 1 --2- 172.21.15.16:0/277007598 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc33c0b4d20 0x7fc33c0b7230 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:33.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.346+0000 7fc35e000640 1 --2- 172.21.15.16:0/277007598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc358078030 0x7fc3581a0c40 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:33.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.346+0000 7fc35e000640 1 --2- 172.21.15.16:0/277007598 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc358114320 0x7fc3581a5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:33.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.346+0000 7fc35e000640 1 --2- 172.21.15.16:0/277007598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3580776e0 0x7fc3581a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:33.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.346+0000 7fc35e000640 1 -- 172.21.15.16:0/277007598 >> 172.21.15.16:0/277007598 conn(0x7fc3581013d0 msgr2=0x7fc358116390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:33.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.346+0000 7fc35e000640 1 -- 172.21.15.16:0/277007598 shutdown_connections 2024-08-31T06:17:33.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:33.346+0000 7fc35e000640 1 -- 172.21.15.16:0/277007598 wait complete. 2024-08-31T06:17:33.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:33 smithi016 ceph-mon[17194]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:17:33.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:33 smithi184 ceph-mon[22331]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:17:34.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:33 smithi155 ceph-mon[21742]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:17:34.342 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T06:17:34.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:34 smithi016 ceph-mon[17194]: pgmap v684: 97 pgs: 97 active+clean; 3.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 21 MiB/s wr, 1.33k op/s 2024-08-31T06:17:34.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:34 smithi016 ceph-mon[17194]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:17:34.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:34 smithi184 ceph-mon[22331]: pgmap v684: 97 pgs: 97 active+clean; 3.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 21 MiB/s wr, 1.33k op/s 2024-08-31T06:17:34.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:34 smithi184 ceph-mon[22331]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:17:35.019 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:17:35.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:34 smithi155 ceph-mon[21742]: pgmap v684: 97 pgs: 97 active+clean; 3.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 21 MiB/s wr, 1.33k op/s 2024-08-31T06:17:35.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:34 smithi155 ceph-mon[21742]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:17:35.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.966+0000 7f64e3c00640 1 -- 172.21.15.16:0/3626824786 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64dc109270 msgr2=0x7f64dc109680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:35.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.966+0000 7f64e3c00640 1 --2- 172.21.15.16:0/3626824786 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64dc109270 0x7f64dc109680 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f64c8009900 tx=0x7f64c802a400 comp rx=0 tx=0).stop 2024-08-31T06:17:35.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.966+0000 7f64e3c00640 1 -- 172.21.15.16:0/3626824786 shutdown_connections 2024-08-31T06:17:35.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.966+0000 7f64e3c00640 1 --2- 172.21.15.16:0/3626824786 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64dc109270 0x7f64dc109680 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:35.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.966+0000 7f64e3c00640 1 --2- 172.21.15.16:0/3626824786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64dc109bc0 0x7f64dc1103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:35.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.966+0000 7f64e3c00640 1 --2- 172.21.15.16:0/3626824786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64dc103930 0x7f64dc103d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:35.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.966+0000 7f64e3c00640 1 -- 172.21.15.16:0/3626824786 >> 172.21.15.16:0/3626824786 conn(0x7f64dc0ff380 msgr2=0x7f64dc1017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:35.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.966+0000 7f64e3c00640 1 -- 172.21.15.16:0/3626824786 shutdown_connections 2024-08-31T06:17:35.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.966+0000 7f64e3c00640 1 -- 172.21.15.16:0/3626824786 wait complete. 2024-08-31T06:17:35.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.968+0000 7f64e3c00640 1 Processor -- start 2024-08-31T06:17:35.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.968+0000 7f64e3c00640 1 -- start start 2024-08-31T06:17:35.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.969+0000 7f64e3c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64dc103930 0x7f64dc076ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:35.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.969+0000 7f64e3c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64dc109270 0x7f64dc0754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:35.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.969+0000 7f64e3c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64dc109bc0 0x7f64dc0759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:35.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.969+0000 7f64e3c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f64dc113880 con 0x7f64dc109270 2024-08-31T06:17:35.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.969+0000 7f64e3c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f64dc113700 con 0x7f64dc103930 2024-08-31T06:17:35.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.969+0000 7f64e3c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f64dc113a00 con 0x7f64dc109bc0 2024-08-31T06:17:35.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.969+0000 7f64e1200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64dc103930 0x7f64dc076ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:35.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.969+0000 7f64e1200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64dc103930 0x7f64dc076ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:38058/0 (socket says 172.21.15.16:38058) 2024-08-31T06:17:35.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.969+0000 7f64e1200640 1 -- 172.21.15.16:0/3658552937 learned_addr learned my addr 172.21.15.16:0/3658552937 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:35.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.969+0000 7f64dbe00640 1 --2- 172.21.15.16:0/3658552937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64dc109270 0x7f64dc0754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:35.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.969+0000 7f64e1c00640 1 --2- 172.21.15.16:0/3658552937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64dc109bc0 0x7f64dc0759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:35.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.970+0000 7f64e1c00640 1 -- 172.21.15.16:0/3658552937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64dc103930 msgr2=0x7f64dc076ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:35.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.970+0000 7f64e1c00640 1 --2- 172.21.15.16:0/3658552937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64dc103930 0x7f64dc076ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:35.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.970+0000 7f64e1c00640 1 -- 172.21.15.16:0/3658552937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64dc109270 msgr2=0x7f64dc0754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:35.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.970+0000 7f64e1c00640 1 --2- 172.21.15.16:0/3658552937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64dc109270 0x7f64dc0754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:35.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.970+0000 7f64e1c00640 1 -- 172.21.15.16:0/3658552937 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64dc076210 con 0x7f64dc109bc0 2024-08-31T06:17:35.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.970+0000 7f64e1c00640 1 --2- 172.21.15.16:0/3658552937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64dc109bc0 0x7f64dc0759f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f64d400ec50 tx=0x7f64d400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:35.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.971+0000 7f64d9600640 1 -- 172.21.15.16:0/3658552937 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d4003b70 con 0x7f64dc109bc0 2024-08-31T06:17:35.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.972+0000 7f64e3c00640 1 -- 172.21.15.16:0/3658552937 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64dc19e430 con 0x7f64dc109bc0 2024-08-31T06:17:35.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.972+0000 7f64d9600640 1 -- 172.21.15.16:0/3658552937 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64d4003d10 con 0x7f64dc109bc0 2024-08-31T06:17:35.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.972+0000 7f64d9600640 1 -- 172.21.15.16:0/3658552937 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d4003b70 con 0x7f64dc109bc0 2024-08-31T06:17:35.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.972+0000 7f64e3c00640 1 -- 172.21.15.16:0/3658552937 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64dc19e940 con 0x7f64dc109bc0 2024-08-31T06:17:35.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.978+0000 7f64ea898740 1 Processor -- start 2024-08-31T06:17:35.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.978+0000 7f64d9600640 1 -- 172.21.15.16:0/3658552937 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f64d4010540 con 0x7f64dc109bc0 2024-08-31T06:17:35.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.980+0000 7f64ea898740 1 -- start start 2024-08-31T06:17:35.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.981+0000 7f64ea898740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed140f5070 0x55ed1449a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:35.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.981+0000 7f64ea898740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed140cc1b0 0x55ed1449aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:35.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.981+0000 7f64ea898740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed140d4390 0x55ed1449b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:35.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.981+0000 7f64ea898740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ed144117b0 con 0x55ed140cc1b0 2024-08-31T06:17:35.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.981+0000 7f64ea898740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ed14400d40 con 0x55ed140f5070 2024-08-31T06:17:35.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.981+0000 7f64ea898740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ed14404cf0 con 0x55ed140d4390 2024-08-31T06:17:35.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.981+0000 7f64d9600640 1 --2- 172.21.15.16:0/3658552937 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f64c40c60b0 0x7f64c40c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:35.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.981+0000 7f64d9600640 1 -- 172.21.15.16:0/3658552937 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8470+0+0 (secure 0 0 0) 0x7f64d4003970 con 0x7f64dc109bc0 2024-08-31T06:17:35.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.981+0000 7f64dbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed140f5070 0x55ed1449a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:35.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.981+0000 7f64dbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed140f5070 0x55ed1449a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:38074/0 (socket says 172.21.15.16:38074) 2024-08-31T06:17:35.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.981+0000 7f64dbe00640 1 -- 172.21.15.16:0/2385517824 learned_addr learned my addr 172.21.15.16:0/2385517824 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:35.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.982+0000 7f64e1200640 1 --2- 172.21.15.16:0/2385517824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed140cc1b0 0x55ed1449aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:35.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.982+0000 7f64e1c00640 1 --2- 172.21.15.16:0/3658552937 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f64c40c60b0 0x7f64c40c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:35.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.983+0000 7f64dbe00640 1 --2- 172.21.15.16:0/2385517824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed140d4390 0x55ed1449b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:35.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.983+0000 7f64e1c00640 1 --2- 172.21.15.16:0/3658552937 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f64c40c60b0 0x7f64c40c85c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f64d40a7a50 tx=0x7f64d4005f70 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:35.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.983+0000 7f64dbe00640 1 -- 172.21.15.16:0/2385517824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed140d4390 msgr2=0x55ed1449b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:35.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.983+0000 7f64dbe00640 1 --2- 172.21.15.16:0/2385517824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed140d4390 0x55ed1449b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:35.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.983+0000 7f64dbe00640 1 -- 172.21.15.16:0/2385517824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed140cc1b0 msgr2=0x55ed1449aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:35.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.983+0000 7f64dbe00640 1 --2- 172.21.15.16:0/2385517824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed140cc1b0 0x55ed1449aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:35.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.983+0000 7f64dbe00640 1 -- 172.21.15.16:0/2385517824 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed14160650 con 0x55ed140f5070 2024-08-31T06:17:35.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.983+0000 7f64e1200640 1 --2- 172.21.15.16:0/2385517824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed140cc1b0 0x55ed1449aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:17:35.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.984+0000 7f64dbe00640 1 --2- 172.21.15.16:0/2385517824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed140f5070 0x55ed1449a990 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f64dc06b8d0 tx=0x7f64c8000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=256f27f7b395ddbd server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:35.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.984+0000 7f64ce000640 1 -- 172.21.15.16:0/2385517824 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c8035040 con 0x55ed140f5070 2024-08-31T06:17:35.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.984+0000 7f64ce000640 1 -- 172.21.15.16:0/2385517824 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64c80314a0 con 0x55ed140f5070 2024-08-31T06:17:35.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.984+0000 7f64ce000640 1 -- 172.21.15.16:0/2385517824 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c8005040 con 0x55ed140f5070 2024-08-31T06:17:35.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.986+0000 7f64ea898740 1 -- 172.21.15.16:0/2385517824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed140f5070 msgr2=0x55ed1449a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:35.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.986+0000 7f64ea898740 1 --2- 172.21.15.16:0/2385517824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed140f5070 0x55ed1449a990 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f64dc06b8d0 tx=0x7f64c8000bc0 comp rx=0 tx=0).stop 2024-08-31T06:17:35.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.991+0000 7f64ea898740 1 -- 172.21.15.16:0/2385517824 shutdown_connections 2024-08-31T06:17:35.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.991+0000 7f64ea898740 1 --2- 172.21.15.16:0/2385517824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed140d4390 0x55ed1449b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:35.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.991+0000 7f64ea898740 1 --2- 172.21.15.16:0/2385517824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed140cc1b0 0x55ed1449aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:35.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.991+0000 7f64ea898740 1 --2- 172.21.15.16:0/2385517824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed140f5070 0x55ed1449a990 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:35.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.992+0000 7f64ea898740 1 -- 172.21.15.16:0/2385517824 >> 172.21.15.16:0/2385517824 conn(0x55ed140e9460 msgr2=0x55ed14498880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:35.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.993+0000 7f64ea898740 1 -- 172.21.15.16:0/2385517824 shutdown_connections 2024-08-31T06:17:35.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.993+0000 7f64ea898740 1 -- 172.21.15.16:0/2385517824 wait complete. 2024-08-31T06:17:35.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.994+0000 7f64ea898740 1 -- start start 2024-08-31T06:17:35.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.995+0000 7f64ea898740 1 Processor -- start 2024-08-31T06:17:36.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.996+0000 7f64ea898740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed140cc1b0 0x55ed144a6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:36.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.996+0000 7f64ea898740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed140d4390 0x55ed144a6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:36.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.996+0000 7f64ea898740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed140f5070 0x55ed144b2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:36.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.996+0000 7f64ea898740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ed14263700 con 0x55ed140d4390 2024-08-31T06:17:36.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.996+0000 7f64ea898740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ed14387cf0 con 0x55ed140f5070 2024-08-31T06:17:36.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.996+0000 7f64ea898740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ed144a9110 con 0x55ed140cc1b0 2024-08-31T06:17:36.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.996+0000 7f64ea898740 10 client.0 _ll_register_callbacks cb 0x55ed143ad470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:17:36.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:35.996+0000 7f64e1200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed140d4390 0x55ed144a6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:36.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.000+0000 7f64e1200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed140d4390 0x55ed144a6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39944/0 (socket says 172.21.15.16:39944) 2024-08-31T06:17:36.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.000+0000 7f64e1200640 1 -- 172.21.15.16:0/3685677473 learned_addr learned my addr 172.21.15.16:0/3685677473 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:36.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.000+0000 7f64dbe00640 1 --2- 172.21.15.16:0/3685677473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed140f5070 0x55ed144b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:36.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.001+0000 7f64dbe00640 1 -- 172.21.15.16:0/3685677473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed140cc1b0 msgr2=0x55ed144a6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:36.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.001+0000 7f64dbe00640 1 --2- 172.21.15.16:0/3685677473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed140cc1b0 0x55ed144a6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:36.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.001+0000 7f64dbe00640 1 -- 172.21.15.16:0/3685677473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed140d4390 msgr2=0x55ed144a6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:36.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.001+0000 7f64dbe00640 1 --2- 172.21.15.16:0/3685677473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed140d4390 0x55ed144a6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:36.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.001+0000 7f64dbe00640 1 -- 172.21.15.16:0/3685677473 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed143c88b0 con 0x55ed140f5070 2024-08-31T06:17:36.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.002+0000 7f64dbe00640 1 --2- 172.21.15.16:0/3685677473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed140f5070 0x55ed144b2490 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f64c8004320 tx=0x7f64c80045f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2658faee9e487b5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:36.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.002+0000 7f64c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:17:36.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.003+0000 7f64c2a00640 1 -- 172.21.15.16:0/3685677473 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c8035030 con 0x55ed140f5070 2024-08-31T06:17:36.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.004+0000 7f64c2a00640 1 -- 172.21.15.16:0/3685677473 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8470+0+0 (secure 0 0 0) 0x7f64c8031b10 con 0x55ed140f5070 2024-08-31T06:17:36.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.005+0000 7f64ea898740 1 -- 172.21.15.16:0/3685677473 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64c80314a0 con 0x55ed140f5070 2024-08-31T06:17:36.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.007+0000 7f64c2a00640 1 -- 172.21.15.16:0/3685677473 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f64c8031d10 con 0x55ed140f5070 2024-08-31T06:17:36.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.007+0000 7f64ea898740 10 client.27902 fetch_fsmap learned FSMap version 45 2024-08-31T06:17:36.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.007+0000 7f64ea898740 1 -- 172.21.15.16:0/3685677473 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55ed144c6170 con 0x55ed140f5070 2024-08-31T06:17:36.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.008+0000 7f64c2a00640 1 -- 172.21.15.16:0/3685677473 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64c80314a0 con 0x55ed140f5070 2024-08-31T06:17:36.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.008+0000 7f64c2a00640 1 -- 172.21.15.16:0/3685677473 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f64c8052070 con 0x55ed140f5070 2024-08-31T06:17:36.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.008+0000 7f64ea898740 10 client.27902 fetch_fsmap finished waiting for FSMap version 45 2024-08-31T06:17:36.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.008+0000 7f64ea898740 10 client.27902 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-31T06:17:36.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.008+0000 7f64ea898740 20 client.27902 populate_metadata read hostname 'smithi016' 2024-08-31T06:17:36.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.009+0000 7f64ea898740 1 --2- 172.21.15.16:0/3685677473 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55ed140b39f0 0x55ed144c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:36.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.009+0000 7f64ea898740 4 client.27902 mds_command: new command op to 24724 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:17:36.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.009+0000 7f64ea898740 1 -- 172.21.15.16:0/3685677473 --> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed14413700 con 0x55ed140b39f0 2024-08-31T06:17:36.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.009+0000 7f64e1200640 1 --2- 172.21.15.16:0/3685677473 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55ed140b39f0 0x55ed144c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:36.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.013+0000 7f64e1200640 1 --2- 172.21.15.16:0/3685677473 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55ed140b39f0 0x55ed144c94d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6653cbd8758a674a server_cookie=d2ce31bcf311805b in_seq=0 out_seq=0 2024-08-31T06:17:36.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.013+0000 7f64c2a00640 10 client.27902 ms_handle_connect on v2:172.21.15.16:6838/219664460 2024-08-31T06:17:36.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.015+0000 7f64c2a00640 1 -- 172.21.15.16:0/3685677473 <== mds.1 v2:172.21.15.16:6838/219664460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ed14413700 con 0x55ed140b39f0 2024-08-31T06:17:36.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.015+0000 7f64c2a00640 10 client.27902 handle_command_reply: tid=0 2024-08-31T06:17:36.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.016+0000 7f64ea898740 1 client.27902 shutdown 2024-08-31T06:17:36.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.016+0000 7f64ea898740 1 -- 172.21.15.16:0/3685677473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed140f5070 msgr2=0x55ed144b2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:36.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.017+0000 7f64ea898740 1 --2- 172.21.15.16:0/3685677473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed140f5070 0x55ed144b2490 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f64c8004320 tx=0x7f64c80045f0 comp rx=0 tx=0).stop 2024-08-31T06:17:36.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.018+0000 7f64ea898740 1 -- 172.21.15.16:0/3685677473 shutdown_connections 2024-08-31T06:17:36.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.018+0000 7f64ea898740 1 --2- 172.21.15.16:0/3685677473 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55ed140b39f0 0x55ed144c94d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.018+0000 7f64ea898740 1 --2- 172.21.15.16:0/3685677473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed140f5070 0x55ed144b2490 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.018+0000 7f64ea898740 1 --2- 172.21.15.16:0/3685677473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed140d4390 0x55ed144a6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.018+0000 7f64ea898740 1 --2- 172.21.15.16:0/3685677473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed140cc1b0 0x55ed144a6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.018+0000 7f64ea898740 1 -- 172.21.15.16:0/3685677473 >> 172.21.15.16:0/3685677473 conn(0x55ed140e9460 msgr2=0x55ed1449db20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.018+0000 7f64c2a00640 0 client.27902 ms_handle_reset on v2:172.21.15.16:6838/219664460 2024-08-31T06:17:36.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.019+0000 7f64ea898740 1 -- 172.21.15.16:0/3685677473 shutdown_connections 2024-08-31T06:17:36.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.023+0000 7f64ea898740 1 -- 172.21.15.16:0/3685677473 wait complete. 2024-08-31T06:17:36.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.024+0000 7f64ea898740 20 client.27902 trim_cache size 0 max 16384 2024-08-31T06:17:36.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.052+0000 7f64ea898740 1 Processor -- start 2024-08-31T06:17:36.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.053+0000 7f64ea898740 1 -- start start 2024-08-31T06:17:36.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.053+0000 7f64ea898740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed140cc1b0 0x55ed144b2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:36.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.053+0000 7f64ea898740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed140d4390 0x55ed144a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:36.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.053+0000 7f64ea898740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed140f5070 0x55ed144a5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:36.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.053+0000 7f64ea898740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc011160 con 0x55ed140f5070 2024-08-31T06:17:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.053+0000 7f64ea898740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc00e9d0 con 0x55ed140cc1b0 2024-08-31T06:17:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.053+0000 7f64ea898740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc00c800 con 0x55ed140d4390 2024-08-31T06:17:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.054+0000 7f64dbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed140cc1b0 0x55ed144b2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.054+0000 7f64e1200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed140d4390 0x55ed144a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.054+0000 7f64e1200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed140d4390 0x55ed144a4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:60922/0 (socket says 172.21.15.16:60922) 2024-08-31T06:17:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.054+0000 7f64dbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed140cc1b0 0x55ed144b2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:38094/0 (socket says 172.21.15.16:38094) 2024-08-31T06:17:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.054+0000 7f64e1200640 1 -- 172.21.15.16:0/2537214001 learned_addr learned my addr 172.21.15.16:0/2537214001 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.054+0000 7f64dbe00640 1 --2- 172.21.15.16:0/2537214001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed140f5070 0x55ed144a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.054+0000 7f64dbe00640 1 -- 172.21.15.16:0/2537214001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed140d4390 msgr2=0x55ed144a4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.054+0000 7f64dbe00640 1 --2- 172.21.15.16:0/2537214001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed140d4390 0x55ed144a4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.054+0000 7f64dbe00640 1 -- 172.21.15.16:0/2537214001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed140f5070 msgr2=0x55ed144a5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.054+0000 7f64dbe00640 1 --2- 172.21.15.16:0/2537214001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed140f5070 0x55ed144a5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.054+0000 7f64dbe00640 1 -- 172.21.15.16:0/2537214001 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed144a7610 con 0x55ed140cc1b0 2024-08-31T06:17:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.055+0000 7f64dbe00640 1 --2- 172.21.15.16:0/2537214001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed140cc1b0 0x55ed144b2400 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f64c8033000 tx=0x7f64c802a400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63742e35b33c6a39 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:36.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.055+0000 7f64cd600640 1 -- 172.21.15.16:0/2537214001 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c8059080 con 0x55ed140cc1b0 2024-08-31T06:17:36.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.057+0000 7f64cd600640 1 -- 172.21.15.16:0/2537214001 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64c80314a0 con 0x55ed140cc1b0 2024-08-31T06:17:36.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.057+0000 7f64cd600640 1 -- 172.21.15.16:0/2537214001 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c8031d10 con 0x55ed140cc1b0 2024-08-31T06:17:36.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.058+0000 7f64ea898740 1 -- 172.21.15.16:0/2537214001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed140cc1b0 msgr2=0x55ed144b2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:36.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.058+0000 7f64ea898740 1 --2- 172.21.15.16:0/2537214001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed140cc1b0 0x55ed144b2400 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f64c8033000 tx=0x7f64c802a400 comp rx=0 tx=0).stop 2024-08-31T06:17:36.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.059+0000 7f64ea898740 1 -- 172.21.15.16:0/2537214001 shutdown_connections 2024-08-31T06:17:36.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.059+0000 7f64ea898740 1 --2- 172.21.15.16:0/2537214001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed140d4390 0x55ed144a4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:36.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.059+0000 7f64ea898740 1 --2- 172.21.15.16:0/2537214001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed140f5070 0x55ed144a5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:36.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.059+0000 7f64ea898740 1 --2- 172.21.15.16:0/2537214001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed140cc1b0 0x55ed144b2400 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:36.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.059+0000 7f64ea898740 1 -- 172.21.15.16:0/2537214001 >> 172.21.15.16:0/2537214001 conn(0x55ed140e9460 msgr2=0x55ed144a1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:36.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.059+0000 7f64ea898740 1 -- 172.21.15.16:0/2537214001 shutdown_connections 2024-08-31T06:17:36.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.059+0000 7f64ea898740 1 -- 172.21.15.16:0/2537214001 wait complete. 2024-08-31T06:17:36.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.060+0000 7f64ea898740 1 -- start start 2024-08-31T06:17:36.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.060+0000 7f64ea898740 1 Processor -- start 2024-08-31T06:17:36.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.061+0000 7f64ea898740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed140cc1b0 0x55ed144ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:36.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.062+0000 7f64ea898740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed140d4390 0x55ed144ca440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:36.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.062+0000 7f64ea898740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed140f5070 0x55ed144cab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:36.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.062+0000 7f64ea898740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc00ce90 con 0x55ed140d4390 2024-08-31T06:17:36.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.062+0000 7f64ea898740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc00e300 con 0x55ed140f5070 2024-08-31T06:17:36.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.062+0000 7f64ea898740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc00d560 con 0x55ed140cc1b0 2024-08-31T06:17:36.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.062+0000 7f64ea898740 10 client.0 _ll_register_callbacks cb 0x7f64c8009540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:17:36.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.063+0000 7f64dbe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed140cc1b0 0x55ed144ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:36.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.063+0000 7f64dbe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed140cc1b0 0x55ed144ccf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:60930/0 (socket says 172.21.15.16:60930) 2024-08-31T06:17:36.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.063+0000 7f64dbe00640 1 -- 172.21.15.16:0/2463810435 learned_addr learned my addr 172.21.15.16:0/2463810435 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:36.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.063+0000 7f64dbe00640 1 --2- 172.21.15.16:0/2463810435 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed140f5070 0x55ed144cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:36.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.064+0000 7f64dbe00640 1 -- 172.21.15.16:0/2463810435 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed140cc1b0 msgr2=0x55ed144ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:36.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.064+0000 7f64dbe00640 1 --2- 172.21.15.16:0/2463810435 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed140cc1b0 0x55ed144ccf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:36.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.064+0000 7f64dbe00640 1 -- 172.21.15.16:0/2463810435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed140d4390 msgr2=0x55ed144ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:36.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.064+0000 7f64dbe00640 1 --2- 172.21.15.16:0/2463810435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed140d4390 0x55ed144ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:36.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.064+0000 7f64dbe00640 1 -- 172.21.15.16:0/2463810435 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed14532c50 con 0x55ed140f5070 2024-08-31T06:17:36.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.065+0000 7f64dbe00640 1 --2- 172.21.15.16:0/2463810435 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed140f5070 0x55ed144cab50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f64c8009900 tx=0x7f64c8000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bc5bdb8b73ee09e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:36.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.065+0000 7f64c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:17:36.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.065+0000 7f64c2a00640 1 -- 172.21.15.16:0/2463810435 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c8059040 con 0x55ed140f5070 2024-08-31T06:17:36.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.065+0000 7f64c2a00640 1 -- 172.21.15.16:0/2463810435 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8470+0+0 (secure 0 0 0) 0x7f64c8067030 con 0x55ed140f5070 2024-08-31T06:17:36.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.065+0000 7f64ea898740 1 -- 172.21.15.16:0/2463810435 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64c80314a0 con 0x55ed140f5070 2024-08-31T06:17:36.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.066+0000 7f64c2a00640 1 -- 172.21.15.16:0/2463810435 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f64c80046c0 con 0x55ed140f5070 2024-08-31T06:17:36.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.066+0000 7f64ea898740 10 client.27908 fetch_fsmap learned FSMap version 45 2024-08-31T06:17:36.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.066+0000 7f64ea898740 1 -- 172.21.15.16:0/2463810435 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55ed14533dd0 con 0x55ed140f5070 2024-08-31T06:17:36.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.066+0000 7f64c2a00640 1 -- 172.21.15.16:0/2463810435 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64c80314a0 con 0x55ed140f5070 2024-08-31T06:17:36.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.067+0000 7f64c2a00640 1 -- 172.21.15.16:0/2463810435 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f64c8052070 con 0x55ed140f5070 2024-08-31T06:17:36.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.067+0000 7f64ea898740 10 client.27908 fetch_fsmap finished waiting for FSMap version 45 2024-08-31T06:17:36.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.067+0000 7f64ea898740 10 client.27908 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-31T06:17:36.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.067+0000 7f64ea898740 20 client.27908 populate_metadata read hostname 'smithi016' 2024-08-31T06:17:36.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.067+0000 7f64ea898740 1 --2- 172.21.15.16:0/2463810435 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55ed14088da0 0x55ed145363a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:36.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.067+0000 7f64ea898740 4 client.27908 mds_command: new command op to 24724 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:17:36.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.068+0000 7f64ea898740 1 -- 172.21.15.16:0/2463810435 --> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ed1440f1a0 con 0x55ed14088da0 2024-08-31T06:17:36.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.068+0000 7f64e1200640 1 --2- 172.21.15.16:0/2463810435 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55ed14088da0 0x55ed145363a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:36.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.070+0000 7f64c2a00640 10 client.27908 ms_handle_connect on v2:172.21.15.16:6838/219664460 2024-08-31T06:17:36.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.070+0000 7f64e1200640 1 --2- 172.21.15.16:0/2463810435 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55ed14088da0 0x55ed145363a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=84b655462a59cbe8 server_cookie=30b85e1a6d73786f in_seq=0 out_seq=0 2024-08-31T06:17:36.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.071+0000 7f64c2a00640 1 -- 172.21.15.16:0/2463810435 <== mds.1 v2:172.21.15.16:6838/219664460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ed1440f1a0 con 0x55ed14088da0 2024-08-31T06:17:36.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.071+0000 7f64c2a00640 10 client.27908 handle_command_reply: tid=0 2024-08-31T06:17:36.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.071+0000 7f64ea898740 1 client.27908 shutdown 2024-08-31T06:17:36.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.071+0000 7f64ea898740 1 -- 172.21.15.16:0/2463810435 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed140f5070 msgr2=0x55ed144cab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:36.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.071+0000 7f64ea898740 1 --2- 172.21.15.16:0/2463810435 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed140f5070 0x55ed144cab50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f64c8009900 tx=0x7f64c8000bc0 comp rx=0 tx=0).stop 2024-08-31T06:17:36.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.071+0000 7f64ea898740 1 -- 172.21.15.16:0/2463810435 shutdown_connections 2024-08-31T06:17:36.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.071+0000 7f64ea898740 1 --2- 172.21.15.16:0/2463810435 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55ed14088da0 0x55ed145363a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:36.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.071+0000 7f64ea898740 1 --2- 172.21.15.16:0/2463810435 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed140f5070 0x55ed144cab50 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:36.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.071+0000 7f64ea898740 1 --2- 172.21.15.16:0/2463810435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed140d4390 0x55ed144ca440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:36.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.071+0000 7f64ea898740 1 --2- 172.21.15.16:0/2463810435 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed140cc1b0 0x55ed144ccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:36.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.071+0000 7f64ea898740 1 -- 172.21.15.16:0/2463810435 >> 172.21.15.16:0/2463810435 conn(0x55ed140e9460 msgr2=0x55ed144a4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:36.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.072+0000 7f64c2a00640 0 client.27908 ms_handle_reset on v2:172.21.15.16:6838/219664460 2024-08-31T06:17:36.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.072+0000 7f64ea898740 1 -- 172.21.15.16:0/2463810435 shutdown_connections 2024-08-31T06:17:36.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.072+0000 7f64ea898740 1 -- 172.21.15.16:0/2463810435 wait complete. 2024-08-31T06:17:36.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.072+0000 7f64ea898740 20 client.27908 trim_cache size 0 max 16384 2024-08-31T06:17:36.076 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:17:36.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.073+0000 7f64e3c00640 1 -- 172.21.15.16:0/3658552937 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f64c40c60b0 msgr2=0x7f64c40c85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:36.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.073+0000 7f64e3c00640 1 --2- 172.21.15.16:0/3658552937 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f64c40c60b0 0x7f64c40c85c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f64d40a7a50 tx=0x7f64d4005f70 comp rx=0 tx=0).stop 2024-08-31T06:17:36.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.073+0000 7f64e3c00640 1 -- 172.21.15.16:0/3658552937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64dc109bc0 msgr2=0x7f64dc0759f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:36.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.074+0000 7f64e3c00640 1 --2- 172.21.15.16:0/3658552937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64dc109bc0 0x7f64dc0759f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f64d400ec50 tx=0x7f64d400c6a0 comp rx=0 tx=0).stop 2024-08-31T06:17:36.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.074+0000 7f64e3c00640 1 -- 172.21.15.16:0/3658552937 shutdown_connections 2024-08-31T06:17:36.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.074+0000 7f64e3c00640 1 --2- 172.21.15.16:0/3658552937 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f64c40c60b0 0x7f64c40c85c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:36.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.074+0000 7f64e3c00640 1 --2- 172.21.15.16:0/3658552937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64dc109bc0 0x7f64dc0759f0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:36.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.074+0000 7f64e3c00640 1 --2- 172.21.15.16:0/3658552937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64dc109270 0x7f64dc0754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:36.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.074+0000 7f64e3c00640 1 --2- 172.21.15.16:0/3658552937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64dc103930 0x7f64dc076ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:36.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.074+0000 7f64e3c00640 1 -- 172.21.15.16:0/3658552937 >> 172.21.15.16:0/3658552937 conn(0x7f64dc0ff380 msgr2=0x7f64dc1017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:36.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.075+0000 7f64e3c00640 1 -- 172.21.15.16:0/3658552937 shutdown_connections 2024-08-31T06:17:36.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:36.075+0000 7f64e3c00640 1 -- 172.21.15.16:0/3658552937 wait complete. 2024-08-31T06:17:36.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 610.0 s, 1785.6 tps, lat 55.762 ms stddev 62.633 2024-08-31T06:17:36.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:36 smithi016 ceph-mon[17194]: pgmap v685: 97 pgs: 97 active+clean; 3.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 24 MiB/s wr, 1.43k op/s 2024-08-31T06:17:36.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:36 smithi184 ceph-mon[22331]: pgmap v685: 97 pgs: 97 active+clean; 3.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 24 MiB/s wr, 1.43k op/s 2024-08-31T06:17:37.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:36 smithi155 ceph-mon[21742]: pgmap v685: 97 pgs: 97 active+clean; 3.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 24 MiB/s wr, 1.43k op/s 2024-08-31T06:17:37.306 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T06:17:37.934 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:17:38.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.795+0000 7faec4a00640 1 -- 172.21.15.16:0/1547568556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faec010e820 msgr2=0x7faec010ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:38.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.795+0000 7faec4a00640 1 --2- 172.21.15.16:0/1547568556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faec010e820 0x7faec010ec30 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7faeac009900 tx=0x7faeac02a3f0 comp rx=0 tx=0).stop 2024-08-31T06:17:38.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.795+0000 7faec4a00640 1 -- 172.21.15.16:0/1547568556 shutdown_connections 2024-08-31T06:17:38.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.795+0000 7faec4a00640 1 --2- 172.21.15.16:0/1547568556 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faec010f170 0x7faec0118be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:38.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.795+0000 7faec4a00640 1 --2- 172.21.15.16:0/1547568556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faec010e820 0x7faec010ec30 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:38.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.795+0000 7faec4a00640 1 --2- 172.21.15.16:0/1547568556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faec006bca0 0x7faec006c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:38.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.795+0000 7faec4a00640 1 -- 172.21.15.16:0/1547568556 >> 172.21.15.16:0/1547568556 conn(0x7faec006e6d0 msgr2=0x7faec0070af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:38.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.795+0000 7faec4a00640 1 -- 172.21.15.16:0/1547568556 shutdown_connections 2024-08-31T06:17:38.800 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:38 smithi016 ceph-mon[17194]: pgmap v686: 97 pgs: 97 active+clean; 3.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 20 MiB/s wr, 1.29k op/s 2024-08-31T06:17:38.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.796+0000 7faec4a00640 1 -- 172.21.15.16:0/1547568556 wait complete. 2024-08-31T06:17:38.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.797+0000 7faec4a00640 1 Processor -- start 2024-08-31T06:17:38.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.800+0000 7faec4a00640 1 -- start start 2024-08-31T06:17:38.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.800+0000 7faec4a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faec006bca0 0x7faec01a2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:38.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.800+0000 7faec4a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faec010e820 0x7faec01a2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:38.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.800+0000 7faec4a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faec010f170 0x7faec01a8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:38.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.800+0000 7faec4a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faec011bfe0 con 0x7faec010f170 2024-08-31T06:17:38.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.800+0000 7faec4a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faec011be60 con 0x7faec010e820 2024-08-31T06:17:38.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.800+0000 7faec4a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faec011c160 con 0x7faec006bca0 2024-08-31T06:17:38.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.801+0000 7faebde00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faec006bca0 0x7faec01a2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:38.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.801+0000 7faebde00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faec006bca0 0x7faec01a2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:60956/0 (socket says 172.21.15.16:60956) 2024-08-31T06:17:38.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.801+0000 7faebde00640 1 -- 172.21.15.16:0/1330746233 learned_addr learned my addr 172.21.15.16:0/1330746233 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:38.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.801+0000 7faebd400640 1 --2- 172.21.15.16:0/1330746233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faec010e820 0x7faec01a2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:38.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.801+0000 7faebe800640 1 --2- 172.21.15.16:0/1330746233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faec010f170 0x7faec01a8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:38.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.801+0000 7faebde00640 1 -- 172.21.15.16:0/1330746233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faec010e820 msgr2=0x7faec01a2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:38.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.801+0000 7faebde00640 1 --2- 172.21.15.16:0/1330746233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faec010e820 0x7faec01a2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:38.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.801+0000 7faebde00640 1 -- 172.21.15.16:0/1330746233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faec010f170 msgr2=0x7faec01a8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:38.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.801+0000 7faebde00640 1 --2- 172.21.15.16:0/1330746233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faec010f170 0x7faec01a8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:38.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.801+0000 7faebde00640 1 -- 172.21.15.16:0/1330746233 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faec01a89c0 con 0x7faec006bca0 2024-08-31T06:17:38.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.801+0000 7faebe800640 1 --2- 172.21.15.16:0/1330746233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faec010f170 0x7faec01a8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:17:38.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.802+0000 7faebde00640 1 --2- 172.21.15.16:0/1330746233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faec006bca0 0x7faec01a2a50 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7faeb800eab0 tx=0x7faeb800ef70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:38.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.802+0000 7faeb6a00640 1 -- 172.21.15.16:0/1330746233 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb800ce40 con 0x7faec006bca0 2024-08-31T06:17:38.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.802+0000 7faec4a00640 1 -- 172.21.15.16:0/1330746233 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faec01a8cb0 con 0x7faec006bca0 2024-08-31T06:17:38.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.802+0000 7faeb6a00640 1 -- 172.21.15.16:0/1330746233 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faeb8011920 con 0x7faec006bca0 2024-08-31T06:17:38.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.802+0000 7faeb6a00640 1 -- 172.21.15.16:0/1330746233 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb8010a30 con 0x7faec006bca0 2024-08-31T06:17:38.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.802+0000 7faec4a00640 1 -- 172.21.15.16:0/1330746233 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faec01af410 con 0x7faec006bca0 2024-08-31T06:17:38.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.806+0000 7faecb64c740 1 Processor -- start 2024-08-31T06:17:38.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.807+0000 7faecb64c740 1 -- start start 2024-08-31T06:17:38.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.807+0000 7faecb64c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589235dc070 0x558923981990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:38.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.807+0000 7faecb64c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589235b31b0 0x558923981ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:38.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.807+0000 7faecb64c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589235bb390 0x558923982410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:38.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.808+0000 7faecb64c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5589238f87b0 con 0x5589235bb390 2024-08-31T06:17:38.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.808+0000 7faecb64c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5589238e7d40 con 0x5589235b31b0 2024-08-31T06:17:38.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.808+0000 7faeb6a00640 1 -- 172.21.15.16:0/1330746233 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7faeb8010bd0 con 0x7faec006bca0 2024-08-31T06:17:38.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.808+0000 7faecb64c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5589238ebcf0 con 0x5589235dc070 2024-08-31T06:17:38.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.808+0000 7faebe800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589235dc070 0x558923981990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:38.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.808+0000 7faebd400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589235bb390 0x558923982410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:38.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.808+0000 7faebd400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589235bb390 0x558923982410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39986/0 (socket says 172.21.15.16:39986) 2024-08-31T06:17:38.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.808+0000 7faebd400640 1 -- 172.21.15.16:0/3642226702 learned_addr learned my addr 172.21.15.16:0/3642226702 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:38.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.808+0000 7faebde00640 1 --2- 172.21.15.16:0/3642226702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589235b31b0 0x558923981ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:38.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.808+0000 7faebe800640 1 -- 172.21.15.16:0/3642226702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589235b31b0 msgr2=0x558923981ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:38.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.808+0000 7faebe800640 1 --2- 172.21.15.16:0/3642226702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589235b31b0 0x558923981ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:38.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.808+0000 7faebe800640 1 -- 172.21.15.16:0/3642226702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589235bb390 msgr2=0x558923982410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:38.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.808+0000 7faebe800640 1 --2- 172.21.15.16:0/3642226702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589235bb390 0x558923982410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:38.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.808+0000 7faebe800640 1 -- 172.21.15.16:0/3642226702 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558923647650 con 0x5589235dc070 2024-08-31T06:17:38.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.809+0000 7faebe800640 1 --2- 172.21.15.16:0/3642226702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589235dc070 0x558923981990 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7faeb0011ef0 tx=0x7faeb000b770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc6855d47a2baf server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:38.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.809+0000 7faeab000640 1 -- 172.21.15.16:0/3642226702 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb001f070 con 0x5589235dc070 2024-08-31T06:17:38.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.809+0000 7faeab000640 1 -- 172.21.15.16:0/3642226702 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faeb001a8c0 con 0x5589235dc070 2024-08-31T06:17:38.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.809+0000 7faeab000640 1 -- 172.21.15.16:0/3642226702 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb0019940 con 0x5589235dc070 2024-08-31T06:17:38.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.810+0000 7faeb6a00640 1 --2- 172.21.15.16:0/1330746233 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7faea40b4d20 0x7faea40b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:38.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.810+0000 7faeb6a00640 1 -- 172.21.15.16:0/1330746233 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8470+0+0 (secure 0 0 0) 0x7faeb80e5bb0 con 0x7faec006bca0 2024-08-31T06:17:38.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.810+0000 7faebde00640 1 --2- 172.21.15.16:0/1330746233 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7faea40b4d20 0x7faea40b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:38.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.811+0000 7faecb64c740 1 -- 172.21.15.16:0/3642226702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589235dc070 msgr2=0x558923981990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:38.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.811+0000 7faecb64c740 1 --2- 172.21.15.16:0/3642226702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589235dc070 0x558923981990 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7faeb0011ef0 tx=0x7faeb000b770 comp rx=0 tx=0).stop 2024-08-31T06:17:38.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.811+0000 7faebde00640 1 --2- 172.21.15.16:0/1330746233 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7faea40b4d20 0x7faea40b7230 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x558923980610 tx=0x7faeb80e9660 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:38.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.811+0000 7faecb64c740 1 -- 172.21.15.16:0/3642226702 shutdown_connections 2024-08-31T06:17:38.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.811+0000 7faecb64c740 1 --2- 172.21.15.16:0/3642226702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589235bb390 0x558923982410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:38.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.811+0000 7faecb64c740 1 --2- 172.21.15.16:0/3642226702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589235b31b0 0x558923981ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:38.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.811+0000 7faecb64c740 1 --2- 172.21.15.16:0/3642226702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589235dc070 0x558923981990 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:38.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.811+0000 7faecb64c740 1 -- 172.21.15.16:0/3642226702 >> 172.21.15.16:0/3642226702 conn(0x5589235d0460 msgr2=0x55892397f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:38.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.811+0000 7faecb64c740 1 -- 172.21.15.16:0/3642226702 shutdown_connections 2024-08-31T06:17:38.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.811+0000 7faecb64c740 1 -- 172.21.15.16:0/3642226702 wait complete. 2024-08-31T06:17:38.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.812+0000 7faecb64c740 1 -- start start 2024-08-31T06:17:38.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.812+0000 7faecb64c740 1 Processor -- start 2024-08-31T06:17:38.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.813+0000 7faecb64c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589235b31b0 0x55892398d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:38.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.813+0000 7faecb64c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589235bb390 0x55892398db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:38.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.813+0000 7faecb64c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589235dc070 0x558923999460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:38.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.813+0000 7faecb64c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55892374a700 con 0x5589235b31b0 2024-08-31T06:17:38.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.813+0000 7faecb64c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55892386ecf0 con 0x5589235bb390 2024-08-31T06:17:38.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.813+0000 7faecb64c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558923990170 con 0x5589235dc070 2024-08-31T06:17:38.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.813+0000 7faecb64c740 10 client.0 _ll_register_callbacks cb 0x558923894470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:17:38.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.814+0000 7faebe800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589235b31b0 0x55892398d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:38.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.814+0000 7faebd400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589235bb390 0x55892398db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:38.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.814+0000 7faebd400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589235bb390 0x55892398db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:38136/0 (socket says 172.21.15.16:38136) 2024-08-31T06:17:38.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.814+0000 7faebd400640 1 -- 172.21.15.16:0/3017745615 learned_addr learned my addr 172.21.15.16:0/3017745615 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:38.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.814+0000 7faebe800640 1 --2- 172.21.15.16:0/3017745615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589235dc070 0x558923999460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:38.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.815+0000 7faebe800640 1 -- 172.21.15.16:0/3017745615 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589235bb390 msgr2=0x55892398db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:38.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.815+0000 7faebe800640 1 --2- 172.21.15.16:0/3017745615 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589235bb390 0x55892398db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:38.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.815+0000 7faebe800640 1 -- 172.21.15.16:0/3017745615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589235b31b0 msgr2=0x55892398d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:38.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.815+0000 7faebe800640 1 --2- 172.21.15.16:0/3017745615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589235b31b0 0x55892398d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:38.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.815+0000 7faebe800640 1 -- 172.21.15.16:0/3017745615 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589238af8b0 con 0x5589235dc070 2024-08-31T06:17:38.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.815+0000 7faebd400640 1 --2- 172.21.15.16:0/3017745615 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589235bb390 0x55892398db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:17:38.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.815+0000 7faebe800640 1 --2- 172.21.15.16:0/3017745615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589235dc070 0x558923999460 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7faeb001c000 tx=0x7faeb00118a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4abb80f28d30b0da server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:38.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.815+0000 7fae9f400640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:17:38.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.816+0000 7fae9f400640 1 -- 172.21.15.16:0/3017745615 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb002d060 con 0x5589235dc070 2024-08-31T06:17:38.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.816+0000 7fae9f400640 1 -- 172.21.15.16:0/3017745615 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8470+0+0 (secure 0 0 0) 0x7faeb0011040 con 0x5589235dc070 2024-08-31T06:17:38.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.816+0000 7faecb64c740 1 -- 172.21.15.16:0/3017745615 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faeb001a8c0 con 0x5589235dc070 2024-08-31T06:17:38.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.816+0000 7fae9f400640 1 -- 172.21.15.16:0/3017745615 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7faeb001f070 con 0x5589235dc070 2024-08-31T06:17:38.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.816+0000 7faecb64c740 10 client.27922 fetch_fsmap learned FSMap version 46 2024-08-31T06:17:38.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.816+0000 7faecb64c740 1 -- 172.21.15.16:0/3017745615 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55892399c200 con 0x5589235dc070 2024-08-31T06:17:38.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.817+0000 7fae9f400640 1 -- 172.21.15.16:0/3017745615 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faeb001a8c0 con 0x5589235dc070 2024-08-31T06:17:38.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.817+0000 7fae9f400640 1 -- 172.21.15.16:0/3017745615 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7faeb0025070 con 0x5589235dc070 2024-08-31T06:17:38.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.817+0000 7faecb64c740 10 client.27922 fetch_fsmap finished waiting for FSMap version 46 2024-08-31T06:17:38.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.817+0000 7faecb64c740 10 client.27922 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:17:38.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.817+0000 7faecb64c740 20 client.27922 populate_metadata read hostname 'smithi016' 2024-08-31T06:17:38.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.817+0000 7faecb64c740 1 --2- 172.21.15.16:0/3017745615 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55892359a9f0 0x55892399f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:38.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.818+0000 7faecb64c740 4 client.27922 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:17:38.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.818+0000 7faecb64c740 1 -- 172.21.15.16:0/3017745615 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589238fa700 con 0x55892359a9f0 2024-08-31T06:17:38.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.818+0000 7faebd400640 1 --2- 172.21.15.16:0/3017745615 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55892359a9f0 0x55892399f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:38.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.819+0000 7fae9f400640 10 client.27922 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:17:38.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.820+0000 7faebd400640 1 --2- 172.21.15.16:0/3017745615 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55892359a9f0 0x55892399f620 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8cec42afa2a244f server_cookie=238f5e0aa491364f in_seq=0 out_seq=0 2024-08-31T06:17:38.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.821+0000 7fae9f400640 1 -- 172.21.15.16:0/3017745615 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5589238fa700 con 0x55892359a9f0 2024-08-31T06:17:38.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.821+0000 7fae9f400640 10 client.27922 handle_command_reply: tid=0 2024-08-31T06:17:38.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.821+0000 7faecb64c740 1 client.27922 shutdown 2024-08-31T06:17:38.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.821+0000 7faecb64c740 1 -- 172.21.15.16:0/3017745615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589235dc070 msgr2=0x558923999460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:38.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.821+0000 7faecb64c740 1 --2- 172.21.15.16:0/3017745615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589235dc070 0x558923999460 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7faeb001c000 tx=0x7faeb00118a0 comp rx=0 tx=0).stop 2024-08-31T06:17:38.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.822+0000 7faecb64c740 1 -- 172.21.15.16:0/3017745615 shutdown_connections 2024-08-31T06:17:38.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.822+0000 7faecb64c740 1 --2- 172.21.15.16:0/3017745615 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55892359a9f0 0x55892399f620 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:38.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.822+0000 7faecb64c740 1 --2- 172.21.15.16:0/3017745615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589235dc070 0x558923999460 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:38.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.822+0000 7faecb64c740 1 --2- 172.21.15.16:0/3017745615 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589235bb390 0x55892398db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:38.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.822+0000 7faecb64c740 1 --2- 172.21.15.16:0/3017745615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589235b31b0 0x55892398d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:38.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.822+0000 7faecb64c740 1 -- 172.21.15.16:0/3017745615 >> 172.21.15.16:0/3017745615 conn(0x5589235d0460 msgr2=0x558923984b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:38.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.822+0000 7fae9f400640 0 client.27922 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:17:38.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.823+0000 7faecb64c740 1 -- 172.21.15.16:0/3017745615 shutdown_connections 2024-08-31T06:17:38.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.823+0000 7faecb64c740 1 -- 172.21.15.16:0/3017745615 wait complete. 2024-08-31T06:17:38.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.824+0000 7faecb64c740 20 client.27922 trim_cache size 0 max 16384 2024-08-31T06:17:38.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.848+0000 7faecb64c740 1 Processor -- start 2024-08-31T06:17:38.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.848+0000 7faecb64c740 1 -- start start 2024-08-31T06:17:38.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.849+0000 7faecb64c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589235b31b0 0x558923999360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:38.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.849+0000 7faecb64c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589235bb390 0x558923986aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:38.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.849+0000 7faecb64c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589235dc070 0x558923986fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:38.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.849+0000 7faecb64c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faec01b8420 con 0x5589235bb390 2024-08-31T06:17:38.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.849+0000 7faecb64c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faec01b5c60 con 0x5589235dc070 2024-08-31T06:17:38.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.849+0000 7faecb64c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faec01b3a90 con 0x5589235b31b0 2024-08-31T06:17:38.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.849+0000 7faebe800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589235dc070 0x558923986fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:38.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.850+0000 7faebd400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589235bb390 0x558923986aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:38.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.849+0000 7faebe800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589235dc070 0x558923986fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:38140/0 (socket says 172.21.15.16:38140) 2024-08-31T06:17:38.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.850+0000 7faebe800640 1 -- 172.21.15.16:0/3082683668 learned_addr learned my addr 172.21.15.16:0/3082683668 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:38.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.850+0000 7faebe800640 1 --2- 172.21.15.16:0/3082683668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589235b31b0 0x558923999360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:38.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.850+0000 7faebe800640 1 -- 172.21.15.16:0/3082683668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589235b31b0 msgr2=0x558923999360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:38.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.850+0000 7faebe800640 1 --2- 172.21.15.16:0/3082683668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589235b31b0 0x558923999360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:38.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.850+0000 7faebe800640 1 -- 172.21.15.16:0/3082683668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589235bb390 msgr2=0x558923986aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:38.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.850+0000 7faebe800640 1 --2- 172.21.15.16:0/3082683668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589235bb390 0x558923986aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:38.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.850+0000 7faebe800640 1 -- 172.21.15.16:0/3082683668 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589239847f0 con 0x5589235dc070 2024-08-31T06:17:38.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.850+0000 7faebe800640 1 --2- 172.21.15.16:0/3082683668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589235dc070 0x558923986fe0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7faeb0001eb0 tx=0x7faeb0005970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f211b5d4092dd5b8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:38.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.851+0000 7faeaa600640 1 -- 172.21.15.16:0/3082683668 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb0037080 con 0x5589235dc070 2024-08-31T06:17:38.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.851+0000 7faeaa600640 1 -- 172.21.15.16:0/3082683668 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faeb00194f0 con 0x5589235dc070 2024-08-31T06:17:38.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.851+0000 7faeaa600640 1 -- 172.21.15.16:0/3082683668 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb001f070 con 0x5589235dc070 2024-08-31T06:17:38.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.852+0000 7faecb64c740 1 -- 172.21.15.16:0/3082683668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589235dc070 msgr2=0x558923986fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:38.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.852+0000 7faecb64c740 1 --2- 172.21.15.16:0/3082683668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589235dc070 0x558923986fe0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7faeb0001eb0 tx=0x7faeb0005970 comp rx=0 tx=0).stop 2024-08-31T06:17:38.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.853+0000 7faecb64c740 1 -- 172.21.15.16:0/3082683668 shutdown_connections 2024-08-31T06:17:38.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.853+0000 7faecb64c740 1 --2- 172.21.15.16:0/3082683668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589235dc070 0x558923986fe0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:38.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.853+0000 7faecb64c740 1 --2- 172.21.15.16:0/3082683668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589235bb390 0x558923986aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:38.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.853+0000 7faecb64c740 1 --2- 172.21.15.16:0/3082683668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589235b31b0 0x558923999360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:38.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.853+0000 7faecb64c740 1 -- 172.21.15.16:0/3082683668 >> 172.21.15.16:0/3082683668 conn(0x5589235d0460 msgr2=0x55892398bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:38.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.853+0000 7faecb64c740 1 -- 172.21.15.16:0/3082683668 shutdown_connections 2024-08-31T06:17:38.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.853+0000 7faecb64c740 1 -- 172.21.15.16:0/3082683668 wait complete. 2024-08-31T06:17:38.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.854+0000 7faecb64c740 1 -- start start 2024-08-31T06:17:38.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.854+0000 7faecb64c740 1 Processor -- start 2024-08-31T06:17:38.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.855+0000 7faecb64c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589235b31b0 0x558923991390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:38.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.855+0000 7faecb64c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589235bb390 0x5589239918d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:38.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.855+0000 7faecb64c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589235dc070 0x558923991e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:38.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.855+0000 7faecb64c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faec01b4120 con 0x5589235dc070 2024-08-31T06:17:38.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.855+0000 7faecb64c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faec01b5590 con 0x5589235b31b0 2024-08-31T06:17:38.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.855+0000 7faecb64c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faec01b47f0 con 0x5589235bb390 2024-08-31T06:17:38.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.855+0000 7faecb64c740 10 client.0 _ll_register_callbacks cb 0x7faeb80e9530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:17:38.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.855+0000 7faebe800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589235dc070 0x558923991e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:38.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.855+0000 7faebe800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589235b31b0 0x558923991390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:38.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.855+0000 7faebe800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589235dc070 0x558923991e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40002/0 (socket says 172.21.15.16:40002) 2024-08-31T06:17:38.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.855+0000 7faebe800640 1 -- 172.21.15.16:0/4021911096 learned_addr learned my addr 172.21.15.16:0/4021911096 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:38.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.856+0000 7faebd400640 1 --2- 172.21.15.16:0/4021911096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589235bb390 0x5589239918d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:38.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.856+0000 7faebd400640 1 -- 172.21.15.16:0/4021911096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589235b31b0 msgr2=0x558923991390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:38.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.856+0000 7faebd400640 1 --2- 172.21.15.16:0/4021911096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589235b31b0 0x558923991390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:38.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.856+0000 7faebd400640 1 -- 172.21.15.16:0/4021911096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589235dc070 msgr2=0x558923991e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:38.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.857+0000 7faebd400640 1 --2- 172.21.15.16:0/4021911096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589235dc070 0x558923991e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:38.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.857+0000 7faebd400640 1 -- 172.21.15.16:0/4021911096 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558923a11600 con 0x5589235bb390 2024-08-31T06:17:38.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.857+0000 7faebe800640 1 --2- 172.21.15.16:0/4021911096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589235dc070 0x558923991e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:17:38.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.857+0000 7faebd400640 1 --2- 172.21.15.16:0/4021911096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589235bb390 0x5589239918d0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7faeac006010 tx=0x7faeac02a990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd32fca7480a94c6 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:38.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.857+0000 7fae9f400640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:17:38.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.858+0000 7fae9f400640 1 -- 172.21.15.16:0/4021911096 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeac035040 con 0x5589235bb390 2024-08-31T06:17:38.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.858+0000 7faecb64c740 1 -- 172.21.15.16:0/4021911096 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faeb00194f0 con 0x5589235bb390 2024-08-31T06:17:38.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.858+0000 7fae9f400640 1 -- 172.21.15.16:0/4021911096 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8470+0+0 (secure 0 0 0) 0x7faeac005040 con 0x5589235bb390 2024-08-31T06:17:38.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.859+0000 7fae9f400640 1 -- 172.21.15.16:0/4021911096 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7faeac005240 con 0x5589235bb390 2024-08-31T06:17:38.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.862+0000 7faecb64c740 10 client.27928 fetch_fsmap learned FSMap version 46 2024-08-31T06:17:38.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.862+0000 7faecb64c740 1 -- 172.21.15.16:0/4021911096 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x558923a128d0 con 0x5589235bb390 2024-08-31T06:17:38.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.863+0000 7fae9f400640 1 -- 172.21.15.16:0/4021911096 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faeb00194f0 con 0x5589235bb390 2024-08-31T06:17:38.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.863+0000 7fae9f400640 1 -- 172.21.15.16:0/4021911096 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7faeac004750 con 0x5589235bb390 2024-08-31T06:17:38.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.867+0000 7faecb64c740 10 client.27928 fetch_fsmap finished waiting for FSMap version 46 2024-08-31T06:17:38.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.867+0000 7faecb64c740 10 client.27928 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:17:38.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.867+0000 7faecb64c740 20 client.27928 populate_metadata read hostname 'smithi016' 2024-08-31T06:17:38.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.867+0000 7faecb64c740 1 --2- 172.21.15.16:0/4021911096 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55892356fda0 0x558923a15050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:38.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.867+0000 7faecb64c740 4 client.27928 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:17:38.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.867+0000 7faecb64c740 1 -- 172.21.15.16:0/4021911096 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5589238f61a0 con 0x55892356fda0 2024-08-31T06:17:38.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.867+0000 7faebe800640 1 --2- 172.21.15.16:0/4021911096 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55892356fda0 0x558923a15050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:38.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.868+0000 7fae9f400640 10 client.27928 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:17:38.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.868+0000 7faebe800640 1 --2- 172.21.15.16:0/4021911096 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55892356fda0 0x558923a15050 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ede9d394ece02e3f server_cookie=5ccd7f42fb877828 in_seq=0 out_seq=0 2024-08-31T06:17:38.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.869+0000 7fae9f400640 1 -- 172.21.15.16:0/4021911096 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5589238f61a0 con 0x55892356fda0 2024-08-31T06:17:38.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.869+0000 7fae9f400640 10 client.27928 handle_command_reply: tid=0 2024-08-31T06:17:38.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.869+0000 7faecb64c740 1 client.27928 shutdown 2024-08-31T06:17:38.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.870+0000 7faecb64c740 1 -- 172.21.15.16:0/4021911096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589235bb390 msgr2=0x5589239918d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:38.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.870+0000 7faecb64c740 1 --2- 172.21.15.16:0/4021911096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589235bb390 0x5589239918d0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7faeac006010 tx=0x7faeac02a990 comp rx=0 tx=0).stop 2024-08-31T06:17:38.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.870+0000 7faecb64c740 1 -- 172.21.15.16:0/4021911096 shutdown_connections 2024-08-31T06:17:38.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.870+0000 7faecb64c740 1 --2- 172.21.15.16:0/4021911096 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55892356fda0 0x558923a15050 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:38.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.870+0000 7faecb64c740 1 --2- 172.21.15.16:0/4021911096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589235bb390 0x5589239918d0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:38.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.870+0000 7faecb64c740 1 --2- 172.21.15.16:0/4021911096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589235dc070 0x558923991e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:38.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.870+0000 7faecb64c740 1 --2- 172.21.15.16:0/4021911096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589235b31b0 0x558923991390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:38.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.870+0000 7faecb64c740 1 -- 172.21.15.16:0/4021911096 >> 172.21.15.16:0/4021911096 conn(0x5589235d0460 msgr2=0x558923986aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:38.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.870+0000 7fae9f400640 0 client.27928 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:17:38.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.870+0000 7faecb64c740 1 -- 172.21.15.16:0/4021911096 shutdown_connections 2024-08-31T06:17:38.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.871+0000 7faecb64c740 1 -- 172.21.15.16:0/4021911096 wait complete. 2024-08-31T06:17:38.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.871+0000 7faecb64c740 20 client.27928 trim_cache size 0 max 16384 2024-08-31T06:17:38.874 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:17:38.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.872+0000 7faec4a00640 1 -- 172.21.15.16:0/1330746233 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7faea40b4d20 msgr2=0x7faea40b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:38.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.872+0000 7faec4a00640 1 --2- 172.21.15.16:0/1330746233 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7faea40b4d20 0x7faea40b7230 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x558923980610 tx=0x7faeb80e9660 comp rx=0 tx=0).stop 2024-08-31T06:17:38.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.872+0000 7faec4a00640 1 -- 172.21.15.16:0/1330746233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faec006bca0 msgr2=0x7faec01a2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:38.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.872+0000 7faec4a00640 1 --2- 172.21.15.16:0/1330746233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faec006bca0 0x7faec01a2a50 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7faeb800eab0 tx=0x7faeb800ef70 comp rx=0 tx=0).stop 2024-08-31T06:17:38.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.873+0000 7faec4a00640 1 -- 172.21.15.16:0/1330746233 shutdown_connections 2024-08-31T06:17:38.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.873+0000 7faec4a00640 1 --2- 172.21.15.16:0/1330746233 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7faea40b4d20 0x7faea40b7230 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:38.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.873+0000 7faec4a00640 1 --2- 172.21.15.16:0/1330746233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faec010f170 0x7faec01a8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:38.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.873+0000 7faec4a00640 1 --2- 172.21.15.16:0/1330746233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faec010e820 0x7faec01a2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:38.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.873+0000 7faec4a00640 1 --2- 172.21.15.16:0/1330746233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faec006bca0 0x7faec01a2a50 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:38.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.873+0000 7faec4a00640 1 -- 172.21.15.16:0/1330746233 >> 172.21.15.16:0/1330746233 conn(0x7faec006e6d0 msgr2=0x7faec010a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:38.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.873+0000 7faec4a00640 1 -- 172.21.15.16:0/1330746233 shutdown_connections 2024-08-31T06:17:38.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:38.874+0000 7faec4a00640 1 -- 172.21.15.16:0/1330746233 wait complete. 2024-08-31T06:17:38.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:38 smithi184 ceph-mon[22331]: pgmap v686: 97 pgs: 97 active+clean; 3.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 20 MiB/s wr, 1.29k op/s 2024-08-31T06:17:39.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:38 smithi155 ceph-mon[21742]: pgmap v686: 97 pgs: 97 active+clean; 3.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 20 MiB/s wr, 1.29k op/s 2024-08-31T06:17:39.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:39 smithi184 ceph-mon[22331]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:17:40.022 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:39 smithi155 ceph-mon[21742]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:17:40.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:39 smithi016 ceph-mon[17194]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:17:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.226+0000 7f5a07200640 1 -- 172.21.15.16:0/3699118292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a08102150 msgr2=0x7f5a08102560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.226+0000 7f5a07200640 1 --2- 172.21.15.16:0/3699118292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a08102150 0x7f5a08102560 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f59f8009900 tx=0x7f59f802a410 comp rx=0 tx=0).stop 2024-08-31T06:17:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.227+0000 7f5a07200640 1 -- 172.21.15.16:0/3699118292 shutdown_connections 2024-08-31T06:17:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.227+0000 7f5a07200640 1 --2- 172.21.15.16:0/3699118292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a0810fd00 0x7f5a08112120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.227+0000 7f5a07200640 1 --2- 172.21.15.16:0/3699118292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a08102aa0 0x7f5a0810f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.227+0000 7f5a07200640 1 --2- 172.21.15.16:0/3699118292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a08102150 0x7f5a08102560 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.227+0000 7f5a07200640 1 -- 172.21.15.16:0/3699118292 >> 172.21.15.16:0/3699118292 conn(0x7f5a080fdfd0 msgr2=0x7f5a08100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.227+0000 7f5a07200640 1 -- 172.21.15.16:0/3699118292 shutdown_connections 2024-08-31T06:17:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.227+0000 7f5a07200640 1 -- 172.21.15.16:0/3699118292 wait complete. 2024-08-31T06:17:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.229+0000 7f5a07200640 1 Processor -- start 2024-08-31T06:17:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.229+0000 7f5a07200640 1 -- start start 2024-08-31T06:17:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.229+0000 7f5a07200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a08102150 0x7f5a0819c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.229+0000 7f5a07200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a08102aa0 0x7f5a0819c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.230+0000 7f5a07200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a0810fd00 0x7f5a081a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.230+0000 7f5a07200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5a08115820 con 0x7f5a08102aa0 2024-08-31T06:17:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.230+0000 7f5a07200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5a081156a0 con 0x7f5a0810fd00 2024-08-31T06:17:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.230+0000 7f5a07200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5a081159a0 con 0x7f5a08102150 2024-08-31T06:17:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.230+0000 7f5a05e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a08102150 0x7f5a0819c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.230+0000 7f5a05e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a08102150 0x7f5a0819c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:32782/0 (socket says 172.21.15.16:32782) 2024-08-31T06:17:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.230+0000 7f5a05e00640 1 -- 172.21.15.16:0/3907042481 learned_addr learned my addr 172.21.15.16:0/3907042481 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.231+0000 7f59ffe00640 1 --2- 172.21.15.16:0/3907042481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a08102aa0 0x7f5a0819c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.231+0000 7f5a05e00640 1 -- 172.21.15.16:0/3907042481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a0810fd00 msgr2=0x7f5a081a1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.231+0000 7f5a05e00640 1 --2- 172.21.15.16:0/3907042481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a0810fd00 0x7f5a081a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.231+0000 7f5a05e00640 1 -- 172.21.15.16:0/3907042481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a08102aa0 msgr2=0x7f5a0819c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.231+0000 7f5a05e00640 1 --2- 172.21.15.16:0/3907042481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a08102aa0 0x7f5a0819c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.231+0000 7f5a05e00640 1 -- 172.21.15.16:0/3907042481 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a081a2170 con 0x7f5a08102150 2024-08-31T06:17:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.232+0000 7f5a05e00640 1 --2- 172.21.15.16:0/3907042481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a08102150 0x7f5a0819c240 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f59f8031440 tx=0x7f59f8003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.233+0000 7f59fea00640 1 -- 172.21.15.16:0/3907042481 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f803f070 con 0x7f5a08102150 2024-08-31T06:17:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.233+0000 7f5a07200640 1 -- 172.21.15.16:0/3907042481 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a081a2400 con 0x7f5a08102150 2024-08-31T06:17:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.233+0000 7f5a07200640 1 -- 172.21.15.16:0/3907042481 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a081a8bf0 con 0x7f5a08102150 2024-08-31T06:17:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.233+0000 7f59fea00640 1 -- 172.21.15.16:0/3907042481 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59f8031b50 con 0x7f5a08102150 2024-08-31T06:17:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.234+0000 7f59fea00640 1 -- 172.21.15.16:0/3907042481 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f8035030 con 0x7f5a08102150 2024-08-31T06:17:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.238+0000 7f5a13e7a740 1 Processor -- start 2024-08-31T06:17:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.238+0000 7f5a13e7a740 1 -- start start 2024-08-31T06:17:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.239+0000 7f5a13e7a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd6c17a1b0 0x55dd6c51db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.239+0000 7f5a13e7a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd6c1512f0 0x55dd6c51e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.239+0000 7f5a13e7a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd6c1594d0 0x55dd6c51e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.239+0000 7f5a13e7a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dd6c492f70 con 0x55dd6c1594d0 2024-08-31T06:17:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.239+0000 7f5a13e7a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55dd6c482500 con 0x55dd6c1512f0 2024-08-31T06:17:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.239+0000 7f5a13e7a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55dd6c4864b0 con 0x55dd6c17a1b0 2024-08-31T06:17:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.239+0000 7f59fea00640 1 -- 172.21.15.16:0/3907042481 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f59f8042070 con 0x7f5a08102150 2024-08-31T06:17:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.240+0000 7f5a06800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd6c17a1b0 0x55dd6c51db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.240+0000 7f5a06800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd6c17a1b0 0x55dd6c51db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:32794/0 (socket says 172.21.15.16:32794) 2024-08-31T06:17:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.240+0000 7f5a06800640 1 -- 172.21.15.16:0/1124534728 learned_addr learned my addr 172.21.15.16:0/1124534728 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.240+0000 7f5a05e00640 1 --2- 172.21.15.16:0/1124534728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd6c1512f0 0x55dd6c51e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.240+0000 7f59ffe00640 1 --2- 172.21.15.16:0/1124534728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd6c1594d0 0x55dd6c51e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.241+0000 7f5a06800640 1 -- 172.21.15.16:0/1124534728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd6c1512f0 msgr2=0x55dd6c51e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.241+0000 7f5a06800640 1 --2- 172.21.15.16:0/1124534728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd6c1512f0 0x55dd6c51e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.241+0000 7f5a06800640 1 -- 172.21.15.16:0/1124534728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd6c1594d0 msgr2=0x55dd6c51e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.241+0000 7f5a06800640 1 --2- 172.21.15.16:0/1124534728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd6c1594d0 0x55dd6c51e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.241+0000 7f5a06800640 1 -- 172.21.15.16:0/1124534728 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd6c43cc10 con 0x55dd6c17a1b0 2024-08-31T06:17:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.241+0000 7f59ffe00640 1 --2- 172.21.15.16:0/1124534728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd6c1594d0 0x55dd6c51e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:17:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.241+0000 7f5a06800640 1 --2- 172.21.15.16:0/1124534728 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd6c17a1b0 0x55dd6c51db00 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5a00009ed0 tx=0x7f5a0000e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87f5a3c47fd13b4e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.242+0000 7f59f7400640 1 -- 172.21.15.16:0/1124534728 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a00003b70 con 0x55dd6c17a1b0 2024-08-31T06:17:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.242+0000 7f59f7400640 1 -- 172.21.15.16:0/1124534728 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a00003d10 con 0x55dd6c17a1b0 2024-08-31T06:17:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.242+0000 7f59f7400640 1 -- 172.21.15.16:0/1124534728 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a000108d0 con 0x55dd6c17a1b0 2024-08-31T06:17:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.243+0000 7f5a13e7a740 1 -- 172.21.15.16:0/1124534728 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd6c17a1b0 msgr2=0x55dd6c51db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.243+0000 7f5a13e7a740 1 --2- 172.21.15.16:0/1124534728 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd6c17a1b0 0x55dd6c51db00 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5a00009ed0 tx=0x7f5a0000e720 comp rx=0 tx=0).stop 2024-08-31T06:17:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.243+0000 7f59fea00640 1 --2- 172.21.15.16:0/3907042481 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f59ec0b4d50 0x7f59ec0b7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.243+0000 7f5a13e7a740 1 -- 172.21.15.16:0/1124534728 shutdown_connections 2024-08-31T06:17:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.243+0000 7f5a13e7a740 1 --2- 172.21.15.16:0/1124534728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd6c1594d0 0x55dd6c51e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.243+0000 7f5a13e7a740 1 --2- 172.21.15.16:0/1124534728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd6c1512f0 0x55dd6c51e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.243+0000 7f5a13e7a740 1 --2- 172.21.15.16:0/1124534728 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd6c17a1b0 0x55dd6c51db00 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.243+0000 7f59fea00640 1 -- 172.21.15.16:0/3907042481 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8470+0+0 (secure 0 0 0) 0x7f59f8105810 con 0x7f5a08102150 2024-08-31T06:17:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.243+0000 7f5a13e7a740 1 -- 172.21.15.16:0/1124534728 >> 172.21.15.16:0/1124534728 conn(0x55dd6c16e300 msgr2=0x55dd6c51b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.243+0000 7f5a05e00640 1 --2- 172.21.15.16:0/3907042481 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f59ec0b4d50 0x7f59ec0b7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.244+0000 7f5a13e7a740 1 -- 172.21.15.16:0/1124534728 shutdown_connections 2024-08-31T06:17:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.244+0000 7f5a13e7a740 1 -- 172.21.15.16:0/1124534728 wait complete. 2024-08-31T06:17:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.244+0000 7f5a05e00640 1 --2- 172.21.15.16:0/3907042481 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f59ec0b4d50 0x7f59ec0b7260 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x55dd6c51c780 tx=0x7f59f8000bc0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.245+0000 7f5a13e7a740 1 -- start start 2024-08-31T06:17:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.245+0000 7f5a13e7a740 1 Processor -- start 2024-08-31T06:17:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.247+0000 7f5a13e7a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd6c1512f0 0x55dd6c5290f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.247+0000 7f5a13e7a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd6c1594d0 0x55dd6c529630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.247+0000 7f5a13e7a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd6c17a1b0 0x55dd6c529b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.247+0000 7f5a13e7a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dd6c3bbab0 con 0x55dd6c1594d0 2024-08-31T06:17:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.247+0000 7f5a13e7a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55dd6c429320 con 0x55dd6c1512f0 2024-08-31T06:17:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.247+0000 7f5a13e7a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55dd6c534a90 con 0x55dd6c17a1b0 2024-08-31T06:17:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.247+0000 7f5a13e7a740 10 client.0 _ll_register_callbacks cb 0x55dd6c42f6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:17:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.247+0000 7f5a06800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd6c1512f0 0x55dd6c5290f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.247+0000 7f5a06800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd6c1512f0 0x55dd6c5290f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:38186/0 (socket says 172.21.15.16:38186) 2024-08-31T06:17:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.247+0000 7f5a06800640 1 -- 172.21.15.16:0/2607048839 learned_addr learned my addr 172.21.15.16:0/2607048839 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.248+0000 7f59ffe00640 1 --2- 172.21.15.16:0/2607048839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd6c1594d0 0x55dd6c529630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.248+0000 7f5a06800640 1 --2- 172.21.15.16:0/2607048839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd6c17a1b0 0x55dd6c529b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.248+0000 7f5a06800640 1 -- 172.21.15.16:0/2607048839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd6c17a1b0 msgr2=0x55dd6c529b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.248+0000 7f5a06800640 1 --2- 172.21.15.16:0/2607048839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd6c17a1b0 0x55dd6c529b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.248+0000 7f5a06800640 1 -- 172.21.15.16:0/2607048839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd6c1594d0 msgr2=0x55dd6c529630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.248+0000 7f5a06800640 1 --2- 172.21.15.16:0/2607048839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd6c1594d0 0x55dd6c529630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.248+0000 7f5a06800640 1 -- 172.21.15.16:0/2607048839 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd6c1e5790 con 0x55dd6c1512f0 2024-08-31T06:17:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.248+0000 7f59ffe00640 1 --2- 172.21.15.16:0/2607048839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd6c1594d0 0x55dd6c529630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:17:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.249+0000 7f5a06800640 1 --2- 172.21.15.16:0/2607048839 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd6c1512f0 0x55dd6c5290f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5a00004a80 tx=0x7f5a00010570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9d547f4db7bf7a1 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.249+0000 7f59ebe00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:17:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.249+0000 7f59ebe00640 1 -- 172.21.15.16:0/2607048839 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0001d040 con 0x55dd6c1512f0 2024-08-31T06:17:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.249+0000 7f5a13e7a740 1 -- 172.21.15.16:0/2607048839 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a00003d10 con 0x55dd6c1512f0 2024-08-31T06:17:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.249+0000 7f59ebe00640 1 -- 172.21.15.16:0/2607048839 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8470+0+0 (secure 0 0 0) 0x7f5a00018430 con 0x55dd6c1512f0 2024-08-31T06:17:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.249+0000 7f59ebe00640 1 -- 172.21.15.16:0/2607048839 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f5a00010650 con 0x55dd6c1512f0 2024-08-31T06:17:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.249+0000 7f5a13e7a740 10 client.27935 fetch_fsmap learned FSMap version 46 2024-08-31T06:17:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.249+0000 7f5a13e7a740 1 -- 172.21.15.16:0/2607048839 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55dd6c25e620 con 0x55dd6c1512f0 2024-08-31T06:17:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.249+0000 7f59ebe00640 1 -- 172.21.15.16:0/2607048839 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a00003d10 con 0x55dd6c1512f0 2024-08-31T06:17:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.250+0000 7f59ebe00640 1 -- 172.21.15.16:0/2607048839 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7f5a00018a40 con 0x55dd6c1512f0 2024-08-31T06:17:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.250+0000 7f5a13e7a740 10 client.27935 fetch_fsmap finished waiting for FSMap version 46 2024-08-31T06:17:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.250+0000 7f5a13e7a740 10 client.27935 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:17:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.250+0000 7f5a13e7a740 20 client.27935 populate_metadata read hostname 'smithi016' 2024-08-31T06:17:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.250+0000 7f5a13e7a740 1 --2- 172.21.15.16:0/2607048839 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55dd6c138d80 0x55dd6c543930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.250+0000 7f5a13e7a740 4 client.27935 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:17:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.250+0000 7f5a13e7a740 1 -- 172.21.15.16:0/2607048839 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd6c494ec0 con 0x55dd6c138d80 2024-08-31T06:17:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.250+0000 7f59ffe00640 1 --2- 172.21.15.16:0/2607048839 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55dd6c138d80 0x55dd6c543930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.252+0000 7f59ffe00640 1 --2- 172.21.15.16:0/2607048839 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55dd6c138d80 0x55dd6c543930 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90dc83f73747ea88 server_cookie=dc258807d778dc9e in_seq=0 out_seq=0 2024-08-31T06:17:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.252+0000 7f59ebe00640 10 client.27935 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:17:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.253+0000 7f59ebe00640 1 -- 172.21.15.16:0/2607048839 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dd6c494ec0 con 0x55dd6c138d80 2024-08-31T06:17:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.253+0000 7f59ebe00640 10 client.27935 handle_command_reply: tid=0 2024-08-31T06:17:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.253+0000 7f5a13e7a740 1 client.27935 shutdown 2024-08-31T06:17:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.253+0000 7f5a13e7a740 1 -- 172.21.15.16:0/2607048839 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd6c1512f0 msgr2=0x55dd6c5290f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.253+0000 7f5a13e7a740 1 --2- 172.21.15.16:0/2607048839 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd6c1512f0 0x55dd6c5290f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5a00004a80 tx=0x7f5a00010570 comp rx=0 tx=0).stop 2024-08-31T06:17:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.253+0000 7f5a13e7a740 1 -- 172.21.15.16:0/2607048839 shutdown_connections 2024-08-31T06:17:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.253+0000 7f5a13e7a740 1 --2- 172.21.15.16:0/2607048839 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55dd6c138d80 0x55dd6c543930 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.253+0000 7f5a13e7a740 1 --2- 172.21.15.16:0/2607048839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd6c17a1b0 0x55dd6c529b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.253+0000 7f5a13e7a740 1 --2- 172.21.15.16:0/2607048839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd6c1594d0 0x55dd6c529630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.253+0000 7f5a13e7a740 1 --2- 172.21.15.16:0/2607048839 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd6c1512f0 0x55dd6c5290f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.253+0000 7f5a13e7a740 1 -- 172.21.15.16:0/2607048839 >> 172.21.15.16:0/2607048839 conn(0x55dd6c16e300 msgr2=0x55dd6c51eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.253+0000 7f59ebe00640 0 client.27935 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:17:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.254+0000 7f5a13e7a740 1 -- 172.21.15.16:0/2607048839 shutdown_connections 2024-08-31T06:17:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.254+0000 7f5a13e7a740 1 -- 172.21.15.16:0/2607048839 wait complete. 2024-08-31T06:17:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.254+0000 7f5a13e7a740 20 client.27935 trim_cache size 0 max 16384 2024-08-31T06:17:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.276+0000 7f5a13e7a740 1 Processor -- start 2024-08-31T06:17:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.276+0000 7f5a13e7a740 1 -- start start 2024-08-31T06:17:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.277+0000 7f5a13e7a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd6c1512f0 0x55dd6c52b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.277+0000 7f5a13e7a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd6c1594d0 0x55dd6c5262c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.277+0000 7f5a13e7a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd6c17a1b0 0x55dd6c526800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.277+0000 7f5a13e7a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f59e4011080 con 0x55dd6c17a1b0 2024-08-31T06:17:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.277+0000 7f5a13e7a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f59e400e8c0 con 0x55dd6c1594d0 2024-08-31T06:17:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.277+0000 7f5a13e7a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f59e400c730 con 0x55dd6c1512f0 2024-08-31T06:17:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.278+0000 7f5a06800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd6c1512f0 0x55dd6c52b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.278+0000 7f5a06800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd6c1512f0 0x55dd6c52b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:32804/0 (socket says 172.21.15.16:32804) 2024-08-31T06:17:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.278+0000 7f5a06800640 1 -- 172.21.15.16:0/1415457516 learned_addr learned my addr 172.21.15.16:0/1415457516 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.278+0000 7f5a06800640 1 --2- 172.21.15.16:0/1415457516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd6c17a1b0 0x55dd6c526800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.278+0000 7f59ffe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd6c1594d0 0x55dd6c5262c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:40.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.279+0000 7f5a06800640 1 -- 172.21.15.16:0/1415457516 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd6c1512f0 msgr2=0x55dd6c52b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:40.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.279+0000 7f5a06800640 1 --2- 172.21.15.16:0/1415457516 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd6c1512f0 0x55dd6c52b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:40.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.279+0000 7f5a06800640 1 -- 172.21.15.16:0/1415457516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd6c1594d0 msgr2=0x55dd6c5262c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:40.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.279+0000 7f5a06800640 1 --2- 172.21.15.16:0/1415457516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd6c1594d0 0x55dd6c5262c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:40.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.279+0000 7f5a06800640 1 -- 172.21.15.16:0/1415457516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd6c51fe00 con 0x55dd6c17a1b0 2024-08-31T06:17:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.279+0000 7f5a06800640 1 --2- 172.21.15.16:0/1415457516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd6c17a1b0 0x55dd6c526800 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f5a0000ed20 tx=0x7f5a00000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c09fe905027a4085 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.280+0000 7f59f6a00640 1 -- 172.21.15.16:0/1415457516 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a00009820 con 0x55dd6c17a1b0 2024-08-31T06:17:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.280+0000 7f59f6a00640 1 -- 172.21.15.16:0/1415457516 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a00037070 con 0x55dd6c17a1b0 2024-08-31T06:17:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.280+0000 7f59f6a00640 1 -- 172.21.15.16:0/1415457516 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a00010a50 con 0x55dd6c17a1b0 2024-08-31T06:17:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.281+0000 7f5a13e7a740 1 -- 172.21.15.16:0/1415457516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd6c17a1b0 msgr2=0x55dd6c526800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.281+0000 7f5a13e7a740 1 --2- 172.21.15.16:0/1415457516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd6c17a1b0 0x55dd6c526800 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f5a0000ed20 tx=0x7f5a00000f40 comp rx=0 tx=0).stop 2024-08-31T06:17:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.281+0000 7f5a13e7a740 1 -- 172.21.15.16:0/1415457516 shutdown_connections 2024-08-31T06:17:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.281+0000 7f5a13e7a740 1 --2- 172.21.15.16:0/1415457516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd6c17a1b0 0x55dd6c526800 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.281+0000 7f5a13e7a740 1 --2- 172.21.15.16:0/1415457516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd6c1594d0 0x55dd6c5262c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.281+0000 7f5a13e7a740 1 --2- 172.21.15.16:0/1415457516 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd6c1512f0 0x55dd6c52b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.282+0000 7f5a13e7a740 1 -- 172.21.15.16:0/1415457516 >> 172.21.15.16:0/1415457516 conn(0x55dd6c16e300 msgr2=0x55dd6c51f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.282+0000 7f5a13e7a740 1 -- 172.21.15.16:0/1415457516 shutdown_connections 2024-08-31T06:17:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.282+0000 7f5a13e7a740 1 -- 172.21.15.16:0/1415457516 wait complete. 2024-08-31T06:17:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.282+0000 7f5a13e7a740 1 -- start start 2024-08-31T06:17:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.283+0000 7f5a13e7a740 1 Processor -- start 2024-08-31T06:17:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.284+0000 7f5a13e7a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd6c1512f0 0x55dd6c53a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.284+0000 7f5a13e7a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd6c1594d0 0x55dd6c5378a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.284+0000 7f5a13e7a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd6c17a1b0 0x55dd6c537de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.284+0000 7f5a13e7a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f59e400cd80 con 0x55dd6c1594d0 2024-08-31T06:17:40.287 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.284+0000 7f5a13e7a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f59e400e1f0 con 0x55dd6c1512f0 2024-08-31T06:17:40.287 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.284+0000 7f5a13e7a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f59e400d450 con 0x55dd6c17a1b0 2024-08-31T06:17:40.287 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.284+0000 7f5a13e7a740 10 client.0 _ll_register_callbacks cb 0x7f5a00004020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:17:40.287 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.284+0000 7f5a06800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd6c1512f0 0x55dd6c53a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:40.287 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.284+0000 7f5a06800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd6c1512f0 0x55dd6c53a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:38212/0 (socket says 172.21.15.16:38212) 2024-08-31T06:17:40.287 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.284+0000 7f5a06800640 1 -- 172.21.15.16:0/3182791429 learned_addr learned my addr 172.21.15.16:0/3182791429 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.284+0000 7f5a06800640 1 --2- 172.21.15.16:0/3182791429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd6c17a1b0 0x55dd6c537de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.285+0000 7f59ffe00640 1 --2- 172.21.15.16:0/3182791429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd6c1594d0 0x55dd6c5378a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.285+0000 7f5a06800640 1 -- 172.21.15.16:0/3182791429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd6c17a1b0 msgr2=0x55dd6c537de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.285+0000 7f5a06800640 1 --2- 172.21.15.16:0/3182791429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd6c17a1b0 0x55dd6c537de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.285+0000 7f5a06800640 1 -- 172.21.15.16:0/3182791429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd6c1594d0 msgr2=0x55dd6c5378a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.285+0000 7f5a06800640 1 --2- 172.21.15.16:0/3182791429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd6c1594d0 0x55dd6c5378a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.285+0000 7f5a06800640 1 -- 172.21.15.16:0/3182791429 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd6c5383f0 con 0x55dd6c1512f0 2024-08-31T06:17:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.285+0000 7f59ebe00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:17:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.285+0000 7f5a06800640 1 --2- 172.21.15.16:0/3182791429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd6c1512f0 0x55dd6c53a3e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5a00004130 tx=0x7f5a00009610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21c9df8591bc930c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.286+0000 7f59ebe00640 1 -- 172.21.15.16:0/3182791429 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a00047020 con 0x55dd6c1512f0 2024-08-31T06:17:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.286+0000 7f5a13e7a740 1 -- 172.21.15.16:0/3182791429 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a00037070 con 0x55dd6c1512f0 2024-08-31T06:17:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.286+0000 7f59ebe00640 1 -- 172.21.15.16:0/3182791429 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8470+0+0 (secure 0 0 0) 0x7f5a00004720 con 0x55dd6c1512f0 2024-08-31T06:17:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.286+0000 7f59ebe00640 1 -- 172.21.15.16:0/3182791429 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f5a00047020 con 0x55dd6c1512f0 2024-08-31T06:17:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.286+0000 7f5a13e7a740 10 client.27941 fetch_fsmap learned FSMap version 46 2024-08-31T06:17:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.286+0000 7f5a13e7a740 1 -- 172.21.15.16:0/3182791429 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55dd6c5ae020 con 0x55dd6c1512f0 2024-08-31T06:17:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.286+0000 7f59ebe00640 1 -- 172.21.15.16:0/3182791429 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a00037070 con 0x55dd6c1512f0 2024-08-31T06:17:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.286+0000 7f59ebe00640 1 -- 172.21.15.16:0/3182791429 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7f5a00030bf0 con 0x55dd6c1512f0 2024-08-31T06:17:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.287+0000 7f5a13e7a740 10 client.27941 fetch_fsmap finished waiting for FSMap version 46 2024-08-31T06:17:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.287+0000 7f5a13e7a740 10 client.27941 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:17:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.287+0000 7f5a13e7a740 20 client.27941 populate_metadata read hostname 'smithi016' 2024-08-31T06:17:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.287+0000 7f5a13e7a740 1 --2- 172.21.15.16:0/3182791429 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55dd6c10e1c0 0x55dd6c5b0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.287+0000 7f5a13e7a740 4 client.27941 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:17:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.287+0000 7f5a13e7a740 1 -- 172.21.15.16:0/3182791429 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dd6c51bc60 con 0x55dd6c10e1c0 2024-08-31T06:17:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.287+0000 7f59ffe00640 1 --2- 172.21.15.16:0/3182791429 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55dd6c10e1c0 0x55dd6c5b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.288+0000 7f59ebe00640 10 client.27941 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:17:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.289+0000 7f59ffe00640 1 --2- 172.21.15.16:0/3182791429 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55dd6c10e1c0 0x55dd6c5b0710 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12416b8240ed2f9a server_cookie=dadc1bf965f4339b in_seq=0 out_seq=0 2024-08-31T06:17:40.377 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.375+0000 7f59ebe00640 1 -- 172.21.15.16:0/3182791429 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dd6c51bc60 con 0x55dd6c10e1c0 2024-08-31T06:17:40.377 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.375+0000 7f59ebe00640 10 client.27941 handle_command_reply: tid=0 2024-08-31T06:17:40.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.375+0000 7f5a13e7a740 1 client.27941 shutdown 2024-08-31T06:17:40.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.375+0000 7f5a13e7a740 1 -- 172.21.15.16:0/3182791429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd6c1512f0 msgr2=0x55dd6c53a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:40.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.375+0000 7f5a13e7a740 1 --2- 172.21.15.16:0/3182791429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd6c1512f0 0x55dd6c53a3e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5a00004130 tx=0x7f5a00009610 comp rx=0 tx=0).stop 2024-08-31T06:17:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.377+0000 7f5a13e7a740 1 -- 172.21.15.16:0/3182791429 shutdown_connections 2024-08-31T06:17:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.377+0000 7f5a13e7a740 1 --2- 172.21.15.16:0/3182791429 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55dd6c10e1c0 0x55dd6c5b0710 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.377+0000 7f5a13e7a740 1 --2- 172.21.15.16:0/3182791429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd6c17a1b0 0x55dd6c537de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.377+0000 7f5a13e7a740 1 --2- 172.21.15.16:0/3182791429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dd6c1594d0 0x55dd6c5378a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.377+0000 7f59ebe00640 0 client.27941 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:17:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.377+0000 7f5a13e7a740 1 --2- 172.21.15.16:0/3182791429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd6c1512f0 0x55dd6c53a3e0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.377+0000 7f5a13e7a740 1 -- 172.21.15.16:0/3182791429 >> 172.21.15.16:0/3182791429 conn(0x55dd6c16e300 msgr2=0x55dd6c51a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.377+0000 7f5a13e7a740 1 -- 172.21.15.16:0/3182791429 shutdown_connections 2024-08-31T06:17:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.378+0000 7f5a13e7a740 1 -- 172.21.15.16:0/3182791429 wait complete. 2024-08-31T06:17:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.378+0000 7f5a13e7a740 20 client.27941 trim_cache size 0 max 16384 2024-08-31T06:17:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:17:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:17:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:17:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:17:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.379+0000 7f5a07200640 1 -- 172.21.15.16:0/3907042481 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f59ec0b4d50 msgr2=0x7f59ec0b7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.379+0000 7f5a07200640 1 --2- 172.21.15.16:0/3907042481 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f59ec0b4d50 0x7f59ec0b7260 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x55dd6c51c780 tx=0x7f59f8000bc0 comp rx=0 tx=0).stop 2024-08-31T06:17:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.379+0000 7f5a07200640 1 -- 172.21.15.16:0/3907042481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a08102150 msgr2=0x7f5a0819c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.379+0000 7f5a07200640 1 --2- 172.21.15.16:0/3907042481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a08102150 0x7f5a0819c240 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f59f8031440 tx=0x7f59f8003b90 comp rx=0 tx=0).stop 2024-08-31T06:17:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.379+0000 7f5a07200640 1 -- 172.21.15.16:0/3907042481 shutdown_connections 2024-08-31T06:17:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.379+0000 7f5a07200640 1 --2- 172.21.15.16:0/3907042481 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f59ec0b4d50 0x7f59ec0b7260 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.379+0000 7f5a07200640 1 --2- 172.21.15.16:0/3907042481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a0810fd00 0x7f5a081a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.379+0000 7f5a07200640 1 --2- 172.21.15.16:0/3907042481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a08102aa0 0x7f5a0819c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.379+0000 7f5a07200640 1 --2- 172.21.15.16:0/3907042481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a08102150 0x7f5a0819c240 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.379+0000 7f5a07200640 1 -- 172.21.15.16:0/3907042481 >> 172.21.15.16:0/3907042481 conn(0x7f5a080fdfd0 msgr2=0x7f5a08100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.379+0000 7f5a07200640 1 -- 172.21.15.16:0/3907042481 shutdown_connections 2024-08-31T06:17:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:40.379+0000 7f5a07200640 1 -- 172.21.15.16:0/3907042481 wait complete. 2024-08-31T06:17:40.890 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T06:17:40.891 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T06:17:40.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:40 smithi184 ceph-mon[22331]: pgmap v687: 97 pgs: 97 active+clean; 3.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 21 MiB/s wr, 1.37k op/s 2024-08-31T06:17:40.985 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:40 smithi016 ceph-mon[17194]: pgmap v687: 97 pgs: 97 active+clean; 3.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 21 MiB/s wr, 1.37k op/s 2024-08-31T06:17:41.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:40 smithi155 ceph-mon[21742]: pgmap v687: 97 pgs: 97 active+clean; 3.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 21 MiB/s wr, 1.37k op/s 2024-08-31T06:17:41.514 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:17:41.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 615.0 s, 1587.6 tps, lat 62.384 ms stddev 72.984 2024-08-31T06:17:42.194 INFO:tasks.quiescer.fs.[cephfs]:Total duration (50.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-08-31T06:17:42.194 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.25 2024-08-31T06:17:42.195 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:17:42.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.365+0000 7fe11f200640 1 -- 172.21.15.16:0/2796045669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe12011c940 msgr2=0x7fe12011ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:42.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.366+0000 7fe11f200640 1 --2- 172.21.15.16:0/2796045669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe12011c940 0x7fe12011ed60 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe118009be0 tx=0x7fe11802a6c0 comp rx=0 tx=0).stop 2024-08-31T06:17:42.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.367+0000 7fe11f200640 1 -- 172.21.15.16:0/2796045669 shutdown_connections 2024-08-31T06:17:42.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.367+0000 7fe11f200640 1 --2- 172.21.15.16:0/2796045669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe12011c940 0x7fe12011ed60 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:42.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.367+0000 7fe11f200640 1 --2- 172.21.15.16:0/2796045669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe12010d990 0x7fe12010dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:42.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.368+0000 7fe11f200640 1 --2- 172.21.15.16:0/2796045669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe120111a70 0x7fe12010d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:42.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.368+0000 7fe11f200640 1 -- 172.21.15.16:0/2796045669 >> 172.21.15.16:0/2796045669 conn(0x7fe12006e790 msgr2=0x7fe12006ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:42.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.369+0000 7fe11f200640 1 -- 172.21.15.16:0/2796045669 shutdown_connections 2024-08-31T06:17:42.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.369+0000 7fe11f200640 1 -- 172.21.15.16:0/2796045669 wait complete. 2024-08-31T06:17:42.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.370+0000 7fe11f200640 1 Processor -- start 2024-08-31T06:17:42.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.375+0000 7fe11f200640 1 -- start start 2024-08-31T06:17:42.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.376+0000 7fe11f200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe12010d990 0x7fe1201a8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:42.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.376+0000 7fe11f200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe120111a70 0x7fe1201a93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:42.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.376+0000 7fe11f200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe12011c940 0x7fe1201ae690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:42.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.376+0000 7fe11f200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe120122460 con 0x7fe120111a70 2024-08-31T06:17:42.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.376+0000 7fe11f200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe1201222e0 con 0x7fe12011c940 2024-08-31T06:17:42.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.376+0000 7fe11f200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe1201225e0 con 0x7fe12010d990 2024-08-31T06:17:42.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.378+0000 7fe11de00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe12010d990 0x7fe1201a8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:42.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.378+0000 7fe11de00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe12010d990 0x7fe1201a8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:32834/0 (socket says 172.21.15.16:32834) 2024-08-31T06:17:42.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.378+0000 7fe11de00640 1 -- 172.21.15.16:0/2331225755 learned_addr learned my addr 172.21.15.16:0/2331225755 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:42.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.378+0000 7fe11e800640 1 --2- 172.21.15.16:0/2331225755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe12011c940 0x7fe1201ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:42.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.379+0000 7fe11de00640 1 -- 172.21.15.16:0/2331225755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe12011c940 msgr2=0x7fe1201ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:42.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.379+0000 7fe11de00640 1 --2- 172.21.15.16:0/2331225755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe12011c940 0x7fe1201ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:42.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.379+0000 7fe11de00640 1 -- 172.21.15.16:0/2331225755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe120111a70 msgr2=0x7fe1201a93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:42.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.379+0000 7fe11de00640 1 --2- 172.21.15.16:0/2331225755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe120111a70 0x7fe1201a93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:42.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.379+0000 7fe11de00640 1 -- 172.21.15.16:0/2331225755 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1201aedc0 con 0x7fe12010d990 2024-08-31T06:17:42.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.380+0000 7fe11de00640 1 --2- 172.21.15.16:0/2331225755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe12010d990 0x7fe1201a8e50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe11000e9f0 tx=0x7fe11000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:42.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.381+0000 7fe116a00640 1 -- 172.21.15.16:0/2331225755 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11000ccc0 con 0x7fe12010d990 2024-08-31T06:17:42.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.381+0000 7fe11f200640 1 -- 172.21.15.16:0/2331225755 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1201af0b0 con 0x7fe12010d990 2024-08-31T06:17:42.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.381+0000 7fe11f200640 1 -- 172.21.15.16:0/2331225755 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1201b5810 con 0x7fe12010d990 2024-08-31T06:17:42.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.382+0000 7fe116a00640 1 -- 172.21.15.16:0/2331225755 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1100035f0 con 0x7fe12010d990 2024-08-31T06:17:42.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.382+0000 7fe116a00640 1 -- 172.21.15.16:0/2331225755 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11000ce60 con 0x7fe12010d990 2024-08-31T06:17:42.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.388+0000 7fe12bfdf740 1 Processor -- start 2024-08-31T06:17:42.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.391+0000 7fe12bfdf740 1 -- start start 2024-08-31T06:17:42.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.391+0000 7fe12bfdf740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c3c976350 0x559c3cd1c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:42.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.391+0000 7fe12bfdf740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c3c94d490 0x559c3cd1cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:42.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.391+0000 7fe12bfdf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c3c955670 0x559c3cd1d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:42.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.391+0000 7fe12bfdf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559c3cc92750 con 0x559c3c955670 2024-08-31T06:17:42.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.391+0000 7fe12bfdf740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559c3cc81ce0 con 0x559c3c94d490 2024-08-31T06:17:42.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.391+0000 7fe12bfdf740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559c3cc85c90 con 0x559c3c976350 2024-08-31T06:17:42.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.392+0000 7fe11e800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c3c976350 0x559c3cd1c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:42.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.392+0000 7fe11de00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c3c94d490 0x559c3cd1cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:42.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.393+0000 7fe11de00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c3c94d490 0x559c3cd1cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:38242/0 (socket says 172.21.15.16:38242) 2024-08-31T06:17:42.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.393+0000 7fe11de00640 1 -- 172.21.15.16:0/1811360098 learned_addr learned my addr 172.21.15.16:0/1811360098 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:42.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.393+0000 7fe11e800640 1 -- 172.21.15.16:0/1811360098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c3c94d490 msgr2=0x559c3cd1cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:42.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.393+0000 7fe11e800640 1 --2- 172.21.15.16:0/1811360098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c3c94d490 0x559c3cd1cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:42.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.393+0000 7fe11e800640 1 -- 172.21.15.16:0/1811360098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c3c955670 msgr2=0x559c3cd1d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:42.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.393+0000 7fe11e800640 1 --2- 172.21.15.16:0/1811360098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c3c955670 0x559c3cd1d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:42.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.393+0000 7fe11e800640 1 -- 172.21.15.16:0/1811360098 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c3c9e1930 con 0x559c3c976350 2024-08-31T06:17:42.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.395+0000 7fe116a00640 1 -- 172.21.15.16:0/2331225755 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fe110003fb0 con 0x7fe12010d990 2024-08-31T06:17:42.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.395+0000 7fe11e800640 1 --2- 172.21.15.16:0/1811360098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c3c976350 0x559c3cd1c7d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe118009b30 tx=0x7fe1180098c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df3de9463647f75c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:42.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.395+0000 7fe10b400640 1 -- 172.21.15.16:0/1811360098 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe118041030 con 0x559c3c976350 2024-08-31T06:17:42.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.396+0000 7fe10b400640 1 -- 172.21.15.16:0/1811360098 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe118003970 con 0x559c3c976350 2024-08-31T06:17:42.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.396+0000 7fe10b400640 1 -- 172.21.15.16:0/1811360098 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe118036040 con 0x559c3c976350 2024-08-31T06:17:42.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.398+0000 7fe116a00640 1 --2- 172.21.15.16:0/2331225755 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe1040b4dd0 0x7fe1040b72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:42.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.398+0000 7fe116a00640 1 -- 172.21.15.16:0/2331225755 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8470+0+0 (secure 0 0 0) 0x7fe1100e5800 con 0x7fe12010d990 2024-08-31T06:17:42.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.398+0000 7fe12bfdf740 1 -- 172.21.15.16:0/1811360098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c3c976350 msgr2=0x559c3cd1c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:42.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.399+0000 7fe11de00640 1 --2- 172.21.15.16:0/2331225755 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe1040b4dd0 0x7fe1040b72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:42.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.399+0000 7fe12bfdf740 1 --2- 172.21.15.16:0/1811360098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c3c976350 0x559c3cd1c7d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe118009b30 tx=0x7fe1180098c0 comp rx=0 tx=0).stop 2024-08-31T06:17:42.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.400+0000 7fe12bfdf740 1 -- 172.21.15.16:0/1811360098 shutdown_connections 2024-08-31T06:17:42.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.400+0000 7fe12bfdf740 1 --2- 172.21.15.16:0/1811360098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c3c955670 0x559c3cd1d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:42.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.400+0000 7fe12bfdf740 1 --2- 172.21.15.16:0/1811360098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c3c94d490 0x559c3cd1cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:42.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.400+0000 7fe11de00640 1 --2- 172.21.15.16:0/2331225755 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe1040b4dd0 0x7fe1040b72e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe1100a9130 tx=0x7fe1100eb530 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:42.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.401+0000 7fe12bfdf740 1 --2- 172.21.15.16:0/1811360098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c3c976350 0x559c3cd1c7d0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:42.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.401+0000 7fe12bfdf740 1 -- 172.21.15.16:0/1811360098 >> 172.21.15.16:0/1811360098 conn(0x559c3c96a740 msgr2=0x559c3cd1a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:42.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.403+0000 7fe12bfdf740 1 -- 172.21.15.16:0/1811360098 shutdown_connections 2024-08-31T06:17:42.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.405+0000 7fe12bfdf740 1 -- 172.21.15.16:0/1811360098 wait complete. 2024-08-31T06:17:42.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.406+0000 7fe12bfdf740 1 -- start start 2024-08-31T06:17:42.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.406+0000 7fe12bfdf740 1 Processor -- start 2024-08-31T06:17:42.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.408+0000 7fe12bfdf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c3c94d490 0x559c3cd28370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:42.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.408+0000 7fe11e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c3c94d490 0x559c3cd28370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:42.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.409+0000 7fe12bfdf740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c3c955670 0x559c3cd288b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:42.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.409+0000 7fe12bfdf740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c3c976350 0x559c3cd45250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:42.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.409+0000 7fe11e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c3c94d490 0x559c3cd28370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40112/0 (socket says 172.21.15.16:40112) 2024-08-31T06:17:42.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.409+0000 7fe11e800640 1 -- 172.21.15.16:0/1942321093 learned_addr learned my addr 172.21.15.16:0/1942321093 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:42.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.410+0000 7fe12bfdf740 1 -- 172.21.15.16:0/1942321093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559c3cae4890 con 0x559c3c94d490 2024-08-31T06:17:42.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.410+0000 7fe11d400640 1 --2- 172.21.15.16:0/1942321093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c3c955670 0x559c3cd288b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:42.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.410+0000 7fe12bfdf740 1 -- 172.21.15.16:0/1942321093 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559c3cc08cd0 con 0x559c3c955670 2024-08-31T06:17:42.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.410+0000 7fe12bfdf740 1 -- 172.21.15.16:0/1942321093 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559c3cd3bf60 con 0x559c3c976350 2024-08-31T06:17:42.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.410+0000 7fe11e800640 1 --2- 172.21.15.16:0/1942321093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c3c976350 0x559c3cd45250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:42.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.411+0000 7fe12bfdf740 10 client.0 _ll_register_callbacks cb 0x559c3cc7cef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:17:42.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.413+0000 7fe11d400640 1 -- 172.21.15.16:0/1942321093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c3c976350 msgr2=0x559c3cd45250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:42.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.413+0000 7fe11d400640 1 --2- 172.21.15.16:0/1942321093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c3c976350 0x559c3cd45250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:42.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.413+0000 7fe11d400640 1 -- 172.21.15.16:0/1942321093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c3c94d490 msgr2=0x559c3cd28370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:42.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.413+0000 7fe11d400640 1 --2- 172.21.15.16:0/1942321093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c3c94d490 0x559c3cd28370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:42.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.413+0000 7fe11d400640 1 -- 172.21.15.16:0/1942321093 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c3cc521b0 con 0x559c3c955670 2024-08-31T06:17:42.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.413+0000 7fe12bfdf740 1 -- 172.21.15.16:0/1942321093 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe118003970 con 0x559c3c955670 2024-08-31T06:17:42.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.414+0000 7fe0ffe00640 10 client.27953 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:17:42.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.414+0000 7fe11d400640 1 --2- 172.21.15.16:0/1942321093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c3c955670 0x559c3cd288b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe12006c420 tx=0x7fe10c00e7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b02b7246a80bddc6 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:42.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.414+0000 7fe0ffe00640 1 -- 172.21.15.16:0/1942321093 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10c004460 con 0x559c3c955670 2024-08-31T06:17:42.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.415+0000 7fe0ffe00640 1 -- 172.21.15.16:0/1942321093 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8470+0+0 (secure 0 0 0) 0x7fe10c005020 con 0x559c3c955670 2024-08-31T06:17:42.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.415+0000 7fe0ffe00640 1 -- 172.21.15.16:0/1942321093 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fe10c005260 con 0x559c3c955670 2024-08-31T06:17:42.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.415+0000 7fe12bfdf740 10 client.27953 fetch_fsmap learned FSMap version 46 2024-08-31T06:17:42.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.415+0000 7fe12bfdf740 1 -- 172.21.15.16:0/1942321093 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({fsmap=46}) -- 0x559c3cd47f00 con 0x559c3c955670 2024-08-31T06:17:42.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.416+0000 7fe0ffe00640 1 -- 172.21.15.16:0/1942321093 <== mon.2 v2:172.21.15.155:3300/0 4 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7fe10c005400 con 0x559c3c955670 2024-08-31T06:17:42.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.416+0000 7fe12bfdf740 10 client.27953 fetch_fsmap finished waiting for FSMap version 46 2024-08-31T06:17:42.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.416+0000 7fe12bfdf740 10 client.27953 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:17:42.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.416+0000 7fe12bfdf740 20 client.27953 populate_metadata read hostname 'smithi016' 2024-08-31T06:17:42.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.416+0000 7fe12bfdf740 1 --2- 172.21.15.16:0/1942321093 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559c3c934cd0 0x559c3cd4b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:42.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.416+0000 7fe12bfdf740 4 client.27953 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:17:42.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.416+0000 7fe12bfdf740 1 -- 172.21.15.16:0/1942321093 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c3cc946a0 con 0x559c3c934cd0 2024-08-31T06:17:42.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.420+0000 7fe11e800640 1 --2- 172.21.15.16:0/1942321093 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559c3c934cd0 0x559c3cd4b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:42.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.420+0000 7fe11e800640 1 --2- 172.21.15.16:0/1942321093 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559c3c934cd0 0x559c3cd4b3e0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baf0d7d74816c6a server_cookie=7faf5bf1a4bf352c in_seq=0 out_seq=0 2024-08-31T06:17:42.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.421+0000 7fe0ffe00640 10 client.27953 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:17:42.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.422+0000 7fe0ffe00640 1 -- 172.21.15.16:0/1942321093 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559c3cc946a0 con 0x559c3c934cd0 2024-08-31T06:17:42.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.422+0000 7fe0ffe00640 10 client.27953 handle_command_reply: tid=0 2024-08-31T06:17:42.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.422+0000 7fe12bfdf740 1 client.27953 shutdown 2024-08-31T06:17:42.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.422+0000 7fe12bfdf740 1 -- 172.21.15.16:0/1942321093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c3c955670 msgr2=0x559c3cd288b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:42.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.422+0000 7fe12bfdf740 1 --2- 172.21.15.16:0/1942321093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c3c955670 0x559c3cd288b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe12006c420 tx=0x7fe10c00e7e0 comp rx=0 tx=0).stop 2024-08-31T06:17:42.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.422+0000 7fe12bfdf740 1 -- 172.21.15.16:0/1942321093 shutdown_connections 2024-08-31T06:17:42.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.422+0000 7fe12bfdf740 1 --2- 172.21.15.16:0/1942321093 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559c3c934cd0 0x559c3cd4b3e0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:42.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.423+0000 7fe12bfdf740 1 --2- 172.21.15.16:0/1942321093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c3c976350 0x559c3cd45250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:42.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.423+0000 7fe12bfdf740 1 --2- 172.21.15.16:0/1942321093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c3c955670 0x559c3cd288b0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:42.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.423+0000 7fe12bfdf740 1 --2- 172.21.15.16:0/1942321093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c3c94d490 0x559c3cd28370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:42.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.423+0000 7fe12bfdf740 1 -- 172.21.15.16:0/1942321093 >> 172.21.15.16:0/1942321093 conn(0x559c3c96a740 msgr2=0x559c3cd1f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:42.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.423+0000 7fe0ffe00640 0 client.27953 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:17:42.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.423+0000 7fe12bfdf740 1 -- 172.21.15.16:0/1942321093 shutdown_connections 2024-08-31T06:17:42.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.423+0000 7fe12bfdf740 1 -- 172.21.15.16:0/1942321093 wait complete. 2024-08-31T06:17:42.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.423+0000 7fe12bfdf740 20 client.27953 trim_cache size 0 max 16384 2024-08-31T06:17:42.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.461+0000 7fe12bfdf740 1 Processor -- start 2024-08-31T06:17:42.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.461+0000 7fe12bfdf740 1 -- start start 2024-08-31T06:17:42.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.462+0000 7fe12bfdf740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c3c94d490 0x559c3cd29da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:42.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.462+0000 7fe11e800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c3c94d490 0x559c3cd29da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:42.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.463+0000 7fe11e800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c3c94d490 0x559c3cd29da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:32870/0 (socket says 172.21.15.16:32870) 2024-08-31T06:17:42.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.463+0000 7fe12bfdf740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c3c955670 0x559c3cd218b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:42.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.463+0000 7fe12bfdf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c3c976350 0x559c3cd21df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:42.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.463+0000 7fe11e800640 1 -- 172.21.15.16:0/33625592 learned_addr learned my addr 172.21.15.16:0/33625592 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:42.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.463+0000 7fe12bfdf740 1 -- 172.21.15.16:0/33625592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe1201b81f0 con 0x559c3c976350 2024-08-31T06:17:42.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.463+0000 7fe12bfdf740 1 -- 172.21.15.16:0/33625592 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe1201b5aa0 con 0x559c3c955670 2024-08-31T06:17:42.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.463+0000 7fe12bfdf740 1 -- 172.21.15.16:0/33625592 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe120074f80 con 0x559c3c94d490 2024-08-31T06:17:42.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.463+0000 7fe11d400640 1 --2- 172.21.15.16:0/33625592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c3c955670 0x559c3cd218b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:42.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.464+0000 7fe11d400640 1 -- 172.21.15.16:0/33625592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c3c94d490 msgr2=0x559c3cd29da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:42.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.464+0000 7fe11d400640 1 --2- 172.21.15.16:0/33625592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c3c94d490 0x559c3cd29da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:42.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.464+0000 7fe11d400640 1 -- 172.21.15.16:0/33625592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c3c976350 msgr2=0x559c3cd21df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:17:42.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.464+0000 7fe11d400640 1 --2- 172.21.15.16:0/33625592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c3c976350 0x559c3cd21df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:42.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.464+0000 7fe11d400640 1 -- 172.21.15.16:0/33625592 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c3cd1eac0 con 0x559c3c955670 2024-08-31T06:17:42.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.464+0000 7fe11d400640 1 --2- 172.21.15.16:0/33625592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c3c955670 0x559c3cd218b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe10c0047c0 tx=0x7fe10c012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5cf0b06c0833e1 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:42.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.464+0000 7fe10aa00640 1 -- 172.21.15.16:0/33625592 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10c004260 con 0x559c3c955670 2024-08-31T06:17:42.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.465+0000 7fe10aa00640 1 -- 172.21.15.16:0/33625592 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe118003970 con 0x559c3c955670 2024-08-31T06:17:42.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.465+0000 7fe10aa00640 1 -- 172.21.15.16:0/33625592 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10c003850 con 0x559c3c955670 2024-08-31T06:17:42.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.466+0000 7fe12bfdf740 1 -- 172.21.15.16:0/33625592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c3c955670 msgr2=0x559c3cd218b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:42.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.466+0000 7fe12bfdf740 1 --2- 172.21.15.16:0/33625592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c3c955670 0x559c3cd218b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe10c0047c0 tx=0x7fe10c012000 comp rx=0 tx=0).stop 2024-08-31T06:17:42.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.466+0000 7fe12bfdf740 1 -- 172.21.15.16:0/33625592 shutdown_connections 2024-08-31T06:17:42.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.466+0000 7fe12bfdf740 1 --2- 172.21.15.16:0/33625592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c3c976350 0x559c3cd21df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:42.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.466+0000 7fe12bfdf740 1 --2- 172.21.15.16:0/33625592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c3c955670 0x559c3cd218b0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:42.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.466+0000 7fe12bfdf740 1 --2- 172.21.15.16:0/33625592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c3c94d490 0x559c3cd29da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:42.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.466+0000 7fe12bfdf740 1 -- 172.21.15.16:0/33625592 >> 172.21.15.16:0/33625592 conn(0x559c3c96a740 msgr2=0x559c3cd29690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:42.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.467+0000 7fe12bfdf740 1 -- 172.21.15.16:0/33625592 shutdown_connections 2024-08-31T06:17:42.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.469+0000 7fe12bfdf740 1 -- 172.21.15.16:0/33625592 wait complete. 2024-08-31T06:17:42.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.469+0000 7fe12bfdf740 1 -- start start 2024-08-31T06:17:42.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.469+0000 7fe12bfdf740 1 Processor -- start 2024-08-31T06:17:42.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.470+0000 7fe12bfdf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c3c94d490 0x559c3cd418f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:42.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.471+0000 7fe12bfdf740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c3c955670 0x559c3cd3ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:42.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.471+0000 7fe12bfdf740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c3c976350 0x559c3cd3e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:42.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.471+0000 7fe12bfdf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe120118de0 con 0x559c3c94d490 2024-08-31T06:17:42.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.471+0000 7fe11e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c3c94d490 0x559c3cd418f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:42.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.471+0000 7fe11e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c3c94d490 0x559c3cd418f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40140/0 (socket says 172.21.15.16:40140) 2024-08-31T06:17:42.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.471+0000 7fe11d400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c3c955670 0x559c3cd3ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:42.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.471+0000 7fe11e800640 1 -- 172.21.15.16:0/3123063019 learned_addr learned my addr 172.21.15.16:0/3123063019 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:42.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.471+0000 7fe12bfdf740 1 -- 172.21.15.16:0/3123063019 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe12011a1f0 con 0x559c3c955670 2024-08-31T06:17:42.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.471+0000 7fe12bfdf740 1 -- 172.21.15.16:0/3123063019 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe120119450 con 0x559c3c976350 2024-08-31T06:17:42.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.471+0000 7fe12bfdf740 10 client.0 _ll_register_callbacks cb 0x7fe10c009390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:17:42.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.471+0000 7fe11e800640 1 --2- 172.21.15.16:0/3123063019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c3c976350 0x559c3cd3e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:42.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.472+0000 7fe11d400640 1 -- 172.21.15.16:0/3123063019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c3c976350 msgr2=0x559c3cd3e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:42.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.472+0000 7fe11d400640 1 --2- 172.21.15.16:0/3123063019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c3c976350 0x559c3cd3e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:42.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.472+0000 7fe11d400640 1 -- 172.21.15.16:0/3123063019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c3c94d490 msgr2=0x559c3cd418f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:42.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.472+0000 7fe11d400640 1 --2- 172.21.15.16:0/3123063019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c3c94d490 0x559c3cd418f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:42.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.472+0000 7fe11d400640 1 -- 172.21.15.16:0/3123063019 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c3cd3e9e0 con 0x559c3c955670 2024-08-31T06:17:42.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.472+0000 7fe0ffe00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:17:42.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.472+0000 7fe11d400640 1 --2- 172.21.15.16:0/3123063019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c3c955670 0x559c3cd3ddb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe10c000c00 tx=0x7fe10c004490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58a5d01e796f25d8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:42.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.473+0000 7fe0ffe00640 1 -- 172.21.15.16:0/3123063019 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10c004580 con 0x559c3c955670 2024-08-31T06:17:42.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.473+0000 7fe12bfdf740 1 -- 172.21.15.16:0/3123063019 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe118003970 con 0x559c3c955670 2024-08-31T06:17:42.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.473+0000 7fe0ffe00640 1 -- 172.21.15.16:0/3123063019 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8470+0+0 (secure 0 0 0) 0x7fe10c033070 con 0x559c3c955670 2024-08-31T06:17:42.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.474+0000 7fe0ffe00640 1 -- 172.21.15.16:0/3123063019 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fe10c03d050 con 0x559c3c955670 2024-08-31T06:17:42.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.474+0000 7fe12bfdf740 10 client.27959 fetch_fsmap learned FSMap version 46 2024-08-31T06:17:42.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.474+0000 7fe12bfdf740 1 -- 172.21.15.16:0/3123063019 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x559c3cdb6cd0 con 0x559c3c955670 2024-08-31T06:17:42.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.474+0000 7fe0ffe00640 1 -- 172.21.15.16:0/3123063019 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe118003970 con 0x559c3c955670 2024-08-31T06:17:42.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.474+0000 7fe0ffe00640 1 -- 172.21.15.16:0/3123063019 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7fe10c040020 con 0x559c3c955670 2024-08-31T06:17:42.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.474+0000 7fe12bfdf740 10 client.27959 fetch_fsmap finished waiting for FSMap version 46 2024-08-31T06:17:42.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.474+0000 7fe12bfdf740 10 client.27959 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:17:42.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.474+0000 7fe12bfdf740 20 client.27959 populate_metadata read hostname 'smithi016' 2024-08-31T06:17:42.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.475+0000 7fe12bfdf740 1 --2- 172.21.15.16:0/3123063019 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559c3c90a080 0x559c3cdb93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:42.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.475+0000 7fe12bfdf740 4 client.27959 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T06:17:42.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.475+0000 7fe12bfdf740 1 -- 172.21.15.16:0/3123063019 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559c3cc90140 con 0x559c3c90a080 2024-08-31T06:17:42.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.475+0000 7fe11e800640 1 --2- 172.21.15.16:0/3123063019 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559c3c90a080 0x559c3cdb93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.476+0000 7fe11e800640 1 --2- 172.21.15.16:0/3123063019 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559c3c90a080 0x559c3cdb93d0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcd479a78faf9592 server_cookie=99af765536485e67 in_seq=0 out_seq=0 2024-08-31T06:17:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.476+0000 7fe0ffe00640 10 client.27959 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:17:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.477+0000 7fe0ffe00640 1 -- 172.21.15.16:0/3123063019 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559c3cc90140 con 0x559c3c90a080 2024-08-31T06:17:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.477+0000 7fe0ffe00640 10 client.27959 handle_command_reply: tid=0 2024-08-31T06:17:42.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.477+0000 7fe12bfdf740 1 client.27959 shutdown 2024-08-31T06:17:42.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.480+0000 7fe12bfdf740 1 -- 172.21.15.16:0/3123063019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c3c955670 msgr2=0x559c3cd3ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:42.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.480+0000 7fe12bfdf740 1 --2- 172.21.15.16:0/3123063019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c3c955670 0x559c3cd3ddb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe10c000c00 tx=0x7fe10c004490 comp rx=0 tx=0).stop 2024-08-31T06:17:42.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.480+0000 7fe12bfdf740 1 -- 172.21.15.16:0/3123063019 shutdown_connections 2024-08-31T06:17:42.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.480+0000 7fe12bfdf740 1 --2- 172.21.15.16:0/3123063019 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559c3c90a080 0x559c3cdb93d0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:42.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.480+0000 7fe12bfdf740 1 --2- 172.21.15.16:0/3123063019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c3c976350 0x559c3cd3e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:42.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.480+0000 7fe12bfdf740 1 --2- 172.21.15.16:0/3123063019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c3c955670 0x559c3cd3ddb0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:42.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.480+0000 7fe12bfdf740 1 --2- 172.21.15.16:0/3123063019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c3c94d490 0x559c3cd418f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:42.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.480+0000 7fe12bfdf740 1 -- 172.21.15.16:0/3123063019 >> 172.21.15.16:0/3123063019 conn(0x559c3c96a740 msgr2=0x559c3cd1d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:42.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.480+0000 7fe0ffe00640 0 client.27959 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:17:42.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.481+0000 7fe12bfdf740 1 -- 172.21.15.16:0/3123063019 shutdown_connections 2024-08-31T06:17:42.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.481+0000 7fe12bfdf740 1 -- 172.21.15.16:0/3123063019 wait complete. 2024-08-31T06:17:42.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.481+0000 7fe12bfdf740 20 client.27959 trim_cache size 0 max 16384 2024-08-31T06:17:42.484 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:17:42.485 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-08-31T06:17:42.485 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "e497cd88-1e7c-4631-a4a8-cdd8e1fa6c83", 2024-08-31T06:17:42.485 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-08-31T06:17:42.485 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:17:42.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.483+0000 7fe11f200640 1 -- 172.21.15.16:0/2331225755 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe1040b4dd0 msgr2=0x7fe1040b72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:42.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.483+0000 7fe11f200640 1 --2- 172.21.15.16:0/2331225755 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe1040b4dd0 0x7fe1040b72e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe1100a9130 tx=0x7fe1100eb530 comp rx=0 tx=0).stop 2024-08-31T06:17:42.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.483+0000 7fe11f200640 1 -- 172.21.15.16:0/2331225755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe12010d990 msgr2=0x7fe1201a8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:42.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.483+0000 7fe11f200640 1 --2- 172.21.15.16:0/2331225755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe12010d990 0x7fe1201a8e50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe11000e9f0 tx=0x7fe11000eeb0 comp rx=0 tx=0).stop 2024-08-31T06:17:42.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.484+0000 7fe11f200640 1 -- 172.21.15.16:0/2331225755 shutdown_connections 2024-08-31T06:17:42.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.484+0000 7fe11f200640 1 --2- 172.21.15.16:0/2331225755 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe1040b4dd0 0x7fe1040b72e0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:42.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.484+0000 7fe11f200640 1 --2- 172.21.15.16:0/2331225755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe12011c940 0x7fe1201ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:42.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.484+0000 7fe11f200640 1 --2- 172.21.15.16:0/2331225755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe120111a70 0x7fe1201a93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:42.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.484+0000 7fe11f200640 1 --2- 172.21.15.16:0/2331225755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe12010d990 0x7fe1201a8e50 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:42.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.484+0000 7fe11f200640 1 -- 172.21.15.16:0/2331225755 >> 172.21.15.16:0/2331225755 conn(0x7fe12006e790 msgr2=0x7fe120110720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:42.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.484+0000 7fe11f200640 1 -- 172.21.15.16:0/2331225755 shutdown_connections 2024-08-31T06:17:42.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:42.484+0000 7fe11f200640 1 -- 172.21.15.16:0/2331225755 wait complete. 2024-08-31T06:17:42.872 INFO:journalctl@ceph.mds.b.smithi155.stdout:Aug 31 06:17:42 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-b[53353]: 2024-08-31T06:17:42.560+0000 7f6c22b03640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000000f [...12,head] /volumes/qa/sv_1/ba5423bc-5c83-4d64-ae1c-ad8850110ce4/postgres/data/pg_subtrans/ auth{1=1,2=1} v3306 ap=2 f(v13 m2024-08-31T06:17:14.533742+0000 6=6+0) n(v34 rc2024-08-31T06:17:14.533742+0000 b1196032 7=6+1) old_inodes=6 (inest sync->mix g=1,2) mcw={1=AsLsXsFs} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x558498a41600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000000f:[<0x10000000006/pg_subtrans v252>,<0x10000000005/data v106>,<0x100000001fb/postgres v19>,<0x100000001fa/ba5423bc-5c83-4d64-ae1c-ad8850110ce4 v35>,<0x10000000001/sv_1 v56>,<0x10000000000/qa v44>,<0x1/volumes v40>]//[]","memoryvalue":"(2)0x1000000000f:[<0x10000000006/pg_subtrans v3306>,<0x10000000005/data v934>,<0x100000001fb/postgres v576>,<0x100000001fa/ba5423bc-5c83-4d64-ae1c-ad8850110ce4 v600>,<0x10000000001/sv_1 v598>,<0x10000000000/qa v600>,<0x1/volumes v546>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-08-31T06:16:41.777254+0000 5=5+0)","ondisk_value.rstat":"n(v0 rc2024-08-31T06:17:14.533742+0000 b1196032 7=6+1)","memory_value.dirstat":"f(v13 m2024-08-31T06:17:14.533742+0000 6=6+0)","memory_value.rstat":"n(v34 rc2024-08-31T06:17:14.533742+0000 b1196032 7=6+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-08-31T06:17:42.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:42 smithi155 ceph-mon[21742]: pgmap v688: 97 pgs: 97 active+clean; 3.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 16 MiB/s wr, 959 op/s 2024-08-31T06:17:42.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:42 smithi184 ceph-mon[22331]: pgmap v688: 97 pgs: 97 active+clean; 3.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 16 MiB/s wr, 959 op/s 2024-08-31T06:17:43.043 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:42 smithi016 ceph-mon[17194]: pgmap v688: 97 pgs: 97 active+clean; 3.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 16 MiB/s wr, 959 op/s 2024-08-31T06:17:43.596 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T06:17:43.845 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:17:43.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:43 smithi016 ceph-mon[17194]: scrub queued for path: / 2024-08-31T06:17:43.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:43 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:17:43.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:43 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:17:43.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:43 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:17:43.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:43 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:17:43.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:43 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:17:43.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:43 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:17:43.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:43 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:17:43.915 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:43 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:17:43.915 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:43 smithi016 ceph-mon[17194]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_1/ba5423bc-5c83-4d64-ae1c-ad8850110ce4/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-08-31T06:17:43.915 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:43 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:17:43.915 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:43 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:17:43.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:43 smithi184 ceph-mon[22331]: scrub queued for path: / 2024-08-31T06:17:43.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:43 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:17:43.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:43 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:17:43.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:43 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:17:43.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:43 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:17:43.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:43 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:17:43.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:43 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:17:43.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:43 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:17:43.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:43 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:17:43.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:43 smithi184 ceph-mon[22331]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_1/ba5423bc-5c83-4d64-ae1c-ad8850110ce4/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-08-31T06:17:43.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:43 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:17:43.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:43 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:17:44.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:43 smithi155 ceph-mon[21742]: scrub queued for path: / 2024-08-31T06:17:44.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:43 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:17:44.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:43 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:17:44.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:43 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:17:44.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:43 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:17:44.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:43 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:17:44.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:43 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:17:44.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:43 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:17:44.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:43 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:17:44.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:43 smithi155 ceph-mon[21742]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_1/ba5423bc-5c83-4d64-ae1c-ad8850110ce4/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-08-31T06:17:44.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:43 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:17:44.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:43 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:17:44.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:44 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:17:44.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:44 smithi184 ceph-mon[22331]: pgmap v689: 97 pgs: 97 active+clean; 3.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 13 MiB/s wr, 763 op/s 2024-08-31T06:17:45.026 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:44 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:17:45.026 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:44 smithi016 ceph-mon[17194]: pgmap v689: 97 pgs: 97 active+clean; 3.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 13 MiB/s wr, 763 op/s 2024-08-31T06:17:45.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:44 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:17:45.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:44 smithi155 ceph-mon[21742]: pgmap v689: 97 pgs: 97 active+clean; 3.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 13 MiB/s wr, 763 op/s 2024-08-31T06:17:45.149 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:17:45.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.154+0000 7f859d200640 1 -- 172.21.15.16:0/2260062462 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f859810e360 msgr2=0x7f859810e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:45.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.154+0000 7f859d200640 1 --2- 172.21.15.16:0/2260062462 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f859810e360 0x7f859810e770 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8588009900 tx=0x7f858802a420 comp rx=0 tx=0).stop 2024-08-31T06:17:45.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.154+0000 7f859d200640 1 -- 172.21.15.16:0/2260062462 shutdown_connections 2024-08-31T06:17:45.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.154+0000 7f859d200640 1 --2- 172.21.15.16:0/2260062462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85981145a0 0x7f859811add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:45.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.154+0000 7f859d200640 1 --2- 172.21.15.16:0/2260062462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8598113c50 0x7f8598114060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:45.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.154+0000 7f859d200640 1 --2- 172.21.15.16:0/2260062462 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f859810e360 0x7f859810e770 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:45.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.154+0000 7f859d200640 1 -- 172.21.15.16:0/2260062462 >> 172.21.15.16:0/2260062462 conn(0x7f859810a160 msgr2=0x7f859810c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:45.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.155+0000 7f859d200640 1 -- 172.21.15.16:0/2260062462 shutdown_connections 2024-08-31T06:17:45.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.155+0000 7f859d200640 1 -- 172.21.15.16:0/2260062462 wait complete. 2024-08-31T06:17:45.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.157+0000 7f859d200640 1 Processor -- start 2024-08-31T06:17:45.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.157+0000 7f859d200640 1 -- start start 2024-08-31T06:17:45.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.158+0000 7f859d200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f859810e360 0x7f85981a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:45.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.158+0000 7f859d200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8598113c50 0x7f85981a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:45.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.158+0000 7f859d200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85981145a0 0x7f85981aa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:45.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.158+0000 7f859d200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f859811e2a0 con 0x7f8598113c50 2024-08-31T06:17:45.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.158+0000 7f859d200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f859811e120 con 0x7f85981145a0 2024-08-31T06:17:45.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.158+0000 7f859d200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f859811e420 con 0x7f859810e360 2024-08-31T06:17:45.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.159+0000 7f8595e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f859810e360 0x7f85981a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:45.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.159+0000 7f8595e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f859810e360 0x7f85981a4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:32908/0 (socket says 172.21.15.16:32908) 2024-08-31T06:17:45.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.159+0000 7f8595e00640 1 -- 172.21.15.16:0/2581136909 learned_addr learned my addr 172.21.15.16:0/2581136909 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:45.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.159+0000 7f8596800640 1 --2- 172.21.15.16:0/2581136909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85981145a0 0x7f85981aa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:45.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.159+0000 7f8595400640 1 --2- 172.21.15.16:0/2581136909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8598113c50 0x7f85981a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:45.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.160+0000 7f8595e00640 1 -- 172.21.15.16:0/2581136909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85981145a0 msgr2=0x7f85981aa4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:45.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.160+0000 7f8595e00640 1 --2- 172.21.15.16:0/2581136909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85981145a0 0x7f85981aa4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:45.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.160+0000 7f8595e00640 1 -- 172.21.15.16:0/2581136909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8598113c50 msgr2=0x7f85981a5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:45.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.160+0000 7f8595e00640 1 --2- 172.21.15.16:0/2581136909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8598113c50 0x7f85981a5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:45.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.160+0000 7f8595e00640 1 -- 172.21.15.16:0/2581136909 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85981aabe0 con 0x7f859810e360 2024-08-31T06:17:45.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.160+0000 7f8595400640 1 --2- 172.21.15.16:0/2581136909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8598113c50 0x7f85981a5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:17:45.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.160+0000 7f8595e00640 1 --2- 172.21.15.16:0/2581136909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f859810e360 0x7f85981a4c70 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8588004770 tx=0x7f8588003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:45.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.161+0000 7f858ea00640 1 -- 172.21.15.16:0/2581136909 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f858803f070 con 0x7f859810e360 2024-08-31T06:17:45.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.161+0000 7f858ea00640 1 -- 172.21.15.16:0/2581136909 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f858803bb10 con 0x7f859810e360 2024-08-31T06:17:45.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.161+0000 7f858ea00640 1 -- 172.21.15.16:0/2581136909 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8588035030 con 0x7f859810e360 2024-08-31T06:17:45.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.161+0000 7f859d200640 1 -- 172.21.15.16:0/2581136909 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85981aae70 con 0x7f859810e360 2024-08-31T06:17:45.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.161+0000 7f859d200640 1 -- 172.21.15.16:0/2581136909 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85981b1630 con 0x7f859810e360 2024-08-31T06:17:45.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.162+0000 7f859d200640 1 -- 172.21.15.16:0/2581136909 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f859810e770 con 0x7f859810e360 2024-08-31T06:17:45.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.165+0000 7f858ea00640 1 -- 172.21.15.16:0/2581136909 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f8588042070 con 0x7f859810e360 2024-08-31T06:17:45.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.168+0000 7f858ea00640 1 --2- 172.21.15.16:0/2581136909 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f857c0b4d20 0x7f857c0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:45.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.168+0000 7f858ea00640 1 -- 172.21.15.16:0/2581136909 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8470+0+0 (secure 0 0 0) 0x7f8588105f20 con 0x7f859810e360 2024-08-31T06:17:45.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.169+0000 7f8595400640 1 --2- 172.21.15.16:0/2581136909 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f857c0b4d20 0x7f857c0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:45.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.170+0000 7f8595400640 1 --2- 172.21.15.16:0/2581136909 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f857c0b4d20 0x7f857c0b7230 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f859810d970 tx=0x7f8584005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:45.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.171+0000 7f858ea00640 1 -- 172.21.15.16:0/2581136909 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f8588049030 con 0x7f859810e360 2024-08-31T06:17:45.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.581+0000 7f859d200640 1 -- 172.21.15.16:0/2581136909 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f859802e430 con 0x7f859810e360 2024-08-31T06:17:45.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.582+0000 7f858ea00640 1 -- 172.21.15.16:0/2581136909 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11921 (secure 0 0 0) 0x7f85880ccce0 con 0x7f859810e360 2024-08-31T06:17:45.585 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:17:45.585 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 46 2024-08-31T06:17:45.586 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":46,"btime":"2024-08-31T06:17:38:653550+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:17:38.342854+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24712,24724,24737]},"id":1}]} 2024-08-31T06:17:45.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.586+0000 7f859d200640 1 -- 172.21.15.16:0/2581136909 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f857c0b4d20 msgr2=0x7f857c0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:45.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.586+0000 7f859d200640 1 --2- 172.21.15.16:0/2581136909 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f857c0b4d20 0x7f857c0b7230 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f859810d970 tx=0x7f8584005f90 comp rx=0 tx=0).stop 2024-08-31T06:17:45.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.586+0000 7f859d200640 1 -- 172.21.15.16:0/2581136909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f859810e360 msgr2=0x7f85981a4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:45.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.586+0000 7f859d200640 1 --2- 172.21.15.16:0/2581136909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f859810e360 0x7f85981a4c70 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8588004770 tx=0x7f8588003b90 comp rx=0 tx=0).stop 2024-08-31T06:17:45.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.587+0000 7f859d200640 1 -- 172.21.15.16:0/2581136909 shutdown_connections 2024-08-31T06:17:45.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.587+0000 7f859d200640 1 --2- 172.21.15.16:0/2581136909 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f857c0b4d20 0x7f857c0b7230 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:45.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.587+0000 7f859d200640 1 --2- 172.21.15.16:0/2581136909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85981145a0 0x7f85981aa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:45.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.587+0000 7f859d200640 1 --2- 172.21.15.16:0/2581136909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8598113c50 0x7f85981a5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:45.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.587+0000 7f859d200640 1 --2- 172.21.15.16:0/2581136909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f859810e360 0x7f85981a4c70 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:45.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.587+0000 7f859d200640 1 -- 172.21.15.16:0/2581136909 >> 172.21.15.16:0/2581136909 conn(0x7f859810a160 msgr2=0x7f8598113350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:45.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.587+0000 7f859d200640 1 -- 172.21.15.16:0/2581136909 shutdown_connections 2024-08-31T06:17:45.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.587+0000 7f859d200640 1 -- 172.21.15.16:0/2581136909 wait complete. 2024-08-31T06:17:45.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:45 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/2581136909' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:17:45.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:45 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/2581136909' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:17:46.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:45 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/2581136909' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:17:46.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.956+0000 7f5aaee00640 1 -- 172.21.15.16:0/1025824394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5aa8075b70 msgr2=0x7f5aa8075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:46.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.956+0000 7f5aaee00640 1 --2- 172.21.15.16:0/1025824394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5aa8075b70 0x7f5aa8075f80 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5a94009860 tx=0x7f5a9402a340 comp rx=0 tx=0).stop 2024-08-31T06:17:46.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.957+0000 7f5aaee00640 1 -- 172.21.15.16:0/1025824394 shutdown_connections 2024-08-31T06:17:46.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.957+0000 7f5aaee00640 1 --2- 172.21.15.16:0/1025824394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5aa8114020 0x7f5aa81164f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:46.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.957+0000 7f5aaee00640 1 --2- 172.21.15.16:0/1025824394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5aa8075b70 0x7f5aa8075f80 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:46.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.957+0000 7f5aaee00640 1 --2- 172.21.15.16:0/1025824394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5aa8077120 0x7f5aa8075550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:46.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.957+0000 7f5aaee00640 1 -- 172.21.15.16:0/1025824394 >> 172.21.15.16:0/1025824394 conn(0x7f5aa80ff1e0 msgr2=0x7f5aa8101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:46.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.957+0000 7f5aaee00640 1 -- 172.21.15.16:0/1025824394 shutdown_connections 2024-08-31T06:17:46.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.957+0000 7f5aaee00640 1 -- 172.21.15.16:0/1025824394 wait complete. 2024-08-31T06:17:46.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.958+0000 7f5aaee00640 1 Processor -- start 2024-08-31T06:17:46.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.959+0000 7f5aaee00640 1 -- start start 2024-08-31T06:17:46.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.960+0000 7f5aaee00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5aa8075b70 0x7f5aa81a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:46.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.960+0000 7f5aaee00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5aa8077120 0x7f5aa81a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:46.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.960+0000 7f5aaee00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5aa8114020 0x7f5aa81a5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:46.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.960+0000 7f5aaee00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa8119c60 con 0x7f5aa8077120 2024-08-31T06:17:46.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.960+0000 7f5aaee00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa8119ae0 con 0x7f5aa8075b70 2024-08-31T06:17:46.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.960+0000 7f5aaee00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa8119de0 con 0x7f5aa8114020 2024-08-31T06:17:46.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.961+0000 7f5aae400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5aa8114020 0x7f5aa81a5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:46.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.961+0000 7f5aae400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5aa8114020 0x7f5aa81a5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:57108/0 (socket says 172.21.15.16:57108) 2024-08-31T06:17:46.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.961+0000 7f5aae400640 1 -- 172.21.15.16:0/355237182 learned_addr learned my addr 172.21.15.16:0/355237182 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:46.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.961+0000 7f5aada00640 1 --2- 172.21.15.16:0/355237182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5aa8075b70 0x7f5aa81a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:46.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.961+0000 7f5aad000640 1 --2- 172.21.15.16:0/355237182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5aa8077120 0x7f5aa81a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:46.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.961+0000 7f5aae400640 1 -- 172.21.15.16:0/355237182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5aa8075b70 msgr2=0x7f5aa81a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:46.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.961+0000 7f5aae400640 1 --2- 172.21.15.16:0/355237182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5aa8075b70 0x7f5aa81a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:46.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.961+0000 7f5aae400640 1 -- 172.21.15.16:0/355237182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5aa8077120 msgr2=0x7f5aa81a0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:46.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.961+0000 7f5aae400640 1 --2- 172.21.15.16:0/355237182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5aa8077120 0x7f5aa81a0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:46.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.961+0000 7f5aae400640 1 -- 172.21.15.16:0/355237182 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aa81a66c0 con 0x7f5aa8114020 2024-08-31T06:17:46.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.961+0000 7f5aad000640 1 --2- 172.21.15.16:0/355237182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5aa8077120 0x7f5aa81a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:17:46.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.961+0000 7f5aada00640 1 --2- 172.21.15.16:0/355237182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5aa8075b70 0x7f5aa81a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:17:46.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.962+0000 7f5aae400640 1 --2- 172.21.15.16:0/355237182 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5aa8114020 0x7f5aa81a5f90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5aa000efc0 tx=0x7f5aa000c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:46.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.962+0000 7f5aa6000640 1 -- 172.21.15.16:0/355237182 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa0003970 con 0x7f5aa8114020 2024-08-31T06:17:46.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.962+0000 7f5aa6000640 1 -- 172.21.15.16:0/355237182 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5aa0004020 con 0x7f5aa8114020 2024-08-31T06:17:46.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.962+0000 7f5aa6000640 1 -- 172.21.15.16:0/355237182 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa000b940 con 0x7f5aa8114020 2024-08-31T06:17:46.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.962+0000 7f5aaee00640 1 -- 172.21.15.16:0/355237182 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5aa81a69b0 con 0x7f5aa8114020 2024-08-31T06:17:46.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.962+0000 7f5aaee00640 1 -- 172.21.15.16:0/355237182 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5aa81ad110 con 0x7f5aa8114020 2024-08-31T06:17:46.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.966+0000 7f5ab7082740 1 Processor -- start 2024-08-31T06:17:46.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.969+0000 7f5aa6000640 1 -- 172.21.15.16:0/355237182 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f5aa001d020 con 0x7f5aa8114020 2024-08-31T06:17:46.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.970+0000 7f5ab7082740 1 -- start start 2024-08-31T06:17:46.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.970+0000 7f5ab7082740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b6301070 0x5624b66a6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:46.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.970+0000 7f5ab7082740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624b62d81b0 0x5624b66a6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:46.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.970+0000 7f5ab7082740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624b62e0390 0x5624b66a74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:46.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.970+0000 7f5ab7082740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5624b661d770 con 0x5624b6301070 2024-08-31T06:17:46.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.970+0000 7f5ab7082740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5624b660cd00 con 0x5624b62e0390 2024-08-31T06:17:46.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.970+0000 7f5ab7082740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5624b6610cb0 con 0x5624b62d81b0 2024-08-31T06:17:46.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.971+0000 7f5aad000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624b62e0390 0x5624b66a74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:46.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.971+0000 7f5aada00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624b62d81b0 0x5624b66a6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:46.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.971+0000 7f5aad000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624b62e0390 0x5624b66a74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:44030/0 (socket says 172.21.15.16:44030) 2024-08-31T06:17:46.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.971+0000 7f5aad000640 1 -- 172.21.15.16:0/2999454916 learned_addr learned my addr 172.21.15.16:0/2999454916 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:46.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.971+0000 7f5aad000640 1 --2- 172.21.15.16:0/2999454916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b6301070 0x5624b66a6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:46.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.972+0000 7f5aad000640 1 -- 172.21.15.16:0/2999454916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624b62d81b0 msgr2=0x5624b66a6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:46.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.972+0000 7f5aad000640 1 --2- 172.21.15.16:0/2999454916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624b62d81b0 0x5624b66a6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:46.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.972+0000 7f5aad000640 1 -- 172.21.15.16:0/2999454916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b6301070 msgr2=0x5624b66a6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:46.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.972+0000 7f5aad000640 1 --2- 172.21.15.16:0/2999454916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b6301070 0x5624b66a6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:46.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.972+0000 7f5aad000640 1 -- 172.21.15.16:0/2999454916 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624b636c650 con 0x5624b62e0390 2024-08-31T06:17:46.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.973+0000 7f5aad000640 1 --2- 172.21.15.16:0/2999454916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624b62e0390 0x5624b66a74b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5aa806b730 tx=0x7f5a94005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1965aa7cdda19ad6 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:46.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.973+0000 7f5aa6000640 1 --2- 172.21.15.16:0/355237182 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f5a900b4d20 0x7f5a900b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:46.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.973+0000 7f5a9aa00640 1 -- 172.21.15.16:0/2999454916 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a94037070 con 0x5624b62e0390 2024-08-31T06:17:46.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.973+0000 7f5a9aa00640 1 -- 172.21.15.16:0/2999454916 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a94038040 con 0x5624b62e0390 2024-08-31T06:17:46.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.973+0000 7f5a9aa00640 1 -- 172.21.15.16:0/2999454916 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a940053d0 con 0x5624b62e0390 2024-08-31T06:17:46.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.973+0000 7f5aa6000640 1 -- 172.21.15.16:0/355237182 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8470+0+0 (secure 0 0 0) 0x7f5aa00e5410 con 0x7f5aa8114020 2024-08-31T06:17:46.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.974+0000 7f5aada00640 1 --2- 172.21.15.16:0/355237182 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f5a900b4d20 0x7f5a900b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:46.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.974+0000 7f5ab7082740 1 -- 172.21.15.16:0/2999454916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624b62e0390 msgr2=0x5624b66a74b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:46.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.974+0000 7f5ab7082740 1 --2- 172.21.15.16:0/2999454916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624b62e0390 0x5624b66a74b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5aa806b730 tx=0x7f5a94005f90 comp rx=0 tx=0).stop 2024-08-31T06:17:46.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.974+0000 7f5ab7082740 1 -- 172.21.15.16:0/2999454916 shutdown_connections 2024-08-31T06:17:46.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.974+0000 7f5ab7082740 1 --2- 172.21.15.16:0/2999454916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624b62d81b0 0x5624b66a6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:46.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.974+0000 7f5ab7082740 1 --2- 172.21.15.16:0/2999454916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624b62e0390 0x5624b66a74b0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:46.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.974+0000 7f5ab7082740 1 --2- 172.21.15.16:0/2999454916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b6301070 0x5624b66a6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:46.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.974+0000 7f5ab7082740 1 -- 172.21.15.16:0/2999454916 >> 172.21.15.16:0/2999454916 conn(0x5624b62f5460 msgr2=0x5624b66a4920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:46.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.975+0000 7f5ab7082740 1 -- 172.21.15.16:0/2999454916 shutdown_connections 2024-08-31T06:17:46.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.975+0000 7f5ab7082740 1 -- 172.21.15.16:0/2999454916 wait complete. 2024-08-31T06:17:46.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.975+0000 7f5aada00640 1 --2- 172.21.15.16:0/355237182 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f5a900b4d20 0x7f5a900b7230 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x5624b66a56b0 tx=0x7f5a9c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:46.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.976+0000 7f5ab7082740 1 -- start start 2024-08-31T06:17:46.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.976+0000 7f5ab7082740 1 Processor -- start 2024-08-31T06:17:46.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.976+0000 7f5ab7082740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624b62d81b0 0x5624b66b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:46.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.977+0000 7f5ab7082740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624b62e0390 0x5624b66b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:46.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.977+0000 7f5ab7082740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b6301070 0x5624b66b84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:46.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.977+0000 7f5ab7082740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5624b646f700 con 0x5624b6301070 2024-08-31T06:17:46.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.977+0000 7f5ab7082740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5624b6593cb0 con 0x5624b62d81b0 2024-08-31T06:17:46.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.977+0000 7f5ab7082740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5624b66b51b0 con 0x5624b62e0390 2024-08-31T06:17:46.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.977+0000 7f5ab7082740 10 client.0 _ll_register_callbacks cb 0x5624b65b9430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:17:46.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.977+0000 7f5aada00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624b62d81b0 0x5624b66b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:46.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.977+0000 7f5aada00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624b62d81b0 0x5624b66b2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:44036/0 (socket says 172.21.15.16:44036) 2024-08-31T06:17:46.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.977+0000 7f5aada00640 1 -- 172.21.15.16:0/1888876146 learned_addr learned my addr 172.21.15.16:0/1888876146 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:46.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.977+0000 7f5aad000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624b62e0390 0x5624b66b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:46.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.977+0000 7f5aae400640 1 --2- 172.21.15.16:0/1888876146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b6301070 0x5624b66b84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:46.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.978+0000 7f5aad000640 1 -- 172.21.15.16:0/1888876146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624b62d81b0 msgr2=0x5624b66b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:46.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.978+0000 7f5aad000640 1 --2- 172.21.15.16:0/1888876146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624b62d81b0 0x5624b66b2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:46.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.978+0000 7f5aad000640 1 -- 172.21.15.16:0/1888876146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b6301070 msgr2=0x5624b66b84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:46.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.978+0000 7f5aad000640 1 --2- 172.21.15.16:0/1888876146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b6301070 0x5624b66b84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:46.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.978+0000 7f5aad000640 1 -- 172.21.15.16:0/1888876146 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624b65d4870 con 0x5624b62e0390 2024-08-31T06:17:46.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.978+0000 7f5aad000640 1 --2- 172.21.15.16:0/1888876146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624b62e0390 0x5624b66b2cc0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5a94009830 tx=0x7f5a940056f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=691caa810d8b005a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:46.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.979+0000 7f5a8f400640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:17:46.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.979+0000 7f5a8f400640 1 -- 172.21.15.16:0/1888876146 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9404e060 con 0x5624b62e0390 2024-08-31T06:17:46.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.979+0000 7f5a8f400640 1 -- 172.21.15.16:0/1888876146 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8470+0+0 (secure 0 0 0) 0x7f5a94030450 con 0x5624b62e0390 2024-08-31T06:17:46.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.979+0000 7f5ab7082740 1 -- 172.21.15.16:0/1888876146 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a94038040 con 0x5624b62e0390 2024-08-31T06:17:46.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.980+0000 7f5a8f400640 1 -- 172.21.15.16:0/1888876146 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f5a94037070 con 0x5624b62e0390 2024-08-31T06:17:46.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.980+0000 7f5ab7082740 10 client.27979 fetch_fsmap learned FSMap version 46 2024-08-31T06:17:46.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.980+0000 7f5ab7082740 1 -- 172.21.15.16:0/1888876146 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5624b66c12e0 con 0x5624b62e0390 2024-08-31T06:17:46.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.980+0000 7f5a8f400640 1 -- 172.21.15.16:0/1888876146 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a94038040 con 0x5624b62e0390 2024-08-31T06:17:46.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.980+0000 7f5a8f400640 1 -- 172.21.15.16:0/1888876146 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7f5a94032440 con 0x5624b62e0390 2024-08-31T06:17:46.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.980+0000 7f5ab7082740 10 client.27979 fetch_fsmap finished waiting for FSMap version 46 2024-08-31T06:17:46.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.980+0000 7f5ab7082740 10 client.27979 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:17:46.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.980+0000 7f5ab7082740 20 client.27979 populate_metadata read hostname 'smithi016' 2024-08-31T06:17:46.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.981+0000 7f5ab7082740 1 --2- 172.21.15.16:0/1888876146 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5624b62bf9f0 0x5624b66c4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:46.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.981+0000 7f5ab7082740 4 client.27979 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:17:46.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.981+0000 7f5ab7082740 1 -- 172.21.15.16:0/1888876146 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624b661f6c0 con 0x5624b62bf9f0 2024-08-31T06:17:46.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.981+0000 7f5aada00640 1 --2- 172.21.15.16:0/1888876146 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5624b62bf9f0 0x5624b66c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:46.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.982+0000 7f5a8f400640 10 client.27979 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:17:46.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.982+0000 7f5aada00640 1 --2- 172.21.15.16:0/1888876146 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5624b62bf9f0 0x5624b66c4640 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be66840d7f3b3a6 server_cookie=16fe3e973f85f788 in_seq=0 out_seq=0 2024-08-31T06:17:46.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.983+0000 7f5a8f400640 1 -- 172.21.15.16:0/1888876146 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5624b661f6c0 con 0x5624b62bf9f0 2024-08-31T06:17:46.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.983+0000 7f5a8f400640 10 client.27979 handle_command_reply: tid=0 2024-08-31T06:17:46.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.983+0000 7f5ab7082740 1 client.27979 shutdown 2024-08-31T06:17:46.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.984+0000 7f5ab7082740 1 -- 172.21.15.16:0/1888876146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624b62e0390 msgr2=0x5624b66b2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:46.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.984+0000 7f5ab7082740 1 --2- 172.21.15.16:0/1888876146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624b62e0390 0x5624b66b2cc0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5a94009830 tx=0x7f5a940056f0 comp rx=0 tx=0).stop 2024-08-31T06:17:46.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.984+0000 7f5ab7082740 1 -- 172.21.15.16:0/1888876146 shutdown_connections 2024-08-31T06:17:46.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.984+0000 7f5ab7082740 1 --2- 172.21.15.16:0/1888876146 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5624b62bf9f0 0x5624b66c4640 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:46.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.984+0000 7f5a8f400640 0 client.27979 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:17:46.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.984+0000 7f5ab7082740 1 --2- 172.21.15.16:0/1888876146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624b62e0390 0x5624b66b2cc0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:46.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.984+0000 7f5ab7082740 1 --2- 172.21.15.16:0/1888876146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b6301070 0x5624b66b84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:46.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.984+0000 7f5ab7082740 1 --2- 172.21.15.16:0/1888876146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624b62d81b0 0x5624b66b2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:46.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.984+0000 7f5ab7082740 1 -- 172.21.15.16:0/1888876146 >> 172.21.15.16:0/1888876146 conn(0x5624b62f5460 msgr2=0x5624b66a9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:46.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.984+0000 7f5ab7082740 1 -- 172.21.15.16:0/1888876146 shutdown_connections 2024-08-31T06:17:46.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.984+0000 7f5ab7082740 1 -- 172.21.15.16:0/1888876146 wait complete. 2024-08-31T06:17:46.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:45.984+0000 7f5ab7082740 20 client.27979 trim_cache size 0 max 16384 2024-08-31T06:17:46.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.013+0000 7f5ab7082740 1 Processor -- start 2024-08-31T06:17:46.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.014+0000 7f5ab7082740 1 -- start start 2024-08-31T06:17:46.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.014+0000 7f5ab7082740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b62d81b0 0x5624b66abb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:46.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.014+0000 7f5ab7082740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624b62e0390 0x5624b66ac050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:46.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.014+0000 7f5ab7082740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624b6301070 0x5624b66af710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:46.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.014+0000 7f5ab7082740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa81b8370 con 0x5624b62d81b0 2024-08-31T06:17:46.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.014+0000 7f5ab7082740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa81b5bb0 con 0x5624b6301070 2024-08-31T06:17:46.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.014+0000 7f5ab7082740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa81b3a20 con 0x5624b62e0390 2024-08-31T06:17:46.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.015+0000 7f5aae400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624b6301070 0x5624b66af710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:46.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.015+0000 7f5aae400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624b6301070 0x5624b66af710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:44042/0 (socket says 172.21.15.16:44042) 2024-08-31T06:17:46.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.015+0000 7f5aae400640 1 -- 172.21.15.16:0/1182754098 learned_addr learned my addr 172.21.15.16:0/1182754098 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:46.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.015+0000 7f5aad000640 1 --2- 172.21.15.16:0/1182754098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624b62e0390 0x5624b66ac050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:46.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.015+0000 7f5aada00640 1 --2- 172.21.15.16:0/1182754098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b62d81b0 0x5624b66abb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:46.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.016+0000 7f5aae400640 1 -- 172.21.15.16:0/1182754098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624b62e0390 msgr2=0x5624b66ac050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:46.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.016+0000 7f5aae400640 1 --2- 172.21.15.16:0/1182754098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624b62e0390 0x5624b66ac050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:46.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.016+0000 7f5aae400640 1 -- 172.21.15.16:0/1182754098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b62d81b0 msgr2=0x5624b66abb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:46.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.016+0000 7f5aae400640 1 --2- 172.21.15.16:0/1182754098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b62d81b0 0x5624b66abb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:46.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.016+0000 7f5aae400640 1 -- 172.21.15.16:0/1182754098 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624b66b37e0 con 0x5624b6301070 2024-08-31T06:17:46.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.016+0000 7f5aad000640 1 --2- 172.21.15.16:0/1182754098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624b62e0390 0x5624b66ac050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:17:46.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.017+0000 7f5aada00640 1 --2- 172.21.15.16:0/1182754098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b62d81b0 0x5624b66abb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:17:46.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.017+0000 7f5aae400640 1 --2- 172.21.15.16:0/1182754098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624b6301070 0x5624b66af710 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5aa000cd00 tx=0x7f5aa00a72c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5a79a30c65d7348 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:46.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.018+0000 7f5a9a000640 1 -- 172.21.15.16:0/1182754098 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa00b0070 con 0x5624b6301070 2024-08-31T06:17:46.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.018+0000 7f5a9a000640 1 -- 172.21.15.16:0/1182754098 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5aa00abb30 con 0x5624b6301070 2024-08-31T06:17:46.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.018+0000 7f5a9a000640 1 -- 172.21.15.16:0/1182754098 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa00ace60 con 0x5624b6301070 2024-08-31T06:17:46.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.019+0000 7f5ab7082740 1 -- 172.21.15.16:0/1182754098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624b6301070 msgr2=0x5624b66af710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:46.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.019+0000 7f5ab7082740 1 --2- 172.21.15.16:0/1182754098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624b6301070 0x5624b66af710 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5aa000cd00 tx=0x7f5aa00a72c0 comp rx=0 tx=0).stop 2024-08-31T06:17:46.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.019+0000 7f5ab7082740 1 -- 172.21.15.16:0/1182754098 shutdown_connections 2024-08-31T06:17:46.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.019+0000 7f5ab7082740 1 --2- 172.21.15.16:0/1182754098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624b62e0390 0x5624b66ac050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:46.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.019+0000 7f5ab7082740 1 --2- 172.21.15.16:0/1182754098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624b6301070 0x5624b66af710 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:46.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.019+0000 7f5ab7082740 1 --2- 172.21.15.16:0/1182754098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b62d81b0 0x5624b66abb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:46.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.019+0000 7f5ab7082740 1 -- 172.21.15.16:0/1182754098 >> 172.21.15.16:0/1182754098 conn(0x5624b62f5460 msgr2=0x5624b66a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:46.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.019+0000 7f5ab7082740 1 -- 172.21.15.16:0/1182754098 shutdown_connections 2024-08-31T06:17:46.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.019+0000 7f5ab7082740 1 -- 172.21.15.16:0/1182754098 wait complete. 2024-08-31T06:17:46.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.020+0000 7f5ab7082740 1 -- start start 2024-08-31T06:17:46.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.020+0000 7f5ab7082740 1 Processor -- start 2024-08-31T06:17:46.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.021+0000 7f5ab7082740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624b62d81b0 0x5624b66bebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:46.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.021+0000 7f5ab7082740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b62e0390 0x5624b66bd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:46.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.021+0000 7f5ab7082740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624b6301070 0x5624b66bd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:46.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.021+0000 7f5ab7082740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa81b4070 con 0x5624b62e0390 2024-08-31T06:17:46.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.021+0000 7f5ab7082740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa81b54e0 con 0x5624b62d81b0 2024-08-31T06:17:46.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.021+0000 7f5ab7082740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa81b4740 con 0x5624b6301070 2024-08-31T06:17:46.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.021+0000 7f5ab7082740 10 client.0 _ll_register_callbacks cb 0x7f5a940096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:17:46.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.021+0000 7f5aad000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b62e0390 0x5624b66bd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:46.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.021+0000 7f5aada00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624b62d81b0 0x5624b66bebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:46.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.021+0000 7f5aad000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b62e0390 0x5624b66bd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33998/0 (socket says 172.21.15.16:33998) 2024-08-31T06:17:46.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.021+0000 7f5aad000640 1 -- 172.21.15.16:0/1542561974 learned_addr learned my addr 172.21.15.16:0/1542561974 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:46.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.021+0000 7f5aae400640 1 --2- 172.21.15.16:0/1542561974 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624b6301070 0x5624b66bd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:46.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.022+0000 7f5aada00640 1 -- 172.21.15.16:0/1542561974 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624b6301070 msgr2=0x5624b66bd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:46.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.022+0000 7f5aada00640 1 --2- 172.21.15.16:0/1542561974 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624b6301070 0x5624b66bd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:46.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.022+0000 7f5aada00640 1 -- 172.21.15.16:0/1542561974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b62e0390 msgr2=0x5624b66bd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:46.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.022+0000 7f5aada00640 1 --2- 172.21.15.16:0/1542561974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b62e0390 0x5624b66bd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:46.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.022+0000 7f5aada00640 1 -- 172.21.15.16:0/1542561974 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624b66bdde0 con 0x5624b62d81b0 2024-08-31T06:17:46.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.022+0000 7f5aada00640 1 --2- 172.21.15.16:0/1542561974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624b62d81b0 0x5624b66bebb0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x5624b661d7c0 tx=0x7f5a9c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be049443c53477c2 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:46.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.023+0000 7f5a8f400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:17:46.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.023+0000 7f5a8f400640 1 -- 172.21.15.16:0/1542561974 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c034020 con 0x5624b62d81b0 2024-08-31T06:17:46.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.023+0000 7f5ab7082740 1 -- 172.21.15.16:0/1542561974 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5aa00abb30 con 0x5624b62d81b0 2024-08-31T06:17:46.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.023+0000 7f5a8f400640 1 -- 172.21.15.16:0/1542561974 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8470+0+0 (secure 0 0 0) 0x7f5a9c02d040 con 0x5624b62d81b0 2024-08-31T06:17:46.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.024+0000 7f5a8f400640 1 -- 172.21.15.16:0/1542561974 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f5a9c034020 con 0x5624b62d81b0 2024-08-31T06:17:46.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.024+0000 7f5ab7082740 10 client.27983 fetch_fsmap learned FSMap version 46 2024-08-31T06:17:46.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.024+0000 7f5ab7082740 1 -- 172.21.15.16:0/1542561974 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5624b672fda0 con 0x5624b62d81b0 2024-08-31T06:17:46.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.024+0000 7f5a8f400640 1 -- 172.21.15.16:0/1542561974 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5aa00abb30 con 0x5624b62d81b0 2024-08-31T06:17:46.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.025+0000 7f5a8f400640 1 -- 172.21.15.16:0/1542561974 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7f5a9c02b3f0 con 0x5624b62d81b0 2024-08-31T06:17:46.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.025+0000 7f5ab7082740 10 client.27983 fetch_fsmap finished waiting for FSMap version 46 2024-08-31T06:17:46.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.025+0000 7f5ab7082740 10 client.27983 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:17:46.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.025+0000 7f5ab7082740 20 client.27983 populate_metadata read hostname 'smithi016' 2024-08-31T06:17:46.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.025+0000 7f5ab7082740 1 --2- 172.21.15.16:0/1542561974 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5624b6294da0 0x5624b6732520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:46.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.025+0000 7f5ab7082740 4 client.27983 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T06:17:46.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.025+0000 7f5ab7082740 1 -- 172.21.15.16:0/1542561974 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5624b661b160 con 0x5624b6294da0 2024-08-31T06:17:46.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.025+0000 7f5aad000640 1 --2- 172.21.15.16:0/1542561974 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5624b6294da0 0x5624b6732520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:46.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.026+0000 7f5aad000640 1 --2- 172.21.15.16:0/1542561974 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5624b6294da0 0x5624b6732520 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffb218020bc810a4 server_cookie=5f98dc515aeeccb7 in_seq=0 out_seq=0 2024-08-31T06:17:46.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.026+0000 7f5a8f400640 10 client.27983 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:17:46.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.026+0000 7f5a8f400640 1 -- 172.21.15.16:0/1542561974 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4262 (crc 0 0 0) 0x5624b661b160 con 0x5624b6294da0 2024-08-31T06:17:46.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.026+0000 7f5a8f400640 10 client.27983 handle_command_reply: tid=0 2024-08-31T06:17:46.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.026+0000 7f5ab7082740 1 client.27983 shutdown 2024-08-31T06:17:46.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.026+0000 7f5ab7082740 1 -- 172.21.15.16:0/1542561974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624b62d81b0 msgr2=0x5624b66bebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:46.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.026+0000 7f5ab7082740 1 --2- 172.21.15.16:0/1542561974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624b62d81b0 0x5624b66bebb0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x5624b661d7c0 tx=0x7f5a9c0046c0 comp rx=0 tx=0).stop 2024-08-31T06:17:46.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.026+0000 7f5ab7082740 1 -- 172.21.15.16:0/1542561974 shutdown_connections 2024-08-31T06:17:46.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.026+0000 7f5ab7082740 1 --2- 172.21.15.16:0/1542561974 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5624b6294da0 0x5624b6732520 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:46.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.026+0000 7f5ab7082740 1 --2- 172.21.15.16:0/1542561974 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624b6301070 0x5624b66bd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:46.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.026+0000 7f5ab7082740 1 --2- 172.21.15.16:0/1542561974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624b62e0390 0x5624b66bd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:46.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.026+0000 7f5ab7082740 1 --2- 172.21.15.16:0/1542561974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5624b62d81b0 0x5624b66bebb0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:46.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.026+0000 7f5ab7082740 1 -- 172.21.15.16:0/1542561974 >> 172.21.15.16:0/1542561974 conn(0x5624b62f5460 msgr2=0x5624b66a7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:46.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.027+0000 7f5a8f400640 0 client.27983 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:17:46.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.027+0000 7f5ab7082740 1 -- 172.21.15.16:0/1542561974 shutdown_connections 2024-08-31T06:17:46.172 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:17:46.172 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T06:17:46.173 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-08-31T06:17:46.173 INFO:teuthology.orchestra.run.smithi016.stdout: "e497cd88-1e7c-4631-a4a8-cdd8e1fa6c83": { 2024-08-31T06:17:46.173 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:17:46.173 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "e497cd88-1e7c-4631-a4a8-cdd8e1fa6c83", 2024-08-31T06:17:46.173 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-08-31T06:17:46.173 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:17:46.173 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:17:46.173 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-08-31T06:17:46.173 INFO:teuthology.orchestra.run.smithi016.stdout: "e497cd88-1e7c-4631-a4a8-cdd8e1fa6c83": { 2024-08-31T06:17:46.173 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:17:42.476795+0000", 2024-08-31T06:17:46.173 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:17:46.173 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:17:46.173 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:17:46.173 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:17:46.174 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:17:46.174 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:17:46.174 INFO:teuthology.orchestra.run.smithi016.stdout: "c300b7e8-f3fd-482c-a909-9329bc61e570": { 2024-08-31T06:17:46.174 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:06:49.868046+0000", 2024-08-31T06:17:46.174 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:17:46.174 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:17:46.174 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:17:46.174 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:17:46.174 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:17:46.174 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:17:46.174 INFO:teuthology.orchestra.run.smithi016.stdout: "6a03046e-cdf4-4a14-b9bd-b560b8968467": { 2024-08-31T06:17:46.174 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:12:38.002354+0000", 2024-08-31T06:17:46.174 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:17:46.174 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:17:46.175 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:17:46.175 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:17:46.175 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:17:46.175 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:17:46.175 INFO:teuthology.orchestra.run.smithi016.stdout: "004b47bb-0fe8-4d28-9d9d-78bde6a61199": { 2024-08-31T06:17:46.175 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:16:48.915926+0000", 2024-08-31T06:17:46.175 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:17:46.175 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:17:46.175 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:17:46.175 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:17:46.175 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:17:46.175 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:17:46.175 INFO:teuthology.orchestra.run.smithi016.stdout: "88d7bdb8-cb1c-45e5-ad29-4cd8da6c6003": { 2024-08-31T06:17:46.175 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:07:39.648942+0000", 2024-08-31T06:17:46.176 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:17:46.176 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:17:46.176 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:17:46.176 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:17:46.176 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:17:46.176 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:17:46.176 INFO:teuthology.orchestra.run.smithi016.stdout: "0fdcd762-533d-42be-9289-b71c8e3b81b1": { 2024-08-31T06:17:46.176 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:15:06.169112+0000", 2024-08-31T06:17:46.176 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:17:46.176 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:17:46.176 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:17:46.176 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:17:46.176 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:17:46.177 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:17:46.177 INFO:teuthology.orchestra.run.smithi016.stdout: "c8455d74-2fe7-4d93-bcf2-19b419d16016": { 2024-08-31T06:17:46.177 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:08:24.035960+0000", 2024-08-31T06:17:46.177 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:17:46.177 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:17:46.177 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:17:46.177 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:17:46.177 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:17:46.177 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:17:46.177 INFO:teuthology.orchestra.run.smithi016.stdout: "78d6ef02-8551-45e8-ad60-9627eb573fc1": { 2024-08-31T06:17:46.177 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:09:13.547865+0000", 2024-08-31T06:17:46.177 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:17:46.177 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:17:46.177 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:17:46.177 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:17:46.178 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:17:46.178 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:17:46.178 INFO:teuthology.orchestra.run.smithi016.stdout: "c1fd6904-2e02-47fa-939a-47349b9fe36b": { 2024-08-31T06:17:46.178 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:06.678691+0000", 2024-08-31T06:17:46.178 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:17:46.178 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:17:46.178 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:17:46.178 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:17:46.178 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:17:46.178 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:17:46.178 INFO:teuthology.orchestra.run.smithi016.stdout: "a10cce76-99ab-46b5-b3b2-07b82899b8f9": { 2024-08-31T06:17:46.178 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:11:43.967546+0000", 2024-08-31T06:17:46.178 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:17:46.178 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:17:46.178 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:17:46.178 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:17:46.179 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:17:46.179 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:17:46.179 INFO:teuthology.orchestra.run.smithi016.stdout: "91b72a34-4ba9-4126-a414-8b862f8c5f78": { 2024-08-31T06:17:46.179 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:13:28.663237+0000", 2024-08-31T06:17:46.179 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:17:46.179 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:17:46.179 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:17:46.179 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:17:46.179 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:17:46.179 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:17:46.179 INFO:teuthology.orchestra.run.smithi016.stdout: "e739a16e-8ace-43a8-a1ce-ac27d39ef8a3": { 2024-08-31T06:17:46.179 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:54.084646+0000", 2024-08-31T06:17:46.179 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:17:46.179 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:17:46.179 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:17:46.179 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:17:46.180 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:17:46.180 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:17:46.180 INFO:teuthology.orchestra.run.smithi016.stdout: "a64d5301-8558-4e9c-ac9b-8d0ac7057e74": { 2024-08-31T06:17:46.180 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:14:20.338104+0000", 2024-08-31T06:17:46.180 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:17:46.180 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:17:46.180 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:17:46.180 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:17:46.180 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:17:46.180 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:17:46.180 INFO:teuthology.orchestra.run.smithi016.stdout: "f9c4a22e-0c8a-4928-9b0d-e2ce1db5a86b": { 2024-08-31T06:17:46.180 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:15:57.496015+0000", 2024-08-31T06:17:46.180 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:17:46.180 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:17:46.180 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:17:46.181 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:17:46.181 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:17:46.181 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:17:46.181 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:17:46.181 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:17:46.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.027+0000 7f5ab7082740 1 -- 172.21.15.16:0/1542561974 wait complete. 2024-08-31T06:17:46.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.027+0000 7f5ab7082740 20 client.27983 trim_cache size 0 max 16384 2024-08-31T06:17:46.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.028+0000 7f5aaee00640 1 -- 172.21.15.16:0/355237182 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f5a900b4d20 msgr2=0x7f5a900b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:46.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.028+0000 7f5aaee00640 1 --2- 172.21.15.16:0/355237182 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f5a900b4d20 0x7f5a900b7230 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x5624b66a56b0 tx=0x7f5a9c005f90 comp rx=0 tx=0).stop 2024-08-31T06:17:46.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.028+0000 7f5aaee00640 1 -- 172.21.15.16:0/355237182 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5aa8114020 msgr2=0x7f5aa81a5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:46.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.028+0000 7f5aaee00640 1 --2- 172.21.15.16:0/355237182 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5aa8114020 0x7f5aa81a5f90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5aa000efc0 tx=0x7f5aa000c800 comp rx=0 tx=0).stop 2024-08-31T06:17:46.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.028+0000 7f5aaee00640 1 -- 172.21.15.16:0/355237182 shutdown_connections 2024-08-31T06:17:46.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.028+0000 7f5aaee00640 1 --2- 172.21.15.16:0/355237182 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f5a900b4d20 0x7f5a900b7230 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:46.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.028+0000 7f5aaee00640 1 --2- 172.21.15.16:0/355237182 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5aa8114020 0x7f5aa81a5f90 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:46.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.028+0000 7f5aaee00640 1 --2- 172.21.15.16:0/355237182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5aa8077120 0x7f5aa81a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:46.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.028+0000 7f5aaee00640 1 --2- 172.21.15.16:0/355237182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5aa8075b70 0x7f5aa81a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:46.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.028+0000 7f5aaee00640 1 -- 172.21.15.16:0/355237182 >> 172.21.15.16:0/355237182 conn(0x7f5aa80ff1e0 msgr2=0x7f5aa810ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:46.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.029+0000 7f5aaee00640 1 -- 172.21.15.16:0/355237182 shutdown_connections 2024-08-31T06:17:46.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:46.029+0000 7f5aaee00640 1 -- 172.21.15.16:0/355237182 wait complete. 2024-08-31T06:17:46.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 620.0 s, 1332.4 tps, lat 74.841 ms stddev 86.387 2024-08-31T06:17:46.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:46 smithi016 ceph-mon[17194]: pgmap v690: 97 pgs: 97 active+clean; 3.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 15 MiB/s wr, 872 op/s 2024-08-31T06:17:47.001 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24737 quiesce db / --timeout 90 --expiration 145.25 --await-for 45.0' 2024-08-31T06:17:47.001 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24737 quiesce db / --timeout 90 --expiration 145.25 --await-for 45.0 2024-08-31T06:17:47.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:46 smithi155 ceph-mon[21742]: pgmap v690: 97 pgs: 97 active+clean; 3.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 15 MiB/s wr, 872 op/s 2024-08-31T06:17:47.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:46 smithi184 ceph-mon[22331]: pgmap v690: 97 pgs: 97 active+clean; 3.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 15 MiB/s wr, 872 op/s 2024-08-31T06:17:47.444 INFO:tasks.cephfs.filesystem:scrub status for tag:e497cd88-1e7c-4631-a4a8-cdd8e1fa6c83 - {'path': '/', 'tag': 'e497cd88-1e7c-4631-a4a8-cdd8e1fa6c83', 'options': 'recursive,force'} 2024-08-31T06:17:47.625 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:17:48.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.471+0000 7f9eac600640 1 -- 172.21.15.16:0/1158037025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ea410a160 msgr2=0x7f9ea410a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:48.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.471+0000 7f9eac600640 1 --2- 172.21.15.16:0/1158037025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ea410a160 0x7f9ea410a570 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f9e98009900 tx=0x7f9e9802a480 comp rx=0 tx=0).stop 2024-08-31T06:17:48.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.471+0000 7f9eac600640 1 -- 172.21.15.16:0/1158037025 shutdown_connections 2024-08-31T06:17:48.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.471+0000 7f9eac600640 1 --2- 172.21.15.16:0/1158037025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ea41186a0 0x7f9ea411aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:48.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.471+0000 7f9eac600640 1 --2- 172.21.15.16:0/1158037025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ea410aab0 0x7f9ea4118160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:48.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.471+0000 7f9eac600640 1 --2- 172.21.15.16:0/1158037025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ea410a160 0x7f9ea410a570 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:48.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.471+0000 7f9eac600640 1 -- 172.21.15.16:0/1158037025 >> 172.21.15.16:0/1158037025 conn(0x7f9ea4070c20 msgr2=0x7f9ea4071030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:48.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.471+0000 7f9eac600640 1 -- 172.21.15.16:0/1158037025 shutdown_connections 2024-08-31T06:17:48.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.471+0000 7f9eac600640 1 -- 172.21.15.16:0/1158037025 wait complete. 2024-08-31T06:17:48.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.473+0000 7f9eac600640 1 Processor -- start 2024-08-31T06:17:48.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.474+0000 7f9eac600640 1 -- start start 2024-08-31T06:17:48.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.474+0000 7f9eac600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ea410a160 0x7f9ea41155f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:48.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.474+0000 7f9eac600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ea410aab0 0x7f9ea4115b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:48.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.474+0000 7f9eac600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ea41186a0 0x7f9ea4116070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:48.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.474+0000 7f9eac600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea411e290 con 0x7f9ea410aab0 2024-08-31T06:17:48.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.475+0000 7f9eac600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea411e110 con 0x7f9ea410a160 2024-08-31T06:17:48.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.475+0000 7f9eac600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea411e410 con 0x7f9ea41186a0 2024-08-31T06:17:48.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.475+0000 7f9ea9c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ea410a160 0x7f9ea41155f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:48.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.475+0000 7f9eaa600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ea41186a0 0x7f9ea4116070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:48.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.475+0000 7f9eaa600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ea41186a0 0x7f9ea4116070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:57158/0 (socket says 172.21.15.16:57158) 2024-08-31T06:17:48.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.475+0000 7f9eaa600640 1 -- 172.21.15.16:0/1740381389 learned_addr learned my addr 172.21.15.16:0/1740381389 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:48.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.475+0000 7f9ea9c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ea410a160 0x7f9ea41155f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:44068/0 (socket says 172.21.15.16:44068) 2024-08-31T06:17:48.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.476+0000 7f9eaa600640 1 -- 172.21.15.16:0/1740381389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ea410a160 msgr2=0x7f9ea41155f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:48.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.476+0000 7f9eaa600640 1 --2- 172.21.15.16:0/1740381389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ea410a160 0x7f9ea41155f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:48.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.476+0000 7f9eaa600640 1 -- 172.21.15.16:0/1740381389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ea410aab0 msgr2=0x7f9ea4115b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:48.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.476+0000 7f9ea9200640 1 --2- 172.21.15.16:0/1740381389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ea410aab0 0x7f9ea4115b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:48.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.477+0000 7f9eaa600640 1 --2- 172.21.15.16:0/1740381389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ea410aab0 0x7f9ea4115b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:48.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.477+0000 7f9eaa600640 1 -- 172.21.15.16:0/1740381389 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ea41a3890 con 0x7f9ea41186a0 2024-08-31T06:17:48.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.477+0000 7f9ea9c00640 1 --2- 172.21.15.16:0/1740381389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ea410a160 0x7f9ea41155f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:17:48.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.477+0000 7f9ea9200640 1 --2- 172.21.15.16:0/1740381389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ea410aab0 0x7f9ea4115b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:17:48.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.478+0000 7f9eaa600640 1 --2- 172.21.15.16:0/1740381389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ea41186a0 0x7f9ea4116070 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00eb40 tx=0x7f9e9c00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:48.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.478+0000 7f9ea2000640 1 -- 172.21.15.16:0/1740381389 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c004b20 con 0x7f9ea41186a0 2024-08-31T06:17:48.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.479+0000 7f9ea2000640 1 -- 172.21.15.16:0/1740381389 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e9c004cc0 con 0x7f9ea41186a0 2024-08-31T06:17:48.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.479+0000 7f9eac600640 1 -- 172.21.15.16:0/1740381389 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ea41a3b80 con 0x7f9ea41186a0 2024-08-31T06:17:48.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.479+0000 7f9ea2000640 1 -- 172.21.15.16:0/1740381389 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c010720 con 0x7f9ea41186a0 2024-08-31T06:17:48.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.479+0000 7f9eac600640 1 -- 172.21.15.16:0/1740381389 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ea41a4110 con 0x7f9ea41186a0 2024-08-31T06:17:48.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.485+0000 7f9ea2000640 1 -- 172.21.15.16:0/1740381389 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f9e9c005560 con 0x7f9ea41186a0 2024-08-31T06:17:48.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.486+0000 7f9eb328a740 1 Processor -- start 2024-08-31T06:17:48.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.487+0000 7f9eb328a740 1 -- start start 2024-08-31T06:17:48.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.487+0000 7f9eb328a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2d08e3850 0x55d2d0c89c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:48.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.487+0000 7f9eb328a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2d08ba990 0x55d2d0c8a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:48.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.487+0000 7f9eb328a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2d08c2b70 0x55d2d0c8a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:48.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.487+0000 7f9eb328a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d2d0bff450 con 0x55d2d08e3850 2024-08-31T06:17:48.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.487+0000 7f9eb328a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d2d0bee9e0 con 0x55d2d08c2b70 2024-08-31T06:17:48.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.487+0000 7f9eb328a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d2d0bf2990 con 0x55d2d08ba990 2024-08-31T06:17:48.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.488+0000 7f9ea9200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2d08c2b70 0x55d2d0c8a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:48.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.488+0000 7f9ea9200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2d08c2b70 0x55d2d0c8a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:44082/0 (socket says 172.21.15.16:44082) 2024-08-31T06:17:48.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.488+0000 7f9ea9200640 1 -- 172.21.15.16:0/2063830746 learned_addr learned my addr 172.21.15.16:0/2063830746 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:48.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.488+0000 7f9ea2000640 1 --2- 172.21.15.16:0/1740381389 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9e8c0b4da0 0x7f9e8c0b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:48.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.489+0000 7f9ea9200640 1 -- 172.21.15.16:0/2063830746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2d08ba990 msgr2=0x55d2d0c8a150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:17:48.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.489+0000 7f9ea9200640 1 --2- 172.21.15.16:0/2063830746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2d08ba990 0x55d2d0c8a150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:48.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.489+0000 7f9ea9200640 1 -- 172.21.15.16:0/2063830746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2d08e3850 msgr2=0x55d2d0c89c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:48.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.489+0000 7f9ea9200640 1 --2- 172.21.15.16:0/2063830746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2d08e3850 0x55d2d0c89c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:48.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.489+0000 7f9ea9200640 1 -- 172.21.15.16:0/2063830746 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2d0b81fa0 con 0x55d2d08c2b70 2024-08-31T06:17:48.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.489+0000 7f9ea2000640 1 -- 172.21.15.16:0/1740381389 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8470+0+0 (secure 0 0 0) 0x7f9e9c0e5d40 con 0x7f9ea41186a0 2024-08-31T06:17:48.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.489+0000 7f9ea9200640 1 --2- 172.21.15.16:0/2063830746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2d08c2b70 0x55d2d0c8a690 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9e90009ca0 tx=0x7f9e900117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2321b7d3e8d104f8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:48.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.490+0000 7f9eaa600640 1 --2- 172.21.15.16:0/1740381389 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9e8c0b4da0 0x7f9e8c0b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:48.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.490+0000 7f9e96a00640 1 -- 172.21.15.16:0/2063830746 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9001b470 con 0x55d2d08c2b70 2024-08-31T06:17:48.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.490+0000 7f9e96a00640 1 -- 172.21.15.16:0/2063830746 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e90016070 con 0x55d2d08c2b70 2024-08-31T06:17:48.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.490+0000 7f9e96a00640 1 -- 172.21.15.16:0/2063830746 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9000fac0 con 0x55d2d08c2b70 2024-08-31T06:17:48.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.491+0000 7f9eaa600640 1 --2- 172.21.15.16:0/1740381389 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9e8c0b4da0 0x7f9e8c0b72b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c0a8480 tx=0x7f9e9c005850 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:48.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.491+0000 7f9eb328a740 1 -- 172.21.15.16:0/2063830746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2d08c2b70 msgr2=0x55d2d0c8a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:48.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.491+0000 7f9eb328a740 1 --2- 172.21.15.16:0/2063830746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2d08c2b70 0x55d2d0c8a690 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9e90009ca0 tx=0x7f9e900117b0 comp rx=0 tx=0).stop 2024-08-31T06:17:48.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.492+0000 7f9eb328a740 1 -- 172.21.15.16:0/2063830746 shutdown_connections 2024-08-31T06:17:48.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.492+0000 7f9eb328a740 1 --2- 172.21.15.16:0/2063830746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2d08ba990 0x55d2d0c8a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:48.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.492+0000 7f9eb328a740 1 --2- 172.21.15.16:0/2063830746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2d08c2b70 0x55d2d0c8a690 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:48.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.492+0000 7f9eb328a740 1 --2- 172.21.15.16:0/2063830746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2d08e3850 0x55d2d0c89c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:48.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.492+0000 7f9eb328a740 1 -- 172.21.15.16:0/2063830746 >> 172.21.15.16:0/2063830746 conn(0x55d2d08d7c40 msgr2=0x55d2d0c87b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:48.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.492+0000 7f9eb328a740 1 -- 172.21.15.16:0/2063830746 shutdown_connections 2024-08-31T06:17:48.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.492+0000 7f9eb328a740 1 -- 172.21.15.16:0/2063830746 wait complete. 2024-08-31T06:17:48.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.493+0000 7f9eb328a740 1 -- start start 2024-08-31T06:17:48.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.493+0000 7f9eb328a740 1 Processor -- start 2024-08-31T06:17:48.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.494+0000 7f9eb328a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2d08ba990 0x55d2d0c953d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:48.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.494+0000 7f9eb328a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2d08c2b70 0x55d2d0c95910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:48.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.494+0000 7f9eb328a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2d08e3850 0x55d2d0ca16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:48.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.494+0000 7f9eb328a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d2d0a517b0 con 0x55d2d08ba990 2024-08-31T06:17:48.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.494+0000 7f9eb328a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d2d0b89eb0 con 0x55d2d08c2b70 2024-08-31T06:17:48.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.494+0000 7f9eb328a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d2d0c983f0 con 0x55d2d08e3850 2024-08-31T06:17:48.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.494+0000 7f9eb328a740 10 client.0 _ll_register_callbacks cb 0x55d2d0bec0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:17:48.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.495+0000 7f9ea9c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2d08c2b70 0x55d2d0c95910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:48.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.495+0000 7f9ea9c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2d08c2b70 0x55d2d0c95910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:44084/0 (socket says 172.21.15.16:44084) 2024-08-31T06:17:48.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.495+0000 7f9ea9c00640 1 -- 172.21.15.16:0/2596031294 learned_addr learned my addr 172.21.15.16:0/2596031294 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:48.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.495+0000 7f9ea9200640 1 --2- 172.21.15.16:0/2596031294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2d08ba990 0x55d2d0c953d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:48.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.495+0000 7f9ea9200640 1 --2- 172.21.15.16:0/2596031294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2d08e3850 0x55d2d0ca16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:48.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.495+0000 7f9ea9c00640 1 -- 172.21.15.16:0/2596031294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2d08e3850 msgr2=0x55d2d0ca16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:48.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.495+0000 7f9ea9c00640 1 --2- 172.21.15.16:0/2596031294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2d08e3850 0x55d2d0ca16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:48.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.496+0000 7f9ea9c00640 1 -- 172.21.15.16:0/2596031294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2d08ba990 msgr2=0x55d2d0c953d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:48.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.496+0000 7f9ea9c00640 1 --2- 172.21.15.16:0/2596031294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2d08ba990 0x55d2d0c953d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:48.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.496+0000 7f9ea9c00640 1 -- 172.21.15.16:0/2596031294 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2d0b94f40 con 0x55d2d08c2b70 2024-08-31T06:17:48.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.496+0000 7f9ea9200640 1 --2- 172.21.15.16:0/2596031294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2d08e3850 0x55d2d0ca16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:17:48.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.496+0000 7f9ea9c00640 1 --2- 172.21.15.16:0/2596031294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2d08c2b70 0x55d2d0c95910 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f9e98009230 tx=0x7f9e98005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=856806dacb961f0b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:48.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.496+0000 7f9e8b400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:17:48.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.497+0000 7f9e8b400640 1 -- 172.21.15.16:0/2596031294 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e98035030 con 0x55d2d08c2b70 2024-08-31T06:17:48.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.497+0000 7f9e8b400640 1 -- 172.21.15.16:0/2596031294 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8470+0+0 (secure 0 0 0) 0x7f9e980046c0 con 0x55d2d08c2b70 2024-08-31T06:17:48.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.497+0000 7f9eb328a740 1 -- 172.21.15.16:0/2596031294 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e90016070 con 0x55d2d08c2b70 2024-08-31T06:17:48.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.497+0000 7f9e8b400640 1 -- 172.21.15.16:0/2596031294 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f9e980048c0 con 0x55d2d08c2b70 2024-08-31T06:17:48.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.497+0000 7f9eb328a740 10 client.27995 fetch_fsmap learned FSMap version 46 2024-08-31T06:17:48.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.497+0000 7f9eb328a740 1 -- 172.21.15.16:0/2596031294 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55d2d095dba0 con 0x55d2d08c2b70 2024-08-31T06:17:48.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.497+0000 7f9e8b400640 1 -- 172.21.15.16:0/2596031294 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e90016070 con 0x55d2d08c2b70 2024-08-31T06:17:48.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.499+0000 7f9e8b400640 1 -- 172.21.15.16:0/2596031294 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7f9e98003bf0 con 0x55d2d08c2b70 2024-08-31T06:17:48.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.499+0000 7f9eb328a740 10 client.27995 fetch_fsmap finished waiting for FSMap version 46 2024-08-31T06:17:48.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.499+0000 7f9eb328a740 10 client.27995 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:17:48.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.499+0000 7f9eb328a740 20 client.27995 populate_metadata read hostname 'smithi016' 2024-08-31T06:17:48.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.499+0000 7f9eb328a740 1 --2- 172.21.15.16:0/2596031294 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d2d08a21d0 0x55d2d0ca74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:48.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.500+0000 7f9eb328a740 4 client.27995 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:17:48.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.500+0000 7f9eb328a740 1 -- 172.21.15.16:0/2596031294 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2d0c013a0 con 0x55d2d08a21d0 2024-08-31T06:17:48.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.500+0000 7f9ea9200640 1 --2- 172.21.15.16:0/2596031294 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d2d08a21d0 0x55d2d0ca74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:48.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.501+0000 7f9ea9200640 1 --2- 172.21.15.16:0/2596031294 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d2d08a21d0 0x55d2d0ca74e0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27676e37d14cee42 server_cookie=54872fa69c7756ee in_seq=0 out_seq=0 2024-08-31T06:17:48.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.501+0000 7f9e8b400640 10 client.27995 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:17:48.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.501+0000 7f9e8b400640 1 -- 172.21.15.16:0/2596031294 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d2d0c013a0 con 0x55d2d08a21d0 2024-08-31T06:17:48.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.501+0000 7f9e8b400640 10 client.27995 handle_command_reply: tid=0 2024-08-31T06:17:48.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.502+0000 7f9eb328a740 1 client.27995 shutdown 2024-08-31T06:17:48.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.502+0000 7f9eb328a740 1 -- 172.21.15.16:0/2596031294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2d08c2b70 msgr2=0x55d2d0c95910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:48.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.502+0000 7f9eb328a740 1 --2- 172.21.15.16:0/2596031294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2d08c2b70 0x55d2d0c95910 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f9e98009230 tx=0x7f9e98005270 comp rx=0 tx=0).stop 2024-08-31T06:17:48.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.502+0000 7f9eb328a740 1 -- 172.21.15.16:0/2596031294 shutdown_connections 2024-08-31T06:17:48.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.502+0000 7f9eb328a740 1 --2- 172.21.15.16:0/2596031294 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d2d08a21d0 0x55d2d0ca74e0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:48.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.502+0000 7f9eb328a740 1 --2- 172.21.15.16:0/2596031294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2d08e3850 0x55d2d0ca16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:48.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.502+0000 7f9eb328a740 1 --2- 172.21.15.16:0/2596031294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2d08c2b70 0x55d2d0c95910 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:48.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.502+0000 7f9eb328a740 1 --2- 172.21.15.16:0/2596031294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2d08ba990 0x55d2d0c953d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:48.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.502+0000 7f9eb328a740 1 -- 172.21.15.16:0/2596031294 >> 172.21.15.16:0/2596031294 conn(0x55d2d08d7c40 msgr2=0x55d2d0c8cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:48.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.502+0000 7f9e8b400640 0 client.27995 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:17:48.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.502+0000 7f9eb328a740 1 -- 172.21.15.16:0/2596031294 shutdown_connections 2024-08-31T06:17:48.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.503+0000 7f9eb328a740 1 -- 172.21.15.16:0/2596031294 wait complete. 2024-08-31T06:17:48.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.503+0000 7f9eb328a740 20 client.27995 trim_cache size 0 max 16384 2024-08-31T06:17:48.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.536+0000 7f9eb328a740 1 Processor -- start 2024-08-31T06:17:48.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.536+0000 7f9eb328a740 1 -- start start 2024-08-31T06:17:48.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.536+0000 7f9eb328a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2d08ba990 0x55d2d0c87b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:48.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.536+0000 7f9eb328a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2d08c2b70 0x55d2d0c925c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:48.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.536+0000 7f9eb328a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2d08e3850 0x55d2d0c92b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:48.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.536+0000 7f9eb328a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea41c93c0 con 0x55d2d08c2b70 2024-08-31T06:17:48.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.536+0000 7f9eb328a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea41c6c00 con 0x55d2d08e3850 2024-08-31T06:17:48.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.536+0000 7f9eb328a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea410a570 con 0x55d2d08ba990 2024-08-31T06:17:48.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.537+0000 7f9ea9200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2d08ba990 0x55d2d0c87b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:48.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.537+0000 7f9ea9200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2d08ba990 0x55d2d0c87b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:57174/0 (socket says 172.21.15.16:57174) 2024-08-31T06:17:48.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.537+0000 7f9ea9200640 1 -- 172.21.15.16:0/1712891151 learned_addr learned my addr 172.21.15.16:0/1712891151 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:48.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.537+0000 7f9ea9200640 1 --2- 172.21.15.16:0/1712891151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2d08e3850 0x55d2d0c92b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:48.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.537+0000 7f9ea9200640 1 -- 172.21.15.16:0/1712891151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2d08e3850 msgr2=0x55d2d0c92b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:48.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.537+0000 7f9ea9200640 1 --2- 172.21.15.16:0/1712891151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2d08e3850 0x55d2d0c92b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:48.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.537+0000 7f9ea9200640 1 -- 172.21.15.16:0/1712891151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2d08c2b70 msgr2=0x55d2d0c925c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:17:48.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.537+0000 7f9ea9200640 1 --2- 172.21.15.16:0/1712891151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2d08c2b70 0x55d2d0c925c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:48.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.537+0000 7f9ea9200640 1 -- 172.21.15.16:0/1712891151 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2d0c8c4a0 con 0x55d2d08ba990 2024-08-31T06:17:48.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.538+0000 7f9ea9200640 1 --2- 172.21.15.16:0/1712891151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2d08ba990 0x55d2d0c87b60 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9e9000f770 tx=0x7f9e900058d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9c9d8b65e40c39b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:48.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.538+0000 7f9e96000640 1 -- 172.21.15.16:0/1712891151 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e90023040 con 0x55d2d08ba990 2024-08-31T06:17:48.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.538+0000 7f9e96000640 1 -- 172.21.15.16:0/1712891151 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e90016040 con 0x55d2d08ba990 2024-08-31T06:17:48.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.538+0000 7f9e96000640 1 -- 172.21.15.16:0/1712891151 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9001b6d0 con 0x55d2d08ba990 2024-08-31T06:17:48.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.539+0000 7f9eb328a740 1 -- 172.21.15.16:0/1712891151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2d08ba990 msgr2=0x55d2d0c87b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:48.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.539+0000 7f9eb328a740 1 --2- 172.21.15.16:0/1712891151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2d08ba990 0x55d2d0c87b60 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9e9000f770 tx=0x7f9e900058d0 comp rx=0 tx=0).stop 2024-08-31T06:17:48.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.540+0000 7f9eb328a740 1 -- 172.21.15.16:0/1712891151 shutdown_connections 2024-08-31T06:17:48.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.540+0000 7f9eb328a740 1 --2- 172.21.15.16:0/1712891151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2d08e3850 0x55d2d0c92b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:48.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.540+0000 7f9eb328a740 1 --2- 172.21.15.16:0/1712891151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2d08c2b70 0x55d2d0c925c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:48.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.540+0000 7f9eb328a740 1 --2- 172.21.15.16:0/1712891151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2d08ba990 0x55d2d0c87b60 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:48.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.540+0000 7f9eb328a740 1 -- 172.21.15.16:0/1712891151 >> 172.21.15.16:0/1712891151 conn(0x55d2d08d7c40 msgr2=0x55d2d0c95340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:48.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.540+0000 7f9eb328a740 1 -- 172.21.15.16:0/1712891151 shutdown_connections 2024-08-31T06:17:48.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.540+0000 7f9eb328a740 1 -- 172.21.15.16:0/1712891151 wait complete. 2024-08-31T06:17:48.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.541+0000 7f9eb328a740 1 -- start start 2024-08-31T06:17:48.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.541+0000 7f9eb328a740 1 Processor -- start 2024-08-31T06:17:48.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.542+0000 7f9eb328a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2d08ba990 0x55d2d0c9d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:48.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.542+0000 7f9eb328a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2d08c2b70 0x55d2d0c9db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:48.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.542+0000 7f9eb328a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2d08e3850 0x55d2d0c98600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:48.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.542+0000 7f9eb328a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea41ab090 con 0x55d2d08e3850 2024-08-31T06:17:48.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.542+0000 7f9eb328a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea41c6530 con 0x55d2d08c2b70 2024-08-31T06:17:48.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.542+0000 7f9eb328a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea41ab760 con 0x55d2d08ba990 2024-08-31T06:17:48.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.542+0000 7f9eb328a740 10 client.0 _ll_register_callbacks cb 0x7f9e90005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:17:48.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.542+0000 7f9ea9c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2d08c2b70 0x55d2d0c9db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:48.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.542+0000 7f9ea9c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2d08c2b70 0x55d2d0c9db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:44100/0 (socket says 172.21.15.16:44100) 2024-08-31T06:17:48.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.542+0000 7f9ea9c00640 1 -- 172.21.15.16:0/1079635337 learned_addr learned my addr 172.21.15.16:0/1079635337 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:48.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.542+0000 7f9ea9200640 1 --2- 172.21.15.16:0/1079635337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2d08e3850 0x55d2d0c98600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:48.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.543+0000 7f9ea9200640 1 --2- 172.21.15.16:0/1079635337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2d08ba990 0x55d2d0c9d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:48.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.543+0000 7f9ea9200640 1 -- 172.21.15.16:0/1079635337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2d08c2b70 msgr2=0x55d2d0c9db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:48.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.543+0000 7f9ea9200640 1 --2- 172.21.15.16:0/1079635337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2d08c2b70 0x55d2d0c9db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:48.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.543+0000 7f9ea9200640 1 -- 172.21.15.16:0/1079635337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2d08e3850 msgr2=0x55d2d0c98600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:48.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.543+0000 7f9ea9200640 1 --2- 172.21.15.16:0/1079635337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2d08e3850 0x55d2d0c98600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:48.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.543+0000 7f9ea9200640 1 -- 172.21.15.16:0/1079635337 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2d0c9e380 con 0x55d2d08ba990 2024-08-31T06:17:48.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.544+0000 7f9ea9200640 1 --2- 172.21.15.16:0/1079635337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2d08ba990 0x55d2d0c9d650 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9e90011cb0 tx=0x7f9e9000f7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc2ba8c8bc00734f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:48.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.544+0000 7f9e8b400640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:17:48.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.544+0000 7f9e8b400640 1 -- 172.21.15.16:0/1079635337 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e900035f0 con 0x55d2d08ba990 2024-08-31T06:17:48.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.544+0000 7f9eb328a740 1 -- 172.21.15.16:0/1079635337 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e90016040 con 0x55d2d08ba990 2024-08-31T06:17:48.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.544+0000 7f9e8b400640 1 -- 172.21.15.16:0/1079635337 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8470+0+0 (secure 0 0 0) 0x7f9e90003790 con 0x55d2d08ba990 2024-08-31T06:17:48.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.544+0000 7f9e8b400640 1 -- 172.21.15.16:0/1079635337 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f9e90003990 con 0x55d2d08ba990 2024-08-31T06:17:48.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.545+0000 7f9eb328a740 10 client.28000 fetch_fsmap learned FSMap version 46 2024-08-31T06:17:48.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.545+0000 7f9eb328a740 1 -- 172.21.15.16:0/1079635337 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55d2d0d12430 con 0x55d2d08ba990 2024-08-31T06:17:48.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.545+0000 7f9e8b400640 1 -- 172.21.15.16:0/1079635337 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e90016040 con 0x55d2d08ba990 2024-08-31T06:17:48.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.546+0000 7f9e8b400640 1 -- 172.21.15.16:0/1079635337 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7f9e9002bcd0 con 0x55d2d08ba990 2024-08-31T06:17:48.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.547+0000 7f9eb328a740 10 client.28000 fetch_fsmap finished waiting for FSMap version 46 2024-08-31T06:17:48.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.547+0000 7f9eb328a740 10 client.28000 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:17:48.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.547+0000 7f9eb328a740 20 client.28000 populate_metadata read hostname 'smithi016' 2024-08-31T06:17:48.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.547+0000 7f9eb328a740 1 --2- 172.21.15.16:0/1079635337 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d2d0877580 0x55d2d0d148e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:48.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.547+0000 7f9eb328a740 4 client.28000 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.25, "await_for": 45.0}] 2024-08-31T06:17:48.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.547+0000 7f9eb328a740 1 -- 172.21.15.16:0/1079635337 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.25, "await_for": 45.0}) -- 0x55d2d0bfce40 con 0x55d2d0877580 2024-08-31T06:17:48.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.550+0000 7f9ea9c00640 1 --2- 172.21.15.16:0/1079635337 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d2d0877580 0x55d2d0d148e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:48.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.551+0000 7f9ea9c00640 1 --2- 172.21.15.16:0/1079635337 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d2d0877580 0x55d2d0d148e0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=474a6e32b1e901d2 server_cookie=bbeecaf904e6e4f0 in_seq=0 out_seq=0 2024-08-31T06:17:48.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.551+0000 7f9e8b400640 10 client.28000 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:17:48.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.803+0000 7f9e8b400640 1 -- 172.21.15.16:0/1079635337 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d2d0bfce40 con 0x55d2d0877580 2024-08-31T06:17:48.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.803+0000 7f9e8b400640 10 client.28000 handle_command_reply: tid=0 2024-08-31T06:17:48.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.803+0000 7f9eb328a740 1 client.28000 shutdown 2024-08-31T06:17:48.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.803+0000 7f9eb328a740 1 -- 172.21.15.16:0/1079635337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2d08ba990 msgr2=0x55d2d0c9d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:48.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.803+0000 7f9eb328a740 1 --2- 172.21.15.16:0/1079635337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2d08ba990 0x55d2d0c9d650 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9e90011cb0 tx=0x7f9e9000f7d0 comp rx=0 tx=0).stop 2024-08-31T06:17:48.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.803+0000 7f9eb328a740 1 -- 172.21.15.16:0/1079635337 shutdown_connections 2024-08-31T06:17:48.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.803+0000 7f9eb328a740 1 --2- 172.21.15.16:0/1079635337 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d2d0877580 0x55d2d0d148e0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:48.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.803+0000 7f9eb328a740 1 --2- 172.21.15.16:0/1079635337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2d08e3850 0x55d2d0c98600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:48.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.803+0000 7f9eb328a740 1 --2- 172.21.15.16:0/1079635337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d2d08c2b70 0x55d2d0c9db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:48.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.803+0000 7f9eb328a740 1 --2- 172.21.15.16:0/1079635337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2d08ba990 0x55d2d0c9d650 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:48.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.803+0000 7f9e8b400640 0 client.28000 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:17:48.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.803+0000 7f9eb328a740 1 -- 172.21.15.16:0/1079635337 >> 172.21.15.16:0/1079635337 conn(0x55d2d08d7c40 msgr2=0x55d2d0c8cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:48.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.803+0000 7f9eb328a740 1 -- 172.21.15.16:0/1079635337 shutdown_connections 2024-08-31T06:17:48.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.803+0000 7f9eb328a740 1 -- 172.21.15.16:0/1079635337 wait complete. 2024-08-31T06:17:48.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.803+0000 7f9eb328a740 20 client.28000 trim_cache size 0 max 16384 2024-08-31T06:17:48.807 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:17:48.807 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 46, 2024-08-31T06:17:48.807 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24737, 2024-08-31T06:17:48.807 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 34, 2024-08-31T06:17:48.807 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-08-31T06:17:48.807 INFO:teuthology.orchestra.run.smithi016.stdout: "7a70d06a": { 2024-08-31T06:17:48.807 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 34, 2024-08-31T06:17:48.807 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-08-31T06:17:48.807 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:17:48.807 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-08-31T06:17:48.807 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:17:48.807 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:17:48.808 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-08-31T06:17:48.808 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 145.2, 2024-08-31T06:17:48.808 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-08-31T06:17:48.808 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-08-31T06:17:48.808 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-08-31T06:17:48.808 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:17:48.808 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-08-31T06:17:48.808 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:17:48.808 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:17:48.808 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:17:48.808 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:17:48.808 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:17:48.808 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:17:48.808 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:17:48.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.804+0000 7f9eac600640 1 -- 172.21.15.16:0/1740381389 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9e8c0b4da0 msgr2=0x7f9e8c0b72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:48.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.804+0000 7f9eac600640 1 --2- 172.21.15.16:0/1740381389 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9e8c0b4da0 0x7f9e8c0b72b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c0a8480 tx=0x7f9e9c005850 comp rx=0 tx=0).stop 2024-08-31T06:17:48.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.804+0000 7f9eac600640 1 -- 172.21.15.16:0/1740381389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ea41186a0 msgr2=0x7f9ea4116070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:48.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.804+0000 7f9eac600640 1 --2- 172.21.15.16:0/1740381389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ea41186a0 0x7f9ea4116070 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00eb40 tx=0x7f9e9c00c2d0 comp rx=0 tx=0).stop 2024-08-31T06:17:48.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.804+0000 7f9eac600640 1 -- 172.21.15.16:0/1740381389 shutdown_connections 2024-08-31T06:17:48.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.804+0000 7f9eac600640 1 --2- 172.21.15.16:0/1740381389 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9e8c0b4da0 0x7f9e8c0b72b0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:48.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.804+0000 7f9eac600640 1 --2- 172.21.15.16:0/1740381389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ea41186a0 0x7f9ea4116070 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:48.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.804+0000 7f9eac600640 1 --2- 172.21.15.16:0/1740381389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ea410aab0 0x7f9ea4115b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:48.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.804+0000 7f9eac600640 1 --2- 172.21.15.16:0/1740381389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ea410a160 0x7f9ea41155f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:48.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.804+0000 7f9eac600640 1 -- 172.21.15.16:0/1740381389 >> 172.21.15.16:0/1740381389 conn(0x7f9ea4070c20 msgr2=0x7f9ea4118bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:48.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.804+0000 7f9eac600640 1 -- 172.21.15.16:0/1740381389 shutdown_connections 2024-08-31T06:17:48.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:17:48.804+0000 7f9eac600640 1 -- 172.21.15.16:0/1740381389 wait complete. 2024-08-31T06:17:49.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:48 smithi155 ceph-mon[21742]: pgmap v691: 97 pgs: 97 active+clean; 3.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 11 MiB/s wr, 761 op/s 2024-08-31T06:17:49.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:48 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:17:49.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:48 smithi016 ceph-mon[17194]: pgmap v691: 97 pgs: 97 active+clean; 3.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 11 MiB/s wr, 761 op/s 2024-08-31T06:17:49.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:48 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:17:49.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:48 smithi184 ceph-mon[22331]: pgmap v691: 97 pgs: 97 active+clean; 3.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 11 MiB/s wr, 761 op/s 2024-08-31T06:17:49.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:48 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:17:49.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:49 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:17:49.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:49 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:17:49.908 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '7a70d06a', quiesce took 7.7 seconds. Will release after: 17.55 2024-08-31T06:17:50.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:49 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:17:50.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:49 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:17:50.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:49 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:17:50.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:49 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:17:51.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:50 smithi155 ceph-mon[21742]: pgmap v692: 97 pgs: 97 active+clean; 3.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 13 MiB/s wr, 842 op/s 2024-08-31T06:17:51.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:50 smithi016 ceph-mon[17194]: pgmap v692: 97 pgs: 97 active+clean; 3.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 13 MiB/s wr, 842 op/s 2024-08-31T06:17:51.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:50 smithi184 ceph-mon[22331]: pgmap v692: 97 pgs: 97 active+clean; 3.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 13 MiB/s wr, 842 op/s 2024-08-31T06:17:51.627 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 625.0 s, 536.3 tps, lat 76.606 ms stddev 147.193 2024-08-31T06:17:53.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:52 smithi155 ceph-mon[21742]: pgmap v693: 97 pgs: 97 active+clean; 3.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 10 MiB/s wr, 756 op/s 2024-08-31T06:17:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:52 smithi016 ceph-mon[17194]: pgmap v693: 97 pgs: 97 active+clean; 3.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 10 MiB/s wr, 756 op/s 2024-08-31T06:17:53.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:52 smithi184 ceph-mon[22331]: pgmap v693: 97 pgs: 97 active+clean; 3.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 10 MiB/s wr, 756 op/s 2024-08-31T06:17:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.589+0000 7f73a7e00640 1 -- 172.21.15.16:0/810953670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73a81036b0 msgr2=0x7f73a8103ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.589+0000 7f73a7e00640 1 --2- 172.21.15.16:0/810953670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73a81036b0 0x7f73a8103ac0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7398009900 tx=0x7f739802a3f0 comp rx=0 tx=0).stop 2024-08-31T06:17:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.590+0000 7f73a7e00640 1 -- 172.21.15.16:0/810953670 shutdown_connections 2024-08-31T06:17:53.593 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.590+0000 7f73a7e00640 1 --2- 172.21.15.16:0/810953670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73a8108fe0 0x7f73a81093f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:53.593 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.590+0000 7f73a7e00640 1 --2- 172.21.15.16:0/810953670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73a8109930 0x7f73a8110160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:53.593 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.590+0000 7f73a7e00640 1 --2- 172.21.15.16:0/810953670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73a81036b0 0x7f73a8103ac0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:53.593 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.590+0000 7f73a7e00640 1 -- 172.21.15.16:0/810953670 >> 172.21.15.16:0/810953670 conn(0x7f73a80ff0d0 msgr2=0x7f73a8101520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:53.593 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.590+0000 7f73a7e00640 1 -- 172.21.15.16:0/810953670 shutdown_connections 2024-08-31T06:17:53.593 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.590+0000 7f73a7e00640 1 -- 172.21.15.16:0/810953670 wait complete. 2024-08-31T06:17:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.591+0000 7f73a7e00640 1 Processor -- start 2024-08-31T06:17:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.591+0000 7f73a7e00640 1 -- start start 2024-08-31T06:17:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.591+0000 7f73a7e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73a81036b0 0x7f73a819c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.591+0000 7f73a7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73a8108fe0 0x7f73a819c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.591+0000 7f73a7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73a8109930 0x7f73a81a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.591+0000 7f73a7e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f73a8113630 con 0x7f73a8108fe0 2024-08-31T06:17:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.591+0000 7f73a7e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f73a81134b0 con 0x7f73a8109930 2024-08-31T06:17:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.591+0000 7f73a7e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f73a81137b0 con 0x7f73a81036b0 2024-08-31T06:17:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.591+0000 7f73a6000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73a8108fe0 0x7f73a819c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.592+0000 7f73a6000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73a8108fe0 0x7f73a819c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34052/0 (socket says 172.21.15.16:34052) 2024-08-31T06:17:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.592+0000 7f73a6000640 1 -- 172.21.15.16:0/3948747160 learned_addr learned my addr 172.21.15.16:0/3948747160 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.592+0000 7f73a6a00640 1 --2- 172.21.15.16:0/3948747160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73a81036b0 0x7f73a819c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.592+0000 7f73a7400640 1 --2- 172.21.15.16:0/3948747160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73a8109930 0x7f73a81a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.592+0000 7f73a6000640 1 -- 172.21.15.16:0/3948747160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73a81036b0 msgr2=0x7f73a819c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.592+0000 7f73a6000640 1 --2- 172.21.15.16:0/3948747160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73a81036b0 0x7f73a819c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.592+0000 7f73a6000640 1 -- 172.21.15.16:0/3948747160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73a8109930 msgr2=0x7f73a81a1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.592+0000 7f73a6000640 1 --2- 172.21.15.16:0/3948747160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73a8109930 0x7f73a81a1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.592+0000 7f73a6000640 1 -- 172.21.15.16:0/3948747160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a81a21a0 con 0x7f73a8108fe0 2024-08-31T06:17:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.592+0000 7f73a6000640 1 --2- 172.21.15.16:0/3948747160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73a8108fe0 0x7f73a819c810 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f739400d740 tx=0x7f739400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.593+0000 7f739f400640 1 -- 172.21.15.16:0/3948747160 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7394017070 con 0x7f73a8108fe0 2024-08-31T06:17:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.593+0000 7f73a7e00640 1 -- 172.21.15.16:0/3948747160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73a81a2490 con 0x7f73a8108fe0 2024-08-31T06:17:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.593+0000 7f739f400640 1 -- 172.21.15.16:0/3948747160 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f739400de50 con 0x7f73a8108fe0 2024-08-31T06:17:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.593+0000 7f739f400640 1 -- 172.21.15.16:0/3948747160 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739400f620 con 0x7f73a8108fe0 2024-08-31T06:17:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.593+0000 7f73a7e00640 1 -- 172.21.15.16:0/3948747160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73a81a8c20 con 0x7f73a8108fe0 2024-08-31T06:17:53.598 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.596+0000 7f73b4a12740 1 Processor -- start 2024-08-31T06:17:53.598 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.596+0000 7f73b4a12740 1 -- start start 2024-08-31T06:17:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.596+0000 7f73b4a12740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619ecc191b0 0x5619ecfbc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.596+0000 7f73b4a12740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619ecbf02f0 0x5619ecfbcde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.596+0000 7f73b4a12740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619ecbf84d0 0x5619ecfbd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.596+0000 7f73b4a12740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5619ecf31f70 con 0x5619ecbf02f0 2024-08-31T06:17:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.596+0000 7f73b4a12740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5619ecf21500 con 0x5619ecbf84d0 2024-08-31T06:17:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.596+0000 7f73b4a12740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5619ecf254b0 con 0x5619ecc191b0 2024-08-31T06:17:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.596+0000 7f73a6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619ecbf02f0 0x5619ecfbcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.597+0000 7f73a6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619ecbf02f0 0x5619ecfbcde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34058/0 (socket says 172.21.15.16:34058) 2024-08-31T06:17:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.597+0000 7f73a6a00640 1 -- 172.21.15.16:0/1548807994 learned_addr learned my addr 172.21.15.16:0/1548807994 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.597+0000 7f73a7400640 1 --2- 172.21.15.16:0/1548807994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619ecc191b0 0x5619ecfbc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.597+0000 7f73a6a00640 1 -- 172.21.15.16:0/1548807994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619ecc191b0 msgr2=0x5619ecfbc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.597+0000 7f73a6a00640 1 --2- 172.21.15.16:0/1548807994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619ecc191b0 0x5619ecfbc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.597+0000 7f73a6a00640 1 -- 172.21.15.16:0/1548807994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619ecbf84d0 msgr2=0x5619ecfbd320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:17:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.597+0000 7f73a6a00640 1 --2- 172.21.15.16:0/1548807994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619ecbf84d0 0x5619ecfbd320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.597+0000 7f73a6a00640 1 -- 172.21.15.16:0/1548807994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619ecedbc10 con 0x5619ecbf02f0 2024-08-31T06:17:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.597+0000 7f739f400640 1 -- 172.21.15.16:0/3948747160 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f739401a070 con 0x7f73a8108fe0 2024-08-31T06:17:53.601 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.597+0000 7f73a6a00640 1 --2- 172.21.15.16:0/1548807994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619ecbf02f0 0x5619ecfbcde0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f7398009230 tx=0x7f7398004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c8f96d5dbb2d376 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:53.602 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.597+0000 7f7393e00640 1 -- 172.21.15.16:0/1548807994 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7398035040 con 0x5619ecbf02f0 2024-08-31T06:17:53.602 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.598+0000 7f7393e00640 1 -- 172.21.15.16:0/1548807994 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73980054f0 con 0x5619ecbf02f0 2024-08-31T06:17:53.602 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.598+0000 7f7393e00640 1 -- 172.21.15.16:0/1548807994 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73980035f0 con 0x5619ecbf02f0 2024-08-31T06:17:53.602 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.598+0000 7f73b4a12740 1 -- 172.21.15.16:0/1548807994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619ecbf02f0 msgr2=0x5619ecfbcde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:53.602 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.598+0000 7f73b4a12740 1 --2- 172.21.15.16:0/1548807994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619ecbf02f0 0x5619ecfbcde0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f7398009230 tx=0x7f7398004170 comp rx=0 tx=0).stop 2024-08-31T06:17:53.602 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.599+0000 7f73b4a12740 1 -- 172.21.15.16:0/1548807994 shutdown_connections 2024-08-31T06:17:53.602 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.599+0000 7f73b4a12740 1 --2- 172.21.15.16:0/1548807994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619ecbf84d0 0x5619ecfbd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.599+0000 7f73b4a12740 1 --2- 172.21.15.16:0/1548807994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619ecbf02f0 0x5619ecfbcde0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.599+0000 7f73b4a12740 1 --2- 172.21.15.16:0/1548807994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619ecc191b0 0x5619ecfbc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.599+0000 7f73b4a12740 1 -- 172.21.15.16:0/1548807994 >> 172.21.15.16:0/1548807994 conn(0x5619ecc0d300 msgr2=0x5619ecfba790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.599+0000 7f739f400640 1 --2- 172.21.15.16:0/3948747160 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f738c0b4ca0 0x7f738c0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.599+0000 7f73b4a12740 1 -- 172.21.15.16:0/1548807994 shutdown_connections 2024-08-31T06:17:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.599+0000 7f739f400640 1 -- 172.21.15.16:0/3948747160 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8470+0+0 (secure 0 0 0) 0x7f73940e4eb0 con 0x7f73a8108fe0 2024-08-31T06:17:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.599+0000 7f73b4a12740 1 -- 172.21.15.16:0/1548807994 wait complete. 2024-08-31T06:17:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.599+0000 7f73a7400640 1 --2- 172.21.15.16:0/3948747160 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f738c0b4ca0 0x7f738c0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.599+0000 7f73b4a12740 1 -- start start 2024-08-31T06:17:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.600+0000 7f73a7400640 1 --2- 172.21.15.16:0/3948747160 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f738c0b4ca0 0x7f738c0b71b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f73a0005fd0 tx=0x7f73a0009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.600+0000 7f73b4a12740 1 Processor -- start 2024-08-31T06:17:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.600+0000 7f73b4a12740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619ecbf02f0 0x5619ecfc7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.600+0000 7f73b4a12740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619ecbf84d0 0x5619ecfc84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.600+0000 7f73b4a12740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619ecc191b0 0x5619ecfd4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.600+0000 7f73b4a12740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5619ece5aab0 con 0x5619ecbf02f0 2024-08-31T06:17:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.600+0000 7f73b4a12740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5619ecec8320 con 0x5619ecc191b0 2024-08-31T06:17:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.600+0000 7f73b4a12740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5619ecfcafc0 con 0x5619ecbf84d0 2024-08-31T06:17:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.600+0000 7f73b4a12740 10 client.0 _ll_register_callbacks cb 0x5619ecf01e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:17:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.600+0000 7f73a6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619ecbf02f0 0x5619ecfc7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.600+0000 7f73a6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619ecbf02f0 0x5619ecfc7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34070/0 (socket says 172.21.15.16:34070) 2024-08-31T06:17:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.600+0000 7f73a6a00640 1 -- 172.21.15.16:0/2223452674 learned_addr learned my addr 172.21.15.16:0/2223452674 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.601+0000 7f73a6000640 1 --2- 172.21.15.16:0/2223452674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619ecbf84d0 0x5619ecfc84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.601+0000 7f73a7400640 1 --2- 172.21.15.16:0/2223452674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619ecc191b0 0x5619ecfd4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.601+0000 7f73a6a00640 1 -- 172.21.15.16:0/2223452674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619ecbf84d0 msgr2=0x5619ecfc84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.601+0000 7f73a6a00640 1 --2- 172.21.15.16:0/2223452674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619ecbf84d0 0x5619ecfc84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.601+0000 7f73a6a00640 1 -- 172.21.15.16:0/2223452674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619ecc191b0 msgr2=0x5619ecfd4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.601+0000 7f73a6a00640 1 --2- 172.21.15.16:0/2223452674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619ecc191b0 0x5619ecfd4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.601+0000 7f73a6a00640 1 -- 172.21.15.16:0/2223452674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619ecc84790 con 0x5619ecbf02f0 2024-08-31T06:17:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.601+0000 7f73a6a00640 1 --2- 172.21.15.16:0/2223452674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619ecbf02f0 0x5619ecfc7f90 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7398031440 tx=0x7f7398009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a57a6f677c68e159 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.601+0000 7f7390c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:17:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.601+0000 7f7390c00640 1 -- 172.21.15.16:0/2223452674 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7398053060 con 0x5619ecbf02f0 2024-08-31T06:17:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.602+0000 7f7390c00640 1 -- 172.21.15.16:0/2223452674 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8470+0+0 (secure 0 0 0) 0x7f7398004980 con 0x5619ecbf02f0 2024-08-31T06:17:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.602+0000 7f73b4a12740 1 -- 172.21.15.16:0/2223452674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73980054f0 con 0x5619ecbf02f0 2024-08-31T06:17:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.602+0000 7f7390c00640 1 -- 172.21.15.16:0/2223452674 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f7398035050 con 0x5619ecbf02f0 2024-08-31T06:17:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.602+0000 7f73b4a12740 10 client.17649 fetch_fsmap learned FSMap version 46 2024-08-31T06:17:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.602+0000 7f73b4a12740 1 -- 172.21.15.16:0/2223452674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5619eccfd620 con 0x5619ecbf02f0 2024-08-31T06:17:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.602+0000 7f7390c00640 1 -- 172.21.15.16:0/2223452674 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73980054f0 con 0x5619ecbf02f0 2024-08-31T06:17:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.602+0000 7f7390c00640 1 -- 172.21.15.16:0/2223452674 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7f7398003c10 con 0x5619ecbf02f0 2024-08-31T06:17:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.602+0000 7f73b4a12740 10 client.17649 fetch_fsmap finished waiting for FSMap version 46 2024-08-31T06:17:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.602+0000 7f73b4a12740 10 client.17649 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:17:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.602+0000 7f73b4a12740 20 client.17649 populate_metadata read hostname 'smithi016' 2024-08-31T06:17:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.602+0000 7f73b4a12740 1 --2- 172.21.15.16:0/2223452674 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5619ecbd7d80 0x5619ecfd9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.603+0000 7f73b4a12740 4 client.17649 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:17:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.603+0000 7f73b4a12740 1 -- 172.21.15.16:0/2223452674 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619ecf33ec0 con 0x5619ecbd7d80 2024-08-31T06:17:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.603+0000 7f73a6000640 1 --2- 172.21.15.16:0/2223452674 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5619ecbd7d80 0x5619ecfd9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.603+0000 7f73a6000640 1 --2- 172.21.15.16:0/2223452674 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5619ecbd7d80 0x5619ecfd9f20 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cf62a5636d49afe server_cookie=86a7d1dd4ce9168c in_seq=0 out_seq=0 2024-08-31T06:17:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.603+0000 7f7390c00640 10 client.17649 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:17:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.604+0000 7f7390c00640 1 -- 172.21.15.16:0/2223452674 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5619ecf33ec0 con 0x5619ecbd7d80 2024-08-31T06:17:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.604+0000 7f7390c00640 10 client.17649 handle_command_reply: tid=0 2024-08-31T06:17:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.604+0000 7f73b4a12740 1 client.17649 shutdown 2024-08-31T06:17:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.604+0000 7f73b4a12740 1 -- 172.21.15.16:0/2223452674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619ecbf02f0 msgr2=0x5619ecfc7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.604+0000 7f73b4a12740 1 --2- 172.21.15.16:0/2223452674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619ecbf02f0 0x5619ecfc7f90 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7398031440 tx=0x7f7398009260 comp rx=0 tx=0).stop 2024-08-31T06:17:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.605+0000 7f73b4a12740 1 -- 172.21.15.16:0/2223452674 shutdown_connections 2024-08-31T06:17:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.605+0000 7f73b4a12740 1 --2- 172.21.15.16:0/2223452674 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5619ecbd7d80 0x5619ecfd9f20 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.605+0000 7f73b4a12740 1 --2- 172.21.15.16:0/2223452674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619ecbf84d0 0x5619ecfc84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.605+0000 7f73b4a12740 1 --2- 172.21.15.16:0/2223452674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619ecc191b0 0x5619ecfd4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.605+0000 7f7390c00640 0 client.17649 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:17:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.605+0000 7f73b4a12740 1 --2- 172.21.15.16:0/2223452674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619ecbf02f0 0x5619ecfc7f90 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.605+0000 7f73b4a12740 1 -- 172.21.15.16:0/2223452674 >> 172.21.15.16:0/2223452674 conn(0x5619ecc0d300 msgr2=0x5619ecfbd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.605+0000 7f73b4a12740 1 -- 172.21.15.16:0/2223452674 shutdown_connections 2024-08-31T06:17:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.605+0000 7f73b4a12740 1 -- 172.21.15.16:0/2223452674 wait complete. 2024-08-31T06:17:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.605+0000 7f73b4a12740 20 client.17649 trim_cache size 0 max 16384 2024-08-31T06:17:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.620+0000 7f73b4a12740 1 Processor -- start 2024-08-31T06:17:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.620+0000 7f73b4a12740 1 -- start start 2024-08-31T06:17:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.621+0000 7f73b4a12740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619ecbf02f0 0x5619ecfc8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.621+0000 7f73b4a12740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619ecbf84d0 0x5619ecfc9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.621+0000 7f73b4a12740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619ecc191b0 0x5619ecfbdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.621+0000 7f73b4a12740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f73a81b3e40 con 0x5619ecc191b0 2024-08-31T06:17:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.621+0000 7f73b4a12740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f73a81b1680 con 0x5619ecbf02f0 2024-08-31T06:17:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.621+0000 7f73b4a12740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f73a81af4f0 con 0x5619ecbf84d0 2024-08-31T06:17:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.621+0000 7f73a7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619ecc191b0 0x5619ecfbdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.621+0000 7f73a7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619ecc191b0 0x5619ecfbdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34084/0 (socket says 172.21.15.16:34084) 2024-08-31T06:17:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.621+0000 7f73a7400640 1 -- 172.21.15.16:0/3490334598 learned_addr learned my addr 172.21.15.16:0/3490334598 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.621+0000 7f73a6000640 1 --2- 172.21.15.16:0/3490334598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619ecbf84d0 0x5619ecfc9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.621+0000 7f73a6a00640 1 --2- 172.21.15.16:0/3490334598 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619ecbf02f0 0x5619ecfc8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.622+0000 7f73a7400640 1 -- 172.21.15.16:0/3490334598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619ecbf84d0 msgr2=0x5619ecfc9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.622+0000 7f73a7400640 1 --2- 172.21.15.16:0/3490334598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619ecbf84d0 0x5619ecfc9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.622+0000 7f73a7400640 1 -- 172.21.15.16:0/3490334598 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619ecbf02f0 msgr2=0x5619ecfc8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.622+0000 7f73a7400640 1 --2- 172.21.15.16:0/3490334598 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619ecbf02f0 0x5619ecfc8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.622+0000 7f73a7400640 1 -- 172.21.15.16:0/3490334598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619ecfc24a0 con 0x5619ecc191b0 2024-08-31T06:17:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.622+0000 7f73a6000640 1 --2- 172.21.15.16:0/3490334598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619ecbf84d0 0x5619ecfc9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:17:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.622+0000 7f73a7400640 1 --2- 172.21.15.16:0/3490334598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619ecc191b0 0x5619ecfbdfe0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f73a0003f50 tx=0x7f73a001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef1107ad51ddc803 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.622+0000 7f7393400640 1 -- 172.21.15.16:0/3490334598 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a002d070 con 0x5619ecc191b0 2024-08-31T06:17:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.622+0000 7f7393400640 1 -- 172.21.15.16:0/3490334598 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73a000f9f0 con 0x5619ecc191b0 2024-08-31T06:17:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.622+0000 7f7393400640 1 -- 172.21.15.16:0/3490334598 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a001db40 con 0x5619ecc191b0 2024-08-31T06:17:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.623+0000 7f73b4a12740 1 -- 172.21.15.16:0/3490334598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619ecc191b0 msgr2=0x5619ecfbdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.623+0000 7f73b4a12740 1 --2- 172.21.15.16:0/3490334598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619ecc191b0 0x5619ecfbdfe0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f73a0003f50 tx=0x7f73a001b000 comp rx=0 tx=0).stop 2024-08-31T06:17:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.623+0000 7f73b4a12740 1 -- 172.21.15.16:0/3490334598 shutdown_connections 2024-08-31T06:17:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.623+0000 7f73b4a12740 1 --2- 172.21.15.16:0/3490334598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619ecbf84d0 0x5619ecfc9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.623+0000 7f73b4a12740 1 --2- 172.21.15.16:0/3490334598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619ecc191b0 0x5619ecfbdfe0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.623+0000 7f73b4a12740 1 --2- 172.21.15.16:0/3490334598 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619ecbf02f0 0x5619ecfc8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.623+0000 7f73b4a12740 1 -- 172.21.15.16:0/3490334598 >> 172.21.15.16:0/3490334598 conn(0x5619ecc0d300 msgr2=0x5619ecfbd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.623+0000 7f73b4a12740 1 -- 172.21.15.16:0/3490334598 shutdown_connections 2024-08-31T06:17:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.623+0000 7f73b4a12740 1 -- 172.21.15.16:0/3490334598 wait complete. 2024-08-31T06:17:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.624+0000 7f73b4a12740 1 -- start start 2024-08-31T06:17:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.624+0000 7f73b4a12740 1 Processor -- start 2024-08-31T06:17:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.624+0000 7f73b4a12740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619ecbf02f0 0x5619ecfdd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.624+0000 7f73b4a12740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619ecbf84d0 0x5619ecfdaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.624+0000 7f73b4a12740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619ecc191b0 0x5619ecfdb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.624+0000 7f73b4a12740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f73a81afb40 con 0x5619ecbf84d0 2024-08-31T06:17:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.624+0000 7f73b4a12740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f73a81b0fb0 con 0x5619ecc191b0 2024-08-31T06:17:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.624+0000 7f73b4a12740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f73a81b0210 con 0x5619ecbf02f0 2024-08-31T06:17:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.625+0000 7f73b4a12740 10 client.0 _ll_register_callbacks cb 0x7f7398004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:17:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.625+0000 7f73a6a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619ecbf02f0 0x5619ecfdd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.625+0000 7f73a6000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619ecbf84d0 0x5619ecfdaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.625+0000 7f73a6a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619ecbf02f0 0x5619ecfdd620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:57236/0 (socket says 172.21.15.16:57236) 2024-08-31T06:17:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.625+0000 7f73a6a00640 1 -- 172.21.15.16:0/2102496307 learned_addr learned my addr 172.21.15.16:0/2102496307 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:17:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.625+0000 7f73a7400640 1 --2- 172.21.15.16:0/2102496307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619ecc191b0 0x5619ecfdb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.625+0000 7f73a6000640 1 -- 172.21.15.16:0/2102496307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619ecbf02f0 msgr2=0x5619ecfdd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.625+0000 7f73a6000640 1 --2- 172.21.15.16:0/2102496307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619ecbf02f0 0x5619ecfdd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.625+0000 7f73a6000640 1 -- 172.21.15.16:0/2102496307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619ecc191b0 msgr2=0x5619ecfdb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:17:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.625+0000 7f73a6000640 1 --2- 172.21.15.16:0/2102496307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619ecc191b0 0x5619ecfdb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.625+0000 7f73a6000640 1 -- 172.21.15.16:0/2102496307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619ecfdb690 con 0x5619ecbf84d0 2024-08-31T06:17:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.625+0000 7f73a6a00640 1 --2- 172.21.15.16:0/2102496307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619ecbf02f0 0x5619ecfdd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:17:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.625+0000 7f73a6000640 1 --2- 172.21.15.16:0/2102496307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619ecbf84d0 0x5619ecfdaae0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f73940e2630 tx=0x7f7394010c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c3f4e47ef5dd437 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:17:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.625+0000 7f7390c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:17:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.626+0000 7f7390c00640 1 -- 172.21.15.16:0/2102496307 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7394021040 con 0x5619ecbf84d0 2024-08-31T06:17:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.626+0000 7f7390c00640 1 -- 172.21.15.16:0/2102496307 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8470+0+0 (secure 0 0 0) 0x7f73940d2170 con 0x5619ecbf84d0 2024-08-31T06:17:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.626+0000 7f73b4a12740 1 -- 172.21.15.16:0/2102496307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73a000f9f0 con 0x5619ecbf84d0 2024-08-31T06:17:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.626+0000 7f7390c00640 1 -- 172.21.15.16:0/2102496307 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f73940a7910 con 0x5619ecbf84d0 2024-08-31T06:17:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.626+0000 7f73b4a12740 10 client.17655 fetch_fsmap learned FSMap version 46 2024-08-31T06:17:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.626+0000 7f73b4a12740 1 -- 172.21.15.16:0/2102496307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5619ed044540 con 0x5619ecbf84d0 2024-08-31T06:17:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.626+0000 7f7390c00640 1 -- 172.21.15.16:0/2102496307 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73a000f9f0 con 0x5619ecbf84d0 2024-08-31T06:17:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.626+0000 7f7390c00640 1 -- 172.21.15.16:0/2102496307 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7f73940ce140 con 0x5619ecbf84d0 2024-08-31T06:17:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.626+0000 7f73b4a12740 10 client.17655 fetch_fsmap finished waiting for FSMap version 46 2024-08-31T06:17:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.626+0000 7f73b4a12740 10 client.17655 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:17:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.626+0000 7f73b4a12740 20 client.17655 populate_metadata read hostname 'smithi016' 2024-08-31T06:17:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.626+0000 7f73b4a12740 1 --2- 172.21.15.16:0/2102496307 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5619ecbad1c0 0x5619ed046d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:17:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.626+0000 7f73b4a12740 4 client.17655 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:17:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.626+0000 7f73b4a12740 1 -- 172.21.15.16:0/2102496307 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5619ecf2f960 con 0x5619ecbad1c0 2024-08-31T06:17:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.627+0000 7f73a6a00640 1 --2- 172.21.15.16:0/2102496307 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5619ecbad1c0 0x5619ed046d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:17:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.627+0000 7f73a6a00640 1 --2- 172.21.15.16:0/2102496307 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5619ecbad1c0 0x5619ed046d60 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=990d10c7ed5f9a59 server_cookie=427e54775dbe9a33 in_seq=0 out_seq=0 2024-08-31T06:17:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.627+0000 7f7390c00640 10 client.17655 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:17:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.648+0000 7f7390c00640 1 -- 172.21.15.16:0/2102496307 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5619ecf2f960 con 0x5619ecbad1c0 2024-08-31T06:17:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.648+0000 7f7390c00640 10 client.17655 handle_command_reply: tid=0 2024-08-31T06:17:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.648+0000 7f73b4a12740 1 client.17655 shutdown 2024-08-31T06:17:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.648+0000 7f73b4a12740 1 -- 172.21.15.16:0/2102496307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619ecbf84d0 msgr2=0x5619ecfdaae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.648+0000 7f73b4a12740 1 --2- 172.21.15.16:0/2102496307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619ecbf84d0 0x5619ecfdaae0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f73940e2630 tx=0x7f7394010c70 comp rx=0 tx=0).stop 2024-08-31T06:17:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.648+0000 7f73b4a12740 1 -- 172.21.15.16:0/2102496307 shutdown_connections 2024-08-31T06:17:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.648+0000 7f73b4a12740 1 --2- 172.21.15.16:0/2102496307 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5619ecbad1c0 0x5619ed046d60 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.648+0000 7f73b4a12740 1 --2- 172.21.15.16:0/2102496307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619ecc191b0 0x5619ecfdb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.648+0000 7f73b4a12740 1 --2- 172.21.15.16:0/2102496307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619ecbf84d0 0x5619ecfdaae0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.648+0000 7f7390c00640 0 client.17655 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:17:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.648+0000 7f73b4a12740 1 --2- 172.21.15.16:0/2102496307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619ecbf02f0 0x5619ecfdd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.648+0000 7f73b4a12740 1 -- 172.21.15.16:0/2102496307 >> 172.21.15.16:0/2102496307 conn(0x5619ecc0d300 msgr2=0x5619ecfc9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.648+0000 7f73b4a12740 1 -- 172.21.15.16:0/2102496307 shutdown_connections 2024-08-31T06:17:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.648+0000 7f73b4a12740 1 -- 172.21.15.16:0/2102496307 wait complete. 2024-08-31T06:17:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.648+0000 7f73b4a12740 20 client.17655 trim_cache size 0 max 16384 2024-08-31T06:17:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:17:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:17:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:17:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:17:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.649+0000 7f73a7e00640 1 -- 172.21.15.16:0/3948747160 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f738c0b4ca0 msgr2=0x7f738c0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.649+0000 7f73a7e00640 1 --2- 172.21.15.16:0/3948747160 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f738c0b4ca0 0x7f738c0b71b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f73a0005fd0 tx=0x7f73a0009500 comp rx=0 tx=0).stop 2024-08-31T06:17:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.649+0000 7f73a7e00640 1 -- 172.21.15.16:0/3948747160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73a8108fe0 msgr2=0x7f73a819c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:17:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.649+0000 7f73a7e00640 1 --2- 172.21.15.16:0/3948747160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73a8108fe0 0x7f73a819c810 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f739400d740 tx=0x7f739400dc00 comp rx=0 tx=0).stop 2024-08-31T06:17:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.649+0000 7f73a7e00640 1 -- 172.21.15.16:0/3948747160 shutdown_connections 2024-08-31T06:17:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.649+0000 7f73a7e00640 1 --2- 172.21.15.16:0/3948747160 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f738c0b4ca0 0x7f738c0b71b0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.649+0000 7f73a7e00640 1 --2- 172.21.15.16:0/3948747160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73a8109930 0x7f73a81a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.649+0000 7f73a7e00640 1 --2- 172.21.15.16:0/3948747160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73a8108fe0 0x7f73a819c810 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.649+0000 7f73a7e00640 1 --2- 172.21.15.16:0/3948747160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73a81036b0 0x7f73a819c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:17:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.649+0000 7f73a7e00640 1 -- 172.21.15.16:0/3948747160 >> 172.21.15.16:0/3948747160 conn(0x7f73a80ff0d0 msgr2=0x7f73a81086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:17:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.649+0000 7f73a7e00640 1 -- 172.21.15.16:0/3948747160 shutdown_connections 2024-08-31T06:17:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:17:53.649+0000 7f73a7e00640 1 -- 172.21.15.16:0/3948747160 wait complete. 2024-08-31T06:17:55.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:54 smithi155 ceph-mon[21742]: pgmap v694: 97 pgs: 97 active+clean; 3.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 9.5 MiB/s wr, 701 op/s 2024-08-31T06:17:55.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:54 smithi016 ceph-mon[17194]: pgmap v694: 97 pgs: 97 active+clean; 3.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 9.5 MiB/s wr, 701 op/s 2024-08-31T06:17:55.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:54 smithi184 ceph-mon[22331]: pgmap v694: 97 pgs: 97 active+clean; 3.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 9.5 MiB/s wr, 701 op/s 2024-08-31T06:17:56.627 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 630.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-31T06:17:57.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:56 smithi155 ceph-mon[21742]: pgmap v695: 97 pgs: 97 active+clean; 3.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 7.9 MiB/s wr, 621 op/s 2024-08-31T06:17:57.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:56 smithi016 ceph-mon[17194]: pgmap v695: 97 pgs: 97 active+clean; 3.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 7.9 MiB/s wr, 621 op/s 2024-08-31T06:17:57.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:56 smithi184 ceph-mon[22331]: pgmap v695: 97 pgs: 97 active+clean; 3.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 7.9 MiB/s wr, 621 op/s 2024-08-31T06:17:59.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:17:58 smithi155 ceph-mon[21742]: pgmap v696: 97 pgs: 97 active+clean; 3.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 4.1 MiB/s wr, 295 op/s 2024-08-31T06:17:59.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:17:58 smithi016 ceph-mon[17194]: pgmap v696: 97 pgs: 97 active+clean; 3.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 4.1 MiB/s wr, 295 op/s 2024-08-31T06:17:59.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:17:58 smithi184 ceph-mon[22331]: pgmap v696: 97 pgs: 97 active+clean; 3.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 4.1 MiB/s wr, 295 op/s 2024-08-31T06:18:01.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:00 smithi155 ceph-mon[21742]: pgmap v697: 97 pgs: 97 active+clean; 3.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 3.7 MiB/s wr, 222 op/s 2024-08-31T06:18:01.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:00 smithi016 ceph-mon[17194]: pgmap v697: 97 pgs: 97 active+clean; 3.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 3.7 MiB/s wr, 222 op/s 2024-08-31T06:18:01.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:00 smithi184 ceph-mon[22331]: pgmap v697: 97 pgs: 97 active+clean; 3.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 3.7 MiB/s wr, 222 op/s 2024-08-31T06:18:01.628 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 635.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-31T06:18:02.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:01 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:18:02.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:01 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:18:02.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:01 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:18:02.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:01 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:18:02.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:01 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:18:02.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:01 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:18:02.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:01 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:18:02.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:01 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:18:02.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:01 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:18:02.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:01 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:18:02.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:01 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:18:02.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:01 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:18:02.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:01 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:18:02.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:01 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:18:02.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:01 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:18:02.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:01 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:18:02.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:01 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:18:02.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:01 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:18:03.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:02 smithi155 ceph-mon[21742]: pgmap v698: 97 pgs: 97 active+clean; 3.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 1.1 MiB/s wr, 85 op/s 2024-08-31T06:18:03.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:02 smithi016 ceph-mon[17194]: pgmap v698: 97 pgs: 97 active+clean; 3.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 1.1 MiB/s wr, 85 op/s 2024-08-31T06:18:03.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:02 smithi184 ceph-mon[22331]: pgmap v698: 97 pgs: 97 active+clean; 3.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 1.1 MiB/s wr, 85 op/s 2024-08-31T06:18:05.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:04 smithi155 ceph-mon[21742]: pgmap v699: 97 pgs: 97 active+clean; 3.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 308 KiB/s wr, 30 op/s 2024-08-31T06:18:05.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:04 smithi155 ceph-mon[21742]: osdmap e92: 12 total, 12 up, 12 in 2024-08-31T06:18:05.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:04 smithi016 ceph-mon[17194]: pgmap v699: 97 pgs: 97 active+clean; 3.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 308 KiB/s wr, 30 op/s 2024-08-31T06:18:05.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:04 smithi016 ceph-mon[17194]: osdmap e92: 12 total, 12 up, 12 in 2024-08-31T06:18:05.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:04 smithi184 ceph-mon[22331]: pgmap v699: 97 pgs: 97 active+clean; 3.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 308 KiB/s wr, 30 op/s 2024-08-31T06:18:05.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:04 smithi184 ceph-mon[22331]: osdmap e92: 12 total, 12 up, 12 in 2024-08-31T06:18:06.629 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 640.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-31T06:18:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.853+0000 7f6c0ca00640 1 -- 172.21.15.16:0/4265481897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c081058c0 msgr2=0x7f6c08105cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.853+0000 7f6c0ca00640 1 --2- 172.21.15.16:0/4265481897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c081058c0 0x7f6c08105cd0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8009900 tx=0x7f6bf802a3f0 comp rx=0 tx=0).stop 2024-08-31T06:18:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.853+0000 7f6c0ca00640 1 -- 172.21.15.16:0/4265481897 shutdown_connections 2024-08-31T06:18:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.853+0000 7f6c0ca00640 1 --2- 172.21.15.16:0/4265481897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c0810b1f0 0x7f6c0810b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.853+0000 7f6c0ca00640 1 --2- 172.21.15.16:0/4265481897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c0810bb40 0x7f6c08112370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.853+0000 7f6c0ca00640 1 --2- 172.21.15.16:0/4265481897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c081058c0 0x7f6c08105cd0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.853+0000 7f6c0ca00640 1 -- 172.21.15.16:0/4265481897 >> 172.21.15.16:0/4265481897 conn(0x7f6c08101300 msgr2=0x7f6c08103750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.853+0000 7f6c0ca00640 1 -- 172.21.15.16:0/4265481897 shutdown_connections 2024-08-31T06:18:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.853+0000 7f6c0ca00640 1 -- 172.21.15.16:0/4265481897 wait complete. 2024-08-31T06:18:06.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.854+0000 7f6c0ca00640 1 Processor -- start 2024-08-31T06:18:06.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.854+0000 7f6c0ca00640 1 -- start start 2024-08-31T06:18:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.855+0000 7f6c0ca00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c081058c0 0x7f6c0819c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.855+0000 7f6c0ca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c0810b1f0 0x7f6c0819c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.855+0000 7f6c0ca00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c0810bb40 0x7f6c081a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.855+0000 7f6c0ca00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6c08115800 con 0x7f6c0810b1f0 2024-08-31T06:18:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.855+0000 7f6c0ca00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6c08115680 con 0x7f6c0810bb40 2024-08-31T06:18:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.855+0000 7f6c0ca00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6c08115980 con 0x7f6c081058c0 2024-08-31T06:18:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.855+0000 7f6bffe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c0810b1f0 0x7f6c0819c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.855+0000 7f6bffe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c0810b1f0 0x7f6c0819c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56150/0 (socket says 172.21.15.16:56150) 2024-08-31T06:18:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.855+0000 7f6bffe00640 1 -- 172.21.15.16:0/2169133254 learned_addr learned my addr 172.21.15.16:0/2169133254 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.855+0000 7f6c06800640 1 --2- 172.21.15.16:0/2169133254 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c081058c0 0x7f6c0819c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.855+0000 7f6c07200640 1 --2- 172.21.15.16:0/2169133254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c0810bb40 0x7f6c081a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.855+0000 7f6bffe00640 1 -- 172.21.15.16:0/2169133254 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c081058c0 msgr2=0x7f6c0819c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.855+0000 7f6bffe00640 1 --2- 172.21.15.16:0/2169133254 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c081058c0 0x7f6c0819c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.855+0000 7f6bffe00640 1 -- 172.21.15.16:0/2169133254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c0810bb40 msgr2=0x7f6c081a1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.855+0000 7f6bffe00640 1 --2- 172.21.15.16:0/2169133254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c0810bb40 0x7f6c081a1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.855+0000 7f6bffe00640 1 -- 172.21.15.16:0/2169133254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c081a2120 con 0x7f6c0810b1f0 2024-08-31T06:18:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.856+0000 7f6bffe00640 1 --2- 172.21.15.16:0/2169133254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c0810b1f0 0x7f6c0819c820 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6bf000e9c0 tx=0x7f6bf000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.856+0000 7f6bff400640 1 -- 172.21.15.16:0/2169133254 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf000ccc0 con 0x7f6c0810b1f0 2024-08-31T06:18:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.856+0000 7f6bff400640 1 -- 172.21.15.16:0/2169133254 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bf00054d0 con 0x7f6c0810b1f0 2024-08-31T06:18:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.856+0000 7f6bff400640 1 -- 172.21.15.16:0/2169133254 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf0010430 con 0x7f6c0810b1f0 2024-08-31T06:18:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.856+0000 7f6c0ca00640 1 -- 172.21.15.16:0/2169133254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c081a2410 con 0x7f6c0810b1f0 2024-08-31T06:18:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.856+0000 7f6c0ca00640 1 -- 172.21.15.16:0/2169133254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c081a8c30 con 0x7f6c0810b1f0 2024-08-31T06:18:06.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.859+0000 7f6c1442f740 1 Processor -- start 2024-08-31T06:18:06.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.859+0000 7f6c1442f740 1 -- start start 2024-08-31T06:18:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.860+0000 7f6c1442f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8e1531b0 0x556c8e4f68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.860+0000 7f6c1442f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8e12a2f0 0x556c8e4f6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.860+0000 7f6c1442f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c8e1324d0 0x556c8e4f7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.860+0000 7f6c1442f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556c8e46bf70 con 0x556c8e12a2f0 2024-08-31T06:18:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.860+0000 7f6c1442f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556c8e45b500 con 0x556c8e1324d0 2024-08-31T06:18:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.860+0000 7f6c1442f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556c8e45f4b0 con 0x556c8e1531b0 2024-08-31T06:18:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.860+0000 7f6bff400640 1 -- 172.21.15.16:0/2169133254 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f6bf0005020 con 0x7f6c0810b1f0 2024-08-31T06:18:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.860+0000 7f6c06800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8e12a2f0 0x556c8e4f6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.860+0000 7f6c06800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8e12a2f0 0x556c8e4f6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56156/0 (socket says 172.21.15.16:56156) 2024-08-31T06:18:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.860+0000 7f6c06800640 1 -- 172.21.15.16:0/4099585957 learned_addr learned my addr 172.21.15.16:0/4099585957 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.860+0000 7f6c07200640 1 --2- 172.21.15.16:0/4099585957 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8e1531b0 0x556c8e4f68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.860+0000 7f6bffe00640 1 --2- 172.21.15.16:0/4099585957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c8e1324d0 0x556c8e4f7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.861+0000 7f6c06800640 1 -- 172.21.15.16:0/4099585957 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8e1531b0 msgr2=0x556c8e4f68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.861+0000 7f6c06800640 1 --2- 172.21.15.16:0/4099585957 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8e1531b0 0x556c8e4f68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.861+0000 7f6c06800640 1 -- 172.21.15.16:0/4099585957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c8e1324d0 msgr2=0x556c8e4f7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.861+0000 7f6c06800640 1 --2- 172.21.15.16:0/4099585957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c8e1324d0 0x556c8e4f7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.861+0000 7f6c06800640 1 -- 172.21.15.16:0/4099585957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c8e415c10 con 0x556c8e12a2f0 2024-08-31T06:18:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.861+0000 7f6bffe00640 1 --2- 172.21.15.16:0/4099585957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c8e1324d0 0x556c8e4f7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:18:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.861+0000 7f6c06800640 1 --2- 172.21.15.16:0/4099585957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8e12a2f0 0x556c8e4f6de0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8009900 tx=0x7f6bf8032cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b2b1f694b8b230d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.861+0000 7f6bf7400640 1 -- 172.21.15.16:0/4099585957 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf8040070 con 0x556c8e12a2f0 2024-08-31T06:18:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.861+0000 7f6bf7400640 1 -- 172.21.15.16:0/4099585957 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bf80316a0 con 0x556c8e12a2f0 2024-08-31T06:18:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.861+0000 7f6bf7400640 1 -- 172.21.15.16:0/4099585957 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf8035040 con 0x556c8e12a2f0 2024-08-31T06:18:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.862+0000 7f6bff400640 1 --2- 172.21.15.16:0/2169133254 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6bec0b4ca0 0x7f6bec0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.862+0000 7f6bff400640 1 -- 172.21.15.16:0/2169133254 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8514+0+0 (secure 0 0 0) 0x7f6bf00e59b0 con 0x7f6c0810b1f0 2024-08-31T06:18:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.862+0000 7f6c07200640 1 --2- 172.21.15.16:0/2169133254 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6bec0b4ca0 0x7f6bec0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.862+0000 7f6c1442f740 1 -- 172.21.15.16:0/4099585957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8e12a2f0 msgr2=0x556c8e4f6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.862+0000 7f6c1442f740 1 --2- 172.21.15.16:0/4099585957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8e12a2f0 0x556c8e4f6de0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8009900 tx=0x7f6bf8032cd0 comp rx=0 tx=0).stop 2024-08-31T06:18:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.863+0000 7f6c1442f740 1 -- 172.21.15.16:0/4099585957 shutdown_connections 2024-08-31T06:18:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.863+0000 7f6c1442f740 1 --2- 172.21.15.16:0/4099585957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c8e1324d0 0x556c8e4f7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.863+0000 7f6c1442f740 1 --2- 172.21.15.16:0/4099585957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8e12a2f0 0x556c8e4f6de0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.863+0000 7f6c1442f740 1 --2- 172.21.15.16:0/4099585957 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8e1531b0 0x556c8e4f68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.863+0000 7f6c1442f740 1 -- 172.21.15.16:0/4099585957 >> 172.21.15.16:0/4099585957 conn(0x556c8e147300 msgr2=0x556c8e4f4790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.863+0000 7f6c1442f740 1 -- 172.21.15.16:0/4099585957 shutdown_connections 2024-08-31T06:18:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.863+0000 7f6c1442f740 1 -- 172.21.15.16:0/4099585957 wait complete. 2024-08-31T06:18:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.863+0000 7f6c07200640 1 --2- 172.21.15.16:0/2169133254 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6bec0b4ca0 0x7f6bec0b71b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6c00007ed0 tx=0x7f6c00007680 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.864+0000 7f6c1442f740 1 -- start start 2024-08-31T06:18:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.864+0000 7f6c1442f740 1 Processor -- start 2024-08-31T06:18:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.864+0000 7f6c1442f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c8e12a2f0 0x556c8e501de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.864+0000 7f6c1442f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8e1324d0 0x556c8e502320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.864+0000 7f6c1442f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8e1531b0 0x556c8e502860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.864+0000 7f6c1442f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556c8e394ab0 con 0x556c8e1531b0 2024-08-31T06:18:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.864+0000 7f6c1442f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556c8e402320 con 0x556c8e12a2f0 2024-08-31T06:18:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.864+0000 7f6c1442f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556c8e504fc0 con 0x556c8e1324d0 2024-08-31T06:18:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.864+0000 7f6c1442f740 10 client.0 _ll_register_callbacks cb 0x556c8e43be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:18:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.865+0000 7f6c07200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8e1531b0 0x556c8e502860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.865+0000 7f6c07200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8e1531b0 0x556c8e502860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56170/0 (socket says 172.21.15.16:56170) 2024-08-31T06:18:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.865+0000 7f6c07200640 1 -- 172.21.15.16:0/1226511248 learned_addr learned my addr 172.21.15.16:0/1226511248 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.865+0000 7f6bffe00640 1 --2- 172.21.15.16:0/1226511248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8e1324d0 0x556c8e502320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.865+0000 7f6c06800640 1 --2- 172.21.15.16:0/1226511248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c8e12a2f0 0x556c8e501de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.865+0000 7f6c07200640 1 -- 172.21.15.16:0/1226511248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8e1324d0 msgr2=0x556c8e502320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.865+0000 7f6c07200640 1 --2- 172.21.15.16:0/1226511248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8e1324d0 0x556c8e502320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.865+0000 7f6c07200640 1 -- 172.21.15.16:0/1226511248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c8e12a2f0 msgr2=0x556c8e501de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.865+0000 7f6c07200640 1 --2- 172.21.15.16:0/1226511248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c8e12a2f0 0x556c8e501de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.865+0000 7f6c07200640 1 -- 172.21.15.16:0/1226511248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c8e1be790 con 0x556c8e1531b0 2024-08-31T06:18:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.865+0000 7f6bffe00640 1 --2- 172.21.15.16:0/1226511248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8e1324d0 0x556c8e502320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:18:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.865+0000 7f6c07200640 1 --2- 172.21.15.16:0/1226511248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8e1531b0 0x556c8e502860 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6c0000edc0 tx=0x7f6c00004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb72c26fe8ae51d7 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.865+0000 7f6bebe00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:18:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.866+0000 7f6bebe00640 1 -- 172.21.15.16:0/1226511248 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0001cd30 con 0x556c8e1531b0 2024-08-31T06:18:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.866+0000 7f6c1442f740 1 -- 172.21.15.16:0/1226511248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bf80316a0 con 0x556c8e1531b0 2024-08-31T06:18:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.866+0000 7f6bebe00640 1 -- 172.21.15.16:0/1226511248 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8514+0+0 (secure 0 0 0) 0x7f6c0001d410 con 0x556c8e1531b0 2024-08-31T06:18:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.866+0000 7f6bebe00640 1 -- 172.21.15.16:0/1226511248 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f6c0001d610 con 0x556c8e1531b0 2024-08-31T06:18:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.866+0000 7f6c1442f740 10 client.17667 fetch_fsmap learned FSMap version 46 2024-08-31T06:18:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.866+0000 7f6c1442f740 1 -- 172.21.15.16:0/1226511248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x556c8e237620 con 0x556c8e1531b0 2024-08-31T06:18:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.866+0000 7f6bebe00640 1 -- 172.21.15.16:0/1226511248 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bf80316a0 con 0x556c8e1531b0 2024-08-31T06:18:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.866+0000 7f6bebe00640 1 -- 172.21.15.16:0/1226511248 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7f6c00031930 con 0x556c8e1531b0 2024-08-31T06:18:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.866+0000 7f6c1442f740 10 client.17667 fetch_fsmap finished waiting for FSMap version 46 2024-08-31T06:18:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.866+0000 7f6c1442f740 10 client.17667 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:18:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.866+0000 7f6c1442f740 20 client.17667 populate_metadata read hostname 'smithi016' 2024-08-31T06:18:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.867+0000 7f6c1442f740 1 --2- 172.21.15.16:0/1226511248 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556c8e111d80 0x556c8e513f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.867+0000 7f6c1442f740 4 client.17667 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:18:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.867+0000 7f6c1442f740 1 -- 172.21.15.16:0/1226511248 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c8e46dec0 con 0x556c8e111d80 2024-08-31T06:18:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.867+0000 7f6c06800640 1 --2- 172.21.15.16:0/1226511248 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556c8e111d80 0x556c8e513f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.868+0000 7f6c06800640 1 --2- 172.21.15.16:0/1226511248 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556c8e111d80 0x556c8e513f70 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73691ef4a73bcecd server_cookie=f294104be32764ee in_seq=0 out_seq=0 2024-08-31T06:18:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.868+0000 7f6bebe00640 10 client.17667 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:18:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.869+0000 7f6bebe00640 1 -- 172.21.15.16:0/1226511248 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556c8e46dec0 con 0x556c8e111d80 2024-08-31T06:18:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.869+0000 7f6bebe00640 10 client.17667 handle_command_reply: tid=0 2024-08-31T06:18:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.869+0000 7f6c1442f740 1 client.17667 shutdown 2024-08-31T06:18:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.869+0000 7f6c1442f740 1 -- 172.21.15.16:0/1226511248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8e1531b0 msgr2=0x556c8e502860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.869+0000 7f6c1442f740 1 --2- 172.21.15.16:0/1226511248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8e1531b0 0x556c8e502860 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6c0000edc0 tx=0x7f6c00004a40 comp rx=0 tx=0).stop 2024-08-31T06:18:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.869+0000 7f6c1442f740 1 -- 172.21.15.16:0/1226511248 shutdown_connections 2024-08-31T06:18:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.869+0000 7f6c1442f740 1 --2- 172.21.15.16:0/1226511248 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556c8e111d80 0x556c8e513f70 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.869+0000 7f6c1442f740 1 --2- 172.21.15.16:0/1226511248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8e1324d0 0x556c8e502320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.869+0000 7f6c1442f740 1 --2- 172.21.15.16:0/1226511248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8e1531b0 0x556c8e502860 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.869+0000 7f6c1442f740 1 --2- 172.21.15.16:0/1226511248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c8e12a2f0 0x556c8e501de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.869+0000 7f6bebe00640 0 client.17667 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:18:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.869+0000 7f6c1442f740 1 -- 172.21.15.16:0/1226511248 >> 172.21.15.16:0/1226511248 conn(0x556c8e147300 msgr2=0x556c8e4f78d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.869+0000 7f6c1442f740 1 -- 172.21.15.16:0/1226511248 shutdown_connections 2024-08-31T06:18:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.869+0000 7f6c1442f740 1 -- 172.21.15.16:0/1226511248 wait complete. 2024-08-31T06:18:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.869+0000 7f6c1442f740 20 client.17667 trim_cache size 0 max 16384 2024-08-31T06:18:06.887 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.885+0000 7f6c1442f740 1 Processor -- start 2024-08-31T06:18:06.887 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.885+0000 7f6c1442f740 1 -- start start 2024-08-31T06:18:06.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.885+0000 7f6c1442f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8e12a2f0 0x556c8e4f3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:06.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.885+0000 7f6c1442f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8e1324d0 0x556c8e4f7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:06.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.885+0000 7f6c1442f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c8e1531b0 0x556c8e4fefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:06.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.885+0000 7f6c1442f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6c081b3d50 con 0x556c8e12a2f0 2024-08-31T06:18:06.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.885+0000 7f6c1442f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6c081b1590 con 0x556c8e1531b0 2024-08-31T06:18:06.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.885+0000 7f6c1442f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6c081af400 con 0x556c8e1324d0 2024-08-31T06:18:06.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.885+0000 7f6c06800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8e12a2f0 0x556c8e4f3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:06.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.885+0000 7f6bffe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8e1324d0 0x556c8e4f7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:06.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.885+0000 7f6bffe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8e1324d0 0x556c8e4f7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:33346/0 (socket says 172.21.15.16:33346) 2024-08-31T06:18:06.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.885+0000 7f6bffe00640 1 -- 172.21.15.16:0/1703320983 learned_addr learned my addr 172.21.15.16:0/1703320983 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:06.889 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.885+0000 7f6c06800640 1 --2- 172.21.15.16:0/1703320983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8e12a2f0 0x556c8e4f3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56180/0 (socket says 172.21.15.16:56180) 2024-08-31T06:18:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.886+0000 7f6c07200640 1 --2- 172.21.15.16:0/1703320983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c8e1531b0 0x556c8e4fefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.886+0000 7f6bffe00640 1 -- 172.21.15.16:0/1703320983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c8e1531b0 msgr2=0x556c8e4fefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.886+0000 7f6bffe00640 1 --2- 172.21.15.16:0/1703320983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c8e1531b0 0x556c8e4fefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.886+0000 7f6bffe00640 1 -- 172.21.15.16:0/1703320983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8e12a2f0 msgr2=0x556c8e4f3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.886+0000 7f6bffe00640 1 --2- 172.21.15.16:0/1703320983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8e12a2f0 0x556c8e4f3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.886+0000 7f6bffe00640 1 -- 172.21.15.16:0/1703320983 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c8e4eeaa0 con 0x556c8e1324d0 2024-08-31T06:18:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.886+0000 7f6bffe00640 1 --2- 172.21.15.16:0/1703320983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8e1324d0 0x556c8e4f7fe0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6bf00a85e0 tx=0x7f6bf00e8f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50150ce56bb08395 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.886+0000 7f6bf6a00640 1 -- 172.21.15.16:0/1703320983 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf0005680 con 0x556c8e1324d0 2024-08-31T06:18:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.886+0000 7f6bf6a00640 1 -- 172.21.15.16:0/1703320983 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bf00abb50 con 0x556c8e1324d0 2024-08-31T06:18:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.886+0000 7f6bf6a00640 1 -- 172.21.15.16:0/1703320983 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf00ac2a0 con 0x556c8e1324d0 2024-08-31T06:18:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.887+0000 7f6c1442f740 1 -- 172.21.15.16:0/1703320983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8e1324d0 msgr2=0x556c8e4f7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.887+0000 7f6c1442f740 1 --2- 172.21.15.16:0/1703320983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8e1324d0 0x556c8e4f7fe0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6bf00a85e0 tx=0x7f6bf00e8f90 comp rx=0 tx=0).stop 2024-08-31T06:18:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.887+0000 7f6c1442f740 1 -- 172.21.15.16:0/1703320983 shutdown_connections 2024-08-31T06:18:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.887+0000 7f6c1442f740 1 --2- 172.21.15.16:0/1703320983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8e1324d0 0x556c8e4f7fe0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.887+0000 7f6c1442f740 1 --2- 172.21.15.16:0/1703320983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c8e1531b0 0x556c8e4fefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.887+0000 7f6c1442f740 1 --2- 172.21.15.16:0/1703320983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8e12a2f0 0x556c8e4f3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.887+0000 7f6c1442f740 1 -- 172.21.15.16:0/1703320983 >> 172.21.15.16:0/1703320983 conn(0x556c8e147300 msgr2=0x556c8e4f78d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.887+0000 7f6c1442f740 1 -- 172.21.15.16:0/1703320983 shutdown_connections 2024-08-31T06:18:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.887+0000 7f6c1442f740 1 -- 172.21.15.16:0/1703320983 wait complete. 2024-08-31T06:18:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.888+0000 7f6c1442f740 1 -- start start 2024-08-31T06:18:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.888+0000 7f6c1442f740 1 Processor -- start 2024-08-31T06:18:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.888+0000 7f6c1442f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c8e12a2f0 0x556c8e508150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.888+0000 7f6c1442f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8e1324d0 0x556c8e508690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.888+0000 7f6c1442f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8e1531b0 0x556c8e508bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.888+0000 7f6c1442f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6c081afa50 con 0x556c8e1531b0 2024-08-31T06:18:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.888+0000 7f6c1442f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6c081b0ec0 con 0x556c8e12a2f0 2024-08-31T06:18:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.888+0000 7f6c1442f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6c081b0120 con 0x556c8e1324d0 2024-08-31T06:18:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.888+0000 7f6c1442f740 10 client.0 _ll_register_callbacks cb 0x7f6c00007b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:18:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.889+0000 7f6bffe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8e1324d0 0x556c8e508690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.889+0000 7f6bffe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8e1324d0 0x556c8e508690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:33356/0 (socket says 172.21.15.16:33356) 2024-08-31T06:18:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.889+0000 7f6bffe00640 1 -- 172.21.15.16:0/3447189063 learned_addr learned my addr 172.21.15.16:0/3447189063 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.889+0000 7f6c07200640 1 --2- 172.21.15.16:0/3447189063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8e1531b0 0x556c8e508bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.889+0000 7f6c06800640 1 --2- 172.21.15.16:0/3447189063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c8e12a2f0 0x556c8e508150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.889+0000 7f6bffe00640 1 -- 172.21.15.16:0/3447189063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c8e12a2f0 msgr2=0x556c8e508150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.889+0000 7f6bffe00640 1 --2- 172.21.15.16:0/3447189063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c8e12a2f0 0x556c8e508150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.889+0000 7f6bffe00640 1 -- 172.21.15.16:0/3447189063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8e1531b0 msgr2=0x556c8e508bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.889+0000 7f6bffe00640 1 --2- 172.21.15.16:0/3447189063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8e1531b0 0x556c8e508bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.889+0000 7f6bffe00640 1 -- 172.21.15.16:0/3447189063 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c8e57d980 con 0x556c8e1324d0 2024-08-31T06:18:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.889+0000 7f6c06800640 1 --2- 172.21.15.16:0/3447189063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c8e12a2f0 0x556c8e508150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:18:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.889+0000 7f6bffe00640 1 --2- 172.21.15.16:0/3447189063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8e1324d0 0x556c8e508690 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6bf00a80c0 tx=0x7f6bf0004920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8c9e1beb8e3cf8b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.889+0000 7f6bebe00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:18:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.890+0000 7f6bebe00640 1 -- 172.21.15.16:0/3447189063 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf00b08b0 con 0x556c8e1324d0 2024-08-31T06:18:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.890+0000 7f6c1442f740 1 -- 172.21.15.16:0/3447189063 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bf00abb50 con 0x556c8e1324d0 2024-08-31T06:18:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.890+0000 7f6bebe00640 1 -- 172.21.15.16:0/3447189063 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8514+0+0 (secure 0 0 0) 0x7f6bf00b50e0 con 0x556c8e1324d0 2024-08-31T06:18:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.890+0000 7f6bebe00640 1 -- 172.21.15.16:0/3447189063 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f6bf00ac740 con 0x556c8e1324d0 2024-08-31T06:18:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.890+0000 7f6c1442f740 10 client.28036 fetch_fsmap learned FSMap version 46 2024-08-31T06:18:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.890+0000 7f6c1442f740 1 -- 172.21.15.16:0/3447189063 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x556c8e57eb80 con 0x556c8e1324d0 2024-08-31T06:18:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.890+0000 7f6bebe00640 1 -- 172.21.15.16:0/3447189063 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bf00abb50 con 0x556c8e1324d0 2024-08-31T06:18:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.891+0000 7f6bebe00640 1 -- 172.21.15.16:0/3447189063 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7f6bf00a79b0 con 0x556c8e1324d0 2024-08-31T06:18:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.891+0000 7f6c1442f740 10 client.28036 fetch_fsmap finished waiting for FSMap version 46 2024-08-31T06:18:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.891+0000 7f6c1442f740 10 client.28036 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:18:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.891+0000 7f6c1442f740 20 client.28036 populate_metadata read hostname 'smithi016' 2024-08-31T06:18:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.891+0000 7f6c1442f740 1 --2- 172.21.15.16:0/3447189063 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556c8e0e71c0 0x556c8e581300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.891+0000 7f6c1442f740 4 client.28036 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:18:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.891+0000 7f6c1442f740 1 -- 172.21.15.16:0/3447189063 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556c8e469960 con 0x556c8e0e71c0 2024-08-31T06:18:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.891+0000 7f6c06800640 1 --2- 172.21.15.16:0/3447189063 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556c8e0e71c0 0x556c8e581300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.891+0000 7f6c06800640 1 --2- 172.21.15.16:0/3447189063 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556c8e0e71c0 0x556c8e581300 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58db3ee9e0955ca server_cookie=c063a9529c77c0e in_seq=0 out_seq=0 2024-08-31T06:18:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.891+0000 7f6bebe00640 10 client.28036 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:18:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.909+0000 7f6bebe00640 1 -- 172.21.15.16:0/3447189063 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556c8e469960 con 0x556c8e0e71c0 2024-08-31T06:18:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.909+0000 7f6bebe00640 10 client.28036 handle_command_reply: tid=0 2024-08-31T06:18:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.909+0000 7f6c1442f740 1 client.28036 shutdown 2024-08-31T06:18:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.909+0000 7f6c1442f740 1 -- 172.21.15.16:0/3447189063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8e1324d0 msgr2=0x556c8e508690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.909+0000 7f6c1442f740 1 --2- 172.21.15.16:0/3447189063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8e1324d0 0x556c8e508690 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6bf00a80c0 tx=0x7f6bf0004920 comp rx=0 tx=0).stop 2024-08-31T06:18:06.913 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.909+0000 7f6c1442f740 1 -- 172.21.15.16:0/3447189063 shutdown_connections 2024-08-31T06:18:06.913 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.909+0000 7f6c1442f740 1 --2- 172.21.15.16:0/3447189063 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556c8e0e71c0 0x556c8e581300 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:06.913 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.910+0000 7f6c1442f740 1 --2- 172.21.15.16:0/3447189063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8e1324d0 0x556c8e508690 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:06.913 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.910+0000 7f6c1442f740 1 --2- 172.21.15.16:0/3447189063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8e1531b0 0x556c8e508bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:06.913 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.910+0000 7f6bebe00640 0 client.28036 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:18:06.913 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.910+0000 7f6c1442f740 1 --2- 172.21.15.16:0/3447189063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c8e12a2f0 0x556c8e508150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:06.913 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.910+0000 7f6c1442f740 1 -- 172.21.15.16:0/3447189063 >> 172.21.15.16:0/3447189063 conn(0x556c8e147300 msgr2=0x556c8e4f3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:06.913 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.910+0000 7f6c1442f740 1 -- 172.21.15.16:0/3447189063 shutdown_connections 2024-08-31T06:18:06.913 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.910+0000 7f6c1442f740 1 -- 172.21.15.16:0/3447189063 wait complete. 2024-08-31T06:18:06.913 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.910+0000 7f6c1442f740 20 client.28036 trim_cache size 0 max 16384 2024-08-31T06:18:06.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:18:06.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:18:06.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:18:06.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:18:06.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.910+0000 7f6c0ca00640 1 -- 172.21.15.16:0/2169133254 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6bec0b4ca0 msgr2=0x7f6bec0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:06.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.910+0000 7f6c0ca00640 1 --2- 172.21.15.16:0/2169133254 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6bec0b4ca0 0x7f6bec0b71b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6c00007ed0 tx=0x7f6c00007680 comp rx=0 tx=0).stop 2024-08-31T06:18:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.910+0000 7f6c0ca00640 1 -- 172.21.15.16:0/2169133254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c0810b1f0 msgr2=0x7f6c0819c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.910+0000 7f6c0ca00640 1 --2- 172.21.15.16:0/2169133254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c0810b1f0 0x7f6c0819c820 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6bf000e9c0 tx=0x7f6bf000ee80 comp rx=0 tx=0).stop 2024-08-31T06:18:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.911+0000 7f6c0ca00640 1 -- 172.21.15.16:0/2169133254 shutdown_connections 2024-08-31T06:18:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.911+0000 7f6c0ca00640 1 --2- 172.21.15.16:0/2169133254 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6bec0b4ca0 0x7f6bec0b71b0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.911+0000 7f6c0ca00640 1 --2- 172.21.15.16:0/2169133254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c0810bb40 0x7f6c081a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.911+0000 7f6c0ca00640 1 --2- 172.21.15.16:0/2169133254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c0810b1f0 0x7f6c0819c820 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.911+0000 7f6c0ca00640 1 --2- 172.21.15.16:0/2169133254 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c081058c0 0x7f6c0819c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.911+0000 7f6c0ca00640 1 -- 172.21.15.16:0/2169133254 >> 172.21.15.16:0/2169133254 conn(0x7f6c08101300 msgr2=0x7f6c08103720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.911+0000 7f6c0ca00640 1 -- 172.21.15.16:0/2169133254 shutdown_connections 2024-08-31T06:18:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:06.911+0000 7f6c0ca00640 1 -- 172.21.15.16:0/2169133254 wait complete. 2024-08-31T06:18:07.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:06 smithi155 ceph-mon[21742]: pgmap v701: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 3.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 190 KiB/s wr, 33 op/s 2024-08-31T06:18:07.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:06 smithi016 ceph-mon[17194]: pgmap v701: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 3.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 190 KiB/s wr, 33 op/s 2024-08-31T06:18:07.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:06 smithi184 ceph-mon[22331]: pgmap v701: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 3.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 190 KiB/s wr, 33 op/s 2024-08-31T06:18:07.459 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:18:07.850 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:18:08.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.492+0000 7f267d400640 1 -- 172.21.15.16:0/1027218772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2678075e20 msgr2=0x7f267810f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:08.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.492+0000 7f267d400640 1 --2- 172.21.15.16:0/1027218772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2678075e20 0x7f267810f7d0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f2664009900 tx=0x7f266402a3f0 comp rx=0 tx=0).stop 2024-08-31T06:18:08.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.492+0000 7f267d400640 1 -- 172.21.15.16:0/1027218772 shutdown_connections 2024-08-31T06:18:08.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.492+0000 7f267d400640 1 --2- 172.21.15.16:0/1027218772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f267810fd10 0x7f2678112130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:08.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.492+0000 7f267d400640 1 --2- 172.21.15.16:0/1027218772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2678075e20 0x7f267810f7d0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:08.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.492+0000 7f267d400640 1 --2- 172.21.15.16:0/1027218772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26780754d0 0x7f26780758e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:08.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.492+0000 7f267d400640 1 -- 172.21.15.16:0/1027218772 >> 172.21.15.16:0/1027218772 conn(0x7f26780ff180 msgr2=0x7f26781015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:08.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.492+0000 7f267d400640 1 -- 172.21.15.16:0/1027218772 shutdown_connections 2024-08-31T06:18:08.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.492+0000 7f267d400640 1 -- 172.21.15.16:0/1027218772 wait complete. 2024-08-31T06:18:08.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.493+0000 7f267d400640 1 Processor -- start 2024-08-31T06:18:08.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.494+0000 7f267d400640 1 -- start start 2024-08-31T06:18:08.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.494+0000 7f267d400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26780754d0 0x7f267819c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:08.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.494+0000 7f267d400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2678075e20 0x7f267819c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:08.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.494+0000 7f267d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f267810fd10 0x7f26781a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:08.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.494+0000 7f267d400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2678115850 con 0x7f267810fd10 2024-08-31T06:18:08.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.494+0000 7f267d400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f26781156d0 con 0x7f26780754d0 2024-08-31T06:18:08.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.494+0000 7f267d400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f26781159d0 con 0x7f2678075e20 2024-08-31T06:18:08.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.494+0000 7f267ca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f267810fd10 0x7f26781a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:08.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.495+0000 7f267ca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f267810fd10 0x7f26781a1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56204/0 (socket says 172.21.15.16:56204) 2024-08-31T06:18:08.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.495+0000 7f267ca00640 1 -- 172.21.15.16:0/3860251780 learned_addr learned my addr 172.21.15.16:0/3860251780 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:08.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.495+0000 7f2677e00640 1 --2- 172.21.15.16:0/3860251780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26780754d0 0x7f267819c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:08.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.495+0000 7f2677400640 1 --2- 172.21.15.16:0/3860251780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2678075e20 0x7f267819c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:08.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.495+0000 7f2677e00640 1 -- 172.21.15.16:0/3860251780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2678075e20 msgr2=0x7f267819c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:08.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.495+0000 7f2677e00640 1 --2- 172.21.15.16:0/3860251780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2678075e20 0x7f267819c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:08.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.495+0000 7f2677e00640 1 -- 172.21.15.16:0/3860251780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f267810fd10 msgr2=0x7f26781a1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:08.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.495+0000 7f2677e00640 1 --2- 172.21.15.16:0/3860251780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f267810fd10 0x7f26781a1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:08.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.495+0000 7f2677e00640 1 -- 172.21.15.16:0/3860251780 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26781a2230 con 0x7f26780754d0 2024-08-31T06:18:08.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.495+0000 7f267ca00640 1 --2- 172.21.15.16:0/3860251780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f267810fd10 0x7f26781a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:18:08.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.495+0000 7f2677e00640 1 --2- 172.21.15.16:0/3860251780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26780754d0 0x7f267819c2c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f266800e9f0 tx=0x7f266800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:08.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.496+0000 7f2674c00640 1 -- 172.21.15.16:0/3860251780 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266800ccc0 con 0x7f26780754d0 2024-08-31T06:18:08.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.496+0000 7f267d400640 1 -- 172.21.15.16:0/3860251780 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26781a2520 con 0x7f26780754d0 2024-08-31T06:18:08.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.496+0000 7f2674c00640 1 -- 172.21.15.16:0/3860251780 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26680108e0 con 0x7f26780754d0 2024-08-31T06:18:08.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.496+0000 7f2674c00640 1 -- 172.21.15.16:0/3860251780 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2668018880 con 0x7f26780754d0 2024-08-31T06:18:08.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.496+0000 7f267d400640 1 -- 172.21.15.16:0/3860251780 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26781a8c80 con 0x7f26780754d0 2024-08-31T06:18:08.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.497+0000 7f267d400640 1 -- 172.21.15.16:0/3860251780 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26780758e0 con 0x7f26780754d0 2024-08-31T06:18:08.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.500+0000 7f2674c00640 1 -- 172.21.15.16:0/3860251780 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f2668010430 con 0x7f26780754d0 2024-08-31T06:18:08.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.501+0000 7f2674c00640 1 --2- 172.21.15.16:0/3860251780 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f265c0b4ca0 0x7f265c0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:08.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.501+0000 7f2674c00640 1 -- 172.21.15.16:0/3860251780 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8514+0+0 (secure 0 0 0) 0x7f26680e5dc0 con 0x7f26780754d0 2024-08-31T06:18:08.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.501+0000 7f2677400640 1 --2- 172.21.15.16:0/3860251780 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f265c0b4ca0 0x7f265c0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:08.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.502+0000 7f2677400640 1 --2- 172.21.15.16:0/3860251780 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f265c0b4ca0 0x7f265c0b71b0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2664009230 tx=0x7f2664031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:08.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.503+0000 7f2674c00640 1 -- 172.21.15.16:0/3860251780 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f26680ac500 con 0x7f26780754d0 2024-08-31T06:18:08.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.740+0000 7f267d400640 1 -- 172.21.15.16:0/3860251780 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26781a8ee0 con 0x7f26780754d0 2024-08-31T06:18:08.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.741+0000 7f2674c00640 1 -- 172.21.15.16:0/3860251780 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11921 (secure 0 0 0) 0x7f26680b5090 con 0x7f26780754d0 2024-08-31T06:18:08.744 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:18:08.744 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":46,"btime":"2024-08-31T06:17:38:653550+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:17:38.342854+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24712,24724,24737]},"id":1}]} 2024-08-31T06:18:08.745 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 46 2024-08-31T06:18:08.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.744+0000 7f267d400640 1 -- 172.21.15.16:0/3860251780 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f265c0b4ca0 msgr2=0x7f265c0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:08.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.744+0000 7f267d400640 1 --2- 172.21.15.16:0/3860251780 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f265c0b4ca0 0x7f265c0b71b0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2664009230 tx=0x7f2664031400 comp rx=0 tx=0).stop 2024-08-31T06:18:08.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.744+0000 7f267d400640 1 -- 172.21.15.16:0/3860251780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26780754d0 msgr2=0x7f267819c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:08.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.744+0000 7f267d400640 1 --2- 172.21.15.16:0/3860251780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26780754d0 0x7f267819c2c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f266800e9f0 tx=0x7f266800eeb0 comp rx=0 tx=0).stop 2024-08-31T06:18:08.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.744+0000 7f267d400640 1 -- 172.21.15.16:0/3860251780 shutdown_connections 2024-08-31T06:18:08.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.744+0000 7f267d400640 1 --2- 172.21.15.16:0/3860251780 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f265c0b4ca0 0x7f265c0b71b0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:08.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.744+0000 7f267d400640 1 --2- 172.21.15.16:0/3860251780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2678075e20 0x7f267819c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:08.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.744+0000 7f267d400640 1 --2- 172.21.15.16:0/3860251780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f267810fd10 0x7f26781a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:08.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.744+0000 7f267d400640 1 --2- 172.21.15.16:0/3860251780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26780754d0 0x7f267819c2c0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:08.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.744+0000 7f267d400640 1 -- 172.21.15.16:0/3860251780 >> 172.21.15.16:0/3860251780 conn(0x7f26780ff180 msgr2=0x7f26781098f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:08.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.744+0000 7f267d400640 1 -- 172.21.15.16:0/3860251780 shutdown_connections 2024-08-31T06:18:08.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:08.744+0000 7f267d400640 1 -- 172.21.15.16:0/3860251780 wait complete. 2024-08-31T06:18:09.102 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:08 smithi016 ceph-mon[17194]: pgmap v702: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 3.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 194 KiB/s wr, 33 op/s 2024-08-31T06:18:09.103 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:08 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/3860251780' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:18:09.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:08 smithi155 ceph-mon[21742]: pgmap v702: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 3.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 194 KiB/s wr, 33 op/s 2024-08-31T06:18:09.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:08 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/3860251780' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:18:09.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:08 smithi184 ceph-mon[22331]: pgmap v702: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 3.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 194 KiB/s wr, 33 op/s 2024-08-31T06:18:09.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:08 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/3860251780' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:18:09.615 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24737 quiesce db --set-id 7a70d06a --release --await' 2024-08-31T06:18:09.615 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24737 quiesce db --set-id 7a70d06a --release --await 2024-08-31T06:18:09.998 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:18:10.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.650+0000 7f7e79600640 1 -- 172.21.15.16:0/77997445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e741141f0 msgr2=0x7f7e741166c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:10.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.650+0000 7f7e79600640 1 --2- 172.21.15.16:0/77997445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e741141f0 0x7f7e741166c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c009ca0 tx=0x7f7e6c02a8c0 comp rx=0 tx=0).stop 2024-08-31T06:18:10.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.651+0000 7f7e79600640 1 -- 172.21.15.16:0/77997445 shutdown_connections 2024-08-31T06:18:10.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.651+0000 7f7e79600640 1 --2- 172.21.15.16:0/77997445 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e74075b70 0x7f7e74075f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:10.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.651+0000 7f7e79600640 1 --2- 172.21.15.16:0/77997445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e741141f0 0x7f7e741166c0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:10.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.651+0000 7f7e79600640 1 --2- 172.21.15.16:0/77997445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e74077120 0x7f7e74075550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:10.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.651+0000 7f7e79600640 1 -- 172.21.15.16:0/77997445 >> 172.21.15.16:0/77997445 conn(0x7f7e740ff370 msgr2=0x7f7e741017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:10.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.651+0000 7f7e79600640 1 -- 172.21.15.16:0/77997445 shutdown_connections 2024-08-31T06:18:10.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.651+0000 7f7e79600640 1 -- 172.21.15.16:0/77997445 wait complete. 2024-08-31T06:18:10.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.652+0000 7f7e79600640 1 Processor -- start 2024-08-31T06:18:10.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.652+0000 7f7e79600640 1 -- start start 2024-08-31T06:18:10.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.652+0000 7f7e79600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e74075b70 0x7f7e741a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:10.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.652+0000 7f7e79600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e74077120 0x7f7e741a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:10.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.652+0000 7f7e79600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e741141f0 0x7f7e741a6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:10.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.652+0000 7f7e79600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7e74119e30 con 0x7f7e74077120 2024-08-31T06:18:10.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.652+0000 7f7e79600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7e74119cb0 con 0x7f7e741141f0 2024-08-31T06:18:10.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.652+0000 7f7e79600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7e74119fb0 con 0x7f7e74075b70 2024-08-31T06:18:10.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.653+0000 7f7e72000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e74077120 0x7f7e741a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:10.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.653+0000 7f7e72a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e74075b70 0x7f7e741a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:10.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.653+0000 7f7e72a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e74075b70 0x7f7e741a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:33412/0 (socket says 172.21.15.16:33412) 2024-08-31T06:18:10.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.653+0000 7f7e72a00640 1 -- 172.21.15.16:0/2946097449 learned_addr learned my addr 172.21.15.16:0/2946097449 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:10.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.653+0000 7f7e72000640 1 --2- 172.21.15.16:0/2946097449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e74077120 0x7f7e741a0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56212/0 (socket says 172.21.15.16:56212) 2024-08-31T06:18:10.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.653+0000 7f7e73400640 1 --2- 172.21.15.16:0/2946097449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e741141f0 0x7f7e741a6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:10.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.653+0000 7f7e72a00640 1 -- 172.21.15.16:0/2946097449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e741141f0 msgr2=0x7f7e741a6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:10.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.653+0000 7f7e72a00640 1 --2- 172.21.15.16:0/2946097449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e741141f0 0x7f7e741a6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:10.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.653+0000 7f7e72a00640 1 -- 172.21.15.16:0/2946097449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e74077120 msgr2=0x7f7e741a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:10.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.653+0000 7f7e72a00640 1 --2- 172.21.15.16:0/2946097449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e74077120 0x7f7e741a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:10.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.653+0000 7f7e72a00640 1 -- 172.21.15.16:0/2946097449 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e741a6750 con 0x7f7e74075b70 2024-08-31T06:18:10.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.653+0000 7f7e72a00640 1 --2- 172.21.15.16:0/2946097449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e74075b70 0x7f7e741a07e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7e6400e9f0 tx=0x7f7e6400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:10.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.654+0000 7f7e6b400640 1 -- 172.21.15.16:0/2946097449 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6400cd80 con 0x7f7e74075b70 2024-08-31T06:18:10.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.654+0000 7f7e79600640 1 -- 172.21.15.16:0/2946097449 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e741a6a40 con 0x7f7e74075b70 2024-08-31T06:18:10.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.654+0000 7f7e6b400640 1 -- 172.21.15.16:0/2946097449 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e640108e0 con 0x7f7e74075b70 2024-08-31T06:18:10.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.654+0000 7f7e6b400640 1 -- 172.21.15.16:0/2946097449 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e64018980 con 0x7f7e74075b70 2024-08-31T06:18:10.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.654+0000 7f7e79600640 1 -- 172.21.15.16:0/2946097449 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e741ad1a0 con 0x7f7e74075b70 2024-08-31T06:18:10.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.658+0000 7f7e80252740 1 Processor -- start 2024-08-31T06:18:10.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.658+0000 7f7e80252740 1 -- start start 2024-08-31T06:18:10.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.659+0000 7f7e80252740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55674944a520 0x5567497efef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:10.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.659+0000 7f7e80252740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556749421660 0x5567497f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:10.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.659+0000 7f7e80252740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556749429840 0x5567497f0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:10.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.659+0000 7f7e80252740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556749766a90 con 0x556749429840 2024-08-31T06:18:10.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.659+0000 7f7e80252740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556749756020 con 0x556749421660 2024-08-31T06:18:10.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.659+0000 7f7e80252740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556749759fd0 con 0x55674944a520 2024-08-31T06:18:10.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.659+0000 7f7e73400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55674944a520 0x5567497efef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:10.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.659+0000 7f7e6b400640 1 -- 172.21.15.16:0/2946097449 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f7e64018b20 con 0x7f7e74075b70 2024-08-31T06:18:10.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.659+0000 7f7e73400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55674944a520 0x5567497efef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:33424/0 (socket says 172.21.15.16:33424) 2024-08-31T06:18:10.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.659+0000 7f7e73400640 1 -- 172.21.15.16:0/3206809491 learned_addr learned my addr 172.21.15.16:0/3206809491 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:10.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.659+0000 7f7e72000640 1 --2- 172.21.15.16:0/3206809491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556749429840 0x5567497f0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:10.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.660+0000 7f7e73400640 1 -- 172.21.15.16:0/3206809491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556749421660 msgr2=0x5567497f0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:10.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.660+0000 7f7e73400640 1 --2- 172.21.15.16:0/3206809491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556749421660 0x5567497f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:10.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.660+0000 7f7e73400640 1 -- 172.21.15.16:0/3206809491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556749429840 msgr2=0x5567497f0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:10.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.660+0000 7f7e73400640 1 --2- 172.21.15.16:0/3206809491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556749429840 0x5567497f0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:10.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.660+0000 7f7e73400640 1 -- 172.21.15.16:0/3206809491 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567494b5b00 con 0x55674944a520 2024-08-31T06:18:10.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.660+0000 7f7e73400640 1 --2- 172.21.15.16:0/3206809491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55674944a520 0x5567497efef0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c02f410 tx=0x7f7e6c02f440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6472f83fde6ffe8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:10.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.661+0000 7f7e5f400640 1 -- 172.21.15.16:0/3206809491 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c031cf0 con 0x55674944a520 2024-08-31T06:18:10.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.661+0000 7f7e5f400640 1 -- 172.21.15.16:0/3206809491 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e6c036030 con 0x55674944a520 2024-08-31T06:18:10.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.661+0000 7f7e5f400640 1 -- 172.21.15.16:0/3206809491 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c0042a0 con 0x55674944a520 2024-08-31T06:18:10.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.661+0000 7f7e6b400640 1 --2- 172.21.15.16:0/2946097449 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7e580b4ca0 0x7f7e580b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:10.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.662+0000 7f7e6b400640 1 -- 172.21.15.16:0/2946097449 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8514+0+0 (secure 0 0 0) 0x7f7e640e5920 con 0x7f7e74075b70 2024-08-31T06:18:10.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.662+0000 7f7e72a00640 1 --2- 172.21.15.16:0/2946097449 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7e580b4ca0 0x7f7e580b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:10.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.662+0000 7f7e80252740 1 -- 172.21.15.16:0/3206809491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55674944a520 msgr2=0x5567497efef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:10.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.662+0000 7f7e80252740 1 --2- 172.21.15.16:0/3206809491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55674944a520 0x5567497efef0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c02f410 tx=0x7f7e6c02f440 comp rx=0 tx=0).stop 2024-08-31T06:18:10.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.662+0000 7f7e80252740 1 -- 172.21.15.16:0/3206809491 shutdown_connections 2024-08-31T06:18:10.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.662+0000 7f7e80252740 1 --2- 172.21.15.16:0/3206809491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556749429840 0x5567497f0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:10.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.662+0000 7f7e80252740 1 --2- 172.21.15.16:0/3206809491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556749421660 0x5567497f0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:10.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.662+0000 7f7e80252740 1 --2- 172.21.15.16:0/3206809491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55674944a520 0x5567497efef0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:10.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.662+0000 7f7e80252740 1 -- 172.21.15.16:0/3206809491 >> 172.21.15.16:0/3206809491 conn(0x55674943e910 msgr2=0x5567497edde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:10.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.662+0000 7f7e80252740 1 -- 172.21.15.16:0/3206809491 shutdown_connections 2024-08-31T06:18:10.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.662+0000 7f7e80252740 1 -- 172.21.15.16:0/3206809491 wait complete. 2024-08-31T06:18:10.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.662+0000 7f7e72a00640 1 --2- 172.21.15.16:0/2946097449 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7e580b4ca0 0x7f7e580b71b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x5567497eeb70 tx=0x7f7e64018420 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:10.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.663+0000 7f7e80252740 1 -- start start 2024-08-31T06:18:10.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.663+0000 7f7e80252740 1 Processor -- start 2024-08-31T06:18:10.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.664+0000 7f7e80252740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556749421660 0x5567497fb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:10.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.664+0000 7f7e80252740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556749429840 0x5567497fb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:10.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.664+0000 7f7e80252740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55674944a520 0x5567497fbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:10.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.664+0000 7f7e80252740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5567495b8bb0 con 0x556749421660 2024-08-31T06:18:10.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.664+0000 7f7e80252740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5567496dcf90 con 0x55674944a520 2024-08-31T06:18:10.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.664+0000 7f7e80252740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5567497fe4f0 con 0x556749429840 2024-08-31T06:18:10.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.664+0000 7f7e80252740 10 client.0 _ll_register_callbacks cb 0x556749702710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:18:10.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.664+0000 7f7e72000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556749429840 0x5567497fb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:10.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.664+0000 7f7e73400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556749421660 0x5567497fb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:10.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.664+0000 7f7e72000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556749429840 0x5567497fb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:33426/0 (socket says 172.21.15.16:33426) 2024-08-31T06:18:10.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.664+0000 7f7e72000640 1 -- 172.21.15.16:0/1498565358 learned_addr learned my addr 172.21.15.16:0/1498565358 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:10.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.664+0000 7f7e73400640 1 --2- 172.21.15.16:0/1498565358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55674944a520 0x5567497fbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:10.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.665+0000 7f7e72000640 1 -- 172.21.15.16:0/1498565358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55674944a520 msgr2=0x5567497fbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:10.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.665+0000 7f7e72000640 1 --2- 172.21.15.16:0/1498565358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55674944a520 0x5567497fbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:10.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.665+0000 7f7e72000640 1 -- 172.21.15.16:0/1498565358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556749421660 msgr2=0x5567497fb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:10.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.665+0000 7f7e72000640 1 --2- 172.21.15.16:0/1498565358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556749421660 0x5567497fb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:10.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.665+0000 7f7e72000640 1 -- 172.21.15.16:0/1498565358 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55674952c960 con 0x556749429840 2024-08-31T06:18:10.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.665+0000 7f7e72000640 1 --2- 172.21.15.16:0/1498565358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556749429840 0x5567497fb800 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7e60012700 tx=0x7f7e60012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec7ad40eb6087c65 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:10.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.665+0000 7f7e53e00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:18:10.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.665+0000 7f7e53e00640 1 -- 172.21.15.16:0/1498565358 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6000f8c0 con 0x556749429840 2024-08-31T06:18:10.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.665+0000 7f7e80252740 1 -- 172.21.15.16:0/1498565358 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5567496f4e30 con 0x556749429840 2024-08-31T06:18:10.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.665+0000 7f7e53e00640 1 -- 172.21.15.16:0/1498565358 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8514+0+0 (secure 0 0 0) 0x7f7e6000fa60 con 0x556749429840 2024-08-31T06:18:10.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.666+0000 7f7e53e00640 1 -- 172.21.15.16:0/1498565358 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f7e6000fc60 con 0x556749429840 2024-08-31T06:18:10.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.666+0000 7f7e80252740 10 client.28054 fetch_fsmap learned FSMap version 46 2024-08-31T06:18:10.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.666+0000 7f7e80252740 1 -- 172.21.15.16:0/1498565358 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x556749809fc0 con 0x556749429840 2024-08-31T06:18:10.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.666+0000 7f7e53e00640 1 -- 172.21.15.16:0/1498565358 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5567496f4e30 con 0x556749429840 2024-08-31T06:18:10.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.666+0000 7f7e53e00640 1 -- 172.21.15.16:0/1498565358 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7f7e6001c420 con 0x556749429840 2024-08-31T06:18:10.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.666+0000 7f7e80252740 10 client.28054 fetch_fsmap finished waiting for FSMap version 46 2024-08-31T06:18:10.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.666+0000 7f7e80252740 10 client.28054 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:18:10.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.666+0000 7f7e80252740 20 client.28054 populate_metadata read hostname 'smithi016' 2024-08-31T06:18:10.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.666+0000 7f7e80252740 1 --2- 172.21.15.16:0/1498565358 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556749408ea0 0x55674980d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:10.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.666+0000 7f7e80252740 4 client.28054 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:18:10.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.666+0000 7f7e80252740 1 -- 172.21.15.16:0/1498565358 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567497689e0 con 0x556749408ea0 2024-08-31T06:18:10.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.667+0000 7f7e73400640 1 --2- 172.21.15.16:0/1498565358 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556749408ea0 0x55674980d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:10.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.667+0000 7f7e73400640 1 --2- 172.21.15.16:0/1498565358 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556749408ea0 0x55674980d560 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbf19361b243326d server_cookie=5a9887cb3d95c6da in_seq=0 out_seq=0 2024-08-31T06:18:10.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.667+0000 7f7e53e00640 10 client.28054 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:18:10.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.668+0000 7f7e53e00640 1 -- 172.21.15.16:0/1498565358 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5567497689e0 con 0x556749408ea0 2024-08-31T06:18:10.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.668+0000 7f7e53e00640 10 client.28054 handle_command_reply: tid=0 2024-08-31T06:18:10.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.668+0000 7f7e80252740 1 client.28054 shutdown 2024-08-31T06:18:10.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.669+0000 7f7e80252740 1 -- 172.21.15.16:0/1498565358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556749429840 msgr2=0x5567497fb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:10.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.669+0000 7f7e80252740 1 --2- 172.21.15.16:0/1498565358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556749429840 0x5567497fb800 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7e60012700 tx=0x7f7e60012bc0 comp rx=0 tx=0).stop 2024-08-31T06:18:10.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.669+0000 7f7e80252740 1 -- 172.21.15.16:0/1498565358 shutdown_connections 2024-08-31T06:18:10.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.669+0000 7f7e80252740 1 --2- 172.21.15.16:0/1498565358 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x556749408ea0 0x55674980d560 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:10.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.669+0000 7f7e80252740 1 --2- 172.21.15.16:0/1498565358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556749429840 0x5567497fb800 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:10.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.669+0000 7f7e80252740 1 --2- 172.21.15.16:0/1498565358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55674944a520 0x5567497fbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:10.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.669+0000 7f7e80252740 1 --2- 172.21.15.16:0/1498565358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556749421660 0x5567497fb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:10.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.669+0000 7f7e53e00640 0 client.28054 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:18:10.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.669+0000 7f7e80252740 1 -- 172.21.15.16:0/1498565358 >> 172.21.15.16:0/1498565358 conn(0x55674943e910 msgr2=0x5567497f3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:10.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.669+0000 7f7e80252740 1 -- 172.21.15.16:0/1498565358 shutdown_connections 2024-08-31T06:18:10.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.669+0000 7f7e80252740 1 -- 172.21.15.16:0/1498565358 wait complete. 2024-08-31T06:18:10.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.669+0000 7f7e80252740 20 client.28054 trim_cache size 0 max 16384 2024-08-31T06:18:10.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.687+0000 7f7e80252740 1 Processor -- start 2024-08-31T06:18:10.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.687+0000 7f7e80252740 1 -- start start 2024-08-31T06:18:10.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.688+0000 7f7e80252740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556749421660 0x5567497fd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:10.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.688+0000 7f7e80252740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556749429840 0x5567497f84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:10.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.688+0000 7f7e80252740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674944a520 0x5567497f89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:10.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.688+0000 7f7e80252740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7e741b8350 con 0x55674944a520 2024-08-31T06:18:10.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.688+0000 7f7e80252740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7e741b5b90 con 0x556749429840 2024-08-31T06:18:10.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.688+0000 7f7e80252740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7e741b3a00 con 0x556749421660 2024-08-31T06:18:10.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.688+0000 7f7e72000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556749429840 0x5567497f84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:10.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.688+0000 7f7e73400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674944a520 0x5567497f89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:10.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.688+0000 7f7e72000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556749429840 0x5567497f84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:37254/0 (socket says 172.21.15.16:37254) 2024-08-31T06:18:10.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.688+0000 7f7e72000640 1 -- 172.21.15.16:0/3020006241 learned_addr learned my addr 172.21.15.16:0/3020006241 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:10.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.688+0000 7f7e73400640 1 --2- 172.21.15.16:0/3020006241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674944a520 0x5567497f89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56250/0 (socket says 172.21.15.16:56250) 2024-08-31T06:18:10.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.688+0000 7f7e73400640 1 --2- 172.21.15.16:0/3020006241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556749421660 0x5567497fd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:10.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.688+0000 7f7e72000640 1 -- 172.21.15.16:0/3020006241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556749421660 msgr2=0x5567497fd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:10.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.688+0000 7f7e72000640 1 --2- 172.21.15.16:0/3020006241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556749421660 0x5567497fd2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:10.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.688+0000 7f7e72000640 1 -- 172.21.15.16:0/3020006241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674944a520 msgr2=0x5567497f89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:10.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.688+0000 7f7e72000640 1 --2- 172.21.15.16:0/3020006241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674944a520 0x5567497f89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:10.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.689+0000 7f7e72000640 1 -- 172.21.15.16:0/3020006241 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556749809dd0 con 0x556749429840 2024-08-31T06:18:10.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.689+0000 7f7e72000640 1 --2- 172.21.15.16:0/3020006241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556749429840 0x5567497f84b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f7e6001c890 tx=0x7f7e6000ff90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34ce3ff7ce5b1f27 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:10.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.689+0000 7f7e5ea00640 1 -- 172.21.15.16:0/3020006241 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6001c4e0 con 0x556749429840 2024-08-31T06:18:10.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.689+0000 7f7e5ea00640 1 -- 172.21.15.16:0/3020006241 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e6001c680 con 0x556749429840 2024-08-31T06:18:10.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.689+0000 7f7e5ea00640 1 -- 172.21.15.16:0/3020006241 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e600053f0 con 0x556749429840 2024-08-31T06:18:10.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.690+0000 7f7e80252740 1 -- 172.21.15.16:0/3020006241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556749429840 msgr2=0x5567497f84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:10.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.690+0000 7f7e80252740 1 --2- 172.21.15.16:0/3020006241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556749429840 0x5567497f84b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f7e6001c890 tx=0x7f7e6000ff90 comp rx=0 tx=0).stop 2024-08-31T06:18:10.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.690+0000 7f7e80252740 1 -- 172.21.15.16:0/3020006241 shutdown_connections 2024-08-31T06:18:10.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.690+0000 7f7e80252740 1 --2- 172.21.15.16:0/3020006241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674944a520 0x5567497f89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:10.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.690+0000 7f7e80252740 1 --2- 172.21.15.16:0/3020006241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556749429840 0x5567497f84b0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:10.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.690+0000 7f7e80252740 1 --2- 172.21.15.16:0/3020006241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556749421660 0x5567497fd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:10.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.690+0000 7f7e80252740 1 -- 172.21.15.16:0/3020006241 >> 172.21.15.16:0/3020006241 conn(0x55674943e910 msgr2=0x5567497f18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:10.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.690+0000 7f7e80252740 1 -- 172.21.15.16:0/3020006241 shutdown_connections 2024-08-31T06:18:10.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.690+0000 7f7e80252740 1 -- 172.21.15.16:0/3020006241 wait complete. 2024-08-31T06:18:10.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.691+0000 7f7e80252740 1 -- start start 2024-08-31T06:18:10.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.691+0000 7f7e80252740 1 Processor -- start 2024-08-31T06:18:10.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.691+0000 7f7e80252740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556749421660 0x5567498020e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:10.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.691+0000 7f7e80252740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556749429840 0x556749802680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:10.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.691+0000 7f7e80252740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55674944a520 0x556749870f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:10.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.691+0000 7f7e80252740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7e741b4050 con 0x556749421660 2024-08-31T06:18:10.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.691+0000 7f7e80252740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7e741b54c0 con 0x55674944a520 2024-08-31T06:18:10.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.691+0000 7f7e80252740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7e741b4720 con 0x556749429840 2024-08-31T06:18:10.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.691+0000 7f7e80252740 10 client.0 _ll_register_callbacks cb 0x7f7e60009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:18:10.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.692+0000 7f7e73400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556749421660 0x5567498020e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:10.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.692+0000 7f7e72000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556749429840 0x556749802680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:10.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.692+0000 7f7e73400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556749421660 0x5567498020e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56256/0 (socket says 172.21.15.16:56256) 2024-08-31T06:18:10.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.692+0000 7f7e73400640 1 -- 172.21.15.16:0/490021917 learned_addr learned my addr 172.21.15.16:0/490021917 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:10.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.692+0000 7f7e73400640 1 --2- 172.21.15.16:0/490021917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55674944a520 0x556749870f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:10.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.692+0000 7f7e72000640 1 -- 172.21.15.16:0/490021917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55674944a520 msgr2=0x556749870f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:10.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.692+0000 7f7e72000640 1 --2- 172.21.15.16:0/490021917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55674944a520 0x556749870f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:10.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.692+0000 7f7e72000640 1 -- 172.21.15.16:0/490021917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556749421660 msgr2=0x5567498020e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:10.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.692+0000 7f7e72000640 1 --2- 172.21.15.16:0/490021917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556749421660 0x5567498020e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:10.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.692+0000 7f7e72000640 1 -- 172.21.15.16:0/490021917 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556749871570 con 0x556749429840 2024-08-31T06:18:10.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.692+0000 7f7e73400640 1 --2- 172.21.15.16:0/490021917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556749421660 0x5567498020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:18:10.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.693+0000 7f7e72000640 1 --2- 172.21.15.16:0/490021917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556749429840 0x556749802680 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x5567497f35d0 tx=0x7f7e60005090 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68794a2444018dd0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:10.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.693+0000 7f7e53e00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:18:10.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.693+0000 7f7e53e00640 1 -- 172.21.15.16:0/490021917 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e60031020 con 0x556749429840 2024-08-31T06:18:10.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.693+0000 7f7e80252740 1 -- 172.21.15.16:0/490021917 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556749764480 con 0x556749429840 2024-08-31T06:18:10.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.693+0000 7f7e53e00640 1 -- 172.21.15.16:0/490021917 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8514+0+0 (secure 0 0 0) 0x7f7e6000f9a0 con 0x556749429840 2024-08-31T06:18:10.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.693+0000 7f7e53e00640 1 -- 172.21.15.16:0/490021917 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f7e6003c050 con 0x556749429840 2024-08-31T06:18:10.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.693+0000 7f7e80252740 10 client.28060 fetch_fsmap learned FSMap version 46 2024-08-31T06:18:10.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.693+0000 7f7e80252740 1 -- 172.21.15.16:0/490021917 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x556749878980 con 0x556749429840 2024-08-31T06:18:10.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.694+0000 7f7e53e00640 1 -- 172.21.15.16:0/490021917 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556749764480 con 0x556749429840 2024-08-31T06:18:10.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.694+0000 7f7e53e00640 1 -- 172.21.15.16:0/490021917 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7f7e60003890 con 0x556749429840 2024-08-31T06:18:10.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.694+0000 7f7e80252740 10 client.28060 fetch_fsmap finished waiting for FSMap version 46 2024-08-31T06:18:10.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.694+0000 7f7e80252740 10 client.28060 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:18:10.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.694+0000 7f7e80252740 20 client.28060 populate_metadata read hostname 'smithi016' 2024-08-31T06:18:10.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.694+0000 7f7e80252740 1 --2- 172.21.15.16:0/490021917 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5567493de250 0x55674987b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:10.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.694+0000 7f7e80252740 4 client.28060 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "7a70d06a", "release": true, "await": true}] 2024-08-31T06:18:10.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.694+0000 7f7e80252740 1 -- 172.21.15.16:0/490021917 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "quiesce db", "set_id": "7a70d06a", "release": true, "await": true}) -- 0x556749761bc0 con 0x5567493de250 2024-08-31T06:18:10.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.694+0000 7f7e73400640 1 --2- 172.21.15.16:0/490021917 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5567493de250 0x55674987b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:10.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.695+0000 7f7e73400640 1 --2- 172.21.15.16:0/490021917 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5567493de250 0x55674987b080 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e76bd82a14cdb9a8 server_cookie=2ee4157344e90149 in_seq=0 out_seq=0 2024-08-31T06:18:10.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.695+0000 7f7e53e00640 10 client.28060 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:18:10.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.722+0000 7f7e53e00640 1 -- 172.21.15.16:0/490021917 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556749761bc0 con 0x5567493de250 2024-08-31T06:18:10.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.722+0000 7f7e53e00640 10 client.28060 handle_command_reply: tid=0 2024-08-31T06:18:10.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.722+0000 7f7e80252740 1 client.28060 shutdown 2024-08-31T06:18:10.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.723+0000 7f7e80252740 1 -- 172.21.15.16:0/490021917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556749429840 msgr2=0x556749802680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:10.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.723+0000 7f7e80252740 1 --2- 172.21.15.16:0/490021917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556749429840 0x556749802680 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x5567497f35d0 tx=0x7f7e60005090 comp rx=0 tx=0).stop 2024-08-31T06:18:10.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.723+0000 7f7e80252740 1 -- 172.21.15.16:0/490021917 shutdown_connections 2024-08-31T06:18:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.723+0000 7f7e80252740 1 --2- 172.21.15.16:0/490021917 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5567493de250 0x55674987b080 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.723+0000 7f7e80252740 1 --2- 172.21.15.16:0/490021917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556749429840 0x556749802680 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.723+0000 7f7e80252740 1 --2- 172.21.15.16:0/490021917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55674944a520 0x556749870f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.723+0000 7f7e80252740 1 --2- 172.21.15.16:0/490021917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556749421660 0x5567498020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.723+0000 7f7e80252740 1 -- 172.21.15.16:0/490021917 >> 172.21.15.16:0/490021917 conn(0x55674943e910 msgr2=0x5567497f16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.723+0000 7f7e53e00640 0 client.28060 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:18:10.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.723+0000 7f7e80252740 1 -- 172.21.15.16:0/490021917 shutdown_connections 2024-08-31T06:18:10.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.723+0000 7f7e80252740 1 -- 172.21.15.16:0/490021917 wait complete. 2024-08-31T06:18:10.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.724+0000 7f7e80252740 20 client.28060 trim_cache size 0 max 16384 2024-08-31T06:18:10.727 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:18:10.727 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 46, 2024-08-31T06:18:10.728 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24737, 2024-08-31T06:18:10.728 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 36, 2024-08-31T06:18:10.728 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-08-31T06:18:10.728 INFO:teuthology.orchestra.run.smithi016.stdout: "7a70d06a": { 2024-08-31T06:18:10.728 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 36, 2024-08-31T06:18:10.728 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-08-31T06:18:10.728 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:18:10.728 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-08-31T06:18:10.728 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:18:10.728 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:18:10.728 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-08-31T06:18:10.728 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 145.2, 2024-08-31T06:18:10.728 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-08-31T06:18:10.729 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-08-31T06:18:10.729 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-08-31T06:18:10.729 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:18:10.729 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-08-31T06:18:10.729 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:18:10.729 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:18:10.729 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:18:10.729 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:18:10.729 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:18:10.729 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:18:10.729 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:18:10.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.725+0000 7f7e79600640 1 -- 172.21.15.16:0/2946097449 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7e580b4ca0 msgr2=0x7f7e580b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:10.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.725+0000 7f7e79600640 1 --2- 172.21.15.16:0/2946097449 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7e580b4ca0 0x7f7e580b71b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x5567497eeb70 tx=0x7f7e64018420 comp rx=0 tx=0).stop 2024-08-31T06:18:10.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.725+0000 7f7e79600640 1 -- 172.21.15.16:0/2946097449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e74075b70 msgr2=0x7f7e741a07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:10.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.725+0000 7f7e79600640 1 --2- 172.21.15.16:0/2946097449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e74075b70 0x7f7e741a07e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7e6400e9f0 tx=0x7f7e6400eeb0 comp rx=0 tx=0).stop 2024-08-31T06:18:10.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.726+0000 7f7e79600640 1 -- 172.21.15.16:0/2946097449 shutdown_connections 2024-08-31T06:18:10.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.726+0000 7f7e79600640 1 --2- 172.21.15.16:0/2946097449 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7e580b4ca0 0x7f7e580b71b0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:10.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.726+0000 7f7e79600640 1 --2- 172.21.15.16:0/2946097449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e741141f0 0x7f7e741a6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:10.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.726+0000 7f7e79600640 1 --2- 172.21.15.16:0/2946097449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e74077120 0x7f7e741a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:10.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.726+0000 7f7e79600640 1 --2- 172.21.15.16:0/2946097449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e74075b70 0x7f7e741a07e0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:10.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.726+0000 7f7e79600640 1 -- 172.21.15.16:0/2946097449 >> 172.21.15.16:0/2946097449 conn(0x7f7e740ff370 msgr2=0x7f7e741100f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:10.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.727+0000 7f7e79600640 1 -- 172.21.15.16:0/2946097449 shutdown_connections 2024-08-31T06:18:10.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:10.727+0000 7f7e79600640 1 -- 172.21.15.16:0/2946097449 wait complete. 2024-08-31T06:18:11.077 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:10 smithi016 ceph-mon[17194]: pgmap v703: 97 pgs: 97 active+clean; 3.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 101 KiB/s wr, 33 op/s 2024-08-31T06:18:11.078 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:10 smithi016 ceph-mon[17194]: osdmap e93: 12 total, 12 up, 12 in 2024-08-31T06:18:11.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:10 smithi155 ceph-mon[21742]: pgmap v703: 97 pgs: 97 active+clean; 3.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 101 KiB/s wr, 33 op/s 2024-08-31T06:18:11.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:10 smithi155 ceph-mon[21742]: osdmap e93: 12 total, 12 up, 12 in 2024-08-31T06:18:11.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:10 smithi184 ceph-mon[22331]: pgmap v703: 97 pgs: 97 active+clean; 3.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 101 KiB/s wr, 33 op/s 2024-08-31T06:18:11.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:10 smithi184 ceph-mon[22331]: osdmap e93: 12 total, 12 up, 12 in 2024-08-31T06:18:11.609 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '7a70d06a', total seconds elapsed: 29.4 2024-08-31T06:18:11.610 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.25 2024-08-31T06:18:11.610 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:18:11.625 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 645.0 s, 124.1 tps, lat 3676.485 ms stddev 8128.457 2024-08-31T06:18:12.358 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:18:13.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:12 smithi155 ceph-mon[21742]: pgmap v705: 97 pgs: 97 active+clean; 3.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 129 KiB/s wr, 44 op/s 2024-08-31T06:18:13.165 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:12 smithi016 ceph-mon[17194]: pgmap v705: 97 pgs: 97 active+clean; 3.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 129 KiB/s wr, 44 op/s 2024-08-31T06:18:13.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.163+0000 7efe96000640 1 -- 172.21.15.16:0/2018223180 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe90111ce0 msgr2=0x7efe9010a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:13.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.163+0000 7efe96000640 1 --2- 172.21.15.16:0/2018223180 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe90111ce0 0x7efe9010a160 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7efe80009900 tx=0x7efe8002a3f0 comp rx=0 tx=0).stop 2024-08-31T06:18:13.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.163+0000 7efe96000640 1 -- 172.21.15.16:0/2018223180 shutdown_connections 2024-08-31T06:18:13.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.163+0000 7efe96000640 1 --2- 172.21.15.16:0/2018223180 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe9011c9d0 0x7efe9011eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:13.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.163+0000 7efe96000640 1 --2- 172.21.15.16:0/2018223180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe9010a780 0x7efe9010ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:13.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.163+0000 7efe96000640 1 --2- 172.21.15.16:0/2018223180 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe90111ce0 0x7efe9010a160 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:13.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.163+0000 7efe96000640 1 -- 172.21.15.16:0/2018223180 >> 172.21.15.16:0/2018223180 conn(0x7efe90070c60 msgr2=0x7efe90071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:13.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.164+0000 7efe96000640 1 -- 172.21.15.16:0/2018223180 shutdown_connections 2024-08-31T06:18:13.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.164+0000 7efe96000640 1 -- 172.21.15.16:0/2018223180 wait complete. 2024-08-31T06:18:13.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.165+0000 7efe96000640 1 Processor -- start 2024-08-31T06:18:13.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.166+0000 7efe96000640 1 -- start start 2024-08-31T06:18:13.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.166+0000 7efe96000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe9010a780 0x7efe901a90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:13.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.166+0000 7efe96000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe90111ce0 0x7efe901a9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:13.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.166+0000 7efe96000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe9011c9d0 0x7efe901ae920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:13.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.166+0000 7efe96000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efe90122690 con 0x7efe90111ce0 2024-08-31T06:18:13.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.166+0000 7efe96000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efe90122510 con 0x7efe9010a780 2024-08-31T06:18:13.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.166+0000 7efe96000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efe90122810 con 0x7efe9011c9d0 2024-08-31T06:18:13.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.167+0000 7efe8f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe9010a780 0x7efe901a90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:13.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.167+0000 7efe8fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe9011c9d0 0x7efe901ae920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:13.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.167+0000 7efe8fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe9011c9d0 0x7efe901ae920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:33456/0 (socket says 172.21.15.16:33456) 2024-08-31T06:18:13.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.167+0000 7efe8fe00640 1 -- 172.21.15.16:0/52116553 learned_addr learned my addr 172.21.15.16:0/52116553 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:13.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.167+0000 7efe8ea00640 1 --2- 172.21.15.16:0/52116553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe90111ce0 0x7efe901a9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:13.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.168+0000 7efe8f400640 1 -- 172.21.15.16:0/52116553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe9011c9d0 msgr2=0x7efe901ae920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:13.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.168+0000 7efe8f400640 1 --2- 172.21.15.16:0/52116553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe9011c9d0 0x7efe901ae920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:13.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.168+0000 7efe8f400640 1 -- 172.21.15.16:0/52116553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe90111ce0 msgr2=0x7efe901a9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:13.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.168+0000 7efe8f400640 1 --2- 172.21.15.16:0/52116553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe90111ce0 0x7efe901a9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:13.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.168+0000 7efe8f400640 1 -- 172.21.15.16:0/52116553 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe901aef90 con 0x7efe9010a780 2024-08-31T06:18:13.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.168+0000 7efe8f400640 1 --2- 172.21.15.16:0/52116553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe9010a780 0x7efe901a90e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7efe8002a9e0 tx=0x7efe80003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:13.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.169+0000 7efe87e00640 1 -- 172.21.15.16:0/52116553 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8003f070 con 0x7efe9010a780 2024-08-31T06:18:13.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.169+0000 7efe96000640 1 -- 172.21.15.16:0/52116553 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe901af220 con 0x7efe9010a780 2024-08-31T06:18:13.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.169+0000 7efe96000640 1 -- 172.21.15.16:0/52116553 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe901af720 con 0x7efe9010a780 2024-08-31T06:18:13.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.169+0000 7efe87e00640 1 -- 172.21.15.16:0/52116553 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe80031b50 con 0x7efe9010a780 2024-08-31T06:18:13.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.169+0000 7efe87e00640 1 -- 172.21.15.16:0/52116553 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe80035030 con 0x7efe9010a780 2024-08-31T06:18:13.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.172+0000 7efe96000640 1 -- 172.21.15.16:0/52116553 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe700052c0 con 0x7efe9010a780 2024-08-31T06:18:13.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.183+0000 7efe87e00640 1 -- 172.21.15.16:0/52116553 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7efe80042070 con 0x7efe9010a780 2024-08-31T06:18:13.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.186+0000 7efe87e00640 1 --2- 172.21.15.16:0/52116553 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7efe740b4dd0 0x7efe740b72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:13.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.186+0000 7efe87e00640 1 -- 172.21.15.16:0/52116553 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8470+0+0 (secure 0 0 0) 0x7efe80105f70 con 0x7efe9010a780 2024-08-31T06:18:13.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.186+0000 7efe8ea00640 1 --2- 172.21.15.16:0/52116553 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7efe740b4dd0 0x7efe740b72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:13.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.187+0000 7efe87e00640 1 -- 172.21.15.16:0/52116553 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7efe80049030 con 0x7efe9010a780 2024-08-31T06:18:13.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.188+0000 7efe8ea00640 1 --2- 172.21.15.16:0/52116553 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7efe740b4dd0 0x7efe740b72e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7efe7c0058c0 tx=0x7efe7c005850 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:13.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:12 smithi184 ceph-mon[22331]: pgmap v705: 97 pgs: 97 active+clean; 3.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 129 KiB/s wr, 44 op/s 2024-08-31T06:18:13.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.582+0000 7efe96000640 1 -- 172.21.15.16:0/52116553 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe700055b0 con 0x7efe9010a780 2024-08-31T06:18:13.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.583+0000 7efe87e00640 1 -- 172.21.15.16:0/52116553 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11921 (secure 0 0 0) 0x7efe800ccd30 con 0x7efe9010a780 2024-08-31T06:18:13.585 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:18:13.586 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":46,"btime":"2024-08-31T06:17:38:653550+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:17:38.342854+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24712,24724,24737]},"id":1}]} 2024-08-31T06:18:13.586 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 46 2024-08-31T06:18:13.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.587+0000 7efe96000640 1 -- 172.21.15.16:0/52116553 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7efe740b4dd0 msgr2=0x7efe740b72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:13.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.587+0000 7efe96000640 1 --2- 172.21.15.16:0/52116553 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7efe740b4dd0 0x7efe740b72e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7efe7c0058c0 tx=0x7efe7c005850 comp rx=0 tx=0).stop 2024-08-31T06:18:13.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.587+0000 7efe96000640 1 -- 172.21.15.16:0/52116553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe9010a780 msgr2=0x7efe901a90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:13.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.587+0000 7efe96000640 1 --2- 172.21.15.16:0/52116553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe9010a780 0x7efe901a90e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7efe8002a9e0 tx=0x7efe80003b90 comp rx=0 tx=0).stop 2024-08-31T06:18:13.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.587+0000 7efe96000640 1 -- 172.21.15.16:0/52116553 shutdown_connections 2024-08-31T06:18:13.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.587+0000 7efe96000640 1 --2- 172.21.15.16:0/52116553 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7efe740b4dd0 0x7efe740b72e0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:13.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.587+0000 7efe96000640 1 --2- 172.21.15.16:0/52116553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe9011c9d0 0x7efe901ae920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:13.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.587+0000 7efe96000640 1 --2- 172.21.15.16:0/52116553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe90111ce0 0x7efe901a9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:13.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.587+0000 7efe96000640 1 --2- 172.21.15.16:0/52116553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe9010a780 0x7efe901a90e0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:13.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.587+0000 7efe96000640 1 -- 172.21.15.16:0/52116553 >> 172.21.15.16:0/52116553 conn(0x7efe90070c60 msgr2=0x7efe9011cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:13.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.588+0000 7efe96000640 1 -- 172.21.15.16:0/52116553 shutdown_connections 2024-08-31T06:18:13.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:13.588+0000 7efe96000640 1 -- 172.21.15.16:0/52116553 wait complete. 2024-08-31T06:18:13.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:13 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/52116553' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:18:14.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:13 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/52116553' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:18:14.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:13 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/52116553' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:18:14.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:14 smithi016 ceph-mon[17194]: pgmap v706: 97 pgs: 97 active+clean; 3.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 3.6 MiB/s wr, 367 op/s 2024-08-31T06:18:15.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:14 smithi155 ceph-mon[21742]: pgmap v706: 97 pgs: 97 active+clean; 3.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 3.6 MiB/s wr, 367 op/s 2024-08-31T06:18:15.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:14 smithi184 ceph-mon[22331]: pgmap v706: 97 pgs: 97 active+clean; 3.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 3.6 MiB/s wr, 367 op/s 2024-08-31T06:18:15.669 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24737 quiesce db / --timeout 90 --expiration 145.25 --await-for 45.0' 2024-08-31T06:18:15.670 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24737 quiesce db / --timeout 90 --expiration 145.25 --await-for 45.0 2024-08-31T06:18:16.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:15 smithi155 ceph-mon[21742]: pgmap v707: 97 pgs: 97 active+clean; 3.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 13 MiB/s wr, 1.22k op/s 2024-08-31T06:18:16.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:15 smithi016 ceph-mon[17194]: pgmap v707: 97 pgs: 97 active+clean; 3.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 13 MiB/s wr, 1.22k op/s 2024-08-31T06:18:16.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:15 smithi184 ceph-mon[22331]: pgmap v707: 97 pgs: 97 active+clean; 3.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 13 MiB/s wr, 1.22k op/s 2024-08-31T06:18:16.323 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:18:16.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 650.0 s, 1047.0 tps, lat 99.460 ms stddev 208.770 2024-08-31T06:18:17.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.242+0000 7f199a000640 1 -- 172.21.15.16:0/1168571012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f199410b2d0 msgr2=0x7f199410b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:17.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.242+0000 7f199a000640 1 --2- 172.21.15.16:0/1168571012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f199410b2d0 0x7f199410b6e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1980009900 tx=0x7f198002a400 comp rx=0 tx=0).stop 2024-08-31T06:18:17.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.242+0000 7f199a000640 1 -- 172.21.15.16:0/1168571012 shutdown_connections 2024-08-31T06:18:17.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.242+0000 7f199a000640 1 --2- 172.21.15.16:0/1168571012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f199410b2d0 0x7f199410b6e0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:17.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.242+0000 7f199a000640 1 --2- 172.21.15.16:0/1168571012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f199410bc20 0x7f1994112450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:17.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.242+0000 7f199a000640 1 --2- 172.21.15.16:0/1168571012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1994105990 0x7f1994105da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:17.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.242+0000 7f199a000640 1 -- 172.21.15.16:0/1168571012 >> 172.21.15.16:0/1168571012 conn(0x7f1994101420 msgr2=0x7f1994103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:17.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.242+0000 7f199a000640 1 -- 172.21.15.16:0/1168571012 shutdown_connections 2024-08-31T06:18:17.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.243+0000 7f199a000640 1 -- 172.21.15.16:0/1168571012 wait complete. 2024-08-31T06:18:17.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.244+0000 7f199a000640 1 Processor -- start 2024-08-31T06:18:17.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.244+0000 7f199a000640 1 -- start start 2024-08-31T06:18:17.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.245+0000 7f199a000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1994105990 0x7f199419c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:17.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.245+0000 7f199a000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f199410b2d0 0x7f199419c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:17.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.245+0000 7f199a000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f199410bc20 0x7f19941a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:17.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.245+0000 7f199a000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1994115930 con 0x7f199410b2d0 2024-08-31T06:18:17.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.245+0000 7f199a000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f19941157b0 con 0x7f1994105990 2024-08-31T06:18:17.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.245+0000 7f199a000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1994115ab0 con 0x7f199410bc20 2024-08-31T06:18:17.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.245+0000 7f1998c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1994105990 0x7f199419c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:17.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.245+0000 7f1998c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1994105990 0x7f199419c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:33022/0 (socket says 172.21.15.16:33022) 2024-08-31T06:18:17.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.245+0000 7f1998c00640 1 -- 172.21.15.16:0/3738336926 learned_addr learned my addr 172.21.15.16:0/3738336926 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:17.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.245+0000 7f1993e00640 1 --2- 172.21.15.16:0/3738336926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f199410b2d0 0x7f199419c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:17.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.245+0000 7f1999600640 1 --2- 172.21.15.16:0/3738336926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f199410bc20 0x7f19941a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:17.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.246+0000 7f1998c00640 1 -- 172.21.15.16:0/3738336926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f199410bc20 msgr2=0x7f19941a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:17.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.246+0000 7f1998c00640 1 --2- 172.21.15.16:0/3738336926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f199410bc20 0x7f19941a1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:17.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.246+0000 7f1998c00640 1 -- 172.21.15.16:0/3738336926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f199410b2d0 msgr2=0x7f199419c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:17.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.246+0000 7f1998c00640 1 --2- 172.21.15.16:0/3738336926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f199410b2d0 0x7f199419c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:17.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.246+0000 7f1998c00640 1 -- 172.21.15.16:0/3738336926 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19941a2250 con 0x7f1994105990 2024-08-31T06:18:17.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.246+0000 7f1998c00640 1 --2- 172.21.15.16:0/3738336926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1994105990 0x7f199419c2e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f198800ea60 tx=0x7f198800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:17.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.246+0000 7f1991600640 1 -- 172.21.15.16:0/3738336926 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198800cde0 con 0x7f1994105990 2024-08-31T06:18:17.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.246+0000 7f1991600640 1 -- 172.21.15.16:0/3738336926 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19880035f0 con 0x7f1994105990 2024-08-31T06:18:17.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.246+0000 7f1991600640 1 -- 172.21.15.16:0/3738336926 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1988010690 con 0x7f1994105990 2024-08-31T06:18:17.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.246+0000 7f199a000640 1 -- 172.21.15.16:0/3738336926 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19941a2540 con 0x7f1994105990 2024-08-31T06:18:17.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.247+0000 7f199a000640 1 -- 172.21.15.16:0/3738336926 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19941a8ca0 con 0x7f1994105990 2024-08-31T06:18:17.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.250+0000 7f19a2259740 1 Processor -- start 2024-08-31T06:18:17.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.250+0000 7f19a2259740 1 -- start start 2024-08-31T06:18:17.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.251+0000 7f19a2259740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bacab66850 0x55bacaf0cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:17.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.251+0000 7f1991600640 1 -- 172.21.15.16:0/3738336926 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f19880037a0 con 0x7f1994105990 2024-08-31T06:18:17.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.252+0000 7f1999600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bacab66850 0x55bacaf0cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:17.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.252+0000 7f1999600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bacab66850 0x55bacaf0cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:33038/0 (socket says 172.21.15.16:33038) 2024-08-31T06:18:17.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.252+0000 7f1999600640 1 -- 172.21.15.16:0/2666736338 learned_addr learned my addr 172.21.15.16:0/2666736338 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:17.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.252+0000 7f19a2259740 1 --2- 172.21.15.16:0/2666736338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bacab3d990 0x55bacaf0d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:17.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.252+0000 7f19a2259740 1 --2- 172.21.15.16:0/2666736338 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bacab45b70 0x55bacaf0d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:17.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.252+0000 7f19a2259740 1 -- 172.21.15.16:0/2666736338 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bacae82450 con 0x55bacab3d990 2024-08-31T06:18:17.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.252+0000 7f19a2259740 1 -- 172.21.15.16:0/2666736338 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55bacae719e0 con 0x55bacab66850 2024-08-31T06:18:17.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.252+0000 7f19a2259740 1 -- 172.21.15.16:0/2666736338 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55bacae75990 con 0x55bacab45b70 2024-08-31T06:18:17.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.253+0000 7f1999600640 1 -- 172.21.15.16:0/2666736338 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bacab45b70 msgr2=0x55bacaf0d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:17.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.253+0000 7f1998c00640 1 --2- 172.21.15.16:0/2666736338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bacab3d990 0x55bacaf0d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:17.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.253+0000 7f1993e00640 1 --2- 172.21.15.16:0/2666736338 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bacab45b70 0x55bacaf0d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:17.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.253+0000 7f1999600640 1 --2- 172.21.15.16:0/2666736338 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bacab45b70 0x55bacaf0d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:17.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.253+0000 7f1999600640 1 -- 172.21.15.16:0/2666736338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bacab3d990 msgr2=0x55bacaf0d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:17.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.253+0000 7f1999600640 1 --2- 172.21.15.16:0/2666736338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bacab3d990 0x55bacaf0d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:17.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.253+0000 7f1999600640 1 -- 172.21.15.16:0/2666736338 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bacae04fa0 con 0x55bacab66850 2024-08-31T06:18:17.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.253+0000 7f1993e00640 1 --2- 172.21.15.16:0/2666736338 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bacab45b70 0x55bacaf0d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:18:17.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.253+0000 7f1998c00640 1 --2- 172.21.15.16:0/2666736338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bacab3d990 0x55bacaf0d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:18:17.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.254+0000 7f1999600640 1 --2- 172.21.15.16:0/2666736338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bacab66850 0x55bacaf0cdd0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f198c009ee0 tx=0x7f198c010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef4a335aa3bdef3f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:17.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.254+0000 7f1985c00640 1 -- 172.21.15.16:0/2666736338 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198c016070 con 0x55bacab66850 2024-08-31T06:18:17.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.254+0000 7f1985c00640 1 -- 172.21.15.16:0/2666736338 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f198c004d20 con 0x55bacab66850 2024-08-31T06:18:17.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.255+0000 7f1985c00640 1 -- 172.21.15.16:0/2666736338 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198c016070 con 0x55bacab66850 2024-08-31T06:18:17.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.255+0000 7f1991600640 1 --2- 172.21.15.16:0/3738336926 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f197c0b4d10 0x7f197c0b7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:17.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.255+0000 7f19a2259740 1 -- 172.21.15.16:0/2666736338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bacab66850 msgr2=0x55bacaf0cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:17.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.256+0000 7f19a2259740 1 --2- 172.21.15.16:0/2666736338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bacab66850 0x55bacaf0cdd0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f198c009ee0 tx=0x7f198c010740 comp rx=0 tx=0).stop 2024-08-31T06:18:17.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.256+0000 7f19a2259740 1 -- 172.21.15.16:0/2666736338 shutdown_connections 2024-08-31T06:18:17.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.256+0000 7f19a2259740 1 --2- 172.21.15.16:0/2666736338 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bacab45b70 0x55bacaf0d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:17.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.256+0000 7f19a2259740 1 --2- 172.21.15.16:0/2666736338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bacab3d990 0x55bacaf0d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:17.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.256+0000 7f19a2259740 1 --2- 172.21.15.16:0/2666736338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bacab66850 0x55bacaf0cdd0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:17.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.256+0000 7f19a2259740 1 -- 172.21.15.16:0/2666736338 >> 172.21.15.16:0/2666736338 conn(0x55bacab5ac40 msgr2=0x55bacaf0acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:17.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.256+0000 7f1991600640 1 -- 172.21.15.16:0/3738336926 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8470+0+0 (secure 0 0 0) 0x7f19880e57e0 con 0x7f1994105990 2024-08-31T06:18:17.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.256+0000 7f19a2259740 1 -- 172.21.15.16:0/2666736338 shutdown_connections 2024-08-31T06:18:17.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.256+0000 7f1998c00640 1 --2- 172.21.15.16:0/3738336926 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f197c0b4d10 0x7f197c0b7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:17.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.256+0000 7f19a2259740 1 -- 172.21.15.16:0/2666736338 wait complete. 2024-08-31T06:18:17.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.257+0000 7f1998c00640 1 --2- 172.21.15.16:0/3738336926 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f197c0b4d10 0x7f197c0b7220 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f19880e3010 tx=0x7f198800ef90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:17.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.257+0000 7f19a2259740 1 -- start start 2024-08-31T06:18:17.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.257+0000 7f19a2259740 1 Processor -- start 2024-08-31T06:18:17.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.258+0000 7f19a2259740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bacab3d990 0x55bacaf20c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:17.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.258+0000 7f19a2259740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bacab45b70 0x55bacaf211d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:17.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.258+0000 7f19a2259740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bacab66850 0x55bacaf2d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:17.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.258+0000 7f19a2259740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bacacd47b0 con 0x55bacab45b70 2024-08-31T06:18:17.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.258+0000 7f19a2259740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55bacae0ceb0 con 0x55bacab66850 2024-08-31T06:18:17.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.258+0000 7f19a2259740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55bacaf23d30 con 0x55bacab3d990 2024-08-31T06:18:17.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.258+0000 7f19a2259740 10 client.0 _ll_register_callbacks cb 0x55bacae6f0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:18:17.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.258+0000 7f1993e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bacab45b70 0x55bacaf211d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:17.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.258+0000 7f1993e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bacab45b70 0x55bacaf211d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57092/0 (socket says 172.21.15.16:57092) 2024-08-31T06:18:17.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.258+0000 7f1993e00640 1 -- 172.21.15.16:0/1705118930 learned_addr learned my addr 172.21.15.16:0/1705118930 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:17.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.258+0000 7f1999600640 1 --2- 172.21.15.16:0/1705118930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bacab3d990 0x55bacaf20c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:17.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.258+0000 7f1999600640 1 --2- 172.21.15.16:0/1705118930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bacab66850 0x55bacaf2d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:17.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.259+0000 7f1993e00640 1 -- 172.21.15.16:0/1705118930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bacab3d990 msgr2=0x55bacaf20c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:17.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.259+0000 7f1993e00640 1 --2- 172.21.15.16:0/1705118930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bacab3d990 0x55bacaf20c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:17.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.259+0000 7f1993e00640 1 -- 172.21.15.16:0/1705118930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bacab66850 msgr2=0x55bacaf2d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:17.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.259+0000 7f1993e00640 1 --2- 172.21.15.16:0/1705118930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bacab66850 0x55bacaf2d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:17.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.259+0000 7f1993e00640 1 -- 172.21.15.16:0/1705118930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bacae17f40 con 0x55bacab45b70 2024-08-31T06:18:17.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.259+0000 7f1999600640 1 --2- 172.21.15.16:0/1705118930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bacab3d990 0x55bacaf20c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:18:17.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.259+0000 7f1993e00640 1 --2- 172.21.15.16:0/1705118930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bacab45b70 0x55bacaf211d0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f1980000c00 tx=0x7f1980004460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ed85591de5335f7 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:17.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.259+0000 7f197a000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:18:17.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.260+0000 7f197a000640 1 -- 172.21.15.16:0/1705118930 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1980035040 con 0x55bacab45b70 2024-08-31T06:18:17.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.260+0000 7f19a2259740 1 -- 172.21.15.16:0/1705118930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f198c004d20 con 0x55bacab45b70 2024-08-31T06:18:17.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.260+0000 7f197a000640 1 -- 172.21.15.16:0/1705118930 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8470+0+0 (secure 0 0 0) 0x7f1980005450 con 0x55bacab45b70 2024-08-31T06:18:17.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.260+0000 7f197a000640 1 -- 172.21.15.16:0/1705118930 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f1980005650 con 0x55bacab45b70 2024-08-31T06:18:17.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.260+0000 7f19a2259740 10 client.17709 fetch_fsmap learned FSMap version 46 2024-08-31T06:18:17.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.260+0000 7f19a2259740 1 -- 172.21.15.16:0/1705118930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55bacabe0ba0 con 0x55bacab45b70 2024-08-31T06:18:17.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.260+0000 7f197a000640 1 -- 172.21.15.16:0/1705118930 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f198c004d20 con 0x55bacab45b70 2024-08-31T06:18:17.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.260+0000 7f197a000640 1 -- 172.21.15.16:0/1705118930 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7f1980003ab0 con 0x55bacab45b70 2024-08-31T06:18:17.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.261+0000 7f19a2259740 10 client.17709 fetch_fsmap finished waiting for FSMap version 46 2024-08-31T06:18:17.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.261+0000 7f19a2259740 10 client.17709 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:18:17.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.261+0000 7f19a2259740 20 client.17709 populate_metadata read hostname 'smithi016' 2024-08-31T06:18:17.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.261+0000 7f19a2259740 1 --2- 172.21.15.16:0/1705118930 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55bacab251d0 0x55bacaf32db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:17.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.261+0000 7f19a2259740 4 client.17709 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:18:17.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.261+0000 7f19a2259740 1 -- 172.21.15.16:0/1705118930 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bacae843a0 con 0x55bacab251d0 2024-08-31T06:18:17.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.261+0000 7f1999600640 1 --2- 172.21.15.16:0/1705118930 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55bacab251d0 0x55bacaf32db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:17.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.261+0000 7f1999600640 1 --2- 172.21.15.16:0/1705118930 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55bacab251d0 0x55bacaf32db0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6868bd87c76ee16 server_cookie=67764f3d72c112a6 in_seq=0 out_seq=0 2024-08-31T06:18:17.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.261+0000 7f197a000640 10 client.17709 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:18:17.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.262+0000 7f197a000640 1 -- 172.21.15.16:0/1705118930 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bacae843a0 con 0x55bacab251d0 2024-08-31T06:18:17.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.262+0000 7f197a000640 10 client.17709 handle_command_reply: tid=0 2024-08-31T06:18:17.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.263+0000 7f19a2259740 1 client.17709 shutdown 2024-08-31T06:18:17.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.263+0000 7f19a2259740 1 -- 172.21.15.16:0/1705118930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bacab45b70 msgr2=0x55bacaf211d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:17.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.263+0000 7f19a2259740 1 --2- 172.21.15.16:0/1705118930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bacab45b70 0x55bacaf211d0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f1980000c00 tx=0x7f1980004460 comp rx=0 tx=0).stop 2024-08-31T06:18:17.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.263+0000 7f19a2259740 1 -- 172.21.15.16:0/1705118930 shutdown_connections 2024-08-31T06:18:17.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.263+0000 7f19a2259740 1 --2- 172.21.15.16:0/1705118930 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55bacab251d0 0x55bacaf32db0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:17.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.263+0000 7f19a2259740 1 --2- 172.21.15.16:0/1705118930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bacab66850 0x55bacaf2d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:17.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.263+0000 7f19a2259740 1 --2- 172.21.15.16:0/1705118930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bacab45b70 0x55bacaf211d0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:17.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.263+0000 7f19a2259740 1 --2- 172.21.15.16:0/1705118930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bacab3d990 0x55bacaf20c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:17.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.263+0000 7f19a2259740 1 -- 172.21.15.16:0/1705118930 >> 172.21.15.16:0/1705118930 conn(0x55bacab5ac40 msgr2=0x55bacaf092a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:17.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.264+0000 7f197a000640 0 client.17709 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:18:17.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.264+0000 7f19a2259740 1 -- 172.21.15.16:0/1705118930 shutdown_connections 2024-08-31T06:18:17.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.264+0000 7f19a2259740 1 -- 172.21.15.16:0/1705118930 wait complete. 2024-08-31T06:18:17.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.264+0000 7f19a2259740 20 client.17709 trim_cache size 0 max 16384 2024-08-31T06:18:17.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.288+0000 7f19a2259740 1 Processor -- start 2024-08-31T06:18:17.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.288+0000 7f19a2259740 1 -- start start 2024-08-31T06:18:17.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.289+0000 7f19a2259740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bacab3d990 0x55bacaf228c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:17.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.289+0000 7f19a2259740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bacab45b70 0x55bacaf2cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:17.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.289+0000 7f19a2259740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bacab66850 0x55bacaf1de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:17.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.289+0000 7f19a2259740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f19941afa80 con 0x55bacab3d990 2024-08-31T06:18:17.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.289+0000 7f19a2259740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f19941ad2c0 con 0x55bacab45b70 2024-08-31T06:18:17.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.289+0000 7f19a2259740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f19941ab150 con 0x55bacab66850 2024-08-31T06:18:17.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.289+0000 7f1999600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bacab66850 0x55bacaf1de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:17.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.290+0000 7f1999600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bacab66850 0x55bacaf1de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:36390/0 (socket says 172.21.15.16:36390) 2024-08-31T06:18:17.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.290+0000 7f1999600640 1 -- 172.21.15.16:0/2612809736 learned_addr learned my addr 172.21.15.16:0/2612809736 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:17.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.290+0000 7f1993e00640 1 --2- 172.21.15.16:0/2612809736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bacab45b70 0x55bacaf2cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:17.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.290+0000 7f1999600640 1 --2- 172.21.15.16:0/2612809736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bacab3d990 0x55bacaf228c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:17.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.290+0000 7f1993e00640 1 -- 172.21.15.16:0/2612809736 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bacab66850 msgr2=0x55bacaf1de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:17.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.290+0000 7f1993e00640 1 --2- 172.21.15.16:0/2612809736 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bacab66850 0x55bacaf1de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:17.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.290+0000 7f1993e00640 1 -- 172.21.15.16:0/2612809736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bacab3d990 msgr2=0x55bacaf228c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:17.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.290+0000 7f1993e00640 1 --2- 172.21.15.16:0/2612809736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bacab3d990 0x55bacaf228c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:17.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.290+0000 7f1993e00640 1 -- 172.21.15.16:0/2612809736 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bacaf21f50 con 0x55bacab45b70 2024-08-31T06:18:17.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.290+0000 7f1993e00640 1 --2- 172.21.15.16:0/2612809736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bacab45b70 0x55bacaf2cfb0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1980005ab0 tx=0x7f1980001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b516b919794c54d0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:17.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.291+0000 7f1985200640 1 -- 172.21.15.16:0/2612809736 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1980051040 con 0x55bacab45b70 2024-08-31T06:18:17.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.291+0000 7f1985200640 1 -- 172.21.15.16:0/2612809736 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1980031e40 con 0x55bacab45b70 2024-08-31T06:18:17.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.291+0000 7f1985200640 1 -- 172.21.15.16:0/2612809736 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1980035040 con 0x55bacab45b70 2024-08-31T06:18:17.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.292+0000 7f19a2259740 1 -- 172.21.15.16:0/2612809736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bacab45b70 msgr2=0x55bacaf2cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:17.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.292+0000 7f19a2259740 1 --2- 172.21.15.16:0/2612809736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bacab45b70 0x55bacaf2cfb0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1980005ab0 tx=0x7f1980001e70 comp rx=0 tx=0).stop 2024-08-31T06:18:17.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.292+0000 7f19a2259740 1 -- 172.21.15.16:0/2612809736 shutdown_connections 2024-08-31T06:18:17.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.292+0000 7f19a2259740 1 --2- 172.21.15.16:0/2612809736 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bacab66850 0x55bacaf1de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:17.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.292+0000 7f19a2259740 1 --2- 172.21.15.16:0/2612809736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bacab45b70 0x55bacaf2cfb0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:17.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.292+0000 7f19a2259740 1 --2- 172.21.15.16:0/2612809736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bacab3d990 0x55bacaf228c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:17.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.292+0000 7f19a2259740 1 -- 172.21.15.16:0/2612809736 >> 172.21.15.16:0/2612809736 conn(0x55bacab5ac40 msgr2=0x55bacaf221b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:17.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.292+0000 7f19a2259740 1 -- 172.21.15.16:0/2612809736 shutdown_connections 2024-08-31T06:18:17.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.292+0000 7f19a2259740 1 -- 172.21.15.16:0/2612809736 wait complete. 2024-08-31T06:18:17.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.293+0000 7f19a2259740 1 -- start start 2024-08-31T06:18:17.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.293+0000 7f19a2259740 1 Processor -- start 2024-08-31T06:18:17.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.294+0000 7f19a2259740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bacab3d990 0x55bacaf25920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:17.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.294+0000 7f19a2259740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bacab45b70 0x55bacaf25ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:17.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.294+0000 7f19a2259740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bacab66850 0x55bacaf967b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:17.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.294+0000 7f19a2259740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f19941ab7a0 con 0x55bacab66850 2024-08-31T06:18:17.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.294+0000 7f19a2259740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f19941acbf0 con 0x55bacab3d990 2024-08-31T06:18:17.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.294+0000 7f19a2259740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f19941abe50 con 0x55bacab45b70 2024-08-31T06:18:17.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.294+0000 7f19a2259740 10 client.0 _ll_register_callbacks cb 0x7f1980009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:18:17.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.294+0000 7f1999600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bacab3d990 0x55bacaf25920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:17.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.294+0000 7f1993e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bacab45b70 0x55bacaf25ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:17.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.294+0000 7f1999600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bacab3d990 0x55bacaf25920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:33070/0 (socket says 172.21.15.16:33070) 2024-08-31T06:18:17.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.294+0000 7f1999600640 1 -- 172.21.15.16:0/3969723355 learned_addr learned my addr 172.21.15.16:0/3969723355 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:17.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.295+0000 7f1999600640 1 -- 172.21.15.16:0/3969723355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bacab45b70 msgr2=0x55bacaf25ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:17.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.295+0000 7f1999600640 1 --2- 172.21.15.16:0/3969723355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bacab45b70 0x55bacaf25ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:17.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.295+0000 7f1999600640 1 -- 172.21.15.16:0/3969723355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bacab66850 msgr2=0x55bacaf967b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:17.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.295+0000 7f1999600640 1 --2- 172.21.15.16:0/3969723355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bacab66850 0x55bacaf967b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:17.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.295+0000 7f1999600640 1 -- 172.21.15.16:0/3969723355 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bacaf96e20 con 0x55bacab3d990 2024-08-31T06:18:17.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.295+0000 7f1999600640 1 --2- 172.21.15.16:0/3969723355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bacab3d990 0x55bacaf25920 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f198c005a70 tx=0x7f198c005120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d534ee173726447b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:17.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.295+0000 7f197a000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:18:17.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.296+0000 7f197a000640 1 -- 172.21.15.16:0/3969723355 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198c036020 con 0x55bacab3d990 2024-08-31T06:18:17.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.296+0000 7f197a000640 1 -- 172.21.15.16:0/3969723355 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8470+0+0 (secure 0 0 0) 0x7f198c005380 con 0x55bacab3d990 2024-08-31T06:18:17.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.296+0000 7f19a2259740 1 -- 172.21.15.16:0/3969723355 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bacae7fe40 con 0x55bacab3d990 2024-08-31T06:18:17.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.296+0000 7f197a000640 1 -- 172.21.15.16:0/3969723355 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f198c016070 con 0x55bacab3d990 2024-08-31T06:18:17.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.296+0000 7f19a2259740 10 client.28073 fetch_fsmap learned FSMap version 46 2024-08-31T06:18:17.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.296+0000 7f19a2259740 1 -- 172.21.15.16:0/3969723355 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55bacaf9e100 con 0x55bacab3d990 2024-08-31T06:18:17.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.297+0000 7f197a000640 1 -- 172.21.15.16:0/3969723355 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55bacae7fe40 con 0x55bacab3d990 2024-08-31T06:18:17.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.297+0000 7f197a000640 1 -- 172.21.15.16:0/3969723355 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7f198c00f040 con 0x55bacab3d990 2024-08-31T06:18:17.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.297+0000 7f19a2259740 10 client.28073 fetch_fsmap finished waiting for FSMap version 46 2024-08-31T06:18:17.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.297+0000 7f19a2259740 10 client.28073 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:18:17.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.297+0000 7f19a2259740 20 client.28073 populate_metadata read hostname 'smithi016' 2024-08-31T06:18:17.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.297+0000 7f19a2259740 1 --2- 172.21.15.16:0/3969723355 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55bacaafa580 0x55bacafa0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:17.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.297+0000 7f19a2259740 4 client.28073 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.25, "await_for": 45.0}] 2024-08-31T06:18:17.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.297+0000 7f19a2259740 1 -- 172.21.15.16:0/3969723355 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.25, "await_for": 45.0}) -- 0x55bacae7d580 con 0x55bacaafa580 2024-08-31T06:18:17.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.297+0000 7f1993e00640 1 --2- 172.21.15.16:0/3969723355 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55bacaafa580 0x55bacafa0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:17.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.298+0000 7f1993e00640 1 --2- 172.21.15.16:0/3969723355 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55bacaafa580 0x55bacafa0810 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bbb691ecaf85298 server_cookie=25755f8b63d268e6 in_seq=0 out_seq=0 2024-08-31T06:18:17.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.298+0000 7f197a000640 10 client.28073 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:18:17.446 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T06:18:17.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.549+0000 7f197a000640 1 -- 172.21.15.16:0/3969723355 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55bacae7d580 con 0x55bacaafa580 2024-08-31T06:18:17.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.549+0000 7f197a000640 10 client.28073 handle_command_reply: tid=0 2024-08-31T06:18:17.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.549+0000 7f19a2259740 1 client.28073 shutdown 2024-08-31T06:18:17.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.549+0000 7f19a2259740 1 -- 172.21.15.16:0/3969723355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bacab3d990 msgr2=0x55bacaf25920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:17.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.549+0000 7f19a2259740 1 --2- 172.21.15.16:0/3969723355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bacab3d990 0x55bacaf25920 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f198c005a70 tx=0x7f198c005120 comp rx=0 tx=0).stop 2024-08-31T06:18:17.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.549+0000 7f19a2259740 1 -- 172.21.15.16:0/3969723355 shutdown_connections 2024-08-31T06:18:17.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.549+0000 7f19a2259740 1 --2- 172.21.15.16:0/3969723355 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55bacaafa580 0x55bacafa0810 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:17.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.549+0000 7f19a2259740 1 --2- 172.21.15.16:0/3969723355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bacab45b70 0x55bacaf25ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:17.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.549+0000 7f19a2259740 1 --2- 172.21.15.16:0/3969723355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bacab66850 0x55bacaf967b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:17.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.549+0000 7f19a2259740 1 --2- 172.21.15.16:0/3969723355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bacab3d990 0x55bacaf25920 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:17.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.549+0000 7f197a000640 0 client.28073 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:18:17.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.549+0000 7f19a2259740 1 -- 172.21.15.16:0/3969723355 >> 172.21.15.16:0/3969723355 conn(0x55bacab5ac40 msgr2=0x55bacaf22120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:17.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.549+0000 7f19a2259740 1 -- 172.21.15.16:0/3969723355 shutdown_connections 2024-08-31T06:18:17.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.549+0000 7f19a2259740 1 -- 172.21.15.16:0/3969723355 wait complete. 2024-08-31T06:18:17.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.550+0000 7f19a2259740 20 client.28073 trim_cache size 0 max 16384 2024-08-31T06:18:17.554 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:18:17.554 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 46, 2024-08-31T06:18:17.554 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24737, 2024-08-31T06:18:17.554 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 38, 2024-08-31T06:18:17.554 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-08-31T06:18:17.554 INFO:teuthology.orchestra.run.smithi016.stdout: "f2a792d6": { 2024-08-31T06:18:17.554 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 38, 2024-08-31T06:18:17.554 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-08-31T06:18:17.554 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:18:17.555 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-08-31T06:18:17.555 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:18:17.555 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:18:17.555 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-08-31T06:18:17.555 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 145.2, 2024-08-31T06:18:17.555 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-08-31T06:18:17.555 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-08-31T06:18:17.555 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-08-31T06:18:17.555 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:18:17.555 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-08-31T06:18:17.555 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:18:17.555 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:18:17.555 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:18:17.556 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:18:17.556 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:18:17.556 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:18:17.556 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:18:17.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.550+0000 7f199a000640 1 -- 172.21.15.16:0/3738336926 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f197c0b4d10 msgr2=0x7f197c0b7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:17.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.550+0000 7f199a000640 1 --2- 172.21.15.16:0/3738336926 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f197c0b4d10 0x7f197c0b7220 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f19880e3010 tx=0x7f198800ef90 comp rx=0 tx=0).stop 2024-08-31T06:18:17.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.550+0000 7f199a000640 1 -- 172.21.15.16:0/3738336926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1994105990 msgr2=0x7f199419c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:17.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.550+0000 7f199a000640 1 --2- 172.21.15.16:0/3738336926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1994105990 0x7f199419c2e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f198800ea60 tx=0x7f198800ef20 comp rx=0 tx=0).stop 2024-08-31T06:18:17.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.550+0000 7f199a000640 1 -- 172.21.15.16:0/3738336926 shutdown_connections 2024-08-31T06:18:17.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.550+0000 7f199a000640 1 --2- 172.21.15.16:0/3738336926 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f197c0b4d10 0x7f197c0b7220 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:17.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.550+0000 7f199a000640 1 --2- 172.21.15.16:0/3738336926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f199410bc20 0x7f19941a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:17.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.550+0000 7f199a000640 1 --2- 172.21.15.16:0/3738336926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f199410b2d0 0x7f199419c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:17.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.550+0000 7f199a000640 1 --2- 172.21.15.16:0/3738336926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1994105990 0x7f199419c2e0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:17.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.550+0000 7f199a000640 1 -- 172.21.15.16:0/3738336926 >> 172.21.15.16:0/3738336926 conn(0x7f1994101420 msgr2=0x7f1994103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:17.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.551+0000 7f199a000640 1 -- 172.21.15.16:0/3738336926 shutdown_connections 2024-08-31T06:18:17.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:17.551+0000 7f199a000640 1 -- 172.21.15.16:0/3738336926 wait complete. 2024-08-31T06:18:18.422 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f2a792d6', quiesce took 6.8 seconds. Will release after: 18.45 2024-08-31T06:18:18.448 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:18:18.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:18 smithi016 ceph-mon[17194]: pgmap v708: 97 pgs: 97 active+clean; 3.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 14 MiB/s wr, 1.32k op/s 2024-08-31T06:18:18.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:18 smithi184 ceph-mon[22331]: pgmap v708: 97 pgs: 97 active+clean; 3.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 14 MiB/s wr, 1.32k op/s 2024-08-31T06:18:19.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:18 smithi155 ceph-mon[21742]: pgmap v708: 97 pgs: 97 active+clean; 3.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 14 MiB/s wr, 1.32k op/s 2024-08-31T06:18:19.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.305+0000 7f1239400640 1 -- 172.21.15.16:0/1267177469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f123410bbe0 msgr2=0x7f1234112410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:19.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.305+0000 7f1239400640 1 --2- 172.21.15.16:0/1267177469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f123410bbe0 0x7f1234112410 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f122c009ca0 tx=0x7f122c02a8c0 comp rx=0 tx=0).stop 2024-08-31T06:18:19.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.305+0000 7f1239400640 1 -- 172.21.15.16:0/1267177469 shutdown_connections 2024-08-31T06:18:19.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.305+0000 7f1239400640 1 --2- 172.21.15.16:0/1267177469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f123410bbe0 0x7f1234112410 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:19.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.305+0000 7f1239400640 1 --2- 172.21.15.16:0/1267177469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f123410b290 0x7f123410b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:19.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.305+0000 7f1239400640 1 --2- 172.21.15.16:0/1267177469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1234105950 0x7f1234105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:19.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.305+0000 7f1239400640 1 -- 172.21.15.16:0/1267177469 >> 172.21.15.16:0/1267177469 conn(0x7f1234101420 msgr2=0x7f1234103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:19.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.305+0000 7f1239400640 1 -- 172.21.15.16:0/1267177469 shutdown_connections 2024-08-31T06:18:19.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.305+0000 7f1239400640 1 -- 172.21.15.16:0/1267177469 wait complete. 2024-08-31T06:18:19.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.306+0000 7f1239400640 1 Processor -- start 2024-08-31T06:18:19.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.306+0000 7f1239400640 1 -- start start 2024-08-31T06:18:19.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.307+0000 7f1239400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1234105950 0x7f123419c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:19.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.307+0000 7f1239400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f123410b290 0x7f123419c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:19.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.307+0000 7f1239400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f123410bbe0 0x7f12341a1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:19.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.307+0000 7f1239400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f12341158a0 con 0x7f123410b290 2024-08-31T06:18:19.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.307+0000 7f1239400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1234115720 con 0x7f123410bbe0 2024-08-31T06:18:19.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.307+0000 7f1239400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1234115a20 con 0x7f1234105950 2024-08-31T06:18:19.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.307+0000 7f1233400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f123410b290 0x7f123419c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:19.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.307+0000 7f1233400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f123410b290 0x7f123419c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57136/0 (socket says 172.21.15.16:57136) 2024-08-31T06:18:19.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.307+0000 7f1233400640 1 -- 172.21.15.16:0/351236454 learned_addr learned my addr 172.21.15.16:0/351236454 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:19.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.307+0000 7f1233e00640 1 --2- 172.21.15.16:0/351236454 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1234105950 0x7f123419c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:19.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.307+0000 7f1238a00640 1 --2- 172.21.15.16:0/351236454 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f123410bbe0 0x7f12341a1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:19.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.307+0000 7f1233400640 1 -- 172.21.15.16:0/351236454 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1234105950 msgr2=0x7f123419c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:19.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.307+0000 7f1233400640 1 --2- 172.21.15.16:0/351236454 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1234105950 0x7f123419c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:19.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.307+0000 7f1233400640 1 -- 172.21.15.16:0/351236454 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f123410bbe0 msgr2=0x7f12341a1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:19.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.307+0000 7f1233400640 1 --2- 172.21.15.16:0/351236454 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f123410bbe0 0x7f12341a1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:19.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.308+0000 7f1233400640 1 -- 172.21.15.16:0/351236454 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12341a22c0 con 0x7f123410b290 2024-08-31T06:18:19.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.308+0000 7f1233400640 1 --2- 172.21.15.16:0/351236454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f123410b290 0x7f123419c8f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f122000e9f0 tx=0x7f122000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:19.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.308+0000 7f1230c00640 1 -- 172.21.15.16:0/351236454 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122000ccc0 con 0x7f123410b290 2024-08-31T06:18:19.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.308+0000 7f1230c00640 1 -- 172.21.15.16:0/351236454 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12200108e0 con 0x7f123410b290 2024-08-31T06:18:19.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.308+0000 7f1239400640 1 -- 172.21.15.16:0/351236454 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12341a25b0 con 0x7f123410b290 2024-08-31T06:18:19.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.308+0000 7f1230c00640 1 -- 172.21.15.16:0/351236454 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1220018860 con 0x7f123410b290 2024-08-31T06:18:19.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.308+0000 7f1239400640 1 -- 172.21.15.16:0/351236454 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12341a8d10 con 0x7f123410b290 2024-08-31T06:18:19.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.312+0000 7f124165f740 1 Processor -- start 2024-08-31T06:18:19.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.313+0000 7f124165f740 1 -- start start 2024-08-31T06:18:19.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.313+0000 7f124165f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf579d4070 0x55cf57d79b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:19.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.313+0000 7f124165f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf579ab1b0 0x55cf57d7a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:19.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.313+0000 7f124165f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf579b3390 0x55cf57d7a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:19.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.313+0000 7f124165f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cf57cf0770 con 0x55cf579d4070 2024-08-31T06:18:19.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.313+0000 7f124165f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55cf57cdfd00 con 0x55cf579b3390 2024-08-31T06:18:19.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.313+0000 7f124165f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55cf57ce3cb0 con 0x55cf579ab1b0 2024-08-31T06:18:19.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.313+0000 7f1238a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf579d4070 0x55cf57d79b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:19.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.314+0000 7f1233e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf579ab1b0 0x55cf57d7a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:19.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.314+0000 7f1238a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf579d4070 0x55cf57d79b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57146/0 (socket says 172.21.15.16:57146) 2024-08-31T06:18:19.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.314+0000 7f1233e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf579ab1b0 0x55cf57d7a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:36428/0 (socket says 172.21.15.16:36428) 2024-08-31T06:18:19.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.314+0000 7f1238a00640 1 -- 172.21.15.16:0/3583231532 learned_addr learned my addr 172.21.15.16:0/3583231532 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:19.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.314+0000 7f1230c00640 1 -- 172.21.15.16:0/351236454 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f1220010430 con 0x7f123410b290 2024-08-31T06:18:19.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.314+0000 7f1238a00640 1 -- 172.21.15.16:0/3583231532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf579ab1b0 msgr2=0x55cf57d7a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:19.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.314+0000 7f1238a00640 1 --2- 172.21.15.16:0/3583231532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf579ab1b0 0x55cf57d7a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:19.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.314+0000 7f1238a00640 1 -- 172.21.15.16:0/3583231532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf579b3390 msgr2=0x55cf57d7a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:19.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.314+0000 7f1238a00640 1 --2- 172.21.15.16:0/3583231532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf579b3390 0x55cf57d7a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:19.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.314+0000 7f1238a00640 1 -- 172.21.15.16:0/3583231532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf57a3f650 con 0x55cf579d4070 2024-08-31T06:18:19.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.314+0000 7f1238a00640 1 --2- 172.21.15.16:0/3583231532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf579d4070 0x55cf57d79b20 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f122c02f410 tx=0x7f122c02f440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbab312579a5c6be server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:19.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.315+0000 7f1229200640 1 -- 172.21.15.16:0/3583231532 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122c031cf0 con 0x55cf579d4070 2024-08-31T06:18:19.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.315+0000 7f1229200640 1 -- 172.21.15.16:0/3583231532 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f122c036030 con 0x55cf579d4070 2024-08-31T06:18:19.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.315+0000 7f1229200640 1 -- 172.21.15.16:0/3583231532 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122c0042a0 con 0x55cf579d4070 2024-08-31T06:18:19.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.315+0000 7f1230c00640 1 --2- 172.21.15.16:0/351236454 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f12180b4ca0 0x7f12180b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:19.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.316+0000 7f1230c00640 1 -- 172.21.15.16:0/351236454 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8470+0+0 (secure 0 0 0) 0x7f12200e5780 con 0x7f123410b290 2024-08-31T06:18:19.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.316+0000 7f1233e00640 1 --2- 172.21.15.16:0/351236454 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f12180b4ca0 0x7f12180b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:19.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.316+0000 7f124165f740 1 -- 172.21.15.16:0/3583231532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf579d4070 msgr2=0x55cf57d79b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:19.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.316+0000 7f124165f740 1 --2- 172.21.15.16:0/3583231532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf579d4070 0x55cf57d79b20 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f122c02f410 tx=0x7f122c02f440 comp rx=0 tx=0).stop 2024-08-31T06:18:19.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.316+0000 7f124165f740 1 -- 172.21.15.16:0/3583231532 shutdown_connections 2024-08-31T06:18:19.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.316+0000 7f124165f740 1 --2- 172.21.15.16:0/3583231532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf579ab1b0 0x55cf57d7a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:19.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.316+0000 7f124165f740 1 --2- 172.21.15.16:0/3583231532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf579b3390 0x55cf57d7a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:19.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.316+0000 7f124165f740 1 --2- 172.21.15.16:0/3583231532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf579d4070 0x55cf57d79b20 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:19.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.316+0000 7f124165f740 1 -- 172.21.15.16:0/3583231532 >> 172.21.15.16:0/3583231532 conn(0x55cf579c8460 msgr2=0x55cf57d77a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:19.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.316+0000 7f124165f740 1 -- 172.21.15.16:0/3583231532 shutdown_connections 2024-08-31T06:18:19.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.316+0000 7f124165f740 1 -- 172.21.15.16:0/3583231532 wait complete. 2024-08-31T06:18:19.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.317+0000 7f1233e00640 1 --2- 172.21.15.16:0/351236454 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f12180b4ca0 0x7f12180b71b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x55cf57d787a0 tx=0x7f1224005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:19.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.317+0000 7f124165f740 1 -- start start 2024-08-31T06:18:19.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.317+0000 7f124165f740 1 Processor -- start 2024-08-31T06:18:19.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.318+0000 7f124165f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf579ab1b0 0x55cf57d85800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:19.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.318+0000 7f124165f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf579b3390 0x55cf57d85de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:19.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.318+0000 7f124165f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf579d4070 0x55cf57d91440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:19.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.318+0000 7f124165f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cf57b42700 con 0x55cf579d4070 2024-08-31T06:18:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.318+0000 7f124165f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55cf57c66cb0 con 0x55cf579ab1b0 2024-08-31T06:18:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.318+0000 7f124165f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55cf57d88120 con 0x55cf579b3390 2024-08-31T06:18:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.318+0000 7f124165f740 10 client.0 _ll_register_callbacks cb 0x55cf57c8c430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:18:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.318+0000 7f1238a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf579d4070 0x55cf57d91440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.318+0000 7f1233e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf579ab1b0 0x55cf57d85800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.318+0000 7f1238a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf579d4070 0x55cf57d91440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57150/0 (socket says 172.21.15.16:57150) 2024-08-31T06:18:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.318+0000 7f1233e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf579ab1b0 0x55cf57d85800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:33114/0 (socket says 172.21.15.16:33114) 2024-08-31T06:18:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.318+0000 7f1238a00640 1 -- 172.21.15.16:0/2081870620 learned_addr learned my addr 172.21.15.16:0/2081870620 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.318+0000 7f1233400640 1 --2- 172.21.15.16:0/2081870620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf579b3390 0x55cf57d85de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.319+0000 7f1233e00640 1 -- 172.21.15.16:0/2081870620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf579b3390 msgr2=0x55cf57d85de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.319+0000 7f1233e00640 1 --2- 172.21.15.16:0/2081870620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf579b3390 0x55cf57d85de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.319+0000 7f1233e00640 1 -- 172.21.15.16:0/2081870620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf579d4070 msgr2=0x55cf57d91440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.319+0000 7f1233e00640 1 --2- 172.21.15.16:0/2081870620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf579d4070 0x55cf57d91440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:19.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.319+0000 7f1233e00640 1 -- 172.21.15.16:0/2081870620 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf57ca7870 con 0x55cf579ab1b0 2024-08-31T06:18:19.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.319+0000 7f1233e00640 1 --2- 172.21.15.16:0/2081870620 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf579ab1b0 0x55cf57d85800 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f12240045f0 tx=0x7f1224004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de229a6beb095b90 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:19.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.319+0000 7f121d600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:18:19.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.319+0000 7f121d600640 1 -- 172.21.15.16:0/2081870620 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122401f640 con 0x55cf579ab1b0 2024-08-31T06:18:19.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.319+0000 7f124165f740 1 -- 172.21.15.16:0/2081870620 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f122c036030 con 0x55cf579ab1b0 2024-08-31T06:18:19.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.319+0000 7f121d600640 1 -- 172.21.15.16:0/2081870620 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8470+0+0 (secure 0 0 0) 0x7f122401f7e0 con 0x55cf579ab1b0 2024-08-31T06:18:19.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.320+0000 7f121d600640 1 -- 172.21.15.16:0/2081870620 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f122401f9e0 con 0x55cf579ab1b0 2024-08-31T06:18:19.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.320+0000 7f124165f740 10 client.28082 fetch_fsmap learned FSMap version 46 2024-08-31T06:18:19.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.320+0000 7f124165f740 1 -- 172.21.15.16:0/2081870620 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55cf57d94270 con 0x55cf579ab1b0 2024-08-31T06:18:19.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.320+0000 7f121d600640 1 -- 172.21.15.16:0/2081870620 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f122c036030 con 0x55cf579ab1b0 2024-08-31T06:18:19.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.321+0000 7f121d600640 1 -- 172.21.15.16:0/2081870620 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7f1224027070 con 0x55cf579ab1b0 2024-08-31T06:18:19.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.321+0000 7f124165f740 10 client.28082 fetch_fsmap finished waiting for FSMap version 46 2024-08-31T06:18:19.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.321+0000 7f124165f740 10 client.28082 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:18:19.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.321+0000 7f124165f740 20 client.28082 populate_metadata read hostname 'smithi016' 2024-08-31T06:18:19.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.321+0000 7f124165f740 1 --2- 172.21.15.16:0/2081870620 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55cf579929f0 0x55cf57d977e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:19.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.321+0000 7f124165f740 4 client.28082 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:18:19.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.321+0000 7f124165f740 1 -- 172.21.15.16:0/2081870620 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf57cf26c0 con 0x55cf579929f0 2024-08-31T06:18:19.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.321+0000 7f1233400640 1 --2- 172.21.15.16:0/2081870620 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55cf579929f0 0x55cf57d977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:19.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.322+0000 7f1233400640 1 --2- 172.21.15.16:0/2081870620 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55cf579929f0 0x55cf57d977e0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f81495593e63c8fa server_cookie=1d8766fcd2d79c13 in_seq=0 out_seq=0 2024-08-31T06:18:19.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.322+0000 7f121d600640 10 client.28082 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:18:19.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.322+0000 7f121d600640 1 -- 172.21.15.16:0/2081870620 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cf57cf26c0 con 0x55cf579929f0 2024-08-31T06:18:19.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.322+0000 7f121d600640 10 client.28082 handle_command_reply: tid=0 2024-08-31T06:18:19.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.323+0000 7f124165f740 1 client.28082 shutdown 2024-08-31T06:18:19.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.323+0000 7f124165f740 1 -- 172.21.15.16:0/2081870620 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf579ab1b0 msgr2=0x55cf57d85800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:19.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.323+0000 7f124165f740 1 --2- 172.21.15.16:0/2081870620 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf579ab1b0 0x55cf57d85800 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f12240045f0 tx=0x7f1224004730 comp rx=0 tx=0).stop 2024-08-31T06:18:19.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.323+0000 7f124165f740 1 -- 172.21.15.16:0/2081870620 shutdown_connections 2024-08-31T06:18:19.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.323+0000 7f124165f740 1 --2- 172.21.15.16:0/2081870620 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55cf579929f0 0x55cf57d977e0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:19.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.323+0000 7f124165f740 1 --2- 172.21.15.16:0/2081870620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf579b3390 0x55cf57d85de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:19.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.323+0000 7f124165f740 1 --2- 172.21.15.16:0/2081870620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf579d4070 0x55cf57d91440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:19.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.323+0000 7f124165f740 1 --2- 172.21.15.16:0/2081870620 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf579ab1b0 0x55cf57d85800 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:19.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.323+0000 7f124165f740 1 -- 172.21.15.16:0/2081870620 >> 172.21.15.16:0/2081870620 conn(0x55cf579c8460 msgr2=0x55cf57d7cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:19.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.323+0000 7f121d600640 0 client.28082 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:18:19.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.323+0000 7f124165f740 1 -- 172.21.15.16:0/2081870620 shutdown_connections 2024-08-31T06:18:19.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.323+0000 7f124165f740 1 -- 172.21.15.16:0/2081870620 wait complete. 2024-08-31T06:18:19.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.323+0000 7f124165f740 20 client.28082 trim_cache size 0 max 16384 2024-08-31T06:18:19.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.341+0000 7f124165f740 1 Processor -- start 2024-08-31T06:18:19.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.342+0000 7f124165f740 1 -- start start 2024-08-31T06:18:19.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.342+0000 7f124165f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf579ab1b0 0x55cf57d807b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:19.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.342+0000 7f124165f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf579b3390 0x55cf57d7ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:19.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.342+0000 7f124165f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf579d4070 0x55cf57d7f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:19.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.342+0000 7f124165f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f12341b3e30 con 0x55cf579d4070 2024-08-31T06:18:19.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.342+0000 7f124165f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f12341b1670 con 0x55cf579ab1b0 2024-08-31T06:18:19.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.342+0000 7f124165f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f12341af4e0 con 0x55cf579b3390 2024-08-31T06:18:19.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.342+0000 7f1238a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf579d4070 0x55cf57d7f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:19.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.342+0000 7f1233400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf579b3390 0x55cf57d7ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:19.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.342+0000 7f1238a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf579d4070 0x55cf57d7f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57164/0 (socket says 172.21.15.16:57164) 2024-08-31T06:18:19.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.342+0000 7f1238a00640 1 -- 172.21.15.16:0/32210270 learned_addr learned my addr 172.21.15.16:0/32210270 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:19.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.342+0000 7f1233e00640 1 --2- 172.21.15.16:0/32210270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf579ab1b0 0x55cf57d807b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:19.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.343+0000 7f1238a00640 1 -- 172.21.15.16:0/32210270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf579b3390 msgr2=0x55cf57d7ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:19.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.343+0000 7f1238a00640 1 --2- 172.21.15.16:0/32210270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf579b3390 0x55cf57d7ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:19.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.343+0000 7f1238a00640 1 -- 172.21.15.16:0/32210270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf579ab1b0 msgr2=0x55cf57d807b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:19.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.343+0000 7f1238a00640 1 --2- 172.21.15.16:0/32210270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf579ab1b0 0x55cf57d807b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:19.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.343+0000 7f1238a00640 1 -- 172.21.15.16:0/32210270 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf57d78380 con 0x55cf579d4070 2024-08-31T06:18:19.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.343+0000 7f1238a00640 1 --2- 172.21.15.16:0/32210270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf579d4070 0x55cf57d7f140 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f122c000f40 tx=0x7f122c003c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b88622d67a9c9ff8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:19.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.343+0000 7f121fe00640 1 -- 172.21.15.16:0/32210270 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122c055020 con 0x55cf579d4070 2024-08-31T06:18:19.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.343+0000 7f121fe00640 1 -- 172.21.15.16:0/32210270 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f122c036040 con 0x55cf579d4070 2024-08-31T06:18:19.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.344+0000 7f121fe00640 1 -- 172.21.15.16:0/32210270 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122c04a040 con 0x55cf579d4070 2024-08-31T06:18:19.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.344+0000 7f124165f740 1 -- 172.21.15.16:0/32210270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf579d4070 msgr2=0x55cf57d7f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:19.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.344+0000 7f124165f740 1 --2- 172.21.15.16:0/32210270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf579d4070 0x55cf57d7f140 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f122c000f40 tx=0x7f122c003c70 comp rx=0 tx=0).stop 2024-08-31T06:18:19.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.344+0000 7f124165f740 1 -- 172.21.15.16:0/32210270 shutdown_connections 2024-08-31T06:18:19.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.344+0000 7f124165f740 1 --2- 172.21.15.16:0/32210270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf579b3390 0x55cf57d7ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:19.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.344+0000 7f124165f740 1 --2- 172.21.15.16:0/32210270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf579d4070 0x55cf57d7f140 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:19.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.344+0000 7f124165f740 1 --2- 172.21.15.16:0/32210270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf579ab1b0 0x55cf57d807b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:19.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.344+0000 7f124165f740 1 -- 172.21.15.16:0/32210270 >> 172.21.15.16:0/32210270 conn(0x55cf579c8460 msgr2=0x55cf57d83e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:19.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.345+0000 7f124165f740 1 -- 172.21.15.16:0/32210270 shutdown_connections 2024-08-31T06:18:19.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.345+0000 7f124165f740 1 -- 172.21.15.16:0/32210270 wait complete. 2024-08-31T06:18:19.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.345+0000 7f124165f740 1 -- start start 2024-08-31T06:18:19.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.345+0000 7f124165f740 1 Processor -- start 2024-08-31T06:18:19.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.346+0000 7f124165f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf579ab1b0 0x55cf57d8a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:19.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.346+0000 7f124165f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf579b3390 0x55cf57d8aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:19.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.346+0000 7f124165f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf579d4070 0x55cf57d8af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:19.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.346+0000 7f124165f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f12341afb30 con 0x55cf579b3390 2024-08-31T06:18:19.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.346+0000 7f124165f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f12341b0fa0 con 0x55cf579d4070 2024-08-31T06:18:19.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.346+0000 7f124165f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f12341b0200 con 0x55cf579ab1b0 2024-08-31T06:18:19.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.346+0000 7f124165f740 10 client.0 _ll_register_callbacks cb 0x7f12240044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:18:19.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.346+0000 7f1233e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf579ab1b0 0x55cf57d8a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:19.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.346+0000 7f1233400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf579b3390 0x55cf57d8aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:19.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.346+0000 7f1233e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf579ab1b0 0x55cf57d8a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:36474/0 (socket says 172.21.15.16:36474) 2024-08-31T06:18:19.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.346+0000 7f1233e00640 1 -- 172.21.15.16:0/1997435464 learned_addr learned my addr 172.21.15.16:0/1997435464 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:19.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.346+0000 7f1238a00640 1 --2- 172.21.15.16:0/1997435464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf579d4070 0x55cf57d8af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:19.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.346+0000 7f1233400640 1 -- 172.21.15.16:0/1997435464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf579ab1b0 msgr2=0x55cf57d8a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:19.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.346+0000 7f1233400640 1 --2- 172.21.15.16:0/1997435464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf579ab1b0 0x55cf57d8a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:19.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.346+0000 7f1233400640 1 -- 172.21.15.16:0/1997435464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf579d4070 msgr2=0x55cf57d8af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:19.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.346+0000 7f1233400640 1 --2- 172.21.15.16:0/1997435464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf579d4070 0x55cf57d8af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:19.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.346+0000 7f1233400640 1 -- 172.21.15.16:0/1997435464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf57e02310 con 0x55cf579b3390 2024-08-31T06:18:19.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.346+0000 7f1238a00640 1 --2- 172.21.15.16:0/1997435464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf579d4070 0x55cf57d8af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:18:19.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.347+0000 7f1233400640 1 --2- 172.21.15.16:0/1997435464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf579b3390 0x55cf57d8aa30 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f12200e2ea0 tx=0x7f12200096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5081afe4a2801eba server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:19.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.347+0000 7f121d600640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:18:19.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.347+0000 7f121d600640 1 -- 172.21.15.16:0/1997435464 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12200ae070 con 0x55cf579b3390 2024-08-31T06:18:19.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.347+0000 7f121d600640 1 -- 172.21.15.16:0/1997435464 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8470+0+0 (secure 0 0 0) 0x7f12200e3120 con 0x55cf579b3390 2024-08-31T06:18:19.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.347+0000 7f124165f740 1 -- 172.21.15.16:0/1997435464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f122c036040 con 0x55cf579b3390 2024-08-31T06:18:19.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.347+0000 7f121d600640 1 -- 172.21.15.16:0/1997435464 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f12200e3320 con 0x55cf579b3390 2024-08-31T06:18:19.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.347+0000 7f124165f740 10 client.17730 fetch_fsmap learned FSMap version 46 2024-08-31T06:18:19.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.348+0000 7f124165f740 1 -- 172.21.15.16:0/1997435464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55cf57e03600 con 0x55cf579b3390 2024-08-31T06:18:19.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.348+0000 7f121d600640 1 -- 172.21.15.16:0/1997435464 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f122c036040 con 0x55cf579b3390 2024-08-31T06:18:19.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.348+0000 7f121d600640 1 -- 172.21.15.16:0/1997435464 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7f1220091ce0 con 0x55cf579b3390 2024-08-31T06:18:19.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.348+0000 7f124165f740 10 client.17730 fetch_fsmap finished waiting for FSMap version 46 2024-08-31T06:18:19.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.348+0000 7f124165f740 10 client.17730 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:18:19.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.348+0000 7f124165f740 20 client.17730 populate_metadata read hostname 'smithi016' 2024-08-31T06:18:19.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.348+0000 7f124165f740 1 --2- 172.21.15.16:0/1997435464 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55cf57967da0 0x55cf57e05dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:19.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.348+0000 7f124165f740 4 client.17730 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T06:18:19.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.348+0000 7f124165f740 1 -- 172.21.15.16:0/1997435464 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cf57cee160 con 0x55cf57967da0 2024-08-31T06:18:19.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.349+0000 7f1233e00640 1 --2- 172.21.15.16:0/1997435464 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55cf57967da0 0x55cf57e05dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:19.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.349+0000 7f1233e00640 1 --2- 172.21.15.16:0/1997435464 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55cf57967da0 0x55cf57e05dd0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac2540f5d30566e1 server_cookie=c67f0b8ee3a69ec9 in_seq=0 out_seq=0 2024-08-31T06:18:19.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.349+0000 7f121d600640 10 client.17730 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:18:19.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.349+0000 7f121d600640 1 -- 172.21.15.16:0/1997435464 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4066 (crc 0 0 0) 0x55cf57cee160 con 0x55cf57967da0 2024-08-31T06:18:19.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.349+0000 7f121d600640 10 client.17730 handle_command_reply: tid=0 2024-08-31T06:18:19.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.349+0000 7f124165f740 1 client.17730 shutdown 2024-08-31T06:18:19.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.350+0000 7f124165f740 1 -- 172.21.15.16:0/1997435464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf579b3390 msgr2=0x55cf57d8aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:19.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.350+0000 7f124165f740 1 --2- 172.21.15.16:0/1997435464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf579b3390 0x55cf57d8aa30 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f12200e2ea0 tx=0x7f12200096a0 comp rx=0 tx=0).stop 2024-08-31T06:18:19.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.350+0000 7f124165f740 1 -- 172.21.15.16:0/1997435464 shutdown_connections 2024-08-31T06:18:19.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.350+0000 7f124165f740 1 --2- 172.21.15.16:0/1997435464 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55cf57967da0 0x55cf57e05dd0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:19.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.350+0000 7f124165f740 1 --2- 172.21.15.16:0/1997435464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf579d4070 0x55cf57d8af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:19.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.350+0000 7f124165f740 1 --2- 172.21.15.16:0/1997435464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf579b3390 0x55cf57d8aa30 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:19.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.350+0000 7f124165f740 1 --2- 172.21.15.16:0/1997435464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf579ab1b0 0x55cf57d8a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:19.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.350+0000 7f121d600640 0 client.17730 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:18:19.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.350+0000 7f124165f740 1 -- 172.21.15.16:0/1997435464 >> 172.21.15.16:0/1997435464 conn(0x55cf579c8460 msgr2=0x55cf57d7cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:19.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.350+0000 7f124165f740 1 -- 172.21.15.16:0/1997435464 shutdown_connections 2024-08-31T06:18:19.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.350+0000 7f124165f740 1 -- 172.21.15.16:0/1997435464 wait complete. 2024-08-31T06:18:19.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.350+0000 7f124165f740 20 client.17730 trim_cache size 0 max 16384 2024-08-31T06:18:19.356 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:18:19.356 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-08-31T06:18:19.356 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {}, 2024-08-31T06:18:19.356 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-08-31T06:18:19.356 INFO:teuthology.orchestra.run.smithi016.stdout: "e497cd88-1e7c-4631-a4a8-cdd8e1fa6c83": { 2024-08-31T06:18:19.356 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:17:42.476795+0000", 2024-08-31T06:18:19.356 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:18:19.356 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:18:19.356 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:18:19.356 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:18:19.357 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:18:19.357 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:18:19.357 INFO:teuthology.orchestra.run.smithi016.stdout: "c300b7e8-f3fd-482c-a909-9329bc61e570": { 2024-08-31T06:18:19.357 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:06:49.868046+0000", 2024-08-31T06:18:19.357 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:18:19.357 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:18:19.357 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:18:19.357 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:18:19.357 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:18:19.357 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:18:19.357 INFO:teuthology.orchestra.run.smithi016.stdout: "6a03046e-cdf4-4a14-b9bd-b560b8968467": { 2024-08-31T06:18:19.357 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:12:38.002354+0000", 2024-08-31T06:18:19.357 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:18:19.357 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:18:19.358 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:18:19.358 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:18:19.358 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:18:19.358 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:18:19.358 INFO:teuthology.orchestra.run.smithi016.stdout: "004b47bb-0fe8-4d28-9d9d-78bde6a61199": { 2024-08-31T06:18:19.358 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:16:48.915926+0000", 2024-08-31T06:18:19.358 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:18:19.358 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:18:19.358 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:18:19.358 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:18:19.358 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:18:19.358 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:18:19.358 INFO:teuthology.orchestra.run.smithi016.stdout: "88d7bdb8-cb1c-45e5-ad29-4cd8da6c6003": { 2024-08-31T06:18:19.358 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:07:39.648942+0000", 2024-08-31T06:18:19.359 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:18:19.359 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:18:19.359 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:18:19.359 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:18:19.359 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:18:19.359 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:18:19.359 INFO:teuthology.orchestra.run.smithi016.stdout: "0fdcd762-533d-42be-9289-b71c8e3b81b1": { 2024-08-31T06:18:19.359 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:15:06.169112+0000", 2024-08-31T06:18:19.359 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:18:19.359 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:18:19.359 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:18:19.359 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:18:19.359 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:18:19.359 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:18:19.360 INFO:teuthology.orchestra.run.smithi016.stdout: "c8455d74-2fe7-4d93-bcf2-19b419d16016": { 2024-08-31T06:18:19.360 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:08:24.035960+0000", 2024-08-31T06:18:19.360 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:18:19.360 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:18:19.360 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:18:19.360 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:18:19.360 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:18:19.360 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:18:19.360 INFO:teuthology.orchestra.run.smithi016.stdout: "78d6ef02-8551-45e8-ad60-9627eb573fc1": { 2024-08-31T06:18:19.360 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:09:13.547865+0000", 2024-08-31T06:18:19.360 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:18:19.360 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:18:19.360 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:18:19.360 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:18:19.361 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:18:19.361 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:18:19.361 INFO:teuthology.orchestra.run.smithi016.stdout: "c1fd6904-2e02-47fa-939a-47349b9fe36b": { 2024-08-31T06:18:19.361 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:06.678691+0000", 2024-08-31T06:18:19.361 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:18:19.361 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:18:19.361 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:18:19.361 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:18:19.361 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:18:19.361 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:18:19.361 INFO:teuthology.orchestra.run.smithi016.stdout: "a10cce76-99ab-46b5-b3b2-07b82899b8f9": { 2024-08-31T06:18:19.361 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:11:43.967546+0000", 2024-08-31T06:18:19.361 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:18:19.361 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:18:19.362 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:18:19.362 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:18:19.362 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:18:19.362 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:18:19.362 INFO:teuthology.orchestra.run.smithi016.stdout: "91b72a34-4ba9-4126-a414-8b862f8c5f78": { 2024-08-31T06:18:19.362 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:13:28.663237+0000", 2024-08-31T06:18:19.362 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:18:19.362 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:18:19.362 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:18:19.362 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:18:19.362 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:18:19.362 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:18:19.362 INFO:teuthology.orchestra.run.smithi016.stdout: "e739a16e-8ace-43a8-a1ce-ac27d39ef8a3": { 2024-08-31T06:18:19.363 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:54.084646+0000", 2024-08-31T06:18:19.363 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:18:19.363 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:18:19.363 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:18:19.363 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:18:19.363 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:18:19.363 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:18:19.363 INFO:teuthology.orchestra.run.smithi016.stdout: "a64d5301-8558-4e9c-ac9b-8d0ac7057e74": { 2024-08-31T06:18:19.363 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:14:20.338104+0000", 2024-08-31T06:18:19.363 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:18:19.363 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:18:19.363 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:18:19.363 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:18:19.364 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:18:19.364 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:18:19.364 INFO:teuthology.orchestra.run.smithi016.stdout: "f9c4a22e-0c8a-4928-9b0d-e2ce1db5a86b": { 2024-08-31T06:18:19.364 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:15:57.496015+0000", 2024-08-31T06:18:19.364 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:18:19.364 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:18:19.364 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:18:19.364 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:18:19.364 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:18:19.364 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:18:19.364 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:18:19.364 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:18:19.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.351+0000 7f1239400640 1 -- 172.21.15.16:0/351236454 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f12180b4ca0 msgr2=0x7f12180b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:19.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.351+0000 7f1239400640 1 --2- 172.21.15.16:0/351236454 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f12180b4ca0 0x7f12180b71b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x55cf57d787a0 tx=0x7f1224005f90 comp rx=0 tx=0).stop 2024-08-31T06:18:19.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.351+0000 7f1239400640 1 -- 172.21.15.16:0/351236454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f123410b290 msgr2=0x7f123419c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:19.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.351+0000 7f1239400640 1 --2- 172.21.15.16:0/351236454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f123410b290 0x7f123419c8f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f122000e9f0 tx=0x7f122000eeb0 comp rx=0 tx=0).stop 2024-08-31T06:18:19.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.351+0000 7f1239400640 1 -- 172.21.15.16:0/351236454 shutdown_connections 2024-08-31T06:18:19.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.351+0000 7f1239400640 1 --2- 172.21.15.16:0/351236454 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f12180b4ca0 0x7f12180b71b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:19.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.351+0000 7f1239400640 1 --2- 172.21.15.16:0/351236454 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f123410bbe0 0x7f12341a1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:19.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.351+0000 7f1239400640 1 --2- 172.21.15.16:0/351236454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f123410b290 0x7f123419c8f0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:19.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.351+0000 7f1239400640 1 --2- 172.21.15.16:0/351236454 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1234105950 0x7f123419c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:19.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.351+0000 7f1239400640 1 -- 172.21.15.16:0/351236454 >> 172.21.15.16:0/351236454 conn(0x7f1234101420 msgr2=0x7f1234108840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:19.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.351+0000 7f1239400640 1 -- 172.21.15.16:0/351236454 shutdown_connections 2024-08-31T06:18:19.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:19.351+0000 7f1239400640 1 -- 172.21.15.16:0/351236454 wait complete. 2024-08-31T06:18:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.099+0000 7f38f9a00640 1 -- 172.21.15.16:0/3134437588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38f4109930 msgr2=0x7f38f4110160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.099+0000 7f38f9a00640 1 --2- 172.21.15.16:0/3134437588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38f4109930 0x7f38f4110160 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f38ec009ca0 tx=0x7f38ec02a8c0 comp rx=0 tx=0).stop 2024-08-31T06:18:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.099+0000 7f38f9a00640 1 -- 172.21.15.16:0/3134437588 shutdown_connections 2024-08-31T06:18:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.099+0000 7f38f9a00640 1 --2- 172.21.15.16:0/3134437588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38f4109930 0x7f38f4110160 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.099+0000 7f38f9a00640 1 --2- 172.21.15.16:0/3134437588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38f4108fe0 0x7f38f41093f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.099+0000 7f38f9a00640 1 --2- 172.21.15.16:0/3134437588 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38f41036b0 0x7f38f4103ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.099+0000 7f38f9a00640 1 -- 172.21.15.16:0/3134437588 >> 172.21.15.16:0/3134437588 conn(0x7f38f40ff0d0 msgr2=0x7f38f4101520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.099+0000 7f38f9a00640 1 -- 172.21.15.16:0/3134437588 shutdown_connections 2024-08-31T06:18:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.099+0000 7f38f9a00640 1 -- 172.21.15.16:0/3134437588 wait complete. 2024-08-31T06:18:20.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.100+0000 7f38f9a00640 1 Processor -- start 2024-08-31T06:18:20.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.100+0000 7f38f9a00640 1 -- start start 2024-08-31T06:18:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.100+0000 7f38f9a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38f41036b0 0x7f38f419a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.100+0000 7f38f9a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38f4108fe0 0x7f38f419a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.100+0000 7f38f9a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38f4109930 0x7f38f419f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.100+0000 7f38f9a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f38f4113650 con 0x7f38f4109930 2024-08-31T06:18:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.100+0000 7f38f9a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f38f41134d0 con 0x7f38f41036b0 2024-08-31T06:18:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.100+0000 7f38f9a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f38f41137d0 con 0x7f38f4108fe0 2024-08-31T06:18:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.101+0000 7f38f9000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38f4109930 0x7f38f419f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.101+0000 7f38f9000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38f4109930 0x7f38f419f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57176/0 (socket says 172.21.15.16:57176) 2024-08-31T06:18:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.101+0000 7f38f9000640 1 -- 172.21.15.16:0/2773959517 learned_addr learned my addr 172.21.15.16:0/2773959517 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.101+0000 7f38f3400640 1 --2- 172.21.15.16:0/2773959517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38f4108fe0 0x7f38f419a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.101+0000 7f38f3e00640 1 --2- 172.21.15.16:0/2773959517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38f41036b0 0x7f38f419a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.101+0000 7f38f9000640 1 -- 172.21.15.16:0/2773959517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38f4108fe0 msgr2=0x7f38f419a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.101+0000 7f38f9000640 1 --2- 172.21.15.16:0/2773959517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38f4108fe0 0x7f38f419a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.101+0000 7f38f9000640 1 -- 172.21.15.16:0/2773959517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38f41036b0 msgr2=0x7f38f419a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.101+0000 7f38f9000640 1 --2- 172.21.15.16:0/2773959517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38f41036b0 0x7f38f419a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.101+0000 7f38f9000640 1 -- 172.21.15.16:0/2773959517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38f419ffa0 con 0x7f38f4109930 2024-08-31T06:18:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.101+0000 7f38f9000640 1 --2- 172.21.15.16:0/2773959517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38f4109930 0x7f38f419f8a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f38ec02f540 tx=0x7f38ec0316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.102+0000 7f38f0c00640 1 -- 172.21.15.16:0/2773959517 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ec003b70 con 0x7f38f4109930 2024-08-31T06:18:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.102+0000 7f38f0c00640 1 -- 172.21.15.16:0/2773959517 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38ec003d10 con 0x7f38f4109930 2024-08-31T06:18:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.102+0000 7f38f9a00640 1 -- 172.21.15.16:0/2773959517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38f41a0230 con 0x7f38f4109930 2024-08-31T06:18:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.102+0000 7f38f0c00640 1 -- 172.21.15.16:0/2773959517 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ec02f990 con 0x7f38f4109930 2024-08-31T06:18:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.102+0000 7f38f9a00640 1 -- 172.21.15.16:0/2773959517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38f41a6a20 con 0x7f38f4109930 2024-08-31T06:18:20.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.105+0000 7f3902015740 1 Processor -- start 2024-08-31T06:18:20.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.105+0000 7f38f0c00640 1 -- 172.21.15.16:0/2773959517 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f38ec02fb30 con 0x7f38f4109930 2024-08-31T06:18:20.108 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.105+0000 7f3902015740 1 -- start start 2024-08-31T06:18:20.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.106+0000 7f3902015740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b167a7e1b0 0x55b167e21b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.106+0000 7f3902015740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b167a552f0 0x55b167e22040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.106+0000 7f3902015740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b167a5d4d0 0x55b167e22580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.106+0000 7f3902015740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b167d96f70 con 0x55b167a5d4d0 2024-08-31T06:18:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.106+0000 7f3902015740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b167d86500 con 0x55b167a552f0 2024-08-31T06:18:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.106+0000 7f3902015740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b167d8a4b0 con 0x55b167a7e1b0 2024-08-31T06:18:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.106+0000 7f38f3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b167a552f0 0x55b167e22040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.106+0000 7f38f3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b167a552f0 0x55b167e22040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:33158/0 (socket says 172.21.15.16:33158) 2024-08-31T06:18:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.106+0000 7f38f3e00640 1 -- 172.21.15.16:0/3113654543 learned_addr learned my addr 172.21.15.16:0/3113654543 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.106+0000 7f38f3400640 1 --2- 172.21.15.16:0/3113654543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b167a5d4d0 0x55b167e22580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.106+0000 7f38f3400640 1 --2- 172.21.15.16:0/3113654543 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b167a7e1b0 0x55b167e21b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.107+0000 7f38f3e00640 1 -- 172.21.15.16:0/3113654543 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b167a7e1b0 msgr2=0x55b167e21b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.107+0000 7f38f3e00640 1 --2- 172.21.15.16:0/3113654543 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b167a7e1b0 0x55b167e21b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.107+0000 7f38f0c00640 1 --2- 172.21.15.16:0/2773959517 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f38d80b4d20 0x7f38d80b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.107+0000 7f38f3e00640 1 -- 172.21.15.16:0/3113654543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b167a5d4d0 msgr2=0x55b167e22580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.107+0000 7f38f3e00640 1 --2- 172.21.15.16:0/3113654543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b167a5d4d0 0x55b167e22580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.107+0000 7f38f3e00640 1 -- 172.21.15.16:0/3113654543 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b167d40c10 con 0x55b167a552f0 2024-08-31T06:18:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.107+0000 7f38f3400640 1 --2- 172.21.15.16:0/3113654543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b167a5d4d0 0x55b167e22580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:18:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.107+0000 7f38f0c00640 1 -- 172.21.15.16:0/2773959517 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8470+0+0 (secure 0 0 0) 0x7f38ec1066b0 con 0x7f38f4109930 2024-08-31T06:18:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.107+0000 7f38f9000640 1 --2- 172.21.15.16:0/2773959517 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f38d80b4d20 0x7f38d80b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.107+0000 7f38f3e00640 1 --2- 172.21.15.16:0/3113654543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b167a552f0 0x55b167e22040 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f38e4009d20 tx=0x7f38e4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f32ebff837c372ea server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.107+0000 7f38e9600640 1 -- 172.21.15.16:0/3113654543 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e401e070 con 0x55b167a552f0 2024-08-31T06:18:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.107+0000 7f38e9600640 1 -- 172.21.15.16:0/3113654543 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38e4019ae0 con 0x55b167a552f0 2024-08-31T06:18:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.107+0000 7f38e9600640 1 -- 172.21.15.16:0/3113654543 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e400eab0 con 0x55b167a552f0 2024-08-31T06:18:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.108+0000 7f3902015740 1 -- 172.21.15.16:0/3113654543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b167a552f0 msgr2=0x55b167e22040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.108+0000 7f3902015740 1 --2- 172.21.15.16:0/3113654543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b167a552f0 0x55b167e22040 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f38e4009d20 tx=0x7f38e4011810 comp rx=0 tx=0).stop 2024-08-31T06:18:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.108+0000 7f38f9000640 1 --2- 172.21.15.16:0/2773959517 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f38d80b4d20 0x7f38d80b7230 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f38ec0d0560 tx=0x7f38ec005df0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.108+0000 7f3902015740 1 -- 172.21.15.16:0/3113654543 shutdown_connections 2024-08-31T06:18:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.108+0000 7f3902015740 1 --2- 172.21.15.16:0/3113654543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b167a5d4d0 0x55b167e22580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.108+0000 7f3902015740 1 --2- 172.21.15.16:0/3113654543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b167a552f0 0x55b167e22040 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.108+0000 7f3902015740 1 --2- 172.21.15.16:0/3113654543 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b167a7e1b0 0x55b167e21b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.108+0000 7f3902015740 1 -- 172.21.15.16:0/3113654543 >> 172.21.15.16:0/3113654543 conn(0x55b167a72300 msgr2=0x55b167e1f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.108+0000 7f3902015740 1 -- 172.21.15.16:0/3113654543 shutdown_connections 2024-08-31T06:18:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.108+0000 7f3902015740 1 -- 172.21.15.16:0/3113654543 wait complete. 2024-08-31T06:18:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.109+0000 7f3902015740 1 -- start start 2024-08-31T06:18:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.109+0000 7f3902015740 1 Processor -- start 2024-08-31T06:18:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.109+0000 7f3902015740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b167a552f0 0x55b167e314d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.110+0000 7f3902015740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b167a5d4d0 0x55b167e31a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.110+0000 7f3902015740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b167a7e1b0 0x55b167e31f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.110+0000 7f3902015740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b167cbfab0 con 0x55b167a7e1b0 2024-08-31T06:18:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.110+0000 7f3902015740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b167d2d320 con 0x55b167a5d4d0 2024-08-31T06:18:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.110+0000 7f3902015740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b167e34660 con 0x55b167a552f0 2024-08-31T06:18:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.110+0000 7f3902015740 10 client.0 _ll_register_callbacks cb 0x55b167d336a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:18:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.110+0000 7f38f3400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b167a552f0 0x55b167e314d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.110+0000 7f38f3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b167a5d4d0 0x55b167e31a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.110+0000 7f38f3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b167a5d4d0 0x55b167e31a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:33172/0 (socket says 172.21.15.16:33172) 2024-08-31T06:18:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.110+0000 7f38f3e00640 1 -- 172.21.15.16:0/4033632896 learned_addr learned my addr 172.21.15.16:0/4033632896 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.110+0000 7f38f3400640 1 --2- 172.21.15.16:0/4033632896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b167a7e1b0 0x55b167e31f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.110+0000 7f38f3e00640 1 -- 172.21.15.16:0/4033632896 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b167a552f0 msgr2=0x55b167e314d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.110+0000 7f38f3e00640 1 --2- 172.21.15.16:0/4033632896 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b167a552f0 0x55b167e314d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.110+0000 7f38f3e00640 1 -- 172.21.15.16:0/4033632896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b167a7e1b0 msgr2=0x55b167e31f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.110+0000 7f38f3e00640 1 --2- 172.21.15.16:0/4033632896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b167a7e1b0 0x55b167e31f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.110+0000 7f38f3e00640 1 -- 172.21.15.16:0/4033632896 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b167ae9790 con 0x55b167a5d4d0 2024-08-31T06:18:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.111+0000 7f38f3e00640 1 --2- 172.21.15.16:0/4033632896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b167a5d4d0 0x55b167e31a10 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f38e4011d10 tx=0x7f38e4004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89378a4415d11bf3 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.111+0000 7f38de000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:18:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.111+0000 7f38de000640 1 -- 172.21.15.16:0/4033632896 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e401e050 con 0x55b167a5d4d0 2024-08-31T06:18:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.111+0000 7f3902015740 1 -- 172.21.15.16:0/4033632896 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38e4019ae0 con 0x55b167a5d4d0 2024-08-31T06:18:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.111+0000 7f38de000640 1 -- 172.21.15.16:0/4033632896 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8470+0+0 (secure 0 0 0) 0x7f38e4019630 con 0x55b167a5d4d0 2024-08-31T06:18:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.111+0000 7f38de000640 1 -- 172.21.15.16:0/4033632896 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f38e4019870 con 0x55b167a5d4d0 2024-08-31T06:18:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.111+0000 7f3902015740 10 client.28100 fetch_fsmap learned FSMap version 46 2024-08-31T06:18:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.111+0000 7f3902015740 1 -- 172.21.15.16:0/4033632896 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55b167b62620 con 0x55b167a5d4d0 2024-08-31T06:18:20.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.112+0000 7f38de000640 1 -- 172.21.15.16:0/4033632896 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38e4019ae0 con 0x55b167a5d4d0 2024-08-31T06:18:20.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.112+0000 7f38de000640 1 -- 172.21.15.16:0/4033632896 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7f38e402d030 con 0x55b167a5d4d0 2024-08-31T06:18:20.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.112+0000 7f3902015740 10 client.28100 fetch_fsmap finished waiting for FSMap version 46 2024-08-31T06:18:20.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.112+0000 7f3902015740 10 client.28100 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:18:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.112+0000 7f3902015740 20 client.28100 populate_metadata read hostname 'smithi016' 2024-08-31T06:18:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.112+0000 7f3902015740 1 --2- 172.21.15.16:0/4033632896 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55b167a3cd80 0x55b167e43510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.112+0000 7f3902015740 4 client.28100 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:18:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.112+0000 7f3902015740 1 -- 172.21.15.16:0/4033632896 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b167d98ec0 con 0x55b167a3cd80 2024-08-31T06:18:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.112+0000 7f38f3400640 1 --2- 172.21.15.16:0/4033632896 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55b167a3cd80 0x55b167e43510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.113+0000 7f38f3400640 1 --2- 172.21.15.16:0/4033632896 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55b167a3cd80 0x55b167e43510 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b6121e3de6d3eb5 server_cookie=89e48860cddb2661 in_seq=0 out_seq=0 2024-08-31T06:18:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.113+0000 7f38de000640 10 client.28100 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:18:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.114+0000 7f38de000640 1 -- 172.21.15.16:0/4033632896 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b167d98ec0 con 0x55b167a3cd80 2024-08-31T06:18:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.114+0000 7f38de000640 10 client.28100 handle_command_reply: tid=0 2024-08-31T06:18:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.114+0000 7f3902015740 1 client.28100 shutdown 2024-08-31T06:18:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.114+0000 7f3902015740 1 -- 172.21.15.16:0/4033632896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b167a5d4d0 msgr2=0x55b167e31a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.114+0000 7f3902015740 1 --2- 172.21.15.16:0/4033632896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b167a5d4d0 0x55b167e31a10 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f38e4011d10 tx=0x7f38e4004270 comp rx=0 tx=0).stop 2024-08-31T06:18:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.114+0000 7f3902015740 1 -- 172.21.15.16:0/4033632896 shutdown_connections 2024-08-31T06:18:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.114+0000 7f3902015740 1 --2- 172.21.15.16:0/4033632896 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55b167a3cd80 0x55b167e43510 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.114+0000 7f3902015740 1 --2- 172.21.15.16:0/4033632896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b167a7e1b0 0x55b167e31f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.114+0000 7f38de000640 0 client.28100 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:18:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.114+0000 7f3902015740 1 --2- 172.21.15.16:0/4033632896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b167a5d4d0 0x55b167e31a10 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.114+0000 7f3902015740 1 --2- 172.21.15.16:0/4033632896 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b167a552f0 0x55b167e314d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.114+0000 7f3902015740 1 -- 172.21.15.16:0/4033632896 >> 172.21.15.16:0/4033632896 conn(0x55b167a72300 msgr2=0x55b167e24c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.115+0000 7f3902015740 1 -- 172.21.15.16:0/4033632896 shutdown_connections 2024-08-31T06:18:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.115+0000 7f3902015740 1 -- 172.21.15.16:0/4033632896 wait complete. 2024-08-31T06:18:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.115+0000 7f3902015740 20 client.28100 trim_cache size 0 max 16384 2024-08-31T06:18:20.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.131+0000 7f3902015740 1 Processor -- start 2024-08-31T06:18:20.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.131+0000 7f3902015740 1 -- start start 2024-08-31T06:18:20.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.131+0000 7f3902015740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b167a552f0 0x55b167e333b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.131+0000 7f3902015740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b167a5d4d0 0x55b167e23240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.131+0000 7f3902015740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b167a7e1b0 0x55b167e2be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.131+0000 7f3902015740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f38f41afa20 con 0x55b167a5d4d0 2024-08-31T06:18:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.131+0000 7f3902015740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f38f41ad260 con 0x55b167a7e1b0 2024-08-31T06:18:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.131+0000 7f3902015740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f38f41ab0d0 con 0x55b167a552f0 2024-08-31T06:18:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.132+0000 7f38f3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b167a5d4d0 0x55b167e23240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.132+0000 7f38f3400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b167a552f0 0x55b167e333b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.132+0000 7f38f3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b167a5d4d0 0x55b167e23240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57186/0 (socket says 172.21.15.16:57186) 2024-08-31T06:18:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.132+0000 7f38f3e00640 1 -- 172.21.15.16:0/3670753281 learned_addr learned my addr 172.21.15.16:0/3670753281 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.132+0000 7f38f3400640 1 --2- 172.21.15.16:0/3670753281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b167a7e1b0 0x55b167e2be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.132+0000 7f38f3e00640 1 -- 172.21.15.16:0/3670753281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b167a552f0 msgr2=0x55b167e333b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.132+0000 7f38f3e00640 1 --2- 172.21.15.16:0/3670753281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b167a552f0 0x55b167e333b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.132+0000 7f38f3e00640 1 -- 172.21.15.16:0/3670753281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b167a7e1b0 msgr2=0x55b167e2be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.132+0000 7f38f3e00640 1 --2- 172.21.15.16:0/3670753281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b167a7e1b0 0x55b167e2be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.132+0000 7f38f3e00640 1 -- 172.21.15.16:0/3670753281 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b167e43970 con 0x55b167a5d4d0 2024-08-31T06:18:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.132+0000 7f38f3e00640 1 --2- 172.21.15.16:0/3670753281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b167a5d4d0 0x55b167e23240 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f38e4011850 tx=0x7f38e4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acdd6a3a455f2145 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.133+0000 7f38e8c00640 1 -- 172.21.15.16:0/3670753281 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e401e070 con 0x55b167a5d4d0 2024-08-31T06:18:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.133+0000 7f38e8c00640 1 -- 172.21.15.16:0/3670753281 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38e401f040 con 0x55b167a5d4d0 2024-08-31T06:18:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.133+0000 7f38e8c00640 1 -- 172.21.15.16:0/3670753281 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e40039b0 con 0x55b167a5d4d0 2024-08-31T06:18:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.133+0000 7f3902015740 1 -- 172.21.15.16:0/3670753281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b167a5d4d0 msgr2=0x55b167e23240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.133+0000 7f3902015740 1 --2- 172.21.15.16:0/3670753281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b167a5d4d0 0x55b167e23240 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f38e4011850 tx=0x7f38e4005f90 comp rx=0 tx=0).stop 2024-08-31T06:18:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.134+0000 7f3902015740 1 -- 172.21.15.16:0/3670753281 shutdown_connections 2024-08-31T06:18:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.134+0000 7f3902015740 1 --2- 172.21.15.16:0/3670753281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b167a7e1b0 0x55b167e2be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.134+0000 7f3902015740 1 --2- 172.21.15.16:0/3670753281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b167a5d4d0 0x55b167e23240 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.134+0000 7f3902015740 1 --2- 172.21.15.16:0/3670753281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b167a552f0 0x55b167e333b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.134+0000 7f3902015740 1 -- 172.21.15.16:0/3670753281 >> 172.21.15.16:0/3670753281 conn(0x55b167a72300 msgr2=0x55b167e22b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.134+0000 7f3902015740 1 -- 172.21.15.16:0/3670753281 shutdown_connections 2024-08-31T06:18:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.134+0000 7f3902015740 1 -- 172.21.15.16:0/3670753281 wait complete. 2024-08-31T06:18:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.134+0000 7f3902015740 1 -- start start 2024-08-31T06:18:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.134+0000 7f3902015740 1 Processor -- start 2024-08-31T06:18:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.135+0000 7f3902015740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b167a552f0 0x55b167e2b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.135+0000 7f3902015740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b167a5d4d0 0x55b167e2b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.135+0000 7f3902015740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b167a7e1b0 0x55b167e388f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.135+0000 7f3902015740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f38f41ab720 con 0x55b167a5d4d0 2024-08-31T06:18:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.135+0000 7f3902015740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f38f41acb90 con 0x55b167a552f0 2024-08-31T06:18:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.135+0000 7f3902015740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f38f41abdf0 con 0x55b167a7e1b0 2024-08-31T06:18:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.135+0000 7f3902015740 10 client.0 _ll_register_callbacks cb 0x7f38e4009ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:18:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.135+0000 7f38f3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b167a5d4d0 0x55b167e2b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.135+0000 7f38f3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b167a5d4d0 0x55b167e2b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57200/0 (socket says 172.21.15.16:57200) 2024-08-31T06:18:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.135+0000 7f38f3e00640 1 -- 172.21.15.16:0/2451834493 learned_addr learned my addr 172.21.15.16:0/2451834493 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.135+0000 7f38f3400640 1 --2- 172.21.15.16:0/2451834493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b167a552f0 0x55b167e2b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.135+0000 7f38f3400640 1 --2- 172.21.15.16:0/2451834493 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b167a7e1b0 0x55b167e388f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.135+0000 7f38f3e00640 1 -- 172.21.15.16:0/2451834493 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b167a7e1b0 msgr2=0x55b167e388f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.135+0000 7f38f3e00640 1 --2- 172.21.15.16:0/2451834493 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b167a7e1b0 0x55b167e388f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.135+0000 7f38f3e00640 1 -- 172.21.15.16:0/2451834493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b167a552f0 msgr2=0x55b167e2b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.135+0000 7f38f3e00640 1 --2- 172.21.15.16:0/2451834493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b167a552f0 0x55b167e2b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.135+0000 7f38f3e00640 1 -- 172.21.15.16:0/2451834493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b167e38e60 con 0x55b167a5d4d0 2024-08-31T06:18:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.136+0000 7f38f3e00640 1 --2- 172.21.15.16:0/2451834493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b167a5d4d0 0x55b167e2b680 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f38e4009d20 tx=0x7f38e4019c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=378d46ea5a213b91 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.136+0000 7f38de000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:18:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.136+0000 7f38de000640 1 -- 172.21.15.16:0/2451834493 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e401e040 con 0x55b167a5d4d0 2024-08-31T06:18:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.136+0000 7f38de000640 1 -- 172.21.15.16:0/2451834493 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8470+0+0 (secure 0 0 0) 0x7f38e4003bc0 con 0x55b167a5d4d0 2024-08-31T06:18:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.136+0000 7f3902015740 1 -- 172.21.15.16:0/2451834493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38e401f040 con 0x55b167a5d4d0 2024-08-31T06:18:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.136+0000 7f38de000640 1 -- 172.21.15.16:0/2451834493 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f38e400e3b0 con 0x55b167a5d4d0 2024-08-31T06:18:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.136+0000 7f3902015740 10 client.17748 fetch_fsmap learned FSMap version 46 2024-08-31T06:18:20.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.136+0000 7f3902015740 1 -- 172.21.15.16:0/2451834493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55b167e3a0d0 con 0x55b167a5d4d0 2024-08-31T06:18:20.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.136+0000 7f38de000640 1 -- 172.21.15.16:0/2451834493 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38e401f040 con 0x55b167a5d4d0 2024-08-31T06:18:20.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.137+0000 7f38de000640 1 -- 172.21.15.16:0/2451834493 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 46) ==== 8358+0+0 (secure 0 0 0) 0x7f38e4037030 con 0x55b167a5d4d0 2024-08-31T06:18:20.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.137+0000 7f3902015740 10 client.17748 fetch_fsmap finished waiting for FSMap version 46 2024-08-31T06:18:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.137+0000 7f3902015740 10 client.17748 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:18:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.137+0000 7f3902015740 20 client.17748 populate_metadata read hostname 'smithi016' 2024-08-31T06:18:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.137+0000 7f3902015740 1 --2- 172.21.15.16:0/2451834493 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55b167a121c0 0x55b167eb4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.137+0000 7f3902015740 4 client.17748 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:18:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.137+0000 7f3902015740 1 -- 172.21.15.16:0/2451834493 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b167e1fc20 con 0x55b167a121c0 2024-08-31T06:18:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.137+0000 7f38f3400640 1 --2- 172.21.15.16:0/2451834493 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55b167a121c0 0x55b167eb4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.138+0000 7f38f3400640 1 --2- 172.21.15.16:0/2451834493 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55b167a121c0 0x55b167eb4630 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=863e6c3f3f71dc74 server_cookie=d60811f8ff8b0902 in_seq=0 out_seq=0 2024-08-31T06:18:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.138+0000 7f38de000640 10 client.17748 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:18:20.158 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.155+0000 7f38de000640 1 -- 172.21.15.16:0/2451834493 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b167e1fc20 con 0x55b167a121c0 2024-08-31T06:18:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.155+0000 7f38de000640 10 client.17748 handle_command_reply: tid=0 2024-08-31T06:18:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.155+0000 7f3902015740 1 client.17748 shutdown 2024-08-31T06:18:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.156+0000 7f3902015740 1 -- 172.21.15.16:0/2451834493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b167a5d4d0 msgr2=0x55b167e2b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.156+0000 7f3902015740 1 --2- 172.21.15.16:0/2451834493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b167a5d4d0 0x55b167e2b680 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f38e4009d20 tx=0x7f38e4019c40 comp rx=0 tx=0).stop 2024-08-31T06:18:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.156+0000 7f3902015740 1 -- 172.21.15.16:0/2451834493 shutdown_connections 2024-08-31T06:18:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.156+0000 7f3902015740 1 --2- 172.21.15.16:0/2451834493 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55b167a121c0 0x55b167eb4630 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.156+0000 7f3902015740 1 --2- 172.21.15.16:0/2451834493 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b167a7e1b0 0x55b167e388f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.156+0000 7f3902015740 1 --2- 172.21.15.16:0/2451834493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b167a5d4d0 0x55b167e2b680 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.156+0000 7f38de000640 0 client.17748 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:18:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.156+0000 7f3902015740 1 --2- 172.21.15.16:0/2451834493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b167a552f0 0x55b167e2b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.156+0000 7f3902015740 1 -- 172.21.15.16:0/2451834493 >> 172.21.15.16:0/2451834493 conn(0x55b167a72300 msgr2=0x55b167e33110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.156+0000 7f3902015740 1 -- 172.21.15.16:0/2451834493 shutdown_connections 2024-08-31T06:18:20.160 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.156+0000 7f3902015740 1 -- 172.21.15.16:0/2451834493 wait complete. 2024-08-31T06:18:20.160 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.156+0000 7f3902015740 20 client.17748 trim_cache size 0 max 16384 2024-08-31T06:18:20.160 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:18:20.160 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:18:20.160 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:18:20.160 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:18:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.157+0000 7f38f9a00640 1 -- 172.21.15.16:0/2773959517 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f38d80b4d20 msgr2=0x7f38d80b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.157+0000 7f38f9a00640 1 --2- 172.21.15.16:0/2773959517 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f38d80b4d20 0x7f38d80b7230 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f38ec0d0560 tx=0x7f38ec005df0 comp rx=0 tx=0).stop 2024-08-31T06:18:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.157+0000 7f38f9a00640 1 -- 172.21.15.16:0/2773959517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38f4109930 msgr2=0x7f38f419f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.157+0000 7f38f9a00640 1 --2- 172.21.15.16:0/2773959517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38f4109930 0x7f38f419f8a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f38ec02f540 tx=0x7f38ec0316f0 comp rx=0 tx=0).stop 2024-08-31T06:18:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.157+0000 7f38f9a00640 1 -- 172.21.15.16:0/2773959517 shutdown_connections 2024-08-31T06:18:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.157+0000 7f38f9a00640 1 --2- 172.21.15.16:0/2773959517 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f38d80b4d20 0x7f38d80b7230 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.157+0000 7f38f9a00640 1 --2- 172.21.15.16:0/2773959517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38f4108fe0 0x7f38f419a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.157+0000 7f38f9a00640 1 --2- 172.21.15.16:0/2773959517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38f4109930 0x7f38f419f8a0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.157+0000 7f38f9a00640 1 --2- 172.21.15.16:0/2773959517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38f41036b0 0x7f38f419a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.157+0000 7f38f9a00640 1 -- 172.21.15.16:0/2773959517 >> 172.21.15.16:0/2773959517 conn(0x7f38f40ff0d0 msgr2=0x7f38f4108630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.157+0000 7f38f9a00640 1 -- 172.21.15.16:0/2773959517 shutdown_connections 2024-08-31T06:18:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:20.157+0000 7f38f9a00640 1 -- 172.21.15.16:0/2773959517 wait complete. 2024-08-31T06:18:20.471 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T06:18:20.471 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:18:20.890 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:18:20.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:20 smithi016 ceph-mon[17194]: pgmap v709: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 1.43k op/s 2024-08-31T06:18:20.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:20 smithi184 ceph-mon[22331]: pgmap v709: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 1.43k op/s 2024-08-31T06:18:21.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:20 smithi155 ceph-mon[21742]: pgmap v709: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 1.43k op/s 2024-08-31T06:18:21.629 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 655.0 s, 170.3 tps, lat 80.154 ms stddev 90.293 2024-08-31T06:18:21.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.675+0000 7f1c1b200640 1 -- 172.21.15.16:0/3069310212 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c1c109090 msgr2=0x7f1c1c1094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:21.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.675+0000 7f1c1b200640 1 --2- 172.21.15.16:0/3069310212 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c1c109090 0x7f1c1c1094a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1c08009900 tx=0x7f1c0802a400 comp rx=0 tx=0).stop 2024-08-31T06:18:21.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.675+0000 7f1c1b200640 1 -- 172.21.15.16:0/3069310212 shutdown_connections 2024-08-31T06:18:21.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.675+0000 7f1c1b200640 1 --2- 172.21.15.16:0/3069310212 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c1c109090 0x7f1c1c1094a0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:21.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.675+0000 7f1c1b200640 1 --2- 172.21.15.16:0/3069310212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c1c1099e0 0x7f1c1c110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:21.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.675+0000 7f1c1b200640 1 --2- 172.21.15.16:0/3069310212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c1c103750 0x7f1c1c103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:21.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.675+0000 7f1c1b200640 1 -- 172.21.15.16:0/3069310212 >> 172.21.15.16:0/3069310212 conn(0x7f1c1c0ff1a0 msgr2=0x7f1c1c1015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:21.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.675+0000 7f1c1b200640 1 -- 172.21.15.16:0/3069310212 shutdown_connections 2024-08-31T06:18:21.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.675+0000 7f1c1b200640 1 -- 172.21.15.16:0/3069310212 wait complete. 2024-08-31T06:18:21.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.676+0000 7f1c1b200640 1 Processor -- start 2024-08-31T06:18:21.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.676+0000 7f1c1b200640 1 -- start start 2024-08-31T06:18:21.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.677+0000 7f1c1b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c1c103750 0x7f1c1c0754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:21.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.677+0000 7f1c1b200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c1c109090 0x7f1c1c0759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:21.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.677+0000 7f1c1b200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c1c1099e0 0x7f1c1c075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:21.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.677+0000 7f1c1b200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c113650 con 0x7f1c1c103750 2024-08-31T06:18:21.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.677+0000 7f1c1b200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c1134d0 con 0x7f1c1c1099e0 2024-08-31T06:18:21.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.677+0000 7f1c1b200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c1137d0 con 0x7f1c1c109090 2024-08-31T06:18:21.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.677+0000 7f1c19e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c1c103750 0x7f1c1c0754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:21.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.677+0000 7f1c19400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c1c109090 0x7f1c1c0759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:21.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.677+0000 7f1c1a800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c1c1099e0 0x7f1c1c075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:21.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.677+0000 7f1c1a800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c1c1099e0 0x7f1c1c075f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:33208/0 (socket says 172.21.15.16:33208) 2024-08-31T06:18:21.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.677+0000 7f1c19400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c1c109090 0x7f1c1c0759f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:36558/0 (socket says 172.21.15.16:36558) 2024-08-31T06:18:21.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.677+0000 7f1c1a800640 1 -- 172.21.15.16:0/2311317825 learned_addr learned my addr 172.21.15.16:0/2311317825 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:21.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.678+0000 7f1c19400640 1 -- 172.21.15.16:0/2311317825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c1c1099e0 msgr2=0x7f1c1c075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:21.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.678+0000 7f1c19400640 1 --2- 172.21.15.16:0/2311317825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c1c1099e0 0x7f1c1c075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:21.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.678+0000 7f1c19400640 1 -- 172.21.15.16:0/2311317825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c1c103750 msgr2=0x7f1c1c0754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:21.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.678+0000 7f1c19400640 1 --2- 172.21.15.16:0/2311317825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c1c103750 0x7f1c1c0754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:21.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.678+0000 7f1c19400640 1 -- 172.21.15.16:0/2311317825 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c1c1a8340 con 0x7f1c1c109090 2024-08-31T06:18:21.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.678+0000 7f1c19e00640 1 --2- 172.21.15.16:0/2311317825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c1c103750 0x7f1c1c0754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:18:21.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.678+0000 7f1c19400640 1 --2- 172.21.15.16:0/2311317825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c1c109090 0x7f1c1c0759f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1c0802a920 tx=0x7f1c08031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:21.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.678+0000 7f1c12a00640 1 -- 172.21.15.16:0/2311317825 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0803f070 con 0x7f1c1c109090 2024-08-31T06:18:21.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.678+0000 7f1c1b200640 1 -- 172.21.15.16:0/2311317825 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c1c1a85d0 con 0x7f1c1c109090 2024-08-31T06:18:21.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.678+0000 7f1c12a00640 1 -- 172.21.15.16:0/2311317825 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c08004490 con 0x7f1c1c109090 2024-08-31T06:18:21.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.678+0000 7f1c12a00640 1 -- 172.21.15.16:0/2311317825 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c08035070 con 0x7f1c1c109090 2024-08-31T06:18:21.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.679+0000 7f1c1b200640 1 -- 172.21.15.16:0/2311317825 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c1c1a8b00 con 0x7f1c1c109090 2024-08-31T06:18:21.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.680+0000 7f1c1b200640 1 -- 172.21.15.16:0/2311317825 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bfc0052c0 con 0x7f1c1c109090 2024-08-31T06:18:21.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.682+0000 7f1c12a00640 1 -- 172.21.15.16:0/2311317825 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f1c0803b460 con 0x7f1c1c109090 2024-08-31T06:18:21.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.683+0000 7f1c12a00640 1 --2- 172.21.15.16:0/2311317825 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f1c000b4ca0 0x7f1c000b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:21.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.684+0000 7f1c12a00640 1 -- 172.21.15.16:0/2311317825 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8470+0+0 (secure 0 0 0) 0x7f1c08105990 con 0x7f1c1c109090 2024-08-31T06:18:21.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.684+0000 7f1c19e00640 1 --2- 172.21.15.16:0/2311317825 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f1c000b4ca0 0x7f1c000b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:21.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.684+0000 7f1c19e00640 1 --2- 172.21.15.16:0/2311317825 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f1c000b4ca0 0x7f1c000b71b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c076f00 tx=0x7f1c0c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:21.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.686+0000 7f1c12a00640 1 -- 172.21.15.16:0/2311317825 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f1c080cc750 con 0x7f1c1c109090 2024-08-31T06:18:21.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.917+0000 7f1c1b200640 1 -- 172.21.15.16:0/2311317825 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1bfc0055b0 con 0x7f1c1c109090 2024-08-31T06:18:21.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.918+0000 7f1c12a00640 1 -- 172.21.15.16:0/2311317825 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11921 (secure 0 0 0) 0x7f1c080cc050 con 0x7f1c1c109090 2024-08-31T06:18:21.921 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:18:21.921 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":46,"btime":"2024-08-31T06:17:38:653550+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:17:38.342854+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24712,24724,24737]},"id":1}]} 2024-08-31T06:18:21.921 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 46 2024-08-31T06:18:21.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.920+0000 7f1c1b200640 1 -- 172.21.15.16:0/2311317825 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f1c000b4ca0 msgr2=0x7f1c000b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:21.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.920+0000 7f1c1b200640 1 --2- 172.21.15.16:0/2311317825 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f1c000b4ca0 0x7f1c000b71b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c076f00 tx=0x7f1c0c005f90 comp rx=0 tx=0).stop 2024-08-31T06:18:21.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.920+0000 7f1c1b200640 1 -- 172.21.15.16:0/2311317825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c1c109090 msgr2=0x7f1c1c0759f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:21.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.920+0000 7f1c1b200640 1 --2- 172.21.15.16:0/2311317825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c1c109090 0x7f1c1c0759f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1c0802a920 tx=0x7f1c08031a20 comp rx=0 tx=0).stop 2024-08-31T06:18:21.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.920+0000 7f1c1b200640 1 -- 172.21.15.16:0/2311317825 shutdown_connections 2024-08-31T06:18:21.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.920+0000 7f1c1b200640 1 --2- 172.21.15.16:0/2311317825 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f1c000b4ca0 0x7f1c000b71b0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:21.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.921+0000 7f1c1b200640 1 --2- 172.21.15.16:0/2311317825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c1c109090 0x7f1c1c0759f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:21.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.921+0000 7f1c1b200640 1 --2- 172.21.15.16:0/2311317825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c1c1099e0 0x7f1c1c075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:21.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.921+0000 7f1c1b200640 1 --2- 172.21.15.16:0/2311317825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c1c103750 0x7f1c1c0754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:21.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.921+0000 7f1c1b200640 1 -- 172.21.15.16:0/2311317825 >> 172.21.15.16:0/2311317825 conn(0x7f1c1c0ff1a0 msgr2=0x7f1c1c1015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:21.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.921+0000 7f1c1b200640 1 -- 172.21.15.16:0/2311317825 shutdown_connections 2024-08-31T06:18:21.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:21.921+0000 7f1c1b200640 1 -- 172.21.15.16:0/2311317825 wait complete. 2024-08-31T06:18:22.802 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T06:18:22.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:22 smithi016 ceph-mon[17194]: pgmap v710: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 1.37k op/s 2024-08-31T06:18:22.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:22 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/2311317825' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:18:22.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:22 smithi184 ceph-mon[22331]: pgmap v710: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 1.37k op/s 2024-08-31T06:18:22.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:22 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/2311317825' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:18:23.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:22 smithi155 ceph-mon[21742]: pgmap v710: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 1.37k op/s 2024-08-31T06:18:23.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:22 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/2311317825' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:18:23.184 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:18:23.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.870+0000 7f5f4ac00640 1 -- 172.21.15.16:0/356094060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f44109070 msgr2=0x7f5f44109480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:23.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.870+0000 7f5f4ac00640 1 --2- 172.21.15.16:0/356094060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f44109070 0x7f5f44109480 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f5f30009900 tx=0x7f5f3002a410 comp rx=0 tx=0).stop 2024-08-31T06:18:23.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.870+0000 7f5f4ac00640 1 -- 172.21.15.16:0/356094060 shutdown_connections 2024-08-31T06:18:23.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.870+0000 7f5f4ac00640 1 --2- 172.21.15.16:0/356094060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f441099c0 0x7f5f441101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:23.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.870+0000 7f5f4ac00640 1 --2- 172.21.15.16:0/356094060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f44109070 0x7f5f44109480 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:23.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.870+0000 7f5f4ac00640 1 --2- 172.21.15.16:0/356094060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f44103730 0x7f5f44103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:23.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.870+0000 7f5f4ac00640 1 -- 172.21.15.16:0/356094060 >> 172.21.15.16:0/356094060 conn(0x7f5f440ff180 msgr2=0x7f5f441015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:23.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.870+0000 7f5f4ac00640 1 -- 172.21.15.16:0/356094060 shutdown_connections 2024-08-31T06:18:23.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.870+0000 7f5f4ac00640 1 -- 172.21.15.16:0/356094060 wait complete. 2024-08-31T06:18:23.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.871+0000 7f5f4ac00640 1 Processor -- start 2024-08-31T06:18:23.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.872+0000 7f5f4ac00640 1 -- start start 2024-08-31T06:18:23.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.872+0000 7f5f4ac00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f44103730 0x7f5f440754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:23.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.872+0000 7f5f4ac00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f44109070 0x7f5f440759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:23.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.872+0000 7f5f4ac00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f441099c0 0x7f5f44075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:23.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.872+0000 7f5f4ac00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5f44113650 con 0x7f5f44103730 2024-08-31T06:18:23.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.872+0000 7f5f4ac00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5f441134d0 con 0x7f5f44109070 2024-08-31T06:18:23.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.872+0000 7f5f4ac00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5f441137d0 con 0x7f5f441099c0 2024-08-31T06:18:23.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.872+0000 7f5f48e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f44109070 0x7f5f440759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:23.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.872+0000 7f5f49800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f44103730 0x7f5f440754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:23.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.872+0000 7f5f4a200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f441099c0 0x7f5f44075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:23.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.872+0000 7f5f48e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f44109070 0x7f5f440759f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:33234/0 (socket says 172.21.15.16:33234) 2024-08-31T06:18:23.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.872+0000 7f5f48e00640 1 -- 172.21.15.16:0/1880578437 learned_addr learned my addr 172.21.15.16:0/1880578437 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:23.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.873+0000 7f5f4a200640 1 -- 172.21.15.16:0/1880578437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f44109070 msgr2=0x7f5f440759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:23.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.873+0000 7f5f4a200640 1 --2- 172.21.15.16:0/1880578437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f44109070 0x7f5f440759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:23.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.873+0000 7f5f4a200640 1 -- 172.21.15.16:0/1880578437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f44103730 msgr2=0x7f5f440754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:23.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.873+0000 7f5f4a200640 1 --2- 172.21.15.16:0/1880578437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f44103730 0x7f5f440754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:23.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.873+0000 7f5f4a200640 1 -- 172.21.15.16:0/1880578437 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f441a8360 con 0x7f5f441099c0 2024-08-31T06:18:23.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.873+0000 7f5f4a200640 1 --2- 172.21.15.16:0/1880578437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f441099c0 0x7f5f44075f30 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c009fd0 tx=0x7f5f3c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:23.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.873+0000 7f5f42000640 1 -- 172.21.15.16:0/1880578437 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c00ce20 con 0x7f5f441099c0 2024-08-31T06:18:23.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.873+0000 7f5f42000640 1 -- 172.21.15.16:0/1880578437 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f3c005020 con 0x7f5f441099c0 2024-08-31T06:18:23.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.873+0000 7f5f4ac00640 1 -- 172.21.15.16:0/1880578437 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f441a8650 con 0x7f5f441099c0 2024-08-31T06:18:23.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.874+0000 7f5f42000640 1 -- 172.21.15.16:0/1880578437 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c010690 con 0x7f5f441099c0 2024-08-31T06:18:23.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.874+0000 7f5f4ac00640 1 -- 172.21.15.16:0/1880578437 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f441a8b90 con 0x7f5f441099c0 2024-08-31T06:18:23.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.877+0000 7f5f52e63740 1 Processor -- start 2024-08-31T06:18:23.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.878+0000 7f5f42000640 1 -- 172.21.15.16:0/1880578437 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f5f3c0054d0 con 0x7f5f441099c0 2024-08-31T06:18:23.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.878+0000 7f5f52e63740 1 -- start start 2024-08-31T06:18:23.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.878+0000 7f5f52e63740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f037823070 0x55f037bc8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:23.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.878+0000 7f5f52e63740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0377fa1b0 0x55f037bc8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:23.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.878+0000 7f5f52e63740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f037802390 0x55f037bc94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:23.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.878+0000 7f5f52e63740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f037b3f7b0 con 0x55f037823070 2024-08-31T06:18:23.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.878+0000 7f5f52e63740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f037b2ed40 con 0x55f037802390 2024-08-31T06:18:23.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.878+0000 7f5f52e63740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f037b32cf0 con 0x55f0377fa1b0 2024-08-31T06:18:23.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.879+0000 7f5f48e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f037823070 0x55f037bc8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:23.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.879+0000 7f5f49800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0377fa1b0 0x55f037bc8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:23.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.879+0000 7f5f48e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f037823070 0x55f037bc8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57254/0 (socket says 172.21.15.16:57254) 2024-08-31T06:18:23.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.879+0000 7f5f48e00640 1 -- 172.21.15.16:0/722167164 learned_addr learned my addr 172.21.15.16:0/722167164 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:23.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.879+0000 7f5f48e00640 1 --2- 172.21.15.16:0/722167164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f037802390 0x55f037bc94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:23.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.879+0000 7f5f49800640 1 -- 172.21.15.16:0/722167164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f037802390 msgr2=0x55f037bc94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:23.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.879+0000 7f5f49800640 1 --2- 172.21.15.16:0/722167164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f037802390 0x55f037bc94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:23.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.879+0000 7f5f49800640 1 -- 172.21.15.16:0/722167164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f037823070 msgr2=0x55f037bc8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:23.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.879+0000 7f5f49800640 1 --2- 172.21.15.16:0/722167164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f037823070 0x55f037bc8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:23.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.879+0000 7f5f49800640 1 -- 172.21.15.16:0/722167164 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f03788e650 con 0x55f0377fa1b0 2024-08-31T06:18:23.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.879+0000 7f5f48e00640 1 --2- 172.21.15.16:0/722167164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f037823070 0x55f037bc8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:18:23.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.879+0000 7f5f42000640 1 --2- 172.21.15.16:0/1880578437 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f5f2c0b4ca0 0x7f5f2c0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:23.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.879+0000 7f5f49800640 1 --2- 172.21.15.16:0/722167164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0377fa1b0 0x55f037bc8fb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5f38005d00 tx=0x7f5f380107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7de2cf4973d8a17 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:23.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.880+0000 7f5f42000640 1 -- 172.21.15.16:0/1880578437 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8470+0+0 (secure 0 0 0) 0x7f5f3c0e5b30 con 0x7f5f441099c0 2024-08-31T06:18:23.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.880+0000 7f5f36a00640 1 -- 172.21.15.16:0/722167164 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3801c680 con 0x55f0377fa1b0 2024-08-31T06:18:23.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.880+0000 7f5f48e00640 1 --2- 172.21.15.16:0/1880578437 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f5f2c0b4ca0 0x7f5f2c0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:23.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.880+0000 7f5f36a00640 1 -- 172.21.15.16:0/722167164 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f3800be20 con 0x55f0377fa1b0 2024-08-31T06:18:23.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.880+0000 7f5f36a00640 1 -- 172.21.15.16:0/722167164 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f38011ba0 con 0x55f0377fa1b0 2024-08-31T06:18:23.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.881+0000 7f5f52e63740 1 -- 172.21.15.16:0/722167164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0377fa1b0 msgr2=0x55f037bc8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:23.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.881+0000 7f5f52e63740 1 --2- 172.21.15.16:0/722167164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0377fa1b0 0x55f037bc8fb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5f38005d00 tx=0x7f5f380107a0 comp rx=0 tx=0).stop 2024-08-31T06:18:23.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.881+0000 7f5f48e00640 1 --2- 172.21.15.16:0/1880578437 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f5f2c0b4ca0 0x7f5f2c0b71b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5f30005f70 tx=0x7f5f30005900 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:23.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.881+0000 7f5f52e63740 1 -- 172.21.15.16:0/722167164 shutdown_connections 2024-08-31T06:18:23.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.881+0000 7f5f52e63740 1 --2- 172.21.15.16:0/722167164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0377fa1b0 0x55f037bc8fb0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:23.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.881+0000 7f5f52e63740 1 --2- 172.21.15.16:0/722167164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f037802390 0x55f037bc94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:23.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.881+0000 7f5f52e63740 1 --2- 172.21.15.16:0/722167164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f037823070 0x55f037bc8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:23.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.881+0000 7f5f52e63740 1 -- 172.21.15.16:0/722167164 >> 172.21.15.16:0/722167164 conn(0x55f037817460 msgr2=0x55f037bc6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:23.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.881+0000 7f5f52e63740 1 -- 172.21.15.16:0/722167164 shutdown_connections 2024-08-31T06:18:23.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.881+0000 7f5f52e63740 1 -- 172.21.15.16:0/722167164 wait complete. 2024-08-31T06:18:23.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.882+0000 7f5f52e63740 1 -- start start 2024-08-31T06:18:23.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.882+0000 7f5f52e63740 1 Processor -- start 2024-08-31T06:18:23.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.882+0000 7f5f52e63740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0377fa1b0 0x55f037bd46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:23.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.882+0000 7f5f52e63740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f037802390 0x55f037bd4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:23.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.882+0000 7f5f52e63740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f037823070 0x55f037be0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:23.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.882+0000 7f5f52e63740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f037991700 con 0x55f037823070 2024-08-31T06:18:23.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.882+0000 7f5f52e63740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f037ab5cf0 con 0x55f037802390 2024-08-31T06:18:23.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.882+0000 7f5f52e63740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f037bd7250 con 0x55f0377fa1b0 2024-08-31T06:18:23.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.883+0000 7f5f52e63740 10 client.0 _ll_register_callbacks cb 0x55f037adb470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:18:23.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.883+0000 7f5f4a200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f037823070 0x55f037be0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:23.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.883+0000 7f5f49800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0377fa1b0 0x55f037bd46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:23.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.883+0000 7f5f4a200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f037823070 0x55f037be0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57260/0 (socket says 172.21.15.16:57260) 2024-08-31T06:18:23.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.883+0000 7f5f4a200640 1 -- 172.21.15.16:0/689846612 learned_addr learned my addr 172.21.15.16:0/689846612 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:23.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.883+0000 7f5f48e00640 1 --2- 172.21.15.16:0/689846612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f037802390 0x55f037bd4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:23.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.883+0000 7f5f49800640 1 -- 172.21.15.16:0/689846612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f037802390 msgr2=0x55f037bd4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:23.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.883+0000 7f5f49800640 1 --2- 172.21.15.16:0/689846612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f037802390 0x55f037bd4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:23.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.883+0000 7f5f49800640 1 -- 172.21.15.16:0/689846612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f037823070 msgr2=0x55f037be0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:23.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.883+0000 7f5f49800640 1 --2- 172.21.15.16:0/689846612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f037823070 0x55f037be0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:23.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.883+0000 7f5f49800640 1 -- 172.21.15.16:0/689846612 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f037af68b0 con 0x55f0377fa1b0 2024-08-31T06:18:23.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.883+0000 7f5f4a200640 1 --2- 172.21.15.16:0/689846612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f037823070 0x55f037be0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:18:23.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.884+0000 7f5f49800640 1 --2- 172.21.15.16:0/689846612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0377fa1b0 0x55f037bd46c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x55f037bc76f0 tx=0x7f5f380117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1efcc323ccec35d8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:23.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.884+0000 7f5f2b400640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:18:23.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.884+0000 7f5f2b400640 1 -- 172.21.15.16:0/689846612 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3802a060 con 0x55f0377fa1b0 2024-08-31T06:18:23.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.884+0000 7f5f52e63740 1 -- 172.21.15.16:0/689846612 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f3800be20 con 0x55f0377fa1b0 2024-08-31T06:18:23.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.884+0000 7f5f2b400640 1 -- 172.21.15.16:0/689846612 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8470+0+0 (secure 0 0 0) 0x7f5f38005040 con 0x55f0377fa1b0 2024-08-31T06:18:23.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.884+0000 7f5f2b400640 1 -- 172.21.15.16:0/689846612 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f5f38005240 con 0x55f0377fa1b0 2024-08-31T06:18:23.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.885+0000 7f5f52e63740 10 client.28135 fetch_fsmap learned FSMap version 47 2024-08-31T06:18:23.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.885+0000 7f5f52e63740 1 -- 172.21.15.16:0/689846612 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55f037be32e0 con 0x55f0377fa1b0 2024-08-31T06:18:23.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.885+0000 7f5f2b400640 1 -- 172.21.15.16:0/689846612 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f3800be20 con 0x55f0377fa1b0 2024-08-31T06:18:23.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.885+0000 7f5f2b400640 1 -- 172.21.15.16:0/689846612 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 47) ==== 8354+0+0 (secure 0 0 0) 0x7f5f3801c480 con 0x55f0377fa1b0 2024-08-31T06:18:23.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.885+0000 7f5f52e63740 10 client.28135 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T06:18:23.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.885+0000 7f5f52e63740 10 client.28135 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T06:18:23.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.885+0000 7f5f52e63740 20 client.28135 populate_metadata read hostname 'smithi016' 2024-08-31T06:18:23.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.885+0000 7f5f52e63740 1 --2- 172.21.15.16:0/689846612 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55f0377e19f0 0x55f037be66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:23.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.885+0000 7f5f52e63740 4 client.28135 mds_command: new command op to 24712 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:18:23.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.885+0000 7f5f52e63740 1 -- 172.21.15.16:0/689846612 --> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f037b41700 con 0x55f0377e19f0 2024-08-31T06:18:23.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.886+0000 7f5f48e00640 1 --2- 172.21.15.16:0/689846612 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55f0377e19f0 0x55f037be66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:23.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.886+0000 7f5f48e00640 1 --2- 172.21.15.16:0/689846612 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55f0377e19f0 0x55f037be66a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c54d285b03f7962d server_cookie=c0ae00dc3cc029b9 in_seq=0 out_seq=0 2024-08-31T06:18:23.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.886+0000 7f5f2b400640 10 client.28135 ms_handle_connect on v2:172.21.15.184:6838/4011051479 2024-08-31T06:18:23.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.887+0000 7f5f2b400640 1 -- 172.21.15.16:0/689846612 <== mds.2 v2:172.21.15.184:6838/4011051479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f037b41700 con 0x55f0377e19f0 2024-08-31T06:18:23.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.887+0000 7f5f2b400640 10 client.28135 handle_command_reply: tid=0 2024-08-31T06:18:23.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.887+0000 7f5f52e63740 1 client.28135 shutdown 2024-08-31T06:18:23.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.887+0000 7f5f52e63740 1 -- 172.21.15.16:0/689846612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0377fa1b0 msgr2=0x55f037bd46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:23.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.887+0000 7f5f52e63740 1 --2- 172.21.15.16:0/689846612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0377fa1b0 0x55f037bd46c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x55f037bc76f0 tx=0x7f5f380117b0 comp rx=0 tx=0).stop 2024-08-31T06:18:23.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.887+0000 7f5f52e63740 1 -- 172.21.15.16:0/689846612 shutdown_connections 2024-08-31T06:18:23.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.887+0000 7f5f52e63740 1 --2- 172.21.15.16:0/689846612 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55f0377e19f0 0x55f037be66a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:23.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.887+0000 7f5f52e63740 1 --2- 172.21.15.16:0/689846612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f037823070 0x55f037be0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:23.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.887+0000 7f5f52e63740 1 --2- 172.21.15.16:0/689846612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f037802390 0x55f037bd4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:23.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.887+0000 7f5f52e63740 1 --2- 172.21.15.16:0/689846612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0377fa1b0 0x55f037bd46c0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:23.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.887+0000 7f5f52e63740 1 -- 172.21.15.16:0/689846612 >> 172.21.15.16:0/689846612 conn(0x55f037817460 msgr2=0x55f037bcbc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:23.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.887+0000 7f5f2b400640 0 client.28135 ms_handle_reset on v2:172.21.15.184:6838/4011051479 2024-08-31T06:18:23.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.888+0000 7f5f52e63740 1 -- 172.21.15.16:0/689846612 shutdown_connections 2024-08-31T06:18:23.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.888+0000 7f5f52e63740 1 -- 172.21.15.16:0/689846612 wait complete. 2024-08-31T06:18:23.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.888+0000 7f5f52e63740 20 client.28135 trim_cache size 0 max 16384 2024-08-31T06:18:23.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.902+0000 7f5f52e63740 1 Processor -- start 2024-08-31T06:18:23.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.902+0000 7f5f52e63740 1 -- start start 2024-08-31T06:18:23.905 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:23 smithi016 ceph-mon[17194]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:18:23.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.903+0000 7f5f52e63740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0377fa1b0 0x55f037bcfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:23.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.903+0000 7f5f52e63740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f037802390 0x55f037bca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:23.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.903+0000 7f5f52e63740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f037823070 0x55f037bc5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:23.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.903+0000 7f5f52e63740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5f441b3d90 con 0x55f0377fa1b0 2024-08-31T06:18:23.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.903+0000 7f5f52e63740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5f441b1600 con 0x55f037823070 2024-08-31T06:18:23.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.903+0000 7f5f52e63740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5f441af470 con 0x55f037802390 2024-08-31T06:18:23.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.903+0000 7f5f48e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f037802390 0x55f037bca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:23.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.903+0000 7f5f49800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0377fa1b0 0x55f037bcfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:23.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.903+0000 7f5f48e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f037802390 0x55f037bca1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:36606/0 (socket says 172.21.15.16:36606) 2024-08-31T06:18:23.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.903+0000 7f5f48e00640 1 -- 172.21.15.16:0/459032213 learned_addr learned my addr 172.21.15.16:0/459032213 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:23.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.903+0000 7f5f4a200640 1 --2- 172.21.15.16:0/459032213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f037823070 0x55f037bc5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:23.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.903+0000 7f5f48e00640 1 -- 172.21.15.16:0/459032213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f037823070 msgr2=0x55f037bc5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:23.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.903+0000 7f5f48e00640 1 --2- 172.21.15.16:0/459032213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f037823070 0x55f037bc5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:23.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.903+0000 7f5f48e00640 1 -- 172.21.15.16:0/459032213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0377fa1b0 msgr2=0x55f037bcfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:23.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.903+0000 7f5f48e00640 1 --2- 172.21.15.16:0/459032213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0377fa1b0 0x55f037bcfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:23.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.904+0000 7f5f48e00640 1 -- 172.21.15.16:0/459032213 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f037bcb8d0 con 0x55f037802390 2024-08-31T06:18:23.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.904+0000 7f5f48e00640 1 --2- 172.21.15.16:0/459032213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f037802390 0x55f037bca1b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5f30006010 tx=0x7f5f3003b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cca51cfda5a7be7 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:23.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.904+0000 7f5f36000640 1 -- 172.21.15.16:0/459032213 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f30035040 con 0x55f037802390 2024-08-31T06:18:23.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.904+0000 7f5f36000640 1 -- 172.21.15.16:0/459032213 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f30004480 con 0x55f037802390 2024-08-31T06:18:23.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.904+0000 7f5f36000640 1 -- 172.21.15.16:0/459032213 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3003b920 con 0x55f037802390 2024-08-31T06:18:23.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.905+0000 7f5f52e63740 1 -- 172.21.15.16:0/459032213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f037802390 msgr2=0x55f037bca1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:23.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.905+0000 7f5f52e63740 1 --2- 172.21.15.16:0/459032213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f037802390 0x55f037bca1b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5f30006010 tx=0x7f5f3003b000 comp rx=0 tx=0).stop 2024-08-31T06:18:23.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.905+0000 7f5f52e63740 1 -- 172.21.15.16:0/459032213 shutdown_connections 2024-08-31T06:18:23.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.905+0000 7f5f52e63740 1 --2- 172.21.15.16:0/459032213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f037802390 0x55f037bca1b0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:23.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.905+0000 7f5f52e63740 1 --2- 172.21.15.16:0/459032213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f037823070 0x55f037bc5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:23.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.905+0000 7f5f52e63740 1 --2- 172.21.15.16:0/459032213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0377fa1b0 0x55f037bcfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:23.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.905+0000 7f5f52e63740 1 -- 172.21.15.16:0/459032213 >> 172.21.15.16:0/459032213 conn(0x55f037817460 msgr2=0x55f037bc9aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:23.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.905+0000 7f5f52e63740 1 -- 172.21.15.16:0/459032213 shutdown_connections 2024-08-31T06:18:23.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.905+0000 7f5f52e63740 1 -- 172.21.15.16:0/459032213 wait complete. 2024-08-31T06:18:23.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.906+0000 7f5f52e63740 1 -- start start 2024-08-31T06:18:23.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.906+0000 7f5f52e63740 1 Processor -- start 2024-08-31T06:18:23.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.907+0000 7f5f52e63740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0377fa1b0 0x55f037be9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:23.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.907+0000 7f5f52e63740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f037802390 0x55f037be72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:23.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.907+0000 7f5f52e63740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f037823070 0x55f037be7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:23.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.907+0000 7f5f52e63740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5f441afac0 con 0x55f037823070 2024-08-31T06:18:23.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.907+0000 7f5f52e63740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5f441b0f30 con 0x55f0377fa1b0 2024-08-31T06:18:23.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.907+0000 7f5f52e63740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5f441b0190 con 0x55f037802390 2024-08-31T06:18:23.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.907+0000 7f5f52e63740 10 client.0 _ll_register_callbacks cb 0x7f5f38011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:18:23.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.907+0000 7f5f48e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f037802390 0x55f037be72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:23.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.907+0000 7f5f49800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0377fa1b0 0x55f037be9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:23.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.907+0000 7f5f48e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f037802390 0x55f037be72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:36608/0 (socket says 172.21.15.16:36608) 2024-08-31T06:18:23.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.907+0000 7f5f48e00640 1 -- 172.21.15.16:0/1010256194 learned_addr learned my addr 172.21.15.16:0/1010256194 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:23.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.907+0000 7f5f4a200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f037823070 0x55f037be7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:23.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.907+0000 7f5f48e00640 1 -- 172.21.15.16:0/1010256194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0377fa1b0 msgr2=0x55f037be9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:23.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.907+0000 7f5f48e00640 1 --2- 172.21.15.16:0/1010256194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0377fa1b0 0x55f037be9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:23.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.907+0000 7f5f48e00640 1 -- 172.21.15.16:0/1010256194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f037823070 msgr2=0x55f037be7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:23.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.907+0000 7f5f48e00640 1 --2- 172.21.15.16:0/1010256194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f037823070 0x55f037be7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:23.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.907+0000 7f5f48e00640 1 -- 172.21.15.16:0/1010256194 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f037be7f00 con 0x55f037802390 2024-08-31T06:18:23.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.907+0000 7f5f4a200640 1 --2- 172.21.15.16:0/1010256194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f037823070 0x55f037be7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:18:23.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.908+0000 7f5f48e00640 1 --2- 172.21.15.16:0/1010256194 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f037802390 0x55f037be72d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5f30042040 tx=0x7f5f30005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bef9fa059f3628d2 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:23.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.908+0000 7f5f2b400640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:18:23.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.908+0000 7f5f2b400640 1 -- 172.21.15.16:0/1010256194 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f30062060 con 0x55f037802390 2024-08-31T06:18:23.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.908+0000 7f5f52e63740 1 -- 172.21.15.16:0/1010256194 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f30004480 con 0x55f037802390 2024-08-31T06:18:23.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.908+0000 7f5f2b400640 1 -- 172.21.15.16:0/1010256194 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8470+0+0 (secure 0 0 0) 0x7f5f3003b8a0 con 0x55f037802390 2024-08-31T06:18:23.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.908+0000 7f5f2b400640 1 -- 172.21.15.16:0/1010256194 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f5f30035040 con 0x55f037802390 2024-08-31T06:18:23.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.908+0000 7f5f52e63740 10 client.28141 fetch_fsmap learned FSMap version 47 2024-08-31T06:18:23.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.909+0000 7f5f52e63740 1 -- 172.21.15.16:0/1010256194 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55f037c50cb0 con 0x55f037802390 2024-08-31T06:18:23.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.909+0000 7f5f2b400640 1 -- 172.21.15.16:0/1010256194 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f30004480 con 0x55f037802390 2024-08-31T06:18:23.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.909+0000 7f5f2b400640 1 -- 172.21.15.16:0/1010256194 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 47) ==== 8354+0+0 (secure 0 0 0) 0x7f5f30053050 con 0x55f037802390 2024-08-31T06:18:23.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.909+0000 7f5f52e63740 10 client.28141 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T06:18:23.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.909+0000 7f5f52e63740 10 client.28141 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T06:18:23.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.909+0000 7f5f52e63740 20 client.28141 populate_metadata read hostname 'smithi016' 2024-08-31T06:18:23.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.909+0000 7f5f52e63740 1 --2- 172.21.15.16:0/1010256194 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55f0377b6da0 0x55f037c53410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:23.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.909+0000 7f5f52e63740 4 client.28141 mds_command: new command op to 24712 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:18:23.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.909+0000 7f5f52e63740 1 -- 172.21.15.16:0/1010256194 --> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f037b3d1a0 con 0x55f0377b6da0 2024-08-31T06:18:23.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.910+0000 7f5f49800640 1 --2- 172.21.15.16:0/1010256194 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55f0377b6da0 0x55f037c53410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:23.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.910+0000 7f5f49800640 1 --2- 172.21.15.16:0/1010256194 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55f0377b6da0 0x55f037c53410 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=17acf874db240ac2 server_cookie=2ac311b3ae72a78c in_seq=0 out_seq=0 2024-08-31T06:18:23.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.910+0000 7f5f2b400640 10 client.28141 ms_handle_connect on v2:172.21.15.184:6838/4011051479 2024-08-31T06:18:23.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.911+0000 7f5f2b400640 1 -- 172.21.15.16:0/1010256194 <== mds.2 v2:172.21.15.184:6838/4011051479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f037b3d1a0 con 0x55f0377b6da0 2024-08-31T06:18:23.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.911+0000 7f5f2b400640 10 client.28141 handle_command_reply: tid=0 2024-08-31T06:18:23.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.911+0000 7f5f52e63740 1 client.28141 shutdown 2024-08-31T06:18:23.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.911+0000 7f5f52e63740 1 -- 172.21.15.16:0/1010256194 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f037802390 msgr2=0x55f037be72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:23.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.911+0000 7f5f52e63740 1 --2- 172.21.15.16:0/1010256194 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f037802390 0x55f037be72d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5f30042040 tx=0x7f5f30005a20 comp rx=0 tx=0).stop 2024-08-31T06:18:23.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.911+0000 7f5f52e63740 1 -- 172.21.15.16:0/1010256194 shutdown_connections 2024-08-31T06:18:23.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.911+0000 7f5f52e63740 1 --2- 172.21.15.16:0/1010256194 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55f0377b6da0 0x55f037c53410 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:23.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.911+0000 7f5f52e63740 1 --2- 172.21.15.16:0/1010256194 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f037802390 0x55f037be72d0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:23.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.911+0000 7f5f52e63740 1 --2- 172.21.15.16:0/1010256194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f037823070 0x55f037be7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:23.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.911+0000 7f5f2b400640 0 client.28141 ms_handle_reset on v2:172.21.15.184:6838/4011051479 2024-08-31T06:18:23.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.911+0000 7f5f52e63740 1 --2- 172.21.15.16:0/1010256194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0377fa1b0 0x55f037be9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:23.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.911+0000 7f5f52e63740 1 -- 172.21.15.16:0/1010256194 >> 172.21.15.16:0/1010256194 conn(0x55f037817460 msgr2=0x55f037bca460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:23.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.911+0000 7f5f52e63740 1 -- 172.21.15.16:0/1010256194 shutdown_connections 2024-08-31T06:18:23.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.911+0000 7f5f52e63740 1 -- 172.21.15.16:0/1010256194 wait complete. 2024-08-31T06:18:23.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.911+0000 7f5f52e63740 20 client.28141 trim_cache size 0 max 16384 2024-08-31T06:18:23.918 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:18:23.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.912+0000 7f5f4ac00640 1 -- 172.21.15.16:0/1880578437 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f5f2c0b4ca0 msgr2=0x7f5f2c0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:23.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.912+0000 7f5f4ac00640 1 --2- 172.21.15.16:0/1880578437 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f5f2c0b4ca0 0x7f5f2c0b71b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5f30005f70 tx=0x7f5f30005900 comp rx=0 tx=0).stop 2024-08-31T06:18:23.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.912+0000 7f5f4ac00640 1 -- 172.21.15.16:0/1880578437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f441099c0 msgr2=0x7f5f44075f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:23.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.912+0000 7f5f4ac00640 1 --2- 172.21.15.16:0/1880578437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f441099c0 0x7f5f44075f30 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c009fd0 tx=0x7f5f3c00ef10 comp rx=0 tx=0).stop 2024-08-31T06:18:23.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.912+0000 7f5f4ac00640 1 -- 172.21.15.16:0/1880578437 shutdown_connections 2024-08-31T06:18:23.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.912+0000 7f5f4ac00640 1 --2- 172.21.15.16:0/1880578437 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f5f2c0b4ca0 0x7f5f2c0b71b0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:23.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.912+0000 7f5f4ac00640 1 --2- 172.21.15.16:0/1880578437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f441099c0 0x7f5f44075f30 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:23.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.912+0000 7f5f4ac00640 1 --2- 172.21.15.16:0/1880578437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f44109070 0x7f5f440759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:23.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.912+0000 7f5f4ac00640 1 --2- 172.21.15.16:0/1880578437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f44103730 0x7f5f440754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:23.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.912+0000 7f5f4ac00640 1 -- 172.21.15.16:0/1880578437 >> 172.21.15.16:0/1880578437 conn(0x7f5f440ff180 msgr2=0x7f5f441015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:23.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.912+0000 7f5f4ac00640 1 -- 172.21.15.16:0/1880578437 shutdown_connections 2024-08-31T06:18:23.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:23.913+0000 7f5f4ac00640 1 -- 172.21.15.16:0/1880578437 wait complete. 2024-08-31T06:18:23.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:23 smithi184 ceph-mon[22331]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:18:24.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:23 smithi155 ceph-mon[21742]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:18:24.802 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T06:18:24.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:24 smithi016 ceph-mon[17194]: pgmap v711: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 1.23k op/s 2024-08-31T06:18:24.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:24 smithi184 ceph-mon[22331]: pgmap v711: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 1.23k op/s 2024-08-31T06:18:25.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:24 smithi155 ceph-mon[21742]: pgmap v711: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 1.23k op/s 2024-08-31T06:18:25.182 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:18:25.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.846+0000 7fd860c00640 1 -- 172.21.15.16:0/1804855956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd85c075550 msgr2=0x7fd85c075960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:25.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.846+0000 7fd860c00640 1 --2- 172.21.15.16:0/1804855956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd85c075550 0x7fd85c075960 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd84c009900 tx=0x7fd84c02a3f0 comp rx=0 tx=0).stop 2024-08-31T06:18:25.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.846+0000 7fd860c00640 1 -- 172.21.15.16:0/1804855956 shutdown_connections 2024-08-31T06:18:25.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.846+0000 7fd860c00640 1 --2- 172.21.15.16:0/1804855956 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd85c075ea0 0x7fd85c10f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:25.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.846+0000 7fd860c00640 1 --2- 172.21.15.16:0/1804855956 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd85c10fe80 0x7fd85c1122c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:25.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.846+0000 7fd860c00640 1 --2- 172.21.15.16:0/1804855956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd85c075550 0x7fd85c075960 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:25.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.846+0000 7fd860c00640 1 -- 172.21.15.16:0/1804855956 >> 172.21.15.16:0/1804855956 conn(0x7fd85c0ff370 msgr2=0x7fd85c1017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:25.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.846+0000 7fd860c00640 1 -- 172.21.15.16:0/1804855956 shutdown_connections 2024-08-31T06:18:25.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.846+0000 7fd860c00640 1 -- 172.21.15.16:0/1804855956 wait complete. 2024-08-31T06:18:25.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.847+0000 7fd860c00640 1 Processor -- start 2024-08-31T06:18:25.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.847+0000 7fd860c00640 1 -- start start 2024-08-31T06:18:25.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.848+0000 7fd860c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd85c075550 0x7fd85c19c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:25.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.848+0000 7fd860c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd85c075ea0 0x7fd85c19c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:25.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.848+0000 7fd860c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd85c10fe80 0x7fd85c1a1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:25.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.848+0000 7fd860c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd85c115a80 con 0x7fd85c10fe80 2024-08-31T06:18:25.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.848+0000 7fd860c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd85c115900 con 0x7fd85c075ea0 2024-08-31T06:18:25.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.848+0000 7fd860c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd85c115c00 con 0x7fd85c075550 2024-08-31T06:18:25.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.848+0000 7fd85a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd85c10fe80 0x7fd85c1a1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:25.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.848+0000 7fd859e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd85c075550 0x7fd85c19c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:25.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.848+0000 7fd85a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd85c10fe80 0x7fd85c1a1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44382/0 (socket says 172.21.15.16:44382) 2024-08-31T06:18:25.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.848+0000 7fd859e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd85c075550 0x7fd85c19c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:51614/0 (socket says 172.21.15.16:51614) 2024-08-31T06:18:25.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.848+0000 7fd85a800640 1 -- 172.21.15.16:0/3372687932 learned_addr learned my addr 172.21.15.16:0/3372687932 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:25.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.848+0000 7fd859400640 1 --2- 172.21.15.16:0/3372687932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd85c075ea0 0x7fd85c19c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:25.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.848+0000 7fd85a800640 1 -- 172.21.15.16:0/3372687932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd85c075550 msgr2=0x7fd85c19c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:25.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.848+0000 7fd85a800640 1 --2- 172.21.15.16:0/3372687932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd85c075550 0x7fd85c19c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:25.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.848+0000 7fd85a800640 1 -- 172.21.15.16:0/3372687932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd85c075ea0 msgr2=0x7fd85c19c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:25.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.848+0000 7fd85a800640 1 --2- 172.21.15.16:0/3372687932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd85c075ea0 0x7fd85c19c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:25.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.849+0000 7fd85a800640 1 -- 172.21.15.16:0/3372687932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd85c1a2360 con 0x7fd85c10fe80 2024-08-31T06:18:25.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.849+0000 7fd859e00640 1 --2- 172.21.15.16:0/3372687932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd85c075550 0x7fd85c19c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:18:25.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.849+0000 7fd859400640 1 --2- 172.21.15.16:0/3372687932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd85c075ea0 0x7fd85c19c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:18:25.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.849+0000 7fd85a800640 1 --2- 172.21.15.16:0/3372687932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd85c10fe80 0x7fd85c1a1c30 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd85400eb10 tx=0x7fd85400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:25.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.849+0000 7fd852a00640 1 -- 172.21.15.16:0/3372687932 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd854004b20 con 0x7fd85c10fe80 2024-08-31T06:18:25.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.849+0000 7fd852a00640 1 -- 172.21.15.16:0/3372687932 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd854004cc0 con 0x7fd85c10fe80 2024-08-31T06:18:25.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.849+0000 7fd852a00640 1 -- 172.21.15.16:0/3372687932 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd854010670 con 0x7fd85c10fe80 2024-08-31T06:18:25.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.849+0000 7fd860c00640 1 -- 172.21.15.16:0/3372687932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd85c1a2650 con 0x7fd85c10fe80 2024-08-31T06:18:25.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.849+0000 7fd860c00640 1 -- 172.21.15.16:0/3372687932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd85c1a8db0 con 0x7fd85c10fe80 2024-08-31T06:18:25.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.853+0000 7fd8678cb740 1 Processor -- start 2024-08-31T06:18:25.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.853+0000 7fd852a00640 1 -- 172.21.15.16:0/3372687932 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fd854005020 con 0x7fd85c10fe80 2024-08-31T06:18:25.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.853+0000 7fd8678cb740 1 -- start start 2024-08-31T06:18:25.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.853+0000 7fd8678cb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbed2ef070 0x55bbed694990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:25.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.853+0000 7fd8678cb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bbed2c61b0 0x55bbed694ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:25.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.853+0000 7fd8678cb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbed2ce390 0x55bbed695410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:25.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.853+0000 7fd8678cb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bbed60b7b0 con 0x55bbed2ef070 2024-08-31T06:18:25.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.853+0000 7fd8678cb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55bbed5fad40 con 0x55bbed2ce390 2024-08-31T06:18:25.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.853+0000 7fd8678cb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55bbed5fecf0 con 0x55bbed2c61b0 2024-08-31T06:18:25.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.854+0000 7fd859e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bbed2c61b0 0x55bbed694ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:25.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.854+0000 7fd859e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bbed2c61b0 0x55bbed694ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:51622/0 (socket says 172.21.15.16:51622) 2024-08-31T06:18:25.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.854+0000 7fd859e00640 1 -- 172.21.15.16:0/1484482298 learned_addr learned my addr 172.21.15.16:0/1484482298 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:25.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.854+0000 7fd859400640 1 --2- 172.21.15.16:0/1484482298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbed2ef070 0x55bbed694990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:25.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.854+0000 7fd859400640 1 --2- 172.21.15.16:0/1484482298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbed2ce390 0x55bbed695410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:25.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.854+0000 7fd859e00640 1 -- 172.21.15.16:0/1484482298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbed2ce390 msgr2=0x55bbed695410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:25.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.854+0000 7fd859e00640 1 --2- 172.21.15.16:0/1484482298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbed2ce390 0x55bbed695410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:25.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.854+0000 7fd859e00640 1 -- 172.21.15.16:0/1484482298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbed2ef070 msgr2=0x55bbed694990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:25.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.854+0000 7fd859e00640 1 --2- 172.21.15.16:0/1484482298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbed2ef070 0x55bbed694990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:25.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.854+0000 7fd859e00640 1 -- 172.21.15.16:0/1484482298 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbed35a650 con 0x55bbed2c61b0 2024-08-31T06:18:25.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.854+0000 7fd859400640 1 --2- 172.21.15.16:0/1484482298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbed2ef070 0x55bbed694990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:18:25.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.855+0000 7fd859e00640 1 --2- 172.21.15.16:0/1484482298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bbed2c61b0 0x55bbed694ed0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd84c009230 tx=0x7fd84c004470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8ed3ae259b8be72 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:25.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.855+0000 7fd852a00640 1 --2- 172.21.15.16:0/3372687932 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd8400b4ca0 0x7fd8400b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:25.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.855+0000 7fd847000640 1 -- 172.21.15.16:0/1484482298 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84c035040 con 0x55bbed2c61b0 2024-08-31T06:18:25.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.855+0000 7fd847000640 1 -- 172.21.15.16:0/1484482298 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd84c004770 con 0x55bbed2c61b0 2024-08-31T06:18:25.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.855+0000 7fd847000640 1 -- 172.21.15.16:0/1484482298 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84c0039d0 con 0x55bbed2c61b0 2024-08-31T06:18:25.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.855+0000 7fd852a00640 1 -- 172.21.15.16:0/3372687932 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8470+0+0 (secure 0 0 0) 0x7fd8540e5b60 con 0x7fd85c10fe80 2024-08-31T06:18:25.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.855+0000 7fd859400640 1 --2- 172.21.15.16:0/3372687932 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd8400b4ca0 0x7fd8400b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:25.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.856+0000 7fd8678cb740 1 -- 172.21.15.16:0/1484482298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bbed2c61b0 msgr2=0x55bbed694ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:25.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.856+0000 7fd859400640 1 --2- 172.21.15.16:0/3372687932 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd8400b4ca0 0x7fd8400b71b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd848009fd0 tx=0x7fd848005850 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:25.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.856+0000 7fd8678cb740 1 --2- 172.21.15.16:0/1484482298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bbed2c61b0 0x55bbed694ed0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd84c009230 tx=0x7fd84c004470 comp rx=0 tx=0).stop 2024-08-31T06:18:25.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.856+0000 7fd8678cb740 1 -- 172.21.15.16:0/1484482298 shutdown_connections 2024-08-31T06:18:25.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.856+0000 7fd8678cb740 1 --2- 172.21.15.16:0/1484482298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bbed2c61b0 0x55bbed694ed0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:25.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.856+0000 7fd8678cb740 1 --2- 172.21.15.16:0/1484482298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbed2ce390 0x55bbed695410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:25.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.856+0000 7fd8678cb740 1 --2- 172.21.15.16:0/1484482298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbed2ef070 0x55bbed694990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:25.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.856+0000 7fd8678cb740 1 -- 172.21.15.16:0/1484482298 >> 172.21.15.16:0/1484482298 conn(0x55bbed2e3460 msgr2=0x55bbed692880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:25.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.856+0000 7fd8678cb740 1 -- 172.21.15.16:0/1484482298 shutdown_connections 2024-08-31T06:18:25.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.856+0000 7fd8678cb740 1 -- 172.21.15.16:0/1484482298 wait complete. 2024-08-31T06:18:25.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.857+0000 7fd8678cb740 1 -- start start 2024-08-31T06:18:25.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.857+0000 7fd8678cb740 1 Processor -- start 2024-08-31T06:18:25.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.858+0000 7fd8678cb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbed2c61b0 0x55bbed6a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:25.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.858+0000 7fd8678cb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bbed2ce390 0x55bbed6a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:25.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.858+0000 7fd8678cb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbed2ef070 0x55bbed6ac430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:25.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.858+0000 7fd8678cb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bbed45d700 con 0x55bbed2c61b0 2024-08-31T06:18:25.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.858+0000 7fd8678cb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55bbed581cf0 con 0x55bbed2ef070 2024-08-31T06:18:25.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.858+0000 7fd8678cb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55bbed6a3110 con 0x55bbed2ce390 2024-08-31T06:18:25.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.858+0000 7fd8678cb740 10 client.0 _ll_register_callbacks cb 0x55bbed5a7470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:18:25.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.858+0000 7fd859400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bbed2ce390 0x55bbed6a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:25.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.858+0000 7fd859e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbed2c61b0 0x55bbed6a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:25.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.858+0000 7fd859400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bbed2ce390 0x55bbed6a0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:51628/0 (socket says 172.21.15.16:51628) 2024-08-31T06:18:25.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.858+0000 7fd859e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbed2c61b0 0x55bbed6a05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44394/0 (socket says 172.21.15.16:44394) 2024-08-31T06:18:25.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.858+0000 7fd859400640 1 -- 172.21.15.16:0/1110378130 learned_addr learned my addr 172.21.15.16:0/1110378130 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:25.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.858+0000 7fd85a800640 1 --2- 172.21.15.16:0/1110378130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbed2ef070 0x55bbed6ac430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:25.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.859+0000 7fd859e00640 1 -- 172.21.15.16:0/1110378130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bbed2ce390 msgr2=0x55bbed6a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:25.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.859+0000 7fd859e00640 1 --2- 172.21.15.16:0/1110378130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bbed2ce390 0x55bbed6a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:25.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.859+0000 7fd859e00640 1 -- 172.21.15.16:0/1110378130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbed2ef070 msgr2=0x55bbed6ac430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:25.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.859+0000 7fd859e00640 1 --2- 172.21.15.16:0/1110378130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbed2ef070 0x55bbed6ac430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:25.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.859+0000 7fd859e00640 1 -- 172.21.15.16:0/1110378130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbed5c28b0 con 0x55bbed2c61b0 2024-08-31T06:18:25.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.859+0000 7fd859400640 1 --2- 172.21.15.16:0/1110378130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bbed2ce390 0x55bbed6a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:18:25.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.859+0000 7fd859e00640 1 --2- 172.21.15.16:0/1110378130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbed2c61b0 0x55bbed6a05c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd84c005ab0 tx=0x7fd84c005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf7699e7a9102ae server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:25.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.859+0000 7fd83b400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:18:25.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.859+0000 7fd83b400640 1 -- 172.21.15.16:0/1110378130 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84c053060 con 0x55bbed2c61b0 2024-08-31T06:18:25.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.859+0000 7fd83b400640 1 -- 172.21.15.16:0/1110378130 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8470+0+0 (secure 0 0 0) 0x7fd84c004920 con 0x55bbed2c61b0 2024-08-31T06:18:25.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.859+0000 7fd8678cb740 1 -- 172.21.15.16:0/1110378130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd84c004770 con 0x55bbed2c61b0 2024-08-31T06:18:25.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.860+0000 7fd83b400640 1 -- 172.21.15.16:0/1110378130 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fd84c035050 con 0x55bbed2c61b0 2024-08-31T06:18:25.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.860+0000 7fd8678cb740 10 client.17784 fetch_fsmap learned FSMap version 47 2024-08-31T06:18:25.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.860+0000 7fd8678cb740 1 -- 172.21.15.16:0/1110378130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55bbed6af160 con 0x55bbed2c61b0 2024-08-31T06:18:25.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.860+0000 7fd83b400640 1 -- 172.21.15.16:0/1110378130 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd84c004770 con 0x55bbed2c61b0 2024-08-31T06:18:25.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.860+0000 7fd83b400640 1 -- 172.21.15.16:0/1110378130 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 47) ==== 8354+0+0 (secure 0 0 0) 0x7fd84c0314a0 con 0x55bbed2c61b0 2024-08-31T06:18:25.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.860+0000 7fd8678cb740 10 client.17784 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T06:18:25.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.860+0000 7fd8678cb740 10 client.17784 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-31T06:18:25.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.860+0000 7fd8678cb740 20 client.17784 populate_metadata read hostname 'smithi016' 2024-08-31T06:18:25.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.860+0000 7fd8678cb740 1 --2- 172.21.15.16:0/1110378130 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55bbed2ad9f0 0x55bbed6b24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:25.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.860+0000 7fd8678cb740 4 client.17784 mds_command: new command op to 24724 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:18:25.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.860+0000 7fd8678cb740 1 -- 172.21.15.16:0/1110378130 --> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbed60d700 con 0x55bbed2ad9f0 2024-08-31T06:18:25.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.861+0000 7fd859400640 1 --2- 172.21.15.16:0/1110378130 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55bbed2ad9f0 0x55bbed6b24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:25.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.861+0000 7fd859400640 1 --2- 172.21.15.16:0/1110378130 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55bbed2ad9f0 0x55bbed6b24c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a9ed3b9d841ae50 server_cookie=25faef4462f99fe9 in_seq=0 out_seq=0 2024-08-31T06:18:25.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.861+0000 7fd83b400640 10 client.17784 ms_handle_connect on v2:172.21.15.16:6838/219664460 2024-08-31T06:18:25.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.862+0000 7fd83b400640 1 -- 172.21.15.16:0/1110378130 <== mds.1 v2:172.21.15.16:6838/219664460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bbed60d700 con 0x55bbed2ad9f0 2024-08-31T06:18:25.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.862+0000 7fd83b400640 10 client.17784 handle_command_reply: tid=0 2024-08-31T06:18:25.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.862+0000 7fd8678cb740 1 client.17784 shutdown 2024-08-31T06:18:25.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.862+0000 7fd8678cb740 1 -- 172.21.15.16:0/1110378130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbed2c61b0 msgr2=0x55bbed6a05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:25.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.862+0000 7fd8678cb740 1 --2- 172.21.15.16:0/1110378130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbed2c61b0 0x55bbed6a05c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd84c005ab0 tx=0x7fd84c005d90 comp rx=0 tx=0).stop 2024-08-31T06:18:25.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.863+0000 7fd8678cb740 1 -- 172.21.15.16:0/1110378130 shutdown_connections 2024-08-31T06:18:25.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.863+0000 7fd8678cb740 1 --2- 172.21.15.16:0/1110378130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bbed2ce390 0x55bbed6a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:25.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.863+0000 7fd8678cb740 1 --2- 172.21.15.16:0/1110378130 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55bbed2ad9f0 0x55bbed6b24c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:25.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.863+0000 7fd8678cb740 1 --2- 172.21.15.16:0/1110378130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbed2ef070 0x55bbed6ac430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:25.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.863+0000 7fd8678cb740 1 --2- 172.21.15.16:0/1110378130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbed2c61b0 0x55bbed6a05c0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:25.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.863+0000 7fd83b400640 0 client.17784 ms_handle_reset on v2:172.21.15.16:6838/219664460 2024-08-31T06:18:25.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.863+0000 7fd8678cb740 1 -- 172.21.15.16:0/1110378130 >> 172.21.15.16:0/1110378130 conn(0x55bbed2e3460 msgr2=0x55bbed697b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:25.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.863+0000 7fd8678cb740 1 -- 172.21.15.16:0/1110378130 shutdown_connections 2024-08-31T06:18:25.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.863+0000 7fd8678cb740 1 -- 172.21.15.16:0/1110378130 wait complete. 2024-08-31T06:18:25.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.863+0000 7fd8678cb740 20 client.17784 trim_cache size 0 max 16384 2024-08-31T06:18:25.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.877+0000 7fd8678cb740 1 Processor -- start 2024-08-31T06:18:25.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.877+0000 7fd8678cb740 1 -- start start 2024-08-31T06:18:25.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.877+0000 7fd8678cb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bbed2c61b0 0x55bbed6981a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:25.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.877+0000 7fd8678cb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbed2ce390 0x55bbed699c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:25.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.877+0000 7fd8678cb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbed2ef070 0x55bbed69a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:25.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.877+0000 7fd8678cb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd85c1b3e50 con 0x55bbed2ce390 2024-08-31T06:18:25.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.877+0000 7fd8678cb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd85c1b16c0 con 0x55bbed2ef070 2024-08-31T06:18:25.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.877+0000 7fd8678cb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd85c1af530 con 0x55bbed2c61b0 2024-08-31T06:18:25.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.878+0000 7fd859400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbed2ce390 0x55bbed699c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:25.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.878+0000 7fd859400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbed2ce390 0x55bbed699c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44396/0 (socket says 172.21.15.16:44396) 2024-08-31T06:18:25.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.878+0000 7fd859400640 1 -- 172.21.15.16:0/1930462988 learned_addr learned my addr 172.21.15.16:0/1930462988 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:25.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.878+0000 7fd859e00640 1 --2- 172.21.15.16:0/1930462988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bbed2c61b0 0x55bbed6981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:25.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.878+0000 7fd85a800640 1 --2- 172.21.15.16:0/1930462988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbed2ef070 0x55bbed69a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:25.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.878+0000 7fd859e00640 1 -- 172.21.15.16:0/1930462988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbed2ef070 msgr2=0x55bbed69a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:25.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.878+0000 7fd859e00640 1 --2- 172.21.15.16:0/1930462988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbed2ef070 0x55bbed69a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:25.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.878+0000 7fd859e00640 1 -- 172.21.15.16:0/1930462988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbed2ce390 msgr2=0x55bbed699c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:25.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.878+0000 7fd859e00640 1 --2- 172.21.15.16:0/1930462988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbed2ce390 0x55bbed699c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:25.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.878+0000 7fd859e00640 1 -- 172.21.15.16:0/1930462988 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbed696c80 con 0x55bbed2c61b0 2024-08-31T06:18:25.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.878+0000 7fd859e00640 1 --2- 172.21.15.16:0/1930462988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bbed2c61b0 0x55bbed6981a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0318c0 tx=0x7fd84c031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=754d966b1c67fc7f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:25.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.879+0000 7fd846600640 1 -- 172.21.15.16:0/1930462988 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84c0314a0 con 0x55bbed2c61b0 2024-08-31T06:18:25.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.879+0000 7fd846600640 1 -- 172.21.15.16:0/1930462988 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd84c031640 con 0x55bbed2c61b0 2024-08-31T06:18:25.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.879+0000 7fd846600640 1 -- 172.21.15.16:0/1930462988 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84c004940 con 0x55bbed2c61b0 2024-08-31T06:18:25.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.879+0000 7fd8678cb740 1 -- 172.21.15.16:0/1930462988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bbed2c61b0 msgr2=0x55bbed6981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:25.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.879+0000 7fd8678cb740 1 --2- 172.21.15.16:0/1930462988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bbed2c61b0 0x55bbed6981a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0318c0 tx=0x7fd84c031400 comp rx=0 tx=0).stop 2024-08-31T06:18:25.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.880+0000 7fd8678cb740 1 -- 172.21.15.16:0/1930462988 shutdown_connections 2024-08-31T06:18:25.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.880+0000 7fd8678cb740 1 --2- 172.21.15.16:0/1930462988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbed2ef070 0x55bbed69a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:25.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.880+0000 7fd8678cb740 1 --2- 172.21.15.16:0/1930462988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbed2ce390 0x55bbed699c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:25.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.880+0000 7fd8678cb740 1 --2- 172.21.15.16:0/1930462988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bbed2c61b0 0x55bbed6981a0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:25.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.880+0000 7fd8678cb740 1 -- 172.21.15.16:0/1930462988 >> 172.21.15.16:0/1930462988 conn(0x55bbed2e3460 msgr2=0x55bbed697a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:25.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.880+0000 7fd8678cb740 1 -- 172.21.15.16:0/1930462988 shutdown_connections 2024-08-31T06:18:25.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.880+0000 7fd8678cb740 1 -- 172.21.15.16:0/1930462988 wait complete. 2024-08-31T06:18:25.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.880+0000 7fd8678cb740 1 -- start start 2024-08-31T06:18:25.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.880+0000 7fd8678cb740 1 Processor -- start 2024-08-31T06:18:25.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.881+0000 7fd8678cb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbed2c61b0 0x55bbed6b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:25.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.881+0000 7fd8678cb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bbed2ce390 0x55bbed6b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:25.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.881+0000 7fd8678cb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbed2ef070 0x55bbed71bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:25.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.881+0000 7fd8678cb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd85c1afb80 con 0x55bbed2ef070 2024-08-31T06:18:25.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.881+0000 7fd8678cb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd85c1b0ff0 con 0x55bbed2c61b0 2024-08-31T06:18:25.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.881+0000 7fd8678cb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd85c1b0250 con 0x55bbed2ce390 2024-08-31T06:18:25.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.881+0000 7fd8678cb740 10 client.0 _ll_register_callbacks cb 0x7fd84c005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:18:25.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.881+0000 7fd85a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbed2ef070 0x55bbed71bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:25.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.881+0000 7fd859400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bbed2ce390 0x55bbed6b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:25.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.881+0000 7fd859400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bbed2ce390 0x55bbed6b20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:51650/0 (socket says 172.21.15.16:51650) 2024-08-31T06:18:25.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.881+0000 7fd859e00640 1 --2- 172.21.15.16:0/1646068985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbed2c61b0 0x55bbed6b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:25.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.881+0000 7fd859400640 1 -- 172.21.15.16:0/1646068985 learned_addr learned my addr 172.21.15.16:0/1646068985 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:25.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.882+0000 7fd85a800640 1 -- 172.21.15.16:0/1646068985 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bbed2ce390 msgr2=0x55bbed6b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:25.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.882+0000 7fd85a800640 1 --2- 172.21.15.16:0/1646068985 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bbed2ce390 0x55bbed6b20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:25.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.882+0000 7fd85a800640 1 -- 172.21.15.16:0/1646068985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbed2c61b0 msgr2=0x55bbed6b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:25.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.882+0000 7fd85a800640 1 --2- 172.21.15.16:0/1646068985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbed2c61b0 0x55bbed6b1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:25.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.882+0000 7fd85a800640 1 -- 172.21.15.16:0/1646068985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbed71c170 con 0x55bbed2ef070 2024-08-31T06:18:25.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.882+0000 7fd85a800640 1 --2- 172.21.15.16:0/1646068985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbed2ef070 0x55bbed71bc30 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd8540e3280 tx=0x7fd854005650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fcdf82c5789471c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:25.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.882+0000 7fd83b400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:18:25.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.882+0000 7fd83b400640 1 -- 172.21.15.16:0/1646068985 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8540ac920 con 0x55bbed2ef070 2024-08-31T06:18:25.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.882+0000 7fd83b400640 1 -- 172.21.15.16:0/1646068985 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8470+0+0 (secure 0 0 0) 0x7fd8540b52c0 con 0x55bbed2ef070 2024-08-31T06:18:25.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.882+0000 7fd8678cb740 1 -- 172.21.15.16:0/1646068985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd84c031640 con 0x55bbed2ef070 2024-08-31T06:18:25.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.883+0000 7fd83b400640 1 -- 172.21.15.16:0/1646068985 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fd8540ac220 con 0x55bbed2ef070 2024-08-31T06:18:25.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.883+0000 7fd8678cb740 10 client.17790 fetch_fsmap learned FSMap version 47 2024-08-31T06:18:25.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.883+0000 7fd8678cb740 1 -- 172.21.15.16:0/1646068985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55bbed71d3b0 con 0x55bbed2ef070 2024-08-31T06:18:25.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.883+0000 7fd83b400640 1 -- 172.21.15.16:0/1646068985 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd84c031640 con 0x55bbed2ef070 2024-08-31T06:18:25.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.883+0000 7fd83b400640 1 -- 172.21.15.16:0/1646068985 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 47) ==== 8354+0+0 (secure 0 0 0) 0x7fd8540aaef0 con 0x55bbed2ef070 2024-08-31T06:18:25.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.883+0000 7fd8678cb740 10 client.17790 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T06:18:25.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.883+0000 7fd8678cb740 10 client.17790 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-31T06:18:25.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.883+0000 7fd8678cb740 20 client.17790 populate_metadata read hostname 'smithi016' 2024-08-31T06:18:25.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.883+0000 7fd8678cb740 1 --2- 172.21.15.16:0/1646068985 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55bbed282da0 0x55bbed71faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:25.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.883+0000 7fd8678cb740 4 client.17790 mds_command: new command op to 24724 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:18:25.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.883+0000 7fd8678cb740 1 -- 172.21.15.16:0/1646068985 --> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bbed6091a0 con 0x55bbed282da0 2024-08-31T06:18:25.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.884+0000 7fd859e00640 1 --2- 172.21.15.16:0/1646068985 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55bbed282da0 0x55bbed71faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:25.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.884+0000 7fd83b400640 10 client.17790 ms_handle_connect on v2:172.21.15.16:6838/219664460 2024-08-31T06:18:25.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.884+0000 7fd859e00640 1 --2- 172.21.15.16:0/1646068985 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55bbed282da0 0x55bbed71faf0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d0f7058fcf43b276 server_cookie=9ee8e3a49bd1ab3e in_seq=0 out_seq=0 2024-08-31T06:18:25.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.884+0000 7fd83b400640 1 -- 172.21.15.16:0/1646068985 <== mds.1 v2:172.21.15.16:6838/219664460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bbed6091a0 con 0x55bbed282da0 2024-08-31T06:18:25.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.884+0000 7fd83b400640 10 client.17790 handle_command_reply: tid=0 2024-08-31T06:18:25.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.884+0000 7fd8678cb740 1 client.17790 shutdown 2024-08-31T06:18:25.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.885+0000 7fd8678cb740 1 -- 172.21.15.16:0/1646068985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbed2ef070 msgr2=0x55bbed71bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:25.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.885+0000 7fd8678cb740 1 --2- 172.21.15.16:0/1646068985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbed2ef070 0x55bbed71bc30 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd8540e3280 tx=0x7fd854005650 comp rx=0 tx=0).stop 2024-08-31T06:18:25.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.885+0000 7fd8678cb740 1 -- 172.21.15.16:0/1646068985 shutdown_connections 2024-08-31T06:18:25.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.885+0000 7fd8678cb740 1 --2- 172.21.15.16:0/1646068985 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bbed2ce390 0x55bbed6b20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:25.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.885+0000 7fd8678cb740 1 --2- 172.21.15.16:0/1646068985 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55bbed282da0 0x55bbed71faf0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:25.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.885+0000 7fd8678cb740 1 --2- 172.21.15.16:0/1646068985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbed2ef070 0x55bbed71bc30 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:25.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.885+0000 7fd8678cb740 1 --2- 172.21.15.16:0/1646068985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbed2c61b0 0x55bbed6b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:25.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.885+0000 7fd83b400640 0 client.17790 ms_handle_reset on v2:172.21.15.16:6838/219664460 2024-08-31T06:18:25.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.885+0000 7fd8678cb740 1 -- 172.21.15.16:0/1646068985 >> 172.21.15.16:0/1646068985 conn(0x55bbed2e3460 msgr2=0x55bbed6a2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:25.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.885+0000 7fd8678cb740 1 -- 172.21.15.16:0/1646068985 shutdown_connections 2024-08-31T06:18:25.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.885+0000 7fd8678cb740 1 -- 172.21.15.16:0/1646068985 wait complete. 2024-08-31T06:18:25.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.885+0000 7fd8678cb740 20 client.17790 trim_cache size 0 max 16384 2024-08-31T06:18:25.891 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:18:25.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.886+0000 7fd860c00640 1 -- 172.21.15.16:0/3372687932 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd8400b4ca0 msgr2=0x7fd8400b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:25.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.886+0000 7fd860c00640 1 --2- 172.21.15.16:0/3372687932 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd8400b4ca0 0x7fd8400b71b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd848009fd0 tx=0x7fd848005850 comp rx=0 tx=0).stop 2024-08-31T06:18:25.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.886+0000 7fd860c00640 1 -- 172.21.15.16:0/3372687932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd85c10fe80 msgr2=0x7fd85c1a1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:25.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.886+0000 7fd860c00640 1 --2- 172.21.15.16:0/3372687932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd85c10fe80 0x7fd85c1a1c30 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd85400eb10 tx=0x7fd85400c2d0 comp rx=0 tx=0).stop 2024-08-31T06:18:25.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.886+0000 7fd860c00640 1 -- 172.21.15.16:0/3372687932 shutdown_connections 2024-08-31T06:18:25.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.886+0000 7fd860c00640 1 --2- 172.21.15.16:0/3372687932 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd8400b4ca0 0x7fd8400b71b0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:25.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.886+0000 7fd860c00640 1 --2- 172.21.15.16:0/3372687932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd85c10fe80 0x7fd85c1a1c30 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:25.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.886+0000 7fd860c00640 1 --2- 172.21.15.16:0/3372687932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd85c075ea0 0x7fd85c19c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:25.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.886+0000 7fd860c00640 1 --2- 172.21.15.16:0/3372687932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd85c075550 0x7fd85c19c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:25.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.886+0000 7fd860c00640 1 -- 172.21.15.16:0/3372687932 >> 172.21.15.16:0/3372687932 conn(0x7fd85c0ff370 msgr2=0x7fd85c1078e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:25.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.886+0000 7fd860c00640 1 -- 172.21.15.16:0/3372687932 shutdown_connections 2024-08-31T06:18:25.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:25.886+0000 7fd860c00640 1 -- 172.21.15.16:0/3372687932 wait complete. 2024-08-31T06:18:26.629 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 660.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-31T06:18:26.821 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T06:18:26.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:26 smithi016 ceph-mon[17194]: pgmap v712: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 MiB/s wr, 968 op/s 2024-08-31T06:18:26.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:26 smithi184 ceph-mon[22331]: pgmap v712: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 MiB/s wr, 968 op/s 2024-08-31T06:18:27.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:26 smithi155 ceph-mon[21742]: pgmap v712: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 MiB/s wr, 968 op/s 2024-08-31T06:18:27.203 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:18:28.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.053+0000 7f2f09600640 1 -- 172.21.15.16:0/1320362412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f04075550 msgr2=0x7f2f04075960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:28.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.054+0000 7f2f09600640 1 --2- 172.21.15.16:0/1320362412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f04075550 0x7f2f04075960 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4009860 tx=0x7f2ef402a340 comp rx=0 tx=0).stop 2024-08-31T06:18:28.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.054+0000 7f2f09600640 1 -- 172.21.15.16:0/1320362412 shutdown_connections 2024-08-31T06:18:28.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.054+0000 7f2f09600640 1 --2- 172.21.15.16:0/1320362412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f04075ea0 0x7f2f0410f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:28.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.054+0000 7f2f09600640 1 --2- 172.21.15.16:0/1320362412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f0410fcb0 0x7f2f041120f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:28.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.054+0000 7f2f09600640 1 --2- 172.21.15.16:0/1320362412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f04075550 0x7f2f04075960 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:28.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.054+0000 7f2f09600640 1 -- 172.21.15.16:0/1320362412 >> 172.21.15.16:0/1320362412 conn(0x7f2f040ff1e0 msgr2=0x7f2f04101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:28.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.054+0000 7f2f09600640 1 -- 172.21.15.16:0/1320362412 shutdown_connections 2024-08-31T06:18:28.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.054+0000 7f2f09600640 1 -- 172.21.15.16:0/1320362412 wait complete. 2024-08-31T06:18:28.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.055+0000 7f2f09600640 1 Processor -- start 2024-08-31T06:18:28.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.055+0000 7f2f09600640 1 -- start start 2024-08-31T06:18:28.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.055+0000 7f2f09600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f04075550 0x7f2f0419c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:28.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.055+0000 7f2f09600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f04075ea0 0x7f2f0419c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:28.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.055+0000 7f2f09600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f0410fcb0 0x7f2f041a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:28.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.055+0000 7f2f09600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2f04115860 con 0x7f2f0410fcb0 2024-08-31T06:18:28.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.055+0000 7f2f09600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2f041156e0 con 0x7f2f04075ea0 2024-08-31T06:18:28.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.055+0000 7f2f09600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2f041159e0 con 0x7f2f04075550 2024-08-31T06:18:28.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.056+0000 7f2f03400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f04075ea0 0x7f2f0419c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:28.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.056+0000 7f2f03400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f04075ea0 0x7f2f0419c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:53758/0 (socket says 172.21.15.16:53758) 2024-08-31T06:18:28.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.056+0000 7f2f08c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f0410fcb0 0x7f2f041a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:28.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.056+0000 7f2f03400640 1 -- 172.21.15.16:0/2480336526 learned_addr learned my addr 172.21.15.16:0/2480336526 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:28.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.056+0000 7f2f03e00640 1 --2- 172.21.15.16:0/2480336526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f04075550 0x7f2f0419c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:28.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.056+0000 7f2f03400640 1 -- 172.21.15.16:0/2480336526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f04075550 msgr2=0x7f2f0419c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:28.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.056+0000 7f2f03400640 1 --2- 172.21.15.16:0/2480336526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f04075550 0x7f2f0419c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:28.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.056+0000 7f2f03400640 1 -- 172.21.15.16:0/2480336526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f0410fcb0 msgr2=0x7f2f041a1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:28.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.056+0000 7f2f03400640 1 --2- 172.21.15.16:0/2480336526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f0410fcb0 0x7f2f041a1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:28.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.056+0000 7f2f03400640 1 -- 172.21.15.16:0/2480336526 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f041a2240 con 0x7f2f04075ea0 2024-08-31T06:18:28.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.056+0000 7f2f03400640 1 --2- 172.21.15.16:0/2480336526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f04075ea0 0x7f2f0419c870 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f2ef000e9c0 tx=0x7f2ef000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:28.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.057+0000 7f2f00c00640 1 -- 172.21.15.16:0/2480336526 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef000cd80 con 0x7f2f04075ea0 2024-08-31T06:18:28.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.057+0000 7f2f09600640 1 -- 172.21.15.16:0/2480336526 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f041a2530 con 0x7f2f04075ea0 2024-08-31T06:18:28.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.057+0000 7f2f00c00640 1 -- 172.21.15.16:0/2480336526 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ef00035f0 con 0x7f2f04075ea0 2024-08-31T06:18:28.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.057+0000 7f2f00c00640 1 -- 172.21.15.16:0/2480336526 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef0010690 con 0x7f2f04075ea0 2024-08-31T06:18:28.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.057+0000 7f2f09600640 1 -- 172.21.15.16:0/2480336526 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f041a8c90 con 0x7f2f04075ea0 2024-08-31T06:18:28.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.061+0000 7f2f11834740 1 Processor -- start 2024-08-31T06:18:28.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.061+0000 7f2f11834740 1 -- start start 2024-08-31T06:18:28.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.061+0000 7f2f11834740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594de7ce070 0x5594deb7c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:28.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.061+0000 7f2f11834740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594de7a51b0 0x5594deb7c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:28.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.062+0000 7f2f11834740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594de7ad390 0x5594deb7cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:28.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.062+0000 7f2f11834740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5594deaea7b0 con 0x5594de7ad390 2024-08-31T06:18:28.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.062+0000 7f2f11834740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5594dead9d40 con 0x5594de7a51b0 2024-08-31T06:18:28.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.062+0000 7f2f11834740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5594deaddcf0 con 0x5594de7ce070 2024-08-31T06:18:28.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.062+0000 7f2f08c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594de7ce070 0x5594deb7c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:28.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.062+0000 7f2f08c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594de7ce070 0x5594deb7c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:51672/0 (socket says 172.21.15.16:51672) 2024-08-31T06:18:28.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.062+0000 7f2f08c00640 1 -- 172.21.15.16:0/3237497639 learned_addr learned my addr 172.21.15.16:0/3237497639 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:28.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.062+0000 7f2f03e00640 1 --2- 172.21.15.16:0/3237497639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594de7a51b0 0x5594deb7c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:28.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.062+0000 7f2f03e00640 1 -- 172.21.15.16:0/3237497639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594de7ce070 msgr2=0x5594deb7c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:28.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.062+0000 7f2f00c00640 1 -- 172.21.15.16:0/2480336526 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f2ef00037a0 con 0x7f2f04075ea0 2024-08-31T06:18:28.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.062+0000 7f2f03e00640 1 --2- 172.21.15.16:0/3237497639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594de7ce070 0x5594deb7c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:28.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.062+0000 7f2f03e00640 1 -- 172.21.15.16:0/3237497639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594de7ad390 msgr2=0x5594deb7cdf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:18:28.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.062+0000 7f2f03e00640 1 --2- 172.21.15.16:0/3237497639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594de7ad390 0x5594deb7cdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:28.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.062+0000 7f2f03e00640 1 -- 172.21.15.16:0/3237497639 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594de839650 con 0x5594de7a51b0 2024-08-31T06:18:28.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.063+0000 7f2f03e00640 1 --2- 172.21.15.16:0/3237497639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594de7a51b0 0x5594deb7c8b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4035040 tx=0x7f2ef4005d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4558a4cc681d55e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:28.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.063+0000 7f2ef9200640 1 -- 172.21.15.16:0/3237497639 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4041020 con 0x5594de7a51b0 2024-08-31T06:18:28.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.063+0000 7f2ef9200640 1 -- 172.21.15.16:0/3237497639 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ef4030450 con 0x5594de7a51b0 2024-08-31T06:18:28.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.063+0000 7f2ef9200640 1 -- 172.21.15.16:0/3237497639 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4037040 con 0x5594de7a51b0 2024-08-31T06:18:28.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.064+0000 7f2f11834740 1 -- 172.21.15.16:0/3237497639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594de7a51b0 msgr2=0x5594deb7c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:28.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.064+0000 7f2f11834740 1 --2- 172.21.15.16:0/3237497639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594de7a51b0 0x5594deb7c8b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4035040 tx=0x7f2ef4005d10 comp rx=0 tx=0).stop 2024-08-31T06:18:28.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.064+0000 7f2f11834740 1 -- 172.21.15.16:0/3237497639 shutdown_connections 2024-08-31T06:18:28.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.064+0000 7f2f11834740 1 --2- 172.21.15.16:0/3237497639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594de7ad390 0x5594deb7cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:28.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.064+0000 7f2f11834740 1 --2- 172.21.15.16:0/3237497639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594de7a51b0 0x5594deb7c8b0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:28.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.064+0000 7f2f11834740 1 --2- 172.21.15.16:0/3237497639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594de7ce070 0x5594deb7c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:28.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.064+0000 7f2f11834740 1 -- 172.21.15.16:0/3237497639 >> 172.21.15.16:0/3237497639 conn(0x5594de7c2460 msgr2=0x5594deb7a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:28.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.064+0000 7f2f11834740 1 -- 172.21.15.16:0/3237497639 shutdown_connections 2024-08-31T06:18:28.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.064+0000 7f2f11834740 1 -- 172.21.15.16:0/3237497639 wait complete. 2024-08-31T06:18:28.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.064+0000 7f2f11834740 1 -- start start 2024-08-31T06:18:28.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.064+0000 7f2f00c00640 1 --2- 172.21.15.16:0/2480336526 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f2ee80b4bc0 0x7f2ee80b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:28.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.065+0000 7f2f00c00640 1 -- 172.21.15.16:0/2480336526 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8470+0+0 (secure 0 0 0) 0x7f2ef00e5910 con 0x7f2f04075ea0 2024-08-31T06:18:28.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.065+0000 7f2f11834740 1 Processor -- start 2024-08-31T06:18:28.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.065+0000 7f2f08c00640 1 --2- 172.21.15.16:0/2480336526 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f2ee80b4bc0 0x7f2ee80b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:28.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.065+0000 7f2f11834740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594de7a51b0 0x5594deb87f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:28.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.065+0000 7f2f11834740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594de7ad390 0x5594deb884e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:28.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.065+0000 7f2f11834740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594de7ce070 0x5594deb93de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:28.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.065+0000 7f2f11834740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5594de93c700 con 0x5594de7a51b0 2024-08-31T06:18:28.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.065+0000 7f2f11834740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5594dea60cf0 con 0x5594de7ad390 2024-08-31T06:18:28.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.065+0000 7f2f11834740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5594deb8aaf0 con 0x5594de7ce070 2024-08-31T06:18:28.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.066+0000 7f2f11834740 10 client.0 _ll_register_callbacks cb 0x5594dea86470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:18:28.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.066+0000 7f2f08c00640 1 --2- 172.21.15.16:0/2480336526 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f2ee80b4bc0 0x7f2ee80b70d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2efc005c10 tx=0x7f2efc010040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:28.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.066+0000 7f2f03e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594de7a51b0 0x5594deb87f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:28.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.066+0000 7f2f03400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594de7ad390 0x5594deb884e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:28.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.066+0000 7f2f08c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594de7ce070 0x5594deb93de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:28.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.066+0000 7f2f03400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594de7ad390 0x5594deb884e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:53788/0 (socket says 172.21.15.16:53788) 2024-08-31T06:18:28.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.066+0000 7f2f03400640 1 -- 172.21.15.16:0/666999459 learned_addr learned my addr 172.21.15.16:0/666999459 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:28.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.066+0000 7f2f03400640 1 -- 172.21.15.16:0/666999459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594de7ce070 msgr2=0x5594deb93de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:28.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.066+0000 7f2f03400640 1 --2- 172.21.15.16:0/666999459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594de7ce070 0x5594deb93de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:28.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.066+0000 7f2f03400640 1 -- 172.21.15.16:0/666999459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594de7a51b0 msgr2=0x5594deb87f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:28.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.066+0000 7f2f03400640 1 --2- 172.21.15.16:0/666999459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594de7a51b0 0x5594deb87f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:28.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.066+0000 7f2f03400640 1 -- 172.21.15.16:0/666999459 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594deaa18b0 con 0x5594de7ad390 2024-08-31T06:18:28.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.067+0000 7f2f03400640 1 --2- 172.21.15.16:0/666999459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594de7ad390 0x5594deb884e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f2ef00abcd0 tx=0x7f2ef0003cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3284362cc1f16720 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:28.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.067+0000 7f2eed600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:18:28.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.067+0000 7f2eed600640 1 -- 172.21.15.16:0/666999459 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef00b0840 con 0x5594de7ad390 2024-08-31T06:18:28.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.067+0000 7f2eed600640 1 -- 172.21.15.16:0/666999459 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8470+0+0 (secure 0 0 0) 0x7f2ef00b4e60 con 0x5594de7ad390 2024-08-31T06:18:28.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.067+0000 7f2f11834740 1 -- 172.21.15.16:0/666999459 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ef4030450 con 0x5594de7ad390 2024-08-31T06:18:28.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.067+0000 7f2eed600640 1 -- 172.21.15.16:0/666999459 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f2ef00b52f0 con 0x5594de7ad390 2024-08-31T06:18:28.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.067+0000 7f2f11834740 10 client.28160 fetch_fsmap learned FSMap version 47 2024-08-31T06:18:28.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.067+0000 7f2f11834740 1 -- 172.21.15.16:0/666999459 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5594deb96bb0 con 0x5594de7ad390 2024-08-31T06:18:28.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.068+0000 7f2eed600640 1 -- 172.21.15.16:0/666999459 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ef4030450 con 0x5594de7ad390 2024-08-31T06:18:28.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.068+0000 7f2eed600640 1 -- 172.21.15.16:0/666999459 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 47) ==== 8354+0+0 (secure 0 0 0) 0x7f2ef0093110 con 0x5594de7ad390 2024-08-31T06:18:28.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.068+0000 7f2f11834740 10 client.28160 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T06:18:28.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.068+0000 7f2f11834740 10 client.28160 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:18:28.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.068+0000 7f2f11834740 20 client.28160 populate_metadata read hostname 'smithi016' 2024-08-31T06:18:28.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.068+0000 7f2f11834740 1 --2- 172.21.15.16:0/666999459 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5594de78c9f0 0x5594deb9a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:28.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.068+0000 7f2f11834740 4 client.28160 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:18:28.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.068+0000 7f2f11834740 1 -- 172.21.15.16:0/666999459 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594deaec700 con 0x5594de78c9f0 2024-08-31T06:18:28.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.068+0000 7f2f03e00640 1 --2- 172.21.15.16:0/666999459 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5594de78c9f0 0x5594deb9a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:28.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.069+0000 7f2f03e00640 1 --2- 172.21.15.16:0/666999459 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5594de78c9f0 0x5594deb9a170 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a7b1f09c879841e server_cookie=5739165a6cf225ba in_seq=0 out_seq=0 2024-08-31T06:18:28.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.069+0000 7f2eed600640 10 client.28160 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:18:28.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.070+0000 7f2eed600640 1 -- 172.21.15.16:0/666999459 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5594deaec700 con 0x5594de78c9f0 2024-08-31T06:18:28.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.070+0000 7f2eed600640 10 client.28160 handle_command_reply: tid=0 2024-08-31T06:18:28.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.070+0000 7f2f11834740 1 client.28160 shutdown 2024-08-31T06:18:28.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.070+0000 7f2f11834740 1 -- 172.21.15.16:0/666999459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594de7ad390 msgr2=0x5594deb884e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:28.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.070+0000 7f2f11834740 1 --2- 172.21.15.16:0/666999459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594de7ad390 0x5594deb884e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f2ef00abcd0 tx=0x7f2ef0003cf0 comp rx=0 tx=0).stop 2024-08-31T06:18:28.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.070+0000 7f2f11834740 1 -- 172.21.15.16:0/666999459 shutdown_connections 2024-08-31T06:18:28.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.070+0000 7f2f11834740 1 --2- 172.21.15.16:0/666999459 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5594de78c9f0 0x5594deb9a170 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:28.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.070+0000 7f2f11834740 1 --2- 172.21.15.16:0/666999459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594de7ce070 0x5594deb93de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:28.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.070+0000 7f2f11834740 1 --2- 172.21.15.16:0/666999459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594de7ad390 0x5594deb884e0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:28.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.070+0000 7f2f11834740 1 --2- 172.21.15.16:0/666999459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594de7a51b0 0x5594deb87f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:28.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.070+0000 7f2eed600640 0 client.28160 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:18:28.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.070+0000 7f2f11834740 1 -- 172.21.15.16:0/666999459 >> 172.21.15.16:0/666999459 conn(0x5594de7c2460 msgr2=0x5594deb7f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:28.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.070+0000 7f2f11834740 1 -- 172.21.15.16:0/666999459 shutdown_connections 2024-08-31T06:18:28.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.070+0000 7f2f11834740 1 -- 172.21.15.16:0/666999459 wait complete. 2024-08-31T06:18:28.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.070+0000 7f2f11834740 20 client.28160 trim_cache size 0 max 16384 2024-08-31T06:18:28.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.084+0000 7f2f11834740 1 Processor -- start 2024-08-31T06:18:28.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.085+0000 7f2f11834740 1 -- start start 2024-08-31T06:18:28.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.085+0000 7f2f11834740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594de7a51b0 0x5594deb93ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:28.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.085+0000 7f2f11834740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594de7ad390 0x5594deb89990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:28.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.085+0000 7f2f11834740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594de7ce070 0x5594deb815f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:28.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.085+0000 7f2f11834740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2f041af900 con 0x5594de7ce070 2024-08-31T06:18:28.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.085+0000 7f2f11834740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2f041ad170 con 0x5594de7a51b0 2024-08-31T06:18:28.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.085+0000 7f2f11834740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2f041aafe0 con 0x5594de7ad390 2024-08-31T06:18:28.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.085+0000 7f2f03e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594de7a51b0 0x5594deb93ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:28.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.085+0000 7f2f03e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594de7a51b0 0x5594deb93ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:53804/0 (socket says 172.21.15.16:53804) 2024-08-31T06:18:28.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.085+0000 7f2f03e00640 1 -- 172.21.15.16:0/785984464 learned_addr learned my addr 172.21.15.16:0/785984464 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:28.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.085+0000 7f2f08c00640 1 --2- 172.21.15.16:0/785984464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594de7ce070 0x5594deb815f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:28.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.085+0000 7f2f03400640 1 --2- 172.21.15.16:0/785984464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594de7ad390 0x5594deb89990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:28.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.086+0000 7f2f03e00640 1 -- 172.21.15.16:0/785984464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594de7ad390 msgr2=0x5594deb89990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:28.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.086+0000 7f2f03e00640 1 --2- 172.21.15.16:0/785984464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594de7ad390 0x5594deb89990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:28.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.086+0000 7f2f03e00640 1 -- 172.21.15.16:0/785984464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594de7ce070 msgr2=0x5594deb815f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:28.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.086+0000 7f2f03e00640 1 --2- 172.21.15.16:0/785984464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594de7ce070 0x5594deb815f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:28.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.086+0000 7f2f03e00640 1 -- 172.21.15.16:0/785984464 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594deb8a550 con 0x5594de7a51b0 2024-08-31T06:18:28.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.086+0000 7f2f03e00640 1 --2- 172.21.15.16:0/785984464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594de7a51b0 0x5594deb93ce0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x5594deb7aff0 tx=0x7f2ef402a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f22b6658b5eea99f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:28.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.086+0000 7f2eefe00640 1 -- 172.21.15.16:0/785984464 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4041040 con 0x5594de7a51b0 2024-08-31T06:18:28.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.086+0000 7f2eefe00640 1 -- 172.21.15.16:0/785984464 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ef4052030 con 0x5594de7a51b0 2024-08-31T06:18:28.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.086+0000 7f2eefe00640 1 -- 172.21.15.16:0/785984464 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4037040 con 0x5594de7a51b0 2024-08-31T06:18:28.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.087+0000 7f2f11834740 1 -- 172.21.15.16:0/785984464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594de7a51b0 msgr2=0x5594deb93ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:28.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.087+0000 7f2f11834740 1 --2- 172.21.15.16:0/785984464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594de7a51b0 0x5594deb93ce0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x5594deb7aff0 tx=0x7f2ef402a340 comp rx=0 tx=0).stop 2024-08-31T06:18:28.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.087+0000 7f2f11834740 1 -- 172.21.15.16:0/785984464 shutdown_connections 2024-08-31T06:18:28.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.087+0000 7f2f11834740 1 --2- 172.21.15.16:0/785984464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594de7ad390 0x5594deb89990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:28.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.087+0000 7f2f11834740 1 --2- 172.21.15.16:0/785984464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594de7ce070 0x5594deb815f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:28.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.087+0000 7f2f11834740 1 --2- 172.21.15.16:0/785984464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594de7a51b0 0x5594deb93ce0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:28.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.087+0000 7f2f11834740 1 -- 172.21.15.16:0/785984464 >> 172.21.15.16:0/785984464 conn(0x5594de7c2460 msgr2=0x5594deb89280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:28.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.087+0000 7f2f11834740 1 -- 172.21.15.16:0/785984464 shutdown_connections 2024-08-31T06:18:28.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.087+0000 7f2f11834740 1 -- 172.21.15.16:0/785984464 wait complete. 2024-08-31T06:18:28.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.088+0000 7f2f11834740 1 -- start start 2024-08-31T06:18:28.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.088+0000 7f2f11834740 1 Processor -- start 2024-08-31T06:18:28.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.089+0000 7f2f11834740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594de7a51b0 0x5594deb8fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:28.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.089+0000 7f2f11834740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594de7ad390 0x5594deb90290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:28.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.089+0000 7f2f11834740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594de7ce070 0x5594deb8ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:28.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.089+0000 7f2f11834740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2f041ab630 con 0x5594de7ce070 2024-08-31T06:18:28.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.089+0000 7f2f11834740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2f041acaa0 con 0x5594de7a51b0 2024-08-31T06:18:28.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.089+0000 7f2f11834740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2f041abd00 con 0x5594de7ad390 2024-08-31T06:18:28.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.089+0000 7f2f11834740 10 client.0 _ll_register_callbacks cb 0x7f2ef00ab740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:18:28.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.089+0000 7f2f03e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594de7a51b0 0x5594deb8fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:28.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.089+0000 7f2f03e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594de7a51b0 0x5594deb8fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:53818/0 (socket says 172.21.15.16:53818) 2024-08-31T06:18:28.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.089+0000 7f2f03e00640 1 -- 172.21.15.16:0/3446619508 learned_addr learned my addr 172.21.15.16:0/3446619508 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:28.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.089+0000 7f2f03400640 1 --2- 172.21.15.16:0/3446619508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594de7ad390 0x5594deb90290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:28.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.089+0000 7f2f08c00640 1 --2- 172.21.15.16:0/3446619508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594de7ce070 0x5594deb8ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:28.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.089+0000 7f2f03e00640 1 -- 172.21.15.16:0/3446619508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594de7ad390 msgr2=0x5594deb90290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:28.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.089+0000 7f2f03e00640 1 --2- 172.21.15.16:0/3446619508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594de7ad390 0x5594deb90290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:28.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.089+0000 7f2f03e00640 1 -- 172.21.15.16:0/3446619508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594de7ce070 msgr2=0x5594deb8ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:28.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.089+0000 7f2f03e00640 1 --2- 172.21.15.16:0/3446619508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594de7ce070 0x5594deb8ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:28.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.089+0000 7f2f03e00640 1 -- 172.21.15.16:0/3446619508 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594deb8b240 con 0x5594de7a51b0 2024-08-31T06:18:28.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.090+0000 7f2f03e00640 1 --2- 172.21.15.16:0/3446619508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594de7a51b0 0x5594deb8fd50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f2ef402a930 tx=0x7f2ef4005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a4e7bc7a40830fc server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:28.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.090+0000 7f2eed600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:18:28.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.090+0000 7f2eed600640 1 -- 172.21.15.16:0/3446619508 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef405b070 con 0x5594de7a51b0 2024-08-31T06:18:28.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.090+0000 7f2f11834740 1 -- 172.21.15.16:0/3446619508 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ef4052030 con 0x5594de7a51b0 2024-08-31T06:18:28.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.090+0000 7f2eed600640 1 -- 172.21.15.16:0/3446619508 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8470+0+0 (secure 0 0 0) 0x7f2ef4004040 con 0x5594de7a51b0 2024-08-31T06:18:28.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.090+0000 7f2eed600640 1 -- 172.21.15.16:0/3446619508 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f2ef4067050 con 0x5594de7a51b0 2024-08-31T06:18:28.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.090+0000 7f2f11834740 10 client.28166 fetch_fsmap learned FSMap version 47 2024-08-31T06:18:28.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.090+0000 7f2f11834740 1 -- 172.21.15.16:0/3446619508 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5594dec05170 con 0x5594de7a51b0 2024-08-31T06:18:28.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.091+0000 7f2eed600640 1 -- 172.21.15.16:0/3446619508 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ef4052030 con 0x5594de7a51b0 2024-08-31T06:18:28.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.091+0000 7f2eed600640 1 -- 172.21.15.16:0/3446619508 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 47) ==== 8354+0+0 (secure 0 0 0) 0x7f2ef403d480 con 0x5594de7a51b0 2024-08-31T06:18:28.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.091+0000 7f2f11834740 10 client.28166 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T06:18:28.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.091+0000 7f2f11834740 10 client.28166 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:18:28.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.091+0000 7f2f11834740 20 client.28166 populate_metadata read hostname 'smithi016' 2024-08-31T06:18:28.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.091+0000 7f2f11834740 1 --2- 172.21.15.16:0/3446619508 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5594de761da0 0x5594dec07400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:28.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.091+0000 7f2f11834740 4 client.28166 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:18:28.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.091+0000 7f2f11834740 1 -- 172.21.15.16:0/3446619508 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5594deae81a0 con 0x5594de761da0 2024-08-31T06:18:28.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.091+0000 7f2f03400640 1 --2- 172.21.15.16:0/3446619508 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5594de761da0 0x5594dec07400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:28.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.092+0000 7f2f03400640 1 --2- 172.21.15.16:0/3446619508 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5594de761da0 0x5594dec07400 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ff02ffce3fe635 server_cookie=787194916fdd2a59 in_seq=0 out_seq=0 2024-08-31T06:18:28.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.092+0000 7f2eed600640 10 client.28166 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:18:28.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.092+0000 7f2eed600640 1 -- 172.21.15.16:0/3446619508 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5594deae81a0 con 0x5594de761da0 2024-08-31T06:18:28.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.092+0000 7f2eed600640 10 client.28166 handle_command_reply: tid=0 2024-08-31T06:18:28.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.092+0000 7f2f11834740 1 client.28166 shutdown 2024-08-31T06:18:28.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.092+0000 7f2f11834740 1 -- 172.21.15.16:0/3446619508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594de7a51b0 msgr2=0x5594deb8fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:28.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.092+0000 7f2f11834740 1 --2- 172.21.15.16:0/3446619508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594de7a51b0 0x5594deb8fd50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f2ef402a930 tx=0x7f2ef4005f90 comp rx=0 tx=0).stop 2024-08-31T06:18:28.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.092+0000 7f2f11834740 1 -- 172.21.15.16:0/3446619508 shutdown_connections 2024-08-31T06:18:28.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.092+0000 7f2f11834740 1 --2- 172.21.15.16:0/3446619508 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5594de761da0 0x5594dec07400 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:28.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.092+0000 7f2f11834740 1 --2- 172.21.15.16:0/3446619508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594de7ad390 0x5594deb90290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:28.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.092+0000 7f2f11834740 1 --2- 172.21.15.16:0/3446619508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594de7ce070 0x5594deb8ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:28.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.092+0000 7f2f11834740 1 --2- 172.21.15.16:0/3446619508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594de7a51b0 0x5594deb8fd50 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:28.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.092+0000 7f2eed600640 0 client.28166 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:18:28.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.092+0000 7f2f11834740 1 -- 172.21.15.16:0/3446619508 >> 172.21.15.16:0/3446619508 conn(0x5594de7c2460 msgr2=0x5594deb7a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:28.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.093+0000 7f2f11834740 1 -- 172.21.15.16:0/3446619508 shutdown_connections 2024-08-31T06:18:28.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.093+0000 7f2f11834740 1 -- 172.21.15.16:0/3446619508 wait complete. 2024-08-31T06:18:28.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.093+0000 7f2f11834740 20 client.28166 trim_cache size 0 max 16384 2024-08-31T06:18:28.099 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:18:28.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.093+0000 7f2f09600640 1 -- 172.21.15.16:0/2480336526 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f2ee80b4bc0 msgr2=0x7f2ee80b70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:28.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.093+0000 7f2f09600640 1 --2- 172.21.15.16:0/2480336526 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f2ee80b4bc0 0x7f2ee80b70d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2efc005c10 tx=0x7f2efc010040 comp rx=0 tx=0).stop 2024-08-31T06:18:28.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.093+0000 7f2f09600640 1 -- 172.21.15.16:0/2480336526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f04075ea0 msgr2=0x7f2f0419c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:28.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.093+0000 7f2f09600640 1 --2- 172.21.15.16:0/2480336526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f04075ea0 0x7f2f0419c870 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f2ef000e9c0 tx=0x7f2ef000ee80 comp rx=0 tx=0).stop 2024-08-31T06:18:28.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.094+0000 7f2f09600640 1 -- 172.21.15.16:0/2480336526 shutdown_connections 2024-08-31T06:18:28.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.094+0000 7f2f09600640 1 --2- 172.21.15.16:0/2480336526 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f2ee80b4bc0 0x7f2ee80b70d0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:28.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.094+0000 7f2f09600640 1 --2- 172.21.15.16:0/2480336526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f0410fcb0 0x7f2f041a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:28.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.094+0000 7f2f09600640 1 --2- 172.21.15.16:0/2480336526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f04075ea0 0x7f2f0419c870 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:28.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.094+0000 7f2f09600640 1 --2- 172.21.15.16:0/2480336526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f04075550 0x7f2f0419c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:28.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.094+0000 7f2f09600640 1 -- 172.21.15.16:0/2480336526 >> 172.21.15.16:0/2480336526 conn(0x7f2f040ff1e0 msgr2=0x7f2f040ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:28.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.094+0000 7f2f09600640 1 -- 172.21.15.16:0/2480336526 shutdown_connections 2024-08-31T06:18:28.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:28.094+0000 7f2f09600640 1 -- 172.21.15.16:0/2480336526 wait complete. 2024-08-31T06:18:28.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:28 smithi016 ceph-mon[17194]: pgmap v713: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 3.1 MiB/s wr, 230 op/s 2024-08-31T06:18:28.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:28 smithi184 ceph-mon[22331]: pgmap v713: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 3.1 MiB/s wr, 230 op/s 2024-08-31T06:18:29.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:28 smithi155 ceph-mon[21742]: pgmap v713: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 3.1 MiB/s wr, 230 op/s 2024-08-31T06:18:30.033 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T06:18:30.033 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T06:18:30.427 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:18:30.906 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:30 smithi016 ceph-mon[17194]: pgmap v714: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 3.5 MiB/s wr, 272 op/s 2024-08-31T06:18:30.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:30 smithi184 ceph-mon[22331]: pgmap v714: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 3.5 MiB/s wr, 272 op/s 2024-08-31T06:18:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.088+0000 7f9336e00640 1 -- 172.21.15.16:0/2292071452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9330109b90 msgr2=0x7f93301103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.088+0000 7f9336e00640 1 --2- 172.21.15.16:0/2292071452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9330109b90 0x7f93301103c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f9328009ca0 tx=0x7f932802a8c0 comp rx=0 tx=0).stop 2024-08-31T06:18:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.088+0000 7f9336e00640 1 -- 172.21.15.16:0/2292071452 shutdown_connections 2024-08-31T06:18:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.088+0000 7f9336e00640 1 --2- 172.21.15.16:0/2292071452 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9330109240 0x7f9330109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.088+0000 7f9336e00640 1 --2- 172.21.15.16:0/2292071452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9330109b90 0x7f93301103c0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.088+0000 7f9336e00640 1 --2- 172.21.15.16:0/2292071452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9330103900 0x7f9330103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.088+0000 7f9336e00640 1 -- 172.21.15.16:0/2292071452 >> 172.21.15.16:0/2292071452 conn(0x7f933007b5f0 msgr2=0x7f933007ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.088+0000 7f9336e00640 1 -- 172.21.15.16:0/2292071452 shutdown_connections 2024-08-31T06:18:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.088+0000 7f9336e00640 1 -- 172.21.15.16:0/2292071452 wait complete. 2024-08-31T06:18:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.089+0000 7f9336e00640 1 Processor -- start 2024-08-31T06:18:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.090+0000 7f9336e00640 1 -- start start 2024-08-31T06:18:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.090+0000 7f9336e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9330103900 0x7f933019c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:31.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.090+0000 7f9336e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9330109240 0x7f933019c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:31.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.090+0000 7f9336e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9330109b90 0x7f93301a1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:31.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.090+0000 7f9336e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9330113850 con 0x7f9330109240 2024-08-31T06:18:31.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.090+0000 7f9336e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f93301136d0 con 0x7f9330109b90 2024-08-31T06:18:31.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.090+0000 7f9336e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f93301139d0 con 0x7f9330103900 2024-08-31T06:18:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.090+0000 7f9327e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9330109240 0x7f933019c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.090+0000 7f9327e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9330109240 0x7f933019c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44486/0 (socket says 172.21.15.16:44486) 2024-08-31T06:18:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.090+0000 7f9327e00640 1 -- 172.21.15.16:0/3159564762 learned_addr learned my addr 172.21.15.16:0/3159564762 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.090+0000 7f932fe00640 1 --2- 172.21.15.16:0/3159564762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9330103900 0x7f933019c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.090+0000 7f9334e00640 1 --2- 172.21.15.16:0/3159564762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9330109b90 0x7f93301a1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.091+0000 7f932fe00640 1 -- 172.21.15.16:0/3159564762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9330109b90 msgr2=0x7f93301a1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.091+0000 7f932fe00640 1 --2- 172.21.15.16:0/3159564762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9330109b90 0x7f93301a1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.091+0000 7f932fe00640 1 -- 172.21.15.16:0/3159564762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9330109240 msgr2=0x7f933019c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:31.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.091+0000 7f932fe00640 1 --2- 172.21.15.16:0/3159564762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9330109240 0x7f933019c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:31.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.091+0000 7f932fe00640 1 -- 172.21.15.16:0/3159564762 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93301a2390 con 0x7f9330103900 2024-08-31T06:18:31.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.091+0000 7f9327e00640 1 --2- 172.21.15.16:0/3159564762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9330109240 0x7f933019c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:18:31.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.091+0000 7f932fe00640 1 --2- 172.21.15.16:0/3159564762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9330103900 0x7f933019c420 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f932000d6d0 tx=0x7f932000db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:31.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.092+0000 7f932d600640 1 -- 172.21.15.16:0/3159564762 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9320017070 con 0x7f9330103900 2024-08-31T06:18:31.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.092+0000 7f9336e00640 1 -- 172.21.15.16:0/3159564762 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93301a2680 con 0x7f9330103900 2024-08-31T06:18:31.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.092+0000 7f932d600640 1 -- 172.21.15.16:0/3159564762 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9320009820 con 0x7f9330103900 2024-08-31T06:18:31.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.092+0000 7f932d600640 1 -- 172.21.15.16:0/3159564762 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9320004c40 con 0x7f9330103900 2024-08-31T06:18:31.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.092+0000 7f9336e00640 1 -- 172.21.15.16:0/3159564762 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93301a8de0 con 0x7f9330103900 2024-08-31T06:18:31.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.095+0000 7f933da2e740 1 Processor -- start 2024-08-31T06:18:31.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.095+0000 7f933da2e740 1 -- start start 2024-08-31T06:18:31.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.095+0000 7f933da2e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dacd2c1350 0x55dacd667600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:31.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.095+0000 7f933da2e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dacd298490 0x55dacd667b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:31.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.095+0000 7f933da2e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dacd2a0670 0x55dacd668080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:31.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.095+0000 7f933da2e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dacd5dd750 con 0x55dacd298490 2024-08-31T06:18:31.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.095+0000 7f933da2e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55dacd5ccce0 con 0x55dacd2c1350 2024-08-31T06:18:31.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.095+0000 7f933da2e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55dacd5d0c90 con 0x55dacd2a0670 2024-08-31T06:18:31.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.096+0000 7f932d600640 1 -- 172.21.15.16:0/3159564762 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f9320005220 con 0x7f9330103900 2024-08-31T06:18:31.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.096+0000 7f9327e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dacd2a0670 0x55dacd668080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:31.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.096+0000 7f9327e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dacd2a0670 0x55dacd668080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:51732/0 (socket says 172.21.15.16:51732) 2024-08-31T06:18:31.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.096+0000 7f9327e00640 1 -- 172.21.15.16:0/1241842545 learned_addr learned my addr 172.21.15.16:0/1241842545 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:31.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.096+0000 7f9334e00640 1 --2- 172.21.15.16:0/1241842545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dacd2c1350 0x55dacd667600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:31.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.096+0000 7f932fe00640 1 --2- 172.21.15.16:0/1241842545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dacd298490 0x55dacd667b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:31.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.096+0000 7f9327e00640 1 -- 172.21.15.16:0/1241842545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dacd2c1350 msgr2=0x55dacd667600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:31.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.096+0000 7f9327e00640 1 --2- 172.21.15.16:0/1241842545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dacd2c1350 0x55dacd667600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:31.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.096+0000 7f9327e00640 1 -- 172.21.15.16:0/1241842545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dacd298490 msgr2=0x55dacd667b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:31.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.096+0000 7f9327e00640 1 --2- 172.21.15.16:0/1241842545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dacd298490 0x55dacd667b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:31.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.096+0000 7f9327e00640 1 -- 172.21.15.16:0/1241842545 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dacd32c930 con 0x55dacd2a0670 2024-08-31T06:18:31.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.097+0000 7f9327e00640 1 --2- 172.21.15.16:0/1241842545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dacd2a0670 0x55dacd668080 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f93180104a0 tx=0x7f9318010960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5051702263ed6a75 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:31.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.097+0000 7f9325600640 1 -- 172.21.15.16:0/1241842545 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9318005220 con 0x55dacd2a0670 2024-08-31T06:18:31.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.097+0000 7f9325600640 1 -- 172.21.15.16:0/1241842545 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9318004c50 con 0x55dacd2a0670 2024-08-31T06:18:31.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.097+0000 7f9325600640 1 -- 172.21.15.16:0/1241842545 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9318011a60 con 0x55dacd2a0670 2024-08-31T06:18:31.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.097+0000 7f932d600640 1 --2- 172.21.15.16:0/3159564762 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f93140b4ca0 0x7f93140b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:31.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.098+0000 7f932d600640 1 -- 172.21.15.16:0/3159564762 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8470+0+0 (secure 0 0 0) 0x7f93200e4fe0 con 0x7f9330103900 2024-08-31T06:18:31.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.098+0000 7f9334e00640 1 --2- 172.21.15.16:0/3159564762 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f93140b4ca0 0x7f93140b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:31.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.098+0000 7f933da2e740 1 -- 172.21.15.16:0/1241842545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dacd2a0670 msgr2=0x55dacd668080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:31.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.098+0000 7f933da2e740 1 --2- 172.21.15.16:0/1241842545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dacd2a0670 0x55dacd668080 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f93180104a0 tx=0x7f9318010960 comp rx=0 tx=0).stop 2024-08-31T06:18:31.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.098+0000 7f933da2e740 1 -- 172.21.15.16:0/1241842545 shutdown_connections 2024-08-31T06:18:31.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.098+0000 7f933da2e740 1 --2- 172.21.15.16:0/1241842545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dacd2a0670 0x55dacd668080 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:31.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.098+0000 7f933da2e740 1 --2- 172.21.15.16:0/1241842545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dacd298490 0x55dacd667b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:31.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.098+0000 7f933da2e740 1 --2- 172.21.15.16:0/1241842545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dacd2c1350 0x55dacd667600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:31.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.098+0000 7f933da2e740 1 -- 172.21.15.16:0/1241842545 >> 172.21.15.16:0/1241842545 conn(0x55dacd2b5740 msgr2=0x55dacd6654f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:31.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.098+0000 7f933da2e740 1 -- 172.21.15.16:0/1241842545 shutdown_connections 2024-08-31T06:18:31.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.098+0000 7f933da2e740 1 -- 172.21.15.16:0/1241842545 wait complete. 2024-08-31T06:18:31.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.099+0000 7f9334e00640 1 --2- 172.21.15.16:0/3159564762 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f93140b4ca0 0x7f93140b71b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f932802f500 tx=0x7f932802f370 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:31.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.099+0000 7f933da2e740 1 -- start start 2024-08-31T06:18:31.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.099+0000 7f933da2e740 1 Processor -- start 2024-08-31T06:18:31.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.100+0000 7f933da2e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dacd298490 0x55dacd673190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:31.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.100+0000 7f933da2e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dacd2a0670 0x55dacd6736d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:31.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.100+0000 7f933da2e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dacd2c1350 0x55dacd67f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:31.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.100+0000 7f933da2e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dacd42f890 con 0x55dacd2a0670 2024-08-31T06:18:31.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.100+0000 7f933da2e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55dacd553cd0 con 0x55dacd2c1350 2024-08-31T06:18:31.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.100+0000 7f933da2e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55dacd675de0 con 0x55dacd298490 2024-08-31T06:18:31.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.100+0000 7f933da2e740 10 client.0 _ll_register_callbacks cb 0x55dacd5c7ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:18:31.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.100+0000 7f9327e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dacd2a0670 0x55dacd6736d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:31.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.100+0000 7f932fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dacd298490 0x55dacd673190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:31.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.100+0000 7f9334e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dacd2c1350 0x55dacd67f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:31.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.100+0000 7f932fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dacd298490 0x55dacd673190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:51746/0 (socket says 172.21.15.16:51746) 2024-08-31T06:18:31.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.100+0000 7f932fe00640 1 -- 172.21.15.16:0/1185837130 learned_addr learned my addr 172.21.15.16:0/1185837130 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:31.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.100+0000 7f932fe00640 1 -- 172.21.15.16:0/1185837130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dacd2c1350 msgr2=0x55dacd67f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:31.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.100+0000 7f932fe00640 1 --2- 172.21.15.16:0/1185837130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dacd2c1350 0x55dacd67f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:31.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.100+0000 7f932fe00640 1 -- 172.21.15.16:0/1185837130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dacd2a0670 msgr2=0x55dacd6736d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:31.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.100+0000 7f932fe00640 1 --2- 172.21.15.16:0/1185837130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dacd2a0670 0x55dacd6736d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:31.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.100+0000 7f932fe00640 1 -- 172.21.15.16:0/1185837130 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dacd59d1b0 con 0x55dacd298490 2024-08-31T06:18:31.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.100+0000 7f9327e00640 1 --2- 172.21.15.16:0/1185837130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dacd2a0670 0x55dacd6736d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:18:31.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.101+0000 7f932fe00640 1 --2- 172.21.15.16:0/1185837130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dacd298490 0x55dacd673190 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f932000d6a0 tx=0x7f9320009f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eac16d63d6e78dd8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:31.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.101+0000 7f931e000640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:18:31.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.101+0000 7f931e000640 1 -- 172.21.15.16:0/1185837130 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9320020070 con 0x55dacd298490 2024-08-31T06:18:31.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.101+0000 7f931e000640 1 -- 172.21.15.16:0/1185837130 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8470+0+0 (secure 0 0 0) 0x7f93200a7a20 con 0x55dacd298490 2024-08-31T06:18:31.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.101+0000 7f933da2e740 1 -- 172.21.15.16:0/1185837130 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9318004c50 con 0x55dacd298490 2024-08-31T06:18:31.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.101+0000 7f931e000640 1 -- 172.21.15.16:0/1185837130 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f93200b4340 con 0x55dacd298490 2024-08-31T06:18:31.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.101+0000 7f933da2e740 10 client.28189 fetch_fsmap learned FSMap version 47 2024-08-31T06:18:31.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.101+0000 7f933da2e740 1 -- 172.21.15.16:0/1185837130 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55dacd681d70 con 0x55dacd298490 2024-08-31T06:18:31.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.102+0000 7f931e000640 1 -- 172.21.15.16:0/1185837130 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9318004c50 con 0x55dacd298490 2024-08-31T06:18:31.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.102+0000 7f931e000640 1 -- 172.21.15.16:0/1185837130 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 47) ==== 8354+0+0 (secure 0 0 0) 0x7f93200e70d0 con 0x55dacd298490 2024-08-31T06:18:31.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.102+0000 7f933da2e740 10 client.28189 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T06:18:31.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.102+0000 7f933da2e740 10 client.28189 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:18:31.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.102+0000 7f933da2e740 20 client.28189 populate_metadata read hostname 'smithi016' 2024-08-31T06:18:31.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.102+0000 7f933da2e740 1 --2- 172.21.15.16:0/1185837130 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55dacd27fcd0 0x55dacd6852b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:31.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.102+0000 7f933da2e740 4 client.28189 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:18:31.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.102+0000 7f933da2e740 1 -- 172.21.15.16:0/1185837130 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dacd5df6a0 con 0x55dacd27fcd0 2024-08-31T06:18:31.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.102+0000 7f9327e00640 1 --2- 172.21.15.16:0/1185837130 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55dacd27fcd0 0x55dacd6852b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:31.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.103+0000 7f9327e00640 1 --2- 172.21.15.16:0/1185837130 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55dacd27fcd0 0x55dacd6852b0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=498fcf22da921b2c server_cookie=b8e865e55f54afb3 in_seq=0 out_seq=0 2024-08-31T06:18:31.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.103+0000 7f931e000640 10 client.28189 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:18:31.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.104+0000 7f931e000640 1 -- 172.21.15.16:0/1185837130 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dacd5df6a0 con 0x55dacd27fcd0 2024-08-31T06:18:31.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.104+0000 7f931e000640 10 client.28189 handle_command_reply: tid=0 2024-08-31T06:18:31.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.104+0000 7f933da2e740 1 client.28189 shutdown 2024-08-31T06:18:31.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.104+0000 7f933da2e740 1 -- 172.21.15.16:0/1185837130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dacd298490 msgr2=0x55dacd673190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:31.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.104+0000 7f933da2e740 1 --2- 172.21.15.16:0/1185837130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dacd298490 0x55dacd673190 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f932000d6a0 tx=0x7f9320009f00 comp rx=0 tx=0).stop 2024-08-31T06:18:31.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.104+0000 7f933da2e740 1 -- 172.21.15.16:0/1185837130 shutdown_connections 2024-08-31T06:18:31.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.104+0000 7f933da2e740 1 --2- 172.21.15.16:0/1185837130 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55dacd27fcd0 0x55dacd6852b0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:31.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.104+0000 7f933da2e740 1 --2- 172.21.15.16:0/1185837130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dacd2c1350 0x55dacd67f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:31.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.104+0000 7f933da2e740 1 --2- 172.21.15.16:0/1185837130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dacd2a0670 0x55dacd6736d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:31.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.104+0000 7f933da2e740 1 --2- 172.21.15.16:0/1185837130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dacd298490 0x55dacd673190 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:31.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.104+0000 7f933da2e740 1 -- 172.21.15.16:0/1185837130 >> 172.21.15.16:0/1185837130 conn(0x55dacd2b5740 msgr2=0x55dacd66a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:31.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.104+0000 7f931e000640 0 client.28189 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:18:31.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.104+0000 7f933da2e740 1 -- 172.21.15.16:0/1185837130 shutdown_connections 2024-08-31T06:18:31.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.105+0000 7f933da2e740 1 -- 172.21.15.16:0/1185837130 wait complete. 2024-08-31T06:18:31.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.105+0000 7f933da2e740 20 client.28189 trim_cache size 0 max 16384 2024-08-31T06:18:31.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:30 smithi155 ceph-mon[21742]: pgmap v714: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 3.5 MiB/s wr, 272 op/s 2024-08-31T06:18:31.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.123+0000 7f933da2e740 1 Processor -- start 2024-08-31T06:18:31.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.123+0000 7f933da2e740 1 -- start start 2024-08-31T06:18:31.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.124+0000 7f933da2e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dacd298490 0x55dacd674bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:31.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.124+0000 7f933da2e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dacd2a0670 0x55dacd66c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:31.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.124+0000 7f933da2e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dacd2c1350 0x55dacd66cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:31.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.124+0000 7f933da2e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f93301b3f40 con 0x55dacd298490 2024-08-31T06:18:31.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.124+0000 7f933da2e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f93301b17b0 con 0x55dacd2a0670 2024-08-31T06:18:31.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.124+0000 7f933da2e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f93301af620 con 0x55dacd2c1350 2024-08-31T06:18:31.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.124+0000 7f932fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dacd298490 0x55dacd674bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:31.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.124+0000 7f9327e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dacd2a0670 0x55dacd66c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:31.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.124+0000 7f932fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dacd298490 0x55dacd674bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44510/0 (socket says 172.21.15.16:44510) 2024-08-31T06:18:31.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.124+0000 7f932fe00640 1 -- 172.21.15.16:0/227652577 learned_addr learned my addr 172.21.15.16:0/227652577 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:31.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.124+0000 7f9334e00640 1 --2- 172.21.15.16:0/227652577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dacd2c1350 0x55dacd66cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:31.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.124+0000 7f932fe00640 1 -- 172.21.15.16:0/227652577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dacd2c1350 msgr2=0x55dacd66cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:31.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.124+0000 7f932fe00640 1 --2- 172.21.15.16:0/227652577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dacd2c1350 0x55dacd66cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:31.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.124+0000 7f932fe00640 1 -- 172.21.15.16:0/227652577 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dacd2a0670 msgr2=0x55dacd66c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:31.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.124+0000 7f932fe00640 1 --2- 172.21.15.16:0/227652577 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dacd2a0670 0x55dacd66c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:31.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.124+0000 7f932fe00640 1 -- 172.21.15.16:0/227652577 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dacd674250 con 0x55dacd298490 2024-08-31T06:18:31.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.124+0000 7f9327e00640 1 --2- 172.21.15.16:0/227652577 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dacd2a0670 0x55dacd66c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:18:31.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.125+0000 7f932fe00640 1 --2- 172.21.15.16:0/227652577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dacd298490 0x55dacd674bc0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f93200e29e0 tx=0x7f93200e5260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2880ca111ca1c8c2 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:31.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.125+0000 7f9324c00640 1 -- 172.21.15.16:0/227652577 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9320020070 con 0x55dacd298490 2024-08-31T06:18:31.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.125+0000 7f9324c00640 1 -- 172.21.15.16:0/227652577 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93200cce50 con 0x55dacd298490 2024-08-31T06:18:31.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.125+0000 7f9324c00640 1 -- 172.21.15.16:0/227652577 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93200e8140 con 0x55dacd298490 2024-08-31T06:18:31.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.126+0000 7f933da2e740 1 -- 172.21.15.16:0/227652577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dacd298490 msgr2=0x55dacd674bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:31.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.126+0000 7f933da2e740 1 --2- 172.21.15.16:0/227652577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dacd298490 0x55dacd674bc0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f93200e29e0 tx=0x7f93200e5260 comp rx=0 tx=0).stop 2024-08-31T06:18:31.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.126+0000 7f933da2e740 1 -- 172.21.15.16:0/227652577 shutdown_connections 2024-08-31T06:18:31.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.126+0000 7f933da2e740 1 --2- 172.21.15.16:0/227652577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dacd2c1350 0x55dacd66cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:31.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.126+0000 7f933da2e740 1 --2- 172.21.15.16:0/227652577 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dacd2a0670 0x55dacd66c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:31.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.126+0000 7f933da2e740 1 --2- 172.21.15.16:0/227652577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dacd298490 0x55dacd674bc0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:31.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.126+0000 7f933da2e740 1 -- 172.21.15.16:0/227652577 >> 172.21.15.16:0/227652577 conn(0x55dacd2b5740 msgr2=0x55dacd6744b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:31.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.126+0000 7f933da2e740 1 -- 172.21.15.16:0/227652577 shutdown_connections 2024-08-31T06:18:31.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.126+0000 7f933da2e740 1 -- 172.21.15.16:0/227652577 wait complete. 2024-08-31T06:18:31.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.126+0000 7f933da2e740 1 -- start start 2024-08-31T06:18:31.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.127+0000 7f933da2e740 1 Processor -- start 2024-08-31T06:18:31.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.127+0000 7f933da2e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dacd298490 0x55dacd677200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:31.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.127+0000 7f933da2e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dacd2a0670 0x55dacd677740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:31.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.127+0000 7f933da2e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dacd2c1350 0x55dacd677c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:31.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.127+0000 7f933da2e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f93301afc70 con 0x55dacd2c1350 2024-08-31T06:18:31.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.127+0000 7f933da2e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f93301b10e0 con 0x55dacd2a0670 2024-08-31T06:18:31.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.127+0000 7f933da2e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f93301b0340 con 0x55dacd298490 2024-08-31T06:18:31.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.127+0000 7f933da2e740 10 client.0 _ll_register_callbacks cb 0x7f93200057f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:18:31.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.128+0000 7f932fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dacd298490 0x55dacd677200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:31.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.128+0000 7f9334e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dacd2c1350 0x55dacd677c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:31.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.128+0000 7f932fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dacd298490 0x55dacd677200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:51770/0 (socket says 172.21.15.16:51770) 2024-08-31T06:18:31.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.128+0000 7f932fe00640 1 -- 172.21.15.16:0/79476847 learned_addr learned my addr 172.21.15.16:0/79476847 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:31.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.128+0000 7f9327e00640 1 --2- 172.21.15.16:0/79476847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dacd2a0670 0x55dacd677740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:31.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.128+0000 7f932fe00640 1 -- 172.21.15.16:0/79476847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dacd2a0670 msgr2=0x55dacd677740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:31.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.128+0000 7f932fe00640 1 --2- 172.21.15.16:0/79476847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dacd2a0670 0x55dacd677740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:31.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.128+0000 7f932fe00640 1 -- 172.21.15.16:0/79476847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dacd2c1350 msgr2=0x55dacd677c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:31.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.128+0000 7f932fe00640 1 --2- 172.21.15.16:0/79476847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dacd2c1350 0x55dacd677c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:31.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.128+0000 7f932fe00640 1 -- 172.21.15.16:0/79476847 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dacd6eeeb0 con 0x55dacd298490 2024-08-31T06:18:31.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.128+0000 7f932fe00640 1 --2- 172.21.15.16:0/79476847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dacd298490 0x55dacd677200 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f93200e2d40 tx=0x7f93200d23e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdfb8981c21667b5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:31.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.128+0000 7f931e000640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:18:31.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.129+0000 7f931e000640 1 -- 172.21.15.16:0/79476847 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9320020070 con 0x55dacd298490 2024-08-31T06:18:31.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.129+0000 7f931e000640 1 -- 172.21.15.16:0/79476847 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8470+0+0 (secure 0 0 0) 0x7f93200e8350 con 0x55dacd298490 2024-08-31T06:18:31.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.129+0000 7f933da2e740 1 -- 172.21.15.16:0/79476847 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93200cce50 con 0x55dacd298490 2024-08-31T06:18:31.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.129+0000 7f931e000640 1 -- 172.21.15.16:0/79476847 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f93200ab790 con 0x55dacd298490 2024-08-31T06:18:31.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.129+0000 7f933da2e740 10 client.28195 fetch_fsmap learned FSMap version 47 2024-08-31T06:18:31.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.129+0000 7f933da2e740 1 -- 172.21.15.16:0/79476847 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55dacd6f0170 con 0x55dacd298490 2024-08-31T06:18:31.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.129+0000 7f931e000640 1 -- 172.21.15.16:0/79476847 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93200cce50 con 0x55dacd298490 2024-08-31T06:18:31.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.129+0000 7f931e000640 1 -- 172.21.15.16:0/79476847 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 47) ==== 8354+0+0 (secure 0 0 0) 0x7f93200aa9f0 con 0x55dacd298490 2024-08-31T06:18:31.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.129+0000 7f933da2e740 10 client.28195 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T06:18:31.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.129+0000 7f933da2e740 10 client.28195 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:18:31.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.129+0000 7f933da2e740 20 client.28195 populate_metadata read hostname 'smithi016' 2024-08-31T06:18:31.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.130+0000 7f933da2e740 1 --2- 172.21.15.16:0/79476847 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55dacd255080 0x55dacd6f26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:31.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.130+0000 7f933da2e740 4 client.28195 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T06:18:31.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.130+0000 7f933da2e740 1 -- 172.21.15.16:0/79476847 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55dacd5db140 con 0x55dacd255080 2024-08-31T06:18:31.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.130+0000 7f9327e00640 1 --2- 172.21.15.16:0/79476847 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55dacd255080 0x55dacd6f26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:31.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.130+0000 7f9327e00640 1 --2- 172.21.15.16:0/79476847 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55dacd255080 0x55dacd6f26a0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec4ffad93c80174 server_cookie=9719674c2d366826 in_seq=0 out_seq=0 2024-08-31T06:18:31.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.130+0000 7f931e000640 10 client.28195 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:18:31.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.132+0000 7f931e000640 1 -- 172.21.15.16:0/79476847 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55dacd5db140 con 0x55dacd255080 2024-08-31T06:18:31.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.132+0000 7f931e000640 10 client.28195 handle_command_reply: tid=0 2024-08-31T06:18:31.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.132+0000 7f933da2e740 1 client.28195 shutdown 2024-08-31T06:18:31.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.132+0000 7f933da2e740 1 -- 172.21.15.16:0/79476847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dacd298490 msgr2=0x55dacd677200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:31.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.132+0000 7f933da2e740 1 --2- 172.21.15.16:0/79476847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dacd298490 0x55dacd677200 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f93200e2d40 tx=0x7f93200d23e0 comp rx=0 tx=0).stop 2024-08-31T06:18:31.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.132+0000 7f933da2e740 1 -- 172.21.15.16:0/79476847 shutdown_connections 2024-08-31T06:18:31.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.132+0000 7f933da2e740 1 --2- 172.21.15.16:0/79476847 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55dacd255080 0x55dacd6f26a0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:31.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.132+0000 7f933da2e740 1 --2- 172.21.15.16:0/79476847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dacd2c1350 0x55dacd677c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:31.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.132+0000 7f933da2e740 1 --2- 172.21.15.16:0/79476847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dacd2a0670 0x55dacd677740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:31.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.132+0000 7f933da2e740 1 --2- 172.21.15.16:0/79476847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dacd298490 0x55dacd677200 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:31.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.132+0000 7f931e000640 0 client.28195 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:18:31.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.132+0000 7f933da2e740 1 -- 172.21.15.16:0/79476847 >> 172.21.15.16:0/79476847 conn(0x55dacd2b5740 msgr2=0x55dacd674de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:31.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.133+0000 7f933da2e740 1 -- 172.21.15.16:0/79476847 shutdown_connections 2024-08-31T06:18:31.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.133+0000 7f933da2e740 1 -- 172.21.15.16:0/79476847 wait complete. 2024-08-31T06:18:31.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.133+0000 7f933da2e740 20 client.28195 trim_cache size 0 max 16384 2024-08-31T06:18:31.138 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:18:31.138 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-08-31T06:18:31.138 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "0e44eb18-439c-46e4-a298-7d4d2ecd0b4a", 2024-08-31T06:18:31.138 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-08-31T06:18:31.138 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:18:31.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.133+0000 7f9336e00640 1 -- 172.21.15.16:0/3159564762 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f93140b4ca0 msgr2=0x7f93140b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:31.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.133+0000 7f9336e00640 1 --2- 172.21.15.16:0/3159564762 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f93140b4ca0 0x7f93140b71b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f932802f500 tx=0x7f932802f370 comp rx=0 tx=0).stop 2024-08-31T06:18:31.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.133+0000 7f9336e00640 1 -- 172.21.15.16:0/3159564762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9330103900 msgr2=0x7f933019c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:31.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.133+0000 7f9336e00640 1 --2- 172.21.15.16:0/3159564762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9330103900 0x7f933019c420 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f932000d6d0 tx=0x7f932000db90 comp rx=0 tx=0).stop 2024-08-31T06:18:31.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.134+0000 7f9336e00640 1 -- 172.21.15.16:0/3159564762 shutdown_connections 2024-08-31T06:18:31.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.134+0000 7f9336e00640 1 --2- 172.21.15.16:0/3159564762 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f93140b4ca0 0x7f93140b71b0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:31.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.134+0000 7f9336e00640 1 --2- 172.21.15.16:0/3159564762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9330109b90 0x7f93301a1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:31.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.134+0000 7f9336e00640 1 --2- 172.21.15.16:0/3159564762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9330109240 0x7f933019c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:31.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.134+0000 7f9336e00640 1 --2- 172.21.15.16:0/3159564762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9330103900 0x7f933019c420 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:31.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.134+0000 7f9336e00640 1 -- 172.21.15.16:0/3159564762 >> 172.21.15.16:0/3159564762 conn(0x7f933007b5f0 msgr2=0x7f9330107050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:31.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.134+0000 7f9336e00640 1 -- 172.21.15.16:0/3159564762 shutdown_connections 2024-08-31T06:18:31.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:31.134+0000 7f9336e00640 1 -- 172.21.15.16:0/3159564762 wait complete. 2024-08-31T06:18:31.622 INFO:journalctl@ceph.mds.b.smithi155.stdout:Aug 31 06:18:31 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-b[53353]: 2024-08-31T06:18:31.157+0000 7f6c22b03640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000000f [...12,head] /volumes/qa/sv_1/ba5423bc-5c83-4d64-ae1c-ad8850110ce4/postgres/data/pg_subtrans/ auth{1=1,2=1} v3373 ap=3 f(v13 m2024-08-31T06:17:14.533742+0000 6=6+0) n(v35 rc2024-08-31T06:17:14.537742+0000 b1466368 7=6+1) old_inodes=6 (inest sync->mix g=1,2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:8531 nref=3)) | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x558498a41600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000000f:[<0x10000000006/pg_subtrans v252>,<0x10000000005/data v106>,<0x100000001fb/postgres v19>,<0x100000001fa/ba5423bc-5c83-4d64-ae1c-ad8850110ce4 v35>,<0x10000000001/sv_1 v56>,<0x10000000000/qa v44>,<0x1/volumes v40>]//[]","memoryvalue":"(2)0x1000000000f:[<0x10000000006/pg_subtrans v3373>,<0x10000000005/data v976>,<0x100000001fb/postgres v611>,<0x100000001fa/ba5423bc-5c83-4d64-ae1c-ad8850110ce4 v634>,<0x10000000001/sv_1 v635>,<0x10000000000/qa v637>,<0x1/volumes v583>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-08-31T06:16:41.777254+0000 5=5+0)","ondisk_value.rstat":"n(v0 rc2024-08-31T06:17:14.537742+0000 b1466368 7=6+1)","memory_value.dirstat":"f(v13 m2024-08-31T06:17:14.533742+0000 6=6+0)","memory_value.rstat":"n(v35 rc2024-08-31T06:17:14.537742+0000 b1466368 7=6+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-08-31T06:18:31.626 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 665.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-31T06:18:31.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:31 smithi184 ceph-mon[22331]: scrub queued for path: / 2024-08-31T06:18:31.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:31 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:18:31.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:31 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:18:31.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:31 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:18:31.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:31 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:18:31.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:31 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:18:31.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:31 smithi184 ceph-mon[22331]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_1/ba5423bc-5c83-4d64-ae1c-ad8850110ce4/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-08-31T06:18:31.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:31 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:18:31.974 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:31 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:18:32.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:31 smithi155 ceph-mon[21742]: scrub queued for path: / 2024-08-31T06:18:32.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:31 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:18:32.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:31 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:18:32.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:31 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:18:32.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:31 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:18:32.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:31 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:18:32.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:31 smithi155 ceph-mon[21742]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_1/ba5423bc-5c83-4d64-ae1c-ad8850110ce4/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-08-31T06:18:32.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:31 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:18:32.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:31 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:18:32.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:31 smithi016 ceph-mon[17194]: scrub queued for path: / 2024-08-31T06:18:32.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:31 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:18:32.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:31 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:18:32.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:31 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:18:32.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:31 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:18:32.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:31 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:18:32.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:31 smithi016 ceph-mon[17194]: Scrub error on inode 0x1000000000f (/volumes/qa/sv_1/ba5423bc-5c83-4d64-ae1c-ad8850110ce4/postgres/data/pg_subtrans) see mds.b log and `damage ls` output for details 2024-08-31T06:18:32.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:31 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:18:32.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:31 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:18:32.203 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T06:18:32.582 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:18:32.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:32 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:18:32.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:32 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:18:32.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:32 smithi184 ceph-mon[22331]: pgmap v715: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 2.5 MiB/s wr, 240 op/s 2024-08-31T06:18:33.035 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:32 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:18:33.035 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:32 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:18:33.035 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:32 smithi016 ceph-mon[17194]: pgmap v715: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 2.5 MiB/s wr, 240 op/s 2024-08-31T06:18:33.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:32 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:18:33.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:32 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:18:33.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:32 smithi155 ceph-mon[21742]: pgmap v715: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 2.5 MiB/s wr, 240 op/s 2024-08-31T06:18:33.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.213+0000 7fe4e3600640 1 -- 172.21.15.16:0/2580635112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4dc109260 msgr2=0x7fe4dc109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:33.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.213+0000 7fe4e3600640 1 --2- 172.21.15.16:0/2580635112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4dc109260 0x7fe4dc109670 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8009900 tx=0x7fe4c802a3f0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.213+0000 7fe4e3600640 1 -- 172.21.15.16:0/2580635112 shutdown_connections 2024-08-31T06:18:33.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.213+0000 7fe4e3600640 1 --2- 172.21.15.16:0/2580635112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4dc109bb0 0x7fe4dc1103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.213+0000 7fe4e3600640 1 --2- 172.21.15.16:0/2580635112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4dc109260 0x7fe4dc109670 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.213+0000 7fe4e3600640 1 --2- 172.21.15.16:0/2580635112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4dc103920 0x7fe4dc103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.213+0000 7fe4e3600640 1 -- 172.21.15.16:0/2580635112 >> 172.21.15.16:0/2580635112 conn(0x7fe4dc0ff3b0 msgr2=0x7fe4dc101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:33.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.213+0000 7fe4e3600640 1 -- 172.21.15.16:0/2580635112 shutdown_connections 2024-08-31T06:18:33.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.213+0000 7fe4e3600640 1 -- 172.21.15.16:0/2580635112 wait complete. 2024-08-31T06:18:33.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.215+0000 7fe4e3600640 1 Processor -- start 2024-08-31T06:18:33.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.215+0000 7fe4e3600640 1 -- start start 2024-08-31T06:18:33.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.216+0000 7fe4e3600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4dc103920 0x7fe4dc19a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:33.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.216+0000 7fe4e3600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4dc109260 0x7fe4dc19a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:33.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.216+0000 7fe4e3600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4dc109bb0 0x7fe4dc19fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:33.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.216+0000 7fe4e3600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc1138c0 con 0x7fe4dc109260 2024-08-31T06:18:33.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.216+0000 7fe4e3600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc113740 con 0x7fe4dc103920 2024-08-31T06:18:33.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.216+0000 7fe4e3600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc113a40 con 0x7fe4dc109bb0 2024-08-31T06:18:33.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.216+0000 7fe4dbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4dc109260 0x7fe4dc19a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:33.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.216+0000 7fe4dbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4dc109260 0x7fe4dc19a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44534/0 (socket says 172.21.15.16:44534) 2024-08-31T06:18:33.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.216+0000 7fe4dbe00640 1 -- 172.21.15.16:0/1486275907 learned_addr learned my addr 172.21.15.16:0/1486275907 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:33.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.216+0000 7fe4e0c00640 1 --2- 172.21.15.16:0/1486275907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4dc103920 0x7fe4dc19a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:33.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.216+0000 7fe4e1600640 1 --2- 172.21.15.16:0/1486275907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4dc109bb0 0x7fe4dc19fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:33.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.216+0000 7fe4dbe00640 1 -- 172.21.15.16:0/1486275907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4dc109bb0 msgr2=0x7fe4dc19fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:33.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.216+0000 7fe4dbe00640 1 --2- 172.21.15.16:0/1486275907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4dc109bb0 0x7fe4dc19fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.216+0000 7fe4dbe00640 1 -- 172.21.15.16:0/1486275907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4dc103920 msgr2=0x7fe4dc19a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:33.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.216+0000 7fe4dbe00640 1 --2- 172.21.15.16:0/1486275907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4dc103920 0x7fe4dc19a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.217+0000 7fe4dbe00640 1 -- 172.21.15.16:0/1486275907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4dc1a01e0 con 0x7fe4dc109260 2024-08-31T06:18:33.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.217+0000 7fe4dbe00640 1 --2- 172.21.15.16:0/1486275907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4dc109260 0x7fe4dc19a810 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8005ab0 tx=0x7fe4c8003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:33.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.217+0000 7fe4d9600640 1 -- 172.21.15.16:0/1486275907 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c803f070 con 0x7fe4dc109260 2024-08-31T06:18:33.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.217+0000 7fe4e3600640 1 -- 172.21.15.16:0/1486275907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4dc1a0470 con 0x7fe4dc109260 2024-08-31T06:18:33.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.217+0000 7fe4d9600640 1 -- 172.21.15.16:0/1486275907 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4c8031b50 con 0x7fe4dc109260 2024-08-31T06:18:33.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.217+0000 7fe4d9600640 1 -- 172.21.15.16:0/1486275907 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8035030 con 0x7fe4dc109260 2024-08-31T06:18:33.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.217+0000 7fe4e3600640 1 -- 172.21.15.16:0/1486275907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4dc1a6c30 con 0x7fe4dc109260 2024-08-31T06:18:33.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.221+0000 7fe4d9600640 1 -- 172.21.15.16:0/1486275907 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fe4c8042070 con 0x7fe4dc109260 2024-08-31T06:18:33.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.222+0000 7fe4ea246740 1 Processor -- start 2024-08-31T06:18:33.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.222+0000 7fe4ea246740 1 -- start start 2024-08-31T06:18:33.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.223+0000 7fe4d9600640 1 --2- 172.21.15.16:0/1486275907 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe4c40b4ca0 0x7fe4c40b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:33.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.223+0000 7fe4ea246740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9f6341070 0x55c9f66e6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:33.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.223+0000 7fe4ea246740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9f63181b0 0x55c9f66e6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:33.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.223+0000 7fe4ea246740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9f6320390 0x55c9f66e73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:33.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.223+0000 7fe4ea246740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c9f665d770 con 0x55c9f6320390 2024-08-31T06:18:33.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.223+0000 7fe4ea246740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c9f664cd00 con 0x55c9f6341070 2024-08-31T06:18:33.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.223+0000 7fe4ea246740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c9f6650cb0 con 0x55c9f63181b0 2024-08-31T06:18:33.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.223+0000 7fe4d9600640 1 -- 172.21.15.16:0/1486275907 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8470+0+0 (secure 0 0 0) 0x7fe4c8105b00 con 0x7fe4dc109260 2024-08-31T06:18:33.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.223+0000 7fe4e0c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9f6341070 0x55c9f66e6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:33.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.223+0000 7fe4e1600640 1 --2- 172.21.15.16:0/1486275907 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe4c40b4ca0 0x7fe4c40b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:33.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.223+0000 7fe4e0c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9f6341070 0x55c9f66e6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:53914/0 (socket says 172.21.15.16:53914) 2024-08-31T06:18:33.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.223+0000 7fe4e0c00640 1 -- 172.21.15.16:0/3540927681 learned_addr learned my addr 172.21.15.16:0/3540927681 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:33.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.224+0000 7fe4e0c00640 1 -- 172.21.15.16:0/3540927681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9f63181b0 msgr2=0x55c9f66e6e90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T06:18:33.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.224+0000 7fe4e1600640 1 --2- 172.21.15.16:0/3540927681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9f6320390 0x55c9f66e73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:33.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.224+0000 7fe4e0c00640 1 --2- 172.21.15.16:0/3540927681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9f63181b0 0x55c9f66e6e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.224+0000 7fe4e0c00640 1 -- 172.21.15.16:0/3540927681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9f6320390 msgr2=0x55c9f66e73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:33.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.224+0000 7fe4e0c00640 1 --2- 172.21.15.16:0/3540927681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9f6320390 0x55c9f66e73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.224+0000 7fe4e0c00640 1 -- 172.21.15.16:0/3540927681 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9f63ac650 con 0x55c9f6341070 2024-08-31T06:18:33.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.224+0000 7fe4e1600640 1 --2- 172.21.15.16:0/3540927681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9f6320390 0x55c9f66e73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:18:33.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.224+0000 7fe4e0c00640 1 --2- 172.21.15.16:0/3540927681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9f6341070 0x55c9f66e6950 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0009d20 tx=0x7fe4d0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40289a5752bc01b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:33.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.225+0000 7fe4e1600640 1 --2- 172.21.15.16:0/1486275907 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe4c40b4ca0 0x7fe4c40b71b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4005fd0 tx=0x7fe4d4009bf0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:33.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.225+0000 7fe4cdc00640 1 -- 172.21.15.16:0/3540927681 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d001e070 con 0x55c9f6341070 2024-08-31T06:18:33.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.225+0000 7fe4cdc00640 1 -- 172.21.15.16:0/3540927681 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4d0019ae0 con 0x55c9f6341070 2024-08-31T06:18:33.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.225+0000 7fe4cdc00640 1 -- 172.21.15.16:0/3540927681 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d000ea50 con 0x55c9f6341070 2024-08-31T06:18:33.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.226+0000 7fe4ea246740 1 -- 172.21.15.16:0/3540927681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9f6341070 msgr2=0x55c9f66e6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:33.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.226+0000 7fe4ea246740 1 --2- 172.21.15.16:0/3540927681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9f6341070 0x55c9f66e6950 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0009d20 tx=0x7fe4d0011810 comp rx=0 tx=0).stop 2024-08-31T06:18:33.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.226+0000 7fe4ea246740 1 -- 172.21.15.16:0/3540927681 shutdown_connections 2024-08-31T06:18:33.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.226+0000 7fe4ea246740 1 --2- 172.21.15.16:0/3540927681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9f63181b0 0x55c9f66e6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.226+0000 7fe4ea246740 1 --2- 172.21.15.16:0/3540927681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9f6320390 0x55c9f66e73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.226+0000 7fe4ea246740 1 --2- 172.21.15.16:0/3540927681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9f6341070 0x55c9f66e6950 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.226+0000 7fe4ea246740 1 -- 172.21.15.16:0/3540927681 >> 172.21.15.16:0/3540927681 conn(0x55c9f6335460 msgr2=0x55c9f66e4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:33.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.226+0000 7fe4ea246740 1 -- 172.21.15.16:0/3540927681 shutdown_connections 2024-08-31T06:18:33.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.226+0000 7fe4ea246740 1 -- 172.21.15.16:0/3540927681 wait complete. 2024-08-31T06:18:33.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.227+0000 7fe4ea246740 1 -- start start 2024-08-31T06:18:33.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.227+0000 7fe4ea246740 1 Processor -- start 2024-08-31T06:18:33.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.228+0000 7fe4ea246740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9f63181b0 0x55c9f66f68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:33.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.228+0000 7fe4ea246740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9f6320390 0x55c9f66f6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:33.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.228+0000 7fe4ea246740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9f6341070 0x55c9f67027f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:33.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.228+0000 7fe4ea246740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c9f64af700 con 0x55c9f6341070 2024-08-31T06:18:33.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.228+0000 7fe4ea246740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c9f65d3cb0 con 0x55c9f63181b0 2024-08-31T06:18:33.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.228+0000 7fe4ea246740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c9f66f9460 con 0x55c9f6320390 2024-08-31T06:18:33.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.228+0000 7fe4ea246740 10 client.0 _ll_register_callbacks cb 0x55c9f65f9430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:18:33.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.228+0000 7fe4e0c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9f63181b0 0x55c9f66f68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:33.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.228+0000 7fe4dbe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9f6320390 0x55c9f66f6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:33.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.228+0000 7fe4dbe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9f6320390 0x55c9f66f6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:51806/0 (socket says 172.21.15.16:51806) 2024-08-31T06:18:33.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.228+0000 7fe4dbe00640 1 -- 172.21.15.16:0/486349778 learned_addr learned my addr 172.21.15.16:0/486349778 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:33.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.228+0000 7fe4e0c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9f63181b0 0x55c9f66f68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:53918/0 (socket says 172.21.15.16:53918) 2024-08-31T06:18:33.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.228+0000 7fe4e1600640 1 --2- 172.21.15.16:0/486349778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9f6341070 0x55c9f67027f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:33.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.229+0000 7fe4e0c00640 1 -- 172.21.15.16:0/486349778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9f6320390 msgr2=0x55c9f66f6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:33.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.229+0000 7fe4e0c00640 1 --2- 172.21.15.16:0/486349778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9f6320390 0x55c9f66f6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.229+0000 7fe4e0c00640 1 -- 172.21.15.16:0/486349778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9f6341070 msgr2=0x55c9f67027f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:33.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.229+0000 7fe4e0c00640 1 --2- 172.21.15.16:0/486349778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9f6341070 0x55c9f67027f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.229+0000 7fe4e0c00640 1 -- 172.21.15.16:0/486349778 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9f6614870 con 0x55c9f63181b0 2024-08-31T06:18:33.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.229+0000 7fe4dbe00640 1 --2- 172.21.15.16:0/486349778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9f6320390 0x55c9f66f6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:18:33.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.229+0000 7fe4e0c00640 1 --2- 172.21.15.16:0/486349778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9f63181b0 0x55c9f66f68c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0011d10 tx=0x7fe4d0004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2df7aac405fa5b1b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:33.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.229+0000 7fe4c2000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:18:33.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.229+0000 7fe4c2000640 1 -- 172.21.15.16:0/486349778 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d001e050 con 0x55c9f63181b0 2024-08-31T06:18:33.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.229+0000 7fe4ea246740 1 -- 172.21.15.16:0/486349778 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4d0019ae0 con 0x55c9f63181b0 2024-08-31T06:18:33.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.229+0000 7fe4c2000640 1 -- 172.21.15.16:0/486349778 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8470+0+0 (secure 0 0 0) 0x7fe4d0019630 con 0x55c9f63181b0 2024-08-31T06:18:33.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.230+0000 7fe4c2000640 1 -- 172.21.15.16:0/486349778 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fe4d00098f0 con 0x55c9f63181b0 2024-08-31T06:18:33.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.230+0000 7fe4ea246740 10 client.28196 fetch_fsmap learned FSMap version 47 2024-08-31T06:18:33.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.230+0000 7fe4ea246740 1 -- 172.21.15.16:0/486349778 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55c9f6705430 con 0x55c9f63181b0 2024-08-31T06:18:33.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.230+0000 7fe4c2000640 1 -- 172.21.15.16:0/486349778 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4d0019ae0 con 0x55c9f63181b0 2024-08-31T06:18:33.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.230+0000 7fe4c2000640 1 -- 172.21.15.16:0/486349778 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 47) ==== 8354+0+0 (secure 0 0 0) 0x7fe4d00046c0 con 0x55c9f63181b0 2024-08-31T06:18:33.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.231+0000 7fe4ea246740 10 client.28196 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T06:18:33.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.231+0000 7fe4ea246740 10 client.28196 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:18:33.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.231+0000 7fe4ea246740 20 client.28196 populate_metadata read hostname 'smithi016' 2024-08-31T06:18:33.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.231+0000 7fe4ea246740 1 --2- 172.21.15.16:0/486349778 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55c9f62ff9f0 0x55c9f6708790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:33.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.231+0000 7fe4ea246740 4 client.28196 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:18:33.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.231+0000 7fe4ea246740 1 -- 172.21.15.16:0/486349778 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9f665f6c0 con 0x55c9f62ff9f0 2024-08-31T06:18:33.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.231+0000 7fe4dbe00640 1 --2- 172.21.15.16:0/486349778 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55c9f62ff9f0 0x55c9f6708790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:33.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.232+0000 7fe4dbe00640 1 --2- 172.21.15.16:0/486349778 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55c9f62ff9f0 0x55c9f6708790 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9706df2dac3373c2 server_cookie=f24e32a18e8711dd in_seq=0 out_seq=0 2024-08-31T06:18:33.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.232+0000 7fe4c2000640 10 client.28196 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:18:33.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.233+0000 7fe4c2000640 1 -- 172.21.15.16:0/486349778 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c9f665f6c0 con 0x55c9f62ff9f0 2024-08-31T06:18:33.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.233+0000 7fe4c2000640 10 client.28196 handle_command_reply: tid=0 2024-08-31T06:18:33.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.233+0000 7fe4ea246740 1 client.28196 shutdown 2024-08-31T06:18:33.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.233+0000 7fe4ea246740 1 -- 172.21.15.16:0/486349778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9f63181b0 msgr2=0x55c9f66f68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:33.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.233+0000 7fe4ea246740 1 --2- 172.21.15.16:0/486349778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9f63181b0 0x55c9f66f68c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0011d10 tx=0x7fe4d0004270 comp rx=0 tx=0).stop 2024-08-31T06:18:33.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.233+0000 7fe4ea246740 1 -- 172.21.15.16:0/486349778 shutdown_connections 2024-08-31T06:18:33.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.233+0000 7fe4ea246740 1 --2- 172.21.15.16:0/486349778 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55c9f62ff9f0 0x55c9f6708790 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.233+0000 7fe4ea246740 1 --2- 172.21.15.16:0/486349778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9f6320390 0x55c9f66f6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.233+0000 7fe4ea246740 1 --2- 172.21.15.16:0/486349778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9f6341070 0x55c9f67027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.233+0000 7fe4c2000640 0 client.28196 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:18:33.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.233+0000 7fe4ea246740 1 --2- 172.21.15.16:0/486349778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9f63181b0 0x55c9f66f68c0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.233+0000 7fe4ea246740 1 -- 172.21.15.16:0/486349778 >> 172.21.15.16:0/486349778 conn(0x55c9f6335460 msgr2=0x55c9f66e9aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:33.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.233+0000 7fe4ea246740 1 -- 172.21.15.16:0/486349778 shutdown_connections 2024-08-31T06:18:33.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.233+0000 7fe4ea246740 1 -- 172.21.15.16:0/486349778 wait complete. 2024-08-31T06:18:33.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.234+0000 7fe4ea246740 20 client.28196 trim_cache size 0 max 16384 2024-08-31T06:18:33.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.252+0000 7fe4ea246740 1 Processor -- start 2024-08-31T06:18:33.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.252+0000 7fe4ea246740 1 -- start start 2024-08-31T06:18:33.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.253+0000 7fe4ea246740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9f63181b0 0x55c9f66e34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:33.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.253+0000 7fe4ea246740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9f6320390 0x55c9f67026f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:33.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.253+0000 7fe4ea246740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9f6341070 0x55c9f6702c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:33.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.253+0000 7fe4ea246740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc1afad0 con 0x55c9f63181b0 2024-08-31T06:18:33.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.253+0000 7fe4ea246740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc1ad340 con 0x55c9f6341070 2024-08-31T06:18:33.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.253+0000 7fe4ea246740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc1ab1b0 con 0x55c9f6320390 2024-08-31T06:18:33.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.253+0000 7fe4e0c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9f63181b0 0x55c9f66e34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:33.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.253+0000 7fe4dbe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9f6320390 0x55c9f67026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:33.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.253+0000 7fe4e0c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9f63181b0 0x55c9f66e34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44556/0 (socket says 172.21.15.16:44556) 2024-08-31T06:18:33.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.253+0000 7fe4e0c00640 1 -- 172.21.15.16:0/272082691 learned_addr learned my addr 172.21.15.16:0/272082691 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:33.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.253+0000 7fe4e1600640 1 --2- 172.21.15.16:0/272082691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9f6341070 0x55c9f6702c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:33.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.254+0000 7fe4dbe00640 1 -- 172.21.15.16:0/272082691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9f6341070 msgr2=0x55c9f6702c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:33.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.254+0000 7fe4dbe00640 1 --2- 172.21.15.16:0/272082691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9f6341070 0x55c9f6702c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.254+0000 7fe4dbe00640 1 -- 172.21.15.16:0/272082691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9f63181b0 msgr2=0x55c9f66e34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:33.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.254+0000 7fe4dbe00640 1 --2- 172.21.15.16:0/272082691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9f63181b0 0x55c9f66e34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.254+0000 7fe4dbe00640 1 -- 172.21.15.16:0/272082691 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9f66e9750 con 0x55c9f6320390 2024-08-31T06:18:33.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.254+0000 7fe4e0c00640 1 --2- 172.21.15.16:0/272082691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9f63181b0 0x55c9f66e34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:18:33.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.254+0000 7fe4dbe00640 1 --2- 172.21.15.16:0/272082691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9f6320390 0x55c9f67026f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8031440 tx=0x7fe4c80ef040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5788ca844d2e7d3 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:33.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.254+0000 7fe4cd200640 1 -- 172.21.15.16:0/272082691 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8049040 con 0x55c9f6320390 2024-08-31T06:18:33.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.254+0000 7fe4cd200640 1 -- 172.21.15.16:0/272082691 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4c80d0a30 con 0x55c9f6320390 2024-08-31T06:18:33.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.254+0000 7fe4cd200640 1 -- 172.21.15.16:0/272082691 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c81034a0 con 0x55c9f6320390 2024-08-31T06:18:33.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.255+0000 7fe4ea246740 1 -- 172.21.15.16:0/272082691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9f6320390 msgr2=0x55c9f67026f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:33.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.255+0000 7fe4ea246740 1 --2- 172.21.15.16:0/272082691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9f6320390 0x55c9f67026f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8031440 tx=0x7fe4c80ef040 comp rx=0 tx=0).stop 2024-08-31T06:18:33.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.255+0000 7fe4ea246740 1 -- 172.21.15.16:0/272082691 shutdown_connections 2024-08-31T06:18:33.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.255+0000 7fe4ea246740 1 --2- 172.21.15.16:0/272082691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9f6320390 0x55c9f67026f0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.255+0000 7fe4ea246740 1 --2- 172.21.15.16:0/272082691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9f6341070 0x55c9f6702c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.255+0000 7fe4ea246740 1 --2- 172.21.15.16:0/272082691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9f63181b0 0x55c9f66e34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.255+0000 7fe4ea246740 1 -- 172.21.15.16:0/272082691 >> 172.21.15.16:0/272082691 conn(0x55c9f6335460 msgr2=0x55c9f66e2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:33.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.256+0000 7fe4ea246740 1 -- 172.21.15.16:0/272082691 shutdown_connections 2024-08-31T06:18:33.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.256+0000 7fe4ea246740 1 -- 172.21.15.16:0/272082691 wait complete. 2024-08-31T06:18:33.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.256+0000 7fe4ea246740 1 -- start start 2024-08-31T06:18:33.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.256+0000 7fe4ea246740 1 Processor -- start 2024-08-31T06:18:33.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.257+0000 7fe4ea246740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9f63181b0 0x55c9f66fa7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:33.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.257+0000 7fe4ea246740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9f6320390 0x55c9f66face0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:33.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.257+0000 7fe4ea246740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9f6341070 0x55c9f66fb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:33.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.257+0000 7fe4ea246740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc1ab800 con 0x55c9f63181b0 2024-08-31T06:18:33.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.257+0000 7fe4ea246740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc1acc70 con 0x55c9f6341070 2024-08-31T06:18:33.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.257+0000 7fe4ea246740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc1abed0 con 0x55c9f6320390 2024-08-31T06:18:33.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.257+0000 7fe4ea246740 10 client.0 _ll_register_callbacks cb 0x7fe4d000e450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:18:33.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.257+0000 7fe4e0c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9f63181b0 0x55c9f66fa7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:33.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.257+0000 7fe4e0c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9f63181b0 0x55c9f66fa7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44558/0 (socket says 172.21.15.16:44558) 2024-08-31T06:18:33.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.257+0000 7fe4e0c00640 1 -- 172.21.15.16:0/3944570433 learned_addr learned my addr 172.21.15.16:0/3944570433 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:33.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.257+0000 7fe4dbe00640 1 --2- 172.21.15.16:0/3944570433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9f6320390 0x55c9f66face0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:33.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.257+0000 7fe4e1600640 1 --2- 172.21.15.16:0/3944570433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9f6341070 0x55c9f66fb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:33.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.258+0000 7fe4dbe00640 1 -- 172.21.15.16:0/3944570433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9f6341070 msgr2=0x55c9f66fb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:33.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.258+0000 7fe4dbe00640 1 --2- 172.21.15.16:0/3944570433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9f6341070 0x55c9f66fb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.258+0000 7fe4dbe00640 1 -- 172.21.15.16:0/3944570433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9f63181b0 msgr2=0x55c9f66fa7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:33.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.258+0000 7fe4dbe00640 1 --2- 172.21.15.16:0/3944570433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9f63181b0 0x55c9f66fa7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.258+0000 7fe4dbe00640 1 -- 172.21.15.16:0/3944570433 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9f6772160 con 0x55c9f6320390 2024-08-31T06:18:33.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.258+0000 7fe4dbe00640 1 --2- 172.21.15.16:0/3944570433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9f6320390 0x55c9f66face0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8031440 tx=0x7fe4c8031f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a91b0a643824e78e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:33.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.258+0000 7fe4c2000640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:18:33.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.258+0000 7fe4c2000640 1 -- 172.21.15.16:0/3944570433 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8049040 con 0x55c9f6320390 2024-08-31T06:18:33.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.258+0000 7fe4c2000640 1 -- 172.21.15.16:0/3944570433 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8470+0+0 (secure 0 0 0) 0x7fe4c81034a0 con 0x55c9f6320390 2024-08-31T06:18:33.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.258+0000 7fe4ea246740 1 -- 172.21.15.16:0/3944570433 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4c80d0a30 con 0x55c9f6320390 2024-08-31T06:18:33.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.259+0000 7fe4c2000640 1 -- 172.21.15.16:0/3944570433 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fe4c80d5050 con 0x55c9f6320390 2024-08-31T06:18:33.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.259+0000 7fe4ea246740 10 client.28210 fetch_fsmap learned FSMap version 47 2024-08-31T06:18:33.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.259+0000 7fe4ea246740 1 -- 172.21.15.16:0/3944570433 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55c9f67733f0 con 0x55c9f6320390 2024-08-31T06:18:33.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.259+0000 7fe4c2000640 1 -- 172.21.15.16:0/3944570433 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4c80d0a30 con 0x55c9f6320390 2024-08-31T06:18:33.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.259+0000 7fe4c2000640 1 -- 172.21.15.16:0/3944570433 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 47) ==== 8354+0+0 (secure 0 0 0) 0x7fe4c80cb4d0 con 0x55c9f6320390 2024-08-31T06:18:33.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.259+0000 7fe4ea246740 10 client.28210 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T06:18:33.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.259+0000 7fe4ea246740 10 client.28210 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:18:33.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.259+0000 7fe4ea246740 20 client.28210 populate_metadata read hostname 'smithi016' 2024-08-31T06:18:33.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.259+0000 7fe4ea246740 1 --2- 172.21.15.16:0/3944570433 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55c9f62d4da0 0x55c9f6775ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:33.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.259+0000 7fe4ea246740 4 client.28210 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T06:18:33.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.259+0000 7fe4ea246740 1 -- 172.21.15.16:0/3944570433 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c9f665b160 con 0x55c9f62d4da0 2024-08-31T06:18:33.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.260+0000 7fe4e0c00640 1 --2- 172.21.15.16:0/3944570433 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55c9f62d4da0 0x55c9f6775ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:33.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.260+0000 7fe4e0c00640 1 --2- 172.21.15.16:0/3944570433 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55c9f62d4da0 0x55c9f6775ad0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c0149b321173b4d server_cookie=f9c208967cb29c in_seq=0 out_seq=0 2024-08-31T06:18:33.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.260+0000 7fe4c2000640 10 client.28210 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:18:33.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.261+0000 7fe4c2000640 1 -- 172.21.15.16:0/3944570433 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4546 (crc 0 0 0) 0x55c9f665b160 con 0x55c9f62d4da0 2024-08-31T06:18:33.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.261+0000 7fe4c2000640 10 client.28210 handle_command_reply: tid=0 2024-08-31T06:18:33.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.261+0000 7fe4ea246740 1 client.28210 shutdown 2024-08-31T06:18:33.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.261+0000 7fe4ea246740 1 -- 172.21.15.16:0/3944570433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9f6320390 msgr2=0x55c9f66face0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:33.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.261+0000 7fe4ea246740 1 --2- 172.21.15.16:0/3944570433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9f6320390 0x55c9f66face0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8031440 tx=0x7fe4c8031f50 comp rx=0 tx=0).stop 2024-08-31T06:18:33.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.261+0000 7fe4ea246740 1 -- 172.21.15.16:0/3944570433 shutdown_connections 2024-08-31T06:18:33.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.261+0000 7fe4ea246740 1 --2- 172.21.15.16:0/3944570433 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55c9f62d4da0 0x55c9f6775ad0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.261+0000 7fe4ea246740 1 --2- 172.21.15.16:0/3944570433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9f6320390 0x55c9f66face0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.261+0000 7fe4ea246740 1 --2- 172.21.15.16:0/3944570433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9f6341070 0x55c9f66fb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.261+0000 7fe4ea246740 1 --2- 172.21.15.16:0/3944570433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9f63181b0 0x55c9f66fa7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.261+0000 7fe4c2000640 0 client.28210 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:18:33.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.261+0000 7fe4ea246740 1 -- 172.21.15.16:0/3944570433 >> 172.21.15.16:0/3944570433 conn(0x55c9f6335460 msgr2=0x55c9f66e7980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:33.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.261+0000 7fe4ea246740 1 -- 172.21.15.16:0/3944570433 shutdown_connections 2024-08-31T06:18:33.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.261+0000 7fe4ea246740 1 -- 172.21.15.16:0/3944570433 wait complete. 2024-08-31T06:18:33.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.261+0000 7fe4ea246740 20 client.28210 trim_cache size 0 max 16384 2024-08-31T06:18:33.267 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:18:33.267 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T06:18:33.267 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-08-31T06:18:33.267 INFO:teuthology.orchestra.run.smithi016.stdout: "0e44eb18-439c-46e4-a298-7d4d2ecd0b4a": { 2024-08-31T06:18:33.267 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:18:33.268 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "0e44eb18-439c-46e4-a298-7d4d2ecd0b4a", 2024-08-31T06:18:33.268 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-08-31T06:18:33.268 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:18:33.268 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:18:33.268 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-08-31T06:18:33.268 INFO:teuthology.orchestra.run.smithi016.stdout: "0e44eb18-439c-46e4-a298-7d4d2ecd0b4a": { 2024-08-31T06:18:33.268 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:18:31.130765+0000", 2024-08-31T06:18:33.268 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:18:33.268 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:18:33.268 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:18:33.268 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:18:33.268 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:18:33.268 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:18:33.269 INFO:teuthology.orchestra.run.smithi016.stdout: "e497cd88-1e7c-4631-a4a8-cdd8e1fa6c83": { 2024-08-31T06:18:33.269 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:17:42.476795+0000", 2024-08-31T06:18:33.269 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:18:33.269 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:18:33.269 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:18:33.269 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:18:33.269 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:18:33.269 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:18:33.269 INFO:teuthology.orchestra.run.smithi016.stdout: "c300b7e8-f3fd-482c-a909-9329bc61e570": { 2024-08-31T06:18:33.269 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:06:49.868046+0000", 2024-08-31T06:18:33.269 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:18:33.270 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:18:33.270 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:18:33.270 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:18:33.270 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:18:33.270 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:18:33.270 INFO:teuthology.orchestra.run.smithi016.stdout: "6a03046e-cdf4-4a14-b9bd-b560b8968467": { 2024-08-31T06:18:33.270 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:12:38.002354+0000", 2024-08-31T06:18:33.270 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:18:33.270 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:18:33.270 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:18:33.270 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:18:33.270 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:18:33.270 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:18:33.270 INFO:teuthology.orchestra.run.smithi016.stdout: "004b47bb-0fe8-4d28-9d9d-78bde6a61199": { 2024-08-31T06:18:33.271 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:16:48.915926+0000", 2024-08-31T06:18:33.271 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:18:33.271 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:18:33.271 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:18:33.271 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:18:33.271 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:18:33.271 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:18:33.271 INFO:teuthology.orchestra.run.smithi016.stdout: "88d7bdb8-cb1c-45e5-ad29-4cd8da6c6003": { 2024-08-31T06:18:33.271 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:07:39.648942+0000", 2024-08-31T06:18:33.271 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:18:33.271 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:18:33.271 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:18:33.271 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:18:33.271 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:18:33.272 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:18:33.272 INFO:teuthology.orchestra.run.smithi016.stdout: "0fdcd762-533d-42be-9289-b71c8e3b81b1": { 2024-08-31T06:18:33.272 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:15:06.169112+0000", 2024-08-31T06:18:33.272 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:18:33.272 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:18:33.272 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:18:33.272 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:18:33.272 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:18:33.272 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:18:33.272 INFO:teuthology.orchestra.run.smithi016.stdout: "c8455d74-2fe7-4d93-bcf2-19b419d16016": { 2024-08-31T06:18:33.272 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:08:24.035960+0000", 2024-08-31T06:18:33.272 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:18:33.272 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:18:33.273 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:18:33.273 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:18:33.273 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:18:33.273 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:18:33.273 INFO:teuthology.orchestra.run.smithi016.stdout: "78d6ef02-8551-45e8-ad60-9627eb573fc1": { 2024-08-31T06:18:33.273 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:09:13.547865+0000", 2024-08-31T06:18:33.273 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:18:33.273 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:18:33.273 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:18:33.273 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:18:33.273 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:18:33.273 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:18:33.274 INFO:teuthology.orchestra.run.smithi016.stdout: "c1fd6904-2e02-47fa-939a-47349b9fe36b": { 2024-08-31T06:18:33.274 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:06.678691+0000", 2024-08-31T06:18:33.274 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:18:33.274 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:18:33.274 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:18:33.274 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:18:33.274 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:18:33.274 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:18:33.274 INFO:teuthology.orchestra.run.smithi016.stdout: "a10cce76-99ab-46b5-b3b2-07b82899b8f9": { 2024-08-31T06:18:33.274 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:11:43.967546+0000", 2024-08-31T06:18:33.274 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:18:33.274 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:18:33.274 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:18:33.274 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:18:33.275 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:18:33.275 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:18:33.275 INFO:teuthology.orchestra.run.smithi016.stdout: "91b72a34-4ba9-4126-a414-8b862f8c5f78": { 2024-08-31T06:18:33.275 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:13:28.663237+0000", 2024-08-31T06:18:33.275 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:18:33.275 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:18:33.275 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:18:33.275 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:18:33.275 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:18:33.275 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:18:33.275 INFO:teuthology.orchestra.run.smithi016.stdout: "e739a16e-8ace-43a8-a1ce-ac27d39ef8a3": { 2024-08-31T06:18:33.275 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:54.084646+0000", 2024-08-31T06:18:33.275 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:18:33.276 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:18:33.276 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:18:33.276 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:18:33.276 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:18:33.276 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:18:33.276 INFO:teuthology.orchestra.run.smithi016.stdout: "a64d5301-8558-4e9c-ac9b-8d0ac7057e74": { 2024-08-31T06:18:33.276 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:14:20.338104+0000", 2024-08-31T06:18:33.276 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:18:33.276 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:18:33.276 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:18:33.276 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:18:33.276 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:18:33.276 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:18:33.277 INFO:teuthology.orchestra.run.smithi016.stdout: "f9c4a22e-0c8a-4928-9b0d-e2ce1db5a86b": { 2024-08-31T06:18:33.277 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:15:57.496015+0000", 2024-08-31T06:18:33.277 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:18:33.277 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:18:33.277 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:18:33.277 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:18:33.277 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:18:33.277 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:18:33.277 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:18:33.277 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:18:33.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.262+0000 7fe4e3600640 1 -- 172.21.15.16:0/1486275907 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe4c40b4ca0 msgr2=0x7fe4c40b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:33.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.262+0000 7fe4e3600640 1 --2- 172.21.15.16:0/1486275907 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe4c40b4ca0 0x7fe4c40b71b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4005fd0 tx=0x7fe4d4009bf0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.262+0000 7fe4e3600640 1 -- 172.21.15.16:0/1486275907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4dc109260 msgr2=0x7fe4dc19a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:33.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.262+0000 7fe4e3600640 1 --2- 172.21.15.16:0/1486275907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4dc109260 0x7fe4dc19a810 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8005ab0 tx=0x7fe4c8003b90 comp rx=0 tx=0).stop 2024-08-31T06:18:33.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.262+0000 7fe4e3600640 1 -- 172.21.15.16:0/1486275907 shutdown_connections 2024-08-31T06:18:33.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.262+0000 7fe4e3600640 1 --2- 172.21.15.16:0/1486275907 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe4c40b4ca0 0x7fe4c40b71b0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.262+0000 7fe4e3600640 1 --2- 172.21.15.16:0/1486275907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4dc109bb0 0x7fe4dc19fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.262+0000 7fe4e3600640 1 --2- 172.21.15.16:0/1486275907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4dc109260 0x7fe4dc19a810 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.262+0000 7fe4e3600640 1 --2- 172.21.15.16:0/1486275907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4dc103920 0x7fe4dc19a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.262+0000 7fe4e3600640 1 -- 172.21.15.16:0/1486275907 >> 172.21.15.16:0/1486275907 conn(0x7fe4dc0ff3b0 msgr2=0x7fe4dc1088f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:33.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.263+0000 7fe4e3600640 1 -- 172.21.15.16:0/1486275907 shutdown_connections 2024-08-31T06:18:33.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:33.263+0000 7fe4e3600640 1 -- 172.21.15.16:0/1486275907 wait complete. 2024-08-31T06:18:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.372+0000 7fd3a7e00640 1 -- 172.21.15.16:0/1508170456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3a8075550 msgr2=0x7fd3a8075960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.372+0000 7fd3a7e00640 1 --2- 172.21.15.16:0/1508170456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3a8075550 0x7fd3a8075960 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fd398009900 tx=0x7fd39802a3f0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.372+0000 7fd3a7e00640 1 -- 172.21.15.16:0/1508170456 shutdown_connections 2024-08-31T06:18:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.372+0000 7fd3a7e00640 1 --2- 172.21.15.16:0/1508170456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3a8075ea0 0x7fd3a810f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.372+0000 7fd3a7e00640 1 --2- 172.21.15.16:0/1508170456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3a810fc00 0x7fd3a8112060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.372+0000 7fd3a7e00640 1 --2- 172.21.15.16:0/1508170456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3a8075550 0x7fd3a8075960 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.372+0000 7fd3a7e00640 1 -- 172.21.15.16:0/1508170456 >> 172.21.15.16:0/1508170456 conn(0x7fd3a80ff0d0 msgr2=0x7fd3a8101520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.372+0000 7fd3a7e00640 1 -- 172.21.15.16:0/1508170456 shutdown_connections 2024-08-31T06:18:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.372+0000 7fd3a7e00640 1 -- 172.21.15.16:0/1508170456 wait complete. 2024-08-31T06:18:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.373+0000 7fd3a7e00640 1 Processor -- start 2024-08-31T06:18:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.373+0000 7fd3a7e00640 1 -- start start 2024-08-31T06:18:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.374+0000 7fd3a7e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3a8075550 0x7fd3a819c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.374+0000 7fd3a7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3a8075ea0 0x7fd3a819c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.374+0000 7fd3a7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3a810fc00 0x7fd3a81a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.374+0000 7fd3a7e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a81157f0 con 0x7fd3a8075ea0 2024-08-31T06:18:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.374+0000 7fd3a7e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a8115670 con 0x7fd3a810fc00 2024-08-31T06:18:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.374+0000 7fd3a7e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a8115970 con 0x7fd3a8075550 2024-08-31T06:18:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.374+0000 7fd39fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3a8075ea0 0x7fd3a819c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.374+0000 7fd3a7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3a810fc00 0x7fd3a81a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.374+0000 7fd3a7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3a810fc00 0x7fd3a81a1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:53948/0 (socket says 172.21.15.16:53948) 2024-08-31T06:18:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.374+0000 7fd3a7400640 1 -- 172.21.15.16:0/158883758 learned_addr learned my addr 172.21.15.16:0/158883758 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.374+0000 7fd3a6a00640 1 --2- 172.21.15.16:0/158883758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3a8075550 0x7fd3a819c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.374+0000 7fd3a7400640 1 -- 172.21.15.16:0/158883758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3a8075550 msgr2=0x7fd3a819c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.374+0000 7fd3a7400640 1 --2- 172.21.15.16:0/158883758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3a8075550 0x7fd3a819c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.374+0000 7fd3a7400640 1 -- 172.21.15.16:0/158883758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3a8075ea0 msgr2=0x7fd3a819c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.374+0000 7fd3a7400640 1 --2- 172.21.15.16:0/158883758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3a8075ea0 0x7fd3a819c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.374+0000 7fd3a7400640 1 -- 172.21.15.16:0/158883758 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a81a2140 con 0x7fd3a810fc00 2024-08-31T06:18:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.374+0000 7fd39fe00640 1 --2- 172.21.15.16:0/158883758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3a8075ea0 0x7fd3a819c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:18:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.375+0000 7fd3a7400640 1 --2- 172.21.15.16:0/158883758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3a810fc00 0x7fd3a81a1a40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd3a000eb10 tx=0x7fd3a000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.375+0000 7fd39f400640 1 -- 172.21.15.16:0/158883758 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a0004b20 con 0x7fd3a810fc00 2024-08-31T06:18:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.375+0000 7fd3a7e00640 1 -- 172.21.15.16:0/158883758 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3a81a2430 con 0x7fd3a810fc00 2024-08-31T06:18:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.375+0000 7fd39f400640 1 -- 172.21.15.16:0/158883758 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3a0004cc0 con 0x7fd3a810fc00 2024-08-31T06:18:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.375+0000 7fd39f400640 1 -- 172.21.15.16:0/158883758 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a0010670 con 0x7fd3a810fc00 2024-08-31T06:18:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.375+0000 7fd3a7e00640 1 -- 172.21.15.16:0/158883758 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3a81a8bc0 con 0x7fd3a810fc00 2024-08-31T06:18:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.378+0000 7fd3b4818740 1 Processor -- start 2024-08-31T06:18:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.379+0000 7fd39f400640 1 -- 172.21.15.16:0/158883758 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fd3a0005020 con 0x7fd3a810fc00 2024-08-31T06:18:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.379+0000 7fd3b4818740 1 -- start start 2024-08-31T06:18:33.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.379+0000 7fd3b4818740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5646f00d41b0 0x5646f04778a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.379+0000 7fd3b4818740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646f00ab2f0 0x5646f0477de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.379+0000 7fd3b4818740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646f00b34d0 0x5646f0478320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.379+0000 7fd3b4818740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5646f03ecf70 con 0x5646f00ab2f0 2024-08-31T06:18:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.379+0000 7fd3b4818740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5646f03dc500 con 0x5646f00b34d0 2024-08-31T06:18:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.379+0000 7fd3b4818740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5646f03e04b0 con 0x5646f00d41b0 2024-08-31T06:18:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.380+0000 7fd3a6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646f00ab2f0 0x5646f0477de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.380+0000 7fd39fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5646f00d41b0 0x5646f04778a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.380+0000 7fd3a6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646f00ab2f0 0x5646f0477de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44590/0 (socket says 172.21.15.16:44590) 2024-08-31T06:18:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.380+0000 7fd3a6a00640 1 -- 172.21.15.16:0/2794561805 learned_addr learned my addr 172.21.15.16:0/2794561805 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.380+0000 7fd39fe00640 1 --2- 172.21.15.16:0/2794561805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646f00b34d0 0x5646f0478320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.380+0000 7fd3a6a00640 1 -- 172.21.15.16:0/2794561805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5646f00d41b0 msgr2=0x5646f04778a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.380+0000 7fd3a6a00640 1 --2- 172.21.15.16:0/2794561805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5646f00d41b0 0x5646f04778a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.380+0000 7fd3a6a00640 1 -- 172.21.15.16:0/2794561805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646f00b34d0 msgr2=0x5646f0478320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.380+0000 7fd3a6a00640 1 --2- 172.21.15.16:0/2794561805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646f00b34d0 0x5646f0478320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.380+0000 7fd3a6a00640 1 -- 172.21.15.16:0/2794561805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646f0396c10 con 0x5646f00ab2f0 2024-08-31T06:18:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.380+0000 7fd39fe00640 1 --2- 172.21.15.16:0/2794561805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646f00b34d0 0x5646f0478320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:18:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.381+0000 7fd3a6a00640 1 --2- 172.21.15.16:0/2794561805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646f00ab2f0 0x5646f0477de0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fd398009900 tx=0x7fd398032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d5ccc0cf93f091f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.381+0000 7fd39f400640 1 --2- 172.21.15.16:0/158883758 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd38c0b4ca0 0x7fd38c0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.381+0000 7fd397e00640 1 -- 172.21.15.16:0/2794561805 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd398040070 con 0x5646f00ab2f0 2024-08-31T06:18:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.381+0000 7fd397e00640 1 -- 172.21.15.16:0/2794561805 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd398004030 con 0x5646f00ab2f0 2024-08-31T06:18:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.381+0000 7fd397e00640 1 -- 172.21.15.16:0/2794561805 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd398035040 con 0x5646f00ab2f0 2024-08-31T06:18:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.381+0000 7fd39f400640 1 -- 172.21.15.16:0/158883758 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8470+0+0 (secure 0 0 0) 0x7fd3a00e5b60 con 0x7fd3a810fc00 2024-08-31T06:18:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.381+0000 7fd39fe00640 1 --2- 172.21.15.16:0/158883758 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd38c0b4ca0 0x7fd38c0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.382+0000 7fd3b4818740 1 -- 172.21.15.16:0/2794561805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646f00ab2f0 msgr2=0x5646f0477de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.382+0000 7fd39fe00640 1 --2- 172.21.15.16:0/158883758 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd38c0b4ca0 0x7fd38c0b71b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8112160 tx=0x7fd390013040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.382+0000 7fd3b4818740 1 --2- 172.21.15.16:0/2794561805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646f00ab2f0 0x5646f0477de0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fd398009900 tx=0x7fd398032b90 comp rx=0 tx=0).stop 2024-08-31T06:18:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.382+0000 7fd3b4818740 1 -- 172.21.15.16:0/2794561805 shutdown_connections 2024-08-31T06:18:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.382+0000 7fd3b4818740 1 --2- 172.21.15.16:0/2794561805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646f00b34d0 0x5646f0478320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.382+0000 7fd3b4818740 1 --2- 172.21.15.16:0/2794561805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646f00ab2f0 0x5646f0477de0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.382+0000 7fd3b4818740 1 --2- 172.21.15.16:0/2794561805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5646f00d41b0 0x5646f04778a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.382+0000 7fd3b4818740 1 -- 172.21.15.16:0/2794561805 >> 172.21.15.16:0/2794561805 conn(0x5646f00c8300 msgr2=0x5646f0475790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.382+0000 7fd3b4818740 1 -- 172.21.15.16:0/2794561805 shutdown_connections 2024-08-31T06:18:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.382+0000 7fd3b4818740 1 -- 172.21.15.16:0/2794561805 wait complete. 2024-08-31T06:18:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.383+0000 7fd3b4818740 1 -- start start 2024-08-31T06:18:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.383+0000 7fd3b4818740 1 Processor -- start 2024-08-31T06:18:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.383+0000 7fd3b4818740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646f00ab2f0 0x5646f0482de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.383+0000 7fd3b4818740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646f00b34d0 0x5646f0483320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.383+0000 7fd3b4818740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5646f00d41b0 0x5646f0483860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.383+0000 7fd3b4818740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5646f0315ab0 con 0x5646f00b34d0 2024-08-31T06:18:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.383+0000 7fd3b4818740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5646f0383320 con 0x5646f00ab2f0 2024-08-31T06:18:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.383+0000 7fd3b4818740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5646f0485fc0 con 0x5646f00d41b0 2024-08-31T06:18:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.383+0000 7fd3b4818740 10 client.0 _ll_register_callbacks cb 0x5646f03bce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:18:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.384+0000 7fd3a7400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5646f00d41b0 0x5646f0483860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.384+0000 7fd39fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646f00b34d0 0x5646f0483320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.384+0000 7fd3a7400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5646f00d41b0 0x5646f0483860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:51870/0 (socket says 172.21.15.16:51870) 2024-08-31T06:18:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.384+0000 7fd3a7400640 1 -- 172.21.15.16:0/3852892011 learned_addr learned my addr 172.21.15.16:0/3852892011 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.384+0000 7fd3a6a00640 1 --2- 172.21.15.16:0/3852892011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646f00ab2f0 0x5646f0482de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.384+0000 7fd3a7400640 1 -- 172.21.15.16:0/3852892011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646f00ab2f0 msgr2=0x5646f0482de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.384+0000 7fd3a7400640 1 --2- 172.21.15.16:0/3852892011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646f00ab2f0 0x5646f0482de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.384+0000 7fd3a7400640 1 -- 172.21.15.16:0/3852892011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646f00b34d0 msgr2=0x5646f0483320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.384+0000 7fd3a7400640 1 --2- 172.21.15.16:0/3852892011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646f00b34d0 0x5646f0483320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.384+0000 7fd3a7400640 1 -- 172.21.15.16:0/3852892011 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646f013f790 con 0x5646f00d41b0 2024-08-31T06:18:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.384+0000 7fd3a7400640 1 --2- 172.21.15.16:0/3852892011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5646f00d41b0 0x5646f0483860 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fd3a000cfb0 tx=0x7fd3a00ac120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a95c13b912c0d17 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.384+0000 7fd394c00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:18:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.385+0000 7fd394c00640 1 -- 172.21.15.16:0/3852892011 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a00ac920 con 0x5646f00d41b0 2024-08-31T06:18:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.385+0000 7fd394c00640 1 -- 172.21.15.16:0/3852892011 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8470+0+0 (secure 0 0 0) 0x7fd3a00ac690 con 0x5646f00d41b0 2024-08-31T06:18:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.385+0000 7fd3b4818740 1 -- 172.21.15.16:0/3852892011 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd398004030 con 0x5646f00d41b0 2024-08-31T06:18:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.385+0000 7fd394c00640 1 -- 172.21.15.16:0/3852892011 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fd3a00ac220 con 0x5646f00d41b0 2024-08-31T06:18:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.385+0000 7fd3b4818740 10 client.28222 fetch_fsmap learned FSMap version 47 2024-08-31T06:18:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.385+0000 7fd3b4818740 1 -- 172.21.15.16:0/3852892011 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5646f01b8620 con 0x5646f00d41b0 2024-08-31T06:18:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.385+0000 7fd394c00640 1 -- 172.21.15.16:0/3852892011 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd398004030 con 0x5646f00d41b0 2024-08-31T06:18:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.385+0000 7fd394c00640 1 -- 172.21.15.16:0/3852892011 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 47) ==== 8354+0+0 (secure 0 0 0) 0x7fd3a00a7b90 con 0x5646f00d41b0 2024-08-31T06:18:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.385+0000 7fd3b4818740 10 client.28222 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T06:18:33.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.385+0000 7fd3b4818740 10 client.28222 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:18:33.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.385+0000 7fd3b4818740 20 client.28222 populate_metadata read hostname 'smithi016' 2024-08-31T06:18:33.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.385+0000 7fd3b4818740 1 --2- 172.21.15.16:0/3852892011 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5646f0092d80 0x5646f0494fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:33.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.385+0000 7fd3b4818740 4 client.28222 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:18:33.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.385+0000 7fd3b4818740 1 -- 172.21.15.16:0/3852892011 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646f03eeec0 con 0x5646f0092d80 2024-08-31T06:18:33.392 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.386+0000 7fd3a6a00640 1 --2- 172.21.15.16:0/3852892011 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5646f0092d80 0x5646f0494fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.386+0000 7fd3a6a00640 1 --2- 172.21.15.16:0/3852892011 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5646f0092d80 0x5646f0494fc0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b80b664608a42c35 server_cookie=373f3d8ad4c6a726 in_seq=0 out_seq=0 2024-08-31T06:18:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.386+0000 7fd394c00640 10 client.28222 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:18:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.387+0000 7fd394c00640 1 -- 172.21.15.16:0/3852892011 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5646f03eeec0 con 0x5646f0092d80 2024-08-31T06:18:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.387+0000 7fd394c00640 10 client.28222 handle_command_reply: tid=0 2024-08-31T06:18:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.387+0000 7fd3b4818740 1 client.28222 shutdown 2024-08-31T06:18:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.388+0000 7fd3b4818740 1 -- 172.21.15.16:0/3852892011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5646f00d41b0 msgr2=0x5646f0483860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.388+0000 7fd3b4818740 1 --2- 172.21.15.16:0/3852892011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5646f00d41b0 0x5646f0483860 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fd3a000cfb0 tx=0x7fd3a00ac120 comp rx=0 tx=0).stop 2024-08-31T06:18:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.388+0000 7fd3b4818740 1 -- 172.21.15.16:0/3852892011 shutdown_connections 2024-08-31T06:18:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.388+0000 7fd3b4818740 1 --2- 172.21.15.16:0/3852892011 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5646f0092d80 0x5646f0494fc0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.388+0000 7fd3b4818740 1 --2- 172.21.15.16:0/3852892011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5646f00d41b0 0x5646f0483860 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.388+0000 7fd3b4818740 1 --2- 172.21.15.16:0/3852892011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646f00b34d0 0x5646f0483320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.388+0000 7fd394c00640 0 client.28222 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:18:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.388+0000 7fd3b4818740 1 --2- 172.21.15.16:0/3852892011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646f00ab2f0 0x5646f0482de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.388+0000 7fd3b4818740 1 -- 172.21.15.16:0/3852892011 >> 172.21.15.16:0/3852892011 conn(0x5646f00c8300 msgr2=0x5646f04788d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.388+0000 7fd3b4818740 1 -- 172.21.15.16:0/3852892011 shutdown_connections 2024-08-31T06:18:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.388+0000 7fd3b4818740 1 -- 172.21.15.16:0/3852892011 wait complete. 2024-08-31T06:18:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.388+0000 7fd3b4818740 20 client.28222 trim_cache size 0 max 16384 2024-08-31T06:18:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.404+0000 7fd3b4818740 1 Processor -- start 2024-08-31T06:18:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.404+0000 7fd3b4818740 1 -- start start 2024-08-31T06:18:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.404+0000 7fd3b4818740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5646f00ab2f0 0x5646f0484a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.404+0000 7fd3b4818740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646f00b34d0 0x5646f047ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.404+0000 7fd3b4818740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646f00d41b0 0x5646f04804f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.404+0000 7fd3b4818740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a81b3ce0 con 0x5646f00d41b0 2024-08-31T06:18:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.404+0000 7fd3b4818740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a81b1550 con 0x5646f00b34d0 2024-08-31T06:18:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.404+0000 7fd3b4818740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a81af3c0 con 0x5646f00ab2f0 2024-08-31T06:18:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.404+0000 7fd3a7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646f00d41b0 0x5646f04804f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.404+0000 7fd3a7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646f00d41b0 0x5646f04804f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44612/0 (socket says 172.21.15.16:44612) 2024-08-31T06:18:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.404+0000 7fd3a7400640 1 -- 172.21.15.16:0/664607392 learned_addr learned my addr 172.21.15.16:0/664607392 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.404+0000 7fd3a6a00640 1 --2- 172.21.15.16:0/664607392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5646f00ab2f0 0x5646f0484a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.404+0000 7fd39fe00640 1 --2- 172.21.15.16:0/664607392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646f00b34d0 0x5646f047ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.405+0000 7fd3a7400640 1 -- 172.21.15.16:0/664607392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5646f00ab2f0 msgr2=0x5646f0484a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.405+0000 7fd3a7400640 1 --2- 172.21.15.16:0/664607392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5646f00ab2f0 0x5646f0484a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.405+0000 7fd3a7400640 1 -- 172.21.15.16:0/664607392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646f00b34d0 msgr2=0x5646f047ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.405+0000 7fd3a7400640 1 --2- 172.21.15.16:0/664607392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646f00b34d0 0x5646f047ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.405+0000 7fd3a7400640 1 -- 172.21.15.16:0/664607392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646f046faa0 con 0x5646f00d41b0 2024-08-31T06:18:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.405+0000 7fd3a7400640 1 --2- 172.21.15.16:0/664607392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646f00d41b0 0x5646f04804f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fd3a00a8080 tx=0x7fd3a000efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0fd9f12d1ba2bd server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.405+0000 7fd397400640 1 -- 172.21.15.16:0/664607392 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a00ac920 con 0x5646f00d41b0 2024-08-31T06:18:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.405+0000 7fd397400640 1 -- 172.21.15.16:0/664607392 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3a00b0a90 con 0x5646f00d41b0 2024-08-31T06:18:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.405+0000 7fd397400640 1 -- 172.21.15.16:0/664607392 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a00b50b0 con 0x5646f00d41b0 2024-08-31T06:18:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.406+0000 7fd3b4818740 1 -- 172.21.15.16:0/664607392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646f00d41b0 msgr2=0x5646f04804f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.406+0000 7fd3b4818740 1 --2- 172.21.15.16:0/664607392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646f00d41b0 0x5646f04804f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fd3a00a8080 tx=0x7fd3a000efd0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.406+0000 7fd3b4818740 1 -- 172.21.15.16:0/664607392 shutdown_connections 2024-08-31T06:18:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.406+0000 7fd3b4818740 1 --2- 172.21.15.16:0/664607392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646f00d41b0 0x5646f04804f0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.406+0000 7fd3b4818740 1 --2- 172.21.15.16:0/664607392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646f00b34d0 0x5646f047ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.406+0000 7fd3b4818740 1 --2- 172.21.15.16:0/664607392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5646f00ab2f0 0x5646f0484a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.406+0000 7fd3b4818740 1 -- 172.21.15.16:0/664607392 >> 172.21.15.16:0/664607392 conn(0x5646f00c8300 msgr2=0x5646f0484300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.406+0000 7fd3b4818740 1 -- 172.21.15.16:0/664607392 shutdown_connections 2024-08-31T06:18:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.406+0000 7fd3b4818740 1 -- 172.21.15.16:0/664607392 wait complete. 2024-08-31T06:18:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.406+0000 7fd3b4818740 1 -- start start 2024-08-31T06:18:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.407+0000 7fd3b4818740 1 Processor -- start 2024-08-31T06:18:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.407+0000 7fd3b4818740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646f00ab2f0 0x5646f048b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.407+0000 7fd3b4818740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646f00b34d0 0x5646f0488d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.407+0000 7fd3b4818740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5646f00d41b0 0x5646f0489250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.407+0000 7fd3b4818740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a81afa10 con 0x5646f00ab2f0 2024-08-31T06:18:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.407+0000 7fd3b4818740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a81b0e80 con 0x5646f00b34d0 2024-08-31T06:18:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.407+0000 7fd3b4818740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a81b00e0 con 0x5646f00d41b0 2024-08-31T06:18:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.407+0000 7fd3b4818740 10 client.0 _ll_register_callbacks cb 0x7fd3a00a8150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:18:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.408+0000 7fd3a6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646f00ab2f0 0x5646f048b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.408+0000 7fd3a6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646f00ab2f0 0x5646f048b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44620/0 (socket says 172.21.15.16:44620) 2024-08-31T06:18:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.408+0000 7fd3a6a00640 1 -- 172.21.15.16:0/2119322072 learned_addr learned my addr 172.21.15.16:0/2119322072 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.408+0000 7fd39fe00640 1 --2- 172.21.15.16:0/2119322072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646f00b34d0 0x5646f0488d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.408+0000 7fd3a7400640 1 --2- 172.21.15.16:0/2119322072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5646f00d41b0 0x5646f0489250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.408+0000 7fd3a6a00640 1 -- 172.21.15.16:0/2119322072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5646f00d41b0 msgr2=0x5646f0489250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.408+0000 7fd3a6a00640 1 --2- 172.21.15.16:0/2119322072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5646f00d41b0 0x5646f0489250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.408+0000 7fd3a6a00640 1 -- 172.21.15.16:0/2119322072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646f00b34d0 msgr2=0x5646f0488d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.408+0000 7fd3a6a00640 1 --2- 172.21.15.16:0/2119322072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646f00b34d0 0x5646f0488d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.408+0000 7fd3a6a00640 1 -- 172.21.15.16:0/2119322072 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646f04897c0 con 0x5646f00ab2f0 2024-08-31T06:18:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.408+0000 7fd3a7400640 1 --2- 172.21.15.16:0/2119322072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5646f00d41b0 0x5646f0489250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:18:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.408+0000 7fd39fe00640 1 --2- 172.21.15.16:0/2119322072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646f00b34d0 0x5646f0488d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:18:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.408+0000 7fd3a6a00640 1 --2- 172.21.15.16:0/2119322072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646f00ab2f0 0x5646f048b800 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fd398005150 tx=0x7fd39802a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b0b12e4fbe10fc6 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.408+0000 7fd394c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:18:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.409+0000 7fd394c00640 1 -- 172.21.15.16:0/2119322072 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd398040030 con 0x5646f00ab2f0 2024-08-31T06:18:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.409+0000 7fd394c00640 1 -- 172.21.15.16:0/2119322072 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8470+0+0 (secure 0 0 0) 0x7fd398031d10 con 0x5646f00ab2f0 2024-08-31T06:18:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.409+0000 7fd3b4818740 1 -- 172.21.15.16:0/2119322072 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3a00b0a90 con 0x5646f00ab2f0 2024-08-31T06:18:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.409+0000 7fd394c00640 1 -- 172.21.15.16:0/2119322072 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fd39805d050 con 0x5646f00ab2f0 2024-08-31T06:18:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.409+0000 7fd3b4818740 10 client.17856 fetch_fsmap learned FSMap version 47 2024-08-31T06:18:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.409+0000 7fd3b4818740 1 -- 172.21.15.16:0/2119322072 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5646f04ff700 con 0x5646f00ab2f0 2024-08-31T06:18:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.409+0000 7fd394c00640 1 -- 172.21.15.16:0/2119322072 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3a00b0a90 con 0x5646f00ab2f0 2024-08-31T06:18:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.409+0000 7fd394c00640 1 -- 172.21.15.16:0/2119322072 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 47) ==== 8354+0+0 (secure 0 0 0) 0x7fd3980047e0 con 0x5646f00ab2f0 2024-08-31T06:18:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.410+0000 7fd3b4818740 10 client.17856 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T06:18:33.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.410+0000 7fd3b4818740 10 client.17856 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:18:33.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.410+0000 7fd3b4818740 20 client.17856 populate_metadata read hostname 'smithi016' 2024-08-31T06:18:33.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.410+0000 7fd3b4818740 1 --2- 172.21.15.16:0/2119322072 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5646f00681c0 0x5646f0501b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:33.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.410+0000 7fd3b4818740 4 client.17856 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:18:33.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.410+0000 7fd3b4818740 1 -- 172.21.15.16:0/2119322072 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5646f03ea960 con 0x5646f00681c0 2024-08-31T06:18:33.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.410+0000 7fd39fe00640 1 --2- 172.21.15.16:0/2119322072 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5646f00681c0 0x5646f0501b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:33.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.410+0000 7fd39fe00640 1 --2- 172.21.15.16:0/2119322072 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5646f00681c0 0x5646f0501b00 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f91536ffdb5d700 server_cookie=7ac7f0d64e0b27b1 in_seq=0 out_seq=0 2024-08-31T06:18:33.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.410+0000 7fd394c00640 10 client.17856 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:18:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.429+0000 7fd394c00640 1 -- 172.21.15.16:0/2119322072 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5646f03ea960 con 0x5646f00681c0 2024-08-31T06:18:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.429+0000 7fd394c00640 10 client.17856 handle_command_reply: tid=0 2024-08-31T06:18:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.429+0000 7fd3b4818740 1 client.17856 shutdown 2024-08-31T06:18:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.429+0000 7fd3b4818740 1 -- 172.21.15.16:0/2119322072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646f00ab2f0 msgr2=0x5646f048b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.429+0000 7fd3b4818740 1 --2- 172.21.15.16:0/2119322072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646f00ab2f0 0x5646f048b800 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fd398005150 tx=0x7fd39802a3f0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.429+0000 7fd3b4818740 1 -- 172.21.15.16:0/2119322072 shutdown_connections 2024-08-31T06:18:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.429+0000 7fd3b4818740 1 --2- 172.21.15.16:0/2119322072 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5646f00681c0 0x5646f0501b00 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.429+0000 7fd3b4818740 1 --2- 172.21.15.16:0/2119322072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5646f00d41b0 0x5646f0489250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.430+0000 7fd3b4818740 1 --2- 172.21.15.16:0/2119322072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646f00b34d0 0x5646f0488d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.430+0000 7fd394c00640 0 client.17856 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:18:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.430+0000 7fd3b4818740 1 --2- 172.21.15.16:0/2119322072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646f00ab2f0 0x5646f048b800 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.430+0000 7fd3b4818740 1 -- 172.21.15.16:0/2119322072 >> 172.21.15.16:0/2119322072 conn(0x5646f00c8300 msgr2=0x5646f04788d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.430+0000 7fd3b4818740 1 -- 172.21.15.16:0/2119322072 shutdown_connections 2024-08-31T06:18:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.430+0000 7fd3b4818740 1 -- 172.21.15.16:0/2119322072 wait complete. 2024-08-31T06:18:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.430+0000 7fd3b4818740 20 client.17856 trim_cache size 0 max 16384 2024-08-31T06:18:33.434 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:18:33.434 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:18:33.434 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:18:33.434 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:18:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.431+0000 7fd3a7e00640 1 -- 172.21.15.16:0/158883758 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd38c0b4ca0 msgr2=0x7fd38c0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.431+0000 7fd3a7e00640 1 --2- 172.21.15.16:0/158883758 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd38c0b4ca0 0x7fd38c0b71b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8112160 tx=0x7fd390013040 comp rx=0 tx=0).stop 2024-08-31T06:18:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.431+0000 7fd3a7e00640 1 -- 172.21.15.16:0/158883758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3a810fc00 msgr2=0x7fd3a81a1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.431+0000 7fd3a7e00640 1 --2- 172.21.15.16:0/158883758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3a810fc00 0x7fd3a81a1a40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd3a000eb10 tx=0x7fd3a000c2d0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.431+0000 7fd3a7e00640 1 -- 172.21.15.16:0/158883758 shutdown_connections 2024-08-31T06:18:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.431+0000 7fd3a7e00640 1 --2- 172.21.15.16:0/158883758 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd38c0b4ca0 0x7fd38c0b71b0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.431+0000 7fd3a7e00640 1 --2- 172.21.15.16:0/158883758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3a810fc00 0x7fd3a81a1a40 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.431+0000 7fd3a7e00640 1 --2- 172.21.15.16:0/158883758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3a8075ea0 0x7fd3a819c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.431+0000 7fd3a7e00640 1 --2- 172.21.15.16:0/158883758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3a8075550 0x7fd3a819c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:33.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.431+0000 7fd3a7e00640 1 -- 172.21.15.16:0/158883758 >> 172.21.15.16:0/158883758 conn(0x7fd3a80ff0d0 msgr2=0x7fd3a8101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:33.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.431+0000 7fd3a7e00640 1 -- 172.21.15.16:0/158883758 shutdown_connections 2024-08-31T06:18:33.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:33.431+0000 7fd3a7e00640 1 -- 172.21.15.16:0/158883758 wait complete. 2024-08-31T06:18:34.164 INFO:tasks.cephfs.filesystem:scrub status for tag:0e44eb18-439c-46e4-a298-7d4d2ecd0b4a - {'path': '/', 'tag': '0e44eb18-439c-46e4-a298-7d4d2ecd0b4a', 'options': 'recursive,force'} 2024-08-31T06:18:34.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:34 smithi184 ceph-mon[22331]: pgmap v716: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 2.4 MiB/s wr, 275 op/s 2024-08-31T06:18:35.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:34 smithi155 ceph-mon[21742]: pgmap v716: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 2.4 MiB/s wr, 275 op/s 2024-08-31T06:18:35.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:34 smithi016 ceph-mon[17194]: pgmap v716: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 2.4 MiB/s wr, 275 op/s 2024-08-31T06:18:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 670.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-31T06:18:36.873 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:18:36.929 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:36 smithi016 ceph-mon[17194]: pgmap v717: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 2.4 MiB/s wr, 344 op/s 2024-08-31T06:18:36.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:36 smithi184 ceph-mon[22331]: pgmap v717: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 2.4 MiB/s wr, 344 op/s 2024-08-31T06:18:37.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:36 smithi155 ceph-mon[21742]: pgmap v717: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 2.4 MiB/s wr, 344 op/s 2024-08-31T06:18:37.267 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:18:37.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:37.965+0000 7f4251000640 1 -- 172.21.15.16:0/3354796550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f424c114020 msgr2=0x7f424c1164f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:37.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:37.965+0000 7f4251000640 1 --2- 172.21.15.16:0/3354796550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f424c114020 0x7f424c1164f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f4244009ca0 tx=0x7f424402a8c0 comp rx=0 tx=0).stop 2024-08-31T06:18:37.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:37.965+0000 7f4251000640 1 -- 172.21.15.16:0/3354796550 shutdown_connections 2024-08-31T06:18:37.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:37.965+0000 7f4251000640 1 --2- 172.21.15.16:0/3354796550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f424c075b70 0x7f424c075f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:37.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:37.965+0000 7f4251000640 1 --2- 172.21.15.16:0/3354796550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f424c114020 0x7f424c1164f0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:37.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:37.965+0000 7f4251000640 1 --2- 172.21.15.16:0/3354796550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f424c077120 0x7f424c075550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:37.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:37.965+0000 7f4251000640 1 -- 172.21.15.16:0/3354796550 >> 172.21.15.16:0/3354796550 conn(0x7f424c0ff1e0 msgr2=0x7f424c101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:37.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:37.965+0000 7f4251000640 1 -- 172.21.15.16:0/3354796550 shutdown_connections 2024-08-31T06:18:37.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:37.965+0000 7f4251000640 1 -- 172.21.15.16:0/3354796550 wait complete. 2024-08-31T06:18:37.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:37.966+0000 7f4251000640 1 Processor -- start 2024-08-31T06:18:37.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:37.967+0000 7f4251000640 1 -- start start 2024-08-31T06:18:37.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:37.967+0000 7f4251000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f424c075b70 0x7f424c1a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:37.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:37.967+0000 7f4251000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f424c077120 0x7f424c1a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:37.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:37.967+0000 7f4251000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f424c114020 0x7f424c1a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:37.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:37.967+0000 7f4251000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f424c119c60 con 0x7f424c075b70 2024-08-31T06:18:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:37.967+0000 7f4251000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f424c119ae0 con 0x7f424c114020 2024-08-31T06:18:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:37.967+0000 7f4251000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f424c119de0 con 0x7f424c077120 2024-08-31T06:18:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:37.967+0000 7f424b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f424c075b70 0x7f424c1a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:37.967+0000 7f424aa00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f424c077120 0x7f424c1a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:37.968+0000 7f424aa00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f424c077120 0x7f424c1a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:56942/0 (socket says 172.21.15.16:56942) 2024-08-31T06:18:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:37.968+0000 7f424aa00640 1 -- 172.21.15.16:0/1406475816 learned_addr learned my addr 172.21.15.16:0/1406475816 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:37.968+0000 7f424be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f424c114020 0x7f424c1a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:37.968+0000 7f424aa00640 1 -- 172.21.15.16:0/1406475816 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f424c114020 msgr2=0x7f424c1a5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:37.968+0000 7f424aa00640 1 --2- 172.21.15.16:0/1406475816 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f424c114020 0x7f424c1a5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:37.968+0000 7f424aa00640 1 -- 172.21.15.16:0/1406475816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f424c075b70 msgr2=0x7f424c1a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:37.968+0000 7f424aa00640 1 --2- 172.21.15.16:0/1406475816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f424c075b70 0x7f424c1a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:37.968+0000 7f424aa00640 1 -- 172.21.15.16:0/1406475816 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f424c1a65a0 con 0x7f424c077120 2024-08-31T06:18:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:37.968+0000 7f424aa00640 1 --2- 172.21.15.16:0/1406475816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f424c077120 0x7f424c1a0bd0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f423400e9f0 tx=0x7f423400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:37.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:37.969+0000 7f4243e00640 1 -- 172.21.15.16:0/1406475816 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423400ccc0 con 0x7f424c077120 2024-08-31T06:18:37.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:37.969+0000 7f4243e00640 1 -- 172.21.15.16:0/1406475816 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42340054d0 con 0x7f424c077120 2024-08-31T06:18:37.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:37.969+0000 7f4251000640 1 -- 172.21.15.16:0/1406475816 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f424c1a6890 con 0x7f424c077120 2024-08-31T06:18:37.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:37.969+0000 7f4243e00640 1 -- 172.21.15.16:0/1406475816 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4234010430 con 0x7f424c077120 2024-08-31T06:18:37.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:37.969+0000 7f4251000640 1 -- 172.21.15.16:0/1406475816 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f424c1acff0 con 0x7f424c077120 2024-08-31T06:18:37.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:37.970+0000 7f4251000640 1 -- 172.21.15.16:0/1406475816 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f424c075f80 con 0x7f424c077120 2024-08-31T06:18:37.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:37.972+0000 7f4243e00640 1 -- 172.21.15.16:0/1406475816 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f42340046c0 con 0x7f424c077120 2024-08-31T06:18:37.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:37.974+0000 7f4243e00640 1 --2- 172.21.15.16:0/1406475816 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f42300b4d20 0x7f42300b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:37.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:37.974+0000 7f4243e00640 1 -- 172.21.15.16:0/1406475816 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8470+0+0 (secure 0 0 0) 0x7f42340e5a30 con 0x7f424c077120 2024-08-31T06:18:37.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:37.974+0000 7f424b400640 1 --2- 172.21.15.16:0/1406475816 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f42300b4d20 0x7f42300b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:37.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:37.975+0000 7f424b400640 1 --2- 172.21.15.16:0/1406475816 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f42300b4d20 0x7f42300b7230 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f423c009bd0 tx=0x7f423c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:37.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:37.976+0000 7f4243e00640 1 -- 172.21.15.16:0/1406475816 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f4234056130 con 0x7f424c077120 2024-08-31T06:18:38.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:38.212+0000 7f4251000640 1 -- 172.21.15.16:0/1406475816 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f424c1a1760 con 0x7f424c077120 2024-08-31T06:18:38.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:38.213+0000 7f4243e00640 1 -- 172.21.15.16:0/1406475816 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11920 (secure 0 0 0) 0x7f424c1a1760 con 0x7f424c077120 2024-08-31T06:18:38.216 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:18:38.216 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":47,"btime":"2024-08-31T06:18:22:635831+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":47,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:18:22.030000+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24737,24724,24712]},"id":1}]} 2024-08-31T06:18:38.216 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 47 2024-08-31T06:18:38.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:38.216+0000 7f4251000640 1 -- 172.21.15.16:0/1406475816 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f42300b4d20 msgr2=0x7f42300b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:38.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:38.216+0000 7f4251000640 1 --2- 172.21.15.16:0/1406475816 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f42300b4d20 0x7f42300b7230 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f423c009bd0 tx=0x7f423c005f90 comp rx=0 tx=0).stop 2024-08-31T06:18:38.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:38.216+0000 7f4251000640 1 -- 172.21.15.16:0/1406475816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f424c077120 msgr2=0x7f424c1a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:38.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:38.216+0000 7f4251000640 1 --2- 172.21.15.16:0/1406475816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f424c077120 0x7f424c1a0bd0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f423400e9f0 tx=0x7f423400eeb0 comp rx=0 tx=0).stop 2024-08-31T06:18:38.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:38.216+0000 7f4251000640 1 -- 172.21.15.16:0/1406475816 shutdown_connections 2024-08-31T06:18:38.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:38.216+0000 7f4251000640 1 --2- 172.21.15.16:0/1406475816 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f42300b4d20 0x7f42300b7230 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:38.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:38.216+0000 7f4251000640 1 --2- 172.21.15.16:0/1406475816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f424c077120 0x7f424c1a0bd0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:38.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:38.216+0000 7f4251000640 1 --2- 172.21.15.16:0/1406475816 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f424c114020 0x7f424c1a5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:38.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:38.216+0000 7f4251000640 1 --2- 172.21.15.16:0/1406475816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f424c075b70 0x7f424c1a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:38.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:38.216+0000 7f4251000640 1 -- 172.21.15.16:0/1406475816 >> 172.21.15.16:0/1406475816 conn(0x7f424c0ff1e0 msgr2=0x7f424c10ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:38.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:38.216+0000 7f4251000640 1 -- 172.21.15.16:0/1406475816 shutdown_connections 2024-08-31T06:18:38.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:38.216+0000 7f4251000640 1 -- 172.21.15.16:0/1406475816 wait complete. 2024-08-31T06:18:38.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:38 smithi016 ceph-mon[17194]: pgmap v718: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 2.4 MiB/s wr, 339 op/s 2024-08-31T06:18:38.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:38 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/1406475816' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:18:38.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:38 smithi184 ceph-mon[22331]: pgmap v718: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 2.4 MiB/s wr, 339 op/s 2024-08-31T06:18:38.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:38 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/1406475816' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:18:39.045 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24737 quiesce db --set-id f2a792d6 --release --await' 2024-08-31T06:18:39.045 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24737 quiesce db --set-id f2a792d6 --release --await 2024-08-31T06:18:39.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:38 smithi155 ceph-mon[21742]: pgmap v718: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 2.4 MiB/s wr, 339 op/s 2024-08-31T06:18:39.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:38 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/1406475816' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:18:39.424 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:18:40.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.082+0000 7fd75b200640 1 -- 172.21.15.16:0/1510381871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd75c105950 msgr2=0x7fd75c105d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:40.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.082+0000 7fd75b200640 1 --2- 172.21.15.16:0/1510381871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd75c105950 0x7fd75c105d60 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd74c009900 tx=0x7fd74c02a400 comp rx=0 tx=0).stop 2024-08-31T06:18:40.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.082+0000 7fd75b200640 1 -- 172.21.15.16:0/1510381871 shutdown_connections 2024-08-31T06:18:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.082+0000 7fd75b200640 1 --2- 172.21.15.16:0/1510381871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd75c10bbe0 0x7fd75c112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.082+0000 7fd75b200640 1 --2- 172.21.15.16:0/1510381871 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd75c10b290 0x7fd75c10b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.082+0000 7fd75b200640 1 --2- 172.21.15.16:0/1510381871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd75c105950 0x7fd75c105d60 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.082+0000 7fd75b200640 1 -- 172.21.15.16:0/1510381871 >> 172.21.15.16:0/1510381871 conn(0x7fd75c101420 msgr2=0x7fd75c103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.083+0000 7fd75b200640 1 -- 172.21.15.16:0/1510381871 shutdown_connections 2024-08-31T06:18:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.083+0000 7fd75b200640 1 -- 172.21.15.16:0/1510381871 wait complete. 2024-08-31T06:18:40.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.084+0000 7fd75b200640 1 Processor -- start 2024-08-31T06:18:40.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.084+0000 7fd75b200640 1 -- start start 2024-08-31T06:18:40.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.084+0000 7fd75b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd75c105950 0x7fd75c19c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:40.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.084+0000 7fd75b200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd75c10b290 0x7fd75c19c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:40.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.084+0000 7fd75b200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd75c10bbe0 0x7fd75c1a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:40.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.084+0000 7fd75b200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c1158a0 con 0x7fd75c105950 2024-08-31T06:18:40.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.084+0000 7fd75b200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c115720 con 0x7fd75c10bbe0 2024-08-31T06:18:40.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.084+0000 7fd75b200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c115a20 con 0x7fd75c10b290 2024-08-31T06:18:40.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.085+0000 7fd759e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd75c105950 0x7fd75c19c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:40.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.085+0000 7fd753e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd75c10b290 0x7fd75c19c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.085+0000 7fd75a800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd75c10bbe0 0x7fd75c1a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.085+0000 7fd753e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd75c10b290 0x7fd75c19c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:56964/0 (socket says 172.21.15.16:56964) 2024-08-31T06:18:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.085+0000 7fd753e00640 1 -- 172.21.15.16:0/3642170067 learned_addr learned my addr 172.21.15.16:0/3642170067 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.085+0000 7fd75a800640 1 -- 172.21.15.16:0/3642170067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd75c10b290 msgr2=0x7fd75c19c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.085+0000 7fd75a800640 1 --2- 172.21.15.16:0/3642170067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd75c10b290 0x7fd75c19c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.085+0000 7fd75a800640 1 -- 172.21.15.16:0/3642170067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd75c105950 msgr2=0x7fd75c19c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.085+0000 7fd75a800640 1 --2- 172.21.15.16:0/3642170067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd75c105950 0x7fd75c19c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.085+0000 7fd75a800640 1 -- 172.21.15.16:0/3642170067 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd75c1a2200 con 0x7fd75c10bbe0 2024-08-31T06:18:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.086+0000 7fd75a800640 1 --2- 172.21.15.16:0/3642170067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd75c10bbe0 0x7fd75c1a1ad0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fd75400ec50 tx=0x7fd75400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.086+0000 7fd752a00640 1 -- 172.21.15.16:0/3642170067 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd754003b70 con 0x7fd75c10bbe0 2024-08-31T06:18:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.086+0000 7fd75b200640 1 -- 172.21.15.16:0/3642170067 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd75c1a24f0 con 0x7fd75c10bbe0 2024-08-31T06:18:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.086+0000 7fd752a00640 1 -- 172.21.15.16:0/3642170067 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd754003d10 con 0x7fd75c10bbe0 2024-08-31T06:18:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.086+0000 7fd752a00640 1 -- 172.21.15.16:0/3642170067 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd754010830 con 0x7fd75c10bbe0 2024-08-31T06:18:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.086+0000 7fd75b200640 1 -- 172.21.15.16:0/3642170067 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd75c1a8c50 con 0x7fd75c10bbe0 2024-08-31T06:18:40.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.090+0000 7fd767866740 1 Processor -- start 2024-08-31T06:18:40.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.091+0000 7fd752a00640 1 -- 172.21.15.16:0/3642170067 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fd754003ec0 con 0x7fd75c10bbe0 2024-08-31T06:18:40.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.091+0000 7fd767866740 1 -- start start 2024-08-31T06:18:40.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.091+0000 7fd767866740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ff1a4520 0x5615ff54a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:40.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.091+0000 7fd767866740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615ff17b660 0x5615ff54a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:40.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.091+0000 7fd767866740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615ff183840 0x5615ff54ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:40.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.091+0000 7fd767866740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5615ff4c0a90 con 0x5615ff183840 2024-08-31T06:18:40.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.091+0000 7fd767866740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5615ff4b0020 con 0x5615ff1a4520 2024-08-31T06:18:40.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.091+0000 7fd767866740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5615ff4b3fd0 con 0x5615ff17b660 2024-08-31T06:18:40.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.092+0000 7fd753e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615ff183840 0x5615ff54ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:40.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.092+0000 7fd759e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615ff17b660 0x5615ff54a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:40.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.092+0000 7fd759e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615ff17b660 0x5615ff54a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:56966/0 (socket says 172.21.15.16:56966) 2024-08-31T06:18:40.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.092+0000 7fd759e00640 1 -- 172.21.15.16:0/1281747055 learned_addr learned my addr 172.21.15.16:0/1281747055 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:40.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.092+0000 7fd753e00640 1 --2- 172.21.15.16:0/1281747055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ff1a4520 0x5615ff54a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:40.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.092+0000 7fd759e00640 1 -- 172.21.15.16:0/1281747055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ff1a4520 msgr2=0x5615ff54a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:40.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.092+0000 7fd759e00640 1 --2- 172.21.15.16:0/1281747055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ff1a4520 0x5615ff54a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:40.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.093+0000 7fd759e00640 1 -- 172.21.15.16:0/1281747055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615ff183840 msgr2=0x5615ff54ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:40.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.093+0000 7fd759e00640 1 --2- 172.21.15.16:0/1281747055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615ff183840 0x5615ff54ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:40.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.093+0000 7fd759e00640 1 -- 172.21.15.16:0/1281747055 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615ff20fb00 con 0x5615ff17b660 2024-08-31T06:18:40.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.093+0000 7fd753e00640 1 --2- 172.21.15.16:0/1281747055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615ff183840 0x5615ff54ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:18:40.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.093+0000 7fd759e00640 1 --2- 172.21.15.16:0/1281747055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615ff17b660 0x5615ff54a5d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd74c009900 tx=0x7fd74c032b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90aa3bd76dd6f2ba server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:40.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.093+0000 7fd752a00640 1 --2- 172.21.15.16:0/3642170067 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd7400b4ca0 0x7fd7400b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:40.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.093+0000 7fd752a00640 1 -- 172.21.15.16:0/3642170067 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8470+0+0 (secure 0 0 0) 0x7fd7540e5a90 con 0x7fd75c10bbe0 2024-08-31T06:18:40.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.093+0000 7fd74b000640 1 -- 172.21.15.16:0/1281747055 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74c040070 con 0x5615ff17b660 2024-08-31T06:18:40.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.093+0000 7fd74b000640 1 -- 172.21.15.16:0/1281747055 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd74c004030 con 0x5615ff17b660 2024-08-31T06:18:40.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.093+0000 7fd74b000640 1 -- 172.21.15.16:0/1281747055 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74c035040 con 0x5615ff17b660 2024-08-31T06:18:40.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.093+0000 7fd753e00640 1 --2- 172.21.15.16:0/3642170067 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd7400b4ca0 0x7fd7400b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:40.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.094+0000 7fd767866740 1 -- 172.21.15.16:0/1281747055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615ff17b660 msgr2=0x5615ff54a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:40.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.094+0000 7fd767866740 1 --2- 172.21.15.16:0/1281747055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615ff17b660 0x5615ff54a5d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd74c009900 tx=0x7fd74c032b90 comp rx=0 tx=0).stop 2024-08-31T06:18:40.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.094+0000 7fd753e00640 1 --2- 172.21.15.16:0/3642170067 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd7400b4ca0 0x7fd7400b71b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd744009fd0 tx=0x7fd744004730 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:40.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.094+0000 7fd767866740 1 -- 172.21.15.16:0/1281747055 shutdown_connections 2024-08-31T06:18:40.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.094+0000 7fd767866740 1 --2- 172.21.15.16:0/1281747055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615ff17b660 0x5615ff54a5d0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:40.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.094+0000 7fd767866740 1 --2- 172.21.15.16:0/1281747055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615ff183840 0x5615ff54ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:40.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.094+0000 7fd767866740 1 --2- 172.21.15.16:0/1281747055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ff1a4520 0x5615ff54a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:40.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.094+0000 7fd767866740 1 -- 172.21.15.16:0/1281747055 >> 172.21.15.16:0/1281747055 conn(0x5615ff198910 msgr2=0x5615ff547f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:40.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.094+0000 7fd767866740 1 -- 172.21.15.16:0/1281747055 shutdown_connections 2024-08-31T06:18:40.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.095+0000 7fd767866740 1 -- 172.21.15.16:0/1281747055 wait complete. 2024-08-31T06:18:40.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.095+0000 7fd767866740 1 -- start start 2024-08-31T06:18:40.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.095+0000 7fd767866740 1 Processor -- start 2024-08-31T06:18:40.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.096+0000 7fd767866740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615ff17b660 0x5615ff557ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:40.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.096+0000 7fd767866740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615ff183840 0x5615ff555410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:40.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.096+0000 7fd767866740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ff1a4520 0x5615ff555b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:40.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.096+0000 7fd767866740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5615ff312bb0 con 0x5615ff17b660 2024-08-31T06:18:40.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.096+0000 7fd767866740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5615ff436f90 con 0x5615ff1a4520 2024-08-31T06:18:40.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.096+0000 7fd767866740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5615ff558810 con 0x5615ff183840 2024-08-31T06:18:40.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.096+0000 7fd767866740 10 client.0 _ll_register_callbacks cb 0x5615ff45c710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:18:40.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.096+0000 7fd759e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615ff17b660 0x5615ff557ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:40.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.096+0000 7fd759e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615ff17b660 0x5615ff557ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58924/0 (socket says 172.21.15.16:58924) 2024-08-31T06:18:40.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.096+0000 7fd759e00640 1 -- 172.21.15.16:0/1047970399 learned_addr learned my addr 172.21.15.16:0/1047970399 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:40.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.096+0000 7fd75a800640 1 --2- 172.21.15.16:0/1047970399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ff1a4520 0x5615ff555b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:40.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.096+0000 7fd753e00640 1 --2- 172.21.15.16:0/1047970399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615ff183840 0x5615ff555410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:40.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.097+0000 7fd759e00640 1 -- 172.21.15.16:0/1047970399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615ff183840 msgr2=0x5615ff555410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:40.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.097+0000 7fd759e00640 1 --2- 172.21.15.16:0/1047970399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615ff183840 0x5615ff555410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:40.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.097+0000 7fd759e00640 1 -- 172.21.15.16:0/1047970399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ff1a4520 msgr2=0x5615ff555b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:40.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.097+0000 7fd759e00640 1 --2- 172.21.15.16:0/1047970399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ff1a4520 0x5615ff555b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:40.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.097+0000 7fd759e00640 1 -- 172.21.15.16:0/1047970399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615ff286960 con 0x5615ff17b660 2024-08-31T06:18:40.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.097+0000 7fd759e00640 1 --2- 172.21.15.16:0/1047970399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615ff17b660 0x5615ff557ec0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd74c009310 tx=0x7fd74c031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76c63d6b46bd85a7 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:40.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.097+0000 7fd73f400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:18:40.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.097+0000 7fd73f400640 1 -- 172.21.15.16:0/1047970399 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74c035040 con 0x5615ff17b660 2024-08-31T06:18:40.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.097+0000 7fd73f400640 1 -- 172.21.15.16:0/1047970399 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8470+0+0 (secure 0 0 0) 0x7fd74c0051c0 con 0x5615ff17b660 2024-08-31T06:18:40.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.097+0000 7fd767866740 1 -- 172.21.15.16:0/1047970399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5615ff44ee30 con 0x5615ff17b660 2024-08-31T06:18:40.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.098+0000 7fd73f400640 1 -- 172.21.15.16:0/1047970399 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fd74c040040 con 0x5615ff17b660 2024-08-31T06:18:40.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.098+0000 7fd767866740 10 client.17874 fetch_fsmap learned FSMap version 47 2024-08-31T06:18:40.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.098+0000 7fd767866740 1 -- 172.21.15.16:0/1047970399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5615ff5641a0 con 0x5615ff17b660 2024-08-31T06:18:40.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.098+0000 7fd73f400640 1 -- 172.21.15.16:0/1047970399 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5615ff44ee30 con 0x5615ff17b660 2024-08-31T06:18:40.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.098+0000 7fd73f400640 1 -- 172.21.15.16:0/1047970399 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 47) ==== 8354+0+0 (secure 0 0 0) 0x7fd74c0048e0 con 0x5615ff17b660 2024-08-31T06:18:40.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.098+0000 7fd767866740 10 client.17874 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T06:18:40.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.098+0000 7fd767866740 10 client.17874 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:18:40.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.098+0000 7fd767866740 20 client.17874 populate_metadata read hostname 'smithi016' 2024-08-31T06:18:40.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.098+0000 7fd767866740 1 --2- 172.21.15.16:0/1047970399 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5615ff162ea0 0x5615ff567500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:40.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.098+0000 7fd767866740 4 client.17874 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:18:40.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.098+0000 7fd767866740 1 -- 172.21.15.16:0/1047970399 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615ff4c29e0 con 0x5615ff162ea0 2024-08-31T06:18:40.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.100+0000 7fd753e00640 1 --2- 172.21.15.16:0/1047970399 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5615ff162ea0 0x5615ff567500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:40.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.101+0000 7fd753e00640 1 --2- 172.21.15.16:0/1047970399 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5615ff162ea0 0x5615ff567500 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3b876f938378ab2 server_cookie=513218391ae1855c in_seq=0 out_seq=0 2024-08-31T06:18:40.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.101+0000 7fd73f400640 10 client.17874 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:18:40.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.102+0000 7fd73f400640 1 -- 172.21.15.16:0/1047970399 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5615ff4c29e0 con 0x5615ff162ea0 2024-08-31T06:18:40.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.102+0000 7fd73f400640 10 client.17874 handle_command_reply: tid=0 2024-08-31T06:18:40.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.102+0000 7fd767866740 1 client.17874 shutdown 2024-08-31T06:18:40.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.102+0000 7fd767866740 1 -- 172.21.15.16:0/1047970399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615ff17b660 msgr2=0x5615ff557ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:40.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.102+0000 7fd767866740 1 --2- 172.21.15.16:0/1047970399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615ff17b660 0x5615ff557ec0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd74c009310 tx=0x7fd74c031400 comp rx=0 tx=0).stop 2024-08-31T06:18:40.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.102+0000 7fd767866740 1 -- 172.21.15.16:0/1047970399 shutdown_connections 2024-08-31T06:18:40.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.102+0000 7fd767866740 1 --2- 172.21.15.16:0/1047970399 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5615ff162ea0 0x5615ff567500 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:40.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.102+0000 7fd767866740 1 --2- 172.21.15.16:0/1047970399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615ff183840 0x5615ff555410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:40.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.102+0000 7fd767866740 1 --2- 172.21.15.16:0/1047970399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ff1a4520 0x5615ff555b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:40.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.102+0000 7fd767866740 1 --2- 172.21.15.16:0/1047970399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615ff17b660 0x5615ff557ec0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:40.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.102+0000 7fd767866740 1 -- 172.21.15.16:0/1047970399 >> 172.21.15.16:0/1047970399 conn(0x5615ff198910 msgr2=0x5615ff54d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:40.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.102+0000 7fd73f400640 0 client.17874 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:18:40.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.102+0000 7fd767866740 1 -- 172.21.15.16:0/1047970399 shutdown_connections 2024-08-31T06:18:40.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.102+0000 7fd767866740 1 -- 172.21.15.16:0/1047970399 wait complete. 2024-08-31T06:18:40.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.103+0000 7fd767866740 20 client.17874 trim_cache size 0 max 16384 2024-08-31T06:18:40.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.121+0000 7fd767866740 1 Processor -- start 2024-08-31T06:18:40.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.121+0000 7fd767866740 1 -- start start 2024-08-31T06:18:40.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.121+0000 7fd767866740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615ff17b660 0x5615ff556fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:40.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.121+0000 7fd767866740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ff183840 0x5615ff557500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:40.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.121+0000 7fd767866740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615ff1a4520 0x5615ff547530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:40.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.121+0000 7fd767866740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c1b3d40 con 0x5615ff17b660 2024-08-31T06:18:40.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.121+0000 7fd767866740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c1b15b0 con 0x5615ff183840 2024-08-31T06:18:40.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.121+0000 7fd767866740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c1af420 con 0x5615ff1a4520 2024-08-31T06:18:40.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.121+0000 7fd759e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615ff17b660 0x5615ff556fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:40.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.121+0000 7fd759e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615ff17b660 0x5615ff556fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58938/0 (socket says 172.21.15.16:58938) 2024-08-31T06:18:40.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.121+0000 7fd759e00640 1 -- 172.21.15.16:0/497458586 learned_addr learned my addr 172.21.15.16:0/497458586 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:40.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.122+0000 7fd753e00640 1 --2- 172.21.15.16:0/497458586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ff183840 0x5615ff557500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:40.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.122+0000 7fd75a800640 1 --2- 172.21.15.16:0/497458586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615ff1a4520 0x5615ff547530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:40.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.122+0000 7fd753e00640 1 -- 172.21.15.16:0/497458586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615ff1a4520 msgr2=0x5615ff547530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:40.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.122+0000 7fd753e00640 1 --2- 172.21.15.16:0/497458586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615ff1a4520 0x5615ff547530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:40.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.122+0000 7fd753e00640 1 -- 172.21.15.16:0/497458586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615ff17b660 msgr2=0x5615ff556fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:40.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.122+0000 7fd753e00640 1 --2- 172.21.15.16:0/497458586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615ff17b660 0x5615ff556fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:40.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.122+0000 7fd753e00640 1 -- 172.21.15.16:0/497458586 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615ff555e90 con 0x5615ff183840 2024-08-31T06:18:40.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.122+0000 7fd753e00640 1 --2- 172.21.15.16:0/497458586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ff183840 0x5615ff557500 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x5615ff558840 tx=0x7fd744013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f453bb6d7944667 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:40.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.123+0000 7fd74a600640 1 -- 172.21.15.16:0/497458586 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd744011050 con 0x5615ff183840 2024-08-31T06:18:40.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.123+0000 7fd74a600640 1 -- 172.21.15.16:0/497458586 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7440111f0 con 0x5615ff183840 2024-08-31T06:18:40.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.123+0000 7fd74a600640 1 -- 172.21.15.16:0/497458586 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74402b670 con 0x5615ff183840 2024-08-31T06:18:40.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.123+0000 7fd767866740 1 -- 172.21.15.16:0/497458586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ff183840 msgr2=0x5615ff557500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:40.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.123+0000 7fd767866740 1 --2- 172.21.15.16:0/497458586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ff183840 0x5615ff557500 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x5615ff558840 tx=0x7fd744013040 comp rx=0 tx=0).stop 2024-08-31T06:18:40.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.123+0000 7fd767866740 1 -- 172.21.15.16:0/497458586 shutdown_connections 2024-08-31T06:18:40.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.123+0000 7fd767866740 1 --2- 172.21.15.16:0/497458586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615ff1a4520 0x5615ff547530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:40.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.123+0000 7fd767866740 1 --2- 172.21.15.16:0/497458586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ff183840 0x5615ff557500 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:40.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.123+0000 7fd767866740 1 --2- 172.21.15.16:0/497458586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615ff17b660 0x5615ff556fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:40.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.123+0000 7fd767866740 1 -- 172.21.15.16:0/497458586 >> 172.21.15.16:0/497458586 conn(0x5615ff198910 msgr2=0x5615ff5568b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:40.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.123+0000 7fd767866740 1 -- 172.21.15.16:0/497458586 shutdown_connections 2024-08-31T06:18:40.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.123+0000 7fd767866740 1 -- 172.21.15.16:0/497458586 wait complete. 2024-08-31T06:18:40.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.124+0000 7fd767866740 1 -- start start 2024-08-31T06:18:40.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.124+0000 7fd767866740 1 Processor -- start 2024-08-31T06:18:40.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.125+0000 7fd767866740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615ff17b660 0x5615ff5c5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:40.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.125+0000 7fd767866740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ff183840 0x5615ff5c63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:40.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.125+0000 7fd767866740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615ff1a4520 0x5615ff5cac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:40.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.125+0000 7fd767866740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c1afa70 con 0x5615ff17b660 2024-08-31T06:18:40.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.125+0000 7fd767866740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c1b0ee0 con 0x5615ff183840 2024-08-31T06:18:40.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.125+0000 7fd767866740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c1b0140 con 0x5615ff1a4520 2024-08-31T06:18:40.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.125+0000 7fd767866740 10 client.0 _ll_register_callbacks cb 0x7fd74c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:18:40.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.125+0000 7fd759e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615ff17b660 0x5615ff5c5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:40.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.125+0000 7fd753e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ff183840 0x5615ff5c63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:40.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.125+0000 7fd753e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ff183840 0x5615ff5c63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:41098/0 (socket says 172.21.15.16:41098) 2024-08-31T06:18:40.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.125+0000 7fd753e00640 1 -- 172.21.15.16:0/1126676289 learned_addr learned my addr 172.21.15.16:0/1126676289 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:40.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.125+0000 7fd75a800640 1 --2- 172.21.15.16:0/1126676289 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615ff1a4520 0x5615ff5cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:40.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.125+0000 7fd753e00640 1 -- 172.21.15.16:0/1126676289 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615ff1a4520 msgr2=0x5615ff5cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:40.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.125+0000 7fd753e00640 1 --2- 172.21.15.16:0/1126676289 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615ff1a4520 0x5615ff5cac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:40.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.125+0000 7fd753e00640 1 -- 172.21.15.16:0/1126676289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615ff17b660 msgr2=0x5615ff5c5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:40.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.125+0000 7fd753e00640 1 --2- 172.21.15.16:0/1126676289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615ff17b660 0x5615ff5c5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:40.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.125+0000 7fd753e00640 1 -- 172.21.15.16:0/1126676289 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615ff5cb2d0 con 0x5615ff183840 2024-08-31T06:18:40.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.126+0000 7fd753e00640 1 --2- 172.21.15.16:0/1126676289 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ff183840 0x5615ff5c63d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd744011470 tx=0x7fd74402b3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7a5d41ed17eac10 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:40.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.126+0000 7fd73f400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:18:40.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.126+0000 7fd73f400640 1 -- 172.21.15.16:0/1126676289 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74402b5f0 con 0x5615ff183840 2024-08-31T06:18:40.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.126+0000 7fd767866740 1 -- 172.21.15.16:0/1126676289 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7440111f0 con 0x5615ff183840 2024-08-31T06:18:40.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.126+0000 7fd73f400640 1 -- 172.21.15.16:0/1126676289 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8470+0+0 (secure 0 0 0) 0x7fd74402b790 con 0x5615ff183840 2024-08-31T06:18:40.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.126+0000 7fd73f400640 1 -- 172.21.15.16:0/1126676289 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fd74402b990 con 0x5615ff183840 2024-08-31T06:18:40.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.126+0000 7fd767866740 10 client.28244 fetch_fsmap learned FSMap version 47 2024-08-31T06:18:40.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.126+0000 7fd767866740 1 -- 172.21.15.16:0/1126676289 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5615ff5d2610 con 0x5615ff183840 2024-08-31T06:18:40.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.127+0000 7fd73f400640 1 -- 172.21.15.16:0/1126676289 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7440111f0 con 0x5615ff183840 2024-08-31T06:18:40.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.127+0000 7fd73f400640 1 -- 172.21.15.16:0/1126676289 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 47) ==== 8354+0+0 (secure 0 0 0) 0x7fd74403a070 con 0x5615ff183840 2024-08-31T06:18:40.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.127+0000 7fd767866740 10 client.28244 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T06:18:40.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.127+0000 7fd767866740 10 client.28244 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:18:40.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.127+0000 7fd767866740 20 client.28244 populate_metadata read hostname 'smithi016' 2024-08-31T06:18:40.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.127+0000 7fd767866740 1 --2- 172.21.15.16:0/1126676289 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5615ff138250 0x5615ff5d4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:40.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.127+0000 7fd767866740 4 client.28244 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f2a792d6", "release": true, "await": true}] 2024-08-31T06:18:40.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.127+0000 7fd767866740 1 -- 172.21.15.16:0/1126676289 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f2a792d6", "release": true, "await": true}) -- 0x5615ff4be480 con 0x5615ff138250 2024-08-31T06:18:40.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.127+0000 7fd759e00640 1 --2- 172.21.15.16:0/1126676289 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5615ff138250 0x5615ff5d4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:40.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.128+0000 7fd759e00640 1 --2- 172.21.15.16:0/1126676289 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5615ff138250 0x5615ff5d4db0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2463dc451e95a392 server_cookie=2b454371f38984a3 in_seq=0 out_seq=0 2024-08-31T06:18:40.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.128+0000 7fd73f400640 10 client.28244 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:18:40.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.345+0000 7fd73f400640 1 -- 172.21.15.16:0/1126676289 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5615ff4be480 con 0x5615ff138250 2024-08-31T06:18:40.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.345+0000 7fd73f400640 10 client.28244 handle_command_reply: tid=0 2024-08-31T06:18:40.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.345+0000 7fd767866740 1 client.28244 shutdown 2024-08-31T06:18:40.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.346+0000 7fd767866740 1 -- 172.21.15.16:0/1126676289 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ff183840 msgr2=0x5615ff5c63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:40.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.346+0000 7fd767866740 1 --2- 172.21.15.16:0/1126676289 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ff183840 0x5615ff5c63d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd744011470 tx=0x7fd74402b3d0 comp rx=0 tx=0).stop 2024-08-31T06:18:40.351 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:18:40.351 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 47, 2024-08-31T06:18:40.351 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24737, 2024-08-31T06:18:40.351 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 40, 2024-08-31T06:18:40.351 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-08-31T06:18:40.351 INFO:teuthology.orchestra.run.smithi016.stdout: "f2a792d6": { 2024-08-31T06:18:40.351 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 40, 2024-08-31T06:18:40.351 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-08-31T06:18:40.352 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:18:40.352 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-08-31T06:18:40.352 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:18:40.352 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:18:40.352 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-08-31T06:18:40.352 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 145.2, 2024-08-31T06:18:40.352 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-08-31T06:18:40.352 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-08-31T06:18:40.352 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-08-31T06:18:40.352 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:18:40.352 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-08-31T06:18:40.352 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:18:40.352 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:18:40.353 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:18:40.353 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:18:40.353 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:18:40.353 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:18:40.353 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:18:40.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.346+0000 7fd767866740 1 -- 172.21.15.16:0/1126676289 shutdown_connections 2024-08-31T06:18:40.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.346+0000 7fd767866740 1 --2- 172.21.15.16:0/1126676289 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5615ff138250 0x5615ff5d4db0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:40.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.346+0000 7fd767866740 1 --2- 172.21.15.16:0/1126676289 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615ff1a4520 0x5615ff5cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:40.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.346+0000 7fd767866740 1 --2- 172.21.15.16:0/1126676289 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ff183840 0x5615ff5c63d0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:40.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.346+0000 7fd767866740 1 --2- 172.21.15.16:0/1126676289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615ff17b660 0x5615ff5c5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:40.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.347+0000 7fd767866740 1 -- 172.21.15.16:0/1126676289 >> 172.21.15.16:0/1126676289 conn(0x5615ff198910 msgr2=0x5615ff557120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:40.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.347+0000 7fd73f400640 0 client.28244 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:18:40.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.347+0000 7fd767866740 1 -- 172.21.15.16:0/1126676289 shutdown_connections 2024-08-31T06:18:40.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.347+0000 7fd767866740 1 -- 172.21.15.16:0/1126676289 wait complete. 2024-08-31T06:18:40.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.347+0000 7fd767866740 20 client.28244 trim_cache size 0 max 16384 2024-08-31T06:18:40.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.350+0000 7fd75b200640 1 -- 172.21.15.16:0/3642170067 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd7400b4ca0 msgr2=0x7fd7400b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:40.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.350+0000 7fd75b200640 1 --2- 172.21.15.16:0/3642170067 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd7400b4ca0 0x7fd7400b71b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd744009fd0 tx=0x7fd744004730 comp rx=0 tx=0).stop 2024-08-31T06:18:40.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.350+0000 7fd75b200640 1 -- 172.21.15.16:0/3642170067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd75c10bbe0 msgr2=0x7fd75c1a1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:40.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.350+0000 7fd75b200640 1 --2- 172.21.15.16:0/3642170067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd75c10bbe0 0x7fd75c1a1ad0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fd75400ec50 tx=0x7fd75400c6a0 comp rx=0 tx=0).stop 2024-08-31T06:18:40.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.350+0000 7fd75b200640 1 -- 172.21.15.16:0/3642170067 shutdown_connections 2024-08-31T06:18:40.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.350+0000 7fd75b200640 1 --2- 172.21.15.16:0/3642170067 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd7400b4ca0 0x7fd7400b71b0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:40.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.350+0000 7fd75b200640 1 --2- 172.21.15.16:0/3642170067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd75c10b290 0x7fd75c19c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:40.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.350+0000 7fd75b200640 1 --2- 172.21.15.16:0/3642170067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd75c10bbe0 0x7fd75c1a1ad0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:40.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.350+0000 7fd75b200640 1 --2- 172.21.15.16:0/3642170067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd75c105950 0x7fd75c19c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:40.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.350+0000 7fd75b200640 1 -- 172.21.15.16:0/3642170067 >> 172.21.15.16:0/3642170067 conn(0x7fd75c101420 msgr2=0x7fd75c103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:40.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.352+0000 7fd75b200640 1 -- 172.21.15.16:0/3642170067 shutdown_connections 2024-08-31T06:18:40.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:18:40.352+0000 7fd75b200640 1 -- 172.21.15.16:0/3642170067 wait complete. 2024-08-31T06:18:40.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:40 smithi184 ceph-mon[22331]: pgmap v719: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 2.4 MiB/s wr, 341 op/s 2024-08-31T06:18:41.096 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:40 smithi016 ceph-mon[17194]: pgmap v719: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 2.4 MiB/s wr, 341 op/s 2024-08-31T06:18:41.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:40 smithi155 ceph-mon[21742]: pgmap v719: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 2.4 MiB/s wr, 341 op/s 2024-08-31T06:18:41.159 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f2a792d6', total seconds elapsed: 29.5 2024-08-31T06:18:41.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 675.0 s, 496.3 tps, lat 979.284 ms stddev 4549.041 2024-08-31T06:18:42.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:42 smithi184 ceph-mon[22331]: pgmap v720: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 1.0 MiB/s wr, 220 op/s 2024-08-31T06:18:43.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:42 smithi155 ceph-mon[21742]: pgmap v720: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 1.0 MiB/s wr, 220 op/s 2024-08-31T06:18:43.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:42 smithi016 ceph-mon[17194]: pgmap v720: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 1.0 MiB/s wr, 220 op/s 2024-08-31T06:18:44.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:44 smithi184 ceph-mon[22331]: pgmap v721: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 1.1 MiB/s wr, 186 op/s 2024-08-31T06:18:45.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:44 smithi155 ceph-mon[21742]: pgmap v721: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 1.1 MiB/s wr, 186 op/s 2024-08-31T06:18:45.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:44 smithi016 ceph-mon[17194]: pgmap v721: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 1.1 MiB/s wr, 186 op/s 2024-08-31T06:18:46.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 680.0 s, 1749.4 tps, lat 56.284 ms stddev 93.102 2024-08-31T06:18:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.709+0000 7f16f9400640 1 -- 172.21.15.16:0/2949464018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16f4113bb0 msgr2=0x7f16f4113fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.709+0000 7f16f9400640 1 --2- 172.21.15.16:0/2949464018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16f4113bb0 0x7f16f4113fc0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f16e0009860 tx=0x7f16e002a340 comp rx=0 tx=0).stop 2024-08-31T06:18:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.710+0000 7f16f9400640 1 -- 172.21.15.16:0/2949464018 shutdown_connections 2024-08-31T06:18:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.710+0000 7f16f9400640 1 --2- 172.21.15.16:0/2949464018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16f4114500 0x7f16f411ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.710+0000 7f16f9400640 1 --2- 172.21.15.16:0/2949464018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16f4113bb0 0x7f16f4113fc0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.710+0000 7f16f9400640 1 --2- 172.21.15.16:0/2949464018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16f410e280 0x7f16f410e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.710+0000 7f16f9400640 1 -- 172.21.15.16:0/2949464018 >> 172.21.15.16:0/2949464018 conn(0x7f16f410a160 msgr2=0x7f16f410c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:46.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.710+0000 7f16f9400640 1 -- 172.21.15.16:0/2949464018 shutdown_connections 2024-08-31T06:18:46.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.710+0000 7f16f9400640 1 -- 172.21.15.16:0/2949464018 wait complete. 2024-08-31T06:18:46.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.711+0000 7f16f9400640 1 Processor -- start 2024-08-31T06:18:46.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.711+0000 7f16f9400640 1 -- start start 2024-08-31T06:18:46.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.711+0000 7f16f9400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16f410e280 0x7f16f4119dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:46.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.711+0000 7f16f9400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16f4113bb0 0x7f16f411a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:46.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.711+0000 7f16f9400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16f4114500 0x7f16f41ab640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:46.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.711+0000 7f16f9400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f16f406fc40 con 0x7f16f410e280 2024-08-31T06:18:46.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.712+0000 7f16f9400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f16f406fac0 con 0x7f16f4113bb0 2024-08-31T06:18:46.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.712+0000 7f16f9400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f16f406fdc0 con 0x7f16f4114500 2024-08-31T06:18:46.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.712+0000 7f16f3200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16f4114500 0x7f16f41ab640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:46.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.712+0000 7f16f3200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16f4114500 0x7f16f41ab640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:43218/0 (socket says 172.21.15.16:43218) 2024-08-31T06:18:46.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.712+0000 7f16f3200640 1 -- 172.21.15.16:0/669775965 learned_addr learned my addr 172.21.15.16:0/669775965 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:46.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.712+0000 7f16f1e00640 1 --2- 172.21.15.16:0/669775965 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16f4113bb0 0x7f16f411a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:46.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.712+0000 7f16f2800640 1 --2- 172.21.15.16:0/669775965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16f410e280 0x7f16f4119dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:46.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.712+0000 7f16f3200640 1 -- 172.21.15.16:0/669775965 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16f4113bb0 msgr2=0x7f16f411a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:46.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.712+0000 7f16f3200640 1 --2- 172.21.15.16:0/669775965 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16f4113bb0 0x7f16f411a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:46.716 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.712+0000 7f16f3200640 1 -- 172.21.15.16:0/669775965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16f410e280 msgr2=0x7f16f4119dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:46.716 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.712+0000 7f16f3200640 1 --2- 172.21.15.16:0/669775965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16f410e280 0x7f16f4119dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:46.716 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.712+0000 7f16f3200640 1 -- 172.21.15.16:0/669775965 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16f41abd10 con 0x7f16f4114500 2024-08-31T06:18:46.716 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.713+0000 7f16f3200640 1 --2- 172.21.15.16:0/669775965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16f4114500 0x7f16f41ab640 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f16ec00eb10 tx=0x7f16ec00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:46.716 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.713+0000 7f16eb400640 1 -- 172.21.15.16:0/669775965 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ec004b20 con 0x7f16f4114500 2024-08-31T06:18:46.716 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.713+0000 7f16eb400640 1 -- 172.21.15.16:0/669775965 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16ec004cc0 con 0x7f16f4114500 2024-08-31T06:18:46.717 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.713+0000 7f16eb400640 1 -- 172.21.15.16:0/669775965 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ec010670 con 0x7f16f4114500 2024-08-31T06:18:46.717 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.713+0000 7f16f9400640 1 -- 172.21.15.16:0/669775965 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16f41ac000 con 0x7f16f4114500 2024-08-31T06:18:46.717 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.713+0000 7f16f9400640 1 -- 172.21.15.16:0/669775965 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16f411cfc0 con 0x7f16f4114500 2024-08-31T06:18:46.720 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.718+0000 7f16eb400640 1 -- 172.21.15.16:0/669775965 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f16ec005020 con 0x7f16f4114500 2024-08-31T06:18:46.720 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.718+0000 7f1700418740 1 Processor -- start 2024-08-31T06:18:46.721 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.718+0000 7f1700418740 1 -- start start 2024-08-31T06:18:46.721 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.719+0000 7f1700418740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55925273e1b0 0x559252ae1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:46.722 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.719+0000 7f1700418740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592527152f0 0x559252ae1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:46.722 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.719+0000 7f1700418740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55925271d4d0 0x559252ae2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:46.722 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.719+0000 7f1700418740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559252a56f70 con 0x55925273e1b0 2024-08-31T06:18:46.722 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.719+0000 7f1700418740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559252a46500 con 0x5592527152f0 2024-08-31T06:18:46.722 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.719+0000 7f1700418740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559252a4a4b0 con 0x55925271d4d0 2024-08-31T06:18:46.722 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.719+0000 7f16f1e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55925273e1b0 0x559252ae1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:46.723 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.719+0000 7f16f2800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592527152f0 0x559252ae1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:46.723 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.720+0000 7f16f1e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55925273e1b0 0x559252ae1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49260/0 (socket says 172.21.15.16:49260) 2024-08-31T06:18:46.723 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.720+0000 7f16f1e00640 1 -- 172.21.15.16:0/3084318291 learned_addr learned my addr 172.21.15.16:0/3084318291 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:46.723 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.720+0000 7f16f1e00640 1 --2- 172.21.15.16:0/3084318291 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55925271d4d0 0x559252ae2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:46.723 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.720+0000 7f16eb400640 1 --2- 172.21.15.16:0/669775965 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f16d80b4d50 0x7f16d80b7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:46.725 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.720+0000 7f16eb400640 1 -- 172.21.15.16:0/669775965 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8470+0+0 (secure 0 0 0) 0x7f16ec0e5ab0 con 0x7f16f4114500 2024-08-31T06:18:46.725 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.721+0000 7f16f3200640 1 --2- 172.21.15.16:0/669775965 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f16d80b4d50 0x7f16d80b7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:46.725 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.721+0000 7f16f2800640 1 -- 172.21.15.16:0/3084318291 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55925271d4d0 msgr2=0x559252ae2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:46.725 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.721+0000 7f16f2800640 1 --2- 172.21.15.16:0/3084318291 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55925271d4d0 0x559252ae2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:46.725 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.721+0000 7f16f2800640 1 -- 172.21.15.16:0/3084318291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55925273e1b0 msgr2=0x559252ae1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:46.725 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.721+0000 7f16f2800640 1 --2- 172.21.15.16:0/3084318291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55925273e1b0 0x559252ae1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:46.725 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.721+0000 7f16f2800640 1 -- 172.21.15.16:0/3084318291 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559252a00c10 con 0x5592527152f0 2024-08-31T06:18:46.725 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.722+0000 7f16f1e00640 1 --2- 172.21.15.16:0/3084318291 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55925271d4d0 0x559252ae2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:18:46.726 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.723+0000 7f16f2800640 1 --2- 172.21.15.16:0/3084318291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592527152f0 0x559252ae1bd0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f16e4009ae0 tx=0x7f16e4010370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50c546584c06f888 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:46.726 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.723+0000 7f16f3200640 1 --2- 172.21.15.16:0/669775965 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f16d80b4d50 0x7f16d80b7260 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f16ec0e32e0 tx=0x7f16ec00cee0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:46.726 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.724+0000 7f16df400640 1 -- 172.21.15.16:0/3084318291 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e4010d40 con 0x5592527152f0 2024-08-31T06:18:46.726 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.724+0000 7f16df400640 1 -- 172.21.15.16:0/3084318291 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16e40035f0 con 0x5592527152f0 2024-08-31T06:18:46.727 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.724+0000 7f16df400640 1 -- 172.21.15.16:0/3084318291 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e4012690 con 0x5592527152f0 2024-08-31T06:18:46.727 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.725+0000 7f1700418740 1 -- 172.21.15.16:0/3084318291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592527152f0 msgr2=0x559252ae1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:46.727 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.725+0000 7f1700418740 1 --2- 172.21.15.16:0/3084318291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592527152f0 0x559252ae1bd0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f16e4009ae0 tx=0x7f16e4010370 comp rx=0 tx=0).stop 2024-08-31T06:18:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.725+0000 7f1700418740 1 -- 172.21.15.16:0/3084318291 shutdown_connections 2024-08-31T06:18:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.725+0000 7f1700418740 1 --2- 172.21.15.16:0/3084318291 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55925271d4d0 0x559252ae2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.725+0000 7f1700418740 1 --2- 172.21.15.16:0/3084318291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592527152f0 0x559252ae1bd0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.725+0000 7f1700418740 1 --2- 172.21.15.16:0/3084318291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55925273e1b0 0x559252ae1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.725+0000 7f1700418740 1 -- 172.21.15.16:0/3084318291 >> 172.21.15.16:0/3084318291 conn(0x559252732300 msgr2=0x559252adf580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.725+0000 7f1700418740 1 -- 172.21.15.16:0/3084318291 shutdown_connections 2024-08-31T06:18:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.726+0000 7f1700418740 1 -- 172.21.15.16:0/3084318291 wait complete. 2024-08-31T06:18:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.726+0000 7f1700418740 1 -- start start 2024-08-31T06:18:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.727+0000 7f1700418740 1 Processor -- start 2024-08-31T06:18:46.730 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.728+0000 7f1700418740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592527152f0 0x559252aecc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:46.731 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.728+0000 7f1700418740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55925271d4d0 0x559252aed1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:46.731 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.728+0000 7f1700418740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55925273e1b0 0x559252aed700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:46.731 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.728+0000 7f1700418740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55925297fab0 con 0x5592527152f0 2024-08-31T06:18:46.731 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.728+0000 7f1700418740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5592529ed320 con 0x55925271d4d0 2024-08-31T06:18:46.731 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.728+0000 7f1700418740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559252aefe10 con 0x55925273e1b0 2024-08-31T06:18:46.731 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.728+0000 7f1700418740 10 client.0 _ll_register_callbacks cb 0x559252a26e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:18:46.731 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.728+0000 7f16f1e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592527152f0 0x559252aecc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.729+0000 7f16f1e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55925273e1b0 0x559252aed700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.729+0000 7f16f1e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55925273e1b0 0x559252aed700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:43234/0 (socket says 172.21.15.16:43234) 2024-08-31T06:18:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.729+0000 7f16f1e00640 1 -- 172.21.15.16:0/3730504899 learned_addr learned my addr 172.21.15.16:0/3730504899 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.729+0000 7f16f2800640 1 --2- 172.21.15.16:0/3730504899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55925271d4d0 0x559252aed1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.730+0000 7f16f1e00640 1 -- 172.21.15.16:0/3730504899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55925273e1b0 msgr2=0x559252aed700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.730+0000 7f16f1e00640 1 --2- 172.21.15.16:0/3730504899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55925273e1b0 0x559252aed700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.730+0000 7f16f1e00640 1 -- 172.21.15.16:0/3730504899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55925271d4d0 msgr2=0x559252aed1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.730+0000 7f16f1e00640 1 --2- 172.21.15.16:0/3730504899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55925271d4d0 0x559252aed1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.730+0000 7f16f1e00640 1 -- 172.21.15.16:0/3730504899 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592527a9790 con 0x5592527152f0 2024-08-31T06:18:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.731+0000 7f16f1e00640 1 --2- 172.21.15.16:0/3730504899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592527152f0 0x559252aecc80 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f16e0035040 tx=0x7f16e002a840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=131653d071584100 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.731+0000 7f16d3e00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:18:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.732+0000 7f16d3e00640 1 -- 172.21.15.16:0/3730504899 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e0037040 con 0x5592527152f0 2024-08-31T06:18:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.732+0000 7f16d3e00640 1 -- 172.21.15.16:0/3730504899 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8470+0+0 (secure 0 0 0) 0x7f16e0005250 con 0x5592527152f0 2024-08-31T06:18:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.733+0000 7f1700418740 1 -- 172.21.15.16:0/3730504899 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16e40035f0 con 0x5592527152f0 2024-08-31T06:18:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.734+0000 7f16d3e00640 1 -- 172.21.15.16:0/3730504899 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f16e00308e0 con 0x5592527152f0 2024-08-31T06:18:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.734+0000 7f1700418740 10 client.17889 fetch_fsmap learned FSMap version 47 2024-08-31T06:18:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.734+0000 7f1700418740 1 -- 172.21.15.16:0/3730504899 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x559252822620 con 0x5592527152f0 2024-08-31T06:18:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.734+0000 7f16d3e00640 1 -- 172.21.15.16:0/3730504899 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16e40035f0 con 0x5592527152f0 2024-08-31T06:18:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.735+0000 7f16d3e00640 1 -- 172.21.15.16:0/3730504899 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 47) ==== 8354+0+0 (secure 0 0 0) 0x7f16e003d040 con 0x5592527152f0 2024-08-31T06:18:46.738 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.735+0000 7f1700418740 10 client.17889 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T06:18:46.738 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.735+0000 7f1700418740 10 client.17889 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:18:46.738 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.735+0000 7f1700418740 20 client.17889 populate_metadata read hostname 'smithi016' 2024-08-31T06:18:46.738 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.735+0000 7f1700418740 1 --2- 172.21.15.16:0/3730504899 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5592526fcd80 0x559252afed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:46.738 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.735+0000 7f1700418740 4 client.17889 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:18:46.738 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.735+0000 7f1700418740 1 -- 172.21.15.16:0/3730504899 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559252a58ec0 con 0x5592526fcd80 2024-08-31T06:18:46.738 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.735+0000 7f16f2800640 1 --2- 172.21.15.16:0/3730504899 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5592526fcd80 0x559252afed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:46.739 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.736+0000 7f16d3e00640 10 client.17889 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:18:46.739 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.736+0000 7f16f2800640 1 --2- 172.21.15.16:0/3730504899 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5592526fcd80 0x559252afed30 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5d318211460a43c server_cookie=2eee7a8519a22ce2 in_seq=0 out_seq=0 2024-08-31T06:18:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.737+0000 7f16d3e00640 1 -- 172.21.15.16:0/3730504899 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559252a58ec0 con 0x5592526fcd80 2024-08-31T06:18:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.737+0000 7f16d3e00640 10 client.17889 handle_command_reply: tid=0 2024-08-31T06:18:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.738+0000 7f1700418740 1 client.17889 shutdown 2024-08-31T06:18:46.743 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.740+0000 7f1700418740 1 -- 172.21.15.16:0/3730504899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592527152f0 msgr2=0x559252aecc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:46.743 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.740+0000 7f1700418740 1 --2- 172.21.15.16:0/3730504899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592527152f0 0x559252aecc80 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f16e0035040 tx=0x7f16e002a840 comp rx=0 tx=0).stop 2024-08-31T06:18:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.742+0000 7f1700418740 1 -- 172.21.15.16:0/3730504899 shutdown_connections 2024-08-31T06:18:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.742+0000 7f1700418740 1 --2- 172.21.15.16:0/3730504899 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5592526fcd80 0x559252afed30 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.743+0000 7f1700418740 1 --2- 172.21.15.16:0/3730504899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55925273e1b0 0x559252aed700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.743+0000 7f1700418740 1 --2- 172.21.15.16:0/3730504899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55925271d4d0 0x559252aed1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.743+0000 7f1700418740 1 --2- 172.21.15.16:0/3730504899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592527152f0 0x559252aecc80 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.743+0000 7f1700418740 1 -- 172.21.15.16:0/3730504899 >> 172.21.15.16:0/3730504899 conn(0x559252732300 msgr2=0x559252ae26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.743+0000 7f16d3e00640 0 client.17889 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:18:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.743+0000 7f1700418740 1 -- 172.21.15.16:0/3730504899 shutdown_connections 2024-08-31T06:18:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.743+0000 7f1700418740 1 -- 172.21.15.16:0/3730504899 wait complete. 2024-08-31T06:18:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.744+0000 7f1700418740 20 client.17889 trim_cache size 0 max 16384 2024-08-31T06:18:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.786+0000 7f1700418740 1 Processor -- start 2024-08-31T06:18:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.787+0000 7f1700418740 1 -- start start 2024-08-31T06:18:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.787+0000 7f1700418740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592527152f0 0x559252ae2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.787+0000 7f1700418740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55925271d4d0 0x559252ae9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.787+0000 7f1700418740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55925273e1b0 0x559252aea390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.787+0000 7f1700418740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f16f41c9430 con 0x55925273e1b0 2024-08-31T06:18:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.788+0000 7f1700418740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f16f41c6ca0 con 0x5592527152f0 2024-08-31T06:18:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.788+0000 7f1700418740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f16f41238d0 con 0x55925271d4d0 2024-08-31T06:18:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.788+0000 7f16f1e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592527152f0 0x559252ae2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.788+0000 7f16f1e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592527152f0 0x559252ae2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:47916/0 (socket says 172.21.15.16:47916) 2024-08-31T06:18:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.788+0000 7f16f1e00640 1 -- 172.21.15.16:0/713724240 learned_addr learned my addr 172.21.15.16:0/713724240 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.789+0000 7f16f2800640 1 --2- 172.21.15.16:0/713724240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55925271d4d0 0x559252ae9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.790+0000 7f16f2800640 1 -- 172.21.15.16:0/713724240 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592527152f0 msgr2=0x559252ae2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.790+0000 7f16f2800640 1 --2- 172.21.15.16:0/713724240 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592527152f0 0x559252ae2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.790+0000 7f16f2800640 1 -- 172.21.15.16:0/713724240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55925273e1b0 msgr2=0x559252aea390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.790+0000 7f16f2800640 1 --2- 172.21.15.16:0/713724240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55925273e1b0 0x559252aea390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.791+0000 7f16f2800640 1 -- 172.21.15.16:0/713724240 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559252ae3990 con 0x55925271d4d0 2024-08-31T06:18:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.791+0000 7f16f1e00640 1 --2- 172.21.15.16:0/713724240 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592527152f0 0x559252ae2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:18:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.792+0000 7f16f2800640 1 --2- 172.21.15.16:0/713724240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55925271d4d0 0x559252ae9e50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f16e4009ab0 tx=0x7f16e4009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=542848da70ee3b95 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.792+0000 7f16dea00640 1 -- 172.21.15.16:0/713724240 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e4010d40 con 0x55925271d4d0 2024-08-31T06:18:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.792+0000 7f16dea00640 1 -- 172.21.15.16:0/713724240 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16e4005040 con 0x55925271d4d0 2024-08-31T06:18:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.792+0000 7f16dea00640 1 -- 172.21.15.16:0/713724240 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e40048c0 con 0x55925271d4d0 2024-08-31T06:18:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.793+0000 7f1700418740 1 -- 172.21.15.16:0/713724240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55925271d4d0 msgr2=0x559252ae9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.793+0000 7f1700418740 1 --2- 172.21.15.16:0/713724240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55925271d4d0 0x559252ae9e50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f16e4009ab0 tx=0x7f16e4009260 comp rx=0 tx=0).stop 2024-08-31T06:18:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.794+0000 7f1700418740 1 -- 172.21.15.16:0/713724240 shutdown_connections 2024-08-31T06:18:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.794+0000 7f1700418740 1 --2- 172.21.15.16:0/713724240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55925271d4d0 0x559252ae9e50 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.794+0000 7f1700418740 1 --2- 172.21.15.16:0/713724240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55925273e1b0 0x559252aea390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.794+0000 7f1700418740 1 --2- 172.21.15.16:0/713724240 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592527152f0 0x559252ae2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.794+0000 7f1700418740 1 -- 172.21.15.16:0/713724240 >> 172.21.15.16:0/713724240 conn(0x559252732300 msgr2=0x559252ae26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.794+0000 7f1700418740 1 -- 172.21.15.16:0/713724240 shutdown_connections 2024-08-31T06:18:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.794+0000 7f1700418740 1 -- 172.21.15.16:0/713724240 wait complete. 2024-08-31T06:18:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.795+0000 7f1700418740 1 -- start start 2024-08-31T06:18:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.795+0000 7f1700418740 1 Processor -- start 2024-08-31T06:18:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.797+0000 7f1700418740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592527152f0 0x559252af56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.797+0000 7f1700418740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55925271d4d0 0x559252af2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.797+0000 7f1700418740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55925273e1b0 0x559252af30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.797+0000 7f1700418740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f16f4123f20 con 0x55925271d4d0 2024-08-31T06:18:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.797+0000 7f1700418740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f16f4125390 con 0x55925273e1b0 2024-08-31T06:18:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.797+0000 7f1700418740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f16f41245f0 con 0x5592527152f0 2024-08-31T06:18:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.797+0000 7f1700418740 10 client.0 _ll_register_callbacks cb 0x7f16e0030570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:18:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.798+0000 7f16f1e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592527152f0 0x559252af56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.798+0000 7f16f1e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592527152f0 0x559252af56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:43252/0 (socket says 172.21.15.16:43252) 2024-08-31T06:18:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.798+0000 7f16f1e00640 1 -- 172.21.15.16:0/1219770497 learned_addr learned my addr 172.21.15.16:0/1219770497 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:18:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.798+0000 7f16f2800640 1 --2- 172.21.15.16:0/1219770497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55925271d4d0 0x559252af2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.798+0000 7f16f1e00640 1 --2- 172.21.15.16:0/1219770497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55925273e1b0 0x559252af30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.799+0000 7f16f1e00640 1 -- 172.21.15.16:0/1219770497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55925273e1b0 msgr2=0x559252af30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.799+0000 7f16f1e00640 1 --2- 172.21.15.16:0/1219770497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55925273e1b0 0x559252af30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.799+0000 7f16f1e00640 1 -- 172.21.15.16:0/1219770497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55925271d4d0 msgr2=0x559252af2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:18:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.799+0000 7f16f1e00640 1 --2- 172.21.15.16:0/1219770497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55925271d4d0 0x559252af2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.799+0000 7f16f1e00640 1 -- 172.21.15.16:0/1219770497 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559252af3690 con 0x5592527152f0 2024-08-31T06:18:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.799+0000 7f16d3e00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:18:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.799+0000 7f16f1e00640 1 --2- 172.21.15.16:0/1219770497 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592527152f0 0x559252af56a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f16e0009780 tx=0x7f16e0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=134dc6c4e1329966 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:18:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.800+0000 7f16d3e00640 1 -- 172.21.15.16:0/1219770497 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e0037040 con 0x5592527152f0 2024-08-31T06:18:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.800+0000 7f16d3e00640 1 -- 172.21.15.16:0/1219770497 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8470+0+0 (secure 0 0 0) 0x7f16e0032880 con 0x5592527152f0 2024-08-31T06:18:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.800+0000 7f1700418740 1 -- 172.21.15.16:0/1219770497 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16e4005040 con 0x5592527152f0 2024-08-31T06:18:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.800+0000 7f16d3e00640 1 -- 172.21.15.16:0/1219770497 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f16e0052080 con 0x5592527152f0 2024-08-31T06:18:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.800+0000 7f1700418740 10 client.28270 fetch_fsmap learned FSMap version 47 2024-08-31T06:18:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.801+0000 7f1700418740 1 -- 172.21.15.16:0/1219770497 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x559252b694f0 con 0x5592527152f0 2024-08-31T06:18:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.801+0000 7f16d3e00640 1 -- 172.21.15.16:0/1219770497 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16e4005040 con 0x5592527152f0 2024-08-31T06:18:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.803+0000 7f16d3e00640 1 -- 172.21.15.16:0/1219770497 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 47) ==== 8354+0+0 (secure 0 0 0) 0x7f16e003d070 con 0x5592527152f0 2024-08-31T06:18:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.803+0000 7f1700418740 10 client.28270 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T06:18:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.803+0000 7f1700418740 10 client.28270 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:18:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.803+0000 7f1700418740 20 client.28270 populate_metadata read hostname 'smithi016' 2024-08-31T06:18:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.803+0000 7f1700418740 1 --2- 172.21.15.16:0/1219770497 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5592526d21c0 0x559252b6bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:18:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.803+0000 7f1700418740 4 client.28270 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:18:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.803+0000 7f1700418740 1 -- 172.21.15.16:0/1219770497 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559252a54960 con 0x5592526d21c0 2024-08-31T06:18:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.804+0000 7f16f2800640 1 --2- 172.21.15.16:0/1219770497 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5592526d21c0 0x559252b6bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:18:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.804+0000 7f16f2800640 1 --2- 172.21.15.16:0/1219770497 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5592526d21c0 0x559252b6bbf0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acaca67c431c5227 server_cookie=3bb817cc7765472b in_seq=0 out_seq=0 2024-08-31T06:18:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.804+0000 7f16d3e00640 10 client.28270 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:18:46.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.830+0000 7f16d3e00640 1 -- 172.21.15.16:0/1219770497 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559252a54960 con 0x5592526d21c0 2024-08-31T06:18:46.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.830+0000 7f16d3e00640 10 client.28270 handle_command_reply: tid=0 2024-08-31T06:18:46.833 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.830+0000 7f1700418740 1 client.28270 shutdown 2024-08-31T06:18:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.831+0000 7f1700418740 1 -- 172.21.15.16:0/1219770497 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592527152f0 msgr2=0x559252af56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.831+0000 7f1700418740 1 --2- 172.21.15.16:0/1219770497 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592527152f0 0x559252af56a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f16e0009780 tx=0x7f16e0000bc0 comp rx=0 tx=0).stop 2024-08-31T06:18:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.831+0000 7f1700418740 1 -- 172.21.15.16:0/1219770497 shutdown_connections 2024-08-31T06:18:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.831+0000 7f1700418740 1 --2- 172.21.15.16:0/1219770497 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5592526d21c0 0x559252b6bbf0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.831+0000 7f1700418740 1 --2- 172.21.15.16:0/1219770497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55925273e1b0 0x559252af30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.831+0000 7f1700418740 1 --2- 172.21.15.16:0/1219770497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55925271d4d0 0x559252af2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.831+0000 7f1700418740 1 --2- 172.21.15.16:0/1219770497 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592527152f0 0x559252af56a0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.831+0000 7f1700418740 1 -- 172.21.15.16:0/1219770497 >> 172.21.15.16:0/1219770497 conn(0x559252732300 msgr2=0x559252adf500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.831+0000 7f16d3e00640 0 client.28270 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:18:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.831+0000 7f1700418740 1 -- 172.21.15.16:0/1219770497 shutdown_connections 2024-08-31T06:18:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.831+0000 7f1700418740 1 -- 172.21.15.16:0/1219770497 wait complete. 2024-08-31T06:18:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.832+0000 7f1700418740 20 client.28270 trim_cache size 0 max 16384 2024-08-31T06:18:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:18:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:18:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:18:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:18:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.832+0000 7f16f9400640 1 -- 172.21.15.16:0/669775965 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f16d80b4d50 msgr2=0x7f16d80b7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.832+0000 7f16f9400640 1 --2- 172.21.15.16:0/669775965 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f16d80b4d50 0x7f16d80b7260 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f16ec0e32e0 tx=0x7f16ec00cee0 comp rx=0 tx=0).stop 2024-08-31T06:18:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.832+0000 7f16f9400640 1 -- 172.21.15.16:0/669775965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16f4114500 msgr2=0x7f16f41ab640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:18:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.832+0000 7f16f9400640 1 --2- 172.21.15.16:0/669775965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16f4114500 0x7f16f41ab640 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f16ec00eb10 tx=0x7f16ec00c2d0 comp rx=0 tx=0).stop 2024-08-31T06:18:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.833+0000 7f16f9400640 1 -- 172.21.15.16:0/669775965 shutdown_connections 2024-08-31T06:18:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.833+0000 7f16f9400640 1 --2- 172.21.15.16:0/669775965 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f16d80b4d50 0x7f16d80b7260 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.833+0000 7f16f9400640 1 --2- 172.21.15.16:0/669775965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16f4114500 0x7f16f41ab640 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.833+0000 7f16f9400640 1 --2- 172.21.15.16:0/669775965 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16f4113bb0 0x7f16f411a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.833+0000 7f16f9400640 1 --2- 172.21.15.16:0/669775965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16f410e280 0x7f16f4119dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:18:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.833+0000 7f16f9400640 1 -- 172.21.15.16:0/669775965 >> 172.21.15.16:0/669775965 conn(0x7f16f410a160 msgr2=0x7f16f4072410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:18:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.833+0000 7f16f9400640 1 -- 172.21.15.16:0/669775965 shutdown_connections 2024-08-31T06:18:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:18:46.834+0000 7f16f9400640 1 -- 172.21.15.16:0/669775965 wait complete. 2024-08-31T06:18:46.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:46 smithi184 ceph-mon[22331]: pgmap v722: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 4.4 MiB/s wr, 391 op/s 2024-08-31T06:18:47.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:46 smithi155 ceph-mon[21742]: pgmap v722: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 4.4 MiB/s wr, 391 op/s 2024-08-31T06:18:47.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:46 smithi016 ceph-mon[17194]: pgmap v722: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 4.4 MiB/s wr, 391 op/s 2024-08-31T06:18:48.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:48 smithi184 ceph-mon[22331]: pgmap v723: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 6.0 MiB/s wr, 478 op/s 2024-08-31T06:18:48.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:48 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:18:49.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:48 smithi155 ceph-mon[21742]: pgmap v723: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 6.0 MiB/s wr, 478 op/s 2024-08-31T06:18:49.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:48 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:18:49.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:48 smithi016 ceph-mon[17194]: pgmap v723: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 6.0 MiB/s wr, 478 op/s 2024-08-31T06:18:49.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:48 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:18:49.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:49 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:18:49.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:49 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:18:50.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:49 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:18:50.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:49 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:18:50.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:49 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:18:50.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:49 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:18:51.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:50 smithi155 ceph-mon[21742]: pgmap v724: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 10 MiB/s wr, 851 op/s 2024-08-31T06:18:51.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:50 smithi016 ceph-mon[17194]: pgmap v724: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 10 MiB/s wr, 851 op/s 2024-08-31T06:18:51.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:50 smithi184 ceph-mon[22331]: pgmap v724: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 10 MiB/s wr, 851 op/s 2024-08-31T06:18:51.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 685.0 s, 2186.2 tps, lat 47.267 ms stddev 64.980 2024-08-31T06:18:53.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:52 smithi155 ceph-mon[21742]: pgmap v725: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 11 MiB/s wr, 941 op/s 2024-08-31T06:18:53.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:52 smithi016 ceph-mon[17194]: pgmap v725: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 11 MiB/s wr, 941 op/s 2024-08-31T06:18:53.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:52 smithi184 ceph-mon[22331]: pgmap v725: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 11 MiB/s wr, 941 op/s 2024-08-31T06:18:55.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:54 smithi155 ceph-mon[21742]: pgmap v726: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 12 MiB/s wr, 1.03k op/s 2024-08-31T06:18:55.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:54 smithi016 ceph-mon[17194]: pgmap v726: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 12 MiB/s wr, 1.03k op/s 2024-08-31T06:18:55.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:54 smithi184 ceph-mon[22331]: pgmap v726: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 12 MiB/s wr, 1.03k op/s 2024-08-31T06:18:56.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 690.0 s, 2054.6 tps, lat 48.391 ms stddev 56.837 2024-08-31T06:18:57.085 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:56 smithi016 ceph-mon[17194]: pgmap v727: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 15 MiB/s wr, 1.20k op/s 2024-08-31T06:18:57.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:56 smithi155 ceph-mon[21742]: pgmap v727: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 15 MiB/s wr, 1.20k op/s 2024-08-31T06:18:57.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:56 smithi184 ceph-mon[22331]: pgmap v727: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 15 MiB/s wr, 1.20k op/s 2024-08-31T06:18:59.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:18:58 smithi155 ceph-mon[21742]: pgmap v728: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 12 MiB/s wr, 966 op/s 2024-08-31T06:18:59.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:18:58 smithi016 ceph-mon[17194]: pgmap v728: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 12 MiB/s wr, 966 op/s 2024-08-31T06:18:59.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:18:58 smithi184 ceph-mon[22331]: pgmap v728: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 12 MiB/s wr, 966 op/s 2024-08-31T06:19:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.138+0000 7eff04200640 1 -- 172.21.15.16:0/3107151291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efefc105a90 msgr2=0x7efefc105ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.138+0000 7eff04200640 1 --2- 172.21.15.16:0/3107151291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efefc105a90 0x7efefc105ea0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7efef0009860 tx=0x7efef002a340 comp rx=0 tx=0).stop 2024-08-31T06:19:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.142+0000 7eff04200640 1 -- 172.21.15.16:0/3107151291 shutdown_connections 2024-08-31T06:19:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.142+0000 7eff04200640 1 --2- 172.21.15.16:0/3107151291 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efefc10b3c0 0x7efefc10b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.142+0000 7eff04200640 1 --2- 172.21.15.16:0/3107151291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efefc10bd10 0x7efefc112540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.142+0000 7eff04200640 1 --2- 172.21.15.16:0/3107151291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efefc105a90 0x7efefc105ea0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.142+0000 7eff04200640 1 -- 172.21.15.16:0/3107151291 >> 172.21.15.16:0/3107151291 conn(0x7efefc101490 msgr2=0x7efefc1038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.142+0000 7eff04200640 1 -- 172.21.15.16:0/3107151291 shutdown_connections 2024-08-31T06:19:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.142+0000 7eff04200640 1 -- 172.21.15.16:0/3107151291 wait complete. 2024-08-31T06:19:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.144+0000 7eff04200640 1 Processor -- start 2024-08-31T06:19:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.144+0000 7eff04200640 1 -- start start 2024-08-31T06:19:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.145+0000 7eff04200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efefc105a90 0x7efefc10fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.145+0000 7eff04200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efefc10b3c0 0x7efefc10ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.145+0000 7eff04200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efefc10bd10 0x7efefc110480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.145+0000 7eff04200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efefc1159d0 con 0x7efefc105a90 2024-08-31T06:19:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.145+0000 7eff04200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efefc115850 con 0x7efefc10b3c0 2024-08-31T06:19:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.145+0000 7eff04200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efefc115b50 con 0x7efefc10bd10 2024-08-31T06:19:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.145+0000 7eff02200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efefc10bd10 0x7efefc110480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.145+0000 7eff02200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efefc10bd10 0x7efefc110480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:51796/0 (socket says 172.21.15.16:51796) 2024-08-31T06:19:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.145+0000 7eff02200640 1 -- 172.21.15.16:0/692235250 learned_addr learned my addr 172.21.15.16:0/692235250 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.146+0000 7eff01800640 1 --2- 172.21.15.16:0/692235250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efefc105a90 0x7efefc10fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.146+0000 7eff02200640 1 -- 172.21.15.16:0/692235250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efefc10b3c0 msgr2=0x7efefc10ff40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-31T06:19:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.146+0000 7eff02200640 1 --2- 172.21.15.16:0/692235250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efefc10b3c0 0x7efefc10ff40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.146+0000 7eff02200640 1 -- 172.21.15.16:0/692235250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efefc105a90 msgr2=0x7efefc10fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.146+0000 7eff02200640 1 --2- 172.21.15.16:0/692235250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efefc105a90 0x7efefc10fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.146+0000 7eff02200640 1 -- 172.21.15.16:0/692235250 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efefc1a2070 con 0x7efefc10bd10 2024-08-31T06:19:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.146+0000 7eff01800640 1 --2- 172.21.15.16:0/692235250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efefc105a90 0x7efefc10fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:19:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.146+0000 7eff02200640 1 --2- 172.21.15.16:0/692235250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efefc10bd10 0x7efefc110480 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7efef400eb10 tx=0x7efef400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.148+0000 7efefa000640 1 -- 172.21.15.16:0/692235250 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef4004b20 con 0x7efefc10bd10 2024-08-31T06:19:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.148+0000 7eff04200640 1 -- 172.21.15.16:0/692235250 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efefc1a2360 con 0x7efefc10bd10 2024-08-31T06:19:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.148+0000 7eff04200640 1 -- 172.21.15.16:0/692235250 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efefc1a2900 con 0x7efefc10bd10 2024-08-31T06:19:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.148+0000 7efefa000640 1 -- 172.21.15.16:0/692235250 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efef4004cc0 con 0x7efefc10bd10 2024-08-31T06:19:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.148+0000 7efefa000640 1 -- 172.21.15.16:0/692235250 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef4010720 con 0x7efefc10bd10 2024-08-31T06:19:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.162+0000 7eff0b280740 1 Processor -- start 2024-08-31T06:19:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.163+0000 7efefa000640 1 -- 172.21.15.16:0/692235250 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7efef4005020 con 0x7efefc10bd10 2024-08-31T06:19:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.163+0000 7eff0b280740 1 -- start start 2024-08-31T06:19:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.163+0000 7eff0b280740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ccc2cba1b0 0x55ccc305d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.163+0000 7eff0b280740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccc2c912f0 0x55ccc305de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.163+0000 7eff0b280740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc2c994d0 0x55ccc305e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.163+0000 7eff0b280740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ccc2fd2f70 con 0x55ccc2c994d0 2024-08-31T06:19:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.163+0000 7eff0b280740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ccc2fc2500 con 0x55ccc2c912f0 2024-08-31T06:19:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.163+0000 7eff0b280740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ccc2fc64b0 con 0x55ccc2cba1b0 2024-08-31T06:19:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.164+0000 7eff00e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ccc2cba1b0 0x55ccc305d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.164+0000 7eff00e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ccc2cba1b0 0x55ccc305d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:51800/0 (socket says 172.21.15.16:51800) 2024-08-31T06:19:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.164+0000 7eff00e00640 1 -- 172.21.15.16:0/1957794469 learned_addr learned my addr 172.21.15.16:0/1957794469 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.164+0000 7eff00e00640 1 -- 172.21.15.16:0/1957794469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccc2c912f0 msgr2=0x55ccc305de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.164+0000 7eff00e00640 1 --2- 172.21.15.16:0/1957794469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccc2c912f0 0x55ccc305de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.164+0000 7eff00e00640 1 -- 172.21.15.16:0/1957794469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc2c994d0 msgr2=0x55ccc305e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.164+0000 7eff00e00640 1 --2- 172.21.15.16:0/1957794469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc2c994d0 0x55ccc305e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.164+0000 7eff00e00640 1 -- 172.21.15.16:0/1957794469 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccc2f7cc10 con 0x55ccc2cba1b0 2024-08-31T06:19:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.165+0000 7eff00e00640 1 --2- 172.21.15.16:0/1957794469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ccc2cba1b0 0x55ccc305d920 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7efee8009b40 tx=0x7efee80103d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be9e09317865cd86 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.165+0000 7efeeea00640 1 -- 172.21.15.16:0/1957794469 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee8010e40 con 0x55ccc2cba1b0 2024-08-31T06:19:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.165+0000 7efeeea00640 1 -- 172.21.15.16:0/1957794469 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efee8015030 con 0x55ccc2cba1b0 2024-08-31T06:19:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.167+0000 7eff0b280740 1 -- 172.21.15.16:0/1957794469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ccc2cba1b0 msgr2=0x55ccc305d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:00.169 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.167+0000 7eff0b280740 1 --2- 172.21.15.16:0/1957794469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ccc2cba1b0 0x55ccc305d920 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7efee8009b40 tx=0x7efee80103d0 comp rx=0 tx=0).stop 2024-08-31T06:19:00.169 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.167+0000 7efefa000640 1 --2- 172.21.15.16:0/692235250 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7efee40b4ca0 0x7efee40b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:00.169 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.167+0000 7efefa000640 1 -- 172.21.15.16:0/692235250 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8470+0+0 (secure 0 0 0) 0x7efef40e59a0 con 0x7efefc10bd10 2024-08-31T06:19:00.169 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.168+0000 7eff0b280740 1 -- 172.21.15.16:0/1957794469 shutdown_connections 2024-08-31T06:19:00.170 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.168+0000 7eff0b280740 1 --2- 172.21.15.16:0/1957794469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc2c994d0 0x55ccc305e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:00.170 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.168+0000 7eff0b280740 1 --2- 172.21.15.16:0/1957794469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccc2c912f0 0x55ccc305de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:00.170 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.168+0000 7eff0b280740 1 --2- 172.21.15.16:0/1957794469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ccc2cba1b0 0x55ccc305d920 secure :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7efee8009b40 tx=0x7efee80103d0 comp rx=0 tx=0).stop 2024-08-31T06:19:00.170 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.168+0000 7eff0b280740 1 -- 172.21.15.16:0/1957794469 >> 172.21.15.16:0/1957794469 conn(0x55ccc2cae300 msgr2=0x55ccc305b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:00.171 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.169+0000 7eff0b280740 1 -- 172.21.15.16:0/1957794469 shutdown_connections 2024-08-31T06:19:00.171 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.169+0000 7eff0b280740 1 -- 172.21.15.16:0/1957794469 wait complete. 2024-08-31T06:19:00.171 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.169+0000 7eff00e00640 1 --2- 172.21.15.16:0/692235250 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7efee40b4ca0 0x7efee40b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.170+0000 7eff0b280740 1 -- start start 2024-08-31T06:19:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.170+0000 7eff0b280740 1 Processor -- start 2024-08-31T06:19:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.171+0000 7eff0b280740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc2c912f0 0x55ccc3068e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.171+0000 7eff0b280740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ccc2c994d0 0x55ccc3069370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.171+0000 7eff0b280740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccc2c78d80 0x55ccc30698b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.171+0000 7eff0b280740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ccc2efbab0 con 0x55ccc2c912f0 2024-08-31T06:19:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.171+0000 7eff0b280740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ccc2f69320 con 0x55ccc2c78d80 2024-08-31T06:19:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.171+0000 7eff0b280740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ccc306c0f0 con 0x55ccc2c994d0 2024-08-31T06:19:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.171+0000 7eff02200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccc2c78d80 0x55ccc30698b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.172+0000 7eff02200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccc2c78d80 0x55ccc30698b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51094/0 (socket says 172.21.15.16:51094) 2024-08-31T06:19:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.172+0000 7eff02200640 1 -- 172.21.15.16:0/2065717102 learned_addr learned my addr 172.21.15.16:0/2065717102 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.172+0000 7eff0b280740 10 client.0 _ll_register_callbacks cb 0x55ccc2f6f6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:19:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.172+0000 7eff02200640 1 -- 172.21.15.16:0/2065717102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ccc2c994d0 msgr2=0x55ccc3069370 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T06:19:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.172+0000 7eff02200640 1 --2- 172.21.15.16:0/2065717102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ccc2c994d0 0x55ccc3069370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.172+0000 7eff02200640 1 -- 172.21.15.16:0/2065717102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc2c912f0 msgr2=0x55ccc3068e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.172+0000 7eff02200640 1 --2- 172.21.15.16:0/2065717102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc2c912f0 0x55ccc3068e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.172+0000 7eff02200640 1 -- 172.21.15.16:0/2065717102 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccc2d25790 con 0x55ccc2c78d80 2024-08-31T06:19:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.172+0000 7efee3400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:19:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.172+0000 7eff02200640 1 --2- 172.21.15.16:0/2065717102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccc2c78d80 0x55ccc30698b0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7efef40abdc0 tx=0x7efef40abbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0fef0a79edb1041 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.173+0000 7efee3400640 1 -- 172.21.15.16:0/2065717102 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef40ac0c0 con 0x55ccc2c78d80 2024-08-31T06:19:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.173+0000 7eff0b280740 1 -- 172.21.15.16:0/2065717102 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efee8015030 con 0x55ccc2c78d80 2024-08-31T06:19:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.173+0000 7eff00e00640 1 --2- 172.21.15.16:0/692235250 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7efee40b4ca0 0x7efee40b71b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7efee8009530 tx=0x7efee8009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.173+0000 7efee3400640 1 -- 172.21.15.16:0/2065717102 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8470+0+0 (secure 0 0 0) 0x7efef40a86b0 con 0x55ccc2c78d80 2024-08-31T06:19:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.175+0000 7efee3400640 1 -- 172.21.15.16:0/2065717102 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7efef40ac0c0 con 0x55ccc2c78d80 2024-08-31T06:19:00.178 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.177+0000 7eff0b280740 10 client.28268 fetch_fsmap learned FSMap version 47 2024-08-31T06:19:00.178 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.177+0000 7eff0b280740 1 -- 172.21.15.16:0/2065717102 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55ccc2d9e620 con 0x55ccc2c78d80 2024-08-31T06:19:00.178 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.177+0000 7efee3400640 1 -- 172.21.15.16:0/2065717102 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efee8015030 con 0x55ccc2c78d80 2024-08-31T06:19:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.178+0000 7efee3400640 1 -- 172.21.15.16:0/2065717102 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 47) ==== 8354+0+0 (secure 0 0 0) 0x7efef40a7a30 con 0x55ccc2c78d80 2024-08-31T06:19:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.178+0000 7eff0b280740 10 client.28268 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T06:19:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.178+0000 7eff0b280740 10 client.28268 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:19:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.178+0000 7eff0b280740 20 client.28268 populate_metadata read hostname 'smithi016' 2024-08-31T06:19:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.178+0000 7eff0b280740 1 --2- 172.21.15.16:0/2065717102 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55ccc2c4e1c0 0x55ccc308c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.178+0000 7eff0b280740 4 client.28268 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:19:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.178+0000 7eff0b280740 1 -- 172.21.15.16:0/2065717102 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccc2fd4ec0 con 0x55ccc2c4e1c0 2024-08-31T06:19:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.178+0000 7eff01800640 1 --2- 172.21.15.16:0/2065717102 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55ccc2c4e1c0 0x55ccc308c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.179+0000 7eff01800640 1 --2- 172.21.15.16:0/2065717102 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55ccc2c4e1c0 0x55ccc308c190 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ef52899e6a5f284 server_cookie=f0a67b8fc15e309c in_seq=0 out_seq=0 2024-08-31T06:19:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.179+0000 7efee3400640 10 client.28268 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:19:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.182+0000 7efee3400640 1 -- 172.21.15.16:0/2065717102 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ccc2fd4ec0 con 0x55ccc2c4e1c0 2024-08-31T06:19:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.182+0000 7efee3400640 10 client.28268 handle_command_reply: tid=0 2024-08-31T06:19:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.182+0000 7eff0b280740 1 client.28268 shutdown 2024-08-31T06:19:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.182+0000 7eff0b280740 1 -- 172.21.15.16:0/2065717102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccc2c78d80 msgr2=0x55ccc30698b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.183+0000 7eff0b280740 1 --2- 172.21.15.16:0/2065717102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccc2c78d80 0x55ccc30698b0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7efef40abdc0 tx=0x7efef40abbd0 comp rx=0 tx=0).stop 2024-08-31T06:19:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.183+0000 7eff0b280740 1 -- 172.21.15.16:0/2065717102 shutdown_connections 2024-08-31T06:19:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.183+0000 7eff0b280740 1 --2- 172.21.15.16:0/2065717102 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55ccc2c4e1c0 0x55ccc308c190 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.183+0000 7eff0b280740 1 --2- 172.21.15.16:0/2065717102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ccc2c994d0 0x55ccc3069370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.183+0000 7eff0b280740 1 --2- 172.21.15.16:0/2065717102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccc2c78d80 0x55ccc30698b0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.183+0000 7efee3400640 0 client.28268 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:19:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.183+0000 7eff0b280740 1 --2- 172.21.15.16:0/2065717102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc2c912f0 0x55ccc3068e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.183+0000 7eff0b280740 1 -- 172.21.15.16:0/2065717102 >> 172.21.15.16:0/2065717102 conn(0x55ccc2cae300 msgr2=0x55ccc305e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.185+0000 7eff0b280740 1 -- 172.21.15.16:0/2065717102 shutdown_connections 2024-08-31T06:19:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.185+0000 7eff0b280740 1 -- 172.21.15.16:0/2065717102 wait complete. 2024-08-31T06:19:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.185+0000 7eff0b280740 20 client.28268 trim_cache size 0 max 16384 2024-08-31T06:19:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.227+0000 7eff0b280740 1 Processor -- start 2024-08-31T06:19:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.228+0000 7eff0b280740 1 -- start start 2024-08-31T06:19:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.228+0000 7eff0b280740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc2c78d80 0x55ccc3067d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.228+0000 7eff0b280740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccc2c912f0 0x55ccc305b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.228+0000 7eff0b280740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ccc2c994d0 0x55ccc3066000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.228+0000 7eff0b280740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efedc011320 con 0x55ccc2c78d80 2024-08-31T06:19:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.228+0000 7eff0b280740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efedc00eb90 con 0x55ccc2c912f0 2024-08-31T06:19:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.228+0000 7eff0b280740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efedc00c980 con 0x55ccc2c994d0 2024-08-31T06:19:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.229+0000 7eff00e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccc2c912f0 0x55ccc305b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.229+0000 7eff02200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ccc2c994d0 0x55ccc3066000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.229+0000 7eff00e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccc2c912f0 0x55ccc305b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51100/0 (socket says 172.21.15.16:51100) 2024-08-31T06:19:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.229+0000 7eff00e00640 1 -- 172.21.15.16:0/1856386007 learned_addr learned my addr 172.21.15.16:0/1856386007 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.229+0000 7eff02200640 1 --2- 172.21.15.16:0/1856386007 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ccc2c994d0 0x55ccc3066000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:51822/0 (socket says 172.21.15.16:51822) 2024-08-31T06:19:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.229+0000 7eff00e00640 1 -- 172.21.15.16:0/1856386007 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ccc2c994d0 msgr2=0x55ccc3066000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:00.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.229+0000 7eff00e00640 1 --2- 172.21.15.16:0/1856386007 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ccc2c994d0 0x55ccc3066000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:00.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.229+0000 7eff00e00640 1 -- 172.21.15.16:0/1856386007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc2c78d80 msgr2=0x55ccc3067d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:00.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.229+0000 7eff00e00640 1 --2- 172.21.15.16:0/1856386007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc2c78d80 0x55ccc3067d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:00.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.229+0000 7eff00e00640 1 -- 172.21.15.16:0/1856386007 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccc3060730 con 0x55ccc2c912f0 2024-08-31T06:19:00.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.229+0000 7eff02200640 1 --2- 172.21.15.16:0/1856386007 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ccc2c994d0 0x55ccc3066000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:19:00.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.230+0000 7eff00e00640 1 --2- 172.21.15.16:0/1856386007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccc2c912f0 0x55ccc305b6f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7efee8009b40 tx=0x7efee80103d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba4c880203bb22dd server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:00.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.230+0000 7efeee000640 1 -- 172.21.15.16:0/1856386007 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee8003800 con 0x55ccc2c912f0 2024-08-31T06:19:00.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.231+0000 7efeee000640 1 -- 172.21.15.16:0/1856386007 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efee802c070 con 0x55ccc2c912f0 2024-08-31T06:19:00.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.231+0000 7efeee000640 1 -- 172.21.15.16:0/1856386007 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee8027690 con 0x55ccc2c912f0 2024-08-31T06:19:00.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.232+0000 7eff0b280740 1 -- 172.21.15.16:0/1856386007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccc2c912f0 msgr2=0x55ccc305b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:00.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.232+0000 7eff0b280740 1 --2- 172.21.15.16:0/1856386007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccc2c912f0 0x55ccc305b6f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7efee8009b40 tx=0x7efee80103d0 comp rx=0 tx=0).stop 2024-08-31T06:19:00.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.232+0000 7eff0b280740 1 -- 172.21.15.16:0/1856386007 shutdown_connections 2024-08-31T06:19:00.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.232+0000 7eff0b280740 1 --2- 172.21.15.16:0/1856386007 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ccc2c994d0 0x55ccc3066000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:00.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.232+0000 7eff0b280740 1 --2- 172.21.15.16:0/1856386007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccc2c912f0 0x55ccc305b6f0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:00.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.232+0000 7eff0b280740 1 --2- 172.21.15.16:0/1856386007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc2c78d80 0x55ccc3067d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:00.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.232+0000 7eff0b280740 1 -- 172.21.15.16:0/1856386007 >> 172.21.15.16:0/1856386007 conn(0x55ccc2cae300 msgr2=0x55ccc305afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:00.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.232+0000 7eff0b280740 1 -- 172.21.15.16:0/1856386007 shutdown_connections 2024-08-31T06:19:00.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.232+0000 7eff0b280740 1 -- 172.21.15.16:0/1856386007 wait complete. 2024-08-31T06:19:00.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.233+0000 7eff0b280740 1 -- start start 2024-08-31T06:19:00.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.233+0000 7eff0b280740 1 Processor -- start 2024-08-31T06:19:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.234+0000 7eff0b280740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ccc2c78d80 0x55ccc3071a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.234+0000 7eff0b280740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccc2c912f0 0x55ccc306eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.235+0000 7eff01800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ccc2c78d80 0x55ccc3071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.235+0000 7eff01800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ccc2c78d80 0x55ccc3071a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:51832/0 (socket says 172.21.15.16:51832) 2024-08-31T06:19:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.235+0000 7eff00e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccc2c912f0 0x55ccc306eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.235+0000 7eff0b280740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc2c994d0 0x55ccc306f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.236+0000 7eff00e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccc2c912f0 0x55ccc306eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51102/0 (socket says 172.21.15.16:51102) 2024-08-31T06:19:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.236+0000 7eff0b280740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efedc00d050 con 0x55ccc2c994d0 2024-08-31T06:19:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.236+0000 7eff0b280740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efedc00e4c0 con 0x55ccc2c912f0 2024-08-31T06:19:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.236+0000 7eff0b280740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efedc00d720 con 0x55ccc2c78d80 2024-08-31T06:19:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.236+0000 7eff01800640 1 -- 172.21.15.16:0/1392318775 learned_addr learned my addr 172.21.15.16:0/1392318775 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.236+0000 7eff0b280740 10 client.0 _ll_register_callbacks cb 0x7efef40aba10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:19:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.236+0000 7eff01800640 1 -- 172.21.15.16:0/1392318775 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccc2c912f0 msgr2=0x55ccc306eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.237+0000 7eff01800640 1 --2- 172.21.15.16:0/1392318775 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccc2c912f0 0x55ccc306eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.237+0000 7eff01800640 1 -- 172.21.15.16:0/1392318775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc2c994d0 msgr2=0x55ccc306f410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:19:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.237+0000 7eff02200640 1 --2- 172.21.15.16:0/1392318775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc2c994d0 0x55ccc306f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.237+0000 7eff01800640 1 --2- 172.21.15.16:0/1392318775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc2c994d0 0x55ccc306f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.238+0000 7eff01800640 1 -- 172.21.15.16:0/1392318775 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccc306f9a0 con 0x55ccc2c78d80 2024-08-31T06:19:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.239+0000 7eff01800640 1 --2- 172.21.15.16:0/1392318775 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ccc2c78d80 0x55ccc3071a10 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7efef0009780 tx=0x7efef0030be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c8f4889738baff8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.239+0000 7efee3400640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:19:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.239+0000 7efee3400640 1 -- 172.21.15.16:0/1392318775 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef0037040 con 0x55ccc2c78d80 2024-08-31T06:19:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.240+0000 7efee3400640 1 -- 172.21.15.16:0/1392318775 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8470+0+0 (secure 0 0 0) 0x7efef0004450 con 0x55ccc2c78d80 2024-08-31T06:19:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.240+0000 7eff0b280740 1 -- 172.21.15.16:0/1392318775 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efee802c070 con 0x55ccc2c78d80 2024-08-31T06:19:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.240+0000 7efee3400640 1 -- 172.21.15.16:0/1392318775 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7efef0030450 con 0x55ccc2c78d80 2024-08-31T06:19:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.240+0000 7eff0b280740 10 client.28285 fetch_fsmap learned FSMap version 47 2024-08-31T06:19:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.240+0000 7eff0b280740 1 -- 172.21.15.16:0/1392318775 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55ccc30f6980 con 0x55ccc2c78d80 2024-08-31T06:19:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.241+0000 7efee3400640 1 -- 172.21.15.16:0/1392318775 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efee802c070 con 0x55ccc2c78d80 2024-08-31T06:19:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.241+0000 7efee3400640 1 -- 172.21.15.16:0/1392318775 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 47) ==== 8354+0+0 (secure 0 0 0) 0x7efef00046c0 con 0x55ccc2c78d80 2024-08-31T06:19:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.241+0000 7eff0b280740 10 client.28285 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T06:19:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.241+0000 7eff0b280740 10 client.28285 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:19:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.241+0000 7eff0b280740 20 client.28285 populate_metadata read hostname 'smithi016' 2024-08-31T06:19:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.241+0000 7eff0b280740 1 --2- 172.21.15.16:0/1392318775 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55ccc30f7110 0x55ccc30f9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.241+0000 7eff0b280740 4 client.28285 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:19:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.241+0000 7eff0b280740 1 -- 172.21.15.16:0/1392318775 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ccc2fd0960 con 0x55ccc30f7110 2024-08-31T06:19:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.242+0000 7eff00e00640 1 --2- 172.21.15.16:0/1392318775 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55ccc30f7110 0x55ccc30f9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.242+0000 7efee3400640 10 client.28285 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:19:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.243+0000 7eff00e00640 1 --2- 172.21.15.16:0/1392318775 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55ccc30f7110 0x55ccc30f9580 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=788e18720f972adb server_cookie=35dbe36616eda807 in_seq=0 out_seq=0 2024-08-31T06:19:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.266+0000 7efee3400640 1 -- 172.21.15.16:0/1392318775 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ccc2fd0960 con 0x55ccc30f7110 2024-08-31T06:19:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.266+0000 7efee3400640 10 client.28285 handle_command_reply: tid=0 2024-08-31T06:19:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.266+0000 7eff0b280740 1 client.28285 shutdown 2024-08-31T06:19:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.267+0000 7eff0b280740 1 -- 172.21.15.16:0/1392318775 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ccc2c78d80 msgr2=0x55ccc3071a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.267+0000 7eff0b280740 1 --2- 172.21.15.16:0/1392318775 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ccc2c78d80 0x55ccc3071a10 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7efef0009780 tx=0x7efef0030be0 comp rx=0 tx=0).stop 2024-08-31T06:19:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.267+0000 7eff0b280740 1 -- 172.21.15.16:0/1392318775 shutdown_connections 2024-08-31T06:19:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.267+0000 7eff0b280740 1 --2- 172.21.15.16:0/1392318775 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55ccc30f7110 0x55ccc30f9580 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.267+0000 7eff0b280740 1 --2- 172.21.15.16:0/1392318775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc2c994d0 0x55ccc306f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.267+0000 7eff0b280740 1 --2- 172.21.15.16:0/1392318775 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccc2c912f0 0x55ccc306eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.267+0000 7eff0b280740 1 --2- 172.21.15.16:0/1392318775 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ccc2c78d80 0x55ccc3071a10 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.267+0000 7eff0b280740 1 -- 172.21.15.16:0/1392318775 >> 172.21.15.16:0/1392318775 conn(0x55ccc2cae300 msgr2=0x55ccc30743b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.267+0000 7efee3400640 0 client.28285 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:19:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.267+0000 7eff0b280740 1 -- 172.21.15.16:0/1392318775 shutdown_connections 2024-08-31T06:19:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.268+0000 7eff0b280740 1 -- 172.21.15.16:0/1392318775 wait complete. 2024-08-31T06:19:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.268+0000 7eff0b280740 20 client.28285 trim_cache size 0 max 16384 2024-08-31T06:19:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:19:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:19:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:19:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:19:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.270+0000 7eff04200640 1 -- 172.21.15.16:0/692235250 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7efee40b4ca0 msgr2=0x7efee40b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.270+0000 7eff04200640 1 --2- 172.21.15.16:0/692235250 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7efee40b4ca0 0x7efee40b71b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7efee8009530 tx=0x7efee8009260 comp rx=0 tx=0).stop 2024-08-31T06:19:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.270+0000 7eff04200640 1 -- 172.21.15.16:0/692235250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efefc10bd10 msgr2=0x7efefc110480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.270+0000 7eff04200640 1 --2- 172.21.15.16:0/692235250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efefc10bd10 0x7efefc110480 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7efef400eb10 tx=0x7efef400c2d0 comp rx=0 tx=0).stop 2024-08-31T06:19:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.270+0000 7eff04200640 1 -- 172.21.15.16:0/692235250 shutdown_connections 2024-08-31T06:19:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.270+0000 7eff04200640 1 --2- 172.21.15.16:0/692235250 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7efee40b4ca0 0x7efee40b71b0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.270+0000 7eff04200640 1 --2- 172.21.15.16:0/692235250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efefc10bd10 0x7efefc110480 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.270+0000 7eff04200640 1 --2- 172.21.15.16:0/692235250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efefc10b3c0 0x7efefc10ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.270+0000 7eff04200640 1 --2- 172.21.15.16:0/692235250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efefc105a90 0x7efefc10fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.270+0000 7eff04200640 1 -- 172.21.15.16:0/692235250 >> 172.21.15.16:0/692235250 conn(0x7efefc101490 msgr2=0x7efefc10aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.270+0000 7eff04200640 1 -- 172.21.15.16:0/692235250 shutdown_connections 2024-08-31T06:19:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:00.271+0000 7eff04200640 1 -- 172.21.15.16:0/692235250 wait complete. 2024-08-31T06:19:01.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:00 smithi155 ceph-mon[21742]: pgmap v729: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 13 MiB/s wr, 959 op/s 2024-08-31T06:19:01.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:00 smithi016 ceph-mon[17194]: pgmap v729: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 13 MiB/s wr, 959 op/s 2024-08-31T06:19:01.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:00 smithi184 ceph-mon[22331]: pgmap v729: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 13 MiB/s wr, 959 op/s 2024-08-31T06:19:01.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 695.0 s, 1462.0 tps, lat 67.713 ms stddev 72.023 2024-08-31T06:19:03.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:02 smithi155 ceph-mon[21742]: pgmap v730: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.0 MiB/s wr, 652 op/s 2024-08-31T06:19:03.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:02 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:19:03.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:02 smithi016 ceph-mon[17194]: pgmap v730: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.0 MiB/s wr, 652 op/s 2024-08-31T06:19:03.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:02 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:19:03.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:02 smithi184 ceph-mon[22331]: pgmap v730: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.0 MiB/s wr, 652 op/s 2024-08-31T06:19:03.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:02 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:19:04.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:03 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:19:04.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:03 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:19:04.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:03 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:19:04.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:03 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:19:04.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:03 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:19:04.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:03 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:19:04.166 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T06:19:04.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:03 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:19:04.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:03 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:19:04.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:03 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:19:04.895 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:19:05.076 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:04 smithi016 ceph-mon[17194]: pgmap v731: 97 pgs: 97 active+clean; 4.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.6 MiB/s wr, 690 op/s 2024-08-31T06:19:05.076 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:04 smithi016 ceph-mon[17194]: osdmap e94: 12 total, 12 up, 12 in 2024-08-31T06:19:05.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:04 smithi155 ceph-mon[21742]: pgmap v731: 97 pgs: 97 active+clean; 4.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.6 MiB/s wr, 690 op/s 2024-08-31T06:19:05.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:04 smithi155 ceph-mon[21742]: osdmap e94: 12 total, 12 up, 12 in 2024-08-31T06:19:05.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:04 smithi184 ceph-mon[22331]: pgmap v731: 97 pgs: 97 active+clean; 4.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.6 MiB/s wr, 690 op/s 2024-08-31T06:19:05.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:04 smithi184 ceph-mon[22331]: osdmap e94: 12 total, 12 up, 12 in 2024-08-31T06:19:05.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.805+0000 7f6517200640 1 -- 172.21.15.16:0/1272549777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6518102ca0 msgr2=0x7f65181071d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:05.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.807+0000 7f6517200640 1 --2- 172.21.15.16:0/1272549777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6518102ca0 0x7f65181071d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f6500009860 tx=0x7f650002a340 comp rx=0 tx=0).stop 2024-08-31T06:19:05.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.807+0000 7f6517200640 1 -- 172.21.15.16:0/1272549777 shutdown_connections 2024-08-31T06:19:05.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.807+0000 7f6517200640 1 --2- 172.21.15.16:0/1272549777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6518102ca0 0x7f65181071d0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:05.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.807+0000 7f6517200640 1 --2- 172.21.15.16:0/1272549777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6518107790 0x7f6518107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:05.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.807+0000 7f6517200640 1 --2- 172.21.15.16:0/1272549777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6518102350 0x7f6518102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:05.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.807+0000 7f6517200640 1 -- 172.21.15.16:0/1272549777 >> 172.21.15.16:0/1272549777 conn(0x7f65180fe060 msgr2=0x7f65181004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:05.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.807+0000 7f6517200640 1 -- 172.21.15.16:0/1272549777 shutdown_connections 2024-08-31T06:19:05.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.808+0000 7f6517200640 1 -- 172.21.15.16:0/1272549777 wait complete. 2024-08-31T06:19:05.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.809+0000 7f6517200640 1 Processor -- start 2024-08-31T06:19:05.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.809+0000 7f6517200640 1 -- start start 2024-08-31T06:19:05.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.810+0000 7f6517200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6518102350 0x7f65181a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:05.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.810+0000 7f6517200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6518102ca0 0x7f65181a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:05.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.810+0000 7f6517200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6518107790 0x7f65181a5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:05.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.810+0000 7f6517200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6518119c30 con 0x7f6518102350 2024-08-31T06:19:05.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.810+0000 7f6517200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6518119ab0 con 0x7f6518107790 2024-08-31T06:19:05.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.810+0000 7f6517200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6518119db0 con 0x7f6518102ca0 2024-08-31T06:19:05.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.810+0000 7f6516800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6518107790 0x7f65181a5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:05.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.811+0000 7f6516800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6518107790 0x7f65181a5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51160/0 (socket says 172.21.15.16:51160) 2024-08-31T06:19:05.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.811+0000 7f6516800640 1 -- 172.21.15.16:0/1081681861 learned_addr learned my addr 172.21.15.16:0/1081681861 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:05.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.811+0000 7f650fe00640 1 --2- 172.21.15.16:0/1081681861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6518102ca0 0x7f65181a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:05.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.811+0000 7f650fe00640 1 -- 172.21.15.16:0/1081681861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6518107790 msgr2=0x7f65181a5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:05.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.811+0000 7f650fe00640 1 --2- 172.21.15.16:0/1081681861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6518107790 0x7f65181a5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:05.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.811+0000 7f650fe00640 1 -- 172.21.15.16:0/1081681861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6518102350 msgr2=0x7f65181a0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:05.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.811+0000 7f650fe00640 1 --2- 172.21.15.16:0/1081681861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6518102350 0x7f65181a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:05.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.811+0000 7f650fe00640 1 -- 172.21.15.16:0/1081681861 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65181a6690 con 0x7f6518102ca0 2024-08-31T06:19:05.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.812+0000 7f650fe00640 1 --2- 172.21.15.16:0/1081681861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6518102ca0 0x7f65181a0cc0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6500009780 tx=0x7f6500032dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:05.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.813+0000 7f650ea00640 1 -- 172.21.15.16:0/1081681861 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6500037040 con 0x7f6518102ca0 2024-08-31T06:19:05.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.813+0000 7f650ea00640 1 -- 172.21.15.16:0/1081681861 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6500030c80 con 0x7f6518102ca0 2024-08-31T06:19:05.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.813+0000 7f650ea00640 1 -- 172.21.15.16:0/1081681861 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6500003850 con 0x7f6518102ca0 2024-08-31T06:19:05.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.813+0000 7f6517200640 1 -- 172.21.15.16:0/1081681861 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65181a6920 con 0x7f6518102ca0 2024-08-31T06:19:05.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.813+0000 7f6517200640 1 -- 172.21.15.16:0/1081681861 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65181a6d20 con 0x7f6518102ca0 2024-08-31T06:19:05.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.820+0000 7f652365f740 1 Processor -- start 2024-08-31T06:19:05.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.821+0000 7f652365f740 1 -- start start 2024-08-31T06:19:05.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.821+0000 7f652365f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55772dce8070 0x55772e08da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:05.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.821+0000 7f652365f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55772dcbf1b0 0x55772e08df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:05.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.822+0000 7f652365f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55772dcc7390 0x55772e08e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:05.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.822+0000 7f652365f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55772e004770 con 0x55772dcc7390 2024-08-31T06:19:05.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.822+0000 7f652365f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55772dff3d00 con 0x55772dce8070 2024-08-31T06:19:05.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.822+0000 7f652365f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55772dff7cb0 con 0x55772dcbf1b0 2024-08-31T06:19:05.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.822+0000 7f650ea00640 1 -- 172.21.15.16:0/1081681861 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f6500043020 con 0x7f6518102ca0 2024-08-31T06:19:05.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.822+0000 7f6516800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55772dce8070 0x55772e08da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:05.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.822+0000 7f650fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55772dcc7390 0x55772e08e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:05.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.822+0000 7f650fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55772dcc7390 0x55772e08e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50308/0 (socket says 172.21.15.16:50308) 2024-08-31T06:19:05.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.822+0000 7f650fe00640 1 -- 172.21.15.16:0/2554345278 learned_addr learned my addr 172.21.15.16:0/2554345278 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:05.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.823+0000 7f6515e00640 1 --2- 172.21.15.16:0/2554345278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55772dcbf1b0 0x55772e08df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:05.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.823+0000 7f650fe00640 1 -- 172.21.15.16:0/2554345278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55772dcbf1b0 msgr2=0x55772e08df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:05.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.823+0000 7f650fe00640 1 --2- 172.21.15.16:0/2554345278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55772dcbf1b0 0x55772e08df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:05.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.824+0000 7f650fe00640 1 -- 172.21.15.16:0/2554345278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55772dce8070 msgr2=0x55772e08da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:05.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.824+0000 7f650fe00640 1 --2- 172.21.15.16:0/2554345278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55772dce8070 0x55772e08da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:05.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.824+0000 7f650fe00640 1 -- 172.21.15.16:0/2554345278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55772dd53650 con 0x55772dcc7390 2024-08-31T06:19:05.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.825+0000 7f6516800640 1 --2- 172.21.15.16:0/2554345278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55772dce8070 0x55772e08da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:19:05.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.825+0000 7f650fe00640 1 --2- 172.21.15.16:0/2554345278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55772dcc7390 0x55772e08e4b0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f6500101c80 tx=0x7f650010a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=723e7802fd79f74c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:05.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.825+0000 7f650ea00640 1 --2- 172.21.15.16:0/1081681861 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f64fc0b4da0 0x7f64fc0b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:05.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.825+0000 7f6516800640 1 --2- 172.21.15.16:0/1081681861 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f64fc0b4da0 0x7f64fc0b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:05.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.825+0000 7f650ea00640 1 -- 172.21.15.16:0/1081681861 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8514+0+0 (secure 0 0 0) 0x7f6500104620 con 0x7f6518102ca0 2024-08-31T06:19:05.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.826+0000 7f6507000640 1 -- 172.21.15.16:0/2554345278 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65000c6b70 con 0x55772dcc7390 2024-08-31T06:19:05.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.826+0000 7f6507000640 1 -- 172.21.15.16:0/2554345278 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6500111030 con 0x55772dcc7390 2024-08-31T06:19:05.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.827+0000 7f6516800640 1 --2- 172.21.15.16:0/1081681861 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f64fc0b4da0 0x7f64fc0b72b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f65100049d0 tx=0x7f6510009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:05.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.827+0000 7f6507000640 1 -- 172.21.15.16:0/2554345278 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65000ca420 con 0x55772dcc7390 2024-08-31T06:19:05.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.828+0000 7f652365f740 1 -- 172.21.15.16:0/2554345278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55772dcc7390 msgr2=0x55772e08e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:05.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.828+0000 7f652365f740 1 --2- 172.21.15.16:0/2554345278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55772dcc7390 0x55772e08e4b0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f6500101c80 tx=0x7f650010a620 comp rx=0 tx=0).stop 2024-08-31T06:19:05.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.828+0000 7f652365f740 1 -- 172.21.15.16:0/2554345278 shutdown_connections 2024-08-31T06:19:05.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.828+0000 7f652365f740 1 --2- 172.21.15.16:0/2554345278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55772dcbf1b0 0x55772e08df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:05.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.828+0000 7f652365f740 1 --2- 172.21.15.16:0/2554345278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55772dcc7390 0x55772e08e4b0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:05.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.828+0000 7f652365f740 1 --2- 172.21.15.16:0/2554345278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55772dce8070 0x55772e08da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:05.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.828+0000 7f652365f740 1 -- 172.21.15.16:0/2554345278 >> 172.21.15.16:0/2554345278 conn(0x55772dcdc460 msgr2=0x55772e08b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:05.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.828+0000 7f652365f740 1 -- 172.21.15.16:0/2554345278 shutdown_connections 2024-08-31T06:19:05.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.828+0000 7f652365f740 1 -- 172.21.15.16:0/2554345278 wait complete. 2024-08-31T06:19:05.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.829+0000 7f652365f740 1 -- start start 2024-08-31T06:19:05.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.829+0000 7f652365f740 1 Processor -- start 2024-08-31T06:19:05.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.830+0000 7f652365f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55772dcbf1b0 0x55772e0995c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:05.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.830+0000 7f652365f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55772dcc7390 0x55772e099b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:05.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.830+0000 7f652365f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55772dce8070 0x55772e0a5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:05.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.830+0000 7f652365f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55772de56700 con 0x55772dcbf1b0 2024-08-31T06:19:05.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.830+0000 7f652365f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55772df7acb0 con 0x55772dce8070 2024-08-31T06:19:05.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.830+0000 7f652365f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55772e09bfd0 con 0x55772dcc7390 2024-08-31T06:19:05.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.830+0000 7f652365f740 10 client.0 _ll_register_callbacks cb 0x55772dfa0430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:19:05.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.831+0000 7f6515e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55772dcbf1b0 0x55772e0995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:05.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.831+0000 7f6515e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55772dcbf1b0 0x55772e0995c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50318/0 (socket says 172.21.15.16:50318) 2024-08-31T06:19:05.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.831+0000 7f6515e00640 1 -- 172.21.15.16:0/578405556 learned_addr learned my addr 172.21.15.16:0/578405556 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:05.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.831+0000 7f650fe00640 1 --2- 172.21.15.16:0/578405556 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55772dcc7390 0x55772e099b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:05.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.831+0000 7f6516800640 1 --2- 172.21.15.16:0/578405556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55772dce8070 0x55772e0a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:05.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.832+0000 7f6516800640 1 -- 172.21.15.16:0/578405556 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55772dcc7390 msgr2=0x55772e099b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:05.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.832+0000 7f6516800640 1 --2- 172.21.15.16:0/578405556 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55772dcc7390 0x55772e099b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:05.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.832+0000 7f6516800640 1 -- 172.21.15.16:0/578405556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55772dcbf1b0 msgr2=0x55772e0995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:05.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.832+0000 7f6516800640 1 --2- 172.21.15.16:0/578405556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55772dcbf1b0 0x55772e0995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:05.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.832+0000 7f6516800640 1 -- 172.21.15.16:0/578405556 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55772dfbb870 con 0x55772dce8070 2024-08-31T06:19:05.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.832+0000 7f6516800640 1 --2- 172.21.15.16:0/578405556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55772dce8070 0x55772e0a5280 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f6510011c40 tx=0x7f6510004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e88bdb66c9bed49 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:05.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.832+0000 7f64fb400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:19:05.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.833+0000 7f64fb400640 1 -- 172.21.15.16:0/578405556 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651000fb60 con 0x55772dce8070 2024-08-31T06:19:05.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.833+0000 7f64fb400640 1 -- 172.21.15.16:0/578405556 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8514+0+0 (secure 0 0 0) 0x7f651000fd00 con 0x55772dce8070 2024-08-31T06:19:05.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.833+0000 7f652365f740 1 -- 172.21.15.16:0/578405556 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6500111030 con 0x55772dce8070 2024-08-31T06:19:05.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.833+0000 7f64fb400640 1 -- 172.21.15.16:0/578405556 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f6510029470 con 0x55772dce8070 2024-08-31T06:19:05.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.834+0000 7f652365f740 10 client.28286 fetch_fsmap learned FSMap version 47 2024-08-31T06:19:05.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.834+0000 7f652365f740 1 -- 172.21.15.16:0/578405556 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55772e0a7fd0 con 0x55772dce8070 2024-08-31T06:19:05.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.834+0000 7f64fb400640 1 -- 172.21.15.16:0/578405556 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6500111030 con 0x55772dce8070 2024-08-31T06:19:05.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.835+0000 7f64fb400640 1 -- 172.21.15.16:0/578405556 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 47) ==== 8354+0+0 (secure 0 0 0) 0x7f65100324c0 con 0x55772dce8070 2024-08-31T06:19:05.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.836+0000 7f652365f740 10 client.28286 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T06:19:05.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.836+0000 7f652365f740 10 client.28286 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:19:05.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.836+0000 7f652365f740 20 client.28286 populate_metadata read hostname 'smithi016' 2024-08-31T06:19:05.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.836+0000 7f652365f740 1 --2- 172.21.15.16:0/578405556 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55772dca69f0 0x55772e0ab490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:05.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.836+0000 7f652365f740 4 client.28286 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:19:05.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.836+0000 7f652365f740 1 -- 172.21.15.16:0/578405556 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55772e0066c0 con 0x55772dca69f0 2024-08-31T06:19:05.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.836+0000 7f6515e00640 1 --2- 172.21.15.16:0/578405556 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55772dca69f0 0x55772e0ab490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:05.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.837+0000 7f6515e00640 1 --2- 172.21.15.16:0/578405556 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55772dca69f0 0x55772e0ab490 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcd13e93a785a83f server_cookie=7b8029f2c885402b in_seq=0 out_seq=0 2024-08-31T06:19:05.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.837+0000 7f64fb400640 10 client.28286 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:19:05.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.838+0000 7f64fb400640 1 -- 172.21.15.16:0/578405556 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55772e0066c0 con 0x55772dca69f0 2024-08-31T06:19:05.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.838+0000 7f64fb400640 10 client.28286 handle_command_reply: tid=0 2024-08-31T06:19:05.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.839+0000 7f652365f740 1 client.28286 shutdown 2024-08-31T06:19:05.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.839+0000 7f652365f740 1 -- 172.21.15.16:0/578405556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55772dce8070 msgr2=0x55772e0a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:05.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.839+0000 7f652365f740 1 --2- 172.21.15.16:0/578405556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55772dce8070 0x55772e0a5280 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f6510011c40 tx=0x7f6510004ab0 comp rx=0 tx=0).stop 2024-08-31T06:19:05.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.840+0000 7f652365f740 1 -- 172.21.15.16:0/578405556 shutdown_connections 2024-08-31T06:19:05.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.840+0000 7f652365f740 1 --2- 172.21.15.16:0/578405556 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55772dca69f0 0x55772e0ab490 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:05.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.840+0000 7f652365f740 1 --2- 172.21.15.16:0/578405556 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55772dcc7390 0x55772e099b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:05.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.840+0000 7f652365f740 1 --2- 172.21.15.16:0/578405556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55772dce8070 0x55772e0a5280 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:05.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.840+0000 7f652365f740 1 --2- 172.21.15.16:0/578405556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55772dcbf1b0 0x55772e0995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:05.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.840+0000 7f64fb400640 0 client.28286 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:19:05.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.840+0000 7f652365f740 1 -- 172.21.15.16:0/578405556 >> 172.21.15.16:0/578405556 conn(0x55772dcdc460 msgr2=0x55772e090c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:05.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.840+0000 7f652365f740 1 -- 172.21.15.16:0/578405556 shutdown_connections 2024-08-31T06:19:05.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.840+0000 7f652365f740 1 -- 172.21.15.16:0/578405556 wait complete. 2024-08-31T06:19:05.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.841+0000 7f652365f740 20 client.28286 trim_cache size 0 max 16384 2024-08-31T06:19:05.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.879+0000 7f652365f740 1 Processor -- start 2024-08-31T06:19:05.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.886+0000 7f652365f740 1 -- start start 2024-08-31T06:19:05.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.886+0000 7f652365f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55772dcbf1b0 0x55772e0a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:05.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.886+0000 7f652365f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55772dcc7390 0x55772e092b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:05.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.886+0000 7f652365f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55772dce8070 0x55772e093080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:05.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.886+0000 7f652365f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f65181b8310 con 0x55772dce8070 2024-08-31T06:19:05.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.886+0000 7f652365f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f65181b5b80 con 0x55772dcbf1b0 2024-08-31T06:19:05.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.886+0000 7f652365f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f65181b39f0 con 0x55772dcc7390 2024-08-31T06:19:05.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.887+0000 7f650fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55772dcc7390 0x55772e092b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:05.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.887+0000 7f650fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55772dcc7390 0x55772e092b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:58580/0 (socket says 172.21.15.16:58580) 2024-08-31T06:19:05.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.887+0000 7f650fe00640 1 -- 172.21.15.16:0/2649911860 learned_addr learned my addr 172.21.15.16:0/2649911860 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:05.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.887+0000 7f6516800640 1 --2- 172.21.15.16:0/2649911860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55772dce8070 0x55772e093080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:05.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.888+0000 7f650fe00640 1 -- 172.21.15.16:0/2649911860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55772dcbf1b0 msgr2=0x55772e0a53d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:19:05.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.888+0000 7f650fe00640 1 --2- 172.21.15.16:0/2649911860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55772dcbf1b0 0x55772e0a53d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:05.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.888+0000 7f650fe00640 1 -- 172.21.15.16:0/2649911860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55772dce8070 msgr2=0x55772e093080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:05.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.888+0000 7f650fe00640 1 --2- 172.21.15.16:0/2649911860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55772dce8070 0x55772e093080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:05.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.888+0000 7f650fe00640 1 -- 172.21.15.16:0/2649911860 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55772e094c50 con 0x55772dcc7390 2024-08-31T06:19:05.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.888+0000 7f6516800640 1 --2- 172.21.15.16:0/2649911860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55772dce8070 0x55772e093080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:19:05.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.892+0000 7f650fe00640 1 --2- 172.21.15.16:0/2649911860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55772dcc7390 0x55772e092b40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f65000ca9b0 tx=0x7f65000303b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7235f1581e1988d5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:05.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.892+0000 7f6506600640 1 -- 172.21.15.16:0/2649911860 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65000ca600 con 0x55772dcc7390 2024-08-31T06:19:05.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.892+0000 7f6506600640 1 -- 172.21.15.16:0/2649911860 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6500111070 con 0x55772dcc7390 2024-08-31T06:19:05.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.892+0000 7f6506600640 1 -- 172.21.15.16:0/2649911860 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65000caa80 con 0x55772dcc7390 2024-08-31T06:19:05.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.893+0000 7f652365f740 1 -- 172.21.15.16:0/2649911860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55772dcc7390 msgr2=0x55772e092b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:05.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.893+0000 7f652365f740 1 --2- 172.21.15.16:0/2649911860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55772dcc7390 0x55772e092b40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f65000ca9b0 tx=0x7f65000303b0 comp rx=0 tx=0).stop 2024-08-31T06:19:05.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.894+0000 7f652365f740 1 -- 172.21.15.16:0/2649911860 shutdown_connections 2024-08-31T06:19:05.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.894+0000 7f652365f740 1 --2- 172.21.15.16:0/2649911860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55772dcc7390 0x55772e092b40 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:05.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.894+0000 7f652365f740 1 --2- 172.21.15.16:0/2649911860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55772dce8070 0x55772e093080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:05.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.894+0000 7f652365f740 1 --2- 172.21.15.16:0/2649911860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55772dcbf1b0 0x55772e0a53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:05.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.894+0000 7f652365f740 1 -- 172.21.15.16:0/2649911860 >> 172.21.15.16:0/2649911860 conn(0x55772dcdc460 msgr2=0x55772e097c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:05.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.894+0000 7f652365f740 1 -- 172.21.15.16:0/2649911860 shutdown_connections 2024-08-31T06:19:05.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.895+0000 7f652365f740 1 -- 172.21.15.16:0/2649911860 wait complete. 2024-08-31T06:19:05.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.896+0000 7f652365f740 1 -- start start 2024-08-31T06:19:05.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.896+0000 7f652365f740 1 Processor -- start 2024-08-31T06:19:05.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.897+0000 7f652365f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55772dcbf1b0 0x55772e0a1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:05.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.897+0000 7f652365f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55772dcc7390 0x55772e0a1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:05.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.897+0000 7f652365f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55772dce8070 0x55772e09c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:05.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.898+0000 7f6515e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55772dcbf1b0 0x55772e0a1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:05.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.898+0000 7f650fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55772dcc7390 0x55772e0a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:05.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.898+0000 7f650fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55772dcc7390 0x55772e0a1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51196/0 (socket says 172.21.15.16:51196) 2024-08-31T06:19:05.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.898+0000 7f650fe00640 1 -- 172.21.15.16:0/542302195 learned_addr learned my addr 172.21.15.16:0/542302195 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:05.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.899+0000 7f652365f740 1 -- 172.21.15.16:0/542302195 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f65181b4040 con 0x55772dce8070 2024-08-31T06:19:05.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.899+0000 7f652365f740 1 -- 172.21.15.16:0/542302195 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f65181b54b0 con 0x55772dcc7390 2024-08-31T06:19:05.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.899+0000 7f652365f740 1 -- 172.21.15.16:0/542302195 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f65181b4710 con 0x55772dcbf1b0 2024-08-31T06:19:05.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.899+0000 7f652365f740 10 client.0 _ll_register_callbacks cb 0x7f6510011a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:19:05.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.899+0000 7f650fe00640 1 -- 172.21.15.16:0/542302195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55772dcbf1b0 msgr2=0x55772e0a1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:05.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.899+0000 7f650fe00640 1 --2- 172.21.15.16:0/542302195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55772dcbf1b0 0x55772e0a1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:05.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.900+0000 7f650fe00640 1 -- 172.21.15.16:0/542302195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55772dce8070 msgr2=0x55772e09c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:05.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.900+0000 7f650fe00640 1 --2- 172.21.15.16:0/542302195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55772dce8070 0x55772e09c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:05.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.900+0000 7f650fe00640 1 -- 172.21.15.16:0/542302195 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55772e09c850 con 0x55772dcc7390 2024-08-31T06:19:05.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.900+0000 7f6515e00640 1 --2- 172.21.15.16:0/542302195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55772dcbf1b0 0x55772e0a1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:19:05.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.900+0000 7f650fe00640 1 --2- 172.21.15.16:0/542302195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55772dcc7390 0x55772e0a1970 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f650002a860 tx=0x7f6500005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=917ec1dbebd9fe61 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:05.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.901+0000 7f64fb400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:19:05.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.902+0000 7f64fb400640 1 -- 172.21.15.16:0/542302195 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65000caa80 con 0x55772dcc7390 2024-08-31T06:19:05.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.902+0000 7f64fb400640 1 -- 172.21.15.16:0/542302195 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8514+0+0 (secure 0 0 0) 0x7f65000c72a0 con 0x55772dcc7390 2024-08-31T06:19:05.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.902+0000 7f652365f740 1 -- 172.21.15.16:0/542302195 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6500111070 con 0x55772dcc7390 2024-08-31T06:19:05.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.902+0000 7f64fb400640 1 -- 172.21.15.16:0/542302195 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f65000ca600 con 0x55772dcc7390 2024-08-31T06:19:05.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.903+0000 7f652365f740 10 client.28289 fetch_fsmap learned FSMap version 47 2024-08-31T06:19:05.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.903+0000 7f652365f740 1 -- 172.21.15.16:0/542302195 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55772e116760 con 0x55772dcc7390 2024-08-31T06:19:05.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.903+0000 7f64fb400640 1 -- 172.21.15.16:0/542302195 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6500111070 con 0x55772dcc7390 2024-08-31T06:19:05.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.903+0000 7f64fb400640 1 -- 172.21.15.16:0/542302195 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 47) ==== 8354+0+0 (secure 0 0 0) 0x7f65000c6f40 con 0x55772dcc7390 2024-08-31T06:19:05.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.906+0000 7f652365f740 10 client.28289 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T06:19:05.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.906+0000 7f652365f740 10 client.28289 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:19:05.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.906+0000 7f652365f740 20 client.28289 populate_metadata read hostname 'smithi016' 2024-08-31T06:19:05.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.906+0000 7f652365f740 1 --2- 172.21.15.16:0/542302195 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55772dc7bda0 0x55772e118990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:05.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.907+0000 7f652365f740 4 client.28289 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T06:19:05.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.907+0000 7f652365f740 1 -- 172.21.15.16:0/542302195 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55772e002160 con 0x55772dc7bda0 2024-08-31T06:19:05.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.907+0000 7f6515e00640 1 --2- 172.21.15.16:0/542302195 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55772dc7bda0 0x55772e118990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:05.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.912+0000 7f6515e00640 1 --2- 172.21.15.16:0/542302195 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55772dc7bda0 0x55772e118990 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e146041df265c0e server_cookie=37a45a922fdbe239 in_seq=0 out_seq=0 2024-08-31T06:19:05.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.912+0000 7f64fb400640 10 client.28289 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:19:05.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.914+0000 7f64fb400640 1 -- 172.21.15.16:0/542302195 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4350 (crc 0 0 0) 0x55772e002160 con 0x55772dc7bda0 2024-08-31T06:19:05.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.915+0000 7f64fb400640 10 client.28289 handle_command_reply: tid=0 2024-08-31T06:19:05.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.915+0000 7f652365f740 1 client.28289 shutdown 2024-08-31T06:19:05.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.915+0000 7f652365f740 1 -- 172.21.15.16:0/542302195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55772dcc7390 msgr2=0x55772e0a1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:05.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.915+0000 7f652365f740 1 --2- 172.21.15.16:0/542302195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55772dcc7390 0x55772e0a1970 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f650002a860 tx=0x7f6500005f90 comp rx=0 tx=0).stop 2024-08-31T06:19:05.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.916+0000 7f652365f740 1 -- 172.21.15.16:0/542302195 shutdown_connections 2024-08-31T06:19:05.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.916+0000 7f652365f740 1 --2- 172.21.15.16:0/542302195 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55772dc7bda0 0x55772e118990 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:05.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.916+0000 7f652365f740 1 --2- 172.21.15.16:0/542302195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55772dce8070 0x55772e09c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:05.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.916+0000 7f652365f740 1 --2- 172.21.15.16:0/542302195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55772dcc7390 0x55772e0a1970 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:05.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.916+0000 7f652365f740 1 --2- 172.21.15.16:0/542302195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55772dcbf1b0 0x55772e0a1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:05.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.916+0000 7f652365f740 1 -- 172.21.15.16:0/542302195 >> 172.21.15.16:0/542302195 conn(0x55772dcdc460 msgr2=0x55772e08b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:05.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.916+0000 7f64fb400640 0 client.28289 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:19:05.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.916+0000 7f652365f740 1 -- 172.21.15.16:0/542302195 shutdown_connections 2024-08-31T06:19:05.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.916+0000 7f652365f740 1 -- 172.21.15.16:0/542302195 wait complete. 2024-08-31T06:19:05.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.917+0000 7f652365f740 20 client.28289 trim_cache size 0 max 16384 2024-08-31T06:19:05.922 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:19:05.922 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-08-31T06:19:05.922 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {}, 2024-08-31T06:19:05.922 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-08-31T06:19:05.922 INFO:teuthology.orchestra.run.smithi016.stdout: "0e44eb18-439c-46e4-a298-7d4d2ecd0b4a": { 2024-08-31T06:19:05.922 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:18:31.130765+0000", 2024-08-31T06:19:05.922 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:05.922 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:05.922 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:05.922 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:05.922 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:05.922 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:05.922 INFO:teuthology.orchestra.run.smithi016.stdout: "e497cd88-1e7c-4631-a4a8-cdd8e1fa6c83": { 2024-08-31T06:19:05.923 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:17:42.476795+0000", 2024-08-31T06:19:05.923 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:05.923 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:05.923 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:05.923 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:05.923 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:05.923 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:05.923 INFO:teuthology.orchestra.run.smithi016.stdout: "c300b7e8-f3fd-482c-a909-9329bc61e570": { 2024-08-31T06:19:05.923 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:06:49.868046+0000", 2024-08-31T06:19:05.923 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:05.923 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:05.923 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:05.923 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:05.923 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:05.923 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:05.924 INFO:teuthology.orchestra.run.smithi016.stdout: "6a03046e-cdf4-4a14-b9bd-b560b8968467": { 2024-08-31T06:19:05.924 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:12:38.002354+0000", 2024-08-31T06:19:05.924 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:05.924 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:05.924 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:05.924 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:05.924 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:05.924 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:05.924 INFO:teuthology.orchestra.run.smithi016.stdout: "004b47bb-0fe8-4d28-9d9d-78bde6a61199": { 2024-08-31T06:19:05.924 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:16:48.915926+0000", 2024-08-31T06:19:05.924 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:05.924 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:05.924 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:05.924 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:05.924 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:05.925 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:05.925 INFO:teuthology.orchestra.run.smithi016.stdout: "88d7bdb8-cb1c-45e5-ad29-4cd8da6c6003": { 2024-08-31T06:19:05.925 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:07:39.648942+0000", 2024-08-31T06:19:05.925 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:05.925 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:05.925 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:05.925 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:05.925 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:05.925 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:05.925 INFO:teuthology.orchestra.run.smithi016.stdout: "0fdcd762-533d-42be-9289-b71c8e3b81b1": { 2024-08-31T06:19:05.925 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:15:06.169112+0000", 2024-08-31T06:19:05.925 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:05.925 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:05.925 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:05.925 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:05.926 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:05.926 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:05.926 INFO:teuthology.orchestra.run.smithi016.stdout: "c8455d74-2fe7-4d93-bcf2-19b419d16016": { 2024-08-31T06:19:05.926 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:08:24.035960+0000", 2024-08-31T06:19:05.926 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:05.926 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:05.926 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:05.926 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:05.926 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:05.926 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:05.926 INFO:teuthology.orchestra.run.smithi016.stdout: "78d6ef02-8551-45e8-ad60-9627eb573fc1": { 2024-08-31T06:19:05.926 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:09:13.547865+0000", 2024-08-31T06:19:05.926 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:05.926 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:05.926 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:05.927 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:05.927 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:05.927 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:05.927 INFO:teuthology.orchestra.run.smithi016.stdout: "c1fd6904-2e02-47fa-939a-47349b9fe36b": { 2024-08-31T06:19:05.927 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:06.678691+0000", 2024-08-31T06:19:05.927 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:05.927 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:05.927 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:05.927 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:05.927 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:05.927 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:05.927 INFO:teuthology.orchestra.run.smithi016.stdout: "a10cce76-99ab-46b5-b3b2-07b82899b8f9": { 2024-08-31T06:19:05.927 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:11:43.967546+0000", 2024-08-31T06:19:05.927 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:05.928 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:05.928 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:05.928 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:05.928 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:05.928 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:05.928 INFO:teuthology.orchestra.run.smithi016.stdout: "91b72a34-4ba9-4126-a414-8b862f8c5f78": { 2024-08-31T06:19:05.928 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:13:28.663237+0000", 2024-08-31T06:19:05.928 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:05.928 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:05.928 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:05.928 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:05.928 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:05.928 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:05.928 INFO:teuthology.orchestra.run.smithi016.stdout: "e739a16e-8ace-43a8-a1ce-ac27d39ef8a3": { 2024-08-31T06:19:05.928 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:54.084646+0000", 2024-08-31T06:19:05.929 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:05.929 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:05.929 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:05.929 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:05.929 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:05.929 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:05.929 INFO:teuthology.orchestra.run.smithi016.stdout: "a64d5301-8558-4e9c-ac9b-8d0ac7057e74": { 2024-08-31T06:19:05.929 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:14:20.338104+0000", 2024-08-31T06:19:05.929 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:05.929 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:05.929 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:05.929 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:05.929 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:05.929 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:05.930 INFO:teuthology.orchestra.run.smithi016.stdout: "f9c4a22e-0c8a-4928-9b0d-e2ce1db5a86b": { 2024-08-31T06:19:05.930 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:15:57.496015+0000", 2024-08-31T06:19:05.930 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:05.930 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:05.930 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:05.930 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:05.930 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:05.930 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:19:05.930 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:19:05.930 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:19:05.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.918+0000 7f6517200640 1 -- 172.21.15.16:0/1081681861 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f64fc0b4da0 msgr2=0x7f64fc0b72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:05.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.918+0000 7f6517200640 1 --2- 172.21.15.16:0/1081681861 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f64fc0b4da0 0x7f64fc0b72b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f65100049d0 tx=0x7f6510009500 comp rx=0 tx=0).stop 2024-08-31T06:19:05.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.918+0000 7f6517200640 1 -- 172.21.15.16:0/1081681861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6518102ca0 msgr2=0x7f65181a0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:05.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.918+0000 7f6517200640 1 --2- 172.21.15.16:0/1081681861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6518102ca0 0x7f65181a0cc0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6500009780 tx=0x7f6500032dc0 comp rx=0 tx=0).stop 2024-08-31T06:19:05.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.919+0000 7f6517200640 1 -- 172.21.15.16:0/1081681861 shutdown_connections 2024-08-31T06:19:05.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.919+0000 7f6517200640 1 --2- 172.21.15.16:0/1081681861 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f64fc0b4da0 0x7f64fc0b72b0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:05.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.919+0000 7f6517200640 1 --2- 172.21.15.16:0/1081681861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6518102ca0 0x7f65181a0cc0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:05.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.919+0000 7f6517200640 1 --2- 172.21.15.16:0/1081681861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6518107790 0x7f65181a5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:05.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.919+0000 7f6517200640 1 --2- 172.21.15.16:0/1081681861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6518102350 0x7f65181a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:05.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.919+0000 7f6517200640 1 -- 172.21.15.16:0/1081681861 >> 172.21.15.16:0/1081681861 conn(0x7f65180fe060 msgr2=0x7f65181051a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:05.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.919+0000 7f6517200640 1 -- 172.21.15.16:0/1081681861 shutdown_connections 2024-08-31T06:19:05.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:05.919+0000 7f6517200640 1 -- 172.21.15.16:0/1081681861 wait complete. 2024-08-31T06:19:06.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 700.0 s, 922.8 tps, lat 105.541 ms stddev 164.036 2024-08-31T06:19:07.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:06 smithi155 ceph-mon[21742]: pgmap v733: 97 pgs: 10 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 67 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 16 MiB/s wr, 1.43k op/s 2024-08-31T06:19:07.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:06 smithi016 ceph-mon[17194]: pgmap v733: 97 pgs: 10 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 67 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 16 MiB/s wr, 1.43k op/s 2024-08-31T06:19:07.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:06 smithi184 ceph-mon[22331]: pgmap v733: 97 pgs: 10 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 67 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 16 MiB/s wr, 1.43k op/s 2024-08-31T06:19:07.604 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T06:19:07.605 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:19:08.194 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:19:09.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.044+0000 7fd2ef200640 1 -- 172.21.15.16:0/3208665343 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2e8102380 msgr2=0x7fd2e8102790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:09.048 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:08 smithi016 ceph-mon[17194]: pgmap v734: 97 pgs: 10 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 67 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 19 MiB/s wr, 1.66k op/s 2024-08-31T06:19:09.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.044+0000 7fd2ef200640 1 --2- 172.21.15.16:0/3208665343 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2e8102380 0x7fd2e8102790 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8009900 tx=0x7fd2d802a410 comp rx=0 tx=0).stop 2024-08-31T06:19:09.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.045+0000 7fd2ef200640 1 -- 172.21.15.16:0/3208665343 shutdown_connections 2024-08-31T06:19:09.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.045+0000 7fd2ef200640 1 --2- 172.21.15.16:0/3208665343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2e81077c0 0x7fd2e8107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:09.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.045+0000 7fd2ef200640 1 --2- 172.21.15.16:0/3208665343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2e8102cd0 0x7fd2e8107200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:09.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.045+0000 7fd2ef200640 1 --2- 172.21.15.16:0/3208665343 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2e8102380 0x7fd2e8102790 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:09.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.045+0000 7fd2ef200640 1 -- 172.21.15.16:0/3208665343 >> 172.21.15.16:0/3208665343 conn(0x7fd2e80fe090 msgr2=0x7fd2e81004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:09.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.046+0000 7fd2ef200640 1 -- 172.21.15.16:0/3208665343 shutdown_connections 2024-08-31T06:19:09.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.046+0000 7fd2ef200640 1 -- 172.21.15.16:0/3208665343 wait complete. 2024-08-31T06:19:09.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.048+0000 7fd2ef200640 1 Processor -- start 2024-08-31T06:19:09.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.048+0000 7fd2ef200640 1 -- start start 2024-08-31T06:19:09.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.049+0000 7fd2ef200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2e8102380 0x7fd2e81a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:09.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.049+0000 7fd2ef200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2e8102cd0 0x7fd2e81a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:09.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.049+0000 7fd2ef200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2e81077c0 0x7fd2e81a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:09.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.049+0000 7fd2ef200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e8119d30 con 0x7fd2e8102cd0 2024-08-31T06:19:09.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.049+0000 7fd2ef200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e8119bb0 con 0x7fd2e8102380 2024-08-31T06:19:09.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.049+0000 7fd2ef200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e8119eb0 con 0x7fd2e81077c0 2024-08-31T06:19:09.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.049+0000 7fd2e7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2e8102380 0x7fd2e81a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:09.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.049+0000 7fd2e7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2e8102380 0x7fd2e81a06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51216/0 (socket says 172.21.15.16:51216) 2024-08-31T06:19:09.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.049+0000 7fd2e7e00640 1 -- 172.21.15.16:0/2527746185 learned_addr learned my addr 172.21.15.16:0/2527746185 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:09.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.049+0000 7fd2e7400640 1 --2- 172.21.15.16:0/2527746185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2e8102cd0 0x7fd2e81a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:09.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.049+0000 7fd2ed200640 1 --2- 172.21.15.16:0/2527746185 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2e81077c0 0x7fd2e81a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:09.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.050+0000 7fd2e7e00640 1 -- 172.21.15.16:0/2527746185 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2e81077c0 msgr2=0x7fd2e81a5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:09.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.050+0000 7fd2e7e00640 1 --2- 172.21.15.16:0/2527746185 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2e81077c0 0x7fd2e81a5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:09.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.050+0000 7fd2e7e00640 1 -- 172.21.15.16:0/2527746185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2e8102cd0 msgr2=0x7fd2e81a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:09.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.050+0000 7fd2e7e00640 1 --2- 172.21.15.16:0/2527746185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2e8102cd0 0x7fd2e81a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:09.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.050+0000 7fd2e7e00640 1 -- 172.21.15.16:0/2527746185 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2e81a6590 con 0x7fd2e8102380 2024-08-31T06:19:09.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.050+0000 7fd2e7400640 1 --2- 172.21.15.16:0/2527746185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2e8102cd0 0x7fd2e81a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:19:09.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.050+0000 7fd2e7e00640 1 --2- 172.21.15.16:0/2527746185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2e8102380 0x7fd2e81a06e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8006010 tx=0x7fd2d8003d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:09.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.052+0000 7fd2e4c00640 1 -- 172.21.15.16:0/2527746185 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d803f070 con 0x7fd2e8102380 2024-08-31T06:19:09.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.052+0000 7fd2e4c00640 1 -- 172.21.15.16:0/2527746185 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2d803bd20 con 0x7fd2e8102380 2024-08-31T06:19:09.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.052+0000 7fd2e4c00640 1 -- 172.21.15.16:0/2527746185 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d8035030 con 0x7fd2e8102380 2024-08-31T06:19:09.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.052+0000 7fd2ef200640 1 -- 172.21.15.16:0/2527746185 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2e81a6820 con 0x7fd2e8102380 2024-08-31T06:19:09.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.052+0000 7fd2ef200640 1 -- 172.21.15.16:0/2527746185 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2e81a6cb0 con 0x7fd2e8102380 2024-08-31T06:19:09.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.054+0000 7fd2ef200640 1 -- 172.21.15.16:0/2527746185 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2c80052c0 con 0x7fd2e8102380 2024-08-31T06:19:09.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.057+0000 7fd2e4c00640 1 -- 172.21.15.16:0/2527746185 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fd2d8042070 con 0x7fd2e8102380 2024-08-31T06:19:09.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.059+0000 7fd2e4c00640 1 --2- 172.21.15.16:0/2527746185 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd2cc0b4e20 0x7fd2cc0b7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:09.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.059+0000 7fd2e4c00640 1 -- 172.21.15.16:0/2527746185 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8514+0+0 (secure 0 0 0) 0x7fd2d8105c50 con 0x7fd2e8102380 2024-08-31T06:19:09.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.060+0000 7fd2e7400640 1 --2- 172.21.15.16:0/2527746185 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd2cc0b4e20 0x7fd2cc0b7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:09.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.061+0000 7fd2e7400640 1 --2- 172.21.15.16:0/2527746185 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd2cc0b4e20 0x7fd2cc0b7330 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8108190 tx=0x7fd2d400a2d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:09.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.063+0000 7fd2e4c00640 1 -- 172.21.15.16:0/2527746185 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fd2d80cc9e0 con 0x7fd2e8102380 2024-08-31T06:19:09.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:08 smithi155 ceph-mon[21742]: pgmap v734: 97 pgs: 10 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 67 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 19 MiB/s wr, 1.66k op/s 2024-08-31T06:19:09.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:08 smithi184 ceph-mon[22331]: pgmap v734: 97 pgs: 10 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 67 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 19 MiB/s wr, 1.66k op/s 2024-08-31T06:19:09.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.458+0000 7fd2ef200640 1 -- 172.21.15.16:0/2527746185 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd2c8005880 con 0x7fd2e8102380 2024-08-31T06:19:09.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.459+0000 7fd2e4c00640 1 -- 172.21.15.16:0/2527746185 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11920 (secure 0 0 0) 0x7fd2d80cc2e0 con 0x7fd2e8102380 2024-08-31T06:19:09.462 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:19:09.462 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":47,"btime":"2024-08-31T06:18:22:635831+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":47,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:18:22.030000+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24737,24724,24712]},"id":1}]} 2024-08-31T06:19:09.462 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 47 2024-08-31T06:19:09.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.465+0000 7fd2ef200640 1 -- 172.21.15.16:0/2527746185 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd2cc0b4e20 msgr2=0x7fd2cc0b7330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:09.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.465+0000 7fd2ef200640 1 --2- 172.21.15.16:0/2527746185 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd2cc0b4e20 0x7fd2cc0b7330 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8108190 tx=0x7fd2d400a2d0 comp rx=0 tx=0).stop 2024-08-31T06:19:09.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.465+0000 7fd2ef200640 1 -- 172.21.15.16:0/2527746185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2e8102380 msgr2=0x7fd2e81a06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:09.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.465+0000 7fd2ef200640 1 --2- 172.21.15.16:0/2527746185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2e8102380 0x7fd2e81a06e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8006010 tx=0x7fd2d8003d50 comp rx=0 tx=0).stop 2024-08-31T06:19:09.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.465+0000 7fd2ef200640 1 -- 172.21.15.16:0/2527746185 shutdown_connections 2024-08-31T06:19:09.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.465+0000 7fd2ef200640 1 --2- 172.21.15.16:0/2527746185 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd2cc0b4e20 0x7fd2cc0b7330 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:09.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.465+0000 7fd2ef200640 1 --2- 172.21.15.16:0/2527746185 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2e81077c0 0x7fd2e81a5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:09.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.465+0000 7fd2ef200640 1 --2- 172.21.15.16:0/2527746185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2e8102cd0 0x7fd2e81a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:09.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.465+0000 7fd2ef200640 1 --2- 172.21.15.16:0/2527746185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2e8102380 0x7fd2e81a06e0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:09.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.465+0000 7fd2ef200640 1 -- 172.21.15.16:0/2527746185 >> 172.21.15.16:0/2527746185 conn(0x7fd2e80fe090 msgr2=0x7fd2e81051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:09.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.465+0000 7fd2ef200640 1 -- 172.21.15.16:0/2527746185 shutdown_connections 2024-08-31T06:19:09.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:09.466+0000 7fd2ef200640 1 -- 172.21.15.16:0/2527746185 wait complete. 2024-08-31T06:19:10.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:09 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/2527746185' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:19:10.123 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:09 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/2527746185' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:19:10.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:09 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/2527746185' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:19:10.485 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T06:19:11.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:10 smithi155 ceph-mon[21742]: pgmap v735: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 3.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 17 MiB/s wr, 1.56k op/s 2024-08-31T06:19:11.147 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:19:11.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:10 smithi016 ceph-mon[17194]: pgmap v735: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 3.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 17 MiB/s wr, 1.56k op/s 2024-08-31T06:19:11.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:10 smithi184 ceph-mon[22331]: pgmap v735: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 3.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 17 MiB/s wr, 1.56k op/s 2024-08-31T06:19:11.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 705.0 s, 1381.0 tps, lat 75.247 ms stddev 122.195 2024-08-31T06:19:12.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.052+0000 7facb0e00640 1 -- 172.21.15.16:0/195379708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7facac105960 msgr2=0x7facac105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:12.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.052+0000 7facb0e00640 1 --2- 172.21.15.16:0/195379708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7facac105960 0x7facac105d70 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fac98009900 tx=0x7fac9802a410 comp rx=0 tx=0).stop 2024-08-31T06:19:12.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.052+0000 7facb0e00640 1 -- 172.21.15.16:0/195379708 shutdown_connections 2024-08-31T06:19:12.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.052+0000 7facb0e00640 1 --2- 172.21.15.16:0/195379708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facac10bbf0 0x7facac112420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:12.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.052+0000 7facb0e00640 1 --2- 172.21.15.16:0/195379708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facac10b2a0 0x7facac10b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:12.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.052+0000 7facb0e00640 1 --2- 172.21.15.16:0/195379708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7facac105960 0x7facac105d70 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:12.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.052+0000 7facb0e00640 1 -- 172.21.15.16:0/195379708 >> 172.21.15.16:0/195379708 conn(0x7facac1013d0 msgr2=0x7facac103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:12.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.052+0000 7facb0e00640 1 -- 172.21.15.16:0/195379708 shutdown_connections 2024-08-31T06:19:12.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.052+0000 7facb0e00640 1 -- 172.21.15.16:0/195379708 wait complete. 2024-08-31T06:19:12.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.053+0000 7facb0e00640 1 Processor -- start 2024-08-31T06:19:12.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.054+0000 7facb0e00640 1 -- start start 2024-08-31T06:19:12.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.055+0000 7facb0e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facac105960 0x7facac07b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:12.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.055+0000 7facb0e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facac10b2a0 0x7facac0776c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:12.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.055+0000 7facb0e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7facac10bbf0 0x7facac077de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:12.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.055+0000 7facb0e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7facac115900 con 0x7facac105960 2024-08-31T06:19:12.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.055+0000 7facb0e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7facac115780 con 0x7facac10b2a0 2024-08-31T06:19:12.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.055+0000 7facb0e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7facac115a80 con 0x7facac10bbf0 2024-08-31T06:19:12.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.055+0000 7facab400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facac105960 0x7facac07b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:12.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.055+0000 7facaaa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facac10b2a0 0x7facac0776c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:12.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.055+0000 7facaaa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facac10b2a0 0x7facac0776c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51242/0 (socket says 172.21.15.16:51242) 2024-08-31T06:19:12.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.055+0000 7facaaa00640 1 -- 172.21.15.16:0/3559794071 learned_addr learned my addr 172.21.15.16:0/3559794071 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:12.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.055+0000 7facab400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facac105960 0x7facac07b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50358/0 (socket says 172.21.15.16:50358) 2024-08-31T06:19:12.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.056+0000 7facaaa00640 1 -- 172.21.15.16:0/3559794071 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7facac10bbf0 msgr2=0x7facac077de0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-31T06:19:12.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.056+0000 7facaaa00640 1 --2- 172.21.15.16:0/3559794071 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7facac10bbf0 0x7facac077de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:12.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.056+0000 7facaaa00640 1 -- 172.21.15.16:0/3559794071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facac105960 msgr2=0x7facac07b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:12.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.056+0000 7facaaa00640 1 --2- 172.21.15.16:0/3559794071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facac105960 0x7facac07b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:12.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.056+0000 7facaaa00640 1 -- 172.21.15.16:0/3559794071 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facac19bfc0 con 0x7facac10b2a0 2024-08-31T06:19:12.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.057+0000 7facaaa00640 1 --2- 172.21.15.16:0/3559794071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facac10b2a0 0x7facac0776c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fac9c00e9f0 tx=0x7fac9c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:12.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.057+0000 7faca3e00640 1 -- 172.21.15.16:0/3559794071 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c00ccc0 con 0x7facac10b2a0 2024-08-31T06:19:12.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.058+0000 7faca3e00640 1 -- 172.21.15.16:0/3559794071 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac9c0108e0 con 0x7facac10b2a0 2024-08-31T06:19:12.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.058+0000 7faca3e00640 1 -- 172.21.15.16:0/3559794071 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c0188e0 con 0x7facac10b2a0 2024-08-31T06:19:12.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.059+0000 7facb0e00640 1 -- 172.21.15.16:0/3559794071 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facac19c2b0 con 0x7facac10b2a0 2024-08-31T06:19:12.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.060+0000 7facb0e00640 1 -- 172.21.15.16:0/3559794071 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facac19c770 con 0x7facac10b2a0 2024-08-31T06:19:12.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.069+0000 7facb906b740 1 Processor -- start 2024-08-31T06:19:12.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.069+0000 7facb906b740 1 -- start start 2024-08-31T06:19:12.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.069+0000 7facb906b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c120fe4070 0x55c121389b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:12.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.070+0000 7facb906b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c120fbb1b0 0x55c12138a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:12.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.070+0000 7facb906b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c120fc3390 0x55c12138a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:12.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.070+0000 7facb906b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c1213007b0 con 0x55c120fbb1b0 2024-08-31T06:19:12.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.070+0000 7facb906b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c1212efd40 con 0x55c120fc3390 2024-08-31T06:19:12.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.070+0000 7facb906b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c1212f3cf0 con 0x55c120fe4070 2024-08-31T06:19:12.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.070+0000 7facab400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c120fbb1b0 0x55c12138a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:12.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.071+0000 7facab400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c120fbb1b0 0x55c12138a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50362/0 (socket says 172.21.15.16:50362) 2024-08-31T06:19:12.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.073+0000 7facab400640 1 -- 172.21.15.16:0/2077950198 learned_addr learned my addr 172.21.15.16:0/2077950198 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:12.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.073+0000 7facab400640 1 -- 172.21.15.16:0/2077950198 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c120fe4070 msgr2=0x55c121389b60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T06:19:12.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.073+0000 7facab400640 1 --2- 172.21.15.16:0/2077950198 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c120fe4070 0x55c121389b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:12.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.073+0000 7facab400640 1 -- 172.21.15.16:0/2077950198 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c120fc3390 msgr2=0x55c12138a5e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:19:12.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.073+0000 7facab400640 1 --2- 172.21.15.16:0/2077950198 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c120fc3390 0x55c12138a5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:12.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.073+0000 7facab400640 1 -- 172.21.15.16:0/2077950198 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c12104f650 con 0x55c120fbb1b0 2024-08-31T06:19:12.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.074+0000 7facab400640 1 --2- 172.21.15.16:0/2077950198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c120fbb1b0 0x55c12138a0a0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fac98000c00 tx=0x7fac98004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=792f65ebc8dcfec5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.075+0000 7faca0c00640 1 -- 172.21.15.16:0/2077950198 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac98035040 con 0x55c120fbb1b0 2024-08-31T06:19:12.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.076+0000 7faca0c00640 1 -- 172.21.15.16:0/2077950198 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac980054f0 con 0x55c120fbb1b0 2024-08-31T06:19:12.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.077+0000 7faca0c00640 1 -- 172.21.15.16:0/2077950198 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac98003850 con 0x55c120fbb1b0 2024-08-31T06:19:12.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.078+0000 7facb906b740 1 -- 172.21.15.16:0/2077950198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c120fbb1b0 msgr2=0x55c12138a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:12.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.078+0000 7facb906b740 1 --2- 172.21.15.16:0/2077950198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c120fbb1b0 0x55c12138a0a0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fac98000c00 tx=0x7fac98004170 comp rx=0 tx=0).stop 2024-08-31T06:19:12.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.079+0000 7facb906b740 1 -- 172.21.15.16:0/2077950198 shutdown_connections 2024-08-31T06:19:12.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.079+0000 7facb906b740 1 --2- 172.21.15.16:0/2077950198 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c120fc3390 0x55c12138a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:12.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.079+0000 7facb906b740 1 --2- 172.21.15.16:0/2077950198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c120fbb1b0 0x55c12138a0a0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:12.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.079+0000 7facb906b740 1 --2- 172.21.15.16:0/2077950198 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c120fe4070 0x55c121389b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:12.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.079+0000 7facb906b740 1 -- 172.21.15.16:0/2077950198 >> 172.21.15.16:0/2077950198 conn(0x55c120fd8460 msgr2=0x55c121387a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:12.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.079+0000 7faca3e00640 1 -- 172.21.15.16:0/3559794071 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fac9c010430 con 0x7facac10b2a0 2024-08-31T06:19:12.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.082+0000 7facb906b740 1 -- 172.21.15.16:0/2077950198 shutdown_connections 2024-08-31T06:19:12.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.083+0000 7facb906b740 1 -- 172.21.15.16:0/2077950198 wait complete. 2024-08-31T06:19:12.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.084+0000 7faca3e00640 1 --2- 172.21.15.16:0/3559794071 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fac900b4c60 0x7fac900b7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:12.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.084+0000 7faca3e00640 1 -- 172.21.15.16:0/3559794071 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8514+0+0 (secure 0 0 0) 0x7fac9c0e59e0 con 0x7facac10b2a0 2024-08-31T06:19:12.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.085+0000 7facb906b740 1 -- start start 2024-08-31T06:19:12.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.085+0000 7facb906b740 1 Processor -- start 2024-08-31T06:19:12.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.087+0000 7facab400640 1 --2- 172.21.15.16:0/3559794071 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fac900b4c60 0x7fac900b7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.089+0000 7facab400640 1 --2- 172.21.15.16:0/3559794071 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fac900b4c60 0x7fac900b7170 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fac980057c0 tx=0x7fac98005fd0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.090+0000 7facb906b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c120fe4070 0x55c1213950c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:12.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.091+0000 7facb906b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c120fa29f0 0x55c121395600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:12.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.091+0000 7facb906b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c120f77da0 0x55c121395b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:12.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.091+0000 7facb906b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c121152700 con 0x55c120fe4070 2024-08-31T06:19:12.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.091+0000 7facb906b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c121276cf0 con 0x55c120f77da0 2024-08-31T06:19:12.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.091+0000 7facb906b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c121398330 con 0x55c120fa29f0 2024-08-31T06:19:12.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.091+0000 7facb906b740 10 client.0 _ll_register_callbacks cb 0x55c12129c470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:19:12.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.091+0000 7facaaa00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c120fa29f0 0x55c121395600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:12.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.091+0000 7facaaa00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c120fa29f0 0x55c121395600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:58630/0 (socket says 172.21.15.16:58630) 2024-08-31T06:19:12.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.091+0000 7facaaa00640 1 -- 172.21.15.16:0/3287675622 learned_addr learned my addr 172.21.15.16:0/3287675622 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:12.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.091+0000 7facabe00640 1 --2- 172.21.15.16:0/3287675622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c120f77da0 0x55c121395b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:12.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.092+0000 7facaaa00640 1 -- 172.21.15.16:0/3287675622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c120f77da0 msgr2=0x55c121395b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:12.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.092+0000 7facaaa00640 1 --2- 172.21.15.16:0/3287675622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c120f77da0 0x55c121395b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:12.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.092+0000 7facaaa00640 1 -- 172.21.15.16:0/3287675622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c120fe4070 msgr2=0x55c1213950c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:19:12.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.093+0000 7facab400640 1 --2- 172.21.15.16:0/3287675622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c120fe4070 0x55c1213950c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:12.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.093+0000 7facaaa00640 1 --2- 172.21.15.16:0/3287675622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c120fe4070 0x55c1213950c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.093+0000 7facaaa00640 1 -- 172.21.15.16:0/3287675622 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1212b78b0 con 0x55c120fa29f0 2024-08-31T06:19:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.094+0000 7facaaa00640 1 --2- 172.21.15.16:0/3287675622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c120fa29f0 0x55c121395600 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fac9c0a92e0 tx=0x7fac9c0abb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88279dc1212fb3fe server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.094+0000 7fac95600640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:19:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.095+0000 7fac95600640 1 -- 172.21.15.16:0/3287675622 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c091e60 con 0x55c120fa29f0 2024-08-31T06:19:12.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.095+0000 7facb906b740 1 -- 172.21.15.16:0/3287675622 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac980054f0 con 0x55c120fa29f0 2024-08-31T06:19:12.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.095+0000 7fac95600640 1 -- 172.21.15.16:0/3287675622 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8514+0+0 (secure 0 0 0) 0x7fac9c0ab2d0 con 0x55c120fa29f0 2024-08-31T06:19:12.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.096+0000 7fac95600640 1 -- 172.21.15.16:0/3287675622 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fac9c0e3560 con 0x55c120fa29f0 2024-08-31T06:19:12.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.096+0000 7facb906b740 10 client.28315 fetch_fsmap learned FSMap version 47 2024-08-31T06:19:12.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.096+0000 7facb906b740 1 -- 172.21.15.16:0/3287675622 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55c1213a3fe0 con 0x55c120fa29f0 2024-08-31T06:19:12.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.097+0000 7fac95600640 1 -- 172.21.15.16:0/3287675622 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac980054f0 con 0x55c120fa29f0 2024-08-31T06:19:12.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.099+0000 7fac95600640 1 -- 172.21.15.16:0/3287675622 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 47) ==== 8354+0+0 (secure 0 0 0) 0x7fac9c0aad40 con 0x55c120fa29f0 2024-08-31T06:19:12.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.099+0000 7facb906b740 10 client.28315 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T06:19:12.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.099+0000 7facb906b740 10 client.28315 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T06:19:12.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.099+0000 7facb906b740 20 client.28315 populate_metadata read hostname 'smithi016' 2024-08-31T06:19:12.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.100+0000 7facb906b740 1 --2- 172.21.15.16:0/3287675622 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55c1213a54a0 0x55c1213a78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:12.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.100+0000 7facb906b740 4 client.28315 mds_command: new command op to 24712 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:19:12.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.100+0000 7facb906b740 1 -- 172.21.15.16:0/3287675622 --> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c121302700 con 0x55c1213a54a0 2024-08-31T06:19:12.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.100+0000 7facab400640 1 --2- 172.21.15.16:0/3287675622 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55c1213a54a0 0x55c1213a78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:12.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.103+0000 7facab400640 1 --2- 172.21.15.16:0/3287675622 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55c1213a54a0 0x55c1213a78c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3ab897056ea4379f server_cookie=479671e7adc7b372 in_seq=0 out_seq=0 2024-08-31T06:19:12.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.103+0000 7fac95600640 10 client.28315 ms_handle_connect on v2:172.21.15.184:6838/4011051479 2024-08-31T06:19:12.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.105+0000 7fac95600640 1 -- 172.21.15.16:0/3287675622 <== mds.2 v2:172.21.15.184:6838/4011051479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c121302700 con 0x55c1213a54a0 2024-08-31T06:19:12.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.105+0000 7fac95600640 10 client.28315 handle_command_reply: tid=0 2024-08-31T06:19:12.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.105+0000 7facb906b740 1 client.28315 shutdown 2024-08-31T06:19:12.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.105+0000 7facb906b740 1 -- 172.21.15.16:0/3287675622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c120fa29f0 msgr2=0x55c121395600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:12.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.105+0000 7facb906b740 1 --2- 172.21.15.16:0/3287675622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c120fa29f0 0x55c121395600 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fac9c0a92e0 tx=0x7fac9c0abb80 comp rx=0 tx=0).stop 2024-08-31T06:19:12.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.105+0000 7facb906b740 1 -- 172.21.15.16:0/3287675622 shutdown_connections 2024-08-31T06:19:12.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.105+0000 7facb906b740 1 --2- 172.21.15.16:0/3287675622 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55c1213a54a0 0x55c1213a78c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:12.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.105+0000 7facb906b740 1 --2- 172.21.15.16:0/3287675622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c120fa29f0 0x55c121395600 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:12.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.105+0000 7facb906b740 1 --2- 172.21.15.16:0/3287675622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c120f77da0 0x55c121395b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:12.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.105+0000 7facb906b740 1 --2- 172.21.15.16:0/3287675622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c120fe4070 0x55c1213950c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:12.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.105+0000 7facb906b740 1 -- 172.21.15.16:0/3287675622 >> 172.21.15.16:0/3287675622 conn(0x55c120fd8460 msgr2=0x55c121388b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:12.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.105+0000 7fac95600640 0 client.28315 ms_handle_reset on v2:172.21.15.184:6838/4011051479 2024-08-31T06:19:12.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.106+0000 7facb906b740 1 -- 172.21.15.16:0/3287675622 shutdown_connections 2024-08-31T06:19:12.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.106+0000 7facb906b740 1 -- 172.21.15.16:0/3287675622 wait complete. 2024-08-31T06:19:12.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.106+0000 7facb906b740 20 client.28315 trim_cache size 0 max 16384 2024-08-31T06:19:12.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.136+0000 7facb906b740 1 Processor -- start 2024-08-31T06:19:12.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.136+0000 7facb906b740 1 -- start start 2024-08-31T06:19:12.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.137+0000 7facb906b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c120f77da0 0x55c1213957d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:12.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.137+0000 7facb906b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c120fe4070 0x55c121392c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:12.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.137+0000 7facb906b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c121389380 0x55c1213acf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:12.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.137+0000 7facb906b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7facac1c0a80 con 0x55c121389380 2024-08-31T06:19:12.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.137+0000 7facb906b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7facac1be2f0 con 0x55c120fe4070 2024-08-31T06:19:12.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.137+0000 7facb906b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7facac1a30a0 con 0x55c120f77da0 2024-08-31T06:19:12.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.138+0000 7facab400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c120f77da0 0x55c1213957d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:12.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.138+0000 7facabe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c121389380 0x55c1213acf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:12.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.138+0000 7facab400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c120f77da0 0x55c1213957d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:58632/0 (socket says 172.21.15.16:58632) 2024-08-31T06:19:12.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.138+0000 7facab400640 1 -- 172.21.15.16:0/2791699630 learned_addr learned my addr 172.21.15.16:0/2791699630 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:12.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.138+0000 7facaaa00640 1 --2- 172.21.15.16:0/2791699630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c120fe4070 0x55c121392c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:12.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.138+0000 7facab400640 1 -- 172.21.15.16:0/2791699630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c120fe4070 msgr2=0x55c121392c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:12.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.138+0000 7facab400640 1 --2- 172.21.15.16:0/2791699630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c120fe4070 0x55c121392c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:12.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.138+0000 7facab400640 1 -- 172.21.15.16:0/2791699630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c121389380 msgr2=0x55c1213acf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:12.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.138+0000 7facab400640 1 --2- 172.21.15.16:0/2791699630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c121389380 0x55c1213acf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:12.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.138+0000 7facab400640 1 -- 172.21.15.16:0/2791699630 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c12138be50 con 0x55c120f77da0 2024-08-31T06:19:12.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.138+0000 7facabe00640 1 --2- 172.21.15.16:0/2791699630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c121389380 0x55c1213acf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:19:12.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.140+0000 7facab400640 1 --2- 172.21.15.16:0/2791699630 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c120f77da0 0x55c1213957d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fac98004d00 tx=0x7fac98004c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dd40c21a1814814 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.140+0000 7fac97e00640 1 -- 172.21.15.16:0/2791699630 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac98035050 con 0x55c120f77da0 2024-08-31T06:19:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.140+0000 7fac97e00640 1 -- 172.21.15.16:0/2791699630 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac98066070 con 0x55c120f77da0 2024-08-31T06:19:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.140+0000 7fac97e00640 1 -- 172.21.15.16:0/2791699630 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac98003b00 con 0x55c120f77da0 2024-08-31T06:19:12.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.142+0000 7facb906b740 1 -- 172.21.15.16:0/2791699630 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c120f77da0 msgr2=0x55c1213957d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:12.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.142+0000 7facb906b740 1 --2- 172.21.15.16:0/2791699630 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c120f77da0 0x55c1213957d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fac98004d00 tx=0x7fac98004c90 comp rx=0 tx=0).stop 2024-08-31T06:19:12.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.143+0000 7facb906b740 1 -- 172.21.15.16:0/2791699630 shutdown_connections 2024-08-31T06:19:12.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.143+0000 7facb906b740 1 --2- 172.21.15.16:0/2791699630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c121389380 0x55c1213acf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:12.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.143+0000 7facb906b740 1 --2- 172.21.15.16:0/2791699630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c120fe4070 0x55c121392c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:12.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.143+0000 7facb906b740 1 --2- 172.21.15.16:0/2791699630 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c120f77da0 0x55c1213957d0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:12.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.143+0000 7facb906b740 1 -- 172.21.15.16:0/2791699630 >> 172.21.15.16:0/2791699630 conn(0x55c120fd8460 msgr2=0x55c1213950c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:12.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.144+0000 7facb906b740 1 -- 172.21.15.16:0/2791699630 shutdown_connections 2024-08-31T06:19:12.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.144+0000 7facb906b740 1 -- 172.21.15.16:0/2791699630 wait complete. 2024-08-31T06:19:12.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.145+0000 7facb906b740 1 -- start start 2024-08-31T06:19:12.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.146+0000 7facb906b740 1 Processor -- start 2024-08-31T06:19:12.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.147+0000 7facb906b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c120f77da0 0x55c1213928c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:12.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.148+0000 7facb906b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c120fe4070 0x55c12138ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:12.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.148+0000 7facb906b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c121389380 0x55c12138f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.148+0000 7facab400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c120f77da0 0x55c1213928c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.148+0000 7facb906b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7facac1a3750 con 0x55c120f77da0 2024-08-31T06:19:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.148+0000 7facb906b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7facac1bdc20 con 0x55c120fe4070 2024-08-31T06:19:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.148+0000 7facaaa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c120fe4070 0x55c12138ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.148+0000 7facaaa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c120fe4070 0x55c12138ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51264/0 (socket says 172.21.15.16:51264) 2024-08-31T06:19:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.148+0000 7facaaa00640 1 -- 172.21.15.16:0/1759713693 learned_addr learned my addr 172.21.15.16:0/1759713693 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.148+0000 7facb906b740 1 -- 172.21.15.16:0/1759713693 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7facac1a3e20 con 0x55c121389380 2024-08-31T06:19:12.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.148+0000 7facb906b740 10 client.0 _ll_register_callbacks cb 0x7fac9c0ab870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:19:12.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.149+0000 7facabe00640 1 --2- 172.21.15.16:0/1759713693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c121389380 0x55c12138f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:12.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.149+0000 7facaaa00640 1 -- 172.21.15.16:0/1759713693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c121389380 msgr2=0x55c12138f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:12.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.149+0000 7facaaa00640 1 --2- 172.21.15.16:0/1759713693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c121389380 0x55c12138f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:12.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.149+0000 7facaaa00640 1 -- 172.21.15.16:0/1759713693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c120f77da0 msgr2=0x55c1213928c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:12.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.149+0000 7facaaa00640 1 --2- 172.21.15.16:0/1759713693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c120f77da0 0x55c1213928c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:12.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.149+0000 7facaaa00640 1 -- 172.21.15.16:0/1759713693 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c12138f8d0 con 0x55c120fe4070 2024-08-31T06:19:12.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.150+0000 7facaaa00640 1 --2- 172.21.15.16:0/1759713693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c120fe4070 0x55c12138ed80 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fac9c0ab0e0 tx=0x7fac9c0ab700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d48679e03f533f4 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:12.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.150+0000 7fac95600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:19:12.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.151+0000 7fac95600640 1 -- 172.21.15.16:0/1759713693 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c091e60 con 0x55c120fe4070 2024-08-31T06:19:12.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.151+0000 7fac95600640 1 -- 172.21.15.16:0/1759713693 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8514+0+0 (secure 0 0 0) 0x7fac9c0a8660 con 0x55c120fe4070 2024-08-31T06:19:12.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.152+0000 7facb906b740 1 -- 172.21.15.16:0/1759713693 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac98066070 con 0x55c120fe4070 2024-08-31T06:19:12.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.152+0000 7fac95600640 1 -- 172.21.15.16:0/1759713693 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fac9c0b08e0 con 0x55c120fe4070 2024-08-31T06:19:12.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.152+0000 7facb906b740 10 client.28307 fetch_fsmap learned FSMap version 47 2024-08-31T06:19:12.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.152+0000 7facb906b740 1 -- 172.21.15.16:0/1759713693 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55c121412e70 con 0x55c120fe4070 2024-08-31T06:19:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.152+0000 7fac95600640 1 -- 172.21.15.16:0/1759713693 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac98066070 con 0x55c120fe4070 2024-08-31T06:19:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.152+0000 7fac95600640 1 -- 172.21.15.16:0/1759713693 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 47) ==== 8354+0+0 (secure 0 0 0) 0x7fac9c018c80 con 0x55c120fe4070 2024-08-31T06:19:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.152+0000 7facb906b740 10 client.28307 fetch_fsmap finished waiting for FSMap version 47 2024-08-31T06:19:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.152+0000 7facb906b740 10 client.28307 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T06:19:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.152+0000 7facb906b740 20 client.28307 populate_metadata read hostname 'smithi016' 2024-08-31T06:19:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.152+0000 7facb906b740 1 --2- 172.21.15.16:0/1759713693 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55c1214135f0 0x55c121415a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.153+0000 7facb906b740 4 client.28307 mds_command: new command op to 24712 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:19:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.153+0000 7facb906b740 1 -- 172.21.15.16:0/1759713693 --> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c1212fe1a0 con 0x55c1214135f0 2024-08-31T06:19:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.153+0000 7facab400640 1 --2- 172.21.15.16:0/1759713693 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55c1214135f0 0x55c121415a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:12.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.154+0000 7facab400640 1 --2- 172.21.15.16:0/1759713693 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55c1214135f0 0x55c121415a10 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3555d786581ada63 server_cookie=7711801bdf9f1e79 in_seq=0 out_seq=0 2024-08-31T06:19:12.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.154+0000 7fac95600640 10 client.28307 ms_handle_connect on v2:172.21.15.184:6838/4011051479 2024-08-31T06:19:12.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.154+0000 7fac95600640 1 -- 172.21.15.16:0/1759713693 <== mds.2 v2:172.21.15.184:6838/4011051479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c1212fe1a0 con 0x55c1214135f0 2024-08-31T06:19:12.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.154+0000 7fac95600640 10 client.28307 handle_command_reply: tid=0 2024-08-31T06:19:12.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.154+0000 7facb906b740 1 client.28307 shutdown 2024-08-31T06:19:12.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.155+0000 7facb906b740 1 -- 172.21.15.16:0/1759713693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c120fe4070 msgr2=0x55c12138ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:12.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.155+0000 7facb906b740 1 --2- 172.21.15.16:0/1759713693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c120fe4070 0x55c12138ed80 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fac9c0ab0e0 tx=0x7fac9c0ab700 comp rx=0 tx=0).stop 2024-08-31T06:19:12.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.155+0000 7facb906b740 1 -- 172.21.15.16:0/1759713693 shutdown_connections 2024-08-31T06:19:12.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.155+0000 7facb906b740 1 --2- 172.21.15.16:0/1759713693 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55c1214135f0 0x55c121415a10 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:12.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.155+0000 7facb906b740 1 --2- 172.21.15.16:0/1759713693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c121389380 0x55c12138f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:12.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.155+0000 7facb906b740 1 --2- 172.21.15.16:0/1759713693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c120fe4070 0x55c12138ed80 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:12.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.155+0000 7facb906b740 1 --2- 172.21.15.16:0/1759713693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c120f77da0 0x55c1213928c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:12.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.155+0000 7facb906b740 1 -- 172.21.15.16:0/1759713693 >> 172.21.15.16:0/1759713693 conn(0x55c120fd8460 msgr2=0x55c1213a0650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:12.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.155+0000 7fac95600640 0 client.28307 ms_handle_reset on v2:172.21.15.184:6838/4011051479 2024-08-31T06:19:12.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.155+0000 7facb906b740 1 -- 172.21.15.16:0/1759713693 shutdown_connections 2024-08-31T06:19:12.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.155+0000 7facb906b740 1 -- 172.21.15.16:0/1759713693 wait complete. 2024-08-31T06:19:12.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.156+0000 7facb906b740 20 client.28307 trim_cache size 0 max 16384 2024-08-31T06:19:12.159 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:19:12.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.157+0000 7facb0e00640 1 -- 172.21.15.16:0/3559794071 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fac900b4c60 msgr2=0x7fac900b7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:12.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.157+0000 7facb0e00640 1 --2- 172.21.15.16:0/3559794071 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fac900b4c60 0x7fac900b7170 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fac980057c0 tx=0x7fac98005fd0 comp rx=0 tx=0).stop 2024-08-31T06:19:12.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.158+0000 7facb0e00640 1 -- 172.21.15.16:0/3559794071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facac10b2a0 msgr2=0x7facac0776c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:12.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.158+0000 7facb0e00640 1 --2- 172.21.15.16:0/3559794071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facac10b2a0 0x7facac0776c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fac9c00e9f0 tx=0x7fac9c00eeb0 comp rx=0 tx=0).stop 2024-08-31T06:19:12.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.158+0000 7facb0e00640 1 -- 172.21.15.16:0/3559794071 shutdown_connections 2024-08-31T06:19:12.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.158+0000 7facb0e00640 1 --2- 172.21.15.16:0/3559794071 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fac900b4c60 0x7fac900b7170 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:12.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.158+0000 7facb0e00640 1 --2- 172.21.15.16:0/3559794071 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7facac10bbf0 0x7facac077de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:12.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.158+0000 7facb0e00640 1 --2- 172.21.15.16:0/3559794071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facac10b2a0 0x7facac0776c0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:12.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.158+0000 7facb0e00640 1 --2- 172.21.15.16:0/3559794071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facac105960 0x7facac07b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:12.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.158+0000 7facb0e00640 1 -- 172.21.15.16:0/3559794071 >> 172.21.15.16:0/3559794071 conn(0x7facac1013d0 msgr2=0x7facac1037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:12.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.158+0000 7facb0e00640 1 -- 172.21.15.16:0/3559794071 shutdown_connections 2024-08-31T06:19:12.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:12.158+0000 7facb0e00640 1 -- 172.21.15.16:0/3559794071 wait complete. 2024-08-31T06:19:13.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:12 smithi155 ceph-mon[21742]: pgmap v736: 97 pgs: 97 active+clean; 3.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 18 MiB/s wr, 1.63k op/s 2024-08-31T06:19:13.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:12 smithi016 ceph-mon[17194]: pgmap v736: 97 pgs: 97 active+clean; 3.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 18 MiB/s wr, 1.63k op/s 2024-08-31T06:19:13.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:12 smithi184 ceph-mon[22331]: pgmap v736: 97 pgs: 97 active+clean; 3.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 18 MiB/s wr, 1.63k op/s 2024-08-31T06:19:13.357 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T06:19:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.575+0000 7f2487200640 1 -- 172.21.15.16:0/602926595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24881044e0 msgr2=0x7f24881048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.575+0000 7f2487200640 1 --2- 172.21.15.16:0/602926595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24881044e0 0x7f24881048f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f2478009900 tx=0x7f247802a400 comp rx=0 tx=0).stop 2024-08-31T06:19:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.576+0000 7f2487200640 1 -- 172.21.15.16:0/602926595 shutdown_connections 2024-08-31T06:19:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.576+0000 7f2487200640 1 --2- 172.21.15.16:0/602926595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f248810d940 0x7f248810fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.576+0000 7f2487200640 1 --2- 172.21.15.16:0/602926595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2488104e30 0x7f248810d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.576+0000 7f2487200640 1 --2- 172.21.15.16:0/602926595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24881044e0 0x7f24881048f0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.576+0000 7f2487200640 1 -- 172.21.15.16:0/602926595 >> 172.21.15.16:0/602926595 conn(0x7f24880fdfd0 msgr2=0x7f2488100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.576+0000 7f2487200640 1 -- 172.21.15.16:0/602926595 shutdown_connections 2024-08-31T06:19:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.576+0000 7f2487200640 1 -- 172.21.15.16:0/602926595 wait complete. 2024-08-31T06:19:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.577+0000 7f2487200640 1 Processor -- start 2024-08-31T06:19:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.577+0000 7f2487200640 1 -- start start 2024-08-31T06:19:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.578+0000 7f2487200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24881044e0 0x7f248810a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.578+0000 7f2487200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2488104e30 0x7f248810abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.578+0000 7f2487200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f248810d940 0x7f248810b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.578+0000 7f2487200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2488103960 con 0x7f2488104e30 2024-08-31T06:19:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.578+0000 7f2487200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f24881037e0 con 0x7f24881044e0 2024-08-31T06:19:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.578+0000 7f2487200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2488103ae0 con 0x7f248810d940 2024-08-31T06:19:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.578+0000 7f2486800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f248810d940 0x7f248810b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.578+0000 7f2486800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f248810d940 0x7f248810b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:58654/0 (socket says 172.21.15.16:58654) 2024-08-31T06:19:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.578+0000 7f2486800640 1 -- 172.21.15.16:0/3358060278 learned_addr learned my addr 172.21.15.16:0/3358060278 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.578+0000 7f2485e00640 1 --2- 172.21.15.16:0/3358060278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24881044e0 0x7f248810a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.578+0000 7f2485400640 1 --2- 172.21.15.16:0/3358060278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2488104e30 0x7f248810abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.579+0000 7f2486800640 1 -- 172.21.15.16:0/3358060278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24881044e0 msgr2=0x7f248810a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.579+0000 7f2486800640 1 --2- 172.21.15.16:0/3358060278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24881044e0 0x7f248810a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.579+0000 7f2486800640 1 -- 172.21.15.16:0/3358060278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2488104e30 msgr2=0x7f248810abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.579+0000 7f2486800640 1 --2- 172.21.15.16:0/3358060278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2488104e30 0x7f248810abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.579+0000 7f2486800640 1 -- 172.21.15.16:0/3358060278 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2488073bd0 con 0x7f248810d940 2024-08-31T06:19:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.579+0000 7f2486800640 1 --2- 172.21.15.16:0/3358060278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f248810d940 0x7f248810b110 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f248000efd0 tx=0x7f248000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.579+0000 7f247ea00640 1 -- 172.21.15.16:0/3358060278 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2480003b70 con 0x7f248810d940 2024-08-31T06:19:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.579+0000 7f2487200640 1 -- 172.21.15.16:0/3358060278 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2488073ec0 con 0x7f248810d940 2024-08-31T06:19:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.579+0000 7f247ea00640 1 -- 172.21.15.16:0/3358060278 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2480003d10 con 0x7f248810d940 2024-08-31T06:19:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.579+0000 7f247ea00640 1 -- 172.21.15.16:0/3358060278 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24800108a0 con 0x7f248810d940 2024-08-31T06:19:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.580+0000 7f2487200640 1 -- 172.21.15.16:0/3358060278 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2488074410 con 0x7f248810d940 2024-08-31T06:19:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.585+0000 7f247ea00640 1 -- 172.21.15.16:0/3358060278 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f2480004020 con 0x7f248810d940 2024-08-31T06:19:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.586+0000 7f24939e6740 1 Processor -- start 2024-08-31T06:19:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.586+0000 7f24939e6740 1 -- start start 2024-08-31T06:19:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.586+0000 7f24939e6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55859d04e1b0 0x55859d3f18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.586+0000 7f24939e6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55859d0252f0 0x55859d3f1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.586+0000 7f24939e6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55859d02d4d0 0x55859d3f2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.586+0000 7f24939e6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55859d366f70 con 0x55859d0252f0 2024-08-31T06:19:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.586+0000 7f24939e6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55859d356500 con 0x55859d04e1b0 2024-08-31T06:19:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.586+0000 7f24939e6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55859d35a4b0 con 0x55859d02d4d0 2024-08-31T06:19:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.587+0000 7f2485e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55859d0252f0 0x55859d3f1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.587+0000 7f2485e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55859d0252f0 0x55859d3f1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50402/0 (socket says 172.21.15.16:50402) 2024-08-31T06:19:13.590 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.587+0000 7f2485e00640 1 -- 172.21.15.16:0/1944138058 learned_addr learned my addr 172.21.15.16:0/1944138058 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:13.590 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.587+0000 7f2485e00640 1 -- 172.21.15.16:0/1944138058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55859d02d4d0 msgr2=0x55859d3f2320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:19:13.590 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.587+0000 7f2485e00640 1 --2- 172.21.15.16:0/1944138058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55859d02d4d0 0x55859d3f2320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:13.590 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.587+0000 7f2485e00640 1 -- 172.21.15.16:0/1944138058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55859d04e1b0 msgr2=0x55859d3f18a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:19:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.589+0000 7f2485e00640 1 --2- 172.21.15.16:0/1944138058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55859d04e1b0 0x55859d3f18a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.589+0000 7f2485e00640 1 -- 172.21.15.16:0/1944138058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55859d310c10 con 0x55859d0252f0 2024-08-31T06:19:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.589+0000 7f2485e00640 1 --2- 172.21.15.16:0/1944138058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55859d0252f0 0x55859d3f1de0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f2478009900 tx=0x7f2478032b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a32e7a7b741f4ec6 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.590+0000 7f2477000640 1 -- 172.21.15.16:0/1944138058 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2478040070 con 0x55859d0252f0 2024-08-31T06:19:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.590+0000 7f2477000640 1 -- 172.21.15.16:0/1944138058 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2478004030 con 0x55859d0252f0 2024-08-31T06:19:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.590+0000 7f2477000640 1 -- 172.21.15.16:0/1944138058 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2478035040 con 0x55859d0252f0 2024-08-31T06:19:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.590+0000 7f247ea00640 1 --2- 172.21.15.16:0/3358060278 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f246c0b4ca0 0x7f246c0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.591+0000 7f247ea00640 1 -- 172.21.15.16:0/3358060278 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8514+0+0 (secure 0 0 0) 0x7f24800e5ac0 con 0x7f248810d940 2024-08-31T06:19:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.591+0000 7f24939e6740 1 -- 172.21.15.16:0/1944138058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55859d0252f0 msgr2=0x55859d3f1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.591+0000 7f24939e6740 1 --2- 172.21.15.16:0/1944138058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55859d0252f0 0x55859d3f1de0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f2478009900 tx=0x7f2478032b20 comp rx=0 tx=0).stop 2024-08-31T06:19:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.591+0000 7f24939e6740 1 -- 172.21.15.16:0/1944138058 shutdown_connections 2024-08-31T06:19:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.591+0000 7f24939e6740 1 --2- 172.21.15.16:0/1944138058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55859d02d4d0 0x55859d3f2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.591+0000 7f24939e6740 1 --2- 172.21.15.16:0/1944138058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55859d0252f0 0x55859d3f1de0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.591+0000 7f24939e6740 1 --2- 172.21.15.16:0/1944138058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55859d04e1b0 0x55859d3f18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.591+0000 7f24939e6740 1 -- 172.21.15.16:0/1944138058 >> 172.21.15.16:0/1944138058 conn(0x55859d042300 msgr2=0x55859d3ef790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.591+0000 7f2485400640 1 --2- 172.21.15.16:0/3358060278 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f246c0b4ca0 0x7f246c0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:13.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.592+0000 7f24939e6740 1 -- 172.21.15.16:0/1944138058 shutdown_connections 2024-08-31T06:19:13.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.592+0000 7f24939e6740 1 -- 172.21.15.16:0/1944138058 wait complete. 2024-08-31T06:19:13.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.593+0000 7f24939e6740 1 -- start start 2024-08-31T06:19:13.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.593+0000 7f2485400640 1 --2- 172.21.15.16:0/3358060278 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f246c0b4ca0 0x7f246c0b71b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f24700095f0 tx=0x7f2470005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:13.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.593+0000 7f24939e6740 1 Processor -- start 2024-08-31T06:19:13.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.593+0000 7f24939e6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55859d0252f0 0x55859d3fcf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:13.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.593+0000 7f24939e6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55859d02d4d0 0x55859d3fd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:13.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.593+0000 7f24939e6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55859d04e1b0 0x55859d409310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:13.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.593+0000 7f24939e6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55859d28fab0 con 0x55859d0252f0 2024-08-31T06:19:13.597 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.593+0000 7f24939e6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55859d2fd320 con 0x55859d02d4d0 2024-08-31T06:19:13.597 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.593+0000 7f24939e6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55859d3fffc0 con 0x55859d04e1b0 2024-08-31T06:19:13.597 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.593+0000 7f24939e6740 10 client.0 _ll_register_callbacks cb 0x55859d336e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:19:13.597 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.594+0000 7f2485e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55859d0252f0 0x55859d3fcf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:13.597 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.594+0000 7f2485400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55859d02d4d0 0x55859d3fd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:13.597 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.594+0000 7f2485e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55859d0252f0 0x55859d3fcf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50410/0 (socket says 172.21.15.16:50410) 2024-08-31T06:19:13.597 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.594+0000 7f2485e00640 1 -- 172.21.15.16:0/2669290262 learned_addr learned my addr 172.21.15.16:0/2669290262 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:13.597 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.594+0000 7f2486800640 1 --2- 172.21.15.16:0/2669290262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55859d04e1b0 0x55859d409310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.594+0000 7f2485e00640 1 -- 172.21.15.16:0/2669290262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55859d04e1b0 msgr2=0x55859d409310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.594+0000 7f2485e00640 1 --2- 172.21.15.16:0/2669290262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55859d04e1b0 0x55859d409310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.594+0000 7f2485e00640 1 -- 172.21.15.16:0/2669290262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55859d02d4d0 msgr2=0x55859d3fd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.594+0000 7f2485e00640 1 --2- 172.21.15.16:0/2669290262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55859d02d4d0 0x55859d3fd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.594+0000 7f2485e00640 1 -- 172.21.15.16:0/2669290262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55859d0b9790 con 0x55859d0252f0 2024-08-31T06:19:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.594+0000 7f2486800640 1 --2- 172.21.15.16:0/2669290262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55859d04e1b0 0x55859d409310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:19:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.595+0000 7f2485e00640 1 --2- 172.21.15.16:0/2669290262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55859d0252f0 0x55859d3fcf10 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f247802a920 tx=0x7f2478004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a240be2e880b7890 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.595+0000 7f246b400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:19:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.595+0000 7f246b400640 1 -- 172.21.15.16:0/2669290262 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2478035070 con 0x55859d0252f0 2024-08-31T06:19:13.599 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.595+0000 7f246b400640 1 -- 172.21.15.16:0/2669290262 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8514+0+0 (secure 0 0 0) 0x7f24780051b0 con 0x55859d0252f0 2024-08-31T06:19:13.599 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.595+0000 7f24939e6740 1 -- 172.21.15.16:0/2669290262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2478004030 con 0x55859d0252f0 2024-08-31T06:19:13.599 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.596+0000 7f246b400640 1 -- 172.21.15.16:0/2669290262 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f2478040040 con 0x55859d0252f0 2024-08-31T06:19:13.599 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.596+0000 7f24939e6740 10 client.17934 fetch_fsmap learned FSMap version 48 2024-08-31T06:19:13.599 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.596+0000 7f24939e6740 1 -- 172.21.15.16:0/2669290262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55859d132620 con 0x55859d0252f0 2024-08-31T06:19:13.600 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.597+0000 7f246b400640 1 -- 172.21.15.16:0/2669290262 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2478004030 con 0x55859d0252f0 2024-08-31T06:19:13.600 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.597+0000 7f246b400640 1 -- 172.21.15.16:0/2669290262 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7f24780035f0 con 0x55859d0252f0 2024-08-31T06:19:13.600 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.598+0000 7f24939e6740 10 client.17934 fetch_fsmap finished waiting for FSMap version 48 2024-08-31T06:19:13.600 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.598+0000 7f24939e6740 10 client.17934 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:19:13.600 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.598+0000 7f24939e6740 20 client.17934 populate_metadata read hostname 'smithi016' 2024-08-31T06:19:13.600 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.598+0000 7f24939e6740 1 --2- 172.21.15.16:0/2669290262 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55859d00cd80 0x55859d40ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:13.600 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.598+0000 7f24939e6740 4 client.17934 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:19:13.601 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.598+0000 7f24939e6740 1 -- 172.21.15.16:0/2669290262 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55859d368ec0 con 0x55859d00cd80 2024-08-31T06:19:13.601 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.598+0000 7f2485400640 1 --2- 172.21.15.16:0/2669290262 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55859d00cd80 0x55859d40ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:13.602 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.599+0000 7f2485400640 1 --2- 172.21.15.16:0/2669290262 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55859d00cd80 0x55859d40ee80 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7f18826a47fc816 server_cookie=50a77d10856af199 in_seq=0 out_seq=0 2024-08-31T06:19:13.602 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.599+0000 7f246b400640 10 client.17934 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:19:13.602 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.600+0000 7f246b400640 1 -- 172.21.15.16:0/2669290262 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55859d368ec0 con 0x55859d00cd80 2024-08-31T06:19:13.603 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.600+0000 7f246b400640 10 client.17934 handle_command_reply: tid=0 2024-08-31T06:19:13.603 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.600+0000 7f24939e6740 1 client.17934 shutdown 2024-08-31T06:19:13.604 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.600+0000 7f24939e6740 1 -- 172.21.15.16:0/2669290262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55859d0252f0 msgr2=0x55859d3fcf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:13.604 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.600+0000 7f24939e6740 1 --2- 172.21.15.16:0/2669290262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55859d0252f0 0x55859d3fcf10 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f247802a920 tx=0x7f2478004410 comp rx=0 tx=0).stop 2024-08-31T06:19:13.604 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.600+0000 7f24939e6740 1 -- 172.21.15.16:0/2669290262 shutdown_connections 2024-08-31T06:19:13.604 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.600+0000 7f24939e6740 1 --2- 172.21.15.16:0/2669290262 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55859d00cd80 0x55859d40ee80 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:13.604 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.600+0000 7f24939e6740 1 --2- 172.21.15.16:0/2669290262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55859d04e1b0 0x55859d409310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:13.604 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.600+0000 7f24939e6740 1 --2- 172.21.15.16:0/2669290262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55859d02d4d0 0x55859d3fd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:13.604 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.600+0000 7f24939e6740 1 --2- 172.21.15.16:0/2669290262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55859d0252f0 0x55859d3fcf10 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:13.604 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.600+0000 7f24939e6740 1 -- 172.21.15.16:0/2669290262 >> 172.21.15.16:0/2669290262 conn(0x55859d042300 msgr2=0x55859d3f28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:13.604 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.601+0000 7f246b400640 0 client.17934 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:19:13.604 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.601+0000 7f24939e6740 1 -- 172.21.15.16:0/2669290262 shutdown_connections 2024-08-31T06:19:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.601+0000 7f24939e6740 1 -- 172.21.15.16:0/2669290262 wait complete. 2024-08-31T06:19:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.601+0000 7f24939e6740 20 client.17934 trim_cache size 0 max 16384 2024-08-31T06:19:13.626 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.623+0000 7f24939e6740 1 Processor -- start 2024-08-31T06:19:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.623+0000 7f24939e6740 1 -- start start 2024-08-31T06:19:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.623+0000 7f24939e6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55859d0252f0 0x55859d3f3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.623+0000 7f24939e6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55859d02d4d0 0x55859d3ee480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.623+0000 7f24939e6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55859d04e1b0 0x55859d3fa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.624+0000 7f24939e6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f24881c0a30 con 0x55859d04e1b0 2024-08-31T06:19:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.624+0000 7f24939e6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f24881be270 con 0x55859d0252f0 2024-08-31T06:19:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.624+0000 7f24939e6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f24881bc0e0 con 0x55859d02d4d0 2024-08-31T06:19:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.624+0000 7f2485e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55859d0252f0 0x55859d3f3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.624+0000 7f2485e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55859d0252f0 0x55859d3f3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51314/0 (socket says 172.21.15.16:51314) 2024-08-31T06:19:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.624+0000 7f2485e00640 1 -- 172.21.15.16:0/4164752512 learned_addr learned my addr 172.21.15.16:0/4164752512 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.624+0000 7f2485400640 1 --2- 172.21.15.16:0/4164752512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55859d02d4d0 0x55859d3ee480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.624+0000 7f2486800640 1 --2- 172.21.15.16:0/4164752512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55859d04e1b0 0x55859d3fa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.624+0000 7f2485400640 1 -- 172.21.15.16:0/4164752512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55859d0252f0 msgr2=0x55859d3f3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:13.628 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.624+0000 7f2485400640 1 --2- 172.21.15.16:0/4164752512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55859d0252f0 0x55859d3f3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:13.628 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.624+0000 7f2485400640 1 -- 172.21.15.16:0/4164752512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55859d04e1b0 msgr2=0x55859d3fa0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:13.628 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.624+0000 7f2485400640 1 --2- 172.21.15.16:0/4164752512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55859d04e1b0 0x55859d3fa0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:13.628 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.624+0000 7f2485400640 1 -- 172.21.15.16:0/4164752512 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55859d3ff880 con 0x55859d02d4d0 2024-08-31T06:19:13.628 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.625+0000 7f2485400640 1 --2- 172.21.15.16:0/4164752512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55859d02d4d0 0x55859d3ee480 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x55859d3ffff0 tx=0x7f2470014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e6b6658c06d8ec5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:13.629 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.625+0000 7f2476600640 1 -- 172.21.15.16:0/4164752512 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247002e070 con 0x55859d02d4d0 2024-08-31T06:19:13.629 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.625+0000 7f2476600640 1 -- 172.21.15.16:0/4164752512 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f247001f990 con 0x55859d02d4d0 2024-08-31T06:19:13.629 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.625+0000 7f2476600640 1 -- 172.21.15.16:0/4164752512 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247000f930 con 0x55859d02d4d0 2024-08-31T06:19:13.629 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.626+0000 7f24939e6740 1 -- 172.21.15.16:0/4164752512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55859d02d4d0 msgr2=0x55859d3ee480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:13.629 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.626+0000 7f24939e6740 1 --2- 172.21.15.16:0/4164752512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55859d02d4d0 0x55859d3ee480 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x55859d3ffff0 tx=0x7f2470014000 comp rx=0 tx=0).stop 2024-08-31T06:19:13.629 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.627+0000 7f24939e6740 1 -- 172.21.15.16:0/4164752512 shutdown_connections 2024-08-31T06:19:13.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.627+0000 7f24939e6740 1 --2- 172.21.15.16:0/4164752512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55859d02d4d0 0x55859d3ee480 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:13.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.627+0000 7f24939e6740 1 --2- 172.21.15.16:0/4164752512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55859d04e1b0 0x55859d3fa0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:13.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.627+0000 7f24939e6740 1 --2- 172.21.15.16:0/4164752512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55859d0252f0 0x55859d3f3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:13.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.627+0000 7f24939e6740 1 -- 172.21.15.16:0/4164752512 >> 172.21.15.16:0/4164752512 conn(0x55859d042300 msgr2=0x55859d3edd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:13.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.627+0000 7f24939e6740 1 -- 172.21.15.16:0/4164752512 shutdown_connections 2024-08-31T06:19:13.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.628+0000 7f24939e6740 1 -- 172.21.15.16:0/4164752512 wait complete. 2024-08-31T06:19:13.632 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.629+0000 7f24939e6740 1 -- start start 2024-08-31T06:19:13.632 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.630+0000 7f24939e6740 1 Processor -- start 2024-08-31T06:19:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.630+0000 7f24939e6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55859d0252f0 0x55859d411800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.630+0000 7f24939e6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55859d02d4d0 0x55859d411dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.631+0000 7f2485e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55859d0252f0 0x55859d411800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.631+0000 7f2485e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55859d0252f0 0x55859d411800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:58702/0 (socket says 172.21.15.16:58702) 2024-08-31T06:19:13.634 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.631+0000 7f2485400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55859d02d4d0 0x55859d411dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:13.634 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.631+0000 7f24939e6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55859d04e1b0 0x55859d471640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.631+0000 7f24939e6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f24881bc730 con 0x55859d02d4d0 2024-08-31T06:19:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.631+0000 7f24939e6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f24881bdba0 con 0x55859d04e1b0 2024-08-31T06:19:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.631+0000 7f24939e6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f24881bce00 con 0x55859d0252f0 2024-08-31T06:19:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.631+0000 7f24939e6740 10 client.0 _ll_register_callbacks cb 0x7f2478009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:19:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.632+0000 7f2485e00640 1 -- 172.21.15.16:0/485427322 learned_addr learned my addr 172.21.15.16:0/485427322 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.632+0000 7f2485e00640 1 -- 172.21.15.16:0/485427322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55859d04e1b0 msgr2=0x55859d471640 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T06:19:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.632+0000 7f2485e00640 1 --2- 172.21.15.16:0/485427322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55859d04e1b0 0x55859d471640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.632+0000 7f2485e00640 1 -- 172.21.15.16:0/485427322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55859d02d4d0 msgr2=0x55859d411dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.633+0000 7f2485e00640 1 --2- 172.21.15.16:0/485427322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55859d02d4d0 0x55859d411dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:13.636 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.633+0000 7f2485e00640 1 -- 172.21.15.16:0/485427322 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55859d471cb0 con 0x55859d0252f0 2024-08-31T06:19:13.636 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.633+0000 7f2485400640 1 --2- 172.21.15.16:0/485427322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55859d02d4d0 0x55859d411dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:19:13.636 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.633+0000 7f2485e00640 1 --2- 172.21.15.16:0/485427322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55859d0252f0 0x55859d411800 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2478003820 tx=0x7f2478003850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9044f53cfd6f389 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:13.636 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.633+0000 7f246b400640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:19:13.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.634+0000 7f246b400640 1 -- 172.21.15.16:0/485427322 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2478035040 con 0x55859d0252f0 2024-08-31T06:19:13.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.634+0000 7f246b400640 1 -- 172.21.15.16:0/485427322 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8514+0+0 (secure 0 0 0) 0x7f2478032e20 con 0x55859d0252f0 2024-08-31T06:19:13.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.634+0000 7f24939e6740 1 -- 172.21.15.16:0/485427322 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f247001f990 con 0x55859d0252f0 2024-08-31T06:19:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.634+0000 7f246b400640 1 -- 172.21.15.16:0/485427322 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f2478003be0 con 0x55859d0252f0 2024-08-31T06:19:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.634+0000 7f24939e6740 10 client.28336 fetch_fsmap learned FSMap version 48 2024-08-31T06:19:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.634+0000 7f24939e6740 1 -- 172.21.15.16:0/485427322 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55859d478fe0 con 0x55859d0252f0 2024-08-31T06:19:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.635+0000 7f246b400640 1 -- 172.21.15.16:0/485427322 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f247001f990 con 0x55859d0252f0 2024-08-31T06:19:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.635+0000 7f246b400640 1 -- 172.21.15.16:0/485427322 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7f2478032030 con 0x55859d0252f0 2024-08-31T06:19:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.635+0000 7f24939e6740 10 client.28336 fetch_fsmap finished waiting for FSMap version 48 2024-08-31T06:19:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.635+0000 7f24939e6740 10 client.28336 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:19:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.635+0000 7f24939e6740 20 client.28336 populate_metadata read hostname 'smithi016' 2024-08-31T06:19:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.635+0000 7f24939e6740 1 --2- 172.21.15.16:0/485427322 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55859cfe21c0 0x55859d47b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.635+0000 7f24939e6740 4 client.28336 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:19:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.635+0000 7f24939e6740 1 -- 172.21.15.16:0/485427322 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55859d364960 con 0x55859cfe21c0 2024-08-31T06:19:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.636+0000 7f2485400640 1 --2- 172.21.15.16:0/485427322 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55859cfe21c0 0x55859d47b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.637+0000 7f2485400640 1 --2- 172.21.15.16:0/485427322 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55859cfe21c0 0x55859d47b850 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5a34aa1b0679266 server_cookie=bb2c4cd5c622850b in_seq=0 out_seq=0 2024-08-31T06:19:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.637+0000 7f246b400640 10 client.28336 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:19:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.666+0000 7f246b400640 1 -- 172.21.15.16:0/485427322 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55859d364960 con 0x55859cfe21c0 2024-08-31T06:19:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.666+0000 7f246b400640 10 client.28336 handle_command_reply: tid=0 2024-08-31T06:19:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.666+0000 7f24939e6740 1 client.28336 shutdown 2024-08-31T06:19:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.667+0000 7f24939e6740 1 -- 172.21.15.16:0/485427322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55859d0252f0 msgr2=0x55859d411800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.667+0000 7f24939e6740 1 --2- 172.21.15.16:0/485427322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55859d0252f0 0x55859d411800 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2478003820 tx=0x7f2478003850 comp rx=0 tx=0).stop 2024-08-31T06:19:13.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.667+0000 7f24939e6740 1 -- 172.21.15.16:0/485427322 shutdown_connections 2024-08-31T06:19:13.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.667+0000 7f24939e6740 1 --2- 172.21.15.16:0/485427322 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55859cfe21c0 0x55859d47b850 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:13.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.668+0000 7f24939e6740 1 --2- 172.21.15.16:0/485427322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55859d04e1b0 0x55859d471640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:13.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.668+0000 7f246b400640 0 client.28336 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:19:13.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.668+0000 7f24939e6740 1 --2- 172.21.15.16:0/485427322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55859d02d4d0 0x55859d411dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:13.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.668+0000 7f24939e6740 1 --2- 172.21.15.16:0/485427322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55859d0252f0 0x55859d411800 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:13.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.668+0000 7f24939e6740 1 -- 172.21.15.16:0/485427322 >> 172.21.15.16:0/485427322 conn(0x55859d042300 msgr2=0x55859d3ee500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.668+0000 7f24939e6740 1 -- 172.21.15.16:0/485427322 shutdown_connections 2024-08-31T06:19:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.668+0000 7f24939e6740 1 -- 172.21.15.16:0/485427322 wait complete. 2024-08-31T06:19:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.669+0000 7f24939e6740 20 client.28336 trim_cache size 0 max 16384 2024-08-31T06:19:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:19:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:19:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:19:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:19:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.670+0000 7f2487200640 1 -- 172.21.15.16:0/3358060278 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f246c0b4ca0 msgr2=0x7f246c0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.670+0000 7f2487200640 1 --2- 172.21.15.16:0/3358060278 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f246c0b4ca0 0x7f246c0b71b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f24700095f0 tx=0x7f2470005f90 comp rx=0 tx=0).stop 2024-08-31T06:19:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.670+0000 7f2487200640 1 -- 172.21.15.16:0/3358060278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f248810d940 msgr2=0x7f248810b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.670+0000 7f2487200640 1 --2- 172.21.15.16:0/3358060278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f248810d940 0x7f248810b110 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f248000efd0 tx=0x7f248000c6a0 comp rx=0 tx=0).stop 2024-08-31T06:19:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.671+0000 7f2487200640 1 -- 172.21.15.16:0/3358060278 shutdown_connections 2024-08-31T06:19:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.671+0000 7f2487200640 1 --2- 172.21.15.16:0/3358060278 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f246c0b4ca0 0x7f246c0b71b0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.671+0000 7f2487200640 1 --2- 172.21.15.16:0/3358060278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f248810d940 0x7f248810b110 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.671+0000 7f2487200640 1 --2- 172.21.15.16:0/3358060278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2488104e30 0x7f248810abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.671+0000 7f2487200640 1 --2- 172.21.15.16:0/3358060278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24881044e0 0x7f248810a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.671+0000 7f2487200640 1 -- 172.21.15.16:0/3358060278 >> 172.21.15.16:0/3358060278 conn(0x7f24880fdfd0 msgr2=0x7f2488114e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.671+0000 7f2487200640 1 -- 172.21.15.16:0/3358060278 shutdown_connections 2024-08-31T06:19:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:13.671+0000 7f2487200640 1 -- 172.21.15.16:0/3358060278 wait complete. 2024-08-31T06:19:14.054 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:19:14.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:13 smithi155 ceph-mon[21742]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:19:14.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:13 smithi155 ceph-mon[21742]: pgmap v737: 97 pgs: 97 active+clean; 3.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 18 MiB/s wr, 1.56k op/s 2024-08-31T06:19:14.167 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:13 smithi016 ceph-mon[17194]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:19:14.167 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:13 smithi016 ceph-mon[17194]: pgmap v737: 97 pgs: 97 active+clean; 3.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 18 MiB/s wr, 1.56k op/s 2024-08-31T06:19:14.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:13 smithi184 ceph-mon[22331]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:19:14.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:13 smithi184 ceph-mon[22331]: pgmap v737: 97 pgs: 97 active+clean; 3.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 18 MiB/s wr, 1.56k op/s 2024-08-31T06:19:14.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.902+0000 7f29d3200640 1 -- 172.21.15.16:0/1387040334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29cc104750 msgr2=0x7f29cc104b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:14.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.902+0000 7f29d3200640 1 --2- 172.21.15.16:0/1387040334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29cc104750 0x7f29cc104b60 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f29bc009900 tx=0x7f29bc02a3f0 comp rx=0 tx=0).stop 2024-08-31T06:19:14.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.902+0000 7f29d3200640 1 -- 172.21.15.16:0/1387040334 shutdown_connections 2024-08-31T06:19:14.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.902+0000 7f29d3200640 1 --2- 172.21.15.16:0/1387040334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29cc10dbb0 0x7f29cc1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:14.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.902+0000 7f29d3200640 1 --2- 172.21.15.16:0/1387040334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29cc1050a0 0x7f29cc10d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:14.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.902+0000 7f29d3200640 1 --2- 172.21.15.16:0/1387040334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29cc104750 0x7f29cc104b60 secure :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f29bc009900 tx=0x7f29bc02a3f0 comp rx=0 tx=0).stop 2024-08-31T06:19:14.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.902+0000 7f29d3200640 1 -- 172.21.15.16:0/1387040334 >> 172.21.15.16:0/1387040334 conn(0x7f29cc0794d0 msgr2=0x7f29cc0798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:14.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.903+0000 7f29d3200640 1 -- 172.21.15.16:0/1387040334 shutdown_connections 2024-08-31T06:19:14.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.904+0000 7f29d3200640 1 -- 172.21.15.16:0/1387040334 wait complete. 2024-08-31T06:19:14.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.906+0000 7f29d3200640 1 Processor -- start 2024-08-31T06:19:14.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.906+0000 7f29d3200640 1 -- start start 2024-08-31T06:19:14.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.907+0000 7f29d3200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29cc1050a0 0x7f29cc102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:14.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.907+0000 7f29d3200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29cc10dbb0 0x7f29cc1029d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:14.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.907+0000 7f29d3200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29cc102f10 0x7f29cc1a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:14.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.907+0000 7f29d3200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f29cc113860 con 0x7f29cc1050a0 2024-08-31T06:19:14.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.907+0000 7f29d3200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f29cc1136e0 con 0x7f29cc102f10 2024-08-31T06:19:14.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.907+0000 7f29d3200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f29cc1139e0 con 0x7f29cc10dbb0 2024-08-31T06:19:14.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.908+0000 7f29cbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29cc1050a0 0x7f29cc102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:14.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.908+0000 7f29cbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29cc1050a0 0x7f29cc102490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50446/0 (socket says 172.21.15.16:50446) 2024-08-31T06:19:14.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.908+0000 7f29cbe00640 1 -- 172.21.15.16:0/4061110610 learned_addr learned my addr 172.21.15.16:0/4061110610 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:14.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.908+0000 7f29cb400640 1 --2- 172.21.15.16:0/4061110610 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29cc10dbb0 0x7f29cc1029d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:14.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.908+0000 7f29d1200640 1 --2- 172.21.15.16:0/4061110610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29cc102f10 0x7f29cc1a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:14.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.909+0000 7f29d1200640 1 -- 172.21.15.16:0/4061110610 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29cc10dbb0 msgr2=0x7f29cc1029d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:14.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.909+0000 7f29d1200640 1 --2- 172.21.15.16:0/4061110610 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29cc10dbb0 0x7f29cc1029d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:14.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.909+0000 7f29d1200640 1 -- 172.21.15.16:0/4061110610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29cc1050a0 msgr2=0x7f29cc102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:14.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.909+0000 7f29d1200640 1 --2- 172.21.15.16:0/4061110610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29cc1050a0 0x7f29cc102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:14.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.909+0000 7f29d1200640 1 -- 172.21.15.16:0/4061110610 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29cc19a020 con 0x7f29cc102f10 2024-08-31T06:19:14.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.909+0000 7f29cbe00640 1 --2- 172.21.15.16:0/4061110610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29cc1050a0 0x7f29cc102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:19:14.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.909+0000 7f29cb400640 1 --2- 172.21.15.16:0/4061110610 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29cc10dbb0 0x7f29cc1029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:19:14.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.910+0000 7f29d1200640 1 --2- 172.21.15.16:0/4061110610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29cc102f10 0x7f29cc1a84a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f29c400ea80 tx=0x7f29c400ef40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:14.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.910+0000 7f29c8c00640 1 -- 172.21.15.16:0/4061110610 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c400cea0 con 0x7f29cc102f10 2024-08-31T06:19:14.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.910+0000 7f29c8c00640 1 -- 172.21.15.16:0/4061110610 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29c4005020 con 0x7f29cc102f10 2024-08-31T06:19:14.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.910+0000 7f29c8c00640 1 -- 172.21.15.16:0/4061110610 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c4010690 con 0x7f29cc102f10 2024-08-31T06:19:14.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.910+0000 7f29d3200640 1 -- 172.21.15.16:0/4061110610 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29cc19a310 con 0x7f29cc102f10 2024-08-31T06:19:14.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.911+0000 7f29d3200640 1 -- 172.21.15.16:0/4061110610 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29cc19a800 con 0x7f29cc102f10 2024-08-31T06:19:14.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.915+0000 7f29d9ec1740 1 Processor -- start 2024-08-31T06:19:14.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.917+0000 7f29c8c00640 1 -- 172.21.15.16:0/4061110610 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f29c40054d0 con 0x7f29cc102f10 2024-08-31T06:19:14.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.917+0000 7f29d9ec1740 1 -- start start 2024-08-31T06:19:14.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.918+0000 7f29d9ec1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ae6add070 0x556ae6e82990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:14.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.919+0000 7f29cb400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ae6add070 0x556ae6e82990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:14.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.919+0000 7f29cb400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ae6add070 0x556ae6e82990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51352/0 (socket says 172.21.15.16:51352) 2024-08-31T06:19:14.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.920+0000 7f29c8c00640 1 --2- 172.21.15.16:0/4061110610 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f29b00b4df0 0x7f29b00b7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:14.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.920+0000 7f29c8c00640 1 -- 172.21.15.16:0/4061110610 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8514+0+0 (secure 0 0 0) 0x7f29c40e5bc0 con 0x7f29cc102f10 2024-08-31T06:19:14.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.921+0000 7f29cbe00640 1 --2- 172.21.15.16:0/4061110610 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f29b00b4df0 0x7f29b00b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:14.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.921+0000 7f29cb400640 1 -- 172.21.15.16:0/2515161570 learned_addr learned my addr 172.21.15.16:0/2515161570 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:14.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.921+0000 7f29d9ec1740 1 --2- 172.21.15.16:0/2515161570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556ae6ab41b0 0x556ae6e82ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:14.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.921+0000 7f29d9ec1740 1 --2- 172.21.15.16:0/2515161570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae6abc390 0x556ae6e83410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:14.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.921+0000 7f29d9ec1740 1 -- 172.21.15.16:0/2515161570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556ae6df97b0 con 0x556ae6abc390 2024-08-31T06:19:14.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.921+0000 7f29d9ec1740 1 -- 172.21.15.16:0/2515161570 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556ae6de8d40 con 0x556ae6add070 2024-08-31T06:19:14.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.921+0000 7f29d9ec1740 1 -- 172.21.15.16:0/2515161570 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556ae6deccf0 con 0x556ae6ab41b0 2024-08-31T06:19:14.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.922+0000 7f29cb400640 1 --2- 172.21.15.16:0/2515161570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556ae6ab41b0 0x556ae6e82ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:14.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.922+0000 7f29d1200640 1 --2- 172.21.15.16:0/2515161570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae6abc390 0x556ae6e83410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:14.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.922+0000 7f29cb400640 1 -- 172.21.15.16:0/2515161570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556ae6ab41b0 msgr2=0x556ae6e82ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:14.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.922+0000 7f29cb400640 1 --2- 172.21.15.16:0/2515161570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556ae6ab41b0 0x556ae6e82ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:14.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.922+0000 7f29cb400640 1 -- 172.21.15.16:0/2515161570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae6abc390 msgr2=0x556ae6e83410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:14.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.922+0000 7f29cb400640 1 --2- 172.21.15.16:0/2515161570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae6abc390 0x556ae6e83410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:14.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.922+0000 7f29cb400640 1 -- 172.21.15.16:0/2515161570 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ae6b48650 con 0x556ae6add070 2024-08-31T06:19:14.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.922+0000 7f29cbe00640 1 --2- 172.21.15.16:0/4061110610 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f29b00b4df0 0x7f29b00b7300 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f29cc103ee0 tx=0x7f29bc005a40 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:14.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.922+0000 7f29cb400640 1 --2- 172.21.15.16:0/2515161570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ae6add070 0x556ae6e82990 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f29b8015580 tx=0x7f29b80098f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f20f5febd68b48a6 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:14.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.923+0000 7f29c1600640 1 -- 172.21.15.16:0/2515161570 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b801b020 con 0x556ae6add070 2024-08-31T06:19:14.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.923+0000 7f29c1600640 1 -- 172.21.15.16:0/2515161570 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29b80035f0 con 0x556ae6add070 2024-08-31T06:19:14.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.923+0000 7f29c1600640 1 -- 172.21.15.16:0/2515161570 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b80044e0 con 0x556ae6add070 2024-08-31T06:19:14.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.924+0000 7f29d9ec1740 1 -- 172.21.15.16:0/2515161570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ae6add070 msgr2=0x556ae6e82990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:14.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.924+0000 7f29d9ec1740 1 --2- 172.21.15.16:0/2515161570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ae6add070 0x556ae6e82990 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f29b8015580 tx=0x7f29b80098f0 comp rx=0 tx=0).stop 2024-08-31T06:19:14.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.925+0000 7f29d9ec1740 1 -- 172.21.15.16:0/2515161570 shutdown_connections 2024-08-31T06:19:14.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.925+0000 7f29d9ec1740 1 --2- 172.21.15.16:0/2515161570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556ae6ab41b0 0x556ae6e82ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:14.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.925+0000 7f29d9ec1740 1 --2- 172.21.15.16:0/2515161570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae6abc390 0x556ae6e83410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:14.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.925+0000 7f29d9ec1740 1 --2- 172.21.15.16:0/2515161570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ae6add070 0x556ae6e82990 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:14.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.925+0000 7f29d9ec1740 1 -- 172.21.15.16:0/2515161570 >> 172.21.15.16:0/2515161570 conn(0x556ae6ad1460 msgr2=0x556ae6e80880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:14.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.927+0000 7f29d9ec1740 1 -- 172.21.15.16:0/2515161570 shutdown_connections 2024-08-31T06:19:14.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.928+0000 7f29d9ec1740 1 -- 172.21.15.16:0/2515161570 wait complete. 2024-08-31T06:19:14.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.928+0000 7f29d9ec1740 1 -- start start 2024-08-31T06:19:14.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.929+0000 7f29d9ec1740 1 Processor -- start 2024-08-31T06:19:14.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.931+0000 7f29d9ec1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ae6ab41b0 0x556ae6e8e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:14.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.931+0000 7f29d9ec1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556ae6abc390 0x556ae6e8ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:14.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.931+0000 7f29d9ec1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae6add070 0x556ae6e9a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:14.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.931+0000 7f29d9ec1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556ae6c4b700 con 0x556ae6add070 2024-08-31T06:19:14.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.931+0000 7f29d9ec1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556ae6d6fcf0 con 0x556ae6ab41b0 2024-08-31T06:19:14.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.931+0000 7f29d9ec1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556ae6e91170 con 0x556ae6abc390 2024-08-31T06:19:14.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.931+0000 7f29d9ec1740 10 client.0 _ll_register_callbacks cb 0x556ae6d95470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:19:14.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.931+0000 7f29d1200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae6add070 0x556ae6e9a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:14.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.931+0000 7f29d1200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae6add070 0x556ae6e9a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50462/0 (socket says 172.21.15.16:50462) 2024-08-31T06:19:14.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.931+0000 7f29d1200640 1 -- 172.21.15.16:0/2492706837 learned_addr learned my addr 172.21.15.16:0/2492706837 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:14.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.932+0000 7f29cbe00640 1 --2- 172.21.15.16:0/2492706837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ae6ab41b0 0x556ae6e8e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:14.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.932+0000 7f29cb400640 1 --2- 172.21.15.16:0/2492706837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556ae6abc390 0x556ae6e8ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:14.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.932+0000 7f29cb400640 1 -- 172.21.15.16:0/2492706837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ae6ab41b0 msgr2=0x556ae6e8e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:14.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.932+0000 7f29cb400640 1 --2- 172.21.15.16:0/2492706837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ae6ab41b0 0x556ae6e8e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:14.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.932+0000 7f29cb400640 1 -- 172.21.15.16:0/2492706837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae6add070 msgr2=0x556ae6e9a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:14.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.933+0000 7f29cb400640 1 --2- 172.21.15.16:0/2492706837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae6add070 0x556ae6e9a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:14.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.933+0000 7f29cb400640 1 -- 172.21.15.16:0/2492706837 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ae6db08b0 con 0x556ae6abc390 2024-08-31T06:19:14.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.933+0000 7f29d1200640 1 --2- 172.21.15.16:0/2492706837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae6add070 0x556ae6e9a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:19:14.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.933+0000 7f29cb400640 1 --2- 172.21.15.16:0/2492706837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556ae6abc390 0x556ae6e8ebf0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f29b8015550 tx=0x7f29b8004230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d25fef1767178b93 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:14.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.933+0000 7f29b6000640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:19:14.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.934+0000 7f29b6000640 1 -- 172.21.15.16:0/2492706837 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b80044e0 con 0x556ae6abc390 2024-08-31T06:19:14.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.934+0000 7f29b6000640 1 -- 172.21.15.16:0/2492706837 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8514+0+0 (secure 0 0 0) 0x7f29b8003bb0 con 0x556ae6abc390 2024-08-31T06:19:14.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.934+0000 7f29d9ec1740 1 -- 172.21.15.16:0/2492706837 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29b80035f0 con 0x556ae6abc390 2024-08-31T06:19:14.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.934+0000 7f29b6000640 1 -- 172.21.15.16:0/2492706837 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f29b8005040 con 0x556ae6abc390 2024-08-31T06:19:14.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.934+0000 7f29d9ec1740 10 client.28348 fetch_fsmap learned FSMap version 48 2024-08-31T06:19:14.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.934+0000 7f29d9ec1740 1 -- 172.21.15.16:0/2492706837 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x556ae6e9d230 con 0x556ae6abc390 2024-08-31T06:19:14.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.935+0000 7f29b6000640 1 -- 172.21.15.16:0/2492706837 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29b80035f0 con 0x556ae6abc390 2024-08-31T06:19:14.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.935+0000 7f29b6000640 1 -- 172.21.15.16:0/2492706837 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7f29b8004c40 con 0x556ae6abc390 2024-08-31T06:19:14.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.935+0000 7f29d9ec1740 10 client.28348 fetch_fsmap finished waiting for FSMap version 48 2024-08-31T06:19:14.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.935+0000 7f29d9ec1740 10 client.28348 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-31T06:19:14.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.935+0000 7f29d9ec1740 20 client.28348 populate_metadata read hostname 'smithi016' 2024-08-31T06:19:14.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.935+0000 7f29d9ec1740 1 --2- 172.21.15.16:0/2492706837 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x556ae6a9b9f0 0x556ae6ea0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:14.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.935+0000 7f29d9ec1740 4 client.28348 mds_command: new command op to 24724 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:19:14.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.935+0000 7f29d9ec1740 1 -- 172.21.15.16:0/2492706837 --> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ae6dfb700 con 0x556ae6a9b9f0 2024-08-31T06:19:14.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.936+0000 7f29cbe00640 1 --2- 172.21.15.16:0/2492706837 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x556ae6a9b9f0 0x556ae6ea0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:14.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.938+0000 7f29cbe00640 1 --2- 172.21.15.16:0/2492706837 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x556ae6a9b9f0 0x556ae6ea0590 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7824c3028a6c69b4 server_cookie=11e4621f0a3a8df7 in_seq=0 out_seq=0 2024-08-31T06:19:14.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.938+0000 7f29b6000640 10 client.28348 ms_handle_connect on v2:172.21.15.16:6838/219664460 2024-08-31T06:19:14.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.939+0000 7f29b6000640 1 -- 172.21.15.16:0/2492706837 <== mds.1 v2:172.21.15.16:6838/219664460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556ae6dfb700 con 0x556ae6a9b9f0 2024-08-31T06:19:14.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.939+0000 7f29b6000640 10 client.28348 handle_command_reply: tid=0 2024-08-31T06:19:14.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.939+0000 7f29d9ec1740 1 client.28348 shutdown 2024-08-31T06:19:14.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.941+0000 7f29d9ec1740 1 -- 172.21.15.16:0/2492706837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556ae6abc390 msgr2=0x556ae6e8ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:14.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.941+0000 7f29d9ec1740 1 --2- 172.21.15.16:0/2492706837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556ae6abc390 0x556ae6e8ebf0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f29b8015550 tx=0x7f29b8004230 comp rx=0 tx=0).stop 2024-08-31T06:19:14.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.941+0000 7f29d9ec1740 1 -- 172.21.15.16:0/2492706837 shutdown_connections 2024-08-31T06:19:14.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.941+0000 7f29d9ec1740 1 --2- 172.21.15.16:0/2492706837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556ae6abc390 0x556ae6e8ebf0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:14.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.941+0000 7f29d9ec1740 1 --2- 172.21.15.16:0/2492706837 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x556ae6a9b9f0 0x556ae6ea0590 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:14.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.941+0000 7f29d9ec1740 1 --2- 172.21.15.16:0/2492706837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae6add070 0x556ae6e9a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:14.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.941+0000 7f29d9ec1740 1 --2- 172.21.15.16:0/2492706837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ae6ab41b0 0x556ae6e8e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:14.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.941+0000 7f29d9ec1740 1 -- 172.21.15.16:0/2492706837 >> 172.21.15.16:0/2492706837 conn(0x556ae6ad1460 msgr2=0x556ae6e85b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:14.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.941+0000 7f29b6000640 0 client.28348 ms_handle_reset on v2:172.21.15.16:6838/219664460 2024-08-31T06:19:14.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.942+0000 7f29d9ec1740 1 -- 172.21.15.16:0/2492706837 shutdown_connections 2024-08-31T06:19:14.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.942+0000 7f29d9ec1740 1 -- 172.21.15.16:0/2492706837 wait complete. 2024-08-31T06:19:14.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.942+0000 7f29d9ec1740 20 client.28348 trim_cache size 0 max 16384 2024-08-31T06:19:14.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.965+0000 7f29d9ec1740 1 Processor -- start 2024-08-31T06:19:14.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.965+0000 7f29d9ec1740 1 -- start start 2024-08-31T06:19:14.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.965+0000 7f29d9ec1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae6ab41b0 0x556ae6e881b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:14.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.965+0000 7f29d9ec1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556ae6abc390 0x556ae6e9a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:14.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.965+0000 7f29d9ec1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ae6add070 0x556ae6e9a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:14.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.965+0000 7f29d9ec1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f29cc1b89f0 con 0x556ae6ab41b0 2024-08-31T06:19:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.965+0000 7f29d9ec1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f29cc1b6230 con 0x556ae6add070 2024-08-31T06:19:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.965+0000 7f29d9ec1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f29cc1b40c0 con 0x556ae6abc390 2024-08-31T06:19:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.966+0000 7f29cb400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556ae6abc390 0x556ae6e9a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.966+0000 7f29cb400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556ae6abc390 0x556ae6e9a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:58748/0 (socket says 172.21.15.16:58748) 2024-08-31T06:19:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.966+0000 7f29cb400640 1 -- 172.21.15.16:0/3097382682 learned_addr learned my addr 172.21.15.16:0/3097382682 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:14.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.966+0000 7f29cb400640 1 -- 172.21.15.16:0/3097382682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ae6add070 msgr2=0x556ae6e9a910 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T06:19:14.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.966+0000 7f29cb400640 1 --2- 172.21.15.16:0/3097382682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ae6add070 0x556ae6e9a910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:14.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.966+0000 7f29cb400640 1 -- 172.21.15.16:0/3097382682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae6ab41b0 msgr2=0x556ae6e881b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:19:14.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.966+0000 7f29cbe00640 1 --2- 172.21.15.16:0/3097382682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae6ab41b0 0x556ae6e881b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:14.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.966+0000 7f29cb400640 1 --2- 172.21.15.16:0/3097382682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae6ab41b0 0x556ae6e881b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:14.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.966+0000 7f29cb400640 1 -- 172.21.15.16:0/3097382682 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ae6e84c80 con 0x556ae6abc390 2024-08-31T06:19:14.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.967+0000 7f29cb400640 1 --2- 172.21.15.16:0/3097382682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556ae6abc390 0x556ae6e9a3d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f29b8011710 tx=0x7f29b8011740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f34c539e62de1304 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:14.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.967+0000 7f29c0c00640 1 -- 172.21.15.16:0/3097382682 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b801b040 con 0x556ae6abc390 2024-08-31T06:19:14.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.967+0000 7f29c0c00640 1 -- 172.21.15.16:0/3097382682 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29b80097b0 con 0x556ae6abc390 2024-08-31T06:19:14.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.967+0000 7f29c0c00640 1 -- 172.21.15.16:0/3097382682 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b80039b0 con 0x556ae6abc390 2024-08-31T06:19:14.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.968+0000 7f29d9ec1740 1 -- 172.21.15.16:0/3097382682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556ae6abc390 msgr2=0x556ae6e9a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:14.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.968+0000 7f29d9ec1740 1 --2- 172.21.15.16:0/3097382682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556ae6abc390 0x556ae6e9a3d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f29b8011710 tx=0x7f29b8011740 comp rx=0 tx=0).stop 2024-08-31T06:19:14.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.968+0000 7f29d9ec1740 1 -- 172.21.15.16:0/3097382682 shutdown_connections 2024-08-31T06:19:14.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.968+0000 7f29d9ec1740 1 --2- 172.21.15.16:0/3097382682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556ae6abc390 0x556ae6e9a3d0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:14.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.968+0000 7f29d9ec1740 1 --2- 172.21.15.16:0/3097382682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ae6add070 0x556ae6e9a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:14.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.968+0000 7f29d9ec1740 1 --2- 172.21.15.16:0/3097382682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae6ab41b0 0x556ae6e881b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:14.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.969+0000 7f29d9ec1740 1 -- 172.21.15.16:0/3097382682 >> 172.21.15.16:0/3097382682 conn(0x556ae6ad1460 msgr2=0x556ae6e87aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:14.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.969+0000 7f29d9ec1740 1 -- 172.21.15.16:0/3097382682 shutdown_connections 2024-08-31T06:19:14.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.969+0000 7f29d9ec1740 1 -- 172.21.15.16:0/3097382682 wait complete. 2024-08-31T06:19:14.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.969+0000 7f29d9ec1740 1 -- start start 2024-08-31T06:19:14.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.969+0000 7f29d9ec1740 1 Processor -- start 2024-08-31T06:19:14.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.970+0000 7f29d9ec1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ae6ab41b0 0x556ae6ea3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:14.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.970+0000 7f29d9ec1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556ae6abc390 0x556ae6ea1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:14.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.970+0000 7f29d9ec1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae6add070 0x556ae6ea1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:14.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.970+0000 7f29d9ec1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f29cc1b4710 con 0x556ae6add070 2024-08-31T06:19:14.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.970+0000 7f29d9ec1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f29cc1b5b60 con 0x556ae6ab41b0 2024-08-31T06:19:14.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.970+0000 7f29d9ec1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f29cc1b4dc0 con 0x556ae6abc390 2024-08-31T06:19:14.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.970+0000 7f29d9ec1740 10 client.0 _ll_register_callbacks cb 0x7f29b8009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:19:14.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.970+0000 7f29cb400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556ae6abc390 0x556ae6ea1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:14.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.970+0000 7f29cbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ae6ab41b0 0x556ae6ea3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:14.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.970+0000 7f29cb400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556ae6abc390 0x556ae6ea1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:58762/0 (socket says 172.21.15.16:58762) 2024-08-31T06:19:14.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.970+0000 7f29cbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ae6ab41b0 0x556ae6ea3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51376/0 (socket says 172.21.15.16:51376) 2024-08-31T06:19:14.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.970+0000 7f29cb400640 1 -- 172.21.15.16:0/212888065 learned_addr learned my addr 172.21.15.16:0/212888065 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:14.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.971+0000 7f29d1200640 1 --2- 172.21.15.16:0/212888065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae6add070 0x556ae6ea1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:14.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.971+0000 7f29cbe00640 1 -- 172.21.15.16:0/212888065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556ae6abc390 msgr2=0x556ae6ea1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:14.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.971+0000 7f29cbe00640 1 --2- 172.21.15.16:0/212888065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556ae6abc390 0x556ae6ea1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:14.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.971+0000 7f29cbe00640 1 -- 172.21.15.16:0/212888065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae6add070 msgr2=0x556ae6ea1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:14.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.971+0000 7f29cbe00640 1 --2- 172.21.15.16:0/212888065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae6add070 0x556ae6ea1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:14.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.971+0000 7f29cbe00640 1 -- 172.21.15.16:0/212888065 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ae6ea1e90 con 0x556ae6ab41b0 2024-08-31T06:19:14.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.971+0000 7f29cb400640 1 --2- 172.21.15.16:0/212888065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556ae6abc390 0x556ae6ea1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:19:14.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.971+0000 7f29cbe00640 1 --2- 172.21.15.16:0/212888065 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ae6ab41b0 0x556ae6ea3d40 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f29bc02a9e0 tx=0x7f29bc02a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e337179ddf06273a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:14.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.971+0000 7f29b6000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:19:14.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.972+0000 7f29b6000640 1 -- 172.21.15.16:0/212888065 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29bc056070 con 0x556ae6ab41b0 2024-08-31T06:19:14.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.972+0000 7f29d9ec1740 1 -- 172.21.15.16:0/212888065 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29b80097b0 con 0x556ae6ab41b0 2024-08-31T06:19:14.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.972+0000 7f29b6000640 1 -- 172.21.15.16:0/212888065 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8514+0+0 (secure 0 0 0) 0x7f29bc0314a0 con 0x556ae6ab41b0 2024-08-31T06:19:14.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.972+0000 7f29b6000640 1 -- 172.21.15.16:0/212888065 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f29bc062050 con 0x556ae6ab41b0 2024-08-31T06:19:14.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.972+0000 7f29d9ec1740 10 client.28334 fetch_fsmap learned FSMap version 48 2024-08-31T06:19:14.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.972+0000 7f29d9ec1740 1 -- 172.21.15.16:0/212888065 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x556ae6f0ac90 con 0x556ae6ab41b0 2024-08-31T06:19:14.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.972+0000 7f29b6000640 1 -- 172.21.15.16:0/212888065 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29b80097b0 con 0x556ae6ab41b0 2024-08-31T06:19:14.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.972+0000 7f29b6000640 1 -- 172.21.15.16:0/212888065 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7f29bc03c4d0 con 0x556ae6ab41b0 2024-08-31T06:19:14.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.972+0000 7f29d9ec1740 10 client.28334 fetch_fsmap finished waiting for FSMap version 48 2024-08-31T06:19:14.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.972+0000 7f29d9ec1740 10 client.28334 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-31T06:19:14.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.973+0000 7f29d9ec1740 20 client.28334 populate_metadata read hostname 'smithi016' 2024-08-31T06:19:14.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.973+0000 7f29d9ec1740 1 --2- 172.21.15.16:0/212888065 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x556ae6a70da0 0x556ae6f0d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:14.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.973+0000 7f29d9ec1740 4 client.28334 mds_command: new command op to 24724 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:19:14.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.973+0000 7f29d9ec1740 1 -- 172.21.15.16:0/212888065 --> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556ae6df71a0 con 0x556ae6a70da0 2024-08-31T06:19:14.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.973+0000 7f29cb400640 1 --2- 172.21.15.16:0/212888065 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x556ae6a70da0 0x556ae6f0d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:14.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.973+0000 7f29cb400640 1 --2- 172.21.15.16:0/212888065 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x556ae6a70da0 0x556ae6f0d430 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=edf8b9d31100763a server_cookie=5e8dccf5e4e088b2 in_seq=0 out_seq=0 2024-08-31T06:19:14.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.973+0000 7f29b6000640 10 client.28334 ms_handle_connect on v2:172.21.15.16:6838/219664460 2024-08-31T06:19:14.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.974+0000 7f29b6000640 1 -- 172.21.15.16:0/212888065 <== mds.1 v2:172.21.15.16:6838/219664460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556ae6df71a0 con 0x556ae6a70da0 2024-08-31T06:19:14.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.974+0000 7f29b6000640 10 client.28334 handle_command_reply: tid=0 2024-08-31T06:19:14.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.974+0000 7f29d9ec1740 1 client.28334 shutdown 2024-08-31T06:19:14.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.974+0000 7f29d9ec1740 1 -- 172.21.15.16:0/212888065 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ae6ab41b0 msgr2=0x556ae6ea3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:14.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.974+0000 7f29d9ec1740 1 --2- 172.21.15.16:0/212888065 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ae6ab41b0 0x556ae6ea3d40 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f29bc02a9e0 tx=0x7f29bc02a3f0 comp rx=0 tx=0).stop 2024-08-31T06:19:14.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.974+0000 7f29d9ec1740 1 -- 172.21.15.16:0/212888065 shutdown_connections 2024-08-31T06:19:14.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.974+0000 7f29d9ec1740 1 --2- 172.21.15.16:0/212888065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556ae6abc390 0x556ae6ea1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:14.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.974+0000 7f29d9ec1740 1 --2- 172.21.15.16:0/212888065 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x556ae6a70da0 0x556ae6f0d430 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:14.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.974+0000 7f29d9ec1740 1 --2- 172.21.15.16:0/212888065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae6add070 0x556ae6ea1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:14.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.974+0000 7f29d9ec1740 1 --2- 172.21.15.16:0/212888065 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ae6ab41b0 0x556ae6ea3d40 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:14.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.974+0000 7f29b6000640 0 client.28334 ms_handle_reset on v2:172.21.15.16:6838/219664460 2024-08-31T06:19:14.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.974+0000 7f29d9ec1740 1 -- 172.21.15.16:0/212888065 >> 172.21.15.16:0/212888065 conn(0x556ae6ad1460 msgr2=0x556ae6e7ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:14.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.974+0000 7f29d9ec1740 1 -- 172.21.15.16:0/212888065 shutdown_connections 2024-08-31T06:19:14.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.974+0000 7f29d9ec1740 1 -- 172.21.15.16:0/212888065 wait complete. 2024-08-31T06:19:14.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.974+0000 7f29d9ec1740 20 client.28334 trim_cache size 0 max 16384 2024-08-31T06:19:14.979 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:19:14.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.975+0000 7f29d3200640 1 -- 172.21.15.16:0/4061110610 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f29b00b4df0 msgr2=0x7f29b00b7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:14.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.975+0000 7f29d3200640 1 --2- 172.21.15.16:0/4061110610 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f29b00b4df0 0x7f29b00b7300 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f29cc103ee0 tx=0x7f29bc005a40 comp rx=0 tx=0).stop 2024-08-31T06:19:14.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.975+0000 7f29d3200640 1 -- 172.21.15.16:0/4061110610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29cc102f10 msgr2=0x7f29cc1a84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:14.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.975+0000 7f29d3200640 1 --2- 172.21.15.16:0/4061110610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29cc102f10 0x7f29cc1a84a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f29c400ea80 tx=0x7f29c400ef40 comp rx=0 tx=0).stop 2024-08-31T06:19:14.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.975+0000 7f29d3200640 1 -- 172.21.15.16:0/4061110610 shutdown_connections 2024-08-31T06:19:14.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.975+0000 7f29d3200640 1 --2- 172.21.15.16:0/4061110610 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f29b00b4df0 0x7f29b00b7300 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:14.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.975+0000 7f29d3200640 1 --2- 172.21.15.16:0/4061110610 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29cc10dbb0 0x7f29cc1029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:14.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.975+0000 7f29d3200640 1 --2- 172.21.15.16:0/4061110610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29cc102f10 0x7f29cc1a84a0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:14.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.975+0000 7f29d3200640 1 --2- 172.21.15.16:0/4061110610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29cc1050a0 0x7f29cc102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:14.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.975+0000 7f29d3200640 1 -- 172.21.15.16:0/4061110610 >> 172.21.15.16:0/4061110610 conn(0x7f29cc0794d0 msgr2=0x7f29cc0ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:14.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.976+0000 7f29d3200640 1 -- 172.21.15.16:0/4061110610 shutdown_connections 2024-08-31T06:19:14.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:14.976+0000 7f29d3200640 1 -- 172.21.15.16:0/4061110610 wait complete. 2024-08-31T06:19:16.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 710.0 s, 1834.6 tps, lat 54.299 ms stddev 64.793 2024-08-31T06:19:16.967 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T06:19:17.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:16 smithi155 ceph-mon[21742]: pgmap v738: 97 pgs: 97 active+clean; 3.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 17 MiB/s wr, 1.53k op/s 2024-08-31T06:19:17.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:16 smithi155 ceph-mon[21742]: osdmap e95: 12 total, 12 up, 12 in 2024-08-31T06:19:17.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:16 smithi016 ceph-mon[17194]: pgmap v738: 97 pgs: 97 active+clean; 3.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 17 MiB/s wr, 1.53k op/s 2024-08-31T06:19:17.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:16 smithi016 ceph-mon[17194]: osdmap e95: 12 total, 12 up, 12 in 2024-08-31T06:19:17.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:16 smithi184 ceph-mon[22331]: pgmap v738: 97 pgs: 97 active+clean; 3.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 17 MiB/s wr, 1.53k op/s 2024-08-31T06:19:17.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:16 smithi184 ceph-mon[22331]: osdmap e95: 12 total, 12 up, 12 in 2024-08-31T06:19:17.587 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:19:18.650 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:19:18.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.362+0000 7faa09c00640 1 -- 172.21.15.16:0/1053385417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa041099c0 msgr2=0x7faa041101f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:18.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.362+0000 7faa09c00640 1 --2- 172.21.15.16:0/1053385417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa041099c0 0x7faa041101f0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc009ca0 tx=0x7fa9fc02a8e0 comp rx=0 tx=0).stop 2024-08-31T06:19:18.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.362+0000 7faa09c00640 1 -- 172.21.15.16:0/1053385417 shutdown_connections 2024-08-31T06:19:18.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.362+0000 7faa09c00640 1 --2- 172.21.15.16:0/1053385417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa04109070 0x7faa04109480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:18.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.362+0000 7faa09c00640 1 --2- 172.21.15.16:0/1053385417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa041099c0 0x7faa041101f0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:18.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.362+0000 7faa09c00640 1 --2- 172.21.15.16:0/1053385417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa04103730 0x7faa04103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:18.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.362+0000 7faa09c00640 1 -- 172.21.15.16:0/1053385417 >> 172.21.15.16:0/1053385417 conn(0x7faa040ff180 msgr2=0x7faa041015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:18.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.362+0000 7faa09c00640 1 -- 172.21.15.16:0/1053385417 shutdown_connections 2024-08-31T06:19:18.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.362+0000 7faa09c00640 1 -- 172.21.15.16:0/1053385417 wait complete. 2024-08-31T06:19:18.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.363+0000 7faa09c00640 1 Processor -- start 2024-08-31T06:19:18.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.363+0000 7faa09c00640 1 -- start start 2024-08-31T06:19:18.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.363+0000 7faa09c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa04103730 0x7faa0419c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:18.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.364+0000 7faa09c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa04109070 0x7faa0419c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:18.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.364+0000 7faa09c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa041099c0 0x7faa041a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:18.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.364+0000 7faa09c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faa04113770 con 0x7faa04103730 2024-08-31T06:19:18.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.364+0000 7faa09c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faa041135f0 con 0x7faa04109070 2024-08-31T06:19:18.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.364+0000 7faa09c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faa041138f0 con 0x7faa041099c0 2024-08-31T06:19:18.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.364+0000 7faa03e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa04103730 0x7faa0419c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:18.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.364+0000 7faa03e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa04103730 0x7faa0419c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53602/0 (socket says 172.21.15.16:53602) 2024-08-31T06:19:18.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.364+0000 7faa03e00640 1 -- 172.21.15.16:0/3499985645 learned_addr learned my addr 172.21.15.16:0/3499985645 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:18.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.364+0000 7faa09200640 1 --2- 172.21.15.16:0/3499985645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa041099c0 0x7faa041a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:18.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.364+0000 7faa03400640 1 --2- 172.21.15.16:0/3499985645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa04109070 0x7faa0419c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:18.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.364+0000 7faa03e00640 1 -- 172.21.15.16:0/3499985645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa041099c0 msgr2=0x7faa041a1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:18.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.364+0000 7faa03e00640 1 --2- 172.21.15.16:0/3499985645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa041099c0 0x7faa041a1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:18.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.364+0000 7faa03e00640 1 -- 172.21.15.16:0/3499985645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa04109070 msgr2=0x7faa0419c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:18.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.364+0000 7faa03e00640 1 --2- 172.21.15.16:0/3499985645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa04109070 0x7faa0419c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:18.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.364+0000 7faa03e00640 1 -- 172.21.15.16:0/3499985645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa041a2270 con 0x7faa04103730 2024-08-31T06:19:18.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.364+0000 7faa09200640 1 --2- 172.21.15.16:0/3499985645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa041099c0 0x7faa041a1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:19:18.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.364+0000 7faa03400640 1 --2- 172.21.15.16:0/3499985645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa04109070 0x7faa0419c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:19:18.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.364+0000 7faa03e00640 1 --2- 172.21.15.16:0/3499985645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa04103730 0x7faa0419c300 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa9f400b450 tx=0x7fa9f400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:18.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.365+0000 7faa00c00640 1 -- 172.21.15.16:0/3499985645 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4015020 con 0x7faa04103730 2024-08-31T06:19:18.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.365+0000 7faa00c00640 1 -- 172.21.15.16:0/3499985645 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9f4011d40 con 0x7faa04103730 2024-08-31T06:19:18.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.365+0000 7faa09c00640 1 -- 172.21.15.16:0/3499985645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa041a2560 con 0x7faa04103730 2024-08-31T06:19:18.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.365+0000 7faa00c00640 1 -- 172.21.15.16:0/3499985645 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f400f340 con 0x7faa04103730 2024-08-31T06:19:18.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.365+0000 7faa09c00640 1 -- 172.21.15.16:0/3499985645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa041a8cc0 con 0x7faa04103730 2024-08-31T06:19:18.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.368+0000 7faa11ebe740 1 Processor -- start 2024-08-31T06:19:18.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.368+0000 7faa11ebe740 1 -- start start 2024-08-31T06:19:18.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.369+0000 7faa11ebe740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560fd90b1070 0x560fd9456b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:18.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.369+0000 7faa11ebe740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fd90881b0 0x560fd94570a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:18.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.369+0000 7faa11ebe740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fd9090390 0x560fd94575e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:18.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.369+0000 7faa11ebe740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560fd93cd7b0 con 0x560fd9090390 2024-08-31T06:19:18.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.369+0000 7faa11ebe740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560fd93bcd40 con 0x560fd90b1070 2024-08-31T06:19:18.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.369+0000 7faa11ebe740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560fd93c0cf0 con 0x560fd90881b0 2024-08-31T06:19:18.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.369+0000 7faa00c00640 1 -- 172.21.15.16:0/3499985645 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fa9f4018020 con 0x7faa04103730 2024-08-31T06:19:18.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.369+0000 7faa03400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fd9090390 0x560fd94575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:18.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.369+0000 7faa03400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fd9090390 0x560fd94575e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53606/0 (socket says 172.21.15.16:53606) 2024-08-31T06:19:18.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.369+0000 7faa09200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560fd90b1070 0x560fd9456b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:18.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.369+0000 7faa03400640 1 -- 172.21.15.16:0/3534011676 learned_addr learned my addr 172.21.15.16:0/3534011676 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:18.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.369+0000 7faa03e00640 1 --2- 172.21.15.16:0/3534011676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fd90881b0 0x560fd94570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:18.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.369+0000 7faa09200640 1 -- 172.21.15.16:0/3534011676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fd90881b0 msgr2=0x560fd94570a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:18.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.369+0000 7faa09200640 1 --2- 172.21.15.16:0/3534011676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fd90881b0 0x560fd94570a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:18.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.369+0000 7faa09200640 1 -- 172.21.15.16:0/3534011676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fd9090390 msgr2=0x560fd94575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:18.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.369+0000 7faa09200640 1 --2- 172.21.15.16:0/3534011676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fd9090390 0x560fd94575e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:18.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.369+0000 7faa09200640 1 -- 172.21.15.16:0/3534011676 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fd911c650 con 0x560fd90b1070 2024-08-31T06:19:18.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.369+0000 7faa03e00640 1 --2- 172.21.15.16:0/3534011676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fd90881b0 0x560fd94570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:19:18.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.370+0000 7faa09200640 1 --2- 172.21.15.16:0/3534011676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560fd90b1070 0x560fd9456b60 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc006010 tx=0x7fa9fc031580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a66490ad8f1d5285 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:18.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.370+0000 7faa00c00640 1 --2- 172.21.15.16:0/3499985645 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa9e80b4ca0 0x7fa9e80b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:18.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.370+0000 7fa9f9600640 1 -- 172.21.15.16:0/3534011676 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc031ad0 con 0x560fd90b1070 2024-08-31T06:19:18.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.370+0000 7fa9f9600640 1 -- 172.21.15.16:0/3534011676 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9fc036040 con 0x560fd90b1070 2024-08-31T06:19:18.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.371+0000 7fa9f9600640 1 -- 172.21.15.16:0/3534011676 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc0045c0 con 0x560fd90b1070 2024-08-31T06:19:18.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.371+0000 7faa00c00640 1 -- 172.21.15.16:0/3499985645 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8470+0+0 (secure 0 0 0) 0x7fa9f40e4ff0 con 0x7faa04103730 2024-08-31T06:19:18.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.371+0000 7faa03e00640 1 --2- 172.21.15.16:0/3499985645 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa9e80b4ca0 0x7fa9e80b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:18.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.371+0000 7faa11ebe740 1 -- 172.21.15.16:0/3534011676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560fd90b1070 msgr2=0x560fd9456b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:18.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.371+0000 7faa11ebe740 1 --2- 172.21.15.16:0/3534011676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560fd90b1070 0x560fd9456b60 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc006010 tx=0x7fa9fc031580 comp rx=0 tx=0).stop 2024-08-31T06:19:18.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.372+0000 7faa11ebe740 1 -- 172.21.15.16:0/3534011676 shutdown_connections 2024-08-31T06:19:18.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.372+0000 7faa11ebe740 1 --2- 172.21.15.16:0/3534011676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fd90881b0 0x560fd94570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:18.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.372+0000 7faa11ebe740 1 --2- 172.21.15.16:0/3534011676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fd9090390 0x560fd94575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:18.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.372+0000 7faa11ebe740 1 --2- 172.21.15.16:0/3534011676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560fd90b1070 0x560fd9456b60 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:18.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.372+0000 7faa03e00640 1 --2- 172.21.15.16:0/3499985645 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa9e80b4ca0 0x7fa9e80b71b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x560fd94557e0 tx=0x7fa9f40108d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:18.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.372+0000 7faa11ebe740 1 -- 172.21.15.16:0/3534011676 >> 172.21.15.16:0/3534011676 conn(0x560fd90a5460 msgr2=0x560fd9454a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:18.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.372+0000 7faa11ebe740 1 -- 172.21.15.16:0/3534011676 shutdown_connections 2024-08-31T06:19:18.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.372+0000 7faa11ebe740 1 -- 172.21.15.16:0/3534011676 wait complete. 2024-08-31T06:19:18.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.372+0000 7faa11ebe740 1 -- start start 2024-08-31T06:19:18.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.373+0000 7faa11ebe740 1 Processor -- start 2024-08-31T06:19:18.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.373+0000 7faa11ebe740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fd90881b0 0x560fd9462770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:18.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.373+0000 7faa11ebe740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fd9090390 0x560fd9462cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:18.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.373+0000 7faa11ebe740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560fd90b1070 0x560fd946e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:18.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.373+0000 7faa11ebe740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560fd921f700 con 0x560fd9090390 2024-08-31T06:19:18.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.373+0000 7faa11ebe740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560fd9343cf0 con 0x560fd90b1070 2024-08-31T06:19:18.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.373+0000 7faa11ebe740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560fd94652e0 con 0x560fd90881b0 2024-08-31T06:19:18.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.373+0000 7faa11ebe740 10 client.0 _ll_register_callbacks cb 0x560fd9369470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:19:18.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.373+0000 7faa03400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fd9090390 0x560fd9462cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:18.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.373+0000 7faa03400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fd9090390 0x560fd9462cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53608/0 (socket says 172.21.15.16:53608) 2024-08-31T06:19:18.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.373+0000 7faa03400640 1 -- 172.21.15.16:0/1660715322 learned_addr learned my addr 172.21.15.16:0/1660715322 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:18.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.374+0000 7faa03400640 1 -- 172.21.15.16:0/1660715322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fd90881b0 msgr2=0x560fd9462770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:18.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.374+0000 7faa09200640 1 --2- 172.21.15.16:0/1660715322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fd90881b0 0x560fd9462770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:18.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.374+0000 7faa03400640 1 --2- 172.21.15.16:0/1660715322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fd90881b0 0x560fd9462770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:18.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.374+0000 7faa03400640 1 -- 172.21.15.16:0/1660715322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560fd90b1070 msgr2=0x560fd946e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:18.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.374+0000 7faa03400640 1 --2- 172.21.15.16:0/1660715322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560fd90b1070 0x560fd946e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:18.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.374+0000 7faa03400640 1 -- 172.21.15.16:0/1660715322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fd93848b0 con 0x560fd9090390 2024-08-31T06:19:18.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.374+0000 7faa09200640 1 --2- 172.21.15.16:0/1660715322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fd90881b0 0x560fd9462770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:19:18.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.374+0000 7faa03400640 1 --2- 172.21.15.16:0/1660715322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fd9090390 0x560fd9462cf0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0013760 tx=0x7fa9f0013c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e0e2ef212201a7e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:18.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.374+0000 7fa9ee000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:19:18.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.375+0000 7fa9ee000640 1 -- 172.21.15.16:0/1660715322 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0020060 con 0x560fd9090390 2024-08-31T06:19:18.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.375+0000 7faa11ebe740 1 -- 172.21.15.16:0/1660715322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9fc036040 con 0x560fd9090390 2024-08-31T06:19:18.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.375+0000 7fa9ee000640 1 -- 172.21.15.16:0/1660715322 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8470+0+0 (secure 0 0 0) 0x7fa9f0011e30 con 0x560fd9090390 2024-08-31T06:19:18.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.375+0000 7fa9ee000640 1 -- 172.21.15.16:0/1660715322 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fa9f0025480 con 0x560fd9090390 2024-08-31T06:19:18.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.375+0000 7faa11ebe740 10 client.17964 fetch_fsmap learned FSMap version 49 2024-08-31T06:19:18.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.375+0000 7faa11ebe740 1 -- 172.21.15.16:0/1660715322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x560fd94713d0 con 0x560fd9090390 2024-08-31T06:19:18.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.375+0000 7fa9ee000640 1 -- 172.21.15.16:0/1660715322 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9fc036040 con 0x560fd9090390 2024-08-31T06:19:18.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.375+0000 7fa9ee000640 1 -- 172.21.15.16:0/1660715322 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fa9f001a030 con 0x560fd9090390 2024-08-31T06:19:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.375+0000 7faa11ebe740 10 client.17964 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:19:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.375+0000 7faa11ebe740 10 client.17964 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:19:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.375+0000 7faa11ebe740 20 client.17964 populate_metadata read hostname 'smithi016' 2024-08-31T06:19:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.375+0000 7faa11ebe740 1 --2- 172.21.15.16:0/1660715322 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x560fd906f9f0 0x560fd94749a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.375+0000 7faa11ebe740 4 client.17964 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:19:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.375+0000 7faa11ebe740 1 -- 172.21.15.16:0/1660715322 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560fd93cf700 con 0x560fd906f9f0 2024-08-31T06:19:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.376+0000 7faa09200640 1 --2- 172.21.15.16:0/1660715322 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x560fd906f9f0 0x560fd94749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.377+0000 7faa09200640 1 --2- 172.21.15.16:0/1660715322 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x560fd906f9f0 0x560fd94749a0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59129970c3e68fa1 server_cookie=78e118d1a0c666f1 in_seq=0 out_seq=0 2024-08-31T06:19:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.377+0000 7fa9ee000640 10 client.17964 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:19:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.377+0000 7fa9ee000640 1 -- 172.21.15.16:0/1660715322 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560fd93cf700 con 0x560fd906f9f0 2024-08-31T06:19:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.377+0000 7fa9ee000640 10 client.17964 handle_command_reply: tid=0 2024-08-31T06:19:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.377+0000 7faa11ebe740 1 client.17964 shutdown 2024-08-31T06:19:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.378+0000 7faa11ebe740 1 -- 172.21.15.16:0/1660715322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fd9090390 msgr2=0x560fd9462cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.378+0000 7faa11ebe740 1 --2- 172.21.15.16:0/1660715322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fd9090390 0x560fd9462cf0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0013760 tx=0x7fa9f0013c20 comp rx=0 tx=0).stop 2024-08-31T06:19:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.378+0000 7faa11ebe740 1 -- 172.21.15.16:0/1660715322 shutdown_connections 2024-08-31T06:19:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.378+0000 7faa11ebe740 1 --2- 172.21.15.16:0/1660715322 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x560fd906f9f0 0x560fd94749a0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.378+0000 7faa11ebe740 1 --2- 172.21.15.16:0/1660715322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560fd90b1070 0x560fd946e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.378+0000 7faa11ebe740 1 --2- 172.21.15.16:0/1660715322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fd9090390 0x560fd9462cf0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.378+0000 7faa11ebe740 1 --2- 172.21.15.16:0/1660715322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fd90881b0 0x560fd9462770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.378+0000 7faa11ebe740 1 -- 172.21.15.16:0/1660715322 >> 172.21.15.16:0/1660715322 conn(0x560fd90a5460 msgr2=0x560fd9459cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.378+0000 7fa9ee000640 0 client.17964 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:19:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.378+0000 7faa11ebe740 1 -- 172.21.15.16:0/1660715322 shutdown_connections 2024-08-31T06:19:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.378+0000 7faa11ebe740 1 -- 172.21.15.16:0/1660715322 wait complete. 2024-08-31T06:19:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.378+0000 7faa11ebe740 20 client.17964 trim_cache size 0 max 16384 2024-08-31T06:19:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.392+0000 7faa11ebe740 1 Processor -- start 2024-08-31T06:19:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.392+0000 7faa11ebe740 1 -- start start 2024-08-31T06:19:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.392+0000 7faa11ebe740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560fd90881b0 0x560fd945bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.392+0000 7faa11ebe740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fd9090390 0x560fd945c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.392+0000 7faa11ebe740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fd90b1070 0x560fd946e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.392+0000 7faa11ebe740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faa041b3e40 con 0x560fd90b1070 2024-08-31T06:19:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.392+0000 7faa11ebe740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faa041b1650 con 0x560fd90881b0 2024-08-31T06:19:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.392+0000 7faa11ebe740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faa041af4c0 con 0x560fd9090390 2024-08-31T06:19:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.393+0000 7faa03400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fd9090390 0x560fd945c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.393+0000 7faa03400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fd9090390 0x560fd945c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:36354/0 (socket says 172.21.15.16:36354) 2024-08-31T06:19:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.393+0000 7faa03400640 1 -- 172.21.15.16:0/793627169 learned_addr learned my addr 172.21.15.16:0/793627169 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.393+0000 7faa09200640 1 --2- 172.21.15.16:0/793627169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fd90b1070 0x560fd946e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.393+0000 7faa09200640 1 --2- 172.21.15.16:0/793627169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560fd90881b0 0x560fd945bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.393+0000 7faa03400640 1 -- 172.21.15.16:0/793627169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560fd90881b0 msgr2=0x560fd945bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.393+0000 7faa03400640 1 --2- 172.21.15.16:0/793627169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560fd90881b0 0x560fd945bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.393+0000 7faa03400640 1 -- 172.21.15.16:0/793627169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fd90b1070 msgr2=0x560fd946e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.393+0000 7faa03400640 1 --2- 172.21.15.16:0/793627169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fd90b1070 0x560fd946e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.393+0000 7faa03400640 1 -- 172.21.15.16:0/793627169 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fd9458e50 con 0x560fd9090390 2024-08-31T06:19:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.393+0000 7faa03400640 1 --2- 172.21.15.16:0/793627169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fd9090390 0x560fd945c1b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0016040 tx=0x7fa9f00092a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6d72d79167b6c6d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.394+0000 7fa9f8c00640 1 -- 172.21.15.16:0/793627169 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f000ecd0 con 0x560fd9090390 2024-08-31T06:19:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.394+0000 7fa9f8c00640 1 -- 172.21.15.16:0/793627169 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9f000ee70 con 0x560fd9090390 2024-08-31T06:19:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.394+0000 7fa9f8c00640 1 -- 172.21.15.16:0/793627169 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0025d30 con 0x560fd9090390 2024-08-31T06:19:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.394+0000 7faa11ebe740 1 -- 172.21.15.16:0/793627169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fd9090390 msgr2=0x560fd945c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.394+0000 7faa11ebe740 1 --2- 172.21.15.16:0/793627169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fd9090390 0x560fd945c1b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0016040 tx=0x7fa9f00092a0 comp rx=0 tx=0).stop 2024-08-31T06:19:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.394+0000 7faa11ebe740 1 -- 172.21.15.16:0/793627169 shutdown_connections 2024-08-31T06:19:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.394+0000 7faa11ebe740 1 --2- 172.21.15.16:0/793627169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fd9090390 0x560fd945c1b0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.394+0000 7faa11ebe740 1 --2- 172.21.15.16:0/793627169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fd90b1070 0x560fd946e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.394+0000 7faa11ebe740 1 --2- 172.21.15.16:0/793627169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560fd90881b0 0x560fd945bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.394+0000 7faa11ebe740 1 -- 172.21.15.16:0/793627169 >> 172.21.15.16:0/793627169 conn(0x560fd90a5460 msgr2=0x560fd9460df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.395+0000 7faa11ebe740 1 -- 172.21.15.16:0/793627169 shutdown_connections 2024-08-31T06:19:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.395+0000 7faa11ebe740 1 -- 172.21.15.16:0/793627169 wait complete. 2024-08-31T06:19:18.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.395+0000 7faa11ebe740 1 -- start start 2024-08-31T06:19:18.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.395+0000 7faa11ebe740 1 Processor -- start 2024-08-31T06:19:18.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.396+0000 7faa11ebe740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560fd90881b0 0x560fd9467080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:18.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.396+0000 7faa11ebe740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fd9090390 0x560fd9467660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:18.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.396+0000 7faa11ebe740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fd90b1070 0x560fd94d8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:18.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.396+0000 7faa11ebe740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faa041afb10 con 0x560fd9090390 2024-08-31T06:19:18.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.396+0000 7faa11ebe740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faa041b0f80 con 0x560fd90881b0 2024-08-31T06:19:18.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.396+0000 7faa11ebe740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faa041b01e0 con 0x560fd90b1070 2024-08-31T06:19:18.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.396+0000 7faa11ebe740 10 client.0 _ll_register_callbacks cb 0x7fa9f0009f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:19:18.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.396+0000 7faa09200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560fd90881b0 0x560fd9467080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:18.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.396+0000 7faa03400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fd9090390 0x560fd9467660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:18.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.396+0000 7faa09200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560fd90881b0 0x560fd9467080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:45036/0 (socket says 172.21.15.16:45036) 2024-08-31T06:19:18.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.396+0000 7faa09200640 1 -- 172.21.15.16:0/3008403953 learned_addr learned my addr 172.21.15.16:0/3008403953 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:18.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.396+0000 7faa09200640 1 --2- 172.21.15.16:0/3008403953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fd90b1070 0x560fd94d8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:18.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.396+0000 7faa03400640 1 -- 172.21.15.16:0/3008403953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fd90b1070 msgr2=0x560fd94d8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:18.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.396+0000 7faa03400640 1 --2- 172.21.15.16:0/3008403953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fd90b1070 0x560fd94d8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:18.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.396+0000 7faa03400640 1 -- 172.21.15.16:0/3008403953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560fd90881b0 msgr2=0x560fd9467080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:18.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.396+0000 7faa03400640 1 --2- 172.21.15.16:0/3008403953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560fd90881b0 0x560fd9467080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:18.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.396+0000 7faa03400640 1 -- 172.21.15.16:0/3008403953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fd94d8b70 con 0x560fd9090390 2024-08-31T06:19:18.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.396+0000 7faa09200640 1 --2- 172.21.15.16:0/3008403953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560fd90881b0 0x560fd9467080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:19:18.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.397+0000 7faa03400640 1 --2- 172.21.15.16:0/3008403953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fd9090390 0x560fd9467660 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0013760 tx=0x7fa9f0011c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99568e521ddbc42e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:18.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.397+0000 7fa9ee000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:19:18.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.397+0000 7fa9ee000640 1 -- 172.21.15.16:0/3008403953 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0020040 con 0x560fd9090390 2024-08-31T06:19:18.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.397+0000 7faa11ebe740 1 -- 172.21.15.16:0/3008403953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9f000ee70 con 0x560fd9090390 2024-08-31T06:19:18.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.397+0000 7fa9ee000640 1 -- 172.21.15.16:0/3008403953 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8470+0+0 (secure 0 0 0) 0x7fa9f0025a30 con 0x560fd9090390 2024-08-31T06:19:18.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.397+0000 7fa9ee000640 1 -- 172.21.15.16:0/3008403953 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fa9f0011e70 con 0x560fd9090390 2024-08-31T06:19:18.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.397+0000 7faa11ebe740 10 client.17970 fetch_fsmap learned FSMap version 49 2024-08-31T06:19:18.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.397+0000 7faa11ebe740 1 -- 172.21.15.16:0/3008403953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x560fd94dff60 con 0x560fd9090390 2024-08-31T06:19:18.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.398+0000 7fa9ee000640 1 -- 172.21.15.16:0/3008403953 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9f000ee70 con 0x560fd9090390 2024-08-31T06:19:18.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.398+0000 7fa9ee000640 1 -- 172.21.15.16:0/3008403953 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fa9f002d030 con 0x560fd9090390 2024-08-31T06:19:18.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.398+0000 7faa11ebe740 10 client.17970 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:19:18.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.398+0000 7faa11ebe740 10 client.17970 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:19:18.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.398+0000 7faa11ebe740 20 client.17970 populate_metadata read hostname 'smithi016' 2024-08-31T06:19:18.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.398+0000 7faa11ebe740 1 --2- 172.21.15.16:0/3008403953 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x560fd9044da0 0x560fd94e2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:18.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.398+0000 7faa11ebe740 4 client.17970 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:19:18.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.398+0000 7faa11ebe740 1 -- 172.21.15.16:0/3008403953 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560fd93cb1a0 con 0x560fd9044da0 2024-08-31T06:19:18.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.399+0000 7faa09200640 1 --2- 172.21.15.16:0/3008403953 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x560fd9044da0 0x560fd94e2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:18.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.400+0000 7faa09200640 1 --2- 172.21.15.16:0/3008403953 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x560fd9044da0 0x560fd94e2640 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0b4ff5f5302c49e server_cookie=beba99c79ea49ca5 in_seq=0 out_seq=0 2024-08-31T06:19:18.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.400+0000 7fa9ee000640 10 client.17970 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:19:18.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.400+0000 7fa9ee000640 1 -- 172.21.15.16:0/3008403953 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560fd93cb1a0 con 0x560fd9044da0 2024-08-31T06:19:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.400+0000 7fa9ee000640 10 client.17970 handle_command_reply: tid=0 2024-08-31T06:19:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.400+0000 7faa11ebe740 1 client.17970 shutdown 2024-08-31T06:19:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.400+0000 7faa11ebe740 1 -- 172.21.15.16:0/3008403953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fd9090390 msgr2=0x560fd9467660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.400+0000 7faa11ebe740 1 --2- 172.21.15.16:0/3008403953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fd9090390 0x560fd9467660 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0013760 tx=0x7fa9f0011c30 comp rx=0 tx=0).stop 2024-08-31T06:19:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.400+0000 7faa11ebe740 1 -- 172.21.15.16:0/3008403953 shutdown_connections 2024-08-31T06:19:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.400+0000 7faa11ebe740 1 --2- 172.21.15.16:0/3008403953 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x560fd9044da0 0x560fd94e2640 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.400+0000 7faa11ebe740 1 --2- 172.21.15.16:0/3008403953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fd90b1070 0x560fd94d8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.400+0000 7faa11ebe740 1 --2- 172.21.15.16:0/3008403953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fd9090390 0x560fd9467660 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.400+0000 7faa11ebe740 1 --2- 172.21.15.16:0/3008403953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560fd90881b0 0x560fd9467080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.400+0000 7fa9ee000640 0 client.17970 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:19:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.400+0000 7faa11ebe740 1 -- 172.21.15.16:0/3008403953 >> 172.21.15.16:0/3008403953 conn(0x560fd90a5460 msgr2=0x560fd945bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.401+0000 7faa11ebe740 1 -- 172.21.15.16:0/3008403953 shutdown_connections 2024-08-31T06:19:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.401+0000 7faa11ebe740 1 -- 172.21.15.16:0/3008403953 wait complete. 2024-08-31T06:19:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.401+0000 7faa11ebe740 20 client.17970 trim_cache size 0 max 16384 2024-08-31T06:19:18.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.401+0000 7faa09c00640 1 -- 172.21.15.16:0/3499985645 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa9e80b4ca0 msgr2=0x7fa9e80b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:18.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.401+0000 7faa09c00640 1 --2- 172.21.15.16:0/3499985645 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa9e80b4ca0 0x7fa9e80b71b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x560fd94557e0 tx=0x7fa9f40108d0 comp rx=0 tx=0).stop 2024-08-31T06:19:18.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.401+0000 7faa09c00640 1 -- 172.21.15.16:0/3499985645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa04103730 msgr2=0x7faa0419c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:18.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.401+0000 7faa09c00640 1 --2- 172.21.15.16:0/3499985645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa04103730 0x7faa0419c300 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa9f400b450 tx=0x7fa9f400b910 comp rx=0 tx=0).stop 2024-08-31T06:19:18.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.401+0000 7faa09c00640 1 -- 172.21.15.16:0/3499985645 shutdown_connections 2024-08-31T06:19:18.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.401+0000 7faa09c00640 1 --2- 172.21.15.16:0/3499985645 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa9e80b4ca0 0x7fa9e80b71b0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:18.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.401+0000 7faa09c00640 1 --2- 172.21.15.16:0/3499985645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa041099c0 0x7faa041a1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:18.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.401+0000 7faa09c00640 1 --2- 172.21.15.16:0/3499985645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa04109070 0x7faa0419c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:18.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.401+0000 7faa09c00640 1 --2- 172.21.15.16:0/3499985645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa04103730 0x7faa0419c300 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:18.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.401+0000 7faa09c00640 1 -- 172.21.15.16:0/3499985645 >> 172.21.15.16:0/3499985645 conn(0x7faa040ff180 msgr2=0x7faa04107690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:18.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.402+0000 7faa09c00640 1 -- 172.21.15.16:0/3499985645 shutdown_connections 2024-08-31T06:19:18.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:18.402+0000 7faa09c00640 1 -- 172.21.15.16:0/3499985645 wait complete. 2024-08-31T06:19:18.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:18 smithi016 ceph-mon[17194]: pgmap v740: 97 pgs: 97 active+clean; 3.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 8.1 MiB/s wr, 647 op/s 2024-08-31T06:19:18.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:18 smithi016 ceph-mon[17194]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:19:19.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:18 smithi155 ceph-mon[21742]: pgmap v740: 97 pgs: 97 active+clean; 3.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 8.1 MiB/s wr, 647 op/s 2024-08-31T06:19:19.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:18 smithi155 ceph-mon[21742]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:19:19.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:18 smithi184 ceph-mon[22331]: pgmap v740: 97 pgs: 97 active+clean; 3.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 8.1 MiB/s wr, 647 op/s 2024-08-31T06:19:19.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:18 smithi184 ceph-mon[22331]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:19:20.665 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T06:19:20.665 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T06:19:21.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:20 smithi155 ceph-mon[21742]: pgmap v741: 97 pgs: 97 active+clean; 3.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 8.9 MiB/s wr, 727 op/s 2024-08-31T06:19:21.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:20 smithi016 ceph-mon[17194]: pgmap v741: 97 pgs: 97 active+clean; 3.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 8.9 MiB/s wr, 727 op/s 2024-08-31T06:19:21.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:20 smithi184 ceph-mon[22331]: pgmap v741: 97 pgs: 97 active+clean; 3.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 8.9 MiB/s wr, 727 op/s 2024-08-31T06:19:21.336 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:19:21.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 715.0 s, 1692.2 tps, lat 58.882 ms stddev 72.857 2024-08-31T06:19:22.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.171+0000 7f68af200640 1 -- 172.21.15.16:0/115358137 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68a8075ea0 msgr2=0x7f68a810f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:22.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.171+0000 7f68af200640 1 --2- 172.21.15.16:0/115358137 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68a8075ea0 0x7f68a810f770 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f6898009900 tx=0x7f689802a3f0 comp rx=0 tx=0).stop 2024-08-31T06:19:22.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.172+0000 7f68af200640 1 -- 172.21.15.16:0/115358137 shutdown_connections 2024-08-31T06:19:22.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.172+0000 7f68af200640 1 --2- 172.21.15.16:0/115358137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68a810fcb0 0x7f68a81120f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:22.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.172+0000 7f68af200640 1 --2- 172.21.15.16:0/115358137 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68a8075ea0 0x7f68a810f770 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:22.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.172+0000 7f68af200640 1 --2- 172.21.15.16:0/115358137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68a8075550 0x7f68a8075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:22.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.172+0000 7f68af200640 1 -- 172.21.15.16:0/115358137 >> 172.21.15.16:0/115358137 conn(0x7f68a80ff1e0 msgr2=0x7f68a8101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:22.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.172+0000 7f68af200640 1 -- 172.21.15.16:0/115358137 shutdown_connections 2024-08-31T06:19:22.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.172+0000 7f68af200640 1 -- 172.21.15.16:0/115358137 wait complete. 2024-08-31T06:19:22.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.173+0000 7f68af200640 1 Processor -- start 2024-08-31T06:19:22.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.173+0000 7f68af200640 1 -- start start 2024-08-31T06:19:22.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.174+0000 7f68af200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68a8075550 0x7f68a810e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:22.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.174+0000 7f68af200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68a8075ea0 0x7f68a810e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:22.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.174+0000 7f68af200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68a810fcb0 0x7f68a81ac1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:22.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.174+0000 7f68af200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f68a8103f50 con 0x7f68a8075550 2024-08-31T06:19:22.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.174+0000 7f68af200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f68a8103dd0 con 0x7f68a810fcb0 2024-08-31T06:19:22.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.174+0000 7f68af200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f68a81040d0 con 0x7f68a8075ea0 2024-08-31T06:19:22.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.174+0000 7f68ae800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68a810fcb0 0x7f68a81ac1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:22.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.174+0000 7f68ae800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68a810fcb0 0x7f68a81ac1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:45060/0 (socket says 172.21.15.16:45060) 2024-08-31T06:19:22.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.174+0000 7f68ae800640 1 -- 172.21.15.16:0/1641736529 learned_addr learned my addr 172.21.15.16:0/1641736529 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:22.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.175+0000 7f68ad400640 1 --2- 172.21.15.16:0/1641736529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68a8075ea0 0x7f68a810e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:22.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.175+0000 7f68ade00640 1 --2- 172.21.15.16:0/1641736529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68a8075550 0x7f68a810e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:22.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.175+0000 7f68ae800640 1 -- 172.21.15.16:0/1641736529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68a8075ea0 msgr2=0x7f68a810e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:22.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.175+0000 7f68ae800640 1 --2- 172.21.15.16:0/1641736529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68a8075ea0 0x7f68a810e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:22.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.175+0000 7f68ae800640 1 -- 172.21.15.16:0/1641736529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68a8075550 msgr2=0x7f68a810e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:22.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.175+0000 7f68ae800640 1 --2- 172.21.15.16:0/1641736529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68a8075550 0x7f68a810e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:22.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.175+0000 7f68ae800640 1 -- 172.21.15.16:0/1641736529 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68a81ac6f0 con 0x7f68a810fcb0 2024-08-31T06:19:22.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.175+0000 7f68ad400640 1 --2- 172.21.15.16:0/1641736529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68a8075ea0 0x7f68a810e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:19:22.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.175+0000 7f68ade00640 1 --2- 172.21.15.16:0/1641736529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68a8075550 0x7f68a810e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:19:22.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.176+0000 7f68ae800640 1 --2- 172.21.15.16:0/1641736529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68a810fcb0 0x7f68a81ac1b0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f68a400eb10 tx=0x7f68a400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:22.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.176+0000 7f68a2a00640 1 -- 172.21.15.16:0/1641736529 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a4004b20 con 0x7f68a810fcb0 2024-08-31T06:19:22.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.177+0000 7f68a2a00640 1 -- 172.21.15.16:0/1641736529 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68a4004cc0 con 0x7f68a810fcb0 2024-08-31T06:19:22.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.177+0000 7f68a2a00640 1 -- 172.21.15.16:0/1641736529 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a4010690 con 0x7f68a810fcb0 2024-08-31T06:19:22.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.177+0000 7f68af200640 1 -- 172.21.15.16:0/1641736529 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68a81ac9e0 con 0x7f68a810fcb0 2024-08-31T06:19:22.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.177+0000 7f68af200640 1 -- 172.21.15.16:0/1641736529 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68a81acea0 con 0x7f68a810fcb0 2024-08-31T06:19:22.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.184+0000 7f68b75ed740 1 Processor -- start 2024-08-31T06:19:22.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.185+0000 7f68a2a00640 1 -- 172.21.15.16:0/1641736529 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f68a4005000 con 0x7f68a810fcb0 2024-08-31T06:19:22.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.188+0000 7f68a2a00640 1 --2- 172.21.15.16:0/1641736529 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f68900b4da0 0x7f68900b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:22.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.188+0000 7f68a2a00640 1 -- 172.21.15.16:0/1641736529 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8470+0+0 (secure 0 0 0) 0x7f68a40e5c10 con 0x7f68a810fcb0 2024-08-31T06:19:22.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.189+0000 7f68ad400640 1 --2- 172.21.15.16:0/1641736529 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f68900b4da0 0x7f68900b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:22.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.189+0000 7f68b75ed740 1 -- start start 2024-08-31T06:19:22.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.189+0000 7f68b75ed740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d59fe0a350 0x55d5a01b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:22.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.190+0000 7f68b75ed740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d59fde1490 0x55d5a01b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:22.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.190+0000 7f68b75ed740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d59fde9670 0x55d5a01b1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:22.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.190+0000 7f68b75ed740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d5a0126750 con 0x55d59fde9670 2024-08-31T06:19:22.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.190+0000 7f68b75ed740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d5a0115ce0 con 0x55d59fe0a350 2024-08-31T06:19:22.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.190+0000 7f68b75ed740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d5a0119c90 con 0x55d59fde1490 2024-08-31T06:19:22.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.190+0000 7f68ae800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d59fde9670 0x55d5a01b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:22.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.190+0000 7f68ae800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d59fde9670 0x55d5a01b1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53674/0 (socket says 172.21.15.16:53674) 2024-08-31T06:19:22.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.190+0000 7f68ae800640 1 -- 172.21.15.16:0/3743762822 learned_addr learned my addr 172.21.15.16:0/3743762822 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:22.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.190+0000 7f68ad400640 1 --2- 172.21.15.16:0/3743762822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d59fde1490 0x55d5a01b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:22.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.191+0000 7f68ade00640 1 --2- 172.21.15.16:0/3743762822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d59fe0a350 0x55d5a01b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:22.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.192+0000 7f68ae800640 1 -- 172.21.15.16:0/3743762822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d59fde1490 msgr2=0x55d5a01b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:22.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.192+0000 7f68ae800640 1 --2- 172.21.15.16:0/3743762822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d59fde1490 0x55d5a01b0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:22.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.192+0000 7f68ae800640 1 -- 172.21.15.16:0/3743762822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d59fe0a350 msgr2=0x55d5a01b07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:22.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.192+0000 7f68ae800640 1 --2- 172.21.15.16:0/3743762822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d59fe0a350 0x55d5a01b07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:22.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.192+0000 7f68ae800640 1 -- 172.21.15.16:0/3743762822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d59fe75930 con 0x55d59fde9670 2024-08-31T06:19:22.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.192+0000 7f68ade00640 1 --2- 172.21.15.16:0/3743762822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d59fe0a350 0x55d5a01b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:19:22.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.193+0000 7f68ae800640 1 --2- 172.21.15.16:0/3743762822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d59fde9670 0x55d5a01b1250 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f68a40ac260 tx=0x7f68a40abd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da6be61759f1f9d4 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:22.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.193+0000 7f68ad400640 1 --2- 172.21.15.16:0/1641736529 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f68900b4da0 0x7f68900b72b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6898006010 tx=0x7f6898004350 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:22.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.193+0000 7f6897000640 1 -- 172.21.15.16:0/3743762822 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a40ac9d0 con 0x55d59fde9670 2024-08-31T06:19:22.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.193+0000 7f6897000640 1 -- 172.21.15.16:0/3743762822 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68a40ac2d0 con 0x55d59fde9670 2024-08-31T06:19:22.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.193+0000 7f6897000640 1 -- 172.21.15.16:0/3743762822 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a40b53c0 con 0x55d59fde9670 2024-08-31T06:19:22.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.196+0000 7f68b75ed740 1 -- 172.21.15.16:0/3743762822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d59fde9670 msgr2=0x55d5a01b1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:22.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.196+0000 7f68b75ed740 1 --2- 172.21.15.16:0/3743762822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d59fde9670 0x55d5a01b1250 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f68a40ac260 tx=0x7f68a40abd30 comp rx=0 tx=0).stop 2024-08-31T06:19:22.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.197+0000 7f68b75ed740 1 -- 172.21.15.16:0/3743762822 shutdown_connections 2024-08-31T06:19:22.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.197+0000 7f68b75ed740 1 --2- 172.21.15.16:0/3743762822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d59fde1490 0x55d5a01b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:22.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.197+0000 7f68b75ed740 1 --2- 172.21.15.16:0/3743762822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d59fde9670 0x55d5a01b1250 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:22.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.197+0000 7f68b75ed740 1 --2- 172.21.15.16:0/3743762822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d59fe0a350 0x55d5a01b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:22.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.197+0000 7f68b75ed740 1 -- 172.21.15.16:0/3743762822 >> 172.21.15.16:0/3743762822 conn(0x55d59fdfe740 msgr2=0x55d5a01ae6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:22.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.197+0000 7f68b75ed740 1 -- 172.21.15.16:0/3743762822 shutdown_connections 2024-08-31T06:19:22.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.197+0000 7f68b75ed740 1 -- 172.21.15.16:0/3743762822 wait complete. 2024-08-31T06:19:22.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.198+0000 7f68b75ed740 1 -- start start 2024-08-31T06:19:22.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.198+0000 7f68b75ed740 1 Processor -- start 2024-08-31T06:19:22.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.198+0000 7f68b75ed740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d59fde1490 0x55d5a01bc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:22.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.198+0000 7f68b75ed740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d59fde9670 0x55d5a01bc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:22.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.198+0000 7f68b75ed740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d59fe0a350 0x55d5a01c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:22.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.198+0000 7f68b75ed740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d59ff78890 con 0x55d59fde9670 2024-08-31T06:19:22.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.198+0000 7f68b75ed740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d5a009ccd0 con 0x55d59fde1490 2024-08-31T06:19:22.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.198+0000 7f68b75ed740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d5a01befb0 con 0x55d59fe0a350 2024-08-31T06:19:22.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.199+0000 7f68b75ed740 10 client.0 _ll_register_callbacks cb 0x55d5a0110ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:19:22.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.199+0000 7f68ad400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d59fde9670 0x55d5a01bc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:22.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.199+0000 7f68ad400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d59fde9670 0x55d5a01bc8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53686/0 (socket says 172.21.15.16:53686) 2024-08-31T06:19:22.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.199+0000 7f68ad400640 1 -- 172.21.15.16:0/2302705633 learned_addr learned my addr 172.21.15.16:0/2302705633 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:22.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.199+0000 7f68ae800640 1 --2- 172.21.15.16:0/2302705633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d59fe0a350 0x55d5a01c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:22.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.199+0000 7f68ade00640 1 --2- 172.21.15.16:0/2302705633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d59fde1490 0x55d5a01bc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:22.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.200+0000 7f68ad400640 1 -- 172.21.15.16:0/2302705633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d59fe0a350 msgr2=0x55d5a01c82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:22.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.200+0000 7f68ad400640 1 --2- 172.21.15.16:0/2302705633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d59fe0a350 0x55d5a01c82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:22.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.200+0000 7f68ad400640 1 -- 172.21.15.16:0/2302705633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d59fde1490 msgr2=0x55d5a01bc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:22.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.200+0000 7f68ad400640 1 --2- 172.21.15.16:0/2302705633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d59fde1490 0x55d5a01bc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:22.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.200+0000 7f68ad400640 1 -- 172.21.15.16:0/2302705633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5a00e61b0 con 0x55d59fde9670 2024-08-31T06:19:22.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.201+0000 7f68ade00640 1 --2- 172.21.15.16:0/2302705633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d59fde1490 0x55d5a01bc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:19:22.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.201+0000 7f68ad400640 1 --2- 172.21.15.16:0/2302705633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d59fde9670 0x55d5a01bc8a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f68a806b4a0 tx=0x7f689802a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a85b08593fbc122 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:22.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.201+0000 7f688b400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:19:22.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.202+0000 7f688b400640 1 -- 172.21.15.16:0/2302705633 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689804c070 con 0x55d59fde9670 2024-08-31T06:19:22.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.202+0000 7f688b400640 1 -- 172.21.15.16:0/2302705633 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8470+0+0 (secure 0 0 0) 0x7f68980037f0 con 0x55d59fde9670 2024-08-31T06:19:22.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.202+0000 7f68b75ed740 1 -- 172.21.15.16:0/2302705633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68a40ac2d0 con 0x55d59fde9670 2024-08-31T06:19:22.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.203+0000 7f688b400640 1 -- 172.21.15.16:0/2302705633 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f6898058050 con 0x55d59fde9670 2024-08-31T06:19:22.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.203+0000 7f68b75ed740 10 client.17979 fetch_fsmap learned FSMap version 49 2024-08-31T06:19:22.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.203+0000 7f68b75ed740 1 -- 172.21.15.16:0/2302705633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55d5a01caf90 con 0x55d59fde9670 2024-08-31T06:19:22.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.203+0000 7f688b400640 1 -- 172.21.15.16:0/2302705633 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68a40ac2d0 con 0x55d59fde9670 2024-08-31T06:19:22.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.203+0000 7f688b400640 1 -- 172.21.15.16:0/2302705633 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f6898047040 con 0x55d59fde9670 2024-08-31T06:19:22.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.204+0000 7f68b75ed740 10 client.17979 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:19:22.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.204+0000 7f68b75ed740 10 client.17979 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:19:22.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.204+0000 7f68b75ed740 20 client.17979 populate_metadata read hostname 'smithi016' 2024-08-31T06:19:22.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.204+0000 7f68b75ed740 1 --2- 172.21.15.16:0/2302705633 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d59fdc8cd0 0x55d5a01ce500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:22.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.204+0000 7f68b75ed740 4 client.17979 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:19:22.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.204+0000 7f68b75ed740 1 -- 172.21.15.16:0/2302705633 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5a01286a0 con 0x55d59fdc8cd0 2024-08-31T06:19:22.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.204+0000 7f68ade00640 1 --2- 172.21.15.16:0/2302705633 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d59fdc8cd0 0x55d5a01ce500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:22.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.205+0000 7f68ade00640 1 --2- 172.21.15.16:0/2302705633 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d59fdc8cd0 0x55d5a01ce500 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce62d44c3b37a628 server_cookie=445c242a69253edb in_seq=0 out_seq=0 2024-08-31T06:19:22.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.205+0000 7f688b400640 10 client.17979 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:19:22.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.206+0000 7f688b400640 1 -- 172.21.15.16:0/2302705633 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d5a01286a0 con 0x55d59fdc8cd0 2024-08-31T06:19:22.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.206+0000 7f688b400640 10 client.17979 handle_command_reply: tid=0 2024-08-31T06:19:22.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.206+0000 7f68b75ed740 1 client.17979 shutdown 2024-08-31T06:19:22.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.206+0000 7f68b75ed740 1 -- 172.21.15.16:0/2302705633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d59fde9670 msgr2=0x55d5a01bc8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:22.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.206+0000 7f68b75ed740 1 --2- 172.21.15.16:0/2302705633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d59fde9670 0x55d5a01bc8a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f68a806b4a0 tx=0x7f689802a3f0 comp rx=0 tx=0).stop 2024-08-31T06:19:22.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.206+0000 7f68b75ed740 1 -- 172.21.15.16:0/2302705633 shutdown_connections 2024-08-31T06:19:22.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.206+0000 7f68b75ed740 1 --2- 172.21.15.16:0/2302705633 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d59fdc8cd0 0x55d5a01ce500 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:22.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.206+0000 7f68b75ed740 1 --2- 172.21.15.16:0/2302705633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d59fe0a350 0x55d5a01c82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:22.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.206+0000 7f68b75ed740 1 --2- 172.21.15.16:0/2302705633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d59fde9670 0x55d5a01bc8a0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:22.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.206+0000 7f68b75ed740 1 --2- 172.21.15.16:0/2302705633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d59fde1490 0x55d5a01bc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:22.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.206+0000 7f68b75ed740 1 -- 172.21.15.16:0/2302705633 >> 172.21.15.16:0/2302705633 conn(0x55d59fdfe740 msgr2=0x55d5a01b3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:22.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.207+0000 7f688b400640 0 client.17979 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:19:22.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.207+0000 7f68b75ed740 1 -- 172.21.15.16:0/2302705633 shutdown_connections 2024-08-31T06:19:22.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.207+0000 7f68b75ed740 1 -- 172.21.15.16:0/2302705633 wait complete. 2024-08-31T06:19:22.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.207+0000 7f68b75ed740 20 client.17979 trim_cache size 0 max 16384 2024-08-31T06:19:22.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.245+0000 7f68b75ed740 1 Processor -- start 2024-08-31T06:19:22.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.245+0000 7f68b75ed740 1 -- start start 2024-08-31T06:19:22.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.246+0000 7f68b75ed740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d59fde1490 0x55d5a01b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:22.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.246+0000 7f68b75ed740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d59fe0a350 0x55d5a01bb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:22.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.246+0000 7f68b75ed740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d59fd9e080 0x55d5a01c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:22.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.246+0000 7f68b75ed740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f68a81b81c0 con 0x55d59fd9e080 2024-08-31T06:19:22.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.246+0000 7f68b75ed740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f68a81b59d0 con 0x55d59fe0a350 2024-08-31T06:19:22.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.246+0000 7f68b75ed740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f68a81b3840 con 0x55d59fde1490 2024-08-31T06:19:22.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.247+0000 7f68ade00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d59fde1490 0x55d5a01b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:22.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.247+0000 7f68ade00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d59fde1490 0x55d5a01b3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:36406/0 (socket says 172.21.15.16:36406) 2024-08-31T06:19:22.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.247+0000 7f68ade00640 1 -- 172.21.15.16:0/2453660301 learned_addr learned my addr 172.21.15.16:0/2453660301 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:22.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.247+0000 7f68ad400640 1 --2- 172.21.15.16:0/2453660301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d59fe0a350 0x55d5a01bb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:22.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.247+0000 7f68ad400640 1 -- 172.21.15.16:0/2453660301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d59fde1490 msgr2=0x55d5a01b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:22.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.247+0000 7f68ad400640 1 --2- 172.21.15.16:0/2453660301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d59fde1490 0x55d5a01b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:22.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.247+0000 7f68ad400640 1 -- 172.21.15.16:0/2453660301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d59fd9e080 msgr2=0x55d5a01c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:22.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.247+0000 7f68ad400640 1 --2- 172.21.15.16:0/2453660301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d59fd9e080 0x55d5a01c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:22.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.247+0000 7f68ad400640 1 -- 172.21.15.16:0/2453660301 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5a01bd420 con 0x55d59fe0a350 2024-08-31T06:19:22.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.251+0000 7f68ad400640 1 --2- 172.21.15.16:0/2453660301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d59fe0a350 0x55d5a01bb0f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f689803d000 tx=0x7f6898004510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=862a2dbe5d9f1d9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:22.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.251+0000 7f6896600640 1 -- 172.21.15.16:0/2453660301 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689804c070 con 0x55d59fe0a350 2024-08-31T06:19:22.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.252+0000 7f6896600640 1 -- 172.21.15.16:0/2453660301 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6898032cf0 con 0x55d59fe0a350 2024-08-31T06:19:22.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.252+0000 7f6896600640 1 -- 172.21.15.16:0/2453660301 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6898035050 con 0x55d59fe0a350 2024-08-31T06:19:22.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.255+0000 7f68b75ed740 1 -- 172.21.15.16:0/2453660301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d59fe0a350 msgr2=0x55d5a01bb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:22.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.255+0000 7f68b75ed740 1 --2- 172.21.15.16:0/2453660301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d59fe0a350 0x55d5a01bb0f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f689803d000 tx=0x7f6898004510 comp rx=0 tx=0).stop 2024-08-31T06:19:22.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.255+0000 7f68b75ed740 1 -- 172.21.15.16:0/2453660301 shutdown_connections 2024-08-31T06:19:22.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.255+0000 7f68b75ed740 1 --2- 172.21.15.16:0/2453660301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d59fd9e080 0x55d5a01c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:22.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.255+0000 7f68b75ed740 1 --2- 172.21.15.16:0/2453660301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d59fe0a350 0x55d5a01bb0f0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:22.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.255+0000 7f68b75ed740 1 --2- 172.21.15.16:0/2453660301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d59fde1490 0x55d5a01b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:22.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.255+0000 7f68b75ed740 1 -- 172.21.15.16:0/2453660301 >> 172.21.15.16:0/2453660301 conn(0x55d59fdfe740 msgr2=0x55d5a01ba9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:22.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.256+0000 7f68b75ed740 1 -- 172.21.15.16:0/2453660301 shutdown_connections 2024-08-31T06:19:22.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.257+0000 7f68b75ed740 1 -- 172.21.15.16:0/2453660301 wait complete. 2024-08-31T06:19:22.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.257+0000 7f68b75ed740 1 -- start start 2024-08-31T06:19:22.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.258+0000 7f68b75ed740 1 Processor -- start 2024-08-31T06:19:22.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.259+0000 7f68b75ed740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d59fd9e080 0x55d5a01c0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:22.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.260+0000 7f68b75ed740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d59fde1490 0x55d5a01c1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:22.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.260+0000 7f68ade00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d59fd9e080 0x55d5a01c0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:22.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.260+0000 7f68ad400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d59fde1490 0x55d5a01c1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:22.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.260+0000 7f68ade00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d59fd9e080 0x55d5a01c0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:36412/0 (socket says 172.21.15.16:36412) 2024-08-31T06:19:22.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.260+0000 7f68ad400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d59fde1490 0x55d5a01c1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:45100/0 (socket says 172.21.15.16:45100) 2024-08-31T06:19:22.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.260+0000 7f68ade00640 1 -- 172.21.15.16:0/3269846395 learned_addr learned my addr 172.21.15.16:0/3269846395 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:22.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.261+0000 7f68b75ed740 1 --2- 172.21.15.16:0/3269846395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d59fe0a350 0x55d5a0242f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:22.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.262+0000 7f68b75ed740 1 -- 172.21.15.16:0/3269846395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f68a81b3e90 con 0x55d59fe0a350 2024-08-31T06:19:22.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.262+0000 7f68b75ed740 1 -- 172.21.15.16:0/3269846395 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f68a81b5300 con 0x55d59fde1490 2024-08-31T06:19:22.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.262+0000 7f68b75ed740 1 -- 172.21.15.16:0/3269846395 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f68a81b4560 con 0x55d59fd9e080 2024-08-31T06:19:22.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.262+0000 7f68b75ed740 10 client.0 _ll_register_callbacks cb 0x7f68a40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:19:22.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.263+0000 7f68ade00640 1 -- 172.21.15.16:0/3269846395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d59fde1490 msgr2=0x55d5a01c1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:22.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.263+0000 7f68ade00640 1 --2- 172.21.15.16:0/3269846395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d59fde1490 0x55d5a01c1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:22.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.264+0000 7f68ade00640 1 -- 172.21.15.16:0/3269846395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d59fe0a350 msgr2=0x55d5a0242f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:22.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.264+0000 7f68ade00640 1 --2- 172.21.15.16:0/3269846395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d59fe0a350 0x55d5a0242f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:22.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.265+0000 7f68ade00640 1 -- 172.21.15.16:0/3269846395 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5a0243580 con 0x55d59fd9e080 2024-08-31T06:19:22.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.265+0000 7f68ad400640 1 --2- 172.21.15.16:0/3269846395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d59fde1490 0x55d5a01c1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:19:22.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.265+0000 7f68ade00640 1 --2- 172.21.15.16:0/3269846395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d59fd9e080 0x55d5a01c0b60 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f689c00d010 tx=0x7f689c0127c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3edf2a645f723fa server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:22.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.266+0000 7f688b400640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:19:22.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.267+0000 7f688b400640 1 -- 172.21.15.16:0/3269846395 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689c005220 con 0x55d59fd9e080 2024-08-31T06:19:22.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.267+0000 7f688b400640 1 -- 172.21.15.16:0/3269846395 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8470+0+0 (secure 0 0 0) 0x7f689c0053c0 con 0x55d59fd9e080 2024-08-31T06:19:22.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.267+0000 7f68b75ed740 1 -- 172.21.15.16:0/3269846395 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6898032cf0 con 0x55d59fd9e080 2024-08-31T06:19:22.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.270+0000 7f688b400640 1 -- 172.21.15.16:0/3269846395 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f689c0055c0 con 0x55d59fd9e080 2024-08-31T06:19:22.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.270+0000 7f68b75ed740 10 client.28384 fetch_fsmap learned FSMap version 49 2024-08-31T06:19:22.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.270+0000 7f68b75ed740 1 -- 172.21.15.16:0/3269846395 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55d5a024a960 con 0x55d59fd9e080 2024-08-31T06:19:22.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.273+0000 7f688b400640 1 -- 172.21.15.16:0/3269846395 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6898032cf0 con 0x55d59fd9e080 2024-08-31T06:19:22.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.273+0000 7f688b400640 1 -- 172.21.15.16:0/3269846395 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f689c00f040 con 0x55d59fd9e080 2024-08-31T06:19:22.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.273+0000 7f68b75ed740 10 client.28384 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:19:22.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.273+0000 7f68b75ed740 10 client.28384 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:19:22.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.273+0000 7f68b75ed740 20 client.28384 populate_metadata read hostname 'smithi016' 2024-08-31T06:19:22.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.273+0000 7f68b75ed740 1 --2- 172.21.15.16:0/3269846395 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d5a024b1a0 0x55d5a024d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:22.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.273+0000 7f68b75ed740 4 client.28384 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T06:19:22.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.273+0000 7f68b75ed740 1 -- 172.21.15.16:0/3269846395 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d5a0124140 con 0x55d5a024b1a0 2024-08-31T06:19:22.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.273+0000 7f68ad400640 1 --2- 172.21.15.16:0/3269846395 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d5a024b1a0 0x55d5a024d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:22.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.274+0000 7f688b400640 10 client.28384 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:19:22.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.275+0000 7f68ad400640 1 --2- 172.21.15.16:0/3269846395 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d5a024b1a0 0x55d5a024d5c0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cc007d753864c68 server_cookie=29836c77c07f2eec in_seq=0 out_seq=0 2024-08-31T06:19:22.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.276+0000 7f688b400640 1 -- 172.21.15.16:0/3269846395 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d5a0124140 con 0x55d5a024b1a0 2024-08-31T06:19:22.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.276+0000 7f688b400640 10 client.28384 handle_command_reply: tid=0 2024-08-31T06:19:22.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.277+0000 7f68b75ed740 1 client.28384 shutdown 2024-08-31T06:19:22.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.277+0000 7f68b75ed740 1 -- 172.21.15.16:0/3269846395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d59fd9e080 msgr2=0x55d5a01c0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:22.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.277+0000 7f68b75ed740 1 --2- 172.21.15.16:0/3269846395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d59fd9e080 0x55d5a01c0b60 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f689c00d010 tx=0x7f689c0127c0 comp rx=0 tx=0).stop 2024-08-31T06:19:22.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.278+0000 7f68b75ed740 1 -- 172.21.15.16:0/3269846395 shutdown_connections 2024-08-31T06:19:22.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.278+0000 7f68b75ed740 1 --2- 172.21.15.16:0/3269846395 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d5a024b1a0 0x55d5a024d5c0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:22.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.278+0000 7f68b75ed740 1 --2- 172.21.15.16:0/3269846395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d59fe0a350 0x55d5a0242f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:22.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.278+0000 7f68b75ed740 1 --2- 172.21.15.16:0/3269846395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d59fde1490 0x55d5a01c1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:22.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.278+0000 7f68b75ed740 1 --2- 172.21.15.16:0/3269846395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d59fd9e080 0x55d5a01c0b60 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:22.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.278+0000 7f688b400640 0 client.28384 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:19:22.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.278+0000 7f68b75ed740 1 -- 172.21.15.16:0/3269846395 >> 172.21.15.16:0/3269846395 conn(0x55d59fdfe740 msgr2=0x55d5a01ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:22.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.279+0000 7f68b75ed740 1 -- 172.21.15.16:0/3269846395 shutdown_connections 2024-08-31T06:19:22.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.279+0000 7f68b75ed740 1 -- 172.21.15.16:0/3269846395 wait complete. 2024-08-31T06:19:22.282 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:19:22.282 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-08-31T06:19:22.283 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "f1cdcb27-e3e4-4dc8-a0ae-7ace761d04c5", 2024-08-31T06:19:22.283 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-08-31T06:19:22.283 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:19:22.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.279+0000 7f68b75ed740 20 client.28384 trim_cache size 0 max 16384 2024-08-31T06:19:22.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.282+0000 7f68af200640 1 -- 172.21.15.16:0/1641736529 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f68900b4da0 msgr2=0x7f68900b72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:22.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.282+0000 7f68af200640 1 --2- 172.21.15.16:0/1641736529 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f68900b4da0 0x7f68900b72b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6898006010 tx=0x7f6898004350 comp rx=0 tx=0).stop 2024-08-31T06:19:22.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.282+0000 7f68af200640 1 -- 172.21.15.16:0/1641736529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68a810fcb0 msgr2=0x7f68a81ac1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:22.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.282+0000 7f68af200640 1 --2- 172.21.15.16:0/1641736529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68a810fcb0 0x7f68a81ac1b0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f68a400eb10 tx=0x7f68a400c2d0 comp rx=0 tx=0).stop 2024-08-31T06:19:22.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.282+0000 7f68af200640 1 -- 172.21.15.16:0/1641736529 shutdown_connections 2024-08-31T06:19:22.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.282+0000 7f68af200640 1 --2- 172.21.15.16:0/1641736529 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f68900b4da0 0x7f68900b72b0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:22.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.282+0000 7f68af200640 1 --2- 172.21.15.16:0/1641736529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68a8075ea0 0x7f68a810e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:22.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.282+0000 7f68af200640 1 --2- 172.21.15.16:0/1641736529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68a810fcb0 0x7f68a81ac1b0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:22.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.282+0000 7f68af200640 1 --2- 172.21.15.16:0/1641736529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68a8075550 0x7f68a810e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:22.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.282+0000 7f68af200640 1 -- 172.21.15.16:0/1641736529 >> 172.21.15.16:0/1641736529 conn(0x7f68a80ff1e0 msgr2=0x7f68a8106850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:22.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.283+0000 7f68af200640 1 -- 172.21.15.16:0/1641736529 shutdown_connections 2024-08-31T06:19:22.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:22.283+0000 7f68af200640 1 -- 172.21.15.16:0/1641736529 wait complete. 2024-08-31T06:19:23.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:22 smithi155 ceph-mon[21742]: pgmap v742: 97 pgs: 97 active+clean; 3.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 8.3 MiB/s wr, 749 op/s 2024-08-31T06:19:23.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:22 smithi155 ceph-mon[21742]: scrub queued for path: / 2024-08-31T06:19:23.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:22 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:19:23.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:22 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:19:23.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:22 smithi016 ceph-mon[17194]: pgmap v742: 97 pgs: 97 active+clean; 3.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 8.3 MiB/s wr, 749 op/s 2024-08-31T06:19:23.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:22 smithi016 ceph-mon[17194]: scrub queued for path: / 2024-08-31T06:19:23.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:22 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:19:23.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:22 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:19:23.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:22 smithi184 ceph-mon[22331]: pgmap v742: 97 pgs: 97 active+clean; 3.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 8.3 MiB/s wr, 749 op/s 2024-08-31T06:19:23.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:22 smithi184 ceph-mon[22331]: scrub queued for path: / 2024-08-31T06:19:23.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:22 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:19:23.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:22 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:19:24.120 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:23 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:19:24.121 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:23 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:19:24.121 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:23 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:19:24.121 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:23 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:19:24.121 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:23 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:19:24.121 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:23 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:19:24.121 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:23 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:19:24.121 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:23 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:19:24.121 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:23 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:19:24.121 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:23 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:19:24.121 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:23 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:19:24.121 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:23 smithi016 ceph-mon[17194]: pgmap v743: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 8.5 MiB/s wr, 771 op/s 2024-08-31T06:19:24.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:23 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:19:24.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:23 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:19:24.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:23 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:19:24.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:23 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:19:24.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:23 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:19:24.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:23 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:19:24.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:23 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:19:24.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:23 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:19:24.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:23 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:19:24.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:23 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:19:24.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:23 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:19:24.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:23 smithi155 ceph-mon[21742]: pgmap v743: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 8.5 MiB/s wr, 771 op/s 2024-08-31T06:19:24.188 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T06:19:24.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:23 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:19:24.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:23 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:19:24.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:23 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:19:24.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:23 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:19:24.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:23 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:19:24.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:23 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:19:24.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:23 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:19:24.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:23 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:19:24.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:23 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:19:24.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:23 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:19:24.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:23 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:19:24.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:23 smithi184 ceph-mon[22331]: pgmap v743: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 8.5 MiB/s wr, 771 op/s 2024-08-31T06:19:24.789 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:19:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.577+0000 7f09f6600640 1 -- 172.21.15.16:0/957635393 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09f0109250 msgr2=0x7f09f0109660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.577+0000 7f09f6600640 1 --2- 172.21.15.16:0/957635393 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09f0109250 0x7f09f0109660 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f09dc009860 tx=0x7f09dc02a340 comp rx=0 tx=0).stop 2024-08-31T06:19:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.579+0000 7f09f6600640 1 -- 172.21.15.16:0/957635393 shutdown_connections 2024-08-31T06:19:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.579+0000 7f09f6600640 1 --2- 172.21.15.16:0/957635393 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09f0109250 0x7f09f0109660 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.579+0000 7f09f6600640 1 --2- 172.21.15.16:0/957635393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09f0109ba0 0x7f09f01103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.579+0000 7f09f6600640 1 --2- 172.21.15.16:0/957635393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09f0103910 0x7f09f0103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.579+0000 7f09f6600640 1 -- 172.21.15.16:0/957635393 >> 172.21.15.16:0/957635393 conn(0x7f09f00ff380 msgr2=0x7f09f01017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:25.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.580+0000 7f09f6600640 1 -- 172.21.15.16:0/957635393 shutdown_connections 2024-08-31T06:19:25.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.580+0000 7f09f6600640 1 -- 172.21.15.16:0/957635393 wait complete. 2024-08-31T06:19:25.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.582+0000 7f09f6600640 1 Processor -- start 2024-08-31T06:19:25.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.582+0000 7f09f6600640 1 -- start start 2024-08-31T06:19:25.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.583+0000 7f09f6600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09f0103910 0x7f09f0198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:25.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.583+0000 7f09f6600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09f0109250 0x7f09f0199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:25.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.584+0000 7f09ef400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09f0103910 0x7f09f0198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:25.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.584+0000 7f09f6600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09f0109ba0 0x7f09f019fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:25.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.584+0000 7f09f6600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f09f0113860 con 0x7f09f0109250 2024-08-31T06:19:25.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.584+0000 7f09f6600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f09f01136e0 con 0x7f09f0109ba0 2024-08-31T06:19:25.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.584+0000 7f09f6600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f09f01139e0 con 0x7f09f0103910 2024-08-31T06:19:25.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.584+0000 7f09ef400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09f0103910 0x7f09f0198cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:36426/0 (socket says 172.21.15.16:36426) 2024-08-31T06:19:25.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.584+0000 7f09ef400640 1 -- 172.21.15.16:0/2495207686 learned_addr learned my addr 172.21.15.16:0/2495207686 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:25.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.585+0000 7f09eea00640 1 --2- 172.21.15.16:0/2495207686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09f0109250 0x7f09f0199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:25.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.585+0000 7f09efe00640 1 --2- 172.21.15.16:0/2495207686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09f0109ba0 0x7f09f019fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:25.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.585+0000 7f09ef400640 1 -- 172.21.15.16:0/2495207686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09f0109ba0 msgr2=0x7f09f019fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:25.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.585+0000 7f09ef400640 1 --2- 172.21.15.16:0/2495207686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09f0109ba0 0x7f09f019fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:25.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.585+0000 7f09ef400640 1 -- 172.21.15.16:0/2495207686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09f0109250 msgr2=0x7f09f0199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:25.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.587+0000 7f09ef400640 1 --2- 172.21.15.16:0/2495207686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09f0109250 0x7f09f0199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:25.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.587+0000 7f09ef400640 1 -- 172.21.15.16:0/2495207686 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09f01a0270 con 0x7f09f0103910 2024-08-31T06:19:25.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.587+0000 7f09eea00640 1 --2- 172.21.15.16:0/2495207686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09f0109250 0x7f09f0199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:19:25.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.587+0000 7f09ef400640 1 --2- 172.21.15.16:0/2495207686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09f0103910 0x7f09f0198cd0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f09e000e9c0 tx=0x7f09e000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:25.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.588+0000 7f09e7e00640 1 -- 172.21.15.16:0/2495207686 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e000ccc0 con 0x7f09f0103910 2024-08-31T06:19:25.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.588+0000 7f09e7e00640 1 -- 172.21.15.16:0/2495207686 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09e00054d0 con 0x7f09f0103910 2024-08-31T06:19:25.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.588+0000 7f09f6600640 1 -- 172.21.15.16:0/2495207686 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09f01a0560 con 0x7f09f0103910 2024-08-31T06:19:25.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.589+0000 7f09f6600640 1 -- 172.21.15.16:0/2495207686 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09f01a0af0 con 0x7f09f0103910 2024-08-31T06:19:25.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.589+0000 7f09e7e00640 1 -- 172.21.15.16:0/2495207686 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e0010430 con 0x7f09f0103910 2024-08-31T06:19:25.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.600+0000 7f09fd1e2740 1 Processor -- start 2024-08-31T06:19:25.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.602+0000 7f09e7e00640 1 -- 172.21.15.16:0/2495207686 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f09e00046c0 con 0x7f09f0103910 2024-08-31T06:19:25.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.602+0000 7f09fd1e2740 1 -- start start 2024-08-31T06:19:25.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.602+0000 7f09fd1e2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55602e6b5070 0x55602ea5a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:25.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.603+0000 7f09fd1e2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55602e68c1b0 0x55602ea5adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:25.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.603+0000 7f09fd1e2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55602e694390 0x55602ea5b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:25.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.603+0000 7f09fd1e2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55602e9d1770 con 0x55602e68c1b0 2024-08-31T06:19:25.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.603+0000 7f09fd1e2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55602e9c0d00 con 0x55602e694390 2024-08-31T06:19:25.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.603+0000 7f09fd1e2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55602e9c4cb0 con 0x55602e6b5070 2024-08-31T06:19:25.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.603+0000 7f09eea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55602e694390 0x55602ea5b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:25.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.603+0000 7f09eea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55602e694390 0x55602ea5b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:45128/0 (socket says 172.21.15.16:45128) 2024-08-31T06:19:25.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.603+0000 7f09eea00640 1 -- 172.21.15.16:0/2610260714 learned_addr learned my addr 172.21.15.16:0/2610260714 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:25.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.604+0000 7f09eea00640 1 -- 172.21.15.16:0/2610260714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55602e6b5070 msgr2=0x55602ea5a870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:19:25.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.604+0000 7f09eea00640 1 --2- 172.21.15.16:0/2610260714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55602e6b5070 0x55602ea5a870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:25.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.604+0000 7f09eea00640 1 -- 172.21.15.16:0/2610260714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55602e68c1b0 msgr2=0x55602ea5adb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:19:25.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.604+0000 7f09eea00640 1 --2- 172.21.15.16:0/2610260714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55602e68c1b0 0x55602ea5adb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:25.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.604+0000 7f09eea00640 1 -- 172.21.15.16:0/2610260714 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55602e720650 con 0x55602e694390 2024-08-31T06:19:25.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.607+0000 7f09eea00640 1 --2- 172.21.15.16:0/2610260714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55602e694390 0x55602ea5b2f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f09dc02a860 tx=0x7f09dc030b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45f3be94f2fca46c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:25.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.607+0000 7f09e4c00640 1 -- 172.21.15.16:0/2610260714 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09dc040020 con 0x55602e694390 2024-08-31T06:19:25.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.607+0000 7f09e7e00640 1 --2- 172.21.15.16:0/2495207686 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f09d40b4dd0 0x7f09d40b72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:25.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.608+0000 7f09e4c00640 1 -- 172.21.15.16:0/2610260714 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09dc038040 con 0x55602e694390 2024-08-31T06:19:25.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.608+0000 7f09e7e00640 1 -- 172.21.15.16:0/2495207686 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8470+0+0 (secure 0 0 0) 0x7f09e00e5570 con 0x7f09f0103910 2024-08-31T06:19:25.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.609+0000 7f09fd1e2740 1 -- 172.21.15.16:0/2610260714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55602e694390 msgr2=0x55602ea5b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:25.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.609+0000 7f09fd1e2740 1 --2- 172.21.15.16:0/2610260714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55602e694390 0x55602ea5b2f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f09dc02a860 tx=0x7f09dc030b50 comp rx=0 tx=0).stop 2024-08-31T06:19:25.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.609+0000 7f09efe00640 1 --2- 172.21.15.16:0/2495207686 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f09d40b4dd0 0x7f09d40b72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:25.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.609+0000 7f09fd1e2740 1 -- 172.21.15.16:0/2610260714 shutdown_connections 2024-08-31T06:19:25.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.609+0000 7f09fd1e2740 1 --2- 172.21.15.16:0/2610260714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55602e694390 0x55602ea5b2f0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:25.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.609+0000 7f09fd1e2740 1 --2- 172.21.15.16:0/2610260714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55602e68c1b0 0x55602ea5adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:25.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.609+0000 7f09fd1e2740 1 --2- 172.21.15.16:0/2610260714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55602e6b5070 0x55602ea5a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:25.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.609+0000 7f09fd1e2740 1 -- 172.21.15.16:0/2610260714 >> 172.21.15.16:0/2610260714 conn(0x55602e6a9460 msgr2=0x55602ea58760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:25.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.610+0000 7f09efe00640 1 --2- 172.21.15.16:0/2495207686 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f09d40b4dd0 0x7f09d40b72e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f09e8005fd0 tx=0x7f09e80096a0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:25.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.612+0000 7f09fd1e2740 1 -- 172.21.15.16:0/2610260714 shutdown_connections 2024-08-31T06:19:25.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.612+0000 7f09fd1e2740 1 -- 172.21.15.16:0/2610260714 wait complete. 2024-08-31T06:19:25.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.613+0000 7f09fd1e2740 1 -- start start 2024-08-31T06:19:25.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.613+0000 7f09fd1e2740 1 Processor -- start 2024-08-31T06:19:25.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.615+0000 7f09fd1e2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55602e68c1b0 0x55602ea65f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:25.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.615+0000 7f09fd1e2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55602e6b5070 0x55602ea66450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:25.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.615+0000 7f09ef400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55602e68c1b0 0x55602ea65f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:25.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.615+0000 7f09ef400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55602e68c1b0 0x55602ea65f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:45132/0 (socket says 172.21.15.16:45132) 2024-08-31T06:19:25.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.615+0000 7f09fd1e2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55602e6739f0 0x55602ea66990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:25.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.616+0000 7f09ef400640 1 -- 172.21.15.16:0/507950682 learned_addr learned my addr 172.21.15.16:0/507950682 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:25.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.616+0000 7f09fd1e2740 1 -- 172.21.15.16:0/507950682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55602e823700 con 0x55602e6739f0 2024-08-31T06:19:25.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.616+0000 7f09eea00640 1 --2- 172.21.15.16:0/507950682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55602e6b5070 0x55602ea66450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:25.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.616+0000 7f09fd1e2740 1 -- 172.21.15.16:0/507950682 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55602e947cb0 con 0x55602e68c1b0 2024-08-31T06:19:25.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.616+0000 7f09fd1e2740 1 -- 172.21.15.16:0/507950682 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55602ea69090 con 0x55602e6b5070 2024-08-31T06:19:25.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.616+0000 7f09efe00640 1 --2- 172.21.15.16:0/507950682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55602e6739f0 0x55602ea66990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:25.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.616+0000 7f09fd1e2740 10 client.0 _ll_register_callbacks cb 0x55602e96d430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:19:25.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.617+0000 7f09ef400640 1 -- 172.21.15.16:0/507950682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55602e6b5070 msgr2=0x55602ea66450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:25.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.617+0000 7f09ef400640 1 --2- 172.21.15.16:0/507950682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55602e6b5070 0x55602ea66450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:25.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.617+0000 7f09ef400640 1 -- 172.21.15.16:0/507950682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55602e6739f0 msgr2=0x55602ea66990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:25.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.617+0000 7f09ef400640 1 --2- 172.21.15.16:0/507950682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55602e6739f0 0x55602ea66990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:25.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.617+0000 7f09ef400640 1 -- 172.21.15.16:0/507950682 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55602e988870 con 0x55602e68c1b0 2024-08-31T06:19:25.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.617+0000 7f09ef400640 1 --2- 172.21.15.16:0/507950682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55602e68c1b0 0x55602ea65f10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f09e000c6a0 tx=0x7f09e00ab460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7192ebe80331a5ce server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:25.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.619+0000 7f09d9600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:19:25.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.619+0000 7f09d9600640 1 -- 172.21.15.16:0/507950682 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e00ac330 con 0x55602e68c1b0 2024-08-31T06:19:25.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.619+0000 7f09d9600640 1 -- 172.21.15.16:0/507950682 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8470+0+0 (secure 0 0 0) 0x7f09e00e2f10 con 0x55602e68c1b0 2024-08-31T06:19:25.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.619+0000 7f09fd1e2740 1 -- 172.21.15.16:0/507950682 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55602e9d36c0 con 0x55602e68c1b0 2024-08-31T06:19:25.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.619+0000 7f09d9600640 1 -- 172.21.15.16:0/507950682 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f09e00e3110 con 0x55602e68c1b0 2024-08-31T06:19:25.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.620+0000 7f09fd1e2740 10 client.28373 fetch_fsmap learned FSMap version 49 2024-08-31T06:19:25.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.620+0000 7f09fd1e2740 1 -- 172.21.15.16:0/507950682 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55602ea74e40 con 0x55602e68c1b0 2024-08-31T06:19:25.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.621+0000 7f09d9600640 1 -- 172.21.15.16:0/507950682 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55602e9d36c0 con 0x55602e68c1b0 2024-08-31T06:19:25.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.621+0000 7f09d9600640 1 -- 172.21.15.16:0/507950682 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f09e00b4ac0 con 0x55602e68c1b0 2024-08-31T06:19:25.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.621+0000 7f09fd1e2740 10 client.28373 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:19:25.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.621+0000 7f09fd1e2740 10 client.28373 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:19:25.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.621+0000 7f09fd1e2740 20 client.28373 populate_metadata read hostname 'smithi016' 2024-08-31T06:19:25.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.621+0000 7f09fd1e2740 1 --2- 172.21.15.16:0/507950682 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55602e648da0 0x55602ea78310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:25.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.621+0000 7f09fd1e2740 4 client.28373 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:19:25.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.621+0000 7f09fd1e2740 1 -- 172.21.15.16:0/507950682 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55602e9cf160 con 0x55602e648da0 2024-08-31T06:19:25.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.622+0000 7f09eea00640 1 --2- 172.21.15.16:0/507950682 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55602e648da0 0x55602ea78310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:25.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.623+0000 7f09d9600640 10 client.28373 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:19:25.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.624+0000 7f09eea00640 1 --2- 172.21.15.16:0/507950682 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55602e648da0 0x55602ea78310 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a468ed3cb7bd3ed2 server_cookie=56c98b70389e3fa4 in_seq=0 out_seq=0 2024-08-31T06:19:25.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.625+0000 7f09d9600640 1 -- 172.21.15.16:0/507950682 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55602e9cf160 con 0x55602e648da0 2024-08-31T06:19:25.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.625+0000 7f09d9600640 10 client.28373 handle_command_reply: tid=0 2024-08-31T06:19:25.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.625+0000 7f09fd1e2740 1 client.28373 shutdown 2024-08-31T06:19:25.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.628+0000 7f09fd1e2740 1 -- 172.21.15.16:0/507950682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55602e68c1b0 msgr2=0x55602ea65f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:25.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.628+0000 7f09fd1e2740 1 --2- 172.21.15.16:0/507950682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55602e68c1b0 0x55602ea65f10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f09e000c6a0 tx=0x7f09e00ab460 comp rx=0 tx=0).stop 2024-08-31T06:19:25.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.631+0000 7f09fd1e2740 1 -- 172.21.15.16:0/507950682 shutdown_connections 2024-08-31T06:19:25.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.631+0000 7f09fd1e2740 1 --2- 172.21.15.16:0/507950682 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55602e648da0 0x55602ea78310 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:25.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.631+0000 7f09fd1e2740 1 --2- 172.21.15.16:0/507950682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55602e6b5070 0x55602ea66450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:25.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.631+0000 7f09fd1e2740 1 --2- 172.21.15.16:0/507950682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55602e6739f0 0x55602ea66990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:25.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.631+0000 7f09fd1e2740 1 --2- 172.21.15.16:0/507950682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55602e68c1b0 0x55602ea65f10 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:25.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.631+0000 7f09d9600640 0 client.28373 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:19:25.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.631+0000 7f09fd1e2740 1 -- 172.21.15.16:0/507950682 >> 172.21.15.16:0/507950682 conn(0x55602e6a9460 msgr2=0x55602ea5da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:25.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.631+0000 7f09fd1e2740 1 -- 172.21.15.16:0/507950682 shutdown_connections 2024-08-31T06:19:25.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.631+0000 7f09fd1e2740 1 -- 172.21.15.16:0/507950682 wait complete. 2024-08-31T06:19:25.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.632+0000 7f09fd1e2740 20 client.28373 trim_cache size 0 max 16384 2024-08-31T06:19:25.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.665+0000 7f09fd1e2740 1 Processor -- start 2024-08-31T06:19:25.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.666+0000 7f09fd1e2740 1 -- start start 2024-08-31T06:19:25.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.666+0000 7f09fd1e2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55602e6739f0 0x55602ea57760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:25.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.666+0000 7f09fd1e2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55602e68c1b0 0x55602ea67b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:25.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.666+0000 7f09fd1e2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55602e6b5070 0x55602ea63100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:25.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.666+0000 7f09fd1e2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f09d0011170 con 0x55602e6739f0 2024-08-31T06:19:25.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.666+0000 7f09fd1e2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f09d000e980 con 0x55602e6b5070 2024-08-31T06:19:25.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.666+0000 7f09fd1e2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f09d000c7f0 con 0x55602e68c1b0 2024-08-31T06:19:25.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.667+0000 7f09efe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55602e6b5070 0x55602ea63100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:25.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.667+0000 7f09efe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55602e6b5070 0x55602ea63100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:45138/0 (socket says 172.21.15.16:45138) 2024-08-31T06:19:25.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.667+0000 7f09efe00640 1 -- 172.21.15.16:0/2096355389 learned_addr learned my addr 172.21.15.16:0/2096355389 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:25.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.667+0000 7f09eea00640 1 --2- 172.21.15.16:0/2096355389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55602e68c1b0 0x55602ea67b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:25.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.667+0000 7f09ef400640 1 --2- 172.21.15.16:0/2096355389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55602e6739f0 0x55602ea57760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:25.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.667+0000 7f09efe00640 1 -- 172.21.15.16:0/2096355389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55602e68c1b0 msgr2=0x55602ea67b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:25.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.667+0000 7f09efe00640 1 --2- 172.21.15.16:0/2096355389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55602e68c1b0 0x55602ea67b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:25.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.667+0000 7f09efe00640 1 -- 172.21.15.16:0/2096355389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55602e6739f0 msgr2=0x55602ea57760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:25.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.667+0000 7f09efe00640 1 --2- 172.21.15.16:0/2096355389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55602e6739f0 0x55602ea57760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:25.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.667+0000 7f09efe00640 1 -- 172.21.15.16:0/2096355389 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55602ea64410 con 0x55602e6b5070 2024-08-31T06:19:25.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.668+0000 7f09efe00640 1 --2- 172.21.15.16:0/2096355389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55602e6b5070 0x55602ea63100 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f09e80049b0 tx=0x7f09e8013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6993db652e354cf server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:25.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.668+0000 7f09dbe00640 1 -- 172.21.15.16:0/2096355389 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e8028670 con 0x55602e6b5070 2024-08-31T06:19:25.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.668+0000 7f09dbe00640 1 -- 172.21.15.16:0/2096355389 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09e8023070 con 0x55602e6b5070 2024-08-31T06:19:25.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.668+0000 7f09dbe00640 1 -- 172.21.15.16:0/2096355389 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e801fba0 con 0x55602e6b5070 2024-08-31T06:19:25.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.669+0000 7f09fd1e2740 1 -- 172.21.15.16:0/2096355389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55602e6b5070 msgr2=0x55602ea63100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:25.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.669+0000 7f09fd1e2740 1 --2- 172.21.15.16:0/2096355389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55602e6b5070 0x55602ea63100 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f09e80049b0 tx=0x7f09e8013000 comp rx=0 tx=0).stop 2024-08-31T06:19:25.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.670+0000 7f09fd1e2740 1 -- 172.21.15.16:0/2096355389 shutdown_connections 2024-08-31T06:19:25.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.670+0000 7f09fd1e2740 1 --2- 172.21.15.16:0/2096355389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55602e68c1b0 0x55602ea67b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:25.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.670+0000 7f09fd1e2740 1 --2- 172.21.15.16:0/2096355389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55602e6b5070 0x55602ea63100 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:25.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.670+0000 7f09fd1e2740 1 --2- 172.21.15.16:0/2096355389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55602e6739f0 0x55602ea57760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:25.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.670+0000 7f09fd1e2740 1 -- 172.21.15.16:0/2096355389 >> 172.21.15.16:0/2096355389 conn(0x55602e6a9460 msgr2=0x55602ea67430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:25.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.670+0000 7f09fd1e2740 1 -- 172.21.15.16:0/2096355389 shutdown_connections 2024-08-31T06:19:25.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.670+0000 7f09fd1e2740 1 -- 172.21.15.16:0/2096355389 wait complete. 2024-08-31T06:19:25.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.671+0000 7f09fd1e2740 1 -- start start 2024-08-31T06:19:25.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.672+0000 7f09fd1e2740 1 Processor -- start 2024-08-31T06:19:25.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.673+0000 7f09fd1e2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55602e6739f0 0x55602ea6ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:25.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.673+0000 7f09fd1e2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55602e68c1b0 0x55602ea6a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:25.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.673+0000 7f09fd1e2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55602e6b5070 0x55602ea6a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:25.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.673+0000 7f09fd1e2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f09d000ce40 con 0x55602e6b5070 2024-08-31T06:19:25.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.673+0000 7f09fd1e2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f09d000e2b0 con 0x55602e6739f0 2024-08-31T06:19:25.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.673+0000 7f09fd1e2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f09d000d510 con 0x55602e68c1b0 2024-08-31T06:19:25.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.673+0000 7f09fd1e2740 10 client.0 _ll_register_callbacks cb 0x7f09e00aba90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:19:25.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.673+0000 7f09ef400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55602e6739f0 0x55602ea6ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:25.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.673+0000 7f09ef400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55602e6739f0 0x55602ea6ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:45144/0 (socket says 172.21.15.16:45144) 2024-08-31T06:19:25.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.673+0000 7f09ef400640 1 -- 172.21.15.16:0/515685564 learned_addr learned my addr 172.21.15.16:0/515685564 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:25.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.673+0000 7f09eea00640 1 --2- 172.21.15.16:0/515685564 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55602e68c1b0 0x55602ea6a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:25.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.673+0000 7f09ef400640 1 -- 172.21.15.16:0/515685564 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55602e68c1b0 msgr2=0x55602ea6a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:25.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.673+0000 7f09ef400640 1 --2- 172.21.15.16:0/515685564 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55602e68c1b0 0x55602ea6a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:25.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.673+0000 7f09ef400640 1 -- 172.21.15.16:0/515685564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55602e6b5070 msgr2=0x55602ea6a850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:19:25.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.673+0000 7f09ef400640 1 --2- 172.21.15.16:0/515685564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55602e6b5070 0x55602ea6a850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:25.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.673+0000 7f09ef400640 1 -- 172.21.15.16:0/515685564 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55602ea6afc0 con 0x55602e6739f0 2024-08-31T06:19:25.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.674+0000 7f09ef400640 1 --2- 172.21.15.16:0/515685564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55602e6739f0 0x55602ea6ecb0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f09e00ab930 tx=0x7f09e0009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f849a30373c27f9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:25.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.674+0000 7f09d9600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:19:25.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.674+0000 7f09d9600640 1 -- 172.21.15.16:0/515685564 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e000ce60 con 0x55602e6739f0 2024-08-31T06:19:25.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.674+0000 7f09fd1e2740 1 -- 172.21.15.16:0/515685564 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09e8023070 con 0x55602e6739f0 2024-08-31T06:19:25.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.674+0000 7f09d9600640 1 -- 172.21.15.16:0/515685564 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8470+0+0 (secure 0 0 0) 0x7f09e00e5250 con 0x55602e6739f0 2024-08-31T06:19:25.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.674+0000 7f09d9600640 1 -- 172.21.15.16:0/515685564 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f09e00a7860 con 0x55602e6739f0 2024-08-31T06:19:25.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.674+0000 7f09fd1e2740 10 client.28379 fetch_fsmap learned FSMap version 49 2024-08-31T06:19:25.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.674+0000 7f09fd1e2740 1 -- 172.21.15.16:0/515685564 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55602eae2d40 con 0x55602e6739f0 2024-08-31T06:19:25.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.675+0000 7f09d9600640 1 -- 172.21.15.16:0/515685564 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09e8023070 con 0x55602e6739f0 2024-08-31T06:19:25.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.675+0000 7f09d9600640 1 -- 172.21.15.16:0/515685564 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f09e0004f80 con 0x55602e6739f0 2024-08-31T06:19:25.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.675+0000 7f09fd1e2740 10 client.28379 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:19:25.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.675+0000 7f09fd1e2740 10 client.28379 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:19:25.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.675+0000 7f09fd1e2740 20 client.28379 populate_metadata read hostname 'smithi016' 2024-08-31T06:19:25.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.675+0000 7f09fd1e2740 1 --2- 172.21.15.16:0/515685564 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55602eae3550 0x55602eae5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:25.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.675+0000 7f09fd1e2740 4 client.28379 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T06:19:25.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.675+0000 7f09fd1e2740 1 -- 172.21.15.16:0/515685564 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55602e9cc8a0 con 0x55602eae3550 2024-08-31T06:19:25.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.675+0000 7f09eea00640 1 --2- 172.21.15.16:0/515685564 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55602eae3550 0x55602eae5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:25.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.676+0000 7f09eea00640 1 --2- 172.21.15.16:0/515685564 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55602eae3550 0x55602eae5970 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65c1da721cd32b93 server_cookie=9af553f754e45689 in_seq=0 out_seq=0 2024-08-31T06:19:25.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.676+0000 7f09d9600640 10 client.28379 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:19:25.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.676+0000 7f09d9600640 1 -- 172.21.15.16:0/515685564 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4830 (crc 0 0 0) 0x55602e9cc8a0 con 0x55602eae3550 2024-08-31T06:19:25.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.676+0000 7f09d9600640 10 client.28379 handle_command_reply: tid=0 2024-08-31T06:19:25.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.676+0000 7f09fd1e2740 1 client.28379 shutdown 2024-08-31T06:19:25.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.676+0000 7f09fd1e2740 1 -- 172.21.15.16:0/515685564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55602e6739f0 msgr2=0x55602ea6ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:25.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.676+0000 7f09fd1e2740 1 --2- 172.21.15.16:0/515685564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55602e6739f0 0x55602ea6ecb0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f09e00ab930 tx=0x7f09e0009f90 comp rx=0 tx=0).stop 2024-08-31T06:19:25.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.677+0000 7f09fd1e2740 1 -- 172.21.15.16:0/515685564 shutdown_connections 2024-08-31T06:19:25.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.677+0000 7f09fd1e2740 1 --2- 172.21.15.16:0/515685564 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55602eae3550 0x55602eae5970 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:25.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.677+0000 7f09fd1e2740 1 --2- 172.21.15.16:0/515685564 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55602e68c1b0 0x55602ea6a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:25.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.677+0000 7f09fd1e2740 1 --2- 172.21.15.16:0/515685564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55602e6b5070 0x55602ea6a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:25.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.677+0000 7f09fd1e2740 1 --2- 172.21.15.16:0/515685564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55602e6739f0 0x55602ea6ecb0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:25.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.677+0000 7f09d9600640 0 client.28379 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:19:25.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.677+0000 7f09fd1e2740 1 -- 172.21.15.16:0/515685564 >> 172.21.15.16:0/515685564 conn(0x55602e6a9460 msgr2=0x55602ea668d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:25.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.677+0000 7f09fd1e2740 1 -- 172.21.15.16:0/515685564 shutdown_connections 2024-08-31T06:19:25.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.677+0000 7f09fd1e2740 1 -- 172.21.15.16:0/515685564 wait complete. 2024-08-31T06:19:25.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.677+0000 7f09fd1e2740 20 client.28379 trim_cache size 0 max 16384 2024-08-31T06:19:25.682 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:19:25.682 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T06:19:25.682 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-08-31T06:19:25.682 INFO:teuthology.orchestra.run.smithi016.stdout: "f1cdcb27-e3e4-4dc8-a0ae-7ace761d04c5": { 2024-08-31T06:19:25.682 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:25.682 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "f1cdcb27-e3e4-4dc8-a0ae-7ace761d04c5", 2024-08-31T06:19:25.683 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-08-31T06:19:25.683 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:19:25.683 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:25.683 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-08-31T06:19:25.683 INFO:teuthology.orchestra.run.smithi016.stdout: "f1cdcb27-e3e4-4dc8-a0ae-7ace761d04c5": { 2024-08-31T06:19:25.683 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:19:22.275683+0000", 2024-08-31T06:19:25.683 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:25.683 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:25.683 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:25.683 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:25.683 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:25.683 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:25.683 INFO:teuthology.orchestra.run.smithi016.stdout: "0e44eb18-439c-46e4-a298-7d4d2ecd0b4a": { 2024-08-31T06:19:25.684 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:18:31.130765+0000", 2024-08-31T06:19:25.684 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:25.684 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:25.684 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:25.684 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:25.684 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:25.684 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:25.684 INFO:teuthology.orchestra.run.smithi016.stdout: "e497cd88-1e7c-4631-a4a8-cdd8e1fa6c83": { 2024-08-31T06:19:25.684 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:17:42.476795+0000", 2024-08-31T06:19:25.684 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:25.684 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:25.684 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:25.684 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:25.684 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:25.685 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:25.685 INFO:teuthology.orchestra.run.smithi016.stdout: "c300b7e8-f3fd-482c-a909-9329bc61e570": { 2024-08-31T06:19:25.685 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:06:49.868046+0000", 2024-08-31T06:19:25.685 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:25.685 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:25.685 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:25.685 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:25.685 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:25.685 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:25.685 INFO:teuthology.orchestra.run.smithi016.stdout: "6a03046e-cdf4-4a14-b9bd-b560b8968467": { 2024-08-31T06:19:25.685 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:12:38.002354+0000", 2024-08-31T06:19:25.685 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:25.685 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:25.685 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:25.686 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:25.686 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:25.686 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:25.686 INFO:teuthology.orchestra.run.smithi016.stdout: "004b47bb-0fe8-4d28-9d9d-78bde6a61199": { 2024-08-31T06:19:25.686 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:16:48.915926+0000", 2024-08-31T06:19:25.686 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:25.686 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:25.686 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:25.686 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:25.686 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:25.686 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:25.686 INFO:teuthology.orchestra.run.smithi016.stdout: "88d7bdb8-cb1c-45e5-ad29-4cd8da6c6003": { 2024-08-31T06:19:25.686 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:07:39.648942+0000", 2024-08-31T06:19:25.686 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:25.687 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:25.687 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:25.687 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:25.687 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:25.687 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:25.687 INFO:teuthology.orchestra.run.smithi016.stdout: "0fdcd762-533d-42be-9289-b71c8e3b81b1": { 2024-08-31T06:19:25.687 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:15:06.169112+0000", 2024-08-31T06:19:25.687 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:25.687 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:25.687 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:25.687 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:25.687 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:25.687 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:25.687 INFO:teuthology.orchestra.run.smithi016.stdout: "c8455d74-2fe7-4d93-bcf2-19b419d16016": { 2024-08-31T06:19:25.688 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:08:24.035960+0000", 2024-08-31T06:19:25.688 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:25.688 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:25.688 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:25.688 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:25.688 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:25.688 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:25.688 INFO:teuthology.orchestra.run.smithi016.stdout: "78d6ef02-8551-45e8-ad60-9627eb573fc1": { 2024-08-31T06:19:25.688 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:09:13.547865+0000", 2024-08-31T06:19:25.688 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:25.688 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:25.688 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:25.688 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:25.689 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:25.689 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:25.689 INFO:teuthology.orchestra.run.smithi016.stdout: "c1fd6904-2e02-47fa-939a-47349b9fe36b": { 2024-08-31T06:19:25.689 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:06.678691+0000", 2024-08-31T06:19:25.689 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:25.689 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:25.689 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:25.689 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:25.689 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:25.689 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:25.689 INFO:teuthology.orchestra.run.smithi016.stdout: "a10cce76-99ab-46b5-b3b2-07b82899b8f9": { 2024-08-31T06:19:25.689 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:11:43.967546+0000", 2024-08-31T06:19:25.689 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:25.689 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:25.690 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:25.690 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:25.690 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:25.690 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:25.690 INFO:teuthology.orchestra.run.smithi016.stdout: "91b72a34-4ba9-4126-a414-8b862f8c5f78": { 2024-08-31T06:19:25.690 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:13:28.663237+0000", 2024-08-31T06:19:25.690 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:25.690 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:25.690 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:25.690 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:25.690 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:25.690 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:25.690 INFO:teuthology.orchestra.run.smithi016.stdout: "e739a16e-8ace-43a8-a1ce-ac27d39ef8a3": { 2024-08-31T06:19:25.690 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:54.084646+0000", 2024-08-31T06:19:25.691 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:25.691 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:25.691 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:25.691 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:25.691 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:25.691 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:25.691 INFO:teuthology.orchestra.run.smithi016.stdout: "a64d5301-8558-4e9c-ac9b-8d0ac7057e74": { 2024-08-31T06:19:25.691 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:14:20.338104+0000", 2024-08-31T06:19:25.691 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:25.691 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:25.691 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:25.691 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:25.691 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:25.691 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:25.692 INFO:teuthology.orchestra.run.smithi016.stdout: "f9c4a22e-0c8a-4928-9b0d-e2ce1db5a86b": { 2024-08-31T06:19:25.692 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:15:57.496015+0000", 2024-08-31T06:19:25.692 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:25.692 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:25.692 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:25.692 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:25.692 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:25.692 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:19:25.692 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:19:25.692 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:19:25.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.681+0000 7f09f6600640 1 -- 172.21.15.16:0/2495207686 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f09d40b4dd0 msgr2=0x7f09d40b72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:25.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.681+0000 7f09f6600640 1 --2- 172.21.15.16:0/2495207686 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f09d40b4dd0 0x7f09d40b72e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f09e8005fd0 tx=0x7f09e80096a0 comp rx=0 tx=0).stop 2024-08-31T06:19:25.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.681+0000 7f09f6600640 1 -- 172.21.15.16:0/2495207686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09f0103910 msgr2=0x7f09f0198cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:25.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.681+0000 7f09f6600640 1 --2- 172.21.15.16:0/2495207686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09f0103910 0x7f09f0198cd0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f09e000e9c0 tx=0x7f09e000ee80 comp rx=0 tx=0).stop 2024-08-31T06:19:25.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.681+0000 7f09f6600640 1 -- 172.21.15.16:0/2495207686 shutdown_connections 2024-08-31T06:19:25.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.681+0000 7f09f6600640 1 --2- 172.21.15.16:0/2495207686 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f09d40b4dd0 0x7f09d40b72e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:25.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.681+0000 7f09f6600640 1 --2- 172.21.15.16:0/2495207686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09f0109ba0 0x7f09f019fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:25.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.681+0000 7f09f6600640 1 --2- 172.21.15.16:0/2495207686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09f0109250 0x7f09f0199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:25.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.681+0000 7f09f6600640 1 --2- 172.21.15.16:0/2495207686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09f0103910 0x7f09f0198cd0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:25.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.681+0000 7f09f6600640 1 -- 172.21.15.16:0/2495207686 >> 172.21.15.16:0/2495207686 conn(0x7f09f00ff380 msgr2=0x7f09f00ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:25.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.683+0000 7f09f6600640 1 -- 172.21.15.16:0/2495207686 shutdown_connections 2024-08-31T06:19:25.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:25.683+0000 7f09f6600640 1 -- 172.21.15.16:0/2495207686 wait complete. 2024-08-31T06:19:26.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 720.0 s, 1629.1 tps, lat 59.009 ms stddev 71.898 2024-08-31T06:19:26.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:26 smithi184 ceph-mon[22331]: pgmap v744: 97 pgs: 97 active+clean; 3.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 8.6 MiB/s wr, 856 op/s 2024-08-31T06:19:26.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.980+0000 7fa9e8200640 1 -- 172.21.15.16:0/2930810051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9e0102750 msgr2=0x7fa9e0102b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:26.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.980+0000 7fa9e8200640 1 --2- 172.21.15.16:0/2930810051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9e0102750 0x7fa9e0102b60 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4009900 tx=0x7fa9d402a410 comp rx=0 tx=0).stop 2024-08-31T06:19:26.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.980+0000 7fa9e8200640 1 -- 172.21.15.16:0/2930810051 shutdown_connections 2024-08-31T06:19:26.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.980+0000 7fa9e8200640 1 --2- 172.21.15.16:0/2930810051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9e01089d0 0x7fa9e010f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:26.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.980+0000 7fa9e8200640 1 --2- 172.21.15.16:0/2930810051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9e0108080 0x7fa9e0108490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:26.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.980+0000 7fa9e8200640 1 --2- 172.21.15.16:0/2930810051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9e0102750 0x7fa9e0102b60 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:26.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.980+0000 7fa9e8200640 1 -- 172.21.15.16:0/2930810051 >> 172.21.15.16:0/2930810051 conn(0x7fa9e007a4e0 msgr2=0x7fa9e007a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:26.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.981+0000 7fa9e8200640 1 -- 172.21.15.16:0/2930810051 shutdown_connections 2024-08-31T06:19:26.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.981+0000 7fa9e8200640 1 -- 172.21.15.16:0/2930810051 wait complete. 2024-08-31T06:19:26.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.982+0000 7fa9e8200640 1 Processor -- start 2024-08-31T06:19:26.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.982+0000 7fa9e8200640 1 -- start start 2024-08-31T06:19:26.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.982+0000 7fa9e8200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9e0102750 0x7fa9e019c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:26.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.982+0000 7fa9e8200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9e0108080 0x7fa9e019c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:26.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.982+0000 7fa9e8200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9e01089d0 0x7fa9e01a1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:26.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.982+0000 7fa9e8200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e01115c0 con 0x7fa9e0108080 2024-08-31T06:19:26.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.982+0000 7fa9e8200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e0111440 con 0x7fa9e01089d0 2024-08-31T06:19:26.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.982+0000 7fa9e8200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e0111740 con 0x7fa9e0102750 2024-08-31T06:19:26.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.983+0000 7fa9e4e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9e0108080 0x7fa9e019c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:26.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.983+0000 7fa9e4e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9e0108080 0x7fa9e019c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34088/0 (socket says 172.21.15.16:34088) 2024-08-31T06:19:26.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.983+0000 7fa9e4e00640 1 -- 172.21.15.16:0/1216846321 learned_addr learned my addr 172.21.15.16:0/1216846321 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:26.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.983+0000 7fa9e5800640 1 --2- 172.21.15.16:0/1216846321 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9e0102750 0x7fa9e019c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:26.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.984+0000 7fa9e4e00640 1 -- 172.21.15.16:0/1216846321 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9e0102750 msgr2=0x7fa9e019c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:26.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.984+0000 7fa9e4e00640 1 --2- 172.21.15.16:0/1216846321 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9e0102750 0x7fa9e019c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:26.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.984+0000 7fa9e4e00640 1 -- 172.21.15.16:0/1216846321 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9e01089d0 msgr2=0x7fa9e01a1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:26.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.985+0000 7fa9e6200640 1 --2- 172.21.15.16:0/1216846321 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9e01089d0 0x7fa9e01a1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:26.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.985+0000 7fa9e4e00640 1 --2- 172.21.15.16:0/1216846321 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9e01089d0 0x7fa9e01a1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.985+0000 7fa9e4e00640 1 -- 172.21.15.16:0/1216846321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e01a2370 con 0x7fa9e0108080 2024-08-31T06:19:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.985+0000 7fa9e6200640 1 --2- 172.21.15.16:0/1216846321 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9e01089d0 0x7fa9e01a1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:19:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.986+0000 7fa9e4e00640 1 --2- 172.21.15.16:0/1216846321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9e0108080 0x7fa9e019c9e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc00e9f0 tx=0x7fa9cc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.986+0000 7fa9de000640 1 -- 172.21.15.16:0/1216846321 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc00ccc0 con 0x7fa9e0108080 2024-08-31T06:19:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.986+0000 7fa9de000640 1 -- 172.21.15.16:0/1216846321 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9cc0108e0 con 0x7fa9e0108080 2024-08-31T06:19:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.986+0000 7fa9e8200640 1 -- 172.21.15.16:0/1216846321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9e01a2660 con 0x7fa9e0108080 2024-08-31T06:19:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.986+0000 7fa9de000640 1 -- 172.21.15.16:0/1216846321 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc018860 con 0x7fa9e0108080 2024-08-31T06:19:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.986+0000 7fa9e8200640 1 -- 172.21.15.16:0/1216846321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9e01a8df0 con 0x7fa9e0108080 2024-08-31T06:19:26.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.992+0000 7fa9de000640 1 -- 172.21.15.16:0/1216846321 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fa9cc010430 con 0x7fa9e0108080 2024-08-31T06:19:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.993+0000 7fa9ef23c740 1 Processor -- start 2024-08-31T06:19:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.993+0000 7fa9ef23c740 1 -- start start 2024-08-31T06:19:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.994+0000 7fa9ef23c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e06dd11b0 0x560e07174920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.994+0000 7fa9ef23c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e06da82f0 0x560e07174e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.994+0000 7fa9ef23c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e06db04d0 0x560e071753a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.994+0000 7fa9ef23c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560e070e9f70 con 0x560e06dd11b0 2024-08-31T06:19:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.994+0000 7fa9ef23c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560e070d9500 con 0x560e06db04d0 2024-08-31T06:19:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.994+0000 7fa9ef23c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560e070dd4b0 con 0x560e06da82f0 2024-08-31T06:19:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.994+0000 7fa9e5800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e06da82f0 0x560e07174e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.994+0000 7fa9e6200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e06dd11b0 0x560e07174920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.994+0000 7fa9e5800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e06da82f0 0x560e07174e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:46532/0 (socket says 172.21.15.16:46532) 2024-08-31T06:19:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.994+0000 7fa9e5800640 1 -- 172.21.15.16:0/4206539444 learned_addr learned my addr 172.21.15.16:0/4206539444 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.994+0000 7fa9e4e00640 1 --2- 172.21.15.16:0/4206539444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e06db04d0 0x560e071753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:26.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.994+0000 7fa9de000640 1 --2- 172.21.15.16:0/1216846321 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa9c80b4d20 0x7fa9c80b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:26.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.995+0000 7fa9de000640 1 -- 172.21.15.16:0/1216846321 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8470+0+0 (secure 0 0 0) 0x7fa9cc0e57d0 con 0x7fa9e0108080 2024-08-31T06:19:26.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.995+0000 7fa9e5800640 1 -- 172.21.15.16:0/4206539444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e06db04d0 msgr2=0x560e071753a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:26.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.995+0000 7fa9e5800640 1 --2- 172.21.15.16:0/4206539444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e06db04d0 0x560e071753a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:26.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.995+0000 7fa9e5800640 1 -- 172.21.15.16:0/4206539444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e06dd11b0 msgr2=0x560e07174920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.995+0000 7fa9e5800640 1 --2- 172.21.15.16:0/4206539444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e06dd11b0 0x560e07174920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.995+0000 7fa9e6200640 1 --2- 172.21.15.16:0/1216846321 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa9c80b4d20 0x7fa9c80b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.995+0000 7fa9e5800640 1 -- 172.21.15.16:0/4206539444 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e07093c10 con 0x560e06da82f0 2024-08-31T06:19:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.995+0000 7fa9e4e00640 1 --2- 172.21.15.16:0/4206539444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e06db04d0 0x560e071753a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:19:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.995+0000 7fa9e5800640 1 --2- 172.21.15.16:0/4206539444 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e06da82f0 0x560e07174e60 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4009230 tx=0x7fa9d4004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=693df561207d942 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.996+0000 7fa9d2a00640 1 -- 172.21.15.16:0/4206539444 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4035040 con 0x560e06da82f0 2024-08-31T06:19:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.996+0000 7fa9d2a00640 1 -- 172.21.15.16:0/4206539444 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9d40054f0 con 0x560e06da82f0 2024-08-31T06:19:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.996+0000 7fa9d2a00640 1 -- 172.21.15.16:0/4206539444 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d40035f0 con 0x560e06da82f0 2024-08-31T06:19:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.996+0000 7fa9e6200640 1 --2- 172.21.15.16:0/1216846321 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa9c80b4d20 0x7fa9c80b7230 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8005fd0 tx=0x7fa9d8004ab0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:27.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.998+0000 7fa9ef23c740 1 -- 172.21.15.16:0/4206539444 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e06da82f0 msgr2=0x560e07174e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:27.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.998+0000 7fa9ef23c740 1 --2- 172.21.15.16:0/4206539444 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e06da82f0 0x560e07174e60 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4009230 tx=0x7fa9d4004040 comp rx=0 tx=0).stop 2024-08-31T06:19:27.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.999+0000 7fa9ef23c740 1 -- 172.21.15.16:0/4206539444 shutdown_connections 2024-08-31T06:19:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.999+0000 7fa9ef23c740 1 --2- 172.21.15.16:0/4206539444 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e06da82f0 0x560e07174e60 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.999+0000 7fa9ef23c740 1 --2- 172.21.15.16:0/4206539444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e06db04d0 0x560e071753a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.999+0000 7fa9ef23c740 1 --2- 172.21.15.16:0/4206539444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e06dd11b0 0x560e07174920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.999+0000 7fa9ef23c740 1 -- 172.21.15.16:0/4206539444 >> 172.21.15.16:0/4206539444 conn(0x560e06dc5300 msgr2=0x560e07172810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.999+0000 7fa9ef23c740 1 -- 172.21.15.16:0/4206539444 shutdown_connections 2024-08-31T06:19:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:26.999+0000 7fa9ef23c740 1 -- 172.21.15.16:0/4206539444 wait complete. 2024-08-31T06:19:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.000+0000 7fa9ef23c740 1 -- start start 2024-08-31T06:19:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.000+0000 7fa9ef23c740 1 Processor -- start 2024-08-31T06:19:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.001+0000 7fa9ef23c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e06da82f0 0x560e07188700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.001+0000 7fa9ef23c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e06db04d0 0x560e07188c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.001+0000 7fa9ef23c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e06dd11b0 0x560e07189180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.001+0000 7fa9ef23c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560e07012ab0 con 0x560e06db04d0 2024-08-31T06:19:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.001+0000 7fa9ef23c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560e07080320 con 0x560e06dd11b0 2024-08-31T06:19:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.001+0000 7fa9ef23c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560e0718b800 con 0x560e06da82f0 2024-08-31T06:19:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.001+0000 7fa9ef23c740 10 client.0 _ll_register_callbacks cb 0x560e070866a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:19:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.001+0000 7fa9e5800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e06da82f0 0x560e07188700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.001+0000 7fa9e5800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e06da82f0 0x560e07188700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:46536/0 (socket says 172.21.15.16:46536) 2024-08-31T06:19:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.001+0000 7fa9e5800640 1 -- 172.21.15.16:0/1809194993 learned_addr learned my addr 172.21.15.16:0/1809194993 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.002+0000 7fa9e6200640 1 --2- 172.21.15.16:0/1809194993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e06dd11b0 0x560e07189180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.002+0000 7fa9e4e00640 1 --2- 172.21.15.16:0/1809194993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e06db04d0 0x560e07188c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.002+0000 7fa9e5800640 1 -- 172.21.15.16:0/1809194993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e06dd11b0 msgr2=0x560e07189180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.002+0000 7fa9e5800640 1 --2- 172.21.15.16:0/1809194993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e06dd11b0 0x560e07189180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.002+0000 7fa9e5800640 1 -- 172.21.15.16:0/1809194993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e06db04d0 msgr2=0x560e07188c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.002+0000 7fa9e5800640 1 --2- 172.21.15.16:0/1809194993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e06db04d0 0x560e07188c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.003+0000 7fa9e5800640 1 -- 172.21.15.16:0/1809194993 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e06e3c790 con 0x560e06da82f0 2024-08-31T06:19:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.004+0000 7fa9e5800640 1 --2- 172.21.15.16:0/1809194993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e06da82f0 0x560e07188700 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4005ab0 tx=0x7fa9d402a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32dd79b6094e8b7 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.004+0000 7fa9c7400640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:19:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.004+0000 7fa9c7400640 1 -- 172.21.15.16:0/1809194993 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4035040 con 0x560e06da82f0 2024-08-31T06:19:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.005+0000 7fa9c7400640 1 -- 172.21.15.16:0/1809194993 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8470+0+0 (secure 0 0 0) 0x7fa9d4004ee0 con 0x560e06da82f0 2024-08-31T06:19:27.008 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.005+0000 7fa9ef23c740 1 -- 172.21.15.16:0/1809194993 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9d40054f0 con 0x560e06da82f0 2024-08-31T06:19:27.008 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.006+0000 7fa9c7400640 1 -- 172.21.15.16:0/1809194993 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fa9d4009520 con 0x560e06da82f0 2024-08-31T06:19:27.008 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.006+0000 7fa9ef23c740 10 client.28408 fetch_fsmap learned FSMap version 49 2024-08-31T06:19:27.009 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.006+0000 7fa9ef23c740 1 -- 172.21.15.16:0/1809194993 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x560e06eb5620 con 0x560e06da82f0 2024-08-31T06:19:27.009 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.006+0000 7fa9c7400640 1 -- 172.21.15.16:0/1809194993 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9d40054f0 con 0x560e06da82f0 2024-08-31T06:19:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.006+0000 7fa9c7400640 1 -- 172.21.15.16:0/1809194993 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fa9d404c070 con 0x560e06da82f0 2024-08-31T06:19:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.007+0000 7fa9ef23c740 10 client.28408 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:19:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.007+0000 7fa9ef23c740 10 client.28408 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:19:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.007+0000 7fa9ef23c740 20 client.28408 populate_metadata read hostname 'smithi016' 2024-08-31T06:19:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.007+0000 7fa9ef23c740 1 --2- 172.21.15.16:0/1809194993 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x560e06d8fd80 0x560e0719a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.007+0000 7fa9ef23c740 4 client.28408 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:19:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.007+0000 7fa9ef23c740 1 -- 172.21.15.16:0/1809194993 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e070ebec0 con 0x560e06d8fd80 2024-08-31T06:19:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.007+0000 7fa9e4e00640 1 --2- 172.21.15.16:0/1809194993 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x560e06d8fd80 0x560e0719a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.008+0000 7fa9e4e00640 1 --2- 172.21.15.16:0/1809194993 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x560e06d8fd80 0x560e0719a600 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c047e53b5f18c9c6 server_cookie=cbe42eb833e10ad1 in_seq=0 out_seq=0 2024-08-31T06:19:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.008+0000 7fa9c7400640 10 client.28408 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:19:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.009+0000 7fa9c7400640 1 -- 172.21.15.16:0/1809194993 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560e070ebec0 con 0x560e06d8fd80 2024-08-31T06:19:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.009+0000 7fa9c7400640 10 client.28408 handle_command_reply: tid=0 2024-08-31T06:19:27.012 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.009+0000 7fa9ef23c740 1 client.28408 shutdown 2024-08-31T06:19:27.012 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.009+0000 7fa9ef23c740 1 -- 172.21.15.16:0/1809194993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e06da82f0 msgr2=0x560e07188700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:27.012 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.009+0000 7fa9ef23c740 1 --2- 172.21.15.16:0/1809194993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e06da82f0 0x560e07188700 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4005ab0 tx=0x7fa9d402a410 comp rx=0 tx=0).stop 2024-08-31T06:19:27.012 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.009+0000 7fa9ef23c740 1 -- 172.21.15.16:0/1809194993 shutdown_connections 2024-08-31T06:19:27.012 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.009+0000 7fa9ef23c740 1 --2- 172.21.15.16:0/1809194993 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x560e06d8fd80 0x560e0719a600 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:27.012 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.009+0000 7fa9ef23c740 1 --2- 172.21.15.16:0/1809194993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e06dd11b0 0x560e07189180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:27.012 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.009+0000 7fa9ef23c740 1 --2- 172.21.15.16:0/1809194993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e06db04d0 0x560e07188c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.009+0000 7fa9ef23c740 1 --2- 172.21.15.16:0/1809194993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e06da82f0 0x560e07188700 secure :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4005ab0 tx=0x7fa9d402a410 comp rx=0 tx=0).stop 2024-08-31T06:19:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.010+0000 7fa9ef23c740 1 -- 172.21.15.16:0/1809194993 >> 172.21.15.16:0/1809194993 conn(0x560e06dc5300 msgr2=0x560e07170cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.010+0000 7fa9c7400640 0 client.28408 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:19:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.010+0000 7fa9ef23c740 1 -- 172.21.15.16:0/1809194993 shutdown_connections 2024-08-31T06:19:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.010+0000 7fa9ef23c740 1 -- 172.21.15.16:0/1809194993 wait complete. 2024-08-31T06:19:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.011+0000 7fa9ef23c740 20 client.28408 trim_cache size 0 max 16384 2024-08-31T06:19:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.071+0000 7fa9ef23c740 1 Processor -- start 2024-08-31T06:19:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.071+0000 7fa9ef23c740 1 -- start start 2024-08-31T06:19:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.071+0000 7fa9ef23c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e06db04d0 0x560e07171400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.072+0000 7fa9ef23c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e06dd11b0 0x560e071858d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.072+0000 7fa9ef23c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e06d651c0 0x560e07185e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.072+0000 7fa9ef23c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e01afd30 con 0x560e06dd11b0 2024-08-31T06:19:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.072+0000 7fa9ef23c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e01ad540 con 0x560e06db04d0 2024-08-31T06:19:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.072+0000 7fa9ef23c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e01ab3d0 con 0x560e06d651c0 2024-08-31T06:19:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.072+0000 7fa9e5800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e06db04d0 0x560e07171400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.072+0000 7fa9e6200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e06d651c0 0x560e07185e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.072+0000 7fa9e5800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e06db04d0 0x560e07171400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:56060/0 (socket says 172.21.15.16:56060) 2024-08-31T06:19:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.072+0000 7fa9e4e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e06dd11b0 0x560e071858d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.072+0000 7fa9e5800640 1 -- 172.21.15.16:0/2765416356 learned_addr learned my addr 172.21.15.16:0/2765416356 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.074+0000 7fa9e5800640 1 -- 172.21.15.16:0/2765416356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e06d651c0 msgr2=0x560e07185e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.074+0000 7fa9e5800640 1 --2- 172.21.15.16:0/2765416356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e06d651c0 0x560e07185e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.074+0000 7fa9e5800640 1 -- 172.21.15.16:0/2765416356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e06dd11b0 msgr2=0x560e071858d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.074+0000 7fa9e5800640 1 --2- 172.21.15.16:0/2765416356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e06dd11b0 0x560e071858d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.074+0000 7fa9e5800640 1 -- 172.21.15.16:0/2765416356 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e06d8ffb0 con 0x560e06db04d0 2024-08-31T06:19:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.075+0000 7fa9e5800640 1 --2- 172.21.15.16:0/2765416356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e06db04d0 0x560e07171400 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa9d402aa00 tx=0x7fa9d4004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d73ef908f1a704e1 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.078+0000 7fa9d2000640 1 -- 172.21.15.16:0/2765416356 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4064070 con 0x560e06db04d0 2024-08-31T06:19:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.078+0000 7fa9d2000640 1 -- 172.21.15.16:0/2765416356 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9d40035f0 con 0x560e06db04d0 2024-08-31T06:19:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.078+0000 7fa9d2000640 1 -- 172.21.15.16:0/2765416356 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4004df0 con 0x560e06db04d0 2024-08-31T06:19:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.080+0000 7fa9ef23c740 1 -- 172.21.15.16:0/2765416356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e06db04d0 msgr2=0x560e07171400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.080+0000 7fa9ef23c740 1 --2- 172.21.15.16:0/2765416356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e06db04d0 0x560e07171400 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa9d402aa00 tx=0x7fa9d4004040 comp rx=0 tx=0).stop 2024-08-31T06:19:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.080+0000 7fa9ef23c740 1 -- 172.21.15.16:0/2765416356 shutdown_connections 2024-08-31T06:19:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.080+0000 7fa9ef23c740 1 --2- 172.21.15.16:0/2765416356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e06d651c0 0x560e07185e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.080+0000 7fa9ef23c740 1 --2- 172.21.15.16:0/2765416356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e06dd11b0 0x560e071858d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.080+0000 7fa9ef23c740 1 --2- 172.21.15.16:0/2765416356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e06db04d0 0x560e07171400 secure :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa9d402aa00 tx=0x7fa9d4004040 comp rx=0 tx=0).stop 2024-08-31T06:19:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.080+0000 7fa9ef23c740 1 -- 172.21.15.16:0/2765416356 >> 172.21.15.16:0/2765416356 conn(0x560e06dc5300 msgr2=0x560e07170cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.080+0000 7fa9ef23c740 1 -- 172.21.15.16:0/2765416356 shutdown_connections 2024-08-31T06:19:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.080+0000 7fa9ef23c740 1 -- 172.21.15.16:0/2765416356 wait complete. 2024-08-31T06:19:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.081+0000 7fa9ef23c740 1 -- start start 2024-08-31T06:19:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.081+0000 7fa9ef23c740 1 Processor -- start 2024-08-31T06:19:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.082+0000 7fa9ef23c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e06d651c0 0x560e0719b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.083+0000 7fa9ef23c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e06dd11b0 0x560e0719bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.083+0000 7fa9ef23c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e0719c080 0x560e07203d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.083+0000 7fa9e4e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e06dd11b0 0x560e0719bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.083+0000 7fa9e4e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e06dd11b0 0x560e0719bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:56066/0 (socket says 172.21.15.16:56066) 2024-08-31T06:19:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.084+0000 7fa9ef23c740 1 -- 172.21.15.16:0/359884039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e01aba20 con 0x560e0719c080 2024-08-31T06:19:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.084+0000 7fa9ef23c740 1 -- 172.21.15.16:0/359884039 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e01ace70 con 0x560e06dd11b0 2024-08-31T06:19:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.084+0000 7fa9ef23c740 1 -- 172.21.15.16:0/359884039 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e01ac0d0 con 0x560e06d651c0 2024-08-31T06:19:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.084+0000 7fa9e4e00640 1 -- 172.21.15.16:0/359884039 learned_addr learned my addr 172.21.15.16:0/359884039 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.084+0000 7fa9e5800640 1 --2- 172.21.15.16:0/359884039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e06d651c0 0x560e0719b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.084+0000 7fa9e6200640 1 --2- 172.21.15.16:0/359884039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e0719c080 0x560e07203d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.084+0000 7fa9ef23c740 10 client.0 _ll_register_callbacks cb 0x7fa9d4005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:19:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.084+0000 7fa9e4e00640 1 -- 172.21.15.16:0/359884039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e06d651c0 msgr2=0x560e0719b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.084+0000 7fa9e4e00640 1 --2- 172.21.15.16:0/359884039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e06d651c0 0x560e0719b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.084+0000 7fa9e4e00640 1 -- 172.21.15.16:0/359884039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e0719c080 msgr2=0x560e07203d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.084+0000 7fa9e4e00640 1 --2- 172.21.15.16:0/359884039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e0719c080 0x560e07203d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.085+0000 7fa9e4e00640 1 -- 172.21.15.16:0/359884039 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e072042c0 con 0x560e06dd11b0 2024-08-31T06:19:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.086+0000 7fa9e4e00640 1 --2- 172.21.15.16:0/359884039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e06dd11b0 0x560e0719bb40 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc00c6a0 tx=0x7fa9cc009740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7afda0285646cb9c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.086+0000 7fa9c7400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:19:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.086+0000 7fa9c7400640 1 -- 172.21.15.16:0/359884039 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc0b0760 con 0x560e06dd11b0 2024-08-31T06:19:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.086+0000 7fa9c7400640 1 -- 172.21.15.16:0/359884039 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8470+0+0 (secure 0 0 0) 0x7fa9cc0a8180 con 0x560e06dd11b0 2024-08-31T06:19:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.086+0000 7fa9ef23c740 1 -- 172.21.15.16:0/359884039 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9d40035f0 con 0x560e06dd11b0 2024-08-31T06:19:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.088+0000 7fa9c7400640 1 -- 172.21.15.16:0/359884039 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fa9cc0b5210 con 0x560e06dd11b0 2024-08-31T06:19:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.088+0000 7fa9ef23c740 10 client.28394 fetch_fsmap learned FSMap version 49 2024-08-31T06:19:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.088+0000 7fa9ef23c740 1 -- 172.21.15.16:0/359884039 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x560e072053e0 con 0x560e06dd11b0 2024-08-31T06:19:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.088+0000 7fa9c7400640 1 -- 172.21.15.16:0/359884039 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9d40035f0 con 0x560e06dd11b0 2024-08-31T06:19:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.089+0000 7fa9c7400640 1 -- 172.21.15.16:0/359884039 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fa9cc091ce0 con 0x560e06dd11b0 2024-08-31T06:19:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.089+0000 7fa9ef23c740 10 client.28394 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:19:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.089+0000 7fa9ef23c740 10 client.28394 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:19:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.089+0000 7fa9ef23c740 20 client.28394 populate_metadata read hostname 'smithi016' 2024-08-31T06:19:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.089+0000 7fa9ef23c740 1 --2- 172.21.15.16:0/359884039 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x560e07205b90 0x560e07208010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.089+0000 7fa9ef23c740 4 client.28394 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:19:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.089+0000 7fa9ef23c740 1 -- 172.21.15.16:0/359884039 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560e070e7960 con 0x560e07205b90 2024-08-31T06:19:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.089+0000 7fa9e5800640 1 --2- 172.21.15.16:0/359884039 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x560e07205b90 0x560e07208010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.090+0000 7fa9c7400640 10 client.28394 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:19:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.092+0000 7fa9e5800640 1 --2- 172.21.15.16:0/359884039 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x560e07205b90 0x560e07208010 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f5c9e5815ce87e9 server_cookie=86fd50cea5955d33 in_seq=0 out_seq=0 2024-08-31T06:19:27.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:26 smithi155 ceph-mon[21742]: pgmap v744: 97 pgs: 97 active+clean; 3.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 8.6 MiB/s wr, 856 op/s 2024-08-31T06:19:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.136+0000 7fa9c7400640 1 -- 172.21.15.16:0/359884039 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560e070e7960 con 0x560e07205b90 2024-08-31T06:19:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.136+0000 7fa9c7400640 10 client.28394 handle_command_reply: tid=0 2024-08-31T06:19:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.136+0000 7fa9ef23c740 1 client.28394 shutdown 2024-08-31T06:19:27.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.137+0000 7fa9ef23c740 1 -- 172.21.15.16:0/359884039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e06dd11b0 msgr2=0x560e0719bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:27.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.137+0000 7fa9ef23c740 1 --2- 172.21.15.16:0/359884039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e06dd11b0 0x560e0719bb40 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc00c6a0 tx=0x7fa9cc009740 comp rx=0 tx=0).stop 2024-08-31T06:19:27.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.138+0000 7fa9ef23c740 1 -- 172.21.15.16:0/359884039 shutdown_connections 2024-08-31T06:19:27.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.138+0000 7fa9ef23c740 1 --2- 172.21.15.16:0/359884039 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x560e07205b90 0x560e07208010 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:27.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.138+0000 7fa9ef23c740 1 --2- 172.21.15.16:0/359884039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e0719c080 0x560e07203d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:27.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.138+0000 7fa9ef23c740 1 --2- 172.21.15.16:0/359884039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e06dd11b0 0x560e0719bb40 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:27.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.138+0000 7fa9ef23c740 1 --2- 172.21.15.16:0/359884039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e06d651c0 0x560e0719b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.138+0000 7fa9ef23c740 1 -- 172.21.15.16:0/359884039 >> 172.21.15.16:0/359884039 conn(0x560e06dc5300 msgr2=0x560e07183460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.138+0000 7fa9c7400640 0 client.28394 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:19:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.138+0000 7fa9ef23c740 1 -- 172.21.15.16:0/359884039 shutdown_connections 2024-08-31T06:19:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.139+0000 7fa9ef23c740 1 -- 172.21.15.16:0/359884039 wait complete. 2024-08-31T06:19:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.139+0000 7fa9ef23c740 20 client.28394 trim_cache size 0 max 16384 2024-08-31T06:19:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:19:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:19:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:19:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:19:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.140+0000 7fa9e8200640 1 -- 172.21.15.16:0/1216846321 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa9c80b4d20 msgr2=0x7fa9c80b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.140+0000 7fa9e8200640 1 --2- 172.21.15.16:0/1216846321 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa9c80b4d20 0x7fa9c80b7230 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8005fd0 tx=0x7fa9d8004ab0 comp rx=0 tx=0).stop 2024-08-31T06:19:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.140+0000 7fa9e8200640 1 -- 172.21.15.16:0/1216846321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9e0108080 msgr2=0x7fa9e019c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.140+0000 7fa9e8200640 1 --2- 172.21.15.16:0/1216846321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9e0108080 0x7fa9e019c9e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc00e9f0 tx=0x7fa9cc00eeb0 comp rx=0 tx=0).stop 2024-08-31T06:19:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.141+0000 7fa9e8200640 1 -- 172.21.15.16:0/1216846321 shutdown_connections 2024-08-31T06:19:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.141+0000 7fa9e8200640 1 --2- 172.21.15.16:0/1216846321 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa9c80b4d20 0x7fa9c80b7230 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.141+0000 7fa9e8200640 1 --2- 172.21.15.16:0/1216846321 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9e01089d0 0x7fa9e01a1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.141+0000 7fa9e8200640 1 --2- 172.21.15.16:0/1216846321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9e0108080 0x7fa9e019c9e0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.141+0000 7fa9e8200640 1 --2- 172.21.15.16:0/1216846321 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9e0102750 0x7fa9e019c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.141+0000 7fa9e8200640 1 -- 172.21.15.16:0/1216846321 >> 172.21.15.16:0/1216846321 conn(0x7fa9e007a4e0 msgr2=0x7fa9e01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.141+0000 7fa9e8200640 1 -- 172.21.15.16:0/1216846321 shutdown_connections 2024-08-31T06:19:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:27.141+0000 7fa9e8200640 1 -- 172.21.15.16:0/1216846321 wait complete. 2024-08-31T06:19:27.155 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:26 smithi016 ceph-mon[17194]: pgmap v744: 97 pgs: 97 active+clean; 3.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 8.6 MiB/s wr, 856 op/s 2024-08-31T06:19:27.214 INFO:tasks.cephfs.filesystem:scrub status for tag:f1cdcb27-e3e4-4dc8-a0ae-7ace761d04c5 - {'path': '/', 'tag': 'f1cdcb27-e3e4-4dc8-a0ae-7ace761d04c5', 'options': 'recursive,force'} 2024-08-31T06:19:28.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:28 smithi184 ceph-mon[22331]: pgmap v745: 97 pgs: 97 active+clean; 3.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 7.6 MiB/s wr, 785 op/s 2024-08-31T06:19:28.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:28 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:19:29.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:28 smithi155 ceph-mon[21742]: pgmap v745: 97 pgs: 97 active+clean; 3.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 7.6 MiB/s wr, 785 op/s 2024-08-31T06:19:29.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:28 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:19:29.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:28 smithi016 ceph-mon[17194]: pgmap v745: 97 pgs: 97 active+clean; 3.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 7.6 MiB/s wr, 785 op/s 2024-08-31T06:19:29.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:28 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:19:29.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:29 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:19:29.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:29 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:19:30.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:29 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:19:30.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:29 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:19:30.165 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:29 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:19:30.165 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:29 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:19:30.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:30 smithi184 ceph-mon[22331]: pgmap v746: 97 pgs: 97 active+clean; 3.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2024-08-31T06:19:31.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:30 smithi155 ceph-mon[21742]: pgmap v746: 97 pgs: 97 active+clean; 3.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2024-08-31T06:19:31.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:30 smithi016 ceph-mon[17194]: pgmap v746: 97 pgs: 97 active+clean; 3.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2024-08-31T06:19:31.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 725.0 s, 1186.8 tps, lat 87.881 ms stddev 156.249 2024-08-31T06:19:32.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:32 smithi184 ceph-mon[22331]: pgmap v747: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 16 MiB/s wr, 1.49k op/s 2024-08-31T06:19:33.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:32 smithi155 ceph-mon[21742]: pgmap v747: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 16 MiB/s wr, 1.49k op/s 2024-08-31T06:19:33.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:32 smithi016 ceph-mon[17194]: pgmap v747: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 16 MiB/s wr, 1.49k op/s 2024-08-31T06:19:35.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:34 smithi155 ceph-mon[21742]: pgmap v748: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 18 MiB/s wr, 1.65k op/s 2024-08-31T06:19:35.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:34 smithi016 ceph-mon[17194]: pgmap v748: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 18 MiB/s wr, 1.65k op/s 2024-08-31T06:19:35.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:34 smithi184 ceph-mon[22331]: pgmap v748: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 18 MiB/s wr, 1.65k op/s 2024-08-31T06:19:36.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 730.0 s, 1401.4 tps, lat 70.956 ms stddev 90.084 2024-08-31T06:19:37.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:36 smithi155 ceph-mon[21742]: pgmap v749: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 19 MiB/s wr, 1.74k op/s 2024-08-31T06:19:37.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:36 smithi016 ceph-mon[17194]: pgmap v749: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 19 MiB/s wr, 1.74k op/s 2024-08-31T06:19:37.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:36 smithi184 ceph-mon[22331]: pgmap v749: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 19 MiB/s wr, 1.74k op/s 2024-08-31T06:19:39.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:38 smithi155 ceph-mon[21742]: pgmap v750: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 17 MiB/s wr, 1.52k op/s 2024-08-31T06:19:39.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:38 smithi016 ceph-mon[17194]: pgmap v750: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 17 MiB/s wr, 1.52k op/s 2024-08-31T06:19:39.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:38 smithi184 ceph-mon[22331]: pgmap v750: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 17 MiB/s wr, 1.52k op/s 2024-08-31T06:19:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.428+0000 7f3b80800640 1 -- 172.21.15.16:0/3468552708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b7810fd10 msgr2=0x7f3b78112220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.428+0000 7f3b80800640 1 --2- 172.21.15.16:0/3468552708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b7810fd10 0x7f3b78112220 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f3b74009be0 tx=0x7f3b7402a6d0 comp rx=0 tx=0).stop 2024-08-31T06:19:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.428+0000 7f3b80800640 1 -- 172.21.15.16:0/3468552708 shutdown_connections 2024-08-31T06:19:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.428+0000 7f3b80800640 1 --2- 172.21.15.16:0/3468552708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b7810fd10 0x7f3b78112220 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.428+0000 7f3b80800640 1 --2- 172.21.15.16:0/3468552708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b78102210 0x7f3b7810f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.428+0000 7f3b80800640 1 --2- 172.21.15.16:0/3468552708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b78101830 0x7f3b78101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.428+0000 7f3b80800640 1 -- 172.21.15.16:0/3468552708 >> 172.21.15.16:0/3468552708 conn(0x7f3b78077510 msgr2=0x7f3b78077950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.428+0000 7f3b80800640 1 -- 172.21.15.16:0/3468552708 shutdown_connections 2024-08-31T06:19:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.428+0000 7f3b80800640 1 -- 172.21.15.16:0/3468552708 wait complete. 2024-08-31T06:19:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.429+0000 7f3b80800640 1 Processor -- start 2024-08-31T06:19:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.430+0000 7f3b80800640 1 -- start start 2024-08-31T06:19:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.430+0000 7f3b80800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b78101830 0x7f3b7810e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.430+0000 7f3b80800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b78102210 0x7f3b7810ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.430+0000 7f3b80800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b7810fd10 0x7f3b781a5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.430+0000 7f3b80800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3b781063a0 con 0x7f3b78102210 2024-08-31T06:19:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.430+0000 7f3b80800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3b78106220 con 0x7f3b7810fd10 2024-08-31T06:19:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.430+0000 7f3b80800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3b78106520 con 0x7f3b78101830 2024-08-31T06:19:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.430+0000 7f3b7d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b78102210 0x7f3b7810ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.431+0000 7f3b7d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b78102210 0x7f3b7810ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34374/0 (socket says 172.21.15.16:34374) 2024-08-31T06:19:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.431+0000 7f3b7d400640 1 -- 172.21.15.16:0/455221523 learned_addr learned my addr 172.21.15.16:0/455221523 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.431+0000 7f3b7de00640 1 --2- 172.21.15.16:0/455221523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b78101830 0x7f3b7810e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.431+0000 7f3b7e800640 1 --2- 172.21.15.16:0/455221523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b7810fd10 0x7f3b781a5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.431+0000 7f3b7de00640 1 -- 172.21.15.16:0/455221523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b7810fd10 msgr2=0x7f3b781a5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.431+0000 7f3b7de00640 1 --2- 172.21.15.16:0/455221523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b7810fd10 0x7f3b781a5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.432+0000 7f3b7de00640 1 -- 172.21.15.16:0/455221523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b78102210 msgr2=0x7f3b7810ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.432+0000 7f3b7de00640 1 --2- 172.21.15.16:0/455221523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b78102210 0x7f3b7810ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.432+0000 7f3b7de00640 1 -- 172.21.15.16:0/455221523 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b781a66f0 con 0x7f3b78101830 2024-08-31T06:19:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.432+0000 7f3b7d400640 1 --2- 172.21.15.16:0/455221523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b78102210 0x7f3b7810ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:19:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.432+0000 7f3b7de00640 1 --2- 172.21.15.16:0/455221523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b78101830 0x7f3b7810e6e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c00e790 tx=0x7f3b6c00ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.432+0000 7f3b72a00640 1 -- 172.21.15.16:0/455221523 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c00eea0 con 0x7f3b78101830 2024-08-31T06:19:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.433+0000 7f3b72a00640 1 -- 172.21.15.16:0/455221523 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b6c00ccc0 con 0x7f3b78101830 2024-08-31T06:19:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.433+0000 7f3b72a00640 1 -- 172.21.15.16:0/455221523 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c018660 con 0x7f3b78101830 2024-08-31T06:19:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.433+0000 7f3b80800640 1 -- 172.21.15.16:0/455221523 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b781a69e0 con 0x7f3b78101830 2024-08-31T06:19:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.433+0000 7f3b80800640 1 -- 172.21.15.16:0/455221523 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b781a6e20 con 0x7f3b78101830 2024-08-31T06:19:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.439+0000 7f3b877f1740 1 Processor -- start 2024-08-31T06:19:40.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.439+0000 7f3b72a00640 1 -- 172.21.15.16:0/455221523 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f3b6c010430 con 0x7f3b78101830 2024-08-31T06:19:40.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.439+0000 7f3b877f1740 1 -- start start 2024-08-31T06:19:40.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.440+0000 7f3b877f1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56285d2b41b0 0x56285d657690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:40.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.440+0000 7f3b877f1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56285d28b2f0 0x56285d657bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:40.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.440+0000 7f3b877f1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56285d2934d0 0x56285d658110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:40.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.440+0000 7f3b877f1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56285d5ccf70 con 0x56285d28b2f0 2024-08-31T06:19:40.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.440+0000 7f3b877f1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56285d5bc500 con 0x56285d2b41b0 2024-08-31T06:19:40.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.440+0000 7f3b877f1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56285d5c04b0 con 0x56285d2934d0 2024-08-31T06:19:40.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.440+0000 7f3b7de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56285d28b2f0 0x56285d657bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:40.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.440+0000 7f3b7e800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56285d2b41b0 0x56285d657690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:40.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.440+0000 7f3b7e800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56285d2b41b0 0x56285d657690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:57872/0 (socket says 172.21.15.16:57872) 2024-08-31T06:19:40.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.440+0000 7f3b7e800640 1 -- 172.21.15.16:0/388932147 learned_addr learned my addr 172.21.15.16:0/388932147 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:40.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.441+0000 7f3b7e800640 1 -- 172.21.15.16:0/388932147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56285d2934d0 msgr2=0x56285d658110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:40.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.441+0000 7f3b7d400640 1 --2- 172.21.15.16:0/388932147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56285d2934d0 0x56285d658110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:40.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.441+0000 7f3b7e800640 1 --2- 172.21.15.16:0/388932147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56285d2934d0 0x56285d658110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:40.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.441+0000 7f3b7e800640 1 -- 172.21.15.16:0/388932147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56285d28b2f0 msgr2=0x56285d657bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:40.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.441+0000 7f3b7e800640 1 --2- 172.21.15.16:0/388932147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56285d28b2f0 0x56285d657bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:40.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.441+0000 7f3b7e800640 1 -- 172.21.15.16:0/388932147 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56285d576c10 con 0x56285d2b41b0 2024-08-31T06:19:40.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.441+0000 7f3b7d400640 1 --2- 172.21.15.16:0/388932147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56285d2934d0 0x56285d658110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:19:40.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.441+0000 7f3b7e800640 1 --2- 172.21.15.16:0/388932147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56285d2b41b0 0x56285d657690 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f3b74033000 tx=0x7f3b74030430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cc77762c3c34c9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:40.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.442+0000 7f3b72a00640 1 --2- 172.21.15.16:0/455221523 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3b600b4da0 0x7f3b600b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:40.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.442+0000 7f3b67000640 1 -- 172.21.15.16:0/388932147 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b74041070 con 0x56285d2b41b0 2024-08-31T06:19:40.446 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.442+0000 7f3b67000640 1 -- 172.21.15.16:0/388932147 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b74003e20 con 0x56285d2b41b0 2024-08-31T06:19:40.446 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.442+0000 7f3b67000640 1 -- 172.21.15.16:0/388932147 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b74036040 con 0x56285d2b41b0 2024-08-31T06:19:40.446 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.442+0000 7f3b72a00640 1 -- 172.21.15.16:0/455221523 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8470+0+0 (secure 0 0 0) 0x7f3b6c0e5700 con 0x7f3b78101830 2024-08-31T06:19:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.443+0000 7f3b7de00640 1 --2- 172.21.15.16:0/455221523 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3b600b4da0 0x7f3b600b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.444+0000 7f3b877f1740 1 -- 172.21.15.16:0/388932147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56285d2b41b0 msgr2=0x56285d657690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.444+0000 7f3b877f1740 1 --2- 172.21.15.16:0/388932147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56285d2b41b0 0x56285d657690 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f3b74033000 tx=0x7f3b74030430 comp rx=0 tx=0).stop 2024-08-31T06:19:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.444+0000 7f3b7de00640 1 --2- 172.21.15.16:0/455221523 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3b600b4da0 0x7f3b600b72b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x56285d656310 tx=0x7f3b6c0eb530 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.445+0000 7f3b877f1740 1 -- 172.21.15.16:0/388932147 shutdown_connections 2024-08-31T06:19:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.445+0000 7f3b877f1740 1 --2- 172.21.15.16:0/388932147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56285d2934d0 0x56285d658110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.445+0000 7f3b877f1740 1 --2- 172.21.15.16:0/388932147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56285d28b2f0 0x56285d657bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.445+0000 7f3b877f1740 1 --2- 172.21.15.16:0/388932147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56285d2b41b0 0x56285d657690 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.445+0000 7f3b877f1740 1 -- 172.21.15.16:0/388932147 >> 172.21.15.16:0/388932147 conn(0x56285d2a8300 msgr2=0x56285d655580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:40.448 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.445+0000 7f3b877f1740 1 -- 172.21.15.16:0/388932147 shutdown_connections 2024-08-31T06:19:40.448 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.445+0000 7f3b877f1740 1 -- 172.21.15.16:0/388932147 wait complete. 2024-08-31T06:19:40.448 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.446+0000 7f3b877f1740 1 -- start start 2024-08-31T06:19:40.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.446+0000 7f3b877f1740 1 Processor -- start 2024-08-31T06:19:40.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.447+0000 7f3b877f1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56285d28b2f0 0x56285d662cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:40.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.447+0000 7f3b877f1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56285d2934d0 0x56285d663200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:40.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.447+0000 7f3b877f1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56285d2b41b0 0x56285d663780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:40.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.447+0000 7f3b877f1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56285d4f5ab0 con 0x56285d2b41b0 2024-08-31T06:19:40.450 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.447+0000 7f3b877f1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56285d563320 con 0x56285d2934d0 2024-08-31T06:19:40.450 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.447+0000 7f3b877f1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56285d665db0 con 0x56285d28b2f0 2024-08-31T06:19:40.450 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.447+0000 7f3b877f1740 10 client.0 _ll_register_callbacks cb 0x56285d59ce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:19:40.450 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.447+0000 7f3b7e800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56285d28b2f0 0x56285d662cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:40.451 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.447+0000 7f3b7e800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56285d28b2f0 0x56285d662cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:56756/0 (socket says 172.21.15.16:56756) 2024-08-31T06:19:40.451 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.447+0000 7f3b7e800640 1 -- 172.21.15.16:0/1509828245 learned_addr learned my addr 172.21.15.16:0/1509828245 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:40.451 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.447+0000 7f3b7e800640 1 --2- 172.21.15.16:0/1509828245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56285d2b41b0 0x56285d663780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:40.451 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.447+0000 7f3b7d400640 1 --2- 172.21.15.16:0/1509828245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56285d2934d0 0x56285d663200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:40.451 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.448+0000 7f3b7d400640 1 -- 172.21.15.16:0/1509828245 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56285d28b2f0 msgr2=0x56285d662cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:40.451 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.448+0000 7f3b7d400640 1 --2- 172.21.15.16:0/1509828245 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56285d28b2f0 0x56285d662cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:40.451 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.448+0000 7f3b7d400640 1 -- 172.21.15.16:0/1509828245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56285d2b41b0 msgr2=0x56285d663780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:40.451 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.448+0000 7f3b7d400640 1 --2- 172.21.15.16:0/1509828245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56285d2b41b0 0x56285d663780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:40.451 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.448+0000 7f3b7d400640 1 -- 172.21.15.16:0/1509828245 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56285d31f790 con 0x56285d2934d0 2024-08-31T06:19:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.449+0000 7f3b7d400640 1 --2- 172.21.15.16:0/1509828245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56285d2934d0 0x56285d663200 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3b781042c0 tx=0x7f3b680117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ceff43de7587dea2 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.449+0000 7f3b5b400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:19:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.449+0000 7f3b5b400640 1 -- 172.21.15.16:0/1509828245 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b68004230 con 0x56285d2934d0 2024-08-31T06:19:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.449+0000 7f3b5b400640 1 -- 172.21.15.16:0/1509828245 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8470+0+0 (secure 0 0 0) 0x7f3b680043d0 con 0x56285d2934d0 2024-08-31T06:19:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.450+0000 7f3b877f1740 1 -- 172.21.15.16:0/1509828245 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b74003e20 con 0x56285d2934d0 2024-08-31T06:19:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.450+0000 7f3b5b400640 1 -- 172.21.15.16:0/1509828245 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f3b68005020 con 0x56285d2934d0 2024-08-31T06:19:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.450+0000 7f3b877f1740 10 client.28406 fetch_fsmap learned FSMap version 49 2024-08-31T06:19:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.450+0000 7f3b877f1740 1 -- 172.21.15.16:0/1509828245 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x56285d398620 con 0x56285d2934d0 2024-08-31T06:19:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.451+0000 7f3b5b400640 1 -- 172.21.15.16:0/1509828245 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b74003e20 con 0x56285d2934d0 2024-08-31T06:19:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.451+0000 7f3b5b400640 1 -- 172.21.15.16:0/1509828245 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f3b6801a070 con 0x56285d2934d0 2024-08-31T06:19:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.452+0000 7f3b877f1740 10 client.28406 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:19:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.452+0000 7f3b877f1740 10 client.28406 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:19:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.452+0000 7f3b877f1740 20 client.28406 populate_metadata read hostname 'smithi016' 2024-08-31T06:19:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.452+0000 7f3b877f1740 1 --2- 172.21.15.16:0/1509828245 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56285d272d80 0x56285d674e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.452+0000 7f3b877f1740 4 client.28406 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:19:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.452+0000 7f3b877f1740 1 -- 172.21.15.16:0/1509828245 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56285d5ceec0 con 0x56285d272d80 2024-08-31T06:19:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.453+0000 7f3b7e800640 1 --2- 172.21.15.16:0/1509828245 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56285d272d80 0x56285d674e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.460+0000 7f3b7e800640 1 --2- 172.21.15.16:0/1509828245 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56285d272d80 0x56285d674e90 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e93e29023afe0b3 server_cookie=ef04697c5944facc in_seq=0 out_seq=0 2024-08-31T06:19:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.460+0000 7f3b5b400640 10 client.28406 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:19:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.461+0000 7f3b5b400640 1 -- 172.21.15.16:0/1509828245 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56285d5ceec0 con 0x56285d272d80 2024-08-31T06:19:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.461+0000 7f3b5b400640 10 client.28406 handle_command_reply: tid=0 2024-08-31T06:19:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.461+0000 7f3b877f1740 1 client.28406 shutdown 2024-08-31T06:19:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.462+0000 7f3b877f1740 1 -- 172.21.15.16:0/1509828245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56285d2934d0 msgr2=0x56285d663200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.462+0000 7f3b877f1740 1 --2- 172.21.15.16:0/1509828245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56285d2934d0 0x56285d663200 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3b781042c0 tx=0x7f3b680117b0 comp rx=0 tx=0).stop 2024-08-31T06:19:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.462+0000 7f3b877f1740 1 -- 172.21.15.16:0/1509828245 shutdown_connections 2024-08-31T06:19:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.462+0000 7f3b877f1740 1 --2- 172.21.15.16:0/1509828245 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56285d272d80 0x56285d674e90 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.462+0000 7f3b877f1740 1 --2- 172.21.15.16:0/1509828245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56285d2b41b0 0x56285d663780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.462+0000 7f3b877f1740 1 --2- 172.21.15.16:0/1509828245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56285d2934d0 0x56285d663200 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.462+0000 7f3b877f1740 1 --2- 172.21.15.16:0/1509828245 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56285d28b2f0 0x56285d662cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.462+0000 7f3b877f1740 1 -- 172.21.15.16:0/1509828245 >> 172.21.15.16:0/1509828245 conn(0x56285d2a8300 msgr2=0x56285d6586c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.462+0000 7f3b5b400640 0 client.28406 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:19:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.463+0000 7f3b877f1740 1 -- 172.21.15.16:0/1509828245 shutdown_connections 2024-08-31T06:19:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.463+0000 7f3b877f1740 1 -- 172.21.15.16:0/1509828245 wait complete. 2024-08-31T06:19:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.463+0000 7f3b877f1740 20 client.28406 trim_cache size 0 max 16384 2024-08-31T06:19:40.504 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.501+0000 7f3b877f1740 1 Processor -- start 2024-08-31T06:19:40.504 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.501+0000 7f3b877f1740 1 -- start start 2024-08-31T06:19:40.504 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.501+0000 7f3b877f1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56285d28b2f0 0x56285d659080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:40.504 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.501+0000 7f3b877f1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56285d2934d0 0x56285d6648f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:40.504 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.501+0000 7f3b877f1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56285d2b41b0 0x56285d65fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:40.504 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.501+0000 7f3b877f1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3b781b8440 con 0x56285d2b41b0 2024-08-31T06:19:40.504 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.501+0000 7f3b877f1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3b781b5c50 con 0x56285d28b2f0 2024-08-31T06:19:40.504 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.501+0000 7f3b877f1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3b781b3ac0 con 0x56285d2934d0 2024-08-31T06:19:40.505 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.502+0000 7f3b7d400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56285d2934d0 0x56285d6648f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:40.505 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.502+0000 7f3b7d400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56285d2934d0 0x56285d6648f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:56764/0 (socket says 172.21.15.16:56764) 2024-08-31T06:19:40.505 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.502+0000 7f3b7d400640 1 -- 172.21.15.16:0/2988604910 learned_addr learned my addr 172.21.15.16:0/2988604910 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.503+0000 7f3b7e800640 1 --2- 172.21.15.16:0/2988604910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56285d2b41b0 0x56285d65fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.503+0000 7f3b7d400640 1 -- 172.21.15.16:0/2988604910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56285d28b2f0 msgr2=0x56285d659080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.503+0000 7f3b7d400640 1 --2- 172.21.15.16:0/2988604910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56285d28b2f0 0x56285d659080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.503+0000 7f3b7d400640 1 -- 172.21.15.16:0/2988604910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56285d2b41b0 msgr2=0x56285d65fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.503+0000 7f3b7d400640 1 --2- 172.21.15.16:0/2988604910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56285d2b41b0 0x56285d65fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.503+0000 7f3b7d400640 1 -- 172.21.15.16:0/2988604910 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56285d6718e0 con 0x56285d2934d0 2024-08-31T06:19:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.503+0000 7f3b7d400640 1 --2- 172.21.15.16:0/2988604910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56285d2934d0 0x56285d6648f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f3b6800d7b0 tx=0x7f3b68011fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fde4b153eadd14d9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.504+0000 7f3b66600640 1 -- 172.21.15.16:0/2988604910 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6800dc50 con 0x56285d2934d0 2024-08-31T06:19:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.504+0000 7f3b66600640 1 -- 172.21.15.16:0/2988604910 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b6800ddf0 con 0x56285d2934d0 2024-08-31T06:19:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.504+0000 7f3b66600640 1 -- 172.21.15.16:0/2988604910 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6800e8a0 con 0x56285d2934d0 2024-08-31T06:19:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.505+0000 7f3b877f1740 1 -- 172.21.15.16:0/2988604910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56285d2934d0 msgr2=0x56285d6648f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.505+0000 7f3b877f1740 1 --2- 172.21.15.16:0/2988604910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56285d2934d0 0x56285d6648f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f3b6800d7b0 tx=0x7f3b68011fb0 comp rx=0 tx=0).stop 2024-08-31T06:19:40.509 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.507+0000 7f3b877f1740 1 -- 172.21.15.16:0/2988604910 shutdown_connections 2024-08-31T06:19:40.509 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.507+0000 7f3b877f1740 1 --2- 172.21.15.16:0/2988604910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56285d2934d0 0x56285d6648f0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:40.509 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.507+0000 7f3b877f1740 1 --2- 172.21.15.16:0/2988604910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56285d2b41b0 0x56285d65fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:40.509 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.507+0000 7f3b877f1740 1 --2- 172.21.15.16:0/2988604910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56285d28b2f0 0x56285d659080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:40.510 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.507+0000 7f3b877f1740 1 -- 172.21.15.16:0/2988604910 >> 172.21.15.16:0/2988604910 conn(0x56285d2a8300 msgr2=0x56285d6641e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:40.510 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.507+0000 7f3b877f1740 1 -- 172.21.15.16:0/2988604910 shutdown_connections 2024-08-31T06:19:40.510 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.507+0000 7f3b877f1740 1 -- 172.21.15.16:0/2988604910 wait complete. 2024-08-31T06:19:40.510 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.507+0000 7f3b877f1740 1 -- start start 2024-08-31T06:19:40.511 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.508+0000 7f3b877f1740 1 Processor -- start 2024-08-31T06:19:40.511 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.508+0000 7f3b877f1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56285d28b2f0 0x56285d66b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:40.511 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.508+0000 7f3b877f1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56285d2934d0 0x56285d668a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.508+0000 7f3b877f1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56285d2b41b0 0x56285d668f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.509+0000 7f3b877f1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3b781b4110 con 0x56285d28b2f0 2024-08-31T06:19:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.509+0000 7f3b877f1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3b781b5580 con 0x56285d2934d0 2024-08-31T06:19:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.509+0000 7f3b877f1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3b781b47e0 con 0x56285d2b41b0 2024-08-31T06:19:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.509+0000 7f3b877f1740 10 client.0 _ll_register_callbacks cb 0x7f3b68009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:19:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.510+0000 7f3b7d400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56285d2934d0 0x56285d668a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.510+0000 7f3b7d400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56285d2934d0 0x56285d668a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:57896/0 (socket says 172.21.15.16:57896) 2024-08-31T06:19:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.510+0000 7f3b7d400640 1 -- 172.21.15.16:0/3580697742 learned_addr learned my addr 172.21.15.16:0/3580697742 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.510+0000 7f3b7e800640 1 --2- 172.21.15.16:0/3580697742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56285d28b2f0 0x56285d66b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.511+0000 7f3b7e800640 1 --2- 172.21.15.16:0/3580697742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56285d2b41b0 0x56285d668f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:40.515 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.512+0000 7f3b7e800640 1 -- 172.21.15.16:0/3580697742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56285d2b41b0 msgr2=0x56285d668f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:40.515 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.512+0000 7f3b7e800640 1 --2- 172.21.15.16:0/3580697742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56285d2b41b0 0x56285d668f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:40.515 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.512+0000 7f3b7e800640 1 -- 172.21.15.16:0/3580697742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56285d2934d0 msgr2=0x56285d668a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:40.515 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.512+0000 7f3b7e800640 1 --2- 172.21.15.16:0/3580697742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56285d2934d0 0x56285d668a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:40.515 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.512+0000 7f3b7e800640 1 -- 172.21.15.16:0/3580697742 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56285d6694b0 con 0x56285d28b2f0 2024-08-31T06:19:40.515 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.513+0000 7f3b7e800640 1 --2- 172.21.15.16:0/3580697742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56285d28b2f0 0x56285d66b570 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f3b740099a0 tx=0x7f3b74005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48115360b2adcbea server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:40.516 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.513+0000 7f3b5b400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:19:40.516 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.513+0000 7f3b5b400640 1 -- 172.21.15.16:0/3580697742 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b74036040 con 0x56285d28b2f0 2024-08-31T06:19:40.516 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.514+0000 7f3b5b400640 1 -- 172.21.15.16:0/3580697742 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8470+0+0 (secure 0 0 0) 0x7f3b74050040 con 0x56285d28b2f0 2024-08-31T06:19:40.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.514+0000 7f3b877f1740 1 -- 172.21.15.16:0/3580697742 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b6800ddf0 con 0x56285d28b2f0 2024-08-31T06:19:40.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.515+0000 7f3b5b400640 1 -- 172.21.15.16:0/3580697742 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f3b74036040 con 0x56285d28b2f0 2024-08-31T06:19:40.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.515+0000 7f3b877f1740 10 client.18021 fetch_fsmap learned FSMap version 49 2024-08-31T06:19:40.517 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.515+0000 7f3b877f1740 1 -- 172.21.15.16:0/3580697742 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x56285d6f0480 con 0x56285d28b2f0 2024-08-31T06:19:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.516+0000 7f3b5b400640 1 -- 172.21.15.16:0/3580697742 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b6800ddf0 con 0x56285d28b2f0 2024-08-31T06:19:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.516+0000 7f3b5b400640 1 -- 172.21.15.16:0/3580697742 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f3b74004310 con 0x56285d28b2f0 2024-08-31T06:19:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.516+0000 7f3b877f1740 10 client.18021 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:19:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.516+0000 7f3b877f1740 10 client.18021 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:19:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.516+0000 7f3b877f1740 20 client.18021 populate_metadata read hostname 'smithi016' 2024-08-31T06:19:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.516+0000 7f3b877f1740 1 --2- 172.21.15.16:0/3580697742 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56285d2481c0 0x56285d6f26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.516+0000 7f3b877f1740 4 client.18021 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:19:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.516+0000 7f3b877f1740 1 -- 172.21.15.16:0/3580697742 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56285d5ca960 con 0x56285d2481c0 2024-08-31T06:19:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.517+0000 7f3b7d400640 1 --2- 172.21.15.16:0/3580697742 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56285d2481c0 0x56285d6f26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:40.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.518+0000 7f3b5b400640 10 client.18021 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:19:40.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.518+0000 7f3b7d400640 1 --2- 172.21.15.16:0/3580697742 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56285d2481c0 0x56285d6f26c0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74d4cfae2bcbd261 server_cookie=80e2b06941cf0919 in_seq=0 out_seq=0 2024-08-31T06:19:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.576+0000 7f3b5b400640 1 -- 172.21.15.16:0/3580697742 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56285d5ca960 con 0x56285d2481c0 2024-08-31T06:19:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.576+0000 7f3b5b400640 10 client.18021 handle_command_reply: tid=0 2024-08-31T06:19:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.576+0000 7f3b877f1740 1 client.18021 shutdown 2024-08-31T06:19:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.576+0000 7f3b877f1740 1 -- 172.21.15.16:0/3580697742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56285d28b2f0 msgr2=0x56285d66b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.576+0000 7f3b877f1740 1 --2- 172.21.15.16:0/3580697742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56285d28b2f0 0x56285d66b570 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f3b740099a0 tx=0x7f3b74005fd0 comp rx=0 tx=0).stop 2024-08-31T06:19:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.576+0000 7f3b877f1740 1 -- 172.21.15.16:0/3580697742 shutdown_connections 2024-08-31T06:19:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.576+0000 7f3b877f1740 1 --2- 172.21.15.16:0/3580697742 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56285d2481c0 0x56285d6f26c0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.576+0000 7f3b877f1740 1 --2- 172.21.15.16:0/3580697742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56285d2b41b0 0x56285d668f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.576+0000 7f3b5b400640 0 client.18021 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:19:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.576+0000 7f3b877f1740 1 --2- 172.21.15.16:0/3580697742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56285d2934d0 0x56285d668a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.576+0000 7f3b877f1740 1 --2- 172.21.15.16:0/3580697742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56285d28b2f0 0x56285d66b570 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.576+0000 7f3b877f1740 1 -- 172.21.15.16:0/3580697742 >> 172.21.15.16:0/3580697742 conn(0x56285d2a8300 msgr2=0x56285d658e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.577+0000 7f3b877f1740 1 -- 172.21.15.16:0/3580697742 shutdown_connections 2024-08-31T06:19:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.577+0000 7f3b877f1740 1 -- 172.21.15.16:0/3580697742 wait complete. 2024-08-31T06:19:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.577+0000 7f3b877f1740 20 client.18021 trim_cache size 0 max 16384 2024-08-31T06:19:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:19:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:19:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:19:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:19:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.578+0000 7f3b80800640 1 -- 172.21.15.16:0/455221523 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3b600b4da0 msgr2=0x7f3b600b72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.578+0000 7f3b80800640 1 --2- 172.21.15.16:0/455221523 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3b600b4da0 0x7f3b600b72b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x56285d656310 tx=0x7f3b6c0eb530 comp rx=0 tx=0).stop 2024-08-31T06:19:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.578+0000 7f3b80800640 1 -- 172.21.15.16:0/455221523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b78101830 msgr2=0x7f3b7810e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.578+0000 7f3b80800640 1 --2- 172.21.15.16:0/455221523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b78101830 0x7f3b7810e6e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c00e790 tx=0x7f3b6c00ec50 comp rx=0 tx=0).stop 2024-08-31T06:19:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.578+0000 7f3b80800640 1 -- 172.21.15.16:0/455221523 shutdown_connections 2024-08-31T06:19:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.578+0000 7f3b80800640 1 --2- 172.21.15.16:0/455221523 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3b600b4da0 0x7f3b600b72b0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.578+0000 7f3b80800640 1 --2- 172.21.15.16:0/455221523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b7810fd10 0x7f3b781a5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.578+0000 7f3b80800640 1 --2- 172.21.15.16:0/455221523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b78102210 0x7f3b7810ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.578+0000 7f3b80800640 1 --2- 172.21.15.16:0/455221523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b78101830 0x7f3b7810e6e0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.578+0000 7f3b80800640 1 -- 172.21.15.16:0/455221523 >> 172.21.15.16:0/455221523 conn(0x7f3b78077510 msgr2=0x7f3b78110300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.578+0000 7f3b80800640 1 -- 172.21.15.16:0/455221523 shutdown_connections 2024-08-31T06:19:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:40.579+0000 7f3b80800640 1 -- 172.21.15.16:0/455221523 wait complete. 2024-08-31T06:19:41.023 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:40 smithi184 ceph-mon[22331]: pgmap v751: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 18 MiB/s wr, 1.57k op/s 2024-08-31T06:19:41.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:40 smithi155 ceph-mon[21742]: pgmap v751: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 18 MiB/s wr, 1.57k op/s 2024-08-31T06:19:41.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:40 smithi016 ceph-mon[17194]: pgmap v751: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 18 MiB/s wr, 1.57k op/s 2024-08-31T06:19:41.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 735.0 s, 1647.4 tps, lat 60.322 ms stddev 71.965 2024-08-31T06:19:43.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:42 smithi155 ceph-mon[21742]: pgmap v752: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 14 MiB/s wr, 1.25k op/s 2024-08-31T06:19:43.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:42 smithi016 ceph-mon[17194]: pgmap v752: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 14 MiB/s wr, 1.25k op/s 2024-08-31T06:19:43.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:42 smithi184 ceph-mon[22331]: pgmap v752: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 14 MiB/s wr, 1.25k op/s 2024-08-31T06:19:45.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:44 smithi155 ceph-mon[21742]: pgmap v753: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 8.5 MiB/s wr, 807 op/s 2024-08-31T06:19:45.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:44 smithi016 ceph-mon[17194]: pgmap v753: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 8.5 MiB/s wr, 807 op/s 2024-08-31T06:19:45.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:44 smithi184 ceph-mon[22331]: pgmap v753: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 8.5 MiB/s wr, 807 op/s 2024-08-31T06:19:46.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 740.0 s, 1456.6 tps, lat 69.275 ms stddev 75.905 2024-08-31T06:19:47.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:46 smithi155 ceph-mon[21742]: pgmap v754: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 7.5 MiB/s wr, 720 op/s 2024-08-31T06:19:47.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:46 smithi016 ceph-mon[17194]: pgmap v754: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 7.5 MiB/s wr, 720 op/s 2024-08-31T06:19:47.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:46 smithi184 ceph-mon[22331]: pgmap v754: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 7.5 MiB/s wr, 720 op/s 2024-08-31T06:19:49.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:48 smithi155 ceph-mon[21742]: pgmap v755: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.9 MiB/s wr, 595 op/s 2024-08-31T06:19:49.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:48 smithi016 ceph-mon[17194]: pgmap v755: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.9 MiB/s wr, 595 op/s 2024-08-31T06:19:49.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:48 smithi184 ceph-mon[22331]: pgmap v755: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.9 MiB/s wr, 595 op/s 2024-08-31T06:19:51.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:50 smithi155 ceph-mon[21742]: pgmap v756: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.8 MiB/s wr, 669 op/s 2024-08-31T06:19:51.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:50 smithi016 ceph-mon[17194]: pgmap v756: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.8 MiB/s wr, 669 op/s 2024-08-31T06:19:51.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:50 smithi184 ceph-mon[22331]: pgmap v756: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.8 MiB/s wr, 669 op/s 2024-08-31T06:19:51.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 745.0 s, 1415.8 tps, lat 70.401 ms stddev 79.940 2024-08-31T06:19:53.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:52 smithi155 ceph-mon[21742]: pgmap v757: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.8 MiB/s wr, 692 op/s 2024-08-31T06:19:53.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:52 smithi016 ceph-mon[17194]: pgmap v757: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.8 MiB/s wr, 692 op/s 2024-08-31T06:19:53.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:52 smithi184 ceph-mon[22331]: pgmap v757: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.8 MiB/s wr, 692 op/s 2024-08-31T06:19:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.850+0000 7f426c200640 1 -- 172.21.15.16:0/1840016913 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42641091c0 msgr2=0x7f42641095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.850+0000 7f426c200640 1 --2- 172.21.15.16:0/1840016913 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42641091c0 0x7f42641095d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f4250009900 tx=0x7f425002a3f0 comp rx=0 tx=0).stop 2024-08-31T06:19:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.850+0000 7f426c200640 1 -- 172.21.15.16:0/1840016913 shutdown_connections 2024-08-31T06:19:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.850+0000 7f426c200640 1 --2- 172.21.15.16:0/1840016913 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42641091c0 0x7f42641095d0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.850+0000 7f426c200640 1 --2- 172.21.15.16:0/1840016913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4264109b10 0x7f4264110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.850+0000 7f426c200640 1 --2- 172.21.15.16:0/1840016913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4264103890 0x7f4264103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.850+0000 7f426c200640 1 -- 172.21.15.16:0/1840016913 >> 172.21.15.16:0/1840016913 conn(0x7f42640ff310 msgr2=0x7f4264101760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.851+0000 7f426c200640 1 -- 172.21.15.16:0/1840016913 shutdown_connections 2024-08-31T06:19:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.851+0000 7f426c200640 1 -- 172.21.15.16:0/1840016913 wait complete. 2024-08-31T06:19:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.852+0000 7f426c200640 1 Processor -- start 2024-08-31T06:19:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.853+0000 7f426c200640 1 -- start start 2024-08-31T06:19:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.853+0000 7f426c200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4264103890 0x7f426419c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.853+0000 7f426c200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42641091c0 0x7f426419ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.853+0000 7f426c200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4264109b10 0x7f42641a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.853+0000 7f426c200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f42641137a0 con 0x7f42641091c0 2024-08-31T06:19:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.853+0000 7f426c200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4264113620 con 0x7f4264109b10 2024-08-31T06:19:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.853+0000 7f426c200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4264113920 con 0x7f4264103890 2024-08-31T06:19:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.853+0000 7f426a200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4264109b10 0x7f42641a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.853+0000 7f426a200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4264109b10 0x7f42641a1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:49324/0 (socket says 172.21.15.16:49324) 2024-08-31T06:19:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.853+0000 7f426a200640 1 -- 172.21.15.16:0/1890783723 learned_addr learned my addr 172.21.15.16:0/1890783723 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.853+0000 7f4269800640 1 --2- 172.21.15.16:0/1890783723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4264103890 0x7f426419c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.854+0000 7f426a200640 1 -- 172.21.15.16:0/1890783723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4264103890 msgr2=0x7f426419c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.854+0000 7f426a200640 1 --2- 172.21.15.16:0/1890783723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4264103890 0x7f426419c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.854+0000 7f426a200640 1 -- 172.21.15.16:0/1890783723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42641091c0 msgr2=0x7f426419ca00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-31T06:19:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.855+0000 7f4268e00640 1 --2- 172.21.15.16:0/1890783723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42641091c0 0x7f426419ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.855+0000 7f426a200640 1 --2- 172.21.15.16:0/1890783723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42641091c0 0x7f426419ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.855+0000 7f426a200640 1 -- 172.21.15.16:0/1890783723 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42641a2390 con 0x7f4264109b10 2024-08-31T06:19:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.855+0000 7f426a200640 1 --2- 172.21.15.16:0/1890783723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4264109b10 0x7f42641a1c90 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f425c00eb10 tx=0x7f425c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.857+0000 7f4262000640 1 -- 172.21.15.16:0/1890783723 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425c004b20 con 0x7f4264109b10 2024-08-31T06:19:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.857+0000 7f4262000640 1 -- 172.21.15.16:0/1890783723 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f425c004cc0 con 0x7f4264109b10 2024-08-31T06:19:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.857+0000 7f4262000640 1 -- 172.21.15.16:0/1890783723 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425c010670 con 0x7f4264109b10 2024-08-31T06:19:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.858+0000 7f426c200640 1 -- 172.21.15.16:0/1890783723 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42641a2680 con 0x7f4264109b10 2024-08-31T06:19:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.858+0000 7f426c200640 1 -- 172.21.15.16:0/1890783723 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42641a8e10 con 0x7f4264109b10 2024-08-31T06:19:53.867 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.865+0000 7f42732b6740 1 Processor -- start 2024-08-31T06:19:53.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.870+0000 7f4262000640 1 -- 172.21.15.16:0/1890783723 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f425c010810 con 0x7f4264109b10 2024-08-31T06:19:53.873 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.871+0000 7f42732b6740 1 -- start start 2024-08-31T06:19:53.877 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.871+0000 7f42732b6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3efa0e1b0 0x55f3efdb1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:53.877 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.871+0000 7f42732b6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3ef9e52f0 0x55f3efdb1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:53.877 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.871+0000 7f42732b6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3ef9ed4d0 0x55f3efdb23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:53.877 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.871+0000 7f42732b6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f3efd26f70 con 0x55f3ef9e52f0 2024-08-31T06:19:53.878 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.871+0000 7f42732b6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f3efd16500 con 0x55f3efa0e1b0 2024-08-31T06:19:53.878 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.871+0000 7f42732b6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f3efd1a4b0 con 0x55f3ef9ed4d0 2024-08-31T06:19:53.878 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.872+0000 7f4269800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3ef9e52f0 0x55f3efdb1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:53.878 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.872+0000 7f4269800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3ef9e52f0 0x55f3efdb1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55842/0 (socket says 172.21.15.16:55842) 2024-08-31T06:19:53.878 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.872+0000 7f4269800640 1 -- 172.21.15.16:0/3142903190 learned_addr learned my addr 172.21.15.16:0/3142903190 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:53.878 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.872+0000 7f4262000640 1 --2- 172.21.15.16:0/1890783723 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f424c0b4df0 0x7f424c0b7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:53.878 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.875+0000 7f4268e00640 1 --2- 172.21.15.16:0/3142903190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3ef9ed4d0 0x55f3efdb23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:53.878 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.875+0000 7f4269800640 1 -- 172.21.15.16:0/3142903190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3ef9ed4d0 msgr2=0x55f3efdb23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:53.879 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.875+0000 7f4269800640 1 --2- 172.21.15.16:0/3142903190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3ef9ed4d0 0x55f3efdb23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:53.879 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.875+0000 7f4269800640 1 -- 172.21.15.16:0/3142903190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3efa0e1b0 msgr2=0x55f3efdb1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:53.879 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.875+0000 7f4269800640 1 --2- 172.21.15.16:0/3142903190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3efa0e1b0 0x55f3efdb1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:53.879 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.875+0000 7f4269800640 1 -- 172.21.15.16:0/3142903190 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3efcd0c10 con 0x55f3ef9e52f0 2024-08-31T06:19:53.879 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.875+0000 7f4268e00640 1 --2- 172.21.15.16:0/3142903190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3ef9ed4d0 0x55f3efdb23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:19:53.879 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.875+0000 7f4262000640 1 -- 172.21.15.16:0/1890783723 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8470+0+0 (secure 0 0 0) 0x7f425c0e59c0 con 0x7f4264109b10 2024-08-31T06:19:53.879 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.876+0000 7f4269800640 1 --2- 172.21.15.16:0/3142903190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3ef9e52f0 0x55f3efdb1e60 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f4258009c80 tx=0x7f42580117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d2c91d7fdb04c6 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:53.880 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.877+0000 7f426a200640 1 --2- 172.21.15.16:0/1890783723 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f424c0b4df0 0x7f424c0b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.878+0000 7f426a200640 1 --2- 172.21.15.16:0/1890783723 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f424c0b4df0 0x7f424c0b7300 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f425c0a8160 tx=0x7f425c009200 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.879+0000 7f4256a00640 1 -- 172.21.15.16:0/3142903190 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4258012450 con 0x55f3ef9e52f0 2024-08-31T06:19:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.879+0000 7f4256a00640 1 -- 172.21.15.16:0/3142903190 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4258012aa0 con 0x55f3ef9e52f0 2024-08-31T06:19:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.879+0000 7f4256a00640 1 -- 172.21.15.16:0/3142903190 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4258012450 con 0x55f3ef9e52f0 2024-08-31T06:19:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.881+0000 7f42732b6740 1 -- 172.21.15.16:0/3142903190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3ef9e52f0 msgr2=0x55f3efdb1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:53.884 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.881+0000 7f42732b6740 1 --2- 172.21.15.16:0/3142903190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3ef9e52f0 0x55f3efdb1e60 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f4258009c80 tx=0x7f42580117b0 comp rx=0 tx=0).stop 2024-08-31T06:19:53.885 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.883+0000 7f42732b6740 1 -- 172.21.15.16:0/3142903190 shutdown_connections 2024-08-31T06:19:53.885 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.883+0000 7f42732b6740 1 --2- 172.21.15.16:0/3142903190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3ef9ed4d0 0x55f3efdb23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:53.885 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.883+0000 7f42732b6740 1 --2- 172.21.15.16:0/3142903190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3ef9e52f0 0x55f3efdb1e60 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:53.885 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.883+0000 7f42732b6740 1 --2- 172.21.15.16:0/3142903190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3efa0e1b0 0x55f3efdb1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:53.885 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.883+0000 7f42732b6740 1 -- 172.21.15.16:0/3142903190 >> 172.21.15.16:0/3142903190 conn(0x55f3efa02300 msgr2=0x55f3efdaf810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:53.886 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.883+0000 7f42732b6740 1 -- 172.21.15.16:0/3142903190 shutdown_connections 2024-08-31T06:19:53.886 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.884+0000 7f42732b6740 1 -- 172.21.15.16:0/3142903190 wait complete. 2024-08-31T06:19:53.887 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.884+0000 7f42732b6740 1 -- start start 2024-08-31T06:19:53.888 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.886+0000 7f42732b6740 1 Processor -- start 2024-08-31T06:19:53.893 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.890+0000 7f42732b6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3ef9e52f0 0x55f3efdc7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:53.893 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.890+0000 7f42732b6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3ef9ed4d0 0x55f3efdbf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:53.893 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.890+0000 7f42732b6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3efa0e1b0 0x55f3efdc7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:53.893 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.890+0000 7f42732b6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f3efc4fab0 con 0x55f3efa0e1b0 2024-08-31T06:19:53.893 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.890+0000 7f42732b6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f3efcbd320 con 0x55f3ef9e52f0 2024-08-31T06:19:53.893 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.890+0000 7f42732b6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f3efdbff60 con 0x55f3ef9ed4d0 2024-08-31T06:19:53.893 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.890+0000 7f42732b6740 10 client.0 _ll_register_callbacks cb 0x55f3efcc36a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:19:53.893 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.890+0000 7f4269800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3ef9ed4d0 0x55f3efdbf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:53.893 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.890+0000 7f4268e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3ef9e52f0 0x55f3efdc7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:53.894 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.890+0000 7f4269800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3ef9ed4d0 0x55f3efdbf580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:32960/0 (socket says 172.21.15.16:32960) 2024-08-31T06:19:53.894 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.890+0000 7f4269800640 1 -- 172.21.15.16:0/517397028 learned_addr learned my addr 172.21.15.16:0/517397028 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:53.894 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.891+0000 7f4269800640 1 -- 172.21.15.16:0/517397028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3ef9e52f0 msgr2=0x55f3efdc7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:53.894 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.891+0000 7f4269800640 1 --2- 172.21.15.16:0/517397028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3ef9e52f0 0x55f3efdc7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:53.894 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.891+0000 7f4269800640 1 -- 172.21.15.16:0/517397028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3efa0e1b0 msgr2=0x55f3efdc7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:53.894 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.891+0000 7f4269800640 1 --2- 172.21.15.16:0/517397028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3efa0e1b0 0x55f3efdc7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:53.894 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.891+0000 7f4269800640 1 -- 172.21.15.16:0/517397028 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3efa79790 con 0x55f3ef9ed4d0 2024-08-31T06:19:53.895 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.892+0000 7f4269800640 1 --2- 172.21.15.16:0/517397028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3ef9ed4d0 0x55f3efdbf580 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f425801c040 tx=0x7f425800f7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64fc9d3bf57cd72d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:53.895 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.892+0000 7f424b400640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:19:53.897 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.895+0000 7f424b400640 1 -- 172.21.15.16:0/517397028 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4258029020 con 0x55f3ef9ed4d0 2024-08-31T06:19:53.897 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.895+0000 7f42732b6740 1 -- 172.21.15.16:0/517397028 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4258012aa0 con 0x55f3ef9ed4d0 2024-08-31T06:19:53.898 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.896+0000 7f424b400640 1 -- 172.21.15.16:0/517397028 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8470+0+0 (secure 0 0 0) 0x7f4258012450 con 0x55f3ef9ed4d0 2024-08-31T06:19:53.898 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.896+0000 7f424b400640 1 -- 172.21.15.16:0/517397028 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f4258034050 con 0x55f3ef9ed4d0 2024-08-31T06:19:53.899 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.896+0000 7f42732b6740 10 client.28438 fetch_fsmap learned FSMap version 49 2024-08-31T06:19:53.899 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.896+0000 7f42732b6740 1 -- 172.21.15.16:0/517397028 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55f3efaf2620 con 0x55f3ef9ed4d0 2024-08-31T06:19:53.899 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.897+0000 7f424b400640 1 -- 172.21.15.16:0/517397028 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4258012aa0 con 0x55f3ef9ed4d0 2024-08-31T06:19:53.900 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.897+0000 7f424b400640 1 -- 172.21.15.16:0/517397028 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f42580037b0 con 0x55f3ef9ed4d0 2024-08-31T06:19:53.900 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.897+0000 7f42732b6740 10 client.28438 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:19:53.901 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.897+0000 7f42732b6740 10 client.28438 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:19:53.901 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.897+0000 7f42732b6740 20 client.28438 populate_metadata read hostname 'smithi016' 2024-08-31T06:19:53.901 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.897+0000 7f42732b6740 1 --2- 172.21.15.16:0/517397028 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f3ef9ccd80 0x55f3efddffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:53.901 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.897+0000 7f42732b6740 4 client.28438 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:19:53.901 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.897+0000 7f42732b6740 1 -- 172.21.15.16:0/517397028 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3efd28ec0 con 0x55f3ef9ccd80 2024-08-31T06:19:53.901 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.898+0000 7f4268e00640 1 --2- 172.21.15.16:0/517397028 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f3ef9ccd80 0x55f3efddffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:53.901 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.899+0000 7f4268e00640 1 --2- 172.21.15.16:0/517397028 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f3ef9ccd80 0x55f3efddffe0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8724d018672306c0 server_cookie=3da2e86c2e077763 in_seq=0 out_seq=0 2024-08-31T06:19:53.901 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.899+0000 7f424b400640 10 client.28438 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:19:53.903 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.900+0000 7f424b400640 1 -- 172.21.15.16:0/517397028 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f3efd28ec0 con 0x55f3ef9ccd80 2024-08-31T06:19:53.903 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.900+0000 7f424b400640 10 client.28438 handle_command_reply: tid=0 2024-08-31T06:19:53.903 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.900+0000 7f42732b6740 1 client.28438 shutdown 2024-08-31T06:19:53.903 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.900+0000 7f42732b6740 1 -- 172.21.15.16:0/517397028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3ef9ed4d0 msgr2=0x55f3efdbf580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:53.903 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.900+0000 7f42732b6740 1 --2- 172.21.15.16:0/517397028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3ef9ed4d0 0x55f3efdbf580 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f425801c040 tx=0x7f425800f7c0 comp rx=0 tx=0).stop 2024-08-31T06:19:53.903 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.900+0000 7f42732b6740 1 -- 172.21.15.16:0/517397028 shutdown_connections 2024-08-31T06:19:53.903 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.900+0000 7f42732b6740 1 --2- 172.21.15.16:0/517397028 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f3ef9ccd80 0x55f3efddffe0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:53.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.900+0000 7f42732b6740 1 --2- 172.21.15.16:0/517397028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3ef9ed4d0 0x55f3efdbf580 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:53.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.900+0000 7f42732b6740 1 --2- 172.21.15.16:0/517397028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3efa0e1b0 0x55f3efdc7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:53.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.900+0000 7f42732b6740 1 --2- 172.21.15.16:0/517397028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3ef9e52f0 0x55f3efdc7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:53.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.900+0000 7f42732b6740 1 -- 172.21.15.16:0/517397028 >> 172.21.15.16:0/517397028 conn(0x55f3efa02300 msgr2=0x55f3efdb2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:53.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.901+0000 7f424b400640 0 client.28438 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:19:53.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.901+0000 7f42732b6740 1 -- 172.21.15.16:0/517397028 shutdown_connections 2024-08-31T06:19:53.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.901+0000 7f42732b6740 1 -- 172.21.15.16:0/517397028 wait complete. 2024-08-31T06:19:53.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.901+0000 7f42732b6740 20 client.28438 trim_cache size 0 max 16384 2024-08-31T06:19:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.933+0000 7f42732b6740 1 Processor -- start 2024-08-31T06:19:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.933+0000 7f42732b6740 1 -- start start 2024-08-31T06:19:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.934+0000 7f42732b6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3ef9e52f0 0x55f3efdc2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.934+0000 7f42732b6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3ef9ed4d0 0x55f3efdc46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.934+0000 7f42732b6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3efa0e1b0 0x55f3efdc4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.934+0000 7f42732b6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4244011140 con 0x55f3ef9e52f0 2024-08-31T06:19:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.934+0000 7f42732b6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f424400e950 con 0x55f3efa0e1b0 2024-08-31T06:19:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.934+0000 7f42732b6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f424400c7c0 con 0x55f3ef9ed4d0 2024-08-31T06:19:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.934+0000 7f4268e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3ef9e52f0 0x55f3efdc2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.934+0000 7f4268e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3ef9e52f0 0x55f3efdc2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55856/0 (socket says 172.21.15.16:55856) 2024-08-31T06:19:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.934+0000 7f4268e00640 1 -- 172.21.15.16:0/3293127494 learned_addr learned my addr 172.21.15.16:0/3293127494 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.934+0000 7f4269800640 1 --2- 172.21.15.16:0/3293127494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3ef9ed4d0 0x55f3efdc46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.934+0000 7f4268e00640 1 --2- 172.21.15.16:0/3293127494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3efa0e1b0 0x55f3efdc4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.935+0000 7f4268e00640 1 -- 172.21.15.16:0/3293127494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3ef9ed4d0 msgr2=0x55f3efdc46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.935+0000 7f4268e00640 1 --2- 172.21.15.16:0/3293127494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3ef9ed4d0 0x55f3efdc46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.935+0000 7f4268e00640 1 -- 172.21.15.16:0/3293127494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3efa0e1b0 msgr2=0x55f3efdc4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.935+0000 7f4268e00640 1 --2- 172.21.15.16:0/3293127494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3efa0e1b0 0x55f3efdc4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.935+0000 7f4268e00640 1 -- 172.21.15.16:0/3293127494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3efdc8350 con 0x55f3ef9e52f0 2024-08-31T06:19:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.935+0000 7f4268e00640 1 --2- 172.21.15.16:0/3293127494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3ef9e52f0 0x55f3efdc2920 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f425002a910 tx=0x7f4250009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cea205fe23ea19dd server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.935+0000 7f4256000640 1 -- 172.21.15.16:0/3293127494 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4250041070 con 0x55f3ef9e52f0 2024-08-31T06:19:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.935+0000 7f4256000640 1 -- 172.21.15.16:0/3293127494 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4250047040 con 0x55f3ef9e52f0 2024-08-31T06:19:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.935+0000 7f4256000640 1 -- 172.21.15.16:0/3293127494 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4250035040 con 0x55f3ef9e52f0 2024-08-31T06:19:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.936+0000 7f42732b6740 1 -- 172.21.15.16:0/3293127494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3ef9e52f0 msgr2=0x55f3efdc2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.936+0000 7f42732b6740 1 --2- 172.21.15.16:0/3293127494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3ef9e52f0 0x55f3efdc2920 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f425002a910 tx=0x7f4250009260 comp rx=0 tx=0).stop 2024-08-31T06:19:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.936+0000 7f42732b6740 1 -- 172.21.15.16:0/3293127494 shutdown_connections 2024-08-31T06:19:53.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.936+0000 7f42732b6740 1 --2- 172.21.15.16:0/3293127494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3ef9ed4d0 0x55f3efdc46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:53.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.936+0000 7f42732b6740 1 --2- 172.21.15.16:0/3293127494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3efa0e1b0 0x55f3efdc4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:53.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.936+0000 7f42732b6740 1 --2- 172.21.15.16:0/3293127494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3ef9e52f0 0x55f3efdc2920 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:53.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.936+0000 7f42732b6740 1 -- 172.21.15.16:0/3293127494 >> 172.21.15.16:0/3293127494 conn(0x55f3efa02300 msgr2=0x55f3efdc2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:53.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.936+0000 7f42732b6740 1 -- 172.21.15.16:0/3293127494 shutdown_connections 2024-08-31T06:19:53.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.936+0000 7f42732b6740 1 -- 172.21.15.16:0/3293127494 wait complete. 2024-08-31T06:19:53.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.937+0000 7f42732b6740 1 -- start start 2024-08-31T06:19:53.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.937+0000 7f42732b6740 1 Processor -- start 2024-08-31T06:19:53.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.938+0000 7f42732b6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3ef9e52f0 0x55f3efdef5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:53.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.938+0000 7f42732b6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3ef9ed4d0 0x55f3efdefaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:53.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.938+0000 7f42732b6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3efa0e1b0 0x55f3efdf0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:53.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.938+0000 7f42732b6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f424400ce10 con 0x55f3ef9e52f0 2024-08-31T06:19:53.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.938+0000 7f42732b6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f424400e280 con 0x55f3ef9ed4d0 2024-08-31T06:19:53.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.938+0000 7f42732b6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f424400d4e0 con 0x55f3efa0e1b0 2024-08-31T06:19:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.938+0000 7f42732b6740 10 client.0 _ll_register_callbacks cb 0x7f4258004420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:19:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.938+0000 7f4269800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3ef9ed4d0 0x55f3efdefaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.938+0000 7f4268e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3ef9e52f0 0x55f3efdef5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.938+0000 7f4269800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3ef9ed4d0 0x55f3efdefaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:49346/0 (socket says 172.21.15.16:49346) 2024-08-31T06:19:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.938+0000 7f4269800640 1 -- 172.21.15.16:0/1764230417 learned_addr learned my addr 172.21.15.16:0/1764230417 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.938+0000 7f4269800640 1 -- 172.21.15.16:0/1764230417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3efa0e1b0 msgr2=0x55f3efdf0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.938+0000 7f4268e00640 1 --2- 172.21.15.16:0/1764230417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3efa0e1b0 0x55f3efdf0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.938+0000 7f4269800640 1 --2- 172.21.15.16:0/1764230417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3efa0e1b0 0x55f3efdf0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.938+0000 7f4269800640 1 -- 172.21.15.16:0/1764230417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3ef9e52f0 msgr2=0x55f3efdef5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.938+0000 7f4269800640 1 --2- 172.21.15.16:0/1764230417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3ef9e52f0 0x55f3efdef5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.938+0000 7f4269800640 1 -- 172.21.15.16:0/1764230417 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3efe49830 con 0x55f3ef9ed4d0 2024-08-31T06:19:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.938+0000 7f4268e00640 1 --2- 172.21.15.16:0/1764230417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3efa0e1b0 0x55f3efdf0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:19:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.939+0000 7f4269800640 1 --2- 172.21.15.16:0/1764230417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3ef9ed4d0 0x55f3efdefaf0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4258003800 tx=0x7f4258000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5abd542f1cd007e8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.939+0000 7f424b400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:19:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.939+0000 7f424b400640 1 -- 172.21.15.16:0/1764230417 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425803d060 con 0x55f3ef9ed4d0 2024-08-31T06:19:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.939+0000 7f424b400640 1 -- 172.21.15.16:0/1764230417 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8470+0+0 (secure 0 0 0) 0x7f4258005240 con 0x55f3ef9ed4d0 2024-08-31T06:19:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.939+0000 7f42732b6740 1 -- 172.21.15.16:0/1764230417 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4250047040 con 0x55f3ef9ed4d0 2024-08-31T06:19:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.939+0000 7f424b400640 1 -- 172.21.15.16:0/1764230417 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f4258029040 con 0x55f3ef9ed4d0 2024-08-31T06:19:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.939+0000 7f42732b6740 10 client.28424 fetch_fsmap learned FSMap version 49 2024-08-31T06:19:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.939+0000 7f42732b6740 1 -- 172.21.15.16:0/1764230417 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55f3efe4aaa0 con 0x55f3ef9ed4d0 2024-08-31T06:19:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.940+0000 7f424b400640 1 -- 172.21.15.16:0/1764230417 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4250047040 con 0x55f3ef9ed4d0 2024-08-31T06:19:53.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.941+0000 7f424b400640 1 -- 172.21.15.16:0/1764230417 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f4258004680 con 0x55f3ef9ed4d0 2024-08-31T06:19:53.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.941+0000 7f42732b6740 10 client.28424 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:19:53.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.941+0000 7f42732b6740 10 client.28424 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:19:53.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.941+0000 7f42732b6740 20 client.28424 populate_metadata read hostname 'smithi016' 2024-08-31T06:19:53.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.941+0000 7f42732b6740 1 --2- 172.21.15.16:0/1764230417 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f3ef9a21c0 0x55f3efe4d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:53.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.941+0000 7f42732b6740 4 client.28424 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:19:53.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.941+0000 7f42732b6740 1 -- 172.21.15.16:0/1764230417 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f3efd24960 con 0x55f3ef9a21c0 2024-08-31T06:19:53.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.942+0000 7f4268e00640 1 --2- 172.21.15.16:0/1764230417 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f3ef9a21c0 0x55f3efe4d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:53.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.943+0000 7f4268e00640 1 --2- 172.21.15.16:0/1764230417 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f3ef9a21c0 0x55f3efe4d240 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d03c26baf9591c3 server_cookie=9819b742a038a9ee in_seq=0 out_seq=0 2024-08-31T06:19:53.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.943+0000 7f424b400640 10 client.28424 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:19:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.959+0000 7f424b400640 1 -- 172.21.15.16:0/1764230417 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f3efd24960 con 0x55f3ef9a21c0 2024-08-31T06:19:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.960+0000 7f424b400640 10 client.28424 handle_command_reply: tid=0 2024-08-31T06:19:53.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.960+0000 7f42732b6740 1 client.28424 shutdown 2024-08-31T06:19:53.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.960+0000 7f42732b6740 1 -- 172.21.15.16:0/1764230417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3ef9ed4d0 msgr2=0x55f3efdefaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:53.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.960+0000 7f42732b6740 1 --2- 172.21.15.16:0/1764230417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3ef9ed4d0 0x55f3efdefaf0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4258003800 tx=0x7f4258000bc0 comp rx=0 tx=0).stop 2024-08-31T06:19:53.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.961+0000 7f42732b6740 1 -- 172.21.15.16:0/1764230417 shutdown_connections 2024-08-31T06:19:53.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.961+0000 7f42732b6740 1 --2- 172.21.15.16:0/1764230417 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55f3ef9a21c0 0x55f3efe4d240 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:53.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.961+0000 7f42732b6740 1 --2- 172.21.15.16:0/1764230417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3efa0e1b0 0x55f3efdf0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:53.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.961+0000 7f42732b6740 1 --2- 172.21.15.16:0/1764230417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3ef9ed4d0 0x55f3efdefaf0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:53.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.961+0000 7f42732b6740 1 --2- 172.21.15.16:0/1764230417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3ef9e52f0 0x55f3efdef5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:53.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.961+0000 7f424b400640 0 client.28424 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:19:53.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.961+0000 7f42732b6740 1 -- 172.21.15.16:0/1764230417 >> 172.21.15.16:0/1764230417 conn(0x55f3efa02300 msgr2=0x55f3efdaf6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:53.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.961+0000 7f42732b6740 1 -- 172.21.15.16:0/1764230417 shutdown_connections 2024-08-31T06:19:53.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.961+0000 7f42732b6740 1 -- 172.21.15.16:0/1764230417 wait complete. 2024-08-31T06:19:53.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.961+0000 7f42732b6740 20 client.28424 trim_cache size 0 max 16384 2024-08-31T06:19:53.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:19:53.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:19:53.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:19:53.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:19:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.962+0000 7f426c200640 1 -- 172.21.15.16:0/1890783723 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f424c0b4df0 msgr2=0x7f424c0b7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.962+0000 7f426c200640 1 --2- 172.21.15.16:0/1890783723 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f424c0b4df0 0x7f424c0b7300 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f425c0a8160 tx=0x7f425c009200 comp rx=0 tx=0).stop 2024-08-31T06:19:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.962+0000 7f426c200640 1 -- 172.21.15.16:0/1890783723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4264109b10 msgr2=0x7f42641a1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.962+0000 7f426c200640 1 --2- 172.21.15.16:0/1890783723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4264109b10 0x7f42641a1c90 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f425c00eb10 tx=0x7f425c00c2d0 comp rx=0 tx=0).stop 2024-08-31T06:19:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.962+0000 7f426c200640 1 -- 172.21.15.16:0/1890783723 shutdown_connections 2024-08-31T06:19:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.962+0000 7f426c200640 1 --2- 172.21.15.16:0/1890783723 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f424c0b4df0 0x7f424c0b7300 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.962+0000 7f426c200640 1 --2- 172.21.15.16:0/1890783723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4264109b10 0x7f42641a1c90 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.962+0000 7f426c200640 1 --2- 172.21.15.16:0/1890783723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42641091c0 0x7f426419ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.962+0000 7f426c200640 1 --2- 172.21.15.16:0/1890783723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4264103890 0x7f426419c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.962+0000 7f426c200640 1 -- 172.21.15.16:0/1890783723 >> 172.21.15.16:0/1890783723 conn(0x7f42640ff310 msgr2=0x7f426410e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.962+0000 7f426c200640 1 -- 172.21.15.16:0/1890783723 shutdown_connections 2024-08-31T06:19:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:19:53.962+0000 7f426c200640 1 -- 172.21.15.16:0/1890783723 wait complete. 2024-08-31T06:19:55.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:54 smithi155 ceph-mon[21742]: pgmap v758: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 6.9 MiB/s wr, 671 op/s 2024-08-31T06:19:55.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:54 smithi016 ceph-mon[17194]: pgmap v758: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 6.9 MiB/s wr, 671 op/s 2024-08-31T06:19:55.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:54 smithi184 ceph-mon[22331]: pgmap v758: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 6.9 MiB/s wr, 671 op/s 2024-08-31T06:19:56.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 750.0 s, 1380.0 tps, lat 71.222 ms stddev 84.047 2024-08-31T06:19:57.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:56 smithi155 ceph-mon[21742]: pgmap v759: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.6 MiB/s wr, 764 op/s 2024-08-31T06:19:57.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:56 smithi016 ceph-mon[17194]: pgmap v759: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.6 MiB/s wr, 764 op/s 2024-08-31T06:19:57.214 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T06:19:57.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:56 smithi184 ceph-mon[22331]: pgmap v759: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.6 MiB/s wr, 764 op/s 2024-08-31T06:19:57.850 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:19:58.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.723+0000 7f032f200640 1 -- 172.21.15.16:0/4159979508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0330109090 msgr2=0x7f03301094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:58.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.723+0000 7f032f200640 1 --2- 172.21.15.16:0/4159979508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0330109090 0x7f03301094a0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f031c009860 tx=0x7f031c02a340 comp rx=0 tx=0).stop 2024-08-31T06:19:58.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.724+0000 7f032f200640 1 -- 172.21.15.16:0/4159979508 shutdown_connections 2024-08-31T06:19:58.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.724+0000 7f032f200640 1 --2- 172.21.15.16:0/4159979508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03301099e0 0x7f0330110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:58.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.724+0000 7f032f200640 1 --2- 172.21.15.16:0/4159979508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0330109090 0x7f03301094a0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:58.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.724+0000 7f032f200640 1 --2- 172.21.15.16:0/4159979508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0330103750 0x7f0330103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:58.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.724+0000 7f032f200640 1 -- 172.21.15.16:0/4159979508 >> 172.21.15.16:0/4159979508 conn(0x7f03300ff1a0 msgr2=0x7f03301015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:58.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.724+0000 7f032f200640 1 -- 172.21.15.16:0/4159979508 shutdown_connections 2024-08-31T06:19:58.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.724+0000 7f032f200640 1 -- 172.21.15.16:0/4159979508 wait complete. 2024-08-31T06:19:58.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.726+0000 7f032f200640 1 Processor -- start 2024-08-31T06:19:58.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.726+0000 7f032f200640 1 -- start start 2024-08-31T06:19:58.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.726+0000 7f032f200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0330103750 0x7f0330198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:58.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.726+0000 7f032f200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0330109090 0x7f0330199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:58.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.726+0000 7f032f200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03301099e0 0x7f033019fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:58.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.726+0000 7f032f200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0330113650 con 0x7f0330103750 2024-08-31T06:19:58.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.726+0000 7f032f200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f03301134d0 con 0x7f0330109090 2024-08-31T06:19:58.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.726+0000 7f032f200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f03301137d0 con 0x7f03301099e0 2024-08-31T06:19:58.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.727+0000 7f032de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0330103750 0x7f0330198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:58.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.727+0000 7f032de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0330103750 0x7f0330198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55086/0 (socket says 172.21.15.16:55086) 2024-08-31T06:19:58.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.727+0000 7f032de00640 1 -- 172.21.15.16:0/810440262 learned_addr learned my addr 172.21.15.16:0/810440262 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:58.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.727+0000 7f032e800640 1 --2- 172.21.15.16:0/810440262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03301099e0 0x7f033019fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:58.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.727+0000 7f032d400640 1 --2- 172.21.15.16:0/810440262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0330109090 0x7f0330199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:58.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.728+0000 7f032e800640 1 -- 172.21.15.16:0/810440262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0330109090 msgr2=0x7f0330199060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:58.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.728+0000 7f032e800640 1 --2- 172.21.15.16:0/810440262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0330109090 0x7f0330199060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:58.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.728+0000 7f032e800640 1 -- 172.21.15.16:0/810440262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0330103750 msgr2=0x7f0330198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:58.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.728+0000 7f032e800640 1 --2- 172.21.15.16:0/810440262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0330103750 0x7f0330198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:58.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.728+0000 7f032e800640 1 -- 172.21.15.16:0/810440262 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0330199930 con 0x7f03301099e0 2024-08-31T06:19:58.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.728+0000 7f032e800640 1 --2- 172.21.15.16:0/810440262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03301099e0 0x7f033019fb80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f032800cab0 tx=0x7f032800cf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:58.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.728+0000 7f0326a00640 1 -- 172.21.15.16:0/810440262 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032800ae20 con 0x7f03301099e0 2024-08-31T06:19:58.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.729+0000 7f0326a00640 1 -- 172.21.15.16:0/810440262 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f032800f900 con 0x7f03301099e0 2024-08-31T06:19:58.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.729+0000 7f032f200640 1 -- 172.21.15.16:0/810440262 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03301a0180 con 0x7f03301099e0 2024-08-31T06:19:58.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.729+0000 7f0326a00640 1 -- 172.21.15.16:0/810440262 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0328018a00 con 0x7f03301099e0 2024-08-31T06:19:58.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.729+0000 7f032f200640 1 -- 172.21.15.16:0/810440262 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03301a06e0 con 0x7f03301099e0 2024-08-31T06:19:58.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.733+0000 7f033bdce740 1 Processor -- start 2024-08-31T06:19:58.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.735+0000 7f0326a00640 1 -- 172.21.15.16:0/810440262 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f032800f450 con 0x7f03301099e0 2024-08-31T06:19:58.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.735+0000 7f033bdce740 1 -- start start 2024-08-31T06:19:58.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.735+0000 7f033bdce740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562dbeff1070 0x562dbf396b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:58.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.735+0000 7f033bdce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562dbefc81b0 0x562dbf397060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:58.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.735+0000 7f033bdce740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dbefd0390 0x562dbf3975a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:58.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.735+0000 7f033bdce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562dbf30d770 con 0x562dbefc81b0 2024-08-31T06:19:58.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.735+0000 7f033bdce740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562dbf2fcd00 con 0x562dbefd0390 2024-08-31T06:19:58.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.735+0000 7f033bdce740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562dbf300cb0 con 0x562dbeff1070 2024-08-31T06:19:58.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.736+0000 7f032d400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dbefd0390 0x562dbf3975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:58.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.736+0000 7f032d400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dbefd0390 0x562dbf3975a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:33566/0 (socket says 172.21.15.16:33566) 2024-08-31T06:19:58.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.736+0000 7f032d400640 1 -- 172.21.15.16:0/223210024 learned_addr learned my addr 172.21.15.16:0/223210024 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:58.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.736+0000 7f032d400640 1 --2- 172.21.15.16:0/223210024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562dbeff1070 0x562dbf396b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:58.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.737+0000 7f032de00640 1 --2- 172.21.15.16:0/223210024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562dbefc81b0 0x562dbf397060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:58.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.737+0000 7f032d400640 1 -- 172.21.15.16:0/223210024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562dbeff1070 msgr2=0x562dbf396b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:58.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.737+0000 7f032d400640 1 --2- 172.21.15.16:0/223210024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562dbeff1070 0x562dbf396b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:58.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.737+0000 7f032d400640 1 -- 172.21.15.16:0/223210024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562dbefc81b0 msgr2=0x562dbf397060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:58.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.737+0000 7f032d400640 1 --2- 172.21.15.16:0/223210024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562dbefc81b0 0x562dbf397060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:58.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.737+0000 7f032d400640 1 -- 172.21.15.16:0/223210024 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dbf05c650 con 0x562dbefd0390 2024-08-31T06:19:58.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.737+0000 7f032de00640 1 --2- 172.21.15.16:0/223210024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562dbefc81b0 0x562dbf397060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:19:58.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.738+0000 7f032d400640 1 --2- 172.21.15.16:0/223210024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dbefd0390 0x562dbf3975a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f031c009860 tx=0x7f031c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb10b2f11e681b62 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:58.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.738+0000 7f0326a00640 1 --2- 172.21.15.16:0/810440262 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f03140b4da0 0x7f03140b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:58.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.738+0000 7f0326a00640 1 -- 172.21.15.16:0/810440262 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8470+0+0 (secure 0 0 0) 0x7f03280e5950 con 0x7f03301099e0 2024-08-31T06:19:58.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.738+0000 7f031b400640 1 -- 172.21.15.16:0/223210024 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031c037040 con 0x562dbefd0390 2024-08-31T06:19:58.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.738+0000 7f031b400640 1 -- 172.21.15.16:0/223210024 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f031c030900 con 0x562dbefd0390 2024-08-31T06:19:58.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.738+0000 7f031b400640 1 -- 172.21.15.16:0/223210024 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031c0055a0 con 0x562dbefd0390 2024-08-31T06:19:58.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.738+0000 7f032de00640 1 --2- 172.21.15.16:0/810440262 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f03140b4da0 0x7f03140b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:58.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.739+0000 7f033bdce740 1 -- 172.21.15.16:0/223210024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dbefd0390 msgr2=0x562dbf3975a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:58.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.739+0000 7f033bdce740 1 --2- 172.21.15.16:0/223210024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dbefd0390 0x562dbf3975a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f031c009860 tx=0x7f031c005f90 comp rx=0 tx=0).stop 2024-08-31T06:19:58.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.739+0000 7f033bdce740 1 -- 172.21.15.16:0/223210024 shutdown_connections 2024-08-31T06:19:58.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.739+0000 7f033bdce740 1 --2- 172.21.15.16:0/223210024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dbefd0390 0x562dbf3975a0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:58.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.740+0000 7f033bdce740 1 --2- 172.21.15.16:0/223210024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562dbefc81b0 0x562dbf397060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:58.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.740+0000 7f033bdce740 1 --2- 172.21.15.16:0/223210024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562dbeff1070 0x562dbf396b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:58.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.740+0000 7f032de00640 1 --2- 172.21.15.16:0/810440262 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f03140b4da0 0x7f03140b72b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x562dbf3957a0 tx=0x7f0320009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:58.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.740+0000 7f033bdce740 1 -- 172.21.15.16:0/223210024 >> 172.21.15.16:0/223210024 conn(0x562dbefe5460 msgr2=0x562dbf394a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:58.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.740+0000 7f033bdce740 1 -- 172.21.15.16:0/223210024 shutdown_connections 2024-08-31T06:19:58.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.740+0000 7f033bdce740 1 -- 172.21.15.16:0/223210024 wait complete. 2024-08-31T06:19:58.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.741+0000 7f033bdce740 1 -- start start 2024-08-31T06:19:58.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.741+0000 7f033bdce740 1 Processor -- start 2024-08-31T06:19:58.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.742+0000 7f033bdce740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562dbefc81b0 0x562dbf3a2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:58.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.742+0000 7f033bdce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562dbefd0390 0x562dbf3a2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:58.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.742+0000 7f033bdce740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dbeff1070 0x562dbf3ae560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:58.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.742+0000 7f033bdce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562dbf15f700 con 0x562dbefd0390 2024-08-31T06:19:58.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.742+0000 7f033bdce740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562dbf283cb0 con 0x562dbeff1070 2024-08-31T06:19:58.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.742+0000 7f033bdce740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562dbf3a5240 con 0x562dbefc81b0 2024-08-31T06:19:58.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.742+0000 7f033bdce740 10 client.0 _ll_register_callbacks cb 0x562dbf2a9430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:19:58.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.742+0000 7f032e800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dbeff1070 0x562dbf3ae560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:58.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.742+0000 7f032e800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dbeff1070 0x562dbf3ae560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:33574/0 (socket says 172.21.15.16:33574) 2024-08-31T06:19:58.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.743+0000 7f032e800640 1 -- 172.21.15.16:0/3415509297 learned_addr learned my addr 172.21.15.16:0/3415509297 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:58.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.743+0000 7f032de00640 1 --2- 172.21.15.16:0/3415509297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562dbefc81b0 0x562dbf3a2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:58.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.743+0000 7f032d400640 1 --2- 172.21.15.16:0/3415509297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562dbefd0390 0x562dbf3a2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:58.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.743+0000 7f032e800640 1 -- 172.21.15.16:0/3415509297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562dbefc81b0 msgr2=0x562dbf3a2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:58.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.743+0000 7f032e800640 1 --2- 172.21.15.16:0/3415509297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562dbefc81b0 0x562dbf3a2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:58.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.743+0000 7f032e800640 1 -- 172.21.15.16:0/3415509297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562dbefd0390 msgr2=0x562dbf3a2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:58.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.743+0000 7f032e800640 1 --2- 172.21.15.16:0/3415509297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562dbefd0390 0x562dbf3a2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:58.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.744+0000 7f032e800640 1 -- 172.21.15.16:0/3415509297 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dbf2c4870 con 0x562dbeff1070 2024-08-31T06:19:58.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.744+0000 7f032d400640 1 --2- 172.21.15.16:0/3415509297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562dbefd0390 0x562dbf3a2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:19:58.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.744+0000 7f032e800640 1 --2- 172.21.15.16:0/3415509297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dbeff1070 0x562dbf3ae560 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f03280a8040 tx=0x7f032800f880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f642e6946b58732a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:58.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.744+0000 7f030fe00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:19:58.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.744+0000 7f030fe00640 1 -- 172.21.15.16:0/3415509297 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03280b0880 con 0x562dbeff1070 2024-08-31T06:19:58.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.744+0000 7f030fe00640 1 -- 172.21.15.16:0/3415509297 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8470+0+0 (secure 0 0 0) 0x7f03280a8600 con 0x562dbeff1070 2024-08-31T06:19:58.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.744+0000 7f033bdce740 1 -- 172.21.15.16:0/3415509297 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f031c030900 con 0x562dbeff1070 2024-08-31T06:19:58.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.745+0000 7f030fe00640 1 -- 172.21.15.16:0/3415509297 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f03280b5330 con 0x562dbeff1070 2024-08-31T06:19:58.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.745+0000 7f033bdce740 10 client.28436 fetch_fsmap learned FSMap version 49 2024-08-31T06:19:58.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.745+0000 7f033bdce740 1 -- 172.21.15.16:0/3415509297 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x562dbf3b1280 con 0x562dbeff1070 2024-08-31T06:19:58.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.745+0000 7f030fe00640 1 -- 172.21.15.16:0/3415509297 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f031c030900 con 0x562dbeff1070 2024-08-31T06:19:58.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.745+0000 7f030fe00640 1 -- 172.21.15.16:0/3415509297 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f03280b4ea0 con 0x562dbeff1070 2024-08-31T06:19:58.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.746+0000 7f033bdce740 10 client.28436 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:19:58.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.746+0000 7f033bdce740 10 client.28436 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:19:58.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.746+0000 7f033bdce740 20 client.28436 populate_metadata read hostname 'smithi016' 2024-08-31T06:19:58.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.746+0000 7f033bdce740 1 --2- 172.21.15.16:0/3415509297 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x562dbefaf9f0 0x562dbf3b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:58.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.746+0000 7f033bdce740 4 client.28436 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:19:58.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.746+0000 7f033bdce740 1 -- 172.21.15.16:0/3415509297 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562dbf30f6c0 con 0x562dbefaf9f0 2024-08-31T06:19:58.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.746+0000 7f032de00640 1 --2- 172.21.15.16:0/3415509297 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x562dbefaf9f0 0x562dbf3b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:58.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.747+0000 7f032de00640 1 --2- 172.21.15.16:0/3415509297 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x562dbefaf9f0 0x562dbf3b4760 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e57b6fe88007806 server_cookie=9be40dccc2fe4fb5 in_seq=0 out_seq=0 2024-08-31T06:19:58.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.747+0000 7f030fe00640 10 client.28436 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:19:58.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.748+0000 7f030fe00640 1 -- 172.21.15.16:0/3415509297 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562dbf30f6c0 con 0x562dbefaf9f0 2024-08-31T06:19:58.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.748+0000 7f030fe00640 10 client.28436 handle_command_reply: tid=0 2024-08-31T06:19:58.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.748+0000 7f033bdce740 1 client.28436 shutdown 2024-08-31T06:19:58.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.749+0000 7f033bdce740 1 -- 172.21.15.16:0/3415509297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dbeff1070 msgr2=0x562dbf3ae560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:58.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.749+0000 7f033bdce740 1 --2- 172.21.15.16:0/3415509297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dbeff1070 0x562dbf3ae560 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f03280a8040 tx=0x7f032800f880 comp rx=0 tx=0).stop 2024-08-31T06:19:58.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.749+0000 7f033bdce740 1 -- 172.21.15.16:0/3415509297 shutdown_connections 2024-08-31T06:19:58.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.749+0000 7f033bdce740 1 --2- 172.21.15.16:0/3415509297 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x562dbefaf9f0 0x562dbf3b4760 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:58.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.749+0000 7f033bdce740 1 --2- 172.21.15.16:0/3415509297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dbeff1070 0x562dbf3ae560 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:58.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.749+0000 7f033bdce740 1 --2- 172.21.15.16:0/3415509297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562dbefd0390 0x562dbf3a2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:58.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.749+0000 7f033bdce740 1 --2- 172.21.15.16:0/3415509297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562dbefc81b0 0x562dbf3a2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:58.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.749+0000 7f033bdce740 1 -- 172.21.15.16:0/3415509297 >> 172.21.15.16:0/3415509297 conn(0x562dbefe5460 msgr2=0x562dbf399cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:58.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.749+0000 7f030fe00640 0 client.28436 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:19:58.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.750+0000 7f033bdce740 1 -- 172.21.15.16:0/3415509297 shutdown_connections 2024-08-31T06:19:58.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.750+0000 7f033bdce740 1 -- 172.21.15.16:0/3415509297 wait complete. 2024-08-31T06:19:58.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.750+0000 7f033bdce740 20 client.28436 trim_cache size 0 max 16384 2024-08-31T06:19:58.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.781+0000 7f033bdce740 1 Processor -- start 2024-08-31T06:19:58.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.781+0000 7f033bdce740 1 -- start start 2024-08-31T06:19:58.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.781+0000 7f033bdce740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dbefc81b0 0x562dbf3ae460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:58.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.781+0000 7f033bdce740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562dbefd0390 0x562dbf39bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:58.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.781+0000 7f033bdce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562dbeff1070 0x562dbf39c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:58.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.782+0000 7f033bdce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f03301c0890 con 0x562dbeff1070 2024-08-31T06:19:58.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.782+0000 7f033bdce740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f03301be0a0 con 0x562dbefc81b0 2024-08-31T06:19:58.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.782+0000 7f032de00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dbefc81b0 0x562dbf3ae460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:58.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.782+0000 7f033bdce740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f03301bbef0 con 0x562dbefd0390 2024-08-31T06:19:58.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.782+0000 7f032de00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dbefc81b0 0x562dbf3ae460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:33588/0 (socket says 172.21.15.16:33588) 2024-08-31T06:19:58.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.782+0000 7f032de00640 1 -- 172.21.15.16:0/1712393037 learned_addr learned my addr 172.21.15.16:0/1712393037 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:58.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.782+0000 7f032d400640 1 --2- 172.21.15.16:0/1712393037 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562dbefd0390 0x562dbf39bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:58.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.782+0000 7f032de00640 1 -- 172.21.15.16:0/1712393037 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562dbefd0390 msgr2=0x562dbf39bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:58.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.782+0000 7f032de00640 1 --2- 172.21.15.16:0/1712393037 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562dbefd0390 0x562dbf39bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:58.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.782+0000 7f032de00640 1 -- 172.21.15.16:0/1712393037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562dbeff1070 msgr2=0x562dbf39c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:58.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.782+0000 7f032e800640 1 --2- 172.21.15.16:0/1712393037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562dbeff1070 0x562dbf39c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:58.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.782+0000 7f032de00640 1 --2- 172.21.15.16:0/1712393037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562dbeff1070 0x562dbf39c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:58.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.782+0000 7f032de00640 1 -- 172.21.15.16:0/1712393037 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dbf3a3820 con 0x562dbefc81b0 2024-08-31T06:19:58.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.783+0000 7f032e800640 1 --2- 172.21.15.16:0/1712393037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562dbeff1070 0x562dbf39c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:19:58.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.783+0000 7f032de00640 1 --2- 172.21.15.16:0/1712393037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dbefc81b0 0x562dbf3ae460 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f03200096c0 tx=0x7f0320015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f87c76df84e47f9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:58.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.783+0000 7f031aa00640 1 -- 172.21.15.16:0/1712393037 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0320031640 con 0x562dbefc81b0 2024-08-31T06:19:58.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.783+0000 7f031aa00640 1 -- 172.21.15.16:0/1712393037 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0320031c90 con 0x562dbefc81b0 2024-08-31T06:19:58.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.783+0000 7f031aa00640 1 -- 172.21.15.16:0/1712393037 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0320029c70 con 0x562dbefc81b0 2024-08-31T06:19:58.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.785+0000 7f033bdce740 1 -- 172.21.15.16:0/1712393037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dbefc81b0 msgr2=0x562dbf3ae460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:58.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.785+0000 7f033bdce740 1 --2- 172.21.15.16:0/1712393037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dbefc81b0 0x562dbf3ae460 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f03200096c0 tx=0x7f0320015000 comp rx=0 tx=0).stop 2024-08-31T06:19:58.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.785+0000 7f033bdce740 1 -- 172.21.15.16:0/1712393037 shutdown_connections 2024-08-31T06:19:58.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.785+0000 7f033bdce740 1 --2- 172.21.15.16:0/1712393037 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562dbefd0390 0x562dbf39bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:58.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.785+0000 7f033bdce740 1 --2- 172.21.15.16:0/1712393037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562dbeff1070 0x562dbf39c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:58.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.785+0000 7f033bdce740 1 --2- 172.21.15.16:0/1712393037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dbefc81b0 0x562dbf3ae460 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:58.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.785+0000 7f033bdce740 1 -- 172.21.15.16:0/1712393037 >> 172.21.15.16:0/1712393037 conn(0x562dbefe5460 msgr2=0x562dbf3a0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:58.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.786+0000 7f033bdce740 1 -- 172.21.15.16:0/1712393037 shutdown_connections 2024-08-31T06:19:58.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.786+0000 7f033bdce740 1 -- 172.21.15.16:0/1712393037 wait complete. 2024-08-31T06:19:58.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.786+0000 7f033bdce740 1 -- start start 2024-08-31T06:19:58.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.787+0000 7f033bdce740 1 Processor -- start 2024-08-31T06:19:58.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.787+0000 7f033bdce740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dbefc81b0 0x562dbf3aac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:58.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.787+0000 7f033bdce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562dbefd0390 0x562dbf3a70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:58.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.788+0000 7f033bdce740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562dbeff1070 0x562dbf3a7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:58.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.788+0000 7f033bdce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f03301bc560 con 0x562dbefd0390 2024-08-31T06:19:58.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.788+0000 7f033bdce740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f03301bd9d0 con 0x562dbefc81b0 2024-08-31T06:19:58.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.788+0000 7f033bdce740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f03301bcc30 con 0x562dbeff1070 2024-08-31T06:19:58.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.788+0000 7f033bdce740 10 client.0 _ll_register_callbacks cb 0x7f03280ab780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:19:58.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.788+0000 7f032e800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562dbeff1070 0x562dbf3a7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:58.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.788+0000 7f032de00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dbefc81b0 0x562dbf3aac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:58.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.788+0000 7f032de00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dbefc81b0 0x562dbf3aac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:33600/0 (socket says 172.21.15.16:33600) 2024-08-31T06:19:58.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.788+0000 7f032e800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562dbeff1070 0x562dbf3a7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:43866/0 (socket says 172.21.15.16:43866) 2024-08-31T06:19:58.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.788+0000 7f032de00640 1 -- 172.21.15.16:0/3632063009 learned_addr learned my addr 172.21.15.16:0/3632063009 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:19:58.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.788+0000 7f032d400640 1 --2- 172.21.15.16:0/3632063009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562dbefd0390 0x562dbf3a70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:58.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.789+0000 7f032de00640 1 -- 172.21.15.16:0/3632063009 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562dbeff1070 msgr2=0x562dbf3a7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:58.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.789+0000 7f032de00640 1 --2- 172.21.15.16:0/3632063009 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562dbeff1070 0x562dbf3a7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:58.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.789+0000 7f032de00640 1 -- 172.21.15.16:0/3632063009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562dbefd0390 msgr2=0x562dbf3a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:19:58.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.789+0000 7f032de00640 1 --2- 172.21.15.16:0/3632063009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562dbefd0390 0x562dbf3a70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:58.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.789+0000 7f032de00640 1 -- 172.21.15.16:0/3632063009 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dbf3a7cf0 con 0x562dbefc81b0 2024-08-31T06:19:58.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.789+0000 7f032de00640 1 --2- 172.21.15.16:0/3632063009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dbefc81b0 0x562dbf3aac00 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0320029b20 tx=0x7f0320029b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ff3256be86fda1d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:19:58.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.789+0000 7f030fe00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:19:58.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.790+0000 7f030fe00640 1 -- 172.21.15.16:0/3632063009 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0320005050 con 0x562dbefc81b0 2024-08-31T06:19:58.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.790+0000 7f030fe00640 1 -- 172.21.15.16:0/3632063009 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8470+0+0 (secure 0 0 0) 0x7f03200051f0 con 0x562dbefc81b0 2024-08-31T06:19:58.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.790+0000 7f033bdce740 1 -- 172.21.15.16:0/3632063009 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0320031c90 con 0x562dbefc81b0 2024-08-31T06:19:58.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.790+0000 7f030fe00640 1 -- 172.21.15.16:0/3632063009 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f03200053f0 con 0x562dbefc81b0 2024-08-31T06:19:58.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.790+0000 7f033bdce740 10 client.28442 fetch_fsmap learned FSMap version 49 2024-08-31T06:19:58.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.790+0000 7f033bdce740 1 -- 172.21.15.16:0/3632063009 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x562dbf41ffc0 con 0x562dbefc81b0 2024-08-31T06:19:58.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.790+0000 7f030fe00640 1 -- 172.21.15.16:0/3632063009 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0320031c90 con 0x562dbefc81b0 2024-08-31T06:19:58.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.791+0000 7f030fe00640 1 -- 172.21.15.16:0/3632063009 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f032002f020 con 0x562dbefc81b0 2024-08-31T06:19:58.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.791+0000 7f033bdce740 10 client.28442 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:19:58.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.791+0000 7f033bdce740 10 client.28442 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:19:58.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.791+0000 7f033bdce740 20 client.28442 populate_metadata read hostname 'smithi016' 2024-08-31T06:19:58.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.791+0000 7f033bdce740 1 --2- 172.21.15.16:0/3632063009 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x562dbef84da0 0x562dbf422740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:19:58.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.791+0000 7f033bdce740 4 client.28442 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T06:19:58.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.791+0000 7f033bdce740 1 -- 172.21.15.16:0/3632063009 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562dbf30b160 con 0x562dbef84da0 2024-08-31T06:19:58.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.791+0000 7f032d400640 1 --2- 172.21.15.16:0/3632063009 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x562dbef84da0 0x562dbf422740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:19:58.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.792+0000 7f032d400640 1 --2- 172.21.15.16:0/3632063009 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x562dbef84da0 0x562dbf422740 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12278a767b48fe07 server_cookie=636e08a50c3c9cee in_seq=0 out_seq=0 2024-08-31T06:19:58.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.792+0000 7f030fe00640 10 client.28442 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:19:58.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.792+0000 7f030fe00640 1 -- 172.21.15.16:0/3632063009 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4634 (crc 0 0 0) 0x562dbf30b160 con 0x562dbef84da0 2024-08-31T06:19:58.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.792+0000 7f030fe00640 10 client.28442 handle_command_reply: tid=0 2024-08-31T06:19:58.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.793+0000 7f033bdce740 1 client.28442 shutdown 2024-08-31T06:19:58.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.793+0000 7f033bdce740 1 -- 172.21.15.16:0/3632063009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dbefc81b0 msgr2=0x562dbf3aac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:58.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.793+0000 7f033bdce740 1 --2- 172.21.15.16:0/3632063009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dbefc81b0 0x562dbf3aac00 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0320029b20 tx=0x7f0320029b50 comp rx=0 tx=0).stop 2024-08-31T06:19:58.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.794+0000 7f033bdce740 1 -- 172.21.15.16:0/3632063009 shutdown_connections 2024-08-31T06:19:58.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.794+0000 7f033bdce740 1 --2- 172.21.15.16:0/3632063009 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x562dbef84da0 0x562dbf422740 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:58.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.794+0000 7f033bdce740 1 --2- 172.21.15.16:0/3632063009 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562dbeff1070 0x562dbf3a7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:58.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.794+0000 7f033bdce740 1 --2- 172.21.15.16:0/3632063009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562dbefd0390 0x562dbf3a70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:58.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.794+0000 7f033bdce740 1 --2- 172.21.15.16:0/3632063009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dbefc81b0 0x562dbf3aac00 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:58.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.794+0000 7f030fe00640 0 client.28442 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:19:58.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.794+0000 7f033bdce740 1 -- 172.21.15.16:0/3632063009 >> 172.21.15.16:0/3632063009 conn(0x562dbefe5460 msgr2=0x562dbf397b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:58.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.795+0000 7f033bdce740 1 -- 172.21.15.16:0/3632063009 shutdown_connections 2024-08-31T06:19:58.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.795+0000 7f033bdce740 1 -- 172.21.15.16:0/3632063009 wait complete. 2024-08-31T06:19:58.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.795+0000 7f033bdce740 20 client.28442 trim_cache size 0 max 16384 2024-08-31T06:19:58.800 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:19:58.800 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-08-31T06:19:58.800 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {}, 2024-08-31T06:19:58.800 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-08-31T06:19:58.800 INFO:teuthology.orchestra.run.smithi016.stdout: "f1cdcb27-e3e4-4dc8-a0ae-7ace761d04c5": { 2024-08-31T06:19:58.800 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:19:22.275683+0000", 2024-08-31T06:19:58.801 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:58.801 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:58.801 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:58.801 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:58.801 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:58.801 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:58.801 INFO:teuthology.orchestra.run.smithi016.stdout: "0e44eb18-439c-46e4-a298-7d4d2ecd0b4a": { 2024-08-31T06:19:58.801 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:18:31.130765+0000", 2024-08-31T06:19:58.801 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:58.801 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:58.801 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:58.801 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:58.801 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:58.801 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:58.802 INFO:teuthology.orchestra.run.smithi016.stdout: "e497cd88-1e7c-4631-a4a8-cdd8e1fa6c83": { 2024-08-31T06:19:58.802 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:17:42.476795+0000", 2024-08-31T06:19:58.802 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:58.802 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:58.802 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:58.802 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:58.802 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:58.802 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:58.802 INFO:teuthology.orchestra.run.smithi016.stdout: "c300b7e8-f3fd-482c-a909-9329bc61e570": { 2024-08-31T06:19:58.802 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:06:49.868046+0000", 2024-08-31T06:19:58.802 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:58.802 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:58.802 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:58.803 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:58.803 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:58.803 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:58.803 INFO:teuthology.orchestra.run.smithi016.stdout: "6a03046e-cdf4-4a14-b9bd-b560b8968467": { 2024-08-31T06:19:58.803 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:12:38.002354+0000", 2024-08-31T06:19:58.803 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:58.803 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:58.803 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:58.803 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:58.803 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:58.803 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:58.803 INFO:teuthology.orchestra.run.smithi016.stdout: "004b47bb-0fe8-4d28-9d9d-78bde6a61199": { 2024-08-31T06:19:58.803 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:16:48.915926+0000", 2024-08-31T06:19:58.803 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:58.804 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:58.804 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:58.804 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:58.804 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:58.804 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:58.804 INFO:teuthology.orchestra.run.smithi016.stdout: "88d7bdb8-cb1c-45e5-ad29-4cd8da6c6003": { 2024-08-31T06:19:58.804 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:07:39.648942+0000", 2024-08-31T06:19:58.804 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:58.804 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:58.804 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:58.804 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:58.804 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:58.804 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:58.804 INFO:teuthology.orchestra.run.smithi016.stdout: "0fdcd762-533d-42be-9289-b71c8e3b81b1": { 2024-08-31T06:19:58.805 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:15:06.169112+0000", 2024-08-31T06:19:58.805 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:58.805 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:58.805 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:58.805 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:58.805 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:58.805 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:58.805 INFO:teuthology.orchestra.run.smithi016.stdout: "c8455d74-2fe7-4d93-bcf2-19b419d16016": { 2024-08-31T06:19:58.805 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:08:24.035960+0000", 2024-08-31T06:19:58.805 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:58.805 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:58.805 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:58.805 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:58.805 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:58.806 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:58.806 INFO:teuthology.orchestra.run.smithi016.stdout: "78d6ef02-8551-45e8-ad60-9627eb573fc1": { 2024-08-31T06:19:58.806 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:09:13.547865+0000", 2024-08-31T06:19:58.806 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:58.806 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:58.806 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:58.806 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:58.806 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:58.806 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:58.806 INFO:teuthology.orchestra.run.smithi016.stdout: "c1fd6904-2e02-47fa-939a-47349b9fe36b": { 2024-08-31T06:19:58.806 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:06.678691+0000", 2024-08-31T06:19:58.806 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:58.806 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:58.807 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:58.807 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:58.807 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:58.807 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:58.807 INFO:teuthology.orchestra.run.smithi016.stdout: "a10cce76-99ab-46b5-b3b2-07b82899b8f9": { 2024-08-31T06:19:58.807 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:11:43.967546+0000", 2024-08-31T06:19:58.807 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:58.807 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:58.807 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:58.807 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:58.807 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:58.807 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:58.807 INFO:teuthology.orchestra.run.smithi016.stdout: "91b72a34-4ba9-4126-a414-8b862f8c5f78": { 2024-08-31T06:19:58.808 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:13:28.663237+0000", 2024-08-31T06:19:58.808 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:58.808 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:58.808 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:58.808 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:58.808 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:58.808 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:58.808 INFO:teuthology.orchestra.run.smithi016.stdout: "e739a16e-8ace-43a8-a1ce-ac27d39ef8a3": { 2024-08-31T06:19:58.808 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:54.084646+0000", 2024-08-31T06:19:58.808 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:58.808 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:58.808 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:58.808 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:58.808 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:58.809 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:58.809 INFO:teuthology.orchestra.run.smithi016.stdout: "a64d5301-8558-4e9c-ac9b-8d0ac7057e74": { 2024-08-31T06:19:58.809 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:14:20.338104+0000", 2024-08-31T06:19:58.809 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:58.809 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:58.809 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:58.809 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:58.809 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:58.809 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:19:58.809 INFO:teuthology.orchestra.run.smithi016.stdout: "f9c4a22e-0c8a-4928-9b0d-e2ce1db5a86b": { 2024-08-31T06:19:58.809 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:15:57.496015+0000", 2024-08-31T06:19:58.809 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:19:58.809 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:19:58.809 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:19:58.809 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:19:58.810 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:19:58.810 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:19:58.810 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:19:58.810 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:19:58.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.796+0000 7f032f200640 1 -- 172.21.15.16:0/810440262 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f03140b4da0 msgr2=0x7f03140b72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:58.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.796+0000 7f032f200640 1 --2- 172.21.15.16:0/810440262 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f03140b4da0 0x7f03140b72b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x562dbf3957a0 tx=0x7f0320009260 comp rx=0 tx=0).stop 2024-08-31T06:19:58.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.797+0000 7f032f200640 1 -- 172.21.15.16:0/810440262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03301099e0 msgr2=0x7f033019fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:19:58.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.797+0000 7f032f200640 1 --2- 172.21.15.16:0/810440262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03301099e0 0x7f033019fb80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f032800cab0 tx=0x7f032800cf70 comp rx=0 tx=0).stop 2024-08-31T06:19:58.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.798+0000 7f032f200640 1 -- 172.21.15.16:0/810440262 shutdown_connections 2024-08-31T06:19:58.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.798+0000 7f032f200640 1 --2- 172.21.15.16:0/810440262 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f03140b4da0 0x7f03140b72b0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:58.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.798+0000 7f032f200640 1 --2- 172.21.15.16:0/810440262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03301099e0 0x7f033019fb80 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:58.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.798+0000 7f032f200640 1 --2- 172.21.15.16:0/810440262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0330109090 0x7f0330199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:58.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.798+0000 7f032f200640 1 --2- 172.21.15.16:0/810440262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0330103750 0x7f0330198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:19:58.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.798+0000 7f032f200640 1 -- 172.21.15.16:0/810440262 >> 172.21.15.16:0/810440262 conn(0x7f03300ff1a0 msgr2=0x7f0330107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:19:58.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.798+0000 7f032f200640 1 -- 172.21.15.16:0/810440262 shutdown_connections 2024-08-31T06:19:58.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:19:58.798+0000 7f032f200640 1 -- 172.21.15.16:0/810440262 wait complete. 2024-08-31T06:19:59.092 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:19:58 smithi016 ceph-mon[17194]: pgmap v760: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.5 MiB/s wr, 642 op/s 2024-08-31T06:19:59.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:19:58 smithi155 ceph-mon[21742]: pgmap v760: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.5 MiB/s wr, 642 op/s 2024-08-31T06:19:59.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:19:58 smithi184 ceph-mon[22331]: pgmap v760: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.5 MiB/s wr, 642 op/s 2024-08-31T06:20:00.592 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T06:20:00.593 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:20:01.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:00 smithi155 ceph-mon[21742]: pgmap v761: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 6.6 MiB/s wr, 655 op/s 2024-08-31T06:20:01.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:00 smithi155 ceph-mon[21742]: overall HEALTH_OK 2024-08-31T06:20:01.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:00 smithi016 ceph-mon[17194]: pgmap v761: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 6.6 MiB/s wr, 655 op/s 2024-08-31T06:20:01.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:00 smithi016 ceph-mon[17194]: overall HEALTH_OK 2024-08-31T06:20:01.186 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:20:01.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:00 smithi184 ceph-mon[22331]: pgmap v761: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 6.6 MiB/s wr, 655 op/s 2024-08-31T06:20:01.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:00 smithi184 ceph-mon[22331]: overall HEALTH_OK 2024-08-31T06:20:01.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 755.0 s, 1063.2 tps, lat 87.893 ms stddev 99.545 2024-08-31T06:20:02.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.141+0000 7f57dd000640 1 -- 172.21.15.16:0/188955282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57d810a230 msgr2=0x7f57d810a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:02.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.141+0000 7f57dd000640 1 --2- 172.21.15.16:0/188955282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57d810a230 0x7f57d810a640 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f57c8009ca0 tx=0x7f57c802a8b0 comp rx=0 tx=0).stop 2024-08-31T06:20:02.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.141+0000 7f57dd000640 1 -- 172.21.15.16:0/188955282 shutdown_connections 2024-08-31T06:20:02.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.141+0000 7f57dd000640 1 --2- 172.21.15.16:0/188955282 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57d810cda0 0x7f57d8116830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:02.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.141+0000 7f57dd000640 1 --2- 172.21.15.16:0/188955282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57d810c450 0x7f57d810c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:02.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.141+0000 7f57dd000640 1 --2- 172.21.15.16:0/188955282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57d810a230 0x7f57d810a640 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:02.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.141+0000 7f57dd000640 1 -- 172.21.15.16:0/188955282 >> 172.21.15.16:0/188955282 conn(0x7f57d80791c0 msgr2=0x7f57d807b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:02.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.141+0000 7f57dd000640 1 -- 172.21.15.16:0/188955282 shutdown_connections 2024-08-31T06:20:02.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.141+0000 7f57dd000640 1 -- 172.21.15.16:0/188955282 wait complete. 2024-08-31T06:20:02.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.144+0000 7f57dd000640 1 Processor -- start 2024-08-31T06:20:02.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.144+0000 7f57dd000640 1 -- start start 2024-08-31T06:20:02.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.149+0000 7f57dd000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57d810a230 0x7f57d8085a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:02.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.150+0000 7f57dd000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57d810c450 0x7f57d807ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:02.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.150+0000 7f57d7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57d810a230 0x7f57d8085a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:02.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.150+0000 7f57dd000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57d810cda0 0x7f57d807f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:02.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.150+0000 7f57dd000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f57d8119cc0 con 0x7f57d810c450 2024-08-31T06:20:02.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.150+0000 7f57dd000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f57d8119b40 con 0x7f57d810a230 2024-08-31T06:20:02.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.150+0000 7f57dd000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f57d8119e40 con 0x7f57d810cda0 2024-08-31T06:20:02.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.150+0000 7f57cea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57d810c450 0x7f57d807ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:02.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.150+0000 7f57cea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57d810c450 0x7f57d807ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55156/0 (socket says 172.21.15.16:55156) 2024-08-31T06:20:02.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.151+0000 7f57cea00640 1 -- 172.21.15.16:0/139082579 learned_addr learned my addr 172.21.15.16:0/139082579 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:02.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.151+0000 7f57d7e00640 1 --2- 172.21.15.16:0/139082579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57d810cda0 0x7f57d807f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:02.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.152+0000 7f57cea00640 1 -- 172.21.15.16:0/139082579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57d810cda0 msgr2=0x7f57d807f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:02.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.152+0000 7f57cea00640 1 --2- 172.21.15.16:0/139082579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57d810cda0 0x7f57d807f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:02.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.152+0000 7f57cea00640 1 -- 172.21.15.16:0/139082579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57d810a230 msgr2=0x7f57d8085a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:02.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.152+0000 7f57cea00640 1 --2- 172.21.15.16:0/139082579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57d810a230 0x7f57d8085a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:02.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.153+0000 7f57cea00640 1 -- 172.21.15.16:0/139082579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d807fa40 con 0x7f57d810c450 2024-08-31T06:20:02.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.153+0000 7f57d7400640 1 --2- 172.21.15.16:0/139082579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57d810a230 0x7f57d8085a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:20:02.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.153+0000 7f57cea00640 1 --2- 172.21.15.16:0/139082579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57d810c450 0x7f57d807ece0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f57c000d740 tx=0x7f57c000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:02.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.154+0000 7f57d4c00640 1 -- 172.21.15.16:0/139082579 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c0017070 con 0x7f57d810c450 2024-08-31T06:20:02.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.157+0000 7f57d4c00640 1 -- 172.21.15.16:0/139082579 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57c000de50 con 0x7f57d810c450 2024-08-31T06:20:02.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.157+0000 7f57d4c00640 1 -- 172.21.15.16:0/139082579 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c000f620 con 0x7f57d810c450 2024-08-31T06:20:02.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.157+0000 7f57dd000640 1 -- 172.21.15.16:0/139082579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57d80846e0 con 0x7f57d810c450 2024-08-31T06:20:02.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.157+0000 7f57dd000640 1 -- 172.21.15.16:0/139082579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57d8084bd0 con 0x7f57d810c450 2024-08-31T06:20:02.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.159+0000 7f57dd000640 1 -- 172.21.15.16:0/139082579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57d810a640 con 0x7f57d810c450 2024-08-31T06:20:02.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.163+0000 7f57d4c00640 1 -- 172.21.15.16:0/139082579 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f57c001a070 con 0x7f57d810c450 2024-08-31T06:20:02.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.166+0000 7f57d4c00640 1 --2- 172.21.15.16:0/139082579 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f57bc0b4d20 0x7f57bc0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:02.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.167+0000 7f57d7400640 1 --2- 172.21.15.16:0/139082579 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f57bc0b4d20 0x7f57bc0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:02.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.167+0000 7f57d4c00640 1 -- 172.21.15.16:0/139082579 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8470+0+0 (secure 0 0 0) 0x7f57c00e4ba0 con 0x7f57d810c450 2024-08-31T06:20:02.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.168+0000 7f57d7400640 1 --2- 172.21.15.16:0/139082579 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f57bc0b4d20 0x7f57bc0b7230 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f57c8000f80 tx=0x7f57c8009a00 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:02.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.177+0000 7f57d4c00640 1 -- 172.21.15.16:0/139082579 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f57c00aba40 con 0x7f57d810c450 2024-08-31T06:20:02.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.611+0000 7f57dd000640 1 -- 172.21.15.16:0/139082579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57d8111a30 con 0x7f57d810c450 2024-08-31T06:20:02.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.618+0000 7f57d4c00640 1 -- 172.21.15.16:0/139082579 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+11923 (secure 0 0 0) 0x7f57c00ab340 con 0x7f57d810c450 2024-08-31T06:20:02.623 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:20:02.623 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 49 2024-08-31T06:20:02.623 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":49,"btime":"2024-08-31T06:19:17:811356+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":49,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:19:17.033815+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24737,24724,24712]},"id":1}]} 2024-08-31T06:20:02.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.622+0000 7f57dd000640 1 -- 172.21.15.16:0/139082579 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f57bc0b4d20 msgr2=0x7f57bc0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:02.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.622+0000 7f57dd000640 1 --2- 172.21.15.16:0/139082579 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f57bc0b4d20 0x7f57bc0b7230 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f57c8000f80 tx=0x7f57c8009a00 comp rx=0 tx=0).stop 2024-08-31T06:20:02.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.622+0000 7f57dd000640 1 -- 172.21.15.16:0/139082579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57d810c450 msgr2=0x7f57d807ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:02.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.622+0000 7f57dd000640 1 --2- 172.21.15.16:0/139082579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57d810c450 0x7f57d807ece0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f57c000d740 tx=0x7f57c000dc00 comp rx=0 tx=0).stop 2024-08-31T06:20:02.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.622+0000 7f57dd000640 1 -- 172.21.15.16:0/139082579 shutdown_connections 2024-08-31T06:20:02.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.622+0000 7f57dd000640 1 --2- 172.21.15.16:0/139082579 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f57bc0b4d20 0x7f57bc0b7230 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:02.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.622+0000 7f57dd000640 1 --2- 172.21.15.16:0/139082579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57d810cda0 0x7f57d807f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:02.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.622+0000 7f57dd000640 1 --2- 172.21.15.16:0/139082579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57d810c450 0x7f57d807ece0 secure :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f57c000d740 tx=0x7f57c000dc00 comp rx=0 tx=0).stop 2024-08-31T06:20:02.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.622+0000 7f57dd000640 1 --2- 172.21.15.16:0/139082579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57d810a230 0x7f57d8085a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:02.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.622+0000 7f57dd000640 1 -- 172.21.15.16:0/139082579 >> 172.21.15.16:0/139082579 conn(0x7f57d80791c0 msgr2=0x7f57d810ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:02.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.623+0000 7f57dd000640 1 -- 172.21.15.16:0/139082579 shutdown_connections 2024-08-31T06:20:02.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:02.623+0000 7f57dd000640 1 -- 172.21.15.16:0/139082579 wait complete. 2024-08-31T06:20:02.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:02 smithi016 ceph-mon[17194]: pgmap v762: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 6.4 MiB/s wr, 641 op/s 2024-08-31T06:20:02.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:02 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/139082579' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:20:03.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:02 smithi155 ceph-mon[21742]: pgmap v762: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 6.4 MiB/s wr, 641 op/s 2024-08-31T06:20:03.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:02 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/139082579' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:20:03.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:02 smithi184 ceph-mon[22331]: pgmap v762: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 6.4 MiB/s wr, 641 op/s 2024-08-31T06:20:03.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:02 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/139082579' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:20:04.030 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T06:20:04.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:03 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:20:04.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:03 smithi155 ceph-mon[21742]: pgmap v763: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 9.2 MiB/s wr, 864 op/s 2024-08-31T06:20:04.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:03 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:20:04.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:03 smithi016 ceph-mon[17194]: pgmap v763: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 9.2 MiB/s wr, 864 op/s 2024-08-31T06:20:04.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:03 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:20:04.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:03 smithi184 ceph-mon[22331]: pgmap v763: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 9.2 MiB/s wr, 864 op/s 2024-08-31T06:20:04.930 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:20:05.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:04 smithi155 ceph-mon[21742]: osdmap e96: 12 total, 12 up, 12 in 2024-08-31T06:20:05.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:04 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:20:05.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:04 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:20:05.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:04 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:20:05.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:04 smithi016 ceph-mon[17194]: osdmap e96: 12 total, 12 up, 12 in 2024-08-31T06:20:05.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:04 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:20:05.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:04 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:20:05.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:04 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:20:05.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:04 smithi184 ceph-mon[22331]: osdmap e96: 12 total, 12 up, 12 in 2024-08-31T06:20:05.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:04 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:20:05.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:04 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:20:05.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:04 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:20:05.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:05 smithi016 ceph-mon[17194]: pgmap v765: 97 pgs: 20 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 69 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 19 MiB/s wr, 1.74k op/s 2024-08-31T06:20:05.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.933+0000 7fe838c00640 1 -- 172.21.15.16:0/1346458588 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe834078030 msgr2=0x7fe834113fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:05.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.934+0000 7fe838c00640 1 --2- 172.21.15.16:0/1346458588 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe834078030 0x7fe834113fd0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe820009900 tx=0x7fe82002a3f0 comp rx=0 tx=0).stop 2024-08-31T06:20:05.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.936+0000 7fe838c00640 1 -- 172.21.15.16:0/1346458588 shutdown_connections 2024-08-31T06:20:05.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.936+0000 7fe838c00640 1 --2- 172.21.15.16:0/1346458588 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe834078030 0x7fe834113fd0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:05.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.936+0000 7fe838c00640 1 --2- 172.21.15.16:0/1346458588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe834114510 0x7fe834116930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:05.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.936+0000 7fe838c00640 1 --2- 172.21.15.16:0/1346458588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8340776e0 0x7fe834077af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:05.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.936+0000 7fe838c00640 1 -- 172.21.15.16:0/1346458588 >> 172.21.15.16:0/1346458588 conn(0x7fe834101560 msgr2=0x7fe8341039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:05.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.936+0000 7fe838c00640 1 -- 172.21.15.16:0/1346458588 shutdown_connections 2024-08-31T06:20:05.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.936+0000 7fe838c00640 1 -- 172.21.15.16:0/1346458588 wait complete. 2024-08-31T06:20:05.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.939+0000 7fe838c00640 1 Processor -- start 2024-08-31T06:20:05.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.939+0000 7fe838c00640 1 -- start start 2024-08-31T06:20:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.940+0000 7fe838c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8340776e0 0x7fe834074ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.940+0000 7fe838c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe834078030 0x7fe83406de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.941+0000 7fe831e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8340776e0 0x7fe834074ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.941+0000 7fe838c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe834114510 0x7fe83406e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:05.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.941+0000 7fe838c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe834119ec0 con 0x7fe834078030 2024-08-31T06:20:05.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.941+0000 7fe831400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe834078030 0x7fe83406de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:05.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.941+0000 7fe838c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe834119d40 con 0x7fe834114510 2024-08-31T06:20:05.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.941+0000 7fe838c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe83411a040 con 0x7fe8340776e0 2024-08-31T06:20:05.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.941+0000 7fe831e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8340776e0 0x7fe834074ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:51242/0 (socket says 172.21.15.16:51242) 2024-08-31T06:20:05.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.941+0000 7fe831e00640 1 -- 172.21.15.16:0/4120532028 learned_addr learned my addr 172.21.15.16:0/4120532028 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:05.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.941+0000 7fe832800640 1 --2- 172.21.15.16:0/4120532028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe834114510 0x7fe83406e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:05.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.943+0000 7fe831400640 1 -- 172.21.15.16:0/4120532028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8340776e0 msgr2=0x7fe834074ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:05.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.943+0000 7fe831400640 1 --2- 172.21.15.16:0/4120532028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8340776e0 0x7fe834074ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:05.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.943+0000 7fe831400640 1 -- 172.21.15.16:0/4120532028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe834114510 msgr2=0x7fe83406e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:05.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.943+0000 7fe831400640 1 --2- 172.21.15.16:0/4120532028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe834114510 0x7fe83406e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:05.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.943+0000 7fe831400640 1 -- 172.21.15.16:0/4120532028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe83406eb20 con 0x7fe834078030 2024-08-31T06:20:05.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.944+0000 7fe831400640 1 --2- 172.21.15.16:0/4120532028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe834078030 0x7fe83406de70 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe820009230 tx=0x7fe820003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:05.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.945+0000 7fe82aa00640 1 -- 172.21.15.16:0/4120532028 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82003f070 con 0x7fe834078030 2024-08-31T06:20:05.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.945+0000 7fe838c00640 1 -- 172.21.15.16:0/4120532028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe834073810 con 0x7fe834078030 2024-08-31T06:20:05.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.945+0000 7fe838c00640 1 -- 172.21.15.16:0/4120532028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe834073d10 con 0x7fe834078030 2024-08-31T06:20:05.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.946+0000 7fe82aa00640 1 -- 172.21.15.16:0/4120532028 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe820031b60 con 0x7fe834078030 2024-08-31T06:20:05.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.946+0000 7fe82aa00640 1 -- 172.21.15.16:0/4120532028 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe820049060 con 0x7fe834078030 2024-08-31T06:20:05.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.950+0000 7fe83f7da740 1 Processor -- start 2024-08-31T06:20:05.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.952+0000 7fe83f7da740 1 -- start start 2024-08-31T06:20:05.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.953+0000 7fe83f7da740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38d2f2070 0x55c38d6978b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:05.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.954+0000 7fe83f7da740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38d2c91b0 0x55c38d697df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:05.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.954+0000 7fe83f7da740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c38d2d1390 0x55c38d698330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:05.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.954+0000 7fe832800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38d2f2070 0x55c38d6978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:05.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.954+0000 7fe832800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38d2f2070 0x55c38d6978b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:51244/0 (socket says 172.21.15.16:51244) 2024-08-31T06:20:05.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.954+0000 7fe832800640 1 -- 172.21.15.16:0/439272011 learned_addr learned my addr 172.21.15.16:0/439272011 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:05.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.954+0000 7fe83f7da740 1 -- 172.21.15.16:0/439272011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c38d60e7b0 con 0x55c38d2d1390 2024-08-31T06:20:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.954+0000 7fe83f7da740 1 -- 172.21.15.16:0/439272011 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c38d5fdd40 con 0x55c38d2c91b0 2024-08-31T06:20:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.954+0000 7fe83f7da740 1 -- 172.21.15.16:0/439272011 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c38d601cf0 con 0x55c38d2f2070 2024-08-31T06:20:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.955+0000 7fe831e00640 1 --2- 172.21.15.16:0/439272011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38d2c91b0 0x55c38d697df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.955+0000 7fe82aa00640 1 -- 172.21.15.16:0/4120532028 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fe820042070 con 0x7fe834078030 2024-08-31T06:20:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.955+0000 7fe832800640 1 -- 172.21.15.16:0/439272011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38d2c91b0 msgr2=0x55c38d697df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.955+0000 7fe832800640 1 --2- 172.21.15.16:0/439272011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38d2c91b0 0x55c38d697df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.955+0000 7fe832800640 1 -- 172.21.15.16:0/439272011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c38d2d1390 msgr2=0x55c38d698330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:20:05.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.955+0000 7fe832800640 1 --2- 172.21.15.16:0/439272011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c38d2d1390 0x55c38d698330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:05.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.955+0000 7fe832800640 1 -- 172.21.15.16:0/439272011 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c38d35d650 con 0x55c38d2f2070 2024-08-31T06:20:05.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.956+0000 7fe832800640 1 --2- 172.21.15.16:0/439272011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38d2f2070 0x55c38d6978b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe82c009fc0 tx=0x7fe82c010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edb8497184de1549 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:05.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.956+0000 7fe81f000640 1 -- 172.21.15.16:0/439272011 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82c003b70 con 0x55c38d2f2070 2024-08-31T06:20:05.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.957+0000 7fe81f000640 1 -- 172.21.15.16:0/439272011 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe82c003d10 con 0x55c38d2f2070 2024-08-31T06:20:05.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.957+0000 7fe81f000640 1 -- 172.21.15.16:0/439272011 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82c011a30 con 0x55c38d2f2070 2024-08-31T06:20:05.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.958+0000 7fe82aa00640 1 --2- 172.21.15.16:0/4120532028 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe8180b4d20 0x7fe8180b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:05.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.958+0000 7fe82aa00640 1 -- 172.21.15.16:0/4120532028 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8514+0+0 (secure 0 0 0) 0x7fe820105630 con 0x7fe834078030 2024-08-31T06:20:05.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.959+0000 7fe831e00640 1 --2- 172.21.15.16:0/4120532028 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe8180b4d20 0x7fe8180b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:05.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.960+0000 7fe83f7da740 1 -- 172.21.15.16:0/439272011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38d2f2070 msgr2=0x55c38d6978b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:05.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.960+0000 7fe831e00640 1 --2- 172.21.15.16:0/4120532028 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe8180b4d20 0x7fe8180b7230 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x55c38d696530 tx=0x7fe82400a380 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:05.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.960+0000 7fe83f7da740 1 --2- 172.21.15.16:0/439272011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38d2f2070 0x55c38d6978b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe82c009fc0 tx=0x7fe82c010860 comp rx=0 tx=0).stop 2024-08-31T06:20:05.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.961+0000 7fe83f7da740 1 -- 172.21.15.16:0/439272011 shutdown_connections 2024-08-31T06:20:05.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.961+0000 7fe83f7da740 1 --2- 172.21.15.16:0/439272011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c38d2d1390 0x55c38d698330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:05.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.961+0000 7fe83f7da740 1 --2- 172.21.15.16:0/439272011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38d2c91b0 0x55c38d697df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:05.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.961+0000 7fe83f7da740 1 --2- 172.21.15.16:0/439272011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38d2f2070 0x55c38d6978b0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:05.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.961+0000 7fe83f7da740 1 -- 172.21.15.16:0/439272011 >> 172.21.15.16:0/439272011 conn(0x55c38d2e6460 msgr2=0x55c38d6957a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:05.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.961+0000 7fe83f7da740 1 -- 172.21.15.16:0/439272011 shutdown_connections 2024-08-31T06:20:05.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.961+0000 7fe83f7da740 1 -- 172.21.15.16:0/439272011 wait complete. 2024-08-31T06:20:05.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.962+0000 7fe83f7da740 1 -- start start 2024-08-31T06:20:05.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.962+0000 7fe83f7da740 1 Processor -- start 2024-08-31T06:20:05.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.963+0000 7fe83f7da740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c38d2c91b0 0x55c38d6a3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:05.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.963+0000 7fe83f7da740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38d2d1390 0x55c38d6a3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:05.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.963+0000 7fe83f7da740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38d2f2070 0x55c38d6af410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:05.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.963+0000 7fe83f7da740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c38d460700 con 0x55c38d2c91b0 2024-08-31T06:20:05.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.963+0000 7fe83f7da740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c38d584cf0 con 0x55c38d2f2070 2024-08-31T06:20:05.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.963+0000 7fe83f7da740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c38d6a6090 con 0x55c38d2d1390 2024-08-31T06:20:05.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.963+0000 7fe83f7da740 10 client.0 _ll_register_callbacks cb 0x55c38d5aa470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:20:05.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.964+0000 7fe831e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c38d2c91b0 0x55c38d6a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:05.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.964+0000 7fe832800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38d2f2070 0x55c38d6af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:05.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.964+0000 7fe832800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38d2f2070 0x55c38d6af410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:34750/0 (socket says 172.21.15.16:34750) 2024-08-31T06:20:05.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.964+0000 7fe832800640 1 -- 172.21.15.16:0/2391605134 learned_addr learned my addr 172.21.15.16:0/2391605134 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:05.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.964+0000 7fe831400640 1 --2- 172.21.15.16:0/2391605134 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38d2d1390 0x55c38d6a3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:05.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.964+0000 7fe832800640 1 -- 172.21.15.16:0/2391605134 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38d2d1390 msgr2=0x55c38d6a3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:05.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.964+0000 7fe832800640 1 --2- 172.21.15.16:0/2391605134 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38d2d1390 0x55c38d6a3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:05.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.964+0000 7fe832800640 1 -- 172.21.15.16:0/2391605134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c38d2c91b0 msgr2=0x55c38d6a3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:05.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.964+0000 7fe832800640 1 --2- 172.21.15.16:0/2391605134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c38d2c91b0 0x55c38d6a3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:05.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.965+0000 7fe832800640 1 -- 172.21.15.16:0/2391605134 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c38d5c58b0 con 0x55c38d2f2070 2024-08-31T06:20:05.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.965+0000 7fe831e00640 1 --2- 172.21.15.16:0/2391605134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c38d2c91b0 0x55c38d6a3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:20:05.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.966+0000 7fe832800640 1 --2- 172.21.15.16:0/2391605134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38d2f2070 0x55c38d6af410 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fe82c0049a0 tx=0x7fe82c004c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=975fc5fa4400d0ba server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:05.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.966+0000 7fe813400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:20:05.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.967+0000 7fe813400640 1 -- 172.21.15.16:0/2391605134 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82c011bc0 con 0x55c38d2f2070 2024-08-31T06:20:05.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.967+0000 7fe813400640 1 -- 172.21.15.16:0/2391605134 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8514+0+0 (secure 0 0 0) 0x7fe82c011d60 con 0x55c38d2f2070 2024-08-31T06:20:05.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.967+0000 7fe83f7da740 1 -- 172.21.15.16:0/2391605134 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe82c003d10 con 0x55c38d2f2070 2024-08-31T06:20:05.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.967+0000 7fe813400640 1 -- 172.21.15.16:0/2391605134 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fe82c003ec0 con 0x55c38d2f2070 2024-08-31T06:20:05.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.967+0000 7fe83f7da740 10 client.28460 fetch_fsmap learned FSMap version 49 2024-08-31T06:20:05.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.967+0000 7fe83f7da740 1 -- 172.21.15.16:0/2391605134 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55c38d6b21c0 con 0x55c38d2f2070 2024-08-31T06:20:05.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.968+0000 7fe813400640 1 -- 172.21.15.16:0/2391605134 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe82c003d10 con 0x55c38d2f2070 2024-08-31T06:20:05.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.968+0000 7fe813400640 1 -- 172.21.15.16:0/2391605134 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fe82c0051d0 con 0x55c38d2f2070 2024-08-31T06:20:05.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.968+0000 7fe83f7da740 10 client.28460 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:20:05.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.968+0000 7fe83f7da740 10 client.28460 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T06:20:05.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.968+0000 7fe83f7da740 20 client.28460 populate_metadata read hostname 'smithi016' 2024-08-31T06:20:05.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.968+0000 7fe83f7da740 1 --2- 172.21.15.16:0/2391605134 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55c38d2b09f0 0x55c38d6b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:05.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.968+0000 7fe83f7da740 4 client.28460 mds_command: new command op to 24712 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:20:05.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.968+0000 7fe83f7da740 1 -- 172.21.15.16:0/2391605134 --> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c38d610700 con 0x55c38d2b09f0 2024-08-31T06:20:05.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.969+0000 7fe831e00640 1 --2- 172.21.15.16:0/2391605134 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55c38d2b09f0 0x55c38d6b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:05.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.969+0000 7fe831e00640 1 --2- 172.21.15.16:0/2391605134 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55c38d2b09f0 0x55c38d6b5520 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2833197a44cf77cc server_cookie=8faf93bbe15c4746 in_seq=0 out_seq=0 2024-08-31T06:20:05.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.969+0000 7fe813400640 10 client.28460 ms_handle_connect on v2:172.21.15.184:6838/4011051479 2024-08-31T06:20:05.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.971+0000 7fe813400640 1 -- 172.21.15.16:0/2391605134 <== mds.2 v2:172.21.15.184:6838/4011051479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c38d610700 con 0x55c38d2b09f0 2024-08-31T06:20:05.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.971+0000 7fe813400640 10 client.28460 handle_command_reply: tid=0 2024-08-31T06:20:05.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.971+0000 7fe83f7da740 1 client.28460 shutdown 2024-08-31T06:20:05.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.972+0000 7fe83f7da740 1 -- 172.21.15.16:0/2391605134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38d2f2070 msgr2=0x55c38d6af410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:05.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.972+0000 7fe83f7da740 1 --2- 172.21.15.16:0/2391605134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38d2f2070 0x55c38d6af410 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fe82c0049a0 tx=0x7fe82c004c50 comp rx=0 tx=0).stop 2024-08-31T06:20:05.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.972+0000 7fe83f7da740 1 -- 172.21.15.16:0/2391605134 shutdown_connections 2024-08-31T06:20:05.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.972+0000 7fe83f7da740 1 --2- 172.21.15.16:0/2391605134 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55c38d2b09f0 0x55c38d6b5520 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:05.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.972+0000 7fe83f7da740 1 --2- 172.21.15.16:0/2391605134 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38d2d1390 0x55c38d6a3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:05.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.972+0000 7fe83f7da740 1 --2- 172.21.15.16:0/2391605134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38d2f2070 0x55c38d6af410 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:05.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.972+0000 7fe83f7da740 1 --2- 172.21.15.16:0/2391605134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c38d2c91b0 0x55c38d6a3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:05.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.972+0000 7fe83f7da740 1 -- 172.21.15.16:0/2391605134 >> 172.21.15.16:0/2391605134 conn(0x55c38d2e6460 msgr2=0x55c38d69aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:05.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.972+0000 7fe813400640 0 client.28460 ms_handle_reset on v2:172.21.15.184:6838/4011051479 2024-08-31T06:20:05.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.972+0000 7fe83f7da740 1 -- 172.21.15.16:0/2391605134 shutdown_connections 2024-08-31T06:20:05.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.972+0000 7fe83f7da740 1 -- 172.21.15.16:0/2391605134 wait complete. 2024-08-31T06:20:05.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.972+0000 7fe83f7da740 20 client.28460 trim_cache size 0 max 16384 2024-08-31T06:20:05.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.992+0000 7fe83f7da740 1 Processor -- start 2024-08-31T06:20:05.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.992+0000 7fe83f7da740 1 -- start start 2024-08-31T06:20:05.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.993+0000 7fe83f7da740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38d2c91b0 0x55c38d6a1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:05.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.993+0000 7fe83f7da740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38d2d1390 0x55c38d6a2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:05.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.993+0000 7fe83f7da740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c38d2f2070 0x55c38d69c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:05.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.993+0000 7fe83f7da740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe814011210 con 0x55c38d2f2070 2024-08-31T06:20:05.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.993+0000 7fe83f7da740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe81400ea20 con 0x55c38d2c91b0 2024-08-31T06:20:05.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.993+0000 7fe83f7da740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe81400c890 con 0x55c38d2d1390 2024-08-31T06:20:05.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.993+0000 7fe831e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38d2c91b0 0x55c38d6a1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:05.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.993+0000 7fe831e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38d2c91b0 0x55c38d6a1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:34762/0 (socket says 172.21.15.16:34762) 2024-08-31T06:20:05.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.994+0000 7fe831e00640 1 -- 172.21.15.16:0/1229541014 learned_addr learned my addr 172.21.15.16:0/1229541014 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:05.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.994+0000 7fe832800640 1 --2- 172.21.15.16:0/1229541014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c38d2f2070 0x55c38d69c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:05.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.994+0000 7fe831400640 1 --2- 172.21.15.16:0/1229541014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38d2d1390 0x55c38d6a2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:05.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.995+0000 7fe831e00640 1 -- 172.21.15.16:0/1229541014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38d2d1390 msgr2=0x55c38d6a2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:05.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.995+0000 7fe831e00640 1 --2- 172.21.15.16:0/1229541014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38d2d1390 0x55c38d6a2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:05.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.995+0000 7fe831e00640 1 -- 172.21.15.16:0/1229541014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c38d2f2070 msgr2=0x55c38d69c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:05.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.995+0000 7fe831e00640 1 --2- 172.21.15.16:0/1229541014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c38d2f2070 0x55c38d69c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:05.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.995+0000 7fe831e00640 1 -- 172.21.15.16:0/1229541014 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c38d6a43f0 con 0x55c38d2c91b0 2024-08-31T06:20:05.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.995+0000 7fe832800640 1 --2- 172.21.15.16:0/1229541014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c38d2f2070 0x55c38d69c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:20:05.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.996+0000 7fe831e00640 1 --2- 172.21.15.16:0/1229541014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38d2c91b0 0x55c38d6a1bd0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe824012cb0 tx=0x7fe824015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=414dfac3c7ddd098 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:05.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.996+0000 7fe81e600640 1 -- 172.21.15.16:0/1229541014 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe824010e70 con 0x55c38d2c91b0 2024-08-31T06:20:05.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.996+0000 7fe81e600640 1 -- 172.21.15.16:0/1229541014 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe82401e8c0 con 0x55c38d2c91b0 2024-08-31T06:20:05.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.996+0000 7fe81e600640 1 -- 172.21.15.16:0/1229541014 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe824029960 con 0x55c38d2c91b0 2024-08-31T06:20:05.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.997+0000 7fe83f7da740 1 -- 172.21.15.16:0/1229541014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38d2c91b0 msgr2=0x55c38d6a1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:05.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.997+0000 7fe83f7da740 1 --2- 172.21.15.16:0/1229541014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38d2c91b0 0x55c38d6a1bd0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe824012cb0 tx=0x7fe824015000 comp rx=0 tx=0).stop 2024-08-31T06:20:05.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.998+0000 7fe83f7da740 1 -- 172.21.15.16:0/1229541014 shutdown_connections 2024-08-31T06:20:05.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.998+0000 7fe83f7da740 1 --2- 172.21.15.16:0/1229541014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38d2d1390 0x55c38d6a2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:05.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.998+0000 7fe83f7da740 1 --2- 172.21.15.16:0/1229541014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c38d2f2070 0x55c38d69c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:06.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.998+0000 7fe83f7da740 1 --2- 172.21.15.16:0/1229541014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38d2c91b0 0x55c38d6a1bd0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:06.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.998+0000 7fe83f7da740 1 -- 172.21.15.16:0/1229541014 >> 172.21.15.16:0/1229541014 conn(0x55c38d2e6460 msgr2=0x55c38d6af520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:06.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.998+0000 7fe83f7da740 1 -- 172.21.15.16:0/1229541014 shutdown_connections 2024-08-31T06:20:06.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.998+0000 7fe83f7da740 1 -- 172.21.15.16:0/1229541014 wait complete. 2024-08-31T06:20:06.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.999+0000 7fe83f7da740 1 -- start start 2024-08-31T06:20:06.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:05.999+0000 7fe83f7da740 1 Processor -- start 2024-08-31T06:20:06.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.000+0000 7fe83f7da740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c38d2d1390 0x55c38d6abda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:06.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.000+0000 7fe83f7da740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38d2f2070 0x55c38d6a72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:06.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.000+0000 7fe83f7da740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38d285da0 0x55c38d6a79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:06.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.000+0000 7fe83f7da740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe81400cee0 con 0x55c38d2d1390 2024-08-31T06:20:06.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.000+0000 7fe83f7da740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe81400e350 con 0x55c38d285da0 2024-08-31T06:20:06.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.000+0000 7fe83f7da740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe81400d5b0 con 0x55c38d2f2070 2024-08-31T06:20:06.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.000+0000 7fe83f7da740 10 client.0 _ll_register_callbacks cb 0x7fe82c009280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:20:06.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.001+0000 7fe831400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38d2f2070 0x55c38d6a72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:06.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.001+0000 7fe831e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c38d2d1390 0x55c38d6abda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:06.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.001+0000 7fe831400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38d2f2070 0x55c38d6a72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:51284/0 (socket says 172.21.15.16:51284) 2024-08-31T06:20:06.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.001+0000 7fe831400640 1 -- 172.21.15.16:0/3225189572 learned_addr learned my addr 172.21.15.16:0/3225189572 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:06.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.001+0000 7fe832800640 1 --2- 172.21.15.16:0/3225189572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38d285da0 0x55c38d6a79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:06.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.001+0000 7fe831400640 1 -- 172.21.15.16:0/3225189572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38d285da0 msgr2=0x55c38d6a79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:06.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.001+0000 7fe831400640 1 --2- 172.21.15.16:0/3225189572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38d285da0 0x55c38d6a79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:06.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.001+0000 7fe831400640 1 -- 172.21.15.16:0/3225189572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c38d2d1390 msgr2=0x55c38d6abda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:06.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.001+0000 7fe831400640 1 --2- 172.21.15.16:0/3225189572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c38d2d1390 0x55c38d6abda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:06.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.001+0000 7fe831400640 1 -- 172.21.15.16:0/3225189572 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c38d71edb0 con 0x55c38d2f2070 2024-08-31T06:20:06.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.001+0000 7fe832800640 1 --2- 172.21.15.16:0/3225189572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38d285da0 0x55c38d6a79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:20:06.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.001+0000 7fe831400640 1 --2- 172.21.15.16:0/3225189572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38d2f2070 0x55c38d6a72d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe8200c7da0 tx=0x7fe820031f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d37c7a7854aa79c1 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:06.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.001+0000 7fe813400640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:20:06.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.002+0000 7fe813400640 1 -- 172.21.15.16:0/3225189572 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8200d0590 con 0x55c38d2f2070 2024-08-31T06:20:06.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.002+0000 7fe83f7da740 1 -- 172.21.15.16:0/3225189572 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe82401e8c0 con 0x55c38d2f2070 2024-08-31T06:20:06.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.002+0000 7fe813400640 1 -- 172.21.15.16:0/3225189572 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8514+0+0 (secure 0 0 0) 0x7fe820103000 con 0x55c38d2f2070 2024-08-31T06:20:06.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.003+0000 7fe813400640 1 -- 172.21.15.16:0/3225189572 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fe8200d5040 con 0x55c38d2f2070 2024-08-31T06:20:06.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.003+0000 7fe83f7da740 10 client.28480 fetch_fsmap learned FSMap version 49 2024-08-31T06:20:06.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.003+0000 7fe83f7da740 1 -- 172.21.15.16:0/3225189572 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55c38d71fff0 con 0x55c38d2f2070 2024-08-31T06:20:06.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.004+0000 7fe813400640 1 -- 172.21.15.16:0/3225189572 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8200d0200 con 0x55c38d2f2070 2024-08-31T06:20:06.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.004+0000 7fe813400640 1 -- 172.21.15.16:0/3225189572 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fe82004c060 con 0x55c38d2f2070 2024-08-31T06:20:06.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.004+0000 7fe83f7da740 10 client.28480 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:20:06.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.004+0000 7fe83f7da740 10 client.28480 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T06:20:06.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.004+0000 7fe83f7da740 20 client.28480 populate_metadata read hostname 'smithi016' 2024-08-31T06:20:06.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.004+0000 7fe83f7da740 1 --2- 172.21.15.16:0/3225189572 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55c38d720720 0x55c38d722b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:06.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.004+0000 7fe83f7da740 4 client.28480 mds_command: new command op to 24712 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:20:06.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.004+0000 7fe83f7da740 1 -- 172.21.15.16:0/3225189572 --> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fe82401e8c0 con 0x55c38d720720 2024-08-31T06:20:06.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.005+0000 7fe831e00640 1 --2- 172.21.15.16:0/3225189572 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55c38d720720 0x55c38d722b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:06.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.006+0000 7fe831e00640 1 --2- 172.21.15.16:0/3225189572 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55c38d720720 0x55c38d722b40 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3565a76c740fa4ef server_cookie=5d8f452ddbd1efdd in_seq=0 out_seq=0 2024-08-31T06:20:06.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.006+0000 7fe813400640 10 client.28480 ms_handle_connect on v2:172.21.15.184:6838/4011051479 2024-08-31T06:20:06.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.006+0000 7fe813400640 1 -- 172.21.15.16:0/3225189572 <== mds.2 v2:172.21.15.184:6838/4011051479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fe82401e8c0 con 0x55c38d720720 2024-08-31T06:20:06.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.006+0000 7fe813400640 10 client.28480 handle_command_reply: tid=0 2024-08-31T06:20:06.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.006+0000 7fe83f7da740 1 client.28480 shutdown 2024-08-31T06:20:06.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.006+0000 7fe83f7da740 1 -- 172.21.15.16:0/3225189572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38d2f2070 msgr2=0x55c38d6a72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:06.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.006+0000 7fe83f7da740 1 --2- 172.21.15.16:0/3225189572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38d2f2070 0x55c38d6a72d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe8200c7da0 tx=0x7fe820031f90 comp rx=0 tx=0).stop 2024-08-31T06:20:06.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.006+0000 7fe83f7da740 1 -- 172.21.15.16:0/3225189572 shutdown_connections 2024-08-31T06:20:06.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.006+0000 7fe83f7da740 1 --2- 172.21.15.16:0/3225189572 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55c38d720720 0x55c38d722b40 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:06.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.006+0000 7fe813400640 0 client.28480 ms_handle_reset on v2:172.21.15.184:6838/4011051479 2024-08-31T06:20:06.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.006+0000 7fe83f7da740 1 --2- 172.21.15.16:0/3225189572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38d2f2070 0x55c38d6a72d0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:06.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.006+0000 7fe83f7da740 1 --2- 172.21.15.16:0/3225189572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38d285da0 0x55c38d6a79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:06.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.006+0000 7fe83f7da740 1 --2- 172.21.15.16:0/3225189572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c38d2d1390 0x55c38d6abda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:06.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.006+0000 7fe83f7da740 1 -- 172.21.15.16:0/3225189572 >> 172.21.15.16:0/3225189572 conn(0x55c38d2e6460 msgr2=0x55c38d6af520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:06.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.007+0000 7fe83f7da740 1 -- 172.21.15.16:0/3225189572 shutdown_connections 2024-08-31T06:20:06.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.007+0000 7fe83f7da740 1 -- 172.21.15.16:0/3225189572 wait complete. 2024-08-31T06:20:06.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.007+0000 7fe83f7da740 20 client.28480 trim_cache size 0 max 16384 2024-08-31T06:20:06.010 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:20:06.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.009+0000 7fe838c00640 1 -- 172.21.15.16:0/4120532028 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe8180b4d20 msgr2=0x7fe8180b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:06.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.009+0000 7fe838c00640 1 --2- 172.21.15.16:0/4120532028 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe8180b4d20 0x7fe8180b7230 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x55c38d696530 tx=0x7fe82400a380 comp rx=0 tx=0).stop 2024-08-31T06:20:06.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.009+0000 7fe838c00640 1 -- 172.21.15.16:0/4120532028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe834078030 msgr2=0x7fe83406de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:06.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.009+0000 7fe838c00640 1 --2- 172.21.15.16:0/4120532028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe834078030 0x7fe83406de70 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe820009230 tx=0x7fe820003c10 comp rx=0 tx=0).stop 2024-08-31T06:20:06.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.009+0000 7fe838c00640 1 -- 172.21.15.16:0/4120532028 shutdown_connections 2024-08-31T06:20:06.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.009+0000 7fe838c00640 1 --2- 172.21.15.16:0/4120532028 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe8180b4d20 0x7fe8180b7230 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:06.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.009+0000 7fe838c00640 1 --2- 172.21.15.16:0/4120532028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe834114510 0x7fe83406e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:06.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.009+0000 7fe838c00640 1 --2- 172.21.15.16:0/4120532028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe834078030 0x7fe83406de70 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:06.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.009+0000 7fe838c00640 1 --2- 172.21.15.16:0/4120532028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8340776e0 0x7fe834074ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:06.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.009+0000 7fe838c00640 1 -- 172.21.15.16:0/4120532028 >> 172.21.15.16:0/4120532028 conn(0x7fe834101560 msgr2=0x7fe834103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:06.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.010+0000 7fe838c00640 1 -- 172.21.15.16:0/4120532028 shutdown_connections 2024-08-31T06:20:06.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:06.010+0000 7fe838c00640 1 -- 172.21.15.16:0/4120532028 wait complete. 2024-08-31T06:20:06.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:05 smithi184 ceph-mon[22331]: pgmap v765: 97 pgs: 20 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 69 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 19 MiB/s wr, 1.74k op/s 2024-08-31T06:20:06.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:05 smithi155 ceph-mon[21742]: pgmap v765: 97 pgs: 20 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 69 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 19 MiB/s wr, 1.74k op/s 2024-08-31T06:20:06.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 760.0 s, 1008.8 tps, lat 107.193 ms stddev 191.896 2024-08-31T06:20:07.144 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T06:20:07.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.250+0000 7f8247c00640 1 -- 172.21.15.16:0/2219494275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f824010b390 msgr2=0x7f824010b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:07.252 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.250+0000 7f8247c00640 1 --2- 172.21.15.16:0/2219494275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f824010b390 0x7f824010b7a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f822c009900 tx=0x7f822c02a3f0 comp rx=0 tx=0).stop 2024-08-31T06:20:07.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.251+0000 7f8247c00640 1 -- 172.21.15.16:0/2219494275 shutdown_connections 2024-08-31T06:20:07.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.251+0000 7f8247c00640 1 --2- 172.21.15.16:0/2219494275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f824010bce0 0x7f8240112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:07.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.251+0000 7f8247c00640 1 --2- 172.21.15.16:0/2219494275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f824010b390 0x7f824010b7a0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:07.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.251+0000 7f8247c00640 1 --2- 172.21.15.16:0/2219494275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8240105a60 0x7f8240105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:07.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.251+0000 7f8247c00640 1 -- 172.21.15.16:0/2219494275 >> 172.21.15.16:0/2219494275 conn(0x7f8240101480 msgr2=0x7f82401038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:07.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.251+0000 7f8247c00640 1 -- 172.21.15.16:0/2219494275 shutdown_connections 2024-08-31T06:20:07.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.252+0000 7f8247c00640 1 -- 172.21.15.16:0/2219494275 wait complete. 2024-08-31T06:20:07.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.253+0000 7f8247c00640 1 Processor -- start 2024-08-31T06:20:07.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.253+0000 7f8247c00640 1 -- start start 2024-08-31T06:20:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.253+0000 7f8247c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8240105a60 0x7f824019c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.254+0000 7f8247c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f824010b390 0x7f824019c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.254+0000 7f8247c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f824010bce0 0x7f82401a1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.254+0000 7f8247c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f82401159a0 con 0x7f8240105a60 2024-08-31T06:20:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.254+0000 7f8247c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8240115820 con 0x7f824010bce0 2024-08-31T06:20:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.254+0000 7f8247c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8240115b20 con 0x7f824010b390 2024-08-31T06:20:07.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.254+0000 7f8245200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8240105a60 0x7f824019c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:07.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.254+0000 7f8245c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f824010bce0 0x7f82401a1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:07.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.254+0000 7f823fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f824010b390 0x7f824019c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:07.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.254+0000 7f8245c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f824010bce0 0x7f82401a1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:34788/0 (socket says 172.21.15.16:34788) 2024-08-31T06:20:07.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.254+0000 7f823fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f824010b390 0x7f824019c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:51308/0 (socket says 172.21.15.16:51308) 2024-08-31T06:20:07.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.254+0000 7f8245c00640 1 -- 172.21.15.16:0/3182934611 learned_addr learned my addr 172.21.15.16:0/3182934611 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:07.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.254+0000 7f8245c00640 1 -- 172.21.15.16:0/3182934611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f824010b390 msgr2=0x7f824019c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:07.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.254+0000 7f8245c00640 1 --2- 172.21.15.16:0/3182934611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f824010b390 0x7f824019c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:07.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.254+0000 7f8245c00640 1 -- 172.21.15.16:0/3182934611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8240105a60 msgr2=0x7f824019c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:07.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.254+0000 7f8245c00640 1 --2- 172.21.15.16:0/3182934611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8240105a60 0x7f824019c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:07.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.255+0000 7f8245c00640 1 -- 172.21.15.16:0/3182934611 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82401a22f0 con 0x7f824010bce0 2024-08-31T06:20:07.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.255+0000 7f823fe00640 1 --2- 172.21.15.16:0/3182934611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f824010b390 0x7f824019c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:20:07.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.255+0000 7f8245c00640 1 --2- 172.21.15.16:0/3182934611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f824010bce0 0x7f82401a1bf0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f823800d950 tx=0x7f823800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:07.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.255+0000 7f823d600640 1 -- 172.21.15.16:0/3182934611 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8238017030 con 0x7f824010bce0 2024-08-31T06:20:07.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.255+0000 7f823d600640 1 -- 172.21.15.16:0/3182934611 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8238004020 con 0x7f824010bce0 2024-08-31T06:20:07.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.255+0000 7f8247c00640 1 -- 172.21.15.16:0/3182934611 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82401a25e0 con 0x7f824010bce0 2024-08-31T06:20:07.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.255+0000 7f823d600640 1 -- 172.21.15.16:0/3182934611 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8238003b50 con 0x7f824010bce0 2024-08-31T06:20:07.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.255+0000 7f8247c00640 1 -- 172.21.15.16:0/3182934611 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82401a8d70 con 0x7f824010bce0 2024-08-31T06:20:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.259+0000 7f824edf9740 1 Processor -- start 2024-08-31T06:20:07.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.260+0000 7f823d600640 1 -- 172.21.15.16:0/3182934611 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f823801a070 con 0x7f824010bce0 2024-08-31T06:20:07.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.261+0000 7f824edf9740 1 -- start start 2024-08-31T06:20:07.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.261+0000 7f824edf9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb705751b0 0x55fb70918920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.261+0000 7f824edf9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb7054c2f0 0x55fb70918e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.261+0000 7f824edf9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb705544d0 0x55fb709193a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.261+0000 7f824edf9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fb7088df70 con 0x55fb705544d0 2024-08-31T06:20:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.261+0000 7f824edf9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55fb7087d500 con 0x55fb7054c2f0 2024-08-31T06:20:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.261+0000 7f824edf9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55fb708814b0 con 0x55fb705751b0 2024-08-31T06:20:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.261+0000 7f8245200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb7054c2f0 0x55fb70918e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.261+0000 7f823fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb705751b0 0x55fb70918920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.262+0000 7f8245200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb7054c2f0 0x55fb70918e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:34792/0 (socket says 172.21.15.16:34792) 2024-08-31T06:20:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.262+0000 7f8245200640 1 -- 172.21.15.16:0/859380351 learned_addr learned my addr 172.21.15.16:0/859380351 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.262+0000 7f823fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb705544d0 0x55fb709193a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.262+0000 7f823fe00640 1 -- 172.21.15.16:0/859380351 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb705751b0 msgr2=0x55fb70918920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.262+0000 7f823fe00640 1 --2- 172.21.15.16:0/859380351 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb705751b0 0x55fb70918920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.262+0000 7f823fe00640 1 -- 172.21.15.16:0/859380351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb7054c2f0 msgr2=0x55fb70918e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.262+0000 7f823fe00640 1 --2- 172.21.15.16:0/859380351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb7054c2f0 0x55fb70918e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.262+0000 7f823fe00640 1 -- 172.21.15.16:0/859380351 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb70837c10 con 0x55fb705544d0 2024-08-31T06:20:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.263+0000 7f823fe00640 1 --2- 172.21.15.16:0/859380351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb705544d0 0x55fb709193a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f822c02a910 tx=0x7f822c031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c01122864bf243 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.263+0000 7f823d600640 1 --2- 172.21.15.16:0/3182934611 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f82280b4d20 0x7f82280b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.263+0000 7f823d600640 1 -- 172.21.15.16:0/3182934611 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8514+0+0 (secure 0 0 0) 0x7f82380e4f30 con 0x7f824010bce0 2024-08-31T06:20:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.263+0000 7f8232000640 1 -- 172.21.15.16:0/859380351 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822c035050 con 0x55fb705544d0 2024-08-31T06:20:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.263+0000 7f8232000640 1 -- 172.21.15.16:0/859380351 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f822c005020 con 0x55fb705544d0 2024-08-31T06:20:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.263+0000 7f8232000640 1 -- 172.21.15.16:0/859380351 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822c004ad0 con 0x55fb705544d0 2024-08-31T06:20:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.263+0000 7f8245200640 1 --2- 172.21.15.16:0/3182934611 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f82280b4d20 0x7f82280b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.264+0000 7f824edf9740 1 -- 172.21.15.16:0/859380351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb705544d0 msgr2=0x55fb709193a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.264+0000 7f824edf9740 1 --2- 172.21.15.16:0/859380351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb705544d0 0x55fb709193a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f822c02a910 tx=0x7f822c031400 comp rx=0 tx=0).stop 2024-08-31T06:20:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.264+0000 7f8245200640 1 --2- 172.21.15.16:0/3182934611 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f82280b4d20 0x7f82280b7230 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x55fb709175a0 tx=0x7f8234009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.264+0000 7f824edf9740 1 -- 172.21.15.16:0/859380351 shutdown_connections 2024-08-31T06:20:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.264+0000 7f824edf9740 1 --2- 172.21.15.16:0/859380351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb705544d0 0x55fb709193a0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.264+0000 7f824edf9740 1 --2- 172.21.15.16:0/859380351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb7054c2f0 0x55fb70918e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.264+0000 7f824edf9740 1 --2- 172.21.15.16:0/859380351 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb705751b0 0x55fb70918920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.264+0000 7f824edf9740 1 -- 172.21.15.16:0/859380351 >> 172.21.15.16:0/859380351 conn(0x55fb70569300 msgr2=0x55fb70916810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.265+0000 7f824edf9740 1 -- 172.21.15.16:0/859380351 shutdown_connections 2024-08-31T06:20:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.265+0000 7f824edf9740 1 -- 172.21.15.16:0/859380351 wait complete. 2024-08-31T06:20:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.266+0000 7f824edf9740 1 -- start start 2024-08-31T06:20:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.267+0000 7f824edf9740 1 Processor -- start 2024-08-31T06:20:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.268+0000 7f824edf9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb7054c2f0 0x55fb70923f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.268+0000 7f824edf9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb705544d0 0x55fb70924450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.268+0000 7f824edf9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb705751b0 0x55fb70924990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.268+0000 7f824edf9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fb707b6ab0 con 0x55fb705751b0 2024-08-31T06:20:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.268+0000 7f824edf9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55fb70824320 con 0x55fb705544d0 2024-08-31T06:20:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.268+0000 7f824edf9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55fb70927040 con 0x55fb7054c2f0 2024-08-31T06:20:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.268+0000 7f824edf9740 10 client.0 _ll_register_callbacks cb 0x55fb7082a6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:20:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.269+0000 7f8245200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb7054c2f0 0x55fb70923f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.269+0000 7f823fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb705544d0 0x55fb70924450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.269+0000 7f8245200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb7054c2f0 0x55fb70923f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:51332/0 (socket says 172.21.15.16:51332) 2024-08-31T06:20:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.269+0000 7f8245200640 1 -- 172.21.15.16:0/2440487932 learned_addr learned my addr 172.21.15.16:0/2440487932 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.269+0000 7f8245c00640 1 --2- 172.21.15.16:0/2440487932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb705751b0 0x55fb70924990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.269+0000 7f8245200640 1 -- 172.21.15.16:0/2440487932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb705544d0 msgr2=0x55fb70924450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.269+0000 7f8245200640 1 --2- 172.21.15.16:0/2440487932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb705544d0 0x55fb70924450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.269+0000 7f8245200640 1 -- 172.21.15.16:0/2440487932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb705751b0 msgr2=0x55fb70924990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.269+0000 7f8245200640 1 --2- 172.21.15.16:0/2440487932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb705751b0 0x55fb70924990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.269+0000 7f8245200640 1 -- 172.21.15.16:0/2440487932 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb705e0790 con 0x55fb7054c2f0 2024-08-31T06:20:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.269+0000 7f8245c00640 1 --2- 172.21.15.16:0/2440487932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb705751b0 0x55fb70924990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:20:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.269+0000 7f8245200640 1 --2- 172.21.15.16:0/2440487932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb7054c2f0 0x55fb70923f10 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f823400f860 tx=0x7f8234004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6d98ff70f92c154 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.269+0000 7f8226a00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:20:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.270+0000 7f8226a00640 1 -- 172.21.15.16:0/2440487932 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8234003b80 con 0x55fb7054c2f0 2024-08-31T06:20:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.270+0000 7f824edf9740 1 -- 172.21.15.16:0/2440487932 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f822c005020 con 0x55fb7054c2f0 2024-08-31T06:20:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.270+0000 7f8226a00640 1 -- 172.21.15.16:0/2440487932 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8514+0+0 (secure 0 0 0) 0x7f8234017030 con 0x55fb7054c2f0 2024-08-31T06:20:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.270+0000 7f8226a00640 1 -- 172.21.15.16:0/2440487932 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f8234003d20 con 0x55fb7054c2f0 2024-08-31T06:20:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.270+0000 7f824edf9740 10 client.28492 fetch_fsmap learned FSMap version 49 2024-08-31T06:20:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.270+0000 7f824edf9740 1 -- 172.21.15.16:0/2440487932 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55fb70659620 con 0x55fb7054c2f0 2024-08-31T06:20:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.270+0000 7f8226a00640 1 -- 172.21.15.16:0/2440487932 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f822c005020 con 0x55fb7054c2f0 2024-08-31T06:20:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.270+0000 7f8226a00640 1 -- 172.21.15.16:0/2440487932 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f8234004420 con 0x55fb7054c2f0 2024-08-31T06:20:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.270+0000 7f824edf9740 10 client.28492 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:20:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.270+0000 7f824edf9740 10 client.28492 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:20:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.270+0000 7f824edf9740 20 client.28492 populate_metadata read hostname 'smithi016' 2024-08-31T06:20:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.270+0000 7f824edf9740 1 --2- 172.21.15.16:0/2440487932 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fb70533d80 0x55fb7093e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.270+0000 7f824edf9740 4 client.28492 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:20:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.270+0000 7f824edf9740 1 -- 172.21.15.16:0/2440487932 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb7088fec0 con 0x55fb70533d80 2024-08-31T06:20:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.271+0000 7f823fe00640 1 --2- 172.21.15.16:0/2440487932 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fb70533d80 0x55fb7093e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.271+0000 7f823fe00640 1 --2- 172.21.15.16:0/2440487932 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fb70533d80 0x55fb7093e920 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89450a740250b9e1 server_cookie=f3940e6b2c38ac in_seq=0 out_seq=0 2024-08-31T06:20:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.271+0000 7f8226a00640 10 client.28492 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:20:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.272+0000 7f8226a00640 1 -- 172.21.15.16:0/2440487932 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fb7088fec0 con 0x55fb70533d80 2024-08-31T06:20:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.272+0000 7f8226a00640 10 client.28492 handle_command_reply: tid=0 2024-08-31T06:20:07.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.273+0000 7f824edf9740 1 client.28492 shutdown 2024-08-31T06:20:07.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.274+0000 7f824edf9740 1 -- 172.21.15.16:0/2440487932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb7054c2f0 msgr2=0x55fb70923f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:07.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.274+0000 7f824edf9740 1 --2- 172.21.15.16:0/2440487932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb7054c2f0 0x55fb70923f10 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f823400f860 tx=0x7f8234004ab0 comp rx=0 tx=0).stop 2024-08-31T06:20:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.275+0000 7f824edf9740 1 -- 172.21.15.16:0/2440487932 shutdown_connections 2024-08-31T06:20:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.275+0000 7f824edf9740 1 --2- 172.21.15.16:0/2440487932 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fb70533d80 0x55fb7093e920 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.275+0000 7f824edf9740 1 --2- 172.21.15.16:0/2440487932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb705751b0 0x55fb70924990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.275+0000 7f824edf9740 1 --2- 172.21.15.16:0/2440487932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb705544d0 0x55fb70924450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.275+0000 7f824edf9740 1 --2- 172.21.15.16:0/2440487932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb7054c2f0 0x55fb70923f10 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.275+0000 7f8226a00640 0 client.28492 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:20:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.275+0000 7f824edf9740 1 -- 172.21.15.16:0/2440487932 >> 172.21.15.16:0/2440487932 conn(0x55fb70569300 msgr2=0x55fb70919950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.275+0000 7f824edf9740 1 -- 172.21.15.16:0/2440487932 shutdown_connections 2024-08-31T06:20:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.275+0000 7f824edf9740 1 -- 172.21.15.16:0/2440487932 wait complete. 2024-08-31T06:20:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.275+0000 7f824edf9740 20 client.28492 trim_cache size 0 max 16384 2024-08-31T06:20:07.339 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.337+0000 7f824edf9740 1 Processor -- start 2024-08-31T06:20:07.339 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.337+0000 7f824edf9740 1 -- start start 2024-08-31T06:20:07.339 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.338+0000 7f824edf9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb7054c2f0 0x55fb70925df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:07.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.338+0000 7f824edf9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb705544d0 0x55fb7091a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:07.340 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.338+0000 7f824edf9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb705751b0 0x55fb709210e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.338+0000 7f8245200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb7054c2f0 0x55fb70925df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.339+0000 7f8245200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb7054c2f0 0x55fb70925df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:34818/0 (socket says 172.21.15.16:34818) 2024-08-31T06:20:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.339+0000 7f824edf9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f82401afc10 con 0x55fb705751b0 2024-08-31T06:20:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.339+0000 7f824edf9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f82401ad420 con 0x55fb7054c2f0 2024-08-31T06:20:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.339+0000 7f824edf9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f82401ab290 con 0x55fb705544d0 2024-08-31T06:20:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.339+0000 7f823fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb705544d0 0x55fb7091a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.339+0000 7f823fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb705544d0 0x55fb7091a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:51344/0 (socket says 172.21.15.16:51344) 2024-08-31T06:20:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.339+0000 7f8245c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb705751b0 0x55fb709210e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.340+0000 7f8245c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb705751b0 0x55fb709210e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50836/0 (socket says 172.21.15.16:50836) 2024-08-31T06:20:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.340+0000 7f8245c00640 1 -- 172.21.15.16:0/1943761275 learned_addr learned my addr 172.21.15.16:0/1943761275 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.340+0000 7f8245200640 1 -- 172.21.15.16:0/1943761275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb705544d0 msgr2=0x55fb7091a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.340+0000 7f8245200640 1 --2- 172.21.15.16:0/1943761275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb705544d0 0x55fb7091a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.340+0000 7f8245200640 1 -- 172.21.15.16:0/1943761275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb705751b0 msgr2=0x55fb709210e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.340+0000 7f8245200640 1 --2- 172.21.15.16:0/1943761275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb705751b0 0x55fb709210e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.340+0000 7f8245200640 1 -- 172.21.15.16:0/1943761275 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb709223f0 con 0x55fb7054c2f0 2024-08-31T06:20:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.340+0000 7f8245c00640 1 --2- 172.21.15.16:0/1943761275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb705751b0 0x55fb709210e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:20:07.343 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.341+0000 7f8245200640 1 --2- 172.21.15.16:0/1943761275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb7054c2f0 0x55fb70925df0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f8234018040 tx=0x7f8234005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f0c76853a42afb8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.343+0000 7f8231600640 1 -- 172.21.15.16:0/1943761275 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823401fa00 con 0x55fb7054c2f0 2024-08-31T06:20:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.343+0000 7f8231600640 1 -- 172.21.15.16:0/1943761275 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f823401fba0 con 0x55fb7054c2f0 2024-08-31T06:20:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.343+0000 7f8231600640 1 -- 172.21.15.16:0/1943761275 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8234004120 con 0x55fb7054c2f0 2024-08-31T06:20:07.346 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.344+0000 7f824edf9740 1 -- 172.21.15.16:0/1943761275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb7054c2f0 msgr2=0x55fb70925df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.344+0000 7f824edf9740 1 --2- 172.21.15.16:0/1943761275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb7054c2f0 0x55fb70925df0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f8234018040 tx=0x7f8234005f90 comp rx=0 tx=0).stop 2024-08-31T06:20:07.348 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.346+0000 7f824edf9740 1 -- 172.21.15.16:0/1943761275 shutdown_connections 2024-08-31T06:20:07.348 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.346+0000 7f824edf9740 1 --2- 172.21.15.16:0/1943761275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb705544d0 0x55fb7091a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:07.348 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.346+0000 7f824edf9740 1 --2- 172.21.15.16:0/1943761275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb705751b0 0x55fb709210e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.348+0000 7f824edf9740 1 --2- 172.21.15.16:0/1943761275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb7054c2f0 0x55fb70925df0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.348+0000 7f824edf9740 1 -- 172.21.15.16:0/1943761275 >> 172.21.15.16:0/1943761275 conn(0x55fb70569300 msgr2=0x55fb70919950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.349+0000 7f824edf9740 1 -- 172.21.15.16:0/1943761275 shutdown_connections 2024-08-31T06:20:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.349+0000 7f824edf9740 1 -- 172.21.15.16:0/1943761275 wait complete. 2024-08-31T06:20:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.349+0000 7f824edf9740 1 -- start start 2024-08-31T06:20:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.350+0000 7f824edf9740 1 Processor -- start 2024-08-31T06:20:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.351+0000 7f824edf9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb7054c2f0 0x55fb70933310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.351+0000 7f824edf9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb705544d0 0x55fb70933870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.351+0000 7f824edf9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb705751b0 0x55fb709a8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:07.355 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.353+0000 7f8245200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb7054c2f0 0x55fb70933310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:07.355 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.353+0000 7f8245200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb7054c2f0 0x55fb70933310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:51360/0 (socket says 172.21.15.16:51360) 2024-08-31T06:20:07.355 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.353+0000 7f8245200640 1 -- 172.21.15.16:0/4168555858 learned_addr learned my addr 172.21.15.16:0/4168555858 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:07.356 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.354+0000 7f8245c00640 1 --2- 172.21.15.16:0/4168555858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb705751b0 0x55fb709a8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:07.356 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.354+0000 7f824edf9740 1 -- 172.21.15.16:0/4168555858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f82401ab8e0 con 0x55fb705751b0 2024-08-31T06:20:07.356 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.354+0000 7f824edf9740 1 -- 172.21.15.16:0/4168555858 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f82401acd50 con 0x55fb705544d0 2024-08-31T06:20:07.357 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.354+0000 7f824edf9740 1 -- 172.21.15.16:0/4168555858 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f82401abfb0 con 0x55fb7054c2f0 2024-08-31T06:20:07.357 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.354+0000 7f824edf9740 10 client.0 _ll_register_callbacks cb 0x7f823400f6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:20:07.357 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.355+0000 7f823fe00640 1 --2- 172.21.15.16:0/4168555858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb705544d0 0x55fb70933870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:07.357 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.356+0000 7f823fe00640 1 -- 172.21.15.16:0/4168555858 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb7054c2f0 msgr2=0x55fb70933310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:07.358 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.356+0000 7f823fe00640 1 --2- 172.21.15.16:0/4168555858 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb7054c2f0 0x55fb70933310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:07.358 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.356+0000 7f823fe00640 1 -- 172.21.15.16:0/4168555858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb705751b0 msgr2=0x55fb709a8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:07.358 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.356+0000 7f823fe00640 1 --2- 172.21.15.16:0/4168555858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb705751b0 0x55fb709a8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:07.358 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.356+0000 7f823fe00640 1 -- 172.21.15.16:0/4168555858 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb709a8940 con 0x55fb705544d0 2024-08-31T06:20:07.358 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.356+0000 7f8245c00640 1 --2- 172.21.15.16:0/4168555858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb705751b0 0x55fb709a8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:20:07.359 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.357+0000 7f823fe00640 1 --2- 172.21.15.16:0/4168555858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb705544d0 0x55fb70933870 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f822c031640 tx=0x7f822c0317b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b99ed4736ea34f7a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:07.359 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.357+0000 7f8226a00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:20:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.358+0000 7f8226a00640 1 -- 172.21.15.16:0/4168555858 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822c035070 con 0x55fb705544d0 2024-08-31T06:20:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.359+0000 7f8226a00640 1 -- 172.21.15.16:0/4168555858 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8514+0+0 (secure 0 0 0) 0x7f822c050070 con 0x55fb705544d0 2024-08-31T06:20:07.361 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.359+0000 7f824edf9740 1 -- 172.21.15.16:0/4168555858 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f823401fba0 con 0x55fb705544d0 2024-08-31T06:20:07.361 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.360+0000 7f8226a00640 1 -- 172.21.15.16:0/4168555858 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f822c035070 con 0x55fb705544d0 2024-08-31T06:20:07.362 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.360+0000 7f824edf9740 10 client.28484 fetch_fsmap learned FSMap version 49 2024-08-31T06:20:07.362 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.361+0000 7f824edf9740 1 -- 172.21.15.16:0/4168555858 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55fb709a9bb0 con 0x55fb705544d0 2024-08-31T06:20:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.362+0000 7f8226a00640 1 -- 172.21.15.16:0/4168555858 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f823401fba0 con 0x55fb705544d0 2024-08-31T06:20:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.362+0000 7f8226a00640 1 -- 172.21.15.16:0/4168555858 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f822c004900 con 0x55fb705544d0 2024-08-31T06:20:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.362+0000 7f824edf9740 10 client.28484 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:20:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.362+0000 7f824edf9740 10 client.28484 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:20:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.362+0000 7f824edf9740 20 client.28484 populate_metadata read hostname 'smithi016' 2024-08-31T06:20:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.362+0000 7f824edf9740 1 --2- 172.21.15.16:0/4168555858 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fb705091c0 0x55fb709abea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.362+0000 7f824edf9740 4 client.28484 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:20:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.362+0000 7f824edf9740 1 -- 172.21.15.16:0/4168555858 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fb7088b960 con 0x55fb705091c0 2024-08-31T06:20:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.363+0000 7f8245200640 1 --2- 172.21.15.16:0/4168555858 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fb705091c0 0x55fb709abea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:07.367 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.366+0000 7f8226a00640 10 client.28484 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:20:07.367 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.366+0000 7f8245200640 1 --2- 172.21.15.16:0/4168555858 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fb705091c0 0x55fb709abea0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33323aa700892116 server_cookie=3d07cada210f3490 in_seq=0 out_seq=0 2024-08-31T06:20:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.455+0000 7f8226a00640 1 -- 172.21.15.16:0/4168555858 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fb7088b960 con 0x55fb705091c0 2024-08-31T06:20:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.455+0000 7f8226a00640 10 client.28484 handle_command_reply: tid=0 2024-08-31T06:20:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.455+0000 7f824edf9740 1 client.28484 shutdown 2024-08-31T06:20:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.456+0000 7f824edf9740 1 -- 172.21.15.16:0/4168555858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb705544d0 msgr2=0x55fb70933870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.456+0000 7f824edf9740 1 --2- 172.21.15.16:0/4168555858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb705544d0 0x55fb70933870 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f822c031640 tx=0x7f822c0317b0 comp rx=0 tx=0).stop 2024-08-31T06:20:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.456+0000 7f824edf9740 1 -- 172.21.15.16:0/4168555858 shutdown_connections 2024-08-31T06:20:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.456+0000 7f824edf9740 1 --2- 172.21.15.16:0/4168555858 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55fb705091c0 0x55fb709abea0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.456+0000 7f824edf9740 1 --2- 172.21.15.16:0/4168555858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb705751b0 0x55fb709a8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.456+0000 7f824edf9740 1 --2- 172.21.15.16:0/4168555858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb705544d0 0x55fb70933870 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.456+0000 7f824edf9740 1 --2- 172.21.15.16:0/4168555858 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fb7054c2f0 0x55fb70933310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.456+0000 7f824edf9740 1 -- 172.21.15.16:0/4168555858 >> 172.21.15.16:0/4168555858 conn(0x55fb70569300 msgr2=0x55fb70937ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.456+0000 7f8226a00640 0 client.28484 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:20:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.456+0000 7f824edf9740 1 -- 172.21.15.16:0/4168555858 shutdown_connections 2024-08-31T06:20:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.456+0000 7f824edf9740 1 -- 172.21.15.16:0/4168555858 wait complete. 2024-08-31T06:20:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.456+0000 7f824edf9740 20 client.28484 trim_cache size 0 max 16384 2024-08-31T06:20:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:20:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:20:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:20:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:20:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.460+0000 7f8247c00640 1 -- 172.21.15.16:0/3182934611 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f82280b4d20 msgr2=0x7f82280b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.460+0000 7f8247c00640 1 --2- 172.21.15.16:0/3182934611 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f82280b4d20 0x7f82280b7230 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x55fb709175a0 tx=0x7f8234009500 comp rx=0 tx=0).stop 2024-08-31T06:20:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.461+0000 7f8247c00640 1 -- 172.21.15.16:0/3182934611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f824010bce0 msgr2=0x7f82401a1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.461+0000 7f8247c00640 1 --2- 172.21.15.16:0/3182934611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f824010bce0 0x7f82401a1bf0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f823800d950 tx=0x7f823800de10 comp rx=0 tx=0).stop 2024-08-31T06:20:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.461+0000 7f8247c00640 1 -- 172.21.15.16:0/3182934611 shutdown_connections 2024-08-31T06:20:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.461+0000 7f8247c00640 1 --2- 172.21.15.16:0/3182934611 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f82280b4d20 0x7f82280b7230 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.461+0000 7f8247c00640 1 --2- 172.21.15.16:0/3182934611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f824010b390 0x7f824019c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.461+0000 7f8247c00640 1 --2- 172.21.15.16:0/3182934611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f824010bce0 0x7f82401a1bf0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.461+0000 7f8247c00640 1 --2- 172.21.15.16:0/3182934611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8240105a60 0x7f824019c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.461+0000 7f8247c00640 1 -- 172.21.15.16:0/3182934611 >> 172.21.15.16:0/3182934611 conn(0x7f8240101480 msgr2=0x7f82401038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.461+0000 7f8247c00640 1 -- 172.21.15.16:0/3182934611 shutdown_connections 2024-08-31T06:20:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:07.461+0000 7f8247c00640 1 -- 172.21.15.16:0/3182934611 wait complete. 2024-08-31T06:20:07.823 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:20:08.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.622+0000 7ff5f6000640 1 -- 172.21.15.16:0/874154615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5f0104770 msgr2=0x7ff5f0104b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:08.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.622+0000 7ff5f6000640 1 --2- 172.21.15.16:0/874154615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5f0104770 0x7ff5f0104b80 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0009900 tx=0x7ff5e002a400 comp rx=0 tx=0).stop 2024-08-31T06:20:08.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.622+0000 7ff5f6000640 1 -- 172.21.15.16:0/874154615 shutdown_connections 2024-08-31T06:20:08.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.622+0000 7ff5f6000640 1 --2- 172.21.15.16:0/874154615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5f01050c0 0x7ff5f010d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:08.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.622+0000 7ff5f6000640 1 --2- 172.21.15.16:0/874154615 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5f010dbd0 0x7ff5f01100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:08.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.622+0000 7ff5f6000640 1 --2- 172.21.15.16:0/874154615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5f0104770 0x7ff5f0104b80 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:08.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.622+0000 7ff5f6000640 1 -- 172.21.15.16:0/874154615 >> 172.21.15.16:0/874154615 conn(0x7ff5f00794e0 msgr2=0x7ff5f00798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:08.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.622+0000 7ff5f6000640 1 -- 172.21.15.16:0/874154615 shutdown_connections 2024-08-31T06:20:08.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.622+0000 7ff5f6000640 1 -- 172.21.15.16:0/874154615 wait complete. 2024-08-31T06:20:08.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.624+0000 7ff5f6000640 1 Processor -- start 2024-08-31T06:20:08.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.624+0000 7ff5f6000640 1 -- start start 2024-08-31T06:20:08.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.624+0000 7ff5f6000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5f0104770 0x7ff5f019c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:08.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.625+0000 7ff5f6000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5f01050c0 0x7ff5f019ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:08.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.625+0000 7ff5f6000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5f010dbd0 0x7ff5f01a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:08.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.625+0000 7ff5f6000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f0113830 con 0x7ff5f010dbd0 2024-08-31T06:20:08.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.625+0000 7ff5f6000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f01136b0 con 0x7ff5f0104770 2024-08-31T06:20:08.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.625+0000 7ff5f6000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f01139b0 con 0x7ff5f01050c0 2024-08-31T06:20:08.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.625+0000 7ff5ef400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5f0104770 0x7ff5f019c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:08.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.625+0000 7ff5ef400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5f0104770 0x7ff5f019c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:34854/0 (socket says 172.21.15.16:34854) 2024-08-31T06:20:08.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.625+0000 7ff5ef400640 1 -- 172.21.15.16:0/389275209 learned_addr learned my addr 172.21.15.16:0/389275209 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:08.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.625+0000 7ff5e6a00640 1 --2- 172.21.15.16:0/389275209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5f01050c0 0x7ff5f019ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:08.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.625+0000 7ff5efe00640 1 --2- 172.21.15.16:0/389275209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5f010dbd0 0x7ff5f01a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:08.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.626+0000 7ff5ef400640 1 -- 172.21.15.16:0/389275209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5f01050c0 msgr2=0x7ff5f019ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:08.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.626+0000 7ff5ef400640 1 --2- 172.21.15.16:0/389275209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5f01050c0 0x7ff5f019ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:08.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.626+0000 7ff5ef400640 1 -- 172.21.15.16:0/389275209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5f010dbd0 msgr2=0x7ff5f01a1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:08.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.626+0000 7ff5ef400640 1 --2- 172.21.15.16:0/389275209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5f010dbd0 0x7ff5f01a1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:08.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.626+0000 7ff5ef400640 1 -- 172.21.15.16:0/389275209 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f01a23f0 con 0x7ff5f0104770 2024-08-31T06:20:08.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.626+0000 7ff5ef400640 1 --2- 172.21.15.16:0/389275209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5f0104770 0x7ff5f019c480 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0032fd0 tx=0x7ff5e0031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:08.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.627+0000 7ff5ecc00640 1 -- 172.21.15.16:0/389275209 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e003f030 con 0x7ff5f0104770 2024-08-31T06:20:08.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.627+0000 7ff5ecc00640 1 -- 172.21.15.16:0/389275209 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5e0004490 con 0x7ff5f0104770 2024-08-31T06:20:08.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.627+0000 7ff5ecc00640 1 -- 172.21.15.16:0/389275209 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e0035070 con 0x7ff5f0104770 2024-08-31T06:20:08.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.627+0000 7ff5f6000640 1 -- 172.21.15.16:0/389275209 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5f01a2680 con 0x7ff5f0104770 2024-08-31T06:20:08.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.627+0000 7ff5f6000640 1 -- 172.21.15.16:0/389275209 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5f01a8e40 con 0x7ff5f0104770 2024-08-31T06:20:08.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.634+0000 7ff5fcc5f740 1 Processor -- start 2024-08-31T06:20:08.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.634+0000 7ff5fcc5f740 1 -- start start 2024-08-31T06:20:08.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.635+0000 7ff5fcc5f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d11fddb070 0x55d120180990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:08.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.635+0000 7ff5fcc5f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11fdb21b0 0x55d120180ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:08.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.635+0000 7ff5fcc5f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11fdba390 0x55d120181410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:08.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.635+0000 7ff5fcc5f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d1200f77b0 con 0x55d11fdba390 2024-08-31T06:20:08.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.635+0000 7ff5fcc5f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d1200e6d40 con 0x55d11fddb070 2024-08-31T06:20:08.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.635+0000 7ff5fcc5f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d1200eacf0 con 0x55d11fdb21b0 2024-08-31T06:20:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.635+0000 7ff5ecc00640 1 -- 172.21.15.16:0/389275209 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7ff5e0042030 con 0x7ff5f0104770 2024-08-31T06:20:08.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.636+0000 7ff5ef400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11fdb21b0 0x55d120180ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:08.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.636+0000 7ff5e6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11fdba390 0x55d120181410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:08.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.636+0000 7ff5ef400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11fdb21b0 0x55d120180ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:51402/0 (socket says 172.21.15.16:51402) 2024-08-31T06:20:08.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.636+0000 7ff5ef400640 1 -- 172.21.15.16:0/3613302693 learned_addr learned my addr 172.21.15.16:0/3613302693 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:08.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.636+0000 7ff5ef400640 1 -- 172.21.15.16:0/3613302693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d11fddb070 msgr2=0x55d120180990 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T06:20:08.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.636+0000 7ff5ef400640 1 --2- 172.21.15.16:0/3613302693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d11fddb070 0x55d120180990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:08.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.636+0000 7ff5ef400640 1 -- 172.21.15.16:0/3613302693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11fdba390 msgr2=0x55d120181410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:08.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.636+0000 7ff5ef400640 1 --2- 172.21.15.16:0/3613302693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11fdba390 0x55d120181410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:08.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.636+0000 7ff5ef400640 1 -- 172.21.15.16:0/3613302693 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d11fe46650 con 0x55d11fdb21b0 2024-08-31T06:20:08.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.637+0000 7ff5ef400640 1 --2- 172.21.15.16:0/3613302693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11fdb21b0 0x55d120180ed0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0102ed0 tx=0x7ff5e010bb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9bf733d09534889 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:08.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.638+0000 7ff5dfe00640 1 -- 172.21.15.16:0/3613302693 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e0114070 con 0x55d11fdb21b0 2024-08-31T06:20:08.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.638+0000 7ff5dfe00640 1 -- 172.21.15.16:0/3613302693 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5e0110440 con 0x55d11fdb21b0 2024-08-31T06:20:08.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.638+0000 7ff5dfe00640 1 -- 172.21.15.16:0/3613302693 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e010f600 con 0x55d11fdb21b0 2024-08-31T06:20:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.639+0000 7ff5ecc00640 1 --2- 172.21.15.16:0/389275209 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff5d40b4d20 0x7ff5d40b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.639+0000 7ff5fcc5f740 1 -- 172.21.15.16:0/3613302693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11fdb21b0 msgr2=0x55d120180ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.639+0000 7ff5fcc5f740 1 --2- 172.21.15.16:0/3613302693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11fdb21b0 0x55d120180ed0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0102ed0 tx=0x7ff5e010bb40 comp rx=0 tx=0).stop 2024-08-31T06:20:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.639+0000 7ff5ecc00640 1 -- 172.21.15.16:0/389275209 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8514+0+0 (secure 0 0 0) 0x7ff5e0105810 con 0x7ff5f0104770 2024-08-31T06:20:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.639+0000 7ff5efe00640 1 --2- 172.21.15.16:0/389275209 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff5d40b4d20 0x7ff5d40b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.639+0000 7ff5fcc5f740 1 -- 172.21.15.16:0/3613302693 shutdown_connections 2024-08-31T06:20:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.639+0000 7ff5fcc5f740 1 --2- 172.21.15.16:0/3613302693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11fdb21b0 0x55d120180ed0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.639+0000 7ff5fcc5f740 1 --2- 172.21.15.16:0/3613302693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11fdba390 0x55d120181410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.639+0000 7ff5fcc5f740 1 --2- 172.21.15.16:0/3613302693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d11fddb070 0x55d120180990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.639+0000 7ff5fcc5f740 1 -- 172.21.15.16:0/3613302693 >> 172.21.15.16:0/3613302693 conn(0x55d11fdcf460 msgr2=0x55d12017e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.640+0000 7ff5fcc5f740 1 -- 172.21.15.16:0/3613302693 shutdown_connections 2024-08-31T06:20:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.640+0000 7ff5fcc5f740 1 -- 172.21.15.16:0/3613302693 wait complete. 2024-08-31T06:20:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.640+0000 7ff5efe00640 1 --2- 172.21.15.16:0/389275209 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff5d40b4d20 0x7ff5d40b7230 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8005fd0 tx=0x7ff5e8009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.641+0000 7ff5fcc5f740 1 -- start start 2024-08-31T06:20:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.641+0000 7ff5fcc5f740 1 Processor -- start 2024-08-31T06:20:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.642+0000 7ff5fcc5f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11fdb21b0 0x55d12018c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.642+0000 7ff5fcc5f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d11fdba390 0x55d12018ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.642+0000 7ff5fcc5f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11fddb070 0x55d1201981e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.642+0000 7ff5fcc5f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d11ff49700 con 0x55d11fdb21b0 2024-08-31T06:20:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.642+0000 7ff5fcc5f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d12006dcf0 con 0x55d11fdba390 2024-08-31T06:20:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.642+0000 7ff5fcc5f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d12018ef30 con 0x55d11fddb070 2024-08-31T06:20:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.642+0000 7ff5fcc5f740 10 client.0 _ll_register_callbacks cb 0x55d120093470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:20:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.642+0000 7ff5ef400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11fdb21b0 0x55d12018c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.642+0000 7ff5e6a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d11fdba390 0x55d12018ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.642+0000 7ff5ef400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11fdb21b0 0x55d12018c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50884/0 (socket says 172.21.15.16:50884) 2024-08-31T06:20:08.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.642+0000 7ff5ef400640 1 -- 172.21.15.16:0/474137949 learned_addr learned my addr 172.21.15.16:0/474137949 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:08.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.642+0000 7ff5efe00640 1 --2- 172.21.15.16:0/474137949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11fddb070 0x55d1201981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:08.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.643+0000 7ff5efe00640 1 -- 172.21.15.16:0/474137949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d11fdba390 msgr2=0x55d12018ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:08.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.643+0000 7ff5efe00640 1 --2- 172.21.15.16:0/474137949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d11fdba390 0x55d12018ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:08.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.643+0000 7ff5efe00640 1 -- 172.21.15.16:0/474137949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11fdb21b0 msgr2=0x55d12018c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:08.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.643+0000 7ff5efe00640 1 --2- 172.21.15.16:0/474137949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11fdb21b0 0x55d12018c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:08.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.643+0000 7ff5efe00640 1 -- 172.21.15.16:0/474137949 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1200ae8b0 con 0x55d11fddb070 2024-08-31T06:20:08.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.643+0000 7ff5ef400640 1 --2- 172.21.15.16:0/474137949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11fdb21b0 0x55d12018c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:20:08.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.643+0000 7ff5efe00640 1 --2- 172.21.15.16:0/474137949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11fddb070 0x55d1201981e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8011ca0 tx=0x7ff5e8004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad0d4803d1b39bdb server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:08.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.643+0000 7ff5dcc00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:20:08.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.644+0000 7ff5dcc00640 1 -- 172.21.15.16:0/474137949 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e800fd50 con 0x55d11fddb070 2024-08-31T06:20:08.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.644+0000 7ff5dcc00640 1 -- 172.21.15.16:0/474137949 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8514+0+0 (secure 0 0 0) 0x7ff5e801e420 con 0x55d11fddb070 2024-08-31T06:20:08.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.645+0000 7ff5fcc5f740 1 -- 172.21.15.16:0/474137949 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5e0110440 con 0x55d11fddb070 2024-08-31T06:20:08.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.645+0000 7ff5dcc00640 1 -- 172.21.15.16:0/474137949 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7ff5e801e620 con 0x55d11fddb070 2024-08-31T06:20:08.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.645+0000 7ff5fcc5f740 10 client.28510 fetch_fsmap learned FSMap version 49 2024-08-31T06:20:08.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.645+0000 7ff5fcc5f740 1 -- 172.21.15.16:0/474137949 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55d12019af30 con 0x55d11fddb070 2024-08-31T06:20:08.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.646+0000 7ff5dcc00640 1 -- 172.21.15.16:0/474137949 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5e0110440 con 0x55d11fddb070 2024-08-31T06:20:08.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.646+0000 7ff5dcc00640 1 -- 172.21.15.16:0/474137949 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7ff5e802f7d0 con 0x55d11fddb070 2024-08-31T06:20:08.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.646+0000 7ff5fcc5f740 10 client.28510 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:20:08.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.646+0000 7ff5fcc5f740 10 client.28510 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-31T06:20:08.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.646+0000 7ff5fcc5f740 20 client.28510 populate_metadata read hostname 'smithi016' 2024-08-31T06:20:08.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.646+0000 7ff5fcc5f740 1 --2- 172.21.15.16:0/474137949 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55d11fd999f0 0x55d12019e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:08.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.646+0000 7ff5fcc5f740 4 client.28510 mds_command: new command op to 24724 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:20:08.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.646+0000 7ff5fcc5f740 1 -- 172.21.15.16:0/474137949 --> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1200f9700 con 0x55d11fd999f0 2024-08-31T06:20:08.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.647+0000 7ff5ef400640 1 --2- 172.21.15.16:0/474137949 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55d11fd999f0 0x55d12019e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:08.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.648+0000 7ff5ef400640 1 --2- 172.21.15.16:0/474137949 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55d11fd999f0 0x55d12019e420 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=127612aac1af3b05 server_cookie=38a3665afc9ce4b3 in_seq=0 out_seq=0 2024-08-31T06:20:08.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.648+0000 7ff5dcc00640 10 client.28510 ms_handle_connect on v2:172.21.15.16:6838/219664460 2024-08-31T06:20:08.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.649+0000 7ff5dcc00640 1 -- 172.21.15.16:0/474137949 <== mds.1 v2:172.21.15.16:6838/219664460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d1200f9700 con 0x55d11fd999f0 2024-08-31T06:20:08.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.649+0000 7ff5dcc00640 10 client.28510 handle_command_reply: tid=0 2024-08-31T06:20:08.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.649+0000 7ff5fcc5f740 1 client.28510 shutdown 2024-08-31T06:20:08.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.649+0000 7ff5fcc5f740 1 -- 172.21.15.16:0/474137949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11fddb070 msgr2=0x55d1201981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:08.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.649+0000 7ff5fcc5f740 1 --2- 172.21.15.16:0/474137949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11fddb070 0x55d1201981e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8011ca0 tx=0x7ff5e8004ab0 comp rx=0 tx=0).stop 2024-08-31T06:20:08.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.649+0000 7ff5fcc5f740 1 -- 172.21.15.16:0/474137949 shutdown_connections 2024-08-31T06:20:08.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.649+0000 7ff5fcc5f740 1 --2- 172.21.15.16:0/474137949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11fddb070 0x55d1201981e0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:08.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.649+0000 7ff5fcc5f740 1 --2- 172.21.15.16:0/474137949 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55d11fd999f0 0x55d12019e420 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:08.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.650+0000 7ff5fcc5f740 1 --2- 172.21.15.16:0/474137949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d11fdba390 0x55d12018ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:08.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.650+0000 7ff5fcc5f740 1 --2- 172.21.15.16:0/474137949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11fdb21b0 0x55d12018c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:08.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.650+0000 7ff5dcc00640 0 client.28510 ms_handle_reset on v2:172.21.15.16:6838/219664460 2024-08-31T06:20:08.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.650+0000 7ff5fcc5f740 1 -- 172.21.15.16:0/474137949 >> 172.21.15.16:0/474137949 conn(0x55d11fdcf460 msgr2=0x55d120183b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:08.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.650+0000 7ff5fcc5f740 1 -- 172.21.15.16:0/474137949 shutdown_connections 2024-08-31T06:20:08.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.650+0000 7ff5fcc5f740 1 -- 172.21.15.16:0/474137949 wait complete. 2024-08-31T06:20:08.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.650+0000 7ff5fcc5f740 20 client.28510 trim_cache size 0 max 16384 2024-08-31T06:20:08.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.670+0000 7ff5fcc5f740 1 Processor -- start 2024-08-31T06:20:08.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.670+0000 7ff5fcc5f740 1 -- start start 2024-08-31T06:20:08.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.671+0000 7ff5fcc5f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11fdb21b0 0x55d120198120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:08.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.671+0000 7ff5fcc5f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d11fdba390 0x55d120198660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:08.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.671+0000 7ff5fcc5f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11fddb070 0x55d120189510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:08.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.671+0000 7ff5fcc5f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f01b41a0 con 0x55d11fdb21b0 2024-08-31T06:20:08.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.671+0000 7ff5fcc5f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f01b19b0 con 0x55d11fdba390 2024-08-31T06:20:08.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.671+0000 7ff5fcc5f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f01af820 con 0x55d11fddb070 2024-08-31T06:20:08.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.671+0000 7ff5ef400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11fdb21b0 0x55d120198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:08.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.671+0000 7ff5e6a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d11fdba390 0x55d120198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:08.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.671+0000 7ff5ef400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11fdb21b0 0x55d120198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50888/0 (socket says 172.21.15.16:50888) 2024-08-31T06:20:08.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.671+0000 7ff5ef400640 1 -- 172.21.15.16:0/2593787848 learned_addr learned my addr 172.21.15.16:0/2593787848 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:08.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.671+0000 7ff5efe00640 1 --2- 172.21.15.16:0/2593787848 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11fddb070 0x55d120189510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:08.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.671+0000 7ff5e6a00640 1 -- 172.21.15.16:0/2593787848 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11fddb070 msgr2=0x55d120189510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:08.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.671+0000 7ff5e6a00640 1 --2- 172.21.15.16:0/2593787848 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11fddb070 0x55d120189510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:08.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.671+0000 7ff5e6a00640 1 -- 172.21.15.16:0/2593787848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11fdb21b0 msgr2=0x55d120198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:08.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.671+0000 7ff5e6a00640 1 --2- 172.21.15.16:0/2593787848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11fdb21b0 0x55d120198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:08.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.671+0000 7ff5e6a00640 1 -- 172.21.15.16:0/2593787848 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1201837f0 con 0x55d11fdba390 2024-08-31T06:20:08.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.671+0000 7ff5ef400640 1 --2- 172.21.15.16:0/2593787848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11fdb21b0 0x55d120198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:20:08.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.672+0000 7ff5e6a00640 1 --2- 172.21.15.16:0/2593787848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d11fdba390 0x55d120198660 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff5f006b6f0 tx=0x7ff5d8005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8c94166e6a86e1a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:08.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.672+0000 7ff5df400640 1 -- 172.21.15.16:0/2593787848 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d8009dc0 con 0x55d11fdba390 2024-08-31T06:20:08.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.672+0000 7ff5df400640 1 -- 172.21.15.16:0/2593787848 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5d80054d0 con 0x55d11fdba390 2024-08-31T06:20:08.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.672+0000 7ff5df400640 1 -- 172.21.15.16:0/2593787848 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d8004ca0 con 0x55d11fdba390 2024-08-31T06:20:08.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.673+0000 7ff5fcc5f740 1 -- 172.21.15.16:0/2593787848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d11fdba390 msgr2=0x55d120198660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:08.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.673+0000 7ff5fcc5f740 1 --2- 172.21.15.16:0/2593787848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d11fdba390 0x55d120198660 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff5f006b6f0 tx=0x7ff5d8005f90 comp rx=0 tx=0).stop 2024-08-31T06:20:08.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.673+0000 7ff5fcc5f740 1 -- 172.21.15.16:0/2593787848 shutdown_connections 2024-08-31T06:20:08.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.673+0000 7ff5fcc5f740 1 --2- 172.21.15.16:0/2593787848 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11fddb070 0x55d120189510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:08.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.673+0000 7ff5fcc5f740 1 --2- 172.21.15.16:0/2593787848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d11fdba390 0x55d120198660 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:08.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.673+0000 7ff5fcc5f740 1 --2- 172.21.15.16:0/2593787848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11fdb21b0 0x55d120198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:08.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.674+0000 7ff5fcc5f740 1 -- 172.21.15.16:0/2593787848 >> 172.21.15.16:0/2593787848 conn(0x55d11fdcf460 msgr2=0x55d12018c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:08.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.674+0000 7ff5fcc5f740 1 -- 172.21.15.16:0/2593787848 shutdown_connections 2024-08-31T06:20:08.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.674+0000 7ff5fcc5f740 1 -- 172.21.15.16:0/2593787848 wait complete. 2024-08-31T06:20:08.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.674+0000 7ff5fcc5f740 1 -- start start 2024-08-31T06:20:08.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.675+0000 7ff5fcc5f740 1 Processor -- start 2024-08-31T06:20:08.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.676+0000 7ff5fcc5f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11fdb21b0 0x55d120190100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:08.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.676+0000 7ff5fcc5f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11fdba390 0x55d120190640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:08.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.676+0000 7ff5fcc5f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d11fddb070 0x55d120190b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:08.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.676+0000 7ff5fcc5f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f01afe70 con 0x55d11fdb21b0 2024-08-31T06:20:08.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.676+0000 7ff5fcc5f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f01b12e0 con 0x55d11fddb070 2024-08-31T06:20:08.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.676+0000 7ff5fcc5f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f01b0540 con 0x55d11fdba390 2024-08-31T06:20:08.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.676+0000 7ff5fcc5f740 10 client.0 _ll_register_callbacks cb 0x7ff5e8003e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:20:08.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.676+0000 7ff5e6a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11fdba390 0x55d120190640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:08.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.676+0000 7ff5e6a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11fdba390 0x55d120190640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:51432/0 (socket says 172.21.15.16:51432) 2024-08-31T06:20:08.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.677+0000 7ff5e6a00640 1 -- 172.21.15.16:0/314995091 learned_addr learned my addr 172.21.15.16:0/314995091 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:08.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.677+0000 7ff5efe00640 1 --2- 172.21.15.16:0/314995091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d11fddb070 0x55d120190b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:08.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.677+0000 7ff5e6a00640 1 -- 172.21.15.16:0/314995091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d11fddb070 msgr2=0x55d120190b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:08.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.677+0000 7ff5e6a00640 1 --2- 172.21.15.16:0/314995091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d11fddb070 0x55d120190b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:08.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.677+0000 7ff5e6a00640 1 -- 172.21.15.16:0/314995091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11fdb21b0 msgr2=0x55d120190100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:08.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.677+0000 7ff5e6a00640 1 --2- 172.21.15.16:0/314995091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11fdb21b0 0x55d120190100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:08.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.677+0000 7ff5e6a00640 1 -- 172.21.15.16:0/314995091 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d120207dc0 con 0x55d11fdba390 2024-08-31T06:20:08.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.678+0000 7ff5e6a00640 1 --2- 172.21.15.16:0/314995091 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11fdba390 0x55d120190640 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8005890 tx=0x7ff5d80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bf5dabbe400d345 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:08.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.678+0000 7ff5dcc00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:20:08.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.678+0000 7ff5dcc00640 1 -- 172.21.15.16:0/314995091 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d802a070 con 0x55d11fdba390 2024-08-31T06:20:08.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.678+0000 7ff5dcc00640 1 -- 172.21.15.16:0/314995091 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8514+0+0 (secure 0 0 0) 0x7ff5d80052b0 con 0x55d11fdba390 2024-08-31T06:20:08.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.679+0000 7ff5fcc5f740 1 -- 172.21.15.16:0/314995091 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5d80054d0 con 0x55d11fdba390 2024-08-31T06:20:08.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.679+0000 7ff5dcc00640 1 -- 172.21.15.16:0/314995091 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7ff5d8004920 con 0x55d11fdba390 2024-08-31T06:20:08.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.679+0000 7ff5fcc5f740 10 client.28516 fetch_fsmap learned FSMap version 49 2024-08-31T06:20:08.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.679+0000 7ff5fcc5f740 1 -- 172.21.15.16:0/314995091 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55d120208fe0 con 0x55d11fdba390 2024-08-31T06:20:08.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.679+0000 7ff5dcc00640 1 -- 172.21.15.16:0/314995091 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5d80054d0 con 0x55d11fdba390 2024-08-31T06:20:08.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.680+0000 7ff5dcc00640 1 -- 172.21.15.16:0/314995091 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7ff5d8037020 con 0x55d11fdba390 2024-08-31T06:20:08.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.680+0000 7ff5fcc5f740 10 client.28516 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:20:08.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.680+0000 7ff5fcc5f740 10 client.28516 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-31T06:20:08.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.680+0000 7ff5fcc5f740 20 client.28516 populate_metadata read hostname 'smithi016' 2024-08-31T06:20:08.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.680+0000 7ff5fcc5f740 1 --2- 172.21.15.16:0/314995091 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55d11fd6eda0 0x55d12020b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:08.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.680+0000 7ff5fcc5f740 4 client.28516 mds_command: new command op to 24724 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:20:08.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.680+0000 7ff5fcc5f740 1 -- 172.21.15.16:0/314995091 --> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d1200f51a0 con 0x55d11fd6eda0 2024-08-31T06:20:08.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.680+0000 7ff5ef400640 1 --2- 172.21.15.16:0/314995091 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55d11fd6eda0 0x55d12020b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:08.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.681+0000 7ff5dcc00640 10 client.28516 ms_handle_connect on v2:172.21.15.16:6838/219664460 2024-08-31T06:20:08.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.681+0000 7ff5ef400640 1 --2- 172.21.15.16:0/314995091 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55d11fd6eda0 0x55d12020b820 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a56db8d2774ddb3a server_cookie=e1d766d201e4e87f in_seq=0 out_seq=0 2024-08-31T06:20:08.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.681+0000 7ff5dcc00640 1 -- 172.21.15.16:0/314995091 <== mds.1 v2:172.21.15.16:6838/219664460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d1200f51a0 con 0x55d11fd6eda0 2024-08-31T06:20:08.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.681+0000 7ff5dcc00640 10 client.28516 handle_command_reply: tid=0 2024-08-31T06:20:08.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.681+0000 7ff5fcc5f740 1 client.28516 shutdown 2024-08-31T06:20:08.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.681+0000 7ff5fcc5f740 1 -- 172.21.15.16:0/314995091 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11fdba390 msgr2=0x55d120190640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:08.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.681+0000 7ff5fcc5f740 1 --2- 172.21.15.16:0/314995091 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11fdba390 0x55d120190640 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8005890 tx=0x7ff5d80046c0 comp rx=0 tx=0).stop 2024-08-31T06:20:08.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.682+0000 7ff5fcc5f740 1 -- 172.21.15.16:0/314995091 shutdown_connections 2024-08-31T06:20:08.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.682+0000 7ff5fcc5f740 1 --2- 172.21.15.16:0/314995091 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d11fdba390 0x55d120190640 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:08.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.682+0000 7ff5fcc5f740 1 --2- 172.21.15.16:0/314995091 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55d11fd6eda0 0x55d12020b820 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:08.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.682+0000 7ff5fcc5f740 1 --2- 172.21.15.16:0/314995091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d11fddb070 0x55d120190b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:08.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.682+0000 7ff5fcc5f740 1 --2- 172.21.15.16:0/314995091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d11fdb21b0 0x55d120190100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:08.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.682+0000 7ff5fcc5f740 1 -- 172.21.15.16:0/314995091 >> 172.21.15.16:0/314995091 conn(0x55d11fdcf460 msgr2=0x55d12018d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:08.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.682+0000 7ff5dcc00640 0 client.28516 ms_handle_reset on v2:172.21.15.16:6838/219664460 2024-08-31T06:20:08.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.682+0000 7ff5fcc5f740 1 -- 172.21.15.16:0/314995091 shutdown_connections 2024-08-31T06:20:08.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.682+0000 7ff5fcc5f740 1 -- 172.21.15.16:0/314995091 wait complete. 2024-08-31T06:20:08.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.682+0000 7ff5fcc5f740 20 client.28516 trim_cache size 0 max 16384 2024-08-31T06:20:08.686 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:20:08.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.683+0000 7ff5f6000640 1 -- 172.21.15.16:0/389275209 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff5d40b4d20 msgr2=0x7ff5d40b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:08.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.683+0000 7ff5f6000640 1 --2- 172.21.15.16:0/389275209 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff5d40b4d20 0x7ff5d40b7230 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8005fd0 tx=0x7ff5e8009500 comp rx=0 tx=0).stop 2024-08-31T06:20:08.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.683+0000 7ff5f6000640 1 -- 172.21.15.16:0/389275209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5f0104770 msgr2=0x7ff5f019c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:08.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.683+0000 7ff5f6000640 1 --2- 172.21.15.16:0/389275209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5f0104770 0x7ff5f019c480 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0032fd0 tx=0x7ff5e0031a20 comp rx=0 tx=0).stop 2024-08-31T06:20:08.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.684+0000 7ff5f6000640 1 -- 172.21.15.16:0/389275209 shutdown_connections 2024-08-31T06:20:08.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.684+0000 7ff5f6000640 1 --2- 172.21.15.16:0/389275209 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff5d40b4d20 0x7ff5d40b7230 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:08.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.684+0000 7ff5f6000640 1 --2- 172.21.15.16:0/389275209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5f01050c0 0x7ff5f019ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:08.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.684+0000 7ff5f6000640 1 --2- 172.21.15.16:0/389275209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5f010dbd0 0x7ff5f01a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:08.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.684+0000 7ff5f6000640 1 --2- 172.21.15.16:0/389275209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5f0104770 0x7ff5f019c480 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:08.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.684+0000 7ff5f6000640 1 -- 172.21.15.16:0/389275209 >> 172.21.15.16:0/389275209 conn(0x7ff5f00794e0 msgr2=0x7ff5f01003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:08.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.684+0000 7ff5f6000640 1 -- 172.21.15.16:0/389275209 shutdown_connections 2024-08-31T06:20:08.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:08.685+0000 7ff5f6000640 1 -- 172.21.15.16:0/389275209 wait complete. 2024-08-31T06:20:08.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:08 smithi016 ceph-mon[17194]: pgmap v766: 97 pgs: 20 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 69 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 19 MiB/s wr, 1.74k op/s 2024-08-31T06:20:08.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:08 smithi184 ceph-mon[22331]: pgmap v766: 97 pgs: 20 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 69 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 19 MiB/s wr, 1.74k op/s 2024-08-31T06:20:09.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:08 smithi155 ceph-mon[21742]: pgmap v766: 97 pgs: 20 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 69 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 19 MiB/s wr, 1.74k op/s 2024-08-31T06:20:10.707 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T06:20:10.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:10 smithi016 ceph-mon[17194]: pgmap v767: 97 pgs: 12 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 81 active+clean; 3.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 19 MiB/s wr, 1.73k op/s 2024-08-31T06:20:10.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:10 smithi184 ceph-mon[22331]: pgmap v767: 97 pgs: 12 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 81 active+clean; 3.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 19 MiB/s wr, 1.73k op/s 2024-08-31T06:20:11.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:10 smithi155 ceph-mon[21742]: pgmap v767: 97 pgs: 12 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 81 active+clean; 3.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 19 MiB/s wr, 1.73k op/s 2024-08-31T06:20:11.314 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:20:11.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 765.0 s, 1213.2 tps, lat 81.374 ms stddev 92.544 2024-08-31T06:20:12.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.137+0000 7fab4b400640 1 -- 172.21.15.16:0/513356944 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab380318a0 con 0x7fab4c104640 2024-08-31T06:20:12.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.138+0000 7fab53600640 1 -- 172.21.15.16:0/513356944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab4c104640 msgr2=0x7fab4c104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:12.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.138+0000 7fab53600640 1 --2- 172.21.15.16:0/513356944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab4c104640 0x7fab4c104a50 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fab38009900 tx=0x7fab3802a400 comp rx=0 tx=0).stop 2024-08-31T06:20:12.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.138+0000 7fab53600640 1 -- 172.21.15.16:0/513356944 shutdown_connections 2024-08-31T06:20:12.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.138+0000 7fab53600640 1 --2- 172.21.15.16:0/513356944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab4c10dbf0 0x7fab4c1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:12.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.138+0000 7fab53600640 1 --2- 172.21.15.16:0/513356944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab4c105020 0x7fab4c10d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:12.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.138+0000 7fab53600640 1 --2- 172.21.15.16:0/513356944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab4c104640 0x7fab4c104a50 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:12.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.138+0000 7fab53600640 1 -- 172.21.15.16:0/513356944 >> 172.21.15.16:0/513356944 conn(0x7fab4c0794c0 msgr2=0x7fab4c0798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:12.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.138+0000 7fab53600640 1 -- 172.21.15.16:0/513356944 shutdown_connections 2024-08-31T06:20:12.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.138+0000 7fab53600640 1 -- 172.21.15.16:0/513356944 wait complete. 2024-08-31T06:20:12.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.140+0000 7fab53600640 1 Processor -- start 2024-08-31T06:20:12.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.142+0000 7fab53600640 1 -- start start 2024-08-31T06:20:12.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.143+0000 7fab53600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab4c104640 0x7fab4c1a1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:12.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.143+0000 7fab53600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab4c105020 0x7fab4c1a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:12.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.143+0000 7fab53600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab4c10dbf0 0x7fab4c1a1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:12.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.143+0000 7fab53600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c1138a0 con 0x7fab4c105020 2024-08-31T06:20:12.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.143+0000 7fab53600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c113720 con 0x7fab4c10dbf0 2024-08-31T06:20:12.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.143+0000 7fab53600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c113a20 con 0x7fab4c104640 2024-08-31T06:20:12.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.143+0000 7fab4be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab4c105020 0x7fab4c1a1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:12.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.143+0000 7fab51600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab4c10dbf0 0x7fab4c1a1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:12.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.143+0000 7fab4be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab4c105020 0x7fab4c1a1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50906/0 (socket says 172.21.15.16:50906) 2024-08-31T06:20:12.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.143+0000 7fab51600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab4c10dbf0 0x7fab4c1a1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:34902/0 (socket says 172.21.15.16:34902) 2024-08-31T06:20:12.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.144+0000 7fab4be00640 1 -- 172.21.15.16:0/2779168159 learned_addr learned my addr 172.21.15.16:0/2779168159 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:12.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.144+0000 7fab50c00640 1 --2- 172.21.15.16:0/2779168159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab4c104640 0x7fab4c1a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:12.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.144+0000 7fab51600640 1 -- 172.21.15.16:0/2779168159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab4c104640 msgr2=0x7fab4c1a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:12.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.144+0000 7fab51600640 1 --2- 172.21.15.16:0/2779168159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab4c104640 0x7fab4c1a1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:12.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.144+0000 7fab51600640 1 -- 172.21.15.16:0/2779168159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab4c105020 msgr2=0x7fab4c1a1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:12.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.144+0000 7fab51600640 1 --2- 172.21.15.16:0/2779168159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab4c105020 0x7fab4c1a1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:12.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.144+0000 7fab51600640 1 -- 172.21.15.16:0/2779168159 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab4c1ac880 con 0x7fab4c10dbf0 2024-08-31T06:20:12.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.144+0000 7fab4be00640 1 --2- 172.21.15.16:0/2779168159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab4c105020 0x7fab4c1a1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:20:12.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.145+0000 7fab51600640 1 --2- 172.21.15.16:0/2779168159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab4c10dbf0 0x7fab4c1a1fe0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fab44009fd0 tx=0x7fab4400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:12.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.145+0000 7fab49600640 1 -- 172.21.15.16:0/2779168159 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4400ce20 con 0x7fab4c10dbf0 2024-08-31T06:20:12.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.145+0000 7fab49600640 1 -- 172.21.15.16:0/2779168159 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab44005020 con 0x7fab4c10dbf0 2024-08-31T06:20:12.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.145+0000 7fab49600640 1 -- 172.21.15.16:0/2779168159 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab44010690 con 0x7fab4c10dbf0 2024-08-31T06:20:12.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.145+0000 7fab53600640 1 -- 172.21.15.16:0/2779168159 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab4c1acb70 con 0x7fab4c10dbf0 2024-08-31T06:20:12.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.146+0000 7fab53600640 1 -- 172.21.15.16:0/2779168159 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab4c1ad060 con 0x7fab4c10dbf0 2024-08-31T06:20:12.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.151+0000 7fab49600640 1 -- 172.21.15.16:0/2779168159 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fab440054d0 con 0x7fab4c10dbf0 2024-08-31T06:20:12.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.151+0000 7fab5a292740 1 Processor -- start 2024-08-31T06:20:12.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.151+0000 7fab5a292740 1 -- start start 2024-08-31T06:20:12.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.151+0000 7fab5a292740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f5517c070 0x559f55521990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:12.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.151+0000 7fab5a292740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f551531b0 0x559f55521ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:12.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.151+0000 7fab5a292740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f5515b390 0x559f55522410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:12.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.151+0000 7fab5a292740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559f554987b0 con 0x559f551531b0 2024-08-31T06:20:12.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.152+0000 7fab5a292740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559f55487d40 con 0x559f5515b390 2024-08-31T06:20:12.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.152+0000 7fab5a292740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559f5548bcf0 con 0x559f5517c070 2024-08-31T06:20:12.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.152+0000 7fab4be00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f5517c070 0x559f55521990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:12.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.152+0000 7fab4be00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f5517c070 0x559f55521990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:51464/0 (socket says 172.21.15.16:51464) 2024-08-31T06:20:12.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.152+0000 7fab4be00640 1 -- 172.21.15.16:0/2860521803 learned_addr learned my addr 172.21.15.16:0/2860521803 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:12.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.152+0000 7fab4be00640 1 --2- 172.21.15.16:0/2860521803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f5515b390 0x559f55522410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:12.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.152+0000 7fab50c00640 1 --2- 172.21.15.16:0/2860521803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f551531b0 0x559f55521ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.152+0000 7fab4be00640 1 -- 172.21.15.16:0/2860521803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f5515b390 msgr2=0x559f55522410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.152+0000 7fab4be00640 1 --2- 172.21.15.16:0/2860521803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f5515b390 0x559f55522410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.152+0000 7fab4be00640 1 -- 172.21.15.16:0/2860521803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f551531b0 msgr2=0x559f55521ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:12.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.152+0000 7fab4be00640 1 --2- 172.21.15.16:0/2860521803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f551531b0 0x559f55521ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:12.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.152+0000 7fab4be00640 1 -- 172.21.15.16:0/2860521803 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f551e7650 con 0x559f5517c070 2024-08-31T06:20:12.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.152+0000 7fab50c00640 1 --2- 172.21.15.16:0/2860521803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f551531b0 0x559f55521ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:20:12.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.153+0000 7fab49600640 1 --2- 172.21.15.16:0/2779168159 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fab300b4da0 0x7fab300b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:12.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.153+0000 7fab4be00640 1 --2- 172.21.15.16:0/2860521803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f5517c070 0x559f55521990 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fab3c011c30 tx=0x7fab3c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c031032922cc3a7b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:12.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.153+0000 7fab49600640 1 -- 172.21.15.16:0/2779168159 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8514+0+0 (secure 0 0 0) 0x7fab440e5c70 con 0x7fab4c10dbf0 2024-08-31T06:20:12.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.153+0000 7fab41c00640 1 -- 172.21.15.16:0/2860521803 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab3c00fc50 con 0x559f5517c070 2024-08-31T06:20:12.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.153+0000 7fab41c00640 1 -- 172.21.15.16:0/2860521803 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab3c00fdf0 con 0x559f5517c070 2024-08-31T06:20:12.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.153+0000 7fab50c00640 1 --2- 172.21.15.16:0/2779168159 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fab300b4da0 0x7fab300b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:12.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.153+0000 7fab41c00640 1 -- 172.21.15.16:0/2860521803 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab3c00fc50 con 0x559f5517c070 2024-08-31T06:20:12.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.154+0000 7fab5a292740 1 -- 172.21.15.16:0/2860521803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f5517c070 msgr2=0x559f55521990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:12.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.154+0000 7fab5a292740 1 --2- 172.21.15.16:0/2860521803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f5517c070 0x559f55521990 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fab3c011c30 tx=0x7fab3c009c00 comp rx=0 tx=0).stop 2024-08-31T06:20:12.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.155+0000 7fab5a292740 1 -- 172.21.15.16:0/2860521803 shutdown_connections 2024-08-31T06:20:12.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.155+0000 7fab5a292740 1 --2- 172.21.15.16:0/2860521803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f5515b390 0x559f55522410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:12.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.155+0000 7fab5a292740 1 --2- 172.21.15.16:0/2860521803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f551531b0 0x559f55521ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:12.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.155+0000 7fab5a292740 1 --2- 172.21.15.16:0/2860521803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f5517c070 0x559f55521990 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:12.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.155+0000 7fab5a292740 1 -- 172.21.15.16:0/2860521803 >> 172.21.15.16:0/2860521803 conn(0x559f55170460 msgr2=0x559f5551f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:12.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.155+0000 7fab50c00640 1 --2- 172.21.15.16:0/2779168159 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fab300b4da0 0x7fab300b72b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fab38000c00 tx=0x7fab38032d00 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:12.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.155+0000 7fab5a292740 1 -- 172.21.15.16:0/2860521803 shutdown_connections 2024-08-31T06:20:12.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.155+0000 7fab5a292740 1 -- 172.21.15.16:0/2860521803 wait complete. 2024-08-31T06:20:12.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.156+0000 7fab5a292740 1 -- start start 2024-08-31T06:20:12.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.157+0000 7fab5a292740 1 Processor -- start 2024-08-31T06:20:12.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.158+0000 7fab5a292740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f551531b0 0x559f5552d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:12.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.158+0000 7fab5a292740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f5515b390 0x559f5552dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:12.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.158+0000 7fab5a292740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f5517c070 0x559f5554a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:12.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.158+0000 7fab5a292740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559f552ea700 con 0x559f551531b0 2024-08-31T06:20:12.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.158+0000 7fab5a292740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559f5540ecf0 con 0x559f5515b390 2024-08-31T06:20:12.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.158+0000 7fab5a292740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559f55541180 con 0x559f5517c070 2024-08-31T06:20:12.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.158+0000 7fab5a292740 10 client.0 _ll_register_callbacks cb 0x559f55434470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:20:12.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.159+0000 7fab51600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f5517c070 0x559f5554a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:12.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.159+0000 7fab51600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f5517c070 0x559f5554a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:51474/0 (socket says 172.21.15.16:51474) 2024-08-31T06:20:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.159+0000 7fab51600640 1 -- 172.21.15.16:0/3169799434 learned_addr learned my addr 172.21.15.16:0/3169799434 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.160+0000 7fab51600640 1 -- 172.21.15.16:0/3169799434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f5515b390 msgr2=0x559f5552dbf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T06:20:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.160+0000 7fab51600640 1 --2- 172.21.15.16:0/3169799434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f5515b390 0x559f5552dbf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.160+0000 7fab51600640 1 -- 172.21.15.16:0/3169799434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f551531b0 msgr2=0x559f5552d610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:20:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.160+0000 7fab50c00640 1 --2- 172.21.15.16:0/3169799434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f551531b0 0x559f5552d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.160+0000 7fab51600640 1 --2- 172.21.15.16:0/3169799434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f551531b0 0x559f5552d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.160+0000 7fab51600640 1 -- 172.21.15.16:0/3169799434 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f5544f8b0 con 0x559f5517c070 2024-08-31T06:20:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.160+0000 7fab51600640 1 --2- 172.21.15.16:0/3169799434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f5517c070 0x559f5554a470 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fab44005820 tx=0x7fab440acac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=589afd28ec5a8a56 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.160+0000 7fab36000640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:20:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.161+0000 7fab36000640 1 -- 172.21.15.16:0/3169799434 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab440b5620 con 0x559f5517c070 2024-08-31T06:20:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.161+0000 7fab36000640 1 -- 172.21.15.16:0/3169799434 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8514+0+0 (secure 0 0 0) 0x7fab440e35e0 con 0x559f5517c070 2024-08-31T06:20:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.161+0000 7fab5a292740 1 -- 172.21.15.16:0/3169799434 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab3c00fdf0 con 0x559f5517c070 2024-08-31T06:20:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.161+0000 7fab36000640 1 -- 172.21.15.16:0/3169799434 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fab440e37e0 con 0x559f5517c070 2024-08-31T06:20:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.161+0000 7fab5a292740 10 client.28528 fetch_fsmap learned FSMap version 49 2024-08-31T06:20:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.161+0000 7fab5a292740 1 -- 172.21.15.16:0/3169799434 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x559f5554d210 con 0x559f5517c070 2024-08-31T06:20:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.161+0000 7fab36000640 1 -- 172.21.15.16:0/3169799434 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab3c00fdf0 con 0x559f5517c070 2024-08-31T06:20:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.162+0000 7fab36000640 1 -- 172.21.15.16:0/3169799434 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fab440e3ab0 con 0x559f5517c070 2024-08-31T06:20:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.162+0000 7fab5a292740 10 client.28528 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:20:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.162+0000 7fab5a292740 10 client.28528 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:20:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.162+0000 7fab5a292740 20 client.28528 populate_metadata read hostname 'smithi016' 2024-08-31T06:20:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.162+0000 7fab5a292740 1 --2- 172.21.15.16:0/3169799434 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559f5513a9f0 0x559f55550750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.162+0000 7fab5a292740 4 client.28528 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:20:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.162+0000 7fab5a292740 1 -- 172.21.15.16:0/3169799434 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f5549a700 con 0x559f5513a9f0 2024-08-31T06:20:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.162+0000 7fab50c00640 1 --2- 172.21.15.16:0/3169799434 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559f5513a9f0 0x559f55550750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.163+0000 7fab50c00640 1 --2- 172.21.15.16:0/3169799434 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559f5513a9f0 0x559f55550750 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e24da1af244338f server_cookie=38ec59c0e640727a in_seq=0 out_seq=0 2024-08-31T06:20:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.163+0000 7fab36000640 10 client.28528 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:20:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.164+0000 7fab36000640 1 -- 172.21.15.16:0/3169799434 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559f5549a700 con 0x559f5513a9f0 2024-08-31T06:20:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.164+0000 7fab36000640 10 client.28528 handle_command_reply: tid=0 2024-08-31T06:20:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.164+0000 7fab5a292740 1 client.28528 shutdown 2024-08-31T06:20:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.165+0000 7fab5a292740 1 -- 172.21.15.16:0/3169799434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f5517c070 msgr2=0x559f5554a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.165+0000 7fab5a292740 1 --2- 172.21.15.16:0/3169799434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f5517c070 0x559f5554a470 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fab44005820 tx=0x7fab440acac0 comp rx=0 tx=0).stop 2024-08-31T06:20:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.165+0000 7fab5a292740 1 -- 172.21.15.16:0/3169799434 shutdown_connections 2024-08-31T06:20:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.165+0000 7fab5a292740 1 --2- 172.21.15.16:0/3169799434 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559f5513a9f0 0x559f55550750 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.165+0000 7fab5a292740 1 --2- 172.21.15.16:0/3169799434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f5517c070 0x559f5554a470 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.165+0000 7fab5a292740 1 --2- 172.21.15.16:0/3169799434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f5515b390 0x559f5552dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.165+0000 7fab5a292740 1 --2- 172.21.15.16:0/3169799434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f551531b0 0x559f5552d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.165+0000 7fab5a292740 1 -- 172.21.15.16:0/3169799434 >> 172.21.15.16:0/3169799434 conn(0x559f55170460 msgr2=0x559f55524b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.165+0000 7fab36000640 0 client.28528 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:20:12.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.165+0000 7fab5a292740 1 -- 172.21.15.16:0/3169799434 shutdown_connections 2024-08-31T06:20:12.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.166+0000 7fab5a292740 1 -- 172.21.15.16:0/3169799434 wait complete. 2024-08-31T06:20:12.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.166+0000 7fab5a292740 20 client.28528 trim_cache size 0 max 16384 2024-08-31T06:20:12.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.191+0000 7fab5a292740 1 Processor -- start 2024-08-31T06:20:12.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.191+0000 7fab5a292740 1 -- start start 2024-08-31T06:20:12.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.192+0000 7fab5a292740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f551531b0 0x559f5552f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:12.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.192+0000 7fab5a292740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f5515b390 0x559f555230d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:12.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.192+0000 7fab5a292740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f5517c070 0x559f55526bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:12.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.192+0000 7fab5a292740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c1afc20 con 0x559f551531b0 2024-08-31T06:20:12.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.192+0000 7fab5a292740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c1ad450 con 0x559f5515b390 2024-08-31T06:20:12.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.192+0000 7fab5a292740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c10cfa0 con 0x559f5517c070 2024-08-31T06:20:12.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.193+0000 7fab51600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f5517c070 0x559f55526bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:12.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.193+0000 7fab51600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f5517c070 0x559f55526bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:51486/0 (socket says 172.21.15.16:51486) 2024-08-31T06:20:12.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.193+0000 7fab51600640 1 -- 172.21.15.16:0/1141427959 learned_addr learned my addr 172.21.15.16:0/1141427959 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:12.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.193+0000 7fab4be00640 1 --2- 172.21.15.16:0/1141427959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f5515b390 0x559f555230d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:12.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.194+0000 7fab51600640 1 -- 172.21.15.16:0/1141427959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f5515b390 msgr2=0x559f555230d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:12.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.194+0000 7fab51600640 1 --2- 172.21.15.16:0/1141427959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f5515b390 0x559f555230d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:12.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.194+0000 7fab51600640 1 -- 172.21.15.16:0/1141427959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f551531b0 msgr2=0x559f5552f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:12.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.194+0000 7fab51600640 1 --2- 172.21.15.16:0/1141427959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f551531b0 0x559f5552f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:12.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.194+0000 7fab51600640 1 -- 172.21.15.16:0/1141427959 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f5552b910 con 0x559f5517c070 2024-08-31T06:20:12.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.194+0000 7fab51600640 1 --2- 172.21.15.16:0/1141427959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f5517c070 0x559f55526bf0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fab440ac000 tx=0x7fab440053e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab150daaba16ef76 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:12.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.195+0000 7fab41200640 1 -- 172.21.15.16:0/1141427959 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab440e3ab0 con 0x559f5517c070 2024-08-31T06:20:12.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.195+0000 7fab41200640 1 -- 172.21.15.16:0/1141427959 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab440e35e0 con 0x559f5517c070 2024-08-31T06:20:12.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.195+0000 7fab41200640 1 -- 172.21.15.16:0/1141427959 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab440a7c70 con 0x559f5517c070 2024-08-31T06:20:12.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.197+0000 7fab5a292740 1 -- 172.21.15.16:0/1141427959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f5517c070 msgr2=0x559f55526bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:12.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.197+0000 7fab5a292740 1 --2- 172.21.15.16:0/1141427959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f5517c070 0x559f55526bf0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fab440ac000 tx=0x7fab440053e0 comp rx=0 tx=0).stop 2024-08-31T06:20:12.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.199+0000 7fab5a292740 1 -- 172.21.15.16:0/1141427959 shutdown_connections 2024-08-31T06:20:12.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.199+0000 7fab5a292740 1 --2- 172.21.15.16:0/1141427959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f5517c070 0x559f55526bf0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:12.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.199+0000 7fab5a292740 1 --2- 172.21.15.16:0/1141427959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f5515b390 0x559f555230d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:12.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.199+0000 7fab5a292740 1 --2- 172.21.15.16:0/1141427959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f551531b0 0x559f5552f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:12.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.199+0000 7fab5a292740 1 -- 172.21.15.16:0/1141427959 >> 172.21.15.16:0/1141427959 conn(0x559f55170460 msgr2=0x559f555229c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:12.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.200+0000 7fab5a292740 1 -- 172.21.15.16:0/1141427959 shutdown_connections 2024-08-31T06:20:12.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.200+0000 7fab5a292740 1 -- 172.21.15.16:0/1141427959 wait complete. 2024-08-31T06:20:12.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.200+0000 7fab5a292740 1 -- start start 2024-08-31T06:20:12.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.201+0000 7fab5a292740 1 Processor -- start 2024-08-31T06:20:12.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.201+0000 7fab5a292740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f551531b0 0x559f555463e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:12.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.201+0000 7fab5a292740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f5515b390 0x559f55546920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:12.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.202+0000 7fab5a292740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f5517c070 0x559f55541390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:12.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.202+0000 7fab5a292740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c073a50 con 0x559f5515b390 2024-08-31T06:20:12.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.202+0000 7fab5a292740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c074ec0 con 0x559f551531b0 2024-08-31T06:20:12.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.202+0000 7fab5a292740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c074120 con 0x559f5517c070 2024-08-31T06:20:12.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.202+0000 7fab4be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f5515b390 0x559f55546920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:12.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.202+0000 7fab5a292740 10 client.0 _ll_register_callbacks cb 0x7fab440abe10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:20:12.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.202+0000 7fab50c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f551531b0 0x559f555463e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:12.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.202+0000 7fab51600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f5517c070 0x559f55541390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:12.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.203+0000 7fab51600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f5517c070 0x559f55541390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:51502/0 (socket says 172.21.15.16:51502) 2024-08-31T06:20:12.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.203+0000 7fab51600640 1 -- 172.21.15.16:0/2981854806 learned_addr learned my addr 172.21.15.16:0/2981854806 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:12.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.205+0000 7fab51600640 1 -- 172.21.15.16:0/2981854806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f551531b0 msgr2=0x559f555463e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:12.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.205+0000 7fab51600640 1 --2- 172.21.15.16:0/2981854806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f551531b0 0x559f555463e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:12.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.205+0000 7fab51600640 1 -- 172.21.15.16:0/2981854806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f5515b390 msgr2=0x559f55546920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:12.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.205+0000 7fab51600640 1 --2- 172.21.15.16:0/2981854806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f5515b390 0x559f55546920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:12.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.205+0000 7fab51600640 1 -- 172.21.15.16:0/2981854806 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f55547090 con 0x559f5517c070 2024-08-31T06:20:12.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.205+0000 7fab50c00640 1 --2- 172.21.15.16:0/2981854806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f551531b0 0x559f555463e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:20:12.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.206+0000 7fab36000640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:20:12.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.206+0000 7fab51600640 1 --2- 172.21.15.16:0/2981854806 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f5517c070 0x559f55541390 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fab440b57e0 tx=0x7fab440057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23a60db947064a57 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:12.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.206+0000 7fab36000640 1 -- 172.21.15.16:0/2981854806 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4408e060 con 0x559f5517c070 2024-08-31T06:20:12.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.206+0000 7fab36000640 1 -- 172.21.15.16:0/2981854806 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8514+0+0 (secure 0 0 0) 0x7fab440e3790 con 0x559f5517c070 2024-08-31T06:20:12.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.207+0000 7fab5a292740 1 -- 172.21.15.16:0/2981854806 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab440e35e0 con 0x559f5517c070 2024-08-31T06:20:12.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.207+0000 7fab36000640 1 -- 172.21.15.16:0/2981854806 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fab440ab230 con 0x559f5517c070 2024-08-31T06:20:12.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.207+0000 7fab5a292740 10 client.28534 fetch_fsmap learned FSMap version 49 2024-08-31T06:20:12.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.207+0000 7fab5a292740 1 -- 172.21.15.16:0/2981854806 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x559f55547090 con 0x559f5517c070 2024-08-31T06:20:12.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.208+0000 7fab36000640 1 -- 172.21.15.16:0/2981854806 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab440e35e0 con 0x559f5517c070 2024-08-31T06:20:12.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.208+0000 7fab36000640 1 -- 172.21.15.16:0/2981854806 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fab440b1e20 con 0x559f5517c070 2024-08-31T06:20:12.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.208+0000 7fab5a292740 10 client.28534 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:20:12.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.208+0000 7fab5a292740 10 client.28534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:20:12.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.208+0000 7fab5a292740 20 client.28534 populate_metadata read hostname 'smithi016' 2024-08-31T06:20:12.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.208+0000 7fab5a292740 1 --2- 172.21.15.16:0/2981854806 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559f5510fda0 0x559f555bd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:12.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.209+0000 7fab5a292740 4 client.28534 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:20:12.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.209+0000 7fab5a292740 1 -- 172.21.15.16:0/2981854806 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559f554961a0 con 0x559f5510fda0 2024-08-31T06:20:12.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.210+0000 7fab50c00640 1 --2- 172.21.15.16:0/2981854806 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559f5510fda0 0x559f555bd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:12.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.213+0000 7fab50c00640 1 --2- 172.21.15.16:0/2981854806 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559f5510fda0 0x559f555bd8c0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=885157a5075f1c11 server_cookie=61575f8759255e1d in_seq=0 out_seq=0 2024-08-31T06:20:12.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.213+0000 7fab36000640 10 client.28534 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:20:12.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.214+0000 7fab36000640 1 -- 172.21.15.16:0/2981854806 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559f554961a0 con 0x559f5510fda0 2024-08-31T06:20:12.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.214+0000 7fab36000640 10 client.28534 handle_command_reply: tid=0 2024-08-31T06:20:12.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.214+0000 7fab5a292740 1 client.28534 shutdown 2024-08-31T06:20:12.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.214+0000 7fab5a292740 1 -- 172.21.15.16:0/2981854806 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f5517c070 msgr2=0x559f55541390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:12.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.214+0000 7fab5a292740 1 --2- 172.21.15.16:0/2981854806 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f5517c070 0x559f55541390 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fab440b57e0 tx=0x7fab440057c0 comp rx=0 tx=0).stop 2024-08-31T06:20:12.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.215+0000 7fab5a292740 1 -- 172.21.15.16:0/2981854806 shutdown_connections 2024-08-31T06:20:12.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.215+0000 7fab5a292740 1 --2- 172.21.15.16:0/2981854806 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559f5510fda0 0x559f555bd8c0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:12.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.215+0000 7fab5a292740 1 --2- 172.21.15.16:0/2981854806 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f5517c070 0x559f55541390 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:12.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.215+0000 7fab5a292740 1 --2- 172.21.15.16:0/2981854806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559f5515b390 0x559f55546920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:12.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.215+0000 7fab5a292740 1 --2- 172.21.15.16:0/2981854806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f551531b0 0x559f555463e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:12.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.215+0000 7fab5a292740 1 -- 172.21.15.16:0/2981854806 >> 172.21.15.16:0/2981854806 conn(0x559f55170460 msgr2=0x559f55523150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:12.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.215+0000 7fab36000640 0 client.28534 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:20:12.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.215+0000 7fab5a292740 1 -- 172.21.15.16:0/2981854806 shutdown_connections 2024-08-31T06:20:12.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.215+0000 7fab5a292740 1 -- 172.21.15.16:0/2981854806 wait complete. 2024-08-31T06:20:12.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.215+0000 7fab5a292740 20 client.28534 trim_cache size 0 max 16384 2024-08-31T06:20:12.219 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:20:12.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.217+0000 7fab53600640 1 -- 172.21.15.16:0/2779168159 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fab300b4da0 msgr2=0x7fab300b72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:12.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.217+0000 7fab53600640 1 --2- 172.21.15.16:0/2779168159 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fab300b4da0 0x7fab300b72b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fab38000c00 tx=0x7fab38032d00 comp rx=0 tx=0).stop 2024-08-31T06:20:12.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.218+0000 7fab53600640 1 -- 172.21.15.16:0/2779168159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab4c10dbf0 msgr2=0x7fab4c1a1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:12.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.218+0000 7fab53600640 1 --2- 172.21.15.16:0/2779168159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab4c10dbf0 0x7fab4c1a1fe0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fab44009fd0 tx=0x7fab4400ef10 comp rx=0 tx=0).stop 2024-08-31T06:20:12.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.219+0000 7fab53600640 1 -- 172.21.15.16:0/2779168159 shutdown_connections 2024-08-31T06:20:12.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.219+0000 7fab53600640 1 --2- 172.21.15.16:0/2779168159 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fab300b4da0 0x7fab300b72b0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:12.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.219+0000 7fab53600640 1 --2- 172.21.15.16:0/2779168159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab4c10dbf0 0x7fab4c1a1fe0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:12.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.219+0000 7fab53600640 1 --2- 172.21.15.16:0/2779168159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab4c105020 0x7fab4c1a1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:12.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.219+0000 7fab53600640 1 --2- 172.21.15.16:0/2779168159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab4c104640 0x7fab4c1a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:12.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.219+0000 7fab53600640 1 -- 172.21.15.16:0/2779168159 >> 172.21.15.16:0/2779168159 conn(0x7fab4c0794c0 msgr2=0x7fab4c107720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:12.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.219+0000 7fab53600640 1 -- 172.21.15.16:0/2779168159 shutdown_connections 2024-08-31T06:20:12.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:12.220+0000 7fab53600640 1 -- 172.21.15.16:0/2779168159 wait complete. 2024-08-31T06:20:12.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:12 smithi016 ceph-mon[17194]: pgmap v768: 97 pgs: 97 active+clean; 3.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 19 MiB/s wr, 1.73k op/s 2024-08-31T06:20:12.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:12 smithi184 ceph-mon[22331]: pgmap v768: 97 pgs: 97 active+clean; 3.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 19 MiB/s wr, 1.73k op/s 2024-08-31T06:20:13.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:12 smithi155 ceph-mon[21742]: pgmap v768: 97 pgs: 97 active+clean; 3.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 19 MiB/s wr, 1.73k op/s 2024-08-31T06:20:14.648 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T06:20:14.649 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T06:20:14.946 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 14.7 2024-08-31T06:20:14.946 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:20:14.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:14 smithi184 ceph-mon[22331]: pgmap v769: 97 pgs: 97 active+clean; 3.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 15 MiB/s wr, 1.39k op/s 2024-08-31T06:20:15.035 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:14 smithi016 ceph-mon[17194]: pgmap v769: 97 pgs: 97 active+clean; 3.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 15 MiB/s wr, 1.39k op/s 2024-08-31T06:20:15.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:14 smithi155 ceph-mon[21742]: pgmap v769: 97 pgs: 97 active+clean; 3.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 15 MiB/s wr, 1.39k op/s 2024-08-31T06:20:15.274 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:20:16.463 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:20:16.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.462+0000 7fe6a4e00640 1 -- 172.21.15.16:0/698827270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6a0105b30 msgr2=0x7fe6a0105f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:16.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.462+0000 7fe6a4e00640 1 --2- 172.21.15.16:0/698827270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6a0105b30 0x7fe6a0105f40 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe690009860 tx=0x7fe69002a340 comp rx=0 tx=0).stop 2024-08-31T06:20:16.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.462+0000 7fe6a4e00640 1 -- 172.21.15.16:0/698827270 shutdown_connections 2024-08-31T06:20:16.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.462+0000 7fe6a4e00640 1 --2- 172.21.15.16:0/698827270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6a010bdc0 0x7fe6a01125f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:16.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.462+0000 7fe6a4e00640 1 --2- 172.21.15.16:0/698827270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6a010b470 0x7fe6a010b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:16.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.462+0000 7fe6a4e00640 1 --2- 172.21.15.16:0/698827270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6a0105b30 0x7fe6a0105f40 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:16.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.462+0000 7fe6a4e00640 1 -- 172.21.15.16:0/698827270 >> 172.21.15.16:0/698827270 conn(0x7fe6a0101600 msgr2=0x7fe6a0103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:16.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.463+0000 7fe6a4e00640 1 -- 172.21.15.16:0/698827270 shutdown_connections 2024-08-31T06:20:16.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.464+0000 7fe6a4e00640 1 -- 172.21.15.16:0/698827270 wait complete. 2024-08-31T06:20:16.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.468+0000 7fe6a4e00640 1 Processor -- start 2024-08-31T06:20:16.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.468+0000 7fe6a4e00640 1 -- start start 2024-08-31T06:20:16.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.468+0000 7fe6a4e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6a010b470 0x7fe6a010fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:16.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.468+0000 7fe6a4e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6a010bdc0 0x7fe6a0110050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:16.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.469+0000 7fe69d400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6a010bdc0 0x7fe6a0110050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:16.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.469+0000 7fe69d400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6a010bdc0 0x7fe6a0110050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:41490/0 (socket says 172.21.15.16:41490) 2024-08-31T06:20:16.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.469+0000 7fe6a4e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6a0110590 0x7fe6a01ac7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:16.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.469+0000 7fe6a4e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a0115ad0 con 0x7fe6a0110590 2024-08-31T06:20:16.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.469+0000 7fe6a4e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a0115950 con 0x7fe6a010bdc0 2024-08-31T06:20:16.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.469+0000 7fe6a4e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a0115c50 con 0x7fe6a010b470 2024-08-31T06:20:16.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.469+0000 7fe69de00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6a010b470 0x7fe6a010fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.469+0000 7fe69de00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6a010b470 0x7fe6a010fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:57332/0 (socket says 172.21.15.16:57332) 2024-08-31T06:20:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.469+0000 7fe69de00640 1 -- 172.21.15.16:0/1867418844 learned_addr learned my addr 172.21.15.16:0/1867418844 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.471+0000 7fe69de00640 1 -- 172.21.15.16:0/1867418844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6a010bdc0 msgr2=0x7fe6a0110050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.471+0000 7fe69de00640 1 --2- 172.21.15.16:0/1867418844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6a010bdc0 0x7fe6a0110050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.471+0000 7fe69de00640 1 -- 172.21.15.16:0/1867418844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6a0110590 msgr2=0x7fe6a01ac7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.471+0000 7fe69de00640 1 --2- 172.21.15.16:0/1867418844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6a0110590 0x7fe6a01ac7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.471+0000 7fe69de00640 1 -- 172.21.15.16:0/1867418844 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a01acec0 con 0x7fe6a010b470 2024-08-31T06:20:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.471+0000 7fe69de00640 1 --2- 172.21.15.16:0/1867418844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6a010b470 0x7fe6a010fb10 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe69002a8b0 tx=0x7fe690030d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:16.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.474+0000 7fe696a00640 1 -- 172.21.15.16:0/1867418844 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe690041070 con 0x7fe6a010b470 2024-08-31T06:20:16.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.474+0000 7fe696a00640 1 -- 172.21.15.16:0/1867418844 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe690030900 con 0x7fe6a010b470 2024-08-31T06:20:16.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.474+0000 7fe696a00640 1 -- 172.21.15.16:0/1867418844 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe690037040 con 0x7fe6a010b470 2024-08-31T06:20:16.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.474+0000 7fe6a4e00640 1 -- 172.21.15.16:0/1867418844 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6a01ad150 con 0x7fe6a010b470 2024-08-31T06:20:16.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.474+0000 7fe6a4e00640 1 -- 172.21.15.16:0/1867418844 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6a01ad5e0 con 0x7fe6a010b470 2024-08-31T06:20:16.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.486+0000 7fe6aba49740 1 Processor -- start 2024-08-31T06:20:16.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.486+0000 7fe6aba49740 1 -- start start 2024-08-31T06:20:16.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.487+0000 7fe6aba49740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df406f0350 0x55df40a96600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:16.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.487+0000 7fe696a00640 1 -- 172.21.15.16:0/1867418844 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fe6900052b0 con 0x7fe6a010b470 2024-08-31T06:20:16.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.487+0000 7fe6aba49740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df406c7490 0x55df40a96b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:16.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.487+0000 7fe6aba49740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df406cf670 0x55df40a97080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:16.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.487+0000 7fe6aba49740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55df40a0c750 con 0x55df406cf670 2024-08-31T06:20:16.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.487+0000 7fe6aba49740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55df409fbce0 con 0x55df406f0350 2024-08-31T06:20:16.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.487+0000 7fe6aba49740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55df409ffc90 con 0x55df406c7490 2024-08-31T06:20:16.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.488+0000 7fe69e800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df406f0350 0x55df40a96600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:16.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.488+0000 7fe69e800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df406f0350 0x55df40a96600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:41498/0 (socket says 172.21.15.16:41498) 2024-08-31T06:20:16.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.488+0000 7fe69e800640 1 -- 172.21.15.16:0/415170087 learned_addr learned my addr 172.21.15.16:0/415170087 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:16.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.489+0000 7fe69de00640 1 --2- 172.21.15.16:0/415170087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df406c7490 0x55df40a96b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:16.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.489+0000 7fe69d400640 1 --2- 172.21.15.16:0/415170087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df406cf670 0x55df40a97080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:16.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.489+0000 7fe69e800640 1 -- 172.21.15.16:0/415170087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df406c7490 msgr2=0x55df40a96b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:16.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.489+0000 7fe69e800640 1 --2- 172.21.15.16:0/415170087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df406c7490 0x55df40a96b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:16.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.489+0000 7fe69e800640 1 -- 172.21.15.16:0/415170087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df406cf670 msgr2=0x55df40a97080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:16.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.490+0000 7fe69e800640 1 --2- 172.21.15.16:0/415170087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df406cf670 0x55df40a97080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:16.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.490+0000 7fe69e800640 1 -- 172.21.15.16:0/415170087 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df4075b930 con 0x55df406f0350 2024-08-31T06:20:16.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.490+0000 7fe69d400640 1 --2- 172.21.15.16:0/415170087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df406cf670 0x55df40a97080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:20:16.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.491+0000 7fe69e800640 1 --2- 172.21.15.16:0/415170087 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df406f0350 0x55df40a96600 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe69800ba00 tx=0x7fe69800bec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=feb7213e56f66efa server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:16.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.493+0000 7fe68f400640 1 -- 172.21.15.16:0/415170087 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe698016070 con 0x55df406f0350 2024-08-31T06:20:16.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.493+0000 7fe68f400640 1 -- 172.21.15.16:0/415170087 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6980054d0 con 0x55df406f0350 2024-08-31T06:20:16.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.493+0000 7fe68f400640 1 -- 172.21.15.16:0/415170087 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe698004e10 con 0x55df406f0350 2024-08-31T06:20:16.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.494+0000 7fe6aba49740 1 -- 172.21.15.16:0/415170087 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df406f0350 msgr2=0x55df40a96600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:16.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.494+0000 7fe6aba49740 1 --2- 172.21.15.16:0/415170087 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df406f0350 0x55df40a96600 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe69800ba00 tx=0x7fe69800bec0 comp rx=0 tx=0).stop 2024-08-31T06:20:16.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.494+0000 7fe696a00640 1 --2- 172.21.15.16:0/1867418844 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe6800b4eb0 0x7fe6800b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:16.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.495+0000 7fe696a00640 1 -- 172.21.15.16:0/1867418844 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8470+0+0 (secure 0 0 0) 0x7fe690104760 con 0x7fe6a010b470 2024-08-31T06:20:16.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.496+0000 7fe6aba49740 1 -- 172.21.15.16:0/415170087 shutdown_connections 2024-08-31T06:20:16.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.496+0000 7fe6aba49740 1 --2- 172.21.15.16:0/415170087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df406c7490 0x55df40a96b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:16.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.496+0000 7fe6aba49740 1 --2- 172.21.15.16:0/415170087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df406cf670 0x55df40a97080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:16.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.497+0000 7fe6aba49740 1 --2- 172.21.15.16:0/415170087 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df406f0350 0x55df40a96600 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:16.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.497+0000 7fe6aba49740 1 -- 172.21.15.16:0/415170087 >> 172.21.15.16:0/415170087 conn(0x55df406e4740 msgr2=0x55df40a944f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:16.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.497+0000 7fe69e800640 1 --2- 172.21.15.16:0/1867418844 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe6800b4eb0 0x7fe6800b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:16.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.497+0000 7fe6aba49740 1 -- 172.21.15.16:0/415170087 shutdown_connections 2024-08-31T06:20:16.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.497+0000 7fe6aba49740 1 -- 172.21.15.16:0/415170087 wait complete. 2024-08-31T06:20:16.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.499+0000 7fe6aba49740 1 -- start start 2024-08-31T06:20:16.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.499+0000 7fe6aba49740 1 Processor -- start 2024-08-31T06:20:16.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.502+0000 7fe69e800640 1 --2- 172.21.15.16:0/1867418844 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe6800b4eb0 0x7fe6800b73c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe6980057c0 tx=0x7fe698005850 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:16.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.504+0000 7fe6aba49740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df406c7490 0x55df40aa21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:16.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.504+0000 7fe6aba49740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df406cf670 0x55df40aa2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:16.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.504+0000 7fe6aba49740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df406f0350 0x55df40aae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:16.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.504+0000 7fe6aba49740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55df4085e890 con 0x55df406cf670 2024-08-31T06:20:16.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.504+0000 7fe6aba49740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55df40982cd0 con 0x55df406f0350 2024-08-31T06:20:16.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.504+0000 7fe6aba49740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55df40aa4de0 con 0x55df406c7490 2024-08-31T06:20:16.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.504+0000 7fe6aba49740 10 client.0 _ll_register_callbacks cb 0x55df409f6ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:20:16.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.505+0000 7fe69d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df406cf670 0x55df40aa2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:16.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.505+0000 7fe69d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df406cf670 0x55df40aa2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51422/0 (socket says 172.21.15.16:51422) 2024-08-31T06:20:16.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.505+0000 7fe69d400640 1 -- 172.21.15.16:0/3334885507 learned_addr learned my addr 172.21.15.16:0/3334885507 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:16.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.505+0000 7fe69e800640 1 --2- 172.21.15.16:0/3334885507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df406f0350 0x55df40aae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:16.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.505+0000 7fe69de00640 1 --2- 172.21.15.16:0/3334885507 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df406c7490 0x55df40aa21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:16.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.506+0000 7fe69e800640 1 -- 172.21.15.16:0/3334885507 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df406c7490 msgr2=0x55df40aa21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:16.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.506+0000 7fe69e800640 1 --2- 172.21.15.16:0/3334885507 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df406c7490 0x55df40aa21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:16.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.506+0000 7fe69e800640 1 -- 172.21.15.16:0/3334885507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df406cf670 msgr2=0x55df40aa2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:16.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.506+0000 7fe69e800640 1 --2- 172.21.15.16:0/3334885507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df406cf670 0x55df40aa2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:16.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.506+0000 7fe69e800640 1 -- 172.21.15.16:0/3334885507 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df409cc1b0 con 0x55df406f0350 2024-08-31T06:20:16.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.506+0000 7fe69d400640 1 --2- 172.21.15.16:0/3334885507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df406cf670 0x55df40aa2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:20:16.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.506+0000 7fe69e800640 1 --2- 172.21.15.16:0/3334885507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df406f0350 0x55df40aae0d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe69800b9d0 tx=0x7fe6980048d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=309d375580c37461 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:16.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.506+0000 7fe687e00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:20:16.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.507+0000 7fe687e00640 1 -- 172.21.15.16:0/3334885507 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe698016040 con 0x55df406f0350 2024-08-31T06:20:16.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.507+0000 7fe687e00640 1 -- 172.21.15.16:0/3334885507 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8470+0+0 (secure 0 0 0) 0x7fe698004450 con 0x55df406f0350 2024-08-31T06:20:16.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.507+0000 7fe6aba49740 1 -- 172.21.15.16:0/3334885507 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6980054d0 con 0x55df406f0350 2024-08-31T06:20:16.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.507+0000 7fe687e00640 1 -- 172.21.15.16:0/3334885507 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fe698004df0 con 0x55df406f0350 2024-08-31T06:20:16.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.508+0000 7fe6aba49740 10 client.28526 fetch_fsmap learned FSMap version 49 2024-08-31T06:20:16.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.508+0000 7fe6aba49740 1 -- 172.21.15.16:0/3334885507 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55df40ab0d70 con 0x55df406f0350 2024-08-31T06:20:16.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.508+0000 7fe687e00640 1 -- 172.21.15.16:0/3334885507 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6980054d0 con 0x55df406f0350 2024-08-31T06:20:16.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.509+0000 7fe687e00640 1 -- 172.21.15.16:0/3334885507 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fe698010760 con 0x55df406f0350 2024-08-31T06:20:16.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.509+0000 7fe6aba49740 10 client.28526 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:20:16.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.509+0000 7fe6aba49740 10 client.28526 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:20:16.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.509+0000 7fe6aba49740 20 client.28526 populate_metadata read hostname 'smithi016' 2024-08-31T06:20:16.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.509+0000 7fe6aba49740 1 --2- 172.21.15.16:0/3334885507 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55df406aecd0 0x55df40ab4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:16.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.509+0000 7fe6aba49740 4 client.28526 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:20:16.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.509+0000 7fe6aba49740 1 -- 172.21.15.16:0/3334885507 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df40a0e6a0 con 0x55df406aecd0 2024-08-31T06:20:16.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.510+0000 7fe69de00640 1 --2- 172.21.15.16:0/3334885507 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55df406aecd0 0x55df40ab4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:16.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.510+0000 7fe69de00640 1 --2- 172.21.15.16:0/3334885507 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55df406aecd0 0x55df40ab4130 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60be29c6465a10b6 server_cookie=dc18138b70f97b13 in_seq=0 out_seq=0 2024-08-31T06:20:16.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.511+0000 7fe687e00640 10 client.28526 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:20:16.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.511+0000 7fe687e00640 1 -- 172.21.15.16:0/3334885507 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55df40a0e6a0 con 0x55df406aecd0 2024-08-31T06:20:16.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.511+0000 7fe687e00640 10 client.28526 handle_command_reply: tid=0 2024-08-31T06:20:16.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.511+0000 7fe6aba49740 1 client.28526 shutdown 2024-08-31T06:20:16.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.512+0000 7fe6aba49740 1 -- 172.21.15.16:0/3334885507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df406f0350 msgr2=0x55df40aae0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:16.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.512+0000 7fe6aba49740 1 --2- 172.21.15.16:0/3334885507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df406f0350 0x55df40aae0d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe69800b9d0 tx=0x7fe6980048d0 comp rx=0 tx=0).stop 2024-08-31T06:20:16.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.513+0000 7fe6aba49740 1 -- 172.21.15.16:0/3334885507 shutdown_connections 2024-08-31T06:20:16.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.513+0000 7fe6aba49740 1 --2- 172.21.15.16:0/3334885507 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55df406aecd0 0x55df40ab4130 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:16.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.513+0000 7fe687e00640 0 client.28526 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:20:16.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.513+0000 7fe6aba49740 1 --2- 172.21.15.16:0/3334885507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df406f0350 0x55df40aae0d0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:16.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.513+0000 7fe6aba49740 1 --2- 172.21.15.16:0/3334885507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df406cf670 0x55df40aa2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:16.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.513+0000 7fe6aba49740 1 --2- 172.21.15.16:0/3334885507 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df406c7490 0x55df40aa21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:16.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.513+0000 7fe6aba49740 1 -- 172.21.15.16:0/3334885507 >> 172.21.15.16:0/3334885507 conn(0x55df406e4740 msgr2=0x55df40a997e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:16.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.513+0000 7fe6aba49740 1 -- 172.21.15.16:0/3334885507 shutdown_connections 2024-08-31T06:20:16.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.513+0000 7fe6aba49740 1 -- 172.21.15.16:0/3334885507 wait complete. 2024-08-31T06:20:16.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.514+0000 7fe6aba49740 20 client.28526 trim_cache size 0 max 16384 2024-08-31T06:20:16.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.549+0000 7fe6aba49740 1 Processor -- start 2024-08-31T06:20:16.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.549+0000 7fe6aba49740 1 -- start start 2024-08-31T06:20:16.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.550+0000 7fe6aba49740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df406c7490 0x55df40aadfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:16.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.550+0000 7fe6aba49740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df406cf670 0x55df40aae510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:16.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.550+0000 7fe6aba49740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df406f0350 0x55df40a9f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:16.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.550+0000 7fe6aba49740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a01b89c0 con 0x55df406c7490 2024-08-31T06:20:16.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.550+0000 7fe6aba49740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a01b61d0 con 0x55df406cf670 2024-08-31T06:20:16.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.550+0000 7fe6aba49740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a01b4040 con 0x55df406f0350 2024-08-31T06:20:16.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.551+0000 7fe69de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df406c7490 0x55df40aadfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:16.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.551+0000 7fe69de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df406c7490 0x55df40aadfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51432/0 (socket says 172.21.15.16:51432) 2024-08-31T06:20:16.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.551+0000 7fe69de00640 1 -- 172.21.15.16:0/3149606472 learned_addr learned my addr 172.21.15.16:0/3149606472 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:16.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.552+0000 7fe69d400640 1 --2- 172.21.15.16:0/3149606472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df406cf670 0x55df40aae510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:16.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.552+0000 7fe69e800640 1 --2- 172.21.15.16:0/3149606472 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df406f0350 0x55df40a9f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:16.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.552+0000 7fe69de00640 1 -- 172.21.15.16:0/3149606472 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df406f0350 msgr2=0x55df40a9f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:16.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.552+0000 7fe69de00640 1 --2- 172.21.15.16:0/3149606472 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df406f0350 0x55df40a9f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:16.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.552+0000 7fe69de00640 1 -- 172.21.15.16:0/3149606472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df406cf670 msgr2=0x55df40aae510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:16.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.552+0000 7fe69de00640 1 --2- 172.21.15.16:0/3149606472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df406cf670 0x55df40aae510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:16.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.552+0000 7fe69de00640 1 -- 172.21.15.16:0/3149606472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df40a9d890 con 0x55df406c7490 2024-08-31T06:20:16.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.553+0000 7fe69e800640 1 --2- 172.21.15.16:0/3149606472 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df406f0350 0x55df40a9f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:20:16.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.553+0000 7fe69de00640 1 --2- 172.21.15.16:0/3149606472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df406c7490 0x55df40aadfd0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe6900c6f20 tx=0x7fe690106b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=207290cc01386d8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:16.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.554+0000 7fe68ea00640 1 -- 172.21.15.16:0/3149606472 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe690004230 con 0x55df406c7490 2024-08-31T06:20:16.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.554+0000 7fe68ea00640 1 -- 172.21.15.16:0/3149606472 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6900ca9b0 con 0x55df406c7490 2024-08-31T06:20:16.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.554+0000 7fe68ea00640 1 -- 172.21.15.16:0/3149606472 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6900c7130 con 0x55df406c7490 2024-08-31T06:20:16.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.556+0000 7fe6aba49740 1 -- 172.21.15.16:0/3149606472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df406c7490 msgr2=0x55df40aadfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:16.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.556+0000 7fe6aba49740 1 --2- 172.21.15.16:0/3149606472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df406c7490 0x55df40aadfd0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe6900c6f20 tx=0x7fe690106b30 comp rx=0 tx=0).stop 2024-08-31T06:20:16.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.557+0000 7fe6aba49740 1 -- 172.21.15.16:0/3149606472 shutdown_connections 2024-08-31T06:20:16.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.557+0000 7fe6aba49740 1 --2- 172.21.15.16:0/3149606472 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df406f0350 0x55df40a9f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:16.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.557+0000 7fe6aba49740 1 --2- 172.21.15.16:0/3149606472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df406cf670 0x55df40aae510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:16.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.557+0000 7fe6aba49740 1 --2- 172.21.15.16:0/3149606472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df406c7490 0x55df40aadfd0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:16.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.557+0000 7fe6aba49740 1 -- 172.21.15.16:0/3149606472 >> 172.21.15.16:0/3149606472 conn(0x55df406e4740 msgr2=0x55df40aa0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:16.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.557+0000 7fe6aba49740 1 -- 172.21.15.16:0/3149606472 shutdown_connections 2024-08-31T06:20:16.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.557+0000 7fe6aba49740 1 -- 172.21.15.16:0/3149606472 wait complete. 2024-08-31T06:20:16.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.558+0000 7fe6aba49740 1 -- start start 2024-08-31T06:20:16.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.558+0000 7fe6aba49740 1 Processor -- start 2024-08-31T06:20:16.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.560+0000 7fe6aba49740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df406c7490 0x55df40aa6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:16.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.560+0000 7fe6aba49740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df406cf670 0x55df40aa6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:16.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.560+0000 7fe6aba49740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df406f0350 0x55df40b1daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:16.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.560+0000 7fe6aba49740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a01b4690 con 0x55df406c7490 2024-08-31T06:20:16.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.560+0000 7fe6aba49740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a01b5b00 con 0x55df406f0350 2024-08-31T06:20:16.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.560+0000 7fe6aba49740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a01b4d60 con 0x55df406cf670 2024-08-31T06:20:16.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.560+0000 7fe6aba49740 10 client.0 _ll_register_callbacks cb 0x7fe69800b850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:20:16.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.561+0000 7fe69de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df406c7490 0x55df40aa6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:16.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.561+0000 7fe69de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df406c7490 0x55df40aa6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51434/0 (socket says 172.21.15.16:51434) 2024-08-31T06:20:16.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.561+0000 7fe69de00640 1 -- 172.21.15.16:0/3857537617 learned_addr learned my addr 172.21.15.16:0/3857537617 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:16.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.562+0000 7fe69d400640 1 --2- 172.21.15.16:0/3857537617 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df406cf670 0x55df40aa6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:16.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.562+0000 7fe69e800640 1 --2- 172.21.15.16:0/3857537617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df406f0350 0x55df40b1daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:16.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.563+0000 7fe69e800640 1 -- 172.21.15.16:0/3857537617 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df406cf670 msgr2=0x55df40aa6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:16.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.563+0000 7fe69e800640 1 --2- 172.21.15.16:0/3857537617 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df406cf670 0x55df40aa6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:16.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.563+0000 7fe69e800640 1 -- 172.21.15.16:0/3857537617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df406c7490 msgr2=0x55df40aa6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:16.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.563+0000 7fe69e800640 1 --2- 172.21.15.16:0/3857537617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df406c7490 0x55df40aa6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:16.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.563+0000 7fe69e800640 1 -- 172.21.15.16:0/3857537617 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df40b1dfe0 con 0x55df406f0350 2024-08-31T06:20:16.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.563+0000 7fe687e00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:20:16.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.564+0000 7fe69e800640 1 --2- 172.21.15.16:0/3857537617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df406f0350 0x55df40b1daa0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe698000c00 tx=0x7fe6980049c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8c7fb7e7f83dceb server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:16.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.564+0000 7fe687e00640 1 -- 172.21.15.16:0/3857537617 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe698005490 con 0x55df406f0350 2024-08-31T06:20:16.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.564+0000 7fe687e00640 1 -- 172.21.15.16:0/3857537617 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8470+0+0 (secure 0 0 0) 0x7fe69803e060 con 0x55df406f0350 2024-08-31T06:20:16.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.565+0000 7fe6aba49740 1 -- 172.21.15.16:0/3857537617 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6900ca9b0 con 0x55df406f0350 2024-08-31T06:20:16.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.565+0000 7fe687e00640 1 -- 172.21.15.16:0/3857537617 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fe698005490 con 0x55df406f0350 2024-08-31T06:20:16.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.565+0000 7fe6aba49740 10 client.28532 fetch_fsmap learned FSMap version 49 2024-08-31T06:20:16.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.565+0000 7fe6aba49740 1 -- 172.21.15.16:0/3857537617 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55df40b1f1b0 con 0x55df406f0350 2024-08-31T06:20:16.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.566+0000 7fe687e00640 1 -- 172.21.15.16:0/3857537617 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe69803d020 con 0x55df406f0350 2024-08-31T06:20:16.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.566+0000 7fe687e00640 1 -- 172.21.15.16:0/3857537617 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fe698003af0 con 0x55df406f0350 2024-08-31T06:20:16.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.569+0000 7fe6aba49740 10 client.28532 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:20:16.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.569+0000 7fe6aba49740 10 client.28532 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:20:16.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.569+0000 7fe6aba49740 20 client.28532 populate_metadata read hostname 'smithi016' 2024-08-31T06:20:16.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.569+0000 7fe6aba49740 1 --2- 172.21.15.16:0/3857537617 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55df40684080 0x55df40b21960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:16.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.569+0000 7fe6aba49740 4 client.28532 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T06:20:16.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.569+0000 7fe6aba49740 1 -- 172.21.15.16:0/3857537617 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fe6900ca9b0 con 0x55df40684080 2024-08-31T06:20:16.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.570+0000 7fe69de00640 1 --2- 172.21.15.16:0/3857537617 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55df40684080 0x55df40b21960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:16.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.570+0000 7fe69de00640 1 --2- 172.21.15.16:0/3857537617 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55df40684080 0x55df40b21960 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c179cfc24289ab06 server_cookie=d0270dc527592ff2 in_seq=0 out_seq=0 2024-08-31T06:20:16.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.571+0000 7fe687e00640 10 client.28532 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:20:16.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.576+0000 7fe687e00640 1 -- 172.21.15.16:0/3857537617 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fe6900ca9b0 con 0x55df40684080 2024-08-31T06:20:16.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.576+0000 7fe687e00640 10 client.28532 handle_command_reply: tid=0 2024-08-31T06:20:16.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.576+0000 7fe6aba49740 1 client.28532 shutdown 2024-08-31T06:20:16.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.577+0000 7fe6aba49740 1 -- 172.21.15.16:0/3857537617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df406f0350 msgr2=0x55df40b1daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:16.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.577+0000 7fe6aba49740 1 --2- 172.21.15.16:0/3857537617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df406f0350 0x55df40b1daa0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe698000c00 tx=0x7fe6980049c0 comp rx=0 tx=0).stop 2024-08-31T06:20:16.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.578+0000 7fe6aba49740 1 -- 172.21.15.16:0/3857537617 shutdown_connections 2024-08-31T06:20:16.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.578+0000 7fe6aba49740 1 --2- 172.21.15.16:0/3857537617 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55df40684080 0x55df40b21960 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:16.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.578+0000 7fe6aba49740 1 --2- 172.21.15.16:0/3857537617 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df406cf670 0x55df40aa6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:16.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.578+0000 7fe6aba49740 1 --2- 172.21.15.16:0/3857537617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55df406f0350 0x55df40b1daa0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:16.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.578+0000 7fe6aba49740 1 --2- 172.21.15.16:0/3857537617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df406c7490 0x55df40aa6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:16.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.578+0000 7fe6aba49740 1 -- 172.21.15.16:0/3857537617 >> 172.21.15.16:0/3857537617 conn(0x55df406e4740 msgr2=0x55df40aa3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:16.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.579+0000 7fe687e00640 0 client.28532 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:20:16.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.579+0000 7fe6aba49740 1 -- 172.21.15.16:0/3857537617 shutdown_connections 2024-08-31T06:20:16.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.579+0000 7fe6aba49740 1 -- 172.21.15.16:0/3857537617 wait complete. 2024-08-31T06:20:16.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.579+0000 7fe6aba49740 20 client.28532 trim_cache size 0 max 16384 2024-08-31T06:20:16.583 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:20:16.583 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-08-31T06:20:16.583 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "18e112f4-a5f9-4602-aaad-3b13fa5a4c2e", 2024-08-31T06:20:16.583 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-08-31T06:20:16.583 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:20:16.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.581+0000 7fe6a4e00640 1 -- 172.21.15.16:0/1867418844 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe6800b4eb0 msgr2=0x7fe6800b73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:16.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.581+0000 7fe6a4e00640 1 --2- 172.21.15.16:0/1867418844 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe6800b4eb0 0x7fe6800b73c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe6980057c0 tx=0x7fe698005850 comp rx=0 tx=0).stop 2024-08-31T06:20:16.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.581+0000 7fe6a4e00640 1 -- 172.21.15.16:0/1867418844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6a010b470 msgr2=0x7fe6a010fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:16.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.581+0000 7fe6a4e00640 1 --2- 172.21.15.16:0/1867418844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6a010b470 0x7fe6a010fb10 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe69002a8b0 tx=0x7fe690030d80 comp rx=0 tx=0).stop 2024-08-31T06:20:16.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.581+0000 7fe6a4e00640 1 -- 172.21.15.16:0/1867418844 shutdown_connections 2024-08-31T06:20:16.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.581+0000 7fe6a4e00640 1 --2- 172.21.15.16:0/1867418844 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe6800b4eb0 0x7fe6800b73c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:16.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.581+0000 7fe6a4e00640 1 --2- 172.21.15.16:0/1867418844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6a0110590 0x7fe6a01ac7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:16.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.581+0000 7fe6a4e00640 1 --2- 172.21.15.16:0/1867418844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6a010bdc0 0x7fe6a0110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:16.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.581+0000 7fe6a4e00640 1 --2- 172.21.15.16:0/1867418844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6a010b470 0x7fe6a010fb10 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:16.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.581+0000 7fe6a4e00640 1 -- 172.21.15.16:0/1867418844 >> 172.21.15.16:0/1867418844 conn(0x7fe6a0101600 msgr2=0x7fe6a0103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:16.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.582+0000 7fe6a4e00640 1 -- 172.21.15.16:0/1867418844 shutdown_connections 2024-08-31T06:20:16.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:16.582+0000 7fe6a4e00640 1 -- 172.21.15.16:0/1867418844 wait complete. 2024-08-31T06:20:16.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 770.0 s, 1453.4 tps, lat 69.729 ms stddev 80.287 2024-08-31T06:20:17.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:16 smithi155 ceph-mon[21742]: pgmap v770: 97 pgs: 97 active+clean; 3.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 15 MiB/s wr, 1.38k op/s 2024-08-31T06:20:17.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:16 smithi155 ceph-mon[21742]: osdmap e97: 12 total, 12 up, 12 in 2024-08-31T06:20:17.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:16 smithi016 ceph-mon[17194]: pgmap v770: 97 pgs: 97 active+clean; 3.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 15 MiB/s wr, 1.38k op/s 2024-08-31T06:20:17.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:16 smithi016 ceph-mon[17194]: osdmap e97: 12 total, 12 up, 12 in 2024-08-31T06:20:17.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:16 smithi184 ceph-mon[22331]: pgmap v770: 97 pgs: 97 active+clean; 3.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 15 MiB/s wr, 1.38k op/s 2024-08-31T06:20:17.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:16 smithi184 ceph-mon[22331]: osdmap e97: 12 total, 12 up, 12 in 2024-08-31T06:20:17.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:17.506+0000 7f0366000640 1 -- 172.21.15.16:0/3105064944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f036010ac80 msgr2=0x7f0360115d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:17.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:17.506+0000 7f0366000640 1 --2- 172.21.15.16:0/3105064944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f036010ac80 0x7f0360115d70 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0350009900 tx=0x7f035002a440 comp rx=0 tx=0).stop 2024-08-31T06:20:17.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:17.507+0000 7f0366000640 1 -- 172.21.15.16:0/3105064944 shutdown_connections 2024-08-31T06:20:17.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:17.507+0000 7f0366000640 1 --2- 172.21.15.16:0/3105064944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f036010ac80 0x7f0360115d70 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:17.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:17.507+0000 7f0366000640 1 --2- 172.21.15.16:0/3105064944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03601163e0 0x7f03601188f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:17.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:17.507+0000 7f0366000640 1 --2- 172.21.15.16:0/3105064944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f036010a330 0x7f036010a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:17.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:17.507+0000 7f0366000640 1 -- 172.21.15.16:0/3105064944 >> 172.21.15.16:0/3105064944 conn(0x7f0360070c60 msgr2=0x7f0360071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:17.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:17.508+0000 7f0366000640 1 -- 172.21.15.16:0/3105064944 shutdown_connections 2024-08-31T06:20:17.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:17.508+0000 7f0366000640 1 -- 172.21.15.16:0/3105064944 wait complete. 2024-08-31T06:20:17.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:17.510+0000 7f0366000640 1 Processor -- start 2024-08-31T06:20:17.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:17.511+0000 7f0366000640 1 -- start start 2024-08-31T06:20:17.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:17.512+0000 7f0366000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f036010a330 0x7f036010bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:17.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:17.512+0000 7f0366000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f036010ac80 0x7f036010c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:17.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:17.512+0000 7f0366000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03601163e0 0x7f036010c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:17.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:17.513+0000 7f0366000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f036011c0f0 con 0x7f03601163e0 2024-08-31T06:20:17.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:17.513+0000 7f0366000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f036011bf70 con 0x7f036010a330 2024-08-31T06:20:17.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:17.513+0000 7f035fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03601163e0 0x7f036010c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:17.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:17.513+0000 7f0366000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f036011c270 con 0x7f036010ac80 2024-08-31T06:20:17.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:17.513+0000 7f035ea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f036010ac80 0x7f036010c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:17.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:17.514+0000 7f035ea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f036010ac80 0x7f036010c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:57398/0 (socket says 172.21.15.16:57398) 2024-08-31T06:20:17.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:17.514+0000 7f035ea00640 1 -- 172.21.15.16:0/4110264099 learned_addr learned my addr 172.21.15.16:0/4110264099 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:17.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:17.515+0000 7f035f400640 1 --2- 172.21.15.16:0/4110264099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f036010a330 0x7f036010bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:17.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:17.515+0000 7f035fe00640 1 -- 172.21.15.16:0/4110264099 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f036010ac80 msgr2=0x7f036010c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:17.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:17.515+0000 7f035fe00640 1 --2- 172.21.15.16:0/4110264099 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f036010ac80 0x7f036010c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:17.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:17.515+0000 7f035fe00640 1 -- 172.21.15.16:0/4110264099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f036010a330 msgr2=0x7f036010bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:17.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:17.515+0000 7f035fe00640 1 --2- 172.21.15.16:0/4110264099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f036010a330 0x7f036010bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:17.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:17.515+0000 7f035fe00640 1 -- 172.21.15.16:0/4110264099 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03601a6960 con 0x7f03601163e0 2024-08-31T06:20:17.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:17.519+0000 7f035fe00640 1 --2- 172.21.15.16:0/4110264099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03601163e0 0x7f036010c6a0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f034c00e970 tx=0x7f034c00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:17.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:17.520+0000 7f0357e00640 1 -- 172.21.15.16:0/4110264099 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034c00ccc0 con 0x7f03601163e0 2024-08-31T06:20:17.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:17.523+0000 7f0366000640 1 -- 172.21.15.16:0/4110264099 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03601b54e0 con 0x7f03601163e0 2024-08-31T06:20:17.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:17.523+0000 7f0366000640 1 -- 172.21.15.16:0/4110264099 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03601b59f0 con 0x7f03601163e0 2024-08-31T06:20:17.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:17.523+0000 7f0357e00640 1 -- 172.21.15.16:0/4110264099 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f034c005020 con 0x7f03601163e0 2024-08-31T06:20:17.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:17.523+0000 7f0357e00640 1 -- 172.21.15.16:0/4110264099 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034c00ccc0 con 0x7f03601163e0 2024-08-31T06:20:17.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:17.527+0000 7f0366000640 1 -- 172.21.15.16:0/4110264099 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0340005290 con 0x7f03601163e0 2024-08-31T06:20:17.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:17.538+0000 7f0357e00640 1 -- 172.21.15.16:0/4110264099 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f034c0051d0 con 0x7f03601163e0 2024-08-31T06:20:17.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:17.544+0000 7f0357e00640 1 --2- 172.21.15.16:0/4110264099 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f03440b4fa0 0x7f03440b74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:17.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:17.548+0000 7f035f400640 1 --2- 172.21.15.16:0/4110264099 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f03440b4fa0 0x7f03440b74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:17.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:17.549+0000 7f0357e00640 1 -- 172.21.15.16:0/4110264099 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8470+0+0 (secure 0 0 0) 0x7f034c0e4ec0 con 0x7f03601163e0 2024-08-31T06:20:17.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:17.551+0000 7f035f400640 1 --2- 172.21.15.16:0/4110264099 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f03440b4fa0 0x7f03440b74b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f036010d670 tx=0x7f035800d040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:17.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:17.554+0000 7f0357e00640 1 -- 172.21.15.16:0/4110264099 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f034c0abc80 con 0x7f03601163e0 2024-08-31T06:20:18.006 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T06:20:18.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:18.038+0000 7f0366000640 1 -- 172.21.15.16:0/4110264099 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0340005850 con 0x7f03601163e0 2024-08-31T06:20:18.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:18.042+0000 7f0357e00640 1 -- 172.21.15.16:0/4110264099 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+11923 (secure 0 0 0) 0x7f034c0ab580 con 0x7f03601163e0 2024-08-31T06:20:18.045 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:20:18.045 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":49,"btime":"2024-08-31T06:19:17:811356+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":49,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:19:17.033815+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24737,24724,24712]},"id":1}]} 2024-08-31T06:20:18.045 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 49 2024-08-31T06:20:18.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:18.047+0000 7f0366000640 1 -- 172.21.15.16:0/4110264099 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f03440b4fa0 msgr2=0x7f03440b74b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:18.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:18.047+0000 7f0366000640 1 --2- 172.21.15.16:0/4110264099 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f03440b4fa0 0x7f03440b74b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f036010d670 tx=0x7f035800d040 comp rx=0 tx=0).stop 2024-08-31T06:20:18.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:18.047+0000 7f0366000640 1 -- 172.21.15.16:0/4110264099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03601163e0 msgr2=0x7f036010c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:18.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:18.047+0000 7f0366000640 1 --2- 172.21.15.16:0/4110264099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03601163e0 0x7f036010c6a0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f034c00e970 tx=0x7f034c00ee30 comp rx=0 tx=0).stop 2024-08-31T06:20:18.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:18.050+0000 7f0366000640 1 -- 172.21.15.16:0/4110264099 shutdown_connections 2024-08-31T06:20:18.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:18.050+0000 7f0366000640 1 --2- 172.21.15.16:0/4110264099 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f03440b4fa0 0x7f03440b74b0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:18.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:18.050+0000 7f0366000640 1 --2- 172.21.15.16:0/4110264099 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f036010ac80 0x7f036010c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:18.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:18.050+0000 7f0366000640 1 --2- 172.21.15.16:0/4110264099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03601163e0 0x7f036010c6a0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:18.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:18.050+0000 7f0366000640 1 --2- 172.21.15.16:0/4110264099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f036010a330 0x7f036010bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:18.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:18.050+0000 7f0366000640 1 -- 172.21.15.16:0/4110264099 >> 172.21.15.16:0/4110264099 conn(0x7f0360070c60 msgr2=0x7f0360071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:18.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:18.050+0000 7f0366000640 1 -- 172.21.15.16:0/4110264099 shutdown_connections 2024-08-31T06:20:18.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:18.052+0000 7f0366000640 1 -- 172.21.15.16:0/4110264099 wait complete. 2024-08-31T06:20:18.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:17 smithi155 ceph-mon[21742]: scrub queued for path: / 2024-08-31T06:20:18.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:17 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:20:18.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:17 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:20:18.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:17 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:20:18.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:17 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:20:18.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:17 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:20:18.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:17 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:20:18.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:17 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:20:18.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:17 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:20:18.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:17 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:20:18.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:17 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:20:18.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:17 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:20:18.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:17 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:20:18.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:17 smithi016 ceph-mon[17194]: scrub queued for path: / 2024-08-31T06:20:18.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:17 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:20:18.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:17 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:20:18.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:17 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:20:18.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:17 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:20:18.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:17 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:20:18.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:17 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:20:18.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:17 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:20:18.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:17 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:20:18.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:17 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:20:18.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:17 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:20:18.165 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:17 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:20:18.165 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:17 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:20:18.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:17 smithi184 ceph-mon[22331]: scrub queued for path: / 2024-08-31T06:20:18.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:17 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:20:18.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:17 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:20:18.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:17 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:20:18.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:17 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:20:18.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:17 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:20:18.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:17 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:20:18.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:17 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:20:18.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:17 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:20:18.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:17 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:20:18.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:17 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:20:18.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:17 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:20:18.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:17 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:20:18.946 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24737 quiesce db / --timeout 90 --expiration 134.7 --await-for 45.0' 2024-08-31T06:20:18.947 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24737 quiesce db / --timeout 90 --expiration 134.7 --await-for 45.0 2024-08-31T06:20:19.045 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:20:19.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:18 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:20:19.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:18 smithi155 ceph-mon[21742]: pgmap v772: 97 pgs: 97 active+clean; 3.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 5.4 MiB/s wr, 609 op/s 2024-08-31T06:20:19.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:18 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/4110264099' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:20:19.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:18 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:20:19.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:18 smithi016 ceph-mon[17194]: pgmap v772: 97 pgs: 97 active+clean; 3.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 5.4 MiB/s wr, 609 op/s 2024-08-31T06:20:19.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:18 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/4110264099' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:20:19.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:18 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:20:19.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:18 smithi184 ceph-mon[22331]: pgmap v772: 97 pgs: 97 active+clean; 3.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 5.4 MiB/s wr, 609 op/s 2024-08-31T06:20:19.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:18 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/4110264099' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:20:19.901 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:20:19.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.953+0000 7ff1fae00640 1 -- 172.21.15.16:0/2220658078 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1f4109d60 msgr2=0x7ff1f410a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:19.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.953+0000 7ff1fae00640 1 --2- 172.21.15.16:0/2220658078 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1f4109d60 0x7ff1f410a170 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009900 tx=0x7ff1e002a410 comp rx=0 tx=0).stop 2024-08-31T06:20:19.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.954+0000 7ff1fae00640 1 -- 172.21.15.16:0/2220658078 shutdown_connections 2024-08-31T06:20:19.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.954+0000 7ff1fae00640 1 --2- 172.21.15.16:0/2220658078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1f410fff0 0x7ff1f4116820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:19.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.954+0000 7ff1fae00640 1 --2- 172.21.15.16:0/2220658078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1f410f6a0 0x7ff1f410fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:19.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.954+0000 7ff1fae00640 1 --2- 172.21.15.16:0/2220658078 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1f4109d60 0x7ff1f410a170 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:19.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.954+0000 7ff1fae00640 1 -- 172.21.15.16:0/2220658078 >> 172.21.15.16:0/2220658078 conn(0x7ff1f4079360 msgr2=0x7ff1f4107f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:19.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.954+0000 7ff1fae00640 1 -- 172.21.15.16:0/2220658078 shutdown_connections 2024-08-31T06:20:19.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.954+0000 7ff1fae00640 1 -- 172.21.15.16:0/2220658078 wait complete. 2024-08-31T06:20:19.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.957+0000 7ff1fae00640 1 Processor -- start 2024-08-31T06:20:19.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.957+0000 7ff1fae00640 1 -- start start 2024-08-31T06:20:19.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.958+0000 7ff1fae00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1f4109d60 0x7ff1f41a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:19.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.958+0000 7ff1fae00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1f410f6a0 0x7ff1f41a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:19.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.958+0000 7ff1fae00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1f410fff0 0x7ff1f41a5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:19.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.958+0000 7ff1fae00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f4119d40 con 0x7ff1f410fff0 2024-08-31T06:20:19.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.958+0000 7ff1fae00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f4119bc0 con 0x7ff1f4109d60 2024-08-31T06:20:19.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.958+0000 7ff1fae00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f4119ec0 con 0x7ff1f410f6a0 2024-08-31T06:20:19.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.958+0000 7ff1f9a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1f4109d60 0x7ff1f41a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:19.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.958+0000 7ff1f9a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1f4109d60 0x7ff1f41a0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:41568/0 (socket says 172.21.15.16:41568) 2024-08-31T06:20:19.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.958+0000 7ff1f9a00640 1 -- 172.21.15.16:0/2674427317 learned_addr learned my addr 172.21.15.16:0/2674427317 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:19.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.958+0000 7ff1fa400640 1 --2- 172.21.15.16:0/2674427317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1f410fff0 0x7ff1f41a5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:19.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.958+0000 7ff1f9000640 1 --2- 172.21.15.16:0/2674427317 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1f410f6a0 0x7ff1f41a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:19.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.959+0000 7ff1f9a00640 1 -- 172.21.15.16:0/2674427317 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1f410f6a0 msgr2=0x7ff1f41a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:19.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.959+0000 7ff1f9a00640 1 --2- 172.21.15.16:0/2674427317 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1f410f6a0 0x7ff1f41a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:19.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.959+0000 7ff1f9a00640 1 -- 172.21.15.16:0/2674427317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1f410fff0 msgr2=0x7ff1f41a5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:19.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.959+0000 7ff1f9a00640 1 --2- 172.21.15.16:0/2674427317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1f410fff0 0x7ff1f41a5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:19.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.959+0000 7ff1f9a00640 1 -- 172.21.15.16:0/2674427317 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f41a65e0 con 0x7ff1f4109d60 2024-08-31T06:20:19.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.960+0000 7ff1f9a00640 1 --2- 172.21.15.16:0/2674427317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1f4109d60 0x7ff1f41a0730 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0005ab0 tx=0x7ff1e0003d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:19.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.960+0000 7ff1f2000640 1 -- 172.21.15.16:0/2674427317 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e003f070 con 0x7ff1f4109d60 2024-08-31T06:20:19.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.960+0000 7ff1f2000640 1 -- 172.21.15.16:0/2674427317 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1e0031d60 con 0x7ff1f4109d60 2024-08-31T06:20:19.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.960+0000 7ff1f2000640 1 -- 172.21.15.16:0/2674427317 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0035030 con 0x7ff1f4109d60 2024-08-31T06:20:19.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.960+0000 7ff1fae00640 1 -- 172.21.15.16:0/2674427317 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f41a6870 con 0x7ff1f4109d60 2024-08-31T06:20:19.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.961+0000 7ff1fae00640 1 -- 172.21.15.16:0/2674427317 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f41a6d80 con 0x7ff1f4109d60 2024-08-31T06:20:19.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.964+0000 7ff203078740 1 Processor -- start 2024-08-31T06:20:19.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.964+0000 7ff203078740 1 -- start start 2024-08-31T06:20:19.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.965+0000 7ff1f2000640 1 -- 172.21.15.16:0/2674427317 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7ff1e0042070 con 0x7ff1f4109d60 2024-08-31T06:20:19.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.965+0000 7ff203078740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aab4c81070 0x55aab5026a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:19.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.965+0000 7ff203078740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aab4c581b0 0x55aab5026f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:19.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.965+0000 7ff203078740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aab4c60390 0x55aab50274b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:19.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.965+0000 7ff203078740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55aab4f9d770 con 0x55aab4c81070 2024-08-31T06:20:19.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.965+0000 7ff203078740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55aab4f8cd00 con 0x55aab4c60390 2024-08-31T06:20:19.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.965+0000 7ff203078740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55aab4f90cb0 con 0x55aab4c581b0 2024-08-31T06:20:19.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.966+0000 7ff1f9000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aab4c60390 0x55aab50274b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:19.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.966+0000 7ff1fa400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aab4c81070 0x55aab5026a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:19.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.966+0000 7ff1f9000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aab4c60390 0x55aab50274b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:41570/0 (socket says 172.21.15.16:41570) 2024-08-31T06:20:19.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.966+0000 7ff1f9000640 1 -- 172.21.15.16:0/3896887843 learned_addr learned my addr 172.21.15.16:0/3896887843 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:19.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.966+0000 7ff1f9a00640 1 --2- 172.21.15.16:0/3896887843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aab4c581b0 0x55aab5026f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:19.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.968+0000 7ff1f9a00640 1 -- 172.21.15.16:0/3896887843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aab4c60390 msgr2=0x55aab50274b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:19.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.968+0000 7ff1f9a00640 1 --2- 172.21.15.16:0/3896887843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aab4c60390 0x55aab50274b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:19.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.968+0000 7ff1f9a00640 1 -- 172.21.15.16:0/3896887843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aab4c81070 msgr2=0x55aab5026a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:19.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.968+0000 7ff1f9a00640 1 --2- 172.21.15.16:0/3896887843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aab4c81070 0x55aab5026a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:19.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.968+0000 7ff1f9a00640 1 -- 172.21.15.16:0/3896887843 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aab4cec650 con 0x55aab4c581b0 2024-08-31T06:20:19.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.969+0000 7ff1f2000640 1 --2- 172.21.15.16:0/2674427317 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff1dc0b4cd0 0x7ff1dc0b71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:19.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.969+0000 7ff1f9a00640 1 --2- 172.21.15.16:0/3896887843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aab4c581b0 0x55aab5026f70 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0103700 tx=0x7ff1e010e2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b4449ca7380a3b7 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:19.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.969+0000 7ff1f2000640 1 -- 172.21.15.16:0/2674427317 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8470+0+0 (secure 0 0 0) 0x7ff1e0105f80 con 0x7ff1f4109d60 2024-08-31T06:20:19.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.969+0000 7ff1e6a00640 1 -- 172.21.15.16:0/3896887843 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0049070 con 0x55aab4c581b0 2024-08-31T06:20:19.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.970+0000 7ff1fa400640 1 --2- 172.21.15.16:0/2674427317 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff1dc0b4cd0 0x7ff1dc0b71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:19.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.970+0000 7ff1e6a00640 1 -- 172.21.15.16:0/3896887843 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1e00cc1b0 con 0x55aab4c581b0 2024-08-31T06:20:19.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.970+0000 7ff1e6a00640 1 -- 172.21.15.16:0/3896887843 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e00cc470 con 0x55aab4c581b0 2024-08-31T06:20:19.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.971+0000 7ff203078740 1 -- 172.21.15.16:0/3896887843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aab4c581b0 msgr2=0x55aab5026f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:19.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.971+0000 7ff203078740 1 --2- 172.21.15.16:0/3896887843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aab4c581b0 0x55aab5026f70 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0103700 tx=0x7ff1e010e2a0 comp rx=0 tx=0).stop 2024-08-31T06:20:19.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.971+0000 7ff203078740 1 -- 172.21.15.16:0/3896887843 shutdown_connections 2024-08-31T06:20:19.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.971+0000 7ff203078740 1 --2- 172.21.15.16:0/3896887843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aab4c581b0 0x55aab5026f70 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:19.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.971+0000 7ff203078740 1 --2- 172.21.15.16:0/3896887843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aab4c60390 0x55aab50274b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:19.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.971+0000 7ff203078740 1 --2- 172.21.15.16:0/3896887843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aab4c81070 0x55aab5026a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:19.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.971+0000 7ff203078740 1 -- 172.21.15.16:0/3896887843 >> 172.21.15.16:0/3896887843 conn(0x55aab4c75460 msgr2=0x55aab5024920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:19.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.972+0000 7ff203078740 1 -- 172.21.15.16:0/3896887843 shutdown_connections 2024-08-31T06:20:19.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.972+0000 7ff203078740 1 -- 172.21.15.16:0/3896887843 wait complete. 2024-08-31T06:20:19.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.972+0000 7ff1fa400640 1 --2- 172.21.15.16:0/2674427317 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff1dc0b4cd0 0x7ff1dc0b71e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec007ed0 tx=0x7ff1ec007520 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:19.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.973+0000 7ff203078740 1 -- start start 2024-08-31T06:20:19.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.973+0000 7ff203078740 1 Processor -- start 2024-08-31T06:20:19.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.974+0000 7ff203078740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aab4c581b0 0x55aab50326e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:19.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.974+0000 7ff203078740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aab4c60390 0x55aab5032ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:19.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.974+0000 7ff203078740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aab4c81070 0x55aab5038180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:19.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.974+0000 7ff203078740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55aab4def700 con 0x55aab4c60390 2024-08-31T06:20:19.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.974+0000 7ff203078740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55aab4f13cb0 con 0x55aab4c81070 2024-08-31T06:20:19.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.974+0000 7ff203078740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55aab5034fd0 con 0x55aab4c581b0 2024-08-31T06:20:19.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.974+0000 7ff203078740 10 client.0 _ll_register_callbacks cb 0x55aab4f39430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:20:19.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.975+0000 7ff1fa400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aab4c81070 0x55aab5038180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:19.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.975+0000 7ff1fa400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aab4c81070 0x55aab5038180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:41574/0 (socket says 172.21.15.16:41574) 2024-08-31T06:20:19.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.975+0000 7ff1fa400640 1 -- 172.21.15.16:0/3986297703 learned_addr learned my addr 172.21.15.16:0/3986297703 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:19.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.975+0000 7ff1f9000640 1 --2- 172.21.15.16:0/3986297703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aab4c60390 0x55aab5032ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:19.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.976+0000 7ff1fa400640 1 -- 172.21.15.16:0/3986297703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aab4c581b0 msgr2=0x55aab50326e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:19.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.976+0000 7ff1f9a00640 1 --2- 172.21.15.16:0/3986297703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aab4c581b0 0x55aab50326e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:19.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.976+0000 7ff1fa400640 1 --2- 172.21.15.16:0/3986297703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aab4c581b0 0x55aab50326e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:19.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.976+0000 7ff1fa400640 1 -- 172.21.15.16:0/3986297703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aab4c60390 msgr2=0x55aab5032ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:19.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.976+0000 7ff1fa400640 1 --2- 172.21.15.16:0/3986297703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aab4c60390 0x55aab5032ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:19.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.976+0000 7ff1fa400640 1 -- 172.21.15.16:0/3986297703 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aab4f54870 con 0x55aab4c81070 2024-08-31T06:20:19.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.977+0000 7ff1f9000640 1 --2- 172.21.15.16:0/3986297703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aab4c60390 0x55aab5032ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:20:19.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.977+0000 7ff1fa400640 1 --2- 172.21.15.16:0/3986297703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aab4c81070 0x55aab5038180 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec0049b0 tx=0x7ff1ec004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27f0370ff79dc6e4 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:19.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.977+0000 7ff1db400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:20:19.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.978+0000 7ff1db400640 1 -- 172.21.15.16:0/3986297703 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec004d20 con 0x55aab4c81070 2024-08-31T06:20:19.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.978+0000 7ff1db400640 1 -- 172.21.15.16:0/3986297703 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8470+0+0 (secure 0 0 0) 0x7ff1ec005020 con 0x55aab4c81070 2024-08-31T06:20:19.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.978+0000 7ff203078740 1 -- 172.21.15.16:0/3986297703 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1e00cc1b0 con 0x55aab4c81070 2024-08-31T06:20:19.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.978+0000 7ff1db400640 1 -- 172.21.15.16:0/3986297703 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7ff1ec005220 con 0x55aab4c81070 2024-08-31T06:20:19.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.979+0000 7ff203078740 10 client.28550 fetch_fsmap learned FSMap version 49 2024-08-31T06:20:19.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.979+0000 7ff203078740 1 -- 172.21.15.16:0/3986297703 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55aab5052050 con 0x55aab4c81070 2024-08-31T06:20:19.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.979+0000 7ff1db400640 1 -- 172.21.15.16:0/3986297703 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1e00cc1b0 con 0x55aab4c81070 2024-08-31T06:20:19.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.979+0000 7ff1db400640 1 -- 172.21.15.16:0/3986297703 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7ff1ec030830 con 0x55aab4c81070 2024-08-31T06:20:19.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.981+0000 7ff203078740 10 client.28550 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:20:19.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.981+0000 7ff203078740 10 client.28550 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:20:19.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.981+0000 7ff203078740 20 client.28550 populate_metadata read hostname 'smithi016' 2024-08-31T06:20:19.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.981+0000 7ff203078740 1 --2- 172.21.15.16:0/3986297703 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55aab4c3f9f0 0x55aab5055660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:19.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.981+0000 7ff203078740 4 client.28550 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:20:19.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.981+0000 7ff203078740 1 -- 172.21.15.16:0/3986297703 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aab4f9f6c0 con 0x55aab4c3f9f0 2024-08-31T06:20:19.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.982+0000 7ff1f9a00640 1 --2- 172.21.15.16:0/3986297703 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55aab4c3f9f0 0x55aab5055660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:19.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.982+0000 7ff1f9a00640 1 --2- 172.21.15.16:0/3986297703 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55aab4c3f9f0 0x55aab5055660 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfa114e152423f70 server_cookie=670c637aa07e22db in_seq=0 out_seq=0 2024-08-31T06:20:19.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.982+0000 7ff1db400640 10 client.28550 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:20:19.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.983+0000 7ff1db400640 1 -- 172.21.15.16:0/3986297703 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55aab4f9f6c0 con 0x55aab4c3f9f0 2024-08-31T06:20:19.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.983+0000 7ff1db400640 10 client.28550 handle_command_reply: tid=0 2024-08-31T06:20:19.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.983+0000 7ff203078740 1 client.28550 shutdown 2024-08-31T06:20:19.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.983+0000 7ff203078740 1 -- 172.21.15.16:0/3986297703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aab4c81070 msgr2=0x55aab5038180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:19.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.983+0000 7ff203078740 1 --2- 172.21.15.16:0/3986297703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aab4c81070 0x55aab5038180 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec0049b0 tx=0x7ff1ec004a40 comp rx=0 tx=0).stop 2024-08-31T06:20:19.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.984+0000 7ff203078740 1 -- 172.21.15.16:0/3986297703 shutdown_connections 2024-08-31T06:20:19.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.984+0000 7ff203078740 1 --2- 172.21.15.16:0/3986297703 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55aab4c3f9f0 0x55aab5055660 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:19.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.984+0000 7ff203078740 1 --2- 172.21.15.16:0/3986297703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aab4c81070 0x55aab5038180 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:19.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.984+0000 7ff203078740 1 --2- 172.21.15.16:0/3986297703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aab4c60390 0x55aab5032ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:19.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.984+0000 7ff203078740 1 --2- 172.21.15.16:0/3986297703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aab4c581b0 0x55aab50326e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:19.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.984+0000 7ff203078740 1 -- 172.21.15.16:0/3986297703 >> 172.21.15.16:0/3986297703 conn(0x55aab4c75460 msgr2=0x55aab5027a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:19.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.984+0000 7ff1db400640 0 client.28550 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:20:19.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.984+0000 7ff203078740 1 -- 172.21.15.16:0/3986297703 shutdown_connections 2024-08-31T06:20:19.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.985+0000 7ff203078740 1 -- 172.21.15.16:0/3986297703 wait complete. 2024-08-31T06:20:19.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:19.985+0000 7ff203078740 20 client.28550 trim_cache size 0 max 16384 2024-08-31T06:20:20.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.011+0000 7ff203078740 1 Processor -- start 2024-08-31T06:20:20.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.011+0000 7ff203078740 1 -- start start 2024-08-31T06:20:20.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.012+0000 7ff203078740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aab4c581b0 0x55aab5038170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:20.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.012+0000 7ff203078740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aab4c60390 0x55aab5030d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:20.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.012+0000 7ff203078740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aab4c81070 0x55aab50312a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:20.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.012+0000 7ff203078740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f41afa50 con 0x55aab4c81070 2024-08-31T06:20:20.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.012+0000 7ff203078740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f41ad260 con 0x55aab4c60390 2024-08-31T06:20:20.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.012+0000 7ff203078740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f4073200 con 0x55aab4c581b0 2024-08-31T06:20:20.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.012+0000 7ff1f9000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aab4c60390 0x55aab5030d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:20.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.012+0000 7ff1f9000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aab4c60390 0x55aab5030d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:41580/0 (socket says 172.21.15.16:41580) 2024-08-31T06:20:20.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.012+0000 7ff1f9000640 1 -- 172.21.15.16:0/3732922012 learned_addr learned my addr 172.21.15.16:0/3732922012 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:20.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.013+0000 7ff1f9a00640 1 --2- 172.21.15.16:0/3732922012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aab4c581b0 0x55aab5038170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:20.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.013+0000 7ff1f9000640 1 -- 172.21.15.16:0/3732922012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aab4c581b0 msgr2=0x55aab5038170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:20.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.013+0000 7ff1f9000640 1 --2- 172.21.15.16:0/3732922012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aab4c581b0 0x55aab5038170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.013+0000 7ff1f9000640 1 -- 172.21.15.16:0/3732922012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aab4c81070 msgr2=0x55aab50312a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:20.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.013+0000 7ff1f9000640 1 --2- 172.21.15.16:0/3732922012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aab4c81070 0x55aab50312a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.013+0000 7ff1f9000640 1 -- 172.21.15.16:0/3732922012 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aab50309e0 con 0x55aab4c60390 2024-08-31T06:20:20.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.013+0000 7ff1f9000640 1 --2- 172.21.15.16:0/3732922012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aab4c60390 0x55aab5030d60 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff1e800b940 tx=0x7ff1e800be00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=835a964d531b2b89 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:20.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.014+0000 7ff1e6000640 1 -- 172.21.15.16:0/3732922012 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e8019070 con 0x55aab4c60390 2024-08-31T06:20:20.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.014+0000 7ff1e6000640 1 -- 172.21.15.16:0/3732922012 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1e8011d30 con 0x55aab4c60390 2024-08-31T06:20:20.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.014+0000 7ff1e6000640 1 -- 172.21.15.16:0/3732922012 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e8015dc0 con 0x55aab4c60390 2024-08-31T06:20:20.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.015+0000 7ff203078740 1 -- 172.21.15.16:0/3732922012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aab4c60390 msgr2=0x55aab5030d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:20.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.015+0000 7ff203078740 1 --2- 172.21.15.16:0/3732922012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aab4c60390 0x55aab5030d60 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff1e800b940 tx=0x7ff1e800be00 comp rx=0 tx=0).stop 2024-08-31T06:20:20.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.015+0000 7ff203078740 1 -- 172.21.15.16:0/3732922012 shutdown_connections 2024-08-31T06:20:20.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.015+0000 7ff203078740 1 --2- 172.21.15.16:0/3732922012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aab4c81070 0x55aab50312a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.015+0000 7ff203078740 1 --2- 172.21.15.16:0/3732922012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aab4c60390 0x55aab5030d60 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.015+0000 7ff203078740 1 --2- 172.21.15.16:0/3732922012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aab4c581b0 0x55aab5038170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.015+0000 7ff203078740 1 -- 172.21.15.16:0/3732922012 >> 172.21.15.16:0/3732922012 conn(0x55aab4c75460 msgr2=0x55aab502d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:20.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.015+0000 7ff203078740 1 -- 172.21.15.16:0/3732922012 shutdown_connections 2024-08-31T06:20:20.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.016+0000 7ff203078740 1 -- 172.21.15.16:0/3732922012 wait complete. 2024-08-31T06:20:20.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.016+0000 7ff203078740 1 -- start start 2024-08-31T06:20:20.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.017+0000 7ff203078740 1 Processor -- start 2024-08-31T06:20:20.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.017+0000 7ff203078740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aab4c581b0 0x55aab503a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:20.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.017+0000 7ff203078740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aab4c60390 0x55aab503ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:20.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.017+0000 7ff203078740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aab4c81070 0x55aab50bf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:20.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.017+0000 7ff203078740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f4073890 con 0x55aab4c581b0 2024-08-31T06:20:20.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.017+0000 7ff203078740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f4074d00 con 0x55aab4c60390 2024-08-31T06:20:20.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.017+0000 7ff203078740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f4073f60 con 0x55aab4c81070 2024-08-31T06:20:20.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.017+0000 7ff203078740 10 client.0 _ll_register_callbacks cb 0x7ff1ec00fad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:20:20.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.018+0000 7ff1f9000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aab4c60390 0x55aab503ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:20.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.018+0000 7ff1f9000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aab4c60390 0x55aab503ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:41592/0 (socket says 172.21.15.16:41592) 2024-08-31T06:20:20.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.018+0000 7ff1f9000640 1 -- 172.21.15.16:0/2828138121 learned_addr learned my addr 172.21.15.16:0/2828138121 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:20.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.018+0000 7ff1f9a00640 1 --2- 172.21.15.16:0/2828138121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aab4c581b0 0x55aab503a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:20.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.018+0000 7ff1fa400640 1 --2- 172.21.15.16:0/2828138121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aab4c81070 0x55aab50bf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:20.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.018+0000 7ff1f9000640 1 -- 172.21.15.16:0/2828138121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aab4c81070 msgr2=0x55aab50bf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:20.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.018+0000 7ff1f9000640 1 --2- 172.21.15.16:0/2828138121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aab4c81070 0x55aab50bf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.018+0000 7ff1f9000640 1 -- 172.21.15.16:0/2828138121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aab4c581b0 msgr2=0x55aab503a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:20.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.018+0000 7ff1f9000640 1 --2- 172.21.15.16:0/2828138121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aab4c581b0 0x55aab503a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.018+0000 7ff1f9000640 1 -- 172.21.15.16:0/2828138121 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aab50bf5a0 con 0x55aab4c60390 2024-08-31T06:20:20.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.018+0000 7ff1fa400640 1 --2- 172.21.15.16:0/2828138121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aab4c81070 0x55aab50bf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:20:20.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.018+0000 7ff1f9000640 1 --2- 172.21.15.16:0/2828138121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aab4c60390 0x55aab503ab40 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8015960 tx=0x7ff1e8011b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b36bd5830b708619 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:20.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.019+0000 7ff1db400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:20:20.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.019+0000 7ff1db400640 1 -- 172.21.15.16:0/2828138121 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e8019030 con 0x55aab4c60390 2024-08-31T06:20:20.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.019+0000 7ff1db400640 1 -- 172.21.15.16:0/2828138121 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8470+0+0 (secure 0 0 0) 0x7ff1e801e450 con 0x55aab4c60390 2024-08-31T06:20:20.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.019+0000 7ff203078740 1 -- 172.21.15.16:0/2828138121 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1e8011d30 con 0x55aab4c60390 2024-08-31T06:20:20.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.022+0000 7ff1db400640 1 -- 172.21.15.16:0/2828138121 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7ff1e801e670 con 0x55aab4c60390 2024-08-31T06:20:20.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.022+0000 7ff203078740 10 client.28556 fetch_fsmap learned FSMap version 49 2024-08-31T06:20:20.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.022+0000 7ff203078740 1 -- 172.21.15.16:0/2828138121 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55aab50c07e0 con 0x55aab4c60390 2024-08-31T06:20:20.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.023+0000 7ff1db400640 1 -- 172.21.15.16:0/2828138121 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1e8011d30 con 0x55aab4c60390 2024-08-31T06:20:20.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.023+0000 7ff1db400640 1 -- 172.21.15.16:0/2828138121 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7ff1e8025070 con 0x55aab4c60390 2024-08-31T06:20:20.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.023+0000 7ff203078740 10 client.28556 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:20:20.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.023+0000 7ff203078740 10 client.28556 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:20:20.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.023+0000 7ff203078740 20 client.28556 populate_metadata read hostname 'smithi016' 2024-08-31T06:20:20.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.023+0000 7ff203078740 1 --2- 172.21.15.16:0/2828138121 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55aab4c14da0 0x55aab50c2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:20.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.023+0000 7ff203078740 4 client.28556 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T06:20:20.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.023+0000 7ff203078740 1 -- 172.21.15.16:0/2828138121 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55aab4f9b160 con 0x55aab4c14da0 2024-08-31T06:20:20.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.024+0000 7ff1f9a00640 1 --2- 172.21.15.16:0/2828138121 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55aab4c14da0 0x55aab50c2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:20.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.025+0000 7ff1f9a00640 1 --2- 172.21.15.16:0/2828138121 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55aab4c14da0 0x55aab50c2e20 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62fa41f22c1793b9 server_cookie=78907a6d486b3011 in_seq=0 out_seq=0 2024-08-31T06:20:20.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.025+0000 7ff1db400640 10 client.28556 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:20:20.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.025+0000 7ff1db400640 1 -- 172.21.15.16:0/2828138121 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5114 (crc 0 0 0) 0x55aab4f9b160 con 0x55aab4c14da0 2024-08-31T06:20:20.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.025+0000 7ff1db400640 10 client.28556 handle_command_reply: tid=0 2024-08-31T06:20:20.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.025+0000 7ff203078740 1 client.28556 shutdown 2024-08-31T06:20:20.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.025+0000 7ff203078740 1 -- 172.21.15.16:0/2828138121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aab4c60390 msgr2=0x55aab503ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:20.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.025+0000 7ff203078740 1 --2- 172.21.15.16:0/2828138121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aab4c60390 0x55aab503ab40 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8015960 tx=0x7ff1e8011b20 comp rx=0 tx=0).stop 2024-08-31T06:20:20.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.026+0000 7ff203078740 1 -- 172.21.15.16:0/2828138121 shutdown_connections 2024-08-31T06:20:20.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.026+0000 7ff203078740 1 --2- 172.21.15.16:0/2828138121 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55aab4c14da0 0x55aab50c2e20 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.026+0000 7ff203078740 1 --2- 172.21.15.16:0/2828138121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aab4c81070 0x55aab50bf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.026+0000 7ff203078740 1 --2- 172.21.15.16:0/2828138121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aab4c60390 0x55aab503ab40 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.026+0000 7ff203078740 1 --2- 172.21.15.16:0/2828138121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aab4c581b0 0x55aab503a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.026+0000 7ff203078740 1 -- 172.21.15.16:0/2828138121 >> 172.21.15.16:0/2828138121 conn(0x55aab4c75460 msgr2=0x55aab5030d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:20.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.026+0000 7ff1db400640 0 client.28556 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:20:20.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.026+0000 7ff203078740 1 -- 172.21.15.16:0/2828138121 shutdown_connections 2024-08-31T06:20:20.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.026+0000 7ff203078740 1 -- 172.21.15.16:0/2828138121 wait complete. 2024-08-31T06:20:20.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.026+0000 7ff203078740 20 client.28556 trim_cache size 0 max 16384 2024-08-31T06:20:20.030 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:20:20.030 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T06:20:20.030 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-08-31T06:20:20.031 INFO:teuthology.orchestra.run.smithi016.stdout: "18e112f4-a5f9-4602-aaad-3b13fa5a4c2e": { 2024-08-31T06:20:20.031 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:20:20.031 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "18e112f4-a5f9-4602-aaad-3b13fa5a4c2e", 2024-08-31T06:20:20.031 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-08-31T06:20:20.031 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:20:20.031 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:20:20.031 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-08-31T06:20:20.031 INFO:teuthology.orchestra.run.smithi016.stdout: "18e112f4-a5f9-4602-aaad-3b13fa5a4c2e": { 2024-08-31T06:20:20.031 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:20:16.570535+0000", 2024-08-31T06:20:20.031 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:20:20.031 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:20:20.031 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:20:20.031 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:20:20.032 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:20:20.032 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:20:20.032 INFO:teuthology.orchestra.run.smithi016.stdout: "f1cdcb27-e3e4-4dc8-a0ae-7ace761d04c5": { 2024-08-31T06:20:20.032 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:19:22.275683+0000", 2024-08-31T06:20:20.032 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:20:20.032 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:20:20.032 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:20:20.032 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:20:20.032 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:20:20.032 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:20:20.032 INFO:teuthology.orchestra.run.smithi016.stdout: "0e44eb18-439c-46e4-a298-7d4d2ecd0b4a": { 2024-08-31T06:20:20.032 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:18:31.130765+0000", 2024-08-31T06:20:20.032 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:20:20.032 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:20:20.033 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:20:20.033 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:20:20.033 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:20:20.033 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:20:20.033 INFO:teuthology.orchestra.run.smithi016.stdout: "e497cd88-1e7c-4631-a4a8-cdd8e1fa6c83": { 2024-08-31T06:20:20.033 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:17:42.476795+0000", 2024-08-31T06:20:20.033 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:20:20.033 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:20:20.033 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:20:20.033 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:20:20.033 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:20:20.033 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:20:20.034 INFO:teuthology.orchestra.run.smithi016.stdout: "c300b7e8-f3fd-482c-a909-9329bc61e570": { 2024-08-31T06:20:20.034 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:06:49.868046+0000", 2024-08-31T06:20:20.034 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:20:20.034 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:20:20.034 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:20:20.034 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:20:20.034 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:20:20.034 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:20:20.034 INFO:teuthology.orchestra.run.smithi016.stdout: "6a03046e-cdf4-4a14-b9bd-b560b8968467": { 2024-08-31T06:20:20.034 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:12:38.002354+0000", 2024-08-31T06:20:20.034 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:20:20.034 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:20:20.034 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:20:20.034 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:20:20.035 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:20:20.035 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:20:20.035 INFO:teuthology.orchestra.run.smithi016.stdout: "004b47bb-0fe8-4d28-9d9d-78bde6a61199": { 2024-08-31T06:20:20.035 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:16:48.915926+0000", 2024-08-31T06:20:20.035 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:20:20.035 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:20:20.035 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:20:20.035 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:20:20.035 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:20:20.035 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:20:20.035 INFO:teuthology.orchestra.run.smithi016.stdout: "88d7bdb8-cb1c-45e5-ad29-4cd8da6c6003": { 2024-08-31T06:20:20.035 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:07:39.648942+0000", 2024-08-31T06:20:20.035 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:20:20.035 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:20:20.035 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:20:20.036 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:20:20.036 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:20:20.036 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:20:20.036 INFO:teuthology.orchestra.run.smithi016.stdout: "0fdcd762-533d-42be-9289-b71c8e3b81b1": { 2024-08-31T06:20:20.036 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:15:06.169112+0000", 2024-08-31T06:20:20.036 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:20:20.036 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:20:20.036 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:20:20.036 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:20:20.036 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:20:20.036 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:20:20.036 INFO:teuthology.orchestra.run.smithi016.stdout: "c8455d74-2fe7-4d93-bcf2-19b419d16016": { 2024-08-31T06:20:20.036 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:08:24.035960+0000", 2024-08-31T06:20:20.037 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:20:20.037 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:20:20.037 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:20:20.037 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:20:20.037 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:20:20.037 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:20:20.037 INFO:teuthology.orchestra.run.smithi016.stdout: "78d6ef02-8551-45e8-ad60-9627eb573fc1": { 2024-08-31T06:20:20.037 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:09:13.547865+0000", 2024-08-31T06:20:20.037 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:20:20.037 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:20:20.038 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:20:20.038 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:20:20.038 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:20:20.038 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:20:20.038 INFO:teuthology.orchestra.run.smithi016.stdout: "c1fd6904-2e02-47fa-939a-47349b9fe36b": { 2024-08-31T06:20:20.038 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:06.678691+0000", 2024-08-31T06:20:20.038 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:20:20.038 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:20:20.038 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:20:20.038 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:20:20.038 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:20:20.038 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:20:20.038 INFO:teuthology.orchestra.run.smithi016.stdout: "a10cce76-99ab-46b5-b3b2-07b82899b8f9": { 2024-08-31T06:20:20.038 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:11:43.967546+0000", 2024-08-31T06:20:20.039 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:20:20.039 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:20:20.039 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:20:20.039 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:20:20.039 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:20:20.039 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:20:20.039 INFO:teuthology.orchestra.run.smithi016.stdout: "91b72a34-4ba9-4126-a414-8b862f8c5f78": { 2024-08-31T06:20:20.039 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:13:28.663237+0000", 2024-08-31T06:20:20.039 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:20:20.039 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:20:20.039 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:20:20.039 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:20:20.039 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:20:20.039 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:20:20.040 INFO:teuthology.orchestra.run.smithi016.stdout: "e739a16e-8ace-43a8-a1ce-ac27d39ef8a3": { 2024-08-31T06:20:20.040 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:54.084646+0000", 2024-08-31T06:20:20.040 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:20:20.040 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:20:20.040 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:20:20.040 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:20:20.040 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:20:20.040 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:20:20.040 INFO:teuthology.orchestra.run.smithi016.stdout: "a64d5301-8558-4e9c-ac9b-8d0ac7057e74": { 2024-08-31T06:20:20.040 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:14:20.338104+0000", 2024-08-31T06:20:20.040 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:20:20.040 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:20:20.041 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:20:20.041 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:20:20.041 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:20:20.041 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:20:20.041 INFO:teuthology.orchestra.run.smithi016.stdout: "f9c4a22e-0c8a-4928-9b0d-e2ce1db5a86b": { 2024-08-31T06:20:20.041 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:15:57.496015+0000", 2024-08-31T06:20:20.041 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:20:20.041 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:20:20.041 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:20:20.041 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:20:20.041 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:20:20.041 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:20:20.041 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:20:20.042 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:20:20.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.027+0000 7ff1fae00640 1 -- 172.21.15.16:0/2674427317 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff1dc0b4cd0 msgr2=0x7ff1dc0b71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:20.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.027+0000 7ff1fae00640 1 --2- 172.21.15.16:0/2674427317 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff1dc0b4cd0 0x7ff1dc0b71e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec007ed0 tx=0x7ff1ec007520 comp rx=0 tx=0).stop 2024-08-31T06:20:20.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.027+0000 7ff1fae00640 1 -- 172.21.15.16:0/2674427317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1f4109d60 msgr2=0x7ff1f41a0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:20.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.027+0000 7ff1fae00640 1 --2- 172.21.15.16:0/2674427317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1f4109d60 0x7ff1f41a0730 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0005ab0 tx=0x7ff1e0003d50 comp rx=0 tx=0).stop 2024-08-31T06:20:20.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.027+0000 7ff1fae00640 1 -- 172.21.15.16:0/2674427317 shutdown_connections 2024-08-31T06:20:20.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.027+0000 7ff1fae00640 1 --2- 172.21.15.16:0/2674427317 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff1dc0b4cd0 0x7ff1dc0b71e0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.027+0000 7ff1fae00640 1 --2- 172.21.15.16:0/2674427317 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1f410f6a0 0x7ff1f41a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.027+0000 7ff1fae00640 1 --2- 172.21.15.16:0/2674427317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1f410fff0 0x7ff1f41a5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.027+0000 7ff1fae00640 1 --2- 172.21.15.16:0/2674427317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1f4109d60 0x7ff1f41a0730 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.027+0000 7ff1fae00640 1 -- 172.21.15.16:0/2674427317 >> 172.21.15.16:0/2674427317 conn(0x7ff1f4079360 msgr2=0x7ff1f4079ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.027+0000 7ff1fae00640 1 -- 172.21.15.16:0/2674427317 shutdown_connections 2024-08-31T06:20:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:20.027+0000 7ff1fae00640 1 -- 172.21.15.16:0/2674427317 wait complete. 2024-08-31T06:20:20.043 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:19 smithi016 ceph-mon[17194]: pgmap v773: 97 pgs: 97 active+clean; 3.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 5.3 MiB/s wr, 647 op/s 2024-08-31T06:20:20.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:19 smithi184 ceph-mon[22331]: pgmap v773: 97 pgs: 97 active+clean; 3.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 5.3 MiB/s wr, 647 op/s 2024-08-31T06:20:20.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:19 smithi155 ceph-mon[21742]: pgmap v773: 97 pgs: 97 active+clean; 3.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 5.3 MiB/s wr, 647 op/s 2024-08-31T06:20:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.743+0000 7f2921200640 1 -- 172.21.15.16:0/2482665911 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f291c111a60 msgr2=0x7f291c10a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.743+0000 7f2921200640 1 --2- 172.21.15.16:0/2482665911 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f291c111a60 0x7f291c10a160 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2908009860 tx=0x7f290802a340 comp rx=0 tx=0).stop 2024-08-31T06:20:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.745+0000 7f2921200640 1 -- 172.21.15.16:0/2482665911 shutdown_connections 2024-08-31T06:20:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.745+0000 7f2921200640 1 --2- 172.21.15.16:0/2482665911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f291c10d470 0x7f291c10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.745+0000 7f2921200640 1 --2- 172.21.15.16:0/2482665911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f291c10a6a0 0x7f291c10aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.745+0000 7f2921200640 1 --2- 172.21.15.16:0/2482665911 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f291c111a60 0x7f291c10a160 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.745+0000 7f2921200640 1 -- 172.21.15.16:0/2482665911 >> 172.21.15.16:0/2482665911 conn(0x7f291c070e00 msgr2=0x7f291c071210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.746+0000 7f2921200640 1 -- 172.21.15.16:0/2482665911 shutdown_connections 2024-08-31T06:20:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.746+0000 7f2921200640 1 -- 172.21.15.16:0/2482665911 wait complete. 2024-08-31T06:20:20.750 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.748+0000 7f2921200640 1 Processor -- start 2024-08-31T06:20:20.751 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.748+0000 7f2921200640 1 -- start start 2024-08-31T06:20:20.751 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.749+0000 7f2921200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f291c10a6a0 0x7f291c1a8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:20.751 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.749+0000 7f2921200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f291c10d470 0x7f291c1a9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:20.751 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.749+0000 7f2921200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f291c111a60 0x7f291c1ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:20.751 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.749+0000 7f2921200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f291c122450 con 0x7f291c10d470 2024-08-31T06:20:20.751 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.749+0000 7f2921200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f291c1222d0 con 0x7f291c10a6a0 2024-08-31T06:20:20.752 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.749+0000 7f2921200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f291c1225d0 con 0x7f291c111a60 2024-08-31T06:20:20.753 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.749+0000 7f291be00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f291c111a60 0x7f291c1ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:20.753 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.749+0000 7f291b400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f291c10a6a0 0x7f291c1a8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:20.753 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.749+0000 7f291b400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f291c10a6a0 0x7f291c1a8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:41608/0 (socket says 172.21.15.16:41608) 2024-08-31T06:20:20.753 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.749+0000 7f291b400640 1 -- 172.21.15.16:0/680676562 learned_addr learned my addr 172.21.15.16:0/680676562 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:20.754 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.749+0000 7f291be00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f291c111a60 0x7f291c1ae6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:57470/0 (socket says 172.21.15.16:57470) 2024-08-31T06:20:20.754 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.750+0000 7f291aa00640 1 --2- 172.21.15.16:0/680676562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f291c10d470 0x7f291c1a9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:20.754 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.750+0000 7f291b400640 1 -- 172.21.15.16:0/680676562 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f291c111a60 msgr2=0x7f291c1ae6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:20.754 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.750+0000 7f291b400640 1 --2- 172.21.15.16:0/680676562 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f291c111a60 0x7f291c1ae6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.754 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.750+0000 7f291b400640 1 -- 172.21.15.16:0/680676562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f291c10d470 msgr2=0x7f291c1a9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:20.754 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.750+0000 7f291b400640 1 --2- 172.21.15.16:0/680676562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f291c10d470 0x7f291c1a9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.754 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.750+0000 7f291b400640 1 -- 172.21.15.16:0/680676562 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f291c1aedc0 con 0x7f291c10a6a0 2024-08-31T06:20:20.754 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.750+0000 7f291aa00640 1 --2- 172.21.15.16:0/680676562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f291c10d470 0x7f291c1a9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:20:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.752+0000 7f291be00640 1 --2- 172.21.15.16:0/680676562 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f291c111a60 0x7f291c1ae6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:20:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.752+0000 7f291b400640 1 --2- 172.21.15.16:0/680676562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f291c10a6a0 0x7f291c1a8e90 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f290802a8b0 tx=0x7f2908032d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.752+0000 7f2913e00640 1 -- 172.21.15.16:0/680676562 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2908042070 con 0x7f291c10a6a0 2024-08-31T06:20:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.752+0000 7f2913e00640 1 -- 172.21.15.16:0/680676562 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2908030e10 con 0x7f291c10a6a0 2024-08-31T06:20:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.753+0000 7f2913e00640 1 -- 172.21.15.16:0/680676562 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2908037040 con 0x7f291c10a6a0 2024-08-31T06:20:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.753+0000 7f2921200640 1 -- 172.21.15.16:0/680676562 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f291c1af050 con 0x7f291c10a6a0 2024-08-31T06:20:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.753+0000 7f2921200640 1 -- 172.21.15.16:0/680676562 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f291c1af4f0 con 0x7f291c10a6a0 2024-08-31T06:20:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.760+0000 7f2929891740 1 Processor -- start 2024-08-31T06:20:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.760+0000 7f2929891740 1 -- start start 2024-08-31T06:20:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.761+0000 7f2929891740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56399de831b0 0x56399e2268a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.762+0000 7f2929891740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56399de5a2f0 0x56399e226de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.762+0000 7f291be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56399de831b0 0x56399e2268a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:20.765 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.762+0000 7f2929891740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56399de624d0 0x56399e227320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:20.765 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.762+0000 7f2929891740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56399e19bf70 con 0x56399de831b0 2024-08-31T06:20:20.765 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.762+0000 7f2929891740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56399e18b500 con 0x56399de624d0 2024-08-31T06:20:20.765 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.762+0000 7f2929891740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56399e18f4b0 con 0x56399de5a2f0 2024-08-31T06:20:20.765 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.762+0000 7f291be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56399de831b0 0x56399e2268a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51532/0 (socket says 172.21.15.16:51532) 2024-08-31T06:20:20.765 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.762+0000 7f291be00640 1 -- 172.21.15.16:0/1275986918 learned_addr learned my addr 172.21.15.16:0/1275986918 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:20.767 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.764+0000 7f291aa00640 1 --2- 172.21.15.16:0/1275986918 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56399de624d0 0x56399e227320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:20.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.765+0000 7f291be00640 1 -- 172.21.15.16:0/1275986918 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56399de5a2f0 msgr2=0x56399e226de0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:20:20.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.765+0000 7f291be00640 1 --2- 172.21.15.16:0/1275986918 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56399de5a2f0 0x56399e226de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.765+0000 7f291be00640 1 -- 172.21.15.16:0/1275986918 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56399de624d0 msgr2=0x56399e227320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:20.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.765+0000 7f291be00640 1 --2- 172.21.15.16:0/1275986918 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56399de624d0 0x56399e227320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.765+0000 7f291be00640 1 -- 172.21.15.16:0/1275986918 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56399e145c10 con 0x56399de831b0 2024-08-31T06:20:20.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.766+0000 7f291be00640 1 --2- 172.21.15.16:0/1275986918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56399de831b0 0x56399e2268a0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f29140105d0 tx=0x7f2914010a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad1588a60d4e1e2 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:20.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.767+0000 7f2910c00640 1 -- 172.21.15.16:0/1275986918 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291401d070 con 0x56399de831b0 2024-08-31T06:20:20.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.767+0000 7f2910c00640 1 -- 172.21.15.16:0/1275986918 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f291400ecd0 con 0x56399de831b0 2024-08-31T06:20:20.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.767+0000 7f2910c00640 1 -- 172.21.15.16:0/1275986918 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2914018de0 con 0x56399de831b0 2024-08-31T06:20:20.774 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.768+0000 7f2929891740 1 -- 172.21.15.16:0/1275986918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56399de831b0 msgr2=0x56399e2268a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:20.774 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.768+0000 7f2929891740 1 --2- 172.21.15.16:0/1275986918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56399de831b0 0x56399e2268a0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f29140105d0 tx=0x7f2914010a90 comp rx=0 tx=0).stop 2024-08-31T06:20:20.774 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.769+0000 7f2929891740 1 -- 172.21.15.16:0/1275986918 shutdown_connections 2024-08-31T06:20:20.775 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.769+0000 7f2929891740 1 --2- 172.21.15.16:0/1275986918 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56399de5a2f0 0x56399e226de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.775 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.769+0000 7f2929891740 1 --2- 172.21.15.16:0/1275986918 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56399de624d0 0x56399e227320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.775 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.769+0000 7f2929891740 1 --2- 172.21.15.16:0/1275986918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56399de831b0 0x56399e2268a0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.775 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.769+0000 7f2929891740 1 -- 172.21.15.16:0/1275986918 >> 172.21.15.16:0/1275986918 conn(0x56399de77300 msgr2=0x56399e224790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:20.775 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.771+0000 7f2913e00640 1 -- 172.21.15.16:0/680676562 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f2908005470 con 0x7f291c10a6a0 2024-08-31T06:20:20.775 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.772+0000 7f2929891740 1 -- 172.21.15.16:0/1275986918 shutdown_connections 2024-08-31T06:20:20.775 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.773+0000 7f2929891740 1 -- 172.21.15.16:0/1275986918 wait complete. 2024-08-31T06:20:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.777+0000 7f2929891740 1 -- start start 2024-08-31T06:20:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.778+0000 7f2929891740 1 Processor -- start 2024-08-31T06:20:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.779+0000 7f2913e00640 1 --2- 172.21.15.16:0/680676562 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f29000b4ce0 0x7f29000b71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.780+0000 7f291be00640 1 --2- 172.21.15.16:0/680676562 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f29000b4ce0 0x7f29000b71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.780+0000 7f2913e00640 1 -- 172.21.15.16:0/680676562 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8470+0+0 (secure 0 0 0) 0x7f2908104e50 con 0x7f291c10a6a0 2024-08-31T06:20:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.780+0000 7f2929891740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56399de5a2f0 0x56399e231f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.781+0000 7f2929891740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56399de624d0 0x56399e232480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.781+0000 7f2929891740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56399de831b0 0x56399e23e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:20.785 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.781+0000 7f291aa00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56399de624d0 0x56399e232480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:20.785 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.781+0000 7f291b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56399de5a2f0 0x56399e231f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:20.785 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.781+0000 7f291aa00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56399de624d0 0x56399e232480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:57476/0 (socket says 172.21.15.16:57476) 2024-08-31T06:20:20.785 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.781+0000 7f291aa00640 1 -- 172.21.15.16:0/3251313505 learned_addr learned my addr 172.21.15.16:0/3251313505 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:20.785 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.782+0000 7f2929891740 1 -- 172.21.15.16:0/3251313505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56399e0c4ab0 con 0x56399de5a2f0 2024-08-31T06:20:20.785 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.782+0000 7f2929891740 1 -- 172.21.15.16:0/3251313505 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56399e132320 con 0x56399de831b0 2024-08-31T06:20:20.785 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.782+0000 7f2929891740 1 -- 172.21.15.16:0/3251313505 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56399e2350c0 con 0x56399de624d0 2024-08-31T06:20:20.785 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.782+0000 7f2929891740 10 client.0 _ll_register_callbacks cb 0x56399e16be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:20:20.786 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.782+0000 7f291be00640 1 --2- 172.21.15.16:0/3251313505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56399de831b0 0x56399e23e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:20.786 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.782+0000 7f291aa00640 1 -- 172.21.15.16:0/3251313505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56399de831b0 msgr2=0x56399e23e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:20.786 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.782+0000 7f291aa00640 1 --2- 172.21.15.16:0/3251313505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56399de831b0 0x56399e23e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.786 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.782+0000 7f291aa00640 1 -- 172.21.15.16:0/3251313505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56399de5a2f0 msgr2=0x56399e231f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:20.786 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.783+0000 7f291aa00640 1 --2- 172.21.15.16:0/3251313505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56399de5a2f0 0x56399e231f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.786 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.784+0000 7f291aa00640 1 -- 172.21.15.16:0/3251313505 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56399deee790 con 0x56399de624d0 2024-08-31T06:20:20.787 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.784+0000 7f291b400640 1 --2- 172.21.15.16:0/3251313505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56399de5a2f0 0x56399e231f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:20:20.787 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.784+0000 7f291be00640 1 --2- 172.21.15.16:0/680676562 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f29000b4ce0 0x7f29000b71f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f29140099a0 tx=0x7f2914009360 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:20.787 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.785+0000 7f2905600640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:20:20.788 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.785+0000 7f291aa00640 1 --2- 172.21.15.16:0/3251313505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56399de624d0 0x56399e232480 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f290c0126a0 tx=0x7f290c012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57b004cdeb582e17 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:20.790 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.786+0000 7f2905600640 1 -- 172.21.15.16:0/3251313505 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c01f060 con 0x56399de624d0 2024-08-31T06:20:20.790 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.786+0000 7f2905600640 1 -- 172.21.15.16:0/3251313505 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8470+0+0 (secure 0 0 0) 0x7f290c010d60 con 0x56399de624d0 2024-08-31T06:20:20.790 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.787+0000 7f2929891740 1 -- 172.21.15.16:0/3251313505 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f291400ecd0 con 0x56399de624d0 2024-08-31T06:20:20.791 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.788+0000 7f2905600640 1 -- 172.21.15.16:0/3251313505 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f290c01a3f0 con 0x56399de624d0 2024-08-31T06:20:20.791 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.788+0000 7f2929891740 10 client.28576 fetch_fsmap learned FSMap version 49 2024-08-31T06:20:20.791 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.788+0000 7f2929891740 1 -- 172.21.15.16:0/3251313505 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x56399df67620 con 0x56399de624d0 2024-08-31T06:20:20.791 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.789+0000 7f2905600640 1 -- 172.21.15.16:0/3251313505 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f291400ecd0 con 0x56399de624d0 2024-08-31T06:20:20.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.790+0000 7f2905600640 1 -- 172.21.15.16:0/3251313505 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f290c019030 con 0x56399de624d0 2024-08-31T06:20:20.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.791+0000 7f2929891740 10 client.28576 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:20:20.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.791+0000 7f2929891740 10 client.28576 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:20:20.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.791+0000 7f2929891740 20 client.28576 populate_metadata read hostname 'smithi016' 2024-08-31T06:20:20.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.791+0000 7f2929891740 1 --2- 172.21.15.16:0/3251313505 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56399de41d80 0x56399e2441a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:20.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.791+0000 7f2929891740 4 client.28576 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:20:20.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.791+0000 7f2929891740 1 -- 172.21.15.16:0/3251313505 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56399e19dec0 con 0x56399de41d80 2024-08-31T06:20:20.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.792+0000 7f291b400640 1 --2- 172.21.15.16:0/3251313505 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56399de41d80 0x56399e2441a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:20.796 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.793+0000 7f2905600640 10 client.28576 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:20:20.796 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.793+0000 7f291b400640 1 --2- 172.21.15.16:0/3251313505 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56399de41d80 0x56399e2441a0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a218fabda38a26bd server_cookie=a5e4661d544912bc in_seq=0 out_seq=0 2024-08-31T06:20:20.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.794+0000 7f2905600640 1 -- 172.21.15.16:0/3251313505 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56399e19dec0 con 0x56399de41d80 2024-08-31T06:20:20.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.794+0000 7f2905600640 10 client.28576 handle_command_reply: tid=0 2024-08-31T06:20:20.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.794+0000 7f2929891740 1 client.28576 shutdown 2024-08-31T06:20:20.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.795+0000 7f2929891740 1 -- 172.21.15.16:0/3251313505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56399de624d0 msgr2=0x56399e232480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:20.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.795+0000 7f2929891740 1 --2- 172.21.15.16:0/3251313505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56399de624d0 0x56399e232480 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f290c0126a0 tx=0x7f290c012b60 comp rx=0 tx=0).stop 2024-08-31T06:20:20.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.795+0000 7f2929891740 1 -- 172.21.15.16:0/3251313505 shutdown_connections 2024-08-31T06:20:20.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.795+0000 7f2929891740 1 --2- 172.21.15.16:0/3251313505 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56399de41d80 0x56399e2441a0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.795+0000 7f2929891740 1 --2- 172.21.15.16:0/3251313505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56399de624d0 0x56399e232480 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.795+0000 7f2929891740 1 --2- 172.21.15.16:0/3251313505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56399de831b0 0x56399e23e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.795+0000 7f2929891740 1 --2- 172.21.15.16:0/3251313505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56399de5a2f0 0x56399e231f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.795+0000 7f2929891740 1 -- 172.21.15.16:0/3251313505 >> 172.21.15.16:0/3251313505 conn(0x56399de77300 msgr2=0x56399e2278d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:20.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.795+0000 7f2905600640 0 client.28576 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:20:20.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.795+0000 7f2929891740 1 -- 172.21.15.16:0/3251313505 shutdown_connections 2024-08-31T06:20:20.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.796+0000 7f2929891740 1 -- 172.21.15.16:0/3251313505 wait complete. 2024-08-31T06:20:20.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.796+0000 7f2929891740 20 client.28576 trim_cache size 0 max 16384 2024-08-31T06:20:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.822+0000 7f2929891740 1 Processor -- start 2024-08-31T06:20:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.822+0000 7f2929891740 1 -- start start 2024-08-31T06:20:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.823+0000 7f2929891740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56399de5a2f0 0x56399e227fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.823+0000 7f2929891740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56399de831b0 0x56399e22f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.823+0000 7f2929891740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56399de171c0 0x56399e22f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.823+0000 7f2929891740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f291c1c0aa0 con 0x56399de831b0 2024-08-31T06:20:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.823+0000 7f2929891740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f291c1be2b0 con 0x56399de5a2f0 2024-08-31T06:20:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.823+0000 7f2929891740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f291c1bc120 con 0x56399de171c0 2024-08-31T06:20:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.823+0000 7f291be00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56399de171c0 0x56399e22f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.824+0000 7f291be00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56399de171c0 0x56399e22f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:57490/0 (socket says 172.21.15.16:57490) 2024-08-31T06:20:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.824+0000 7f291be00640 1 -- 172.21.15.16:0/1921067734 learned_addr learned my addr 172.21.15.16:0/1921067734 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.824+0000 7f291b400640 1 --2- 172.21.15.16:0/1921067734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56399de5a2f0 0x56399e227fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.824+0000 7f291aa00640 1 --2- 172.21.15.16:0/1921067734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56399de831b0 0x56399e22f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.825+0000 7f291b400640 1 -- 172.21.15.16:0/1921067734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56399de171c0 msgr2=0x56399e22f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.825+0000 7f291b400640 1 --2- 172.21.15.16:0/1921067734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56399de171c0 0x56399e22f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.825+0000 7f291b400640 1 -- 172.21.15.16:0/1921067734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56399de831b0 msgr2=0x56399e22f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.825+0000 7f291b400640 1 --2- 172.21.15.16:0/1921067734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56399de831b0 0x56399e22f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.825+0000 7f291b400640 1 -- 172.21.15.16:0/1921067734 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56399e225500 con 0x56399de5a2f0 2024-08-31T06:20:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.825+0000 7f291aa00640 1 --2- 172.21.15.16:0/1921067734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56399de831b0 0x56399e22f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:20:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.825+0000 7f291be00640 1 --2- 172.21.15.16:0/1921067734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56399de171c0 0x56399e22f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:20:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.826+0000 7f291b400640 1 --2- 172.21.15.16:0/1921067734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56399de5a2f0 0x56399e227fe0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f2908102570 tx=0x7f29080ee040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d713d33b50d5eece server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.827+0000 7f2907e00640 1 -- 172.21.15.16:0/1921067734 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29080d4170 con 0x56399de5a2f0 2024-08-31T06:20:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.827+0000 7f2907e00640 1 -- 172.21.15.16:0/1921067734 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29080cb020 con 0x56399de5a2f0 2024-08-31T06:20:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.827+0000 7f2907e00640 1 -- 172.21.15.16:0/1921067734 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29080f2100 con 0x56399de5a2f0 2024-08-31T06:20:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.828+0000 7f2929891740 1 -- 172.21.15.16:0/1921067734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56399de5a2f0 msgr2=0x56399e227fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.828+0000 7f2929891740 1 --2- 172.21.15.16:0/1921067734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56399de5a2f0 0x56399e227fe0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f2908102570 tx=0x7f29080ee040 comp rx=0 tx=0).stop 2024-08-31T06:20:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.828+0000 7f2929891740 1 -- 172.21.15.16:0/1921067734 shutdown_connections 2024-08-31T06:20:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.829+0000 7f2929891740 1 --2- 172.21.15.16:0/1921067734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56399de171c0 0x56399e22f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.829+0000 7f2929891740 1 --2- 172.21.15.16:0/1921067734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56399de831b0 0x56399e22f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.829+0000 7f2929891740 1 --2- 172.21.15.16:0/1921067734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56399de5a2f0 0x56399e227fe0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.829+0000 7f2929891740 1 -- 172.21.15.16:0/1921067734 >> 172.21.15.16:0/1921067734 conn(0x56399de77300 msgr2=0x56399e2278d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.829+0000 7f2929891740 1 -- 172.21.15.16:0/1921067734 shutdown_connections 2024-08-31T06:20:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.829+0000 7f2929891740 1 -- 172.21.15.16:0/1921067734 wait complete. 2024-08-31T06:20:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.829+0000 7f2929891740 1 -- start start 2024-08-31T06:20:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.830+0000 7f2929891740 1 Processor -- start 2024-08-31T06:20:20.833 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.830+0000 7f2929891740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56399de171c0 0x56399e237dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:20.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.830+0000 7f2929891740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56399de5a2f0 0x56399e2383b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:20.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.830+0000 7f2929891740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56399de831b0 0x56399e2a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:20.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.830+0000 7f2929891740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f291c1bc770 con 0x56399de831b0 2024-08-31T06:20:20.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.830+0000 7f2929891740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f291c1bdbe0 con 0x56399de171c0 2024-08-31T06:20:20.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.830+0000 7f2929891740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f291c1bce40 con 0x56399de5a2f0 2024-08-31T06:20:20.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.830+0000 7f2929891740 10 client.0 _ll_register_callbacks cb 0x7f290c0094f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:20:20.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.831+0000 7f291b400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56399de171c0 0x56399e237dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:20.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.831+0000 7f291b400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56399de171c0 0x56399e237dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:41646/0 (socket says 172.21.15.16:41646) 2024-08-31T06:20:20.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.831+0000 7f291b400640 1 -- 172.21.15.16:0/3856652669 learned_addr learned my addr 172.21.15.16:0/3856652669 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:20.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.831+0000 7f291b400640 1 -- 172.21.15.16:0/3856652669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56399de5a2f0 msgr2=0x56399e2383b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:20:20.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.831+0000 7f291be00640 1 --2- 172.21.15.16:0/3856652669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56399de831b0 0x56399e2a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.831+0000 7f291b400640 1 --2- 172.21.15.16:0/3856652669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56399de5a2f0 0x56399e2383b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.831+0000 7f291b400640 1 -- 172.21.15.16:0/3856652669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56399de831b0 msgr2=0x56399e2a5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.831+0000 7f291b400640 1 --2- 172.21.15.16:0/3856652669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56399de831b0 0x56399e2a5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.831+0000 7f291b400640 1 -- 172.21.15.16:0/3856652669 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56399e2a6200 con 0x56399de171c0 2024-08-31T06:20:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.832+0000 7f291be00640 1 --2- 172.21.15.16:0/3856652669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56399de831b0 0x56399e2a5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:20:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.832+0000 7f291b400640 1 --2- 172.21.15.16:0/3856652669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56399de171c0 0x56399e237dd0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f29080303f0 tx=0x7f290802aa40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dcd8aad2b62fa01 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.832+0000 7f2905600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:20:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.832+0000 7f2905600640 1 -- 172.21.15.16:0/3856652669 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29080e5060 con 0x56399de171c0 2024-08-31T06:20:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.832+0000 7f2905600640 1 -- 172.21.15.16:0/3856652669 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8470+0+0 (secure 0 0 0) 0x7f29080c77a0 con 0x56399de171c0 2024-08-31T06:20:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.832+0000 7f2929891740 1 -- 172.21.15.16:0/3856652669 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29080cb020 con 0x56399de171c0 2024-08-31T06:20:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.833+0000 7f2905600640 1 -- 172.21.15.16:0/3856652669 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f29080cb510 con 0x56399de171c0 2024-08-31T06:20:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.833+0000 7f2929891740 10 client.28574 fetch_fsmap learned FSMap version 49 2024-08-31T06:20:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.833+0000 7f2929891740 1 -- 172.21.15.16:0/3856652669 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x56399e2ae650 con 0x56399de171c0 2024-08-31T06:20:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.833+0000 7f2905600640 1 -- 172.21.15.16:0/3856652669 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29080cb020 con 0x56399de171c0 2024-08-31T06:20:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.833+0000 7f2905600640 1 -- 172.21.15.16:0/3856652669 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f29080a5070 con 0x56399de171c0 2024-08-31T06:20:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.833+0000 7f2929891740 10 client.28574 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:20:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.833+0000 7f2929891740 10 client.28574 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:20:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.833+0000 7f2929891740 20 client.28574 populate_metadata read hostname 'smithi016' 2024-08-31T06:20:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.833+0000 7f2929891740 1 --2- 172.21.15.16:0/3856652669 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56399e2aed70 0x56399e2b1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.833+0000 7f2929891740 4 client.28574 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:20:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.833+0000 7f2929891740 1 -- 172.21.15.16:0/3856652669 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56399e199960 con 0x56399e2aed70 2024-08-31T06:20:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.834+0000 7f291aa00640 1 --2- 172.21.15.16:0/3856652669 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56399e2aed70 0x56399e2b1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.834+0000 7f291aa00640 1 --2- 172.21.15.16:0/3856652669 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56399e2aed70 0x56399e2b1190 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=755ab343d5f55e47 server_cookie=f229171986b68467 in_seq=0 out_seq=0 2024-08-31T06:20:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.834+0000 7f2905600640 10 client.28574 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:20:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.849+0000 7f2905600640 1 -- 172.21.15.16:0/3856652669 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56399e199960 con 0x56399e2aed70 2024-08-31T06:20:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.849+0000 7f2905600640 10 client.28574 handle_command_reply: tid=0 2024-08-31T06:20:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.849+0000 7f2929891740 1 client.28574 shutdown 2024-08-31T06:20:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.849+0000 7f2929891740 1 -- 172.21.15.16:0/3856652669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56399de171c0 msgr2=0x56399e237dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.849+0000 7f2929891740 1 --2- 172.21.15.16:0/3856652669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56399de171c0 0x56399e237dd0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f29080303f0 tx=0x7f290802aa40 comp rx=0 tx=0).stop 2024-08-31T06:20:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.850+0000 7f2929891740 1 -- 172.21.15.16:0/3856652669 shutdown_connections 2024-08-31T06:20:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.850+0000 7f2929891740 1 --2- 172.21.15.16:0/3856652669 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56399e2aed70 0x56399e2b1190 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.850+0000 7f2929891740 1 --2- 172.21.15.16:0/3856652669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56399de5a2f0 0x56399e2383b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.850+0000 7f2929891740 1 --2- 172.21.15.16:0/3856652669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56399de831b0 0x56399e2a5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.850+0000 7f2929891740 1 --2- 172.21.15.16:0/3856652669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56399de171c0 0x56399e237dd0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.850+0000 7f2929891740 1 -- 172.21.15.16:0/3856652669 >> 172.21.15.16:0/3856652669 conn(0x56399de77300 msgr2=0x56399e2333f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.850+0000 7f2905600640 0 client.28574 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:20:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.850+0000 7f2929891740 1 -- 172.21.15.16:0/3856652669 shutdown_connections 2024-08-31T06:20:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.850+0000 7f2929891740 1 -- 172.21.15.16:0/3856652669 wait complete. 2024-08-31T06:20:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.850+0000 7f2929891740 20 client.28574 trim_cache size 0 max 16384 2024-08-31T06:20:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:20:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:20:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:20:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:20:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.851+0000 7f2921200640 1 -- 172.21.15.16:0/680676562 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f29000b4ce0 msgr2=0x7f29000b71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.851+0000 7f2921200640 1 --2- 172.21.15.16:0/680676562 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f29000b4ce0 0x7f29000b71f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f29140099a0 tx=0x7f2914009360 comp rx=0 tx=0).stop 2024-08-31T06:20:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.851+0000 7f2921200640 1 -- 172.21.15.16:0/680676562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f291c10a6a0 msgr2=0x7f291c1a8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.851+0000 7f2921200640 1 --2- 172.21.15.16:0/680676562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f291c10a6a0 0x7f291c1a8e90 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f290802a8b0 tx=0x7f2908032d40 comp rx=0 tx=0).stop 2024-08-31T06:20:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.851+0000 7f2921200640 1 -- 172.21.15.16:0/680676562 shutdown_connections 2024-08-31T06:20:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.851+0000 7f2921200640 1 --2- 172.21.15.16:0/680676562 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f29000b4ce0 0x7f29000b71f0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.851+0000 7f2921200640 1 --2- 172.21.15.16:0/680676562 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f291c111a60 0x7f291c1ae6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.851+0000 7f2921200640 1 --2- 172.21.15.16:0/680676562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f291c10d470 0x7f291c1a9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.851+0000 7f2921200640 1 --2- 172.21.15.16:0/680676562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f291c10a6a0 0x7f291c1a8e90 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.851+0000 7f2921200640 1 -- 172.21.15.16:0/680676562 >> 172.21.15.16:0/680676562 conn(0x7f291c070e00 msgr2=0x7f291c1106e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.851+0000 7f2921200640 1 -- 172.21.15.16:0/680676562 shutdown_connections 2024-08-31T06:20:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:20.851+0000 7f2921200640 1 -- 172.21.15.16:0/680676562 wait complete. 2024-08-31T06:20:21.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.272+0000 7f78c9c00640 1 -- 172.21.15.16:0/4017275369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78c40776e0 msgr2=0x7f78c4077af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:21.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.272+0000 7f78c9c00640 1 --2- 172.21.15.16:0/4017275369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78c40776e0 0x7f78c4077af0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f78bc009ca0 tx=0x7f78bc02a8c0 comp rx=0 tx=0).stop 2024-08-31T06:20:21.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.273+0000 7f78c9c00640 1 -- 172.21.15.16:0/4017275369 shutdown_connections 2024-08-31T06:20:21.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.273+0000 7f78c9c00640 1 --2- 172.21.15.16:0/4017275369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78c4078030 0x7f78c4113fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:21.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.273+0000 7f78c9c00640 1 --2- 172.21.15.16:0/4017275369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c4114510 0x7f78c4116930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:21.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.273+0000 7f78c9c00640 1 --2- 172.21.15.16:0/4017275369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78c40776e0 0x7f78c4077af0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:21.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.273+0000 7f78c9c00640 1 -- 172.21.15.16:0/4017275369 >> 172.21.15.16:0/4017275369 conn(0x7f78c41015a0 msgr2=0x7f78c41039f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:21.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.273+0000 7f78c9c00640 1 -- 172.21.15.16:0/4017275369 shutdown_connections 2024-08-31T06:20:21.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.273+0000 7f78c9c00640 1 -- 172.21.15.16:0/4017275369 wait complete. 2024-08-31T06:20:21.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.274+0000 7f78c9c00640 1 Processor -- start 2024-08-31T06:20:21.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.274+0000 7f78c9c00640 1 -- start start 2024-08-31T06:20:21.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.275+0000 7f78c9c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78c40776e0 0x7f78c41a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:21.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.275+0000 7f78c9c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78c4078030 0x7f78c41a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:21.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.275+0000 7f78c9c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c4114510 0x7f78c41a60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:21.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.275+0000 7f78c9c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f78c4119e70 con 0x7f78c4114510 2024-08-31T06:20:21.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.275+0000 7f78c9c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f78c4119cf0 con 0x7f78c4078030 2024-08-31T06:20:21.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.275+0000 7f78c9c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f78c4119ff0 con 0x7f78c40776e0 2024-08-31T06:20:21.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.275+0000 7f78c2a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78c40776e0 0x7f78c41a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:21.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.275+0000 7f78c2000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78c4078030 0x7f78c41a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:21.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.275+0000 7f78c2a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78c40776e0 0x7f78c41a0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:57524/0 (socket says 172.21.15.16:57524) 2024-08-31T06:20:21.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.275+0000 7f78c2a00640 1 -- 172.21.15.16:0/871111923 learned_addr learned my addr 172.21.15.16:0/871111923 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:21.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.275+0000 7f78c3400640 1 --2- 172.21.15.16:0/871111923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c4114510 0x7f78c41a60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:21.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.275+0000 7f78c2000640 1 -- 172.21.15.16:0/871111923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78c40776e0 msgr2=0x7f78c41a0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:21.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.275+0000 7f78c2000640 1 --2- 172.21.15.16:0/871111923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78c40776e0 0x7f78c41a0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:21.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.275+0000 7f78c2000640 1 -- 172.21.15.16:0/871111923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c4114510 msgr2=0x7f78c41a60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:21.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.275+0000 7f78c2000640 1 --2- 172.21.15.16:0/871111923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c4114510 0x7f78c41a60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:21.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.275+0000 7f78c2000640 1 -- 172.21.15.16:0/871111923 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c41a67d0 con 0x7f78c4078030 2024-08-31T06:20:21.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.275+0000 7f78c3400640 1 --2- 172.21.15.16:0/871111923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c4114510 0x7f78c41a60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:20:21.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.276+0000 7f78c2000640 1 --2- 172.21.15.16:0/871111923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78c4078030 0x7f78c41a0e00 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f78c406b870 tx=0x7f78ac00ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:21.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.277+0000 7f78bb400640 1 -- 172.21.15.16:0/871111923 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78ac00cd00 con 0x7f78c4078030 2024-08-31T06:20:21.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.277+0000 7f78c9c00640 1 -- 172.21.15.16:0/871111923 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78c41a6ac0 con 0x7f78c4078030 2024-08-31T06:20:21.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.277+0000 7f78bb400640 1 -- 172.21.15.16:0/871111923 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78ac0035f0 con 0x7f78c4078030 2024-08-31T06:20:21.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.277+0000 7f78c9c00640 1 -- 172.21.15.16:0/871111923 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78c41ad220 con 0x7f78c4078030 2024-08-31T06:20:21.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.277+0000 7f78bb400640 1 -- 172.21.15.16:0/871111923 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78ac00cd00 con 0x7f78c4078030 2024-08-31T06:20:21.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.283+0000 7f78d08b6740 1 Processor -- start 2024-08-31T06:20:21.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.283+0000 7f78d08b6740 1 -- start start 2024-08-31T06:20:21.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.283+0000 7f78d08b6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eec35c5840 0x55eec396c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:21.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.284+0000 7f78d08b6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eec359c980 0x55eec396c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:21.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.284+0000 7f78d08b6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eec35a4b60 0x55eec396cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:21.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.284+0000 7f78d08b6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55eec38e1530 con 0x55eec35c5840 2024-08-31T06:20:21.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.284+0000 7f78d08b6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55eec38d0ac0 con 0x55eec359c980 2024-08-31T06:20:21.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.284+0000 7f78d08b6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55eec38d4a70 con 0x55eec35a4b60 2024-08-31T06:20:21.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.284+0000 7f78bb400640 1 -- 172.21.15.16:0/871111923 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f78ac004020 con 0x7f78c4078030 2024-08-31T06:20:21.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.284+0000 7f78c2a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eec359c980 0x55eec396c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:21.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.284+0000 7f78c2a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eec359c980 0x55eec396c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:41680/0 (socket says 172.21.15.16:41680) 2024-08-31T06:20:21.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.284+0000 7f78c2a00640 1 -- 172.21.15.16:0/1580286015 learned_addr learned my addr 172.21.15.16:0/1580286015 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:21.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.284+0000 7f78c3400640 1 --2- 172.21.15.16:0/1580286015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eec35c5840 0x55eec396c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:21.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.284+0000 7f78c2a00640 1 -- 172.21.15.16:0/1580286015 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eec35a4b60 msgr2=0x55eec396cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:21.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.284+0000 7f78c2a00640 1 --2- 172.21.15.16:0/1580286015 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eec35a4b60 0x55eec396cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:21.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.284+0000 7f78c2a00640 1 -- 172.21.15.16:0/1580286015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eec35c5840 msgr2=0x55eec396c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:21.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.284+0000 7f78c2a00640 1 --2- 172.21.15.16:0/1580286015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eec35c5840 0x55eec396c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:21.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.284+0000 7f78c2a00640 1 -- 172.21.15.16:0/1580286015 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eec3883880 con 0x55eec359c980 2024-08-31T06:20:21.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.285+0000 7f78c3400640 1 --2- 172.21.15.16:0/1580286015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eec35c5840 0x55eec396c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:20:21.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.286+0000 7f78c2a00640 1 --2- 172.21.15.16:0/1580286015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eec359c980 0x55eec396c650 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f78bc009480 tx=0x7f78bc009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc637ed51ead5cc5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:21.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.286+0000 7f78bb400640 1 --2- 172.21.15.16:0/871111923 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f78a80b4f20 0x7f78a80b7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:21.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.286+0000 7f78bb400640 1 -- 172.21.15.16:0/871111923 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8470+0+0 (secure 0 0 0) 0x7f78ac0e4da0 con 0x7f78c4078030 2024-08-31T06:20:21.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.287+0000 7f78b3e00640 1 -- 172.21.15.16:0/1580286015 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78bc031db0 con 0x55eec359c980 2024-08-31T06:20:21.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.287+0000 7f78b3e00640 1 -- 172.21.15.16:0/1580286015 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78bc036030 con 0x55eec359c980 2024-08-31T06:20:21.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.287+0000 7f78b3e00640 1 -- 172.21.15.16:0/1580286015 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78bc0042a0 con 0x55eec359c980 2024-08-31T06:20:21.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.287+0000 7f78c3400640 1 --2- 172.21.15.16:0/871111923 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f78a80b4f20 0x7f78a80b7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:21.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.288+0000 7f78d08b6740 1 -- 172.21.15.16:0/1580286015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eec359c980 msgr2=0x55eec396c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:21.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.288+0000 7f78d08b6740 1 --2- 172.21.15.16:0/1580286015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eec359c980 0x55eec396c650 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f78bc009480 tx=0x7f78bc009840 comp rx=0 tx=0).stop 2024-08-31T06:20:21.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.288+0000 7f78c3400640 1 --2- 172.21.15.16:0/871111923 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f78a80b4f20 0x7f78a80b7430 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f78b4009d40 tx=0x7f78b4005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:21.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.288+0000 7f78d08b6740 1 -- 172.21.15.16:0/1580286015 shutdown_connections 2024-08-31T06:20:21.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.288+0000 7f78d08b6740 1 --2- 172.21.15.16:0/1580286015 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eec35a4b60 0x55eec396cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:21.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.288+0000 7f78d08b6740 1 --2- 172.21.15.16:0/1580286015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eec359c980 0x55eec396c650 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:21.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.288+0000 7f78d08b6740 1 --2- 172.21.15.16:0/1580286015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eec35c5840 0x55eec396c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:21.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.288+0000 7f78d08b6740 1 -- 172.21.15.16:0/1580286015 >> 172.21.15.16:0/1580286015 conn(0x55eec35b9990 msgr2=0x55eec396a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:21.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.289+0000 7f78d08b6740 1 -- 172.21.15.16:0/1580286015 shutdown_connections 2024-08-31T06:20:21.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.289+0000 7f78d08b6740 1 -- 172.21.15.16:0/1580286015 wait complete. 2024-08-31T06:20:21.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.289+0000 7f78d08b6740 1 -- start start 2024-08-31T06:20:21.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.290+0000 7f78d08b6740 1 Processor -- start 2024-08-31T06:20:21.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.291+0000 7f78d08b6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eec359c980 0x55eec3977de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:21.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.291+0000 7f78d08b6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eec35a4b60 0x55eec39783c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:21.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.291+0000 7f78d08b6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eec35c5840 0x55eec3983a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:21.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.291+0000 7f78d08b6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55eec37337a0 con 0x55eec35c5840 2024-08-31T06:20:21.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.291+0000 7f78d08b6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55eec38577f0 con 0x55eec359c980 2024-08-31T06:20:21.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.291+0000 7f78d08b6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55eec397a710 con 0x55eec35a4b60 2024-08-31T06:20:21.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.291+0000 7f78d08b6740 10 client.0 _ll_register_callbacks cb 0x55eec387ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:20:21.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.291+0000 7f78c2000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eec35a4b60 0x55eec39783c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:21.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.291+0000 7f78c2000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eec35a4b60 0x55eec39783c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:57534/0 (socket says 172.21.15.16:57534) 2024-08-31T06:20:21.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.291+0000 7f78c2000640 1 -- 172.21.15.16:0/915193202 learned_addr learned my addr 172.21.15.16:0/915193202 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:21.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.291+0000 7f78c3400640 1 --2- 172.21.15.16:0/915193202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eec35c5840 0x55eec3983a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:21.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.291+0000 7f78c2a00640 1 --2- 172.21.15.16:0/915193202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eec359c980 0x55eec3977de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:21.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.292+0000 7f78c2000640 1 -- 172.21.15.16:0/915193202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eec359c980 msgr2=0x55eec3977de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:21.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.292+0000 7f78c2000640 1 --2- 172.21.15.16:0/915193202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eec359c980 0x55eec3977de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:21.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.292+0000 7f78c2000640 1 -- 172.21.15.16:0/915193202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eec35c5840 msgr2=0x55eec3983a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:21.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.292+0000 7f78c2000640 1 --2- 172.21.15.16:0/915193202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eec35c5840 0x55eec3983a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:21.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.292+0000 7f78c2000640 1 -- 172.21.15.16:0/915193202 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eec363fb90 con 0x55eec35a4b60 2024-08-31T06:20:21.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.292+0000 7f78c3400640 1 --2- 172.21.15.16:0/915193202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eec35c5840 0x55eec3983a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:20:21.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.292+0000 7f78c2a00640 1 --2- 172.21.15.16:0/915193202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eec359c980 0x55eec3977de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:20:21.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.292+0000 7f78c2000640 1 --2- 172.21.15.16:0/915193202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eec35a4b60 0x55eec39783c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f78ac0a7540 tx=0x7f78ac0aafd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49df327d115be181 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:21.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.292+0000 7f78b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:20:21.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.293+0000 7f78b0c00640 1 -- 172.21.15.16:0/915193202 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78ac0d21c0 con 0x55eec35a4b60 2024-08-31T06:20:21.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.293+0000 7f78b0c00640 1 -- 172.21.15.16:0/915193202 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8470+0+0 (secure 0 0 0) 0x7f78ac0037a0 con 0x55eec35a4b60 2024-08-31T06:20:21.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.294+0000 7f78d08b6740 1 -- 172.21.15.16:0/915193202 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78bc036030 con 0x55eec35a4b60 2024-08-31T06:20:21.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.294+0000 7f78b0c00640 1 -- 172.21.15.16:0/915193202 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f78ac0ab4c0 con 0x55eec35a4b60 2024-08-31T06:20:21.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.294+0000 7f78d08b6740 10 client.28588 fetch_fsmap learned FSMap version 49 2024-08-31T06:20:21.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.294+0000 7f78d08b6740 1 -- 172.21.15.16:0/915193202 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55eec3986830 con 0x55eec35a4b60 2024-08-31T06:20:21.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.294+0000 7f78b0c00640 1 -- 172.21.15.16:0/915193202 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78bc036030 con 0x55eec35a4b60 2024-08-31T06:20:21.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.296+0000 7f78b0c00640 1 -- 172.21.15.16:0/915193202 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f78ac0ce0d0 con 0x55eec35a4b60 2024-08-31T06:20:21.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.296+0000 7f78d08b6740 10 client.28588 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:20:21.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.296+0000 7f78d08b6740 10 client.28588 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:20:21.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.296+0000 7f78d08b6740 20 client.28588 populate_metadata read hostname 'smithi016' 2024-08-31T06:20:21.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.296+0000 7f78d08b6740 1 --2- 172.21.15.16:0/915193202 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55eec35841c0 0x55eec3989d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:21.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.296+0000 7f78d08b6740 4 client.28588 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:20:21.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.296+0000 7f78d08b6740 1 -- 172.21.15.16:0/915193202 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eec38e3480 con 0x55eec35841c0 2024-08-31T06:20:21.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.297+0000 7f78c2a00640 1 --2- 172.21.15.16:0/915193202 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55eec35841c0 0x55eec3989d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:21.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.298+0000 7f78c2a00640 1 --2- 172.21.15.16:0/915193202 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55eec35841c0 0x55eec3989d80 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e49b4331885e91d server_cookie=fe80bc58f19bd3eb in_seq=0 out_seq=0 2024-08-31T06:20:21.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.298+0000 7f78b0c00640 10 client.28588 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:20:21.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.304+0000 7f78b0c00640 1 -- 172.21.15.16:0/915193202 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f78bc003970 con 0x55eec35841c0 2024-08-31T06:20:21.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.304+0000 7f78b0c00640 10 client.28588 handle_command_reply: tid=0 2024-08-31T06:20:21.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.304+0000 7f78d08b6740 1 client.28588 shutdown 2024-08-31T06:20:21.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.305+0000 7f78d08b6740 1 -- 172.21.15.16:0/915193202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eec35a4b60 msgr2=0x55eec39783c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:21.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.305+0000 7f78d08b6740 1 --2- 172.21.15.16:0/915193202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eec35a4b60 0x55eec39783c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f78ac0a7540 tx=0x7f78ac0aafd0 comp rx=0 tx=0).stop 2024-08-31T06:20:21.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.307+0000 7f78d08b6740 1 -- 172.21.15.16:0/915193202 shutdown_connections 2024-08-31T06:20:21.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.307+0000 7f78d08b6740 1 --2- 172.21.15.16:0/915193202 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55eec35841c0 0x55eec3989d80 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:21.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.307+0000 7f78d08b6740 1 --2- 172.21.15.16:0/915193202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eec35a4b60 0x55eec39783c0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:21.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.307+0000 7f78d08b6740 1 --2- 172.21.15.16:0/915193202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eec35c5840 0x55eec3983a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:21.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.307+0000 7f78d08b6740 1 --2- 172.21.15.16:0/915193202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eec359c980 0x55eec3977de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:21.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.307+0000 7f78d08b6740 1 -- 172.21.15.16:0/915193202 >> 172.21.15.16:0/915193202 conn(0x55eec35b9990 msgr2=0x55eec39687e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:21.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.307+0000 7f78b0c00640 0 client.28588 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:20:21.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.308+0000 7f78d08b6740 1 -- 172.21.15.16:0/915193202 shutdown_connections 2024-08-31T06:20:21.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.308+0000 7f78d08b6740 1 -- 172.21.15.16:0/915193202 wait complete. 2024-08-31T06:20:21.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.308+0000 7f78d08b6740 20 client.28588 trim_cache size 0 max 16384 2024-08-31T06:20:21.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.347+0000 7f78d08b6740 1 Processor -- start 2024-08-31T06:20:21.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.347+0000 7f78d08b6740 1 -- start start 2024-08-31T06:20:21.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.347+0000 7f78d08b6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eec359c980 0x55eec3984070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:21.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.347+0000 7f78d08b6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eec35a4b60 0x55eec39845b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:21.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.347+0000 7f78d08b6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eec35c5840 0x55eec39687e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:21.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.347+0000 7f78d08b6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f78c41b82d0 con 0x55eec35a4b60 2024-08-31T06:20:21.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.348+0000 7f78d08b6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f78c41b5ae0 con 0x55eec35c5840 2024-08-31T06:20:21.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.348+0000 7f78d08b6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f78c41b3950 con 0x55eec359c980 2024-08-31T06:20:21.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.349+0000 7f78c3400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eec35c5840 0x55eec39687e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:21.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.349+0000 7f78c3400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eec35c5840 0x55eec39687e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:41698/0 (socket says 172.21.15.16:41698) 2024-08-31T06:20:21.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.349+0000 7f78c3400640 1 -- 172.21.15.16:0/3628043952 learned_addr learned my addr 172.21.15.16:0/3628043952 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:21.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.349+0000 7f78c2000640 1 --2- 172.21.15.16:0/3628043952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eec35a4b60 0x55eec39845b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:21.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.350+0000 7f78c3400640 1 -- 172.21.15.16:0/3628043952 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eec359c980 msgr2=0x55eec3984070 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T06:20:21.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.350+0000 7f78c3400640 1 --2- 172.21.15.16:0/3628043952 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eec359c980 0x55eec3984070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:21.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.350+0000 7f78c3400640 1 -- 172.21.15.16:0/3628043952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eec35a4b60 msgr2=0x55eec39845b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:21.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.350+0000 7f78c3400640 1 --2- 172.21.15.16:0/3628043952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eec35a4b60 0x55eec39845b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:21.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.350+0000 7f78c3400640 1 -- 172.21.15.16:0/3628043952 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eec3964f00 con 0x55eec35c5840 2024-08-31T06:20:21.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.350+0000 7f78c2000640 1 --2- 172.21.15.16:0/3628043952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eec35a4b60 0x55eec39845b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:20:21.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.352+0000 7f78c3400640 1 --2- 172.21.15.16:0/3628043952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eec35c5840 0x55eec39687e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f78b4009680 tx=0x7f78b4016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1edf095d0e76320 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:21.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.353+0000 7f78b3400640 1 -- 172.21.15.16:0/3628043952 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b402d070 con 0x55eec35c5840 2024-08-31T06:20:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.355+0000 7f78b3400640 1 -- 172.21.15.16:0/3628043952 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78b4020d40 con 0x55eec35c5840 2024-08-31T06:20:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.355+0000 7f78b3400640 1 -- 172.21.15.16:0/3628043952 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b4028810 con 0x55eec35c5840 2024-08-31T06:20:21.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.356+0000 7f78d08b6740 1 -- 172.21.15.16:0/3628043952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eec35c5840 msgr2=0x55eec39687e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:21.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.356+0000 7f78d08b6740 1 --2- 172.21.15.16:0/3628043952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eec35c5840 0x55eec39687e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f78b4009680 tx=0x7f78b4016000 comp rx=0 tx=0).stop 2024-08-31T06:20:21.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.358+0000 7f78d08b6740 1 -- 172.21.15.16:0/3628043952 shutdown_connections 2024-08-31T06:20:21.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.358+0000 7f78d08b6740 1 --2- 172.21.15.16:0/3628043952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eec35c5840 0x55eec39687e0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:21.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.358+0000 7f78d08b6740 1 --2- 172.21.15.16:0/3628043952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eec35a4b60 0x55eec39845b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:21.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.358+0000 7f78d08b6740 1 --2- 172.21.15.16:0/3628043952 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eec359c980 0x55eec3984070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:21.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.358+0000 7f78d08b6740 1 -- 172.21.15.16:0/3628043952 >> 172.21.15.16:0/3628043952 conn(0x55eec35b9990 msgr2=0x55eec3976460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:21.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.360+0000 7f78d08b6740 1 -- 172.21.15.16:0/3628043952 shutdown_connections 2024-08-31T06:20:21.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.360+0000 7f78d08b6740 1 -- 172.21.15.16:0/3628043952 wait complete. 2024-08-31T06:20:21.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.361+0000 7f78d08b6740 1 -- start start 2024-08-31T06:20:21.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.361+0000 7f78d08b6740 1 Processor -- start 2024-08-31T06:20:21.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.362+0000 7f78d08b6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eec359c980 0x55eec397f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:21.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.363+0000 7f78d08b6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eec35a4b60 0x55eec397feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:21.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.363+0000 7f78d08b6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eec35c5840 0x55eec397a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:21.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.363+0000 7f78c2a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eec359c980 0x55eec397f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:21.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.363+0000 7f78c2a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eec359c980 0x55eec397f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:41708/0 (socket says 172.21.15.16:41708) 2024-08-31T06:20:21.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.363+0000 7f78c2000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eec35a4b60 0x55eec397feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:21.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.363+0000 7f78c2000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eec35a4b60 0x55eec397feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:57554/0 (socket says 172.21.15.16:57554) 2024-08-31T06:20:21.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.363+0000 7f78c2a00640 1 -- 172.21.15.16:0/3116931218 learned_addr learned my addr 172.21.15.16:0/3116931218 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:21.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.363+0000 7f78d08b6740 1 -- 172.21.15.16:0/3116931218 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f78c41b3fa0 con 0x55eec35c5840 2024-08-31T06:20:21.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.363+0000 7f78d08b6740 1 -- 172.21.15.16:0/3116931218 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f78c41b5410 con 0x55eec359c980 2024-08-31T06:20:21.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.363+0000 7f78d08b6740 1 -- 172.21.15.16:0/3116931218 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f78c41b4670 con 0x55eec35a4b60 2024-08-31T06:20:21.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.364+0000 7f78d08b6740 10 client.0 _ll_register_callbacks cb 0x7f78ac0aae10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:20:21.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.364+0000 7f78c3400640 1 --2- 172.21.15.16:0/3116931218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eec35c5840 0x55eec397a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:21.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.364+0000 7f78c2a00640 1 -- 172.21.15.16:0/3116931218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eec35a4b60 msgr2=0x55eec397feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:21.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.364+0000 7f78c2a00640 1 --2- 172.21.15.16:0/3116931218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eec35a4b60 0x55eec397feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:21.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.364+0000 7f78c2a00640 1 -- 172.21.15.16:0/3116931218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eec35c5840 msgr2=0x55eec397a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:21.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.364+0000 7f78c2a00640 1 --2- 172.21.15.16:0/3116931218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eec35c5840 0x55eec397a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:21.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.364+0000 7f78c2a00640 1 -- 172.21.15.16:0/3116931218 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eec397ae60 con 0x55eec359c980 2024-08-31T06:20:21.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.365+0000 7f78c2a00640 1 --2- 172.21.15.16:0/3116931218 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eec359c980 0x55eec397f970 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f78bc003de0 tx=0x7f78bc02a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9871602994fd7068 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:21.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.365+0000 7f78b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:20:21.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.366+0000 7f78b0c00640 1 -- 172.21.15.16:0/3116931218 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78bc055070 con 0x55eec359c980 2024-08-31T06:20:21.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.366+0000 7f78b0c00640 1 -- 172.21.15.16:0/3116931218 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8470+0+0 (secure 0 0 0) 0x7f78bc003970 con 0x55eec359c980 2024-08-31T06:20:21.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.366+0000 7f78d08b6740 1 -- 172.21.15.16:0/3116931218 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78b4020d40 con 0x55eec359c980 2024-08-31T06:20:21.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.367+0000 7f78b0c00640 1 -- 172.21.15.16:0/3116931218 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f78bc031bb0 con 0x55eec359c980 2024-08-31T06:20:21.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.367+0000 7f78d08b6740 10 client.28589 fetch_fsmap learned FSMap version 49 2024-08-31T06:20:21.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.367+0000 7f78d08b6740 1 -- 172.21.15.16:0/3116931218 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55eec39f4d70 con 0x55eec359c980 2024-08-31T06:20:21.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.368+0000 7f78b0c00640 1 -- 172.21.15.16:0/3116931218 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78b4020d40 con 0x55eec359c980 2024-08-31T06:20:21.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.368+0000 7f78b0c00640 1 -- 172.21.15.16:0/3116931218 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f78bc043070 con 0x55eec359c980 2024-08-31T06:20:21.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.368+0000 7f78d08b6740 10 client.28589 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:20:21.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.369+0000 7f78d08b6740 10 client.28589 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:20:21.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.369+0000 7f78d08b6740 20 client.28589 populate_metadata read hostname 'smithi016' 2024-08-31T06:20:21.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.369+0000 7f78d08b6740 1 --2- 172.21.15.16:0/3116931218 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55eec3559570 0x55eec39f72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:21.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.369+0000 7f78d08b6740 4 client.28589 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 134.7, "await_for": 45.0}] 2024-08-31T06:20:21.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.369+0000 7f78d08b6740 1 -- 172.21.15.16:0/3116931218 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 134.7, "await_for": 45.0}) -- 0x55eec38def20 con 0x55eec3559570 2024-08-31T06:20:21.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.369+0000 7f78c2000640 1 --2- 172.21.15.16:0/3116931218 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55eec3559570 0x55eec39f72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:21.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.374+0000 7f78c2000640 1 --2- 172.21.15.16:0/3116931218 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55eec3559570 0x55eec39f72c0 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa8660f88e79ed70 server_cookie=118a6b1be6d29dbd in_seq=0 out_seq=0 2024-08-31T06:20:21.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.374+0000 7f78b0c00640 10 client.28589 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:20:21.462 INFO:tasks.cephfs.filesystem:scrub status for tag:18e112f4-a5f9-4602-aaad-3b13fa5a4c2e - {'path': '/', 'tag': '18e112f4-a5f9-4602-aaad-3b13fa5a4c2e', 'options': 'recursive,force'} 2024-08-31T06:20:21.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.589+0000 7f78b0c00640 1 -- 172.21.15.16:0/3116931218 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55eec38def20 con 0x55eec3559570 2024-08-31T06:20:21.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.589+0000 7f78b0c00640 10 client.28589 handle_command_reply: tid=0 2024-08-31T06:20:21.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.589+0000 7f78d08b6740 1 client.28589 shutdown 2024-08-31T06:20:21.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.590+0000 7f78d08b6740 1 -- 172.21.15.16:0/3116931218 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eec359c980 msgr2=0x55eec397f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:21.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.590+0000 7f78d08b6740 1 --2- 172.21.15.16:0/3116931218 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eec359c980 0x55eec397f970 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f78bc003de0 tx=0x7f78bc02a8c0 comp rx=0 tx=0).stop 2024-08-31T06:20:21.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.590+0000 7f78d08b6740 1 -- 172.21.15.16:0/3116931218 shutdown_connections 2024-08-31T06:20:21.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.590+0000 7f78d08b6740 1 --2- 172.21.15.16:0/3116931218 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55eec3559570 0x55eec39f72c0 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:21.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.590+0000 7f78d08b6740 1 --2- 172.21.15.16:0/3116931218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eec35a4b60 0x55eec397feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:21.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.590+0000 7f78d08b6740 1 --2- 172.21.15.16:0/3116931218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eec35c5840 0x55eec397a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:21.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.590+0000 7f78b0c00640 0 client.28589 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:20:21.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.590+0000 7f78d08b6740 1 --2- 172.21.15.16:0/3116931218 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eec359c980 0x55eec397f970 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:21.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.590+0000 7f78d08b6740 1 -- 172.21.15.16:0/3116931218 >> 172.21.15.16:0/3116931218 conn(0x55eec35b9990 msgr2=0x55eec39762f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:21.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.590+0000 7f78d08b6740 1 -- 172.21.15.16:0/3116931218 shutdown_connections 2024-08-31T06:20:21.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.590+0000 7f78d08b6740 1 -- 172.21.15.16:0/3116931218 wait complete. 2024-08-31T06:20:21.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.590+0000 7f78d08b6740 20 client.28589 trim_cache size 0 max 16384 2024-08-31T06:20:21.595 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:20:21.595 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 49, 2024-08-31T06:20:21.595 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24737, 2024-08-31T06:20:21.595 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 42, 2024-08-31T06:20:21.595 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-08-31T06:20:21.595 INFO:teuthology.orchestra.run.smithi016.stdout: "8fb05a68": { 2024-08-31T06:20:21.595 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 42, 2024-08-31T06:20:21.595 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-08-31T06:20:21.595 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:20:21.595 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-08-31T06:20:21.596 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:20:21.596 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:20:21.596 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-08-31T06:20:21.596 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 134.7, 2024-08-31T06:20:21.596 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-08-31T06:20:21.596 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-08-31T06:20:21.596 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-08-31T06:20:21.596 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:20:21.596 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-08-31T06:20:21.596 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:20:21.596 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:20:21.596 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:20:21.596 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:20:21.597 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:20:21.597 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:20:21.597 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:20:21.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.591+0000 7f78c9c00640 1 -- 172.21.15.16:0/871111923 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f78a80b4f20 msgr2=0x7f78a80b7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:21.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.591+0000 7f78c9c00640 1 --2- 172.21.15.16:0/871111923 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f78a80b4f20 0x7f78a80b7430 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f78b4009d40 tx=0x7f78b4005f90 comp rx=0 tx=0).stop 2024-08-31T06:20:21.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.591+0000 7f78c9c00640 1 -- 172.21.15.16:0/871111923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78c4078030 msgr2=0x7f78c41a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:21.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.591+0000 7f78c9c00640 1 --2- 172.21.15.16:0/871111923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78c4078030 0x7f78c41a0e00 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f78c406b870 tx=0x7f78ac00ee50 comp rx=0 tx=0).stop 2024-08-31T06:20:21.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.591+0000 7f78c9c00640 1 -- 172.21.15.16:0/871111923 shutdown_connections 2024-08-31T06:20:21.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.591+0000 7f78c9c00640 1 --2- 172.21.15.16:0/871111923 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f78a80b4f20 0x7f78a80b7430 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:21.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.591+0000 7f78c9c00640 1 --2- 172.21.15.16:0/871111923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c4114510 0x7f78c41a60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:21.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.591+0000 7f78c9c00640 1 --2- 172.21.15.16:0/871111923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78c4078030 0x7f78c41a0e00 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:21.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.591+0000 7f78c9c00640 1 --2- 172.21.15.16:0/871111923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78c40776e0 0x7f78c41a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:21.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.591+0000 7f78c9c00640 1 -- 172.21.15.16:0/871111923 >> 172.21.15.16:0/871111923 conn(0x7f78c41015a0 msgr2=0x7f78c41165e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:21.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.591+0000 7f78c9c00640 1 -- 172.21.15.16:0/871111923 shutdown_connections 2024-08-31T06:20:21.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:21.591+0000 7f78c9c00640 1 -- 172.21.15.16:0/871111923 wait complete. 2024-08-31T06:20:21.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 775.0 s, 1566.2 tps, lat 59.710 ms stddev 71.543 2024-08-31T06:20:22.379 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '8fb05a68', quiesce took 7.4 seconds. Will release after: 7.299999999999999 2024-08-31T06:20:22.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:22 smithi184 ceph-mon[22331]: pgmap v774: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 5.7 MiB/s wr, 740 op/s 2024-08-31T06:20:23.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:22 smithi155 ceph-mon[21742]: pgmap v774: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 5.7 MiB/s wr, 740 op/s 2024-08-31T06:20:23.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:22 smithi016 ceph-mon[17194]: pgmap v774: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 5.7 MiB/s wr, 740 op/s 2024-08-31T06:20:23.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:23 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:20:24.064 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:23 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:20:24.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:23 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:20:24.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:24 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:20:24.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:24 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:20:24.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:24 smithi184 ceph-mon[22331]: pgmap v775: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 6.6 MiB/s wr, 795 op/s 2024-08-31T06:20:25.092 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:24 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:20:25.092 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:24 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:20:25.092 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:24 smithi016 ceph-mon[17194]: pgmap v775: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 6.6 MiB/s wr, 795 op/s 2024-08-31T06:20:25.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:24 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:20:25.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:24 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:20:25.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:24 smithi155 ceph-mon[21742]: pgmap v775: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 6.6 MiB/s wr, 795 op/s 2024-08-31T06:20:26.628 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 780.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-31T06:20:26.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:26 smithi016 ceph-mon[17194]: pgmap v776: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 4.7 MiB/s wr, 621 op/s 2024-08-31T06:20:26.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:26 smithi184 ceph-mon[22331]: pgmap v776: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 4.7 MiB/s wr, 621 op/s 2024-08-31T06:20:27.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:26 smithi155 ceph-mon[21742]: pgmap v776: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 4.7 MiB/s wr, 621 op/s 2024-08-31T06:20:28.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:28 smithi184 ceph-mon[22331]: pgmap v777: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 4.2 MiB/s wr, 528 op/s 2024-08-31T06:20:29.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:28 smithi155 ceph-mon[21742]: pgmap v777: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 4.2 MiB/s wr, 528 op/s 2024-08-31T06:20:29.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:28 smithi016 ceph-mon[17194]: pgmap v777: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 4.2 MiB/s wr, 528 op/s 2024-08-31T06:20:29.680 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:20:30.097 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:20:30.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:30 smithi184 ceph-mon[22331]: pgmap v778: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 5.6 MiB/s wr, 640 op/s 2024-08-31T06:20:31.065 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:30 smithi016 ceph-mon[17194]: pgmap v778: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 5.6 MiB/s wr, 640 op/s 2024-08-31T06:20:31.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:30 smithi155 ceph-mon[21742]: pgmap v778: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 5.6 MiB/s wr, 640 op/s 2024-08-31T06:20:31.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.248+0000 7f63ea400640 1 -- 172.21.15.16:0/3519070218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63e4075ea0 msgr2=0x7f63e410f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:31.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.248+0000 7f63ea400640 1 --2- 172.21.15.16:0/3519070218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63e4075ea0 0x7f63e410f920 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f63d0009900 tx=0x7f63d002a410 comp rx=0 tx=0).stop 2024-08-31T06:20:31.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.248+0000 7f63ea400640 1 -- 172.21.15.16:0/3519070218 shutdown_connections 2024-08-31T06:20:31.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.248+0000 7f63ea400640 1 --2- 172.21.15.16:0/3519070218 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63e410fe60 0x7f63e41122a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:31.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.248+0000 7f63ea400640 1 --2- 172.21.15.16:0/3519070218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63e4075ea0 0x7f63e410f920 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:31.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.248+0000 7f63ea400640 1 --2- 172.21.15.16:0/3519070218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63e4075550 0x7f63e4075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:31.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.248+0000 7f63ea400640 1 -- 172.21.15.16:0/3519070218 >> 172.21.15.16:0/3519070218 conn(0x7f63e407b5f0 msgr2=0x7f63e407ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:31.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.248+0000 7f63ea400640 1 -- 172.21.15.16:0/3519070218 shutdown_connections 2024-08-31T06:20:31.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.248+0000 7f63ea400640 1 -- 172.21.15.16:0/3519070218 wait complete. 2024-08-31T06:20:31.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.249+0000 7f63ea400640 1 Processor -- start 2024-08-31T06:20:31.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.249+0000 7f63ea400640 1 -- start start 2024-08-31T06:20:31.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.250+0000 7f63ea400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63e4075550 0x7f63e419c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:31.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.250+0000 7f63ea400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63e4075ea0 0x7f63e419c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:31.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.250+0000 7f63ea400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63e410fe60 0x7f63e41a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:31.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.250+0000 7f63ea400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f63e4115a20 con 0x7f63e4075ea0 2024-08-31T06:20:31.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.250+0000 7f63ea400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f63e41158a0 con 0x7f63e4075550 2024-08-31T06:20:31.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.250+0000 7f63ea400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f63e4115ba0 con 0x7f63e410fe60 2024-08-31T06:20:31.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.250+0000 7f63e2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63e4075ea0 0x7f63e419c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:31.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.250+0000 7f63e3400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63e4075550 0x7f63e419c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:31.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.250+0000 7f63e2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63e4075ea0 0x7f63e419c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51042/0 (socket says 172.21.15.16:51042) 2024-08-31T06:20:31.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.250+0000 7f63e2a00640 1 -- 172.21.15.16:0/3408437250 learned_addr learned my addr 172.21.15.16:0/3408437250 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:31.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.250+0000 7f63e3e00640 1 --2- 172.21.15.16:0/3408437250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63e410fe60 0x7f63e41a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:31.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.251+0000 7f63e2a00640 1 -- 172.21.15.16:0/3408437250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63e410fe60 msgr2=0x7f63e41a1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:31.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.251+0000 7f63e2a00640 1 --2- 172.21.15.16:0/3408437250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63e410fe60 0x7f63e41a1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:31.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.251+0000 7f63e2a00640 1 -- 172.21.15.16:0/3408437250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63e4075550 msgr2=0x7f63e419c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:31.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.251+0000 7f63e2a00640 1 --2- 172.21.15.16:0/3408437250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63e4075550 0x7f63e419c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:31.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.251+0000 7f63e2a00640 1 -- 172.21.15.16:0/3408437250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e41a2380 con 0x7f63e4075ea0 2024-08-31T06:20:31.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.251+0000 7f63e3400640 1 --2- 172.21.15.16:0/3408437250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63e4075550 0x7f63e419c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:20:31.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.251+0000 7f63e2a00640 1 --2- 172.21.15.16:0/3408437250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63e4075ea0 0x7f63e419c9b0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f63d0005ab0 tx=0x7f63d0003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:31.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.252+0000 7f63dbe00640 1 -- 172.21.15.16:0/3408437250 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d003f070 con 0x7f63e4075ea0 2024-08-31T06:20:31.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.252+0000 7f63dbe00640 1 -- 172.21.15.16:0/3408437250 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63d0031b50 con 0x7f63e4075ea0 2024-08-31T06:20:31.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.252+0000 7f63dbe00640 1 -- 172.21.15.16:0/3408437250 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d0035030 con 0x7f63e4075ea0 2024-08-31T06:20:31.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.252+0000 7f63ea400640 1 -- 172.21.15.16:0/3408437250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63e41a2610 con 0x7f63e4075ea0 2024-08-31T06:20:31.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.252+0000 7f63ea400640 1 -- 172.21.15.16:0/3408437250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63e41a8dd0 con 0x7f63e4075ea0 2024-08-31T06:20:31.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.253+0000 7f63ea400640 1 -- 172.21.15.16:0/3408437250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63e4075960 con 0x7f63e4075ea0 2024-08-31T06:20:31.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.256+0000 7f63dbe00640 1 -- 172.21.15.16:0/3408437250 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f63d0042070 con 0x7f63e4075ea0 2024-08-31T06:20:31.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.257+0000 7f63dbe00640 1 --2- 172.21.15.16:0/3408437250 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f63c80b4d20 0x7f63c80b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:31.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.258+0000 7f63dbe00640 1 -- 172.21.15.16:0/3408437250 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8470+0+0 (secure 0 0 0) 0x7f63d0105ea0 con 0x7f63e4075ea0 2024-08-31T06:20:31.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.258+0000 7f63e3400640 1 --2- 172.21.15.16:0/3408437250 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f63c80b4d20 0x7f63c80b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:31.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.258+0000 7f63e3400640 1 --2- 172.21.15.16:0/3408437250 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f63c80b4d20 0x7f63c80b7230 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f63d4009fc0 tx=0x7f63d4005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:31.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.259+0000 7f63dbe00640 1 -- 172.21.15.16:0/3408437250 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f63d00cc610 con 0x7f63e4075ea0 2024-08-31T06:20:31.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.493+0000 7f63ea400640 1 -- 172.21.15.16:0/3408437250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63e410afe0 con 0x7f63e4075ea0 2024-08-31T06:20:31.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.494+0000 7f63dbe00640 1 -- 172.21.15.16:0/3408437250 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+11923 (secure 0 0 0) 0x7f63d00d0dd0 con 0x7f63e4075ea0 2024-08-31T06:20:31.496 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:20:31.497 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":49,"btime":"2024-08-31T06:19:17:811356+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":49,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:19:17.033815+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24737,24724,24712]},"id":1}]} 2024-08-31T06:20:31.497 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 49 2024-08-31T06:20:31.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.496+0000 7f63ea400640 1 -- 172.21.15.16:0/3408437250 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f63c80b4d20 msgr2=0x7f63c80b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:31.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.496+0000 7f63ea400640 1 --2- 172.21.15.16:0/3408437250 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f63c80b4d20 0x7f63c80b7230 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f63d4009fc0 tx=0x7f63d4005f90 comp rx=0 tx=0).stop 2024-08-31T06:20:31.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.496+0000 7f63ea400640 1 -- 172.21.15.16:0/3408437250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63e4075ea0 msgr2=0x7f63e419c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:31.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.496+0000 7f63ea400640 1 --2- 172.21.15.16:0/3408437250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63e4075ea0 0x7f63e419c9b0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f63d0005ab0 tx=0x7f63d0003b90 comp rx=0 tx=0).stop 2024-08-31T06:20:31.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.496+0000 7f63ea400640 1 -- 172.21.15.16:0/3408437250 shutdown_connections 2024-08-31T06:20:31.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.496+0000 7f63ea400640 1 --2- 172.21.15.16:0/3408437250 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f63c80b4d20 0x7f63c80b7230 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:31.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.496+0000 7f63ea400640 1 --2- 172.21.15.16:0/3408437250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63e410fe60 0x7f63e41a1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:31.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.496+0000 7f63ea400640 1 --2- 172.21.15.16:0/3408437250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63e4075ea0 0x7f63e419c9b0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:31.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.496+0000 7f63ea400640 1 --2- 172.21.15.16:0/3408437250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63e4075550 0x7f63e419c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:31.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.496+0000 7f63ea400640 1 -- 172.21.15.16:0/3408437250 >> 172.21.15.16:0/3408437250 conn(0x7f63e407b5f0 msgr2=0x7f63e4101620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:31.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.496+0000 7f63ea400640 1 -- 172.21.15.16:0/3408437250 shutdown_connections 2024-08-31T06:20:31.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:31.496+0000 7f63ea400640 1 -- 172.21.15.16:0/3408437250 wait complete. 2024-08-31T06:20:31.629 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 785.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-31T06:20:31.925 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:31 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/3408437250' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:20:31.972 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:31 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/3408437250' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:20:32.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:31 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/3408437250' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:20:32.265 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24737 quiesce db --set-id 8fb05a68 --release --await' 2024-08-31T06:20:32.266 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24737 quiesce db --set-id 8fb05a68 --release --await 2024-08-31T06:20:32.646 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:20:32.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:32 smithi184 ceph-mon[22331]: pgmap v779: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 9.5 MiB/s wr, 974 op/s 2024-08-31T06:20:32.976 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:32 smithi016 ceph-mon[17194]: pgmap v779: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 9.5 MiB/s wr, 974 op/s 2024-08-31T06:20:33.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:32 smithi155 ceph-mon[21742]: pgmap v779: 97 pgs: 97 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 9.5 MiB/s wr, 974 op/s 2024-08-31T06:20:33.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.284+0000 7ff2c0800640 1 -- 172.21.15.16:0/4274056832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2b8105b20 msgr2=0x7ff2b8105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:33.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.284+0000 7ff2c0800640 1 --2- 172.21.15.16:0/4274056832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2b8105b20 0x7ff2b8105f30 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac009900 tx=0x7ff2ac02a3f0 comp rx=0 tx=0).stop 2024-08-31T06:20:33.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.284+0000 7ff2c0800640 1 -- 172.21.15.16:0/4274056832 shutdown_connections 2024-08-31T06:20:33.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.284+0000 7ff2c0800640 1 --2- 172.21.15.16:0/4274056832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2b810bdb0 0x7ff2b81125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:33.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.284+0000 7ff2c0800640 1 --2- 172.21.15.16:0/4274056832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2b810b460 0x7ff2b810b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:33.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.284+0000 7ff2c0800640 1 --2- 172.21.15.16:0/4274056832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2b8105b20 0x7ff2b8105f30 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:33.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.284+0000 7ff2c0800640 1 -- 172.21.15.16:0/4274056832 >> 172.21.15.16:0/4274056832 conn(0x7ff2b81015b0 msgr2=0x7ff2b8103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:33.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.285+0000 7ff2c0800640 1 -- 172.21.15.16:0/4274056832 shutdown_connections 2024-08-31T06:20:33.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.285+0000 7ff2c0800640 1 -- 172.21.15.16:0/4274056832 wait complete. 2024-08-31T06:20:33.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.286+0000 7ff2c0800640 1 Processor -- start 2024-08-31T06:20:33.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.286+0000 7ff2c0800640 1 -- start start 2024-08-31T06:20:33.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.286+0000 7ff2c0800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2b8105b20 0x7ff2b819c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:33.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.286+0000 7ff2c0800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2b810b460 0x7ff2b819ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:33.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.286+0000 7ff2c0800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2b810bdb0 0x7ff2b81a1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:33.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.286+0000 7ff2c0800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b8115a70 con 0x7ff2b810bdb0 2024-08-31T06:20:33.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.286+0000 7ff2c0800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b81158f0 con 0x7ff2b8105b20 2024-08-31T06:20:33.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.286+0000 7ff2c0800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b8115bf0 con 0x7ff2b810b460 2024-08-31T06:20:33.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.287+0000 7ff2be800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2b810bdb0 0x7ff2b81a1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:33.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.287+0000 7ff2bde00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2b8105b20 0x7ff2b819c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:33.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.287+0000 7ff2bd400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2b810b460 0x7ff2b819ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:33.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.287+0000 7ff2bde00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2b8105b20 0x7ff2b819c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:47882/0 (socket says 172.21.15.16:47882) 2024-08-31T06:20:33.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.287+0000 7ff2be800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2b810bdb0 0x7ff2b81a1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51064/0 (socket says 172.21.15.16:51064) 2024-08-31T06:20:33.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.287+0000 7ff2bde00640 1 -- 172.21.15.16:0/1146844334 learned_addr learned my addr 172.21.15.16:0/1146844334 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:33.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.287+0000 7ff2bde00640 1 -- 172.21.15.16:0/1146844334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2b810b460 msgr2=0x7ff2b819ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:33.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.287+0000 7ff2bde00640 1 --2- 172.21.15.16:0/1146844334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2b810b460 0x7ff2b819ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:33.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.287+0000 7ff2bde00640 1 -- 172.21.15.16:0/1146844334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2b810bdb0 msgr2=0x7ff2b81a1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:33.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.287+0000 7ff2bde00640 1 --2- 172.21.15.16:0/1146844334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2b810bdb0 0x7ff2b81a1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:33.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.287+0000 7ff2bde00640 1 -- 172.21.15.16:0/1146844334 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b81a2430 con 0x7ff2b8105b20 2024-08-31T06:20:33.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.287+0000 7ff2be800640 1 --2- 172.21.15.16:0/1146844334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2b810bdb0 0x7ff2b81a1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:20:33.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.287+0000 7ff2bde00640 1 --2- 172.21.15.16:0/1146844334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2b8105b20 0x7ff2b819c4c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac005940 tx=0x7ff2ac0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:33.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.288+0000 7ff2b2a00640 1 -- 172.21.15.16:0/1146844334 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac03f030 con 0x7ff2b8105b20 2024-08-31T06:20:33.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.288+0000 7ff2c0800640 1 -- 172.21.15.16:0/1146844334 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2b81a26c0 con 0x7ff2b8105b20 2024-08-31T06:20:33.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.288+0000 7ff2b2a00640 1 -- 172.21.15.16:0/1146844334 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2ac0044c0 con 0x7ff2b8105b20 2024-08-31T06:20:33.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.288+0000 7ff2b2a00640 1 -- 172.21.15.16:0/1146844334 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac035070 con 0x7ff2b8105b20 2024-08-31T06:20:33.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.288+0000 7ff2c0800640 1 -- 172.21.15.16:0/1146844334 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2b81a8e80 con 0x7ff2b8105b20 2024-08-31T06:20:33.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.291+0000 7ff2c7473740 1 Processor -- start 2024-08-31T06:20:33.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.291+0000 7ff2c7473740 1 -- start start 2024-08-31T06:20:33.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.292+0000 7ff2c7473740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a7b075520 0x557a7b41aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:33.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.292+0000 7ff2c7473740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a7b04c660 0x557a7b41b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:33.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.292+0000 7ff2c7473740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a7b054840 0x557a7b41b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:33.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.292+0000 7ff2c7473740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557a7b391a90 con 0x557a7b04c660 2024-08-31T06:20:33.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.292+0000 7ff2c7473740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557a7b381020 con 0x557a7b075520 2024-08-31T06:20:33.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.292+0000 7ff2c7473740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557a7b384fd0 con 0x557a7b054840 2024-08-31T06:20:33.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.292+0000 7ff2be800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a7b075520 0x557a7b41aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:33.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.292+0000 7ff2be800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a7b075520 0x557a7b41aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:47884/0 (socket says 172.21.15.16:47884) 2024-08-31T06:20:33.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.292+0000 7ff2be800640 1 -- 172.21.15.16:0/144343370 learned_addr learned my addr 172.21.15.16:0/144343370 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:33.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.292+0000 7ff2bd400640 1 --2- 172.21.15.16:0/144343370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a7b054840 0x557a7b41b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:33.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.292+0000 7ff2b2a00640 1 -- 172.21.15.16:0/1146844334 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7ff2ac042030 con 0x7ff2b8105b20 2024-08-31T06:20:33.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.292+0000 7ff2bde00640 1 --2- 172.21.15.16:0/144343370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a7b04c660 0x557a7b41b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:33.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.292+0000 7ff2be800640 1 -- 172.21.15.16:0/144343370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a7b054840 msgr2=0x557a7b41b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:33.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.293+0000 7ff2be800640 1 --2- 172.21.15.16:0/144343370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a7b054840 0x557a7b41b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:33.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.293+0000 7ff2be800640 1 -- 172.21.15.16:0/144343370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a7b04c660 msgr2=0x557a7b41b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:33.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.293+0000 7ff2be800640 1 --2- 172.21.15.16:0/144343370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a7b04c660 0x557a7b41b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:33.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.293+0000 7ff2be800640 1 -- 172.21.15.16:0/144343370 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a7b0e0b00 con 0x557a7b075520 2024-08-31T06:20:33.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.293+0000 7ff2bd400640 1 --2- 172.21.15.16:0/144343370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a7b054840 0x557a7b41b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:20:33.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.293+0000 7ff2bde00640 1 --2- 172.21.15.16:0/144343370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a7b04c660 0x557a7b41b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:20:33.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.293+0000 7ff2be800640 1 --2- 172.21.15.16:0/144343370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a7b075520 0x557a7b41aef0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4011790 tx=0x7ff2b4011c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=beaa21742c4f19f1 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:33.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.293+0000 7ff2a7000640 1 -- 172.21.15.16:0/144343370 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b4003c30 con 0x557a7b075520 2024-08-31T06:20:33.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.293+0000 7ff2a7000640 1 -- 172.21.15.16:0/144343370 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2b400fe50 con 0x557a7b075520 2024-08-31T06:20:33.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.293+0000 7ff2a7000640 1 -- 172.21.15.16:0/144343370 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b4004860 con 0x557a7b075520 2024-08-31T06:20:33.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.294+0000 7ff2b2a00640 1 --2- 172.21.15.16:0/1146844334 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff2a00b4ca0 0x7ff2a00b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:33.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.294+0000 7ff2c7473740 1 -- 172.21.15.16:0/144343370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a7b075520 msgr2=0x557a7b41aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:33.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.294+0000 7ff2c7473740 1 --2- 172.21.15.16:0/144343370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a7b075520 0x557a7b41aef0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4011790 tx=0x7ff2b4011c50 comp rx=0 tx=0).stop 2024-08-31T06:20:33.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.294+0000 7ff2b2a00640 1 -- 172.21.15.16:0/1146844334 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8470+0+0 (secure 0 0 0) 0x7ff2ac1058e0 con 0x7ff2b8105b20 2024-08-31T06:20:33.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.294+0000 7ff2bde00640 1 --2- 172.21.15.16:0/1146844334 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff2a00b4ca0 0x7ff2a00b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:33.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.294+0000 7ff2c7473740 1 -- 172.21.15.16:0/144343370 shutdown_connections 2024-08-31T06:20:33.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.294+0000 7ff2c7473740 1 --2- 172.21.15.16:0/144343370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a7b054840 0x557a7b41b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:33.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.294+0000 7ff2c7473740 1 --2- 172.21.15.16:0/144343370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a7b04c660 0x557a7b41b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:33.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.294+0000 7ff2c7473740 1 --2- 172.21.15.16:0/144343370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a7b075520 0x557a7b41aef0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:33.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.294+0000 7ff2c7473740 1 -- 172.21.15.16:0/144343370 >> 172.21.15.16:0/144343370 conn(0x557a7b069910 msgr2=0x557a7b418de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:33.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.295+0000 7ff2c7473740 1 -- 172.21.15.16:0/144343370 shutdown_connections 2024-08-31T06:20:33.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.295+0000 7ff2c7473740 1 -- 172.21.15.16:0/144343370 wait complete. 2024-08-31T06:20:33.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.295+0000 7ff2bde00640 1 --2- 172.21.15.16:0/1146844334 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff2a00b4ca0 0x7ff2a00b71b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x557a7b419b70 tx=0x7ff2ac02a3f0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:33.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.295+0000 7ff2c7473740 1 -- start start 2024-08-31T06:20:33.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.295+0000 7ff2c7473740 1 Processor -- start 2024-08-31T06:20:33.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.296+0000 7ff2c7473740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a7b04c660 0x557a7b426350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:33.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.296+0000 7ff2c7473740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a7b054840 0x557a7b426890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:33.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.296+0000 7ff2c7473740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a7b075520 0x557a7b426dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:33.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.296+0000 7ff2c7473740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557a7b1e3bb0 con 0x557a7b04c660 2024-08-31T06:20:33.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.296+0000 7ff2c7473740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557a7b307f90 con 0x557a7b075520 2024-08-31T06:20:33.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.296+0000 7ff2c7473740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557a7b4296d0 con 0x557a7b054840 2024-08-31T06:20:33.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.296+0000 7ff2c7473740 10 client.0 _ll_register_callbacks cb 0x557a7b32d710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:20:33.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.296+0000 7ff2be800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a7b04c660 0x557a7b426350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:33.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.296+0000 7ff2be800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a7b04c660 0x557a7b426350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51092/0 (socket says 172.21.15.16:51092) 2024-08-31T06:20:33.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.296+0000 7ff2be800640 1 -- 172.21.15.16:0/3903138049 learned_addr learned my addr 172.21.15.16:0/3903138049 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:33.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.296+0000 7ff2bd400640 1 --2- 172.21.15.16:0/3903138049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a7b054840 0x557a7b426890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:33.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.297+0000 7ff2be800640 1 --2- 172.21.15.16:0/3903138049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a7b075520 0x557a7b426dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:33.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.297+0000 7ff2be800640 1 -- 172.21.15.16:0/3903138049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a7b054840 msgr2=0x557a7b426890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:33.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.297+0000 7ff2be800640 1 --2- 172.21.15.16:0/3903138049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a7b054840 0x557a7b426890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:33.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.297+0000 7ff2be800640 1 -- 172.21.15.16:0/3903138049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a7b075520 msgr2=0x557a7b426dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:33.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.297+0000 7ff2be800640 1 --2- 172.21.15.16:0/3903138049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a7b075520 0x557a7b426dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:33.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.297+0000 7ff2be800640 1 -- 172.21.15.16:0/3903138049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a7b157960 con 0x557a7b04c660 2024-08-31T06:20:33.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.297+0000 7ff2bd400640 1 --2- 172.21.15.16:0/3903138049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a7b054840 0x557a7b426890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:20:33.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.297+0000 7ff2be800640 1 --2- 172.21.15.16:0/3903138049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a7b04c660 0x557a7b426350 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4006010 tx=0x7ff2b4005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15346bce74522c98 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:33.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.297+0000 7ff29b400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:20:33.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.298+0000 7ff29b400640 1 -- 172.21.15.16:0/3903138049 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b402e060 con 0x557a7b04c660 2024-08-31T06:20:33.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.298+0000 7ff29b400640 1 -- 172.21.15.16:0/3903138049 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8470+0+0 (secure 0 0 0) 0x7ff2b4003c30 con 0x557a7b04c660 2024-08-31T06:20:33.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.298+0000 7ff2c7473740 1 -- 172.21.15.16:0/3903138049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557a7b31fe30 con 0x557a7b04c660 2024-08-31T06:20:33.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.298+0000 7ff29b400640 1 -- 172.21.15.16:0/3903138049 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7ff2b401f040 con 0x557a7b04c660 2024-08-31T06:20:33.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.298+0000 7ff2c7473740 10 client.18150 fetch_fsmap learned FSMap version 49 2024-08-31T06:20:33.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.298+0000 7ff2c7473740 1 -- 172.21.15.16:0/3903138049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x557a7b435020 con 0x557a7b04c660 2024-08-31T06:20:33.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.298+0000 7ff29b400640 1 -- 172.21.15.16:0/3903138049 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557a7b31fe30 con 0x557a7b04c660 2024-08-31T06:20:33.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.298+0000 7ff29b400640 1 -- 172.21.15.16:0/3903138049 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7ff2b4018070 con 0x557a7b04c660 2024-08-31T06:20:33.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.299+0000 7ff2c7473740 10 client.18150 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:20:33.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.299+0000 7ff2c7473740 10 client.18150 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:20:33.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.299+0000 7ff2c7473740 20 client.18150 populate_metadata read hostname 'smithi016' 2024-08-31T06:20:33.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.299+0000 7ff2c7473740 1 --2- 172.21.15.16:0/3903138049 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557a7b033ea0 0x557a7b438430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:33.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.299+0000 7ff2c7473740 4 client.18150 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:20:33.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.299+0000 7ff2c7473740 1 -- 172.21.15.16:0/3903138049 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a7b3939e0 con 0x557a7b033ea0 2024-08-31T06:20:33.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.299+0000 7ff2bd400640 1 --2- 172.21.15.16:0/3903138049 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557a7b033ea0 0x557a7b438430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:33.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.300+0000 7ff2bd400640 1 --2- 172.21.15.16:0/3903138049 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557a7b033ea0 0x557a7b438430 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=346d1a66fc0d0fd9 server_cookie=76a89cff9515e250 in_seq=0 out_seq=0 2024-08-31T06:20:33.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.300+0000 7ff29b400640 10 client.18150 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:20:33.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.301+0000 7ff29b400640 1 -- 172.21.15.16:0/3903138049 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557a7b3939e0 con 0x557a7b033ea0 2024-08-31T06:20:33.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.301+0000 7ff29b400640 10 client.18150 handle_command_reply: tid=0 2024-08-31T06:20:33.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.301+0000 7ff2c7473740 1 client.18150 shutdown 2024-08-31T06:20:33.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.301+0000 7ff2c7473740 1 -- 172.21.15.16:0/3903138049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a7b04c660 msgr2=0x557a7b426350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:33.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.301+0000 7ff2c7473740 1 --2- 172.21.15.16:0/3903138049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a7b04c660 0x557a7b426350 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4006010 tx=0x7ff2b4005a50 comp rx=0 tx=0).stop 2024-08-31T06:20:33.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.301+0000 7ff2c7473740 1 -- 172.21.15.16:0/3903138049 shutdown_connections 2024-08-31T06:20:33.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.301+0000 7ff2c7473740 1 --2- 172.21.15.16:0/3903138049 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557a7b033ea0 0x557a7b438430 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:33.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.301+0000 7ff2c7473740 1 --2- 172.21.15.16:0/3903138049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a7b054840 0x557a7b426890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:33.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.301+0000 7ff2c7473740 1 --2- 172.21.15.16:0/3903138049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a7b075520 0x557a7b426dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:33.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.301+0000 7ff2c7473740 1 --2- 172.21.15.16:0/3903138049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a7b04c660 0x557a7b426350 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:33.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.301+0000 7ff2c7473740 1 -- 172.21.15.16:0/3903138049 >> 172.21.15.16:0/3903138049 conn(0x557a7b069910 msgr2=0x557a7b41e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:33.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.301+0000 7ff29b400640 0 client.18150 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:20:33.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.301+0000 7ff2c7473740 1 -- 172.21.15.16:0/3903138049 shutdown_connections 2024-08-31T06:20:33.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.301+0000 7ff2c7473740 1 -- 172.21.15.16:0/3903138049 wait complete. 2024-08-31T06:20:33.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.301+0000 7ff2c7473740 20 client.18150 trim_cache size 0 max 16384 2024-08-31T06:20:33.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.319+0000 7ff2c7473740 1 Processor -- start 2024-08-31T06:20:33.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.319+0000 7ff2c7473740 1 -- start start 2024-08-31T06:20:33.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.320+0000 7ff2c7473740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a7b04c660 0x557a7b41c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:33.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.320+0000 7ff2c7473740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a7b054840 0x557a7b423540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:33.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.320+0000 7ff2c7473740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a7b075520 0x557a7b423a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:33.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.320+0000 7ff2c7473740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b81b4030 con 0x557a7b054840 2024-08-31T06:20:33.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.320+0000 7ff2c7473740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b81b1840 con 0x557a7b075520 2024-08-31T06:20:33.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.320+0000 7ff2c7473740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b81af6b0 con 0x557a7b04c660 2024-08-31T06:20:33.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.320+0000 7ff2bd400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a7b054840 0x557a7b423540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:33.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.320+0000 7ff2bd400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a7b054840 0x557a7b423540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51098/0 (socket says 172.21.15.16:51098) 2024-08-31T06:20:33.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.320+0000 7ff2bd400640 1 -- 172.21.15.16:0/1807523895 learned_addr learned my addr 172.21.15.16:0/1807523895 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:33.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.320+0000 7ff2be800640 1 --2- 172.21.15.16:0/1807523895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a7b04c660 0x557a7b41c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:33.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.320+0000 7ff2be800640 1 --2- 172.21.15.16:0/1807523895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a7b075520 0x557a7b423a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:33.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.321+0000 7ff2bd400640 1 -- 172.21.15.16:0/1807523895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a7b04c660 msgr2=0x557a7b41c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:33.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.321+0000 7ff2bd400640 1 --2- 172.21.15.16:0/1807523895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a7b04c660 0x557a7b41c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:33.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.321+0000 7ff2bd400640 1 -- 172.21.15.16:0/1807523895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a7b075520 msgr2=0x557a7b423a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:33.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.321+0000 7ff2bd400640 1 --2- 172.21.15.16:0/1807523895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a7b075520 0x557a7b423a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:33.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.321+0000 7ff2bd400640 1 -- 172.21.15.16:0/1807523895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a7b427610 con 0x557a7b054840 2024-08-31T06:20:33.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.321+0000 7ff2be800640 1 --2- 172.21.15.16:0/1807523895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a7b075520 0x557a7b423a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:20:33.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.321+0000 7ff2bd400640 1 --2- 172.21.15.16:0/1807523895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a7b054840 0x557a7b423540 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ff2b806c840 tx=0x7ff2a8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7a3ddb8cfd74092 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:33.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.321+0000 7ff2a6600640 1 -- 172.21.15.16:0/1807523895 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a802b070 con 0x557a7b054840 2024-08-31T06:20:33.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.321+0000 7ff2a6600640 1 -- 172.21.15.16:0/1807523895 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2a80122d0 con 0x557a7b054840 2024-08-31T06:20:33.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.321+0000 7ff2a6600640 1 -- 172.21.15.16:0/1807523895 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a8010b10 con 0x557a7b054840 2024-08-31T06:20:33.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.322+0000 7ff2c7473740 1 -- 172.21.15.16:0/1807523895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a7b054840 msgr2=0x557a7b423540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:33.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.322+0000 7ff2c7473740 1 --2- 172.21.15.16:0/1807523895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a7b054840 0x557a7b423540 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ff2b806c840 tx=0x7ff2a8005f90 comp rx=0 tx=0).stop 2024-08-31T06:20:33.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.322+0000 7ff2c7473740 1 -- 172.21.15.16:0/1807523895 shutdown_connections 2024-08-31T06:20:33.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.322+0000 7ff2c7473740 1 --2- 172.21.15.16:0/1807523895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a7b075520 0x557a7b423a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:33.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.322+0000 7ff2c7473740 1 --2- 172.21.15.16:0/1807523895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a7b054840 0x557a7b423540 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:33.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.322+0000 7ff2c7473740 1 --2- 172.21.15.16:0/1807523895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a7b04c660 0x557a7b41c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:33.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.322+0000 7ff2c7473740 1 -- 172.21.15.16:0/1807523895 >> 172.21.15.16:0/1807523895 conn(0x557a7b069910 msgr2=0x557a7b428110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:33.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.322+0000 7ff2c7473740 1 -- 172.21.15.16:0/1807523895 shutdown_connections 2024-08-31T06:20:33.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.322+0000 7ff2c7473740 1 -- 172.21.15.16:0/1807523895 wait complete. 2024-08-31T06:20:33.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.323+0000 7ff2c7473740 1 -- start start 2024-08-31T06:20:33.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.323+0000 7ff2c7473740 1 Processor -- start 2024-08-31T06:20:33.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.324+0000 7ff2c7473740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a7b04c660 0x557a7b42abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:33.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.324+0000 7ff2c7473740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a7b054840 0x557a7b42b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:33.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.324+0000 7ff2c7473740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a7b075520 0x557a7b4a1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:33.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.324+0000 7ff2c7473740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b81afd00 con 0x557a7b054840 2024-08-31T06:20:33.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.324+0000 7ff2c7473740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b81b1170 con 0x557a7b04c660 2024-08-31T06:20:33.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.324+0000 7ff2c7473740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b81b03d0 con 0x557a7b075520 2024-08-31T06:20:33.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.324+0000 7ff2c7473740 10 client.0 _ll_register_callbacks cb 0x7ff2b40048e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:20:33.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.324+0000 7ff2be800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a7b04c660 0x557a7b42abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:33.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.324+0000 7ff2bd400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a7b054840 0x557a7b42b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:33.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.324+0000 7ff2be800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a7b04c660 0x557a7b42abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:47912/0 (socket says 172.21.15.16:47912) 2024-08-31T06:20:33.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.324+0000 7ff2be800640 1 -- 172.21.15.16:0/2697773131 learned_addr learned my addr 172.21.15.16:0/2697773131 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:33.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.324+0000 7ff2be800640 1 --2- 172.21.15.16:0/2697773131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a7b075520 0x557a7b4a1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:33.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.324+0000 7ff2bd400640 1 -- 172.21.15.16:0/2697773131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a7b075520 msgr2=0x557a7b4a1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:33.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.324+0000 7ff2bd400640 1 --2- 172.21.15.16:0/2697773131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a7b075520 0x557a7b4a1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:33.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.324+0000 7ff2bd400640 1 -- 172.21.15.16:0/2697773131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a7b04c660 msgr2=0x557a7b42abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:33.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.324+0000 7ff2bd400640 1 --2- 172.21.15.16:0/2697773131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a7b04c660 0x557a7b42abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:33.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.324+0000 7ff2bd400640 1 -- 172.21.15.16:0/2697773131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a7b42b990 con 0x557a7b054840 2024-08-31T06:20:33.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.325+0000 7ff2bd400640 1 --2- 172.21.15.16:0/2697773131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a7b054840 0x557a7b42b0f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x557a7b337e00 tx=0x7ff2a8012630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f005488615a75112 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:33.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.325+0000 7ff29b400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:20:33.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.325+0000 7ff29b400640 1 -- 172.21.15.16:0/2697773131 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a8034060 con 0x557a7b054840 2024-08-31T06:20:33.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.325+0000 7ff2c7473740 1 -- 172.21.15.16:0/2697773131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2a80122d0 con 0x557a7b054840 2024-08-31T06:20:33.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.325+0000 7ff29b400640 1 -- 172.21.15.16:0/2697773131 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8470+0+0 (secure 0 0 0) 0x7ff2a8010890 con 0x557a7b054840 2024-08-31T06:20:33.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.325+0000 7ff29b400640 1 -- 172.21.15.16:0/2697773131 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7ff2a802b040 con 0x557a7b054840 2024-08-31T06:20:33.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.325+0000 7ff2c7473740 10 client.18156 fetch_fsmap learned FSMap version 49 2024-08-31T06:20:33.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.325+0000 7ff2c7473740 1 -- 172.21.15.16:0/2697773131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x557a7b4a31d0 con 0x557a7b054840 2024-08-31T06:20:33.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.326+0000 7ff29b400640 1 -- 172.21.15.16:0/2697773131 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2a80122d0 con 0x557a7b054840 2024-08-31T06:20:33.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.326+0000 7ff29b400640 1 -- 172.21.15.16:0/2697773131 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7ff2a802e070 con 0x557a7b054840 2024-08-31T06:20:33.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.326+0000 7ff2c7473740 10 client.18156 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:20:33.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.326+0000 7ff2c7473740 10 client.18156 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:20:33.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.326+0000 7ff2c7473740 20 client.18156 populate_metadata read hostname 'smithi016' 2024-08-31T06:20:33.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.326+0000 7ff2c7473740 1 --2- 172.21.15.16:0/2697773131 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557a7b009250 0x557a7b4a5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:33.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.326+0000 7ff2c7473740 4 client.18156 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "8fb05a68", "release": true, "await": true}] 2024-08-31T06:20:33.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.326+0000 7ff2c7473740 1 -- 172.21.15.16:0/2697773131 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "quiesce db", "set_id": "8fb05a68", "release": true, "await": true}) -- 0x557a7b38f480 con 0x557a7b009250 2024-08-31T06:20:33.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.326+0000 7ff2be800640 1 --2- 172.21.15.16:0/2697773131 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557a7b009250 0x557a7b4a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:33.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.327+0000 7ff2be800640 1 --2- 172.21.15.16:0/2697773131 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557a7b009250 0x557a7b4a5920 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4f3ec13a052fef0 server_cookie=748129bcc5b38762 in_seq=0 out_seq=0 2024-08-31T06:20:33.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.327+0000 7ff29b400640 10 client.18156 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:20:33.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.360+0000 7ff29b400640 1 -- 172.21.15.16:0/2697773131 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557a7b38f480 con 0x557a7b009250 2024-08-31T06:20:33.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.360+0000 7ff29b400640 10 client.18156 handle_command_reply: tid=0 2024-08-31T06:20:33.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.360+0000 7ff2c7473740 1 client.18156 shutdown 2024-08-31T06:20:33.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.361+0000 7ff2c7473740 1 -- 172.21.15.16:0/2697773131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a7b054840 msgr2=0x557a7b42b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:33.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.361+0000 7ff2c7473740 1 --2- 172.21.15.16:0/2697773131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a7b054840 0x557a7b42b0f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x557a7b337e00 tx=0x7ff2a8012630 comp rx=0 tx=0).stop 2024-08-31T06:20:33.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.361+0000 7ff2c7473740 1 -- 172.21.15.16:0/2697773131 shutdown_connections 2024-08-31T06:20:33.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.361+0000 7ff2c7473740 1 --2- 172.21.15.16:0/2697773131 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x557a7b009250 0x557a7b4a5920 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:33.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.361+0000 7ff2c7473740 1 --2- 172.21.15.16:0/2697773131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a7b075520 0x557a7b4a1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:33.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.361+0000 7ff2c7473740 1 --2- 172.21.15.16:0/2697773131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a7b054840 0x557a7b42b0f0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:33.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.361+0000 7ff2c7473740 1 --2- 172.21.15.16:0/2697773131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a7b04c660 0x557a7b42abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:33.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.361+0000 7ff2c7473740 1 -- 172.21.15.16:0/2697773131 >> 172.21.15.16:0/2697773131 conn(0x557a7b069910 msgr2=0x557a7b428110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:33.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.362+0000 7ff29b400640 0 client.18156 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:20:33.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.364+0000 7ff2c7473740 1 -- 172.21.15.16:0/2697773131 shutdown_connections 2024-08-31T06:20:33.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.364+0000 7ff2c7473740 1 -- 172.21.15.16:0/2697773131 wait complete. 2024-08-31T06:20:33.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.365+0000 7ff2c7473740 20 client.18156 trim_cache size 0 max 16384 2024-08-31T06:20:33.368 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:20:33.368 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 49, 2024-08-31T06:20:33.368 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24737, 2024-08-31T06:20:33.368 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 44, 2024-08-31T06:20:33.368 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-08-31T06:20:33.368 INFO:teuthology.orchestra.run.smithi016.stdout: "8fb05a68": { 2024-08-31T06:20:33.368 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 44, 2024-08-31T06:20:33.368 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-08-31T06:20:33.368 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:20:33.369 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-08-31T06:20:33.369 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:20:33.369 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:20:33.369 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-08-31T06:20:33.369 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 134.7, 2024-08-31T06:20:33.369 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-08-31T06:20:33.369 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-08-31T06:20:33.369 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-08-31T06:20:33.369 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:20:33.369 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-08-31T06:20:33.369 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:20:33.369 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:20:33.369 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:20:33.369 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:20:33.369 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:20:33.370 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:20:33.370 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:20:33.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.366+0000 7ff2c0800640 1 -- 172.21.15.16:0/1146844334 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff2a00b4ca0 msgr2=0x7ff2a00b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:33.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.366+0000 7ff2c0800640 1 --2- 172.21.15.16:0/1146844334 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff2a00b4ca0 0x7ff2a00b71b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x557a7b419b70 tx=0x7ff2ac02a3f0 comp rx=0 tx=0).stop 2024-08-31T06:20:33.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.366+0000 7ff2c0800640 1 -- 172.21.15.16:0/1146844334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2b8105b20 msgr2=0x7ff2b819c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:33.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.366+0000 7ff2c0800640 1 --2- 172.21.15.16:0/1146844334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2b8105b20 0x7ff2b819c4c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac005940 tx=0x7ff2ac0040c0 comp rx=0 tx=0).stop 2024-08-31T06:20:33.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.367+0000 7ff2c0800640 1 -- 172.21.15.16:0/1146844334 shutdown_connections 2024-08-31T06:20:33.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.367+0000 7ff2c0800640 1 --2- 172.21.15.16:0/1146844334 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff2a00b4ca0 0x7ff2a00b71b0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:33.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.367+0000 7ff2c0800640 1 --2- 172.21.15.16:0/1146844334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2b810b460 0x7ff2b819ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:33.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.367+0000 7ff2c0800640 1 --2- 172.21.15.16:0/1146844334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2b810bdb0 0x7ff2b81a1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:33.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.367+0000 7ff2c0800640 1 --2- 172.21.15.16:0/1146844334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2b8105b20 0x7ff2b819c4c0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:33.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.367+0000 7ff2c0800640 1 -- 172.21.15.16:0/1146844334 >> 172.21.15.16:0/1146844334 conn(0x7ff2b81015b0 msgr2=0x7ff2b81039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:33.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.367+0000 7ff2c0800640 1 -- 172.21.15.16:0/1146844334 shutdown_connections 2024-08-31T06:20:33.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:33.367+0000 7ff2c0800640 1 -- 172.21.15.16:0/1146844334 wait complete. 2024-08-31T06:20:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.145+0000 7feb16200640 1 -- 172.21.15.16:0/420807540 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb10075b70 msgr2=0x7feb10075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.145+0000 7feb16200640 1 --2- 172.21.15.16:0/420807540 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb10075b70 0x7feb10075f80 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7feafc009900 tx=0x7feafc02a3f0 comp rx=0 tx=0).stop 2024-08-31T06:20:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.145+0000 7feb16200640 1 -- 172.21.15.16:0/420807540 shutdown_connections 2024-08-31T06:20:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.145+0000 7feb16200640 1 --2- 172.21.15.16:0/420807540 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb10075b70 0x7feb10075f80 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.145+0000 7feb16200640 1 --2- 172.21.15.16:0/420807540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb10113f70 0x7feb10116440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.145+0000 7feb16200640 1 --2- 172.21.15.16:0/420807540 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb10077130 0x7feb10075550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.145+0000 7feb16200640 1 -- 172.21.15.16:0/420807540 >> 172.21.15.16:0/420807540 conn(0x7feb100ff0d0 msgr2=0x7feb10101520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.145+0000 7feb16200640 1 -- 172.21.15.16:0/420807540 shutdown_connections 2024-08-31T06:20:34.149 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.145+0000 7feb16200640 1 -- 172.21.15.16:0/420807540 wait complete. 2024-08-31T06:20:34.149 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.146+0000 7feb16200640 1 Processor -- start 2024-08-31T06:20:34.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.146+0000 7feb16200640 1 -- start start 2024-08-31T06:20:34.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.147+0000 7feb16200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb10075b70 0x7feb101a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:34.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.147+0000 7feb16200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb10077130 0x7feb101a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:34.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.147+0000 7feb16200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb10113f70 0x7feb101a5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:34.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.147+0000 7feb16200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feb10119c00 con 0x7feb10113f70 2024-08-31T06:20:34.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.147+0000 7feb16200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7feb10119a80 con 0x7feb10077130 2024-08-31T06:20:34.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.147+0000 7feb16200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7feb10119d80 con 0x7feb10075b70 2024-08-31T06:20:34.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.147+0000 7feb14e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb10075b70 0x7feb101a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.147+0000 7feb0fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb10077130 0x7feb101a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.147+0000 7feb14e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb10075b70 0x7feb101a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:53010/0 (socket says 172.21.15.16:53010) 2024-08-31T06:20:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.147+0000 7feb14e00640 1 -- 172.21.15.16:0/2240580144 learned_addr learned my addr 172.21.15.16:0/2240580144 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.147+0000 7feb15800640 1 --2- 172.21.15.16:0/2240580144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb10113f70 0x7feb101a5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.148+0000 7feb14e00640 1 -- 172.21.15.16:0/2240580144 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb10077130 msgr2=0x7feb101a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.148+0000 7feb14e00640 1 --2- 172.21.15.16:0/2240580144 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb10077130 0x7feb101a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.148+0000 7feb14e00640 1 -- 172.21.15.16:0/2240580144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb10113f70 msgr2=0x7feb101a5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.148+0000 7feb14e00640 1 --2- 172.21.15.16:0/2240580144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb10113f70 0x7feb101a5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.148+0000 7feb14e00640 1 -- 172.21.15.16:0/2240580144 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb101a6590 con 0x7feb10075b70 2024-08-31T06:20:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.148+0000 7feb15800640 1 --2- 172.21.15.16:0/2240580144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb10113f70 0x7feb101a5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:20:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.148+0000 7feb14e00640 1 --2- 172.21.15.16:0/2240580144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb10075b70 0x7feb101a0660 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7feb0400e9f0 tx=0x7feb0400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.149+0000 7feb0d600640 1 -- 172.21.15.16:0/2240580144 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0400cd80 con 0x7feb10075b70 2024-08-31T06:20:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.149+0000 7feb0d600640 1 -- 172.21.15.16:0/2240580144 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb040054d0 con 0x7feb10075b70 2024-08-31T06:20:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.149+0000 7feb0d600640 1 -- 172.21.15.16:0/2240580144 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb04010690 con 0x7feb10075b70 2024-08-31T06:20:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.149+0000 7feb16200640 1 -- 172.21.15.16:0/2240580144 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb101a6880 con 0x7feb10075b70 2024-08-31T06:20:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.149+0000 7feb16200640 1 -- 172.21.15.16:0/2240580144 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb101ad010 con 0x7feb10075b70 2024-08-31T06:20:34.155 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.153+0000 7feb1e85b740 1 Processor -- start 2024-08-31T06:20:34.155 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.153+0000 7feb0d600640 1 -- 172.21.15.16:0/2240580144 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7feb04010830 con 0x7feb10075b70 2024-08-31T06:20:34.155 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.153+0000 7feb1e85b740 1 -- start start 2024-08-31T06:20:34.156 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.153+0000 7feb1e85b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735e11c1b0 0x55735e4bfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:34.156 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.153+0000 7feb1e85b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55735e0f32f0 0x55735e4c0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:34.156 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.153+0000 7feb1e85b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55735e0fb4d0 0x55735e4c0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:34.156 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.153+0000 7feb1e85b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55735e434f70 con 0x55735e0fb4d0 2024-08-31T06:20:34.156 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.153+0000 7feb1e85b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55735e424500 con 0x55735e0f32f0 2024-08-31T06:20:34.156 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.153+0000 7feb1e85b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55735e4284b0 con 0x55735e11c1b0 2024-08-31T06:20:34.156 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.154+0000 7feb15800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735e11c1b0 0x55735e4bfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:34.156 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.154+0000 7feb14e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55735e0f32f0 0x55735e4c0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:34.157 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.154+0000 7feb0fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55735e0fb4d0 0x55735e4c0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:34.157 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.154+0000 7feb15800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735e11c1b0 0x55735e4bfb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:53020/0 (socket says 172.21.15.16:53020) 2024-08-31T06:20:34.157 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.154+0000 7feb15800640 1 -- 172.21.15.16:0/2467928899 learned_addr learned my addr 172.21.15.16:0/2467928899 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:34.157 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.154+0000 7feb15800640 1 -- 172.21.15.16:0/2467928899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55735e0f32f0 msgr2=0x55735e4c0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:34.157 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.154+0000 7feb15800640 1 --2- 172.21.15.16:0/2467928899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55735e0f32f0 0x55735e4c0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:34.158 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.154+0000 7feb15800640 1 -- 172.21.15.16:0/2467928899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55735e0fb4d0 msgr2=0x55735e4c0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:34.158 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.154+0000 7feb15800640 1 --2- 172.21.15.16:0/2467928899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55735e0fb4d0 0x55735e4c0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:34.158 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.154+0000 7feb15800640 1 -- 172.21.15.16:0/2467928899 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55735e3dec10 con 0x55735e11c1b0 2024-08-31T06:20:34.158 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.155+0000 7feb15800640 1 --2- 172.21.15.16:0/2467928899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735e11c1b0 0x55735e4bfb00 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7feb080103d0 tx=0x7feb08010890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57fecc60d240a87 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:34.158 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.155+0000 7feb0d600640 1 --2- 172.21.15.16:0/2240580144 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7feaf80b4e70 0x7feaf80b7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:34.158 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.155+0000 7feb02000640 1 -- 172.21.15.16:0/2467928899 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb08011700 con 0x55735e11c1b0 2024-08-31T06:20:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.155+0000 7feb02000640 1 -- 172.21.15.16:0/2467928899 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb08011d50 con 0x55735e11c1b0 2024-08-31T06:20:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.156+0000 7feb0d600640 1 -- 172.21.15.16:0/2240580144 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8470+0+0 (secure 0 0 0) 0x7feb040e5790 con 0x7feb10075b70 2024-08-31T06:20:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.156+0000 7feb02000640 1 -- 172.21.15.16:0/2467928899 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0800e3c0 con 0x55735e11c1b0 2024-08-31T06:20:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.156+0000 7feb14e00640 1 --2- 172.21.15.16:0/2240580144 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7feaf80b4e70 0x7feaf80b7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.157+0000 7feb1e85b740 1 -- 172.21.15.16:0/2467928899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735e11c1b0 msgr2=0x55735e4bfb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.157+0000 7feb1e85b740 1 --2- 172.21.15.16:0/2467928899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735e11c1b0 0x55735e4bfb00 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7feb080103d0 tx=0x7feb08010890 comp rx=0 tx=0).stop 2024-08-31T06:20:34.160 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.157+0000 7feb1e85b740 1 -- 172.21.15.16:0/2467928899 shutdown_connections 2024-08-31T06:20:34.160 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.157+0000 7feb1e85b740 1 --2- 172.21.15.16:0/2467928899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55735e0fb4d0 0x55735e4c0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:34.160 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.157+0000 7feb1e85b740 1 --2- 172.21.15.16:0/2467928899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55735e0f32f0 0x55735e4c0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:34.160 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.157+0000 7feb1e85b740 1 --2- 172.21.15.16:0/2467928899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735e11c1b0 0x55735e4bfb00 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:34.160 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.157+0000 7feb1e85b740 1 -- 172.21.15.16:0/2467928899 >> 172.21.15.16:0/2467928899 conn(0x55735e110300 msgr2=0x55735e4bd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:34.160 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.158+0000 7feb1e85b740 1 -- 172.21.15.16:0/2467928899 shutdown_connections 2024-08-31T06:20:34.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.158+0000 7feb1e85b740 1 -- 172.21.15.16:0/2467928899 wait complete. 2024-08-31T06:20:34.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.159+0000 7feb14e00640 1 --2- 172.21.15.16:0/2240580144 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7feaf80b4e70 0x7feaf80b7380 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x55735e4be780 tx=0x7feb040eb530 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:34.161 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.159+0000 7feb1e85b740 1 -- start start 2024-08-31T06:20:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.159+0000 7feb1e85b740 1 Processor -- start 2024-08-31T06:20:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.160+0000 7feb1e85b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735e0f32f0 0x55735e4cb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.160+0000 7feb1e85b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55735e0fb4d0 0x55735e4cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.160+0000 7feb1e85b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55735e11c1b0 0x55735e4cba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.160+0000 7feb1e85b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55735e35dab0 con 0x55735e0fb4d0 2024-08-31T06:20:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.160+0000 7feb1e85b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55735e3cb320 con 0x55735e11c1b0 2024-08-31T06:20:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.160+0000 7feb1e85b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55735e4ce320 con 0x55735e0f32f0 2024-08-31T06:20:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.160+0000 7feb15800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735e0f32f0 0x55735e4cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.160+0000 7feb1e85b740 10 client.0 _ll_register_callbacks cb 0x55735e3d16a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:20:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.160+0000 7feb15800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735e0f32f0 0x55735e4cb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:53026/0 (socket says 172.21.15.16:53026) 2024-08-31T06:20:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.160+0000 7feb15800640 1 -- 172.21.15.16:0/3326986437 learned_addr learned my addr 172.21.15.16:0/3326986437 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.160+0000 7feb0fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55735e0fb4d0 0x55735e4cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.161+0000 7feb15800640 1 --2- 172.21.15.16:0/3326986437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55735e11c1b0 0x55735e4cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.161+0000 7feb15800640 1 -- 172.21.15.16:0/3326986437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55735e11c1b0 msgr2=0x55735e4cba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.161+0000 7feb15800640 1 --2- 172.21.15.16:0/3326986437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55735e11c1b0 0x55735e4cba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.161+0000 7feb15800640 1 -- 172.21.15.16:0/3326986437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55735e0fb4d0 msgr2=0x55735e4cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.161+0000 7feb15800640 1 --2- 172.21.15.16:0/3326986437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55735e0fb4d0 0x55735e4cb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.161+0000 7feb15800640 1 -- 172.21.15.16:0/3326986437 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55735e187790 con 0x55735e0f32f0 2024-08-31T06:20:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.161+0000 7feb15800640 1 --2- 172.21.15.16:0/3326986437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735e0f32f0 0x55735e4cb000 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7feb08006010 tx=0x7feb0801bac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b48c31ce6e05d794 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.161+0000 7feaf6a00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:20:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.162+0000 7feaf6a00640 1 -- 172.21.15.16:0/3326986437 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb08011500 con 0x55735e0f32f0 2024-08-31T06:20:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.162+0000 7feaf6a00640 1 -- 172.21.15.16:0/3326986437 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8470+0+0 (secure 0 0 0) 0x7feb080116a0 con 0x55735e0f32f0 2024-08-31T06:20:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.162+0000 7feb1e85b740 1 -- 172.21.15.16:0/3326986437 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb08011d50 con 0x55735e0f32f0 2024-08-31T06:20:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.162+0000 7feaf6a00640 1 -- 172.21.15.16:0/3326986437 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7feb080118a0 con 0x55735e0f32f0 2024-08-31T06:20:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.162+0000 7feb1e85b740 10 client.28627 fetch_fsmap learned FSMap version 49 2024-08-31T06:20:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.162+0000 7feb1e85b740 1 -- 172.21.15.16:0/3326986437 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55735e200620 con 0x55735e0f32f0 2024-08-31T06:20:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.162+0000 7feaf6a00640 1 -- 172.21.15.16:0/3326986437 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb08011d50 con 0x55735e0f32f0 2024-08-31T06:20:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.162+0000 7feaf6a00640 1 -- 172.21.15.16:0/3326986437 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7feb0802c4f0 con 0x55735e0f32f0 2024-08-31T06:20:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.163+0000 7feb1e85b740 10 client.28627 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:20:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.163+0000 7feb1e85b740 10 client.28627 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:20:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.163+0000 7feb1e85b740 20 client.28627 populate_metadata read hostname 'smithi016' 2024-08-31T06:20:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.163+0000 7feb1e85b740 1 --2- 172.21.15.16:0/3326986437 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55735e0dad80 0x55735e4dd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.163+0000 7feb1e85b740 4 client.28627 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:20:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.163+0000 7feb1e85b740 1 -- 172.21.15.16:0/3326986437 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55735e436ec0 con 0x55735e0dad80 2024-08-31T06:20:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.163+0000 7feb0fe00640 1 --2- 172.21.15.16:0/3326986437 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55735e0dad80 0x55735e4dd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.164+0000 7feb0fe00640 1 --2- 172.21.15.16:0/3326986437 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55735e0dad80 0x55735e4dd100 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f125edb61db2611 server_cookie=89759a9bea1edda3 in_seq=0 out_seq=0 2024-08-31T06:20:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.164+0000 7feaf6a00640 10 client.28627 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:20:34.168 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.165+0000 7feaf6a00640 1 -- 172.21.15.16:0/3326986437 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55735e436ec0 con 0x55735e0dad80 2024-08-31T06:20:34.168 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.165+0000 7feaf6a00640 10 client.28627 handle_command_reply: tid=0 2024-08-31T06:20:34.168 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.165+0000 7feb1e85b740 1 client.28627 shutdown 2024-08-31T06:20:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.165+0000 7feb1e85b740 1 -- 172.21.15.16:0/3326986437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735e0f32f0 msgr2=0x55735e4cb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.165+0000 7feb1e85b740 1 --2- 172.21.15.16:0/3326986437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735e0f32f0 0x55735e4cb000 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7feb08006010 tx=0x7feb0801bac0 comp rx=0 tx=0).stop 2024-08-31T06:20:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.166+0000 7feb1e85b740 1 -- 172.21.15.16:0/3326986437 shutdown_connections 2024-08-31T06:20:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.166+0000 7feb1e85b740 1 --2- 172.21.15.16:0/3326986437 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55735e0dad80 0x55735e4dd100 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.166+0000 7feb1e85b740 1 --2- 172.21.15.16:0/3326986437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55735e11c1b0 0x55735e4cba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.166+0000 7feb1e85b740 1 --2- 172.21.15.16:0/3326986437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55735e0fb4d0 0x55735e4cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.166+0000 7feb1e85b740 1 --2- 172.21.15.16:0/3326986437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735e0f32f0 0x55735e4cb000 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.166+0000 7feb1e85b740 1 -- 172.21.15.16:0/3326986437 >> 172.21.15.16:0/3326986437 conn(0x55735e110300 msgr2=0x55735e4c0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.166+0000 7feaf6a00640 0 client.28627 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:20:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.166+0000 7feb1e85b740 1 -- 172.21.15.16:0/3326986437 shutdown_connections 2024-08-31T06:20:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.166+0000 7feb1e85b740 1 -- 172.21.15.16:0/3326986437 wait complete. 2024-08-31T06:20:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.166+0000 7feb1e85b740 20 client.28627 trim_cache size 0 max 16384 2024-08-31T06:20:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.192+0000 7feb1e85b740 1 Processor -- start 2024-08-31T06:20:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.196+0000 7feb1e85b740 1 -- start start 2024-08-31T06:20:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.196+0000 7feb1e85b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55735e0f32f0 0x55735e4ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.196+0000 7feb1e85b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735e0fb4d0 0x55735e4d75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.196+0000 7feb1e85b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55735e11c1b0 0x55735e4c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.196+0000 7feb1e85b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feaf00110c0 con 0x55735e11c1b0 2024-08-31T06:20:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.196+0000 7feb1e85b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7feaf000e8d0 con 0x55735e0f32f0 2024-08-31T06:20:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.196+0000 7feb1e85b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7feaf000c740 con 0x55735e0fb4d0 2024-08-31T06:20:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.197+0000 7feb15800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55735e0f32f0 0x55735e4ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.197+0000 7feb15800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55735e0f32f0 0x55735e4ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:47958/0 (socket says 172.21.15.16:47958) 2024-08-31T06:20:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.197+0000 7feb15800640 1 -- 172.21.15.16:0/765002554 learned_addr learned my addr 172.21.15.16:0/765002554 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.197+0000 7feb15800640 1 -- 172.21.15.16:0/765002554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735e0fb4d0 msgr2=0x55735e4d75a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T06:20:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.197+0000 7feb15800640 1 --2- 172.21.15.16:0/765002554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735e0fb4d0 0x55735e4d75a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.197+0000 7feb15800640 1 -- 172.21.15.16:0/765002554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55735e11c1b0 msgr2=0x55735e4c81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.197+0000 7feb15800640 1 --2- 172.21.15.16:0/765002554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55735e11c1b0 0x55735e4c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.197+0000 7feb15800640 1 -- 172.21.15.16:0/765002554 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55735e4c20a0 con 0x55735e0f32f0 2024-08-31T06:20:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.198+0000 7feb15800640 1 --2- 172.21.15.16:0/765002554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55735e0f32f0 0x55735e4ccff0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7feb0802df80 tx=0x7feb0802dfb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13b797be25b86ccc server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.198+0000 7feb01600640 1 -- 172.21.15.16:0/765002554 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0801b720 con 0x55735e0f32f0 2024-08-31T06:20:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.198+0000 7feb01600640 1 -- 172.21.15.16:0/765002554 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb0801f030 con 0x55735e0f32f0 2024-08-31T06:20:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.198+0000 7feb01600640 1 -- 172.21.15.16:0/765002554 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0801bd90 con 0x55735e0f32f0 2024-08-31T06:20:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.200+0000 7feb1e85b740 1 -- 172.21.15.16:0/765002554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55735e0f32f0 msgr2=0x55735e4ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.200+0000 7feb1e85b740 1 --2- 172.21.15.16:0/765002554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55735e0f32f0 0x55735e4ccff0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7feb0802df80 tx=0x7feb0802dfb0 comp rx=0 tx=0).stop 2024-08-31T06:20:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.200+0000 7feb1e85b740 1 -- 172.21.15.16:0/765002554 shutdown_connections 2024-08-31T06:20:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.200+0000 7feb1e85b740 1 --2- 172.21.15.16:0/765002554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735e0fb4d0 0x55735e4d75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.200+0000 7feb1e85b740 1 --2- 172.21.15.16:0/765002554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55735e11c1b0 0x55735e4c81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.200+0000 7feb1e85b740 1 --2- 172.21.15.16:0/765002554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55735e0f32f0 0x55735e4ccff0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.200+0000 7feb1e85b740 1 -- 172.21.15.16:0/765002554 >> 172.21.15.16:0/765002554 conn(0x55735e110300 msgr2=0x55735e4c14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.201+0000 7feb1e85b740 1 -- 172.21.15.16:0/765002554 shutdown_connections 2024-08-31T06:20:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.202+0000 7feb1e85b740 1 -- 172.21.15.16:0/765002554 wait complete. 2024-08-31T06:20:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.202+0000 7feb1e85b740 1 -- start start 2024-08-31T06:20:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.203+0000 7feb1e85b740 1 Processor -- start 2024-08-31T06:20:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.204+0000 7feb1e85b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735e0f32f0 0x55735e4d3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.204+0000 7feb1e85b740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55735e0fb4d0 0x55735e542890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.204+0000 7feb1e85b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55735e11c1b0 0x55735e542dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.205+0000 7feb15800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735e0f32f0 0x55735e4d3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.205+0000 7feb1e85b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feaf000cd90 con 0x55735e11c1b0 2024-08-31T06:20:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.205+0000 7feb1e85b740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7feaf000e200 con 0x55735e0fb4d0 2024-08-31T06:20:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.205+0000 7feb1e85b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7feaf000d460 con 0x55735e0f32f0 2024-08-31T06:20:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.205+0000 7feb1e85b740 10 client.0 _ll_register_callbacks cb 0x7feb08009e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:20:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.205+0000 7feb15800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735e0f32f0 0x55735e4d3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:53054/0 (socket says 172.21.15.16:53054) 2024-08-31T06:20:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.205+0000 7feb15800640 1 -- 172.21.15.16:0/1635440606 learned_addr learned my addr 172.21.15.16:0/1635440606 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.207+0000 7feb0fe00640 1 --2- 172.21.15.16:0/1635440606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55735e0fb4d0 0x55735e542890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.207+0000 7feb0fe00640 1 -- 172.21.15.16:0/1635440606 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735e0f32f0 msgr2=0x55735e4d3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.207+0000 7feb0fe00640 1 --2- 172.21.15.16:0/1635440606 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735e0f32f0 0x55735e4d3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.207+0000 7feb0fe00640 1 -- 172.21.15.16:0/1635440606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55735e11c1b0 msgr2=0x55735e542dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.207+0000 7feb0fe00640 1 --2- 172.21.15.16:0/1635440606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55735e11c1b0 0x55735e542dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.207+0000 7feb0fe00640 1 -- 172.21.15.16:0/1635440606 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55735e543310 con 0x55735e0fb4d0 2024-08-31T06:20:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.208+0000 7feaf6a00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:20:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.208+0000 7feb0fe00640 1 --2- 172.21.15.16:0/1635440606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55735e0fb4d0 0x55735e542890 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7feafc02a9e0 tx=0x7feafc02a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b324d823a34369b5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.209+0000 7feaf6a00640 1 -- 172.21.15.16:0/1635440606 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feafc04a020 con 0x55735e0fb4d0 2024-08-31T06:20:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.209+0000 7feb1e85b740 1 -- 172.21.15.16:0/1635440606 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb0801f030 con 0x55735e0fb4d0 2024-08-31T06:20:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.210+0000 7feaf6a00640 1 -- 172.21.15.16:0/1635440606 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8470+0+0 (secure 0 0 0) 0x7feafc005310 con 0x55735e0fb4d0 2024-08-31T06:20:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.210+0000 7feaf6a00640 1 -- 172.21.15.16:0/1635440606 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7feafc055050 con 0x55735e0fb4d0 2024-08-31T06:20:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.211+0000 7feb1e85b740 10 client.28631 fetch_fsmap learned FSMap version 49 2024-08-31T06:20:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.211+0000 7feb1e85b740 1 -- 172.21.15.16:0/1635440606 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55735e4d1d10 con 0x55735e0fb4d0 2024-08-31T06:20:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.212+0000 7feaf6a00640 1 -- 172.21.15.16:0/1635440606 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb0801f030 con 0x55735e0fb4d0 2024-08-31T06:20:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.212+0000 7feaf6a00640 1 -- 172.21.15.16:0/1635440606 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7feafc003680 con 0x55735e0fb4d0 2024-08-31T06:20:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.212+0000 7feb1e85b740 10 client.28631 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:20:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.212+0000 7feb1e85b740 10 client.28631 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:20:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.212+0000 7feb1e85b740 20 client.28631 populate_metadata read hostname 'smithi016' 2024-08-31T06:20:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.212+0000 7feb1e85b740 1 --2- 172.21.15.16:0/1635440606 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55735e0b01c0 0x55735e55ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.212+0000 7feb1e85b740 4 client.28631 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:20:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.212+0000 7feb1e85b740 1 -- 172.21.15.16:0/1635440606 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55735e432960 con 0x55735e0b01c0 2024-08-31T06:20:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.214+0000 7feb15800640 1 --2- 172.21.15.16:0/1635440606 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55735e0b01c0 0x55735e55ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.215+0000 7feb15800640 1 --2- 172.21.15.16:0/1635440606 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55735e0b01c0 0x55735e55ae70 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e76efd4d31b75ec9 server_cookie=48752eeb68e3637f in_seq=0 out_seq=0 2024-08-31T06:20:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.215+0000 7feaf6a00640 10 client.28631 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:20:34.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.251+0000 7feaf6a00640 1 -- 172.21.15.16:0/1635440606 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55735e432960 con 0x55735e0b01c0 2024-08-31T06:20:34.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.251+0000 7feaf6a00640 10 client.28631 handle_command_reply: tid=0 2024-08-31T06:20:34.253 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.251+0000 7feb1e85b740 1 client.28631 shutdown 2024-08-31T06:20:34.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.251+0000 7feb1e85b740 1 -- 172.21.15.16:0/1635440606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55735e0fb4d0 msgr2=0x55735e542890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:34.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.251+0000 7feb1e85b740 1 --2- 172.21.15.16:0/1635440606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55735e0fb4d0 0x55735e542890 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7feafc02a9e0 tx=0x7feafc02a3f0 comp rx=0 tx=0).stop 2024-08-31T06:20:34.254 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.251+0000 7feb1e85b740 1 -- 172.21.15.16:0/1635440606 shutdown_connections 2024-08-31T06:20:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.251+0000 7feb1e85b740 1 --2- 172.21.15.16:0/1635440606 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55735e0b01c0 0x55735e55ae70 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.251+0000 7feb1e85b740 1 --2- 172.21.15.16:0/1635440606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55735e11c1b0 0x55735e542dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.251+0000 7feb1e85b740 1 --2- 172.21.15.16:0/1635440606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55735e0fb4d0 0x55735e542890 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.251+0000 7feb1e85b740 1 --2- 172.21.15.16:0/1635440606 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735e0f32f0 0x55735e4d3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.251+0000 7feb1e85b740 1 -- 172.21.15.16:0/1635440606 >> 172.21.15.16:0/1635440606 conn(0x55735e110300 msgr2=0x55735e4c12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.251+0000 7feaf6a00640 0 client.28631 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:20:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.252+0000 7feb1e85b740 1 -- 172.21.15.16:0/1635440606 shutdown_connections 2024-08-31T06:20:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.252+0000 7feb1e85b740 1 -- 172.21.15.16:0/1635440606 wait complete. 2024-08-31T06:20:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.252+0000 7feb1e85b740 20 client.28631 trim_cache size 0 max 16384 2024-08-31T06:20:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:20:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:20:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:20:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:20:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.253+0000 7feb16200640 1 -- 172.21.15.16:0/2240580144 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7feaf80b4e70 msgr2=0x7feaf80b7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.253+0000 7feb16200640 1 --2- 172.21.15.16:0/2240580144 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7feaf80b4e70 0x7feaf80b7380 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x55735e4be780 tx=0x7feb040eb530 comp rx=0 tx=0).stop 2024-08-31T06:20:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.253+0000 7feb16200640 1 -- 172.21.15.16:0/2240580144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb10075b70 msgr2=0x7feb101a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.253+0000 7feb16200640 1 --2- 172.21.15.16:0/2240580144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb10075b70 0x7feb101a0660 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7feb0400e9f0 tx=0x7feb0400edf0 comp rx=0 tx=0).stop 2024-08-31T06:20:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.253+0000 7feb16200640 1 -- 172.21.15.16:0/2240580144 shutdown_connections 2024-08-31T06:20:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.253+0000 7feb16200640 1 --2- 172.21.15.16:0/2240580144 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7feaf80b4e70 0x7feaf80b7380 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.253+0000 7feb16200640 1 --2- 172.21.15.16:0/2240580144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb10113f70 0x7feb101a5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.253+0000 7feb16200640 1 --2- 172.21.15.16:0/2240580144 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb10077130 0x7feb101a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.253+0000 7feb16200640 1 --2- 172.21.15.16:0/2240580144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb10075b70 0x7feb101a0660 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.253+0000 7feb16200640 1 -- 172.21.15.16:0/2240580144 >> 172.21.15.16:0/2240580144 conn(0x7feb100ff0d0 msgr2=0x7feb1010fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.254+0000 7feb16200640 1 -- 172.21.15.16:0/2240580144 shutdown_connections 2024-08-31T06:20:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:34.254+0000 7feb16200640 1 -- 172.21.15.16:0/2240580144 wait complete. 2024-08-31T06:20:34.347 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '8fb05a68', total seconds elapsed: 19.4 2024-08-31T06:20:35.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:34 smithi155 ceph-mon[21742]: pgmap v780: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 10 MiB/s wr, 961 op/s 2024-08-31T06:20:35.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:34 smithi016 ceph-mon[17194]: pgmap v780: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 10 MiB/s wr, 961 op/s 2024-08-31T06:20:35.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:34 smithi184 ceph-mon[22331]: pgmap v780: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 10 MiB/s wr, 961 op/s 2024-08-31T06:20:36.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 790.0 s, 828.8 tps, lat 369.496 ms stddev 1859.572 2024-08-31T06:20:37.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:36 smithi155 ceph-mon[21742]: pgmap v781: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 9.1 MiB/s wr, 897 op/s 2024-08-31T06:20:37.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:36 smithi016 ceph-mon[17194]: pgmap v781: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 9.1 MiB/s wr, 897 op/s 2024-08-31T06:20:37.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:36 smithi184 ceph-mon[22331]: pgmap v781: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 9.1 MiB/s wr, 897 op/s 2024-08-31T06:20:39.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:38 smithi155 ceph-mon[21742]: pgmap v782: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 9.4 MiB/s wr, 909 op/s 2024-08-31T06:20:39.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:38 smithi016 ceph-mon[17194]: pgmap v782: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 9.4 MiB/s wr, 909 op/s 2024-08-31T06:20:39.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:38 smithi184 ceph-mon[22331]: pgmap v782: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 9.4 MiB/s wr, 909 op/s 2024-08-31T06:20:41.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:40 smithi155 ceph-mon[21742]: pgmap v783: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 9.6 MiB/s wr, 960 op/s 2024-08-31T06:20:41.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:40 smithi016 ceph-mon[17194]: pgmap v783: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 9.6 MiB/s wr, 960 op/s 2024-08-31T06:20:41.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:40 smithi184 ceph-mon[22331]: pgmap v783: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 9.6 MiB/s wr, 960 op/s 2024-08-31T06:20:41.625 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 795.0 s, 1505.9 tps, lat 66.458 ms stddev 91.770 2024-08-31T06:20:43.025 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:42 smithi184 ceph-mon[22331]: pgmap v784: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 9.1 MiB/s wr, 922 op/s 2024-08-31T06:20:43.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:42 smithi155 ceph-mon[21742]: pgmap v784: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 9.1 MiB/s wr, 922 op/s 2024-08-31T06:20:43.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:42 smithi016 ceph-mon[17194]: pgmap v784: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 9.1 MiB/s wr, 922 op/s 2024-08-31T06:20:45.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:44 smithi155 ceph-mon[21742]: pgmap v785: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 5.2 MiB/s wr, 554 op/s 2024-08-31T06:20:45.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:44 smithi016 ceph-mon[17194]: pgmap v785: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 5.2 MiB/s wr, 554 op/s 2024-08-31T06:20:45.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:44 smithi184 ceph-mon[22331]: pgmap v785: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 5.2 MiB/s wr, 554 op/s 2024-08-31T06:20:46.626 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 800.0 s, 685.3 tps, lat 80.854 ms stddev 89.729 2024-08-31T06:20:47.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:46 smithi155 ceph-mon[21742]: pgmap v786: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 5.1 MiB/s wr, 558 op/s 2024-08-31T06:20:47.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:46 smithi016 ceph-mon[17194]: pgmap v786: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 5.1 MiB/s wr, 558 op/s 2024-08-31T06:20:47.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:46 smithi184 ceph-mon[22331]: pgmap v786: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 5.1 MiB/s wr, 558 op/s 2024-08-31T06:20:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.560+0000 7fe676800640 1 -- 172.21.15.16:0/2211463446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe670108fe0 msgr2=0x7fe6701093f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.560+0000 7fe676800640 1 --2- 172.21.15.16:0/2211463446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe670108fe0 0x7fe6701093f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe65c009900 tx=0x7fe65c02a410 comp rx=0 tx=0).stop 2024-08-31T06:20:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.561+0000 7fe676800640 1 -- 172.21.15.16:0/2211463446 shutdown_connections 2024-08-31T06:20:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.561+0000 7fe676800640 1 --2- 172.21.15.16:0/2211463446 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe670109930 0x7fe670110160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.561+0000 7fe676800640 1 --2- 172.21.15.16:0/2211463446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe670108fe0 0x7fe6701093f0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.561+0000 7fe676800640 1 --2- 172.21.15.16:0/2211463446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6701036b0 0x7fe670103ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.561+0000 7fe676800640 1 -- 172.21.15.16:0/2211463446 >> 172.21.15.16:0/2211463446 conn(0x7fe6700ff0d0 msgr2=0x7fe670101520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.562+0000 7fe676800640 1 -- 172.21.15.16:0/2211463446 shutdown_connections 2024-08-31T06:20:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.563+0000 7fe676800640 1 -- 172.21.15.16:0/2211463446 wait complete. 2024-08-31T06:20:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.564+0000 7fe676800640 1 Processor -- start 2024-08-31T06:20:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.566+0000 7fe676800640 1 -- start start 2024-08-31T06:20:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.567+0000 7fe676800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6701036b0 0x7fe67019a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.567+0000 7fe676800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe670108fe0 0x7fe67019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.567+0000 7fe676800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe670109930 0x7fe67019f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.567+0000 7fe676800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe6701135f0 con 0x7fe670108fe0 2024-08-31T06:20:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.567+0000 7fe676800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe670113470 con 0x7fe670109930 2024-08-31T06:20:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.567+0000 7fe676800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe670113770 con 0x7fe6701036b0 2024-08-31T06:20:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.567+0000 7fe675400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6701036b0 0x7fe67019a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.567+0000 7fe675e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe670109930 0x7fe67019f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.567+0000 7fe675400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6701036b0 0x7fe67019a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:53604/0 (socket says 172.21.15.16:53604) 2024-08-31T06:20:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.567+0000 7fe675400640 1 -- 172.21.15.16:0/2895089741 learned_addr learned my addr 172.21.15.16:0/2895089741 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.567+0000 7fe674a00640 1 --2- 172.21.15.16:0/2895089741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe670108fe0 0x7fe67019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.568+0000 7fe675e00640 1 -- 172.21.15.16:0/2895089741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6701036b0 msgr2=0x7fe67019a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.568+0000 7fe675e00640 1 --2- 172.21.15.16:0/2895089741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6701036b0 0x7fe67019a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.568+0000 7fe675e00640 1 -- 172.21.15.16:0/2895089741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe670108fe0 msgr2=0x7fe67019a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.568+0000 7fe675e00640 1 --2- 172.21.15.16:0/2895089741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe670108fe0 0x7fe67019a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.568+0000 7fe675e00640 1 -- 172.21.15.16:0/2895089741 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe67019ff60 con 0x7fe670109930 2024-08-31T06:20:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.568+0000 7fe675400640 1 --2- 172.21.15.16:0/2895089741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6701036b0 0x7fe67019a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:20:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.568+0000 7fe675e00640 1 --2- 172.21.15.16:0/2895089741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe670109930 0x7fe67019f860 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fe668009fd0 tx=0x7fe66800b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.569+0000 7fe66e000640 1 -- 172.21.15.16:0/2895089741 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe668004b20 con 0x7fe670109930 2024-08-31T06:20:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.569+0000 7fe66e000640 1 -- 172.21.15.16:0/2895089741 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe668004cc0 con 0x7fe670109930 2024-08-31T06:20:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.569+0000 7fe66e000640 1 -- 172.21.15.16:0/2895089741 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe668005410 con 0x7fe670109930 2024-08-31T06:20:47.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.572+0000 7fe676800640 1 -- 172.21.15.16:0/2895089741 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6701a0250 con 0x7fe670109930 2024-08-31T06:20:47.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.572+0000 7fe676800640 1 -- 172.21.15.16:0/2895089741 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe670075390 con 0x7fe670109930 2024-08-31T06:20:47.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.576+0000 7fe67ede7740 1 Processor -- start 2024-08-31T06:20:47.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.581+0000 7fe66e000640 1 -- 172.21.15.16:0/2895089741 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fe668019450 con 0x7fe670109930 2024-08-31T06:20:47.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.581+0000 7fe67ede7740 1 -- start start 2024-08-31T06:20:47.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.583+0000 7fe67ede7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576fbd421b0 0x5576fc0e58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:47.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.583+0000 7fe67ede7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576fbd192f0 0x5576fc0e5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:47.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.583+0000 7fe67ede7740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576fbd214d0 0x5576fc0e6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:47.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.583+0000 7fe67ede7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5576fc05af70 con 0x5576fbd421b0 2024-08-31T06:20:47.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.583+0000 7fe67ede7740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5576fc04a500 con 0x5576fbd214d0 2024-08-31T06:20:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.583+0000 7fe67ede7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5576fc04e4b0 con 0x5576fbd192f0 2024-08-31T06:20:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.584+0000 7fe675400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576fbd192f0 0x5576fc0e5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.584+0000 7fe675400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576fbd192f0 0x5576fc0e5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:53608/0 (socket says 172.21.15.16:53608) 2024-08-31T06:20:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.584+0000 7fe675400640 1 -- 172.21.15.16:0/3985636034 learned_addr learned my addr 172.21.15.16:0/3985636034 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.584+0000 7fe675400640 1 -- 172.21.15.16:0/3985636034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576fbd214d0 msgr2=0x5576fc0e6320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:20:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.584+0000 7fe675400640 1 --2- 172.21.15.16:0/3985636034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576fbd214d0 0x5576fc0e6320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.584+0000 7fe675400640 1 -- 172.21.15.16:0/3985636034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576fbd421b0 msgr2=0x5576fc0e58a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:20:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.585+0000 7fe674a00640 1 --2- 172.21.15.16:0/3985636034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576fbd421b0 0x5576fc0e58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.585+0000 7fe675400640 1 --2- 172.21.15.16:0/3985636034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576fbd421b0 0x5576fc0e58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.585+0000 7fe675400640 1 -- 172.21.15.16:0/3985636034 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576fc004c10 con 0x5576fbd192f0 2024-08-31T06:20:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.585+0000 7fe674a00640 1 --2- 172.21.15.16:0/3985636034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576fbd421b0 0x5576fc0e58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:20:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.585+0000 7fe675400640 1 --2- 172.21.15.16:0/3985636034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576fbd192f0 0x5576fc0e5de0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe664010580 tx=0x7fe664010a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f14d2c923e47978 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.586+0000 7fe662600640 1 -- 172.21.15.16:0/3985636034 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66400e830 con 0x5576fbd192f0 2024-08-31T06:20:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.586+0000 7fe66e000640 1 --2- 172.21.15.16:0/2895089741 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe6580b4d20 0x7fe6580b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.586+0000 7fe662600640 1 -- 172.21.15.16:0/3985636034 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe66400ee80 con 0x5576fbd192f0 2024-08-31T06:20:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.586+0000 7fe662600640 1 -- 172.21.15.16:0/3985636034 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe664011e60 con 0x5576fbd192f0 2024-08-31T06:20:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.586+0000 7fe66e000640 1 -- 172.21.15.16:0/2895089741 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8470+0+0 (secure 0 0 0) 0x7fe6680e9530 con 0x7fe670109930 2024-08-31T06:20:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.588+0000 7fe674a00640 1 --2- 172.21.15.16:0/2895089741 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe6580b4d20 0x7fe6580b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.588+0000 7fe67ede7740 1 -- 172.21.15.16:0/3985636034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576fbd192f0 msgr2=0x5576fc0e5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.588+0000 7fe67ede7740 1 --2- 172.21.15.16:0/3985636034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576fbd192f0 0x5576fc0e5de0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe664010580 tx=0x7fe664010a40 comp rx=0 tx=0).stop 2024-08-31T06:20:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.588+0000 7fe67ede7740 1 -- 172.21.15.16:0/3985636034 shutdown_connections 2024-08-31T06:20:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.588+0000 7fe67ede7740 1 --2- 172.21.15.16:0/3985636034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576fbd192f0 0x5576fc0e5de0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.588+0000 7fe67ede7740 1 --2- 172.21.15.16:0/3985636034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576fbd214d0 0x5576fc0e6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.588+0000 7fe67ede7740 1 --2- 172.21.15.16:0/3985636034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576fbd421b0 0x5576fc0e58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.588+0000 7fe67ede7740 1 -- 172.21.15.16:0/3985636034 >> 172.21.15.16:0/3985636034 conn(0x5576fbd36300 msgr2=0x5576fc0e3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.590+0000 7fe67ede7740 1 -- 172.21.15.16:0/3985636034 shutdown_connections 2024-08-31T06:20:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.591+0000 7fe67ede7740 1 -- 172.21.15.16:0/3985636034 wait complete. 2024-08-31T06:20:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.591+0000 7fe674a00640 1 --2- 172.21.15.16:0/2895089741 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe6580b4d20 0x7fe6580b7230 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe65c009230 tx=0x7fe65c005fd0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.592+0000 7fe67ede7740 1 -- start start 2024-08-31T06:20:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.592+0000 7fe67ede7740 1 Processor -- start 2024-08-31T06:20:47.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.593+0000 7fe67ede7740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576fbd192f0 0x5576fc0f95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.593+0000 7fe67ede7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576fbd214d0 0x5576fc0f9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.593+0000 7fe67ede7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576fbd421b0 0x5576fc0fa070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.593+0000 7fe67ede7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5576fbf83ab0 con 0x5576fbd421b0 2024-08-31T06:20:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.593+0000 7fe67ede7740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5576fbff1320 con 0x5576fbd192f0 2024-08-31T06:20:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.593+0000 7fe67ede7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5576fc0fc7e0 con 0x5576fbd214d0 2024-08-31T06:20:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.593+0000 7fe67ede7740 10 client.0 _ll_register_callbacks cb 0x5576fc02ae80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:20:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.594+0000 7fe675400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576fbd192f0 0x5576fc0f95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.594+0000 7fe675400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576fbd192f0 0x5576fc0f95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51414/0 (socket says 172.21.15.16:51414) 2024-08-31T06:20:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.594+0000 7fe675400640 1 -- 172.21.15.16:0/2338765329 learned_addr learned my addr 172.21.15.16:0/2338765329 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.594+0000 7fe674a00640 1 --2- 172.21.15.16:0/2338765329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576fbd214d0 0x5576fc0f9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.594+0000 7fe675e00640 1 --2- 172.21.15.16:0/2338765329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576fbd421b0 0x5576fc0fa070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.595+0000 7fe675400640 1 -- 172.21.15.16:0/2338765329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576fbd214d0 msgr2=0x5576fc0f9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.595+0000 7fe675400640 1 --2- 172.21.15.16:0/2338765329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576fbd214d0 0x5576fc0f9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.595+0000 7fe675400640 1 -- 172.21.15.16:0/2338765329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576fbd421b0 msgr2=0x5576fc0fa070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.595+0000 7fe675400640 1 --2- 172.21.15.16:0/2338765329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576fbd421b0 0x5576fc0fa070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.595+0000 7fe675400640 1 -- 172.21.15.16:0/2338765329 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576fbdad790 con 0x5576fbd192f0 2024-08-31T06:20:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.595+0000 7fe675e00640 1 --2- 172.21.15.16:0/2338765329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576fbd421b0 0x5576fc0fa070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:20:47.599 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.595+0000 7fe675400640 1 --2- 172.21.15.16:0/2338765329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576fbd192f0 0x5576fc0f95f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fe664005bc0 tx=0x7fe66400b700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=436a5524e64f95cc server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:47.599 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.596+0000 7fe656a00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:20:47.599 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.596+0000 7fe656a00640 1 -- 172.21.15.16:0/2338765329 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66400e630 con 0x5576fbd192f0 2024-08-31T06:20:47.599 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.596+0000 7fe656a00640 1 -- 172.21.15.16:0/2338765329 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8470+0+0 (secure 0 0 0) 0x7fe66400e7d0 con 0x5576fbd192f0 2024-08-31T06:20:47.599 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.597+0000 7fe67ede7740 1 -- 172.21.15.16:0/2338765329 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5576fc05cec0 con 0x5576fbd192f0 2024-08-31T06:20:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.597+0000 7fe656a00640 1 -- 172.21.15.16:0/2338765329 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fe66400e9d0 con 0x5576fbd192f0 2024-08-31T06:20:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.598+0000 7fe67ede7740 10 client.28640 fetch_fsmap learned FSMap version 49 2024-08-31T06:20:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.598+0000 7fe67ede7740 1 -- 172.21.15.16:0/2338765329 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5576fbe26620 con 0x5576fbd192f0 2024-08-31T06:20:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.599+0000 7fe656a00640 1 -- 172.21.15.16:0/2338765329 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5576fc05cec0 con 0x5576fbd192f0 2024-08-31T06:20:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.599+0000 7fe656a00640 1 -- 172.21.15.16:0/2338765329 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fe6640253f0 con 0x5576fbd192f0 2024-08-31T06:20:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.599+0000 7fe67ede7740 10 client.28640 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:20:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.599+0000 7fe67ede7740 10 client.28640 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:20:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.599+0000 7fe67ede7740 20 client.28640 populate_metadata read hostname 'smithi016' 2024-08-31T06:20:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.599+0000 7fe67ede7740 1 --2- 172.21.15.16:0/2338765329 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5576fbd00d80 0x5576fc0f15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.599+0000 7fe67ede7740 4 client.28640 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:20:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.599+0000 7fe67ede7740 1 -- 172.21.15.16:0/2338765329 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576fc058960 con 0x5576fbd00d80 2024-08-31T06:20:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.600+0000 7fe674a00640 1 --2- 172.21.15.16:0/2338765329 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5576fbd00d80 0x5576fc0f15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.601+0000 7fe656a00640 10 client.28640 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:20:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.605+0000 7fe674a00640 1 --2- 172.21.15.16:0/2338765329 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5576fbd00d80 0x5576fc0f15f0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c5649cd38b69128 server_cookie=b962b5e0199cfe19 in_seq=0 out_seq=0 2024-08-31T06:20:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.605+0000 7fe656a00640 1 -- 172.21.15.16:0/2338765329 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5576fc058960 con 0x5576fbd00d80 2024-08-31T06:20:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.605+0000 7fe656a00640 10 client.28640 handle_command_reply: tid=0 2024-08-31T06:20:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.606+0000 7fe67ede7740 1 client.28640 shutdown 2024-08-31T06:20:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.606+0000 7fe67ede7740 1 -- 172.21.15.16:0/2338765329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576fbd192f0 msgr2=0x5576fc0f95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.606+0000 7fe67ede7740 1 --2- 172.21.15.16:0/2338765329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576fbd192f0 0x5576fc0f95f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fe664005bc0 tx=0x7fe66400b700 comp rx=0 tx=0).stop 2024-08-31T06:20:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.606+0000 7fe67ede7740 1 -- 172.21.15.16:0/2338765329 shutdown_connections 2024-08-31T06:20:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.606+0000 7fe67ede7740 1 --2- 172.21.15.16:0/2338765329 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5576fbd00d80 0x5576fc0f15f0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.606+0000 7fe67ede7740 1 --2- 172.21.15.16:0/2338765329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576fbd214d0 0x5576fc0f9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.606+0000 7fe67ede7740 1 --2- 172.21.15.16:0/2338765329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576fbd421b0 0x5576fc0fa070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.606+0000 7fe67ede7740 1 --2- 172.21.15.16:0/2338765329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576fbd192f0 0x5576fc0f95f0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.606+0000 7fe656a00640 0 client.28640 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:20:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.606+0000 7fe67ede7740 1 -- 172.21.15.16:0/2338765329 >> 172.21.15.16:0/2338765329 conn(0x5576fbd36300 msgr2=0x5576fc0e8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.606+0000 7fe67ede7740 1 -- 172.21.15.16:0/2338765329 shutdown_connections 2024-08-31T06:20:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.606+0000 7fe67ede7740 1 -- 172.21.15.16:0/2338765329 wait complete. 2024-08-31T06:20:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.607+0000 7fe67ede7740 20 client.28640 trim_cache size 0 max 16384 2024-08-31T06:20:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.633+0000 7fe67ede7740 1 Processor -- start 2024-08-31T06:20:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.634+0000 7fe67ede7740 1 -- start start 2024-08-31T06:20:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.634+0000 7fe67ede7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576fbd192f0 0x5576fc0f4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.634+0000 7fe67ede7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576fbd214d0 0x5576fc0e2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.634+0000 7fe67ede7740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576fbd421b0 0x5576fc105170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.634+0000 7fe67ede7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe6701afc00 con 0x5576fbd192f0 2024-08-31T06:20:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.634+0000 7fe67ede7740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe6701ad410 con 0x5576fbd421b0 2024-08-31T06:20:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.634+0000 7fe67ede7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe6701ab280 con 0x5576fbd214d0 2024-08-31T06:20:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.634+0000 7fe675400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576fbd192f0 0x5576fc0f4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.635+0000 7fe674a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576fbd214d0 0x5576fc0e2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.635+0000 7fe674a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576fbd214d0 0x5576fc0e2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:53632/0 (socket says 172.21.15.16:53632) 2024-08-31T06:20:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.635+0000 7fe674a00640 1 -- 172.21.15.16:0/3630969488 learned_addr learned my addr 172.21.15.16:0/3630969488 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.635+0000 7fe675e00640 1 --2- 172.21.15.16:0/3630969488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576fbd421b0 0x5576fc105170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.635+0000 7fe674a00640 1 -- 172.21.15.16:0/3630969488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576fbd421b0 msgr2=0x5576fc105170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.635+0000 7fe674a00640 1 --2- 172.21.15.16:0/3630969488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576fbd421b0 0x5576fc105170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.635+0000 7fe674a00640 1 -- 172.21.15.16:0/3630969488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576fbd192f0 msgr2=0x5576fc0f4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.635+0000 7fe674a00640 1 --2- 172.21.15.16:0/3630969488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576fbd192f0 0x5576fc0f4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.635+0000 7fe674a00640 1 -- 172.21.15.16:0/3630969488 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576fc0e4390 con 0x5576fbd214d0 2024-08-31T06:20:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.635+0000 7fe675e00640 1 --2- 172.21.15.16:0/3630969488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576fbd421b0 0x5576fc105170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:20:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.637+0000 7fe674a00640 1 --2- 172.21.15.16:0/3630969488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576fbd214d0 0x5576fc0e2730 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe65c009900 tx=0x7fe65c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a33144c96285652 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.637+0000 7fe661c00640 1 -- 172.21.15.16:0/3630969488 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c035050 con 0x5576fbd214d0 2024-08-31T06:20:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.637+0000 7fe661c00640 1 -- 172.21.15.16:0/3630969488 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe65c056070 con 0x5576fbd214d0 2024-08-31T06:20:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.637+0000 7fe661c00640 1 -- 172.21.15.16:0/3630969488 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c003bf0 con 0x5576fbd214d0 2024-08-31T06:20:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.638+0000 7fe67ede7740 1 -- 172.21.15.16:0/3630969488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576fbd214d0 msgr2=0x5576fc0e2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.638+0000 7fe67ede7740 1 --2- 172.21.15.16:0/3630969488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576fbd214d0 0x5576fc0e2730 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe65c009900 tx=0x7fe65c009260 comp rx=0 tx=0).stop 2024-08-31T06:20:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.638+0000 7fe67ede7740 1 -- 172.21.15.16:0/3630969488 shutdown_connections 2024-08-31T06:20:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.638+0000 7fe67ede7740 1 --2- 172.21.15.16:0/3630969488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576fbd214d0 0x5576fc0e2730 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.638+0000 7fe67ede7740 1 --2- 172.21.15.16:0/3630969488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576fbd421b0 0x5576fc105170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.638+0000 7fe67ede7740 1 --2- 172.21.15.16:0/3630969488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576fbd192f0 0x5576fc0f4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.638+0000 7fe67ede7740 1 -- 172.21.15.16:0/3630969488 >> 172.21.15.16:0/3630969488 conn(0x5576fbd36300 msgr2=0x5576fc104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.639+0000 7fe67ede7740 1 -- 172.21.15.16:0/3630969488 shutdown_connections 2024-08-31T06:20:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.639+0000 7fe67ede7740 1 -- 172.21.15.16:0/3630969488 wait complete. 2024-08-31T06:20:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.639+0000 7fe67ede7740 1 -- start start 2024-08-31T06:20:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.640+0000 7fe67ede7740 1 Processor -- start 2024-08-31T06:20:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.641+0000 7fe67ede7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576fbd192f0 0x5576fc1771d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.641+0000 7fe67ede7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576fbd421b0 0x5576fc177710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.641+0000 7fe67ede7740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576fbcd61c0 0x5576fc177c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.641+0000 7fe67ede7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe6701ab8d0 con 0x5576fbd192f0 2024-08-31T06:20:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.641+0000 7fe67ede7740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe6701acd40 con 0x5576fbcd61c0 2024-08-31T06:20:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.641+0000 7fe67ede7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe6701abfa0 con 0x5576fbd421b0 2024-08-31T06:20:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.641+0000 7fe67ede7740 10 client.0 _ll_register_callbacks cb 0x7fe6640103d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:20:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.641+0000 7fe674a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576fbd421b0 0x5576fc177710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.642+0000 7fe675e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576fbcd61c0 0x5576fc177c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.642+0000 7fe675400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576fbd192f0 0x5576fc1771d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.642+0000 7fe675e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576fbcd61c0 0x5576fc177c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51426/0 (socket says 172.21.15.16:51426) 2024-08-31T06:20:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.642+0000 7fe675400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576fbd192f0 0x5576fc1771d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35326/0 (socket says 172.21.15.16:35326) 2024-08-31T06:20:47.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.642+0000 7fe675e00640 1 -- 172.21.15.16:0/1608995800 learned_addr learned my addr 172.21.15.16:0/1608995800 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.642+0000 7fe675e00640 1 -- 172.21.15.16:0/1608995800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576fbd421b0 msgr2=0x5576fc177710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.642+0000 7fe675e00640 1 --2- 172.21.15.16:0/1608995800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576fbd421b0 0x5576fc177710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.642+0000 7fe675e00640 1 -- 172.21.15.16:0/1608995800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576fbd192f0 msgr2=0x5576fc1771d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.642+0000 7fe675e00640 1 --2- 172.21.15.16:0/1608995800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576fbd192f0 0x5576fc1771d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.642+0000 7fe675e00640 1 -- 172.21.15.16:0/1608995800 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576fc100cc0 con 0x5576fbcd61c0 2024-08-31T06:20:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.643+0000 7fe675e00640 1 --2- 172.21.15.16:0/1608995800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576fbcd61c0 0x5576fc177c50 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe6680e6cb0 tx=0x7fe668019c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e172ebe379f11201 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.643+0000 7fe656a00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:20:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.643+0000 7fe656a00640 1 -- 172.21.15.16:0/1608995800 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6680b4020 con 0x5576fbcd61c0 2024-08-31T06:20:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.643+0000 7fe656a00640 1 -- 172.21.15.16:0/1608995800 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8470+0+0 (secure 0 0 0) 0x7fe6680b41c0 con 0x5576fbcd61c0 2024-08-31T06:20:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.643+0000 7fe67ede7740 1 -- 172.21.15.16:0/1608995800 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe65c056070 con 0x5576fbcd61c0 2024-08-31T06:20:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.643+0000 7fe656a00640 1 -- 172.21.15.16:0/1608995800 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fe6680e70e0 con 0x5576fbcd61c0 2024-08-31T06:20:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.643+0000 7fe67ede7740 10 client.28643 fetch_fsmap learned FSMap version 49 2024-08-31T06:20:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.643+0000 7fe67ede7740 1 -- 172.21.15.16:0/1608995800 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5576fc101f00 con 0x5576fbcd61c0 2024-08-31T06:20:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.644+0000 7fe656a00640 1 -- 172.21.15.16:0/1608995800 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe65c056070 con 0x5576fbcd61c0 2024-08-31T06:20:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.644+0000 7fe656a00640 1 -- 172.21.15.16:0/1608995800 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fe668005640 con 0x5576fbcd61c0 2024-08-31T06:20:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.644+0000 7fe67ede7740 10 client.28643 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:20:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.644+0000 7fe67ede7740 10 client.28643 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:20:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.644+0000 7fe67ede7740 20 client.28643 populate_metadata read hostname 'smithi016' 2024-08-31T06:20:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.644+0000 7fe67ede7740 1 --2- 172.21.15.16:0/1608995800 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5576fc18fde0 0x5576fc192200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.644+0000 7fe67ede7740 4 client.28643 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:20:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.644+0000 7fe67ede7740 1 -- 172.21.15.16:0/1608995800 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5576fc0560a0 con 0x5576fc18fde0 2024-08-31T06:20:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.645+0000 7fe675400640 1 --2- 172.21.15.16:0/1608995800 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5576fc18fde0 0x5576fc192200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.646+0000 7fe675400640 1 --2- 172.21.15.16:0/1608995800 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5576fc18fde0 0x5576fc192200 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb3f2de374c2821d server_cookie=8f16c26a0214174 in_seq=0 out_seq=0 2024-08-31T06:20:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.646+0000 7fe656a00640 10 client.28643 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:20:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.659+0000 7fe656a00640 1 -- 172.21.15.16:0/1608995800 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5576fc0560a0 con 0x5576fc18fde0 2024-08-31T06:20:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.659+0000 7fe656a00640 10 client.28643 handle_command_reply: tid=0 2024-08-31T06:20:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.659+0000 7fe67ede7740 1 client.28643 shutdown 2024-08-31T06:20:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.660+0000 7fe67ede7740 1 -- 172.21.15.16:0/1608995800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576fbcd61c0 msgr2=0x5576fc177c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.660+0000 7fe67ede7740 1 --2- 172.21.15.16:0/1608995800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576fbcd61c0 0x5576fc177c50 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe6680e6cb0 tx=0x7fe668019c80 comp rx=0 tx=0).stop 2024-08-31T06:20:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.661+0000 7fe67ede7740 1 -- 172.21.15.16:0/1608995800 shutdown_connections 2024-08-31T06:20:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.661+0000 7fe67ede7740 1 --2- 172.21.15.16:0/1608995800 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5576fc18fde0 0x5576fc192200 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.661+0000 7fe67ede7740 1 --2- 172.21.15.16:0/1608995800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576fbd421b0 0x5576fc177710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.661+0000 7fe67ede7740 1 --2- 172.21.15.16:0/1608995800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5576fbcd61c0 0x5576fc177c50 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.661+0000 7fe67ede7740 1 --2- 172.21.15.16:0/1608995800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576fbd192f0 0x5576fc1771d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.661+0000 7fe656a00640 0 client.28643 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:20:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.661+0000 7fe67ede7740 1 -- 172.21.15.16:0/1608995800 >> 172.21.15.16:0/1608995800 conn(0x5576fbd36300 msgr2=0x5576fc0e2500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.661+0000 7fe67ede7740 1 -- 172.21.15.16:0/1608995800 shutdown_connections 2024-08-31T06:20:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.661+0000 7fe67ede7740 1 -- 172.21.15.16:0/1608995800 wait complete. 2024-08-31T06:20:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.661+0000 7fe67ede7740 20 client.28643 trim_cache size 0 max 16384 2024-08-31T06:20:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:20:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:20:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:20:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:20:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.662+0000 7fe676800640 1 -- 172.21.15.16:0/2895089741 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe6580b4d20 msgr2=0x7fe6580b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.662+0000 7fe676800640 1 --2- 172.21.15.16:0/2895089741 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe6580b4d20 0x7fe6580b7230 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe65c009230 tx=0x7fe65c005fd0 comp rx=0 tx=0).stop 2024-08-31T06:20:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.662+0000 7fe676800640 1 -- 172.21.15.16:0/2895089741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe670109930 msgr2=0x7fe67019f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.662+0000 7fe676800640 1 --2- 172.21.15.16:0/2895089741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe670109930 0x7fe67019f860 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fe668009fd0 tx=0x7fe66800b690 comp rx=0 tx=0).stop 2024-08-31T06:20:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.663+0000 7fe676800640 1 -- 172.21.15.16:0/2895089741 shutdown_connections 2024-08-31T06:20:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.663+0000 7fe676800640 1 --2- 172.21.15.16:0/2895089741 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe6580b4d20 0x7fe6580b7230 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.663+0000 7fe676800640 1 --2- 172.21.15.16:0/2895089741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe670109930 0x7fe67019f860 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.663+0000 7fe676800640 1 --2- 172.21.15.16:0/2895089741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe670108fe0 0x7fe67019a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.663+0000 7fe676800640 1 --2- 172.21.15.16:0/2895089741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6701036b0 0x7fe67019a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.663+0000 7fe676800640 1 -- 172.21.15.16:0/2895089741 >> 172.21.15.16:0/2895089741 conn(0x7fe6700ff0d0 msgr2=0x7fe6701014f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.663+0000 7fe676800640 1 -- 172.21.15.16:0/2895089741 shutdown_connections 2024-08-31T06:20:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:20:47.663+0000 7fe676800640 1 -- 172.21.15.16:0/2895089741 wait complete. 2024-08-31T06:20:49.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:48 smithi155 ceph-mon[21742]: pgmap v787: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 5.0 MiB/s wr, 541 op/s 2024-08-31T06:20:49.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:48 smithi016 ceph-mon[17194]: pgmap v787: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 5.0 MiB/s wr, 541 op/s 2024-08-31T06:20:49.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:48 smithi184 ceph-mon[22331]: pgmap v787: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 5.0 MiB/s wr, 541 op/s 2024-08-31T06:20:51.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:50 smithi155 ceph-mon[21742]: pgmap v788: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.3 MiB/s wr, 566 op/s 2024-08-31T06:20:51.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:50 smithi016 ceph-mon[17194]: pgmap v788: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.3 MiB/s wr, 566 op/s 2024-08-31T06:20:51.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:50 smithi184 ceph-mon[22331]: pgmap v788: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.3 MiB/s wr, 566 op/s 2024-08-31T06:20:51.463 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T06:20:51.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 805.0 s, 1910.0 tps, lat 75.383 ms stddev 283.030 2024-08-31T06:20:52.109 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:20:52.915 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:52 smithi016 ceph-mon[17194]: pgmap v789: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.7 MiB/s wr, 614 op/s 2024-08-31T06:20:52.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.938+0000 7f3231e00640 1 -- 172.21.15.16:0/3793486334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f322c0776e0 msgr2=0x7f322c077af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:52.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.938+0000 7f3231e00640 1 --2- 172.21.15.16:0/3793486334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f322c0776e0 0x7f322c077af0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f321c009ca0 tx=0x7f321c02a8d0 comp rx=0 tx=0).stop 2024-08-31T06:20:52.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.939+0000 7f3231e00640 1 -- 172.21.15.16:0/3793486334 shutdown_connections 2024-08-31T06:20:52.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.939+0000 7f3231e00640 1 --2- 172.21.15.16:0/3793486334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f322c078030 0x7f322c113fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:52.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.939+0000 7f3231e00640 1 --2- 172.21.15.16:0/3793486334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f322c114510 0x7f322c116930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:52.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.939+0000 7f3231e00640 1 --2- 172.21.15.16:0/3793486334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f322c0776e0 0x7f322c077af0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:52.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.939+0000 7f3231e00640 1 -- 172.21.15.16:0/3793486334 >> 172.21.15.16:0/3793486334 conn(0x7f322c1015a0 msgr2=0x7f322c1039f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:52.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.939+0000 7f3231e00640 1 -- 172.21.15.16:0/3793486334 shutdown_connections 2024-08-31T06:20:52.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.939+0000 7f3231e00640 1 -- 172.21.15.16:0/3793486334 wait complete. 2024-08-31T06:20:52.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.940+0000 7f3231e00640 1 Processor -- start 2024-08-31T06:20:52.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.941+0000 7f3231e00640 1 -- start start 2024-08-31T06:20:52.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.941+0000 7f3231e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f322c0776e0 0x7f322c113dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:52.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.942+0000 7f3231e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f322c078030 0x7f322c10d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:52.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.942+0000 7f3231e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f322c114510 0x7f322c10d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:52.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.942+0000 7f3231e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f322c119ec0 con 0x7f322c0776e0 2024-08-31T06:20:52.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.942+0000 7f3231e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f322c119d40 con 0x7f322c078030 2024-08-31T06:20:52.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.942+0000 7f3231e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f322c11a040 con 0x7f322c114510 2024-08-31T06:20:52.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.942+0000 7f322b400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f322c114510 0x7f322c10d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:52.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.942+0000 7f322b400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f322c114510 0x7f322c10d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:53666/0 (socket says 172.21.15.16:53666) 2024-08-31T06:20:52.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.942+0000 7f322b400640 1 -- 172.21.15.16:0/1239659774 learned_addr learned my addr 172.21.15.16:0/1239659774 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:52.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.942+0000 7f322aa00640 1 --2- 172.21.15.16:0/1239659774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f322c0776e0 0x7f322c113dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:52.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.942+0000 7f3222000640 1 --2- 172.21.15.16:0/1239659774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f322c078030 0x7f322c10d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:52.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.943+0000 7f322b400640 1 -- 172.21.15.16:0/1239659774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f322c078030 msgr2=0x7f322c10d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:52.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.943+0000 7f322b400640 1 --2- 172.21.15.16:0/1239659774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f322c078030 0x7f322c10d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:52.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.943+0000 7f322b400640 1 -- 172.21.15.16:0/1239659774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f322c0776e0 msgr2=0x7f322c113dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:52.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.943+0000 7f322b400640 1 --2- 172.21.15.16:0/1239659774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f322c0776e0 0x7f322c113dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:52.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.943+0000 7f322b400640 1 -- 172.21.15.16:0/1239659774 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f322c10de90 con 0x7f322c114510 2024-08-31T06:20:52.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.943+0000 7f322b400640 1 --2- 172.21.15.16:0/1239659774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f322c114510 0x7f322c10d670 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f322400e9b0 tx=0x7f322400ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:52.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.943+0000 7f3230a00640 1 -- 172.21.15.16:0/1239659774 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322400cd80 con 0x7f322c114510 2024-08-31T06:20:52.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.943+0000 7f3230a00640 1 -- 172.21.15.16:0/1239659774 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32240035f0 con 0x7f322c114510 2024-08-31T06:20:52.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.943+0000 7f3230a00640 1 -- 172.21.15.16:0/1239659774 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3224010670 con 0x7f322c114510 2024-08-31T06:20:52.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.944+0000 7f3231e00640 1 -- 172.21.15.16:0/1239659774 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f322c112b60 con 0x7f322c114510 2024-08-31T06:20:52.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.944+0000 7f3231e00640 1 -- 172.21.15.16:0/1239659774 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f322c113050 con 0x7f322c114510 2024-08-31T06:20:52.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.949+0000 7f3230a00640 1 -- 172.21.15.16:0/1239659774 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f3224010810 con 0x7f322c114510 2024-08-31T06:20:52.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.950+0000 7f3238a50740 1 Processor -- start 2024-08-31T06:20:52.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.951+0000 7f3238a50740 1 -- start start 2024-08-31T06:20:52.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.951+0000 7f3230a00640 1 --2- 172.21.15.16:0/1239659774 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f32100b4d50 0x7f32100b7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:52.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.952+0000 7f3238a50740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321d81f070 0x56321dbc4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:52.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.952+0000 7f3238a50740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56321d7f61b0 0x56321dbc4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:52.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.952+0000 7f3238a50740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56321d7fe390 0x56321dbc53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:52.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.952+0000 7f3238a50740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56321db3b770 con 0x56321d81f070 2024-08-31T06:20:52.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.952+0000 7f3238a50740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56321db2ad00 con 0x56321d7fe390 2024-08-31T06:20:52.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.952+0000 7f3238a50740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56321db2ecb0 con 0x56321d7f61b0 2024-08-31T06:20:52.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.952+0000 7f3222000640 1 --2- 172.21.15.16:0/1239659774 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f32100b4d50 0x7f32100b7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:52.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.952+0000 7f3222000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56321d7f61b0 0x56321dbc4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:52.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.952+0000 7f3222000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56321d7f61b0 0x56321dbc4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:53670/0 (socket says 172.21.15.16:53670) 2024-08-31T06:20:52.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.952+0000 7f3222000640 1 -- 172.21.15.16:0/1828352867 learned_addr learned my addr 172.21.15.16:0/1828352867 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:52.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.953+0000 7f322b400640 1 --2- 172.21.15.16:0/1828352867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56321d7fe390 0x56321dbc53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:52.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.952+0000 7f3230a00640 1 -- 172.21.15.16:0/1239659774 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8470+0+0 (secure 0 0 0) 0x7f32240e5940 con 0x7f322c114510 2024-08-31T06:20:52.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.953+0000 7f3222000640 1 -- 172.21.15.16:0/1828352867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56321d7fe390 msgr2=0x56321dbc53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:52.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.953+0000 7f3222000640 1 --2- 172.21.15.16:0/1828352867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56321d7fe390 0x56321dbc53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:52.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.953+0000 7f3222000640 1 -- 172.21.15.16:0/1828352867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321d81f070 msgr2=0x56321dbc4950 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:20:52.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.953+0000 7f322aa00640 1 --2- 172.21.15.16:0/1828352867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321d81f070 0x56321dbc4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:52.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.954+0000 7f3222000640 1 --2- 172.21.15.16:0/1828352867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321d81f070 0x56321dbc4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:52.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.954+0000 7f3222000640 1 -- 172.21.15.16:0/1828352867 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56321d88a650 con 0x56321d7f61b0 2024-08-31T06:20:52.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.954+0000 7f322aa00640 1 --2- 172.21.15.16:0/1828352867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321d81f070 0x56321dbc4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:20:52.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.954+0000 7f3222000640 1 --2- 172.21.15.16:0/1828352867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56321d7f61b0 0x56321dbc4e90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f3214014b20 tx=0x7f3214011380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21a1be9ffd4da00c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:52.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.954+0000 7f3222000640 1 --2- 172.21.15.16:0/1239659774 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f32100b4d50 0x7f32100b7260 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3214011880 tx=0x7f321400f000 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:52.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.955+0000 7f321be00640 1 -- 172.21.15.16:0/1828352867 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3214023070 con 0x56321d7f61b0 2024-08-31T06:20:52.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.955+0000 7f321be00640 1 -- 172.21.15.16:0/1828352867 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32140111a0 con 0x56321d7f61b0 2024-08-31T06:20:52.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.955+0000 7f321be00640 1 -- 172.21.15.16:0/1828352867 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3214012740 con 0x56321d7f61b0 2024-08-31T06:20:52.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.956+0000 7f3238a50740 1 -- 172.21.15.16:0/1828352867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56321d7f61b0 msgr2=0x56321dbc4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:52.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.956+0000 7f3238a50740 1 --2- 172.21.15.16:0/1828352867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56321d7f61b0 0x56321dbc4e90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f3214014b20 tx=0x7f3214011380 comp rx=0 tx=0).stop 2024-08-31T06:20:52.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.956+0000 7f3238a50740 1 -- 172.21.15.16:0/1828352867 shutdown_connections 2024-08-31T06:20:52.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.956+0000 7f3238a50740 1 --2- 172.21.15.16:0/1828352867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56321d7f61b0 0x56321dbc4e90 secure :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f3214014b20 tx=0x7f3214011380 comp rx=0 tx=0).stop 2024-08-31T06:20:52.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.956+0000 7f3238a50740 1 --2- 172.21.15.16:0/1828352867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56321d7fe390 0x56321dbc53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:52.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.956+0000 7f3238a50740 1 --2- 172.21.15.16:0/1828352867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321d81f070 0x56321dbc4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:52.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.956+0000 7f3238a50740 1 -- 172.21.15.16:0/1828352867 >> 172.21.15.16:0/1828352867 conn(0x56321d813460 msgr2=0x56321dbc2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:52.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.956+0000 7f3238a50740 1 -- 172.21.15.16:0/1828352867 shutdown_connections 2024-08-31T06:20:52.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.957+0000 7f3238a50740 1 -- 172.21.15.16:0/1828352867 wait complete. 2024-08-31T06:20:52.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.958+0000 7f3238a50740 1 -- start start 2024-08-31T06:20:52.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.958+0000 7f3238a50740 1 Processor -- start 2024-08-31T06:20:52.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.960+0000 7f3238a50740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321d7fe390 0x56321dbd0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:52.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.960+0000 7f3238a50740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56321d81f070 0x56321dbd0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:52.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.960+0000 7f3238a50740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56321d7dd9f0 0x56321dbe73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:52.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.960+0000 7f3238a50740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56321d98d700 con 0x56321d7fe390 2024-08-31T06:20:52.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.960+0000 7f3238a50740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56321dab1cb0 con 0x56321d7dd9f0 2024-08-31T06:20:52.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.960+0000 7f3238a50740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56321dbe40e0 con 0x56321d81f070 2024-08-31T06:20:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.960+0000 7f3238a50740 10 client.0 _ll_register_callbacks cb 0x56321dad7430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:20:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.961+0000 7f322aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321d7fe390 0x56321dbd0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.961+0000 7f322aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321d7fe390 0x56321dbd0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35372/0 (socket says 172.21.15.16:35372) 2024-08-31T06:20:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.961+0000 7f322aa00640 1 -- 172.21.15.16:0/3531616521 learned_addr learned my addr 172.21.15.16:0/3531616521 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.961+0000 7f322b400640 1 --2- 172.21.15.16:0/3531616521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56321d7dd9f0 0x56321dbe73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:52.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.962+0000 7f322b400640 1 -- 172.21.15.16:0/3531616521 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56321d81f070 msgr2=0x56321dbd0be0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T06:20:52.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.962+0000 7f322b400640 1 --2- 172.21.15.16:0/3531616521 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56321d81f070 0x56321dbd0be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:52.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.962+0000 7f322b400640 1 -- 172.21.15.16:0/3531616521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321d7fe390 msgr2=0x56321dbd0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:52.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.962+0000 7f322b400640 1 --2- 172.21.15.16:0/3531616521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321d7fe390 0x56321dbd0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:52.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.962+0000 7f322b400640 1 -- 172.21.15.16:0/3531616521 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56321daf2870 con 0x56321d7dd9f0 2024-08-31T06:20:52.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.962+0000 7f322b400640 1 --2- 172.21.15.16:0/3531616521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56321d7dd9f0 0x56321dbe73a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f32240e30c0 tx=0x7f32240ad600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dea3d730327d7a62 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:52.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.962+0000 7f3218c00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:20:52.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.963+0000 7f3218c00640 1 -- 172.21.15.16:0/3531616521 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32240981b0 con 0x56321d7dd9f0 2024-08-31T06:20:52.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.963+0000 7f3218c00640 1 -- 172.21.15.16:0/3531616521 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8470+0+0 (secure 0 0 0) 0x7f32240a9190 con 0x56321d7dd9f0 2024-08-31T06:20:52.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.963+0000 7f3238a50740 1 -- 172.21.15.16:0/3531616521 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32140111a0 con 0x56321d7dd9f0 2024-08-31T06:20:52.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.963+0000 7f3218c00640 1 -- 172.21.15.16:0/3531616521 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f32240e3340 con 0x56321d7dd9f0 2024-08-31T06:20:52.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.963+0000 7f3238a50740 10 client.28655 fetch_fsmap learned FSMap version 49 2024-08-31T06:20:52.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.963+0000 7f3238a50740 1 -- 172.21.15.16:0/3531616521 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x56321dbf01f0 con 0x56321d7dd9f0 2024-08-31T06:20:52.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.963+0000 7f3218c00640 1 -- 172.21.15.16:0/3531616521 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32140111a0 con 0x56321d7dd9f0 2024-08-31T06:20:52.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.964+0000 7f3218c00640 1 -- 172.21.15.16:0/3531616521 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f32240940c0 con 0x56321d7dd9f0 2024-08-31T06:20:52.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.964+0000 7f3238a50740 10 client.28655 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:20:52.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.964+0000 7f3238a50740 10 client.28655 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:20:52.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.964+0000 7f3238a50740 20 client.28655 populate_metadata read hostname 'smithi016' 2024-08-31T06:20:52.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.964+0000 7f3238a50740 1 --2- 172.21.15.16:0/3531616521 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56321d7b2da0 0x56321dbf37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:52.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.964+0000 7f3238a50740 4 client.28655 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:20:52.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.964+0000 7f3238a50740 1 -- 172.21.15.16:0/3531616521 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56321db3d6c0 con 0x56321d7b2da0 2024-08-31T06:20:52.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.965+0000 7f322aa00640 1 --2- 172.21.15.16:0/3531616521 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56321d7b2da0 0x56321dbf37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:52.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.967+0000 7f322aa00640 1 --2- 172.21.15.16:0/3531616521 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56321d7b2da0 0x56321dbf37b0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6945b1e6182fd2b6 server_cookie=b323877af69bdf7f in_seq=0 out_seq=0 2024-08-31T06:20:52.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.967+0000 7f3218c00640 10 client.28655 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:20:52.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.968+0000 7f3218c00640 1 -- 172.21.15.16:0/3531616521 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56321db3d6c0 con 0x56321d7b2da0 2024-08-31T06:20:52.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.968+0000 7f3218c00640 10 client.28655 handle_command_reply: tid=0 2024-08-31T06:20:52.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.968+0000 7f3238a50740 1 client.28655 shutdown 2024-08-31T06:20:52.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.968+0000 7f3238a50740 1 -- 172.21.15.16:0/3531616521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56321d7dd9f0 msgr2=0x56321dbe73a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:52.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.968+0000 7f3238a50740 1 --2- 172.21.15.16:0/3531616521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56321d7dd9f0 0x56321dbe73a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f32240e30c0 tx=0x7f32240ad600 comp rx=0 tx=0).stop 2024-08-31T06:20:52.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.968+0000 7f3238a50740 1 -- 172.21.15.16:0/3531616521 shutdown_connections 2024-08-31T06:20:52.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.968+0000 7f3238a50740 1 --2- 172.21.15.16:0/3531616521 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56321d7b2da0 0x56321dbf37b0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:52.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.968+0000 7f3238a50740 1 --2- 172.21.15.16:0/3531616521 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56321d81f070 0x56321dbd0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:52.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.968+0000 7f3238a50740 1 --2- 172.21.15.16:0/3531616521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56321d7dd9f0 0x56321dbe73a0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:52.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.968+0000 7f3238a50740 1 --2- 172.21.15.16:0/3531616521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321d7fe390 0x56321dbd0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:52.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.968+0000 7f3238a50740 1 -- 172.21.15.16:0/3531616521 >> 172.21.15.16:0/3531616521 conn(0x56321d813460 msgr2=0x56321dbc7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:52.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.968+0000 7f3218c00640 0 client.28655 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:20:52.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.969+0000 7f3238a50740 1 -- 172.21.15.16:0/3531616521 shutdown_connections 2024-08-31T06:20:52.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.969+0000 7f3238a50740 1 -- 172.21.15.16:0/3531616521 wait complete. 2024-08-31T06:20:52.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:52.969+0000 7f3238a50740 20 client.28655 trim_cache size 0 max 16384 2024-08-31T06:20:53.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.010+0000 7f3238a50740 1 Processor -- start 2024-08-31T06:20:53.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.011+0000 7f3238a50740 1 -- start start 2024-08-31T06:20:53.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.011+0000 7f3238a50740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56321d7dd9f0 0x56321dbc2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:53.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.011+0000 7f3238a50740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321d7fe390 0x56321dbee170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:53.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.011+0000 7f3238a50740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56321d81f070 0x56321dbc9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:53.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.011+0000 7f3238a50740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f322c1b83c0 con 0x56321d7fe390 2024-08-31T06:20:53.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.011+0000 7f3238a50740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f322c074c10 con 0x56321d7dd9f0 2024-08-31T06:20:53.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.011+0000 7f3238a50740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f322c077af0 con 0x56321d81f070 2024-08-31T06:20:53.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.012+0000 7f3222000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321d7fe390 0x56321dbee170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:53.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.012+0000 7f322b400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56321d81f070 0x56321dbc9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:53.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.012+0000 7f322b400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56321d81f070 0x56321dbc9bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:53696/0 (socket says 172.21.15.16:53696) 2024-08-31T06:20:53.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.012+0000 7f322b400640 1 -- 172.21.15.16:0/3524091145 learned_addr learned my addr 172.21.15.16:0/3524091145 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:53.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.013+0000 7f322b400640 1 -- 172.21.15.16:0/3524091145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56321d7dd9f0 msgr2=0x56321dbc2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:53.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.013+0000 7f322b400640 1 --2- 172.21.15.16:0/3524091145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56321d7dd9f0 0x56321dbc2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:53.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.013+0000 7f322b400640 1 -- 172.21.15.16:0/3524091145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321d7fe390 msgr2=0x56321dbee170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:53.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.013+0000 7f322b400640 1 --2- 172.21.15.16:0/3524091145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321d7fe390 0x56321dbee170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:53.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.013+0000 7f322b400640 1 -- 172.21.15.16:0/3524091145 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56321dbbde90 con 0x56321d81f070 2024-08-31T06:20:53.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.015+0000 7f3222000640 1 --2- 172.21.15.16:0/3524091145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321d7fe390 0x56321dbee170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:20:53.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.015+0000 7f322b400640 1 --2- 172.21.15.16:0/3524091145 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56321d81f070 0x56321dbc9bd0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f32240ac7e0 tx=0x7f32240ad770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b447418c93265ca server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:53.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.015+0000 7f321b400640 1 -- 172.21.15.16:0/3524091145 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32240cb070 con 0x56321d81f070 2024-08-31T06:20:53.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.016+0000 7f321b400640 1 -- 172.21.15.16:0/3524091145 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32240e34e0 con 0x56321d81f070 2024-08-31T06:20:53.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.016+0000 7f321b400640 1 -- 172.21.15.16:0/3524091145 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32240b5150 con 0x56321d81f070 2024-08-31T06:20:53.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.017+0000 7f3238a50740 1 -- 172.21.15.16:0/3524091145 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56321d81f070 msgr2=0x56321dbc9bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.017+0000 7f3238a50740 1 --2- 172.21.15.16:0/3524091145 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56321d81f070 0x56321dbc9bd0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f32240ac7e0 tx=0x7f32240ad770 comp rx=0 tx=0).stop 2024-08-31T06:20:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.017+0000 7f3238a50740 1 -- 172.21.15.16:0/3524091145 shutdown_connections 2024-08-31T06:20:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.017+0000 7f3238a50740 1 --2- 172.21.15.16:0/3524091145 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56321d81f070 0x56321dbc9bd0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.017+0000 7f3238a50740 1 --2- 172.21.15.16:0/3524091145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321d7fe390 0x56321dbee170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.017+0000 7f3238a50740 1 --2- 172.21.15.16:0/3524091145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56321d7dd9f0 0x56321dbc2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.017+0000 7f3238a50740 1 -- 172.21.15.16:0/3524091145 >> 172.21.15.16:0/3524091145 conn(0x56321d813460 msgr2=0x56321dbc2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.018+0000 7f3238a50740 1 -- 172.21.15.16:0/3524091145 shutdown_connections 2024-08-31T06:20:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.018+0000 7f3238a50740 1 -- 172.21.15.16:0/3524091145 wait complete. 2024-08-31T06:20:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.018+0000 7f3238a50740 1 -- start start 2024-08-31T06:20:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.019+0000 7f3238a50740 1 Processor -- start 2024-08-31T06:20:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.021+0000 7f3238a50740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56321d7dd9f0 0x56321dbe8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.021+0000 7f3238a50740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56321d7fe390 0x56321dbe8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.022+0000 7f3238a50740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321d81f070 0x56321dbe8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.022+0000 7f3238a50740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f322c0730f0 con 0x56321d81f070 2024-08-31T06:20:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.022+0000 7f322aa00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56321d7dd9f0 0x56321dbe8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.023+0000 7f322aa00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56321d7dd9f0 0x56321dbe8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:53706/0 (socket says 172.21.15.16:53706) 2024-08-31T06:20:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.023+0000 7f3222000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56321d7fe390 0x56321dbe8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.024+0000 7f322aa00640 1 -- 172.21.15.16:0/2388080320 learned_addr learned my addr 172.21.15.16:0/2388080320 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.024+0000 7f3238a50740 1 -- 172.21.15.16:0/2388080320 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f322c074540 con 0x56321d7fe390 2024-08-31T06:20:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.024+0000 7f3238a50740 1 -- 172.21.15.16:0/2388080320 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f322c0737a0 con 0x56321d7dd9f0 2024-08-31T06:20:53.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.024+0000 7f3238a50740 10 client.0 _ll_register_callbacks cb 0x7f32240aca10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:20:53.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.024+0000 7f322b400640 1 --2- 172.21.15.16:0/2388080320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321d81f070 0x56321dbe8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.025+0000 7f322aa00640 1 -- 172.21.15.16:0/2388080320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56321d7fe390 msgr2=0x56321dbe8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.026+0000 7f322aa00640 1 --2- 172.21.15.16:0/2388080320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56321d7fe390 0x56321dbe8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.026+0000 7f322aa00640 1 -- 172.21.15.16:0/2388080320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321d81f070 msgr2=0x56321dbe8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.026+0000 7f322aa00640 1 --2- 172.21.15.16:0/2388080320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321d81f070 0x56321dbe8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.026+0000 7f322aa00640 1 -- 172.21.15.16:0/2388080320 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56321dbed250 con 0x56321d7dd9f0 2024-08-31T06:20:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.026+0000 7f322b400640 1 --2- 172.21.15.16:0/2388080320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321d81f070 0x56321dbe8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:20:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.026+0000 7f3222000640 1 --2- 172.21.15.16:0/2388080320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56321d7fe390 0x56321dbe8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:20:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.027+0000 7f322aa00640 1 --2- 172.21.15.16:0/2388080320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56321d7dd9f0 0x56321dbe8050 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f321c005e30 tx=0x7f321c0316e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4872dcc0382b6069 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.027+0000 7f3218c00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:20:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.027+0000 7f3218c00640 1 -- 172.21.15.16:0/2388080320 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321c031e50 con 0x56321d7dd9f0 2024-08-31T06:20:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.027+0000 7f3218c00640 1 -- 172.21.15.16:0/2388080320 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8470+0+0 (secure 0 0 0) 0x7f321c003970 con 0x56321d7dd9f0 2024-08-31T06:20:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.028+0000 7f3238a50740 1 -- 172.21.15.16:0/2388080320 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32240e34e0 con 0x56321d7dd9f0 2024-08-31T06:20:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.028+0000 7f3218c00640 1 -- 172.21.15.16:0/2388080320 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f321c031850 con 0x56321d7dd9f0 2024-08-31T06:20:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.028+0000 7f3238a50740 10 client.28663 fetch_fsmap learned FSMap version 49 2024-08-31T06:20:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.028+0000 7f3238a50740 1 -- 172.21.15.16:0/2388080320 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x56321dc5e6b0 con 0x56321d7dd9f0 2024-08-31T06:20:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.028+0000 7f3218c00640 1 -- 172.21.15.16:0/2388080320 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32240e34e0 con 0x56321d7dd9f0 2024-08-31T06:20:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.029+0000 7f3218c00640 1 -- 172.21.15.16:0/2388080320 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f321c030660 con 0x56321d7dd9f0 2024-08-31T06:20:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.029+0000 7f3238a50740 10 client.28663 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:20:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.029+0000 7f3238a50740 10 client.28663 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:20:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.029+0000 7f3238a50740 20 client.28663 populate_metadata read hostname 'smithi016' 2024-08-31T06:20:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.029+0000 7f3238a50740 1 --2- 172.21.15.16:0/2388080320 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56321dc5eec0 0x56321dc612e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.029+0000 7f3238a50740 4 client.28663 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T06:20:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.029+0000 7f3238a50740 1 -- 172.21.15.16:0/2388080320 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56321db39160 con 0x56321dc5eec0 2024-08-31T06:20:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.030+0000 7f3222000640 1 --2- 172.21.15.16:0/2388080320 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56321dc5eec0 0x56321dc612e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.031+0000 7f3218c00640 10 client.28663 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:20:53.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.032+0000 7f3222000640 1 --2- 172.21.15.16:0/2388080320 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56321dc5eec0 0x56321dc612e0 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c623e223a31ae47e server_cookie=e8a60bdfb8168de1 in_seq=0 out_seq=0 2024-08-31T06:20:53.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.033+0000 7f3218c00640 1 -- 172.21.15.16:0/2388080320 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4918 (crc 0 0 0) 0x56321db39160 con 0x56321dc5eec0 2024-08-31T06:20:53.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.033+0000 7f3218c00640 10 client.28663 handle_command_reply: tid=0 2024-08-31T06:20:53.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.033+0000 7f3238a50740 1 client.28663 shutdown 2024-08-31T06:20:53.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.033+0000 7f3238a50740 1 -- 172.21.15.16:0/2388080320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56321d7dd9f0 msgr2=0x56321dbe8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:53.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.033+0000 7f3238a50740 1 --2- 172.21.15.16:0/2388080320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56321d7dd9f0 0x56321dbe8050 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f321c005e30 tx=0x7f321c0316e0 comp rx=0 tx=0).stop 2024-08-31T06:20:53.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.033+0000 7f3238a50740 1 -- 172.21.15.16:0/2388080320 shutdown_connections 2024-08-31T06:20:53.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.033+0000 7f3238a50740 1 --2- 172.21.15.16:0/2388080320 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x56321dc5eec0 0x56321dc612e0 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:53.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.033+0000 7f3238a50740 1 --2- 172.21.15.16:0/2388080320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321d81f070 0x56321dbe8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:53.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.033+0000 7f3238a50740 1 --2- 172.21.15.16:0/2388080320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56321d7fe390 0x56321dbe8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:53.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.033+0000 7f3238a50740 1 --2- 172.21.15.16:0/2388080320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56321d7dd9f0 0x56321dbe8050 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:53.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.033+0000 7f3238a50740 1 -- 172.21.15.16:0/2388080320 >> 172.21.15.16:0/2388080320 conn(0x56321d813460 msgr2=0x56321dbc27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:53.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.034+0000 7f3218c00640 0 client.28663 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:20:53.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.034+0000 7f3238a50740 1 -- 172.21.15.16:0/2388080320 shutdown_connections 2024-08-31T06:20:53.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.034+0000 7f3238a50740 1 -- 172.21.15.16:0/2388080320 wait complete. 2024-08-31T06:20:53.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.034+0000 7f3238a50740 20 client.28663 trim_cache size 0 max 16384 2024-08-31T06:20:53.039 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:20:53.039 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-08-31T06:20:53.039 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {}, 2024-08-31T06:20:53.039 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-08-31T06:20:53.039 INFO:teuthology.orchestra.run.smithi016.stdout: "18e112f4-a5f9-4602-aaad-3b13fa5a4c2e": { 2024-08-31T06:20:53.039 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:20:16.570535+0000", 2024-08-31T06:20:53.040 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:20:53.040 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:20:53.040 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:20:53.040 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:20:53.040 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:20:53.040 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:20:53.040 INFO:teuthology.orchestra.run.smithi016.stdout: "f1cdcb27-e3e4-4dc8-a0ae-7ace761d04c5": { 2024-08-31T06:20:53.040 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:19:22.275683+0000", 2024-08-31T06:20:53.040 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:20:53.040 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:20:53.040 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:20:53.040 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:20:53.040 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:20:53.041 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:20:53.041 INFO:teuthology.orchestra.run.smithi016.stdout: "0e44eb18-439c-46e4-a298-7d4d2ecd0b4a": { 2024-08-31T06:20:53.041 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:18:31.130765+0000", 2024-08-31T06:20:53.041 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:20:53.041 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:20:53.041 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:20:53.041 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:20:53.041 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:20:53.041 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:20:53.041 INFO:teuthology.orchestra.run.smithi016.stdout: "e497cd88-1e7c-4631-a4a8-cdd8e1fa6c83": { 2024-08-31T06:20:53.041 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:17:42.476795+0000", 2024-08-31T06:20:53.041 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:20:53.041 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:20:53.041 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:20:53.042 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:20:53.042 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:20:53.042 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:20:53.042 INFO:teuthology.orchestra.run.smithi016.stdout: "c300b7e8-f3fd-482c-a909-9329bc61e570": { 2024-08-31T06:20:53.042 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:06:49.868046+0000", 2024-08-31T06:20:53.042 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:20:53.042 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:20:53.042 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:20:53.042 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:20:53.042 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:20:53.042 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:20:53.042 INFO:teuthology.orchestra.run.smithi016.stdout: "6a03046e-cdf4-4a14-b9bd-b560b8968467": { 2024-08-31T06:20:53.042 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:12:38.002354+0000", 2024-08-31T06:20:53.042 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:20:53.042 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:20:53.043 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:20:53.043 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:20:53.043 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:20:53.043 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:20:53.043 INFO:teuthology.orchestra.run.smithi016.stdout: "004b47bb-0fe8-4d28-9d9d-78bde6a61199": { 2024-08-31T06:20:53.043 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:16:48.915926+0000", 2024-08-31T06:20:53.043 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:20:53.043 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:20:53.043 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:20:53.043 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:20:53.043 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:20:53.043 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:20:53.043 INFO:teuthology.orchestra.run.smithi016.stdout: "88d7bdb8-cb1c-45e5-ad29-4cd8da6c6003": { 2024-08-31T06:20:53.043 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:07:39.648942+0000", 2024-08-31T06:20:53.044 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:20:53.044 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:20:53.044 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:20:53.044 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:20:53.044 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:20:53.044 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:20:53.044 INFO:teuthology.orchestra.run.smithi016.stdout: "0fdcd762-533d-42be-9289-b71c8e3b81b1": { 2024-08-31T06:20:53.044 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:15:06.169112+0000", 2024-08-31T06:20:53.044 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:20:53.044 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:20:53.044 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:20:53.044 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:20:53.044 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:20:53.045 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:20:53.045 INFO:teuthology.orchestra.run.smithi016.stdout: "c8455d74-2fe7-4d93-bcf2-19b419d16016": { 2024-08-31T06:20:53.045 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:08:24.035960+0000", 2024-08-31T06:20:53.045 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:20:53.045 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:20:53.045 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:20:53.045 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:20:53.045 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:20:53.045 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:20:53.045 INFO:teuthology.orchestra.run.smithi016.stdout: "78d6ef02-8551-45e8-ad60-9627eb573fc1": { 2024-08-31T06:20:53.045 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:09:13.547865+0000", 2024-08-31T06:20:53.045 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:20:53.045 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:20:53.045 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:20:53.045 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:20:53.046 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:20:53.046 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:20:53.046 INFO:teuthology.orchestra.run.smithi016.stdout: "c1fd6904-2e02-47fa-939a-47349b9fe36b": { 2024-08-31T06:20:53.046 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:06.678691+0000", 2024-08-31T06:20:53.046 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:20:53.046 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:20:53.046 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:20:53.046 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:20:53.046 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:20:53.046 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:20:53.046 INFO:teuthology.orchestra.run.smithi016.stdout: "a10cce76-99ab-46b5-b3b2-07b82899b8f9": { 2024-08-31T06:20:53.046 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:11:43.967546+0000", 2024-08-31T06:20:53.046 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:20:53.046 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:20:53.047 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:20:53.047 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:20:53.047 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:20:53.047 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:20:53.047 INFO:teuthology.orchestra.run.smithi016.stdout: "91b72a34-4ba9-4126-a414-8b862f8c5f78": { 2024-08-31T06:20:53.047 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:13:28.663237+0000", 2024-08-31T06:20:53.047 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:20:53.047 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:20:53.047 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:20:53.047 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:20:53.048 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:20:53.048 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:20:53.048 INFO:teuthology.orchestra.run.smithi016.stdout: "e739a16e-8ace-43a8-a1ce-ac27d39ef8a3": { 2024-08-31T06:20:53.048 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:54.084646+0000", 2024-08-31T06:20:53.048 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:20:53.048 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:20:53.048 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:20:53.048 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:20:53.048 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:20:53.048 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:20:53.048 INFO:teuthology.orchestra.run.smithi016.stdout: "a64d5301-8558-4e9c-ac9b-8d0ac7057e74": { 2024-08-31T06:20:53.048 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:14:20.338104+0000", 2024-08-31T06:20:53.048 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:20:53.048 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:20:53.049 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:20:53.049 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:20:53.049 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:20:53.049 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:20:53.049 INFO:teuthology.orchestra.run.smithi016.stdout: "f9c4a22e-0c8a-4928-9b0d-e2ce1db5a86b": { 2024-08-31T06:20:53.049 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:15:57.496015+0000", 2024-08-31T06:20:53.049 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:20:53.049 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:20:53.049 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:20:53.049 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:20:53.049 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:20:53.049 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:20:53.049 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:20:53.049 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:20:53.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.037+0000 7f3231e00640 1 -- 172.21.15.16:0/1239659774 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f32100b4d50 msgr2=0x7f32100b7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:53.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.037+0000 7f3231e00640 1 --2- 172.21.15.16:0/1239659774 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f32100b4d50 0x7f32100b7260 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3214011880 tx=0x7f321400f000 comp rx=0 tx=0).stop 2024-08-31T06:20:53.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.038+0000 7f3231e00640 1 -- 172.21.15.16:0/1239659774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f322c114510 msgr2=0x7f322c10d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:53.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.038+0000 7f3231e00640 1 --2- 172.21.15.16:0/1239659774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f322c114510 0x7f322c10d670 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f322400e9b0 tx=0x7f322400ee70 comp rx=0 tx=0).stop 2024-08-31T06:20:53.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.038+0000 7f3231e00640 1 -- 172.21.15.16:0/1239659774 shutdown_connections 2024-08-31T06:20:53.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.038+0000 7f3231e00640 1 --2- 172.21.15.16:0/1239659774 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f32100b4d50 0x7f32100b7260 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:53.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.038+0000 7f3231e00640 1 --2- 172.21.15.16:0/1239659774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f322c114510 0x7f322c10d670 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:53.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.038+0000 7f3231e00640 1 --2- 172.21.15.16:0/1239659774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f322c078030 0x7f322c10d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:53.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.038+0000 7f3231e00640 1 --2- 172.21.15.16:0/1239659774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f322c0776e0 0x7f322c113dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:53.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.038+0000 7f3231e00640 1 -- 172.21.15.16:0/1239659774 >> 172.21.15.16:0/1239659774 conn(0x7f322c1015a0 msgr2=0x7f322c116580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:53.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.038+0000 7f3231e00640 1 -- 172.21.15.16:0/1239659774 shutdown_connections 2024-08-31T06:20:53.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:53.038+0000 7f3231e00640 1 -- 172.21.15.16:0/1239659774 wait complete. 2024-08-31T06:20:53.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:52 smithi155 ceph-mon[21742]: pgmap v789: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.7 MiB/s wr, 614 op/s 2024-08-31T06:20:53.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:52 smithi184 ceph-mon[22331]: pgmap v789: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.7 MiB/s wr, 614 op/s 2024-08-31T06:20:54.171 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T06:20:54.171 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:20:54.820 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:20:55.052 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:54 smithi016 ceph-mon[17194]: pgmap v790: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.2 MiB/s wr, 568 op/s 2024-08-31T06:20:55.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:54 smithi155 ceph-mon[21742]: pgmap v790: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.2 MiB/s wr, 568 op/s 2024-08-31T06:20:55.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:54 smithi184 ceph-mon[22331]: pgmap v790: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.2 MiB/s wr, 568 op/s 2024-08-31T06:20:55.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:55.818+0000 7f3c60e00640 1 -- 172.21.15.16:0/814009076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c5c105940 msgr2=0x7f3c5c105d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:55.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:55.818+0000 7f3c60e00640 1 --2- 172.21.15.16:0/814009076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c5c105940 0x7f3c5c105d50 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f3c54013fd0 tx=0x7f3c54032d90 comp rx=0 tx=0).stop 2024-08-31T06:20:55.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:55.818+0000 7f3c60e00640 1 -- 172.21.15.16:0/814009076 shutdown_connections 2024-08-31T06:20:55.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:55.818+0000 7f3c60e00640 1 --2- 172.21.15.16:0/814009076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c5c10bbd0 0x7f3c5c112400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:55.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:55.818+0000 7f3c60e00640 1 --2- 172.21.15.16:0/814009076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c5c10b280 0x7f3c5c10b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:55.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:55.818+0000 7f3c60e00640 1 --2- 172.21.15.16:0/814009076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c5c105940 0x7f3c5c105d50 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:55.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:55.818+0000 7f3c60e00640 1 -- 172.21.15.16:0/814009076 >> 172.21.15.16:0/814009076 conn(0x7f3c5c1013b0 msgr2=0x7f3c5c103800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:55.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:55.819+0000 7f3c60e00640 1 -- 172.21.15.16:0/814009076 shutdown_connections 2024-08-31T06:20:55.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:55.819+0000 7f3c60e00640 1 -- 172.21.15.16:0/814009076 wait complete. 2024-08-31T06:20:55.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:55.821+0000 7f3c60e00640 1 Processor -- start 2024-08-31T06:20:55.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:55.821+0000 7f3c60e00640 1 -- start start 2024-08-31T06:20:55.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:55.822+0000 7f3c60e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c5c105940 0x7f3c5c197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:55.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:55.822+0000 7f3c60e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c5c10b280 0x7f3c5c1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:55.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:55.822+0000 7f3c60e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c5c10bbd0 0x7f3c5c19d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:55.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:55.822+0000 7f3c60e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c079e30 con 0x7f3c5c105940 2024-08-31T06:20:55.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:55.822+0000 7f3c60e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c079cb0 con 0x7f3c5c10b280 2024-08-31T06:20:55.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:55.822+0000 7f3c60e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c079fb0 con 0x7f3c5c10bbd0 2024-08-31T06:20:55.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:55.823+0000 7f3c5be00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c5c10bbd0 0x7f3c5c19d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:55.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:55.823+0000 7f3c5be00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c5c10bbd0 0x7f3c5c19d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:39154/0 (socket says 172.21.15.16:39154) 2024-08-31T06:20:55.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:55.823+0000 7f3c5be00640 1 -- 172.21.15.16:0/3879604138 learned_addr learned my addr 172.21.15.16:0/3879604138 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:55.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:55.823+0000 7f3c5aa00640 1 --2- 172.21.15.16:0/3879604138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c5c10b280 0x7f3c5c1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:55.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:55.823+0000 7f3c5be00640 1 -- 172.21.15.16:0/3879604138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c5c10b280 msgr2=0x7f3c5c1984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:55.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:55.823+0000 7f3c5be00640 1 --2- 172.21.15.16:0/3879604138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c5c10b280 0x7f3c5c1984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:55.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:55.823+0000 7f3c5be00640 1 -- 172.21.15.16:0/3879604138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c5c105940 msgr2=0x7f3c5c197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:55.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:55.823+0000 7f3c5be00640 1 --2- 172.21.15.16:0/3879604138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c5c105940 0x7f3c5c197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:55.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:55.823+0000 7f3c5be00640 1 -- 172.21.15.16:0/3879604138 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c5c19de10 con 0x7f3c5c10bbd0 2024-08-31T06:20:55.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:55.824+0000 7f3c5be00640 1 --2- 172.21.15.16:0/3879604138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c5c10bbd0 0x7f3c5c19d6e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c00e970 tx=0x7f3c4c00ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:55.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:55.825+0000 7f3c53e00640 1 -- 172.21.15.16:0/3879604138 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c00ccc0 con 0x7f3c5c10bbd0 2024-08-31T06:20:55.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:55.825+0000 7f3c53e00640 1 -- 172.21.15.16:0/3879604138 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c4c00ce60 con 0x7f3c5c10bbd0 2024-08-31T06:20:55.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:55.825+0000 7f3c53e00640 1 -- 172.21.15.16:0/3879604138 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c010430 con 0x7f3c5c10bbd0 2024-08-31T06:20:55.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:55.825+0000 7f3c60e00640 1 -- 172.21.15.16:0/3879604138 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c5c19e0a0 con 0x7f3c5c10bbd0 2024-08-31T06:20:55.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:55.825+0000 7f3c60e00640 1 -- 172.21.15.16:0/3879604138 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c5c1a4860 con 0x7f3c5c10bbd0 2024-08-31T06:20:55.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:55.827+0000 7f3c60e00640 1 -- 172.21.15.16:0/3879604138 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c5c105d50 con 0x7f3c5c10bbd0 2024-08-31T06:20:55.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:55.830+0000 7f3c53e00640 1 -- 172.21.15.16:0/3879604138 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f3c4c004ff0 con 0x7f3c5c10bbd0 2024-08-31T06:20:55.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:55.833+0000 7f3c53e00640 1 --2- 172.21.15.16:0/3879604138 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3c400b4eb0 0x7f3c400b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:55.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:55.833+0000 7f3c53e00640 1 -- 172.21.15.16:0/3879604138 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8470+0+0 (secure 0 0 0) 0x7f3c4c0e59c0 con 0x7f3c5c10bbd0 2024-08-31T06:20:55.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:55.833+0000 7f3c5b400640 1 --2- 172.21.15.16:0/3879604138 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3c400b4eb0 0x7f3c400b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:55.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:55.834+0000 7f3c5b400640 1 --2- 172.21.15.16:0/3879604138 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3c400b4eb0 0x7f3c400b73c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3c5403e040 tx=0x7f3c54000f40 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:55.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:55.835+0000 7f3c53e00640 1 -- 172.21.15.16:0/3879604138 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f3c4c0ac780 con 0x7f3c5c10bbd0 2024-08-31T06:20:56.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:56.271+0000 7f3c60e00640 1 -- 172.21.15.16:0/3879604138 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c5c1a4c20 con 0x7f3c5c10bbd0 2024-08-31T06:20:56.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:56.272+0000 7f3c53e00640 1 -- 172.21.15.16:0/3879604138 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+11923 (secure 0 0 0) 0x7f3c4c0ac080 con 0x7f3c5c10bbd0 2024-08-31T06:20:56.276 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:20:56.276 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 49 2024-08-31T06:20:56.277 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":49,"btime":"2024-08-31T06:19:17:811356+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":49,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:19:17.033815+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24737,24724,24712]},"id":1}]} 2024-08-31T06:20:56.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:56.278+0000 7f3c60e00640 1 -- 172.21.15.16:0/3879604138 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3c400b4eb0 msgr2=0x7f3c400b73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:56.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:56.278+0000 7f3c60e00640 1 --2- 172.21.15.16:0/3879604138 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3c400b4eb0 0x7f3c400b73c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3c5403e040 tx=0x7f3c54000f40 comp rx=0 tx=0).stop 2024-08-31T06:20:56.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:56.278+0000 7f3c60e00640 1 -- 172.21.15.16:0/3879604138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c5c10bbd0 msgr2=0x7f3c5c19d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:56.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:56.278+0000 7f3c60e00640 1 --2- 172.21.15.16:0/3879604138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c5c10bbd0 0x7f3c5c19d6e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c00e970 tx=0x7f3c4c00ed70 comp rx=0 tx=0).stop 2024-08-31T06:20:56.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:56.279+0000 7f3c60e00640 1 -- 172.21.15.16:0/3879604138 shutdown_connections 2024-08-31T06:20:56.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:56.279+0000 7f3c60e00640 1 --2- 172.21.15.16:0/3879604138 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3c400b4eb0 0x7f3c400b73c0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:56.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:56.279+0000 7f3c60e00640 1 --2- 172.21.15.16:0/3879604138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c5c10bbd0 0x7f3c5c19d6e0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:56.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:56.279+0000 7f3c60e00640 1 --2- 172.21.15.16:0/3879604138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c5c10b280 0x7f3c5c1984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:56.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:56.279+0000 7f3c60e00640 1 --2- 172.21.15.16:0/3879604138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c5c105940 0x7f3c5c197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:56.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:56.279+0000 7f3c60e00640 1 -- 172.21.15.16:0/3879604138 >> 172.21.15.16:0/3879604138 conn(0x7f3c5c1013b0 msgr2=0x7f3c5c1037d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:56.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:56.279+0000 7f3c60e00640 1 -- 172.21.15.16:0/3879604138 shutdown_connections 2024-08-31T06:20:56.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:56.279+0000 7f3c60e00640 1 -- 172.21.15.16:0/3879604138 wait complete. 2024-08-31T06:20:56.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 810.0 s, 1415.6 tps, lat 70.272 ms stddev 84.798 2024-08-31T06:20:57.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:56 smithi155 ceph-mon[21742]: pgmap v791: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.2 MiB/s wr, 671 op/s 2024-08-31T06:20:57.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:56 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/3879604138' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:20:57.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:56 smithi016 ceph-mon[17194]: pgmap v791: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.2 MiB/s wr, 671 op/s 2024-08-31T06:20:57.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:56 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/3879604138' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:20:57.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:56 smithi184 ceph-mon[22331]: pgmap v791: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.2 MiB/s wr, 671 op/s 2024-08-31T06:20:57.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:56 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/3879604138' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:20:57.339 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T06:20:57.999 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:20:58.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.859+0000 7f87ba600640 1 -- 172.21.15.16:0/1230393421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87b410f6a0 msgr2=0x7f87b410fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:58.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.859+0000 7f87ba600640 1 --2- 172.21.15.16:0/1230393421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87b410f6a0 0x7f87b410fab0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f87a8009900 tx=0x7f87a802a410 comp rx=0 tx=0).stop 2024-08-31T06:20:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.861+0000 7f87ba600640 1 -- 172.21.15.16:0/1230393421 shutdown_connections 2024-08-31T06:20:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.861+0000 7f87ba600640 1 --2- 172.21.15.16:0/1230393421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87b410f6a0 0x7f87b410fab0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.861+0000 7f87ba600640 1 --2- 172.21.15.16:0/1230393421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b410fff0 0x7f87b4116820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.861+0000 7f87ba600640 1 --2- 172.21.15.16:0/1230393421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87b4109d60 0x7f87b410a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.861+0000 7f87ba600640 1 -- 172.21.15.16:0/1230393421 >> 172.21.15.16:0/1230393421 conn(0x7f87b4079360 msgr2=0x7f87b4107f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:58.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.863+0000 7f87ba600640 1 -- 172.21.15.16:0/1230393421 shutdown_connections 2024-08-31T06:20:58.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.863+0000 7f87ba600640 1 -- 172.21.15.16:0/1230393421 wait complete. 2024-08-31T06:20:58.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.864+0000 7f87ba600640 1 Processor -- start 2024-08-31T06:20:58.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.866+0000 7f87ba600640 1 -- start start 2024-08-31T06:20:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.868+0000 7f87ba600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b4109d60 0x7f87b4083af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.868+0000 7f87ba600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87b410f6a0 0x7f87b407ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.869+0000 7f87ba600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87b410fff0 0x7f87b407e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.869+0000 7f87b9200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b4109d60 0x7f87b4083af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.869+0000 7f87ba600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f87b407c2b0 con 0x7f87b4109d60 2024-08-31T06:20:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.869+0000 7f87ba600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f87b407c130 con 0x7f87b410fff0 2024-08-31T06:20:58.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.869+0000 7f87ba600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f87b407c430 con 0x7f87b410f6a0 2024-08-31T06:20:58.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.869+0000 7f87b3e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87b410f6a0 0x7f87b407ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:58.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.869+0000 7f87b9200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b4109d60 0x7f87b4083af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47236/0 (socket says 172.21.15.16:47236) 2024-08-31T06:20:58.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.869+0000 7f87b9200640 1 -- 172.21.15.16:0/3844303937 learned_addr learned my addr 172.21.15.16:0/3844303937 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.869+0000 7f87b9c00640 1 --2- 172.21.15.16:0/3844303937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87b410fff0 0x7f87b407e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.870+0000 7f87b9200640 1 -- 172.21.15.16:0/3844303937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87b410f6a0 msgr2=0x7f87b407ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.870+0000 7f87b9200640 1 --2- 172.21.15.16:0/3844303937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87b410f6a0 0x7f87b407ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.870+0000 7f87b9200640 1 -- 172.21.15.16:0/3844303937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87b410fff0 msgr2=0x7f87b407e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.870+0000 7f87b9200640 1 --2- 172.21.15.16:0/3844303937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87b410fff0 0x7f87b407e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.870+0000 7f87b9200640 1 -- 172.21.15.16:0/3844303937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87b4072bb0 con 0x7f87b4109d60 2024-08-31T06:20:58.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.872+0000 7f87b9200640 1 --2- 172.21.15.16:0/3844303937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b4109d60 0x7f87b4083af0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f87a000b450 tx=0x7f87a000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:58.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.873+0000 7f87b1600640 1 -- 172.21.15.16:0/3844303937 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a0015020 con 0x7f87b4109d60 2024-08-31T06:20:58.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.873+0000 7f87b1600640 1 -- 172.21.15.16:0/3844303937 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87a0011d40 con 0x7f87b4109d60 2024-08-31T06:20:58.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.873+0000 7f87b1600640 1 -- 172.21.15.16:0/3844303937 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a000f340 con 0x7f87b4109d60 2024-08-31T06:20:58.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.873+0000 7f87ba600640 1 -- 172.21.15.16:0/3844303937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87b4072e40 con 0x7f87b4109d60 2024-08-31T06:20:58.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.873+0000 7f87ba600640 1 -- 172.21.15.16:0/3844303937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87b4073380 con 0x7f87b4109d60 2024-08-31T06:20:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.880+0000 7f87c29ce740 1 Processor -- start 2024-08-31T06:20:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.880+0000 7f87c29ce740 1 -- start start 2024-08-31T06:20:58.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.880+0000 7f87c29ce740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e943318070 0x55e9436bdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:58.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.881+0000 7f87c29ce740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9432ef1b0 0x55e9436be0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:58.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.881+0000 7f87c29ce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9432f7390 0x55e9436be5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:58.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.881+0000 7f87c29ce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e9436347b0 con 0x55e9432f7390 2024-08-31T06:20:58.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.881+0000 7f87c29ce740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e943623d40 con 0x55e9432ef1b0 2024-08-31T06:20:58.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.881+0000 7f87c29ce740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e943627cf0 con 0x55e943318070 2024-08-31T06:20:58.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.882+0000 7f87b9200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9432ef1b0 0x55e9436be0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:58.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.882+0000 7f87b9c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e943318070 0x55e9436bdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:58.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.882+0000 7f87b9200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9432ef1b0 0x55e9436be0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:48004/0 (socket says 172.21.15.16:48004) 2024-08-31T06:20:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.882+0000 7f87b9200640 1 -- 172.21.15.16:0/3377994124 learned_addr learned my addr 172.21.15.16:0/3377994124 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.882+0000 7f87b9c00640 1 -- 172.21.15.16:0/3377994124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9432ef1b0 msgr2=0x55e9436be0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.882+0000 7f87b9c00640 1 --2- 172.21.15.16:0/3377994124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9432ef1b0 0x55e9436be0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.882+0000 7f87b9c00640 1 -- 172.21.15.16:0/3377994124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9432f7390 msgr2=0x55e9436be5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.882+0000 7f87b9c00640 1 --2- 172.21.15.16:0/3377994124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9432f7390 0x55e9436be5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.882+0000 7f87b9c00640 1 -- 172.21.15.16:0/3377994124 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e943383650 con 0x55e943318070 2024-08-31T06:20:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.883+0000 7f87b9c00640 1 --2- 172.21.15.16:0/3377994124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e943318070 0x55e9436bdb60 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f87ac009f30 tx=0x7f87ac010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e6d60e3643425c7 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:58.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.886+0000 7f87a6000640 1 -- 172.21.15.16:0/3377994124 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ac016070 con 0x55e943318070 2024-08-31T06:20:58.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.886+0000 7f87a6000640 1 -- 172.21.15.16:0/3377994124 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87ac004d20 con 0x55e943318070 2024-08-31T06:20:58.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.886+0000 7f87a6000640 1 -- 172.21.15.16:0/3377994124 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ac012780 con 0x55e943318070 2024-08-31T06:20:58.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.889+0000 7f87c29ce740 1 -- 172.21.15.16:0/3377994124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e943318070 msgr2=0x55e9436bdb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:58.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.889+0000 7f87c29ce740 1 --2- 172.21.15.16:0/3377994124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e943318070 0x55e9436bdb60 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f87ac009f30 tx=0x7f87ac010800 comp rx=0 tx=0).stop 2024-08-31T06:20:58.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.889+0000 7f87c29ce740 1 -- 172.21.15.16:0/3377994124 shutdown_connections 2024-08-31T06:20:58.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.889+0000 7f87c29ce740 1 --2- 172.21.15.16:0/3377994124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9432f7390 0x55e9436be5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:58.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.889+0000 7f87c29ce740 1 --2- 172.21.15.16:0/3377994124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9432ef1b0 0x55e9436be0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:58.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.889+0000 7f87c29ce740 1 --2- 172.21.15.16:0/3377994124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e943318070 0x55e9436bdb60 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:58.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.889+0000 7f87c29ce740 1 -- 172.21.15.16:0/3377994124 >> 172.21.15.16:0/3377994124 conn(0x55e94330c460 msgr2=0x55e9436bba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:58.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.893+0000 7f87c29ce740 1 -- 172.21.15.16:0/3377994124 shutdown_connections 2024-08-31T06:20:58.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.893+0000 7f87c29ce740 1 -- 172.21.15.16:0/3377994124 wait complete. 2024-08-31T06:20:58.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.893+0000 7f87b1600640 1 -- 172.21.15.16:0/3844303937 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f87a0018020 con 0x7f87b4109d60 2024-08-31T06:20:58.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.894+0000 7f87c29ce740 1 -- start start 2024-08-31T06:20:58.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.895+0000 7f87c29ce740 1 Processor -- start 2024-08-31T06:20:58.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.896+0000 7f87c29ce740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9432f7390 0x55e9436c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.896+0000 7f87c29ce740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e943318070 0x55e9436c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.897+0000 7f87b9c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9432f7390 0x55e9436c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.897+0000 7f87c29ce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9432d69f0 0x55e9436d5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.897+0000 7f87b9c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9432f7390 0x55e9436c9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:39194/0 (socket says 172.21.15.16:39194) 2024-08-31T06:20:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.897+0000 7f87b9200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e943318070 0x55e9436c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.897+0000 7f87b9200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e943318070 0x55e9436c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:48016/0 (socket says 172.21.15.16:48016) 2024-08-31T06:20:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.897+0000 7f87b9c00640 1 -- 172.21.15.16:0/3106360949 learned_addr learned my addr 172.21.15.16:0/3106360949 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:58.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.897+0000 7f87c29ce740 1 -- 172.21.15.16:0/3106360949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e943486700 con 0x55e9432d69f0 2024-08-31T06:20:58.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.897+0000 7f87c29ce740 1 -- 172.21.15.16:0/3106360949 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e9435aacf0 con 0x55e943318070 2024-08-31T06:20:58.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.897+0000 7f87c29ce740 1 -- 172.21.15.16:0/3106360949 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e9436cc340 con 0x55e9432f7390 2024-08-31T06:20:58.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.897+0000 7f87c29ce740 10 client.0 _ll_register_callbacks cb 0x55e9435d0470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:20:58.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.898+0000 7f87b9c00640 1 -- 172.21.15.16:0/3106360949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e943318070 msgr2=0x55e9436c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:58.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.898+0000 7f87b9c00640 1 --2- 172.21.15.16:0/3106360949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e943318070 0x55e9436c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:58.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.898+0000 7f87b9c00640 1 -- 172.21.15.16:0/3106360949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9432d69f0 msgr2=0x55e9436d5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:58.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.898+0000 7f87b9c00640 1 --2- 172.21.15.16:0/3106360949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9432d69f0 0x55e9436d5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:58.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.898+0000 7f87b9c00640 1 -- 172.21.15.16:0/3106360949 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9435eb8b0 con 0x55e9432f7390 2024-08-31T06:20:58.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.899+0000 7f87b9c00640 1 --2- 172.21.15.16:0/3106360949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9432f7390 0x55e9436c9770 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f87ac000f80 tx=0x7f87ac005100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af8966bc8d4ce93c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:58.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.899+0000 7f879aa00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:20:58.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.899+0000 7f879aa00640 1 -- 172.21.15.16:0/3106360949 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ac029060 con 0x55e9432f7390 2024-08-31T06:20:58.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.899+0000 7f879aa00640 1 -- 172.21.15.16:0/3106360949 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8470+0+0 (secure 0 0 0) 0x7f87ac005680 con 0x55e9432f7390 2024-08-31T06:20:58.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.901+0000 7f87c29ce740 1 -- 172.21.15.16:0/3106360949 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87ac004d20 con 0x55e9432f7390 2024-08-31T06:20:58.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.901+0000 7f879aa00640 1 -- 172.21.15.16:0/3106360949 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f87ac016070 con 0x55e9432f7390 2024-08-31T06:20:58.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.901+0000 7f87b1600640 1 --2- 172.21.15.16:0/3844303937 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f879c0b4dc0 0x7f879c0b72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:58.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.902+0000 7f87c29ce740 10 client.28681 fetch_fsmap learned FSMap version 49 2024-08-31T06:20:58.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.902+0000 7f87c29ce740 1 -- 172.21.15.16:0/3106360949 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55e9436d83a0 con 0x55e9432f7390 2024-08-31T06:20:58.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.903+0000 7f879aa00640 1 -- 172.21.15.16:0/3106360949 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87ac004d20 con 0x55e9432f7390 2024-08-31T06:20:58.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.903+0000 7f87b1600640 1 -- 172.21.15.16:0/3844303937 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8470+0+0 (secure 0 0 0) 0x7f87a00ea300 con 0x7f87b4109d60 2024-08-31T06:20:58.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.903+0000 7f87b9200640 1 --2- 172.21.15.16:0/3844303937 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f879c0b4dc0 0x7f879c0b72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:58.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.904+0000 7f87b9200640 1 --2- 172.21.15.16:0/3844303937 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f879c0b4dc0 0x7f879c0b72d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f87a0010910 tx=0x7f87a00ec040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:58.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.907+0000 7f879aa00640 1 -- 172.21.15.16:0/3106360949 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f87ac00f040 con 0x55e9432f7390 2024-08-31T06:20:58.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.907+0000 7f87c29ce740 10 client.28681 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:20:58.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.907+0000 7f87c29ce740 10 client.28681 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T06:20:58.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.907+0000 7f87c29ce740 20 client.28681 populate_metadata read hostname 'smithi016' 2024-08-31T06:20:58.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.907+0000 7f87c29ce740 1 --2- 172.21.15.16:0/3106360949 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55e9432abda0 0x55e9436dbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:58.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.907+0000 7f87c29ce740 4 client.28681 mds_command: new command op to 24712 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:20:58.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.907+0000 7f87c29ce740 1 -- 172.21.15.16:0/3106360949 --> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e943636700 con 0x55e9432abda0 2024-08-31T06:20:58.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.910+0000 7f87b3e00640 1 --2- 172.21.15.16:0/3106360949 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55e9432abda0 0x55e9436dbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:58.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.911+0000 7f879aa00640 10 client.28681 ms_handle_connect on v2:172.21.15.184:6838/4011051479 2024-08-31T06:20:58.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.911+0000 7f87b3e00640 1 --2- 172.21.15.16:0/3106360949 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55e9432abda0 0x55e9436dbad0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bc3b27ca2e78ee4e server_cookie=ccef9ab9d9734a78 in_seq=0 out_seq=0 2024-08-31T06:20:58.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.913+0000 7f879aa00640 1 -- 172.21.15.16:0/3106360949 <== mds.2 v2:172.21.15.184:6838/4011051479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e943636700 con 0x55e9432abda0 2024-08-31T06:20:58.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.913+0000 7f879aa00640 10 client.28681 handle_command_reply: tid=0 2024-08-31T06:20:58.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.913+0000 7f87c29ce740 1 client.28681 shutdown 2024-08-31T06:20:58.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.914+0000 7f87c29ce740 1 -- 172.21.15.16:0/3106360949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9432f7390 msgr2=0x55e9436c9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:58.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.914+0000 7f87c29ce740 1 --2- 172.21.15.16:0/3106360949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9432f7390 0x55e9436c9770 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f87ac000f80 tx=0x7f87ac005100 comp rx=0 tx=0).stop 2024-08-31T06:20:58.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.914+0000 7f87c29ce740 1 -- 172.21.15.16:0/3106360949 shutdown_connections 2024-08-31T06:20:58.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.914+0000 7f87c29ce740 1 --2- 172.21.15.16:0/3106360949 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55e9432abda0 0x55e9436dbad0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:58.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.914+0000 7f87c29ce740 1 --2- 172.21.15.16:0/3106360949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9432d69f0 0x55e9436d5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:58.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.914+0000 7f87c29ce740 1 --2- 172.21.15.16:0/3106360949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e943318070 0x55e9436c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:58.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.914+0000 7f879aa00640 0 client.28681 ms_handle_reset on v2:172.21.15.184:6838/4011051479 2024-08-31T06:20:58.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.914+0000 7f87c29ce740 1 --2- 172.21.15.16:0/3106360949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9432f7390 0x55e9436c9770 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:58.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.914+0000 7f87c29ce740 1 -- 172.21.15.16:0/3106360949 >> 172.21.15.16:0/3106360949 conn(0x55e94330c460 msgr2=0x55e9436c0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:58.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.914+0000 7f87c29ce740 1 -- 172.21.15.16:0/3106360949 shutdown_connections 2024-08-31T06:20:58.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.915+0000 7f87c29ce740 1 -- 172.21.15.16:0/3106360949 wait complete. 2024-08-31T06:20:58.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.915+0000 7f87c29ce740 20 client.28681 trim_cache size 0 max 16384 2024-08-31T06:20:58.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.950+0000 7f87c29ce740 1 Processor -- start 2024-08-31T06:20:58.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.950+0000 7f87c29ce740 1 -- start start 2024-08-31T06:20:58.953 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:20:58 smithi016 ceph-mon[17194]: pgmap v792: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.1 MiB/s wr, 553 op/s 2024-08-31T06:20:58.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.950+0000 7f87c29ce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9432d69f0 0x55e9436d5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:58.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.950+0000 7f87c29ce740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9432f7390 0x55e9436c4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:58.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.950+0000 7f87c29ce740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e943318070 0x55e9436c0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:58.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.950+0000 7f87c29ce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41c9390 con 0x55e9432d69f0 2024-08-31T06:20:58.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.950+0000 7f87c29ce740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41c6ba0 con 0x55e943318070 2024-08-31T06:20:58.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.950+0000 7f87c29ce740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41c4a60 con 0x55e9432f7390 2024-08-31T06:20:58.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.951+0000 7f87b3e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9432f7390 0x55e9436c4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:58.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.951+0000 7f87b3e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9432f7390 0x55e9436c4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:39200/0 (socket says 172.21.15.16:39200) 2024-08-31T06:20:58.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.951+0000 7f87b3e00640 1 -- 172.21.15.16:0/2983143302 learned_addr learned my addr 172.21.15.16:0/2983143302 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:58.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.951+0000 7f87b9c00640 1 --2- 172.21.15.16:0/2983143302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e943318070 0x55e9436c0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:58.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.952+0000 7f87b3e00640 1 -- 172.21.15.16:0/2983143302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e943318070 msgr2=0x55e9436c0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:58.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.952+0000 7f87b9c00640 1 --2- 172.21.15.16:0/2983143302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9432d69f0 0x55e9436d5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:58.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.952+0000 7f87b3e00640 1 --2- 172.21.15.16:0/2983143302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e943318070 0x55e9436c0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:58.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.952+0000 7f87b3e00640 1 -- 172.21.15.16:0/2983143302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9432d69f0 msgr2=0x55e9436d5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:58.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.952+0000 7f87b3e00640 1 --2- 172.21.15.16:0/2983143302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9432d69f0 0x55e9436d5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:58.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.952+0000 7f87b3e00640 1 -- 172.21.15.16:0/2983143302 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9436cbcb0 con 0x55e9432f7390 2024-08-31T06:20:58.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.952+0000 7f87b3e00640 1 --2- 172.21.15.16:0/2983143302 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9432f7390 0x55e9436c4c90 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f87a8004180 tx=0x7f87a80045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1198d0a8c8f31f8e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:58.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.953+0000 7f87b9c00640 1 --2- 172.21.15.16:0/2983143302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9432d69f0 0x55e9436d5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:20:58.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.953+0000 7f87a5600640 1 -- 172.21.15.16:0/2983143302 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a804a070 con 0x55e9432f7390 2024-08-31T06:20:58.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.953+0000 7f87a5600640 1 -- 172.21.15.16:0/2983143302 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87a80054f0 con 0x55e9432f7390 2024-08-31T06:20:58.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.953+0000 7f87a5600640 1 -- 172.21.15.16:0/2983143302 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a8035040 con 0x55e9432f7390 2024-08-31T06:20:58.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.954+0000 7f87c29ce740 1 -- 172.21.15.16:0/2983143302 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9432f7390 msgr2=0x55e9436c4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:58.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.954+0000 7f87c29ce740 1 --2- 172.21.15.16:0/2983143302 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9432f7390 0x55e9436c4c90 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f87a8004180 tx=0x7f87a80045e0 comp rx=0 tx=0).stop 2024-08-31T06:20:58.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.956+0000 7f87c29ce740 1 -- 172.21.15.16:0/2983143302 shutdown_connections 2024-08-31T06:20:58.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.956+0000 7f87c29ce740 1 --2- 172.21.15.16:0/2983143302 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9432f7390 0x55e9436c4c90 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:58.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.956+0000 7f87c29ce740 1 --2- 172.21.15.16:0/2983143302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e943318070 0x55e9436c0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:58.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.956+0000 7f87c29ce740 1 --2- 172.21.15.16:0/2983143302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9432d69f0 0x55e9436d5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:58.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.956+0000 7f87c29ce740 1 -- 172.21.15.16:0/2983143302 >> 172.21.15.16:0/2983143302 conn(0x55e94330c460 msgr2=0x55e9436c7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:58.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.956+0000 7f87c29ce740 1 -- 172.21.15.16:0/2983143302 shutdown_connections 2024-08-31T06:20:58.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.956+0000 7f87c29ce740 1 -- 172.21.15.16:0/2983143302 wait complete. 2024-08-31T06:20:58.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.957+0000 7f87c29ce740 1 -- start start 2024-08-31T06:20:58.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.958+0000 7f87c29ce740 1 Processor -- start 2024-08-31T06:20:58.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.958+0000 7f87c29ce740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9432d69f0 0x55e9436c7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:58.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.958+0000 7f87c29ce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9432f7390 0x55e9436d1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:58.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.959+0000 7f87c29ce740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e943318070 0x55e9436d1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:58.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.959+0000 7f87c29ce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41c5060 con 0x55e9432f7390 2024-08-31T06:20:58.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.959+0000 7f87c29ce740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41c64d0 con 0x55e943318070 2024-08-31T06:20:58.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.959+0000 7f87c29ce740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41c5730 con 0x55e9432d69f0 2024-08-31T06:20:58.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.959+0000 7f87c29ce740 10 client.0 _ll_register_callbacks cb 0x7f87ac004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:20:58.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.959+0000 7f87b9c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9432d69f0 0x55e9436c7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:58.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.959+0000 7f87b9c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9432d69f0 0x55e9436c7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:39212/0 (socket says 172.21.15.16:39212) 2024-08-31T06:20:58.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.959+0000 7f87b9c00640 1 -- 172.21.15.16:0/3328614184 learned_addr learned my addr 172.21.15.16:0/3328614184 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:20:58.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.959+0000 7f87b9c00640 1 --2- 172.21.15.16:0/3328614184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e943318070 0x55e9436d1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:58.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.960+0000 7f87b3e00640 1 --2- 172.21.15.16:0/3328614184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9432f7390 0x55e9436d1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:58.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.960+0000 7f87b9c00640 1 -- 172.21.15.16:0/3328614184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e943318070 msgr2=0x55e9436d1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:58.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.961+0000 7f87b9c00640 1 --2- 172.21.15.16:0/3328614184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e943318070 0x55e9436d1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:58.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.961+0000 7f87b9c00640 1 -- 172.21.15.16:0/3328614184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9432f7390 msgr2=0x55e9436d1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:20:58.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.961+0000 7f87b9c00640 1 --2- 172.21.15.16:0/3328614184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9432f7390 0x55e9436d1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:58.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.961+0000 7f87b9c00640 1 -- 172.21.15.16:0/3328614184 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9436d2260 con 0x55e9432d69f0 2024-08-31T06:20:58.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.961+0000 7f87b3e00640 1 --2- 172.21.15.16:0/3328614184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9432f7390 0x55e9436d1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:20:58.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.961+0000 7f87b9c00640 1 --2- 172.21.15.16:0/3328614184 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9432d69f0 0x55e9436c7020 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f87ac024940 tx=0x7f87ac009600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15ca8c94727cf86e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:20:58.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.961+0000 7f879aa00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:20:58.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.962+0000 7f879aa00640 1 -- 172.21.15.16:0/3328614184 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ac029040 con 0x55e9432d69f0 2024-08-31T06:20:58.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.962+0000 7f879aa00640 1 -- 172.21.15.16:0/3328614184 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8470+0+0 (secure 0 0 0) 0x7f87ac012450 con 0x55e9432d69f0 2024-08-31T06:20:58.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.962+0000 7f87c29ce740 1 -- 172.21.15.16:0/3328614184 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87a80054f0 con 0x55e9432d69f0 2024-08-31T06:20:58.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.962+0000 7f879aa00640 1 -- 172.21.15.16:0/3328614184 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f87ac041050 con 0x55e9432d69f0 2024-08-31T06:20:58.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.962+0000 7f87c29ce740 10 client.28687 fetch_fsmap learned FSMap version 49 2024-08-31T06:20:58.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.962+0000 7f87c29ce740 1 -- 172.21.15.16:0/3328614184 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55e9437464d0 con 0x55e9432d69f0 2024-08-31T06:20:58.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.963+0000 7f879aa00640 1 -- 172.21.15.16:0/3328614184 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87a80054f0 con 0x55e9432d69f0 2024-08-31T06:20:58.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.963+0000 7f879aa00640 1 -- 172.21.15.16:0/3328614184 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f87ac036070 con 0x55e9432d69f0 2024-08-31T06:20:58.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.964+0000 7f87c29ce740 10 client.28687 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:20:58.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.964+0000 7f87c29ce740 10 client.28687 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T06:20:58.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.964+0000 7f87c29ce740 20 client.28687 populate_metadata read hostname 'smithi016' 2024-08-31T06:20:58.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.964+0000 7f87c29ce740 1 --2- 172.21.15.16:0/3328614184 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55e943746a30 0x55e943748e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:20:58.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.964+0000 7f87c29ce740 4 client.28687 mds_command: new command op to 24712 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:20:58.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.964+0000 7f87c29ce740 1 -- 172.21.15.16:0/3328614184 --> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e9436321a0 con 0x55e943746a30 2024-08-31T06:20:58.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.965+0000 7f87b3e00640 1 --2- 172.21.15.16:0/3328614184 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55e943746a30 0x55e943748e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:20:58.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.967+0000 7f87b3e00640 1 --2- 172.21.15.16:0/3328614184 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55e943746a30 0x55e943748e50 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2c94a525f37ae003 server_cookie=8c75473f9d5e7279 in_seq=0 out_seq=0 2024-08-31T06:20:58.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.967+0000 7f879aa00640 10 client.28687 ms_handle_connect on v2:172.21.15.184:6838/4011051479 2024-08-31T06:20:58.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.967+0000 7f879aa00640 1 -- 172.21.15.16:0/3328614184 <== mds.2 v2:172.21.15.184:6838/4011051479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e9436321a0 con 0x55e943746a30 2024-08-31T06:20:58.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.967+0000 7f879aa00640 10 client.28687 handle_command_reply: tid=0 2024-08-31T06:20:58.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.967+0000 7f87c29ce740 1 client.28687 shutdown 2024-08-31T06:20:58.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.968+0000 7f87c29ce740 1 -- 172.21.15.16:0/3328614184 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9432d69f0 msgr2=0x55e9436c7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:58.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.968+0000 7f87c29ce740 1 --2- 172.21.15.16:0/3328614184 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9432d69f0 0x55e9436c7020 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f87ac024940 tx=0x7f87ac009600 comp rx=0 tx=0).stop 2024-08-31T06:20:58.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.968+0000 7f87c29ce740 1 -- 172.21.15.16:0/3328614184 shutdown_connections 2024-08-31T06:20:58.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.968+0000 7f87c29ce740 1 --2- 172.21.15.16:0/3328614184 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55e943746a30 0x55e943748e50 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:58.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.968+0000 7f87c29ce740 1 --2- 172.21.15.16:0/3328614184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e943318070 0x55e9436d1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:58.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.968+0000 7f879aa00640 0 client.28687 ms_handle_reset on v2:172.21.15.184:6838/4011051479 2024-08-31T06:20:58.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.968+0000 7f87c29ce740 1 --2- 172.21.15.16:0/3328614184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9432f7390 0x55e9436d1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:58.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.968+0000 7f87c29ce740 1 --2- 172.21.15.16:0/3328614184 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9432d69f0 0x55e9436c7020 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:58.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.969+0000 7f87c29ce740 1 -- 172.21.15.16:0/3328614184 >> 172.21.15.16:0/3328614184 conn(0x55e94330c460 msgr2=0x55e9436c0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:58.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.969+0000 7f87c29ce740 1 -- 172.21.15.16:0/3328614184 shutdown_connections 2024-08-31T06:20:58.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.969+0000 7f87c29ce740 1 -- 172.21.15.16:0/3328614184 wait complete. 2024-08-31T06:20:58.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.970+0000 7f87c29ce740 20 client.28687 trim_cache size 0 max 16384 2024-08-31T06:20:58.973 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:20:58.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.973+0000 7f87ba600640 1 -- 172.21.15.16:0/3844303937 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f879c0b4dc0 msgr2=0x7f879c0b72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:58.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.973+0000 7f87ba600640 1 --2- 172.21.15.16:0/3844303937 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f879c0b4dc0 0x7f879c0b72d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f87a0010910 tx=0x7f87a00ec040 comp rx=0 tx=0).stop 2024-08-31T06:20:58.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.973+0000 7f87ba600640 1 -- 172.21.15.16:0/3844303937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b4109d60 msgr2=0x7f87b4083af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:20:58.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.973+0000 7f87ba600640 1 --2- 172.21.15.16:0/3844303937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b4109d60 0x7f87b4083af0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f87a000b450 tx=0x7f87a000b910 comp rx=0 tx=0).stop 2024-08-31T06:20:58.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.974+0000 7f87ba600640 1 -- 172.21.15.16:0/3844303937 shutdown_connections 2024-08-31T06:20:58.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.974+0000 7f87ba600640 1 --2- 172.21.15.16:0/3844303937 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f879c0b4dc0 0x7f879c0b72d0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:58.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.974+0000 7f87ba600640 1 --2- 172.21.15.16:0/3844303937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87b410f6a0 0x7f87b407ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:58.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.974+0000 7f87ba600640 1 --2- 172.21.15.16:0/3844303937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87b410fff0 0x7f87b407e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:58.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.974+0000 7f87ba600640 1 --2- 172.21.15.16:0/3844303937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b4109d60 0x7f87b4083af0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:20:58.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.974+0000 7f87ba600640 1 -- 172.21.15.16:0/3844303937 >> 172.21.15.16:0/3844303937 conn(0x7f87b4079360 msgr2=0x7f87b410dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:20:58.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.974+0000 7f87ba600640 1 -- 172.21.15.16:0/3844303937 shutdown_connections 2024-08-31T06:20:58.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:20:58.974+0000 7f87ba600640 1 -- 172.21.15.16:0/3844303937 wait complete. 2024-08-31T06:20:59.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:20:58 smithi155 ceph-mon[21742]: pgmap v792: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.1 MiB/s wr, 553 op/s 2024-08-31T06:20:59.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:20:58 smithi184 ceph-mon[22331]: pgmap v792: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.1 MiB/s wr, 553 op/s 2024-08-31T06:20:59.944 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T06:21:00.512 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:21:00.903 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.900+0000 7fe2dc200640 1 -- 172.21.15.16:0/2513722910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2d410fd90 msgr2=0x7fe2d41122a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:00.903 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.900+0000 7fe2dc200640 1 --2- 172.21.15.16:0/2513722910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2d410fd90 0x7fe2d41122a0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc009be0 tx=0x7fe2cc02a750 comp rx=0 tx=0).stop 2024-08-31T06:21:00.903 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.901+0000 7fe2dc200640 1 -- 172.21.15.16:0/2513722910 shutdown_connections 2024-08-31T06:21:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.901+0000 7fe2dc200640 1 --2- 172.21.15.16:0/2513722910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2d410fd90 0x7fe2d41122a0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.901+0000 7fe2dc200640 1 --2- 172.21.15.16:0/2513722910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2d41021a0 0x7fe2d410f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.901+0000 7fe2dc200640 1 --2- 172.21.15.16:0/2513722910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2d4101850 0x7fe2d4101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.901+0000 7fe2dc200640 1 -- 172.21.15.16:0/2513722910 >> 172.21.15.16:0/2513722910 conn(0x7fe2d4077510 msgr2=0x7fe2d4077950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.901+0000 7fe2dc200640 1 -- 172.21.15.16:0/2513722910 shutdown_connections 2024-08-31T06:21:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.901+0000 7fe2dc200640 1 -- 172.21.15.16:0/2513722910 wait complete. 2024-08-31T06:21:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.902+0000 7fe2dc200640 1 Processor -- start 2024-08-31T06:21:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.902+0000 7fe2dc200640 1 -- start start 2024-08-31T06:21:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.903+0000 7fe2dc200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2d4101850 0x7fe2d419c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.903+0000 7fe2dc200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2d41021a0 0x7fe2d419c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.903+0000 7fe2dc200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2d410fd90 0x7fe2d41a1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.903+0000 7fe2dc200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d41159b0 con 0x7fe2d410fd90 2024-08-31T06:21:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.903+0000 7fe2dc200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d4115830 con 0x7fe2d41021a0 2024-08-31T06:21:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.903+0000 7fe2dc200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d4115b30 con 0x7fe2d4101850 2024-08-31T06:21:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.903+0000 7fe2d9800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2d4101850 0x7fe2d419c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.903+0000 7fe2d8e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2d41021a0 0x7fe2d419c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.903+0000 7fe2d9800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2d4101850 0x7fe2d419c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:39236/0 (socket says 172.21.15.16:39236) 2024-08-31T06:21:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.903+0000 7fe2d9800640 1 -- 172.21.15.16:0/1679497935 learned_addr learned my addr 172.21.15.16:0/1679497935 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.903+0000 7fe2da200640 1 --2- 172.21.15.16:0/1679497935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2d410fd90 0x7fe2d41a1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.904+0000 7fe2d9800640 1 -- 172.21.15.16:0/1679497935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2d41021a0 msgr2=0x7fe2d419c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.904+0000 7fe2d9800640 1 --2- 172.21.15.16:0/1679497935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2d41021a0 0x7fe2d419c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.904+0000 7fe2d9800640 1 -- 172.21.15.16:0/1679497935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2d410fd90 msgr2=0x7fe2d41a1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.904+0000 7fe2d9800640 1 --2- 172.21.15.16:0/1679497935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2d410fd90 0x7fe2d41a1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.904+0000 7fe2d9800640 1 -- 172.21.15.16:0/1679497935 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2d41a2390 con 0x7fe2d4101850 2024-08-31T06:21:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.904+0000 7fe2d8e00640 1 --2- 172.21.15.16:0/1679497935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2d41021a0 0x7fe2d419c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:21:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.904+0000 7fe2d9800640 1 --2- 172.21.15.16:0/1679497935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2d4101850 0x7fe2d419c450 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe2c800e9c0 tx=0x7fe2c800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.904+0000 7fe2d2000640 1 -- 172.21.15.16:0/1679497935 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c800cde0 con 0x7fe2d4101850 2024-08-31T06:21:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.904+0000 7fe2d2000640 1 -- 172.21.15.16:0/1679497935 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2c80035f0 con 0x7fe2d4101850 2024-08-31T06:21:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.904+0000 7fe2dc200640 1 -- 172.21.15.16:0/1679497935 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2d41a2680 con 0x7fe2d4101850 2024-08-31T06:21:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.904+0000 7fe2d2000640 1 -- 172.21.15.16:0/1679497935 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c8010690 con 0x7fe2d4101850 2024-08-31T06:21:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.904+0000 7fe2dc200640 1 -- 172.21.15.16:0/1679497935 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2d41a8f20 con 0x7fe2d4101850 2024-08-31T06:21:00.914 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.911+0000 7fe2d2000640 1 -- 172.21.15.16:0/1679497935 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fe2c8010830 con 0x7fe2d4101850 2024-08-31T06:21:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.913+0000 7fe2e342e740 1 Processor -- start 2024-08-31T06:21:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.913+0000 7fe2e342e740 1 -- start start 2024-08-31T06:21:00.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.913+0000 7fe2d2000640 1 --2- 172.21.15.16:0/1679497935 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe2bc0b4d20 0x7fe2bc0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:00.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.914+0000 7fe2e342e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f064f1b0 0x5570f09f2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:00.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.914+0000 7fe2e342e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570f06262f0 0x5570f09f2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:00.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.914+0000 7fe2e342e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570f062e4d0 0x5570f09f33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:00.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.914+0000 7fe2e342e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5570f0967f70 con 0x5570f064f1b0 2024-08-31T06:21:00.918 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.914+0000 7fe2e342e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5570f0957500 con 0x5570f06262f0 2024-08-31T06:21:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.914+0000 7fe2e342e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5570f095b4b0 con 0x5570f062e4d0 2024-08-31T06:21:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.914+0000 7fe2d2000640 1 -- 172.21.15.16:0/1679497935 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8470+0+0 (secure 0 0 0) 0x7fe2c80e57a0 con 0x7fe2d4101850 2024-08-31T06:21:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.914+0000 7fe2d9800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f064f1b0 0x5570f09f2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.914+0000 7fe2d8e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570f06262f0 0x5570f09f2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.914+0000 7fe2da200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570f062e4d0 0x5570f09f33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.914+0000 7fe2d9800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f064f1b0 0x5570f09f2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47306/0 (socket says 172.21.15.16:47306) 2024-08-31T06:21:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.914+0000 7fe2d9800640 1 -- 172.21.15.16:0/62905185 learned_addr learned my addr 172.21.15.16:0/62905185 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.914+0000 7fe2da200640 1 --2- 172.21.15.16:0/1679497935 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe2bc0b4d20 0x7fe2bc0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.915+0000 7fe2d9800640 1 -- 172.21.15.16:0/62905185 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570f062e4d0 msgr2=0x5570f09f33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.915+0000 7fe2d9800640 1 --2- 172.21.15.16:0/62905185 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570f062e4d0 0x5570f09f33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.915+0000 7fe2d9800640 1 -- 172.21.15.16:0/62905185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570f06262f0 msgr2=0x5570f09f2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.915+0000 7fe2d9800640 1 --2- 172.21.15.16:0/62905185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570f06262f0 0x5570f09f2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.915+0000 7fe2d9800640 1 -- 172.21.15.16:0/62905185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570f0911c10 con 0x5570f064f1b0 2024-08-31T06:21:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.915+0000 7fe2d9800640 1 --2- 172.21.15.16:0/62905185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f064f1b0 0x5570f09f2920 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe2c80a8e30 tx=0x7fe2c8050dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d978542d70600b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.915+0000 7fe2da200640 1 --2- 172.21.15.16:0/1679497935 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe2bc0b4d20 0x7fe2bc0b7230 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc006010 tx=0x7fe2cc004a40 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.916+0000 7fe2c6a00640 1 -- 172.21.15.16:0/62905185 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c80ad4b0 con 0x5570f064f1b0 2024-08-31T06:21:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.916+0000 7fe2c6a00640 1 -- 172.21.15.16:0/62905185 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2c80acdb0 con 0x5570f064f1b0 2024-08-31T06:21:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.916+0000 7fe2c6a00640 1 -- 172.21.15.16:0/62905185 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c80e31a0 con 0x5570f064f1b0 2024-08-31T06:21:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.917+0000 7fe2e342e740 1 -- 172.21.15.16:0/62905185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f064f1b0 msgr2=0x5570f09f2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.917+0000 7fe2e342e740 1 --2- 172.21.15.16:0/62905185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f064f1b0 0x5570f09f2920 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe2c80a8e30 tx=0x7fe2c8050dc0 comp rx=0 tx=0).stop 2024-08-31T06:21:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.917+0000 7fe2e342e740 1 -- 172.21.15.16:0/62905185 shutdown_connections 2024-08-31T06:21:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.917+0000 7fe2e342e740 1 --2- 172.21.15.16:0/62905185 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570f062e4d0 0x5570f09f33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.917+0000 7fe2e342e740 1 --2- 172.21.15.16:0/62905185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570f06262f0 0x5570f09f2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.917+0000 7fe2e342e740 1 --2- 172.21.15.16:0/62905185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f064f1b0 0x5570f09f2920 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.917+0000 7fe2e342e740 1 -- 172.21.15.16:0/62905185 >> 172.21.15.16:0/62905185 conn(0x5570f0643300 msgr2=0x5570f09f0810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.918+0000 7fe2e342e740 1 -- 172.21.15.16:0/62905185 shutdown_connections 2024-08-31T06:21:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.918+0000 7fe2e342e740 1 -- 172.21.15.16:0/62905185 wait complete. 2024-08-31T06:21:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.919+0000 7fe2e342e740 1 -- start start 2024-08-31T06:21:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.919+0000 7fe2e342e740 1 Processor -- start 2024-08-31T06:21:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.920+0000 7fe2e342e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570f06262f0 0x5570f09fdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.920+0000 7fe2e342e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570f062e4d0 0x5570f09fe420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.920+0000 7fe2e342e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f064f1b0 0x5570f09fe960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.920+0000 7fe2e342e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5570f0890ab0 con 0x5570f064f1b0 2024-08-31T06:21:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.920+0000 7fe2e342e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5570f08fe320 con 0x5570f06262f0 2024-08-31T06:21:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.920+0000 7fe2e342e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5570f0a010a0 con 0x5570f062e4d0 2024-08-31T06:21:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.920+0000 7fe2e342e740 10 client.0 _ll_register_callbacks cb 0x5570f09046a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:21:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.920+0000 7fe2d8e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570f062e4d0 0x5570f09fe420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.921+0000 7fe2d8e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570f062e4d0 0x5570f09fe420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:39256/0 (socket says 172.21.15.16:39256) 2024-08-31T06:21:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.921+0000 7fe2d8e00640 1 -- 172.21.15.16:0/3868985066 learned_addr learned my addr 172.21.15.16:0/3868985066 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.921+0000 7fe2d9800640 1 --2- 172.21.15.16:0/3868985066 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570f06262f0 0x5570f09fdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.921+0000 7fe2da200640 1 --2- 172.21.15.16:0/3868985066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f064f1b0 0x5570f09fe960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.921+0000 7fe2d9800640 1 -- 172.21.15.16:0/3868985066 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570f062e4d0 msgr2=0x5570f09fe420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.921+0000 7fe2d9800640 1 --2- 172.21.15.16:0/3868985066 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570f062e4d0 0x5570f09fe420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.921+0000 7fe2d9800640 1 -- 172.21.15.16:0/3868985066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f064f1b0 msgr2=0x5570f09fe960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.921+0000 7fe2d9800640 1 --2- 172.21.15.16:0/3868985066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f064f1b0 0x5570f09fe960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.921+0000 7fe2d9800640 1 -- 172.21.15.16:0/3868985066 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570f06ba790 con 0x5570f06262f0 2024-08-31T06:21:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.921+0000 7fe2d8e00640 1 --2- 172.21.15.16:0/3868985066 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570f062e4d0 0x5570f09fe420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:21:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.921+0000 7fe2da200640 1 --2- 172.21.15.16:0/3868985066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f064f1b0 0x5570f09fe960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:21:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.921+0000 7fe2d9800640 1 --2- 172.21.15.16:0/3868985066 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570f06262f0 0x5570f09fdee0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe2c80acd40 tx=0x7fe2c80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=513a2b1a3ee05f14 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.921+0000 7fe2bb400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:21:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.922+0000 7fe2bb400640 1 -- 172.21.15.16:0/3868985066 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c80b60d0 con 0x5570f06262f0 2024-08-31T06:21:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.922+0000 7fe2bb400640 1 -- 172.21.15.16:0/3868985066 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8470+0+0 (secure 0 0 0) 0x7fe2c80a93a0 con 0x5570f06262f0 2024-08-31T06:21:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.922+0000 7fe2e342e740 1 -- 172.21.15.16:0/3868985066 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2c80acdb0 con 0x5570f06262f0 2024-08-31T06:21:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.922+0000 7fe2bb400640 1 -- 172.21.15.16:0/3868985066 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fe2c80c7050 con 0x5570f06262f0 2024-08-31T06:21:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.922+0000 7fe2e342e740 10 client.28682 fetch_fsmap learned FSMap version 49 2024-08-31T06:21:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.922+0000 7fe2e342e740 1 -- 172.21.15.16:0/3868985066 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5570f0733620 con 0x5570f06262f0 2024-08-31T06:21:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.922+0000 7fe2bb400640 1 -- 172.21.15.16:0/3868985066 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2c80acdb0 con 0x5570f06262f0 2024-08-31T06:21:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.922+0000 7fe2bb400640 1 -- 172.21.15.16:0/3868985066 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fe2c80e31a0 con 0x5570f06262f0 2024-08-31T06:21:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.923+0000 7fe2e342e740 10 client.28682 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:21:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.923+0000 7fe2e342e740 10 client.28682 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:21:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.923+0000 7fe2e342e740 20 client.28682 populate_metadata read hostname 'smithi016' 2024-08-31T06:21:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.923+0000 7fe2e342e740 1 --2- 172.21.15.16:0/3868985066 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5570f060dd80 0x5570f0a18670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.923+0000 7fe2e342e740 4 client.28682 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:21:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.923+0000 7fe2e342e740 1 -- 172.21.15.16:0/3868985066 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570f0969ec0 con 0x5570f060dd80 2024-08-31T06:21:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.923+0000 7fe2d8e00640 1 --2- 172.21.15.16:0/3868985066 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5570f060dd80 0x5570f0a18670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.924+0000 7fe2bb400640 10 client.28682 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:21:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.924+0000 7fe2d8e00640 1 --2- 172.21.15.16:0/3868985066 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5570f060dd80 0x5570f0a18670 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67c4747a86660d98 server_cookie=153b5197fde6b55e in_seq=0 out_seq=0 2024-08-31T06:21:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.926+0000 7fe2bb400640 1 -- 172.21.15.16:0/3868985066 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5570f0969ec0 con 0x5570f060dd80 2024-08-31T06:21:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.926+0000 7fe2bb400640 10 client.28682 handle_command_reply: tid=0 2024-08-31T06:21:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.926+0000 7fe2e342e740 1 client.28682 shutdown 2024-08-31T06:21:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.927+0000 7fe2e342e740 1 -- 172.21.15.16:0/3868985066 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570f06262f0 msgr2=0x5570f09fdee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.927+0000 7fe2e342e740 1 --2- 172.21.15.16:0/3868985066 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570f06262f0 0x5570f09fdee0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe2c80acd40 tx=0x7fe2c80046c0 comp rx=0 tx=0).stop 2024-08-31T06:21:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.927+0000 7fe2e342e740 1 -- 172.21.15.16:0/3868985066 shutdown_connections 2024-08-31T06:21:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.927+0000 7fe2e342e740 1 --2- 172.21.15.16:0/3868985066 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5570f060dd80 0x5570f0a18670 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.927+0000 7fe2e342e740 1 --2- 172.21.15.16:0/3868985066 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570f062e4d0 0x5570f09fe420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.927+0000 7fe2e342e740 1 --2- 172.21.15.16:0/3868985066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f064f1b0 0x5570f09fe960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.927+0000 7fe2e342e740 1 --2- 172.21.15.16:0/3868985066 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570f06262f0 0x5570f09fdee0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.927+0000 7fe2bb400640 0 client.28682 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:21:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.927+0000 7fe2e342e740 1 -- 172.21.15.16:0/3868985066 >> 172.21.15.16:0/3868985066 conn(0x5570f0643300 msgr2=0x5570f09f3950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.928+0000 7fe2e342e740 1 -- 172.21.15.16:0/3868985066 shutdown_connections 2024-08-31T06:21:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.928+0000 7fe2e342e740 1 -- 172.21.15.16:0/3868985066 wait complete. 2024-08-31T06:21:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.928+0000 7fe2e342e740 20 client.28682 trim_cache size 0 max 16384 2024-08-31T06:21:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.950+0000 7fe2e342e740 1 Processor -- start 2024-08-31T06:21:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.950+0000 7fe2e342e740 1 -- start start 2024-08-31T06:21:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.950+0000 7fe2e342e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570f062e4d0 0x5570f09ffdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.950+0000 7fe2e342e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f064f1b0 0x5570f0a12290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.950+0000 7fe2e342e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570f05e31c0 0x5570f09fb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.950+0000 7fe2e342e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d41afbc0 con 0x5570f064f1b0 2024-08-31T06:21:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.950+0000 7fe2e342e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d41ad3d0 con 0x5570f062e4d0 2024-08-31T06:21:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.950+0000 7fe2e342e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d41ab260 con 0x5570f05e31c0 2024-08-31T06:21:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.951+0000 7fe2d9800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570f062e4d0 0x5570f09ffdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.951+0000 7fe2da200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570f05e31c0 0x5570f09fb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.951+0000 7fe2d8e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f064f1b0 0x5570f0a12290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.951+0000 7fe2d9800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570f062e4d0 0x5570f09ffdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:48078/0 (socket says 172.21.15.16:48078) 2024-08-31T06:21:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.951+0000 7fe2d9800640 1 -- 172.21.15.16:0/451509820 learned_addr learned my addr 172.21.15.16:0/451509820 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.951+0000 7fe2d9800640 1 -- 172.21.15.16:0/451509820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570f05e31c0 msgr2=0x5570f09fb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.951+0000 7fe2d9800640 1 --2- 172.21.15.16:0/451509820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570f05e31c0 0x5570f09fb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.951+0000 7fe2d9800640 1 -- 172.21.15.16:0/451509820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f064f1b0 msgr2=0x5570f0a12290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.951+0000 7fe2d9800640 1 --2- 172.21.15.16:0/451509820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f064f1b0 0x5570f0a12290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.951+0000 7fe2d9800640 1 -- 172.21.15.16:0/451509820 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570f09f5730 con 0x5570f062e4d0 2024-08-31T06:21:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.951+0000 7fe2d9800640 1 --2- 172.21.15.16:0/451509820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570f062e4d0 0x5570f09ffdc0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe2c80aca60 tx=0x7fe2c80acc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=709c063ee44dc6f7 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.951+0000 7fe2c6000640 1 -- 172.21.15.16:0/451509820 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c80cb070 con 0x5570f062e4d0 2024-08-31T06:21:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.952+0000 7fe2c6000640 1 -- 172.21.15.16:0/451509820 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2c80b0070 con 0x5570f062e4d0 2024-08-31T06:21:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.952+0000 7fe2c6000640 1 -- 172.21.15.16:0/451509820 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c80b60d0 con 0x5570f062e4d0 2024-08-31T06:21:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.952+0000 7fe2e342e740 1 -- 172.21.15.16:0/451509820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570f062e4d0 msgr2=0x5570f09ffdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.952+0000 7fe2e342e740 1 --2- 172.21.15.16:0/451509820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570f062e4d0 0x5570f09ffdc0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe2c80aca60 tx=0x7fe2c80acc80 comp rx=0 tx=0).stop 2024-08-31T06:21:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.953+0000 7fe2e342e740 1 -- 172.21.15.16:0/451509820 shutdown_connections 2024-08-31T06:21:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.953+0000 7fe2e342e740 1 --2- 172.21.15.16:0/451509820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570f05e31c0 0x5570f09fb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.953+0000 7fe2e342e740 1 --2- 172.21.15.16:0/451509820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f064f1b0 0x5570f0a12290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.953+0000 7fe2e342e740 1 --2- 172.21.15.16:0/451509820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570f062e4d0 0x5570f09ffdc0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.953+0000 7fe2e342e740 1 -- 172.21.15.16:0/451509820 >> 172.21.15.16:0/451509820 conn(0x5570f0643300 msgr2=0x5570f0a11b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.953+0000 7fe2e342e740 1 -- 172.21.15.16:0/451509820 shutdown_connections 2024-08-31T06:21:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.953+0000 7fe2e342e740 1 -- 172.21.15.16:0/451509820 wait complete. 2024-08-31T06:21:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.953+0000 7fe2e342e740 1 -- start start 2024-08-31T06:21:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.953+0000 7fe2e342e740 1 Processor -- start 2024-08-31T06:21:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.954+0000 7fe2e342e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570f05e31c0 0x5570f0a76d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.954+0000 7fe2e342e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f062e4d0 0x5570f0a772c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.954+0000 7fe2e342e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570f064f1b0 0x5570f0a81e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.954+0000 7fe2e342e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d41ab8b0 con 0x5570f062e4d0 2024-08-31T06:21:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.954+0000 7fe2e342e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d41acd00 con 0x5570f05e31c0 2024-08-31T06:21:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.954+0000 7fe2e342e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d41abf60 con 0x5570f064f1b0 2024-08-31T06:21:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.954+0000 7fe2e342e740 10 client.0 _ll_register_callbacks cb 0x7fe2c80ac5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:21:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.954+0000 7fe2d9800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570f05e31c0 0x5570f0a76d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.954+0000 7fe2d9800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570f05e31c0 0x5570f0a76d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:48092/0 (socket says 172.21.15.16:48092) 2024-08-31T06:21:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.954+0000 7fe2d9800640 1 -- 172.21.15.16:0/2608921109 learned_addr learned my addr 172.21.15.16:0/2608921109 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.954+0000 7fe2d8e00640 1 --2- 172.21.15.16:0/2608921109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f062e4d0 0x5570f0a772c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.954+0000 7fe2da200640 1 --2- 172.21.15.16:0/2608921109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570f064f1b0 0x5570f0a81e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.954+0000 7fe2d9800640 1 -- 172.21.15.16:0/2608921109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570f064f1b0 msgr2=0x5570f0a81e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.954+0000 7fe2d9800640 1 --2- 172.21.15.16:0/2608921109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570f064f1b0 0x5570f0a81e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.954+0000 7fe2d9800640 1 -- 172.21.15.16:0/2608921109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f062e4d0 msgr2=0x5570f0a772c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.954+0000 7fe2d9800640 1 --2- 172.21.15.16:0/2608921109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f062e4d0 0x5570f0a772c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.954+0000 7fe2d9800640 1 -- 172.21.15.16:0/2608921109 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570f0a77ad0 con 0x5570f05e31c0 2024-08-31T06:21:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.954+0000 7fe2da200640 1 --2- 172.21.15.16:0/2608921109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570f064f1b0 0x5570f0a81e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:21:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.955+0000 7fe2d9800640 1 --2- 172.21.15.16:0/2608921109 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570f05e31c0 0x5570f0a76d80 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe2c80acd40 tx=0x7fe2c80ac770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acbe4130ca5d67e6 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.955+0000 7fe2bb400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:21:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.955+0000 7fe2bb400640 1 -- 172.21.15.16:0/2608921109 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c80cb020 con 0x5570f05e31c0 2024-08-31T06:21:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.955+0000 7fe2bb400640 1 -- 172.21.15.16:0/2608921109 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8470+0+0 (secure 0 0 0) 0x7fe2c80a93a0 con 0x5570f05e31c0 2024-08-31T06:21:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.955+0000 7fe2e342e740 1 -- 172.21.15.16:0/2608921109 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2c80b0070 con 0x5570f05e31c0 2024-08-31T06:21:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.955+0000 7fe2bb400640 1 -- 172.21.15.16:0/2608921109 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fe2c8098160 con 0x5570f05e31c0 2024-08-31T06:21:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.955+0000 7fe2e342e740 10 client.28688 fetch_fsmap learned FSMap version 49 2024-08-31T06:21:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.955+0000 7fe2e342e740 1 -- 172.21.15.16:0/2608921109 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5570f0a77ad0 con 0x5570f05e31c0 2024-08-31T06:21:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.955+0000 7fe2bb400640 1 -- 172.21.15.16:0/2608921109 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2c80b0070 con 0x5570f05e31c0 2024-08-31T06:21:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.955+0000 7fe2bb400640 1 -- 172.21.15.16:0/2608921109 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fe2c80ce020 con 0x5570f05e31c0 2024-08-31T06:21:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.955+0000 7fe2e342e740 10 client.28688 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:21:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.955+0000 7fe2e342e740 10 client.28688 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:21:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.955+0000 7fe2e342e740 20 client.28688 populate_metadata read hostname 'smithi016' 2024-08-31T06:21:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.955+0000 7fe2e342e740 1 --2- 172.21.15.16:0/2608921109 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5570f0a83750 0x5570f0a85b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.955+0000 7fe2e342e740 4 client.28688 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:21:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.955+0000 7fe2e342e740 1 -- 172.21.15.16:0/2608921109 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5570f0965960 con 0x5570f0a83750 2024-08-31T06:21:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.956+0000 7fe2d8e00640 1 --2- 172.21.15.16:0/2608921109 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5570f0a83750 0x5570f0a85b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.956+0000 7fe2d8e00640 1 --2- 172.21.15.16:0/2608921109 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5570f0a83750 0x5570f0a85b70 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50f1933e842ee531 server_cookie=ee2914e9dd979012 in_seq=0 out_seq=0 2024-08-31T06:21:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.956+0000 7fe2bb400640 10 client.28688 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:21:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.969+0000 7fe2bb400640 1 -- 172.21.15.16:0/2608921109 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5570f0965960 con 0x5570f0a83750 2024-08-31T06:21:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.969+0000 7fe2bb400640 10 client.28688 handle_command_reply: tid=0 2024-08-31T06:21:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.969+0000 7fe2e342e740 1 client.28688 shutdown 2024-08-31T06:21:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.969+0000 7fe2e342e740 1 -- 172.21.15.16:0/2608921109 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570f05e31c0 msgr2=0x5570f0a76d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.969+0000 7fe2e342e740 1 --2- 172.21.15.16:0/2608921109 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570f05e31c0 0x5570f0a76d80 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe2c80acd40 tx=0x7fe2c80ac770 comp rx=0 tx=0).stop 2024-08-31T06:21:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.969+0000 7fe2e342e740 1 -- 172.21.15.16:0/2608921109 shutdown_connections 2024-08-31T06:21:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.969+0000 7fe2e342e740 1 --2- 172.21.15.16:0/2608921109 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5570f0a83750 0x5570f0a85b70 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.969+0000 7fe2e342e740 1 --2- 172.21.15.16:0/2608921109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570f064f1b0 0x5570f0a81e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.969+0000 7fe2e342e740 1 --2- 172.21.15.16:0/2608921109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f062e4d0 0x5570f0a772c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.969+0000 7fe2bb400640 0 client.28688 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:21:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.969+0000 7fe2e342e740 1 --2- 172.21.15.16:0/2608921109 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570f05e31c0 0x5570f0a76d80 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.969+0000 7fe2e342e740 1 -- 172.21.15.16:0/2608921109 >> 172.21.15.16:0/2608921109 conn(0x5570f0643300 msgr2=0x5570f09f41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.970+0000 7fe2e342e740 1 -- 172.21.15.16:0/2608921109 shutdown_connections 2024-08-31T06:21:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.970+0000 7fe2e342e740 1 -- 172.21.15.16:0/2608921109 wait complete. 2024-08-31T06:21:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.970+0000 7fe2e342e740 20 client.28688 trim_cache size 0 max 16384 2024-08-31T06:21:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:21:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:21:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:21:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:21:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.970+0000 7fe2dc200640 1 -- 172.21.15.16:0/1679497935 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe2bc0b4d20 msgr2=0x7fe2bc0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.970+0000 7fe2dc200640 1 --2- 172.21.15.16:0/1679497935 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe2bc0b4d20 0x7fe2bc0b7230 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc006010 tx=0x7fe2cc004a40 comp rx=0 tx=0).stop 2024-08-31T06:21:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.970+0000 7fe2dc200640 1 -- 172.21.15.16:0/1679497935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2d4101850 msgr2=0x7fe2d419c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.970+0000 7fe2dc200640 1 --2- 172.21.15.16:0/1679497935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2d4101850 0x7fe2d419c450 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe2c800e9c0 tx=0x7fe2c800ee80 comp rx=0 tx=0).stop 2024-08-31T06:21:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.971+0000 7fe2dc200640 1 -- 172.21.15.16:0/1679497935 shutdown_connections 2024-08-31T06:21:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.971+0000 7fe2dc200640 1 --2- 172.21.15.16:0/1679497935 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe2bc0b4d20 0x7fe2bc0b7230 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.971+0000 7fe2dc200640 1 --2- 172.21.15.16:0/1679497935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2d410fd90 0x7fe2d41a1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.971+0000 7fe2dc200640 1 --2- 172.21.15.16:0/1679497935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2d41021a0 0x7fe2d419c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.971+0000 7fe2dc200640 1 --2- 172.21.15.16:0/1679497935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2d4101850 0x7fe2d419c450 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.971+0000 7fe2dc200640 1 -- 172.21.15.16:0/1679497935 >> 172.21.15.16:0/1679497935 conn(0x7fe2d4077510 msgr2=0x7fe2d41103d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.971+0000 7fe2dc200640 1 -- 172.21.15.16:0/1679497935 shutdown_connections 2024-08-31T06:21:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:00.971+0000 7fe2dc200640 1 -- 172.21.15.16:0/1679497935 wait complete. 2024-08-31T06:21:01.013 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:00 smithi016 ceph-mon[17194]: pgmap v793: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.2 MiB/s wr, 568 op/s 2024-08-31T06:21:01.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:00 smithi155 ceph-mon[21742]: pgmap v793: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.2 MiB/s wr, 568 op/s 2024-08-31T06:21:01.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:00 smithi184 ceph-mon[22331]: pgmap v793: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.2 MiB/s wr, 568 op/s 2024-08-31T06:21:01.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.231+0000 7f26a0e00640 1 -- 172.21.15.16:0/2768169192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f269c10dbb0 msgr2=0x7f269c1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:01.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.231+0000 7f26a0e00640 1 --2- 172.21.15.16:0/2768169192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f269c10dbb0 0x7f269c1100a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f2694009be0 tx=0x7f269402a6c0 comp rx=0 tx=0).stop 2024-08-31T06:21:01.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.232+0000 7f26a0e00640 1 -- 172.21.15.16:0/2768169192 shutdown_connections 2024-08-31T06:21:01.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.232+0000 7f26a0e00640 1 --2- 172.21.15.16:0/2768169192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f269c10dbb0 0x7f269c1100a0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:01.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.232+0000 7f26a0e00640 1 --2- 172.21.15.16:0/2768169192 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f269c1050a0 0x7f269c10d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:01.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.232+0000 7f26a0e00640 1 --2- 172.21.15.16:0/2768169192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f269c104750 0x7f269c104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:01.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.232+0000 7f26a0e00640 1 -- 172.21.15.16:0/2768169192 >> 172.21.15.16:0/2768169192 conn(0x7f269c0794d0 msgr2=0x7f269c0798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:01.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.232+0000 7f26a0e00640 1 -- 172.21.15.16:0/2768169192 shutdown_connections 2024-08-31T06:21:01.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.232+0000 7f26a0e00640 1 -- 172.21.15.16:0/2768169192 wait complete. 2024-08-31T06:21:01.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.234+0000 7f26a0e00640 1 Processor -- start 2024-08-31T06:21:01.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.234+0000 7f26a0e00640 1 -- start start 2024-08-31T06:21:01.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.234+0000 7f26a0e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f269c104750 0x7f269c198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:01.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.235+0000 7f26a0e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f269c1050a0 0x7f269c199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:01.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.235+0000 7f26a0e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f269c10dbb0 0x7f269c19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:01.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.235+0000 7f26a0e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f269c113810 con 0x7f269c104750 2024-08-31T06:21:01.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.235+0000 7f26a0e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f269c113690 con 0x7f269c10dbb0 2024-08-31T06:21:01.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.235+0000 7f26a0e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f269c113990 con 0x7f269c1050a0 2024-08-31T06:21:01.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.235+0000 7f2699400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f269c1050a0 0x7f269c199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:01.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.235+0000 7f2699e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f269c104750 0x7f269c198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:01.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.235+0000 7f2699400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f269c1050a0 0x7f269c199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:39290/0 (socket says 172.21.15.16:39290) 2024-08-31T06:21:01.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.235+0000 7f2699400640 1 -- 172.21.15.16:0/2967706396 learned_addr learned my addr 172.21.15.16:0/2967706396 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:01.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.235+0000 7f269a800640 1 --2- 172.21.15.16:0/2967706396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f269c10dbb0 0x7f269c19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:01.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.236+0000 7f2699400640 1 -- 172.21.15.16:0/2967706396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f269c10dbb0 msgr2=0x7f269c19fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:01.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.236+0000 7f2699400640 1 --2- 172.21.15.16:0/2967706396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f269c10dbb0 0x7f269c19fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:01.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.236+0000 7f2699400640 1 -- 172.21.15.16:0/2967706396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f269c104750 msgr2=0x7f269c198ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:01.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.236+0000 7f2699400640 1 --2- 172.21.15.16:0/2967706396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f269c104750 0x7f269c198ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:01.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.236+0000 7f2699400640 1 -- 172.21.15.16:0/2967706396 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f269c199af0 con 0x7f269c1050a0 2024-08-31T06:21:01.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.236+0000 7f269a800640 1 --2- 172.21.15.16:0/2967706396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f269c10dbb0 0x7f269c19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:21:01.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.236+0000 7f2699400640 1 --2- 172.21.15.16:0/2967706396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f269c1050a0 0x7f269c199220 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f268800e9c0 tx=0x7f268800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:01.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.236+0000 7f2692a00640 1 -- 172.21.15.16:0/2967706396 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268800ccc0 con 0x7f269c1050a0 2024-08-31T06:21:01.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.236+0000 7f2692a00640 1 -- 172.21.15.16:0/2967706396 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26880108e0 con 0x7f269c1050a0 2024-08-31T06:21:01.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.236+0000 7f26a0e00640 1 -- 172.21.15.16:0/2967706396 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f269c1a03a0 con 0x7f269c1050a0 2024-08-31T06:21:01.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.237+0000 7f26a0e00640 1 -- 172.21.15.16:0/2967706396 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f269c1a08b0 con 0x7f269c1050a0 2024-08-31T06:21:01.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.237+0000 7f2692a00640 1 -- 172.21.15.16:0/2967706396 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26880187e0 con 0x7f269c1050a0 2024-08-31T06:21:01.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.241+0000 7f2692a00640 1 -- 172.21.15.16:0/2967706396 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f2688010430 con 0x7f269c1050a0 2024-08-31T06:21:01.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.241+0000 7f26a7ab4740 1 Processor -- start 2024-08-31T06:21:01.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.241+0000 7f26a7ab4740 1 -- start start 2024-08-31T06:21:01.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.242+0000 7f26a7ab4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9cd43070 0x55ac9d0e8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:01.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.242+0000 7f26a7ab4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac9cd1a1b0 0x55ac9d0e8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:01.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.242+0000 7f26a7ab4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac9cd22390 0x55ac9d0e9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:01.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.242+0000 7f26a7ab4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ac9d05f7b0 con 0x55ac9cd22390 2024-08-31T06:21:01.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.242+0000 7f26a7ab4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ac9d04ed40 con 0x55ac9cd43070 2024-08-31T06:21:01.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.242+0000 7f26a7ab4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ac9d052cf0 con 0x55ac9cd1a1b0 2024-08-31T06:21:01.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.242+0000 7f2699e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac9cd1a1b0 0x55ac9d0e8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:01.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.242+0000 7f269a800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9cd43070 0x55ac9d0e8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:01.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.242+0000 7f2699e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac9cd1a1b0 0x55ac9d0e8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:39298/0 (socket says 172.21.15.16:39298) 2024-08-31T06:21:01.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.242+0000 7f2699e00640 1 -- 172.21.15.16:0/1807564579 learned_addr learned my addr 172.21.15.16:0/1807564579 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:01.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.242+0000 7f2699400640 1 --2- 172.21.15.16:0/1807564579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac9cd22390 0x55ac9d0e9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:01.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.242+0000 7f2699e00640 1 -- 172.21.15.16:0/1807564579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9cd43070 msgr2=0x55ac9d0e8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:01.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.242+0000 7f2699e00640 1 --2- 172.21.15.16:0/1807564579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9cd43070 0x55ac9d0e8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:01.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.242+0000 7f2699e00640 1 -- 172.21.15.16:0/1807564579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac9cd22390 msgr2=0x55ac9d0e9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:01.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.242+0000 7f2699e00640 1 --2- 172.21.15.16:0/1807564579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac9cd22390 0x55ac9d0e9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:01.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.242+0000 7f2699e00640 1 -- 172.21.15.16:0/1807564579 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac9cdae650 con 0x55ac9cd1a1b0 2024-08-31T06:21:01.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.243+0000 7f2699400640 1 --2- 172.21.15.16:0/1807564579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac9cd22390 0x55ac9d0e9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:21:01.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.243+0000 7f2692a00640 1 --2- 172.21.15.16:0/2967706396 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f26800b4df0 0x7f26800b7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:01.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.243+0000 7f2699e00640 1 --2- 172.21.15.16:0/1807564579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac9cd1a1b0 0x55ac9d0e8ed0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f268c00b750 tx=0x7f268c00bc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f8ddbbd767441c4 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:01.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.243+0000 7f2692a00640 1 -- 172.21.15.16:0/2967706396 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8470+0+0 (secure 0 0 0) 0x7f26880e5400 con 0x7f269c1050a0 2024-08-31T06:21:01.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.243+0000 7f269a800640 1 --2- 172.21.15.16:0/2967706396 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f26800b4df0 0x7f26800b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:01.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.243+0000 7f2687400640 1 -- 172.21.15.16:0/1807564579 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268c00be60 con 0x55ac9cd1a1b0 2024-08-31T06:21:01.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.243+0000 7f2687400640 1 -- 172.21.15.16:0/1807564579 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f268c015070 con 0x55ac9cd1a1b0 2024-08-31T06:21:01.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.243+0000 7f2687400640 1 -- 172.21.15.16:0/1807564579 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268c0106a0 con 0x55ac9cd1a1b0 2024-08-31T06:21:01.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.244+0000 7f26a7ab4740 1 -- 172.21.15.16:0/1807564579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac9cd1a1b0 msgr2=0x55ac9d0e8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:01.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.244+0000 7f269a800640 1 --2- 172.21.15.16:0/2967706396 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f26800b4df0 0x7f26800b7300 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2694004af0 tx=0x7f26940093a0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:01.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.244+0000 7f26a7ab4740 1 --2- 172.21.15.16:0/1807564579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac9cd1a1b0 0x55ac9d0e8ed0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f268c00b750 tx=0x7f268c00bc10 comp rx=0 tx=0).stop 2024-08-31T06:21:01.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.244+0000 7f26a7ab4740 1 -- 172.21.15.16:0/1807564579 shutdown_connections 2024-08-31T06:21:01.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.244+0000 7f26a7ab4740 1 --2- 172.21.15.16:0/1807564579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac9cd1a1b0 0x55ac9d0e8ed0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:01.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.244+0000 7f26a7ab4740 1 --2- 172.21.15.16:0/1807564579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac9cd22390 0x55ac9d0e9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:01.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.245+0000 7f26a7ab4740 1 --2- 172.21.15.16:0/1807564579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9cd43070 0x55ac9d0e8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:01.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.245+0000 7f26a7ab4740 1 -- 172.21.15.16:0/1807564579 >> 172.21.15.16:0/1807564579 conn(0x55ac9cd37460 msgr2=0x55ac9d0e6880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:01.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.245+0000 7f26a7ab4740 1 -- 172.21.15.16:0/1807564579 shutdown_connections 2024-08-31T06:21:01.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.245+0000 7f26a7ab4740 1 -- 172.21.15.16:0/1807564579 wait complete. 2024-08-31T06:21:01.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.245+0000 7f26a7ab4740 1 -- start start 2024-08-31T06:21:01.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.246+0000 7f26a7ab4740 1 Processor -- start 2024-08-31T06:21:01.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.246+0000 7f26a7ab4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac9cd1a1b0 0x55ac9d0f4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:01.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.246+0000 7f26a7ab4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac9cd22390 0x55ac9d0f4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:01.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.246+0000 7f26a7ab4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9cd43070 0x55ac9d1004f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:01.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.247+0000 7f26a7ab4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ac9ceb1700 con 0x55ac9cd1a1b0 2024-08-31T06:21:01.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.247+0000 7f26a7ab4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ac9cfd5cf0 con 0x55ac9cd43070 2024-08-31T06:21:01.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.247+0000 7f2699e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac9cd1a1b0 0x55ac9d0f4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:01.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.247+0000 7f26a7ab4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ac9d0f7170 con 0x55ac9cd22390 2024-08-31T06:21:01.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.247+0000 7f26a7ab4740 10 client.0 _ll_register_callbacks cb 0x55ac9cffb470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:21:01.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.247+0000 7f2699e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac9cd1a1b0 0x55ac9d0f4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47348/0 (socket says 172.21.15.16:47348) 2024-08-31T06:21:01.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.247+0000 7f2699400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac9cd22390 0x55ac9d0f4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:01.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.247+0000 7f2699e00640 1 -- 172.21.15.16:0/1127751216 learned_addr learned my addr 172.21.15.16:0/1127751216 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:01.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.247+0000 7f269a800640 1 --2- 172.21.15.16:0/1127751216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9cd43070 0x55ac9d1004f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:01.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.248+0000 7f2699400640 1 -- 172.21.15.16:0/1127751216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9cd43070 msgr2=0x55ac9d1004f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:01.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.248+0000 7f2699400640 1 --2- 172.21.15.16:0/1127751216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9cd43070 0x55ac9d1004f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:01.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.248+0000 7f2699400640 1 -- 172.21.15.16:0/1127751216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac9cd1a1b0 msgr2=0x55ac9d0f4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:01.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.248+0000 7f2699400640 1 --2- 172.21.15.16:0/1127751216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac9cd1a1b0 0x55ac9d0f4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:01.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.248+0000 7f2699400640 1 -- 172.21.15.16:0/1127751216 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac9d0168b0 con 0x55ac9cd22390 2024-08-31T06:21:01.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.248+0000 7f2699e00640 1 --2- 172.21.15.16:0/1127751216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac9cd1a1b0 0x55ac9d0f4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:21:01.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.248+0000 7f2699400640 1 --2- 172.21.15.16:0/1127751216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac9cd22390 0x55ac9d0f4ab0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f26880e2b20 tx=0x7f26880e30a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c8c0e29ebbbf513 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:01.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.248+0000 7f267be00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:21:01.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.248+0000 7f267be00640 1 -- 172.21.15.16:0/1127751216 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26880a7300 con 0x55ac9cd22390 2024-08-31T06:21:01.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.248+0000 7f267be00640 1 -- 172.21.15.16:0/1127751216 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8470+0+0 (secure 0 0 0) 0x7f26880e8dd0 con 0x55ac9cd22390 2024-08-31T06:21:01.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.248+0000 7f26a7ab4740 1 -- 172.21.15.16:0/1127751216 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f268c015070 con 0x55ac9cd22390 2024-08-31T06:21:01.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.248+0000 7f267be00640 1 -- 172.21.15.16:0/1127751216 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f2688010720 con 0x55ac9cd22390 2024-08-31T06:21:01.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.249+0000 7f26a7ab4740 10 client.28717 fetch_fsmap learned FSMap version 49 2024-08-31T06:21:01.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.249+0000 7f26a7ab4740 1 -- 172.21.15.16:0/1127751216 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55ac9d103240 con 0x55ac9cd22390 2024-08-31T06:21:01.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.249+0000 7f267be00640 1 -- 172.21.15.16:0/1127751216 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f268c015070 con 0x55ac9cd22390 2024-08-31T06:21:01.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.249+0000 7f267be00640 1 -- 172.21.15.16:0/1127751216 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f26880ceba0 con 0x55ac9cd22390 2024-08-31T06:21:01.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.249+0000 7f26a7ab4740 10 client.28717 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:21:01.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.249+0000 7f26a7ab4740 10 client.28717 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-31T06:21:01.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.249+0000 7f26a7ab4740 20 client.28717 populate_metadata read hostname 'smithi016' 2024-08-31T06:21:01.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.249+0000 7f26a7ab4740 1 --2- 172.21.15.16:0/1127751216 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55ac9cd019f0 0x55ac9d106850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:01.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.249+0000 7f26a7ab4740 4 client.28717 mds_command: new command op to 24724 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:21:01.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.249+0000 7f26a7ab4740 1 -- 172.21.15.16:0/1127751216 --> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac9d061700 con 0x55ac9cd019f0 2024-08-31T06:21:01.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.250+0000 7f2699e00640 1 --2- 172.21.15.16:0/1127751216 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55ac9cd019f0 0x55ac9d106850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:01.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.250+0000 7f2699e00640 1 --2- 172.21.15.16:0/1127751216 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55ac9cd019f0 0x55ac9d106850 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=470741e63ed30df0 server_cookie=4f77eccc56d870ba in_seq=0 out_seq=0 2024-08-31T06:21:01.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.250+0000 7f267be00640 10 client.28717 ms_handle_connect on v2:172.21.15.16:6838/219664460 2024-08-31T06:21:01.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.251+0000 7f267be00640 1 -- 172.21.15.16:0/1127751216 <== mds.1 v2:172.21.15.16:6838/219664460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ac9d061700 con 0x55ac9cd019f0 2024-08-31T06:21:01.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.251+0000 7f267be00640 10 client.28717 handle_command_reply: tid=0 2024-08-31T06:21:01.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.251+0000 7f26a7ab4740 1 client.28717 shutdown 2024-08-31T06:21:01.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.251+0000 7f26a7ab4740 1 -- 172.21.15.16:0/1127751216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac9cd22390 msgr2=0x55ac9d0f4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.251+0000 7f26a7ab4740 1 --2- 172.21.15.16:0/1127751216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac9cd22390 0x55ac9d0f4ab0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f26880e2b20 tx=0x7f26880e30a0 comp rx=0 tx=0).stop 2024-08-31T06:21:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.252+0000 7f26a7ab4740 1 -- 172.21.15.16:0/1127751216 shutdown_connections 2024-08-31T06:21:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.252+0000 7f26a7ab4740 1 --2- 172.21.15.16:0/1127751216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac9cd22390 0x55ac9d0f4ab0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.252+0000 7f26a7ab4740 1 --2- 172.21.15.16:0/1127751216 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55ac9cd019f0 0x55ac9d106850 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.252+0000 7f26a7ab4740 1 --2- 172.21.15.16:0/1127751216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9cd43070 0x55ac9d1004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.252+0000 7f26a7ab4740 1 --2- 172.21.15.16:0/1127751216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac9cd1a1b0 0x55ac9d0f4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.252+0000 7f267be00640 0 client.28717 ms_handle_reset on v2:172.21.15.16:6838/219664460 2024-08-31T06:21:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.252+0000 7f26a7ab4740 1 -- 172.21.15.16:0/1127751216 >> 172.21.15.16:0/1127751216 conn(0x55ac9cd37460 msgr2=0x55ac9d0ebb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.252+0000 7f26a7ab4740 1 -- 172.21.15.16:0/1127751216 shutdown_connections 2024-08-31T06:21:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.252+0000 7f26a7ab4740 1 -- 172.21.15.16:0/1127751216 wait complete. 2024-08-31T06:21:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.252+0000 7f26a7ab4740 20 client.28717 trim_cache size 0 max 16384 2024-08-31T06:21:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.272+0000 7f26a7ab4740 1 Processor -- start 2024-08-31T06:21:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.272+0000 7f26a7ab4740 1 -- start start 2024-08-31T06:21:01.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.272+0000 7f26a7ab4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac9cd1a1b0 0x55ac9d1003f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:01.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.272+0000 7f26a7ab4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9cd22390 0x55ac9d0f2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:01.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.273+0000 7f26a7ab4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac9cd43070 0x55ac9d0f3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:01.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.273+0000 7f26a7ab4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f267c0110b0 con 0x55ac9cd43070 2024-08-31T06:21:01.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.273+0000 7f26a7ab4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f267c00e8c0 con 0x55ac9cd22390 2024-08-31T06:21:01.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.273+0000 7f26a7ab4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f267c00c730 con 0x55ac9cd1a1b0 2024-08-31T06:21:01.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.273+0000 7f2699400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9cd22390 0x55ac9d0f2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:01.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.273+0000 7f2699e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac9cd1a1b0 0x55ac9d1003f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:01.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.273+0000 7f2699400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9cd22390 0x55ac9d0f2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:48134/0 (socket says 172.21.15.16:48134) 2024-08-31T06:21:01.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.273+0000 7f2699e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac9cd1a1b0 0x55ac9d1003f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:39324/0 (socket says 172.21.15.16:39324) 2024-08-31T06:21:01.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.273+0000 7f2699400640 1 -- 172.21.15.16:0/1374241602 learned_addr learned my addr 172.21.15.16:0/1374241602 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:01.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.273+0000 7f269a800640 1 --2- 172.21.15.16:0/1374241602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac9cd43070 0x55ac9d0f3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:01.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.273+0000 7f2699e00640 1 -- 172.21.15.16:0/1374241602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9cd22390 msgr2=0x55ac9d0f2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:01.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.273+0000 7f2699e00640 1 --2- 172.21.15.16:0/1374241602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9cd22390 0x55ac9d0f2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:01.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.273+0000 7f2699e00640 1 -- 172.21.15.16:0/1374241602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac9cd43070 msgr2=0x55ac9d0f3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:01.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.273+0000 7f2699e00640 1 --2- 172.21.15.16:0/1374241602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac9cd43070 0x55ac9d0f3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:01.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.273+0000 7f2699e00640 1 -- 172.21.15.16:0/1374241602 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac9d0f5630 con 0x55ac9cd1a1b0 2024-08-31T06:21:01.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.274+0000 7f2699e00640 1 --2- 172.21.15.16:0/1374241602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac9cd1a1b0 0x55ac9d1003f0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f268c006010 tx=0x7f268c013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6c8716447f9ab2c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:01.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.274+0000 7f2699400640 1 --2- 172.21.15.16:0/1374241602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9cd22390 0x55ac9d0f2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:21:01.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.274+0000 7f2686a00640 1 -- 172.21.15.16:0/1374241602 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268c003dd0 con 0x55ac9cd1a1b0 2024-08-31T06:21:01.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.274+0000 7f2686a00640 1 -- 172.21.15.16:0/1374241602 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f268c015070 con 0x55ac9cd1a1b0 2024-08-31T06:21:01.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.274+0000 7f2686a00640 1 -- 172.21.15.16:0/1374241602 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268c004c80 con 0x55ac9cd1a1b0 2024-08-31T06:21:01.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.275+0000 7f26a7ab4740 1 -- 172.21.15.16:0/1374241602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac9cd1a1b0 msgr2=0x55ac9d1003f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:01.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.275+0000 7f26a7ab4740 1 --2- 172.21.15.16:0/1374241602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac9cd1a1b0 0x55ac9d1003f0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f268c006010 tx=0x7f268c013040 comp rx=0 tx=0).stop 2024-08-31T06:21:01.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.275+0000 7f26a7ab4740 1 -- 172.21.15.16:0/1374241602 shutdown_connections 2024-08-31T06:21:01.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.275+0000 7f26a7ab4740 1 --2- 172.21.15.16:0/1374241602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac9cd43070 0x55ac9d0f3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:01.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.275+0000 7f26a7ab4740 1 --2- 172.21.15.16:0/1374241602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9cd22390 0x55ac9d0f2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:01.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.275+0000 7f26a7ab4740 1 --2- 172.21.15.16:0/1374241602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac9cd1a1b0 0x55ac9d1003f0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:01.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.275+0000 7f26a7ab4740 1 -- 172.21.15.16:0/1374241602 >> 172.21.15.16:0/1374241602 conn(0x55ac9cd37460 msgr2=0x55ac9d0ef520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:01.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.275+0000 7f26a7ab4740 1 -- 172.21.15.16:0/1374241602 shutdown_connections 2024-08-31T06:21:01.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.275+0000 7f26a7ab4740 1 -- 172.21.15.16:0/1374241602 wait complete. 2024-08-31T06:21:01.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.276+0000 7f26a7ab4740 1 -- start start 2024-08-31T06:21:01.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.276+0000 7f26a7ab4740 1 Processor -- start 2024-08-31T06:21:01.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.276+0000 7f26a7ab4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9cd1a1b0 0x55ac9d0f87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:01.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.276+0000 7f26a7ab4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac9cd22390 0x55ac9d0f8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:01.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.276+0000 7f26a7ab4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac9cd43070 0x55ac9d170210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:01.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.276+0000 7f26a7ab4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f267c00cd80 con 0x55ac9cd22390 2024-08-31T06:21:01.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.276+0000 7f26a7ab4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f267c00e1f0 con 0x55ac9cd1a1b0 2024-08-31T06:21:01.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.276+0000 7f26a7ab4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f267c00d450 con 0x55ac9cd43070 2024-08-31T06:21:01.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.276+0000 7f26a7ab4740 10 client.0 _ll_register_callbacks cb 0x7f26880ab940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:21:01.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.276+0000 7f2699e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9cd1a1b0 0x55ac9d0f87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:01.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.276+0000 7f2699400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac9cd22390 0x55ac9d0f8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:01.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.277+0000 7f2699e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9cd1a1b0 0x55ac9d0f87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:48146/0 (socket says 172.21.15.16:48146) 2024-08-31T06:21:01.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.277+0000 7f2699e00640 1 -- 172.21.15.16:0/1878090659 learned_addr learned my addr 172.21.15.16:0/1878090659 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:01.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.277+0000 7f269a800640 1 --2- 172.21.15.16:0/1878090659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac9cd43070 0x55ac9d170210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:01.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.277+0000 7f2699400640 1 -- 172.21.15.16:0/1878090659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac9cd43070 msgr2=0x55ac9d170210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:01.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.277+0000 7f2699400640 1 --2- 172.21.15.16:0/1878090659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac9cd43070 0x55ac9d170210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:01.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.277+0000 7f2699400640 1 -- 172.21.15.16:0/1878090659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9cd1a1b0 msgr2=0x55ac9d0f87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:01.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.277+0000 7f2699400640 1 --2- 172.21.15.16:0/1878090659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9cd1a1b0 0x55ac9d0f87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:01.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.277+0000 7f2699400640 1 -- 172.21.15.16:0/1878090659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac9d170750 con 0x55ac9cd22390 2024-08-31T06:21:01.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.277+0000 7f2699e00640 1 --2- 172.21.15.16:0/1878090659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9cd1a1b0 0x55ac9d0f87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:21:01.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.277+0000 7f2699400640 1 --2- 172.21.15.16:0/1878090659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac9cd22390 0x55ac9d0f8d20 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f26880a8de0 tx=0x7f26880b0260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bfd588ffb8e0a75 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:01.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.277+0000 7f267be00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:21:01.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.277+0000 7f267be00640 1 -- 172.21.15.16:0/1878090659 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26880a7a30 con 0x55ac9cd22390 2024-08-31T06:21:01.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.278+0000 7f267be00640 1 -- 172.21.15.16:0/1878090659 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8470+0+0 (secure 0 0 0) 0x7f26880aadd0 con 0x55ac9cd22390 2024-08-31T06:21:01.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.278+0000 7f26a7ab4740 1 -- 172.21.15.16:0/1878090659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f268c015070 con 0x55ac9cd22390 2024-08-31T06:21:01.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.278+0000 7f267be00640 1 -- 172.21.15.16:0/1878090659 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f26880b4de0 con 0x55ac9cd22390 2024-08-31T06:21:01.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.278+0000 7f26a7ab4740 10 client.18240 fetch_fsmap learned FSMap version 49 2024-08-31T06:21:01.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.278+0000 7f26a7ab4740 1 -- 172.21.15.16:0/1878090659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55ac9d171840 con 0x55ac9cd22390 2024-08-31T06:21:01.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.278+0000 7f267be00640 1 -- 172.21.15.16:0/1878090659 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f268c015070 con 0x55ac9cd22390 2024-08-31T06:21:01.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.278+0000 7f267be00640 1 -- 172.21.15.16:0/1878090659 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f2688010a90 con 0x55ac9cd22390 2024-08-31T06:21:01.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.278+0000 7f26a7ab4740 10 client.18240 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:21:01.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.278+0000 7f26a7ab4740 10 client.18240 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-31T06:21:01.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.278+0000 7f26a7ab4740 20 client.18240 populate_metadata read hostname 'smithi016' 2024-08-31T06:21:01.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.278+0000 7f26a7ab4740 1 --2- 172.21.15.16:0/1878090659 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55ac9ccd6da0 0x55ac9d173ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:01.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.278+0000 7f26a7ab4740 4 client.18240 mds_command: new command op to 24724 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:21:01.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.278+0000 7f26a7ab4740 1 -- 172.21.15.16:0/1878090659 --> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ac9d05d1a0 con 0x55ac9ccd6da0 2024-08-31T06:21:01.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.279+0000 7f2699e00640 1 --2- 172.21.15.16:0/1878090659 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55ac9ccd6da0 0x55ac9d173ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:01.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.279+0000 7f2699e00640 1 --2- 172.21.15.16:0/1878090659 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55ac9ccd6da0 0x55ac9d173ea0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ea1ff07f05504400 server_cookie=a36bf364fac3bf04 in_seq=0 out_seq=0 2024-08-31T06:21:01.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.279+0000 7f267be00640 10 client.18240 ms_handle_connect on v2:172.21.15.16:6838/219664460 2024-08-31T06:21:01.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.280+0000 7f267be00640 1 -- 172.21.15.16:0/1878090659 <== mds.1 v2:172.21.15.16:6838/219664460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ac9d05d1a0 con 0x55ac9ccd6da0 2024-08-31T06:21:01.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.280+0000 7f267be00640 10 client.18240 handle_command_reply: tid=0 2024-08-31T06:21:01.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.280+0000 7f26a7ab4740 1 client.18240 shutdown 2024-08-31T06:21:01.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.280+0000 7f26a7ab4740 1 -- 172.21.15.16:0/1878090659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac9cd22390 msgr2=0x55ac9d0f8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:01.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.280+0000 7f26a7ab4740 1 --2- 172.21.15.16:0/1878090659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac9cd22390 0x55ac9d0f8d20 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f26880a8de0 tx=0x7f26880b0260 comp rx=0 tx=0).stop 2024-08-31T06:21:01.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.280+0000 7f26a7ab4740 1 -- 172.21.15.16:0/1878090659 shutdown_connections 2024-08-31T06:21:01.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.280+0000 7f26a7ab4740 1 --2- 172.21.15.16:0/1878090659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac9cd43070 0x55ac9d170210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:01.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.280+0000 7f26a7ab4740 1 --2- 172.21.15.16:0/1878090659 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x55ac9ccd6da0 0x55ac9d173ea0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:01.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.280+0000 7f26a7ab4740 1 --2- 172.21.15.16:0/1878090659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac9cd22390 0x55ac9d0f8d20 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:01.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.280+0000 7f26a7ab4740 1 --2- 172.21.15.16:0/1878090659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9cd1a1b0 0x55ac9d0f87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:01.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.280+0000 7f267be00640 0 client.18240 ms_handle_reset on v2:172.21.15.16:6838/219664460 2024-08-31T06:21:01.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.280+0000 7f26a7ab4740 1 -- 172.21.15.16:0/1878090659 >> 172.21.15.16:0/1878090659 conn(0x55ac9cd37460 msgr2=0x55ac9d0f2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:01.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.280+0000 7f26a7ab4740 1 -- 172.21.15.16:0/1878090659 shutdown_connections 2024-08-31T06:21:01.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.280+0000 7f26a7ab4740 1 -- 172.21.15.16:0/1878090659 wait complete. 2024-08-31T06:21:01.467 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:21:01.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.280+0000 7f26a7ab4740 20 client.18240 trim_cache size 0 max 16384 2024-08-31T06:21:01.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.281+0000 7f26a0e00640 1 -- 172.21.15.16:0/2967706396 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f26800b4df0 msgr2=0x7f26800b7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:01.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.281+0000 7f26a0e00640 1 --2- 172.21.15.16:0/2967706396 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f26800b4df0 0x7f26800b7300 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2694004af0 tx=0x7f26940093a0 comp rx=0 tx=0).stop 2024-08-31T06:21:01.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.281+0000 7f26a0e00640 1 -- 172.21.15.16:0/2967706396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f269c1050a0 msgr2=0x7f269c199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:01.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.281+0000 7f26a0e00640 1 --2- 172.21.15.16:0/2967706396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f269c1050a0 0x7f269c199220 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f268800e9c0 tx=0x7f268800ee80 comp rx=0 tx=0).stop 2024-08-31T06:21:01.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.281+0000 7f26a0e00640 1 -- 172.21.15.16:0/2967706396 shutdown_connections 2024-08-31T06:21:01.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.281+0000 7f26a0e00640 1 --2- 172.21.15.16:0/2967706396 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f26800b4df0 0x7f26800b7300 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:01.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.281+0000 7f26a0e00640 1 --2- 172.21.15.16:0/2967706396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f269c1050a0 0x7f269c199220 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:01.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.281+0000 7f26a0e00640 1 --2- 172.21.15.16:0/2967706396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f269c10dbb0 0x7f269c19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:01.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.281+0000 7f26a0e00640 1 --2- 172.21.15.16:0/2967706396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f269c104750 0x7f269c198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:01.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.281+0000 7f26a0e00640 1 -- 172.21.15.16:0/2967706396 >> 172.21.15.16:0/2967706396 conn(0x7f269c0794d0 msgr2=0x7f269c101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:01.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.281+0000 7f26a0e00640 1 -- 172.21.15.16:0/2967706396 shutdown_connections 2024-08-31T06:21:01.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:01.281+0000 7f26a0e00640 1 -- 172.21.15.16:0/2967706396 wait complete. 2024-08-31T06:21:01.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 815.0 s, 1132.8 tps, lat 86.961 ms stddev 105.611 2024-08-31T06:21:01.648 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 35.0 2024-08-31T06:21:01.648 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:21:02.431 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T06:21:02.718 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:21:03.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:02 smithi155 ceph-mon[21742]: pgmap v794: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.7 MiB/s wr, 626 op/s 2024-08-31T06:21:03.148 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:02 smithi016 ceph-mon[17194]: pgmap v794: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.7 MiB/s wr, 626 op/s 2024-08-31T06:21:03.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:02 smithi184 ceph-mon[22331]: pgmap v794: 97 pgs: 97 active+clean; 4.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.7 MiB/s wr, 626 op/s 2024-08-31T06:21:03.608 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:21:03.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:03.661+0000 7f449f200640 1 -- 172.21.15.16:0/4039871932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f449806e8b0 msgr2=0x7f449806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:03.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:03.661+0000 7f449f200640 1 --2- 172.21.15.16:0/4039871932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f449806e8b0 0x7f449806cce0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f4484009900 tx=0x7f448402a410 comp rx=0 tx=0).stop 2024-08-31T06:21:03.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:03.661+0000 7f449f200640 1 -- 172.21.15.16:0/4039871932 shutdown_connections 2024-08-31T06:21:03.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:03.661+0000 7f449f200640 1 --2- 172.21.15.16:0/4039871932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f449811c9e0 0x7f449811eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:03.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:03.661+0000 7f449f200640 1 --2- 172.21.15.16:0/4039871932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f449806d300 0x7f449806d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:03.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:03.661+0000 7f449f200640 1 --2- 172.21.15.16:0/4039871932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f449806e8b0 0x7f449806cce0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:03.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:03.661+0000 7f449f200640 1 -- 172.21.15.16:0/4039871932 >> 172.21.15.16:0/4039871932 conn(0x7f4498071820 msgr2=0x7f449810a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:03.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:03.662+0000 7f449f200640 1 -- 172.21.15.16:0/4039871932 shutdown_connections 2024-08-31T06:21:03.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:03.662+0000 7f449f200640 1 -- 172.21.15.16:0/4039871932 wait complete. 2024-08-31T06:21:03.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:03.664+0000 7f449f200640 1 Processor -- start 2024-08-31T06:21:03.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:03.666+0000 7f449f200640 1 -- start start 2024-08-31T06:21:03.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:03.668+0000 7f449f200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f449806d300 0x7f44981a9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:03.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:03.668+0000 7f449f200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f449806e8b0 0x7f44981a9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:03.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:03.669+0000 7f4497e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f449806d300 0x7f44981a9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:03.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:03.669+0000 7f4497400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f449806e8b0 0x7f44981a9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:03.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:03.669+0000 7f4497400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f449806e8b0 0x7f44981a9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:39354/0 (socket says 172.21.15.16:39354) 2024-08-31T06:21:03.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:03.669+0000 7f4497e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f449806d300 0x7f44981a9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47382/0 (socket says 172.21.15.16:47382) 2024-08-31T06:21:03.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:03.670+0000 7f449f200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f449811c9e0 0x7f44981ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:03.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:03.670+0000 7f4497400640 1 -- 172.21.15.16:0/1060583811 learned_addr learned my addr 172.21.15.16:0/1060583811 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:03.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:03.670+0000 7f449f200640 1 -- 172.21.15.16:0/1060583811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f44981226f0 con 0x7f449806d300 2024-08-31T06:21:03.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:03.670+0000 7f449f200640 1 -- 172.21.15.16:0/1060583811 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4498122570 con 0x7f449811c9e0 2024-08-31T06:21:03.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:03.670+0000 7f449d200640 1 --2- 172.21.15.16:0/1060583811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f449811c9e0 0x7f44981ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:03.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:03.671+0000 7f449f200640 1 -- 172.21.15.16:0/1060583811 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4498122870 con 0x7f449806e8b0 2024-08-31T06:21:03.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:03.672+0000 7f449d200640 1 -- 172.21.15.16:0/1060583811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f449806e8b0 msgr2=0x7f44981a9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:03.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:03.672+0000 7f449d200640 1 --2- 172.21.15.16:0/1060583811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f449806e8b0 0x7f44981a9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:03.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:03.672+0000 7f449d200640 1 -- 172.21.15.16:0/1060583811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f449806d300 msgr2=0x7f44981a9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:03.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:03.672+0000 7f449d200640 1 --2- 172.21.15.16:0/1060583811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f449806d300 0x7f44981a9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:03.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:03.672+0000 7f449d200640 1 -- 172.21.15.16:0/1060583811 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44981aeff0 con 0x7f449811c9e0 2024-08-31T06:21:03.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:03.672+0000 7f449d200640 1 --2- 172.21.15.16:0/1060583811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f449811c9e0 0x7f44981ae8c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f448800e970 tx=0x7f448800ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:03.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:03.673+0000 7f4494c00640 1 -- 172.21.15.16:0/1060583811 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448800ccc0 con 0x7f449811c9e0 2024-08-31T06:21:03.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:03.673+0000 7f4494c00640 1 -- 172.21.15.16:0/1060583811 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4488005020 con 0x7f449811c9e0 2024-08-31T06:21:03.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:03.673+0000 7f4494c00640 1 -- 172.21.15.16:0/1060583811 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4488004a00 con 0x7f449811c9e0 2024-08-31T06:21:03.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:03.673+0000 7f449f200640 1 -- 172.21.15.16:0/1060583811 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44981af2e0 con 0x7f449811c9e0 2024-08-31T06:21:03.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:03.673+0000 7f449f200640 1 -- 172.21.15.16:0/1060583811 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44981b5a40 con 0x7f449811c9e0 2024-08-31T06:21:03.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:03.674+0000 7f449f200640 1 -- 172.21.15.16:0/1060583811 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44780052c0 con 0x7f449811c9e0 2024-08-31T06:21:03.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:03.681+0000 7f4494c00640 1 -- 172.21.15.16:0/1060583811 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f4488004ba0 con 0x7f449811c9e0 2024-08-31T06:21:03.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:03.683+0000 7f4494c00640 1 --2- 172.21.15.16:0/1060583811 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f447c0b4e70 0x7f447c0b7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:03.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:03.684+0000 7f4494c00640 1 -- 172.21.15.16:0/1060583811 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8470+0+0 (secure 0 0 0) 0x7f44880e58c0 con 0x7f449811c9e0 2024-08-31T06:21:03.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:03.685+0000 7f4497e00640 1 --2- 172.21.15.16:0/1060583811 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f447c0b4e70 0x7f447c0b7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:03.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:03.686+0000 7f4497e00640 1 --2- 172.21.15.16:0/1060583811 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f447c0b4e70 0x7f447c0b7380 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4484005ab0 tx=0x7f4484005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:03.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:03.692+0000 7f4494c00640 1 -- 172.21.15.16:0/1060583811 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f448801c020 con 0x7f449811c9e0 2024-08-31T06:21:04.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.144+0000 7f449f200640 1 -- 172.21.15.16:0/1060583811 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f44780055b0 con 0x7f449811c9e0 2024-08-31T06:21:04.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.146+0000 7f4494c00640 1 -- 172.21.15.16:0/1060583811 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+11923 (secure 0 0 0) 0x7f44880ac700 con 0x7f449811c9e0 2024-08-31T06:21:04.149 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:21:04.149 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":49,"btime":"2024-08-31T06:19:17:811356+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":49,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:19:17.033815+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24737,24724,24712]},"id":1}]} 2024-08-31T06:21:04.150 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 49 2024-08-31T06:21:04.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.150+0000 7f449f200640 1 -- 172.21.15.16:0/1060583811 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f447c0b4e70 msgr2=0x7f447c0b7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:04.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.150+0000 7f449f200640 1 --2- 172.21.15.16:0/1060583811 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f447c0b4e70 0x7f447c0b7380 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4484005ab0 tx=0x7f4484005a20 comp rx=0 tx=0).stop 2024-08-31T06:21:04.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.150+0000 7f449f200640 1 -- 172.21.15.16:0/1060583811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f449811c9e0 msgr2=0x7f44981ae8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:04.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.150+0000 7f449f200640 1 --2- 172.21.15.16:0/1060583811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f449811c9e0 0x7f44981ae8c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f448800e970 tx=0x7f448800ee30 comp rx=0 tx=0).stop 2024-08-31T06:21:04.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.150+0000 7f449f200640 1 -- 172.21.15.16:0/1060583811 shutdown_connections 2024-08-31T06:21:04.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.150+0000 7f449f200640 1 --2- 172.21.15.16:0/1060583811 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f447c0b4e70 0x7f447c0b7380 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:04.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.150+0000 7f449f200640 1 --2- 172.21.15.16:0/1060583811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f449806e8b0 0x7f44981a9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:04.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.150+0000 7f449f200640 1 --2- 172.21.15.16:0/1060583811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f449811c9e0 0x7f44981ae8c0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:04.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.150+0000 7f449f200640 1 --2- 172.21.15.16:0/1060583811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f449806d300 0x7f44981a9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:04.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.150+0000 7f449f200640 1 -- 172.21.15.16:0/1060583811 >> 172.21.15.16:0/1060583811 conn(0x7f4498071820 msgr2=0x7f44981188e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:04.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.150+0000 7f449f200640 1 -- 172.21.15.16:0/1060583811 shutdown_connections 2024-08-31T06:21:04.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.150+0000 7f449f200640 1 -- 172.21.15.16:0/1060583811 wait complete. 2024-08-31T06:21:04.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.505+0000 7fdb95800640 1 -- 172.21.15.16:0/2346828950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb9010ffd0 msgr2=0x7fdb901103e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:04.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.505+0000 7fdb95800640 1 --2- 172.21.15.16:0/2346828950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb9010ffd0 0x7fdb901103e0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fdb88009be0 tx=0x7fdb8802a6c0 comp rx=0 tx=0).stop 2024-08-31T06:21:04.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.505+0000 7fdb95800640 1 -- 172.21.15.16:0/2346828950 shutdown_connections 2024-08-31T06:21:04.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.505+0000 7fdb95800640 1 --2- 172.21.15.16:0/2346828950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb9010ffd0 0x7fdb901103e0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:04.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.505+0000 7fdb95800640 1 --2- 172.21.15.16:0/2346828950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb9010aab0 0x7fdb9010fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:04.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.505+0000 7fdb95800640 1 --2- 172.21.15.16:0/2346828950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb9010a160 0x7fdb9010a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:04.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.505+0000 7fdb95800640 1 -- 172.21.15.16:0/2346828950 >> 172.21.15.16:0/2346828950 conn(0x7fdb9006ebf0 msgr2=0x7fdb9006f030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:04.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.506+0000 7fdb95800640 1 -- 172.21.15.16:0/2346828950 shutdown_connections 2024-08-31T06:21:04.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.506+0000 7fdb95800640 1 -- 172.21.15.16:0/2346828950 wait complete. 2024-08-31T06:21:04.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.508+0000 7fdb95800640 1 Processor -- start 2024-08-31T06:21:04.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.509+0000 7fdb95800640 1 -- start start 2024-08-31T06:21:04.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.509+0000 7fdb95800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb9010a160 0x7fdb901a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:04.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.509+0000 7fdb95800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb9010aab0 0x7fdb901a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:04.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.510+0000 7fdb95800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb9010ffd0 0x7fdb901a5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:04.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.510+0000 7fdb95800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdb90113c10 con 0x7fdb9010a160 2024-08-31T06:21:04.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.510+0000 7fdb95800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdb90113a90 con 0x7fdb9010aab0 2024-08-31T06:21:04.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.510+0000 7fdb95800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdb901a6550 con 0x7fdb9010ffd0 2024-08-31T06:21:04.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.511+0000 7fdb94e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb9010ffd0 0x7fdb901a5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:04.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.511+0000 7fdb94e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb9010ffd0 0x7fdb901a5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:39362/0 (socket says 172.21.15.16:39362) 2024-08-31T06:21:04.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.511+0000 7fdb94e00640 1 -- 172.21.15.16:0/3367525387 learned_addr learned my addr 172.21.15.16:0/3367525387 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:04.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.512+0000 7fdb8fe00640 1 --2- 172.21.15.16:0/3367525387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb9010a160 0x7fdb901a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:04.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.512+0000 7fdb8f400640 1 --2- 172.21.15.16:0/3367525387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb9010aab0 0x7fdb901a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:04.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.513+0000 7fdb94e00640 1 -- 172.21.15.16:0/3367525387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb9010aab0 msgr2=0x7fdb901a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:04.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.513+0000 7fdb94e00640 1 --2- 172.21.15.16:0/3367525387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb9010aab0 0x7fdb901a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:04.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.513+0000 7fdb94e00640 1 -- 172.21.15.16:0/3367525387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb9010a160 msgr2=0x7fdb901a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:04.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.514+0000 7fdb94e00640 1 --2- 172.21.15.16:0/3367525387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb9010a160 0x7fdb901a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:04.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.514+0000 7fdb94e00640 1 -- 172.21.15.16:0/3367525387 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb901a6750 con 0x7fdb9010ffd0 2024-08-31T06:21:04.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.514+0000 7fdb8fe00640 1 --2- 172.21.15.16:0/3367525387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb9010a160 0x7fdb901a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:21:04.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.514+0000 7fdb94e00640 1 --2- 172.21.15.16:0/3367525387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb9010ffd0 0x7fdb901a5ea0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fdb88009b30 tx=0x7fdb88003cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:04.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.515+0000 7fdb8cc00640 1 -- 172.21.15.16:0/3367525387 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb88040020 con 0x7fdb9010ffd0 2024-08-31T06:21:04.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.515+0000 7fdb8cc00640 1 -- 172.21.15.16:0/3367525387 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb88030970 con 0x7fdb9010ffd0 2024-08-31T06:21:04.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.515+0000 7fdb8cc00640 1 -- 172.21.15.16:0/3367525387 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb88036040 con 0x7fdb9010ffd0 2024-08-31T06:21:04.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.516+0000 7fdb95800640 1 -- 172.21.15.16:0/3367525387 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb901a69e0 con 0x7fdb9010ffd0 2024-08-31T06:21:04.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.516+0000 7fdb95800640 1 -- 172.21.15.16:0/3367525387 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb901ad020 con 0x7fdb9010ffd0 2024-08-31T06:21:04.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.521+0000 7fdb9d722740 1 Processor -- start 2024-08-31T06:21:04.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.525+0000 7fdb8cc00640 1 -- 172.21.15.16:0/3367525387 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fdb88043020 con 0x7fdb9010ffd0 2024-08-31T06:21:04.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.528+0000 7fdb8cc00640 1 --2- 172.21.15.16:0/3367525387 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdb740b4da0 0x7fdb740b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:04.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.528+0000 7fdb8cc00640 1 -- 172.21.15.16:0/3367525387 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8514+0+0 (secure 0 0 0) 0x7fdb88105c60 con 0x7fdb9010ffd0 2024-08-31T06:21:04.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.529+0000 7fdb9d722740 1 -- start start 2024-08-31T06:21:04.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.529+0000 7fdb9d722740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613e220f070 0x5613e25b4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:04.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.529+0000 7fdb9d722740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613e21e61b0 0x5613e25b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:04.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.529+0000 7fdb9d722740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613e21ee390 0x5613e25b55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:04.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.529+0000 7fdb9d722740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5613e252b7b0 con 0x5613e21e61b0 2024-08-31T06:21:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.529+0000 7fdb9d722740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5613e251ad40 con 0x5613e21ee390 2024-08-31T06:21:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.529+0000 7fdb9d722740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5613e251ecf0 con 0x5613e220f070 2024-08-31T06:21:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.530+0000 7fdb8fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613e220f070 0x5613e25b4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.530+0000 7fdb8fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613e220f070 0x5613e25b4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:39378/0 (socket says 172.21.15.16:39378) 2024-08-31T06:21:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.530+0000 7fdb8fe00640 1 -- 172.21.15.16:0/4173208467 learned_addr learned my addr 172.21.15.16:0/4173208467 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.530+0000 7fdb94e00640 1 --2- 172.21.15.16:0/4173208467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613e21ee390 0x5613e25b55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.530+0000 7fdb8f400640 1 --2- 172.21.15.16:0/4173208467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613e21e61b0 0x5613e25b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.530+0000 7fdb8fe00640 1 -- 172.21.15.16:0/4173208467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613e21ee390 msgr2=0x5613e25b55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.530+0000 7fdb8fe00640 1 --2- 172.21.15.16:0/4173208467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613e21ee390 0x5613e25b55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.530+0000 7fdb8fe00640 1 -- 172.21.15.16:0/4173208467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613e21e61b0 msgr2=0x5613e25b50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.530+0000 7fdb8fe00640 1 --2- 172.21.15.16:0/4173208467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613e21e61b0 0x5613e25b50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.530+0000 7fdb8fe00640 1 -- 172.21.15.16:0/4173208467 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613e227a650 con 0x5613e220f070 2024-08-31T06:21:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.531+0000 7fdb8f400640 1 --2- 172.21.15.16:0/3367525387 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdb740b4da0 0x7fdb740b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:04.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.531+0000 7fdb8fe00640 1 --2- 172.21.15.16:0/4173208467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613e220f070 0x5613e25b4b60 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fdb80015870 tx=0x7fdb8000fa50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a196e3796c075eea server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:04.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.533+0000 7fdb85600640 1 -- 172.21.15.16:0/4173208467 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb80022020 con 0x5613e220f070 2024-08-31T06:21:04.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.533+0000 7fdb85600640 1 -- 172.21.15.16:0/4173208467 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb80018bb0 con 0x5613e220f070 2024-08-31T06:21:04.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.533+0000 7fdb8f400640 1 --2- 172.21.15.16:0/3367525387 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdb740b4da0 0x7fdb740b72b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x5613e25b37e0 tx=0x7fdb7c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.534+0000 7fdb9d722740 1 -- 172.21.15.16:0/4173208467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613e220f070 msgr2=0x5613e25b4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.534+0000 7fdb9d722740 1 --2- 172.21.15.16:0/4173208467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613e220f070 0x5613e25b4b60 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fdb80015870 tx=0x7fdb8000fa50 comp rx=0 tx=0).stop 2024-08-31T06:21:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.535+0000 7fdb9d722740 1 -- 172.21.15.16:0/4173208467 shutdown_connections 2024-08-31T06:21:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.535+0000 7fdb9d722740 1 --2- 172.21.15.16:0/4173208467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613e21ee390 0x5613e25b55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.535+0000 7fdb9d722740 1 --2- 172.21.15.16:0/4173208467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613e21e61b0 0x5613e25b50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.535+0000 7fdb9d722740 1 --2- 172.21.15.16:0/4173208467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613e220f070 0x5613e25b4b60 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.535+0000 7fdb9d722740 1 -- 172.21.15.16:0/4173208467 >> 172.21.15.16:0/4173208467 conn(0x5613e2203460 msgr2=0x5613e25b2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:04.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.536+0000 7fdb9d722740 1 -- 172.21.15.16:0/4173208467 shutdown_connections 2024-08-31T06:21:04.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.536+0000 7fdb9d722740 1 -- 172.21.15.16:0/4173208467 wait complete. 2024-08-31T06:21:04.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.537+0000 7fdb9d722740 1 -- start start 2024-08-31T06:21:04.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.539+0000 7fdb9d722740 1 Processor -- start 2024-08-31T06:21:04.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.539+0000 7fdb9d722740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613e21e61b0 0x5613e25c00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:04.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.540+0000 7fdb9d722740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613e21ee390 0x5613e25c0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:04.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.540+0000 7fdb9d722740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613e21cd9f0 0x5613e25c0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:04.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.540+0000 7fdb9d722740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5613e237d700 con 0x5613e21e61b0 2024-08-31T06:21:04.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.540+0000 7fdb9d722740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5613e24a1cf0 con 0x5613e21cd9f0 2024-08-31T06:21:04.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.540+0000 7fdb9d722740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5613e25c3390 con 0x5613e21ee390 2024-08-31T06:21:04.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.540+0000 7fdb9d722740 10 client.0 _ll_register_callbacks cb 0x5613e24c7470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:21:04.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.540+0000 7fdb8fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613e21e61b0 0x5613e25c00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:04.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.540+0000 7fdb8f400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613e21ee390 0x5613e25c0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:04.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.540+0000 7fdb8fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613e21e61b0 0x5613e25c00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47412/0 (socket says 172.21.15.16:47412) 2024-08-31T06:21:04.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.540+0000 7fdb8fe00640 1 -- 172.21.15.16:0/2110798501 learned_addr learned my addr 172.21.15.16:0/2110798501 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:04.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.540+0000 7fdb94e00640 1 --2- 172.21.15.16:0/2110798501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613e21cd9f0 0x5613e25c0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:04.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.541+0000 7fdb8fe00640 1 -- 172.21.15.16:0/2110798501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613e21ee390 msgr2=0x5613e25c0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:04.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.541+0000 7fdb8fe00640 1 --2- 172.21.15.16:0/2110798501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613e21ee390 0x5613e25c0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:04.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.541+0000 7fdb8fe00640 1 -- 172.21.15.16:0/2110798501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613e21cd9f0 msgr2=0x5613e25c0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:04.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.541+0000 7fdb8fe00640 1 --2- 172.21.15.16:0/2110798501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613e21cd9f0 0x5613e25c0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:04.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.541+0000 7fdb8fe00640 1 -- 172.21.15.16:0/2110798501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613e24e28b0 con 0x5613e21e61b0 2024-08-31T06:21:04.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.542+0000 7fdb8fe00640 1 --2- 172.21.15.16:0/2110798501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613e21e61b0 0x5613e25c00e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fdb80018590 tx=0x7fdb80005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe9d4aee16d69f9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:04.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.543+0000 7fdb7a000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:21:04.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.543+0000 7fdb7a000640 1 -- 172.21.15.16:0/2110798501 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8001d640 con 0x5613e21e61b0 2024-08-31T06:21:04.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.544+0000 7fdb7a000640 1 -- 172.21.15.16:0/2110798501 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8514+0+0 (secure 0 0 0) 0x7fdb8001d7e0 con 0x5613e21e61b0 2024-08-31T06:21:04.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.544+0000 7fdb9d722740 1 -- 172.21.15.16:0/2110798501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5613e252d700 con 0x5613e21e61b0 2024-08-31T06:21:04.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.544+0000 7fdb7a000640 1 -- 172.21.15.16:0/2110798501 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fdb80015480 con 0x5613e21e61b0 2024-08-31T06:21:04.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.545+0000 7fdb9d722740 10 client.18252 fetch_fsmap learned FSMap version 49 2024-08-31T06:21:04.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.545+0000 7fdb9d722740 1 -- 172.21.15.16:0/2110798501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5613e25cf040 con 0x5613e21e61b0 2024-08-31T06:21:04.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.546+0000 7fdb7a000640 1 -- 172.21.15.16:0/2110798501 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5613e252d700 con 0x5613e21e61b0 2024-08-31T06:21:04.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.547+0000 7fdb7a000640 1 -- 172.21.15.16:0/2110798501 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fdb80003b90 con 0x5613e21e61b0 2024-08-31T06:21:04.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.547+0000 7fdb9d722740 10 client.18252 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:21:04.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.547+0000 7fdb9d722740 10 client.18252 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:21:04.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.547+0000 7fdb9d722740 20 client.18252 populate_metadata read hostname 'smithi016' 2024-08-31T06:21:04.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.547+0000 7fdb9d722740 1 --2- 172.21.15.16:0/2110798501 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5613e21a2da0 0x5613e25d2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:04.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.547+0000 7fdb9d722740 4 client.18252 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:21:04.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.547+0000 7fdb9d722740 1 -- 172.21.15.16:0/2110798501 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613e25291a0 con 0x5613e21a2da0 2024-08-31T06:21:04.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.548+0000 7fdb8f400640 1 --2- 172.21.15.16:0/2110798501 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5613e21a2da0 0x5613e25d2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:04.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.550+0000 7fdb8f400640 1 --2- 172.21.15.16:0/2110798501 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5613e21a2da0 0x5613e25d2450 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3d15955fb3e90ae server_cookie=b60cb90a226062d1 in_seq=0 out_seq=0 2024-08-31T06:21:04.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.550+0000 7fdb7a000640 10 client.18252 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:21:04.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.552+0000 7fdb7a000640 1 -- 172.21.15.16:0/2110798501 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5613e25291a0 con 0x5613e21a2da0 2024-08-31T06:21:04.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.552+0000 7fdb7a000640 10 client.18252 handle_command_reply: tid=0 2024-08-31T06:21:04.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.552+0000 7fdb9d722740 1 client.18252 shutdown 2024-08-31T06:21:04.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.555+0000 7fdb9d722740 1 -- 172.21.15.16:0/2110798501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613e21e61b0 msgr2=0x5613e25c00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:04.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.555+0000 7fdb9d722740 1 --2- 172.21.15.16:0/2110798501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613e21e61b0 0x5613e25c00e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fdb80018590 tx=0x7fdb80005280 comp rx=0 tx=0).stop 2024-08-31T06:21:04.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.555+0000 7fdb9d722740 1 -- 172.21.15.16:0/2110798501 shutdown_connections 2024-08-31T06:21:04.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.555+0000 7fdb9d722740 1 --2- 172.21.15.16:0/2110798501 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5613e21a2da0 0x5613e25d2450 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:04.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.555+0000 7fdb7a000640 0 client.18252 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:21:04.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.555+0000 7fdb9d722740 1 --2- 172.21.15.16:0/2110798501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613e21ee390 0x5613e25c0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:04.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.555+0000 7fdb9d722740 1 --2- 172.21.15.16:0/2110798501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613e21cd9f0 0x5613e25c0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:04.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.555+0000 7fdb9d722740 1 --2- 172.21.15.16:0/2110798501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613e21e61b0 0x5613e25c00e0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:04.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.555+0000 7fdb9d722740 1 -- 172.21.15.16:0/2110798501 >> 172.21.15.16:0/2110798501 conn(0x5613e2203460 msgr2=0x5613e25b7d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:04.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.555+0000 7fdb9d722740 1 -- 172.21.15.16:0/2110798501 shutdown_connections 2024-08-31T06:21:04.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.557+0000 7fdb9d722740 1 -- 172.21.15.16:0/2110798501 wait complete. 2024-08-31T06:21:04.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.559+0000 7fdb9d722740 20 client.18252 trim_cache size 0 max 16384 2024-08-31T06:21:04.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.588+0000 7fdb9d722740 1 Processor -- start 2024-08-31T06:21:04.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.588+0000 7fdb9d722740 1 -- start start 2024-08-31T06:21:04.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.589+0000 7fdb9d722740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613e21cd9f0 0x5613e25c07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:04.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.589+0000 7fdb9d722740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613e21e61b0 0x5613e25b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:04.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.589+0000 7fdb9d722740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613e21ee390 0x5613e25b2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:04.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.589+0000 7fdb8fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613e21cd9f0 0x5613e25c07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:04.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.589+0000 7fdb8fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613e21cd9f0 0x5613e25c07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:48222/0 (socket says 172.21.15.16:48222) 2024-08-31T06:21:04.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.590+0000 7fdb9d722740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdb901b8330 con 0x5613e21ee390 2024-08-31T06:21:04.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.590+0000 7fdb9d722740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdb901b5b40 con 0x5613e21cd9f0 2024-08-31T06:21:04.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.590+0000 7fdb9d722740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdb901b39b0 con 0x5613e21e61b0 2024-08-31T06:21:04.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.590+0000 7fdb8f400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613e21e61b0 0x5613e25b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:04.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.590+0000 7fdb8fe00640 1 -- 172.21.15.16:0/4087233125 learned_addr learned my addr 172.21.15.16:0/4087233125 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:04.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.591+0000 7fdb94e00640 1 --2- 172.21.15.16:0/4087233125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613e21ee390 0x5613e25b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:04.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.591+0000 7fdb8fe00640 1 -- 172.21.15.16:0/4087233125 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613e21e61b0 msgr2=0x5613e25b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:04.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.591+0000 7fdb8fe00640 1 --2- 172.21.15.16:0/4087233125 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613e21e61b0 0x5613e25b2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:04.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.591+0000 7fdb8fe00640 1 -- 172.21.15.16:0/4087233125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613e21ee390 msgr2=0x5613e25b2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:04.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.591+0000 7fdb8fe00640 1 --2- 172.21.15.16:0/4087233125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613e21ee390 0x5613e25b2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:04.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.591+0000 7fdb8fe00640 1 -- 172.21.15.16:0/4087233125 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613e25d2780 con 0x5613e21cd9f0 2024-08-31T06:21:04.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.593+0000 7fdb8fe00640 1 --2- 172.21.15.16:0/4087233125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613e21cd9f0 0x5613e25c07f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fdb80005dc0 tx=0x7fdb80015fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e5782442cf6b07 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:04.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.594+0000 7fdb84c00640 1 -- 172.21.15.16:0/4087233125 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8003d070 con 0x5613e21cd9f0 2024-08-31T06:21:04.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.594+0000 7fdb84c00640 1 -- 172.21.15.16:0/4087233125 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb8000fe50 con 0x5613e21cd9f0 2024-08-31T06:21:04.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.594+0000 7fdb84c00640 1 -- 172.21.15.16:0/4087233125 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8001dd10 con 0x5613e21cd9f0 2024-08-31T06:21:04.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.595+0000 7fdb9d722740 1 -- 172.21.15.16:0/4087233125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613e21cd9f0 msgr2=0x5613e25c07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:04.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.595+0000 7fdb9d722740 1 --2- 172.21.15.16:0/4087233125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613e21cd9f0 0x5613e25c07f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fdb80005dc0 tx=0x7fdb80015fc0 comp rx=0 tx=0).stop 2024-08-31T06:21:04.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.595+0000 7fdb9d722740 1 -- 172.21.15.16:0/4087233125 shutdown_connections 2024-08-31T06:21:04.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.595+0000 7fdb9d722740 1 --2- 172.21.15.16:0/4087233125 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613e21e61b0 0x5613e25b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:04.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.595+0000 7fdb9d722740 1 --2- 172.21.15.16:0/4087233125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613e21ee390 0x5613e25b2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:04.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.595+0000 7fdb9d722740 1 --2- 172.21.15.16:0/4087233125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613e21cd9f0 0x5613e25c07f0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:04.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.595+0000 7fdb9d722740 1 -- 172.21.15.16:0/4087233125 >> 172.21.15.16:0/4087233125 conn(0x5613e2203460 msgr2=0x5613e25c00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:04.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.596+0000 7fdb9d722740 1 -- 172.21.15.16:0/4087233125 shutdown_connections 2024-08-31T06:21:04.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.596+0000 7fdb9d722740 1 -- 172.21.15.16:0/4087233125 wait complete. 2024-08-31T06:21:04.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.597+0000 7fdb9d722740 1 -- start start 2024-08-31T06:21:04.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.597+0000 7fdb9d722740 1 Processor -- start 2024-08-31T06:21:04.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.598+0000 7fdb9d722740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613e21cd9f0 0x5613e25c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:04.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.598+0000 7fdb9d722740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613e21e61b0 0x5613e25c4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:04.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.599+0000 7fdb9d722740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613e21ee390 0x5613e25c4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:04.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.599+0000 7fdb9d722740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdb901b4000 con 0x5613e21ee390 2024-08-31T06:21:04.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.599+0000 7fdb9d722740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdb901b5470 con 0x5613e21cd9f0 2024-08-31T06:21:04.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.599+0000 7fdb8fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613e21cd9f0 0x5613e25c8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:04.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.599+0000 7fdb8fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613e21cd9f0 0x5613e25c8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:48232/0 (socket says 172.21.15.16:48232) 2024-08-31T06:21:04.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.599+0000 7fdb8fe00640 1 -- 172.21.15.16:0/3595421826 learned_addr learned my addr 172.21.15.16:0/3595421826 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:04.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.599+0000 7fdb9d722740 1 -- 172.21.15.16:0/3595421826 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdb901b46d0 con 0x5613e21e61b0 2024-08-31T06:21:04.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.599+0000 7fdb9d722740 10 client.0 _ll_register_callbacks cb 0x7fdb80018f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:21:04.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.599+0000 7fdb8f400640 1 --2- 172.21.15.16:0/3595421826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613e21e61b0 0x5613e25c4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:04.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.600+0000 7fdb8f400640 1 -- 172.21.15.16:0/3595421826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613e21cd9f0 msgr2=0x5613e25c8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:04.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.600+0000 7fdb8f400640 1 --2- 172.21.15.16:0/3595421826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613e21cd9f0 0x5613e25c8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:04.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.600+0000 7fdb8f400640 1 -- 172.21.15.16:0/3595421826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613e21ee390 msgr2=0x5613e25c4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:04.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.600+0000 7fdb8f400640 1 --2- 172.21.15.16:0/3595421826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613e21ee390 0x5613e25c4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:04.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.600+0000 7fdb8f400640 1 -- 172.21.15.16:0/3595421826 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613e25c4e40 con 0x5613e21e61b0 2024-08-31T06:21:04.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.601+0000 7fdb8f400640 1 --2- 172.21.15.16:0/3595421826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613e21e61b0 0x5613e25c4210 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x5613e25d29f0 tx=0x7fdb7c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da16be152ddb62e9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:04.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.601+0000 7fdb7a000640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:21:04.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.601+0000 7fdb7a000640 1 -- 172.21.15.16:0/3595421826 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c034060 con 0x5613e21e61b0 2024-08-31T06:21:04.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.601+0000 7fdb7a000640 1 -- 172.21.15.16:0/3595421826 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8514+0+0 (secure 0 0 0) 0x7fdb7c029770 con 0x5613e21e61b0 2024-08-31T06:21:04.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.602+0000 7fdb9d722740 1 -- 172.21.15.16:0/3595421826 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb8000fe50 con 0x5613e21e61b0 2024-08-31T06:21:04.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.602+0000 7fdb7a000640 1 -- 172.21.15.16:0/3595421826 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fdb7c029970 con 0x5613e21e61b0 2024-08-31T06:21:04.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.603+0000 7fdb9d722740 10 client.28747 fetch_fsmap learned FSMap version 49 2024-08-31T06:21:04.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.603+0000 7fdb9d722740 1 -- 172.21.15.16:0/3595421826 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5613e263cb40 con 0x5613e21e61b0 2024-08-31T06:21:04.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.603+0000 7fdb7a000640 1 -- 172.21.15.16:0/3595421826 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb8000fe50 con 0x5613e21e61b0 2024-08-31T06:21:04.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.604+0000 7fdb7a000640 1 -- 172.21.15.16:0/3595421826 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fdb7c02e030 con 0x5613e21e61b0 2024-08-31T06:21:04.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.604+0000 7fdb9d722740 10 client.28747 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:21:04.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.605+0000 7fdb9d722740 10 client.28747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:21:04.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.605+0000 7fdb9d722740 20 client.28747 populate_metadata read hostname 'smithi016' 2024-08-31T06:21:04.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.605+0000 7fdb9d722740 1 --2- 172.21.15.16:0/3595421826 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5613e263d280 0x5613e263f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:04.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.605+0000 7fdb9d722740 4 client.28747 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:21:04.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.605+0000 7fdb9d722740 1 -- 172.21.15.16:0/3595421826 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5613e25268e0 con 0x5613e263d280 2024-08-31T06:21:04.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.607+0000 7fdb8fe00640 1 --2- 172.21.15.16:0/3595421826 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5613e263d280 0x5613e263f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:04.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.608+0000 7fdb8fe00640 1 --2- 172.21.15.16:0/3595421826 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5613e263d280 0x5613e263f6a0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee84582b2b634439 server_cookie=efaf0ba85b946033 in_seq=0 out_seq=0 2024-08-31T06:21:04.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.608+0000 7fdb7a000640 10 client.28747 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:21:04.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.609+0000 7fdb7a000640 1 -- 172.21.15.16:0/3595421826 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5613e25268e0 con 0x5613e263d280 2024-08-31T06:21:04.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.609+0000 7fdb7a000640 10 client.28747 handle_command_reply: tid=0 2024-08-31T06:21:04.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.609+0000 7fdb9d722740 1 client.28747 shutdown 2024-08-31T06:21:04.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.610+0000 7fdb9d722740 1 -- 172.21.15.16:0/3595421826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613e21e61b0 msgr2=0x5613e25c4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:04.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.610+0000 7fdb9d722740 1 --2- 172.21.15.16:0/3595421826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613e21e61b0 0x5613e25c4210 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x5613e25d29f0 tx=0x7fdb7c004730 comp rx=0 tx=0).stop 2024-08-31T06:21:04.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.610+0000 7fdb9d722740 1 -- 172.21.15.16:0/3595421826 shutdown_connections 2024-08-31T06:21:04.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.610+0000 7fdb9d722740 1 --2- 172.21.15.16:0/3595421826 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5613e263d280 0x5613e263f6a0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:04.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.610+0000 7fdb7a000640 0 client.28747 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:21:04.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.610+0000 7fdb9d722740 1 --2- 172.21.15.16:0/3595421826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613e21e61b0 0x5613e25c4210 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:04.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.610+0000 7fdb9d722740 1 --2- 172.21.15.16:0/3595421826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613e21ee390 0x5613e25c4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:04.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.610+0000 7fdb9d722740 1 --2- 172.21.15.16:0/3595421826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613e21cd9f0 0x5613e25c8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:04.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.610+0000 7fdb9d722740 1 -- 172.21.15.16:0/3595421826 >> 172.21.15.16:0/3595421826 conn(0x5613e2203460 msgr2=0x5613e25b5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:04.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.611+0000 7fdb9d722740 1 -- 172.21.15.16:0/3595421826 shutdown_connections 2024-08-31T06:21:04.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.611+0000 7fdb9d722740 1 -- 172.21.15.16:0/3595421826 wait complete. 2024-08-31T06:21:04.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.612+0000 7fdb9d722740 20 client.28747 trim_cache size 0 max 16384 2024-08-31T06:21:04.616 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:21:04.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.614+0000 7fdb95800640 1 -- 172.21.15.16:0/3367525387 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdb740b4da0 msgr2=0x7fdb740b72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:04.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.614+0000 7fdb95800640 1 --2- 172.21.15.16:0/3367525387 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdb740b4da0 0x7fdb740b72b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x5613e25b37e0 tx=0x7fdb7c005f90 comp rx=0 tx=0).stop 2024-08-31T06:21:04.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.614+0000 7fdb95800640 1 -- 172.21.15.16:0/3367525387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb9010ffd0 msgr2=0x7fdb901a5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:04.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.614+0000 7fdb95800640 1 --2- 172.21.15.16:0/3367525387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb9010ffd0 0x7fdb901a5ea0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fdb88009b30 tx=0x7fdb88003cc0 comp rx=0 tx=0).stop 2024-08-31T06:21:04.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.614+0000 7fdb95800640 1 -- 172.21.15.16:0/3367525387 shutdown_connections 2024-08-31T06:21:04.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.614+0000 7fdb95800640 1 --2- 172.21.15.16:0/3367525387 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdb740b4da0 0x7fdb740b72b0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:04.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.614+0000 7fdb95800640 1 --2- 172.21.15.16:0/3367525387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb9010ffd0 0x7fdb901a5ea0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:04.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.614+0000 7fdb95800640 1 --2- 172.21.15.16:0/3367525387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb9010aab0 0x7fdb901a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:04.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.614+0000 7fdb95800640 1 --2- 172.21.15.16:0/3367525387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb9010a160 0x7fdb901a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:04.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.614+0000 7fdb95800640 1 -- 172.21.15.16:0/3367525387 >> 172.21.15.16:0/3367525387 conn(0x7fdb9006ebf0 msgr2=0x7fdb900739f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:04.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.614+0000 7fdb95800640 1 -- 172.21.15.16:0/3367525387 shutdown_connections 2024-08-31T06:21:04.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:04.617+0000 7fdb95800640 1 -- 172.21.15.16:0/3367525387 wait complete. 2024-08-31T06:21:05.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:04 smithi155 ceph-mon[21742]: pgmap v795: 97 pgs: 97 active+clean; 4.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 7.6 MiB/s wr, 786 op/s 2024-08-31T06:21:05.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:04 smithi155 ceph-mon[21742]: osdmap e98: 12 total, 12 up, 12 in 2024-08-31T06:21:05.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:04 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/1060583811' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:21:05.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:04 smithi016 ceph-mon[17194]: pgmap v795: 97 pgs: 97 active+clean; 4.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 7.6 MiB/s wr, 786 op/s 2024-08-31T06:21:05.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:04 smithi016 ceph-mon[17194]: osdmap e98: 12 total, 12 up, 12 in 2024-08-31T06:21:05.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:04 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/1060583811' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:21:05.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:04 smithi184 ceph-mon[22331]: pgmap v795: 97 pgs: 97 active+clean; 4.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 7.6 MiB/s wr, 786 op/s 2024-08-31T06:21:05.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:04 smithi184 ceph-mon[22331]: osdmap e98: 12 total, 12 up, 12 in 2024-08-31T06:21:05.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:04 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/1060583811' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:21:05.789 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24737 quiesce db / --timeout 90 --expiration 155.0 --await-for 45.0' 2024-08-31T06:21:05.789 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24737 quiesce db / --timeout 90 --expiration 155.0 --await-for 45.0 2024-08-31T06:21:06.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:05 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:21:06.142 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:05 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:21:06.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:05 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:21:06.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 820.0 s, 523.6 tps, lat 184.571 ms stddev 274.685 2024-08-31T06:21:07.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:06 smithi155 ceph-mon[21742]: pgmap v797: 97 pgs: 19 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 69 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 16 MiB/s wr, 1.56k op/s 2024-08-31T06:21:07.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:06 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:21:07.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:06 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:21:07.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:06 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:21:07.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:06 smithi016 ceph-mon[17194]: pgmap v797: 97 pgs: 19 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 69 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 16 MiB/s wr, 1.56k op/s 2024-08-31T06:21:07.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:06 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:21:07.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:06 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:21:07.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:06 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:21:07.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:06 smithi184 ceph-mon[22331]: pgmap v797: 97 pgs: 19 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 69 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 16 MiB/s wr, 1.56k op/s 2024-08-31T06:21:07.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:06 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:21:07.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:06 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:21:07.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:06 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:21:08.493 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:21:08.923 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:08 smithi016 ceph-mon[17194]: pgmap v798: 97 pgs: 19 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 69 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 16 MiB/s wr, 1.54k op/s 2024-08-31T06:21:09.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:08 smithi184 ceph-mon[22331]: pgmap v798: 97 pgs: 19 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 69 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 16 MiB/s wr, 1.54k op/s 2024-08-31T06:21:09.330 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T06:21:09.330 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T06:21:09.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:08 smithi155 ceph-mon[21742]: pgmap v798: 97 pgs: 19 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 69 active+clean; 3.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 16 MiB/s wr, 1.54k op/s 2024-08-31T06:21:09.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.375+0000 7fe33c800640 1 -- 172.21.15.16:0/1162247601 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3340754f0 msgr2=0x7fe334075900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:09.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.375+0000 7fe33c800640 1 --2- 172.21.15.16:0/1162247601 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3340754f0 0x7fe334075900 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe328009900 tx=0x7fe32802a400 comp rx=0 tx=0).stop 2024-08-31T06:21:09.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.376+0000 7fe33c800640 1 -- 172.21.15.16:0/1162247601 shutdown_connections 2024-08-31T06:21:09.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.376+0000 7fe33c800640 1 --2- 172.21.15.16:0/1162247601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe33410fef0 0x7fe334112310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:09.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.376+0000 7fe33c800640 1 --2- 172.21.15.16:0/1162247601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe334075e40 0x7fe33410f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:09.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.376+0000 7fe33c800640 1 --2- 172.21.15.16:0/1162247601 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3340754f0 0x7fe334075900 secure :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe328009900 tx=0x7fe32802a400 comp rx=0 tx=0).stop 2024-08-31T06:21:09.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.376+0000 7fe33c800640 1 -- 172.21.15.16:0/1162247601 >> 172.21.15.16:0/1162247601 conn(0x7fe3340ff380 msgr2=0x7fe3341017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:09.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.376+0000 7fe33c800640 1 -- 172.21.15.16:0/1162247601 shutdown_connections 2024-08-31T06:21:09.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.376+0000 7fe33c800640 1 -- 172.21.15.16:0/1162247601 wait complete. 2024-08-31T06:21:09.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.378+0000 7fe33c800640 1 Processor -- start 2024-08-31T06:21:09.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.378+0000 7fe33c800640 1 -- start start 2024-08-31T06:21:09.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.379+0000 7fe33c800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe334075e40 0x7fe33410f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:09.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.379+0000 7fe33c800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe33410fef0 0x7fe33410dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:09.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.379+0000 7fe33c800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe33410e050 0x7fe33410e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:09.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.379+0000 7fe33c800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe3341039f0 con 0x7fe33410e050 2024-08-31T06:21:09.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.379+0000 7fe33c800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe334103870 con 0x7fe33410fef0 2024-08-31T06:21:09.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.379+0000 7fe33c800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe334103b70 con 0x7fe334075e40 2024-08-31T06:21:09.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.379+0000 7fe33a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe33410e050 0x7fe33410e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:09.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.379+0000 7fe33a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe33410e050 0x7fe33410e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39368/0 (socket says 172.21.15.16:39368) 2024-08-31T06:21:09.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.379+0000 7fe33a800640 1 -- 172.21.15.16:0/528244074 learned_addr learned my addr 172.21.15.16:0/528244074 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:09.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.380+0000 7fe339e00640 1 --2- 172.21.15.16:0/528244074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe334075e40 0x7fe33410f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:09.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.380+0000 7fe339400640 1 --2- 172.21.15.16:0/528244074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe33410fef0 0x7fe33410dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:09.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.381+0000 7fe33a800640 1 -- 172.21.15.16:0/528244074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe334075e40 msgr2=0x7fe33410f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:09.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.381+0000 7fe33a800640 1 --2- 172.21.15.16:0/528244074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe334075e40 0x7fe33410f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:09.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.381+0000 7fe33a800640 1 -- 172.21.15.16:0/528244074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe33410fef0 msgr2=0x7fe33410dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:09.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.381+0000 7fe33a800640 1 --2- 172.21.15.16:0/528244074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe33410fef0 0x7fe33410dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:09.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.381+0000 7fe33a800640 1 -- 172.21.15.16:0/528244074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3341acdc0 con 0x7fe33410e050 2024-08-31T06:21:09.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.382+0000 7fe33a800640 1 --2- 172.21.15.16:0/528244074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe33410e050 0x7fe33410e550 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe33000eb10 tx=0x7fe33000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:09.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.382+0000 7fe32ea00640 1 -- 172.21.15.16:0/528244074 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe330004b20 con 0x7fe33410e050 2024-08-31T06:21:09.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.382+0000 7fe32ea00640 1 -- 172.21.15.16:0/528244074 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe330004cc0 con 0x7fe33410e050 2024-08-31T06:21:09.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.382+0000 7fe33c800640 1 -- 172.21.15.16:0/528244074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3341ad0b0 con 0x7fe33410e050 2024-08-31T06:21:09.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.382+0000 7fe32ea00640 1 -- 172.21.15.16:0/528244074 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe330010670 con 0x7fe33410e050 2024-08-31T06:21:09.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.382+0000 7fe33c800640 1 -- 172.21.15.16:0/528244074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3341ad610 con 0x7fe33410e050 2024-08-31T06:21:09.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.389+0000 7fe32ea00640 1 -- 172.21.15.16:0/528244074 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fe330005020 con 0x7fe33410e050 2024-08-31T06:21:09.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.392+0000 7fe34348a740 1 Processor -- start 2024-08-31T06:21:09.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.392+0000 7fe32ea00640 1 --2- 172.21.15.16:0/528244074 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe31c0b4da0 0x7fe31c0b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:09.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.392+0000 7fe32ea00640 1 -- 172.21.15.16:0/528244074 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8514+0+0 (secure 0 0 0) 0x7fe3300e5b30 con 0x7fe33410e050 2024-08-31T06:21:09.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.394+0000 7fe34348a740 1 -- start start 2024-08-31T06:21:09.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.394+0000 7fe339400640 1 --2- 172.21.15.16:0/528244074 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe31c0b4da0 0x7fe31c0b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:09.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.395+0000 7fe34348a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a456a9a840 0x55a456e41110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:09.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.395+0000 7fe34348a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a456a71980 0x55a456e41650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:09.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.396+0000 7fe34348a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a456a79b60 0x55a456e41b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:09.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.396+0000 7fe34348a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a456db6530 con 0x55a456a71980 2024-08-31T06:21:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.396+0000 7fe339e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a456a9a840 0x55a456e41110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.396+0000 7fe34348a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a456da5ac0 con 0x55a456a79b60 2024-08-31T06:21:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.396+0000 7fe339e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a456a9a840 0x55a456e41110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:45440/0 (socket says 172.21.15.16:45440) 2024-08-31T06:21:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.396+0000 7fe339e00640 1 -- 172.21.15.16:0/121633646 learned_addr learned my addr 172.21.15.16:0/121633646 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.396+0000 7fe34348a740 1 -- 172.21.15.16:0/121633646 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a456da9a70 con 0x55a456a9a840 2024-08-31T06:21:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.396+0000 7fe339400640 1 --2- 172.21.15.16:0/528244074 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe31c0b4da0 0x7fe31c0b72b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe324005e10 tx=0x7fe324005da0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.396+0000 7fe33a800640 1 --2- 172.21.15.16:0/121633646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a456a79b60 0x55a456e41b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.396+0000 7fe339e00640 1 -- 172.21.15.16:0/121633646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a456a79b60 msgr2=0x55a456e41b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.396+0000 7fe339e00640 1 --2- 172.21.15.16:0/121633646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a456a79b60 0x55a456e41b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:09.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.396+0000 7fe339e00640 1 -- 172.21.15.16:0/121633646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a456a71980 msgr2=0x55a456e41650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:21:09.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.398+0000 7fe339400640 1 --2- 172.21.15.16:0/121633646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a456a71980 0x55a456e41650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:09.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.398+0000 7fe339e00640 1 --2- 172.21.15.16:0/121633646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a456a71980 0x55a456e41650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:09.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.398+0000 7fe339e00640 1 -- 172.21.15.16:0/121633646 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a456d58880 con 0x55a456a9a840 2024-08-31T06:21:09.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.399+0000 7fe339400640 1 --2- 172.21.15.16:0/121633646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a456a71980 0x55a456e41650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:21:09.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.399+0000 7fe339e00640 1 --2- 172.21.15.16:0/121633646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a456a9a840 0x55a456e41110 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fe32802a9f0 tx=0x7fe328004200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=678bb4b26bdb458e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:09.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.400+0000 7fe323000640 1 -- 172.21.15.16:0/121633646 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe328040070 con 0x55a456a9a840 2024-08-31T06:21:09.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.400+0000 7fe323000640 1 -- 172.21.15.16:0/121633646 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3280054f0 con 0x55a456a9a840 2024-08-31T06:21:09.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.400+0000 7fe323000640 1 -- 172.21.15.16:0/121633646 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe328035040 con 0x55a456a9a840 2024-08-31T06:21:09.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.402+0000 7fe34348a740 1 -- 172.21.15.16:0/121633646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a456a9a840 msgr2=0x55a456e41110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:09.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.402+0000 7fe34348a740 1 --2- 172.21.15.16:0/121633646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a456a9a840 0x55a456e41110 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fe32802a9f0 tx=0x7fe328004200 comp rx=0 tx=0).stop 2024-08-31T06:21:09.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.403+0000 7fe34348a740 1 -- 172.21.15.16:0/121633646 shutdown_connections 2024-08-31T06:21:09.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.403+0000 7fe34348a740 1 --2- 172.21.15.16:0/121633646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a456a79b60 0x55a456e41b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:09.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.403+0000 7fe34348a740 1 --2- 172.21.15.16:0/121633646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a456a71980 0x55a456e41650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:09.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.403+0000 7fe34348a740 1 --2- 172.21.15.16:0/121633646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a456a9a840 0x55a456e41110 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:09.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.403+0000 7fe34348a740 1 -- 172.21.15.16:0/121633646 >> 172.21.15.16:0/121633646 conn(0x55a456a8e990 msgr2=0x55a456e3f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:09.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.404+0000 7fe34348a740 1 -- 172.21.15.16:0/121633646 shutdown_connections 2024-08-31T06:21:09.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.404+0000 7fe34348a740 1 -- 172.21.15.16:0/121633646 wait complete. 2024-08-31T06:21:09.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.405+0000 7fe34348a740 1 -- start start 2024-08-31T06:21:09.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.405+0000 7fe34348a740 1 Processor -- start 2024-08-31T06:21:09.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.406+0000 7fe34348a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a456a71980 0x55a456e4cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:09.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.406+0000 7fe34348a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a456a79b60 0x55a456e4d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:09.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.407+0000 7fe34348a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a456a9a840 0x55a456e58b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:09.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.407+0000 7fe34348a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a456c087a0 con 0x55a456a71980 2024-08-31T06:21:09.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.407+0000 7fe34348a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a456d2c7f0 con 0x55a456a9a840 2024-08-31T06:21:09.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.407+0000 7fe34348a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a456e4f890 con 0x55a456a79b60 2024-08-31T06:21:09.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.407+0000 7fe34348a740 10 client.0 _ll_register_callbacks cb 0x55a456d51e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:21:09.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.407+0000 7fe339e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a456a71980 0x55a456e4cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:09.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.408+0000 7fe33a800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a456a9a840 0x55a456e58b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:09.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.408+0000 7fe33a800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a456a9a840 0x55a456e58b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:54038/0 (socket says 172.21.15.16:54038) 2024-08-31T06:21:09.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.408+0000 7fe339400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a456a79b60 0x55a456e4d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:09.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.408+0000 7fe33a800640 1 -- 172.21.15.16:0/3119686931 learned_addr learned my addr 172.21.15.16:0/3119686931 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:09.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.408+0000 7fe339e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a456a71980 0x55a456e4cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39380/0 (socket says 172.21.15.16:39380) 2024-08-31T06:21:09.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.408+0000 7fe33a800640 1 -- 172.21.15.16:0/3119686931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a456a79b60 msgr2=0x55a456e4d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:09.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.408+0000 7fe33a800640 1 --2- 172.21.15.16:0/3119686931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a456a79b60 0x55a456e4d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:09.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.408+0000 7fe33a800640 1 -- 172.21.15.16:0/3119686931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a456a71980 msgr2=0x55a456e4cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:09.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.408+0000 7fe33a800640 1 --2- 172.21.15.16:0/3119686931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a456a71980 0x55a456e4cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:09.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.408+0000 7fe33a800640 1 -- 172.21.15.16:0/3119686931 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a456b14b90 con 0x55a456a9a840 2024-08-31T06:21:09.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.408+0000 7fe339e00640 1 --2- 172.21.15.16:0/3119686931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a456a71980 0x55a456e4cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:21:09.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.409+0000 7fe33a800640 1 --2- 172.21.15.16:0/3119686931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a456a9a840 0x55a456e58b80 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe3300049d0 tx=0x7fe3300a8190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d54530460dd159a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:09.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.409+0000 7fe317400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:21:09.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.409+0000 7fe317400640 1 -- 172.21.15.16:0/3119686931 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3300abcd0 con 0x55a456a9a840 2024-08-31T06:21:09.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.409+0000 7fe317400640 1 -- 172.21.15.16:0/3119686931 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8514+0+0 (secure 0 0 0) 0x7fe3300b5260 con 0x55a456a9a840 2024-08-31T06:21:09.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.409+0000 7fe34348a740 1 -- 172.21.15.16:0/3119686931 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3280054f0 con 0x55a456a9a840 2024-08-31T06:21:09.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.409+0000 7fe317400640 1 -- 172.21.15.16:0/3119686931 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fe3300ac8c0 con 0x55a456a9a840 2024-08-31T06:21:09.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.409+0000 7fe34348a740 10 client.28742 fetch_fsmap learned FSMap version 49 2024-08-31T06:21:09.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.409+0000 7fe34348a740 1 -- 172.21.15.16:0/3119686931 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55a456e5b920 con 0x55a456a9a840 2024-08-31T06:21:09.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.410+0000 7fe317400640 1 -- 172.21.15.16:0/3119686931 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3280054f0 con 0x55a456a9a840 2024-08-31T06:21:09.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.410+0000 7fe317400640 1 -- 172.21.15.16:0/3119686931 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fe3300a7d30 con 0x55a456a9a840 2024-08-31T06:21:09.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.410+0000 7fe34348a740 10 client.28742 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:21:09.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.410+0000 7fe34348a740 10 client.28742 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:21:09.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.410+0000 7fe34348a740 20 client.28742 populate_metadata read hostname 'smithi016' 2024-08-31T06:21:09.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.410+0000 7fe34348a740 1 --2- 172.21.15.16:0/3119686931 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a456a591c0 0x55a456e5ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:09.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.410+0000 7fe34348a740 4 client.28742 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:21:09.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.410+0000 7fe34348a740 1 -- 172.21.15.16:0/3119686931 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a456db8480 con 0x55a456a591c0 2024-08-31T06:21:09.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.410+0000 7fe339e00640 1 --2- 172.21.15.16:0/3119686931 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a456a591c0 0x55a456e5ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:09.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.411+0000 7fe317400640 10 client.28742 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:21:09.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.411+0000 7fe339e00640 1 --2- 172.21.15.16:0/3119686931 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a456a591c0 0x55a456e5ed60 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bf16a745d442a8e server_cookie=55b7072b6f94b398 in_seq=0 out_seq=0 2024-08-31T06:21:09.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.412+0000 7fe317400640 1 -- 172.21.15.16:0/3119686931 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a456db8480 con 0x55a456a591c0 2024-08-31T06:21:09.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.412+0000 7fe317400640 10 client.28742 handle_command_reply: tid=0 2024-08-31T06:21:09.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.412+0000 7fe34348a740 1 client.28742 shutdown 2024-08-31T06:21:09.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.413+0000 7fe34348a740 1 -- 172.21.15.16:0/3119686931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a456a9a840 msgr2=0x55a456e58b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:09.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.413+0000 7fe34348a740 1 --2- 172.21.15.16:0/3119686931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a456a9a840 0x55a456e58b80 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe3300049d0 tx=0x7fe3300a8190 comp rx=0 tx=0).stop 2024-08-31T06:21:09.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.414+0000 7fe34348a740 1 -- 172.21.15.16:0/3119686931 shutdown_connections 2024-08-31T06:21:09.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.414+0000 7fe34348a740 1 --2- 172.21.15.16:0/3119686931 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a456a591c0 0x55a456e5ed60 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:09.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.414+0000 7fe34348a740 1 --2- 172.21.15.16:0/3119686931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a456a79b60 0x55a456e4d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:09.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.414+0000 7fe34348a740 1 --2- 172.21.15.16:0/3119686931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a456a9a840 0x55a456e58b80 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:09.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.414+0000 7fe34348a740 1 --2- 172.21.15.16:0/3119686931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a456a71980 0x55a456e4cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:09.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.414+0000 7fe317400640 0 client.28742 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:21:09.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.414+0000 7fe34348a740 1 -- 172.21.15.16:0/3119686931 >> 172.21.15.16:0/3119686931 conn(0x55a456a8e990 msgr2=0x55a456e3d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:09.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.415+0000 7fe34348a740 1 -- 172.21.15.16:0/3119686931 shutdown_connections 2024-08-31T06:21:09.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.415+0000 7fe34348a740 1 -- 172.21.15.16:0/3119686931 wait complete. 2024-08-31T06:21:09.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.415+0000 7fe34348a740 20 client.28742 trim_cache size 0 max 16384 2024-08-31T06:21:09.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.443+0000 7fe34348a740 1 Processor -- start 2024-08-31T06:21:09.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.443+0000 7fe34348a740 1 -- start start 2024-08-31T06:21:09.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.444+0000 7fe34348a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a456a71980 0x55a456e58a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:09.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.444+0000 7fe34348a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a456a79b60 0x55a456e46220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:09.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.444+0000 7fe34348a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a456a9a840 0x55a456e46760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:09.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.444+0000 7fe34348a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe3341b8990 con 0x55a456a79b60 2024-08-31T06:21:09.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.444+0000 7fe34348a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe3341b61a0 con 0x55a456a9a840 2024-08-31T06:21:09.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.444+0000 7fe34348a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe3341b4010 con 0x55a456a71980 2024-08-31T06:21:09.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.447+0000 7fe339e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a456a71980 0x55a456e58a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:09.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.447+0000 7fe339e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a456a71980 0x55a456e58a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:45462/0 (socket says 172.21.15.16:45462) 2024-08-31T06:21:09.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.447+0000 7fe339e00640 1 -- 172.21.15.16:0/2817089587 learned_addr learned my addr 172.21.15.16:0/2817089587 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:09.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.447+0000 7fe339400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a456a79b60 0x55a456e46220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:09.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.448+0000 7fe339e00640 1 -- 172.21.15.16:0/2817089587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a456a9a840 msgr2=0x55a456e46760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:09.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.448+0000 7fe339e00640 1 --2- 172.21.15.16:0/2817089587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a456a9a840 0x55a456e46760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:09.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.448+0000 7fe339e00640 1 -- 172.21.15.16:0/2817089587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a456a79b60 msgr2=0x55a456e46220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:09.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.448+0000 7fe339e00640 1 --2- 172.21.15.16:0/2817089587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a456a79b60 0x55a456e46220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:09.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.448+0000 7fe339e00640 1 -- 172.21.15.16:0/2817089587 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a456e3e1a0 con 0x55a456a71980 2024-08-31T06:21:09.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.449+0000 7fe339e00640 1 --2- 172.21.15.16:0/2817089587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a456a71980 0x55a456e58a80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fe3280057c0 tx=0x7fe328000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=731f56ba5073c9b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:09.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.449+0000 7fe322600640 1 -- 172.21.15.16:0/2817089587 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe328040070 con 0x55a456a71980 2024-08-31T06:21:09.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.449+0000 7fe322600640 1 -- 172.21.15.16:0/2817089587 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe328004e20 con 0x55a456a71980 2024-08-31T06:21:09.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.449+0000 7fe322600640 1 -- 172.21.15.16:0/2817089587 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe328040070 con 0x55a456a71980 2024-08-31T06:21:09.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.453+0000 7fe34348a740 1 -- 172.21.15.16:0/2817089587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a456a71980 msgr2=0x55a456e58a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:09.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.453+0000 7fe34348a740 1 --2- 172.21.15.16:0/2817089587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a456a71980 0x55a456e58a80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fe3280057c0 tx=0x7fe328000bc0 comp rx=0 tx=0).stop 2024-08-31T06:21:09.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.458+0000 7fe34348a740 1 -- 172.21.15.16:0/2817089587 shutdown_connections 2024-08-31T06:21:09.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.458+0000 7fe34348a740 1 --2- 172.21.15.16:0/2817089587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a456a9a840 0x55a456e46760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:09.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.458+0000 7fe34348a740 1 --2- 172.21.15.16:0/2817089587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a456a79b60 0x55a456e46220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:09.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.458+0000 7fe34348a740 1 --2- 172.21.15.16:0/2817089587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a456a71980 0x55a456e58a80 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:09.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.458+0000 7fe34348a740 1 -- 172.21.15.16:0/2817089587 >> 172.21.15.16:0/2817089587 conn(0x55a456a8e990 msgr2=0x55a456e47d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:09.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.458+0000 7fe34348a740 1 -- 172.21.15.16:0/2817089587 shutdown_connections 2024-08-31T06:21:09.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.458+0000 7fe34348a740 1 -- 172.21.15.16:0/2817089587 wait complete. 2024-08-31T06:21:09.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.460+0000 7fe34348a740 1 -- start start 2024-08-31T06:21:09.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.460+0000 7fe34348a740 1 Processor -- start 2024-08-31T06:21:09.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.464+0000 7fe34348a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a456a71980 0x55a456e511f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:09.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.464+0000 7fe34348a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a456a79b60 0x55a456e51730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:09.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.465+0000 7fe34348a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a456a9a840 0x55a456ec8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:09.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.465+0000 7fe339e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a456a71980 0x55a456e511f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:09.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.465+0000 7fe339e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a456a71980 0x55a456e511f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:54056/0 (socket says 172.21.15.16:54056) 2024-08-31T06:21:09.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.465+0000 7fe339e00640 1 -- 172.21.15.16:0/3706018935 learned_addr learned my addr 172.21.15.16:0/3706018935 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:09.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.465+0000 7fe339400640 1 --2- 172.21.15.16:0/3706018935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a456a79b60 0x55a456e51730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:09.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.465+0000 7fe33a800640 1 --2- 172.21.15.16:0/3706018935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a456a9a840 0x55a456ec8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:09.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.465+0000 7fe34348a740 1 -- 172.21.15.16:0/3706018935 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe3341b4660 con 0x55a456a79b60 2024-08-31T06:21:09.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.466+0000 7fe34348a740 1 -- 172.21.15.16:0/3706018935 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe3341b5ad0 con 0x55a456a71980 2024-08-31T06:21:09.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.466+0000 7fe34348a740 1 -- 172.21.15.16:0/3706018935 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe3341b4d30 con 0x55a456a9a840 2024-08-31T06:21:09.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.466+0000 7fe34348a740 10 client.0 _ll_register_callbacks cb 0x7fe3300e36d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:21:09.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.466+0000 7fe33a800640 1 -- 172.21.15.16:0/3706018935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a456a71980 msgr2=0x55a456e511f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:09.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.466+0000 7fe33a800640 1 --2- 172.21.15.16:0/3706018935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a456a71980 0x55a456e511f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:09.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.466+0000 7fe33a800640 1 -- 172.21.15.16:0/3706018935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a456a79b60 msgr2=0x55a456e51730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:09.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.466+0000 7fe33a800640 1 --2- 172.21.15.16:0/3706018935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a456a79b60 0x55a456e51730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:09.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.466+0000 7fe33a800640 1 -- 172.21.15.16:0/3706018935 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a456ec8c60 con 0x55a456a9a840 2024-08-31T06:21:09.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.467+0000 7fe33a800640 1 --2- 172.21.15.16:0/3706018935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a456a9a840 0x55a456ec8720 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe3300056d0 tx=0x7fe3300e5d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44ed8a5dfa796679 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:09.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.467+0000 7fe317400640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:21:09.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.468+0000 7fe317400640 1 -- 172.21.15.16:0/3706018935 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3300abb10 con 0x55a456a9a840 2024-08-31T06:21:09.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.468+0000 7fe34348a740 1 -- 172.21.15.16:0/3706018935 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe328004e20 con 0x55a456a9a840 2024-08-31T06:21:09.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.468+0000 7fe317400640 1 -- 172.21.15.16:0/3706018935 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8514+0+0 (secure 0 0 0) 0x7fe3300a87b0 con 0x55a456a9a840 2024-08-31T06:21:09.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.468+0000 7fe317400640 1 -- 172.21.15.16:0/3706018935 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fe3300abcd0 con 0x55a456a9a840 2024-08-31T06:21:09.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.469+0000 7fe34348a740 10 client.28765 fetch_fsmap learned FSMap version 49 2024-08-31T06:21:09.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.469+0000 7fe34348a740 1 -- 172.21.15.16:0/3706018935 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55a456ec9e80 con 0x55a456a9a840 2024-08-31T06:21:09.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.470+0000 7fe317400640 1 -- 172.21.15.16:0/3706018935 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe328004e20 con 0x55a456a9a840 2024-08-31T06:21:09.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.471+0000 7fe317400640 1 -- 172.21.15.16:0/3706018935 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fe33008b050 con 0x55a456a9a840 2024-08-31T06:21:09.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.471+0000 7fe34348a740 10 client.28765 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:21:09.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.471+0000 7fe34348a740 10 client.28765 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:21:09.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.471+0000 7fe34348a740 20 client.28765 populate_metadata read hostname 'smithi016' 2024-08-31T06:21:09.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.471+0000 7fe34348a740 1 --2- 172.21.15.16:0/3706018935 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a456a2e570 0x55a456ecc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:09.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.471+0000 7fe34348a740 4 client.28765 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 155.0, "await_for": 45.0}] 2024-08-31T06:21:09.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.471+0000 7fe34348a740 1 -- 172.21.15.16:0/3706018935 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 155.0, "await_for": 45.0}) -- 0x55a456db3f20 con 0x55a456a2e570 2024-08-31T06:21:09.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.472+0000 7fe339e00640 1 --2- 172.21.15.16:0/3706018935 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a456a2e570 0x55a456ecc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:09.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.473+0000 7fe339e00640 1 --2- 172.21.15.16:0/3706018935 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a456a2e570 0x55a456ecc520 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c2104b1c6f3326e server_cookie=13426c5f0ea845de in_seq=0 out_seq=0 2024-08-31T06:21:09.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.473+0000 7fe317400640 10 client.28765 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:21:09.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.724+0000 7fe317400640 1 -- 172.21.15.16:0/3706018935 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a456db3f20 con 0x55a456a2e570 2024-08-31T06:21:09.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.724+0000 7fe317400640 10 client.28765 handle_command_reply: tid=0 2024-08-31T06:21:09.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.725+0000 7fe34348a740 1 client.28765 shutdown 2024-08-31T06:21:09.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.725+0000 7fe34348a740 1 -- 172.21.15.16:0/3706018935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a456a9a840 msgr2=0x55a456ec8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:09.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.725+0000 7fe34348a740 1 --2- 172.21.15.16:0/3706018935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a456a9a840 0x55a456ec8720 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe3300056d0 tx=0x7fe3300e5d80 comp rx=0 tx=0).stop 2024-08-31T06:21:09.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.725+0000 7fe34348a740 1 -- 172.21.15.16:0/3706018935 shutdown_connections 2024-08-31T06:21:09.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.725+0000 7fe34348a740 1 --2- 172.21.15.16:0/3706018935 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a456a2e570 0x55a456ecc520 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:09.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.725+0000 7fe34348a740 1 --2- 172.21.15.16:0/3706018935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a456a9a840 0x55a456ec8720 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:09.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.725+0000 7fe34348a740 1 --2- 172.21.15.16:0/3706018935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a456a79b60 0x55a456e51730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:09.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.725+0000 7fe34348a740 1 --2- 172.21.15.16:0/3706018935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a456a71980 0x55a456e511f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:09.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.725+0000 7fe34348a740 1 -- 172.21.15.16:0/3706018935 >> 172.21.15.16:0/3706018935 conn(0x55a456a8e990 msgr2=0x55a456e46220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:09.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.725+0000 7fe317400640 0 client.28765 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:21:09.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.725+0000 7fe34348a740 1 -- 172.21.15.16:0/3706018935 shutdown_connections 2024-08-31T06:21:09.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.725+0000 7fe34348a740 1 -- 172.21.15.16:0/3706018935 wait complete. 2024-08-31T06:21:09.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.726+0000 7fe34348a740 20 client.28765 trim_cache size 0 max 16384 2024-08-31T06:21:09.728 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:21:09.728 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 49, 2024-08-31T06:21:09.728 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24737, 2024-08-31T06:21:09.729 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 46, 2024-08-31T06:21:09.729 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-08-31T06:21:09.729 INFO:teuthology.orchestra.run.smithi016.stdout: "603ae143": { 2024-08-31T06:21:09.729 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 46, 2024-08-31T06:21:09.729 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-08-31T06:21:09.729 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:21:09.729 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-08-31T06:21:09.729 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:21:09.729 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:21:09.729 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-08-31T06:21:09.729 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 155.0, 2024-08-31T06:21:09.729 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-08-31T06:21:09.729 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-08-31T06:21:09.730 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-08-31T06:21:09.730 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:21:09.730 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-08-31T06:21:09.730 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:21:09.730 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:21:09.730 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:21:09.730 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:21:09.730 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:21:09.730 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:21:09.730 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:21:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.726+0000 7fe33c800640 1 -- 172.21.15.16:0/528244074 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe31c0b4da0 msgr2=0x7fe31c0b72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.726+0000 7fe33c800640 1 --2- 172.21.15.16:0/528244074 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe31c0b4da0 0x7fe31c0b72b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe324005e10 tx=0x7fe324005da0 comp rx=0 tx=0).stop 2024-08-31T06:21:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.726+0000 7fe33c800640 1 -- 172.21.15.16:0/528244074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe33410e050 msgr2=0x7fe33410e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.726+0000 7fe33c800640 1 --2- 172.21.15.16:0/528244074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe33410e050 0x7fe33410e550 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe33000eb10 tx=0x7fe33000c2d0 comp rx=0 tx=0).stop 2024-08-31T06:21:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.727+0000 7fe33c800640 1 -- 172.21.15.16:0/528244074 shutdown_connections 2024-08-31T06:21:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.727+0000 7fe33c800640 1 --2- 172.21.15.16:0/528244074 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fe31c0b4da0 0x7fe31c0b72b0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.727+0000 7fe33c800640 1 --2- 172.21.15.16:0/528244074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe33410e050 0x7fe33410e550 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.727+0000 7fe33c800640 1 --2- 172.21.15.16:0/528244074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe33410fef0 0x7fe33410dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.727+0000 7fe33c800640 1 --2- 172.21.15.16:0/528244074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe334075e40 0x7fe33410f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.727+0000 7fe33c800640 1 -- 172.21.15.16:0/528244074 >> 172.21.15.16:0/528244074 conn(0x7fe3340ff380 msgr2=0x7fe3341062d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.727+0000 7fe33c800640 1 -- 172.21.15.16:0/528244074 shutdown_connections 2024-08-31T06:21:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:09.727+0000 7fe33c800640 1 -- 172.21.15.16:0/528244074 wait complete. 2024-08-31T06:21:10.067 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:09 smithi016 ceph-mon[17194]: pgmap v799: 97 pgs: 11 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 81 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 18 MiB/s wr, 1.62k op/s 2024-08-31T06:21:10.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:09 smithi184 ceph-mon[22331]: pgmap v799: 97 pgs: 11 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 81 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 18 MiB/s wr, 1.62k op/s 2024-08-31T06:21:10.272 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:09 smithi155 ceph-mon[21742]: pgmap v799: 97 pgs: 11 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 81 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 18 MiB/s wr, 1.62k op/s 2024-08-31T06:21:10.704 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:21:10.742 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '603ae143', quiesce took 9.1 seconds. Will release after: 25.9 2024-08-31T06:21:11.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.402+0000 7f56bac00640 1 -- 172.21.15.16:0/2316003034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56b41101e0 msgr2=0x7f56b4116a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:11.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.402+0000 7f56bac00640 1 --2- 172.21.15.16:0/2316003034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56b41101e0 0x7f56b4116a10 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f56ac009ca0 tx=0x7f56ac02a8c0 comp rx=0 tx=0).stop 2024-08-31T06:21:11.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.402+0000 7f56bac00640 1 -- 172.21.15.16:0/2316003034 shutdown_connections 2024-08-31T06:21:11.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.402+0000 7f56bac00640 1 --2- 172.21.15.16:0/2316003034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56b41101e0 0x7f56b4116a10 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:11.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.402+0000 7f56bac00640 1 --2- 172.21.15.16:0/2316003034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56b410f890 0x7f56b410fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:11.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.402+0000 7f56bac00640 1 --2- 172.21.15.16:0/2316003034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56b4109f50 0x7f56b410a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:11.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.402+0000 7f56bac00640 1 -- 172.21.15.16:0/2316003034 >> 172.21.15.16:0/2316003034 conn(0x7f56b4077e80 msgr2=0x7f56b4107f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:11.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.402+0000 7f56bac00640 1 -- 172.21.15.16:0/2316003034 shutdown_connections 2024-08-31T06:21:11.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.402+0000 7f56bac00640 1 -- 172.21.15.16:0/2316003034 wait complete. 2024-08-31T06:21:11.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.403+0000 7f56bac00640 1 Processor -- start 2024-08-31T06:21:11.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.404+0000 7f56bac00640 1 -- start start 2024-08-31T06:21:11.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.404+0000 7f56bac00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56b4109f50 0x7f56b41a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:11.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.404+0000 7f56bac00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56b410f890 0x7f56b41a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:11.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.404+0000 7f56bac00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56b41101e0 0x7f56b41a6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:11.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.404+0000 7f56bac00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f56b4119ef0 con 0x7f56b4109f50 2024-08-31T06:21:11.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.404+0000 7f56bac00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f56b4119d70 con 0x7f56b410f890 2024-08-31T06:21:11.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.404+0000 7f56bac00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f56b411a070 con 0x7f56b41101e0 2024-08-31T06:21:11.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.404+0000 7f56b3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56b4109f50 0x7f56b41a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:11.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.404+0000 7f56b3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56b4109f50 0x7f56b41a0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39414/0 (socket says 172.21.15.16:39414) 2024-08-31T06:21:11.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.404+0000 7f56b3e00640 1 -- 172.21.15.16:0/2429476758 learned_addr learned my addr 172.21.15.16:0/2429476758 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:11.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.405+0000 7f56ab400640 1 --2- 172.21.15.16:0/2429476758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56b410f890 0x7f56b41a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:11.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.405+0000 7f56b8c00640 1 --2- 172.21.15.16:0/2429476758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56b41101e0 0x7f56b41a6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:11.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.405+0000 7f56b3e00640 1 -- 172.21.15.16:0/2429476758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56b41101e0 msgr2=0x7f56b41a6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:11.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.405+0000 7f56b3e00640 1 --2- 172.21.15.16:0/2429476758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56b41101e0 0x7f56b41a6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:11.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.405+0000 7f56b3e00640 1 -- 172.21.15.16:0/2429476758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56b410f890 msgr2=0x7f56b41a0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:11.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.405+0000 7f56b3e00640 1 --2- 172.21.15.16:0/2429476758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56b410f890 0x7f56b41a0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:11.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.405+0000 7f56b3e00640 1 -- 172.21.15.16:0/2429476758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b41a67b0 con 0x7f56b4109f50 2024-08-31T06:21:11.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.405+0000 7f56ab400640 1 --2- 172.21.15.16:0/2429476758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56b410f890 0x7f56b41a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:21:11.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.405+0000 7f56b3e00640 1 --2- 172.21.15.16:0/2429476758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56b4109f50 0x7f56b41a0840 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f56a400e9f0 tx=0x7f56a400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:11.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.405+0000 7f56b1600640 1 -- 172.21.15.16:0/2429476758 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a400cd80 con 0x7f56b4109f50 2024-08-31T06:21:11.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.406+0000 7f56bac00640 1 -- 172.21.15.16:0/2429476758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56b41a6aa0 con 0x7f56b4109f50 2024-08-31T06:21:11.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.406+0000 7f56b1600640 1 -- 172.21.15.16:0/2429476758 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56a40054d0 con 0x7f56b4109f50 2024-08-31T06:21:11.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.406+0000 7f56bac00640 1 -- 172.21.15.16:0/2429476758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56b41ad200 con 0x7f56b4109f50 2024-08-31T06:21:11.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.406+0000 7f56b1600640 1 -- 172.21.15.16:0/2429476758 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a4010690 con 0x7f56b4109f50 2024-08-31T06:21:11.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.409+0000 7f56c184c740 1 Processor -- start 2024-08-31T06:21:11.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.409+0000 7f56c184c740 1 -- start start 2024-08-31T06:21:11.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.409+0000 7f56c184c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594dd6a8350 0x5594dda4e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:11.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.409+0000 7f56c184c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594dd67f490 0x5594dda4eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:11.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.409+0000 7f56c184c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594dd687670 0x5594dda4f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:11.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.409+0000 7f56c184c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5594dd9c4750 con 0x5594dd6a8350 2024-08-31T06:21:11.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.409+0000 7f56c184c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5594dd9b3ce0 con 0x5594dd67f490 2024-08-31T06:21:11.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.409+0000 7f56c184c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5594dd9b7c90 con 0x5594dd687670 2024-08-31T06:21:11.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.410+0000 7f56b8c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594dd6a8350 0x5594dda4e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:11.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.410+0000 7f56b8c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594dd6a8350 0x5594dda4e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39416/0 (socket says 172.21.15.16:39416) 2024-08-31T06:21:11.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.410+0000 7f56b8c00640 1 -- 172.21.15.16:0/776870267 learned_addr learned my addr 172.21.15.16:0/776870267 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:11.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.410+0000 7f56ab400640 1 --2- 172.21.15.16:0/776870267 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594dd687670 0x5594dda4f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:11.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.410+0000 7f56b1600640 1 -- 172.21.15.16:0/2429476758 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f56a4004e90 con 0x7f56b4109f50 2024-08-31T06:21:11.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.410+0000 7f56b8c00640 1 -- 172.21.15.16:0/776870267 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594dd687670 msgr2=0x5594dda4f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:11.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.410+0000 7f56b8c00640 1 --2- 172.21.15.16:0/776870267 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594dd687670 0x5594dda4f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:11.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.410+0000 7f56b8c00640 1 -- 172.21.15.16:0/776870267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594dd67f490 msgr2=0x5594dda4eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:11.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.410+0000 7f56b8c00640 1 --2- 172.21.15.16:0/776870267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594dd67f490 0x5594dda4eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:11.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.410+0000 7f56b8c00640 1 -- 172.21.15.16:0/776870267 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594dd713930 con 0x5594dd6a8350 2024-08-31T06:21:11.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.411+0000 7f56b8c00640 1 --2- 172.21.15.16:0/776870267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594dd6a8350 0x5594dda4e600 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f56ac009ca0 tx=0x7f56ac009430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d52d4a1bcc277b0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:11.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.411+0000 7f56a9200640 1 -- 172.21.15.16:0/776870267 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac031cf0 con 0x5594dd6a8350 2024-08-31T06:21:11.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.411+0000 7f56a9200640 1 -- 172.21.15.16:0/776870267 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56ac036030 con 0x5594dd6a8350 2024-08-31T06:21:11.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.411+0000 7f56a9200640 1 -- 172.21.15.16:0/776870267 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac0042a0 con 0x5594dd6a8350 2024-08-31T06:21:11.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.412+0000 7f56b1600640 1 --2- 172.21.15.16:0/2429476758 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f56980b4d20 0x7f56980b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:11.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.412+0000 7f56b1600640 1 -- 172.21.15.16:0/2429476758 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8514+0+0 (secure 0 0 0) 0x7f56a40e5aa0 con 0x7f56b4109f50 2024-08-31T06:21:11.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.412+0000 7f56b3e00640 1 --2- 172.21.15.16:0/2429476758 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f56980b4d20 0x7f56980b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:11.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.412+0000 7f56c184c740 1 -- 172.21.15.16:0/776870267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594dd6a8350 msgr2=0x5594dda4e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:11.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.412+0000 7f56c184c740 1 --2- 172.21.15.16:0/776870267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594dd6a8350 0x5594dda4e600 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f56ac009ca0 tx=0x7f56ac009430 comp rx=0 tx=0).stop 2024-08-31T06:21:11.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.412+0000 7f56c184c740 1 -- 172.21.15.16:0/776870267 shutdown_connections 2024-08-31T06:21:11.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.412+0000 7f56c184c740 1 --2- 172.21.15.16:0/776870267 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594dd687670 0x5594dda4f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:11.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.412+0000 7f56c184c740 1 --2- 172.21.15.16:0/776870267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594dd67f490 0x5594dda4eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:11.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.412+0000 7f56c184c740 1 --2- 172.21.15.16:0/776870267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594dd6a8350 0x5594dda4e600 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:11.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.412+0000 7f56c184c740 1 -- 172.21.15.16:0/776870267 >> 172.21.15.16:0/776870267 conn(0x5594dd69c740 msgr2=0x5594dda4c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:11.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.412+0000 7f56c184c740 1 -- 172.21.15.16:0/776870267 shutdown_connections 2024-08-31T06:21:11.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.413+0000 7f56c184c740 1 -- 172.21.15.16:0/776870267 wait complete. 2024-08-31T06:21:11.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.413+0000 7f56b3e00640 1 --2- 172.21.15.16:0/2429476758 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f56980b4d20 0x7f56980b7230 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x5594dda4d280 tx=0x7f56a40cf040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:11.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.413+0000 7f56c184c740 1 -- start start 2024-08-31T06:21:11.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.414+0000 7f56c184c740 1 Processor -- start 2024-08-31T06:21:11.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.414+0000 7f56c184c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594dd67f490 0x5594dda5a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:11.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.414+0000 7f56c184c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594dd687670 0x5594dda5a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:11.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.414+0000 7f56c184c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594dd6a8350 0x5594dda65ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:11.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.414+0000 7f56c184c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5594dd816890 con 0x5594dd6a8350 2024-08-31T06:21:11.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.414+0000 7f56c184c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5594dd93acd0 con 0x5594dd67f490 2024-08-31T06:21:11.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.414+0000 7f56c184c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5594dda5cc00 con 0x5594dd687670 2024-08-31T06:21:11.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.414+0000 7f56c184c740 10 client.0 _ll_register_callbacks cb 0x5594dd9aeef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:21:11.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.415+0000 7f56b8c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594dd67f490 0x5594dda5a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:11.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.415+0000 7f56ab400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594dd687670 0x5594dda5a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:11.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.415+0000 7f56b8c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594dd67f490 0x5594dda5a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:54088/0 (socket says 172.21.15.16:54088) 2024-08-31T06:21:11.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.415+0000 7f56b8c00640 1 -- 172.21.15.16:0/286604292 learned_addr learned my addr 172.21.15.16:0/286604292 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:11.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.415+0000 7f56ab400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594dd687670 0x5594dda5a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:45504/0 (socket says 172.21.15.16:45504) 2024-08-31T06:21:11.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.415+0000 7f56b8c00640 1 --2- 172.21.15.16:0/286604292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594dd6a8350 0x5594dda65ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:11.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.415+0000 7f56ab400640 1 -- 172.21.15.16:0/286604292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594dd67f490 msgr2=0x5594dda5a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:11.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.415+0000 7f56ab400640 1 --2- 172.21.15.16:0/286604292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594dd67f490 0x5594dda5a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:11.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.415+0000 7f56ab400640 1 -- 172.21.15.16:0/286604292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594dd6a8350 msgr2=0x5594dda65ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:11.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.415+0000 7f56ab400640 1 --2- 172.21.15.16:0/286604292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594dd6a8350 0x5594dda65ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:11.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.415+0000 7f56ab400640 1 -- 172.21.15.16:0/286604292 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594dd9841b0 con 0x5594dd687670 2024-08-31T06:21:11.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.416+0000 7f56ab400640 1 --2- 172.21.15.16:0/286604292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594dd687670 0x5594dda5a6f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f56b406c530 tx=0x7f569c012b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c1584cd212ef0f5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:11.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.416+0000 7f56a1600640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:21:11.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.416+0000 7f56a1600640 1 -- 172.21.15.16:0/286604292 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569c004320 con 0x5594dd687670 2024-08-31T06:21:11.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.416+0000 7f56c184c740 1 -- 172.21.15.16:0/286604292 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56ac036030 con 0x5594dd687670 2024-08-31T06:21:11.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.416+0000 7f56a1600640 1 -- 172.21.15.16:0/286604292 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8514+0+0 (secure 0 0 0) 0x7f569c011970 con 0x5594dd687670 2024-08-31T06:21:11.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.416+0000 7f56a1600640 1 -- 172.21.15.16:0/286604292 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f569c0044c0 con 0x5594dd687670 2024-08-31T06:21:11.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.416+0000 7f56c184c740 10 client.28777 fetch_fsmap learned FSMap version 49 2024-08-31T06:21:11.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.416+0000 7f56c184c740 1 -- 172.21.15.16:0/286604292 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5594dda68c10 con 0x5594dd687670 2024-08-31T06:21:11.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.417+0000 7f56a1600640 1 -- 172.21.15.16:0/286604292 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56ac036030 con 0x5594dd687670 2024-08-31T06:21:11.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.417+0000 7f56a1600640 1 -- 172.21.15.16:0/286604292 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f569c025480 con 0x5594dd687670 2024-08-31T06:21:11.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.417+0000 7f56c184c740 10 client.28777 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:21:11.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.417+0000 7f56c184c740 10 client.28777 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:21:11.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.417+0000 7f56c184c740 20 client.28777 populate_metadata read hostname 'smithi016' 2024-08-31T06:21:11.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.417+0000 7f56c184c740 1 --2- 172.21.15.16:0/286604292 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5594dd666cd0 0x5594dda6c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:11.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.417+0000 7f56c184c740 4 client.28777 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:21:11.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.417+0000 7f56c184c740 1 -- 172.21.15.16:0/286604292 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594dd9c66a0 con 0x5594dd666cd0 2024-08-31T06:21:11.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.418+0000 7f56b8c00640 1 --2- 172.21.15.16:0/286604292 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5594dd666cd0 0x5594dda6c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:11.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.418+0000 7f56b8c00640 1 --2- 172.21.15.16:0/286604292 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5594dd666cd0 0x5594dda6c190 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3a03ef74c8e5405 server_cookie=aeffb3376ea4442 in_seq=0 out_seq=0 2024-08-31T06:21:11.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.418+0000 7f56a1600640 10 client.28777 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:21:11.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.419+0000 7f56a1600640 1 -- 172.21.15.16:0/286604292 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5594dd9c66a0 con 0x5594dd666cd0 2024-08-31T06:21:11.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.419+0000 7f56a1600640 10 client.28777 handle_command_reply: tid=0 2024-08-31T06:21:11.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.419+0000 7f56c184c740 1 client.28777 shutdown 2024-08-31T06:21:11.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.419+0000 7f56c184c740 1 -- 172.21.15.16:0/286604292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594dd687670 msgr2=0x5594dda5a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:11.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.419+0000 7f56c184c740 1 --2- 172.21.15.16:0/286604292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594dd687670 0x5594dda5a6f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f56b406c530 tx=0x7f569c012b10 comp rx=0 tx=0).stop 2024-08-31T06:21:11.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.419+0000 7f56c184c740 1 -- 172.21.15.16:0/286604292 shutdown_connections 2024-08-31T06:21:11.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.419+0000 7f56c184c740 1 --2- 172.21.15.16:0/286604292 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5594dd666cd0 0x5594dda6c190 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:11.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.419+0000 7f56c184c740 1 --2- 172.21.15.16:0/286604292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594dd687670 0x5594dda5a6f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:11.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.419+0000 7f56c184c740 1 --2- 172.21.15.16:0/286604292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594dd6a8350 0x5594dda65ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:11.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.419+0000 7f56a1600640 0 client.28777 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:21:11.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.419+0000 7f56c184c740 1 --2- 172.21.15.16:0/286604292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594dd67f490 0x5594dda5a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:11.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.419+0000 7f56c184c740 1 -- 172.21.15.16:0/286604292 >> 172.21.15.16:0/286604292 conn(0x5594dd69c740 msgr2=0x5594dda517e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:11.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.420+0000 7f56c184c740 1 -- 172.21.15.16:0/286604292 shutdown_connections 2024-08-31T06:21:11.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.420+0000 7f56c184c740 1 -- 172.21.15.16:0/286604292 wait complete. 2024-08-31T06:21:11.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.420+0000 7f56c184c740 20 client.28777 trim_cache size 0 max 16384 2024-08-31T06:21:11.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.438+0000 7f56c184c740 1 Processor -- start 2024-08-31T06:21:11.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.438+0000 7f56c184c740 1 -- start start 2024-08-31T06:21:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.439+0000 7f56c184c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594dd67f490 0x5594dda5bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.439+0000 7f56c184c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594dd687670 0x5594dda65df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.439+0000 7f56c184c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594dd6a8350 0x5594dda66330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.439+0000 7f56c184c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f56b41b83e0 con 0x5594dd6a8350 2024-08-31T06:21:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.439+0000 7f56c184c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f56b41b5bf0 con 0x5594dd67f490 2024-08-31T06:21:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.439+0000 7f56c184c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f56b41b3a60 con 0x5594dd687670 2024-08-31T06:21:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.439+0000 7f56ab400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594dd687670 0x5594dda65df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.439+0000 7f56b8c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594dd67f490 0x5594dda5bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.439+0000 7f56ab400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594dd687670 0x5594dda65df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:45516/0 (socket says 172.21.15.16:45516) 2024-08-31T06:21:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.439+0000 7f56ab400640 1 -- 172.21.15.16:0/1906812463 learned_addr learned my addr 172.21.15.16:0/1906812463 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.439+0000 7f56b8c00640 1 --2- 172.21.15.16:0/1906812463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594dd6a8350 0x5594dda66330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:11.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.440+0000 7f56ab400640 1 -- 172.21.15.16:0/1906812463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594dd67f490 msgr2=0x5594dda5bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:11.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.440+0000 7f56ab400640 1 --2- 172.21.15.16:0/1906812463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594dd67f490 0x5594dda5bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:11.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.440+0000 7f56ab400640 1 -- 172.21.15.16:0/1906812463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594dd6a8350 msgr2=0x5594dda66330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:11.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.440+0000 7f56ab400640 1 --2- 172.21.15.16:0/1906812463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594dd6a8350 0x5594dda66330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:11.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.440+0000 7f56ab400640 1 -- 172.21.15.16:0/1906812463 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594dda50920 con 0x5594dd687670 2024-08-31T06:21:11.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.440+0000 7f56ab400640 1 --2- 172.21.15.16:0/1906812463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594dd687670 0x5594dda65df0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f569c001eb0 tx=0x7f569c009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6ef188b1aaef19a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:11.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.440+0000 7f56a3e00640 1 -- 172.21.15.16:0/1906812463 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569c025480 con 0x5594dd687670 2024-08-31T06:21:11.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.440+0000 7f56a3e00640 1 -- 172.21.15.16:0/1906812463 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f569c025620 con 0x5594dd687670 2024-08-31T06:21:11.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.440+0000 7f56a3e00640 1 -- 172.21.15.16:0/1906812463 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569c005260 con 0x5594dd687670 2024-08-31T06:21:11.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.441+0000 7f56c184c740 1 -- 172.21.15.16:0/1906812463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594dd687670 msgr2=0x5594dda65df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:11.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.441+0000 7f56c184c740 1 --2- 172.21.15.16:0/1906812463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594dd687670 0x5594dda65df0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f569c001eb0 tx=0x7f569c009700 comp rx=0 tx=0).stop 2024-08-31T06:21:11.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.441+0000 7f56c184c740 1 -- 172.21.15.16:0/1906812463 shutdown_connections 2024-08-31T06:21:11.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.441+0000 7f56c184c740 1 --2- 172.21.15.16:0/1906812463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594dd687670 0x5594dda65df0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:11.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.441+0000 7f56c184c740 1 --2- 172.21.15.16:0/1906812463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594dd6a8350 0x5594dda66330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:11.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.441+0000 7f56c184c740 1 --2- 172.21.15.16:0/1906812463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594dd67f490 0x5594dda5bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:11.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.441+0000 7f56c184c740 1 -- 172.21.15.16:0/1906812463 >> 172.21.15.16:0/1906812463 conn(0x5594dd69c740 msgr2=0x5594dda5b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:11.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.442+0000 7f56c184c740 1 -- 172.21.15.16:0/1906812463 shutdown_connections 2024-08-31T06:21:11.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.442+0000 7f56c184c740 1 -- 172.21.15.16:0/1906812463 wait complete. 2024-08-31T06:21:11.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.442+0000 7f56c184c740 1 -- start start 2024-08-31T06:21:11.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.442+0000 7f56c184c740 1 Processor -- start 2024-08-31T06:21:11.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.443+0000 7f56c184c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594dd67f490 0x5594dda626a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:11.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.443+0000 7f56c184c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594dd687670 0x5594dda5eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:11.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.443+0000 7f56c184c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594dd6a8350 0x5594dda5f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:11.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.443+0000 7f56c184c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f56b41b40b0 con 0x5594dd6a8350 2024-08-31T06:21:11.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.443+0000 7f56c184c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f56b41b5520 con 0x5594dd687670 2024-08-31T06:21:11.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.443+0000 7f56c184c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f56b41b4780 con 0x5594dd67f490 2024-08-31T06:21:11.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.443+0000 7f56c184c740 10 client.0 _ll_register_callbacks cb 0x7f569c009860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:21:11.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.443+0000 7f56ab400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594dd687670 0x5594dda5eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:11.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.443+0000 7f56ab400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594dd687670 0x5594dda5eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:54106/0 (socket says 172.21.15.16:54106) 2024-08-31T06:21:11.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.443+0000 7f56ab400640 1 -- 172.21.15.16:0/293217516 learned_addr learned my addr 172.21.15.16:0/293217516 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:11.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.443+0000 7f56b8c00640 1 --2- 172.21.15.16:0/293217516 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594dd67f490 0x5594dda626a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:11.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.443+0000 7f56b8c00640 1 --2- 172.21.15.16:0/293217516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594dd6a8350 0x5594dda5f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:11.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.444+0000 7f56ab400640 1 -- 172.21.15.16:0/293217516 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594dd67f490 msgr2=0x5594dda626a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:11.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.444+0000 7f56ab400640 1 --2- 172.21.15.16:0/293217516 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594dd67f490 0x5594dda626a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:11.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.444+0000 7f56ab400640 1 -- 172.21.15.16:0/293217516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594dd6a8350 msgr2=0x5594dda5f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:11.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.444+0000 7f56ab400640 1 --2- 172.21.15.16:0/293217516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594dd6a8350 0x5594dda5f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:11.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.444+0000 7f56ab400640 1 -- 172.21.15.16:0/293217516 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594dda5f890 con 0x5594dd687670 2024-08-31T06:21:11.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.444+0000 7f56b8c00640 1 --2- 172.21.15.16:0/293217516 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594dd67f490 0x5594dda626a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:21:11.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.444+0000 7f56ab400640 1 --2- 172.21.15.16:0/293217516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594dd687670 0x5594dda5eb60 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f569c00a010 tx=0x7f569c005410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c55be295119fc47 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:11.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.444+0000 7f56a1600640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:21:11.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.444+0000 7f56a1600640 1 -- 172.21.15.16:0/293217516 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569c025480 con 0x5594dd687670 2024-08-31T06:21:11.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.444+0000 7f56c184c740 1 -- 172.21.15.16:0/293217516 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f569c025620 con 0x5594dd687670 2024-08-31T06:21:11.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.445+0000 7f56a1600640 1 -- 172.21.15.16:0/293217516 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8514+0+0 (secure 0 0 0) 0x7f569c018040 con 0x5594dd687670 2024-08-31T06:21:11.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.445+0000 7f56a1600640 1 -- 172.21.15.16:0/293217516 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f569c005630 con 0x5594dd687670 2024-08-31T06:21:11.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.445+0000 7f56c184c740 10 client.28760 fetch_fsmap learned FSMap version 49 2024-08-31T06:21:11.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.445+0000 7f56c184c740 1 -- 172.21.15.16:0/293217516 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5594ddad7b50 con 0x5594dd687670 2024-08-31T06:21:11.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.445+0000 7f56a1600640 1 -- 172.21.15.16:0/293217516 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f569c025620 con 0x5594dd687670 2024-08-31T06:21:11.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.445+0000 7f56a1600640 1 -- 172.21.15.16:0/293217516 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f569c037020 con 0x5594dd687670 2024-08-31T06:21:11.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.445+0000 7f56c184c740 10 client.28760 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:21:11.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.445+0000 7f56c184c740 10 client.28760 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:21:11.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.445+0000 7f56c184c740 20 client.28760 populate_metadata read hostname 'smithi016' 2024-08-31T06:21:11.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.445+0000 7f56c184c740 1 --2- 172.21.15.16:0/293217516 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5594dd63c080 0x5594ddada270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:11.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.445+0000 7f56c184c740 4 client.28760 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T06:21:11.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.445+0000 7f56c184c740 1 -- 172.21.15.16:0/293217516 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5594dd9c2140 con 0x5594dd63c080 2024-08-31T06:21:11.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.446+0000 7f56b8c00640 1 --2- 172.21.15.16:0/293217516 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5594dd63c080 0x5594ddada270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:11.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.446+0000 7f56b8c00640 1 --2- 172.21.15.16:0/293217516 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5594dd63c080 0x5594ddada270 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5be34959de71f73 server_cookie=dca28d5275518427 in_seq=0 out_seq=0 2024-08-31T06:21:11.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.446+0000 7f56a1600640 10 client.28760 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:21:11.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.448+0000 7f56a1600640 1 -- 172.21.15.16:0/293217516 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5594dd9c2140 con 0x5594dd63c080 2024-08-31T06:21:11.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.448+0000 7f56a1600640 10 client.28760 handle_command_reply: tid=0 2024-08-31T06:21:11.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.448+0000 7f56c184c740 1 client.28760 shutdown 2024-08-31T06:21:11.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.448+0000 7f56c184c740 1 -- 172.21.15.16:0/293217516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594dd687670 msgr2=0x5594dda5eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:11.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.448+0000 7f56c184c740 1 --2- 172.21.15.16:0/293217516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594dd687670 0x5594dda5eb60 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f569c00a010 tx=0x7f569c005410 comp rx=0 tx=0).stop 2024-08-31T06:21:11.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.448+0000 7f56c184c740 1 -- 172.21.15.16:0/293217516 shutdown_connections 2024-08-31T06:21:11.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.448+0000 7f56c184c740 1 --2- 172.21.15.16:0/293217516 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5594dd63c080 0x5594ddada270 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:11.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.448+0000 7f56c184c740 1 --2- 172.21.15.16:0/293217516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594dd6a8350 0x5594dda5f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:11.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.448+0000 7f56c184c740 1 --2- 172.21.15.16:0/293217516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594dd687670 0x5594dda5eb60 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:11.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.448+0000 7f56a1600640 0 client.28760 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:21:11.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.448+0000 7f56c184c740 1 --2- 172.21.15.16:0/293217516 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594dd67f490 0x5594dda626a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:11.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.448+0000 7f56c184c740 1 -- 172.21.15.16:0/293217516 >> 172.21.15.16:0/293217516 conn(0x5594dd69c740 msgr2=0x5594dda4a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:11.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.448+0000 7f56c184c740 1 -- 172.21.15.16:0/293217516 shutdown_connections 2024-08-31T06:21:11.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.449+0000 7f56c184c740 1 -- 172.21.15.16:0/293217516 wait complete. 2024-08-31T06:21:11.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.449+0000 7f56c184c740 20 client.28760 trim_cache size 0 max 16384 2024-08-31T06:21:11.452 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:21:11.452 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-08-31T06:21:11.452 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "0015da36-6634-47c5-b8d4-219f9eba8250", 2024-08-31T06:21:11.452 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-08-31T06:21:11.453 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:21:11.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.449+0000 7f56bac00640 1 -- 172.21.15.16:0/2429476758 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f56980b4d20 msgr2=0x7f56980b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:11.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.449+0000 7f56bac00640 1 --2- 172.21.15.16:0/2429476758 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f56980b4d20 0x7f56980b7230 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x5594dda4d280 tx=0x7f56a40cf040 comp rx=0 tx=0).stop 2024-08-31T06:21:11.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.449+0000 7f56bac00640 1 -- 172.21.15.16:0/2429476758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56b4109f50 msgr2=0x7f56b41a0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:11.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.449+0000 7f56bac00640 1 --2- 172.21.15.16:0/2429476758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56b4109f50 0x7f56b41a0840 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f56a400e9f0 tx=0x7f56a400edf0 comp rx=0 tx=0).stop 2024-08-31T06:21:11.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.450+0000 7f56bac00640 1 -- 172.21.15.16:0/2429476758 shutdown_connections 2024-08-31T06:21:11.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.450+0000 7f56bac00640 1 --2- 172.21.15.16:0/2429476758 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f56980b4d20 0x7f56980b7230 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:11.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.450+0000 7f56bac00640 1 --2- 172.21.15.16:0/2429476758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56b41101e0 0x7f56b41a6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:11.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.450+0000 7f56bac00640 1 --2- 172.21.15.16:0/2429476758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56b410f890 0x7f56b41a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:11.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.450+0000 7f56bac00640 1 --2- 172.21.15.16:0/2429476758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56b4109f50 0x7f56b41a0840 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:11.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.450+0000 7f56bac00640 1 -- 172.21.15.16:0/2429476758 >> 172.21.15.16:0/2429476758 conn(0x7f56b4077e80 msgr2=0x7f56b40784e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:11.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.450+0000 7f56bac00640 1 -- 172.21.15.16:0/2429476758 shutdown_connections 2024-08-31T06:21:11.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:11.450+0000 7f56bac00640 1 -- 172.21.15.16:0/2429476758 wait complete. 2024-08-31T06:21:11.626 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 825.0 s, 869.3 tps, lat 73.243 ms stddev 121.904 2024-08-31T06:21:11.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:11 smithi016 ceph-mon[17194]: scrub queued for path: / 2024-08-31T06:21:11.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:11 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:21:11.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:11 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:21:11.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:11 smithi155 ceph-mon[21742]: scrub queued for path: / 2024-08-31T06:21:11.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:11 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:21:11.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:11 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:21:11.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:11 smithi184 ceph-mon[22331]: scrub queued for path: / 2024-08-31T06:21:11.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:11 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:21:11.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:11 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:21:12.476 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T06:21:12.860 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:21:12.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:12 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:21:12.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:12 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:21:12.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:12 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:21:12.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:12 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:21:12.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:12 smithi155 ceph-mon[21742]: pgmap v800: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 18 MiB/s wr, 1.61k op/s 2024-08-31T06:21:12.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:12 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:21:12.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:12 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:21:12.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:12 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:21:12.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:12 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:21:12.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:12 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:21:12.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:12 smithi016 ceph-mon[17194]: pgmap v800: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 18 MiB/s wr, 1.61k op/s 2024-08-31T06:21:12.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:12 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:21:12.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:12 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:21:12.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:12 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:21:12.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:12 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:21:12.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:12 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:21:12.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:12 smithi184 ceph-mon[22331]: pgmap v800: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 18 MiB/s wr, 1.61k op/s 2024-08-31T06:21:12.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:12 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:21:13.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.865+0000 7f26b9200640 1 -- 172.21.15.16:0/3042556643 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26b4102230 msgr2=0x7f26b4102640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:13.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.865+0000 7f26b9200640 1 --2- 172.21.15.16:0/3042556643 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26b4102230 0x7f26b4102640 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f26a4009860 tx=0x7f26a402a3d0 comp rx=0 tx=0).stop 2024-08-31T06:21:13.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.866+0000 7f26b9200640 1 -- 172.21.15.16:0/3042556643 shutdown_connections 2024-08-31T06:21:13.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.866+0000 7f26b9200640 1 --2- 172.21.15.16:0/3042556643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26b410fde0 0x7f26b4112200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:13.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.866+0000 7f26b9200640 1 --2- 172.21.15.16:0/3042556643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26b4102b80 0x7f26b410f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:13.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.866+0000 7f26b9200640 1 --2- 172.21.15.16:0/3042556643 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26b4102230 0x7f26b4102640 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:13.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.866+0000 7f26b9200640 1 -- 172.21.15.16:0/3042556643 >> 172.21.15.16:0/3042556643 conn(0x7f26b40fe0d0 msgr2=0x7f26b4100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:13.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.866+0000 7f26b9200640 1 -- 172.21.15.16:0/3042556643 shutdown_connections 2024-08-31T06:21:13.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.866+0000 7f26b9200640 1 -- 172.21.15.16:0/3042556643 wait complete. 2024-08-31T06:21:13.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.867+0000 7f26b9200640 1 Processor -- start 2024-08-31T06:21:13.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.867+0000 7f26b9200640 1 -- start start 2024-08-31T06:21:13.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.867+0000 7f26b9200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26b4102230 0x7f26b419c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:13.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.867+0000 7f26b9200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26b4102b80 0x7f26b419c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:13.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.867+0000 7f26b9200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26b410fde0 0x7f26b41a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:13.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.868+0000 7f26b9200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f26b41158d0 con 0x7f26b4102b80 2024-08-31T06:21:13.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.868+0000 7f26b9200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f26b4115750 con 0x7f26b410fde0 2024-08-31T06:21:13.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.868+0000 7f26b9200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f26b4115a50 con 0x7f26b4102230 2024-08-31T06:21:13.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.868+0000 7f26b3400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26b4102230 0x7f26b419c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:13.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.868+0000 7f26b2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26b4102b80 0x7f26b419c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:13.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.868+0000 7f26b3400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26b4102230 0x7f26b419c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:45556/0 (socket says 172.21.15.16:45556) 2024-08-31T06:21:13.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.868+0000 7f26b3400640 1 -- 172.21.15.16:0/3238904417 learned_addr learned my addr 172.21.15.16:0/3238904417 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:13.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.868+0000 7f26b3e00640 1 --2- 172.21.15.16:0/3238904417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26b410fde0 0x7f26b41a1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:13.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.868+0000 7f26b3400640 1 -- 172.21.15.16:0/3238904417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26b410fde0 msgr2=0x7f26b41a1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:13.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.868+0000 7f26b3400640 1 --2- 172.21.15.16:0/3238904417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26b410fde0 0x7f26b41a1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:13.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.868+0000 7f26b3400640 1 -- 172.21.15.16:0/3238904417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26b4102b80 msgr2=0x7f26b419c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:13.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.868+0000 7f26b3400640 1 --2- 172.21.15.16:0/3238904417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26b4102b80 0x7f26b419c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:13.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.868+0000 7f26b3400640 1 -- 172.21.15.16:0/3238904417 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26b41a21b0 con 0x7f26b4102230 2024-08-31T06:21:13.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.869+0000 7f26b3400640 1 --2- 172.21.15.16:0/3238904417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26b4102230 0x7f26b419c240 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f26a4005940 tx=0x7f26a4030fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:13.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.869+0000 7f26abe00640 1 -- 172.21.15.16:0/3238904417 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a4040070 con 0x7f26b4102230 2024-08-31T06:21:13.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.869+0000 7f26b9200640 1 -- 172.21.15.16:0/3238904417 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26b41a2440 con 0x7f26b4102230 2024-08-31T06:21:13.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.869+0000 7f26b9200640 1 -- 172.21.15.16:0/3238904417 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26b41a8c00 con 0x7f26b4102230 2024-08-31T06:21:13.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.869+0000 7f26abe00640 1 -- 172.21.15.16:0/3238904417 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26a4005570 con 0x7f26b4102230 2024-08-31T06:21:13.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.869+0000 7f26abe00640 1 -- 172.21.15.16:0/3238904417 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a404a060 con 0x7f26b4102230 2024-08-31T06:21:13.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.872+0000 7f26abe00640 1 -- 172.21.15.16:0/3238904417 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f26a4043070 con 0x7f26b4102230 2024-08-31T06:21:13.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.873+0000 7f26c1461740 1 Processor -- start 2024-08-31T06:21:13.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.873+0000 7f26c1461740 1 -- start start 2024-08-31T06:21:13.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.873+0000 7f26c1461740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632cb497070 0x5632cb83ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:13.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.873+0000 7f26c1461740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cb46e1b0 0x5632cb83cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:13.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.873+0000 7f26c1461740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632cb476390 0x5632cb83d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:13.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.873+0000 7f26c1461740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5632cb7b3770 con 0x5632cb46e1b0 2024-08-31T06:21:13.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.873+0000 7f26c1461740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5632cb7a2d00 con 0x5632cb476390 2024-08-31T06:21:13.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.873+0000 7f26c1461740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5632cb7a6cb0 con 0x5632cb497070 2024-08-31T06:21:13.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.874+0000 7f26b3e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632cb497070 0x5632cb83ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:13.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.874+0000 7f26b3e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632cb497070 0x5632cb83ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:45562/0 (socket says 172.21.15.16:45562) 2024-08-31T06:21:13.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.874+0000 7f26b3e00640 1 -- 172.21.15.16:0/2994051239 learned_addr learned my addr 172.21.15.16:0/2994051239 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:13.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.874+0000 7f26b2a00640 1 --2- 172.21.15.16:0/2994051239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632cb476390 0x5632cb83d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:13.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.874+0000 7f26b3400640 1 --2- 172.21.15.16:0/2994051239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cb46e1b0 0x5632cb83cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:13.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.874+0000 7f26b3e00640 1 -- 172.21.15.16:0/2994051239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632cb476390 msgr2=0x5632cb83d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:13.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.874+0000 7f26b3e00640 1 --2- 172.21.15.16:0/2994051239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632cb476390 0x5632cb83d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:13.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.874+0000 7f26b3e00640 1 -- 172.21.15.16:0/2994051239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cb46e1b0 msgr2=0x5632cb83cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:13.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.874+0000 7f26b3e00640 1 --2- 172.21.15.16:0/2994051239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cb46e1b0 0x5632cb83cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:13.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.874+0000 7f26b3e00640 1 -- 172.21.15.16:0/2994051239 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632cb502650 con 0x5632cb497070 2024-08-31T06:21:13.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.874+0000 7f26b3400640 1 --2- 172.21.15.16:0/2994051239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cb46e1b0 0x5632cb83cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:21:13.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.874+0000 7f26abe00640 1 --2- 172.21.15.16:0/3238904417 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f26980b4d20 0x7f26980b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:13.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.875+0000 7f26b3e00640 1 --2- 172.21.15.16:0/2994051239 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632cb497070 0x5632cb83ca30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f26ac00e450 tx=0x7f26ac0095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5feb54c726a5c14 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:13.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.875+0000 7f26abe00640 1 -- 172.21.15.16:0/3238904417 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8514+0+0 (secure 0 0 0) 0x7f26a4105cd0 con 0x7f26b4102230 2024-08-31T06:21:13.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.875+0000 7f26b3400640 1 --2- 172.21.15.16:0/3238904417 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f26980b4d20 0x7f26980b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:13.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.875+0000 7f26a8c00640 1 -- 172.21.15.16:0/2994051239 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac00d910 con 0x5632cb497070 2024-08-31T06:21:13.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.875+0000 7f26a8c00640 1 -- 172.21.15.16:0/2994051239 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26ac01c070 con 0x5632cb497070 2024-08-31T06:21:13.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.875+0000 7f26a8c00640 1 -- 172.21.15.16:0/2994051239 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac017e50 con 0x5632cb497070 2024-08-31T06:21:13.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.876+0000 7f26c1461740 1 -- 172.21.15.16:0/2994051239 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632cb497070 msgr2=0x5632cb83ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:13.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.876+0000 7f26c1461740 1 --2- 172.21.15.16:0/2994051239 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632cb497070 0x5632cb83ca30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f26ac00e450 tx=0x7f26ac0095c0 comp rx=0 tx=0).stop 2024-08-31T06:21:13.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.876+0000 7f26b3400640 1 --2- 172.21.15.16:0/3238904417 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f26980b4d20 0x7f26980b7230 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x5632cb83b6b0 tx=0x7f26a40316c0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:13.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.876+0000 7f26c1461740 1 -- 172.21.15.16:0/2994051239 shutdown_connections 2024-08-31T06:21:13.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.876+0000 7f26c1461740 1 --2- 172.21.15.16:0/2994051239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632cb476390 0x5632cb83d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:13.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.876+0000 7f26c1461740 1 --2- 172.21.15.16:0/2994051239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cb46e1b0 0x5632cb83cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:13.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.876+0000 7f26c1461740 1 --2- 172.21.15.16:0/2994051239 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632cb497070 0x5632cb83ca30 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:13.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.876+0000 7f26c1461740 1 -- 172.21.15.16:0/2994051239 >> 172.21.15.16:0/2994051239 conn(0x5632cb48b460 msgr2=0x5632cb83a920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:13.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.876+0000 7f26c1461740 1 -- 172.21.15.16:0/2994051239 shutdown_connections 2024-08-31T06:21:13.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.876+0000 7f26c1461740 1 -- 172.21.15.16:0/2994051239 wait complete. 2024-08-31T06:21:13.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.877+0000 7f26c1461740 1 -- start start 2024-08-31T06:21:13.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.877+0000 7f26c1461740 1 Processor -- start 2024-08-31T06:21:13.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.877+0000 7f26c1461740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632cb46e1b0 0x5632cb848650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:13.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.877+0000 7f26c1461740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632cb476390 0x5632cb848b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:13.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.878+0000 7f26c1461740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cb497070 0x5632cb854530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:13.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.878+0000 7f26c1461740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5632cb605700 con 0x5632cb497070 2024-08-31T06:21:13.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.878+0000 7f26c1461740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5632cb729cb0 con 0x5632cb46e1b0 2024-08-31T06:21:13.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.878+0000 7f26c1461740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5632cb84b210 con 0x5632cb476390 2024-08-31T06:21:13.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.878+0000 7f26c1461740 10 client.0 _ll_register_callbacks cb 0x5632cb74f430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:21:13.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.878+0000 7f26b3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cb497070 0x5632cb854530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:13.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.878+0000 7f26b3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cb497070 0x5632cb854530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39464/0 (socket says 172.21.15.16:39464) 2024-08-31T06:21:13.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.878+0000 7f26b3e00640 1 -- 172.21.15.16:0/98289398 learned_addr learned my addr 172.21.15.16:0/98289398 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:13.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.878+0000 7f26b2a00640 1 --2- 172.21.15.16:0/98289398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632cb476390 0x5632cb848b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:13.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.878+0000 7f26b3e00640 1 --2- 172.21.15.16:0/98289398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632cb46e1b0 0x5632cb848650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:13.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.878+0000 7f26b2a00640 1 -- 172.21.15.16:0/98289398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632cb46e1b0 msgr2=0x5632cb848650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:13.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.878+0000 7f26b2a00640 1 --2- 172.21.15.16:0/98289398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632cb46e1b0 0x5632cb848650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:13.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.878+0000 7f26b2a00640 1 -- 172.21.15.16:0/98289398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cb497070 msgr2=0x5632cb854530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:13.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.878+0000 7f26b2a00640 1 --2- 172.21.15.16:0/98289398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cb497070 0x5632cb854530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:13.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.878+0000 7f26b2a00640 1 -- 172.21.15.16:0/98289398 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632cb76a870 con 0x5632cb476390 2024-08-31T06:21:13.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.878+0000 7f26b3e00640 1 --2- 172.21.15.16:0/98289398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632cb46e1b0 0x5632cb848650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:21:13.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.879+0000 7f26b2a00640 1 --2- 172.21.15.16:0/98289398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632cb476390 0x5632cb848b90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f26a00126d0 tx=0x7f26a0012b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c96ee6ecfa3d669e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:13.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.879+0000 7f269d600640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:21:13.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.879+0000 7f269d600640 1 -- 172.21.15.16:0/98289398 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a000f920 con 0x5632cb476390 2024-08-31T06:21:13.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.879+0000 7f26c1461740 1 -- 172.21.15.16:0/98289398 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26ac01c070 con 0x5632cb476390 2024-08-31T06:21:13.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.879+0000 7f269d600640 1 -- 172.21.15.16:0/98289398 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8514+0+0 (secure 0 0 0) 0x7f26a000fac0 con 0x5632cb476390 2024-08-31T06:21:13.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.879+0000 7f269d600640 1 -- 172.21.15.16:0/98289398 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f26a000fcc0 con 0x5632cb476390 2024-08-31T06:21:13.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.879+0000 7f26c1461740 10 client.28795 fetch_fsmap learned FSMap version 49 2024-08-31T06:21:13.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.879+0000 7f26c1461740 1 -- 172.21.15.16:0/98289398 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5632cb8572c0 con 0x5632cb476390 2024-08-31T06:21:13.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.880+0000 7f269d600640 1 -- 172.21.15.16:0/98289398 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26ac01c070 con 0x5632cb476390 2024-08-31T06:21:13.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.880+0000 7f269d600640 1 -- 172.21.15.16:0/98289398 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f26a001c420 con 0x5632cb476390 2024-08-31T06:21:13.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.880+0000 7f26c1461740 10 client.28795 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:21:13.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.880+0000 7f26c1461740 10 client.28795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:21:13.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.880+0000 7f26c1461740 20 client.28795 populate_metadata read hostname 'smithi016' 2024-08-31T06:21:13.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.880+0000 7f26c1461740 1 --2- 172.21.15.16:0/98289398 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5632cb4559f0 0x5632cb85a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:13.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.880+0000 7f26c1461740 4 client.28795 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:21:13.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.880+0000 7f26c1461740 1 -- 172.21.15.16:0/98289398 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632cb7b56c0 con 0x5632cb4559f0 2024-08-31T06:21:13.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.881+0000 7f26b3e00640 1 --2- 172.21.15.16:0/98289398 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5632cb4559f0 0x5632cb85a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:13.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.881+0000 7f26b3e00640 1 --2- 172.21.15.16:0/98289398 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5632cb4559f0 0x5632cb85a880 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac71180fb803a10d server_cookie=3d039d837ca69465 in_seq=0 out_seq=0 2024-08-31T06:21:13.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.881+0000 7f269d600640 10 client.28795 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:21:13.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.882+0000 7f269d600640 1 -- 172.21.15.16:0/98289398 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5632cb7b56c0 con 0x5632cb4559f0 2024-08-31T06:21:13.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.882+0000 7f269d600640 10 client.28795 handle_command_reply: tid=0 2024-08-31T06:21:13.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.882+0000 7f26c1461740 1 client.28795 shutdown 2024-08-31T06:21:13.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.883+0000 7f26c1461740 1 -- 172.21.15.16:0/98289398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632cb476390 msgr2=0x5632cb848b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:13.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.883+0000 7f26c1461740 1 --2- 172.21.15.16:0/98289398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632cb476390 0x5632cb848b90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f26a00126d0 tx=0x7f26a0012b90 comp rx=0 tx=0).stop 2024-08-31T06:21:13.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.883+0000 7f26c1461740 1 -- 172.21.15.16:0/98289398 shutdown_connections 2024-08-31T06:21:13.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.883+0000 7f26c1461740 1 --2- 172.21.15.16:0/98289398 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5632cb4559f0 0x5632cb85a880 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:13.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.883+0000 7f26c1461740 1 --2- 172.21.15.16:0/98289398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632cb476390 0x5632cb848b90 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:13.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.883+0000 7f26c1461740 1 --2- 172.21.15.16:0/98289398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cb497070 0x5632cb854530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:13.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.883+0000 7f26c1461740 1 --2- 172.21.15.16:0/98289398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632cb46e1b0 0x5632cb848650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:13.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.883+0000 7f269d600640 0 client.28795 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:21:13.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.883+0000 7f26c1461740 1 -- 172.21.15.16:0/98289398 >> 172.21.15.16:0/98289398 conn(0x5632cb48b460 msgr2=0x5632cb83fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:13.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.883+0000 7f26c1461740 1 -- 172.21.15.16:0/98289398 shutdown_connections 2024-08-31T06:21:13.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.883+0000 7f26c1461740 1 -- 172.21.15.16:0/98289398 wait complete. 2024-08-31T06:21:13.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.883+0000 7f26c1461740 20 client.28795 trim_cache size 0 max 16384 2024-08-31T06:21:13.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.901+0000 7f26c1461740 1 Processor -- start 2024-08-31T06:21:13.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.902+0000 7f26c1461740 1 -- start start 2024-08-31T06:21:13.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.902+0000 7f26c1461740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632cb46e1b0 0x5632cb84a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:13.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.902+0000 7f26c1461740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cb476390 0x5632cb854430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:13.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.902+0000 7f26c1461740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632cb497070 0x5632cb854970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:13.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.902+0000 7f26c1461740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f26b41b3e90 con 0x5632cb476390 2024-08-31T06:21:13.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.902+0000 7f26c1461740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f26b41b16a0 con 0x5632cb497070 2024-08-31T06:21:13.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.902+0000 7f26c1461740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f26b41af510 con 0x5632cb46e1b0 2024-08-31T06:21:13.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.902+0000 7f26b2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cb476390 0x5632cb854430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:13.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.902+0000 7f26b2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cb476390 0x5632cb854430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39472/0 (socket says 172.21.15.16:39472) 2024-08-31T06:21:13.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.902+0000 7f26b2a00640 1 -- 172.21.15.16:0/1049132805 learned_addr learned my addr 172.21.15.16:0/1049132805 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:13.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.902+0000 7f26b3e00640 1 --2- 172.21.15.16:0/1049132805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632cb46e1b0 0x5632cb84a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:13.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.903+0000 7f26b3e00640 1 --2- 172.21.15.16:0/1049132805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632cb497070 0x5632cb854970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:13.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.903+0000 7f26b2a00640 1 -- 172.21.15.16:0/1049132805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632cb46e1b0 msgr2=0x5632cb84a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:13.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.903+0000 7f26b2a00640 1 --2- 172.21.15.16:0/1049132805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632cb46e1b0 0x5632cb84a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:13.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.903+0000 7f26b2a00640 1 -- 172.21.15.16:0/1049132805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632cb497070 msgr2=0x5632cb854970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:13.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.903+0000 7f26b2a00640 1 --2- 172.21.15.16:0/1049132805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632cb497070 0x5632cb854970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:13.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.903+0000 7f26b2a00640 1 -- 172.21.15.16:0/1049132805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632cb849710 con 0x5632cb476390 2024-08-31T06:21:13.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.903+0000 7f26b2a00640 1 --2- 172.21.15.16:0/1049132805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cb476390 0x5632cb854430 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f26a0000c00 tx=0x7f26a0005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed6099630c6dc53e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:13.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.903+0000 7f269fe00640 1 -- 172.21.15.16:0/1049132805 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a0009e90 con 0x5632cb476390 2024-08-31T06:21:13.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.903+0000 7f269fe00640 1 -- 172.21.15.16:0/1049132805 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26a000f920 con 0x5632cb476390 2024-08-31T06:21:13.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.903+0000 7f269fe00640 1 -- 172.21.15.16:0/1049132805 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a00035f0 con 0x5632cb476390 2024-08-31T06:21:13.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.904+0000 7f26c1461740 1 -- 172.21.15.16:0/1049132805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cb476390 msgr2=0x5632cb854430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:13.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.904+0000 7f26c1461740 1 --2- 172.21.15.16:0/1049132805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cb476390 0x5632cb854430 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f26a0000c00 tx=0x7f26a0005870 comp rx=0 tx=0).stop 2024-08-31T06:21:13.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.904+0000 7f26c1461740 1 -- 172.21.15.16:0/1049132805 shutdown_connections 2024-08-31T06:21:13.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.904+0000 7f26c1461740 1 --2- 172.21.15.16:0/1049132805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632cb497070 0x5632cb854970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:13.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.904+0000 7f26c1461740 1 --2- 172.21.15.16:0/1049132805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cb476390 0x5632cb854430 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:13.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.904+0000 7f26c1461740 1 --2- 172.21.15.16:0/1049132805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632cb46e1b0 0x5632cb84a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:13.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.904+0000 7f26c1461740 1 -- 172.21.15.16:0/1049132805 >> 172.21.15.16:0/1049132805 conn(0x5632cb48b460 msgr2=0x5632cb849970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:13.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.904+0000 7f26c1461740 1 -- 172.21.15.16:0/1049132805 shutdown_connections 2024-08-31T06:21:13.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.904+0000 7f26c1461740 1 -- 172.21.15.16:0/1049132805 wait complete. 2024-08-31T06:21:13.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.905+0000 7f26c1461740 1 -- start start 2024-08-31T06:21:13.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.905+0000 7f26c1461740 1 Processor -- start 2024-08-31T06:21:13.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.905+0000 7f26c1461740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632cb46e1b0 0x5632cb84cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:13.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.905+0000 7f26c1461740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cb476390 0x5632cb84d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:13.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.905+0000 7f26c1461740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632cb497070 0x5632cb8be310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:13.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.905+0000 7f26c1461740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f26b41afb60 con 0x5632cb476390 2024-08-31T06:21:13.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.905+0000 7f26c1461740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f26b41b0fd0 con 0x5632cb46e1b0 2024-08-31T06:21:13.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.905+0000 7f26c1461740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f26b41b0230 con 0x5632cb497070 2024-08-31T06:21:13.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.906+0000 7f26c1461740 10 client.0 _ll_register_callbacks cb 0x7f26a0009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:21:13.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.906+0000 7f26b2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cb476390 0x5632cb84d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:13.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.906+0000 7f26b2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cb476390 0x5632cb84d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39478/0 (socket says 172.21.15.16:39478) 2024-08-31T06:21:13.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.906+0000 7f26b2a00640 1 -- 172.21.15.16:0/3576768875 learned_addr learned my addr 172.21.15.16:0/3576768875 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:13.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.906+0000 7f26b3e00640 1 --2- 172.21.15.16:0/3576768875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632cb46e1b0 0x5632cb84cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:13.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.906+0000 7f26b3e00640 1 --2- 172.21.15.16:0/3576768875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632cb497070 0x5632cb8be310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:13.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.906+0000 7f26b2a00640 1 -- 172.21.15.16:0/3576768875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632cb497070 msgr2=0x5632cb8be310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:13.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.906+0000 7f26b2a00640 1 --2- 172.21.15.16:0/3576768875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632cb497070 0x5632cb8be310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:13.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.906+0000 7f26b2a00640 1 -- 172.21.15.16:0/3576768875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632cb46e1b0 msgr2=0x5632cb84cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:13.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.906+0000 7f26b2a00640 1 --2- 172.21.15.16:0/3576768875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632cb46e1b0 0x5632cb84cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:13.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.906+0000 7f26b2a00640 1 -- 172.21.15.16:0/3576768875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632cb8be980 con 0x5632cb476390 2024-08-31T06:21:13.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.907+0000 7f26b2a00640 1 --2- 172.21.15.16:0/3576768875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cb476390 0x5632cb84d490 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f26a0005fe0 tx=0x7f26a001c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1dd0a97e6534983 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:13.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.907+0000 7f269d600640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:21:13.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.907+0000 7f269d600640 1 -- 172.21.15.16:0/3576768875 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a0005290 con 0x5632cb476390 2024-08-31T06:21:13.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.907+0000 7f269d600640 1 -- 172.21.15.16:0/3576768875 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8514+0+0 (secure 0 0 0) 0x7f26a0005430 con 0x5632cb476390 2024-08-31T06:21:13.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.907+0000 7f26c1461740 1 -- 172.21.15.16:0/3576768875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26a000f920 con 0x5632cb476390 2024-08-31T06:21:13.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.907+0000 7f269d600640 1 -- 172.21.15.16:0/3576768875 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f26a00035f0 con 0x5632cb476390 2024-08-31T06:21:13.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.907+0000 7f26c1461740 10 client.18300 fetch_fsmap learned FSMap version 49 2024-08-31T06:21:13.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.907+0000 7f26c1461740 1 -- 172.21.15.16:0/3576768875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5632cb8c5d60 con 0x5632cb476390 2024-08-31T06:21:13.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.907+0000 7f269d600640 1 -- 172.21.15.16:0/3576768875 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26a000f920 con 0x5632cb476390 2024-08-31T06:21:13.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.908+0000 7f269d600640 1 -- 172.21.15.16:0/3576768875 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f26a002b020 con 0x5632cb476390 2024-08-31T06:21:13.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.908+0000 7f26c1461740 10 client.18300 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:21:13.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.908+0000 7f26c1461740 10 client.18300 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:21:13.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.908+0000 7f26c1461740 20 client.18300 populate_metadata read hostname 'smithi016' 2024-08-31T06:21:13.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.908+0000 7f26c1461740 1 --2- 172.21.15.16:0/3576768875 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5632cb42ada0 0x5632cb8c83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:13.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.908+0000 7f26c1461740 4 client.18300 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T06:21:13.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.908+0000 7f26c1461740 1 -- 172.21.15.16:0/3576768875 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5632cb7b1160 con 0x5632cb42ada0 2024-08-31T06:21:13.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.908+0000 7f26b3e00640 1 --2- 172.21.15.16:0/3576768875 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5632cb42ada0 0x5632cb8c83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:13.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.909+0000 7f26b3e00640 1 --2- 172.21.15.16:0/3576768875 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5632cb42ada0 0x5632cb8c83e0 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81aafc237a755d50 server_cookie=ca4dd34ccda2999d in_seq=0 out_seq=0 2024-08-31T06:21:13.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.909+0000 7f269d600640 10 client.18300 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:21:13.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.909+0000 7f269d600640 1 -- 172.21.15.16:0/3576768875 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5398 (crc 0 0 0) 0x5632cb7b1160 con 0x5632cb42ada0 2024-08-31T06:21:13.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.909+0000 7f269d600640 10 client.18300 handle_command_reply: tid=0 2024-08-31T06:21:13.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.909+0000 7f26c1461740 1 client.18300 shutdown 2024-08-31T06:21:13.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.909+0000 7f26c1461740 1 -- 172.21.15.16:0/3576768875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cb476390 msgr2=0x5632cb84d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:13.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.909+0000 7f26c1461740 1 --2- 172.21.15.16:0/3576768875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cb476390 0x5632cb84d490 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f26a0005fe0 tx=0x7f26a001c920 comp rx=0 tx=0).stop 2024-08-31T06:21:13.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.909+0000 7f26c1461740 1 -- 172.21.15.16:0/3576768875 shutdown_connections 2024-08-31T06:21:13.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.909+0000 7f26c1461740 1 --2- 172.21.15.16:0/3576768875 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5632cb42ada0 0x5632cb8c83e0 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:13.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.909+0000 7f26c1461740 1 --2- 172.21.15.16:0/3576768875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632cb497070 0x5632cb8be310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:13.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.909+0000 7f26c1461740 1 --2- 172.21.15.16:0/3576768875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cb476390 0x5632cb84d490 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:13.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.909+0000 7f26c1461740 1 --2- 172.21.15.16:0/3576768875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632cb46e1b0 0x5632cb84cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:13.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.909+0000 7f269d600640 0 client.18300 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:21:13.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.909+0000 7f26c1461740 1 -- 172.21.15.16:0/3576768875 >> 172.21.15.16:0/3576768875 conn(0x5632cb48b460 msgr2=0x5632cb8498e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:13.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.910+0000 7f26c1461740 1 -- 172.21.15.16:0/3576768875 shutdown_connections 2024-08-31T06:21:13.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.910+0000 7f26c1461740 1 -- 172.21.15.16:0/3576768875 wait complete. 2024-08-31T06:21:13.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.910+0000 7f26c1461740 20 client.18300 trim_cache size 0 max 16384 2024-08-31T06:21:13.915 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:21:13.916 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T06:21:13.916 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-08-31T06:21:13.916 INFO:teuthology.orchestra.run.smithi016.stdout: "0015da36-6634-47c5-b8d4-219f9eba8250": { 2024-08-31T06:21:13.916 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:21:13.916 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "0015da36-6634-47c5-b8d4-219f9eba8250", 2024-08-31T06:21:13.916 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-08-31T06:21:13.916 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:21:13.916 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:21:13.916 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-08-31T06:21:13.916 INFO:teuthology.orchestra.run.smithi016.stdout: "18e112f4-a5f9-4602-aaad-3b13fa5a4c2e": { 2024-08-31T06:21:13.916 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:20:16.570535+0000", 2024-08-31T06:21:13.916 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:21:13.916 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:21:13.917 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:21:13.917 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:21:13.917 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:21:13.917 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:21:13.917 INFO:teuthology.orchestra.run.smithi016.stdout: "f1cdcb27-e3e4-4dc8-a0ae-7ace761d04c5": { 2024-08-31T06:21:13.917 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:19:22.275683+0000", 2024-08-31T06:21:13.917 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:21:13.917 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:21:13.917 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:21:13.917 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:21:13.917 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:21:13.917 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:21:13.918 INFO:teuthology.orchestra.run.smithi016.stdout: "0e44eb18-439c-46e4-a298-7d4d2ecd0b4a": { 2024-08-31T06:21:13.918 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:18:31.130765+0000", 2024-08-31T06:21:13.918 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:21:13.918 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:21:13.918 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:21:13.918 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:21:13.918 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:21:13.918 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:21:13.918 INFO:teuthology.orchestra.run.smithi016.stdout: "e497cd88-1e7c-4631-a4a8-cdd8e1fa6c83": { 2024-08-31T06:21:13.918 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:17:42.476795+0000", 2024-08-31T06:21:13.918 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:21:13.918 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:21:13.918 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:21:13.919 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:21:13.919 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:21:13.919 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:21:13.919 INFO:teuthology.orchestra.run.smithi016.stdout: "c300b7e8-f3fd-482c-a909-9329bc61e570": { 2024-08-31T06:21:13.919 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:06:49.868046+0000", 2024-08-31T06:21:13.919 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:21:13.919 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:21:13.919 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:21:13.919 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:21:13.919 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:21:13.919 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:21:13.919 INFO:teuthology.orchestra.run.smithi016.stdout: "6a03046e-cdf4-4a14-b9bd-b560b8968467": { 2024-08-31T06:21:13.919 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:12:38.002354+0000", 2024-08-31T06:21:13.919 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:21:13.919 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:21:13.920 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:21:13.920 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:21:13.920 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:21:13.920 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:21:13.920 INFO:teuthology.orchestra.run.smithi016.stdout: "004b47bb-0fe8-4d28-9d9d-78bde6a61199": { 2024-08-31T06:21:13.920 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:16:48.915926+0000", 2024-08-31T06:21:13.920 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:21:13.920 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:21:13.920 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:21:13.920 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:21:13.920 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:21:13.921 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:21:13.921 INFO:teuthology.orchestra.run.smithi016.stdout: "88d7bdb8-cb1c-45e5-ad29-4cd8da6c6003": { 2024-08-31T06:21:13.921 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:07:39.648942+0000", 2024-08-31T06:21:13.921 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:21:13.921 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:21:13.921 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:21:13.921 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:21:13.921 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:21:13.921 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:21:13.921 INFO:teuthology.orchestra.run.smithi016.stdout: "0fdcd762-533d-42be-9289-b71c8e3b81b1": { 2024-08-31T06:21:13.921 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:15:06.169112+0000", 2024-08-31T06:21:13.921 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:21:13.921 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:21:13.921 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:21:13.922 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:21:13.922 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:21:13.922 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:21:13.922 INFO:teuthology.orchestra.run.smithi016.stdout: "0015da36-6634-47c5-b8d4-219f9eba8250": { 2024-08-31T06:21:13.922 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:21:11.445380+0000", 2024-08-31T06:21:13.922 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:21:13.922 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:21:13.922 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:21:13.922 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:21:13.922 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:21:13.922 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:21:13.922 INFO:teuthology.orchestra.run.smithi016.stdout: "c8455d74-2fe7-4d93-bcf2-19b419d16016": { 2024-08-31T06:21:13.922 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:08:24.035960+0000", 2024-08-31T06:21:13.922 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:21:13.922 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:21:13.923 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:21:13.923 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:21:13.923 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:21:13.923 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:21:13.923 INFO:teuthology.orchestra.run.smithi016.stdout: "78d6ef02-8551-45e8-ad60-9627eb573fc1": { 2024-08-31T06:21:13.923 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:09:13.547865+0000", 2024-08-31T06:21:13.923 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:21:13.923 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:21:13.923 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:21:13.923 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:21:13.923 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:21:13.923 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:21:13.923 INFO:teuthology.orchestra.run.smithi016.stdout: "c1fd6904-2e02-47fa-939a-47349b9fe36b": { 2024-08-31T06:21:13.923 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:06.678691+0000", 2024-08-31T06:21:13.924 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:21:13.924 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:21:13.924 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:21:13.924 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:21:13.924 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:21:13.924 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:21:13.924 INFO:teuthology.orchestra.run.smithi016.stdout: "a10cce76-99ab-46b5-b3b2-07b82899b8f9": { 2024-08-31T06:21:13.924 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:11:43.967546+0000", 2024-08-31T06:21:13.924 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:21:13.924 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:21:13.924 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:21:13.924 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:21:13.924 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:21:13.924 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:21:13.924 INFO:teuthology.orchestra.run.smithi016.stdout: "91b72a34-4ba9-4126-a414-8b862f8c5f78": { 2024-08-31T06:21:13.925 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:13:28.663237+0000", 2024-08-31T06:21:13.925 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:21:13.925 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:21:13.925 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:21:13.925 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:21:13.925 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:21:13.925 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:21:13.925 INFO:teuthology.orchestra.run.smithi016.stdout: "e739a16e-8ace-43a8-a1ce-ac27d39ef8a3": { 2024-08-31T06:21:13.925 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:54.084646+0000", 2024-08-31T06:21:13.925 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:21:13.925 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:21:13.925 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:21:13.925 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:21:13.925 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:21:13.926 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:21:13.926 INFO:teuthology.orchestra.run.smithi016.stdout: "a64d5301-8558-4e9c-ac9b-8d0ac7057e74": { 2024-08-31T06:21:13.926 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:14:20.338104+0000", 2024-08-31T06:21:13.926 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:21:13.926 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:21:13.926 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:21:13.926 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:21:13.926 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:21:13.926 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:21:13.926 INFO:teuthology.orchestra.run.smithi016.stdout: "f9c4a22e-0c8a-4928-9b0d-e2ce1db5a86b": { 2024-08-31T06:21:13.926 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:15:57.496015+0000", 2024-08-31T06:21:13.926 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:21:13.926 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:21:13.926 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:21:13.927 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:21:13.927 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:21:13.927 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:21:13.927 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:21:13.927 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:21:13.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.911+0000 7f26b9200640 1 -- 172.21.15.16:0/3238904417 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f26980b4d20 msgr2=0x7f26980b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:13.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.911+0000 7f26b9200640 1 --2- 172.21.15.16:0/3238904417 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f26980b4d20 0x7f26980b7230 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x5632cb83b6b0 tx=0x7f26a40316c0 comp rx=0 tx=0).stop 2024-08-31T06:21:13.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.911+0000 7f26b9200640 1 -- 172.21.15.16:0/3238904417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26b4102230 msgr2=0x7f26b419c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:13.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.911+0000 7f26b9200640 1 --2- 172.21.15.16:0/3238904417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26b4102230 0x7f26b419c240 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f26a4005940 tx=0x7f26a4030fa0 comp rx=0 tx=0).stop 2024-08-31T06:21:13.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.911+0000 7f26b9200640 1 -- 172.21.15.16:0/3238904417 shutdown_connections 2024-08-31T06:21:13.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.911+0000 7f26b9200640 1 --2- 172.21.15.16:0/3238904417 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f26980b4d20 0x7f26980b7230 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:13.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.911+0000 7f26b9200640 1 --2- 172.21.15.16:0/3238904417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26b410fde0 0x7f26b41a1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:13.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.911+0000 7f26b9200640 1 --2- 172.21.15.16:0/3238904417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26b4102b80 0x7f26b419c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:13.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.911+0000 7f26b9200640 1 --2- 172.21.15.16:0/3238904417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26b4102230 0x7f26b419c240 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:13.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.911+0000 7f26b9200640 1 -- 172.21.15.16:0/3238904417 >> 172.21.15.16:0/3238904417 conn(0x7f26b40fe0d0 msgr2=0x7f26b4100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:13.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.911+0000 7f26b9200640 1 -- 172.21.15.16:0/3238904417 shutdown_connections 2024-08-31T06:21:13.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:13.911+0000 7f26b9200640 1 -- 172.21.15.16:0/3238904417 wait complete. 2024-08-31T06:21:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.169+0000 7fa2dcc00640 1 -- 172.21.15.16:0/1062521618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d810d960 msgr2=0x7fa2d810fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.169+0000 7fa2dcc00640 1 --2- 172.21.15.16:0/1062521618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d810d960 0x7fa2d810fe50 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0009ca0 tx=0x7fa2d002a8c0 comp rx=0 tx=0).stop 2024-08-31T06:21:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.169+0000 7fa2dcc00640 1 -- 172.21.15.16:0/1062521618 shutdown_connections 2024-08-31T06:21:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.169+0000 7fa2dcc00640 1 --2- 172.21.15.16:0/1062521618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d810d960 0x7fa2d810fe50 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.169+0000 7fa2dcc00640 1 --2- 172.21.15.16:0/1062521618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2d8104e50 0x7fa2d810d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.169+0000 7fa2dcc00640 1 --2- 172.21.15.16:0/1062521618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2d8104500 0x7fa2d8104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.169+0000 7fa2dcc00640 1 -- 172.21.15.16:0/1062521618 >> 172.21.15.16:0/1062521618 conn(0x7fa2d80fdfd0 msgr2=0x7fa2d8100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.169+0000 7fa2dcc00640 1 -- 172.21.15.16:0/1062521618 shutdown_connections 2024-08-31T06:21:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.169+0000 7fa2dcc00640 1 -- 172.21.15.16:0/1062521618 wait complete. 2024-08-31T06:21:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.170+0000 7fa2dcc00640 1 Processor -- start 2024-08-31T06:21:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.171+0000 7fa2dcc00640 1 -- start start 2024-08-31T06:21:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.171+0000 7fa2dcc00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2d8104500 0x7fa2d819c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.171+0000 7fa2dcc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2d8104e50 0x7fa2d819c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.171+0000 7fa2dcc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d810d960 0x7fa2d81a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.171+0000 7fa2dcc00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d81135d0 con 0x7fa2d810d960 2024-08-31T06:21:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.171+0000 7fa2dcc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d8113450 con 0x7fa2d8104e50 2024-08-31T06:21:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.171+0000 7fa2dcc00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d8113750 con 0x7fa2d8104500 2024-08-31T06:21:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.171+0000 7fa2d7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d810d960 0x7fa2d81a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.171+0000 7fa2cfe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2d8104e50 0x7fa2d819c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.171+0000 7fa2d7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d810d960 0x7fa2d81a1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39496/0 (socket says 172.21.15.16:39496) 2024-08-31T06:21:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.171+0000 7fa2d7e00640 1 -- 172.21.15.16:0/101004112 learned_addr learned my addr 172.21.15.16:0/101004112 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.171+0000 7fa2d7400640 1 --2- 172.21.15.16:0/101004112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2d8104500 0x7fa2d819c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.172+0000 7fa2d7400640 1 -- 172.21.15.16:0/101004112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2d8104e50 msgr2=0x7fa2d819c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.172+0000 7fa2d7400640 1 --2- 172.21.15.16:0/101004112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2d8104e50 0x7fa2d819c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.172+0000 7fa2d7400640 1 -- 172.21.15.16:0/101004112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d810d960 msgr2=0x7fa2d81a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.172+0000 7fa2d7400640 1 --2- 172.21.15.16:0/101004112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d810d960 0x7fa2d81a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.172+0000 7fa2d7400640 1 -- 172.21.15.16:0/101004112 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d81a2160 con 0x7fa2d8104500 2024-08-31T06:21:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.172+0000 7fa2cfe00640 1 --2- 172.21.15.16:0/101004112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2d8104e50 0x7fa2d819c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:21:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.172+0000 7fa2d7e00640 1 --2- 172.21.15.16:0/101004112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d810d960 0x7fa2d81a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:21:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.172+0000 7fa2d7400640 1 --2- 172.21.15.16:0/101004112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2d8104500 0x7fa2d819c230 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fa2c800e9f0 tx=0x7fa2c800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.172+0000 7fa2d4c00640 1 -- 172.21.15.16:0/101004112 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c800ccc0 con 0x7fa2d8104500 2024-08-31T06:21:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.172+0000 7fa2dcc00640 1 -- 172.21.15.16:0/101004112 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2d81a2450 con 0x7fa2d8104500 2024-08-31T06:21:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.172+0000 7fa2d4c00640 1 -- 172.21.15.16:0/101004112 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2c80108e0 con 0x7fa2d8104500 2024-08-31T06:21:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.172+0000 7fa2d4c00640 1 -- 172.21.15.16:0/101004112 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8018880 con 0x7fa2d8104500 2024-08-31T06:21:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.173+0000 7fa2dcc00640 1 -- 172.21.15.16:0/101004112 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2d81a8be0 con 0x7fa2d8104500 2024-08-31T06:21:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.175+0000 7fa2e541e740 1 Processor -- start 2024-08-31T06:21:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.176+0000 7fa2e541e740 1 -- start start 2024-08-31T06:21:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.176+0000 7fa2e541e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581959081b0 0x558195cabb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.176+0000 7fa2e541e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581958df2f0 0x558195cac040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.176+0000 7fa2e541e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581958e74d0 0x558195cac580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.176+0000 7fa2e541e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558195c20f70 con 0x5581959081b0 2024-08-31T06:21:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.176+0000 7fa2e541e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x558195c10500 con 0x5581958df2f0 2024-08-31T06:21:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.176+0000 7fa2e541e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558195c144b0 con 0x5581958e74d0 2024-08-31T06:21:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.176+0000 7fa2d7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581959081b0 0x558195cabb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.176+0000 7fa2d7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581959081b0 0x558195cabb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39500/0 (socket says 172.21.15.16:39500) 2024-08-31T06:21:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.176+0000 7fa2d7e00640 1 -- 172.21.15.16:0/3677501284 learned_addr learned my addr 172.21.15.16:0/3677501284 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.176+0000 7fa2cfe00640 1 --2- 172.21.15.16:0/3677501284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581958e74d0 0x558195cac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.177+0000 7fa2d4c00640 1 -- 172.21.15.16:0/101004112 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fa2c8010430 con 0x7fa2d8104500 2024-08-31T06:21:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.177+0000 7fa2d7e00640 1 -- 172.21.15.16:0/3677501284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581958e74d0 msgr2=0x558195cac580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.177+0000 7fa2d7e00640 1 --2- 172.21.15.16:0/3677501284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581958e74d0 0x558195cac580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.177+0000 7fa2d7e00640 1 -- 172.21.15.16:0/3677501284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581958df2f0 msgr2=0x558195cac040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:21:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.177+0000 7fa2d7e00640 1 --2- 172.21.15.16:0/3677501284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581958df2f0 0x558195cac040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.177+0000 7fa2d7e00640 1 -- 172.21.15.16:0/3677501284 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558195bcac10 con 0x5581959081b0 2024-08-31T06:21:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.177+0000 7fa2d7e00640 1 --2- 172.21.15.16:0/3677501284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581959081b0 0x558195cabb00 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa2d00094b0 tx=0x7fa2d002f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e10133e8a7ecb3a4 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.177+0000 7fa2c7e00640 1 -- 172.21.15.16:0/3677501284 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0031450 con 0x5581959081b0 2024-08-31T06:21:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.177+0000 7fa2c7e00640 1 -- 172.21.15.16:0/3677501284 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2d0036040 con 0x5581959081b0 2024-08-31T06:21:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.177+0000 7fa2c7e00640 1 -- 172.21.15.16:0/3677501284 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0004500 con 0x5581959081b0 2024-08-31T06:21:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.178+0000 7fa2e541e740 1 -- 172.21.15.16:0/3677501284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581959081b0 msgr2=0x558195cabb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.178+0000 7fa2e541e740 1 --2- 172.21.15.16:0/3677501284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581959081b0 0x558195cabb00 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa2d00094b0 tx=0x7fa2d002f310 comp rx=0 tx=0).stop 2024-08-31T06:21:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.178+0000 7fa2d4c00640 1 --2- 172.21.15.16:0/101004112 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa2bc0b4ca0 0x7fa2bc0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.178+0000 7fa2e541e740 1 -- 172.21.15.16:0/3677501284 shutdown_connections 2024-08-31T06:21:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.178+0000 7fa2e541e740 1 --2- 172.21.15.16:0/3677501284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581958e74d0 0x558195cac580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.178+0000 7fa2e541e740 1 --2- 172.21.15.16:0/3677501284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581958df2f0 0x558195cac040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.178+0000 7fa2e541e740 1 --2- 172.21.15.16:0/3677501284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581959081b0 0x558195cabb00 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.178+0000 7fa2e541e740 1 -- 172.21.15.16:0/3677501284 >> 172.21.15.16:0/3677501284 conn(0x5581958fc300 msgr2=0x558195ca99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.179+0000 7fa2d4c00640 1 -- 172.21.15.16:0/101004112 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8514+0+0 (secure 0 0 0) 0x7fa2c80e58f0 con 0x7fa2d8104500 2024-08-31T06:21:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.179+0000 7fa2e541e740 1 -- 172.21.15.16:0/3677501284 shutdown_connections 2024-08-31T06:21:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.179+0000 7fa2e541e740 1 -- 172.21.15.16:0/3677501284 wait complete. 2024-08-31T06:21:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.179+0000 7fa2d7e00640 1 --2- 172.21.15.16:0/101004112 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa2bc0b4ca0 0x7fa2bc0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.179+0000 7fa2e541e740 1 -- start start 2024-08-31T06:21:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.179+0000 7fa2d7e00640 1 --2- 172.21.15.16:0/101004112 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa2bc0b4ca0 0x7fa2bc0b71b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0005cc0 tx=0x7fa2d002a8c0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.179+0000 7fa2e541e740 1 Processor -- start 2024-08-31T06:21:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.180+0000 7fa2e541e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581958df2f0 0x558195cb7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.180+0000 7fa2e541e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581958e74d0 0x558195cb7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.180+0000 7fa2e541e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581959081b0 0x558195cc33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.180+0000 7fa2e541e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558195b49ab0 con 0x5581959081b0 2024-08-31T06:21:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.180+0000 7fa2e541e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x558195bb7320 con 0x5581958e74d0 2024-08-31T06:21:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.180+0000 7fa2e541e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558195cba0a0 con 0x5581958df2f0 2024-08-31T06:21:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.180+0000 7fa2e541e740 10 client.0 _ll_register_callbacks cb 0x558195bbd6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:21:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.180+0000 7fa2d7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581959081b0 0x558195cc33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.180+0000 7fa2d7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581959081b0 0x558195cc33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39516/0 (socket says 172.21.15.16:39516) 2024-08-31T06:21:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.180+0000 7fa2d7e00640 1 -- 172.21.15.16:0/4275626803 learned_addr learned my addr 172.21.15.16:0/4275626803 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.180+0000 7fa2cfe00640 1 --2- 172.21.15.16:0/4275626803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581958e74d0 0x558195cb7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:14.186 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.180+0000 7fa2d7400640 1 --2- 172.21.15.16:0/4275626803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581958df2f0 0x558195cb7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:14.186 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.181+0000 7fa2d7e00640 1 -- 172.21.15.16:0/4275626803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581958df2f0 msgr2=0x558195cb7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:14.186 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.181+0000 7fa2d7e00640 1 --2- 172.21.15.16:0/4275626803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581958df2f0 0x558195cb7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:14.186 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.181+0000 7fa2d7e00640 1 -- 172.21.15.16:0/4275626803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581958e74d0 msgr2=0x558195cb7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:14.186 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.181+0000 7fa2d7e00640 1 --2- 172.21.15.16:0/4275626803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581958e74d0 0x558195cb7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:14.186 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.181+0000 7fa2d7e00640 1 -- 172.21.15.16:0/4275626803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558195973790 con 0x5581959081b0 2024-08-31T06:21:14.186 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.181+0000 7fa2d7e00640 1 --2- 172.21.15.16:0/4275626803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581959081b0 0x558195cc33c0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa2d00094b0 tx=0x7fa2d0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aaa50bc3c8de41f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:14.186 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.181+0000 7fa2c4c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:21:14.186 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.181+0000 7fa2c4c00640 1 -- 172.21.15.16:0/4275626803 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0004490 con 0x5581959081b0 2024-08-31T06:21:14.186 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.181+0000 7fa2c4c00640 1 -- 172.21.15.16:0/4275626803 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8514+0+0 (secure 0 0 0) 0x7fa2d0004630 con 0x5581959081b0 2024-08-31T06:21:14.186 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.181+0000 7fa2e541e740 1 -- 172.21.15.16:0/4275626803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2d0036040 con 0x5581959081b0 2024-08-31T06:21:14.188 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.182+0000 7fa2c4c00640 1 -- 172.21.15.16:0/4275626803 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fa2d0068050 con 0x5581959081b0 2024-08-31T06:21:14.188 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.182+0000 7fa2e541e740 10 client.18312 fetch_fsmap learned FSMap version 49 2024-08-31T06:21:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.182+0000 7fa2e541e740 1 -- 172.21.15.16:0/4275626803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5581959ec620 con 0x5581959081b0 2024-08-31T06:21:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.182+0000 7fa2c4c00640 1 -- 172.21.15.16:0/4275626803 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2d0036040 con 0x5581959081b0 2024-08-31T06:21:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.182+0000 7fa2c4c00640 1 -- 172.21.15.16:0/4275626803 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fa2d004f050 con 0x5581959081b0 2024-08-31T06:21:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.182+0000 7fa2e541e740 10 client.18312 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:21:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.182+0000 7fa2e541e740 10 client.18312 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:21:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.182+0000 7fa2e541e740 20 client.18312 populate_metadata read hostname 'smithi016' 2024-08-31T06:21:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.182+0000 7fa2e541e740 1 --2- 172.21.15.16:0/4275626803 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5581958c6d80 0x558195cc8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.182+0000 7fa2e541e740 4 client.18312 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:21:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.182+0000 7fa2e541e740 1 -- 172.21.15.16:0/4275626803 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558195c22ec0 con 0x5581958c6d80 2024-08-31T06:21:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.183+0000 7fa2d7400640 1 --2- 172.21.15.16:0/4275626803 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5581958c6d80 0x558195cc8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.183+0000 7fa2d7400640 1 --2- 172.21.15.16:0/4275626803 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5581958c6d80 0x558195cc8f10 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=701c63716f198e8d server_cookie=fe120cffade298c0 in_seq=0 out_seq=0 2024-08-31T06:21:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.183+0000 7fa2c4c00640 10 client.18312 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:21:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.184+0000 7fa2c4c00640 1 -- 172.21.15.16:0/4275626803 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558195c22ec0 con 0x5581958c6d80 2024-08-31T06:21:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.184+0000 7fa2c4c00640 10 client.18312 handle_command_reply: tid=0 2024-08-31T06:21:14.190 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.184+0000 7fa2e541e740 1 client.18312 shutdown 2024-08-31T06:21:14.190 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.184+0000 7fa2e541e740 1 -- 172.21.15.16:0/4275626803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581959081b0 msgr2=0x558195cc33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:14.190 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.184+0000 7fa2e541e740 1 --2- 172.21.15.16:0/4275626803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581959081b0 0x558195cc33c0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa2d00094b0 tx=0x7fa2d0005fd0 comp rx=0 tx=0).stop 2024-08-31T06:21:14.190 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.184+0000 7fa2e541e740 1 -- 172.21.15.16:0/4275626803 shutdown_connections 2024-08-31T06:21:14.190 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.184+0000 7fa2e541e740 1 --2- 172.21.15.16:0/4275626803 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5581958c6d80 0x558195cc8f10 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:14.190 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.184+0000 7fa2e541e740 1 --2- 172.21.15.16:0/4275626803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581959081b0 0x558195cc33c0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:14.190 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.184+0000 7fa2e541e740 1 --2- 172.21.15.16:0/4275626803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581958e74d0 0x558195cb7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:14.190 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.184+0000 7fa2c4c00640 0 client.18312 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:21:14.190 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.184+0000 7fa2e541e740 1 --2- 172.21.15.16:0/4275626803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581958df2f0 0x558195cb7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:14.190 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.184+0000 7fa2e541e740 1 -- 172.21.15.16:0/4275626803 >> 172.21.15.16:0/4275626803 conn(0x5581958fc300 msgr2=0x558195cacb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:14.190 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.185+0000 7fa2e541e740 1 -- 172.21.15.16:0/4275626803 shutdown_connections 2024-08-31T06:21:14.190 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.185+0000 7fa2e541e740 1 -- 172.21.15.16:0/4275626803 wait complete. 2024-08-31T06:21:14.191 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.185+0000 7fa2e541e740 20 client.18312 trim_cache size 0 max 16384 2024-08-31T06:21:14.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.200+0000 7fa2e541e740 1 Processor -- start 2024-08-31T06:21:14.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.201+0000 7fa2e541e740 1 -- start start 2024-08-31T06:21:14.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.201+0000 7fa2e541e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581958df2f0 0x558195cb7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:14.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.201+0000 7fa2e541e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581958e74d0 0x558195cb9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:14.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.201+0000 7fa2e541e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581959081b0 0x558195ca8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:14.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.201+0000 7fa2e541e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d81b3e10 con 0x5581958e74d0 2024-08-31T06:21:14.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.201+0000 7fa2e541e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d81b1620 con 0x5581959081b0 2024-08-31T06:21:14.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.201+0000 7fa2e541e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d81af490 con 0x5581958df2f0 2024-08-31T06:21:14.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.201+0000 7fa2cfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581958e74d0 0x558195cb9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:14.205 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.201+0000 7fa2cfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581958e74d0 0x558195cb9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39522/0 (socket says 172.21.15.16:39522) 2024-08-31T06:21:14.205 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.201+0000 7fa2cfe00640 1 -- 172.21.15.16:0/3864866900 learned_addr learned my addr 172.21.15.16:0/3864866900 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:14.205 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.201+0000 7fa2d7400640 1 --2- 172.21.15.16:0/3864866900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581958df2f0 0x558195cb7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:14.205 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.201+0000 7fa2d7e00640 1 --2- 172.21.15.16:0/3864866900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581959081b0 0x558195ca8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:14.205 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.202+0000 7fa2cfe00640 1 -- 172.21.15.16:0/3864866900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581958df2f0 msgr2=0x558195cb7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:14.205 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.202+0000 7fa2cfe00640 1 --2- 172.21.15.16:0/3864866900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581958df2f0 0x558195cb7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:14.205 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.202+0000 7fa2cfe00640 1 -- 172.21.15.16:0/3864866900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581959081b0 msgr2=0x558195ca8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:14.205 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.202+0000 7fa2cfe00640 1 --2- 172.21.15.16:0/3864866900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581959081b0 0x558195ca8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:14.205 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.202+0000 7fa2cfe00640 1 -- 172.21.15.16:0/3864866900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558195ca3be0 con 0x5581958e74d0 2024-08-31T06:21:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.202+0000 7fa2cfe00640 1 --2- 172.21.15.16:0/3864866900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581958e74d0 0x558195cb9030 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0012610 tx=0x7fa2c0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27736a1476a5d331 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.202+0000 7fa2c7400640 1 -- 172.21.15.16:0/3864866900 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0012190 con 0x5581958e74d0 2024-08-31T06:21:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.202+0000 7fa2c7400640 1 -- 172.21.15.16:0/3864866900 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2c00054d0 con 0x5581958e74d0 2024-08-31T06:21:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.202+0000 7fa2c7400640 1 -- 172.21.15.16:0/3864866900 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c000f580 con 0x5581958e74d0 2024-08-31T06:21:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.203+0000 7fa2e541e740 1 -- 172.21.15.16:0/3864866900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581958e74d0 msgr2=0x558195cb9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.203+0000 7fa2e541e740 1 --2- 172.21.15.16:0/3864866900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581958e74d0 0x558195cb9030 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0012610 tx=0x7fa2c0005f90 comp rx=0 tx=0).stop 2024-08-31T06:21:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.203+0000 7fa2e541e740 1 -- 172.21.15.16:0/3864866900 shutdown_connections 2024-08-31T06:21:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.203+0000 7fa2e541e740 1 --2- 172.21.15.16:0/3864866900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581959081b0 0x558195ca8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.203+0000 7fa2e541e740 1 --2- 172.21.15.16:0/3864866900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581958e74d0 0x558195cb9030 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.203+0000 7fa2e541e740 1 --2- 172.21.15.16:0/3864866900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581958df2f0 0x558195cb7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.203+0000 7fa2e541e740 1 -- 172.21.15.16:0/3864866900 >> 172.21.15.16:0/3864866900 conn(0x5581958fc300 msgr2=0x558195ca7f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.203+0000 7fa2e541e740 1 -- 172.21.15.16:0/3864866900 shutdown_connections 2024-08-31T06:21:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.203+0000 7fa2e541e740 1 -- 172.21.15.16:0/3864866900 wait complete. 2024-08-31T06:21:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.204+0000 7fa2e541e740 1 -- start start 2024-08-31T06:21:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.204+0000 7fa2e541e740 1 Processor -- start 2024-08-31T06:21:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.205+0000 7fa2e541e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581958df2f0 0x558195cbf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.205+0000 7fa2e541e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581958e74d0 0x558195cbcd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.205+0000 7fa2e541e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581959081b0 0x558195cbd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.205+0000 7fa2e541e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d81afae0 con 0x5581958e74d0 2024-08-31T06:21:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.205+0000 7fa2e541e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d81b0f50 con 0x5581959081b0 2024-08-31T06:21:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.205+0000 7fa2e541e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d81b01b0 con 0x5581958df2f0 2024-08-31T06:21:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.205+0000 7fa2e541e740 10 client.0 _ll_register_callbacks cb 0x7fa2d00098f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:21:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.205+0000 7fa2cfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581958e74d0 0x558195cbcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:14.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.205+0000 7fa2cfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581958e74d0 0x558195cbcd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39524/0 (socket says 172.21.15.16:39524) 2024-08-31T06:21:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.205+0000 7fa2cfe00640 1 -- 172.21.15.16:0/35640894 learned_addr learned my addr 172.21.15.16:0/35640894 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.205+0000 7fa2d7e00640 1 --2- 172.21.15.16:0/35640894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581959081b0 0x558195cbd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.205+0000 7fa2d7400640 1 --2- 172.21.15.16:0/35640894 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581958df2f0 0x558195cbf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.205+0000 7fa2cfe00640 1 -- 172.21.15.16:0/35640894 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581958df2f0 msgr2=0x558195cbf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.205+0000 7fa2cfe00640 1 --2- 172.21.15.16:0/35640894 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581958df2f0 0x558195cbf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.205+0000 7fa2cfe00640 1 -- 172.21.15.16:0/35640894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581959081b0 msgr2=0x558195cbd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.205+0000 7fa2cfe00640 1 --2- 172.21.15.16:0/35640894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581959081b0 0x558195cbd2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.205+0000 7fa2cfe00640 1 -- 172.21.15.16:0/35640894 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558195cbd7f0 con 0x5581958e74d0 2024-08-31T06:21:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.205+0000 7fa2d7e00640 1 --2- 172.21.15.16:0/35640894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581959081b0 0x558195cbd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:21:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.206+0000 7fa2cfe00640 1 --2- 172.21.15.16:0/35640894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581958e74d0 0x558195cbcd70 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0000c00 tx=0x7fa2c0005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a58bd26069edf7 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.206+0000 7fa2c4c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:21:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.206+0000 7fa2c4c00640 1 -- 172.21.15.16:0/35640894 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c002a070 con 0x5581958e74d0 2024-08-31T06:21:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.206+0000 7fa2c4c00640 1 -- 172.21.15.16:0/35640894 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8514+0+0 (secure 0 0 0) 0x7fa2c0005020 con 0x5581958e74d0 2024-08-31T06:21:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.206+0000 7fa2e541e740 1 -- 172.21.15.16:0/35640894 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2c00054d0 con 0x5581958e74d0 2024-08-31T06:21:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.206+0000 7fa2c4c00640 1 -- 172.21.15.16:0/35640894 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fa2c0035050 con 0x5581958e74d0 2024-08-31T06:21:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.206+0000 7fa2e541e740 10 client.18318 fetch_fsmap learned FSMap version 49 2024-08-31T06:21:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.206+0000 7fa2e541e740 1 -- 172.21.15.16:0/35640894 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x558195d334d0 con 0x5581958e74d0 2024-08-31T06:21:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.207+0000 7fa2c4c00640 1 -- 172.21.15.16:0/35640894 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2c00054d0 con 0x5581958e74d0 2024-08-31T06:21:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.207+0000 7fa2c4c00640 1 -- 172.21.15.16:0/35640894 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7fa2c00038f0 con 0x5581958e74d0 2024-08-31T06:21:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.207+0000 7fa2e541e740 10 client.18318 fetch_fsmap finished waiting for FSMap version 49 2024-08-31T06:21:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.207+0000 7fa2e541e740 10 client.18318 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:21:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.207+0000 7fa2e541e740 20 client.18318 populate_metadata read hostname 'smithi016' 2024-08-31T06:21:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.207+0000 7fa2e541e740 1 --2- 172.21.15.16:0/35640894 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55819589c1c0 0x558195d35c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.207+0000 7fa2e541e740 4 client.18318 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:21:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.207+0000 7fa2e541e740 1 -- 172.21.15.16:0/35640894 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558195c1e960 con 0x55819589c1c0 2024-08-31T06:21:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.207+0000 7fa2d7400640 1 --2- 172.21.15.16:0/35640894 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55819589c1c0 0x558195d35c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.208+0000 7fa2d7400640 1 --2- 172.21.15.16:0/35640894 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55819589c1c0 0x558195d35c40 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4d192a4c2a23d7c server_cookie=9cf55a4fec2577ab in_seq=0 out_seq=0 2024-08-31T06:21:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.208+0000 7fa2c4c00640 10 client.18318 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:21:14.227 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.224+0000 7fa2c4c00640 1 -- 172.21.15.16:0/35640894 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558195c1e960 con 0x55819589c1c0 2024-08-31T06:21:14.227 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.224+0000 7fa2c4c00640 10 client.18318 handle_command_reply: tid=0 2024-08-31T06:21:14.227 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.224+0000 7fa2e541e740 1 client.18318 shutdown 2024-08-31T06:21:14.227 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.224+0000 7fa2e541e740 1 -- 172.21.15.16:0/35640894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581958e74d0 msgr2=0x558195cbcd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:14.227 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.224+0000 7fa2e541e740 1 --2- 172.21.15.16:0/35640894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581958e74d0 0x558195cbcd70 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0000c00 tx=0x7fa2c0005750 comp rx=0 tx=0).stop 2024-08-31T06:21:14.227 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.224+0000 7fa2e541e740 1 -- 172.21.15.16:0/35640894 shutdown_connections 2024-08-31T06:21:14.227 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.224+0000 7fa2e541e740 1 --2- 172.21.15.16:0/35640894 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55819589c1c0 0x558195d35c40 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:14.227 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.224+0000 7fa2e541e740 1 --2- 172.21.15.16:0/35640894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581959081b0 0x558195cbd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:14.227 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.224+0000 7fa2e541e740 1 --2- 172.21.15.16:0/35640894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581958e74d0 0x558195cbcd70 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:14.228 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.224+0000 7fa2c4c00640 0 client.18318 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:21:14.228 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.224+0000 7fa2e541e740 1 --2- 172.21.15.16:0/35640894 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581958df2f0 0x558195cbf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:14.228 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.225+0000 7fa2e541e740 1 -- 172.21.15.16:0/35640894 >> 172.21.15.16:0/35640894 conn(0x5581958fc300 msgr2=0x558195cb8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:14.228 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.225+0000 7fa2e541e740 1 -- 172.21.15.16:0/35640894 shutdown_connections 2024-08-31T06:21:14.228 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.225+0000 7fa2e541e740 1 -- 172.21.15.16:0/35640894 wait complete. 2024-08-31T06:21:14.228 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.225+0000 7fa2e541e740 20 client.18318 trim_cache size 0 max 16384 2024-08-31T06:21:14.228 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:21:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:21:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:21:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:21:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.226+0000 7fa2dcc00640 1 -- 172.21.15.16:0/101004112 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa2bc0b4ca0 msgr2=0x7fa2bc0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.226+0000 7fa2dcc00640 1 --2- 172.21.15.16:0/101004112 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa2bc0b4ca0 0x7fa2bc0b71b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0005cc0 tx=0x7fa2d002a8c0 comp rx=0 tx=0).stop 2024-08-31T06:21:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.226+0000 7fa2dcc00640 1 -- 172.21.15.16:0/101004112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2d8104500 msgr2=0x7fa2d819c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.226+0000 7fa2dcc00640 1 --2- 172.21.15.16:0/101004112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2d8104500 0x7fa2d819c230 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fa2c800e9f0 tx=0x7fa2c800eeb0 comp rx=0 tx=0).stop 2024-08-31T06:21:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.226+0000 7fa2dcc00640 1 -- 172.21.15.16:0/101004112 shutdown_connections 2024-08-31T06:21:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.226+0000 7fa2dcc00640 1 --2- 172.21.15.16:0/101004112 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fa2bc0b4ca0 0x7fa2bc0b71b0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.226+0000 7fa2dcc00640 1 --2- 172.21.15.16:0/101004112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d810d960 0x7fa2d81a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.226+0000 7fa2dcc00640 1 --2- 172.21.15.16:0/101004112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2d8104e50 0x7fa2d819c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.226+0000 7fa2dcc00640 1 --2- 172.21.15.16:0/101004112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2d8104500 0x7fa2d819c230 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.226+0000 7fa2dcc00640 1 -- 172.21.15.16:0/101004112 >> 172.21.15.16:0/101004112 conn(0x7fa2d80fdfd0 msgr2=0x7fa2d810c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.226+0000 7fa2dcc00640 1 -- 172.21.15.16:0/101004112 shutdown_connections 2024-08-31T06:21:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:14.226+0000 7fa2dcc00640 1 -- 172.21.15.16:0/101004112 wait complete. 2024-08-31T06:21:14.919 INFO:tasks.cephfs.filesystem:scrub status for tag:0015da36-6634-47c5-b8d4-219f9eba8250 - {'path': '/', 'tag': '0015da36-6634-47c5-b8d4-219f9eba8250', 'options': 'recursive,force'} 2024-08-31T06:21:15.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:14 smithi155 ceph-mon[21742]: pgmap v801: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 15 MiB/s wr, 1.35k op/s 2024-08-31T06:21:15.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:14 smithi016 ceph-mon[17194]: pgmap v801: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 15 MiB/s wr, 1.35k op/s 2024-08-31T06:21:15.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:14 smithi184 ceph-mon[22331]: pgmap v801: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 15 MiB/s wr, 1.35k op/s 2024-08-31T06:21:16.629 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 830.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-31T06:21:17.124 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:16 smithi155 ceph-mon[21742]: pgmap v802: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 13 MiB/s wr, 1.21k op/s 2024-08-31T06:21:17.124 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:16 smithi155 ceph-mon[21742]: osdmap e99: 12 total, 12 up, 12 in 2024-08-31T06:21:17.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:16 smithi016 ceph-mon[17194]: pgmap v802: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 13 MiB/s wr, 1.21k op/s 2024-08-31T06:21:17.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:16 smithi016 ceph-mon[17194]: osdmap e99: 12 total, 12 up, 12 in 2024-08-31T06:21:17.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:16 smithi184 ceph-mon[22331]: pgmap v802: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 13 MiB/s wr, 1.21k op/s 2024-08-31T06:21:17.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:16 smithi184 ceph-mon[22331]: osdmap e99: 12 total, 12 up, 12 in 2024-08-31T06:21:19.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:18 smithi155 ceph-mon[21742]: pgmap v804: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 4.4 MiB/s wr, 443 op/s 2024-08-31T06:21:19.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:18 smithi155 ceph-mon[21742]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:21:19.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:18 smithi016 ceph-mon[17194]: pgmap v804: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 4.4 MiB/s wr, 443 op/s 2024-08-31T06:21:19.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:18 smithi016 ceph-mon[17194]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:21:19.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:18 smithi184 ceph-mon[22331]: pgmap v804: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 4.4 MiB/s wr, 443 op/s 2024-08-31T06:21:19.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:18 smithi184 ceph-mon[22331]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:21:21.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:20 smithi155 ceph-mon[21742]: pgmap v805: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 2.0 MiB/s wr, 323 op/s 2024-08-31T06:21:21.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:20 smithi016 ceph-mon[17194]: pgmap v805: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 2.0 MiB/s wr, 323 op/s 2024-08-31T06:21:21.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:20 smithi184 ceph-mon[22331]: pgmap v805: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 2.0 MiB/s wr, 323 op/s 2024-08-31T06:21:21.629 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 835.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-31T06:21:23.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:22 smithi016 ceph-mon[17194]: pgmap v806: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 887 KiB/s wr, 216 op/s 2024-08-31T06:21:23.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:22 smithi184 ceph-mon[22331]: pgmap v806: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 887 KiB/s wr, 216 op/s 2024-08-31T06:21:23.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:22 smithi155 ceph-mon[21742]: pgmap v806: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 887 KiB/s wr, 216 op/s 2024-08-31T06:21:24.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:23 smithi184 ceph-mon[22331]: pgmap v807: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 1.5 MiB/s wr, 267 op/s 2024-08-31T06:21:24.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:23 smithi155 ceph-mon[21742]: pgmap v807: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 1.5 MiB/s wr, 267 op/s 2024-08-31T06:21:24.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:23 smithi016 ceph-mon[17194]: pgmap v807: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 1.5 MiB/s wr, 267 op/s 2024-08-31T06:21:26.629 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 840.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-31T06:21:26.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:26 smithi016 ceph-mon[17194]: pgmap v808: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 3.1 MiB/s wr, 378 op/s 2024-08-31T06:21:27.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:26 smithi155 ceph-mon[21742]: pgmap v808: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 3.1 MiB/s wr, 378 op/s 2024-08-31T06:21:27.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:26 smithi184 ceph-mon[22331]: pgmap v808: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 3.1 MiB/s wr, 378 op/s 2024-08-31T06:21:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.420+0000 7f6c3f200640 1 -- 172.21.15.16:0/421098642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c40108fd0 msgr2=0x7f6c401093e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.420+0000 7f6c3f200640 1 --2- 172.21.15.16:0/421098642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c40108fd0 0x7f6c401093e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f6c28009900 tx=0x7f6c2802a410 comp rx=0 tx=0).stop 2024-08-31T06:21:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.420+0000 7f6c3f200640 1 -- 172.21.15.16:0/421098642 shutdown_connections 2024-08-31T06:21:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.420+0000 7f6c3f200640 1 --2- 172.21.15.16:0/421098642 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c40109920 0x7f6c40110150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.420+0000 7f6c3f200640 1 --2- 172.21.15.16:0/421098642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c40108fd0 0x7f6c401093e0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.420+0000 7f6c3f200640 1 --2- 172.21.15.16:0/421098642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c401036a0 0x7f6c40103ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.420+0000 7f6c3f200640 1 -- 172.21.15.16:0/421098642 >> 172.21.15.16:0/421098642 conn(0x7f6c400ff100 msgr2=0x7f6c40101550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.420+0000 7f6c3f200640 1 -- 172.21.15.16:0/421098642 shutdown_connections 2024-08-31T06:21:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.420+0000 7f6c3f200640 1 -- 172.21.15.16:0/421098642 wait complete. 2024-08-31T06:21:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.421+0000 7f6c3f200640 1 Processor -- start 2024-08-31T06:21:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.422+0000 7f6c3f200640 1 -- start start 2024-08-31T06:21:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.422+0000 7f6c3f200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c401036a0 0x7f6c4019a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.422+0000 7f6c3f200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c40108fd0 0x7f6c4019a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.422+0000 7f6c3f200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c40109920 0x7f6c4019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.422+0000 7f6c3f200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6c40113590 con 0x7f6c40109920 2024-08-31T06:21:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.422+0000 7f6c3f200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6c40113410 con 0x7f6c401036a0 2024-08-31T06:21:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.422+0000 7f6c3f200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6c40113710 con 0x7f6c40108fd0 2024-08-31T06:21:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.422+0000 7f6c3e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c40109920 0x7f6c4019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.422+0000 7f6c3e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c40109920 0x7f6c4019f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34116/0 (socket says 172.21.15.16:34116) 2024-08-31T06:21:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.422+0000 7f6c3e800640 1 -- 172.21.15.16:0/2762253119 learned_addr learned my addr 172.21.15.16:0/2762253119 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.422+0000 7f6c37e00640 1 --2- 172.21.15.16:0/2762253119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c40108fd0 0x7f6c4019a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.422+0000 7f6c3de00640 1 --2- 172.21.15.16:0/2762253119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c401036a0 0x7f6c4019a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.423+0000 7f6c3e800640 1 -- 172.21.15.16:0/2762253119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c40108fd0 msgr2=0x7f6c4019a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.423+0000 7f6c3e800640 1 --2- 172.21.15.16:0/2762253119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c40108fd0 0x7f6c4019a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.423+0000 7f6c3e800640 1 -- 172.21.15.16:0/2762253119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c401036a0 msgr2=0x7f6c4019a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.423+0000 7f6c3e800640 1 --2- 172.21.15.16:0/2762253119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c401036a0 0x7f6c4019a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.423+0000 7f6c3e800640 1 -- 172.21.15.16:0/2762253119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c4019ff40 con 0x7f6c40109920 2024-08-31T06:21:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.423+0000 7f6c3e800640 1 --2- 172.21.15.16:0/2762253119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c40109920 0x7f6c4019f840 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f6c3800eb10 tx=0x7f6c3800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.423+0000 7f6c36a00640 1 -- 172.21.15.16:0/2762253119 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38004b20 con 0x7f6c40109920 2024-08-31T06:21:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.423+0000 7f6c36a00640 1 -- 172.21.15.16:0/2762253119 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c38004cc0 con 0x7f6c40109920 2024-08-31T06:21:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.423+0000 7f6c3f200640 1 -- 172.21.15.16:0/2762253119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c401a0230 con 0x7f6c40109920 2024-08-31T06:21:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.423+0000 7f6c36a00640 1 -- 172.21.15.16:0/2762253119 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38010670 con 0x7f6c40109920 2024-08-31T06:21:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.423+0000 7f6c3f200640 1 -- 172.21.15.16:0/2762253119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c401a69c0 con 0x7f6c40109920 2024-08-31T06:21:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.426+0000 7f6c4be0e740 1 Processor -- start 2024-08-31T06:21:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.427+0000 7f6c36a00640 1 -- 172.21.15.16:0/2762253119 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f6c38005020 con 0x7f6c40109920 2024-08-31T06:21:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.427+0000 7f6c4be0e740 1 -- start start 2024-08-31T06:21:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.427+0000 7f6c4be0e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e6f7211b0 0x560e6fac8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.427+0000 7f6c4be0e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e6f6f82f0 0x560e6fac91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.427+0000 7f6c4be0e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e6f7004d0 0x560e6fac9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.427+0000 7f6c4be0e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560e6fa39f70 con 0x560e6f7211b0 2024-08-31T06:21:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.427+0000 7f6c4be0e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560e6fa29500 con 0x560e6f7004d0 2024-08-31T06:21:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.427+0000 7f6c4be0e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560e6fa2d4b0 con 0x560e6f6f82f0 2024-08-31T06:21:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.428+0000 7f6c37e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e6f7211b0 0x560e6fac8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.428+0000 7f6c37e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e6f7211b0 0x560e6fac8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34120/0 (socket says 172.21.15.16:34120) 2024-08-31T06:21:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.428+0000 7f6c37e00640 1 -- 172.21.15.16:0/4262851888 learned_addr learned my addr 172.21.15.16:0/4262851888 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.428+0000 7f6c3de00640 1 --2- 172.21.15.16:0/4262851888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e6f6f82f0 0x560e6fac91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.428+0000 7f6c37e00640 1 --2- 172.21.15.16:0/4262851888 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e6f7004d0 0x560e6fac9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.428+0000 7f6c37e00640 1 -- 172.21.15.16:0/4262851888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e6f6f82f0 msgr2=0x560e6fac91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.428+0000 7f6c37e00640 1 --2- 172.21.15.16:0/4262851888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e6f6f82f0 0x560e6fac91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.428+0000 7f6c37e00640 1 -- 172.21.15.16:0/4262851888 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e6f7004d0 msgr2=0x560e6fac9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.428+0000 7f6c37e00640 1 --2- 172.21.15.16:0/4262851888 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e6f7004d0 0x560e6fac9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.428+0000 7f6c37e00640 1 -- 172.21.15.16:0/4262851888 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e6f9e3c10 con 0x560e6f7211b0 2024-08-31T06:21:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.428+0000 7f6c3de00640 1 --2- 172.21.15.16:0/4262851888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e6f6f82f0 0x560e6fac91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:21:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.429+0000 7f6c37e00640 1 --2- 172.21.15.16:0/4262851888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e6f7211b0 0x560e6fac8cb0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f6c4006c5e0 tx=0x7f6c28000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806b189d6c8fc3a6 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.429+0000 7f6c2f000640 1 -- 172.21.15.16:0/4262851888 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c28044020 con 0x560e6f7211b0 2024-08-31T06:21:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.429+0000 7f6c2f000640 1 -- 172.21.15.16:0/4262851888 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c28031d10 con 0x560e6f7211b0 2024-08-31T06:21:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.429+0000 7f6c2f000640 1 -- 172.21.15.16:0/4262851888 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c28035040 con 0x560e6f7211b0 2024-08-31T06:21:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.429+0000 7f6c36a00640 1 --2- 172.21.15.16:0/2762253119 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6c240b4ca0 0x7f6c240b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.429+0000 7f6c36a00640 1 -- 172.21.15.16:0/2762253119 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8470+0+0 (secure 0 0 0) 0x7f6c380e5b60 con 0x7f6c40109920 2024-08-31T06:21:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.429+0000 7f6c3de00640 1 --2- 172.21.15.16:0/2762253119 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6c240b4ca0 0x7f6c240b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.430+0000 7f6c4be0e740 1 -- 172.21.15.16:0/4262851888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e6f7211b0 msgr2=0x560e6fac8cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.430+0000 7f6c4be0e740 1 --2- 172.21.15.16:0/4262851888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e6f7211b0 0x560e6fac8cb0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f6c4006c5e0 tx=0x7f6c28000bc0 comp rx=0 tx=0).stop 2024-08-31T06:21:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.430+0000 7f6c4be0e740 1 -- 172.21.15.16:0/4262851888 shutdown_connections 2024-08-31T06:21:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.430+0000 7f6c4be0e740 1 --2- 172.21.15.16:0/4262851888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e6f6f82f0 0x560e6fac91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.430+0000 7f6c4be0e740 1 --2- 172.21.15.16:0/4262851888 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e6f7004d0 0x560e6fac9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.430+0000 7f6c4be0e740 1 --2- 172.21.15.16:0/4262851888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e6f7211b0 0x560e6fac8cb0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.430+0000 7f6c4be0e740 1 -- 172.21.15.16:0/4262851888 >> 172.21.15.16:0/4262851888 conn(0x560e6f715300 msgr2=0x560e6fac6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.430+0000 7f6c3de00640 1 --2- 172.21.15.16:0/2762253119 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6c240b4ca0 0x7f6c240b71b0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x560e6fac7930 tx=0x7f6c30005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.430+0000 7f6c4be0e740 1 -- 172.21.15.16:0/4262851888 shutdown_connections 2024-08-31T06:21:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.430+0000 7f6c4be0e740 1 -- 172.21.15.16:0/4262851888 wait complete. 2024-08-31T06:21:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.431+0000 7f6c4be0e740 1 -- start start 2024-08-31T06:21:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.431+0000 7f6c4be0e740 1 Processor -- start 2024-08-31T06:21:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.431+0000 7f6c4be0e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e6f6f82f0 0x560e6fad42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.431+0000 7f6c4be0e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e6f7004d0 0x560e6fad47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.431+0000 7f6c4be0e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e6f7211b0 0x560e6fad4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.431+0000 7f6c4be0e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560e6f962ab0 con 0x560e6f7004d0 2024-08-31T06:21:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.431+0000 7f6c4be0e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560e6f9d0320 con 0x560e6f7211b0 2024-08-31T06:21:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.431+0000 7f6c4be0e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560e6fad73d0 con 0x560e6f6f82f0 2024-08-31T06:21:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.431+0000 7f6c4be0e740 10 client.0 _ll_register_callbacks cb 0x560e6fa09e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:21:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.432+0000 7f6c37e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e6f7004d0 0x560e6fad47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.432+0000 7f6c37e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e6f7004d0 0x560e6fad47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34124/0 (socket says 172.21.15.16:34124) 2024-08-31T06:21:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.432+0000 7f6c37e00640 1 -- 172.21.15.16:0/4225959778 learned_addr learned my addr 172.21.15.16:0/4225959778 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.432+0000 7f6c3de00640 1 --2- 172.21.15.16:0/4225959778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e6f6f82f0 0x560e6fad42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.432+0000 7f6c3e800640 1 --2- 172.21.15.16:0/4225959778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e6f7211b0 0x560e6fad4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.432+0000 7f6c37e00640 1 -- 172.21.15.16:0/4225959778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e6f6f82f0 msgr2=0x560e6fad42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.432+0000 7f6c37e00640 1 --2- 172.21.15.16:0/4225959778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e6f6f82f0 0x560e6fad42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:27.438 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.432+0000 7f6c37e00640 1 -- 172.21.15.16:0/4225959778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e6f7211b0 msgr2=0x560e6fad4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:27.438 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.432+0000 7f6c37e00640 1 --2- 172.21.15.16:0/4225959778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e6f7211b0 0x560e6fad4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:27.438 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.432+0000 7f6c37e00640 1 -- 172.21.15.16:0/4225959778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e6f78c790 con 0x560e6f7004d0 2024-08-31T06:21:27.438 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.432+0000 7f6c37e00640 1 --2- 172.21.15.16:0/4225959778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e6f7004d0 0x560e6fad47f0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f6c28031440 tx=0x7f6c28031730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84853846fa50ed00 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:27.438 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.432+0000 7f6c23400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:21:27.438 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.433+0000 7f6c23400640 1 -- 172.21.15.16:0/4225959778 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c28044070 con 0x560e6f7004d0 2024-08-31T06:21:27.438 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.433+0000 7f6c4be0e740 1 -- 172.21.15.16:0/4225959778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c28031d10 con 0x560e6f7004d0 2024-08-31T06:21:27.438 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.433+0000 7f6c23400640 1 -- 172.21.15.16:0/4225959778 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8470+0+0 (secure 0 0 0) 0x7f6c28005240 con 0x560e6f7004d0 2024-08-31T06:21:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.433+0000 7f6c23400640 1 -- 172.21.15.16:0/4225959778 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f6c28035070 con 0x560e6f7004d0 2024-08-31T06:21:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.433+0000 7f6c4be0e740 10 client.18330 fetch_fsmap learned FSMap version 50 2024-08-31T06:21:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.433+0000 7f6c4be0e740 1 -- 172.21.15.16:0/4225959778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x560e6f805620 con 0x560e6f7004d0 2024-08-31T06:21:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.433+0000 7f6c23400640 1 -- 172.21.15.16:0/4225959778 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c28031d10 con 0x560e6f7004d0 2024-08-31T06:21:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.434+0000 7f6c23400640 1 -- 172.21.15.16:0/4225959778 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 50) ==== 8358+0+0 (secure 0 0 0) 0x7f6c28049440 con 0x560e6f7004d0 2024-08-31T06:21:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.434+0000 7f6c4be0e740 10 client.18330 fetch_fsmap finished waiting for FSMap version 50 2024-08-31T06:21:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.434+0000 7f6c4be0e740 10 client.18330 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:21:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.434+0000 7f6c4be0e740 20 client.18330 populate_metadata read hostname 'smithi016' 2024-08-31T06:21:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.434+0000 7f6c4be0e740 1 --2- 172.21.15.16:0/4225959778 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x560e6f6dfd80 0x560e6fae6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.434+0000 7f6c4be0e740 4 client.18330 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:21:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.434+0000 7f6c4be0e740 1 -- 172.21.15.16:0/4225959778 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e6fa3bec0 con 0x560e6f6dfd80 2024-08-31T06:21:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.434+0000 7f6c3de00640 1 --2- 172.21.15.16:0/4225959778 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x560e6f6dfd80 0x560e6fae6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.435+0000 7f6c23400640 10 client.18330 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:21:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.435+0000 7f6c3de00640 1 --2- 172.21.15.16:0/4225959778 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x560e6f6dfd80 0x560e6fae6240 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dfc38033b936284 server_cookie=fcf3693a2d393003 in_seq=0 out_seq=0 2024-08-31T06:21:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.436+0000 7f6c23400640 1 -- 172.21.15.16:0/4225959778 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560e6fa3bec0 con 0x560e6f6dfd80 2024-08-31T06:21:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.436+0000 7f6c23400640 10 client.18330 handle_command_reply: tid=0 2024-08-31T06:21:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.436+0000 7f6c4be0e740 1 client.18330 shutdown 2024-08-31T06:21:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.436+0000 7f6c4be0e740 1 -- 172.21.15.16:0/4225959778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e6f7004d0 msgr2=0x560e6fad47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.436+0000 7f6c4be0e740 1 --2- 172.21.15.16:0/4225959778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e6f7004d0 0x560e6fad47f0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f6c28031440 tx=0x7f6c28031730 comp rx=0 tx=0).stop 2024-08-31T06:21:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.436+0000 7f6c4be0e740 1 -- 172.21.15.16:0/4225959778 shutdown_connections 2024-08-31T06:21:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.436+0000 7f6c4be0e740 1 --2- 172.21.15.16:0/4225959778 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x560e6f6dfd80 0x560e6fae6240 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.436+0000 7f6c4be0e740 1 --2- 172.21.15.16:0/4225959778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e6f7211b0 0x560e6fad4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.436+0000 7f6c4be0e740 1 --2- 172.21.15.16:0/4225959778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e6f7004d0 0x560e6fad47f0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.436+0000 7f6c4be0e740 1 --2- 172.21.15.16:0/4225959778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e6f6f82f0 0x560e6fad42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.436+0000 7f6c23400640 0 client.18330 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:21:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.436+0000 7f6c4be0e740 1 -- 172.21.15.16:0/4225959778 >> 172.21.15.16:0/4225959778 conn(0x560e6f715300 msgr2=0x560e6fac9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.436+0000 7f6c4be0e740 1 -- 172.21.15.16:0/4225959778 shutdown_connections 2024-08-31T06:21:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.436+0000 7f6c4be0e740 1 -- 172.21.15.16:0/4225959778 wait complete. 2024-08-31T06:21:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.436+0000 7f6c4be0e740 20 client.18330 trim_cache size 0 max 16384 2024-08-31T06:21:27.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.452+0000 7f6c4be0e740 1 Processor -- start 2024-08-31T06:21:27.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.452+0000 7f6c4be0e740 1 -- start start 2024-08-31T06:21:27.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.452+0000 7f6c4be0e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e6f6f82f0 0x560e6fac1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:27.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.452+0000 7f6c4be0e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e6f7004d0 0x560e6faca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:27.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.452+0000 7f6c4be0e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e6f7211b0 0x560e6fad1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:27.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.452+0000 7f6c4be0e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6c401af870 con 0x560e6f7004d0 2024-08-31T06:21:27.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.452+0000 7f6c4be0e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6c401ad0b0 con 0x560e6f7211b0 2024-08-31T06:21:27.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.452+0000 7f6c4be0e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6c401aaf40 con 0x560e6f6f82f0 2024-08-31T06:21:27.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.452+0000 7f6c37e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e6f7004d0 0x560e6faca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:27.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.453+0000 7f6c37e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e6f7004d0 0x560e6faca3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34136/0 (socket says 172.21.15.16:34136) 2024-08-31T06:21:27.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.453+0000 7f6c37e00640 1 -- 172.21.15.16:0/3327001796 learned_addr learned my addr 172.21.15.16:0/3327001796 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:27.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.453+0000 7f6c3de00640 1 --2- 172.21.15.16:0/3327001796 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e6f6f82f0 0x560e6fac1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:27.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.453+0000 7f6c3e800640 1 --2- 172.21.15.16:0/3327001796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e6f7211b0 0x560e6fad1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:27.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.453+0000 7f6c37e00640 1 -- 172.21.15.16:0/3327001796 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e6f6f82f0 msgr2=0x560e6fac1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:27.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.453+0000 7f6c37e00640 1 --2- 172.21.15.16:0/3327001796 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e6f6f82f0 0x560e6fac1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:27.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.453+0000 7f6c37e00640 1 -- 172.21.15.16:0/3327001796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e6f7211b0 msgr2=0x560e6fad1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:27.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.453+0000 7f6c37e00640 1 --2- 172.21.15.16:0/3327001796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e6f7211b0 0x560e6fad1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:27.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.453+0000 7f6c37e00640 1 -- 172.21.15.16:0/3327001796 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e6facbac0 con 0x560e6f7004d0 2024-08-31T06:21:27.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.453+0000 7f6c37e00640 1 --2- 172.21.15.16:0/3327001796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e6f7004d0 0x560e6faca3f0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f6c2802a930 tx=0x7f6c28000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c39ae9ccd1f5a6 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:27.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.453+0000 7f6c2e600640 1 -- 172.21.15.16:0/3327001796 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c28061070 con 0x560e6f7004d0 2024-08-31T06:21:27.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.454+0000 7f6c2e600640 1 -- 172.21.15.16:0/3327001796 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c28049a80 con 0x560e6f7004d0 2024-08-31T06:21:27.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.454+0000 7f6c2e600640 1 -- 172.21.15.16:0/3327001796 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c28044040 con 0x560e6f7004d0 2024-08-31T06:21:27.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.454+0000 7f6c4be0e740 1 -- 172.21.15.16:0/3327001796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e6f7004d0 msgr2=0x560e6faca3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:27.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.454+0000 7f6c4be0e740 1 --2- 172.21.15.16:0/3327001796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e6f7004d0 0x560e6faca3f0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f6c2802a930 tx=0x7f6c28000bc0 comp rx=0 tx=0).stop 2024-08-31T06:21:27.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.454+0000 7f6c4be0e740 1 -- 172.21.15.16:0/3327001796 shutdown_connections 2024-08-31T06:21:27.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.454+0000 7f6c4be0e740 1 --2- 172.21.15.16:0/3327001796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e6f7211b0 0x560e6fad1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:27.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.454+0000 7f6c4be0e740 1 --2- 172.21.15.16:0/3327001796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e6f7004d0 0x560e6faca3f0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:27.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.454+0000 7f6c4be0e740 1 --2- 172.21.15.16:0/3327001796 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e6f6f82f0 0x560e6fac1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:27.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.454+0000 7f6c4be0e740 1 -- 172.21.15.16:0/3327001796 >> 172.21.15.16:0/3327001796 conn(0x560e6f715300 msgr2=0x560e6fac9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:27.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.454+0000 7f6c4be0e740 1 -- 172.21.15.16:0/3327001796 shutdown_connections 2024-08-31T06:21:27.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.454+0000 7f6c4be0e740 1 -- 172.21.15.16:0/3327001796 wait complete. 2024-08-31T06:21:27.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.455+0000 7f6c4be0e740 1 -- start start 2024-08-31T06:21:27.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.455+0000 7f6c4be0e740 1 Processor -- start 2024-08-31T06:21:27.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.455+0000 7f6c4be0e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e6f6f82f0 0x560e6fae9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:27.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.455+0000 7f6c4be0e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e6f7004d0 0x560e6fae9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:27.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.455+0000 7f6c4be0e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e6f7211b0 0x560e6fb46ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:27.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.455+0000 7f6c4be0e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6c401ab590 con 0x560e6f7211b0 2024-08-31T06:21:27.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.455+0000 7f6c4be0e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6c401ac9e0 con 0x560e6f6f82f0 2024-08-31T06:21:27.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.455+0000 7f6c4be0e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6c401abc40 con 0x560e6f7004d0 2024-08-31T06:21:27.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.455+0000 7f6c4be0e740 10 client.0 _ll_register_callbacks cb 0x7f6c300097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:21:27.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.456+0000 7f6c37e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e6f7004d0 0x560e6fae9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:27.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.456+0000 7f6c3e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e6f7211b0 0x560e6fb46ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:27.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.456+0000 7f6c37e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e6f7004d0 0x560e6fae9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:43158/0 (socket says 172.21.15.16:43158) 2024-08-31T06:21:27.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.456+0000 7f6c37e00640 1 -- 172.21.15.16:0/4172067556 learned_addr learned my addr 172.21.15.16:0/4172067556 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:27.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.456+0000 7f6c3de00640 1 --2- 172.21.15.16:0/4172067556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e6f6f82f0 0x560e6fae9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:27.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.456+0000 7f6c37e00640 1 -- 172.21.15.16:0/4172067556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e6f6f82f0 msgr2=0x560e6fae9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:27.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.456+0000 7f6c37e00640 1 --2- 172.21.15.16:0/4172067556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e6f6f82f0 0x560e6fae9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:27.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.456+0000 7f6c37e00640 1 -- 172.21.15.16:0/4172067556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e6f7211b0 msgr2=0x560e6fb46ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:27.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.456+0000 7f6c37e00640 1 --2- 172.21.15.16:0/4172067556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e6f7211b0 0x560e6fb46ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:27.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.456+0000 7f6c37e00640 1 -- 172.21.15.16:0/4172067556 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e6fb47240 con 0x560e6f7004d0 2024-08-31T06:21:27.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.456+0000 7f6c3e800640 1 --2- 172.21.15.16:0/4172067556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e6f7211b0 0x560e6fb46ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:21:27.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.456+0000 7f6c37e00640 1 --2- 172.21.15.16:0/4172067556 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e6f7004d0 0x560e6fae9620 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6c28049930 tx=0x7f6c2802a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62c64b1185bd551f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:27.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.456+0000 7f6c23400640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:21:27.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.457+0000 7f6c23400640 1 -- 172.21.15.16:0/4172067556 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c28044070 con 0x560e6f7004d0 2024-08-31T06:21:27.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.457+0000 7f6c4be0e740 1 -- 172.21.15.16:0/4172067556 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c28049a80 con 0x560e6f7004d0 2024-08-31T06:21:27.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.457+0000 7f6c23400640 1 -- 172.21.15.16:0/4172067556 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8470+0+0 (secure 0 0 0) 0x7f6c28005440 con 0x560e6f7004d0 2024-08-31T06:21:27.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.457+0000 7f6c23400640 1 -- 172.21.15.16:0/4172067556 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f6c28061070 con 0x560e6f7004d0 2024-08-31T06:21:27.466 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.457+0000 7f6c4be0e740 10 client.28837 fetch_fsmap learned FSMap version 50 2024-08-31T06:21:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.457+0000 7f6c4be0e740 1 -- 172.21.15.16:0/4172067556 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x560e6fb50820 con 0x560e6f7004d0 2024-08-31T06:21:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.458+0000 7f6c23400640 1 -- 172.21.15.16:0/4172067556 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c28049a80 con 0x560e6f7004d0 2024-08-31T06:21:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.458+0000 7f6c23400640 1 -- 172.21.15.16:0/4172067556 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 50) ==== 8358+0+0 (secure 0 0 0) 0x7f6c2805c510 con 0x560e6f7004d0 2024-08-31T06:21:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.458+0000 7f6c4be0e740 10 client.28837 fetch_fsmap finished waiting for FSMap version 50 2024-08-31T06:21:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.458+0000 7f6c4be0e740 10 client.28837 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:21:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.458+0000 7f6c4be0e740 20 client.28837 populate_metadata read hostname 'smithi016' 2024-08-31T06:21:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.458+0000 7f6c4be0e740 1 --2- 172.21.15.16:0/4172067556 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x560e6f6b51c0 0x560e6fb52f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.458+0000 7f6c4be0e740 4 client.28837 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:21:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.458+0000 7f6c4be0e740 1 -- 172.21.15.16:0/4172067556 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560e6fa37960 con 0x560e6f6b51c0 2024-08-31T06:21:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.458+0000 7f6c3de00640 1 --2- 172.21.15.16:0/4172067556 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x560e6f6b51c0 0x560e6fb52f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.459+0000 7f6c3de00640 1 --2- 172.21.15.16:0/4172067556 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x560e6f6b51c0 0x560e6fb52f40 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d5c3117c9c6efb6 server_cookie=b0ecbfc04a91b1d1 in_seq=0 out_seq=0 2024-08-31T06:21:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.459+0000 7f6c23400640 10 client.28837 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:21:27.480 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.476+0000 7f6c23400640 1 -- 172.21.15.16:0/4172067556 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560e6fa37960 con 0x560e6f6b51c0 2024-08-31T06:21:27.480 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.476+0000 7f6c23400640 10 client.28837 handle_command_reply: tid=0 2024-08-31T06:21:27.480 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.476+0000 7f6c4be0e740 1 client.28837 shutdown 2024-08-31T06:21:27.480 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.476+0000 7f6c4be0e740 1 -- 172.21.15.16:0/4172067556 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e6f7004d0 msgr2=0x560e6fae9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:27.480 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.476+0000 7f6c4be0e740 1 --2- 172.21.15.16:0/4172067556 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e6f7004d0 0x560e6fae9620 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6c28049930 tx=0x7f6c2802a410 comp rx=0 tx=0).stop 2024-08-31T06:21:27.480 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.476+0000 7f6c4be0e740 1 -- 172.21.15.16:0/4172067556 shutdown_connections 2024-08-31T06:21:27.480 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.476+0000 7f6c4be0e740 1 --2- 172.21.15.16:0/4172067556 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x560e6f6b51c0 0x560e6fb52f40 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:27.480 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.476+0000 7f6c4be0e740 1 --2- 172.21.15.16:0/4172067556 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e6f7004d0 0x560e6fae9620 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.476+0000 7f6c23400640 0 client.28837 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:21:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.476+0000 7f6c4be0e740 1 --2- 172.21.15.16:0/4172067556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e6f7211b0 0x560e6fb46ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.476+0000 7f6c4be0e740 1 --2- 172.21.15.16:0/4172067556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e6f6f82f0 0x560e6fae9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.476+0000 7f6c4be0e740 1 -- 172.21.15.16:0/4172067556 >> 172.21.15.16:0/4172067556 conn(0x560e6f715300 msgr2=0x560e6fac1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.476+0000 7f6c4be0e740 1 -- 172.21.15.16:0/4172067556 shutdown_connections 2024-08-31T06:21:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.476+0000 7f6c4be0e740 1 -- 172.21.15.16:0/4172067556 wait complete. 2024-08-31T06:21:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.476+0000 7f6c4be0e740 20 client.28837 trim_cache size 0 max 16384 2024-08-31T06:21:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:21:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:21:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:21:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:21:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.477+0000 7f6c3f200640 1 -- 172.21.15.16:0/2762253119 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6c240b4ca0 msgr2=0x7f6c240b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.477+0000 7f6c3f200640 1 --2- 172.21.15.16:0/2762253119 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6c240b4ca0 0x7f6c240b71b0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x560e6fac7930 tx=0x7f6c30005f90 comp rx=0 tx=0).stop 2024-08-31T06:21:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.477+0000 7f6c3f200640 1 -- 172.21.15.16:0/2762253119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c40109920 msgr2=0x7f6c4019f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.477+0000 7f6c3f200640 1 --2- 172.21.15.16:0/2762253119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c40109920 0x7f6c4019f840 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f6c3800eb10 tx=0x7f6c3800c2d0 comp rx=0 tx=0).stop 2024-08-31T06:21:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.477+0000 7f6c3f200640 1 -- 172.21.15.16:0/2762253119 shutdown_connections 2024-08-31T06:21:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.477+0000 7f6c3f200640 1 --2- 172.21.15.16:0/2762253119 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6c240b4ca0 0x7f6c240b71b0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.477+0000 7f6c3f200640 1 --2- 172.21.15.16:0/2762253119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c40108fd0 0x7f6c4019a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:27.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.477+0000 7f6c3f200640 1 --2- 172.21.15.16:0/2762253119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c40109920 0x7f6c4019f840 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:27.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.477+0000 7f6c3f200640 1 --2- 172.21.15.16:0/2762253119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c401036a0 0x7f6c4019a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:27.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.477+0000 7f6c3f200640 1 -- 172.21.15.16:0/2762253119 >> 172.21.15.16:0/2762253119 conn(0x7f6c400ff100 msgr2=0x7f6c40101520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:27.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.477+0000 7f6c3f200640 1 -- 172.21.15.16:0/2762253119 shutdown_connections 2024-08-31T06:21:27.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:27.477+0000 7f6c3f200640 1 -- 172.21.15.16:0/2762253119 wait complete. 2024-08-31T06:21:29.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:28 smithi155 ceph-mon[21742]: pgmap v809: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 2.7 MiB/s wr, 326 op/s 2024-08-31T06:21:29.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:28 smithi016 ceph-mon[17194]: pgmap v809: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 2.7 MiB/s wr, 326 op/s 2024-08-31T06:21:29.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:28 smithi184 ceph-mon[22331]: pgmap v809: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 2.7 MiB/s wr, 326 op/s 2024-08-31T06:21:31.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:30 smithi155 ceph-mon[21742]: pgmap v810: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 2.6 MiB/s wr, 321 op/s 2024-08-31T06:21:31.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:30 smithi016 ceph-mon[17194]: pgmap v810: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 2.6 MiB/s wr, 321 op/s 2024-08-31T06:21:31.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:30 smithi184 ceph-mon[22331]: pgmap v810: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 2.6 MiB/s wr, 321 op/s 2024-08-31T06:21:31.627 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 845.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-31T06:21:33.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:32 smithi155 ceph-mon[21742]: pgmap v811: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.5 MiB/s wr, 309 op/s 2024-08-31T06:21:33.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:32 smithi016 ceph-mon[17194]: pgmap v811: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.5 MiB/s wr, 309 op/s 2024-08-31T06:21:33.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:32 smithi184 ceph-mon[22331]: pgmap v811: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.5 MiB/s wr, 309 op/s 2024-08-31T06:21:34.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:33 smithi155 ceph-mon[21742]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:21:34.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:33 smithi016 ceph-mon[17194]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:21:34.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:33 smithi184 ceph-mon[22331]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:21:35.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:34 smithi155 ceph-mon[21742]: pgmap v812: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 2.2 MiB/s wr, 265 op/s 2024-08-31T06:21:35.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:34 smithi016 ceph-mon[17194]: pgmap v812: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 2.2 MiB/s wr, 265 op/s 2024-08-31T06:21:35.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:34 smithi184 ceph-mon[22331]: pgmap v812: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 2.2 MiB/s wr, 265 op/s 2024-08-31T06:21:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 850.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-31T06:21:36.643 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:21:37.037 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:21:37.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:36 smithi155 ceph-mon[21742]: pgmap v813: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 1.5 MiB/s wr, 180 op/s 2024-08-31T06:21:37.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:36 smithi016 ceph-mon[17194]: pgmap v813: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 1.5 MiB/s wr, 180 op/s 2024-08-31T06:21:37.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:36 smithi184 ceph-mon[22331]: pgmap v813: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 1.5 MiB/s wr, 180 op/s 2024-08-31T06:21:37.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.706+0000 7f72b3e00640 1 -- 172.21.15.16:0/2946396298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b4075ea0 msgr2=0x7f72b410f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:37.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.706+0000 7f72b3e00640 1 --2- 172.21.15.16:0/2946396298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b4075ea0 0x7f72b410f770 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f72a0009900 tx=0x7f72a002a410 comp rx=0 tx=0).stop 2024-08-31T06:21:37.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.706+0000 7f72b3e00640 1 -- 172.21.15.16:0/2946396298 shutdown_connections 2024-08-31T06:21:37.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.706+0000 7f72b3e00640 1 --2- 172.21.15.16:0/2946396298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72b410fcb0 0x7f72b41120f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:37.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.706+0000 7f72b3e00640 1 --2- 172.21.15.16:0/2946396298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b4075ea0 0x7f72b410f770 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:37.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.706+0000 7f72b3e00640 1 --2- 172.21.15.16:0/2946396298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72b4075550 0x7f72b4075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:37.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.706+0000 7f72b3e00640 1 -- 172.21.15.16:0/2946396298 >> 172.21.15.16:0/2946396298 conn(0x7f72b40ff1e0 msgr2=0x7f72b4101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:37.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.706+0000 7f72b3e00640 1 -- 172.21.15.16:0/2946396298 shutdown_connections 2024-08-31T06:21:37.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.706+0000 7f72b3e00640 1 -- 172.21.15.16:0/2946396298 wait complete. 2024-08-31T06:21:37.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.707+0000 7f72b3e00640 1 Processor -- start 2024-08-31T06:21:37.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.707+0000 7f72b3e00640 1 -- start start 2024-08-31T06:21:37.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.707+0000 7f72b3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b4075550 0x7f72b419c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:37.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.707+0000 7f72b3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72b4075ea0 0x7f72b419c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:37.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.707+0000 7f72b3e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72b410fcb0 0x7f72b41a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:37.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.707+0000 7f72b3e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f72b41158b0 con 0x7f72b4075550 2024-08-31T06:21:37.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.707+0000 7f72b3e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f72b4115730 con 0x7f72b4075ea0 2024-08-31T06:21:37.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.707+0000 7f72b3e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f72b4115a30 con 0x7f72b410fcb0 2024-08-31T06:21:37.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.708+0000 7f72b2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b4075550 0x7f72b419c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:37.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.708+0000 7f72b2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b4075550 0x7f72b419c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46162/0 (socket says 172.21.15.16:46162) 2024-08-31T06:21:37.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.708+0000 7f72b2a00640 1 -- 172.21.15.16:0/3287606780 learned_addr learned my addr 172.21.15.16:0/3287606780 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:37.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.708+0000 7f72b2000640 1 --2- 172.21.15.16:0/3287606780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72b4075ea0 0x7f72b419c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:37.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.708+0000 7f72b3400640 1 --2- 172.21.15.16:0/3287606780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72b410fcb0 0x7f72b41a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:37.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.708+0000 7f72b2a00640 1 -- 172.21.15.16:0/3287606780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72b410fcb0 msgr2=0x7f72b41a1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:37.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.708+0000 7f72b2a00640 1 --2- 172.21.15.16:0/3287606780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72b410fcb0 0x7f72b41a1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:37.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.708+0000 7f72b2a00640 1 -- 172.21.15.16:0/3287606780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72b4075ea0 msgr2=0x7f72b419c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:37.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.708+0000 7f72b2a00640 1 --2- 172.21.15.16:0/3287606780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72b4075ea0 0x7f72b419c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:37.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.708+0000 7f72b2a00640 1 -- 172.21.15.16:0/3287606780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b41a2210 con 0x7f72b4075550 2024-08-31T06:21:37.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.709+0000 7f72b2a00640 1 --2- 172.21.15.16:0/3287606780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b4075550 0x7f72b419c2a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f72a400e9f0 tx=0x7f72a400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:37.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.709+0000 7f72ab400640 1 -- 172.21.15.16:0/3287606780 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a400cd80 con 0x7f72b4075550 2024-08-31T06:21:37.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.709+0000 7f72ab400640 1 -- 172.21.15.16:0/3287606780 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72a40054d0 con 0x7f72b4075550 2024-08-31T06:21:37.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.709+0000 7f72ab400640 1 -- 172.21.15.16:0/3287606780 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a4010690 con 0x7f72b4075550 2024-08-31T06:21:37.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.709+0000 7f72b3e00640 1 -- 172.21.15.16:0/3287606780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72b41a2500 con 0x7f72b4075550 2024-08-31T06:21:37.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.709+0000 7f72b3e00640 1 -- 172.21.15.16:0/3287606780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72b41a8c60 con 0x7f72b4075550 2024-08-31T06:21:37.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.710+0000 7f72b3e00640 1 -- 172.21.15.16:0/3287606780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72b4075960 con 0x7f72b4075550 2024-08-31T06:21:37.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.712+0000 7f72ab400640 1 -- 172.21.15.16:0/3287606780 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f72a4004e30 con 0x7f72b4075550 2024-08-31T06:21:37.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.714+0000 7f72ab400640 1 --2- 172.21.15.16:0/3287606780 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f72980b4d20 0x7f72980b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:37.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.714+0000 7f72ab400640 1 -- 172.21.15.16:0/3287606780 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8470+0+0 (secure 0 0 0) 0x7f72a40e5af0 con 0x7f72b4075550 2024-08-31T06:21:37.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.714+0000 7f72b2000640 1 --2- 172.21.15.16:0/3287606780 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f72980b4d20 0x7f72980b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:37.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.715+0000 7f72b2000640 1 --2- 172.21.15.16:0/3287606780 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f72980b4d20 0x7f72980b7230 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f72a0031440 tx=0x7f72a0005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:37.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.716+0000 7f72ab400640 1 -- 172.21.15.16:0/3287606780 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f72a4056130 con 0x7f72b4075550 2024-08-31T06:21:37.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.950+0000 7f72b3e00640 1 -- 172.21.15.16:0/3287606780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72b419d3d0 con 0x7f72b4075550 2024-08-31T06:21:37.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.950+0000 7f72ab400640 1 -- 172.21.15.16:0/3287606780 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+11920 (secure 0 0 0) 0x7f72a40ac8b0 con 0x7f72b4075550 2024-08-31T06:21:37.953 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:21:37.953 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":51,"btime":"2024-08-31T06:21:32:734970+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":51,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:21:32.045138+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24737,24724,24712]},"id":1}]} 2024-08-31T06:21:37.954 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 51 2024-08-31T06:21:37.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.953+0000 7f72b3e00640 1 -- 172.21.15.16:0/3287606780 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f72980b4d20 msgr2=0x7f72980b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:37.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.953+0000 7f72b3e00640 1 --2- 172.21.15.16:0/3287606780 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f72980b4d20 0x7f72980b7230 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f72a0031440 tx=0x7f72a0005a20 comp rx=0 tx=0).stop 2024-08-31T06:21:37.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.953+0000 7f72b3e00640 1 -- 172.21.15.16:0/3287606780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b4075550 msgr2=0x7f72b419c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:37.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.953+0000 7f72b3e00640 1 --2- 172.21.15.16:0/3287606780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b4075550 0x7f72b419c2a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f72a400e9f0 tx=0x7f72a400edf0 comp rx=0 tx=0).stop 2024-08-31T06:21:37.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.953+0000 7f72b3e00640 1 -- 172.21.15.16:0/3287606780 shutdown_connections 2024-08-31T06:21:37.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.953+0000 7f72b3e00640 1 --2- 172.21.15.16:0/3287606780 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f72980b4d20 0x7f72980b7230 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:37.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.953+0000 7f72b3e00640 1 --2- 172.21.15.16:0/3287606780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72b410fcb0 0x7f72b41a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:37.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.953+0000 7f72b3e00640 1 --2- 172.21.15.16:0/3287606780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72b4075ea0 0x7f72b419c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:37.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.953+0000 7f72b3e00640 1 --2- 172.21.15.16:0/3287606780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b4075550 0x7f72b419c2a0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:37.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.953+0000 7f72b3e00640 1 -- 172.21.15.16:0/3287606780 >> 172.21.15.16:0/3287606780 conn(0x7f72b40ff1e0 msgr2=0x7f72b4109830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:37.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.953+0000 7f72b3e00640 1 -- 172.21.15.16:0/3287606780 shutdown_connections 2024-08-31T06:21:37.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:37.953+0000 7f72b3e00640 1 -- 172.21.15.16:0/3287606780 wait complete. 2024-08-31T06:21:39.035 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:38 smithi016 ceph-mon[17194]: pgmap v814: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 80 KiB/s wr, 16 op/s 2024-08-31T06:21:39.035 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:38 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/3287606780' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:21:39.081 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24737 quiesce db --set-id 603ae143 --release --await' 2024-08-31T06:21:39.081 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24737 quiesce db --set-id 603ae143 --release --await 2024-08-31T06:21:39.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:38 smithi155 ceph-mon[21742]: pgmap v814: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 80 KiB/s wr, 16 op/s 2024-08-31T06:21:39.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:38 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/3287606780' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:21:39.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:38 smithi184 ceph-mon[22331]: pgmap v814: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 80 KiB/s wr, 16 op/s 2024-08-31T06:21:39.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:38 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/3287606780' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:21:39.475 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:21:40.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.410+0000 7f799fe00640 1 -- 172.21.15.16:0/38626972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79a0075b70 msgr2=0x7f79a0075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:40.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.410+0000 7f799fe00640 1 --2- 172.21.15.16:0/38626972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79a0075b70 0x7f79a0075f80 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f798c009900 tx=0x7f798c02a3f0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.410+0000 7f799fe00640 1 -- 172.21.15.16:0/38626972 shutdown_connections 2024-08-31T06:21:40.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.410+0000 7f799fe00640 1 --2- 172.21.15.16:0/38626972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79a0114020 0x7f79a01164f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.410+0000 7f799fe00640 1 --2- 172.21.15.16:0/38626972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79a0075b70 0x7f79a0075f80 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.410+0000 7f799fe00640 1 --2- 172.21.15.16:0/38626972 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79a0077120 0x7f79a0075550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.410+0000 7f799fe00640 1 -- 172.21.15.16:0/38626972 >> 172.21.15.16:0/38626972 conn(0x7f79a00ff1e0 msgr2=0x7f79a0101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:40.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.410+0000 7f799fe00640 1 -- 172.21.15.16:0/38626972 shutdown_connections 2024-08-31T06:21:40.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.410+0000 7f799fe00640 1 -- 172.21.15.16:0/38626972 wait complete. 2024-08-31T06:21:40.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.411+0000 7f799fe00640 1 Processor -- start 2024-08-31T06:21:40.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.412+0000 7f799fe00640 1 -- start start 2024-08-31T06:21:40.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.412+0000 7f799fe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79a0075b70 0x7f79a01a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:40.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.412+0000 7f799fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79a0077120 0x7f79a01a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:40.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.412+0000 7f799fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79a0114020 0x7f79a01a5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:40.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.412+0000 7f799fe00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f79a0119c60 con 0x7f79a0114020 2024-08-31T06:21:40.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.412+0000 7f799fe00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f79a0119ae0 con 0x7f79a0077120 2024-08-31T06:21:40.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.412+0000 7f799fe00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f79a0119de0 con 0x7f79a0075b70 2024-08-31T06:21:40.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.412+0000 7f799f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79a0114020 0x7f79a01a5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:40.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.412+0000 7f799f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79a0114020 0x7f79a01a5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46182/0 (socket says 172.21.15.16:46182) 2024-08-31T06:21:40.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.412+0000 7f799f400640 1 -- 172.21.15.16:0/2244129051 learned_addr learned my addr 172.21.15.16:0/2244129051 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:40.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.412+0000 7f799e000640 1 --2- 172.21.15.16:0/2244129051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79a0077120 0x7f79a01a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:40.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.413+0000 7f799ea00640 1 --2- 172.21.15.16:0/2244129051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79a0075b70 0x7f79a01a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:40.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.413+0000 7f799e000640 1 -- 172.21.15.16:0/2244129051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79a0075b70 msgr2=0x7f79a01a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:40.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.413+0000 7f799e000640 1 --2- 172.21.15.16:0/2244129051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79a0075b70 0x7f79a01a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.413+0000 7f799e000640 1 -- 172.21.15.16:0/2244129051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79a0114020 msgr2=0x7f79a01a5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:40.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.413+0000 7f799e000640 1 --2- 172.21.15.16:0/2244129051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79a0114020 0x7f79a01a5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.413+0000 7f799e000640 1 -- 172.21.15.16:0/2244129051 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a01a65e0 con 0x7f79a0077120 2024-08-31T06:21:40.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.413+0000 7f799f400640 1 --2- 172.21.15.16:0/2244129051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79a0114020 0x7f79a01a5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:21:40.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.413+0000 7f799e000640 1 --2- 172.21.15.16:0/2244129051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79a0077120 0x7f79a01a0c10 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f798c02a910 tx=0x7f798c031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:40.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.414+0000 7f7997400640 1 -- 172.21.15.16:0/2244129051 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c03f070 con 0x7f79a0077120 2024-08-31T06:21:40.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.414+0000 7f799fe00640 1 -- 172.21.15.16:0/2244129051 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79a01a6870 con 0x7f79a0077120 2024-08-31T06:21:40.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.414+0000 7f7997400640 1 -- 172.21.15.16:0/2244129051 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f798c004490 con 0x7f79a0077120 2024-08-31T06:21:40.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.414+0000 7f7997400640 1 -- 172.21.15.16:0/2244129051 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c035070 con 0x7f79a0077120 2024-08-31T06:21:40.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.414+0000 7f799fe00640 1 -- 172.21.15.16:0/2244129051 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79a01ad030 con 0x7f79a0077120 2024-08-31T06:21:40.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.417+0000 7f79ac659740 1 Processor -- start 2024-08-31T06:21:40.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.417+0000 7f79ac659740 1 -- start start 2024-08-31T06:21:40.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.418+0000 7f79ac659740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a44a534520 0x55a44a8da090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:40.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.418+0000 7f79ac659740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a44a50b660 0x55a44a8da5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:40.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.418+0000 7f79ac659740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44a513840 0x55a44a8dab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:40.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.418+0000 7f79ac659740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a44a850a90 con 0x55a44a50b660 2024-08-31T06:21:40.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.418+0000 7f79ac659740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a44a840020 con 0x55a44a534520 2024-08-31T06:21:40.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.418+0000 7f79ac659740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a44a843fd0 con 0x55a44a513840 2024-08-31T06:21:40.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.418+0000 7f799ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a44a50b660 0x55a44a8da5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:40.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.418+0000 7f799f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a44a534520 0x55a44a8da090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:40.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.418+0000 7f799ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a44a50b660 0x55a44a8da5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46192/0 (socket says 172.21.15.16:46192) 2024-08-31T06:21:40.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.418+0000 7f799ea00640 1 -- 172.21.15.16:0/810789329 learned_addr learned my addr 172.21.15.16:0/810789329 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:40.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.418+0000 7f7997400640 1 -- 172.21.15.16:0/2244129051 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f798c03b460 con 0x7f79a0077120 2024-08-31T06:21:40.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.419+0000 7f799ea00640 1 -- 172.21.15.16:0/810789329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44a513840 msgr2=0x55a44a8dab10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:21:40.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.419+0000 7f799ea00640 1 --2- 172.21.15.16:0/810789329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44a513840 0x55a44a8dab10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.419+0000 7f799ea00640 1 -- 172.21.15.16:0/810789329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a44a534520 msgr2=0x55a44a8da090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:40.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.419+0000 7f799ea00640 1 --2- 172.21.15.16:0/810789329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a44a534520 0x55a44a8da090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.419+0000 7f799ea00640 1 -- 172.21.15.16:0/810789329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a44a59fb00 con 0x55a44a50b660 2024-08-31T06:21:40.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.419+0000 7f799f400640 1 --2- 172.21.15.16:0/810789329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a44a534520 0x55a44a8da090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:21:40.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.419+0000 7f799ea00640 1 --2- 172.21.15.16:0/810789329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a44a50b660 0x55a44a8da5d0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f7990009d20 tx=0x7f7990011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca395e7fcff3ef46 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:40.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.419+0000 7f798be00640 1 -- 172.21.15.16:0/810789329 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799001e070 con 0x55a44a50b660 2024-08-31T06:21:40.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.419+0000 7f798be00640 1 -- 172.21.15.16:0/810789329 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7990009e10 con 0x55a44a50b660 2024-08-31T06:21:40.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.419+0000 7f798be00640 1 -- 172.21.15.16:0/810789329 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799000e9b0 con 0x55a44a50b660 2024-08-31T06:21:40.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.420+0000 7f79ac659740 1 -- 172.21.15.16:0/810789329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a44a50b660 msgr2=0x55a44a8da5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:40.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.420+0000 7f79ac659740 1 --2- 172.21.15.16:0/810789329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a44a50b660 0x55a44a8da5d0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f7990009d20 tx=0x7f7990011810 comp rx=0 tx=0).stop 2024-08-31T06:21:40.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.420+0000 7f7997400640 1 --2- 172.21.15.16:0/2244129051 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f79840b4ca0 0x7f79840b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:40.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.421+0000 7f79ac659740 1 -- 172.21.15.16:0/810789329 shutdown_connections 2024-08-31T06:21:40.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.421+0000 7f79ac659740 1 --2- 172.21.15.16:0/810789329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44a513840 0x55a44a8dab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.421+0000 7f79ac659740 1 --2- 172.21.15.16:0/810789329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a44a50b660 0x55a44a8da5d0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.421+0000 7f79ac659740 1 --2- 172.21.15.16:0/810789329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a44a534520 0x55a44a8da090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.421+0000 7f79ac659740 1 -- 172.21.15.16:0/810789329 >> 172.21.15.16:0/810789329 conn(0x55a44a528910 msgr2=0x55a44a8d7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:40.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.421+0000 7f7997400640 1 -- 172.21.15.16:0/2244129051 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8470+0+0 (secure 0 0 0) 0x7f798c1059e0 con 0x7f79a0077120 2024-08-31T06:21:40.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.421+0000 7f799f400640 1 --2- 172.21.15.16:0/2244129051 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f79840b4ca0 0x7f79840b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:40.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.421+0000 7f79ac659740 1 -- 172.21.15.16:0/810789329 shutdown_connections 2024-08-31T06:21:40.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.421+0000 7f79ac659740 1 -- 172.21.15.16:0/810789329 wait complete. 2024-08-31T06:21:40.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.422+0000 7f799f400640 1 --2- 172.21.15.16:0/2244129051 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f79840b4ca0 0x7f79840b71b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f79980049d0 tx=0x7f7998009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:40.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.422+0000 7f79ac659740 1 -- start start 2024-08-31T06:21:40.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.422+0000 7f79ac659740 1 Processor -- start 2024-08-31T06:21:40.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.422+0000 7f79ac659740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44a50b660 0x55a44a8e54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:40.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.422+0000 7f79ac659740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a44a513840 0x55a44a8e5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:40.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.422+0000 7f79ac659740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a44a534520 0x55a44a8e5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:40.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.422+0000 7f79ac659740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a44a6a2bb0 con 0x55a44a534520 2024-08-31T06:21:40.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.422+0000 7f79ac659740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a44a7c6f90 con 0x55a44a513840 2024-08-31T06:21:40.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.422+0000 7f79ac659740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a44a8e8870 con 0x55a44a50b660 2024-08-31T06:21:40.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.423+0000 7f79ac659740 10 client.0 _ll_register_callbacks cb 0x55a44a7ec710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:21:40.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.423+0000 7f799ea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44a50b660 0x55a44a8e54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:40.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.423+0000 7f799f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a44a534520 0x55a44a8e5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:40.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.423+0000 7f799ea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44a50b660 0x55a44a8e54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:43598/0 (socket says 172.21.15.16:43598) 2024-08-31T06:21:40.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.423+0000 7f799ea00640 1 -- 172.21.15.16:0/3420652170 learned_addr learned my addr 172.21.15.16:0/3420652170 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:40.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.423+0000 7f799e000640 1 --2- 172.21.15.16:0/3420652170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a44a513840 0x55a44a8e5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:40.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.423+0000 7f799ea00640 1 -- 172.21.15.16:0/3420652170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a44a513840 msgr2=0x55a44a8e5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:40.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.423+0000 7f799ea00640 1 --2- 172.21.15.16:0/3420652170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a44a513840 0x55a44a8e5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.423+0000 7f799ea00640 1 -- 172.21.15.16:0/3420652170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a44a534520 msgr2=0x55a44a8e5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:40.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.423+0000 7f799ea00640 1 --2- 172.21.15.16:0/3420652170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a44a534520 0x55a44a8e5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.423+0000 7f799ea00640 1 -- 172.21.15.16:0/3420652170 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a44a616960 con 0x55a44a50b660 2024-08-31T06:21:40.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.424+0000 7f799ea00640 1 --2- 172.21.15.16:0/3420652170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44a50b660 0x55a44a8e54f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f7990005890 tx=0x7f7990005730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8b10d7ffd1a383b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:40.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.424+0000 7f7988c00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:21:40.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.424+0000 7f7988c00640 1 -- 172.21.15.16:0/3420652170 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799001e070 con 0x55a44a50b660 2024-08-31T06:21:40.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.424+0000 7f79ac659740 1 -- 172.21.15.16:0/3420652170 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a44a7dee30 con 0x55a44a50b660 2024-08-31T06:21:40.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.424+0000 7f7988c00640 1 -- 172.21.15.16:0/3420652170 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8470+0+0 (secure 0 0 0) 0x7f7990005440 con 0x55a44a50b660 2024-08-31T06:21:40.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.424+0000 7f7988c00640 1 -- 172.21.15.16:0/3420652170 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f7990031050 con 0x55a44a50b660 2024-08-31T06:21:40.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.424+0000 7f79ac659740 10 client.28852 fetch_fsmap learned FSMap version 51 2024-08-31T06:21:40.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.424+0000 7f79ac659740 1 -- 172.21.15.16:0/3420652170 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55a44a8f42c0 con 0x55a44a50b660 2024-08-31T06:21:40.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.425+0000 7f7988c00640 1 -- 172.21.15.16:0/3420652170 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a44a7dee30 con 0x55a44a50b660 2024-08-31T06:21:40.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.425+0000 7f7988c00640 1 -- 172.21.15.16:0/3420652170 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 51) ==== 8354+0+0 (secure 0 0 0) 0x7f7990004850 con 0x55a44a50b660 2024-08-31T06:21:40.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.425+0000 7f79ac659740 10 client.28852 fetch_fsmap finished waiting for FSMap version 51 2024-08-31T06:21:40.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.425+0000 7f79ac659740 10 client.28852 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:21:40.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.425+0000 7f79ac659740 20 client.28852 populate_metadata read hostname 'smithi016' 2024-08-31T06:21:40.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.425+0000 7f79ac659740 1 --2- 172.21.15.16:0/3420652170 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a44a4f2ea0 0x55a44a8f7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:40.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.425+0000 7f79ac659740 4 client.28852 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:21:40.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.425+0000 7f79ac659740 1 -- 172.21.15.16:0/3420652170 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a44a8529e0 con 0x55a44a4f2ea0 2024-08-31T06:21:40.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.426+0000 7f799e000640 1 --2- 172.21.15.16:0/3420652170 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a44a4f2ea0 0x55a44a8f7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:40.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.426+0000 7f799e000640 1 --2- 172.21.15.16:0/3420652170 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a44a4f2ea0 0x55a44a8f7680 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7a7b82f09bea7df server_cookie=94ad84c4451b525a in_seq=0 out_seq=0 2024-08-31T06:21:40.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.426+0000 7f7988c00640 10 client.28852 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:21:40.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.427+0000 7f7988c00640 1 -- 172.21.15.16:0/3420652170 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a44a8529e0 con 0x55a44a4f2ea0 2024-08-31T06:21:40.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.427+0000 7f7988c00640 10 client.28852 handle_command_reply: tid=0 2024-08-31T06:21:40.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.427+0000 7f79ac659740 1 client.28852 shutdown 2024-08-31T06:21:40.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.427+0000 7f79ac659740 1 -- 172.21.15.16:0/3420652170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44a50b660 msgr2=0x55a44a8e54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:40.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.427+0000 7f79ac659740 1 --2- 172.21.15.16:0/3420652170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44a50b660 0x55a44a8e54f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f7990005890 tx=0x7f7990005730 comp rx=0 tx=0).stop 2024-08-31T06:21:40.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.428+0000 7f79ac659740 1 -- 172.21.15.16:0/3420652170 shutdown_connections 2024-08-31T06:21:40.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.428+0000 7f79ac659740 1 --2- 172.21.15.16:0/3420652170 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a44a4f2ea0 0x55a44a8f7680 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.428+0000 7f79ac659740 1 --2- 172.21.15.16:0/3420652170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a44a534520 0x55a44a8e5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.428+0000 7f79ac659740 1 --2- 172.21.15.16:0/3420652170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a44a513840 0x55a44a8e5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.428+0000 7f79ac659740 1 --2- 172.21.15.16:0/3420652170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44a50b660 0x55a44a8e54f0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.428+0000 7f7988c00640 0 client.28852 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:21:40.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.428+0000 7f79ac659740 1 -- 172.21.15.16:0/3420652170 >> 172.21.15.16:0/3420652170 conn(0x55a44a528910 msgr2=0x55a44a8dd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:40.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.428+0000 7f79ac659740 1 -- 172.21.15.16:0/3420652170 shutdown_connections 2024-08-31T06:21:40.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.428+0000 7f79ac659740 1 -- 172.21.15.16:0/3420652170 wait complete. 2024-08-31T06:21:40.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.428+0000 7f79ac659740 20 client.28852 trim_cache size 0 max 16384 2024-08-31T06:21:40.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.447+0000 7f79ac659740 1 Processor -- start 2024-08-31T06:21:40.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.447+0000 7f79ac659740 1 -- start start 2024-08-31T06:21:40.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.447+0000 7f79ac659740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a44a50b660 0x55a44a8e5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:40.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.447+0000 7f79ac659740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44a513840 0x55a44a8dba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.447+0000 7f79ac659740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a44a534520 0x55a44a8f1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.447+0000 7f79ac659740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f79a01b80e0 con 0x55a44a534520 2024-08-31T06:21:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.447+0000 7f79ac659740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f79a01b5950 con 0x55a44a50b660 2024-08-31T06:21:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.447+0000 7f79ac659740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f79a01b37c0 con 0x55a44a513840 2024-08-31T06:21:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.448+0000 7f799ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a44a50b660 0x55a44a8e5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.448+0000 7f799e000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44a513840 0x55a44a8dba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.448+0000 7f799ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a44a50b660 0x55a44a8e5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:41168/0 (socket says 172.21.15.16:41168) 2024-08-31T06:21:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.448+0000 7f799ea00640 1 -- 172.21.15.16:0/1494454787 learned_addr learned my addr 172.21.15.16:0/1494454787 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.448+0000 7f799f400640 1 --2- 172.21.15.16:0/1494454787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a44a534520 0x55a44a8f1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.448+0000 7f799e000640 1 -- 172.21.15.16:0/1494454787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a44a50b660 msgr2=0x55a44a8e5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.448+0000 7f799e000640 1 --2- 172.21.15.16:0/1494454787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a44a50b660 0x55a44a8e5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.448+0000 7f799e000640 1 -- 172.21.15.16:0/1494454787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a44a534520 msgr2=0x55a44a8f1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.448+0000 7f799e000640 1 --2- 172.21.15.16:0/1494454787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a44a534520 0x55a44a8f1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.448+0000 7f799e000640 1 -- 172.21.15.16:0/1494454787 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a44a8e67b0 con 0x55a44a513840 2024-08-31T06:21:40.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.449+0000 7f799e000640 1 --2- 172.21.15.16:0/1494454787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44a513840 0x55a44a8dba80 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f798c103100 tx=0x7f798c0b3040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ad3920844e34f99 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:40.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.449+0000 7f798b400640 1 -- 172.21.15.16:0/1494454787 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c048040 con 0x55a44a513840 2024-08-31T06:21:40.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.449+0000 7f798b400640 1 -- 172.21.15.16:0/1494454787 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f798c0d0910 con 0x55a44a513840 2024-08-31T06:21:40.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.449+0000 7f798b400640 1 -- 172.21.15.16:0/1494454787 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c0d4f30 con 0x55a44a513840 2024-08-31T06:21:40.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.450+0000 7f79ac659740 1 -- 172.21.15.16:0/1494454787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44a513840 msgr2=0x55a44a8dba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:40.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.450+0000 7f79ac659740 1 --2- 172.21.15.16:0/1494454787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44a513840 0x55a44a8dba80 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f798c103100 tx=0x7f798c0b3040 comp rx=0 tx=0).stop 2024-08-31T06:21:40.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.450+0000 7f79ac659740 1 -- 172.21.15.16:0/1494454787 shutdown_connections 2024-08-31T06:21:40.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.450+0000 7f79ac659740 1 --2- 172.21.15.16:0/1494454787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44a513840 0x55a44a8dba80 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.450+0000 7f79ac659740 1 --2- 172.21.15.16:0/1494454787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a44a534520 0x55a44a8f1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.450+0000 7f79ac659740 1 --2- 172.21.15.16:0/1494454787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a44a50b660 0x55a44a8e5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.450+0000 7f79ac659740 1 -- 172.21.15.16:0/1494454787 >> 172.21.15.16:0/1494454787 conn(0x55a44a528910 msgr2=0x55a44a8e6a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:40.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.450+0000 7f79ac659740 1 -- 172.21.15.16:0/1494454787 shutdown_connections 2024-08-31T06:21:40.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.450+0000 7f79ac659740 1 -- 172.21.15.16:0/1494454787 wait complete. 2024-08-31T06:21:40.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.450+0000 7f79ac659740 1 -- start start 2024-08-31T06:21:40.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.451+0000 7f79ac659740 1 Processor -- start 2024-08-31T06:21:40.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.451+0000 7f79ac659740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a44a50b660 0x55a44a95b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:40.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.451+0000 7f79ac659740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a44a513840 0x55a44a95b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:40.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.451+0000 7f79ac659740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44a534520 0x55a44a8ea1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:40.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.451+0000 7f79ac659740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f79a01b3e10 con 0x55a44a513840 2024-08-31T06:21:40.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.451+0000 7f79ac659740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f79a01b5280 con 0x55a44a50b660 2024-08-31T06:21:40.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.451+0000 7f79ac659740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f79a01b44e0 con 0x55a44a534520 2024-08-31T06:21:40.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.451+0000 7f79ac659740 10 client.0 _ll_register_callbacks cb 0x7f7990004400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:21:40.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.452+0000 7f799e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a44a513840 0x55a44a95b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:40.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.452+0000 7f799f400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44a534520 0x55a44a8ea1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:40.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.452+0000 7f799e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a44a513840 0x55a44a95b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46224/0 (socket says 172.21.15.16:46224) 2024-08-31T06:21:40.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.452+0000 7f799e000640 1 -- 172.21.15.16:0/3372040650 learned_addr learned my addr 172.21.15.16:0/3372040650 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:40.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.452+0000 7f799ea00640 1 --2- 172.21.15.16:0/3372040650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a44a50b660 0x55a44a95b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:40.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.452+0000 7f799f400640 1 -- 172.21.15.16:0/3372040650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a44a50b660 msgr2=0x55a44a95b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:40.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.452+0000 7f799f400640 1 --2- 172.21.15.16:0/3372040650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a44a50b660 0x55a44a95b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.452+0000 7f799f400640 1 -- 172.21.15.16:0/3372040650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a44a513840 msgr2=0x55a44a95b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:40.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.452+0000 7f799f400640 1 --2- 172.21.15.16:0/3372040650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a44a513840 0x55a44a95b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.452+0000 7f799f400640 1 -- 172.21.15.16:0/3372040650 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a44a8ea950 con 0x55a44a534520 2024-08-31T06:21:40.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.452+0000 7f799e000640 1 --2- 172.21.15.16:0/3372040650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a44a513840 0x55a44a95b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:21:40.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.452+0000 7f799f400640 1 --2- 172.21.15.16:0/3372040650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44a534520 0x55a44a8ea1e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f7998013010 tx=0x7f7998004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8748aba34ad85c6d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:40.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.452+0000 7f7988c00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:21:40.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.453+0000 7f7988c00640 1 -- 172.21.15.16:0/3372040650 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7998011ba0 con 0x55a44a534520 2024-08-31T06:21:40.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.453+0000 7f79ac659740 1 -- 172.21.15.16:0/3372040650 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f798c0d0910 con 0x55a44a534520 2024-08-31T06:21:40.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.453+0000 7f7988c00640 1 -- 172.21.15.16:0/3372040650 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8470+0+0 (secure 0 0 0) 0x7f7998011d40 con 0x55a44a534520 2024-08-31T06:21:40.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.453+0000 7f7988c00640 1 -- 172.21.15.16:0/3372040650 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f7998004b20 con 0x55a44a534520 2024-08-31T06:21:40.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.453+0000 7f79ac659740 10 client.28858 fetch_fsmap learned FSMap version 51 2024-08-31T06:21:40.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.453+0000 7f79ac659740 1 -- 172.21.15.16:0/3372040650 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55a44a962a90 con 0x55a44a534520 2024-08-31T06:21:40.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.453+0000 7f7988c00640 1 -- 172.21.15.16:0/3372040650 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f798c0d0910 con 0x55a44a534520 2024-08-31T06:21:40.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.453+0000 7f7988c00640 1 -- 172.21.15.16:0/3372040650 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 51) ==== 8354+0+0 (secure 0 0 0) 0x7f7998033430 con 0x55a44a534520 2024-08-31T06:21:40.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.454+0000 7f79ac659740 10 client.28858 fetch_fsmap finished waiting for FSMap version 51 2024-08-31T06:21:40.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.454+0000 7f79ac659740 10 client.28858 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:21:40.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.454+0000 7f79ac659740 20 client.28858 populate_metadata read hostname 'smithi016' 2024-08-31T06:21:40.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.454+0000 7f79ac659740 1 --2- 172.21.15.16:0/3372040650 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a44a4c8250 0x55a44a965310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:40.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.454+0000 7f79ac659740 4 client.28858 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "603ae143", "release": true, "await": true}] 2024-08-31T06:21:40.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.454+0000 7f79ac659740 1 -- 172.21.15.16:0/3372040650 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "quiesce db", "set_id": "603ae143", "release": true, "await": true}) -- 0x55a44a84e480 con 0x55a44a4c8250 2024-08-31T06:21:40.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.454+0000 7f799ea00640 1 --2- 172.21.15.16:0/3372040650 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a44a4c8250 0x55a44a965310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:40.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.454+0000 7f799ea00640 1 --2- 172.21.15.16:0/3372040650 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a44a4c8250 0x55a44a965310 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f79fd7725f8e3fd server_cookie=b79b69f29a26811f in_seq=0 out_seq=0 2024-08-31T06:21:40.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.454+0000 7f7988c00640 10 client.28858 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:21:40.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.636+0000 7f7988c00640 1 -- 172.21.15.16:0/3372040650 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a44a84e480 con 0x55a44a4c8250 2024-08-31T06:21:40.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.636+0000 7f7988c00640 10 client.28858 handle_command_reply: tid=0 2024-08-31T06:21:40.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.636+0000 7f79ac659740 1 client.28858 shutdown 2024-08-31T06:21:40.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.636+0000 7f79ac659740 1 -- 172.21.15.16:0/3372040650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44a534520 msgr2=0x55a44a8ea1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:40.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.636+0000 7f79ac659740 1 --2- 172.21.15.16:0/3372040650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44a534520 0x55a44a8ea1e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f7998013010 tx=0x7f7998004a40 comp rx=0 tx=0).stop 2024-08-31T06:21:40.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.636+0000 7f79ac659740 1 -- 172.21.15.16:0/3372040650 shutdown_connections 2024-08-31T06:21:40.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.636+0000 7f79ac659740 1 --2- 172.21.15.16:0/3372040650 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a44a4c8250 0x55a44a965310 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.636+0000 7f79ac659740 1 --2- 172.21.15.16:0/3372040650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44a534520 0x55a44a8ea1e0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.636+0000 7f79ac659740 1 --2- 172.21.15.16:0/3372040650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a44a513840 0x55a44a95b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.636+0000 7f79ac659740 1 --2- 172.21.15.16:0/3372040650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a44a50b660 0x55a44a95b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.636+0000 7f7988c00640 0 client.28858 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:21:40.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.636+0000 7f79ac659740 1 -- 172.21.15.16:0/3372040650 >> 172.21.15.16:0/3372040650 conn(0x55a44a528910 msgr2=0x55a44a8db850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:40.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.637+0000 7f79ac659740 1 -- 172.21.15.16:0/3372040650 shutdown_connections 2024-08-31T06:21:40.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.637+0000 7f79ac659740 1 -- 172.21.15.16:0/3372040650 wait complete. 2024-08-31T06:21:40.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.637+0000 7f79ac659740 20 client.28858 trim_cache size 0 max 16384 2024-08-31T06:21:40.641 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:21:40.641 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 51, 2024-08-31T06:21:40.641 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24737, 2024-08-31T06:21:40.641 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 48, 2024-08-31T06:21:40.642 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-08-31T06:21:40.642 INFO:teuthology.orchestra.run.smithi016.stdout: "603ae143": { 2024-08-31T06:21:40.642 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 48, 2024-08-31T06:21:40.642 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-08-31T06:21:40.642 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:21:40.642 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-08-31T06:21:40.642 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:21:40.642 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:21:40.642 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-08-31T06:21:40.642 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 155.0, 2024-08-31T06:21:40.642 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-08-31T06:21:40.642 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-08-31T06:21:40.642 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-08-31T06:21:40.642 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-08-31T06:21:40.643 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-08-31T06:21:40.643 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-08-31T06:21:40.643 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:21:40.643 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:21:40.643 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:21:40.643 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:21:40.643 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:21:40.643 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:21:40.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.638+0000 7f799fe00640 1 -- 172.21.15.16:0/2244129051 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f79840b4ca0 msgr2=0x7f79840b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:40.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.638+0000 7f799fe00640 1 --2- 172.21.15.16:0/2244129051 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f79840b4ca0 0x7f79840b71b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f79980049d0 tx=0x7f7998009500 comp rx=0 tx=0).stop 2024-08-31T06:21:40.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.638+0000 7f799fe00640 1 -- 172.21.15.16:0/2244129051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79a0077120 msgr2=0x7f79a01a0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:40.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.638+0000 7f799fe00640 1 --2- 172.21.15.16:0/2244129051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79a0077120 0x7f79a01a0c10 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f798c02a910 tx=0x7f798c031a20 comp rx=0 tx=0).stop 2024-08-31T06:21:40.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.639+0000 7f799fe00640 1 -- 172.21.15.16:0/2244129051 shutdown_connections 2024-08-31T06:21:40.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.639+0000 7f799fe00640 1 --2- 172.21.15.16:0/2244129051 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f79840b4ca0 0x7f79840b71b0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.639+0000 7f799fe00640 1 --2- 172.21.15.16:0/2244129051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79a0114020 0x7f79a01a5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.639+0000 7f799fe00640 1 --2- 172.21.15.16:0/2244129051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79a0077120 0x7f79a01a0c10 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.639+0000 7f799fe00640 1 --2- 172.21.15.16:0/2244129051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79a0075b70 0x7f79a01a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.639+0000 7f799fe00640 1 -- 172.21.15.16:0/2244129051 >> 172.21.15.16:0/2244129051 conn(0x7f79a00ff1e0 msgr2=0x7f79a010ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:40.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.639+0000 7f799fe00640 1 -- 172.21.15.16:0/2244129051 shutdown_connections 2024-08-31T06:21:40.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:40.639+0000 7f799fe00640 1 -- 172.21.15.16:0/2244129051 wait complete. 2024-08-31T06:21:40.730 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.728+0000 7f36eaa00640 1 -- 172.21.15.16:0/1334006510 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d40318a0 con 0x7f36ec1044e0 2024-08-31T06:21:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.729+0000 7f36f1600640 1 -- 172.21.15.16:0/1334006510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36ec1044e0 msgr2=0x7f36ec1048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.729+0000 7f36f1600640 1 --2- 172.21.15.16:0/1334006510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36ec1044e0 0x7f36ec1048f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f36d4009900 tx=0x7f36d402a3f0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.729+0000 7f36f1600640 1 -- 172.21.15.16:0/1334006510 shutdown_connections 2024-08-31T06:21:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.729+0000 7f36f1600640 1 --2- 172.21.15.16:0/1334006510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36ec104e30 0x7f36ec10d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.729+0000 7f36f1600640 1 --2- 172.21.15.16:0/1334006510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36ec10d940 0x7f36ec10fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.729+0000 7f36f1600640 1 --2- 172.21.15.16:0/1334006510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36ec1044e0 0x7f36ec1048f0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.729+0000 7f36f1600640 1 -- 172.21.15.16:0/1334006510 >> 172.21.15.16:0/1334006510 conn(0x7f36ec0fdfd0 msgr2=0x7f36ec100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.730+0000 7f36f1600640 1 -- 172.21.15.16:0/1334006510 shutdown_connections 2024-08-31T06:21:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.730+0000 7f36f1600640 1 -- 172.21.15.16:0/1334006510 wait complete. 2024-08-31T06:21:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.732+0000 7f36f1600640 1 Processor -- start 2024-08-31T06:21:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.732+0000 7f36f1600640 1 -- start start 2024-08-31T06:21:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.732+0000 7f36f1600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36ec1044e0 0x7f36ec19c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.733+0000 7f36f1600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36ec104e30 0x7f36ec19c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.733+0000 7f36f1600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36ec10d940 0x7f36ec1a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.733+0000 7f36f1600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec113600 con 0x7f36ec10d940 2024-08-31T06:21:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.733+0000 7f36f1600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec113480 con 0x7f36ec104e30 2024-08-31T06:21:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.733+0000 7f36f1600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec113780 con 0x7f36ec1044e0 2024-08-31T06:21:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.733+0000 7f36ebe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36ec1044e0 0x7f36ec19c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.733+0000 7f36ebe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36ec1044e0 0x7f36ec19c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:43636/0 (socket says 172.21.15.16:43636) 2024-08-31T06:21:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.733+0000 7f36ebe00640 1 -- 172.21.15.16:0/473192242 learned_addr learned my addr 172.21.15.16:0/473192242 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.733+0000 7f36eb400640 1 --2- 172.21.15.16:0/473192242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36ec104e30 0x7f36ec19c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.733+0000 7f36f0c00640 1 --2- 172.21.15.16:0/473192242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36ec10d940 0x7f36ec1a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.734+0000 7f36ebe00640 1 -- 172.21.15.16:0/473192242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36ec104e30 msgr2=0x7f36ec19c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.734+0000 7f36ebe00640 1 --2- 172.21.15.16:0/473192242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36ec104e30 0x7f36ec19c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.734+0000 7f36ebe00640 1 -- 172.21.15.16:0/473192242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36ec10d940 msgr2=0x7f36ec1a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.734+0000 7f36ebe00640 1 --2- 172.21.15.16:0/473192242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36ec10d940 0x7f36ec1a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.734+0000 7f36ebe00640 1 -- 172.21.15.16:0/473192242 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36ec1a2150 con 0x7f36ec1044e0 2024-08-31T06:21:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.734+0000 7f36eb400640 1 --2- 172.21.15.16:0/473192242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36ec104e30 0x7f36ec19c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:21:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.735+0000 7f36ebe00640 1 --2- 172.21.15.16:0/473192242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36ec1044e0 0x7f36ec19c220 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f36d4005ab0 tx=0x7f36d4003c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:40.739 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.735+0000 7f36e8c00640 1 -- 172.21.15.16:0/473192242 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d403f070 con 0x7f36ec1044e0 2024-08-31T06:21:40.739 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.735+0000 7f36e8c00640 1 -- 172.21.15.16:0/473192242 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36d4031b50 con 0x7f36ec1044e0 2024-08-31T06:21:40.739 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.735+0000 7f36e8c00640 1 -- 172.21.15.16:0/473192242 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d4035030 con 0x7f36ec1044e0 2024-08-31T06:21:40.739 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.735+0000 7f36f1600640 1 -- 172.21.15.16:0/473192242 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36ec1a23e0 con 0x7f36ec1044e0 2024-08-31T06:21:40.740 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.736+0000 7f36f1600640 1 -- 172.21.15.16:0/473192242 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36ec1a2880 con 0x7f36ec1044e0 2024-08-31T06:21:40.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.743+0000 7f36f9922740 1 Processor -- start 2024-08-31T06:21:40.747 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.743+0000 7f36f9922740 1 -- start start 2024-08-31T06:21:40.747 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.744+0000 7f36f9922740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589f6b381b0 0x5589f6edbb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:40.747 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.744+0000 7f36f9922740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589f6b0f2f0 0x5589f6edc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:40.747 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.744+0000 7f36f9922740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589f6b174d0 0x5589f6edc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:40.747 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.744+0000 7f36f9922740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5589f6e50f70 con 0x5589f6b381b0 2024-08-31T06:21:40.747 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.744+0000 7f36f9922740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5589f6e40500 con 0x5589f6b0f2f0 2024-08-31T06:21:40.747 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.744+0000 7f36e8c00640 1 -- 172.21.15.16:0/473192242 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f36d4042070 con 0x7f36ec1044e0 2024-08-31T06:21:40.747 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.744+0000 7f36f9922740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5589f6e444b0 con 0x5589f6b174d0 2024-08-31T06:21:40.747 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.745+0000 7f36f0c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589f6b381b0 0x5589f6edbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:40.748 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.745+0000 7f36eb400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589f6b174d0 0x5589f6edc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:40.748 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.745+0000 7f36eb400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589f6b174d0 0x5589f6edc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:43638/0 (socket says 172.21.15.16:43638) 2024-08-31T06:21:40.748 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.745+0000 7f36eb400640 1 -- 172.21.15.16:0/1106777051 learned_addr learned my addr 172.21.15.16:0/1106777051 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:40.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.746+0000 7f36eb400640 1 -- 172.21.15.16:0/1106777051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589f6b0f2f0 msgr2=0x5589f6edc040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:21:40.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.746+0000 7f36eb400640 1 --2- 172.21.15.16:0/1106777051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589f6b0f2f0 0x5589f6edc040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.746+0000 7f36eb400640 1 -- 172.21.15.16:0/1106777051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589f6b381b0 msgr2=0x5589f6edbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:40.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.746+0000 7f36eb400640 1 --2- 172.21.15.16:0/1106777051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589f6b381b0 0x5589f6edbb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.746+0000 7f36eb400640 1 -- 172.21.15.16:0/1106777051 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589f6dfac10 con 0x5589f6b174d0 2024-08-31T06:21:40.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.747+0000 7f36eb400640 1 --2- 172.21.15.16:0/1106777051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589f6b174d0 0x5589f6edc580 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f36dc009c70 tx=0x7f36dc0117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adb4d50a3f4c7242 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:40.750 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.747+0000 7f36e1200640 1 -- 172.21.15.16:0/1106777051 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc012450 con 0x5589f6b174d0 2024-08-31T06:21:40.750 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.748+0000 7f36e1200640 1 -- 172.21.15.16:0/1106777051 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36dc012aa0 con 0x5589f6b174d0 2024-08-31T06:21:40.750 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.748+0000 7f36e1200640 1 -- 172.21.15.16:0/1106777051 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc00fb90 con 0x5589f6b174d0 2024-08-31T06:21:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.749+0000 7f36e8c00640 1 --2- 172.21.15.16:0/473192242 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f36d00b4eb0 0x7f36d00b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:40.752 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.749+0000 7f36e8c00640 1 -- 172.21.15.16:0/473192242 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8470+0+0 (secure 0 0 0) 0x7f36d4105e50 con 0x7f36ec1044e0 2024-08-31T06:21:40.752 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.749+0000 7f36f0c00640 1 --2- 172.21.15.16:0/473192242 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f36d00b4eb0 0x7f36d00b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:40.752 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.749+0000 7f36f9922740 1 -- 172.21.15.16:0/1106777051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589f6b174d0 msgr2=0x5589f6edc580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:40.752 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.749+0000 7f36f9922740 1 --2- 172.21.15.16:0/1106777051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589f6b174d0 0x5589f6edc580 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f36dc009c70 tx=0x7f36dc0117b0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.752 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.750+0000 7f36f9922740 1 -- 172.21.15.16:0/1106777051 shutdown_connections 2024-08-31T06:21:40.753 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.750+0000 7f36f9922740 1 --2- 172.21.15.16:0/1106777051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589f6b174d0 0x5589f6edc580 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.753 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.750+0000 7f36f9922740 1 --2- 172.21.15.16:0/1106777051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589f6b0f2f0 0x5589f6edc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.753 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.750+0000 7f36f9922740 1 --2- 172.21.15.16:0/1106777051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589f6b381b0 0x5589f6edbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.753 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.750+0000 7f36f9922740 1 -- 172.21.15.16:0/1106777051 >> 172.21.15.16:0/1106777051 conn(0x5589f6b2c300 msgr2=0x5589f6ed99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:40.753 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.750+0000 7f36f9922740 1 -- 172.21.15.16:0/1106777051 shutdown_connections 2024-08-31T06:21:40.754 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.750+0000 7f36f9922740 1 -- 172.21.15.16:0/1106777051 wait complete. 2024-08-31T06:21:40.754 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.751+0000 7f36f9922740 1 -- start start 2024-08-31T06:21:40.754 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.751+0000 7f36f9922740 1 Processor -- start 2024-08-31T06:21:40.754 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.752+0000 7f36f0c00640 1 --2- 172.21.15.16:0/473192242 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f36d00b4eb0 0x7f36d00b73c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f36e4005c40 tx=0x7f36e4005bd0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:40.755 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.752+0000 7f36f9922740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589f6b0f2f0 0x5589f6ee7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:40.755 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.753+0000 7f36f9922740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589f6b174d0 0x5589f6ef20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:40.755 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.753+0000 7f36f9922740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589f6b381b0 0x5589f6ee8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:40.756 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.753+0000 7f36f9922740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5589f6d79ab0 con 0x5589f6b0f2f0 2024-08-31T06:21:40.756 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.753+0000 7f36ebe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589f6b0f2f0 0x5589f6ee7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:40.756 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.753+0000 7f36f9922740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5589f6de7320 con 0x5589f6b174d0 2024-08-31T06:21:40.756 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.753+0000 7f36f9922740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5589f6ee21e0 con 0x5589f6b381b0 2024-08-31T06:21:40.756 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.754+0000 7f36ebe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589f6b0f2f0 0x5589f6ee7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46262/0 (socket says 172.21.15.16:46262) 2024-08-31T06:21:40.756 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.754+0000 7f36ebe00640 1 -- 172.21.15.16:0/850520637 learned_addr learned my addr 172.21.15.16:0/850520637 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.753+0000 7f36eb400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589f6b174d0 0x5589f6ef20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.754+0000 7f36f9922740 10 client.0 _ll_register_callbacks cb 0x5589f6ded6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:21:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.753+0000 7f36f0c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589f6b381b0 0x5589f6ee8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.754+0000 7f36ebe00640 1 -- 172.21.15.16:0/850520637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589f6b381b0 msgr2=0x5589f6ee8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.754+0000 7f36ebe00640 1 --2- 172.21.15.16:0/850520637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589f6b381b0 0x5589f6ee8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.754+0000 7f36ebe00640 1 -- 172.21.15.16:0/850520637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589f6b174d0 msgr2=0x5589f6ef20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.754+0000 7f36ebe00640 1 --2- 172.21.15.16:0/850520637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589f6b174d0 0x5589f6ef20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.754+0000 7f36ebe00640 1 -- 172.21.15.16:0/850520637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589f6ba3790 con 0x5589f6b0f2f0 2024-08-31T06:21:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.755+0000 7f36ebe00640 1 --2- 172.21.15.16:0/850520637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589f6b0f2f0 0x5589f6ee7940 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f36d4031440 tx=0x7f36d40cc020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7ff271600c78cf server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.755+0000 7f36d9600640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:21:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.755+0000 7f36d9600640 1 -- 172.21.15.16:0/850520637 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d40cc510 con 0x5589f6b0f2f0 2024-08-31T06:21:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.755+0000 7f36f9922740 1 -- 172.21.15.16:0/850520637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36dc012aa0 con 0x5589f6b0f2f0 2024-08-31T06:21:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.755+0000 7f36d9600640 1 -- 172.21.15.16:0/850520637 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8470+0+0 (secure 0 0 0) 0x7f36d40c87a0 con 0x5589f6b0f2f0 2024-08-31T06:21:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.756+0000 7f36d9600640 1 -- 172.21.15.16:0/850520637 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f36d40d0d80 con 0x5589f6b0f2f0 2024-08-31T06:21:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.756+0000 7f36f9922740 10 client.18369 fetch_fsmap learned FSMap version 51 2024-08-31T06:21:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.756+0000 7f36f9922740 1 -- 172.21.15.16:0/850520637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5589f6c1c620 con 0x5589f6b0f2f0 2024-08-31T06:21:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.756+0000 7f36d9600640 1 -- 172.21.15.16:0/850520637 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36dc012aa0 con 0x5589f6b0f2f0 2024-08-31T06:21:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.756+0000 7f36d9600640 1 -- 172.21.15.16:0/850520637 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 51) ==== 8354+0+0 (secure 0 0 0) 0x7f36d40ef0b0 con 0x5589f6b0f2f0 2024-08-31T06:21:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.757+0000 7f36f9922740 10 client.18369 fetch_fsmap finished waiting for FSMap version 51 2024-08-31T06:21:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.757+0000 7f36f9922740 10 client.18369 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:21:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.757+0000 7f36f9922740 20 client.18369 populate_metadata read hostname 'smithi016' 2024-08-31T06:21:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.757+0000 7f36f9922740 1 --2- 172.21.15.16:0/850520637 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5589f6af6d80 0x5589f6f0a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.757+0000 7f36f9922740 4 client.18369 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:21:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.757+0000 7f36f9922740 1 -- 172.21.15.16:0/850520637 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589f6e52ec0 con 0x5589f6af6d80 2024-08-31T06:21:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.757+0000 7f36eb400640 1 --2- 172.21.15.16:0/850520637 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5589f6af6d80 0x5589f6f0a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.758+0000 7f36eb400640 1 --2- 172.21.15.16:0/850520637 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5589f6af6d80 0x5589f6f0a1d0 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4db0e3c0f9aeef14 server_cookie=87dff789626b412a in_seq=0 out_seq=0 2024-08-31T06:21:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.758+0000 7f36d9600640 10 client.18369 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:21:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.759+0000 7f36d9600640 1 -- 172.21.15.16:0/850520637 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5589f6e52ec0 con 0x5589f6af6d80 2024-08-31T06:21:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.759+0000 7f36d9600640 10 client.18369 handle_command_reply: tid=0 2024-08-31T06:21:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.760+0000 7f36f9922740 1 client.18369 shutdown 2024-08-31T06:21:40.763 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.760+0000 7f36f9922740 1 -- 172.21.15.16:0/850520637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589f6b0f2f0 msgr2=0x5589f6ee7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:40.763 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.760+0000 7f36f9922740 1 --2- 172.21.15.16:0/850520637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589f6b0f2f0 0x5589f6ee7940 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f36d4031440 tx=0x7f36d40cc020 comp rx=0 tx=0).stop 2024-08-31T06:21:40.763 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.761+0000 7f36f9922740 1 -- 172.21.15.16:0/850520637 shutdown_connections 2024-08-31T06:21:40.763 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.761+0000 7f36f9922740 1 --2- 172.21.15.16:0/850520637 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5589f6af6d80 0x5589f6f0a1d0 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.763 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.761+0000 7f36f9922740 1 --2- 172.21.15.16:0/850520637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589f6b381b0 0x5589f6ee8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.763 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.761+0000 7f36f9922740 1 --2- 172.21.15.16:0/850520637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589f6b174d0 0x5589f6ef20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.763 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.761+0000 7f36f9922740 1 --2- 172.21.15.16:0/850520637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589f6b0f2f0 0x5589f6ee7940 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.763 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.761+0000 7f36d9600640 0 client.18369 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:21:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.761+0000 7f36f9922740 1 -- 172.21.15.16:0/850520637 >> 172.21.15.16:0/850520637 conn(0x5589f6b2c300 msgr2=0x5589f6eded20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.761+0000 7f36f9922740 1 -- 172.21.15.16:0/850520637 shutdown_connections 2024-08-31T06:21:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.761+0000 7f36f9922740 1 -- 172.21.15.16:0/850520637 wait complete. 2024-08-31T06:21:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.761+0000 7f36f9922740 20 client.18369 trim_cache size 0 max 16384 2024-08-31T06:21:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.787+0000 7f36f9922740 1 Processor -- start 2024-08-31T06:21:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.788+0000 7f36f9922740 1 -- start start 2024-08-31T06:21:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.788+0000 7f36f9922740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589f6b0f2f0 0x5589f6ee2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.788+0000 7f36f9922740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589f6b174d0 0x5589f6edd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.788+0000 7f36f9922740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589f6b381b0 0x5589f6ee4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.788+0000 7f36f9922740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec1af840 con 0x5589f6b174d0 2024-08-31T06:21:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.788+0000 7f36f9922740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec1ad0b0 con 0x5589f6b381b0 2024-08-31T06:21:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.788+0000 7f36f9922740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec1aaf80 con 0x5589f6b0f2f0 2024-08-31T06:21:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.788+0000 7f36ebe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589f6b0f2f0 0x5589f6ee2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.789+0000 7f36f0c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589f6b381b0 0x5589f6ee4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.789+0000 7f36eb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589f6b174d0 0x5589f6edd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.789+0000 7f36eb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589f6b174d0 0x5589f6edd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46272/0 (socket says 172.21.15.16:46272) 2024-08-31T06:21:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.789+0000 7f36f0c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589f6b381b0 0x5589f6ee4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:41192/0 (socket says 172.21.15.16:41192) 2024-08-31T06:21:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.789+0000 7f36eb400640 1 -- 172.21.15.16:0/1155833015 learned_addr learned my addr 172.21.15.16:0/1155833015 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.789+0000 7f36f0c00640 1 -- 172.21.15.16:0/1155833015 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589f6b0f2f0 msgr2=0x5589f6ee2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.789+0000 7f36f0c00640 1 --2- 172.21.15.16:0/1155833015 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589f6b0f2f0 0x5589f6ee2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.789+0000 7f36f0c00640 1 -- 172.21.15.16:0/1155833015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589f6b174d0 msgr2=0x5589f6edd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.789+0000 7f36f0c00640 1 --2- 172.21.15.16:0/1155833015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589f6b174d0 0x5589f6edd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.789+0000 7f36f0c00640 1 -- 172.21.15.16:0/1155833015 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589f6ee20c0 con 0x5589f6b381b0 2024-08-31T06:21:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.790+0000 7f36f0c00640 1 --2- 172.21.15.16:0/1155833015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589f6b381b0 0x5589f6ee4920 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f36e4009bc0 tx=0x7f36e4014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba995958690ee9ad server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.791+0000 7f36dbe00640 1 -- 172.21.15.16:0/1155833015 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e401f8c0 con 0x5589f6b381b0 2024-08-31T06:21:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.791+0000 7f36dbe00640 1 -- 172.21.15.16:0/1155833015 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36e402b480 con 0x5589f6b381b0 2024-08-31T06:21:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.791+0000 7f36dbe00640 1 -- 172.21.15.16:0/1155833015 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e4020690 con 0x5589f6b381b0 2024-08-31T06:21:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.792+0000 7f36f9922740 1 -- 172.21.15.16:0/1155833015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589f6b381b0 msgr2=0x5589f6ee4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.792+0000 7f36f9922740 1 --2- 172.21.15.16:0/1155833015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589f6b381b0 0x5589f6ee4920 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f36e4009bc0 tx=0x7f36e4014000 comp rx=0 tx=0).stop 2024-08-31T06:21:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.792+0000 7f36f9922740 1 -- 172.21.15.16:0/1155833015 shutdown_connections 2024-08-31T06:21:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.792+0000 7f36f9922740 1 --2- 172.21.15.16:0/1155833015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589f6b381b0 0x5589f6ee4920 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.792+0000 7f36f9922740 1 --2- 172.21.15.16:0/1155833015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589f6b174d0 0x5589f6edd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.792+0000 7f36f9922740 1 --2- 172.21.15.16:0/1155833015 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589f6b0f2f0 0x5589f6ee2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.792+0000 7f36f9922740 1 -- 172.21.15.16:0/1155833015 >> 172.21.15.16:0/1155833015 conn(0x5589f6b2c300 msgr2=0x5589f6edcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.792+0000 7f36f9922740 1 -- 172.21.15.16:0/1155833015 shutdown_connections 2024-08-31T06:21:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.792+0000 7f36f9922740 1 -- 172.21.15.16:0/1155833015 wait complete. 2024-08-31T06:21:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.793+0000 7f36f9922740 1 -- start start 2024-08-31T06:21:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.793+0000 7f36f9922740 1 Processor -- start 2024-08-31T06:21:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.794+0000 7f36f9922740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589f6b0f2f0 0x5589f6f17ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.794+0000 7f36f9922740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589f6b174d0 0x5589f6f180e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.794+0000 7f36f9922740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589f6b381b0 0x5589f6f73b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.794+0000 7f36f9922740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec1ab5d0 con 0x5589f6b381b0 2024-08-31T06:21:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.794+0000 7f36f9922740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec1ac9e0 con 0x5589f6b174d0 2024-08-31T06:21:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.794+0000 7f36f9922740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec1abc40 con 0x5589f6b0f2f0 2024-08-31T06:21:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.794+0000 7f36f9922740 10 client.0 _ll_register_callbacks cb 0x7f36d40cbe60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:21:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.795+0000 7f36f0c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589f6b381b0 0x5589f6f73b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.795+0000 7f36f0c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589f6b381b0 0x5589f6f73b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46284/0 (socket says 172.21.15.16:46284) 2024-08-31T06:21:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.795+0000 7f36f0c00640 1 -- 172.21.15.16:0/839823160 learned_addr learned my addr 172.21.15.16:0/839823160 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.795+0000 7f36ebe00640 1 --2- 172.21.15.16:0/839823160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589f6b0f2f0 0x5589f6f17ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.795+0000 7f36eb400640 1 --2- 172.21.15.16:0/839823160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589f6b174d0 0x5589f6f180e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.795+0000 7f36ebe00640 1 -- 172.21.15.16:0/839823160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589f6b174d0 msgr2=0x5589f6f180e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.795+0000 7f36ebe00640 1 --2- 172.21.15.16:0/839823160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589f6b174d0 0x5589f6f180e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.795+0000 7f36ebe00640 1 -- 172.21.15.16:0/839823160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589f6b381b0 msgr2=0x5589f6f73b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.795+0000 7f36ebe00640 1 --2- 172.21.15.16:0/839823160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589f6b381b0 0x5589f6f73b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.795+0000 7f36ebe00640 1 -- 172.21.15.16:0/839823160 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589f6f188d0 con 0x5589f6b0f2f0 2024-08-31T06:21:40.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.796+0000 7f36ebe00640 1 --2- 172.21.15.16:0/839823160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589f6b0f2f0 0x5589f6f17ba0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f36d40cc4a0 tx=0x7f36d407c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efb3b0643d37f0ea server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.796+0000 7f36d9600640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:21:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.796+0000 7f36d9600640 1 -- 172.21.15.16:0/839823160 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d40cc510 con 0x5589f6b0f2f0 2024-08-31T06:21:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.796+0000 7f36d9600640 1 -- 172.21.15.16:0/839823160 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8470+0+0 (secure 0 0 0) 0x7f36d41037f0 con 0x5589f6b0f2f0 2024-08-31T06:21:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.796+0000 7f36f9922740 1 -- 172.21.15.16:0/839823160 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36e402b480 con 0x5589f6b0f2f0 2024-08-31T06:21:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.797+0000 7f36d9600640 1 -- 172.21.15.16:0/839823160 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f36d41039f0 con 0x5589f6b0f2f0 2024-08-31T06:21:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.797+0000 7f36f9922740 10 client.28876 fetch_fsmap learned FSMap version 51 2024-08-31T06:21:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.797+0000 7f36f9922740 1 -- 172.21.15.16:0/839823160 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5589f6f75060 con 0x5589f6b0f2f0 2024-08-31T06:21:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.797+0000 7f36d9600640 1 -- 172.21.15.16:0/839823160 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36e402b480 con 0x5589f6b0f2f0 2024-08-31T06:21:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.797+0000 7f36d9600640 1 -- 172.21.15.16:0/839823160 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 51) ==== 8354+0+0 (secure 0 0 0) 0x7f36d40cb770 con 0x5589f6b0f2f0 2024-08-31T06:21:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.797+0000 7f36f9922740 10 client.28876 fetch_fsmap finished waiting for FSMap version 51 2024-08-31T06:21:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.797+0000 7f36f9922740 10 client.28876 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:21:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.797+0000 7f36f9922740 20 client.28876 populate_metadata read hostname 'smithi016' 2024-08-31T06:21:40.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.797+0000 7f36f9922740 1 --2- 172.21.15.16:0/839823160 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5589f6acc1c0 0x5589f6f77820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:40.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.797+0000 7f36f9922740 4 client.28876 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:21:40.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.797+0000 7f36f9922740 1 -- 172.21.15.16:0/839823160 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5589f6e4e960 con 0x5589f6acc1c0 2024-08-31T06:21:40.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.798+0000 7f36eb400640 1 --2- 172.21.15.16:0/839823160 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5589f6acc1c0 0x5589f6f77820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:40.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.798+0000 7f36eb400640 1 --2- 172.21.15.16:0/839823160 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5589f6acc1c0 0x5589f6f77820 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1ebf124ca9cb974 server_cookie=381242a50d626f82 in_seq=0 out_seq=0 2024-08-31T06:21:40.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.798+0000 7f36d9600640 10 client.28876 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:21:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.818+0000 7f36d9600640 1 -- 172.21.15.16:0/839823160 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5589f6e4e960 con 0x5589f6acc1c0 2024-08-31T06:21:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.818+0000 7f36d9600640 10 client.28876 handle_command_reply: tid=0 2024-08-31T06:21:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.818+0000 7f36f9922740 1 client.28876 shutdown 2024-08-31T06:21:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.818+0000 7f36f9922740 1 -- 172.21.15.16:0/839823160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589f6b0f2f0 msgr2=0x5589f6f17ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.818+0000 7f36f9922740 1 --2- 172.21.15.16:0/839823160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589f6b0f2f0 0x5589f6f17ba0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f36d40cc4a0 tx=0x7f36d407c3c0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.819+0000 7f36f9922740 1 -- 172.21.15.16:0/839823160 shutdown_connections 2024-08-31T06:21:40.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.819+0000 7f36f9922740 1 --2- 172.21.15.16:0/839823160 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5589f6acc1c0 0x5589f6f77820 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.819+0000 7f36f9922740 1 --2- 172.21.15.16:0/839823160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589f6b381b0 0x5589f6f73b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.819+0000 7f36f9922740 1 --2- 172.21.15.16:0/839823160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589f6b174d0 0x5589f6f180e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.819+0000 7f36f9922740 1 --2- 172.21.15.16:0/839823160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589f6b0f2f0 0x5589f6f17ba0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.819+0000 7f36d9600640 0 client.28876 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:21:40.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.819+0000 7f36f9922740 1 -- 172.21.15.16:0/839823160 >> 172.21.15.16:0/839823160 conn(0x5589f6b2c300 msgr2=0x5589f6ee94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:40.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.820+0000 7f36f9922740 1 -- 172.21.15.16:0/839823160 shutdown_connections 2024-08-31T06:21:40.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.820+0000 7f36f9922740 1 -- 172.21.15.16:0/839823160 wait complete. 2024-08-31T06:21:40.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.820+0000 7f36f9922740 20 client.28876 trim_cache size 0 max 16384 2024-08-31T06:21:40.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:21:40.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:21:40.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:21:40.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:21:40.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.821+0000 7f36f1600640 1 -- 172.21.15.16:0/473192242 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f36d00b4eb0 msgr2=0x7f36d00b73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:40.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.821+0000 7f36f1600640 1 --2- 172.21.15.16:0/473192242 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f36d00b4eb0 0x7f36d00b73c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f36e4005c40 tx=0x7f36e4005bd0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.821+0000 7f36f1600640 1 -- 172.21.15.16:0/473192242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36ec1044e0 msgr2=0x7f36ec19c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:40.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.821+0000 7f36f1600640 1 --2- 172.21.15.16:0/473192242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36ec1044e0 0x7f36ec19c220 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f36d4005ab0 tx=0x7f36d4003c20 comp rx=0 tx=0).stop 2024-08-31T06:21:40.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.821+0000 7f36f1600640 1 -- 172.21.15.16:0/473192242 shutdown_connections 2024-08-31T06:21:40.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.822+0000 7f36f1600640 1 --2- 172.21.15.16:0/473192242 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f36d00b4eb0 0x7f36d00b73c0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.822+0000 7f36f1600640 1 --2- 172.21.15.16:0/473192242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36ec10d940 0x7f36ec1a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.822+0000 7f36f1600640 1 --2- 172.21.15.16:0/473192242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36ec104e30 0x7f36ec19c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.822+0000 7f36f1600640 1 --2- 172.21.15.16:0/473192242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36ec1044e0 0x7f36ec19c220 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:40.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.822+0000 7f36f1600640 1 -- 172.21.15.16:0/473192242 >> 172.21.15.16:0/473192242 conn(0x7f36ec0fdfd0 msgr2=0x7f36ec1020f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:40.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.822+0000 7f36f1600640 1 -- 172.21.15.16:0/473192242 shutdown_connections 2024-08-31T06:21:40.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:40.822+0000 7f36f1600640 1 -- 172.21.15.16:0/473192242 wait complete. 2024-08-31T06:21:41.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:40 smithi016 ceph-mon[17194]: pgmap v815: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 10 op/s 2024-08-31T06:21:41.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:40 smithi184 ceph-mon[22331]: pgmap v815: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 10 op/s 2024-08-31T06:21:41.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:40 smithi155 ceph-mon[21742]: pgmap v815: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 10 op/s 2024-08-31T06:21:41.623 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '603ae143', total seconds elapsed: 40.0 2024-08-31T06:21:41.625 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 855.0 s, 314.0 tps, lat 2047.533 ms stddev 7631.598 2024-08-31T06:21:43.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:42 smithi016 ceph-mon[17194]: pgmap v816: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-08-31T06:21:43.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:42 smithi184 ceph-mon[22331]: pgmap v816: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-08-31T06:21:43.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:42 smithi155 ceph-mon[21742]: pgmap v816: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-08-31T06:21:44.921 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T06:21:45.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:44 smithi016 ceph-mon[17194]: pgmap v817: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 268 KiB/s wr, 37 op/s 2024-08-31T06:21:45.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:44 smithi184 ceph-mon[22331]: pgmap v817: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 268 KiB/s wr, 37 op/s 2024-08-31T06:21:45.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:44 smithi155 ceph-mon[21742]: pgmap v817: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 268 KiB/s wr, 37 op/s 2024-08-31T06:21:45.540 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:21:45.963 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:45 smithi016 ceph-mon[17194]: pgmap v818: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 2.0 MiB/s wr, 217 op/s 2024-08-31T06:21:46.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:45 smithi184 ceph-mon[22331]: pgmap v818: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 2.0 MiB/s wr, 217 op/s 2024-08-31T06:21:46.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:45 smithi155 ceph-mon[21742]: pgmap v818: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 2.0 MiB/s wr, 217 op/s 2024-08-31T06:21:46.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.578+0000 7f1dc6600640 1 -- 172.21.15.16:0/2685185956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dc01021e0 msgr2=0x7f1dc01025f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:46.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.578+0000 7f1dc6600640 1 --2- 172.21.15.16:0/2685185956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dc01021e0 0x7f1dc01025f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f1db4009900 tx=0x7f1db402a480 comp rx=0 tx=0).stop 2024-08-31T06:21:46.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.579+0000 7f1dc6600640 1 -- 172.21.15.16:0/2685185956 shutdown_connections 2024-08-31T06:21:46.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.579+0000 7f1dc6600640 1 --2- 172.21.15.16:0/2685185956 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dc0102b30 0x7f1dc010f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:46.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.579+0000 7f1dc6600640 1 --2- 172.21.15.16:0/2685185956 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1dc010fd90 0x7f1dc01121b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:46.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.579+0000 7f1dc6600640 1 --2- 172.21.15.16:0/2685185956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dc01021e0 0x7f1dc01025f0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:46.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.579+0000 7f1dc6600640 1 -- 172.21.15.16:0/2685185956 >> 172.21.15.16:0/2685185956 conn(0x7f1dc00fe060 msgr2=0x7f1dc01004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:46.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.580+0000 7f1dc6600640 1 -- 172.21.15.16:0/2685185956 shutdown_connections 2024-08-31T06:21:46.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.580+0000 7f1dc6600640 1 -- 172.21.15.16:0/2685185956 wait complete. 2024-08-31T06:21:46.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.581+0000 7f1dc6600640 1 Processor -- start 2024-08-31T06:21:46.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.582+0000 7f1dc6600640 1 -- start start 2024-08-31T06:21:46.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.583+0000 7f1dc6600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dc01021e0 0x7f1dc019c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:46.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.584+0000 7f1dc6600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1dc0102b30 0x7f1dc019c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:46.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.584+0000 7f1dc6600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dc010fd90 0x7f1dc01a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:46.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.584+0000 7f1dc6600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc0115820 con 0x7f1dc010fd90 2024-08-31T06:21:46.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.584+0000 7f1dbfe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1dc0102b30 0x7f1dc019c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:46.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.584+0000 7f1dc5200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dc01021e0 0x7f1dc019c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:46.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.584+0000 7f1dbfe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1dc0102b30 0x7f1dc019c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:53420/0 (socket says 172.21.15.16:53420) 2024-08-31T06:21:46.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.584+0000 7f1dbfe00640 1 -- 172.21.15.16:0/2512861572 learned_addr learned my addr 172.21.15.16:0/2512861572 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:46.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.584+0000 7f1dc6600640 1 -- 172.21.15.16:0/2512861572 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc01156a0 con 0x7f1dc0102b30 2024-08-31T06:21:46.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.584+0000 7f1dc6600640 1 -- 172.21.15.16:0/2512861572 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc01159a0 con 0x7f1dc01021e0 2024-08-31T06:21:46.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.585+0000 7f1dc5c00640 1 --2- 172.21.15.16:0/2512861572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dc010fd90 0x7f1dc01a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:46.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.585+0000 7f1dbfe00640 1 -- 172.21.15.16:0/2512861572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dc01021e0 msgr2=0x7f1dc019c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:46.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.585+0000 7f1dbfe00640 1 --2- 172.21.15.16:0/2512861572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dc01021e0 0x7f1dc019c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:46.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.585+0000 7f1dbfe00640 1 -- 172.21.15.16:0/2512861572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dc010fd90 msgr2=0x7f1dc01a1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:46.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.585+0000 7f1dbfe00640 1 --2- 172.21.15.16:0/2512861572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dc010fd90 0x7f1dc01a1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:46.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.585+0000 7f1dbfe00640 1 -- 172.21.15.16:0/2512861572 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc01a21a0 con 0x7f1dc0102b30 2024-08-31T06:21:46.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.585+0000 7f1dc5c00640 1 --2- 172.21.15.16:0/2512861572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dc010fd90 0x7f1dc01a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:21:46.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.586+0000 7f1dbfe00640 1 --2- 172.21.15.16:0/2512861572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1dc0102b30 0x7f1dc019c7d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1dac00ea60 tx=0x7f1dac00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:46.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.587+0000 7f1dbd600640 1 -- 172.21.15.16:0/2512861572 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac00cde0 con 0x7f1dc0102b30 2024-08-31T06:21:46.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.588+0000 7f1dbd600640 1 -- 172.21.15.16:0/2512861572 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dac0035f0 con 0x7f1dc0102b30 2024-08-31T06:21:46.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.588+0000 7f1dbd600640 1 -- 172.21.15.16:0/2512861572 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac010690 con 0x7f1dc0102b30 2024-08-31T06:21:46.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.588+0000 7f1dc6600640 1 -- 172.21.15.16:0/2512861572 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dc01a2490 con 0x7f1dc0102b30 2024-08-31T06:21:46.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.588+0000 7f1dc6600640 1 -- 172.21.15.16:0/2512861572 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dc01a8bf0 con 0x7f1dc0102b30 2024-08-31T06:21:46.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.592+0000 7f1dce8bf740 1 Processor -- start 2024-08-31T06:21:46.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.592+0000 7f1dce8bf740 1 -- start start 2024-08-31T06:21:46.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.593+0000 7f1dce8bf740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594813f3070 0x559481798b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:46.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.593+0000 7f1dce8bf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594813ca1b0 0x559481799060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:46.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.593+0000 7f1dce8bf740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594813d2390 0x5594817995a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:46.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.593+0000 7f1dce8bf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55948170f770 con 0x5594813ca1b0 2024-08-31T06:21:46.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.593+0000 7f1dce8bf740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5594816fed00 con 0x5594813f3070 2024-08-31T06:21:46.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.593+0000 7f1dce8bf740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559481702cb0 con 0x5594813d2390 2024-08-31T06:21:46.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.594+0000 7f1dc5200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594813ca1b0 0x559481799060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:46.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.594+0000 7f1dbd600640 1 -- 172.21.15.16:0/2512861572 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f1dac0037a0 con 0x7f1dc0102b30 2024-08-31T06:21:46.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.595+0000 7f1dc5200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594813ca1b0 0x559481799060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37992/0 (socket says 172.21.15.16:37992) 2024-08-31T06:21:46.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.595+0000 7f1dc5200640 1 -- 172.21.15.16:0/2747232748 learned_addr learned my addr 172.21.15.16:0/2747232748 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:46.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.595+0000 7f1dc5c00640 1 --2- 172.21.15.16:0/2747232748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594813f3070 0x559481798b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:46.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.596+0000 7f1dc5c00640 1 -- 172.21.15.16:0/2747232748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594813d2390 msgr2=0x5594817995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:46.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.596+0000 7f1dbfe00640 1 --2- 172.21.15.16:0/2747232748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594813d2390 0x5594817995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:46.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.597+0000 7f1dc5c00640 1 --2- 172.21.15.16:0/2747232748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594813d2390 0x5594817995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:46.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.597+0000 7f1dc5c00640 1 -- 172.21.15.16:0/2747232748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594813ca1b0 msgr2=0x559481799060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:46.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.597+0000 7f1dc5c00640 1 --2- 172.21.15.16:0/2747232748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594813ca1b0 0x559481799060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:46.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.597+0000 7f1dc5c00640 1 -- 172.21.15.16:0/2747232748 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55948145e650 con 0x5594813f3070 2024-08-31T06:21:46.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.597+0000 7f1dc5200640 1 --2- 172.21.15.16:0/2747232748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594813ca1b0 0x559481799060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:21:46.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.599+0000 7f1dbfe00640 1 --2- 172.21.15.16:0/2747232748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594813d2390 0x5594817995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:21:46.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.598+0000 7f1dc5c00640 1 --2- 172.21.15.16:0/2747232748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594813f3070 0x559481798b20 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f1db8009ee0 tx=0x7f1db8010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e2ca63811f85d08 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:46.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.599+0000 7f1dbd600640 1 --2- 172.21.15.16:0/2512861572 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f1da80b4e70 0x7f1da80b7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:46.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.599+0000 7f1db2000640 1 -- 172.21.15.16:0/2747232748 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db8016070 con 0x5594813f3070 2024-08-31T06:21:46.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.599+0000 7f1dbd600640 1 -- 172.21.15.16:0/2512861572 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8470+0+0 (secure 0 0 0) 0x7f1dac0e5800 con 0x7f1dc0102b30 2024-08-31T06:21:46.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.599+0000 7f1db2000640 1 -- 172.21.15.16:0/2747232748 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1db801bb10 con 0x5594813f3070 2024-08-31T06:21:46.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.599+0000 7f1db2000640 1 -- 172.21.15.16:0/2747232748 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db8012ad0 con 0x5594813f3070 2024-08-31T06:21:46.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.600+0000 7f1dc5200640 1 --2- 172.21.15.16:0/2512861572 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f1da80b4e70 0x7f1da80b7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:46.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.600+0000 7f1dce8bf740 1 -- 172.21.15.16:0/2747232748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594813f3070 msgr2=0x559481798b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:46.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.600+0000 7f1dce8bf740 1 --2- 172.21.15.16:0/2747232748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594813f3070 0x559481798b20 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f1db8009ee0 tx=0x7f1db8010740 comp rx=0 tx=0).stop 2024-08-31T06:21:46.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.601+0000 7f1dce8bf740 1 -- 172.21.15.16:0/2747232748 shutdown_connections 2024-08-31T06:21:46.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.601+0000 7f1dce8bf740 1 --2- 172.21.15.16:0/2747232748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594813d2390 0x5594817995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:46.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.601+0000 7f1dce8bf740 1 --2- 172.21.15.16:0/2747232748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594813ca1b0 0x559481799060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:46.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.601+0000 7f1dce8bf740 1 --2- 172.21.15.16:0/2747232748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594813f3070 0x559481798b20 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:46.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.601+0000 7f1dce8bf740 1 -- 172.21.15.16:0/2747232748 >> 172.21.15.16:0/2747232748 conn(0x5594813e7460 msgr2=0x559481796a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:46.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.601+0000 7f1dc5200640 1 --2- 172.21.15.16:0/2512861572 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f1da80b4e70 0x7f1da80b7380 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1db4000c00 tx=0x7f1db4005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:46.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.601+0000 7f1dce8bf740 1 -- 172.21.15.16:0/2747232748 shutdown_connections 2024-08-31T06:21:46.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.601+0000 7f1dce8bf740 1 -- 172.21.15.16:0/2747232748 wait complete. 2024-08-31T06:21:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.602+0000 7f1dce8bf740 1 -- start start 2024-08-31T06:21:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.602+0000 7f1dce8bf740 1 Processor -- start 2024-08-31T06:21:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.604+0000 7f1dce8bf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594813ca1b0 0x5594817a4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.604+0000 7f1dce8bf740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594813d2390 0x5594817a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.604+0000 7f1dce8bf740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594813f3070 0x5594817b0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.604+0000 7f1dce8bf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559481561700 con 0x5594813ca1b0 2024-08-31T06:21:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.604+0000 7f1dce8bf740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559481685cb0 con 0x5594813f3070 2024-08-31T06:21:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.604+0000 7f1dce8bf740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5594817a7300 con 0x5594813d2390 2024-08-31T06:21:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.604+0000 7f1dc5c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594813f3070 0x5594817b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.604+0000 7f1dc5c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594813f3070 0x5594817b0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:53450/0 (socket says 172.21.15.16:53450) 2024-08-31T06:21:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.604+0000 7f1dc5c00640 1 -- 172.21.15.16:0/475583312 learned_addr learned my addr 172.21.15.16:0/475583312 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.604+0000 7f1dc5200640 1 --2- 172.21.15.16:0/475583312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594813ca1b0 0x5594817a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:46.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.604+0000 7f1dce8bf740 10 client.0 _ll_register_callbacks cb 0x5594816ab430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:21:46.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.604+0000 7f1dbfe00640 1 --2- 172.21.15.16:0/475583312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594813d2390 0x5594817a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:46.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.605+0000 7f1dc5c00640 1 -- 172.21.15.16:0/475583312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594813d2390 msgr2=0x5594817a4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:46.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.605+0000 7f1dc5c00640 1 --2- 172.21.15.16:0/475583312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594813d2390 0x5594817a4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:46.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.605+0000 7f1dc5c00640 1 -- 172.21.15.16:0/475583312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594813ca1b0 msgr2=0x5594817a4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:46.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.605+0000 7f1dc5c00640 1 --2- 172.21.15.16:0/475583312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594813ca1b0 0x5594817a4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:46.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.605+0000 7f1dc5c00640 1 -- 172.21.15.16:0/475583312 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594816c6870 con 0x5594813f3070 2024-08-31T06:21:46.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.605+0000 7f1dbfe00640 1 --2- 172.21.15.16:0/475583312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594813d2390 0x5594817a4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:21:46.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.606+0000 7f1dc5c00640 1 --2- 172.21.15.16:0/475583312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594813f3070 0x5594817b0620 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f1db8006010 tx=0x7f1db801bcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=386d7e1e5730cd7f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:46.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.606+0000 7f1da6a00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:21:46.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.606+0000 7f1da6a00640 1 -- 172.21.15.16:0/475583312 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db8016050 con 0x5594813f3070 2024-08-31T06:21:46.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.606+0000 7f1dce8bf740 1 -- 172.21.15.16:0/475583312 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1db801bb10 con 0x5594813f3070 2024-08-31T06:21:46.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.606+0000 7f1da6a00640 1 -- 172.21.15.16:0/475583312 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8470+0+0 (secure 0 0 0) 0x7f1db801b870 con 0x5594813f3070 2024-08-31T06:21:46.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.607+0000 7f1da6a00640 1 -- 172.21.15.16:0/475583312 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f1db8012850 con 0x5594813f3070 2024-08-31T06:21:46.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.607+0000 7f1dce8bf740 10 client.28862 fetch_fsmap learned FSMap version 51 2024-08-31T06:21:46.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.607+0000 7f1dce8bf740 1 -- 172.21.15.16:0/475583312 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5594817b32c0 con 0x5594813f3070 2024-08-31T06:21:46.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.607+0000 7f1da6a00640 1 -- 172.21.15.16:0/475583312 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1db801bb10 con 0x5594813f3070 2024-08-31T06:21:46.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.607+0000 7f1da6a00640 1 -- 172.21.15.16:0/475583312 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 51) ==== 8354+0+0 (secure 0 0 0) 0x7f1db8024070 con 0x5594813f3070 2024-08-31T06:21:46.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.607+0000 7f1dce8bf740 10 client.28862 fetch_fsmap finished waiting for FSMap version 51 2024-08-31T06:21:46.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.607+0000 7f1dce8bf740 10 client.28862 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:21:46.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.607+0000 7f1dce8bf740 20 client.28862 populate_metadata read hostname 'smithi016' 2024-08-31T06:21:46.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.607+0000 7f1dce8bf740 1 --2- 172.21.15.16:0/475583312 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5594813b19f0 0x5594817b6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:46.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.608+0000 7f1dce8bf740 4 client.28862 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:21:46.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.608+0000 7f1dce8bf740 1 -- 172.21.15.16:0/475583312 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594817116c0 con 0x5594813b19f0 2024-08-31T06:21:46.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.608+0000 7f1dc5200640 1 --2- 172.21.15.16:0/475583312 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5594813b19f0 0x5594817b6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:46.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.609+0000 7f1dc5200640 1 --2- 172.21.15.16:0/475583312 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5594813b19f0 0x5594817b6620 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0612f1517320cc5 server_cookie=1b4a2e84d93fdf17 in_seq=0 out_seq=0 2024-08-31T06:21:46.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.609+0000 7f1da6a00640 10 client.28862 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:21:46.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.609+0000 7f1da6a00640 1 -- 172.21.15.16:0/475583312 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5594817116c0 con 0x5594813b19f0 2024-08-31T06:21:46.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.609+0000 7f1da6a00640 10 client.28862 handle_command_reply: tid=0 2024-08-31T06:21:46.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.610+0000 7f1dce8bf740 1 client.28862 shutdown 2024-08-31T06:21:46.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.615+0000 7f1dce8bf740 1 -- 172.21.15.16:0/475583312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594813f3070 msgr2=0x5594817b0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:46.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.615+0000 7f1dce8bf740 1 --2- 172.21.15.16:0/475583312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594813f3070 0x5594817b0620 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f1db8006010 tx=0x7f1db801bcc0 comp rx=0 tx=0).stop 2024-08-31T06:21:46.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.615+0000 7f1dce8bf740 1 -- 172.21.15.16:0/475583312 shutdown_connections 2024-08-31T06:21:46.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.615+0000 7f1dce8bf740 1 --2- 172.21.15.16:0/475583312 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5594813b19f0 0x5594817b6620 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:46.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.615+0000 7f1dce8bf740 1 --2- 172.21.15.16:0/475583312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594813d2390 0x5594817a4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:46.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.615+0000 7f1dce8bf740 1 --2- 172.21.15.16:0/475583312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594813f3070 0x5594817b0620 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:46.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.615+0000 7f1dce8bf740 1 --2- 172.21.15.16:0/475583312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594813ca1b0 0x5594817a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:46.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.615+0000 7f1dce8bf740 1 -- 172.21.15.16:0/475583312 >> 172.21.15.16:0/475583312 conn(0x5594813e7460 msgr2=0x55948179bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:46.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.616+0000 7f1da6a00640 0 client.28862 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:21:46.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.616+0000 7f1dce8bf740 1 -- 172.21.15.16:0/475583312 shutdown_connections 2024-08-31T06:21:46.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.616+0000 7f1dce8bf740 1 -- 172.21.15.16:0/475583312 wait complete. 2024-08-31T06:21:46.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.617+0000 7f1dce8bf740 20 client.28862 trim_cache size 0 max 16384 2024-08-31T06:21:46.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 860.0 s, 1658.0 tps, lat 60.111 ms stddev 70.486 2024-08-31T06:21:46.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.665+0000 7f1dce8bf740 1 Processor -- start 2024-08-31T06:21:46.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.666+0000 7f1dce8bf740 1 -- start start 2024-08-31T06:21:46.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.666+0000 7f1dce8bf740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594813ca1b0 0x55948179dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:46.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.666+0000 7f1dce8bf740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594813d2390 0x55948179e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:46.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.666+0000 7f1dce8bf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594813f3070 0x5594817b0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:46.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.666+0000 7f1dce8bf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc01b81c0 con 0x5594813f3070 2024-08-31T06:21:46.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.666+0000 7f1dce8bf740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc01b5a30 con 0x5594813ca1b0 2024-08-31T06:21:46.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.666+0000 7f1dce8bf740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc01b38a0 con 0x5594813d2390 2024-08-31T06:21:46.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.667+0000 7f1dc5200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594813ca1b0 0x55948179dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:46.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.667+0000 7f1dbfe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594813d2390 0x55948179e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:46.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.667+0000 7f1dbfe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594813d2390 0x55948179e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:33480/0 (socket says 172.21.15.16:33480) 2024-08-31T06:21:46.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.667+0000 7f1dbfe00640 1 -- 172.21.15.16:0/4221732468 learned_addr learned my addr 172.21.15.16:0/4221732468 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:46.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.668+0000 7f1dc5c00640 1 --2- 172.21.15.16:0/4221732468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594813f3070 0x5594817b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:46.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.668+0000 7f1dbfe00640 1 -- 172.21.15.16:0/4221732468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594813ca1b0 msgr2=0x55948179dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:46.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.668+0000 7f1dbfe00640 1 --2- 172.21.15.16:0/4221732468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594813ca1b0 0x55948179dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:46.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.668+0000 7f1dbfe00640 1 -- 172.21.15.16:0/4221732468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594813f3070 msgr2=0x5594817b0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:46.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.668+0000 7f1dbfe00640 1 --2- 172.21.15.16:0/4221732468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594813f3070 0x5594817b0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:46.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.668+0000 7f1dbfe00640 1 -- 172.21.15.16:0/4221732468 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594817b6990 con 0x5594813d2390 2024-08-31T06:21:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.670+0000 7f1dbfe00640 1 --2- 172.21.15.16:0/4221732468 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594813d2390 0x55948179e2c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1dac00c760 tx=0x7f1dac0ab8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=496d92e6cbd9bc0b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.670+0000 7f1db1600640 1 -- 172.21.15.16:0/4221732468 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac0ac5c0 con 0x5594813d2390 2024-08-31T06:21:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.671+0000 7f1db1600640 1 -- 172.21.15.16:0/4221732468 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dac0abec0 con 0x5594813d2390 2024-08-31T06:21:46.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.675+0000 7f1dce8bf740 1 -- 172.21.15.16:0/4221732468 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594813d2390 msgr2=0x55948179e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:46.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.675+0000 7f1dce8bf740 1 --2- 172.21.15.16:0/4221732468 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594813d2390 0x55948179e2c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1dac00c760 tx=0x7f1dac0ab8d0 comp rx=0 tx=0).stop 2024-08-31T06:21:46.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.675+0000 7f1dce8bf740 1 -- 172.21.15.16:0/4221732468 shutdown_connections 2024-08-31T06:21:46.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.675+0000 7f1dce8bf740 1 --2- 172.21.15.16:0/4221732468 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594813d2390 0x55948179e2c0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:46.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.675+0000 7f1dce8bf740 1 --2- 172.21.15.16:0/4221732468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594813f3070 0x5594817b0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:46.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.675+0000 7f1dce8bf740 1 --2- 172.21.15.16:0/4221732468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594813ca1b0 0x55948179dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:46.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.675+0000 7f1dce8bf740 1 -- 172.21.15.16:0/4221732468 >> 172.21.15.16:0/4221732468 conn(0x5594813e7460 msgr2=0x5594817a2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:46.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.676+0000 7f1dce8bf740 1 -- 172.21.15.16:0/4221732468 shutdown_connections 2024-08-31T06:21:46.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.676+0000 7f1dce8bf740 1 -- 172.21.15.16:0/4221732468 wait complete. 2024-08-31T06:21:46.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.677+0000 7f1dce8bf740 1 -- start start 2024-08-31T06:21:46.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.677+0000 7f1dce8bf740 1 Processor -- start 2024-08-31T06:21:46.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.678+0000 7f1dce8bf740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594813ca1b0 0x5594817ac590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:46.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.679+0000 7f1dce8bf740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594813d2390 0x5594817acb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:46.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.679+0000 7f1dc5200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594813ca1b0 0x5594817ac590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:46.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.679+0000 7f1dc5200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594813ca1b0 0x5594817ac590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:33488/0 (socket says 172.21.15.16:33488) 2024-08-31T06:21:46.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.679+0000 7f1dbfe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594813d2390 0x5594817acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:46.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.679+0000 7f1dc5200640 1 -- 172.21.15.16:0/3563814034 learned_addr learned my addr 172.21.15.16:0/3563814034 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:46.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.679+0000 7f1dce8bf740 1 --2- 172.21.15.16:0/3563814034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594813f3070 0x5594817a7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:46.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.680+0000 7f1dce8bf740 1 -- 172.21.15.16:0/3563814034 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc01b3ef0 con 0x5594813f3070 2024-08-31T06:21:46.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.680+0000 7f1dce8bf740 1 -- 172.21.15.16:0/3563814034 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc01b5360 con 0x5594813d2390 2024-08-31T06:21:46.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.680+0000 7f1dce8bf740 1 -- 172.21.15.16:0/3563814034 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc01b45c0 con 0x5594813ca1b0 2024-08-31T06:21:46.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.680+0000 7f1dce8bf740 10 client.0 _ll_register_callbacks cb 0x7f1db8009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:21:46.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.680+0000 7f1dc5200640 1 -- 172.21.15.16:0/3563814034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594813d2390 msgr2=0x5594817acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:46.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.680+0000 7f1dc5200640 1 --2- 172.21.15.16:0/3563814034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594813d2390 0x5594817acb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:46.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.680+0000 7f1dc5200640 1 -- 172.21.15.16:0/3563814034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594813f3070 msgr2=0x5594817a7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:46.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.681+0000 7f1dc5200640 1 --2- 172.21.15.16:0/3563814034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594813f3070 0x5594817a7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:46.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.681+0000 7f1dc5200640 1 -- 172.21.15.16:0/3563814034 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594817a7a80 con 0x5594813ca1b0 2024-08-31T06:21:46.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.681+0000 7f1dbfe00640 1 --2- 172.21.15.16:0/3563814034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594813d2390 0x5594817acb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:21:46.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.682+0000 7f1da6a00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:21:46.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.683+0000 7f1dc5200640 1 --2- 172.21.15.16:0/3563814034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594813ca1b0 0x5594817ac590 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f1db4031440 tx=0x7f1db4009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=805252c1b8273f9a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:46.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.683+0000 7f1da6a00640 1 -- 172.21.15.16:0/3563814034 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db405f060 con 0x5594813ca1b0 2024-08-31T06:21:46.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.683+0000 7f1da6a00640 1 -- 172.21.15.16:0/3563814034 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8470+0+0 (secure 0 0 0) 0x7f1db4004b80 con 0x5594813ca1b0 2024-08-31T06:21:46.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.683+0000 7f1dce8bf740 1 -- 172.21.15.16:0/3563814034 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1dac0abec0 con 0x5594813ca1b0 2024-08-31T06:21:46.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.685+0000 7f1da6a00640 1 -- 172.21.15.16:0/3563814034 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f1db4035040 con 0x5594813ca1b0 2024-08-31T06:21:46.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.685+0000 7f1dce8bf740 10 client.28891 fetch_fsmap learned FSMap version 51 2024-08-31T06:21:46.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.685+0000 7f1dce8bf740 1 -- 172.21.15.16:0/3563814034 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5594818215e0 con 0x5594813ca1b0 2024-08-31T06:21:46.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.685+0000 7f1da6a00640 1 -- 172.21.15.16:0/3563814034 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dac0abec0 con 0x5594813ca1b0 2024-08-31T06:21:46.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.687+0000 7f1da6a00640 1 -- 172.21.15.16:0/3563814034 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 51) ==== 8354+0+0 (secure 0 0 0) 0x7f1db4032520 con 0x5594813ca1b0 2024-08-31T06:21:46.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.687+0000 7f1dce8bf740 10 client.28891 fetch_fsmap finished waiting for FSMap version 51 2024-08-31T06:21:46.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.687+0000 7f1dce8bf740 10 client.28891 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:21:46.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.687+0000 7f1dce8bf740 20 client.28891 populate_metadata read hostname 'smithi016' 2024-08-31T06:21:46.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.687+0000 7f1dce8bf740 1 --2- 172.21.15.16:0/3563814034 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559481386da0 0x559481823d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:46.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.687+0000 7f1dce8bf740 4 client.28891 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T06:21:46.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.687+0000 7f1dce8bf740 1 -- 172.21.15.16:0/3563814034 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55948170d160 con 0x559481386da0 2024-08-31T06:21:46.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.687+0000 7f1dbfe00640 1 --2- 172.21.15.16:0/3563814034 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559481386da0 0x559481823d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:46.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.688+0000 7f1dbfe00640 1 --2- 172.21.15.16:0/3563814034 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559481386da0 0x559481823d60 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c3ba96630f0d6ee server_cookie=8f0490832d472d1c in_seq=0 out_seq=0 2024-08-31T06:21:46.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.688+0000 7f1da6a00640 10 client.28891 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:21:46.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.689+0000 7f1da6a00640 1 -- 172.21.15.16:0/3563814034 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5398 (crc 0 0 0) 0x55948170d160 con 0x559481386da0 2024-08-31T06:21:46.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.689+0000 7f1da6a00640 10 client.28891 handle_command_reply: tid=0 2024-08-31T06:21:46.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.689+0000 7f1dce8bf740 1 client.28891 shutdown 2024-08-31T06:21:46.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.690+0000 7f1dce8bf740 1 -- 172.21.15.16:0/3563814034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594813ca1b0 msgr2=0x5594817ac590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:46.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.690+0000 7f1dce8bf740 1 --2- 172.21.15.16:0/3563814034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594813ca1b0 0x5594817ac590 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f1db4031440 tx=0x7f1db4009260 comp rx=0 tx=0).stop 2024-08-31T06:21:46.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.690+0000 7f1dce8bf740 1 -- 172.21.15.16:0/3563814034 shutdown_connections 2024-08-31T06:21:46.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.690+0000 7f1dce8bf740 1 --2- 172.21.15.16:0/3563814034 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x559481386da0 0x559481823d60 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:46.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.690+0000 7f1dce8bf740 1 --2- 172.21.15.16:0/3563814034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594813f3070 0x5594817a7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:46.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.690+0000 7f1dce8bf740 1 --2- 172.21.15.16:0/3563814034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594813d2390 0x5594817acb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:46.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.690+0000 7f1dce8bf740 1 --2- 172.21.15.16:0/3563814034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594813ca1b0 0x5594817ac590 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:46.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.690+0000 7f1dce8bf740 1 -- 172.21.15.16:0/3563814034 >> 172.21.15.16:0/3563814034 conn(0x5594813e7460 msgr2=0x559481799b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:46.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.690+0000 7f1da6a00640 0 client.28891 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:21:46.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.691+0000 7f1dce8bf740 1 -- 172.21.15.16:0/3563814034 shutdown_connections 2024-08-31T06:21:46.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.691+0000 7f1dce8bf740 1 -- 172.21.15.16:0/3563814034 wait complete. 2024-08-31T06:21:46.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.694+0000 7f1dce8bf740 20 client.28891 trim_cache size 0 max 16384 2024-08-31T06:21:46.699 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:21:46.699 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T06:21:46.699 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-08-31T06:21:46.699 INFO:teuthology.orchestra.run.smithi016.stdout: "0015da36-6634-47c5-b8d4-219f9eba8250": { 2024-08-31T06:21:46.699 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:21:46.699 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "0015da36-6634-47c5-b8d4-219f9eba8250", 2024-08-31T06:21:46.699 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-08-31T06:21:46.699 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:21:46.699 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:21:46.700 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-08-31T06:21:46.700 INFO:teuthology.orchestra.run.smithi016.stdout: "18e112f4-a5f9-4602-aaad-3b13fa5a4c2e": { 2024-08-31T06:21:46.700 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:20:16.570535+0000", 2024-08-31T06:21:46.700 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:21:46.700 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:21:46.700 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:21:46.700 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:21:46.700 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:21:46.700 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:21:46.700 INFO:teuthology.orchestra.run.smithi016.stdout: "f1cdcb27-e3e4-4dc8-a0ae-7ace761d04c5": { 2024-08-31T06:21:46.700 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:19:22.275683+0000", 2024-08-31T06:21:46.700 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:21:46.700 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:21:46.700 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:21:46.701 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:21:46.701 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:21:46.701 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:21:46.701 INFO:teuthology.orchestra.run.smithi016.stdout: "0e44eb18-439c-46e4-a298-7d4d2ecd0b4a": { 2024-08-31T06:21:46.701 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:18:31.130765+0000", 2024-08-31T06:21:46.701 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:21:46.701 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:21:46.701 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:21:46.701 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:21:46.701 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:21:46.701 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:21:46.701 INFO:teuthology.orchestra.run.smithi016.stdout: "e497cd88-1e7c-4631-a4a8-cdd8e1fa6c83": { 2024-08-31T06:21:46.701 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:17:42.476795+0000", 2024-08-31T06:21:46.701 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:21:46.702 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:21:46.702 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:21:46.702 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:21:46.702 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:21:46.702 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:21:46.702 INFO:teuthology.orchestra.run.smithi016.stdout: "c300b7e8-f3fd-482c-a909-9329bc61e570": { 2024-08-31T06:21:46.702 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:06:49.868046+0000", 2024-08-31T06:21:46.702 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:21:46.702 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:21:46.702 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:21:46.702 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:21:46.702 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:21:46.702 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:21:46.702 INFO:teuthology.orchestra.run.smithi016.stdout: "6a03046e-cdf4-4a14-b9bd-b560b8968467": { 2024-08-31T06:21:46.703 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:12:38.002354+0000", 2024-08-31T06:21:46.703 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:21:46.703 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:21:46.703 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:21:46.703 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:21:46.703 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:21:46.703 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:21:46.703 INFO:teuthology.orchestra.run.smithi016.stdout: "004b47bb-0fe8-4d28-9d9d-78bde6a61199": { 2024-08-31T06:21:46.703 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:16:48.915926+0000", 2024-08-31T06:21:46.703 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:21:46.703 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:21:46.703 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:21:46.703 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:21:46.703 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:21:46.703 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:21:46.704 INFO:teuthology.orchestra.run.smithi016.stdout: "88d7bdb8-cb1c-45e5-ad29-4cd8da6c6003": { 2024-08-31T06:21:46.704 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:07:39.648942+0000", 2024-08-31T06:21:46.704 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:21:46.704 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:21:46.704 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:21:46.704 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:21:46.704 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:21:46.704 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:21:46.704 INFO:teuthology.orchestra.run.smithi016.stdout: "0fdcd762-533d-42be-9289-b71c8e3b81b1": { 2024-08-31T06:21:46.704 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:15:06.169112+0000", 2024-08-31T06:21:46.704 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:21:46.704 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:21:46.704 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:21:46.704 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:21:46.705 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:21:46.705 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:21:46.705 INFO:teuthology.orchestra.run.smithi016.stdout: "0015da36-6634-47c5-b8d4-219f9eba8250": { 2024-08-31T06:21:46.705 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:21:11.445380+0000", 2024-08-31T06:21:46.705 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:21:46.705 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:21:46.705 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:21:46.705 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:21:46.705 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:21:46.705 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:21:46.705 INFO:teuthology.orchestra.run.smithi016.stdout: "c8455d74-2fe7-4d93-bcf2-19b419d16016": { 2024-08-31T06:21:46.705 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:08:24.035960+0000", 2024-08-31T06:21:46.705 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:21:46.705 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:21:46.706 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:21:46.706 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:21:46.706 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:21:46.706 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:21:46.706 INFO:teuthology.orchestra.run.smithi016.stdout: "78d6ef02-8551-45e8-ad60-9627eb573fc1": { 2024-08-31T06:21:46.706 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:09:13.547865+0000", 2024-08-31T06:21:46.706 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:21:46.706 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:21:46.706 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:21:46.706 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:21:46.706 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:21:46.706 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:21:46.706 INFO:teuthology.orchestra.run.smithi016.stdout: "c1fd6904-2e02-47fa-939a-47349b9fe36b": { 2024-08-31T06:21:46.706 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:06.678691+0000", 2024-08-31T06:21:46.706 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:21:46.707 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:21:46.707 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:21:46.707 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:21:46.707 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:21:46.707 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:21:46.707 INFO:teuthology.orchestra.run.smithi016.stdout: "a10cce76-99ab-46b5-b3b2-07b82899b8f9": { 2024-08-31T06:21:46.707 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:11:43.967546+0000", 2024-08-31T06:21:46.707 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:21:46.707 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:21:46.707 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:21:46.707 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:21:46.707 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:21:46.707 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:21:46.707 INFO:teuthology.orchestra.run.smithi016.stdout: "91b72a34-4ba9-4126-a414-8b862f8c5f78": { 2024-08-31T06:21:46.707 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:13:28.663237+0000", 2024-08-31T06:21:46.707 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:21:46.708 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:21:46.708 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:21:46.708 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:21:46.708 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:21:46.708 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:21:46.708 INFO:teuthology.orchestra.run.smithi016.stdout: "e739a16e-8ace-43a8-a1ce-ac27d39ef8a3": { 2024-08-31T06:21:46.708 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:54.084646+0000", 2024-08-31T06:21:46.708 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:21:46.708 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:21:46.708 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:21:46.708 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:21:46.708 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:21:46.708 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:21:46.708 INFO:teuthology.orchestra.run.smithi016.stdout: "a64d5301-8558-4e9c-ac9b-8d0ac7057e74": { 2024-08-31T06:21:46.708 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:14:20.338104+0000", 2024-08-31T06:21:46.709 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:21:46.709 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:21:46.709 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:21:46.709 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:21:46.709 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:21:46.709 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:21:46.709 INFO:teuthology.orchestra.run.smithi016.stdout: "f9c4a22e-0c8a-4928-9b0d-e2ce1db5a86b": { 2024-08-31T06:21:46.709 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:15:57.496015+0000", 2024-08-31T06:21:46.709 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:21:46.709 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:21:46.709 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:21:46.709 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:21:46.709 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:21:46.709 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:21:46.709 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:21:46.710 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:21:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.695+0000 7f1dc6600640 1 -- 172.21.15.16:0/2512861572 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f1da80b4e70 msgr2=0x7f1da80b7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.695+0000 7f1dc6600640 1 --2- 172.21.15.16:0/2512861572 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f1da80b4e70 0x7f1da80b7380 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1db4000c00 tx=0x7f1db4005a20 comp rx=0 tx=0).stop 2024-08-31T06:21:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.695+0000 7f1dc6600640 1 -- 172.21.15.16:0/2512861572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1dc0102b30 msgr2=0x7f1dc019c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.695+0000 7f1dc6600640 1 --2- 172.21.15.16:0/2512861572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1dc0102b30 0x7f1dc019c7d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1dac00ea60 tx=0x7f1dac00ef20 comp rx=0 tx=0).stop 2024-08-31T06:21:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.696+0000 7f1dc6600640 1 -- 172.21.15.16:0/2512861572 shutdown_connections 2024-08-31T06:21:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.696+0000 7f1dc6600640 1 --2- 172.21.15.16:0/2512861572 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f1da80b4e70 0x7f1da80b7380 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.696+0000 7f1dc6600640 1 --2- 172.21.15.16:0/2512861572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dc010fd90 0x7f1dc01a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.696+0000 7f1dc6600640 1 --2- 172.21.15.16:0/2512861572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1dc0102b30 0x7f1dc019c7d0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.696+0000 7f1dc6600640 1 --2- 172.21.15.16:0/2512861572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dc01021e0 0x7f1dc019c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.696+0000 7f1dc6600640 1 -- 172.21.15.16:0/2512861572 >> 172.21.15.16:0/2512861572 conn(0x7f1dc00fe060 msgr2=0x7f1dc0111ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.697+0000 7f1dc6600640 1 -- 172.21.15.16:0/2512861572 shutdown_connections 2024-08-31T06:21:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:21:46.697+0000 7f1dc6600640 1 -- 172.21.15.16:0/2512861572 wait complete. 2024-08-31T06:21:47.735 INFO:tasks.cephfs.filesystem:scrub status for tag:0015da36-6634-47c5-b8d4-219f9eba8250 - {'path': '/', 'tag': '0015da36-6634-47c5-b8d4-219f9eba8250', 'options': 'recursive,force'} 2024-08-31T06:21:49.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:48 smithi155 ceph-mon[21742]: pgmap v819: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 2.0 MiB/s wr, 222 op/s 2024-08-31T06:21:49.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:48 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:21:49.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:48 smithi016 ceph-mon[17194]: pgmap v819: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 2.0 MiB/s wr, 222 op/s 2024-08-31T06:21:49.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:48 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:21:49.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:48 smithi184 ceph-mon[22331]: pgmap v819: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 2.0 MiB/s wr, 222 op/s 2024-08-31T06:21:49.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:48 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:21:50.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:49 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:21:50.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:49 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:21:50.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:49 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:21:50.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:49 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:21:50.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:49 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:21:50.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:49 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:21:51.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:50 smithi155 ceph-mon[21742]: pgmap v820: 97 pgs: 97 active+clean; 3.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 2.8 MiB/s wr, 316 op/s 2024-08-31T06:21:51.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:50 smithi016 ceph-mon[17194]: pgmap v820: 97 pgs: 97 active+clean; 3.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 2.8 MiB/s wr, 316 op/s 2024-08-31T06:21:51.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:50 smithi184 ceph-mon[22331]: pgmap v820: 97 pgs: 97 active+clean; 3.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 2.8 MiB/s wr, 316 op/s 2024-08-31T06:21:51.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 865.0 s, 1492.2 tps, lat 67.235 ms stddev 82.630 2024-08-31T06:21:53.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:52 smithi155 ceph-mon[21742]: pgmap v821: 97 pgs: 97 active+clean; 3.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 4.2 MiB/s wr, 467 op/s 2024-08-31T06:21:53.168 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:52 smithi016 ceph-mon[17194]: pgmap v821: 97 pgs: 97 active+clean; 3.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 4.2 MiB/s wr, 467 op/s 2024-08-31T06:21:53.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:52 smithi184 ceph-mon[22331]: pgmap v821: 97 pgs: 97 active+clean; 3.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 4.2 MiB/s wr, 467 op/s 2024-08-31T06:21:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.109+0000 7f6264000640 1 -- 172.21.15.16:0/2953677682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f625c1091d0 msgr2=0x7f625c1095e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.109+0000 7f6264000640 1 --2- 172.21.15.16:0/2953677682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f625c1091d0 0x7f625c1095e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f6248009900 tx=0x7f624802a410 comp rx=0 tx=0).stop 2024-08-31T06:21:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.111+0000 7f6264000640 1 -- 172.21.15.16:0/2953677682 shutdown_connections 2024-08-31T06:21:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.111+0000 7f6264000640 1 --2- 172.21.15.16:0/2953677682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f625c109b20 0x7f625c110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.111+0000 7f6264000640 1 --2- 172.21.15.16:0/2953677682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f625c1091d0 0x7f625c1095e0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.111+0000 7f6264000640 1 --2- 172.21.15.16:0/2953677682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f625c1038a0 0x7f625c103cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.111+0000 7f6264000640 1 -- 172.21.15.16:0/2953677682 >> 172.21.15.16:0/2953677682 conn(0x7f625c0ff2a0 msgr2=0x7f625c1016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.112+0000 7f6264000640 1 -- 172.21.15.16:0/2953677682 shutdown_connections 2024-08-31T06:21:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.112+0000 7f6264000640 1 -- 172.21.15.16:0/2953677682 wait complete. 2024-08-31T06:21:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.113+0000 7f6264000640 1 Processor -- start 2024-08-31T06:21:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.114+0000 7f6264000640 1 -- start start 2024-08-31T06:21:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.114+0000 7f6264000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f625c1038a0 0x7f625c076ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.114+0000 7f6264000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f625c1091d0 0x7f625c0754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.114+0000 7f6264000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f625c109b20 0x7f625c0759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.115+0000 7f6260c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f625c1091d0 0x7f625c0754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.115+0000 7f6260c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f625c1091d0 0x7f625c0754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:53488/0 (socket says 172.21.15.16:53488) 2024-08-31T06:21:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.115+0000 7f6261600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f625c1038a0 0x7f625c076ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.115+0000 7f6260c00640 1 -- 172.21.15.16:0/2697734145 learned_addr learned my addr 172.21.15.16:0/2697734145 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.115+0000 7f6264000640 1 -- 172.21.15.16:0/2697734145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f625c113780 con 0x7f625c109b20 2024-08-31T06:21:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.115+0000 7f6264000640 1 -- 172.21.15.16:0/2697734145 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f625c113600 con 0x7f625c1091d0 2024-08-31T06:21:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.115+0000 7f6264000640 1 -- 172.21.15.16:0/2697734145 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f625c113900 con 0x7f625c1038a0 2024-08-31T06:21:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.115+0000 7f6262000640 1 --2- 172.21.15.16:0/2697734145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f625c109b20 0x7f625c0759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.116+0000 7f6260c00640 1 -- 172.21.15.16:0/2697734145 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f625c1038a0 msgr2=0x7f625c076ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.116+0000 7f6260c00640 1 --2- 172.21.15.16:0/2697734145 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f625c1038a0 0x7f625c076ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.116+0000 7f6260c00640 1 -- 172.21.15.16:0/2697734145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f625c109b20 msgr2=0x7f625c0759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.116+0000 7f6260c00640 1 --2- 172.21.15.16:0/2697734145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f625c109b20 0x7f625c0759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.116+0000 7f6260c00640 1 -- 172.21.15.16:0/2697734145 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f625c076210 con 0x7f625c1091d0 2024-08-31T06:21:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.116+0000 7f6261600640 1 --2- 172.21.15.16:0/2697734145 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f625c1038a0 0x7f625c076ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:21:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.117+0000 7f6260c00640 1 --2- 172.21.15.16:0/2697734145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f625c1091d0 0x7f625c0754b0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6248006010 tx=0x7f6248004060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:54.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.117+0000 7f625a000640 1 -- 172.21.15.16:0/2697734145 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624803f030 con 0x7f625c1091d0 2024-08-31T06:21:54.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.117+0000 7f6264000640 1 -- 172.21.15.16:0/2697734145 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f625c1acb90 con 0x7f625c1091d0 2024-08-31T06:21:54.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.117+0000 7f625a000640 1 -- 172.21.15.16:0/2697734145 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6248003aa0 con 0x7f625c1091d0 2024-08-31T06:21:54.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.117+0000 7f625a000640 1 -- 172.21.15.16:0/2697734145 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6248035070 con 0x7f625c1091d0 2024-08-31T06:21:54.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.117+0000 7f6264000640 1 -- 172.21.15.16:0/2697734145 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f625c1ad0d0 con 0x7f625c1091d0 2024-08-31T06:21:54.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.124+0000 7f625a000640 1 -- 172.21.15.16:0/2697734145 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f6248042030 con 0x7f625c1091d0 2024-08-31T06:21:54.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.124+0000 7f626b204740 1 Processor -- start 2024-08-31T06:21:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.124+0000 7f626b204740 1 -- start start 2024-08-31T06:21:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.125+0000 7f626b204740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5031ae1b0 0x55a503551920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.125+0000 7f626b204740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5031852f0 0x55a503551e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.125+0000 7f626b204740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a50318d4d0 0x55a5035523a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.125+0000 7f626b204740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a5034c6f70 con 0x55a50318d4d0 2024-08-31T06:21:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.125+0000 7f626b204740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a5034b6500 con 0x55a5031852f0 2024-08-31T06:21:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.125+0000 7f626b204740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a5034ba4b0 con 0x55a5031ae1b0 2024-08-31T06:21:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.125+0000 7f6262000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5031ae1b0 0x55a503551920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.125+0000 7f6260c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a50318d4d0 0x55a5035523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.125+0000 7f6261600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5031852f0 0x55a503551e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.125+0000 7f6261600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5031852f0 0x55a503551e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:53502/0 (socket says 172.21.15.16:53502) 2024-08-31T06:21:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.125+0000 7f6261600640 1 -- 172.21.15.16:0/3373894552 learned_addr learned my addr 172.21.15.16:0/3373894552 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.125+0000 7f6262000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5031ae1b0 0x55a503551920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:33510/0 (socket says 172.21.15.16:33510) 2024-08-31T06:21:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.126+0000 7f6261600640 1 -- 172.21.15.16:0/3373894552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5031ae1b0 msgr2=0x55a503551920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.126+0000 7f6261600640 1 --2- 172.21.15.16:0/3373894552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5031ae1b0 0x55a503551920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.126+0000 7f6261600640 1 -- 172.21.15.16:0/3373894552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a50318d4d0 msgr2=0x55a5035523a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.126+0000 7f6261600640 1 --2- 172.21.15.16:0/3373894552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a50318d4d0 0x55a5035523a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.126+0000 7f6261600640 1 -- 172.21.15.16:0/3373894552 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a503470c10 con 0x55a5031852f0 2024-08-31T06:21:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.126+0000 7f6260c00640 1 --2- 172.21.15.16:0/3373894552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a50318d4d0 0x55a5035523a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:21:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.126+0000 7f625a000640 1 --2- 172.21.15.16:0/2697734145 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f62440b4ca0 0x7f62440b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.126+0000 7f6262000640 1 --2- 172.21.15.16:0/3373894552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5031ae1b0 0x55a503551920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:21:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.126+0000 7f625a000640 1 -- 172.21.15.16:0/2697734145 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8470+0+0 (secure 0 0 0) 0x7f6248105680 con 0x7f625c1091d0 2024-08-31T06:21:54.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.127+0000 7f6261600640 1 --2- 172.21.15.16:0/3373894552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5031852f0 0x55a503551e60 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6250011520 tx=0x7f62500119e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93dec65ebdea57af server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:54.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.127+0000 7f624e600640 1 -- 172.21.15.16:0/3373894552 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625001e070 con 0x55a5031852f0 2024-08-31T06:21:54.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.127+0000 7f6262000640 1 --2- 172.21.15.16:0/2697734145 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f62440b4ca0 0x7f62440b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:54.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.127+0000 7f624e600640 1 -- 172.21.15.16:0/3373894552 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62500035f0 con 0x55a5031852f0 2024-08-31T06:21:54.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.127+0000 7f624e600640 1 -- 172.21.15.16:0/3373894552 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625000f840 con 0x55a5031852f0 2024-08-31T06:21:54.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.128+0000 7f626b204740 1 -- 172.21.15.16:0/3373894552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5031852f0 msgr2=0x55a503551e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:54.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.128+0000 7f626b204740 1 --2- 172.21.15.16:0/3373894552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5031852f0 0x55a503551e60 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6250011520 tx=0x7f62500119e0 comp rx=0 tx=0).stop 2024-08-31T06:21:54.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.129+0000 7f6262000640 1 --2- 172.21.15.16:0/2697734145 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f62440b4ca0 0x7f62440b71b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6254005fd0 tx=0x7f62540096a0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:54.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.129+0000 7f626b204740 1 -- 172.21.15.16:0/3373894552 shutdown_connections 2024-08-31T06:21:54.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.129+0000 7f626b204740 1 --2- 172.21.15.16:0/3373894552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a50318d4d0 0x55a5035523a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:54.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.129+0000 7f626b204740 1 --2- 172.21.15.16:0/3373894552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5031852f0 0x55a503551e60 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:54.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.129+0000 7f626b204740 1 --2- 172.21.15.16:0/3373894552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5031ae1b0 0x55a503551920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:54.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.129+0000 7f626b204740 1 -- 172.21.15.16:0/3373894552 >> 172.21.15.16:0/3373894552 conn(0x55a5031a2300 msgr2=0x55a50354f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:54.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.129+0000 7f626b204740 1 -- 172.21.15.16:0/3373894552 shutdown_connections 2024-08-31T06:21:54.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.129+0000 7f626b204740 1 -- 172.21.15.16:0/3373894552 wait complete. 2024-08-31T06:21:54.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.129+0000 7f626b204740 1 -- start start 2024-08-31T06:21:54.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.130+0000 7f626b204740 1 Processor -- start 2024-08-31T06:21:54.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.130+0000 7f626b204740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5031852f0 0x55a50355ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.130+0000 7f626b204740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a50318d4d0 0x55a50355d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.130+0000 7f626b204740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5031ae1b0 0x55a50355d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.130+0000 7f626b204740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a5033efab0 con 0x55a5031ae1b0 2024-08-31T06:21:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.130+0000 7f626b204740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a50345d320 con 0x55a50318d4d0 2024-08-31T06:21:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.130+0000 7f626b204740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a5035600a0 con 0x55a5031852f0 2024-08-31T06:21:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.130+0000 7f626b204740 10 client.0 _ll_register_callbacks cb 0x55a5034636a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:21:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.130+0000 7f6261600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5031852f0 0x55a50355ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.131+0000 7f6261600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5031852f0 0x55a50355ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:33516/0 (socket says 172.21.15.16:33516) 2024-08-31T06:21:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.131+0000 7f6261600640 1 -- 172.21.15.16:0/3708253364 learned_addr learned my addr 172.21.15.16:0/3708253364 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.131+0000 7f6260c00640 1 --2- 172.21.15.16:0/3708253364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a50318d4d0 0x55a50355d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.131+0000 7f6262000640 1 --2- 172.21.15.16:0/3708253364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5031ae1b0 0x55a50355d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.131+0000 7f6260c00640 1 -- 172.21.15.16:0/3708253364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5031852f0 msgr2=0x55a50355ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.131+0000 7f6260c00640 1 --2- 172.21.15.16:0/3708253364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5031852f0 0x55a50355ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.131+0000 7f6260c00640 1 -- 172.21.15.16:0/3708253364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5031ae1b0 msgr2=0x55a50355d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.131+0000 7f6260c00640 1 --2- 172.21.15.16:0/3708253364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5031ae1b0 0x55a50355d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.131+0000 7f6260c00640 1 -- 172.21.15.16:0/3708253364 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a503219790 con 0x55a50318d4d0 2024-08-31T06:21:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.132+0000 7f6260c00640 1 --2- 172.21.15.16:0/3708253364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a50318d4d0 0x55a50355d3f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f6248102e00 tx=0x7f624803b5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50b43f6c108ff299 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.132+0000 7f6242a00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:21:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.132+0000 7f6262000640 1 --2- 172.21.15.16:0/3708253364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5031ae1b0 0x55a50355d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:21:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.132+0000 7f6242a00640 1 -- 172.21.15.16:0/3708253364 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6248049030 con 0x55a50318d4d0 2024-08-31T06:21:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.132+0000 7f6261600640 1 --2- 172.21.15.16:0/3708253364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5031852f0 0x55a50355ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:21:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.132+0000 7f626b204740 1 -- 172.21.15.16:0/3708253364 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62500035f0 con 0x55a50318d4d0 2024-08-31T06:21:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.132+0000 7f6242a00640 1 -- 172.21.15.16:0/3708253364 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8470+0+0 (secure 0 0 0) 0x7f62480d0290 con 0x55a50318d4d0 2024-08-31T06:21:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.132+0000 7f6242a00640 1 -- 172.21.15.16:0/3708253364 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f62480cbc20 con 0x55a50318d4d0 2024-08-31T06:21:54.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.132+0000 7f626b204740 10 client.28880 fetch_fsmap learned FSMap version 51 2024-08-31T06:21:54.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.132+0000 7f626b204740 1 -- 172.21.15.16:0/3708253364 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55a503292620 con 0x55a50318d4d0 2024-08-31T06:21:54.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.133+0000 7f6242a00640 1 -- 172.21.15.16:0/3708253364 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62500035f0 con 0x55a50318d4d0 2024-08-31T06:21:54.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.133+0000 7f6242a00640 1 -- 172.21.15.16:0/3708253364 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 51) ==== 8354+0+0 (secure 0 0 0) 0x7f6248108ce0 con 0x55a50318d4d0 2024-08-31T06:21:54.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.135+0000 7f626b204740 10 client.28880 fetch_fsmap finished waiting for FSMap version 51 2024-08-31T06:21:54.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.135+0000 7f626b204740 10 client.28880 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:21:54.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.135+0000 7f626b204740 20 client.28880 populate_metadata read hostname 'smithi016' 2024-08-31T06:21:54.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.135+0000 7f626b204740 1 --2- 172.21.15.16:0/3708253364 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a50316cd80 0x55a50356f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:54.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.135+0000 7f626b204740 4 client.28880 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:21:54.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.135+0000 7f626b204740 1 -- 172.21.15.16:0/3708253364 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5034c8ec0 con 0x55a50316cd80 2024-08-31T06:21:54.138 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.136+0000 7f6261600640 1 --2- 172.21.15.16:0/3708253364 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a50316cd80 0x55a50356f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.137+0000 7f6261600640 1 --2- 172.21.15.16:0/3708253364 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a50316cd80 0x55a50356f0b0 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d81119e5353f5b4 server_cookie=5392035b5fee2745 in_seq=0 out_seq=0 2024-08-31T06:21:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.137+0000 7f6242a00640 10 client.28880 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:21:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.138+0000 7f6242a00640 1 -- 172.21.15.16:0/3708253364 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a5034c8ec0 con 0x55a50316cd80 2024-08-31T06:21:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.139+0000 7f6242a00640 10 client.28880 handle_command_reply: tid=0 2024-08-31T06:21:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.139+0000 7f626b204740 1 client.28880 shutdown 2024-08-31T06:21:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.140+0000 7f626b204740 1 -- 172.21.15.16:0/3708253364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a50318d4d0 msgr2=0x55a50355d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.140+0000 7f626b204740 1 --2- 172.21.15.16:0/3708253364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a50318d4d0 0x55a50355d3f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f6248102e00 tx=0x7f624803b5d0 comp rx=0 tx=0).stop 2024-08-31T06:21:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.140+0000 7f626b204740 1 -- 172.21.15.16:0/3708253364 shutdown_connections 2024-08-31T06:21:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.140+0000 7f626b204740 1 --2- 172.21.15.16:0/3708253364 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a50316cd80 0x55a50356f0b0 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.140+0000 7f626b204740 1 --2- 172.21.15.16:0/3708253364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5031ae1b0 0x55a50355d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.140+0000 7f6242a00640 0 client.28880 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:21:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.140+0000 7f626b204740 1 --2- 172.21.15.16:0/3708253364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a50318d4d0 0x55a50355d3f0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.140+0000 7f626b204740 1 --2- 172.21.15.16:0/3708253364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5031852f0 0x55a50355ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.140+0000 7f626b204740 1 -- 172.21.15.16:0/3708253364 >> 172.21.15.16:0/3708253364 conn(0x55a5031a2300 msgr2=0x55a503552950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.140+0000 7f626b204740 1 -- 172.21.15.16:0/3708253364 shutdown_connections 2024-08-31T06:21:54.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.140+0000 7f626b204740 1 -- 172.21.15.16:0/3708253364 wait complete. 2024-08-31T06:21:54.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.141+0000 7f626b204740 20 client.28880 trim_cache size 0 max 16384 2024-08-31T06:21:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.171+0000 7f626b204740 1 Processor -- start 2024-08-31T06:21:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.172+0000 7f626b204740 1 -- start start 2024-08-31T06:21:54.175 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.172+0000 7f626b204740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5031852f0 0x55a50355e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:54.175 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.172+0000 7f626b204740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a50318d4d0 0x55a503574870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:54.176 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.172+0000 7f626b204740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5031ae1b0 0x55a503574db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:54.176 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.172+0000 7f626b204740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f625c1b8410 con 0x55a50318d4d0 2024-08-31T06:21:54.176 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.172+0000 7f626b204740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f625c1b5c80 con 0x55a5031852f0 2024-08-31T06:21:54.176 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.172+0000 7f626b204740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f625c1b3af0 con 0x55a5031ae1b0 2024-08-31T06:21:54.176 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.173+0000 7f6260c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a50318d4d0 0x55a503574870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:54.177 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.173+0000 7f6260c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a50318d4d0 0x55a503574870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38052/0 (socket says 172.21.15.16:38052) 2024-08-31T06:21:54.178 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.173+0000 7f6260c00640 1 -- 172.21.15.16:0/1857596901 learned_addr learned my addr 172.21.15.16:0/1857596901 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:54.178 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.173+0000 7f6261600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5031852f0 0x55a50355e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:54.178 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.175+0000 7f6260c00640 1 -- 172.21.15.16:0/1857596901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5031ae1b0 msgr2=0x55a503574db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:54.178 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.175+0000 7f6260c00640 1 --2- 172.21.15.16:0/1857596901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5031ae1b0 0x55a503574db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:54.178 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.175+0000 7f6260c00640 1 -- 172.21.15.16:0/1857596901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5031852f0 msgr2=0x55a50355e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:54.178 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.175+0000 7f6260c00640 1 --2- 172.21.15.16:0/1857596901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5031852f0 0x55a50355e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:54.178 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.175+0000 7f6260c00640 1 -- 172.21.15.16:0/1857596901 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a50355f280 con 0x55a50318d4d0 2024-08-31T06:21:54.178 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.175+0000 7f6260c00640 1 --2- 172.21.15.16:0/1857596901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a50318d4d0 0x55a503574870 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f6248032af0 tx=0x7f6248003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a114fa2f4c191101 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:54.181 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.179+0000 7f624dc00640 1 -- 172.21.15.16:0/1857596901 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6248049070 con 0x55a50318d4d0 2024-08-31T06:21:54.181 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.179+0000 7f624dc00640 1 -- 172.21.15.16:0/1857596901 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62480cb960 con 0x55a50318d4d0 2024-08-31T06:21:54.181 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.179+0000 7f624dc00640 1 -- 172.21.15.16:0/1857596901 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62480cb5c0 con 0x55a50318d4d0 2024-08-31T06:21:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.180+0000 7f626b204740 1 -- 172.21.15.16:0/1857596901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a50318d4d0 msgr2=0x55a503574870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.180+0000 7f626b204740 1 --2- 172.21.15.16:0/1857596901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a50318d4d0 0x55a503574870 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f6248032af0 tx=0x7f6248003f50 comp rx=0 tx=0).stop 2024-08-31T06:21:54.183 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.181+0000 7f626b204740 1 -- 172.21.15.16:0/1857596901 shutdown_connections 2024-08-31T06:21:54.183 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.181+0000 7f626b204740 1 --2- 172.21.15.16:0/1857596901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5031ae1b0 0x55a503574db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:54.183 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.181+0000 7f626b204740 1 --2- 172.21.15.16:0/1857596901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a50318d4d0 0x55a503574870 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:54.183 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.181+0000 7f626b204740 1 --2- 172.21.15.16:0/1857596901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5031852f0 0x55a50355e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:54.183 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.181+0000 7f626b204740 1 -- 172.21.15.16:0/1857596901 >> 172.21.15.16:0/1857596901 conn(0x55a5031a2300 msgr2=0x55a503553310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:54.184 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.181+0000 7f626b204740 1 -- 172.21.15.16:0/1857596901 shutdown_connections 2024-08-31T06:21:54.184 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.181+0000 7f626b204740 1 -- 172.21.15.16:0/1857596901 wait complete. 2024-08-31T06:21:54.187 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.182+0000 7f626b204740 1 -- start start 2024-08-31T06:21:54.187 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.182+0000 7f626b204740 1 Processor -- start 2024-08-31T06:21:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.186+0000 7f626b204740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5031852f0 0x55a5035d2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.186+0000 7f626b204740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a50318d4d0 0x55a5035d2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.186+0000 7f626b204740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5031ae1b0 0x55a503564fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.186+0000 7f626b204740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f625c1b4140 con 0x55a50318d4d0 2024-08-31T06:21:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.186+0000 7f626b204740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f625c1b55b0 con 0x55a5031ae1b0 2024-08-31T06:21:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.186+0000 7f626b204740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f625c1b4810 con 0x55a5031852f0 2024-08-31T06:21:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.186+0000 7f626b204740 10 client.0 _ll_register_callbacks cb 0x7f6248004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:21:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.186+0000 7f6261600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5031852f0 0x55a5035d2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.188+0000 7f6261600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5031852f0 0x55a5035d2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:33532/0 (socket says 172.21.15.16:33532) 2024-08-31T06:21:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.188+0000 7f6261600640 1 -- 172.21.15.16:0/714139294 learned_addr learned my addr 172.21.15.16:0/714139294 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:21:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.187+0000 7f6262000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5031ae1b0 0x55a503564fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.187+0000 7f6260c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a50318d4d0 0x55a5035d2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.189+0000 7f6262000640 1 -- 172.21.15.16:0/714139294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5031852f0 msgr2=0x55a5035d2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.189+0000 7f6262000640 1 --2- 172.21.15.16:0/714139294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5031852f0 0x55a5035d2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.189+0000 7f6262000640 1 -- 172.21.15.16:0/714139294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a50318d4d0 msgr2=0x55a5035d2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:21:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.189+0000 7f6262000640 1 --2- 172.21.15.16:0/714139294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a50318d4d0 0x55a5035d2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.189+0000 7f6262000640 1 -- 172.21.15.16:0/714139294 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a503565710 con 0x55a5031ae1b0 2024-08-31T06:21:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.189+0000 7f6261600640 1 --2- 172.21.15.16:0/714139294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5031852f0 0x55a5035d2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:21:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.190+0000 7f6262000640 1 --2- 172.21.15.16:0/714139294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5031ae1b0 0x55a503564fd0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f625401f570 tx=0x7f625401f970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2b51f4ab761e5b7 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:21:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.190+0000 7f6242a00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:21:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.191+0000 7f6242a00640 1 -- 172.21.15.16:0/714139294 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625401c610 con 0x55a5031ae1b0 2024-08-31T06:21:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.191+0000 7f6242a00640 1 -- 172.21.15.16:0/714139294 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8470+0+0 (secure 0 0 0) 0x7f6254017030 con 0x55a5031ae1b0 2024-08-31T06:21:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.192+0000 7f626b204740 1 -- 172.21.15.16:0/714139294 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62480cb960 con 0x55a5031ae1b0 2024-08-31T06:21:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.192+0000 7f6242a00640 1 -- 172.21.15.16:0/714139294 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f625401c7b0 con 0x55a5031ae1b0 2024-08-31T06:21:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.192+0000 7f626b204740 10 client.28883 fetch_fsmap learned FSMap version 51 2024-08-31T06:21:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.192+0000 7f626b204740 1 -- 172.21.15.16:0/714139294 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55a5035eaa70 con 0x55a5031ae1b0 2024-08-31T06:21:54.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.195+0000 7f6242a00640 1 -- 172.21.15.16:0/714139294 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f625401ce00 con 0x55a5031ae1b0 2024-08-31T06:21:54.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.195+0000 7f6242a00640 1 -- 172.21.15.16:0/714139294 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 51) ==== 8354+0+0 (secure 0 0 0) 0x7f6254011440 con 0x55a5031ae1b0 2024-08-31T06:21:54.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.195+0000 7f626b204740 10 client.28883 fetch_fsmap finished waiting for FSMap version 51 2024-08-31T06:21:54.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.195+0000 7f626b204740 10 client.28883 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:21:54.198 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.195+0000 7f626b204740 20 client.28883 populate_metadata read hostname 'smithi016' 2024-08-31T06:21:54.198 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.195+0000 7f626b204740 1 --2- 172.21.15.16:0/714139294 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a5031421c0 0x55a5035ed340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:21:54.198 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.195+0000 7f626b204740 4 client.28883 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:21:54.198 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.195+0000 7f626b204740 1 -- 172.21.15.16:0/714139294 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f62480cb960 con 0x55a5031421c0 2024-08-31T06:21:54.199 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.197+0000 7f6261600640 1 --2- 172.21.15.16:0/714139294 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a5031421c0 0x55a5035ed340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:21:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.198+0000 7f6261600640 1 --2- 172.21.15.16:0/714139294 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a5031421c0 0x55a5035ed340 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f0c1e9ead2ea5a6 server_cookie=37f9137822f7070c in_seq=0 out_seq=0 2024-08-31T06:21:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.198+0000 7f6242a00640 10 client.28883 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:21:54.228 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.226+0000 7f6242a00640 1 -- 172.21.15.16:0/714139294 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f62480cb960 con 0x55a5031421c0 2024-08-31T06:21:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.226+0000 7f6242a00640 10 client.28883 handle_command_reply: tid=0 2024-08-31T06:21:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.226+0000 7f626b204740 1 client.28883 shutdown 2024-08-31T06:21:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.226+0000 7f626b204740 1 -- 172.21.15.16:0/714139294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5031ae1b0 msgr2=0x55a503564fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.226+0000 7f626b204740 1 --2- 172.21.15.16:0/714139294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5031ae1b0 0x55a503564fd0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f625401f570 tx=0x7f625401f970 comp rx=0 tx=0).stop 2024-08-31T06:21:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.226+0000 7f626b204740 1 -- 172.21.15.16:0/714139294 shutdown_connections 2024-08-31T06:21:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.226+0000 7f626b204740 1 --2- 172.21.15.16:0/714139294 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55a5031421c0 0x55a5035ed340 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.226+0000 7f626b204740 1 --2- 172.21.15.16:0/714139294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5031ae1b0 0x55a503564fd0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.226+0000 7f626b204740 1 --2- 172.21.15.16:0/714139294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a50318d4d0 0x55a5035d2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.227+0000 7f6242a00640 0 client.28883 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:21:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.227+0000 7f626b204740 1 --2- 172.21.15.16:0/714139294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a5031852f0 0x55a5035d2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.227+0000 7f626b204740 1 -- 172.21.15.16:0/714139294 >> 172.21.15.16:0/714139294 conn(0x55a5031a2300 msgr2=0x55a503553310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.227+0000 7f626b204740 1 -- 172.21.15.16:0/714139294 shutdown_connections 2024-08-31T06:21:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.227+0000 7f626b204740 1 -- 172.21.15.16:0/714139294 wait complete. 2024-08-31T06:21:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.227+0000 7f626b204740 20 client.28883 trim_cache size 0 max 16384 2024-08-31T06:21:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:21:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:21:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:21:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:21:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.229+0000 7f6264000640 1 -- 172.21.15.16:0/2697734145 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f62440b4ca0 msgr2=0x7f62440b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.229+0000 7f6264000640 1 --2- 172.21.15.16:0/2697734145 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f62440b4ca0 0x7f62440b71b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6254005fd0 tx=0x7f62540096a0 comp rx=0 tx=0).stop 2024-08-31T06:21:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.229+0000 7f6264000640 1 -- 172.21.15.16:0/2697734145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f625c1091d0 msgr2=0x7f625c0754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:21:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.229+0000 7f6264000640 1 --2- 172.21.15.16:0/2697734145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f625c1091d0 0x7f625c0754b0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6248006010 tx=0x7f6248004060 comp rx=0 tx=0).stop 2024-08-31T06:21:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.229+0000 7f6264000640 1 -- 172.21.15.16:0/2697734145 shutdown_connections 2024-08-31T06:21:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.229+0000 7f6264000640 1 --2- 172.21.15.16:0/2697734145 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f62440b4ca0 0x7f62440b71b0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.229+0000 7f6264000640 1 --2- 172.21.15.16:0/2697734145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f625c109b20 0x7f625c0759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.229+0000 7f6264000640 1 --2- 172.21.15.16:0/2697734145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f625c1091d0 0x7f625c0754b0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.229+0000 7f6264000640 1 --2- 172.21.15.16:0/2697734145 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f625c1038a0 0x7f625c076ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:21:54.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.229+0000 7f6264000640 1 -- 172.21.15.16:0/2697734145 >> 172.21.15.16:0/2697734145 conn(0x7f625c0ff2a0 msgr2=0x7f625c108890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:21:54.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.230+0000 7f6264000640 1 -- 172.21.15.16:0/2697734145 shutdown_connections 2024-08-31T06:21:54.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:21:54.230+0000 7f6264000640 1 -- 172.21.15.16:0/2697734145 wait complete. 2024-08-31T06:21:55.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:54 smithi155 ceph-mon[21742]: pgmap v822: 97 pgs: 97 active+clean; 3.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 5.4 MiB/s wr, 579 op/s 2024-08-31T06:21:55.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:54 smithi016 ceph-mon[17194]: pgmap v822: 97 pgs: 97 active+clean; 3.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 5.4 MiB/s wr, 579 op/s 2024-08-31T06:21:55.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:54 smithi184 ceph-mon[22331]: pgmap v822: 97 pgs: 97 active+clean; 3.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 5.4 MiB/s wr, 579 op/s 2024-08-31T06:21:56.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 870.0 s, 1623.2 tps, lat 61.404 ms stddev 75.084 2024-08-31T06:21:57.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:56 smithi155 ceph-mon[21742]: pgmap v823: 97 pgs: 97 active+clean; 3.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 6.9 MiB/s wr, 741 op/s 2024-08-31T06:21:57.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:56 smithi016 ceph-mon[17194]: pgmap v823: 97 pgs: 97 active+clean; 3.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 6.9 MiB/s wr, 741 op/s 2024-08-31T06:21:57.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:56 smithi184 ceph-mon[22331]: pgmap v823: 97 pgs: 97 active+clean; 3.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 6.9 MiB/s wr, 741 op/s 2024-08-31T06:21:59.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:21:58 smithi155 ceph-mon[21742]: pgmap v824: 97 pgs: 97 active+clean; 3.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 5.2 MiB/s wr, 569 op/s 2024-08-31T06:21:59.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:21:58 smithi016 ceph-mon[17194]: pgmap v824: 97 pgs: 97 active+clean; 3.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 5.2 MiB/s wr, 569 op/s 2024-08-31T06:21:59.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:21:58 smithi184 ceph-mon[22331]: pgmap v824: 97 pgs: 97 active+clean; 3.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 5.2 MiB/s wr, 569 op/s 2024-08-31T06:22:01.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:00 smithi155 ceph-mon[21742]: pgmap v825: 97 pgs: 97 active+clean; 3.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 5.8 MiB/s wr, 649 op/s 2024-08-31T06:22:01.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:00 smithi016 ceph-mon[17194]: pgmap v825: 97 pgs: 97 active+clean; 3.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 5.8 MiB/s wr, 649 op/s 2024-08-31T06:22:01.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:00 smithi184 ceph-mon[22331]: pgmap v825: 97 pgs: 97 active+clean; 3.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 5.8 MiB/s wr, 649 op/s 2024-08-31T06:22:01.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 875.0 s, 1306.2 tps, lat 69.694 ms stddev 86.199 2024-08-31T06:22:03.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:02 smithi155 ceph-mon[21742]: pgmap v826: 97 pgs: 97 active+clean; 3.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 6.6 MiB/s wr, 712 op/s 2024-08-31T06:22:03.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:02 smithi016 ceph-mon[17194]: pgmap v826: 97 pgs: 97 active+clean; 3.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 6.6 MiB/s wr, 712 op/s 2024-08-31T06:22:03.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:02 smithi184 ceph-mon[22331]: pgmap v826: 97 pgs: 97 active+clean; 3.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 6.6 MiB/s wr, 712 op/s 2024-08-31T06:22:05.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:04 smithi155 ceph-mon[21742]: pgmap v827: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 8.6 MiB/s wr, 882 op/s 2024-08-31T06:22:05.165 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:04 smithi016 ceph-mon[17194]: pgmap v827: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 8.6 MiB/s wr, 882 op/s 2024-08-31T06:22:05.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:04 smithi184 ceph-mon[22331]: pgmap v827: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 8.6 MiB/s wr, 882 op/s 2024-08-31T06:22:06.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 880.0 s, 1806.2 tps, lat 60.409 ms stddev 115.269 2024-08-31T06:22:07.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:06 smithi155 ceph-mon[21742]: pgmap v828: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 16 MiB/s wr, 1.58k op/s 2024-08-31T06:22:07.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:06 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:22:07.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:06 smithi016 ceph-mon[17194]: pgmap v828: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 16 MiB/s wr, 1.58k op/s 2024-08-31T06:22:07.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:06 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:22:07.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:06 smithi184 ceph-mon[22331]: pgmap v828: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 16 MiB/s wr, 1.58k op/s 2024-08-31T06:22:07.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:06 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:22:07.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.645+0000 7efd32a00640 1 -- 172.21.15.16:0/402756238 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd20030850 con 0x7efd34105020 2024-08-31T06:22:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.646+0000 7efd3ae00640 1 -- 172.21.15.16:0/402756238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd34105020 msgr2=0x7efd3410d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.646+0000 7efd3ae00640 1 --2- 172.21.15.16:0/402756238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd34105020 0x7efd3410d560 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7efd20009860 tx=0x7efd2002a340 comp rx=0 tx=0).stop 2024-08-31T06:22:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.646+0000 7efd3ae00640 1 -- 172.21.15.16:0/402756238 shutdown_connections 2024-08-31T06:22:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.646+0000 7efd3ae00640 1 --2- 172.21.15.16:0/402756238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd34105020 0x7efd3410d560 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:07.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.646+0000 7efd3ae00640 1 --2- 172.21.15.16:0/402756238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd3410db30 0x7efd34110020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:07.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.646+0000 7efd3ae00640 1 --2- 172.21.15.16:0/402756238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd341046d0 0x7efd34104ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:07.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.646+0000 7efd3ae00640 1 -- 172.21.15.16:0/402756238 >> 172.21.15.16:0/402756238 conn(0x7efd340fe160 msgr2=0x7efd341005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:07.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.646+0000 7efd3ae00640 1 -- 172.21.15.16:0/402756238 shutdown_connections 2024-08-31T06:22:07.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.646+0000 7efd3ae00640 1 -- 172.21.15.16:0/402756238 wait complete. 2024-08-31T06:22:07.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.648+0000 7efd3ae00640 1 Processor -- start 2024-08-31T06:22:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.648+0000 7efd3ae00640 1 -- start start 2024-08-31T06:22:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.648+0000 7efd3ae00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd341046d0 0x7efd3419a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.648+0000 7efd3ae00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd34105020 0x7efd3419a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.648+0000 7efd3ae00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd3410db30 0x7efd3419fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.648+0000 7efd3ae00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efd341137b0 con 0x7efd3410db30 2024-08-31T06:22:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.649+0000 7efd3ae00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efd34113630 con 0x7efd34105020 2024-08-31T06:22:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.649+0000 7efd3ae00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efd34113930 con 0x7efd341046d0 2024-08-31T06:22:07.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.649+0000 7efd33e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd341046d0 0x7efd3419a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:07.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.650+0000 7efd33e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd341046d0 0x7efd3419a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:56212/0 (socket says 172.21.15.16:56212) 2024-08-31T06:22:07.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.650+0000 7efd33e00640 1 -- 172.21.15.16:0/958674398 learned_addr learned my addr 172.21.15.16:0/958674398 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:07.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.650+0000 7efd38e00640 1 --2- 172.21.15.16:0/958674398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd3410db30 0x7efd3419fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:07.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.650+0000 7efd33400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd34105020 0x7efd3419a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:07.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.651+0000 7efd33e00640 1 -- 172.21.15.16:0/958674398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd34105020 msgr2=0x7efd3419a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:07.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.651+0000 7efd33e00640 1 --2- 172.21.15.16:0/958674398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd34105020 0x7efd3419a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:07.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.651+0000 7efd33e00640 1 -- 172.21.15.16:0/958674398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd3410db30 msgr2=0x7efd3419fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:07.654 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.651+0000 7efd33e00640 1 --2- 172.21.15.16:0/958674398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd3410db30 0x7efd3419fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:07.654 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.651+0000 7efd33e00640 1 -- 172.21.15.16:0/958674398 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd341a0180 con 0x7efd341046d0 2024-08-31T06:22:07.654 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.651+0000 7efd33e00640 1 --2- 172.21.15.16:0/958674398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd341046d0 0x7efd3419a250 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7efd2400e9c0 tx=0x7efd2400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:07.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.652+0000 7efd30c00640 1 -- 172.21.15.16:0/958674398 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2400ccc0 con 0x7efd341046d0 2024-08-31T06:22:07.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.652+0000 7efd3ae00640 1 -- 172.21.15.16:0/958674398 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd341a0470 con 0x7efd341046d0 2024-08-31T06:22:07.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.652+0000 7efd30c00640 1 -- 172.21.15.16:0/958674398 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd240054d0 con 0x7efd341046d0 2024-08-31T06:22:07.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.652+0000 7efd3ae00640 1 -- 172.21.15.16:0/958674398 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd341022e0 con 0x7efd341046d0 2024-08-31T06:22:07.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.652+0000 7efd30c00640 1 -- 172.21.15.16:0/958674398 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd24010430 con 0x7efd341046d0 2024-08-31T06:22:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.657+0000 7efd42006740 1 Processor -- start 2024-08-31T06:22:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.658+0000 7efd42006740 1 -- start start 2024-08-31T06:22:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.658+0000 7efd42006740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562952eac1b0 0x56295324f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.658+0000 7efd42006740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562952e832f0 0x56295324fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.658+0000 7efd42006740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562952e8b4d0 0x5629532503a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.658+0000 7efd42006740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5629531c4f70 con 0x562952e8b4d0 2024-08-31T06:22:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.658+0000 7efd42006740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5629531b4500 con 0x562952e832f0 2024-08-31T06:22:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.658+0000 7efd42006740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5629531b84b0 con 0x562952eac1b0 2024-08-31T06:22:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.658+0000 7efd38e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562952eac1b0 0x56295324f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.658+0000 7efd38e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562952eac1b0 0x56295324f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:56228/0 (socket says 172.21.15.16:56228) 2024-08-31T06:22:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.658+0000 7efd38e00640 1 -- 172.21.15.16:0/1931303210 learned_addr learned my addr 172.21.15.16:0/1931303210 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.658+0000 7efd33400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562952e8b4d0 0x5629532503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.659+0000 7efd38e00640 1 -- 172.21.15.16:0/1931303210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562952e832f0 msgr2=0x56295324fe60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:22:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.659+0000 7efd38e00640 1 --2- 172.21.15.16:0/1931303210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562952e832f0 0x56295324fe60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.659+0000 7efd38e00640 1 -- 172.21.15.16:0/1931303210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562952e8b4d0 msgr2=0x5629532503a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.659+0000 7efd38e00640 1 --2- 172.21.15.16:0/1931303210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562952e8b4d0 0x5629532503a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.659+0000 7efd38e00640 1 -- 172.21.15.16:0/1931303210 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56295316ec10 con 0x562952eac1b0 2024-08-31T06:22:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.659+0000 7efd33400640 1 --2- 172.21.15.16:0/1931303210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562952e8b4d0 0x5629532503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:22:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.660+0000 7efd38e00640 1 --2- 172.21.15.16:0/1931303210 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562952eac1b0 0x56295324f920 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7efd2c009fb0 tx=0x7efd2c012b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d729f3caa9e9e7e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.660+0000 7efd30c00640 1 -- 172.21.15.16:0/958674398 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7efd24004f80 con 0x7efd341046d0 2024-08-31T06:22:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.660+0000 7efd29600640 1 -- 172.21.15.16:0/1931303210 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2c020070 con 0x562952eac1b0 2024-08-31T06:22:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.661+0000 7efd29600640 1 -- 172.21.15.16:0/1931303210 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd2c00cd60 con 0x562952eac1b0 2024-08-31T06:22:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.661+0000 7efd29600640 1 -- 172.21.15.16:0/1931303210 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2c01a370 con 0x562952eac1b0 2024-08-31T06:22:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.662+0000 7efd42006740 1 -- 172.21.15.16:0/1931303210 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562952eac1b0 msgr2=0x56295324f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.662+0000 7efd42006740 1 --2- 172.21.15.16:0/1931303210 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562952eac1b0 0x56295324f920 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7efd2c009fb0 tx=0x7efd2c012b90 comp rx=0 tx=0).stop 2024-08-31T06:22:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.663+0000 7efd42006740 1 -- 172.21.15.16:0/1931303210 shutdown_connections 2024-08-31T06:22:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.663+0000 7efd42006740 1 --2- 172.21.15.16:0/1931303210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562952e8b4d0 0x5629532503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.663+0000 7efd42006740 1 --2- 172.21.15.16:0/1931303210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562952e832f0 0x56295324fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.663+0000 7efd42006740 1 --2- 172.21.15.16:0/1931303210 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562952eac1b0 0x56295324f920 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.663+0000 7efd42006740 1 -- 172.21.15.16:0/1931303210 >> 172.21.15.16:0/1931303210 conn(0x562952ea0300 msgr2=0x56295324d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.664+0000 7efd42006740 1 -- 172.21.15.16:0/1931303210 shutdown_connections 2024-08-31T06:22:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.664+0000 7efd42006740 1 -- 172.21.15.16:0/1931303210 wait complete. 2024-08-31T06:22:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.665+0000 7efd42006740 1 -- start start 2024-08-31T06:22:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.665+0000 7efd42006740 1 Processor -- start 2024-08-31T06:22:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.666+0000 7efd42006740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562952e832f0 0x56295325b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.666+0000 7efd42006740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562952e8b4d0 0x56295325b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.666+0000 7efd42006740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562952eac1b0 0x5629532673f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.666+0000 7efd42006740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5629530edab0 con 0x562952eac1b0 2024-08-31T06:22:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.666+0000 7efd42006740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56295315b320 con 0x562952e832f0 2024-08-31T06:22:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.666+0000 7efd42006740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56295325e0a0 con 0x562952e8b4d0 2024-08-31T06:22:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.667+0000 7efd30c00640 1 --2- 172.21.15.16:0/958674398 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7efd180b4be0 0x7efd180b70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.667+0000 7efd30c00640 1 -- 172.21.15.16:0/958674398 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8470+0+0 (secure 0 0 0) 0x7efd240e58e0 con 0x7efd341046d0 2024-08-31T06:22:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.667+0000 7efd42006740 10 client.0 _ll_register_callbacks cb 0x5629531616a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:22:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.668+0000 7efd38e00640 1 --2- 172.21.15.16:0/958674398 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7efd180b4be0 0x7efd180b70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.668+0000 7efd38e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562952e832f0 0x56295325b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.668+0000 7efd33e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562952e8b4d0 0x56295325b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.668+0000 7efd33e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562952e8b4d0 0x56295325b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:56242/0 (socket says 172.21.15.16:56242) 2024-08-31T06:22:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.668+0000 7efd38e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562952e832f0 0x56295325b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:50828/0 (socket says 172.21.15.16:50828) 2024-08-31T06:22:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.668+0000 7efd33e00640 1 -- 172.21.15.16:0/1289244143 learned_addr learned my addr 172.21.15.16:0/1289244143 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.668+0000 7efd33400640 1 --2- 172.21.15.16:0/1289244143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562952eac1b0 0x5629532673f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.669+0000 7efd38e00640 1 -- 172.21.15.16:0/1289244143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562952e8b4d0 msgr2=0x56295325b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.669+0000 7efd38e00640 1 --2- 172.21.15.16:0/1289244143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562952e8b4d0 0x56295325b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.669+0000 7efd38e00640 1 -- 172.21.15.16:0/1289244143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562952eac1b0 msgr2=0x5629532673f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.669+0000 7efd38e00640 1 --2- 172.21.15.16:0/1289244143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562952eac1b0 0x5629532673f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.669+0000 7efd38e00640 1 -- 172.21.15.16:0/1289244143 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562952f17790 con 0x562952e832f0 2024-08-31T06:22:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.669+0000 7efd33400640 1 --2- 172.21.15.16:0/1289244143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562952eac1b0 0x5629532673f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:22:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.669+0000 7efd38e00640 1 --2- 172.21.15.16:0/1289244143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562952e832f0 0x56295325b020 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7efd2c009fb0 tx=0x7efd2c009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=190a6d959317890d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.669+0000 7efd1e000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:22:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.670+0000 7efd38e00640 1 --2- 172.21.15.16:0/958674398 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7efd180b4be0 0x7efd180b70f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7efd2c009d90 tx=0x7efd2c01ba00 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.670+0000 7efd1e000640 1 -- 172.21.15.16:0/1289244143 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2c020070 con 0x562952e832f0 2024-08-31T06:22:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.670+0000 7efd42006740 1 -- 172.21.15.16:0/1289244143 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd2c00cd60 con 0x562952e832f0 2024-08-31T06:22:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.670+0000 7efd1e000640 1 -- 172.21.15.16:0/1289244143 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8470+0+0 (secure 0 0 0) 0x7efd2c010de0 con 0x562952e832f0 2024-08-31T06:22:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.670+0000 7efd1e000640 1 -- 172.21.15.16:0/1289244143 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7efd2c037050 con 0x562952e832f0 2024-08-31T06:22:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.670+0000 7efd42006740 10 client.28892 fetch_fsmap learned FSMap version 51 2024-08-31T06:22:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.670+0000 7efd42006740 1 -- 172.21.15.16:0/1289244143 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x562952f90620 con 0x562952e832f0 2024-08-31T06:22:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.670+0000 7efd1e000640 1 -- 172.21.15.16:0/1289244143 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd2c00cd60 con 0x562952e832f0 2024-08-31T06:22:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.671+0000 7efd1e000640 1 -- 172.21.15.16:0/1289244143 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 51) ==== 8354+0+0 (secure 0 0 0) 0x7efd2c026040 con 0x562952e832f0 2024-08-31T06:22:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.671+0000 7efd42006740 10 client.28892 fetch_fsmap finished waiting for FSMap version 51 2024-08-31T06:22:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.671+0000 7efd42006740 10 client.28892 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:22:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.671+0000 7efd42006740 20 client.28892 populate_metadata read hostname 'smithi016' 2024-08-31T06:22:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.671+0000 7efd42006740 1 --2- 172.21.15.16:0/1289244143 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x562952e6ad80 0x56295326ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.671+0000 7efd42006740 4 client.28892 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:22:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.671+0000 7efd42006740 1 -- 172.21.15.16:0/1289244143 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629531c6ec0 con 0x562952e6ad80 2024-08-31T06:22:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.672+0000 7efd33e00640 1 --2- 172.21.15.16:0/1289244143 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x562952e6ad80 0x56295326ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.673+0000 7efd1e000640 10 client.28892 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:22:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.673+0000 7efd33e00640 1 --2- 172.21.15.16:0/1289244143 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x562952e6ad80 0x56295326ace0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c05e3af3536ab90d server_cookie=315cc446bd41c775 in_seq=0 out_seq=0 2024-08-31T06:22:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.674+0000 7efd1e000640 1 -- 172.21.15.16:0/1289244143 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5629531c6ec0 con 0x562952e6ad80 2024-08-31T06:22:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.674+0000 7efd1e000640 10 client.28892 handle_command_reply: tid=0 2024-08-31T06:22:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.674+0000 7efd42006740 1 client.28892 shutdown 2024-08-31T06:22:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.674+0000 7efd42006740 1 -- 172.21.15.16:0/1289244143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562952e832f0 msgr2=0x56295325b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.674+0000 7efd42006740 1 --2- 172.21.15.16:0/1289244143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562952e832f0 0x56295325b020 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7efd2c009fb0 tx=0x7efd2c009840 comp rx=0 tx=0).stop 2024-08-31T06:22:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.674+0000 7efd42006740 1 -- 172.21.15.16:0/1289244143 shutdown_connections 2024-08-31T06:22:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.674+0000 7efd42006740 1 --2- 172.21.15.16:0/1289244143 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x562952e6ad80 0x56295326ace0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.674+0000 7efd42006740 1 --2- 172.21.15.16:0/1289244143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562952e8b4d0 0x56295325b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.674+0000 7efd42006740 1 --2- 172.21.15.16:0/1289244143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562952eac1b0 0x5629532673f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.674+0000 7efd42006740 1 --2- 172.21.15.16:0/1289244143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562952e832f0 0x56295325b020 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.674+0000 7efd42006740 1 -- 172.21.15.16:0/1289244143 >> 172.21.15.16:0/1289244143 conn(0x562952ea0300 msgr2=0x562953250950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.674+0000 7efd1e000640 0 client.28892 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:22:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.675+0000 7efd42006740 1 -- 172.21.15.16:0/1289244143 shutdown_connections 2024-08-31T06:22:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.675+0000 7efd42006740 1 -- 172.21.15.16:0/1289244143 wait complete. 2024-08-31T06:22:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.675+0000 7efd42006740 20 client.28892 trim_cache size 0 max 16384 2024-08-31T06:22:07.707 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.704+0000 7efd42006740 1 Processor -- start 2024-08-31T06:22:07.707 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.704+0000 7efd42006740 1 -- start start 2024-08-31T06:22:07.707 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.704+0000 7efd42006740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562952e832f0 0x562953256800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:07.707 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.704+0000 7efd42006740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562952e8b4d0 0x5629532672f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:07.707 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.704+0000 7efd42006740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562952eac1b0 0x56295325cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:07.707 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.704+0000 7efd42006740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efd341b8490 con 0x562952e832f0 2024-08-31T06:22:07.707 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.704+0000 7efd42006740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efd34073480 con 0x562952e8b4d0 2024-08-31T06:22:07.707 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.704+0000 7efd42006740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efd340712f0 con 0x562952eac1b0 2024-08-31T06:22:07.707 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.704+0000 7efd33400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562952e8b4d0 0x5629532672f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:07.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.705+0000 7efd33e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562952e832f0 0x562953256800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:07.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.705+0000 7efd33e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562952e832f0 0x562953256800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44014/0 (socket says 172.21.15.16:44014) 2024-08-31T06:22:07.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.705+0000 7efd33e00640 1 -- 172.21.15.16:0/1654286620 learned_addr learned my addr 172.21.15.16:0/1654286620 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:07.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.705+0000 7efd38e00640 1 --2- 172.21.15.16:0/1654286620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562952eac1b0 0x56295325cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:07.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.704+0000 7efd33400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562952e8b4d0 0x5629532672f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:50844/0 (socket says 172.21.15.16:50844) 2024-08-31T06:22:07.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.706+0000 7efd38e00640 1 -- 172.21.15.16:0/1654286620 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562952e8b4d0 msgr2=0x5629532672f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:07.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.706+0000 7efd38e00640 1 --2- 172.21.15.16:0/1654286620 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562952e8b4d0 0x5629532672f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:07.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.706+0000 7efd38e00640 1 -- 172.21.15.16:0/1654286620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562952e832f0 msgr2=0x562953256800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:07.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.706+0000 7efd38e00640 1 --2- 172.21.15.16:0/1654286620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562952e832f0 0x562953256800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:07.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.706+0000 7efd38e00640 1 -- 172.21.15.16:0/1654286620 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56295326b050 con 0x562952eac1b0 2024-08-31T06:22:07.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.706+0000 7efd33400640 1 --2- 172.21.15.16:0/1654286620 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562952e8b4d0 0x5629532672f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:22:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.707+0000 7efd38e00640 1 --2- 172.21.15.16:0/1654286620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562952eac1b0 0x56295325cc50 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7efd2c0098b0 tx=0x7efd2c00c750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3076f87163869a52 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.707+0000 7efd28c00640 1 -- 172.21.15.16:0/1654286620 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2c020070 con 0x562952eac1b0 2024-08-31T06:22:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.708+0000 7efd28c00640 1 -- 172.21.15.16:0/1654286620 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd2c02b030 con 0x562952eac1b0 2024-08-31T06:22:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.708+0000 7efd28c00640 1 -- 172.21.15.16:0/1654286620 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2c0056e0 con 0x562952eac1b0 2024-08-31T06:22:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.709+0000 7efd42006740 1 -- 172.21.15.16:0/1654286620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562952eac1b0 msgr2=0x56295325cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.709+0000 7efd42006740 1 --2- 172.21.15.16:0/1654286620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562952eac1b0 0x56295325cc50 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7efd2c0098b0 tx=0x7efd2c00c750 comp rx=0 tx=0).stop 2024-08-31T06:22:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.709+0000 7efd42006740 1 -- 172.21.15.16:0/1654286620 shutdown_connections 2024-08-31T06:22:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.709+0000 7efd42006740 1 --2- 172.21.15.16:0/1654286620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562952eac1b0 0x56295325cc50 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.709+0000 7efd42006740 1 --2- 172.21.15.16:0/1654286620 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562952e8b4d0 0x5629532672f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.709+0000 7efd42006740 1 --2- 172.21.15.16:0/1654286620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562952e832f0 0x562953256800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.709+0000 7efd42006740 1 -- 172.21.15.16:0/1654286620 >> 172.21.15.16:0/1654286620 conn(0x562952ea0300 msgr2=0x56295325c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.709+0000 7efd42006740 1 -- 172.21.15.16:0/1654286620 shutdown_connections 2024-08-31T06:22:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.709+0000 7efd42006740 1 -- 172.21.15.16:0/1654286620 wait complete. 2024-08-31T06:22:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.710+0000 7efd42006740 1 -- start start 2024-08-31T06:22:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.710+0000 7efd42006740 1 Processor -- start 2024-08-31T06:22:07.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.710+0000 7efd42006740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562952e832f0 0x56295325fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:07.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.710+0000 7efd42006740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562952e8b4d0 0x5629532601b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:07.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.710+0000 7efd42006740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562952eac1b0 0x5629532cf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:07.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.710+0000 7efd42006740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efd34071940 con 0x562952e832f0 2024-08-31T06:22:07.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.710+0000 7efd42006740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efd34072db0 con 0x562952eac1b0 2024-08-31T06:22:07.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.710+0000 7efd42006740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efd34072010 con 0x562952e8b4d0 2024-08-31T06:22:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.710+0000 7efd42006740 10 client.0 _ll_register_callbacks cb 0x7efd2c01b470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:22:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.711+0000 7efd38e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562952eac1b0 0x5629532cf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.711+0000 7efd33e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562952e832f0 0x56295325fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.711+0000 7efd38e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562952eac1b0 0x5629532cf6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:50856/0 (socket says 172.21.15.16:50856) 2024-08-31T06:22:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.711+0000 7efd38e00640 1 -- 172.21.15.16:0/4288142284 learned_addr learned my addr 172.21.15.16:0/4288142284 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.711+0000 7efd33400640 1 --2- 172.21.15.16:0/4288142284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562952e8b4d0 0x5629532601b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.711+0000 7efd33400640 1 -- 172.21.15.16:0/4288142284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562952eac1b0 msgr2=0x5629532cf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.711+0000 7efd33400640 1 --2- 172.21.15.16:0/4288142284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562952eac1b0 0x5629532cf6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.711+0000 7efd33400640 1 -- 172.21.15.16:0/4288142284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562952e832f0 msgr2=0x56295325fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.711+0000 7efd33400640 1 --2- 172.21.15.16:0/4288142284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562952e832f0 0x56295325fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.711+0000 7efd33400640 1 -- 172.21.15.16:0/4288142284 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629532cfd60 con 0x562952e8b4d0 2024-08-31T06:22:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.712+0000 7efd38e00640 1 --2- 172.21.15.16:0/4288142284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562952eac1b0 0x5629532cf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:22:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.712+0000 7efd1e000640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:22:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.712+0000 7efd33400640 1 --2- 172.21.15.16:0/4288142284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562952e8b4d0 0x5629532601b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7efd20009780 tx=0x7efd20030f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11a2428e20f63ac9 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.713+0000 7efd1e000640 1 -- 172.21.15.16:0/4288142284 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd20041070 con 0x562952e8b4d0 2024-08-31T06:22:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.713+0000 7efd1e000640 1 -- 172.21.15.16:0/4288142284 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8470+0+0 (secure 0 0 0) 0x7efd20005020 con 0x562952e8b4d0 2024-08-31T06:22:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.713+0000 7efd42006740 1 -- 172.21.15.16:0/4288142284 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd2c02b030 con 0x562952e8b4d0 2024-08-31T06:22:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.714+0000 7efd1e000640 1 -- 172.21.15.16:0/4288142284 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7efd20037040 con 0x562952e8b4d0 2024-08-31T06:22:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.714+0000 7efd42006740 10 client.28924 fetch_fsmap learned FSMap version 51 2024-08-31T06:22:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.714+0000 7efd42006740 1 -- 172.21.15.16:0/4288142284 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5629532d7090 con 0x562952e8b4d0 2024-08-31T06:22:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.714+0000 7efd1e000640 1 -- 172.21.15.16:0/4288142284 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd2c02b030 con 0x562952e8b4d0 2024-08-31T06:22:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.715+0000 7efd1e000640 1 -- 172.21.15.16:0/4288142284 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 51) ==== 8354+0+0 (secure 0 0 0) 0x7efd200035f0 con 0x562952e8b4d0 2024-08-31T06:22:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.715+0000 7efd42006740 10 client.28924 fetch_fsmap finished waiting for FSMap version 51 2024-08-31T06:22:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.715+0000 7efd42006740 10 client.28924 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:22:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.715+0000 7efd42006740 20 client.28924 populate_metadata read hostname 'smithi016' 2024-08-31T06:22:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.715+0000 7efd42006740 1 --2- 172.21.15.16:0/4288142284 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x562952e401c0 0x5629532d9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.715+0000 7efd42006740 4 client.28924 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:22:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.715+0000 7efd42006740 1 -- 172.21.15.16:0/4288142284 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5629531c2960 con 0x562952e401c0 2024-08-31T06:22:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.715+0000 7efd33e00640 1 --2- 172.21.15.16:0/4288142284 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x562952e401c0 0x5629532d9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.716+0000 7efd33e00640 1 --2- 172.21.15.16:0/4288142284 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x562952e401c0 0x5629532d9880 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3191c071699c6b74 server_cookie=604729843ed23728 in_seq=0 out_seq=0 2024-08-31T06:22:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.716+0000 7efd1e000640 10 client.28924 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:22:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.731+0000 7efd1e000640 1 -- 172.21.15.16:0/4288142284 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5629531c2960 con 0x562952e401c0 2024-08-31T06:22:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.731+0000 7efd1e000640 10 client.28924 handle_command_reply: tid=0 2024-08-31T06:22:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.731+0000 7efd42006740 1 client.28924 shutdown 2024-08-31T06:22:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.732+0000 7efd42006740 1 -- 172.21.15.16:0/4288142284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562952e8b4d0 msgr2=0x5629532601b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.732+0000 7efd42006740 1 --2- 172.21.15.16:0/4288142284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562952e8b4d0 0x5629532601b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7efd20009780 tx=0x7efd20030f40 comp rx=0 tx=0).stop 2024-08-31T06:22:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.733+0000 7efd42006740 1 -- 172.21.15.16:0/4288142284 shutdown_connections 2024-08-31T06:22:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.733+0000 7efd42006740 1 --2- 172.21.15.16:0/4288142284 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x562952e401c0 0x5629532d9880 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.733+0000 7efd42006740 1 --2- 172.21.15.16:0/4288142284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562952e8b4d0 0x5629532601b0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.733+0000 7efd42006740 1 --2- 172.21.15.16:0/4288142284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562952eac1b0 0x5629532cf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.733+0000 7efd42006740 1 --2- 172.21.15.16:0/4288142284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562952e832f0 0x56295325fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.733+0000 7efd42006740 1 -- 172.21.15.16:0/4288142284 >> 172.21.15.16:0/4288142284 conn(0x562952ea0300 msgr2=0x56295325cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.733+0000 7efd1e000640 0 client.28924 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:22:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.733+0000 7efd42006740 1 -- 172.21.15.16:0/4288142284 shutdown_connections 2024-08-31T06:22:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.733+0000 7efd42006740 1 -- 172.21.15.16:0/4288142284 wait complete. 2024-08-31T06:22:07.737 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.734+0000 7efd42006740 20 client.28924 trim_cache size 0 max 16384 2024-08-31T06:22:07.737 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:22:07.737 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:22:07.737 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:22:07.737 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:22:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.737+0000 7efd3ae00640 1 -- 172.21.15.16:0/958674398 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7efd180b4be0 msgr2=0x7efd180b70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.737+0000 7efd3ae00640 1 --2- 172.21.15.16:0/958674398 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7efd180b4be0 0x7efd180b70f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7efd2c009d90 tx=0x7efd2c01ba00 comp rx=0 tx=0).stop 2024-08-31T06:22:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.737+0000 7efd3ae00640 1 -- 172.21.15.16:0/958674398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd341046d0 msgr2=0x7efd3419a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.737+0000 7efd3ae00640 1 --2- 172.21.15.16:0/958674398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd341046d0 0x7efd3419a250 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7efd2400e9c0 tx=0x7efd2400ee80 comp rx=0 tx=0).stop 2024-08-31T06:22:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.738+0000 7efd3ae00640 1 -- 172.21.15.16:0/958674398 shutdown_connections 2024-08-31T06:22:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.738+0000 7efd3ae00640 1 --2- 172.21.15.16:0/958674398 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7efd180b4be0 0x7efd180b70f0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.738+0000 7efd3ae00640 1 --2- 172.21.15.16:0/958674398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd3410db30 0x7efd3419fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.738+0000 7efd3ae00640 1 --2- 172.21.15.16:0/958674398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd34105020 0x7efd3419a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.738+0000 7efd3ae00640 1 --2- 172.21.15.16:0/958674398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd341046d0 0x7efd3419a250 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.738+0000 7efd3ae00640 1 -- 172.21.15.16:0/958674398 >> 172.21.15.16:0/958674398 conn(0x7efd340fe160 msgr2=0x7efd34107640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.739+0000 7efd3ae00640 1 -- 172.21.15.16:0/958674398 shutdown_connections 2024-08-31T06:22:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:07.741+0000 7efd3ae00640 1 -- 172.21.15.16:0/958674398 wait complete. 2024-08-31T06:22:09.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:08 smithi155 ceph-mon[21742]: pgmap v829: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 14 MiB/s wr, 1.39k op/s 2024-08-31T06:22:09.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:08 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:22:09.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:08 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:22:09.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:08 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:22:09.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:08 smithi016 ceph-mon[17194]: pgmap v829: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 14 MiB/s wr, 1.39k op/s 2024-08-31T06:22:09.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:08 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:22:09.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:08 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:22:09.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:08 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:22:09.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:08 smithi184 ceph-mon[22331]: pgmap v829: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 14 MiB/s wr, 1.39k op/s 2024-08-31T06:22:09.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:08 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:22:09.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:08 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:22:09.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:08 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:22:10.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:10 smithi155 ceph-mon[21742]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:22:10.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:10 smithi155 ceph-mon[21742]: osdmap e100: 12 total, 12 up, 12 in 2024-08-31T06:22:10.622 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:10 smithi155 ceph-mon[21742]: pgmap v831: 97 pgs: 3 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 91 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 18 MiB/s wr, 1.70k op/s 2024-08-31T06:22:10.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:10 smithi016 ceph-mon[17194]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:22:10.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:10 smithi016 ceph-mon[17194]: osdmap e100: 12 total, 12 up, 12 in 2024-08-31T06:22:10.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:10 smithi016 ceph-mon[17194]: pgmap v831: 97 pgs: 3 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 91 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 18 MiB/s wr, 1.70k op/s 2024-08-31T06:22:10.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:10 smithi184 ceph-mon[22331]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:22:10.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:10 smithi184 ceph-mon[22331]: osdmap e100: 12 total, 12 up, 12 in 2024-08-31T06:22:10.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:10 smithi184 ceph-mon[22331]: pgmap v831: 97 pgs: 3 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 91 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 18 MiB/s wr, 1.70k op/s 2024-08-31T06:22:11.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 885.0 s, 1691.4 tps, lat 58.988 ms stddev 83.475 2024-08-31T06:22:13.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:12 smithi155 ceph-mon[21742]: pgmap v832: 97 pgs: 3 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 90 active+clean; 3.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 17 MiB/s wr, 1.67k op/s 2024-08-31T06:22:13.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:12 smithi016 ceph-mon[17194]: pgmap v832: 97 pgs: 3 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 90 active+clean; 3.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 17 MiB/s wr, 1.67k op/s 2024-08-31T06:22:13.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:12 smithi184 ceph-mon[22331]: pgmap v832: 97 pgs: 3 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 90 active+clean; 3.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 17 MiB/s wr, 1.67k op/s 2024-08-31T06:22:15.023 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:14 smithi184 ceph-mon[22331]: pgmap v833: 97 pgs: 3 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 90 active+clean; 3.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 14 MiB/s wr, 1.35k op/s 2024-08-31T06:22:15.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:14 smithi155 ceph-mon[21742]: pgmap v833: 97 pgs: 3 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 90 active+clean; 3.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 14 MiB/s wr, 1.35k op/s 2024-08-31T06:22:15.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:14 smithi016 ceph-mon[17194]: pgmap v833: 97 pgs: 3 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 90 active+clean; 3.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 14 MiB/s wr, 1.35k op/s 2024-08-31T06:22:16.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 890.0 s, 1279.2 tps, lat 77.190 ms stddev 101.738 2024-08-31T06:22:17.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:16 smithi184 ceph-mon[22331]: pgmap v834: 97 pgs: 97 active+clean; 3.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.2 MiB/s wr, 645 op/s 2024-08-31T06:22:17.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:16 smithi184 ceph-mon[22331]: osdmap e101: 12 total, 12 up, 12 in 2024-08-31T06:22:17.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:16 smithi155 ceph-mon[21742]: pgmap v834: 97 pgs: 97 active+clean; 3.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.2 MiB/s wr, 645 op/s 2024-08-31T06:22:17.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:16 smithi155 ceph-mon[21742]: osdmap e101: 12 total, 12 up, 12 in 2024-08-31T06:22:17.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:16 smithi016 ceph-mon[17194]: pgmap v834: 97 pgs: 97 active+clean; 3.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.2 MiB/s wr, 645 op/s 2024-08-31T06:22:17.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:16 smithi016 ceph-mon[17194]: osdmap e101: 12 total, 12 up, 12 in 2024-08-31T06:22:17.737 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T06:22:18.166 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:17 smithi016 ceph-mon[17194]: pgmap v836: 97 pgs: 97 active+clean; 3.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.2 MiB/s wr, 679 op/s 2024-08-31T06:22:18.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:17 smithi184 ceph-mon[22331]: pgmap v836: 97 pgs: 97 active+clean; 3.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.2 MiB/s wr, 679 op/s 2024-08-31T06:22:18.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:17 smithi155 ceph-mon[21742]: pgmap v836: 97 pgs: 97 active+clean; 3.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.2 MiB/s wr, 679 op/s 2024-08-31T06:22:18.376 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:22:19.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.569+0000 7f7003e00640 1 -- 172.21.15.16:0/2117164368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7004103750 msgr2=0x7f7004103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:19.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.569+0000 7f7003e00640 1 --2- 172.21.15.16:0/2117164368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7004103750 0x7f7004103b60 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4009900 tx=0x7f6ff402a3f0 comp rx=0 tx=0).stop 2024-08-31T06:22:19.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.569+0000 7f7003e00640 1 -- 172.21.15.16:0/2117164368 shutdown_connections 2024-08-31T06:22:19.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.569+0000 7f7003e00640 1 --2- 172.21.15.16:0/2117164368 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7004109090 0x7f70041094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:19.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.569+0000 7f7003e00640 1 --2- 172.21.15.16:0/2117164368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70041099e0 0x7f7004110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:19.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.569+0000 7f7003e00640 1 --2- 172.21.15.16:0/2117164368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7004103750 0x7f7004103b60 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:19.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.569+0000 7f7003e00640 1 -- 172.21.15.16:0/2117164368 >> 172.21.15.16:0/2117164368 conn(0x7f70040ff1e0 msgr2=0x7f7004101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:19.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.569+0000 7f7003e00640 1 -- 172.21.15.16:0/2117164368 shutdown_connections 2024-08-31T06:22:19.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.569+0000 7f7003e00640 1 -- 172.21.15.16:0/2117164368 wait complete. 2024-08-31T06:22:19.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.570+0000 7f7003e00640 1 Processor -- start 2024-08-31T06:22:19.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.570+0000 7f7003e00640 1 -- start start 2024-08-31T06:22:19.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.571+0000 7f7003e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7004103750 0x7f70040754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:19.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.571+0000 7f7003e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7004109090 0x7f70040759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:19.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.571+0000 7f7003e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70041099e0 0x7f7004075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:19.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.571+0000 7f7003e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f70041136e0 con 0x7f7004109090 2024-08-31T06:22:19.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.571+0000 7f7003e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7004113560 con 0x7f7004103750 2024-08-31T06:22:19.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.571+0000 7f7003e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7004113860 con 0x7f70041099e0 2024-08-31T06:22:19.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.571+0000 7f7002a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7004103750 0x7f70040754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:19.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.571+0000 7f7002a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7004103750 0x7f70040754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:37206/0 (socket says 172.21.15.16:37206) 2024-08-31T06:22:19.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.571+0000 7f7002a00640 1 -- 172.21.15.16:0/2385479279 learned_addr learned my addr 172.21.15.16:0/2385479279 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:19.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.572+0000 7f7002000640 1 --2- 172.21.15.16:0/2385479279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7004109090 0x7f70040759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:19.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.572+0000 7f7003400640 1 --2- 172.21.15.16:0/2385479279 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70041099e0 0x7f7004075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:19.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.572+0000 7f7002a00640 1 -- 172.21.15.16:0/2385479279 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70041099e0 msgr2=0x7f7004075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:19.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.572+0000 7f7002a00640 1 --2- 172.21.15.16:0/2385479279 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70041099e0 0x7f7004075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:19.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.572+0000 7f7002a00640 1 -- 172.21.15.16:0/2385479279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7004109090 msgr2=0x7f70040759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:19.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.572+0000 7f7002a00640 1 --2- 172.21.15.16:0/2385479279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7004109090 0x7f70040759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:19.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.572+0000 7f7002a00640 1 -- 172.21.15.16:0/2385479279 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70041a8350 con 0x7f7004103750 2024-08-31T06:22:19.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.573+0000 7f7002a00640 1 --2- 172.21.15.16:0/2385479279 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7004103750 0x7f70040754b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4031440 tx=0x7f6ff4003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:19.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.573+0000 7f6ffb400640 1 -- 172.21.15.16:0/2385479279 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff403f070 con 0x7f7004103750 2024-08-31T06:22:19.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.573+0000 7f6ffb400640 1 -- 172.21.15.16:0/2385479279 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ff4031b50 con 0x7f7004103750 2024-08-31T06:22:19.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.573+0000 7f6ffb400640 1 -- 172.21.15.16:0/2385479279 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff4035030 con 0x7f7004103750 2024-08-31T06:22:19.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.573+0000 7f7003e00640 1 -- 172.21.15.16:0/2385479279 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70041a85e0 con 0x7f7004103750 2024-08-31T06:22:19.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.574+0000 7f7003e00640 1 -- 172.21.15.16:0/2385479279 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70041a8a70 con 0x7f7004103750 2024-08-31T06:22:19.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.581+0000 7f6ffb400640 1 -- 172.21.15.16:0/2385479279 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f6ff4042070 con 0x7f7004103750 2024-08-31T06:22:19.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.583+0000 7f7010675740 1 Processor -- start 2024-08-31T06:22:19.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.584+0000 7f6ffb400640 1 --2- 172.21.15.16:0/2385479279 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6fe80b4dd0 0x7f6fe80b72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:19.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.584+0000 7f6ffb400640 1 -- 172.21.15.16:0/2385479279 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8470+0+0 (secure 0 0 0) 0x7f6ff4105820 con 0x7f7004103750 2024-08-31T06:22:19.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.585+0000 7f7003400640 1 --2- 172.21.15.16:0/2385479279 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6fe80b4dd0 0x7f6fe80b72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:19.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.586+0000 7f7010675740 1 -- start start 2024-08-31T06:22:19.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.587+0000 7f7003400640 1 --2- 172.21.15.16:0/2385479279 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6fe80b4dd0 0x7f6fe80b72e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc005fd0 tx=0x7f6ffc009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:19.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.587+0000 7f7010675740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ac80e070 0x5623acbb3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:19.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.587+0000 7f7010675740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5623ac7e51b0 0x5623acbb4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:19.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.587+0000 7f7010675740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623ac7ed390 0x5623acbb45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:19.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.587+0000 7f7010675740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5623acb2a770 con 0x5623ac80e070 2024-08-31T06:22:19.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.587+0000 7f7010675740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5623acb19d00 con 0x5623ac7e51b0 2024-08-31T06:22:19.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.587+0000 7f7010675740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5623acb1dcb0 con 0x5623ac7ed390 2024-08-31T06:22:19.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.588+0000 7f7003400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623ac7ed390 0x5623acbb45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:19.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.588+0000 7f7003400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623ac7ed390 0x5623acbb45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:45716/0 (socket says 172.21.15.16:45716) 2024-08-31T06:22:19.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.588+0000 7f7003400640 1 -- 172.21.15.16:0/587730590 learned_addr learned my addr 172.21.15.16:0/587730590 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:19.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.588+0000 7f7002a00640 1 --2- 172.21.15.16:0/587730590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ac80e070 0x5623acbb3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:19.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.588+0000 7f7002000640 1 --2- 172.21.15.16:0/587730590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5623ac7e51b0 0x5623acbb4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:19.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.589+0000 7f7002a00640 1 -- 172.21.15.16:0/587730590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623ac7ed390 msgr2=0x5623acbb45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:19.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.589+0000 7f7002a00640 1 --2- 172.21.15.16:0/587730590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623ac7ed390 0x5623acbb45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:19.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.589+0000 7f7002a00640 1 -- 172.21.15.16:0/587730590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5623ac7e51b0 msgr2=0x5623acbb4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:19.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.589+0000 7f7002a00640 1 --2- 172.21.15.16:0/587730590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5623ac7e51b0 0x5623acbb4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:19.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.589+0000 7f7002a00640 1 -- 172.21.15.16:0/587730590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623ac879650 con 0x5623ac80e070 2024-08-31T06:22:19.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.591+0000 7f7002a00640 1 --2- 172.21.15.16:0/587730590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ac80e070 0x5623acbb3b20 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4102f40 tx=0x7f6ff402a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd97fb55939292f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:19.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.592+0000 7f6fefe00640 1 -- 172.21.15.16:0/587730590 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff4049040 con 0x5623ac80e070 2024-08-31T06:22:19.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.592+0000 7f6fefe00640 1 -- 172.21.15.16:0/587730590 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ff40d5200 con 0x5623ac80e070 2024-08-31T06:22:19.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.592+0000 7f6fefe00640 1 -- 172.21.15.16:0/587730590 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff40d4d70 con 0x5623ac80e070 2024-08-31T06:22:19.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.594+0000 7f7010675740 1 -- 172.21.15.16:0/587730590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ac80e070 msgr2=0x5623acbb3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:19.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.594+0000 7f7010675740 1 --2- 172.21.15.16:0/587730590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ac80e070 0x5623acbb3b20 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4102f40 tx=0x7f6ff402a8f0 comp rx=0 tx=0).stop 2024-08-31T06:22:19.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.594+0000 7f7010675740 1 -- 172.21.15.16:0/587730590 shutdown_connections 2024-08-31T06:22:19.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.594+0000 7f7010675740 1 --2- 172.21.15.16:0/587730590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623ac7ed390 0x5623acbb45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:19.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.594+0000 7f7010675740 1 --2- 172.21.15.16:0/587730590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5623ac7e51b0 0x5623acbb4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:19.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.594+0000 7f7010675740 1 --2- 172.21.15.16:0/587730590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ac80e070 0x5623acbb3b20 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:19.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.594+0000 7f7010675740 1 -- 172.21.15.16:0/587730590 >> 172.21.15.16:0/587730590 conn(0x5623ac802460 msgr2=0x5623acbb1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:19.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.594+0000 7f7010675740 1 -- 172.21.15.16:0/587730590 shutdown_connections 2024-08-31T06:22:19.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.594+0000 7f7010675740 1 -- 172.21.15.16:0/587730590 wait complete. 2024-08-31T06:22:19.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.595+0000 7f7010675740 1 -- start start 2024-08-31T06:22:19.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.596+0000 7f7010675740 1 Processor -- start 2024-08-31T06:22:19.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.597+0000 7f7010675740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5623ac7e51b0 0x5623acbbf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:19.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.597+0000 7f7010675740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623ac7ed390 0x5623acbbfc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:19.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.597+0000 7f7010675740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ac80e070 0x5623acbd3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:19.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.597+0000 7f7010675740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5623ac97c700 con 0x5623ac80e070 2024-08-31T06:22:19.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.597+0000 7f7010675740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5623acaa0cb0 con 0x5623ac7e51b0 2024-08-31T06:22:19.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.597+0000 7f7010675740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5623acbc20c0 con 0x5623ac7ed390 2024-08-31T06:22:19.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.597+0000 7f7010675740 10 client.0 _ll_register_callbacks cb 0x5623acac6430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:22:19.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.598+0000 7f7002a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5623ac7e51b0 0x5623acbbf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:19.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.598+0000 7f7003400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ac80e070 0x5623acbd3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:19.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.598+0000 7f7002a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5623ac7e51b0 0x5623acbbf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:37220/0 (socket says 172.21.15.16:37220) 2024-08-31T06:22:19.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.598+0000 7f7002a00640 1 -- 172.21.15.16:0/3112244803 learned_addr learned my addr 172.21.15.16:0/3112244803 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:19.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.598+0000 7f7002000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623ac7ed390 0x5623acbbfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:19.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.599+0000 7f7002000640 1 -- 172.21.15.16:0/3112244803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5623ac7e51b0 msgr2=0x5623acbbf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:19.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.599+0000 7f7002000640 1 --2- 172.21.15.16:0/3112244803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5623ac7e51b0 0x5623acbbf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:19.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.599+0000 7f7002000640 1 -- 172.21.15.16:0/3112244803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ac80e070 msgr2=0x5623acbd3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:19.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.599+0000 7f7002000640 1 --2- 172.21.15.16:0/3112244803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ac80e070 0x5623acbd3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:19.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.599+0000 7f7002000640 1 -- 172.21.15.16:0/3112244803 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623acae1870 con 0x5623ac7ed390 2024-08-31T06:22:19.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.600+0000 7f7002000640 1 --2- 172.21.15.16:0/3112244803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623ac7ed390 0x5623acbbfc50 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f7004108a60 tx=0x7f6ff0012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef5e5258f56b2065 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:19.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.600+0000 7f6fecc00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:22:19.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.600+0000 7f6fecc00640 1 -- 172.21.15.16:0/3112244803 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff0004030 con 0x5623ac7ed390 2024-08-31T06:22:19.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.600+0000 7f6fecc00640 1 -- 172.21.15.16:0/3112244803 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8470+0+0 (secure 0 0 0) 0x7f6ff00041d0 con 0x5623ac7ed390 2024-08-31T06:22:19.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.600+0000 7f7010675740 1 -- 172.21.15.16:0/3112244803 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ff40d5200 con 0x5623ac7ed390 2024-08-31T06:22:19.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.601+0000 7f6fecc00640 1 -- 172.21.15.16:0/3112244803 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f6ff00043d0 con 0x5623ac7ed390 2024-08-31T06:22:19.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.601+0000 7f7010675740 10 client.28936 fetch_fsmap learned FSMap version 52 2024-08-31T06:22:19.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.601+0000 7f7010675740 1 -- 172.21.15.16:0/3112244803 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5623acbd6950 con 0x5623ac7ed390 2024-08-31T06:22:19.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.602+0000 7f6fecc00640 1 -- 172.21.15.16:0/3112244803 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ff40d5200 con 0x5623ac7ed390 2024-08-31T06:22:19.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.603+0000 7f6fecc00640 1 -- 172.21.15.16:0/3112244803 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 52) ==== 8350+0+0 (secure 0 0 0) 0x7f6ff0003aa0 con 0x5623ac7ed390 2024-08-31T06:22:19.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.603+0000 7f7010675740 10 client.28936 fetch_fsmap finished waiting for FSMap version 52 2024-08-31T06:22:19.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.603+0000 7f7010675740 10 client.28936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:22:19.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.603+0000 7f7010675740 20 client.28936 populate_metadata read hostname 'smithi016' 2024-08-31T06:22:19.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.603+0000 7f7010675740 1 --2- 172.21.15.16:0/3112244803 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5623ac7cc9f0 0x5623acbd9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:19.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.603+0000 7f7010675740 4 client.28936 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:22:19.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.603+0000 7f7010675740 1 -- 172.21.15.16:0/3112244803 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623acb2c6c0 con 0x5623ac7cc9f0 2024-08-31T06:22:19.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.606+0000 7f7002a00640 1 --2- 172.21.15.16:0/3112244803 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5623ac7cc9f0 0x5623acbd9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:19.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.607+0000 7f7002a00640 1 --2- 172.21.15.16:0/3112244803 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5623ac7cc9f0 0x5623acbd9e90 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc0013a5e217749c server_cookie=d41d97e17549b5c0 in_seq=0 out_seq=0 2024-08-31T06:22:19.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.607+0000 7f6fecc00640 10 client.28936 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:22:19.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.608+0000 7f6fecc00640 1 -- 172.21.15.16:0/3112244803 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5623acb2c6c0 con 0x5623ac7cc9f0 2024-08-31T06:22:19.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.608+0000 7f6fecc00640 10 client.28936 handle_command_reply: tid=0 2024-08-31T06:22:19.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.608+0000 7f7010675740 1 client.28936 shutdown 2024-08-31T06:22:19.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.609+0000 7f7010675740 1 -- 172.21.15.16:0/3112244803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623ac7ed390 msgr2=0x5623acbbfc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:19.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.609+0000 7f7010675740 1 --2- 172.21.15.16:0/3112244803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623ac7ed390 0x5623acbbfc50 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f7004108a60 tx=0x7f6ff0012b60 comp rx=0 tx=0).stop 2024-08-31T06:22:19.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.609+0000 7f7010675740 1 -- 172.21.15.16:0/3112244803 shutdown_connections 2024-08-31T06:22:19.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.609+0000 7f7010675740 1 --2- 172.21.15.16:0/3112244803 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5623ac7cc9f0 0x5623acbd9e90 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:19.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.609+0000 7f7010675740 1 --2- 172.21.15.16:0/3112244803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623ac7ed390 0x5623acbbfc50 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:19.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.609+0000 7f7010675740 1 --2- 172.21.15.16:0/3112244803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ac80e070 0x5623acbd3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:19.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.609+0000 7f7010675740 1 --2- 172.21.15.16:0/3112244803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5623ac7e51b0 0x5623acbbf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:19.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.609+0000 7f7010675740 1 -- 172.21.15.16:0/3112244803 >> 172.21.15.16:0/3112244803 conn(0x5623ac802460 msgr2=0x5623acbb6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:19.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.609+0000 7f6fecc00640 0 client.28936 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:22:19.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.609+0000 7f7010675740 1 -- 172.21.15.16:0/3112244803 shutdown_connections 2024-08-31T06:22:19.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.610+0000 7f7010675740 1 -- 172.21.15.16:0/3112244803 wait complete. 2024-08-31T06:22:19.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.610+0000 7f7010675740 20 client.28936 trim_cache size 0 max 16384 2024-08-31T06:22:19.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.646+0000 7f7010675740 1 Processor -- start 2024-08-31T06:22:19.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.646+0000 7f7010675740 1 -- start start 2024-08-31T06:22:19.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.646+0000 7f7010675740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623ac7e51b0 0x5623acbbf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:19.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.646+0000 7f7010675740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ac7ed390 0x5623acbb5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:19.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.646+0000 7f7010675740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5623ac80e070 0x5623acbb0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:19.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.646+0000 7f7010675740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f70041af940 con 0x5623ac7ed390 2024-08-31T06:22:19.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.646+0000 7f7010675740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f70041ad1e0 con 0x5623ac80e070 2024-08-31T06:22:19.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.646+0000 7f7010675740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f70041ab050 con 0x5623ac7e51b0 2024-08-31T06:22:19.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.647+0000 7f7002a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623ac7e51b0 0x5623acbbf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:19.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.647+0000 7f7002a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623ac7e51b0 0x5623acbbf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:45734/0 (socket says 172.21.15.16:45734) 2024-08-31T06:22:19.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.647+0000 7f7002a00640 1 -- 172.21.15.16:0/3836593112 learned_addr learned my addr 172.21.15.16:0/3836593112 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:19.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.647+0000 7f7003400640 1 --2- 172.21.15.16:0/3836593112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5623ac80e070 0x5623acbb0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:19.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.648+0000 7f7002000640 1 --2- 172.21.15.16:0/3836593112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ac7ed390 0x5623acbb5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:19.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.648+0000 7f7003400640 1 -- 172.21.15.16:0/3836593112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623ac7e51b0 msgr2=0x5623acbbf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:19.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.648+0000 7f7003400640 1 --2- 172.21.15.16:0/3836593112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623ac7e51b0 0x5623acbbf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:19.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.648+0000 7f7003400640 1 -- 172.21.15.16:0/3836593112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ac7ed390 msgr2=0x5623acbb5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:19.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.648+0000 7f7003400640 1 --2- 172.21.15.16:0/3836593112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ac7ed390 0x5623acbb5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:19.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.648+0000 7f7003400640 1 -- 172.21.15.16:0/3836593112 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623acbbe470 con 0x5623ac80e070 2024-08-31T06:22:19.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.648+0000 7f7003400640 1 --2- 172.21.15.16:0/3836593112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5623ac80e070 0x5623acbb0fc0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc011ca0 tx=0x7f6ffc014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=937bfa0c92cd0ff0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:19.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.649+0000 7f6fef400640 1 -- 172.21.15.16:0/3836593112 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc004b20 con 0x5623ac80e070 2024-08-31T06:22:19.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.649+0000 7f6fef400640 1 -- 172.21.15.16:0/3836593112 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ffc003e20 con 0x5623ac80e070 2024-08-31T06:22:19.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.649+0000 7f6fef400640 1 -- 172.21.15.16:0/3836593112 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc01fd60 con 0x5623ac80e070 2024-08-31T06:22:19.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.651+0000 7f7010675740 1 -- 172.21.15.16:0/3836593112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5623ac80e070 msgr2=0x5623acbb0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:19.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.651+0000 7f7010675740 1 --2- 172.21.15.16:0/3836593112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5623ac80e070 0x5623acbb0fc0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc011ca0 tx=0x7f6ffc014000 comp rx=0 tx=0).stop 2024-08-31T06:22:19.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.651+0000 7f7010675740 1 -- 172.21.15.16:0/3836593112 shutdown_connections 2024-08-31T06:22:19.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.651+0000 7f7010675740 1 --2- 172.21.15.16:0/3836593112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5623ac80e070 0x5623acbb0fc0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:19.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.651+0000 7f7010675740 1 --2- 172.21.15.16:0/3836593112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ac7ed390 0x5623acbb5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:19.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.651+0000 7f7010675740 1 --2- 172.21.15.16:0/3836593112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623ac7e51b0 0x5623acbbf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:19.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.651+0000 7f7010675740 1 -- 172.21.15.16:0/3836593112 >> 172.21.15.16:0/3836593112 conn(0x5623ac802460 msgr2=0x5623acbb4b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:19.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.651+0000 7f7010675740 1 -- 172.21.15.16:0/3836593112 shutdown_connections 2024-08-31T06:22:19.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.651+0000 7f7010675740 1 -- 172.21.15.16:0/3836593112 wait complete. 2024-08-31T06:22:19.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.652+0000 7f7010675740 1 -- start start 2024-08-31T06:22:19.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.652+0000 7f7010675740 1 Processor -- start 2024-08-31T06:22:19.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.653+0000 7f7010675740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5623ac7e51b0 0x5623acbd04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:19.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.653+0000 7f7010675740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ac7ed390 0x5623acbcc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:19.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.653+0000 7f7010675740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623ac80e070 0x5623acbcd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:19.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.653+0000 7f7010675740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f70041ab6a0 con 0x5623ac7ed390 2024-08-31T06:22:19.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.653+0000 7f7010675740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f70041acb10 con 0x5623ac7e51b0 2024-08-31T06:22:19.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.653+0000 7f7010675740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f70041abd70 con 0x5623ac80e070 2024-08-31T06:22:19.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.653+0000 7f7010675740 10 client.0 _ll_register_callbacks cb 0x7f6ff0009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:22:19.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.654+0000 7f7002000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ac7ed390 0x5623acbcc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:19.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.654+0000 7f7003400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623ac80e070 0x5623acbcd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:19.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.654+0000 7f7002a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5623ac7e51b0 0x5623acbd04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:19.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.654+0000 7f7003400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623ac80e070 0x5623acbcd020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:45746/0 (socket says 172.21.15.16:45746) 2024-08-31T06:22:19.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.654+0000 7f7003400640 1 -- 172.21.15.16:0/3806344175 learned_addr learned my addr 172.21.15.16:0/3806344175 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:19.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.655+0000 7f7003400640 1 -- 172.21.15.16:0/3806344175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5623ac7e51b0 msgr2=0x5623acbd04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:19.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.655+0000 7f7003400640 1 --2- 172.21.15.16:0/3806344175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5623ac7e51b0 0x5623acbd04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:19.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.655+0000 7f7003400640 1 -- 172.21.15.16:0/3806344175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ac7ed390 msgr2=0x5623acbcc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:19.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.655+0000 7f7003400640 1 --2- 172.21.15.16:0/3806344175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ac7ed390 0x5623acbcc980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:19.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.655+0000 7f7003400640 1 -- 172.21.15.16:0/3806344175 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623acbcd790 con 0x5623ac80e070 2024-08-31T06:22:19.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.657+0000 7f7003400640 1 --2- 172.21.15.16:0/3806344175 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623ac80e070 0x5623acbcd020 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc00e4c0 tx=0x7f6ffc004c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8780112db4024f7 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:19.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.657+0000 7f6fecc00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:22:19.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.657+0000 7f6fecc00640 1 -- 172.21.15.16:0/3806344175 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc0041e0 con 0x5623ac80e070 2024-08-31T06:22:19.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.657+0000 7f7010675740 1 -- 172.21.15.16:0/3806344175 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ffc003e20 con 0x5623ac80e070 2024-08-31T06:22:19.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.658+0000 7f6fecc00640 1 -- 172.21.15.16:0/3806344175 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8470+0+0 (secure 0 0 0) 0x7f6ffc025040 con 0x5623ac80e070 2024-08-31T06:22:19.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.658+0000 7f6fecc00640 1 -- 172.21.15.16:0/3806344175 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f6ffc03f050 con 0x5623ac80e070 2024-08-31T06:22:19.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.661+0000 7f7010675740 10 client.28942 fetch_fsmap learned FSMap version 52 2024-08-31T06:22:19.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.661+0000 7f7010675740 1 -- 172.21.15.16:0/3806344175 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5623acc45930 con 0x5623ac80e070 2024-08-31T06:22:19.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.662+0000 7f6fecc00640 1 -- 172.21.15.16:0/3806344175 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ffc003e20 con 0x5623ac80e070 2024-08-31T06:22:19.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.662+0000 7f6fecc00640 1 -- 172.21.15.16:0/3806344175 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 52) ==== 8350+0+0 (secure 0 0 0) 0x7f6ffc01fce0 con 0x5623ac80e070 2024-08-31T06:22:19.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.662+0000 7f7010675740 10 client.28942 fetch_fsmap finished waiting for FSMap version 52 2024-08-31T06:22:19.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.662+0000 7f7010675740 10 client.28942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:22:19.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.662+0000 7f7010675740 20 client.28942 populate_metadata read hostname 'smithi016' 2024-08-31T06:22:19.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.662+0000 7f7010675740 1 --2- 172.21.15.16:0/3806344175 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5623ac7a1da0 0x5623acc480a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:19.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.662+0000 7f7010675740 4 client.28942 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T06:22:19.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.662+0000 7f7010675740 1 -- 172.21.15.16:0/3806344175 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5623acb28160 con 0x5623ac7a1da0 2024-08-31T06:22:19.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.662+0000 7f7002a00640 1 --2- 172.21.15.16:0/3806344175 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5623ac7a1da0 0x5623acc480a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:19.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.663+0000 7f6fecc00640 10 client.28942 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:22:19.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.663+0000 7f7002a00640 1 --2- 172.21.15.16:0/3806344175 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5623ac7a1da0 0x5623acc480a0 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a001ebffbc54a7ac server_cookie=9530b30a8bb4851c in_seq=0 out_seq=0 2024-08-31T06:22:19.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.667+0000 7f6fecc00640 1 -- 172.21.15.16:0/3806344175 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5202 (crc 0 0 0) 0x5623acb28160 con 0x5623ac7a1da0 2024-08-31T06:22:19.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.667+0000 7f6fecc00640 10 client.28942 handle_command_reply: tid=0 2024-08-31T06:22:19.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.667+0000 7f7010675740 1 client.28942 shutdown 2024-08-31T06:22:19.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.667+0000 7f7010675740 1 -- 172.21.15.16:0/3806344175 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623ac80e070 msgr2=0x5623acbcd020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:19.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.667+0000 7f7010675740 1 --2- 172.21.15.16:0/3806344175 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623ac80e070 0x5623acbcd020 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc00e4c0 tx=0x7f6ffc004c70 comp rx=0 tx=0).stop 2024-08-31T06:22:19.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.667+0000 7f7010675740 1 -- 172.21.15.16:0/3806344175 shutdown_connections 2024-08-31T06:22:19.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.667+0000 7f7010675740 1 --2- 172.21.15.16:0/3806344175 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5623ac7a1da0 0x5623acc480a0 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:19.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.667+0000 7f7010675740 1 --2- 172.21.15.16:0/3806344175 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623ac80e070 0x5623acbcd020 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:19.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.667+0000 7f7010675740 1 --2- 172.21.15.16:0/3806344175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ac7ed390 0x5623acbcc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:19.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.667+0000 7f7010675740 1 --2- 172.21.15.16:0/3806344175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5623ac7e51b0 0x5623acbd04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:19.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.667+0000 7f7010675740 1 -- 172.21.15.16:0/3806344175 >> 172.21.15.16:0/3806344175 conn(0x5623ac802460 msgr2=0x5623acbb4b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:19.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.668+0000 7f6fecc00640 0 client.28942 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:22:19.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.668+0000 7f7010675740 1 -- 172.21.15.16:0/3806344175 shutdown_connections 2024-08-31T06:22:19.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.668+0000 7f7010675740 1 -- 172.21.15.16:0/3806344175 wait complete. 2024-08-31T06:22:19.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.669+0000 7f7010675740 20 client.28942 trim_cache size 0 max 16384 2024-08-31T06:22:19.676 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:22:19.676 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-08-31T06:22:19.676 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {}, 2024-08-31T06:22:19.676 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-08-31T06:22:19.676 INFO:teuthology.orchestra.run.smithi016.stdout: "18e112f4-a5f9-4602-aaad-3b13fa5a4c2e": { 2024-08-31T06:22:19.676 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:20:16.570535+0000", 2024-08-31T06:22:19.676 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:22:19.676 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:22:19.676 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:22:19.676 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:22:19.677 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:22:19.677 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:22:19.677 INFO:teuthology.orchestra.run.smithi016.stdout: "f1cdcb27-e3e4-4dc8-a0ae-7ace761d04c5": { 2024-08-31T06:22:19.677 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:19:22.275683+0000", 2024-08-31T06:22:19.677 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:22:19.677 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:22:19.677 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:22:19.677 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:22:19.677 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:22:19.677 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:22:19.677 INFO:teuthology.orchestra.run.smithi016.stdout: "0e44eb18-439c-46e4-a298-7d4d2ecd0b4a": { 2024-08-31T06:22:19.677 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:18:31.130765+0000", 2024-08-31T06:22:19.677 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:22:19.678 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:22:19.678 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:22:19.678 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:22:19.678 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:22:19.678 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:22:19.678 INFO:teuthology.orchestra.run.smithi016.stdout: "e497cd88-1e7c-4631-a4a8-cdd8e1fa6c83": { 2024-08-31T06:22:19.678 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:17:42.476795+0000", 2024-08-31T06:22:19.678 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:22:19.678 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:22:19.678 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:22:19.678 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:22:19.678 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:22:19.678 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:22:19.678 INFO:teuthology.orchestra.run.smithi016.stdout: "c300b7e8-f3fd-482c-a909-9329bc61e570": { 2024-08-31T06:22:19.679 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:06:49.868046+0000", 2024-08-31T06:22:19.679 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:22:19.679 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:22:19.679 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:22:19.679 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:22:19.679 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:22:19.679 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:22:19.679 INFO:teuthology.orchestra.run.smithi016.stdout: "6a03046e-cdf4-4a14-b9bd-b560b8968467": { 2024-08-31T06:22:19.679 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:12:38.002354+0000", 2024-08-31T06:22:19.679 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:22:19.679 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:22:19.679 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:22:19.679 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:22:19.679 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:22:19.680 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:22:19.680 INFO:teuthology.orchestra.run.smithi016.stdout: "004b47bb-0fe8-4d28-9d9d-78bde6a61199": { 2024-08-31T06:22:19.680 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:16:48.915926+0000", 2024-08-31T06:22:19.680 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:22:19.680 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:22:19.680 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:22:19.680 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:22:19.680 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:22:19.680 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:22:19.680 INFO:teuthology.orchestra.run.smithi016.stdout: "88d7bdb8-cb1c-45e5-ad29-4cd8da6c6003": { 2024-08-31T06:22:19.680 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:07:39.648942+0000", 2024-08-31T06:22:19.680 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:22:19.680 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:22:19.681 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:22:19.681 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:22:19.681 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:22:19.681 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:22:19.681 INFO:teuthology.orchestra.run.smithi016.stdout: "0fdcd762-533d-42be-9289-b71c8e3b81b1": { 2024-08-31T06:22:19.681 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:15:06.169112+0000", 2024-08-31T06:22:19.681 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:22:19.681 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:22:19.681 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:22:19.681 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:22:19.681 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:22:19.681 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:22:19.681 INFO:teuthology.orchestra.run.smithi016.stdout: "0015da36-6634-47c5-b8d4-219f9eba8250": { 2024-08-31T06:22:19.681 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:21:11.445380+0000", 2024-08-31T06:22:19.681 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:22:19.682 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:22:19.682 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:22:19.682 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:22:19.682 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:22:19.682 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:22:19.682 INFO:teuthology.orchestra.run.smithi016.stdout: "c8455d74-2fe7-4d93-bcf2-19b419d16016": { 2024-08-31T06:22:19.682 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:08:24.035960+0000", 2024-08-31T06:22:19.682 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:22:19.682 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:22:19.682 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:22:19.682 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:22:19.682 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:22:19.682 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:22:19.682 INFO:teuthology.orchestra.run.smithi016.stdout: "78d6ef02-8551-45e8-ad60-9627eb573fc1": { 2024-08-31T06:22:19.683 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:09:13.547865+0000", 2024-08-31T06:22:19.683 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:22:19.683 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:22:19.683 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:22:19.683 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:22:19.683 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:22:19.683 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:22:19.683 INFO:teuthology.orchestra.run.smithi016.stdout: "c1fd6904-2e02-47fa-939a-47349b9fe36b": { 2024-08-31T06:22:19.683 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:06.678691+0000", 2024-08-31T06:22:19.683 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:22:19.683 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:22:19.683 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:22:19.683 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:22:19.683 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:22:19.684 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:22:19.684 INFO:teuthology.orchestra.run.smithi016.stdout: "a10cce76-99ab-46b5-b3b2-07b82899b8f9": { 2024-08-31T06:22:19.684 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:11:43.967546+0000", 2024-08-31T06:22:19.684 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:22:19.684 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:22:19.684 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:22:19.684 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:22:19.684 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:22:19.684 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:22:19.684 INFO:teuthology.orchestra.run.smithi016.stdout: "91b72a34-4ba9-4126-a414-8b862f8c5f78": { 2024-08-31T06:22:19.684 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:13:28.663237+0000", 2024-08-31T06:22:19.684 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:22:19.684 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:22:19.684 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:22:19.685 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:22:19.685 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:22:19.685 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:22:19.685 INFO:teuthology.orchestra.run.smithi016.stdout: "e739a16e-8ace-43a8-a1ce-ac27d39ef8a3": { 2024-08-31T06:22:19.685 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:54.084646+0000", 2024-08-31T06:22:19.685 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:22:19.685 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:22:19.685 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:22:19.685 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:22:19.685 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:22:19.685 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:22:19.685 INFO:teuthology.orchestra.run.smithi016.stdout: "a64d5301-8558-4e9c-ac9b-8d0ac7057e74": { 2024-08-31T06:22:19.685 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:14:20.338104+0000", 2024-08-31T06:22:19.685 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:22:19.686 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:22:19.686 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:22:19.686 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:22:19.686 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:22:19.686 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:22:19.686 INFO:teuthology.orchestra.run.smithi016.stdout: "f9c4a22e-0c8a-4928-9b0d-e2ce1db5a86b": { 2024-08-31T06:22:19.686 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:15:57.496015+0000", 2024-08-31T06:22:19.686 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:22:19.686 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:22:19.686 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:22:19.686 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:22:19.686 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:22:19.686 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:22:19.686 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:22:19.686 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:22:19.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.671+0000 7f7003e00640 1 -- 172.21.15.16:0/2385479279 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6fe80b4dd0 msgr2=0x7f6fe80b72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:19.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.671+0000 7f7003e00640 1 --2- 172.21.15.16:0/2385479279 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6fe80b4dd0 0x7f6fe80b72e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc005fd0 tx=0x7f6ffc009500 comp rx=0 tx=0).stop 2024-08-31T06:22:19.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.671+0000 7f7003e00640 1 -- 172.21.15.16:0/2385479279 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7004103750 msgr2=0x7f70040754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:19.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.671+0000 7f7003e00640 1 --2- 172.21.15.16:0/2385479279 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7004103750 0x7f70040754b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4031440 tx=0x7f6ff4003b90 comp rx=0 tx=0).stop 2024-08-31T06:22:19.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.671+0000 7f7003e00640 1 -- 172.21.15.16:0/2385479279 shutdown_connections 2024-08-31T06:22:19.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.671+0000 7f7003e00640 1 --2- 172.21.15.16:0/2385479279 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f6fe80b4dd0 0x7f6fe80b72e0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:19.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.671+0000 7f7003e00640 1 --2- 172.21.15.16:0/2385479279 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70041099e0 0x7f7004075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:19.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.671+0000 7f7003e00640 1 --2- 172.21.15.16:0/2385479279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7004109090 0x7f70040759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:19.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.671+0000 7f7003e00640 1 --2- 172.21.15.16:0/2385479279 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7004103750 0x7f70040754b0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:19.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.671+0000 7f7003e00640 1 -- 172.21.15.16:0/2385479279 >> 172.21.15.16:0/2385479279 conn(0x7f70040ff1e0 msgr2=0x7f7004101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:19.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.673+0000 7f7003e00640 1 -- 172.21.15.16:0/2385479279 shutdown_connections 2024-08-31T06:22:19.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:19.673+0000 7f7003e00640 1 -- 172.21.15.16:0/2385479279 wait complete. 2024-08-31T06:22:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.069+0000 7fc38f600640 1 -- 172.21.15.16:0/765579219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc38810bd10 msgr2=0x7fc388112540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.069+0000 7fc38f600640 1 --2- 172.21.15.16:0/765579219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc38810bd10 0x7fc388112540 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fc380009ca0 tx=0x7fc38002a8c0 comp rx=0 tx=0).stop 2024-08-31T06:22:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.069+0000 7fc38f600640 1 -- 172.21.15.16:0/765579219 shutdown_connections 2024-08-31T06:22:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.069+0000 7fc38f600640 1 --2- 172.21.15.16:0/765579219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc38810bd10 0x7fc388112540 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.069+0000 7fc38f600640 1 --2- 172.21.15.16:0/765579219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc38810b3c0 0x7fc38810b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.069+0000 7fc38f600640 1 --2- 172.21.15.16:0/765579219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc388105a90 0x7fc388105ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.069+0000 7fc38f600640 1 -- 172.21.15.16:0/765579219 >> 172.21.15.16:0/765579219 conn(0x7fc388101490 msgr2=0x7fc3881038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.069+0000 7fc38f600640 1 -- 172.21.15.16:0/765579219 shutdown_connections 2024-08-31T06:22:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.069+0000 7fc38f600640 1 -- 172.21.15.16:0/765579219 wait complete. 2024-08-31T06:22:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.074+0000 7fc38f600640 1 Processor -- start 2024-08-31T06:22:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.075+0000 7fc38f600640 1 -- start start 2024-08-31T06:22:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.075+0000 7fc38f600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc388105a90 0x7fc38807b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.075+0000 7fc38f600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc38810b3c0 0x7fc3880776c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.075+0000 7fc38f600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc38810bd10 0x7fc388077df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.076+0000 7fc387e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc38810b3c0 0x7fc3880776c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:21.079 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.076+0000 7fc387e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc38810b3c0 0x7fc3880776c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:37258/0 (socket says 172.21.15.16:37258) 2024-08-31T06:22:21.079 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.077+0000 7fc38cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc388105a90 0x7fc38807b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.077+0000 7fc387e00640 1 -- 172.21.15.16:0/1369813268 learned_addr learned my addr 172.21.15.16:0/1369813268 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.077+0000 7fc38f600640 1 -- 172.21.15.16:0/1369813268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc388115a20 con 0x7fc388105a90 2024-08-31T06:22:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.077+0000 7fc38f600640 1 -- 172.21.15.16:0/1369813268 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc3881158a0 con 0x7fc38810b3c0 2024-08-31T06:22:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.077+0000 7fc38f600640 1 -- 172.21.15.16:0/1369813268 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc388115ba0 con 0x7fc38810bd10 2024-08-31T06:22:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.078+0000 7fc387e00640 1 -- 172.21.15.16:0/1369813268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc38810bd10 msgr2=0x7fc388077df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.078+0000 7fc387e00640 1 --2- 172.21.15.16:0/1369813268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc38810bd10 0x7fc388077df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.078+0000 7fc387e00640 1 -- 172.21.15.16:0/1369813268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc388105a90 msgr2=0x7fc38807b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.078+0000 7fc387e00640 1 --2- 172.21.15.16:0/1369813268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc388105a90 0x7fc38807b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.078+0000 7fc387e00640 1 -- 172.21.15.16:0/1369813268 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3881ac9a0 con 0x7fc38810b3c0 2024-08-31T06:22:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.081+0000 7fc387e00640 1 --2- 172.21.15.16:0/1369813268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc38810b3c0 0x7fc3880776c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fc37400e9f0 tx=0x7fc37400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.081+0000 7fc385600640 1 -- 172.21.15.16:0/1369813268 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37400ccc0 con 0x7fc38810b3c0 2024-08-31T06:22:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.081+0000 7fc38f600640 1 -- 172.21.15.16:0/1369813268 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3881acbd0 con 0x7fc38810b3c0 2024-08-31T06:22:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.082+0000 7fc38f600640 1 -- 172.21.15.16:0/1369813268 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3881ad170 con 0x7fc38810b3c0 2024-08-31T06:22:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.082+0000 7fc385600640 1 -- 172.21.15.16:0/1369813268 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3740108e0 con 0x7fc38810b3c0 2024-08-31T06:22:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.082+0000 7fc385600640 1 -- 172.21.15.16:0/1369813268 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc374018860 con 0x7fc38810b3c0 2024-08-31T06:22:21.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.092+0000 7fc3966d2740 1 Processor -- start 2024-08-31T06:22:21.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.092+0000 7fc385600640 1 -- 172.21.15.16:0/1369813268 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fc374018a00 con 0x7fc38810b3c0 2024-08-31T06:22:21.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.096+0000 7fc3966d2740 1 -- start start 2024-08-31T06:22:21.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.097+0000 7fc3966d2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6340f71b0 0x55b63449a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:21.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.097+0000 7fc3966d2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6340ce2f0 0x55b63449abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:21.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.097+0000 7fc3966d2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6340d64d0 0x55b63449b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.097+0000 7fc3966d2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b63440ff70 con 0x55b6340ce2f0 2024-08-31T06:22:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.097+0000 7fc3966d2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b6343ff500 con 0x55b6340f71b0 2024-08-31T06:22:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.097+0000 7fc3966d2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b6344034b0 con 0x55b6340d64d0 2024-08-31T06:22:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.097+0000 7fc38cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6340ce2f0 0x55b63449abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.098+0000 7fc38d600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6340f71b0 0x55b63449a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.098+0000 7fc387e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6340d64d0 0x55b63449b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.098+0000 7fc38cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6340ce2f0 0x55b63449abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34680/0 (socket says 172.21.15.16:34680) 2024-08-31T06:22:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.098+0000 7fc38cc00640 1 -- 172.21.15.16:0/2519706917 learned_addr learned my addr 172.21.15.16:0/2519706917 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.099+0000 7fc38d600640 1 -- 172.21.15.16:0/2519706917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6340d64d0 msgr2=0x55b63449b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.099+0000 7fc38d600640 1 --2- 172.21.15.16:0/2519706917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6340d64d0 0x55b63449b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.099+0000 7fc38d600640 1 -- 172.21.15.16:0/2519706917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6340ce2f0 msgr2=0x55b63449abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.099+0000 7fc38d600640 1 --2- 172.21.15.16:0/2519706917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6340ce2f0 0x55b63449abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.099+0000 7fc38d600640 1 -- 172.21.15.16:0/2519706917 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6343b9c10 con 0x55b6340f71b0 2024-08-31T06:22:21.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.101+0000 7fc38cc00640 1 --2- 172.21.15.16:0/2519706917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6340ce2f0 0x55b63449abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:22:21.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.101+0000 7fc38d600640 1 --2- 172.21.15.16:0/2519706917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6340f71b0 0x55b63449a690 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fc38002ade0 tx=0x7fc380009460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be01a5cac4539dc5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:21.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.102+0000 7fc385600640 1 --2- 172.21.15.16:0/1369813268 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc3700b4da0 0x7fc3700b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:21.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.102+0000 7fc379c00640 1 -- 172.21.15.16:0/2519706917 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc380031cf0 con 0x55b6340f71b0 2024-08-31T06:22:21.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.102+0000 7fc379c00640 1 -- 172.21.15.16:0/2519706917 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc38003f070 con 0x55b6340f71b0 2024-08-31T06:22:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.102+0000 7fc379c00640 1 -- 172.21.15.16:0/2519706917 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc380004040 con 0x55b6340f71b0 2024-08-31T06:22:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.103+0000 7fc3966d2740 1 -- 172.21.15.16:0/2519706917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6340f71b0 msgr2=0x55b63449a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.103+0000 7fc3966d2740 1 --2- 172.21.15.16:0/2519706917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6340f71b0 0x55b63449a690 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fc38002ade0 tx=0x7fc380009460 comp rx=0 tx=0).stop 2024-08-31T06:22:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.104+0000 7fc385600640 1 -- 172.21.15.16:0/1369813268 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8470+0+0 (secure 0 0 0) 0x7fc3740e59f0 con 0x7fc38810b3c0 2024-08-31T06:22:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.105+0000 7fc3966d2740 1 -- 172.21.15.16:0/2519706917 shutdown_connections 2024-08-31T06:22:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.105+0000 7fc3966d2740 1 --2- 172.21.15.16:0/2519706917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6340d64d0 0x55b63449b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.105+0000 7fc3966d2740 1 --2- 172.21.15.16:0/2519706917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6340ce2f0 0x55b63449abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.105+0000 7fc3966d2740 1 --2- 172.21.15.16:0/2519706917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6340f71b0 0x55b63449a690 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.105+0000 7fc3966d2740 1 -- 172.21.15.16:0/2519706917 >> 172.21.15.16:0/2519706917 conn(0x55b6340eb300 msgr2=0x55b634498580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.105+0000 7fc38cc00640 1 --2- 172.21.15.16:0/1369813268 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc3700b4da0 0x7fc3700b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.106+0000 7fc3966d2740 1 -- 172.21.15.16:0/2519706917 shutdown_connections 2024-08-31T06:22:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.106+0000 7fc3966d2740 1 -- 172.21.15.16:0/2519706917 wait complete. 2024-08-31T06:22:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.107+0000 7fc3966d2740 1 -- start start 2024-08-31T06:22:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.107+0000 7fc3966d2740 1 Processor -- start 2024-08-31T06:22:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.108+0000 7fc38cc00640 1 --2- 172.21.15.16:0/1369813268 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc3700b4da0 0x7fc3700b72b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x55b634499310 tx=0x7fc37c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:21.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.110+0000 7fc3966d2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6340ce2f0 0x55b6344a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:21.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.110+0000 7fc3966d2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6340d64d0 0x55b6344a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:21.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.110+0000 7fc3966d2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6340f71b0 0x55b6344a6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:21.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.110+0000 7fc3966d2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b634338ab0 con 0x55b6340ce2f0 2024-08-31T06:22:21.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.110+0000 7fc3966d2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b6343a6320 con 0x55b6340d64d0 2024-08-31T06:22:21.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.110+0000 7fc3966d2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b6344b9dc0 con 0x55b6340f71b0 2024-08-31T06:22:21.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.110+0000 7fc3966d2740 10 client.0 _ll_register_callbacks cb 0x55b6343dfe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:22:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.111+0000 7fc38cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6340ce2f0 0x55b6344a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.111+0000 7fc38cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6340ce2f0 0x55b6344a5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34688/0 (socket says 172.21.15.16:34688) 2024-08-31T06:22:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.111+0000 7fc38cc00640 1 -- 172.21.15.16:0/3318249445 learned_addr learned my addr 172.21.15.16:0/3318249445 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.111+0000 7fc38d600640 1 --2- 172.21.15.16:0/3318249445 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6340f71b0 0x55b6344a6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.111+0000 7fc387e00640 1 --2- 172.21.15.16:0/3318249445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6340d64d0 0x55b6344a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.112+0000 7fc38cc00640 1 -- 172.21.15.16:0/3318249445 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6340f71b0 msgr2=0x55b6344a6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.112+0000 7fc38cc00640 1 --2- 172.21.15.16:0/3318249445 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6340f71b0 0x55b6344a6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.112+0000 7fc38cc00640 1 -- 172.21.15.16:0/3318249445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6340d64d0 msgr2=0x55b6344a61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.112+0000 7fc38cc00640 1 --2- 172.21.15.16:0/3318249445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6340d64d0 0x55b6344a61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.112+0000 7fc38cc00640 1 -- 172.21.15.16:0/3318249445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b634162790 con 0x55b6340ce2f0 2024-08-31T06:22:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.112+0000 7fc38cc00640 1 --2- 172.21.15.16:0/3318249445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6340ce2f0 0x55b6344a5c90 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fc37c01e680 tx=0x7fc37c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e04c577366d4d873 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.113+0000 7fc36e000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:22:21.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.113+0000 7fc36e000640 1 -- 172.21.15.16:0/3318249445 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37c02b060 con 0x55b6340ce2f0 2024-08-31T06:22:21.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.113+0000 7fc36e000640 1 -- 172.21.15.16:0/3318249445 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8470+0+0 (secure 0 0 0) 0x7fc37c01c670 con 0x55b6340ce2f0 2024-08-31T06:22:21.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.114+0000 7fc3966d2740 1 -- 172.21.15.16:0/3318249445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc38003f070 con 0x55b6340ce2f0 2024-08-31T06:22:21.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.115+0000 7fc36e000640 1 -- 172.21.15.16:0/3318249445 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fc37c01c870 con 0x55b6340ce2f0 2024-08-31T06:22:21.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.115+0000 7fc3966d2740 10 client.18444 fetch_fsmap learned FSMap version 52 2024-08-31T06:22:21.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.115+0000 7fc3966d2740 1 -- 172.21.15.16:0/3318249445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55b6341db620 con 0x55b6340ce2f0 2024-08-31T06:22:21.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.118+0000 7fc36e000640 1 -- 172.21.15.16:0/3318249445 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc38003f070 con 0x55b6340ce2f0 2024-08-31T06:22:21.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:20 smithi155 ceph-mon[21742]: pgmap v837: 97 pgs: 97 active+clean; 3.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.7 MiB/s wr, 675 op/s 2024-08-31T06:22:21.123 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.120+0000 7fc36e000640 1 -- 172.21.15.16:0/3318249445 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 52) ==== 8350+0+0 (secure 0 0 0) 0x7fc37c01a040 con 0x55b6340ce2f0 2024-08-31T06:22:21.124 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.121+0000 7fc3966d2740 10 client.18444 fetch_fsmap finished waiting for FSMap version 52 2024-08-31T06:22:21.124 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.121+0000 7fc3966d2740 10 client.18444 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:22:21.124 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.121+0000 7fc3966d2740 20 client.18444 populate_metadata read hostname 'smithi016' 2024-08-31T06:22:21.124 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.121+0000 7fc3966d2740 1 --2- 172.21.15.16:0/3318249445 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55b6340b5d80 0x55b6344c93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:21.124 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.121+0000 7fc3966d2740 4 client.18444 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:22:21.124 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.121+0000 7fc3966d2740 1 -- 172.21.15.16:0/3318249445 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b634411ec0 con 0x55b6340b5d80 2024-08-31T06:22:21.124 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.121+0000 7fc387e00640 1 --2- 172.21.15.16:0/3318249445 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55b6340b5d80 0x55b6344c93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:21.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.124+0000 7fc36e000640 10 client.18444 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:22:21.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.124+0000 7fc387e00640 1 --2- 172.21.15.16:0/3318249445 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55b6340b5d80 0x55b6344c93b0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98194414652692fc server_cookie=9e51ca3a9e2313ff in_seq=0 out_seq=0 2024-08-31T06:22:21.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.125+0000 7fc36e000640 1 -- 172.21.15.16:0/3318249445 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b634411ec0 con 0x55b6340b5d80 2024-08-31T06:22:21.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.125+0000 7fc36e000640 10 client.18444 handle_command_reply: tid=0 2024-08-31T06:22:21.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.125+0000 7fc3966d2740 1 client.18444 shutdown 2024-08-31T06:22:21.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.129+0000 7fc3966d2740 1 -- 172.21.15.16:0/3318249445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6340ce2f0 msgr2=0x55b6344a5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:21.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.129+0000 7fc3966d2740 1 --2- 172.21.15.16:0/3318249445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6340ce2f0 0x55b6344a5c90 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fc37c01e680 tx=0x7fc37c0046c0 comp rx=0 tx=0).stop 2024-08-31T06:22:21.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.131+0000 7fc3966d2740 1 -- 172.21.15.16:0/3318249445 shutdown_connections 2024-08-31T06:22:21.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.131+0000 7fc3966d2740 1 --2- 172.21.15.16:0/3318249445 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55b6340b5d80 0x55b6344c93b0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:21.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.131+0000 7fc3966d2740 1 --2- 172.21.15.16:0/3318249445 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6340f71b0 0x55b6344a6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:21.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.131+0000 7fc3966d2740 1 --2- 172.21.15.16:0/3318249445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6340d64d0 0x55b6344a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:21.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.131+0000 7fc36e000640 0 client.18444 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:22:21.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.131+0000 7fc3966d2740 1 --2- 172.21.15.16:0/3318249445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6340ce2f0 0x55b6344a5c90 secure :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fc37c01e680 tx=0x7fc37c0046c0 comp rx=0 tx=0).stop 2024-08-31T06:22:21.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.131+0000 7fc3966d2740 1 -- 172.21.15.16:0/3318249445 >> 172.21.15.16:0/3318249445 conn(0x55b6340eb300 msgr2=0x55b63449b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:21.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.132+0000 7fc3966d2740 1 -- 172.21.15.16:0/3318249445 shutdown_connections 2024-08-31T06:22:21.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.132+0000 7fc3966d2740 1 -- 172.21.15.16:0/3318249445 wait complete. 2024-08-31T06:22:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.132+0000 7fc3966d2740 20 client.18444 trim_cache size 0 max 16384 2024-08-31T06:22:21.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:20 smithi016 ceph-mon[17194]: pgmap v837: 97 pgs: 97 active+clean; 3.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.7 MiB/s wr, 675 op/s 2024-08-31T06:22:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.172+0000 7fc3966d2740 1 Processor -- start 2024-08-31T06:22:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.172+0000 7fc3966d2740 1 -- start start 2024-08-31T06:22:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.173+0000 7fc3966d2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6340d64d0 0x55b6344974e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.173+0000 7fc3966d2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6340f71b0 0x55b6344c3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.173+0000 7fc3966d2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b63408b1c0 0x55b6344a2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.173+0000 7fc387e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6340f71b0 0x55b6344c3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.173+0000 7fc3966d2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc368011060 con 0x55b6340d64d0 2024-08-31T06:22:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.174+0000 7fc387e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6340f71b0 0x55b6344c3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:45780/0 (socket says 172.21.15.16:45780) 2024-08-31T06:22:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.174+0000 7fc387e00640 1 -- 172.21.15.16:0/2862247718 learned_addr learned my addr 172.21.15.16:0/2862247718 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.174+0000 7fc38cc00640 1 --2- 172.21.15.16:0/2862247718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6340d64d0 0x55b6344974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.174+0000 7fc3966d2740 1 -- 172.21.15.16:0/2862247718 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc36800e900 con 0x55b63408b1c0 2024-08-31T06:22:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.174+0000 7fc3966d2740 1 -- 172.21.15.16:0/2862247718 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc36800c730 con 0x55b6340f71b0 2024-08-31T06:22:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.174+0000 7fc38d600640 1 --2- 172.21.15.16:0/2862247718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b63408b1c0 0x55b6344a2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.174+0000 7fc38d600640 1 -- 172.21.15.16:0/2862247718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6340f71b0 msgr2=0x55b6344c3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.174+0000 7fc38d600640 1 --2- 172.21.15.16:0/2862247718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6340f71b0 0x55b6344c3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.174+0000 7fc38d600640 1 -- 172.21.15.16:0/2862247718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6340d64d0 msgr2=0x55b6344974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.174+0000 7fc38d600640 1 --2- 172.21.15.16:0/2862247718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6340d64d0 0x55b6344974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.174+0000 7fc38d600640 1 -- 172.21.15.16:0/2862247718 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b63449c990 con 0x55b63408b1c0 2024-08-31T06:22:21.179 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.176+0000 7fc38d600640 1 --2- 172.21.15.16:0/2862247718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b63408b1c0 0x55b6344a2ee0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x55b6344a6c70 tx=0x7fc380031af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f058d492eeb5bf server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:21.179 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.177+0000 7fc379200640 1 -- 172.21.15.16:0/2862247718 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc380061070 con 0x55b63408b1c0 2024-08-31T06:22:21.180 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.177+0000 7fc379200640 1 -- 172.21.15.16:0/2862247718 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc380036030 con 0x55b63408b1c0 2024-08-31T06:22:21.180 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.177+0000 7fc379200640 1 -- 172.21.15.16:0/2862247718 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc380004280 con 0x55b63408b1c0 2024-08-31T06:22:21.181 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.178+0000 7fc3966d2740 1 -- 172.21.15.16:0/2862247718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b63408b1c0 msgr2=0x55b6344a2ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:21.181 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.178+0000 7fc3966d2740 1 --2- 172.21.15.16:0/2862247718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b63408b1c0 0x55b6344a2ee0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x55b6344a6c70 tx=0x7fc380031af0 comp rx=0 tx=0).stop 2024-08-31T06:22:21.181 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.178+0000 7fc3966d2740 1 -- 172.21.15.16:0/2862247718 shutdown_connections 2024-08-31T06:22:21.181 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.178+0000 7fc3966d2740 1 --2- 172.21.15.16:0/2862247718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6340f71b0 0x55b6344c3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:21.181 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.178+0000 7fc3966d2740 1 --2- 172.21.15.16:0/2862247718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b63408b1c0 0x55b6344a2ee0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:21.181 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.178+0000 7fc3966d2740 1 --2- 172.21.15.16:0/2862247718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6340d64d0 0x55b6344974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:21.181 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.178+0000 7fc3966d2740 1 -- 172.21.15.16:0/2862247718 >> 172.21.15.16:0/2862247718 conn(0x55b6340eb300 msgr2=0x55b6344983e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:21.181 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.178+0000 7fc3966d2740 1 -- 172.21.15.16:0/2862247718 shutdown_connections 2024-08-31T06:22:21.182 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.178+0000 7fc3966d2740 1 -- 172.21.15.16:0/2862247718 wait complete. 2024-08-31T06:22:21.182 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.179+0000 7fc3966d2740 1 -- start start 2024-08-31T06:22:21.182 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.179+0000 7fc3966d2740 1 Processor -- start 2024-08-31T06:22:21.183 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.180+0000 7fc3966d2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b63408b1c0 0x55b6344bf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.181+0000 7fc3966d2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6340d64d0 0x55b6344bfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.181+0000 7fc3966d2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6340f71b0 0x55b6344bab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.181+0000 7fc3966d2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc36800cdc0 con 0x55b63408b1c0 2024-08-31T06:22:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.181+0000 7fc3966d2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc36800e230 con 0x55b6340d64d0 2024-08-31T06:22:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.181+0000 7fc3966d2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc36800d490 con 0x55b6340f71b0 2024-08-31T06:22:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.181+0000 7fc3966d2740 10 client.0 _ll_register_callbacks cb 0x7fc37c009550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:22:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.181+0000 7fc38d600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6340f71b0 0x55b6344bab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.182+0000 7fc38d600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6340f71b0 0x55b6344bab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:45782/0 (socket says 172.21.15.16:45782) 2024-08-31T06:22:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.182+0000 7fc387e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6340d64d0 0x55b6344bfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.182+0000 7fc38d600640 1 -- 172.21.15.16:0/2397764350 learned_addr learned my addr 172.21.15.16:0/2397764350 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.183+0000 7fc38cc00640 1 --2- 172.21.15.16:0/2397764350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b63408b1c0 0x55b6344bf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.184+0000 7fc38d600640 1 -- 172.21.15.16:0/2397764350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6340d64d0 msgr2=0x55b6344bfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.184+0000 7fc38d600640 1 --2- 172.21.15.16:0/2397764350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6340d64d0 0x55b6344bfaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.184+0000 7fc38d600640 1 -- 172.21.15.16:0/2397764350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b63408b1c0 msgr2=0x55b6344bf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.184+0000 7fc38d600640 1 --2- 172.21.15.16:0/2397764350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b63408b1c0 0x55b6344bf4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:21.188 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.184+0000 7fc38d600640 1 -- 172.21.15.16:0/2397764350 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6344bb2f0 con 0x55b6340f71b0 2024-08-31T06:22:21.188 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.184+0000 7fc38d600640 1 --2- 172.21.15.16:0/2397764350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6340f71b0 0x55b6344bab60 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fc380006010 tx=0x7fc3800318c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca5dd65b83d42904 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:21.188 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.184+0000 7fc36e000640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:22:21.188 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.185+0000 7fc36e000640 1 -- 172.21.15.16:0/2397764350 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc38006b020 con 0x55b6340f71b0 2024-08-31T06:22:21.189 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.185+0000 7fc36e000640 1 -- 172.21.15.16:0/2397764350 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8470+0+0 (secure 0 0 0) 0x7fc380003b70 con 0x55b6340f71b0 2024-08-31T06:22:21.189 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.185+0000 7fc3966d2740 1 -- 172.21.15.16:0/2397764350 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc380036030 con 0x55b6340f71b0 2024-08-31T06:22:21.189 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.185+0000 7fc36e000640 1 -- 172.21.15.16:0/2397764350 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fc380061040 con 0x55b6340f71b0 2024-08-31T06:22:21.189 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.186+0000 7fc3966d2740 10 client.28957 fetch_fsmap learned FSMap version 52 2024-08-31T06:22:21.189 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.186+0000 7fc3966d2740 1 -- 172.21.15.16:0/2397764350 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55b634533920 con 0x55b6340f71b0 2024-08-31T06:22:21.189 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.186+0000 7fc36e000640 1 -- 172.21.15.16:0/2397764350 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc380036030 con 0x55b6340f71b0 2024-08-31T06:22:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.188+0000 7fc36e000640 1 -- 172.21.15.16:0/2397764350 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 52) ==== 8350+0+0 (secure 0 0 0) 0x7fc38002f610 con 0x55b6340f71b0 2024-08-31T06:22:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.188+0000 7fc3966d2740 10 client.28957 fetch_fsmap finished waiting for FSMap version 52 2024-08-31T06:22:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.188+0000 7fc3966d2740 10 client.28957 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:22:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.188+0000 7fc3966d2740 20 client.28957 populate_metadata read hostname 'smithi016' 2024-08-31T06:22:21.191 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.188+0000 7fc3966d2740 1 --2- 172.21.15.16:0/2397764350 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55b634533b40 0x55b634535f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:21.191 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.188+0000 7fc3966d2740 4 client.28957 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:22:21.191 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.188+0000 7fc3966d2740 1 -- 172.21.15.16:0/2397764350 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b63440d960 con 0x55b634533b40 2024-08-31T06:22:21.191 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.188+0000 7fc38cc00640 1 --2- 172.21.15.16:0/2397764350 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55b634533b40 0x55b634535f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:21.191 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.189+0000 7fc38cc00640 1 --2- 172.21.15.16:0/2397764350 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55b634533b40 0x55b634535f60 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0dbbbed10b262b4 server_cookie=a184735ef75567ae in_seq=0 out_seq=0 2024-08-31T06:22:21.191 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.189+0000 7fc36e000640 10 client.28957 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:22:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.217+0000 7fc36e000640 1 -- 172.21.15.16:0/2397764350 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b63440d960 con 0x55b634533b40 2024-08-31T06:22:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.217+0000 7fc36e000640 10 client.28957 handle_command_reply: tid=0 2024-08-31T06:22:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.217+0000 7fc3966d2740 1 client.28957 shutdown 2024-08-31T06:22:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.217+0000 7fc3966d2740 1 -- 172.21.15.16:0/2397764350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6340f71b0 msgr2=0x55b6344bab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.217+0000 7fc3966d2740 1 --2- 172.21.15.16:0/2397764350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6340f71b0 0x55b6344bab60 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fc380006010 tx=0x7fc3800318c0 comp rx=0 tx=0).stop 2024-08-31T06:22:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.218+0000 7fc3966d2740 1 -- 172.21.15.16:0/2397764350 shutdown_connections 2024-08-31T06:22:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.218+0000 7fc3966d2740 1 --2- 172.21.15.16:0/2397764350 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55b634533b40 0x55b634535f60 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.218+0000 7fc3966d2740 1 --2- 172.21.15.16:0/2397764350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b6340f71b0 0x55b6344bab60 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.218+0000 7fc3966d2740 1 --2- 172.21.15.16:0/2397764350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6340d64d0 0x55b6344bfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.218+0000 7fc3966d2740 1 --2- 172.21.15.16:0/2397764350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b63408b1c0 0x55b6344bf4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.218+0000 7fc3966d2740 1 -- 172.21.15.16:0/2397764350 >> 172.21.15.16:0/2397764350 conn(0x55b6340eb300 msgr2=0x55b6344a09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.218+0000 7fc36e000640 0 client.28957 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:22:21.222 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.218+0000 7fc3966d2740 1 -- 172.21.15.16:0/2397764350 shutdown_connections 2024-08-31T06:22:21.222 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.218+0000 7fc3966d2740 1 -- 172.21.15.16:0/2397764350 wait complete. 2024-08-31T06:22:21.222 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.218+0000 7fc3966d2740 20 client.28957 trim_cache size 0 max 16384 2024-08-31T06:22:21.222 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:22:21.222 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:22:21.222 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:22:21.222 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:22:21.223 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.219+0000 7fc38f600640 1 -- 172.21.15.16:0/1369813268 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc3700b4da0 msgr2=0x7fc3700b72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:21.223 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.219+0000 7fc38f600640 1 --2- 172.21.15.16:0/1369813268 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc3700b4da0 0x7fc3700b72b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x55b634499310 tx=0x7fc37c005f90 comp rx=0 tx=0).stop 2024-08-31T06:22:21.223 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.219+0000 7fc38f600640 1 -- 172.21.15.16:0/1369813268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc38810b3c0 msgr2=0x7fc3880776c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:21.223 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.219+0000 7fc38f600640 1 --2- 172.21.15.16:0/1369813268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc38810b3c0 0x7fc3880776c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fc37400e9f0 tx=0x7fc37400eeb0 comp rx=0 tx=0).stop 2024-08-31T06:22:21.223 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.219+0000 7fc38f600640 1 -- 172.21.15.16:0/1369813268 shutdown_connections 2024-08-31T06:22:21.223 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.219+0000 7fc38f600640 1 --2- 172.21.15.16:0/1369813268 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fc3700b4da0 0x7fc3700b72b0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:21.223 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.219+0000 7fc38f600640 1 --2- 172.21.15.16:0/1369813268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc38810bd10 0x7fc388077df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.219+0000 7fc38f600640 1 --2- 172.21.15.16:0/1369813268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc38810b3c0 0x7fc3880776c0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.219+0000 7fc38f600640 1 --2- 172.21.15.16:0/1369813268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc388105a90 0x7fc38807b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.219+0000 7fc38f600640 1 -- 172.21.15.16:0/1369813268 >> 172.21.15.16:0/1369813268 conn(0x7fc388101490 msgr2=0x7fc38810a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.220+0000 7fc38f600640 1 -- 172.21.15.16:0/1369813268 shutdown_connections 2024-08-31T06:22:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:21.220+0000 7fc38f600640 1 -- 172.21.15.16:0/1369813268 wait complete. 2024-08-31T06:22:21.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:20 smithi184 ceph-mon[22331]: pgmap v837: 97 pgs: 97 active+clean; 3.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.7 MiB/s wr, 675 op/s 2024-08-31T06:22:21.465 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T06:22:21.466 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:22:21.625 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 895.0 s, 947.9 tps, lat 105.455 ms stddev 118.234 2024-08-31T06:22:22.088 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:22:22.818 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:22 smithi016 ceph-mon[17194]: pgmap v838: 97 pgs: 97 active+clean; 3.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 10 MiB/s wr, 729 op/s 2024-08-31T06:22:23.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.109+0000 7fafdb600640 1 -- 172.21.15.16:0/848714080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafd4104750 msgr2=0x7fafd4104b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:23.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.109+0000 7fafdb600640 1 --2- 172.21.15.16:0/848714080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafd4104750 0x7fafd4104b60 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fafc800dc90 tx=0x7fafc802e370 comp rx=0 tx=0).stop 2024-08-31T06:22:23.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.111+0000 7fafdb600640 1 -- 172.21.15.16:0/848714080 shutdown_connections 2024-08-31T06:22:23.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.111+0000 7fafdb600640 1 --2- 172.21.15.16:0/848714080 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafd41050a0 0x7fafd410d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:23.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.111+0000 7fafdb600640 1 --2- 172.21.15.16:0/848714080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd410dbb0 0x7fafd41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:23.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.111+0000 7fafdb600640 1 --2- 172.21.15.16:0/848714080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafd4104750 0x7fafd4104b60 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:23.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.111+0000 7fafdb600640 1 -- 172.21.15.16:0/848714080 >> 172.21.15.16:0/848714080 conn(0x7fafd40794d0 msgr2=0x7fafd40798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:23.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.111+0000 7fafdb600640 1 -- 172.21.15.16:0/848714080 shutdown_connections 2024-08-31T06:22:23.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.111+0000 7fafdb600640 1 -- 172.21.15.16:0/848714080 wait complete. 2024-08-31T06:22:23.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.113+0000 7fafdb600640 1 Processor -- start 2024-08-31T06:22:23.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.114+0000 7fafdb600640 1 -- start start 2024-08-31T06:22:23.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.114+0000 7fafdb600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd4104750 0x7fafd41a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:23.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.115+0000 7fafdb600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafd41050a0 0x7fafd41a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:23.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.115+0000 7fafdb600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafd410dbb0 0x7fafd41a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:23.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.115+0000 7fafd3e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafd41050a0 0x7fafd41a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:23.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.115+0000 7fafd8c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd4104750 0x7fafd41a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:23.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.115+0000 7fafd3e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafd41050a0 0x7fafd41a0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:45806/0 (socket says 172.21.15.16:45806) 2024-08-31T06:22:23.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.115+0000 7fafd8c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd4104750 0x7fafd41a08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34722/0 (socket says 172.21.15.16:34722) 2024-08-31T06:22:23.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.115+0000 7fafd9600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafd410dbb0 0x7fafd41a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:23.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.116+0000 7fafd3e00640 1 -- 172.21.15.16:0/722046871 learned_addr learned my addr 172.21.15.16:0/722046871 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:23.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.116+0000 7fafdb600640 1 -- 172.21.15.16:0/722046871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fafd4111650 con 0x7fafd4104750 2024-08-31T06:22:23.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.116+0000 7fafdb600640 1 -- 172.21.15.16:0/722046871 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fafd41114d0 con 0x7fafd410dbb0 2024-08-31T06:22:23.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.116+0000 7fafdb600640 1 -- 172.21.15.16:0/722046871 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fafd41117d0 con 0x7fafd41050a0 2024-08-31T06:22:23.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.117+0000 7fafd3e00640 1 -- 172.21.15.16:0/722046871 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafd410dbb0 msgr2=0x7fafd41a60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:23.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.117+0000 7fafd3e00640 1 --2- 172.21.15.16:0/722046871 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafd410dbb0 0x7fafd41a60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:23.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.117+0000 7fafd3e00640 1 -- 172.21.15.16:0/722046871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd4104750 msgr2=0x7fafd41a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:23.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.117+0000 7fafd3e00640 1 --2- 172.21.15.16:0/722046871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd4104750 0x7fafd41a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:23.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.117+0000 7fafd3e00640 1 -- 172.21.15.16:0/722046871 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd41a6820 con 0x7fafd41050a0 2024-08-31T06:22:23.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.117+0000 7fafd9600640 1 --2- 172.21.15.16:0/722046871 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafd410dbb0 0x7fafd41a60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:22:23.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.117+0000 7fafd3e00640 1 --2- 172.21.15.16:0/722046871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafd41050a0 0x7fafd41a0e50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fafc000c970 tx=0x7fafc000ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:23.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.118+0000 7fafd1600640 1 -- 172.21.15.16:0/722046871 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc0007bb0 con 0x7fafd41050a0 2024-08-31T06:22:23.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.118+0000 7fafd1600640 1 -- 172.21.15.16:0/722046871 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafc0007d50 con 0x7fafd41050a0 2024-08-31T06:22:23.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.118+0000 7fafd1600640 1 -- 172.21.15.16:0/722046871 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc000f450 con 0x7fafd41050a0 2024-08-31T06:22:23.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.118+0000 7fafdb600640 1 -- 172.21.15.16:0/722046871 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafd41a6b10 con 0x7fafd41050a0 2024-08-31T06:22:23.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.118+0000 7fafdb600640 1 -- 172.21.15.16:0/722046871 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafd41ad270 con 0x7fafd41050a0 2024-08-31T06:22:23.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:22 smithi155 ceph-mon[21742]: pgmap v838: 97 pgs: 97 active+clean; 3.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 10 MiB/s wr, 729 op/s 2024-08-31T06:22:23.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.121+0000 7fafdb600640 1 -- 172.21.15.16:0/722046871 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafd4104b60 con 0x7fafd41050a0 2024-08-31T06:22:23.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.126+0000 7fafd1600640 1 -- 172.21.15.16:0/722046871 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fafc0004eb0 con 0x7fafd41050a0 2024-08-31T06:22:23.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.130+0000 7fafd1600640 1 --2- 172.21.15.16:0/722046871 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fafbc0b4da0 0x7fafbc0b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:23.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.131+0000 7fafd1600640 1 -- 172.21.15.16:0/722046871 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8470+0+0 (secure 0 0 0) 0x7fafc00e5190 con 0x7fafd41050a0 2024-08-31T06:22:23.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.131+0000 7fafd8c00640 1 --2- 172.21.15.16:0/722046871 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fafbc0b4da0 0x7fafbc0b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:23.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.133+0000 7fafd1600640 1 -- 172.21.15.16:0/722046871 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fafc0014030 con 0x7fafd41050a0 2024-08-31T06:22:23.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.134+0000 7fafd8c00640 1 --2- 172.21.15.16:0/722046871 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fafbc0b4da0 0x7fafbc0b72b0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fafc800dad0 tx=0x7fafc8000bc0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:23.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:22 smithi184 ceph-mon[22331]: pgmap v838: 97 pgs: 97 active+clean; 3.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 10 MiB/s wr, 729 op/s 2024-08-31T06:22:23.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.531+0000 7fafdb600640 1 -- 172.21.15.16:0/722046871 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fafd402e430 con 0x7fafd41050a0 2024-08-31T06:22:23.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.531+0000 7fafd1600640 1 -- 172.21.15.16:0/722046871 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+11918 (secure 0 0 0) 0x7fafc00ab900 con 0x7fafd41050a0 2024-08-31T06:22:23.535 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:22:23.535 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":52,"btime":"2024-08-31T06:22:09:205644+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":52,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:22:08.355818+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24712,24724,24737]},"id":1}]} 2024-08-31T06:22:23.536 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 52 2024-08-31T06:22:23.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.535+0000 7fafdb600640 1 -- 172.21.15.16:0/722046871 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fafbc0b4da0 msgr2=0x7fafbc0b72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:23.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.535+0000 7fafdb600640 1 --2- 172.21.15.16:0/722046871 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fafbc0b4da0 0x7fafbc0b72b0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fafc800dad0 tx=0x7fafc8000bc0 comp rx=0 tx=0).stop 2024-08-31T06:22:23.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.535+0000 7fafdb600640 1 -- 172.21.15.16:0/722046871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafd41050a0 msgr2=0x7fafd41a0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:23.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.535+0000 7fafdb600640 1 --2- 172.21.15.16:0/722046871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafd41050a0 0x7fafd41a0e50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fafc000c970 tx=0x7fafc000ce30 comp rx=0 tx=0).stop 2024-08-31T06:22:23.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.535+0000 7fafdb600640 1 -- 172.21.15.16:0/722046871 shutdown_connections 2024-08-31T06:22:23.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.535+0000 7fafdb600640 1 --2- 172.21.15.16:0/722046871 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fafbc0b4da0 0x7fafbc0b72b0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:23.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.535+0000 7fafdb600640 1 --2- 172.21.15.16:0/722046871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafd41050a0 0x7fafd41a0e50 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:23.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.535+0000 7fafdb600640 1 --2- 172.21.15.16:0/722046871 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafd410dbb0 0x7fafd41a60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:23.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.535+0000 7fafdb600640 1 --2- 172.21.15.16:0/722046871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd4104750 0x7fafd41a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:23.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.535+0000 7fafdb600640 1 -- 172.21.15.16:0/722046871 >> 172.21.15.16:0/722046871 conn(0x7fafd40794d0 msgr2=0x7fafd40ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:23.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.536+0000 7fafdb600640 1 -- 172.21.15.16:0/722046871 shutdown_connections 2024-08-31T06:22:23.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:23.536+0000 7fafdb600640 1 -- 172.21.15.16:0/722046871 wait complete. 2024-08-31T06:22:23.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:23 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/722046871' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:22:24.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:23 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/722046871' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:22:24.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:23 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/722046871' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:22:24.431 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T06:22:25.081 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:22:25.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:24 smithi155 ceph-mon[21742]: pgmap v839: 97 pgs: 97 active+clean; 3.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 11 MiB/s wr, 743 op/s 2024-08-31T06:22:25.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:24 smithi016 ceph-mon[17194]: pgmap v839: 97 pgs: 97 active+clean; 3.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 11 MiB/s wr, 743 op/s 2024-08-31T06:22:25.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:24 smithi184 ceph-mon[22331]: pgmap v839: 97 pgs: 97 active+clean; 3.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 11 MiB/s wr, 743 op/s 2024-08-31T06:22:26.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.067+0000 7fabb1e00640 1 -- 172.21.15.16:0/2917367440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fabac105950 msgr2=0x7fabac105d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:26.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.067+0000 7fabb1e00640 1 --2- 172.21.15.16:0/2917367440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fabac105950 0x7fabac105d60 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7faba0009900 tx=0x7faba002a410 comp rx=0 tx=0).stop 2024-08-31T06:22:26.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.068+0000 7fabb1e00640 1 -- 172.21.15.16:0/2917367440 shutdown_connections 2024-08-31T06:22:26.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.068+0000 7fabb1e00640 1 --2- 172.21.15.16:0/2917367440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabac10bbe0 0x7fabac112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:26.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.068+0000 7fabb1e00640 1 --2- 172.21.15.16:0/2917367440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabac10b290 0x7fabac10b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:26.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.068+0000 7fabb1e00640 1 --2- 172.21.15.16:0/2917367440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fabac105950 0x7fabac105d60 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:26.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.068+0000 7fabb1e00640 1 -- 172.21.15.16:0/2917367440 >> 172.21.15.16:0/2917367440 conn(0x7fabac1013e0 msgr2=0x7fabac103830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:26.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.068+0000 7fabb1e00640 1 -- 172.21.15.16:0/2917367440 shutdown_connections 2024-08-31T06:22:26.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.068+0000 7fabb1e00640 1 -- 172.21.15.16:0/2917367440 wait complete. 2024-08-31T06:22:26.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.069+0000 7fabb1e00640 1 Processor -- start 2024-08-31T06:22:26.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.069+0000 7fabb1e00640 1 -- start start 2024-08-31T06:22:26.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.070+0000 7fabb1e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabac105950 0x7fabac19c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:26.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.070+0000 7fabb1e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fabac10b290 0x7fabac19c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:26.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.070+0000 7fabb1e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabac10bbe0 0x7fabac1a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:26.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.070+0000 7fabb1e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fabac1158f0 con 0x7fabac105950 2024-08-31T06:22:26.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.070+0000 7fabb1e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fabac115770 con 0x7fabac10bbe0 2024-08-31T06:22:26.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.070+0000 7fabb1e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fabac115a70 con 0x7fabac10b290 2024-08-31T06:22:26.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.070+0000 7fabb0a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabac105950 0x7fabac19c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:26.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.070+0000 7fababe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fabac10b290 0x7fabac19c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:26.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.070+0000 7fababe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fabac10b290 0x7fabac19c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:54078/0 (socket says 172.21.15.16:54078) 2024-08-31T06:22:26.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.070+0000 7fababe00640 1 -- 172.21.15.16:0/2042876762 learned_addr learned my addr 172.21.15.16:0/2042876762 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:26.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.070+0000 7fabb1400640 1 --2- 172.21.15.16:0/2042876762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabac10bbe0 0x7fabac1a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:26.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.071+0000 7fababe00640 1 -- 172.21.15.16:0/2042876762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabac10bbe0 msgr2=0x7fabac1a1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:26.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.071+0000 7fababe00640 1 --2- 172.21.15.16:0/2042876762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabac10bbe0 0x7fabac1a1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:26.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.071+0000 7fababe00640 1 -- 172.21.15.16:0/2042876762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabac105950 msgr2=0x7fabac19c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:26.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.071+0000 7fababe00640 1 --2- 172.21.15.16:0/2042876762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabac105950 0x7fabac19c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:26.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.071+0000 7fababe00640 1 -- 172.21.15.16:0/2042876762 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabac1a21d0 con 0x7fabac10b290 2024-08-31T06:22:26.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.071+0000 7fababe00640 1 --2- 172.21.15.16:0/2042876762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fabac10b290 0x7fabac19c800 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fab9800e9f0 tx=0x7fab9800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:26.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.072+0000 7faba9600640 1 -- 172.21.15.16:0/2042876762 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9800ccc0 con 0x7fabac10b290 2024-08-31T06:22:26.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.072+0000 7fabb1e00640 1 -- 172.21.15.16:0/2042876762 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabac1a24c0 con 0x7fabac10b290 2024-08-31T06:22:26.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.072+0000 7fabb1e00640 1 -- 172.21.15.16:0/2042876762 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabac1a8c20 con 0x7fabac10b290 2024-08-31T06:22:26.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.072+0000 7faba9600640 1 -- 172.21.15.16:0/2042876762 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab980108e0 con 0x7fabac10b290 2024-08-31T06:22:26.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.072+0000 7faba9600640 1 -- 172.21.15.16:0/2042876762 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab98018780 con 0x7fabac10b290 2024-08-31T06:22:26.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.076+0000 7fabba06c740 1 Processor -- start 2024-08-31T06:22:26.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.076+0000 7faba9600640 1 -- 172.21.15.16:0/2042876762 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fab98010430 con 0x7fabac10b290 2024-08-31T06:22:26.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.077+0000 7fabba06c740 1 -- start start 2024-08-31T06:22:26.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.077+0000 7fabba06c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc01a10070 0x55fc01db5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:26.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.078+0000 7fabb1400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc01a10070 0x55fc01db5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:26.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.078+0000 7fabb1400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc01a10070 0x55fc01db5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51546/0 (socket says 172.21.15.16:51546) 2024-08-31T06:22:26.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.078+0000 7fabba06c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc019e71b0 0x55fc01db5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:26.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.078+0000 7fabb1400640 1 -- 172.21.15.16:0/1507914215 learned_addr learned my addr 172.21.15.16:0/1507914215 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:26.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.078+0000 7fabba06c740 1 --2- 172.21.15.16:0/1507914215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc019ef390 0x55fc01db64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:26.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.078+0000 7fabba06c740 1 -- 172.21.15.16:0/1507914215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fc01d2c7b0 con 0x55fc019ef390 2024-08-31T06:22:26.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.079+0000 7fabb0a00640 1 --2- 172.21.15.16:0/1507914215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc019e71b0 0x55fc01db5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:26.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.079+0000 7faba9600640 1 --2- 172.21.15.16:0/2042876762 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fab940b4d20 0x7fab940b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:26.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.079+0000 7fabba06c740 1 -- 172.21.15.16:0/1507914215 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55fc01d1bd40 con 0x55fc01a10070 2024-08-31T06:22:26.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.079+0000 7fabba06c740 1 -- 172.21.15.16:0/1507914215 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55fc01d1fcf0 con 0x55fc019e71b0 2024-08-31T06:22:26.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.079+0000 7fababe00640 1 --2- 172.21.15.16:0/1507914215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc019ef390 0x55fc01db64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:26.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.080+0000 7faba9600640 1 -- 172.21.15.16:0/2042876762 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8470+0+0 (secure 0 0 0) 0x7fab980e5690 con 0x7fabac10b290 2024-08-31T06:22:26.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.080+0000 7fabb1400640 1 --2- 172.21.15.16:0/2042876762 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fab940b4d20 0x7fab940b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:26.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.080+0000 7fabb0a00640 1 -- 172.21.15.16:0/1507914215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc01a10070 msgr2=0x55fc01db5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:26.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.080+0000 7fabb0a00640 1 --2- 172.21.15.16:0/1507914215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc01a10070 0x55fc01db5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:26.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.080+0000 7fabb0a00640 1 -- 172.21.15.16:0/1507914215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc019ef390 msgr2=0x55fc01db64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:26.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.080+0000 7fabb0a00640 1 --2- 172.21.15.16:0/1507914215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc019ef390 0x55fc01db64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:26.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.080+0000 7fabb0a00640 1 -- 172.21.15.16:0/1507914215 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc01a7b650 con 0x55fc019e71b0 2024-08-31T06:22:26.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.081+0000 7fabb0a00640 1 --2- 172.21.15.16:0/1507914215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc019e71b0 0x55fc01db5fb0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7faba002a930 tx=0x7faba0003670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e84c52163414bcc6 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:26.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.081+0000 7fab9dc00640 1 -- 172.21.15.16:0/1507914215 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba0041070 con 0x55fc019e71b0 2024-08-31T06:22:26.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.081+0000 7fab9dc00640 1 -- 172.21.15.16:0/1507914215 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faba0004940 con 0x55fc019e71b0 2024-08-31T06:22:26.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.081+0000 7fabb1400640 1 --2- 172.21.15.16:0/2042876762 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fab940b4d20 0x7fab940b7230 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7faba4009fd0 tx=0x7faba4004ab0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:26.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.081+0000 7fab9dc00640 1 -- 172.21.15.16:0/1507914215 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba00316a0 con 0x55fc019e71b0 2024-08-31T06:22:26.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.083+0000 7fabba06c740 1 -- 172.21.15.16:0/1507914215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc019e71b0 msgr2=0x55fc01db5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:26.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.083+0000 7fabba06c740 1 --2- 172.21.15.16:0/1507914215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc019e71b0 0x55fc01db5fb0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7faba002a930 tx=0x7faba0003670 comp rx=0 tx=0).stop 2024-08-31T06:22:26.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.083+0000 7fabba06c740 1 -- 172.21.15.16:0/1507914215 shutdown_connections 2024-08-31T06:22:26.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.083+0000 7fabba06c740 1 --2- 172.21.15.16:0/1507914215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc019e71b0 0x55fc01db5fb0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:26.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.083+0000 7fabba06c740 1 --2- 172.21.15.16:0/1507914215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc019ef390 0x55fc01db64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:26.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.083+0000 7fabba06c740 1 --2- 172.21.15.16:0/1507914215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc01a10070 0x55fc01db5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:26.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.083+0000 7fabba06c740 1 -- 172.21.15.16:0/1507914215 >> 172.21.15.16:0/1507914215 conn(0x55fc01a04460 msgr2=0x55fc01db3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:26.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.083+0000 7fabba06c740 1 -- 172.21.15.16:0/1507914215 shutdown_connections 2024-08-31T06:22:26.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.083+0000 7fabba06c740 1 -- 172.21.15.16:0/1507914215 wait complete. 2024-08-31T06:22:26.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.084+0000 7fabba06c740 1 -- start start 2024-08-31T06:22:26.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.084+0000 7fabba06c740 1 Processor -- start 2024-08-31T06:22:26.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.085+0000 7fabba06c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc019e71b0 0x55fc01dc9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:26.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.085+0000 7fabba06c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc019ef390 0x55fc01dca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:26.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.085+0000 7fabba06c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc01a10070 0x55fc01dd5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:26.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.086+0000 7fabba06c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fc01b7e700 con 0x55fc019ef390 2024-08-31T06:22:26.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.086+0000 7fabba06c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55fc01ca2cf0 con 0x55fc019e71b0 2024-08-31T06:22:26.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.086+0000 7fabba06c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55fc01dcc920 con 0x55fc01a10070 2024-08-31T06:22:26.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.086+0000 7fabb0a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc019e71b0 0x55fc01dc9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:26.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.086+0000 7fabb0a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc019e71b0 0x55fc01dc9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51552/0 (socket says 172.21.15.16:51552) 2024-08-31T06:22:26.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.086+0000 7fabb0a00640 1 -- 172.21.15.16:0/1527918261 learned_addr learned my addr 172.21.15.16:0/1527918261 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:26.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.086+0000 7fabba06c740 10 client.0 _ll_register_callbacks cb 0x55fc01cc8470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:22:26.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.086+0000 7fabb1400640 1 --2- 172.21.15.16:0/1527918261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc01a10070 0x55fc01dd5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:26.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.086+0000 7fababe00640 1 --2- 172.21.15.16:0/1527918261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc019ef390 0x55fc01dca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:26.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.086+0000 7fabb0a00640 1 -- 172.21.15.16:0/1527918261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc01a10070 msgr2=0x55fc01dd5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:26.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.086+0000 7fabb0a00640 1 --2- 172.21.15.16:0/1527918261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc01a10070 0x55fc01dd5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:26.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.086+0000 7fabb0a00640 1 -- 172.21.15.16:0/1527918261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc019ef390 msgr2=0x55fc01dca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:26.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.086+0000 7fabb0a00640 1 --2- 172.21.15.16:0/1527918261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc019ef390 0x55fc01dca2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:26.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.086+0000 7fabb0a00640 1 -- 172.21.15.16:0/1527918261 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc01ce38b0 con 0x55fc019e71b0 2024-08-31T06:22:26.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.086+0000 7fababe00640 1 --2- 172.21.15.16:0/1527918261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc019ef390 0x55fc01dca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:22:26.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.087+0000 7fabb0a00640 1 --2- 172.21.15.16:0/1527918261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc019e71b0 0x55fc01dc9da0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7faba0033040 tx=0x7faba00037e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7e8e09c5443cc93 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:26.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.087+0000 7fab92000640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:22:26.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.087+0000 7fab92000640 1 -- 172.21.15.16:0/1527918261 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba0041050 con 0x55fc019e71b0 2024-08-31T06:22:26.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.087+0000 7fab92000640 1 -- 172.21.15.16:0/1527918261 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8470+0+0 (secure 0 0 0) 0x7faba0031440 con 0x55fc019e71b0 2024-08-31T06:22:26.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.087+0000 7fabba06c740 1 -- 172.21.15.16:0/1527918261 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faba0004940 con 0x55fc019e71b0 2024-08-31T06:22:26.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.088+0000 7fab92000640 1 -- 172.21.15.16:0/1527918261 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7faba0031640 con 0x55fc019e71b0 2024-08-31T06:22:26.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.088+0000 7fabba06c740 10 client.28949 fetch_fsmap learned FSMap version 52 2024-08-31T06:22:26.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.088+0000 7fabba06c740 1 -- 172.21.15.16:0/1527918261 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55fc01dd8910 con 0x55fc019e71b0 2024-08-31T06:22:26.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.088+0000 7fab92000640 1 -- 172.21.15.16:0/1527918261 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faba0031c90 con 0x55fc019e71b0 2024-08-31T06:22:26.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.088+0000 7fab92000640 1 -- 172.21.15.16:0/1527918261 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 52) ==== 8350+0+0 (secure 0 0 0) 0x7faba004f070 con 0x55fc019e71b0 2024-08-31T06:22:26.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.088+0000 7fabba06c740 10 client.28949 fetch_fsmap finished waiting for FSMap version 52 2024-08-31T06:22:26.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.088+0000 7fabba06c740 10 client.28949 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T06:22:26.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.088+0000 7fabba06c740 20 client.28949 populate_metadata read hostname 'smithi016' 2024-08-31T06:22:26.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.088+0000 7fabba06c740 1 --2- 172.21.15.16:0/1527918261 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55fc019ce9f0 0x55fc01ddbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:26.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.088+0000 7fabba06c740 4 client.28949 mds_command: new command op to 24712 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:22:26.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.088+0000 7fabba06c740 1 -- 172.21.15.16:0/1527918261 --> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7faba0004940 con 0x55fc019ce9f0 2024-08-31T06:22:26.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.089+0000 7fababe00640 1 --2- 172.21.15.16:0/1527918261 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55fc019ce9f0 0x55fc01ddbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:26.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.090+0000 7fababe00640 1 --2- 172.21.15.16:0/1527918261 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55fc019ce9f0 0x55fc01ddbc70 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e1a34b4648bc4dd1 server_cookie=9650ee90dd7b8ea5 in_seq=0 out_seq=0 2024-08-31T06:22:26.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.090+0000 7fab92000640 10 client.28949 ms_handle_connect on v2:172.21.15.184:6838/4011051479 2024-08-31T06:22:26.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.090+0000 7fab92000640 1 -- 172.21.15.16:0/1527918261 <== mds.2 v2:172.21.15.184:6838/4011051479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7faba0004940 con 0x55fc019ce9f0 2024-08-31T06:22:26.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.090+0000 7fab92000640 10 client.28949 handle_command_reply: tid=0 2024-08-31T06:22:26.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.091+0000 7fabba06c740 1 client.28949 shutdown 2024-08-31T06:22:26.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.091+0000 7fabba06c740 1 -- 172.21.15.16:0/1527918261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc019e71b0 msgr2=0x55fc01dc9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:26.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.091+0000 7fabba06c740 1 --2- 172.21.15.16:0/1527918261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc019e71b0 0x55fc01dc9da0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7faba0033040 tx=0x7faba00037e0 comp rx=0 tx=0).stop 2024-08-31T06:22:26.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.091+0000 7fabba06c740 1 -- 172.21.15.16:0/1527918261 shutdown_connections 2024-08-31T06:22:26.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.091+0000 7fabba06c740 1 --2- 172.21.15.16:0/1527918261 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55fc019ce9f0 0x55fc01ddbc70 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:26.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.091+0000 7fab92000640 0 client.28949 ms_handle_reset on v2:172.21.15.184:6838/4011051479 2024-08-31T06:22:26.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.091+0000 7fabba06c740 1 --2- 172.21.15.16:0/1527918261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc01a10070 0x55fc01dd5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:26.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.091+0000 7fabba06c740 1 --2- 172.21.15.16:0/1527918261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc019ef390 0x55fc01dca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:26.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.091+0000 7fabba06c740 1 --2- 172.21.15.16:0/1527918261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc019e71b0 0x55fc01dc9da0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:26.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.091+0000 7fabba06c740 1 -- 172.21.15.16:0/1527918261 >> 172.21.15.16:0/1527918261 conn(0x55fc01a04460 msgr2=0x55fc01db1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:26.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.091+0000 7fabba06c740 1 -- 172.21.15.16:0/1527918261 shutdown_connections 2024-08-31T06:22:26.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.091+0000 7fabba06c740 1 -- 172.21.15.16:0/1527918261 wait complete. 2024-08-31T06:22:26.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.092+0000 7fabba06c740 20 client.28949 trim_cache size 0 max 16384 2024-08-31T06:22:26.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.111+0000 7fabba06c740 1 Processor -- start 2024-08-31T06:22:26.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.111+0000 7fabba06c740 1 -- start start 2024-08-31T06:22:26.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.112+0000 7fabba06c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc019ce9f0 0x55fc01db3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:26.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.112+0000 7fabba06c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc019e71b0 0x55fc01dd5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:26.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.112+0000 7fabba06c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc019ef390 0x55fc01dd6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:26.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.112+0000 7fabba06c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c011030 con 0x55fc019e71b0 2024-08-31T06:22:26.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.112+0000 7fabba06c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c00e8d0 con 0x55fc019ef390 2024-08-31T06:22:26.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.112+0000 7fabba06c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c00c740 con 0x55fc019ce9f0 2024-08-31T06:22:26.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.113+0000 7fabb0a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc019ce9f0 0x55fc01db3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:26.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.113+0000 7fabb1400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc019ef390 0x55fc01dd6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:26.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.113+0000 7fabb1400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc019ef390 0x55fc01dd6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51554/0 (socket says 172.21.15.16:51554) 2024-08-31T06:22:26.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.113+0000 7fabb1400640 1 -- 172.21.15.16:0/2962172861 learned_addr learned my addr 172.21.15.16:0/2962172861 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:26.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.113+0000 7fababe00640 1 --2- 172.21.15.16:0/2962172861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc019e71b0 0x55fc01dd5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:26.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.114+0000 7fabb1400640 1 -- 172.21.15.16:0/2962172861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc019ce9f0 msgr2=0x55fc01db3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:26.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.114+0000 7fabb1400640 1 --2- 172.21.15.16:0/2962172861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc019ce9f0 0x55fc01db3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:26.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.114+0000 7fabb1400640 1 -- 172.21.15.16:0/2962172861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc019e71b0 msgr2=0x55fc01dd5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:26.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.114+0000 7fabb1400640 1 --2- 172.21.15.16:0/2962172861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc019e71b0 0x55fc01dd5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:26.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.114+0000 7fabb1400640 1 -- 172.21.15.16:0/2962172861 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc01dc8b00 con 0x55fc019ef390 2024-08-31T06:22:26.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.114+0000 7fabb0a00640 1 --2- 172.21.15.16:0/2962172861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc019ce9f0 0x55fc01db3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:22:26.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.114+0000 7fabb1400640 1 --2- 172.21.15.16:0/2962172861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc019ef390 0x55fc01dd6160 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7faba40123a0 tx=0x7faba4009ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0446dcd77fbae7e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:26.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.114+0000 7fab9d200640 1 -- 172.21.15.16:0/2962172861 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba4010620 con 0x55fc019ef390 2024-08-31T06:22:26.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.114+0000 7fab9d200640 1 -- 172.21.15.16:0/2962172861 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faba4010c70 con 0x55fc019ef390 2024-08-31T06:22:26.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.115+0000 7fab9d200640 1 -- 172.21.15.16:0/2962172861 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba4021c80 con 0x55fc019ef390 2024-08-31T06:22:26.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.115+0000 7fabba06c740 1 -- 172.21.15.16:0/2962172861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc019ef390 msgr2=0x55fc01dd6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:26.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.116+0000 7fabba06c740 1 --2- 172.21.15.16:0/2962172861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc019ef390 0x55fc01dd6160 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7faba40123a0 tx=0x7faba4009ba0 comp rx=0 tx=0).stop 2024-08-31T06:22:26.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.116+0000 7fabba06c740 1 -- 172.21.15.16:0/2962172861 shutdown_connections 2024-08-31T06:22:26.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.116+0000 7fabba06c740 1 --2- 172.21.15.16:0/2962172861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc019ef390 0x55fc01dd6160 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:26.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.116+0000 7fabba06c740 1 --2- 172.21.15.16:0/2962172861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc019e71b0 0x55fc01dd5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:26.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.116+0000 7fabba06c740 1 --2- 172.21.15.16:0/2962172861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc019ce9f0 0x55fc01db3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:26.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.116+0000 7fabba06c740 1 -- 172.21.15.16:0/2962172861 >> 172.21.15.16:0/2962172861 conn(0x55fc01a04460 msgr2=0x55fc01db3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:26.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.116+0000 7fabba06c740 1 -- 172.21.15.16:0/2962172861 shutdown_connections 2024-08-31T06:22:26.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.116+0000 7fabba06c740 1 -- 172.21.15.16:0/2962172861 wait complete. 2024-08-31T06:22:26.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.117+0000 7fabba06c740 1 -- start start 2024-08-31T06:22:26.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.117+0000 7fabba06c740 1 Processor -- start 2024-08-31T06:22:26.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.118+0000 7fabba06c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc019ce9f0 0x55fc01ddd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:26.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.118+0000 7fabba06c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc019e71b0 0x55fc01ddde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:26.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.118+0000 7fabba06c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc019ef390 0x55fc01e453e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:26.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.118+0000 7fabb0a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc019ce9f0 0x55fc01ddd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:26.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.118+0000 7fabb0a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc019ce9f0 0x55fc01ddd940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:54120/0 (socket says 172.21.15.16:54120) 2024-08-31T06:22:26.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.118+0000 7fabb0a00640 1 -- 172.21.15.16:0/1645986964 learned_addr learned my addr 172.21.15.16:0/1645986964 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:26.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.118+0000 7fabba06c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c00cd90 con 0x55fc019ef390 2024-08-31T06:22:26.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.118+0000 7fabba06c740 1 -- 172.21.15.16:0/1645986964 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c00e200 con 0x55fc019e71b0 2024-08-31T06:22:26.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.118+0000 7fabba06c740 1 -- 172.21.15.16:0/1645986964 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c00d460 con 0x55fc019ce9f0 2024-08-31T06:22:26.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.119+0000 7fabba06c740 10 client.0 _ll_register_callbacks cb 0x7faba0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:22:26.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.119+0000 7fabb0a00640 1 -- 172.21.15.16:0/1645986964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc019e71b0 msgr2=0x55fc01ddde80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T06:22:26.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.119+0000 7fabb0a00640 1 --2- 172.21.15.16:0/1645986964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc019e71b0 0x55fc01ddde80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:26.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.119+0000 7fabb0a00640 1 -- 172.21.15.16:0/1645986964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc019ef390 msgr2=0x55fc01e453e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:22:26.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.120+0000 7fabb1400640 1 --2- 172.21.15.16:0/1645986964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc019ef390 0x55fc01e453e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:26.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.121+0000 7fabb0a00640 1 --2- 172.21.15.16:0/1645986964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc019ef390 0x55fc01e453e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:26.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.121+0000 7fabb0a00640 1 -- 172.21.15.16:0/1645986964 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc01e459a0 con 0x55fc019ce9f0 2024-08-31T06:22:26.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.121+0000 7fabba06c740 1 -- 172.21.15.16:0/1645986964 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faba4010c70 con 0x55fc019ce9f0 2024-08-31T06:22:26.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.121+0000 7fabb1400640 1 --2- 172.21.15.16:0/1645986964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc019ef390 0x55fc01e453e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:22:26.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.124+0000 7fabb0a00640 1 --2- 172.21.15.16:0/1645986964 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc019ce9f0 0x55fc01ddd940 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7faba0033040 tx=0x7faba003de50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=377a07037adf595 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:26.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.124+0000 7fab92000640 10 client.28978 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:22:26.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.124+0000 7fab92000640 1 -- 172.21.15.16:0/1645986964 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba0041040 con 0x55fc019ce9f0 2024-08-31T06:22:26.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.124+0000 7fab92000640 1 -- 172.21.15.16:0/1645986964 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8470+0+0 (secure 0 0 0) 0x7faba0031440 con 0x55fc019ce9f0 2024-08-31T06:22:26.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.124+0000 7fab92000640 1 -- 172.21.15.16:0/1645986964 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7faba0004950 con 0x55fc019ce9f0 2024-08-31T06:22:26.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.125+0000 7fabba06c740 10 client.28978 fetch_fsmap learned FSMap version 52 2024-08-31T06:22:26.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.125+0000 7fabba06c740 1 -- 172.21.15.16:0/1645986964 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({fsmap=52}) -- 0x55fc01e46b20 con 0x55fc019ce9f0 2024-08-31T06:22:26.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.126+0000 7fab92000640 1 -- 172.21.15.16:0/1645986964 <== mon.1 v2:172.21.15.184:3300/0 4 ==== fsmap(e 52) ==== 8350+0+0 (secure 0 0 0) 0x7faba004f040 con 0x55fc019ce9f0 2024-08-31T06:22:26.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.126+0000 7fabba06c740 10 client.28978 fetch_fsmap finished waiting for FSMap version 52 2024-08-31T06:22:26.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.126+0000 7fabba06c740 10 client.28978 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T06:22:26.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.126+0000 7fabba06c740 20 client.28978 populate_metadata read hostname 'smithi016' 2024-08-31T06:22:26.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.126+0000 7fabba06c740 1 --2- 172.21.15.16:0/1645986964 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55fc01a10070 0x55fc01e48f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:26.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.126+0000 7fabba06c740 4 client.28978 mds_command: new command op to 24712 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:22:26.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.126+0000 7fabba06c740 1 -- 172.21.15.16:0/1645986964 --> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fc01d2e700 con 0x55fc01a10070 2024-08-31T06:22:26.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.132+0000 7fababe00640 1 --2- 172.21.15.16:0/1645986964 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55fc01a10070 0x55fc01e48f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:26.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.133+0000 7fababe00640 1 --2- 172.21.15.16:0/1645986964 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55fc01a10070 0x55fc01e48f60 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d5a2e56a1c7efa44 server_cookie=b8a1e77bb844286c in_seq=0 out_seq=0 2024-08-31T06:22:26.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.133+0000 7fab92000640 10 client.28978 ms_handle_connect on v2:172.21.15.184:6838/4011051479 2024-08-31T06:22:26.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.133+0000 7fab92000640 1 -- 172.21.15.16:0/1645986964 <== mds.2 v2:172.21.15.184:6838/4011051479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fc01d2e700 con 0x55fc01a10070 2024-08-31T06:22:26.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.133+0000 7fab92000640 10 client.28978 handle_command_reply: tid=0 2024-08-31T06:22:26.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.134+0000 7fabba06c740 1 client.28978 shutdown 2024-08-31T06:22:26.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.135+0000 7fabba06c740 1 -- 172.21.15.16:0/1645986964 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc019ce9f0 msgr2=0x55fc01ddd940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:26.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.135+0000 7fabba06c740 1 --2- 172.21.15.16:0/1645986964 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc019ce9f0 0x55fc01ddd940 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7faba0033040 tx=0x7faba003de50 comp rx=0 tx=0).stop 2024-08-31T06:22:26.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.135+0000 7fabba06c740 1 -- 172.21.15.16:0/1645986964 shutdown_connections 2024-08-31T06:22:26.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.135+0000 7fabba06c740 1 --2- 172.21.15.16:0/1645986964 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x55fc01a10070 0x55fc01e48f60 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:26.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.135+0000 7fabba06c740 1 --2- 172.21.15.16:0/1645986964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc019ef390 0x55fc01e453e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:26.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.135+0000 7fabba06c740 1 --2- 172.21.15.16:0/1645986964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc019e71b0 0x55fc01ddde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:26.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.135+0000 7fabba06c740 1 --2- 172.21.15.16:0/1645986964 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fc019ce9f0 0x55fc01ddd940 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:26.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.135+0000 7fabba06c740 1 -- 172.21.15.16:0/1645986964 >> 172.21.15.16:0/1645986964 conn(0x55fc01a04460 msgr2=0x55fc01db1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:26.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.135+0000 7fab92000640 0 client.28978 ms_handle_reset on v2:172.21.15.184:6838/4011051479 2024-08-31T06:22:26.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.136+0000 7fabba06c740 1 -- 172.21.15.16:0/1645986964 shutdown_connections 2024-08-31T06:22:26.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.136+0000 7fabba06c740 1 -- 172.21.15.16:0/1645986964 wait complete. 2024-08-31T06:22:26.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.137+0000 7fabba06c740 20 client.28978 trim_cache size 0 max 16384 2024-08-31T06:22:26.140 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:22:26.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.139+0000 7fabb1e00640 1 -- 172.21.15.16:0/2042876762 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fab940b4d20 msgr2=0x7fab940b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:26.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.139+0000 7fabb1e00640 1 --2- 172.21.15.16:0/2042876762 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fab940b4d20 0x7fab940b7230 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7faba4009fd0 tx=0x7faba4004ab0 comp rx=0 tx=0).stop 2024-08-31T06:22:26.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.139+0000 7fabb1e00640 1 -- 172.21.15.16:0/2042876762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fabac10b290 msgr2=0x7fabac19c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:26.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.139+0000 7fabb1e00640 1 --2- 172.21.15.16:0/2042876762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fabac10b290 0x7fabac19c800 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fab9800e9f0 tx=0x7fab9800eeb0 comp rx=0 tx=0).stop 2024-08-31T06:22:26.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.139+0000 7fabb1e00640 1 -- 172.21.15.16:0/2042876762 shutdown_connections 2024-08-31T06:22:26.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.139+0000 7fabb1e00640 1 --2- 172.21.15.16:0/2042876762 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fab940b4d20 0x7fab940b7230 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:26.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.139+0000 7fabb1e00640 1 --2- 172.21.15.16:0/2042876762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fabac10b290 0x7fabac19c800 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:26.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.139+0000 7fabb1e00640 1 --2- 172.21.15.16:0/2042876762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabac10bbe0 0x7fabac1a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:26.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.139+0000 7fabb1e00640 1 --2- 172.21.15.16:0/2042876762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabac105950 0x7fabac19c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:26.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.139+0000 7fabb1e00640 1 -- 172.21.15.16:0/2042876762 >> 172.21.15.16:0/2042876762 conn(0x7fabac1013e0 msgr2=0x7fabac10a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:26.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.142+0000 7fabb1e00640 1 -- 172.21.15.16:0/2042876762 shutdown_connections 2024-08-31T06:22:26.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:26.142+0000 7fabb1e00640 1 -- 172.21.15.16:0/2042876762 wait complete. 2024-08-31T06:22:26.624 INFO:teuthology.orchestra.run.smithi016.stderr:progress: 900.0 s, 1124.9 tps, lat 88.700 ms stddev 133.723 2024-08-31T06:22:26.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:26 smithi016 ceph-mon[17194]: pgmap v840: 97 pgs: 97 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 14 MiB/s wr, 791 op/s 2024-08-31T06:22:26.955 INFO:teuthology.orchestra.run.smithi016.stdout:transaction type: 2024-08-31T06:22:26.955 INFO:teuthology.orchestra.run.smithi016.stdout:scaling factor: 32 2024-08-31T06:22:26.955 INFO:teuthology.orchestra.run.smithi016.stdout:query mode: simple 2024-08-31T06:22:26.955 INFO:teuthology.orchestra.run.smithi016.stdout:number of clients: 100 2024-08-31T06:22:26.955 INFO:teuthology.orchestra.run.smithi016.stdout:number of threads: 4 2024-08-31T06:22:26.955 INFO:teuthology.orchestra.run.smithi016.stdout:duration: 900 s 2024-08-31T06:22:26.955 INFO:teuthology.orchestra.run.smithi016.stdout:number of transactions actually processed: 1144395 2024-08-31T06:22:26.955 INFO:teuthology.orchestra.run.smithi016.stdout:latency average = 78.621 ms 2024-08-31T06:22:26.956 INFO:teuthology.orchestra.run.smithi016.stdout:latency stddev = 767.435 ms 2024-08-31T06:22:26.956 INFO:teuthology.orchestra.run.smithi016.stdout:tps = 1271.082742 (including connections establishing) 2024-08-31T06:22:26.956 INFO:teuthology.orchestra.run.smithi016.stdout:tps = 1271.093323 (excluding connections establishing) 2024-08-31T06:22:26.963 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2024-08-31T06:22:27.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:26 smithi155 ceph-mon[21742]: pgmap v840: 97 pgs: 97 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 14 MiB/s wr, 791 op/s 2024-08-31T06:22:27.166 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T06:22:27.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:26 smithi184 ceph-mon[22331]: pgmap v840: 97 pgs: 97 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 14 MiB/s wr, 791 op/s 2024-08-31T06:22:27.930 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:22:28.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.801+0000 7fbb03200640 1 -- 172.21.15.16:0/1954715180 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb0410d9c0 msgr2=0x7fbb0410feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:28.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.802+0000 7fbb03200640 1 --2- 172.21.15.16:0/1954715180 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb0410d9c0 0x7fbb0410feb0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0009900 tx=0x7fbaf002a410 comp rx=0 tx=0).stop 2024-08-31T06:22:28.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.803+0000 7fbb03200640 1 -- 172.21.15.16:0/1954715180 shutdown_connections 2024-08-31T06:22:28.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.803+0000 7fbb03200640 1 --2- 172.21.15.16:0/1954715180 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb0410d9c0 0x7fbb0410feb0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:28.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.803+0000 7fbb03200640 1 --2- 172.21.15.16:0/1954715180 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbb04104eb0 0x7fbb0410d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:28.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.803+0000 7fbb03200640 1 --2- 172.21.15.16:0/1954715180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb04104560 0x7fbb04104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:28.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.804+0000 7fbb03200640 1 -- 172.21.15.16:0/1954715180 >> 172.21.15.16:0/1954715180 conn(0x7fbb040fe060 msgr2=0x7fbb041004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:28.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.804+0000 7fbb03200640 1 -- 172.21.15.16:0/1954715180 shutdown_connections 2024-08-31T06:22:28.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.804+0000 7fbb03200640 1 -- 172.21.15.16:0/1954715180 wait complete. 2024-08-31T06:22:28.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.806+0000 7fbb03200640 1 Processor -- start 2024-08-31T06:22:28.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.807+0000 7fbb03200640 1 -- start start 2024-08-31T06:22:28.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.808+0000 7fbb03200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbb04104560 0x7fbb0419d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:28.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.808+0000 7fbb03200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb04104eb0 0x7fbb0419dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:28.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.809+0000 7fbb03200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb0410d9c0 0x7fbb041a0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:28.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.809+0000 7fbb01e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbb04104560 0x7fbb0419d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:28.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.809+0000 7fbb02800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb0410d9c0 0x7fbb041a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:28.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.809+0000 7fbb01e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbb04104560 0x7fbb0419d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51578/0 (socket says 172.21.15.16:51578) 2024-08-31T06:22:28.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.810+0000 7fbb01400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb04104eb0 0x7fbb0419dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:28.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.810+0000 7fbb01e00640 1 -- 172.21.15.16:0/1776761721 learned_addr learned my addr 172.21.15.16:0/1776761721 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:28.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.810+0000 7fbb03200640 1 -- 172.21.15.16:0/1776761721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbb041136a0 con 0x7fbb0410d9c0 2024-08-31T06:22:28.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.810+0000 7fbb03200640 1 -- 172.21.15.16:0/1776761721 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbb04113520 con 0x7fbb04104560 2024-08-31T06:22:28.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.810+0000 7fbb03200640 1 -- 172.21.15.16:0/1776761721 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbb04113820 con 0x7fbb04104eb0 2024-08-31T06:22:28.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.811+0000 7fbb01400640 1 -- 172.21.15.16:0/1776761721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbb04104560 msgr2=0x7fbb0419d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:28.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.811+0000 7fbb01400640 1 --2- 172.21.15.16:0/1776761721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbb04104560 0x7fbb0419d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:28.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.811+0000 7fbb01400640 1 -- 172.21.15.16:0/1776761721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb0410d9c0 msgr2=0x7fbb041a0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:28.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.811+0000 7fbb01400640 1 --2- 172.21.15.16:0/1776761721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb0410d9c0 0x7fbb041a0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:28.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.811+0000 7fbb01400640 1 -- 172.21.15.16:0/1776761721 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb041a0d20 con 0x7fbb04104eb0 2024-08-31T06:22:28.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.812+0000 7fbb01400640 1 --2- 172.21.15.16:0/1776761721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb04104eb0 0x7fbb0419dcb0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fbb0406a5a0 tx=0x7fbafc00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:28.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.813+0000 7fbafaa00640 1 -- 172.21.15.16:0/1776761721 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc00cd50 con 0x7fbb04104eb0 2024-08-31T06:22:28.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.814+0000 7fbb03200640 1 -- 172.21.15.16:0/1776761721 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb041a0fb0 con 0x7fbb04104eb0 2024-08-31T06:22:28.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.814+0000 7fbb03200640 1 -- 172.21.15.16:0/1776761721 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb0410c9e0 con 0x7fbb04104eb0 2024-08-31T06:22:28.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.815+0000 7fbafaa00640 1 -- 172.21.15.16:0/1776761721 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbafc005020 con 0x7fbb04104eb0 2024-08-31T06:22:28.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.815+0000 7fbafaa00640 1 -- 172.21.15.16:0/1776761721 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc0116d0 con 0x7fbb04104eb0 2024-08-31T06:22:28.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.835+0000 7fbb0fa86740 1 Processor -- start 2024-08-31T06:22:28.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.836+0000 7fbb0fa86740 1 -- start start 2024-08-31T06:22:28.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.836+0000 7fbb0fa86740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625e1168070 0x5625e150db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:28.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.838+0000 7fbb0fa86740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625e113f1b0 0x5625e150e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:28.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.838+0000 7fbafaa00640 1 -- 172.21.15.16:0/1776761721 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fbafc004a40 con 0x7fbb04104eb0 2024-08-31T06:22:28.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.838+0000 7fbb0fa86740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5625e1147390 0x5625e150e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:28.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.838+0000 7fbb0fa86740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5625e14847b0 con 0x5625e1168070 2024-08-31T06:22:28.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.838+0000 7fbb0fa86740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5625e1473d40 con 0x5625e1147390 2024-08-31T06:22:28.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.838+0000 7fbb0fa86740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5625e1477cf0 con 0x5625e113f1b0 2024-08-31T06:22:28.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.838+0000 7fbb02800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625e1168070 0x5625e150db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:28.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.838+0000 7fbb02800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625e1168070 0x5625e150db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45460/0 (socket says 172.21.15.16:45460) 2024-08-31T06:22:28.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.838+0000 7fbb02800640 1 -- 172.21.15.16:0/530270654 learned_addr learned my addr 172.21.15.16:0/530270654 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:28.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.839+0000 7fbb01e00640 1 --2- 172.21.15.16:0/530270654 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625e113f1b0 0x5625e150e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:28.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.840+0000 7fbb01e00640 1 -- 172.21.15.16:0/530270654 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5625e1147390 msgr2=0x5625e150e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:28.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.840+0000 7fbb01e00640 1 --2- 172.21.15.16:0/530270654 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5625e1147390 0x5625e150e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:28.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.840+0000 7fbb01e00640 1 -- 172.21.15.16:0/530270654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625e1168070 msgr2=0x5625e150db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:28.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.840+0000 7fbb01e00640 1 --2- 172.21.15.16:0/530270654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625e1168070 0x5625e150db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:28.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.840+0000 7fbb01e00640 1 -- 172.21.15.16:0/530270654 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625e11d3650 con 0x5625e113f1b0 2024-08-31T06:22:28.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.840+0000 7fbb01e00640 1 --2- 172.21.15.16:0/530270654 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625e113f1b0 0x5625e150e0a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4009c00 tx=0x7fbaf40117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad909c7f2be4f8ef server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:28.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.841+0000 7fbaef400640 1 -- 172.21.15.16:0/530270654 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4004020 con 0x5625e113f1b0 2024-08-31T06:22:28.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.841+0000 7fbaef400640 1 -- 172.21.15.16:0/530270654 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbaf40041c0 con 0x5625e113f1b0 2024-08-31T06:22:28.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.841+0000 7fbaef400640 1 -- 172.21.15.16:0/530270654 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4012780 con 0x5625e113f1b0 2024-08-31T06:22:28.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.845+0000 7fbb0fa86740 1 -- 172.21.15.16:0/530270654 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625e113f1b0 msgr2=0x5625e150e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:28.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.845+0000 7fbb0fa86740 1 --2- 172.21.15.16:0/530270654 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625e113f1b0 0x5625e150e0a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4009c00 tx=0x7fbaf40117b0 comp rx=0 tx=0).stop 2024-08-31T06:22:28.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.845+0000 7fbafaa00640 1 --2- 172.21.15.16:0/1776761721 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbae80b4d20 0x7fbae80b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:28.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.846+0000 7fbafaa00640 1 -- 172.21.15.16:0/1776761721 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8470+0+0 (secure 0 0 0) 0x7fbafc0e5890 con 0x7fbb04104eb0 2024-08-31T06:22:28.848 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:28 smithi016 ceph-mon[17194]: pgmap v841: 97 pgs: 97 active+clean; 3.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 12 MiB/s wr, 687 op/s 2024-08-31T06:22:28.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.848+0000 7fbb0fa86740 1 -- 172.21.15.16:0/530270654 shutdown_connections 2024-08-31T06:22:28.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.848+0000 7fbb0fa86740 1 --2- 172.21.15.16:0/530270654 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625e113f1b0 0x5625e150e0a0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:28.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.848+0000 7fbb0fa86740 1 --2- 172.21.15.16:0/530270654 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5625e1147390 0x5625e150e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:28.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.848+0000 7fbb0fa86740 1 --2- 172.21.15.16:0/530270654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625e1168070 0x5625e150db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:28.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.848+0000 7fbb0fa86740 1 -- 172.21.15.16:0/530270654 >> 172.21.15.16:0/530270654 conn(0x5625e115c460 msgr2=0x5625e150ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:28.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.849+0000 7fbb0fa86740 1 -- 172.21.15.16:0/530270654 shutdown_connections 2024-08-31T06:22:28.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.849+0000 7fbb0fa86740 1 -- 172.21.15.16:0/530270654 wait complete. 2024-08-31T06:22:28.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.850+0000 7fbb0fa86740 1 -- start start 2024-08-31T06:22:28.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.850+0000 7fbb0fa86740 1 Processor -- start 2024-08-31T06:22:28.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.851+0000 7fbb02800640 1 --2- 172.21.15.16:0/1776761721 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbae80b4d20 0x7fbae80b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:28.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.852+0000 7fbb02800640 1 --2- 172.21.15.16:0/1776761721 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbae80b4d20 0x7fbae80b7230 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fbaf00096c0 tx=0x7fbaf000b040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:28.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.853+0000 7fbb0fa86740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625e113f1b0 0x5625e1519780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:28.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.854+0000 7fbb0fa86740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5625e1147390 0x5625e1519cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:28.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.854+0000 7fbb0fa86740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625e1168070 0x5625e15256c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:28.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.855+0000 7fbb01400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5625e1147390 0x5625e1519cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:28.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.855+0000 7fbb01400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5625e1147390 0x5625e1519cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51610/0 (socket says 172.21.15.16:51610) 2024-08-31T06:22:28.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.855+0000 7fbb0fa86740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5625e12d6700 con 0x5625e113f1b0 2024-08-31T06:22:28.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.855+0000 7fbb0fa86740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5625e13facf0 con 0x5625e1147390 2024-08-31T06:22:28.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.855+0000 7fbb0fa86740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5625e151c340 con 0x5625e1168070 2024-08-31T06:22:28.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.855+0000 7fbb02800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625e1168070 0x5625e15256c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:28.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.855+0000 7fbb01400640 1 -- 172.21.15.16:0/3839199275 learned_addr learned my addr 172.21.15.16:0/3839199275 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:28.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.855+0000 7fbb01e00640 1 --2- 172.21.15.16:0/3839199275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625e113f1b0 0x5625e1519780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:28.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.855+0000 7fbb0fa86740 10 client.0 _ll_register_callbacks cb 0x5625e1420470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:22:28.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.856+0000 7fbb01400640 1 -- 172.21.15.16:0/3839199275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625e1168070 msgr2=0x5625e15256c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:28.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.857+0000 7fbb01400640 1 --2- 172.21.15.16:0/3839199275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625e1168070 0x5625e15256c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:28.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.857+0000 7fbb01400640 1 -- 172.21.15.16:0/3839199275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625e113f1b0 msgr2=0x5625e1519780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:28.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.857+0000 7fbb01400640 1 --2- 172.21.15.16:0/3839199275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625e113f1b0 0x5625e1519780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:28.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.857+0000 7fbb01400640 1 -- 172.21.15.16:0/3839199275 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625e143b8b0 con 0x5625e1147390 2024-08-31T06:22:28.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.857+0000 7fbb01e00640 1 --2- 172.21.15.16:0/3839199275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625e113f1b0 0x5625e1519780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:22:28.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.857+0000 7fbb02800640 1 --2- 172.21.15.16:0/3839199275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625e1168070 0x5625e15256c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:22:28.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.858+0000 7fbb01400640 1 --2- 172.21.15.16:0/3839199275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5625e1147390 0x5625e1519cc0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fbafc009fd0 tx=0x7fbafc0aba60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b92d129d17908384 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:28.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.858+0000 7fbae3e00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:22:28.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.861+0000 7fbae3e00640 1 -- 172.21.15.16:0/3839199275 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc01f040 con 0x5625e1147390 2024-08-31T06:22:28.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.861+0000 7fbae3e00640 1 -- 172.21.15.16:0/3839199275 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8470+0+0 (secure 0 0 0) 0x7fbafc0b03d0 con 0x5625e1147390 2024-08-31T06:22:28.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.862+0000 7fbb0fa86740 1 -- 172.21.15.16:0/3839199275 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbaf40041c0 con 0x5625e1147390 2024-08-31T06:22:28.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.862+0000 7fbae3e00640 1 -- 172.21.15.16:0/3839199275 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fbafc0abf50 con 0x5625e1147390 2024-08-31T06:22:28.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.863+0000 7fbb0fa86740 10 client.28961 fetch_fsmap learned FSMap version 53 2024-08-31T06:22:28.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.863+0000 7fbb0fa86740 1 -- 172.21.15.16:0/3839199275 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x5625e1528400 con 0x5625e1147390 2024-08-31T06:22:28.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.864+0000 7fbae3e00640 1 -- 172.21.15.16:0/3839199275 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbaf40041c0 con 0x5625e1147390 2024-08-31T06:22:28.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.864+0000 7fbae3e00640 1 -- 172.21.15.16:0/3839199275 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 53) ==== 8346+0+0 (secure 0 0 0) 0x7fbafc00f040 con 0x5625e1147390 2024-08-31T06:22:28.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.864+0000 7fbb0fa86740 10 client.28961 fetch_fsmap finished waiting for FSMap version 53 2024-08-31T06:22:28.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.864+0000 7fbb0fa86740 10 client.28961 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-31T06:22:28.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.864+0000 7fbb0fa86740 20 client.28961 populate_metadata read hostname 'smithi016' 2024-08-31T06:22:28.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.864+0000 7fbb0fa86740 1 --2- 172.21.15.16:0/3839199275 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x5625e11269f0 0x5625e152b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:28.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.864+0000 7fbb0fa86740 4 client.28961 mds_command: new command op to 24724 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:22:28.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.864+0000 7fbb0fa86740 1 -- 172.21.15.16:0/3839199275 --> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625e1486700 con 0x5625e11269f0 2024-08-31T06:22:28.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.866+0000 7fbb01e00640 1 --2- 172.21.15.16:0/3839199275 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x5625e11269f0 0x5625e152b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:28.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.871+0000 7fbb01e00640 1 --2- 172.21.15.16:0/3839199275 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x5625e11269f0 0x5625e152b970 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d87338e1876e093 server_cookie=e996578d74467924 in_seq=0 out_seq=0 2024-08-31T06:22:28.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.871+0000 7fbae3e00640 10 client.28961 ms_handle_connect on v2:172.21.15.16:6838/219664460 2024-08-31T06:22:28.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.872+0000 7fbae3e00640 1 -- 172.21.15.16:0/3839199275 <== mds.1 v2:172.21.15.16:6838/219664460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5625e1486700 con 0x5625e11269f0 2024-08-31T06:22:28.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.872+0000 7fbae3e00640 10 client.28961 handle_command_reply: tid=0 2024-08-31T06:22:28.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.873+0000 7fbb0fa86740 1 client.28961 shutdown 2024-08-31T06:22:28.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.873+0000 7fbb0fa86740 1 -- 172.21.15.16:0/3839199275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5625e1147390 msgr2=0x5625e1519cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:28.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.873+0000 7fbb0fa86740 1 --2- 172.21.15.16:0/3839199275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5625e1147390 0x5625e1519cc0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fbafc009fd0 tx=0x7fbafc0aba60 comp rx=0 tx=0).stop 2024-08-31T06:22:28.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.873+0000 7fbb0fa86740 1 -- 172.21.15.16:0/3839199275 shutdown_connections 2024-08-31T06:22:28.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.873+0000 7fbb0fa86740 1 --2- 172.21.15.16:0/3839199275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625e1168070 0x5625e15256c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:28.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.873+0000 7fbb0fa86740 1 --2- 172.21.15.16:0/3839199275 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x5625e11269f0 0x5625e152b970 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:28.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.873+0000 7fbb0fa86740 1 --2- 172.21.15.16:0/3839199275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5625e1147390 0x5625e1519cc0 secure :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fbafc009fd0 tx=0x7fbafc0aba60 comp rx=0 tx=0).stop 2024-08-31T06:22:28.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.873+0000 7fbb0fa86740 1 --2- 172.21.15.16:0/3839199275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625e113f1b0 0x5625e1519780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:28.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.873+0000 7fbb0fa86740 1 -- 172.21.15.16:0/3839199275 >> 172.21.15.16:0/3839199275 conn(0x5625e115c460 msgr2=0x5625e1510cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:28.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.875+0000 7fbae3e00640 0 client.28961 ms_handle_reset on v2:172.21.15.16:6838/219664460 2024-08-31T06:22:28.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.875+0000 7fbb0fa86740 1 -- 172.21.15.16:0/3839199275 shutdown_connections 2024-08-31T06:22:28.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.875+0000 7fbb0fa86740 1 -- 172.21.15.16:0/3839199275 wait complete. 2024-08-31T06:22:28.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.875+0000 7fbb0fa86740 20 client.28961 trim_cache size 0 max 16384 2024-08-31T06:22:28.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.908+0000 7fbb0fa86740 1 Processor -- start 2024-08-31T06:22:28.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.908+0000 7fbb0fa86740 1 -- start start 2024-08-31T06:22:28.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.908+0000 7fbb0fa86740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625e113f1b0 0x5625e15255c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:28.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.909+0000 7fbb0fa86740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5625e1168070 0x5625e1517e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:28.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.909+0000 7fbb0fa86740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625e10fbda0 0x5625e1518360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:28.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.909+0000 7fbb0fa86740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbae4010f20 con 0x5625e10fbda0 2024-08-31T06:22:28.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.909+0000 7fbb0fa86740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbae400e7f0 con 0x5625e1168070 2024-08-31T06:22:28.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.909+0000 7fbb0fa86740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbae400c660 con 0x5625e113f1b0 2024-08-31T06:22:28.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.909+0000 7fbb01e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625e113f1b0 0x5625e15255c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:28.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.909+0000 7fbb01e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625e113f1b0 0x5625e15255c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:54178/0 (socket says 172.21.15.16:54178) 2024-08-31T06:22:28.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.909+0000 7fbb01e00640 1 -- 172.21.15.16:0/1905209465 learned_addr learned my addr 172.21.15.16:0/1905209465 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:28.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.909+0000 7fbb01e00640 1 -- 172.21.15.16:0/1905209465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5625e1168070 msgr2=0x5625e1517e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:22:28.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.909+0000 7fbb01e00640 1 --2- 172.21.15.16:0/1905209465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5625e1168070 0x5625e1517e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:28.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.909+0000 7fbb01e00640 1 -- 172.21.15.16:0/1905209465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625e10fbda0 msgr2=0x5625e1518360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:22:28.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.909+0000 7fbb01e00640 1 --2- 172.21.15.16:0/1905209465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625e10fbda0 0x5625e1518360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:28.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.909+0000 7fbb01e00640 1 -- 172.21.15.16:0/1905209465 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625e151a620 con 0x5625e113f1b0 2024-08-31T06:22:28.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.910+0000 7fbb01e00640 1 --2- 172.21.15.16:0/1905209465 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625e113f1b0 0x5625e15255c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x5625e150c7e0 tx=0x7fbaf4003ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d4ed107cdc3bcd server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:28.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.910+0000 7fbaeea00640 1 -- 172.21.15.16:0/1905209465 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4032070 con 0x5625e113f1b0 2024-08-31T06:22:28.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.910+0000 7fbaeea00640 1 -- 172.21.15.16:0/1905209465 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbaf4009510 con 0x5625e113f1b0 2024-08-31T06:22:28.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.911+0000 7fbaeea00640 1 -- 172.21.15.16:0/1905209465 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf401f040 con 0x5625e113f1b0 2024-08-31T06:22:28.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.912+0000 7fbb0fa86740 1 -- 172.21.15.16:0/1905209465 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625e113f1b0 msgr2=0x5625e15255c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:28.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.912+0000 7fbb0fa86740 1 --2- 172.21.15.16:0/1905209465 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625e113f1b0 0x5625e15255c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x5625e150c7e0 tx=0x7fbaf4003ec0 comp rx=0 tx=0).stop 2024-08-31T06:22:28.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.912+0000 7fbb0fa86740 1 -- 172.21.15.16:0/1905209465 shutdown_connections 2024-08-31T06:22:28.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.912+0000 7fbb0fa86740 1 --2- 172.21.15.16:0/1905209465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625e10fbda0 0x5625e1518360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:28.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.912+0000 7fbb0fa86740 1 --2- 172.21.15.16:0/1905209465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5625e1168070 0x5625e1517e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:28.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.912+0000 7fbb0fa86740 1 --2- 172.21.15.16:0/1905209465 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625e113f1b0 0x5625e15255c0 secure :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0x5625e150c7e0 tx=0x7fbaf4003ec0 comp rx=0 tx=0).stop 2024-08-31T06:22:28.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.912+0000 7fbb0fa86740 1 -- 172.21.15.16:0/1905209465 >> 172.21.15.16:0/1905209465 conn(0x5625e115c460 msgr2=0x5625e15146e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:28.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.913+0000 7fbb0fa86740 1 -- 172.21.15.16:0/1905209465 shutdown_connections 2024-08-31T06:22:28.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.913+0000 7fbb0fa86740 1 -- 172.21.15.16:0/1905209465 wait complete. 2024-08-31T06:22:28.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.915+0000 7fbb0fa86740 1 -- start start 2024-08-31T06:22:28.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.915+0000 7fbb0fa86740 1 Processor -- start 2024-08-31T06:22:28.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.916+0000 7fbb0fa86740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5625e10fbda0 0x5625e151d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:28.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.916+0000 7fbb0fa86740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625e1168070 0x5625e151dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:28.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.916+0000 7fbb0fa86740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625e151e210 0x5625e1595370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:28.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.916+0000 7fbb0fa86740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbae400ccb0 con 0x5625e1168070 2024-08-31T06:22:28.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.916+0000 7fbb0fa86740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbae400e120 con 0x5625e10fbda0 2024-08-31T06:22:28.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.916+0000 7fbb0fa86740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbae400d380 con 0x5625e151e210 2024-08-31T06:22:28.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.916+0000 7fbb0fa86740 10 client.0 _ll_register_callbacks cb 0x7fbafc0abdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:22:28.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.917+0000 7fbb02800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625e151e210 0x5625e1595370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:28.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.917+0000 7fbb02800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625e151e210 0x5625e1595370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:54184/0 (socket says 172.21.15.16:54184) 2024-08-31T06:22:28.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.917+0000 7fbb02800640 1 -- 172.21.15.16:0/154355485 learned_addr learned my addr 172.21.15.16:0/154355485 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:28.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.917+0000 7fbb01e00640 1 --2- 172.21.15.16:0/154355485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5625e10fbda0 0x5625e151d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:28.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.918+0000 7fbb02800640 1 -- 172.21.15.16:0/154355485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5625e10fbda0 msgr2=0x5625e151d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:28.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.918+0000 7fbb02800640 1 --2- 172.21.15.16:0/154355485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5625e10fbda0 0x5625e151d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:28.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.918+0000 7fbb02800640 1 -- 172.21.15.16:0/154355485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625e1168070 msgr2=0x5625e151dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:28.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.918+0000 7fbb02800640 1 --2- 172.21.15.16:0/154355485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625e1168070 0x5625e151dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:28.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.918+0000 7fbb02800640 1 -- 172.21.15.16:0/154355485 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625e1595930 con 0x5625e151e210 2024-08-31T06:22:28.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.918+0000 7fbb02800640 1 --2- 172.21.15.16:0/154355485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625e151e210 0x5625e1595370 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0033000 tx=0x7fbaf002a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a58231f777404c0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:28.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.919+0000 7fbae3e00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:22:28.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.919+0000 7fbae3e00640 1 -- 172.21.15.16:0/154355485 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf0035040 con 0x5625e151e210 2024-08-31T06:22:28.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.920+0000 7fbae3e00640 1 -- 172.21.15.16:0/154355485 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8470+0+0 (secure 0 0 0) 0x7fbaf0004300 con 0x5625e151e210 2024-08-31T06:22:28.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.920+0000 7fbb0fa86740 1 -- 172.21.15.16:0/154355485 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbaf4009510 con 0x5625e151e210 2024-08-31T06:22:28.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.920+0000 7fbae3e00640 1 -- 172.21.15.16:0/154355485 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fbaf0005030 con 0x5625e151e210 2024-08-31T06:22:28.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.921+0000 7fbb0fa86740 10 client.28996 fetch_fsmap learned FSMap version 53 2024-08-31T06:22:28.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.921+0000 7fbb0fa86740 1 -- 172.21.15.16:0/154355485 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x5625e1596bc0 con 0x5625e151e210 2024-08-31T06:22:28.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.921+0000 7fbae3e00640 1 -- 172.21.15.16:0/154355485 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbaf4009510 con 0x5625e151e210 2024-08-31T06:22:28.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.922+0000 7fbae3e00640 1 -- 172.21.15.16:0/154355485 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 53) ==== 8346+0+0 (secure 0 0 0) 0x7fbaf0044050 con 0x5625e151e210 2024-08-31T06:22:28.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.922+0000 7fbb0fa86740 10 client.28996 fetch_fsmap finished waiting for FSMap version 53 2024-08-31T06:22:28.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.922+0000 7fbb0fa86740 10 client.28996 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-31T06:22:28.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.922+0000 7fbb0fa86740 20 client.28996 populate_metadata read hostname 'smithi016' 2024-08-31T06:22:28.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.923+0000 7fbb0fa86740 1 --2- 172.21.15.16:0/154355485 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x5625e1597300 0x5625e1599720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:28.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.923+0000 7fbb0fa86740 4 client.28996 mds_command: new command op to 24724 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:22:28.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.923+0000 7fbb0fa86740 1 -- 172.21.15.16:0/154355485 --> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5625e14821a0 con 0x5625e1597300 2024-08-31T06:22:28.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.925+0000 7fbb01e00640 1 --2- 172.21.15.16:0/154355485 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x5625e1597300 0x5625e1599720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:28.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.926+0000 7fbb01e00640 1 --2- 172.21.15.16:0/154355485 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x5625e1597300 0x5625e1599720 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ee9283c9efce3d3e server_cookie=2288269ba71d622b in_seq=0 out_seq=0 2024-08-31T06:22:28.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.926+0000 7fbae3e00640 10 client.28996 ms_handle_connect on v2:172.21.15.16:6838/219664460 2024-08-31T06:22:28.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.926+0000 7fbae3e00640 1 -- 172.21.15.16:0/154355485 <== mds.1 v2:172.21.15.16:6838/219664460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5625e14821a0 con 0x5625e1597300 2024-08-31T06:22:28.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.926+0000 7fbae3e00640 10 client.28996 handle_command_reply: tid=0 2024-08-31T06:22:28.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.926+0000 7fbb0fa86740 1 client.28996 shutdown 2024-08-31T06:22:28.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.927+0000 7fbb0fa86740 1 -- 172.21.15.16:0/154355485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625e151e210 msgr2=0x5625e1595370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:28.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.927+0000 7fbb0fa86740 1 --2- 172.21.15.16:0/154355485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625e151e210 0x5625e1595370 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0033000 tx=0x7fbaf002a410 comp rx=0 tx=0).stop 2024-08-31T06:22:28.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.927+0000 7fbb0fa86740 1 -- 172.21.15.16:0/154355485 shutdown_connections 2024-08-31T06:22:28.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.927+0000 7fbb0fa86740 1 --2- 172.21.15.16:0/154355485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5625e151e210 0x5625e1595370 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:28.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.927+0000 7fbb0fa86740 1 --2- 172.21.15.16:0/154355485 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x5625e1597300 0x5625e1599720 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:28.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.927+0000 7fbae3e00640 0 client.28996 ms_handle_reset on v2:172.21.15.16:6838/219664460 2024-08-31T06:22:28.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.927+0000 7fbb0fa86740 1 --2- 172.21.15.16:0/154355485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625e1168070 0x5625e151dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:28.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.927+0000 7fbb0fa86740 1 --2- 172.21.15.16:0/154355485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5625e10fbda0 0x5625e151d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:28.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.927+0000 7fbb0fa86740 1 -- 172.21.15.16:0/154355485 >> 172.21.15.16:0/154355485 conn(0x5625e115c460 msgr2=0x5625e1512c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:28.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.928+0000 7fbb0fa86740 1 -- 172.21.15.16:0/154355485 shutdown_connections 2024-08-31T06:22:28.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.928+0000 7fbb0fa86740 1 -- 172.21.15.16:0/154355485 wait complete. 2024-08-31T06:22:28.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.928+0000 7fbb0fa86740 20 client.28996 trim_cache size 0 max 16384 2024-08-31T06:22:28.931 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:22:28.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.930+0000 7fbb03200640 1 -- 172.21.15.16:0/1776761721 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbae80b4d20 msgr2=0x7fbae80b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:28.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.930+0000 7fbb03200640 1 --2- 172.21.15.16:0/1776761721 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbae80b4d20 0x7fbae80b7230 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fbaf00096c0 tx=0x7fbaf000b040 comp rx=0 tx=0).stop 2024-08-31T06:22:28.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.930+0000 7fbb03200640 1 -- 172.21.15.16:0/1776761721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb04104eb0 msgr2=0x7fbb0419dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:28.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.930+0000 7fbb03200640 1 --2- 172.21.15.16:0/1776761721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb04104eb0 0x7fbb0419dcb0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fbb0406a5a0 tx=0x7fbafc00ee80 comp rx=0 tx=0).stop 2024-08-31T06:22:28.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.930+0000 7fbb03200640 1 -- 172.21.15.16:0/1776761721 shutdown_connections 2024-08-31T06:22:28.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.930+0000 7fbb03200640 1 --2- 172.21.15.16:0/1776761721 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fbae80b4d20 0x7fbae80b7230 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:28.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.930+0000 7fbb03200640 1 --2- 172.21.15.16:0/1776761721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb04104eb0 0x7fbb0419dcb0 secure :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fbb0406a5a0 tx=0x7fbafc00ee80 comp rx=0 tx=0).stop 2024-08-31T06:22:28.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.930+0000 7fbb03200640 1 --2- 172.21.15.16:0/1776761721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb0410d9c0 0x7fbb041a0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:28.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.930+0000 7fbb03200640 1 --2- 172.21.15.16:0/1776761721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbb04104560 0x7fbb0419d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:28.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.930+0000 7fbb03200640 1 -- 172.21.15.16:0/1776761721 >> 172.21.15.16:0/1776761721 conn(0x7fbb040fe060 msgr2=0x7fbb04102180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:28.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.930+0000 7fbb03200640 1 -- 172.21.15.16:0/1776761721 shutdown_connections 2024-08-31T06:22:28.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:28.930+0000 7fbb03200640 1 -- 172.21.15.16:0/1776761721 wait complete. 2024-08-31T06:22:29.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:28 smithi155 ceph-mon[21742]: pgmap v841: 97 pgs: 97 active+clean; 3.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 12 MiB/s wr, 687 op/s 2024-08-31T06:22:29.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:28 smithi184 ceph-mon[22331]: pgmap v841: 97 pgs: 97 active+clean; 3.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 12 MiB/s wr, 687 op/s 2024-08-31T06:22:29.777 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T06:22:29.987 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:29 smithi016 ceph-mon[17194]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:22:30.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:29 smithi155 ceph-mon[21742]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:22:30.193 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:22:30.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:29 smithi184 ceph-mon[22331]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:22:30.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:30 smithi016 ceph-mon[17194]: pgmap v842: 97 pgs: 97 active+clean; 3.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 15 MiB/s wr, 950 op/s 2024-08-31T06:22:31.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:30 smithi155 ceph-mon[21742]: pgmap v842: 97 pgs: 97 active+clean; 3.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 15 MiB/s wr, 950 op/s 2024-08-31T06:22:31.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:30 smithi184 ceph-mon[22331]: pgmap v842: 97 pgs: 97 active+clean; 3.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 15 MiB/s wr, 950 op/s 2024-08-31T06:22:33.119 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:32 smithi016 ceph-mon[17194]: pgmap v843: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 19 MiB/s wr, 1.37k op/s 2024-08-31T06:22:33.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:32 smithi155 ceph-mon[21742]: pgmap v843: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 19 MiB/s wr, 1.37k op/s 2024-08-31T06:22:33.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:32 smithi184 ceph-mon[22331]: pgmap v843: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 19 MiB/s wr, 1.37k op/s 2024-08-31T06:22:33.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.443+0000 7f055ee00640 1 -- 172.21.15.16:0/249847343 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f055810d9c0 msgr2=0x7f055810feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:33.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.443+0000 7f055ee00640 1 --2- 172.21.15.16:0/249847343 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f055810d9c0 0x7f055810feb0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f0550009ca0 tx=0x7f055002a8f0 comp rx=0 tx=0).stop 2024-08-31T06:22:33.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.443+0000 7f055ee00640 1 -- 172.21.15.16:0/249847343 shutdown_connections 2024-08-31T06:22:33.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.443+0000 7f055ee00640 1 --2- 172.21.15.16:0/249847343 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f055810d9c0 0x7f055810feb0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:33.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.443+0000 7f055ee00640 1 --2- 172.21.15.16:0/249847343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0558104eb0 0x7f055810d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:33.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.443+0000 7f055ee00640 1 --2- 172.21.15.16:0/249847343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0558104560 0x7f0558104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:33.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.443+0000 7f055ee00640 1 -- 172.21.15.16:0/249847343 >> 172.21.15.16:0/249847343 conn(0x7f05580fe060 msgr2=0x7f05581004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:33.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.444+0000 7f055ee00640 1 -- 172.21.15.16:0/249847343 shutdown_connections 2024-08-31T06:22:33.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.444+0000 7f055ee00640 1 -- 172.21.15.16:0/249847343 wait complete. 2024-08-31T06:22:33.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.448+0000 7f055ee00640 1 Processor -- start 2024-08-31T06:22:33.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.449+0000 7f055ee00640 1 -- start start 2024-08-31T06:22:33.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.450+0000 7f055ee00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0558104560 0x7f05581091e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:33.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.451+0000 7f055ee00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0558104eb0 0x7f05581054c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:33.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.451+0000 7f055da00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0558104560 0x7f05581091e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:33.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.451+0000 7f055da00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0558104560 0x7f05581091e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51628/0 (socket says 172.21.15.16:51628) 2024-08-31T06:22:33.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.452+0000 7f055da00640 1 -- 172.21.15.16:0/1672689170 learned_addr learned my addr 172.21.15.16:0/1672689170 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:33.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.452+0000 7f055ee00640 1 --2- 172.21.15.16:0/1672689170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f055810d9c0 0x7f0558105be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:33.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.453+0000 7f055d000640 1 --2- 172.21.15.16:0/1672689170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0558104eb0 0x7f05581054c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:33.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.453+0000 7f055ee00640 1 -- 172.21.15.16:0/1672689170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0558113680 con 0x7f055810d9c0 2024-08-31T06:22:33.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.453+0000 7f055ee00640 1 -- 172.21.15.16:0/1672689170 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0558113500 con 0x7f0558104560 2024-08-31T06:22:33.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.453+0000 7f055ee00640 1 -- 172.21.15.16:0/1672689170 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0558113800 con 0x7f0558104eb0 2024-08-31T06:22:33.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.453+0000 7f055e400640 1 --2- 172.21.15.16:0/1672689170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f055810d9c0 0x7f0558105be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:33.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.454+0000 7f055da00640 1 -- 172.21.15.16:0/1672689170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0558104eb0 msgr2=0x7f05581054c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:33.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.454+0000 7f055da00640 1 --2- 172.21.15.16:0/1672689170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0558104eb0 0x7f05581054c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:33.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.454+0000 7f055da00640 1 -- 172.21.15.16:0/1672689170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f055810d9c0 msgr2=0x7f0558105be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:33.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.454+0000 7f055da00640 1 --2- 172.21.15.16:0/1672689170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f055810d9c0 0x7f0558105be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:33.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.454+0000 7f055da00640 1 -- 172.21.15.16:0/1672689170 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05581ac780 con 0x7f0558104560 2024-08-31T06:22:33.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.454+0000 7f055da00640 1 --2- 172.21.15.16:0/1672689170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0558104560 0x7f05581091e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f054c00c890 tx=0x7f054c00cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:33.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.456+0000 7f0556000640 1 -- 172.21.15.16:0/1672689170 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054c004220 con 0x7f0558104560 2024-08-31T06:22:33.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.456+0000 7f0556000640 1 -- 172.21.15.16:0/1672689170 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f054c0043c0 con 0x7f0558104560 2024-08-31T06:22:33.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.456+0000 7f055ee00640 1 -- 172.21.15.16:0/1672689170 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05581ac9b0 con 0x7f0558104560 2024-08-31T06:22:33.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.456+0000 7f0556000640 1 -- 172.21.15.16:0/1672689170 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054c004a00 con 0x7f0558104560 2024-08-31T06:22:33.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.456+0000 7f055ee00640 1 -- 172.21.15.16:0/1672689170 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05581acfc0 con 0x7f0558104560 2024-08-31T06:22:33.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.461+0000 7f0567067740 1 Processor -- start 2024-08-31T06:22:33.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.461+0000 7f0567067740 1 -- start start 2024-08-31T06:22:33.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.462+0000 7f0567067740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564d33b1c070 0x564d33ec1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:33.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.462+0000 7f0567067740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d33af31b0 0x564d33ec20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:33.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.462+0000 7f0567067740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d33afb390 0x564d33ec25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:33.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.462+0000 7f0567067740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564d33e387b0 con 0x564d33af31b0 2024-08-31T06:22:33.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.462+0000 7f0567067740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x564d33e27d40 con 0x564d33afb390 2024-08-31T06:22:33.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.462+0000 7f0567067740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564d33e2bcf0 con 0x564d33b1c070 2024-08-31T06:22:33.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.463+0000 7f055e400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564d33b1c070 0x564d33ec1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:33.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.463+0000 7f055d000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d33afb390 0x564d33ec25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:33.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.463+0000 7f055d000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d33afb390 0x564d33ec25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51636/0 (socket says 172.21.15.16:51636) 2024-08-31T06:22:33.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.463+0000 7f055e400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564d33b1c070 0x564d33ec1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:54204/0 (socket says 172.21.15.16:54204) 2024-08-31T06:22:33.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.463+0000 7f055e400640 1 -- 172.21.15.16:0/2901164632 learned_addr learned my addr 172.21.15.16:0/2901164632 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:33.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.468+0000 7f0556000640 1 -- 172.21.15.16:0/1672689170 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f054c0079b0 con 0x7f0558104560 2024-08-31T06:22:33.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.468+0000 7f055d000640 1 -- 172.21.15.16:0/2901164632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564d33b1c070 msgr2=0x564d33ec1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:33.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.468+0000 7f055d000640 1 --2- 172.21.15.16:0/2901164632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564d33b1c070 0x564d33ec1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:33.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.468+0000 7f055d000640 1 -- 172.21.15.16:0/2901164632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d33af31b0 msgr2=0x564d33ec20a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:22:33.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.469+0000 7f055da00640 1 --2- 172.21.15.16:0/2901164632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d33af31b0 0x564d33ec20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:33.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.469+0000 7f055d000640 1 --2- 172.21.15.16:0/2901164632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d33af31b0 0x564d33ec20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:33.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.470+0000 7f055d000640 1 -- 172.21.15.16:0/2901164632 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d33b87650 con 0x564d33afb390 2024-08-31T06:22:33.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.472+0000 7f055d000640 1 --2- 172.21.15.16:0/2901164632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d33afb390 0x564d33ec25e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f055806a5a0 tx=0x7f054400bbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73255eb068436ee0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:33.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.472+0000 7f054aa00640 1 -- 172.21.15.16:0/2901164632 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054400be10 con 0x564d33afb390 2024-08-31T06:22:33.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.472+0000 7f054aa00640 1 -- 172.21.15.16:0/2901164632 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0544019070 con 0x564d33afb390 2024-08-31T06:22:33.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.472+0000 7f0556000640 1 --2- 172.21.15.16:0/1672689170 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f05400b4da0 0x7f05400b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:33.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.472+0000 7f054aa00640 1 -- 172.21.15.16:0/2901164632 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0544014d00 con 0x564d33afb390 2024-08-31T06:22:33.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.474+0000 7f0567067740 1 -- 172.21.15.16:0/2901164632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d33afb390 msgr2=0x564d33ec25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:33.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.474+0000 7f0567067740 1 --2- 172.21.15.16:0/2901164632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d33afb390 0x564d33ec25e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f055806a5a0 tx=0x7f054400bbc0 comp rx=0 tx=0).stop 2024-08-31T06:22:33.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.474+0000 7f0556000640 1 -- 172.21.15.16:0/1672689170 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8470+0+0 (secure 0 0 0) 0x7f054c0e5240 con 0x7f0558104560 2024-08-31T06:22:33.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.474+0000 7f055e400640 1 --2- 172.21.15.16:0/1672689170 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f05400b4da0 0x7f05400b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:33.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.475+0000 7f0567067740 1 -- 172.21.15.16:0/2901164632 shutdown_connections 2024-08-31T06:22:33.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.475+0000 7f0567067740 1 --2- 172.21.15.16:0/2901164632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d33afb390 0x564d33ec25e0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:33.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.475+0000 7f0567067740 1 --2- 172.21.15.16:0/2901164632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d33af31b0 0x564d33ec20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:33.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.475+0000 7f0567067740 1 --2- 172.21.15.16:0/2901164632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564d33b1c070 0x564d33ec1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:33.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.475+0000 7f0567067740 1 -- 172.21.15.16:0/2901164632 >> 172.21.15.16:0/2901164632 conn(0x564d33b10460 msgr2=0x564d33ebfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:33.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.475+0000 7f0567067740 1 -- 172.21.15.16:0/2901164632 shutdown_connections 2024-08-31T06:22:33.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.476+0000 7f0567067740 1 -- 172.21.15.16:0/2901164632 wait complete. 2024-08-31T06:22:33.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.477+0000 7f0567067740 1 -- start start 2024-08-31T06:22:33.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.477+0000 7f0567067740 1 Processor -- start 2024-08-31T06:22:33.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.478+0000 7f055e400640 1 --2- 172.21.15.16:0/1672689170 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f05400b4da0 0x7f05400b72b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0550009ca0 tx=0x7f0550009900 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:33.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.478+0000 7f0567067740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564d33af31b0 0x564d33ecd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:33.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.478+0000 7f0567067740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d33afb390 0x564d33ecdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:33.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.478+0000 7f0567067740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d33b1c070 0x564d33eea670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:33.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.478+0000 7f0567067740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564d33c8a700 con 0x564d33b1c070 2024-08-31T06:22:33.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.478+0000 7f0567067740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x564d33daecf0 con 0x564d33afb390 2024-08-31T06:22:33.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.478+0000 7f0567067740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564d33ee1350 con 0x564d33af31b0 2024-08-31T06:22:33.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.478+0000 7f0567067740 10 client.0 _ll_register_callbacks cb 0x564d33dd4470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:22:33.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.479+0000 7f055e400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d33b1c070 0x564d33eea670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:33.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.479+0000 7f055e400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d33b1c070 0x564d33eea670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45520/0 (socket says 172.21.15.16:45520) 2024-08-31T06:22:33.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.480+0000 7f055e400640 1 -- 172.21.15.16:0/3179937342 learned_addr learned my addr 172.21.15.16:0/3179937342 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:33.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.481+0000 7f055d000640 1 --2- 172.21.15.16:0/3179937342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d33afb390 0x564d33ecdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:33.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.481+0000 7f055e400640 1 -- 172.21.15.16:0/3179937342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564d33af31b0 msgr2=0x564d33ecd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:33.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.481+0000 7f055e400640 1 --2- 172.21.15.16:0/3179937342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564d33af31b0 0x564d33ecd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:33.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.481+0000 7f055e400640 1 -- 172.21.15.16:0/3179937342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d33afb390 msgr2=0x564d33ecdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:33.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.481+0000 7f055e400640 1 --2- 172.21.15.16:0/3179937342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d33afb390 0x564d33ecdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:33.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.481+0000 7f055e400640 1 -- 172.21.15.16:0/3179937342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d33def8b0 con 0x564d33b1c070 2024-08-31T06:22:33.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.483+0000 7f055d000640 1 --2- 172.21.15.16:0/3179937342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d33afb390 0x564d33ecdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:22:33.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.483+0000 7f055e400640 1 --2- 172.21.15.16:0/3179937342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d33b1c070 0x564d33eea670 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f0550031040 tx=0x7f055002a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e18a04d2d58d3d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:33.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.483+0000 7f053f400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:22:33.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.484+0000 7f053f400640 1 -- 172.21.15.16:0/3179937342 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0550055060 con 0x564d33b1c070 2024-08-31T06:22:33.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.484+0000 7f053f400640 1 -- 172.21.15.16:0/3179937342 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8470+0+0 (secure 0 0 0) 0x7f0550041030 con 0x564d33b1c070 2024-08-31T06:22:33.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.485+0000 7f0567067740 1 -- 172.21.15.16:0/3179937342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0544019070 con 0x564d33b1c070 2024-08-31T06:22:33.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.485+0000 7f053f400640 1 -- 172.21.15.16:0/3179937342 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f05500309f0 con 0x564d33b1c070 2024-08-31T06:22:33.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.485+0000 7f0567067740 10 client.18483 fetch_fsmap learned FSMap version 53 2024-08-31T06:22:33.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.486+0000 7f0567067740 1 -- 172.21.15.16:0/3179937342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x564d33efe320 con 0x564d33b1c070 2024-08-31T06:22:33.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.486+0000 7f053f400640 1 -- 172.21.15.16:0/3179937342 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0544019070 con 0x564d33b1c070 2024-08-31T06:22:33.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.487+0000 7f053f400640 1 -- 172.21.15.16:0/3179937342 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 53) ==== 8346+0+0 (secure 0 0 0) 0x7f0550004430 con 0x564d33b1c070 2024-08-31T06:22:33.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.487+0000 7f0567067740 10 client.18483 fetch_fsmap finished waiting for FSMap version 53 2024-08-31T06:22:33.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.487+0000 7f0567067740 10 client.18483 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:22:33.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.487+0000 7f0567067740 20 client.18483 populate_metadata read hostname 'smithi016' 2024-08-31T06:22:33.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.487+0000 7f0567067740 1 --2- 172.21.15.16:0/3179937342 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564d33ada9f0 0x564d33f01860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:33.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.487+0000 7f0567067740 4 client.18483 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:22:33.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.487+0000 7f0567067740 1 -- 172.21.15.16:0/3179937342 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d33e3a700 con 0x564d33ada9f0 2024-08-31T06:22:33.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.487+0000 7f055da00640 1 --2- 172.21.15.16:0/3179937342 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564d33ada9f0 0x564d33f01860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:33.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.488+0000 7f055da00640 1 --2- 172.21.15.16:0/3179937342 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564d33ada9f0 0x564d33f01860 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=769b6e71d27404dc server_cookie=c40988198b6a827 in_seq=0 out_seq=0 2024-08-31T06:22:33.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.488+0000 7f053f400640 10 client.18483 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:22:33.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.489+0000 7f053f400640 1 -- 172.21.15.16:0/3179937342 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564d33e3a700 con 0x564d33ada9f0 2024-08-31T06:22:33.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.489+0000 7f053f400640 10 client.18483 handle_command_reply: tid=0 2024-08-31T06:22:33.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.490+0000 7f0567067740 1 client.18483 shutdown 2024-08-31T06:22:33.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.492+0000 7f0567067740 1 -- 172.21.15.16:0/3179937342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d33b1c070 msgr2=0x564d33eea670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:33.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.492+0000 7f0567067740 1 --2- 172.21.15.16:0/3179937342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d33b1c070 0x564d33eea670 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f0550031040 tx=0x7f055002a8f0 comp rx=0 tx=0).stop 2024-08-31T06:22:33.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.494+0000 7f0567067740 1 -- 172.21.15.16:0/3179937342 shutdown_connections 2024-08-31T06:22:33.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.494+0000 7f0567067740 1 --2- 172.21.15.16:0/3179937342 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564d33ada9f0 0x564d33f01860 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:33.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.494+0000 7f0567067740 1 --2- 172.21.15.16:0/3179937342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d33b1c070 0x564d33eea670 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:33.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.494+0000 7f0567067740 1 --2- 172.21.15.16:0/3179937342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d33afb390 0x564d33ecdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:33.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.494+0000 7f0567067740 1 --2- 172.21.15.16:0/3179937342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564d33af31b0 0x564d33ecd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:33.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.494+0000 7f0567067740 1 -- 172.21.15.16:0/3179937342 >> 172.21.15.16:0/3179937342 conn(0x564d33b10460 msgr2=0x564d33ec4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:33.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.494+0000 7f053f400640 0 client.18483 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:22:33.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.494+0000 7f0567067740 1 -- 172.21.15.16:0/3179937342 shutdown_connections 2024-08-31T06:22:33.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.494+0000 7f0567067740 1 -- 172.21.15.16:0/3179937342 wait complete. 2024-08-31T06:22:33.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.495+0000 7f0567067740 20 client.18483 trim_cache size 0 max 16384 2024-08-31T06:22:33.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.536+0000 7f0567067740 1 Processor -- start 2024-08-31T06:22:33.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.537+0000 7f0567067740 1 -- start start 2024-08-31T06:22:33.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.537+0000 7f0567067740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d33af31b0 0x564d33ec7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:33.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.537+0000 7f0567067740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564d33afb390 0x564d33eea570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:33.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.537+0000 7f0567067740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d33b1c070 0x564d33ebf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:33.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.537+0000 7f0567067740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f055819e6a0 con 0x564d33af31b0 2024-08-31T06:22:33.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.537+0000 7f0567067740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f055819bf70 con 0x564d33b1c070 2024-08-31T06:22:33.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.537+0000 7f0567067740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f05581067c0 con 0x564d33afb390 2024-08-31T06:22:33.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.537+0000 7f055d000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564d33afb390 0x564d33eea570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:33.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.538+0000 7f055d000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564d33afb390 0x564d33eea570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:54216/0 (socket says 172.21.15.16:54216) 2024-08-31T06:22:33.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.538+0000 7f055d000640 1 -- 172.21.15.16:0/753158856 learned_addr learned my addr 172.21.15.16:0/753158856 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:33.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.538+0000 7f055e400640 1 --2- 172.21.15.16:0/753158856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d33b1c070 0x564d33ebf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:33.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.538+0000 7f055da00640 1 --2- 172.21.15.16:0/753158856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d33af31b0 0x564d33ec7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:33.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.538+0000 7f055d000640 1 -- 172.21.15.16:0/753158856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d33b1c070 msgr2=0x564d33ebf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:33.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.538+0000 7f055d000640 1 --2- 172.21.15.16:0/753158856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d33b1c070 0x564d33ebf000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:33.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.538+0000 7f055d000640 1 -- 172.21.15.16:0/753158856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d33af31b0 msgr2=0x564d33ec7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:33.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.538+0000 7f055d000640 1 --2- 172.21.15.16:0/753158856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d33af31b0 0x564d33ec7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:33.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.538+0000 7f055d000640 1 -- 172.21.15.16:0/753158856 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d33eb9c70 con 0x564d33afb390 2024-08-31T06:22:33.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.538+0000 7f055e400640 1 --2- 172.21.15.16:0/753158856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d33b1c070 0x564d33ebf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:22:33.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.538+0000 7f055da00640 1 --2- 172.21.15.16:0/753158856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d33af31b0 0x564d33ec7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:22:33.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.539+0000 7f055d000640 1 --2- 172.21.15.16:0/753158856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564d33afb390 0x564d33eea570 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f0544004810 tx=0x7f0544006dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d3ec11c30f6de65 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:33.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.539+0000 7f054a000640 1 -- 172.21.15.16:0/753158856 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054400dbe0 con 0x564d33afb390 2024-08-31T06:22:33.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.539+0000 7f054a000640 1 -- 172.21.15.16:0/753158856 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0544019040 con 0x564d33afb390 2024-08-31T06:22:33.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.539+0000 7f054a000640 1 -- 172.21.15.16:0/753158856 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054400dbe0 con 0x564d33afb390 2024-08-31T06:22:33.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.542+0000 7f0567067740 1 -- 172.21.15.16:0/753158856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564d33afb390 msgr2=0x564d33eea570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:33.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.542+0000 7f0567067740 1 --2- 172.21.15.16:0/753158856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564d33afb390 0x564d33eea570 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f0544004810 tx=0x7f0544006dc0 comp rx=0 tx=0).stop 2024-08-31T06:22:33.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.542+0000 7f0567067740 1 -- 172.21.15.16:0/753158856 shutdown_connections 2024-08-31T06:22:33.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.542+0000 7f0567067740 1 --2- 172.21.15.16:0/753158856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564d33afb390 0x564d33eea570 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:33.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.542+0000 7f0567067740 1 --2- 172.21.15.16:0/753158856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d33b1c070 0x564d33ebf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:33.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.542+0000 7f0567067740 1 --2- 172.21.15.16:0/753158856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d33af31b0 0x564d33ec7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:33.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.542+0000 7f0567067740 1 -- 172.21.15.16:0/753158856 >> 172.21.15.16:0/753158856 conn(0x564d33b10460 msgr2=0x564d33ec6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:33.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.543+0000 7f0567067740 1 -- 172.21.15.16:0/753158856 shutdown_connections 2024-08-31T06:22:33.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.545+0000 7f0567067740 1 -- 172.21.15.16:0/753158856 wait complete. 2024-08-31T06:22:33.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.546+0000 7f0567067740 1 -- start start 2024-08-31T06:22:33.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.547+0000 7f0567067740 1 Processor -- start 2024-08-31T06:22:33.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.548+0000 7f0567067740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d33af31b0 0x564d33ee2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:33.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.548+0000 7f0567067740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d33afb390 0x564d33ee2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:33.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.548+0000 7f0567067740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564d33b1c070 0x564d33ee2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:33.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.549+0000 7f055da00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d33af31b0 0x564d33ee2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:33.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.549+0000 7f055da00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d33af31b0 0x564d33ee2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51666/0 (socket says 172.21.15.16:51666) 2024-08-31T06:22:33.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.550+0000 7f0567067740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f05580738d0 con 0x564d33afb390 2024-08-31T06:22:33.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.550+0000 7f0567067740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0558074d40 con 0x564d33af31b0 2024-08-31T06:22:33.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.550+0000 7f0567067740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0558073fa0 con 0x564d33b1c070 2024-08-31T06:22:33.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.550+0000 7f0567067740 10 client.0 _ll_register_callbacks cb 0x7f0550031480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:22:33.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.550+0000 7f055d000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d33afb390 0x564d33ee2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:33.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.550+0000 7f055d000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d33afb390 0x564d33ee2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45546/0 (socket says 172.21.15.16:45546) 2024-08-31T06:22:33.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.550+0000 7f055d000640 1 -- 172.21.15.16:0/3308653805 learned_addr learned my addr 172.21.15.16:0/3308653805 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:33.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.551+0000 7f055da00640 1 -- 172.21.15.16:0/3308653805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564d33b1c070 msgr2=0x564d33ee2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:33.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.551+0000 7f055da00640 1 --2- 172.21.15.16:0/3308653805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564d33b1c070 0x564d33ee2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:33.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.551+0000 7f055da00640 1 -- 172.21.15.16:0/3308653805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d33afb390 msgr2=0x564d33ee2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:33.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.551+0000 7f055da00640 1 --2- 172.21.15.16:0/3308653805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d33afb390 0x564d33ee2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:33.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.551+0000 7f055da00640 1 -- 172.21.15.16:0/3308653805 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d33f6b220 con 0x564d33af31b0 2024-08-31T06:22:33.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.551+0000 7f055d000640 1 --2- 172.21.15.16:0/3308653805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d33afb390 0x564d33ee2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:22:33.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.551+0000 7f055da00640 1 --2- 172.21.15.16:0/3308653805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d33af31b0 0x564d33ee2510 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f054c0e2960 tx=0x7f054c0ab800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f430da2995c6191c server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:33.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.551+0000 7f053f400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:22:33.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.552+0000 7f053f400640 1 -- 172.21.15.16:0/3308653805 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054c01d070 con 0x564d33af31b0 2024-08-31T06:22:33.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.552+0000 7f0567067740 1 -- 172.21.15.16:0/3308653805 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0544019040 con 0x564d33af31b0 2024-08-31T06:22:33.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.553+0000 7f053f400640 1 -- 172.21.15.16:0/3308653805 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8470+0+0 (secure 0 0 0) 0x7f054c0e2be0 con 0x564d33af31b0 2024-08-31T06:22:33.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.555+0000 7f053f400640 1 -- 172.21.15.16:0/3308653805 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f054c0e2de0 con 0x564d33af31b0 2024-08-31T06:22:33.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.555+0000 7f0567067740 10 client.28976 fetch_fsmap learned FSMap version 53 2024-08-31T06:22:33.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.555+0000 7f0567067740 1 -- 172.21.15.16:0/3308653805 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x564d33f6c370 con 0x564d33af31b0 2024-08-31T06:22:33.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.555+0000 7f053f400640 1 -- 172.21.15.16:0/3308653805 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0544019040 con 0x564d33af31b0 2024-08-31T06:22:33.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.556+0000 7f053f400640 1 -- 172.21.15.16:0/3308653805 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 53) ==== 8346+0+0 (secure 0 0 0) 0x7f054c0a7270 con 0x564d33af31b0 2024-08-31T06:22:33.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.556+0000 7f0567067740 10 client.28976 fetch_fsmap finished waiting for FSMap version 53 2024-08-31T06:22:33.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.556+0000 7f0567067740 10 client.28976 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:22:33.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.556+0000 7f0567067740 20 client.28976 populate_metadata read hostname 'smithi016' 2024-08-31T06:22:33.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.556+0000 7f0567067740 1 --2- 172.21.15.16:0/3308653805 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564d33aafda0 0x564d33f6eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:33.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.556+0000 7f0567067740 4 client.28976 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:22:33.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.556+0000 7f0567067740 1 -- 172.21.15.16:0/3308653805 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564d33e361a0 con 0x564d33aafda0 2024-08-31T06:22:33.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.556+0000 7f055d000640 1 --2- 172.21.15.16:0/3308653805 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564d33aafda0 0x564d33f6eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:33.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.558+0000 7f055d000640 1 --2- 172.21.15.16:0/3308653805 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564d33aafda0 0x564d33f6eb30 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20f5243978664894 server_cookie=1768a57850ecd7ff in_seq=0 out_seq=0 2024-08-31T06:22:33.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.558+0000 7f053f400640 10 client.28976 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:22:33.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.558+0000 7f053f400640 1 -- 172.21.15.16:0/3308653805 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564d33e361a0 con 0x564d33aafda0 2024-08-31T06:22:33.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.558+0000 7f053f400640 10 client.28976 handle_command_reply: tid=0 2024-08-31T06:22:33.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.558+0000 7f0567067740 1 client.28976 shutdown 2024-08-31T06:22:33.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.559+0000 7f0567067740 1 -- 172.21.15.16:0/3308653805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d33af31b0 msgr2=0x564d33ee2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:33.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.559+0000 7f0567067740 1 --2- 172.21.15.16:0/3308653805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d33af31b0 0x564d33ee2510 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f054c0e2960 tx=0x7f054c0ab800 comp rx=0 tx=0).stop 2024-08-31T06:22:33.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.561+0000 7f0567067740 1 -- 172.21.15.16:0/3308653805 shutdown_connections 2024-08-31T06:22:33.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.561+0000 7f0567067740 1 --2- 172.21.15.16:0/3308653805 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564d33aafda0 0x564d33f6eb30 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:33.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.561+0000 7f0567067740 1 --2- 172.21.15.16:0/3308653805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564d33b1c070 0x564d33ee2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:33.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.561+0000 7f0567067740 1 --2- 172.21.15.16:0/3308653805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d33afb390 0x564d33ee2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:33.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.561+0000 7f0567067740 1 --2- 172.21.15.16:0/3308653805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d33af31b0 0x564d33ee2510 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:33.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.561+0000 7f0567067740 1 -- 172.21.15.16:0/3308653805 >> 172.21.15.16:0/3308653805 conn(0x564d33b10460 msgr2=0x564d33ec2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:33.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.561+0000 7f053f400640 0 client.28976 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:22:33.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.562+0000 7f0567067740 1 -- 172.21.15.16:0/3308653805 shutdown_connections 2024-08-31T06:22:33.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.562+0000 7f0567067740 1 -- 172.21.15.16:0/3308653805 wait complete. 2024-08-31T06:22:33.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.562+0000 7f0567067740 20 client.28976 trim_cache size 0 max 16384 2024-08-31T06:22:33.565 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:22:33.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.563+0000 7f055ee00640 1 -- 172.21.15.16:0/1672689170 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f05400b4da0 msgr2=0x7f05400b72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:33.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.563+0000 7f055ee00640 1 --2- 172.21.15.16:0/1672689170 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f05400b4da0 0x7f05400b72b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0550009ca0 tx=0x7f0550009900 comp rx=0 tx=0).stop 2024-08-31T06:22:33.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.563+0000 7f055ee00640 1 -- 172.21.15.16:0/1672689170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0558104560 msgr2=0x7f05581091e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:33.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.563+0000 7f055ee00640 1 --2- 172.21.15.16:0/1672689170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0558104560 0x7f05581091e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f054c00c890 tx=0x7f054c00cd50 comp rx=0 tx=0).stop 2024-08-31T06:22:33.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.563+0000 7f055ee00640 1 -- 172.21.15.16:0/1672689170 shutdown_connections 2024-08-31T06:22:33.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.563+0000 7f055ee00640 1 --2- 172.21.15.16:0/1672689170 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f05400b4da0 0x7f05400b72b0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:33.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.563+0000 7f055ee00640 1 --2- 172.21.15.16:0/1672689170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0558104eb0 0x7f05581054c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:33.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.563+0000 7f055ee00640 1 --2- 172.21.15.16:0/1672689170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f055810d9c0 0x7f0558105be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:33.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.563+0000 7f055ee00640 1 --2- 172.21.15.16:0/1672689170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0558104560 0x7f05581091e0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:33.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.563+0000 7f055ee00640 1 -- 172.21.15.16:0/1672689170 >> 172.21.15.16:0/1672689170 conn(0x7f05580fe060 msgr2=0x7f0558102160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:33.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.563+0000 7f055ee00640 1 -- 172.21.15.16:0/1672689170 shutdown_connections 2024-08-31T06:22:33.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:33.563+0000 7f055ee00640 1 -- 172.21.15.16:0/1672689170 wait complete. 2024-08-31T06:22:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.550+0000 7ffbdf200640 1 -- 172.21.15.16:0/1608012100 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbe01058b0 msgr2=0x7ffbe0105cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.550+0000 7ffbdf200640 1 --2- 172.21.15.16:0/1608012100 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbe01058b0 0x7ffbe0105cc0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0009900 tx=0x7ffbd002a3f0 comp rx=0 tx=0).stop 2024-08-31T06:22:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.550+0000 7ffbdf200640 1 -- 172.21.15.16:0/1608012100 shutdown_connections 2024-08-31T06:22:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.550+0000 7ffbdf200640 1 --2- 172.21.15.16:0/1608012100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbe010b1e0 0x7ffbe010b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.550+0000 7ffbdf200640 1 --2- 172.21.15.16:0/1608012100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbe010bb30 0x7ffbe0112360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.550+0000 7ffbdf200640 1 --2- 172.21.15.16:0/1608012100 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbe01058b0 0x7ffbe0105cc0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.550+0000 7ffbdf200640 1 -- 172.21.15.16:0/1608012100 >> 172.21.15.16:0/1608012100 conn(0x7ffbe0101290 msgr2=0x7ffbe01036e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.550+0000 7ffbdf200640 1 -- 172.21.15.16:0/1608012100 shutdown_connections 2024-08-31T06:22:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.550+0000 7ffbdf200640 1 -- 172.21.15.16:0/1608012100 wait complete. 2024-08-31T06:22:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.552+0000 7ffbdf200640 1 Processor -- start 2024-08-31T06:22:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.553+0000 7ffbdf200640 1 -- start start 2024-08-31T06:22:34.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.554+0000 7ffbdf200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbe01058b0 0x7ffbe019c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:34.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.554+0000 7ffbdf200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbe010b1e0 0x7ffbe019c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:34.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.554+0000 7ffbdf200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbe010bb30 0x7ffbe01a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:34.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.554+0000 7ffbdf200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe0115820 con 0x7ffbe01058b0 2024-08-31T06:22:34.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.554+0000 7ffbdf200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe01156a0 con 0x7ffbe010bb30 2024-08-31T06:22:34.557 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.554+0000 7ffbdf200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe01159a0 con 0x7ffbe010b1e0 2024-08-31T06:22:34.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.555+0000 7ffbde800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbe010bb30 0x7ffbe01a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:34.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.555+0000 7ffbdde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbe01058b0 0x7ffbe019c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:34.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.555+0000 7ffbde800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbe010bb30 0x7ffbe01a1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:51696/0 (socket says 172.21.15.16:51696) 2024-08-31T06:22:34.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.555+0000 7ffbde800640 1 -- 172.21.15.16:0/1964235484 learned_addr learned my addr 172.21.15.16:0/1964235484 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:34.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.555+0000 7ffbde800640 1 -- 172.21.15.16:0/1964235484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbe010b1e0 msgr2=0x7ffbe019c7e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-31T06:22:34.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.555+0000 7ffbde800640 1 --2- 172.21.15.16:0/1964235484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbe010b1e0 0x7ffbe019c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:34.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.555+0000 7ffbde800640 1 -- 172.21.15.16:0/1964235484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbe01058b0 msgr2=0x7ffbe019c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:34.558 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.555+0000 7ffbde800640 1 --2- 172.21.15.16:0/1964235484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbe01058b0 0x7ffbe019c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:34.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.555+0000 7ffbde800640 1 -- 172.21.15.16:0/1964235484 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbe01a2170 con 0x7ffbe010bb30 2024-08-31T06:22:34.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.556+0000 7ffbde800640 1 --2- 172.21.15.16:0/1964235484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbe010bb30 0x7ffbe01a1a70 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ffbd800eaa0 tx=0x7ffbd800ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:34.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.556+0000 7ffbd6a00640 1 -- 172.21.15.16:0/1964235484 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd800cdb0 con 0x7ffbe010bb30 2024-08-31T06:22:34.559 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.557+0000 7ffbd6a00640 1 -- 172.21.15.16:0/1964235484 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbd8005020 con 0x7ffbe010bb30 2024-08-31T06:22:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.557+0000 7ffbd6a00640 1 -- 172.21.15.16:0/1964235484 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd80116d0 con 0x7ffbe010bb30 2024-08-31T06:22:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.557+0000 7ffbdf200640 1 -- 172.21.15.16:0/1964235484 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbe01a2460 con 0x7ffbe010bb30 2024-08-31T06:22:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.557+0000 7ffbdf200640 1 -- 172.21.15.16:0/1964235484 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbe01a8bf0 con 0x7ffbe010bb30 2024-08-31T06:22:34.565 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.561+0000 7ffbebdcb740 1 Processor -- start 2024-08-31T06:22:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.570+0000 7ffbd6a00640 1 -- 172.21.15.16:0/1964235484 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7ffbd80051d0 con 0x7ffbe010bb30 2024-08-31T06:22:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.570+0000 7ffbebdcb740 1 -- start start 2024-08-31T06:22:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.571+0000 7ffbebdcb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565491f6a1b0 0x56549230d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.571+0000 7ffbebdcb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565491f412f0 0x56549230dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.571+0000 7ffbebdcb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565491f494d0 0x56549230e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.571+0000 7ffbebdcb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x565492282f70 con 0x565491f494d0 2024-08-31T06:22:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.571+0000 7ffbebdcb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x565492272500 con 0x565491f6a1b0 2024-08-31T06:22:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.571+0000 7ffbebdcb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5654922764b0 con 0x565491f412f0 2024-08-31T06:22:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.571+0000 7ffbdde00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565491f412f0 0x56549230dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.571+0000 7ffbdde00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565491f412f0 0x56549230dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:54240/0 (socket says 172.21.15.16:54240) 2024-08-31T06:22:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.571+0000 7ffbdde00640 1 -- 172.21.15.16:0/637646367 learned_addr learned my addr 172.21.15.16:0/637646367 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.572+0000 7ffbdde00640 1 -- 172.21.15.16:0/637646367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565491f6a1b0 msgr2=0x56549230d8a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-31T06:22:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.572+0000 7ffbdde00640 1 --2- 172.21.15.16:0/637646367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565491f6a1b0 0x56549230d8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.572+0000 7ffbdde00640 1 -- 172.21.15.16:0/637646367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565491f494d0 msgr2=0x56549230e320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-31T06:22:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.573+0000 7ffbd6a00640 1 --2- 172.21.15.16:0/1964235484 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ffbc40b4df0 0x7ffbc40b7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.574+0000 7ffbdd400640 1 --2- 172.21.15.16:0/637646367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565491f494d0 0x56549230e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.575+0000 7ffbd6a00640 1 -- 172.21.15.16:0/1964235484 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8470+0+0 (secure 0 0 0) 0x7ffbd80e56f0 con 0x7ffbe010bb30 2024-08-31T06:22:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.575+0000 7ffbdde00640 1 --2- 172.21.15.16:0/637646367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565491f494d0 0x56549230e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.575+0000 7ffbdde00640 1 -- 172.21.15.16:0/637646367 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56549222cc10 con 0x565491f412f0 2024-08-31T06:22:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.575+0000 7ffbdd400640 1 --2- 172.21.15.16:0/1964235484 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ffbc40b4df0 0x7ffbc40b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.576+0000 7ffbdde00640 1 --2- 172.21.15.16:0/637646367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565491f412f0 0x56549230dde0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0006010 tx=0x7ffbd0004150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79c65a85b3aa81e4 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.577+0000 7ffbcb400640 1 -- 172.21.15.16:0/637646367 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd003f020 con 0x565491f412f0 2024-08-31T06:22:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.577+0000 7ffbcb400640 1 -- 172.21.15.16:0/637646367 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbd0005500 con 0x565491f412f0 2024-08-31T06:22:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.577+0000 7ffbcb400640 1 -- 172.21.15.16:0/637646367 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd0035070 con 0x565491f412f0 2024-08-31T06:22:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.578+0000 7ffbebdcb740 1 -- 172.21.15.16:0/637646367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565491f412f0 msgr2=0x56549230dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.578+0000 7ffbdd400640 1 --2- 172.21.15.16:0/1964235484 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ffbc40b4df0 0x7ffbc40b7300 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ffbe0104ac0 tx=0x7ffbcc0046c0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.578+0000 7ffbebdcb740 1 --2- 172.21.15.16:0/637646367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565491f412f0 0x56549230dde0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0006010 tx=0x7ffbd0004150 comp rx=0 tx=0).stop 2024-08-31T06:22:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.579+0000 7ffbebdcb740 1 -- 172.21.15.16:0/637646367 shutdown_connections 2024-08-31T06:22:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.579+0000 7ffbebdcb740 1 --2- 172.21.15.16:0/637646367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565491f412f0 0x56549230dde0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.579+0000 7ffbebdcb740 1 --2- 172.21.15.16:0/637646367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565491f494d0 0x56549230e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.579+0000 7ffbebdcb740 1 --2- 172.21.15.16:0/637646367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565491f6a1b0 0x56549230d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.579+0000 7ffbebdcb740 1 -- 172.21.15.16:0/637646367 >> 172.21.15.16:0/637646367 conn(0x565491f5e300 msgr2=0x56549230b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.579+0000 7ffbebdcb740 1 -- 172.21.15.16:0/637646367 shutdown_connections 2024-08-31T06:22:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.579+0000 7ffbebdcb740 1 -- 172.21.15.16:0/637646367 wait complete. 2024-08-31T06:22:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.580+0000 7ffbebdcb740 1 -- start start 2024-08-31T06:22:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.581+0000 7ffbebdcb740 1 Processor -- start 2024-08-31T06:22:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.582+0000 7ffbebdcb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565491f412f0 0x565492318eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:34.585 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.582+0000 7ffbebdcb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565491f494d0 0x5654923193f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:34.585 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.582+0000 7ffbebdcb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565491f6a1b0 0x565492319950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:34.585 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.582+0000 7ffbebdcb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5654921abab0 con 0x565491f6a1b0 2024-08-31T06:22:34.585 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.582+0000 7ffbebdcb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x565492219320 con 0x565491f412f0 2024-08-31T06:22:34.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.582+0000 7ffbebdcb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56549231bfc0 con 0x565491f494d0 2024-08-31T06:22:34.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.582+0000 7ffbdd400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565491f494d0 0x5654923193f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:34.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.582+0000 7ffbebdcb740 10 client.0 _ll_register_callbacks cb 0x565492252e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:22:34.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.582+0000 7ffbdd400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565491f494d0 0x5654923193f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:54250/0 (socket says 172.21.15.16:54250) 2024-08-31T06:22:34.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.582+0000 7ffbdd400640 1 -- 172.21.15.16:0/2779466941 learned_addr learned my addr 172.21.15.16:0/2779466941 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:34.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.583+0000 7ffbde800640 1 --2- 172.21.15.16:0/2779466941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565491f6a1b0 0x565492319950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:34.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.583+0000 7ffbdde00640 1 --2- 172.21.15.16:0/2779466941 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565491f412f0 0x565492318eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:34.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.583+0000 7ffbdd400640 1 -- 172.21.15.16:0/2779466941 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565491f412f0 msgr2=0x565492318eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:34.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.583+0000 7ffbdd400640 1 --2- 172.21.15.16:0/2779466941 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565491f412f0 0x565492318eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.583+0000 7ffbdd400640 1 -- 172.21.15.16:0/2779466941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565491f6a1b0 msgr2=0x565492319950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.583+0000 7ffbdd400640 1 --2- 172.21.15.16:0/2779466941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565491f6a1b0 0x565492319950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.583+0000 7ffbdd400640 1 -- 172.21.15.16:0/2779466941 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565491fd5790 con 0x565491f494d0 2024-08-31T06:22:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.584+0000 7ffbdd400640 1 --2- 172.21.15.16:0/2779466941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565491f494d0 0x5654923193f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc011b10 tx=0x7ffbcc004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e13c447ef294df8 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.584+0000 7ffbbfe00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:22:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.585+0000 7ffbbfe00640 1 -- 172.21.15.16:0/2779466941 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc020d80 con 0x565491f494d0 2024-08-31T06:22:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.585+0000 7ffbbfe00640 1 -- 172.21.15.16:0/2779466941 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8470+0+0 (secure 0 0 0) 0x7ffbcc012a10 con 0x565491f494d0 2024-08-31T06:22:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.585+0000 7ffbebdcb740 1 -- 172.21.15.16:0/2779466941 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbd0005500 con 0x565491f494d0 2024-08-31T06:22:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.585+0000 7ffbbfe00640 1 -- 172.21.15.16:0/2779466941 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7ffbcc012c10 con 0x565491f494d0 2024-08-31T06:22:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.585+0000 7ffbebdcb740 10 client.29017 fetch_fsmap learned FSMap version 53 2024-08-31T06:22:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.585+0000 7ffbebdcb740 1 -- 172.21.15.16:0/2779466941 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x56549204e620 con 0x565491f494d0 2024-08-31T06:22:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.586+0000 7ffbbfe00640 1 -- 172.21.15.16:0/2779466941 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbd0005500 con 0x565491f494d0 2024-08-31T06:22:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.587+0000 7ffbbfe00640 1 -- 172.21.15.16:0/2779466941 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 53) ==== 8346+0+0 (secure 0 0 0) 0x7ffbcc0220d0 con 0x565491f494d0 2024-08-31T06:22:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.587+0000 7ffbebdcb740 10 client.29017 fetch_fsmap finished waiting for FSMap version 53 2024-08-31T06:22:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.588+0000 7ffbebdcb740 10 client.29017 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:22:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.588+0000 7ffbebdcb740 20 client.29017 populate_metadata read hostname 'smithi016' 2024-08-31T06:22:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.588+0000 7ffbebdcb740 1 --2- 172.21.15.16:0/2779466941 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x565491f28d80 0x56549232b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.588+0000 7ffbebdcb740 4 client.29017 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:22:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.588+0000 7ffbebdcb740 1 -- 172.21.15.16:0/2779466941 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565492284ec0 con 0x565491f28d80 2024-08-31T06:22:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.589+0000 7ffbdde00640 1 --2- 172.21.15.16:0/2779466941 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x565491f28d80 0x56549232b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.591+0000 7ffbdde00640 1 --2- 172.21.15.16:0/2779466941 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x565491f28d80 0x56549232b040 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35c24b21769c96a0 server_cookie=a210a2dfdc2a2688 in_seq=0 out_seq=0 2024-08-31T06:22:34.594 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.591+0000 7ffbbfe00640 10 client.29017 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:22:34.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.592+0000 7ffbbfe00640 1 -- 172.21.15.16:0/2779466941 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x565492284ec0 con 0x565491f28d80 2024-08-31T06:22:34.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.593+0000 7ffbbfe00640 10 client.29017 handle_command_reply: tid=0 2024-08-31T06:22:34.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.594+0000 7ffbebdcb740 1 client.29017 shutdown 2024-08-31T06:22:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.596+0000 7ffbebdcb740 1 -- 172.21.15.16:0/2779466941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565491f494d0 msgr2=0x5654923193f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.596+0000 7ffbebdcb740 1 --2- 172.21.15.16:0/2779466941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565491f494d0 0x5654923193f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc011b10 tx=0x7ffbcc004730 comp rx=0 tx=0).stop 2024-08-31T06:22:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.596+0000 7ffbebdcb740 1 -- 172.21.15.16:0/2779466941 shutdown_connections 2024-08-31T06:22:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.596+0000 7ffbebdcb740 1 --2- 172.21.15.16:0/2779466941 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x565491f28d80 0x56549232b040 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.596+0000 7ffbebdcb740 1 --2- 172.21.15.16:0/2779466941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565491f494d0 0x5654923193f0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.596+0000 7ffbebdcb740 1 --2- 172.21.15.16:0/2779466941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565491f6a1b0 0x565492319950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.596+0000 7ffbebdcb740 1 --2- 172.21.15.16:0/2779466941 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565491f412f0 0x565492318eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.596+0000 7ffbebdcb740 1 -- 172.21.15.16:0/2779466941 >> 172.21.15.16:0/2779466941 conn(0x565491f5e300 msgr2=0x56549230e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.597+0000 7ffbbfe00640 0 client.29017 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:22:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.597+0000 7ffbebdcb740 1 -- 172.21.15.16:0/2779466941 shutdown_connections 2024-08-31T06:22:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.597+0000 7ffbebdcb740 1 -- 172.21.15.16:0/2779466941 wait complete. 2024-08-31T06:22:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.598+0000 7ffbebdcb740 20 client.29017 trim_cache size 0 max 16384 2024-08-31T06:22:34.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.647+0000 7ffbebdcb740 1 Processor -- start 2024-08-31T06:22:34.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.650+0000 7ffbebdcb740 1 -- start start 2024-08-31T06:22:34.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.650+0000 7ffbebdcb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565491f412f0 0x56549230efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:34.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.650+0000 7ffbebdcb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565491f494d0 0x565492316080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:34.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.650+0000 7ffbebdcb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565491f6a1b0 0x5654923165c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:34.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.650+0000 7ffbebdcb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe01af870 con 0x565491f412f0 2024-08-31T06:22:34.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.650+0000 7ffbebdcb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe01ad140 con 0x565491f6a1b0 2024-08-31T06:22:34.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.650+0000 7ffbebdcb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe01aafb0 con 0x565491f494d0 2024-08-31T06:22:34.654 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.651+0000 7ffbdd400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565491f494d0 0x565492316080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:34.654 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.651+0000 7ffbdde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565491f412f0 0x56549230efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:34.654 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.651+0000 7ffbdd400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565491f494d0 0x565492316080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:54258/0 (socket says 172.21.15.16:54258) 2024-08-31T06:22:34.654 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.651+0000 7ffbdd400640 1 -- 172.21.15.16:0/2471041953 learned_addr learned my addr 172.21.15.16:0/2471041953 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:34.654 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.651+0000 7ffbde800640 1 --2- 172.21.15.16:0/2471041953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565491f6a1b0 0x5654923165c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.652+0000 7ffbdd400640 1 -- 172.21.15.16:0/2471041953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565491f6a1b0 msgr2=0x5654923165c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.652+0000 7ffbdd400640 1 --2- 172.21.15.16:0/2471041953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565491f6a1b0 0x5654923165c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.652+0000 7ffbdd400640 1 -- 172.21.15.16:0/2471041953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565491f412f0 msgr2=0x56549230efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.652+0000 7ffbdd400640 1 --2- 172.21.15.16:0/2471041953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565491f412f0 0x56549230efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.652+0000 7ffbdd400640 1 -- 172.21.15.16:0/2471041953 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654923106b0 con 0x565491f494d0 2024-08-31T06:22:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.654+0000 7ffbdd400640 1 --2- 172.21.15.16:0/2471041953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565491f494d0 0x565492316080 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc012dd0 tx=0x7ffbcc012820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac720332309a4087 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.654+0000 7ffbcaa00640 1 -- 172.21.15.16:0/2471041953 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc039070 con 0x565491f494d0 2024-08-31T06:22:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.654+0000 7ffbcaa00640 1 -- 172.21.15.16:0/2471041953 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbcc016070 con 0x565491f494d0 2024-08-31T06:22:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.654+0000 7ffbcaa00640 1 -- 172.21.15.16:0/2471041953 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc035b40 con 0x565491f494d0 2024-08-31T06:22:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.656+0000 7ffbebdcb740 1 -- 172.21.15.16:0/2471041953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565491f494d0 msgr2=0x565492316080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.656+0000 7ffbebdcb740 1 --2- 172.21.15.16:0/2471041953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565491f494d0 0x565492316080 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc012dd0 tx=0x7ffbcc012820 comp rx=0 tx=0).stop 2024-08-31T06:22:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.656+0000 7ffbebdcb740 1 -- 172.21.15.16:0/2471041953 shutdown_connections 2024-08-31T06:22:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.656+0000 7ffbebdcb740 1 --2- 172.21.15.16:0/2471041953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565491f494d0 0x565492316080 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.656+0000 7ffbebdcb740 1 --2- 172.21.15.16:0/2471041953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565491f6a1b0 0x5654923165c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.656+0000 7ffbebdcb740 1 --2- 172.21.15.16:0/2471041953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565491f412f0 0x56549230efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.656+0000 7ffbebdcb740 1 -- 172.21.15.16:0/2471041953 >> 172.21.15.16:0/2471041953 conn(0x565491f5e300 msgr2=0x56549230e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.656+0000 7ffbebdcb740 1 -- 172.21.15.16:0/2471041953 shutdown_connections 2024-08-31T06:22:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.656+0000 7ffbebdcb740 1 -- 172.21.15.16:0/2471041953 wait complete. 2024-08-31T06:22:34.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.657+0000 7ffbebdcb740 1 -- start start 2024-08-31T06:22:34.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.657+0000 7ffbebdcb740 1 Processor -- start 2024-08-31T06:22:34.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.660+0000 7ffbebdcb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565491f412f0 0x565492321a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:34.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.660+0000 7ffbebdcb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565491f494d0 0x56549231ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:34.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.660+0000 7ffbebdcb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565491f6a1b0 0x56549231f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:34.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.660+0000 7ffbebdcb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe01ab600 con 0x565491f6a1b0 2024-08-31T06:22:34.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.660+0000 7ffbebdcb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe01aca70 con 0x565491f412f0 2024-08-31T06:22:34.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.660+0000 7ffbebdcb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe01abcd0 con 0x565491f494d0 2024-08-31T06:22:34.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.660+0000 7ffbebdcb740 10 client.0 _ll_register_callbacks cb 0x7ffbcc022ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:22:34.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.661+0000 7ffbde800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565491f6a1b0 0x56549231f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:34.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.661+0000 7ffbde800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565491f6a1b0 0x56549231f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45632/0 (socket says 172.21.15.16:45632) 2024-08-31T06:22:34.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.661+0000 7ffbde800640 1 -- 172.21.15.16:0/1640348560 learned_addr learned my addr 172.21.15.16:0/1640348560 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:34.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.662+0000 7ffbdd400640 1 --2- 172.21.15.16:0/1640348560 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565491f494d0 0x56549231ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:34.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.662+0000 7ffbde800640 1 -- 172.21.15.16:0/1640348560 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565491f494d0 msgr2=0x56549231ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:34.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.662+0000 7ffbde800640 1 --2- 172.21.15.16:0/1640348560 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565491f494d0 0x56549231ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:34.665 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.662+0000 7ffbde800640 1 -- 172.21.15.16:0/1640348560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565491f412f0 msgr2=0x565492321a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:34.665 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.662+0000 7ffbde800640 1 --2- 172.21.15.16:0/1640348560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565491f412f0 0x565492321a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:34.665 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.662+0000 7ffbde800640 1 -- 172.21.15.16:0/1640348560 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56549231fc80 con 0x565491f6a1b0 2024-08-31T06:22:34.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.665+0000 7ffbde800640 1 --2- 172.21.15.16:0/1640348560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565491f6a1b0 0x56549231f4f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ffbd800efd0 tx=0x7ffbd80abe10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f30d9ef63a97cb server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:34.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.665+0000 7ffbbfe00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:22:34.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.665+0000 7ffbbfe00640 1 -- 172.21.15.16:0/1640348560 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd801f040 con 0x565491f6a1b0 2024-08-31T06:22:34.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.665+0000 7ffbebdcb740 1 -- 172.21.15.16:0/1640348560 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbcc016070 con 0x565491f6a1b0 2024-08-31T06:22:34.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.666+0000 7ffbbfe00640 1 -- 172.21.15.16:0/1640348560 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8470+0+0 (secure 0 0 0) 0x7ffbd80ab070 con 0x565491f6a1b0 2024-08-31T06:22:34.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.668+0000 7ffbbfe00640 1 -- 172.21.15.16:0/1640348560 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7ffbd80971b0 con 0x565491f6a1b0 2024-08-31T06:22:34.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.668+0000 7ffbebdcb740 10 client.18498 fetch_fsmap learned FSMap version 53 2024-08-31T06:22:34.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.668+0000 7ffbebdcb740 1 -- 172.21.15.16:0/1640348560 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x565492395a30 con 0x565491f6a1b0 2024-08-31T06:22:34.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.669+0000 7ffbbfe00640 1 -- 172.21.15.16:0/1640348560 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbcc016070 con 0x565491f6a1b0 2024-08-31T06:22:34.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.670+0000 7ffbbfe00640 1 -- 172.21.15.16:0/1640348560 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 53) ==== 8346+0+0 (secure 0 0 0) 0x7ffbd800f040 con 0x565491f6a1b0 2024-08-31T06:22:34.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.670+0000 7ffbebdcb740 10 client.18498 fetch_fsmap finished waiting for FSMap version 53 2024-08-31T06:22:34.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.670+0000 7ffbebdcb740 10 client.18498 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-31T06:22:34.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.671+0000 7ffbebdcb740 20 client.18498 populate_metadata read hostname 'smithi016' 2024-08-31T06:22:34.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.671+0000 7ffbebdcb740 1 --2- 172.21.15.16:0/1640348560 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x565491efe1c0 0x565492398120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:34.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.671+0000 7ffbebdcb740 4 client.18498 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-31T06:22:34.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.671+0000 7ffbebdcb740 1 -- 172.21.15.16:0/1640348560 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565492280960 con 0x565491efe1c0 2024-08-31T06:22:34.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.671+0000 7ffbdde00640 1 --2- 172.21.15.16:0/1640348560 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x565491efe1c0 0x565492398120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:34.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.672+0000 7ffbdde00640 1 --2- 172.21.15.16:0/1640348560 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x565491efe1c0 0x565492398120 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=485a755fb4371ceb server_cookie=48a075a29a957991 in_seq=0 out_seq=0 2024-08-31T06:22:34.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.672+0000 7ffbbfe00640 10 client.18498 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:22:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.794+0000 7ffbbfe00640 1 -- 172.21.15.16:0/1640348560 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x565492280960 con 0x565491efe1c0 2024-08-31T06:22:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.794+0000 7ffbbfe00640 10 client.18498 handle_command_reply: tid=0 2024-08-31T06:22:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.794+0000 7ffbebdcb740 1 client.18498 shutdown 2024-08-31T06:22:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.794+0000 7ffbebdcb740 1 -- 172.21.15.16:0/1640348560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565491f6a1b0 msgr2=0x56549231f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.794+0000 7ffbebdcb740 1 --2- 172.21.15.16:0/1640348560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565491f6a1b0 0x56549231f4f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ffbd800efd0 tx=0x7ffbd80abe10 comp rx=0 tx=0).stop 2024-08-31T06:22:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.795+0000 7ffbebdcb740 1 -- 172.21.15.16:0/1640348560 shutdown_connections 2024-08-31T06:22:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.795+0000 7ffbebdcb740 1 --2- 172.21.15.16:0/1640348560 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x565491efe1c0 0x565492398120 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.795+0000 7ffbebdcb740 1 --2- 172.21.15.16:0/1640348560 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565491f494d0 0x56549231ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.795+0000 7ffbebdcb740 1 --2- 172.21.15.16:0/1640348560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565491f6a1b0 0x56549231f4f0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.795+0000 7ffbebdcb740 1 --2- 172.21.15.16:0/1640348560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565491f412f0 0x565492321a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.795+0000 7ffbebdcb740 1 -- 172.21.15.16:0/1640348560 >> 172.21.15.16:0/1640348560 conn(0x565491f5e300 msgr2=0x56549231a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.795+0000 7ffbbfe00640 0 client.18498 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:22:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.795+0000 7ffbebdcb740 1 -- 172.21.15.16:0/1640348560 shutdown_connections 2024-08-31T06:22:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.795+0000 7ffbebdcb740 1 -- 172.21.15.16:0/1640348560 wait complete. 2024-08-31T06:22:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.796+0000 7ffbebdcb740 20 client.18498 trim_cache size 0 max 16384 2024-08-31T06:22:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-08-31T06:22:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-08-31T06:22:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-08-31T06:22:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-08-31T06:22:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.797+0000 7ffbdf200640 1 -- 172.21.15.16:0/1964235484 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ffbc40b4df0 msgr2=0x7ffbc40b7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.797+0000 7ffbdf200640 1 --2- 172.21.15.16:0/1964235484 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ffbc40b4df0 0x7ffbc40b7300 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ffbe0104ac0 tx=0x7ffbcc0046c0 comp rx=0 tx=0).stop 2024-08-31T06:22:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.797+0000 7ffbdf200640 1 -- 172.21.15.16:0/1964235484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbe010bb30 msgr2=0x7ffbe01a1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.797+0000 7ffbdf200640 1 --2- 172.21.15.16:0/1964235484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbe010bb30 0x7ffbe01a1a70 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ffbd800eaa0 tx=0x7ffbd800ef60 comp rx=0 tx=0).stop 2024-08-31T06:22:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.797+0000 7ffbdf200640 1 -- 172.21.15.16:0/1964235484 shutdown_connections 2024-08-31T06:22:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.797+0000 7ffbdf200640 1 --2- 172.21.15.16:0/1964235484 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ffbc40b4df0 0x7ffbc40b7300 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.797+0000 7ffbdf200640 1 --2- 172.21.15.16:0/1964235484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbe010b1e0 0x7ffbe019c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.797+0000 7ffbdf200640 1 --2- 172.21.15.16:0/1964235484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbe010bb30 0x7ffbe01a1a70 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.797+0000 7ffbdf200640 1 --2- 172.21.15.16:0/1964235484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbe01058b0 0x7ffbe019c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.797+0000 7ffbdf200640 1 -- 172.21.15.16:0/1964235484 >> 172.21.15.16:0/1964235484 conn(0x7ffbe0101290 msgr2=0x7ffbe01036b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.799+0000 7ffbdf200640 1 -- 172.21.15.16:0/1964235484 shutdown_connections 2024-08-31T06:22:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-08-31T06:22:34.799+0000 7ffbdf200640 1 -- 172.21.15.16:0/1964235484 wait complete. 2024-08-31T06:22:35.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:34 smithi155 ceph-mon[21742]: pgmap v844: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 18 MiB/s wr, 1.44k op/s 2024-08-31T06:22:35.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:34 smithi016 ceph-mon[17194]: pgmap v844: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 18 MiB/s wr, 1.44k op/s 2024-08-31T06:22:35.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:34 smithi184 ceph-mon[22331]: pgmap v844: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 18 MiB/s wr, 1.44k op/s 2024-08-31T06:22:35.893 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-31T06:22:35.894 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-31T06:22:36.572 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:22:37.047 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:36 smithi016 ceph-mon[17194]: pgmap v845: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 21 MiB/s wr, 1.75k op/s 2024-08-31T06:22:37.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:36 smithi155 ceph-mon[21742]: pgmap v845: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 21 MiB/s wr, 1.75k op/s 2024-08-31T06:22:37.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:36 smithi184 ceph-mon[22331]: pgmap v845: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 21 MiB/s wr, 1.75k op/s 2024-08-31T06:22:37.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.339+0000 7f3bbbe00640 1 -- 172.21.15.16:0/532622098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bbc1020a0 msgr2=0x7f3bbc10f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:37.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.339+0000 7f3bbbe00640 1 --2- 172.21.15.16:0/532622098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bbc1020a0 0x7f3bbc10f4f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4009900 tx=0x7f3ba402a410 comp rx=0 tx=0).stop 2024-08-31T06:22:37.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.339+0000 7f3bbbe00640 1 -- 172.21.15.16:0/532622098 shutdown_connections 2024-08-31T06:22:37.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.339+0000 7f3bbbe00640 1 --2- 172.21.15.16:0/532622098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bbc10fbb0 0x7f3bbc1120c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:37.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.339+0000 7f3bbbe00640 1 --2- 172.21.15.16:0/532622098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bbc1020a0 0x7f3bbc10f4f0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:37.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.339+0000 7f3bbbe00640 1 --2- 172.21.15.16:0/532622098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3bbc1016c0 0x7f3bbc101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:37.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.339+0000 7f3bbbe00640 1 -- 172.21.15.16:0/532622098 >> 172.21.15.16:0/532622098 conn(0x7f3bbc078420 msgr2=0x7f3bbc078860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:37.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.339+0000 7f3bbbe00640 1 -- 172.21.15.16:0/532622098 shutdown_connections 2024-08-31T06:22:37.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.339+0000 7f3bbbe00640 1 -- 172.21.15.16:0/532622098 wait complete. 2024-08-31T06:22:37.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.341+0000 7f3bbbe00640 1 Processor -- start 2024-08-31T06:22:37.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.341+0000 7f3bbbe00640 1 -- start start 2024-08-31T06:22:37.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.342+0000 7f3bbbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bbc1016c0 0x7f3bbc19c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:37.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.342+0000 7f3bbbe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3bbc1020a0 0x7f3bbc19c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:37.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.342+0000 7f3bbbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bbc10fbb0 0x7f3bbc1a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:37.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.342+0000 7f3bbbe00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3bbc115810 con 0x7f3bbc10fbb0 2024-08-31T06:22:37.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.342+0000 7f3bbbe00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3bbc115690 con 0x7f3bbc1016c0 2024-08-31T06:22:37.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.342+0000 7f3bbbe00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3bbc115990 con 0x7f3bbc1020a0 2024-08-31T06:22:37.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.342+0000 7f3bbaa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bbc1016c0 0x7f3bbc19c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:37.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.342+0000 7f3bbaa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bbc1016c0 0x7f3bbc19c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:41928/0 (socket says 172.21.15.16:41928) 2024-08-31T06:22:37.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.342+0000 7f3bbaa00640 1 -- 172.21.15.16:0/2554341510 learned_addr learned my addr 172.21.15.16:0/2554341510 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:37.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.342+0000 7f3bbb400640 1 --2- 172.21.15.16:0/2554341510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bbc10fbb0 0x7f3bbc1a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:37.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.343+0000 7f3bba000640 1 --2- 172.21.15.16:0/2554341510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3bbc1020a0 0x7f3bbc19c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:37.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.343+0000 7f3bbaa00640 1 -- 172.21.15.16:0/2554341510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3bbc1020a0 msgr2=0x7f3bbc19c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:37.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.343+0000 7f3bbaa00640 1 --2- 172.21.15.16:0/2554341510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3bbc1020a0 0x7f3bbc19c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:37.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.343+0000 7f3bbaa00640 1 -- 172.21.15.16:0/2554341510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bbc10fbb0 msgr2=0x7f3bbc1a1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:37.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.343+0000 7f3bbaa00640 1 --2- 172.21.15.16:0/2554341510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bbc10fbb0 0x7f3bbc1a1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:37.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.343+0000 7f3bbaa00640 1 -- 172.21.15.16:0/2554341510 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bbc1a21f0 con 0x7f3bbc1016c0 2024-08-31T06:22:37.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.343+0000 7f3bbaa00640 1 --2- 172.21.15.16:0/2554341510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bbc1016c0 0x7f3bbc19c280 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f3bac00e9f0 tx=0x7f3bac00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:37.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.344+0000 7f3bb3400640 1 -- 172.21.15.16:0/2554341510 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bac00cd80 con 0x7f3bbc1016c0 2024-08-31T06:22:37.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.344+0000 7f3bb3400640 1 -- 172.21.15.16:0/2554341510 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bac0054d0 con 0x7f3bbc1016c0 2024-08-31T06:22:37.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.344+0000 7f3bb3400640 1 -- 172.21.15.16:0/2554341510 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bac010690 con 0x7f3bbc1016c0 2024-08-31T06:22:37.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.344+0000 7f3bbbe00640 1 -- 172.21.15.16:0/2554341510 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bbc1a24e0 con 0x7f3bbc1016c0 2024-08-31T06:22:37.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.344+0000 7f3bbbe00640 1 -- 172.21.15.16:0/2554341510 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bbc1a8c40 con 0x7f3bbc1016c0 2024-08-31T06:22:37.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.348+0000 7f3bb3400640 1 -- 172.21.15.16:0/2554341510 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f3bac004e90 con 0x7f3bbc1016c0 2024-08-31T06:22:37.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.349+0000 7f3bc8284740 1 Processor -- start 2024-08-31T06:22:37.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.349+0000 7f3bc8284740 1 -- start start 2024-08-31T06:22:37.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.350+0000 7f3bc8284740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b5a6cb2350 0x55b5a7060fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:37.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.350+0000 7f3bc8284740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5a6c89490 0x55b5a7061520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:37.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.350+0000 7f3bc8284740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5a6c91670 0x55b5a7061a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:37.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.350+0000 7f3bc8284740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b5a6fce750 con 0x55b5a6c89490 2024-08-31T06:22:37.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.350+0000 7f3bc8284740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b5a6fbdce0 con 0x55b5a6c91670 2024-08-31T06:22:37.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.350+0000 7f3bc8284740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b5a6fc1c90 con 0x55b5a6cb2350 2024-08-31T06:22:37.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.350+0000 7f3bbb400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b5a6cb2350 0x55b5a7060fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:37.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.350+0000 7f3bbb400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b5a6cb2350 0x55b5a7060fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:39994/0 (socket says 172.21.15.16:39994) 2024-08-31T06:22:37.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.350+0000 7f3bba000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5a6c91670 0x55b5a7061a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:37.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.350+0000 7f3bbb400640 1 -- 172.21.15.16:0/3836098722 learned_addr learned my addr 172.21.15.16:0/3836098722 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:37.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.351+0000 7f3bbaa00640 1 --2- 172.21.15.16:0/3836098722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5a6c89490 0x55b5a7061520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:37.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.351+0000 7f3bb3400640 1 --2- 172.21.15.16:0/2554341510 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3ba00b4da0 0x7f3ba00b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:37.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.351+0000 7f3bbb400640 1 -- 172.21.15.16:0/3836098722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5a6c91670 msgr2=0x55b5a7061a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:37.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.351+0000 7f3bbb400640 1 --2- 172.21.15.16:0/3836098722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5a6c91670 0x55b5a7061a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:37.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.351+0000 7f3bbb400640 1 -- 172.21.15.16:0/3836098722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5a6c89490 msgr2=0x55b5a7061520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:37.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.351+0000 7f3bbb400640 1 --2- 172.21.15.16:0/3836098722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5a6c89490 0x55b5a7061520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:37.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.351+0000 7f3bbb400640 1 -- 172.21.15.16:0/3836098722 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5a6d1d930 con 0x55b5a6cb2350 2024-08-31T06:22:37.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.351+0000 7f3bb3400640 1 -- 172.21.15.16:0/2554341510 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8470+0+0 (secure 0 0 0) 0x7f3bac0e59e0 con 0x7f3bbc1016c0 2024-08-31T06:22:37.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.352+0000 7f3bbb400640 1 --2- 172.21.15.16:0/2554341510 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3ba00b4da0 0x7f3ba00b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:37.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.352+0000 7f3bbb400640 1 --2- 172.21.15.16:0/3836098722 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b5a6cb2350 0x55b5a7060fe0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4009fd0 tx=0x7f3bb40118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=416d872d75d663b7 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:37.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.353+0000 7f3bab400640 1 -- 172.21.15.16:0/3836098722 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4004250 con 0x55b5a6cb2350 2024-08-31T06:22:37.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.353+0000 7f3bab400640 1 -- 172.21.15.16:0/3836098722 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bb400f4e0 con 0x55b5a6cb2350 2024-08-31T06:22:37.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.353+0000 7f3bab400640 1 -- 172.21.15.16:0/3836098722 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb401b670 con 0x55b5a6cb2350 2024-08-31T06:22:37.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.354+0000 7f3bc8284740 1 -- 172.21.15.16:0/3836098722 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b5a6cb2350 msgr2=0x55b5a7060fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:37.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.354+0000 7f3bbb400640 1 --2- 172.21.15.16:0/2554341510 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3ba00b4da0 0x7f3ba00b72b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4020000 tx=0x7f3bb401bd20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:37.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.354+0000 7f3bc8284740 1 --2- 172.21.15.16:0/3836098722 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b5a6cb2350 0x55b5a7060fe0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4009fd0 tx=0x7f3bb40118d0 comp rx=0 tx=0).stop 2024-08-31T06:22:37.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.354+0000 7f3bc8284740 1 -- 172.21.15.16:0/3836098722 shutdown_connections 2024-08-31T06:22:37.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.354+0000 7f3bc8284740 1 --2- 172.21.15.16:0/3836098722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5a6c91670 0x55b5a7061a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:37.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.354+0000 7f3bc8284740 1 --2- 172.21.15.16:0/3836098722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5a6c89490 0x55b5a7061520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:37.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.354+0000 7f3bc8284740 1 --2- 172.21.15.16:0/3836098722 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b5a6cb2350 0x55b5a7060fe0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:37.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.354+0000 7f3bc8284740 1 -- 172.21.15.16:0/3836098722 >> 172.21.15.16:0/3836098722 conn(0x55b5a6ca6740 msgr2=0x55b5a705eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:37.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.354+0000 7f3bc8284740 1 -- 172.21.15.16:0/3836098722 shutdown_connections 2024-08-31T06:22:37.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.354+0000 7f3bc8284740 1 -- 172.21.15.16:0/3836098722 wait complete. 2024-08-31T06:22:37.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.355+0000 7f3bc8284740 1 -- start start 2024-08-31T06:22:37.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.355+0000 7f3bc8284740 1 Processor -- start 2024-08-31T06:22:37.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.356+0000 7f3bc8284740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5a6c89490 0x55b5a706cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:37.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.356+0000 7f3bc8284740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5a6c91670 0x55b5a706d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:37.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.356+0000 7f3bc8284740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b5a6cb2350 0x55b5a7078b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:37.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.356+0000 7f3bc8284740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b5a6e20890 con 0x55b5a6c91670 2024-08-31T06:22:37.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.356+0000 7f3bc8284740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b5a6f44cd0 con 0x55b5a6c89490 2024-08-31T06:22:37.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.356+0000 7f3bc8284740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b5a706f860 con 0x55b5a6cb2350 2024-08-31T06:22:37.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.356+0000 7f3bc8284740 10 client.0 _ll_register_callbacks cb 0x55b5a6fb8ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:22:37.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.356+0000 7f3bbaa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5a6c89490 0x55b5a706cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:37.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.356+0000 7f3bbaa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5a6c89490 0x55b5a706cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:41942/0 (socket says 172.21.15.16:41942) 2024-08-31T06:22:37.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.356+0000 7f3bbaa00640 1 -- 172.21.15.16:0/1688660687 learned_addr learned my addr 172.21.15.16:0/1688660687 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:37.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.356+0000 7f3bbb400640 1 --2- 172.21.15.16:0/1688660687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b5a6cb2350 0x55b5a7078b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:37.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.356+0000 7f3bba000640 1 --2- 172.21.15.16:0/1688660687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5a6c91670 0x55b5a706d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:37.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.357+0000 7f3bbb400640 1 -- 172.21.15.16:0/1688660687 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5a6c89490 msgr2=0x55b5a706cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:37.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.357+0000 7f3bbb400640 1 --2- 172.21.15.16:0/1688660687 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5a6c89490 0x55b5a706cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:37.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.357+0000 7f3bbb400640 1 -- 172.21.15.16:0/1688660687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5a6c91670 msgr2=0x55b5a706d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:37.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.357+0000 7f3bbb400640 1 --2- 172.21.15.16:0/1688660687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5a6c91670 0x55b5a706d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:37.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.357+0000 7f3bbb400640 1 -- 172.21.15.16:0/1688660687 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5a6f8e1b0 con 0x55b5a6cb2350 2024-08-31T06:22:37.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.357+0000 7f3bbaa00640 1 --2- 172.21.15.16:0/1688660687 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5a6c89490 0x55b5a706cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:22:37.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.357+0000 7f3bba000640 1 --2- 172.21.15.16:0/1688660687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5a6c91670 0x55b5a706d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:22:37.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.358+0000 7f3bbb400640 1 --2- 172.21.15.16:0/1688660687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b5a6cb2350 0x55b5a7078b50 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4009fa0 tx=0x7f3bb4017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d19416c55d18e server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:37.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.358+0000 7f3b9fe00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:22:37.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.358+0000 7f3b9fe00640 1 -- 172.21.15.16:0/1688660687 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4019950 con 0x55b5a6cb2350 2024-08-31T06:22:37.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.359+0000 7f3b9fe00640 1 -- 172.21.15.16:0/1688660687 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8470+0+0 (secure 0 0 0) 0x7f3bb4019af0 con 0x55b5a6cb2350 2024-08-31T06:22:37.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.359+0000 7f3bc8284740 1 -- 172.21.15.16:0/1688660687 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bb400f4e0 con 0x55b5a6cb2350 2024-08-31T06:22:37.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.359+0000 7f3b9fe00640 1 -- 172.21.15.16:0/1688660687 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f3bb4019cf0 con 0x55b5a6cb2350 2024-08-31T06:22:37.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.359+0000 7f3bc8284740 10 client.29032 fetch_fsmap learned FSMap version 53 2024-08-31T06:22:37.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.359+0000 7f3bc8284740 1 -- 172.21.15.16:0/1688660687 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55b5a707b7a0 con 0x55b5a6cb2350 2024-08-31T06:22:37.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.360+0000 7f3b9fe00640 1 -- 172.21.15.16:0/1688660687 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bb400f4e0 con 0x55b5a6cb2350 2024-08-31T06:22:37.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.360+0000 7f3b9fe00640 1 -- 172.21.15.16:0/1688660687 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 53) ==== 8346+0+0 (secure 0 0 0) 0x7f3bb4004500 con 0x55b5a6cb2350 2024-08-31T06:22:37.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.360+0000 7f3bc8284740 10 client.29032 fetch_fsmap finished waiting for FSMap version 53 2024-08-31T06:22:37.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.360+0000 7f3bc8284740 10 client.29032 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:22:37.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.360+0000 7f3bc8284740 20 client.29032 populate_metadata read hostname 'smithi016' 2024-08-31T06:22:37.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.360+0000 7f3bc8284740 1 --2- 172.21.15.16:0/1688660687 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55b5a6c70cd0 0x55b5a707ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:37.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.360+0000 7f3bc8284740 4 client.29032 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:22:37.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.360+0000 7f3bc8284740 1 -- 172.21.15.16:0/1688660687 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5a6fd06a0 con 0x55b5a6c70cd0 2024-08-31T06:22:37.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.361+0000 7f3bbaa00640 1 --2- 172.21.15.16:0/1688660687 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55b5a6c70cd0 0x55b5a707ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:37.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.361+0000 7f3bbaa00640 1 --2- 172.21.15.16:0/1688660687 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55b5a6c70cd0 0x55b5a707ec10 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be60867eecbcae2e server_cookie=9e864e220cd0aba0 in_seq=0 out_seq=0 2024-08-31T06:22:37.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.361+0000 7f3b9fe00640 10 client.29032 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:22:37.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.362+0000 7f3b9fe00640 1 -- 172.21.15.16:0/1688660687 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b5a6fd06a0 con 0x55b5a6c70cd0 2024-08-31T06:22:37.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.362+0000 7f3b9fe00640 10 client.29032 handle_command_reply: tid=0 2024-08-31T06:22:37.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.362+0000 7f3bc8284740 1 client.29032 shutdown 2024-08-31T06:22:37.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.362+0000 7f3bc8284740 1 -- 172.21.15.16:0/1688660687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b5a6cb2350 msgr2=0x55b5a7078b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:37.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.362+0000 7f3bc8284740 1 --2- 172.21.15.16:0/1688660687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b5a6cb2350 0x55b5a7078b50 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4009fa0 tx=0x7f3bb4017000 comp rx=0 tx=0).stop 2024-08-31T06:22:37.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.363+0000 7f3bc8284740 1 -- 172.21.15.16:0/1688660687 shutdown_connections 2024-08-31T06:22:37.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.363+0000 7f3bc8284740 1 --2- 172.21.15.16:0/1688660687 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55b5a6c70cd0 0x55b5a707ec10 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:37.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.363+0000 7f3bc8284740 1 --2- 172.21.15.16:0/1688660687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b5a6cb2350 0x55b5a7078b50 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:37.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.363+0000 7f3bc8284740 1 --2- 172.21.15.16:0/1688660687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5a6c91670 0x55b5a706d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:37.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.363+0000 7f3bc8284740 1 --2- 172.21.15.16:0/1688660687 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5a6c89490 0x55b5a706cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:37.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.363+0000 7f3bc8284740 1 -- 172.21.15.16:0/1688660687 >> 172.21.15.16:0/1688660687 conn(0x55b5a6ca6740 msgr2=0x55b5a70629d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:37.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.363+0000 7f3b9fe00640 0 client.29032 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:22:37.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.363+0000 7f3bc8284740 1 -- 172.21.15.16:0/1688660687 shutdown_connections 2024-08-31T06:22:37.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.363+0000 7f3bc8284740 1 -- 172.21.15.16:0/1688660687 wait complete. 2024-08-31T06:22:37.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.363+0000 7f3bc8284740 20 client.29032 trim_cache size 0 max 16384 2024-08-31T06:22:37.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.386+0000 7f3bc8284740 1 Processor -- start 2024-08-31T06:22:37.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.386+0000 7f3bc8284740 1 -- start start 2024-08-31T06:22:37.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.387+0000 7f3bc8284740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b5a6c89490 0x55b5a706b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:37.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.387+0000 7f3bc8284740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5a6c91670 0x55b5a706b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:37.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.387+0000 7f3bc8284740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5a6cb2350 0x55b5a7078a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:37.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.387+0000 7f3bc8284740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3bbc1af9d0 con 0x55b5a6c91670 2024-08-31T06:22:37.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.387+0000 7f3bc8284740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3bbc1ad2a0 con 0x55b5a6cb2350 2024-08-31T06:22:37.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.387+0000 7f3bc8284740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3bbc1ab110 con 0x55b5a6c89490 2024-08-31T06:22:37.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.387+0000 7f3bba000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5a6c91670 0x55b5a706b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:37.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.387+0000 7f3bbaa00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b5a6c89490 0x55b5a706b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:37.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.387+0000 7f3bba000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5a6c91670 0x55b5a706b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51600/0 (socket says 172.21.15.16:51600) 2024-08-31T06:22:37.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.387+0000 7f3bbaa00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b5a6c89490 0x55b5a706b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:40016/0 (socket says 172.21.15.16:40016) 2024-08-31T06:22:37.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.387+0000 7f3bba000640 1 -- 172.21.15.16:0/2077713020 learned_addr learned my addr 172.21.15.16:0/2077713020 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:37.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.387+0000 7f3bbb400640 1 --2- 172.21.15.16:0/2077713020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5a6cb2350 0x55b5a7078a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:37.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.388+0000 7f3bbaa00640 1 -- 172.21.15.16:0/2077713020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5a6cb2350 msgr2=0x55b5a7078a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:37.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.388+0000 7f3bbaa00640 1 --2- 172.21.15.16:0/2077713020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5a6cb2350 0x55b5a7078a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:37.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.388+0000 7f3bbaa00640 1 -- 172.21.15.16:0/2077713020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5a6c91670 msgr2=0x55b5a706b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:37.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.388+0000 7f3bbaa00640 1 --2- 172.21.15.16:0/2077713020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5a6c91670 0x55b5a706b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:37.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.388+0000 7f3bbaa00640 1 -- 172.21.15.16:0/2077713020 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5a70635c0 con 0x55b5a6c89490 2024-08-31T06:22:37.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.388+0000 7f3bba000640 1 --2- 172.21.15.16:0/2077713020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5a6c91670 0x55b5a706b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:22:37.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.388+0000 7f3bbaa00640 1 --2- 172.21.15.16:0/2077713020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b5a6c89490 0x55b5a706b150 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f3bac00e9c0 tx=0x7f3bac0e7e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92b5f3ac844c1765 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:37.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.389+0000 7f3baaa00640 1 -- 172.21.15.16:0/2077713020 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bac0abe70 con 0x55b5a6c89490 2024-08-31T06:22:37.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.389+0000 7f3baaa00640 1 -- 172.21.15.16:0/2077713020 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bac0d0070 con 0x55b5a6c89490 2024-08-31T06:22:37.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.389+0000 7f3baaa00640 1 -- 172.21.15.16:0/2077713020 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bac0b0910 con 0x55b5a6c89490 2024-08-31T06:22:37.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.390+0000 7f3bc8284740 1 -- 172.21.15.16:0/2077713020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b5a6c89490 msgr2=0x55b5a706b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:37.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.390+0000 7f3bc8284740 1 --2- 172.21.15.16:0/2077713020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b5a6c89490 0x55b5a706b150 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f3bac00e9c0 tx=0x7f3bac0e7e20 comp rx=0 tx=0).stop 2024-08-31T06:22:37.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.390+0000 7f3bc8284740 1 -- 172.21.15.16:0/2077713020 shutdown_connections 2024-08-31T06:22:37.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.390+0000 7f3bc8284740 1 --2- 172.21.15.16:0/2077713020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5a6cb2350 0x55b5a7078a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:37.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.390+0000 7f3bc8284740 1 --2- 172.21.15.16:0/2077713020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5a6c91670 0x55b5a706b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:37.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.390+0000 7f3bc8284740 1 --2- 172.21.15.16:0/2077713020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b5a6c89490 0x55b5a706b150 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:37.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.390+0000 7f3bc8284740 1 -- 172.21.15.16:0/2077713020 >> 172.21.15.16:0/2077713020 conn(0x55b5a6ca6740 msgr2=0x55b5a7067a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:37.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.390+0000 7f3bc8284740 1 -- 172.21.15.16:0/2077713020 shutdown_connections 2024-08-31T06:22:37.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.390+0000 7f3bc8284740 1 -- 172.21.15.16:0/2077713020 wait complete. 2024-08-31T06:22:37.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.391+0000 7f3bc8284740 1 -- start start 2024-08-31T06:22:37.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.391+0000 7f3bc8284740 1 Processor -- start 2024-08-31T06:22:37.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.392+0000 7f3bc8284740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5a6c89490 0x55b5a7075430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:37.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.392+0000 7f3bc8284740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b5a6c91670 0x55b5a70708e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:37.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.392+0000 7f3bc8284740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5a6cb2350 0x55b5a7070fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:37.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.392+0000 7f3bc8284740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3bbc1ab760 con 0x55b5a6c89490 2024-08-31T06:22:37.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.392+0000 7f3bc8284740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3bbc1acbd0 con 0x55b5a6cb2350 2024-08-31T06:22:37.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.392+0000 7f3bc8284740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3bbc1abe30 con 0x55b5a6c91670 2024-08-31T06:22:37.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.392+0000 7f3bc8284740 10 client.0 _ll_register_callbacks cb 0x7f3bb40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:22:37.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.392+0000 7f3bbaa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5a6c89490 0x55b5a7075430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:37.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.392+0000 7f3bba000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b5a6c91670 0x55b5a70708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:37.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.392+0000 7f3bbaa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5a6c89490 0x55b5a7075430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51612/0 (socket says 172.21.15.16:51612) 2024-08-31T06:22:37.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.392+0000 7f3bbaa00640 1 -- 172.21.15.16:0/1228007786 learned_addr learned my addr 172.21.15.16:0/1228007786 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:37.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.392+0000 7f3bbb400640 1 --2- 172.21.15.16:0/1228007786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5a6cb2350 0x55b5a7070fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:37.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.393+0000 7f3bbaa00640 1 -- 172.21.15.16:0/1228007786 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b5a6c91670 msgr2=0x55b5a70708e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:37.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.393+0000 7f3bbaa00640 1 --2- 172.21.15.16:0/1228007786 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b5a6c91670 0x55b5a70708e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:37.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.393+0000 7f3bbaa00640 1 -- 172.21.15.16:0/1228007786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5a6cb2350 msgr2=0x55b5a7070fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:37.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.393+0000 7f3bbaa00640 1 --2- 172.21.15.16:0/1228007786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5a6cb2350 0x55b5a7070fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:37.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.393+0000 7f3bbaa00640 1 -- 172.21.15.16:0/1228007786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5a70e8580 con 0x55b5a6c89490 2024-08-31T06:22:37.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.393+0000 7f3bba000640 1 --2- 172.21.15.16:0/1228007786 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b5a6c91670 0x55b5a70708e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:22:37.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.393+0000 7f3bbaa00640 1 --2- 172.21.15.16:0/1228007786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5a6c89490 0x55b5a7075430 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f3bac00c6a0 tx=0x7f3bac0ac110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e9ea1828689d20 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:37.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.393+0000 7f3b9fe00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:22:37.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.393+0000 7f3b9fe00640 1 -- 172.21.15.16:0/1228007786 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bac097020 con 0x55b5a6c89490 2024-08-31T06:22:37.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.394+0000 7f3b9fe00640 1 -- 172.21.15.16:0/1228007786 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8470+0+0 (secure 0 0 0) 0x7f3bac0973c0 con 0x55b5a6c89490 2024-08-31T06:22:37.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.394+0000 7f3bc8284740 1 -- 172.21.15.16:0/1228007786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bac0d0070 con 0x55b5a6c89490 2024-08-31T06:22:37.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.394+0000 7f3b9fe00640 1 -- 172.21.15.16:0/1228007786 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f3bac005680 con 0x55b5a6c89490 2024-08-31T06:22:37.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.394+0000 7f3bc8284740 10 client.18516 fetch_fsmap learned FSMap version 53 2024-08-31T06:22:37.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.394+0000 7f3bc8284740 1 -- 172.21.15.16:0/1228007786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55b5a70e96c0 con 0x55b5a6c89490 2024-08-31T06:22:37.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.394+0000 7f3b9fe00640 1 -- 172.21.15.16:0/1228007786 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bac0c6e50 con 0x55b5a6c89490 2024-08-31T06:22:37.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.395+0000 7f3b9fe00640 1 -- 172.21.15.16:0/1228007786 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 53) ==== 8346+0+0 (secure 0 0 0) 0x7f3bac0a7a10 con 0x55b5a6c89490 2024-08-31T06:22:37.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.395+0000 7f3bc8284740 10 client.18516 fetch_fsmap finished waiting for FSMap version 53 2024-08-31T06:22:37.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.395+0000 7f3bc8284740 10 client.18516 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:22:37.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.395+0000 7f3bc8284740 20 client.18516 populate_metadata read hostname 'smithi016' 2024-08-31T06:22:37.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.395+0000 7f3bc8284740 1 --2- 172.21.15.16:0/1228007786 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55b5a6c46080 0x55b5a70ebe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:37.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.395+0000 7f3bc8284740 4 client.18516 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-31T06:22:37.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.395+0000 7f3bc8284740 1 -- 172.21.15.16:0/1228007786 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b5a6fcc140 con 0x55b5a6c46080 2024-08-31T06:22:37.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.395+0000 7f3bba000640 1 --2- 172.21.15.16:0/1228007786 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55b5a6c46080 0x55b5a70ebe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:37.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.396+0000 7f3bba000640 1 --2- 172.21.15.16:0/1228007786 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55b5a6c46080 0x55b5a70ebe70 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d69d8c8d287a7609 server_cookie=f14bfae1215b71af in_seq=0 out_seq=0 2024-08-31T06:22:37.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.396+0000 7f3b9fe00640 10 client.18516 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:22:37.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.397+0000 7f3b9fe00640 1 -- 172.21.15.16:0/1228007786 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b5a6fcc140 con 0x55b5a6c46080 2024-08-31T06:22:37.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.397+0000 7f3b9fe00640 10 client.18516 handle_command_reply: tid=0 2024-08-31T06:22:37.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.397+0000 7f3bc8284740 1 client.18516 shutdown 2024-08-31T06:22:37.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.398+0000 7f3bc8284740 1 -- 172.21.15.16:0/1228007786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5a6c89490 msgr2=0x55b5a7075430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:37.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.398+0000 7f3bc8284740 1 --2- 172.21.15.16:0/1228007786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5a6c89490 0x55b5a7075430 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f3bac00c6a0 tx=0x7f3bac0ac110 comp rx=0 tx=0).stop 2024-08-31T06:22:37.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.398+0000 7f3bc8284740 1 -- 172.21.15.16:0/1228007786 shutdown_connections 2024-08-31T06:22:37.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.398+0000 7f3bc8284740 1 --2- 172.21.15.16:0/1228007786 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55b5a6c46080 0x55b5a70ebe70 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:37.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.398+0000 7f3bc8284740 1 --2- 172.21.15.16:0/1228007786 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b5a6c91670 0x55b5a70708e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:37.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.398+0000 7f3bc8284740 1 --2- 172.21.15.16:0/1228007786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5a6cb2350 0x55b5a7070fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:37.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.398+0000 7f3bc8284740 1 --2- 172.21.15.16:0/1228007786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b5a6c89490 0x55b5a7075430 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:37.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.398+0000 7f3bc8284740 1 -- 172.21.15.16:0/1228007786 >> 172.21.15.16:0/1228007786 conn(0x55b5a6ca6740 msgr2=0x55b5a706b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:37.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.398+0000 7f3b9fe00640 0 client.18516 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:22:37.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.398+0000 7f3bc8284740 1 -- 172.21.15.16:0/1228007786 shutdown_connections 2024-08-31T06:22:37.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.398+0000 7f3bc8284740 1 -- 172.21.15.16:0/1228007786 wait complete. 2024-08-31T06:22:37.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.399+0000 7f3bc8284740 20 client.18516 trim_cache size 0 max 16384 2024-08-31T06:22:37.402 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:22:37.402 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-08-31T06:22:37.402 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "ab1ded0f-9efd-4559-88af-72427ba8460d", 2024-08-31T06:22:37.402 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-08-31T06:22:37.402 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:22:37.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.400+0000 7f3bbbe00640 1 -- 172.21.15.16:0/2554341510 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3ba00b4da0 msgr2=0x7f3ba00b72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:37.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.400+0000 7f3bbbe00640 1 --2- 172.21.15.16:0/2554341510 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3ba00b4da0 0x7f3ba00b72b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4020000 tx=0x7f3bb401bd20 comp rx=0 tx=0).stop 2024-08-31T06:22:37.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.400+0000 7f3bbbe00640 1 -- 172.21.15.16:0/2554341510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bbc1016c0 msgr2=0x7f3bbc19c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:37.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.400+0000 7f3bbbe00640 1 --2- 172.21.15.16:0/2554341510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bbc1016c0 0x7f3bbc19c280 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f3bac00e9f0 tx=0x7f3bac00edf0 comp rx=0 tx=0).stop 2024-08-31T06:22:37.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.400+0000 7f3bbbe00640 1 -- 172.21.15.16:0/2554341510 shutdown_connections 2024-08-31T06:22:37.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.400+0000 7f3bbbe00640 1 --2- 172.21.15.16:0/2554341510 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3ba00b4da0 0x7f3ba00b72b0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:37.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.400+0000 7f3bbbe00640 1 --2- 172.21.15.16:0/2554341510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3bbc1020a0 0x7f3bbc19c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:37.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.400+0000 7f3bbbe00640 1 --2- 172.21.15.16:0/2554341510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bbc10fbb0 0x7f3bbc1a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:37.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.400+0000 7f3bbbe00640 1 --2- 172.21.15.16:0/2554341510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bbc1016c0 0x7f3bbc19c280 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:37.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.400+0000 7f3bbbe00640 1 -- 172.21.15.16:0/2554341510 >> 172.21.15.16:0/2554341510 conn(0x7f3bbc078420 msgr2=0x7f3bbc110310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:37.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.401+0000 7f3bbbe00640 1 -- 172.21.15.16:0/2554341510 shutdown_connections 2024-08-31T06:22:37.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:37.401+0000 7f3bbbe00640 1 -- 172.21.15.16:0/2554341510 wait complete. 2024-08-31T06:22:37.872 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-08-31T06:22:38.005 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/: 2024-08-31T06:22:38.005 INFO:teuthology.orchestra.run.smithi016.stdout:total 0 2024-08-31T06:22:38.006 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 31 06:07 . 2024-08-31T06:22:38.006 INFO:teuthology.orchestra.run.smithi016.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Aug 31 06:07 .. 2024-08-31T06:22:38.006 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 31 06:22 data 2024-08-31T06:22:38.006 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:22:38.006 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data: 2024-08-31T06:22:38.006 INFO:teuthology.orchestra.run.smithi016.stdout:total 36 2024-08-31T06:22:38.006 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 31 06:22 . 2024-08-31T06:22:38.006 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 31 06:07 .. 2024-08-31T06:22:38.006 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Aug 31 06:07 base 2024-08-31T06:22:38.006 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 Aug 31 06:07 current_logfiles 2024-08-31T06:22:38.006 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Aug 31 06:07 global 2024-08-31T06:22:38.006 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 31 06:07 log 2024-08-31T06:22:38.006 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 pg_commit_ts 2024-08-31T06:22:38.006 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 pg_dynshmem 2024-08-31T06:22:38.007 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Aug 31 06:07 pg_hba.conf 2024-08-31T06:22:38.007 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Aug 31 06:07 pg_ident.conf 2024-08-31T06:22:38.007 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 31 06:22 pg_logical 2024-08-31T06:22:38.007 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 31 06:07 pg_multixact 2024-08-31T06:22:38.007 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 pg_notify 2024-08-31T06:22:38.007 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 pg_replslot 2024-08-31T06:22:38.007 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 pg_serial 2024-08-31T06:22:38.007 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 pg_snapshots 2024-08-31T06:22:38.007 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 31 06:22 pg_stat 2024-08-31T06:22:38.007 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:22 pg_stat_tmp 2024-08-31T06:22:38.008 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 31 06:22 pg_subtrans 2024-08-31T06:22:38.008 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 pg_tblspc 2024-08-31T06:22:38.008 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 pg_twophase 2024-08-31T06:22:38.008 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 31 06:07 PG_VERSION 2024-08-31T06:22:38.008 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 61 Aug 31 06:22 pg_wal 2024-08-31T06:22:38.008 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 31 06:22 pg_xact 2024-08-31T06:22:38.008 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Aug 31 06:07 postgresql.auto.conf 2024-08-31T06:22:38.008 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28074 Aug 31 06:07 postgresql.conf 2024-08-31T06:22:38.008 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 Aug 31 06:07 postmaster.opts 2024-08-31T06:22:38.008 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:22:38.008 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/base: 2024-08-31T06:22:38.008 INFO:teuthology.orchestra.run.smithi016.stdout:total 0 2024-08-31T06:22:38.008 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Aug 31 06:07 . 2024-08-31T06:22:38.009 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 31 06:22 .. 2024-08-31T06:22:38.009 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Aug 31 06:07 1 2024-08-31T06:22:38.009 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Aug 31 06:07 13493 2024-08-31T06:22:38.009 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 Aug 31 06:13 13494 2024-08-31T06:22:38.009 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 pgsql_tmp 2024-08-31T06:22:38.009 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:22:38.009 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/base/1: 2024-08-31T06:22:38.009 INFO:teuthology.orchestra.run.smithi016.stdout:total 7777 2024-08-31T06:22:38.009 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Aug 31 06:07 . 2024-08-31T06:22:38.009 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Aug 31 06:07 .. 2024-08-31T06:22:38.009 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 112 2024-08-31T06:22:38.009 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 113 2024-08-31T06:22:38.009 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 31 06:07 1247 2024-08-31T06:22:38.010 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 1247_fsm 2024-08-31T06:22:38.010 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 1247_vm 2024-08-31T06:22:38.010 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Aug 31 06:07 1249 2024-08-31T06:22:38.010 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 1249_fsm 2024-08-31T06:22:38.010 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 1249_vm 2024-08-31T06:22:38.010 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Aug 31 06:07 1255 2024-08-31T06:22:38.010 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 1255_fsm 2024-08-31T06:22:38.010 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 1255_vm 2024-08-31T06:22:38.010 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Aug 31 06:07 1259 2024-08-31T06:22:38.010 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 1259_fsm 2024-08-31T06:22:38.010 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 1259_vm 2024-08-31T06:22:38.010 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Aug 31 06:07 13344 2024-08-31T06:22:38.010 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 13344_fsm 2024-08-31T06:22:38.010 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13344_vm 2024-08-31T06:22:38.010 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 13346 2024-08-31T06:22:38.010 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13348 2024-08-31T06:22:38.011 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13349 2024-08-31T06:22:38.011 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 13349_fsm 2024-08-31T06:22:38.011 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13349_vm 2024-08-31T06:22:38.011 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 13351 2024-08-31T06:22:38.011 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13353 2024-08-31T06:22:38.011 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13354 2024-08-31T06:22:38.011 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 13354_fsm 2024-08-31T06:22:38.011 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13354_vm 2024-08-31T06:22:38.011 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 13356 2024-08-31T06:22:38.011 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13358 2024-08-31T06:22:38.011 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13359 2024-08-31T06:22:38.011 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 13359_fsm 2024-08-31T06:22:38.011 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13359_vm 2024-08-31T06:22:38.011 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 13361 2024-08-31T06:22:38.011 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13363 2024-08-31T06:22:38.011 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 1417 2024-08-31T06:22:38.012 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 1418 2024-08-31T06:22:38.012 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 174 2024-08-31T06:22:38.012 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 175 2024-08-31T06:22:38.012 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2187 2024-08-31T06:22:38.012 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2224 2024-08-31T06:22:38.012 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2328 2024-08-31T06:22:38.012 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2336 2024-08-31T06:22:38.012 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2337 2024-08-31T06:22:38.012 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2579 2024-08-31T06:22:38.012 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2600 2024-08-31T06:22:38.012 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2600_fsm 2024-08-31T06:22:38.012 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2600_vm 2024-08-31T06:22:38.012 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2601 2024-08-31T06:22:38.012 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2601_fsm 2024-08-31T06:22:38.012 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2601_vm 2024-08-31T06:22:38.013 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 31 06:07 2602 2024-08-31T06:22:38.013 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2602_fsm 2024-08-31T06:22:38.013 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2602_vm 2024-08-31T06:22:38.013 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 2603 2024-08-31T06:22:38.013 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2603_fsm 2024-08-31T06:22:38.013 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2603_vm 2024-08-31T06:22:38.013 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2604 2024-08-31T06:22:38.013 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2605 2024-08-31T06:22:38.013 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2605_fsm 2024-08-31T06:22:38.013 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2605_vm 2024-08-31T06:22:38.013 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2606 2024-08-31T06:22:38.013 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2606_fsm 2024-08-31T06:22:38.013 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2606_vm 2024-08-31T06:22:38.014 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2607 2024-08-31T06:22:38.014 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2607_fsm 2024-08-31T06:22:38.014 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2607_vm 2024-08-31T06:22:38.014 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Aug 31 06:07 2608 2024-08-31T06:22:38.014 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2608_fsm 2024-08-31T06:22:38.014 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2608_vm 2024-08-31T06:22:38.014 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Aug 31 06:07 2609 2024-08-31T06:22:38.014 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2609_fsm 2024-08-31T06:22:38.014 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2609_vm 2024-08-31T06:22:38.014 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 2610 2024-08-31T06:22:38.014 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2610_fsm 2024-08-31T06:22:38.014 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2610_vm 2024-08-31T06:22:38.014 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2611 2024-08-31T06:22:38.014 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2612 2024-08-31T06:22:38.014 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2612_fsm 2024-08-31T06:22:38.014 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2612_vm 2024-08-31T06:22:38.015 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2613 2024-08-31T06:22:38.015 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2615 2024-08-31T06:22:38.015 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2615_fsm 2024-08-31T06:22:38.015 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2615_vm 2024-08-31T06:22:38.015 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2616 2024-08-31T06:22:38.015 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2616_fsm 2024-08-31T06:22:38.015 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2616_vm 2024-08-31T06:22:38.015 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Aug 31 06:07 2617 2024-08-31T06:22:38.015 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2617_fsm 2024-08-31T06:22:38.015 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2617_vm 2024-08-31T06:22:38.015 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Aug 31 06:07 2618 2024-08-31T06:22:38.015 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2618_fsm 2024-08-31T06:22:38.015 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2618_vm 2024-08-31T06:22:38.015 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Aug 31 06:07 2619 2024-08-31T06:22:38.015 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2619_fsm 2024-08-31T06:22:38.015 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2619_vm 2024-08-31T06:22:38.016 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2620 2024-08-31T06:22:38.016 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2650 2024-08-31T06:22:38.016 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2651 2024-08-31T06:22:38.016 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2652 2024-08-31T06:22:38.016 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 31 06:07 2653 2024-08-31T06:22:38.016 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 31 06:07 2654 2024-08-31T06:22:38.016 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 2655 2024-08-31T06:22:38.016 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2656 2024-08-31T06:22:38.016 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2657 2024-08-31T06:22:38.016 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Aug 31 06:07 2658 2024-08-31T06:22:38.016 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 31 06:07 2659 2024-08-31T06:22:38.016 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2660 2024-08-31T06:22:38.016 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2661 2024-08-31T06:22:38.016 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 2662 2024-08-31T06:22:38.016 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 31 06:07 2663 2024-08-31T06:22:38.016 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2664 2024-08-31T06:22:38.017 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2665 2024-08-31T06:22:38.017 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2666 2024-08-31T06:22:38.017 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2667 2024-08-31T06:22:38.017 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2668 2024-08-31T06:22:38.017 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2669 2024-08-31T06:22:38.017 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2670 2024-08-31T06:22:38.017 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Aug 31 06:07 2673 2024-08-31T06:22:38.017 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Aug 31 06:07 2674 2024-08-31T06:22:38.018 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Aug 31 06:07 2675 2024-08-31T06:22:38.018 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2678 2024-08-31T06:22:38.018 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2679 2024-08-31T06:22:38.018 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2680 2024-08-31T06:22:38.018 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2681 2024-08-31T06:22:38.018 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2682 2024-08-31T06:22:38.019 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2683 2024-08-31T06:22:38.019 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2684 2024-08-31T06:22:38.019 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2685 2024-08-31T06:22:38.019 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2686 2024-08-31T06:22:38.019 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2687 2024-08-31T06:22:38.019 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 31 06:07 2688 2024-08-31T06:22:38.019 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 31 06:07 2689 2024-08-31T06:22:38.019 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Aug 31 06:07 2690 2024-08-31T06:22:38.019 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Aug 31 06:07 2691 2024-08-31T06:22:38.019 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2692 2024-08-31T06:22:38.019 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2693 2024-08-31T06:22:38.019 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2696 2024-08-31T06:22:38.019 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2699 2024-08-31T06:22:38.019 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2701 2024-08-31T06:22:38.019 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2702 2024-08-31T06:22:38.020 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 2703 2024-08-31T06:22:38.020 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 31 06:07 2704 2024-08-31T06:22:38.020 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2753 2024-08-31T06:22:38.020 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2753_fsm 2024-08-31T06:22:38.020 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2753_vm 2024-08-31T06:22:38.020 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2754 2024-08-31T06:22:38.020 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2755 2024-08-31T06:22:38.020 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 2756 2024-08-31T06:22:38.020 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 2757 2024-08-31T06:22:38.020 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2830 2024-08-31T06:22:38.020 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2831 2024-08-31T06:22:38.020 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2832 2024-08-31T06:22:38.020 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2833 2024-08-31T06:22:38.021 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2834 2024-08-31T06:22:38.021 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2835 2024-08-31T06:22:38.021 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2836 2024-08-31T06:22:38.021 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2837 2024-08-31T06:22:38.021 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Aug 31 06:07 2838 2024-08-31T06:22:38.021 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2838_fsm 2024-08-31T06:22:38.021 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2838_vm 2024-08-31T06:22:38.021 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2839 2024-08-31T06:22:38.021 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2840 2024-08-31T06:22:38.021 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2840_fsm 2024-08-31T06:22:38.021 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2840_vm 2024-08-31T06:22:38.022 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2841 2024-08-31T06:22:38.022 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2995 2024-08-31T06:22:38.022 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2996 2024-08-31T06:22:38.022 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3079 2024-08-31T06:22:38.022 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3079_fsm 2024-08-31T06:22:38.022 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3079_vm 2024-08-31T06:22:38.022 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3080 2024-08-31T06:22:38.022 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3081 2024-08-31T06:22:38.022 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 31 06:07 3085 2024-08-31T06:22:38.022 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3118 2024-08-31T06:22:38.022 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3119 2024-08-31T06:22:38.022 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Aug 31 06:07 3164 2024-08-31T06:22:38.022 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3256 2024-08-31T06:22:38.022 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3257 2024-08-31T06:22:38.022 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3258 2024-08-31T06:22:38.023 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3350 2024-08-31T06:22:38.023 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3351 2024-08-31T06:22:38.023 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3379 2024-08-31T06:22:38.023 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3380 2024-08-31T06:22:38.023 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3381 2024-08-31T06:22:38.023 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3394 2024-08-31T06:22:38.023 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3394_fsm 2024-08-31T06:22:38.023 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3394_vm 2024-08-31T06:22:38.023 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3395 2024-08-31T06:22:38.023 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3429 2024-08-31T06:22:38.023 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3430 2024-08-31T06:22:38.023 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3431 2024-08-31T06:22:38.024 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3433 2024-08-31T06:22:38.024 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3439 2024-08-31T06:22:38.024 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3440 2024-08-31T06:22:38.024 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 31 06:07 3455 2024-08-31T06:22:38.024 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Aug 31 06:07 3456 2024-08-31T06:22:38.024 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3456_fsm 2024-08-31T06:22:38.024 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3456_vm 2024-08-31T06:22:38.024 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3466 2024-08-31T06:22:38.024 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3467 2024-08-31T06:22:38.024 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3468 2024-08-31T06:22:38.024 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3501 2024-08-31T06:22:38.024 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3502 2024-08-31T06:22:38.025 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3503 2024-08-31T06:22:38.025 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3534 2024-08-31T06:22:38.025 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3541 2024-08-31T06:22:38.025 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3541_fsm 2024-08-31T06:22:38.025 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3541_vm 2024-08-31T06:22:38.025 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3542 2024-08-31T06:22:38.025 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3574 2024-08-31T06:22:38.025 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3575 2024-08-31T06:22:38.025 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3576 2024-08-31T06:22:38.025 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3596 2024-08-31T06:22:38.025 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3597 2024-08-31T06:22:38.025 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3598 2024-08-31T06:22:38.025 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3599 2024-08-31T06:22:38.025 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3600 2024-08-31T06:22:38.025 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3600_fsm 2024-08-31T06:22:38.026 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3600_vm 2024-08-31T06:22:38.026 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3601 2024-08-31T06:22:38.026 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3601_fsm 2024-08-31T06:22:38.026 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3601_vm 2024-08-31T06:22:38.026 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3602 2024-08-31T06:22:38.026 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3602_fsm 2024-08-31T06:22:38.026 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3602_vm 2024-08-31T06:22:38.026 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3603 2024-08-31T06:22:38.026 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3603_fsm 2024-08-31T06:22:38.026 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3603_vm 2024-08-31T06:22:38.026 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3604 2024-08-31T06:22:38.026 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3605 2024-08-31T06:22:38.026 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3606 2024-08-31T06:22:38.026 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3607 2024-08-31T06:22:38.027 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3608 2024-08-31T06:22:38.027 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 3609 2024-08-31T06:22:38.027 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3712 2024-08-31T06:22:38.027 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3764 2024-08-31T06:22:38.027 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3764_fsm 2024-08-31T06:22:38.027 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3764_vm 2024-08-31T06:22:38.027 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3766 2024-08-31T06:22:38.027 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3767 2024-08-31T06:22:38.027 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3997 2024-08-31T06:22:38.027 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4143 2024-08-31T06:22:38.027 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4144 2024-08-31T06:22:38.027 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4145 2024-08-31T06:22:38.027 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4146 2024-08-31T06:22:38.028 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4147 2024-08-31T06:22:38.028 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4148 2024-08-31T06:22:38.028 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4149 2024-08-31T06:22:38.028 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4150 2024-08-31T06:22:38.028 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4151 2024-08-31T06:22:38.028 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4152 2024-08-31T06:22:38.028 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4153 2024-08-31T06:22:38.028 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4154 2024-08-31T06:22:38.028 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4155 2024-08-31T06:22:38.028 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4156 2024-08-31T06:22:38.028 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4157 2024-08-31T06:22:38.028 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4158 2024-08-31T06:22:38.028 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4159 2024-08-31T06:22:38.028 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4160 2024-08-31T06:22:38.028 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4161 2024-08-31T06:22:38.029 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4162 2024-08-31T06:22:38.033 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4163 2024-08-31T06:22:38.034 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4164 2024-08-31T06:22:38.034 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4165 2024-08-31T06:22:38.034 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4166 2024-08-31T06:22:38.034 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4167 2024-08-31T06:22:38.034 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4168 2024-08-31T06:22:38.034 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4169 2024-08-31T06:22:38.034 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4170 2024-08-31T06:22:38.034 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4171 2024-08-31T06:22:38.034 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4172 2024-08-31T06:22:38.034 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4173 2024-08-31T06:22:38.034 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4174 2024-08-31T06:22:38.034 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 5002 2024-08-31T06:22:38.034 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 548 2024-08-31T06:22:38.035 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 549 2024-08-31T06:22:38.035 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 6102 2024-08-31T06:22:38.035 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 6104 2024-08-31T06:22:38.035 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 6106 2024-08-31T06:22:38.035 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 6110 2024-08-31T06:22:38.035 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 6111 2024-08-31T06:22:38.035 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 6112 2024-08-31T06:22:38.035 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 6113 2024-08-31T06:22:38.035 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 6117 2024-08-31T06:22:38.035 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 826 2024-08-31T06:22:38.035 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 827 2024-08-31T06:22:38.035 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 828 2024-08-31T06:22:38.035 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Aug 31 06:07 pg_filenode.map 2024-08-31T06:22:38.035 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 31 06:07 PG_VERSION 2024-08-31T06:22:38.035 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:22:38.035 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-08-31T06:22:38.036 INFO:teuthology.orchestra.run.smithi016.stdout:total 7777 2024-08-31T06:22:38.036 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Aug 31 06:07 . 2024-08-31T06:22:38.036 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Aug 31 06:07 .. 2024-08-31T06:22:38.036 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 112 2024-08-31T06:22:38.036 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 113 2024-08-31T06:22:38.036 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 31 06:07 1247 2024-08-31T06:22:38.036 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 1247_fsm 2024-08-31T06:22:38.036 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 1247_vm 2024-08-31T06:22:38.036 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Aug 31 06:07 1249 2024-08-31T06:22:38.036 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 1249_fsm 2024-08-31T06:22:38.036 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 1249_vm 2024-08-31T06:22:38.036 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Aug 31 06:07 1255 2024-08-31T06:22:38.037 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 1255_fsm 2024-08-31T06:22:38.037 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 1255_vm 2024-08-31T06:22:38.037 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Aug 31 06:07 1259 2024-08-31T06:22:38.037 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 1259_fsm 2024-08-31T06:22:38.037 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 1259_vm 2024-08-31T06:22:38.037 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Aug 31 06:07 13344 2024-08-31T06:22:38.037 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 13344_fsm 2024-08-31T06:22:38.037 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13344_vm 2024-08-31T06:22:38.037 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 13346 2024-08-31T06:22:38.037 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13348 2024-08-31T06:22:38.037 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13349 2024-08-31T06:22:38.037 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 13349_fsm 2024-08-31T06:22:38.037 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13349_vm 2024-08-31T06:22:38.037 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 13351 2024-08-31T06:22:38.038 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13353 2024-08-31T06:22:38.038 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13354 2024-08-31T06:22:38.038 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 13354_fsm 2024-08-31T06:22:38.038 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13354_vm 2024-08-31T06:22:38.038 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 13356 2024-08-31T06:22:38.038 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13358 2024-08-31T06:22:38.038 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13359 2024-08-31T06:22:38.038 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 13359_fsm 2024-08-31T06:22:38.038 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13359_vm 2024-08-31T06:22:38.038 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 13361 2024-08-31T06:22:38.038 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13363 2024-08-31T06:22:38.038 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 1417 2024-08-31T06:22:38.038 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 1418 2024-08-31T06:22:38.038 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 174 2024-08-31T06:22:38.038 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 175 2024-08-31T06:22:38.038 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2187 2024-08-31T06:22:38.039 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2224 2024-08-31T06:22:38.039 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2328 2024-08-31T06:22:38.039 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2336 2024-08-31T06:22:38.039 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2337 2024-08-31T06:22:38.039 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2579 2024-08-31T06:22:38.039 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2600 2024-08-31T06:22:38.039 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2600_fsm 2024-08-31T06:22:38.039 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2600_vm 2024-08-31T06:22:38.039 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2601 2024-08-31T06:22:38.039 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2601_fsm 2024-08-31T06:22:38.039 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2601_vm 2024-08-31T06:22:38.039 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 31 06:07 2602 2024-08-31T06:22:38.039 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2602_fsm 2024-08-31T06:22:38.040 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2602_vm 2024-08-31T06:22:38.040 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 2603 2024-08-31T06:22:38.040 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2603_fsm 2024-08-31T06:22:38.040 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2603_vm 2024-08-31T06:22:38.040 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2604 2024-08-31T06:22:38.040 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2605 2024-08-31T06:22:38.040 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2605_fsm 2024-08-31T06:22:38.040 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2605_vm 2024-08-31T06:22:38.040 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2606 2024-08-31T06:22:38.040 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2606_fsm 2024-08-31T06:22:38.040 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2606_vm 2024-08-31T06:22:38.040 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2607 2024-08-31T06:22:38.040 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2607_fsm 2024-08-31T06:22:38.040 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2607_vm 2024-08-31T06:22:38.040 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Aug 31 06:07 2608 2024-08-31T06:22:38.041 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2608_fsm 2024-08-31T06:22:38.041 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2608_vm 2024-08-31T06:22:38.041 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Aug 31 06:07 2609 2024-08-31T06:22:38.041 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2609_fsm 2024-08-31T06:22:38.041 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2609_vm 2024-08-31T06:22:38.041 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 2610 2024-08-31T06:22:38.041 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2610_fsm 2024-08-31T06:22:38.041 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2610_vm 2024-08-31T06:22:38.041 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2611 2024-08-31T06:22:38.041 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2612 2024-08-31T06:22:38.041 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2612_fsm 2024-08-31T06:22:38.041 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2612_vm 2024-08-31T06:22:38.041 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2613 2024-08-31T06:22:38.041 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2615 2024-08-31T06:22:38.041 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2615_fsm 2024-08-31T06:22:38.042 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2615_vm 2024-08-31T06:22:38.042 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2616 2024-08-31T06:22:38.042 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2616_fsm 2024-08-31T06:22:38.042 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2616_vm 2024-08-31T06:22:38.042 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Aug 31 06:07 2617 2024-08-31T06:22:38.042 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2617_fsm 2024-08-31T06:22:38.042 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2617_vm 2024-08-31T06:22:38.042 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Aug 31 06:07 2618 2024-08-31T06:22:38.042 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2618_fsm 2024-08-31T06:22:38.042 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2618_vm 2024-08-31T06:22:38.042 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Aug 31 06:07 2619 2024-08-31T06:22:38.042 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2619_fsm 2024-08-31T06:22:38.042 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2619_vm 2024-08-31T06:22:38.042 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2620 2024-08-31T06:22:38.042 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2650 2024-08-31T06:22:38.042 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2651 2024-08-31T06:22:38.043 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2652 2024-08-31T06:22:38.043 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 31 06:07 2653 2024-08-31T06:22:38.043 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 31 06:07 2654 2024-08-31T06:22:38.043 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 2655 2024-08-31T06:22:38.043 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2656 2024-08-31T06:22:38.043 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2657 2024-08-31T06:22:38.043 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Aug 31 06:07 2658 2024-08-31T06:22:38.043 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 31 06:07 2659 2024-08-31T06:22:38.043 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2660 2024-08-31T06:22:38.043 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2661 2024-08-31T06:22:38.043 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 2662 2024-08-31T06:22:38.043 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 31 06:07 2663 2024-08-31T06:22:38.043 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2664 2024-08-31T06:22:38.043 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2665 2024-08-31T06:22:38.043 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2666 2024-08-31T06:22:38.044 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2667 2024-08-31T06:22:38.044 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2668 2024-08-31T06:22:38.044 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2669 2024-08-31T06:22:38.044 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2670 2024-08-31T06:22:38.044 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Aug 31 06:07 2673 2024-08-31T06:22:38.044 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Aug 31 06:07 2674 2024-08-31T06:22:38.044 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Aug 31 06:07 2675 2024-08-31T06:22:38.044 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2678 2024-08-31T06:22:38.044 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2679 2024-08-31T06:22:38.044 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2680 2024-08-31T06:22:38.044 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2681 2024-08-31T06:22:38.044 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2682 2024-08-31T06:22:38.044 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2683 2024-08-31T06:22:38.044 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2684 2024-08-31T06:22:38.044 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2685 2024-08-31T06:22:38.045 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2686 2024-08-31T06:22:38.045 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2687 2024-08-31T06:22:38.045 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 31 06:07 2688 2024-08-31T06:22:38.045 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 31 06:07 2689 2024-08-31T06:22:38.045 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Aug 31 06:07 2690 2024-08-31T06:22:38.045 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Aug 31 06:07 2691 2024-08-31T06:22:38.045 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2692 2024-08-31T06:22:38.045 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2693 2024-08-31T06:22:38.045 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2696 2024-08-31T06:22:38.045 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2699 2024-08-31T06:22:38.045 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2701 2024-08-31T06:22:38.045 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2702 2024-08-31T06:22:38.045 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 2703 2024-08-31T06:22:38.045 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 31 06:07 2704 2024-08-31T06:22:38.045 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2753 2024-08-31T06:22:38.046 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2753_fsm 2024-08-31T06:22:38.046 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2753_vm 2024-08-31T06:22:38.046 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2754 2024-08-31T06:22:38.046 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2755 2024-08-31T06:22:38.046 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 2756 2024-08-31T06:22:38.046 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 2757 2024-08-31T06:22:38.046 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2830 2024-08-31T06:22:38.046 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2831 2024-08-31T06:22:38.046 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2832 2024-08-31T06:22:38.046 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2833 2024-08-31T06:22:38.046 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2834 2024-08-31T06:22:38.046 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2835 2024-08-31T06:22:38.046 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2836 2024-08-31T06:22:38.046 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2837 2024-08-31T06:22:38.046 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Aug 31 06:07 2838 2024-08-31T06:22:38.047 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2838_fsm 2024-08-31T06:22:38.047 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2838_vm 2024-08-31T06:22:38.047 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2839 2024-08-31T06:22:38.047 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2840 2024-08-31T06:22:38.047 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2840_fsm 2024-08-31T06:22:38.047 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2840_vm 2024-08-31T06:22:38.047 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2841 2024-08-31T06:22:38.047 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2995 2024-08-31T06:22:38.047 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2996 2024-08-31T06:22:38.047 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3079 2024-08-31T06:22:38.047 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3079_fsm 2024-08-31T06:22:38.047 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3079_vm 2024-08-31T06:22:38.047 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3080 2024-08-31T06:22:38.048 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3081 2024-08-31T06:22:38.048 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 31 06:07 3085 2024-08-31T06:22:38.048 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3118 2024-08-31T06:22:38.048 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3119 2024-08-31T06:22:38.048 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Aug 31 06:07 3164 2024-08-31T06:22:38.048 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3256 2024-08-31T06:22:38.048 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3257 2024-08-31T06:22:38.048 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3258 2024-08-31T06:22:38.048 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3350 2024-08-31T06:22:38.048 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3351 2024-08-31T06:22:38.048 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3379 2024-08-31T06:22:38.048 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3380 2024-08-31T06:22:38.048 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3381 2024-08-31T06:22:38.048 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3394 2024-08-31T06:22:38.048 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3394_fsm 2024-08-31T06:22:38.049 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3394_vm 2024-08-31T06:22:38.049 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3395 2024-08-31T06:22:38.049 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3429 2024-08-31T06:22:38.049 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3430 2024-08-31T06:22:38.049 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3431 2024-08-31T06:22:38.049 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3433 2024-08-31T06:22:38.049 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3439 2024-08-31T06:22:38.049 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3440 2024-08-31T06:22:38.049 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 31 06:07 3455 2024-08-31T06:22:38.049 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Aug 31 06:07 3456 2024-08-31T06:22:38.049 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3456_fsm 2024-08-31T06:22:38.049 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3456_vm 2024-08-31T06:22:38.049 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3466 2024-08-31T06:22:38.049 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3467 2024-08-31T06:22:38.049 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3468 2024-08-31T06:22:38.049 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3501 2024-08-31T06:22:38.050 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3502 2024-08-31T06:22:38.050 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3503 2024-08-31T06:22:38.050 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3534 2024-08-31T06:22:38.050 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3541 2024-08-31T06:22:38.050 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3541_fsm 2024-08-31T06:22:38.050 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3541_vm 2024-08-31T06:22:38.050 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3542 2024-08-31T06:22:38.050 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3574 2024-08-31T06:22:38.050 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3575 2024-08-31T06:22:38.050 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3576 2024-08-31T06:22:38.050 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3596 2024-08-31T06:22:38.050 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3597 2024-08-31T06:22:38.050 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3598 2024-08-31T06:22:38.050 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3599 2024-08-31T06:22:38.050 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3600 2024-08-31T06:22:38.050 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3600_fsm 2024-08-31T06:22:38.051 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3600_vm 2024-08-31T06:22:38.051 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3601 2024-08-31T06:22:38.051 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3601_fsm 2024-08-31T06:22:38.051 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3601_vm 2024-08-31T06:22:38.051 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3602 2024-08-31T06:22:38.051 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3602_fsm 2024-08-31T06:22:38.051 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3602_vm 2024-08-31T06:22:38.051 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3603 2024-08-31T06:22:38.051 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3603_fsm 2024-08-31T06:22:38.051 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3603_vm 2024-08-31T06:22:38.051 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3604 2024-08-31T06:22:38.051 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3605 2024-08-31T06:22:38.051 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3606 2024-08-31T06:22:38.051 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3607 2024-08-31T06:22:38.051 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3608 2024-08-31T06:22:38.052 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 3609 2024-08-31T06:22:38.052 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3712 2024-08-31T06:22:38.052 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3764 2024-08-31T06:22:38.052 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3764_fsm 2024-08-31T06:22:38.052 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3764_vm 2024-08-31T06:22:38.052 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3766 2024-08-31T06:22:38.052 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3767 2024-08-31T06:22:38.052 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3997 2024-08-31T06:22:38.052 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4143 2024-08-31T06:22:38.052 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4144 2024-08-31T06:22:38.052 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4145 2024-08-31T06:22:38.052 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4146 2024-08-31T06:22:38.052 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4147 2024-08-31T06:22:38.052 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4148 2024-08-31T06:22:38.052 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4149 2024-08-31T06:22:38.053 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4150 2024-08-31T06:22:38.053 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4151 2024-08-31T06:22:38.053 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4152 2024-08-31T06:22:38.053 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4153 2024-08-31T06:22:38.053 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4154 2024-08-31T06:22:38.053 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4155 2024-08-31T06:22:38.053 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4156 2024-08-31T06:22:38.053 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4157 2024-08-31T06:22:38.053 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4158 2024-08-31T06:22:38.053 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4159 2024-08-31T06:22:38.053 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4160 2024-08-31T06:22:38.053 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4161 2024-08-31T06:22:38.053 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4162 2024-08-31T06:22:38.053 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4163 2024-08-31T06:22:38.053 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4164 2024-08-31T06:22:38.054 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4165 2024-08-31T06:22:38.054 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4166 2024-08-31T06:22:38.054 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4167 2024-08-31T06:22:38.054 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4168 2024-08-31T06:22:38.054 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4169 2024-08-31T06:22:38.054 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4170 2024-08-31T06:22:38.054 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4171 2024-08-31T06:22:38.054 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4172 2024-08-31T06:22:38.094 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4173 2024-08-31T06:22:38.094 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4174 2024-08-31T06:22:38.094 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 5002 2024-08-31T06:22:38.094 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 548 2024-08-31T06:22:38.094 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 549 2024-08-31T06:22:38.094 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 6102 2024-08-31T06:22:38.094 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 6104 2024-08-31T06:22:38.095 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 6106 2024-08-31T06:22:38.095 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 6110 2024-08-31T06:22:38.095 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 6111 2024-08-31T06:22:38.095 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 6112 2024-08-31T06:22:38.095 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 6113 2024-08-31T06:22:38.095 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 6117 2024-08-31T06:22:38.095 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 826 2024-08-31T06:22:38.095 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 827 2024-08-31T06:22:38.095 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 828 2024-08-31T06:22:38.095 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Aug 31 06:07 pg_filenode.map 2024-08-31T06:22:38.095 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 31 06:07 PG_VERSION 2024-08-31T06:22:38.095 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:22:38.095 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-08-31T06:22:38.095 INFO:teuthology.orchestra.run.smithi016.stdout:total 571454 2024-08-31T06:22:38.095 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 Aug 31 06:13 . 2024-08-31T06:22:38.095 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Aug 31 06:07 .. 2024-08-31T06:22:38.096 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 112 2024-08-31T06:22:38.096 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 113 2024-08-31T06:22:38.096 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 31 06:07 1247 2024-08-31T06:22:38.096 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 1247_fsm 2024-08-31T06:22:38.096 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 1247_vm 2024-08-31T06:22:38.096 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Aug 31 06:07 1249 2024-08-31T06:22:38.096 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 1249_fsm 2024-08-31T06:22:38.096 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 1249_vm 2024-08-31T06:22:38.096 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Aug 31 06:07 1255 2024-08-31T06:22:38.096 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 1255_fsm 2024-08-31T06:22:38.096 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 1255_vm 2024-08-31T06:22:38.096 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Aug 31 06:21 1259 2024-08-31T06:22:38.096 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 1259_fsm 2024-08-31T06:22:38.096 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 1259_vm 2024-08-31T06:22:38.097 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Aug 31 06:07 13344 2024-08-31T06:22:38.097 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 13344_fsm 2024-08-31T06:22:38.097 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13344_vm 2024-08-31T06:22:38.097 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 13346 2024-08-31T06:22:38.097 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13348 2024-08-31T06:22:38.097 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13349 2024-08-31T06:22:38.097 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 13349_fsm 2024-08-31T06:22:38.097 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13349_vm 2024-08-31T06:22:38.097 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 13351 2024-08-31T06:22:38.097 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13353 2024-08-31T06:22:38.098 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13354 2024-08-31T06:22:38.098 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 13354_fsm 2024-08-31T06:22:38.098 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13354_vm 2024-08-31T06:22:38.098 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 13356 2024-08-31T06:22:38.098 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13358 2024-08-31T06:22:38.098 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13359 2024-08-31T06:22:38.098 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 13359_fsm 2024-08-31T06:22:38.098 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13359_vm 2024-08-31T06:22:38.098 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 13361 2024-08-31T06:22:38.098 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 13363 2024-08-31T06:22:38.098 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 1417 2024-08-31T06:22:38.098 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 1418 2024-08-31T06:22:38.099 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 441630720 Aug 31 06:22 16396 2024-08-31T06:22:38.099 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Aug 31 06:19 16396_fsm 2024-08-31T06:22:38.099 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:13 16396_vm 2024-08-31T06:22:38.099 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1130496 Aug 31 06:22 16397 2024-08-31T06:22:38.099 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:22 16397_fsm 2024-08-31T06:22:38.099 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:22 16397_vm 2024-08-31T06:22:38.099 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1409024 Aug 31 06:22 16399 2024-08-31T06:22:38.099 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:22 16399_fsm 2024-08-31T06:22:38.099 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:22 16399_vm 2024-08-31T06:22:38.099 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:22 16400 2024-08-31T06:22:38.099 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Aug 31 06:22 16402 2024-08-31T06:22:38.099 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 71901184 Aug 31 06:22 16404 2024-08-31T06:22:38.099 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 60620800 Aug 31 06:22 16406 2024-08-31T06:22:38.099 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:22 16406_fsm 2024-08-31T06:22:38.099 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:20 16406_vm 2024-08-31T06:22:38.099 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 174 2024-08-31T06:22:38.100 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 175 2024-08-31T06:22:38.100 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2187 2024-08-31T06:22:38.100 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2224 2024-08-31T06:22:38.100 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2328 2024-08-31T06:22:38.100 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2336 2024-08-31T06:22:38.100 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2337 2024-08-31T06:22:38.100 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2579 2024-08-31T06:22:38.100 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2600 2024-08-31T06:22:38.100 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2600_fsm 2024-08-31T06:22:38.100 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2600_vm 2024-08-31T06:22:38.100 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2601 2024-08-31T06:22:38.101 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2601_fsm 2024-08-31T06:22:38.101 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2601_vm 2024-08-31T06:22:38.101 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 31 06:07 2602 2024-08-31T06:22:38.101 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2602_fsm 2024-08-31T06:22:38.101 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2602_vm 2024-08-31T06:22:38.101 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 2603 2024-08-31T06:22:38.101 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2603_fsm 2024-08-31T06:22:38.101 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2603_vm 2024-08-31T06:22:38.101 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2604 2024-08-31T06:22:38.101 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2605 2024-08-31T06:22:38.101 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2605_fsm 2024-08-31T06:22:38.101 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2605_vm 2024-08-31T06:22:38.102 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2606 2024-08-31T06:22:38.102 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2606_fsm 2024-08-31T06:22:38.102 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2606_vm 2024-08-31T06:22:38.102 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2607 2024-08-31T06:22:38.102 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2607_fsm 2024-08-31T06:22:38.102 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2607_vm 2024-08-31T06:22:38.102 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Aug 31 06:07 2608 2024-08-31T06:22:38.102 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2608_fsm 2024-08-31T06:22:38.102 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2608_vm 2024-08-31T06:22:38.102 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Aug 31 06:07 2609 2024-08-31T06:22:38.102 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2609_fsm 2024-08-31T06:22:38.102 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2609_vm 2024-08-31T06:22:38.102 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 2610 2024-08-31T06:22:38.103 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2610_fsm 2024-08-31T06:22:38.103 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2610_vm 2024-08-31T06:22:38.103 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2611 2024-08-31T06:22:38.103 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2612 2024-08-31T06:22:38.103 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2612_fsm 2024-08-31T06:22:38.103 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2612_vm 2024-08-31T06:22:38.103 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2613 2024-08-31T06:22:38.103 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2615 2024-08-31T06:22:38.103 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2615_fsm 2024-08-31T06:22:38.103 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2615_vm 2024-08-31T06:22:38.103 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2616 2024-08-31T06:22:38.103 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2616_fsm 2024-08-31T06:22:38.103 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2616_vm 2024-08-31T06:22:38.104 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Aug 31 06:07 2617 2024-08-31T06:22:38.104 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2617_fsm 2024-08-31T06:22:38.104 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2617_vm 2024-08-31T06:22:38.104 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Aug 31 06:07 2618 2024-08-31T06:22:38.104 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2618_fsm 2024-08-31T06:22:38.104 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2618_vm 2024-08-31T06:22:38.104 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 163840 Aug 31 06:21 2619 2024-08-31T06:22:38.104 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:08 2619_fsm 2024-08-31T06:22:38.104 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2619_vm 2024-08-31T06:22:38.104 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2620 2024-08-31T06:22:38.104 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2650 2024-08-31T06:22:38.104 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2651 2024-08-31T06:22:38.104 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2652 2024-08-31T06:22:38.105 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 31 06:07 2653 2024-08-31T06:22:38.105 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 31 06:07 2654 2024-08-31T06:22:38.105 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 2655 2024-08-31T06:22:38.105 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2656 2024-08-31T06:22:38.105 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2657 2024-08-31T06:22:38.105 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Aug 31 06:07 2658 2024-08-31T06:22:38.105 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 31 06:07 2659 2024-08-31T06:22:38.105 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2660 2024-08-31T06:22:38.105 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2661 2024-08-31T06:22:38.105 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 2662 2024-08-31T06:22:38.105 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 31 06:07 2663 2024-08-31T06:22:38.105 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2664 2024-08-31T06:22:38.105 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2665 2024-08-31T06:22:38.105 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2666 2024-08-31T06:22:38.106 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2667 2024-08-31T06:22:38.106 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2668 2024-08-31T06:22:38.106 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2669 2024-08-31T06:22:38.106 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2670 2024-08-31T06:22:38.106 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Aug 31 06:07 2673 2024-08-31T06:22:38.106 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Aug 31 06:07 2674 2024-08-31T06:22:38.106 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Aug 31 06:07 2675 2024-08-31T06:22:38.106 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2678 2024-08-31T06:22:38.106 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2679 2024-08-31T06:22:38.106 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2680 2024-08-31T06:22:38.106 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2681 2024-08-31T06:22:38.106 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2682 2024-08-31T06:22:38.106 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2683 2024-08-31T06:22:38.107 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2684 2024-08-31T06:22:38.107 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2685 2024-08-31T06:22:38.107 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2686 2024-08-31T06:22:38.107 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2687 2024-08-31T06:22:38.107 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 31 06:07 2688 2024-08-31T06:22:38.107 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 31 06:07 2689 2024-08-31T06:22:38.107 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Aug 31 06:07 2690 2024-08-31T06:22:38.107 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Aug 31 06:07 2691 2024-08-31T06:22:38.107 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2692 2024-08-31T06:22:38.107 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2693 2024-08-31T06:22:38.107 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:13 2696 2024-08-31T06:22:38.108 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2699 2024-08-31T06:22:38.108 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2701 2024-08-31T06:22:38.108 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2702 2024-08-31T06:22:38.108 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 2703 2024-08-31T06:22:38.108 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 31 06:07 2704 2024-08-31T06:22:38.108 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2753 2024-08-31T06:22:38.108 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2753_fsm 2024-08-31T06:22:38.108 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2753_vm 2024-08-31T06:22:38.109 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2754 2024-08-31T06:22:38.109 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2755 2024-08-31T06:22:38.109 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 2756 2024-08-31T06:22:38.109 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 2757 2024-08-31T06:22:38.109 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2830 2024-08-31T06:22:38.109 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2831 2024-08-31T06:22:38.109 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2832 2024-08-31T06:22:38.109 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2833 2024-08-31T06:22:38.109 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2834 2024-08-31T06:22:38.109 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2835 2024-08-31T06:22:38.109 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2836 2024-08-31T06:22:38.109 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2837 2024-08-31T06:22:38.109 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Aug 31 06:07 2838 2024-08-31T06:22:38.110 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2838_fsm 2024-08-31T06:22:38.110 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2838_vm 2024-08-31T06:22:38.110 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2839 2024-08-31T06:22:38.110 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2840 2024-08-31T06:22:38.110 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2840_fsm 2024-08-31T06:22:38.110 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2840_vm 2024-08-31T06:22:38.110 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2841 2024-08-31T06:22:38.110 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2995 2024-08-31T06:22:38.110 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2996 2024-08-31T06:22:38.110 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3079 2024-08-31T06:22:38.110 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3079_fsm 2024-08-31T06:22:38.110 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3079_vm 2024-08-31T06:22:38.110 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3080 2024-08-31T06:22:38.110 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3081 2024-08-31T06:22:38.111 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 31 06:07 3085 2024-08-31T06:22:38.111 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3118 2024-08-31T06:22:38.111 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3119 2024-08-31T06:22:38.111 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Aug 31 06:07 3164 2024-08-31T06:22:38.111 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3256 2024-08-31T06:22:38.111 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3257 2024-08-31T06:22:38.111 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3258 2024-08-31T06:22:38.111 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3350 2024-08-31T06:22:38.111 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3351 2024-08-31T06:22:38.111 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3379 2024-08-31T06:22:38.111 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3380 2024-08-31T06:22:38.111 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3381 2024-08-31T06:22:38.112 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3394 2024-08-31T06:22:38.112 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3394_fsm 2024-08-31T06:22:38.112 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3394_vm 2024-08-31T06:22:38.112 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3395 2024-08-31T06:22:38.112 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3429 2024-08-31T06:22:38.112 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3430 2024-08-31T06:22:38.112 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3431 2024-08-31T06:22:38.112 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3433 2024-08-31T06:22:38.112 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3439 2024-08-31T06:22:38.112 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3440 2024-08-31T06:22:38.112 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 31 06:07 3455 2024-08-31T06:22:38.112 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Aug 31 06:07 3456 2024-08-31T06:22:38.112 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3456_fsm 2024-08-31T06:22:38.112 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3456_vm 2024-08-31T06:22:38.112 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3466 2024-08-31T06:22:38.113 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3467 2024-08-31T06:22:38.113 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3468 2024-08-31T06:22:38.113 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3501 2024-08-31T06:22:38.113 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3502 2024-08-31T06:22:38.113 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3503 2024-08-31T06:22:38.113 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3534 2024-08-31T06:22:38.113 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3541 2024-08-31T06:22:38.113 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3541_fsm 2024-08-31T06:22:38.113 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3541_vm 2024-08-31T06:22:38.113 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3542 2024-08-31T06:22:38.113 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3574 2024-08-31T06:22:38.113 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3575 2024-08-31T06:22:38.113 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3576 2024-08-31T06:22:38.114 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3596 2024-08-31T06:22:38.114 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3597 2024-08-31T06:22:38.114 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3598 2024-08-31T06:22:38.114 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3599 2024-08-31T06:22:38.114 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3600 2024-08-31T06:22:38.114 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3600_fsm 2024-08-31T06:22:38.114 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3600_vm 2024-08-31T06:22:38.114 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3601 2024-08-31T06:22:38.114 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3601_fsm 2024-08-31T06:22:38.114 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3601_vm 2024-08-31T06:22:38.114 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3602 2024-08-31T06:22:38.114 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3602_fsm 2024-08-31T06:22:38.114 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3602_vm 2024-08-31T06:22:38.114 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3603 2024-08-31T06:22:38.115 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3603_fsm 2024-08-31T06:22:38.115 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3603_vm 2024-08-31T06:22:38.115 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3604 2024-08-31T06:22:38.115 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3605 2024-08-31T06:22:38.115 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3606 2024-08-31T06:22:38.115 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3607 2024-08-31T06:22:38.115 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3608 2024-08-31T06:22:38.115 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 31 06:07 3609 2024-08-31T06:22:38.115 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3712 2024-08-31T06:22:38.115 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3764 2024-08-31T06:22:38.115 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 3764_fsm 2024-08-31T06:22:38.115 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3764_vm 2024-08-31T06:22:38.115 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3766 2024-08-31T06:22:38.115 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 3767 2024-08-31T06:22:38.116 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3997 2024-08-31T06:22:38.116 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4143 2024-08-31T06:22:38.116 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4144 2024-08-31T06:22:38.116 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4145 2024-08-31T06:22:38.116 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4146 2024-08-31T06:22:38.116 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4147 2024-08-31T06:22:38.116 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4148 2024-08-31T06:22:38.116 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4149 2024-08-31T06:22:38.116 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4150 2024-08-31T06:22:38.116 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4151 2024-08-31T06:22:38.116 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4152 2024-08-31T06:22:38.116 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4153 2024-08-31T06:22:38.116 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4154 2024-08-31T06:22:38.117 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4155 2024-08-31T06:22:38.117 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4156 2024-08-31T06:22:38.117 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4157 2024-08-31T06:22:38.117 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4158 2024-08-31T06:22:38.117 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4159 2024-08-31T06:22:38.118 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4160 2024-08-31T06:22:38.118 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4161 2024-08-31T06:22:38.118 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4162 2024-08-31T06:22:38.118 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4163 2024-08-31T06:22:38.118 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4164 2024-08-31T06:22:38.118 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4165 2024-08-31T06:22:38.118 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4166 2024-08-31T06:22:38.118 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4167 2024-08-31T06:22:38.118 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4168 2024-08-31T06:22:38.119 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4169 2024-08-31T06:22:38.119 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4170 2024-08-31T06:22:38.119 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4171 2024-08-31T06:22:38.119 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4172 2024-08-31T06:22:38.119 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4173 2024-08-31T06:22:38.119 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4174 2024-08-31T06:22:38.120 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 5002 2024-08-31T06:22:38.120 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 548 2024-08-31T06:22:38.120 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 549 2024-08-31T06:22:38.120 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 6102 2024-08-31T06:22:38.120 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 6104 2024-08-31T06:22:38.120 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 6106 2024-08-31T06:22:38.120 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 6110 2024-08-31T06:22:38.120 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 6111 2024-08-31T06:22:38.120 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 6112 2024-08-31T06:22:38.120 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 6113 2024-08-31T06:22:38.120 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 6117 2024-08-31T06:22:38.120 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 826 2024-08-31T06:22:38.120 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 827 2024-08-31T06:22:38.121 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 828 2024-08-31T06:22:38.121 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Aug 31 06:07 pg_filenode.map 2024-08-31T06:22:38.121 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 151596 Aug 31 06:07 pg_internal.init 2024-08-31T06:22:38.121 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 31 06:07 PG_VERSION 2024-08-31T06:22:38.121 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:22:38.121 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2024-08-31T06:22:38.121 INFO:teuthology.orchestra.run.smithi016.stdout:total 0 2024-08-31T06:22:38.121 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 . 2024-08-31T06:22:38.121 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Aug 31 06:07 .. 2024-08-31T06:22:38.121 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:22:38.121 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/global: 2024-08-31T06:22:38.122 INFO:teuthology.orchestra.run.smithi016.stdout:total 560 2024-08-31T06:22:38.122 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Aug 31 06:07 . 2024-08-31T06:22:38.122 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 31 06:22 .. 2024-08-31T06:22:38.122 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 1213 2024-08-31T06:22:38.122 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 1213_fsm 2024-08-31T06:22:38.122 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 1213_vm 2024-08-31T06:22:38.122 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 1214 2024-08-31T06:22:38.122 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 1214_fsm 2024-08-31T06:22:38.122 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 1214_vm 2024-08-31T06:22:38.122 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 1232 2024-08-31T06:22:38.122 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 1233 2024-08-31T06:22:38.122 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 1260 2024-08-31T06:22:38.122 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 1260_fsm 2024-08-31T06:22:38.123 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 1260_vm 2024-08-31T06:22:38.123 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 1261 2024-08-31T06:22:38.123 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 1261_fsm 2024-08-31T06:22:38.123 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 1261_vm 2024-08-31T06:22:38.123 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 1262 2024-08-31T06:22:38.123 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 1262_fsm 2024-08-31T06:22:38.123 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 1262_vm 2024-08-31T06:22:38.123 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2396 2024-08-31T06:22:38.123 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:07 2396_fsm 2024-08-31T06:22:38.123 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2396_vm 2024-08-31T06:22:38.123 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2397 2024-08-31T06:22:38.123 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2671 2024-08-31T06:22:38.123 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2672 2024-08-31T06:22:38.123 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2676 2024-08-31T06:22:38.124 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2677 2024-08-31T06:22:38.124 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2694 2024-08-31T06:22:38.124 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2695 2024-08-31T06:22:38.124 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2697 2024-08-31T06:22:38.124 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 31 06:07 2698 2024-08-31T06:22:38.124 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2846 2024-08-31T06:22:38.124 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2847 2024-08-31T06:22:38.124 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2964 2024-08-31T06:22:38.124 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2965 2024-08-31T06:22:38.124 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 2966 2024-08-31T06:22:38.124 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 2967 2024-08-31T06:22:38.125 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 3592 2024-08-31T06:22:38.125 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 3593 2024-08-31T06:22:38.125 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4060 2024-08-31T06:22:38.125 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4061 2024-08-31T06:22:38.125 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4175 2024-08-31T06:22:38.125 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4176 2024-08-31T06:22:38.125 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4177 2024-08-31T06:22:38.125 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4178 2024-08-31T06:22:38.125 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4181 2024-08-31T06:22:38.125 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4182 2024-08-31T06:22:38.125 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4183 2024-08-31T06:22:38.125 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4184 2024-08-31T06:22:38.125 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 4185 2024-08-31T06:22:38.126 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 4186 2024-08-31T06:22:38.126 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 6000 2024-08-31T06:22:38.126 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 6001 2024-08-31T06:22:38.126 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 6002 2024-08-31T06:22:38.126 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 6100 2024-08-31T06:22:38.126 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 6114 2024-08-31T06:22:38.126 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 6115 2024-08-31T06:22:38.126 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:22 pg_control 2024-08-31T06:22:38.126 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Aug 31 06:07 pg_filenode.map 2024-08-31T06:22:38.126 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 23144 Aug 31 06:07 pg_internal.init 2024-08-31T06:22:38.126 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:22:38.126 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/log: 2024-08-31T06:22:38.126 INFO:teuthology.orchestra.run.smithi016.stdout:total 2 2024-08-31T06:22:38.127 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 31 06:07 . 2024-08-31T06:22:38.127 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 31 06:22 .. 2024-08-31T06:22:38.127 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1249 Aug 31 06:22 postgresql-Sat.log 2024-08-31T06:22:38.127 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:22:38.127 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-08-31T06:22:38.127 INFO:teuthology.orchestra.run.smithi016.stdout:total 0 2024-08-31T06:22:38.127 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 . 2024-08-31T06:22:38.127 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 31 06:22 .. 2024-08-31T06:22:38.127 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:22:38.127 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-08-31T06:22:38.127 INFO:teuthology.orchestra.run.smithi016.stdout:total 0 2024-08-31T06:22:38.127 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 . 2024-08-31T06:22:38.127 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 31 06:22 .. 2024-08-31T06:22:38.128 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:22:38.128 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-08-31T06:22:38.128 INFO:teuthology.orchestra.run.smithi016.stdout:total 1 2024-08-31T06:22:38.128 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 31 06:22 . 2024-08-31T06:22:38.128 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 31 06:22 .. 2024-08-31T06:22:38.128 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 mappings 2024-08-31T06:22:38.128 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Aug 31 06:22 replorigin_checkpoint 2024-08-31T06:22:38.128 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 snapshots 2024-08-31T06:22:38.128 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:22:38.128 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-08-31T06:22:38.128 INFO:teuthology.orchestra.run.smithi016.stdout:total 0 2024-08-31T06:22:38.128 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 . 2024-08-31T06:22:38.128 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 31 06:22 .. 2024-08-31T06:22:38.129 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:22:38.129 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-08-31T06:22:38.129 INFO:teuthology.orchestra.run.smithi016.stdout:total 0 2024-08-31T06:22:38.129 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 . 2024-08-31T06:22:38.129 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 31 06:22 .. 2024-08-31T06:22:38.129 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:22:38.129 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-08-31T06:22:38.129 INFO:teuthology.orchestra.run.smithi016.stdout:total 0 2024-08-31T06:22:38.129 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 31 06:07 . 2024-08-31T06:22:38.129 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 31 06:22 .. 2024-08-31T06:22:38.129 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 31 06:07 members 2024-08-31T06:22:38.129 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 31 06:07 offsets 2024-08-31T06:22:38.129 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:22:38.130 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-08-31T06:22:38.130 INFO:teuthology.orchestra.run.smithi016.stdout:total 8 2024-08-31T06:22:38.130 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 31 06:07 . 2024-08-31T06:22:38.130 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 31 06:07 .. 2024-08-31T06:22:38.130 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:07 0000 2024-08-31T06:22:38.130 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:22:38.130 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-08-31T06:22:38.130 INFO:teuthology.orchestra.run.smithi016.stdout:total 8 2024-08-31T06:22:38.131 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 31 06:07 . 2024-08-31T06:22:38.131 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 31 06:07 .. 2024-08-31T06:22:38.131 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 31 06:12 0000 2024-08-31T06:22:38.131 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:22:38.131 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-08-31T06:22:38.131 INFO:teuthology.orchestra.run.smithi016.stdout:total 0 2024-08-31T06:22:38.131 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 . 2024-08-31T06:22:38.131 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 31 06:22 .. 2024-08-31T06:22:38.131 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:22:38.131 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-08-31T06:22:38.131 INFO:teuthology.orchestra.run.smithi016.stdout:total 0 2024-08-31T06:22:38.131 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 . 2024-08-31T06:22:38.132 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:37 smithi155 ceph-mon[21742]: scrub queued for path: / 2024-08-31T06:22:38.132 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:37 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:22:38.132 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:37 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:22:38.133 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 31 06:22 .. 2024-08-31T06:22:38.133 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:22:38.133 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-08-31T06:22:38.133 INFO:teuthology.orchestra.run.smithi016.stdout:total 0 2024-08-31T06:22:38.133 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 . 2024-08-31T06:22:38.133 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 31 06:22 .. 2024-08-31T06:22:38.133 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:22:38.133 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-08-31T06:22:38.133 INFO:teuthology.orchestra.run.smithi016.stdout:total 0 2024-08-31T06:22:38.133 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 . 2024-08-31T06:22:38.133 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 31 06:22 .. 2024-08-31T06:22:38.134 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:22:38.134 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-08-31T06:22:38.134 INFO:teuthology.orchestra.run.smithi016.stdout:total 15 2024-08-31T06:22:38.134 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 31 06:22 . 2024-08-31T06:22:38.134 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 31 06:22 .. 2024-08-31T06:22:38.134 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1952 Aug 31 06:22 db_0.stat 2024-08-31T06:22:38.134 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 10979 Aug 31 06:22 db_13494.stat 2024-08-31T06:22:38.134 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1151 Aug 31 06:22 global.stat 2024-08-31T06:22:38.134 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:22:38.134 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-08-31T06:22:38.134 INFO:teuthology.orchestra.run.smithi016.stdout:total 0 2024-08-31T06:22:38.134 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:22 . 2024-08-31T06:22:38.134 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 31 06:22 .. 2024-08-31T06:22:38.134 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:22:38.135 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-08-31T06:22:38.135 INFO:teuthology.orchestra.run.smithi016.stdout:total 128 2024-08-31T06:22:38.135 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 31 06:22 . 2024-08-31T06:22:38.135 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 31 06:22 .. 2024-08-31T06:22:38.135 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Aug 31 06:22 0011 2024-08-31T06:22:38.135 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:22:38.135 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-08-31T06:22:38.135 INFO:teuthology.orchestra.run.smithi016.stdout:total 0 2024-08-31T06:22:38.135 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 . 2024-08-31T06:22:38.135 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 31 06:22 .. 2024-08-31T06:22:38.135 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:22:38.135 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-08-31T06:22:38.136 INFO:teuthology.orchestra.run.smithi016.stdout:total 0 2024-08-31T06:22:38.136 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 . 2024-08-31T06:22:38.136 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 31 06:22 .. 2024-08-31T06:22:38.136 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:22:38.136 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-08-31T06:22:38.136 INFO:teuthology.orchestra.run.smithi016.stdout:total 983040 2024-08-31T06:22:38.136 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 61 Aug 31 06:22 . 2024-08-31T06:22:38.136 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 31 06:22 .. 2024-08-31T06:22:38.136 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:22 000000010000000000000076 2024-08-31T06:22:38.136 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:13 000000010000000000000077 2024-08-31T06:22:38.137 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:12 000000010000000000000078 2024-08-31T06:22:38.137 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:12 000000010000000000000079 2024-08-31T06:22:38.137 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:13 00000001000000000000007A 2024-08-31T06:22:38.137 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:12 00000001000000000000007B 2024-08-31T06:22:38.137 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:14 00000001000000000000007C 2024-08-31T06:22:38.137 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:12 00000001000000000000007D 2024-08-31T06:22:38.137 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:16 00000001000000000000007E 2024-08-31T06:22:38.137 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:13 00000001000000000000007F 2024-08-31T06:22:38.137 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:13 000000010000000000000080 2024-08-31T06:22:38.137 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:16 000000010000000000000081 2024-08-31T06:22:38.137 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:12 000000010000000000000082 2024-08-31T06:22:38.137 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:15 000000010000000000000083 2024-08-31T06:22:38.138 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:13 000000010000000000000084 2024-08-31T06:22:38.138 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:15 000000010000000000000085 2024-08-31T06:22:38.138 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:13 000000010000000000000086 2024-08-31T06:22:38.138 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:12 000000010000000000000087 2024-08-31T06:22:38.138 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:12 000000010000000000000088 2024-08-31T06:22:38.138 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:17 000000010000000000000089 2024-08-31T06:22:38.138 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:19 00000001000000000000008A 2024-08-31T06:22:38.138 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:18 00000001000000000000008B 2024-08-31T06:22:38.138 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:17 00000001000000000000008C 2024-08-31T06:22:38.138 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:17 00000001000000000000008D 2024-08-31T06:22:38.138 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:17 00000001000000000000008E 2024-08-31T06:22:38.138 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:18 00000001000000000000008F 2024-08-31T06:22:38.139 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:19 000000010000000000000090 2024-08-31T06:22:38.139 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:17 000000010000000000000091 2024-08-31T06:22:38.139 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:18 000000010000000000000092 2024-08-31T06:22:38.139 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:19 000000010000000000000093 2024-08-31T06:22:38.139 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:17 000000010000000000000094 2024-08-31T06:22:38.139 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:19 000000010000000000000095 2024-08-31T06:22:38.139 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:17 000000010000000000000096 2024-08-31T06:22:38.139 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:18 000000010000000000000097 2024-08-31T06:22:38.139 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:19 000000010000000000000098 2024-08-31T06:22:38.139 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:17 000000010000000000000099 2024-08-31T06:22:38.139 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:21 00000001000000000000009A 2024-08-31T06:22:38.139 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:17 00000001000000000000009B 2024-08-31T06:22:38.139 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:20 00000001000000000000009C 2024-08-31T06:22:38.139 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:17 00000001000000000000009D 2024-08-31T06:22:38.140 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:18 00000001000000000000009E 2024-08-31T06:22:38.140 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:17 00000001000000000000009F 2024-08-31T06:22:38.140 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:19 0000000100000000000000A0 2024-08-31T06:22:38.140 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:17 0000000100000000000000A1 2024-08-31T06:22:38.140 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:21 0000000100000000000000A2 2024-08-31T06:22:38.140 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:18 0000000100000000000000A3 2024-08-31T06:22:38.140 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:17 0000000100000000000000A4 2024-08-31T06:22:38.140 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:20 0000000100000000000000A5 2024-08-31T06:22:38.140 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:17 0000000100000000000000A6 2024-08-31T06:22:38.140 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:17 0000000100000000000000A7 2024-08-31T06:22:38.141 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:17 0000000100000000000000A8 2024-08-31T06:22:38.141 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:18 0000000100000000000000A9 2024-08-31T06:22:38.141 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:17 0000000100000000000000AA 2024-08-31T06:22:38.141 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:17 0000000100000000000000AB 2024-08-31T06:22:38.141 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:22 0000000100000000000000AC 2024-08-31T06:22:38.141 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:22 0000000100000000000000AD 2024-08-31T06:22:38.141 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:22 0000000100000000000000AE 2024-08-31T06:22:38.141 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:22 0000000100000000000000AF 2024-08-31T06:22:38.141 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:22 0000000100000000000000B0 2024-08-31T06:22:38.142 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 31 06:22 0000000100000000000000B1 2024-08-31T06:22:38.142 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 archive_status 2024-08-31T06:22:38.142 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:22:38.142 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-08-31T06:22:38.142 INFO:teuthology.orchestra.run.smithi016.stdout:total 0 2024-08-31T06:22:38.142 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 31 06:07 . 2024-08-31T06:22:38.142 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 61 Aug 31 06:22 .. 2024-08-31T06:22:38.142 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:22:38.142 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-08-31T06:22:38.142 INFO:teuthology.orchestra.run.smithi016.stdout:total 280 2024-08-31T06:22:38.142 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 31 06:22 . 2024-08-31T06:22:38.142 INFO:teuthology.orchestra.run.smithi016.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 31 06:22 .. 2024-08-31T06:22:38.142 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 262144 Aug 31 06:22 0000 2024-08-31T06:22:38.142 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 31 06:22 0001 2024-08-31T06:22:38.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:37 smithi016 ceph-mon[17194]: scrub queued for path: / 2024-08-31T06:22:38.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:37 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:22:38.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:37 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:22:38.144 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-08-31T06:22:38.158 INFO:tasks.quiescer:joining Quiescers 2024-08-31T06:22:38.159 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-08-31T06:22:38.159 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:22:38.225 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:37 smithi184 ceph-mon[22331]: scrub queued for path: / 2024-08-31T06:22:38.225 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:37 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:22:38.225 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:37 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:22:39.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:38 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:22:39.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:38 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:22:39.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:38 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:22:39.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:38 smithi155 ceph-mon[21742]: scrub summary: active paths [/] 2024-08-31T06:22:39.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:38 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:22:39.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:38 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:22:39.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:38 smithi155 ceph-mon[21742]: pgmap v846: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 17 MiB/s wr, 1.54k op/s 2024-08-31T06:22:39.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:38 smithi155 ceph-mon[21742]: scrub summary: idle+waiting paths [/] 2024-08-31T06:22:39.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:38 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:22:39.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:38 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:22:39.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:38 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:22:39.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:38 smithi016 ceph-mon[17194]: scrub summary: active paths [/] 2024-08-31T06:22:39.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:38 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:22:39.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:38 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:22:39.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:38 smithi016 ceph-mon[17194]: pgmap v846: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 17 MiB/s wr, 1.54k op/s 2024-08-31T06:22:39.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:38 smithi016 ceph-mon[17194]: scrub summary: idle+waiting paths [/] 2024-08-31T06:22:39.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:38 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:22:39.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:38 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:22:39.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:38 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:22:39.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:38 smithi184 ceph-mon[22331]: scrub summary: active paths [/] 2024-08-31T06:22:39.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:38 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:22:39.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:38 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:22:39.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:38 smithi184 ceph-mon[22331]: pgmap v846: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 17 MiB/s wr, 1.54k op/s 2024-08-31T06:22:39.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:38 smithi184 ceph-mon[22331]: scrub summary: idle+waiting paths [/] 2024-08-31T06:22:39.464 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T06:22:39.515 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:22:40.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.149+0000 7f4445400640 1 -- 172.21.15.16:0/3470007675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44401099e0 msgr2=0x7f4440110210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:40.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.149+0000 7f4445400640 1 --2- 172.21.15.16:0/3470007675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44401099e0 0x7f4440110210 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f4438009ca0 tx=0x7f443802a8c0 comp rx=0 tx=0).stop 2024-08-31T06:22:40.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.149+0000 7f4445400640 1 -- 172.21.15.16:0/3470007675 shutdown_connections 2024-08-31T06:22:40.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.149+0000 7f4445400640 1 --2- 172.21.15.16:0/3470007675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44401099e0 0x7f4440110210 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:40.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.149+0000 7f4445400640 1 --2- 172.21.15.16:0/3470007675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4440109090 0x7f44401094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:40.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.149+0000 7f4445400640 1 --2- 172.21.15.16:0/3470007675 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4440103750 0x7f4440103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:40.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.149+0000 7f4445400640 1 -- 172.21.15.16:0/3470007675 >> 172.21.15.16:0/3470007675 conn(0x7f44400ff1e0 msgr2=0x7f4440101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:40.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.149+0000 7f4445400640 1 -- 172.21.15.16:0/3470007675 shutdown_connections 2024-08-31T06:22:40.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.149+0000 7f4445400640 1 -- 172.21.15.16:0/3470007675 wait complete. 2024-08-31T06:22:40.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.150+0000 7f4445400640 1 Processor -- start 2024-08-31T06:22:40.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.150+0000 7f4445400640 1 -- start start 2024-08-31T06:22:40.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.151+0000 7f4445400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4440103750 0x7f444019c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:40.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.151+0000 7f4445400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4440109090 0x7f444019c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:40.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.151+0000 7f4445400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44401099e0 0x7f44401a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:40.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.151+0000 7f4445400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4440113700 con 0x7f4440109090 2024-08-31T06:22:40.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.151+0000 7f4445400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4440113580 con 0x7f44401099e0 2024-08-31T06:22:40.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.151+0000 7f4445400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4440113880 con 0x7f4440103750 2024-08-31T06:22:40.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.151+0000 7f443f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4440109090 0x7f444019c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:40.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.151+0000 7f443f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4440109090 0x7f444019c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51628/0 (socket says 172.21.15.16:51628) 2024-08-31T06:22:40.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.151+0000 7f443f400640 1 -- 172.21.15.16:0/1667311681 learned_addr learned my addr 172.21.15.16:0/1667311681 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:40.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.151+0000 7f443fe00640 1 --2- 172.21.15.16:0/1667311681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4440103750 0x7f444019c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:40.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.151+0000 7f4444a00640 1 --2- 172.21.15.16:0/1667311681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44401099e0 0x7f44401a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:40.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.152+0000 7f443f400640 1 -- 172.21.15.16:0/1667311681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4440103750 msgr2=0x7f444019c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:40.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.152+0000 7f443f400640 1 --2- 172.21.15.16:0/1667311681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4440103750 0x7f444019c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:40.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.152+0000 7f443f400640 1 -- 172.21.15.16:0/1667311681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44401099e0 msgr2=0x7f44401a1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:40.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.152+0000 7f443f400640 1 --2- 172.21.15.16:0/1667311681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44401099e0 0x7f44401a1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:40.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.152+0000 7f443f400640 1 -- 172.21.15.16:0/1667311681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44401a2200 con 0x7f4440109090 2024-08-31T06:22:40.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.152+0000 7f443f400640 1 --2- 172.21.15.16:0/1667311681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4440109090 0x7f444019c830 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f442c00e970 tx=0x7f442c00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:40.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.152+0000 7f443cc00640 1 -- 172.21.15.16:0/1667311681 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c00cd80 con 0x7f4440109090 2024-08-31T06:22:40.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.152+0000 7f443cc00640 1 -- 172.21.15.16:0/1667311681 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f442c0035f0 con 0x7f4440109090 2024-08-31T06:22:40.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.152+0000 7f4445400640 1 -- 172.21.15.16:0/1667311681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44401a24f0 con 0x7f4440109090 2024-08-31T06:22:40.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.152+0000 7f443cc00640 1 -- 172.21.15.16:0/1667311681 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c010690 con 0x7f4440109090 2024-08-31T06:22:40.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.152+0000 7f4445400640 1 -- 172.21.15.16:0/1667311681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44401a8c50 con 0x7f4440109090 2024-08-31T06:22:40.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.154+0000 7f4445400640 1 -- 172.21.15.16:0/1667311681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4440103b60 con 0x7f4440109090 2024-08-31T06:22:40.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.157+0000 7f443cc00640 1 -- 172.21.15.16:0/1667311681 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f442c003aa0 con 0x7f4440109090 2024-08-31T06:22:40.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.159+0000 7f443cc00640 1 --2- 172.21.15.16:0/1667311681 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f44240b4d20 0x7f44240b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:40.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.160+0000 7f443cc00640 1 -- 172.21.15.16:0/1667311681 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8470+0+0 (secure 0 0 0) 0x7f442c0e5790 con 0x7f4440109090 2024-08-31T06:22:40.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.160+0000 7f443fe00640 1 --2- 172.21.15.16:0/1667311681 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f44240b4d20 0x7f44240b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:40.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.161+0000 7f443fe00640 1 --2- 172.21.15.16:0/1667311681 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f44240b4d20 0x7f44240b7230 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4430009bd0 tx=0x7f4430005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:40.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.164+0000 7f443cc00640 1 -- 172.21.15.16:0/1667311681 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f442c0ac550 con 0x7f4440109090 2024-08-31T06:22:40.172 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:22:40.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.405+0000 7f4445400640 1 -- 172.21.15.16:0/1667311681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f444019d3c0 con 0x7f4440109090 2024-08-31T06:22:40.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.406+0000 7f443cc00640 1 -- 172.21.15.16:0/1667311681 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+11917 (secure 0 0 0) 0x7f442c0abe50 con 0x7f4440109090 2024-08-31T06:22:40.409 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:22:40.409 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":53,"btime":"2024-08-31T06:22:28:781487+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":53,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:22:28.366480+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24737,24724,24712]},"id":1}]} 2024-08-31T06:22:40.409 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 53 2024-08-31T06:22:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.408+0000 7f4445400640 1 -- 172.21.15.16:0/1667311681 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f44240b4d20 msgr2=0x7f44240b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.408+0000 7f4445400640 1 --2- 172.21.15.16:0/1667311681 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f44240b4d20 0x7f44240b7230 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4430009bd0 tx=0x7f4430005f90 comp rx=0 tx=0).stop 2024-08-31T06:22:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.408+0000 7f4445400640 1 -- 172.21.15.16:0/1667311681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4440109090 msgr2=0x7f444019c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.408+0000 7f4445400640 1 --2- 172.21.15.16:0/1667311681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4440109090 0x7f444019c830 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f442c00e970 tx=0x7f442c00ee30 comp rx=0 tx=0).stop 2024-08-31T06:22:40.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.408+0000 7f4445400640 1 -- 172.21.15.16:0/1667311681 shutdown_connections 2024-08-31T06:22:40.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.409+0000 7f4445400640 1 --2- 172.21.15.16:0/1667311681 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f44240b4d20 0x7f44240b7230 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:40.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.409+0000 7f4445400640 1 --2- 172.21.15.16:0/1667311681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44401099e0 0x7f44401a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:40.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.409+0000 7f4445400640 1 --2- 172.21.15.16:0/1667311681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4440109090 0x7f444019c830 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:40.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.409+0000 7f4445400640 1 --2- 172.21.15.16:0/1667311681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4440103750 0x7f444019c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:40.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.409+0000 7f4445400640 1 -- 172.21.15.16:0/1667311681 >> 172.21.15.16:0/1667311681 conn(0x7f44400ff1e0 msgr2=0x7f44401065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:40.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.409+0000 7f4445400640 1 -- 172.21.15.16:0/1667311681 shutdown_connections 2024-08-31T06:22:40.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.409+0000 7f4445400640 1 -- 172.21.15.16:0/1667311681 wait complete. 2024-08-31T06:22:40.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.967+0000 7f022ce00640 1 -- 172.21.15.16:0/3239103072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f022810bde0 msgr2=0x7f0228112610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:40.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.967+0000 7f022ce00640 1 --2- 172.21.15.16:0/3239103072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f022810bde0 0x7f0228112610 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f0220009ca0 tx=0x7f022002a8c0 comp rx=0 tx=0).stop 2024-08-31T06:22:40.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.967+0000 7f022ce00640 1 -- 172.21.15.16:0/3239103072 shutdown_connections 2024-08-31T06:22:40.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.967+0000 7f022ce00640 1 --2- 172.21.15.16:0/3239103072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f022810b490 0x7f022810b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:40.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.967+0000 7f022ce00640 1 --2- 172.21.15.16:0/3239103072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f022810bde0 0x7f0228112610 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:40.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.967+0000 7f022ce00640 1 --2- 172.21.15.16:0/3239103072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0228105b50 0x7f0228105f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:40.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.967+0000 7f022ce00640 1 -- 172.21.15.16:0/3239103072 >> 172.21.15.16:0/3239103072 conn(0x7f0228101580 msgr2=0x7f02281039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:40.971 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:40 smithi016 ceph-mon[17194]: pgmap v847: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 17 MiB/s wr, 1.66k op/s 2024-08-31T06:22:40.971 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:40 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/1667311681' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:22:40.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.967+0000 7f022ce00640 1 -- 172.21.15.16:0/3239103072 shutdown_connections 2024-08-31T06:22:40.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.967+0000 7f022ce00640 1 -- 172.21.15.16:0/3239103072 wait complete. 2024-08-31T06:22:40.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.968+0000 7f022ce00640 1 Processor -- start 2024-08-31T06:22:40.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.968+0000 7f022ce00640 1 -- start start 2024-08-31T06:22:40.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.969+0000 7f022ce00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0228105b50 0x7f022819c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:40.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.969+0000 7f022ce00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f022810b490 0x7f022819c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:40.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.969+0000 7f022ce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f022810bde0 0x7f02281a1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:40.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.969+0000 7f022ce00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0228115aa0 con 0x7f022810bde0 2024-08-31T06:22:40.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.969+0000 7f022ce00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0228115920 con 0x7f0228105b50 2024-08-31T06:22:40.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.969+0000 7f022ce00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0228115c20 con 0x7f022810b490 2024-08-31T06:22:40.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.969+0000 7f0226800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f022810bde0 0x7f02281a1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:40.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.969+0000 7f0225e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0228105b50 0x7f022819c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:40.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.969+0000 7f0226800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f022810bde0 0x7f02281a1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51638/0 (socket says 172.21.15.16:51638) 2024-08-31T06:22:40.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.969+0000 7f0226800640 1 -- 172.21.15.16:0/64045190 learned_addr learned my addr 172.21.15.16:0/64045190 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:40.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.969+0000 7f0225e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0228105b50 0x7f022819c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:42004/0 (socket says 172.21.15.16:42004) 2024-08-31T06:22:40.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.969+0000 7f021d400640 1 --2- 172.21.15.16:0/64045190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f022810b490 0x7f022819c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:40.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.970+0000 7f0226800640 1 -- 172.21.15.16:0/64045190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f022810b490 msgr2=0x7f022819c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:40.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.970+0000 7f0226800640 1 --2- 172.21.15.16:0/64045190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f022810b490 0x7f022819c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:40.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.970+0000 7f0226800640 1 -- 172.21.15.16:0/64045190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0228105b50 msgr2=0x7f022819c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:40.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.970+0000 7f0226800640 1 --2- 172.21.15.16:0/64045190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0228105b50 0x7f022819c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:40.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.970+0000 7f0226800640 1 -- 172.21.15.16:0/64045190 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02281a23a0 con 0x7f022810bde0 2024-08-31T06:22:40.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.970+0000 7f0225e00640 1 --2- 172.21.15.16:0/64045190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0228105b50 0x7f022819c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:22:40.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.970+0000 7f0226800640 1 --2- 172.21.15.16:0/64045190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f022810bde0 0x7f02281a1c70 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f02200098d0 tx=0x7f02200317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:40.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.970+0000 7f021f400640 1 -- 172.21.15.16:0/64045190 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0220003b70 con 0x7f022810bde0 2024-08-31T06:22:40.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.970+0000 7f021f400640 1 -- 172.21.15.16:0/64045190 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0220003d10 con 0x7f022810bde0 2024-08-31T06:22:40.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.970+0000 7f021f400640 1 -- 172.21.15.16:0/64045190 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022002fa20 con 0x7f022810bde0 2024-08-31T06:22:40.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.970+0000 7f022ce00640 1 -- 172.21.15.16:0/64045190 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02281a2630 con 0x7f022810bde0 2024-08-31T06:22:40.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.971+0000 7f022ce00640 1 -- 172.21.15.16:0/64045190 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02281a8df0 con 0x7f022810bde0 2024-08-31T06:22:40.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.974+0000 7f0233a41740 1 Processor -- start 2024-08-31T06:22:40.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.974+0000 7f021f400640 1 -- 172.21.15.16:0/64045190 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f022002fc20 con 0x7f022810bde0 2024-08-31T06:22:40.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.974+0000 7f0233a41740 1 -- start start 2024-08-31T06:22:40.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.975+0000 7f0233a41740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ef83cc070 0x564ef8771950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:40.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.975+0000 7f0233a41740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ef83a31b0 0x564ef8771e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:40.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.975+0000 7f0233a41740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ef83ab390 0x564ef87723d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:40.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.975+0000 7f0233a41740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564ef86e8770 con 0x564ef83ab390 2024-08-31T06:22:40.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.975+0000 7f0233a41740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x564ef86d7d00 con 0x564ef83a31b0 2024-08-31T06:22:40.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.975+0000 7f0233a41740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564ef86dbcb0 con 0x564ef83cc070 2024-08-31T06:22:40.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.975+0000 7f021d400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ef83cc070 0x564ef8771950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:40.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.975+0000 7f0225e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ef83a31b0 0x564ef8771e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:40.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.975+0000 7f021d400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ef83cc070 0x564ef8771950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:40060/0 (socket says 172.21.15.16:40060) 2024-08-31T06:22:40.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.975+0000 7f021d400640 1 -- 172.21.15.16:0/1744170777 learned_addr learned my addr 172.21.15.16:0/1744170777 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:40.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.975+0000 7f021d400640 1 --2- 172.21.15.16:0/1744170777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ef83ab390 0x564ef87723d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:40.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.976+0000 7f0225e00640 1 -- 172.21.15.16:0/1744170777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ef83cc070 msgr2=0x564ef8771950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:40.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.976+0000 7f0225e00640 1 --2- 172.21.15.16:0/1744170777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ef83cc070 0x564ef8771950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:40.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.976+0000 7f0225e00640 1 -- 172.21.15.16:0/1744170777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ef83ab390 msgr2=0x564ef87723d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:40.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.976+0000 7f0225e00640 1 --2- 172.21.15.16:0/1744170777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ef83ab390 0x564ef87723d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:40.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.976+0000 7f0225e00640 1 -- 172.21.15.16:0/1744170777 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ef8437650 con 0x564ef83a31b0 2024-08-31T06:22:40.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.976+0000 7f021d400640 1 --2- 172.21.15.16:0/1744170777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ef83ab390 0x564ef87723d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:22:40.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.976+0000 7f0225e00640 1 --2- 172.21.15.16:0/1744170777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ef83a31b0 0x564ef8771e90 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f0218009fd0 tx=0x7f02180118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e04e96430dc522f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:40.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.976+0000 7f021f400640 1 --2- 172.21.15.16:0/64045190 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f020c0b4d20 0x7f020c0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:40.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.976+0000 7f0217400640 1 -- 172.21.15.16:0/1744170777 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0218004030 con 0x564ef83a31b0 2024-08-31T06:22:40.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.976+0000 7f021f400640 1 -- 172.21.15.16:0/64045190 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8470+0+0 (secure 0 0 0) 0x7f0220106920 con 0x7f022810bde0 2024-08-31T06:22:40.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.976+0000 7f0217400640 1 -- 172.21.15.16:0/1744170777 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02180054d0 con 0x564ef83a31b0 2024-08-31T06:22:40.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.976+0000 7f0217400640 1 -- 172.21.15.16:0/1744170777 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0218004c40 con 0x564ef83a31b0 2024-08-31T06:22:40.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.976+0000 7f021d400640 1 --2- 172.21.15.16:0/64045190 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f020c0b4d20 0x7f020c0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:40.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.978+0000 7f0233a41740 1 -- 172.21.15.16:0/1744170777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ef83a31b0 msgr2=0x564ef8771e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:40.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.978+0000 7f021d400640 1 --2- 172.21.15.16:0/64045190 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f020c0b4d20 0x7f020c0b7230 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f022806c7c0 tx=0x7f0210004730 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:40.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.978+0000 7f0233a41740 1 --2- 172.21.15.16:0/1744170777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ef83a31b0 0x564ef8771e90 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f0218009fd0 tx=0x7f02180118d0 comp rx=0 tx=0).stop 2024-08-31T06:22:40.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.978+0000 7f0233a41740 1 -- 172.21.15.16:0/1744170777 shutdown_connections 2024-08-31T06:22:40.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.978+0000 7f0233a41740 1 --2- 172.21.15.16:0/1744170777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ef83ab390 0x564ef87723d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:40.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.978+0000 7f0233a41740 1 --2- 172.21.15.16:0/1744170777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ef83a31b0 0x564ef8771e90 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:40.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.978+0000 7f0233a41740 1 --2- 172.21.15.16:0/1744170777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ef83cc070 0x564ef8771950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:40.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.978+0000 7f0233a41740 1 -- 172.21.15.16:0/1744170777 >> 172.21.15.16:0/1744170777 conn(0x564ef83c0460 msgr2=0x564ef876f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:40.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.978+0000 7f0233a41740 1 -- 172.21.15.16:0/1744170777 shutdown_connections 2024-08-31T06:22:40.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.978+0000 7f0233a41740 1 -- 172.21.15.16:0/1744170777 wait complete. 2024-08-31T06:22:40.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.978+0000 7f0233a41740 1 -- start start 2024-08-31T06:22:40.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.979+0000 7f0233a41740 1 Processor -- start 2024-08-31T06:22:40.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.979+0000 7f0233a41740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ef83a31b0 0x564ef877d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:40.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.979+0000 7f0233a41740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ef83ab390 0x564ef877d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:40.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.979+0000 7f0233a41740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ef83cc070 0x564ef8789450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:40.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.979+0000 7f0233a41740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564ef853a700 con 0x564ef83cc070 2024-08-31T06:22:40.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.979+0000 7f0233a41740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x564ef865ecb0 con 0x564ef83ab390 2024-08-31T06:22:40.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.979+0000 7f0233a41740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564ef8780130 con 0x564ef83a31b0 2024-08-31T06:22:40.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.979+0000 7f0233a41740 10 client.0 _ll_register_callbacks cb 0x564ef8684430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:22:40.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.980+0000 7f0225e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ef83a31b0 0x564ef877d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:40.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.980+0000 7f021d400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ef83ab390 0x564ef877d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:40.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.980+0000 7f0225e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ef83a31b0 0x564ef877d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:40072/0 (socket says 172.21.15.16:40072) 2024-08-31T06:22:40.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.980+0000 7f0226800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ef83cc070 0x564ef8789450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:40.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.980+0000 7f0225e00640 1 -- 172.21.15.16:0/3498270608 learned_addr learned my addr 172.21.15.16:0/3498270608 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:40.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.980+0000 7f0225e00640 1 -- 172.21.15.16:0/3498270608 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ef83ab390 msgr2=0x564ef877d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:40.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.980+0000 7f0225e00640 1 --2- 172.21.15.16:0/3498270608 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ef83ab390 0x564ef877d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:40.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.980+0000 7f0225e00640 1 -- 172.21.15.16:0/3498270608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ef83cc070 msgr2=0x564ef8789450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:40.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.980+0000 7f0225e00640 1 --2- 172.21.15.16:0/3498270608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ef83cc070 0x564ef8789450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:40.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.980+0000 7f0225e00640 1 -- 172.21.15.16:0/3498270608 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ef869f870 con 0x564ef83a31b0 2024-08-31T06:22:40.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.981+0000 7f0225e00640 1 --2- 172.21.15.16:0/3498270608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ef83a31b0 0x564ef877d490 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0218009700 tx=0x7f021800fcd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9da2729c319da3fa server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:40.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.981+0000 7f020be00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:22:40.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.981+0000 7f020be00640 1 -- 172.21.15.16:0/3498270608 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0218029020 con 0x564ef83a31b0 2024-08-31T06:22:40.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.981+0000 7f020be00640 1 -- 172.21.15.16:0/3498270608 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8470+0+0 (secure 0 0 0) 0x7f021800fe40 con 0x564ef83a31b0 2024-08-31T06:22:40.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.981+0000 7f0233a41740 1 -- 172.21.15.16:0/3498270608 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02180054d0 con 0x564ef83a31b0 2024-08-31T06:22:40.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.981+0000 7f020be00640 1 -- 172.21.15.16:0/3498270608 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f0218034050 con 0x564ef83a31b0 2024-08-31T06:22:40.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.981+0000 7f0233a41740 10 client.29056 fetch_fsmap learned FSMap version 53 2024-08-31T06:22:40.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.981+0000 7f0233a41740 1 -- 172.21.15.16:0/3498270608 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x564ef878c0f0 con 0x564ef83a31b0 2024-08-31T06:22:40.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.982+0000 7f020be00640 1 -- 172.21.15.16:0/3498270608 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02180054d0 con 0x564ef83a31b0 2024-08-31T06:22:40.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.982+0000 7f020be00640 1 -- 172.21.15.16:0/3498270608 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 53) ==== 8346+0+0 (secure 0 0 0) 0x7f0218004940 con 0x564ef83a31b0 2024-08-31T06:22:40.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.982+0000 7f0233a41740 10 client.29056 fetch_fsmap finished waiting for FSMap version 53 2024-08-31T06:22:40.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.982+0000 7f0233a41740 10 client.29056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:22:40.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.982+0000 7f0233a41740 20 client.29056 populate_metadata read hostname 'smithi016' 2024-08-31T06:22:40.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.982+0000 7f0233a41740 1 --2- 172.21.15.16:0/3498270608 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564ef838a9f0 0x564ef878f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:40.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.982+0000 7f0233a41740 4 client.29056 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:22:40.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.982+0000 7f0233a41740 1 -- 172.21.15.16:0/3498270608 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ef86ea6c0 con 0x564ef838a9f0 2024-08-31T06:22:40.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.983+0000 7f021d400640 1 --2- 172.21.15.16:0/3498270608 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564ef838a9f0 0x564ef878f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:40.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.983+0000 7f021d400640 1 --2- 172.21.15.16:0/3498270608 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564ef838a9f0 0x564ef878f560 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7512dcad7e6252fc server_cookie=97134804096f43aa in_seq=0 out_seq=0 2024-08-31T06:22:40.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.983+0000 7f020be00640 10 client.29056 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:22:40.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.984+0000 7f020be00640 1 -- 172.21.15.16:0/3498270608 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564ef86ea6c0 con 0x564ef838a9f0 2024-08-31T06:22:40.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.984+0000 7f020be00640 10 client.29056 handle_command_reply: tid=0 2024-08-31T06:22:40.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.984+0000 7f0233a41740 1 client.29056 shutdown 2024-08-31T06:22:40.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.984+0000 7f0233a41740 1 -- 172.21.15.16:0/3498270608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ef83a31b0 msgr2=0x564ef877d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:40.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.984+0000 7f0233a41740 1 --2- 172.21.15.16:0/3498270608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ef83a31b0 0x564ef877d490 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0218009700 tx=0x7f021800fcd0 comp rx=0 tx=0).stop 2024-08-31T06:22:40.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.985+0000 7f0233a41740 1 -- 172.21.15.16:0/3498270608 shutdown_connections 2024-08-31T06:22:40.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.985+0000 7f0233a41740 1 --2- 172.21.15.16:0/3498270608 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564ef838a9f0 0x564ef878f560 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:40.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.985+0000 7f0233a41740 1 --2- 172.21.15.16:0/3498270608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ef83cc070 0x564ef8789450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:40.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.985+0000 7f0233a41740 1 --2- 172.21.15.16:0/3498270608 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ef83ab390 0x564ef877d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:40.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.985+0000 7f0233a41740 1 --2- 172.21.15.16:0/3498270608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ef83a31b0 0x564ef877d490 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:40.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.985+0000 7f0233a41740 1 -- 172.21.15.16:0/3498270608 >> 172.21.15.16:0/3498270608 conn(0x564ef83c0460 msgr2=0x564ef8774ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:40.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.985+0000 7f020be00640 0 client.29056 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:22:40.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.985+0000 7f0233a41740 1 -- 172.21.15.16:0/3498270608 shutdown_connections 2024-08-31T06:22:40.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.985+0000 7f0233a41740 1 -- 172.21.15.16:0/3498270608 wait complete. 2024-08-31T06:22:40.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:40.985+0000 7f0233a41740 20 client.29056 trim_cache size 0 max 16384 2024-08-31T06:22:41.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.004+0000 7f0233a41740 1 Processor -- start 2024-08-31T06:22:41.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.004+0000 7f0233a41740 1 -- start start 2024-08-31T06:22:41.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.004+0000 7f0233a41740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ef83a31b0 0x564ef876f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:41.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.004+0000 7f0233a41740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ef83ab390 0x564ef8778440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:41.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.004+0000 7f0233a41740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ef83cc070 0x564ef8789350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:41.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.004+0000 7f0233a41740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f02281b3f20 con 0x564ef83a31b0 2024-08-31T06:22:41.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.004+0000 7f0233a41740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f02281b17f0 con 0x564ef83ab390 2024-08-31T06:22:41.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.004+0000 7f0233a41740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f02281af660 con 0x564ef83cc070 2024-08-31T06:22:41.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.004+0000 7f0225e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ef83a31b0 0x564ef876f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:41.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.004+0000 7f0225e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ef83a31b0 0x564ef876f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51650/0 (socket says 172.21.15.16:51650) 2024-08-31T06:22:41.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.004+0000 7f0225e00640 1 -- 172.21.15.16:0/3471275927 learned_addr learned my addr 172.21.15.16:0/3471275927 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:41.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.005+0000 7f0226800640 1 --2- 172.21.15.16:0/3471275927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ef83cc070 0x564ef8789350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:41.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.005+0000 7f021d400640 1 --2- 172.21.15.16:0/3471275927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ef83ab390 0x564ef8778440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:41.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.005+0000 7f0225e00640 1 -- 172.21.15.16:0/3471275927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ef83cc070 msgr2=0x564ef8789350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:41.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.005+0000 7f0225e00640 1 --2- 172.21.15.16:0/3471275927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ef83cc070 0x564ef8789350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:41.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.005+0000 7f0225e00640 1 -- 172.21.15.16:0/3471275927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ef83ab390 msgr2=0x564ef8778440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:41.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.005+0000 7f0225e00640 1 --2- 172.21.15.16:0/3471275927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ef83ab390 0x564ef8778440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:41.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.005+0000 7f0225e00640 1 -- 172.21.15.16:0/3471275927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ef8773c40 con 0x564ef83a31b0 2024-08-31T06:22:41.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.005+0000 7f0225e00640 1 --2- 172.21.15.16:0/3471275927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ef83a31b0 0x564ef876f9f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f0218000c00 tx=0x7f02180059b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec9c7190738bcd4 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:41.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.005+0000 7f0216a00640 1 -- 172.21.15.16:0/3471275927 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0218029040 con 0x564ef83a31b0 2024-08-31T06:22:41.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.006+0000 7f0216a00640 1 -- 172.21.15.16:0/3471275927 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0218005420 con 0x564ef83a31b0 2024-08-31T06:22:41.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.006+0000 7f0216a00640 1 -- 172.21.15.16:0/3471275927 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0218025d90 con 0x564ef83a31b0 2024-08-31T06:22:41.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.006+0000 7f0233a41740 1 -- 172.21.15.16:0/3471275927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ef83a31b0 msgr2=0x564ef876f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:41.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.006+0000 7f0233a41740 1 --2- 172.21.15.16:0/3471275927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ef83a31b0 0x564ef876f9f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f0218000c00 tx=0x7f02180059b0 comp rx=0 tx=0).stop 2024-08-31T06:22:41.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.006+0000 7f0233a41740 1 -- 172.21.15.16:0/3471275927 shutdown_connections 2024-08-31T06:22:41.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.006+0000 7f0233a41740 1 --2- 172.21.15.16:0/3471275927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ef83cc070 0x564ef8789350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:41.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.006+0000 7f0233a41740 1 --2- 172.21.15.16:0/3471275927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ef83ab390 0x564ef8778440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:41.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.006+0000 7f0233a41740 1 --2- 172.21.15.16:0/3471275927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ef83a31b0 0x564ef876f9f0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:41.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.007+0000 7f0233a41740 1 -- 172.21.15.16:0/3471275927 >> 172.21.15.16:0/3471275927 conn(0x564ef83c0460 msgr2=0x564ef877bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:41.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.007+0000 7f0233a41740 1 -- 172.21.15.16:0/3471275927 shutdown_connections 2024-08-31T06:22:41.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.007+0000 7f0233a41740 1 -- 172.21.15.16:0/3471275927 wait complete. 2024-08-31T06:22:41.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.007+0000 7f0233a41740 1 -- start start 2024-08-31T06:22:41.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.007+0000 7f0233a41740 1 Processor -- start 2024-08-31T06:22:41.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.008+0000 7f0233a41740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ef83a31b0 0x564ef8781590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:41.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.008+0000 7f0233a41740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ef83ab390 0x564ef8781ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:41.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.008+0000 7f0233a41740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ef83cc070 0x564ef8782010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:41.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.008+0000 7f0233a41740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f02281afcb0 con 0x564ef83cc070 2024-08-31T06:22:41.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.008+0000 7f0233a41740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f02281b1120 con 0x564ef83ab390 2024-08-31T06:22:41.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.008+0000 7f0233a41740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f02281b0380 con 0x564ef83a31b0 2024-08-31T06:22:41.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.008+0000 7f0233a41740 10 client.0 _ll_register_callbacks cb 0x7f021800f960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:22:41.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.008+0000 7f0226800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ef83cc070 0x564ef8782010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:41.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.008+0000 7f0225e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ef83a31b0 0x564ef8781590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:41.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.008+0000 7f0226800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ef83cc070 0x564ef8782010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51662/0 (socket says 172.21.15.16:51662) 2024-08-31T06:22:41.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.008+0000 7f0225e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ef83a31b0 0x564ef8781590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:40098/0 (socket says 172.21.15.16:40098) 2024-08-31T06:22:41.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.008+0000 7f0226800640 1 -- 172.21.15.16:0/485924778 learned_addr learned my addr 172.21.15.16:0/485924778 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:41.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.008+0000 7f021d400640 1 --2- 172.21.15.16:0/485924778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ef83ab390 0x564ef8781ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:41.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.009+0000 7f0226800640 1 -- 172.21.15.16:0/485924778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ef83a31b0 msgr2=0x564ef8781590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:41.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.009+0000 7f0226800640 1 --2- 172.21.15.16:0/485924778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ef83a31b0 0x564ef8781590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:41.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.009+0000 7f0226800640 1 -- 172.21.15.16:0/485924778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ef83ab390 msgr2=0x564ef8781ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:41.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.009+0000 7f0226800640 1 --2- 172.21.15.16:0/485924778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ef83ab390 0x564ef8781ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:41.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.009+0000 7f0226800640 1 -- 172.21.15.16:0/485924778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ef87f8fc0 con 0x564ef83cc070 2024-08-31T06:22:41.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.009+0000 7f0225e00640 1 --2- 172.21.15.16:0/485924778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ef83a31b0 0x564ef8781590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:22:41.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.009+0000 7f0226800640 1 --2- 172.21.15.16:0/485924778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ef83cc070 0x564ef8782010 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f02200c9060 tx=0x7f0220104540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbf9ff1ec3d47090 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:41.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.009+0000 7f020be00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:22:41.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.009+0000 7f020be00640 1 -- 172.21.15.16:0/485924778 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02200cd6e0 con 0x564ef83cc070 2024-08-31T06:22:41.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.009+0000 7f020be00640 1 -- 172.21.15.16:0/485924778 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8470+0+0 (secure 0 0 0) 0x7f02200c95d0 con 0x564ef83cc070 2024-08-31T06:22:41.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.009+0000 7f0233a41740 1 -- 172.21.15.16:0/485924778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0218005420 con 0x564ef83cc070 2024-08-31T06:22:41.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.010+0000 7f020be00640 1 -- 172.21.15.16:0/485924778 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f02200ccfe0 con 0x564ef83cc070 2024-08-31T06:22:41.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.010+0000 7f0233a41740 10 client.18540 fetch_fsmap learned FSMap version 53 2024-08-31T06:22:41.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.010+0000 7f0233a41740 1 -- 172.21.15.16:0/485924778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x564ef87fa2b0 con 0x564ef83cc070 2024-08-31T06:22:41.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.010+0000 7f020be00640 1 -- 172.21.15.16:0/485924778 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0218005420 con 0x564ef83cc070 2024-08-31T06:22:41.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.010+0000 7f020be00640 1 -- 172.21.15.16:0/485924778 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 53) ==== 8346+0+0 (secure 0 0 0) 0x7f02200d5e70 con 0x564ef83cc070 2024-08-31T06:22:41.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.010+0000 7f0233a41740 10 client.18540 fetch_fsmap finished waiting for FSMap version 53 2024-08-31T06:22:41.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.010+0000 7f0233a41740 10 client.18540 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:22:41.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.010+0000 7f0233a41740 20 client.18540 populate_metadata read hostname 'smithi016' 2024-08-31T06:22:41.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.010+0000 7f0233a41740 1 --2- 172.21.15.16:0/485924778 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564ef835fda0 0x564ef87fca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:41.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.010+0000 7f0233a41740 4 client.18540 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T06:22:41.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.010+0000 7f0233a41740 1 -- 172.21.15.16:0/485924778 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564ef86e6160 con 0x564ef835fda0 2024-08-31T06:22:41.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.011+0000 7f0225e00640 1 --2- 172.21.15.16:0/485924778 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564ef835fda0 0x564ef87fca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:41.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.011+0000 7f0225e00640 1 --2- 172.21.15.16:0/485924778 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564ef835fda0 0x564ef87fca70 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12e51e68b4ce2fe server_cookie=8340d7f09f40cf32 in_seq=0 out_seq=0 2024-08-31T06:22:41.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.011+0000 7f020be00640 10 client.18540 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:22:41.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.011+0000 7f020be00640 1 -- 172.21.15.16:0/485924778 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5682 (crc 0 0 0) 0x564ef86e6160 con 0x564ef835fda0 2024-08-31T06:22:41.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.011+0000 7f020be00640 10 client.18540 handle_command_reply: tid=0 2024-08-31T06:22:41.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.012+0000 7f0233a41740 1 client.18540 shutdown 2024-08-31T06:22:41.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.012+0000 7f0233a41740 1 -- 172.21.15.16:0/485924778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ef83cc070 msgr2=0x564ef8782010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:41.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.012+0000 7f0233a41740 1 --2- 172.21.15.16:0/485924778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ef83cc070 0x564ef8782010 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f02200c9060 tx=0x7f0220104540 comp rx=0 tx=0).stop 2024-08-31T06:22:41.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.012+0000 7f0233a41740 1 -- 172.21.15.16:0/485924778 shutdown_connections 2024-08-31T06:22:41.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.012+0000 7f0233a41740 1 --2- 172.21.15.16:0/485924778 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564ef835fda0 0x564ef87fca70 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:41.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.012+0000 7f0233a41740 1 --2- 172.21.15.16:0/485924778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ef83cc070 0x564ef8782010 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:41.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.012+0000 7f0233a41740 1 --2- 172.21.15.16:0/485924778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564ef83ab390 0x564ef8781ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:41.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.012+0000 7f0233a41740 1 --2- 172.21.15.16:0/485924778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ef83a31b0 0x564ef8781590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:41.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.012+0000 7f0233a41740 1 -- 172.21.15.16:0/485924778 >> 172.21.15.16:0/485924778 conn(0x564ef83c0460 msgr2=0x564ef8778440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:41.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.012+0000 7f020be00640 0 client.18540 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:22:41.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.012+0000 7f0233a41740 1 -- 172.21.15.16:0/485924778 shutdown_connections 2024-08-31T06:22:41.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.012+0000 7f0233a41740 1 -- 172.21.15.16:0/485924778 wait complete. 2024-08-31T06:22:41.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.012+0000 7f0233a41740 20 client.18540 trim_cache size 0 max 16384 2024-08-31T06:22:41.018 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:22:41.018 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-31T06:22:41.018 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-08-31T06:22:41.018 INFO:teuthology.orchestra.run.smithi016.stdout: "ab1ded0f-9efd-4559-88af-72427ba8460d": { 2024-08-31T06:22:41.018 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:22:41.018 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "ab1ded0f-9efd-4559-88af-72427ba8460d", 2024-08-31T06:22:41.019 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-08-31T06:22:41.019 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:22:41.019 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:22:41.019 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-08-31T06:22:41.019 INFO:teuthology.orchestra.run.smithi016.stdout: "ab1ded0f-9efd-4559-88af-72427ba8460d": { 2024-08-31T06:22:41.019 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:22:37.396566+0000", 2024-08-31T06:22:41.019 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:22:41.019 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:22:41.019 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:22:41.019 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:22:41.019 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:22:41.019 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:22:41.019 INFO:teuthology.orchestra.run.smithi016.stdout: "18e112f4-a5f9-4602-aaad-3b13fa5a4c2e": { 2024-08-31T06:22:41.020 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:20:16.570535+0000", 2024-08-31T06:22:41.020 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:22:41.020 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:22:41.020 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:22:41.020 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:22:41.020 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:22:41.020 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:22:41.020 INFO:teuthology.orchestra.run.smithi016.stdout: "f1cdcb27-e3e4-4dc8-a0ae-7ace761d04c5": { 2024-08-31T06:22:41.020 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:19:22.275683+0000", 2024-08-31T06:22:41.020 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:22:41.020 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:22:41.020 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:22:41.020 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:22:41.021 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:22:41.021 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:22:41.021 INFO:teuthology.orchestra.run.smithi016.stdout: "0e44eb18-439c-46e4-a298-7d4d2ecd0b4a": { 2024-08-31T06:22:41.021 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:18:31.130765+0000", 2024-08-31T06:22:41.021 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:22:41.021 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:22:41.021 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:22:41.021 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:22:41.021 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:22:41.021 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:22:41.021 INFO:teuthology.orchestra.run.smithi016.stdout: "e497cd88-1e7c-4631-a4a8-cdd8e1fa6c83": { 2024-08-31T06:22:41.021 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:17:42.476795+0000", 2024-08-31T06:22:41.021 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:22:41.021 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:22:41.021 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:22:41.022 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:22:41.022 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:22:41.022 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:22:41.022 INFO:teuthology.orchestra.run.smithi016.stdout: "c300b7e8-f3fd-482c-a909-9329bc61e570": { 2024-08-31T06:22:41.022 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:06:49.868046+0000", 2024-08-31T06:22:41.022 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:22:41.022 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:22:41.022 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:22:41.022 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:22:41.022 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:22:41.022 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:22:41.022 INFO:teuthology.orchestra.run.smithi016.stdout: "6a03046e-cdf4-4a14-b9bd-b560b8968467": { 2024-08-31T06:22:41.022 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:12:38.002354+0000", 2024-08-31T06:22:41.022 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:22:41.023 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:22:41.023 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:22:41.023 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:22:41.023 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:22:41.023 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:22:41.023 INFO:teuthology.orchestra.run.smithi016.stdout: "004b47bb-0fe8-4d28-9d9d-78bde6a61199": { 2024-08-31T06:22:41.023 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:16:48.915926+0000", 2024-08-31T06:22:41.023 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:22:41.023 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:22:41.023 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:22:41.023 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:22:41.023 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:22:41.023 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:22:41.023 INFO:teuthology.orchestra.run.smithi016.stdout: "88d7bdb8-cb1c-45e5-ad29-4cd8da6c6003": { 2024-08-31T06:22:41.023 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:07:39.648942+0000", 2024-08-31T06:22:41.024 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:22:41.024 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:22:41.024 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:22:41.024 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:22:41.024 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:22:41.024 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:22:41.024 INFO:teuthology.orchestra.run.smithi016.stdout: "0fdcd762-533d-42be-9289-b71c8e3b81b1": { 2024-08-31T06:22:41.024 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:15:06.169112+0000", 2024-08-31T06:22:41.024 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:22:41.024 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:22:41.024 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:22:41.024 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:22:41.024 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:22:41.024 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:22:41.025 INFO:teuthology.orchestra.run.smithi016.stdout: "0015da36-6634-47c5-b8d4-219f9eba8250": { 2024-08-31T06:22:41.025 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:21:11.445380+0000", 2024-08-31T06:22:41.025 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:22:41.025 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:22:41.025 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:22:41.025 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:22:41.025 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:22:41.025 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:22:41.025 INFO:teuthology.orchestra.run.smithi016.stdout: "c8455d74-2fe7-4d93-bcf2-19b419d16016": { 2024-08-31T06:22:41.025 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:08:24.035960+0000", 2024-08-31T06:22:41.025 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:22:41.025 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:22:41.025 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:22:41.026 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:22:41.026 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:22:41.026 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:22:41.026 INFO:teuthology.orchestra.run.smithi016.stdout: "78d6ef02-8551-45e8-ad60-9627eb573fc1": { 2024-08-31T06:22:41.026 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:09:13.547865+0000", 2024-08-31T06:22:41.026 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:22:41.026 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:22:41.026 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:22:41.026 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:22:41.026 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:22:41.026 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:22:41.026 INFO:teuthology.orchestra.run.smithi016.stdout: "c1fd6904-2e02-47fa-939a-47349b9fe36b": { 2024-08-31T06:22:41.026 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:06.678691+0000", 2024-08-31T06:22:41.027 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:22:41.027 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:22:41.027 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:22:41.027 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:22:41.027 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:22:41.027 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:22:41.027 INFO:teuthology.orchestra.run.smithi016.stdout: "a10cce76-99ab-46b5-b3b2-07b82899b8f9": { 2024-08-31T06:22:41.027 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:11:43.967546+0000", 2024-08-31T06:22:41.027 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:22:41.027 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:22:41.027 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:22:41.027 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:22:41.027 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:22:41.027 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:22:41.028 INFO:teuthology.orchestra.run.smithi016.stdout: "91b72a34-4ba9-4126-a414-8b862f8c5f78": { 2024-08-31T06:22:41.028 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:13:28.663237+0000", 2024-08-31T06:22:41.028 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:22:41.028 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:22:41.028 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:22:41.028 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:22:41.028 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:22:41.028 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:22:41.028 INFO:teuthology.orchestra.run.smithi016.stdout: "e739a16e-8ace-43a8-a1ce-ac27d39ef8a3": { 2024-08-31T06:22:41.028 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:54.084646+0000", 2024-08-31T06:22:41.028 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:22:41.028 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:22:41.028 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:22:41.028 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:22:41.028 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:22:41.029 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:22:41.029 INFO:teuthology.orchestra.run.smithi016.stdout: "a64d5301-8558-4e9c-ac9b-8d0ac7057e74": { 2024-08-31T06:22:41.029 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:14:20.338104+0000", 2024-08-31T06:22:41.029 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:22:41.029 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:22:41.029 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:22:41.029 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:22:41.029 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:22:41.029 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:22:41.029 INFO:teuthology.orchestra.run.smithi016.stdout: "f9c4a22e-0c8a-4928-9b0d-e2ce1db5a86b": { 2024-08-31T06:22:41.029 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:15:57.496015+0000", 2024-08-31T06:22:41.029 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:22:41.029 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:22:41.029 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:22:41.029 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:22:41.030 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:22:41.030 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:22:41.030 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:22:41.030 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:22:41.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.013+0000 7f022ce00640 1 -- 172.21.15.16:0/64045190 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f020c0b4d20 msgr2=0x7f020c0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:41.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.013+0000 7f022ce00640 1 --2- 172.21.15.16:0/64045190 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f020c0b4d20 0x7f020c0b7230 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f022806c7c0 tx=0x7f0210004730 comp rx=0 tx=0).stop 2024-08-31T06:22:41.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.013+0000 7f022ce00640 1 -- 172.21.15.16:0/64045190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f022810bde0 msgr2=0x7f02281a1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:41.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.013+0000 7f022ce00640 1 --2- 172.21.15.16:0/64045190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f022810bde0 0x7f02281a1c70 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f02200098d0 tx=0x7f02200317b0 comp rx=0 tx=0).stop 2024-08-31T06:22:41.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.013+0000 7f022ce00640 1 -- 172.21.15.16:0/64045190 shutdown_connections 2024-08-31T06:22:41.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.013+0000 7f022ce00640 1 --2- 172.21.15.16:0/64045190 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f020c0b4d20 0x7f020c0b7230 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:41.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.013+0000 7f022ce00640 1 --2- 172.21.15.16:0/64045190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f022810b490 0x7f022819c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:41.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.013+0000 7f022ce00640 1 --2- 172.21.15.16:0/64045190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f022810bde0 0x7f02281a1c70 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:41.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.013+0000 7f022ce00640 1 --2- 172.21.15.16:0/64045190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0228105b50 0x7f022819c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:41.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.013+0000 7f022ce00640 1 -- 172.21.15.16:0/64045190 >> 172.21.15.16:0/64045190 conn(0x7f0228101580 msgr2=0x7f022810aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:41.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.013+0000 7f022ce00640 1 -- 172.21.15.16:0/64045190 shutdown_connections 2024-08-31T06:22:41.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:41.013+0000 7f022ce00640 1 -- 172.21.15.16:0/64045190 wait complete. 2024-08-31T06:22:41.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:40 smithi155 ceph-mon[21742]: pgmap v847: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 17 MiB/s wr, 1.66k op/s 2024-08-31T06:22:41.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:40 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/1667311681' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:22:41.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:40 smithi184 ceph-mon[22331]: pgmap v847: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 17 MiB/s wr, 1.66k op/s 2024-08-31T06:22:41.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:40 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/1667311681' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:22:41.448 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24737 quiesce db --cancel --all' 2024-08-31T06:22:41.448 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- timeout 105 ceph tell mds.24737 quiesce db --cancel --all 2024-08-31T06:22:42.221 INFO:tasks.cephfs.filesystem:scrub status for tag:ab1ded0f-9efd-4559-88af-72427ba8460d - {'path': '/', 'tag': 'ab1ded0f-9efd-4559-88af-72427ba8460d', 'options': 'recursive,force'} 2024-08-31T06:22:42.262 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:22:42.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.918+0000 7fd63f800640 1 -- 172.21.15.16:0/1830249445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6380754f0 msgr2=0x7fd638075900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:42.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.918+0000 7fd63f800640 1 --2- 172.21.15.16:0/1830249445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6380754f0 0x7fd638075900 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fd62c009ca0 tx=0x7fd62c02a8c0 comp rx=0 tx=0).stop 2024-08-31T06:22:42.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.918+0000 7fd63f800640 1 -- 172.21.15.16:0/1830249445 shutdown_connections 2024-08-31T06:22:42.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.918+0000 7fd63f800640 1 --2- 172.21.15.16:0/1830249445 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd63810fec0 0x7fd6381122e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:42.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.918+0000 7fd63f800640 1 --2- 172.21.15.16:0/1830249445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd638075e40 0x7fd63810f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:42.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.918+0000 7fd63f800640 1 --2- 172.21.15.16:0/1830249445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6380754f0 0x7fd638075900 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:42.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.918+0000 7fd63f800640 1 -- 172.21.15.16:0/1830249445 >> 172.21.15.16:0/1830249445 conn(0x7fd63807b5f0 msgr2=0x7fd63807ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:42.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.918+0000 7fd63f800640 1 -- 172.21.15.16:0/1830249445 shutdown_connections 2024-08-31T06:22:42.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.918+0000 7fd63f800640 1 -- 172.21.15.16:0/1830249445 wait complete. 2024-08-31T06:22:42.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.919+0000 7fd63f800640 1 Processor -- start 2024-08-31T06:22:42.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.920+0000 7fd63f800640 1 -- start start 2024-08-31T06:22:42.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.920+0000 7fd63f800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6380754f0 0x7fd63819c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:42.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.920+0000 7fd63f800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd638075e40 0x7fd63819ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:42.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.920+0000 7fd63f800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd63810fec0 0x7fd6381a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:42.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.920+0000 7fd63f800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd638115a80 con 0x7fd63810fec0 2024-08-31T06:22:42.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.920+0000 7fd63f800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd638115900 con 0x7fd6380754f0 2024-08-31T06:22:42.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.920+0000 7fd63f800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd638115c00 con 0x7fd638075e40 2024-08-31T06:22:42.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.920+0000 7fd63d800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd63810fec0 0x7fd6381a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:42.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.920+0000 7fd63d800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd63810fec0 0x7fd6381a1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51670/0 (socket says 172.21.15.16:51670) 2024-08-31T06:22:42.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.920+0000 7fd63d800640 1 -- 172.21.15.16:0/161695004 learned_addr learned my addr 172.21.15.16:0/161695004 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:42.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.920+0000 7fd63ce00640 1 --2- 172.21.15.16:0/161695004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6380754f0 0x7fd63819c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:42.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.921+0000 7fd637e00640 1 --2- 172.21.15.16:0/161695004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd638075e40 0x7fd63819ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:42.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.921+0000 7fd63d800640 1 -- 172.21.15.16:0/161695004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd638075e40 msgr2=0x7fd63819ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:42.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.921+0000 7fd63d800640 1 --2- 172.21.15.16:0/161695004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd638075e40 0x7fd63819ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:42.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.921+0000 7fd63d800640 1 -- 172.21.15.16:0/161695004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6380754f0 msgr2=0x7fd63819c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:42.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.921+0000 7fd63d800640 1 --2- 172.21.15.16:0/161695004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6380754f0 0x7fd63819c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:42.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.921+0000 7fd63d800640 1 -- 172.21.15.16:0/161695004 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6381a23e0 con 0x7fd63810fec0 2024-08-31T06:22:42.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.921+0000 7fd63d800640 1 --2- 172.21.15.16:0/161695004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd63810fec0 0x7fd6381a1cb0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fd63000e970 tx=0x7fd63000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:42.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.922+0000 7fd635600640 1 -- 172.21.15.16:0/161695004 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63000ccc0 con 0x7fd63810fec0 2024-08-31T06:22:42.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.922+0000 7fd635600640 1 -- 172.21.15.16:0/161695004 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd63000ce60 con 0x7fd63810fec0 2024-08-31T06:22:42.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.922+0000 7fd63f800640 1 -- 172.21.15.16:0/161695004 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6381a26d0 con 0x7fd63810fec0 2024-08-31T06:22:42.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.922+0000 7fd635600640 1 -- 172.21.15.16:0/161695004 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd630010430 con 0x7fd63810fec0 2024-08-31T06:22:42.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.922+0000 7fd63f800640 1 -- 172.21.15.16:0/161695004 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6381a8e30 con 0x7fd63810fec0 2024-08-31T06:22:42.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.925+0000 7fd6465f4740 1 Processor -- start 2024-08-31T06:22:42.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.925+0000 7fd635600640 1 -- 172.21.15.16:0/161695004 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fd630004f90 con 0x7fd63810fec0 2024-08-31T06:22:42.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.926+0000 7fd6465f4740 1 -- start start 2024-08-31T06:22:42.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.926+0000 7fd6465f4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d815ab82d0 0x55d815e5f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:42.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.926+0000 7fd6465f4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d815a8f410 0x55d815e5f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:42.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.926+0000 7fd6465f4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d815a975f0 0x55d815e5fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:42.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.926+0000 7fd6465f4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d815dd4790 con 0x55d815a8f410 2024-08-31T06:22:42.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.926+0000 7fd6465f4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d815dc3d20 con 0x55d815ab82d0 2024-08-31T06:22:42.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.926+0000 7fd6465f4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d815dc7cd0 con 0x55d815a975f0 2024-08-31T06:22:42.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.926+0000 7fd637e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d815ab82d0 0x55d815e5f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:42.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.926+0000 7fd637e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d815ab82d0 0x55d815e5f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:42058/0 (socket says 172.21.15.16:42058) 2024-08-31T06:22:42.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.926+0000 7fd637e00640 1 -- 172.21.15.16:0/602833827 learned_addr learned my addr 172.21.15.16:0/602833827 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:42.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.926+0000 7fd63ce00640 1 --2- 172.21.15.16:0/602833827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d815a8f410 0x55d815e5f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:42.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.927+0000 7fd637e00640 1 --2- 172.21.15.16:0/602833827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d815a975f0 0x55d815e5fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:42.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.927+0000 7fd637e00640 1 -- 172.21.15.16:0/602833827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d815a975f0 msgr2=0x55d815e5fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:42.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.927+0000 7fd637e00640 1 --2- 172.21.15.16:0/602833827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d815a975f0 0x55d815e5fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:42.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.927+0000 7fd637e00640 1 -- 172.21.15.16:0/602833827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d815a8f410 msgr2=0x55d815e5f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:42.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.927+0000 7fd637e00640 1 --2- 172.21.15.16:0/602833827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d815a8f410 0x55d815e5f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:42.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.927+0000 7fd637e00640 1 -- 172.21.15.16:0/602833827 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d815b238b0 con 0x55d815ab82d0 2024-08-31T06:22:42.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.927+0000 7fd637e00640 1 --2- 172.21.15.16:0/602833827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d815ab82d0 0x55d815e5f110 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd624011710 tx=0x7fd624005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93a3059a18ffcf7f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:42.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.927+0000 7fd635600640 1 --2- 172.21.15.16:0/161695004 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd6200b4ca0 0x7fd6200b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:42.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.928+0000 7fd635600640 1 -- 172.21.15.16:0/161695004 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8470+0+0 (secure 0 0 0) 0x7fd6300e5930 con 0x7fd63810fec0 2024-08-31T06:22:42.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.928+0000 7fd63ce00640 1 --2- 172.21.15.16:0/161695004 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd6200b4ca0 0x7fd6200b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:42.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.928+0000 7fd62a000640 1 -- 172.21.15.16:0/602833827 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62400f840 con 0x55d815ab82d0 2024-08-31T06:22:42.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.928+0000 7fd62a000640 1 -- 172.21.15.16:0/602833827 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd62400fe90 con 0x55d815ab82d0 2024-08-31T06:22:42.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.928+0000 7fd62a000640 1 -- 172.21.15.16:0/602833827 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62401fca0 con 0x55d815ab82d0 2024-08-31T06:22:42.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.929+0000 7fd6465f4740 1 -- 172.21.15.16:0/602833827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d815ab82d0 msgr2=0x55d815e5f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:42.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.929+0000 7fd6465f4740 1 --2- 172.21.15.16:0/602833827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d815ab82d0 0x55d815e5f110 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd624011710 tx=0x7fd624005850 comp rx=0 tx=0).stop 2024-08-31T06:22:42.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.929+0000 7fd63ce00640 1 --2- 172.21.15.16:0/161695004 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd6200b4ca0 0x7fd6200b71b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd62c006010 tx=0x7fd62c009820 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:42.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.929+0000 7fd6465f4740 1 -- 172.21.15.16:0/602833827 shutdown_connections 2024-08-31T06:22:42.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.929+0000 7fd6465f4740 1 --2- 172.21.15.16:0/602833827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d815a975f0 0x55d815e5fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:42.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.929+0000 7fd6465f4740 1 --2- 172.21.15.16:0/602833827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d815a8f410 0x55d815e5f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:42.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.929+0000 7fd6465f4740 1 --2- 172.21.15.16:0/602833827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d815ab82d0 0x55d815e5f110 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:42.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.929+0000 7fd6465f4740 1 -- 172.21.15.16:0/602833827 >> 172.21.15.16:0/602833827 conn(0x55d815aac6c0 msgr2=0x55d815e5d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:42.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.929+0000 7fd6465f4740 1 -- 172.21.15.16:0/602833827 shutdown_connections 2024-08-31T06:22:42.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.929+0000 7fd6465f4740 1 -- 172.21.15.16:0/602833827 wait complete. 2024-08-31T06:22:42.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.930+0000 7fd6465f4740 1 -- start start 2024-08-31T06:22:42.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.930+0000 7fd6465f4740 1 Processor -- start 2024-08-31T06:22:42.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.930+0000 7fd6465f4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d815a8f410 0x55d815e6ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:42.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.930+0000 7fd6465f4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d815a975f0 0x55d815e6b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:42.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.930+0000 7fd6465f4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d815ab82d0 0x55d815e76be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:42.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.930+0000 7fd6465f4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d815c26810 con 0x55d815a975f0 2024-08-31T06:22:42.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.930+0000 7fd6465f4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d815d5f2b0 con 0x55d815ab82d0 2024-08-31T06:22:42.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.930+0000 7fd6465f4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d815e6d8f0 con 0x55d815a8f410 2024-08-31T06:22:42.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.930+0000 7fd6465f4740 10 client.0 _ll_register_callbacks cb 0x55d815d70040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:22:42.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.931+0000 7fd637e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d815a975f0 0x55d815e6b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:42.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.931+0000 7fd63d800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d815ab82d0 0x55d815e76be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:42.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.931+0000 7fd63ce00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d815a8f410 0x55d815e6ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:42.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.931+0000 7fd63d800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d815ab82d0 0x55d815e76be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:42064/0 (socket says 172.21.15.16:42064) 2024-08-31T06:22:42.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.931+0000 7fd63d800640 1 -- 172.21.15.16:0/2795429234 learned_addr learned my addr 172.21.15.16:0/2795429234 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:42.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.931+0000 7fd63d800640 1 -- 172.21.15.16:0/2795429234 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d815a8f410 msgr2=0x55d815e6ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:42.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.931+0000 7fd63d800640 1 --2- 172.21.15.16:0/2795429234 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d815a8f410 0x55d815e6ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:42.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.931+0000 7fd63d800640 1 -- 172.21.15.16:0/2795429234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d815a975f0 msgr2=0x55d815e6b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:42.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.931+0000 7fd63d800640 1 --2- 172.21.15.16:0/2795429234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d815a975f0 0x55d815e6b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:42.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.931+0000 7fd63d800640 1 -- 172.21.15.16:0/2795429234 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d815d941f0 con 0x55d815ab82d0 2024-08-31T06:22:42.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.931+0000 7fd63d800640 1 --2- 172.21.15.16:0/2795429234 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d815ab82d0 0x55d815e76be0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd6300a8070 tx=0x7fd6300056b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12045c2270689d9b server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:42.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.931+0000 7fd61ea00640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:22:42.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.932+0000 7fd61ea00640 1 -- 172.21.15.16:0/2795429234 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6300ac6f0 con 0x55d815ab82d0 2024-08-31T06:22:42.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.932+0000 7fd6465f4740 1 -- 172.21.15.16:0/2795429234 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd62400fe90 con 0x55d815ab82d0 2024-08-31T06:22:42.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.932+0000 7fd61ea00640 1 -- 172.21.15.16:0/2795429234 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8470+0+0 (secure 0 0 0) 0x7fd6300a85e0 con 0x55d815ab82d0 2024-08-31T06:22:42.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.932+0000 7fd61ea00640 1 -- 172.21.15.16:0/2795429234 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fd6300abff0 con 0x55d815ab82d0 2024-08-31T06:22:42.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.932+0000 7fd6465f4740 10 client.29042 fetch_fsmap learned FSMap version 54 2024-08-31T06:22:42.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.932+0000 7fd6465f4740 1 -- 172.21.15.16:0/2795429234 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55d815e79a00 con 0x55d815ab82d0 2024-08-31T06:22:42.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.932+0000 7fd61ea00640 1 -- 172.21.15.16:0/2795429234 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd62400fe90 con 0x55d815ab82d0 2024-08-31T06:22:42.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.933+0000 7fd61ea00640 1 -- 172.21.15.16:0/2795429234 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 54) ==== 8350+0+0 (secure 0 0 0) 0x7fd6300b03b0 con 0x55d815ab82d0 2024-08-31T06:22:42.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.933+0000 7fd6465f4740 10 client.29042 fetch_fsmap finished waiting for FSMap version 54 2024-08-31T06:22:42.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.933+0000 7fd6465f4740 10 client.29042 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:22:42.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.933+0000 7fd6465f4740 20 client.29042 populate_metadata read hostname 'smithi016' 2024-08-31T06:22:42.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.933+0000 7fd6465f4740 1 --2- 172.21.15.16:0/2795429234 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d815a76c50 0x55d815e7ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:42.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.933+0000 7fd6465f4740 4 client.29042 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:22:42.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.933+0000 7fd6465f4740 1 -- 172.21.15.16:0/2795429234 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d815dd66e0 con 0x55d815a76c50 2024-08-31T06:22:42.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.933+0000 7fd63ce00640 1 --2- 172.21.15.16:0/2795429234 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d815a76c50 0x55d815e7ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:42.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.934+0000 7fd63ce00640 1 --2- 172.21.15.16:0/2795429234 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d815a76c50 0x55d815e7ce20 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db0be4cd6788bd14 server_cookie=55a7f7bb3653ed29 in_seq=0 out_seq=0 2024-08-31T06:22:42.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.934+0000 7fd61ea00640 10 client.29042 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:22:42.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.935+0000 7fd61ea00640 1 -- 172.21.15.16:0/2795429234 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d815dd66e0 con 0x55d815a76c50 2024-08-31T06:22:42.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.935+0000 7fd61ea00640 10 client.29042 handle_command_reply: tid=0 2024-08-31T06:22:42.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.935+0000 7fd6465f4740 1 client.29042 shutdown 2024-08-31T06:22:42.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.935+0000 7fd6465f4740 1 -- 172.21.15.16:0/2795429234 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d815ab82d0 msgr2=0x55d815e76be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:42.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.935+0000 7fd6465f4740 1 --2- 172.21.15.16:0/2795429234 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d815ab82d0 0x55d815e76be0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd6300a8070 tx=0x7fd6300056b0 comp rx=0 tx=0).stop 2024-08-31T06:22:42.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.935+0000 7fd6465f4740 1 -- 172.21.15.16:0/2795429234 shutdown_connections 2024-08-31T06:22:42.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.935+0000 7fd6465f4740 1 --2- 172.21.15.16:0/2795429234 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d815a76c50 0x55d815e7ce20 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:42.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.935+0000 7fd6465f4740 1 --2- 172.21.15.16:0/2795429234 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d815ab82d0 0x55d815e76be0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:42.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.935+0000 7fd6465f4740 1 --2- 172.21.15.16:0/2795429234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d815a975f0 0x55d815e6b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:42.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.935+0000 7fd6465f4740 1 --2- 172.21.15.16:0/2795429234 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d815a8f410 0x55d815e6ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:42.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.935+0000 7fd61ea00640 0 client.29042 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:22:42.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.935+0000 7fd6465f4740 1 -- 172.21.15.16:0/2795429234 >> 172.21.15.16:0/2795429234 conn(0x55d815aac6c0 msgr2=0x55d815e622f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:42.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.935+0000 7fd6465f4740 1 -- 172.21.15.16:0/2795429234 shutdown_connections 2024-08-31T06:22:42.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.935+0000 7fd6465f4740 1 -- 172.21.15.16:0/2795429234 wait complete. 2024-08-31T06:22:42.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.935+0000 7fd6465f4740 20 client.29042 trim_cache size 0 max 16384 2024-08-31T06:22:42.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.954+0000 7fd6465f4740 1 Processor -- start 2024-08-31T06:22:42.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.954+0000 7fd6465f4740 1 -- start start 2024-08-31T06:22:42.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.954+0000 7fd6465f4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d815a8f410 0x55d815e76ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:42.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.954+0000 7fd6465f4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d815a975f0 0x55d815e693f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:42.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.954+0000 7fd6465f4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d815ab82d0 0x55d815e69930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:42.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.954+0000 7fd6465f4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd6381b3ed0 con 0x55d815a8f410 2024-08-31T06:22:42.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.954+0000 7fd6465f4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd6381b1770 con 0x55d815a975f0 2024-08-31T06:22:42.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.954+0000 7fd6465f4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd6381af5e0 con 0x55d815ab82d0 2024-08-31T06:22:42.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.955+0000 7fd63ce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d815a8f410 0x55d815e76ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:42.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.955+0000 7fd637e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d815a975f0 0x55d815e693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:42.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.955+0000 7fd637e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d815a975f0 0x55d815e693f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:42070/0 (socket says 172.21.15.16:42070) 2024-08-31T06:22:42.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.955+0000 7fd637e00640 1 -- 172.21.15.16:0/1065909953 learned_addr learned my addr 172.21.15.16:0/1065909953 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:42.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.955+0000 7fd63d800640 1 --2- 172.21.15.16:0/1065909953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d815ab82d0 0x55d815e69930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:42.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.955+0000 7fd637e00640 1 -- 172.21.15.16:0/1065909953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d815ab82d0 msgr2=0x55d815e69930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:42.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.955+0000 7fd637e00640 1 --2- 172.21.15.16:0/1065909953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d815ab82d0 0x55d815e69930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:42.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.955+0000 7fd637e00640 1 -- 172.21.15.16:0/1065909953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d815a8f410 msgr2=0x55d815e76ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:42.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.955+0000 7fd637e00640 1 --2- 172.21.15.16:0/1065909953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d815a8f410 0x55d815e76ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:42.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.955+0000 7fd637e00640 1 -- 172.21.15.16:0/1065909953 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d815e57270 con 0x55d815a975f0 2024-08-31T06:22:42.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.955+0000 7fd63ce00640 1 --2- 172.21.15.16:0/1065909953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d815a8f410 0x55d815e76ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:22:42.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.956+0000 7fd637e00640 1 --2- 172.21.15.16:0/1065909953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d815a975f0 0x55d815e693f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd624011510 tx=0x7fd624005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4dc851adfdb4fcae server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:42.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.956+0000 7fd629600640 1 -- 172.21.15.16:0/1065909953 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62402f070 con 0x55d815a975f0 2024-08-31T06:22:42.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.956+0000 7fd629600640 1 -- 172.21.15.16:0/1065909953 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd62401f440 con 0x55d815a975f0 2024-08-31T06:22:42.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.956+0000 7fd629600640 1 -- 172.21.15.16:0/1065909953 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62400f840 con 0x55d815a975f0 2024-08-31T06:22:42.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.957+0000 7fd6465f4740 1 -- 172.21.15.16:0/1065909953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d815a975f0 msgr2=0x55d815e693f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:42.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.957+0000 7fd6465f4740 1 --2- 172.21.15.16:0/1065909953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d815a975f0 0x55d815e693f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd624011510 tx=0x7fd624005fd0 comp rx=0 tx=0).stop 2024-08-31T06:22:42.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.957+0000 7fd6465f4740 1 -- 172.21.15.16:0/1065909953 shutdown_connections 2024-08-31T06:22:42.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.957+0000 7fd6465f4740 1 --2- 172.21.15.16:0/1065909953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d815ab82d0 0x55d815e69930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:42.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.957+0000 7fd6465f4740 1 --2- 172.21.15.16:0/1065909953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d815a975f0 0x55d815e693f0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:42.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.957+0000 7fd6465f4740 1 --2- 172.21.15.16:0/1065909953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d815a8f410 0x55d815e76ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:42.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.957+0000 7fd6465f4740 1 -- 172.21.15.16:0/1065909953 >> 172.21.15.16:0/1065909953 conn(0x55d815aac6c0 msgr2=0x55d815e65cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:42.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.957+0000 7fd6465f4740 1 -- 172.21.15.16:0/1065909953 shutdown_connections 2024-08-31T06:22:42.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.957+0000 7fd6465f4740 1 -- 172.21.15.16:0/1065909953 wait complete. 2024-08-31T06:22:42.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.958+0000 7fd6465f4740 1 -- start start 2024-08-31T06:22:42.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.958+0000 7fd6465f4740 1 Processor -- start 2024-08-31T06:22:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.958+0000 7fd6465f4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d815a8f410 0x55d815e6f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.958+0000 7fd6465f4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d815a975f0 0x55d815e6f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.958+0000 7fd6465f4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d815ab82d0 0x55d815ee67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.958+0000 7fd6465f4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd6381afc30 con 0x55d815a8f410 2024-08-31T06:22:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.958+0000 7fd6465f4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd6381b10a0 con 0x55d815ab82d0 2024-08-31T06:22:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.958+0000 7fd6465f4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd6381b0300 con 0x55d815a975f0 2024-08-31T06:22:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.958+0000 7fd6465f4740 10 client.0 _ll_register_callbacks cb 0x7fd630005280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:22:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.959+0000 7fd63ce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d815a8f410 0x55d815e6f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.959+0000 7fd63ce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d815a8f410 0x55d815e6f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51712/0 (socket says 172.21.15.16:51712) 2024-08-31T06:22:42.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.959+0000 7fd63ce00640 1 -- 172.21.15.16:0/2988531497 learned_addr learned my addr 172.21.15.16:0/2988531497 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:42.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.959+0000 7fd637e00640 1 --2- 172.21.15.16:0/2988531497 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d815a975f0 0x55d815e6f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:42.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.959+0000 7fd63d800640 1 --2- 172.21.15.16:0/2988531497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d815ab82d0 0x55d815ee67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:42.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.959+0000 7fd637e00640 1 -- 172.21.15.16:0/2988531497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d815ab82d0 msgr2=0x55d815ee67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:42.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.959+0000 7fd637e00640 1 --2- 172.21.15.16:0/2988531497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d815ab82d0 0x55d815ee67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:42.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.959+0000 7fd637e00640 1 -- 172.21.15.16:0/2988531497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d815a8f410 msgr2=0x55d815e6f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:22:42.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.959+0000 7fd637e00640 1 --2- 172.21.15.16:0/2988531497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d815a8f410 0x55d815e6f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:42.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.959+0000 7fd637e00640 1 -- 172.21.15.16:0/2988531497 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d815e5dce0 con 0x55d815a975f0 2024-08-31T06:22:42.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.959+0000 7fd63ce00640 1 --2- 172.21.15.16:0/2988531497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d815a8f410 0x55d815e6f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:22:42.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.960+0000 7fd637e00640 1 --2- 172.21.15.16:0/2988531497 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d815a975f0 0x55d815e6f860 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fd6240114e0 tx=0x7fd624005cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33fb392694ddf13f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:42.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.960+0000 7fd61ea00640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:22:42.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.960+0000 7fd61ea00640 1 -- 172.21.15.16:0/2988531497 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62402f070 con 0x55d815a975f0 2024-08-31T06:22:42.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.960+0000 7fd6465f4740 1 -- 172.21.15.16:0/2988531497 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd62401f440 con 0x55d815a975f0 2024-08-31T06:22:42.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.960+0000 7fd61ea00640 1 -- 172.21.15.16:0/2988531497 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8470+0+0 (secure 0 0 0) 0x7fd62401f5f0 con 0x55d815a975f0 2024-08-31T06:22:42.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.960+0000 7fd61ea00640 1 -- 172.21.15.16:0/2988531497 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fd62401f960 con 0x55d815a975f0 2024-08-31T06:22:42.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.960+0000 7fd6465f4740 10 client.29080 fetch_fsmap learned FSMap version 54 2024-08-31T06:22:42.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.960+0000 7fd6465f4740 1 -- 172.21.15.16:0/2988531497 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55d815ee7e60 con 0x55d815a975f0 2024-08-31T06:22:42.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.961+0000 7fd61ea00640 1 -- 172.21.15.16:0/2988531497 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd62401f440 con 0x55d815a975f0 2024-08-31T06:22:42.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.961+0000 7fd61ea00640 1 -- 172.21.15.16:0/2988531497 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 54) ==== 8350+0+0 (secure 0 0 0) 0x7fd62401d020 con 0x55d815a975f0 2024-08-31T06:22:42.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.961+0000 7fd6465f4740 10 client.29080 fetch_fsmap finished waiting for FSMap version 54 2024-08-31T06:22:42.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.961+0000 7fd6465f4740 10 client.29080 resolve_mds: validated gid 24737 aka daemon mds.b 2024-08-31T06:22:42.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.961+0000 7fd6465f4740 20 client.29080 populate_metadata read hostname 'smithi016' 2024-08-31T06:22:42.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.961+0000 7fd6465f4740 1 --2- 172.21.15.16:0/2988531497 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d815a4c000 0x55d815eea380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:42.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.961+0000 7fd6465f4740 4 client.29080 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-08-31T06:22:42.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.961+0000 7fd6465f4740 1 -- 172.21.15.16:0/2988531497 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55d815dd2180 con 0x55d815a4c000 2024-08-31T06:22:42.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.961+0000 7fd63ce00640 1 --2- 172.21.15.16:0/2988531497 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d815a4c000 0x55d815eea380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:42.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.962+0000 7fd63ce00640 1 --2- 172.21.15.16:0/2988531497 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d815a4c000 0x55d815eea380 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=678511bf848dd249 server_cookie=df03b5385cd1c114 in_seq=0 out_seq=0 2024-08-31T06:22:42.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.962+0000 7fd61ea00640 10 client.29080 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:22:42.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.962+0000 7fd61ea00640 1 -- 172.21.15.16:0/2988531497 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55d815dd2180 con 0x55d815a4c000 2024-08-31T06:22:42.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.962+0000 7fd61ea00640 10 client.29080 handle_command_reply: tid=0 2024-08-31T06:22:42.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.962+0000 7fd6465f4740 1 client.29080 shutdown 2024-08-31T06:22:42.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.962+0000 7fd6465f4740 1 -- 172.21.15.16:0/2988531497 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d815a975f0 msgr2=0x55d815e6f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:42.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.962+0000 7fd6465f4740 1 --2- 172.21.15.16:0/2988531497 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d815a975f0 0x55d815e6f860 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fd6240114e0 tx=0x7fd624005cd0 comp rx=0 tx=0).stop 2024-08-31T06:22:42.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.962+0000 7fd6465f4740 1 -- 172.21.15.16:0/2988531497 shutdown_connections 2024-08-31T06:22:42.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.962+0000 7fd6465f4740 1 --2- 172.21.15.16:0/2988531497 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x55d815a4c000 0x55d815eea380 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:42.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.962+0000 7fd6465f4740 1 --2- 172.21.15.16:0/2988531497 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d815a975f0 0x55d815e6f860 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:42.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.962+0000 7fd6465f4740 1 --2- 172.21.15.16:0/2988531497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d815ab82d0 0x55d815ee67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:42.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.962+0000 7fd6465f4740 1 --2- 172.21.15.16:0/2988531497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d815a8f410 0x55d815e6f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:42.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.962+0000 7fd61ea00640 0 client.29080 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:22:42.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.962+0000 7fd6465f4740 1 -- 172.21.15.16:0/2988531497 >> 172.21.15.16:0/2988531497 conn(0x55d815aac6c0 msgr2=0x55d815e5c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:42.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.963+0000 7fd6465f4740 1 -- 172.21.15.16:0/2988531497 shutdown_connections 2024-08-31T06:22:42.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.963+0000 7fd6465f4740 1 -- 172.21.15.16:0/2988531497 wait complete. 2024-08-31T06:22:42.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.963+0000 7fd6465f4740 20 client.29080 trim_cache size 0 max 16384 2024-08-31T06:22:42.968 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:22:42.968 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 54, 2024-08-31T06:22:42.968 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24737, 2024-08-31T06:22:42.968 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 48, 2024-08-31T06:22:42.968 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": {} 2024-08-31T06:22:42.968 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:22:42.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.963+0000 7fd63f800640 1 -- 172.21.15.16:0/161695004 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd6200b4ca0 msgr2=0x7fd6200b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:42.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.963+0000 7fd63f800640 1 --2- 172.21.15.16:0/161695004 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd6200b4ca0 0x7fd6200b71b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd62c006010 tx=0x7fd62c009820 comp rx=0 tx=0).stop 2024-08-31T06:22:42.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.963+0000 7fd63f800640 1 -- 172.21.15.16:0/161695004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd63810fec0 msgr2=0x7fd6381a1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:42.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.963+0000 7fd63f800640 1 --2- 172.21.15.16:0/161695004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd63810fec0 0x7fd6381a1cb0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fd63000e970 tx=0x7fd63000ed70 comp rx=0 tx=0).stop 2024-08-31T06:22:42.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.963+0000 7fd63f800640 1 -- 172.21.15.16:0/161695004 shutdown_connections 2024-08-31T06:22:42.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.964+0000 7fd63f800640 1 --2- 172.21.15.16:0/161695004 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fd6200b4ca0 0x7fd6200b71b0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:42.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.964+0000 7fd63f800640 1 --2- 172.21.15.16:0/161695004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd638075e40 0x7fd63819ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:42.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.964+0000 7fd63f800640 1 --2- 172.21.15.16:0/161695004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd63810fec0 0x7fd6381a1cb0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:42.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.964+0000 7fd63f800640 1 --2- 172.21.15.16:0/161695004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6380754f0 0x7fd63819c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:42.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.964+0000 7fd63f800640 1 -- 172.21.15.16:0/161695004 >> 172.21.15.16:0/161695004 conn(0x7fd63807b5f0 msgr2=0x7fd6381099d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:42.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.964+0000 7fd63f800640 1 -- 172.21.15.16:0/161695004 shutdown_connections 2024-08-31T06:22:42.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:42.964+0000 7fd63f800640 1 -- 172.21.15.16:0/161695004 wait complete. 2024-08-31T06:22:42.990 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:42 smithi016 ceph-mon[17194]: pgmap v848: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 15 MiB/s wr, 1.50k op/s 2024-08-31T06:22:43.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:42 smithi184 ceph-mon[22331]: pgmap v848: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 15 MiB/s wr, 1.50k op/s 2024-08-31T06:22:43.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:42 smithi155 ceph-mon[21742]: pgmap v848: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 15 MiB/s wr, 1.50k op/s 2024-08-31T06:22:43.872 INFO:tasks.quiescer:done joining Quiescers 2024-08-31T06:22:43.872 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-08-31T06:22:43.883 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi016.front.sepia.ceph.com 2024-08-31T06:22:43.895 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-08-31T06:22:43.905 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-08-31T06:22:43.905 INFO:teuthology.task.exec:Executing custom commands... 2024-08-31T06:22:43.905 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-08-31T06:22:43.905 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-08-31T06:22:43.996 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-08-31T06:22:44.057 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-08-31T06:22:44.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:43 smithi016 ceph-mon[17194]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:22:44.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:43 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:22:44.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:43 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:22:44.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:43 smithi184 ceph-mon[22331]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:22:44.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:43 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:22:44.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:43 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:22:44.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.275+0000 7eff21600640 1 -- 172.21.15.16:0/1238512259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff1c10b1f0 msgr2=0x7eff1c10b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:44.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.275+0000 7eff21600640 1 --2- 172.21.15.16:0/1238512259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff1c10b1f0 0x7eff1c10b600 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7eff08009900 tx=0x7eff0802a3f0 comp rx=0 tx=0).stop 2024-08-31T06:22:44.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.275+0000 7eff21600640 1 -- 172.21.15.16:0/1238512259 shutdown_connections 2024-08-31T06:22:44.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.275+0000 7eff21600640 1 --2- 172.21.15.16:0/1238512259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff1c10bb40 0x7eff1c112370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:44.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.275+0000 7eff21600640 1 --2- 172.21.15.16:0/1238512259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff1c10b1f0 0x7eff1c10b600 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:44.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.275+0000 7eff21600640 1 --2- 172.21.15.16:0/1238512259 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff1c1058c0 0x7eff1c105cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:44.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.275+0000 7eff21600640 1 -- 172.21.15.16:0/1238512259 >> 172.21.15.16:0/1238512259 conn(0x7eff1c101300 msgr2=0x7eff1c103750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:44.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.275+0000 7eff21600640 1 -- 172.21.15.16:0/1238512259 shutdown_connections 2024-08-31T06:22:44.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.275+0000 7eff21600640 1 -- 172.21.15.16:0/1238512259 wait complete. 2024-08-31T06:22:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.276+0000 7eff21600640 1 Processor -- start 2024-08-31T06:22:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.276+0000 7eff21600640 1 -- start start 2024-08-31T06:22:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.276+0000 7eff21600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff1c1058c0 0x7eff1c19c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.277+0000 7eff21600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff1c10b1f0 0x7eff1c19c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.277+0000 7eff21600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff1c10bb40 0x7eff1c1a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.277+0000 7eff21600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c1157a0 con 0x7eff1c1058c0 2024-08-31T06:22:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.277+0000 7eff21600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c115620 con 0x7eff1c10bb40 2024-08-31T06:22:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.277+0000 7eff21600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c115920 con 0x7eff1c10b1f0 2024-08-31T06:22:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.277+0000 7eff1be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff1c1058c0 0x7eff1c19c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.277+0000 7eff1be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff1c1058c0 0x7eff1c19c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51736/0 (socket says 172.21.15.16:51736) 2024-08-31T06:22:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.277+0000 7eff1be00640 1 -- 172.21.15.16:0/2132414323 learned_addr learned my addr 172.21.15.16:0/2132414323 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.277+0000 7eff1b400640 1 --2- 172.21.15.16:0/2132414323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff1c10b1f0 0x7eff1c19c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.277+0000 7eff20c00640 1 --2- 172.21.15.16:0/2132414323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff1c10bb40 0x7eff1c1a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.277+0000 7eff1be00640 1 -- 172.21.15.16:0/2132414323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff1c10b1f0 msgr2=0x7eff1c19c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.277+0000 7eff1be00640 1 --2- 172.21.15.16:0/2132414323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff1c10b1f0 0x7eff1c19c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.277+0000 7eff1be00640 1 -- 172.21.15.16:0/2132414323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff1c10bb40 msgr2=0x7eff1c1a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.277+0000 7eff1be00640 1 --2- 172.21.15.16:0/2132414323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff1c10bb40 0x7eff1c1a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.277+0000 7eff1be00640 1 -- 172.21.15.16:0/2132414323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff1c1a2150 con 0x7eff1c1058c0 2024-08-31T06:22:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.278+0000 7eff1be00640 1 --2- 172.21.15.16:0/2132414323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff1c1058c0 0x7eff1c19c220 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7eff0c00e9f0 tx=0x7eff0c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.278+0000 7eff18c00640 1 -- 172.21.15.16:0/2132414323 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c00ccc0 con 0x7eff1c1058c0 2024-08-31T06:22:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.278+0000 7eff18c00640 1 -- 172.21.15.16:0/2132414323 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff0c0108e0 con 0x7eff1c1058c0 2024-08-31T06:22:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.278+0000 7eff21600640 1 -- 172.21.15.16:0/2132414323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff1c1a2440 con 0x7eff1c1058c0 2024-08-31T06:22:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.278+0000 7eff18c00640 1 -- 172.21.15.16:0/2132414323 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c018880 con 0x7eff1c1058c0 2024-08-31T06:22:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.278+0000 7eff21600640 1 -- 172.21.15.16:0/2132414323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff1c1a8bd0 con 0x7eff1c1058c0 2024-08-31T06:22:44.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.279+0000 7eff21600640 1 -- 172.21.15.16:0/2132414323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff1c106b30 con 0x7eff1c1058c0 2024-08-31T06:22:44.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.282+0000 7eff18c00640 1 -- 172.21.15.16:0/2132414323 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7eff0c010430 con 0x7eff1c1058c0 2024-08-31T06:22:44.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.283+0000 7eff18c00640 1 --2- 172.21.15.16:0/2132414323 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7eff000b4ca0 0x7eff000b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:44.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.283+0000 7eff18c00640 1 -- 172.21.15.16:0/2132414323 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8470+0+0 (secure 0 0 0) 0x7eff0c0e5de0 con 0x7eff1c1058c0 2024-08-31T06:22:44.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.283+0000 7eff1b400640 1 --2- 172.21.15.16:0/2132414323 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7eff000b4ca0 0x7eff000b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:44.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.284+0000 7eff1b400640 1 --2- 172.21.15.16:0/2132414323 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7eff000b4ca0 0x7eff000b71b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7eff08009230 tx=0x7eff08031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:44.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.285+0000 7eff18c00640 1 -- 172.21.15.16:0/2132414323 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7eff0c0acba0 con 0x7eff1c1058c0 2024-08-31T06:22:44.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:43 smithi155 ceph-mon[21742]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:22:44.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:43 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:22:44.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:43 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:22:44.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.503+0000 7eff21600640 1 -- 172.21.15.16:0/2132414323 --> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7eff1c0635a0 con 0x7eff000b4ca0 2024-08-31T06:22:44.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.513+0000 7eff18c00640 1 -- 172.21.15.16:0/2132414323 <== mgr.14220 v2:172.21.15.16:6800/4279751337 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7eff1c0635a0 con 0x7eff000b4ca0 2024-08-31T06:22:44.515 INFO:teuthology.orchestra.run.smithi016.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-08-31T00:00:00", "created": "2024-08-31T06:06:49", "first": "2024-08-31T06:07:00", "last": "2024-08-31T06:22:00", "last_pruned": "2024-08-31T06:22:00", "created_count": 16, "pruned_count": 10, "active": true} 2024-08-31T06:22:44.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.515+0000 7eff21600640 1 -- 172.21.15.16:0/2132414323 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7eff000b4ca0 msgr2=0x7eff000b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:44.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.515+0000 7eff21600640 1 --2- 172.21.15.16:0/2132414323 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7eff000b4ca0 0x7eff000b71b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7eff08009230 tx=0x7eff08031400 comp rx=0 tx=0).stop 2024-08-31T06:22:44.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.515+0000 7eff21600640 1 -- 172.21.15.16:0/2132414323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff1c1058c0 msgr2=0x7eff1c19c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:44.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.515+0000 7eff21600640 1 --2- 172.21.15.16:0/2132414323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff1c1058c0 0x7eff1c19c220 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7eff0c00e9f0 tx=0x7eff0c00edf0 comp rx=0 tx=0).stop 2024-08-31T06:22:44.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.516+0000 7eff21600640 1 -- 172.21.15.16:0/2132414323 shutdown_connections 2024-08-31T06:22:44.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.516+0000 7eff21600640 1 --2- 172.21.15.16:0/2132414323 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7eff000b4ca0 0x7eff000b71b0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:44.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.516+0000 7eff21600640 1 --2- 172.21.15.16:0/2132414323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff1c10b1f0 0x7eff1c19c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:44.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.516+0000 7eff21600640 1 --2- 172.21.15.16:0/2132414323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff1c10bb40 0x7eff1c1a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:44.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.516+0000 7eff21600640 1 --2- 172.21.15.16:0/2132414323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff1c1058c0 0x7eff1c19c220 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:44.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.516+0000 7eff21600640 1 -- 172.21.15.16:0/2132414323 >> 172.21.15.16:0/2132414323 conn(0x7eff1c101300 msgr2=0x7eff1c103720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:44.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.516+0000 7eff21600640 1 -- 172.21.15.16:0/2132414323 shutdown_connections 2024-08-31T06:22:44.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.516+0000 7eff21600640 1 -- 172.21.15.16:0/2132414323 wait complete. 2024-08-31T06:22:44.543 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-08-31T06:22:44.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.762+0000 7f5450200640 1 -- 172.21.15.16:0/2206824738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5448075e40 msgr2=0x7f544810f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:44.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.762+0000 7f5450200640 1 --2- 172.21.15.16:0/2206824738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5448075e40 0x7f544810f910 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f5434009900 tx=0x7f543402a3f0 comp rx=0 tx=0).stop 2024-08-31T06:22:44.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.762+0000 7f5450200640 1 -- 172.21.15.16:0/2206824738 shutdown_connections 2024-08-31T06:22:44.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.762+0000 7f5450200640 1 --2- 172.21.15.16:0/2206824738 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f544810fe50 0x7f5448112270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:44.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.762+0000 7f5450200640 1 --2- 172.21.15.16:0/2206824738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5448075e40 0x7f544810f910 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:44.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.762+0000 7f5450200640 1 --2- 172.21.15.16:0/2206824738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f54480754f0 0x7f5448075900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:44.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.762+0000 7f5450200640 1 -- 172.21.15.16:0/2206824738 >> 172.21.15.16:0/2206824738 conn(0x7f54480ff2a0 msgr2=0x7f54481016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:44.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.762+0000 7f5450200640 1 -- 172.21.15.16:0/2206824738 shutdown_connections 2024-08-31T06:22:44.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.762+0000 7f5450200640 1 -- 172.21.15.16:0/2206824738 wait complete. 2024-08-31T06:22:44.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.763+0000 7f5450200640 1 Processor -- start 2024-08-31T06:22:44.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.763+0000 7f5450200640 1 -- start start 2024-08-31T06:22:44.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.763+0000 7f5450200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54480754f0 0x7f544819c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:44.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.763+0000 7f5450200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5448075e40 0x7f544819c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:44.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.763+0000 7f5450200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f544810fe50 0x7f54481a1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:44.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.763+0000 7f5450200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5448115a10 con 0x7f54480754f0 2024-08-31T06:22:44.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.763+0000 7f5450200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5448115890 con 0x7f5448075e40 2024-08-31T06:22:44.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.763+0000 7f5450200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5448115b90 con 0x7f544810fe50 2024-08-31T06:22:44.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.764+0000 7f544d800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54480754f0 0x7f544819c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:44.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.764+0000 7f544d800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54480754f0 0x7f544819c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51748/0 (socket says 172.21.15.16:51748) 2024-08-31T06:22:44.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.764+0000 7f544d800640 1 -- 172.21.15.16:0/1922475756 learned_addr learned my addr 172.21.15.16:0/1922475756 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:22:44.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.764+0000 7f544ce00640 1 --2- 172.21.15.16:0/1922475756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5448075e40 0x7f544819c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:44.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.764+0000 7f544e200640 1 --2- 172.21.15.16:0/1922475756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f544810fe50 0x7f54481a1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:44.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.764+0000 7f544d800640 1 -- 172.21.15.16:0/1922475756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f544810fe50 msgr2=0x7f54481a1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:44.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.764+0000 7f544d800640 1 --2- 172.21.15.16:0/1922475756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f544810fe50 0x7f54481a1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:44.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.764+0000 7f544d800640 1 -- 172.21.15.16:0/1922475756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5448075e40 msgr2=0x7f544819c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:44.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.764+0000 7f544d800640 1 --2- 172.21.15.16:0/1922475756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5448075e40 0x7f544819c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:44.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.764+0000 7f544d800640 1 -- 172.21.15.16:0/1922475756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54481a2320 con 0x7f54480754f0 2024-08-31T06:22:44.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.764+0000 7f544d800640 1 --2- 172.21.15.16:0/1922475756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54480754f0 0x7f544819c3f0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f543c00e9f0 tx=0x7f543c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.765+0000 7f5446000640 1 -- 172.21.15.16:0/1922475756 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543c00cd80 con 0x7f54480754f0 2024-08-31T06:22:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.765+0000 7f5446000640 1 -- 172.21.15.16:0/1922475756 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f543c0108e0 con 0x7f54480754f0 2024-08-31T06:22:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.765+0000 7f5450200640 1 -- 172.21.15.16:0/1922475756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54481a2610 con 0x7f54480754f0 2024-08-31T06:22:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.765+0000 7f5446000640 1 -- 172.21.15.16:0/1922475756 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543c018980 con 0x7f54480754f0 2024-08-31T06:22:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.765+0000 7f5450200640 1 -- 172.21.15.16:0/1922475756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54481a8da0 con 0x7f54480754f0 2024-08-31T06:22:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.766+0000 7f5450200640 1 -- 172.21.15.16:0/1922475756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5448075900 con 0x7f54480754f0 2024-08-31T06:22:44.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.768+0000 7f5446000640 1 -- 172.21.15.16:0/1922475756 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f543c010430 con 0x7f54480754f0 2024-08-31T06:22:44.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.771+0000 7f5446000640 1 --2- 172.21.15.16:0/1922475756 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f54300b4d20 0x7f54300b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:22:44.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.771+0000 7f5446000640 1 -- 172.21.15.16:0/1922475756 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8470+0+0 (secure 0 0 0) 0x7f543c0e5cc0 con 0x7f54480754f0 2024-08-31T06:22:44.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.771+0000 7f544ce00640 1 --2- 172.21.15.16:0/1922475756 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f54300b4d20 0x7f54300b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:22:44.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.772+0000 7f544ce00640 1 --2- 172.21.15.16:0/1922475756 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f54300b4d20 0x7f54300b7230 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f5434031440 tx=0x7f5434005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:22:44.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.773+0000 7f5446000640 1 -- 172.21.15.16:0/1922475756 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f543c0ac430 con 0x7f54480754f0 2024-08-31T06:22:44.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:44.992+0000 7f5450200640 1 -- 172.21.15.16:0/1922475756 --> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f544810a090 con 0x7f54300b4d20 2024-08-31T06:22:45.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:45.001+0000 7f5446000640 1 -- 172.21.15.16:0/1922475756 <== mgr.14220 v2:172.21.15.16:6800/4279751337 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f544810a090 con 0x7f54300b4d20 2024-08-31T06:22:45.003 INFO:teuthology.orchestra.run.smithi016.stdout:/ 1m 6m3h 2024-08-31T06:22:45.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:45.003+0000 7f5450200640 1 -- 172.21.15.16:0/1922475756 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f54300b4d20 msgr2=0x7f54300b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:45.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:45.003+0000 7f5450200640 1 --2- 172.21.15.16:0/1922475756 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f54300b4d20 0x7f54300b7230 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f5434031440 tx=0x7f5434005a20 comp rx=0 tx=0).stop 2024-08-31T06:22:45.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:45.003+0000 7f5450200640 1 -- 172.21.15.16:0/1922475756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54480754f0 msgr2=0x7f544819c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:22:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:45.003+0000 7f5450200640 1 --2- 172.21.15.16:0/1922475756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54480754f0 0x7f544819c3f0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f543c00e9f0 tx=0x7f543c00eeb0 comp rx=0 tx=0).stop 2024-08-31T06:22:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:45.003+0000 7f5450200640 1 -- 172.21.15.16:0/1922475756 shutdown_connections 2024-08-31T06:22:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:45.003+0000 7f5450200640 1 --2- 172.21.15.16:0/1922475756 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f54300b4d20 0x7f54300b7230 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:45.003+0000 7f5450200640 1 --2- 172.21.15.16:0/1922475756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f544810fe50 0x7f54481a1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:45.003+0000 7f5450200640 1 --2- 172.21.15.16:0/1922475756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5448075e40 0x7f544819c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:45.003+0000 7f5450200640 1 --2- 172.21.15.16:0/1922475756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54480754f0 0x7f544819c3f0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:22:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:45.003+0000 7f5450200640 1 -- 172.21.15.16:0/1922475756 >> 172.21.15.16:0/1922475756 conn(0x7f54480ff2a0 msgr2=0x7f5448109980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:22:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:45.004+0000 7f5450200640 1 -- 172.21.15.16:0/1922475756 shutdown_connections 2024-08-31T06:22:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:22:45.004+0000 7f5450200640 1 -- 172.21.15.16:0/1922475756 wait complete. 2024-08-31T06:22:45.030 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-08-31T06:22:45.040 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-08-31T06:22:45.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:44 smithi016 ceph-mon[17194]: scrub summary: idle 2024-08-31T06:22:45.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:44 smithi016 ceph-mon[17194]: pgmap v849: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 8.0 MiB/s wr, 942 op/s 2024-08-31T06:22:45.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:44 smithi184 ceph-mon[22331]: scrub summary: idle 2024-08-31T06:22:45.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:44 smithi184 ceph-mon[22331]: pgmap v849: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 8.0 MiB/s wr, 942 op/s 2024-08-31T06:22:45.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:44 smithi155 ceph-mon[21742]: scrub summary: idle 2024-08-31T06:22:45.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:44 smithi155 ceph-mon[21742]: pgmap v849: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 8.0 MiB/s wr, 942 op/s 2024-08-31T06:22:46.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:45 smithi016 ceph-mon[17194]: from='client.18564 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:22:46.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:45 smithi184 ceph-mon[22331]: from='client.18564 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:22:46.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:45 smithi155 ceph-mon[21742]: from='client.18564 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:22:47.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:46 smithi016 ceph-mon[17194]: from='client.18570 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:22:47.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:46 smithi016 ceph-mon[17194]: pgmap v850: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 5.5 MiB/s wr, 702 op/s 2024-08-31T06:22:47.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:46 smithi184 ceph-mon[22331]: from='client.18570 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:22:47.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:46 smithi184 ceph-mon[22331]: pgmap v850: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 5.5 MiB/s wr, 702 op/s 2024-08-31T06:22:47.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:46 smithi155 ceph-mon[21742]: from='client.18570 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-31T06:22:47.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:46 smithi155 ceph-mon[21742]: pgmap v850: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 5.5 MiB/s wr, 702 op/s 2024-08-31T06:22:49.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:48 smithi016 ceph-mon[17194]: pgmap v851: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 2.1 MiB/s wr, 321 op/s 2024-08-31T06:22:49.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:48 smithi016 ceph-mon[17194]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:22:49.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:48 smithi184 ceph-mon[22331]: pgmap v851: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 2.1 MiB/s wr, 321 op/s 2024-08-31T06:22:49.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:48 smithi184 ceph-mon[22331]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:22:49.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:48 smithi155 ceph-mon[21742]: pgmap v851: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 2.1 MiB/s wr, 321 op/s 2024-08-31T06:22:49.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:48 smithi155 ceph-mon[21742]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-31T06:22:51.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:50 smithi016 ceph-mon[17194]: pgmap v852: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 1.6 MiB/s wr, 272 op/s 2024-08-31T06:22:51.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:50 smithi184 ceph-mon[22331]: pgmap v852: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 1.6 MiB/s wr, 272 op/s 2024-08-31T06:22:51.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:50 smithi155 ceph-mon[21742]: pgmap v852: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 1.6 MiB/s wr, 272 op/s 2024-08-31T06:22:53.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:52 smithi184 ceph-mon[22331]: pgmap v853: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 843 KiB/s wr, 144 op/s 2024-08-31T06:22:53.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:52 smithi155 ceph-mon[21742]: pgmap v853: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 843 KiB/s wr, 144 op/s 2024-08-31T06:22:53.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:52 smithi016 ceph-mon[17194]: pgmap v853: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 843 KiB/s wr, 144 op/s 2024-08-31T06:22:54.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:53 smithi184 ceph-mon[22331]: pgmap v854: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 66 KiB/s wr, 20 op/s 2024-08-31T06:22:54.314 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:53 smithi016 ceph-mon[17194]: pgmap v854: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 66 KiB/s wr, 20 op/s 2024-08-31T06:22:54.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:53 smithi155 ceph-mon[21742]: pgmap v854: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 66 KiB/s wr, 20 op/s 2024-08-31T06:22:57.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:56 smithi155 ceph-mon[21742]: pgmap v855: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 15 op/s 2024-08-31T06:22:57.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:56 smithi016 ceph-mon[17194]: pgmap v855: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 15 op/s 2024-08-31T06:22:57.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:56 smithi184 ceph-mon[22331]: pgmap v855: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 15 op/s 2024-08-31T06:22:59.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:22:58 smithi155 ceph-mon[21742]: pgmap v856: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 56 KiB/s wr, 12 op/s 2024-08-31T06:22:59.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:22:58 smithi016 ceph-mon[17194]: pgmap v856: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 56 KiB/s wr, 12 op/s 2024-08-31T06:22:59.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:22:58 smithi184 ceph-mon[22331]: pgmap v856: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 56 KiB/s wr, 12 op/s 2024-08-31T06:23:01.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:00 smithi155 ceph-mon[21742]: pgmap v857: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 43 KiB/s wr, 11 op/s 2024-08-31T06:23:01.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:00 smithi016 ceph-mon[17194]: pgmap v857: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 43 KiB/s wr, 11 op/s 2024-08-31T06:23:01.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:00 smithi184 ceph-mon[22331]: pgmap v857: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 43 KiB/s wr, 11 op/s 2024-08-31T06:23:03.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:02 smithi155 ceph-mon[21742]: pgmap v858: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 22 KiB/s wr, 9 op/s 2024-08-31T06:23:03.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:02 smithi016 ceph-mon[17194]: pgmap v858: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 22 KiB/s wr, 9 op/s 2024-08-31T06:23:03.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:02 smithi184 ceph-mon[22331]: pgmap v858: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 22 KiB/s wr, 9 op/s 2024-08-31T06:23:05.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:04 smithi155 ceph-mon[21742]: pgmap v859: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-08-31T06:23:05.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:04 smithi016 ceph-mon[17194]: pgmap v859: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-08-31T06:23:05.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:04 smithi184 ceph-mon[22331]: pgmap v859: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-08-31T06:23:07.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:06 smithi155 ceph-mon[21742]: pgmap v860: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 52 KiB/s wr, 25 op/s 2024-08-31T06:23:07.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:06 smithi016 ceph-mon[17194]: pgmap v860: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 52 KiB/s wr, 25 op/s 2024-08-31T06:23:07.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:06 smithi184 ceph-mon[22331]: pgmap v860: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 52 KiB/s wr, 25 op/s 2024-08-31T06:23:09.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:08 smithi155 ceph-mon[21742]: pgmap v861: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 60 KiB/s wr, 24 op/s 2024-08-31T06:23:09.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:08 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:23:09.166 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:08 smithi016 ceph-mon[17194]: pgmap v861: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 60 KiB/s wr, 24 op/s 2024-08-31T06:23:09.166 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:08 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:23:09.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:08 smithi184 ceph-mon[22331]: pgmap v861: 97 pgs: 97 active+clean; 3.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 60 KiB/s wr, 24 op/s 2024-08-31T06:23:09.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:08 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-31T06:23:10.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:09 smithi155 ceph-mon[21742]: osdmap e102: 12 total, 12 up, 12 in 2024-08-31T06:23:10.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:09 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T06:23:10.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:09 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T06:23:10.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:09 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T06:23:10.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:09 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T06:23:10.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:09 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-31T06:23:10.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:09 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-31T06:23:10.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:09 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-31T06:23:10.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:09 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-31T06:23:10.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:09 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:23:10.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:09 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:23:10.123 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:09 smithi155 ceph-mon[21742]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:23:10.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:09 smithi016 ceph-mon[17194]: osdmap e102: 12 total, 12 up, 12 in 2024-08-31T06:23:10.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:09 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T06:23:10.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:09 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T06:23:10.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:09 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T06:23:10.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:09 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T06:23:10.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:09 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-31T06:23:10.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:09 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-31T06:23:10.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:09 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-31T06:23:10.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:09 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-31T06:23:10.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:09 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:23:10.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:09 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:23:10.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:09 smithi016 ceph-mon[17194]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:23:10.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:09 smithi184 ceph-mon[22331]: osdmap e102: 12 total, 12 up, 12 in 2024-08-31T06:23:10.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:09 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T06:23:10.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:09 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-31T06:23:10.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:09 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T06:23:10.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:09 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-31T06:23:10.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:09 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-31T06:23:10.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:09 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-31T06:23:10.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:09 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-31T06:23:10.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:09 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-31T06:23:10.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:09 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-31T06:23:10.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:09 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-31T06:23:10.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:09 smithi184 ceph-mon[22331]: from='mgr.14220 ' entity='mgr.x' 2024-08-31T06:23:11.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:10 smithi155 ceph-mon[21742]: Adjusting osd_memory_target on smithi016 to 177.5M 2024-08-31T06:23:11.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:10 smithi155 ceph-mon[21742]: Unable to set osd_memory_target on smithi016 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-08-31T06:23:11.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:10 smithi155 ceph-mon[21742]: pgmap v863: 97 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 92 active+clean; 3.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 65 KiB/s wr, 28 op/s 2024-08-31T06:23:11.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:10 smithi016 ceph-mon[17194]: Adjusting osd_memory_target on smithi016 to 177.5M 2024-08-31T06:23:11.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:10 smithi016 ceph-mon[17194]: Unable to set osd_memory_target on smithi016 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-08-31T06:23:11.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:10 smithi016 ceph-mon[17194]: pgmap v863: 97 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 92 active+clean; 3.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 65 KiB/s wr, 28 op/s 2024-08-31T06:23:11.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:10 smithi184 ceph-mon[22331]: Adjusting osd_memory_target on smithi016 to 177.5M 2024-08-31T06:23:11.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:10 smithi184 ceph-mon[22331]: Unable to set osd_memory_target on smithi016 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-08-31T06:23:11.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:10 smithi184 ceph-mon[22331]: pgmap v863: 97 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 92 active+clean; 3.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 65 KiB/s wr, 28 op/s 2024-08-31T06:23:12.222 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-31T06:23:12.615 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:23:13.086 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:12 smithi016 ceph-mon[17194]: pgmap v864: 97 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 92 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 65 KiB/s wr, 28 op/s 2024-08-31T06:23:13.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:12 smithi184 ceph-mon[22331]: pgmap v864: 97 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 92 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 65 KiB/s wr, 28 op/s 2024-08-31T06:23:13.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.266+0000 7f5065800640 1 -- 172.21.15.16:0/1002137590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f506010b460 msgr2=0x7f506010b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:13.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.266+0000 7f5065800640 1 --2- 172.21.15.16:0/1002137590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f506010b460 0x7f506010b870 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f5048009900 tx=0x7f504802a410 comp rx=0 tx=0).stop 2024-08-31T06:23:13.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.266+0000 7f5065800640 1 -- 172.21.15.16:0/1002137590 shutdown_connections 2024-08-31T06:23:13.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.266+0000 7f5065800640 1 --2- 172.21.15.16:0/1002137590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f506010bdb0 0x7f50601125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:13.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.266+0000 7f5065800640 1 --2- 172.21.15.16:0/1002137590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f506010b460 0x7f506010b870 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:13.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.266+0000 7f5065800640 1 --2- 172.21.15.16:0/1002137590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5060105b20 0x7f5060105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:13.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.266+0000 7f5065800640 1 -- 172.21.15.16:0/1002137590 >> 172.21.15.16:0/1002137590 conn(0x7f50601015b0 msgr2=0x7f5060103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:13.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.266+0000 7f5065800640 1 -- 172.21.15.16:0/1002137590 shutdown_connections 2024-08-31T06:23:13.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.266+0000 7f5065800640 1 -- 172.21.15.16:0/1002137590 wait complete. 2024-08-31T06:23:13.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.267+0000 7f5065800640 1 Processor -- start 2024-08-31T06:23:13.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.267+0000 7f5065800640 1 -- start start 2024-08-31T06:23:13.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.268+0000 7f5065800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5060105b20 0x7f506019c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:13.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.268+0000 7f5065800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f506010b460 0x7f506019c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:13.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.268+0000 7f5065800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f506010bdb0 0x7f50601a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:13.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.268+0000 7f5065800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5060115ac0 con 0x7f506010bdb0 2024-08-31T06:23:13.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.268+0000 7f5065800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5060115940 con 0x7f506010b460 2024-08-31T06:23:13.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.268+0000 7f5065800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5060115c40 con 0x7f5060105b20 2024-08-31T06:23:13.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.268+0000 7f505f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f506010bdb0 0x7f50601a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:13.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.268+0000 7f505f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f506010bdb0 0x7f50601a1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34626/0 (socket says 172.21.15.16:34626) 2024-08-31T06:23:13.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.268+0000 7f505f400640 1 -- 172.21.15.16:0/2436504190 learned_addr learned my addr 172.21.15.16:0/2436504190 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:23:13.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.268+0000 7f5057e00640 1 --2- 172.21.15.16:0/2436504190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f506010b460 0x7f506019c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:13.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.268+0000 7f505ea00640 1 --2- 172.21.15.16:0/2436504190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5060105b20 0x7f506019c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:13.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.269+0000 7f505f400640 1 -- 172.21.15.16:0/2436504190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5060105b20 msgr2=0x7f506019c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:13.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.269+0000 7f505f400640 1 --2- 172.21.15.16:0/2436504190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5060105b20 0x7f506019c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:13.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.269+0000 7f505f400640 1 -- 172.21.15.16:0/2436504190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f506010b460 msgr2=0x7f506019c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:13.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.269+0000 7f505f400640 1 --2- 172.21.15.16:0/2436504190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f506010b460 0x7f506019c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:13.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.269+0000 7f505f400640 1 -- 172.21.15.16:0/2436504190 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50601a23c0 con 0x7f506010bdb0 2024-08-31T06:23:13.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.269+0000 7f5057e00640 1 --2- 172.21.15.16:0/2436504190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f506010b460 0x7f506019c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:23:13.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.269+0000 7f505f400640 1 --2- 172.21.15.16:0/2436504190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f506010bdb0 0x7f50601a1c90 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f505800eb10 tx=0x7f505800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:13.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.269+0000 7f5057400640 1 -- 172.21.15.16:0/2436504190 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5058004b20 con 0x7f506010bdb0 2024-08-31T06:23:13.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.270+0000 7f5057400640 1 -- 172.21.15.16:0/2436504190 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5058004cc0 con 0x7f506010bdb0 2024-08-31T06:23:13.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.270+0000 7f5065800640 1 -- 172.21.15.16:0/2436504190 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50601a26b0 con 0x7f506010bdb0 2024-08-31T06:23:13.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.270+0000 7f5057400640 1 -- 172.21.15.16:0/2436504190 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5058010670 con 0x7f506010bdb0 2024-08-31T06:23:13.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.270+0000 7f5065800640 1 -- 172.21.15.16:0/2436504190 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50601a8e10 con 0x7f506010bdb0 2024-08-31T06:23:13.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.273+0000 7f506c3c5740 1 Processor -- start 2024-08-31T06:23:13.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.274+0000 7f5057400640 1 -- 172.21.15.16:0/2436504190 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f5058005020 con 0x7f506010bdb0 2024-08-31T06:23:13.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.274+0000 7f506c3c5740 1 -- start start 2024-08-31T06:23:13.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.274+0000 7f506c3c5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bd448a070 0x564bd482f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:13.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.275+0000 7f506c3c5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd44611b0 0x564bd482fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:13.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.275+0000 7f506c3c5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd4469390 0x564bd48302f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:13.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.275+0000 7f506c3c5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564bd47a6770 con 0x564bd4469390 2024-08-31T06:23:13.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.275+0000 7f506c3c5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x564bd4795d00 con 0x564bd44611b0 2024-08-31T06:23:13.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.275+0000 7f506c3c5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564bd4799cb0 con 0x564bd448a070 2024-08-31T06:23:13.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.275+0000 7f505ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd44611b0 0x564bd482fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:13.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.275+0000 7f5057e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bd448a070 0x564bd482f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:13.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.275+0000 7f505ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd44611b0 0x564bd482fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:47292/0 (socket says 172.21.15.16:47292) 2024-08-31T06:23:13.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.275+0000 7f505ea00640 1 -- 172.21.15.16:0/2757454387 learned_addr learned my addr 172.21.15.16:0/2757454387 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:23:13.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.275+0000 7f5057e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bd448a070 0x564bd482f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:35062/0 (socket says 172.21.15.16:35062) 2024-08-31T06:23:13.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.275+0000 7f5057e00640 1 --2- 172.21.15.16:0/2757454387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd4469390 0x564bd48302f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:13.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.275+0000 7f505ea00640 1 -- 172.21.15.16:0/2757454387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bd448a070 msgr2=0x564bd482f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:23:13.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.275+0000 7f505ea00640 1 --2- 172.21.15.16:0/2757454387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bd448a070 0x564bd482f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:13.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.275+0000 7f505ea00640 1 -- 172.21.15.16:0/2757454387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd4469390 msgr2=0x564bd48302f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:23:13.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.275+0000 7f505ea00640 1 --2- 172.21.15.16:0/2757454387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd4469390 0x564bd48302f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:13.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.275+0000 7f505ea00640 1 -- 172.21.15.16:0/2757454387 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bd44f5650 con 0x564bd44611b0 2024-08-31T06:23:13.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.275+0000 7f5057e00640 1 --2- 172.21.15.16:0/2757454387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd4469390 0x564bd48302f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:23:13.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.276+0000 7f5057400640 1 --2- 172.21.15.16:0/2436504190 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f50440b4d20 0x7f50440b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:13.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.276+0000 7f505ea00640 1 --2- 172.21.15.16:0/2757454387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd44611b0 0x564bd482fdb0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f5050009cf0 tx=0x7f5050011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3c8136159c44ae1 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:13.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.276+0000 7f5057400640 1 -- 172.21.15.16:0/2436504190 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8514+0+0 (secure 0 0 0) 0x7f50580e5a80 con 0x7f506010bdb0 2024-08-31T06:23:13.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.276+0000 7f5057e00640 1 --2- 172.21.15.16:0/2436504190 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f50440b4d20 0x7f50440b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:13.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.276+0000 7f504fe00640 1 -- 172.21.15.16:0/2757454387 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5050004230 con 0x564bd44611b0 2024-08-31T06:23:13.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.276+0000 7f504fe00640 1 -- 172.21.15.16:0/2757454387 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50500043d0 con 0x564bd44611b0 2024-08-31T06:23:13.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.276+0000 7f504fe00640 1 -- 172.21.15.16:0/2757454387 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5050019880 con 0x564bd44611b0 2024-08-31T06:23:13.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.277+0000 7f506c3c5740 1 -- 172.21.15.16:0/2757454387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd44611b0 msgr2=0x564bd482fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:13.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.277+0000 7f506c3c5740 1 --2- 172.21.15.16:0/2757454387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd44611b0 0x564bd482fdb0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f5050009cf0 tx=0x7f5050011810 comp rx=0 tx=0).stop 2024-08-31T06:23:13.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.277+0000 7f5057e00640 1 --2- 172.21.15.16:0/2436504190 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f50440b4d20 0x7f50440b7230 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f504802aa00 tx=0x7f5048005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:13.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.277+0000 7f506c3c5740 1 -- 172.21.15.16:0/2757454387 shutdown_connections 2024-08-31T06:23:13.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.277+0000 7f506c3c5740 1 --2- 172.21.15.16:0/2757454387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd4469390 0x564bd48302f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:13.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.277+0000 7f506c3c5740 1 --2- 172.21.15.16:0/2757454387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd44611b0 0x564bd482fdb0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:13.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.277+0000 7f506c3c5740 1 --2- 172.21.15.16:0/2757454387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bd448a070 0x564bd482f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:13.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.277+0000 7f506c3c5740 1 -- 172.21.15.16:0/2757454387 >> 172.21.15.16:0/2757454387 conn(0x564bd447e460 msgr2=0x564bd482d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:13.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.277+0000 7f506c3c5740 1 -- 172.21.15.16:0/2757454387 shutdown_connections 2024-08-31T06:23:13.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.277+0000 7f506c3c5740 1 -- 172.21.15.16:0/2757454387 wait complete. 2024-08-31T06:23:13.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.278+0000 7f506c3c5740 1 -- start start 2024-08-31T06:23:13.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.278+0000 7f506c3c5740 1 Processor -- start 2024-08-31T06:23:13.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.279+0000 7f506c3c5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd44611b0 0x564bd483b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:13.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.279+0000 7f506c3c5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd4469390 0x564bd483b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:13.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.279+0000 7f506c3c5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bd448a070 0x564bd48473a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:13.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.279+0000 7f506c3c5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564bd45f8700 con 0x564bd44611b0 2024-08-31T06:23:13.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.279+0000 7f506c3c5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x564bd471ccb0 con 0x564bd4469390 2024-08-31T06:23:13.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.279+0000 7f506c3c5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564bd483e050 con 0x564bd448a070 2024-08-31T06:23:13.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.279+0000 7f506c3c5740 10 client.0 _ll_register_callbacks cb 0x564bd4742430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:23:13.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.279+0000 7f505ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd44611b0 0x564bd483b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:13.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.279+0000 7f505ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd44611b0 0x564bd483b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34640/0 (socket says 172.21.15.16:34640) 2024-08-31T06:23:13.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.279+0000 7f505ea00640 1 -- 172.21.15.16:0/2706746114 learned_addr learned my addr 172.21.15.16:0/2706746114 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:23:13.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.279+0000 7f5057e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd4469390 0x564bd483b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:13.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.279+0000 7f505f400640 1 --2- 172.21.15.16:0/2706746114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bd448a070 0x564bd48473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:13.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.280+0000 7f505ea00640 1 -- 172.21.15.16:0/2706746114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bd448a070 msgr2=0x564bd48473a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:23:13.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.280+0000 7f505ea00640 1 --2- 172.21.15.16:0/2706746114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bd448a070 0x564bd48473a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:13.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.280+0000 7f505ea00640 1 -- 172.21.15.16:0/2706746114 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd4469390 msgr2=0x564bd483b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:23:13.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.280+0000 7f505ea00640 1 --2- 172.21.15.16:0/2706746114 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd4469390 0x564bd483b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:13.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.280+0000 7f505ea00640 1 -- 172.21.15.16:0/2706746114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bd475d870 con 0x564bd44611b0 2024-08-31T06:23:13.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.280+0000 7f505ea00640 1 --2- 172.21.15.16:0/2706746114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd44611b0 0x564bd483b420 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f5050009560 tx=0x7f5050003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=829fb0c0f99e2c52 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:13.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.280+0000 7f504cc00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:23:13.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.280+0000 7f504cc00640 1 -- 172.21.15.16:0/2706746114 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5050027070 con 0x564bd44611b0 2024-08-31T06:23:13.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.280+0000 7f504cc00640 1 -- 172.21.15.16:0/2706746114 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8514+0+0 (secure 0 0 0) 0x7f5050005240 con 0x564bd44611b0 2024-08-31T06:23:13.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.280+0000 7f506c3c5740 1 -- 172.21.15.16:0/2706746114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50500043d0 con 0x564bd44611b0 2024-08-31T06:23:13.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.281+0000 7f504cc00640 1 -- 172.21.15.16:0/2706746114 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f5050033050 con 0x564bd44611b0 2024-08-31T06:23:13.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.281+0000 7f506c3c5740 10 client.18582 fetch_fsmap learned FSMap version 55 2024-08-31T06:23:13.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.281+0000 7f506c3c5740 1 -- 172.21.15.16:0/2706746114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x564bd484a0c0 con 0x564bd44611b0 2024-08-31T06:23:13.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.281+0000 7f504cc00640 1 -- 172.21.15.16:0/2706746114 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50500043d0 con 0x564bd44611b0 2024-08-31T06:23:13.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.281+0000 7f504cc00640 1 -- 172.21.15.16:0/2706746114 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 55) ==== 8358+0+0 (secure 0 0 0) 0x7f5050004b70 con 0x564bd44611b0 2024-08-31T06:23:13.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.281+0000 7f506c3c5740 10 client.18582 fetch_fsmap finished waiting for FSMap version 55 2024-08-31T06:23:13.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.281+0000 7f506c3c5740 10 client.18582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:23:13.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.282+0000 7f506c3c5740 20 client.18582 populate_metadata read hostname 'smithi016' 2024-08-31T06:23:13.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.282+0000 7f506c3c5740 1 --2- 172.21.15.16:0/2706746114 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564bd44489f0 0x564bd484d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:13.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.282+0000 7f506c3c5740 4 client.18582 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:23:13.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.282+0000 7f506c3c5740 1 -- 172.21.15.16:0/2706746114 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564bd47a86c0 con 0x564bd44489f0 2024-08-31T06:23:13.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.282+0000 7f5057e00640 1 --2- 172.21.15.16:0/2706746114 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564bd44489f0 0x564bd484d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:13.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.283+0000 7f5057e00640 1 --2- 172.21.15.16:0/2706746114 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564bd44489f0 0x564bd484d480 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=995ec7fe55bf73ee server_cookie=f633f84bc317b3c in_seq=0 out_seq=0 2024-08-31T06:23:13.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.283+0000 7f504cc00640 10 client.18582 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:23:13.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.283+0000 7f504cc00640 1 -- 172.21.15.16:0/2706746114 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564bd47a86c0 con 0x564bd44489f0 2024-08-31T06:23:13.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.283+0000 7f504cc00640 10 client.18582 handle_command_reply: tid=0 2024-08-31T06:23:13.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.284+0000 7f506c3c5740 1 client.18582 shutdown 2024-08-31T06:23:13.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.284+0000 7f506c3c5740 1 -- 172.21.15.16:0/2706746114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd44611b0 msgr2=0x564bd483b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:13.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.284+0000 7f506c3c5740 1 --2- 172.21.15.16:0/2706746114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd44611b0 0x564bd483b420 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f5050009560 tx=0x7f5050003820 comp rx=0 tx=0).stop 2024-08-31T06:23:13.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.284+0000 7f506c3c5740 1 -- 172.21.15.16:0/2706746114 shutdown_connections 2024-08-31T06:23:13.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.284+0000 7f506c3c5740 1 --2- 172.21.15.16:0/2706746114 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564bd44489f0 0x564bd484d480 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:13.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.284+0000 7f506c3c5740 1 --2- 172.21.15.16:0/2706746114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bd448a070 0x564bd48473a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:13.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.284+0000 7f506c3c5740 1 --2- 172.21.15.16:0/2706746114 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd4469390 0x564bd483b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:13.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.284+0000 7f506c3c5740 1 --2- 172.21.15.16:0/2706746114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd44611b0 0x564bd483b420 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:13.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.284+0000 7f504cc00640 0 client.18582 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:23:13.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.284+0000 7f506c3c5740 1 -- 172.21.15.16:0/2706746114 >> 172.21.15.16:0/2706746114 conn(0x564bd447e460 msgr2=0x564bd4832a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:13.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.284+0000 7f506c3c5740 1 -- 172.21.15.16:0/2706746114 shutdown_connections 2024-08-31T06:23:13.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.284+0000 7f506c3c5740 1 -- 172.21.15.16:0/2706746114 wait complete. 2024-08-31T06:23:13.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.284+0000 7f506c3c5740 20 client.18582 trim_cache size 0 max 16384 2024-08-31T06:23:13.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.303+0000 7f506c3c5740 1 Processor -- start 2024-08-31T06:23:13.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.303+0000 7f506c3c5740 1 -- start start 2024-08-31T06:23:13.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.303+0000 7f506c3c5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd44611b0 0x564bd482d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:13.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.303+0000 7f506c3c5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd4469390 0x564bd482c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:13.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.303+0000 7f506c3c5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bd448a070 0x564bd4835990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:13.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.303+0000 7f506c3c5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f50601b4010 con 0x564bd44611b0 2024-08-31T06:23:13.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.303+0000 7f506c3c5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f50601b18b0 con 0x564bd4469390 2024-08-31T06:23:13.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.303+0000 7f506c3c5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f50601af720 con 0x564bd448a070 2024-08-31T06:23:13.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.304+0000 7f5057e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd4469390 0x564bd482c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:13.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.304+0000 7f5057e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd4469390 0x564bd482c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:47310/0 (socket says 172.21.15.16:47310) 2024-08-31T06:23:13.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.304+0000 7f5057e00640 1 -- 172.21.15.16:0/552903157 learned_addr learned my addr 172.21.15.16:0/552903157 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:23:13.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.304+0000 7f505ea00640 1 --2- 172.21.15.16:0/552903157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd44611b0 0x564bd482d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:13.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.304+0000 7f505f400640 1 --2- 172.21.15.16:0/552903157 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bd448a070 0x564bd4835990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:13.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.304+0000 7f5057e00640 1 -- 172.21.15.16:0/552903157 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bd448a070 msgr2=0x564bd4835990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:23:13.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.304+0000 7f5057e00640 1 --2- 172.21.15.16:0/552903157 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bd448a070 0x564bd4835990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:13.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.304+0000 7f5057e00640 1 -- 172.21.15.16:0/552903157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd44611b0 msgr2=0x564bd482d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:23:13.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.304+0000 7f5057e00640 1 --2- 172.21.15.16:0/552903157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd44611b0 0x564bd482d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:13.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.304+0000 7f5057e00640 1 -- 172.21.15.16:0/552903157 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bd48326d0 con 0x564bd4469390 2024-08-31T06:23:13.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.304+0000 7f505ea00640 1 --2- 172.21.15.16:0/552903157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd44611b0 0x564bd482d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:23:13.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.304+0000 7f5057e00640 1 --2- 172.21.15.16:0/552903157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd4469390 0x564bd482c760 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f506006c840 tx=0x7f5048009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11fee7e6a45a503d server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:13.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.305+0000 7f504f400640 1 -- 172.21.15.16:0/552903157 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5048054440 con 0x564bd4469390 2024-08-31T06:23:13.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.305+0000 7f504f400640 1 -- 172.21.15.16:0/552903157 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5048054a90 con 0x564bd4469390 2024-08-31T06:23:13.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.305+0000 7f504f400640 1 -- 172.21.15.16:0/552903157 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5048031ba0 con 0x564bd4469390 2024-08-31T06:23:13.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.306+0000 7f506c3c5740 1 -- 172.21.15.16:0/552903157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd4469390 msgr2=0x564bd482c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:13.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.306+0000 7f506c3c5740 1 --2- 172.21.15.16:0/552903157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd4469390 0x564bd482c760 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f506006c840 tx=0x7f5048009260 comp rx=0 tx=0).stop 2024-08-31T06:23:13.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.306+0000 7f506c3c5740 1 -- 172.21.15.16:0/552903157 shutdown_connections 2024-08-31T06:23:13.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.306+0000 7f506c3c5740 1 --2- 172.21.15.16:0/552903157 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bd448a070 0x564bd4835990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:13.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.306+0000 7f506c3c5740 1 --2- 172.21.15.16:0/552903157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd4469390 0x564bd482c760 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:13.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.306+0000 7f506c3c5740 1 --2- 172.21.15.16:0/552903157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd44611b0 0x564bd482d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:13.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.306+0000 7f506c3c5740 1 -- 172.21.15.16:0/552903157 >> 172.21.15.16:0/552903157 conn(0x564bd447e460 msgr2=0x564bd48308a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:13.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.306+0000 7f506c3c5740 1 -- 172.21.15.16:0/552903157 shutdown_connections 2024-08-31T06:23:13.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.306+0000 7f506c3c5740 1 -- 172.21.15.16:0/552903157 wait complete. 2024-08-31T06:23:13.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.306+0000 7f506c3c5740 1 -- start start 2024-08-31T06:23:13.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.307+0000 7f506c3c5740 1 Processor -- start 2024-08-31T06:23:13.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.307+0000 7f506c3c5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bd44611b0 0x564bd483f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:13.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.307+0000 7f506c3c5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd4469390 0x564bd483fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:13.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.307+0000 7f506c3c5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd448a070 0x564bd483ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:13.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.307+0000 7f506c3c5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f50601afd70 con 0x564bd448a070 2024-08-31T06:23:13.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.307+0000 7f506c3c5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f50601b11e0 con 0x564bd4469390 2024-08-31T06:23:13.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.307+0000 7f506c3c5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f50601b0440 con 0x564bd44611b0 2024-08-31T06:23:13.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.307+0000 7f506c3c5740 10 client.0 _ll_register_callbacks cb 0x7f5050009680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:23:13.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.307+0000 7f505f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd448a070 0x564bd483ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:13.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.307+0000 7f505f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd448a070 0x564bd483ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34650/0 (socket says 172.21.15.16:34650) 2024-08-31T06:23:13.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.307+0000 7f505f400640 1 -- 172.21.15.16:0/664881191 learned_addr learned my addr 172.21.15.16:0/664881191 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:23:13.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.307+0000 7f5057e00640 1 --2- 172.21.15.16:0/664881191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd4469390 0x564bd483fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:13.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.308+0000 7f505ea00640 1 --2- 172.21.15.16:0/664881191 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bd44611b0 0x564bd483f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:13.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.308+0000 7f505f400640 1 -- 172.21.15.16:0/664881191 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bd44611b0 msgr2=0x564bd483f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:23:13.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.308+0000 7f505f400640 1 --2- 172.21.15.16:0/664881191 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bd44611b0 0x564bd483f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:13.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.308+0000 7f505f400640 1 -- 172.21.15.16:0/664881191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd4469390 msgr2=0x564bd483fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:23:13.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.308+0000 7f505f400640 1 --2- 172.21.15.16:0/664881191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd4469390 0x564bd483fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:13.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.308+0000 7f505f400640 1 -- 172.21.15.16:0/664881191 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bd48b6f00 con 0x564bd448a070 2024-08-31T06:23:13.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.308+0000 7f505f400640 1 --2- 172.21.15.16:0/664881191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd448a070 0x564bd483ff60 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f50580ac0a0 tx=0x7f50580e36f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f41dac45bfbf6ac6 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:13.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.308+0000 7f504cc00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:23:13.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.308+0000 7f504cc00640 1 -- 172.21.15.16:0/664881191 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50580ac810 con 0x564bd448a070 2024-08-31T06:23:13.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.308+0000 7f504cc00640 1 -- 172.21.15.16:0/664881191 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8514+0+0 (secure 0 0 0) 0x7f50580b51b0 con 0x564bd448a070 2024-08-31T06:23:13.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.309+0000 7f506c3c5740 1 -- 172.21.15.16:0/664881191 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5048054a90 con 0x564bd448a070 2024-08-31T06:23:13.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.309+0000 7f504cc00640 1 -- 172.21.15.16:0/664881191 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f50580ac110 con 0x564bd448a070 2024-08-31T06:23:13.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.309+0000 7f506c3c5740 10 client.18588 fetch_fsmap learned FSMap version 55 2024-08-31T06:23:13.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.309+0000 7f506c3c5740 1 -- 172.21.15.16:0/664881191 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x564bd48b8190 con 0x564bd448a070 2024-08-31T06:23:13.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.309+0000 7f504cc00640 1 -- 172.21.15.16:0/664881191 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5048054a90 con 0x564bd448a070 2024-08-31T06:23:13.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.309+0000 7f504cc00640 1 -- 172.21.15.16:0/664881191 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 55) ==== 8358+0+0 (secure 0 0 0) 0x7f50580a7a80 con 0x564bd448a070 2024-08-31T06:23:13.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.309+0000 7f506c3c5740 10 client.18588 fetch_fsmap finished waiting for FSMap version 55 2024-08-31T06:23:13.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.309+0000 7f506c3c5740 10 client.18588 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:23:13.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.309+0000 7f506c3c5740 20 client.18588 populate_metadata read hostname 'smithi016' 2024-08-31T06:23:13.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.309+0000 7f506c3c5740 1 --2- 172.21.15.16:0/664881191 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564bd441dda0 0x564bd48ba950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:13.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.309+0000 7f506c3c5740 4 client.18588 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-31T06:23:13.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.309+0000 7f506c3c5740 1 -- 172.21.15.16:0/664881191 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564bd47a4160 con 0x564bd441dda0 2024-08-31T06:23:13.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.310+0000 7f505ea00640 1 --2- 172.21.15.16:0/664881191 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564bd441dda0 0x564bd48ba950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:13.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.310+0000 7f505ea00640 1 --2- 172.21.15.16:0/664881191 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564bd441dda0 0x564bd48ba950 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7804d0f9288f494 server_cookie=8d81b808c8ade8a6 in_seq=0 out_seq=0 2024-08-31T06:23:13.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.310+0000 7f504cc00640 10 client.18588 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:23:13.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.311+0000 7f504cc00640 1 -- 172.21.15.16:0/664881191 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5486 (crc 0 0 0) 0x564bd47a4160 con 0x564bd441dda0 2024-08-31T06:23:13.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.311+0000 7f504cc00640 10 client.18588 handle_command_reply: tid=0 2024-08-31T06:23:13.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.311+0000 7f506c3c5740 1 client.18588 shutdown 2024-08-31T06:23:13.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.311+0000 7f506c3c5740 1 -- 172.21.15.16:0/664881191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd448a070 msgr2=0x564bd483ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:13.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.311+0000 7f506c3c5740 1 --2- 172.21.15.16:0/664881191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd448a070 0x564bd483ff60 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f50580ac0a0 tx=0x7f50580e36f0 comp rx=0 tx=0).stop 2024-08-31T06:23:13.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.311+0000 7f506c3c5740 1 -- 172.21.15.16:0/664881191 shutdown_connections 2024-08-31T06:23:13.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.311+0000 7f506c3c5740 1 --2- 172.21.15.16:0/664881191 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x564bd441dda0 0x564bd48ba950 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:13.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.311+0000 7f506c3c5740 1 --2- 172.21.15.16:0/664881191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd448a070 0x564bd483ff60 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:13.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.311+0000 7f506c3c5740 1 --2- 172.21.15.16:0/664881191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd4469390 0x564bd483fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:13.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.311+0000 7f506c3c5740 1 --2- 172.21.15.16:0/664881191 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564bd44611b0 0x564bd483f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:13.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.311+0000 7f504cc00640 0 client.18588 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:23:13.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.311+0000 7f506c3c5740 1 -- 172.21.15.16:0/664881191 >> 172.21.15.16:0/664881191 conn(0x564bd447e460 msgr2=0x564bd4831260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:13.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.311+0000 7f506c3c5740 1 -- 172.21.15.16:0/664881191 shutdown_connections 2024-08-31T06:23:13.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.311+0000 7f506c3c5740 1 -- 172.21.15.16:0/664881191 wait complete. 2024-08-31T06:23:13.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.311+0000 7f506c3c5740 20 client.18588 trim_cache size 0 max 16384 2024-08-31T06:23:13.317 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:23:13.317 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-08-31T06:23:13.317 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {}, 2024-08-31T06:23:13.317 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-08-31T06:23:13.318 INFO:teuthology.orchestra.run.smithi016.stdout: "ab1ded0f-9efd-4559-88af-72427ba8460d": { 2024-08-31T06:23:13.318 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:22:37.396566+0000", 2024-08-31T06:23:13.318 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:23:13.318 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:23:13.318 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:23:13.318 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:23:13.318 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:23:13.318 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:13.318 INFO:teuthology.orchestra.run.smithi016.stdout: "18e112f4-a5f9-4602-aaad-3b13fa5a4c2e": { 2024-08-31T06:23:13.318 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:20:16.570535+0000", 2024-08-31T06:23:13.318 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:23:13.318 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:23:13.318 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:23:13.319 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:23:13.319 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:23:13.319 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:13.319 INFO:teuthology.orchestra.run.smithi016.stdout: "f1cdcb27-e3e4-4dc8-a0ae-7ace761d04c5": { 2024-08-31T06:23:13.319 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:19:22.275683+0000", 2024-08-31T06:23:13.319 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:23:13.319 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:23:13.319 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:23:13.319 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:23:13.319 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:23:13.319 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:13.319 INFO:teuthology.orchestra.run.smithi016.stdout: "0e44eb18-439c-46e4-a298-7d4d2ecd0b4a": { 2024-08-31T06:23:13.319 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:18:31.130765+0000", 2024-08-31T06:23:13.319 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:23:13.320 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:23:13.320 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:23:13.320 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:23:13.320 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:23:13.320 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:13.320 INFO:teuthology.orchestra.run.smithi016.stdout: "e497cd88-1e7c-4631-a4a8-cdd8e1fa6c83": { 2024-08-31T06:23:13.320 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:17:42.476795+0000", 2024-08-31T06:23:13.320 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:23:13.320 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:23:13.320 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:23:13.320 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:23:13.320 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:23:13.320 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:13.321 INFO:teuthology.orchestra.run.smithi016.stdout: "c300b7e8-f3fd-482c-a909-9329bc61e570": { 2024-08-31T06:23:13.321 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:06:49.868046+0000", 2024-08-31T06:23:13.321 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:23:13.321 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:23:13.321 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:23:13.321 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:23:13.321 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:23:13.321 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:13.321 INFO:teuthology.orchestra.run.smithi016.stdout: "6a03046e-cdf4-4a14-b9bd-b560b8968467": { 2024-08-31T06:23:13.321 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:12:38.002354+0000", 2024-08-31T06:23:13.321 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:23:13.322 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:23:13.322 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:23:13.322 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:23:13.322 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:23:13.322 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:13.322 INFO:teuthology.orchestra.run.smithi016.stdout: "004b47bb-0fe8-4d28-9d9d-78bde6a61199": { 2024-08-31T06:23:13.322 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:16:48.915926+0000", 2024-08-31T06:23:13.322 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:23:13.322 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:23:13.322 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:23:13.322 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:23:13.322 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:23:13.322 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:13.323 INFO:teuthology.orchestra.run.smithi016.stdout: "88d7bdb8-cb1c-45e5-ad29-4cd8da6c6003": { 2024-08-31T06:23:13.323 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:07:39.648942+0000", 2024-08-31T06:23:13.323 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:23:13.323 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:23:13.323 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:23:13.323 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:23:13.323 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:23:13.323 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:13.323 INFO:teuthology.orchestra.run.smithi016.stdout: "0fdcd762-533d-42be-9289-b71c8e3b81b1": { 2024-08-31T06:23:13.323 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:15:06.169112+0000", 2024-08-31T06:23:13.323 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:23:13.323 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:23:13.323 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:23:13.323 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:23:13.324 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:23:13.324 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:13.324 INFO:teuthology.orchestra.run.smithi016.stdout: "0015da36-6634-47c5-b8d4-219f9eba8250": { 2024-08-31T06:23:13.324 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:21:11.445380+0000", 2024-08-31T06:23:13.324 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:23:13.324 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:23:13.324 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:23:13.324 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:23:13.324 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:23:13.324 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:13.324 INFO:teuthology.orchestra.run.smithi016.stdout: "c8455d74-2fe7-4d93-bcf2-19b419d16016": { 2024-08-31T06:23:13.324 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:08:24.035960+0000", 2024-08-31T06:23:13.324 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:23:13.324 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:23:13.325 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:23:13.325 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:23:13.325 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:23:13.325 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:13.325 INFO:teuthology.orchestra.run.smithi016.stdout: "78d6ef02-8551-45e8-ad60-9627eb573fc1": { 2024-08-31T06:23:13.325 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:09:13.547865+0000", 2024-08-31T06:23:13.325 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:23:13.325 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:23:13.325 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:23:13.325 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:23:13.325 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:23:13.325 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:13.325 INFO:teuthology.orchestra.run.smithi016.stdout: "c1fd6904-2e02-47fa-939a-47349b9fe36b": { 2024-08-31T06:23:13.326 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:06.678691+0000", 2024-08-31T06:23:13.326 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:23:13.326 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:23:13.326 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:23:13.326 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:23:13.326 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:23:13.326 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:13.326 INFO:teuthology.orchestra.run.smithi016.stdout: "a10cce76-99ab-46b5-b3b2-07b82899b8f9": { 2024-08-31T06:23:13.326 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:11:43.967546+0000", 2024-08-31T06:23:13.326 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:23:13.326 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:23:13.326 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:23:13.326 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:23:13.327 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:23:13.327 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:13.327 INFO:teuthology.orchestra.run.smithi016.stdout: "91b72a34-4ba9-4126-a414-8b862f8c5f78": { 2024-08-31T06:23:13.327 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:13:28.663237+0000", 2024-08-31T06:23:13.327 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:23:13.327 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:23:13.327 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:23:13.327 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:23:13.327 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:23:13.327 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:13.327 INFO:teuthology.orchestra.run.smithi016.stdout: "e739a16e-8ace-43a8-a1ce-ac27d39ef8a3": { 2024-08-31T06:23:13.327 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:10:54.084646+0000", 2024-08-31T06:23:13.327 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:23:13.327 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:23:13.328 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:23:13.328 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:23:13.328 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:23:13.328 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:13.328 INFO:teuthology.orchestra.run.smithi016.stdout: "a64d5301-8558-4e9c-ac9b-8d0ac7057e74": { 2024-08-31T06:23:13.328 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:14:20.338104+0000", 2024-08-31T06:23:13.328 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:23:13.328 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:23:13.328 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:23:13.328 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:23:13.328 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:23:13.328 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:13.328 INFO:teuthology.orchestra.run.smithi016.stdout: "f9c4a22e-0c8a-4928-9b0d-e2ce1db5a86b": { 2024-08-31T06:23:13.328 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-08-31T06:15:57.496015+0000", 2024-08-31T06:23:13.329 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-08-31T06:23:13.329 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:23:13.329 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-08-31T06:23:13.329 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-08-31T06:23:13.329 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-08-31T06:23:13.329 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:13.329 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:13.329 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:23:13.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.312+0000 7f5065800640 1 -- 172.21.15.16:0/2436504190 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f50440b4d20 msgr2=0x7f50440b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:13.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.312+0000 7f5065800640 1 --2- 172.21.15.16:0/2436504190 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f50440b4d20 0x7f50440b7230 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f504802aa00 tx=0x7f5048005a20 comp rx=0 tx=0).stop 2024-08-31T06:23:13.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.312+0000 7f5065800640 1 -- 172.21.15.16:0/2436504190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f506010bdb0 msgr2=0x7f50601a1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:13.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.312+0000 7f5065800640 1 --2- 172.21.15.16:0/2436504190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f506010bdb0 0x7f50601a1c90 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f505800eb10 tx=0x7f505800c2d0 comp rx=0 tx=0).stop 2024-08-31T06:23:13.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.312+0000 7f5065800640 1 -- 172.21.15.16:0/2436504190 shutdown_connections 2024-08-31T06:23:13.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.312+0000 7f5065800640 1 --2- 172.21.15.16:0/2436504190 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f50440b4d20 0x7f50440b7230 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:13.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.312+0000 7f5065800640 1 --2- 172.21.15.16:0/2436504190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f506010bdb0 0x7f50601a1c90 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:13.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.312+0000 7f5065800640 1 --2- 172.21.15.16:0/2436504190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f506010b460 0x7f506019c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:13.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.312+0000 7f5065800640 1 --2- 172.21.15.16:0/2436504190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5060105b20 0x7f506019c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:13.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.312+0000 7f5065800640 1 -- 172.21.15.16:0/2436504190 >> 172.21.15.16:0/2436504190 conn(0x7f50601015b0 msgr2=0x7f5060108a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:13.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.312+0000 7f5065800640 1 -- 172.21.15.16:0/2436504190 shutdown_connections 2024-08-31T06:23:13.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:13.312+0000 7f5065800640 1 -- 172.21.15.16:0/2436504190 wait complete. 2024-08-31T06:23:13.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:12 smithi155 ceph-mon[21742]: pgmap v864: 97 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 92 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 65 KiB/s wr, 28 op/s 2024-08-31T06:23:14.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:13 smithi016 ceph-mon[17194]: pgmap v865: 97 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 92 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 64 KiB/s wr, 27 op/s 2024-08-31T06:23:14.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:13 smithi184 ceph-mon[22331]: pgmap v865: 97 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 92 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 64 KiB/s wr, 27 op/s 2024-08-31T06:23:14.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:13 smithi155 ceph-mon[21742]: pgmap v865: 97 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 92 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 64 KiB/s wr, 27 op/s 2024-08-31T06:23:14.467 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-31T06:23:14.467 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:23:14.847 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:23:15.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.511+0000 7f8010000640 1 -- 172.21.15.16:0/2266173698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8008104770 msgr2=0x7f8008104b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:15.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.511+0000 7f8010000640 1 --2- 172.21.15.16:0/2266173698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8008104770 0x7f8008104b80 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc009900 tx=0x7f7ffc02a410 comp rx=0 tx=0).stop 2024-08-31T06:23:15.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.512+0000 7f8010000640 1 -- 172.21.15.16:0/2266173698 shutdown_connections 2024-08-31T06:23:15.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.512+0000 7f8010000640 1 --2- 172.21.15.16:0/2266173698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80081050c0 0x7f800810d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:15.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.512+0000 7f8010000640 1 --2- 172.21.15.16:0/2266173698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f800810dbd0 0x7f80081100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:15.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.512+0000 7f8010000640 1 --2- 172.21.15.16:0/2266173698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8008104770 0x7f8008104b80 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:15.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.512+0000 7f8010000640 1 -- 172.21.15.16:0/2266173698 >> 172.21.15.16:0/2266173698 conn(0x7f80080794e0 msgr2=0x7f80080798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:15.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.512+0000 7f8010000640 1 -- 172.21.15.16:0/2266173698 shutdown_connections 2024-08-31T06:23:15.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.512+0000 7f8010000640 1 -- 172.21.15.16:0/2266173698 wait complete. 2024-08-31T06:23:15.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.513+0000 7f8010000640 1 Processor -- start 2024-08-31T06:23:15.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.513+0000 7f8010000640 1 -- start start 2024-08-31T06:23:15.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.513+0000 7f8010000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8008104770 0x7f800819c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:15.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.513+0000 7f8010000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80081050c0 0x7f800819ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:15.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.513+0000 7f8010000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f800810dbd0 0x7f80081a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:15.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.513+0000 7f8010000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8008113830 con 0x7f80081050c0 2024-08-31T06:23:15.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.513+0000 7f8010000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f80081136b0 con 0x7f800810dbd0 2024-08-31T06:23:15.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.513+0000 7f8010000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f80081139b0 con 0x7f8008104770 2024-08-31T06:23:15.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.514+0000 7f800cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80081050c0 0x7f800819ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:15.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.514+0000 7f800d600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8008104770 0x7f800819c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:15.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.514+0000 7f800cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80081050c0 0x7f800819ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34672/0 (socket says 172.21.15.16:34672) 2024-08-31T06:23:15.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.514+0000 7f800cc00640 1 -- 172.21.15.16:0/1624869745 learned_addr learned my addr 172.21.15.16:0/1624869745 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:23:15.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.514+0000 7f800e000640 1 --2- 172.21.15.16:0/1624869745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f800810dbd0 0x7f80081a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:15.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.514+0000 7f800d600640 1 -- 172.21.15.16:0/1624869745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f800810dbd0 msgr2=0x7f80081a1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:15.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.514+0000 7f800d600640 1 --2- 172.21.15.16:0/1624869745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f800810dbd0 0x7f80081a1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:15.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.514+0000 7f800d600640 1 -- 172.21.15.16:0/1624869745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80081050c0 msgr2=0x7f800819ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:15.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.514+0000 7f800d600640 1 --2- 172.21.15.16:0/1624869745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80081050c0 0x7f800819ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:15.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.514+0000 7f800d600640 1 -- 172.21.15.16:0/1624869745 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80081a23f0 con 0x7f8008104770 2024-08-31T06:23:15.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.515+0000 7f800d600640 1 --2- 172.21.15.16:0/1624869745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8008104770 0x7f800819c480 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc02a930 tx=0x7f7ffc031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:15.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.515+0000 7f8006000640 1 -- 172.21.15.16:0/1624869745 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc03f070 con 0x7f8008104770 2024-08-31T06:23:15.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.515+0000 7f8010000640 1 -- 172.21.15.16:0/1624869745 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80081a2680 con 0x7f8008104770 2024-08-31T06:23:15.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.515+0000 7f8006000640 1 -- 172.21.15.16:0/1624869745 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ffc004490 con 0x7f8008104770 2024-08-31T06:23:15.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.515+0000 7f8006000640 1 -- 172.21.15.16:0/1624869745 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc035070 con 0x7f8008104770 2024-08-31T06:23:15.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.515+0000 7f8010000640 1 -- 172.21.15.16:0/1624869745 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80081a8e40 con 0x7f8008104770 2024-08-31T06:23:15.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.516+0000 7f8010000640 1 -- 172.21.15.16:0/1624869745 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f800806bfe0 con 0x7f8008104770 2024-08-31T06:23:15.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.519+0000 7f8006000640 1 -- 172.21.15.16:0/1624869745 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f7ffc03b460 con 0x7f8008104770 2024-08-31T06:23:15.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.520+0000 7f8006000640 1 --2- 172.21.15.16:0/1624869745 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7ff00b4ca0 0x7f7ff00b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:15.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.521+0000 7f8006000640 1 -- 172.21.15.16:0/1624869745 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8514+0+0 (secure 0 0 0) 0x7f7ffc105ce0 con 0x7f8008104770 2024-08-31T06:23:15.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.521+0000 7f800cc00640 1 --2- 172.21.15.16:0/1624869745 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7ff00b4ca0 0x7f7ff00b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:15.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.522+0000 7f800cc00640 1 --2- 172.21.15.16:0/1624869745 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7ff00b4ca0 0x7f7ff00b71b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f80080fffa0 tx=0x7f7ff4005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:15.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.522+0000 7f8006000640 1 -- 172.21.15.16:0/1624869745 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f7ffc0cc420 con 0x7f8008104770 2024-08-31T06:23:15.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.757+0000 7f8010000640 1 -- 172.21.15.16:0/1624869745 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8008104b80 con 0x7f8008104770 2024-08-31T06:23:15.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.758+0000 7f8006000640 1 -- 172.21.15.16:0/1624869745 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11921 (secure 0 0 0) 0x7f7ffc0d0be0 con 0x7f8008104770 2024-08-31T06:23:15.761 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:23:15.761 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":55,"btime":"2024-08-31T06:22:47:897383+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":55,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:22:47.738871+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24737,24724,24712]},"id":1}]} 2024-08-31T06:23:15.761 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 55 2024-08-31T06:23:15.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.760+0000 7f8010000640 1 -- 172.21.15.16:0/1624869745 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7ff00b4ca0 msgr2=0x7f7ff00b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:15.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.760+0000 7f8010000640 1 --2- 172.21.15.16:0/1624869745 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7ff00b4ca0 0x7f7ff00b71b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f80080fffa0 tx=0x7f7ff4005f90 comp rx=0 tx=0).stop 2024-08-31T06:23:15.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.760+0000 7f8010000640 1 -- 172.21.15.16:0/1624869745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8008104770 msgr2=0x7f800819c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:15.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.760+0000 7f8010000640 1 --2- 172.21.15.16:0/1624869745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8008104770 0x7f800819c480 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc02a930 tx=0x7f7ffc031a20 comp rx=0 tx=0).stop 2024-08-31T06:23:15.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.760+0000 7f8010000640 1 -- 172.21.15.16:0/1624869745 shutdown_connections 2024-08-31T06:23:15.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.760+0000 7f8010000640 1 --2- 172.21.15.16:0/1624869745 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7ff00b4ca0 0x7f7ff00b71b0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:15.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.760+0000 7f8010000640 1 --2- 172.21.15.16:0/1624869745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f800810dbd0 0x7f80081a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:15.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.760+0000 7f8010000640 1 --2- 172.21.15.16:0/1624869745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80081050c0 0x7f800819ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:15.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.760+0000 7f8010000640 1 --2- 172.21.15.16:0/1624869745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8008104770 0x7f800819c480 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:15.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.761+0000 7f8010000640 1 -- 172.21.15.16:0/1624869745 >> 172.21.15.16:0/1624869745 conn(0x7f80080794e0 msgr2=0x7f80081003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:15.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.761+0000 7f8010000640 1 -- 172.21.15.16:0/1624869745 shutdown_connections 2024-08-31T06:23:15.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:15.761+0000 7f8010000640 1 -- 172.21.15.16:0/1624869745 wait complete. 2024-08-31T06:23:16.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:15 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/1624869745' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:23:16.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:15 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/1624869745' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:23:16.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:15 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/1624869745' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:23:16.571 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-31T06:23:16.955 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:23:17.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:16 smithi184 ceph-mon[22331]: pgmap v866: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-08-31T06:23:17.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:16 smithi184 ceph-mon[22331]: osdmap e103: 12 total, 12 up, 12 in 2024-08-31T06:23:17.276 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:16 smithi016 ceph-mon[17194]: pgmap v866: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-08-31T06:23:17.276 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:16 smithi016 ceph-mon[17194]: osdmap e103: 12 total, 12 up, 12 in 2024-08-31T06:23:17.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:16 smithi155 ceph-mon[21742]: pgmap v866: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-08-31T06:23:17.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:16 smithi155 ceph-mon[21742]: osdmap e103: 12 total, 12 up, 12 in 2024-08-31T06:23:17.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.615+0000 7f8557e00640 1 -- 172.21.15.16:0/3269870575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8558105950 msgr2=0x7f8558105d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:17.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.615+0000 7f8557e00640 1 --2- 172.21.15.16:0/3269870575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8558105950 0x7f8558105d60 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f8548009900 tx=0x7f854802a410 comp rx=0 tx=0).stop 2024-08-31T06:23:17.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.615+0000 7f8557e00640 1 -- 172.21.15.16:0/3269870575 shutdown_connections 2024-08-31T06:23:17.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.615+0000 7f8557e00640 1 --2- 172.21.15.16:0/3269870575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f855810bbe0 0x7f8558112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:17.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.615+0000 7f8557e00640 1 --2- 172.21.15.16:0/3269870575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f855810b290 0x7f855810b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:17.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.615+0000 7f8557e00640 1 --2- 172.21.15.16:0/3269870575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8558105950 0x7f8558105d60 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:17.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.615+0000 7f8557e00640 1 -- 172.21.15.16:0/3269870575 >> 172.21.15.16:0/3269870575 conn(0x7f8558101420 msgr2=0x7f8558103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:17.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.615+0000 7f8557e00640 1 -- 172.21.15.16:0/3269870575 shutdown_connections 2024-08-31T06:23:17.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.615+0000 7f8557e00640 1 -- 172.21.15.16:0/3269870575 wait complete. 2024-08-31T06:23:17.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.616+0000 7f8557e00640 1 Processor -- start 2024-08-31T06:23:17.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.617+0000 7f8557e00640 1 -- start start 2024-08-31T06:23:17.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.617+0000 7f8557e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8558105950 0x7f855819c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:17.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.617+0000 7f8557e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f855810b290 0x7f855819c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:17.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.617+0000 7f8557e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f855810bbe0 0x7f85581a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:17.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.617+0000 7f8557e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f85581158f0 con 0x7f855810b290 2024-08-31T06:23:17.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.617+0000 7f8557e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8558115770 con 0x7f855810bbe0 2024-08-31T06:23:17.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.617+0000 7f8557e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8558115a70 con 0x7f8558105950 2024-08-31T06:23:17.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.617+0000 7f8556a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8558105950 0x7f855819c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:17.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.617+0000 7f8556000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f855810b290 0x7f855819c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:17.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.617+0000 7f8556a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8558105950 0x7f855819c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:46126/0 (socket says 172.21.15.16:46126) 2024-08-31T06:23:17.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.617+0000 7f8556a00640 1 -- 172.21.15.16:0/4171709811 learned_addr learned my addr 172.21.15.16:0/4171709811 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:23:17.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.618+0000 7f8557400640 1 --2- 172.21.15.16:0/4171709811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f855810bbe0 0x7f85581a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:17.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.618+0000 7f8556a00640 1 -- 172.21.15.16:0/4171709811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f855810bbe0 msgr2=0x7f85581a1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:17.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.618+0000 7f8556a00640 1 --2- 172.21.15.16:0/4171709811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f855810bbe0 0x7f85581a1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:17.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.618+0000 7f8556a00640 1 -- 172.21.15.16:0/4171709811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f855810b290 msgr2=0x7f855819c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:17.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.618+0000 7f8556a00640 1 --2- 172.21.15.16:0/4171709811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f855810b290 0x7f855819c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:17.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.618+0000 7f8556a00640 1 -- 172.21.15.16:0/4171709811 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85581a2230 con 0x7f8558105950 2024-08-31T06:23:17.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.618+0000 7f8556a00640 1 --2- 172.21.15.16:0/4171709811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8558105950 0x7f855819c2c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f8548005ab0 tx=0x7f8548003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:17.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.619+0000 7f854f400640 1 -- 172.21.15.16:0/4171709811 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f854803f070 con 0x7f8558105950 2024-08-31T06:23:17.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.619+0000 7f8557e00640 1 -- 172.21.15.16:0/4171709811 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85581a24c0 con 0x7f8558105950 2024-08-31T06:23:17.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.619+0000 7f854f400640 1 -- 172.21.15.16:0/4171709811 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8548031b60 con 0x7f8558105950 2024-08-31T06:23:17.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.619+0000 7f854f400640 1 -- 172.21.15.16:0/4171709811 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8548035030 con 0x7f8558105950 2024-08-31T06:23:17.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.619+0000 7f8557e00640 1 -- 172.21.15.16:0/4171709811 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85581a8c80 con 0x7f8558105950 2024-08-31T06:23:17.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.622+0000 7f856482e740 1 Processor -- start 2024-08-31T06:23:17.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.622+0000 7f856482e740 1 -- start start 2024-08-31T06:23:17.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.623+0000 7f856482e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595bc716070 0x5595bcabba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:17.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.623+0000 7f856482e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595bc6ed1b0 0x5595bcabbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:17.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.623+0000 7f856482e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5595bc6f5390 0x5595bcabc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:17.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.623+0000 7f856482e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5595bca327b0 con 0x5595bc6ed1b0 2024-08-31T06:23:17.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.623+0000 7f856482e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5595bca21d40 con 0x5595bc716070 2024-08-31T06:23:17.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.623+0000 7f856482e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5595bca25cf0 con 0x5595bc6f5390 2024-08-31T06:23:17.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.623+0000 7f854f400640 1 -- 172.21.15.16:0/4171709811 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f8548042070 con 0x7f8558105950 2024-08-31T06:23:17.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.623+0000 7f8557400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595bc716070 0x5595bcabba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:17.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.623+0000 7f8557400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595bc716070 0x5595bcabba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:49556/0 (socket says 172.21.15.16:49556) 2024-08-31T06:23:17.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.623+0000 7f8557400640 1 -- 172.21.15.16:0/667285739 learned_addr learned my addr 172.21.15.16:0/667285739 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:23:17.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.623+0000 7f8556000640 1 --2- 172.21.15.16:0/667285739 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5595bc6f5390 0x5595bcabc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:17.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.623+0000 7f8556a00640 1 --2- 172.21.15.16:0/667285739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595bc6ed1b0 0x5595bcabbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:17.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.623+0000 7f8557400640 1 -- 172.21.15.16:0/667285739 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5595bc6f5390 msgr2=0x5595bcabc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:23:17.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.623+0000 7f8557400640 1 --2- 172.21.15.16:0/667285739 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5595bc6f5390 0x5595bcabc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:17.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.623+0000 7f8557400640 1 -- 172.21.15.16:0/667285739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595bc6ed1b0 msgr2=0x5595bcabbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:23:17.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.624+0000 7f8557400640 1 --2- 172.21.15.16:0/667285739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595bc6ed1b0 0x5595bcabbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:17.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.624+0000 7f8557400640 1 -- 172.21.15.16:0/667285739 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595bc781650 con 0x5595bc716070 2024-08-31T06:23:17.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.624+0000 7f8556a00640 1 --2- 172.21.15.16:0/667285739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595bc6ed1b0 0x5595bcabbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-31T06:23:17.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.624+0000 7f8557400640 1 --2- 172.21.15.16:0/667285739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595bc716070 0x5595bcabba70 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f8550011440 tx=0x7f8550011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a76466f6c3c29ff server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:17.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.624+0000 7f8543e00640 1 -- 172.21.15.16:0/667285739 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8550003c30 con 0x5595bc716070 2024-08-31T06:23:17.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.624+0000 7f8543e00640 1 -- 172.21.15.16:0/667285739 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8550003dd0 con 0x5595bc716070 2024-08-31T06:23:17.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.624+0000 7f8543e00640 1 -- 172.21.15.16:0/667285739 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855000fae0 con 0x5595bc716070 2024-08-31T06:23:17.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.625+0000 7f854f400640 1 --2- 172.21.15.16:0/4171709811 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f853c0b4ca0 0x7f853c0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:17.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.625+0000 7f856482e740 1 -- 172.21.15.16:0/667285739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595bc716070 msgr2=0x5595bcabba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:17.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.625+0000 7f856482e740 1 --2- 172.21.15.16:0/667285739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595bc716070 0x5595bcabba70 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f8550011440 tx=0x7f8550011900 comp rx=0 tx=0).stop 2024-08-31T06:23:17.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.625+0000 7f854f400640 1 -- 172.21.15.16:0/4171709811 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8470+0+0 (secure 0 0 0) 0x7f8548105bf0 con 0x7f8558105950 2024-08-31T06:23:17.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.625+0000 7f8556a00640 1 --2- 172.21.15.16:0/4171709811 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f853c0b4ca0 0x7f853c0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:17.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.625+0000 7f856482e740 1 -- 172.21.15.16:0/667285739 shutdown_connections 2024-08-31T06:23:17.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.625+0000 7f856482e740 1 --2- 172.21.15.16:0/667285739 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5595bc6f5390 0x5595bcabc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:17.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.625+0000 7f856482e740 1 --2- 172.21.15.16:0/667285739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595bc6ed1b0 0x5595bcabbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:17.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.625+0000 7f856482e740 1 --2- 172.21.15.16:0/667285739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595bc716070 0x5595bcabba70 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:17.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.625+0000 7f856482e740 1 -- 172.21.15.16:0/667285739 >> 172.21.15.16:0/667285739 conn(0x5595bc70a460 msgr2=0x5595bcab9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:17.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.626+0000 7f856482e740 1 -- 172.21.15.16:0/667285739 shutdown_connections 2024-08-31T06:23:17.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.626+0000 7f856482e740 1 -- 172.21.15.16:0/667285739 wait complete. 2024-08-31T06:23:17.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.626+0000 7f8556a00640 1 --2- 172.21.15.16:0/4171709811 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f853c0b4ca0 0x7f853c0b71b0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f8548103420 tx=0x7f8548009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:17.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.626+0000 7f856482e740 1 -- start start 2024-08-31T06:23:17.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.627+0000 7f856482e740 1 Processor -- start 2024-08-31T06:23:17.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.627+0000 7f856482e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5595bc6ed1b0 0x5595bcac7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:17.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.627+0000 7f856482e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595bc6f5390 0x5595bcac7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:17.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.627+0000 7f856482e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595bc716070 0x5595bcad3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:17.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.627+0000 7f856482e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5595bc884700 con 0x5595bc6f5390 2024-08-31T06:23:17.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.627+0000 7f856482e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5595bc9a8cf0 con 0x5595bc716070 2024-08-31T06:23:17.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.627+0000 7f856482e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5595bcaca250 con 0x5595bc6ed1b0 2024-08-31T06:23:17.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.627+0000 7f856482e740 10 client.0 _ll_register_callbacks cb 0x5595bc9ce470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:23:17.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.627+0000 7f8556000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595bc6f5390 0x5595bcac7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:17.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.628+0000 7f8556000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595bc6f5390 0x5595bcac7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47458/0 (socket says 172.21.15.16:47458) 2024-08-31T06:23:17.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.628+0000 7f8556000640 1 -- 172.21.15.16:0/1729999301 learned_addr learned my addr 172.21.15.16:0/1729999301 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:23:17.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.628+0000 7f8557400640 1 --2- 172.21.15.16:0/1729999301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5595bc6ed1b0 0x5595bcac7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:17.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.628+0000 7f8557400640 1 --2- 172.21.15.16:0/1729999301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595bc716070 0x5595bcad3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:17.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.628+0000 7f8556000640 1 -- 172.21.15.16:0/1729999301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5595bc6ed1b0 msgr2=0x5595bcac7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:23:17.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.628+0000 7f8556000640 1 --2- 172.21.15.16:0/1729999301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5595bc6ed1b0 0x5595bcac7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:17.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.628+0000 7f8556000640 1 -- 172.21.15.16:0/1729999301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595bc716070 msgr2=0x5595bcad3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:23:17.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.628+0000 7f8556000640 1 --2- 172.21.15.16:0/1729999301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595bc716070 0x5595bcad3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:17.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.628+0000 7f8556000640 1 -- 172.21.15.16:0/1729999301 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595bc9e98b0 con 0x5595bc6f5390 2024-08-31T06:23:17.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.628+0000 7f8556000640 1 --2- 172.21.15.16:0/1729999301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595bc6f5390 0x5595bcac7bf0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f8544012700 tx=0x7f8544012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bd75182daf54fbb server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:17.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.628+0000 7f8540c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:23:17.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.629+0000 7f8540c00640 1 -- 172.21.15.16:0/1729999301 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f854400f920 con 0x5595bc6f5390 2024-08-31T06:23:17.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.629+0000 7f8540c00640 1 -- 172.21.15.16:0/1729999301 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8470+0+0 (secure 0 0 0) 0x7f854400fac0 con 0x5595bc6f5390 2024-08-31T06:23:17.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.629+0000 7f856482e740 1 -- 172.21.15.16:0/1729999301 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8550003dd0 con 0x5595bc6f5390 2024-08-31T06:23:17.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.629+0000 7f8540c00640 1 -- 172.21.15.16:0/1729999301 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f854400fcc0 con 0x5595bc6f5390 2024-08-31T06:23:17.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.629+0000 7f856482e740 10 client.18603 fetch_fsmap learned FSMap version 55 2024-08-31T06:23:17.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.629+0000 7f856482e740 1 -- 172.21.15.16:0/1729999301 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5595bcad6280 con 0x5595bc6f5390 2024-08-31T06:23:17.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.629+0000 7f8540c00640 1 -- 172.21.15.16:0/1729999301 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8550003dd0 con 0x5595bc6f5390 2024-08-31T06:23:17.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.630+0000 7f8540c00640 1 -- 172.21.15.16:0/1729999301 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 55) ==== 8358+0+0 (secure 0 0 0) 0x7f854401c420 con 0x5595bc6f5390 2024-08-31T06:23:17.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.630+0000 7f856482e740 10 client.18603 fetch_fsmap finished waiting for FSMap version 55 2024-08-31T06:23:17.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.630+0000 7f856482e740 10 client.18603 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T06:23:17.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.630+0000 7f856482e740 20 client.18603 populate_metadata read hostname 'smithi016' 2024-08-31T06:23:17.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.630+0000 7f856482e740 1 --2- 172.21.15.16:0/1729999301 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x5595bc6d49f0 0x5595bcad9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:17.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.630+0000 7f856482e740 4 client.18603 mds_command: new command op to 24712 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:23:17.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.630+0000 7f856482e740 1 -- 172.21.15.16:0/1729999301 --> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595bca34700 con 0x5595bc6d49f0 2024-08-31T06:23:17.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.630+0000 7f8557400640 1 --2- 172.21.15.16:0/1729999301 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x5595bc6d49f0 0x5595bcad9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:17.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.631+0000 7f8557400640 1 --2- 172.21.15.16:0/1729999301 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x5595bc6d49f0 0x5595bcad9840 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=50e3d0d1f5b72c46 server_cookie=90c999e8040a951f in_seq=0 out_seq=0 2024-08-31T06:23:17.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.631+0000 7f8540c00640 10 client.18603 ms_handle_connect on v2:172.21.15.184:6838/4011051479 2024-08-31T06:23:17.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.632+0000 7f8540c00640 1 -- 172.21.15.16:0/1729999301 <== mds.2 v2:172.21.15.184:6838/4011051479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5595bca34700 con 0x5595bc6d49f0 2024-08-31T06:23:17.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.632+0000 7f8540c00640 10 client.18603 handle_command_reply: tid=0 2024-08-31T06:23:17.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.632+0000 7f856482e740 1 client.18603 shutdown 2024-08-31T06:23:17.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.632+0000 7f856482e740 1 -- 172.21.15.16:0/1729999301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595bc6f5390 msgr2=0x5595bcac7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:17.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.632+0000 7f856482e740 1 --2- 172.21.15.16:0/1729999301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595bc6f5390 0x5595bcac7bf0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f8544012700 tx=0x7f8544012bc0 comp rx=0 tx=0).stop 2024-08-31T06:23:17.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.632+0000 7f856482e740 1 -- 172.21.15.16:0/1729999301 shutdown_connections 2024-08-31T06:23:17.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.632+0000 7f856482e740 1 --2- 172.21.15.16:0/1729999301 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x5595bc6d49f0 0x5595bcad9840 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:17.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.632+0000 7f856482e740 1 --2- 172.21.15.16:0/1729999301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595bc716070 0x5595bcad3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:17.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.632+0000 7f856482e740 1 --2- 172.21.15.16:0/1729999301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595bc6f5390 0x5595bcac7bf0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:17.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.632+0000 7f856482e740 1 --2- 172.21.15.16:0/1729999301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5595bc6ed1b0 0x5595bcac7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:17.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.632+0000 7f856482e740 1 -- 172.21.15.16:0/1729999301 >> 172.21.15.16:0/1729999301 conn(0x5595bc70a460 msgr2=0x5595bcabec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:17.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.632+0000 7f8540c00640 0 client.18603 ms_handle_reset on v2:172.21.15.184:6838/4011051479 2024-08-31T06:23:17.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.632+0000 7f856482e740 1 -- 172.21.15.16:0/1729999301 shutdown_connections 2024-08-31T06:23:17.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.632+0000 7f856482e740 1 -- 172.21.15.16:0/1729999301 wait complete. 2024-08-31T06:23:17.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.633+0000 7f856482e740 20 client.18603 trim_cache size 0 max 16384 2024-08-31T06:23:17.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.647+0000 7f856482e740 1 Processor -- start 2024-08-31T06:23:17.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.647+0000 7f856482e740 1 -- start start 2024-08-31T06:23:17.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.647+0000 7f856482e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595bc6ed1b0 0x5595bcac1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:17.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.647+0000 7f856482e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5595bc6f5390 0x5595bcad3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:17.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.647+0000 7f856482e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595bc716070 0x5595bcab9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:17.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.647+0000 7f856482e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f85581b3dc0 con 0x5595bc716070 2024-08-31T06:23:17.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.647+0000 7f856482e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f85581b1660 con 0x5595bc6ed1b0 2024-08-31T06:23:17.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.647+0000 7f856482e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f85581af4d0 con 0x5595bc6f5390 2024-08-31T06:23:17.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.648+0000 7f8557400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595bc6ed1b0 0x5595bcac1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:17.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.648+0000 7f8557400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595bc6ed1b0 0x5595bcac1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:49572/0 (socket says 172.21.15.16:49572) 2024-08-31T06:23:17.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.648+0000 7f8557400640 1 -- 172.21.15.16:0/3116558537 learned_addr learned my addr 172.21.15.16:0/3116558537 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:23:17.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.648+0000 7f8556000640 1 --2- 172.21.15.16:0/3116558537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5595bc6f5390 0x5595bcad3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:17.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.648+0000 7f8557400640 1 --2- 172.21.15.16:0/3116558537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595bc716070 0x5595bcab9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:17.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.648+0000 7f8556000640 1 -- 172.21.15.16:0/3116558537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595bc6ed1b0 msgr2=0x5595bcac1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:23:17.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.648+0000 7f8556000640 1 --2- 172.21.15.16:0/3116558537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595bc6ed1b0 0x5595bcac1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:17.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.648+0000 7f8556000640 1 -- 172.21.15.16:0/3116558537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595bc716070 msgr2=0x5595bcab9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:23:17.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.648+0000 7f8556000640 1 --2- 172.21.15.16:0/3116558537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595bc716070 0x5595bcab9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:17.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.648+0000 7f8556000640 1 -- 172.21.15.16:0/3116558537 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595bcac8730 con 0x5595bc6f5390 2024-08-31T06:23:17.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.648+0000 7f8556000640 1 --2- 172.21.15.16:0/3116558537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5595bc6f5390 0x5595bcad3440 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f854401c860 tx=0x7f8544009e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e021b6570a10f230 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:17.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.649+0000 7f8543400640 1 -- 172.21.15.16:0/3116558537 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8544029030 con 0x5595bc6f5390 2024-08-31T06:23:17.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.649+0000 7f8543400640 1 -- 172.21.15.16:0/3116558537 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f854400f9a0 con 0x5595bc6f5390 2024-08-31T06:23:17.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.649+0000 7f8543400640 1 -- 172.21.15.16:0/3116558537 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8544005370 con 0x5595bc6f5390 2024-08-31T06:23:17.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.649+0000 7f856482e740 1 -- 172.21.15.16:0/3116558537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5595bc6f5390 msgr2=0x5595bcad3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:17.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.649+0000 7f856482e740 1 --2- 172.21.15.16:0/3116558537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5595bc6f5390 0x5595bcad3440 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f854401c860 tx=0x7f8544009e00 comp rx=0 tx=0).stop 2024-08-31T06:23:17.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.649+0000 7f856482e740 1 -- 172.21.15.16:0/3116558537 shutdown_connections 2024-08-31T06:23:17.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.649+0000 7f856482e740 1 --2- 172.21.15.16:0/3116558537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5595bc6f5390 0x5595bcad3440 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:17.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.649+0000 7f856482e740 1 --2- 172.21.15.16:0/3116558537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595bc716070 0x5595bcab9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:17.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.649+0000 7f856482e740 1 --2- 172.21.15.16:0/3116558537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595bc6ed1b0 0x5595bcac1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:17.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.649+0000 7f856482e740 1 -- 172.21.15.16:0/3116558537 >> 172.21.15.16:0/3116558537 conn(0x5595bc70a460 msgr2=0x5595bcac0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:17.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.650+0000 7f856482e740 1 -- 172.21.15.16:0/3116558537 shutdown_connections 2024-08-31T06:23:17.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.650+0000 7f856482e740 1 -- 172.21.15.16:0/3116558537 wait complete. 2024-08-31T06:23:17.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.650+0000 7f856482e740 1 -- start start 2024-08-31T06:23:17.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.650+0000 7f856482e740 1 Processor -- start 2024-08-31T06:23:17.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.651+0000 7f856482e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5595bc6ed1b0 0x5595bcacb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:17.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.651+0000 7f856482e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595bc6f5390 0x5595bcacbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:17.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.651+0000 7f856482e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595bc716070 0x5595bcacc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:17.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.651+0000 7f856482e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f85581afb20 con 0x5595bc716070 2024-08-31T06:23:17.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.651+0000 7f856482e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f85581b0f90 con 0x5595bc6f5390 2024-08-31T06:23:17.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.651+0000 7f856482e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f85581b01f0 con 0x5595bc6ed1b0 2024-08-31T06:23:17.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.651+0000 7f856482e740 10 client.0 _ll_register_callbacks cb 0x7f8544009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:23:17.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.651+0000 7f8557400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5595bc6ed1b0 0x5595bcacb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:17.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.651+0000 7f8557400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595bc716070 0x5595bcacc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:17.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.651+0000 7f8557400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595bc716070 0x5595bcacc0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47482/0 (socket says 172.21.15.16:47482) 2024-08-31T06:23:17.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.651+0000 7f8557400640 1 -- 172.21.15.16:0/2577687179 learned_addr learned my addr 172.21.15.16:0/2577687179 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:23:17.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.651+0000 7f8556000640 1 --2- 172.21.15.16:0/2577687179 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595bc6f5390 0x5595bcacbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:17.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.652+0000 7f8557400640 1 -- 172.21.15.16:0/2577687179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5595bc6ed1b0 msgr2=0x5595bcacb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:23:17.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.652+0000 7f8557400640 1 --2- 172.21.15.16:0/2577687179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5595bc6ed1b0 0x5595bcacb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:17.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.652+0000 7f8557400640 1 -- 172.21.15.16:0/2577687179 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595bc6f5390 msgr2=0x5595bcacbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:23:17.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.652+0000 7f8557400640 1 --2- 172.21.15.16:0/2577687179 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595bc6f5390 0x5595bcacbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:17.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.652+0000 7f8557400640 1 -- 172.21.15.16:0/2577687179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595bcb43420 con 0x5595bc716070 2024-08-31T06:23:17.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.652+0000 7f8556000640 1 --2- 172.21.15.16:0/2577687179 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595bc6f5390 0x5595bcacbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:23:17.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.652+0000 7f8557400640 1 --2- 172.21.15.16:0/2577687179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595bc716070 0x5595bcacc0f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x5595bcaaaed0 tx=0x7f8550011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=186100b6af68348f server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:17.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.652+0000 7f8540c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-31T06:23:17.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.652+0000 7f8540c00640 1 -- 172.21.15.16:0/2577687179 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855000fc50 con 0x5595bc716070 2024-08-31T06:23:17.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.652+0000 7f8540c00640 1 -- 172.21.15.16:0/2577687179 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8470+0+0 (secure 0 0 0) 0x7f855000fdf0 con 0x5595bc716070 2024-08-31T06:23:17.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.652+0000 7f856482e740 1 -- 172.21.15.16:0/2577687179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f854400f9a0 con 0x5595bc716070 2024-08-31T06:23:17.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.653+0000 7f8540c00640 1 -- 172.21.15.16:0/2577687179 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f85500050c0 con 0x5595bc716070 2024-08-31T06:23:17.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.653+0000 7f856482e740 10 client.18609 fetch_fsmap learned FSMap version 55 2024-08-31T06:23:17.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.653+0000 7f856482e740 1 -- 172.21.15.16:0/2577687179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5595bcb446a0 con 0x5595bc716070 2024-08-31T06:23:17.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.653+0000 7f8540c00640 1 -- 172.21.15.16:0/2577687179 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f854400f9a0 con 0x5595bc716070 2024-08-31T06:23:17.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.653+0000 7f8540c00640 1 -- 172.21.15.16:0/2577687179 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 55) ==== 8358+0+0 (secure 0 0 0) 0x7f8550045720 con 0x5595bc716070 2024-08-31T06:23:17.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.653+0000 7f856482e740 10 client.18609 fetch_fsmap finished waiting for FSMap version 55 2024-08-31T06:23:17.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.653+0000 7f856482e740 10 client.18609 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-31T06:23:17.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.653+0000 7f856482e740 20 client.18609 populate_metadata read hostname 'smithi016' 2024-08-31T06:23:17.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.653+0000 7f856482e740 1 --2- 172.21.15.16:0/2577687179 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x5595bc6a9da0 0x5595bcb46c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:17.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.653+0000 7f856482e740 4 client.18609 mds_command: new command op to 24712 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:23:17.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.653+0000 7f856482e740 1 -- 172.21.15.16:0/2577687179 --> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5595bca301a0 con 0x5595bc6a9da0 2024-08-31T06:23:17.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.654+0000 7f8556000640 1 --2- 172.21.15.16:0/2577687179 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x5595bc6a9da0 0x5595bcb46c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:17.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.654+0000 7f8556000640 1 --2- 172.21.15.16:0/2577687179 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x5595bc6a9da0 0x5595bcb46c30 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5cc2649ac9adcb8d server_cookie=f30d29c7fe333742 in_seq=0 out_seq=0 2024-08-31T06:23:17.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.654+0000 7f8540c00640 10 client.18609 ms_handle_connect on v2:172.21.15.184:6838/4011051479 2024-08-31T06:23:17.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.655+0000 7f8540c00640 1 -- 172.21.15.16:0/2577687179 <== mds.2 v2:172.21.15.184:6838/4011051479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5595bca301a0 con 0x5595bc6a9da0 2024-08-31T06:23:17.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.655+0000 7f8540c00640 10 client.18609 handle_command_reply: tid=0 2024-08-31T06:23:17.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.655+0000 7f856482e740 1 client.18609 shutdown 2024-08-31T06:23:17.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.655+0000 7f856482e740 1 -- 172.21.15.16:0/2577687179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595bc716070 msgr2=0x5595bcacc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:17.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.655+0000 7f856482e740 1 --2- 172.21.15.16:0/2577687179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595bc716070 0x5595bcacc0f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x5595bcaaaed0 tx=0x7f8550011900 comp rx=0 tx=0).stop 2024-08-31T06:23:17.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.655+0000 7f856482e740 1 -- 172.21.15.16:0/2577687179 shutdown_connections 2024-08-31T06:23:17.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.655+0000 7f856482e740 1 --2- 172.21.15.16:0/2577687179 >> [v2:172.21.15.184:6838/4011051479,v1:172.21.15.184:6839/4011051479] conn(0x5595bc6a9da0 0x5595bcb46c30 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:17.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.655+0000 7f856482e740 1 --2- 172.21.15.16:0/2577687179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595bc716070 0x5595bcacc0f0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:17.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.655+0000 7f856482e740 1 --2- 172.21.15.16:0/2577687179 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595bc6f5390 0x5595bcacbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:17.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.655+0000 7f856482e740 1 --2- 172.21.15.16:0/2577687179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5595bc6ed1b0 0x5595bcacb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:17.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.655+0000 7f856482e740 1 -- 172.21.15.16:0/2577687179 >> 172.21.15.16:0/2577687179 conn(0x5595bc70a460 msgr2=0x5595bcab9260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:17.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.655+0000 7f8540c00640 0 client.18609 ms_handle_reset on v2:172.21.15.184:6838/4011051479 2024-08-31T06:23:17.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.655+0000 7f856482e740 1 -- 172.21.15.16:0/2577687179 shutdown_connections 2024-08-31T06:23:17.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.655+0000 7f856482e740 1 -- 172.21.15.16:0/2577687179 wait complete. 2024-08-31T06:23:17.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.656+0000 7f856482e740 20 client.18609 trim_cache size 0 max 16384 2024-08-31T06:23:17.660 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:23:17.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.656+0000 7f8557e00640 1 -- 172.21.15.16:0/4171709811 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f853c0b4ca0 msgr2=0x7f853c0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:17.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.656+0000 7f8557e00640 1 --2- 172.21.15.16:0/4171709811 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f853c0b4ca0 0x7f853c0b71b0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f8548103420 tx=0x7f8548009260 comp rx=0 tx=0).stop 2024-08-31T06:23:17.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.656+0000 7f8557e00640 1 -- 172.21.15.16:0/4171709811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8558105950 msgr2=0x7f855819c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:17.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.656+0000 7f8557e00640 1 --2- 172.21.15.16:0/4171709811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8558105950 0x7f855819c2c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f8548005ab0 tx=0x7f8548003c10 comp rx=0 tx=0).stop 2024-08-31T06:23:17.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.656+0000 7f8557e00640 1 -- 172.21.15.16:0/4171709811 shutdown_connections 2024-08-31T06:23:17.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.656+0000 7f8557e00640 1 --2- 172.21.15.16:0/4171709811 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f853c0b4ca0 0x7f853c0b71b0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:17.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.656+0000 7f8557e00640 1 --2- 172.21.15.16:0/4171709811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f855810bbe0 0x7f85581a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:17.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.656+0000 7f8557e00640 1 --2- 172.21.15.16:0/4171709811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f855810b290 0x7f855819c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:17.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.656+0000 7f8557e00640 1 --2- 172.21.15.16:0/4171709811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8558105950 0x7f855819c2c0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:17.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.656+0000 7f8557e00640 1 -- 172.21.15.16:0/4171709811 >> 172.21.15.16:0/4171709811 conn(0x7f8558101420 msgr2=0x7f8558103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:17.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.657+0000 7f8557e00640 1 -- 172.21.15.16:0/4171709811 shutdown_connections 2024-08-31T06:23:17.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:17.657+0000 7f8557e00640 1 -- 172.21.15.16:0/4171709811 wait complete. 2024-08-31T06:23:18.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:18 smithi016 ceph-mon[17194]: pgmap v868: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.9 KiB/s wr, 8 op/s 2024-08-31T06:23:18.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:18 smithi184 ceph-mon[22331]: pgmap v868: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.9 KiB/s wr, 8 op/s 2024-08-31T06:23:18.763 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-31T06:23:18.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:18 smithi155 ceph-mon[21742]: pgmap v868: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.9 KiB/s wr, 8 op/s 2024-08-31T06:23:19.146 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:23:19.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.759+0000 7ff890200640 1 -- 172.21.15.16:0/2465248034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8881144e0 msgr2=0x7ff888116900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:19.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.759+0000 7ff890200640 1 --2- 172.21.15.16:0/2465248034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8881144e0 0x7ff888116900 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7ff880009ca0 tx=0x7ff88002a8c0 comp rx=0 tx=0).stop 2024-08-31T06:23:19.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.759+0000 7ff890200640 1 -- 172.21.15.16:0/2465248034 shutdown_connections 2024-08-31T06:23:19.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.759+0000 7ff890200640 1 --2- 172.21.15.16:0/2465248034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8881144e0 0x7ff888116900 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:19.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.759+0000 7ff890200640 1 --2- 172.21.15.16:0/2465248034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff888078030 0x7ff888113fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:19.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.759+0000 7ff890200640 1 --2- 172.21.15.16:0/2465248034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8880776e0 0x7ff888077af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:19.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.759+0000 7ff890200640 1 -- 172.21.15.16:0/2465248034 >> 172.21.15.16:0/2465248034 conn(0x7ff8881015b0 msgr2=0x7ff888103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:19.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.759+0000 7ff890200640 1 -- 172.21.15.16:0/2465248034 shutdown_connections 2024-08-31T06:23:19.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.759+0000 7ff890200640 1 -- 172.21.15.16:0/2465248034 wait complete. 2024-08-31T06:23:19.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.760+0000 7ff890200640 1 Processor -- start 2024-08-31T06:23:19.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.760+0000 7ff890200640 1 -- start start 2024-08-31T06:23:19.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.761+0000 7ff890200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8880776e0 0x7ff8881a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:19.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.761+0000 7ff890200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff888078030 0x7ff8881a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:19.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.761+0000 7ff890200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8881144e0 0x7ff8881a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:19.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.761+0000 7ff890200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff888119eb0 con 0x7ff8881144e0 2024-08-31T06:23:19.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.761+0000 7ff890200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff888119d30 con 0x7ff8880776e0 2024-08-31T06:23:19.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.761+0000 7ff890200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff88811a030 con 0x7ff888078030 2024-08-31T06:23:19.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.761+0000 7ff88d800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8880776e0 0x7ff8881a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:19.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.761+0000 7ff88d800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8880776e0 0x7ff8881a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:49602/0 (socket says 172.21.15.16:49602) 2024-08-31T06:23:19.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.761+0000 7ff88d800640 1 -- 172.21.15.16:0/1095284495 learned_addr learned my addr 172.21.15.16:0/1095284495 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:23:19.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.761+0000 7ff88e200640 1 --2- 172.21.15.16:0/1095284495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8881144e0 0x7ff8881a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:19.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.761+0000 7ff88ce00640 1 --2- 172.21.15.16:0/1095284495 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff888078030 0x7ff8881a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:19.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.761+0000 7ff88d800640 1 -- 172.21.15.16:0/1095284495 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff888078030 msgr2=0x7ff8881a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:19.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.761+0000 7ff88d800640 1 --2- 172.21.15.16:0/1095284495 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff888078030 0x7ff8881a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:19.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.761+0000 7ff88d800640 1 -- 172.21.15.16:0/1095284495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8881144e0 msgr2=0x7ff8881a6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:19.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.761+0000 7ff88d800640 1 --2- 172.21.15.16:0/1095284495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8881144e0 0x7ff8881a6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:19.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.762+0000 7ff88d800640 1 -- 172.21.15.16:0/1095284495 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8881a6790 con 0x7ff8880776e0 2024-08-31T06:23:19.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.762+0000 7ff88d800640 1 --2- 172.21.15.16:0/1095284495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8880776e0 0x7ff8881a0820 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ff87c00e9f0 tx=0x7ff87c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:19.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.762+0000 7ff886000640 1 -- 172.21.15.16:0/1095284495 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87c00cd80 con 0x7ff8880776e0 2024-08-31T06:23:19.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.762+0000 7ff890200640 1 -- 172.21.15.16:0/1095284495 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8881a6a80 con 0x7ff8880776e0 2024-08-31T06:23:19.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.762+0000 7ff886000640 1 -- 172.21.15.16:0/1095284495 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff87c0054d0 con 0x7ff8880776e0 2024-08-31T06:23:19.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.762+0000 7ff886000640 1 -- 172.21.15.16:0/1095284495 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87c010690 con 0x7ff8880776e0 2024-08-31T06:23:19.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.762+0000 7ff890200640 1 -- 172.21.15.16:0/1095284495 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8881ad1e0 con 0x7ff8880776e0 2024-08-31T06:23:19.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.765+0000 7ff896e06740 1 Processor -- start 2024-08-31T06:23:19.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.765+0000 7ff896e06740 1 -- start start 2024-08-31T06:23:19.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.766+0000 7ff896e06740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561051668070 0x561051a0d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:19.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.766+0000 7ff896e06740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56105163f1b0 0x561051a0ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:19.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.766+0000 7ff896e06740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561051647390 0x561051a0e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:19.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.766+0000 7ff896e06740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5610519847b0 con 0x56105163f1b0 2024-08-31T06:23:19.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.766+0000 7ff896e06740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561051973d40 con 0x561051647390 2024-08-31T06:23:19.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.766+0000 7ff896e06740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x561051977cf0 con 0x561051668070 2024-08-31T06:23:19.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.766+0000 7ff88e200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561051668070 0x561051a0d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:19.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.766+0000 7ff88ce00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561051647390 0x561051a0e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:19.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.766+0000 7ff88e200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561051668070 0x561051a0d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:46188/0 (socket says 172.21.15.16:46188) 2024-08-31T06:23:19.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.766+0000 7ff88e200640 1 -- 172.21.15.16:0/3030913879 learned_addr learned my addr 172.21.15.16:0/3030913879 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:23:19.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.767+0000 7ff886000640 1 -- 172.21.15.16:0/1095284495 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7ff87c004e90 con 0x7ff8880776e0 2024-08-31T06:23:19.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.767+0000 7ff88ce00640 1 -- 172.21.15.16:0/3030913879 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561051668070 msgr2=0x561051a0d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:23:19.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.767+0000 7ff88ce00640 1 --2- 172.21.15.16:0/3030913879 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561051668070 0x561051a0d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:19.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.767+0000 7ff88ce00640 1 -- 172.21.15.16:0/3030913879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56105163f1b0 msgr2=0x561051a0ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:23:19.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.767+0000 7ff88ce00640 1 --2- 172.21.15.16:0/3030913879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56105163f1b0 0x561051a0ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:19.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.767+0000 7ff88ce00640 1 -- 172.21.15.16:0/3030913879 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610516d3650 con 0x561051647390 2024-08-31T06:23:19.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.767+0000 7ff88e200640 1 --2- 172.21.15.16:0/3030913879 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561051668070 0x561051a0d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:23:19.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.767+0000 7ff88ce00640 1 --2- 172.21.15.16:0/3030913879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561051647390 0x561051a0e410 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ff88806c5b0 tx=0x7ff8740113e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83ed52e61407fb5a server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:19.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.768+0000 7ff87aa00640 1 -- 172.21.15.16:0/3030913879 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff874011de0 con 0x561051647390 2024-08-31T06:23:19.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.768+0000 7ff87aa00640 1 -- 172.21.15.16:0/3030913879 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff874012900 con 0x561051647390 2024-08-31T06:23:19.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.768+0000 7ff87aa00640 1 -- 172.21.15.16:0/3030913879 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87400f960 con 0x561051647390 2024-08-31T06:23:19.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.768+0000 7ff886000640 1 --2- 172.21.15.16:0/1095284495 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff8700b4ca0 0x7ff8700b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:19.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.768+0000 7ff886000640 1 -- 172.21.15.16:0/1095284495 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8470+0+0 (secure 0 0 0) 0x7ff87c0e5a70 con 0x7ff8880776e0 2024-08-31T06:23:19.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.769+0000 7ff896e06740 1 -- 172.21.15.16:0/3030913879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561051647390 msgr2=0x561051a0e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:19.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.769+0000 7ff896e06740 1 --2- 172.21.15.16:0/3030913879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561051647390 0x561051a0e410 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ff88806c5b0 tx=0x7ff8740113e0 comp rx=0 tx=0).stop 2024-08-31T06:23:19.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.769+0000 7ff88e200640 1 --2- 172.21.15.16:0/1095284495 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff8700b4ca0 0x7ff8700b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:19.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.769+0000 7ff896e06740 1 -- 172.21.15.16:0/3030913879 shutdown_connections 2024-08-31T06:23:19.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.769+0000 7ff896e06740 1 --2- 172.21.15.16:0/3030913879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561051647390 0x561051a0e410 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:19.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.769+0000 7ff896e06740 1 --2- 172.21.15.16:0/3030913879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56105163f1b0 0x561051a0ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:19.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.769+0000 7ff896e06740 1 --2- 172.21.15.16:0/3030913879 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561051668070 0x561051a0d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:19.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.769+0000 7ff896e06740 1 -- 172.21.15.16:0/3030913879 >> 172.21.15.16:0/3030913879 conn(0x56105165c460 msgr2=0x561051a0b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:19.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.769+0000 7ff896e06740 1 -- 172.21.15.16:0/3030913879 shutdown_connections 2024-08-31T06:23:19.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.769+0000 7ff896e06740 1 -- 172.21.15.16:0/3030913879 wait complete. 2024-08-31T06:23:19.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.769+0000 7ff88e200640 1 --2- 172.21.15.16:0/1095284495 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff8700b4ca0 0x7ff8700b71b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7ff8800094e0 tx=0x7ff880005d80 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:19.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.770+0000 7ff896e06740 1 -- start start 2024-08-31T06:23:19.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.770+0000 7ff896e06740 1 Processor -- start 2024-08-31T06:23:19.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.771+0000 7ff896e06740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56105163f1b0 0x561051a19510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:19.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.771+0000 7ff896e06740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561051647390 0x561051a19a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:19.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.771+0000 7ff896e06740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561051668070 0x561051a25460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:19.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.771+0000 7ff896e06740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5610517d6700 con 0x561051668070 2024-08-31T06:23:19.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.771+0000 7ff896e06740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5610518facf0 con 0x561051647390 2024-08-31T06:23:19.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.771+0000 7ff896e06740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x561051a1c170 con 0x56105163f1b0 2024-08-31T06:23:19.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.771+0000 7ff896e06740 10 client.0 _ll_register_callbacks cb 0x561051920470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:23:19.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.771+0000 7ff88d800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56105163f1b0 0x561051a19510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:19.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.771+0000 7ff88d800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56105163f1b0 0x561051a19510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:46190/0 (socket says 172.21.15.16:46190) 2024-08-31T06:23:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.771+0000 7ff88d800640 1 -- 172.21.15.16:0/1314346738 learned_addr learned my addr 172.21.15.16:0/1314346738 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:23:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.771+0000 7ff88e200640 1 --2- 172.21.15.16:0/1314346738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561051668070 0x561051a25460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.771+0000 7ff88ce00640 1 --2- 172.21.15.16:0/1314346738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561051647390 0x561051a19a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.771+0000 7ff88d800640 1 -- 172.21.15.16:0/1314346738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561051647390 msgr2=0x561051a19a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:23:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.771+0000 7ff88d800640 1 --2- 172.21.15.16:0/1314346738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561051647390 0x561051a19a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.771+0000 7ff88d800640 1 -- 172.21.15.16:0/1314346738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561051668070 msgr2=0x561051a25460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:23:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.771+0000 7ff88d800640 1 --2- 172.21.15.16:0/1314346738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561051668070 0x561051a25460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.771+0000 7ff88d800640 1 -- 172.21.15.16:0/1314346738 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56105193b8b0 con 0x56105163f1b0 2024-08-31T06:23:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.772+0000 7ff88d800640 1 --2- 172.21.15.16:0/1314346738 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56105163f1b0 0x561051a19510 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ff87c00e9c0 tx=0x7ff87c0ab960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d5928e71bb5afc5 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.772+0000 7ff86f400640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:23:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.772+0000 7ff86f400640 1 -- 172.21.15.16:0/1314346738 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87c0e5e60 con 0x56105163f1b0 2024-08-31T06:23:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.772+0000 7ff896e06740 1 -- 172.21.15.16:0/1314346738 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff874012900 con 0x56105163f1b0 2024-08-31T06:23:19.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.772+0000 7ff86f400640 1 -- 172.21.15.16:0/1314346738 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8470+0+0 (secure 0 0 0) 0x7ff87c0b4fc0 con 0x56105163f1b0 2024-08-31T06:23:19.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.772+0000 7ff86f400640 1 -- 172.21.15.16:0/1314346738 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7ff87c0e3620 con 0x56105163f1b0 2024-08-31T06:23:19.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.772+0000 7ff896e06740 10 client.29146 fetch_fsmap learned FSMap version 55 2024-08-31T06:23:19.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.772+0000 7ff896e06740 1 -- 172.21.15.16:0/1314346738 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x561051a281a0 con 0x56105163f1b0 2024-08-31T06:23:19.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.773+0000 7ff86f400640 1 -- 172.21.15.16:0/1314346738 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff874012900 con 0x56105163f1b0 2024-08-31T06:23:19.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.773+0000 7ff86f400640 1 -- 172.21.15.16:0/1314346738 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 55) ==== 8358+0+0 (secure 0 0 0) 0x7ff87c0a7aa0 con 0x56105163f1b0 2024-08-31T06:23:19.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.773+0000 7ff896e06740 10 client.29146 fetch_fsmap finished waiting for FSMap version 55 2024-08-31T06:23:19.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.773+0000 7ff896e06740 10 client.29146 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-31T06:23:19.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.773+0000 7ff896e06740 20 client.29146 populate_metadata read hostname 'smithi016' 2024-08-31T06:23:19.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.773+0000 7ff896e06740 1 --2- 172.21.15.16:0/1314346738 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x5610516269f0 0x561051a2b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:19.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.773+0000 7ff896e06740 4 client.29146 mds_command: new command op to 24724 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:23:19.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.773+0000 7ff896e06740 1 -- 172.21.15.16:0/1314346738 --> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561051986700 con 0x5610516269f0 2024-08-31T06:23:19.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.774+0000 7ff88ce00640 1 --2- 172.21.15.16:0/1314346738 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x5610516269f0 0x561051a2b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:19.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.774+0000 7ff88ce00640 1 --2- 172.21.15.16:0/1314346738 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x5610516269f0 0x561051a2b6b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=425f982b1a81481c server_cookie=e3510adfae013e69 in_seq=0 out_seq=0 2024-08-31T06:23:19.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.774+0000 7ff86f400640 10 client.29146 ms_handle_connect on v2:172.21.15.16:6838/219664460 2024-08-31T06:23:19.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.775+0000 7ff86f400640 1 -- 172.21.15.16:0/1314346738 <== mds.1 v2:172.21.15.16:6838/219664460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561051986700 con 0x5610516269f0 2024-08-31T06:23:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.775+0000 7ff86f400640 10 client.29146 handle_command_reply: tid=0 2024-08-31T06:23:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.775+0000 7ff896e06740 1 client.29146 shutdown 2024-08-31T06:23:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.776+0000 7ff896e06740 1 -- 172.21.15.16:0/1314346738 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56105163f1b0 msgr2=0x561051a19510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.776+0000 7ff896e06740 1 --2- 172.21.15.16:0/1314346738 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56105163f1b0 0x561051a19510 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ff87c00e9c0 tx=0x7ff87c0ab960 comp rx=0 tx=0).stop 2024-08-31T06:23:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.776+0000 7ff896e06740 1 -- 172.21.15.16:0/1314346738 shutdown_connections 2024-08-31T06:23:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.776+0000 7ff896e06740 1 --2- 172.21.15.16:0/1314346738 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x5610516269f0 0x561051a2b6b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.776+0000 7ff896e06740 1 --2- 172.21.15.16:0/1314346738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561051668070 0x561051a25460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.776+0000 7ff896e06740 1 --2- 172.21.15.16:0/1314346738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561051647390 0x561051a19a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.776+0000 7ff896e06740 1 --2- 172.21.15.16:0/1314346738 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56105163f1b0 0x561051a19510 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.776+0000 7ff86f400640 0 client.29146 ms_handle_reset on v2:172.21.15.16:6838/219664460 2024-08-31T06:23:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.776+0000 7ff896e06740 1 -- 172.21.15.16:0/1314346738 >> 172.21.15.16:0/1314346738 conn(0x56105165c460 msgr2=0x561051a10b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.776+0000 7ff896e06740 1 -- 172.21.15.16:0/1314346738 shutdown_connections 2024-08-31T06:23:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.776+0000 7ff896e06740 1 -- 172.21.15.16:0/1314346738 wait complete. 2024-08-31T06:23:19.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.776+0000 7ff896e06740 20 client.29146 trim_cache size 0 max 16384 2024-08-31T06:23:19.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.790+0000 7ff896e06740 1 Processor -- start 2024-08-31T06:23:19.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.790+0000 7ff896e06740 1 -- start start 2024-08-31T06:23:19.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.791+0000 7ff896e06740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56105163f1b0 0x561051a25360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:19.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.791+0000 7ff896e06740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561051647390 0x561051a258a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:19.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.791+0000 7ff896e06740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561051668070 0x561051a16500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:19.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.791+0000 7ff896e06740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff8881b8320 con 0x561051668070 2024-08-31T06:23:19.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.791+0000 7ff896e06740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff8881b5bc0 con 0x56105163f1b0 2024-08-31T06:23:19.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.791+0000 7ff896e06740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff8881b3a30 con 0x561051647390 2024-08-31T06:23:19.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.791+0000 7ff88e200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561051668070 0x561051a16500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:19.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.791+0000 7ff88ce00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561051647390 0x561051a258a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:19.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.791+0000 7ff88e200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561051668070 0x561051a16500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47520/0 (socket says 172.21.15.16:47520) 2024-08-31T06:23:19.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.791+0000 7ff88ce00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561051647390 0x561051a258a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:46196/0 (socket says 172.21.15.16:46196) 2024-08-31T06:23:19.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.791+0000 7ff88e200640 1 -- 172.21.15.16:0/1903588997 learned_addr learned my addr 172.21.15.16:0/1903588997 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:23:19.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.791+0000 7ff88d800640 1 --2- 172.21.15.16:0/1903588997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56105163f1b0 0x561051a25360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:19.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.792+0000 7ff88ce00640 1 -- 172.21.15.16:0/1903588997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56105163f1b0 msgr2=0x561051a25360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:23:19.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.792+0000 7ff88ce00640 1 --2- 172.21.15.16:0/1903588997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56105163f1b0 0x561051a25360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:19.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.792+0000 7ff88ce00640 1 -- 172.21.15.16:0/1903588997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561051668070 msgr2=0x561051a16500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:23:19.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.792+0000 7ff88ce00640 1 --2- 172.21.15.16:0/1903588997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561051668070 0x561051a16500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:19.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.792+0000 7ff88ce00640 1 -- 172.21.15.16:0/1903588997 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561051a1a5d0 con 0x561051647390 2024-08-31T06:23:19.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.792+0000 7ff88ce00640 1 --2- 172.21.15.16:0/1903588997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561051647390 0x561051a258a0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff87400f520 tx=0x7ff87400f580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddbac5eeab4c054 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:19.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.792+0000 7ff87a000640 1 -- 172.21.15.16:0/1903588997 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff874005240 con 0x561051647390 2024-08-31T06:23:19.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.792+0000 7ff87a000640 1 -- 172.21.15.16:0/1903588997 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff874009770 con 0x561051647390 2024-08-31T06:23:19.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.792+0000 7ff87a000640 1 -- 172.21.15.16:0/1903588997 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff874003d40 con 0x561051647390 2024-08-31T06:23:19.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.793+0000 7ff896e06740 1 -- 172.21.15.16:0/1903588997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561051647390 msgr2=0x561051a258a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:19.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.793+0000 7ff896e06740 1 --2- 172.21.15.16:0/1903588997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561051647390 0x561051a258a0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff87400f520 tx=0x7ff87400f580 comp rx=0 tx=0).stop 2024-08-31T06:23:19.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.793+0000 7ff896e06740 1 -- 172.21.15.16:0/1903588997 shutdown_connections 2024-08-31T06:23:19.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.793+0000 7ff896e06740 1 --2- 172.21.15.16:0/1903588997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561051647390 0x561051a258a0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:19.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.793+0000 7ff896e06740 1 --2- 172.21.15.16:0/1903588997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561051668070 0x561051a16500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:19.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.793+0000 7ff896e06740 1 --2- 172.21.15.16:0/1903588997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56105163f1b0 0x561051a25360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:19.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.793+0000 7ff896e06740 1 -- 172.21.15.16:0/1903588997 >> 172.21.15.16:0/1903588997 conn(0x56105165c460 msgr2=0x561051a12aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:19.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.793+0000 7ff896e06740 1 -- 172.21.15.16:0/1903588997 shutdown_connections 2024-08-31T06:23:19.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.793+0000 7ff896e06740 1 -- 172.21.15.16:0/1903588997 wait complete. 2024-08-31T06:23:19.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.794+0000 7ff896e06740 1 -- start start 2024-08-31T06:23:19.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.794+0000 7ff896e06740 1 Processor -- start 2024-08-31T06:23:19.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.795+0000 7ff896e06740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56105163f1b0 0x561051a1d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:19.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.795+0000 7ff896e06740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561051647390 0x561051a1dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:19.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.795+0000 7ff896e06740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561051668070 0x561051a1e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:19.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.795+0000 7ff896e06740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff8881b4080 con 0x561051668070 2024-08-31T06:23:19.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.795+0000 7ff896e06740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff8881b54f0 con 0x56105163f1b0 2024-08-31T06:23:19.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.795+0000 7ff896e06740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff8881b4750 con 0x561051647390 2024-08-31T06:23:19.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.795+0000 7ff896e06740 10 client.0 _ll_register_callbacks cb 0x7ff87c0aba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:23:19.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.795+0000 7ff88e200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561051668070 0x561051a1e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:19.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.795+0000 7ff88d800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56105163f1b0 0x561051a1d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:19.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.795+0000 7ff88ce00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561051647390 0x561051a1dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:19.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.795+0000 7ff88e200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561051668070 0x561051a1e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47534/0 (socket says 172.21.15.16:47534) 2024-08-31T06:23:19.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.795+0000 7ff88e200640 1 -- 172.21.15.16:0/248824400 learned_addr learned my addr 172.21.15.16:0/248824400 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:23:19.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.795+0000 7ff88d800640 1 -- 172.21.15.16:0/248824400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561051647390 msgr2=0x561051a1dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:23:19.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.795+0000 7ff88d800640 1 --2- 172.21.15.16:0/248824400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561051647390 0x561051a1dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:19.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.795+0000 7ff88d800640 1 -- 172.21.15.16:0/248824400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561051668070 msgr2=0x561051a1e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:23:19.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.795+0000 7ff88d800640 1 --2- 172.21.15.16:0/248824400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561051668070 0x561051a1e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:19.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.795+0000 7ff88d800640 1 -- 172.21.15.16:0/248824400 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561051a95270 con 0x56105163f1b0 2024-08-31T06:23:19.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.796+0000 7ff88d800640 1 --2- 172.21.15.16:0/248824400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56105163f1b0 0x561051a1d580 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7ff87c0abe30 tx=0x7ff87c0abb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12e654d453fd97f7 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:19.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.796+0000 7ff86f400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:23:19.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.796+0000 7ff86f400640 1 -- 172.21.15.16:0/248824400 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87c0971c0 con 0x56105163f1b0 2024-08-31T06:23:19.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.796+0000 7ff86f400640 1 -- 172.21.15.16:0/248824400 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8470+0+0 (secure 0 0 0) 0x7ff87c0ab5a0 con 0x56105163f1b0 2024-08-31T06:23:19.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.796+0000 7ff896e06740 1 -- 172.21.15.16:0/248824400 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5610519821a0 con 0x56105163f1b0 2024-08-31T06:23:19.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.796+0000 7ff86f400640 1 -- 172.21.15.16:0/248824400 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7ff87c097380 con 0x56105163f1b0 2024-08-31T06:23:19.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.796+0000 7ff896e06740 10 client.29120 fetch_fsmap learned FSMap version 55 2024-08-31T06:23:19.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.796+0000 7ff896e06740 1 -- 172.21.15.16:0/248824400 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x561051a964d0 con 0x56105163f1b0 2024-08-31T06:23:19.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.797+0000 7ff86f400640 1 -- 172.21.15.16:0/248824400 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5610519821a0 con 0x56105163f1b0 2024-08-31T06:23:19.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.797+0000 7ff86f400640 1 -- 172.21.15.16:0/248824400 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 55) ==== 8358+0+0 (secure 0 0 0) 0x7ff87c0b1b30 con 0x56105163f1b0 2024-08-31T06:23:19.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.797+0000 7ff896e06740 10 client.29120 fetch_fsmap finished waiting for FSMap version 55 2024-08-31T06:23:19.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.797+0000 7ff896e06740 10 client.29120 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-31T06:23:19.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.797+0000 7ff896e06740 20 client.29120 populate_metadata read hostname 'smithi016' 2024-08-31T06:23:19.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.797+0000 7ff896e06740 1 --2- 172.21.15.16:0/248824400 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x5610515fbda0 0x561051a98cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:19.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.797+0000 7ff896e06740 4 client.29120 mds_command: new command op to 24724 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:23:19.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.797+0000 7ff896e06740 1 -- 172.21.15.16:0/248824400 --> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56105197f8e0 con 0x5610515fbda0 2024-08-31T06:23:19.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.797+0000 7ff88ce00640 1 --2- 172.21.15.16:0/248824400 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x5610515fbda0 0x561051a98cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:19.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.798+0000 7ff88ce00640 1 --2- 172.21.15.16:0/248824400 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x5610515fbda0 0x561051a98cf0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c8da462240ff0a57 server_cookie=b87a11184a0b4500 in_seq=0 out_seq=0 2024-08-31T06:23:19.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.798+0000 7ff86f400640 10 client.29120 ms_handle_connect on v2:172.21.15.16:6838/219664460 2024-08-31T06:23:19.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.798+0000 7ff86f400640 1 -- 172.21.15.16:0/248824400 <== mds.1 v2:172.21.15.16:6838/219664460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56105197f8e0 con 0x5610515fbda0 2024-08-31T06:23:19.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.798+0000 7ff86f400640 10 client.29120 handle_command_reply: tid=0 2024-08-31T06:23:19.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.798+0000 7ff896e06740 1 client.29120 shutdown 2024-08-31T06:23:19.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.798+0000 7ff896e06740 1 -- 172.21.15.16:0/248824400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56105163f1b0 msgr2=0x561051a1d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:19.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.798+0000 7ff896e06740 1 --2- 172.21.15.16:0/248824400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56105163f1b0 0x561051a1d580 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7ff87c0abe30 tx=0x7ff87c0abb30 comp rx=0 tx=0).stop 2024-08-31T06:23:19.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.799+0000 7ff896e06740 1 -- 172.21.15.16:0/248824400 shutdown_connections 2024-08-31T06:23:19.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.799+0000 7ff896e06740 1 --2- 172.21.15.16:0/248824400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561051647390 0x561051a1dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:19.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.799+0000 7ff896e06740 1 --2- 172.21.15.16:0/248824400 >> [v2:172.21.15.16:6838/219664460,v1:172.21.15.16:6839/219664460] conn(0x5610515fbda0 0x561051a98cf0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:19.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.799+0000 7ff896e06740 1 --2- 172.21.15.16:0/248824400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561051668070 0x561051a1e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:19.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.799+0000 7ff896e06740 1 --2- 172.21.15.16:0/248824400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56105163f1b0 0x561051a1d580 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:19.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.799+0000 7ff86f400640 0 client.29120 ms_handle_reset on v2:172.21.15.16:6838/219664460 2024-08-31T06:23:19.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.799+0000 7ff896e06740 1 -- 172.21.15.16:0/248824400 >> 172.21.15.16:0/248824400 conn(0x56105165c460 msgr2=0x561051a0e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:19.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.799+0000 7ff896e06740 1 -- 172.21.15.16:0/248824400 shutdown_connections 2024-08-31T06:23:19.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.799+0000 7ff896e06740 1 -- 172.21.15.16:0/248824400 wait complete. 2024-08-31T06:23:19.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.799+0000 7ff896e06740 20 client.29120 trim_cache size 0 max 16384 2024-08-31T06:23:19.804 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:23:19.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.800+0000 7ff890200640 1 -- 172.21.15.16:0/1095284495 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff8700b4ca0 msgr2=0x7ff8700b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:19.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.800+0000 7ff890200640 1 --2- 172.21.15.16:0/1095284495 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff8700b4ca0 0x7ff8700b71b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7ff8800094e0 tx=0x7ff880005d80 comp rx=0 tx=0).stop 2024-08-31T06:23:19.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.800+0000 7ff890200640 1 -- 172.21.15.16:0/1095284495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8880776e0 msgr2=0x7ff8881a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:19.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.800+0000 7ff890200640 1 --2- 172.21.15.16:0/1095284495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8880776e0 0x7ff8881a0820 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ff87c00e9f0 tx=0x7ff87c00edf0 comp rx=0 tx=0).stop 2024-08-31T06:23:19.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.800+0000 7ff890200640 1 -- 172.21.15.16:0/1095284495 shutdown_connections 2024-08-31T06:23:19.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.800+0000 7ff890200640 1 --2- 172.21.15.16:0/1095284495 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7ff8700b4ca0 0x7ff8700b71b0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:19.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.800+0000 7ff890200640 1 --2- 172.21.15.16:0/1095284495 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff888078030 0x7ff8881a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:19.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.800+0000 7ff890200640 1 --2- 172.21.15.16:0/1095284495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8881144e0 0x7ff8881a6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:19.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.800+0000 7ff890200640 1 --2- 172.21.15.16:0/1095284495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8880776e0 0x7ff8881a0820 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:19.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.800+0000 7ff890200640 1 -- 172.21.15.16:0/1095284495 >> 172.21.15.16:0/1095284495 conn(0x7ff8881015b0 msgr2=0x7ff8881164c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:19.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.800+0000 7ff890200640 1 -- 172.21.15.16:0/1095284495 shutdown_connections 2024-08-31T06:23:19.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:19.800+0000 7ff890200640 1 -- 172.21.15.16:0/1095284495 wait complete. 2024-08-31T06:23:20.803 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-31T06:23:21.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:20 smithi155 ceph-mon[21742]: pgmap v869: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.8 KiB/s wr, 7 op/s 2024-08-31T06:23:21.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:20 smithi016 ceph-mon[17194]: pgmap v869: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.8 KiB/s wr, 7 op/s 2024-08-31T06:23:21.182 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:23:21.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:20 smithi184 ceph-mon[22331]: pgmap v869: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.8 KiB/s wr, 7 op/s 2024-08-31T06:23:21.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.811+0000 7f3590200640 1 -- 172.21.15.16:0/1757149378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3588103900 msgr2=0x7f3588103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:21.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.811+0000 7f3590200640 1 --2- 172.21.15.16:0/1757149378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3588103900 0x7f3588103d10 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f357c009900 tx=0x7f357c02a400 comp rx=0 tx=0).stop 2024-08-31T06:23:21.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.811+0000 7f3590200640 1 -- 172.21.15.16:0/1757149378 shutdown_connections 2024-08-31T06:23:21.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.811+0000 7f3590200640 1 --2- 172.21.15.16:0/1757149378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3588109b90 0x7f35881103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:21.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.811+0000 7f3590200640 1 --2- 172.21.15.16:0/1757149378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3588109240 0x7f3588109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:21.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.811+0000 7f3590200640 1 --2- 172.21.15.16:0/1757149378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3588103900 0x7f3588103d10 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:21.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.811+0000 7f3590200640 1 -- 172.21.15.16:0/1757149378 >> 172.21.15.16:0/1757149378 conn(0x7f358807b5f0 msgr2=0x7f358807ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:21.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.811+0000 7f3590200640 1 -- 172.21.15.16:0/1757149378 shutdown_connections 2024-08-31T06:23:21.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.811+0000 7f3590200640 1 -- 172.21.15.16:0/1757149378 wait complete. 2024-08-31T06:23:21.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.812+0000 7f3590200640 1 Processor -- start 2024-08-31T06:23:21.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.812+0000 7f3590200640 1 -- start start 2024-08-31T06:23:21.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.813+0000 7f3590200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3588103900 0x7f358819a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:21.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.813+0000 7f3590200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3588109240 0x7f358819a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:21.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.813+0000 7f3590200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3588109b90 0x7f358819faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:21.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.813+0000 7f3590200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3588113850 con 0x7f3588109b90 2024-08-31T06:23:21.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.813+0000 7f3590200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f35881136d0 con 0x7f3588109240 2024-08-31T06:23:21.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.813+0000 7f3590200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f35881139d0 con 0x7f3588103900 2024-08-31T06:23:21.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.813+0000 7f358e200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3588109b90 0x7f358819faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:21.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.813+0000 7f358e200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3588109b90 0x7f358819faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47552/0 (socket says 172.21.15.16:47552) 2024-08-31T06:23:21.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.813+0000 7f358e200640 1 -- 172.21.15.16:0/3477323585 learned_addr learned my addr 172.21.15.16:0/3477323585 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:23:21.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.813+0000 7f358ce00640 1 --2- 172.21.15.16:0/3477323585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3588109240 0x7f358819a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:21.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.813+0000 7f358d800640 1 --2- 172.21.15.16:0/3477323585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3588103900 0x7f358819a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:21.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.814+0000 7f358e200640 1 -- 172.21.15.16:0/3477323585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3588103900 msgr2=0x7f358819a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:21.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.814+0000 7f358e200640 1 --2- 172.21.15.16:0/3477323585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3588103900 0x7f358819a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:21.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.814+0000 7f358e200640 1 -- 172.21.15.16:0/3477323585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3588109240 msgr2=0x7f358819a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:21.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.814+0000 7f358e200640 1 --2- 172.21.15.16:0/3477323585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3588109240 0x7f358819a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:21.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.814+0000 7f358e200640 1 -- 172.21.15.16:0/3477323585 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35881a01d0 con 0x7f3588109b90 2024-08-31T06:23:21.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.814+0000 7f358e200640 1 --2- 172.21.15.16:0/3477323585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3588109b90 0x7f358819faa0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f358000efd0 tx=0x7f358000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:21.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.814+0000 7f3586000640 1 -- 172.21.15.16:0/3477323585 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3580003b70 con 0x7f3588109b90 2024-08-31T06:23:21.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.814+0000 7f3590200640 1 -- 172.21.15.16:0/3477323585 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35881a04c0 con 0x7f3588109b90 2024-08-31T06:23:21.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.814+0000 7f3586000640 1 -- 172.21.15.16:0/3477323585 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3580003d10 con 0x7f3588109b90 2024-08-31T06:23:21.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.814+0000 7f3586000640 1 -- 172.21.15.16:0/3477323585 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35800108a0 con 0x7f3588109b90 2024-08-31T06:23:21.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.815+0000 7f3590200640 1 -- 172.21.15.16:0/3477323585 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35881a6c20 con 0x7f3588109b90 2024-08-31T06:23:21.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.818+0000 7f3596e90740 1 Processor -- start 2024-08-31T06:23:21.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.818+0000 7f3586000640 1 -- 172.21.15.16:0/3477323585 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f3580004020 con 0x7f3588109b90 2024-08-31T06:23:21.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.818+0000 7f3596e90740 1 -- start start 2024-08-31T06:23:21.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.819+0000 7f3596e90740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568b5d86070 0x5568b612b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:21.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.819+0000 7f3596e90740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568b5d5d1b0 0x5568b612bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:21.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.819+0000 7f3596e90740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b5d65390 0x5568b612c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:21.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.819+0000 7f3596e90740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5568b60a27b0 con 0x5568b5d65390 2024-08-31T06:23:21.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.819+0000 7f3596e90740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5568b6091d40 con 0x5568b5d5d1b0 2024-08-31T06:23:21.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.819+0000 7f3596e90740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5568b6095cf0 con 0x5568b5d86070 2024-08-31T06:23:21.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.819+0000 7f358ce00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568b5d86070 0x5568b612b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:21.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.819+0000 7f358d800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568b5d5d1b0 0x5568b612bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:21.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.819+0000 7f358ce00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568b5d86070 0x5568b612b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:46252/0 (socket says 172.21.15.16:46252) 2024-08-31T06:23:21.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.819+0000 7f358d800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568b5d5d1b0 0x5568b612bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:49660/0 (socket says 172.21.15.16:49660) 2024-08-31T06:23:21.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.819+0000 7f358ce00640 1 -- 172.21.15.16:0/1599654224 learned_addr learned my addr 172.21.15.16:0/1599654224 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:23:21.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.819+0000 7f358ce00640 1 --2- 172.21.15.16:0/1599654224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b5d65390 0x5568b612c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:21.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.820+0000 7f358d800640 1 -- 172.21.15.16:0/1599654224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568b5d86070 msgr2=0x5568b612b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:23:21.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.820+0000 7f358d800640 1 --2- 172.21.15.16:0/1599654224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568b5d86070 0x5568b612b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:21.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.820+0000 7f358d800640 1 -- 172.21.15.16:0/1599654224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b5d65390 msgr2=0x5568b612c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:23:21.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.820+0000 7f358d800640 1 --2- 172.21.15.16:0/1599654224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b5d65390 0x5568b612c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:21.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.820+0000 7f358d800640 1 -- 172.21.15.16:0/1599654224 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568b5df1650 con 0x5568b5d5d1b0 2024-08-31T06:23:21.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.820+0000 7f3586000640 1 --2- 172.21.15.16:0/3477323585 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f35700b4ca0 0x7f35700b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:21.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.820+0000 7f3586000640 1 -- 172.21.15.16:0/3477323585 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8470+0+0 (secure 0 0 0) 0x7f35800e5990 con 0x7f3588109b90 2024-08-31T06:23:21.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.820+0000 7f358d800640 1 --2- 172.21.15.16:0/1599654224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568b5d5d1b0 0x5568b612bed0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f357c009230 tx=0x7f357c032b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ceef783a5bd7e15 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:21.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.820+0000 7f358ce00640 1 --2- 172.21.15.16:0/3477323585 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f35700b4ca0 0x7f35700b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:21.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.821+0000 7f357aa00640 1 -- 172.21.15.16:0/1599654224 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357c040070 con 0x5568b5d5d1b0 2024-08-31T06:23:21.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.821+0000 7f357aa00640 1 -- 172.21.15.16:0/1599654224 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f357c004030 con 0x5568b5d5d1b0 2024-08-31T06:23:21.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.821+0000 7f357aa00640 1 -- 172.21.15.16:0/1599654224 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357c035040 con 0x5568b5d5d1b0 2024-08-31T06:23:21.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.821+0000 7f358ce00640 1 --2- 172.21.15.16:0/3477323585 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f35700b4ca0 0x7f35700b71b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f35740058c0 tx=0x7f3574005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:21.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.821+0000 7f3596e90740 1 -- 172.21.15.16:0/1599654224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568b5d5d1b0 msgr2=0x5568b612bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:21.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.821+0000 7f3596e90740 1 --2- 172.21.15.16:0/1599654224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568b5d5d1b0 0x5568b612bed0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f357c009230 tx=0x7f357c032b90 comp rx=0 tx=0).stop 2024-08-31T06:23:21.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.822+0000 7f3596e90740 1 -- 172.21.15.16:0/1599654224 shutdown_connections 2024-08-31T06:23:21.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.822+0000 7f3596e90740 1 --2- 172.21.15.16:0/1599654224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b5d65390 0x5568b612c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:21.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.822+0000 7f3596e90740 1 --2- 172.21.15.16:0/1599654224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568b5d5d1b0 0x5568b612bed0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:21.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.822+0000 7f3596e90740 1 --2- 172.21.15.16:0/1599654224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568b5d86070 0x5568b612b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:21.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.822+0000 7f3596e90740 1 -- 172.21.15.16:0/1599654224 >> 172.21.15.16:0/1599654224 conn(0x5568b5d7a460 msgr2=0x5568b6129880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:21.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.822+0000 7f3596e90740 1 -- 172.21.15.16:0/1599654224 shutdown_connections 2024-08-31T06:23:21.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.822+0000 7f3596e90740 1 -- 172.21.15.16:0/1599654224 wait complete. 2024-08-31T06:23:21.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.822+0000 7f3596e90740 1 -- start start 2024-08-31T06:23:21.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.823+0000 7f3596e90740 1 Processor -- start 2024-08-31T06:23:21.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.823+0000 7f3596e90740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568b5d5d1b0 0x5568b613b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:21.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.823+0000 7f3596e90740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b5d65390 0x5568b613be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:21.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.823+0000 7f3596e90740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568b5d86070 0x5568b6147770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:21.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.823+0000 7f3596e90740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5568b5ef4700 con 0x5568b5d65390 2024-08-31T06:23:21.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.823+0000 7f3596e90740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5568b6018cf0 con 0x5568b5d86070 2024-08-31T06:23:21.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.823+0000 7f3596e90740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5568b613e440 con 0x5568b5d5d1b0 2024-08-31T06:23:21.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.823+0000 7f3596e90740 10 client.0 _ll_register_callbacks cb 0x5568b603e470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:23:21.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.823+0000 7f358ce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b5d65390 0x5568b613be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:21.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.823+0000 7f358ce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b5d65390 0x5568b613be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47570/0 (socket says 172.21.15.16:47570) 2024-08-31T06:23:21.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.823+0000 7f358ce00640 1 -- 172.21.15.16:0/2130488509 learned_addr learned my addr 172.21.15.16:0/2130488509 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:23:21.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.823+0000 7f358d800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568b5d5d1b0 0x5568b613b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:21.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.824+0000 7f358e200640 1 --2- 172.21.15.16:0/2130488509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568b5d86070 0x5568b6147770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:21.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.824+0000 7f358d800640 1 -- 172.21.15.16:0/2130488509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568b5d86070 msgr2=0x5568b6147770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:23:21.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.824+0000 7f358d800640 1 --2- 172.21.15.16:0/2130488509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568b5d86070 0x5568b6147770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:21.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.824+0000 7f358d800640 1 -- 172.21.15.16:0/2130488509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b5d65390 msgr2=0x5568b613be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:23:21.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.824+0000 7f358d800640 1 --2- 172.21.15.16:0/2130488509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b5d65390 0x5568b613be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:21.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.824+0000 7f358d800640 1 -- 172.21.15.16:0/2130488509 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568b60598b0 con 0x5568b5d5d1b0 2024-08-31T06:23:21.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.824+0000 7f358d800640 1 --2- 172.21.15.16:0/2130488509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568b5d5d1b0 0x5568b613b900 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f357c009230 tx=0x7f357c032f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e273cab2b635a984 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:21.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.824+0000 7f356f400640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-08-31T06:23:21.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.825+0000 7f356f400640 1 -- 172.21.15.16:0/2130488509 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357c052060 con 0x5568b5d5d1b0 2024-08-31T06:23:21.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.825+0000 7f3596e90740 1 -- 172.21.15.16:0/2130488509 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f357c004030 con 0x5568b5d5d1b0 2024-08-31T06:23:21.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.825+0000 7f356f400640 1 -- 172.21.15.16:0/2130488509 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8470+0+0 (secure 0 0 0) 0x7f357c0316a0 con 0x5568b5d5d1b0 2024-08-31T06:23:21.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.825+0000 7f356f400640 1 -- 172.21.15.16:0/2130488509 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f357c035070 con 0x5568b5d5d1b0 2024-08-31T06:23:21.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.825+0000 7f3596e90740 10 client.29164 fetch_fsmap learned FSMap version 55 2024-08-31T06:23:21.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.825+0000 7f3596e90740 1 -- 172.21.15.16:0/2130488509 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5568b614a440 con 0x5568b5d5d1b0 2024-08-31T06:23:21.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.825+0000 7f356f400640 1 -- 172.21.15.16:0/2130488509 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f357c004030 con 0x5568b5d5d1b0 2024-08-31T06:23:21.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.825+0000 7f356f400640 1 -- 172.21.15.16:0/2130488509 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 55) ==== 8358+0+0 (secure 0 0 0) 0x7f357c004a80 con 0x5568b5d5d1b0 2024-08-31T06:23:21.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.826+0000 7f3596e90740 10 client.29164 fetch_fsmap finished waiting for FSMap version 55 2024-08-31T06:23:21.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.826+0000 7f3596e90740 10 client.29164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:23:21.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.826+0000 7f3596e90740 20 client.29164 populate_metadata read hostname 'smithi016' 2024-08-31T06:23:21.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.826+0000 7f3596e90740 1 --2- 172.21.15.16:0/2130488509 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5568b5d449f0 0x5568b614d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:21.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.826+0000 7f3596e90740 4 client.29164 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-31T06:23:21.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.826+0000 7f3596e90740 1 -- 172.21.15.16:0/2130488509 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568b60a4700 con 0x5568b5d449f0 2024-08-31T06:23:21.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.826+0000 7f358ce00640 1 --2- 172.21.15.16:0/2130488509 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5568b5d449f0 0x5568b614d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:21.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.827+0000 7f358ce00640 1 --2- 172.21.15.16:0/2130488509 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5568b5d449f0 0x5568b614d7a0 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcc3e86bd0934209 server_cookie=161d6ac07221b57b in_seq=0 out_seq=0 2024-08-31T06:23:21.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.827+0000 7f356f400640 10 client.29164 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:23:21.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.827+0000 7f356f400640 1 -- 172.21.15.16:0/2130488509 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5568b60a4700 con 0x5568b5d449f0 2024-08-31T06:23:21.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.827+0000 7f356f400640 10 client.29164 handle_command_reply: tid=0 2024-08-31T06:23:21.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.827+0000 7f3596e90740 1 client.29164 shutdown 2024-08-31T06:23:21.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.828+0000 7f3596e90740 1 -- 172.21.15.16:0/2130488509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568b5d5d1b0 msgr2=0x5568b613b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:21.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.828+0000 7f3596e90740 1 --2- 172.21.15.16:0/2130488509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568b5d5d1b0 0x5568b613b900 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f357c009230 tx=0x7f357c032f30 comp rx=0 tx=0).stop 2024-08-31T06:23:21.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.828+0000 7f3596e90740 1 -- 172.21.15.16:0/2130488509 shutdown_connections 2024-08-31T06:23:21.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.828+0000 7f3596e90740 1 --2- 172.21.15.16:0/2130488509 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5568b5d449f0 0x5568b614d7a0 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:21.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.828+0000 7f3596e90740 1 --2- 172.21.15.16:0/2130488509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568b5d86070 0x5568b6147770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:21.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.828+0000 7f3596e90740 1 --2- 172.21.15.16:0/2130488509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b5d65390 0x5568b613be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:21.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.828+0000 7f3596e90740 1 --2- 172.21.15.16:0/2130488509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568b5d5d1b0 0x5568b613b900 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:21.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.828+0000 7f356f400640 0 client.29164 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:23:21.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.828+0000 7f3596e90740 1 -- 172.21.15.16:0/2130488509 >> 172.21.15.16:0/2130488509 conn(0x5568b5d7a460 msgr2=0x5568b612eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:21.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.828+0000 7f3596e90740 1 -- 172.21.15.16:0/2130488509 shutdown_connections 2024-08-31T06:23:21.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.828+0000 7f3596e90740 1 -- 172.21.15.16:0/2130488509 wait complete. 2024-08-31T06:23:21.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.828+0000 7f3596e90740 20 client.29164 trim_cache size 0 max 16384 2024-08-31T06:23:21.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.842+0000 7f3596e90740 1 Processor -- start 2024-08-31T06:23:21.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.842+0000 7f3596e90740 1 -- start start 2024-08-31T06:23:21.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.843+0000 7f3596e90740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568b5d5d1b0 0x5568b612c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:21.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.843+0000 7f3596e90740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568b5d65390 0x5568b61476e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:21.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.843+0000 7f3596e90740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b5d86070 0x5568b6147c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:21.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.843+0000 7f3596e90740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f35881afa60 con 0x5568b5d86070 2024-08-31T06:23:21.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.843+0000 7f3596e90740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f35881ad300 con 0x5568b5d65390 2024-08-31T06:23:21.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.843+0000 7f3596e90740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f35881ab170 con 0x5568b5d5d1b0 2024-08-31T06:23:21.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.843+0000 7f358e200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b5d86070 0x5568b6147c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:21.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.843+0000 7f358e200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b5d86070 0x5568b6147c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47582/0 (socket says 172.21.15.16:47582) 2024-08-31T06:23:21.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.843+0000 7f358e200640 1 -- 172.21.15.16:0/3607771910 learned_addr learned my addr 172.21.15.16:0/3607771910 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:23:21.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.843+0000 7f358ce00640 1 --2- 172.21.15.16:0/3607771910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568b5d65390 0x5568b61476e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:21.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.843+0000 7f358d800640 1 --2- 172.21.15.16:0/3607771910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568b5d5d1b0 0x5568b612c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:21.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.843+0000 7f358e200640 1 -- 172.21.15.16:0/3607771910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568b5d5d1b0 msgr2=0x5568b612c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:23:21.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.843+0000 7f358e200640 1 --2- 172.21.15.16:0/3607771910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568b5d5d1b0 0x5568b612c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:21.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.843+0000 7f358e200640 1 -- 172.21.15.16:0/3607771910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568b5d65390 msgr2=0x5568b61476e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:23:21.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.843+0000 7f358e200640 1 --2- 172.21.15.16:0/3607771910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568b5d65390 0x5568b61476e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:21.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.843+0000 7f358e200640 1 -- 172.21.15.16:0/3607771910 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568b612fa40 con 0x5568b5d86070 2024-08-31T06:23:21.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.844+0000 7f358e200640 1 --2- 172.21.15.16:0/3607771910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b5d86070 0x5568b6147c20 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f358000cba0 tx=0x7f35800ac750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=556006fad155d3ba server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:21.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.844+0000 7f357a000640 1 -- 172.21.15.16:0/3607771910 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35800b08c0 con 0x5568b5d86070 2024-08-31T06:23:21.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.844+0000 7f357a000640 1 -- 172.21.15.16:0/3607771910 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35800b5370 con 0x5568b5d86070 2024-08-31T06:23:21.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.844+0000 7f357a000640 1 -- 172.21.15.16:0/3607771910 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35800ab510 con 0x5568b5d86070 2024-08-31T06:23:21.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.845+0000 7f3596e90740 1 -- 172.21.15.16:0/3607771910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b5d86070 msgr2=0x5568b6147c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:21.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.845+0000 7f3596e90740 1 --2- 172.21.15.16:0/3607771910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b5d86070 0x5568b6147c20 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f358000cba0 tx=0x7f35800ac750 comp rx=0 tx=0).stop 2024-08-31T06:23:21.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.845+0000 7f3596e90740 1 -- 172.21.15.16:0/3607771910 shutdown_connections 2024-08-31T06:23:21.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.845+0000 7f3596e90740 1 --2- 172.21.15.16:0/3607771910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b5d86070 0x5568b6147c20 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:21.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.845+0000 7f3596e90740 1 --2- 172.21.15.16:0/3607771910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568b5d65390 0x5568b61476e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:21.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.845+0000 7f3596e90740 1 --2- 172.21.15.16:0/3607771910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568b5d5d1b0 0x5568b612c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:21.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.845+0000 7f3596e90740 1 -- 172.21.15.16:0/3607771910 >> 172.21.15.16:0/3607771910 conn(0x5568b5d7a460 msgr2=0x5568b613d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:21.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.845+0000 7f3596e90740 1 -- 172.21.15.16:0/3607771910 shutdown_connections 2024-08-31T06:23:21.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.845+0000 7f3596e90740 1 -- 172.21.15.16:0/3607771910 wait complete. 2024-08-31T06:23:21.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.845+0000 7f3596e90740 1 -- start start 2024-08-31T06:23:21.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.846+0000 7f3596e90740 1 Processor -- start 2024-08-31T06:23:21.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.846+0000 7f3596e90740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b5d5d1b0 0x5568b6150cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:21.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.846+0000 7f3596e90740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568b5d65390 0x5568b614d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:21.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.846+0000 7f3596e90740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568b5d86070 0x5568b614d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:21.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.846+0000 7f3596e90740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f35881ab7c0 con 0x5568b5d5d1b0 2024-08-31T06:23:21.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.846+0000 7f3596e90740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f35881acc30 con 0x5568b5d65390 2024-08-31T06:23:21.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.846+0000 7f3596e90740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f35881abe90 con 0x5568b5d86070 2024-08-31T06:23:21.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.846+0000 7f3596e90740 10 client.0 _ll_register_callbacks cb 0x7f357c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-31T06:23:21.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.846+0000 7f358d800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b5d5d1b0 0x5568b6150cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:21.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.846+0000 7f358d800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b5d5d1b0 0x5568b6150cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47584/0 (socket says 172.21.15.16:47584) 2024-08-31T06:23:21.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.846+0000 7f358d800640 1 -- 172.21.15.16:0/948731018 learned_addr learned my addr 172.21.15.16:0/948731018 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:23:21.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.846+0000 7f358ce00640 1 --2- 172.21.15.16:0/948731018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568b5d65390 0x5568b614d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:21.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.847+0000 7f358e200640 1 --2- 172.21.15.16:0/948731018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568b5d86070 0x5568b614d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:21.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.847+0000 7f358ce00640 1 -- 172.21.15.16:0/948731018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568b5d86070 msgr2=0x5568b614d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:23:21.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.847+0000 7f358ce00640 1 --2- 172.21.15.16:0/948731018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568b5d86070 0x5568b614d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:21.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.847+0000 7f358ce00640 1 -- 172.21.15.16:0/948731018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b5d5d1b0 msgr2=0x5568b6150cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-31T06:23:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.847+0000 7f358ce00640 1 --2- 172.21.15.16:0/948731018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b5d5d1b0 0x5568b6150cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.847+0000 7f358ce00640 1 -- 172.21.15.16:0/948731018 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568b614dbf0 con 0x5568b5d65390 2024-08-31T06:23:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.847+0000 7f358e200640 1 --2- 172.21.15.16:0/948731018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568b5d86070 0x5568b614d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:23:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.847+0000 7f358d800640 1 --2- 172.21.15.16:0/948731018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b5d5d1b0 0x5568b6150cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:23:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.847+0000 7f358ce00640 1 --2- 172.21.15.16:0/948731018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568b5d65390 0x5568b614d170 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x5568b6091e10 tx=0x7f3574004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef067b67690969bc server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.847+0000 7f356f400640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-31T06:23:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.848+0000 7f356f400640 1 -- 172.21.15.16:0/948731018 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3574039060 con 0x5568b5d65390 2024-08-31T06:23:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.848+0000 7f356f400640 1 -- 172.21.15.16:0/948731018 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8470+0+0 (secure 0 0 0) 0x7f3574011d10 con 0x5568b5d65390 2024-08-31T06:23:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.848+0000 7f3596e90740 1 -- 172.21.15.16:0/948731018 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35800b5370 con 0x5568b5d65390 2024-08-31T06:23:21.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.848+0000 7f356f400640 1 -- 172.21.15.16:0/948731018 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f3574011030 con 0x5568b5d65390 2024-08-31T06:23:21.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.848+0000 7f3596e90740 10 client.29138 fetch_fsmap learned FSMap version 55 2024-08-31T06:23:21.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.848+0000 7f3596e90740 1 -- 172.21.15.16:0/948731018 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5568b61b8bc0 con 0x5568b5d65390 2024-08-31T06:23:21.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.848+0000 7f356f400640 1 -- 172.21.15.16:0/948731018 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35800b5370 con 0x5568b5d65390 2024-08-31T06:23:21.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.849+0000 7f356f400640 1 -- 172.21.15.16:0/948731018 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 55) ==== 8358+0+0 (secure 0 0 0) 0x7f35740049e0 con 0x5568b5d65390 2024-08-31T06:23:21.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.849+0000 7f3596e90740 10 client.29138 fetch_fsmap finished waiting for FSMap version 55 2024-08-31T06:23:21.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.849+0000 7f3596e90740 10 client.29138 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-31T06:23:21.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.849+0000 7f3596e90740 20 client.29138 populate_metadata read hostname 'smithi016' 2024-08-31T06:23:21.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.849+0000 7f3596e90740 1 --2- 172.21.15.16:0/948731018 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5568b5d19da0 0x5568b61bb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:21.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.849+0000 7f3596e90740 4 client.29138 mds_command: new command op to 24737 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-31T06:23:21.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.849+0000 7f3596e90740 1 -- 172.21.15.16:0/948731018 --> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5568b60a01a0 con 0x5568b5d19da0 2024-08-31T06:23:21.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.849+0000 7f358d800640 1 --2- 172.21.15.16:0/948731018 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5568b5d19da0 0x5568b61bb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:21.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.850+0000 7f358d800640 1 --2- 172.21.15.16:0/948731018 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5568b5d19da0 0x5568b61bb360 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afc5eab31652323b server_cookie=355f594f3848125 in_seq=0 out_seq=0 2024-08-31T06:23:21.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.850+0000 7f356f400640 10 client.29138 ms_handle_connect on v2:172.21.15.155:6838/390606290 2024-08-31T06:23:21.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.850+0000 7f356f400640 1 -- 172.21.15.16:0/948731018 <== mds.0 v2:172.21.15.155:6838/390606290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5568b60a01a0 con 0x5568b5d19da0 2024-08-31T06:23:21.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.850+0000 7f356f400640 10 client.29138 handle_command_reply: tid=0 2024-08-31T06:23:21.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.850+0000 7f3596e90740 1 client.29138 shutdown 2024-08-31T06:23:21.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.850+0000 7f3596e90740 1 -- 172.21.15.16:0/948731018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568b5d65390 msgr2=0x5568b614d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:21.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.850+0000 7f3596e90740 1 --2- 172.21.15.16:0/948731018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568b5d65390 0x5568b614d170 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x5568b6091e10 tx=0x7f3574004730 comp rx=0 tx=0).stop 2024-08-31T06:23:21.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.850+0000 7f3596e90740 1 -- 172.21.15.16:0/948731018 shutdown_connections 2024-08-31T06:23:21.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.850+0000 7f3596e90740 1 --2- 172.21.15.16:0/948731018 >> [v2:172.21.15.155:6838/390606290,v1:172.21.15.155:6839/390606290] conn(0x5568b5d19da0 0x5568b61bb360 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:21.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.850+0000 7f3596e90740 1 --2- 172.21.15.16:0/948731018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568b5d86070 0x5568b614d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:21.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.850+0000 7f3596e90740 1 --2- 172.21.15.16:0/948731018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568b5d65390 0x5568b614d170 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:21.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.850+0000 7f356f400640 0 client.29138 ms_handle_reset on v2:172.21.15.155:6838/390606290 2024-08-31T06:23:21.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.850+0000 7f3596e90740 1 --2- 172.21.15.16:0/948731018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b5d5d1b0 0x5568b6150cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:21.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.850+0000 7f3596e90740 1 -- 172.21.15.16:0/948731018 >> 172.21.15.16:0/948731018 conn(0x5568b5d7a460 msgr2=0x5568b612f190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:21.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.851+0000 7f3596e90740 1 -- 172.21.15.16:0/948731018 shutdown_connections 2024-08-31T06:23:21.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.851+0000 7f3596e90740 1 -- 172.21.15.16:0/948731018 wait complete. 2024-08-31T06:23:21.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.851+0000 7f3596e90740 20 client.29138 trim_cache size 0 max 16384 2024-08-31T06:23:21.858 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-08-31T06:23:21.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.851+0000 7f3590200640 1 -- 172.21.15.16:0/3477323585 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f35700b4ca0 msgr2=0x7f35700b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:21.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.851+0000 7f3590200640 1 --2- 172.21.15.16:0/3477323585 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f35700b4ca0 0x7f35700b71b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f35740058c0 tx=0x7f3574005f90 comp rx=0 tx=0).stop 2024-08-31T06:23:21.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.851+0000 7f3590200640 1 -- 172.21.15.16:0/3477323585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3588109b90 msgr2=0x7f358819faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:21.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.851+0000 7f3590200640 1 --2- 172.21.15.16:0/3477323585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3588109b90 0x7f358819faa0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f358000efd0 tx=0x7f358000c6a0 comp rx=0 tx=0).stop 2024-08-31T06:23:21.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.852+0000 7f3590200640 1 -- 172.21.15.16:0/3477323585 shutdown_connections 2024-08-31T06:23:21.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.852+0000 7f3590200640 1 --2- 172.21.15.16:0/3477323585 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f35700b4ca0 0x7f35700b71b0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:21.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.852+0000 7f3590200640 1 --2- 172.21.15.16:0/3477323585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3588109b90 0x7f358819faa0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:21.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.852+0000 7f3590200640 1 --2- 172.21.15.16:0/3477323585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3588109240 0x7f358819a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:21.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.852+0000 7f3590200640 1 --2- 172.21.15.16:0/3477323585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3588103900 0x7f358819a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:21.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.852+0000 7f3590200640 1 -- 172.21.15.16:0/3477323585 >> 172.21.15.16:0/3477323585 conn(0x7f358807b5f0 msgr2=0x7f3588107330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:21.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.852+0000 7f3590200640 1 -- 172.21.15.16:0/3477323585 shutdown_connections 2024-08-31T06:23:21.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:21.852+0000 7f3590200640 1 -- 172.21.15.16:0/3477323585 wait complete. 2024-08-31T06:23:22.854 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_d19e6118f7015f0047264006cb0c873528eb3ae6/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_d19e6118f7015f0047264006cb0c873528eb3ae6/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_d19e6118f7015f0047264006cb0c873528eb3ae6/qa/tasks/thrasher.py", line 50, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-08-31T06:23:22.854 INFO:tasks.fwd_scrub:done joining 2024-08-31T06:23:22.855 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-08-31T06:23:22.865 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-31T06:23:23.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.089+0000 7f9d08a00640 1 -- 172.21.15.16:0/2597723403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d0410fc40 msgr2=0x7f9d04112130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:23.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.089+0000 7f9d08a00640 1 --2- 172.21.15.16:0/2597723403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d0410fc40 0x7f9d04112130 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc009900 tx=0x7f9cfc02a3f0 comp rx=0 tx=0).stop 2024-08-31T06:23:23.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.090+0000 7f9d08a00640 1 -- 172.21.15.16:0/2597723403 shutdown_connections 2024-08-31T06:23:23.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.090+0000 7f9d08a00640 1 --2- 172.21.15.16:0/2597723403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d04075f20 0x7f9d0410f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:23.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.090+0000 7f9d08a00640 1 --2- 172.21.15.16:0/2597723403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d0410fc40 0x7f9d04112130 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:23.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.090+0000 7f9d08a00640 1 --2- 172.21.15.16:0/2597723403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d040754f0 0x7f9d04075900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:23.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.090+0000 7f9d08a00640 1 -- 172.21.15.16:0/2597723403 >> 172.21.15.16:0/2597723403 conn(0x7f9d040ff0d0 msgr2=0x7f9d04101520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:23.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.090+0000 7f9d08a00640 1 -- 172.21.15.16:0/2597723403 shutdown_connections 2024-08-31T06:23:23.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.090+0000 7f9d08a00640 1 -- 172.21.15.16:0/2597723403 wait complete. 2024-08-31T06:23:23.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.091+0000 7f9d08a00640 1 Processor -- start 2024-08-31T06:23:23.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.091+0000 7f9d08a00640 1 -- start start 2024-08-31T06:23:23.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.091+0000 7f9d08a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d040754f0 0x7f9d0419c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:23.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.091+0000 7f9d08a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d04075f20 0x7f9d0419c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:23.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.091+0000 7f9d08a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d0410fc40 0x7f9d041a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:23.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.091+0000 7f9d08a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9d041158f0 con 0x7f9d040754f0 2024-08-31T06:23:23.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.091+0000 7f9d08a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9d04115770 con 0x7f9d04075f20 2024-08-31T06:23:23.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.091+0000 7f9d08a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9d04115a70 con 0x7f9d0410fc40 2024-08-31T06:23:23.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.091+0000 7f9d02800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d040754f0 0x7f9d0419c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:23.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.092+0000 7f9d02800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d040754f0 0x7f9d0419c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47604/0 (socket says 172.21.15.16:47604) 2024-08-31T06:23:23.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.092+0000 7f9d02800640 1 -- 172.21.15.16:0/1734539702 learned_addr learned my addr 172.21.15.16:0/1734539702 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:23:23.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.092+0000 7f9cf9e00640 1 --2- 172.21.15.16:0/1734539702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d04075f20 0x7f9d0419c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:23.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.092+0000 7f9d03200640 1 --2- 172.21.15.16:0/1734539702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d0410fc40 0x7f9d041a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:23.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.092+0000 7f9d02800640 1 -- 172.21.15.16:0/1734539702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d0410fc40 msgr2=0x7f9d041a1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:23.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.092+0000 7f9d02800640 1 --2- 172.21.15.16:0/1734539702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d0410fc40 0x7f9d041a1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:23.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.092+0000 7f9d02800640 1 -- 172.21.15.16:0/1734539702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d04075f20 msgr2=0x7f9d0419c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:23.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.092+0000 7f9d02800640 1 --2- 172.21.15.16:0/1734539702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d04075f20 0x7f9d0419c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:23.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.092+0000 7f9d02800640 1 -- 172.21.15.16:0/1734539702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d041a2240 con 0x7f9d040754f0 2024-08-31T06:23:23.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.092+0000 7f9d03200640 1 --2- 172.21.15.16:0/1734539702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d0410fc40 0x7f9d041a1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-31T06:23:23.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.092+0000 7f9d02800640 1 --2- 172.21.15.16:0/1734539702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d040754f0 0x7f9d0419c310 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400ebc0 tx=0x7f9cf400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:23.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.092+0000 7f9cfbe00640 1 -- 172.21.15.16:0/1734539702 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4010630 con 0x7f9d040754f0 2024-08-31T06:23:23.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.093+0000 7f9d08a00640 1 -- 172.21.15.16:0/1734539702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d041a2530 con 0x7f9d040754f0 2024-08-31T06:23:23.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.093+0000 7f9cfbe00640 1 -- 172.21.15.16:0/1734539702 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cf4010c80 con 0x7f9d040754f0 2024-08-31T06:23:23.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.093+0000 7f9cfbe00640 1 -- 172.21.15.16:0/1734539702 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4018b60 con 0x7f9d040754f0 2024-08-31T06:23:23.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.093+0000 7f9d08a00640 1 -- 172.21.15.16:0/1734539702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d041a8cc0 con 0x7f9d040754f0 2024-08-31T06:23:23.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.094+0000 7f9d08a00640 1 -- 172.21.15.16:0/1734539702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d04075900 con 0x7f9d040754f0 2024-08-31T06:23:23.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.096+0000 7f9cfbe00640 1 -- 172.21.15.16:0/1734539702 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f9cf4018d00 con 0x7f9d040754f0 2024-08-31T06:23:23.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.097+0000 7f9cfbe00640 1 --2- 172.21.15.16:0/1734539702 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9ce80b4ca0 0x7f9ce80b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:23.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.098+0000 7f9cfbe00640 1 -- 172.21.15.16:0/1734539702 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8470+0+0 (secure 0 0 0) 0x7f9cf40e60c0 con 0x7f9d040754f0 2024-08-31T06:23:23.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.098+0000 7f9cf9e00640 1 --2- 172.21.15.16:0/1734539702 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9ce80b4ca0 0x7f9ce80b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:23.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.098+0000 7f9cf9e00640 1 --2- 172.21.15.16:0/1734539702 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9ce80b4ca0 0x7f9ce80b71b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f9d04112060 tx=0x7f9cec005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:23.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.100+0000 7f9cfbe00640 1 -- 172.21.15.16:0/1734539702 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f9cf40ace80 con 0x7f9d040754f0 2024-08-31T06:23:23.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:22 smithi155 ceph-mon[21742]: pgmap v870: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.5 KiB/s wr, 7 op/s 2024-08-31T06:23:23.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:22 smithi016 ceph-mon[17194]: pgmap v870: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.5 KiB/s wr, 7 op/s 2024-08-31T06:23:23.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:22 smithi184 ceph-mon[22331]: pgmap v870: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.5 KiB/s wr, 7 op/s 2024-08-31T06:23:23.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.299+0000 7f9d08a00640 1 -- 172.21.15.16:0/1734539702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9d041a8ef0 con 0x7f9d040754f0 2024-08-31T06:23:23.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.300+0000 7f9cfbe00640 1 -- 172.21.15.16:0/1734539702 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+23137 (secure 0 0 0) 0x7f9cf40b53c0 con 0x7f9d040754f0 2024-08-31T06:23:23.303 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:23:23.303 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":103,"fsid":"673e4aa0-675d-11ef-bcd4-c7b262605968","created":"2024-08-31T05:55:09.632536+0000","modified":"2024-08-31T06:23:15.913786+0000","last_up_change":"2024-08-31T06:03:27.893697+0000","last_in_change":"2024-08-31T06:03:05.664945+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-31T05:59:23.642270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-31T06:04:20.871835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-31T06:04:24.119499+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":29,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e349dae-6aa5-4b6d-9107-a945655e11bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6803","nonce":1725037400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6805","nonce":1725037400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6809","nonce":1725037400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6807","nonce":1725037400}]},"public_addr":"172.21.15.16:6803/1725037400","cluster_addr":"172.21.15.16:6805/1725037400","heartbeat_back_addr":"172.21.15.16:6809/1725037400","heartbeat_front_addr":"172.21.15.16:6807/1725037400","state":["exists","up"]},{"osd":1,"uuid":"bd645dc3-93b5-40af-89d8-08ccec22e50d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6801","nonce":4241618440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6803","nonce":4241618440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6807","nonce":4241618440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6805","nonce":4241618440}]},"public_addr":"172.21.15.155:6801/4241618440","cluster_addr":"172.21.15.155:6803/4241618440","heartbeat_back_addr":"172.21.15.155:6807/4241618440","heartbeat_front_addr":"172.21.15.155:6805/4241618440","state":["exists","up"]},{"osd":2,"uuid":"c69c8c19-32f8-400a-86d4-0f4ce240a7de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6801","nonce":2193296422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6803","nonce":2193296422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6807","nonce":2193296422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6805","nonce":2193296422}]},"public_addr":"172.21.15.184:6801/2193296422","cluster_addr":"172.21.15.184:6803/2193296422","heartbeat_back_addr":"172.21.15.184:6807/2193296422","heartbeat_front_addr":"172.21.15.184:6805/2193296422","state":["exists","up"]},{"osd":3,"uuid":"8892e35f-6880-434c-a805-ab8ccbed2bf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6811","nonce":81681815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6813","nonce":81681815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6817","nonce":81681815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6815","nonce":81681815}]},"public_addr":"172.21.15.16:6811/81681815","cluster_addr":"172.21.15.16:6813/81681815","heartbeat_back_addr":"172.21.15.16:6817/81681815","heartbeat_front_addr":"172.21.15.16:6815/81681815","state":["exists","up"]},{"osd":4,"uuid":"d2515e73-88f7-4654-9762-adb8802777ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6809","nonce":3592422019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6811","nonce":3592422019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6815","nonce":3592422019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6813","nonce":3592422019}]},"public_addr":"172.21.15.155:6809/3592422019","cluster_addr":"172.21.15.155:6811/3592422019","heartbeat_back_addr":"172.21.15.155:6815/3592422019","heartbeat_front_addr":"172.21.15.155:6813/3592422019","state":["exists","up"]},{"osd":5,"uuid":"5ce6e2fa-c17c-49b3-9402-f19bfc21076d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6809","nonce":2054894065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6811","nonce":2054894065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6815","nonce":2054894065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6813","nonce":2054894065}]},"public_addr":"172.21.15.184:6809/2054894065","cluster_addr":"172.21.15.184:6811/2054894065","heartbeat_back_addr":"172.21.15.184:6815/2054894065","heartbeat_front_addr":"172.21.15.184:6813/2054894065","state":["exists","up"]},{"osd":6,"uuid":"f257d8d1-ad59-4bd8-821d-bf29e50b547e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6819","nonce":1697421092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6821","nonce":1697421092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6825","nonce":1697421092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6823","nonce":1697421092}]},"public_addr":"172.21.15.16:6819/1697421092","cluster_addr":"172.21.15.16:6821/1697421092","heartbeat_back_addr":"172.21.15.16:6825/1697421092","heartbeat_front_addr":"172.21.15.16:6823/1697421092","state":["exists","up"]},{"osd":7,"uuid":"8e76c20e-8d25-423b-b7e8-de642d9c9a6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6817","nonce":2671827901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6819","nonce":2671827901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6823","nonce":2671827901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6821","nonce":2671827901}]},"public_addr":"172.21.15.155:6817/2671827901","cluster_addr":"172.21.15.155:6819/2671827901","heartbeat_back_addr":"172.21.15.155:6823/2671827901","heartbeat_front_addr":"172.21.15.155:6821/2671827901","state":["exists","up"]},{"osd":8,"uuid":"0f0bb23f-4bc3-4198-8381-5b5cd16ae394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6817","nonce":2479117823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6819","nonce":2479117823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6823","nonce":2479117823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6821","nonce":2479117823}]},"public_addr":"172.21.15.184:6817/2479117823","cluster_addr":"172.21.15.184:6819/2479117823","heartbeat_back_addr":"172.21.15.184:6823/2479117823","heartbeat_front_addr":"172.21.15.184:6821/2479117823","state":["exists","up"]},{"osd":9,"uuid":"bc27afcb-7871-4f72-a043-044fd5fe7d86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6827","nonce":3015700546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6829","nonce":3015700546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6833","nonce":3015700546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6831","nonce":3015700546}]},"public_addr":"172.21.15.16:6827/3015700546","cluster_addr":"172.21.15.16:6829/3015700546","heartbeat_back_addr":"172.21.15.16:6833/3015700546","heartbeat_front_addr":"172.21.15.16:6831/3015700546","state":["exists","up"]},{"osd":10,"uuid":"f2548997-72ee-400e-ab8d-5966d85b2cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6825","nonce":2427122475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6827","nonce":2427122475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6831","nonce":2427122475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6829","nonce":2427122475}]},"public_addr":"172.21.15.155:6825/2427122475","cluster_addr":"172.21.15.155:6827/2427122475","heartbeat_back_addr":"172.21.15.155:6831/2427122475","heartbeat_front_addr":"172.21.15.155:6829/2427122475","state":["exists","up"]},{"osd":11,"uuid":"8297da1d-d3e2-4a4d-9396-426dfafa3e95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6825","nonce":399265701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6827","nonce":399265701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6831","nonce":399265701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6829","nonce":399265701}]},"public_addr":"172.21.15.184:6825/399265701","cluster_addr":"172.21.15.184:6827/399265701","heartbeat_back_addr":"172.21.15.184:6831/399265701","heartbeat_front_addr":"172.21.15.184:6829/399265701","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:58:30.842360+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:58:57.164488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:59:20.778103+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:59:50.123991+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:00:16.789933+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:00:41.788905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:01:12.281376+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:01:39.211858+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:02:04.958771+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:02:35.645947+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:03:01.940433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:03:26.728673+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3923220960":"2024-09-01T05:57:23.153214+0000","172.21.15.16:0/3013533836":"2024-09-01T05:57:23.153214+0000","172.21.15.16:6800/3755683228":"2024-09-01T05:55:45.075902+0000","172.21.15.16:0/3064435470":"2024-09-01T05:55:45.075902+0000","172.21.15.16:0/911709962":"2024-09-01T05:55:45.075902+0000","172.21.15.16:0/2937368967":"2024-09-01T05:55:45.075902+0000","172.21.15.16:6800/1378972230":"2024-09-01T05:56:14.195704+0000","172.21.15.16:0/3047316995":"2024-09-01T05:56:14.195704+0000","172.21.15.16:6801/3755683228":"2024-09-01T05:55:45.075902+0000","172.21.15.16:6801/2571082634":"2024-09-01T05:57:23.153214+0000","172.21.15.16:0/2136704295":"2024-09-01T05:56:14.195704+0000","172.21.15.16:0/2259332778":"2024-09-01T05:56:14.195704+0000","172.21.15.16:6800/2571082634":"2024-09-01T05:57:23.153214+0000","172.21.15.16:6801/1378972230":"2024-09-01T05:56:14.195704+0000","172.21.15.16:0/2126457673":"2024-09-01T05:57:23.153214+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":16,"length":1},{"begin":29,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-31T06:23:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.303+0000 7f9d08a00640 1 -- 172.21.15.16:0/1734539702 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9ce80b4ca0 msgr2=0x7f9ce80b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.303+0000 7f9d08a00640 1 --2- 172.21.15.16:0/1734539702 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9ce80b4ca0 0x7f9ce80b71b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f9d04112060 tx=0x7f9cec005f90 comp rx=0 tx=0).stop 2024-08-31T06:23:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.303+0000 7f9d08a00640 1 -- 172.21.15.16:0/1734539702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d040754f0 msgr2=0x7f9d0419c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.303+0000 7f9d08a00640 1 --2- 172.21.15.16:0/1734539702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d040754f0 0x7f9d0419c310 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400ebc0 tx=0x7f9cf400c6a0 comp rx=0 tx=0).stop 2024-08-31T06:23:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.303+0000 7f9d08a00640 1 -- 172.21.15.16:0/1734539702 shutdown_connections 2024-08-31T06:23:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.303+0000 7f9d08a00640 1 --2- 172.21.15.16:0/1734539702 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f9ce80b4ca0 0x7f9ce80b71b0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.303+0000 7f9d08a00640 1 --2- 172.21.15.16:0/1734539702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d0410fc40 0x7f9d041a1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.303+0000 7f9d08a00640 1 --2- 172.21.15.16:0/1734539702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d04075f20 0x7f9d0419c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.303+0000 7f9d08a00640 1 --2- 172.21.15.16:0/1734539702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d040754f0 0x7f9d0419c310 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.303+0000 7f9d08a00640 1 -- 172.21.15.16:0/1734539702 >> 172.21.15.16:0/1734539702 conn(0x7f9d040ff0d0 msgr2=0x7f9d04109740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.303+0000 7f9d08a00640 1 -- 172.21.15.16:0/1734539702 shutdown_connections 2024-08-31T06:23:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.303+0000 7f9d08a00640 1 -- 172.21.15.16:0/1734539702 wait complete. 2024-08-31T06:23:23.331 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-31T05:59:23.642270+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-31T06:04:20.871835+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-31T06:04:24.119499+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 29, 'snap_epoch': 102, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-08-31T06:23:23.331 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-31T06:23:23.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.556+0000 7fef46400640 1 -- 172.21.15.16:0/1459556227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef4010b1d0 msgr2=0x7fef4010b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:23.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.556+0000 7fef46400640 1 --2- 172.21.15.16:0/1459556227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef4010b1d0 0x7fef4010b5e0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fef2c009900 tx=0x7fef2c02a410 comp rx=0 tx=0).stop 2024-08-31T06:23:23.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.556+0000 7fef46400640 1 -- 172.21.15.16:0/1459556227 shutdown_connections 2024-08-31T06:23:23.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.556+0000 7fef46400640 1 --2- 172.21.15.16:0/1459556227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef4010bb20 0x7fef40112350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:23.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.556+0000 7fef46400640 1 --2- 172.21.15.16:0/1459556227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef4010b1d0 0x7fef4010b5e0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:23.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.556+0000 7fef46400640 1 --2- 172.21.15.16:0/1459556227 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef401058a0 0x7fef40105cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:23.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.556+0000 7fef46400640 1 -- 172.21.15.16:0/1459556227 >> 172.21.15.16:0/1459556227 conn(0x7fef401012c0 msgr2=0x7fef40103710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:23.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.556+0000 7fef46400640 1 -- 172.21.15.16:0/1459556227 shutdown_connections 2024-08-31T06:23:23.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.556+0000 7fef46400640 1 -- 172.21.15.16:0/1459556227 wait complete. 2024-08-31T06:23:23.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.557+0000 7fef46400640 1 Processor -- start 2024-08-31T06:23:23.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.557+0000 7fef46400640 1 -- start start 2024-08-31T06:23:23.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.557+0000 7fef46400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef401058a0 0x7fef4019c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:23.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.557+0000 7fef46400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef4010b1d0 0x7fef4019c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:23.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.557+0000 7fef46400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef4010bb20 0x7fef401a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:23.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.557+0000 7fef46400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fef401157a0 con 0x7fef4010bb20 2024-08-31T06:23:23.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.557+0000 7fef46400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fef40115620 con 0x7fef4010b1d0 2024-08-31T06:23:23.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.557+0000 7fef46400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fef40115920 con 0x7fef401058a0 2024-08-31T06:23:23.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.558+0000 7fef45a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef4010bb20 0x7fef401a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:23.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.558+0000 7fef45000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef401058a0 0x7fef4019c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:23.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.558+0000 7fef45a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef4010bb20 0x7fef401a1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47630/0 (socket says 172.21.15.16:47630) 2024-08-31T06:23:23.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.558+0000 7fef45a00640 1 -- 172.21.15.16:0/877240064 learned_addr learned my addr 172.21.15.16:0/877240064 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:23:23.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.558+0000 7fef3fe00640 1 --2- 172.21.15.16:0/877240064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef4010b1d0 0x7fef4019c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:23.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.558+0000 7fef45000640 1 -- 172.21.15.16:0/877240064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef4010b1d0 msgr2=0x7fef4019c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:23.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.558+0000 7fef45000640 1 --2- 172.21.15.16:0/877240064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef4010b1d0 0x7fef4019c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:23.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.558+0000 7fef45000640 1 -- 172.21.15.16:0/877240064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef4010bb20 msgr2=0x7fef401a1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:23.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.558+0000 7fef45000640 1 --2- 172.21.15.16:0/877240064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef4010bb20 0x7fef401a1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:23.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.558+0000 7fef45000640 1 -- 172.21.15.16:0/877240064 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef401a2110 con 0x7fef401058a0 2024-08-31T06:23:23.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.558+0000 7fef45a00640 1 --2- 172.21.15.16:0/877240064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef4010bb20 0x7fef401a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:23:23.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.558+0000 7fef45000640 1 --2- 172.21.15.16:0/877240064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef401058a0 0x7fef4019c1e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fef3400e9f0 tx=0x7fef3400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:23.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.559+0000 7fef3d600640 1 -- 172.21.15.16:0/877240064 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3400ccc0 con 0x7fef401058a0 2024-08-31T06:23:23.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.559+0000 7fef46400640 1 -- 172.21.15.16:0/877240064 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef401a2400 con 0x7fef401058a0 2024-08-31T06:23:23.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.559+0000 7fef3d600640 1 -- 172.21.15.16:0/877240064 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef34005020 con 0x7fef401058a0 2024-08-31T06:23:23.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.559+0000 7fef3d600640 1 -- 172.21.15.16:0/877240064 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef34004a00 con 0x7fef401058a0 2024-08-31T06:23:23.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.559+0000 7fef46400640 1 -- 172.21.15.16:0/877240064 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef401a8b90 con 0x7fef401058a0 2024-08-31T06:23:23.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.560+0000 7fef46400640 1 -- 172.21.15.16:0/877240064 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef40105cb0 con 0x7fef401058a0 2024-08-31T06:23:23.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.563+0000 7fef3d600640 1 -- 172.21.15.16:0/877240064 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fef34004ba0 con 0x7fef401058a0 2024-08-31T06:23:23.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.564+0000 7fef3d600640 1 --2- 172.21.15.16:0/877240064 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fef280b4ca0 0x7fef280b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:23.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.564+0000 7fef3d600640 1 -- 172.21.15.16:0/877240064 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8470+0+0 (secure 0 0 0) 0x7fef340e55d0 con 0x7fef401058a0 2024-08-31T06:23:23.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.564+0000 7fef3fe00640 1 --2- 172.21.15.16:0/877240064 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fef280b4ca0 0x7fef280b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:23.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.565+0000 7fef3fe00640 1 --2- 172.21.15.16:0/877240064 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fef280b4ca0 0x7fef280b71b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fef2c009230 tx=0x7fef2c031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:23.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.566+0000 7fef3d600640 1 -- 172.21.15.16:0/877240064 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fef340ac420 con 0x7fef401058a0 2024-08-31T06:23:23.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.762+0000 7fef46400640 1 -- 172.21.15.16:0/877240064 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fef4002e430 con 0x7fef401058a0 2024-08-31T06:23:23.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.763+0000 7fef3d600640 1 -- 172.21.15.16:0/877240064 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) ==== 93+0+10 (secure 0 0 0) 0x7fef340abd20 con 0x7fef401058a0 2024-08-31T06:23:23.765 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-08-31T06:23:23.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.765+0000 7fef46400640 1 -- 172.21.15.16:0/877240064 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fef280b4ca0 msgr2=0x7fef280b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:23.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.766+0000 7fef46400640 1 --2- 172.21.15.16:0/877240064 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fef280b4ca0 0x7fef280b71b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fef2c009230 tx=0x7fef2c031400 comp rx=0 tx=0).stop 2024-08-31T06:23:23.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.766+0000 7fef46400640 1 -- 172.21.15.16:0/877240064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef401058a0 msgr2=0x7fef4019c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:23.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.766+0000 7fef46400640 1 --2- 172.21.15.16:0/877240064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef401058a0 0x7fef4019c1e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fef3400e9f0 tx=0x7fef3400eeb0 comp rx=0 tx=0).stop 2024-08-31T06:23:23.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.766+0000 7fef46400640 1 -- 172.21.15.16:0/877240064 shutdown_connections 2024-08-31T06:23:23.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.766+0000 7fef46400640 1 --2- 172.21.15.16:0/877240064 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fef280b4ca0 0x7fef280b71b0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:23.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.766+0000 7fef46400640 1 --2- 172.21.15.16:0/877240064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef4010bb20 0x7fef401a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:23.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.766+0000 7fef46400640 1 --2- 172.21.15.16:0/877240064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef4010b1d0 0x7fef4019c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:23.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.766+0000 7fef46400640 1 --2- 172.21.15.16:0/877240064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef401058a0 0x7fef4019c1e0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:23.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.766+0000 7fef46400640 1 -- 172.21.15.16:0/877240064 >> 172.21.15.16:0/877240064 conn(0x7fef401012c0 msgr2=0x7fef401036e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:23.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.766+0000 7fef46400640 1 -- 172.21.15.16:0/877240064 shutdown_connections 2024-08-31T06:23:23.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:23.766+0000 7fef46400640 1 -- 172.21.15.16:0/877240064 wait complete. 2024-08-31T06:23:23.794 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-31T06:23:24.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.018+0000 7f886da00640 1 -- 172.21.15.16:0/2870926888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8868103850 msgr2=0x7f8868103c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:24.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.018+0000 7f886da00640 1 --2- 172.21.15.16:0/2870926888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8868103850 0x7f8868103c60 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f8858009900 tx=0x7f885802a410 comp rx=0 tx=0).stop 2024-08-31T06:23:24.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.018+0000 7f886da00640 1 -- 172.21.15.16:0/2870926888 shutdown_connections 2024-08-31T06:23:24.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.018+0000 7f886da00640 1 --2- 172.21.15.16:0/2870926888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8868109180 0x7f8868109590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:24.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.018+0000 7f886da00640 1 --2- 172.21.15.16:0/2870926888 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8868109ad0 0x7f8868110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:24.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.018+0000 7f886da00640 1 --2- 172.21.15.16:0/2870926888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8868103850 0x7f8868103c60 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:24.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.018+0000 7f886da00640 1 -- 172.21.15.16:0/2870926888 >> 172.21.15.16:0/2870926888 conn(0x7f88680ff270 msgr2=0x7f88681016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:24.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.018+0000 7f886da00640 1 -- 172.21.15.16:0/2870926888 shutdown_connections 2024-08-31T06:23:24.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.018+0000 7f886da00640 1 -- 172.21.15.16:0/2870926888 wait complete. 2024-08-31T06:23:24.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.019+0000 7f886da00640 1 Processor -- start 2024-08-31T06:23:24.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.020+0000 7f886da00640 1 -- start start 2024-08-31T06:23:24.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.020+0000 7f886da00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8868103850 0x7f886819c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:24.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.020+0000 7f886da00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8868109180 0x7f886819c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:24.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.020+0000 7f886da00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8868109ad0 0x7f88681a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:24.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.020+0000 7f886da00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f88681137f0 con 0x7f8868109180 2024-08-31T06:23:24.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.020+0000 7f886da00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8868113670 con 0x7f8868109ad0 2024-08-31T06:23:24.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.020+0000 7f886da00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8868113970 con 0x7f8868103850 2024-08-31T06:23:24.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.020+0000 7f8866000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8868109180 0x7f886819c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:24.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.020+0000 7f8866000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8868109180 0x7f886819c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47658/0 (socket says 172.21.15.16:47658) 2024-08-31T06:23:24.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.020+0000 7f8867400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8868109ad0 0x7f88681a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:24.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.020+0000 7f8866000640 1 -- 172.21.15.16:0/2492759279 learned_addr learned my addr 172.21.15.16:0/2492759279 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:23:24.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.020+0000 7f8866a00640 1 --2- 172.21.15.16:0/2492759279 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8868103850 0x7f886819c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:24.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.021+0000 7f8867400640 1 -- 172.21.15.16:0/2492759279 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8868103850 msgr2=0x7f886819c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:24.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.021+0000 7f8867400640 1 --2- 172.21.15.16:0/2492759279 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8868103850 0x7f886819c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:24.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.021+0000 7f8867400640 1 -- 172.21.15.16:0/2492759279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8868109180 msgr2=0x7f886819c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:24.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.021+0000 7f8867400640 1 --2- 172.21.15.16:0/2492759279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8868109180 0x7f886819c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:24.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.021+0000 7f8867400640 1 -- 172.21.15.16:0/2492759279 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88681a2300 con 0x7f8868109ad0 2024-08-31T06:23:24.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.021+0000 7f8866000640 1 --2- 172.21.15.16:0/2492759279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8868109180 0x7f886819c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:23:24.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.021+0000 7f8866a00640 1 --2- 172.21.15.16:0/2492759279 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8868103850 0x7f886819c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:23:24.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.021+0000 7f8867400640 1 --2- 172.21.15.16:0/2492759279 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8868109ad0 0x7f88681a1c00 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f8860009fd0 tx=0x7f886000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:24.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.021+0000 7f885f400640 1 -- 172.21.15.16:0/2492759279 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886000ce20 con 0x7f8868109ad0 2024-08-31T06:23:24.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.022+0000 7f886da00640 1 -- 172.21.15.16:0/2492759279 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88681a25f0 con 0x7f8868109ad0 2024-08-31T06:23:24.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.022+0000 7f885f400640 1 -- 172.21.15.16:0/2492759279 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8860005020 con 0x7f8868109ad0 2024-08-31T06:23:24.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.022+0000 7f885f400640 1 -- 172.21.15.16:0/2492759279 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8860010690 con 0x7f8868109ad0 2024-08-31T06:23:24.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.022+0000 7f886da00640 1 -- 172.21.15.16:0/2492759279 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88681a8d80 con 0x7f8868109ad0 2024-08-31T06:23:24.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.023+0000 7f886da00640 1 -- 172.21.15.16:0/2492759279 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8868103c60 con 0x7f8868109ad0 2024-08-31T06:23:24.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.025+0000 7f885f400640 1 -- 172.21.15.16:0/2492759279 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f88600054d0 con 0x7f8868109ad0 2024-08-31T06:23:24.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.027+0000 7f885f400640 1 --2- 172.21.15.16:0/2492759279 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f884c0b4d20 0x7f884c0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:24.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.027+0000 7f885f400640 1 -- 172.21.15.16:0/2492759279 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8470+0+0 (secure 0 0 0) 0x7f88600e5b80 con 0x7f8868109ad0 2024-08-31T06:23:24.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.027+0000 7f8866a00640 1 --2- 172.21.15.16:0/2492759279 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f884c0b4d20 0x7f884c0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:24.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.028+0000 7f8866a00640 1 --2- 172.21.15.16:0/2492759279 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f884c0b4d20 0x7f884c0b7230 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f8858005ab0 tx=0x7f8858005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:24.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.029+0000 7f885f400640 1 -- 172.21.15.16:0/2492759279 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f88600ac940 con 0x7f8868109ad0 2024-08-31T06:23:24.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:23 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/1734539702' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T06:23:24.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:23 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/877240064' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-31T06:23:24.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:23 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/1734539702' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T06:23:24.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:23 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/877240064' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-31T06:23:24.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:23 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/1734539702' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T06:23:24.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:23 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/877240064' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-31T06:23:24.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.225+0000 7f886da00640 1 -- 172.21.15.16:0/2492759279 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f88681a9040 con 0x7f8868109ad0 2024-08-31T06:23:24.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.226+0000 7f885f400640 1 -- 172.21.15.16:0/2492759279 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v103) ==== 104+0+11 (secure 0 0 0) 0x7f88600ac240 con 0x7f8868109ad0 2024-08-31T06:23:24.229 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-08-31T06:23:24.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.228+0000 7f886da00640 1 -- 172.21.15.16:0/2492759279 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f884c0b4d20 msgr2=0x7f884c0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:24.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.228+0000 7f886da00640 1 --2- 172.21.15.16:0/2492759279 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f884c0b4d20 0x7f884c0b7230 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f8858005ab0 tx=0x7f8858005a20 comp rx=0 tx=0).stop 2024-08-31T06:23:24.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.228+0000 7f886da00640 1 -- 172.21.15.16:0/2492759279 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8868109ad0 msgr2=0x7f88681a1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:24.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.228+0000 7f886da00640 1 --2- 172.21.15.16:0/2492759279 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8868109ad0 0x7f88681a1c00 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f8860009fd0 tx=0x7f886000ef10 comp rx=0 tx=0).stop 2024-08-31T06:23:24.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.228+0000 7f886da00640 1 -- 172.21.15.16:0/2492759279 shutdown_connections 2024-08-31T06:23:24.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.228+0000 7f886da00640 1 --2- 172.21.15.16:0/2492759279 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f884c0b4d20 0x7f884c0b7230 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:24.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.228+0000 7f886da00640 1 --2- 172.21.15.16:0/2492759279 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8868109ad0 0x7f88681a1c00 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:24.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.228+0000 7f886da00640 1 --2- 172.21.15.16:0/2492759279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8868109180 0x7f886819c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:24.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.228+0000 7f886da00640 1 --2- 172.21.15.16:0/2492759279 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8868103850 0x7f886819c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:24.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.228+0000 7f886da00640 1 -- 172.21.15.16:0/2492759279 >> 172.21.15.16:0/2492759279 conn(0x7f88680ff270 msgr2=0x7f8868108830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:24.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.228+0000 7f886da00640 1 -- 172.21.15.16:0/2492759279 shutdown_connections 2024-08-31T06:23:24.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.228+0000 7f886da00640 1 -- 172.21.15.16:0/2492759279 wait complete. 2024-08-31T06:23:24.253 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-31T06:23:24.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.488+0000 7fdd2ca00640 1 -- 172.21.15.16:0/2185299995 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd2810bb20 msgr2=0x7fdd28112350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:24.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.488+0000 7fdd2ca00640 1 --2- 172.21.15.16:0/2185299995 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd2810bb20 0x7fdd28112350 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fdd20009be0 tx=0x7fdd2002a6d0 comp rx=0 tx=0).stop 2024-08-31T06:23:24.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.488+0000 7fdd2ca00640 1 -- 172.21.15.16:0/2185299995 shutdown_connections 2024-08-31T06:23:24.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.488+0000 7fdd2ca00640 1 --2- 172.21.15.16:0/2185299995 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd2810bb20 0x7fdd28112350 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:24.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.488+0000 7fdd2ca00640 1 --2- 172.21.15.16:0/2185299995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd2810b1d0 0x7fdd2810b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:24.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.488+0000 7fdd2ca00640 1 --2- 172.21.15.16:0/2185299995 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd281058a0 0x7fdd28105cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:24.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.488+0000 7fdd2ca00640 1 -- 172.21.15.16:0/2185299995 >> 172.21.15.16:0/2185299995 conn(0x7fdd281012c0 msgr2=0x7fdd28103710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:24.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.488+0000 7fdd2ca00640 1 -- 172.21.15.16:0/2185299995 shutdown_connections 2024-08-31T06:23:24.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.488+0000 7fdd2ca00640 1 -- 172.21.15.16:0/2185299995 wait complete. 2024-08-31T06:23:24.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.489+0000 7fdd2ca00640 1 Processor -- start 2024-08-31T06:23:24.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.489+0000 7fdd2ca00640 1 -- start start 2024-08-31T06:23:24.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.490+0000 7fdd2ca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd281058a0 0x7fdd2819c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:24.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.490+0000 7fdd2ca00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd2810b1d0 0x7fdd2819c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:24.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.490+0000 7fdd2ca00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd2810bb20 0x7fdd281a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:24.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.490+0000 7fdd2ca00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdd281157d0 con 0x7fdd281058a0 2024-08-31T06:23:24.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.490+0000 7fdd2ca00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdd28115650 con 0x7fdd2810b1d0 2024-08-31T06:23:24.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.490+0000 7fdd2ca00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdd28115950 con 0x7fdd2810bb20 2024-08-31T06:23:24.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.490+0000 7fdd26800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd281058a0 0x7fdd2819c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:24.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.490+0000 7fdd26800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd281058a0 0x7fdd2819c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47670/0 (socket says 172.21.15.16:47670) 2024-08-31T06:23:24.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.490+0000 7fdd26800640 1 -- 172.21.15.16:0/3227927684 learned_addr learned my addr 172.21.15.16:0/3227927684 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:23:24.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.490+0000 7fdd1fe00640 1 --2- 172.21.15.16:0/3227927684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd2810b1d0 0x7fdd2819c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:24.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.490+0000 7fdd27200640 1 --2- 172.21.15.16:0/3227927684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd2810bb20 0x7fdd281a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:24.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.491+0000 7fdd26800640 1 -- 172.21.15.16:0/3227927684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd2810bb20 msgr2=0x7fdd281a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:24.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.491+0000 7fdd26800640 1 --2- 172.21.15.16:0/3227927684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd2810bb20 0x7fdd281a19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:24.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.491+0000 7fdd26800640 1 -- 172.21.15.16:0/3227927684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd2810b1d0 msgr2=0x7fdd2819c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:24.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.491+0000 7fdd26800640 1 --2- 172.21.15.16:0/3227927684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd2810b1d0 0x7fdd2819c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:24.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.491+0000 7fdd26800640 1 -- 172.21.15.16:0/3227927684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd281a20e0 con 0x7fdd281058a0 2024-08-31T06:23:24.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.491+0000 7fdd26800640 1 --2- 172.21.15.16:0/3227927684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd281058a0 0x7fdd2819c1b0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fdd1800e9f0 tx=0x7fdd1800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:24.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.491+0000 7fdd1f400640 1 -- 172.21.15.16:0/3227927684 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1800cd80 con 0x7fdd281058a0 2024-08-31T06:23:24.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.491+0000 7fdd2ca00640 1 -- 172.21.15.16:0/3227927684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd281a23d0 con 0x7fdd281058a0 2024-08-31T06:23:24.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.491+0000 7fdd1f400640 1 -- 172.21.15.16:0/3227927684 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd180108e0 con 0x7fdd281058a0 2024-08-31T06:23:24.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.491+0000 7fdd1f400640 1 -- 172.21.15.16:0/3227927684 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd18018980 con 0x7fdd281058a0 2024-08-31T06:23:24.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.491+0000 7fdd2ca00640 1 -- 172.21.15.16:0/3227927684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd281a8b60 con 0x7fdd281058a0 2024-08-31T06:23:24.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.492+0000 7fdd2ca00640 1 -- 172.21.15.16:0/3227927684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd28105cb0 con 0x7fdd281058a0 2024-08-31T06:23:24.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.495+0000 7fdd1f400640 1 -- 172.21.15.16:0/3227927684 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fdd18010430 con 0x7fdd281058a0 2024-08-31T06:23:24.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.496+0000 7fdd1f400640 1 --2- 172.21.15.16:0/3227927684 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdd0c0b4d20 0x7fdd0c0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:24.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.496+0000 7fdd1f400640 1 -- 172.21.15.16:0/3227927684 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8470+0+0 (secure 0 0 0) 0x7fdd180e5cc0 con 0x7fdd281058a0 2024-08-31T06:23:24.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.496+0000 7fdd1fe00640 1 --2- 172.21.15.16:0/3227927684 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdd0c0b4d20 0x7fdd0c0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:24.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.497+0000 7fdd1fe00640 1 --2- 172.21.15.16:0/3227927684 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdd0c0b4d20 0x7fdd0c0b7230 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fdd2810acf0 tx=0x7fdd10009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:24.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.498+0000 7fdd1f400640 1 -- 172.21.15.16:0/3227927684 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fdd180ac430 con 0x7fdd281058a0 2024-08-31T06:23:24.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.696+0000 7fdd2ca00640 1 -- 172.21.15.16:0/3227927684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdd2802e430 con 0x7fdd281058a0 2024-08-31T06:23:24.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.697+0000 7fdd1f400640 1 -- 172.21.15.16:0/3227927684 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v103) ==== 100+0+11 (secure 0 0 0) 0x7fdd180b0bf0 con 0x7fdd281058a0 2024-08-31T06:23:24.700 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-08-31T06:23:24.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.699+0000 7fdd2ca00640 1 -- 172.21.15.16:0/3227927684 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdd0c0b4d20 msgr2=0x7fdd0c0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:24.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.699+0000 7fdd2ca00640 1 --2- 172.21.15.16:0/3227927684 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdd0c0b4d20 0x7fdd0c0b7230 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fdd2810acf0 tx=0x7fdd10009260 comp rx=0 tx=0).stop 2024-08-31T06:23:24.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.699+0000 7fdd2ca00640 1 -- 172.21.15.16:0/3227927684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd281058a0 msgr2=0x7fdd2819c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:24.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.699+0000 7fdd2ca00640 1 --2- 172.21.15.16:0/3227927684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd281058a0 0x7fdd2819c1b0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fdd1800e9f0 tx=0x7fdd1800eeb0 comp rx=0 tx=0).stop 2024-08-31T06:23:24.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.699+0000 7fdd2ca00640 1 -- 172.21.15.16:0/3227927684 shutdown_connections 2024-08-31T06:23:24.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.699+0000 7fdd2ca00640 1 --2- 172.21.15.16:0/3227927684 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fdd0c0b4d20 0x7fdd0c0b7230 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:24.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.699+0000 7fdd2ca00640 1 --2- 172.21.15.16:0/3227927684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd2810bb20 0x7fdd281a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:24.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.699+0000 7fdd2ca00640 1 --2- 172.21.15.16:0/3227927684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd2810b1d0 0x7fdd2819c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:24.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.699+0000 7fdd2ca00640 1 --2- 172.21.15.16:0/3227927684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd281058a0 0x7fdd2819c1b0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:24.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.699+0000 7fdd2ca00640 1 -- 172.21.15.16:0/3227927684 >> 172.21.15.16:0/3227927684 conn(0x7fdd281012c0 msgr2=0x7fdd281036e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:24.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.699+0000 7fdd2ca00640 1 -- 172.21.15.16:0/3227927684 shutdown_connections 2024-08-31T06:23:24.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.700+0000 7fdd2ca00640 1 -- 172.21.15.16:0/3227927684 wait complete. 2024-08-31T06:23:24.726 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-08-31T06:23:24.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.949+0000 7faf02a00640 1 -- 172.21.15.16:0/2004471527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faefc1058a0 msgr2=0x7faefc105cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:24.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.949+0000 7faf02a00640 1 --2- 172.21.15.16:0/2004471527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faefc1058a0 0x7faefc105cb0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7faef0009900 tx=0x7faef002a3f0 comp rx=0 tx=0).stop 2024-08-31T06:23:24.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.949+0000 7faf02a00640 1 -- 172.21.15.16:0/2004471527 shutdown_connections 2024-08-31T06:23:24.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.949+0000 7faf02a00640 1 --2- 172.21.15.16:0/2004471527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faefc10bb20 0x7faefc112350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:24.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.949+0000 7faf02a00640 1 --2- 172.21.15.16:0/2004471527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faefc10b1d0 0x7faefc10b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:24.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.949+0000 7faf02a00640 1 --2- 172.21.15.16:0/2004471527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faefc1058a0 0x7faefc105cb0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:24.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.949+0000 7faf02a00640 1 -- 172.21.15.16:0/2004471527 >> 172.21.15.16:0/2004471527 conn(0x7faefc1012c0 msgr2=0x7faefc103710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:24.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.949+0000 7faf02a00640 1 -- 172.21.15.16:0/2004471527 shutdown_connections 2024-08-31T06:23:24.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.949+0000 7faf02a00640 1 -- 172.21.15.16:0/2004471527 wait complete. 2024-08-31T06:23:24.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.950+0000 7faf02a00640 1 Processor -- start 2024-08-31T06:23:24.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.950+0000 7faf02a00640 1 -- start start 2024-08-31T06:23:24.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.951+0000 7faf02a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faefc1058a0 0x7faefc19c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:24.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.951+0000 7faf02a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faefc10b1d0 0x7faefc19c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:24.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.951+0000 7faf02a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faefc10bb20 0x7faefc1a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:24.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.951+0000 7faf02a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faefc1157e0 con 0x7faefc10bb20 2024-08-31T06:23:24.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.951+0000 7faf02a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faefc115660 con 0x7faefc10b1d0 2024-08-31T06:23:24.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.951+0000 7faf02a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faefc115960 con 0x7faefc1058a0 2024-08-31T06:23:24.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.951+0000 7faf02000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faefc10bb20 0x7faefc1a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:24.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.951+0000 7faf00c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faefc10b1d0 0x7faefc19c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:24.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.951+0000 7faf00c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faefc10b1d0 0x7faefc19c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:49782/0 (socket says 172.21.15.16:49782) 2024-08-31T06:23:24.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.951+0000 7faf00c00640 1 -- 172.21.15.16:0/3699661806 learned_addr learned my addr 172.21.15.16:0/3699661806 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:23:24.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.951+0000 7faf02000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faefc10bb20 0x7faefc1a19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47690/0 (socket says 172.21.15.16:47690) 2024-08-31T06:23:24.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.951+0000 7faf01600640 1 --2- 172.21.15.16:0/3699661806 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faefc1058a0 0x7faefc19c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:24.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.951+0000 7faf00c00640 1 -- 172.21.15.16:0/3699661806 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faefc1058a0 msgr2=0x7faefc19c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:24.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.951+0000 7faf00c00640 1 --2- 172.21.15.16:0/3699661806 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faefc1058a0 0x7faefc19c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:24.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.951+0000 7faf00c00640 1 -- 172.21.15.16:0/3699661806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faefc10bb20 msgr2=0x7faefc1a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:24.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.951+0000 7faf00c00640 1 --2- 172.21.15.16:0/3699661806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faefc10bb20 0x7faefc1a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:24.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.952+0000 7faf00c00640 1 -- 172.21.15.16:0/3699661806 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faefc1a20d0 con 0x7faefc10b1d0 2024-08-31T06:23:24.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.952+0000 7faf02000640 1 --2- 172.21.15.16:0/3699661806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faefc10bb20 0x7faefc1a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:23:24.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.952+0000 7faf00c00640 1 --2- 172.21.15.16:0/3699661806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faefc10b1d0 0x7faefc19c740 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7faee800e9f0 tx=0x7faee800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:24.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.952+0000 7faefa000640 1 -- 172.21.15.16:0/3699661806 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee800ccc0 con 0x7faefc10b1d0 2024-08-31T06:23:24.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.952+0000 7faefa000640 1 -- 172.21.15.16:0/3699661806 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faee80108e0 con 0x7faefc10b1d0 2024-08-31T06:23:24.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.952+0000 7faf02a00640 1 -- 172.21.15.16:0/3699661806 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faefc1a23c0 con 0x7faefc10b1d0 2024-08-31T06:23:24.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.952+0000 7faefa000640 1 -- 172.21.15.16:0/3699661806 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee8018860 con 0x7faefc10b1d0 2024-08-31T06:23:24.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.952+0000 7faf02a00640 1 -- 172.21.15.16:0/3699661806 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faefc1a8b50 con 0x7faefc10b1d0 2024-08-31T06:23:24.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.953+0000 7faf02a00640 1 -- 172.21.15.16:0/3699661806 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faefc105cb0 con 0x7faefc10b1d0 2024-08-31T06:23:24.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.956+0000 7faefa000640 1 -- 172.21.15.16:0/3699661806 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7faee8010430 con 0x7faefc10b1d0 2024-08-31T06:23:24.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.958+0000 7faefa000640 1 --2- 172.21.15.16:0/3699661806 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7faee40b4ca0 0x7faee40b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:24.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.958+0000 7faefa000640 1 -- 172.21.15.16:0/3699661806 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8470+0+0 (secure 0 0 0) 0x7faee80e5cf0 con 0x7faefc10b1d0 2024-08-31T06:23:24.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.958+0000 7faf01600640 1 --2- 172.21.15.16:0/3699661806 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7faee40b4ca0 0x7faee40b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:24.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.959+0000 7faf01600640 1 --2- 172.21.15.16:0/3699661806 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7faee40b4ca0 0x7faee40b71b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7faef0031440 tx=0x7faef0009560 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:24.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:24.959+0000 7faefa000640 1 -- 172.21.15.16:0/3699661806 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7faee80ac460 con 0x7faefc10b1d0 2024-08-31T06:23:25.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:24 smithi155 ceph-mon[21742]: pgmap v871: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.0 KiB/s wr, 7 op/s 2024-08-31T06:23:25.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:24 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/2492759279' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-08-31T06:23:25.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:24 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/3227927684' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-08-31T06:23:25.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:24 smithi016 ceph-mon[17194]: pgmap v871: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.0 KiB/s wr, 7 op/s 2024-08-31T06:23:25.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:24 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/2492759279' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-08-31T06:23:25.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:24 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/3227927684' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-08-31T06:23:25.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:25.207+0000 7faf02a00640 1 -- 172.21.15.16:0/3699661806 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7faefc19d2c0 con 0x7faefc10b1d0 2024-08-31T06:23:25.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:25.216+0000 7faefa000640 1 -- 172.21.15.16:0/3699661806 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v24) ==== 81+0+826324 (secure 0 0 0) 0x7faefc19d2c0 con 0x7faefc10b1d0 2024-08-31T06:23:25.227 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:24 smithi184 ceph-mon[22331]: pgmap v871: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.0 KiB/s wr, 7 op/s 2024-08-31T06:23:25.227 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:24 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/2492759279' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-08-31T06:23:25.227 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:24 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/3227927684' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-08-31T06:23:25.231 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:23:25.231 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:23:25.231 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 24, 2024-08-31T06:23:25.231 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.231 INFO:teuthology.orchestra.run.smithi016.stdout: "active_gid": 14220, 2024-08-31T06:23:25.231 INFO:teuthology.orchestra.run.smithi016.stdout: "active_name": "x", 2024-08-31T06:23:25.231 INFO:teuthology.orchestra.run.smithi016.stdout: "active_addrs": { 2024-08-31T06:23:25.232 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-08-31T06:23:25.232 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.232 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-08-31T06:23:25.232 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6800", 2024-08-31T06:23:25.232 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 4279751337 2024-08-31T06:23:25.232 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.232 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.232 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-08-31T06:23:25.232 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6801", 2024-08-31T06:23:25.233 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 4279751337 2024-08-31T06:23:25.233 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.233 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-08-31T06:23:25.233 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.233 INFO:teuthology.orchestra.run.smithi016.stdout: "active_addr": "172.21.15.16:6801/4279751337", 2024-08-31T06:23:25.233 INFO:teuthology.orchestra.run.smithi016.stdout: "active_change": "2024-08-31T05:57:23.153395+0000", 2024-08-31T06:23:25.233 INFO:teuthology.orchestra.run.smithi016.stdout: "active_mgr_features": 4540701547738038271, 2024-08-31T06:23:25.233 INFO:teuthology.orchestra.run.smithi016.stdout: "available": true, 2024-08-31T06:23:25.233 INFO:teuthology.orchestra.run.smithi016.stdout: "standbys": [ 2024-08-31T06:23:25.233 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.233 INFO:teuthology.orchestra.run.smithi016.stdout: "gid": 24166, 2024-08-31T06:23:25.233 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "z", 2024-08-31T06:23:25.233 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr_features": 4540701547738038271, 2024-08-31T06:23:25.233 INFO:teuthology.orchestra.run.smithi016.stdout: "available_modules": [ 2024-08-31T06:23:25.234 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.234 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "alerts", 2024-08-31T06:23:25.234 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.234 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.234 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.234 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-08-31T06:23:25.234 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-08-31T06:23:25.234 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:25.234 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.234 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.234 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-08-31T06:23:25.234 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.234 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.234 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.234 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How frequently to reexamine health status", 2024-08-31T06:23:25.235 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.235 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.235 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.235 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.235 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.235 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.235 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.235 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.235 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.235 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.235 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.235 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.235 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.235 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.236 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.236 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.236 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.236 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.236 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.236 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.236 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.236 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.236 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.236 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.236 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.236 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.236 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.236 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.237 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.237 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.237 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.237 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.237 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.237 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.237 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.237 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.237 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.237 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.237 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.237 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.237 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.237 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.238 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.238 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.238 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.238 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.238 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.238 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.238 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.238 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.238 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.238 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.238 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.238 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.238 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.238 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.239 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.239 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.239 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.239 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.239 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.239 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.239 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.239 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.239 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.239 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.239 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.239 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.239 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.239 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.240 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.240 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.240 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.240 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.240 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_destination": { 2024-08-31T06:23:25.240 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_destination", 2024-08-31T06:23:25.240 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.240 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.240 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.240 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.240 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.240 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.240 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.240 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email address to send alerts to", 2024-08-31T06:23:25.241 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.241 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.241 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.241 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.241 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_from_name": { 2024-08-31T06:23:25.241 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_from_name", 2024-08-31T06:23:25.241 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.241 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.241 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.241 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "Ceph", 2024-08-31T06:23:25.241 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.241 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.241 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.241 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email From: name", 2024-08-31T06:23:25.241 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.242 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.242 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.242 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.242 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_host": { 2024-08-31T06:23:25.242 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_host", 2024-08-31T06:23:25.242 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.242 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.242 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.242 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.242 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.242 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.242 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.242 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP server", 2024-08-31T06:23:25.242 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.243 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.243 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.243 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.243 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_password": { 2024-08-31T06:23:25.243 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_password", 2024-08-31T06:23:25.243 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.243 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.243 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.243 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.243 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.243 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.243 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.243 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Password to authenticate with", 2024-08-31T06:23:25.243 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.243 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.244 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.244 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.244 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_port": { 2024-08-31T06:23:25.244 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_port", 2024-08-31T06:23:25.244 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.244 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.244 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.244 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "465", 2024-08-31T06:23:25.244 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.244 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.244 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.244 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP port", 2024-08-31T06:23:25.244 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.244 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.245 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.245 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.245 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_sender": { 2024-08-31T06:23:25.245 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_sender", 2024-08-31T06:23:25.245 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.245 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.245 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.245 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.245 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.245 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.245 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.245 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP envelope sender", 2024-08-31T06:23:25.245 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.245 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.246 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.246 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.246 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_ssl": { 2024-08-31T06:23:25.246 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_ssl", 2024-08-31T06:23:25.246 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.246 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.246 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.246 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.246 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.246 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.246 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.246 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use SSL to connect to SMTP server", 2024-08-31T06:23:25.246 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.247 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.247 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.247 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.247 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_user": { 2024-08-31T06:23:25.247 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_user", 2024-08-31T06:23:25.247 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.247 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.247 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.247 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.247 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.247 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.247 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.247 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "User to authenticate as", 2024-08-31T06:23:25.247 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.247 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.248 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.248 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.248 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.248 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.248 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.248 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.248 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.248 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.248 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.248 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.248 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.248 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.248 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.249 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.249 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.249 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.249 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.249 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.249 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.249 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "balancer", 2024-08-31T06:23:25.249 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.249 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.249 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.249 INFO:teuthology.orchestra.run.smithi016.stdout: "active": { 2024-08-31T06:23:25.249 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "active", 2024-08-31T06:23:25.249 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.249 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.250 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.250 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.250 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.250 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.250 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.250 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically balance PGs across cluster", 2024-08-31T06:23:25.250 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.250 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.250 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.250 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.250 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_time": { 2024-08-31T06:23:25.251 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_time", 2024-08-31T06:23:25.251 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.251 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.251 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.251 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0000", 2024-08-31T06:23:25.251 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.251 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.251 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.251 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "beginning time of day to automatically balance", 2024-08-31T06:23:25.251 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-31T06:23:25.251 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.251 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.251 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.252 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_weekday": { 2024-08-31T06:23:25.252 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_weekday", 2024-08-31T06:23:25.252 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-08-31T06:23:25.252 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.252 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.252 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.252 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-08-31T06:23:25.252 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-08-31T06:23:25.252 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.252 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-08-31T06:23:25.252 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-31T06:23:25.252 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.252 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.253 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.253 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_max_iterations": { 2024-08-31T06:23:25.253 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_max_iterations", 2024-08-31T06:23:25.253 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-08-31T06:23:25.253 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.253 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.253 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "25", 2024-08-31T06:23:25.253 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-08-31T06:23:25.253 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "250", 2024-08-31T06:23:25.253 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.253 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-08-31T06:23:25.253 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.253 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.253 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.254 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.254 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_metrics": { 2024-08-31T06:23:25.254 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_metrics", 2024-08-31T06:23:25.254 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.254 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.254 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.254 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "pgs,objects,bytes", 2024-08-31T06:23:25.254 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.254 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.254 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.254 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-08-31T06:23:25.254 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-08-31T06:23:25.254 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.254 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.254 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.255 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_step": { 2024-08-31T06:23:25.255 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_step", 2024-08-31T06:23:25.255 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-08-31T06:23:25.255 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.255 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.255 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.5", 2024-08-31T06:23:25.255 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0.001", 2024-08-31T06:23:25.255 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "0.999", 2024-08-31T06:23:25.255 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.255 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "aggressiveness of optimization", 2024-08-31T06:23:25.255 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-08-31T06:23:25.255 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.255 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.256 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.256 INFO:teuthology.orchestra.run.smithi016.stdout: "end_time": { 2024-08-31T06:23:25.256 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_time", 2024-08-31T06:23:25.256 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.256 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.256 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.256 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2359", 2024-08-31T06:23:25.256 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.256 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.256 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.256 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ending time of day to automatically balance", 2024-08-31T06:23:25.256 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-31T06:23:25.256 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.256 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.257 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.257 INFO:teuthology.orchestra.run.smithi016.stdout: "end_weekday": { 2024-08-31T06:23:25.257 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_weekday", 2024-08-31T06:23:25.257 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-08-31T06:23:25.257 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.257 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.257 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.257 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-08-31T06:23:25.257 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-08-31T06:23:25.257 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.257 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-08-31T06:23:25.257 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-31T06:23:25.257 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.257 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.258 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.258 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.258 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.258 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.258 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.258 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.258 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.258 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.258 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.258 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.258 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.258 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.258 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.258 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.259 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.259 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.259 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.259 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.259 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.259 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.259 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.259 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.259 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.259 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.259 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.259 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.259 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.259 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.260 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.260 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.260 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.260 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.260 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.260 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.260 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.260 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.260 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.260 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.260 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.260 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.260 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.260 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.261 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.261 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.261 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.261 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.261 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.261 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.261 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.261 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.261 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.261 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.261 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.261 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.261 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.261 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.262 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.262 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.262 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.262 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.262 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.262 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.262 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.262 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.262 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.262 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.262 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.262 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.262 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.262 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.262 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.263 INFO:teuthology.orchestra.run.smithi016.stdout: "min_score": { 2024-08-31T06:23:25.263 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_score", 2024-08-31T06:23:25.263 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-08-31T06:23:25.263 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.263 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.263 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.263 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.263 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.263 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.263 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-08-31T06:23:25.263 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.263 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.263 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.263 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.264 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-08-31T06:23:25.264 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-08-31T06:23:25.264 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.264 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.264 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.264 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "upmap", 2024-08-31T06:23:25.264 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.264 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.264 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.264 INFO:teuthology.orchestra.run.smithi016.stdout: "crush-compat", 2024-08-31T06:23:25.264 INFO:teuthology.orchestra.run.smithi016.stdout: "none", 2024-08-31T06:23:25.264 INFO:teuthology.orchestra.run.smithi016.stdout: "read", 2024-08-31T06:23:25.265 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap", 2024-08-31T06:23:25.265 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap-read" 2024-08-31T06:23:25.265 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.265 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Balancer mode", 2024-08-31T06:23:25.265 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.265 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.265 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.265 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.265 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_ids": { 2024-08-31T06:23:25.265 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_ids", 2024-08-31T06:23:25.265 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.265 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.265 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.266 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.266 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.266 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.266 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.266 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-08-31T06:23:25.266 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.266 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.266 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.266 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.266 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-08-31T06:23:25.266 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-08-31T06:23:25.266 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:25.266 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.267 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.267 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-08-31T06:23:25.267 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.267 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.267 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.267 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-08-31T06:23:25.267 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.267 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.267 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.267 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.267 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.267 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.267 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.267 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.267 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.268 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.268 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.268 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.268 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.268 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.268 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.268 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.268 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.268 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.268 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_deviation": { 2024-08-31T06:23:25.268 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_deviation", 2024-08-31T06:23:25.268 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.268 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.268 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.269 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-08-31T06:23:25.269 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-08-31T06:23:25.269 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.269 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.269 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "deviation below which no optimization is attempted", 2024-08-31T06:23:25.269 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-08-31T06:23:25.269 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.269 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.269 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.269 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_optimizations": { 2024-08-31T06:23:25.269 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_optimizations", 2024-08-31T06:23:25.269 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-08-31T06:23:25.269 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.269 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.269 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-08-31T06:23:25.270 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.270 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.270 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.270 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-08-31T06:23:25.270 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.270 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.270 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.270 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.270 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.270 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.270 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.270 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm", 2024-08-31T06:23:25.270 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.270 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.271 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.271 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_down_multiplier": { 2024-08-31T06:23:25.271 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_down_multiplier", 2024-08-31T06:23:25.271 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-08-31T06:23:25.271 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.271 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.271 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-08-31T06:23:25.271 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.271 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.271 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.271 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-08-31T06:23:25.271 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.271 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.271 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.272 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.272 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_refresh_rate": { 2024-08-31T06:23:25.272 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_refresh_rate", 2024-08-31T06:23:25.272 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:25.272 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.272 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.272 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "20", 2024-08-31T06:23:25.272 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.272 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.272 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.272 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-08-31T06:23:25.272 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.272 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.273 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.273 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.273 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_starting_port": { 2024-08-31T06:23:25.273 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_starting_port", 2024-08-31T06:23:25.273 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.273 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.273 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.273 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4721", 2024-08-31T06:23:25.273 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.273 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.273 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.273 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-08-31T06:23:25.273 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.273 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.274 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.274 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.274 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_ptrace": { 2024-08-31T06:23:25.274 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_ptrace", 2024-08-31T06:23:25.274 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.274 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.274 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.274 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.274 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.274 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.274 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.274 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-08-31T06:23:25.274 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-08-31T06:23:25.274 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.274 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.275 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.275 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_interval": { 2024-08-31T06:23:25.275 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_interval", 2024-08-31T06:23:25.275 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:25.275 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.275 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.275 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-08-31T06:23:25.275 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.275 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.275 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.275 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to autotune daemon memory", 2024-08-31T06:23:25.275 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.275 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.275 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.276 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.276 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_memory_target_ratio": { 2024-08-31T06:23:25.276 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_memory_target_ratio", 2024-08-31T06:23:25.276 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-08-31T06:23:25.276 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.276 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.276 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.7", 2024-08-31T06:23:25.276 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.276 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.276 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.276 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-08-31T06:23:25.276 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.276 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.277 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.277 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.277 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm_log_destination": { 2024-08-31T06:23:25.277 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm_log_destination", 2024-08-31T06:23:25.277 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.277 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.277 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.277 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.277 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.277 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.277 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.277 INFO:teuthology.orchestra.run.smithi016.stdout: "file", 2024-08-31T06:23:25.277 INFO:teuthology.orchestra.run.smithi016.stdout: "file,syslog", 2024-08-31T06:23:25.277 INFO:teuthology.orchestra.run.smithi016.stdout: "syslog" 2024-08-31T06:23:25.277 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.278 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-08-31T06:23:25.278 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.278 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.278 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.278 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.278 INFO:teuthology.orchestra.run.smithi016.stdout: "cgroups_split": { 2024-08-31T06:23:25.278 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cgroups_split", 2024-08-31T06:23:25.278 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.278 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.278 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.278 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.278 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.278 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.278 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.279 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-08-31T06:23:25.279 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.279 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.279 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.279 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.279 INFO:teuthology.orchestra.run.smithi016.stdout: "config_checks_enabled": { 2024-08-31T06:23:25.279 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_checks_enabled", 2024-08-31T06:23:25.279 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.279 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.279 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.279 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.279 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.279 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.279 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.279 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-08-31T06:23:25.280 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.280 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.280 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.280 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.280 INFO:teuthology.orchestra.run.smithi016.stdout: "config_dashboard": { 2024-08-31T06:23:25.280 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_dashboard", 2024-08-31T06:23:25.280 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.280 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.280 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.280 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.280 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.280 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.280 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.280 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-08-31T06:23:25.280 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.281 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.281 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.281 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.281 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_alertmanager": { 2024-08-31T06:23:25.281 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_alertmanager", 2024-08-31T06:23:25.281 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.281 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.281 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.281 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-08-31T06:23:25.281 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.281 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.281 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.281 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-08-31T06:23:25.281 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.281 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.282 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.282 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.282 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_base": { 2024-08-31T06:23:25.282 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_base", 2024-08-31T06:23:25.282 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.282 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.282 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.282 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/ceph", 2024-08-31T06:23:25.282 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.282 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.282 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.282 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Container image name, without the tag", 2024-08-31T06:23:25.282 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.282 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.282 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.283 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.283 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_elasticsearch": { 2024-08-31T06:23:25.283 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_elasticsearch", 2024-08-31T06:23:25.283 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.283 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.283 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.283 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-08-31T06:23:25.283 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.283 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.283 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.283 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "elasticsearch container image", 2024-08-31T06:23:25.283 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.284 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.284 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.284 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.284 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_grafana": { 2024-08-31T06:23:25.284 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_grafana", 2024-08-31T06:23:25.284 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.284 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.284 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.284 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-08-31T06:23:25.284 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.284 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.284 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.284 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-08-31T06:23:25.284 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.284 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.285 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.285 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.285 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_haproxy": { 2024-08-31T06:23:25.285 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_haproxy", 2024-08-31T06:23:25.285 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.285 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.285 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.285 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-08-31T06:23:25.285 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.285 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.285 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.285 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "HAproxy container image", 2024-08-31T06:23:25.285 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.285 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.286 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.286 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.286 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_agent": { 2024-08-31T06:23:25.286 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_agent", 2024-08-31T06:23:25.286 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.286 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.286 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.286 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-08-31T06:23:25.286 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.286 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.286 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.286 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger agent container image", 2024-08-31T06:23:25.286 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.286 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.287 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.287 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.287 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_collector": { 2024-08-31T06:23:25.287 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_collector", 2024-08-31T06:23:25.287 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.287 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.287 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.287 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-08-31T06:23:25.287 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.287 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.287 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.287 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger collector container image", 2024-08-31T06:23:25.287 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.287 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.287 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.288 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.288 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_query": { 2024-08-31T06:23:25.288 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_query", 2024-08-31T06:23:25.288 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.288 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.288 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.288 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-08-31T06:23:25.288 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.288 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.288 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.288 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger query container image", 2024-08-31T06:23:25.288 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.288 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.288 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.289 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.289 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_keepalived": { 2024-08-31T06:23:25.289 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_keepalived", 2024-08-31T06:23:25.289 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.289 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.289 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.289 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-08-31T06:23:25.289 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.289 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.289 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.289 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Keepalived container image", 2024-08-31T06:23:25.289 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.289 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.289 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.290 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.290 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_loki": { 2024-08-31T06:23:25.290 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_loki", 2024-08-31T06:23:25.290 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.290 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.290 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.290 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-08-31T06:23:25.290 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.290 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.290 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.290 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Loki container image", 2024-08-31T06:23:25.290 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.290 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.290 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.291 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.291 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_nginx": { 2024-08-31T06:23:25.291 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_nginx", 2024-08-31T06:23:25.291 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.291 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.291 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.291 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-08-31T06:23:25.291 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.291 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.291 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.291 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Nginx container image", 2024-08-31T06:23:25.291 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.291 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.291 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.292 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.292 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_node_exporter": { 2024-08-31T06:23:25.292 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_node_exporter", 2024-08-31T06:23:25.292 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.292 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.292 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.292 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-08-31T06:23:25.292 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.292 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.292 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.292 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-08-31T06:23:25.292 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.292 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.292 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.292 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.292 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_nvmeof": { 2024-08-31T06:23:25.293 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_nvmeof", 2024-08-31T06:23:25.293 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.293 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.293 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.293 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-08-31T06:23:25.293 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.293 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.293 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.293 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Nvme-of container image", 2024-08-31T06:23:25.293 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.293 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.293 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.293 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.294 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_oauth2_proxy": { 2024-08-31T06:23:25.294 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_oauth2_proxy", 2024-08-31T06:23:25.294 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.294 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.294 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.294 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-08-31T06:23:25.294 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.294 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.294 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.294 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "oauth2-proxy container image", 2024-08-31T06:23:25.294 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.294 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.294 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.294 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.294 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_prometheus": { 2024-08-31T06:23:25.295 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_prometheus", 2024-08-31T06:23:25.295 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.295 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.295 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.295 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-08-31T06:23:25.295 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.295 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.295 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.295 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-08-31T06:23:25.295 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.295 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.295 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.295 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.296 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_promtail": { 2024-08-31T06:23:25.296 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_promtail", 2024-08-31T06:23:25.296 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.296 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.296 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.296 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-08-31T06:23:25.296 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.296 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.296 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.296 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Promtail container image", 2024-08-31T06:23:25.296 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.296 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.296 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.296 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.297 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_samba": { 2024-08-31T06:23:25.297 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_samba", 2024-08-31T06:23:25.297 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.297 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.297 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.297 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-08-31T06:23:25.297 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.297 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.297 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.297 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Samba/SMB container image", 2024-08-31T06:23:25.297 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.297 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.297 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.297 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.298 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_snmp_gateway": { 2024-08-31T06:23:25.298 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_snmp_gateway", 2024-08-31T06:23:25.298 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.298 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.298 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.298 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-08-31T06:23:25.298 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.298 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.298 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.298 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SNMP Gateway container image", 2024-08-31T06:23:25.298 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.298 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.298 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.298 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.299 INFO:teuthology.orchestra.run.smithi016.stdout: "container_init": { 2024-08-31T06:23:25.299 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_init", 2024-08-31T06:23:25.299 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.299 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.299 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.299 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.299 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.299 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.299 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.299 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Run podman/docker with `--init`", 2024-08-31T06:23:25.299 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.299 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.299 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.299 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.299 INFO:teuthology.orchestra.run.smithi016.stdout: "daemon_cache_timeout": { 2024-08-31T06:23:25.300 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "daemon_cache_timeout", 2024-08-31T06:23:25.300 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:25.300 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.300 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.300 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-08-31T06:23:25.300 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.300 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.300 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.300 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-08-31T06:23:25.300 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.300 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.300 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.300 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.301 INFO:teuthology.orchestra.run.smithi016.stdout: "default_cephadm_command_timeout": { 2024-08-31T06:23:25.301 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_cephadm_command_timeout", 2024-08-31T06:23:25.301 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.301 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.301 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.301 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "900", 2024-08-31T06:23:25.301 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.301 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.301 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.301 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-08-31T06:23:25.301 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.301 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.301 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.301 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.302 INFO:teuthology.orchestra.run.smithi016.stdout: "default_registry": { 2024-08-31T06:23:25.302 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_registry", 2024-08-31T06:23:25.302 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.302 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.302 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.302 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io", 2024-08-31T06:23:25.302 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.302 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.302 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.302 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-08-31T06:23:25.302 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.302 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.302 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.302 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.303 INFO:teuthology.orchestra.run.smithi016.stdout: "device_cache_timeout": { 2024-08-31T06:23:25.303 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_cache_timeout", 2024-08-31T06:23:25.303 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:25.303 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.303 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.303 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1800", 2024-08-31T06:23:25.303 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.303 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.303 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.303 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache device inventory", 2024-08-31T06:23:25.303 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.303 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.304 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.304 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.304 INFO:teuthology.orchestra.run.smithi016.stdout: "device_enhanced_scan": { 2024-08-31T06:23:25.304 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_enhanced_scan", 2024-08-31T06:23:25.304 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.304 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.304 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.304 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.304 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.304 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.304 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.304 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use libstoragemgmt during device scans", 2024-08-31T06:23:25.304 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.304 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.305 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.305 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.305 INFO:teuthology.orchestra.run.smithi016.stdout: "facts_cache_timeout": { 2024-08-31T06:23:25.305 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "facts_cache_timeout", 2024-08-31T06:23:25.305 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:25.305 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.305 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.305 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-08-31T06:23:25.305 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.305 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.305 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.305 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache host facts data", 2024-08-31T06:23:25.305 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.305 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.306 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.306 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.306 INFO:teuthology.orchestra.run.smithi016.stdout: "grafana_dashboards_path": { 2024-08-31T06:23:25.306 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "grafana_dashboards_path", 2024-08-31T06:23:25.306 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.306 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.306 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.306 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-08-31T06:23:25.306 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.306 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.306 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.306 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-08-31T06:23:25.306 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.306 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.306 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.307 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.307 INFO:teuthology.orchestra.run.smithi016.stdout: "host_check_interval": { 2024-08-31T06:23:25.307 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "host_check_interval", 2024-08-31T06:23:25.307 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:25.307 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.307 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.307 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-08-31T06:23:25.307 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.307 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.307 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.307 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to perform a host check", 2024-08-31T06:23:25.307 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.307 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.307 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.308 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.308 INFO:teuthology.orchestra.run.smithi016.stdout: "hw_monitoring": { 2024-08-31T06:23:25.308 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hw_monitoring", 2024-08-31T06:23:25.308 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.308 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.308 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.308 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.308 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.308 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.308 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.308 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-08-31T06:23:25.308 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.308 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.308 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.308 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.309 INFO:teuthology.orchestra.run.smithi016.stdout: "inventory_list_all": { 2024-08-31T06:23:25.309 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "inventory_list_all", 2024-08-31T06:23:25.309 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.309 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.309 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.309 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.309 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.309 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.309 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.309 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-08-31T06:23:25.309 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.309 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.309 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.309 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.309 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.310 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.310 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.310 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.310 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.310 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.310 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.310 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.310 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.310 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.310 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.310 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.310 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.310 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.310 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.310 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.310 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.311 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.311 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.311 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.311 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.311 INFO:teuthology.orchestra.run.smithi016.stdout: "log_refresh_metadata": { 2024-08-31T06:23:25.311 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_refresh_metadata", 2024-08-31T06:23:25.311 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.311 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.311 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.311 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.311 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.311 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.311 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.311 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-08-31T06:23:25.311 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.312 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.312 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.312 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.312 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.312 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.312 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.312 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.312 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.312 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.312 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.312 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.312 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.312 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-08-31T06:23:25.313 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.313 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.313 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.313 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.313 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.313 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.313 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.313 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.313 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.313 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.313 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.313 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.313 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.313 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.314 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.314 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.314 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.314 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.314 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.314 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.314 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.314 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.314 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.314 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.314 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.314 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.314 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.314 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.315 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.315 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.315 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.315 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.315 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.315 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.315 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.315 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.315 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.315 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.315 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.315 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf": { 2024-08-31T06:23:25.315 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-08-31T06:23:25.315 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.316 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.316 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.316 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.316 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.316 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.316 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.316 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-08-31T06:23:25.316 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.316 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.316 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.316 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.316 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-08-31T06:23:25.316 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-08-31T06:23:25.316 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.317 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.317 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.317 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "*", 2024-08-31T06:23:25.317 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.317 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.317 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.317 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-08-31T06:23:25.317 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.317 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.317 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.317 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.317 INFO:teuthology.orchestra.run.smithi016.stdout: "max_count_per_host": { 2024-08-31T06:23:25.317 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_count_per_host", 2024-08-31T06:23:25.317 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.318 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.318 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.318 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-08-31T06:23:25.318 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.318 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.318 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.318 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of daemons per service per host", 2024-08-31T06:23:25.318 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.318 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.318 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.318 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.318 INFO:teuthology.orchestra.run.smithi016.stdout: "max_osd_draining_count": { 2024-08-31T06:23:25.318 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_osd_draining_count", 2024-08-31T06:23:25.318 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.319 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.319 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.319 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-08-31T06:23:25.319 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.319 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.319 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.319 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-08-31T06:23:25.319 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.319 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.319 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.319 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.319 INFO:teuthology.orchestra.run.smithi016.stdout: "migration_current": { 2024-08-31T06:23:25.319 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "migration_current", 2024-08-31T06:23:25.319 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.320 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.320 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.320 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.320 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.320 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.320 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.320 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "internal - do not modify", 2024-08-31T06:23:25.320 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.320 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.320 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.320 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.320 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-08-31T06:23:25.320 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-08-31T06:23:25.321 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.321 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.321 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.321 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "root", 2024-08-31T06:23:25.321 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.321 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.321 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.321 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm-package", 2024-08-31T06:23:25.321 INFO:teuthology.orchestra.run.smithi016.stdout: "root" 2024-08-31T06:23:25.321 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.321 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "mode for remote execution of cephadm", 2024-08-31T06:23:25.321 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.321 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.321 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.322 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.322 INFO:teuthology.orchestra.run.smithi016.stdout: "oob_default_addr": { 2024-08-31T06:23:25.322 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "oob_default_addr", 2024-08-31T06:23:25.322 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.322 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.322 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.322 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "169.254.1.1", 2024-08-31T06:23:25.322 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.322 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.322 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.322 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default address for RedFish API (oob management).", 2024-08-31T06:23:25.322 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.322 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.322 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.323 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.323 INFO:teuthology.orchestra.run.smithi016.stdout: "prometheus_alerts_path": { 2024-08-31T06:23:25.323 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus_alerts_path", 2024-08-31T06:23:25.323 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.323 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.323 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.323 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-08-31T06:23:25.323 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.323 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.323 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.323 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-08-31T06:23:25.323 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.323 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.323 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.324 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.324 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_insecure": { 2024-08-31T06:23:25.324 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_insecure", 2024-08-31T06:23:25.324 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.324 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.324 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.324 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.324 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.324 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.324 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.324 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-08-31T06:23:25.324 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.324 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.324 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.325 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.325 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_password": { 2024-08-31T06:23:25.325 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_password", 2024-08-31T06:23:25.325 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.325 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.325 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.325 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.325 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.325 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.325 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.325 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-08-31T06:23:25.325 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.325 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.326 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.326 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.326 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_url": { 2024-08-31T06:23:25.326 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_url", 2024-08-31T06:23:25.326 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.326 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.326 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.326 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.326 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.326 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.326 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.326 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-08-31T06:23:25.326 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.326 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.326 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.327 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.327 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_username": { 2024-08-31T06:23:25.327 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_username", 2024-08-31T06:23:25.327 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.327 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.327 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.327 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.327 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.327 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.327 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.327 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-08-31T06:23:25.327 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.327 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.327 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.328 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.328 INFO:teuthology.orchestra.run.smithi016.stdout: "secure_monitoring_stack": { 2024-08-31T06:23:25.328 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secure_monitoring_stack", 2024-08-31T06:23:25.328 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.328 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.328 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.328 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.328 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.328 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.328 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.328 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-08-31T06:23:25.328 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.328 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.328 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.329 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.329 INFO:teuthology.orchestra.run.smithi016.stdout: "service_discovery_port": { 2024-08-31T06:23:25.329 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "service_discovery_port", 2024-08-31T06:23:25.329 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.329 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.329 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.329 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8765", 2024-08-31T06:23:25.329 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.329 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.329 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.329 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "cephadm service discovery port", 2024-08-31T06:23:25.329 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.329 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.329 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.330 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.330 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.330 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.330 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.330 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.330 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.330 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.330 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.330 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.330 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.330 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.330 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.330 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.330 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.330 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.331 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_config_file": { 2024-08-31T06:23:25.331 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_config_file", 2024-08-31T06:23:25.331 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.331 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.331 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.331 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.331 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.331 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.331 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.331 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-08-31T06:23:25.331 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.331 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.331 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.331 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.331 INFO:teuthology.orchestra.run.smithi016.stdout: "use_agent": { 2024-08-31T06:23:25.332 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_agent", 2024-08-31T06:23:25.332 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.332 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.332 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.332 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.332 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.332 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.332 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.332 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-08-31T06:23:25.332 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.332 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.332 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.332 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.333 INFO:teuthology.orchestra.run.smithi016.stdout: "use_repo_digest": { 2024-08-31T06:23:25.333 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_repo_digest", 2024-08-31T06:23:25.333 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.333 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.333 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.333 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.333 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.333 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.333 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.333 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-08-31T06:23:25.333 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.333 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.333 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.333 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.334 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_failed_host_check": { 2024-08-31T06:23:25.334 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_failed_host_check", 2024-08-31T06:23:25.334 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.334 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.334 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.334 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.334 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.334 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.334 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.334 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if the host check fails", 2024-08-31T06:23:25.334 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.334 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.334 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.334 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.335 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_daemons": { 2024-08-31T06:23:25.335 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_daemons", 2024-08-31T06:23:25.335 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.335 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.335 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.335 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.335 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.335 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.335 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.335 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-08-31T06:23:25.335 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.335 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.335 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.335 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.336 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_hosts": { 2024-08-31T06:23:25.336 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_hosts", 2024-08-31T06:23:25.336 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.336 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.336 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.336 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.336 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.336 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.336 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.336 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-08-31T06:23:25.336 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.336 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.336 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.336 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.336 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.337 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.337 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.337 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crash", 2024-08-31T06:23:25.337 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.337 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.337 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.337 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.337 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.337 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.337 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.337 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.337 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.337 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.337 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.338 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.338 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.338 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.338 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.338 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.338 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.338 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.338 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.338 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.338 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.338 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.338 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.338 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.338 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.338 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.339 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.339 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.339 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.339 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.339 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.339 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.339 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.339 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.339 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.339 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.339 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.339 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.339 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.339 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.340 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.340 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.340 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.340 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.340 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.340 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.340 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.340 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.340 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.340 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.340 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.340 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.340 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.340 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.341 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.341 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.341 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.341 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.341 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.341 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.341 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.341 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.341 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.341 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.341 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.341 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.341 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.341 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.341 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.342 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.342 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.342 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.342 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.343 INFO:teuthology.orchestra.run.smithi016.stdout: "retain_interval": { 2024-08-31T06:23:25.343 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retain_interval", 2024-08-31T06:23:25.343 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:25.343 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.343 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.343 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "31536000", 2024-08-31T06:23:25.343 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.343 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.343 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.343 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain crashes before pruning them", 2024-08-31T06:23:25.343 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.343 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.343 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.343 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.344 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.344 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.344 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.344 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.344 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.344 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.344 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.344 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.344 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.344 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.344 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.344 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.344 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.344 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.345 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_recent_interval": { 2024-08-31T06:23:25.345 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_recent_interval", 2024-08-31T06:23:25.345 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:25.345 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.345 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.345 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1209600", 2024-08-31T06:23:25.345 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.345 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.345 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.345 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "time interval in which to warn about recent crashes", 2024-08-31T06:23:25.345 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.345 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.345 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.346 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.346 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.346 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.346 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.346 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dashboard", 2024-08-31T06:23:25.346 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.346 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.346 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.346 INFO:teuthology.orchestra.run.smithi016.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-08-31T06:23:25.346 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-08-31T06:23:25.346 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.346 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.346 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.347 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-08-31T06:23:25.347 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.347 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.347 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.347 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.347 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.347 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.347 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.347 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.347 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_HOST": { 2024-08-31T06:23:25.347 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_HOST", 2024-08-31T06:23:25.347 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.347 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.347 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.348 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.348 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.348 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.348 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.348 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.348 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.348 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.348 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.348 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.348 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-08-31T06:23:25.348 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-08-31T06:23:25.348 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.348 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.349 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.349 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.349 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.349 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.349 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.349 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.349 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.349 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.349 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.349 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.349 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_ENABLED": { 2024-08-31T06:23:25.349 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_ENABLED", 2024-08-31T06:23:25.349 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.349 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.350 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.350 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.350 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.350 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.350 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.350 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.350 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.350 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.350 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.350 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.350 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-08-31T06:23:25.350 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-08-31T06:23:25.350 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.350 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.351 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.351 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.351 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.351 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.351 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.351 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.351 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.351 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.351 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.351 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.351 INFO:teuthology.orchestra.run.smithi016.stdout: "ENABLE_BROWSABLE_API": { 2024-08-31T06:23:25.351 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ENABLE_BROWSABLE_API", 2024-08-31T06:23:25.351 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.352 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.352 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.352 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.352 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.352 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.352 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.352 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.352 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.352 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.352 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.352 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.352 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-08-31T06:23:25.352 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-08-31T06:23:25.352 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.353 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.353 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.353 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.353 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.353 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.353 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.353 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.353 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.353 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.353 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.353 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.353 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-08-31T06:23:25.353 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-08-31T06:23:25.354 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.354 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.354 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.354 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.354 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.354 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.354 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.354 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.354 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.354 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.354 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.354 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.354 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-08-31T06:23:25.354 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-08-31T06:23:25.355 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.355 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.355 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.355 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.355 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.355 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.355 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.355 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.355 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.355 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.355 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.355 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.355 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-08-31T06:23:25.355 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-08-31T06:23:25.355 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.356 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.356 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.356 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.356 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.356 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.356 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.356 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.356 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.356 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.356 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.356 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.356 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_NFS": { 2024-08-31T06:23:25.356 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-08-31T06:23:25.356 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.357 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.357 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.357 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.357 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.357 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.357 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.357 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.357 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.357 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.357 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.357 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.357 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RBD": { 2024-08-31T06:23:25.357 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-08-31T06:23:25.357 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.358 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.358 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.358 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.358 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.358 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.358 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.358 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.358 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.358 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.358 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.358 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.358 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RGW": { 2024-08-31T06:23:25.358 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-08-31T06:23:25.358 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.359 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.359 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.359 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.359 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.359 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.359 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.359 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.359 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.359 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.359 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.359 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.359 INFO:teuthology.orchestra.run.smithi016.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-08-31T06:23:25.359 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-08-31T06:23:25.360 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.360 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.360 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.360 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.360 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.360 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.360 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.360 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.360 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.360 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.360 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.360 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.360 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_PASSWORD": { 2024-08-31T06:23:25.360 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_PASSWORD", 2024-08-31T06:23:25.361 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.361 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.361 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.361 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-08-31T06:23:25.361 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.361 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.361 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.361 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.361 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.361 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.361 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.361 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.361 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-08-31T06:23:25.362 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-08-31T06:23:25.362 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.362 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.362 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.362 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.362 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.362 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.362 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.362 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.362 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.362 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.362 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.362 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.362 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_URL": { 2024-08-31T06:23:25.362 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_URL", 2024-08-31T06:23:25.363 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.363 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.363 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.363 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.363 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.363 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.363 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.363 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.363 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.363 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.363 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.363 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.363 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_USERNAME": { 2024-08-31T06:23:25.363 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_USERNAME", 2024-08-31T06:23:25.364 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.364 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.364 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.364 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-08-31T06:23:25.364 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.364 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.364 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.364 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.364 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.364 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.364 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.364 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.364 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-08-31T06:23:25.365 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-08-31T06:23:25.365 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.365 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.365 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.365 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.365 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.365 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.365 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.365 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.365 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.365 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.365 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.365 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.365 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-08-31T06:23:25.366 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-08-31T06:23:25.366 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.366 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.366 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.366 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.366 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.366 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.366 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.366 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.366 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.366 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.366 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.366 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.366 INFO:teuthology.orchestra.run.smithi016.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-08-31T06:23:25.367 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-08-31T06:23:25.367 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.367 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.367 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.367 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.367 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.367 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.367 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.367 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.367 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.367 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.367 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.367 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.367 INFO:teuthology.orchestra.run.smithi016.stdout: "ISSUE_TRACKER_API_KEY": { 2024-08-31T06:23:25.368 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-08-31T06:23:25.368 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.368 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.368 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.368 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.368 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.368 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.368 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.368 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.368 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.368 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.368 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.368 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.369 INFO:teuthology.orchestra.run.smithi016.stdout: "MANAGED_BY_CLUSTERS": { 2024-08-31T06:23:25.369 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-08-31T06:23:25.369 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.369 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.369 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.369 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "[]", 2024-08-31T06:23:25.369 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.369 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.369 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.369 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.369 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.369 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.369 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.369 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.369 INFO:teuthology.orchestra.run.smithi016.stdout: "MULTICLUSTER_CONFIG": { 2024-08-31T06:23:25.370 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "MULTICLUSTER_CONFIG", 2024-08-31T06:23:25.370 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.370 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.370 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.370 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "{}", 2024-08-31T06:23:25.370 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.370 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.370 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.370 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.370 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.370 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.370 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.370 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.370 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_HOST": { 2024-08-31T06:23:25.371 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_HOST", 2024-08-31T06:23:25.371 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.371 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.371 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.371 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.371 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.371 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.371 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.371 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.371 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.371 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.371 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.371 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.371 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-08-31T06:23:25.372 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-08-31T06:23:25.372 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.372 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.372 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.372 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.372 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.372 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.372 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.372 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.372 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.372 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.372 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.372 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.372 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-08-31T06:23:25.373 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-08-31T06:23:25.373 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.373 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.373 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.373 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.373 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.373 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.373 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.373 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.373 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.373 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.373 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.374 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.374 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-08-31T06:23:25.374 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-08-31T06:23:25.374 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.374 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.374 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.374 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.374 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.374 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.374 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.374 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.374 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.374 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.374 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.375 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.375 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-08-31T06:23:25.375 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-08-31T06:23:25.375 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.375 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.375 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.375 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.375 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.375 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.375 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.375 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.375 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.375 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.375 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.376 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.376 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-08-31T06:23:25.376 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-08-31T06:23:25.376 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.376 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.376 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.376 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.376 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.376 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.376 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.376 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.376 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.376 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.377 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.377 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.377 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-08-31T06:23:25.377 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-08-31T06:23:25.377 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.377 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.377 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.377 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.377 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.377 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.377 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.377 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.377 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.377 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.378 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.378 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.378 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-08-31T06:23:25.378 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-08-31T06:23:25.378 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.378 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.378 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.378 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.378 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.378 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.378 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.378 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.378 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.378 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.379 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.379 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.379 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-08-31T06:23:25.379 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-08-31T06:23:25.379 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.379 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.379 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.379 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.379 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.379 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.379 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.379 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.379 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.379 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.379 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.380 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.380 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_ENABLED": { 2024-08-31T06:23:25.380 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_ENABLED", 2024-08-31T06:23:25.380 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.380 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.380 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.380 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.380 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.380 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.380 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.380 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.380 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.380 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.380 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.381 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.381 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-08-31T06:23:25.381 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-08-31T06:23:25.381 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.381 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.381 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.381 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-08-31T06:23:25.381 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.381 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.381 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.381 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.381 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.381 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.381 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.382 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.382 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-08-31T06:23:25.382 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-08-31T06:23:25.382 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.382 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.382 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.382 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-08-31T06:23:25.382 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.382 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.382 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.382 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.382 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.382 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.382 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.382 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.383 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-08-31T06:23:25.383 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-08-31T06:23:25.383 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.383 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.383 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.383 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8", 2024-08-31T06:23:25.383 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.383 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.383 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.383 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.383 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.383 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.383 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.383 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.384 INFO:teuthology.orchestra.run.smithi016.stdout: "REST_REQUESTS_TIMEOUT": { 2024-08-31T06:23:25.384 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-08-31T06:23:25.384 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.384 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.384 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.384 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "45", 2024-08-31T06:23:25.384 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.384 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.384 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.384 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.384 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.384 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.384 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.384 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.385 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ACCESS_KEY": { 2024-08-31T06:23:25.385 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ACCESS_KEY", 2024-08-31T06:23:25.385 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.385 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.385 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.385 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.385 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.385 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.385 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.385 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.385 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.385 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.385 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.385 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.386 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-08-31T06:23:25.386 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-08-31T06:23:25.386 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.386 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.386 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.386 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-08-31T06:23:25.386 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.386 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.386 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.386 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.386 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.386 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.386 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.387 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.387 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SECRET_KEY": { 2024-08-31T06:23:25.387 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SECRET_KEY", 2024-08-31T06:23:25.387 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.387 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.387 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.387 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.387 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.387 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.387 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.387 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.387 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.387 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.387 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.387 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.388 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SSL_VERIFY": { 2024-08-31T06:23:25.388 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SSL_VERIFY", 2024-08-31T06:23:25.388 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.388 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.388 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.388 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.388 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.388 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.388 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.388 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.388 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.388 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.388 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.388 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.389 INFO:teuthology.orchestra.run.smithi016.stdout: "UNSAFE_TLS_v1_2": { 2024-08-31T06:23:25.389 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "UNSAFE_TLS_v1_2", 2024-08-31T06:23:25.389 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.389 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.389 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.389 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.389 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.389 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.389 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.389 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.389 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.389 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.389 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.390 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.390 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-08-31T06:23:25.390 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-08-31T06:23:25.390 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.390 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.390 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.390 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.390 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.390 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.390 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.390 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.390 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.390 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.391 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.391 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.391 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-08-31T06:23:25.391 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-08-31T06:23:25.391 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.391 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.391 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.391 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-08-31T06:23:25.391 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.391 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.391 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.391 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.391 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.391 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.392 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.392 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.392 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-08-31T06:23:25.392 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-08-31T06:23:25.392 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.392 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.392 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.392 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-08-31T06:23:25.392 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.392 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.392 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.392 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.392 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.392 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.393 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.393 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.393 INFO:teuthology.orchestra.run.smithi016.stdout: "cross_origin_url": { 2024-08-31T06:23:25.393 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cross_origin_url", 2024-08-31T06:23:25.393 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.393 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.393 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.393 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.393 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.393 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.393 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.393 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.393 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.393 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.393 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.394 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.394 INFO:teuthology.orchestra.run.smithi016.stdout: "crt_file": { 2024-08-31T06:23:25.394 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crt_file", 2024-08-31T06:23:25.394 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.394 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.394 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.394 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.394 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.394 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.394 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.394 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.394 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.394 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.394 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.395 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.395 INFO:teuthology.orchestra.run.smithi016.stdout: "debug": { 2024-08-31T06:23:25.395 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "debug", 2024-08-31T06:23:25.395 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.395 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.395 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.395 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.395 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.395 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.395 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.395 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable/disable debug options", 2024-08-31T06:23:25.396 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.396 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.396 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.396 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.396 INFO:teuthology.orchestra.run.smithi016.stdout: "jwt_token_ttl": { 2024-08-31T06:23:25.396 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "jwt_token_ttl", 2024-08-31T06:23:25.396 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.396 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.396 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.396 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "28800", 2024-08-31T06:23:25.396 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.396 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.396 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.396 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.397 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.397 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.397 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.397 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.397 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-08-31T06:23:25.397 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-08-31T06:23:25.397 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.397 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.397 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.397 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.397 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.397 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.397 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.398 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.398 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.398 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.398 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.398 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.398 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.398 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.398 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.398 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.398 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.398 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.398 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.398 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.399 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.399 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.399 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.399 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.399 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.399 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.399 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.399 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.399 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.399 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.399 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.399 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.399 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.400 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.400 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.400 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.400 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.400 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.400 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.400 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.400 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.400 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.400 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.400 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.400 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.400 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.400 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.400 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.401 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.401 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.401 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.401 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.401 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.401 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.401 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.401 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.401 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.401 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.401 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.401 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.401 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.402 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.402 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.402 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.402 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.402 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.402 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.402 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.402 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.402 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.402 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.402 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.402 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.402 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.402 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.403 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.403 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.403 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.403 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.403 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.403 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.403 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.403 INFO:teuthology.orchestra.run.smithi016.stdout: "motd": { 2024-08-31T06:23:25.403 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "motd", 2024-08-31T06:23:25.403 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.403 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.403 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.403 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.404 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.404 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.404 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.404 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "The message of the day", 2024-08-31T06:23:25.404 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.404 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.404 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.404 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.404 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect_resolve_ip_addr": { 2024-08-31T06:23:25.404 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "redirect_resolve_ip_addr", 2024-08-31T06:23:25.404 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.404 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.404 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.404 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.405 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.405 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.405 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.405 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.405 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.405 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.405 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.405 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.405 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-08-31T06:23:25.405 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-08-31T06:23:25.405 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.405 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.405 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.405 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-08-31T06:23:25.406 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.406 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.406 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.406 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.406 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.406 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.406 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.406 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.406 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-08-31T06:23:25.406 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-08-31T06:23:25.406 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.406 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.406 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.407 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8080", 2024-08-31T06:23:25.407 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.407 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.407 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.407 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.407 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.407 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.407 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.407 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.407 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.407 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.407 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.407 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.407 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.407 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.408 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.408 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.408 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.408 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.408 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.408 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.408 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.408 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.408 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-08-31T06:23:25.408 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-08-31T06:23:25.408 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.408 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.408 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.408 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.409 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.409 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.409 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.409 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.409 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.409 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.409 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.409 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.409 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl_server_port": { 2024-08-31T06:23:25.409 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl_server_port", 2024-08-31T06:23:25.409 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.409 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.409 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.409 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8443", 2024-08-31T06:23:25.409 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.410 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.410 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.410 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.410 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.410 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.410 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.410 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.410 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-08-31T06:23:25.410 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-08-31T06:23:25.410 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.410 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.410 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.410 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "redirect", 2024-08-31T06:23:25.410 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.411 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.411 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.411 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.411 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect" 2024-08-31T06:23:25.411 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.411 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.411 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.411 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.411 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.411 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.411 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-08-31T06:23:25.411 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-08-31T06:23:25.411 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.411 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.411 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.411 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-08-31T06:23:25.412 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-08-31T06:23:25.412 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-08-31T06:23:25.412 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.412 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.412 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.412 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.412 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.412 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.412 INFO:teuthology.orchestra.run.smithi016.stdout: "url_prefix": { 2024-08-31T06:23:25.412 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url_prefix", 2024-08-31T06:23:25.412 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.412 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.412 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.412 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.413 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.413 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.413 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.413 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.413 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.413 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.413 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.413 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.413 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.413 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.413 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.413 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2024-08-31T06:23:25.414 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.414 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.414 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.414 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_monitoring": { 2024-08-31T06:23:25.414 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_monitoring", 2024-08-31T06:23:25.414 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.414 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.414 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.414 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.414 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.414 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.414 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.414 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "monitor device health metrics", 2024-08-31T06:23:25.414 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.415 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.415 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.415 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.415 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.415 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.415 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.415 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.415 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.415 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.415 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.415 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.415 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.415 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.415 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.416 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.416 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.416 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.416 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.416 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.416 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.416 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.416 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.416 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.416 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.416 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.416 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.416 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.416 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.417 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.417 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.417 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.417 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.417 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.417 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.417 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.417 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.417 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.417 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.417 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.417 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.417 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.417 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.418 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.418 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.418 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.418 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.418 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.418 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.418 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.418 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.418 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.418 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.418 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.418 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.418 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.418 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.419 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.419 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.419 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.419 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.419 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.419 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.419 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.419 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.419 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.419 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.419 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.419 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.419 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.419 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.420 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.420 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.420 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.420 INFO:teuthology.orchestra.run.smithi016.stdout: "mark_out_threshold": { 2024-08-31T06:23:25.420 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mark_out_threshold", 2024-08-31T06:23:25.420 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:25.420 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.420 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.420 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2419200", 2024-08-31T06:23:25.420 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.420 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.420 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.420 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-08-31T06:23:25.420 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.421 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.421 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.421 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.421 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_name": { 2024-08-31T06:23:25.421 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_name", 2024-08-31T06:23:25.421 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.421 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.421 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.421 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "device_health_metrics", 2024-08-31T06:23:25.421 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.421 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.421 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.421 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name of pool in which to store device health metrics", 2024-08-31T06:23:25.421 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.422 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.422 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.422 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.422 INFO:teuthology.orchestra.run.smithi016.stdout: "retention_period": { 2024-08-31T06:23:25.422 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retention_period", 2024-08-31T06:23:25.422 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:25.422 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.422 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.422 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15552000", 2024-08-31T06:23:25.422 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.422 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.422 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.422 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain device health metrics", 2024-08-31T06:23:25.423 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.423 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.423 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.423 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.423 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_frequency": { 2024-08-31T06:23:25.423 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_frequency", 2024-08-31T06:23:25.423 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:25.423 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.423 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.423 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-08-31T06:23:25.423 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.423 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.423 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.424 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to scrape device health metrics", 2024-08-31T06:23:25.424 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.424 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.424 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.424 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.424 INFO:teuthology.orchestra.run.smithi016.stdout: "self_heal": { 2024-08-31T06:23:25.424 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "self_heal", 2024-08-31T06:23:25.424 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.424 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.424 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.424 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.424 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.424 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.424 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.425 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-08-31T06:23:25.425 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.425 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.425 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.425 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.425 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-08-31T06:23:25.425 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-08-31T06:23:25.425 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:25.425 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.425 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.425 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-08-31T06:23:25.425 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.425 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.426 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.426 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and check device health", 2024-08-31T06:23:25.426 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.426 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.426 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.426 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.426 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.426 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.426 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.426 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.426 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.426 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.426 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.426 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.427 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.427 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.427 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.427 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.427 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.427 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.427 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_threshold": { 2024-08-31T06:23:25.427 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_threshold", 2024-08-31T06:23:25.427 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:25.427 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.427 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.427 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7257600", 2024-08-31T06:23:25.427 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.428 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.428 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.428 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-08-31T06:23:25.428 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.428 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.428 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.428 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.428 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.428 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.428 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.428 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "diskprediction_local", 2024-08-31T06:23:25.428 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.428 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.429 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.429 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.429 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.429 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.429 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.429 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.429 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.429 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.429 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.429 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.429 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.429 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.429 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.429 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.430 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.430 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.430 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.430 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.430 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.430 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.430 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.430 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.430 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.430 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.430 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.430 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.430 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.431 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.431 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.431 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.431 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.431 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.431 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.431 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.431 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.431 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.431 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.431 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.431 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.431 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.432 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.432 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.432 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.432 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.432 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.432 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.432 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.432 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.432 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.432 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.432 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.432 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.432 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.432 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.433 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.433 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.433 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.433 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.433 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.433 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.433 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.433 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.433 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.433 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.433 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.433 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.433 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.434 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.434 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.434 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.434 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.434 INFO:teuthology.orchestra.run.smithi016.stdout: "predict_interval": { 2024-08-31T06:23:25.434 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predict_interval", 2024-08-31T06:23:25.434 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.434 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.434 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.434 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-08-31T06:23:25.434 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.434 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.434 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.434 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.435 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.435 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.435 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.435 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.435 INFO:teuthology.orchestra.run.smithi016.stdout: "predictor_model": { 2024-08-31T06:23:25.435 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predictor_model", 2024-08-31T06:23:25.435 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.435 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.435 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.435 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "prophetstor", 2024-08-31T06:23:25.435 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.435 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.435 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.435 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.435 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.436 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.436 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.436 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.436 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-08-31T06:23:25.436 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-08-31T06:23:25.436 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.436 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.436 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.436 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-08-31T06:23:25.436 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.436 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.436 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.436 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.437 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.437 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.437 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.437 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.437 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.437 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.437 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.437 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.437 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.437 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.437 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.437 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.437 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.437 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.438 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.438 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.438 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.438 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.438 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.438 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.438 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.438 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "influx", 2024-08-31T06:23:25.438 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": false, 2024-08-31T06:23:25.438 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "influxdb python module not found", 2024-08-31T06:23:25.438 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.438 INFO:teuthology.orchestra.run.smithi016.stdout: "batch_size": { 2024-08-31T06:23:25.438 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "batch_size", 2024-08-31T06:23:25.439 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.439 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.439 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.439 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5000", 2024-08-31T06:23:25.439 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.439 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.439 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.439 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-08-31T06:23:25.439 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.439 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.439 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.439 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.439 INFO:teuthology.orchestra.run.smithi016.stdout: "database": { 2024-08-31T06:23:25.439 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "database", 2024-08-31T06:23:25.439 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.440 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.440 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.440 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "ceph", 2024-08-31T06:23:25.440 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.440 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.440 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.440 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-08-31T06:23:25.440 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.440 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.440 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.440 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.440 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": { 2024-08-31T06:23:25.440 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hostname", 2024-08-31T06:23:25.441 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.441 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.441 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.441 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.441 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.441 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.441 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.441 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server hostname", 2024-08-31T06:23:25.441 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.441 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.441 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.441 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.441 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-08-31T06:23:25.441 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-08-31T06:23:25.442 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:25.442 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.442 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.442 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "30", 2024-08-31T06:23:25.442 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "5", 2024-08-31T06:23:25.442 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.442 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.442 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-08-31T06:23:25.442 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.442 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.442 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.442 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.442 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.442 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.442 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.443 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.443 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.443 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.443 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.443 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.443 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.443 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.443 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.443 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.443 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.443 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.443 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.443 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.443 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.443 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.444 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.444 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.444 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.444 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.444 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.444 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.444 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.444 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.444 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.444 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.444 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.444 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.444 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.445 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.445 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.445 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.445 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.445 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.445 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.445 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.445 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.445 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.445 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.445 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.445 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.445 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.445 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.445 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.446 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.446 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.446 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.446 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.446 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.446 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.446 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.446 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.446 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.446 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.446 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.446 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.446 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.446 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.447 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.447 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.447 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.447 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.447 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.447 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.447 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.447 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.447 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.447 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.447 INFO:teuthology.orchestra.run.smithi016.stdout: "password": { 2024-08-31T06:23:25.447 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "password", 2024-08-31T06:23:25.447 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.447 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.447 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.448 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.448 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.448 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.448 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.448 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "password of InfluxDB server user", 2024-08-31T06:23:25.448 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.448 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.448 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.448 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.448 INFO:teuthology.orchestra.run.smithi016.stdout: "port": { 2024-08-31T06:23:25.448 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "port", 2024-08-31T06:23:25.448 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.448 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.448 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.449 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8086", 2024-08-31T06:23:25.449 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.449 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.449 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.449 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server port", 2024-08-31T06:23:25.449 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.449 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.449 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.449 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.449 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.449 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.449 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.449 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.449 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.449 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.450 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.450 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.450 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.450 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.450 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.450 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.450 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.450 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.450 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-08-31T06:23:25.450 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-08-31T06:23:25.450 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.450 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.450 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.451 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "false", 2024-08-31T06:23:25.451 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.451 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.451 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.451 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-08-31T06:23:25.451 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.451 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.451 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.451 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.451 INFO:teuthology.orchestra.run.smithi016.stdout: "threads": { 2024-08-31T06:23:25.451 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threads", 2024-08-31T06:23:25.451 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.451 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.451 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.452 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-08-31T06:23:25.452 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-08-31T06:23:25.452 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "32", 2024-08-31T06:23:25.452 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.452 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-08-31T06:23:25.452 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.452 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.452 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.452 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.452 INFO:teuthology.orchestra.run.smithi016.stdout: "username": { 2024-08-31T06:23:25.452 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "username", 2024-08-31T06:23:25.452 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.452 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.452 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.452 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.453 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.453 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.453 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.453 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "username of InfluxDB server user", 2024-08-31T06:23:25.453 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.453 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.453 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.453 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.453 INFO:teuthology.orchestra.run.smithi016.stdout: "verify_ssl": { 2024-08-31T06:23:25.453 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "verify_ssl", 2024-08-31T06:23:25.453 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.453 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.453 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.453 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "true", 2024-08-31T06:23:25.454 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.454 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.454 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.454 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-08-31T06:23:25.454 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.454 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.454 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.454 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.454 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.454 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.454 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.454 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "insights", 2024-08-31T06:23:25.454 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.454 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.455 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.455 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.455 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.455 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.455 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.455 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.455 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.455 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.455 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.455 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.455 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.455 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.455 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.455 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.455 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.456 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.456 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.456 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.456 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.456 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.456 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.456 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.456 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.456 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.456 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.456 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.456 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.456 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.456 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.457 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.457 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.457 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.457 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.457 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.457 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.457 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.457 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.457 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.457 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.457 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.457 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.457 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.458 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.458 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.458 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.458 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.458 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.458 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.458 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.458 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.458 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.458 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.458 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.458 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.458 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.458 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.458 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.459 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.459 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.459 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.459 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.459 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.459 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.459 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.459 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.459 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.459 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.459 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.459 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.460 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.460 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.460 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.460 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.460 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.460 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.460 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.460 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.460 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.460 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.460 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.460 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.460 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.460 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.461 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.461 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.461 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.461 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.461 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.461 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "iostat", 2024-08-31T06:23:25.461 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.461 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.461 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.461 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.461 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.461 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.461 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.461 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.462 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.462 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.462 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.462 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.462 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.462 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.462 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.462 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.462 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.462 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.462 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.462 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.462 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.462 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.463 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.463 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.463 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.463 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.463 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.463 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.463 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.463 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.463 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.463 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.463 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.463 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.463 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.463 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.463 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.463 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.464 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.464 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.464 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.464 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.464 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.464 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.464 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.464 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.464 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.464 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.464 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.464 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.464 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.465 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.465 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.465 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.465 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.465 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.465 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.465 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.465 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.465 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.465 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.465 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.465 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.465 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.465 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.466 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.466 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.466 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.466 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.466 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.466 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.466 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.466 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.466 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.466 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.466 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.466 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.466 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.466 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.467 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.467 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.467 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.467 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.467 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.467 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.467 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.467 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.467 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.467 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.467 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.467 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "k8sevents", 2024-08-31T06:23:25.467 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.467 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.467 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.468 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_event_retention_days": { 2024-08-31T06:23:25.468 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ceph_event_retention_days", 2024-08-31T06:23:25.468 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.468 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.468 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.468 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7", 2024-08-31T06:23:25.468 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.468 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.468 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.468 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Days to hold ceph event information within local cache", 2024-08-31T06:23:25.468 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.468 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.469 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.469 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.469 INFO:teuthology.orchestra.run.smithi016.stdout: "config_check_secs": { 2024-08-31T06:23:25.469 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_check_secs", 2024-08-31T06:23:25.469 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.469 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.469 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.469 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-08-31T06:23:25.469 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "10", 2024-08-31T06:23:25.469 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.469 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.469 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-08-31T06:23:25.469 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.469 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.470 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.470 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.470 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.470 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.470 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.470 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.470 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.470 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.470 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.470 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.470 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.470 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.470 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.471 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.471 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.471 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.471 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.471 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.471 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.471 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.471 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.471 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.471 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.471 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.471 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.471 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.471 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.471 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.472 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.472 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.472 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.472 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.472 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.472 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.472 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.472 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.472 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.472 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.472 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.472 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.472 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.472 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.473 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.473 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.473 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.473 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.473 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.473 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.473 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.473 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.473 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.473 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.473 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.473 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.473 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.473 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.474 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.474 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.474 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.474 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.474 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.474 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.474 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.474 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.474 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.474 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.474 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.474 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.474 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.474 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.475 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.475 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.475 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.475 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.475 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.475 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.475 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.475 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.475 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.475 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.475 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.475 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.475 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.475 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.476 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.476 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.476 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.476 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.476 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.476 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "localpool", 2024-08-31T06:23:25.476 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.476 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.476 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.476 INFO:teuthology.orchestra.run.smithi016.stdout: "failure_domain": { 2024-08-31T06:23:25.476 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "failure_domain", 2024-08-31T06:23:25.476 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.476 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.476 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.476 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "host", 2024-08-31T06:23:25.477 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.477 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.477 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.477 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "failure domain for any created local pool", 2024-08-31T06:23:25.477 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-08-31T06:23:25.477 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.477 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.477 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.477 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.477 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.477 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.477 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.477 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.477 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.478 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.478 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.478 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.478 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.478 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.478 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.478 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.478 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.478 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.478 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.478 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.478 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.478 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.478 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.479 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.479 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.479 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.479 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.479 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.479 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.479 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.479 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.479 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.479 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.479 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.479 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.479 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.479 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.480 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.480 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.480 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.480 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.480 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.480 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.480 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.480 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.480 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.480 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.480 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.480 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.480 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.480 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.481 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.481 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.481 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.481 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.481 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.481 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.481 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.481 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.481 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.481 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.481 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.481 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.481 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.482 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.482 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.482 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.482 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.482 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.482 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.482 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.482 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.482 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.482 INFO:teuthology.orchestra.run.smithi016.stdout: "min_size": { 2024-08-31T06:23:25.482 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_size", 2024-08-31T06:23:25.482 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.482 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.482 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.483 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.483 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.483 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.483 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.483 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default min_size for any created local pool", 2024-08-31T06:23:25.483 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-08-31T06:23:25.483 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.483 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.483 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.483 INFO:teuthology.orchestra.run.smithi016.stdout: "num_rep": { 2024-08-31T06:23:25.483 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "num_rep", 2024-08-31T06:23:25.483 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.483 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.483 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.483 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3", 2024-08-31T06:23:25.484 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.484 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.484 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.484 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default replica count for any created local pool", 2024-08-31T06:23:25.484 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.484 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.484 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.484 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.484 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num": { 2024-08-31T06:23:25.484 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_num", 2024-08-31T06:23:25.484 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.484 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.484 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.485 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "128", 2024-08-31T06:23:25.485 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.485 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.485 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.485 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default pg_num for any created local pool", 2024-08-31T06:23:25.485 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.485 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.485 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.485 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.485 INFO:teuthology.orchestra.run.smithi016.stdout: "prefix": { 2024-08-31T06:23:25.485 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prefix", 2024-08-31T06:23:25.485 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.485 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.485 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.486 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.486 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.486 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.486 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.486 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name prefix for any created local pool", 2024-08-31T06:23:25.486 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.486 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.486 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.486 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.486 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.486 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.486 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.486 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.486 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.487 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.487 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.487 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.487 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.487 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.487 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.487 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.487 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.487 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.487 INFO:teuthology.orchestra.run.smithi016.stdout: "subtree": { 2024-08-31T06:23:25.487 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "subtree", 2024-08-31T06:23:25.487 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.487 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.487 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.488 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "rack", 2024-08-31T06:23:25.488 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.488 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.488 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.488 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "CRUSH level for which to create a local pool", 2024-08-31T06:23:25.488 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-08-31T06:23:25.488 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.488 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.488 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.488 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.488 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.488 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.488 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mds_autoscaler", 2024-08-31T06:23:25.488 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.489 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.489 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.489 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.489 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.489 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.489 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.489 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.489 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.489 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.489 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.489 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.489 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.489 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.489 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.490 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.490 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.490 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.490 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.490 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.490 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.490 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.490 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.490 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.490 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.490 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.490 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.490 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.490 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.491 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.491 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.491 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.491 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.491 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.491 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.491 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.491 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.491 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.491 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.491 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.491 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.491 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.491 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.492 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.492 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.492 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.492 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.492 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.492 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.492 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.492 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.492 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.492 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.492 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.492 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.492 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.492 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.493 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.493 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.493 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.493 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.493 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.493 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.493 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.493 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.493 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.493 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.493 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.493 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.493 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.493 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.494 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.494 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.494 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.494 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.494 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.494 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.494 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.494 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.494 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.494 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.494 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.494 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.494 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.495 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.495 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.495 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.495 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.495 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.495 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.495 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirroring", 2024-08-31T06:23:25.495 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.495 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.495 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.496 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.496 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.496 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.496 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.496 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.496 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.496 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.496 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.496 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.496 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.496 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.496 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.496 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.496 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.496 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.497 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.497 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.497 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.497 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.497 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.497 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.497 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.497 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.497 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.497 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.497 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.497 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.497 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.498 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.498 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.498 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.498 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.498 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.498 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.498 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.498 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.498 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.498 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.498 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.498 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.498 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.498 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.499 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.499 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.499 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.499 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.499 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.499 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.499 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.499 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.499 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.499 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.499 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.499 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.499 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.499 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.499 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.499 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.500 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.500 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.500 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.500 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.500 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.500 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.500 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.500 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.500 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.500 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.500 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.500 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.500 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.500 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.501 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.501 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.501 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.501 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.501 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.501 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.501 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.501 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.501 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.501 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.501 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.501 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.501 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.501 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.501 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.502 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "nfs", 2024-08-31T06:23:25.502 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.502 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.502 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.502 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.502 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.502 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.502 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.502 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.502 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.502 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.502 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.502 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.502 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.503 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.503 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.503 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.503 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.503 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.503 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.503 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.503 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.503 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.503 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.503 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.503 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.503 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.503 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.504 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.504 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.504 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.504 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.504 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.504 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.504 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.504 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.504 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.504 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.504 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.504 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.504 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.504 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.505 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.505 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.505 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.505 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.505 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.505 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.505 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.505 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.505 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.505 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.505 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.505 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.505 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.505 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.505 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.506 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.506 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.506 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.506 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.506 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.506 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.506 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.506 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.506 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.506 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.506 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.506 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.506 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.506 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.506 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.507 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.507 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.507 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.507 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.507 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.507 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.507 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.507 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.507 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.507 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.507 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.507 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.507 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.507 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.507 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.508 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.508 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.508 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.508 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.508 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-08-31T06:23:25.508 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.508 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.508 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.508 INFO:teuthology.orchestra.run.smithi016.stdout: "fail_fs": { 2024-08-31T06:23:25.508 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "fail_fs", 2024-08-31T06:23:25.508 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.508 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.508 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.508 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.509 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.509 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.509 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.509 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-08-31T06:23:25.509 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.509 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.509 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.509 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.509 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.509 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.509 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.509 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.509 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.509 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.510 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.510 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.510 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.510 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.510 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.510 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.510 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.510 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.510 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.510 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.510 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.510 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.510 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.510 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.510 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.510 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.511 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.511 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.511 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.511 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.511 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.511 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.511 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.511 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.511 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.511 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.511 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.511 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.511 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.511 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.511 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.512 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.512 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.512 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.512 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.512 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.512 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.512 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.512 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.512 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.512 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.512 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.512 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.512 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.512 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.513 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.513 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.513 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.513 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.513 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.513 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.513 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.513 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.513 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.513 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.513 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.513 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.513 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.513 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.514 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.514 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.514 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.514 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.514 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.514 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator": { 2024-08-31T06:23:25.514 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-08-31T06:23:25.514 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.514 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.514 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.514 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.514 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.514 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.515 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.515 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2024-08-31T06:23:25.515 INFO:teuthology.orchestra.run.smithi016.stdout: "rook", 2024-08-31T06:23:25.515 INFO:teuthology.orchestra.run.smithi016.stdout: "test_orchestrator" 2024-08-31T06:23:25.515 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.515 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Orchestrator backend", 2024-08-31T06:23:25.515 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.515 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.515 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.515 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.515 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.515 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.515 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.515 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.515 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.515 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.516 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.516 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.516 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.516 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.516 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.516 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.516 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.516 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.516 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.516 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.516 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.516 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_perf_query", 2024-08-31T06:23:25.516 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.516 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.517 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.517 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.517 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.517 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.517 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.517 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.517 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.517 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.517 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.517 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.517 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.517 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.517 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.517 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.517 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.518 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.518 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.518 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.518 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.518 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.518 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.518 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.518 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.518 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.518 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.518 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.518 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.518 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.518 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.518 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.519 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.519 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.519 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.519 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.519 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.519 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.519 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.519 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.519 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.519 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.519 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.519 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.519 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.519 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.519 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.520 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.520 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.520 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.520 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.520 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.520 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.520 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.520 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.520 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.520 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.520 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.520 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.520 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.520 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.520 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.521 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.521 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.521 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.521 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.521 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.521 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.521 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.521 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.521 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.521 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.521 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.521 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.521 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.521 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.521 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.522 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.522 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.522 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.522 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.522 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.522 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.522 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.522 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.522 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.522 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.522 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.522 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.523 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.523 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_support", 2024-08-31T06:23:25.523 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.523 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.523 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.523 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.523 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.523 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.523 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.523 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.523 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.523 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.523 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.523 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.523 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.523 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.524 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.524 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.524 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.524 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.524 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.524 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.524 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.524 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.524 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.524 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.524 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.524 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.524 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.524 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.525 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.525 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.525 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.525 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.525 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.525 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.525 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.525 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.525 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.525 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.525 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.525 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.525 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.525 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.526 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.526 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.526 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.526 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.526 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.526 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.526 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.526 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.526 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.526 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.526 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.526 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.526 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.526 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.526 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.526 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.527 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.527 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.527 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.527 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.527 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.527 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.527 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.527 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.527 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.527 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.527 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.527 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.527 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.527 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.527 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.528 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.528 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.528 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.528 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.528 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.528 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.528 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.528 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.528 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.528 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.528 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.528 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.528 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.528 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.529 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.529 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.529 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.529 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_autoscaler", 2024-08-31T06:23:25.529 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.529 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.529 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.529 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.529 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.529 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.529 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.529 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.529 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.529 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.529 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.530 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.530 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.530 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.530 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.530 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.530 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.530 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.530 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.530 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.530 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.530 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.530 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.530 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.530 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.530 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.531 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.531 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.531 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.531 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.531 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.531 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.531 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.531 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.531 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.531 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.531 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.531 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.531 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.531 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.531 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.531 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.532 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.532 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.532 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.532 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.532 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.532 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.532 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.532 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.532 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.532 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.532 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.532 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.532 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.532 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.532 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.532 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.533 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.533 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.533 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.533 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.533 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.533 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.533 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.533 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.533 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.533 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.533 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.533 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.533 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.533 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.534 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.534 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-08-31T06:23:25.534 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-08-31T06:23:25.534 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:25.534 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.534 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.534 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-08-31T06:23:25.534 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.534 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.534 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.534 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.534 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.534 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.534 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.534 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.535 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.535 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.535 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.535 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.535 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.535 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.535 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.535 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.535 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.535 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.535 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.535 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.535 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.535 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.536 INFO:teuthology.orchestra.run.smithi016.stdout: "threshold": { 2024-08-31T06:23:25.536 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threshold", 2024-08-31T06:23:25.536 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-08-31T06:23:25.536 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.536 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.536 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-08-31T06:23:25.536 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1.0", 2024-08-31T06:23:25.536 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.536 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.536 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "scaling threshold", 2024-08-31T06:23:25.536 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-08-31T06:23:25.536 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.536 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.537 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.537 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.537 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.537 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.537 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "progress", 2024-08-31T06:23:25.537 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.537 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.537 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.537 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_pg_recovery_event": { 2024-08-31T06:23:25.537 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_pg_recovery_event", 2024-08-31T06:23:25.537 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.537 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.537 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.538 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.538 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.538 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.538 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.538 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow the module to show pg recovery progress", 2024-08-31T06:23:25.538 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.538 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.538 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.538 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.538 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-08-31T06:23:25.538 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-08-31T06:23:25.538 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.538 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.538 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.538 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.539 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.539 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.539 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.539 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.539 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.539 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.539 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.539 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.539 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.539 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.539 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.539 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.539 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.539 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.539 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.539 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.540 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.540 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.540 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.540 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.540 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.540 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.540 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.540 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.540 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.540 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.540 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.540 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.540 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.540 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.541 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.541 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.541 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.541 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.541 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.541 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.541 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.541 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.541 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.541 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.541 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.541 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.541 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.541 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.541 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.542 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.542 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.542 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.542 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.542 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.542 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.542 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.542 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.542 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.542 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.542 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.542 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.542 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.542 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.542 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.543 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.543 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.543 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.543 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.543 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.543 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.543 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.543 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.543 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.543 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.543 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.543 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.543 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.543 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.543 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.543 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.544 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.544 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.544 INFO:teuthology.orchestra.run.smithi016.stdout: "max_completed_events": { 2024-08-31T06:23:25.544 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_completed_events", 2024-08-31T06:23:25.544 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.544 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.544 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.544 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "50", 2024-08-31T06:23:25.544 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.544 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.544 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.544 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "number of past completed events to remember", 2024-08-31T06:23:25.544 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.544 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.545 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.545 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.545 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-08-31T06:23:25.545 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-08-31T06:23:25.545 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:25.545 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.545 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.545 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-08-31T06:23:25.545 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.545 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.545 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.545 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long the module is going to sleep", 2024-08-31T06:23:25.545 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.545 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.545 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.546 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.546 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.546 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.546 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.546 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.546 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.546 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.546 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.546 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.546 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.546 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.546 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.546 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.546 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.547 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.547 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.547 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.547 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.547 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus", 2024-08-31T06:23:25.547 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.547 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.547 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.547 INFO:teuthology.orchestra.run.smithi016.stdout: "cache": { 2024-08-31T06:23:25.547 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cache", 2024-08-31T06:23:25.547 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.547 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.547 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.547 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.547 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.547 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.548 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.548 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.548 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.548 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.548 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.548 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.548 INFO:teuthology.orchestra.run.smithi016.stdout: "exclude_perf_counters": { 2024-08-31T06:23:25.548 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "exclude_perf_counters", 2024-08-31T06:23:25.548 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.548 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.548 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.548 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.549 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.549 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.549 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.549 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-08-31T06:23:25.549 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-08-31T06:23:25.549 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.549 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.549 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.549 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.549 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.549 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.549 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.549 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.549 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.550 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.550 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.550 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.550 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.550 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.550 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.550 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.550 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.550 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.550 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.550 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.550 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.550 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.550 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.550 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.550 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.551 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.551 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.551 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.551 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.551 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.551 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.551 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.551 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.551 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.551 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.551 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.551 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.551 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.551 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.552 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.552 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.552 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.552 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.552 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.552 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.552 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.552 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.552 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.552 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.552 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.552 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.552 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.552 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.553 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.553 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.553 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.553 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.553 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.553 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.553 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.553 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.553 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.553 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.553 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.553 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.553 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.553 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.554 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.554 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.554 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.554 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.554 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.554 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.554 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools": { 2024-08-31T06:23:25.554 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools", 2024-08-31T06:23:25.554 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.554 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.554 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.554 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.554 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.554 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.554 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.555 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.555 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.555 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.555 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.555 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.555 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools_refresh_interval": { 2024-08-31T06:23:25.555 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-08-31T06:23:25.555 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.555 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.555 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.555 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "300", 2024-08-31T06:23:25.555 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.555 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.555 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.556 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.556 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.556 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.556 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.556 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.556 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_interval": { 2024-08-31T06:23:25.556 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_interval", 2024-08-31T06:23:25.556 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-08-31T06:23:25.556 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.556 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.556 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15.0", 2024-08-31T06:23:25.556 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.556 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.556 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.556 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.557 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.557 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.557 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.557 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.557 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-08-31T06:23:25.557 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-08-31T06:23:25.557 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.557 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.557 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.557 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-08-31T06:23:25.557 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.557 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.557 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.558 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-08-31T06:23:25.558 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.558 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.558 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.558 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.558 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-08-31T06:23:25.558 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-08-31T06:23:25.558 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.558 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.558 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.558 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "9283", 2024-08-31T06:23:25.558 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.558 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.558 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.559 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-08-31T06:23:25.559 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.559 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.559 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.559 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.559 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.559 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.559 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.559 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.559 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.559 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.559 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.559 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.559 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.559 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.560 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.560 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.560 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.560 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.560 INFO:teuthology.orchestra.run.smithi016.stdout: "stale_cache_strategy": { 2024-08-31T06:23:25.560 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stale_cache_strategy", 2024-08-31T06:23:25.560 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.560 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.560 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.560 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "log", 2024-08-31T06:23:25.560 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.560 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.560 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.560 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.561 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.561 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.561 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.561 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.561 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-08-31T06:23:25.561 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-08-31T06:23:25.561 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.561 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.561 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.561 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "default", 2024-08-31T06:23:25.561 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.561 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.561 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.562 INFO:teuthology.orchestra.run.smithi016.stdout: "default", 2024-08-31T06:23:25.562 INFO:teuthology.orchestra.run.smithi016.stdout: "error" 2024-08-31T06:23:25.562 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.562 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.562 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.562 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.562 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.562 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.562 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-08-31T06:23:25.562 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-08-31T06:23:25.562 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.562 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.562 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.562 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-08-31T06:23:25.562 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-08-31T06:23:25.563 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-08-31T06:23:25.563 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.563 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.563 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.563 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.563 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.563 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.563 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.563 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.563 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.563 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2024-08-31T06:23:25.563 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.563 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.563 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.563 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.564 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.564 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.564 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.564 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.564 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.564 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.564 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.564 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.564 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.564 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.564 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.564 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.564 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.564 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.564 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.565 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.565 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.565 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.565 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.565 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.565 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.565 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.565 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.565 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.565 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.565 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.565 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.565 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.565 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.566 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.566 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.566 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.566 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.566 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.566 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.566 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.566 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.566 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.566 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.566 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.566 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.566 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.566 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.566 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.567 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.567 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.567 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.567 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.567 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.567 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.567 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.567 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.567 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.567 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.567 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.567 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.567 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.567 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.567 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.568 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.568 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.568 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.568 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.568 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.568 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.568 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.568 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.568 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.568 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.568 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_snap_create": { 2024-08-31T06:23:25.568 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_snap_create", 2024-08-31T06:23:25.568 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.568 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.568 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.568 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-08-31T06:23:25.569 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.569 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.569 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.569 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.569 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.569 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.569 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.569 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.569 INFO:teuthology.orchestra.run.smithi016.stdout: "mirror_snapshot_schedule": { 2024-08-31T06:23:25.569 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirror_snapshot_schedule", 2024-08-31T06:23:25.569 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.569 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.569 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.569 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.570 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.570 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.570 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.570 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.570 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.570 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.570 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.570 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.570 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.570 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.570 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.570 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.570 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.570 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.570 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.570 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.570 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.571 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.571 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.571 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.571 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.571 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.571 INFO:teuthology.orchestra.run.smithi016.stdout: "trash_purge_schedule": { 2024-08-31T06:23:25.571 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "trash_purge_schedule", 2024-08-31T06:23:25.571 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.571 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.571 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.571 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.571 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.571 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.571 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.571 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.572 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.572 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.572 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.572 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.572 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.572 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.572 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.572 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "restful", 2024-08-31T06:23:25.572 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.572 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.572 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.572 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_auth": { 2024-08-31T06:23:25.572 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_auth", 2024-08-31T06:23:25.572 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.572 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.573 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.573 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.573 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.573 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.573 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.573 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.573 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.573 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.573 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.573 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.573 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-08-31T06:23:25.573 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-08-31T06:23:25.573 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.573 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.574 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.574 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.574 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.574 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.574 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.574 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.574 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.574 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.574 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.574 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.574 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.574 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.574 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.574 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.575 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.575 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.575 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.575 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.575 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.575 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.575 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.575 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.575 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.575 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.575 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.575 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.575 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.575 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.576 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.576 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.576 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.576 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.576 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.576 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.576 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.576 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.576 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.576 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.576 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.576 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.576 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.576 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.577 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.577 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.577 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.577 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.577 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.577 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.577 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.577 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.577 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.577 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.577 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.577 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.577 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.577 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.578 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.578 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.578 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.578 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.578 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.578 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.578 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.578 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.578 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.578 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.578 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.578 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.578 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.579 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.579 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.579 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.579 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.579 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.579 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.579 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.579 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.579 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.579 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.579 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.579 INFO:teuthology.orchestra.run.smithi016.stdout: "max_requests": { 2024-08-31T06:23:25.579 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_requests", 2024-08-31T06:23:25.579 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.580 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.580 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.580 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-08-31T06:23:25.580 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.580 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.580 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.580 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-08-31T06:23:25.580 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.580 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.580 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.580 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.580 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-08-31T06:23:25.580 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-08-31T06:23:25.580 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.581 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.581 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.581 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.581 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.581 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.581 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.581 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.581 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.581 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.581 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.581 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.581 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-08-31T06:23:25.581 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-08-31T06:23:25.581 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.582 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.582 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.582 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.582 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.582 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.582 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.582 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.582 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.582 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.582 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.582 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.582 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.582 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.582 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.582 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.583 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.583 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.583 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.583 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.583 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.583 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.583 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.583 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.583 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.583 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.583 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.583 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.583 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.583 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rgw", 2024-08-31T06:23:25.584 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.584 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.584 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.584 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.584 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.584 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.584 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.584 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.584 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.584 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.584 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.584 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.584 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.584 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.585 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.585 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.585 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.585 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.585 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.585 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.585 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.585 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.585 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.585 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.585 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.585 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.586 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.586 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.586 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.586 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.586 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.586 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.586 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.586 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.586 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.586 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.586 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.586 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.586 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.587 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.587 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.587 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.587 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.587 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.587 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.587 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.587 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.587 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.587 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.587 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.587 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.587 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.587 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.588 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.588 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.588 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.588 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.588 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.588 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.588 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.588 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.588 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.588 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.588 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.588 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.588 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.589 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.589 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.589 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.589 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.589 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.589 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.589 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.589 INFO:teuthology.orchestra.run.smithi016.stdout: "secondary_zone_period_retry_limit": { 2024-08-31T06:23:25.589 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secondary_zone_period_retry_limit", 2024-08-31T06:23:25.589 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.589 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.589 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.589 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-08-31T06:23:25.590 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.590 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.590 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.590 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-08-31T06:23:25.590 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.590 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.590 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.590 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.590 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.590 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.590 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.590 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.590 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.591 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.591 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.591 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.591 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.591 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.591 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.591 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.591 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.591 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.591 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.591 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.591 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.591 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rook", 2024-08-31T06:23:25.591 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.592 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.592 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.592 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.592 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.592 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.592 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.592 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.592 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.592 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.592 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.592 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.592 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.592 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.592 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.592 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.593 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.593 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.593 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.593 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.593 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.593 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.593 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.593 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.593 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.593 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.593 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.593 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.593 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.594 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.594 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.594 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.594 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.594 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.594 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.594 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.594 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.594 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.594 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.594 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.594 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.594 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.595 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.595 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.595 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.595 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.595 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.595 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.595 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.595 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.595 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.595 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.595 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.595 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.595 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.596 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.596 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.596 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.596 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.596 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.596 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.596 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.596 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.596 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.596 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.596 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.596 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.596 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.596 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.597 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.597 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.597 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.597 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.597 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.597 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.597 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.597 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.597 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.597 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.597 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.597 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.597 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.597 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.598 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.598 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.598 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.598 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.598 INFO:teuthology.orchestra.run.smithi016.stdout: "storage_class": { 2024-08-31T06:23:25.598 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "storage_class", 2024-08-31T06:23:25.598 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.598 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.598 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.598 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "local", 2024-08-31T06:23:25.598 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.598 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.598 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.598 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-08-31T06:23:25.599 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.599 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.599 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.599 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.599 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.599 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.599 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.599 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "selftest", 2024-08-31T06:23:25.599 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.599 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.599 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.599 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.599 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.599 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.599 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.600 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.600 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.600 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.600 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.600 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.600 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.600 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.600 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.600 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.600 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.600 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.600 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.600 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.600 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.601 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.601 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.601 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.601 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.601 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.601 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.601 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.601 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.601 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.601 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.601 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.601 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.601 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.602 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.602 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.602 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.602 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.602 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.602 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.602 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.602 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.602 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.602 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.602 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.602 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.603 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.603 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.603 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.603 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.603 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.603 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.603 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.603 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.603 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.603 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.603 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.603 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.603 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.603 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.604 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.604 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.604 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.604 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.604 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.604 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.604 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.604 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.604 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.604 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.604 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.604 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.605 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.605 INFO:teuthology.orchestra.run.smithi016.stdout: "roption1": { 2024-08-31T06:23:25.605 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption1", 2024-08-31T06:23:25.605 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.605 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.605 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.605 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.605 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.605 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.605 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.605 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.605 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.605 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.605 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.606 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.606 INFO:teuthology.orchestra.run.smithi016.stdout: "roption2": { 2024-08-31T06:23:25.606 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption2", 2024-08-31T06:23:25.606 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.606 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.606 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.606 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "xyz", 2024-08-31T06:23:25.606 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.606 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.606 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.606 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.606 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.606 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.607 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.607 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.607 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption1": { 2024-08-31T06:23:25.607 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption1", 2024-08-31T06:23:25.607 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.607 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.607 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.607 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.607 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.607 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.607 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.607 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.607 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.607 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.608 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.608 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.608 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption2": { 2024-08-31T06:23:25.608 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption2", 2024-08-31T06:23:25.608 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.608 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.608 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.608 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.608 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.608 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.608 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.608 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.608 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.608 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.609 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.609 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.609 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption3": { 2024-08-31T06:23:25.609 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption3", 2024-08-31T06:23:25.609 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-08-31T06:23:25.609 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.609 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.609 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.609 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.609 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.609 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.609 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.609 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.609 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.610 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.610 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.610 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption4": { 2024-08-31T06:23:25.610 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption4", 2024-08-31T06:23:25.610 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.610 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.610 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.610 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.610 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.610 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.610 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.610 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.610 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.610 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.610 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.611 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.611 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption5": { 2024-08-31T06:23:25.611 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption5", 2024-08-31T06:23:25.611 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.611 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.611 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.611 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.611 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.611 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.611 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.611 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.611 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.611 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.612 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.612 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.612 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption6": { 2024-08-31T06:23:25.612 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption6", 2024-08-31T06:23:25.612 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.612 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.612 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.612 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.612 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.612 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.612 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.612 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.612 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.612 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.613 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.613 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.613 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption7": { 2024-08-31T06:23:25.613 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption7", 2024-08-31T06:23:25.613 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.613 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.613 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.613 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.613 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-08-31T06:23:25.613 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "42", 2024-08-31T06:23:25.613 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.613 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.613 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.613 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.614 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.614 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.614 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.614 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.614 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.614 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.614 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.614 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.614 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.614 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.614 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.614 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.614 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.614 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.615 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.615 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.615 INFO:teuthology.orchestra.run.smithi016.stdout: "testkey": { 2024-08-31T06:23:25.615 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testkey", 2024-08-31T06:23:25.615 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.615 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.615 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.615 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.615 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.615 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.615 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.615 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.615 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.615 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.616 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.616 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.616 INFO:teuthology.orchestra.run.smithi016.stdout: "testlkey": { 2024-08-31T06:23:25.616 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testlkey", 2024-08-31T06:23:25.616 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.616 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.616 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.616 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.616 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.616 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.616 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.616 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.616 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.617 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.617 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.617 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.617 INFO:teuthology.orchestra.run.smithi016.stdout: "testnewline": { 2024-08-31T06:23:25.617 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testnewline", 2024-08-31T06:23:25.617 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.617 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.617 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.617 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.617 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.617 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.617 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.617 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.617 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.618 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.618 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.618 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.618 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.618 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.618 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.618 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smb", 2024-08-31T06:23:25.618 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.618 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.618 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.618 INFO:teuthology.orchestra.run.smithi016.stdout: "internal_store_backend": { 2024-08-31T06:23:25.618 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "internal_store_backend", 2024-08-31T06:23:25.618 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.619 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.619 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.619 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.619 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.619 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.619 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.619 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-08-31T06:23:25.619 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.619 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.619 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.619 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.619 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.619 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.620 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.620 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.620 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.620 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.620 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.620 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.620 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.620 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.620 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.620 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.620 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.620 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.620 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.620 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.621 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.621 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.621 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.621 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.621 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.621 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.621 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.621 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.621 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.621 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.621 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.621 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.621 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.621 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.622 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.622 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.622 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.622 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.622 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.622 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.622 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.622 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.622 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.622 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.622 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.622 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.622 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.622 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.623 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.623 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.623 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.623 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.623 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.623 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.623 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.623 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.623 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.623 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.623 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.623 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.623 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.623 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.623 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.624 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.624 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.624 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.624 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.624 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.624 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.624 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.624 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.624 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.624 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.624 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.624 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.624 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.624 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.625 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.625 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.625 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.625 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.625 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.625 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.625 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.625 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.625 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.625 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.625 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.625 INFO:teuthology.orchestra.run.smithi016.stdout: "update_orchestration": { 2024-08-31T06:23:25.625 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "update_orchestration", 2024-08-31T06:23:25.625 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.626 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.626 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.626 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.626 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.626 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.626 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.626 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-08-31T06:23:25.626 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.626 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.626 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.626 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.626 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.626 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.626 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.626 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-08-31T06:23:25.627 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.627 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.627 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.627 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_m_granularity": { 2024-08-31T06:23:25.627 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_m_granularity", 2024-08-31T06:23:25.627 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.627 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.627 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.627 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.627 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.627 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.627 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.627 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow minute scheduled snapshots", 2024-08-31T06:23:25.627 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.628 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.628 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.628 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.628 INFO:teuthology.orchestra.run.smithi016.stdout: "dump_on_update": { 2024-08-31T06:23:25.628 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dump_on_update", 2024-08-31T06:23:25.628 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.628 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.628 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.628 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.628 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.628 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.628 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.628 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "dump database to debug log on update", 2024-08-31T06:23:25.628 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.628 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.629 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.629 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.629 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.629 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.629 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.629 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.629 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.629 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.629 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.629 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.629 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.629 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.629 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.629 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.629 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.630 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.630 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.630 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.630 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.630 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.630 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.630 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.630 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.630 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.630 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.630 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.630 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.630 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.631 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.631 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.631 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.631 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.631 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.631 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.631 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.631 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.631 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.631 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.631 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.631 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.631 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.631 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.631 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.632 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.632 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.632 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.632 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.632 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.632 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.632 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.632 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.632 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.632 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.632 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.632 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.632 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.633 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.633 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.633 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.633 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.633 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.633 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.633 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.633 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.633 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.633 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.633 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.633 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.633 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.633 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.634 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.634 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.634 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.634 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.634 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.634 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.634 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.634 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.634 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.634 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.634 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.634 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.634 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.634 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.635 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.635 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.635 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.635 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.635 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.635 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stats", 2024-08-31T06:23:25.635 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.635 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.635 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.635 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.635 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.635 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.635 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.635 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.636 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.636 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.636 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.636 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.636 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.636 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.636 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.636 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.636 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.636 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.636 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.636 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.636 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.636 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.637 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.637 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.637 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.637 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.637 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.637 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.637 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.637 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.637 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.637 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.637 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.637 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.637 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.637 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.638 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.638 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.638 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.638 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.638 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.638 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.638 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.638 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.638 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.638 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.638 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.638 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.638 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.638 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.639 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.639 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.639 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.639 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.639 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.639 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.639 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.639 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.639 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.639 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.639 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.639 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.639 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.639 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.639 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.640 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.640 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.640 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.640 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.640 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.640 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.640 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.640 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.640 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.640 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.640 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.640 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.640 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.641 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.641 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.641 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.641 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.641 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.641 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.641 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.641 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.641 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.641 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.641 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.641 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.641 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "status", 2024-08-31T06:23:25.641 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.642 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.642 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.642 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.642 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.642 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.642 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.642 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.642 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.642 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.642 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.642 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.642 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.642 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.642 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.643 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.643 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.643 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.643 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.643 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.643 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.643 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.643 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.643 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.643 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.643 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.643 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.643 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.643 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.644 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.644 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.644 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.644 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.644 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.644 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.644 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.644 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.644 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.644 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.644 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.644 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.644 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.645 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.645 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.645 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.645 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.645 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.645 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.645 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.645 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.645 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.645 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.645 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.645 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.645 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.646 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.646 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.646 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.646 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.646 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.646 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.646 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.646 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.646 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.646 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.646 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.646 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.646 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.646 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.647 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.647 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.647 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.647 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.647 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.647 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.647 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.647 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.647 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.647 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.647 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.647 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.647 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.648 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.648 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.648 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.648 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.648 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.648 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.648 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.648 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.648 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telegraf", 2024-08-31T06:23:25.648 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.648 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.648 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.648 INFO:teuthology.orchestra.run.smithi016.stdout: "address": { 2024-08-31T06:23:25.649 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "address", 2024-08-31T06:23:25.649 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.649 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.649 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.649 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-08-31T06:23:25.649 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.649 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.649 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.649 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.649 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.649 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.649 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.649 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.649 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-08-31T06:23:25.650 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-08-31T06:23:25.650 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:25.650 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.650 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.650 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15", 2024-08-31T06:23:25.650 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.650 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.650 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.650 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.650 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.650 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.650 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.650 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.650 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.651 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.651 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.651 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.651 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.651 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.651 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.651 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.651 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.651 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.651 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.651 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.651 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.651 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.652 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.652 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.652 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.652 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.652 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.652 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.652 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.652 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.652 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.652 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.652 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.652 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.652 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.652 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.652 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.653 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.653 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.653 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.653 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.653 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.653 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.653 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.653 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.653 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.653 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.653 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.653 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.653 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.653 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.654 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.654 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.654 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.654 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.654 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.654 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.654 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.654 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.654 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.654 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.654 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.654 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.654 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.654 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.655 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.655 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.655 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.655 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.655 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.655 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.655 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.655 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.655 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.655 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.655 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.655 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.655 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.655 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.656 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.656 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.656 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.656 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.656 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.656 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.656 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.656 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.656 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.656 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.656 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.656 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.656 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.657 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.657 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.657 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.657 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telemetry", 2024-08-31T06:23:25.657 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.657 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.657 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.657 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_basic": { 2024-08-31T06:23:25.657 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_basic", 2024-08-31T06:23:25.657 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.657 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.657 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.657 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.657 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.657 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.658 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.658 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share basic cluster information (size, version)", 2024-08-31T06:23:25.658 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.658 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.658 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.658 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.658 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_crash": { 2024-08-31T06:23:25.658 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_crash", 2024-08-31T06:23:25.658 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.658 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.658 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.658 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.659 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.659 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.659 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.659 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-08-31T06:23:25.659 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.659 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.659 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.659 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.659 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_device": { 2024-08-31T06:23:25.659 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_device", 2024-08-31T06:23:25.659 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.659 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.659 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.659 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.660 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.660 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.660 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.660 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-08-31T06:23:25.660 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.660 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.660 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.660 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.660 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_ident": { 2024-08-31T06:23:25.660 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_ident", 2024-08-31T06:23:25.660 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.660 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.660 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.660 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.661 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.661 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.661 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.661 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-08-31T06:23:25.661 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.661 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.661 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.661 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.661 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_perf": { 2024-08-31T06:23:25.661 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_perf", 2024-08-31T06:23:25.661 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.661 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.661 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.662 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.662 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.662 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.662 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.662 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share various performance metrics of a cluster", 2024-08-31T06:23:25.662 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.662 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.662 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.662 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.662 INFO:teuthology.orchestra.run.smithi016.stdout: "contact": { 2024-08-31T06:23:25.662 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "contact", 2024-08-31T06:23:25.662 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.662 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.663 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.663 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.663 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.663 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.663 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.663 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.663 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.663 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.663 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.663 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.663 INFO:teuthology.orchestra.run.smithi016.stdout: "description": { 2024-08-31T06:23:25.663 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "description", 2024-08-31T06:23:25.663 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.663 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.664 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.664 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.664 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.664 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.664 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.664 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.664 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.664 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.664 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.664 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.664 INFO:teuthology.orchestra.run.smithi016.stdout: "device_url": { 2024-08-31T06:23:25.664 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_url", 2024-08-31T06:23:25.664 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.664 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.665 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.665 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-08-31T06:23:25.665 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.665 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.665 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.665 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.665 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.665 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.665 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.665 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.665 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-08-31T06:23:25.665 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-08-31T06:23:25.665 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.665 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.666 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.666 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.666 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.666 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.666 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.666 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.666 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.666 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.666 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.666 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.666 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-08-31T06:23:25.666 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-08-31T06:23:25.666 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.668 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.668 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.668 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "24", 2024-08-31T06:23:25.668 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "8", 2024-08-31T06:23:25.668 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.668 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.668 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.668 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.668 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.668 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.668 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.669 INFO:teuthology.orchestra.run.smithi016.stdout: "last_opt_revision": { 2024-08-31T06:23:25.669 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "last_opt_revision", 2024-08-31T06:23:25.669 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.669 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.669 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.669 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1", 2024-08-31T06:23:25.669 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.669 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.669 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.669 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.669 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.669 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.669 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.669 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.670 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard": { 2024-08-31T06:23:25.670 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard", 2024-08-31T06:23:25.670 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.670 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.670 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.670 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.670 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.670 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.670 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.670 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.670 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.670 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.670 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.670 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.671 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard_description": { 2024-08-31T06:23:25.671 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard_description", 2024-08-31T06:23:25.671 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.671 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.671 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.671 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.671 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.671 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.671 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.671 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.671 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.671 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.671 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.671 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.672 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.672 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.672 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.672 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.672 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.672 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.672 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.672 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.672 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.672 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.672 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.672 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.672 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.672 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.673 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.673 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.673 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.673 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.673 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.673 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.673 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.673 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.673 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.673 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.673 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.673 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.673 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.674 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.674 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.674 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.674 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.674 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.674 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.674 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.674 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.674 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.674 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.674 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.674 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.674 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.674 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.674 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.675 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.675 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.675 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.675 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.675 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.675 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.675 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.675 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.675 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.675 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.675 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.675 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.675 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.675 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.676 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.676 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.676 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.676 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.676 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.676 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.676 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.676 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.676 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.676 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.676 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.676 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.676 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.677 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.677 INFO:teuthology.orchestra.run.smithi016.stdout: "organization": { 2024-08-31T06:23:25.677 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "organization", 2024-08-31T06:23:25.677 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.677 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.677 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.677 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.677 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.677 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.677 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.677 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.677 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.677 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.677 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.678 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.678 INFO:teuthology.orchestra.run.smithi016.stdout: "proxy": { 2024-08-31T06:23:25.678 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "proxy", 2024-08-31T06:23:25.678 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.678 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.678 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.678 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.678 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.678 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.678 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.678 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.678 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.678 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.678 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.678 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.679 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.679 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.679 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.679 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.679 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.679 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.679 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.679 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.679 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.679 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.679 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.679 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.679 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.680 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.680 INFO:teuthology.orchestra.run.smithi016.stdout: "url": { 2024-08-31T06:23:25.680 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url", 2024-08-31T06:23:25.680 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.680 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.680 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.680 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-08-31T06:23:25.680 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.680 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.681 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.681 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.681 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.681 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.681 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.681 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.681 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.681 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.681 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.681 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "test_orchestrator", 2024-08-31T06:23:25.681 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.681 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.681 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.682 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.682 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.682 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.682 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.682 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.682 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.682 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.682 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.682 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.682 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.682 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.682 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.682 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.682 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.683 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.683 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.683 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.683 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.683 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.683 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.683 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.683 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.683 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.683 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.683 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.683 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.683 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.684 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.684 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.684 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.684 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.684 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.684 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.684 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.684 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.684 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.684 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.684 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.684 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.684 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.684 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.685 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.685 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.685 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.685 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.685 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.685 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.685 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.685 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.685 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.685 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.685 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.685 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.685 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.685 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.686 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.686 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.686 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.686 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.686 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.686 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.686 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.686 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.686 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.686 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.686 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.686 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.686 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.686 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.687 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.687 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.687 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.687 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.687 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.687 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.687 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.687 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.687 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.687 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.687 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.687 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.688 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.688 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.688 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.688 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.688 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.688 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.688 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2024-08-31T06:23:25.688 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.688 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.688 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.688 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.688 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.688 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.688 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.689 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.689 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.689 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.689 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.689 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.689 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.689 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.689 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.689 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.689 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.689 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.689 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.689 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.689 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.690 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.690 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.690 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.690 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.690 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.690 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.690 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.690 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.690 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.690 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.690 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.690 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.690 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.691 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.691 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.691 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.691 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.691 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.691 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.691 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.691 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.691 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.691 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.691 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.691 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.691 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.691 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.692 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.692 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.692 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.692 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.692 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.692 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.692 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.692 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.692 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.692 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.692 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.692 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.692 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.692 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.693 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.693 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.693 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.693 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.693 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.693 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.693 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.693 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.693 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.693 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.693 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.693 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_clones": { 2024-08-31T06:23:25.693 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_clones", 2024-08-31T06:23:25.694 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.694 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.694 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.694 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4", 2024-08-31T06:23:25.694 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.694 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.694 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.694 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Number of asynchronous cloner threads", 2024-08-31T06:23:25.694 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.694 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.694 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.694 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.694 INFO:teuthology.orchestra.run.smithi016.stdout: "periodic_async_work": { 2024-08-31T06:23:25.694 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "periodic_async_work", 2024-08-31T06:23:25.695 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.695 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.695 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.695 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.695 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.695 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.695 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.695 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Periodically check for async work", 2024-08-31T06:23:25.695 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.695 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.695 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.695 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.695 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_delay": { 2024-08-31T06:23:25.695 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_delay", 2024-08-31T06:23:25.696 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.696 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.696 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.696 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.696 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.696 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.696 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.696 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-08-31T06:23:25.696 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.696 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.696 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.696 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.696 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_no_wait": { 2024-08-31T06:23:25.696 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_no_wait", 2024-08-31T06:23:25.697 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.697 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.697 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.697 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.697 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.697 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.697 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.697 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-08-31T06:23:25.697 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.697 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.697 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.697 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.697 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.697 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.698 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.698 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.698 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.698 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.698 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.698 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.698 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.698 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.698 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.698 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.698 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.698 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.698 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.698 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.699 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.699 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix", 2024-08-31T06:23:25.699 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.699 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.699 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.699 INFO:teuthology.orchestra.run.smithi016.stdout: "discovery_interval": { 2024-08-31T06:23:25.699 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "discovery_interval", 2024-08-31T06:23:25.699 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-08-31T06:23:25.699 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.699 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.699 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "100", 2024-08-31T06:23:25.699 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.699 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.699 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.699 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.700 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.700 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.700 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.700 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.700 INFO:teuthology.orchestra.run.smithi016.stdout: "identifier": { 2024-08-31T06:23:25.700 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "identifier", 2024-08-31T06:23:25.700 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.700 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.700 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.700 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.700 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.700 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.700 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.700 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.701 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.701 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.701 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.701 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.701 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-08-31T06:23:25.701 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-08-31T06:23:25.701 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:25.701 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.701 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.701 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-08-31T06:23:25.701 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.701 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.701 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.702 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.702 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.702 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.702 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.702 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.702 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.702 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.702 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.702 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.702 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.702 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.702 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.702 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.702 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.703 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.703 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.703 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.703 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.703 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.703 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.703 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.703 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.703 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.703 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.703 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.703 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.703 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.704 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.704 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.704 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.704 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.704 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.704 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.704 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.704 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.704 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.704 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.704 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.704 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.704 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.704 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.705 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.705 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.705 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.705 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.705 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.705 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.705 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.705 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.705 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.705 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.705 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.705 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.705 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.705 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.706 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.706 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.706 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.706 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.706 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.706 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.706 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.706 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.706 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.706 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.706 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.706 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.706 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.706 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.707 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.707 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.707 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.707 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.707 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.707 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.707 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.707 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.707 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.707 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.707 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.707 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.708 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.708 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.708 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.708 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.708 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.708 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.708 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.708 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.708 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_host": { 2024-08-31T06:23:25.708 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_host", 2024-08-31T06:23:25.708 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.708 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.708 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.708 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.709 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.709 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.709 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.709 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.709 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.709 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.709 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.709 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.709 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_port": { 2024-08-31T06:23:25.709 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_port", 2024-08-31T06:23:25.709 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.709 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.709 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.709 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10051", 2024-08-31T06:23:25.710 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.710 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.710 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.710 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.710 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.710 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.710 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.710 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.710 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_sender": { 2024-08-31T06:23:25.710 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_sender", 2024-08-31T06:23:25.710 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.710 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.710 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.710 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-08-31T06:23:25.711 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.711 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.711 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.711 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.711 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.711 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.711 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.711 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.711 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.711 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.711 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-08-31T06:23:25.711 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.711 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.711 INFO:teuthology.orchestra.run.smithi016.stdout: "gid": 24182, 2024-08-31T06:23:25.712 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "y", 2024-08-31T06:23:25.712 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr_features": 4540701547738038271, 2024-08-31T06:23:25.712 INFO:teuthology.orchestra.run.smithi016.stdout: "available_modules": [ 2024-08-31T06:23:25.712 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.712 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "alerts", 2024-08-31T06:23:25.712 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.712 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.712 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.712 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-08-31T06:23:25.712 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-08-31T06:23:25.712 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:25.712 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.712 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.712 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-08-31T06:23:25.713 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.713 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.713 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.713 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How frequently to reexamine health status", 2024-08-31T06:23:25.713 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.713 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.713 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.713 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.713 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.713 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.713 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.713 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.713 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.713 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.714 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.714 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.714 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.714 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.714 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.714 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.714 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.714 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.714 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.714 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.714 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.714 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.714 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.714 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.715 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.715 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.715 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.715 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.715 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.715 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.715 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.715 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.715 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.715 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.715 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.715 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.715 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.715 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.716 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.716 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.716 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.716 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.716 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.716 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.716 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.716 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.716 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.716 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.716 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.716 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.716 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.717 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.717 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.717 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.717 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.717 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.717 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.717 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.717 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.717 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.717 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.717 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.717 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.717 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.718 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.718 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.718 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.718 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.718 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.718 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.718 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.718 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.718 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.718 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.718 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_destination": { 2024-08-31T06:23:25.718 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_destination", 2024-08-31T06:23:25.718 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.718 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.719 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.719 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.719 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.719 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.719 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.719 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email address to send alerts to", 2024-08-31T06:23:25.719 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.719 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.719 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.719 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.719 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_from_name": { 2024-08-31T06:23:25.719 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_from_name", 2024-08-31T06:23:25.719 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.719 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.720 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.720 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "Ceph", 2024-08-31T06:23:25.720 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.720 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.720 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.720 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email From: name", 2024-08-31T06:23:25.720 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.720 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.720 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.720 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.720 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_host": { 2024-08-31T06:23:25.720 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_host", 2024-08-31T06:23:25.720 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.721 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.721 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.721 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.721 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.721 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.721 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.721 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP server", 2024-08-31T06:23:25.721 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.721 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.721 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.721 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.721 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_password": { 2024-08-31T06:23:25.721 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_password", 2024-08-31T06:23:25.721 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.722 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.722 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.722 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.722 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.722 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.722 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.722 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Password to authenticate with", 2024-08-31T06:23:25.722 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.722 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.722 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.722 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.722 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_port": { 2024-08-31T06:23:25.722 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_port", 2024-08-31T06:23:25.722 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.723 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.723 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.723 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "465", 2024-08-31T06:23:25.723 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.723 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.723 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.723 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP port", 2024-08-31T06:23:25.723 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.723 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.723 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.723 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.723 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_sender": { 2024-08-31T06:23:25.723 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_sender", 2024-08-31T06:23:25.723 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.724 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.724 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.724 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.724 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.724 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.724 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.724 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP envelope sender", 2024-08-31T06:23:25.724 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.724 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.724 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.724 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.724 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_ssl": { 2024-08-31T06:23:25.724 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_ssl", 2024-08-31T06:23:25.724 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.725 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.725 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.725 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.725 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.725 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.725 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.725 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use SSL to connect to SMTP server", 2024-08-31T06:23:25.725 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.725 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.725 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.725 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.725 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_user": { 2024-08-31T06:23:25.725 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_user", 2024-08-31T06:23:25.725 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.726 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.726 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.726 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.726 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.726 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.726 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.726 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "User to authenticate as", 2024-08-31T06:23:25.726 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.726 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.726 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.726 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.726 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.726 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.726 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.726 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.727 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.727 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.727 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.727 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.727 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.727 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.727 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.727 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.727 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.727 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.727 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.727 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.727 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.727 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "balancer", 2024-08-31T06:23:25.727 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.728 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.728 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.728 INFO:teuthology.orchestra.run.smithi016.stdout: "active": { 2024-08-31T06:23:25.728 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "active", 2024-08-31T06:23:25.728 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.728 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.728 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.728 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.728 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.728 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.728 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.728 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically balance PGs across cluster", 2024-08-31T06:23:25.728 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.728 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.728 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.729 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.729 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_time": { 2024-08-31T06:23:25.729 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_time", 2024-08-31T06:23:25.729 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.729 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.729 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.729 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0000", 2024-08-31T06:23:25.729 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.729 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.729 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.729 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "beginning time of day to automatically balance", 2024-08-31T06:23:25.729 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-31T06:23:25.729 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.730 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.730 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.730 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_weekday": { 2024-08-31T06:23:25.730 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_weekday", 2024-08-31T06:23:25.730 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-08-31T06:23:25.730 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.730 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.730 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.730 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-08-31T06:23:25.730 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-08-31T06:23:25.730 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.730 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-08-31T06:23:25.730 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-31T06:23:25.730 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.730 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.731 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.731 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_max_iterations": { 2024-08-31T06:23:25.731 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_max_iterations", 2024-08-31T06:23:25.731 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-08-31T06:23:25.731 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.731 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.731 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "25", 2024-08-31T06:23:25.731 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-08-31T06:23:25.731 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "250", 2024-08-31T06:23:25.731 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.731 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-08-31T06:23:25.731 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.731 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.732 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.732 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.732 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_metrics": { 2024-08-31T06:23:25.732 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_metrics", 2024-08-31T06:23:25.732 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.732 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.732 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.732 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "pgs,objects,bytes", 2024-08-31T06:23:25.732 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.732 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.732 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.732 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-08-31T06:23:25.732 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-08-31T06:23:25.732 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.733 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.733 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.733 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_step": { 2024-08-31T06:23:25.733 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_step", 2024-08-31T06:23:25.733 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-08-31T06:23:25.733 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.733 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.733 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.5", 2024-08-31T06:23:25.733 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0.001", 2024-08-31T06:23:25.733 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "0.999", 2024-08-31T06:23:25.733 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.733 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "aggressiveness of optimization", 2024-08-31T06:23:25.733 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-08-31T06:23:25.733 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.734 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.734 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.734 INFO:teuthology.orchestra.run.smithi016.stdout: "end_time": { 2024-08-31T06:23:25.734 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_time", 2024-08-31T06:23:25.734 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.734 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.734 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.734 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2359", 2024-08-31T06:23:25.734 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.734 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.734 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.734 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ending time of day to automatically balance", 2024-08-31T06:23:25.734 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-31T06:23:25.735 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.735 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.735 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.735 INFO:teuthology.orchestra.run.smithi016.stdout: "end_weekday": { 2024-08-31T06:23:25.735 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_weekday", 2024-08-31T06:23:25.735 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-08-31T06:23:25.735 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.735 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.735 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.735 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-08-31T06:23:25.735 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-08-31T06:23:25.735 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.735 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-08-31T06:23:25.735 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-31T06:23:25.735 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.736 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.736 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.736 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.736 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.736 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.736 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.736 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.736 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.736 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.736 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.736 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.736 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.736 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.736 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.737 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.737 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.737 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.737 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.737 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.737 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.737 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.737 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.737 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.737 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.737 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.737 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.737 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.738 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.738 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.738 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.738 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.738 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.738 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.738 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.738 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.738 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.738 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.738 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.738 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.738 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.738 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.738 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.739 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.739 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.739 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.739 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.739 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.739 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.739 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.739 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.739 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.739 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.739 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.739 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.739 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.739 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.740 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.740 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.740 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.740 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.740 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.740 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.740 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.740 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.740 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.740 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.740 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.740 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.740 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.740 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.741 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.741 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.741 INFO:teuthology.orchestra.run.smithi016.stdout: "min_score": { 2024-08-31T06:23:25.741 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_score", 2024-08-31T06:23:25.741 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-08-31T06:23:25.741 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.741 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.741 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.741 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.741 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.741 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.741 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-08-31T06:23:25.741 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.742 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.742 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.742 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.742 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-08-31T06:23:25.742 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-08-31T06:23:25.742 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.742 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.742 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.742 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "upmap", 2024-08-31T06:23:25.742 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.742 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.742 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.742 INFO:teuthology.orchestra.run.smithi016.stdout: "crush-compat", 2024-08-31T06:23:25.742 INFO:teuthology.orchestra.run.smithi016.stdout: "none", 2024-08-31T06:23:25.743 INFO:teuthology.orchestra.run.smithi016.stdout: "read", 2024-08-31T06:23:25.743 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap", 2024-08-31T06:23:25.743 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap-read" 2024-08-31T06:23:25.743 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.743 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Balancer mode", 2024-08-31T06:23:25.743 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.743 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.743 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.743 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.743 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_ids": { 2024-08-31T06:23:25.743 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_ids", 2024-08-31T06:23:25.743 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.743 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.743 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.744 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.744 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.744 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.744 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.744 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-08-31T06:23:25.744 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.744 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.744 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.744 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.744 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-08-31T06:23:25.744 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-08-31T06:23:25.744 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:25.744 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.745 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.745 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-08-31T06:23:25.745 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.745 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.745 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.745 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-08-31T06:23:25.745 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.745 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.745 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.745 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.745 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.745 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.745 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.745 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.746 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.746 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.746 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.746 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.746 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.746 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.746 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.746 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.746 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.746 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.746 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_deviation": { 2024-08-31T06:23:25.746 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_deviation", 2024-08-31T06:23:25.746 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.747 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.747 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.747 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-08-31T06:23:25.747 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-08-31T06:23:25.747 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.747 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.747 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "deviation below which no optimization is attempted", 2024-08-31T06:23:25.747 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-08-31T06:23:25.747 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.747 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.747 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.747 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_optimizations": { 2024-08-31T06:23:25.747 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_optimizations", 2024-08-31T06:23:25.747 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-08-31T06:23:25.748 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.748 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.748 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-08-31T06:23:25.748 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.748 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.748 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.748 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-08-31T06:23:25.748 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.748 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.748 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.748 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.748 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.748 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.748 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.749 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm", 2024-08-31T06:23:25.749 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.749 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.749 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.749 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_down_multiplier": { 2024-08-31T06:23:25.749 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_down_multiplier", 2024-08-31T06:23:25.749 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-08-31T06:23:25.749 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.749 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.749 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-08-31T06:23:25.749 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.749 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.749 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.749 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-08-31T06:23:25.750 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.750 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.750 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.750 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.750 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_refresh_rate": { 2024-08-31T06:23:25.750 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_refresh_rate", 2024-08-31T06:23:25.750 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:25.750 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.750 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.750 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "20", 2024-08-31T06:23:25.750 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.750 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.750 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.750 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-08-31T06:23:25.751 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.751 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.751 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.751 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.751 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_starting_port": { 2024-08-31T06:23:25.751 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_starting_port", 2024-08-31T06:23:25.751 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.751 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.751 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.751 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4721", 2024-08-31T06:23:25.751 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.751 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.751 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.751 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-08-31T06:23:25.752 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.752 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.752 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.752 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.752 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_ptrace": { 2024-08-31T06:23:25.752 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_ptrace", 2024-08-31T06:23:25.752 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.752 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.752 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.752 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.752 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.752 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.752 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.752 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-08-31T06:23:25.753 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-08-31T06:23:25.753 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.753 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.753 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.753 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_interval": { 2024-08-31T06:23:25.753 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_interval", 2024-08-31T06:23:25.753 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:25.753 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.753 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.753 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-08-31T06:23:25.753 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.753 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.753 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.753 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to autotune daemon memory", 2024-08-31T06:23:25.754 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.754 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.754 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.754 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.754 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_memory_target_ratio": { 2024-08-31T06:23:25.754 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_memory_target_ratio", 2024-08-31T06:23:25.754 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-08-31T06:23:25.754 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.754 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.754 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.7", 2024-08-31T06:23:25.754 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.754 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.754 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.755 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-08-31T06:23:25.755 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.755 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.755 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.755 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.755 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm_log_destination": { 2024-08-31T06:23:25.755 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm_log_destination", 2024-08-31T06:23:25.755 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.755 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.755 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.755 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.755 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.755 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.755 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.756 INFO:teuthology.orchestra.run.smithi016.stdout: "file", 2024-08-31T06:23:25.756 INFO:teuthology.orchestra.run.smithi016.stdout: "file,syslog", 2024-08-31T06:23:25.756 INFO:teuthology.orchestra.run.smithi016.stdout: "syslog" 2024-08-31T06:23:25.756 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.756 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-08-31T06:23:25.756 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.756 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.756 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.756 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.756 INFO:teuthology.orchestra.run.smithi016.stdout: "cgroups_split": { 2024-08-31T06:23:25.756 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cgroups_split", 2024-08-31T06:23:25.756 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.756 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.757 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.757 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.757 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.758 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.758 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.758 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-08-31T06:23:25.758 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.758 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.758 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.758 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.758 INFO:teuthology.orchestra.run.smithi016.stdout: "config_checks_enabled": { 2024-08-31T06:23:25.758 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_checks_enabled", 2024-08-31T06:23:25.758 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.758 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.758 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.758 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.759 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.759 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.759 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.759 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-08-31T06:23:25.759 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.759 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.759 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.759 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.759 INFO:teuthology.orchestra.run.smithi016.stdout: "config_dashboard": { 2024-08-31T06:23:25.759 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_dashboard", 2024-08-31T06:23:25.759 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.759 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.759 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.759 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.760 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.760 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.760 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.760 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-08-31T06:23:25.760 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.760 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.760 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.760 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.760 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_alertmanager": { 2024-08-31T06:23:25.760 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_alertmanager", 2024-08-31T06:23:25.760 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.760 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.760 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.761 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-08-31T06:23:25.761 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.761 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.761 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.761 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-08-31T06:23:25.761 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.761 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.761 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.761 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.761 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_base": { 2024-08-31T06:23:25.761 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_base", 2024-08-31T06:23:25.761 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.761 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.761 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.762 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/ceph", 2024-08-31T06:23:25.762 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.762 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.762 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.762 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Container image name, without the tag", 2024-08-31T06:23:25.762 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.762 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.762 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.762 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.762 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_elasticsearch": { 2024-08-31T06:23:25.762 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_elasticsearch", 2024-08-31T06:23:25.762 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.762 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.762 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.763 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-08-31T06:23:25.763 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.763 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.763 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.763 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "elasticsearch container image", 2024-08-31T06:23:25.763 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.763 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.763 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.763 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.763 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_grafana": { 2024-08-31T06:23:25.763 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_grafana", 2024-08-31T06:23:25.763 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.763 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.764 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.764 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-08-31T06:23:25.764 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.764 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.764 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.764 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-08-31T06:23:25.764 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.764 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.764 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.764 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.764 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_haproxy": { 2024-08-31T06:23:25.764 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_haproxy", 2024-08-31T06:23:25.764 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.764 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.765 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.765 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-08-31T06:23:25.765 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.765 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.765 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.765 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "HAproxy container image", 2024-08-31T06:23:25.765 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.765 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.765 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.765 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.765 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_agent": { 2024-08-31T06:23:25.765 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_agent", 2024-08-31T06:23:25.765 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.765 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.766 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.766 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-08-31T06:23:25.766 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.766 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.766 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.766 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger agent container image", 2024-08-31T06:23:25.766 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.766 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.766 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.766 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.766 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_collector": { 2024-08-31T06:23:25.766 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_collector", 2024-08-31T06:23:25.766 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.766 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.767 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.767 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-08-31T06:23:25.767 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.767 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.767 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.767 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger collector container image", 2024-08-31T06:23:25.767 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.767 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.767 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.767 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.767 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_query": { 2024-08-31T06:23:25.767 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_query", 2024-08-31T06:23:25.768 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.768 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.768 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.768 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-08-31T06:23:25.768 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.768 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.768 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.768 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger query container image", 2024-08-31T06:23:25.768 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.768 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.768 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.768 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.768 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_keepalived": { 2024-08-31T06:23:25.769 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_keepalived", 2024-08-31T06:23:25.769 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.769 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.769 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.769 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-08-31T06:23:25.769 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.769 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.769 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.769 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Keepalived container image", 2024-08-31T06:23:25.769 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.769 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.769 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.769 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.769 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_loki": { 2024-08-31T06:23:25.770 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_loki", 2024-08-31T06:23:25.770 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.770 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.770 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.770 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-08-31T06:23:25.770 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.770 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.770 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.770 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Loki container image", 2024-08-31T06:23:25.770 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.770 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.770 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.770 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.770 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_nginx": { 2024-08-31T06:23:25.771 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_nginx", 2024-08-31T06:23:25.771 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.771 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.771 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.771 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-08-31T06:23:25.771 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.771 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.771 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.771 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Nginx container image", 2024-08-31T06:23:25.771 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.771 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.771 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.771 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.771 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_node_exporter": { 2024-08-31T06:23:25.772 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_node_exporter", 2024-08-31T06:23:25.772 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.772 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.772 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.772 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-08-31T06:23:25.772 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.772 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.772 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.772 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-08-31T06:23:25.772 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.772 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.772 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.772 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.772 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_nvmeof": { 2024-08-31T06:23:25.773 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_nvmeof", 2024-08-31T06:23:25.773 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.773 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.773 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.773 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-08-31T06:23:25.773 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.773 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.773 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.773 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Nvme-of container image", 2024-08-31T06:23:25.773 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.773 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.773 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.773 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.773 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_oauth2_proxy": { 2024-08-31T06:23:25.774 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_oauth2_proxy", 2024-08-31T06:23:25.774 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.774 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.774 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.774 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-08-31T06:23:25.774 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.774 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.774 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.774 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "oauth2-proxy container image", 2024-08-31T06:23:25.774 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.774 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.774 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.774 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.774 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_prometheus": { 2024-08-31T06:23:25.775 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_prometheus", 2024-08-31T06:23:25.775 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.775 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.775 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.775 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-08-31T06:23:25.775 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.775 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.775 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.775 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-08-31T06:23:25.775 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.775 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.775 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.775 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.775 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_promtail": { 2024-08-31T06:23:25.776 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_promtail", 2024-08-31T06:23:25.776 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.776 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.776 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.776 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-08-31T06:23:25.776 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.776 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.776 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.776 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Promtail container image", 2024-08-31T06:23:25.776 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.776 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.776 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.776 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.776 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_samba": { 2024-08-31T06:23:25.777 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_samba", 2024-08-31T06:23:25.777 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.777 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.777 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.777 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-08-31T06:23:25.777 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.777 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.777 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.777 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Samba/SMB container image", 2024-08-31T06:23:25.777 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.777 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.777 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.777 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.777 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_snmp_gateway": { 2024-08-31T06:23:25.778 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_snmp_gateway", 2024-08-31T06:23:25.778 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.778 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.778 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.778 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-08-31T06:23:25.778 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.778 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.778 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.778 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SNMP Gateway container image", 2024-08-31T06:23:25.778 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.778 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.778 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.778 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.778 INFO:teuthology.orchestra.run.smithi016.stdout: "container_init": { 2024-08-31T06:23:25.778 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_init", 2024-08-31T06:23:25.779 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.779 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.779 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.779 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.779 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.779 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.779 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.779 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Run podman/docker with `--init`", 2024-08-31T06:23:25.779 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.779 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.779 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.779 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.779 INFO:teuthology.orchestra.run.smithi016.stdout: "daemon_cache_timeout": { 2024-08-31T06:23:25.780 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "daemon_cache_timeout", 2024-08-31T06:23:25.780 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:25.780 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.780 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.780 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-08-31T06:23:25.780 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.780 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.780 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.780 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-08-31T06:23:25.780 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.780 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.780 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.781 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.781 INFO:teuthology.orchestra.run.smithi016.stdout: "default_cephadm_command_timeout": { 2024-08-31T06:23:25.781 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_cephadm_command_timeout", 2024-08-31T06:23:25.781 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.781 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.781 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.781 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "900", 2024-08-31T06:23:25.781 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.781 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.781 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.781 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-08-31T06:23:25.781 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.781 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.781 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.782 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.782 INFO:teuthology.orchestra.run.smithi016.stdout: "default_registry": { 2024-08-31T06:23:25.782 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_registry", 2024-08-31T06:23:25.782 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.782 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.782 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.782 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io", 2024-08-31T06:23:25.782 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.782 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.782 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.782 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-08-31T06:23:25.782 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.782 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.782 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.783 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.783 INFO:teuthology.orchestra.run.smithi016.stdout: "device_cache_timeout": { 2024-08-31T06:23:25.783 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_cache_timeout", 2024-08-31T06:23:25.783 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:25.783 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.783 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.783 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1800", 2024-08-31T06:23:25.783 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.783 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.783 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.783 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache device inventory", 2024-08-31T06:23:25.783 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.783 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.783 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.784 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.784 INFO:teuthology.orchestra.run.smithi016.stdout: "device_enhanced_scan": { 2024-08-31T06:23:25.784 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_enhanced_scan", 2024-08-31T06:23:25.784 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.784 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.784 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.784 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.784 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.784 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.784 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.784 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use libstoragemgmt during device scans", 2024-08-31T06:23:25.784 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.784 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.784 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.785 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.785 INFO:teuthology.orchestra.run.smithi016.stdout: "facts_cache_timeout": { 2024-08-31T06:23:25.785 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "facts_cache_timeout", 2024-08-31T06:23:25.785 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:25.785 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.785 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.785 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-08-31T06:23:25.785 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.785 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.785 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.785 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache host facts data", 2024-08-31T06:23:25.785 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.785 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.785 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.786 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.786 INFO:teuthology.orchestra.run.smithi016.stdout: "grafana_dashboards_path": { 2024-08-31T06:23:25.786 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "grafana_dashboards_path", 2024-08-31T06:23:25.786 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.786 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.786 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.786 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-08-31T06:23:25.786 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.786 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.786 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.786 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-08-31T06:23:25.786 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.786 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.786 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.787 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.787 INFO:teuthology.orchestra.run.smithi016.stdout: "host_check_interval": { 2024-08-31T06:23:25.787 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "host_check_interval", 2024-08-31T06:23:25.787 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:25.787 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.787 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.787 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-08-31T06:23:25.787 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.787 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.787 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.787 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to perform a host check", 2024-08-31T06:23:25.787 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.787 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.787 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.788 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.788 INFO:teuthology.orchestra.run.smithi016.stdout: "hw_monitoring": { 2024-08-31T06:23:25.788 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hw_monitoring", 2024-08-31T06:23:25.788 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.788 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.788 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.788 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.788 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.788 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.788 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.788 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-08-31T06:23:25.788 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.788 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.788 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.789 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.789 INFO:teuthology.orchestra.run.smithi016.stdout: "inventory_list_all": { 2024-08-31T06:23:25.789 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "inventory_list_all", 2024-08-31T06:23:25.789 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.789 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.789 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.789 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.789 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.789 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.789 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.789 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-08-31T06:23:25.789 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.789 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.789 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.790 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.790 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.790 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.790 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.790 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.790 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.790 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.790 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.790 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.790 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.790 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.790 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.790 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.790 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.790 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.791 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.791 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.791 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.791 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.791 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.791 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.791 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.791 INFO:teuthology.orchestra.run.smithi016.stdout: "log_refresh_metadata": { 2024-08-31T06:23:25.791 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_refresh_metadata", 2024-08-31T06:23:25.791 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.791 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.791 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.791 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.791 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.792 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.792 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.792 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-08-31T06:23:25.792 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.792 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.792 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.792 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.792 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.793 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.793 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.793 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.793 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.793 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.793 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.793 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.793 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.793 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-08-31T06:23:25.793 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.793 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.793 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.793 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.794 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.794 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.794 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.794 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.794 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.794 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.794 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.794 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.794 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.794 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.794 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.794 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.794 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.794 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.795 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.795 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.795 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.795 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.795 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.795 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.795 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.795 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.795 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.795 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.795 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.795 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.795 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.795 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.796 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.796 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.796 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.796 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.796 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.796 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.796 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.796 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf": { 2024-08-31T06:23:25.796 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-08-31T06:23:25.796 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.796 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.796 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.796 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.796 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.797 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.797 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.797 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-08-31T06:23:25.797 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.797 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.797 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.797 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.797 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-08-31T06:23:25.797 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-08-31T06:23:25.797 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.797 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.797 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.797 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "*", 2024-08-31T06:23:25.797 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.798 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.798 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.798 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-08-31T06:23:25.798 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.798 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.798 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.798 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.798 INFO:teuthology.orchestra.run.smithi016.stdout: "max_count_per_host": { 2024-08-31T06:23:25.798 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_count_per_host", 2024-08-31T06:23:25.798 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.798 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.798 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.798 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-08-31T06:23:25.798 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.799 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.799 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.799 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of daemons per service per host", 2024-08-31T06:23:25.799 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.799 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.799 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.799 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.799 INFO:teuthology.orchestra.run.smithi016.stdout: "max_osd_draining_count": { 2024-08-31T06:23:25.799 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_osd_draining_count", 2024-08-31T06:23:25.799 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.799 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.799 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.799 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-08-31T06:23:25.799 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.800 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.800 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.800 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-08-31T06:23:25.800 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.800 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.800 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.800 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.800 INFO:teuthology.orchestra.run.smithi016.stdout: "migration_current": { 2024-08-31T06:23:25.800 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "migration_current", 2024-08-31T06:23:25.800 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.800 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.800 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.800 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.800 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.801 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.801 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.801 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "internal - do not modify", 2024-08-31T06:23:25.801 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.801 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.801 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.801 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.801 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-08-31T06:23:25.801 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-08-31T06:23:25.801 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.801 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.801 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.801 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "root", 2024-08-31T06:23:25.801 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.802 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.802 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.802 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm-package", 2024-08-31T06:23:25.802 INFO:teuthology.orchestra.run.smithi016.stdout: "root" 2024-08-31T06:23:25.802 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.802 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "mode for remote execution of cephadm", 2024-08-31T06:23:25.802 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.802 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.802 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.802 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.802 INFO:teuthology.orchestra.run.smithi016.stdout: "oob_default_addr": { 2024-08-31T06:23:25.802 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "oob_default_addr", 2024-08-31T06:23:25.802 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.802 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.802 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.803 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "169.254.1.1", 2024-08-31T06:23:25.803 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.803 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.803 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.803 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default address for RedFish API (oob management).", 2024-08-31T06:23:25.803 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.803 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.803 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.803 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.803 INFO:teuthology.orchestra.run.smithi016.stdout: "prometheus_alerts_path": { 2024-08-31T06:23:25.803 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus_alerts_path", 2024-08-31T06:23:25.803 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.803 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.803 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.804 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-08-31T06:23:25.804 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.804 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.804 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.804 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-08-31T06:23:25.804 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.804 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.804 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.804 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.804 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_insecure": { 2024-08-31T06:23:25.804 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_insecure", 2024-08-31T06:23:25.804 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.804 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.805 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.805 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.805 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.805 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.805 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.805 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-08-31T06:23:25.805 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.805 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.805 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.805 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.805 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_password": { 2024-08-31T06:23:25.805 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_password", 2024-08-31T06:23:25.805 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.806 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.806 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.806 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.806 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.806 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.806 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.806 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-08-31T06:23:25.806 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.806 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.806 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.806 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.806 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_url": { 2024-08-31T06:23:25.806 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_url", 2024-08-31T06:23:25.807 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.807 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.807 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.807 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.807 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.807 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.807 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.807 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-08-31T06:23:25.807 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.807 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.807 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.807 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.807 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_username": { 2024-08-31T06:23:25.808 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_username", 2024-08-31T06:23:25.808 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.808 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.808 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.808 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.808 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.808 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.808 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.808 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-08-31T06:23:25.808 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.808 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.808 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.808 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.809 INFO:teuthology.orchestra.run.smithi016.stdout: "secure_monitoring_stack": { 2024-08-31T06:23:25.809 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secure_monitoring_stack", 2024-08-31T06:23:25.809 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.809 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.809 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.809 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.809 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.809 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.809 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.809 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-08-31T06:23:25.809 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.809 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.809 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.809 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.810 INFO:teuthology.orchestra.run.smithi016.stdout: "service_discovery_port": { 2024-08-31T06:23:25.810 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "service_discovery_port", 2024-08-31T06:23:25.810 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.810 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.810 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.810 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8765", 2024-08-31T06:23:25.810 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.810 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.810 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.810 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "cephadm service discovery port", 2024-08-31T06:23:25.810 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.810 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.810 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.810 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.811 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.811 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.811 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.811 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.811 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.811 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.811 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.811 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.811 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.811 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.811 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.811 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.811 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.811 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.812 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_config_file": { 2024-08-31T06:23:25.812 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_config_file", 2024-08-31T06:23:25.812 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.812 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.812 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.812 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.812 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.812 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.812 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.812 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-08-31T06:23:25.812 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.812 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.812 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.812 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.813 INFO:teuthology.orchestra.run.smithi016.stdout: "use_agent": { 2024-08-31T06:23:25.813 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_agent", 2024-08-31T06:23:25.813 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.813 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.813 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.813 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.813 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.813 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.813 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.813 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-08-31T06:23:25.813 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.813 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.813 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.813 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.813 INFO:teuthology.orchestra.run.smithi016.stdout: "use_repo_digest": { 2024-08-31T06:23:25.814 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_repo_digest", 2024-08-31T06:23:25.814 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.814 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.814 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.814 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.814 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.814 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.814 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.814 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-08-31T06:23:25.814 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.814 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.814 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.814 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.814 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_failed_host_check": { 2024-08-31T06:23:25.815 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_failed_host_check", 2024-08-31T06:23:25.815 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.815 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.815 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.815 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.815 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.815 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.815 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.815 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if the host check fails", 2024-08-31T06:23:25.815 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.815 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.815 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.815 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.815 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_daemons": { 2024-08-31T06:23:25.815 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_daemons", 2024-08-31T06:23:25.816 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.816 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.816 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.816 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.816 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.816 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.816 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.816 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-08-31T06:23:25.816 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.816 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.816 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.816 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.816 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_hosts": { 2024-08-31T06:23:25.817 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_hosts", 2024-08-31T06:23:25.817 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.817 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.817 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.817 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.817 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.817 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.817 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.817 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-08-31T06:23:25.817 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.817 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.817 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.817 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.818 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.818 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.818 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.818 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crash", 2024-08-31T06:23:25.818 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.818 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.818 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.818 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.818 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.818 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.818 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.818 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.818 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.818 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.818 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.819 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.819 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.819 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.819 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.819 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.819 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.819 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.819 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.819 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.819 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.819 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.819 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.819 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.820 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.820 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.820 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.820 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.820 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.820 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.820 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.820 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.820 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.820 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.820 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.820 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.820 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.820 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.821 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.821 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.821 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.821 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.821 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.821 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.821 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.821 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.821 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.821 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.821 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.821 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.821 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.821 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.822 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.822 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.822 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.822 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.822 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.822 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.822 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.822 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.822 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.822 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.822 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.822 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.822 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.823 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.823 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.823 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.823 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.823 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.823 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.823 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.823 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.823 INFO:teuthology.orchestra.run.smithi016.stdout: "retain_interval": { 2024-08-31T06:23:25.823 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retain_interval", 2024-08-31T06:23:25.823 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:25.823 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.823 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.824 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "31536000", 2024-08-31T06:23:25.824 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.824 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.824 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.824 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain crashes before pruning them", 2024-08-31T06:23:25.824 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.824 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.824 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.824 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.824 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.824 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.824 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.824 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.825 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.825 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.825 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.825 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.825 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.825 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.825 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.825 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.825 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.825 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.825 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_recent_interval": { 2024-08-31T06:23:25.825 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_recent_interval", 2024-08-31T06:23:25.825 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:25.825 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.826 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.826 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1209600", 2024-08-31T06:23:25.826 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.826 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.826 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.826 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "time interval in which to warn about recent crashes", 2024-08-31T06:23:25.826 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.826 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.826 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.826 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.826 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.826 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.826 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.826 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dashboard", 2024-08-31T06:23:25.827 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.827 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.827 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.827 INFO:teuthology.orchestra.run.smithi016.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-08-31T06:23:25.827 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-08-31T06:23:25.827 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.827 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.827 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.827 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-08-31T06:23:25.827 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.827 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.827 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.827 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.827 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.828 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.828 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.828 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.828 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_HOST": { 2024-08-31T06:23:25.828 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_HOST", 2024-08-31T06:23:25.828 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.828 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.828 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.828 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.828 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.828 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.828 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.828 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.828 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.829 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.829 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.829 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.829 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-08-31T06:23:25.829 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-08-31T06:23:25.829 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.829 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.829 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.829 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.829 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.829 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.829 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.829 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.829 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.830 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.830 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.830 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.830 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_ENABLED": { 2024-08-31T06:23:25.830 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_ENABLED", 2024-08-31T06:23:25.830 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.830 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.830 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.830 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.830 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.830 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.830 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.830 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.830 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.830 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.831 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.831 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.831 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-08-31T06:23:25.831 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-08-31T06:23:25.831 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.831 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.831 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.831 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.831 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.831 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.831 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.831 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.831 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.832 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.832 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.832 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.832 INFO:teuthology.orchestra.run.smithi016.stdout: "ENABLE_BROWSABLE_API": { 2024-08-31T06:23:25.832 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ENABLE_BROWSABLE_API", 2024-08-31T06:23:25.832 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.832 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.832 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.832 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.832 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.832 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.832 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.832 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.833 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.833 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.833 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.833 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.833 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-08-31T06:23:25.833 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-08-31T06:23:25.833 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.833 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.833 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.833 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.833 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.833 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.833 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.833 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.834 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.834 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.834 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.834 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.834 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-08-31T06:23:25.834 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-08-31T06:23:25.834 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.834 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.834 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.834 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.834 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.834 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.834 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.835 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.835 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.835 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.835 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.835 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.835 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-08-31T06:23:25.835 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-08-31T06:23:25.835 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.835 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.835 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.835 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.835 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.835 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.835 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.835 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.836 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.836 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.836 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.836 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.836 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-08-31T06:23:25.836 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-08-31T06:23:25.836 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.836 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.836 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.836 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.836 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.836 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.837 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.837 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.837 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.837 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.837 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.837 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.837 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_NFS": { 2024-08-31T06:23:25.837 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-08-31T06:23:25.837 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.837 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.837 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.837 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.837 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.838 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.838 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.838 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.838 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.838 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.838 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.838 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.838 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RBD": { 2024-08-31T06:23:25.838 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-08-31T06:23:25.838 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.838 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.838 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.838 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.838 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.839 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.839 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.839 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.839 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.839 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.839 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.839 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.839 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RGW": { 2024-08-31T06:23:25.839 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-08-31T06:23:25.839 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.839 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.839 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.839 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.840 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.840 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.840 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.840 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.840 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.840 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.840 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.840 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.840 INFO:teuthology.orchestra.run.smithi016.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-08-31T06:23:25.840 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-08-31T06:23:25.840 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.840 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.840 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.841 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.841 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.841 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.841 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.841 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.841 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.841 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.841 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.841 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.841 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_PASSWORD": { 2024-08-31T06:23:25.841 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_PASSWORD", 2024-08-31T06:23:25.841 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.841 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.841 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.842 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-08-31T06:23:25.842 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.842 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.842 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.842 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.842 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.842 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.842 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.842 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.842 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-08-31T06:23:25.842 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-08-31T06:23:25.842 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.843 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.843 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.843 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.843 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.843 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.843 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.843 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.843 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.843 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.843 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.843 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.843 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_URL": { 2024-08-31T06:23:25.843 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_URL", 2024-08-31T06:23:25.843 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.843 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.844 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.844 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.844 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.844 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.844 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.844 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.844 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.844 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.844 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.844 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.844 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_USERNAME": { 2024-08-31T06:23:25.844 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_USERNAME", 2024-08-31T06:23:25.844 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.845 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.845 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.845 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-08-31T06:23:25.845 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.845 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.845 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.845 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.845 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.845 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.845 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.845 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.845 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-08-31T06:23:25.845 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-08-31T06:23:25.846 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.846 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.846 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.846 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.846 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.846 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.846 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.846 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.846 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.846 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.846 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.846 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.846 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-08-31T06:23:25.846 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-08-31T06:23:25.847 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.847 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.847 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.847 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.847 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.847 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.847 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.847 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.848 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.848 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.848 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.848 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.848 INFO:teuthology.orchestra.run.smithi016.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-08-31T06:23:25.848 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-08-31T06:23:25.848 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.848 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.848 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.848 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.848 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.848 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.848 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.849 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.849 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.849 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.849 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.849 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.849 INFO:teuthology.orchestra.run.smithi016.stdout: "ISSUE_TRACKER_API_KEY": { 2024-08-31T06:23:25.849 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-08-31T06:23:25.849 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.849 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.849 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.849 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.849 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.849 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.849 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.850 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.850 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.850 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.850 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.850 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.850 INFO:teuthology.orchestra.run.smithi016.stdout: "MANAGED_BY_CLUSTERS": { 2024-08-31T06:23:25.850 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-08-31T06:23:25.850 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.850 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.850 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.850 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "[]", 2024-08-31T06:23:25.850 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.850 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.850 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.851 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.851 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.851 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.851 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.851 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.851 INFO:teuthology.orchestra.run.smithi016.stdout: "MULTICLUSTER_CONFIG": { 2024-08-31T06:23:25.851 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "MULTICLUSTER_CONFIG", 2024-08-31T06:23:25.851 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.851 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.851 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.851 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "{}", 2024-08-31T06:23:25.851 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.851 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.851 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.852 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.852 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.852 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.852 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.852 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.852 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_HOST": { 2024-08-31T06:23:25.852 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_HOST", 2024-08-31T06:23:25.852 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.852 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.852 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.852 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.852 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.852 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.853 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.853 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.853 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.853 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.853 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.853 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.853 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-08-31T06:23:25.853 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-08-31T06:23:25.853 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.853 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.853 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.853 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.853 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.853 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.854 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.854 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.854 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.854 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.854 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.854 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.854 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-08-31T06:23:25.854 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-08-31T06:23:25.854 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.854 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.854 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.854 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.854 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.854 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.855 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.855 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.855 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.855 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.855 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.855 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.855 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-08-31T06:23:25.855 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-08-31T06:23:25.855 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.855 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.855 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.855 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.855 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.855 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.855 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.856 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.856 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.856 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.856 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.856 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.856 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-08-31T06:23:25.856 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-08-31T06:23:25.856 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.856 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.856 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.856 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.856 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.856 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.857 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.857 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.857 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.857 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.857 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.857 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.857 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-08-31T06:23:25.857 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-08-31T06:23:25.857 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.857 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.857 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.857 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.857 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.857 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.858 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.858 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.858 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.858 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.858 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.858 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.858 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-08-31T06:23:25.858 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-08-31T06:23:25.858 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.858 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.858 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.858 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.858 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.858 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.859 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.859 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.859 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.859 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.859 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.859 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.859 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-08-31T06:23:25.859 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-08-31T06:23:25.859 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.859 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.859 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.859 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.860 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.860 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.860 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.860 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.860 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.860 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.860 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.860 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.860 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-08-31T06:23:25.860 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-08-31T06:23:25.860 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.860 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.860 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.861 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.861 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.861 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.861 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.861 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.861 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.861 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.861 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.861 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.861 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_ENABLED": { 2024-08-31T06:23:25.861 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_ENABLED", 2024-08-31T06:23:25.861 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.861 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.861 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.862 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.862 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.862 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.862 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.862 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.862 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.862 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.862 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.862 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.862 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-08-31T06:23:25.862 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-08-31T06:23:25.862 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.862 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.863 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.863 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-08-31T06:23:25.863 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.863 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.863 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.863 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.863 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.863 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.863 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.863 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.863 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-08-31T06:23:25.863 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-08-31T06:23:25.863 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.864 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.864 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.864 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-08-31T06:23:25.864 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.864 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.864 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.864 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.864 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.864 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.864 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.864 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.864 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-08-31T06:23:25.865 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-08-31T06:23:25.865 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.865 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.865 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.865 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8", 2024-08-31T06:23:25.865 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.865 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.865 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.865 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.865 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.865 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.865 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.865 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.865 INFO:teuthology.orchestra.run.smithi016.stdout: "REST_REQUESTS_TIMEOUT": { 2024-08-31T06:23:25.866 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-08-31T06:23:25.866 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.866 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.866 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.866 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "45", 2024-08-31T06:23:25.866 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.866 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.866 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.866 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.866 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.866 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.866 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.866 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.866 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ACCESS_KEY": { 2024-08-31T06:23:25.866 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ACCESS_KEY", 2024-08-31T06:23:25.867 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.867 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.867 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.867 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.867 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.867 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.867 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.867 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.867 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.867 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.867 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.867 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.867 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-08-31T06:23:25.867 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-08-31T06:23:25.868 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.868 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.868 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.868 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-08-31T06:23:25.868 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.868 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.868 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.868 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.868 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.868 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.868 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.868 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.868 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SECRET_KEY": { 2024-08-31T06:23:25.868 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SECRET_KEY", 2024-08-31T06:23:25.869 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.869 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.869 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.869 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.869 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.869 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.869 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.869 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.869 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.869 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.869 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.869 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.869 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SSL_VERIFY": { 2024-08-31T06:23:25.869 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SSL_VERIFY", 2024-08-31T06:23:25.870 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.870 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.870 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.870 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.870 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.870 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.870 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.870 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.870 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.870 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.870 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.870 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.870 INFO:teuthology.orchestra.run.smithi016.stdout: "UNSAFE_TLS_v1_2": { 2024-08-31T06:23:25.870 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "UNSAFE_TLS_v1_2", 2024-08-31T06:23:25.871 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.871 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.871 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.871 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.871 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.871 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.871 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.871 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.871 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.871 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.871 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.871 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.871 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-08-31T06:23:25.871 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-08-31T06:23:25.872 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.872 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.872 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.872 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.872 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.872 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.872 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.872 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.872 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.872 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.872 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.872 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.872 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-08-31T06:23:25.873 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-08-31T06:23:25.873 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.873 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.873 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.873 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-08-31T06:23:25.873 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.873 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.873 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.873 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.873 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.873 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.873 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.873 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.874 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-08-31T06:23:25.874 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-08-31T06:23:25.874 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.874 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.874 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.874 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-08-31T06:23:25.874 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.874 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.874 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.874 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.874 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.874 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.874 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.874 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.875 INFO:teuthology.orchestra.run.smithi016.stdout: "cross_origin_url": { 2024-08-31T06:23:25.875 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cross_origin_url", 2024-08-31T06:23:25.875 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.875 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.875 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.875 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.875 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.875 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.875 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.875 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.875 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.875 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.875 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.875 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.876 INFO:teuthology.orchestra.run.smithi016.stdout: "crt_file": { 2024-08-31T06:23:25.876 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crt_file", 2024-08-31T06:23:25.876 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.876 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.876 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.876 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.876 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.876 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.876 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.876 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.876 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.876 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.876 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.876 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.877 INFO:teuthology.orchestra.run.smithi016.stdout: "debug": { 2024-08-31T06:23:25.877 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "debug", 2024-08-31T06:23:25.877 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.877 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.877 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.877 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.877 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.877 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.877 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.877 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable/disable debug options", 2024-08-31T06:23:25.877 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.877 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.877 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.878 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.878 INFO:teuthology.orchestra.run.smithi016.stdout: "jwt_token_ttl": { 2024-08-31T06:23:25.878 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "jwt_token_ttl", 2024-08-31T06:23:25.878 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.878 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.878 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.878 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "28800", 2024-08-31T06:23:25.878 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.878 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.878 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.878 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.878 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.878 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.878 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.879 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.879 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-08-31T06:23:25.879 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-08-31T06:23:25.879 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.879 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.879 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.879 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.879 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.879 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.879 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.879 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.879 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.879 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.879 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.880 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.880 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.880 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.880 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.880 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.880 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.880 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.880 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.880 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.880 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.880 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.880 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.880 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.881 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.881 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.881 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.881 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.881 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.881 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.881 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.881 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.881 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.881 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.881 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.881 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.881 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.881 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.882 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.882 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.882 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.882 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.882 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.882 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.882 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.882 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.882 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.882 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.882 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.882 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.882 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.882 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.882 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.883 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.883 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.883 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.883 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.883 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.883 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.883 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.883 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.883 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.883 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.883 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.883 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.883 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.883 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.884 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.884 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.884 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.884 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.884 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.884 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.884 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.884 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.884 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.884 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.884 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.884 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.884 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.885 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.885 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.885 INFO:teuthology.orchestra.run.smithi016.stdout: "motd": { 2024-08-31T06:23:25.885 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "motd", 2024-08-31T06:23:25.885 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.885 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.885 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.885 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.885 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.885 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.885 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.885 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "The message of the day", 2024-08-31T06:23:25.885 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.885 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.885 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.886 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.886 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect_resolve_ip_addr": { 2024-08-31T06:23:25.886 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "redirect_resolve_ip_addr", 2024-08-31T06:23:25.886 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.886 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.886 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.886 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.886 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.886 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.886 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.886 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.886 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.886 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.887 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.887 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.887 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-08-31T06:23:25.887 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-08-31T06:23:25.887 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.887 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.887 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.887 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-08-31T06:23:25.887 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.887 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.887 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.887 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.887 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.887 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.887 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.888 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.888 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-08-31T06:23:25.888 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-08-31T06:23:25.888 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.888 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.888 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.888 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8080", 2024-08-31T06:23:25.888 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.888 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.888 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.888 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.888 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.888 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.889 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.889 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.889 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.889 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.889 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.889 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.889 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.889 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.889 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.889 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.889 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.889 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.889 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.889 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.890 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.890 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.890 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-08-31T06:23:25.890 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-08-31T06:23:25.890 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.890 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.890 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.890 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.890 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.890 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.890 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.890 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.890 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.891 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.891 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.891 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.891 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl_server_port": { 2024-08-31T06:23:25.891 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl_server_port", 2024-08-31T06:23:25.891 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.891 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.891 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.891 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8443", 2024-08-31T06:23:25.891 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.891 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.891 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.891 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.892 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.892 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.892 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.892 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.892 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-08-31T06:23:25.892 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-08-31T06:23:25.892 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.892 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.892 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.893 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "redirect", 2024-08-31T06:23:25.893 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.893 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.893 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.893 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.893 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect" 2024-08-31T06:23:25.893 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.893 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.893 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.893 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.893 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.893 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.893 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-08-31T06:23:25.893 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-08-31T06:23:25.894 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.894 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.894 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.894 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-08-31T06:23:25.894 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-08-31T06:23:25.894 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-08-31T06:23:25.894 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.894 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.894 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.894 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.894 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.894 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.894 INFO:teuthology.orchestra.run.smithi016.stdout: "url_prefix": { 2024-08-31T06:23:25.894 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url_prefix", 2024-08-31T06:23:25.895 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.895 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.895 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.895 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.895 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.895 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.895 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.895 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.895 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.895 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.895 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.895 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.895 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.895 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.896 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.896 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2024-08-31T06:23:25.896 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.896 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.896 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.896 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_monitoring": { 2024-08-31T06:23:25.896 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_monitoring", 2024-08-31T06:23:25.896 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.896 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.896 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.896 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.896 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.896 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.896 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.897 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "monitor device health metrics", 2024-08-31T06:23:25.897 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.897 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.897 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.897 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.897 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.897 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.897 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.897 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.897 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.897 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.897 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.897 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.897 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.898 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.898 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.898 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.898 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.898 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.898 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.898 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.898 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.898 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.898 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.898 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.898 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.898 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.898 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.899 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.899 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.899 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.899 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.899 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.899 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.899 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.899 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.899 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.899 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.899 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.899 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.899 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.899 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.900 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.900 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.900 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.900 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.900 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.900 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.900 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.900 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.900 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.900 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.900 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.901 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.901 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.901 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.901 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.901 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.901 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.901 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.901 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.901 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.901 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.901 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.902 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.902 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.902 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.902 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.902 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.902 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.902 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.902 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.902 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.902 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.902 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.902 INFO:teuthology.orchestra.run.smithi016.stdout: "mark_out_threshold": { 2024-08-31T06:23:25.902 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mark_out_threshold", 2024-08-31T06:23:25.902 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:25.903 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.903 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.903 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2419200", 2024-08-31T06:23:25.903 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.903 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.903 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.903 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-08-31T06:23:25.903 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.903 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.903 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.903 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.903 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_name": { 2024-08-31T06:23:25.903 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_name", 2024-08-31T06:23:25.904 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.904 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.904 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.904 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "device_health_metrics", 2024-08-31T06:23:25.904 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.904 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.904 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.904 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name of pool in which to store device health metrics", 2024-08-31T06:23:25.904 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.904 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.904 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.904 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.904 INFO:teuthology.orchestra.run.smithi016.stdout: "retention_period": { 2024-08-31T06:23:25.905 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retention_period", 2024-08-31T06:23:25.905 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:25.905 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.905 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.905 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15552000", 2024-08-31T06:23:25.905 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.905 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.905 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.905 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain device health metrics", 2024-08-31T06:23:25.905 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.905 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.905 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.905 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.905 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_frequency": { 2024-08-31T06:23:25.906 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_frequency", 2024-08-31T06:23:25.906 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:25.906 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.906 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.906 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-08-31T06:23:25.906 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.906 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.906 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.906 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to scrape device health metrics", 2024-08-31T06:23:25.906 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.906 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.906 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.906 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.906 INFO:teuthology.orchestra.run.smithi016.stdout: "self_heal": { 2024-08-31T06:23:25.907 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "self_heal", 2024-08-31T06:23:25.907 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.907 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.907 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.907 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:25.907 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.907 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.907 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.907 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-08-31T06:23:25.907 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.907 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.907 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.907 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.908 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-08-31T06:23:25.908 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-08-31T06:23:25.908 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:25.908 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.908 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.908 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-08-31T06:23:25.908 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.908 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.908 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.908 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and check device health", 2024-08-31T06:23:25.908 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.908 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.908 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.909 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.909 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.909 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.909 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.909 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.909 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.909 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.909 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.909 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.909 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.909 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.909 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.909 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.909 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.910 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.910 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_threshold": { 2024-08-31T06:23:25.910 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_threshold", 2024-08-31T06:23:25.910 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:25.910 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.910 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.910 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7257600", 2024-08-31T06:23:25.910 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.910 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.910 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.910 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-08-31T06:23:25.910 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.910 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.910 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.911 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.911 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.911 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.911 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.911 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "diskprediction_local", 2024-08-31T06:23:25.911 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.911 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.911 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.911 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.911 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.911 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.911 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.911 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.912 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.912 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.912 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.912 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.912 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.912 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.912 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.912 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.912 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.912 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.912 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.912 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.912 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.913 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.913 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.913 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.913 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.913 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.913 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.913 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.913 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.913 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.913 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.913 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.913 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.913 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.913 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.913 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.914 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.914 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.914 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.914 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.914 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.914 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.914 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.914 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.914 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.914 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.914 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.914 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.915 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.915 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.915 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.915 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.915 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.915 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.915 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.915 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.915 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.915 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.915 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.915 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.915 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.915 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.915 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.916 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.916 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.916 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.916 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.916 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.916 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.916 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.916 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.916 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.916 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.916 INFO:teuthology.orchestra.run.smithi016.stdout: "predict_interval": { 2024-08-31T06:23:25.916 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predict_interval", 2024-08-31T06:23:25.916 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.917 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.917 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.917 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-08-31T06:23:25.917 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.917 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.917 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.917 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.917 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.917 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.917 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.917 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.917 INFO:teuthology.orchestra.run.smithi016.stdout: "predictor_model": { 2024-08-31T06:23:25.917 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predictor_model", 2024-08-31T06:23:25.917 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.918 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.918 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.918 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "prophetstor", 2024-08-31T06:23:25.918 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.918 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.918 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.918 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.918 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.918 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.918 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.918 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.918 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-08-31T06:23:25.918 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-08-31T06:23:25.919 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.919 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.919 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.919 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-08-31T06:23:25.919 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.919 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.919 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.919 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.919 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.919 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.919 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.919 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.919 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.919 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.919 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.920 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.920 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.920 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.920 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.920 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.920 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.920 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.920 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.920 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.920 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.920 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.920 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.920 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.921 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.921 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "influx", 2024-08-31T06:23:25.921 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": false, 2024-08-31T06:23:25.921 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "influxdb python module not found", 2024-08-31T06:23:25.921 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.921 INFO:teuthology.orchestra.run.smithi016.stdout: "batch_size": { 2024-08-31T06:23:25.921 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "batch_size", 2024-08-31T06:23:25.921 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.921 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.921 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.921 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5000", 2024-08-31T06:23:25.921 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.921 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.921 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.921 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-08-31T06:23:25.922 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.922 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.922 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.922 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.922 INFO:teuthology.orchestra.run.smithi016.stdout: "database": { 2024-08-31T06:23:25.922 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "database", 2024-08-31T06:23:25.922 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.922 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.922 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.922 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "ceph", 2024-08-31T06:23:25.922 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.922 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.922 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.922 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-08-31T06:23:25.923 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.923 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.923 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.923 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.923 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": { 2024-08-31T06:23:25.923 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hostname", 2024-08-31T06:23:25.923 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.923 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.923 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.923 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.923 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.923 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.923 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.923 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server hostname", 2024-08-31T06:23:25.924 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.924 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.924 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.924 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.924 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-08-31T06:23:25.924 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-08-31T06:23:25.924 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:25.924 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.924 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.924 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "30", 2024-08-31T06:23:25.924 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "5", 2024-08-31T06:23:25.924 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.924 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.925 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-08-31T06:23:25.925 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.925 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.925 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.925 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.925 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.925 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.925 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.925 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.925 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.925 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.925 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.925 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.925 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.925 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.926 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.926 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.926 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.926 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.926 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.926 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.926 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.926 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.926 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.926 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.926 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.926 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.926 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.926 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.926 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.927 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.927 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.927 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.927 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.927 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.927 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.927 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.927 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.927 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.927 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.927 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.927 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.927 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.927 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.928 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.928 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.928 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.928 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.928 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.928 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.928 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.928 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.928 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.928 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.928 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.928 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.929 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.929 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.929 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.929 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.929 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.929 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.929 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.929 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.929 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.929 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.929 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.929 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.929 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.929 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.930 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.930 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.930 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.930 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.930 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.930 INFO:teuthology.orchestra.run.smithi016.stdout: "password": { 2024-08-31T06:23:25.930 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "password", 2024-08-31T06:23:25.930 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.930 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.930 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.930 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.930 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.930 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.930 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.931 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "password of InfluxDB server user", 2024-08-31T06:23:25.931 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.931 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.931 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.931 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.931 INFO:teuthology.orchestra.run.smithi016.stdout: "port": { 2024-08-31T06:23:25.931 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "port", 2024-08-31T06:23:25.931 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.931 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.931 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.931 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8086", 2024-08-31T06:23:25.931 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.931 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.931 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.931 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server port", 2024-08-31T06:23:25.932 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.932 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.932 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.932 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.932 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.932 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.932 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.932 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.932 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.932 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.932 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.932 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.932 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.933 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.933 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.933 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.933 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.933 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.933 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-08-31T06:23:25.933 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-08-31T06:23:25.933 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.933 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.933 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.933 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "false", 2024-08-31T06:23:25.933 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.933 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.933 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.934 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-08-31T06:23:25.934 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.934 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.934 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.934 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.934 INFO:teuthology.orchestra.run.smithi016.stdout: "threads": { 2024-08-31T06:23:25.934 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threads", 2024-08-31T06:23:25.934 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.934 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.934 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.934 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-08-31T06:23:25.934 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-08-31T06:23:25.934 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "32", 2024-08-31T06:23:25.934 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.935 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-08-31T06:23:25.935 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.935 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.935 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.935 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.935 INFO:teuthology.orchestra.run.smithi016.stdout: "username": { 2024-08-31T06:23:25.935 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "username", 2024-08-31T06:23:25.935 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.935 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.935 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.935 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.935 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.935 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.935 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.936 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "username of InfluxDB server user", 2024-08-31T06:23:25.936 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.936 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.936 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.936 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.936 INFO:teuthology.orchestra.run.smithi016.stdout: "verify_ssl": { 2024-08-31T06:23:25.936 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "verify_ssl", 2024-08-31T06:23:25.936 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.936 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.936 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.936 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "true", 2024-08-31T06:23:25.936 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.936 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.936 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.937 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-08-31T06:23:25.937 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.937 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.937 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.937 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.937 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.937 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.937 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.937 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "insights", 2024-08-31T06:23:25.937 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.937 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.937 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.937 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.937 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.937 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.938 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.938 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.938 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.938 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.938 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.938 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.938 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.938 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.938 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.938 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.938 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.938 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.938 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.938 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.939 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.939 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.939 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.939 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.939 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.939 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.939 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.939 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.939 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.939 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.939 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.939 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.939 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.939 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.940 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.940 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.940 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.940 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.940 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.940 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.940 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.940 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.940 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.940 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.940 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.940 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.940 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.940 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.940 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.941 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.941 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.941 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.941 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.941 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.941 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.941 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.941 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.941 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.941 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.941 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.941 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.941 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.942 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.942 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.942 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.942 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.942 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.942 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.942 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.942 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.942 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.942 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.942 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.942 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.942 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.942 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.943 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.943 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.943 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.943 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.943 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.943 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.943 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.943 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.943 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.943 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.943 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.943 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.943 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.943 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.944 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "iostat", 2024-08-31T06:23:25.944 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.944 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.944 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.944 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.944 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.944 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.944 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.944 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.944 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.944 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.944 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.944 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.945 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.945 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.945 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.945 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.945 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.945 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.945 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.945 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.945 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.945 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.945 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.945 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.945 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.945 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.946 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.946 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.946 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.946 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.946 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.946 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.946 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.946 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.946 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.946 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.946 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.946 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.946 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.946 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.947 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.947 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.947 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.947 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.947 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.947 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.947 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.947 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.947 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.947 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.947 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.947 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.947 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.948 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.948 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.948 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.948 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.948 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.948 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.948 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.948 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.948 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.948 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.948 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.948 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.948 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.949 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.949 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.949 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.949 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.949 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.949 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.949 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.949 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.949 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.949 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.949 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.949 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.949 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.949 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.950 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.950 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.950 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.950 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.950 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.950 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.950 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.950 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.950 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.950 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.950 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "k8sevents", 2024-08-31T06:23:25.950 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.950 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.950 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.950 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_event_retention_days": { 2024-08-31T06:23:25.951 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ceph_event_retention_days", 2024-08-31T06:23:25.951 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.951 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.951 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.951 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7", 2024-08-31T06:23:25.951 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.951 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.951 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.951 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Days to hold ceph event information within local cache", 2024-08-31T06:23:25.951 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.951 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.951 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.951 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.951 INFO:teuthology.orchestra.run.smithi016.stdout: "config_check_secs": { 2024-08-31T06:23:25.952 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_check_secs", 2024-08-31T06:23:25.952 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.952 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.952 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.952 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-08-31T06:23:25.952 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "10", 2024-08-31T06:23:25.952 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.952 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.952 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-08-31T06:23:25.952 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.952 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.952 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.952 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.952 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.953 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.953 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.953 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.953 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.953 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.953 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.953 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.953 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.953 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.953 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.953 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.953 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.953 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.953 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.954 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.954 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.954 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.954 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.954 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.954 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.954 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.954 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.954 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.954 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.954 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.954 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.954 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.954 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.954 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.955 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.955 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.955 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.955 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.955 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.955 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.955 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.955 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.955 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.955 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.955 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.955 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.955 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.956 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.956 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.956 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.956 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.956 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.956 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.956 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.956 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.956 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.956 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.956 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.956 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.956 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.956 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.957 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.957 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.957 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.957 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.957 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.957 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.957 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.957 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.957 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.957 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.957 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.957 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.957 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.957 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.958 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.958 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.958 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.958 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.958 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.958 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.958 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.958 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.958 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.958 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.958 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.958 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.958 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.958 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.959 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.959 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.959 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "localpool", 2024-08-31T06:23:25.959 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.959 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.959 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.959 INFO:teuthology.orchestra.run.smithi016.stdout: "failure_domain": { 2024-08-31T06:23:25.959 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "failure_domain", 2024-08-31T06:23:25.959 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.959 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.959 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.959 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "host", 2024-08-31T06:23:25.959 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.959 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.959 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.960 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "failure domain for any created local pool", 2024-08-31T06:23:25.960 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-08-31T06:23:25.960 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.960 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.960 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.960 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.960 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.960 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.960 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.960 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.960 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.960 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.960 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.961 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.961 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.961 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.961 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.961 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.961 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.961 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.961 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.961 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.961 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.961 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.961 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.961 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.961 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.962 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.962 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.962 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.962 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.962 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.962 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.962 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.962 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.962 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.962 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.962 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.962 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.962 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.962 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.962 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.963 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.963 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.963 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.963 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.963 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.963 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.963 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.963 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.963 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.963 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.963 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.963 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.963 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.963 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.964 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.964 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.964 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.964 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.964 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.964 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.964 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.964 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.964 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.964 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.964 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.964 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.964 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.964 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.965 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.965 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.965 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.965 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.965 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.965 INFO:teuthology.orchestra.run.smithi016.stdout: "min_size": { 2024-08-31T06:23:25.965 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_size", 2024-08-31T06:23:25.965 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.965 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.965 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.965 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.965 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.965 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.965 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.966 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default min_size for any created local pool", 2024-08-31T06:23:25.966 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-08-31T06:23:25.966 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.966 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.966 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.966 INFO:teuthology.orchestra.run.smithi016.stdout: "num_rep": { 2024-08-31T06:23:25.966 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "num_rep", 2024-08-31T06:23:25.966 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.966 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.966 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.966 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3", 2024-08-31T06:23:25.966 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.966 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.966 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.966 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default replica count for any created local pool", 2024-08-31T06:23:25.967 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.967 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.967 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.967 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.967 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num": { 2024-08-31T06:23:25.967 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_num", 2024-08-31T06:23:25.967 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.967 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.967 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.967 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "128", 2024-08-31T06:23:25.967 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.967 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.967 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.967 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default pg_num for any created local pool", 2024-08-31T06:23:25.968 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.968 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.968 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.968 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.968 INFO:teuthology.orchestra.run.smithi016.stdout: "prefix": { 2024-08-31T06:23:25.968 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prefix", 2024-08-31T06:23:25.968 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.968 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.968 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.968 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.968 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.968 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.968 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.969 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name prefix for any created local pool", 2024-08-31T06:23:25.969 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.969 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.969 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.969 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.969 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.969 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.969 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.969 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.969 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.969 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.969 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.969 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.969 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.970 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.970 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.970 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.970 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.970 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.970 INFO:teuthology.orchestra.run.smithi016.stdout: "subtree": { 2024-08-31T06:23:25.970 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "subtree", 2024-08-31T06:23:25.970 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.970 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.970 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.970 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "rack", 2024-08-31T06:23:25.970 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.970 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.970 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.971 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "CRUSH level for which to create a local pool", 2024-08-31T06:23:25.971 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-08-31T06:23:25.971 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.971 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.971 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.971 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.971 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.971 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.971 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mds_autoscaler", 2024-08-31T06:23:25.971 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.971 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.971 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.971 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.971 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.972 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.972 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.972 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.972 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.972 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.972 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.972 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.972 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.972 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.972 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.972 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.972 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.972 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.973 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.973 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.973 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.973 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.973 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.973 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.973 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.973 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.973 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.973 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.973 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.973 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.973 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.973 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.974 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.974 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.974 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.974 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.974 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.974 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.974 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.974 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.974 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.974 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.974 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.974 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.974 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.975 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.975 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.975 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.975 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.975 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.975 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.975 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.975 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.975 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.975 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.975 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.975 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.975 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.975 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.975 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.976 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.976 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.976 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.976 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.976 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.976 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.976 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.976 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.976 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.976 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.976 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.976 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.976 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.976 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.977 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.977 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.977 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.977 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.977 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.977 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.977 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.977 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.977 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.977 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.977 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.977 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.977 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.978 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.978 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.978 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.978 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirroring", 2024-08-31T06:23:25.978 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.978 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.978 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.978 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.978 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.978 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.978 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.978 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.978 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.978 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.979 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.979 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.979 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.979 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.979 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.979 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.979 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.979 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.979 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.979 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.979 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.979 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.979 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.979 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.980 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.980 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.980 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.980 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.980 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.980 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.980 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.980 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.980 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.980 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.980 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.980 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.980 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.980 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.981 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.981 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.981 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.981 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.981 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.981 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.981 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.981 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.981 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.981 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.981 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.981 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.981 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.981 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.981 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.982 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.982 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.982 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.982 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.982 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.982 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.982 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.982 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.982 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.982 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.982 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.982 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.982 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.982 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.983 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.983 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.983 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.983 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.983 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.983 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.983 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.983 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.983 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.983 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.983 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.983 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.983 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.983 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.984 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.984 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.984 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.984 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.984 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.984 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.984 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.984 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.984 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.984 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "nfs", 2024-08-31T06:23:25.984 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.984 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.984 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.985 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.985 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.985 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.985 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.985 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.985 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.985 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.985 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.985 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.985 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.985 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.985 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.985 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.985 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.986 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.986 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.986 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.986 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.986 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.986 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.986 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.986 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.986 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.986 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.986 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.986 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.986 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.986 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.987 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.987 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.987 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.987 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.987 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.987 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.987 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.987 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.987 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.987 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.987 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.987 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.987 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.987 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.988 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.988 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.988 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.988 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.988 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.988 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.988 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.988 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.988 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.988 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.988 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.988 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.988 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.988 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.989 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.989 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.989 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.989 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.989 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.989 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.989 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.989 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.989 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.989 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.989 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.989 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.989 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.989 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.990 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.990 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.990 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.990 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.990 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.990 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.990 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.990 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.990 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.990 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.990 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.990 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.990 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.990 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.990 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:25.991 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.991 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:25.991 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-08-31T06:23:25.991 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:25.991 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:25.991 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:25.991 INFO:teuthology.orchestra.run.smithi016.stdout: "fail_fs": { 2024-08-31T06:23:25.991 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "fail_fs", 2024-08-31T06:23:25.991 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.991 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.991 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:25.991 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.991 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.991 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.992 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.992 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-08-31T06:23:25.992 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.992 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.992 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.992 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.992 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:25.992 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:25.992 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.992 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.992 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.993 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.993 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.993 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.993 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.993 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.993 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.993 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.993 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.993 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.993 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.993 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.993 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.993 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.994 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.994 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.994 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.994 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:25.994 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:25.994 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.994 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.994 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.994 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.994 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.994 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.994 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.994 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.994 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.995 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.995 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.995 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.995 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:25.995 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:25.995 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.995 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.995 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.995 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:25.995 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.995 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.995 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.995 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:25.995 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:25.996 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:25.996 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:25.996 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:25.996 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:25.996 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.996 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.996 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.996 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.996 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.996 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.996 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:25.996 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:25.996 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:25.996 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.997 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.997 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:25.997 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.997 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.997 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.997 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.997 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.997 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.997 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.997 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.997 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator": { 2024-08-31T06:23:25.997 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-08-31T06:23:25.997 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:25.998 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:25.998 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.998 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:25.998 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.998 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.998 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:25.998 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2024-08-31T06:23:25.998 INFO:teuthology.orchestra.run.smithi016.stdout: "rook", 2024-08-31T06:23:25.998 INFO:teuthology.orchestra.run.smithi016.stdout: "test_orchestrator" 2024-08-31T06:23:25.998 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:25.998 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Orchestrator backend", 2024-08-31T06:23:25.998 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.998 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.998 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:25.999 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:25.999 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:25.999 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:25.999 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:25.999 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:25.999 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:25.999 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:25.999 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:25.999 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:25.999 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:25.999 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:25.999 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:25.999 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:25.999 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.000 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.000 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.000 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.000 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.000 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_perf_query", 2024-08-31T06:23:26.000 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.000 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.000 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.000 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.000 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.000 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.000 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.000 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.000 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.001 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.001 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.001 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.001 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.001 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.001 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.001 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.001 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.001 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.001 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.001 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.001 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.001 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.002 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.002 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.002 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.002 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.002 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.002 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.002 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.002 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.002 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.002 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.002 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.002 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.002 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.003 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.003 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.003 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.003 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.003 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.003 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.003 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.003 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.003 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.003 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.003 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.003 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.003 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.004 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.004 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.004 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.004 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.004 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.004 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.004 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.004 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.004 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.004 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.004 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.004 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.004 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.004 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.005 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.005 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.005 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.005 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.005 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.005 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.005 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.005 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.005 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.005 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.005 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.005 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.005 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.005 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.006 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.006 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.006 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.006 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.006 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.006 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.006 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.006 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.006 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.006 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.006 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.006 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.006 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.006 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.007 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_support", 2024-08-31T06:23:26.007 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.007 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.007 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.007 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.007 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.007 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.007 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.007 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.007 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.007 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.007 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.007 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.007 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.007 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.008 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.008 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.008 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.008 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.008 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.008 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.008 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.008 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.008 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.008 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.008 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.008 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.008 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.008 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.009 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.009 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.009 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.009 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.009 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.009 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.009 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.009 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.009 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.009 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.009 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.009 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.009 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.010 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.010 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.010 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.010 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.010 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.010 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.010 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.010 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.010 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.010 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.010 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.010 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.010 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.010 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.011 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.011 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.011 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.011 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.011 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.011 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.011 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.011 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.011 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.011 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.011 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.011 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.011 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.012 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.012 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.012 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.012 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.012 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.012 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.012 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.012 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.012 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.012 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.012 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.012 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.012 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.012 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.013 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.013 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.013 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.013 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.013 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.013 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.013 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.013 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.013 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_autoscaler", 2024-08-31T06:23:26.013 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.013 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.013 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.013 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.013 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.014 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.014 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.014 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.014 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.014 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.014 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.014 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.014 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.014 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.014 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.014 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.014 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.014 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.014 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.015 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.015 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.015 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.015 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.015 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.015 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.015 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.015 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.015 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.015 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.015 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.015 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.015 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.016 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.016 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.016 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.016 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.016 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.016 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.016 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.016 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.016 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.016 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.016 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.016 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.016 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.016 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.017 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.017 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.017 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.017 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.017 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.017 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.017 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.017 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.017 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.017 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.017 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.017 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.017 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.018 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.018 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.018 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.018 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.018 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.018 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.018 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.018 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.018 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.018 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.018 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.018 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.018 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.019 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.019 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-08-31T06:23:26.019 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-08-31T06:23:26.019 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:26.019 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.019 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.019 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-08-31T06:23:26.019 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.019 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.019 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.019 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.019 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.020 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.020 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.020 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.020 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.020 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.020 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.020 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.020 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.020 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.020 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.020 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.020 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.020 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.020 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.021 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.021 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.021 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.021 INFO:teuthology.orchestra.run.smithi016.stdout: "threshold": { 2024-08-31T06:23:26.021 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threshold", 2024-08-31T06:23:26.021 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-08-31T06:23:26.021 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.021 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.021 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-08-31T06:23:26.021 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1.0", 2024-08-31T06:23:26.021 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.021 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.021 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "scaling threshold", 2024-08-31T06:23:26.022 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-08-31T06:23:26.022 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.022 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.022 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.022 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.022 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.022 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.022 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "progress", 2024-08-31T06:23:26.022 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.022 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.022 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.022 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_pg_recovery_event": { 2024-08-31T06:23:26.022 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_pg_recovery_event", 2024-08-31T06:23:26.022 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.023 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.023 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.023 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.023 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.023 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.023 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.023 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow the module to show pg recovery progress", 2024-08-31T06:23:26.023 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.023 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.023 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.023 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.023 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-08-31T06:23:26.023 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-08-31T06:23:26.023 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.024 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.024 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.024 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.024 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.024 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.024 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.024 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.024 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.024 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.024 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.024 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.024 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.024 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.024 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.025 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.025 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.025 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.025 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.025 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.025 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.025 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.025 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.025 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.025 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.025 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.025 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.025 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.025 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.026 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.026 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.026 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.026 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.026 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.026 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.026 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.026 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.026 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.026 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.026 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.026 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.026 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.026 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.027 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.027 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.027 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.027 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.027 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.027 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.027 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.027 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.027 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.027 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.027 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.027 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.027 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.027 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.028 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.028 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.028 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.028 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.028 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.028 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.028 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.028 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.028 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.028 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.028 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.028 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.028 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.028 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.029 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.029 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.029 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.029 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.029 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.029 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.029 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.029 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.029 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.029 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.029 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.029 INFO:teuthology.orchestra.run.smithi016.stdout: "max_completed_events": { 2024-08-31T06:23:26.029 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_completed_events", 2024-08-31T06:23:26.029 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.030 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.030 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.030 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "50", 2024-08-31T06:23:26.030 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.030 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.030 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.030 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "number of past completed events to remember", 2024-08-31T06:23:26.030 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.030 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.030 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.030 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.030 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-08-31T06:23:26.030 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-08-31T06:23:26.030 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:26.031 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.031 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.031 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-08-31T06:23:26.031 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.031 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.031 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.031 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long the module is going to sleep", 2024-08-31T06:23:26.031 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.031 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.031 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.031 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.031 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.031 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.031 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.032 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.032 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.032 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.032 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.032 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.032 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.032 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.032 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.032 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.032 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.032 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.032 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.032 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.032 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.032 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus", 2024-08-31T06:23:26.033 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.033 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.033 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.033 INFO:teuthology.orchestra.run.smithi016.stdout: "cache": { 2024-08-31T06:23:26.033 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cache", 2024-08-31T06:23:26.033 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.033 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.033 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.033 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.033 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.033 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.033 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.033 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.033 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.034 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.034 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.034 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.034 INFO:teuthology.orchestra.run.smithi016.stdout: "exclude_perf_counters": { 2024-08-31T06:23:26.034 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "exclude_perf_counters", 2024-08-31T06:23:26.034 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.034 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.034 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.034 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.034 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.034 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.034 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.034 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-08-31T06:23:26.034 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-08-31T06:23:26.035 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.035 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.035 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.035 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.035 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.035 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.035 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.035 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.035 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.035 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.035 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.035 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.035 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.035 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.035 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.036 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.036 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.036 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.036 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.036 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.036 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.036 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.036 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.036 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.036 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.036 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.036 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.036 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.036 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.037 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.037 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.037 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.037 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.037 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.037 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.037 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.037 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.037 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.037 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.037 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.037 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.037 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.037 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.038 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.038 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.038 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.038 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.038 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.038 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.038 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.038 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.038 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.038 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.038 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.038 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.038 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.039 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.039 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.039 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.039 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.039 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.039 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.039 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.039 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.039 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.039 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.039 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.039 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.039 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.039 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.040 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.040 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.040 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.040 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools": { 2024-08-31T06:23:26.040 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools", 2024-08-31T06:23:26.040 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.040 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.040 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.040 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.040 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.040 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.040 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.040 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.040 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.041 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.041 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.041 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.041 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools_refresh_interval": { 2024-08-31T06:23:26.041 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-08-31T06:23:26.041 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.041 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.041 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.041 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "300", 2024-08-31T06:23:26.041 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.041 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.041 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.041 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.042 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.042 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.042 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.042 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.042 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_interval": { 2024-08-31T06:23:26.042 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_interval", 2024-08-31T06:23:26.042 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-08-31T06:23:26.042 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.042 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.042 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15.0", 2024-08-31T06:23:26.042 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.042 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.042 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.042 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.043 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.043 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.043 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.043 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.043 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-08-31T06:23:26.043 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-08-31T06:23:26.043 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.043 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.043 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.043 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-08-31T06:23:26.043 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.043 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.043 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.043 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-08-31T06:23:26.044 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.044 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.044 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.044 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.044 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-08-31T06:23:26.044 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-08-31T06:23:26.044 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.044 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.044 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.044 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "9283", 2024-08-31T06:23:26.044 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.044 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.044 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.045 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-08-31T06:23:26.045 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.045 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.045 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.045 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.045 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.045 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.045 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.045 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.045 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.045 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.045 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.045 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.045 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.045 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.046 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.046 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.046 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.046 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.046 INFO:teuthology.orchestra.run.smithi016.stdout: "stale_cache_strategy": { 2024-08-31T06:23:26.046 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stale_cache_strategy", 2024-08-31T06:23:26.046 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.046 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.046 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.046 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "log", 2024-08-31T06:23:26.046 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.046 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.046 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.046 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.047 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.047 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.047 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.047 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.047 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-08-31T06:23:26.047 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-08-31T06:23:26.047 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.047 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.047 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.047 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "default", 2024-08-31T06:23:26.047 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.047 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.047 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.047 INFO:teuthology.orchestra.run.smithi016.stdout: "default", 2024-08-31T06:23:26.047 INFO:teuthology.orchestra.run.smithi016.stdout: "error" 2024-08-31T06:23:26.048 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.048 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.048 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.048 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.048 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.048 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.048 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-08-31T06:23:26.048 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-08-31T06:23:26.048 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.048 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.048 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.048 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-08-31T06:23:26.048 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-08-31T06:23:26.049 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-08-31T06:23:26.049 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.049 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.049 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.049 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.049 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.049 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.049 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.049 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.049 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.049 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2024-08-31T06:23:26.049 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.049 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.049 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.049 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.050 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.050 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.050 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.050 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.050 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.050 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.050 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.050 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.050 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.050 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.050 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.050 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.050 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.050 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.051 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.051 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.051 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.051 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.051 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.051 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.051 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.051 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.051 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.051 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.051 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.051 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.051 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.051 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.051 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.052 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.052 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.052 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.052 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.052 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.052 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.052 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.052 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.052 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.052 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.052 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.052 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.052 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.053 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.053 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.053 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.053 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.053 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.053 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.053 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.053 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.053 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.053 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.053 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.053 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.053 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.054 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.054 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.054 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.054 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.054 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.054 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.054 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.054 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.054 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.054 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.054 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.054 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.054 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.054 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.055 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_snap_create": { 2024-08-31T06:23:26.055 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_snap_create", 2024-08-31T06:23:26.055 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.055 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.055 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.055 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-08-31T06:23:26.055 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.055 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.055 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.055 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.055 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.055 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.055 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.055 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.056 INFO:teuthology.orchestra.run.smithi016.stdout: "mirror_snapshot_schedule": { 2024-08-31T06:23:26.056 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirror_snapshot_schedule", 2024-08-31T06:23:26.056 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.056 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.056 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.056 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.056 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.056 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.056 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.056 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.056 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.056 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.056 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.056 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.057 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.057 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.057 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.057 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.057 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.057 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.057 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.057 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.057 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.057 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.057 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.057 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.057 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.057 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.058 INFO:teuthology.orchestra.run.smithi016.stdout: "trash_purge_schedule": { 2024-08-31T06:23:26.058 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "trash_purge_schedule", 2024-08-31T06:23:26.058 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.058 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.058 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.058 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.058 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.058 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.058 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.058 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.058 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.058 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.058 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.058 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.059 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.059 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.059 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.059 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "restful", 2024-08-31T06:23:26.059 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.059 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.059 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.059 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_auth": { 2024-08-31T06:23:26.059 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_auth", 2024-08-31T06:23:26.059 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.059 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.059 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.059 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.059 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.060 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.060 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.060 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.060 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.060 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.060 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.060 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.060 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-08-31T06:23:26.060 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-08-31T06:23:26.060 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.060 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.060 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.060 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.060 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.061 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.061 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.061 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.061 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.061 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.061 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.061 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.061 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.061 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.061 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.061 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.061 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.061 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.061 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.061 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.062 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.062 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.062 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.062 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.062 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.062 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.062 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.062 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.062 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.062 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.062 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.062 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.062 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.062 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.063 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.063 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.063 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.063 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.063 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.063 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.063 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.063 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.063 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.063 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.063 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.063 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.063 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.063 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.064 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.064 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.064 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.064 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.064 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.064 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.064 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.064 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.064 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.064 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.064 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.064 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.064 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.064 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.065 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.065 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.065 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.065 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.065 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.065 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.065 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.065 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.065 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.065 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.065 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.065 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.065 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.065 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.065 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.066 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.066 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.066 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.066 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.066 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.066 INFO:teuthology.orchestra.run.smithi016.stdout: "max_requests": { 2024-08-31T06:23:26.066 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_requests", 2024-08-31T06:23:26.066 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.066 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.066 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.066 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-08-31T06:23:26.066 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.066 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.067 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.067 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-08-31T06:23:26.067 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.067 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.067 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.067 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.067 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-08-31T06:23:26.067 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-08-31T06:23:26.067 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.067 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.067 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.067 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.067 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.067 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.067 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.068 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.068 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.068 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.068 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.068 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.068 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-08-31T06:23:26.068 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-08-31T06:23:26.068 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.068 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.068 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.068 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.068 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.068 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.068 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.069 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.069 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.069 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.069 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.069 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.069 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.069 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.069 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.069 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.069 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.069 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.069 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.069 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.069 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.070 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.070 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.070 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.070 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.070 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.070 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.070 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.070 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.070 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rgw", 2024-08-31T06:23:26.070 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.070 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.070 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.070 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.070 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.071 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.071 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.071 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.071 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.071 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.071 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.071 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.071 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.071 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.071 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.071 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.071 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.071 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.071 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.072 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.072 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.072 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.072 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.072 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.072 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.072 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.072 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.072 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.072 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.072 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.072 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.072 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.072 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.072 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.073 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.073 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.073 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.073 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.073 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.073 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.073 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.073 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.073 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.073 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.073 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.073 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.073 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.073 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.074 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.074 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.074 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.074 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.074 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.074 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.074 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.074 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.074 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.074 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.074 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.074 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.074 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.074 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.075 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.075 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.075 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.075 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.075 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.075 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.075 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.075 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.075 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.075 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.075 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.075 INFO:teuthology.orchestra.run.smithi016.stdout: "secondary_zone_period_retry_limit": { 2024-08-31T06:23:26.075 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secondary_zone_period_retry_limit", 2024-08-31T06:23:26.075 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.075 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.076 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.076 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-08-31T06:23:26.076 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.076 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.076 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.076 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-08-31T06:23:26.076 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.076 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.076 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.076 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.076 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.076 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.076 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.077 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.077 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.077 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.077 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.077 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.077 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.077 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.077 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.077 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.077 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.077 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.077 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.077 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.078 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.078 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rook", 2024-08-31T06:23:26.078 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.078 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.078 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.078 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.078 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.078 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.078 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.078 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.078 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.078 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.078 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.078 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.078 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.078 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.079 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.079 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.079 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.079 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.079 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.079 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.079 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.079 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.079 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.079 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.079 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.079 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.079 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.080 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.080 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.080 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.080 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.080 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.080 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.080 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.080 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.080 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.080 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.080 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.080 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.080 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.080 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.081 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.081 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.081 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.081 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.081 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.081 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.081 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.081 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.081 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.081 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.081 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.081 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.081 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.081 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.081 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.082 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.082 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.082 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.082 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.082 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.082 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.082 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.082 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.082 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.082 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.082 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.082 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.082 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.083 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.083 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.083 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.083 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.083 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.083 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.083 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.083 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.083 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.083 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.083 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.083 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.083 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.083 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.083 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.084 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.084 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.084 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.084 INFO:teuthology.orchestra.run.smithi016.stdout: "storage_class": { 2024-08-31T06:23:26.084 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "storage_class", 2024-08-31T06:23:26.084 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.084 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.084 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.084 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "local", 2024-08-31T06:23:26.084 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.084 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.084 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.084 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-08-31T06:23:26.084 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.085 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.085 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.085 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.085 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.085 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.085 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.085 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "selftest", 2024-08-31T06:23:26.085 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.085 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.085 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.085 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.085 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.085 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.085 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.086 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.086 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.086 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.086 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.086 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.086 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.086 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.086 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.086 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.086 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.086 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.086 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.086 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.086 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.087 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.087 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.087 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.087 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.087 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.087 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.087 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.087 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.087 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.087 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.087 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.087 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.087 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.087 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.088 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.088 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.088 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.088 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.088 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.088 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.088 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.088 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.088 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.088 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.088 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.088 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.088 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.088 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.089 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.089 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.089 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.089 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.089 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.089 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.089 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.089 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.089 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.089 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.089 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.089 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.089 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.089 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.090 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.090 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.090 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.090 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.090 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.090 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.090 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.090 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.090 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.090 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.090 INFO:teuthology.orchestra.run.smithi016.stdout: "roption1": { 2024-08-31T06:23:26.090 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption1", 2024-08-31T06:23:26.090 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.090 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.091 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.091 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.091 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.091 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.091 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.091 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.091 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.091 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.091 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.091 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.091 INFO:teuthology.orchestra.run.smithi016.stdout: "roption2": { 2024-08-31T06:23:26.091 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption2", 2024-08-31T06:23:26.091 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.091 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.091 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.092 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "xyz", 2024-08-31T06:23:26.092 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.092 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.092 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.092 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.092 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.092 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.092 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.092 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.092 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption1": { 2024-08-31T06:23:26.092 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption1", 2024-08-31T06:23:26.092 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.092 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.093 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.093 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.093 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.093 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.093 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.093 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.093 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.093 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.093 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.094 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.094 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption2": { 2024-08-31T06:23:26.094 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption2", 2024-08-31T06:23:26.094 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.094 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.094 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.094 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.094 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.094 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.094 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.094 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.094 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.094 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.094 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.095 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.095 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption3": { 2024-08-31T06:23:26.095 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption3", 2024-08-31T06:23:26.095 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-08-31T06:23:26.095 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.095 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.095 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.095 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.095 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.095 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.095 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.095 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.095 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.095 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.096 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.096 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption4": { 2024-08-31T06:23:26.096 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption4", 2024-08-31T06:23:26.096 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.096 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.096 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.096 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.096 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.096 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.096 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.096 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.096 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.096 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.097 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.097 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.097 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption5": { 2024-08-31T06:23:26.097 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption5", 2024-08-31T06:23:26.097 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.097 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.097 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.097 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.097 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.097 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.097 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.097 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.097 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.097 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.098 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.098 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.098 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption6": { 2024-08-31T06:23:26.098 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption6", 2024-08-31T06:23:26.098 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.098 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.098 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.098 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.098 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.098 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.098 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.098 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.098 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.098 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.099 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.099 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.099 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption7": { 2024-08-31T06:23:26.099 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption7", 2024-08-31T06:23:26.099 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.099 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.099 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.099 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.099 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-08-31T06:23:26.099 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "42", 2024-08-31T06:23:26.099 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.099 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.099 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.099 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.100 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.100 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.100 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.100 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.100 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.100 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.100 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.100 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.100 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.100 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.100 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.100 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.100 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.100 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.101 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.101 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.101 INFO:teuthology.orchestra.run.smithi016.stdout: "testkey": { 2024-08-31T06:23:26.101 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testkey", 2024-08-31T06:23:26.101 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.101 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.101 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.101 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.101 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.101 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.101 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.101 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.101 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.102 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.102 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.102 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.102 INFO:teuthology.orchestra.run.smithi016.stdout: "testlkey": { 2024-08-31T06:23:26.102 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testlkey", 2024-08-31T06:23:26.102 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.102 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.102 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.102 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.102 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.102 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.102 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.102 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.102 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.103 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.103 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.103 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.103 INFO:teuthology.orchestra.run.smithi016.stdout: "testnewline": { 2024-08-31T06:23:26.103 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testnewline", 2024-08-31T06:23:26.103 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.103 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.103 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.103 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.103 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.103 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.103 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.103 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.103 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.104 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.104 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.104 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.104 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.104 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.104 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.104 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smb", 2024-08-31T06:23:26.104 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.104 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.104 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.104 INFO:teuthology.orchestra.run.smithi016.stdout: "internal_store_backend": { 2024-08-31T06:23:26.104 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "internal_store_backend", 2024-08-31T06:23:26.104 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.104 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.105 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.105 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.105 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.105 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.105 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.105 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-08-31T06:23:26.105 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.105 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.105 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.105 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.105 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.105 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.105 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.105 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.106 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.106 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.106 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.106 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.106 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.106 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.106 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.106 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.106 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.106 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.106 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.106 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.106 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.106 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.107 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.107 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.107 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.107 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.107 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.107 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.107 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.107 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.107 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.107 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.107 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.107 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.107 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.108 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.108 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.108 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.108 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.108 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.108 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.108 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.108 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.108 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.108 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.108 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.108 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.108 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.109 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.109 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.109 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.109 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.109 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.109 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.109 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.109 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.109 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.109 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.109 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.109 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.109 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.109 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.110 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.110 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.110 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.110 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.110 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.110 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.110 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.110 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.110 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.110 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.110 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.110 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.110 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.110 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.111 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.111 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.111 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.111 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.111 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.111 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.111 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.111 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.111 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.111 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.111 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.111 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.111 INFO:teuthology.orchestra.run.smithi016.stdout: "update_orchestration": { 2024-08-31T06:23:26.111 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "update_orchestration", 2024-08-31T06:23:26.112 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.112 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.112 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.112 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.112 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.112 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.112 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.112 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-08-31T06:23:26.112 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.112 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.112 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.112 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.112 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.112 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.113 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.113 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-08-31T06:23:26.113 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.113 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.113 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.113 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_m_granularity": { 2024-08-31T06:23:26.113 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_m_granularity", 2024-08-31T06:23:26.113 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.113 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.113 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.113 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.113 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.113 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.114 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.114 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow minute scheduled snapshots", 2024-08-31T06:23:26.114 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.114 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.114 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.114 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.114 INFO:teuthology.orchestra.run.smithi016.stdout: "dump_on_update": { 2024-08-31T06:23:26.114 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dump_on_update", 2024-08-31T06:23:26.114 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.114 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.114 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.114 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.114 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.114 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.115 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.115 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "dump database to debug log on update", 2024-08-31T06:23:26.115 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.115 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.115 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.115 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.115 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.115 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.115 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.115 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.115 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.115 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.115 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.115 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.116 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.116 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.116 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.116 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.116 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.116 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.116 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.116 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.116 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.116 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.116 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.116 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.116 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.117 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.117 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.117 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.117 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.117 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.117 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.117 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.117 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.117 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.117 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.117 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.117 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.117 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.117 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.118 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.118 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.118 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.118 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.118 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.118 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.118 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.118 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.118 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.118 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.118 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.118 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.118 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.118 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.119 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.119 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.119 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.119 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.119 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.119 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.119 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.119 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.119 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.119 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.119 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.119 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.119 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.119 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.120 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.120 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.120 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.120 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.120 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.120 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.120 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.120 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.120 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.120 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.120 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.120 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.120 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.120 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.121 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.121 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.121 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.121 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.121 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.121 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.121 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.121 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.121 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.121 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.121 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stats", 2024-08-31T06:23:26.121 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.121 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.122 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.122 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.122 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.122 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.122 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.122 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.122 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.122 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.122 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.122 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.122 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.122 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.122 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.122 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.123 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.123 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.123 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.123 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.123 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.123 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.123 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.123 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.123 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.123 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.123 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.123 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.123 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.123 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.124 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.124 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.124 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.124 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.124 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.124 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.124 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.124 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.124 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.124 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.124 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.124 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.124 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.124 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.125 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.125 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.125 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.125 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.125 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.125 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.125 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.125 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.125 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.125 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.125 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.125 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.125 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.125 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.126 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.126 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.126 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.126 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.126 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.126 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.126 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.126 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.126 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.126 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.127 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.127 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.127 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.127 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.127 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.127 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.127 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.127 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.127 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.127 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.127 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.127 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.127 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.128 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.128 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.128 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.128 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.128 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.128 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.128 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.128 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.128 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.128 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "status", 2024-08-31T06:23:26.128 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.128 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.128 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.128 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.129 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.129 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.129 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.129 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.129 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.129 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.129 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.129 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.129 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.129 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.129 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.129 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.129 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.130 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.130 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.130 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.130 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.130 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.130 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.130 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.130 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.130 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.130 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.130 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.130 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.130 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.130 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.130 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.131 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.131 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.131 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.131 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.131 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.131 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.131 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.131 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.131 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.131 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.131 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.131 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.131 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.131 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.131 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.132 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.132 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.132 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.132 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.132 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.132 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.132 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.132 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.132 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.132 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.132 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.132 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.132 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.132 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.133 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.133 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.133 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.133 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.133 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.133 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.133 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.133 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.133 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.133 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.133 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.133 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.133 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.133 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.134 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.134 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.134 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.134 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.134 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.134 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.134 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.134 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.134 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.134 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.134 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.134 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.134 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.134 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.135 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.135 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telegraf", 2024-08-31T06:23:26.135 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.135 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.135 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.135 INFO:teuthology.orchestra.run.smithi016.stdout: "address": { 2024-08-31T06:23:26.135 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "address", 2024-08-31T06:23:26.135 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.135 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.135 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.135 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-08-31T06:23:26.135 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.135 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.135 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.136 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.136 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.136 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.136 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.136 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.136 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-08-31T06:23:26.136 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-08-31T06:23:26.136 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:26.136 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.136 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.136 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15", 2024-08-31T06:23:26.136 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.136 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.137 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.137 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.137 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.137 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.137 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.137 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.137 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.137 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.137 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.137 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.137 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.137 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.137 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.138 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.138 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.138 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.138 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.138 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.138 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.138 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.138 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.138 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.138 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.138 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.138 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.138 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.138 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.138 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.139 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.139 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.139 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.139 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.139 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.139 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.139 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.139 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.139 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.139 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.139 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.139 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.139 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.139 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.139 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.140 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.140 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.140 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.140 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.140 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.140 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.140 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.140 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.140 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.140 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.140 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.140 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.140 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.141 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.141 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.141 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.141 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.141 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.141 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.141 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.141 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.141 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.141 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.141 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.141 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.141 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.141 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.142 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.142 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.142 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.142 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.142 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.142 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.142 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.142 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.142 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.142 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.142 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.142 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.142 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.143 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.143 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.143 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.143 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.143 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.143 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.143 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.143 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.143 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.143 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.143 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telemetry", 2024-08-31T06:23:26.143 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.143 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.143 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.144 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_basic": { 2024-08-31T06:23:26.144 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_basic", 2024-08-31T06:23:26.144 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.144 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.144 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.144 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.144 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.144 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.144 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.144 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share basic cluster information (size, version)", 2024-08-31T06:23:26.144 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.144 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.144 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.145 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.145 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_crash": { 2024-08-31T06:23:26.145 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_crash", 2024-08-31T06:23:26.145 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.145 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.145 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.145 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.145 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.145 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.145 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.145 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-08-31T06:23:26.145 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.145 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.145 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.145 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.146 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_device": { 2024-08-31T06:23:26.146 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_device", 2024-08-31T06:23:26.146 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.146 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.146 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.146 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.146 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.146 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.146 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.146 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-08-31T06:23:26.146 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.146 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.146 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.146 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.147 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_ident": { 2024-08-31T06:23:26.147 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_ident", 2024-08-31T06:23:26.147 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.147 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.147 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.147 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.147 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.147 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.147 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.147 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-08-31T06:23:26.147 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.147 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.147 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.147 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.147 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_perf": { 2024-08-31T06:23:26.148 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_perf", 2024-08-31T06:23:26.148 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.148 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.148 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.148 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.148 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.148 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.148 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.148 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share various performance metrics of a cluster", 2024-08-31T06:23:26.148 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.148 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.148 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.148 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.148 INFO:teuthology.orchestra.run.smithi016.stdout: "contact": { 2024-08-31T06:23:26.149 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "contact", 2024-08-31T06:23:26.149 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.149 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.149 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.149 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.149 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.149 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.149 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.149 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.149 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.149 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.149 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.149 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.149 INFO:teuthology.orchestra.run.smithi016.stdout: "description": { 2024-08-31T06:23:26.150 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "description", 2024-08-31T06:23:26.150 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.150 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.150 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.150 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.150 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.150 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.150 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.150 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.150 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.150 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.150 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.150 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.150 INFO:teuthology.orchestra.run.smithi016.stdout: "device_url": { 2024-08-31T06:23:26.151 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_url", 2024-08-31T06:23:26.151 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.151 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.151 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.151 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-08-31T06:23:26.151 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.151 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.151 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.151 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.151 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.151 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.151 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.151 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.151 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-08-31T06:23:26.152 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-08-31T06:23:26.152 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.152 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.152 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.152 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.152 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.152 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.152 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.152 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.152 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.152 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.152 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.152 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.152 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-08-31T06:23:26.153 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-08-31T06:23:26.153 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.153 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.153 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.153 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "24", 2024-08-31T06:23:26.153 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "8", 2024-08-31T06:23:26.153 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.153 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.153 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.153 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.153 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.153 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.153 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.153 INFO:teuthology.orchestra.run.smithi016.stdout: "last_opt_revision": { 2024-08-31T06:23:26.154 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "last_opt_revision", 2024-08-31T06:23:26.154 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.154 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.154 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.154 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1", 2024-08-31T06:23:26.154 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.154 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.154 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.154 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.154 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.154 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.154 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.154 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.154 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard": { 2024-08-31T06:23:26.155 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard", 2024-08-31T06:23:26.155 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.155 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.155 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.155 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.155 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.155 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.155 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.155 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.155 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.155 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.155 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.155 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.155 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard_description": { 2024-08-31T06:23:26.156 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard_description", 2024-08-31T06:23:26.156 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.156 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.156 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.156 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.156 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.156 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.156 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.156 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.156 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.156 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.156 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.156 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.156 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.157 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.157 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.157 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.157 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.157 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.157 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.157 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.157 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.157 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.157 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.157 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.157 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.157 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.157 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.158 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.158 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.158 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.158 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.158 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.158 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.158 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.158 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.158 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.158 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.158 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.158 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.158 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.158 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.159 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.159 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.159 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.159 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.159 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.159 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.159 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.159 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.159 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.159 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.159 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.159 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.159 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.159 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.159 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.160 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.160 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.160 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.160 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.160 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.160 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.160 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.160 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.160 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.160 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.160 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.160 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.160 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.160 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.161 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.161 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.161 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.161 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.161 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.161 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.161 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.161 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.161 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.161 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.161 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.161 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.161 INFO:teuthology.orchestra.run.smithi016.stdout: "organization": { 2024-08-31T06:23:26.161 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "organization", 2024-08-31T06:23:26.162 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.162 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.162 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.162 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.162 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.162 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.162 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.162 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.162 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.162 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.162 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.162 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.162 INFO:teuthology.orchestra.run.smithi016.stdout: "proxy": { 2024-08-31T06:23:26.162 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "proxy", 2024-08-31T06:23:26.162 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.163 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.163 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.163 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.163 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.163 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.163 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.163 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.163 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.163 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.163 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.163 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.163 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.163 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.163 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.164 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.164 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.164 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.164 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.164 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.164 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.164 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.164 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.164 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.164 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.164 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.164 INFO:teuthology.orchestra.run.smithi016.stdout: "url": { 2024-08-31T06:23:26.164 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url", 2024-08-31T06:23:26.165 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.165 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.165 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.165 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-08-31T06:23:26.165 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.165 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.165 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.165 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.165 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.165 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.165 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.165 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.165 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.165 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.166 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.166 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "test_orchestrator", 2024-08-31T06:23:26.166 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.166 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.166 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.166 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.166 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.166 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.166 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.166 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.166 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.166 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.166 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.166 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.167 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.167 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.167 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.167 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.167 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.167 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.167 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.167 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.167 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.167 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.167 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.167 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.167 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.167 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.167 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.168 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.168 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.168 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.168 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.168 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.168 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.168 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.168 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.168 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.168 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.168 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.168 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.168 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.169 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.169 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.169 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.169 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.169 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.169 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.169 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.169 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.169 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.169 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.169 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.169 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.169 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.169 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.170 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.170 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.170 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.170 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.170 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.170 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.170 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.170 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.170 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.170 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.170 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.170 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.170 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.170 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.171 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.171 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.171 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.171 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.171 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.171 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.171 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.171 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.171 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.171 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.171 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.171 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.171 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.171 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.172 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.172 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.172 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.172 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.172 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.172 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.172 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.172 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.172 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2024-08-31T06:23:26.172 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.172 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.172 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.172 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.172 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.173 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.173 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.173 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.173 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.173 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.173 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.173 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.173 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.173 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.173 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.173 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.173 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.173 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.173 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.174 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.174 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.174 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.174 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.174 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.174 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.174 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.174 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.174 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.174 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.174 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.174 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.174 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.174 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.175 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.175 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.175 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.175 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.175 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.175 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.175 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.175 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.175 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.175 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.175 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.175 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.175 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.175 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.176 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.176 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.176 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.176 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.176 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.176 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.176 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.176 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.176 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.176 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.176 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.176 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.176 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.176 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.177 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.177 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.177 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.177 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.177 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.177 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.177 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.177 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.177 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.177 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.177 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.177 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.177 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_clones": { 2024-08-31T06:23:26.178 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_clones", 2024-08-31T06:23:26.178 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.178 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.178 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.178 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4", 2024-08-31T06:23:26.178 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.178 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.178 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.178 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Number of asynchronous cloner threads", 2024-08-31T06:23:26.178 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.178 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.178 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.178 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.178 INFO:teuthology.orchestra.run.smithi016.stdout: "periodic_async_work": { 2024-08-31T06:23:26.179 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "periodic_async_work", 2024-08-31T06:23:26.179 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.179 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.179 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.179 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.179 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.179 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.179 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.179 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Periodically check for async work", 2024-08-31T06:23:26.179 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.179 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.179 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.179 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.179 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_delay": { 2024-08-31T06:23:26.180 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_delay", 2024-08-31T06:23:26.180 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.180 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.180 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.180 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.180 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.180 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.180 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.180 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-08-31T06:23:26.180 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.180 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.180 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.180 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.181 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_no_wait": { 2024-08-31T06:23:26.181 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_no_wait", 2024-08-31T06:23:26.181 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.181 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.181 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.181 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.181 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.181 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.181 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.181 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-08-31T06:23:26.181 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.181 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.181 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.182 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.182 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.182 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.182 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.182 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.182 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.182 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.182 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.182 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.182 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.182 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.182 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.182 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.182 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.183 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.183 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.183 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.183 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.183 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix", 2024-08-31T06:23:26.183 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.183 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.183 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.183 INFO:teuthology.orchestra.run.smithi016.stdout: "discovery_interval": { 2024-08-31T06:23:26.183 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "discovery_interval", 2024-08-31T06:23:26.183 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-08-31T06:23:26.183 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.183 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.183 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "100", 2024-08-31T06:23:26.184 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.184 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.184 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.184 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.184 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.184 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.184 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.184 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.184 INFO:teuthology.orchestra.run.smithi016.stdout: "identifier": { 2024-08-31T06:23:26.184 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "identifier", 2024-08-31T06:23:26.184 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.184 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.184 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.184 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.185 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.185 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.185 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.185 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.185 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.185 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.185 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.185 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.185 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-08-31T06:23:26.185 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-08-31T06:23:26.185 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:26.185 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.185 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.185 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-08-31T06:23:26.186 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.186 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.186 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.186 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.186 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.186 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.186 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.186 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.186 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.186 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.186 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.186 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.186 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.186 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.187 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.187 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.187 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.187 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.187 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.187 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.187 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.187 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.187 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.187 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.187 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.187 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.187 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.187 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.188 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.188 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.188 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.188 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.188 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.188 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.188 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.188 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.188 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.188 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.188 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.188 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.188 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.189 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.189 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.189 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.189 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.189 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.189 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.189 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.189 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.189 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.189 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.189 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.189 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.189 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.189 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.189 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.190 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.190 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.190 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.190 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.190 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.190 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.190 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.190 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.190 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.190 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.190 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.190 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.190 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.191 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.191 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.191 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.191 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.191 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.191 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.191 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.191 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.191 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.191 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.191 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.191 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.191 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.192 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.192 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.192 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.192 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.192 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.192 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.192 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.192 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.192 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.192 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.192 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_host": { 2024-08-31T06:23:26.192 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_host", 2024-08-31T06:23:26.192 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.192 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.193 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.193 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.193 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.193 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.193 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.193 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.193 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.193 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.193 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.194 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.194 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_port": { 2024-08-31T06:23:26.194 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_port", 2024-08-31T06:23:26.194 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.194 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.194 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.194 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10051", 2024-08-31T06:23:26.194 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.194 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.194 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.194 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.194 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.194 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.194 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.195 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.195 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_sender": { 2024-08-31T06:23:26.195 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_sender", 2024-08-31T06:23:26.195 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.195 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.195 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.195 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-08-31T06:23:26.195 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.195 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.195 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.195 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.195 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.195 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.195 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.196 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.196 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.196 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.196 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-08-31T06:23:26.196 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.196 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.196 INFO:teuthology.orchestra.run.smithi016.stdout: "modules": [ 2024-08-31T06:23:26.196 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2024-08-31T06:23:26.196 INFO:teuthology.orchestra.run.smithi016.stdout: "dashboard", 2024-08-31T06:23:26.196 INFO:teuthology.orchestra.run.smithi016.stdout: "iostat", 2024-08-31T06:23:26.196 INFO:teuthology.orchestra.run.smithi016.stdout: "nfs", 2024-08-31T06:23:26.196 INFO:teuthology.orchestra.run.smithi016.stdout: "restful", 2024-08-31T06:23:26.196 INFO:teuthology.orchestra.run.smithi016.stdout: "snap_schedule" 2024-08-31T06:23:26.196 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.197 INFO:teuthology.orchestra.run.smithi016.stdout: "available_modules": [ 2024-08-31T06:23:26.197 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.197 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "alerts", 2024-08-31T06:23:26.197 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.197 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.197 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.197 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-08-31T06:23:26.197 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-08-31T06:23:26.197 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:26.197 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.197 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.197 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-08-31T06:23:26.197 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.197 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.198 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.198 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How frequently to reexamine health status", 2024-08-31T06:23:26.198 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.198 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.198 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.198 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.198 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.198 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.198 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.198 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.198 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.198 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.198 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.198 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.199 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.199 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.199 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.199 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.199 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.199 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.199 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.199 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.199 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.199 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.199 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.199 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.199 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.199 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.200 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.200 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.200 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.200 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.200 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.200 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.200 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.200 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.200 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.200 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.200 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.200 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.200 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.200 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.201 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.201 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.201 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.201 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.201 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.201 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.201 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.201 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.201 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.201 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.201 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.201 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.201 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.201 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.202 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.202 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.202 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.202 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.202 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.202 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.202 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.202 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.202 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.202 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.202 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.202 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.202 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.202 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.203 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.203 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.203 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.203 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.203 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.203 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.203 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_destination": { 2024-08-31T06:23:26.203 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_destination", 2024-08-31T06:23:26.203 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.203 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.203 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.203 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.203 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.203 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.203 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.204 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email address to send alerts to", 2024-08-31T06:23:26.204 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.204 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.204 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.204 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.204 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_from_name": { 2024-08-31T06:23:26.204 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_from_name", 2024-08-31T06:23:26.204 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.204 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.204 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.204 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "Ceph", 2024-08-31T06:23:26.204 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.204 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.204 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.205 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email From: name", 2024-08-31T06:23:26.205 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.205 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.205 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.205 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.205 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_host": { 2024-08-31T06:23:26.205 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_host", 2024-08-31T06:23:26.205 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.205 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.205 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.205 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.205 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.205 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.205 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.205 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP server", 2024-08-31T06:23:26.206 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.206 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.206 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.206 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.206 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_password": { 2024-08-31T06:23:26.206 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_password", 2024-08-31T06:23:26.206 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.206 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.206 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.206 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.206 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.206 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.206 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.207 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Password to authenticate with", 2024-08-31T06:23:26.207 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.207 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.207 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.207 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.207 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_port": { 2024-08-31T06:23:26.207 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_port", 2024-08-31T06:23:26.207 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.207 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.207 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.207 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "465", 2024-08-31T06:23:26.207 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.207 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.207 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.208 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP port", 2024-08-31T06:23:26.208 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.208 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.208 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.208 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.208 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_sender": { 2024-08-31T06:23:26.208 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_sender", 2024-08-31T06:23:26.208 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.208 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.208 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.208 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.208 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.208 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.208 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.209 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP envelope sender", 2024-08-31T06:23:26.209 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.209 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.209 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.209 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.209 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_ssl": { 2024-08-31T06:23:26.209 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_ssl", 2024-08-31T06:23:26.209 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.209 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.209 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.209 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.209 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.209 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.210 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.210 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use SSL to connect to SMTP server", 2024-08-31T06:23:26.210 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.210 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.210 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.210 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.210 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_user": { 2024-08-31T06:23:26.210 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_user", 2024-08-31T06:23:26.210 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.210 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.210 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.210 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.210 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.210 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.211 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.211 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "User to authenticate as", 2024-08-31T06:23:26.211 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.211 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.211 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.211 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.211 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.211 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.211 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.211 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.211 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.211 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.211 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.211 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.212 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.212 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.212 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.212 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.212 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.212 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.212 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.212 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.212 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.212 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "balancer", 2024-08-31T06:23:26.212 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.212 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.212 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.212 INFO:teuthology.orchestra.run.smithi016.stdout: "active": { 2024-08-31T06:23:26.213 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "active", 2024-08-31T06:23:26.213 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.213 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.213 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.213 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.213 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.213 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.213 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.213 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically balance PGs across cluster", 2024-08-31T06:23:26.213 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.213 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.213 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.213 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.213 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_time": { 2024-08-31T06:23:26.214 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_time", 2024-08-31T06:23:26.214 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.214 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.214 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.214 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0000", 2024-08-31T06:23:26.214 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.214 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.214 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.214 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "beginning time of day to automatically balance", 2024-08-31T06:23:26.214 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-31T06:23:26.214 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.214 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.214 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.215 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_weekday": { 2024-08-31T06:23:26.215 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_weekday", 2024-08-31T06:23:26.215 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-08-31T06:23:26.215 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.215 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.215 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.215 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-08-31T06:23:26.215 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-08-31T06:23:26.215 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.215 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-08-31T06:23:26.215 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-31T06:23:26.215 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.215 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.215 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.215 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_max_iterations": { 2024-08-31T06:23:26.216 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_max_iterations", 2024-08-31T06:23:26.216 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-08-31T06:23:26.216 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.216 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.216 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "25", 2024-08-31T06:23:26.216 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-08-31T06:23:26.216 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "250", 2024-08-31T06:23:26.216 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.216 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-08-31T06:23:26.216 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.216 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.216 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.216 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.216 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_metrics": { 2024-08-31T06:23:26.217 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_metrics", 2024-08-31T06:23:26.217 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.217 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.217 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.217 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "pgs,objects,bytes", 2024-08-31T06:23:26.217 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.217 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.217 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.217 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-08-31T06:23:26.217 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-08-31T06:23:26.217 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.217 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.217 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.217 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_step": { 2024-08-31T06:23:26.218 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_step", 2024-08-31T06:23:26.218 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-08-31T06:23:26.218 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.218 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.218 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.5", 2024-08-31T06:23:26.218 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0.001", 2024-08-31T06:23:26.218 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "0.999", 2024-08-31T06:23:26.218 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.218 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "aggressiveness of optimization", 2024-08-31T06:23:26.218 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-08-31T06:23:26.218 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.218 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.218 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.218 INFO:teuthology.orchestra.run.smithi016.stdout: "end_time": { 2024-08-31T06:23:26.218 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_time", 2024-08-31T06:23:26.219 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.219 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.219 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.219 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2359", 2024-08-31T06:23:26.219 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.219 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.219 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.219 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ending time of day to automatically balance", 2024-08-31T06:23:26.219 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-31T06:23:26.219 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.219 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.219 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.219 INFO:teuthology.orchestra.run.smithi016.stdout: "end_weekday": { 2024-08-31T06:23:26.219 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_weekday", 2024-08-31T06:23:26.220 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-08-31T06:23:26.220 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.220 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.220 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.220 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-08-31T06:23:26.220 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-08-31T06:23:26.220 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.220 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-08-31T06:23:26.220 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-31T06:23:26.220 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.220 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.220 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.220 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.220 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.221 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.221 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.221 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.221 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.221 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.221 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.221 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.221 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.221 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.221 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.221 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.221 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.221 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.221 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.221 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.222 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.222 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.222 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.222 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.222 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.222 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.222 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.222 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.222 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.222 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.222 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.222 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.223 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.223 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.223 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.223 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.223 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.223 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.223 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.223 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.223 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.223 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.223 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.223 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.223 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.224 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.224 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.224 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.224 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.224 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.224 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.224 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.224 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.224 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.224 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.224 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.224 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.225 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.225 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.225 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.225 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.225 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.225 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.225 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.225 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.225 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.225 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.225 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.225 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.226 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.226 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.226 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.226 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.226 INFO:teuthology.orchestra.run.smithi016.stdout: "min_score": { 2024-08-31T06:23:26.226 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_score", 2024-08-31T06:23:26.226 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-08-31T06:23:26.226 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.226 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.226 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.226 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.226 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.226 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.227 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-08-31T06:23:26.227 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.227 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.227 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.227 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.227 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-08-31T06:23:26.227 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-08-31T06:23:26.227 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.227 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.227 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.227 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "upmap", 2024-08-31T06:23:26.227 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.227 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.228 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.228 INFO:teuthology.orchestra.run.smithi016.stdout: "crush-compat", 2024-08-31T06:23:26.228 INFO:teuthology.orchestra.run.smithi016.stdout: "none", 2024-08-31T06:23:26.228 INFO:teuthology.orchestra.run.smithi016.stdout: "read", 2024-08-31T06:23:26.228 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap", 2024-08-31T06:23:26.228 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap-read" 2024-08-31T06:23:26.228 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.228 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Balancer mode", 2024-08-31T06:23:26.228 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.228 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.228 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.228 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.228 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_ids": { 2024-08-31T06:23:26.228 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_ids", 2024-08-31T06:23:26.229 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.229 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.229 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.229 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.229 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.229 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.229 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.229 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-08-31T06:23:26.229 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.229 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.229 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.229 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.229 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-08-31T06:23:26.229 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-08-31T06:23:26.230 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:26.230 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.230 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.230 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-08-31T06:23:26.230 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.230 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.230 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.230 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-08-31T06:23:26.230 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.230 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.230 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.230 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.230 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.230 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.231 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.231 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.231 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.231 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.231 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.231 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.231 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.231 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.231 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.231 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.231 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.231 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.231 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_deviation": { 2024-08-31T06:23:26.231 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_deviation", 2024-08-31T06:23:26.232 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.232 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.232 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.232 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-08-31T06:23:26.232 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-08-31T06:23:26.232 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.232 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.232 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "deviation below which no optimization is attempted", 2024-08-31T06:23:26.232 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-08-31T06:23:26.232 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.232 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.232 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.232 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_optimizations": { 2024-08-31T06:23:26.233 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_optimizations", 2024-08-31T06:23:26.233 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-08-31T06:23:26.233 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.233 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.233 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-08-31T06:23:26.233 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.233 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.233 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.233 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-08-31T06:23:26.233 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.233 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.233 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.233 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.233 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.234 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.234 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.234 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm", 2024-08-31T06:23:26.234 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.234 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.234 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.234 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_down_multiplier": { 2024-08-31T06:23:26.234 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_down_multiplier", 2024-08-31T06:23:26.234 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-08-31T06:23:26.234 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.234 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.234 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-08-31T06:23:26.234 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.235 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.235 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.235 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-08-31T06:23:26.235 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.235 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.235 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.235 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.235 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_refresh_rate": { 2024-08-31T06:23:26.235 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_refresh_rate", 2024-08-31T06:23:26.235 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:26.235 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.235 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.235 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "20", 2024-08-31T06:23:26.235 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.236 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.236 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.236 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-08-31T06:23:26.236 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.236 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.236 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.236 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.236 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_starting_port": { 2024-08-31T06:23:26.236 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_starting_port", 2024-08-31T06:23:26.236 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.236 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.236 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.236 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4721", 2024-08-31T06:23:26.236 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.237 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.237 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.237 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-08-31T06:23:26.237 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.237 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.237 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.237 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.237 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_ptrace": { 2024-08-31T06:23:26.237 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_ptrace", 2024-08-31T06:23:26.237 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.237 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.237 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.237 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.237 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.238 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.238 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.238 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-08-31T06:23:26.238 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-08-31T06:23:26.238 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.238 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.238 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.238 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_interval": { 2024-08-31T06:23:26.238 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_interval", 2024-08-31T06:23:26.238 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:26.238 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.238 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.238 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-08-31T06:23:26.239 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.239 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.239 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.239 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to autotune daemon memory", 2024-08-31T06:23:26.239 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.239 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.239 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.239 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.239 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_memory_target_ratio": { 2024-08-31T06:23:26.239 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_memory_target_ratio", 2024-08-31T06:23:26.239 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-08-31T06:23:26.239 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.239 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.239 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.7", 2024-08-31T06:23:26.240 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.240 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.240 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.240 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-08-31T06:23:26.240 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.240 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.240 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.240 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.240 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm_log_destination": { 2024-08-31T06:23:26.240 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm_log_destination", 2024-08-31T06:23:26.240 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.240 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.240 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.241 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.241 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.241 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.241 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.241 INFO:teuthology.orchestra.run.smithi016.stdout: "file", 2024-08-31T06:23:26.241 INFO:teuthology.orchestra.run.smithi016.stdout: "file,syslog", 2024-08-31T06:23:26.241 INFO:teuthology.orchestra.run.smithi016.stdout: "syslog" 2024-08-31T06:23:26.241 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.241 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-08-31T06:23:26.241 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.241 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.241 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.241 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.242 INFO:teuthology.orchestra.run.smithi016.stdout: "cgroups_split": { 2024-08-31T06:23:26.242 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cgroups_split", 2024-08-31T06:23:26.242 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.242 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.242 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.242 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.242 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.242 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.242 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.242 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-08-31T06:23:26.242 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.242 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.242 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.242 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.243 INFO:teuthology.orchestra.run.smithi016.stdout: "config_checks_enabled": { 2024-08-31T06:23:26.243 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_checks_enabled", 2024-08-31T06:23:26.243 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.243 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.243 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.243 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.243 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.243 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.243 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.243 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-08-31T06:23:26.243 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.243 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.243 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.243 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.244 INFO:teuthology.orchestra.run.smithi016.stdout: "config_dashboard": { 2024-08-31T06:23:26.244 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_dashboard", 2024-08-31T06:23:26.244 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.244 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.244 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.244 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.244 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.244 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.244 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.244 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-08-31T06:23:26.244 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.244 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.244 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.244 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.245 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_alertmanager": { 2024-08-31T06:23:26.245 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_alertmanager", 2024-08-31T06:23:26.245 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.245 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.245 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.245 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-08-31T06:23:26.245 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.245 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.245 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.245 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-08-31T06:23:26.245 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.245 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.245 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.245 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.245 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_base": { 2024-08-31T06:23:26.246 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_base", 2024-08-31T06:23:26.246 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.246 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.246 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.246 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/ceph", 2024-08-31T06:23:26.246 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.246 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.246 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.246 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Container image name, without the tag", 2024-08-31T06:23:26.246 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.246 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.246 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.246 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.246 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_elasticsearch": { 2024-08-31T06:23:26.247 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_elasticsearch", 2024-08-31T06:23:26.247 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.247 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.247 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.247 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-08-31T06:23:26.247 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.247 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.247 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.247 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "elasticsearch container image", 2024-08-31T06:23:26.247 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.247 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.247 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.247 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.247 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_grafana": { 2024-08-31T06:23:26.248 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_grafana", 2024-08-31T06:23:26.248 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.248 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.248 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.248 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-08-31T06:23:26.248 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.248 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.248 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.248 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-08-31T06:23:26.248 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.248 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.248 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.248 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.248 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_haproxy": { 2024-08-31T06:23:26.249 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_haproxy", 2024-08-31T06:23:26.249 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.249 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.249 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.249 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-08-31T06:23:26.249 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.249 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.249 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.249 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "HAproxy container image", 2024-08-31T06:23:26.249 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.249 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.249 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.249 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.249 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_agent": { 2024-08-31T06:23:26.250 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_agent", 2024-08-31T06:23:26.250 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.250 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.250 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.250 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-08-31T06:23:26.250 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.250 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.250 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.250 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger agent container image", 2024-08-31T06:23:26.250 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.250 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.250 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.250 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.250 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_collector": { 2024-08-31T06:23:26.251 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_collector", 2024-08-31T06:23:26.251 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.251 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.251 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.251 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-08-31T06:23:26.251 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.251 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.251 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.251 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger collector container image", 2024-08-31T06:23:26.251 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.251 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.251 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.251 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.251 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_query": { 2024-08-31T06:23:26.251 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_query", 2024-08-31T06:23:26.252 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.252 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.252 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.252 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-08-31T06:23:26.252 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.252 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.252 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.252 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger query container image", 2024-08-31T06:23:26.252 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.252 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.252 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.252 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.252 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_keepalived": { 2024-08-31T06:23:26.252 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_keepalived", 2024-08-31T06:23:26.253 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.253 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.253 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.253 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-08-31T06:23:26.253 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.253 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.253 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.253 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Keepalived container image", 2024-08-31T06:23:26.253 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.253 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.253 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.253 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.253 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_loki": { 2024-08-31T06:23:26.254 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_loki", 2024-08-31T06:23:26.254 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.254 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.254 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.254 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-08-31T06:23:26.254 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.254 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.254 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.254 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Loki container image", 2024-08-31T06:23:26.254 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.254 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.254 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.254 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.254 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_nginx": { 2024-08-31T06:23:26.255 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_nginx", 2024-08-31T06:23:26.255 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.255 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.255 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.255 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-08-31T06:23:26.255 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.255 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.255 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.255 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Nginx container image", 2024-08-31T06:23:26.255 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.255 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.255 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.255 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.255 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_node_exporter": { 2024-08-31T06:23:26.256 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_node_exporter", 2024-08-31T06:23:26.256 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.256 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.256 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.256 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-08-31T06:23:26.256 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.256 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.256 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.256 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-08-31T06:23:26.256 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.256 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.256 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.256 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.256 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_nvmeof": { 2024-08-31T06:23:26.257 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_nvmeof", 2024-08-31T06:23:26.257 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.257 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.257 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.257 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-08-31T06:23:26.257 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.257 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.257 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.257 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Nvme-of container image", 2024-08-31T06:23:26.257 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.257 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.257 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.257 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.257 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_oauth2_proxy": { 2024-08-31T06:23:26.258 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_oauth2_proxy", 2024-08-31T06:23:26.258 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.258 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.258 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.258 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-08-31T06:23:26.258 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.258 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.258 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.258 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "oauth2-proxy container image", 2024-08-31T06:23:26.258 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.258 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.258 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.258 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.258 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_prometheus": { 2024-08-31T06:23:26.259 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_prometheus", 2024-08-31T06:23:26.259 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.259 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.259 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.259 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-08-31T06:23:26.259 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.259 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.259 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.259 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-08-31T06:23:26.259 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.259 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.259 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.259 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.260 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_promtail": { 2024-08-31T06:23:26.260 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_promtail", 2024-08-31T06:23:26.260 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.260 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.260 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.260 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-08-31T06:23:26.260 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.260 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.260 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.260 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Promtail container image", 2024-08-31T06:23:26.260 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.260 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.260 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.260 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.261 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_samba": { 2024-08-31T06:23:26.261 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_samba", 2024-08-31T06:23:26.261 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.261 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.261 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.261 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-08-31T06:23:26.261 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.261 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.261 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.261 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Samba/SMB container image", 2024-08-31T06:23:26.261 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.261 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.261 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.261 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.262 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_snmp_gateway": { 2024-08-31T06:23:26.262 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_snmp_gateway", 2024-08-31T06:23:26.262 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.262 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.262 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.262 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-08-31T06:23:26.262 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.262 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.262 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.262 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SNMP Gateway container image", 2024-08-31T06:23:26.262 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.262 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.262 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.262 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.263 INFO:teuthology.orchestra.run.smithi016.stdout: "container_init": { 2024-08-31T06:23:26.263 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_init", 2024-08-31T06:23:26.263 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.263 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.263 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.263 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.263 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.263 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.263 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.263 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Run podman/docker with `--init`", 2024-08-31T06:23:26.263 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.263 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.263 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.264 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.264 INFO:teuthology.orchestra.run.smithi016.stdout: "daemon_cache_timeout": { 2024-08-31T06:23:26.264 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "daemon_cache_timeout", 2024-08-31T06:23:26.264 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:26.264 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.264 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.264 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-08-31T06:23:26.264 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.264 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.264 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.264 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-08-31T06:23:26.264 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.264 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.264 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.265 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.265 INFO:teuthology.orchestra.run.smithi016.stdout: "default_cephadm_command_timeout": { 2024-08-31T06:23:26.265 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_cephadm_command_timeout", 2024-08-31T06:23:26.265 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.265 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.265 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.265 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "900", 2024-08-31T06:23:26.265 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.265 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.265 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.265 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-08-31T06:23:26.265 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.265 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.266 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.266 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.266 INFO:teuthology.orchestra.run.smithi016.stdout: "default_registry": { 2024-08-31T06:23:26.266 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_registry", 2024-08-31T06:23:26.266 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.266 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.266 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.266 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io", 2024-08-31T06:23:26.266 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.266 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.266 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.266 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-08-31T06:23:26.266 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.266 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.267 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.267 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.267 INFO:teuthology.orchestra.run.smithi016.stdout: "device_cache_timeout": { 2024-08-31T06:23:26.267 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_cache_timeout", 2024-08-31T06:23:26.267 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:26.267 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.267 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.267 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1800", 2024-08-31T06:23:26.267 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.267 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.267 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.267 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache device inventory", 2024-08-31T06:23:26.267 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.267 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.268 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.268 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.268 INFO:teuthology.orchestra.run.smithi016.stdout: "device_enhanced_scan": { 2024-08-31T06:23:26.268 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_enhanced_scan", 2024-08-31T06:23:26.268 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.268 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.268 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.268 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.268 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.268 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.269 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.269 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use libstoragemgmt during device scans", 2024-08-31T06:23:26.269 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.269 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.269 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.269 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.269 INFO:teuthology.orchestra.run.smithi016.stdout: "facts_cache_timeout": { 2024-08-31T06:23:26.269 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "facts_cache_timeout", 2024-08-31T06:23:26.269 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:26.269 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.269 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.269 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-08-31T06:23:26.269 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.269 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.269 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.270 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache host facts data", 2024-08-31T06:23:26.270 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.270 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.270 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.270 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.270 INFO:teuthology.orchestra.run.smithi016.stdout: "grafana_dashboards_path": { 2024-08-31T06:23:26.270 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "grafana_dashboards_path", 2024-08-31T06:23:26.270 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.270 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.270 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.270 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-08-31T06:23:26.270 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.270 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.271 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.271 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-08-31T06:23:26.271 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.271 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.271 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.271 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.271 INFO:teuthology.orchestra.run.smithi016.stdout: "host_check_interval": { 2024-08-31T06:23:26.271 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "host_check_interval", 2024-08-31T06:23:26.271 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:26.271 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.271 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.271 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-08-31T06:23:26.271 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.271 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.271 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.272 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to perform a host check", 2024-08-31T06:23:26.272 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.272 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.272 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.272 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.272 INFO:teuthology.orchestra.run.smithi016.stdout: "hw_monitoring": { 2024-08-31T06:23:26.272 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hw_monitoring", 2024-08-31T06:23:26.272 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.272 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.272 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.272 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.272 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.272 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.272 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.273 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-08-31T06:23:26.273 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.273 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.273 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.273 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.273 INFO:teuthology.orchestra.run.smithi016.stdout: "inventory_list_all": { 2024-08-31T06:23:26.273 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "inventory_list_all", 2024-08-31T06:23:26.273 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.273 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.273 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.273 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.273 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.273 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.274 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.274 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-08-31T06:23:26.274 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.274 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.274 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.274 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.274 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.274 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.274 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.274 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.274 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.274 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.274 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.274 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.275 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.275 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.275 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.275 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.275 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.275 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.275 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.275 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.275 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.275 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.275 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.275 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.275 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.275 INFO:teuthology.orchestra.run.smithi016.stdout: "log_refresh_metadata": { 2024-08-31T06:23:26.276 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_refresh_metadata", 2024-08-31T06:23:26.276 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.276 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.276 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.276 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.276 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.276 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.276 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.276 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-08-31T06:23:26.276 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.276 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.276 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.276 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.276 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.277 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.277 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.277 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.277 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.277 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.277 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.277 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.277 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.277 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-08-31T06:23:26.277 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.277 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.277 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.277 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.277 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.277 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.278 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.278 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.278 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.278 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.280 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.280 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.280 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.280 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.280 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.280 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.280 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.280 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.280 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.281 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.281 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.281 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.281 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.281 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.281 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.281 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.281 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.281 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.281 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.281 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.281 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.281 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.281 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.282 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.282 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.282 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.282 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.282 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.282 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.282 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf": { 2024-08-31T06:23:26.282 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-08-31T06:23:26.282 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.282 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.282 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.282 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.282 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.282 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.283 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.283 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-08-31T06:23:26.283 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.283 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.283 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.283 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.283 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-08-31T06:23:26.283 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-08-31T06:23:26.283 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.283 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.283 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.283 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "*", 2024-08-31T06:23:26.283 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.283 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.283 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.284 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-08-31T06:23:26.284 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.284 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.284 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.284 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.284 INFO:teuthology.orchestra.run.smithi016.stdout: "max_count_per_host": { 2024-08-31T06:23:26.284 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_count_per_host", 2024-08-31T06:23:26.284 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.284 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.284 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.284 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-08-31T06:23:26.284 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.284 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.285 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.285 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of daemons per service per host", 2024-08-31T06:23:26.285 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.285 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.285 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.285 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.285 INFO:teuthology.orchestra.run.smithi016.stdout: "max_osd_draining_count": { 2024-08-31T06:23:26.285 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_osd_draining_count", 2024-08-31T06:23:26.285 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.285 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.285 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.285 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-08-31T06:23:26.286 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.286 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.286 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.286 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-08-31T06:23:26.286 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.286 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.286 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.286 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.286 INFO:teuthology.orchestra.run.smithi016.stdout: "migration_current": { 2024-08-31T06:23:26.286 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "migration_current", 2024-08-31T06:23:26.286 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.286 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.286 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.286 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.287 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.287 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.287 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.287 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "internal - do not modify", 2024-08-31T06:23:26.287 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.287 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.287 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.287 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.287 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-08-31T06:23:26.287 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-08-31T06:23:26.287 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.287 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.287 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.287 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "root", 2024-08-31T06:23:26.288 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.288 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.288 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.288 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm-package", 2024-08-31T06:23:26.288 INFO:teuthology.orchestra.run.smithi016.stdout: "root" 2024-08-31T06:23:26.288 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.288 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "mode for remote execution of cephadm", 2024-08-31T06:23:26.288 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.288 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.288 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.288 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.288 INFO:teuthology.orchestra.run.smithi016.stdout: "oob_default_addr": { 2024-08-31T06:23:26.288 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "oob_default_addr", 2024-08-31T06:23:26.288 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.288 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.289 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.289 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "169.254.1.1", 2024-08-31T06:23:26.289 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.289 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.289 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.289 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default address for RedFish API (oob management).", 2024-08-31T06:23:26.289 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.289 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.289 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.289 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.289 INFO:teuthology.orchestra.run.smithi016.stdout: "prometheus_alerts_path": { 2024-08-31T06:23:26.289 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus_alerts_path", 2024-08-31T06:23:26.289 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.290 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.290 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.290 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-08-31T06:23:26.290 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.290 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.290 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.290 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-08-31T06:23:26.290 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.290 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.290 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.290 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.290 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_insecure": { 2024-08-31T06:23:26.290 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_insecure", 2024-08-31T06:23:26.290 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.291 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.291 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.291 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.291 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.291 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.291 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.291 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-08-31T06:23:26.291 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.291 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.291 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.291 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.291 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_password": { 2024-08-31T06:23:26.291 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_password", 2024-08-31T06:23:26.291 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.292 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.292 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.292 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.292 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.292 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.292 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.292 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-08-31T06:23:26.292 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.292 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.292 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.292 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.292 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_url": { 2024-08-31T06:23:26.292 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_url", 2024-08-31T06:23:26.292 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.293 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.293 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.293 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.293 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.293 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.293 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.293 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-08-31T06:23:26.293 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.293 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.293 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.293 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.294 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_username": { 2024-08-31T06:23:26.294 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_username", 2024-08-31T06:23:26.294 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.294 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.294 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.294 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.294 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.294 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.294 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.294 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-08-31T06:23:26.294 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.295 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.295 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.295 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.295 INFO:teuthology.orchestra.run.smithi016.stdout: "secure_monitoring_stack": { 2024-08-31T06:23:26.295 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secure_monitoring_stack", 2024-08-31T06:23:26.295 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.295 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.295 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.295 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.295 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.295 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.295 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.295 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-08-31T06:23:26.296 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.296 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.296 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.296 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.296 INFO:teuthology.orchestra.run.smithi016.stdout: "service_discovery_port": { 2024-08-31T06:23:26.296 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "service_discovery_port", 2024-08-31T06:23:26.296 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.296 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.296 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.296 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8765", 2024-08-31T06:23:26.296 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.296 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.296 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.296 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "cephadm service discovery port", 2024-08-31T06:23:26.297 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.297 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.297 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.297 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.297 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.297 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.297 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.297 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.297 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.297 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.297 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.297 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.297 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.297 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.298 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.298 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.298 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.298 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.298 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_config_file": { 2024-08-31T06:23:26.298 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_config_file", 2024-08-31T06:23:26.298 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.298 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.298 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.298 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.298 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.298 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.298 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.298 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-08-31T06:23:26.299 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.299 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.299 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.299 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.299 INFO:teuthology.orchestra.run.smithi016.stdout: "use_agent": { 2024-08-31T06:23:26.299 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_agent", 2024-08-31T06:23:26.299 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.299 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.299 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.299 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.299 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.299 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.299 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.299 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-08-31T06:23:26.300 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.300 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.300 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.300 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.300 INFO:teuthology.orchestra.run.smithi016.stdout: "use_repo_digest": { 2024-08-31T06:23:26.300 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_repo_digest", 2024-08-31T06:23:26.300 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.300 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.300 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.300 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.300 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.300 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.300 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.301 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-08-31T06:23:26.301 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.301 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.301 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.301 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.301 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_failed_host_check": { 2024-08-31T06:23:26.301 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_failed_host_check", 2024-08-31T06:23:26.301 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.301 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.301 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.301 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.301 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.301 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.301 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.302 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if the host check fails", 2024-08-31T06:23:26.302 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.302 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.302 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.302 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.302 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_daemons": { 2024-08-31T06:23:26.302 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_daemons", 2024-08-31T06:23:26.302 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.302 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.302 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.302 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.302 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.302 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.303 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.303 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-08-31T06:23:26.303 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.303 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.303 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.303 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.303 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_hosts": { 2024-08-31T06:23:26.303 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_hosts", 2024-08-31T06:23:26.303 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.303 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.303 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.303 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.303 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.303 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.304 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.304 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-08-31T06:23:26.304 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.304 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.304 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.304 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.304 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.304 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.304 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.304 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crash", 2024-08-31T06:23:26.304 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.304 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.304 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.304 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.305 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.305 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.305 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.305 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.305 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.305 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.305 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.305 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.305 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.305 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.305 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.305 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.305 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.305 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.306 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.306 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.306 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.306 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.306 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.306 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.306 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.306 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.306 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.306 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.306 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.306 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.306 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.306 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.307 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.307 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.307 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.307 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.307 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.307 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.307 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.307 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.307 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.307 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.307 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.307 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.307 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.307 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.308 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.308 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.308 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.308 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.308 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.308 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.308 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.308 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.308 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.308 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.308 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.308 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.308 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.308 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.309 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.309 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.309 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.309 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.309 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.309 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.309 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.309 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.309 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.309 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.309 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.309 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.309 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.309 INFO:teuthology.orchestra.run.smithi016.stdout: "retain_interval": { 2024-08-31T06:23:26.310 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retain_interval", 2024-08-31T06:23:26.310 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:26.310 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.310 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.310 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "31536000", 2024-08-31T06:23:26.310 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.310 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.310 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.310 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain crashes before pruning them", 2024-08-31T06:23:26.310 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.310 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.310 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.310 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.310 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.310 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.311 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.311 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.311 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.311 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.311 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.311 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.311 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.311 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.311 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.311 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.311 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.311 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.311 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_recent_interval": { 2024-08-31T06:23:26.311 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_recent_interval", 2024-08-31T06:23:26.312 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:26.312 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.312 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.312 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1209600", 2024-08-31T06:23:26.312 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.312 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.312 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.312 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "time interval in which to warn about recent crashes", 2024-08-31T06:23:26.312 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.312 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.312 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.312 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.312 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.312 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.313 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.313 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dashboard", 2024-08-31T06:23:26.313 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.313 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.313 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.313 INFO:teuthology.orchestra.run.smithi016.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-08-31T06:23:26.313 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-08-31T06:23:26.313 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.313 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.313 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.313 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-08-31T06:23:26.313 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.313 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.313 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.314 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.314 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.314 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.314 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.314 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.314 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_HOST": { 2024-08-31T06:23:26.314 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_HOST", 2024-08-31T06:23:26.314 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.314 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.314 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.314 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.314 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.314 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.314 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.315 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.315 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.315 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.315 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.315 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.315 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-08-31T06:23:26.315 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-08-31T06:23:26.315 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.315 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.315 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.315 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.315 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.315 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.315 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.316 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.316 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.316 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.316 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.316 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.316 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_ENABLED": { 2024-08-31T06:23:26.316 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_ENABLED", 2024-08-31T06:23:26.316 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.316 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.316 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.316 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.316 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.316 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.316 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.317 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.317 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.317 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.317 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.317 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.317 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-08-31T06:23:26.317 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-08-31T06:23:26.317 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.317 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.317 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.317 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.317 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.317 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.318 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.318 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.318 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.318 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.318 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.318 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.318 INFO:teuthology.orchestra.run.smithi016.stdout: "ENABLE_BROWSABLE_API": { 2024-08-31T06:23:26.318 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ENABLE_BROWSABLE_API", 2024-08-31T06:23:26.318 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.318 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.318 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.318 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.318 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.318 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.318 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.319 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.319 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.319 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.319 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.319 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.319 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-08-31T06:23:26.319 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-08-31T06:23:26.319 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.319 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.319 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.319 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.319 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.320 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.320 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.320 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.320 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.320 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.320 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.320 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.320 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-08-31T06:23:26.320 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-08-31T06:23:26.320 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.320 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.320 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.320 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.321 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.321 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.321 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.321 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.321 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.321 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.321 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.321 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.321 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-08-31T06:23:26.321 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-08-31T06:23:26.321 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.321 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.321 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.322 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.322 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.322 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.322 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.322 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.322 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.322 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.322 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.322 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.322 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-08-31T06:23:26.322 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-08-31T06:23:26.322 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.322 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.322 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.323 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.323 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.323 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.323 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.323 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.323 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.323 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.323 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.323 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.323 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_NFS": { 2024-08-31T06:23:26.323 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-08-31T06:23:26.323 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.323 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.324 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.324 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.324 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.324 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.324 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.324 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.324 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.324 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.324 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.324 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.324 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RBD": { 2024-08-31T06:23:26.324 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-08-31T06:23:26.324 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.325 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.325 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.325 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.325 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.325 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.325 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.325 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.325 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.325 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.325 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.325 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.325 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RGW": { 2024-08-31T06:23:26.325 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-08-31T06:23:26.325 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.325 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.326 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.326 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.326 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.326 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.326 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.326 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.326 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.326 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.326 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.326 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.326 INFO:teuthology.orchestra.run.smithi016.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-08-31T06:23:26.326 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-08-31T06:23:26.326 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.327 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.327 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.327 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.327 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.327 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.327 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.327 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.327 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.327 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.327 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.327 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.327 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_PASSWORD": { 2024-08-31T06:23:26.327 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_PASSWORD", 2024-08-31T06:23:26.327 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.328 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.328 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.328 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-08-31T06:23:26.328 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.328 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.328 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.328 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.328 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.328 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.328 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.328 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.328 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-08-31T06:23:26.328 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-08-31T06:23:26.329 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.329 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.329 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.329 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.329 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.329 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.329 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.329 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.329 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.329 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.329 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.329 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.329 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_URL": { 2024-08-31T06:23:26.329 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_URL", 2024-08-31T06:23:26.329 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.330 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.330 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.330 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.330 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.330 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.330 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.330 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.330 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.330 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.330 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.330 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.330 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_USERNAME": { 2024-08-31T06:23:26.330 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_USERNAME", 2024-08-31T06:23:26.331 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.331 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.331 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.331 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-08-31T06:23:26.331 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.331 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.331 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.331 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.331 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.331 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.331 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.331 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.331 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-08-31T06:23:26.331 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-08-31T06:23:26.332 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.332 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.332 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.332 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.332 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.332 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.332 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.332 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.332 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.332 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.332 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.332 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.332 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-08-31T06:23:26.332 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-08-31T06:23:26.333 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.333 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.333 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.333 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.333 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.333 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.333 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.333 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.333 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.333 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.333 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.333 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.333 INFO:teuthology.orchestra.run.smithi016.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-08-31T06:23:26.334 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-08-31T06:23:26.334 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.334 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.334 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.334 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.334 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.334 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.334 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.334 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.334 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.334 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.334 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.334 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.334 INFO:teuthology.orchestra.run.smithi016.stdout: "ISSUE_TRACKER_API_KEY": { 2024-08-31T06:23:26.334 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-08-31T06:23:26.335 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.335 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.335 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.335 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.335 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.335 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.335 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.335 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.335 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.335 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.335 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.335 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.335 INFO:teuthology.orchestra.run.smithi016.stdout: "MANAGED_BY_CLUSTERS": { 2024-08-31T06:23:26.335 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-08-31T06:23:26.336 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.336 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.336 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.336 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "[]", 2024-08-31T06:23:26.336 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.336 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.336 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.336 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.336 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.336 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.336 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.336 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.336 INFO:teuthology.orchestra.run.smithi016.stdout: "MULTICLUSTER_CONFIG": { 2024-08-31T06:23:26.337 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "MULTICLUSTER_CONFIG", 2024-08-31T06:23:26.337 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.337 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.337 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.337 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "{}", 2024-08-31T06:23:26.337 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.337 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.337 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.337 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.337 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.337 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.337 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.337 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.337 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_HOST": { 2024-08-31T06:23:26.338 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_HOST", 2024-08-31T06:23:26.338 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.338 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.338 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.338 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.338 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.338 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.338 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.338 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.338 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.338 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.338 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.338 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.338 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-08-31T06:23:26.339 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-08-31T06:23:26.339 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.339 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.339 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.339 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.339 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.339 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.339 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.339 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.339 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.339 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.339 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.339 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.339 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-08-31T06:23:26.339 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-08-31T06:23:26.340 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.340 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.340 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.340 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.340 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.340 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.340 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.340 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.340 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.340 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.340 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.340 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.340 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-08-31T06:23:26.341 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-08-31T06:23:26.341 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.341 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.341 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.341 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.341 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.341 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.341 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.341 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.341 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.341 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.341 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.341 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.342 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-08-31T06:23:26.342 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-08-31T06:23:26.342 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.342 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.342 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.342 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.342 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.342 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.342 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.342 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.342 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.342 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.342 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.342 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.343 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-08-31T06:23:26.343 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-08-31T06:23:26.343 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.343 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.343 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.343 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.343 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.343 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.343 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.343 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.343 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.343 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.343 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.343 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.344 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-08-31T06:23:26.344 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-08-31T06:23:26.344 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.344 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.344 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.344 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.344 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.344 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.344 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.344 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.344 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.344 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.344 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.344 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.344 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-08-31T06:23:26.345 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-08-31T06:23:26.345 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.345 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.345 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.345 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.345 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.345 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.345 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.345 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.345 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.345 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.345 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.345 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.345 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-08-31T06:23:26.346 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-08-31T06:23:26.346 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.346 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.346 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.346 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.346 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.346 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.346 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.346 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.346 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.346 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.346 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.346 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.346 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_ENABLED": { 2024-08-31T06:23:26.347 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_ENABLED", 2024-08-31T06:23:26.347 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.347 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.347 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.347 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.347 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.347 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.347 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.347 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.347 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.347 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.347 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.347 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.347 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-08-31T06:23:26.347 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-08-31T06:23:26.348 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.348 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.348 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.348 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-08-31T06:23:26.348 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.348 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.348 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.348 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.348 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.348 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.348 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.348 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.348 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-08-31T06:23:26.348 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-08-31T06:23:26.349 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.349 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.349 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.349 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-08-31T06:23:26.349 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.349 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.349 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.349 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.349 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.349 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.349 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.349 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.349 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-08-31T06:23:26.350 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-08-31T06:23:26.350 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.350 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.350 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.350 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8", 2024-08-31T06:23:26.350 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.350 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.350 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.350 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.350 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.350 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.350 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.350 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.350 INFO:teuthology.orchestra.run.smithi016.stdout: "REST_REQUESTS_TIMEOUT": { 2024-08-31T06:23:26.351 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-08-31T06:23:26.351 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.351 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.351 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.351 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "45", 2024-08-31T06:23:26.351 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.351 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.351 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.351 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.351 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.351 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.351 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.351 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.351 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ACCESS_KEY": { 2024-08-31T06:23:26.352 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ACCESS_KEY", 2024-08-31T06:23:26.352 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.352 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.352 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.352 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.352 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.352 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.352 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.352 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.352 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.352 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.352 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.352 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.352 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-08-31T06:23:26.353 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-08-31T06:23:26.353 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.353 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.353 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.353 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-08-31T06:23:26.353 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.353 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.353 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.353 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.353 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.353 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.353 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.353 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.353 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SECRET_KEY": { 2024-08-31T06:23:26.354 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SECRET_KEY", 2024-08-31T06:23:26.354 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.354 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.354 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.354 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.354 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.354 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.354 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.354 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.354 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.354 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.354 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.354 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.354 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SSL_VERIFY": { 2024-08-31T06:23:26.355 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SSL_VERIFY", 2024-08-31T06:23:26.355 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.355 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.355 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.355 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.355 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.355 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.355 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.355 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.355 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.355 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.355 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.355 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.355 INFO:teuthology.orchestra.run.smithi016.stdout: "UNSAFE_TLS_v1_2": { 2024-08-31T06:23:26.356 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "UNSAFE_TLS_v1_2", 2024-08-31T06:23:26.356 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.356 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.356 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.356 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.356 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.356 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.356 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.356 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.356 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.356 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.356 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.356 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.356 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-08-31T06:23:26.357 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-08-31T06:23:26.357 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.357 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.357 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.357 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.357 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.357 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.357 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.357 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.357 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.357 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.357 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.357 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.357 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-08-31T06:23:26.358 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-08-31T06:23:26.358 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.358 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.358 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.358 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-08-31T06:23:26.358 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.358 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.358 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.358 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.358 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.358 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.358 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.358 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.358 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-08-31T06:23:26.359 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-08-31T06:23:26.359 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.359 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.359 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.359 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-08-31T06:23:26.359 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.359 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.359 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.359 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.359 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.359 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.359 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.359 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.359 INFO:teuthology.orchestra.run.smithi016.stdout: "cross_origin_url": { 2024-08-31T06:23:26.360 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cross_origin_url", 2024-08-31T06:23:26.360 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.360 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.360 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.360 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.360 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.360 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.360 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.360 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.360 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.360 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.360 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.360 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.360 INFO:teuthology.orchestra.run.smithi016.stdout: "crt_file": { 2024-08-31T06:23:26.361 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crt_file", 2024-08-31T06:23:26.361 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.361 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.361 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.361 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.361 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.361 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.361 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.361 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.361 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.361 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.361 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.361 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.362 INFO:teuthology.orchestra.run.smithi016.stdout: "debug": { 2024-08-31T06:23:26.362 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "debug", 2024-08-31T06:23:26.362 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.362 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.362 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.362 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.362 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.362 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.362 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.362 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable/disable debug options", 2024-08-31T06:23:26.362 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.362 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.362 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.363 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.363 INFO:teuthology.orchestra.run.smithi016.stdout: "jwt_token_ttl": { 2024-08-31T06:23:26.363 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "jwt_token_ttl", 2024-08-31T06:23:26.363 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.363 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.363 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.363 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "28800", 2024-08-31T06:23:26.363 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.363 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.363 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.363 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.363 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.363 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.363 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.364 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.364 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-08-31T06:23:26.364 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-08-31T06:23:26.364 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.364 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.364 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.364 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.364 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.364 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.364 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.364 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.364 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.364 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.364 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.365 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.365 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.365 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.365 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.365 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.365 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.365 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.365 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.365 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.365 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.365 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.365 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.365 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.365 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.366 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.366 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.366 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.366 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.366 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.366 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.366 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.366 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.366 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.366 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.366 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.366 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.366 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.367 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.367 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.367 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.367 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.367 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.367 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.367 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.367 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.367 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.367 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.367 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.367 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.367 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.367 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.367 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.368 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.368 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.368 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.368 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.368 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.368 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.368 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.368 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.368 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.368 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.368 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.368 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.368 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.369 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.369 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.369 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.369 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.369 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.369 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.369 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.369 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.369 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.369 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.369 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.369 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.369 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.369 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.370 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.370 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.370 INFO:teuthology.orchestra.run.smithi016.stdout: "motd": { 2024-08-31T06:23:26.370 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "motd", 2024-08-31T06:23:26.370 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.370 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.370 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.370 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.370 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.370 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.370 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.370 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "The message of the day", 2024-08-31T06:23:26.370 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.370 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.370 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.371 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.371 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect_resolve_ip_addr": { 2024-08-31T06:23:26.371 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "redirect_resolve_ip_addr", 2024-08-31T06:23:26.371 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.371 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.371 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.371 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.371 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.371 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.371 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.371 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.371 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.371 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.371 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.372 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.372 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-08-31T06:23:26.372 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-08-31T06:23:26.372 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.372 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.372 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.372 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-08-31T06:23:26.372 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.372 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.372 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.372 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.372 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.372 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.373 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.373 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.373 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-08-31T06:23:26.373 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-08-31T06:23:26.373 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.373 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.373 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.373 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8080", 2024-08-31T06:23:26.373 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.373 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.373 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.373 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.373 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.373 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.373 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.374 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.374 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.374 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.374 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.374 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.374 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.374 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.374 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.374 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.374 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.374 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.374 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.374 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.374 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.375 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.375 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-08-31T06:23:26.375 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-08-31T06:23:26.375 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.375 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.375 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.375 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.375 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.375 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.375 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.375 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.375 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.375 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.375 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.376 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.376 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl_server_port": { 2024-08-31T06:23:26.376 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl_server_port", 2024-08-31T06:23:26.376 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.376 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.376 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.376 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8443", 2024-08-31T06:23:26.376 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.376 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.376 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.376 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.376 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.376 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.376 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.377 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.377 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-08-31T06:23:26.377 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-08-31T06:23:26.377 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.377 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.377 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.377 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "redirect", 2024-08-31T06:23:26.377 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.377 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.377 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.377 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.377 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect" 2024-08-31T06:23:26.377 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.377 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.378 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.378 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.378 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.378 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.378 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-08-31T06:23:26.378 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-08-31T06:23:26.378 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.378 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.378 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.378 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-08-31T06:23:26.378 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-08-31T06:23:26.378 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-08-31T06:23:26.378 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.378 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.378 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.379 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.379 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.379 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.379 INFO:teuthology.orchestra.run.smithi016.stdout: "url_prefix": { 2024-08-31T06:23:26.379 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url_prefix", 2024-08-31T06:23:26.379 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.379 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.379 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.379 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.379 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.379 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.379 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.379 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.379 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.380 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.380 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.380 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.380 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.380 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.380 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.380 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2024-08-31T06:23:26.380 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.380 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.380 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.380 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_monitoring": { 2024-08-31T06:23:26.380 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_monitoring", 2024-08-31T06:23:26.380 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.380 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.380 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.381 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.381 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.381 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.381 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.381 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "monitor device health metrics", 2024-08-31T06:23:26.381 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.381 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.381 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.381 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.381 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.381 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.381 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.381 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.381 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.382 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.382 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.382 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.382 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.382 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.382 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.382 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.382 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.382 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.382 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.382 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.382 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.383 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.383 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.383 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.383 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.383 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.383 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.383 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.383 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.383 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.383 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.383 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.383 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.383 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.383 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.384 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.384 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.384 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.384 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.384 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.384 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.384 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.384 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.384 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.384 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.384 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.384 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.384 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.384 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.385 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.385 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.385 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.385 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.385 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.385 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.385 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.385 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.385 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.385 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.385 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.385 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.385 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.385 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.386 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.386 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.386 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.386 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.386 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.386 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.386 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.386 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.386 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.386 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.386 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.386 INFO:teuthology.orchestra.run.smithi016.stdout: "mark_out_threshold": { 2024-08-31T06:23:26.386 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mark_out_threshold", 2024-08-31T06:23:26.386 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:26.387 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.387 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.387 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2419200", 2024-08-31T06:23:26.387 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.387 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.387 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.387 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-08-31T06:23:26.387 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.387 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.387 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.387 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.387 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_name": { 2024-08-31T06:23:26.387 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_name", 2024-08-31T06:23:26.387 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.388 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.388 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.388 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "device_health_metrics", 2024-08-31T06:23:26.388 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.388 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.388 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.388 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name of pool in which to store device health metrics", 2024-08-31T06:23:26.388 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.388 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.388 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.388 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.388 INFO:teuthology.orchestra.run.smithi016.stdout: "retention_period": { 2024-08-31T06:23:26.388 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retention_period", 2024-08-31T06:23:26.388 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:26.389 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.389 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.389 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15552000", 2024-08-31T06:23:26.389 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.389 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.389 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.389 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain device health metrics", 2024-08-31T06:23:26.389 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.389 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.389 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.389 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.389 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_frequency": { 2024-08-31T06:23:26.389 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_frequency", 2024-08-31T06:23:26.389 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:26.390 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.390 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.390 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-08-31T06:23:26.390 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.390 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.390 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.390 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to scrape device health metrics", 2024-08-31T06:23:26.390 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.390 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.390 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.390 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.390 INFO:teuthology.orchestra.run.smithi016.stdout: "self_heal": { 2024-08-31T06:23:26.390 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "self_heal", 2024-08-31T06:23:26.390 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.391 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.391 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.391 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.391 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.391 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.391 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.391 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-08-31T06:23:26.391 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.391 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.391 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.391 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.391 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-08-31T06:23:26.391 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-08-31T06:23:26.391 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:26.391 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.392 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.392 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-08-31T06:23:26.392 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.392 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.392 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.392 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and check device health", 2024-08-31T06:23:26.392 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.392 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.392 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.392 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.392 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.392 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.392 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.392 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.393 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.393 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.393 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.393 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.393 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.393 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.393 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.393 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.393 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.393 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.393 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_threshold": { 2024-08-31T06:23:26.393 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_threshold", 2024-08-31T06:23:26.393 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:26.393 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.394 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.394 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7257600", 2024-08-31T06:23:26.394 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.394 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.394 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.394 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-08-31T06:23:26.394 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.394 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.394 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.395 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.395 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.395 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.395 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.395 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "diskprediction_local", 2024-08-31T06:23:26.395 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.395 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.395 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.395 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.395 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.395 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.395 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.395 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.395 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.395 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.396 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.396 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.396 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.396 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.396 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.396 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.396 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.396 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.396 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.396 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.396 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.396 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.396 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.396 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.397 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.397 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.397 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.397 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.397 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.397 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.397 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.397 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.397 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.397 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.397 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.397 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.397 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.397 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.398 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.398 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.398 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.398 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.398 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.398 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.398 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.398 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.398 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.398 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.398 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.398 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.398 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.398 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.399 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.399 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.399 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.399 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.399 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.399 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.399 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.399 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.399 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.399 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.399 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.399 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.399 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.400 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.400 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.400 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.400 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.400 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.400 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.400 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.400 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.400 INFO:teuthology.orchestra.run.smithi016.stdout: "predict_interval": { 2024-08-31T06:23:26.400 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predict_interval", 2024-08-31T06:23:26.400 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.400 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.400 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.400 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-08-31T06:23:26.401 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.401 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.401 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.401 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.401 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.401 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.401 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.401 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.401 INFO:teuthology.orchestra.run.smithi016.stdout: "predictor_model": { 2024-08-31T06:23:26.401 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predictor_model", 2024-08-31T06:23:26.401 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.401 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.401 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.401 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "prophetstor", 2024-08-31T06:23:26.402 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.402 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.402 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.402 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.402 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.402 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.402 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.402 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.402 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-08-31T06:23:26.402 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-08-31T06:23:26.402 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.402 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.402 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.403 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-08-31T06:23:26.403 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.403 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.403 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.403 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.403 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.403 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.403 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.403 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.403 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.403 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.403 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.403 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.404 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.404 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.404 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.404 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.404 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.404 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.404 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.404 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.404 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.404 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.404 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.404 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.404 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.405 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "influx", 2024-08-31T06:23:26.405 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": false, 2024-08-31T06:23:26.405 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "influxdb python module not found", 2024-08-31T06:23:26.405 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.405 INFO:teuthology.orchestra.run.smithi016.stdout: "batch_size": { 2024-08-31T06:23:26.405 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "batch_size", 2024-08-31T06:23:26.405 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.405 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.405 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.405 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5000", 2024-08-31T06:23:26.405 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.405 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.405 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.406 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-08-31T06:23:26.406 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.406 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.406 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.406 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.406 INFO:teuthology.orchestra.run.smithi016.stdout: "database": { 2024-08-31T06:23:26.406 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "database", 2024-08-31T06:23:26.406 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.406 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.406 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.406 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "ceph", 2024-08-31T06:23:26.406 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.406 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.407 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.407 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-08-31T06:23:26.407 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.407 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.407 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.407 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.407 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": { 2024-08-31T06:23:26.407 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hostname", 2024-08-31T06:23:26.407 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.407 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.407 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.407 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.407 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.407 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.408 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.408 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server hostname", 2024-08-31T06:23:26.408 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.408 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.408 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.408 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.408 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-08-31T06:23:26.408 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-08-31T06:23:26.408 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:26.408 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.408 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.408 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "30", 2024-08-31T06:23:26.408 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "5", 2024-08-31T06:23:26.409 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.409 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.409 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-08-31T06:23:26.409 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.409 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.409 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.409 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.409 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.409 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.409 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.409 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.409 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.409 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.410 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.410 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.410 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.410 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.410 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.410 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.410 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.410 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.410 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.410 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.410 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.410 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.410 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.410 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.411 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.411 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.411 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.411 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.411 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.411 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.411 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.411 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.411 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.411 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.411 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.411 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.411 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.411 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.412 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.412 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.412 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.412 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.412 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.412 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.412 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.412 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.412 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.412 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.412 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.412 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.412 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.412 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.413 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.413 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.413 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.413 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.413 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.413 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.413 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.413 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.413 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.413 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.413 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.413 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.413 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.413 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.414 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.414 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.414 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.414 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.414 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.414 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.414 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.414 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.414 INFO:teuthology.orchestra.run.smithi016.stdout: "password": { 2024-08-31T06:23:26.414 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "password", 2024-08-31T06:23:26.414 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.414 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.414 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.414 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.415 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.415 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.415 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.415 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "password of InfluxDB server user", 2024-08-31T06:23:26.415 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.415 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.415 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.415 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.415 INFO:teuthology.orchestra.run.smithi016.stdout: "port": { 2024-08-31T06:23:26.415 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "port", 2024-08-31T06:23:26.415 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.415 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.415 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.415 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8086", 2024-08-31T06:23:26.416 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.416 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.416 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.416 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server port", 2024-08-31T06:23:26.416 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.416 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.416 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.416 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.416 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.416 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.416 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.416 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.417 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.417 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.417 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.417 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.417 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.417 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.417 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.417 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.417 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.417 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.417 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-08-31T06:23:26.417 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-08-31T06:23:26.417 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.417 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.418 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.418 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "false", 2024-08-31T06:23:26.418 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.418 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.418 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.418 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-08-31T06:23:26.418 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.418 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.418 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.418 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.418 INFO:teuthology.orchestra.run.smithi016.stdout: "threads": { 2024-08-31T06:23:26.418 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threads", 2024-08-31T06:23:26.418 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.419 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.419 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.419 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-08-31T06:23:26.419 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-08-31T06:23:26.419 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "32", 2024-08-31T06:23:26.419 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.419 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-08-31T06:23:26.419 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.419 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.419 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.419 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.419 INFO:teuthology.orchestra.run.smithi016.stdout: "username": { 2024-08-31T06:23:26.419 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "username", 2024-08-31T06:23:26.419 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.419 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.420 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.420 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.420 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.420 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.420 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.420 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "username of InfluxDB server user", 2024-08-31T06:23:26.420 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.420 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.420 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.420 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.420 INFO:teuthology.orchestra.run.smithi016.stdout: "verify_ssl": { 2024-08-31T06:23:26.420 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "verify_ssl", 2024-08-31T06:23:26.420 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.421 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.421 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.421 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "true", 2024-08-31T06:23:26.421 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.421 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.421 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.421 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-08-31T06:23:26.421 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.421 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.421 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.421 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.421 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.421 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.422 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.422 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "insights", 2024-08-31T06:23:26.422 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.422 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.422 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.422 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.422 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.422 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.422 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.422 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.422 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.422 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.422 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.422 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.423 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.423 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.423 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.423 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.423 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.423 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.423 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.423 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.423 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.423 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.423 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.423 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.423 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.423 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.424 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.424 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.424 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.424 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.424 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.424 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.424 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.424 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.424 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.424 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.424 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.424 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.424 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.425 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.425 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.425 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.425 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.425 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.425 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.425 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.425 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.425 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.425 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.425 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.425 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.425 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.425 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.426 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.426 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.426 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.426 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.426 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.426 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.426 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.426 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.426 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.426 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.426 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.426 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.426 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.426 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.427 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.427 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.427 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.427 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.427 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.427 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.427 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.427 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.427 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.427 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.427 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.427 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.427 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.427 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.427 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.428 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.428 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.428 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.428 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.428 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.428 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.428 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.428 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.428 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "iostat", 2024-08-31T06:23:26.428 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.428 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.428 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.428 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.428 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.429 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.429 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.429 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.429 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.429 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.429 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.429 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.429 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.429 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.429 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.429 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.429 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.429 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.429 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.430 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.430 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.430 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.430 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.430 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.430 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.430 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.430 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.430 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.430 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.430 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.430 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.430 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.430 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.431 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.431 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.431 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.431 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.431 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.431 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.431 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.431 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.431 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.431 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.431 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.431 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.431 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.431 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.431 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.432 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.432 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.432 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.432 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.432 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.432 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.432 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.432 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.432 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.432 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.432 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.432 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.432 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.432 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.433 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.433 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.433 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.433 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.433 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.433 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.433 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.433 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.433 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.433 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.433 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.433 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.433 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.434 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.434 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.434 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.434 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.434 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.434 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.434 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.434 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.434 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.434 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.434 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.434 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.434 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.434 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.435 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.435 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "k8sevents", 2024-08-31T06:23:26.435 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.435 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.435 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.435 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_event_retention_days": { 2024-08-31T06:23:26.435 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ceph_event_retention_days", 2024-08-31T06:23:26.435 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.435 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.435 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.435 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7", 2024-08-31T06:23:26.435 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.435 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.435 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.436 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Days to hold ceph event information within local cache", 2024-08-31T06:23:26.436 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.436 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.436 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.436 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.436 INFO:teuthology.orchestra.run.smithi016.stdout: "config_check_secs": { 2024-08-31T06:23:26.436 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_check_secs", 2024-08-31T06:23:26.436 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.436 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.436 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.436 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-08-31T06:23:26.436 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "10", 2024-08-31T06:23:26.437 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.437 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.437 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-08-31T06:23:26.437 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.437 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.437 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.437 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.437 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.437 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.437 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.437 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.437 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.437 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.437 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.438 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.438 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.438 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.438 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.438 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.438 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.438 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.438 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.438 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.438 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.438 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.438 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.439 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.439 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.439 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.439 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.439 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.439 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.439 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.439 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.439 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.439 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.439 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.439 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.439 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.440 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.440 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.440 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.440 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.440 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.440 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.440 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.440 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.441 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.441 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.441 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.441 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.441 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.441 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.441 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.441 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.441 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.441 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.441 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.441 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.441 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.441 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.442 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.442 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.442 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.442 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.442 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.442 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.442 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.442 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.442 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.442 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.442 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.442 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.442 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.443 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.443 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.443 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.443 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.443 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.443 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.443 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.443 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.443 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.443 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.443 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.443 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.443 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.443 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.443 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.444 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.444 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.444 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.444 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.444 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.444 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "localpool", 2024-08-31T06:23:26.444 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.444 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.444 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.444 INFO:teuthology.orchestra.run.smithi016.stdout: "failure_domain": { 2024-08-31T06:23:26.444 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "failure_domain", 2024-08-31T06:23:26.444 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.444 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.444 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.445 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "host", 2024-08-31T06:23:26.445 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.445 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.445 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.445 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "failure domain for any created local pool", 2024-08-31T06:23:26.445 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-08-31T06:23:26.445 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.445 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.445 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.445 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.445 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.445 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.445 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.445 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.446 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.446 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.446 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.446 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.446 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.446 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.446 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.446 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.446 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.446 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.446 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.446 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.446 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.446 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.447 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.447 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.447 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.447 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.447 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.447 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.447 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.447 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.447 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.447 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.447 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.447 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.447 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.447 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.448 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.448 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.448 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.448 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.448 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.448 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.448 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.448 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.448 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.448 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.448 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.448 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.448 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.448 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.449 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.449 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.449 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.449 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.449 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.449 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.449 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.449 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.449 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.449 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.449 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.449 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.449 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.449 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.450 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.450 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.450 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.450 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.450 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.450 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.450 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.450 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.450 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.450 INFO:teuthology.orchestra.run.smithi016.stdout: "min_size": { 2024-08-31T06:23:26.450 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_size", 2024-08-31T06:23:26.450 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.450 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.450 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.451 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.451 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.451 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.451 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.451 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default min_size for any created local pool", 2024-08-31T06:23:26.451 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-08-31T06:23:26.451 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.451 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.451 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.451 INFO:teuthology.orchestra.run.smithi016.stdout: "num_rep": { 2024-08-31T06:23:26.451 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "num_rep", 2024-08-31T06:23:26.451 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.451 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.452 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.452 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3", 2024-08-31T06:23:26.452 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.452 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.452 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.452 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default replica count for any created local pool", 2024-08-31T06:23:26.452 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.452 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.452 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.452 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.452 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num": { 2024-08-31T06:23:26.452 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_num", 2024-08-31T06:23:26.452 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.452 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.453 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.453 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "128", 2024-08-31T06:23:26.453 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.453 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.453 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.453 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default pg_num for any created local pool", 2024-08-31T06:23:26.453 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.453 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.453 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.453 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.453 INFO:teuthology.orchestra.run.smithi016.stdout: "prefix": { 2024-08-31T06:23:26.453 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prefix", 2024-08-31T06:23:26.453 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.453 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.454 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.454 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.454 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.454 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.454 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.454 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name prefix for any created local pool", 2024-08-31T06:23:26.454 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.454 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.454 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.454 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.454 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.454 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.454 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.454 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.455 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.455 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.455 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.455 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.455 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.455 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.455 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.455 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.455 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.455 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.455 INFO:teuthology.orchestra.run.smithi016.stdout: "subtree": { 2024-08-31T06:23:26.455 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "subtree", 2024-08-31T06:23:26.455 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.455 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.456 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.456 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "rack", 2024-08-31T06:23:26.456 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.456 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.456 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.456 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "CRUSH level for which to create a local pool", 2024-08-31T06:23:26.456 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-08-31T06:23:26.456 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.456 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.456 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.456 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.456 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.456 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.456 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mds_autoscaler", 2024-08-31T06:23:26.457 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.457 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.457 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.457 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.457 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.457 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.457 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.457 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.457 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.457 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.457 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.457 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.457 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.457 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.458 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.458 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.458 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.458 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.458 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.458 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.458 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.458 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.458 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.458 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.458 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.458 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.458 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.458 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.459 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.459 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.459 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.459 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.459 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.459 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.459 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.459 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.459 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.459 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.459 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.459 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.459 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.459 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.459 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.460 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.460 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.460 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.460 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.460 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.460 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.460 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.460 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.460 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.460 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.460 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.460 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.460 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.461 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.461 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.461 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.461 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.461 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.461 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.461 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.461 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.461 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.461 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.461 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.461 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.461 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.461 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.461 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.462 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.462 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.462 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.462 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.462 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.462 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.462 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.462 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.462 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.462 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.462 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.462 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.462 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.462 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.462 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.463 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.463 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.463 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.463 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.463 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirroring", 2024-08-31T06:23:26.463 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.463 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.463 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.463 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.463 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.463 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.463 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.463 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.463 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.464 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.464 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.464 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.464 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.464 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.464 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.464 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.464 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.464 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.464 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.464 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.464 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.464 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.464 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.465 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.465 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.465 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.465 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.465 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.465 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.465 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.465 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.465 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.465 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.465 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.465 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.465 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.466 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.466 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.466 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.466 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.466 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.466 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.466 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.466 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.466 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.466 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.466 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.466 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.466 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.466 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.466 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.467 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.467 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.467 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.467 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.467 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.467 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.467 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.467 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.467 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.467 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.467 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.467 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.467 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.467 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.468 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.468 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.468 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.468 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.468 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.468 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.468 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.468 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.468 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.468 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.468 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.468 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.468 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.468 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.469 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.469 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.469 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.469 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.469 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.469 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.469 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.469 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.469 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.469 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.469 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.469 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "nfs", 2024-08-31T06:23:26.469 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.469 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.470 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.470 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.470 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.470 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.470 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.470 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.470 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.470 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.470 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.470 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.470 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.470 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.470 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.471 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.471 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.471 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.471 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.471 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.471 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.471 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.471 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.471 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.471 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.471 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.471 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.471 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.471 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.472 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.472 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.472 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.472 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.472 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.472 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.472 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.472 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.472 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.472 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.472 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.472 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.472 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.472 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.473 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.473 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.473 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.473 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.473 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.473 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.473 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.473 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.473 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.473 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.473 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.473 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.473 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.474 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.474 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.474 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.474 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.474 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.474 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.474 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.474 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.474 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.474 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.474 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.474 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.474 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.475 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.475 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.475 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.475 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.475 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.475 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.475 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.475 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.475 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.475 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.475 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.475 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.475 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.476 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.476 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.476 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.476 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.476 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.476 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.476 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.476 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.476 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-08-31T06:23:26.476 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.476 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.476 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.476 INFO:teuthology.orchestra.run.smithi016.stdout: "fail_fs": { 2024-08-31T06:23:26.476 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "fail_fs", 2024-08-31T06:23:26.477 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.477 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.477 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.477 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.477 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.477 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.477 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.477 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-08-31T06:23:26.477 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.477 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.477 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.477 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.478 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.478 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.478 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.478 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.478 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.478 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.478 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.478 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.478 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.478 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.478 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.478 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.478 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.478 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.479 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.479 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.479 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.479 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.479 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.479 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.479 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.479 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.479 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.479 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.479 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.479 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.479 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.479 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.480 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.480 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.480 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.480 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.480 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.480 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.480 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.480 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.480 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.480 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.480 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.480 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.480 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.480 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.481 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.481 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.481 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.481 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.481 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.481 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.481 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.481 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.481 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.481 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.481 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.481 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.481 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.481 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.482 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.482 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.482 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.482 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.482 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.482 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.482 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.482 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.482 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.482 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.482 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.482 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.482 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.482 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.482 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator": { 2024-08-31T06:23:26.483 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-08-31T06:23:26.483 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.483 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.483 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.483 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.483 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.483 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.483 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.483 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2024-08-31T06:23:26.483 INFO:teuthology.orchestra.run.smithi016.stdout: "rook", 2024-08-31T06:23:26.483 INFO:teuthology.orchestra.run.smithi016.stdout: "test_orchestrator" 2024-08-31T06:23:26.483 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.483 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Orchestrator backend", 2024-08-31T06:23:26.483 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.483 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.484 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.484 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.484 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.484 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.484 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.484 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.484 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.484 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.484 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.484 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.484 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.484 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.484 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.484 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.485 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.485 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.485 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.485 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.485 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.485 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_perf_query", 2024-08-31T06:23:26.485 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.485 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.485 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.485 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.485 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.485 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.485 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.486 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.486 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.486 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.486 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.486 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.486 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.486 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.486 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.486 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.486 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.486 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.486 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.486 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.486 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.487 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.487 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.487 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.487 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.487 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.487 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.487 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.487 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.487 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.487 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.487 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.487 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.487 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.487 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.488 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.488 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.488 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.488 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.488 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.488 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.488 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.488 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.488 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.488 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.488 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.488 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.488 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.488 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.489 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.489 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.489 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.489 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.489 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.489 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.489 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.489 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.489 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.489 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.489 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.489 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.489 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.489 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.490 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.490 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.490 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.490 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.490 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.490 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.490 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.490 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.490 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.490 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.490 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.490 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.490 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.490 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.491 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.491 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.491 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.491 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.491 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.491 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.491 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.491 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.491 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.491 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.491 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.491 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.491 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.491 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_support", 2024-08-31T06:23:26.492 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.492 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.492 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.492 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.492 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.492 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.492 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.492 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.492 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.492 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.492 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.492 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.492 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.493 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.493 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.493 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.493 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.493 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.493 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.493 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.493 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.493 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.493 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.493 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.493 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.493 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.493 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.494 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.494 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.494 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.494 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.494 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.494 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.494 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.494 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.494 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.494 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.495 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.495 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.495 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.495 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.495 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.495 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.495 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.495 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.495 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.495 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.495 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.495 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.495 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.495 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.496 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.496 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.496 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.496 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.496 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.496 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.496 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.496 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.496 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.496 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.496 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.496 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.496 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.496 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.497 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.497 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.497 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.497 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.497 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.497 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.497 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.497 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.497 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.497 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.497 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.497 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.497 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.497 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.498 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.498 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.498 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.498 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.498 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.498 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.498 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.498 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.498 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.498 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.498 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.498 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_autoscaler", 2024-08-31T06:23:26.498 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.498 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.499 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.499 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.499 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.499 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.499 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.499 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.499 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.499 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.499 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.499 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.499 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.499 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.499 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.499 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.500 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.500 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.500 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.500 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.500 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.500 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.500 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.500 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.500 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.500 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.500 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.500 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.500 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.500 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.500 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.501 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.501 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.501 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.501 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.501 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.501 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.501 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.501 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.501 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.501 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.501 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.501 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.501 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.501 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.502 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.502 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.502 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.502 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.502 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.502 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.502 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.502 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.502 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.502 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.502 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.502 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.502 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.502 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.502 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.503 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.503 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.503 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.503 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.503 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.503 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.503 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.503 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.503 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.503 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.503 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.503 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.503 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.503 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-08-31T06:23:26.504 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-08-31T06:23:26.504 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:26.504 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.504 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.504 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-08-31T06:23:26.504 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.504 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.504 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.504 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.504 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.504 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.504 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.504 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.504 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.505 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.505 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.505 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.505 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.505 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.505 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.505 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.505 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.505 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.505 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.505 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.505 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.505 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.505 INFO:teuthology.orchestra.run.smithi016.stdout: "threshold": { 2024-08-31T06:23:26.506 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threshold", 2024-08-31T06:23:26.506 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-08-31T06:23:26.506 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.506 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.506 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-08-31T06:23:26.506 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1.0", 2024-08-31T06:23:26.506 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.506 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.506 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "scaling threshold", 2024-08-31T06:23:26.506 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-08-31T06:23:26.506 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.506 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.506 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.506 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.507 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.507 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.507 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "progress", 2024-08-31T06:23:26.507 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.507 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.507 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.507 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_pg_recovery_event": { 2024-08-31T06:23:26.507 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_pg_recovery_event", 2024-08-31T06:23:26.507 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.507 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.507 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.507 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.507 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.507 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.507 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.508 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow the module to show pg recovery progress", 2024-08-31T06:23:26.508 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.508 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.508 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.508 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.508 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-08-31T06:23:26.508 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-08-31T06:23:26.508 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.508 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.508 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.508 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.508 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.508 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.509 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.509 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.509 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.509 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.509 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.509 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.509 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.509 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.509 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.509 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.509 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.509 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.509 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.509 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.510 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.510 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.510 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.510 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.510 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.510 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.510 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.510 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.510 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.510 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.510 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.510 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.510 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.510 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.511 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.511 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.511 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.511 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.511 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.511 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.511 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.511 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.511 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.511 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.511 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.511 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.511 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.511 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.512 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.512 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.512 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.512 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.512 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.512 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.512 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.512 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.512 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.512 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.512 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.512 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.512 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.512 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.513 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.513 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.513 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.513 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.513 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.513 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.513 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.513 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.513 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.513 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.513 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.513 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.513 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.513 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.513 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.514 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.514 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.514 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.514 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.514 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.514 INFO:teuthology.orchestra.run.smithi016.stdout: "max_completed_events": { 2024-08-31T06:23:26.514 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_completed_events", 2024-08-31T06:23:26.514 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.514 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.514 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.514 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "50", 2024-08-31T06:23:26.514 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.514 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.514 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.515 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "number of past completed events to remember", 2024-08-31T06:23:26.515 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.515 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.515 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.515 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.515 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-08-31T06:23:26.515 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-08-31T06:23:26.515 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:26.515 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.515 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.515 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-08-31T06:23:26.515 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.515 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.515 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.515 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long the module is going to sleep", 2024-08-31T06:23:26.516 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.516 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.516 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.516 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.516 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.516 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.516 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.516 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.516 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.516 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.516 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.516 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.516 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.516 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.517 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.517 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.517 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.517 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.517 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.517 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.517 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.517 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus", 2024-08-31T06:23:26.517 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.517 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.517 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.517 INFO:teuthology.orchestra.run.smithi016.stdout: "cache": { 2024-08-31T06:23:26.517 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cache", 2024-08-31T06:23:26.517 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.518 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.518 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.518 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.518 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.518 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.518 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.518 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.518 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.518 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.518 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.518 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.518 INFO:teuthology.orchestra.run.smithi016.stdout: "exclude_perf_counters": { 2024-08-31T06:23:26.518 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "exclude_perf_counters", 2024-08-31T06:23:26.518 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.518 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.519 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.519 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.519 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.519 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.519 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.519 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-08-31T06:23:26.519 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-08-31T06:23:26.519 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.519 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.519 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.519 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.519 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.519 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.519 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.520 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.520 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.520 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.520 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.520 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.520 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.520 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.520 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.520 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.520 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.520 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.520 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.520 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.520 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.521 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.521 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.521 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.521 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.521 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.521 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.521 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.521 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.521 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.521 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.521 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.521 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.521 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.521 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.522 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.522 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.522 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.522 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.522 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.522 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.522 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.522 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.522 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.522 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.522 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.522 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.522 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.522 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.523 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.523 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.523 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.523 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.523 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.523 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.523 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.523 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.523 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.523 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.523 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.523 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.523 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.523 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.524 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.524 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.524 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.524 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.524 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.524 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.524 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.524 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.524 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.524 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.524 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools": { 2024-08-31T06:23:26.524 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools", 2024-08-31T06:23:26.524 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.524 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.525 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.525 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.525 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.525 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.525 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.525 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.525 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.525 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.525 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.525 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.525 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools_refresh_interval": { 2024-08-31T06:23:26.525 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-08-31T06:23:26.525 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.526 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.526 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.526 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "300", 2024-08-31T06:23:26.526 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.526 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.526 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.526 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.526 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.526 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.526 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.526 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.526 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_interval": { 2024-08-31T06:23:26.526 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_interval", 2024-08-31T06:23:26.526 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-08-31T06:23:26.527 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.527 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.527 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15.0", 2024-08-31T06:23:26.527 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.527 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.527 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.527 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.527 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.527 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.527 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.527 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.527 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-08-31T06:23:26.527 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-08-31T06:23:26.527 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.527 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.528 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.528 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-08-31T06:23:26.528 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.528 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.528 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.528 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-08-31T06:23:26.528 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.528 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.528 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.528 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.528 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-08-31T06:23:26.528 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-08-31T06:23:26.528 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.528 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.529 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.529 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "9283", 2024-08-31T06:23:26.529 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.529 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.529 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.529 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-08-31T06:23:26.529 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.529 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.529 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.529 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.529 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.529 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.529 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.529 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.530 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.530 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.530 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.530 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.530 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.530 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.530 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.530 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.530 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.530 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.530 INFO:teuthology.orchestra.run.smithi016.stdout: "stale_cache_strategy": { 2024-08-31T06:23:26.530 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stale_cache_strategy", 2024-08-31T06:23:26.530 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.530 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.530 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.531 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "log", 2024-08-31T06:23:26.531 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.531 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.531 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.531 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.531 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.531 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.531 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.531 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.531 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-08-31T06:23:26.531 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-08-31T06:23:26.531 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.531 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.531 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.531 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "default", 2024-08-31T06:23:26.532 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.532 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.532 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.532 INFO:teuthology.orchestra.run.smithi016.stdout: "default", 2024-08-31T06:23:26.532 INFO:teuthology.orchestra.run.smithi016.stdout: "error" 2024-08-31T06:23:26.532 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.532 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.532 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.532 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.532 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.532 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.532 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-08-31T06:23:26.532 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-08-31T06:23:26.532 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.533 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.533 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.533 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-08-31T06:23:26.533 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-08-31T06:23:26.533 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-08-31T06:23:26.533 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.533 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.533 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.533 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.533 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.533 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.533 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.533 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.533 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.534 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2024-08-31T06:23:26.534 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.534 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.534 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.534 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.534 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.534 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.534 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.534 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.534 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.534 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.534 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.534 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.534 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.535 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.535 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.535 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.535 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.535 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.535 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.535 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.535 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.535 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.535 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.535 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.535 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.535 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.535 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.536 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.536 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.536 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.536 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.536 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.536 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.536 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.536 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.536 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.536 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.536 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.536 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.536 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.537 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.537 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.537 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.537 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.537 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.537 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.537 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.537 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.537 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.537 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.537 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.537 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.537 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.537 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.537 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.538 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.538 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.538 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.538 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.538 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.538 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.538 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.538 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.538 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.538 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.538 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.538 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.538 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.539 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.539 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.539 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.539 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.539 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.539 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_snap_create": { 2024-08-31T06:23:26.539 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_snap_create", 2024-08-31T06:23:26.539 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.539 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.539 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.539 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-08-31T06:23:26.539 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.539 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.539 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.540 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.540 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.540 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.540 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.540 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.540 INFO:teuthology.orchestra.run.smithi016.stdout: "mirror_snapshot_schedule": { 2024-08-31T06:23:26.540 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirror_snapshot_schedule", 2024-08-31T06:23:26.540 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.540 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.540 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.540 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.540 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.540 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.540 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.541 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.541 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.541 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.541 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.541 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.541 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.541 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.541 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.541 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.541 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.541 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.541 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.541 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.541 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.542 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.542 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.542 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.542 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.542 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.542 INFO:teuthology.orchestra.run.smithi016.stdout: "trash_purge_schedule": { 2024-08-31T06:23:26.542 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "trash_purge_schedule", 2024-08-31T06:23:26.542 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.542 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.542 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.542 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.542 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.542 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.542 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.543 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.543 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.543 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.543 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.543 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.543 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.543 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.543 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.543 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "restful", 2024-08-31T06:23:26.543 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.543 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.543 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.543 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_auth": { 2024-08-31T06:23:26.544 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_auth", 2024-08-31T06:23:26.544 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.544 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.544 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.544 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.544 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.544 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.544 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.544 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.544 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.544 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.544 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.544 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.544 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-08-31T06:23:26.545 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-08-31T06:23:26.545 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.545 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.545 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.545 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.545 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.545 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.545 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.545 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.545 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.545 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.545 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.545 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.545 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.546 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.546 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.546 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.546 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.546 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.546 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.546 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.546 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.546 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.546 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.546 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.546 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.546 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.547 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.547 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.547 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.547 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.547 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.547 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.547 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.547 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.547 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.547 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.547 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.547 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.547 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.547 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.548 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.548 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.548 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.548 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.548 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.548 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.548 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.548 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.548 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.548 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.548 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.548 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.548 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.549 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.549 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.549 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.549 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.549 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.549 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.549 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.549 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.549 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.549 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.549 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.549 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.549 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.550 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.550 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.550 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.550 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.550 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.550 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.550 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.550 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.550 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.550 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.550 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.550 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.550 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.550 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.550 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.551 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.551 INFO:teuthology.orchestra.run.smithi016.stdout: "max_requests": { 2024-08-31T06:23:26.551 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_requests", 2024-08-31T06:23:26.551 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.551 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.551 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.551 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-08-31T06:23:26.551 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.551 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.551 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.551 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-08-31T06:23:26.551 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.551 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.552 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.552 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.552 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-08-31T06:23:26.552 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-08-31T06:23:26.552 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.552 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.552 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.552 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.552 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.552 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.552 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.552 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.552 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.553 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.553 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.553 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.553 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-08-31T06:23:26.553 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-08-31T06:23:26.553 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.553 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.553 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.553 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.553 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.553 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.553 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.553 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.554 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.554 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.554 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.554 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.554 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.554 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.554 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.554 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.554 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.554 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.554 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.554 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.554 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.554 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.555 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.555 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.555 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.555 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.555 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.555 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.555 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.555 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rgw", 2024-08-31T06:23:26.555 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.555 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.555 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.555 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.555 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.555 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.556 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.556 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.556 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.556 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.556 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.556 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.556 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.556 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.556 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.556 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.556 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.556 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.556 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.557 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.557 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.557 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.557 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.557 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.557 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.557 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.557 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.557 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.557 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.557 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.557 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.557 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.558 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.558 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.558 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.558 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.558 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.558 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.558 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.558 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.558 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.558 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.558 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.558 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.558 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.558 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.559 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.559 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.559 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.559 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.559 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.559 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.559 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.559 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.559 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.559 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.559 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.559 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.559 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.560 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.560 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.560 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.560 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.560 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.560 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.560 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.560 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.560 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.560 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.560 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.560 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.560 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.560 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.561 INFO:teuthology.orchestra.run.smithi016.stdout: "secondary_zone_period_retry_limit": { 2024-08-31T06:23:26.561 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secondary_zone_period_retry_limit", 2024-08-31T06:23:26.561 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.561 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.561 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.561 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-08-31T06:23:26.561 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.561 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.561 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.561 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-08-31T06:23:26.561 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.561 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.561 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.561 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.561 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.562 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.562 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.562 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.562 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.562 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.562 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.562 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.562 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.562 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.562 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.562 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.562 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.562 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.562 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.562 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.563 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.563 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rook", 2024-08-31T06:23:26.563 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.563 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.563 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.563 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.563 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.563 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.563 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.563 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.563 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.563 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.563 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.564 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.564 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.564 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.564 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.564 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.564 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.564 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.564 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.564 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.564 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.564 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.564 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.564 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.564 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.564 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.565 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.565 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.565 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.565 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.565 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.565 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.565 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.565 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.565 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.565 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.565 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.565 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.565 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.565 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.566 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.566 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.566 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.566 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.566 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.566 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.566 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.566 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.566 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.566 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.566 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.566 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.566 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.566 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.567 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.567 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.567 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.567 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.567 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.567 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.567 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.567 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.567 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.567 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.567 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.567 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.567 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.567 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.567 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.567 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.568 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.568 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.568 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.568 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.568 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.568 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.568 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.568 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.568 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.568 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.568 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.568 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.568 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.568 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.569 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.569 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.569 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.569 INFO:teuthology.orchestra.run.smithi016.stdout: "storage_class": { 2024-08-31T06:23:26.569 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "storage_class", 2024-08-31T06:23:26.569 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.569 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.569 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.569 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "local", 2024-08-31T06:23:26.569 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.569 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.569 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.569 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-08-31T06:23:26.570 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.570 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.570 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.570 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.570 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.570 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.570 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.570 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "selftest", 2024-08-31T06:23:26.570 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.570 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.570 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.570 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.570 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.571 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.571 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.571 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.571 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.571 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.571 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.571 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.571 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.571 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.571 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.571 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.571 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.571 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.571 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.572 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.572 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.572 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.572 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.572 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.572 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.572 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.572 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.572 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.572 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.572 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.572 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.572 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.573 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.573 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.573 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.573 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.573 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.573 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.573 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.573 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.573 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.573 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.573 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.573 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.573 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.574 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.574 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.574 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.574 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.574 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.574 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.574 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.574 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.574 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.574 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.574 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.574 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.574 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.574 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.575 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.575 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.575 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.575 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.575 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.575 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.575 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.575 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.575 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.575 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.575 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.575 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.575 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.576 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.576 INFO:teuthology.orchestra.run.smithi016.stdout: "roption1": { 2024-08-31T06:23:26.576 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption1", 2024-08-31T06:23:26.576 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.576 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.576 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.576 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.576 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.576 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.576 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.576 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.576 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.576 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.577 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.577 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.577 INFO:teuthology.orchestra.run.smithi016.stdout: "roption2": { 2024-08-31T06:23:26.577 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption2", 2024-08-31T06:23:26.577 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.577 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.577 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.577 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "xyz", 2024-08-31T06:23:26.577 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.577 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.577 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.577 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.577 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.577 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.578 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.578 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.578 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption1": { 2024-08-31T06:23:26.578 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption1", 2024-08-31T06:23:26.578 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.578 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.578 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.578 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.578 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.578 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.578 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.578 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.578 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.578 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.579 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.579 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.579 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption2": { 2024-08-31T06:23:26.579 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption2", 2024-08-31T06:23:26.579 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.579 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.579 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.579 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.579 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.579 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.579 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.579 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.579 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.579 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.579 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.580 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.580 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption3": { 2024-08-31T06:23:26.580 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption3", 2024-08-31T06:23:26.580 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-08-31T06:23:26.580 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.580 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.580 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.580 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.580 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.580 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.580 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.580 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.580 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.581 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.581 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.581 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption4": { 2024-08-31T06:23:26.581 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption4", 2024-08-31T06:23:26.581 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.581 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.581 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.581 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.581 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.581 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.581 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.581 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.581 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.581 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.581 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.582 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.582 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption5": { 2024-08-31T06:23:26.582 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption5", 2024-08-31T06:23:26.582 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.582 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.582 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.582 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.582 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.582 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.582 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.582 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.582 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.582 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.582 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.583 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.583 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption6": { 2024-08-31T06:23:26.583 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption6", 2024-08-31T06:23:26.583 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.583 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.583 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.583 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.583 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.583 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.583 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.583 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.583 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.583 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.583 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.583 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.584 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption7": { 2024-08-31T06:23:26.584 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption7", 2024-08-31T06:23:26.584 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.584 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.584 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.584 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.584 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-08-31T06:23:26.584 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "42", 2024-08-31T06:23:26.584 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.584 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.584 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.584 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.584 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.584 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.585 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.585 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.585 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.585 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.585 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.585 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.585 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.585 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.585 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.585 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.585 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.585 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.585 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.585 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.586 INFO:teuthology.orchestra.run.smithi016.stdout: "testkey": { 2024-08-31T06:23:26.586 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testkey", 2024-08-31T06:23:26.586 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.586 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.586 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.586 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.586 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.586 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.586 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.586 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.586 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.586 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.586 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.587 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.587 INFO:teuthology.orchestra.run.smithi016.stdout: "testlkey": { 2024-08-31T06:23:26.587 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testlkey", 2024-08-31T06:23:26.587 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.587 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.587 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.587 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.587 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.587 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.587 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.587 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.587 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.587 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.587 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.588 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.588 INFO:teuthology.orchestra.run.smithi016.stdout: "testnewline": { 2024-08-31T06:23:26.588 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testnewline", 2024-08-31T06:23:26.588 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.588 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.588 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.588 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.588 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.588 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.588 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.588 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.588 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.588 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.588 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.589 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.589 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.589 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.589 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.589 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smb", 2024-08-31T06:23:26.589 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.589 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.589 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.589 INFO:teuthology.orchestra.run.smithi016.stdout: "internal_store_backend": { 2024-08-31T06:23:26.589 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "internal_store_backend", 2024-08-31T06:23:26.589 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.589 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.589 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.590 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.590 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.590 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.590 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.590 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-08-31T06:23:26.590 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.590 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.590 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.590 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.590 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.590 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.590 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.590 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.590 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.590 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.591 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.591 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.591 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.591 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.591 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.591 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.591 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.591 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.591 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.591 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.591 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.591 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.591 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.591 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.592 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.592 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.592 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.592 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.592 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.592 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.592 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.592 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.592 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.592 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.592 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.592 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.592 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.592 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.593 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.593 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.593 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.593 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.593 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.593 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.593 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.593 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.593 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.593 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.593 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.593 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.593 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.593 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.594 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.594 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.594 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.594 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.594 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.594 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.594 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.594 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.594 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.594 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.594 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.594 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.595 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.595 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.595 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.595 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.595 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.595 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.595 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.595 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.595 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.595 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.595 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.595 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.596 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.596 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.596 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.596 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.596 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.596 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.596 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.596 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.596 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.596 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.596 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.596 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.596 INFO:teuthology.orchestra.run.smithi016.stdout: "update_orchestration": { 2024-08-31T06:23:26.597 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "update_orchestration", 2024-08-31T06:23:26.597 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.597 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.597 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.597 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.597 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.597 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.597 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.597 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-08-31T06:23:26.597 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.597 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.597 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.597 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.597 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.598 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.598 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.598 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-08-31T06:23:26.598 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.598 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.598 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.598 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_m_granularity": { 2024-08-31T06:23:26.598 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_m_granularity", 2024-08-31T06:23:26.598 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.598 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.598 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.598 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.598 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.598 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.599 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.599 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow minute scheduled snapshots", 2024-08-31T06:23:26.599 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.599 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.599 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.599 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.599 INFO:teuthology.orchestra.run.smithi016.stdout: "dump_on_update": { 2024-08-31T06:23:26.599 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dump_on_update", 2024-08-31T06:23:26.599 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.599 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.599 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.599 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.599 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.599 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.599 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.600 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "dump database to debug log on update", 2024-08-31T06:23:26.600 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.600 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.600 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.600 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.600 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.600 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.600 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.600 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.600 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.600 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.600 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.600 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.600 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.601 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.601 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.601 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.601 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.601 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.601 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.601 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.601 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.601 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.601 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.601 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.601 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.601 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.601 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.602 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.602 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.602 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.602 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.602 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.602 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.602 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.602 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.602 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.602 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.602 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.602 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.602 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.602 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.603 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.603 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.603 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.603 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.603 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.603 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.603 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.603 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.603 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.603 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.603 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.603 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.603 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.604 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.604 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.604 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.604 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.604 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.604 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.604 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.604 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.604 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.604 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.604 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.604 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.604 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.605 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.605 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.605 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.605 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.605 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.605 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.605 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.605 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.605 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.605 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.605 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.605 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.605 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.606 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.606 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.606 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.606 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.606 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.606 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.606 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.606 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.606 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.606 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.606 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.606 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stats", 2024-08-31T06:23:26.606 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.607 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.607 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.607 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.607 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.607 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.607 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.607 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.607 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.607 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.607 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.607 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.607 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.608 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.608 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.608 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.608 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.608 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.608 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.608 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.608 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.608 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.608 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.608 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.608 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.608 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.608 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.609 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.609 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.609 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.609 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.609 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.609 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.609 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.609 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.609 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.609 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.609 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.609 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.609 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.609 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.610 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.610 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.610 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.610 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.610 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.610 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.610 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.610 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.610 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.610 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.610 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.610 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.610 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.610 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.611 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.611 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.611 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.611 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.611 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.611 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.611 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.611 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.611 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.611 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.611 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.611 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.611 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.611 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.612 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.612 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.612 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.612 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.612 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.612 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.612 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.612 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.612 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.612 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.612 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.612 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.612 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.612 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.613 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.613 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.613 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.613 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.613 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.613 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.613 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.613 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "status", 2024-08-31T06:23:26.613 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.613 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.613 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.613 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.613 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.614 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.614 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.614 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.614 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.614 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.614 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.614 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.614 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.614 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.614 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.614 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.614 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.614 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.614 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.615 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.615 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.615 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.615 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.615 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.615 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.615 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.615 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.615 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.615 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.615 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.615 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.615 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.615 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.616 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.616 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.616 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.616 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.616 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.616 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.616 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.616 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.616 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.616 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.616 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.616 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.616 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.616 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.617 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.617 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.617 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.617 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.617 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.617 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.617 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.617 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.617 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.617 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.617 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.617 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.617 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.618 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.618 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.618 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.618 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.618 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.618 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.618 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.618 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.618 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.618 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.618 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.618 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.618 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.619 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.619 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.619 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.619 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.619 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.619 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.619 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.619 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.619 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.619 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.619 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.619 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.619 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.620 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.620 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.620 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.620 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.620 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telegraf", 2024-08-31T06:23:26.620 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.620 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.620 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.620 INFO:teuthology.orchestra.run.smithi016.stdout: "address": { 2024-08-31T06:23:26.620 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "address", 2024-08-31T06:23:26.620 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.620 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.620 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.620 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-08-31T06:23:26.621 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.621 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.621 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.621 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.621 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.621 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.621 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.621 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.621 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-08-31T06:23:26.621 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-08-31T06:23:26.621 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:26.621 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.621 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.622 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15", 2024-08-31T06:23:26.622 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.622 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.622 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.622 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.622 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.622 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.622 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.622 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.622 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.622 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.622 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.622 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.623 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.623 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.623 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.623 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.623 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.623 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.623 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.623 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.623 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.623 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.623 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.623 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.623 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.623 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.624 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.624 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.624 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.624 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.624 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.624 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.624 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.624 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.624 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.624 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.624 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.624 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.624 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.625 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.625 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.625 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.625 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.625 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.625 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.625 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.625 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.625 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.625 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.625 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.625 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.625 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.626 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.626 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.626 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.626 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.626 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.626 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.626 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.626 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.626 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.626 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.626 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.626 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.626 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.626 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.627 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.627 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.627 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.627 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.627 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.627 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.627 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.627 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.627 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.627 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.627 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.627 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.627 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.627 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.627 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.628 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.628 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.628 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.628 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.628 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.628 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.628 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.628 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.628 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.628 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.628 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.628 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.628 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.629 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.629 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telemetry", 2024-08-31T06:23:26.629 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.629 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.629 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.629 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_basic": { 2024-08-31T06:23:26.629 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_basic", 2024-08-31T06:23:26.629 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.629 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.629 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.629 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.629 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.629 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.629 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.630 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share basic cluster information (size, version)", 2024-08-31T06:23:26.630 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.630 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.630 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.630 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.630 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_crash": { 2024-08-31T06:23:26.630 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_crash", 2024-08-31T06:23:26.630 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.630 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.630 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.630 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.630 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.630 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.630 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.631 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-08-31T06:23:26.631 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.631 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.631 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.631 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.631 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_device": { 2024-08-31T06:23:26.631 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_device", 2024-08-31T06:23:26.631 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.631 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.631 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.631 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.631 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.631 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.631 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.632 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-08-31T06:23:26.632 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.632 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.632 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.632 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.632 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_ident": { 2024-08-31T06:23:26.632 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_ident", 2024-08-31T06:23:26.632 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.632 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.632 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.632 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.632 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.632 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.632 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.633 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-08-31T06:23:26.633 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.633 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.633 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.633 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.633 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_perf": { 2024-08-31T06:23:26.633 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_perf", 2024-08-31T06:23:26.633 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.633 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.633 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.633 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.633 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.633 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.633 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.634 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share various performance metrics of a cluster", 2024-08-31T06:23:26.634 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.634 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.634 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.634 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.634 INFO:teuthology.orchestra.run.smithi016.stdout: "contact": { 2024-08-31T06:23:26.634 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "contact", 2024-08-31T06:23:26.634 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.634 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.634 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.634 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.634 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.634 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.634 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.635 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.635 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.635 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.635 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.635 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.635 INFO:teuthology.orchestra.run.smithi016.stdout: "description": { 2024-08-31T06:23:26.635 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "description", 2024-08-31T06:23:26.635 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.635 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.635 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.635 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.635 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.635 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.635 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.636 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.636 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.636 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.636 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.636 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.636 INFO:teuthology.orchestra.run.smithi016.stdout: "device_url": { 2024-08-31T06:23:26.636 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_url", 2024-08-31T06:23:26.636 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.636 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.636 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.636 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-08-31T06:23:26.636 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.636 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.636 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.637 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.637 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.637 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.637 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.637 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.637 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-08-31T06:23:26.637 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-08-31T06:23:26.637 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.637 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.637 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.637 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.637 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.637 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.637 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.638 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.638 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.638 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.638 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.638 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.638 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-08-31T06:23:26.638 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-08-31T06:23:26.638 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.638 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.638 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.638 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "24", 2024-08-31T06:23:26.638 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "8", 2024-08-31T06:23:26.638 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.638 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.639 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.639 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.639 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.639 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.639 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.639 INFO:teuthology.orchestra.run.smithi016.stdout: "last_opt_revision": { 2024-08-31T06:23:26.639 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "last_opt_revision", 2024-08-31T06:23:26.639 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.639 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.639 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.639 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1", 2024-08-31T06:23:26.639 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.639 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.639 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.639 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.640 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.640 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.640 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.640 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.640 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard": { 2024-08-31T06:23:26.640 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard", 2024-08-31T06:23:26.640 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.640 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.640 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.640 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.640 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.640 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.640 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.640 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.641 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.641 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.641 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.641 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.641 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard_description": { 2024-08-31T06:23:26.641 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard_description", 2024-08-31T06:23:26.641 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.641 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.641 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.641 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.641 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.641 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.641 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.641 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.642 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.642 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.642 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.642 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.642 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.642 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.642 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.642 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.642 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.642 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.642 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.642 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.642 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.642 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.643 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.643 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.643 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.643 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.643 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.643 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.643 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.643 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.643 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.643 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.643 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.643 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.643 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.643 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.644 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.644 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.644 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.644 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.644 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.644 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.644 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.644 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.644 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.644 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.644 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.644 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.644 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.644 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.645 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.645 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.645 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.645 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.645 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.645 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.645 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.645 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.645 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.645 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.645 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.645 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.645 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.645 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.645 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.646 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.646 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.646 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.646 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.646 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.646 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.646 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.646 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.646 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.646 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.646 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.646 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.646 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.647 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.647 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.647 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.647 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.647 INFO:teuthology.orchestra.run.smithi016.stdout: "organization": { 2024-08-31T06:23:26.647 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "organization", 2024-08-31T06:23:26.647 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.647 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.647 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.647 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.647 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.647 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.647 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.648 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.648 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.648 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.648 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.648 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.648 INFO:teuthology.orchestra.run.smithi016.stdout: "proxy": { 2024-08-31T06:23:26.648 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "proxy", 2024-08-31T06:23:26.648 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.648 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.648 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.648 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.648 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.648 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.649 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.649 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.649 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.649 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.649 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.649 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.649 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.649 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.649 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.649 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.649 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.649 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.649 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.649 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.649 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.650 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.650 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.650 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.650 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.650 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.650 INFO:teuthology.orchestra.run.smithi016.stdout: "url": { 2024-08-31T06:23:26.650 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url", 2024-08-31T06:23:26.650 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.650 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.650 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.650 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-08-31T06:23:26.650 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.650 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.650 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.650 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.651 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.651 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.651 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.651 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.651 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.651 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.651 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.651 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "test_orchestrator", 2024-08-31T06:23:26.651 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.651 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.651 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.651 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.651 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.651 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.651 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.651 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.652 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.652 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.652 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.652 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.652 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.652 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.652 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.652 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.652 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.652 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.652 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.652 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.652 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.652 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.653 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.653 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.653 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.653 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.653 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.653 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.653 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.653 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.653 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.653 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.653 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.653 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.653 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.654 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.654 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.654 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.654 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.654 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.654 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.654 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.654 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.654 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.654 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.654 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.654 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.654 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.654 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.654 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.654 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.655 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.655 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.655 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.655 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.655 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.655 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.655 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.655 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.655 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.655 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.655 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.655 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.655 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.655 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.655 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.656 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.656 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.656 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.656 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.656 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.656 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.656 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.656 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.656 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.656 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.656 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.656 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.657 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.657 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.657 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.657 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.657 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.657 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.657 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.657 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.657 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.657 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.657 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.657 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.657 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2024-08-31T06:23:26.657 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.658 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.658 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.658 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.658 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.658 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.658 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.658 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.658 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.658 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.658 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.658 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.658 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.659 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.659 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.659 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.659 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.659 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.659 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.659 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.659 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.659 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.659 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.659 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.659 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.659 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.660 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.660 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.660 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.660 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.660 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.660 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.660 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.660 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.660 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.660 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.660 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.660 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.660 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.661 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.661 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.661 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.661 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.661 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.661 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.661 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.661 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.661 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.661 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.661 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.661 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.661 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.661 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.662 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.662 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.662 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.662 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.662 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.662 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.662 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.662 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.662 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.662 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.662 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.662 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.662 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.662 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.663 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.663 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.663 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.663 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.663 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.663 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.663 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_clones": { 2024-08-31T06:23:26.663 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_clones", 2024-08-31T06:23:26.663 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.663 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.663 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.663 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4", 2024-08-31T06:23:26.663 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.664 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.664 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.664 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Number of asynchronous cloner threads", 2024-08-31T06:23:26.664 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.664 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.664 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.664 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.664 INFO:teuthology.orchestra.run.smithi016.stdout: "periodic_async_work": { 2024-08-31T06:23:26.664 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "periodic_async_work", 2024-08-31T06:23:26.664 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.664 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.664 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.664 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.665 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.665 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.665 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.665 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Periodically check for async work", 2024-08-31T06:23:26.665 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.665 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.665 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.665 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.665 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_delay": { 2024-08-31T06:23:26.665 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_delay", 2024-08-31T06:23:26.665 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.665 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.665 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.665 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.666 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.666 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.666 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.666 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-08-31T06:23:26.666 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.666 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.666 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.666 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.666 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_no_wait": { 2024-08-31T06:23:26.666 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_no_wait", 2024-08-31T06:23:26.666 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.666 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.666 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.666 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-08-31T06:23:26.667 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.667 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.667 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.667 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-08-31T06:23:26.667 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.667 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.667 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.667 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.667 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.667 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.667 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.667 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.667 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.667 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.668 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.668 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.668 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.668 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.668 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.668 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.668 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.668 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.668 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.668 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.668 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.668 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix", 2024-08-31T06:23:26.668 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-08-31T06:23:26.668 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-08-31T06:23:26.669 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-08-31T06:23:26.669 INFO:teuthology.orchestra.run.smithi016.stdout: "discovery_interval": { 2024-08-31T06:23:26.669 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "discovery_interval", 2024-08-31T06:23:26.669 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-08-31T06:23:26.669 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.669 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.669 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "100", 2024-08-31T06:23:26.669 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.669 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.669 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.669 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.669 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.669 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.669 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.670 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.670 INFO:teuthology.orchestra.run.smithi016.stdout: "identifier": { 2024-08-31T06:23:26.670 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "identifier", 2024-08-31T06:23:26.670 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.670 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.670 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.670 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.670 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.670 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.670 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.670 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.670 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.670 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.670 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.671 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.671 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-08-31T06:23:26.671 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-08-31T06:23:26.671 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-08-31T06:23:26.671 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.671 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.671 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-08-31T06:23:26.671 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.671 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.671 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.671 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.671 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.671 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.671 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.671 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.672 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-08-31T06:23:26.672 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-08-31T06:23:26.672 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.672 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.672 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.672 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.672 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.672 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.672 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.672 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.672 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.672 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.672 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.673 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.673 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.673 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.673 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.673 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.673 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.673 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.673 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.673 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-08-31T06:23:26.673 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-08-31T06:23:26.673 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.673 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.673 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.673 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.674 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.674 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.674 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.674 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.674 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.674 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.674 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.674 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.674 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-08-31T06:23:26.674 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-08-31T06:23:26.674 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.674 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.674 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.674 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-08-31T06:23:26.674 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.675 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.675 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-08-31T06:23:26.675 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-08-31T06:23:26.675 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-08-31T06:23:26.675 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-08-31T06:23:26.675 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-08-31T06:23:26.675 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-08-31T06:23:26.675 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-08-31T06:23:26.675 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.675 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.675 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.675 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.675 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.676 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.676 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-08-31T06:23:26.676 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-08-31T06:23:26.676 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-08-31T06:23:26.676 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.676 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.676 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-08-31T06:23:26.676 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.676 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.676 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.676 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.676 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.676 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.676 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.677 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.677 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-08-31T06:23:26.677 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-08-31T06:23:26.677 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.677 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-08-31T06:23:26.677 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-08-31T06:23:26.677 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-08-31T06:23:26.677 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.677 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.677 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.677 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.677 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.677 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.677 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.678 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.678 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_host": { 2024-08-31T06:23:26.678 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_host", 2024-08-31T06:23:26.678 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.678 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.678 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.678 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-08-31T06:23:26.678 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.678 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.678 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.678 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.678 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.678 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.678 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.679 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.679 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_port": { 2024-08-31T06:23:26.679 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_port", 2024-08-31T06:23:26.679 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-08-31T06:23:26.679 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.679 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.679 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10051", 2024-08-31T06:23:26.679 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.679 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.679 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.679 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.679 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.679 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.679 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.680 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.680 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_sender": { 2024-08-31T06:23:26.680 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_sender", 2024-08-31T06:23:26.680 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-08-31T06:23:26.680 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-08-31T06:23:26.680 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-08-31T06:23:26.680 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-08-31T06:23:26.680 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-08-31T06:23:26.680 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-08-31T06:23:26.680 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-08-31T06:23:26.680 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-08-31T06:23:26.680 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-08-31T06:23:26.680 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-08-31T06:23:26.680 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-08-31T06:23:26.681 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.681 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.681 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.681 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.681 INFO:teuthology.orchestra.run.smithi016.stdout: "services": { 2024-08-31T06:23:26.681 INFO:teuthology.orchestra.run.smithi016.stdout: "dashboard": "https://172.21.15.16:8443/" 2024-08-31T06:23:26.681 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.681 INFO:teuthology.orchestra.run.smithi016.stdout: "always_on_modules": { 2024-08-31T06:23:26.681 INFO:teuthology.orchestra.run.smithi016.stdout: "octopus": [ 2024-08-31T06:23:26.681 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-08-31T06:23:26.681 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-08-31T06:23:26.681 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-08-31T06:23:26.681 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-08-31T06:23:26.681 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-08-31T06:23:26.682 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-08-31T06:23:26.682 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-08-31T06:23:26.682 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-08-31T06:23:26.682 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-08-31T06:23:26.682 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-08-31T06:23:26.682 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.682 INFO:teuthology.orchestra.run.smithi016.stdout: "pacific": [ 2024-08-31T06:23:26.682 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-08-31T06:23:26.682 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-08-31T06:23:26.682 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-08-31T06:23:26.682 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-08-31T06:23:26.682 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-08-31T06:23:26.682 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-08-31T06:23:26.682 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-08-31T06:23:26.682 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-08-31T06:23:26.683 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-08-31T06:23:26.683 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-08-31T06:23:26.683 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.683 INFO:teuthology.orchestra.run.smithi016.stdout: "quincy": [ 2024-08-31T06:23:26.683 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-08-31T06:23:26.683 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-08-31T06:23:26.683 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-08-31T06:23:26.683 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-08-31T06:23:26.683 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-08-31T06:23:26.683 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-08-31T06:23:26.683 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-08-31T06:23:26.683 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-08-31T06:23:26.683 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-08-31T06:23:26.684 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-08-31T06:23:26.684 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.684 INFO:teuthology.orchestra.run.smithi016.stdout: "reef": [ 2024-08-31T06:23:26.684 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-08-31T06:23:26.684 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-08-31T06:23:26.684 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-08-31T06:23:26.684 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-08-31T06:23:26.684 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-08-31T06:23:26.684 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-08-31T06:23:26.684 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-08-31T06:23:26.684 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-08-31T06:23:26.684 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-08-31T06:23:26.684 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-08-31T06:23:26.684 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-08-31T06:23:26.685 INFO:teuthology.orchestra.run.smithi016.stdout: "squid": [ 2024-08-31T06:23:26.685 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-08-31T06:23:26.685 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-08-31T06:23:26.685 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-08-31T06:23:26.685 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-08-31T06:23:26.685 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-08-31T06:23:26.685 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-08-31T06:23:26.685 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-08-31T06:23:26.685 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-08-31T06:23:26.685 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-08-31T06:23:26.688 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-08-31T06:23:26.688 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-08-31T06:23:26.688 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.688 INFO:teuthology.orchestra.run.smithi016.stdout: "last_failure_osd_epoch": 5, 2024-08-31T06:23:26.688 INFO:teuthology.orchestra.run.smithi016.stdout: "active_clients": [ 2024-08-31T06:23:26.688 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.688 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2024-08-31T06:23:26.688 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-08-31T06:23:26.688 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.688 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-08-31T06:23:26.688 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-08-31T06:23:26.689 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1836579324 2024-08-31T06:23:26.689 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.689 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-08-31T06:23:26.689 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.689 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.689 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "libcephsqlite", 2024-08-31T06:23:26.689 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-08-31T06:23:26.689 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.689 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-08-31T06:23:26.689 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-08-31T06:23:26.689 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2203903249 2024-08-31T06:23:26.689 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.689 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-08-31T06:23:26.689 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.690 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.690 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2024-08-31T06:23:26.690 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-08-31T06:23:26.690 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.690 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-08-31T06:23:26.690 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-08-31T06:23:26.690 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2975339075 2024-08-31T06:23:26.690 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.690 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-08-31T06:23:26.690 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.690 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.690 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-08-31T06:23:26.690 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-08-31T06:23:26.690 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.690 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-08-31T06:23:26.691 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-08-31T06:23:26.691 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3370086464 2024-08-31T06:23:26.691 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.691 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-08-31T06:23:26.691 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.691 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.691 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-08-31T06:23:26.691 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-08-31T06:23:26.691 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.691 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-08-31T06:23:26.691 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-08-31T06:23:26.691 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2376975678 2024-08-31T06:23:26.691 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.692 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-08-31T06:23:26.692 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:26.692 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.692 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2024-08-31T06:23:26.692 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-08-31T06:23:26.692 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-31T06:23:26.692 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-08-31T06:23:26.692 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-08-31T06:23:26.692 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3165937677 2024-08-31T06:23:26.692 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.692 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-08-31T06:23:26.692 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:26.692 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-08-31T06:23:26.692 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:23:26.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:25.225+0000 7faf02a00640 1 -- 172.21.15.16:0/3699661806 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7faee40b4ca0 msgr2=0x7faee40b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:26.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:25.225+0000 7faf02a00640 1 --2- 172.21.15.16:0/3699661806 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7faee40b4ca0 0x7faee40b71b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7faef0031440 tx=0x7faef0009560 comp rx=0 tx=0).stop 2024-08-31T06:23:26.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:25.225+0000 7faf02a00640 1 -- 172.21.15.16:0/3699661806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faefc10b1d0 msgr2=0x7faefc19c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:26.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:25.225+0000 7faf02a00640 1 --2- 172.21.15.16:0/3699661806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faefc10b1d0 0x7faefc19c740 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7faee800e9f0 tx=0x7faee800eeb0 comp rx=0 tx=0).stop 2024-08-31T06:23:26.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:25.225+0000 7faf02a00640 1 -- 172.21.15.16:0/3699661806 shutdown_connections 2024-08-31T06:23:26.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:25.225+0000 7faf02a00640 1 --2- 172.21.15.16:0/3699661806 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7faee40b4ca0 0x7faee40b71b0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:26.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:25.225+0000 7faf02a00640 1 --2- 172.21.15.16:0/3699661806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faefc10bb20 0x7faefc1a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:26.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:25.225+0000 7faf02a00640 1 --2- 172.21.15.16:0/3699661806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faefc10b1d0 0x7faefc19c740 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:26.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:25.225+0000 7faf02a00640 1 --2- 172.21.15.16:0/3699661806 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faefc1058a0 0x7faefc19c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:26.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:25.225+0000 7faf02a00640 1 -- 172.21.15.16:0/3699661806 >> 172.21.15.16:0/3699661806 conn(0x7faefc1012c0 msgr2=0x7faefc1036e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:26.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:25.225+0000 7faf02a00640 1 -- 172.21.15.16:0/3699661806 shutdown_connections 2024-08-31T06:23:26.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:25.225+0000 7faf02a00640 1 -- 172.21.15.16:0/3699661806 wait complete. 2024-08-31T06:23:26.702 DEBUG:tasks.check_counter:Getting stats from a 2024-08-31T06:23:26.702 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph daemon mds.a perf dump 2024-08-31T06:23:26.703 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:25 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/3699661806' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-08-31T06:23:26.703 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:25 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/3699661806' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-08-31T06:23:26.704 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:25 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/3699661806' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-08-31T06:23:27.095 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:23:27.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:26 smithi155 ceph-mon[21742]: pgmap v872: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-08-31T06:23:27.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:26 smithi016 ceph-mon[17194]: pgmap v872: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-08-31T06:23:27.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:26 smithi184 ceph-mon[22331]: pgmap v872: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-08-31T06:23:27.872 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:23:27.872 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-08-31T06:23:27.872 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 313, 2024-08-31T06:23:27.872 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 308, 2024-08-31T06:23:27.872 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 854544, 2024-08-31T06:23:27.872 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 190480, 2024-08-31T06:23:27.872 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-08-31T06:23:27.872 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 0, 2024-08-31T06:23:27.873 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.088787696, 2024-08-31T06:23:27.873 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.048298772, 2024-08-31T06:23:27.873 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.046146203, 2024-08-31T06:23:27.873 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-31T06:23:27.873 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T06:23:27.873 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 307, 2024-08-31T06:23:27.873 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.057682313, 2024-08-31T06:23:27.873 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000187890 2024-08-31T06:23:27.873 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:27.873 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-08-31T06:23:27.873 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-08-31T06:23:27.873 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-08-31T06:23:27.873 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-08-31T06:23:27.873 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:27.874 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 854544, 2024-08-31T06:23:27.874 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 190480 2024-08-31T06:23:27.874 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:27.874 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-08-31T06:23:27.874 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 6, 2024-08-31T06:23:27.874 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 238, 2024-08-31T06:23:27.874 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 3312, 2024-08-31T06:23:27.874 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 202208, 2024-08-31T06:23:27.874 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-08-31T06:23:27.874 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 2, 2024-08-31T06:23:27.874 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.044071895, 2024-08-31T06:23:27.874 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.034355153, 2024-08-31T06:23:27.874 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.006380983, 2024-08-31T06:23:27.874 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-31T06:23:27.874 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T06:23:27.875 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 237, 2024-08-31T06:23:27.875 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.047491989, 2024-08-31T06:23:27.875 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000200388 2024-08-31T06:23:27.875 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:27.875 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-08-31T06:23:27.875 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-08-31T06:23:27.875 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-08-31T06:23:27.875 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-08-31T06:23:27.875 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:27.875 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-08-31T06:23:27.875 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 202208 2024-08-31T06:23:27.875 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:27.875 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-08-31T06:23:27.875 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 0, 2024-08-31T06:23:27.875 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 2, 2024-08-31T06:23:27.876 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 0, 2024-08-31T06:23:27.876 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 0, 2024-08-31T06:23:27.876 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 2, 2024-08-31T06:23:27.876 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 1, 2024-08-31T06:23:27.876 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.000766038, 2024-08-31T06:23:27.876 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.000000000, 2024-08-31T06:23:27.876 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.000644630, 2024-08-31T06:23:27.876 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-31T06:23:27.876 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T06:23:27.876 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-08-31T06:23:27.876 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-08-31T06:23:27.876 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-08-31T06:23:27.876 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:27.876 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-08-31T06:23:27.877 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-08-31T06:23:27.877 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-08-31T06:23:27.877 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-08-31T06:23:27.877 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:27.877 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-31T06:23:27.877 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 0 2024-08-31T06:23:27.877 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:27.877 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-08-31T06:23:27.877 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 0, 2024-08-31T06:23:27.877 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-08-31T06:23:27.877 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:27.877 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-08-31T06:23:27.877 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-08-31T06:23:27.877 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-08-31T06:23:27.877 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-08-31T06:23:27.878 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-08-31T06:23:27.878 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-08-31T06:23:27.878 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-08-31T06:23:27.878 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-31T06:23:27.878 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-08-31T06:23:27.878 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-31T06:23:27.878 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-08-31T06:23:27.878 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-08-31T06:23:27.878 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-08-31T06:23:27.878 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-31T06:23:27.878 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-08-31T06:23:27.878 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-08-31T06:23:27.878 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-08-31T06:23:27.878 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-08-31T06:23:27.879 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-08-31T06:23:27.879 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-31T06:23:27.879 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-08-31T06:23:27.879 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-31T06:23:27.879 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-08-31T06:23:27.879 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-08-31T06:23:27.879 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-08-31T06:23:27.879 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-08-31T06:23:27.879 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-08-31T06:23:27.879 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-31T06:23:27.879 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-08-31T06:23:27.879 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-08-31T06:23:27.879 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-08-31T06:23:27.879 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-08-31T06:23:27.879 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-08-31T06:23:27.880 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-08-31T06:23:27.880 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-08-31T06:23:27.880 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-08-31T06:23:27.880 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-08-31T06:23:27.880 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 148704, 2024-08-31T06:23:27.880 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 29, 2024-08-31T06:23:27.880 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-08-31T06:23:27.880 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-08-31T06:23:27.880 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-08-31T06:23:27.880 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-08-31T06:23:27.880 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-08-31T06:23:27.880 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-08-31T06:23:27.880 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-08-31T06:23:27.880 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-08-31T06:23:27.881 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 0, 2024-08-31T06:23:27.881 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 0, 2024-08-31T06:23:27.881 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-08-31T06:23:27.881 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-08-31T06:23:27.881 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-08-31T06:23:27.881 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-08-31T06:23:27.881 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 928, 2024-08-31T06:23:27.881 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 2, 2024-08-31T06:23:27.881 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-08-31T06:23:27.881 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-08-31T06:23:27.881 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-08-31T06:23:27.881 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-08-31T06:23:27.881 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:27.881 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-31T06:23:27.882 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-08-31T06:23:27.882 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-08-31T06:23:27.882 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-08-31T06:23:27.882 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 315, 2024-08-31T06:23:27.882 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 810298, 2024-08-31T06:23:27.882 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-08-31T06:23:27.882 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 315, 2024-08-31T06:23:27.882 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-08-31T06:23:27.882 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-08-31T06:23:27.882 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 315, 2024-08-31T06:23:27.882 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 810298, 2024-08-31T06:23:27.882 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-08-31T06:23:27.882 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-08-31T06:23:27.882 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-08-31T06:23:27.882 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-08-31T06:23:27.883 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:27.883 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:27.883 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:23:28.990 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-08-31T06:23:28.990 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-08-31T06:23:28.991 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-08-31T06:23:28.991 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-08-31T06:23:28.991 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-08-31T06:23:28.991 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-08-31T06:23:28.991 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-08-31T06:23:28.991 DEBUG:tasks.check_counter:Getting stats from d 2024-08-31T06:23:28.991 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph daemon mds.d perf dump 2024-08-31T06:23:29.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:28 smithi155 ceph-mon[21742]: pgmap v873: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 8 op/s 2024-08-31T06:23:29.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:28 smithi016 ceph-mon[17194]: pgmap v873: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 8 op/s 2024-08-31T06:23:29.275 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:28 smithi184 ceph-mon[22331]: pgmap v873: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 8 op/s 2024-08-31T06:23:29.378 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:23:30.126 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:23:30.126 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-08-31T06:23:30.126 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 323, 2024-08-31T06:23:30.126 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 316, 2024-08-31T06:23:30.126 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 859168, 2024-08-31T06:23:30.127 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 194768, 2024-08-31T06:23:30.127 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-08-31T06:23:30.127 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 0, 2024-08-31T06:23:30.127 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.076547672, 2024-08-31T06:23:30.127 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.036125623, 2024-08-31T06:23:30.127 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.046810799, 2024-08-31T06:23:30.127 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-31T06:23:30.127 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T06:23:30.127 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 316, 2024-08-31T06:23:30.127 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.057723381, 2024-08-31T06:23:30.127 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000182668 2024-08-31T06:23:30.127 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:30.127 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-08-31T06:23:30.127 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-08-31T06:23:30.128 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-08-31T06:23:30.128 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-08-31T06:23:30.128 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:30.128 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 859168, 2024-08-31T06:23:30.128 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 194768 2024-08-31T06:23:30.128 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:30.128 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-08-31T06:23:30.128 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 2, 2024-08-31T06:23:30.128 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 241, 2024-08-31T06:23:30.128 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 320, 2024-08-31T06:23:30.128 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 202640, 2024-08-31T06:23:30.128 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-08-31T06:23:30.128 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 2, 2024-08-31T06:23:30.129 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.038726219, 2024-08-31T06:23:30.129 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.032125992, 2024-08-31T06:23:30.129 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.003064275, 2024-08-31T06:23:30.129 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-31T06:23:30.129 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T06:23:30.129 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 239, 2024-08-31T06:23:30.129 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.041414943, 2024-08-31T06:23:30.129 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000173284 2024-08-31T06:23:30.129 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:30.129 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-08-31T06:23:30.129 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-08-31T06:23:30.129 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-08-31T06:23:30.129 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-08-31T06:23:30.129 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:30.129 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 320, 2024-08-31T06:23:30.130 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 202640 2024-08-31T06:23:30.130 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:30.130 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-08-31T06:23:30.130 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 0, 2024-08-31T06:23:30.130 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 2, 2024-08-31T06:23:30.130 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 0, 2024-08-31T06:23:30.130 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 0, 2024-08-31T06:23:30.130 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 2, 2024-08-31T06:23:30.130 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 1, 2024-08-31T06:23:30.130 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.001086706, 2024-08-31T06:23:30.130 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.000000000, 2024-08-31T06:23:30.130 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.000920138, 2024-08-31T06:23:30.130 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-31T06:23:30.130 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T06:23:30.131 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-08-31T06:23:30.131 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-08-31T06:23:30.131 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-08-31T06:23:30.131 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:30.131 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-08-31T06:23:30.131 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-08-31T06:23:30.131 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-08-31T06:23:30.131 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-08-31T06:23:30.131 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:30.131 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-31T06:23:30.131 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 0 2024-08-31T06:23:30.131 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:30.131 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-08-31T06:23:30.131 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 0, 2024-08-31T06:23:30.131 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-08-31T06:23:30.132 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:30.132 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-08-31T06:23:30.132 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-08-31T06:23:30.132 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-08-31T06:23:30.132 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-08-31T06:23:30.132 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-08-31T06:23:30.132 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-08-31T06:23:30.132 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-08-31T06:23:30.132 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-31T06:23:30.132 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-08-31T06:23:30.133 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-31T06:23:30.133 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-08-31T06:23:30.133 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-08-31T06:23:30.133 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-08-31T06:23:30.133 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-31T06:23:30.133 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-08-31T06:23:30.133 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-08-31T06:23:30.133 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-08-31T06:23:30.133 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-08-31T06:23:30.133 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-08-31T06:23:30.133 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-31T06:23:30.133 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-08-31T06:23:30.133 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-31T06:23:30.133 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-08-31T06:23:30.134 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-08-31T06:23:30.134 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-08-31T06:23:30.134 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-08-31T06:23:30.134 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-08-31T06:23:30.134 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-31T06:23:30.134 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-08-31T06:23:30.134 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-08-31T06:23:30.134 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-08-31T06:23:30.134 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-08-31T06:23:30.134 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-08-31T06:23:30.134 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-08-31T06:23:30.134 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-08-31T06:23:30.134 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-08-31T06:23:30.134 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-08-31T06:23:30.134 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 149735, 2024-08-31T06:23:30.135 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 30, 2024-08-31T06:23:30.135 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-08-31T06:23:30.135 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-08-31T06:23:30.135 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-08-31T06:23:30.135 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-08-31T06:23:30.135 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-08-31T06:23:30.135 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-08-31T06:23:30.135 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-08-31T06:23:30.135 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-08-31T06:23:30.135 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 0, 2024-08-31T06:23:30.135 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 0, 2024-08-31T06:23:30.135 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-08-31T06:23:30.135 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-08-31T06:23:30.135 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-08-31T06:23:30.136 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-08-31T06:23:30.136 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 928, 2024-08-31T06:23:30.136 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 2, 2024-08-31T06:23:30.136 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-08-31T06:23:30.136 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-08-31T06:23:30.136 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-08-31T06:23:30.136 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-08-31T06:23:30.136 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:30.136 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-31T06:23:30.136 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-08-31T06:23:30.136 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-08-31T06:23:30.136 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-08-31T06:23:30.136 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 321, 2024-08-31T06:23:30.137 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 811078, 2024-08-31T06:23:30.137 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-08-31T06:23:30.137 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 321, 2024-08-31T06:23:30.137 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-08-31T06:23:30.137 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-08-31T06:23:30.137 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 321, 2024-08-31T06:23:30.137 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 811078, 2024-08-31T06:23:30.137 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-08-31T06:23:30.137 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-08-31T06:23:30.137 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-08-31T06:23:30.137 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-08-31T06:23:30.137 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:30.137 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:30.137 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:23:30.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:30 smithi016 ceph-mon[17194]: pgmap v874: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-08-31T06:23:30.901 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-08-31T06:23:30.901 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-08-31T06:23:30.901 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-08-31T06:23:30.901 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-08-31T06:23:30.901 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-08-31T06:23:30.902 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-08-31T06:23:30.902 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-08-31T06:23:30.902 DEBUG:tasks.check_counter:Getting stats from g 2024-08-31T06:23:30.902 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph daemon mds.g perf dump 2024-08-31T06:23:31.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:30 smithi155 ceph-mon[21742]: pgmap v874: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-08-31T06:23:31.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:30 smithi184 ceph-mon[22331]: pgmap v874: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-08-31T06:23:31.283 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:23:32.036 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:23:32.036 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-08-31T06:23:32.036 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 9211, 2024-08-31T06:23:32.036 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 7253, 2024-08-31T06:23:32.036 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 4122179, 2024-08-31T06:23:32.037 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 10040229, 2024-08-31T06:23:32.037 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 49, 2024-08-31T06:23:32.037 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 8, 2024-08-31T06:23:32.037 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 1.252225964, 2024-08-31T06:23:32.037 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.546834004, 2024-08-31T06:23:32.037 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.786301071, 2024-08-31T06:23:32.037 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.119541482, 2024-08-31T06:23:32.037 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T06:23:32.037 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 7251, 2024-08-31T06:23:32.037 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 1.419958844, 2024-08-31T06:23:32.037 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000195829 2024-08-31T06:23:32.037 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.037 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-08-31T06:23:32.037 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 10651, 2024-08-31T06:23:32.038 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.009708524, 2024-08-31T06:23:32.038 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000911 2024-08-31T06:23:32.038 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.038 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-31T06:23:32.038 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 0 2024-08-31T06:23:32.038 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.038 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-08-31T06:23:32.038 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 3286, 2024-08-31T06:23:32.038 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 3421, 2024-08-31T06:23:32.038 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 1697188, 2024-08-31T06:23:32.038 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 1824900, 2024-08-31T06:23:32.038 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 12, 2024-08-31T06:23:32.038 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 6, 2024-08-31T06:23:32.038 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.475282640, 2024-08-31T06:23:32.039 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.251229995, 2024-08-31T06:23:32.039 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.206891798, 2024-08-31T06:23:32.039 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.036391887, 2024-08-31T06:23:32.039 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T06:23:32.039 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 3419, 2024-08-31T06:23:32.039 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.551754075, 2024-08-31T06:23:32.039 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000161378 2024-08-31T06:23:32.039 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.039 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-08-31T06:23:32.039 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 3944, 2024-08-31T06:23:32.039 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.001094904, 2024-08-31T06:23:32.039 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000277 2024-08-31T06:23:32.039 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.039 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-08-31T06:23:32.040 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 505904 2024-08-31T06:23:32.040 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.040 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-08-31T06:23:32.040 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 6448, 2024-08-31T06:23:32.040 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 8468, 2024-08-31T06:23:32.040 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 3184194, 2024-08-31T06:23:32.040 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 4026423, 2024-08-31T06:23:32.040 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 10, 2024-08-31T06:23:32.040 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 6, 2024-08-31T06:23:32.040 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 2.139333309, 2024-08-31T06:23:32.040 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 1.702201036, 2024-08-31T06:23:32.040 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.574782642, 2024-08-31T06:23:32.040 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.054262093, 2024-08-31T06:23:32.040 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T06:23:32.041 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 8468, 2024-08-31T06:23:32.041 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 4.248855446, 2024-08-31T06:23:32.041 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000501754 2024-08-31T06:23:32.041 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.041 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-08-31T06:23:32.041 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 5208, 2024-08-31T06:23:32.041 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.002948607, 2024-08-31T06:23:32.041 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000566 2024-08-31T06:23:32.041 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.041 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 1046320, 2024-08-31T06:23:32.041 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 227168 2024-08-31T06:23:32.041 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.041 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-08-31T06:23:32.042 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 1, 2024-08-31T06:23:32.042 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-08-31T06:23:32.042 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.042 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-MDSRank": { 2024-08-31T06:23:32.042 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-08-31T06:23:32.042 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-08-31T06:23:32.042 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 3255, 2024-08-31T06:23:32.042 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 4.384257931, 2024-08-31T06:23:32.042 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.001346930 2024-08-31T06:23:32.042 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:32.042 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.042 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-PurgeQueue": { 2024-08-31T06:23:32.042 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-08-31T06:23:32.042 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-08-31T06:23:32.043 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 675, 2024-08-31T06:23:32.043 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.054810849, 2024-08-31T06:23:32.043 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000081201 2024-08-31T06:23:32.043 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:32.043 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.043 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-08-31T06:23:32.043 INFO:teuthology.orchestra.run.smithi016.stdout: "request": 1999, 2024-08-31T06:23:32.043 INFO:teuthology.orchestra.run.smithi016.stdout: "reply": 1922, 2024-08-31T06:23:32.043 INFO:teuthology.orchestra.run.smithi016.stdout: "reply_latency": { 2024-08-31T06:23:32.043 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1922, 2024-08-31T06:23:32.043 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 69.570348373, 2024-08-31T06:23:32.043 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.036196851 2024-08-31T06:23:32.043 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.043 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_reply": 0, 2024-08-31T06:23:32.043 INFO:teuthology.orchestra.run.smithi016.stdout: "forward": 77, 2024-08-31T06:23:32.044 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_complete": 8, 2024-08-31T06:23:32.044 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_keys": 2, 2024-08-31T06:23:32.044 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_commit": 11, 2024-08-31T06:23:32.044 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_split": 0, 2024-08-31T06:23:32.044 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_merge": 0, 2024-08-31T06:23:32.044 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes": 765, 2024-08-31T06:23:32.044 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_top": 129, 2024-08-31T06:23:32.044 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_bottom": 571, 2024-08-31T06:23:32.044 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pin_tail": 65, 2024-08-31T06:23:32.044 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pinned": 502, 2024-08-31T06:23:32.044 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_expired": 903, 2024-08-31T06:23:32.044 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_with_caps": 202, 2024-08-31T06:23:32.044 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": 202, 2024-08-31T06:23:32.044 INFO:teuthology.orchestra.run.smithi016.stdout: "subtrees": 62, 2024-08-31T06:23:32.045 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse": 9321, 2024-08-31T06:23:32.045 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_hit": 8641, 2024-08-31T06:23:32.045 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_forward": 73, 2024-08-31T06:23:32.045 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_discover": 70, 2024-08-31T06:23:32.045 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_dir_fetch": 1, 2024-08-31T06:23:32.045 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_remote_ino": 0, 2024-08-31T06:23:32.045 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_lock": 144, 2024-08-31T06:23:32.045 INFO:teuthology.orchestra.run.smithi016.stdout: "load_cent": 117, 2024-08-31T06:23:32.045 INFO:teuthology.orchestra.run.smithi016.stdout: "q": 0, 2024-08-31T06:23:32.045 INFO:teuthology.orchestra.run.smithi016.stdout: "exported": 17, 2024-08-31T06:23:32.045 INFO:teuthology.orchestra.run.smithi016.stdout: "exported_inodes": 820, 2024-08-31T06:23:32.045 INFO:teuthology.orchestra.run.smithi016.stdout: "imported": 42, 2024-08-31T06:23:32.045 INFO:teuthology.orchestra.run.smithi016.stdout: "imported_inodes": 1207, 2024-08-31T06:23:32.045 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_dir_fetch": 0, 2024-08-31T06:23:32.046 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_backtrace_fetch": 3343, 2024-08-31T06:23:32.046 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_peer_discover": 0, 2024-08-31T06:23:32.046 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rfiles": 1003, 2024-08-31T06:23:32.046 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rbytes": 1271739561, 2024-08-31T06:23:32.046 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rsnaps": 6, 2024-08-31T06:23:32.046 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_fetch": 3343, 2024-08-31T06:23:32.046 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_set_tag": 0, 2024-08-31T06:23:32.046 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_repaired": 0, 2024-08-31T06:23:32.046 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_inotable_repaired": 0, 2024-08-31T06:23:32.046 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_inodes": 106, 2024-08-31T06:23:32.046 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_base_inodes": 0, 2024-08-31T06:23:32.046 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dirfrag_rstats": 106, 2024-08-31T06:23:32.046 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_file_inodes": 3237, 2024-08-31T06:23:32.046 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_inode_file_caps": 7, 2024-08-31T06:23:32.046 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_revoke": 1367, 2024-08-31T06:23:32.047 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_grant": 1676, 2024-08-31T06:23:32.047 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_trunc": 0, 2024-08-31T06:23:32.047 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flushsnap_ack": 1, 2024-08-31T06:23:32.047 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-31T06:23:32.047 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps": 2101, 2024-08-31T06:23:32.047 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps_dirty": 471, 2024-08-31T06:23:32.047 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_cap_release": 188, 2024-08-31T06:23:32.047 INFO:teuthology.orchestra.run.smithi016.stdout: "process_request_cap_release": 1117 2024-08-31T06:23:32.047 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.047 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_cache": { 2024-08-31T06:23:32.047 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update": 52, 2024-08-31T06:23:32.047 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update_receipt": 276, 2024-08-31T06:23:32.047 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_try_discover": 153, 2024-08-31T06:23:32.047 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_send_discover": 265, 2024-08-31T06:23:32.048 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_handle_discover": 91, 2024-08-31T06:23:32.048 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays": 8, 2024-08-31T06:23:32.048 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_delayed": 0, 2024-08-31T06:23:32.048 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_enqueuing": 0, 2024-08-31T06:23:32.048 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_created": 274, 2024-08-31T06:23:32.048 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_enqueued": 217, 2024-08-31T06:23:32.048 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_reintegrated": 49, 2024-08-31T06:23:32.048 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_migrated": 7, 2024-08-31T06:23:32.048 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_processing": 0, 2024-08-31T06:23:32.048 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_enqueued": 0, 2024-08-31T06:23:32.048 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_prioritized": 0, 2024-08-31T06:23:32.048 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_started": 0, 2024-08-31T06:23:32.048 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_completed": 0, 2024-08-31T06:23:32.048 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_quiesce_path": 12, 2024-08-31T06:23:32.049 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_quiesce_inode": 2748, 2024-08-31T06:23:32.049 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_enqueue_scrub": 0, 2024-08-31T06:23:32.049 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_exportdir": 17, 2024-08-31T06:23:32.049 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_flush": 0, 2024-08-31T06:23:32.049 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragmentdir": 0, 2024-08-31T06:23:32.049 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragstats": 0, 2024-08-31T06:23:32.049 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_inodestats": 0, 2024-08-31T06:23:32.049 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-08-31T06:23:32.049 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_succeeded": 0, 2024-08-31T06:23:32.049 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_write_failed": 0 2024-08-31T06:23:32.049 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.049 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_log": { 2024-08-31T06:23:32.049 INFO:teuthology.orchestra.run.smithi016.stdout: "evlrg": 0, 2024-08-31T06:23:32.049 INFO:teuthology.orchestra.run.smithi016.stdout: "evadd": 2834, 2024-08-31T06:23:32.049 INFO:teuthology.orchestra.run.smithi016.stdout: "evex": 0, 2024-08-31T06:23:32.050 INFO:teuthology.orchestra.run.smithi016.stdout: "evtrm": 0, 2024-08-31T06:23:32.050 INFO:teuthology.orchestra.run.smithi016.stdout: "ev": 2834, 2024-08-31T06:23:32.050 INFO:teuthology.orchestra.run.smithi016.stdout: "evexg": 0, 2024-08-31T06:23:32.050 INFO:teuthology.orchestra.run.smithi016.stdout: "evexd": 0, 2024-08-31T06:23:32.050 INFO:teuthology.orchestra.run.smithi016.stdout: "segadd": 5, 2024-08-31T06:23:32.050 INFO:teuthology.orchestra.run.smithi016.stdout: "segex": 0, 2024-08-31T06:23:32.050 INFO:teuthology.orchestra.run.smithi016.stdout: "segtrm": 0, 2024-08-31T06:23:32.050 INFO:teuthology.orchestra.run.smithi016.stdout: "seg": 5, 2024-08-31T06:23:32.050 INFO:teuthology.orchestra.run.smithi016.stdout: "segmjr": 2, 2024-08-31T06:23:32.051 INFO:teuthology.orchestra.run.smithi016.stdout: "segexg": 0, 2024-08-31T06:23:32.051 INFO:teuthology.orchestra.run.smithi016.stdout: "segexd": 0, 2024-08-31T06:23:32.051 INFO:teuthology.orchestra.run.smithi016.stdout: "expos": 4194304, 2024-08-31T06:23:32.051 INFO:teuthology.orchestra.run.smithi016.stdout: "wrpos": 11359943, 2024-08-31T06:23:32.051 INFO:teuthology.orchestra.run.smithi016.stdout: "rdpos": 0, 2024-08-31T06:23:32.051 INFO:teuthology.orchestra.run.smithi016.stdout: "jlat": { 2024-08-31T06:23:32.051 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1113, 2024-08-31T06:23:32.051 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 5.595913411, 2024-08-31T06:23:32.051 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.005027774 2024-08-31T06:23:32.051 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.051 INFO:teuthology.orchestra.run.smithi016.stdout: "replayed": 0 2024-08-31T06:23:32.051 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.051 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_mem": { 2024-08-31T06:23:32.051 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 625, 2024-08-31T06:23:32.052 INFO:teuthology.orchestra.run.smithi016.stdout: "ino+": 1652, 2024-08-31T06:23:32.052 INFO:teuthology.orchestra.run.smithi016.stdout: "ino-": 1027, 2024-08-31T06:23:32.052 INFO:teuthology.orchestra.run.smithi016.stdout: "dir": 104, 2024-08-31T06:23:32.052 INFO:teuthology.orchestra.run.smithi016.stdout: "dir+": 231, 2024-08-31T06:23:32.052 INFO:teuthology.orchestra.run.smithi016.stdout: "dir-": 127, 2024-08-31T06:23:32.052 INFO:teuthology.orchestra.run.smithi016.stdout: "dn": 779, 2024-08-31T06:23:32.052 INFO:teuthology.orchestra.run.smithi016.stdout: "dn+": 1937, 2024-08-31T06:23:32.052 INFO:teuthology.orchestra.run.smithi016.stdout: "dn-": 1158, 2024-08-31T06:23:32.052 INFO:teuthology.orchestra.run.smithi016.stdout: "cap": 202, 2024-08-31T06:23:32.052 INFO:teuthology.orchestra.run.smithi016.stdout: "cap+": 803, 2024-08-31T06:23:32.052 INFO:teuthology.orchestra.run.smithi016.stdout: "cap-": 601, 2024-08-31T06:23:32.052 INFO:teuthology.orchestra.run.smithi016.stdout: "rss": 131976, 2024-08-31T06:23:32.052 INFO:teuthology.orchestra.run.smithi016.stdout: "heap": 339619840 2024-08-31T06:23:32.053 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.053 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_server": { 2024-08-31T06:23:32.053 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_client_request": 2807, 2024-08-31T06:23:32.053 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_server_request": 250, 2024-08-31T06:23:32.053 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_request": 1999, 2024-08-31T06:23:32.053 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_session": 134, 2024-08-31T06:23:32.053 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_peer_request": 448, 2024-08-31T06:23:32.053 INFO:teuthology.orchestra.run.smithi016.stdout: "req_create_latency": { 2024-08-31T06:23:32.053 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 232, 2024-08-31T06:23:32.054 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 44.201473481, 2024-08-31T06:23:32.054 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.190523592 2024-08-31T06:23:32.054 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.054 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getattr_latency": { 2024-08-31T06:23:32.054 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 565, 2024-08-31T06:23:32.054 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 18.092788569, 2024-08-31T06:23:32.054 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.032022634 2024-08-31T06:23:32.054 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.054 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getfilelock_latency": { 2024-08-31T06:23:32.054 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-08-31T06:23:32.054 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-08-31T06:23:32.054 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-08-31T06:23:32.054 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.054 INFO:teuthology.orchestra.run.smithi016.stdout: "req_link_latency": { 2024-08-31T06:23:32.055 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 63, 2024-08-31T06:23:32.055 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.317752748, 2024-08-31T06:23:32.055 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.005043694 2024-08-31T06:23:32.055 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.055 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookup_latency": { 2024-08-31T06:23:32.055 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 468, 2024-08-31T06:23:32.055 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 4.776379180, 2024-08-31T06:23:32.055 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.010205938 2024-08-31T06:23:32.055 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.055 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookuphash_latency": { 2024-08-31T06:23:32.055 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-08-31T06:23:32.055 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-08-31T06:23:32.055 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-08-31T06:23:32.055 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.056 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupino_latency": { 2024-08-31T06:23:32.056 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1, 2024-08-31T06:23:32.056 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000145334, 2024-08-31T06:23:32.056 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000145334 2024-08-31T06:23:32.056 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.056 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupname_latency": { 2024-08-31T06:23:32.056 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-08-31T06:23:32.056 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-08-31T06:23:32.056 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-08-31T06:23:32.056 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.056 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupparent_latency": { 2024-08-31T06:23:32.056 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-08-31T06:23:32.056 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-08-31T06:23:32.057 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-08-31T06:23:32.057 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.057 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupsnap_latency": { 2024-08-31T06:23:32.057 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-08-31T06:23:32.057 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-08-31T06:23:32.057 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-08-31T06:23:32.058 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.058 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lssnap_latency": { 2024-08-31T06:23:32.058 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-08-31T06:23:32.058 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-08-31T06:23:32.058 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-08-31T06:23:32.058 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.058 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mkdir_latency": { 2024-08-31T06:23:32.058 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-08-31T06:23:32.058 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-08-31T06:23:32.058 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-08-31T06:23:32.058 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.058 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mknod_latency": { 2024-08-31T06:23:32.058 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-08-31T06:23:32.058 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-08-31T06:23:32.059 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-08-31T06:23:32.059 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.059 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mksnap_latency": { 2024-08-31T06:23:32.059 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-08-31T06:23:32.059 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-08-31T06:23:32.059 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-08-31T06:23:32.059 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.059 INFO:teuthology.orchestra.run.smithi016.stdout: "req_open_latency": { 2024-08-31T06:23:32.059 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 230, 2024-08-31T06:23:32.059 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.251574416, 2024-08-31T06:23:32.059 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.001093801 2024-08-31T06:23:32.059 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.059 INFO:teuthology.orchestra.run.smithi016.stdout: "req_readdir_latency": { 2024-08-31T06:23:32.059 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 33, 2024-08-31T06:23:32.060 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.020635847, 2024-08-31T06:23:32.060 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000625328 2024-08-31T06:23:32.060 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.060 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rename_latency": { 2024-08-31T06:23:32.060 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 250, 2024-08-31T06:23:32.060 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 1.693492443, 2024-08-31T06:23:32.060 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.006773969 2024-08-31T06:23:32.060 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.060 INFO:teuthology.orchestra.run.smithi016.stdout: "req_renamesnap_latency": { 2024-08-31T06:23:32.060 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-08-31T06:23:32.060 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-08-31T06:23:32.060 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-08-31T06:23:32.060 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.060 INFO:teuthology.orchestra.run.smithi016.stdout: "req_snapdiff_latency": { 2024-08-31T06:23:32.061 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-08-31T06:23:32.061 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-08-31T06:23:32.061 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-08-31T06:23:32.061 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.061 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmdir_latency": { 2024-08-31T06:23:32.061 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-08-31T06:23:32.061 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-08-31T06:23:32.061 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-08-31T06:23:32.061 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.061 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmsnap_latency": { 2024-08-31T06:23:32.061 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-08-31T06:23:32.061 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-08-31T06:23:32.061 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-08-31T06:23:32.061 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.062 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmxattr_latency": { 2024-08-31T06:23:32.062 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-08-31T06:23:32.062 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-08-31T06:23:32.062 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-08-31T06:23:32.062 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.062 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setattr_latency": { 2024-08-31T06:23:32.062 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-08-31T06:23:32.062 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-08-31T06:23:32.062 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-08-31T06:23:32.062 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.062 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setdirlayout_latency": { 2024-08-31T06:23:32.062 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-08-31T06:23:32.062 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-08-31T06:23:32.062 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-08-31T06:23:32.063 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.063 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setfilelock_latency": { 2024-08-31T06:23:32.063 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-08-31T06:23:32.063 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-08-31T06:23:32.063 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-08-31T06:23:32.063 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.063 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setlayout_latency": { 2024-08-31T06:23:32.063 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-08-31T06:23:32.063 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-08-31T06:23:32.063 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-08-31T06:23:32.063 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.063 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setxattr_latency": { 2024-08-31T06:23:32.063 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-08-31T06:23:32.063 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-08-31T06:23:32.064 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-08-31T06:23:32.064 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.064 INFO:teuthology.orchestra.run.smithi016.stdout: "req_symlink_latency": { 2024-08-31T06:23:32.064 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-08-31T06:23:32.064 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-08-31T06:23:32.064 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-08-31T06:23:32.064 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.064 INFO:teuthology.orchestra.run.smithi016.stdout: "req_unlink_latency": { 2024-08-31T06:23:32.064 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 80, 2024-08-31T06:23:32.064 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.216106355, 2024-08-31T06:23:32.064 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.002701329 2024-08-31T06:23:32.064 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.064 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_revoke_eviction": 0, 2024-08-31T06:23:32.064 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_acquisition_throttle": 0, 2024-08-31T06:23:32.065 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getvxattr_latency": { 2024-08-31T06:23:32.065 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-08-31T06:23:32.065 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-08-31T06:23:32.065 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-08-31T06:23:32.065 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:32.065 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.065 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_sessions": { 2024-08-31T06:23:32.065 INFO:teuthology.orchestra.run.smithi016.stdout: "session_count": 1, 2024-08-31T06:23:32.065 INFO:teuthology.orchestra.run.smithi016.stdout: "session_add": 2, 2024-08-31T06:23:32.065 INFO:teuthology.orchestra.run.smithi016.stdout: "session_remove": 1, 2024-08-31T06:23:32.065 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_open": 1, 2024-08-31T06:23:32.065 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_stale": 0, 2024-08-31T06:23:32.065 INFO:teuthology.orchestra.run.smithi016.stdout: "total_load": 271, 2024-08-31T06:23:32.065 INFO:teuthology.orchestra.run.smithi016.stdout: "average_load": 271, 2024-08-31T06:23:32.066 INFO:teuthology.orchestra.run.smithi016.stdout: "avg_session_uptime": 980, 2024-08-31T06:23:32.066 INFO:teuthology.orchestra.run.smithi016.stdout: "mdthresh_evicted": 0 2024-08-31T06:23:32.066 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.066 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-08-31T06:23:32.066 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-08-31T06:23:32.066 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-08-31T06:23:32.066 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-08-31T06:23:32.066 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-08-31T06:23:32.066 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-08-31T06:23:32.066 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-08-31T06:23:32.066 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-31T06:23:32.066 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-08-31T06:23:32.066 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-31T06:23:32.066 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-08-31T06:23:32.067 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-08-31T06:23:32.067 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-08-31T06:23:32.067 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-31T06:23:32.067 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-08-31T06:23:32.067 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-08-31T06:23:32.067 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-08-31T06:23:32.067 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-08-31T06:23:32.067 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-08-31T06:23:32.067 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-31T06:23:32.067 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-08-31T06:23:32.067 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-31T06:23:32.067 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-08-31T06:23:32.067 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-08-31T06:23:32.067 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-08-31T06:23:32.067 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-08-31T06:23:32.068 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-08-31T06:23:32.068 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-31T06:23:32.068 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-08-31T06:23:32.068 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-08-31T06:23:32.068 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-08-31T06:23:32.068 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-08-31T06:23:32.068 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-08-31T06:23:32.068 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-08-31T06:23:32.068 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-08-31T06:23:32.068 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-08-31T06:23:32.068 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-08-31T06:23:32.068 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 759897, 2024-08-31T06:23:32.068 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 844, 2024-08-31T06:23:32.068 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-08-31T06:23:32.069 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-08-31T06:23:32.069 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-08-31T06:23:32.069 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-08-31T06:23:32.069 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-08-31T06:23:32.069 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-08-31T06:23:32.069 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-08-31T06:23:32.069 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-08-31T06:23:32.069 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 8064, 2024-08-31T06:23:32.069 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 178, 2024-08-31T06:23:32.069 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-08-31T06:23:32.069 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-08-31T06:23:32.069 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-08-31T06:23:32.069 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-08-31T06:23:32.069 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 2349283, 2024-08-31T06:23:32.069 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 44250, 2024-08-31T06:23:32.070 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-08-31T06:23:32.070 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-08-31T06:23:32.070 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-08-31T06:23:32.070 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-08-31T06:23:32.070 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.070 INFO:teuthology.orchestra.run.smithi016.stdout: "objecter": { 2024-08-31T06:23:32.070 INFO:teuthology.orchestra.run.smithi016.stdout: "op_active": 0, 2024-08-31T06:23:32.070 INFO:teuthology.orchestra.run.smithi016.stdout: "op_laggy": 0, 2024-08-31T06:23:32.070 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send": 5246, 2024-08-31T06:23:32.070 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send_bytes": 7300618, 2024-08-31T06:23:32.070 INFO:teuthology.orchestra.run.smithi016.stdout: "op_resend": 33, 2024-08-31T06:23:32.070 INFO:teuthology.orchestra.run.smithi016.stdout: "op_reply": 5246, 2024-08-31T06:23:32.070 INFO:teuthology.orchestra.run.smithi016.stdout: "op_latency": { 2024-08-31T06:23:32.070 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 5246, 2024-08-31T06:23:32.071 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 22.116657912, 2024-08-31T06:23:32.071 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.004215908 2024-08-31T06:23:32.071 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.071 INFO:teuthology.orchestra.run.smithi016.stdout: "op_inflight": 0, 2024-08-31T06:23:32.071 INFO:teuthology.orchestra.run.smithi016.stdout: "oplen_avg": { 2024-08-31T06:23:32.071 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 5246, 2024-08-31T06:23:32.071 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 5529 2024-08-31T06:23:32.071 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.071 INFO:teuthology.orchestra.run.smithi016.stdout: "op": 5246, 2024-08-31T06:23:32.071 INFO:teuthology.orchestra.run.smithi016.stdout: "op_r": 3442, 2024-08-31T06:23:32.071 INFO:teuthology.orchestra.run.smithi016.stdout: "op_w": 1804, 2024-08-31T06:23:32.071 INFO:teuthology.orchestra.run.smithi016.stdout: "op_rmw": 0, 2024-08-31T06:23:32.071 INFO:teuthology.orchestra.run.smithi016.stdout: "op_pg": 0, 2024-08-31T06:23:32.071 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_stat": 0, 2024-08-31T06:23:32.071 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_create": 10, 2024-08-31T06:23:32.072 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_read": 84, 2024-08-31T06:23:32.072 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_write": 1197, 2024-08-31T06:23:32.072 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writefull": 136, 2024-08-31T06:23:32.072 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writesame": 0, 2024-08-31T06:23:32.072 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_append": 0, 2024-08-31T06:23:32.072 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_zero": 0, 2024-08-31T06:23:32.072 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_truncate": 0, 2024-08-31T06:23:32.072 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_delete": 230, 2024-08-31T06:23:32.072 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_mapext": 0, 2024-08-31T06:23:32.072 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_sparse_read": 0, 2024-08-31T06:23:32.072 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_clonerange": 0, 2024-08-31T06:23:32.072 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_getxattr": 3348, 2024-08-31T06:23:32.072 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_setxattr": 20, 2024-08-31T06:23:32.072 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_cmpxattr": 0, 2024-08-31T06:23:32.073 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_rmxattr": 0, 2024-08-31T06:23:32.073 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_resetxattrs": 0, 2024-08-31T06:23:32.073 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_call": 0, 2024-08-31T06:23:32.073 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_watch": 0, 2024-08-31T06:23:32.073 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_notify": 0, 2024-08-31T06:23:32.073 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_src_cmpxattr": 0, 2024-08-31T06:23:32.073 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls": 0, 2024-08-31T06:23:32.073 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls_filter": 0, 2024-08-31T06:23:32.073 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_other": 0, 2024-08-31T06:23:32.073 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_active": 0, 2024-08-31T06:23:32.073 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_send": 0, 2024-08-31T06:23:32.073 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_resend": 0, 2024-08-31T06:23:32.073 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_ping": 0, 2024-08-31T06:23:32.073 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_active": 0, 2024-08-31T06:23:32.073 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_send": 0, 2024-08-31T06:23:32.074 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_resend": 0, 2024-08-31T06:23:32.074 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_active": 0, 2024-08-31T06:23:32.074 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_send": 0, 2024-08-31T06:23:32.074 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_resend": 0, 2024-08-31T06:23:32.074 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_active": 0, 2024-08-31T06:23:32.074 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_send": 0, 2024-08-31T06:23:32.074 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_resend": 0, 2024-08-31T06:23:32.074 INFO:teuthology.orchestra.run.smithi016.stdout: "command_active": 0, 2024-08-31T06:23:32.074 INFO:teuthology.orchestra.run.smithi016.stdout: "command_send": 0, 2024-08-31T06:23:32.074 INFO:teuthology.orchestra.run.smithi016.stdout: "command_resend": 0, 2024-08-31T06:23:32.074 INFO:teuthology.orchestra.run.smithi016.stdout: "map_epoch": 103, 2024-08-31T06:23:32.074 INFO:teuthology.orchestra.run.smithi016.stdout: "map_full": 0, 2024-08-31T06:23:32.074 INFO:teuthology.orchestra.run.smithi016.stdout: "map_inc": 25, 2024-08-31T06:23:32.074 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_sessions": 12, 2024-08-31T06:23:32.075 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_open": 12, 2024-08-31T06:23:32.075 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_close": 0, 2024-08-31T06:23:32.075 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_laggy": 0, 2024-08-31T06:23:32.075 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_wr": 368, 2024-08-31T06:23:32.075 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_rd": 25, 2024-08-31T06:23:32.075 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_del": 111 2024-08-31T06:23:32.075 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.075 INFO:teuthology.orchestra.run.smithi016.stdout: "oft": { 2024-08-31T06:23:32.075 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_objs": 1, 2024-08-31T06:23:32.075 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_kv_pairs": 0, 2024-08-31T06:23:32.075 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_updates": 0, 2024-08-31T06:23:32.075 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_removes": 0 2024-08-31T06:23:32.075 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.075 INFO:teuthology.orchestra.run.smithi016.stdout: "purge_queue": { 2024-08-31T06:23:32.076 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops": 0, 2024-08-31T06:23:32.076 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops_high_water": 10, 2024-08-31T06:23:32.076 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing": 0, 2024-08-31T06:23:32.076 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_high_water": 10, 2024-08-31T06:23:32.076 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed_ops": 217, 2024-08-31T06:23:32.076 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed": 217, 2024-08-31T06:23:32.076 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_item_in_journal": 0 2024-08-31T06:23:32.076 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.076 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-31T06:23:32.076 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-08-31T06:23:32.076 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-08-31T06:23:32.076 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-08-31T06:23:32.076 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 18941, 2024-08-31T06:23:32.076 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 7344485, 2024-08-31T06:23:32.077 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-08-31T06:23:32.077 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 18941, 2024-08-31T06:23:32.077 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-08-31T06:23:32.077 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-08-31T06:23:32.077 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 18941, 2024-08-31T06:23:32.077 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 7344485, 2024-08-31T06:23:32.077 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-08-31T06:23:32.077 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-08-31T06:23:32.077 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-08-31T06:23:32.077 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-08-31T06:23:32.077 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:32.077 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.077 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_bytes": { 2024-08-31T06:23:32.077 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-08-31T06:23:32.078 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-08-31T06:23:32.078 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-08-31T06:23:32.078 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-08-31T06:23:32.078 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-08-31T06:23:32.078 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-08-31T06:23:32.078 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-08-31T06:23:32.078 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 5246, 2024-08-31T06:23:32.078 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 7323214, 2024-08-31T06:23:32.078 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 5246, 2024-08-31T06:23:32.078 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 7323214, 2024-08-31T06:23:32.078 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-08-31T06:23:32.078 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-08-31T06:23:32.078 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-08-31T06:23:32.078 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-08-31T06:23:32.078 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:32.079 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.079 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_ops": { 2024-08-31T06:23:32.079 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-08-31T06:23:32.079 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 1024, 2024-08-31T06:23:32.079 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-08-31T06:23:32.079 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-08-31T06:23:32.079 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-08-31T06:23:32.079 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-08-31T06:23:32.079 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-08-31T06:23:32.079 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 5246, 2024-08-31T06:23:32.080 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 5246, 2024-08-31T06:23:32.080 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 5246, 2024-08-31T06:23:32.080 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 5246, 2024-08-31T06:23:32.080 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-08-31T06:23:32.080 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-08-31T06:23:32.080 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-08-31T06:23:32.080 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-08-31T06:23:32.080 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:32.080 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.080 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle": { 2024-08-31T06:23:32.080 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-08-31T06:23:32.080 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-08-31T06:23:32.080 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-08-31T06:23:32.081 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 217, 2024-08-31T06:23:32.081 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 22957, 2024-08-31T06:23:32.081 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-08-31T06:23:32.081 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 217, 2024-08-31T06:23:32.081 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-08-31T06:23:32.081 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-08-31T06:23:32.081 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 84, 2024-08-31T06:23:32.081 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 22957, 2024-08-31T06:23:32.081 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-08-31T06:23:32.081 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-08-31T06:23:32.081 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-08-31T06:23:32.081 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-08-31T06:23:32.081 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:32.081 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.081 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle-0x56369aa7a5b0": { 2024-08-31T06:23:32.082 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-08-31T06:23:32.082 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-08-31T06:23:32.082 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-08-31T06:23:32.082 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 2834, 2024-08-31T06:23:32.082 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 7165639, 2024-08-31T06:23:32.082 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-08-31T06:23:32.082 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 2834, 2024-08-31T06:23:32.082 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-08-31T06:23:32.082 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-08-31T06:23:32.082 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 1113, 2024-08-31T06:23:32.082 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 7165639, 2024-08-31T06:23:32.082 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-08-31T06:23:32.082 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-08-31T06:23:32.082 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-08-31T06:23:32.082 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-08-31T06:23:32.083 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:32.083 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:32.083 INFO:teuthology.orchestra.run.smithi016.stdout: "trackedop": { 2024-08-31T06:23:32.083 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_ops_count": 4033 2024-08-31T06:23:32.083 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:32.083 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:23:33.045 INFO:tasks.check_counter:Daemon mds.g mds.exported=17 2024-08-31T06:23:33.046 INFO:tasks.check_counter:Daemon mds.g mds.imported=42 2024-08-31T06:23:33.046 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=52 2024-08-31T06:23:33.046 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=276 2024-08-31T06:23:33.046 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=6 2024-08-31T06:23:33.046 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-08-31T06:23:33.046 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-08-31T06:23:33.046 DEBUG:tasks.check_counter:Getting stats from j 2024-08-31T06:23:33.046 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph daemon mds.j perf dump 2024-08-31T06:23:33.095 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:32 smithi016 ceph-mon[17194]: pgmap v875: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-08-31T06:23:33.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:32 smithi155 ceph-mon[21742]: pgmap v875: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-08-31T06:23:33.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:32 smithi184 ceph-mon[22331]: pgmap v875: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-08-31T06:23:33.424 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:23:34.132 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-31T06:23:34.132 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-08-31T06:23:34.132 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 325, 2024-08-31T06:23:34.132 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 318, 2024-08-31T06:23:34.132 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 859712, 2024-08-31T06:23:34.133 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 196048, 2024-08-31T06:23:34.133 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-08-31T06:23:34.133 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 0, 2024-08-31T06:23:34.133 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.100286651, 2024-08-31T06:23:34.133 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.052303940, 2024-08-31T06:23:34.133 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.061142384, 2024-08-31T06:23:34.133 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-31T06:23:34.133 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T06:23:34.133 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 318, 2024-08-31T06:23:34.133 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.075110974, 2024-08-31T06:23:34.133 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000236198 2024-08-31T06:23:34.133 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:34.133 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-08-31T06:23:34.134 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-08-31T06:23:34.134 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-08-31T06:23:34.134 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-08-31T06:23:34.134 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:34.134 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 859712, 2024-08-31T06:23:34.134 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 196048 2024-08-31T06:23:34.134 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:34.134 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-08-31T06:23:34.134 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 2, 2024-08-31T06:23:34.134 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 243, 2024-08-31T06:23:34.134 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 320, 2024-08-31T06:23:34.134 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 283280, 2024-08-31T06:23:34.134 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 5, 2024-08-31T06:23:34.134 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 2, 2024-08-31T06:23:34.135 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.039602728, 2024-08-31T06:23:34.135 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.033096282, 2024-08-31T06:23:34.135 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.002709965, 2024-08-31T06:23:34.135 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-31T06:23:34.135 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T06:23:34.135 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 241, 2024-08-31T06:23:34.135 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.036514404, 2024-08-31T06:23:34.135 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000151512 2024-08-31T06:23:34.135 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:34.135 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-08-31T06:23:34.135 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-08-31T06:23:34.135 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-08-31T06:23:34.135 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-08-31T06:23:34.135 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:34.136 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 320, 2024-08-31T06:23:34.136 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 283280 2024-08-31T06:23:34.136 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:34.136 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-08-31T06:23:34.136 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 0, 2024-08-31T06:23:34.136 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 2, 2024-08-31T06:23:34.136 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 0, 2024-08-31T06:23:34.136 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 0, 2024-08-31T06:23:34.136 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 2, 2024-08-31T06:23:34.136 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 1, 2024-08-31T06:23:34.136 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.000812485, 2024-08-31T06:23:34.136 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.000000000, 2024-08-31T06:23:34.136 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.000643712, 2024-08-31T06:23:34.136 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-31T06:23:34.136 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T06:23:34.137 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-08-31T06:23:34.137 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-08-31T06:23:34.137 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-08-31T06:23:34.137 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:34.137 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-08-31T06:23:34.137 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-08-31T06:23:34.137 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-08-31T06:23:34.137 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-08-31T06:23:34.137 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:34.137 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-31T06:23:34.137 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 0 2024-08-31T06:23:34.137 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:34.137 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-08-31T06:23:34.137 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 0, 2024-08-31T06:23:34.138 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-08-31T06:23:34.138 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:34.138 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-08-31T06:23:34.138 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-08-31T06:23:34.138 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-08-31T06:23:34.138 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-08-31T06:23:34.138 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-08-31T06:23:34.138 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-08-31T06:23:34.138 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-08-31T06:23:34.138 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-31T06:23:34.138 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-08-31T06:23:34.138 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-31T06:23:34.138 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-08-31T06:23:34.138 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-08-31T06:23:34.139 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-08-31T06:23:34.139 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-31T06:23:34.139 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-08-31T06:23:34.139 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-08-31T06:23:34.139 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-08-31T06:23:34.139 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-08-31T06:23:34.139 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-08-31T06:23:34.139 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-31T06:23:34.139 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-08-31T06:23:34.139 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-31T06:23:34.139 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-08-31T06:23:34.139 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-08-31T06:23:34.139 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-08-31T06:23:34.139 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-08-31T06:23:34.140 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-08-31T06:23:34.140 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-31T06:23:34.140 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-08-31T06:23:34.140 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-08-31T06:23:34.140 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-08-31T06:23:34.140 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-08-31T06:23:34.140 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-08-31T06:23:34.140 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-08-31T06:23:34.141 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-08-31T06:23:34.141 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-08-31T06:23:34.141 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-08-31T06:23:34.141 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 148721, 2024-08-31T06:23:34.141 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 30, 2024-08-31T06:23:34.141 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-08-31T06:23:34.141 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-08-31T06:23:34.141 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-08-31T06:23:34.141 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-08-31T06:23:34.141 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-08-31T06:23:34.141 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-08-31T06:23:34.141 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-08-31T06:23:34.141 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-08-31T06:23:34.141 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 0, 2024-08-31T06:23:34.142 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 0, 2024-08-31T06:23:34.142 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-08-31T06:23:34.142 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-08-31T06:23:34.142 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-08-31T06:23:34.142 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-08-31T06:23:34.142 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 928, 2024-08-31T06:23:34.142 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 2, 2024-08-31T06:23:34.142 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-08-31T06:23:34.142 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-08-31T06:23:34.142 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-08-31T06:23:34.142 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-08-31T06:23:34.142 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-31T06:23:34.142 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-31T06:23:34.142 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-08-31T06:23:34.142 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-08-31T06:23:34.143 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-08-31T06:23:34.143 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 323, 2024-08-31T06:23:34.143 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 811338, 2024-08-31T06:23:34.143 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-08-31T06:23:34.143 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 323, 2024-08-31T06:23:34.143 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-08-31T06:23:34.143 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-08-31T06:23:34.143 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 323, 2024-08-31T06:23:34.143 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 811338, 2024-08-31T06:23:34.143 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-08-31T06:23:34.143 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-08-31T06:23:34.143 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-08-31T06:23:34.143 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-08-31T06:23:34.143 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:34.144 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-31T06:23:34.144 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-31T06:23:35.023 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-08-31T06:23:35.023 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-08-31T06:23:35.023 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-08-31T06:23:35.023 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-08-31T06:23:35.023 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-08-31T06:23:35.023 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-08-31T06:23:35.024 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-08-31T06:23:35.024 DEBUG:tasks.check_counter:Getting stats from b 2024-08-31T06:23:35.024 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph daemon mds.b perf dump 2024-08-31T06:23:35.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:34 smithi155 ceph-mon[21742]: pgmap v876: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-08-31T06:23:35.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:34 smithi016 ceph-mon[17194]: pgmap v876: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-08-31T06:23:35.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:34 smithi184 ceph-mon[22331]: pgmap v876: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-08-31T06:23:35.284 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.b/config 2024-08-31T06:23:35.825 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-08-31T06:23:35.825 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-0": { 2024-08-31T06:23:35.826 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 11778, 2024-08-31T06:23:35.826 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 12800, 2024-08-31T06:23:35.826 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 5019879, 2024-08-31T06:23:35.826 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 19022902, 2024-08-31T06:23:35.826 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 14, 2024-08-31T06:23:35.826 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 4, 2024-08-31T06:23:35.826 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 1.057924372, 2024-08-31T06:23:35.826 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.474809138, 2024-08-31T06:23:35.826 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.395115170, 2024-08-31T06:23:35.826 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.185424841, 2024-08-31T06:23:35.826 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T06:23:35.826 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 12801, 2024-08-31T06:23:35.827 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 1.653015596, 2024-08-31T06:23:35.827 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000129131 2024-08-31T06:23:35.827 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.827 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-08-31T06:23:35.827 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 11509, 2024-08-31T06:23:35.827 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.002464747, 2024-08-31T06:23:35.827 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000214 2024-08-31T06:23:35.827 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.827 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 534272, 2024-08-31T06:23:35.827 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 172928 2024-08-31T06:23:35.827 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.827 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-1": { 2024-08-31T06:23:35.828 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 12770, 2024-08-31T06:23:35.828 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 15465, 2024-08-31T06:23:35.828 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 4869800, 2024-08-31T06:23:35.828 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 28708837, 2024-08-31T06:23:35.828 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 352, 2024-08-31T06:23:35.828 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 6, 2024-08-31T06:23:35.828 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 1.139807377, 2024-08-31T06:23:35.828 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.503948041, 2024-08-31T06:23:35.828 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.477945162, 2024-08-31T06:23:35.828 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.177091221, 2024-08-31T06:23:35.828 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T06:23:35.828 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 15464, 2024-08-31T06:23:35.828 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 1.397475940, 2024-08-31T06:23:35.828 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000090369 2024-08-31T06:23:35.829 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.829 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-08-31T06:23:35.829 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 8839, 2024-08-31T06:23:35.829 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.002762741, 2024-08-31T06:23:35.829 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000312 2024-08-31T06:23:35.829 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.829 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 65024, 2024-08-31T06:23:35.829 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 546880 2024-08-31T06:23:35.829 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.829 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-2": { 2024-08-31T06:23:35.829 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 7218, 2024-08-31T06:23:35.829 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 9487, 2024-08-31T06:23:35.830 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 3970463, 2024-08-31T06:23:35.830 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 9393074, 2024-08-31T06:23:35.830 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 12, 2024-08-31T06:23:35.830 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 7, 2024-08-31T06:23:35.830 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.693918467, 2024-08-31T06:23:35.830 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.346733751, 2024-08-31T06:23:35.830 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.294315931, 2024-08-31T06:23:35.830 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.075029043, 2024-08-31T06:23:35.830 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T06:23:35.830 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 9483, 2024-08-31T06:23:35.830 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 1.056421626, 2024-08-31T06:23:35.830 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000111401 2024-08-31T06:23:35.830 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.831 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-08-31T06:23:35.831 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 8932, 2024-08-31T06:23:35.831 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.001935298, 2024-08-31T06:23:35.831 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000216 2024-08-31T06:23:35.831 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.831 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 1051600, 2024-08-31T06:23:35.831 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 232000 2024-08-31T06:23:35.831 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.831 INFO:teuthology.orchestra.run.smithi155.stdout: "cct": { 2024-08-31T06:23:35.831 INFO:teuthology.orchestra.run.smithi155.stdout: "total_workers": 1, 2024-08-31T06:23:35.831 INFO:teuthology.orchestra.run.smithi155.stdout: "unhealthy_workers": 0 2024-08-31T06:23:35.831 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.831 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-MDSRank": { 2024-08-31T06:23:35.831 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-08-31T06:23:35.831 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-08-31T06:23:35.832 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 8947, 2024-08-31T06:23:35.832 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 7.442486531, 2024-08-31T06:23:35.832 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000831841 2024-08-31T06:23:35.832 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-31T06:23:35.832 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.832 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-PurgeQueue": { 2024-08-31T06:23:35.832 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-08-31T06:23:35.832 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-08-31T06:23:35.832 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 125, 2024-08-31T06:23:35.832 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.006589939, 2024-08-31T06:23:35.832 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000052719 2024-08-31T06:23:35.832 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-31T06:23:35.832 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.832 INFO:teuthology.orchestra.run.smithi155.stdout: "mds": { 2024-08-31T06:23:35.833 INFO:teuthology.orchestra.run.smithi155.stdout: "request": 3111, 2024-08-31T06:23:35.833 INFO:teuthology.orchestra.run.smithi155.stdout: "reply": 2893, 2024-08-31T06:23:35.833 INFO:teuthology.orchestra.run.smithi155.stdout: "reply_latency": { 2024-08-31T06:23:35.833 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 2893, 2024-08-31T06:23:35.833 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 23.988384080, 2024-08-31T06:23:35.833 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.008291871 2024-08-31T06:23:35.833 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.833 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_reply": 0, 2024-08-31T06:23:35.833 INFO:teuthology.orchestra.run.smithi155.stdout: "forward": 218, 2024-08-31T06:23:35.833 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_complete": 23, 2024-08-31T06:23:35.833 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_keys": 11, 2024-08-31T06:23:35.833 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_commit": 1033, 2024-08-31T06:23:35.833 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_split": 5, 2024-08-31T06:23:35.834 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_merge": 0, 2024-08-31T06:23:35.834 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes": 594, 2024-08-31T06:23:35.834 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_top": 32, 2024-08-31T06:23:35.834 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_bottom": 551, 2024-08-31T06:23:35.834 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pin_tail": 11, 2024-08-31T06:23:35.834 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pinned": 548, 2024-08-31T06:23:35.834 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_expired": 1491, 2024-08-31T06:23:35.834 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_with_caps": 475, 2024-08-31T06:23:35.834 INFO:teuthology.orchestra.run.smithi155.stdout: "caps": 475, 2024-08-31T06:23:35.834 INFO:teuthology.orchestra.run.smithi155.stdout: "subtrees": 66, 2024-08-31T06:23:35.834 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse": 14261, 2024-08-31T06:23:35.834 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_hit": 13169, 2024-08-31T06:23:35.834 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_forward": 190, 2024-08-31T06:23:35.835 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_discover": 4, 2024-08-31T06:23:35.835 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_dir_fetch": 0, 2024-08-31T06:23:35.835 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_remote_ino": 0, 2024-08-31T06:23:35.835 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_lock": 146, 2024-08-31T06:23:35.835 INFO:teuthology.orchestra.run.smithi155.stdout: "load_cent": 219, 2024-08-31T06:23:35.835 INFO:teuthology.orchestra.run.smithi155.stdout: "q": 0, 2024-08-31T06:23:35.835 INFO:teuthology.orchestra.run.smithi155.stdout: "exported": 44, 2024-08-31T06:23:35.835 INFO:teuthology.orchestra.run.smithi155.stdout: "exported_inodes": 1417, 2024-08-31T06:23:35.835 INFO:teuthology.orchestra.run.smithi155.stdout: "imported": 15, 2024-08-31T06:23:35.835 INFO:teuthology.orchestra.run.smithi155.stdout: "imported_inodes": 760, 2024-08-31T06:23:35.835 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_dir_fetch": 0, 2024-08-31T06:23:35.835 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_backtrace_fetch": 7449, 2024-08-31T06:23:35.835 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_peer_discover": 0, 2024-08-31T06:23:35.835 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rfiles": 1000, 2024-08-31T06:23:35.836 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rbytes": 1239684560, 2024-08-31T06:23:35.836 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rsnaps": 6, 2024-08-31T06:23:35.836 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_fetch": 7449, 2024-08-31T06:23:35.836 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_set_tag": 0, 2024-08-31T06:23:35.836 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_repaired": 0, 2024-08-31T06:23:35.836 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_inotable_repaired": 0, 2024-08-31T06:23:35.836 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_inodes": 549, 2024-08-31T06:23:35.836 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_base_inodes": 19, 2024-08-31T06:23:35.836 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dirfrag_rstats": 530, 2024-08-31T06:23:35.836 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_file_inodes": 6900, 2024-08-31T06:23:35.836 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_inode_file_caps": 38, 2024-08-31T06:23:35.836 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_revoke": 1529, 2024-08-31T06:23:35.836 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_grant": 2270, 2024-08-31T06:23:35.836 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_trunc": 0, 2024-08-31T06:23:35.836 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flushsnap_ack": 1, 2024-08-31T06:23:35.837 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-31T06:23:35.837 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps": 3089, 2024-08-31T06:23:35.837 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps_dirty": 1238, 2024-08-31T06:23:35.837 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_cap_release": 73, 2024-08-31T06:23:35.837 INFO:teuthology.orchestra.run.smithi155.stdout: "process_request_cap_release": 510 2024-08-31T06:23:35.837 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.837 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_cache": { 2024-08-31T06:23:35.837 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update": 91, 2024-08-31T06:23:35.837 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update_receipt": 6, 2024-08-31T06:23:35.837 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_try_discover": 4, 2024-08-31T06:23:35.837 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_send_discover": 16, 2024-08-31T06:23:35.837 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_handle_discover": 282, 2024-08-31T06:23:35.837 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays": 0, 2024-08-31T06:23:35.837 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_delayed": 0, 2024-08-31T06:23:35.837 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_enqueuing": 0, 2024-08-31T06:23:35.838 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_created": 112, 2024-08-31T06:23:35.838 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_enqueued": 25, 2024-08-31T06:23:35.838 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_reintegrated": 87, 2024-08-31T06:23:35.838 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_migrated": 0, 2024-08-31T06:23:35.838 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_processing": 0, 2024-08-31T06:23:35.838 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_enqueued": 0, 2024-08-31T06:23:35.838 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_prioritized": 0, 2024-08-31T06:23:35.838 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_started": 0, 2024-08-31T06:23:35.838 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_completed": 0, 2024-08-31T06:23:35.838 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_quiesce_path": 12, 2024-08-31T06:23:35.838 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_quiesce_inode": 4883, 2024-08-31T06:23:35.838 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_enqueue_scrub": 19, 2024-08-31T06:23:35.838 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_exportdir": 44, 2024-08-31T06:23:35.838 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_flush": 0, 2024-08-31T06:23:35.838 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragmentdir": 5, 2024-08-31T06:23:35.839 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragstats": 0, 2024-08-31T06:23:35.839 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_inodestats": 0, 2024-08-31T06:23:35.839 INFO:teuthology.orchestra.run.smithi155.stdout: "uninline_started": 0, 2024-08-31T06:23:35.839 INFO:teuthology.orchestra.run.smithi155.stdout: "uninline_succeeded": 0, 2024-08-31T06:23:35.839 INFO:teuthology.orchestra.run.smithi155.stdout: "uninline_write_failed": 0 2024-08-31T06:23:35.839 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.839 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_log": { 2024-08-31T06:23:35.839 INFO:teuthology.orchestra.run.smithi155.stdout: "evlrg": 0, 2024-08-31T06:23:35.839 INFO:teuthology.orchestra.run.smithi155.stdout: "evadd": 4965, 2024-08-31T06:23:35.839 INFO:teuthology.orchestra.run.smithi155.stdout: "evex": 4880, 2024-08-31T06:23:35.839 INFO:teuthology.orchestra.run.smithi155.stdout: "evtrm": 4880, 2024-08-31T06:23:35.839 INFO:teuthology.orchestra.run.smithi155.stdout: "ev": 85, 2024-08-31T06:23:35.839 INFO:teuthology.orchestra.run.smithi155.stdout: "evexg": 0, 2024-08-31T06:23:35.839 INFO:teuthology.orchestra.run.smithi155.stdout: "evexd": 22, 2024-08-31T06:23:35.840 INFO:teuthology.orchestra.run.smithi155.stdout: "segadd": 79, 2024-08-31T06:23:35.840 INFO:teuthology.orchestra.run.smithi155.stdout: "segex": 78, 2024-08-31T06:23:35.840 INFO:teuthology.orchestra.run.smithi155.stdout: "segtrm": 78, 2024-08-31T06:23:35.840 INFO:teuthology.orchestra.run.smithi155.stdout: "seg": 1, 2024-08-31T06:23:35.840 INFO:teuthology.orchestra.run.smithi155.stdout: "segmjr": 1, 2024-08-31T06:23:35.840 INFO:teuthology.orchestra.run.smithi155.stdout: "segexg": 0, 2024-08-31T06:23:35.840 INFO:teuthology.orchestra.run.smithi155.stdout: "segexd": 1, 2024-08-31T06:23:35.840 INFO:teuthology.orchestra.run.smithi155.stdout: "expos": 35082754, 2024-08-31T06:23:35.840 INFO:teuthology.orchestra.run.smithi155.stdout: "wrpos": 36111834, 2024-08-31T06:23:35.840 INFO:teuthology.orchestra.run.smithi155.stdout: "rdpos": 0, 2024-08-31T06:23:35.840 INFO:teuthology.orchestra.run.smithi155.stdout: "jlat": { 2024-08-31T06:23:35.840 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 3143, 2024-08-31T06:23:35.840 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 32.615105036, 2024-08-31T06:23:35.841 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.010377061 2024-08-31T06:23:35.841 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.841 INFO:teuthology.orchestra.run.smithi155.stdout: "replayed": 0 2024-08-31T06:23:35.841 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.841 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_mem": { 2024-08-31T06:23:35.841 INFO:teuthology.orchestra.run.smithi155.stdout: "ino": 588, 2024-08-31T06:23:35.841 INFO:teuthology.orchestra.run.smithi155.stdout: "ino+": 2018, 2024-08-31T06:23:35.841 INFO:teuthology.orchestra.run.smithi155.stdout: "ino-": 1430, 2024-08-31T06:23:35.841 INFO:teuthology.orchestra.run.smithi155.stdout: "dir": 117, 2024-08-31T06:23:35.841 INFO:teuthology.orchestra.run.smithi155.stdout: "dir+": 164, 2024-08-31T06:23:35.841 INFO:teuthology.orchestra.run.smithi155.stdout: "dir-": 47, 2024-08-31T06:23:35.841 INFO:teuthology.orchestra.run.smithi155.stdout: "dn": 594, 2024-08-31T06:23:35.841 INFO:teuthology.orchestra.run.smithi155.stdout: "dn+": 2478, 2024-08-31T06:23:35.841 INFO:teuthology.orchestra.run.smithi155.stdout: "dn-": 1884, 2024-08-31T06:23:35.842 INFO:teuthology.orchestra.run.smithi155.stdout: "cap": 475, 2024-08-31T06:23:35.842 INFO:teuthology.orchestra.run.smithi155.stdout: "cap+": 1687, 2024-08-31T06:23:35.842 INFO:teuthology.orchestra.run.smithi155.stdout: "cap-": 1212, 2024-08-31T06:23:35.842 INFO:teuthology.orchestra.run.smithi155.stdout: "rss": 141268, 2024-08-31T06:23:35.842 INFO:teuthology.orchestra.run.smithi155.stdout: "heap": 366886912 2024-08-31T06:23:35.842 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.842 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_server": { 2024-08-31T06:23:35.842 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_client_request": 3803, 2024-08-31T06:23:35.842 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_server_request": 88, 2024-08-31T06:23:35.842 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_request": 3111, 2024-08-31T06:23:35.842 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_session": 330, 2024-08-31T06:23:35.842 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_peer_request": 176, 2024-08-31T06:23:35.842 INFO:teuthology.orchestra.run.smithi155.stdout: "req_create_latency": { 2024-08-31T06:23:35.842 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 1040, 2024-08-31T06:23:35.843 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 2.085710279, 2024-08-31T06:23:35.843 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.002005490 2024-08-31T06:23:35.843 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.843 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getattr_latency": { 2024-08-31T06:23:35.843 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 437, 2024-08-31T06:23:35.843 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 17.991645257, 2024-08-31T06:23:35.843 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.041170812 2024-08-31T06:23:35.843 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.843 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getfilelock_latency": { 2024-08-31T06:23:35.843 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:35.843 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:35.843 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:35.843 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.843 INFO:teuthology.orchestra.run.smithi155.stdout: "req_link_latency": { 2024-08-31T06:23:35.843 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 87, 2024-08-31T06:23:35.844 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.327019502, 2024-08-31T06:23:35.844 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.003758844 2024-08-31T06:23:35.844 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.844 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookup_latency": { 2024-08-31T06:23:35.844 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 485, 2024-08-31T06:23:35.844 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 1.211733046, 2024-08-31T06:23:35.844 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.002498418 2024-08-31T06:23:35.844 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.844 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookuphash_latency": { 2024-08-31T06:23:35.844 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:35.844 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:35.844 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:35.844 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.845 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupino_latency": { 2024-08-31T06:23:35.845 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 1, 2024-08-31T06:23:35.845 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000224226, 2024-08-31T06:23:35.845 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000224226 2024-08-31T06:23:35.845 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.845 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupname_latency": { 2024-08-31T06:23:35.845 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:35.845 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:35.845 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:35.845 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.845 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupparent_latency": { 2024-08-31T06:23:35.845 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:35.845 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:35.845 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:35.845 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.846 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupsnap_latency": { 2024-08-31T06:23:35.846 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:35.846 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:35.846 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:35.846 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.846 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lssnap_latency": { 2024-08-31T06:23:35.846 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 17, 2024-08-31T06:23:35.847 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.003019721, 2024-08-31T06:23:35.847 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000177630 2024-08-31T06:23:35.847 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.847 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mkdir_latency": { 2024-08-31T06:23:35.847 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 37, 2024-08-31T06:23:35.847 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.526804463, 2024-08-31T06:23:35.847 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.014237958 2024-08-31T06:23:35.847 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.847 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mknod_latency": { 2024-08-31T06:23:35.847 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:35.847 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:35.847 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:35.847 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.847 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mksnap_latency": { 2024-08-31T06:23:35.848 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 17, 2024-08-31T06:23:35.848 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.194824403, 2024-08-31T06:23:35.848 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.011460259 2024-08-31T06:23:35.848 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.848 INFO:teuthology.orchestra.run.smithi155.stdout: "req_open_latency": { 2024-08-31T06:23:35.848 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 480, 2024-08-31T06:23:35.848 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.154010829, 2024-08-31T06:23:35.848 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000320855 2024-08-31T06:23:35.848 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.848 INFO:teuthology.orchestra.run.smithi155.stdout: "req_readdir_latency": { 2024-08-31T06:23:35.849 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 60, 2024-08-31T06:23:35.849 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.099957507, 2024-08-31T06:23:35.849 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.001665958 2024-08-31T06:23:35.849 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.849 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rename_latency": { 2024-08-31T06:23:35.849 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 113, 2024-08-31T06:23:35.849 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 1.063623953, 2024-08-31T06:23:35.849 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.009412601 2024-08-31T06:23:35.849 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.849 INFO:teuthology.orchestra.run.smithi155.stdout: "req_renamesnap_latency": { 2024-08-31T06:23:35.849 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:35.849 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:35.849 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:35.849 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.850 INFO:teuthology.orchestra.run.smithi155.stdout: "req_snapdiff_latency": { 2024-08-31T06:23:35.850 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:35.850 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:35.850 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:35.850 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.850 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmdir_latency": { 2024-08-31T06:23:35.850 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 2, 2024-08-31T06:23:35.850 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.013869569, 2024-08-31T06:23:35.850 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.006934784 2024-08-31T06:23:35.850 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.850 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmsnap_latency": { 2024-08-31T06:23:35.850 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 11, 2024-08-31T06:23:35.850 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.060639552, 2024-08-31T06:23:35.851 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.005512686 2024-08-31T06:23:35.851 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.851 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmxattr_latency": { 2024-08-31T06:23:35.851 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:35.851 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:35.851 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:35.851 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.851 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setattr_latency": { 2024-08-31T06:23:35.851 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 1, 2024-08-31T06:23:35.851 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.001455173, 2024-08-31T06:23:35.851 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.001455173 2024-08-31T06:23:35.851 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.851 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setdirlayout_latency": { 2024-08-31T06:23:35.851 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:35.852 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:35.852 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:35.852 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.852 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setfilelock_latency": { 2024-08-31T06:23:35.852 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:35.852 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:35.852 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:35.852 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.852 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setlayout_latency": { 2024-08-31T06:23:35.852 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:35.852 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:35.852 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:35.852 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.853 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setxattr_latency": { 2024-08-31T06:23:35.853 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 7, 2024-08-31T06:23:35.853 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.020801442, 2024-08-31T06:23:35.853 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.002971634 2024-08-31T06:23:35.853 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.853 INFO:teuthology.orchestra.run.smithi155.stdout: "req_symlink_latency": { 2024-08-31T06:23:35.853 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:35.853 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:35.853 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:35.853 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.853 INFO:teuthology.orchestra.run.smithi155.stdout: "req_unlink_latency": { 2024-08-31T06:23:35.853 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 98, 2024-08-31T06:23:35.854 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.233045158, 2024-08-31T06:23:35.854 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.002378011 2024-08-31T06:23:35.854 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.854 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_revoke_eviction": 0, 2024-08-31T06:23:35.854 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_acquisition_throttle": 0, 2024-08-31T06:23:35.854 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getvxattr_latency": { 2024-08-31T06:23:35.854 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:35.854 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:35.854 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:35.854 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-31T06:23:35.854 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.854 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_sessions": { 2024-08-31T06:23:35.854 INFO:teuthology.orchestra.run.smithi155.stdout: "session_count": 2, 2024-08-31T06:23:35.854 INFO:teuthology.orchestra.run.smithi155.stdout: "session_add": 3, 2024-08-31T06:23:35.854 INFO:teuthology.orchestra.run.smithi155.stdout: "session_remove": 1, 2024-08-31T06:23:35.855 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_open": 2, 2024-08-31T06:23:35.855 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_stale": 0, 2024-08-31T06:23:35.855 INFO:teuthology.orchestra.run.smithi155.stdout: "total_load": 78, 2024-08-31T06:23:35.855 INFO:teuthology.orchestra.run.smithi155.stdout: "average_load": 39, 2024-08-31T06:23:35.855 INFO:teuthology.orchestra.run.smithi155.stdout: "avg_session_uptime": 1004, 2024-08-31T06:23:35.855 INFO:teuthology.orchestra.run.smithi155.stdout: "mdthresh_evicted": 0 2024-08-31T06:23:35.855 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.855 INFO:teuthology.orchestra.run.smithi155.stdout: "mempool": { 2024-08-31T06:23:35.855 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_bytes": 0, 2024-08-31T06:23:35.855 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_items": 0, 2024-08-31T06:23:35.855 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_bytes": 0, 2024-08-31T06:23:35.855 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_items": 0, 2024-08-31T06:23:35.855 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_bytes": 0, 2024-08-31T06:23:35.856 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_items": 0, 2024-08-31T06:23:35.856 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-31T06:23:35.856 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_items": 0, 2024-08-31T06:23:35.856 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-31T06:23:35.856 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_items": 0, 2024-08-31T06:23:35.856 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_bytes": 0, 2024-08-31T06:23:35.856 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_items": 0, 2024-08-31T06:23:35.856 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-31T06:23:35.856 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_items": 0, 2024-08-31T06:23:35.856 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_bytes": 0, 2024-08-31T06:23:35.856 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_items": 0, 2024-08-31T06:23:35.856 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_bytes": 0, 2024-08-31T06:23:35.856 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_items": 0, 2024-08-31T06:23:35.856 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-31T06:23:35.857 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_items": 0, 2024-08-31T06:23:35.857 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-31T06:23:35.857 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_items": 0, 2024-08-31T06:23:35.857 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_bytes": 0, 2024-08-31T06:23:35.857 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_items": 0, 2024-08-31T06:23:35.857 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_bytes": 0, 2024-08-31T06:23:35.857 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_items": 0, 2024-08-31T06:23:35.857 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-31T06:23:35.857 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_items": 0, 2024-08-31T06:23:35.857 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_bytes": 0, 2024-08-31T06:23:35.857 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_items": 0, 2024-08-31T06:23:35.857 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_bytes": 0, 2024-08-31T06:23:35.857 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_items": 0, 2024-08-31T06:23:35.858 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_bytes": 0, 2024-08-31T06:23:35.858 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_items": 0, 2024-08-31T06:23:35.858 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_bytes": 0, 2024-08-31T06:23:35.858 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_items": 0, 2024-08-31T06:23:35.858 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_bytes": 338974, 2024-08-31T06:23:35.858 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_items": 653, 2024-08-31T06:23:35.858 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_bytes": 0, 2024-08-31T06:23:35.858 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_items": 0, 2024-08-31T06:23:35.858 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_bytes": 0, 2024-08-31T06:23:35.858 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_items": 0, 2024-08-31T06:23:35.858 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_bytes": 0, 2024-08-31T06:23:35.858 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_items": 0, 2024-08-31T06:23:35.858 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_bytes": 0, 2024-08-31T06:23:35.858 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_items": 0, 2024-08-31T06:23:35.858 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_bytes": 8064, 2024-08-31T06:23:35.859 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_items": 178, 2024-08-31T06:23:35.859 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_bytes": 0, 2024-08-31T06:23:35.859 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_items": 0, 2024-08-31T06:23:35.859 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_bytes": 0, 2024-08-31T06:23:35.859 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_items": 0, 2024-08-31T06:23:35.859 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_bytes": 2199721, 2024-08-31T06:23:35.859 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_items": 22555, 2024-08-31T06:23:35.859 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_bytes": 0, 2024-08-31T06:23:35.859 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_items": 0, 2024-08-31T06:23:35.859 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_bytes": 0, 2024-08-31T06:23:35.859 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_items": 0 2024-08-31T06:23:35.859 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.859 INFO:teuthology.orchestra.run.smithi155.stdout: "objecter": { 2024-08-31T06:23:35.859 INFO:teuthology.orchestra.run.smithi155.stdout: "op_active": 0, 2024-08-31T06:23:35.860 INFO:teuthology.orchestra.run.smithi155.stdout: "op_laggy": 0, 2024-08-31T06:23:35.860 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send": 14011, 2024-08-31T06:23:35.860 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send_bytes": 36905593, 2024-08-31T06:23:35.860 INFO:teuthology.orchestra.run.smithi155.stdout: "op_resend": 37, 2024-08-31T06:23:35.860 INFO:teuthology.orchestra.run.smithi155.stdout: "op_reply": 14011, 2024-08-31T06:23:35.860 INFO:teuthology.orchestra.run.smithi155.stdout: "op_latency": { 2024-08-31T06:23:35.860 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 14011, 2024-08-31T06:23:35.860 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 157.754532528, 2024-08-31T06:23:35.860 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.011259334 2024-08-31T06:23:35.860 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.860 INFO:teuthology.orchestra.run.smithi155.stdout: "op_inflight": 0, 2024-08-31T06:23:35.860 INFO:teuthology.orchestra.run.smithi155.stdout: "oplen_avg": { 2024-08-31T06:23:35.860 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 14011, 2024-08-31T06:23:35.860 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 18342 2024-08-31T06:23:35.861 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.861 INFO:teuthology.orchestra.run.smithi155.stdout: "op": 14011, 2024-08-31T06:23:35.861 INFO:teuthology.orchestra.run.smithi155.stdout: "op_r": 7550, 2024-08-31T06:23:35.861 INFO:teuthology.orchestra.run.smithi155.stdout: "op_w": 6461, 2024-08-31T06:23:35.861 INFO:teuthology.orchestra.run.smithi155.stdout: "op_rmw": 0, 2024-08-31T06:23:35.861 INFO:teuthology.orchestra.run.smithi155.stdout: "op_pg": 0, 2024-08-31T06:23:35.861 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_stat": 946, 2024-08-31T06:23:35.861 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_create": 1138, 2024-08-31T06:23:35.861 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_read": 33, 2024-08-31T06:23:35.861 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_write": 3157, 2024-08-31T06:23:35.861 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writefull": 269, 2024-08-31T06:23:35.861 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writesame": 0, 2024-08-31T06:23:35.861 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_append": 0, 2024-08-31T06:23:35.861 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_zero": 0, 2024-08-31T06:23:35.862 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_truncate": 4, 2024-08-31T06:23:35.862 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_delete": 566, 2024-08-31T06:23:35.862 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_mapext": 0, 2024-08-31T06:23:35.862 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_sparse_read": 0, 2024-08-31T06:23:35.862 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_clonerange": 0, 2024-08-31T06:23:35.862 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_getxattr": 7499, 2024-08-31T06:23:35.862 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_setxattr": 2276, 2024-08-31T06:23:35.862 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_cmpxattr": 0, 2024-08-31T06:23:35.862 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_rmxattr": 0, 2024-08-31T06:23:35.862 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_resetxattrs": 0, 2024-08-31T06:23:35.862 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_call": 0, 2024-08-31T06:23:35.862 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_watch": 0, 2024-08-31T06:23:35.862 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_notify": 0, 2024-08-31T06:23:35.862 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_src_cmpxattr": 0, 2024-08-31T06:23:35.863 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls": 0, 2024-08-31T06:23:35.863 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls_filter": 0, 2024-08-31T06:23:35.863 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_other": 0, 2024-08-31T06:23:35.863 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_active": 0, 2024-08-31T06:23:35.863 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_send": 0, 2024-08-31T06:23:35.863 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_resend": 0, 2024-08-31T06:23:35.863 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_ping": 0, 2024-08-31T06:23:35.863 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_active": 0, 2024-08-31T06:23:35.863 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_send": 0, 2024-08-31T06:23:35.863 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_resend": 0, 2024-08-31T06:23:35.863 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_active": 0, 2024-08-31T06:23:35.863 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_send": 0, 2024-08-31T06:23:35.863 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_resend": 0, 2024-08-31T06:23:35.864 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_active": 0, 2024-08-31T06:23:35.864 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_send": 0, 2024-08-31T06:23:35.864 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_resend": 0, 2024-08-31T06:23:35.864 INFO:teuthology.orchestra.run.smithi155.stdout: "command_active": 0, 2024-08-31T06:23:35.864 INFO:teuthology.orchestra.run.smithi155.stdout: "command_send": 0, 2024-08-31T06:23:35.864 INFO:teuthology.orchestra.run.smithi155.stdout: "command_resend": 0, 2024-08-31T06:23:35.864 INFO:teuthology.orchestra.run.smithi155.stdout: "map_epoch": 103, 2024-08-31T06:23:35.864 INFO:teuthology.orchestra.run.smithi155.stdout: "map_full": 0, 2024-08-31T06:23:35.864 INFO:teuthology.orchestra.run.smithi155.stdout: "map_inc": 25, 2024-08-31T06:23:35.864 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_sessions": 12, 2024-08-31T06:23:35.864 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_open": 12, 2024-08-31T06:23:35.864 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_close": 0, 2024-08-31T06:23:35.864 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_laggy": 0, 2024-08-31T06:23:35.865 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_wr": 2260, 2024-08-31T06:23:35.865 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_rd": 83, 2024-08-31T06:23:35.865 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_del": 111 2024-08-31T06:23:35.865 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.865 INFO:teuthology.orchestra.run.smithi155.stdout: "oft": { 2024-08-31T06:23:35.865 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_objs": 1, 2024-08-31T06:23:35.865 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_kv_pairs": 0, 2024-08-31T06:23:35.865 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_updates": 0, 2024-08-31T06:23:35.865 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_removes": 0 2024-08-31T06:23:35.865 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.865 INFO:teuthology.orchestra.run.smithi155.stdout: "purge_queue": { 2024-08-31T06:23:35.865 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops": 0, 2024-08-31T06:23:35.865 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops_high_water": 16, 2024-08-31T06:23:35.865 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing": 0, 2024-08-31T06:23:35.866 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_high_water": 8, 2024-08-31T06:23:35.866 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed_ops": 39, 2024-08-31T06:23:35.866 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed": 25, 2024-08-31T06:23:35.866 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_item_in_journal": 0 2024-08-31T06:23:35.866 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.866 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-31T06:23:35.866 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-08-31T06:23:35.866 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-08-31T06:23:35.866 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-08-31T06:23:35.866 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 31762, 2024-08-31T06:23:35.866 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 11029726, 2024-08-31T06:23:35.866 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-08-31T06:23:35.866 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 31762, 2024-08-31T06:23:35.866 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-08-31T06:23:35.867 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-08-31T06:23:35.867 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 31762, 2024-08-31T06:23:35.867 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 11029726, 2024-08-31T06:23:35.867 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-08-31T06:23:35.867 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:35.867 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:35.867 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:35.867 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-31T06:23:35.867 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.867 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_bytes": { 2024-08-31T06:23:35.867 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-08-31T06:23:35.867 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-08-31T06:23:35.867 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-08-31T06:23:35.867 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-08-31T06:23:35.868 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-08-31T06:23:35.868 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-08-31T06:23:35.868 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-08-31T06:23:35.868 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 14011, 2024-08-31T06:23:35.868 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 36907298, 2024-08-31T06:23:35.868 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 14011, 2024-08-31T06:23:35.868 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 36907298, 2024-08-31T06:23:35.868 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-08-31T06:23:35.868 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:35.868 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:35.868 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:35.868 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-31T06:23:35.868 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.869 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_ops": { 2024-08-31T06:23:35.869 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-08-31T06:23:35.869 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 1024, 2024-08-31T06:23:35.869 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-08-31T06:23:35.869 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-08-31T06:23:35.869 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-08-31T06:23:35.869 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-08-31T06:23:35.869 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-08-31T06:23:35.869 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 14011, 2024-08-31T06:23:35.869 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 14011, 2024-08-31T06:23:35.869 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 14011, 2024-08-31T06:23:35.869 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 14011, 2024-08-31T06:23:35.869 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-08-31T06:23:35.869 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:35.870 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:35.870 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:35.870 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-31T06:23:35.870 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.870 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle": { 2024-08-31T06:23:35.870 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-08-31T06:23:35.870 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-08-31T06:23:35.870 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-08-31T06:23:35.870 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 25, 2024-08-31T06:23:35.870 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 2573, 2024-08-31T06:23:35.870 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-08-31T06:23:35.870 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 25, 2024-08-31T06:23:35.870 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-08-31T06:23:35.870 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-08-31T06:23:35.870 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 14, 2024-08-31T06:23:35.871 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 2573, 2024-08-31T06:23:35.871 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-08-31T06:23:35.871 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:35.871 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:35.871 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:35.871 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-31T06:23:35.871 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.871 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle-0x5584985d0750": { 2024-08-31T06:23:35.871 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-08-31T06:23:35.871 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-08-31T06:23:35.871 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-08-31T06:23:35.871 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 4965, 2024-08-31T06:23:35.871 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 31917530, 2024-08-31T06:23:35.871 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-08-31T06:23:35.872 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 4965, 2024-08-31T06:23:35.872 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-08-31T06:23:35.872 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-08-31T06:23:35.872 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 3143, 2024-08-31T06:23:35.872 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 31917530, 2024-08-31T06:23:35.872 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-08-31T06:23:35.872 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:35.872 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:35.872 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:35.872 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-31T06:23:35.872 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:35.872 INFO:teuthology.orchestra.run.smithi155.stdout: "trackedop": { 2024-08-31T06:23:35.872 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_ops_count": 5933 2024-08-31T06:23:35.872 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-31T06:23:35.873 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-08-31T06:23:36.778 INFO:tasks.check_counter:Daemon mds.b mds.exported=44 2024-08-31T06:23:36.778 INFO:tasks.check_counter:Daemon mds.b mds.imported=15 2024-08-31T06:23:36.778 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=91 2024-08-31T06:23:36.778 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=6 2024-08-31T06:23:36.778 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-08-31T06:23:36.779 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=17 2024-08-31T06:23:36.779 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=11 2024-08-31T06:23:36.779 DEBUG:tasks.check_counter:Getting stats from e 2024-08-31T06:23:36.779 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph daemon mds.e perf dump 2024-08-31T06:23:37.035 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.b/config 2024-08-31T06:23:37.094 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:36 smithi155 ceph-mon[21742]: pgmap v877: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-08-31T06:23:37.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:36 smithi016 ceph-mon[17194]: pgmap v877: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-08-31T06:23:37.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:36 smithi184 ceph-mon[22331]: pgmap v877: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-08-31T06:23:37.542 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-08-31T06:23:37.542 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-0": { 2024-08-31T06:23:37.543 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 4, 2024-08-31T06:23:37.543 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 4, 2024-08-31T06:23:37.543 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 2992, 2024-08-31T06:23:37.543 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 448, 2024-08-31T06:23:37.543 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 4, 2024-08-31T06:23:37.543 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 0, 2024-08-31T06:23:37.543 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.003038767, 2024-08-31T06:23:37.543 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.000046996, 2024-08-31T06:23:37.543 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.002083825, 2024-08-31T06:23:37.543 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-31T06:23:37.543 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T06:23:37.543 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 3, 2024-08-31T06:23:37.544 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.003730517, 2024-08-31T06:23:37.544 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.001243505 2024-08-31T06:23:37.544 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:37.544 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-08-31T06:23:37.544 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:37.544 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:37.544 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:37.544 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:37.544 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-08-31T06:23:37.544 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 448 2024-08-31T06:23:37.544 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:37.544 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-1": { 2024-08-31T06:23:37.544 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 2, 2024-08-31T06:23:37.544 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 242, 2024-08-31T06:23:37.545 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 320, 2024-08-31T06:23:37.545 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 202816, 2024-08-31T06:23:37.545 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 4, 2024-08-31T06:23:37.545 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 2, 2024-08-31T06:23:37.545 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.017710761, 2024-08-31T06:23:37.545 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.014430429, 2024-08-31T06:23:37.545 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.000981931, 2024-08-31T06:23:37.545 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-31T06:23:37.545 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T06:23:37.545 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 240, 2024-08-31T06:23:37.545 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.015965274, 2024-08-31T06:23:37.545 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000066521 2024-08-31T06:23:37.545 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:37.546 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-08-31T06:23:37.546 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:37.546 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:37.546 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:37.546 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:37.546 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 320, 2024-08-31T06:23:37.546 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 202816 2024-08-31T06:23:37.546 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:37.546 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-2": { 2024-08-31T06:23:37.546 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 320, 2024-08-31T06:23:37.547 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 315, 2024-08-31T06:23:37.547 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 856448, 2024-08-31T06:23:37.547 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 194960, 2024-08-31T06:23:37.547 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 2, 2024-08-31T06:23:37.547 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 1, 2024-08-31T06:23:37.547 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.052617263, 2024-08-31T06:23:37.547 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.025655022, 2024-08-31T06:23:37.547 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.036038650, 2024-08-31T06:23:37.547 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-31T06:23:37.547 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T06:23:37.547 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 314, 2024-08-31T06:23:37.547 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.016131755, 2024-08-31T06:23:37.547 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000051375 2024-08-31T06:23:37.547 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:37.548 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-08-31T06:23:37.548 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:37.548 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:37.548 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:37.548 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:37.548 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 856448, 2024-08-31T06:23:37.548 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 194960 2024-08-31T06:23:37.548 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:37.548 INFO:teuthology.orchestra.run.smithi155.stdout: "cct": { 2024-08-31T06:23:37.548 INFO:teuthology.orchestra.run.smithi155.stdout: "total_workers": 0, 2024-08-31T06:23:37.548 INFO:teuthology.orchestra.run.smithi155.stdout: "unhealthy_workers": 0 2024-08-31T06:23:37.548 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:37.548 INFO:teuthology.orchestra.run.smithi155.stdout: "mempool": { 2024-08-31T06:23:37.549 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_bytes": 0, 2024-08-31T06:23:37.549 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_items": 0, 2024-08-31T06:23:37.549 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_bytes": 0, 2024-08-31T06:23:37.549 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_items": 0, 2024-08-31T06:23:37.549 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_bytes": 0, 2024-08-31T06:23:37.549 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_items": 0, 2024-08-31T06:23:37.549 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-31T06:23:37.549 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_items": 0, 2024-08-31T06:23:37.549 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-31T06:23:37.549 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_items": 0, 2024-08-31T06:23:37.549 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_bytes": 0, 2024-08-31T06:23:37.549 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_items": 0, 2024-08-31T06:23:37.549 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-31T06:23:37.552 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_items": 0, 2024-08-31T06:23:37.552 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_bytes": 0, 2024-08-31T06:23:37.552 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_items": 0, 2024-08-31T06:23:37.552 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_bytes": 0, 2024-08-31T06:23:37.552 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_items": 0, 2024-08-31T06:23:37.552 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-31T06:23:37.552 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_items": 0, 2024-08-31T06:23:37.552 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-31T06:23:37.553 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_items": 0, 2024-08-31T06:23:37.553 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_bytes": 0, 2024-08-31T06:23:37.553 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_items": 0, 2024-08-31T06:23:37.553 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_bytes": 0, 2024-08-31T06:23:37.553 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_items": 0, 2024-08-31T06:23:37.553 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-31T06:23:37.553 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_items": 0, 2024-08-31T06:23:37.553 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_bytes": 0, 2024-08-31T06:23:37.553 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_items": 0, 2024-08-31T06:23:37.553 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_bytes": 0, 2024-08-31T06:23:37.553 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_items": 0, 2024-08-31T06:23:37.553 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_bytes": 0, 2024-08-31T06:23:37.553 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_items": 0, 2024-08-31T06:23:37.553 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_bytes": 0, 2024-08-31T06:23:37.554 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_items": 0, 2024-08-31T06:23:37.554 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_bytes": 148761, 2024-08-31T06:23:37.554 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_items": 31, 2024-08-31T06:23:37.554 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_bytes": 0, 2024-08-31T06:23:37.554 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_items": 0, 2024-08-31T06:23:37.554 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_bytes": 0, 2024-08-31T06:23:37.554 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_items": 0, 2024-08-31T06:23:37.554 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_bytes": 0, 2024-08-31T06:23:37.554 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_items": 0, 2024-08-31T06:23:37.554 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_bytes": 0, 2024-08-31T06:23:37.554 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_items": 0, 2024-08-31T06:23:37.554 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_bytes": 0, 2024-08-31T06:23:37.554 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_items": 0, 2024-08-31T06:23:37.554 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_bytes": 0, 2024-08-31T06:23:37.555 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_items": 0, 2024-08-31T06:23:37.555 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_bytes": 0, 2024-08-31T06:23:37.555 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_items": 0, 2024-08-31T06:23:37.555 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_bytes": 928, 2024-08-31T06:23:37.555 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_items": 2, 2024-08-31T06:23:37.555 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_bytes": 0, 2024-08-31T06:23:37.555 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_items": 0, 2024-08-31T06:23:37.555 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_bytes": 0, 2024-08-31T06:23:37.555 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_items": 0 2024-08-31T06:23:37.555 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:37.555 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-31T06:23:37.555 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-08-31T06:23:37.555 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-08-31T06:23:37.556 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-08-31T06:23:37.556 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 322, 2024-08-31T06:23:37.556 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 811208, 2024-08-31T06:23:37.556 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-08-31T06:23:37.556 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 322, 2024-08-31T06:23:37.556 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-08-31T06:23:37.556 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-08-31T06:23:37.556 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 322, 2024-08-31T06:23:37.556 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 811208, 2024-08-31T06:23:37.556 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-08-31T06:23:37.556 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:37.556 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:37.556 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:37.556 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-31T06:23:37.557 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-31T06:23:37.557 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-08-31T06:23:38.228 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.e 2024-08-31T06:23:38.228 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.e 2024-08-31T06:23:38.228 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.e 2024-08-31T06:23:38.228 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.e 2024-08-31T06:23:38.229 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2024-08-31T06:23:38.229 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2024-08-31T06:23:38.229 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2024-08-31T06:23:38.229 DEBUG:tasks.check_counter:Getting stats from h 2024-08-31T06:23:38.229 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph daemon mds.h perf dump 2024-08-31T06:23:38.479 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.b/config 2024-08-31T06:23:38.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:38 smithi155 ceph-mon[21742]: pgmap v878: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 6 op/s 2024-08-31T06:23:39.034 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-08-31T06:23:39.034 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-0": { 2024-08-31T06:23:39.034 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 654, 2024-08-31T06:23:39.035 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 656, 2024-08-31T06:23:39.035 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 8550869, 2024-08-31T06:23:39.035 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 207979, 2024-08-31T06:23:39.035 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 8, 2024-08-31T06:23:39.035 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 3, 2024-08-31T06:23:39.035 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.072004987, 2024-08-31T06:23:39.035 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.024139914, 2024-08-31T06:23:39.035 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.023422881, 2024-08-31T06:23:39.035 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.018272153, 2024-08-31T06:23:39.035 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T06:23:39.035 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 654, 2024-08-31T06:23:39.036 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.024561241, 2024-08-31T06:23:39.036 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000037555 2024-08-31T06:23:39.036 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.036 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-08-31T06:23:39.036 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:39.036 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:39.036 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:39.036 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.036 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-31T06:23:39.036 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 0 2024-08-31T06:23:39.036 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.036 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-1": { 2024-08-31T06:23:39.036 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 2432, 2024-08-31T06:23:39.036 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 2629, 2024-08-31T06:23:39.037 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 15839220, 2024-08-31T06:23:39.037 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 1221735, 2024-08-31T06:23:39.037 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 6, 2024-08-31T06:23:39.037 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 5, 2024-08-31T06:23:39.037 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.277383784, 2024-08-31T06:23:39.037 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.114881659, 2024-08-31T06:23:39.037 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.069043187, 2024-08-31T06:23:39.037 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.075854985, 2024-08-31T06:23:39.037 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T06:23:39.037 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 2628, 2024-08-31T06:23:39.037 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.146106745, 2024-08-31T06:23:39.037 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000055596 2024-08-31T06:23:39.037 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.038 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-08-31T06:23:39.038 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:39.038 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:39.038 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:39.038 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.038 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-08-31T06:23:39.038 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 449888 2024-08-31T06:23:39.038 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.038 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-2": { 2024-08-31T06:23:39.038 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 574, 2024-08-31T06:23:39.038 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 545, 2024-08-31T06:23:39.038 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 4638880, 2024-08-31T06:23:39.038 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 247801, 2024-08-31T06:23:39.039 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 5, 2024-08-31T06:23:39.039 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 3, 2024-08-31T06:23:39.039 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.069127461, 2024-08-31T06:23:39.039 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.030860100, 2024-08-31T06:23:39.039 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.033779522, 2024-08-31T06:23:39.039 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.007801739, 2024-08-31T06:23:39.039 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T06:23:39.039 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 544, 2024-08-31T06:23:39.039 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.026625878, 2024-08-31T06:23:39.039 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000048944 2024-08-31T06:23:39.039 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.039 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-08-31T06:23:39.039 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:39.039 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:39.040 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:39.040 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.040 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 380000, 2024-08-31T06:23:39.040 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 166384 2024-08-31T06:23:39.040 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.040 INFO:teuthology.orchestra.run.smithi155.stdout: "cct": { 2024-08-31T06:23:39.040 INFO:teuthology.orchestra.run.smithi155.stdout: "total_workers": 1, 2024-08-31T06:23:39.040 INFO:teuthology.orchestra.run.smithi155.stdout: "unhealthy_workers": 0 2024-08-31T06:23:39.040 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.040 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-MDSRank": { 2024-08-31T06:23:39.040 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-08-31T06:23:39.040 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-08-31T06:23:39.040 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 5632, 2024-08-31T06:23:39.041 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.158556300, 2024-08-31T06:23:39.041 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000028152 2024-08-31T06:23:39.041 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-31T06:23:39.041 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.041 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-PurgeQueue": { 2024-08-31T06:23:39.041 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-08-31T06:23:39.041 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-08-31T06:23:39.041 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:39.041 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:39.041 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:39.041 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-31T06:23:39.041 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.041 INFO:teuthology.orchestra.run.smithi155.stdout: "mds": { 2024-08-31T06:23:39.041 INFO:teuthology.orchestra.run.smithi155.stdout: "request": 0, 2024-08-31T06:23:39.042 INFO:teuthology.orchestra.run.smithi155.stdout: "reply": 0, 2024-08-31T06:23:39.042 INFO:teuthology.orchestra.run.smithi155.stdout: "reply_latency": { 2024-08-31T06:23:39.042 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:39.042 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:39.042 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:39.042 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.042 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_reply": 0, 2024-08-31T06:23:39.042 INFO:teuthology.orchestra.run.smithi155.stdout: "forward": 0, 2024-08-31T06:23:39.042 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_complete": 0, 2024-08-31T06:23:39.042 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_keys": 0, 2024-08-31T06:23:39.042 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_commit": 0, 2024-08-31T06:23:39.042 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_split": 0, 2024-08-31T06:23:39.042 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_merge": 0, 2024-08-31T06:23:39.042 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes": 691, 2024-08-31T06:23:39.043 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_top": 386, 2024-08-31T06:23:39.043 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_bottom": 305, 2024-08-31T06:23:39.043 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pin_tail": 0, 2024-08-31T06:23:39.043 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pinned": 140, 2024-08-31T06:23:39.043 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_expired": 0, 2024-08-31T06:23:39.043 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_with_caps": 0, 2024-08-31T06:23:39.043 INFO:teuthology.orchestra.run.smithi155.stdout: "caps": 0, 2024-08-31T06:23:39.043 INFO:teuthology.orchestra.run.smithi155.stdout: "subtrees": 65, 2024-08-31T06:23:39.043 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse": 0, 2024-08-31T06:23:39.043 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_hit": 0, 2024-08-31T06:23:39.043 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_forward": 0, 2024-08-31T06:23:39.043 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_discover": 0, 2024-08-31T06:23:39.043 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_dir_fetch": 0, 2024-08-31T06:23:39.044 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_remote_ino": 0, 2024-08-31T06:23:39.044 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_lock": 0, 2024-08-31T06:23:39.044 INFO:teuthology.orchestra.run.smithi155.stdout: "load_cent": 0, 2024-08-31T06:23:39.044 INFO:teuthology.orchestra.run.smithi155.stdout: "q": 0, 2024-08-31T06:23:39.044 INFO:teuthology.orchestra.run.smithi155.stdout: "exported": 0, 2024-08-31T06:23:39.044 INFO:teuthology.orchestra.run.smithi155.stdout: "exported_inodes": 0, 2024-08-31T06:23:39.044 INFO:teuthology.orchestra.run.smithi155.stdout: "imported": 0, 2024-08-31T06:23:39.044 INFO:teuthology.orchestra.run.smithi155.stdout: "imported_inodes": 0, 2024-08-31T06:23:39.044 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_dir_fetch": 0, 2024-08-31T06:23:39.044 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_backtrace_fetch": 0, 2024-08-31T06:23:39.044 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_peer_discover": 0, 2024-08-31T06:23:39.044 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rfiles": 1000, 2024-08-31T06:23:39.044 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rbytes": 1239684560, 2024-08-31T06:23:39.044 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rsnaps": 6, 2024-08-31T06:23:39.045 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_fetch": 0, 2024-08-31T06:23:39.045 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_set_tag": 0, 2024-08-31T06:23:39.045 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_repaired": 0, 2024-08-31T06:23:39.045 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_inotable_repaired": 0, 2024-08-31T06:23:39.045 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_inodes": 0, 2024-08-31T06:23:39.045 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_base_inodes": 0, 2024-08-31T06:23:39.045 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dirfrag_rstats": 0, 2024-08-31T06:23:39.045 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_file_inodes": 0, 2024-08-31T06:23:39.045 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_inode_file_caps": 0, 2024-08-31T06:23:39.045 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_revoke": 0, 2024-08-31T06:23:39.045 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_grant": 0, 2024-08-31T06:23:39.045 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_trunc": 0, 2024-08-31T06:23:39.045 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-31T06:23:39.046 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-31T06:23:39.046 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps": 0, 2024-08-31T06:23:39.046 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps_dirty": 0, 2024-08-31T06:23:39.046 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_cap_release": 0, 2024-08-31T06:23:39.046 INFO:teuthology.orchestra.run.smithi155.stdout: "process_request_cap_release": 0 2024-08-31T06:23:39.046 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.046 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_cache": { 2024-08-31T06:23:39.046 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update": 0, 2024-08-31T06:23:39.046 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update_receipt": 0, 2024-08-31T06:23:39.046 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_try_discover": 0, 2024-08-31T06:23:39.046 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_send_discover": 0, 2024-08-31T06:23:39.046 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_handle_discover": 0, 2024-08-31T06:23:39.046 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays": 0, 2024-08-31T06:23:39.046 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_delayed": 0, 2024-08-31T06:23:39.047 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_enqueuing": 0, 2024-08-31T06:23:39.047 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_created": 0, 2024-08-31T06:23:39.047 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_enqueued": 0, 2024-08-31T06:23:39.047 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_reintegrated": 0, 2024-08-31T06:23:39.047 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_migrated": 0, 2024-08-31T06:23:39.047 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_processing": 0, 2024-08-31T06:23:39.047 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_enqueued": 0, 2024-08-31T06:23:39.047 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_prioritized": 0, 2024-08-31T06:23:39.047 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_started": 0, 2024-08-31T06:23:39.047 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_completed": 0, 2024-08-31T06:23:39.047 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_quiesce_path": 0, 2024-08-31T06:23:39.047 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_quiesce_inode": 0, 2024-08-31T06:23:39.047 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_enqueue_scrub": 0, 2024-08-31T06:23:39.047 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_exportdir": 0, 2024-08-31T06:23:39.048 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_flush": 0, 2024-08-31T06:23:39.048 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragmentdir": 0, 2024-08-31T06:23:39.048 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragstats": 0, 2024-08-31T06:23:39.048 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_inodestats": 0, 2024-08-31T06:23:39.048 INFO:teuthology.orchestra.run.smithi155.stdout: "uninline_started": 0, 2024-08-31T06:23:39.048 INFO:teuthology.orchestra.run.smithi155.stdout: "uninline_succeeded": 0, 2024-08-31T06:23:39.048 INFO:teuthology.orchestra.run.smithi155.stdout: "uninline_write_failed": 0 2024-08-31T06:23:39.048 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.048 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_log": { 2024-08-31T06:23:39.048 INFO:teuthology.orchestra.run.smithi155.stdout: "evlrg": 0, 2024-08-31T06:23:39.049 INFO:teuthology.orchestra.run.smithi155.stdout: "evadd": 0, 2024-08-31T06:23:39.049 INFO:teuthology.orchestra.run.smithi155.stdout: "evex": 0, 2024-08-31T06:23:39.049 INFO:teuthology.orchestra.run.smithi155.stdout: "evtrm": 0, 2024-08-31T06:23:39.049 INFO:teuthology.orchestra.run.smithi155.stdout: "ev": 3015, 2024-08-31T06:23:39.049 INFO:teuthology.orchestra.run.smithi155.stdout: "evexg": 0, 2024-08-31T06:23:39.049 INFO:teuthology.orchestra.run.smithi155.stdout: "evexd": 0, 2024-08-31T06:23:39.049 INFO:teuthology.orchestra.run.smithi155.stdout: "segadd": 0, 2024-08-31T06:23:39.049 INFO:teuthology.orchestra.run.smithi155.stdout: "segex": 0, 2024-08-31T06:23:39.049 INFO:teuthology.orchestra.run.smithi155.stdout: "segtrm": 0, 2024-08-31T06:23:39.049 INFO:teuthology.orchestra.run.smithi155.stdout: "seg": 2, 2024-08-31T06:23:39.049 INFO:teuthology.orchestra.run.smithi155.stdout: "segmjr": 69, 2024-08-31T06:23:39.049 INFO:teuthology.orchestra.run.smithi155.stdout: "segexg": 0, 2024-08-31T06:23:39.049 INFO:teuthology.orchestra.run.smithi155.stdout: "segexd": 0, 2024-08-31T06:23:39.050 INFO:teuthology.orchestra.run.smithi155.stdout: "expos": 35082754, 2024-08-31T06:23:39.050 INFO:teuthology.orchestra.run.smithi155.stdout: "wrpos": 36111834, 2024-08-31T06:23:39.050 INFO:teuthology.orchestra.run.smithi155.stdout: "rdpos": 36103191, 2024-08-31T06:23:39.050 INFO:teuthology.orchestra.run.smithi155.stdout: "jlat": { 2024-08-31T06:23:39.050 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:39.050 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:39.050 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:39.050 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.050 INFO:teuthology.orchestra.run.smithi155.stdout: "replayed": 3015 2024-08-31T06:23:39.050 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.050 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_mem": { 2024-08-31T06:23:39.050 INFO:teuthology.orchestra.run.smithi155.stdout: "ino": 579, 2024-08-31T06:23:39.050 INFO:teuthology.orchestra.run.smithi155.stdout: "ino+": 1496, 2024-08-31T06:23:39.050 INFO:teuthology.orchestra.run.smithi155.stdout: "ino-": 917, 2024-08-31T06:23:39.051 INFO:teuthology.orchestra.run.smithi155.stdout: "dir": 110, 2024-08-31T06:23:39.051 INFO:teuthology.orchestra.run.smithi155.stdout: "dir+": 145, 2024-08-31T06:23:39.051 INFO:teuthology.orchestra.run.smithi155.stdout: "dir-": 35, 2024-08-31T06:23:39.051 INFO:teuthology.orchestra.run.smithi155.stdout: "dn": 691, 2024-08-31T06:23:39.051 INFO:teuthology.orchestra.run.smithi155.stdout: "dn+": 1680, 2024-08-31T06:23:39.051 INFO:teuthology.orchestra.run.smithi155.stdout: "dn-": 989, 2024-08-31T06:23:39.051 INFO:teuthology.orchestra.run.smithi155.stdout: "cap": 0, 2024-08-31T06:23:39.051 INFO:teuthology.orchestra.run.smithi155.stdout: "cap+": 0, 2024-08-31T06:23:39.051 INFO:teuthology.orchestra.run.smithi155.stdout: "cap-": 0, 2024-08-31T06:23:39.051 INFO:teuthology.orchestra.run.smithi155.stdout: "rss": 118504, 2024-08-31T06:23:39.051 INFO:teuthology.orchestra.run.smithi155.stdout: "heap": 319623168 2024-08-31T06:23:39.051 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.051 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_server": { 2024-08-31T06:23:39.051 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_client_request": 0, 2024-08-31T06:23:39.052 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_server_request": 0, 2024-08-31T06:23:39.052 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_request": 0, 2024-08-31T06:23:39.052 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_session": 0, 2024-08-31T06:23:39.052 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_peer_request": 0, 2024-08-31T06:23:39.052 INFO:teuthology.orchestra.run.smithi155.stdout: "req_create_latency": { 2024-08-31T06:23:39.052 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:39.052 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:39.052 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:39.052 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.052 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getattr_latency": { 2024-08-31T06:23:39.052 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:39.052 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:39.052 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:39.052 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.053 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getfilelock_latency": { 2024-08-31T06:23:39.053 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:39.053 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:39.053 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:39.053 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.053 INFO:teuthology.orchestra.run.smithi155.stdout: "req_link_latency": { 2024-08-31T06:23:39.053 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:39.053 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:39.053 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:39.053 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.053 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookup_latency": { 2024-08-31T06:23:39.053 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:39.053 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:39.054 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:39.054 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.054 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookuphash_latency": { 2024-08-31T06:23:39.054 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:39.054 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:39.054 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:39.054 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.054 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupino_latency": { 2024-08-31T06:23:39.054 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:39.054 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:39.054 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:39.054 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.054 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupname_latency": { 2024-08-31T06:23:39.055 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:39.055 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:39.055 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:39.055 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.055 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupparent_latency": { 2024-08-31T06:23:39.055 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:39.055 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:39.055 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:39.055 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.055 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupsnap_latency": { 2024-08-31T06:23:39.055 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:39.055 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:39.055 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:39.055 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.056 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lssnap_latency": { 2024-08-31T06:23:39.056 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:39.056 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:39.057 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:39.057 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.057 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mkdir_latency": { 2024-08-31T06:23:39.057 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:39.057 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:39.057 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:39.057 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.057 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mknod_latency": { 2024-08-31T06:23:39.057 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:39.057 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:39.057 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:39.057 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.057 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mksnap_latency": { 2024-08-31T06:23:39.057 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:39.058 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:39.058 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:39.058 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.058 INFO:teuthology.orchestra.run.smithi155.stdout: "req_open_latency": { 2024-08-31T06:23:39.058 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:39.058 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:39.058 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:39.058 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.058 INFO:teuthology.orchestra.run.smithi155.stdout: "req_readdir_latency": { 2024-08-31T06:23:39.058 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:39.058 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:39.058 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:39.058 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.058 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rename_latency": { 2024-08-31T06:23:39.059 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:39.059 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:39.059 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:39.059 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.059 INFO:teuthology.orchestra.run.smithi155.stdout: "req_renamesnap_latency": { 2024-08-31T06:23:39.059 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:39.059 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:39.059 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:39.059 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.059 INFO:teuthology.orchestra.run.smithi155.stdout: "req_snapdiff_latency": { 2024-08-31T06:23:39.059 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:39.059 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:39.059 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:39.060 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.060 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmdir_latency": { 2024-08-31T06:23:39.060 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:39.060 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:39.060 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:39.060 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.060 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmsnap_latency": { 2024-08-31T06:23:39.060 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:39.060 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:39.060 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:39.060 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.060 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmxattr_latency": { 2024-08-31T06:23:39.060 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:39.061 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:39.061 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:39.061 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.061 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setattr_latency": { 2024-08-31T06:23:39.061 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:39.061 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:39.061 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:39.061 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.061 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setdirlayout_latency": { 2024-08-31T06:23:39.061 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:39.062 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:39.062 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:39.062 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.062 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setfilelock_latency": { 2024-08-31T06:23:39.062 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:39.062 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:39.062 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:39.062 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.062 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setlayout_latency": { 2024-08-31T06:23:39.062 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:39.062 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:39.062 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:39.062 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.063 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setxattr_latency": { 2024-08-31T06:23:39.063 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:39.063 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:39.063 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:39.063 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.063 INFO:teuthology.orchestra.run.smithi155.stdout: "req_symlink_latency": { 2024-08-31T06:23:39.063 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:39.063 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:39.063 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:39.063 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.063 INFO:teuthology.orchestra.run.smithi155.stdout: "req_unlink_latency": { 2024-08-31T06:23:39.063 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:39.063 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:39.064 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:39.064 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.064 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_revoke_eviction": 0, 2024-08-31T06:23:39.064 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_acquisition_throttle": 0, 2024-08-31T06:23:39.064 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getvxattr_latency": { 2024-08-31T06:23:39.064 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:39.064 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:39.064 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:39.064 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-31T06:23:39.064 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.064 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_sessions": { 2024-08-31T06:23:39.064 INFO:teuthology.orchestra.run.smithi155.stdout: "session_count": 2, 2024-08-31T06:23:39.064 INFO:teuthology.orchestra.run.smithi155.stdout: "session_add": 2, 2024-08-31T06:23:39.064 INFO:teuthology.orchestra.run.smithi155.stdout: "session_remove": 0, 2024-08-31T06:23:39.065 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_open": 0, 2024-08-31T06:23:39.065 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_stale": 0, 2024-08-31T06:23:39.065 INFO:teuthology.orchestra.run.smithi155.stdout: "total_load": 0, 2024-08-31T06:23:39.065 INFO:teuthology.orchestra.run.smithi155.stdout: "average_load": 0, 2024-08-31T06:23:39.065 INFO:teuthology.orchestra.run.smithi155.stdout: "avg_session_uptime": 2273, 2024-08-31T06:23:39.065 INFO:teuthology.orchestra.run.smithi155.stdout: "mdthresh_evicted": 0 2024-08-31T06:23:39.065 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.065 INFO:teuthology.orchestra.run.smithi155.stdout: "mempool": { 2024-08-31T06:23:39.065 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_bytes": 0, 2024-08-31T06:23:39.065 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_items": 0, 2024-08-31T06:23:39.065 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_bytes": 0, 2024-08-31T06:23:39.065 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_items": 0, 2024-08-31T06:23:39.065 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_bytes": 0, 2024-08-31T06:23:39.065 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_items": 0, 2024-08-31T06:23:39.065 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-31T06:23:39.066 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_items": 0, 2024-08-31T06:23:39.066 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-31T06:23:39.066 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_items": 0, 2024-08-31T06:23:39.066 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_bytes": 0, 2024-08-31T06:23:39.066 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_items": 0, 2024-08-31T06:23:39.066 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-31T06:23:39.066 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_items": 0, 2024-08-31T06:23:39.066 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_bytes": 0, 2024-08-31T06:23:39.066 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_items": 0, 2024-08-31T06:23:39.066 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_bytes": 0, 2024-08-31T06:23:39.066 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_items": 0, 2024-08-31T06:23:39.066 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-31T06:23:39.066 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_items": 0, 2024-08-31T06:23:39.066 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-31T06:23:39.067 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_items": 0, 2024-08-31T06:23:39.067 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_bytes": 0, 2024-08-31T06:23:39.067 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_items": 0, 2024-08-31T06:23:39.067 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_bytes": 0, 2024-08-31T06:23:39.067 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_items": 0, 2024-08-31T06:23:39.067 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-31T06:23:39.067 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_items": 0, 2024-08-31T06:23:39.067 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_bytes": 0, 2024-08-31T06:23:39.067 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_items": 0, 2024-08-31T06:23:39.067 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_bytes": 0, 2024-08-31T06:23:39.067 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_items": 0, 2024-08-31T06:23:39.067 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_bytes": 0, 2024-08-31T06:23:39.067 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_items": 0, 2024-08-31T06:23:39.067 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_bytes": 0, 2024-08-31T06:23:39.067 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_items": 0, 2024-08-31T06:23:39.067 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_bytes": 238432, 2024-08-31T06:23:39.068 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_items": 694, 2024-08-31T06:23:39.068 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_bytes": 0, 2024-08-31T06:23:39.068 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_items": 0, 2024-08-31T06:23:39.068 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_bytes": 0, 2024-08-31T06:23:39.068 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_items": 0, 2024-08-31T06:23:39.068 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_bytes": 0, 2024-08-31T06:23:39.068 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_items": 0, 2024-08-31T06:23:39.068 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_bytes": 0, 2024-08-31T06:23:39.068 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_items": 0, 2024-08-31T06:23:39.068 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_bytes": 8064, 2024-08-31T06:23:39.068 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_items": 178, 2024-08-31T06:23:39.068 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_bytes": 0, 2024-08-31T06:23:39.068 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_items": 0, 2024-08-31T06:23:39.068 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_bytes": 0, 2024-08-31T06:23:39.068 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_items": 0, 2024-08-31T06:23:39.069 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_bytes": 2026181, 2024-08-31T06:23:39.069 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_items": 16085, 2024-08-31T06:23:39.069 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_bytes": 0, 2024-08-31T06:23:39.069 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_items": 0, 2024-08-31T06:23:39.069 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_bytes": 0, 2024-08-31T06:23:39.069 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_items": 0 2024-08-31T06:23:39.069 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.069 INFO:teuthology.orchestra.run.smithi155.stdout: "objecter": { 2024-08-31T06:23:39.069 INFO:teuthology.orchestra.run.smithi155.stdout: "op_active": 0, 2024-08-31T06:23:39.069 INFO:teuthology.orchestra.run.smithi155.stdout: "op_laggy": 0, 2024-08-31T06:23:39.069 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send": 3337, 2024-08-31T06:23:39.069 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send_bytes": 26, 2024-08-31T06:23:39.069 INFO:teuthology.orchestra.run.smithi155.stdout: "op_resend": 4, 2024-08-31T06:23:39.069 INFO:teuthology.orchestra.run.smithi155.stdout: "op_reply": 3337, 2024-08-31T06:23:39.070 INFO:teuthology.orchestra.run.smithi155.stdout: "op_latency": { 2024-08-31T06:23:39.070 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 3337, 2024-08-31T06:23:39.070 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 9.191804731, 2024-08-31T06:23:39.070 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.002754511 2024-08-31T06:23:39.070 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.070 INFO:teuthology.orchestra.run.smithi155.stdout: "op_inflight": 0, 2024-08-31T06:23:39.070 INFO:teuthology.orchestra.run.smithi155.stdout: "oplen_avg": { 2024-08-31T06:23:39.070 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 3337, 2024-08-31T06:23:39.070 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 3338 2024-08-31T06:23:39.070 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.070 INFO:teuthology.orchestra.run.smithi155.stdout: "op": 3337, 2024-08-31T06:23:39.070 INFO:teuthology.orchestra.run.smithi155.stdout: "op_r": 3337, 2024-08-31T06:23:39.070 INFO:teuthology.orchestra.run.smithi155.stdout: "op_w": 0, 2024-08-31T06:23:39.070 INFO:teuthology.orchestra.run.smithi155.stdout: "op_rmw": 0, 2024-08-31T06:23:39.070 INFO:teuthology.orchestra.run.smithi155.stdout: "op_pg": 0, 2024-08-31T06:23:39.070 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_stat": 1944, 2024-08-31T06:23:39.070 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_create": 0, 2024-08-31T06:23:39.071 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_read": 1390, 2024-08-31T06:23:39.071 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_write": 0, 2024-08-31T06:23:39.071 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writefull": 0, 2024-08-31T06:23:39.071 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writesame": 0, 2024-08-31T06:23:39.071 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_append": 0, 2024-08-31T06:23:39.071 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_zero": 0, 2024-08-31T06:23:39.071 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_truncate": 0, 2024-08-31T06:23:39.071 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_delete": 0, 2024-08-31T06:23:39.071 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_mapext": 0, 2024-08-31T06:23:39.071 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_sparse_read": 0, 2024-08-31T06:23:39.071 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_clonerange": 0, 2024-08-31T06:23:39.071 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_getxattr": 2, 2024-08-31T06:23:39.071 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_setxattr": 0, 2024-08-31T06:23:39.071 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_cmpxattr": 0, 2024-08-31T06:23:39.072 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_rmxattr": 0, 2024-08-31T06:23:39.072 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_resetxattrs": 0, 2024-08-31T06:23:39.072 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_call": 0, 2024-08-31T06:23:39.072 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_watch": 0, 2024-08-31T06:23:39.072 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_notify": 0, 2024-08-31T06:23:39.072 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_src_cmpxattr": 0, 2024-08-31T06:23:39.072 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls": 0, 2024-08-31T06:23:39.072 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls_filter": 0, 2024-08-31T06:23:39.072 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_other": 0, 2024-08-31T06:23:39.072 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_active": 0, 2024-08-31T06:23:39.072 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_send": 0, 2024-08-31T06:23:39.072 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_resend": 0, 2024-08-31T06:23:39.072 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_ping": 0, 2024-08-31T06:23:39.072 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_active": 0, 2024-08-31T06:23:39.073 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_send": 0, 2024-08-31T06:23:39.073 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_resend": 0, 2024-08-31T06:23:39.073 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_active": 0, 2024-08-31T06:23:39.073 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_send": 0, 2024-08-31T06:23:39.073 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_resend": 0, 2024-08-31T06:23:39.073 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_active": 0, 2024-08-31T06:23:39.073 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_send": 0, 2024-08-31T06:23:39.073 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_resend": 0, 2024-08-31T06:23:39.073 INFO:teuthology.orchestra.run.smithi155.stdout: "command_active": 0, 2024-08-31T06:23:39.073 INFO:teuthology.orchestra.run.smithi155.stdout: "command_send": 0, 2024-08-31T06:23:39.073 INFO:teuthology.orchestra.run.smithi155.stdout: "command_resend": 0, 2024-08-31T06:23:39.073 INFO:teuthology.orchestra.run.smithi155.stdout: "map_epoch": 103, 2024-08-31T06:23:39.073 INFO:teuthology.orchestra.run.smithi155.stdout: "map_full": 0, 2024-08-31T06:23:39.074 INFO:teuthology.orchestra.run.smithi155.stdout: "map_inc": 22, 2024-08-31T06:23:39.074 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_sessions": 8, 2024-08-31T06:23:39.074 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_open": 10, 2024-08-31T06:23:39.074 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_close": 2, 2024-08-31T06:23:39.074 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_laggy": 0, 2024-08-31T06:23:39.074 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_wr": 0, 2024-08-31T06:23:39.074 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_rd": 2, 2024-08-31T06:23:39.074 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_del": 0 2024-08-31T06:23:39.074 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.074 INFO:teuthology.orchestra.run.smithi155.stdout: "oft": { 2024-08-31T06:23:39.074 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_objs": 0, 2024-08-31T06:23:39.074 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_kv_pairs": 0, 2024-08-31T06:23:39.074 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_updates": 0, 2024-08-31T06:23:39.074 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_removes": 0 2024-08-31T06:23:39.075 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.075 INFO:teuthology.orchestra.run.smithi155.stdout: "purge_queue": { 2024-08-31T06:23:39.075 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops": 0, 2024-08-31T06:23:39.075 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops_high_water": 0, 2024-08-31T06:23:39.075 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing": 0, 2024-08-31T06:23:39.075 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_high_water": 0, 2024-08-31T06:23:39.075 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed_ops": 0, 2024-08-31T06:23:39.075 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed": 0, 2024-08-31T06:23:39.075 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_item_in_journal": 0 2024-08-31T06:23:39.075 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.075 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-31T06:23:39.075 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-08-31T06:23:39.075 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-08-31T06:23:39.075 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-08-31T06:23:39.076 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 3656, 2024-08-31T06:23:39.076 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 28680800, 2024-08-31T06:23:39.076 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-08-31T06:23:39.076 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 3656, 2024-08-31T06:23:39.076 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-08-31T06:23:39.076 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-08-31T06:23:39.076 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 3656, 2024-08-31T06:23:39.076 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 28680800, 2024-08-31T06:23:39.076 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-08-31T06:23:39.076 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:39.076 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:39.076 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:39.076 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-31T06:23:39.076 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.077 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_bytes": { 2024-08-31T06:23:39.077 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-08-31T06:23:39.077 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-08-31T06:23:39.077 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-08-31T06:23:39.077 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-08-31T06:23:39.077 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-08-31T06:23:39.077 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-08-31T06:23:39.077 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-08-31T06:23:39.077 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 3337, 2024-08-31T06:23:39.077 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 27719112, 2024-08-31T06:23:39.077 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 3337, 2024-08-31T06:23:39.077 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 27719112, 2024-08-31T06:23:39.077 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-08-31T06:23:39.078 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:39.078 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:39.078 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:39.078 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-31T06:23:39.078 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.078 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_ops": { 2024-08-31T06:23:39.078 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-08-31T06:23:39.078 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 1024, 2024-08-31T06:23:39.078 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-08-31T06:23:39.078 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-08-31T06:23:39.078 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-08-31T06:23:39.078 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-08-31T06:23:39.078 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-08-31T06:23:39.078 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 3337, 2024-08-31T06:23:39.078 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 3337, 2024-08-31T06:23:39.079 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 3337, 2024-08-31T06:23:39.079 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 3337, 2024-08-31T06:23:39.079 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-08-31T06:23:39.079 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:39.079 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:39.079 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:39.079 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-31T06:23:39.079 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.079 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle": { 2024-08-31T06:23:39.079 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-08-31T06:23:39.079 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-08-31T06:23:39.079 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-08-31T06:23:39.079 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-08-31T06:23:39.079 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-08-31T06:23:39.080 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-08-31T06:23:39.080 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-08-31T06:23:39.080 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-08-31T06:23:39.080 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-08-31T06:23:39.080 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 0, 2024-08-31T06:23:39.080 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 0, 2024-08-31T06:23:39.080 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-08-31T06:23:39.080 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:39.080 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:39.080 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:39.080 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-31T06:23:39.080 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.080 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle-0x555c177d81a0": { 2024-08-31T06:23:39.080 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-08-31T06:23:39.081 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-08-31T06:23:39.081 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-08-31T06:23:39.081 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-08-31T06:23:39.081 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-08-31T06:23:39.081 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-08-31T06:23:39.081 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-08-31T06:23:39.081 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-08-31T06:23:39.081 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-08-31T06:23:39.081 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 0, 2024-08-31T06:23:39.081 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 0, 2024-08-31T06:23:39.081 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-08-31T06:23:39.081 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:39.081 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:39.081 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:39.081 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-31T06:23:39.082 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:39.082 INFO:teuthology.orchestra.run.smithi155.stdout: "trackedop": { 2024-08-31T06:23:39.082 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_ops_count": 0 2024-08-31T06:23:39.082 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-31T06:23:39.082 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-08-31T06:23:39.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:38 smithi016 ceph-mon[17194]: pgmap v878: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 6 op/s 2024-08-31T06:23:39.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:38 smithi184 ceph-mon[22331]: pgmap v878: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 6 op/s 2024-08-31T06:23:39.939 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-08-31T06:23:39.940 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-08-31T06:23:39.940 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-08-31T06:23:39.940 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2024-08-31T06:23:39.940 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-08-31T06:23:39.940 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-08-31T06:23:39.940 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-08-31T06:23:39.940 DEBUG:tasks.check_counter:Getting stats from k 2024-08-31T06:23:39.940 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph daemon mds.k perf dump 2024-08-31T06:23:40.187 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.b/config 2024-08-31T06:23:40.713 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-08-31T06:23:40.714 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-0": { 2024-08-31T06:23:40.714 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 0, 2024-08-31T06:23:40.714 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 2, 2024-08-31T06:23:40.714 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 0, 2024-08-31T06:23:40.714 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 0, 2024-08-31T06:23:40.714 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 4, 2024-08-31T06:23:40.714 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 0, 2024-08-31T06:23:40.714 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.001129578, 2024-08-31T06:23:40.714 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.000000000, 2024-08-31T06:23:40.714 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.000311884, 2024-08-31T06:23:40.714 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-31T06:23:40.714 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T06:23:40.715 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:40.715 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:40.715 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:40.715 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:40.715 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-08-31T06:23:40.715 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:40.715 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:40.715 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:40.715 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:40.715 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-31T06:23:40.715 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 0 2024-08-31T06:23:40.715 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:40.715 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-1": { 2024-08-31T06:23:40.715 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 2, 2024-08-31T06:23:40.716 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 244, 2024-08-31T06:23:40.716 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 320, 2024-08-31T06:23:40.716 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 203168, 2024-08-31T06:23:40.716 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 4, 2024-08-31T06:23:40.716 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 2, 2024-08-31T06:23:40.716 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.018959333, 2024-08-31T06:23:40.716 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.014982681, 2024-08-31T06:23:40.716 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.001369296, 2024-08-31T06:23:40.716 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-31T06:23:40.716 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T06:23:40.716 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 242, 2024-08-31T06:23:40.716 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.022534195, 2024-08-31T06:23:40.716 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000093116 2024-08-31T06:23:40.716 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:40.717 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-08-31T06:23:40.717 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:40.717 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:40.717 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:40.717 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:40.717 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 320, 2024-08-31T06:23:40.717 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 203168 2024-08-31T06:23:40.717 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:40.717 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-2": { 2024-08-31T06:23:40.717 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 327, 2024-08-31T06:23:40.717 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 320, 2024-08-31T06:23:40.717 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 860256, 2024-08-31T06:23:40.717 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 197328, 2024-08-31T06:23:40.717 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 2, 2024-08-31T06:23:40.717 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 1, 2024-08-31T06:23:40.718 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.056863081, 2024-08-31T06:23:40.718 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.026252845, 2024-08-31T06:23:40.718 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.039680581, 2024-08-31T06:23:40.718 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-31T06:23:40.718 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T06:23:40.718 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 320, 2024-08-31T06:23:40.718 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.022949498, 2024-08-31T06:23:40.718 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000071717 2024-08-31T06:23:40.718 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:40.718 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-08-31T06:23:40.718 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:40.718 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:40.718 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:40.718 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:40.719 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 860256, 2024-08-31T06:23:40.719 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 197328 2024-08-31T06:23:40.719 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:40.719 INFO:teuthology.orchestra.run.smithi155.stdout: "cct": { 2024-08-31T06:23:40.719 INFO:teuthology.orchestra.run.smithi155.stdout: "total_workers": 0, 2024-08-31T06:23:40.719 INFO:teuthology.orchestra.run.smithi155.stdout: "unhealthy_workers": 0 2024-08-31T06:23:40.719 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:40.719 INFO:teuthology.orchestra.run.smithi155.stdout: "mempool": { 2024-08-31T06:23:40.719 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_bytes": 0, 2024-08-31T06:23:40.719 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_items": 0, 2024-08-31T06:23:40.719 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_bytes": 0, 2024-08-31T06:23:40.719 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_items": 0, 2024-08-31T06:23:40.719 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_bytes": 0, 2024-08-31T06:23:40.719 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_items": 0, 2024-08-31T06:23:40.720 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-31T06:23:40.720 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_items": 0, 2024-08-31T06:23:40.720 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-31T06:23:40.720 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_items": 0, 2024-08-31T06:23:40.720 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_bytes": 0, 2024-08-31T06:23:40.720 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_items": 0, 2024-08-31T06:23:40.720 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-31T06:23:40.720 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_items": 0, 2024-08-31T06:23:40.720 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_bytes": 0, 2024-08-31T06:23:40.720 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_items": 0, 2024-08-31T06:23:40.720 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_bytes": 0, 2024-08-31T06:23:40.720 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_items": 0, 2024-08-31T06:23:40.720 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-31T06:23:40.720 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_items": 0, 2024-08-31T06:23:40.720 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-31T06:23:40.721 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_items": 0, 2024-08-31T06:23:40.721 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_bytes": 0, 2024-08-31T06:23:40.721 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_items": 0, 2024-08-31T06:23:40.721 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_bytes": 0, 2024-08-31T06:23:40.721 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_items": 0, 2024-08-31T06:23:40.721 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-31T06:23:40.721 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_items": 0, 2024-08-31T06:23:40.721 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_bytes": 0, 2024-08-31T06:23:40.721 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_items": 0, 2024-08-31T06:23:40.721 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_bytes": 0, 2024-08-31T06:23:40.721 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_items": 0, 2024-08-31T06:23:40.721 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_bytes": 0, 2024-08-31T06:23:40.721 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_items": 0, 2024-08-31T06:23:40.721 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_bytes": 0, 2024-08-31T06:23:40.722 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_items": 0, 2024-08-31T06:23:40.722 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_bytes": 148721, 2024-08-31T06:23:40.722 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_items": 30, 2024-08-31T06:23:40.722 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_bytes": 0, 2024-08-31T06:23:40.722 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_items": 0, 2024-08-31T06:23:40.722 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_bytes": 0, 2024-08-31T06:23:40.722 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_items": 0, 2024-08-31T06:23:40.722 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_bytes": 0, 2024-08-31T06:23:40.722 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_items": 0, 2024-08-31T06:23:40.722 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_bytes": 0, 2024-08-31T06:23:40.722 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_items": 0, 2024-08-31T06:23:40.722 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_bytes": 0, 2024-08-31T06:23:40.722 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_items": 0, 2024-08-31T06:23:40.722 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_bytes": 0, 2024-08-31T06:23:40.723 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_items": 0, 2024-08-31T06:23:40.723 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_bytes": 0, 2024-08-31T06:23:40.723 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_items": 0, 2024-08-31T06:23:40.723 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_bytes": 928, 2024-08-31T06:23:40.723 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_items": 2, 2024-08-31T06:23:40.723 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_bytes": 0, 2024-08-31T06:23:40.723 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_items": 0, 2024-08-31T06:23:40.723 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_bytes": 0, 2024-08-31T06:23:40.723 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_items": 0 2024-08-31T06:23:40.723 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-31T06:23:40.723 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-31T06:23:40.724 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-08-31T06:23:40.724 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-08-31T06:23:40.724 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-08-31T06:23:40.724 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 325, 2024-08-31T06:23:40.724 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 811598, 2024-08-31T06:23:40.724 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-08-31T06:23:40.724 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 325, 2024-08-31T06:23:40.724 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-08-31T06:23:40.724 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-08-31T06:23:40.724 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 325, 2024-08-31T06:23:40.724 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 811598, 2024-08-31T06:23:40.724 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-08-31T06:23:40.724 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-31T06:23:40.724 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-31T06:23:40.725 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-31T06:23:40.725 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-31T06:23:40.725 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-31T06:23:40.725 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-08-31T06:23:40.872 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:40 smithi155 ceph-mon[21742]: pgmap v879: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-08-31T06:23:41.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:40 smithi016 ceph-mon[17194]: pgmap v879: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-08-31T06:23:41.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:40 smithi184 ceph-mon[22331]: pgmap v879: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-08-31T06:23:41.417 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-08-31T06:23:41.417 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-08-31T06:23:41.417 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-08-31T06:23:41.417 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-08-31T06:23:41.417 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-08-31T06:23:41.417 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-08-31T06:23:41.417 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-08-31T06:23:41.417 DEBUG:tasks.check_counter:Getting stats from c 2024-08-31T06:23:41.418 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph daemon mds.c perf dump 2024-08-31T06:23:41.680 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.c/config 2024-08-31T06:23:42.462 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-08-31T06:23:42.462 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-0": { 2024-08-31T06:23:42.462 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 3579, 2024-08-31T06:23:42.462 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 3580, 2024-08-31T06:23:42.462 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 8167512, 2024-08-31T06:23:42.462 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 1137811, 2024-08-31T06:23:42.462 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 6, 2024-08-31T06:23:42.463 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 2, 2024-08-31T06:23:42.463 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.354264758, 2024-08-31T06:23:42.463 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.145239243, 2024-08-31T06:23:42.463 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.077625253, 2024-08-31T06:23:42.463 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.110343734, 2024-08-31T06:23:42.463 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T06:23:42.463 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 3578, 2024-08-31T06:23:42.463 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.164781037, 2024-08-31T06:23:42.463 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000046053 2024-08-31T06:23:42.463 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.463 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-08-31T06:23:42.463 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:42.463 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:42.463 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:42.464 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.464 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-31T06:23:42.464 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 0 2024-08-31T06:23:42.464 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.464 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-1": { 2024-08-31T06:23:42.464 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 385, 2024-08-31T06:23:42.464 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 348, 2024-08-31T06:23:42.464 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 1025372, 2024-08-31T06:23:42.464 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 201050, 2024-08-31T06:23:42.464 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 3, 2024-08-31T06:23:42.464 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 2, 2024-08-31T06:23:42.464 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.056724037, 2024-08-31T06:23:42.464 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.026684021, 2024-08-31T06:23:42.464 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.040037820, 2024-08-31T06:23:42.465 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.000084259, 2024-08-31T06:23:42.465 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T06:23:42.465 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 347, 2024-08-31T06:23:42.465 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.020967074, 2024-08-31T06:23:42.465 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000060423 2024-08-31T06:23:42.465 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.467 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-08-31T06:23:42.467 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:42.468 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:42.468 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:42.468 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.468 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 1024816, 2024-08-31T06:23:42.468 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 200352 2024-08-31T06:23:42.468 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.468 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-2": { 2024-08-31T06:23:42.468 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 7, 2024-08-31T06:23:42.468 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 244, 2024-08-31T06:23:42.468 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 3580, 2024-08-31T06:23:42.468 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 598446, 2024-08-31T06:23:42.468 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 6, 2024-08-31T06:23:42.468 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 3, 2024-08-31T06:23:42.468 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.019478835, 2024-08-31T06:23:42.469 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.013147923, 2024-08-31T06:23:42.469 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.004115569, 2024-08-31T06:23:42.469 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.000019057, 2024-08-31T06:23:42.469 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T06:23:42.469 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 243, 2024-08-31T06:23:42.469 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.021378429, 2024-08-31T06:23:42.469 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000087977 2024-08-31T06:23:42.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.469 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-08-31T06:23:42.469 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:42.469 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:42.469 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:42.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.470 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-08-31T06:23:42.470 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 598128 2024-08-31T06:23:42.470 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.470 INFO:teuthology.orchestra.run.smithi184.stdout: "cct": { 2024-08-31T06:23:42.470 INFO:teuthology.orchestra.run.smithi184.stdout: "total_workers": 1, 2024-08-31T06:23:42.470 INFO:teuthology.orchestra.run.smithi184.stdout: "unhealthy_workers": 0 2024-08-31T06:23:42.470 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.470 INFO:teuthology.orchestra.run.smithi184.stdout: "finisher-MDSRank": { 2024-08-31T06:23:42.470 INFO:teuthology.orchestra.run.smithi184.stdout: "queue_len": 0, 2024-08-31T06:23:42.470 INFO:teuthology.orchestra.run.smithi184.stdout: "complete_latency": { 2024-08-31T06:23:42.470 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 6076, 2024-08-31T06:23:42.470 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.158425792, 2024-08-31T06:23:42.470 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000026074 2024-08-31T06:23:42.470 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-08-31T06:23:42.470 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.471 INFO:teuthology.orchestra.run.smithi184.stdout: "finisher-PurgeQueue": { 2024-08-31T06:23:42.471 INFO:teuthology.orchestra.run.smithi184.stdout: "queue_len": 0, 2024-08-31T06:23:42.471 INFO:teuthology.orchestra.run.smithi184.stdout: "complete_latency": { 2024-08-31T06:23:42.471 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:42.471 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:42.471 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:42.471 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-08-31T06:23:42.471 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.471 INFO:teuthology.orchestra.run.smithi184.stdout: "mds": { 2024-08-31T06:23:42.471 INFO:teuthology.orchestra.run.smithi184.stdout: "request": 0, 2024-08-31T06:23:42.471 INFO:teuthology.orchestra.run.smithi184.stdout: "reply": 0, 2024-08-31T06:23:42.471 INFO:teuthology.orchestra.run.smithi184.stdout: "reply_latency": { 2024-08-31T06:23:42.471 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:42.471 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:42.471 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:42.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.472 INFO:teuthology.orchestra.run.smithi184.stdout: "slow_reply": 0, 2024-08-31T06:23:42.472 INFO:teuthology.orchestra.run.smithi184.stdout: "forward": 0, 2024-08-31T06:23:42.472 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_fetch_complete": 0, 2024-08-31T06:23:42.472 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_fetch_keys": 0, 2024-08-31T06:23:42.472 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_commit": 0, 2024-08-31T06:23:42.472 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_split": 0, 2024-08-31T06:23:42.472 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_merge": 0, 2024-08-31T06:23:42.472 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes": 1004, 2024-08-31T06:23:42.472 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_top": 301, 2024-08-31T06:23:42.472 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_bottom": 703, 2024-08-31T06:23:42.472 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_pin_tail": 0, 2024-08-31T06:23:42.472 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_pinned": 574, 2024-08-31T06:23:42.472 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_expired": 0, 2024-08-31T06:23:42.473 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_with_caps": 0, 2024-08-31T06:23:42.473 INFO:teuthology.orchestra.run.smithi184.stdout: "caps": 0, 2024-08-31T06:23:42.473 INFO:teuthology.orchestra.run.smithi184.stdout: "subtrees": 26, 2024-08-31T06:23:42.473 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse": 0, 2024-08-31T06:23:42.473 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_hit": 0, 2024-08-31T06:23:42.473 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_forward": 0, 2024-08-31T06:23:42.473 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_discover": 0, 2024-08-31T06:23:42.473 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_dir_fetch": 0, 2024-08-31T06:23:42.473 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_remote_ino": 0, 2024-08-31T06:23:42.473 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_lock": 0, 2024-08-31T06:23:42.473 INFO:teuthology.orchestra.run.smithi184.stdout: "load_cent": 0, 2024-08-31T06:23:42.473 INFO:teuthology.orchestra.run.smithi184.stdout: "q": 0, 2024-08-31T06:23:42.473 INFO:teuthology.orchestra.run.smithi184.stdout: "exported": 0, 2024-08-31T06:23:42.473 INFO:teuthology.orchestra.run.smithi184.stdout: "exported_inodes": 0, 2024-08-31T06:23:42.474 INFO:teuthology.orchestra.run.smithi184.stdout: "imported": 0, 2024-08-31T06:23:42.474 INFO:teuthology.orchestra.run.smithi184.stdout: "imported_inodes": 0, 2024-08-31T06:23:42.474 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_dir_fetch": 0, 2024-08-31T06:23:42.474 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_backtrace_fetch": 0, 2024-08-31T06:23:42.474 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_peer_discover": 0, 2024-08-31T06:23:42.474 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rfiles": 0, 2024-08-31T06:23:42.474 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rbytes": 0, 2024-08-31T06:23:42.474 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rsnaps": 0, 2024-08-31T06:23:42.474 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_backtrace_fetch": 0, 2024-08-31T06:23:42.474 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_set_tag": 0, 2024-08-31T06:23:42.474 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_backtrace_repaired": 0, 2024-08-31T06:23:42.474 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_inotable_repaired": 0, 2024-08-31T06:23:42.474 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dir_inodes": 0, 2024-08-31T06:23:42.474 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dir_base_inodes": 0, 2024-08-31T06:23:42.474 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dirfrag_rstats": 0, 2024-08-31T06:23:42.474 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_file_inodes": 0, 2024-08-31T06:23:42.475 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_inode_file_caps": 0, 2024-08-31T06:23:42.475 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_revoke": 0, 2024-08-31T06:23:42.475 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_grant": 0, 2024-08-31T06:23:42.475 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_trunc": 0, 2024-08-31T06:23:42.475 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-31T06:23:42.475 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-31T06:23:42.475 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_caps": 0, 2024-08-31T06:23:42.475 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_caps_dirty": 0, 2024-08-31T06:23:42.475 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_cap_release": 0, 2024-08-31T06:23:42.475 INFO:teuthology.orchestra.run.smithi184.stdout: "process_request_cap_release": 0 2024-08-31T06:23:42.475 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.475 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_cache": { 2024-08-31T06:23:42.475 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_update": 0, 2024-08-31T06:23:42.475 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_update_receipt": 0, 2024-08-31T06:23:42.475 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_try_discover": 0, 2024-08-31T06:23:42.476 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_send_discover": 0, 2024-08-31T06:23:42.476 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_handle_discover": 0, 2024-08-31T06:23:42.476 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays": 0, 2024-08-31T06:23:42.476 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays_delayed": 0, 2024-08-31T06:23:42.476 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays_enqueuing": 0, 2024-08-31T06:23:42.476 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_created": 0, 2024-08-31T06:23:42.476 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_enqueued": 0, 2024-08-31T06:23:42.476 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_reintegrated": 0, 2024-08-31T06:23:42.476 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_migrated": 0, 2024-08-31T06:23:42.476 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_processing": 0, 2024-08-31T06:23:42.476 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_enqueued": 0, 2024-08-31T06:23:42.476 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_prioritized": 0, 2024-08-31T06:23:42.476 INFO:teuthology.orchestra.run.smithi184.stdout: "recovery_started": 0, 2024-08-31T06:23:42.476 INFO:teuthology.orchestra.run.smithi184.stdout: "recovery_completed": 0, 2024-08-31T06:23:42.477 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_quiesce_path": 0, 2024-08-31T06:23:42.477 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_quiesce_inode": 0, 2024-08-31T06:23:42.477 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_enqueue_scrub": 0, 2024-08-31T06:23:42.477 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_exportdir": 0, 2024-08-31T06:23:42.477 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_flush": 0, 2024-08-31T06:23:42.477 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_fragmentdir": 0, 2024-08-31T06:23:42.477 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_fragstats": 0, 2024-08-31T06:23:42.477 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_inodestats": 0, 2024-08-31T06:23:42.477 INFO:teuthology.orchestra.run.smithi184.stdout: "uninline_started": 0, 2024-08-31T06:23:42.477 INFO:teuthology.orchestra.run.smithi184.stdout: "uninline_succeeded": 0, 2024-08-31T06:23:42.477 INFO:teuthology.orchestra.run.smithi184.stdout: "uninline_write_failed": 0 2024-08-31T06:23:42.477 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.477 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_log": { 2024-08-31T06:23:42.478 INFO:teuthology.orchestra.run.smithi184.stdout: "evlrg": 0, 2024-08-31T06:23:42.478 INFO:teuthology.orchestra.run.smithi184.stdout: "evadd": 0, 2024-08-31T06:23:42.478 INFO:teuthology.orchestra.run.smithi184.stdout: "evex": 0, 2024-08-31T06:23:42.478 INFO:teuthology.orchestra.run.smithi184.stdout: "evtrm": 0, 2024-08-31T06:23:42.478 INFO:teuthology.orchestra.run.smithi184.stdout: "ev": 2834, 2024-08-31T06:23:42.478 INFO:teuthology.orchestra.run.smithi184.stdout: "evexg": 0, 2024-08-31T06:23:42.478 INFO:teuthology.orchestra.run.smithi184.stdout: "evexd": 0, 2024-08-31T06:23:42.478 INFO:teuthology.orchestra.run.smithi184.stdout: "segadd": 0, 2024-08-31T06:23:42.478 INFO:teuthology.orchestra.run.smithi184.stdout: "segex": 0, 2024-08-31T06:23:42.478 INFO:teuthology.orchestra.run.smithi184.stdout: "segtrm": 0, 2024-08-31T06:23:42.478 INFO:teuthology.orchestra.run.smithi184.stdout: "seg": 5, 2024-08-31T06:23:42.478 INFO:teuthology.orchestra.run.smithi184.stdout: "segmjr": 2, 2024-08-31T06:23:42.478 INFO:teuthology.orchestra.run.smithi184.stdout: "segexg": 0, 2024-08-31T06:23:42.478 INFO:teuthology.orchestra.run.smithi184.stdout: "segexd": 0, 2024-08-31T06:23:42.479 INFO:teuthology.orchestra.run.smithi184.stdout: "expos": 4194304, 2024-08-31T06:23:42.479 INFO:teuthology.orchestra.run.smithi184.stdout: "wrpos": 11359943, 2024-08-31T06:23:42.479 INFO:teuthology.orchestra.run.smithi184.stdout: "rdpos": 11359369, 2024-08-31T06:23:42.479 INFO:teuthology.orchestra.run.smithi184.stdout: "jlat": { 2024-08-31T06:23:42.479 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:42.479 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:42.479 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:42.479 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.479 INFO:teuthology.orchestra.run.smithi184.stdout: "replayed": 2834 2024-08-31T06:23:42.479 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.479 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_mem": { 2024-08-31T06:23:42.479 INFO:teuthology.orchestra.run.smithi184.stdout: "ino": 607, 2024-08-31T06:23:42.479 INFO:teuthology.orchestra.run.smithi184.stdout: "ino+": 1622, 2024-08-31T06:23:42.479 INFO:teuthology.orchestra.run.smithi184.stdout: "ino-": 1015, 2024-08-31T06:23:42.480 INFO:teuthology.orchestra.run.smithi184.stdout: "dir": 42, 2024-08-31T06:23:42.480 INFO:teuthology.orchestra.run.smithi184.stdout: "dir+": 86, 2024-08-31T06:23:42.480 INFO:teuthology.orchestra.run.smithi184.stdout: "dir-": 44, 2024-08-31T06:23:42.480 INFO:teuthology.orchestra.run.smithi184.stdout: "dn": 1004, 2024-08-31T06:23:42.480 INFO:teuthology.orchestra.run.smithi184.stdout: "dn+": 1852, 2024-08-31T06:23:42.480 INFO:teuthology.orchestra.run.smithi184.stdout: "dn-": 848, 2024-08-31T06:23:42.480 INFO:teuthology.orchestra.run.smithi184.stdout: "cap": 0, 2024-08-31T06:23:42.480 INFO:teuthology.orchestra.run.smithi184.stdout: "cap+": 0, 2024-08-31T06:23:42.480 INFO:teuthology.orchestra.run.smithi184.stdout: "cap-": 0, 2024-08-31T06:23:42.480 INFO:teuthology.orchestra.run.smithi184.stdout: "rss": 121956, 2024-08-31T06:23:42.480 INFO:teuthology.orchestra.run.smithi184.stdout: "heap": 321789952 2024-08-31T06:23:42.480 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.480 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_server": { 2024-08-31T06:23:42.480 INFO:teuthology.orchestra.run.smithi184.stdout: "dispatch_client_request": 0, 2024-08-31T06:23:42.480 INFO:teuthology.orchestra.run.smithi184.stdout: "dispatch_server_request": 0, 2024-08-31T06:23:42.481 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_request": 0, 2024-08-31T06:23:42.481 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_session": 0, 2024-08-31T06:23:42.481 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_peer_request": 0, 2024-08-31T06:23:42.481 INFO:teuthology.orchestra.run.smithi184.stdout: "req_create_latency": { 2024-08-31T06:23:42.481 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:42.481 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:42.481 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:42.481 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.481 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getattr_latency": { 2024-08-31T06:23:42.481 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:42.481 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:42.481 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:42.481 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.481 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getfilelock_latency": { 2024-08-31T06:23:42.481 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:42.482 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:42.482 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:42.482 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.482 INFO:teuthology.orchestra.run.smithi184.stdout: "req_link_latency": { 2024-08-31T06:23:42.482 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:42.482 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:42.482 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:42.482 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.482 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookup_latency": { 2024-08-31T06:23:42.482 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:42.482 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:42.482 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:42.482 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.482 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookuphash_latency": { 2024-08-31T06:23:42.483 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:42.483 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:42.483 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:42.483 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.483 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupino_latency": { 2024-08-31T06:23:42.483 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:42.483 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:42.483 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:42.483 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.483 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupname_latency": { 2024-08-31T06:23:42.483 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:42.483 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:42.483 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:42.483 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.483 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupparent_latency": { 2024-08-31T06:23:42.484 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:42.484 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:42.484 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:42.484 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.484 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupsnap_latency": { 2024-08-31T06:23:42.484 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:42.484 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:42.484 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:42.484 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.484 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lssnap_latency": { 2024-08-31T06:23:42.484 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:42.485 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:42.485 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:42.485 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.485 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mkdir_latency": { 2024-08-31T06:23:42.486 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:42.486 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:42.486 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:42.486 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.486 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mknod_latency": { 2024-08-31T06:23:42.486 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:42.486 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:42.486 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:42.486 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.486 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mksnap_latency": { 2024-08-31T06:23:42.486 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:42.486 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:42.486 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:42.486 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.487 INFO:teuthology.orchestra.run.smithi184.stdout: "req_open_latency": { 2024-08-31T06:23:42.487 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:42.487 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:42.487 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:42.487 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.487 INFO:teuthology.orchestra.run.smithi184.stdout: "req_readdir_latency": { 2024-08-31T06:23:42.487 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:42.487 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:42.487 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:42.487 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.487 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rename_latency": { 2024-08-31T06:23:42.487 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:42.487 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:42.487 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:42.488 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.488 INFO:teuthology.orchestra.run.smithi184.stdout: "req_renamesnap_latency": { 2024-08-31T06:23:42.488 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:42.488 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:42.488 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:42.488 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.488 INFO:teuthology.orchestra.run.smithi184.stdout: "req_snapdiff_latency": { 2024-08-31T06:23:42.488 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:42.488 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:42.488 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:42.488 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.488 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmdir_latency": { 2024-08-31T06:23:42.488 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:42.489 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:42.489 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:42.489 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.489 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmsnap_latency": { 2024-08-31T06:23:42.489 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:42.489 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:42.489 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:42.489 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.489 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmxattr_latency": { 2024-08-31T06:23:42.489 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:42.489 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:42.489 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:42.489 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.489 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setattr_latency": { 2024-08-31T06:23:42.490 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:42.490 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:42.490 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:42.490 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.490 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setdirlayout_latency": { 2024-08-31T06:23:42.490 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:42.490 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:42.490 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:42.490 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.490 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setfilelock_latency": { 2024-08-31T06:23:42.490 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:42.490 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:42.490 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:42.490 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.491 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setlayout_latency": { 2024-08-31T06:23:42.491 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:42.491 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:42.491 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:42.491 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.491 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setxattr_latency": { 2024-08-31T06:23:42.491 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:42.491 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:42.491 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:42.491 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.491 INFO:teuthology.orchestra.run.smithi184.stdout: "req_symlink_latency": { 2024-08-31T06:23:42.491 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:42.491 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:42.491 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:42.491 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.492 INFO:teuthology.orchestra.run.smithi184.stdout: "req_unlink_latency": { 2024-08-31T06:23:42.492 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:42.492 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:42.492 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:42.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.492 INFO:teuthology.orchestra.run.smithi184.stdout: "cap_revoke_eviction": 0, 2024-08-31T06:23:42.492 INFO:teuthology.orchestra.run.smithi184.stdout: "cap_acquisition_throttle": 0, 2024-08-31T06:23:42.492 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getvxattr_latency": { 2024-08-31T06:23:42.492 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:42.492 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:42.492 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:42.492 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-08-31T06:23:42.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.492 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_sessions": { 2024-08-31T06:23:42.493 INFO:teuthology.orchestra.run.smithi184.stdout: "session_count": 1, 2024-08-31T06:23:42.493 INFO:teuthology.orchestra.run.smithi184.stdout: "session_add": 2, 2024-08-31T06:23:42.493 INFO:teuthology.orchestra.run.smithi184.stdout: "session_remove": 1, 2024-08-31T06:23:42.493 INFO:teuthology.orchestra.run.smithi184.stdout: "sessions_open": 1, 2024-08-31T06:23:42.493 INFO:teuthology.orchestra.run.smithi184.stdout: "sessions_stale": 0, 2024-08-31T06:23:42.493 INFO:teuthology.orchestra.run.smithi184.stdout: "total_load": 0, 2024-08-31T06:23:42.493 INFO:teuthology.orchestra.run.smithi184.stdout: "average_load": 0, 2024-08-31T06:23:42.493 INFO:teuthology.orchestra.run.smithi184.stdout: "avg_session_uptime": 2280, 2024-08-31T06:23:42.493 INFO:teuthology.orchestra.run.smithi184.stdout: "mdthresh_evicted": 0 2024-08-31T06:23:42.493 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.493 INFO:teuthology.orchestra.run.smithi184.stdout: "mempool": { 2024-08-31T06:23:42.493 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_bytes": 0, 2024-08-31T06:23:42.493 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_items": 0, 2024-08-31T06:23:42.493 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_bytes": 0, 2024-08-31T06:23:42.494 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_items": 0, 2024-08-31T06:23:42.494 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_bytes": 0, 2024-08-31T06:23:42.494 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_items": 0, 2024-08-31T06:23:42.494 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-31T06:23:42.494 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_items": 0, 2024-08-31T06:23:42.494 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-31T06:23:42.494 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_items": 0, 2024-08-31T06:23:42.494 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_bytes": 0, 2024-08-31T06:23:42.494 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_items": 0, 2024-08-31T06:23:42.494 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-31T06:23:42.494 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_items": 0, 2024-08-31T06:23:42.494 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_bytes": 0, 2024-08-31T06:23:42.494 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_items": 0, 2024-08-31T06:23:42.494 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_bytes": 0, 2024-08-31T06:23:42.494 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_items": 0, 2024-08-31T06:23:42.495 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-31T06:23:42.495 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_items": 0, 2024-08-31T06:23:42.495 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-31T06:23:42.495 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_items": 0, 2024-08-31T06:23:42.495 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_bytes": 0, 2024-08-31T06:23:42.495 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_items": 0, 2024-08-31T06:23:42.495 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_bytes": 0, 2024-08-31T06:23:42.495 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_items": 0, 2024-08-31T06:23:42.495 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-31T06:23:42.495 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_items": 0, 2024-08-31T06:23:42.495 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_bytes": 0, 2024-08-31T06:23:42.495 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_items": 0, 2024-08-31T06:23:42.495 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_bytes": 0, 2024-08-31T06:23:42.495 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_items": 0, 2024-08-31T06:23:42.496 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_bytes": 0, 2024-08-31T06:23:42.496 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_items": 0, 2024-08-31T06:23:42.496 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_bytes": 0, 2024-08-31T06:23:42.496 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_items": 0, 2024-08-31T06:23:42.496 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_bytes": 206709, 2024-08-31T06:23:42.496 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_items": 714, 2024-08-31T06:23:42.496 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_bytes": 0, 2024-08-31T06:23:42.496 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_items": 0, 2024-08-31T06:23:42.496 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_bytes": 0, 2024-08-31T06:23:42.496 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_items": 0, 2024-08-31T06:23:42.496 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_bytes": 0, 2024-08-31T06:23:42.496 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_items": 0, 2024-08-31T06:23:42.496 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_bytes": 0, 2024-08-31T06:23:42.496 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_items": 0, 2024-08-31T06:23:42.496 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_bytes": 8064, 2024-08-31T06:23:42.497 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_items": 178, 2024-08-31T06:23:42.497 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_bytes": 0, 2024-08-31T06:23:42.497 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_items": 0, 2024-08-31T06:23:42.497 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_bytes": 0, 2024-08-31T06:23:42.497 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_items": 0, 2024-08-31T06:23:42.497 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_bytes": 2222580, 2024-08-31T06:23:42.497 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_items": 34791, 2024-08-31T06:23:42.497 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_bytes": 0, 2024-08-31T06:23:42.497 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_items": 0, 2024-08-31T06:23:42.497 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_bytes": 0, 2024-08-31T06:23:42.497 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_items": 0 2024-08-31T06:23:42.497 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.497 INFO:teuthology.orchestra.run.smithi184.stdout: "objecter": { 2024-08-31T06:23:42.497 INFO:teuthology.orchestra.run.smithi184.stdout: "op_active": 0, 2024-08-31T06:23:42.497 INFO:teuthology.orchestra.run.smithi184.stdout: "op_laggy": 0, 2024-08-31T06:23:42.498 INFO:teuthology.orchestra.run.smithi184.stdout: "op_send": 3581, 2024-08-31T06:23:42.498 INFO:teuthology.orchestra.run.smithi184.stdout: "op_send_bytes": 21, 2024-08-31T06:23:42.498 INFO:teuthology.orchestra.run.smithi184.stdout: "op_resend": 3, 2024-08-31T06:23:42.498 INFO:teuthology.orchestra.run.smithi184.stdout: "op_reply": 3581, 2024-08-31T06:23:42.498 INFO:teuthology.orchestra.run.smithi184.stdout: "op_latency": { 2024-08-31T06:23:42.498 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 3581, 2024-08-31T06:23:42.498 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 3.955916736, 2024-08-31T06:23:42.498 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.001104696 2024-08-31T06:23:42.498 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.498 INFO:teuthology.orchestra.run.smithi184.stdout: "op_inflight": 0, 2024-08-31T06:23:42.498 INFO:teuthology.orchestra.run.smithi184.stdout: "oplen_avg": { 2024-08-31T06:23:42.498 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 3581, 2024-08-31T06:23:42.498 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 3582 2024-08-31T06:23:42.498 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.499 INFO:teuthology.orchestra.run.smithi184.stdout: "op": 3581, 2024-08-31T06:23:42.499 INFO:teuthology.orchestra.run.smithi184.stdout: "op_r": 3581, 2024-08-31T06:23:42.499 INFO:teuthology.orchestra.run.smithi184.stdout: "op_w": 0, 2024-08-31T06:23:42.499 INFO:teuthology.orchestra.run.smithi184.stdout: "op_rmw": 0, 2024-08-31T06:23:42.499 INFO:teuthology.orchestra.run.smithi184.stdout: "op_pg": 0, 2024-08-31T06:23:42.499 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_stat": 2173, 2024-08-31T06:23:42.499 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_create": 0, 2024-08-31T06:23:42.499 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_read": 1406, 2024-08-31T06:23:42.499 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_write": 0, 2024-08-31T06:23:42.499 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_writefull": 0, 2024-08-31T06:23:42.499 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_writesame": 0, 2024-08-31T06:23:42.499 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_append": 0, 2024-08-31T06:23:42.499 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_zero": 0, 2024-08-31T06:23:42.499 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_truncate": 0, 2024-08-31T06:23:42.499 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_delete": 0, 2024-08-31T06:23:42.500 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_mapext": 0, 2024-08-31T06:23:42.500 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_sparse_read": 0, 2024-08-31T06:23:42.500 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_clonerange": 0, 2024-08-31T06:23:42.500 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_getxattr": 1, 2024-08-31T06:23:42.500 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_setxattr": 0, 2024-08-31T06:23:42.500 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_cmpxattr": 0, 2024-08-31T06:23:42.500 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_rmxattr": 0, 2024-08-31T06:23:42.500 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_resetxattrs": 0, 2024-08-31T06:23:42.500 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_call": 0, 2024-08-31T06:23:42.500 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_watch": 0, 2024-08-31T06:23:42.500 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_notify": 0, 2024-08-31T06:23:42.500 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_src_cmpxattr": 0, 2024-08-31T06:23:42.500 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_pgls": 0, 2024-08-31T06:23:42.500 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_pgls_filter": 0, 2024-08-31T06:23:42.501 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_other": 0, 2024-08-31T06:23:42.501 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_active": 0, 2024-08-31T06:23:42.501 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_send": 0, 2024-08-31T06:23:42.501 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_resend": 0, 2024-08-31T06:23:42.501 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_ping": 0, 2024-08-31T06:23:42.501 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_active": 0, 2024-08-31T06:23:42.501 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_send": 0, 2024-08-31T06:23:42.501 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_resend": 0, 2024-08-31T06:23:42.501 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_active": 0, 2024-08-31T06:23:42.501 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_send": 0, 2024-08-31T06:23:42.501 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_resend": 0, 2024-08-31T06:23:42.501 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_active": 0, 2024-08-31T06:23:42.501 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_send": 0, 2024-08-31T06:23:42.501 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_resend": 0, 2024-08-31T06:23:42.501 INFO:teuthology.orchestra.run.smithi184.stdout: "command_active": 0, 2024-08-31T06:23:42.502 INFO:teuthology.orchestra.run.smithi184.stdout: "command_send": 0, 2024-08-31T06:23:42.502 INFO:teuthology.orchestra.run.smithi184.stdout: "command_resend": 0, 2024-08-31T06:23:42.502 INFO:teuthology.orchestra.run.smithi184.stdout: "map_epoch": 103, 2024-08-31T06:23:42.502 INFO:teuthology.orchestra.run.smithi184.stdout: "map_full": 0, 2024-08-31T06:23:42.502 INFO:teuthology.orchestra.run.smithi184.stdout: "map_inc": 25, 2024-08-31T06:23:42.502 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_sessions": 4, 2024-08-31T06:23:42.502 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_session_open": 5, 2024-08-31T06:23:42.502 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_session_close": 1, 2024-08-31T06:23:42.502 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_laggy": 0, 2024-08-31T06:23:42.502 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_wr": 0, 2024-08-31T06:23:42.502 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_rd": 2, 2024-08-31T06:23:42.502 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_del": 0 2024-08-31T06:23:42.502 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.502 INFO:teuthology.orchestra.run.smithi184.stdout: "oft": { 2024-08-31T06:23:42.503 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_objs": 0, 2024-08-31T06:23:42.503 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_kv_pairs": 0, 2024-08-31T06:23:42.503 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_updates": 0, 2024-08-31T06:23:42.503 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_removes": 0 2024-08-31T06:23:42.503 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.503 INFO:teuthology.orchestra.run.smithi184.stdout: "purge_queue": { 2024-08-31T06:23:42.503 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_ops": 0, 2024-08-31T06:23:42.503 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_ops_high_water": 0, 2024-08-31T06:23:42.503 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing": 0, 2024-08-31T06:23:42.503 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_high_water": 0, 2024-08-31T06:23:42.503 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executed_ops": 0, 2024-08-31T06:23:42.503 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executed": 0, 2024-08-31T06:23:42.503 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_item_in_journal": 0 2024-08-31T06:23:42.503 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.503 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-31T06:23:42.504 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-08-31T06:23:42.504 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2024-08-31T06:23:42.504 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-08-31T06:23:42.504 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 3967, 2024-08-31T06:23:42.504 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 8816933, 2024-08-31T06:23:42.504 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-08-31T06:23:42.504 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 3967, 2024-08-31T06:23:42.504 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-08-31T06:23:42.504 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-08-31T06:23:42.504 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 3967, 2024-08-31T06:23:42.504 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 8816933, 2024-08-31T06:23:42.504 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-08-31T06:23:42.504 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:42.504 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:42.504 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:42.505 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-08-31T06:23:42.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.505 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-objecter_bytes": { 2024-08-31T06:23:42.505 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-08-31T06:23:42.505 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2024-08-31T06:23:42.505 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-08-31T06:23:42.505 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-08-31T06:23:42.505 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-08-31T06:23:42.505 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-08-31T06:23:42.505 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-08-31T06:23:42.505 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 3581, 2024-08-31T06:23:42.505 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 7165644, 2024-08-31T06:23:42.505 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 3581, 2024-08-31T06:23:42.505 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 7165644, 2024-08-31T06:23:42.505 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-08-31T06:23:42.506 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:42.506 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:42.506 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:42.506 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-08-31T06:23:42.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.506 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-objecter_ops": { 2024-08-31T06:23:42.506 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-08-31T06:23:42.506 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 1024, 2024-08-31T06:23:42.506 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-08-31T06:23:42.506 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-08-31T06:23:42.506 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-08-31T06:23:42.506 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-08-31T06:23:42.506 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-08-31T06:23:42.506 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 3581, 2024-08-31T06:23:42.506 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 3581, 2024-08-31T06:23:42.507 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 3581, 2024-08-31T06:23:42.507 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 3581, 2024-08-31T06:23:42.507 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-08-31T06:23:42.507 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:42.507 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:42.507 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:42.507 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-08-31T06:23:42.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.507 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-write_buf_throttle": { 2024-08-31T06:23:42.507 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-08-31T06:23:42.507 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 3758096384, 2024-08-31T06:23:42.507 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-08-31T06:23:42.507 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-08-31T06:23:42.507 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-08-31T06:23:42.507 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-08-31T06:23:42.508 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-08-31T06:23:42.508 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-08-31T06:23:42.508 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-08-31T06:23:42.508 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 0, 2024-08-31T06:23:42.508 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 0, 2024-08-31T06:23:42.508 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-08-31T06:23:42.508 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:42.508 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:42.508 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:42.508 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-08-31T06:23:42.508 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.508 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-write_buf_throttle-0x555b4825c9c0": { 2024-08-31T06:23:42.508 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-08-31T06:23:42.508 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 3758096384, 2024-08-31T06:23:42.509 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-08-31T06:23:42.509 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-08-31T06:23:42.509 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-08-31T06:23:42.509 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-08-31T06:23:42.509 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-08-31T06:23:42.509 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-08-31T06:23:42.509 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-08-31T06:23:42.509 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 0, 2024-08-31T06:23:42.510 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 0, 2024-08-31T06:23:42.510 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-08-31T06:23:42.510 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:42.510 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:42.510 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:42.510 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-08-31T06:23:42.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:42.510 INFO:teuthology.orchestra.run.smithi184.stdout: "trackedop": { 2024-08-31T06:23:42.510 INFO:teuthology.orchestra.run.smithi184.stdout: "slow_ops_count": 0 2024-08-31T06:23:42.511 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-08-31T06:23:42.511 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-08-31T06:23:42.973 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:42 smithi184 ceph-mon[22331]: pgmap v880: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.2 KiB/s wr, 7 op/s 2024-08-31T06:23:43.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:42 smithi155 ceph-mon[21742]: pgmap v880: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.2 KiB/s wr, 7 op/s 2024-08-31T06:23:43.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:42 smithi016 ceph-mon[17194]: pgmap v880: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.2 KiB/s wr, 7 op/s 2024-08-31T06:23:43.555 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-08-31T06:23:43.555 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-08-31T06:23:43.555 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-08-31T06:23:43.555 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-08-31T06:23:43.555 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-08-31T06:23:43.555 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-08-31T06:23:43.555 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-08-31T06:23:43.555 DEBUG:tasks.check_counter:Getting stats from f 2024-08-31T06:23:43.556 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph daemon mds.f perf dump 2024-08-31T06:23:43.806 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.c/config 2024-08-31T06:23:44.386 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-08-31T06:23:44.386 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-0": { 2024-08-31T06:23:44.386 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 385, 2024-08-31T06:23:44.386 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 348, 2024-08-31T06:23:44.386 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 1025922, 2024-08-31T06:23:44.386 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 201316, 2024-08-31T06:23:44.386 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 6, 2024-08-31T06:23:44.386 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 1, 2024-08-31T06:23:44.386 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.058822544, 2024-08-31T06:23:44.387 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.027158419, 2024-08-31T06:23:44.387 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.043237562, 2024-08-31T06:23:44.387 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.000030482, 2024-08-31T06:23:44.387 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T06:23:44.387 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 347, 2024-08-31T06:23:44.387 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.027354259, 2024-08-31T06:23:44.387 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000078830 2024-08-31T06:23:44.387 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.387 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-08-31T06:23:44.387 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:44.387 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:44.387 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:44.388 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.388 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 1025088, 2024-08-31T06:23:44.388 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 200992 2024-08-31T06:23:44.388 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.388 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-1": { 2024-08-31T06:23:44.388 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 1033, 2024-08-31T06:23:44.388 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 1273, 2024-08-31T06:23:44.388 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 270699, 2024-08-31T06:23:44.388 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 835925, 2024-08-31T06:23:44.388 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 6, 2024-08-31T06:23:44.388 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 4, 2024-08-31T06:23:44.388 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.131171999, 2024-08-31T06:23:44.388 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.055608270, 2024-08-31T06:23:44.389 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.025223325, 2024-08-31T06:23:44.389 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.038664303, 2024-08-31T06:23:44.389 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T06:23:44.389 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 1271, 2024-08-31T06:23:44.389 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.063280796, 2024-08-31T06:23:44.389 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000049788 2024-08-31T06:23:44.389 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.389 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-08-31T06:23:44.389 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:44.389 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:44.389 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:44.389 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.389 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 160, 2024-08-31T06:23:44.389 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 507744 2024-08-31T06:23:44.390 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.390 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-2": { 2024-08-31T06:23:44.390 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 2512, 2024-08-31T06:23:44.390 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 2512, 2024-08-31T06:23:44.390 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 4939329, 2024-08-31T06:23:44.390 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 798050, 2024-08-31T06:23:44.390 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 4, 2024-08-31T06:23:44.390 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 3, 2024-08-31T06:23:44.390 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.285434991, 2024-08-31T06:23:44.390 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.112934690, 2024-08-31T06:23:44.390 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.070565979, 2024-08-31T06:23:44.390 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.081840738, 2024-08-31T06:23:44.390 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T06:23:44.390 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 2511, 2024-08-31T06:23:44.391 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.099571698, 2024-08-31T06:23:44.391 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000039654 2024-08-31T06:23:44.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.391 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-08-31T06:23:44.391 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:44.391 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:44.391 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:44.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.391 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-08-31T06:23:44.391 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 448 2024-08-31T06:23:44.391 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.391 INFO:teuthology.orchestra.run.smithi184.stdout: "cct": { 2024-08-31T06:23:44.391 INFO:teuthology.orchestra.run.smithi184.stdout: "total_workers": 1, 2024-08-31T06:23:44.391 INFO:teuthology.orchestra.run.smithi184.stdout: "unhealthy_workers": 0 2024-08-31T06:23:44.392 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.392 INFO:teuthology.orchestra.run.smithi184.stdout: "finisher-MDSRank": { 2024-08-31T06:23:44.392 INFO:teuthology.orchestra.run.smithi184.stdout: "queue_len": 0, 2024-08-31T06:23:44.392 INFO:teuthology.orchestra.run.smithi184.stdout: "complete_latency": { 2024-08-31T06:23:44.392 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 5956, 2024-08-31T06:23:44.392 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.171408340, 2024-08-31T06:23:44.392 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000028779 2024-08-31T06:23:44.392 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-08-31T06:23:44.392 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.392 INFO:teuthology.orchestra.run.smithi184.stdout: "finisher-PurgeQueue": { 2024-08-31T06:23:44.392 INFO:teuthology.orchestra.run.smithi184.stdout: "queue_len": 0, 2024-08-31T06:23:44.392 INFO:teuthology.orchestra.run.smithi184.stdout: "complete_latency": { 2024-08-31T06:23:44.392 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:44.392 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:44.393 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:44.393 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-08-31T06:23:44.393 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.393 INFO:teuthology.orchestra.run.smithi184.stdout: "mds": { 2024-08-31T06:23:44.393 INFO:teuthology.orchestra.run.smithi184.stdout: "request": 0, 2024-08-31T06:23:44.393 INFO:teuthology.orchestra.run.smithi184.stdout: "reply": 0, 2024-08-31T06:23:44.393 INFO:teuthology.orchestra.run.smithi184.stdout: "reply_latency": { 2024-08-31T06:23:44.393 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:44.393 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:44.393 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:44.393 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.393 INFO:teuthology.orchestra.run.smithi184.stdout: "slow_reply": 0, 2024-08-31T06:23:44.393 INFO:teuthology.orchestra.run.smithi184.stdout: "forward": 0, 2024-08-31T06:23:44.393 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_fetch_complete": 0, 2024-08-31T06:23:44.394 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_fetch_keys": 0, 2024-08-31T06:23:44.394 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_commit": 0, 2024-08-31T06:23:44.394 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_split": 0, 2024-08-31T06:23:44.394 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_merge": 0, 2024-08-31T06:23:44.394 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes": 770, 2024-08-31T06:23:44.394 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_top": 62, 2024-08-31T06:23:44.394 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_bottom": 708, 2024-08-31T06:23:44.394 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_pin_tail": 0, 2024-08-31T06:23:44.394 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_pinned": 681, 2024-08-31T06:23:44.395 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_expired": 0, 2024-08-31T06:23:44.395 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_with_caps": 0, 2024-08-31T06:23:44.395 INFO:teuthology.orchestra.run.smithi184.stdout: "caps": 0, 2024-08-31T06:23:44.395 INFO:teuthology.orchestra.run.smithi184.stdout: "subtrees": 41, 2024-08-31T06:23:44.395 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse": 0, 2024-08-31T06:23:44.395 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_hit": 0, 2024-08-31T06:23:44.395 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_forward": 0, 2024-08-31T06:23:44.395 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_discover": 0, 2024-08-31T06:23:44.395 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_dir_fetch": 0, 2024-08-31T06:23:44.395 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_remote_ino": 0, 2024-08-31T06:23:44.395 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_lock": 0, 2024-08-31T06:23:44.395 INFO:teuthology.orchestra.run.smithi184.stdout: "load_cent": 0, 2024-08-31T06:23:44.395 INFO:teuthology.orchestra.run.smithi184.stdout: "q": 0, 2024-08-31T06:23:44.395 INFO:teuthology.orchestra.run.smithi184.stdout: "exported": 0, 2024-08-31T06:23:44.396 INFO:teuthology.orchestra.run.smithi184.stdout: "exported_inodes": 0, 2024-08-31T06:23:44.396 INFO:teuthology.orchestra.run.smithi184.stdout: "imported": 0, 2024-08-31T06:23:44.396 INFO:teuthology.orchestra.run.smithi184.stdout: "imported_inodes": 0, 2024-08-31T06:23:44.396 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_dir_fetch": 0, 2024-08-31T06:23:44.396 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_backtrace_fetch": 0, 2024-08-31T06:23:44.396 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_peer_discover": 0, 2024-08-31T06:23:44.396 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rfiles": 0, 2024-08-31T06:23:44.396 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rbytes": 0, 2024-08-31T06:23:44.396 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rsnaps": 0, 2024-08-31T06:23:44.396 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_backtrace_fetch": 0, 2024-08-31T06:23:44.396 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_set_tag": 0, 2024-08-31T06:23:44.396 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_backtrace_repaired": 0, 2024-08-31T06:23:44.396 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_inotable_repaired": 0, 2024-08-31T06:23:44.396 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dir_inodes": 0, 2024-08-31T06:23:44.397 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dir_base_inodes": 0, 2024-08-31T06:23:44.397 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dirfrag_rstats": 0, 2024-08-31T06:23:44.397 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_file_inodes": 0, 2024-08-31T06:23:44.397 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_inode_file_caps": 0, 2024-08-31T06:23:44.397 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_revoke": 0, 2024-08-31T06:23:44.397 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_grant": 0, 2024-08-31T06:23:44.397 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_trunc": 0, 2024-08-31T06:23:44.397 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-31T06:23:44.397 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-31T06:23:44.397 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_caps": 0, 2024-08-31T06:23:44.397 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_caps_dirty": 0, 2024-08-31T06:23:44.397 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_cap_release": 0, 2024-08-31T06:23:44.397 INFO:teuthology.orchestra.run.smithi184.stdout: "process_request_cap_release": 0 2024-08-31T06:23:44.397 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.398 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_cache": { 2024-08-31T06:23:44.398 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_update": 0, 2024-08-31T06:23:44.398 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_update_receipt": 0, 2024-08-31T06:23:44.398 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_try_discover": 0, 2024-08-31T06:23:44.398 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_send_discover": 0, 2024-08-31T06:23:44.398 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_handle_discover": 0, 2024-08-31T06:23:44.398 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays": 0, 2024-08-31T06:23:44.398 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays_delayed": 0, 2024-08-31T06:23:44.398 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays_enqueuing": 0, 2024-08-31T06:23:44.398 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_created": 0, 2024-08-31T06:23:44.398 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_enqueued": 0, 2024-08-31T06:23:44.398 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_reintegrated": 0, 2024-08-31T06:23:44.398 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_migrated": 0, 2024-08-31T06:23:44.398 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_processing": 0, 2024-08-31T06:23:44.399 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_enqueued": 0, 2024-08-31T06:23:44.399 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_prioritized": 0, 2024-08-31T06:23:44.399 INFO:teuthology.orchestra.run.smithi184.stdout: "recovery_started": 0, 2024-08-31T06:23:44.399 INFO:teuthology.orchestra.run.smithi184.stdout: "recovery_completed": 0, 2024-08-31T06:23:44.399 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_quiesce_path": 0, 2024-08-31T06:23:44.399 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_quiesce_inode": 0, 2024-08-31T06:23:44.399 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_enqueue_scrub": 0, 2024-08-31T06:23:44.399 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_exportdir": 0, 2024-08-31T06:23:44.399 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_flush": 0, 2024-08-31T06:23:44.399 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_fragmentdir": 0, 2024-08-31T06:23:44.399 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_fragstats": 0, 2024-08-31T06:23:44.399 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_inodestats": 0, 2024-08-31T06:23:44.399 INFO:teuthology.orchestra.run.smithi184.stdout: "uninline_started": 0, 2024-08-31T06:23:44.399 INFO:teuthology.orchestra.run.smithi184.stdout: "uninline_succeeded": 0, 2024-08-31T06:23:44.400 INFO:teuthology.orchestra.run.smithi184.stdout: "uninline_write_failed": 0 2024-08-31T06:23:44.400 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.400 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_log": { 2024-08-31T06:23:44.400 INFO:teuthology.orchestra.run.smithi184.stdout: "evlrg": 0, 2024-08-31T06:23:44.400 INFO:teuthology.orchestra.run.smithi184.stdout: "evadd": 0, 2024-08-31T06:23:44.400 INFO:teuthology.orchestra.run.smithi184.stdout: "evex": 0, 2024-08-31T06:23:44.400 INFO:teuthology.orchestra.run.smithi184.stdout: "evtrm": 0, 2024-08-31T06:23:44.400 INFO:teuthology.orchestra.run.smithi184.stdout: "ev": 2959, 2024-08-31T06:23:44.400 INFO:teuthology.orchestra.run.smithi184.stdout: "evexg": 0, 2024-08-31T06:23:44.400 INFO:teuthology.orchestra.run.smithi184.stdout: "evexd": 0, 2024-08-31T06:23:44.400 INFO:teuthology.orchestra.run.smithi184.stdout: "segadd": 0, 2024-08-31T06:23:44.400 INFO:teuthology.orchestra.run.smithi184.stdout: "segex": 0, 2024-08-31T06:23:44.400 INFO:teuthology.orchestra.run.smithi184.stdout: "segtrm": 0, 2024-08-31T06:23:44.400 INFO:teuthology.orchestra.run.smithi184.stdout: "seg": 5, 2024-08-31T06:23:44.401 INFO:teuthology.orchestra.run.smithi184.stdout: "segmjr": 2, 2024-08-31T06:23:44.401 INFO:teuthology.orchestra.run.smithi184.stdout: "segexg": 0, 2024-08-31T06:23:44.401 INFO:teuthology.orchestra.run.smithi184.stdout: "segexd": 0, 2024-08-31T06:23:44.401 INFO:teuthology.orchestra.run.smithi184.stdout: "expos": 4194304, 2024-08-31T06:23:44.401 INFO:teuthology.orchestra.run.smithi184.stdout: "wrpos": 8405221, 2024-08-31T06:23:44.401 INFO:teuthology.orchestra.run.smithi184.stdout: "rdpos": 8400744, 2024-08-31T06:23:44.401 INFO:teuthology.orchestra.run.smithi184.stdout: "jlat": { 2024-08-31T06:23:44.401 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:44.401 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:44.401 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:44.401 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.401 INFO:teuthology.orchestra.run.smithi184.stdout: "replayed": 2959 2024-08-31T06:23:44.401 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.401 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_mem": { 2024-08-31T06:23:44.401 INFO:teuthology.orchestra.run.smithi184.stdout: "ino": 658, 2024-08-31T06:23:44.402 INFO:teuthology.orchestra.run.smithi184.stdout: "ino+": 959, 2024-08-31T06:23:44.402 INFO:teuthology.orchestra.run.smithi184.stdout: "ino-": 301, 2024-08-31T06:23:44.402 INFO:teuthology.orchestra.run.smithi184.stdout: "dir": 51, 2024-08-31T06:23:44.402 INFO:teuthology.orchestra.run.smithi184.stdout: "dir+": 67, 2024-08-31T06:23:44.402 INFO:teuthology.orchestra.run.smithi184.stdout: "dir-": 16, 2024-08-31T06:23:44.402 INFO:teuthology.orchestra.run.smithi184.stdout: "dn": 770, 2024-08-31T06:23:44.402 INFO:teuthology.orchestra.run.smithi184.stdout: "dn+": 1063, 2024-08-31T06:23:44.402 INFO:teuthology.orchestra.run.smithi184.stdout: "dn-": 293, 2024-08-31T06:23:44.402 INFO:teuthology.orchestra.run.smithi184.stdout: "cap": 0, 2024-08-31T06:23:44.402 INFO:teuthology.orchestra.run.smithi184.stdout: "cap+": 0, 2024-08-31T06:23:44.402 INFO:teuthology.orchestra.run.smithi184.stdout: "cap-": 0, 2024-08-31T06:23:44.402 INFO:teuthology.orchestra.run.smithi184.stdout: "rss": 120820, 2024-08-31T06:23:44.402 INFO:teuthology.orchestra.run.smithi184.stdout: "heap": 321662976 2024-08-31T06:23:44.402 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.402 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_server": { 2024-08-31T06:23:44.403 INFO:teuthology.orchestra.run.smithi184.stdout: "dispatch_client_request": 0, 2024-08-31T06:23:44.403 INFO:teuthology.orchestra.run.smithi184.stdout: "dispatch_server_request": 0, 2024-08-31T06:23:44.403 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_request": 0, 2024-08-31T06:23:44.403 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_session": 0, 2024-08-31T06:23:44.403 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_peer_request": 0, 2024-08-31T06:23:44.403 INFO:teuthology.orchestra.run.smithi184.stdout: "req_create_latency": { 2024-08-31T06:23:44.403 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:44.403 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:44.403 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:44.403 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.403 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getattr_latency": { 2024-08-31T06:23:44.403 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:44.403 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:44.403 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:44.404 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.404 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getfilelock_latency": { 2024-08-31T06:23:44.404 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:44.404 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:44.404 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:44.404 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.404 INFO:teuthology.orchestra.run.smithi184.stdout: "req_link_latency": { 2024-08-31T06:23:44.404 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:44.404 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:44.404 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:44.404 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.404 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookup_latency": { 2024-08-31T06:23:44.404 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:44.404 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:44.404 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:44.405 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.405 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookuphash_latency": { 2024-08-31T06:23:44.405 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:44.405 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:44.405 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:44.405 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.405 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupino_latency": { 2024-08-31T06:23:44.405 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:44.405 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:44.405 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:44.405 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.405 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupname_latency": { 2024-08-31T06:23:44.405 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:44.405 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:44.406 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:44.406 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.406 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupparent_latency": { 2024-08-31T06:23:44.406 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:44.406 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:44.406 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:44.406 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.406 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupsnap_latency": { 2024-08-31T06:23:44.406 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:44.406 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:44.406 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:44.406 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.406 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lssnap_latency": { 2024-08-31T06:23:44.406 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:44.407 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:44.407 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:44.407 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.407 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mkdir_latency": { 2024-08-31T06:23:44.407 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:44.407 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:44.408 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:44.408 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.408 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mknod_latency": { 2024-08-31T06:23:44.408 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:44.408 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:44.408 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:44.408 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.408 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mksnap_latency": { 2024-08-31T06:23:44.408 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:44.408 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:44.408 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:44.408 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.408 INFO:teuthology.orchestra.run.smithi184.stdout: "req_open_latency": { 2024-08-31T06:23:44.409 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:44.409 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:44.409 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:44.409 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.409 INFO:teuthology.orchestra.run.smithi184.stdout: "req_readdir_latency": { 2024-08-31T06:23:44.409 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:44.409 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:44.409 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:44.409 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.409 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rename_latency": { 2024-08-31T06:23:44.409 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:44.409 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:44.409 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:44.409 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.409 INFO:teuthology.orchestra.run.smithi184.stdout: "req_renamesnap_latency": { 2024-08-31T06:23:44.410 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:44.410 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:44.410 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:44.410 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.410 INFO:teuthology.orchestra.run.smithi184.stdout: "req_snapdiff_latency": { 2024-08-31T06:23:44.410 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:44.410 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:44.410 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:44.410 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.410 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmdir_latency": { 2024-08-31T06:23:44.410 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:44.410 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:44.410 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:44.410 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.411 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmsnap_latency": { 2024-08-31T06:23:44.411 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:44.411 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:44.411 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:44.411 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.411 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmxattr_latency": { 2024-08-31T06:23:44.411 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:44.411 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:44.411 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:44.411 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.411 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setattr_latency": { 2024-08-31T06:23:44.411 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:44.411 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:44.411 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:44.412 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.412 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setdirlayout_latency": { 2024-08-31T06:23:44.412 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:44.412 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:44.412 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:44.412 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.412 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setfilelock_latency": { 2024-08-31T06:23:44.412 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:44.412 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:44.412 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:44.412 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.412 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setlayout_latency": { 2024-08-31T06:23:44.412 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:44.413 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:44.413 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:44.413 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.413 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setxattr_latency": { 2024-08-31T06:23:44.413 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:44.413 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:44.413 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:44.413 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.413 INFO:teuthology.orchestra.run.smithi184.stdout: "req_symlink_latency": { 2024-08-31T06:23:44.413 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:44.413 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:44.413 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:44.413 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.413 INFO:teuthology.orchestra.run.smithi184.stdout: "req_unlink_latency": { 2024-08-31T06:23:44.414 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:44.414 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:44.414 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:44.414 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.414 INFO:teuthology.orchestra.run.smithi184.stdout: "cap_revoke_eviction": 0, 2024-08-31T06:23:44.414 INFO:teuthology.orchestra.run.smithi184.stdout: "cap_acquisition_throttle": 0, 2024-08-31T06:23:44.414 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getvxattr_latency": { 2024-08-31T06:23:44.414 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:44.414 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:44.414 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:44.414 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-08-31T06:23:44.414 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.414 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_sessions": { 2024-08-31T06:23:44.414 INFO:teuthology.orchestra.run.smithi184.stdout: "session_count": 2, 2024-08-31T06:23:44.415 INFO:teuthology.orchestra.run.smithi184.stdout: "session_add": 3, 2024-08-31T06:23:44.415 INFO:teuthology.orchestra.run.smithi184.stdout: "session_remove": 1, 2024-08-31T06:23:44.415 INFO:teuthology.orchestra.run.smithi184.stdout: "sessions_open": 2, 2024-08-31T06:23:44.415 INFO:teuthology.orchestra.run.smithi184.stdout: "sessions_stale": 0, 2024-08-31T06:23:44.415 INFO:teuthology.orchestra.run.smithi184.stdout: "total_load": 0, 2024-08-31T06:23:44.415 INFO:teuthology.orchestra.run.smithi184.stdout: "average_load": 0, 2024-08-31T06:23:44.415 INFO:teuthology.orchestra.run.smithi184.stdout: "avg_session_uptime": 2278, 2024-08-31T06:23:44.415 INFO:teuthology.orchestra.run.smithi184.stdout: "mdthresh_evicted": 0 2024-08-31T06:23:44.415 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.415 INFO:teuthology.orchestra.run.smithi184.stdout: "mempool": { 2024-08-31T06:23:44.415 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_bytes": 0, 2024-08-31T06:23:44.415 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_items": 0, 2024-08-31T06:23:44.415 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_bytes": 0, 2024-08-31T06:23:44.415 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_items": 0, 2024-08-31T06:23:44.416 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_bytes": 0, 2024-08-31T06:23:44.416 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_items": 0, 2024-08-31T06:23:44.416 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-31T06:23:44.416 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_items": 0, 2024-08-31T06:23:44.416 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-31T06:23:44.416 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_items": 0, 2024-08-31T06:23:44.416 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_bytes": 0, 2024-08-31T06:23:44.416 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_items": 0, 2024-08-31T06:23:44.416 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-31T06:23:44.416 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_items": 0, 2024-08-31T06:23:44.416 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_bytes": 0, 2024-08-31T06:23:44.416 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_items": 0, 2024-08-31T06:23:44.416 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_bytes": 0, 2024-08-31T06:23:44.416 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_items": 0, 2024-08-31T06:23:44.417 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-31T06:23:44.417 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_items": 0, 2024-08-31T06:23:44.417 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-31T06:23:44.417 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_items": 0, 2024-08-31T06:23:44.417 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_bytes": 0, 2024-08-31T06:23:44.417 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_items": 0, 2024-08-31T06:23:44.417 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_bytes": 0, 2024-08-31T06:23:44.417 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_items": 0, 2024-08-31T06:23:44.417 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-31T06:23:44.417 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_items": 0, 2024-08-31T06:23:44.417 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_bytes": 0, 2024-08-31T06:23:44.417 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_items": 0, 2024-08-31T06:23:44.417 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_bytes": 0, 2024-08-31T06:23:44.417 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_items": 0, 2024-08-31T06:23:44.417 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_bytes": 0, 2024-08-31T06:23:44.418 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_items": 0, 2024-08-31T06:23:44.418 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_bytes": 0, 2024-08-31T06:23:44.418 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_items": 0, 2024-08-31T06:23:44.418 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_bytes": 216307, 2024-08-31T06:23:44.418 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_items": 759, 2024-08-31T06:23:44.418 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_bytes": 0, 2024-08-31T06:23:44.418 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_items": 0, 2024-08-31T06:23:44.418 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_bytes": 0, 2024-08-31T06:23:44.418 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_items": 0, 2024-08-31T06:23:44.418 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_bytes": 0, 2024-08-31T06:23:44.418 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_items": 0, 2024-08-31T06:23:44.418 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_bytes": 0, 2024-08-31T06:23:44.418 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_items": 0, 2024-08-31T06:23:44.418 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_bytes": 8064, 2024-08-31T06:23:44.419 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_items": 178, 2024-08-31T06:23:44.419 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_bytes": 0, 2024-08-31T06:23:44.419 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_items": 0, 2024-08-31T06:23:44.419 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_bytes": 0, 2024-08-31T06:23:44.419 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_items": 0, 2024-08-31T06:23:44.419 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_bytes": 2173346, 2024-08-31T06:23:44.419 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_items": 33334, 2024-08-31T06:23:44.419 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_bytes": 0, 2024-08-31T06:23:44.419 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_items": 0, 2024-08-31T06:23:44.419 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_bytes": 0, 2024-08-31T06:23:44.419 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_items": 0 2024-08-31T06:23:44.419 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.419 INFO:teuthology.orchestra.run.smithi184.stdout: "objecter": { 2024-08-31T06:23:44.419 INFO:teuthology.orchestra.run.smithi184.stdout: "op_active": 0, 2024-08-31T06:23:44.419 INFO:teuthology.orchestra.run.smithi184.stdout: "op_laggy": 0, 2024-08-31T06:23:44.420 INFO:teuthology.orchestra.run.smithi184.stdout: "op_send": 3541, 2024-08-31T06:23:44.420 INFO:teuthology.orchestra.run.smithi184.stdout: "op_send_bytes": 21, 2024-08-31T06:23:44.420 INFO:teuthology.orchestra.run.smithi184.stdout: "op_resend": 3, 2024-08-31T06:23:44.420 INFO:teuthology.orchestra.run.smithi184.stdout: "op_reply": 3541, 2024-08-31T06:23:44.420 INFO:teuthology.orchestra.run.smithi184.stdout: "op_latency": { 2024-08-31T06:23:44.420 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 3541, 2024-08-31T06:23:44.420 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 12.421740819, 2024-08-31T06:23:44.420 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.003507975 2024-08-31T06:23:44.420 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.420 INFO:teuthology.orchestra.run.smithi184.stdout: "op_inflight": 0, 2024-08-31T06:23:44.420 INFO:teuthology.orchestra.run.smithi184.stdout: "oplen_avg": { 2024-08-31T06:23:44.420 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 3541, 2024-08-31T06:23:44.420 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 3542 2024-08-31T06:23:44.420 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.421 INFO:teuthology.orchestra.run.smithi184.stdout: "op": 3541, 2024-08-31T06:23:44.421 INFO:teuthology.orchestra.run.smithi184.stdout: "op_r": 3541, 2024-08-31T06:23:44.421 INFO:teuthology.orchestra.run.smithi184.stdout: "op_w": 0, 2024-08-31T06:23:44.421 INFO:teuthology.orchestra.run.smithi184.stdout: "op_rmw": 0, 2024-08-31T06:23:44.421 INFO:teuthology.orchestra.run.smithi184.stdout: "op_pg": 0, 2024-08-31T06:23:44.421 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_stat": 2161, 2024-08-31T06:23:44.421 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_create": 0, 2024-08-31T06:23:44.421 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_read": 1378, 2024-08-31T06:23:44.421 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_write": 0, 2024-08-31T06:23:44.421 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_writefull": 0, 2024-08-31T06:23:44.421 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_writesame": 0, 2024-08-31T06:23:44.421 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_append": 0, 2024-08-31T06:23:44.421 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_zero": 0, 2024-08-31T06:23:44.421 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_truncate": 0, 2024-08-31T06:23:44.421 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_delete": 0, 2024-08-31T06:23:44.422 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_mapext": 0, 2024-08-31T06:23:44.422 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_sparse_read": 0, 2024-08-31T06:23:44.422 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_clonerange": 0, 2024-08-31T06:23:44.422 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_getxattr": 1, 2024-08-31T06:23:44.422 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_setxattr": 0, 2024-08-31T06:23:44.422 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_cmpxattr": 0, 2024-08-31T06:23:44.422 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_rmxattr": 0, 2024-08-31T06:23:44.422 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_resetxattrs": 0, 2024-08-31T06:23:44.422 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_call": 0, 2024-08-31T06:23:44.422 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_watch": 0, 2024-08-31T06:23:44.422 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_notify": 0, 2024-08-31T06:23:44.422 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_src_cmpxattr": 0, 2024-08-31T06:23:44.422 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_pgls": 0, 2024-08-31T06:23:44.422 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_pgls_filter": 0, 2024-08-31T06:23:44.422 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_other": 0, 2024-08-31T06:23:44.423 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_active": 0, 2024-08-31T06:23:44.423 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_send": 0, 2024-08-31T06:23:44.423 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_resend": 0, 2024-08-31T06:23:44.423 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_ping": 0, 2024-08-31T06:23:44.423 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_active": 0, 2024-08-31T06:23:44.423 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_send": 0, 2024-08-31T06:23:44.423 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_resend": 0, 2024-08-31T06:23:44.423 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_active": 0, 2024-08-31T06:23:44.423 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_send": 0, 2024-08-31T06:23:44.423 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_resend": 0, 2024-08-31T06:23:44.423 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_active": 0, 2024-08-31T06:23:44.423 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_send": 0, 2024-08-31T06:23:44.423 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_resend": 0, 2024-08-31T06:23:44.423 INFO:teuthology.orchestra.run.smithi184.stdout: "command_active": 0, 2024-08-31T06:23:44.423 INFO:teuthology.orchestra.run.smithi184.stdout: "command_send": 0, 2024-08-31T06:23:44.424 INFO:teuthology.orchestra.run.smithi184.stdout: "command_resend": 0, 2024-08-31T06:23:44.424 INFO:teuthology.orchestra.run.smithi184.stdout: "map_epoch": 103, 2024-08-31T06:23:44.424 INFO:teuthology.orchestra.run.smithi184.stdout: "map_full": 0, 2024-08-31T06:23:44.424 INFO:teuthology.orchestra.run.smithi184.stdout: "map_inc": 25, 2024-08-31T06:23:44.424 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_sessions": 5, 2024-08-31T06:23:44.424 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_session_open": 7, 2024-08-31T06:23:44.424 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_session_close": 2, 2024-08-31T06:23:44.424 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_laggy": 0, 2024-08-31T06:23:44.424 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_wr": 0, 2024-08-31T06:23:44.424 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_rd": 2, 2024-08-31T06:23:44.424 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_del": 0 2024-08-31T06:23:44.424 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.424 INFO:teuthology.orchestra.run.smithi184.stdout: "oft": { 2024-08-31T06:23:44.424 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_objs": 0, 2024-08-31T06:23:44.425 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_kv_pairs": 0, 2024-08-31T06:23:44.425 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_updates": 0, 2024-08-31T06:23:44.425 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_removes": 0 2024-08-31T06:23:44.425 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.425 INFO:teuthology.orchestra.run.smithi184.stdout: "purge_queue": { 2024-08-31T06:23:44.425 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_ops": 0, 2024-08-31T06:23:44.425 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_ops_high_water": 0, 2024-08-31T06:23:44.425 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing": 0, 2024-08-31T06:23:44.425 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_high_water": 0, 2024-08-31T06:23:44.425 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executed_ops": 0, 2024-08-31T06:23:44.425 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executed": 0, 2024-08-31T06:23:44.425 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_item_in_journal": 0 2024-08-31T06:23:44.425 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.425 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-31T06:23:44.426 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-08-31T06:23:44.426 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2024-08-31T06:23:44.426 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-08-31T06:23:44.426 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 3926, 2024-08-31T06:23:44.426 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 5860100, 2024-08-31T06:23:44.426 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-08-31T06:23:44.426 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 3926, 2024-08-31T06:23:44.426 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-08-31T06:23:44.426 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-08-31T06:23:44.426 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 3926, 2024-08-31T06:23:44.426 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 5860100, 2024-08-31T06:23:44.426 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-08-31T06:23:44.426 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:44.426 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:44.426 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:44.427 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-08-31T06:23:44.427 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.427 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-objecter_bytes": { 2024-08-31T06:23:44.427 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-08-31T06:23:44.427 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2024-08-31T06:23:44.427 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-08-31T06:23:44.427 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-08-31T06:23:44.427 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-08-31T06:23:44.427 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-08-31T06:23:44.427 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-08-31T06:23:44.427 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 3541, 2024-08-31T06:23:44.427 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 4215896, 2024-08-31T06:23:44.427 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 3541, 2024-08-31T06:23:44.427 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 4215896, 2024-08-31T06:23:44.428 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-08-31T06:23:44.428 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:44.428 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:44.428 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:44.428 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-08-31T06:23:44.428 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.428 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-objecter_ops": { 2024-08-31T06:23:44.428 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-08-31T06:23:44.428 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 1024, 2024-08-31T06:23:44.428 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-08-31T06:23:44.428 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-08-31T06:23:44.428 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-08-31T06:23:44.428 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-08-31T06:23:44.428 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-08-31T06:23:44.429 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 3541, 2024-08-31T06:23:44.429 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 3541, 2024-08-31T06:23:44.429 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 3541, 2024-08-31T06:23:44.429 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 3541, 2024-08-31T06:23:44.429 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-08-31T06:23:44.429 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:44.429 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:44.429 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:44.429 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-08-31T06:23:44.429 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.429 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-write_buf_throttle": { 2024-08-31T06:23:44.429 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-08-31T06:23:44.429 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 3758096384, 2024-08-31T06:23:44.429 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-08-31T06:23:44.429 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-08-31T06:23:44.430 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-08-31T06:23:44.430 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-08-31T06:23:44.430 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-08-31T06:23:44.430 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-08-31T06:23:44.430 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-08-31T06:23:44.430 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 0, 2024-08-31T06:23:44.430 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 0, 2024-08-31T06:23:44.430 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-08-31T06:23:44.430 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:44.430 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:44.430 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:44.430 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-08-31T06:23:44.430 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.430 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-write_buf_throttle-0x55a8a67148f0": { 2024-08-31T06:23:44.430 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-08-31T06:23:44.431 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 3758096384, 2024-08-31T06:23:44.431 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-08-31T06:23:44.431 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-08-31T06:23:44.431 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-08-31T06:23:44.431 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-08-31T06:23:44.431 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-08-31T06:23:44.431 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-08-31T06:23:44.431 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-08-31T06:23:44.431 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 0, 2024-08-31T06:23:44.431 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 0, 2024-08-31T06:23:44.431 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-08-31T06:23:44.431 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:44.431 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:44.431 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:44.431 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-08-31T06:23:44.432 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:44.432 INFO:teuthology.orchestra.run.smithi184.stdout: "trackedop": { 2024-08-31T06:23:44.432 INFO:teuthology.orchestra.run.smithi184.stdout: "slow_ops_count": 0 2024-08-31T06:23:44.432 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-08-31T06:23:44.432 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-08-31T06:23:45.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:44 smithi155 ceph-mon[21742]: pgmap v881: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.0 KiB/s wr, 6 op/s 2024-08-31T06:23:45.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:44 smithi016 ceph-mon[17194]: pgmap v881: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.0 KiB/s wr, 6 op/s 2024-08-31T06:23:45.214 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:44 smithi184 ceph-mon[22331]: pgmap v881: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.0 KiB/s wr, 6 op/s 2024-08-31T06:23:45.241 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-08-31T06:23:45.241 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-08-31T06:23:45.241 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-08-31T06:23:45.241 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-08-31T06:23:45.241 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-08-31T06:23:45.241 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-08-31T06:23:45.241 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-08-31T06:23:45.241 DEBUG:tasks.check_counter:Getting stats from i 2024-08-31T06:23:45.242 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph daemon mds.i perf dump 2024-08-31T06:23:45.488 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.c/config 2024-08-31T06:23:46.005 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-08-31T06:23:46.005 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-0": { 2024-08-31T06:23:46.005 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 9985, 2024-08-31T06:23:46.005 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 7701, 2024-08-31T06:23:46.005 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 4415007, 2024-08-31T06:23:46.005 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 7190976, 2024-08-31T06:23:46.005 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 12, 2024-08-31T06:23:46.005 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 4, 2024-08-31T06:23:46.006 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.649527206, 2024-08-31T06:23:46.006 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.295735283, 2024-08-31T06:23:46.006 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.297759582, 2024-08-31T06:23:46.006 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.089165859, 2024-08-31T06:23:46.006 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T06:23:46.006 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 7701, 2024-08-31T06:23:46.006 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.374663754, 2024-08-31T06:23:46.006 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000048651 2024-08-31T06:23:46.006 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.006 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-08-31T06:23:46.006 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 9024, 2024-08-31T06:23:46.006 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.001694257, 2024-08-31T06:23:46.006 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000187 2024-08-31T06:23:46.006 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.007 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-31T06:23:46.007 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 0 2024-08-31T06:23:46.007 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.007 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-1": { 2024-08-31T06:23:46.007 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 4990, 2024-08-31T06:23:46.007 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 5344, 2024-08-31T06:23:46.007 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 2661136, 2024-08-31T06:23:46.007 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 3202301, 2024-08-31T06:23:46.007 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 47, 2024-08-31T06:23:46.007 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 5, 2024-08-31T06:23:46.007 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.360428607, 2024-08-31T06:23:46.007 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.170154190, 2024-08-31T06:23:46.007 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.138893168, 2024-08-31T06:23:46.007 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.049477794, 2024-08-31T06:23:46.008 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T06:23:46.008 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 5345, 2024-08-31T06:23:46.008 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.193090663, 2024-08-31T06:23:46.008 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000036125 2024-08-31T06:23:46.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.008 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-08-31T06:23:46.008 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 3901, 2024-08-31T06:23:46.008 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000677596, 2024-08-31T06:23:46.008 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000173 2024-08-31T06:23:46.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.008 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-08-31T06:23:46.008 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 601568 2024-08-31T06:23:46.008 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.008 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-2": { 2024-08-31T06:23:46.008 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 9461, 2024-08-31T06:23:46.009 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 11579, 2024-08-31T06:23:46.009 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 4462382, 2024-08-31T06:23:46.009 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 5396639, 2024-08-31T06:23:46.009 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 14, 2024-08-31T06:23:46.009 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 8, 2024-08-31T06:23:46.009 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.610682903, 2024-08-31T06:23:46.009 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.337071355, 2024-08-31T06:23:46.009 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.301048378, 2024-08-31T06:23:46.009 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.042010165, 2024-08-31T06:23:46.009 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T06:23:46.009 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 11574, 2024-08-31T06:23:46.009 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.704803693, 2024-08-31T06:23:46.009 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000060895 2024-08-31T06:23:46.009 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.009 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-08-31T06:23:46.010 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 5697, 2024-08-31T06:23:46.010 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.001880168, 2024-08-31T06:23:46.010 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000330 2024-08-31T06:23:46.010 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.010 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 1045488, 2024-08-31T06:23:46.010 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 233888 2024-08-31T06:23:46.010 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.010 INFO:teuthology.orchestra.run.smithi184.stdout: "cct": { 2024-08-31T06:23:46.010 INFO:teuthology.orchestra.run.smithi184.stdout: "total_workers": 1, 2024-08-31T06:23:46.010 INFO:teuthology.orchestra.run.smithi184.stdout: "unhealthy_workers": 0 2024-08-31T06:23:46.010 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.010 INFO:teuthology.orchestra.run.smithi184.stdout: "finisher-MDSRank": { 2024-08-31T06:23:46.010 INFO:teuthology.orchestra.run.smithi184.stdout: "queue_len": 0, 2024-08-31T06:23:46.010 INFO:teuthology.orchestra.run.smithi184.stdout: "complete_latency": { 2024-08-31T06:23:46.010 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 4910, 2024-08-31T06:23:46.011 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 5.268413762, 2024-08-31T06:23:46.011 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.001072996 2024-08-31T06:23:46.011 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-08-31T06:23:46.011 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.011 INFO:teuthology.orchestra.run.smithi184.stdout: "finisher-PurgeQueue": { 2024-08-31T06:23:46.011 INFO:teuthology.orchestra.run.smithi184.stdout: "queue_len": 0, 2024-08-31T06:23:46.011 INFO:teuthology.orchestra.run.smithi184.stdout: "complete_latency": { 2024-08-31T06:23:46.011 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 17, 2024-08-31T06:23:46.011 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000464310, 2024-08-31T06:23:46.011 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000027312 2024-08-31T06:23:46.011 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-08-31T06:23:46.011 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.011 INFO:teuthology.orchestra.run.smithi184.stdout: "mds": { 2024-08-31T06:23:46.012 INFO:teuthology.orchestra.run.smithi184.stdout: "request": 3330, 2024-08-31T06:23:46.012 INFO:teuthology.orchestra.run.smithi184.stdout: "reply": 3187, 2024-08-31T06:23:46.012 INFO:teuthology.orchestra.run.smithi184.stdout: "reply_latency": { 2024-08-31T06:23:46.012 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 3187, 2024-08-31T06:23:46.012 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 117.915763933, 2024-08-31T06:23:46.012 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.036998984 2024-08-31T06:23:46.012 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.012 INFO:teuthology.orchestra.run.smithi184.stdout: "slow_reply": 0, 2024-08-31T06:23:46.012 INFO:teuthology.orchestra.run.smithi184.stdout: "forward": 143, 2024-08-31T06:23:46.012 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_fetch_complete": 2, 2024-08-31T06:23:46.012 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_fetch_keys": 0, 2024-08-31T06:23:46.012 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_commit": 51, 2024-08-31T06:23:46.012 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_split": 5, 2024-08-31T06:23:46.012 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_merge": 0, 2024-08-31T06:23:46.013 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes": 788, 2024-08-31T06:23:46.013 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_top": 109, 2024-08-31T06:23:46.013 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_bottom": 679, 2024-08-31T06:23:46.013 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_pin_tail": 0, 2024-08-31T06:23:46.013 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_pinned": 458, 2024-08-31T06:23:46.013 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_expired": 318, 2024-08-31T06:23:46.013 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_with_caps": 163, 2024-08-31T06:23:46.013 INFO:teuthology.orchestra.run.smithi184.stdout: "caps": 163, 2024-08-31T06:23:46.013 INFO:teuthology.orchestra.run.smithi184.stdout: "subtrees": 63, 2024-08-31T06:23:46.013 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse": 16839, 2024-08-31T06:23:46.013 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_hit": 16520, 2024-08-31T06:23:46.014 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_forward": 141, 2024-08-31T06:23:46.014 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_discover": 52, 2024-08-31T06:23:46.014 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_dir_fetch": 1, 2024-08-31T06:23:46.014 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_remote_ino": 0, 2024-08-31T06:23:46.014 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_lock": 31, 2024-08-31T06:23:46.014 INFO:teuthology.orchestra.run.smithi184.stdout: "load_cent": 426, 2024-08-31T06:23:46.014 INFO:teuthology.orchestra.run.smithi184.stdout: "q": 0, 2024-08-31T06:23:46.014 INFO:teuthology.orchestra.run.smithi184.stdout: "exported": 11, 2024-08-31T06:23:46.014 INFO:teuthology.orchestra.run.smithi184.stdout: "exported_inodes": 292, 2024-08-31T06:23:46.014 INFO:teuthology.orchestra.run.smithi184.stdout: "imported": 15, 2024-08-31T06:23:46.014 INFO:teuthology.orchestra.run.smithi184.stdout: "imported_inodes": 562, 2024-08-31T06:23:46.014 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_dir_fetch": 0, 2024-08-31T06:23:46.014 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_backtrace_fetch": 8096, 2024-08-31T06:23:46.015 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_peer_discover": 0, 2024-08-31T06:23:46.015 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rfiles": 1003, 2024-08-31T06:23:46.015 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rbytes": 1271739561, 2024-08-31T06:23:46.015 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rsnaps": 6, 2024-08-31T06:23:46.015 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_backtrace_fetch": 8096, 2024-08-31T06:23:46.015 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_set_tag": 0, 2024-08-31T06:23:46.015 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_backtrace_repaired": 0, 2024-08-31T06:23:46.015 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_inotable_repaired": 0, 2024-08-31T06:23:46.015 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dir_inodes": 0, 2024-08-31T06:23:46.015 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dir_base_inodes": 0, 2024-08-31T06:23:46.015 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dirfrag_rstats": 0, 2024-08-31T06:23:46.015 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_file_inodes": 8096, 2024-08-31T06:23:46.015 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_inode_file_caps": 8, 2024-08-31T06:23:46.015 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_revoke": 2228, 2024-08-31T06:23:46.016 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_grant": 1850, 2024-08-31T06:23:46.016 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_trunc": 0, 2024-08-31T06:23:46.016 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-31T06:23:46.016 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-31T06:23:46.016 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_caps": 2871, 2024-08-31T06:23:46.016 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_caps_dirty": 389, 2024-08-31T06:23:46.016 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_cap_release": 40, 2024-08-31T06:23:46.016 INFO:teuthology.orchestra.run.smithi184.stdout: "process_request_cap_release": 135 2024-08-31T06:23:46.016 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.016 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_cache": { 2024-08-31T06:23:46.016 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_update": 72, 2024-08-31T06:23:46.016 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_update_receipt": 201, 2024-08-31T06:23:46.016 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_try_discover": 111, 2024-08-31T06:23:46.017 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_send_discover": 200, 2024-08-31T06:23:46.017 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_handle_discover": 108, 2024-08-31T06:23:46.017 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays": 14, 2024-08-31T06:23:46.017 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays_delayed": 0, 2024-08-31T06:23:46.017 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays_enqueuing": 0, 2024-08-31T06:23:46.017 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_created": 36, 2024-08-31T06:23:46.017 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_enqueued": 2, 2024-08-31T06:23:46.017 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_reintegrated": 19, 2024-08-31T06:23:46.017 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_migrated": 7, 2024-08-31T06:23:46.017 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_processing": 0, 2024-08-31T06:23:46.017 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_enqueued": 0, 2024-08-31T06:23:46.017 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_prioritized": 0, 2024-08-31T06:23:46.017 INFO:teuthology.orchestra.run.smithi184.stdout: "recovery_started": 0, 2024-08-31T06:23:46.017 INFO:teuthology.orchestra.run.smithi184.stdout: "recovery_completed": 0, 2024-08-31T06:23:46.018 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_quiesce_path": 12, 2024-08-31T06:23:46.018 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_quiesce_inode": 5641, 2024-08-31T06:23:46.018 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_enqueue_scrub": 0, 2024-08-31T06:23:46.018 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_exportdir": 11, 2024-08-31T06:23:46.018 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_flush": 0, 2024-08-31T06:23:46.018 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_fragmentdir": 5, 2024-08-31T06:23:46.018 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_fragstats": 0, 2024-08-31T06:23:46.018 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_inodestats": 0, 2024-08-31T06:23:46.018 INFO:teuthology.orchestra.run.smithi184.stdout: "uninline_started": 0, 2024-08-31T06:23:46.018 INFO:teuthology.orchestra.run.smithi184.stdout: "uninline_succeeded": 0, 2024-08-31T06:23:46.018 INFO:teuthology.orchestra.run.smithi184.stdout: "uninline_write_failed": 0 2024-08-31T06:23:46.018 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.018 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_log": { 2024-08-31T06:23:46.018 INFO:teuthology.orchestra.run.smithi184.stdout: "evlrg": 0, 2024-08-31T06:23:46.019 INFO:teuthology.orchestra.run.smithi184.stdout: "evadd": 2959, 2024-08-31T06:23:46.019 INFO:teuthology.orchestra.run.smithi184.stdout: "evex": 0, 2024-08-31T06:23:46.019 INFO:teuthology.orchestra.run.smithi184.stdout: "evtrm": 0, 2024-08-31T06:23:46.019 INFO:teuthology.orchestra.run.smithi184.stdout: "ev": 2959, 2024-08-31T06:23:46.019 INFO:teuthology.orchestra.run.smithi184.stdout: "evexg": 0, 2024-08-31T06:23:46.019 INFO:teuthology.orchestra.run.smithi184.stdout: "evexd": 0, 2024-08-31T06:23:46.019 INFO:teuthology.orchestra.run.smithi184.stdout: "segadd": 5, 2024-08-31T06:23:46.019 INFO:teuthology.orchestra.run.smithi184.stdout: "segex": 0, 2024-08-31T06:23:46.019 INFO:teuthology.orchestra.run.smithi184.stdout: "segtrm": 0, 2024-08-31T06:23:46.019 INFO:teuthology.orchestra.run.smithi184.stdout: "seg": 5, 2024-08-31T06:23:46.019 INFO:teuthology.orchestra.run.smithi184.stdout: "segmjr": 2, 2024-08-31T06:23:46.019 INFO:teuthology.orchestra.run.smithi184.stdout: "segexg": 0, 2024-08-31T06:23:46.019 INFO:teuthology.orchestra.run.smithi184.stdout: "segexd": 0, 2024-08-31T06:23:46.020 INFO:teuthology.orchestra.run.smithi184.stdout: "expos": 4194304, 2024-08-31T06:23:46.020 INFO:teuthology.orchestra.run.smithi184.stdout: "wrpos": 8405221, 2024-08-31T06:23:46.020 INFO:teuthology.orchestra.run.smithi184.stdout: "rdpos": 0, 2024-08-31T06:23:46.020 INFO:teuthology.orchestra.run.smithi184.stdout: "jlat": { 2024-08-31T06:23:46.020 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 1320, 2024-08-31T06:23:46.020 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 7.979121728, 2024-08-31T06:23:46.020 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.006044789 2024-08-31T06:23:46.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.020 INFO:teuthology.orchestra.run.smithi184.stdout: "replayed": 0 2024-08-31T06:23:46.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.020 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_mem": { 2024-08-31T06:23:46.020 INFO:teuthology.orchestra.run.smithi184.stdout: "ino": 698, 2024-08-31T06:23:46.020 INFO:teuthology.orchestra.run.smithi184.stdout: "ino+": 994, 2024-08-31T06:23:46.020 INFO:teuthology.orchestra.run.smithi184.stdout: "ino-": 296, 2024-08-31T06:23:46.021 INFO:teuthology.orchestra.run.smithi184.stdout: "dir": 104, 2024-08-31T06:23:46.021 INFO:teuthology.orchestra.run.smithi184.stdout: "dir+": 179, 2024-08-31T06:23:46.021 INFO:teuthology.orchestra.run.smithi184.stdout: "dir-": 75, 2024-08-31T06:23:46.021 INFO:teuthology.orchestra.run.smithi184.stdout: "dn": 796, 2024-08-31T06:23:46.021 INFO:teuthology.orchestra.run.smithi184.stdout: "dn+": 1128, 2024-08-31T06:23:46.021 INFO:teuthology.orchestra.run.smithi184.stdout: "dn-": 332, 2024-08-31T06:23:46.021 INFO:teuthology.orchestra.run.smithi184.stdout: "cap": 163, 2024-08-31T06:23:46.021 INFO:teuthology.orchestra.run.smithi184.stdout: "cap+": 1055, 2024-08-31T06:23:46.021 INFO:teuthology.orchestra.run.smithi184.stdout: "cap-": 892, 2024-08-31T06:23:46.021 INFO:teuthology.orchestra.run.smithi184.stdout: "rss": 133624, 2024-08-31T06:23:46.021 INFO:teuthology.orchestra.run.smithi184.stdout: "heap": 339623936 2024-08-31T06:23:46.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.021 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_server": { 2024-08-31T06:23:46.021 INFO:teuthology.orchestra.run.smithi184.stdout: "dispatch_client_request": 3854, 2024-08-31T06:23:46.021 INFO:teuthology.orchestra.run.smithi184.stdout: "dispatch_server_request": 247, 2024-08-31T06:23:46.022 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_request": 3330, 2024-08-31T06:23:46.022 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_session": 87, 2024-08-31T06:23:46.022 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_peer_request": 386, 2024-08-31T06:23:46.022 INFO:teuthology.orchestra.run.smithi184.stdout: "req_create_latency": { 2024-08-31T06:23:46.022 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:46.022 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:46.022 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:46.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.022 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getattr_latency": { 2024-08-31T06:23:46.022 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 2444, 2024-08-31T06:23:46.022 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 106.028693645, 2024-08-31T06:23:46.022 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.043383262 2024-08-31T06:23:46.022 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.023 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getfilelock_latency": { 2024-08-31T06:23:46.023 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:46.023 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:46.023 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:46.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.023 INFO:teuthology.orchestra.run.smithi184.stdout: "req_link_latency": { 2024-08-31T06:23:46.023 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 27, 2024-08-31T06:23:46.023 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.233483986, 2024-08-31T06:23:46.023 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.008647555 2024-08-31T06:23:46.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.023 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookup_latency": { 2024-08-31T06:23:46.023 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 401, 2024-08-31T06:23:46.023 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 4.330086716, 2024-08-31T06:23:46.023 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.010798221 2024-08-31T06:23:46.023 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.024 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookuphash_latency": { 2024-08-31T06:23:46.024 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:46.024 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:46.024 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:46.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.024 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupino_latency": { 2024-08-31T06:23:46.024 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:46.024 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:46.024 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:46.024 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.024 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupname_latency": { 2024-08-31T06:23:46.024 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:46.024 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:46.025 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:46.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.025 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupparent_latency": { 2024-08-31T06:23:46.025 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:46.025 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:46.025 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:46.025 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.025 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupsnap_latency": { 2024-08-31T06:23:46.025 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:46.026 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:46.026 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:46.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.026 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lssnap_latency": { 2024-08-31T06:23:46.026 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:46.026 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:46.026 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:46.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.026 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mkdir_latency": { 2024-08-31T06:23:46.026 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:46.026 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:46.026 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:46.026 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.027 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mknod_latency": { 2024-08-31T06:23:46.027 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:46.027 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:46.027 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:46.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.027 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mksnap_latency": { 2024-08-31T06:23:46.027 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:46.027 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:46.027 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:46.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.027 INFO:teuthology.orchestra.run.smithi184.stdout: "req_open_latency": { 2024-08-31T06:23:46.027 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 201, 2024-08-31T06:23:46.027 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.164946861, 2024-08-31T06:23:46.027 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000820631 2024-08-31T06:23:46.027 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.028 INFO:teuthology.orchestra.run.smithi184.stdout: "req_readdir_latency": { 2024-08-31T06:23:46.028 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 53, 2024-08-31T06:23:46.028 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.027830810, 2024-08-31T06:23:46.028 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000525109 2024-08-31T06:23:46.028 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.028 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rename_latency": { 2024-08-31T06:23:46.028 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 26, 2024-08-31T06:23:46.028 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 7.006036660, 2024-08-31T06:23:46.028 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.269462948 2024-08-31T06:23:46.028 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.028 INFO:teuthology.orchestra.run.smithi184.stdout: "req_renamesnap_latency": { 2024-08-31T06:23:46.028 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:46.028 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:46.028 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:46.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.029 INFO:teuthology.orchestra.run.smithi184.stdout: "req_snapdiff_latency": { 2024-08-31T06:23:46.029 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:46.029 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:46.029 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:46.029 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.029 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmdir_latency": { 2024-08-31T06:23:46.029 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:46.029 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:46.029 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:46.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.030 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmsnap_latency": { 2024-08-31T06:23:46.030 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:46.030 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:46.030 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:46.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.030 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmxattr_latency": { 2024-08-31T06:23:46.030 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:46.030 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:46.030 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:46.030 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.030 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setattr_latency": { 2024-08-31T06:23:46.030 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:46.030 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:46.031 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:46.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.031 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setdirlayout_latency": { 2024-08-31T06:23:46.031 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:46.031 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:46.031 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:46.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.031 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setfilelock_latency": { 2024-08-31T06:23:46.031 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:46.031 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:46.031 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:46.031 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.031 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setlayout_latency": { 2024-08-31T06:23:46.031 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:46.031 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:46.032 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:46.032 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.032 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setxattr_latency": { 2024-08-31T06:23:46.032 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:46.032 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:46.032 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:46.032 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.032 INFO:teuthology.orchestra.run.smithi184.stdout: "req_symlink_latency": { 2024-08-31T06:23:46.032 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:46.032 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:46.032 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:46.032 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.032 INFO:teuthology.orchestra.run.smithi184.stdout: "req_unlink_latency": { 2024-08-31T06:23:46.033 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 35, 2024-08-31T06:23:46.033 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.124685255, 2024-08-31T06:23:46.033 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.003562435 2024-08-31T06:23:46.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.033 INFO:teuthology.orchestra.run.smithi184.stdout: "cap_revoke_eviction": 0, 2024-08-31T06:23:46.033 INFO:teuthology.orchestra.run.smithi184.stdout: "cap_acquisition_throttle": 0, 2024-08-31T06:23:46.033 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getvxattr_latency": { 2024-08-31T06:23:46.033 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:46.033 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:46.033 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:46.033 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-08-31T06:23:46.033 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.033 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_sessions": { 2024-08-31T06:23:46.033 INFO:teuthology.orchestra.run.smithi184.stdout: "session_count": 2, 2024-08-31T06:23:46.033 INFO:teuthology.orchestra.run.smithi184.stdout: "session_add": 3, 2024-08-31T06:23:46.034 INFO:teuthology.orchestra.run.smithi184.stdout: "session_remove": 1, 2024-08-31T06:23:46.034 INFO:teuthology.orchestra.run.smithi184.stdout: "sessions_open": 2, 2024-08-31T06:23:46.034 INFO:teuthology.orchestra.run.smithi184.stdout: "sessions_stale": 0, 2024-08-31T06:23:46.034 INFO:teuthology.orchestra.run.smithi184.stdout: "total_load": 343, 2024-08-31T06:23:46.034 INFO:teuthology.orchestra.run.smithi184.stdout: "average_load": 171, 2024-08-31T06:23:46.034 INFO:teuthology.orchestra.run.smithi184.stdout: "avg_session_uptime": 994, 2024-08-31T06:23:46.034 INFO:teuthology.orchestra.run.smithi184.stdout: "mdthresh_evicted": 0 2024-08-31T06:23:46.034 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.034 INFO:teuthology.orchestra.run.smithi184.stdout: "mempool": { 2024-08-31T06:23:46.034 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_bytes": 0, 2024-08-31T06:23:46.034 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_items": 0, 2024-08-31T06:23:46.034 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_bytes": 0, 2024-08-31T06:23:46.034 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_items": 0, 2024-08-31T06:23:46.034 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_bytes": 0, 2024-08-31T06:23:46.034 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_items": 0, 2024-08-31T06:23:46.035 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-31T06:23:46.035 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_items": 0, 2024-08-31T06:23:46.035 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-31T06:23:46.035 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_items": 0, 2024-08-31T06:23:46.035 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_bytes": 0, 2024-08-31T06:23:46.035 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_items": 0, 2024-08-31T06:23:46.035 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-31T06:23:46.035 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_items": 0, 2024-08-31T06:23:46.035 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_bytes": 0, 2024-08-31T06:23:46.035 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_items": 0, 2024-08-31T06:23:46.035 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_bytes": 0, 2024-08-31T06:23:46.035 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_items": 0, 2024-08-31T06:23:46.035 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-31T06:23:46.035 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_items": 0, 2024-08-31T06:23:46.036 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-31T06:23:46.036 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_items": 0, 2024-08-31T06:23:46.036 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_bytes": 0, 2024-08-31T06:23:46.036 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_items": 0, 2024-08-31T06:23:46.036 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_bytes": 0, 2024-08-31T06:23:46.036 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_items": 0, 2024-08-31T06:23:46.036 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-31T06:23:46.036 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_items": 0, 2024-08-31T06:23:46.036 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_bytes": 0, 2024-08-31T06:23:46.036 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_items": 0, 2024-08-31T06:23:46.036 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_bytes": 0, 2024-08-31T06:23:46.036 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_items": 0, 2024-08-31T06:23:46.036 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_bytes": 0, 2024-08-31T06:23:46.036 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_items": 0, 2024-08-31T06:23:46.036 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_bytes": 0, 2024-08-31T06:23:46.036 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_items": 0, 2024-08-31T06:23:46.037 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_bytes": 751304, 2024-08-31T06:23:46.037 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_items": 703, 2024-08-31T06:23:46.037 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_bytes": 0, 2024-08-31T06:23:46.037 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_items": 0, 2024-08-31T06:23:46.037 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_bytes": 0, 2024-08-31T06:23:46.037 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_items": 0, 2024-08-31T06:23:46.037 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_bytes": 0, 2024-08-31T06:23:46.037 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_items": 0, 2024-08-31T06:23:46.037 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_bytes": 0, 2024-08-31T06:23:46.037 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_items": 0, 2024-08-31T06:23:46.037 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_bytes": 8064, 2024-08-31T06:23:46.037 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_items": 178, 2024-08-31T06:23:46.037 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_bytes": 0, 2024-08-31T06:23:46.037 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_items": 0, 2024-08-31T06:23:46.037 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_bytes": 0, 2024-08-31T06:23:46.038 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_items": 0, 2024-08-31T06:23:46.038 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_bytes": 2477825, 2024-08-31T06:23:46.038 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_items": 43074, 2024-08-31T06:23:46.038 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_bytes": 0, 2024-08-31T06:23:46.038 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_items": 0, 2024-08-31T06:23:46.038 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_bytes": 0, 2024-08-31T06:23:46.038 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_items": 0 2024-08-31T06:23:46.038 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.038 INFO:teuthology.orchestra.run.smithi184.stdout: "objecter": { 2024-08-31T06:23:46.038 INFO:teuthology.orchestra.run.smithi184.stdout: "op_active": 0, 2024-08-31T06:23:46.038 INFO:teuthology.orchestra.run.smithi184.stdout: "op_laggy": 0, 2024-08-31T06:23:46.038 INFO:teuthology.orchestra.run.smithi184.stdout: "op_send": 9688, 2024-08-31T06:23:46.038 INFO:teuthology.orchestra.run.smithi184.stdout: "op_send_bytes": 4650884, 2024-08-31T06:23:46.038 INFO:teuthology.orchestra.run.smithi184.stdout: "op_resend": 32, 2024-08-31T06:23:46.038 INFO:teuthology.orchestra.run.smithi184.stdout: "op_reply": 9688, 2024-08-31T06:23:46.039 INFO:teuthology.orchestra.run.smithi184.stdout: "op_latency": { 2024-08-31T06:23:46.039 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 9688, 2024-08-31T06:23:46.039 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 18.863602250, 2024-08-31T06:23:46.039 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.001947110 2024-08-31T06:23:46.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.039 INFO:teuthology.orchestra.run.smithi184.stdout: "op_inflight": 0, 2024-08-31T06:23:46.039 INFO:teuthology.orchestra.run.smithi184.stdout: "oplen_avg": { 2024-08-31T06:23:46.039 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 9688, 2024-08-31T06:23:46.039 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 9885 2024-08-31T06:23:46.039 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.039 INFO:teuthology.orchestra.run.smithi184.stdout: "op": 9688, 2024-08-31T06:23:46.039 INFO:teuthology.orchestra.run.smithi184.stdout: "op_r": 8101, 2024-08-31T06:23:46.039 INFO:teuthology.orchestra.run.smithi184.stdout: "op_w": 1587, 2024-08-31T06:23:46.039 INFO:teuthology.orchestra.run.smithi184.stdout: "op_rmw": 0, 2024-08-31T06:23:46.039 INFO:teuthology.orchestra.run.smithi184.stdout: "op_pg": 0, 2024-08-31T06:23:46.039 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_stat": 0, 2024-08-31T06:23:46.040 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_create": 10, 2024-08-31T06:23:46.040 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_read": 1, 2024-08-31T06:23:46.040 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_write": 1321, 2024-08-31T06:23:46.040 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_writefull": 68, 2024-08-31T06:23:46.040 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_writesame": 0, 2024-08-31T06:23:46.040 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_append": 0, 2024-08-31T06:23:46.040 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_zero": 0, 2024-08-31T06:23:46.040 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_truncate": 1, 2024-08-31T06:23:46.040 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_delete": 19, 2024-08-31T06:23:46.040 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_mapext": 0, 2024-08-31T06:23:46.040 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_sparse_read": 0, 2024-08-31T06:23:46.040 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_clonerange": 0, 2024-08-31T06:23:46.040 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_getxattr": 8096, 2024-08-31T06:23:46.040 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_setxattr": 20, 2024-08-31T06:23:46.040 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_cmpxattr": 0, 2024-08-31T06:23:46.041 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_rmxattr": 0, 2024-08-31T06:23:46.041 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_resetxattrs": 0, 2024-08-31T06:23:46.041 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_call": 0, 2024-08-31T06:23:46.041 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_watch": 0, 2024-08-31T06:23:46.041 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_notify": 0, 2024-08-31T06:23:46.041 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_src_cmpxattr": 0, 2024-08-31T06:23:46.041 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_pgls": 0, 2024-08-31T06:23:46.041 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_pgls_filter": 0, 2024-08-31T06:23:46.041 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_other": 0, 2024-08-31T06:23:46.041 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_active": 0, 2024-08-31T06:23:46.041 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_send": 0, 2024-08-31T06:23:46.041 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_resend": 0, 2024-08-31T06:23:46.041 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_ping": 0, 2024-08-31T06:23:46.041 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_active": 0, 2024-08-31T06:23:46.042 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_send": 0, 2024-08-31T06:23:46.042 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_resend": 0, 2024-08-31T06:23:46.042 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_active": 0, 2024-08-31T06:23:46.042 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_send": 0, 2024-08-31T06:23:46.042 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_resend": 0, 2024-08-31T06:23:46.042 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_active": 0, 2024-08-31T06:23:46.042 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_send": 0, 2024-08-31T06:23:46.042 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_resend": 0, 2024-08-31T06:23:46.042 INFO:teuthology.orchestra.run.smithi184.stdout: "command_active": 0, 2024-08-31T06:23:46.042 INFO:teuthology.orchestra.run.smithi184.stdout: "command_send": 0, 2024-08-31T06:23:46.042 INFO:teuthology.orchestra.run.smithi184.stdout: "command_resend": 0, 2024-08-31T06:23:46.042 INFO:teuthology.orchestra.run.smithi184.stdout: "map_epoch": 103, 2024-08-31T06:23:46.042 INFO:teuthology.orchestra.run.smithi184.stdout: "map_full": 0, 2024-08-31T06:23:46.042 INFO:teuthology.orchestra.run.smithi184.stdout: "map_inc": 25, 2024-08-31T06:23:46.042 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_sessions": 12, 2024-08-31T06:23:46.043 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_session_open": 12, 2024-08-31T06:23:46.043 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_session_close": 0, 2024-08-31T06:23:46.043 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_laggy": 0, 2024-08-31T06:23:46.043 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_wr": 282, 2024-08-31T06:23:46.043 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_rd": 6, 2024-08-31T06:23:46.043 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_del": 61 2024-08-31T06:23:46.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.043 INFO:teuthology.orchestra.run.smithi184.stdout: "oft": { 2024-08-31T06:23:46.043 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_objs": 1, 2024-08-31T06:23:46.043 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_kv_pairs": 0, 2024-08-31T06:23:46.043 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_updates": 0, 2024-08-31T06:23:46.043 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_removes": 0 2024-08-31T06:23:46.043 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.043 INFO:teuthology.orchestra.run.smithi184.stdout: "purge_queue": { 2024-08-31T06:23:46.043 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_ops": 0, 2024-08-31T06:23:46.044 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_ops_high_water": 2, 2024-08-31T06:23:46.044 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing": 0, 2024-08-31T06:23:46.044 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_high_water": 2, 2024-08-31T06:23:46.044 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executed_ops": 2, 2024-08-31T06:23:46.044 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executed": 2, 2024-08-31T06:23:46.044 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_item_in_journal": 0 2024-08-31T06:23:46.044 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.044 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-31T06:23:46.044 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-08-31T06:23:46.044 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2024-08-31T06:23:46.044 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-08-31T06:23:46.044 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 24432, 2024-08-31T06:23:46.044 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 9415017, 2024-08-31T06:23:46.044 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-08-31T06:23:46.044 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 24432, 2024-08-31T06:23:46.045 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-08-31T06:23:46.045 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-08-31T06:23:46.045 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 24432, 2024-08-31T06:23:46.045 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 9415017, 2024-08-31T06:23:46.045 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-08-31T06:23:46.045 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:46.045 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:46.045 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:46.045 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-08-31T06:23:46.045 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.045 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-objecter_bytes": { 2024-08-31T06:23:46.045 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-08-31T06:23:46.045 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2024-08-31T06:23:46.045 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-08-31T06:23:46.046 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-08-31T06:23:46.046 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-08-31T06:23:46.046 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-08-31T06:23:46.046 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-08-31T06:23:46.046 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 9688, 2024-08-31T06:23:46.046 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 4651060, 2024-08-31T06:23:46.046 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 9688, 2024-08-31T06:23:46.046 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 4651060, 2024-08-31T06:23:46.046 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-08-31T06:23:46.046 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:46.046 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:46.046 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:46.046 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-08-31T06:23:46.046 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.046 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-objecter_ops": { 2024-08-31T06:23:46.046 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-08-31T06:23:46.047 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 1024, 2024-08-31T06:23:46.047 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-08-31T06:23:46.047 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-08-31T06:23:46.047 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-08-31T06:23:46.047 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-08-31T06:23:46.047 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-08-31T06:23:46.047 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 9688, 2024-08-31T06:23:46.047 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 9688, 2024-08-31T06:23:46.047 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 9688, 2024-08-31T06:23:46.047 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 9688, 2024-08-31T06:23:46.047 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-08-31T06:23:46.047 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:46.047 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:46.047 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:46.047 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-08-31T06:23:46.048 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.048 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-write_buf_throttle": { 2024-08-31T06:23:46.048 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-08-31T06:23:46.048 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 3758096384, 2024-08-31T06:23:46.048 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-08-31T06:23:46.048 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 2, 2024-08-31T06:23:46.048 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 298, 2024-08-31T06:23:46.048 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-08-31T06:23:46.048 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 2, 2024-08-31T06:23:46.048 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-08-31T06:23:46.048 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-08-31T06:23:46.048 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 1, 2024-08-31T06:23:46.048 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 298, 2024-08-31T06:23:46.048 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-08-31T06:23:46.049 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:46.049 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:46.049 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:46.049 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-08-31T06:23:46.049 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.049 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-write_buf_throttle-0x55ecb95ba5b0": { 2024-08-31T06:23:46.049 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-08-31T06:23:46.049 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 3758096384, 2024-08-31T06:23:46.049 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-08-31T06:23:46.049 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 2959, 2024-08-31T06:23:46.049 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 4210917, 2024-08-31T06:23:46.049 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-08-31T06:23:46.049 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 2959, 2024-08-31T06:23:46.049 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-08-31T06:23:46.050 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-08-31T06:23:46.050 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 1320, 2024-08-31T06:23:46.050 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 4210917, 2024-08-31T06:23:46.050 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-08-31T06:23:46.050 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:46.050 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:46.050 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:46.050 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-08-31T06:23:46.050 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:46.050 INFO:teuthology.orchestra.run.smithi184.stdout: "trackedop": { 2024-08-31T06:23:46.050 INFO:teuthology.orchestra.run.smithi184.stdout: "slow_ops_count": 7394 2024-08-31T06:23:46.050 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-08-31T06:23:46.050 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-08-31T06:23:46.844 INFO:tasks.check_counter:Daemon mds.i mds.exported=11 2024-08-31T06:23:46.844 INFO:tasks.check_counter:Daemon mds.i mds.imported=15 2024-08-31T06:23:46.844 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=72 2024-08-31T06:23:46.844 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=201 2024-08-31T06:23:46.844 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-08-31T06:23:46.844 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-08-31T06:23:46.844 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-08-31T06:23:46.844 DEBUG:tasks.check_counter:Getting stats from l 2024-08-31T06:23:46.845 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph daemon mds.l perf dump 2024-08-31T06:23:47.093 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.c/config 2024-08-31T06:23:47.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:46 smithi155 ceph-mon[21742]: pgmap v882: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-08-31T06:23:47.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:46 smithi016 ceph-mon[17194]: pgmap v882: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-08-31T06:23:47.214 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:46 smithi184 ceph-mon[22331]: pgmap v882: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-08-31T06:23:47.675 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-08-31T06:23:47.675 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-0": { 2024-08-31T06:23:47.676 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 4, 2024-08-31T06:23:47.676 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 4, 2024-08-31T06:23:47.676 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 2992, 2024-08-31T06:23:47.676 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 448, 2024-08-31T06:23:47.676 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 4, 2024-08-31T06:23:47.676 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 0, 2024-08-31T06:23:47.676 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.002879410, 2024-08-31T06:23:47.676 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.000041410, 2024-08-31T06:23:47.676 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.002086173, 2024-08-31T06:23:47.676 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-31T06:23:47.676 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T06:23:47.676 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 3, 2024-08-31T06:23:47.677 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.003886871, 2024-08-31T06:23:47.677 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.001295623 2024-08-31T06:23:47.677 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:47.677 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-08-31T06:23:47.677 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:47.677 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:47.677 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:47.677 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:47.677 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-08-31T06:23:47.677 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 448 2024-08-31T06:23:47.677 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:47.677 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-1": { 2024-08-31T06:23:47.677 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 327, 2024-08-31T06:23:47.678 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 322, 2024-08-31T06:23:47.678 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 858416, 2024-08-31T06:23:47.678 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 198960, 2024-08-31T06:23:47.678 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 2, 2024-08-31T06:23:47.678 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 1, 2024-08-31T06:23:47.678 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.051673546, 2024-08-31T06:23:47.678 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.024543539, 2024-08-31T06:23:47.678 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.037316631, 2024-08-31T06:23:47.678 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-31T06:23:47.678 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T06:23:47.678 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 321, 2024-08-31T06:23:47.678 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.017236674, 2024-08-31T06:23:47.678 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000053696 2024-08-31T06:23:47.678 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:47.679 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-08-31T06:23:47.679 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:47.679 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:47.679 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:47.679 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:47.679 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 858416, 2024-08-31T06:23:47.679 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 198960 2024-08-31T06:23:47.679 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:47.679 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-2": { 2024-08-31T06:23:47.679 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 2, 2024-08-31T06:23:47.679 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 247, 2024-08-31T06:23:47.679 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 320, 2024-08-31T06:23:47.679 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 203696, 2024-08-31T06:23:47.679 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 4, 2024-08-31T06:23:47.679 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 2, 2024-08-31T06:23:47.680 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.019237215, 2024-08-31T06:23:47.680 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.015219493, 2024-08-31T06:23:47.680 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.001310648, 2024-08-31T06:23:47.680 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-31T06:23:47.680 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-08-31T06:23:47.680 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 245, 2024-08-31T06:23:47.680 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.016671501, 2024-08-31T06:23:47.680 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000068046 2024-08-31T06:23:47.680 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:47.680 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-08-31T06:23:47.680 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:47.680 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:47.680 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:47.681 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:47.681 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 320, 2024-08-31T06:23:47.681 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 203696 2024-08-31T06:23:47.681 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:47.681 INFO:teuthology.orchestra.run.smithi184.stdout: "cct": { 2024-08-31T06:23:47.681 INFO:teuthology.orchestra.run.smithi184.stdout: "total_workers": 0, 2024-08-31T06:23:47.681 INFO:teuthology.orchestra.run.smithi184.stdout: "unhealthy_workers": 0 2024-08-31T06:23:47.681 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:47.681 INFO:teuthology.orchestra.run.smithi184.stdout: "mempool": { 2024-08-31T06:23:47.681 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_bytes": 0, 2024-08-31T06:23:47.681 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_items": 0, 2024-08-31T06:23:47.681 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_bytes": 0, 2024-08-31T06:23:47.682 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_items": 0, 2024-08-31T06:23:47.682 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_bytes": 0, 2024-08-31T06:23:47.682 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_items": 0, 2024-08-31T06:23:47.682 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-31T06:23:47.682 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_items": 0, 2024-08-31T06:23:47.682 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-31T06:23:47.682 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_items": 0, 2024-08-31T06:23:47.682 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_bytes": 0, 2024-08-31T06:23:47.682 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_items": 0, 2024-08-31T06:23:47.682 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-31T06:23:47.682 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_items": 0, 2024-08-31T06:23:47.682 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_bytes": 0, 2024-08-31T06:23:47.682 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_items": 0, 2024-08-31T06:23:47.682 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_bytes": 0, 2024-08-31T06:23:47.682 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_items": 0, 2024-08-31T06:23:47.683 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-31T06:23:47.683 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_items": 0, 2024-08-31T06:23:47.683 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-31T06:23:47.683 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_items": 0, 2024-08-31T06:23:47.683 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_bytes": 0, 2024-08-31T06:23:47.683 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_items": 0, 2024-08-31T06:23:47.683 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_bytes": 0, 2024-08-31T06:23:47.683 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_items": 0, 2024-08-31T06:23:47.683 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-31T06:23:47.683 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_items": 0, 2024-08-31T06:23:47.683 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_bytes": 0, 2024-08-31T06:23:47.683 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_items": 0, 2024-08-31T06:23:47.683 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_bytes": 0, 2024-08-31T06:23:47.684 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_items": 0, 2024-08-31T06:23:47.684 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_bytes": 0, 2024-08-31T06:23:47.684 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_items": 0, 2024-08-31T06:23:47.684 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_bytes": 0, 2024-08-31T06:23:47.684 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_items": 0, 2024-08-31T06:23:47.684 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_bytes": 148722, 2024-08-31T06:23:47.684 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_items": 30, 2024-08-31T06:23:47.684 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_bytes": 0, 2024-08-31T06:23:47.684 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_items": 0, 2024-08-31T06:23:47.684 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_bytes": 0, 2024-08-31T06:23:47.684 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_items": 0, 2024-08-31T06:23:47.684 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_bytes": 0, 2024-08-31T06:23:47.684 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_items": 0, 2024-08-31T06:23:47.684 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_bytes": 0, 2024-08-31T06:23:47.685 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_items": 0, 2024-08-31T06:23:47.685 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_bytes": 0, 2024-08-31T06:23:47.685 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_items": 0, 2024-08-31T06:23:47.685 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_bytes": 0, 2024-08-31T06:23:47.685 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_items": 0, 2024-08-31T06:23:47.685 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_bytes": 0, 2024-08-31T06:23:47.685 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_items": 0, 2024-08-31T06:23:47.685 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_bytes": 928, 2024-08-31T06:23:47.685 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_items": 2, 2024-08-31T06:23:47.685 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_bytes": 0, 2024-08-31T06:23:47.685 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_items": 0, 2024-08-31T06:23:47.685 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_bytes": 0, 2024-08-31T06:23:47.685 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_items": 0 2024-08-31T06:23:47.685 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-08-31T06:23:47.686 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-31T06:23:47.686 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-08-31T06:23:47.686 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2024-08-31T06:23:47.686 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-08-31T06:23:47.686 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 329, 2024-08-31T06:23:47.686 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 812182, 2024-08-31T06:23:47.686 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-08-31T06:23:47.686 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 329, 2024-08-31T06:23:47.686 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-08-31T06:23:47.686 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-08-31T06:23:47.686 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 329, 2024-08-31T06:23:47.686 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 812182, 2024-08-31T06:23:47.686 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-08-31T06:23:47.687 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-08-31T06:23:47.687 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-08-31T06:23:47.687 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-08-31T06:23:47.687 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-08-31T06:23:47.687 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-08-31T06:23:47.687 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-08-31T06:23:48.379 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-08-31T06:23:48.379 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-08-31T06:23:48.379 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-08-31T06:23:48.380 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-08-31T06:23:48.380 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-08-31T06:23:48.380 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-08-31T06:23:48.380 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-08-31T06:23:48.380 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-08-31T06:23:48.391 INFO:tasks.kclient:Unmounting kernel clients... 2024-08-31T06:23:48.391 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-08-31T06:23:48.391 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-31T06:23:48.433 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-08-31T06:23:48.434 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-31T06:23:48.512 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph osd blocklist ls 2024-08-31T06:23:48.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:48 smithi016 ceph-mon[17194]: pgmap v883: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 7 op/s 2024-08-31T06:23:48.936 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:23:49.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:48 smithi155 ceph-mon[21742]: pgmap v883: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 7 op/s 2024-08-31T06:23:49.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:48 smithi184 ceph-mon[22331]: pgmap v883: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 7 op/s 2024-08-31T06:23:49.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.591+0000 7f11bfe00640 1 -- 172.21.15.16:0/3748939033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11b8105b20 msgr2=0x7f11b8105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:49.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.591+0000 7f11bfe00640 1 --2- 172.21.15.16:0/3748939033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11b8105b20 0x7f11b8105f30 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f11ac009900 tx=0x7f11ac02a410 comp rx=0 tx=0).stop 2024-08-31T06:23:49.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.591+0000 7f11bfe00640 1 -- 172.21.15.16:0/3748939033 shutdown_connections 2024-08-31T06:23:49.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.591+0000 7f11bfe00640 1 --2- 172.21.15.16:0/3748939033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11b810bdb0 0x7f11b81125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:49.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.591+0000 7f11bfe00640 1 --2- 172.21.15.16:0/3748939033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11b810b460 0x7f11b810b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:49.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.591+0000 7f11bfe00640 1 --2- 172.21.15.16:0/3748939033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11b8105b20 0x7f11b8105f30 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:49.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.591+0000 7f11bfe00640 1 -- 172.21.15.16:0/3748939033 >> 172.21.15.16:0/3748939033 conn(0x7f11b81015b0 msgr2=0x7f11b8103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:49.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.591+0000 7f11bfe00640 1 -- 172.21.15.16:0/3748939033 shutdown_connections 2024-08-31T06:23:49.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.591+0000 7f11bfe00640 1 -- 172.21.15.16:0/3748939033 wait complete. 2024-08-31T06:23:49.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.592+0000 7f11bfe00640 1 Processor -- start 2024-08-31T06:23:49.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.592+0000 7f11bfe00640 1 -- start start 2024-08-31T06:23:49.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.593+0000 7f11bfe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11b8105b20 0x7f11b819c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:49.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.593+0000 7f11bfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11b810b460 0x7f11b819ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:49.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.593+0000 7f11bfe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11b810bdb0 0x7f11b81a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:49.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.593+0000 7f11bfe00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f11b8115ac0 con 0x7f11b810b460 2024-08-31T06:23:49.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.593+0000 7f11bfe00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f11b8115940 con 0x7f11b8105b20 2024-08-31T06:23:49.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.593+0000 7f11bfe00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f11b8115c40 con 0x7f11b810bdb0 2024-08-31T06:23:49.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.593+0000 7f11bca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11b810b460 0x7f11b819ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:49.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.593+0000 7f11bca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11b810b460 0x7f11b819ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41834/0 (socket says 172.21.15.16:41834) 2024-08-31T06:23:49.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.593+0000 7f11bd400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11b8105b20 0x7f11b819c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:49.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.593+0000 7f11bca00640 1 -- 172.21.15.16:0/3024281172 learned_addr learned my addr 172.21.15.16:0/3024281172 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:23:49.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.593+0000 7f11bde00640 1 --2- 172.21.15.16:0/3024281172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11b810bdb0 0x7f11b81a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:49.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.594+0000 7f11bca00640 1 -- 172.21.15.16:0/3024281172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11b810bdb0 msgr2=0x7f11b81a1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:49.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.594+0000 7f11bca00640 1 --2- 172.21.15.16:0/3024281172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11b810bdb0 0x7f11b81a1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:49.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.594+0000 7f11bca00640 1 -- 172.21.15.16:0/3024281172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11b8105b20 msgr2=0x7f11b819c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:49.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.594+0000 7f11bca00640 1 --2- 172.21.15.16:0/3024281172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11b8105b20 0x7f11b819c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:49.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.594+0000 7f11bca00640 1 -- 172.21.15.16:0/3024281172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11b81a23e0 con 0x7f11b810b460 2024-08-31T06:23:49.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.594+0000 7f11bde00640 1 --2- 172.21.15.16:0/3024281172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11b810bdb0 0x7f11b81a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:23:49.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.594+0000 7f11bca00640 1 --2- 172.21.15.16:0/3024281172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11b810b460 0x7f11b819ca10 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f11a400e9f0 tx=0x7f11a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:49.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.594+0000 7f11b6000640 1 -- 172.21.15.16:0/3024281172 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a400ccc0 con 0x7f11b810b460 2024-08-31T06:23:49.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.594+0000 7f11b6000640 1 -- 172.21.15.16:0/3024281172 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11a40108e0 con 0x7f11b810b460 2024-08-31T06:23:49.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.594+0000 7f11bfe00640 1 -- 172.21.15.16:0/3024281172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11b81a26d0 con 0x7f11b810b460 2024-08-31T06:23:49.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.594+0000 7f11b6000640 1 -- 172.21.15.16:0/3024281172 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a4018860 con 0x7f11b810b460 2024-08-31T06:23:49.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.594+0000 7f11bfe00640 1 -- 172.21.15.16:0/3024281172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11b81a8e30 con 0x7f11b810b460 2024-08-31T06:23:49.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.596+0000 7f11bfe00640 1 -- 172.21.15.16:0/3024281172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11b8105f30 con 0x7f11b810b460 2024-08-31T06:23:49.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.599+0000 7f11b6000640 1 -- 172.21.15.16:0/3024281172 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f11a4010430 con 0x7f11b810b460 2024-08-31T06:23:49.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.600+0000 7f11b6000640 1 --2- 172.21.15.16:0/3024281172 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f11a00b4ca0 0x7f11a00b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:49.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.600+0000 7f11b6000640 1 -- 172.21.15.16:0/3024281172 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8470+0+0 (secure 0 0 0) 0x7f11a40e5800 con 0x7f11b810b460 2024-08-31T06:23:49.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.600+0000 7f11bd400640 1 --2- 172.21.15.16:0/3024281172 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f11a00b4ca0 0x7f11a00b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:49.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.601+0000 7f11bd400640 1 --2- 172.21.15.16:0/3024281172 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f11a00b4ca0 0x7f11a00b71b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f11ac005ab0 tx=0x7f11ac005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:49.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.604+0000 7f11b6000640 1 -- 172.21.15.16:0/3024281172 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f11a40abf70 con 0x7f11b810b460 2024-08-31T06:23:49.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.813+0000 7f11bfe00640 1 -- 172.21.15.16:0/3024281172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f11b802e430 con 0x7f11b810b460 2024-08-31T06:23:49.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.813+0000 7f11b6000640 1 -- 172.21.15.16:0/3024281172 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v103) ==== 81+0+887 (secure 0 0 0) 0x7f11a40b0730 con 0x7f11b810b460 2024-08-31T06:23:49.816 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3923220960 2024-09-01T05:57:23.153214+0000 2024-08-31T06:23:49.816 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3013533836 2024-09-01T05:57:23.153214+0000 2024-08-31T06:23:49.817 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/3755683228 2024-09-01T05:55:45.075902+0000 2024-08-31T06:23:49.817 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3064435470 2024-09-01T05:55:45.075902+0000 2024-08-31T06:23:49.817 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/911709962 2024-09-01T05:55:45.075902+0000 2024-08-31T06:23:49.817 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/2937368967 2024-09-01T05:55:45.075902+0000 2024-08-31T06:23:49.817 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/1378972230 2024-09-01T05:56:14.195704+0000 2024-08-31T06:23:49.817 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3047316995 2024-09-01T05:56:14.195704+0000 2024-08-31T06:23:49.817 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/3755683228 2024-09-01T05:55:45.075902+0000 2024-08-31T06:23:49.817 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/2571082634 2024-09-01T05:57:23.153214+0000 2024-08-31T06:23:49.817 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/2136704295 2024-09-01T05:56:14.195704+0000 2024-08-31T06:23:49.817 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/2259332778 2024-09-01T05:56:14.195704+0000 2024-08-31T06:23:49.817 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/2571082634 2024-09-01T05:57:23.153214+0000 2024-08-31T06:23:49.817 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/1378972230 2024-09-01T05:56:14.195704+0000 2024-08-31T06:23:49.817 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/2126457673 2024-09-01T05:57:23.153214+0000 2024-08-31T06:23:49.818 INFO:teuthology.orchestra.run.smithi016.stderr:listed 15 entries 2024-08-31T06:23:49.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.816+0000 7f11bfe00640 1 -- 172.21.15.16:0/3024281172 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f11a00b4ca0 msgr2=0x7f11a00b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:49.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.816+0000 7f11bfe00640 1 --2- 172.21.15.16:0/3024281172 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f11a00b4ca0 0x7f11a00b71b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f11ac005ab0 tx=0x7f11ac005a20 comp rx=0 tx=0).stop 2024-08-31T06:23:49.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.816+0000 7f11bfe00640 1 -- 172.21.15.16:0/3024281172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11b810b460 msgr2=0x7f11b819ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:49.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.816+0000 7f11bfe00640 1 --2- 172.21.15.16:0/3024281172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11b810b460 0x7f11b819ca10 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f11a400e9f0 tx=0x7f11a400eeb0 comp rx=0 tx=0).stop 2024-08-31T06:23:49.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.816+0000 7f11bfe00640 1 -- 172.21.15.16:0/3024281172 shutdown_connections 2024-08-31T06:23:49.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.816+0000 7f11bfe00640 1 --2- 172.21.15.16:0/3024281172 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f11a00b4ca0 0x7f11a00b71b0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:49.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.816+0000 7f11bfe00640 1 --2- 172.21.15.16:0/3024281172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11b810bdb0 0x7f11b81a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:49.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.816+0000 7f11bfe00640 1 --2- 172.21.15.16:0/3024281172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11b810b460 0x7f11b819ca10 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:49.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.816+0000 7f11bfe00640 1 --2- 172.21.15.16:0/3024281172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11b8105b20 0x7f11b819c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:49.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.816+0000 7f11bfe00640 1 -- 172.21.15.16:0/3024281172 >> 172.21.15.16:0/3024281172 conn(0x7f11b81015b0 msgr2=0x7f11b810aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:49.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.816+0000 7f11bfe00640 1 -- 172.21.15.16:0/3024281172 shutdown_connections 2024-08-31T06:23:49.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:49.816+0000 7f11bfe00640 1 -- 172.21.15.16:0/3024281172 wait complete. 2024-08-31T06:23:50.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:49 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/3024281172' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-08-31T06:23:50.152 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:49 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/3024281172' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-08-31T06:23:50.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:49 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/3024281172' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-08-31T06:23:50.695 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-08-31T06:23:50.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-31T06:23:50.695 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /tmp/cephfs 2024-08-31T06:23:50.820 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-08-31T06:23:50.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-31T06:23:50.820 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2024-08-31T06:23:50.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-31T06:23:50.904 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-08-31T06:23:50.974 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--tmp-cephfs (id: 0) 2024-08-31T06:23:50.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-31T06:23:50.974 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns delete ceph-ns--tmp-cephfs 2024-08-31T06:23:51.075 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:50 smithi016 ceph-mon[17194]: pgmap v884: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-08-31T06:23:51.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-31T06:23:51.106 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2024-08-31T06:23:51.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:50 smithi155 ceph-mon[21742]: pgmap v884: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-08-31T06:23:51.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:50 smithi184 ceph-mon[22331]: pgmap v884: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-08-31T06:23:51.442 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-08-31T06:23:51.452 INFO:tasks.cephadm:Teardown begin 2024-08-31T06:23:51.452 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-08-31T06:23:51.452 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:23:51.853 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:23:52.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.537+0000 7fb2c0c00640 1 -- 172.21.15.16:0/1697427018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2bc104660 msgr2=0x7fb2bc104a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:52.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.537+0000 7fb2c0c00640 1 --2- 172.21.15.16:0/1697427018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2bc104660 0x7fb2bc104a70 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac009900 tx=0x7fb2ac02a410 comp rx=0 tx=0).stop 2024-08-31T06:23:52.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.537+0000 7fb2c0c00640 1 -- 172.21.15.16:0/1697427018 shutdown_connections 2024-08-31T06:23:52.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.537+0000 7fb2c0c00640 1 --2- 172.21.15.16:0/1697427018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2bc105040 0x7fb2bc10d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:52.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.537+0000 7fb2c0c00640 1 --2- 172.21.15.16:0/1697427018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2bc10dc10 0x7fb2bc1100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:52.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.537+0000 7fb2c0c00640 1 --2- 172.21.15.16:0/1697427018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2bc104660 0x7fb2bc104a70 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:52.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.537+0000 7fb2c0c00640 1 -- 172.21.15.16:0/1697427018 >> 172.21.15.16:0/1697427018 conn(0x7fb2bc079500 msgr2=0x7fb2bc079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:52.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.537+0000 7fb2c0c00640 1 -- 172.21.15.16:0/1697427018 shutdown_connections 2024-08-31T06:23:52.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.537+0000 7fb2c0c00640 1 -- 172.21.15.16:0/1697427018 wait complete. 2024-08-31T06:23:52.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.538+0000 7fb2c0c00640 1 Processor -- start 2024-08-31T06:23:52.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.538+0000 7fb2c0c00640 1 -- start start 2024-08-31T06:23:52.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.539+0000 7fb2c0c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2bc104660 0x7fb2bc19c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:52.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.539+0000 7fb2c0c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2bc105040 0x7fb2bc19ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.539+0000 7fb2c0c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2bc10dc10 0x7fb2bc1a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.539+0000 7fb2c0c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc1138a0 con 0x7fb2bc104660 2024-08-31T06:23:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.539+0000 7fb2c0c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc113720 con 0x7fb2bc105040 2024-08-31T06:23:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.539+0000 7fb2c0c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc113a20 con 0x7fb2bc10dc10 2024-08-31T06:23:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.539+0000 7fb2b9e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2bc104660 0x7fb2bc19c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.539+0000 7fb2b9400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2bc105040 0x7fb2bc19ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.539+0000 7fb2b9e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2bc104660 0x7fb2bc19c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41860/0 (socket says 172.21.15.16:41860) 2024-08-31T06:23:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.539+0000 7fb2b9400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2bc105040 0x7fb2bc19ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:49316/0 (socket says 172.21.15.16:49316) 2024-08-31T06:23:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.539+0000 7fb2b9e00640 1 -- 172.21.15.16:0/702729293 learned_addr learned my addr 172.21.15.16:0/702729293 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:23:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.539+0000 7fb2ba800640 1 --2- 172.21.15.16:0/702729293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2bc10dc10 0x7fb2bc1a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.540+0000 7fb2b9400640 1 -- 172.21.15.16:0/702729293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2bc10dc10 msgr2=0x7fb2bc1a1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.540+0000 7fb2b9400640 1 --2- 172.21.15.16:0/702729293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2bc10dc10 0x7fb2bc1a1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:52.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.540+0000 7fb2b9400640 1 -- 172.21.15.16:0/702729293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2bc104660 msgr2=0x7fb2bc19c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:52.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.540+0000 7fb2b9400640 1 --2- 172.21.15.16:0/702729293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2bc104660 0x7fb2bc19c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:52.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.540+0000 7fb2b9400640 1 -- 172.21.15.16:0/702729293 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2bc1a2460 con 0x7fb2bc105040 2024-08-31T06:23:52.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.540+0000 7fb2b9e00640 1 --2- 172.21.15.16:0/702729293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2bc104660 0x7fb2bc19c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:23:52.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.540+0000 7fb2ba800640 1 --2- 172.21.15.16:0/702729293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2bc10dc10 0x7fb2bc1a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:23:52.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.540+0000 7fb2b9400640 1 --2- 172.21.15.16:0/702729293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2bc105040 0x7fb2bc19ca90 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc06b540 tx=0x7fb2a800ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:52.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.541+0000 7fb2b2a00640 1 -- 172.21.15.16:0/702729293 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a800cc40 con 0x7fb2bc105040 2024-08-31T06:23:52.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.541+0000 7fb2c0c00640 1 -- 172.21.15.16:0/702729293 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2bc1a2750 con 0x7fb2bc105040 2024-08-31T06:23:52.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.541+0000 7fb2c0c00640 1 -- 172.21.15.16:0/702729293 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2bc1a8eb0 con 0x7fb2bc105040 2024-08-31T06:23:52.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.541+0000 7fb2b2a00640 1 -- 172.21.15.16:0/702729293 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2a800cde0 con 0x7fb2bc105040 2024-08-31T06:23:52.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.541+0000 7fb2b2a00640 1 -- 172.21.15.16:0/702729293 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8018640 con 0x7fb2bc105040 2024-08-31T06:23:52.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.542+0000 7fb2c0c00640 1 -- 172.21.15.16:0/702729293 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb29c0052c0 con 0x7fb2bc105040 2024-08-31T06:23:52.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.545+0000 7fb2b2a00640 1 -- 172.21.15.16:0/702729293 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fb2a8005030 con 0x7fb2bc105040 2024-08-31T06:23:52.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.548+0000 7fb2b2a00640 1 --2- 172.21.15.16:0/702729293 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb2a00b4ca0 0x7fb2a00b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:52.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.548+0000 7fb2b2a00640 1 -- 172.21.15.16:0/702729293 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8470+0+0 (secure 0 0 0) 0x7fb2a80e5920 con 0x7fb2bc105040 2024-08-31T06:23:52.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.548+0000 7fb2b9e00640 1 --2- 172.21.15.16:0/702729293 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb2a00b4ca0 0x7fb2a00b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:52.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.548+0000 7fb2b2a00640 1 -- 172.21.15.16:0/702729293 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fb2a80ac090 con 0x7fb2bc105040 2024-08-31T06:23:52.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.549+0000 7fb2b9e00640 1 --2- 172.21.15.16:0/702729293 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb2a00b4ca0 0x7fb2a00b71b0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac000c00 tx=0x7fb2ac005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:52.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.781+0000 7fb2c0c00640 1 -- 172.21.15.16:0/702729293 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb29c0055b0 con 0x7fb2bc105040 2024-08-31T06:23:52.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.782+0000 7fb2b2a00640 1 -- 172.21.15.16:0/702729293 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11921 (secure 0 0 0) 0x7fb2a80b0850 con 0x7fb2bc105040 2024-08-31T06:23:52.785 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:23:52.785 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":55,"btime":"2024-08-31T06:22:47:897383+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":55,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:22:47.738871+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24737,24724,24712]},"id":1}]} 2024-08-31T06:23:52.785 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 55 2024-08-31T06:23:52.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.784+0000 7fb2c0c00640 1 -- 172.21.15.16:0/702729293 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb2a00b4ca0 msgr2=0x7fb2a00b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:52.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.784+0000 7fb2c0c00640 1 --2- 172.21.15.16:0/702729293 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb2a00b4ca0 0x7fb2a00b71b0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac000c00 tx=0x7fb2ac005a20 comp rx=0 tx=0).stop 2024-08-31T06:23:52.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.784+0000 7fb2c0c00640 1 -- 172.21.15.16:0/702729293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2bc105040 msgr2=0x7fb2bc19ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:52.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.784+0000 7fb2c0c00640 1 --2- 172.21.15.16:0/702729293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2bc105040 0x7fb2bc19ca90 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc06b540 tx=0x7fb2a800ee50 comp rx=0 tx=0).stop 2024-08-31T06:23:52.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.784+0000 7fb2c0c00640 1 -- 172.21.15.16:0/702729293 shutdown_connections 2024-08-31T06:23:52.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.784+0000 7fb2c0c00640 1 --2- 172.21.15.16:0/702729293 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fb2a00b4ca0 0x7fb2a00b71b0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:52.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.784+0000 7fb2c0c00640 1 --2- 172.21.15.16:0/702729293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2bc10dc10 0x7fb2bc1a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:52.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.784+0000 7fb2c0c00640 1 --2- 172.21.15.16:0/702729293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2bc105040 0x7fb2bc19ca90 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:52.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.784+0000 7fb2c0c00640 1 --2- 172.21.15.16:0/702729293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2bc104660 0x7fb2bc19c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:52.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.784+0000 7fb2c0c00640 1 -- 172.21.15.16:0/702729293 >> 172.21.15.16:0/702729293 conn(0x7fb2bc079500 msgr2=0x7fb2bc100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:52.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.785+0000 7fb2c0c00640 1 -- 172.21.15.16:0/702729293 shutdown_connections 2024-08-31T06:23:52.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:52.785+0000 7fb2c0c00640 1 -- 172.21.15.16:0/702729293 wait complete. 2024-08-31T06:23:53.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:52 smithi155 ceph-mon[21742]: pgmap v885: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-08-31T06:23:53.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:52 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/702729293' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:23:53.140 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:52 smithi016 ceph-mon[17194]: pgmap v885: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-08-31T06:23:53.140 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:52 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/702729293' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:23:53.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:52 smithi184 ceph-mon[22331]: pgmap v885: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-08-31T06:23:53.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:52 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/702729293' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:23:53.540 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs dump --format=json 2024-08-31T06:23:53.926 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:23:54.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.740+0000 7f7e8da00640 1 -- 172.21.15.16:0/3112463893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e88077120 msgr2=0x7f7e88075550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:54.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.740+0000 7f7e8da00640 1 --2- 172.21.15.16:0/3112463893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e88077120 0x7f7e88075550 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f7e78009900 tx=0x7f7e7802a400 comp rx=0 tx=0).stop 2024-08-31T06:23:54.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.741+0000 7f7e8da00640 1 -- 172.21.15.16:0/3112463893 shutdown_connections 2024-08-31T06:23:54.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.741+0000 7f7e8da00640 1 --2- 172.21.15.16:0/3112463893 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e88075b70 0x7f7e88075f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:54.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.741+0000 7f7e8da00640 1 --2- 172.21.15.16:0/3112463893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e88114020 0x7f7e881164f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:54.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.741+0000 7f7e8da00640 1 --2- 172.21.15.16:0/3112463893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e88077120 0x7f7e88075550 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:54.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.741+0000 7f7e8da00640 1 -- 172.21.15.16:0/3112463893 >> 172.21.15.16:0/3112463893 conn(0x7f7e880ff1e0 msgr2=0x7f7e88101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:54.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.741+0000 7f7e8da00640 1 -- 172.21.15.16:0/3112463893 shutdown_connections 2024-08-31T06:23:54.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.741+0000 7f7e8da00640 1 -- 172.21.15.16:0/3112463893 wait complete. 2024-08-31T06:23:54.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.742+0000 7f7e8da00640 1 Processor -- start 2024-08-31T06:23:54.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.742+0000 7f7e8da00640 1 -- start start 2024-08-31T06:23:54.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.742+0000 7f7e8da00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e88075b70 0x7f7e881a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:54.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.742+0000 7f7e8da00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e88077120 0x7f7e881a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:54.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.743+0000 7f7e8da00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e88114020 0x7f7e881a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:54.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.743+0000 7f7e8da00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7e88119cb0 con 0x7f7e88075b70 2024-08-31T06:23:54.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.743+0000 7f7e8da00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7e88119b30 con 0x7f7e88114020 2024-08-31T06:23:54.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.743+0000 7f7e8da00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7e88119e30 con 0x7f7e88077120 2024-08-31T06:23:54.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.743+0000 7f7e87e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e88075b70 0x7f7e881a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:54.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.743+0000 7f7e87e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e88075b70 0x7f7e881a06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41872/0 (socket says 172.21.15.16:41872) 2024-08-31T06:23:54.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.743+0000 7f7e87e00640 1 -- 172.21.15.16:0/2961409769 learned_addr learned my addr 172.21.15.16:0/2961409769 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:23:54.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.743+0000 7f7e7fe00640 1 --2- 172.21.15.16:0/2961409769 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e88077120 0x7f7e881a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:54.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.743+0000 7f7e8d000640 1 --2- 172.21.15.16:0/2961409769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e88114020 0x7f7e881a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:54.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.743+0000 7f7e87e00640 1 -- 172.21.15.16:0/2961409769 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e88077120 msgr2=0x7f7e881a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:54.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.743+0000 7f7e87e00640 1 --2- 172.21.15.16:0/2961409769 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e88077120 0x7f7e881a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:54.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.743+0000 7f7e87e00640 1 -- 172.21.15.16:0/2961409769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e88114020 msgr2=0x7f7e881a5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:54.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.743+0000 7f7e87e00640 1 --2- 172.21.15.16:0/2961409769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e88114020 0x7f7e881a5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:54.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.743+0000 7f7e87e00640 1 -- 172.21.15.16:0/2961409769 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e881a6650 con 0x7f7e88075b70 2024-08-31T06:23:54.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.744+0000 7f7e87e00640 1 --2- 172.21.15.16:0/2961409769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e88075b70 0x7f7e881a06e0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f7e78005ab0 tx=0x7f7e78003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:54.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.744+0000 7f7e85600640 1 -- 172.21.15.16:0/2961409769 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7803f070 con 0x7f7e88075b70 2024-08-31T06:23:54.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.744+0000 7f7e85600640 1 -- 172.21.15.16:0/2961409769 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e78031b60 con 0x7f7e88075b70 2024-08-31T06:23:54.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.744+0000 7f7e8da00640 1 -- 172.21.15.16:0/2961409769 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e881a68e0 con 0x7f7e88075b70 2024-08-31T06:23:54.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.744+0000 7f7e85600640 1 -- 172.21.15.16:0/2961409769 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e78035030 con 0x7f7e88075b70 2024-08-31T06:23:54.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.744+0000 7f7e8da00640 1 -- 172.21.15.16:0/2961409769 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e881ad0a0 con 0x7f7e88075b70 2024-08-31T06:23:54.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.746+0000 7f7e8da00640 1 -- 172.21.15.16:0/2961409769 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e88075f80 con 0x7f7e88075b70 2024-08-31T06:23:54.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.750+0000 7f7e85600640 1 -- 172.21.15.16:0/2961409769 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f7e78042070 con 0x7f7e88075b70 2024-08-31T06:23:54.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.751+0000 7f7e85600640 1 --2- 172.21.15.16:0/2961409769 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7e6c0b4d20 0x7f7e6c0b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:54.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.751+0000 7f7e85600640 1 -- 172.21.15.16:0/2961409769 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8470+0+0 (secure 0 0 0) 0x7f7e78105ee0 con 0x7f7e88075b70 2024-08-31T06:23:54.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.751+0000 7f7e7fe00640 1 --2- 172.21.15.16:0/2961409769 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7e6c0b4d20 0x7f7e6c0b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:54.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.752+0000 7f7e7fe00640 1 --2- 172.21.15.16:0/2961409769 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7e6c0b4d20 0x7f7e6c0b7230 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7e88115fa0 tx=0x7f7e70005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:54.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.754+0000 7f7e85600640 1 -- 172.21.15.16:0/2961409769 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f7e780ccca0 con 0x7f7e88075b70 2024-08-31T06:23:54.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.987+0000 7f7e8da00640 1 -- 172.21.15.16:0/2961409769 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e881a1810 con 0x7f7e88075b70 2024-08-31T06:23:54.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.987+0000 7f7e85600640 1 -- 172.21.15.16:0/2961409769 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11921 (secure 0 0 0) 0x7f7e881a1810 con 0x7f7e88075b70 2024-08-31T06:23:54.990 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:23:54.990 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":55,"btime":"2024-08-31T06:22:47:897383+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14643,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3365059950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3365059950},{"type":"v1","addr":"172.21.15.16:6835","nonce":3365059950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3621101104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3621101104},{"type":"v1","addr":"172.21.15.16:6841","nonce":3621101104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24637,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/1204885928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1204885928},{"type":"v1","addr":"172.21.15.184:6833","nonce":1204885928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2239981116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2239981116},{"type":"v1","addr":"172.21.15.155:6833","nonce":2239981116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/368986894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":368986894},{"type":"v1","addr":"172.21.15.16:6837","nonce":368986894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1634403570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1634403570},{"type":"v1","addr":"172.21.15.155:6835","nonce":1634403570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":55,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-31T06:04:27.098649+0000","modified":"2024-08-31T06:22:47.738871+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24737,"mds_1":24724,"mds_2":24712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24688":{"gid":24688,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/1925472309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1925472309},{"type":"v1","addr":"172.21.15.184:6835","nonce":1925472309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/1712755588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":1712755588},{"type":"v1","addr":"172.21.15.184:6837","nonce":1712755588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24712":{"gid":24712,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/4011051479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":4011051479},{"type":"v1","addr":"172.21.15.184:6839","nonce":4011051479}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24724":{"gid":24724,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6839/219664460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":219664460},{"type":"v1","addr":"172.21.15.16:6839","nonce":219664460}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24737":{"gid":24737,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/390606290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":390606290},{"type":"v1","addr":"172.21.15.155:6839","nonce":390606290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25511":{"gid":25511,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1791912179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1791912179},{"type":"v1","addr":"172.21.15.155:6837","nonce":1791912179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24737,"qdb_cluster":[24737,24724,24712]},"id":1}]} 2024-08-31T06:23:54.990 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 55 2024-08-31T06:23:54.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.990+0000 7f7e8da00640 1 -- 172.21.15.16:0/2961409769 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7e6c0b4d20 msgr2=0x7f7e6c0b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:54.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.990+0000 7f7e8da00640 1 --2- 172.21.15.16:0/2961409769 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7e6c0b4d20 0x7f7e6c0b7230 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7e88115fa0 tx=0x7f7e70005f90 comp rx=0 tx=0).stop 2024-08-31T06:23:54.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.990+0000 7f7e8da00640 1 -- 172.21.15.16:0/2961409769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e88075b70 msgr2=0x7f7e881a06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:54.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.990+0000 7f7e8da00640 1 --2- 172.21.15.16:0/2961409769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e88075b70 0x7f7e881a06e0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f7e78005ab0 tx=0x7f7e78003c10 comp rx=0 tx=0).stop 2024-08-31T06:23:54.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.990+0000 7f7e8da00640 1 -- 172.21.15.16:0/2961409769 shutdown_connections 2024-08-31T06:23:54.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.990+0000 7f7e8da00640 1 --2- 172.21.15.16:0/2961409769 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f7e6c0b4d20 0x7f7e6c0b7230 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:54.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.990+0000 7f7e8da00640 1 --2- 172.21.15.16:0/2961409769 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e88077120 0x7f7e881a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:54.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.990+0000 7f7e8da00640 1 --2- 172.21.15.16:0/2961409769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e88114020 0x7f7e881a5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:54.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.990+0000 7f7e8da00640 1 --2- 172.21.15.16:0/2961409769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e88075b70 0x7f7e881a06e0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:54.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.990+0000 7f7e8da00640 1 -- 172.21.15.16:0/2961409769 >> 172.21.15.16:0/2961409769 conn(0x7f7e880ff1e0 msgr2=0x7f7e8810ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:54.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.990+0000 7f7e8da00640 1 -- 172.21.15.16:0/2961409769 shutdown_connections 2024-08-31T06:23:54.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:54.990+0000 7f7e8da00640 1 -- 172.21.15.16:0/2961409769 wait complete. 2024-08-31T06:23:55.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:54 smithi016 ceph-mon[17194]: pgmap v886: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 8 op/s 2024-08-31T06:23:55.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:54 smithi155 ceph-mon[21742]: pgmap v886: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 8 op/s 2024-08-31T06:23:55.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:54 smithi184 ceph-mon[22331]: pgmap v886: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 8 op/s 2024-08-31T06:23:55.815 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph osd dump --format=json 2024-08-31T06:23:56.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:55 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/2961409769' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:23:56.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:55 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/2961409769' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:23:56.196 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:23:56.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:55 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/2961409769' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-31T06:23:56.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:56.947+0000 7f3eb9c00640 1 -- 172.21.15.16:0/1780381482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3eb4109090 msgr2=0x7f3eb41094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:56.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:56.947+0000 7f3eb9c00640 1 --2- 172.21.15.16:0/1780381482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3eb4109090 0x7f3eb41094a0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c009900 tx=0x7f3e9c02a3f0 comp rx=0 tx=0).stop 2024-08-31T06:23:56.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:56.948+0000 7f3eb9c00640 1 -- 172.21.15.16:0/1780381482 shutdown_connections 2024-08-31T06:23:56.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:56.948+0000 7f3eb9c00640 1 --2- 172.21.15.16:0/1780381482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3eb41099e0 0x7f3eb4110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:56.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:56.948+0000 7f3eb9c00640 1 --2- 172.21.15.16:0/1780381482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3eb4109090 0x7f3eb41094a0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:56.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:56.948+0000 7f3eb9c00640 1 --2- 172.21.15.16:0/1780381482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3eb4103750 0x7f3eb4103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:56.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:56.948+0000 7f3eb9c00640 1 -- 172.21.15.16:0/1780381482 >> 172.21.15.16:0/1780381482 conn(0x7f3eb40ff1a0 msgr2=0x7f3eb41015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:56.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:56.948+0000 7f3eb9c00640 1 -- 172.21.15.16:0/1780381482 shutdown_connections 2024-08-31T06:23:56.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:56.948+0000 7f3eb9c00640 1 -- 172.21.15.16:0/1780381482 wait complete. 2024-08-31T06:23:56.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:56.949+0000 7f3eb9c00640 1 Processor -- start 2024-08-31T06:23:56.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:56.949+0000 7f3eb9c00640 1 -- start start 2024-08-31T06:23:56.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:56.949+0000 7f3eb9c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3eb4103750 0x7f3eb419c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:56.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:56.949+0000 7f3eb9c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3eb4109090 0x7f3eb419c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:56.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:56.949+0000 7f3eb9c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3eb41099e0 0x7f3eb41a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:56.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:56.949+0000 7f3eb9c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb4113650 con 0x7f3eb41099e0 2024-08-31T06:23:56.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:56.949+0000 7f3eb9c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb41134d0 con 0x7f3eb4109090 2024-08-31T06:23:56.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:56.949+0000 7f3eb9c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb41137d0 con 0x7f3eb4103750 2024-08-31T06:23:56.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:56.950+0000 7f3eb3e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3eb4103750 0x7f3eb419c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:56.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:56.950+0000 7f3eb3e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3eb4103750 0x7f3eb419c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:34994/0 (socket says 172.21.15.16:34994) 2024-08-31T06:23:56.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:56.950+0000 7f3eb3e00640 1 -- 172.21.15.16:0/150033482 learned_addr learned my addr 172.21.15.16:0/150033482 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:23:56.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:56.950+0000 7f3eb9200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3eb41099e0 0x7f3eb41a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:56.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:56.950+0000 7f3eab400640 1 --2- 172.21.15.16:0/150033482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3eb4109090 0x7f3eb419c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:56.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:56.950+0000 7f3eb3e00640 1 -- 172.21.15.16:0/150033482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3eb4109090 msgr2=0x7f3eb419c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:56.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:56.950+0000 7f3eb3e00640 1 --2- 172.21.15.16:0/150033482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3eb4109090 0x7f3eb419c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:56.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:56.950+0000 7f3eb3e00640 1 -- 172.21.15.16:0/150033482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3eb41099e0 msgr2=0x7f3eb41a1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:56.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:56.950+0000 7f3eb3e00640 1 --2- 172.21.15.16:0/150033482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3eb41099e0 0x7f3eb41a1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:56.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:56.950+0000 7f3eb3e00640 1 -- 172.21.15.16:0/150033482 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb41a2230 con 0x7f3eb4103750 2024-08-31T06:23:56.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:56.950+0000 7f3eb3e00640 1 --2- 172.21.15.16:0/150033482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3eb4103750 0x7f3eb419c2c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f3ea400e9f0 tx=0x7f3ea400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:56.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:56.951+0000 7f3eb1600640 1 -- 172.21.15.16:0/150033482 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea400cd80 con 0x7f3eb4103750 2024-08-31T06:23:56.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:56.951+0000 7f3eb9c00640 1 -- 172.21.15.16:0/150033482 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eb41a2520 con 0x7f3eb4103750 2024-08-31T06:23:56.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:56.951+0000 7f3eb9c00640 1 -- 172.21.15.16:0/150033482 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eb41a8c80 con 0x7f3eb4103750 2024-08-31T06:23:56.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:56.951+0000 7f3eb1600640 1 -- 172.21.15.16:0/150033482 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ea40108e0 con 0x7f3eb4103750 2024-08-31T06:23:56.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:56.951+0000 7f3eb1600640 1 -- 172.21.15.16:0/150033482 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea40188e0 con 0x7f3eb4103750 2024-08-31T06:23:56.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:56.952+0000 7f3eb9c00640 1 -- 172.21.15.16:0/150033482 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e900052c0 con 0x7f3eb4103750 2024-08-31T06:23:56.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:56.954+0000 7f3eb1600640 1 -- 172.21.15.16:0/150033482 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f3ea4010430 con 0x7f3eb4103750 2024-08-31T06:23:56.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:56.956+0000 7f3eb1600640 1 --2- 172.21.15.16:0/150033482 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3e980b4d20 0x7f3e980b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:56.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:56.956+0000 7f3eb1600640 1 -- 172.21.15.16:0/150033482 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8470+0+0 (secure 0 0 0) 0x7f3ea40e56a0 con 0x7f3eb4103750 2024-08-31T06:23:56.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:56.956+0000 7f3eab400640 1 --2- 172.21.15.16:0/150033482 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3e980b4d20 0x7f3e980b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:56.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:56.957+0000 7f3eab400640 1 --2- 172.21.15.16:0/150033482 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3e980b4d20 0x7f3e980b7230 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c009230 tx=0x7f3e9c031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:56.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:56.960+0000 7f3eb1600640 1 -- 172.21.15.16:0/150033482 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f3ea40abe10 con 0x7f3eb4103750 2024-08-31T06:23:57.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:56 smithi155 ceph-mon[21742]: pgmap v887: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-08-31T06:23:57.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:57.159+0000 7f3eb9c00640 1 -- 172.21.15.16:0/150033482 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3e90005880 con 0x7f3eb4103750 2024-08-31T06:23:57.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:56 smithi016 ceph-mon[17194]: pgmap v887: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-08-31T06:23:57.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:57.161+0000 7f3eb1600640 1 -- 172.21.15.16:0/150033482 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+23137 (secure 0 0 0) 0x7f3ea40b05d0 con 0x7f3eb4103750 2024-08-31T06:23:57.164 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-31T06:23:57.164 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":103,"fsid":"673e4aa0-675d-11ef-bcd4-c7b262605968","created":"2024-08-31T05:55:09.632536+0000","modified":"2024-08-31T06:23:15.913786+0000","last_up_change":"2024-08-31T06:03:27.893697+0000","last_in_change":"2024-08-31T06:03:05.664945+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-31T05:59:23.642270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-31T06:04:20.871835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-31T06:04:24.119499+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":29,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e349dae-6aa5-4b6d-9107-a945655e11bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6803","nonce":1725037400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6805","nonce":1725037400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6809","nonce":1725037400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1725037400},{"type":"v1","addr":"172.21.15.16:6807","nonce":1725037400}]},"public_addr":"172.21.15.16:6803/1725037400","cluster_addr":"172.21.15.16:6805/1725037400","heartbeat_back_addr":"172.21.15.16:6809/1725037400","heartbeat_front_addr":"172.21.15.16:6807/1725037400","state":["exists","up"]},{"osd":1,"uuid":"bd645dc3-93b5-40af-89d8-08ccec22e50d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6801","nonce":4241618440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6803","nonce":4241618440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6807","nonce":4241618440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4241618440},{"type":"v1","addr":"172.21.15.155:6805","nonce":4241618440}]},"public_addr":"172.21.15.155:6801/4241618440","cluster_addr":"172.21.15.155:6803/4241618440","heartbeat_back_addr":"172.21.15.155:6807/4241618440","heartbeat_front_addr":"172.21.15.155:6805/4241618440","state":["exists","up"]},{"osd":2,"uuid":"c69c8c19-32f8-400a-86d4-0f4ce240a7de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6801","nonce":2193296422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6803","nonce":2193296422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6807","nonce":2193296422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2193296422},{"type":"v1","addr":"172.21.15.184:6805","nonce":2193296422}]},"public_addr":"172.21.15.184:6801/2193296422","cluster_addr":"172.21.15.184:6803/2193296422","heartbeat_back_addr":"172.21.15.184:6807/2193296422","heartbeat_front_addr":"172.21.15.184:6805/2193296422","state":["exists","up"]},{"osd":3,"uuid":"8892e35f-6880-434c-a805-ab8ccbed2bf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6811","nonce":81681815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6813","nonce":81681815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6817","nonce":81681815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":81681815},{"type":"v1","addr":"172.21.15.16:6815","nonce":81681815}]},"public_addr":"172.21.15.16:6811/81681815","cluster_addr":"172.21.15.16:6813/81681815","heartbeat_back_addr":"172.21.15.16:6817/81681815","heartbeat_front_addr":"172.21.15.16:6815/81681815","state":["exists","up"]},{"osd":4,"uuid":"d2515e73-88f7-4654-9762-adb8802777ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6809","nonce":3592422019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6811","nonce":3592422019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6815","nonce":3592422019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":3592422019},{"type":"v1","addr":"172.21.15.155:6813","nonce":3592422019}]},"public_addr":"172.21.15.155:6809/3592422019","cluster_addr":"172.21.15.155:6811/3592422019","heartbeat_back_addr":"172.21.15.155:6815/3592422019","heartbeat_front_addr":"172.21.15.155:6813/3592422019","state":["exists","up"]},{"osd":5,"uuid":"5ce6e2fa-c17c-49b3-9402-f19bfc21076d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6809","nonce":2054894065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6811","nonce":2054894065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6815","nonce":2054894065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":2054894065},{"type":"v1","addr":"172.21.15.184:6813","nonce":2054894065}]},"public_addr":"172.21.15.184:6809/2054894065","cluster_addr":"172.21.15.184:6811/2054894065","heartbeat_back_addr":"172.21.15.184:6815/2054894065","heartbeat_front_addr":"172.21.15.184:6813/2054894065","state":["exists","up"]},{"osd":6,"uuid":"f257d8d1-ad59-4bd8-821d-bf29e50b547e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6819","nonce":1697421092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6821","nonce":1697421092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6825","nonce":1697421092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1697421092},{"type":"v1","addr":"172.21.15.16:6823","nonce":1697421092}]},"public_addr":"172.21.15.16:6819/1697421092","cluster_addr":"172.21.15.16:6821/1697421092","heartbeat_back_addr":"172.21.15.16:6825/1697421092","heartbeat_front_addr":"172.21.15.16:6823/1697421092","state":["exists","up"]},{"osd":7,"uuid":"8e76c20e-8d25-423b-b7e8-de642d9c9a6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6817","nonce":2671827901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6819","nonce":2671827901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6823","nonce":2671827901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":2671827901},{"type":"v1","addr":"172.21.15.155:6821","nonce":2671827901}]},"public_addr":"172.21.15.155:6817/2671827901","cluster_addr":"172.21.15.155:6819/2671827901","heartbeat_back_addr":"172.21.15.155:6823/2671827901","heartbeat_front_addr":"172.21.15.155:6821/2671827901","state":["exists","up"]},{"osd":8,"uuid":"0f0bb23f-4bc3-4198-8381-5b5cd16ae394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6817","nonce":2479117823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6819","nonce":2479117823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6823","nonce":2479117823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":2479117823},{"type":"v1","addr":"172.21.15.184:6821","nonce":2479117823}]},"public_addr":"172.21.15.184:6817/2479117823","cluster_addr":"172.21.15.184:6819/2479117823","heartbeat_back_addr":"172.21.15.184:6823/2479117823","heartbeat_front_addr":"172.21.15.184:6821/2479117823","state":["exists","up"]},{"osd":9,"uuid":"bc27afcb-7871-4f72-a043-044fd5fe7d86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6827","nonce":3015700546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6829","nonce":3015700546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6833","nonce":3015700546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3015700546},{"type":"v1","addr":"172.21.15.16:6831","nonce":3015700546}]},"public_addr":"172.21.15.16:6827/3015700546","cluster_addr":"172.21.15.16:6829/3015700546","heartbeat_back_addr":"172.21.15.16:6833/3015700546","heartbeat_front_addr":"172.21.15.16:6831/3015700546","state":["exists","up"]},{"osd":10,"uuid":"f2548997-72ee-400e-ab8d-5966d85b2cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6825","nonce":2427122475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6827","nonce":2427122475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6831","nonce":2427122475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2427122475},{"type":"v1","addr":"172.21.15.155:6829","nonce":2427122475}]},"public_addr":"172.21.15.155:6825/2427122475","cluster_addr":"172.21.15.155:6827/2427122475","heartbeat_back_addr":"172.21.15.155:6831/2427122475","heartbeat_front_addr":"172.21.15.155:6829/2427122475","state":["exists","up"]},{"osd":11,"uuid":"8297da1d-d3e2-4a4d-9396-426dfafa3e95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6825","nonce":399265701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6827","nonce":399265701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6831","nonce":399265701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":399265701},{"type":"v1","addr":"172.21.15.184:6829","nonce":399265701}]},"public_addr":"172.21.15.184:6825/399265701","cluster_addr":"172.21.15.184:6827/399265701","heartbeat_back_addr":"172.21.15.184:6831/399265701","heartbeat_front_addr":"172.21.15.184:6829/399265701","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:58:30.842360+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:58:57.164488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:59:20.778103+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T05:59:50.123991+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:00:16.789933+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:00:41.788905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:01:12.281376+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:01:39.211858+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:02:04.958771+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:02:35.645947+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:03:01.940433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-31T06:03:26.728673+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3923220960":"2024-09-01T05:57:23.153214+0000","172.21.15.16:0/3013533836":"2024-09-01T05:57:23.153214+0000","172.21.15.16:6800/3755683228":"2024-09-01T05:55:45.075902+0000","172.21.15.16:0/3064435470":"2024-09-01T05:55:45.075902+0000","172.21.15.16:0/911709962":"2024-09-01T05:55:45.075902+0000","172.21.15.16:0/2937368967":"2024-09-01T05:55:45.075902+0000","172.21.15.16:6800/1378972230":"2024-09-01T05:56:14.195704+0000","172.21.15.16:0/3047316995":"2024-09-01T05:56:14.195704+0000","172.21.15.16:6801/3755683228":"2024-09-01T05:55:45.075902+0000","172.21.15.16:6801/2571082634":"2024-09-01T05:57:23.153214+0000","172.21.15.16:0/2136704295":"2024-09-01T05:56:14.195704+0000","172.21.15.16:0/2259332778":"2024-09-01T05:56:14.195704+0000","172.21.15.16:6800/2571082634":"2024-09-01T05:57:23.153214+0000","172.21.15.16:6801/1378972230":"2024-09-01T05:56:14.195704+0000","172.21.15.16:0/2126457673":"2024-09-01T05:57:23.153214+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":16,"length":1},{"begin":29,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-31T06:23:57.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:57.164+0000 7f3eb9c00640 1 -- 172.21.15.16:0/150033482 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3e980b4d20 msgr2=0x7f3e980b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:57.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:57.164+0000 7f3eb9c00640 1 --2- 172.21.15.16:0/150033482 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3e980b4d20 0x7f3e980b7230 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c009230 tx=0x7f3e9c031400 comp rx=0 tx=0).stop 2024-08-31T06:23:57.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:57.164+0000 7f3eb9c00640 1 -- 172.21.15.16:0/150033482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3eb4103750 msgr2=0x7f3eb419c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:57.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:57.164+0000 7f3eb9c00640 1 --2- 172.21.15.16:0/150033482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3eb4103750 0x7f3eb419c2c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f3ea400e9f0 tx=0x7f3ea400eeb0 comp rx=0 tx=0).stop 2024-08-31T06:23:57.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:57.164+0000 7f3eb9c00640 1 -- 172.21.15.16:0/150033482 shutdown_connections 2024-08-31T06:23:57.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:57.164+0000 7f3eb9c00640 1 --2- 172.21.15.16:0/150033482 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f3e980b4d20 0x7f3e980b7230 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:57.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:57.164+0000 7f3eb9c00640 1 --2- 172.21.15.16:0/150033482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3eb41099e0 0x7f3eb41a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:57.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:57.164+0000 7f3eb9c00640 1 --2- 172.21.15.16:0/150033482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3eb4109090 0x7f3eb419c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:57.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:57.164+0000 7f3eb9c00640 1 --2- 172.21.15.16:0/150033482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3eb4103750 0x7f3eb419c2c0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:57.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:57.164+0000 7f3eb9c00640 1 -- 172.21.15.16:0/150033482 >> 172.21.15.16:0/150033482 conn(0x7f3eb40ff1a0 msgr2=0x7f3eb4107780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:57.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:57.164+0000 7f3eb9c00640 1 -- 172.21.15.16:0/150033482 shutdown_connections 2024-08-31T06:23:57.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:57.164+0000 7f3eb9c00640 1 -- 172.21.15.16:0/150033482 wait complete. 2024-08-31T06:23:57.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:56 smithi184 ceph-mon[22331]: pgmap v887: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-08-31T06:23:57.884 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs fail cephfs 2024-08-31T06:23:58.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:57 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/150033482' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T06:23:58.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:57 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/150033482' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T06:23:58.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:57 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/150033482' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-31T06:23:58.264 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:23:58.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:58.908+0000 7fed23400640 1 -- 172.21.15.16:0/2367683157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed1c075550 msgr2=0x7fed1c075960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:58.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:58.908+0000 7fed23400640 1 --2- 172.21.15.16:0/2367683157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed1c075550 0x7fed1c075960 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fed10009900 tx=0x7fed1002a410 comp rx=0 tx=0).stop 2024-08-31T06:23:58.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:58.909+0000 7fed23400640 1 -- 172.21.15.16:0/2367683157 shutdown_connections 2024-08-31T06:23:58.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:58.909+0000 7fed23400640 1 --2- 172.21.15.16:0/2367683157 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed1c075ea0 0x7fed1c10f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:58.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:58.909+0000 7fed23400640 1 --2- 172.21.15.16:0/2367683157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed1c10fe80 0x7fed1c1122c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:58.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:58.909+0000 7fed23400640 1 --2- 172.21.15.16:0/2367683157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed1c075550 0x7fed1c075960 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:58.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:58.909+0000 7fed23400640 1 -- 172.21.15.16:0/2367683157 >> 172.21.15.16:0/2367683157 conn(0x7fed1c0ff370 msgr2=0x7fed1c1017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:58.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:58.909+0000 7fed23400640 1 -- 172.21.15.16:0/2367683157 shutdown_connections 2024-08-31T06:23:58.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:58.909+0000 7fed23400640 1 -- 172.21.15.16:0/2367683157 wait complete. 2024-08-31T06:23:58.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:58.910+0000 7fed23400640 1 Processor -- start 2024-08-31T06:23:58.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:58.910+0000 7fed23400640 1 -- start start 2024-08-31T06:23:58.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:58.911+0000 7fed23400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed1c075550 0x7fed1c19c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:58.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:58.911+0000 7fed23400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed1c075ea0 0x7fed1c19c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:58.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:58.911+0000 7fed23400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed1c10fe80 0x7fed1c1a1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:58.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:58.911+0000 7fed23400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fed1c115a80 con 0x7fed1c10fe80 2024-08-31T06:23:58.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:58.911+0000 7fed23400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fed1c115900 con 0x7fed1c075ea0 2024-08-31T06:23:58.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:58.911+0000 7fed23400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fed1c115c00 con 0x7fed1c075550 2024-08-31T06:23:58.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:58.911+0000 7fed21400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed1c10fe80 0x7fed1c1a1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:58.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:58.911+0000 7fed20a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed1c075550 0x7fed1c19c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:58.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:58.911+0000 7fed21400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed1c10fe80 0x7fed1c1a1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59414/0 (socket says 172.21.15.16:59414) 2024-08-31T06:23:58.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:58.911+0000 7fed21400640 1 -- 172.21.15.16:0/358858506 learned_addr learned my addr 172.21.15.16:0/358858506 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:23:58.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:58.911+0000 7fed1be00640 1 --2- 172.21.15.16:0/358858506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed1c075ea0 0x7fed1c19c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:58.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:58.911+0000 7fed20a00640 1 -- 172.21.15.16:0/358858506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed1c075ea0 msgr2=0x7fed1c19c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:58.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:58.911+0000 7fed20a00640 1 --2- 172.21.15.16:0/358858506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed1c075ea0 0x7fed1c19c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:58.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:58.911+0000 7fed20a00640 1 -- 172.21.15.16:0/358858506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed1c10fe80 msgr2=0x7fed1c1a1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:58.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:58.911+0000 7fed20a00640 1 --2- 172.21.15.16:0/358858506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed1c10fe80 0x7fed1c1a1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:58.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:58.911+0000 7fed20a00640 1 -- 172.21.15.16:0/358858506 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed1c1a2320 con 0x7fed1c075550 2024-08-31T06:23:58.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:58.912+0000 7fed21400640 1 --2- 172.21.15.16:0/358858506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed1c10fe80 0x7fed1c1a1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:23:58.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:58.912+0000 7fed1be00640 1 --2- 172.21.15.16:0/358858506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed1c075ea0 0x7fed1c19c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:23:58.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:58.912+0000 7fed20a00640 1 --2- 172.21.15.16:0/358858506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed1c075550 0x7fed1c19c3b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fed1002aa00 tx=0x7fed10003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:58.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:58.912+0000 7fed19600640 1 -- 172.21.15.16:0/358858506 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1003f070 con 0x7fed1c075550 2024-08-31T06:23:58.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:58.912+0000 7fed23400640 1 -- 172.21.15.16:0/358858506 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed1c1a25b0 con 0x7fed1c075550 2024-08-31T06:23:58.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:58.912+0000 7fed19600640 1 -- 172.21.15.16:0/358858506 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed10031b50 con 0x7fed1c075550 2024-08-31T06:23:58.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:58.912+0000 7fed19600640 1 -- 172.21.15.16:0/358858506 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed10035030 con 0x7fed1c075550 2024-08-31T06:23:58.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:58.912+0000 7fed23400640 1 -- 172.21.15.16:0/358858506 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed1c1a8d70 con 0x7fed1c075550 2024-08-31T06:23:58.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:58.913+0000 7fed23400640 1 -- 172.21.15.16:0/358858506 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed1c076a60 con 0x7fed1c075550 2024-08-31T06:23:58.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:58.916+0000 7fed19600640 1 -- 172.21.15.16:0/358858506 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7fed10042070 con 0x7fed1c075550 2024-08-31T06:23:58.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:58.917+0000 7fed19600640 1 --2- 172.21.15.16:0/358858506 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fed040b4ca0 0x7fed040b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:23:58.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:58.917+0000 7fed19600640 1 -- 172.21.15.16:0/358858506 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8470+0+0 (secure 0 0 0) 0x7fed10105c50 con 0x7fed1c075550 2024-08-31T06:23:58.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:58.918+0000 7fed1be00640 1 --2- 172.21.15.16:0/358858506 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fed040b4ca0 0x7fed040b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:23:58.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:58.918+0000 7fed1be00640 1 --2- 172.21.15.16:0/358858506 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fed040b4ca0 0x7fed040b71b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fed1c1002d0 tx=0x7fed08005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:23:58.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:58.920+0000 7fed19600640 1 -- 172.21.15.16:0/358858506 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7fed100cc420 con 0x7fed1c075550 2024-08-31T06:23:59.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:58 smithi155 ceph-mon[21742]: pgmap v888: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-08-31T06:23:59.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:59.153+0000 7fed23400640 1 -- 172.21.15.16:0/358858506 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fed1c10a770 con 0x7fed1c075550 2024-08-31T06:23:59.156 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:58 smithi016 ceph-mon[17194]: pgmap v888: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-08-31T06:23:59.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:59.165+0000 7fed19600640 1 -- 172.21.15.16:0/358858506 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v56) ==== 161+0+0 (secure 0 0 0) 0x7fed100d0be0 con 0x7fed1c075550 2024-08-31T06:23:59.168 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-08-31T06:23:59.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:59.168+0000 7fed23400640 1 -- 172.21.15.16:0/358858506 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fed040b4ca0 msgr2=0x7fed040b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:59.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:59.168+0000 7fed23400640 1 --2- 172.21.15.16:0/358858506 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fed040b4ca0 0x7fed040b71b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fed1c1002d0 tx=0x7fed08005f90 comp rx=0 tx=0).stop 2024-08-31T06:23:59.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:59.168+0000 7fed23400640 1 -- 172.21.15.16:0/358858506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed1c075550 msgr2=0x7fed1c19c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:23:59.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:59.168+0000 7fed23400640 1 --2- 172.21.15.16:0/358858506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed1c075550 0x7fed1c19c3b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fed1002aa00 tx=0x7fed10003b90 comp rx=0 tx=0).stop 2024-08-31T06:23:59.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:59.168+0000 7fed23400640 1 -- 172.21.15.16:0/358858506 shutdown_connections 2024-08-31T06:23:59.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:59.168+0000 7fed23400640 1 --2- 172.21.15.16:0/358858506 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7fed040b4ca0 0x7fed040b71b0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:59.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:59.168+0000 7fed23400640 1 --2- 172.21.15.16:0/358858506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed1c10fe80 0x7fed1c1a1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:59.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:59.168+0000 7fed23400640 1 --2- 172.21.15.16:0/358858506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed1c075ea0 0x7fed1c19c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:59.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:59.168+0000 7fed23400640 1 --2- 172.21.15.16:0/358858506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed1c075550 0x7fed1c19c3b0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:23:59.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:59.168+0000 7fed23400640 1 -- 172.21.15.16:0/358858506 >> 172.21.15.16:0/358858506 conn(0x7fed1c0ff370 msgr2=0x7fed1c1078e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:23:59.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:59.168+0000 7fed23400640 1 -- 172.21.15.16:0/358858506 shutdown_connections 2024-08-31T06:23:59.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:23:59.168+0000 7fed23400640 1 -- 172.21.15.16:0/358858506 wait complete. 2024-08-31T06:23:59.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:58 smithi184 ceph-mon[22331]: pgmap v888: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-08-31T06:23:59.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:59 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mon-a[17190]: 2024-08-31T06:23:59.155+0000 7f996fe00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-31T06:23:59.413 INFO:journalctl@ceph.mds.g.smithi016.stdout:Aug 31 06:23:59 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-g[62058]: starting mds.g at 2024-08-31T06:23:59.622 INFO:journalctl@ceph.mds.h.smithi155.stdout:Aug 31 06:23:59 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-h[53037]: ignoring --setuser ceph since I am not root 2024-08-31T06:23:59.622 INFO:journalctl@ceph.mds.h.smithi155.stdout:Aug 31 06:23:59 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-h[53037]: ignoring --setgroup ceph since I am not root 2024-08-31T06:23:59.622 INFO:journalctl@ceph.mds.h.smithi155.stdout:Aug 31 06:23:59 smithi155 ceph-mds[53041]: -- 172.21.15.155:0/486442018 <== mon.2 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c04adab860 con 0x55c04ef51a80 2024-08-31T06:23:59.622 INFO:journalctl@ceph.mds.h.smithi155.stdout:Aug 31 06:23:59 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-h[53037]: starting mds.h at 2024-08-31T06:23:59.623 INFO:journalctl@ceph.mds.b.smithi155.stdout:Aug 31 06:23:59 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-b[53353]: ignoring --setuser ceph since I am not root 2024-08-31T06:23:59.623 INFO:journalctl@ceph.mds.b.smithi155.stdout:Aug 31 06:23:59 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-b[53353]: ignoring --setgroup ceph since I am not root 2024-08-31T06:23:59.623 INFO:journalctl@ceph.mds.b.smithi155.stdout:Aug 31 06:23:59 smithi155 ceph-mds[53357]: -- 172.21.15.155:0/3659761884 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55944d66d860 con 0x559451812d00 2024-08-31T06:23:59.623 INFO:journalctl@ceph.mds.b.smithi155.stdout:Aug 31 06:23:59 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-b[53353]: starting mds.b at 2024-08-31T06:23:59.723 INFO:journalctl@ceph.mds.c.smithi184.stdout:Aug 31 06:23:59 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-c[54092]: ignoring --setuser ceph since I am not root 2024-08-31T06:23:59.723 INFO:journalctl@ceph.mds.c.smithi184.stdout:Aug 31 06:23:59 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-c[54092]: ignoring --setgroup ceph since I am not root 2024-08-31T06:23:59.723 INFO:journalctl@ceph.mds.c.smithi184.stdout:Aug 31 06:23:59 smithi184 ceph-mds[54096]: --2- 172.21.15.184:0/1603248340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638d11c1a80 0x5638d11b2b00 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x5638d11d2ba0 tx=0x5638d10fa780 comp rx=0 tx=0).write_message sending message m=0x5638d1088540 seq=3 auth(proto 2 2 bytes epoch 0) 2024-08-31T06:23:59.723 INFO:journalctl@ceph.mds.c.smithi184.stdout:Aug 31 06:23:59 smithi184 ceph-mds[54096]: -- 172.21.15.184:0/1603248340 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5638cd025520 con 0x5638d11c1a80 2024-08-31T06:23:59.723 INFO:journalctl@ceph.mds.c.smithi184.stdout:Aug 31 06:23:59 smithi184 ceph-mds[54096]: -- 172.21.15.184:0/1603248340 <== mon.2 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5638cd01bc20 con 0x5638d11c1a80 2024-08-31T06:23:59.723 INFO:journalctl@ceph.mds.c.smithi184.stdout:Aug 31 06:23:59 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-c[54092]: starting mds.c at 2024-08-31T06:23:59.724 INFO:journalctl@ceph.mds.f.smithi184.stdout:Aug 31 06:23:59 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-f[53791]: ignoring --setuser ceph since I am not root 2024-08-31T06:23:59.724 INFO:journalctl@ceph.mds.f.smithi184.stdout:Aug 31 06:23:59 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-f[53791]: ignoring --setgroup ceph since I am not root 2024-08-31T06:23:59.724 INFO:journalctl@ceph.mds.f.smithi184.stdout:Aug 31 06:23:59 smithi184 ceph-mds[53795]: -- 172.21.15.184:0/7611711 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5610d7669860 con 0x5610db80f600 2024-08-31T06:23:59.724 INFO:journalctl@ceph.mds.f.smithi184.stdout:Aug 31 06:23:59 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-f[53791]: starting mds.f at 2024-08-31T06:23:59.724 INFO:journalctl@ceph.mds.i.smithi184.stdout:Aug 31 06:23:59 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-i[54398]: ignoring --setuser ceph since I am not root 2024-08-31T06:23:59.724 INFO:journalctl@ceph.mds.i.smithi184.stdout:Aug 31 06:23:59 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-i[54398]: ignoring --setgroup ceph since I am not root 2024-08-31T06:23:59.724 INFO:journalctl@ceph.mds.i.smithi184.stdout:Aug 31 06:23:59 smithi184 ceph-mds[54407]: -- 172.21.15.184:0/2816266604 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5650a6a83860 con 0x5650aac29180 2024-08-31T06:23:59.724 INFO:journalctl@ceph.mds.i.smithi184.stdout:Aug 31 06:23:59 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-i[54398]: starting mds.i at 2024-08-31T06:24:00.077 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-08-31T06:24:00.126 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:59 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/358858506' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-08-31T06:24:00.126 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:59 smithi016 ceph-mon[17194]: from='client.? ' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-08-31T06:24:00.126 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:59 smithi016 ceph-mon[17194]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-08-31T06:24:00.127 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:59 smithi016 ceph-mon[17194]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-31T06:24:00.127 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:59 smithi016 ceph-mon[17194]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-08-31T06:24:00.127 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:59 smithi016 ceph-mon[17194]: osdmap e104: 12 total, 12 up, 12 in 2024-08-31T06:24:00.127 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:23:59 smithi016 ceph-mon[17194]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-08-31T06:24:00.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:59 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/358858506' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-08-31T06:24:00.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:59 smithi184 ceph-mon[22331]: from='client.? ' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-08-31T06:24:00.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:59 smithi184 ceph-mon[22331]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-08-31T06:24:00.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:59 smithi184 ceph-mon[22331]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-31T06:24:00.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:59 smithi184 ceph-mon[22331]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-08-31T06:24:00.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:59 smithi184 ceph-mon[22331]: osdmap e104: 12 total, 12 up, 12 in 2024-08-31T06:24:00.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:23:59 smithi184 ceph-mon[22331]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-08-31T06:24:00.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:59 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/358858506' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-08-31T06:24:00.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:59 smithi155 ceph-mon[21742]: from='client.? ' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-08-31T06:24:00.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:59 smithi155 ceph-mon[21742]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-08-31T06:24:00.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:59 smithi155 ceph-mon[21742]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-31T06:24:00.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:59 smithi155 ceph-mon[21742]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-08-31T06:24:00.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:59 smithi155 ceph-mon[21742]: osdmap e104: 12 total, 12 up, 12 in 2024-08-31T06:24:00.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:23:59 smithi155 ceph-mon[21742]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-08-31T06:24:00.465 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:24:01.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.138+0000 7f127b200640 1 -- 172.21.15.16:0/2845471418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1274109260 msgr2=0x7f1274109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:24:01.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.138+0000 7f127b200640 1 --2- 172.21.15.16:0/2845471418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1274109260 0x7f1274109670 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f125c009900 tx=0x7f125c02a3f0 comp rx=0 tx=0).stop 2024-08-31T06:24:01.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.138+0000 7f127b200640 1 -- 172.21.15.16:0/2845471418 shutdown_connections 2024-08-31T06:24:01.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.138+0000 7f127b200640 1 --2- 172.21.15.16:0/2845471418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1274109bb0 0x7f12741103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:24:01.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.138+0000 7f127b200640 1 --2- 172.21.15.16:0/2845471418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1274109260 0x7f1274109670 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:24:01.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.138+0000 7f127b200640 1 --2- 172.21.15.16:0/2845471418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1274103920 0x7f1274103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:24:01.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.138+0000 7f127b200640 1 -- 172.21.15.16:0/2845471418 >> 172.21.15.16:0/2845471418 conn(0x7f12740ff370 msgr2=0x7f12741017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:24:01.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.138+0000 7f127b200640 1 -- 172.21.15.16:0/2845471418 shutdown_connections 2024-08-31T06:24:01.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.138+0000 7f127b200640 1 -- 172.21.15.16:0/2845471418 wait complete. 2024-08-31T06:24:01.142 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:24:00 smithi016 ceph-mon[17194]: pgmap v890: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.3 KiB/s wr, 9 op/s 2024-08-31T06:24:01.142 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:24:00 smithi016 ceph-mon[17194]: mds.? [v2:172.21.15.155:6836/2639002954,v1:172.21.15.155:6838/2639002954] up:boot 2024-08-31T06:24:01.142 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:24:00 smithi016 ceph-mon[17194]: mds.? [v2:172.21.15.155:6837/12264655,v1:172.21.15.155:6839/12264655] up:boot 2024-08-31T06:24:01.142 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:24:00 smithi016 ceph-mon[17194]: mds.? [v2:172.21.15.184:6835/3408067637,v1:172.21.15.184:6838/3408067637] up:boot 2024-08-31T06:24:01.142 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:24:00 smithi016 ceph-mon[17194]: mds.? [v2:172.21.15.184:6836/2420228222,v1:172.21.15.184:6839/2420228222] up:boot 2024-08-31T06:24:01.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:24:00 smithi016 ceph-mon[17194]: mds.? [v2:172.21.15.184:6834/839394481,v1:172.21.15.184:6837/839394481] up:boot 2024-08-31T06:24:01.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:24:00 smithi016 ceph-mon[17194]: mds.? [v2:172.21.15.16:6838/481923897,v1:172.21.15.16:6839/481923897] up:boot 2024-08-31T06:24:01.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:24:00 smithi016 ceph-mon[17194]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-08-31T06:24:01.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:24:00 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-08-31T06:24:01.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:24:00 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-08-31T06:24:01.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:24:00 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-08-31T06:24:01.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:24:00 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-08-31T06:24:01.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:24:00 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-08-31T06:24:01.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:24:00 smithi016 ceph-mon[17194]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-08-31T06:24:01.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.139+0000 7f127b200640 1 Processor -- start 2024-08-31T06:24:01.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.139+0000 7f127b200640 1 -- start start 2024-08-31T06:24:01.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.140+0000 7f127b200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1274103920 0x7f12740754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:24:01.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.140+0000 7f127b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1274109260 0x7f12740759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:24:01.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.140+0000 7f127b200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1274109bb0 0x7f1274075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:24:01.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.140+0000 7f127b200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1274113870 con 0x7f1274109260 2024-08-31T06:24:01.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.140+0000 7f127b200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f12741136f0 con 0x7f1274109bb0 2024-08-31T06:24:01.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.140+0000 7f127b200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f12741139f0 con 0x7f1274103920 2024-08-31T06:24:01.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.140+0000 7f1273e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1274103920 0x7f12740754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:24:01.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.140+0000 7f1273e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1274103920 0x7f12740754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:35042/0 (socket says 172.21.15.16:35042) 2024-08-31T06:24:01.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.140+0000 7f1273e00640 1 -- 172.21.15.16:0/2946516909 learned_addr learned my addr 172.21.15.16:0/2946516909 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:24:01.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.140+0000 7f1279200640 1 --2- 172.21.15.16:0/2946516909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1274109bb0 0x7f1274075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:24:01.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.140+0000 7f126be00640 1 --2- 172.21.15.16:0/2946516909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1274109260 0x7f12740759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:24:01.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.140+0000 7f1273e00640 1 -- 172.21.15.16:0/2946516909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1274109bb0 msgr2=0x7f1274075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:24:01.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.140+0000 7f1273e00640 1 --2- 172.21.15.16:0/2946516909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1274109bb0 0x7f1274075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:24:01.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.140+0000 7f1273e00640 1 -- 172.21.15.16:0/2946516909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1274109260 msgr2=0x7f12740759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:24:01.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.140+0000 7f1273e00640 1 --2- 172.21.15.16:0/2946516909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1274109260 0x7f12740759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:24:01.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.140+0000 7f1273e00640 1 -- 172.21.15.16:0/2946516909 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12741a84e0 con 0x7f1274103920 2024-08-31T06:24:01.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.141+0000 7f1273e00640 1 --2- 172.21.15.16:0/2946516909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1274103920 0x7f12740754b0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f126400ea60 tx=0x7f126400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:24:01.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.141+0000 7f1271600640 1 -- 172.21.15.16:0/2946516909 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f126400cde0 con 0x7f1274103920 2024-08-31T06:24:01.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.141+0000 7f1271600640 1 -- 172.21.15.16:0/2946516909 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12640035f0 con 0x7f1274103920 2024-08-31T06:24:01.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.141+0000 7f127b200640 1 -- 172.21.15.16:0/2946516909 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12741a87d0 con 0x7f1274103920 2024-08-31T06:24:01.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.141+0000 7f1271600640 1 -- 172.21.15.16:0/2946516909 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1264010690 con 0x7f1274103920 2024-08-31T06:24:01.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.141+0000 7f127b200640 1 -- 172.21.15.16:0/2946516909 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12741a8ce0 con 0x7f1274103920 2024-08-31T06:24:01.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.142+0000 7f127b200640 1 -- 172.21.15.16:0/2946516909 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1274103d30 con 0x7f1274103920 2024-08-31T06:24:01.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.146+0000 7f1271600640 1 -- 172.21.15.16:0/2946516909 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f12640037a0 con 0x7f1274103920 2024-08-31T06:24:01.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.147+0000 7f1271600640 1 --2- 172.21.15.16:0/2946516909 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f12580b4ca0 0x7f12580b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:24:01.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.147+0000 7f1271600640 1 -- 172.21.15.16:0/2946516909 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8684+0+0 (secure 0 0 0) 0x7f12640e5980 con 0x7f1274103920 2024-08-31T06:24:01.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.147+0000 7f126be00640 1 --2- 172.21.15.16:0/2946516909 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f12580b4ca0 0x7f12580b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:24:01.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.148+0000 7f126be00640 1 --2- 172.21.15.16:0/2946516909 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f12580b4ca0 0x7f12580b71b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f125c009230 tx=0x7f125c031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:24:01.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.151+0000 7f1271600640 1 -- 172.21.15.16:0/2946516909 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f12640ac670 con 0x7f1274103920 2024-08-31T06:24:01.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:24:00 smithi184 ceph-mon[22331]: pgmap v890: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.3 KiB/s wr, 9 op/s 2024-08-31T06:24:01.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:24:00 smithi184 ceph-mon[22331]: mds.? [v2:172.21.15.155:6836/2639002954,v1:172.21.15.155:6838/2639002954] up:boot 2024-08-31T06:24:01.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:24:00 smithi184 ceph-mon[22331]: mds.? [v2:172.21.15.155:6837/12264655,v1:172.21.15.155:6839/12264655] up:boot 2024-08-31T06:24:01.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:24:00 smithi184 ceph-mon[22331]: mds.? [v2:172.21.15.184:6835/3408067637,v1:172.21.15.184:6838/3408067637] up:boot 2024-08-31T06:24:01.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:24:00 smithi184 ceph-mon[22331]: mds.? [v2:172.21.15.184:6836/2420228222,v1:172.21.15.184:6839/2420228222] up:boot 2024-08-31T06:24:01.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:24:00 smithi184 ceph-mon[22331]: mds.? [v2:172.21.15.184:6834/839394481,v1:172.21.15.184:6837/839394481] up:boot 2024-08-31T06:24:01.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:24:00 smithi184 ceph-mon[22331]: mds.? [v2:172.21.15.16:6838/481923897,v1:172.21.15.16:6839/481923897] up:boot 2024-08-31T06:24:01.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:24:00 smithi184 ceph-mon[22331]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-08-31T06:24:01.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:24:00 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-08-31T06:24:01.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:24:00 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-08-31T06:24:01.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:24:00 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-08-31T06:24:01.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:24:00 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-08-31T06:24:01.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:24:00 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-08-31T06:24:01.224 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:24:00 smithi184 ceph-mon[22331]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-08-31T06:24:01.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:24:00 smithi155 ceph-mon[21742]: pgmap v890: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.3 KiB/s wr, 9 op/s 2024-08-31T06:24:01.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:24:00 smithi155 ceph-mon[21742]: mds.? [v2:172.21.15.155:6836/2639002954,v1:172.21.15.155:6838/2639002954] up:boot 2024-08-31T06:24:01.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:24:00 smithi155 ceph-mon[21742]: mds.? [v2:172.21.15.155:6837/12264655,v1:172.21.15.155:6839/12264655] up:boot 2024-08-31T06:24:01.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:24:00 smithi155 ceph-mon[21742]: mds.? [v2:172.21.15.184:6835/3408067637,v1:172.21.15.184:6838/3408067637] up:boot 2024-08-31T06:24:01.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:24:00 smithi155 ceph-mon[21742]: mds.? [v2:172.21.15.184:6836/2420228222,v1:172.21.15.184:6839/2420228222] up:boot 2024-08-31T06:24:01.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:24:00 smithi155 ceph-mon[21742]: mds.? [v2:172.21.15.184:6834/839394481,v1:172.21.15.184:6837/839394481] up:boot 2024-08-31T06:24:01.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:24:00 smithi155 ceph-mon[21742]: mds.? [v2:172.21.15.16:6838/481923897,v1:172.21.15.16:6839/481923897] up:boot 2024-08-31T06:24:01.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:24:00 smithi155 ceph-mon[21742]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-08-31T06:24:01.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:24:00 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-08-31T06:24:01.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:24:00 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-08-31T06:24:01.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:24:00 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-08-31T06:24:01.374 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:24:00 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-08-31T06:24:01.374 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:24:00 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-08-31T06:24:01.374 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:24:00 smithi155 ceph-mon[21742]: from='mgr.14220 172.21.15.16:0/4229147704' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-08-31T06:24:01.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.384+0000 7f127b200640 1 -- 172.21.15.16:0/2946516909 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f127402e430 con 0x7f1274103920 2024-08-31T06:24:01.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.881+0000 7f1271600640 1 -- 172.21.15.16:0/2946516909 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v58) ==== 104+0+0 (secure 0 0 0) 0x7f12640abf70 con 0x7f1274103920 2024-08-31T06:24:01.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.884+0000 7f127b200640 1 -- 172.21.15.16:0/2946516909 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f12580b4ca0 msgr2=0x7f12580b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:24:01.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.884+0000 7f127b200640 1 --2- 172.21.15.16:0/2946516909 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f12580b4ca0 0x7f12580b71b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f125c009230 tx=0x7f125c031400 comp rx=0 tx=0).stop 2024-08-31T06:24:01.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.884+0000 7f127b200640 1 -- 172.21.15.16:0/2946516909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1274103920 msgr2=0x7f12740754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:24:01.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.884+0000 7f127b200640 1 --2- 172.21.15.16:0/2946516909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1274103920 0x7f12740754b0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f126400ea60 tx=0x7f126400ef20 comp rx=0 tx=0).stop 2024-08-31T06:24:01.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.884+0000 7f127b200640 1 -- 172.21.15.16:0/2946516909 shutdown_connections 2024-08-31T06:24:01.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.884+0000 7f127b200640 1 --2- 172.21.15.16:0/2946516909 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f12580b4ca0 0x7f12580b71b0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:24:01.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.884+0000 7f127b200640 1 --2- 172.21.15.16:0/2946516909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1274109bb0 0x7f1274075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:24:01.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.884+0000 7f127b200640 1 --2- 172.21.15.16:0/2946516909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1274109260 0x7f12740759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:24:01.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.884+0000 7f127b200640 1 --2- 172.21.15.16:0/2946516909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1274103920 0x7f12740754b0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:24:01.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.884+0000 7f127b200640 1 -- 172.21.15.16:0/2946516909 >> 172.21.15.16:0/2946516909 conn(0x7f12740ff370 msgr2=0x7f1274101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:24:01.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.884+0000 7f127b200640 1 -- 172.21.15.16:0/2946516909 shutdown_connections 2024-08-31T06:24:01.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:01.885+0000 7f127b200640 1 -- 172.21.15.16:0/2946516909 wait complete. 2024-08-31T06:24:02.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:24:01 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/2946516909' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-08-31T06:24:02.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:24:01 smithi016 ceph-mon[17194]: from='client.? ' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-08-31T06:24:02.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:24:01 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/2946516909' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-08-31T06:24:02.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:24:01 smithi184 ceph-mon[22331]: from='client.? ' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-08-31T06:24:02.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:24:01 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/2946516909' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-08-31T06:24:02.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:24:01 smithi155 ceph-mon[21742]: from='client.? ' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-08-31T06:24:02.650 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-08-31T06:24:03.029 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:24:03.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:24:02 smithi016 ceph-mon[17194]: pgmap v891: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-08-31T06:24:03.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:24:02 smithi016 ceph-mon[17194]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-08-31T06:24:03.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:24:02 smithi016 ceph-mon[17194]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-08-31T06:24:03.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:24:02 smithi016 ceph-mon[17194]: Cluster is now healthy 2024-08-31T06:24:03.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:24:02 smithi016 ceph-mon[17194]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-08-31T06:24:03.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:24:02 smithi016 ceph-mon[17194]: fsmap 12 up:standby 2024-08-31T06:24:03.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:24:02 smithi184 ceph-mon[22331]: pgmap v891: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-08-31T06:24:03.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:24:02 smithi184 ceph-mon[22331]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-08-31T06:24:03.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:24:02 smithi184 ceph-mon[22331]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-08-31T06:24:03.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:24:02 smithi184 ceph-mon[22331]: Cluster is now healthy 2024-08-31T06:24:03.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:24:02 smithi184 ceph-mon[22331]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-08-31T06:24:03.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:24:02 smithi184 ceph-mon[22331]: fsmap 12 up:standby 2024-08-31T06:24:03.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:24:02 smithi155 ceph-mon[21742]: pgmap v891: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-08-31T06:24:03.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:24:02 smithi155 ceph-mon[21742]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-08-31T06:24:03.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:24:02 smithi155 ceph-mon[21742]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-08-31T06:24:03.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:24:02 smithi155 ceph-mon[21742]: Cluster is now healthy 2024-08-31T06:24:03.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:24:02 smithi155 ceph-mon[21742]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-08-31T06:24:03.373 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:24:02 smithi155 ceph-mon[21742]: fsmap 12 up:standby 2024-08-31T06:24:03.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.664+0000 7f469b200640 1 -- 172.21.15.16:0/151838999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4694109240 msgr2=0x7f4694109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:24:03.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.664+0000 7f469b200640 1 --2- 172.21.15.16:0/151838999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4694109240 0x7f4694109650 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f4680009900 tx=0x7f468002a3f0 comp rx=0 tx=0).stop 2024-08-31T06:24:03.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.665+0000 7f469b200640 1 -- 172.21.15.16:0/151838999 shutdown_connections 2024-08-31T06:24:03.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.665+0000 7f469b200640 1 --2- 172.21.15.16:0/151838999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4694109b90 0x7f46941103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:24:03.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.665+0000 7f469b200640 1 --2- 172.21.15.16:0/151838999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4694109240 0x7f4694109650 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:24:03.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.665+0000 7f469b200640 1 --2- 172.21.15.16:0/151838999 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4694103900 0x7f4694103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:24:03.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.665+0000 7f469b200640 1 -- 172.21.15.16:0/151838999 >> 172.21.15.16:0/151838999 conn(0x7f469407b5f0 msgr2=0x7f469407ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:24:03.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.665+0000 7f469b200640 1 -- 172.21.15.16:0/151838999 shutdown_connections 2024-08-31T06:24:03.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.665+0000 7f469b200640 1 -- 172.21.15.16:0/151838999 wait complete. 2024-08-31T06:24:03.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.666+0000 7f469b200640 1 Processor -- start 2024-08-31T06:24:03.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.666+0000 7f469b200640 1 -- start start 2024-08-31T06:24:03.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.666+0000 7f469b200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4694103900 0x7f469419c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:24:03.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.666+0000 7f469b200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4694109240 0x7f469419c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:24:03.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.666+0000 7f469b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4694109b90 0x7f46941a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:24:03.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.666+0000 7f469b200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4694113850 con 0x7f4694109b90 2024-08-31T06:24:03.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.666+0000 7f469b200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f46941136d0 con 0x7f4694109240 2024-08-31T06:24:03.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.666+0000 7f469b200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f46941139d0 con 0x7f4694103900 2024-08-31T06:24:03.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.667+0000 7f4693e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4694103900 0x7f469419c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:24:03.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.667+0000 7f4693e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4694103900 0x7f469419c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:35058/0 (socket says 172.21.15.16:35058) 2024-08-31T06:24:03.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.667+0000 7f4693e00640 1 -- 172.21.15.16:0/2063127125 learned_addr learned my addr 172.21.15.16:0/2063127125 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:24:03.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.667+0000 7f4693400640 1 --2- 172.21.15.16:0/2063127125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4694109240 0x7f469419c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:24:03.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.667+0000 7f4699200640 1 --2- 172.21.15.16:0/2063127125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4694109b90 0x7f46941a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:24:03.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.667+0000 7f4693e00640 1 -- 172.21.15.16:0/2063127125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4694109240 msgr2=0x7f469419c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:24:03.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.667+0000 7f4693e00640 1 --2- 172.21.15.16:0/2063127125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4694109240 0x7f469419c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:24:03.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.667+0000 7f4693e00640 1 -- 172.21.15.16:0/2063127125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4694109b90 msgr2=0x7f46941a1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:24:03.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.667+0000 7f4693e00640 1 --2- 172.21.15.16:0/2063127125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4694109b90 0x7f46941a1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:24:03.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.667+0000 7f4693e00640 1 -- 172.21.15.16:0/2063127125 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46941a23b0 con 0x7f4694103900 2024-08-31T06:24:03.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.668+0000 7f4693e00640 1 --2- 172.21.15.16:0/2063127125 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4694103900 0x7f469419c440 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f468400e9f0 tx=0x7f468400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:24:03.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.668+0000 7f4690c00640 1 -- 172.21.15.16:0/2063127125 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468400cd80 con 0x7f4694103900 2024-08-31T06:24:03.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.668+0000 7f4690c00640 1 -- 172.21.15.16:0/2063127125 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46840108e0 con 0x7f4694103900 2024-08-31T06:24:03.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.668+0000 7f469b200640 1 -- 172.21.15.16:0/2063127125 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46941a26a0 con 0x7f4694103900 2024-08-31T06:24:03.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.668+0000 7f469b200640 1 -- 172.21.15.16:0/2063127125 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46941a8e00 con 0x7f4694103900 2024-08-31T06:24:03.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.668+0000 7f4690c00640 1 -- 172.21.15.16:0/2063127125 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4684018980 con 0x7f4694103900 2024-08-31T06:24:03.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.669+0000 7f469b200640 1 -- 172.21.15.16:0/2063127125 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46740052c0 con 0x7f4694103900 2024-08-31T06:24:03.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.672+0000 7f4690c00640 1 -- 172.21.15.16:0/2063127125 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f4684010430 con 0x7f4694103900 2024-08-31T06:24:03.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.673+0000 7f4690c00640 1 --2- 172.21.15.16:0/2063127125 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f46780b4d20 0x7f46780b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:24:03.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.673+0000 7f4690c00640 1 -- 172.21.15.16:0/2063127125 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8684+0+0 (secure 0 0 0) 0x7f46840e5820 con 0x7f4694103900 2024-08-31T06:24:03.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.673+0000 7f4693400640 1 --2- 172.21.15.16:0/2063127125 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f46780b4d20 0x7f46780b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:24:03.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.674+0000 7f4693400640 1 --2- 172.21.15.16:0/2063127125 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f46780b4d20 0x7f46780b7230 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4680009230 tx=0x7f4680031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:24:03.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.675+0000 7f4690c00640 1 -- 172.21.15.16:0/2063127125 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f46840abf20 con 0x7f4694103900 2024-08-31T06:24:03.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.872+0000 7f469b200640 1 -- 172.21.15.16:0/2063127125 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f46740055b0 con 0x7f4694103900 2024-08-31T06:24:03.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.890+0000 7f4690c00640 1 -- 172.21.15.16:0/2063127125 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v105) ==== 181+0+0 (secure 0 0 0) 0x7f46840b06e0 con 0x7f4694103900 2024-08-31T06:24:03.893 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-08-31T06:24:03.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.894+0000 7f469b200640 1 -- 172.21.15.16:0/2063127125 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f46780b4d20 msgr2=0x7f46780b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:24:03.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.894+0000 7f469b200640 1 --2- 172.21.15.16:0/2063127125 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f46780b4d20 0x7f46780b7230 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4680009230 tx=0x7f4680031400 comp rx=0 tx=0).stop 2024-08-31T06:24:03.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.894+0000 7f469b200640 1 -- 172.21.15.16:0/2063127125 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4694103900 msgr2=0x7f469419c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:24:03.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.894+0000 7f469b200640 1 --2- 172.21.15.16:0/2063127125 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4694103900 0x7f469419c440 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f468400e9f0 tx=0x7f468400eeb0 comp rx=0 tx=0).stop 2024-08-31T06:24:03.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.894+0000 7f469b200640 1 -- 172.21.15.16:0/2063127125 shutdown_connections 2024-08-31T06:24:03.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.894+0000 7f469b200640 1 --2- 172.21.15.16:0/2063127125 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f46780b4d20 0x7f46780b7230 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:24:03.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.894+0000 7f469b200640 1 --2- 172.21.15.16:0/2063127125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4694109b90 0x7f46941a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:24:03.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.894+0000 7f469b200640 1 --2- 172.21.15.16:0/2063127125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4694109240 0x7f469419c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:24:03.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.894+0000 7f469b200640 1 --2- 172.21.15.16:0/2063127125 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4694103900 0x7f469419c440 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:24:03.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.894+0000 7f469b200640 1 -- 172.21.15.16:0/2063127125 >> 172.21.15.16:0/2063127125 conn(0x7f469407b5f0 msgr2=0x7f46941070b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:24:03.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.894+0000 7f469b200640 1 -- 172.21.15.16:0/2063127125 shutdown_connections 2024-08-31T06:24:03.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:03.894+0000 7f469b200640 1 -- 172.21.15.16:0/2063127125 wait complete. 2024-08-31T06:24:04.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:24:03 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/2063127125' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-08-31T06:24:04.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:24:03 smithi016 ceph-mon[17194]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-08-31T06:24:04.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:24:03 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/2063127125' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-08-31T06:24:04.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:24:03 smithi184 ceph-mon[22331]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-08-31T06:24:04.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:24:03 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/2063127125' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-08-31T06:24:04.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:24:03 smithi155 ceph-mon[21742]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-08-31T06:24:04.865 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6 shell --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-08-31T06:24:05.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:24:04 smithi016 ceph-mon[17194]: pgmap v892: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 4.1 KiB/s wr, 7 op/s 2024-08-31T06:24:05.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:24:04 smithi016 ceph-mon[17194]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-08-31T06:24:05.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:24:04 smithi016 ceph-mon[17194]: osdmap e105: 12 total, 12 up, 12 in 2024-08-31T06:24:05.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:24:04 smithi184 ceph-mon[22331]: pgmap v892: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 4.1 KiB/s wr, 7 op/s 2024-08-31T06:24:05.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:24:04 smithi184 ceph-mon[22331]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-08-31T06:24:05.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:24:04 smithi184 ceph-mon[22331]: osdmap e105: 12 total, 12 up, 12 in 2024-08-31T06:24:05.245 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/mon.a/config 2024-08-31T06:24:05.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:24:04 smithi155 ceph-mon[21742]: pgmap v892: 97 pgs: 97 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 4.1 KiB/s wr, 7 op/s 2024-08-31T06:24:05.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:24:04 smithi155 ceph-mon[21742]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-08-31T06:24:05.372 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:24:04 smithi155 ceph-mon[21742]: osdmap e105: 12 total, 12 up, 12 in 2024-08-31T06:24:05.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:05.912+0000 7f0997e00640 1 -- 172.21.15.16:0/3934407202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0998075e40 msgr2=0x7f099810f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:24:05.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:05.912+0000 7f0997e00640 1 --2- 172.21.15.16:0/3934407202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0998075e40 0x7f099810f7b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f0980009900 tx=0x7f098002a410 comp rx=0 tx=0).stop 2024-08-31T06:24:05.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:05.912+0000 7f0997e00640 1 -- 172.21.15.16:0/3934407202 shutdown_connections 2024-08-31T06:24:05.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:05.912+0000 7f0997e00640 1 --2- 172.21.15.16:0/3934407202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f099810fcf0 0x7f0998112110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:24:05.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:05.912+0000 7f0997e00640 1 --2- 172.21.15.16:0/3934407202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0998075e40 0x7f099810f7b0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:24:05.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:05.912+0000 7f0997e00640 1 --2- 172.21.15.16:0/3934407202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09980754f0 0x7f0998075900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:24:05.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:05.912+0000 7f0997e00640 1 -- 172.21.15.16:0/3934407202 >> 172.21.15.16:0/3934407202 conn(0x7f09980ff180 msgr2=0x7f09981015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:24:05.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:05.912+0000 7f0997e00640 1 -- 172.21.15.16:0/3934407202 shutdown_connections 2024-08-31T06:24:05.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:05.912+0000 7f0997e00640 1 -- 172.21.15.16:0/3934407202 wait complete. 2024-08-31T06:24:05.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:05.913+0000 7f0997e00640 1 Processor -- start 2024-08-31T06:24:05.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:05.913+0000 7f0997e00640 1 -- start start 2024-08-31T06:24:05.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:05.914+0000 7f0997e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09980754f0 0x7f099819c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:24:05.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:05.914+0000 7f0997e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0998075e40 0x7f099819c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:24:05.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:05.914+0000 7f0997e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f099810fcf0 0x7f09981a1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:24:05.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:05.914+0000 7f0997e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f09981158a0 con 0x7f0998075e40 2024-08-31T06:24:05.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:05.914+0000 7f0997e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0998115720 con 0x7f09980754f0 2024-08-31T06:24:05.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:05.914+0000 7f0997e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0998115a20 con 0x7f099810fcf0 2024-08-31T06:24:05.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:05.914+0000 7f098e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0998075e40 0x7f099819c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:24:05.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:05.914+0000 7f0997400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f099810fcf0 0x7f09981a1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:24:05.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:05.914+0000 7f0997400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f099810fcf0 0x7f09981a1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:44890/0 (socket says 172.21.15.16:44890) 2024-08-31T06:24:05.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:05.914+0000 7f098e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0998075e40 0x7f099819c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53654/0 (socket says 172.21.15.16:53654) 2024-08-31T06:24:05.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:05.914+0000 7f0997400640 1 -- 172.21.15.16:0/1054946823 learned_addr learned my addr 172.21.15.16:0/1054946823 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-31T06:24:05.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:05.914+0000 7f0996a00640 1 --2- 172.21.15.16:0/1054946823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09980754f0 0x7f099819c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:24:05.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:05.915+0000 7f098e000640 1 -- 172.21.15.16:0/1054946823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f099810fcf0 msgr2=0x7f09981a1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:24:05.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:05.915+0000 7f098e000640 1 --2- 172.21.15.16:0/1054946823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f099810fcf0 0x7f09981a1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:24:05.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:05.915+0000 7f098e000640 1 -- 172.21.15.16:0/1054946823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09980754f0 msgr2=0x7f099819c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:24:05.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:05.915+0000 7f098e000640 1 --2- 172.21.15.16:0/1054946823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09980754f0 0x7f099819c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:24:05.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:05.915+0000 7f098e000640 1 -- 172.21.15.16:0/1054946823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09981a22a0 con 0x7f0998075e40 2024-08-31T06:24:05.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:05.915+0000 7f0997400640 1 --2- 172.21.15.16:0/1054946823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f099810fcf0 0x7f09981a1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-31T06:24:05.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:05.915+0000 7f098e000640 1 --2- 172.21.15.16:0/1054946823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0998075e40 0x7f099819c8d0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f098002a930 tx=0x7f0980004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:24:05.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:05.915+0000 7f098fe00640 1 -- 172.21.15.16:0/1054946823 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098003f070 con 0x7f0998075e40 2024-08-31T06:24:05.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:05.915+0000 7f098fe00640 1 -- 172.21.15.16:0/1054946823 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0980003aa0 con 0x7f0998075e40 2024-08-31T06:24:05.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:05.915+0000 7f0997e00640 1 -- 172.21.15.16:0/1054946823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09981a2530 con 0x7f0998075e40 2024-08-31T06:24:05.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:05.915+0000 7f098fe00640 1 -- 172.21.15.16:0/1054946823 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0980035070 con 0x7f0998075e40 2024-08-31T06:24:05.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:05.915+0000 7f0997e00640 1 -- 172.21.15.16:0/1054946823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09981a8cf0 con 0x7f0998075e40 2024-08-31T06:24:05.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:05.916+0000 7f0997e00640 1 -- 172.21.15.16:0/1054946823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0998075900 con 0x7f0998075e40 2024-08-31T06:24:05.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:05.919+0000 7f098fe00640 1 -- 172.21.15.16:0/1054946823 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 152708+0+0 (secure 0 0 0) 0x7f09800035f0 con 0x7f0998075e40 2024-08-31T06:24:05.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:05.920+0000 7f098fe00640 1 --2- 172.21.15.16:0/1054946823 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f097c0b4ca0 0x7f097c0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-31T06:24:05.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:05.920+0000 7f098fe00640 1 -- 172.21.15.16:0/1054946823 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8130+0+0 (secure 0 0 0) 0x7f0980105bf0 con 0x7f0998075e40 2024-08-31T06:24:05.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:05.921+0000 7f0996a00640 1 --2- 172.21.15.16:0/1054946823 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f097c0b4ca0 0x7f097c0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-31T06:24:05.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:05.921+0000 7f0996a00640 1 --2- 172.21.15.16:0/1054946823 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f097c0b4ca0 0x7f097c0b71b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0988009990 tx=0x7f0988009110 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-31T06:24:05.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:05.922+0000 7f098fe00640 1 -- 172.21.15.16:0/1054946823 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196895 (secure 0 0 0) 0x7f09800ccb10 con 0x7f0998075e40 2024-08-31T06:24:06.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:06.123+0000 7f0997e00640 1 -- 172.21.15.16:0/1054946823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f09981a8ec0 con 0x7f0998075e40 2024-08-31T06:24:06.895 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:24:06 smithi155 ceph-mon[21742]: pgmap v894: 65 pgs: 65 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:24:06.895 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:24:06 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/1054946823' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-08-31T06:24:06.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:06.897+0000 7f098fe00640 1 -- 172.21.15.16:0/1054946823 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v106) ==== 169+0+0 (secure 0 0 0) 0x7f09800cc410 con 0x7f0998075e40 2024-08-31T06:24:06.900 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-08-31T06:24:06.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:06.902+0000 7f0997e00640 1 -- 172.21.15.16:0/1054946823 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f097c0b4ca0 msgr2=0x7f097c0b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:24:06.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:06.902+0000 7f0997e00640 1 --2- 172.21.15.16:0/1054946823 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f097c0b4ca0 0x7f097c0b71b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0988009990 tx=0x7f0988009110 comp rx=0 tx=0).stop 2024-08-31T06:24:06.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:06.902+0000 7f0997e00640 1 -- 172.21.15.16:0/1054946823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0998075e40 msgr2=0x7f099819c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-31T06:24:06.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:06.902+0000 7f0997e00640 1 --2- 172.21.15.16:0/1054946823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0998075e40 0x7f099819c8d0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f098002a930 tx=0x7f0980004060 comp rx=0 tx=0).stop 2024-08-31T06:24:06.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:06.903+0000 7f0997e00640 1 -- 172.21.15.16:0/1054946823 shutdown_connections 2024-08-31T06:24:06.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:06.903+0000 7f0997e00640 1 --2- 172.21.15.16:0/1054946823 >> [v2:172.21.15.16:6800/4279751337,v1:172.21.15.16:6801/4279751337] conn(0x7f097c0b4ca0 0x7f097c0b71b0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:24:06.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:06.903+0000 7f0997e00640 1 --2- 172.21.15.16:0/1054946823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f099810fcf0 0x7f09981a1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:24:06.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:06.903+0000 7f0997e00640 1 --2- 172.21.15.16:0/1054946823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0998075e40 0x7f099819c8d0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:24:06.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:06.903+0000 7f0997e00640 1 --2- 172.21.15.16:0/1054946823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09980754f0 0x7f099819c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-31T06:24:06.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:06.903+0000 7f0997e00640 1 -- 172.21.15.16:0/1054946823 >> 172.21.15.16:0/1054946823 conn(0x7f09980ff180 msgr2=0x7f0998108780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-31T06:24:06.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:06.904+0000 7f0997e00640 1 -- 172.21.15.16:0/1054946823 shutdown_connections 2024-08-31T06:24:06.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-31T06:24:06.904+0000 7f0997e00640 1 -- 172.21.15.16:0/1054946823 wait complete. 2024-08-31T06:24:07.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:24:06 smithi016 ceph-mon[17194]: pgmap v894: 65 pgs: 65 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:24:07.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:24:06 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/1054946823' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-08-31T06:24:07.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:24:06 smithi184 ceph-mon[22331]: pgmap v894: 65 pgs: 65 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-31T06:24:07.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:24:06 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/1054946823' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-08-31T06:24:08.043 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-31T06:24:08.143 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-31T06:24:08.193 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:24:07 smithi155 ceph-mon[21742]: from='client.? 172.21.15.16:0/1054946823' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-08-31T06:24:08.193 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:24:07 smithi155 ceph-mon[21742]: osdmap e106: 12 total, 12 up, 12 in 2024-08-31T06:24:08.200 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-31T06:24:08.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:24:07 smithi184 ceph-mon[22331]: from='client.? 172.21.15.16:0/1054946823' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-08-31T06:24:08.222 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:24:07 smithi184 ceph-mon[22331]: osdmap e106: 12 total, 12 up, 12 in 2024-08-31T06:24:08.231 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-08-31T06:24:08.231 DEBUG:teuthology.orchestra.run.smithi016:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-08-31T06:24:08.294 DEBUG:teuthology.orchestra.run.smithi155:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-08-31T06:24:08.328 DEBUG:teuthology.orchestra.run.smithi184:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-08-31T06:24:08.343 INFO:tasks.cephadm:Stopping all daemons... 2024-08-31T06:24:08.343 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-08-31T06:24:08.343 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mon.a 2024-08-31T06:24:08.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:24:07 smithi016 ceph-mon[17194]: from='client.? 172.21.15.16:0/1054946823' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-08-31T06:24:08.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:24:07 smithi016 ceph-mon[17194]: osdmap e106: 12 total, 12 up, 12 in 2024-08-31T06:24:08.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:24:08 smithi016 systemd[1]: Stopping Ceph mon.a for 673e4aa0-675d-11ef-bcd4-c7b262605968... 2024-08-31T06:24:09.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:24:08 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mon-a[17190]: 2024-08-31T06:24:08.908+0000 7f9976c00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T06:24:09.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:24:08 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mon-a[17190]: 2024-08-31T06:24:08.908+0000 7f9976c00640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-08-31T06:24:09.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:24:09 smithi016 podman[107138]: 2024-08-31 06:24:09.155643772 +0000 UTC m=+0.407595904 container died b27650bd7483dbf9f61bd09190df2beabe54158928cd34ed3e97f34435dab432 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mon-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-08-31T06:24:10.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:24:09 smithi016 podman[107138]: 2024-08-31 06:24:09.791454281 +0000 UTC m=+1.043406418 container cleanup b27650bd7483dbf9f61bd09190df2beabe54158928cd34ed3e97f34435dab432 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mon-a, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.1, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-08-31T06:24:10.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:24:09 smithi016 bash[107138]: ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mon-a 2024-08-31T06:24:10.634 INFO:journalctl@ceph.mon.a.smithi016.stdout:Aug 31 06:24:10 smithi016 podman[107147]: 2024-08-31 06:24:10.307548988 +0000 UTC m=+1.145693564 container remove b27650bd7483dbf9f61bd09190df2beabe54158928cd34ed3e97f34435dab432 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-08-31T06:24:10.653 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mon.a.service' 2024-08-31T06:24:10.766 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T06:24:10.766 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-08-31T06:24:10.766 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-08-31T06:24:10.767 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mon.b 2024-08-31T06:24:11.083 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:24:10 smithi155 systemd[1]: Stopping Ceph mon.b for 673e4aa0-675d-11ef-bcd4-c7b262605968... 2024-08-31T06:24:11.360 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:24:11 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mon-b[21738]: 2024-08-31T06:24:11.080+0000 7f1e11fe5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T06:24:11.360 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:24:11 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mon-b[21738]: 2024-08-31T06:24:11.080+0000 7f1e11fe5640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-08-31T06:24:11.360 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:24:11 smithi155 podman[58680]: 2024-08-31 06:24:11.294368547 +0000 UTC m=+0.344798817 container died d41e849cd3cb90fe0eb579ec4f702be32a7e97cef70803e7776be254985847e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mon-b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.1) 2024-08-31T06:24:12.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:24:11 smithi155 podman[58680]: 2024-08-31 06:24:11.689796268 +0000 UTC m=+0.740226544 container cleanup d41e849cd3cb90fe0eb579ec4f702be32a7e97cef70803e7776be254985847e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mon-b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-08-31T06:24:12.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 31 06:24:11 smithi155 bash[58680]: ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mon-b 2024-08-31T06:24:12.474 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mon.b.service' 2024-08-31T06:24:12.531 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T06:24:12.532 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-08-31T06:24:12.532 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-08-31T06:24:12.532 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mon.c 2024-08-31T06:24:12.863 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:24:12 smithi184 systemd[1]: Stopping Ceph mon.c for 673e4aa0-675d-11ef-bcd4-c7b262605968... 2024-08-31T06:24:13.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:24:12 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mon-c[22327]: 2024-08-31T06:24:12.860+0000 7f1b2a03c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T06:24:13.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:24:12 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mon-c[22327]: 2024-08-31T06:24:12.860+0000 7f1b2a03c640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-08-31T06:24:13.223 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:24:13 smithi184 podman[59827]: 2024-08-31 06:24:13.054752379 +0000 UTC m=+0.323809818 container died cc3de2bf8812a01cf5f5e345061bd69ae8bb6b09f8528ac1bf2d71bc6dfb899a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mon-c, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.1, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-08-31T06:24:13.722 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:24:13 smithi184 podman[59827]: 2024-08-31 06:24:13.453036588 +0000 UTC m=+0.722094016 container cleanup cc3de2bf8812a01cf5f5e345061bd69ae8bb6b09f8528ac1bf2d71bc6dfb899a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mon-c, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-08-31T06:24:13.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:Aug 31 06:24:13 smithi184 bash[59827]: ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mon-c 2024-08-31T06:24:14.033 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mon.c.service' 2024-08-31T06:24:14.075 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T06:24:14.075 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-08-31T06:24:14.075 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-08-31T06:24:14.076 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mgr.x 2024-08-31T06:24:14.416 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 06:24:14 smithi016 systemd[1]: Stopping Ceph mgr.x for 673e4aa0-675d-11ef-bcd4-c7b262605968... 2024-08-31T06:24:14.913 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 06:24:14 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x[17421]: 2024-08-31T06:24:14.499+0000 7fdf4ac00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T06:24:14.913 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 06:24:14 smithi016 podman[107355]: 2024-08-31 06:24:14.602948316 +0000 UTC m=+0.232790823 container died a9111fa496cbf6132c6a860b0a1f89996258208355acec3b4548aa7e64134eb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.1) 2024-08-31T06:24:15.663 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 06:24:15 smithi016 podman[107355]: 2024-08-31 06:24:15.223988539 +0000 UTC m=+0.853831045 container cleanup a9111fa496cbf6132c6a860b0a1f89996258208355acec3b4548aa7e64134eb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x, io.buildah.version=1.37.1, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2) 2024-08-31T06:24:15.663 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 06:24:15 smithi016 bash[107355]: ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x 2024-08-31T06:24:15.663 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Aug 31 06:24:15 smithi016 podman[107367]: 2024-08-31 06:24:15.451015983 +0000 UTC m=+0.841591690 container remove a9111fa496cbf6132c6a860b0a1f89996258208355acec3b4548aa7e64134eb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d19e6118f7015f0047264006cb0c873528eb3ae6, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-x, io.buildah.version=1.37.1, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-08-31T06:24:15.719 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mgr.x.service' 2024-08-31T06:24:15.794 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T06:24:15.794 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-08-31T06:24:15.794 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-08-31T06:24:15.794 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mgr.y 2024-08-31T06:24:16.118 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 31 06:24:15 smithi155 systemd[1]: Stopping Ceph mgr.y for 673e4aa0-675d-11ef-bcd4-c7b262605968... 2024-08-31T06:24:16.372 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 31 06:24:16 smithi155 podman[58813]: 2024-08-31 06:24:16.116002482 +0000 UTC m=+0.125038572 container died 2368cdbe66a423b1322f71968490f8f68a56cf38ebc11ccb11518a5d98678339 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-y, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.1) 2024-08-31T06:24:16.872 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 31 06:24:16 smithi155 podman[58813]: 2024-08-31 06:24:16.58768016 +0000 UTC m=+0.596716250 container cleanup 2368cdbe66a423b1322f71968490f8f68a56cf38ebc11ccb11518a5d98678339 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-y, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-31T06:24:16.872 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 31 06:24:16 smithi155 bash[58813]: ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-y 2024-08-31T06:24:17.062 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mgr.y.service' 2024-08-31T06:24:17.097 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T06:24:17.097 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-08-31T06:24:17.097 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-08-31T06:24:17.097 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mgr.z 2024-08-31T06:24:17.431 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Aug 31 06:24:17 smithi184 systemd[1]: Stopping Ceph mgr.z for 673e4aa0-675d-11ef-bcd4-c7b262605968... 2024-08-31T06:24:17.707 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Aug 31 06:24:17 smithi184 podman[59951]: 2024-08-31 06:24:17.429313819 +0000 UTC m=+0.133180346 container died 4154939d0c3b069be853dcd43585438787b42a5da926030861ab64c00702d91a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-z, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.1, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_CLEAN=True) 2024-08-31T06:24:17.973 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Aug 31 06:24:17 smithi184 podman[59951]: 2024-08-31 06:24:17.82613131 +0000 UTC m=+0.529997837 container cleanup 4154939d0c3b069be853dcd43585438787b42a5da926030861ab64c00702d91a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-z, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.1, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-31T06:24:17.973 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Aug 31 06:24:17 smithi184 bash[59951]: ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mgr-z 2024-08-31T06:24:18.261 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mgr.z.service' 2024-08-31T06:24:18.299 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T06:24:18.299 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-08-31T06:24:18.299 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-08-31T06:24:18.300 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@osd.0 2024-08-31T06:24:18.663 INFO:journalctl@ceph.osd.0.smithi016.stdout:Aug 31 06:24:18 smithi016 systemd[1]: Stopping Ceph osd.0 for 673e4aa0-675d-11ef-bcd4-c7b262605968... 2024-08-31T06:24:19.163 INFO:journalctl@ceph.osd.0.smithi016.stdout:Aug 31 06:24:18 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-0[35211]: 2024-08-31T06:24:18.731+0000 7fbab8200640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T06:24:19.163 INFO:journalctl@ceph.osd.0.smithi016.stdout:Aug 31 06:24:18 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-0[35211]: 2024-08-31T06:24:18.731+0000 7fbab8200640 -1 osd.0 106 *** Got signal Terminated *** 2024-08-31T06:24:19.163 INFO:journalctl@ceph.osd.0.smithi016.stdout:Aug 31 06:24:18 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-0[35211]: 2024-08-31T06:24:18.731+0000 7fbab8200640 -1 osd.0 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-31T06:24:24.913 INFO:journalctl@ceph.osd.0.smithi016.stdout:Aug 31 06:24:24 smithi016 podman[107473]: 2024-08-31 06:24:24.652842567 +0000 UTC m=+6.059913652 container died 8f392c40d13cc86fa4cd5259908d70d2c3a0ddcf20aa30ee740db53d31f6a449 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-0, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-31T06:24:25.599 INFO:journalctl@ceph.osd.0.smithi016.stdout:Aug 31 06:24:25 smithi016 podman[107473]: 2024-08-31 06:24:25.216346446 +0000 UTC m=+6.623417521 container cleanup 8f392c40d13cc86fa4cd5259908d70d2c3a0ddcf20aa30ee740db53d31f6a449 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-0, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.1) 2024-08-31T06:24:25.599 INFO:journalctl@ceph.osd.0.smithi016.stdout:Aug 31 06:24:25 smithi016 bash[107473]: ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-0 2024-08-31T06:24:25.599 INFO:journalctl@ceph.osd.0.smithi016.stdout:Aug 31 06:24:25 smithi016 podman[107489]: 2024-08-31 06:24:25.426676378 +0000 UTC m=+0.755737467 container remove 8f392c40d13cc86fa4cd5259908d70d2c3a0ddcf20aa30ee740db53d31f6a449 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.1, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-08-31T06:24:26.324 INFO:journalctl@ceph.osd.0.smithi016.stdout:Aug 31 06:24:26 smithi016 podman[107586]: 2024-08-31 06:24:25.952811564 +0000 UTC m=+0.028322944 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T06:24:26.324 INFO:journalctl@ceph.osd.0.smithi016.stdout:Aug 31 06:24:26 smithi016 podman[107586]: 2024-08-31 06:24:26.077759155 +0000 UTC m=+0.153270511 container create c7af7df2ea9097ac07b24f0fe70d399a5801bc2b613532cb4cd6a373eb43f718 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.1, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-31T06:24:26.605 INFO:journalctl@ceph.osd.0.smithi016.stdout:Aug 31 06:24:26 smithi016 podman[107586]: 2024-08-31 06:24:26.322278258 +0000 UTC m=+0.397789640 container init c7af7df2ea9097ac07b24f0fe70d399a5801bc2b613532cb4cd6a373eb43f718 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-31T06:24:26.605 INFO:journalctl@ceph.osd.0.smithi016.stdout:Aug 31 06:24:26 smithi016 podman[107586]: 2024-08-31 06:24:26.32966464 +0000 UTC m=+0.405175999 container start c7af7df2ea9097ac07b24f0fe70d399a5801bc2b613532cb4cd6a373eb43f718 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-31T06:24:26.605 INFO:journalctl@ceph.osd.0.smithi016.stdout:Aug 31 06:24:26 smithi016 podman[107586]: 2024-08-31 06:24:26.439036369 +0000 UTC m=+0.514547741 container attach c7af7df2ea9097ac07b24f0fe70d399a5801bc2b613532cb4cd6a373eb43f718 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.1, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-08-31T06:24:26.913 INFO:journalctl@ceph.osd.0.smithi016.stdout:Aug 31 06:24:26 smithi016 podman[107586]: 2024-08-31 06:24:26.603407238 +0000 UTC m=+0.678918613 container died c7af7df2ea9097ac07b24f0fe70d399a5801bc2b613532cb4cd6a373eb43f718 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.1) 2024-08-31T06:24:27.385 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@osd.0.service' 2024-08-31T06:24:27.462 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T06:24:27.462 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-08-31T06:24:27.462 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-08-31T06:24:27.463 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@osd.1 2024-08-31T06:24:27.795 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 31 06:24:27 smithi155 systemd[1]: Stopping Ceph osd.1 for 673e4aa0-675d-11ef-bcd4-c7b262605968... 2024-08-31T06:24:28.124 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 31 06:24:27 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-1[27505]: 2024-08-31T06:24:27.791+0000 7f37291ab640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T06:24:28.125 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 31 06:24:27 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-1[27505]: 2024-08-31T06:24:27.791+0000 7f37291ab640 -1 osd.1 106 *** Got signal Terminated *** 2024-08-31T06:24:28.125 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 31 06:24:27 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-1[27505]: 2024-08-31T06:24:27.791+0000 7f37291ab640 -1 osd.1 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-31T06:24:33.854 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 31 06:24:33 smithi155 podman[58938]: 2024-08-31 06:24:33.551703903 +0000 UTC m=+5.886861966 container died 850d12528ba851ec1ba1c5f31e50605c74f7a30678e09cbe7b07f106a6f1f56b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-1, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.1, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-08-31T06:24:34.122 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 31 06:24:33 smithi155 podman[58938]: 2024-08-31 06:24:33.951111851 +0000 UTC m=+6.286269897 container cleanup 850d12528ba851ec1ba1c5f31e50605c74f7a30678e09cbe7b07f106a6f1f56b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.1, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-08-31T06:24:34.122 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 31 06:24:33 smithi155 bash[58938]: ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-1 2024-08-31T06:24:34.569 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 31 06:24:34 smithi155 podman[58955]: 2024-08-31 06:24:34.210631181 +0000 UTC m=+0.658894631 container remove 850d12528ba851ec1ba1c5f31e50605c74f7a30678e09cbe7b07f106a6f1f56b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.1, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-08-31T06:24:34.849 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 31 06:24:34 smithi155 podman[59043]: 2024-08-31 06:24:34.470617757 +0000 UTC m=+0.016171112 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T06:24:34.850 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 31 06:24:34 smithi155 podman[59043]: 2024-08-31 06:24:34.611602626 +0000 UTC m=+0.157156019 container create c851921c03c37c4dc5072d8e5ecdc1c08d2d329e003a4bdf18ce91497f639020 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.1, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611) 2024-08-31T06:24:35.122 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 31 06:24:34 smithi155 podman[59043]: 2024-08-31 06:24:34.847773459 +0000 UTC m=+0.393326807 container init c851921c03c37c4dc5072d8e5ecdc1c08d2d329e003a4bdf18ce91497f639020 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.1, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611) 2024-08-31T06:24:35.123 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 31 06:24:34 smithi155 podman[59043]: 2024-08-31 06:24:34.851719677 +0000 UTC m=+0.397273033 container start c851921c03c37c4dc5072d8e5ecdc1c08d2d329e003a4bdf18ce91497f639020 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.1, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-08-31T06:24:35.123 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 31 06:24:34 smithi155 podman[59043]: 2024-08-31 06:24:34.9646157 +0000 UTC m=+0.510169060 container attach c851921c03c37c4dc5072d8e5ecdc1c08d2d329e003a4bdf18ce91497f639020 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.1, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-08-31T06:24:35.123 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 31 06:24:35 smithi155 podman[59043]: 2024-08-31 06:24:35.058225194 +0000 UTC m=+0.603778545 container died c851921c03c37c4dc5072d8e5ecdc1c08d2d329e003a4bdf18ce91497f639020 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-31T06:24:35.690 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@osd.1.service' 2024-08-31T06:24:35.728 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T06:24:35.729 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-08-31T06:24:35.729 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-08-31T06:24:35.729 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@osd.2 2024-08-31T06:24:36.133 INFO:journalctl@ceph.osd.2.smithi184.stdout:Aug 31 06:24:35 smithi184 systemd[1]: Stopping Ceph osd.2 for 673e4aa0-675d-11ef-bcd4-c7b262605968... 2024-08-31T06:24:36.473 INFO:journalctl@ceph.osd.2.smithi184.stdout:Aug 31 06:24:36 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-2[28490]: 2024-08-31T06:24:36.130+0000 7fd7d1676640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T06:24:36.473 INFO:journalctl@ceph.osd.2.smithi184.stdout:Aug 31 06:24:36 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-2[28490]: 2024-08-31T06:24:36.130+0000 7fd7d1676640 -1 osd.2 106 *** Got signal Terminated *** 2024-08-31T06:24:36.473 INFO:journalctl@ceph.osd.2.smithi184.stdout:Aug 31 06:24:36 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-2[28490]: 2024-08-31T06:24:36.130+0000 7fd7d1676640 -1 osd.2 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-31T06:24:42.124 INFO:journalctl@ceph.osd.2.smithi184.stdout:Aug 31 06:24:41 smithi184 podman[60085]: 2024-08-31 06:24:41.827042669 +0000 UTC m=+5.891442076 container died 90887a957a46b4d1144ca6cf58f19319aaae8a049c047347725bd88c788f641c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.1) 2024-08-31T06:24:42.458 INFO:journalctl@ceph.osd.2.smithi184.stdout:Aug 31 06:24:42 smithi184 podman[60085]: 2024-08-31 06:24:42.221931851 +0000 UTC m=+6.286331258 container cleanup 90887a957a46b4d1144ca6cf58f19319aaae8a049c047347725bd88c788f641c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1) 2024-08-31T06:24:42.458 INFO:journalctl@ceph.osd.2.smithi184.stdout:Aug 31 06:24:42 smithi184 bash[60085]: ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-2 2024-08-31T06:24:42.722 INFO:journalctl@ceph.osd.2.smithi184.stdout:Aug 31 06:24:42 smithi184 podman[60099]: 2024-08-31 06:24:42.456946434 +0000 UTC m=+0.625186370 container remove 90887a957a46b4d1144ca6cf58f19319aaae8a049c047347725bd88c788f641c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-2, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-08-31T06:24:43.079 INFO:journalctl@ceph.osd.2.smithi184.stdout:Aug 31 06:24:42 smithi184 podman[60188]: 2024-08-31 06:24:42.681257568 +0000 UTC m=+0.015910003 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T06:24:43.079 INFO:journalctl@ceph.osd.2.smithi184.stdout:Aug 31 06:24:42 smithi184 podman[60188]: 2024-08-31 06:24:42.808165591 +0000 UTC m=+0.142818012 container create 853c313377e28f6bd3b01e1d4d5301185998d09372890b21c63436411f2bda30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-2-deactivate, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-31T06:24:43.473 INFO:journalctl@ceph.osd.2.smithi184.stdout:Aug 31 06:24:43 smithi184 podman[60188]: 2024-08-31 06:24:43.077582094 +0000 UTC m=+0.412234517 container init 853c313377e28f6bd3b01e1d4d5301185998d09372890b21c63436411f2bda30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.1, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-08-31T06:24:43.473 INFO:journalctl@ceph.osd.2.smithi184.stdout:Aug 31 06:24:43 smithi184 podman[60188]: 2024-08-31 06:24:43.081115856 +0000 UTC m=+0.415768279 container start 853c313377e28f6bd3b01e1d4d5301185998d09372890b21c63436411f2bda30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.1, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-08-31T06:24:43.473 INFO:journalctl@ceph.osd.2.smithi184.stdout:Aug 31 06:24:43 smithi184 podman[60188]: 2024-08-31 06:24:43.203021089 +0000 UTC m=+0.537673526 container attach 853c313377e28f6bd3b01e1d4d5301185998d09372890b21c63436411f2bda30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-31T06:24:43.474 INFO:journalctl@ceph.osd.2.smithi184.stdout:Aug 31 06:24:43 smithi184 podman[60188]: 2024-08-31 06:24:43.269714749 +0000 UTC m=+0.604367184 container died 853c313377e28f6bd3b01e1d4d5301185998d09372890b21c63436411f2bda30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-2-deactivate, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-08-31T06:24:43.964 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@osd.2.service' 2024-08-31T06:24:44.002 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T06:24:44.003 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-08-31T06:24:44.003 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-08-31T06:24:44.003 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@osd.3 2024-08-31T06:24:44.413 INFO:journalctl@ceph.osd.3.smithi016.stdout:Aug 31 06:24:44 smithi016 systemd[1]: Stopping Ceph osd.3 for 673e4aa0-675d-11ef-bcd4-c7b262605968... 2024-08-31T06:24:44.913 INFO:journalctl@ceph.osd.3.smithi016.stdout:Aug 31 06:24:44 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-3[42089]: 2024-08-31T06:24:44.440+0000 7fa93fc00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T06:24:44.913 INFO:journalctl@ceph.osd.3.smithi016.stdout:Aug 31 06:24:44 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-3[42089]: 2024-08-31T06:24:44.440+0000 7fa93fc00640 -1 osd.3 106 *** Got signal Terminated *** 2024-08-31T06:24:44.913 INFO:journalctl@ceph.osd.3.smithi016.stdout:Aug 31 06:24:44 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-3[42089]: 2024-08-31T06:24:44.440+0000 7fa93fc00640 -1 osd.3 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-31T06:24:45.913 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:24:45 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:24:45.714+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:21.798609+0000 front 2024-08-31T06:24:21.798847+0000 (oldest deadline 2024-08-31T06:24:45.298817+0000) 2024-08-31T06:24:45.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:24:45 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:24:45.636+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:24:46.555 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:24:46 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:24:46.261+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:24:46.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:24:46 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:24:46.648+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:24:46.913 INFO:journalctl@ceph.osd.3.smithi016.stdout:Aug 31 06:24:46 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-3[42089]: 2024-08-31T06:24:46.674+0000 7fa93b600640 -1 osd.3 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:21.314985+0000 front 2024-08-31T06:24:21.314938+0000 (oldest deadline 2024-08-31T06:24:46.015063+0000) 2024-08-31T06:24:46.914 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:24:46 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:24:46.751+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:21.798609+0000 front 2024-08-31T06:24:21.798847+0000 (oldest deadline 2024-08-31T06:24:45.298817+0000) 2024-08-31T06:24:46.972 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:24:46 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5[34971]: 2024-08-31T06:24:46.553+0000 7f8af3539640 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.836260+0000 front 2024-08-31T06:24:22.836471+0000 (oldest deadline 2024-08-31T06:24:46.336170+0000) 2024-08-31T06:24:47.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:24:46 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:24:46.693+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:24:47.597 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:24:47 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:24:47.228+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:24:47.913 INFO:journalctl@ceph.osd.3.smithi016.stdout:Aug 31 06:24:47 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-3[42089]: 2024-08-31T06:24:47.641+0000 7fa93b600640 -1 osd.3 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:21.314985+0000 front 2024-08-31T06:24:21.314938+0000 (oldest deadline 2024-08-31T06:24:46.015063+0000) 2024-08-31T06:24:47.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:24:47 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:24:47.673+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:24:47.913 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:24:47 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:24:47.757+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:21.798609+0000 front 2024-08-31T06:24:21.798847+0000 (oldest deadline 2024-08-31T06:24:45.298817+0000) 2024-08-31T06:24:47.972 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:24:47 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5[34971]: 2024-08-31T06:24:47.595+0000 7f8af3539640 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.836260+0000 front 2024-08-31T06:24:22.836471+0000 (oldest deadline 2024-08-31T06:24:46.336170+0000) 2024-08-31T06:24:48.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:24:47 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:24:47.650+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:24:48.645 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:24:48 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:24:48.248+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:24:48.972 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:24:48 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5[34971]: 2024-08-31T06:24:48.643+0000 7f8af3539640 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.836260+0000 front 2024-08-31T06:24:22.836471+0000 (oldest deadline 2024-08-31T06:24:46.336170+0000) 2024-08-31T06:24:48.973 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:24:48 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:24:48.818+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:24:49.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:24:48 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:24:48.657+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:24:49.163 INFO:journalctl@ceph.osd.3.smithi016.stdout:Aug 31 06:24:48 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-3[42089]: 2024-08-31T06:24:48.688+0000 7fa93b600640 -1 osd.3 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:21.314985+0000 front 2024-08-31T06:24:21.314938+0000 (oldest deadline 2024-08-31T06:24:46.015063+0000) 2024-08-31T06:24:49.163 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:24:48 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:24:48.779+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:21.798609+0000 front 2024-08-31T06:24:21.798847+0000 (oldest deadline 2024-08-31T06:24:45.298817+0000) 2024-08-31T06:24:49.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:24:48 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:24:48.669+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:24:49.658 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:24:49 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:24:49.237+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:24:49.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:24:49 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:24:49.619+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:24:49.913 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:24:49 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:24:49.732+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:21.798609+0000 front 2024-08-31T06:24:21.798847+0000 (oldest deadline 2024-08-31T06:24:45.298817+0000) 2024-08-31T06:24:49.972 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:24:49 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5[34971]: 2024-08-31T06:24:49.656+0000 7f8af3539640 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.836260+0000 front 2024-08-31T06:24:22.836471+0000 (oldest deadline 2024-08-31T06:24:46.336170+0000) 2024-08-31T06:24:49.973 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:24:49 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:24:49.780+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:24:50.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:24:49 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:24:49.650+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:24:50.694 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:24:50 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:24:50.225+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:24:50.783 INFO:journalctl@ceph.osd.3.smithi016.stdout:Aug 31 06:24:50 smithi016 podman[107688]: 2024-08-31 06:24:50.499873397 +0000 UTC m=+6.208187535 container died 00de9756ea21b2b8db7d7737426f78f7a4f09b07bbb9b5b75575e64758d7146b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-3, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.1, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-31T06:24:50.783 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:24:50 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:24:50.604+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:24:50.972 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:24:50 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5[34971]: 2024-08-31T06:24:50.692+0000 7f8af3539640 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.836260+0000 front 2024-08-31T06:24:22.836471+0000 (oldest deadline 2024-08-31T06:24:46.336170+0000) 2024-08-31T06:24:50.973 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:24:50 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:24:50.828+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:24:51.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:24:50 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:24:50.669+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:24:51.163 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:24:50 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:24:50.781+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:21.798609+0000 front 2024-08-31T06:24:21.798847+0000 (oldest deadline 2024-08-31T06:24:45.298817+0000) 2024-08-31T06:24:51.163 INFO:journalctl@ceph.osd.3.smithi016.stdout:Aug 31 06:24:50 smithi016 podman[107688]: 2024-08-31 06:24:50.946397575 +0000 UTC m=+6.654711712 container cleanup 00de9756ea21b2b8db7d7737426f78f7a4f09b07bbb9b5b75575e64758d7146b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-3, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.1, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-31T06:24:51.163 INFO:journalctl@ceph.osd.3.smithi016.stdout:Aug 31 06:24:50 smithi016 bash[107688]: ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-3 2024-08-31T06:24:51.472 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:24:51 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:24:51.177+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:24:51.580 INFO:journalctl@ceph.osd.3.smithi016.stdout:Aug 31 06:24:51 smithi016 podman[107705]: 2024-08-31 06:24:51.16446894 +0000 UTC m=+0.654210845 container remove 00de9756ea21b2b8db7d7737426f78f7a4f09b07bbb9b5b75575e64758d7146b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.1, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.vendor=CentOS) 2024-08-31T06:24:51.894 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:24:51 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:24:51.795+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:21.798609+0000 front 2024-08-31T06:24:21.798847+0000 (oldest deadline 2024-08-31T06:24:45.298817+0000) 2024-08-31T06:24:51.895 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:24:51 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:24:51.578+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:24:51.895 INFO:journalctl@ceph.osd.3.smithi016.stdout:Aug 31 06:24:51 smithi016 podman[107795]: 2024-08-31 06:24:51.630594381 +0000 UTC m=+0.028083099 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T06:24:51.895 INFO:journalctl@ceph.osd.3.smithi016.stdout:Aug 31 06:24:51 smithi016 podman[107795]: 2024-08-31 06:24:51.74922908 +0000 UTC m=+0.146717798 container create 67f6368a1906d49509e81717298a07087add53bb167f7088a9dd9dd82dd99180 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.1, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-31T06:24:51.972 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:24:51 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5[34971]: 2024-08-31T06:24:51.648+0000 7f8af3539640 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.836260+0000 front 2024-08-31T06:24:22.836471+0000 (oldest deadline 2024-08-31T06:24:46.336170+0000) 2024-08-31T06:24:51.973 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:24:51 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:24:51.851+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:24:52.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:24:51 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:24:51.669+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:24:52.163 INFO:journalctl@ceph.osd.3.smithi016.stdout:Aug 31 06:24:52 smithi016 podman[107795]: 2024-08-31 06:24:52.027322719 +0000 UTC m=+0.424811437 container init 67f6368a1906d49509e81717298a07087add53bb167f7088a9dd9dd82dd99180 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.1, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-08-31T06:24:52.163 INFO:journalctl@ceph.osd.3.smithi016.stdout:Aug 31 06:24:52 smithi016 podman[107795]: 2024-08-31 06:24:52.034723639 +0000 UTC m=+0.432212358 container start 67f6368a1906d49509e81717298a07087add53bb167f7088a9dd9dd82dd99180 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-31T06:24:52.472 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:24:52 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:24:52.193+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:24:52.626 INFO:journalctl@ceph.osd.3.smithi016.stdout:Aug 31 06:24:52 smithi016 podman[107795]: 2024-08-31 06:24:52.1686397 +0000 UTC m=+0.566128428 container attach 67f6368a1906d49509e81717298a07087add53bb167f7088a9dd9dd82dd99180 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-08-31T06:24:52.626 INFO:journalctl@ceph.osd.3.smithi016.stdout:Aug 31 06:24:52 smithi016 podman[107795]: 2024-08-31 06:24:52.300923309 +0000 UTC m=+0.698412030 container died 67f6368a1906d49509e81717298a07087add53bb167f7088a9dd9dd82dd99180 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.1, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-31T06:24:52.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:24:52 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:24:52.623+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:24:52.913 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:24:52 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:24:52.827+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:21.798609+0000 front 2024-08-31T06:24:21.798847+0000 (oldest deadline 2024-08-31T06:24:45.298817+0000) 2024-08-31T06:24:52.972 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:24:52 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5[34971]: 2024-08-31T06:24:52.633+0000 7f8af3539640 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.836260+0000 front 2024-08-31T06:24:22.836471+0000 (oldest deadline 2024-08-31T06:24:46.336170+0000) 2024-08-31T06:24:52.973 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:24:52 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:24:52.843+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:24:53.022 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@osd.3.service' 2024-08-31T06:24:53.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:24:52 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:24:52.652+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:24:53.160 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T06:24:53.160 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-08-31T06:24:53.160 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-08-31T06:24:53.160 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@osd.4 2024-08-31T06:24:53.548 INFO:journalctl@ceph.osd.4.smithi155.stdout:Aug 31 06:24:53 smithi155 systemd[1]: Stopping Ceph osd.4 for 673e4aa0-675d-11ef-bcd4-c7b262605968... 2024-08-31T06:24:53.644 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:24:53 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:24:53.243+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:24:53.872 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:24:53 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:24:53.631+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:24:53.872 INFO:journalctl@ceph.osd.4.smithi155.stdout:Aug 31 06:24:53 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-4[33944]: 2024-08-31T06:24:53.545+0000 7fc8b1e6f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T06:24:53.872 INFO:journalctl@ceph.osd.4.smithi155.stdout:Aug 31 06:24:53 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-4[33944]: 2024-08-31T06:24:53.545+0000 7fc8b1e6f640 -1 osd.4 106 *** Got signal Terminated *** 2024-08-31T06:24:53.873 INFO:journalctl@ceph.osd.4.smithi155.stdout:Aug 31 06:24:53 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-4[33944]: 2024-08-31T06:24:53.545+0000 7fc8b1e6f640 -1 osd.4 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-31T06:24:53.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:24:53 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:24:53.659+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:24:53.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:24:53 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:24:53.659+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:24:53.913 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:24:53 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:24:53.781+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:21.798609+0000 front 2024-08-31T06:24:21.798847+0000 (oldest deadline 2024-08-31T06:24:45.298817+0000) 2024-08-31T06:24:53.972 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:24:53 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5[34971]: 2024-08-31T06:24:53.641+0000 7f8af3539640 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.836260+0000 front 2024-08-31T06:24:22.836471+0000 (oldest deadline 2024-08-31T06:24:46.336170+0000) 2024-08-31T06:24:53.973 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:24:53 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:24:53.840+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:24:54.472 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:24:54 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:24:54.197+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:24:54.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:24:54 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:24:54.197+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.670468+0000 front 2024-08-31T06:24:30.670483+0000 (oldest deadline 2024-08-31T06:24:53.570276+0000) 2024-08-31T06:24:54.872 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:24:54 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:24:54.596+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:24:54.872 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:24:54 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:24:54.596+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:24:54.973 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:24:54 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5[34971]: 2024-08-31T06:24:54.640+0000 7f8af3539640 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.836260+0000 front 2024-08-31T06:24:22.836471+0000 (oldest deadline 2024-08-31T06:24:46.336170+0000) 2024-08-31T06:24:54.973 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:24:54 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:24:54.879+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:24:54.973 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:24:54 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:24:54.879+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:24:55.163 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:24:54 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:24:54.784+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:21.798609+0000 front 2024-08-31T06:24:21.798847+0000 (oldest deadline 2024-08-31T06:24:45.298817+0000) 2024-08-31T06:24:55.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:24:54 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:24:54.662+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:24:55.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:24:54 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:24:54.662+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:24:55.372 INFO:journalctl@ceph.osd.4.smithi155.stdout:Aug 31 06:24:55 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-4[33944]: 2024-08-31T06:24:55.018+0000 7fc8aa452640 -1 osd.4 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.166909+0000 front 2024-08-31T06:24:30.166953+0000 (oldest deadline 2024-08-31T06:24:54.267087+0000) 2024-08-31T06:24:55.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:24:55 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:24:55.150+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:24:55.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:24:55 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:24:55.150+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.670468+0000 front 2024-08-31T06:24:30.670483+0000 (oldest deadline 2024-08-31T06:24:53.570276+0000) 2024-08-31T06:24:55.872 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:24:55 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:24:55.566+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:24:55.872 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:24:55 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:24:55.566+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:24:55.900 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:24:55 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5[34971]: 2024-08-31T06:24:55.628+0000 7f8af3539640 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.836260+0000 front 2024-08-31T06:24:22.836471+0000 (oldest deadline 2024-08-31T06:24:46.336170+0000) 2024-08-31T06:24:56.162 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:24:55 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:24:55.807+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:21.798609+0000 front 2024-08-31T06:24:21.798847+0000 (oldest deadline 2024-08-31T06:24:45.298817+0000) 2024-08-31T06:24:56.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:24:55 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:24:55.662+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:24:56.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:24:55 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:24:55.662+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:24:56.223 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:24:56 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:24:56.143+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:24:56.223 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:24:56 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:24:56.143+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.670468+0000 front 2024-08-31T06:24:30.670483+0000 (oldest deadline 2024-08-31T06:24:53.570276+0000) 2024-08-31T06:24:56.223 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:24:55 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:24:55.898+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:24:56.224 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:24:55 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:24:55.898+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:24:56.372 INFO:journalctl@ceph.osd.4.smithi155.stdout:Aug 31 06:24:55 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-4[33944]: 2024-08-31T06:24:55.986+0000 7fc8aa452640 -1 osd.4 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.166909+0000 front 2024-08-31T06:24:30.166953+0000 (oldest deadline 2024-08-31T06:24:54.267087+0000) 2024-08-31T06:24:56.838 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:24:56 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:24:56.686+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:24:56.838 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:24:56 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:24:56.686+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:24:56.872 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:24:56 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:24:56.566+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:24:56.872 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:24:56 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:24:56.566+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:24:56.952 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:24:56 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5[34971]: 2024-08-31T06:24:56.672+0000 7f8af3539640 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.836260+0000 front 2024-08-31T06:24:22.836471+0000 (oldest deadline 2024-08-31T06:24:46.336170+0000) 2024-08-31T06:24:57.163 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:24:56 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:24:56.835+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:21.798609+0000 front 2024-08-31T06:24:21.798847+0000 (oldest deadline 2024-08-31T06:24:45.298817+0000) 2024-08-31T06:24:57.222 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:24:57 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:24:57.125+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:24:57.223 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:24:57 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:24:57.125+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.670468+0000 front 2024-08-31T06:24:30.670483+0000 (oldest deadline 2024-08-31T06:24:53.570276+0000) 2024-08-31T06:24:57.223 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:24:56 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:24:56.948+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:24:57.223 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:24:56 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:24:56.948+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:24:57.372 INFO:journalctl@ceph.osd.4.smithi155.stdout:Aug 31 06:24:56 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-4[33944]: 2024-08-31T06:24:56.967+0000 7fc8aa452640 -1 osd.4 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.166909+0000 front 2024-08-31T06:24:30.166953+0000 (oldest deadline 2024-08-31T06:24:54.267087+0000) 2024-08-31T06:24:57.872 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:24:57 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:24:57.531+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:24:57.872 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:24:57 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:24:57.531+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:24:57.912 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:24:57 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:24:57.813+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:21.798609+0000 front 2024-08-31T06:24:21.798847+0000 (oldest deadline 2024-08-31T06:24:45.298817+0000) 2024-08-31T06:24:57.913 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:24:57 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:24:57.813+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.300670+0000 front 2024-08-31T06:24:32.300510+0000 (oldest deadline 2024-08-31T06:24:57.000455+0000) 2024-08-31T06:24:57.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:24:57 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:24:57.655+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:24:57.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:24:57 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:24:57.655+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:24:57.973 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:24:57 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5[34971]: 2024-08-31T06:24:57.705+0000 7f8af3539640 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.836260+0000 front 2024-08-31T06:24:22.836471+0000 (oldest deadline 2024-08-31T06:24:46.336170+0000) 2024-08-31T06:24:57.973 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:24:57 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:24:57.910+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:24:57.973 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:24:57 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:24:57.910+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:24:58.372 INFO:journalctl@ceph.osd.4.smithi155.stdout:Aug 31 06:24:57 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-4[33944]: 2024-08-31T06:24:57.996+0000 7fc8aa452640 -1 osd.4 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.166909+0000 front 2024-08-31T06:24:30.166953+0000 (oldest deadline 2024-08-31T06:24:54.267087+0000) 2024-08-31T06:24:58.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:24:58 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:24:58.148+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:24:58.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:24:58 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:24:58.148+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.670468+0000 front 2024-08-31T06:24:30.670483+0000 (oldest deadline 2024-08-31T06:24:53.570276+0000) 2024-08-31T06:24:58.872 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:24:58 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:24:58.744+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.099444+0000 front 2024-08-31T06:24:32.099224+0000 (oldest deadline 2024-08-31T06:24:57.999327+0000) 2024-08-31T06:24:58.872 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:24:58 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:24:58.559+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:24:58.872 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:24:58 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:24:58.559+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:24:58.913 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:24:58 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:24:58.822+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:21.798609+0000 front 2024-08-31T06:24:21.798847+0000 (oldest deadline 2024-08-31T06:24:45.298817+0000) 2024-08-31T06:24:58.914 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:24:58 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:24:58.822+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.300670+0000 front 2024-08-31T06:24:32.300510+0000 (oldest deadline 2024-08-31T06:24:57.000455+0000) 2024-08-31T06:24:58.914 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:24:58 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:24:58.621+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:24:58.914 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:24:58 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:24:58.621+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:24:59.116 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:24:58 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5[34971]: 2024-08-31T06:24:58.750+0000 7f8af3539640 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.836260+0000 front 2024-08-31T06:24:22.836471+0000 (oldest deadline 2024-08-31T06:24:46.336170+0000) 2024-08-31T06:24:59.116 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:24:58 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5[34971]: 2024-08-31T06:24:58.750+0000 7f8af3539640 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:28.637459+0000 front 2024-08-31T06:24:28.637574+0000 (oldest deadline 2024-08-31T06:24:58.037946+0000) 2024-08-31T06:24:59.117 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:24:58 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:24:58.872+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:24:59.117 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:24:58 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:24:58.872+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:24:59.472 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:24:59 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:24:59.114+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:24:59.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:24:59 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:24:59.114+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.670468+0000 front 2024-08-31T06:24:30.670483+0000 (oldest deadline 2024-08-31T06:24:53.570276+0000) 2024-08-31T06:24:59.524 INFO:journalctl@ceph.osd.4.smithi155.stdout:Aug 31 06:24:59 smithi155 podman[59157]: 2024-08-31 06:24:59.239733972 +0000 UTC m=+5.883155578 container died c9fbde35ffdfc2c17c2d1168dbb001500d2ba8b9860ae5e3bd93d5bce94d6b01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-4, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-31T06:24:59.872 INFO:journalctl@ceph.osd.4.smithi155.stdout:Aug 31 06:24:59 smithi155 podman[59157]: 2024-08-31 06:24:59.629689772 +0000 UTC m=+6.273111360 container cleanup c9fbde35ffdfc2c17c2d1168dbb001500d2ba8b9860ae5e3bd93d5bce94d6b01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-4, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-08-31T06:24:59.873 INFO:journalctl@ceph.osd.4.smithi155.stdout:Aug 31 06:24:59 smithi155 bash[59157]: ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-4 2024-08-31T06:24:59.873 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:24:59 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:24:59.736+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.099444+0000 front 2024-08-31T06:24:32.099224+0000 (oldest deadline 2024-08-31T06:24:57.999327+0000) 2024-08-31T06:24:59.873 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:24:59 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:24:59.583+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:24:59.873 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:24:59 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:24:59.583+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:24:59.913 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:24:59 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:24:59.805+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:21.798609+0000 front 2024-08-31T06:24:21.798847+0000 (oldest deadline 2024-08-31T06:24:45.298817+0000) 2024-08-31T06:24:59.913 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:24:59 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:24:59.805+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.300670+0000 front 2024-08-31T06:24:32.300510+0000 (oldest deadline 2024-08-31T06:24:57.000455+0000) 2024-08-31T06:24:59.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:24:59 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:24:59.575+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:24:59.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:24:59 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:24:59.575+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:25:00.135 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:24:59 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5[34971]: 2024-08-31T06:24:59.771+0000 7f8af3539640 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.836260+0000 front 2024-08-31T06:24:22.836471+0000 (oldest deadline 2024-08-31T06:24:46.336170+0000) 2024-08-31T06:25:00.135 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:24:59 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5[34971]: 2024-08-31T06:24:59.771+0000 7f8af3539640 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:28.637459+0000 front 2024-08-31T06:24:28.637574+0000 (oldest deadline 2024-08-31T06:24:58.037946+0000) 2024-08-31T06:25:00.135 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:24:59 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:24:59.914+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:00.135 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:24:59 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:24:59.914+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:00.238 INFO:journalctl@ceph.osd.4.smithi155.stdout:Aug 31 06:24:59 smithi155 podman[59172]: 2024-08-31 06:24:59.88946474 +0000 UTC m=+0.648169040 container remove c9fbde35ffdfc2c17c2d1168dbb001500d2ba8b9860ae5e3bd93d5bce94d6b01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.1, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-31T06:25:00.472 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:00 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:00.133+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:25:00.472 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:00 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:00.133+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.670468+0000 front 2024-08-31T06:24:30.670483+0000 (oldest deadline 2024-08-31T06:24:53.570276+0000) 2024-08-31T06:25:00.544 INFO:journalctl@ceph.osd.4.smithi155.stdout:Aug 31 06:25:00 smithi155 podman[59257]: 2024-08-31 06:25:00.1393119 +0000 UTC m=+0.016198036 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T06:25:00.544 INFO:journalctl@ceph.osd.4.smithi155.stdout:Aug 31 06:25:00 smithi155 podman[59257]: 2024-08-31 06:25:00.282006167 +0000 UTC m=+0.158892291 container create 072071dbdaaf9a88c8c452f69ecfe3586cab3af67b4c7a32006b802daac586fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.1, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611) 2024-08-31T06:25:00.872 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:00 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:00.784+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.099444+0000 front 2024-08-31T06:24:32.099224+0000 (oldest deadline 2024-08-31T06:24:57.999327+0000) 2024-08-31T06:25:00.873 INFO:journalctl@ceph.osd.4.smithi155.stdout:Aug 31 06:25:00 smithi155 podman[59257]: 2024-08-31 06:25:00.542757544 +0000 UTC m=+0.419643667 container init 072071dbdaaf9a88c8c452f69ecfe3586cab3af67b4c7a32006b802daac586fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-4-deactivate, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.1, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-08-31T06:25:00.873 INFO:journalctl@ceph.osd.4.smithi155.stdout:Aug 31 06:25:00 smithi155 podman[59257]: 2024-08-31 06:25:00.546165977 +0000 UTC m=+0.423052109 container start 072071dbdaaf9a88c8c452f69ecfe3586cab3af67b4c7a32006b802daac586fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.1, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611) 2024-08-31T06:25:00.873 INFO:journalctl@ceph.osd.4.smithi155.stdout:Aug 31 06:25:00 smithi155 podman[59257]: 2024-08-31 06:25:00.626306246 +0000 UTC m=+0.503192370 container attach 072071dbdaaf9a88c8c452f69ecfe3586cab3af67b4c7a32006b802daac586fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-4-deactivate, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.vendor=CentOS, io.buildah.version=1.37.1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-08-31T06:25:00.873 INFO:journalctl@ceph.osd.4.smithi155.stdout:Aug 31 06:25:00 smithi155 podman[59257]: 2024-08-31 06:25:00.742819132 +0000 UTC m=+0.619705268 container died 072071dbdaaf9a88c8c452f69ecfe3586cab3af67b4c7a32006b802daac586fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.1, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, CEPH_POINT_RELEASE=) 2024-08-31T06:25:00.873 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:00 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:00.581+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:00.873 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:00 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:00.581+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:00.913 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:00 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:00.836+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:21.798609+0000 front 2024-08-31T06:24:21.798847+0000 (oldest deadline 2024-08-31T06:24:45.298817+0000) 2024-08-31T06:25:00.913 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:00 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:00.836+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.300670+0000 front 2024-08-31T06:24:32.300510+0000 (oldest deadline 2024-08-31T06:24:57.000455+0000) 2024-08-31T06:25:00.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:00 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:00.591+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:25:00.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:00 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:00.591+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:25:01.142 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:25:00 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5[34971]: 2024-08-31T06:25:00.811+0000 7f8af3539640 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.836260+0000 front 2024-08-31T06:24:22.836471+0000 (oldest deadline 2024-08-31T06:24:46.336170+0000) 2024-08-31T06:25:01.142 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:25:00 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5[34971]: 2024-08-31T06:25:00.811+0000 7f8af3539640 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:28.637459+0000 front 2024-08-31T06:24:28.637574+0000 (oldest deadline 2024-08-31T06:24:58.037946+0000) 2024-08-31T06:25:01.143 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:00 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:00.908+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:01.143 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:00 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:00.908+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:01.359 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@osd.4.service' 2024-08-31T06:25:01.399 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T06:25:01.399 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-08-31T06:25:01.400 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-08-31T06:25:01.400 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@osd.5 2024-08-31T06:25:01.429 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:01 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:01.140+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:25:01.429 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:01 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:01.140+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.670468+0000 front 2024-08-31T06:24:30.670483+0000 (oldest deadline 2024-08-31T06:24:53.570276+0000) 2024-08-31T06:25:01.723 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:25:01 smithi184 systemd[1]: Stopping Ceph osd.5 for 673e4aa0-675d-11ef-bcd4-c7b262605968... 2024-08-31T06:25:01.839 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:01 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:01.544+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:25:01.839 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:01 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:01.544+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:25:01.872 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:01 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:01.800+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.099444+0000 front 2024-08-31T06:24:32.099224+0000 (oldest deadline 2024-08-31T06:24:57.999327+0000) 2024-08-31T06:25:01.872 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:01 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:01.616+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:01.872 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:01 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:01.616+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:02.130 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:01 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:01.894+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:02.130 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:01 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:01.894+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:02.130 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:25:01 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5[34971]: 2024-08-31T06:25:01.726+0000 7f8af6f4e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T06:25:02.130 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:25:01 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5[34971]: 2024-08-31T06:25:01.726+0000 7f8af6f4e640 -1 osd.5 106 *** Got signal Terminated *** 2024-08-31T06:25:02.131 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:25:01 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5[34971]: 2024-08-31T06:25:01.726+0000 7f8af6f4e640 -1 osd.5 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-31T06:25:02.131 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:25:01 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5[34971]: 2024-08-31T06:25:01.852+0000 7f8af3539640 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.836260+0000 front 2024-08-31T06:24:22.836471+0000 (oldest deadline 2024-08-31T06:24:46.336170+0000) 2024-08-31T06:25:02.131 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:25:01 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5[34971]: 2024-08-31T06:25:01.852+0000 7f8af3539640 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:28.637459+0000 front 2024-08-31T06:24:28.637574+0000 (oldest deadline 2024-08-31T06:24:58.037946+0000) 2024-08-31T06:25:02.163 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:01 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:01.837+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:21.798609+0000 front 2024-08-31T06:24:21.798847+0000 (oldest deadline 2024-08-31T06:24:45.298817+0000) 2024-08-31T06:25:02.163 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:01 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:01.837+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.300670+0000 front 2024-08-31T06:24:32.300510+0000 (oldest deadline 2024-08-31T06:24:57.000455+0000) 2024-08-31T06:25:02.472 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:02 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:02.128+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:25:02.472 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:02 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:02.128+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.670468+0000 front 2024-08-31T06:24:30.670483+0000 (oldest deadline 2024-08-31T06:24:53.570276+0000) 2024-08-31T06:25:02.872 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:02 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:02.824+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.099444+0000 front 2024-08-31T06:24:32.099224+0000 (oldest deadline 2024-08-31T06:24:57.999327+0000) 2024-08-31T06:25:02.872 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:02 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:02.824+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:38.500433+0000 front 2024-08-31T06:24:38.500454+0000 (oldest deadline 2024-08-31T06:25:02.000525+0000) 2024-08-31T06:25:02.872 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:02 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:02.619+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:02.872 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:02 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:02.619+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:02.913 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:02 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:02.796+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:21.798609+0000 front 2024-08-31T06:24:21.798847+0000 (oldest deadline 2024-08-31T06:24:45.298817+0000) 2024-08-31T06:25:02.913 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:02 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:02.796+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.300670+0000 front 2024-08-31T06:24:32.300510+0000 (oldest deadline 2024-08-31T06:24:57.000455+0000) 2024-08-31T06:25:02.913 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:02 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:02.796+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:40.903301+0000 front 2024-08-31T06:24:40.903322+0000 (oldest deadline 2024-08-31T06:25:02.603246+0000) 2024-08-31T06:25:02.914 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:02 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:02.591+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:25:02.914 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:02 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:02.591+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:25:02.914 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:02 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:02.591+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:35.901359+0000 front 2024-08-31T06:24:35.901304+0000 (oldest deadline 2024-08-31T06:25:02.302188+0000) 2024-08-31T06:25:03.150 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:02 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:02.931+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:03.150 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:02 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:02.931+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:03.150 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:25:02 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5[34971]: 2024-08-31T06:25:02.868+0000 7f8af3539640 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.836260+0000 front 2024-08-31T06:24:22.836471+0000 (oldest deadline 2024-08-31T06:24:46.336170+0000) 2024-08-31T06:25:03.151 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:25:02 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5[34971]: 2024-08-31T06:25:02.868+0000 7f8af3539640 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:28.637459+0000 front 2024-08-31T06:24:28.637574+0000 (oldest deadline 2024-08-31T06:24:58.037946+0000) 2024-08-31T06:25:03.472 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:03 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:03.148+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:25:03.472 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:03 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:03.148+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.670468+0000 front 2024-08-31T06:24:30.670483+0000 (oldest deadline 2024-08-31T06:24:53.570276+0000) 2024-08-31T06:25:03.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:03 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:03.148+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:37.071304+0000 front 2024-08-31T06:24:37.071497+0000 (oldest deadline 2024-08-31T06:25:02.971434+0000) 2024-08-31T06:25:03.870 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:03 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:03.618+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:03.870 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:03 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:03.618+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:03.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:03 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:03.550+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:25:03.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:03 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:03.550+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:25:03.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:03 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:03.550+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:35.901359+0000 front 2024-08-31T06:24:35.901304+0000 (oldest deadline 2024-08-31T06:25:02.302188+0000) 2024-08-31T06:25:03.913 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:03 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:03.756+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:21.798609+0000 front 2024-08-31T06:24:21.798847+0000 (oldest deadline 2024-08-31T06:24:45.298817+0000) 2024-08-31T06:25:03.913 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:03 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:03.756+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.300670+0000 front 2024-08-31T06:24:32.300510+0000 (oldest deadline 2024-08-31T06:24:57.000455+0000) 2024-08-31T06:25:03.914 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:03 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:03.756+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:40.903301+0000 front 2024-08-31T06:24:40.903322+0000 (oldest deadline 2024-08-31T06:25:02.603246+0000) 2024-08-31T06:25:04.122 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:03 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:03.868+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.099444+0000 front 2024-08-31T06:24:32.099224+0000 (oldest deadline 2024-08-31T06:24:57.999327+0000) 2024-08-31T06:25:04.122 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:03 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:03.868+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:38.500433+0000 front 2024-08-31T06:24:38.500454+0000 (oldest deadline 2024-08-31T06:25:02.000525+0000) 2024-08-31T06:25:04.222 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:04 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:04.130+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:25:04.223 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:04 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:04.130+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.670468+0000 front 2024-08-31T06:24:30.670483+0000 (oldest deadline 2024-08-31T06:24:53.570276+0000) 2024-08-31T06:25:04.223 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:04 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:04.130+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:37.071304+0000 front 2024-08-31T06:24:37.071497+0000 (oldest deadline 2024-08-31T06:25:02.971434+0000) 2024-08-31T06:25:04.223 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:03 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:03.946+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:04.223 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:03 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:03.946+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:04.223 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:25:03 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5[34971]: 2024-08-31T06:25:03.890+0000 7f8af3539640 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.836260+0000 front 2024-08-31T06:24:22.836471+0000 (oldest deadline 2024-08-31T06:24:46.336170+0000) 2024-08-31T06:25:04.223 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:25:03 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5[34971]: 2024-08-31T06:25:03.890+0000 7f8af3539640 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:28.637459+0000 front 2024-08-31T06:24:28.637574+0000 (oldest deadline 2024-08-31T06:24:58.037946+0000) 2024-08-31T06:25:04.224 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:25:03 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5[34971]: 2024-08-31T06:25:03.890+0000 7f8af3539640 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:38.038386+0000 front 2024-08-31T06:24:38.038249+0000 (oldest deadline 2024-08-31T06:25:03.338457+0000) 2024-08-31T06:25:04.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:04 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:04.590+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:25:04.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:04 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:04.590+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:25:04.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:04 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:04.590+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:35.901359+0000 front 2024-08-31T06:24:35.901304+0000 (oldest deadline 2024-08-31T06:25:02.302188+0000) 2024-08-31T06:25:04.913 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:04 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:04.750+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:21.798609+0000 front 2024-08-31T06:24:21.798847+0000 (oldest deadline 2024-08-31T06:24:45.298817+0000) 2024-08-31T06:25:04.913 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:04 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:04.750+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.300670+0000 front 2024-08-31T06:24:32.300510+0000 (oldest deadline 2024-08-31T06:24:57.000455+0000) 2024-08-31T06:25:04.913 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:04 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:04.750+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:40.903301+0000 front 2024-08-31T06:24:40.903322+0000 (oldest deadline 2024-08-31T06:25:02.603246+0000) 2024-08-31T06:25:05.122 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:04 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:04.841+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.099444+0000 front 2024-08-31T06:24:32.099224+0000 (oldest deadline 2024-08-31T06:24:57.999327+0000) 2024-08-31T06:25:05.124 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:04 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:04.841+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:38.500433+0000 front 2024-08-31T06:24:38.500454+0000 (oldest deadline 2024-08-31T06:25:02.000525+0000) 2024-08-31T06:25:05.124 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:04 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:04.631+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:05.124 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:04 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:04.631+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:05.222 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:05 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:05.098+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:25:05.222 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:05 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:05.098+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.670468+0000 front 2024-08-31T06:24:30.670483+0000 (oldest deadline 2024-08-31T06:24:53.570276+0000) 2024-08-31T06:25:05.223 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:05 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:05.098+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:37.071304+0000 front 2024-08-31T06:24:37.071497+0000 (oldest deadline 2024-08-31T06:25:02.971434+0000) 2024-08-31T06:25:05.223 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:04 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:04.960+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:05.223 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:04 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:04.960+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:05.223 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:25:04 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5[34971]: 2024-08-31T06:25:04.902+0000 7f8af3539640 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.836260+0000 front 2024-08-31T06:24:22.836471+0000 (oldest deadline 2024-08-31T06:24:46.336170+0000) 2024-08-31T06:25:05.223 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:25:04 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5[34971]: 2024-08-31T06:25:04.902+0000 7f8af3539640 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:28.637459+0000 front 2024-08-31T06:24:28.637574+0000 (oldest deadline 2024-08-31T06:24:58.037946+0000) 2024-08-31T06:25:05.223 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:25:04 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5[34971]: 2024-08-31T06:25:04.902+0000 7f8af3539640 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:38.038386+0000 front 2024-08-31T06:24:38.038249+0000 (oldest deadline 2024-08-31T06:25:03.338457+0000) 2024-08-31T06:25:05.864 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:05 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:05.585+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:05.865 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:05 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:05.585+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:05.865 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:05 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:05.585+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.499981+0000 front 2024-08-31T06:24:39.500178+0000 (oldest deadline 2024-08-31T06:25:04.799749+0000) 2024-08-31T06:25:05.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:05 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:05.627+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:25:05.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:05 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:05.627+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:25:05.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:05 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:05.627+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:35.901359+0000 front 2024-08-31T06:24:35.901304+0000 (oldest deadline 2024-08-31T06:25:02.302188+0000) 2024-08-31T06:25:05.913 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:05 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:05.780+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:21.798609+0000 front 2024-08-31T06:24:21.798847+0000 (oldest deadline 2024-08-31T06:24:45.298817+0000) 2024-08-31T06:25:05.913 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:05 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:05.780+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.300670+0000 front 2024-08-31T06:24:32.300510+0000 (oldest deadline 2024-08-31T06:24:57.000455+0000) 2024-08-31T06:25:05.913 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:05 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:05.780+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:40.903301+0000 front 2024-08-31T06:24:40.903322+0000 (oldest deadline 2024-08-31T06:25:02.603246+0000) 2024-08-31T06:25:06.122 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:05 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:05.862+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.099444+0000 front 2024-08-31T06:24:32.099224+0000 (oldest deadline 2024-08-31T06:24:57.999327+0000) 2024-08-31T06:25:06.122 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:05 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:05.862+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:38.500433+0000 front 2024-08-31T06:24:38.500454+0000 (oldest deadline 2024-08-31T06:25:02.000525+0000) 2024-08-31T06:25:06.222 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:06 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:06.001+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:06.223 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:06 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:06.001+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:06.223 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:25:05 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5[34971]: 2024-08-31T06:25:05.930+0000 7f8af3539640 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.836260+0000 front 2024-08-31T06:24:22.836471+0000 (oldest deadline 2024-08-31T06:24:46.336170+0000) 2024-08-31T06:25:06.223 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:25:05 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5[34971]: 2024-08-31T06:25:05.930+0000 7f8af3539640 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:28.637459+0000 front 2024-08-31T06:24:28.637574+0000 (oldest deadline 2024-08-31T06:24:58.037946+0000) 2024-08-31T06:25:06.223 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:25:05 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5[34971]: 2024-08-31T06:25:05.930+0000 7f8af3539640 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:38.038386+0000 front 2024-08-31T06:24:38.038249+0000 (oldest deadline 2024-08-31T06:25:03.338457+0000) 2024-08-31T06:25:06.223 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:06 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:06.069+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:25:06.223 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:06 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:06.069+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.670468+0000 front 2024-08-31T06:24:30.670483+0000 (oldest deadline 2024-08-31T06:24:53.570276+0000) 2024-08-31T06:25:06.223 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:06 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:06.069+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:37.071304+0000 front 2024-08-31T06:24:37.071497+0000 (oldest deadline 2024-08-31T06:25:02.971434+0000) 2024-08-31T06:25:06.831 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:06 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:06.544+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:06.832 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:06 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:06.544+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:06.832 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:06 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:06.544+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.499981+0000 front 2024-08-31T06:24:39.500178+0000 (oldest deadline 2024-08-31T06:25:04.799749+0000) 2024-08-31T06:25:07.122 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:06 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:06.828+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.099444+0000 front 2024-08-31T06:24:32.099224+0000 (oldest deadline 2024-08-31T06:24:57.999327+0000) 2024-08-31T06:25:07.122 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:06 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:06.828+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:38.500433+0000 front 2024-08-31T06:24:38.500454+0000 (oldest deadline 2024-08-31T06:25:02.000525+0000) 2024-08-31T06:25:07.165 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:06 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:06.730+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:21.798609+0000 front 2024-08-31T06:24:21.798847+0000 (oldest deadline 2024-08-31T06:24:45.298817+0000) 2024-08-31T06:25:07.166 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:06 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:06.730+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.300670+0000 front 2024-08-31T06:24:32.300510+0000 (oldest deadline 2024-08-31T06:24:57.000455+0000) 2024-08-31T06:25:07.166 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:06 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:06.730+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:40.903301+0000 front 2024-08-31T06:24:40.903322+0000 (oldest deadline 2024-08-31T06:25:02.603246+0000) 2024-08-31T06:25:07.166 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:06 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:06.662+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:25:07.166 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:06 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:06.662+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:25:07.166 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:06 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:06.662+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:35.901359+0000 front 2024-08-31T06:24:35.901304+0000 (oldest deadline 2024-08-31T06:25:02.302188+0000) 2024-08-31T06:25:07.472 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:25:07 smithi184 podman[60307]: 2024-08-31 06:25:07.271492869 +0000 UTC m=+5.669512467 container died e21ad848367edee784f2d14acd37cc0e11205dc13589ebd7d1a26cb8a8dc88f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2, io.buildah.version=1.37.1, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-31T06:25:07.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:07 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:07.105+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:25:07.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:07 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:07.105+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.670468+0000 front 2024-08-31T06:24:30.670483+0000 (oldest deadline 2024-08-31T06:24:53.570276+0000) 2024-08-31T06:25:07.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:07 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:07.105+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:37.071304+0000 front 2024-08-31T06:24:37.071497+0000 (oldest deadline 2024-08-31T06:25:02.971434+0000) 2024-08-31T06:25:07.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:07 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:07.023+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:07.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:07 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:07.023+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:07.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:07 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:07.023+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:07.871 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:25:07 smithi184 podman[60307]: 2024-08-31 06:25:07.668276663 +0000 UTC m=+6.066296248 container cleanup e21ad848367edee784f2d14acd37cc0e11205dc13589ebd7d1a26cb8a8dc88f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.1, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-08-31T06:25:07.872 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:25:07 smithi184 bash[60307]: ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5 2024-08-31T06:25:07.872 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:07 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:07.530+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:07.872 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:07 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:07.530+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:07.872 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:07 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:07.530+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.499981+0000 front 2024-08-31T06:24:39.500178+0000 (oldest deadline 2024-08-31T06:25:04.799749+0000) 2024-08-31T06:25:08.122 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:07 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:07.869+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.099444+0000 front 2024-08-31T06:24:32.099224+0000 (oldest deadline 2024-08-31T06:24:57.999327+0000) 2024-08-31T06:25:08.122 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:07 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:07.869+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:38.500433+0000 front 2024-08-31T06:24:38.500454+0000 (oldest deadline 2024-08-31T06:25:02.000525+0000) 2024-08-31T06:25:08.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:07 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:07.712+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:25:08.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:07 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:07.712+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:25:08.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:07 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:07.712+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:35.901359+0000 front 2024-08-31T06:24:35.901304+0000 (oldest deadline 2024-08-31T06:25:02.302188+0000) 2024-08-31T06:25:08.163 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:07 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:07.731+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:21.798609+0000 front 2024-08-31T06:24:21.798847+0000 (oldest deadline 2024-08-31T06:24:45.298817+0000) 2024-08-31T06:25:08.163 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:07 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:07.731+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.300670+0000 front 2024-08-31T06:24:32.300510+0000 (oldest deadline 2024-08-31T06:24:57.000455+0000) 2024-08-31T06:25:08.163 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:07 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:07.731+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:40.903301+0000 front 2024-08-31T06:24:40.903322+0000 (oldest deadline 2024-08-31T06:25:02.603246+0000) 2024-08-31T06:25:08.223 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:25:07 smithi184 podman[60325]: 2024-08-31 06:25:07.869979315 +0000 UTC m=+0.593396020 container remove e21ad848367edee784f2d14acd37cc0e11205dc13589ebd7d1a26cb8a8dc88f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-08-31T06:25:08.223 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:07 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:07.991+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:08.223 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:07 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:07.991+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:08.223 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:07 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:07.991+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:08.223 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:08 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:08.105+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:25:08.223 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:08 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:08.105+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.670468+0000 front 2024-08-31T06:24:30.670483+0000 (oldest deadline 2024-08-31T06:24:53.570276+0000) 2024-08-31T06:25:08.224 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:08 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:08.105+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:37.071304+0000 front 2024-08-31T06:24:37.071497+0000 (oldest deadline 2024-08-31T06:25:02.971434+0000) 2024-08-31T06:25:08.584 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:25:08 smithi184 podman[60410]: 2024-08-31 06:25:08.177321747 +0000 UTC m=+0.016201157 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T06:25:08.584 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:25:08 smithi184 podman[60410]: 2024-08-31 06:25:08.313061268 +0000 UTC m=+0.151940672 container create b0ba7ae882c5d2263f5b9ddfec2077c3f90d966d6679569285d0e8f40c65eac7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.1, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.build-date=20240731, ceph=True) 2024-08-31T06:25:08.584 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:25:08 smithi184 podman[60410]: 2024-08-31 06:25:08.582506013 +0000 UTC m=+0.421385418 container init b0ba7ae882c5d2263f5b9ddfec2077c3f90d966d6679569285d0e8f40c65eac7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-08-31T06:25:08.872 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:08 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:08.567+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:08.872 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:08 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:08.567+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:08.872 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:08 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:08.567+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.499981+0000 front 2024-08-31T06:24:39.500178+0000 (oldest deadline 2024-08-31T06:25:04.799749+0000) 2024-08-31T06:25:08.967 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:25:08 smithi184 podman[60410]: 2024-08-31 06:25:08.586189164 +0000 UTC m=+0.425068570 container start b0ba7ae882c5d2263f5b9ddfec2077c3f90d966d6679569285d0e8f40c65eac7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.1, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-08-31T06:25:08.967 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:25:08 smithi184 podman[60410]: 2024-08-31 06:25:08.68253372 +0000 UTC m=+0.521413135 container attach b0ba7ae882c5d2263f5b9ddfec2077c3f90d966d6679569285d0e8f40c65eac7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.1, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611) 2024-08-31T06:25:08.967 INFO:journalctl@ceph.osd.5.smithi184.stdout:Aug 31 06:25:08 smithi184 podman[60410]: 2024-08-31 06:25:08.779234828 +0000 UTC m=+0.618114234 container died b0ba7ae882c5d2263f5b9ddfec2077c3f90d966d6679569285d0e8f40c65eac7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.1) 2024-08-31T06:25:09.163 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:08 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:08.700+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:21.798609+0000 front 2024-08-31T06:24:21.798847+0000 (oldest deadline 2024-08-31T06:24:45.298817+0000) 2024-08-31T06:25:09.163 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:08 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:08.700+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.300670+0000 front 2024-08-31T06:24:32.300510+0000 (oldest deadline 2024-08-31T06:24:57.000455+0000) 2024-08-31T06:25:09.163 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:08 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:08.700+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:40.903301+0000 front 2024-08-31T06:24:40.903322+0000 (oldest deadline 2024-08-31T06:25:02.603246+0000) 2024-08-31T06:25:09.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:08 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:08.714+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:25:09.164 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:08 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:08.714+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:25:09.164 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:08 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:08.714+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:35.901359+0000 front 2024-08-31T06:24:35.901304+0000 (oldest deadline 2024-08-31T06:25:02.302188+0000) 2024-08-31T06:25:09.222 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:09 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:09.148+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:25:09.223 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:09 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:09.148+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.670468+0000 front 2024-08-31T06:24:30.670483+0000 (oldest deadline 2024-08-31T06:24:53.570276+0000) 2024-08-31T06:25:09.223 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:09 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:09.148+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:37.071304+0000 front 2024-08-31T06:24:37.071497+0000 (oldest deadline 2024-08-31T06:25:02.971434+0000) 2024-08-31T06:25:09.223 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:08 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:08.965+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:09.223 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:08 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:08.965+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:09.223 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:08 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:08.965+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:09.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:08 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:08.877+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.099444+0000 front 2024-08-31T06:24:32.099224+0000 (oldest deadline 2024-08-31T06:24:57.999327+0000) 2024-08-31T06:25:09.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:08 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:08.877+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:38.500433+0000 front 2024-08-31T06:24:38.500454+0000 (oldest deadline 2024-08-31T06:25:02.000525+0000) 2024-08-31T06:25:09.435 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@osd.5.service' 2024-08-31T06:25:09.520 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T06:25:09.520 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-08-31T06:25:09.520 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-08-31T06:25:09.520 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@osd.6 2024-08-31T06:25:09.872 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:09 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:09.599+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:09.872 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:09 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:09.599+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:09.872 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:09 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:09.599+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.499981+0000 front 2024-08-31T06:24:39.500178+0000 (oldest deadline 2024-08-31T06:25:04.799749+0000) 2024-08-31T06:25:09.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:09 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:09.700+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:25:09.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:09 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:09.700+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:25:09.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:09 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:09.700+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:35.901359+0000 front 2024-08-31T06:24:35.901304+0000 (oldest deadline 2024-08-31T06:25:02.302188+0000) 2024-08-31T06:25:09.913 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:09 smithi016 systemd[1]: Stopping Ceph osd.6 for 673e4aa0-675d-11ef-bcd4-c7b262605968... 2024-08-31T06:25:09.914 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:09 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:09.745+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:21.798609+0000 front 2024-08-31T06:24:21.798847+0000 (oldest deadline 2024-08-31T06:24:45.298817+0000) 2024-08-31T06:25:09.914 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:09 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:09.745+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.300670+0000 front 2024-08-31T06:24:32.300510+0000 (oldest deadline 2024-08-31T06:24:57.000455+0000) 2024-08-31T06:25:09.914 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:09 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:09.745+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:40.903301+0000 front 2024-08-31T06:24:40.903322+0000 (oldest deadline 2024-08-31T06:25:02.603246+0000) 2024-08-31T06:25:10.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:09 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:09.900+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.099444+0000 front 2024-08-31T06:24:32.099224+0000 (oldest deadline 2024-08-31T06:24:57.999327+0000) 2024-08-31T06:25:10.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:09 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:09.900+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:38.500433+0000 front 2024-08-31T06:24:38.500454+0000 (oldest deadline 2024-08-31T06:25:02.000525+0000) 2024-08-31T06:25:10.413 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:09 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:09.953+0000 7f41a1600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T06:25:10.413 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:09 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:09.953+0000 7f41a1600640 -1 osd.6 106 *** Got signal Terminated *** 2024-08-31T06:25:10.413 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:09 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:09.953+0000 7f41a1600640 -1 osd.6 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-31T06:25:10.472 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:10 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:10.119+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:25:10.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:10 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:10.119+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.670468+0000 front 2024-08-31T06:24:30.670483+0000 (oldest deadline 2024-08-31T06:24:53.570276+0000) 2024-08-31T06:25:10.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:10 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:10.119+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:37.071304+0000 front 2024-08-31T06:24:37.071497+0000 (oldest deadline 2024-08-31T06:25:02.971434+0000) 2024-08-31T06:25:10.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:10 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:10.011+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:10.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:10 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:10.011+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:10.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:10 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:10.011+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:10.872 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:10 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:10.611+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:10.872 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:10 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:10.611+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:10.872 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:10 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:10.611+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.499981+0000 front 2024-08-31T06:24:39.500178+0000 (oldest deadline 2024-08-31T06:25:04.799749+0000) 2024-08-31T06:25:10.913 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:10 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:10.783+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:21.798609+0000 front 2024-08-31T06:24:21.798847+0000 (oldest deadline 2024-08-31T06:24:45.298817+0000) 2024-08-31T06:25:10.913 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:10 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:10.783+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.300670+0000 front 2024-08-31T06:24:32.300510+0000 (oldest deadline 2024-08-31T06:24:57.000455+0000) 2024-08-31T06:25:10.913 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:10 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:10.783+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:40.903301+0000 front 2024-08-31T06:24:40.903322+0000 (oldest deadline 2024-08-31T06:25:02.603246+0000) 2024-08-31T06:25:10.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:10 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:10.657+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:25:10.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:10 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:10.657+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:25:10.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:10 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:10.657+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:35.901359+0000 front 2024-08-31T06:24:35.901304+0000 (oldest deadline 2024-08-31T06:25:02.302188+0000) 2024-08-31T06:25:11.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:10 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:10.928+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.099444+0000 front 2024-08-31T06:24:32.099224+0000 (oldest deadline 2024-08-31T06:24:57.999327+0000) 2024-08-31T06:25:11.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:10 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:10.928+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:38.500433+0000 front 2024-08-31T06:24:38.500454+0000 (oldest deadline 2024-08-31T06:25:02.000525+0000) 2024-08-31T06:25:11.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:11 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:11.070+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:25:11.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:11 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:11.070+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.670468+0000 front 2024-08-31T06:24:30.670483+0000 (oldest deadline 2024-08-31T06:24:53.570276+0000) 2024-08-31T06:25:11.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:11 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:11.070+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:37.071304+0000 front 2024-08-31T06:24:37.071497+0000 (oldest deadline 2024-08-31T06:25:02.971434+0000) 2024-08-31T06:25:11.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:11 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:11.047+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:11.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:11 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:11.047+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:11.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:11 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:11.047+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:11.873 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:11 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:11.611+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:11.873 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:11 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:11.611+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:11.873 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:11 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:11.611+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.499981+0000 front 2024-08-31T06:24:39.500178+0000 (oldest deadline 2024-08-31T06:25:04.799749+0000) 2024-08-31T06:25:11.913 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:11 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:11.826+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:21.798609+0000 front 2024-08-31T06:24:21.798847+0000 (oldest deadline 2024-08-31T06:24:45.298817+0000) 2024-08-31T06:25:11.913 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:11 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:11.827+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.300670+0000 front 2024-08-31T06:24:32.300510+0000 (oldest deadline 2024-08-31T06:24:57.000455+0000) 2024-08-31T06:25:11.913 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:11 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:11.827+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:40.903301+0000 front 2024-08-31T06:24:40.903322+0000 (oldest deadline 2024-08-31T06:25:02.603246+0000) 2024-08-31T06:25:11.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:11 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:11.651+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:25:11.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:11 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:11.651+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:25:11.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:11 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:11.651+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:35.901359+0000 front 2024-08-31T06:24:35.901304+0000 (oldest deadline 2024-08-31T06:25:02.302188+0000) 2024-08-31T06:25:11.914 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:11 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:11.651+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.203189+0000 front 2024-08-31T06:24:48.203315+0000 (oldest deadline 2024-08-31T06:25:11.003818+0000) 2024-08-31T06:25:12.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:11 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:11.948+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.099444+0000 front 2024-08-31T06:24:32.099224+0000 (oldest deadline 2024-08-31T06:24:57.999327+0000) 2024-08-31T06:25:12.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:11 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:11.948+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:38.500433+0000 front 2024-08-31T06:24:38.500454+0000 (oldest deadline 2024-08-31T06:25:02.000525+0000) 2024-08-31T06:25:12.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:11 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:11.948+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:49.303805+0000 front 2024-08-31T06:24:49.303913+0000 (oldest deadline 2024-08-31T06:25:11.603806+0000) 2024-08-31T06:25:12.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:12 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:12.027+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:25:12.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:12 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:12.027+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.670468+0000 front 2024-08-31T06:24:30.670483+0000 (oldest deadline 2024-08-31T06:24:53.570276+0000) 2024-08-31T06:25:12.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:12 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:12.027+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:37.071304+0000 front 2024-08-31T06:24:37.071497+0000 (oldest deadline 2024-08-31T06:25:02.971434+0000) 2024-08-31T06:25:12.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:12 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:12.075+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:12.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:12 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:12.075+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:12.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:12 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:12.075+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:12.913 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:12 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:12.793+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:21.798609+0000 front 2024-08-31T06:24:21.798847+0000 (oldest deadline 2024-08-31T06:24:45.298817+0000) 2024-08-31T06:25:12.913 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:12 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:12.793+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.300670+0000 front 2024-08-31T06:24:32.300510+0000 (oldest deadline 2024-08-31T06:24:57.000455+0000) 2024-08-31T06:25:12.913 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:12 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:12.793+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:40.903301+0000 front 2024-08-31T06:24:40.903322+0000 (oldest deadline 2024-08-31T06:25:02.603246+0000) 2024-08-31T06:25:12.914 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:12 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:12.622+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:25:12.914 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:12 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:12.622+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:25:12.914 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:12 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:12.622+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:35.901359+0000 front 2024-08-31T06:24:35.901304+0000 (oldest deadline 2024-08-31T06:25:02.302188+0000) 2024-08-31T06:25:12.914 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:12 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:12.622+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.203189+0000 front 2024-08-31T06:24:48.203315+0000 (oldest deadline 2024-08-31T06:25:11.003818+0000) 2024-08-31T06:25:12.975 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:12 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:12.642+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:12.975 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:12 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:12.642+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:12.975 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:12 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:12.642+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.499981+0000 front 2024-08-31T06:24:39.500178+0000 (oldest deadline 2024-08-31T06:25:04.799749+0000) 2024-08-31T06:25:13.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:12 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:12.972+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.099444+0000 front 2024-08-31T06:24:32.099224+0000 (oldest deadline 2024-08-31T06:24:57.999327+0000) 2024-08-31T06:25:13.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:12 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:12.972+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:38.500433+0000 front 2024-08-31T06:24:38.500454+0000 (oldest deadline 2024-08-31T06:25:02.000525+0000) 2024-08-31T06:25:13.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:12 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:12.972+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:49.303805+0000 front 2024-08-31T06:24:49.303913+0000 (oldest deadline 2024-08-31T06:25:11.603806+0000) 2024-08-31T06:25:13.472 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:12 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:12.989+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:25:13.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:12 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:12.989+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.670468+0000 front 2024-08-31T06:24:30.670483+0000 (oldest deadline 2024-08-31T06:24:53.570276+0000) 2024-08-31T06:25:13.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:12 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:12.989+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:37.071304+0000 front 2024-08-31T06:24:37.071497+0000 (oldest deadline 2024-08-31T06:25:02.971434+0000) 2024-08-31T06:25:13.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:13 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:13.117+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:13.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:13 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:13.117+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:13.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:13 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:13.117+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:13.916 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:13 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:13.824+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:21.798609+0000 front 2024-08-31T06:24:21.798847+0000 (oldest deadline 2024-08-31T06:24:45.298817+0000) 2024-08-31T06:25:13.917 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:13 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:13.824+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.300670+0000 front 2024-08-31T06:24:32.300510+0000 (oldest deadline 2024-08-31T06:24:57.000455+0000) 2024-08-31T06:25:13.917 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:13 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:13.824+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:40.903301+0000 front 2024-08-31T06:24:40.903322+0000 (oldest deadline 2024-08-31T06:25:02.603246+0000) 2024-08-31T06:25:13.917 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:13 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:13.824+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:47.304318+0000 front 2024-08-31T06:24:47.304268+0000 (oldest deadline 2024-08-31T06:25:13.204244+0000) 2024-08-31T06:25:13.917 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:13 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:13.625+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:25:13.917 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:13 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:13.625+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:25:13.917 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:13 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:13.625+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:35.901359+0000 front 2024-08-31T06:24:35.901304+0000 (oldest deadline 2024-08-31T06:25:02.302188+0000) 2024-08-31T06:25:13.917 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:13 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:13.625+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.203189+0000 front 2024-08-31T06:24:48.203315+0000 (oldest deadline 2024-08-31T06:25:11.003818+0000) 2024-08-31T06:25:13.960 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:13 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:13.642+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:13.960 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:13 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:13.642+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:13.960 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:13 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:13.642+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.499981+0000 front 2024-08-31T06:24:39.500178+0000 (oldest deadline 2024-08-31T06:25:04.799749+0000) 2024-08-31T06:25:14.222 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:14 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:14.150+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:14.223 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:14 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:14.150+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:14.223 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:14 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:14.150+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:14.223 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:13 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:13.955+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:25:14.223 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:13 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:13.955+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.670468+0000 front 2024-08-31T06:24:30.670483+0000 (oldest deadline 2024-08-31T06:24:53.570276+0000) 2024-08-31T06:25:14.223 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:13 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:13.955+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:37.071304+0000 front 2024-08-31T06:24:37.071497+0000 (oldest deadline 2024-08-31T06:25:02.971434+0000) 2024-08-31T06:25:14.223 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:13 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:13.955+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.173143+0000 front 2024-08-31T06:24:48.173187+0000 (oldest deadline 2024-08-31T06:25:13.472902+0000) 2024-08-31T06:25:14.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:13 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:13.957+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.099444+0000 front 2024-08-31T06:24:32.099224+0000 (oldest deadline 2024-08-31T06:24:57.999327+0000) 2024-08-31T06:25:14.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:13 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:13.957+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:38.500433+0000 front 2024-08-31T06:24:38.500454+0000 (oldest deadline 2024-08-31T06:25:02.000525+0000) 2024-08-31T06:25:14.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:13 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:13.957+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:49.303805+0000 front 2024-08-31T06:24:49.303913+0000 (oldest deadline 2024-08-31T06:25:11.603806+0000) 2024-08-31T06:25:14.913 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:14 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:14.822+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:21.798609+0000 front 2024-08-31T06:24:21.798847+0000 (oldest deadline 2024-08-31T06:24:45.298817+0000) 2024-08-31T06:25:14.913 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:14 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:14.822+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.300670+0000 front 2024-08-31T06:24:32.300510+0000 (oldest deadline 2024-08-31T06:24:57.000455+0000) 2024-08-31T06:25:14.913 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:14 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:14.822+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:40.903301+0000 front 2024-08-31T06:24:40.903322+0000 (oldest deadline 2024-08-31T06:25:02.603246+0000) 2024-08-31T06:25:14.913 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:14 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6[48560]: 2024-08-31T06:25:14.822+0000 7f419d000640 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:47.304318+0000 front 2024-08-31T06:24:47.304268+0000 (oldest deadline 2024-08-31T06:25:13.204244+0000) 2024-08-31T06:25:14.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:14 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:14.634+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:25:14.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:14 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:14.634+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:25:14.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:14 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:14.634+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:35.901359+0000 front 2024-08-31T06:24:35.901304+0000 (oldest deadline 2024-08-31T06:25:02.302188+0000) 2024-08-31T06:25:14.914 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:14 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:14.634+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.203189+0000 front 2024-08-31T06:24:48.203315+0000 (oldest deadline 2024-08-31T06:25:11.003818+0000) 2024-08-31T06:25:14.995 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:14 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:14.622+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:14.995 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:14 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:14.622+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:14.995 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:14 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:14.622+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.499981+0000 front 2024-08-31T06:24:39.500178+0000 (oldest deadline 2024-08-31T06:25:04.799749+0000) 2024-08-31T06:25:15.222 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:15 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:15.129+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:15.223 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:15 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:15.129+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:15.223 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:15 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:15.129+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:15.223 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:14 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:14.969+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:25:15.223 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:14 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:14.969+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.670468+0000 front 2024-08-31T06:24:30.670483+0000 (oldest deadline 2024-08-31T06:24:53.570276+0000) 2024-08-31T06:25:15.223 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:14 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:14.969+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:37.071304+0000 front 2024-08-31T06:24:37.071497+0000 (oldest deadline 2024-08-31T06:25:02.971434+0000) 2024-08-31T06:25:15.223 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:14 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:14.969+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.173143+0000 front 2024-08-31T06:24:48.173187+0000 (oldest deadline 2024-08-31T06:25:13.472902+0000) 2024-08-31T06:25:15.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:14 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:14.993+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.099444+0000 front 2024-08-31T06:24:32.099224+0000 (oldest deadline 2024-08-31T06:24:57.999327+0000) 2024-08-31T06:25:15.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:14 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:14.993+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:38.500433+0000 front 2024-08-31T06:24:38.500454+0000 (oldest deadline 2024-08-31T06:25:02.000525+0000) 2024-08-31T06:25:15.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:14 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:14.993+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:49.303805+0000 front 2024-08-31T06:24:49.303913+0000 (oldest deadline 2024-08-31T06:25:11.603806+0000) 2024-08-31T06:25:15.872 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:15 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:15.607+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:15.872 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:15 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:15.607+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:15.872 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:15 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:15.607+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.499981+0000 front 2024-08-31T06:24:39.500178+0000 (oldest deadline 2024-08-31T06:25:04.799749+0000) 2024-08-31T06:25:15.872 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:15 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:15.607+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:44.800369+0000 front 2024-08-31T06:24:44.800594+0000 (oldest deadline 2024-08-31T06:25:15.400910+0000) 2024-08-31T06:25:15.913 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:15 smithi016 podman[107910]: 2024-08-31 06:25:15.841848218 +0000 UTC m=+6.017205172 container died db7f406efaa3de6513c75dceaed0454c77f964817e4bf29f0e82346e3cc262b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.1) 2024-08-31T06:25:15.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:15 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:15.621+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:25:15.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:15 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:15.621+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:25:15.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:15 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:15.621+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:35.901359+0000 front 2024-08-31T06:24:35.901304+0000 (oldest deadline 2024-08-31T06:25:02.302188+0000) 2024-08-31T06:25:15.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:15 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:15.621+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.203189+0000 front 2024-08-31T06:24:48.203315+0000 (oldest deadline 2024-08-31T06:25:11.003818+0000) 2024-08-31T06:25:16.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:15 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:15.953+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.099444+0000 front 2024-08-31T06:24:32.099224+0000 (oldest deadline 2024-08-31T06:24:57.999327+0000) 2024-08-31T06:25:16.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:15 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:15.953+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:38.500433+0000 front 2024-08-31T06:24:38.500454+0000 (oldest deadline 2024-08-31T06:25:02.000525+0000) 2024-08-31T06:25:16.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:15 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:15.953+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:49.303805+0000 front 2024-08-31T06:24:49.303913+0000 (oldest deadline 2024-08-31T06:25:11.603806+0000) 2024-08-31T06:25:16.413 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:16 smithi016 podman[107910]: 2024-08-31 06:25:16.367967982 +0000 UTC m=+6.543324935 container cleanup db7f406efaa3de6513c75dceaed0454c77f964817e4bf29f0e82346e3cc262b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.1, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-08-31T06:25:16.413 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:16 smithi016 bash[107910]: ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6 2024-08-31T06:25:16.472 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:16 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:16.136+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:16.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:16 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:16.136+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:16.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:16 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:16.136+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:16.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:15 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:15.988+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:25:16.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:15 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:15.988+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.670468+0000 front 2024-08-31T06:24:30.670483+0000 (oldest deadline 2024-08-31T06:24:53.570276+0000) 2024-08-31T06:25:16.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:15 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:15.988+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:37.071304+0000 front 2024-08-31T06:24:37.071497+0000 (oldest deadline 2024-08-31T06:25:02.971434+0000) 2024-08-31T06:25:16.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:15 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:15.988+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.173143+0000 front 2024-08-31T06:24:48.173187+0000 (oldest deadline 2024-08-31T06:25:13.472902+0000) 2024-08-31T06:25:16.641 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:16 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:16.612+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:16.642 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:16 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:16.612+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:16.642 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:16 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:16.612+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.499981+0000 front 2024-08-31T06:24:39.500178+0000 (oldest deadline 2024-08-31T06:25:04.799749+0000) 2024-08-31T06:25:16.642 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:16 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:16.612+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:44.800369+0000 front 2024-08-31T06:24:44.800594+0000 (oldest deadline 2024-08-31T06:25:15.400910+0000) 2024-08-31T06:25:16.912 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:16 smithi016 podman[107928]: 2024-08-31 06:25:16.63642974 +0000 UTC m=+0.780577435 container remove db7f406efaa3de6513c75dceaed0454c77f964817e4bf29f0e82346e3cc262b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.1, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-31T06:25:16.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:16 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:16.651+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:25:16.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:16 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:16.651+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:25:16.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:16 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:16.651+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:35.901359+0000 front 2024-08-31T06:24:35.901304+0000 (oldest deadline 2024-08-31T06:25:02.302188+0000) 2024-08-31T06:25:16.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:16 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:16.651+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.203189+0000 front 2024-08-31T06:24:48.203315+0000 (oldest deadline 2024-08-31T06:25:11.003818+0000) 2024-08-31T06:25:17.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:16 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:16.938+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.099444+0000 front 2024-08-31T06:24:32.099224+0000 (oldest deadline 2024-08-31T06:24:57.999327+0000) 2024-08-31T06:25:17.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:16 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:16.938+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:38.500433+0000 front 2024-08-31T06:24:38.500454+0000 (oldest deadline 2024-08-31T06:25:02.000525+0000) 2024-08-31T06:25:17.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:16 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:16.938+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:49.303805+0000 front 2024-08-31T06:24:49.303913+0000 (oldest deadline 2024-08-31T06:25:11.603806+0000) 2024-08-31T06:25:17.374 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:17 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:17.154+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:17.374 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:17 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:17.154+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:17.374 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:17 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:17.154+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:17.374 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:17 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:17.037+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:25:17.375 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:17 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:17.037+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.670468+0000 front 2024-08-31T06:24:30.670483+0000 (oldest deadline 2024-08-31T06:24:53.570276+0000) 2024-08-31T06:25:17.375 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:17 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:17.037+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:37.071304+0000 front 2024-08-31T06:24:37.071497+0000 (oldest deadline 2024-08-31T06:25:02.971434+0000) 2024-08-31T06:25:17.375 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:17 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:17.037+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.173143+0000 front 2024-08-31T06:24:48.173187+0000 (oldest deadline 2024-08-31T06:25:13.472902+0000) 2024-08-31T06:25:17.458 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:17 smithi016 podman[108011]: 2024-08-31 06:25:17.187227294 +0000 UTC m=+0.119654388 container create d43bb8335fc532a720b719dafccf648998fdcda98a6df8f549fd190dcc6b00ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_BRANCH=HEAD) 2024-08-31T06:25:17.458 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:17 smithi016 podman[108011]: 2024-08-31 06:25:17.094980286 +0000 UTC m=+0.027407396 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T06:25:17.459 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:17 smithi016 podman[108011]: 2024-08-31 06:25:17.456734506 +0000 UTC m=+0.389161595 container init d43bb8335fc532a720b719dafccf648998fdcda98a6df8f549fd190dcc6b00ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-08-31T06:25:17.730 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:17 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:17.698+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:25:17.730 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:17 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:17.698+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:25:17.730 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:17 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:17.698+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:35.901359+0000 front 2024-08-31T06:24:35.901304+0000 (oldest deadline 2024-08-31T06:25:02.302188+0000) 2024-08-31T06:25:17.730 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:17 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:17.698+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.203189+0000 front 2024-08-31T06:24:48.203315+0000 (oldest deadline 2024-08-31T06:25:11.003818+0000) 2024-08-31T06:25:17.730 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:17 smithi016 podman[108011]: 2024-08-31 06:25:17.463875623 +0000 UTC m=+0.396302712 container start d43bb8335fc532a720b719dafccf648998fdcda98a6df8f549fd190dcc6b00ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.1, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-31T06:25:17.730 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:17 smithi016 podman[108011]: 2024-08-31 06:25:17.539596108 +0000 UTC m=+0.472023210 container attach d43bb8335fc532a720b719dafccf648998fdcda98a6df8f549fd190dcc6b00ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.1) 2024-08-31T06:25:17.730 INFO:journalctl@ceph.osd.6.smithi016.stdout:Aug 31 06:25:17 smithi016 podman[108011]: 2024-08-31 06:25:17.728612535 +0000 UTC m=+0.661039622 container died d43bb8335fc532a720b719dafccf648998fdcda98a6df8f549fd190dcc6b00ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.1, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611) 2024-08-31T06:25:17.872 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:17 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:17.587+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:17.872 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:17 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:17.587+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:17.872 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:17 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:17.587+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.499981+0000 front 2024-08-31T06:24:39.500178+0000 (oldest deadline 2024-08-31T06:25:04.799749+0000) 2024-08-31T06:25:17.872 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:17 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:17.587+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:44.800369+0000 front 2024-08-31T06:24:44.800594+0000 (oldest deadline 2024-08-31T06:25:15.400910+0000) 2024-08-31T06:25:18.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:17 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:17.953+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.099444+0000 front 2024-08-31T06:24:32.099224+0000 (oldest deadline 2024-08-31T06:24:57.999327+0000) 2024-08-31T06:25:18.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:17 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:17.953+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:38.500433+0000 front 2024-08-31T06:24:38.500454+0000 (oldest deadline 2024-08-31T06:25:02.000525+0000) 2024-08-31T06:25:18.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:17 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:17.953+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:49.303805+0000 front 2024-08-31T06:24:49.303913+0000 (oldest deadline 2024-08-31T06:25:11.603806+0000) 2024-08-31T06:25:18.398 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:18 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:18.059+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:25:18.398 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:18 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:18.059+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.670468+0000 front 2024-08-31T06:24:30.670483+0000 (oldest deadline 2024-08-31T06:24:53.570276+0000) 2024-08-31T06:25:18.398 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:18 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:18.059+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:37.071304+0000 front 2024-08-31T06:24:37.071497+0000 (oldest deadline 2024-08-31T06:25:02.971434+0000) 2024-08-31T06:25:18.399 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:18 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:18.059+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.173143+0000 front 2024-08-31T06:24:48.173187+0000 (oldest deadline 2024-08-31T06:25:13.472902+0000) 2024-08-31T06:25:18.399 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:18 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:18.192+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:18.399 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:18 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:18.192+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:18.399 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:18 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:18.192+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:18.562 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@osd.6.service' 2024-08-31T06:25:18.639 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T06:25:18.639 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-08-31T06:25:18.639 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-08-31T06:25:18.640 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@osd.7 2024-08-31T06:25:18.880 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:18 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:18.743+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:25:18.880 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:18 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:18.743+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:25:18.880 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:18 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:18.743+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:35.901359+0000 front 2024-08-31T06:24:35.901304+0000 (oldest deadline 2024-08-31T06:25:02.302188+0000) 2024-08-31T06:25:18.880 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:18 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:18.743+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.203189+0000 front 2024-08-31T06:24:48.203315+0000 (oldest deadline 2024-08-31T06:25:11.003818+0000) 2024-08-31T06:25:18.943 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:18 smithi155 systemd[1]: Stopping Ceph osd.7 for 673e4aa0-675d-11ef-bcd4-c7b262605968... 2024-08-31T06:25:18.943 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:18 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:18.940+0000 7f4a1ab6d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T06:25:18.943 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:18 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:18.940+0000 7f4a1ab6d640 -1 osd.7 106 *** Got signal Terminated *** 2024-08-31T06:25:18.943 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:18 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:18.940+0000 7f4a1ab6d640 -1 osd.7 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-31T06:25:18.944 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:18 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:18.628+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:18.944 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:18 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:18.628+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:18.944 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:18 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:18.628+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.499981+0000 front 2024-08-31T06:24:39.500178+0000 (oldest deadline 2024-08-31T06:25:04.799749+0000) 2024-08-31T06:25:18.944 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:18 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:18.628+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:44.800369+0000 front 2024-08-31T06:24:44.800594+0000 (oldest deadline 2024-08-31T06:25:15.400910+0000) 2024-08-31T06:25:19.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:18 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:18.958+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.099444+0000 front 2024-08-31T06:24:32.099224+0000 (oldest deadline 2024-08-31T06:24:57.999327+0000) 2024-08-31T06:25:19.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:18 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:18.958+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:38.500433+0000 front 2024-08-31T06:24:38.500454+0000 (oldest deadline 2024-08-31T06:25:02.000525+0000) 2024-08-31T06:25:19.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:18 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:18.958+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:49.303805+0000 front 2024-08-31T06:24:49.303913+0000 (oldest deadline 2024-08-31T06:25:11.603806+0000) 2024-08-31T06:25:19.472 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:19 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:19.156+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:19.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:19 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:19.156+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:19.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:19 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:19.156+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:19.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:19 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:19.043+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:25:19.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:19 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:19.043+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.670468+0000 front 2024-08-31T06:24:30.670483+0000 (oldest deadline 2024-08-31T06:24:53.570276+0000) 2024-08-31T06:25:19.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:19 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:19.043+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:37.071304+0000 front 2024-08-31T06:24:37.071497+0000 (oldest deadline 2024-08-31T06:25:02.971434+0000) 2024-08-31T06:25:19.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:19 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:19.043+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.173143+0000 front 2024-08-31T06:24:48.173187+0000 (oldest deadline 2024-08-31T06:25:13.472902+0000) 2024-08-31T06:25:19.951 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:19 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:19.664+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:19.951 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:19 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:19.664+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:19.951 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:19 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:19.664+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.499981+0000 front 2024-08-31T06:24:39.500178+0000 (oldest deadline 2024-08-31T06:25:04.799749+0000) 2024-08-31T06:25:19.951 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:19 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:19.664+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:44.800369+0000 front 2024-08-31T06:24:44.800594+0000 (oldest deadline 2024-08-31T06:25:15.400910+0000) 2024-08-31T06:25:20.071 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:19 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:19.765+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:25:20.072 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:19 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:19.765+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:25:20.072 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:19 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:19.765+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:35.901359+0000 front 2024-08-31T06:24:35.901304+0000 (oldest deadline 2024-08-31T06:25:02.302188+0000) 2024-08-31T06:25:20.072 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:19 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:19.765+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.203189+0000 front 2024-08-31T06:24:48.203315+0000 (oldest deadline 2024-08-31T06:25:11.003818+0000) 2024-08-31T06:25:20.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:19 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:19.949+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.099444+0000 front 2024-08-31T06:24:32.099224+0000 (oldest deadline 2024-08-31T06:24:57.999327+0000) 2024-08-31T06:25:20.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:19 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:19.949+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:38.500433+0000 front 2024-08-31T06:24:38.500454+0000 (oldest deadline 2024-08-31T06:25:02.000525+0000) 2024-08-31T06:25:20.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:19 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:19.949+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:49.303805+0000 front 2024-08-31T06:24:49.303913+0000 (oldest deadline 2024-08-31T06:25:11.603806+0000) 2024-08-31T06:25:20.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:20 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:20.014+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:25:20.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:20 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:20.014+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.670468+0000 front 2024-08-31T06:24:30.670483+0000 (oldest deadline 2024-08-31T06:24:53.570276+0000) 2024-08-31T06:25:20.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:20 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:20.014+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:37.071304+0000 front 2024-08-31T06:24:37.071497+0000 (oldest deadline 2024-08-31T06:25:02.971434+0000) 2024-08-31T06:25:20.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:20 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:20.014+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.173143+0000 front 2024-08-31T06:24:48.173187+0000 (oldest deadline 2024-08-31T06:25:13.472902+0000) 2024-08-31T06:25:20.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:20 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:20.168+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:20.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:20 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:20.168+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:20.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:20 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:20.168+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:20.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:20 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:20.168+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:58.403155+0000 front 2024-08-31T06:24:58.402997+0000 (oldest deadline 2024-08-31T06:25:19.502922+0000) 2024-08-31T06:25:20.988 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:20 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:20.678+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:20.988 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:20 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:20.678+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:20.988 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:20 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:20.678+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.499981+0000 front 2024-08-31T06:24:39.500178+0000 (oldest deadline 2024-08-31T06:25:04.799749+0000) 2024-08-31T06:25:20.989 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:20 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:20.678+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:44.800369+0000 front 2024-08-31T06:24:44.800594+0000 (oldest deadline 2024-08-31T06:25:15.400910+0000) 2024-08-31T06:25:21.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:20 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:20.815+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:25:21.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:20 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:20.815+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:25:21.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:20 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:20.815+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:35.901359+0000 front 2024-08-31T06:24:35.901304+0000 (oldest deadline 2024-08-31T06:25:02.302188+0000) 2024-08-31T06:25:21.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:20 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:20.815+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.203189+0000 front 2024-08-31T06:24:48.203315+0000 (oldest deadline 2024-08-31T06:25:11.003818+0000) 2024-08-31T06:25:21.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:20 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:20.986+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.099444+0000 front 2024-08-31T06:24:32.099224+0000 (oldest deadline 2024-08-31T06:24:57.999327+0000) 2024-08-31T06:25:21.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:20 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:20.986+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:38.500433+0000 front 2024-08-31T06:24:38.500454+0000 (oldest deadline 2024-08-31T06:25:02.000525+0000) 2024-08-31T06:25:21.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:20 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:20.986+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:49.303805+0000 front 2024-08-31T06:24:49.303913+0000 (oldest deadline 2024-08-31T06:25:11.603806+0000) 2024-08-31T06:25:21.472 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:21 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:21.009+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:25:21.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:21 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:21.009+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.670468+0000 front 2024-08-31T06:24:30.670483+0000 (oldest deadline 2024-08-31T06:24:53.570276+0000) 2024-08-31T06:25:21.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:21 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:21.009+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:37.071304+0000 front 2024-08-31T06:24:37.071497+0000 (oldest deadline 2024-08-31T06:25:02.971434+0000) 2024-08-31T06:25:21.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:21 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:21.009+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.173143+0000 front 2024-08-31T06:24:48.173187+0000 (oldest deadline 2024-08-31T06:25:13.472902+0000) 2024-08-31T06:25:21.476 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:21 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:21.170+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:21.477 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:21 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:21.170+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:21.477 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:21 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:21.170+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:21.477 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:21 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:21.170+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:58.403155+0000 front 2024-08-31T06:24:58.402997+0000 (oldest deadline 2024-08-31T06:25:19.502922+0000) 2024-08-31T06:25:22.020 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:21 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:21.670+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:22.020 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:21 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:21.670+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:22.020 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:21 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:21.670+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.499981+0000 front 2024-08-31T06:24:39.500178+0000 (oldest deadline 2024-08-31T06:25:04.799749+0000) 2024-08-31T06:25:22.020 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:21 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:21.670+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:44.800369+0000 front 2024-08-31T06:24:44.800594+0000 (oldest deadline 2024-08-31T06:25:15.400910+0000) 2024-08-31T06:25:22.020 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:21 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:21.670+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:55.401302+0000 front 2024-08-31T06:24:55.401342+0000 (oldest deadline 2024-08-31T06:25:20.701389+0000) 2024-08-31T06:25:22.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:21 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:21.789+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:25:22.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:21 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:21.789+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:25:22.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:21 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:21.789+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:35.901359+0000 front 2024-08-31T06:24:35.901304+0000 (oldest deadline 2024-08-31T06:25:02.302188+0000) 2024-08-31T06:25:22.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:21 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:21.789+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.203189+0000 front 2024-08-31T06:24:48.203315+0000 (oldest deadline 2024-08-31T06:25:11.003818+0000) 2024-08-31T06:25:22.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:21 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:21.789+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:56.805532+0000 front 2024-08-31T06:24:56.805595+0000 (oldest deadline 2024-08-31T06:25:20.905505+0000) 2024-08-31T06:25:22.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:22 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:22.018+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.099444+0000 front 2024-08-31T06:24:32.099224+0000 (oldest deadline 2024-08-31T06:24:57.999327+0000) 2024-08-31T06:25:22.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:22 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:22.018+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:38.500433+0000 front 2024-08-31T06:24:38.500454+0000 (oldest deadline 2024-08-31T06:25:02.000525+0000) 2024-08-31T06:25:22.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:22 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:22.018+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:49.303805+0000 front 2024-08-31T06:24:49.303913+0000 (oldest deadline 2024-08-31T06:25:11.603806+0000) 2024-08-31T06:25:22.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:22 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:22.018+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:55.704982+0000 front 2024-08-31T06:24:55.705029+0000 (oldest deadline 2024-08-31T06:25:21.005242+0000) 2024-08-31T06:25:22.472 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:21 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:21.985+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:25:22.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:21 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:21.985+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.670468+0000 front 2024-08-31T06:24:30.670483+0000 (oldest deadline 2024-08-31T06:24:53.570276+0000) 2024-08-31T06:25:22.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:21 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:21.985+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:37.071304+0000 front 2024-08-31T06:24:37.071497+0000 (oldest deadline 2024-08-31T06:25:02.971434+0000) 2024-08-31T06:25:22.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:21 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:21.985+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.173143+0000 front 2024-08-31T06:24:48.173187+0000 (oldest deadline 2024-08-31T06:25:13.472902+0000) 2024-08-31T06:25:22.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:22 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:22.147+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:22.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:22 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:22.147+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:22.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:22 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:22.147+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:22.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:22 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:22.147+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:58.403155+0000 front 2024-08-31T06:24:58.402997+0000 (oldest deadline 2024-08-31T06:25:19.502922+0000) 2024-08-31T06:25:23.025 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:22 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:22.708+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:23.025 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:22 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:22.708+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:23.026 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:22 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:22.708+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.499981+0000 front 2024-08-31T06:24:39.500178+0000 (oldest deadline 2024-08-31T06:25:04.799749+0000) 2024-08-31T06:25:23.026 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:22 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:22.708+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:44.800369+0000 front 2024-08-31T06:24:44.800594+0000 (oldest deadline 2024-08-31T06:25:15.400910+0000) 2024-08-31T06:25:23.026 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:22 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:22.708+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:55.401302+0000 front 2024-08-31T06:24:55.401342+0000 (oldest deadline 2024-08-31T06:25:20.701389+0000) 2024-08-31T06:25:23.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:22 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:22.788+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:25:23.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:22 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:22.788+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:25:23.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:22 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:22.788+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:35.901359+0000 front 2024-08-31T06:24:35.901304+0000 (oldest deadline 2024-08-31T06:25:02.302188+0000) 2024-08-31T06:25:23.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:22 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:22.788+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.203189+0000 front 2024-08-31T06:24:48.203315+0000 (oldest deadline 2024-08-31T06:25:11.003818+0000) 2024-08-31T06:25:23.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:22 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:22.788+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:56.805532+0000 front 2024-08-31T06:24:56.805595+0000 (oldest deadline 2024-08-31T06:25:20.905505+0000) 2024-08-31T06:25:23.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:23 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:23.024+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.099444+0000 front 2024-08-31T06:24:32.099224+0000 (oldest deadline 2024-08-31T06:24:57.999327+0000) 2024-08-31T06:25:23.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:23 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:23.024+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:38.500433+0000 front 2024-08-31T06:24:38.500454+0000 (oldest deadline 2024-08-31T06:25:02.000525+0000) 2024-08-31T06:25:23.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:23 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:23.024+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:49.303805+0000 front 2024-08-31T06:24:49.303913+0000 (oldest deadline 2024-08-31T06:25:11.603806+0000) 2024-08-31T06:25:23.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:23 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7[40855]: 2024-08-31T06:25:23.024+0000 7f4a13150640 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:55.704982+0000 front 2024-08-31T06:24:55.705029+0000 (oldest deadline 2024-08-31T06:25:21.005242+0000) 2024-08-31T06:25:23.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:22 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:22.973+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:25:23.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:22 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:22.973+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.670468+0000 front 2024-08-31T06:24:30.670483+0000 (oldest deadline 2024-08-31T06:24:53.570276+0000) 2024-08-31T06:25:23.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:22 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:22.973+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:37.071304+0000 front 2024-08-31T06:24:37.071497+0000 (oldest deadline 2024-08-31T06:25:02.971434+0000) 2024-08-31T06:25:23.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:22 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:22.973+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.173143+0000 front 2024-08-31T06:24:48.173187+0000 (oldest deadline 2024-08-31T06:25:13.472902+0000) 2024-08-31T06:25:23.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:23 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:23.168+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:23.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:23 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:23.168+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:23.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:23 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:23.168+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:23.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:23 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:23.168+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:58.403155+0000 front 2024-08-31T06:24:58.402997+0000 (oldest deadline 2024-08-31T06:25:19.502922+0000) 2024-08-31T06:25:24.064 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:23 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:23.786+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:25:24.064 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:23 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:23.786+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:25:24.065 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:23 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:23.786+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:35.901359+0000 front 2024-08-31T06:24:35.901304+0000 (oldest deadline 2024-08-31T06:25:02.302188+0000) 2024-08-31T06:25:24.065 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:23 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:23.786+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.203189+0000 front 2024-08-31T06:24:48.203315+0000 (oldest deadline 2024-08-31T06:25:11.003818+0000) 2024-08-31T06:25:24.065 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:23 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:23.786+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:56.805532+0000 front 2024-08-31T06:24:56.805595+0000 (oldest deadline 2024-08-31T06:25:20.905505+0000) 2024-08-31T06:25:24.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:23 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:23.702+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:24.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:23 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:23.702+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:24.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:23 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:23.702+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.499981+0000 front 2024-08-31T06:24:39.500178+0000 (oldest deadline 2024-08-31T06:25:04.799749+0000) 2024-08-31T06:25:24.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:23 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:23.702+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:44.800369+0000 front 2024-08-31T06:24:44.800594+0000 (oldest deadline 2024-08-31T06:25:15.400910+0000) 2024-08-31T06:25:24.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:23 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:23.702+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:55.401302+0000 front 2024-08-31T06:24:55.401342+0000 (oldest deadline 2024-08-31T06:25:20.701389+0000) 2024-08-31T06:25:24.223 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:23 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:23.953+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:25:24.223 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:23 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:23.953+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.670468+0000 front 2024-08-31T06:24:30.670483+0000 (oldest deadline 2024-08-31T06:24:53.570276+0000) 2024-08-31T06:25:24.223 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:23 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:23.953+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:37.071304+0000 front 2024-08-31T06:24:37.071497+0000 (oldest deadline 2024-08-31T06:25:02.971434+0000) 2024-08-31T06:25:24.223 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:23 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:23.953+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.173143+0000 front 2024-08-31T06:24:48.173187+0000 (oldest deadline 2024-08-31T06:25:13.472902+0000) 2024-08-31T06:25:24.223 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:24 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:24.165+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:24.223 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:24 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:24.165+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:24.223 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:24 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:24.165+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:24.223 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:24 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:24.165+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:58.403155+0000 front 2024-08-31T06:24:58.402997+0000 (oldest deadline 2024-08-31T06:25:19.502922+0000) 2024-08-31T06:25:24.751 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:24 smithi155 podman[59372]: 2024-08-31 06:25:24.477181553 +0000 UTC m=+5.642289914 container died 8582e224bf819bb48d5cec0165a0f00a5c824fba91b118faa766d9396c5fe345 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-08-31T06:25:24.751 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:24 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:24.660+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:24.751 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:24 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:24.660+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:24.751 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:24 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:24.660+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.499981+0000 front 2024-08-31T06:24:39.500178+0000 (oldest deadline 2024-08-31T06:25:04.799749+0000) 2024-08-31T06:25:24.751 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:24 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:24.660+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:44.800369+0000 front 2024-08-31T06:24:44.800594+0000 (oldest deadline 2024-08-31T06:25:15.400910+0000) 2024-08-31T06:25:24.751 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:24 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:24.660+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:55.401302+0000 front 2024-08-31T06:24:55.401342+0000 (oldest deadline 2024-08-31T06:25:20.701389+0000) 2024-08-31T06:25:25.081 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:24 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:24.781+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:25:25.081 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:24 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:24.781+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:25:25.081 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:24 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:24.781+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:35.901359+0000 front 2024-08-31T06:24:35.901304+0000 (oldest deadline 2024-08-31T06:25:02.302188+0000) 2024-08-31T06:25:25.081 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:24 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:24.781+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.203189+0000 front 2024-08-31T06:24:48.203315+0000 (oldest deadline 2024-08-31T06:25:11.003818+0000) 2024-08-31T06:25:25.081 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:24 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:24.781+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:56.805532+0000 front 2024-08-31T06:24:56.805595+0000 (oldest deadline 2024-08-31T06:25:20.905505+0000) 2024-08-31T06:25:25.119 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:24 smithi155 podman[59372]: 2024-08-31 06:25:24.857499971 +0000 UTC m=+6.022608328 container cleanup 8582e224bf819bb48d5cec0165a0f00a5c824fba91b118faa766d9396c5fe345 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, CEPH_POINT_RELEASE=, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-31T06:25:25.119 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:24 smithi155 bash[59372]: ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7 2024-08-31T06:25:25.222 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:24 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:24.958+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:25:25.222 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:24 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:24.958+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.670468+0000 front 2024-08-31T06:24:30.670483+0000 (oldest deadline 2024-08-31T06:24:53.570276+0000) 2024-08-31T06:25:25.222 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:24 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:24.958+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:37.071304+0000 front 2024-08-31T06:24:37.071497+0000 (oldest deadline 2024-08-31T06:25:02.971434+0000) 2024-08-31T06:25:25.223 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:24 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:24.958+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.173143+0000 front 2024-08-31T06:24:48.173187+0000 (oldest deadline 2024-08-31T06:25:13.472902+0000) 2024-08-31T06:25:25.223 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:25 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:25.158+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:25.223 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:25 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:25.158+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:25.223 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:25 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:25.158+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:25.223 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:25 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:25.158+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:58.403155+0000 front 2024-08-31T06:24:58.402997+0000 (oldest deadline 2024-08-31T06:25:19.502922+0000) 2024-08-31T06:25:25.372 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:25 smithi155 podman[59387]: 2024-08-31 06:25:25.117216003 +0000 UTC m=+0.634774932 container remove 8582e224bf819bb48d5cec0165a0f00a5c824fba91b118faa766d9396c5fe345 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-08-31T06:25:25.717 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:25 smithi155 podman[59475]: 2024-08-31 06:25:25.366499711 +0000 UTC m=+0.016074224 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T06:25:25.717 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:25 smithi155 podman[59475]: 2024-08-31 06:25:25.476303782 +0000 UTC m=+0.125878285 container create 3be151e240f1b07e6a61d46d45162003581669b46ef4019237dde3110f1c44b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.1, org.label-schema.build-date=20240731, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True) 2024-08-31T06:25:25.717 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:25 smithi155 podman[59475]: 2024-08-31 06:25:25.711960519 +0000 UTC m=+0.361535019 container init 3be151e240f1b07e6a61d46d45162003581669b46ef4019237dde3110f1c44b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.1, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611) 2024-08-31T06:25:25.717 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:25 smithi155 podman[59475]: 2024-08-31 06:25:25.715393743 +0000 UTC m=+0.364968246 container start 3be151e240f1b07e6a61d46d45162003581669b46ef4019237dde3110f1c44b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-31T06:25:25.717 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:25 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:25.697+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:25.717 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:25 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:25.697+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:25.717 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:25 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:25.697+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.499981+0000 front 2024-08-31T06:24:39.500178+0000 (oldest deadline 2024-08-31T06:25:04.799749+0000) 2024-08-31T06:25:25.717 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:25 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:25.697+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:44.800369+0000 front 2024-08-31T06:24:44.800594+0000 (oldest deadline 2024-08-31T06:25:15.400910+0000) 2024-08-31T06:25:25.718 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:25 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:25.697+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:55.401302+0000 front 2024-08-31T06:24:55.401342+0000 (oldest deadline 2024-08-31T06:25:20.701389+0000) 2024-08-31T06:25:26.122 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:25 smithi155 podman[59475]: 2024-08-31 06:25:25.837279339 +0000 UTC m=+0.486853844 container attach 3be151e240f1b07e6a61d46d45162003581669b46ef4019237dde3110f1c44b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240731, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.1) 2024-08-31T06:25:26.122 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 31 06:25:25 smithi155 podman[59475]: 2024-08-31 06:25:25.900484728 +0000 UTC m=+0.550059229 container died 3be151e240f1b07e6a61d46d45162003581669b46ef4019237dde3110f1c44b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.1, org.label-schema.build-date=20240731, ceph=True) 2024-08-31T06:25:26.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:25 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:25.769+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:25:26.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:25 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:25.769+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:25:26.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:25 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:25.769+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:35.901359+0000 front 2024-08-31T06:24:35.901304+0000 (oldest deadline 2024-08-31T06:25:02.302188+0000) 2024-08-31T06:25:26.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:25 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:25.769+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.203189+0000 front 2024-08-31T06:24:48.203315+0000 (oldest deadline 2024-08-31T06:25:11.003818+0000) 2024-08-31T06:25:26.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:25 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:25.769+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:56.805532+0000 front 2024-08-31T06:24:56.805595+0000 (oldest deadline 2024-08-31T06:25:20.905505+0000) 2024-08-31T06:25:26.222 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:25 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:25.950+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:25:26.223 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:25 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:25.950+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.670468+0000 front 2024-08-31T06:24:30.670483+0000 (oldest deadline 2024-08-31T06:24:53.570276+0000) 2024-08-31T06:25:26.223 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:25 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:25.950+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:37.071304+0000 front 2024-08-31T06:24:37.071497+0000 (oldest deadline 2024-08-31T06:25:02.971434+0000) 2024-08-31T06:25:26.223 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:25 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:25.950+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.173143+0000 front 2024-08-31T06:24:48.173187+0000 (oldest deadline 2024-08-31T06:25:13.472902+0000) 2024-08-31T06:25:26.223 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:26 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:26.138+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:26.223 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:26 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:26.138+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:26.223 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:26 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:26.138+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:26.223 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:26 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:26.138+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:58.403155+0000 front 2024-08-31T06:24:58.402997+0000 (oldest deadline 2024-08-31T06:25:19.502922+0000) 2024-08-31T06:25:26.606 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@osd.7.service' 2024-08-31T06:25:26.650 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T06:25:26.650 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-08-31T06:25:26.650 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-08-31T06:25:26.651 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@osd.8 2024-08-31T06:25:26.872 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:26 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:26.729+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:26.872 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:26 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:26.729+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:26.872 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:26 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:26.729+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.499981+0000 front 2024-08-31T06:24:39.500178+0000 (oldest deadline 2024-08-31T06:25:04.799749+0000) 2024-08-31T06:25:26.872 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:26 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:26.729+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:44.800369+0000 front 2024-08-31T06:24:44.800594+0000 (oldest deadline 2024-08-31T06:25:15.400910+0000) 2024-08-31T06:25:26.872 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:26 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:26.729+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:55.401302+0000 front 2024-08-31T06:24:55.401342+0000 (oldest deadline 2024-08-31T06:25:20.701389+0000) 2024-08-31T06:25:26.967 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:26 smithi184 systemd[1]: Stopping Ceph osd.8 for 673e4aa0-675d-11ef-bcd4-c7b262605968... 2024-08-31T06:25:26.967 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:26 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:26.926+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:25:26.967 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:26 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:26.926+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.670468+0000 front 2024-08-31T06:24:30.670483+0000 (oldest deadline 2024-08-31T06:24:53.570276+0000) 2024-08-31T06:25:26.967 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:26 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:26.926+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:37.071304+0000 front 2024-08-31T06:24:37.071497+0000 (oldest deadline 2024-08-31T06:25:02.971434+0000) 2024-08-31T06:25:26.967 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:26 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:26.926+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.173143+0000 front 2024-08-31T06:24:48.173187+0000 (oldest deadline 2024-08-31T06:25:13.472902+0000) 2024-08-31T06:25:26.967 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:26 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:26.964+0000 7f6880245640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T06:25:26.967 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:26 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:26.964+0000 7f6880245640 -1 osd.8 106 *** Got signal Terminated *** 2024-08-31T06:25:26.968 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:26 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:26.964+0000 7f6880245640 -1 osd.8 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-31T06:25:27.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:26 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:26.724+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:25:27.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:26 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:26.724+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:25:27.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:26 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:26.724+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:35.901359+0000 front 2024-08-31T06:24:35.901304+0000 (oldest deadline 2024-08-31T06:25:02.302188+0000) 2024-08-31T06:25:27.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:26 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:26.724+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.203189+0000 front 2024-08-31T06:24:48.203315+0000 (oldest deadline 2024-08-31T06:25:11.003818+0000) 2024-08-31T06:25:27.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:26 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:26.724+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:56.805532+0000 front 2024-08-31T06:24:56.805595+0000 (oldest deadline 2024-08-31T06:25:20.905505+0000) 2024-08-31T06:25:27.222 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:27 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:27.138+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:27.222 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:27 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:27.139+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:27.223 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:27 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:27.139+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:27.223 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:27 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:27.139+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:58.403155+0000 front 2024-08-31T06:24:58.402997+0000 (oldest deadline 2024-08-31T06:25:19.502922+0000) 2024-08-31T06:25:28.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:27 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:27.690+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:28.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:27 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:27.690+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:28.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:27 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:27.690+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.499981+0000 front 2024-08-31T06:24:39.500178+0000 (oldest deadline 2024-08-31T06:25:04.799749+0000) 2024-08-31T06:25:28.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:27 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:27.690+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:44.800369+0000 front 2024-08-31T06:24:44.800594+0000 (oldest deadline 2024-08-31T06:25:15.400910+0000) 2024-08-31T06:25:28.123 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:27 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:27.690+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:55.401302+0000 front 2024-08-31T06:24:55.401342+0000 (oldest deadline 2024-08-31T06:25:20.701389+0000) 2024-08-31T06:25:28.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:27 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:27.716+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:25:28.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:27 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:27.716+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:25:28.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:27 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:27.716+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:35.901359+0000 front 2024-08-31T06:24:35.901304+0000 (oldest deadline 2024-08-31T06:25:02.302188+0000) 2024-08-31T06:25:28.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:27 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:27.716+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.203189+0000 front 2024-08-31T06:24:48.203315+0000 (oldest deadline 2024-08-31T06:25:11.003818+0000) 2024-08-31T06:25:28.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:27 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:27.716+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:56.805532+0000 front 2024-08-31T06:24:56.805595+0000 (oldest deadline 2024-08-31T06:25:20.905505+0000) 2024-08-31T06:25:28.223 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:27 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:27.924+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:25:28.223 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:27 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:27.924+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.670468+0000 front 2024-08-31T06:24:30.670483+0000 (oldest deadline 2024-08-31T06:24:53.570276+0000) 2024-08-31T06:25:28.223 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:27 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:27.924+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:37.071304+0000 front 2024-08-31T06:24:37.071497+0000 (oldest deadline 2024-08-31T06:25:02.971434+0000) 2024-08-31T06:25:28.223 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:27 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:27.924+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.173143+0000 front 2024-08-31T06:24:48.173187+0000 (oldest deadline 2024-08-31T06:25:13.472902+0000) 2024-08-31T06:25:28.223 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:28 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:28.160+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:28.223 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:28 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:28.160+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:28.223 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:28 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:28.160+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:28.224 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:28 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:28.160+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:58.403155+0000 front 2024-08-31T06:24:58.402997+0000 (oldest deadline 2024-08-31T06:25:19.502922+0000) 2024-08-31T06:25:29.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:28 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:28.725+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:29.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:28 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:28.725+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:29.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:28 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:28.725+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.499981+0000 front 2024-08-31T06:24:39.500178+0000 (oldest deadline 2024-08-31T06:25:04.799749+0000) 2024-08-31T06:25:29.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:28 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:28.725+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:44.800369+0000 front 2024-08-31T06:24:44.800594+0000 (oldest deadline 2024-08-31T06:25:15.400910+0000) 2024-08-31T06:25:29.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:28 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:28.725+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:55.401302+0000 front 2024-08-31T06:24:55.401342+0000 (oldest deadline 2024-08-31T06:25:20.701389+0000) 2024-08-31T06:25:29.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:28 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:28.725+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:06.002531+0000 front 2024-08-31T06:25:06.002753+0000 (oldest deadline 2024-08-31T06:25:27.702422+0000) 2024-08-31T06:25:29.162 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:28 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:28.726+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:25:29.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:28 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:28.726+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:25:29.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:28 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:28.726+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:35.901359+0000 front 2024-08-31T06:24:35.901304+0000 (oldest deadline 2024-08-31T06:25:02.302188+0000) 2024-08-31T06:25:29.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:28 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:28.726+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.203189+0000 front 2024-08-31T06:24:48.203315+0000 (oldest deadline 2024-08-31T06:25:11.003818+0000) 2024-08-31T06:25:29.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:28 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:28.726+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:56.805532+0000 front 2024-08-31T06:24:56.805595+0000 (oldest deadline 2024-08-31T06:25:20.905505+0000) 2024-08-31T06:25:29.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:28 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:28.726+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:05.308486+0000 front 2024-08-31T06:25:05.308432+0000 (oldest deadline 2024-08-31T06:25:28.208398+0000) 2024-08-31T06:25:29.223 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:28 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:28.958+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:25:29.223 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:28 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:28.958+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.670468+0000 front 2024-08-31T06:24:30.670483+0000 (oldest deadline 2024-08-31T06:24:53.570276+0000) 2024-08-31T06:25:29.223 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:28 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:28.958+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:37.071304+0000 front 2024-08-31T06:24:37.071497+0000 (oldest deadline 2024-08-31T06:25:02.971434+0000) 2024-08-31T06:25:29.223 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:28 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:28.958+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.173143+0000 front 2024-08-31T06:24:48.173187+0000 (oldest deadline 2024-08-31T06:25:13.472902+0000) 2024-08-31T06:25:29.223 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:29 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:29.202+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:29.223 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:29 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:29.202+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:29.223 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:29 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:29.202+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:29.223 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:29 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:29.202+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:58.403155+0000 front 2024-08-31T06:24:58.402997+0000 (oldest deadline 2024-08-31T06:25:19.502922+0000) 2024-08-31T06:25:30.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:29 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:29.767+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:30.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:29 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:29.767+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:30.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:29 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:29.767+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.499981+0000 front 2024-08-31T06:24:39.500178+0000 (oldest deadline 2024-08-31T06:25:04.799749+0000) 2024-08-31T06:25:30.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:29 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:29.767+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:44.800369+0000 front 2024-08-31T06:24:44.800594+0000 (oldest deadline 2024-08-31T06:25:15.400910+0000) 2024-08-31T06:25:30.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:29 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:29.767+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:55.401302+0000 front 2024-08-31T06:24:55.401342+0000 (oldest deadline 2024-08-31T06:25:20.701389+0000) 2024-08-31T06:25:30.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:29 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:29.767+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:06.002531+0000 front 2024-08-31T06:25:06.002753+0000 (oldest deadline 2024-08-31T06:25:27.702422+0000) 2024-08-31T06:25:30.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:29 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:29.681+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:25:30.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:29 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:29.681+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:25:30.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:29 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:29.681+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:35.901359+0000 front 2024-08-31T06:24:35.901304+0000 (oldest deadline 2024-08-31T06:25:02.302188+0000) 2024-08-31T06:25:30.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:29 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:29.681+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.203189+0000 front 2024-08-31T06:24:48.203315+0000 (oldest deadline 2024-08-31T06:25:11.003818+0000) 2024-08-31T06:25:30.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:29 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:29.681+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:56.805532+0000 front 2024-08-31T06:24:56.805595+0000 (oldest deadline 2024-08-31T06:25:20.905505+0000) 2024-08-31T06:25:30.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:29 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:29.681+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:05.308486+0000 front 2024-08-31T06:25:05.308432+0000 (oldest deadline 2024-08-31T06:25:28.208398+0000) 2024-08-31T06:25:30.222 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:29 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:29.961+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:25:30.222 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:29 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:29.961+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.670468+0000 front 2024-08-31T06:24:30.670483+0000 (oldest deadline 2024-08-31T06:24:53.570276+0000) 2024-08-31T06:25:30.223 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:29 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:29.961+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:37.071304+0000 front 2024-08-31T06:24:37.071497+0000 (oldest deadline 2024-08-31T06:25:02.971434+0000) 2024-08-31T06:25:30.223 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:29 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:29.961+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.173143+0000 front 2024-08-31T06:24:48.173187+0000 (oldest deadline 2024-08-31T06:25:13.472902+0000) 2024-08-31T06:25:30.723 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:30 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:30.245+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:30.723 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:30 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:30.245+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:30.723 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:30 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:30.245+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:30.723 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:30 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:30.245+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:58.403155+0000 front 2024-08-31T06:24:58.402997+0000 (oldest deadline 2024-08-31T06:25:19.502922+0000) 2024-08-31T06:25:30.723 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:30 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:30.245+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:03.603637+0000 front 2024-08-31T06:25:03.603696+0000 (oldest deadline 2024-08-31T06:25:29.503703+0000) 2024-08-31T06:25:31.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:30 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:30.810+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:31.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:30 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:30.810+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:31.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:30 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:30.810+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.499981+0000 front 2024-08-31T06:24:39.500178+0000 (oldest deadline 2024-08-31T06:25:04.799749+0000) 2024-08-31T06:25:31.123 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:30 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:30.810+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:44.800369+0000 front 2024-08-31T06:24:44.800594+0000 (oldest deadline 2024-08-31T06:25:15.400910+0000) 2024-08-31T06:25:31.123 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:30 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:30.810+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:55.401302+0000 front 2024-08-31T06:24:55.401342+0000 (oldest deadline 2024-08-31T06:25:20.701389+0000) 2024-08-31T06:25:31.123 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:30 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:30.810+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:06.002531+0000 front 2024-08-31T06:25:06.002753+0000 (oldest deadline 2024-08-31T06:25:27.702422+0000) 2024-08-31T06:25:31.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:30 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:30.700+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:25:31.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:30 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:30.700+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:25:31.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:30 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:30.700+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:35.901359+0000 front 2024-08-31T06:24:35.901304+0000 (oldest deadline 2024-08-31T06:25:02.302188+0000) 2024-08-31T06:25:31.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:30 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:30.700+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.203189+0000 front 2024-08-31T06:24:48.203315+0000 (oldest deadline 2024-08-31T06:25:11.003818+0000) 2024-08-31T06:25:31.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:30 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:30.700+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:56.805532+0000 front 2024-08-31T06:24:56.805595+0000 (oldest deadline 2024-08-31T06:25:20.905505+0000) 2024-08-31T06:25:31.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:30 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:30.700+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:05.308486+0000 front 2024-08-31T06:25:05.308432+0000 (oldest deadline 2024-08-31T06:25:28.208398+0000) 2024-08-31T06:25:31.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:30 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:30.983+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:25:31.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:30 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:30.983+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.670468+0000 front 2024-08-31T06:24:30.670483+0000 (oldest deadline 2024-08-31T06:24:53.570276+0000) 2024-08-31T06:25:31.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:30 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:30.983+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:37.071304+0000 front 2024-08-31T06:24:37.071497+0000 (oldest deadline 2024-08-31T06:25:02.971434+0000) 2024-08-31T06:25:31.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:30 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:30.983+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.173143+0000 front 2024-08-31T06:24:48.173187+0000 (oldest deadline 2024-08-31T06:25:13.472902+0000) 2024-08-31T06:25:31.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:31 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:31.231+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:31.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:31 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:31.231+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:31.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:31 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:31.231+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:31.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:31 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:31.231+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:58.403155+0000 front 2024-08-31T06:24:58.402997+0000 (oldest deadline 2024-08-31T06:25:19.502922+0000) 2024-08-31T06:25:31.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:31 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:31.231+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:03.603637+0000 front 2024-08-31T06:25:03.603696+0000 (oldest deadline 2024-08-31T06:25:29.503703+0000) 2024-08-31T06:25:32.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:31 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:31.787+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:32.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:31 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:31.787+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:32.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:31 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:31.787+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.499981+0000 front 2024-08-31T06:24:39.500178+0000 (oldest deadline 2024-08-31T06:25:04.799749+0000) 2024-08-31T06:25:32.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:31 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:31.787+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:44.800369+0000 front 2024-08-31T06:24:44.800594+0000 (oldest deadline 2024-08-31T06:25:15.400910+0000) 2024-08-31T06:25:32.123 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:31 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:31.787+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:55.401302+0000 front 2024-08-31T06:24:55.401342+0000 (oldest deadline 2024-08-31T06:25:20.701389+0000) 2024-08-31T06:25:32.123 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:31 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:31.787+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:06.002531+0000 front 2024-08-31T06:25:06.002753+0000 (oldest deadline 2024-08-31T06:25:27.702422+0000) 2024-08-31T06:25:32.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:31 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:31.688+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:25:32.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:31 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:31.688+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:25:32.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:31 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:31.688+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:35.901359+0000 front 2024-08-31T06:24:35.901304+0000 (oldest deadline 2024-08-31T06:25:02.302188+0000) 2024-08-31T06:25:32.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:31 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:31.688+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.203189+0000 front 2024-08-31T06:24:48.203315+0000 (oldest deadline 2024-08-31T06:25:11.003818+0000) 2024-08-31T06:25:32.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:31 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:31.688+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:56.805532+0000 front 2024-08-31T06:24:56.805595+0000 (oldest deadline 2024-08-31T06:25:20.905505+0000) 2024-08-31T06:25:32.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:31 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:31.688+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:05.308486+0000 front 2024-08-31T06:25:05.308432+0000 (oldest deadline 2024-08-31T06:25:28.208398+0000) 2024-08-31T06:25:32.222 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:31 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:31.959+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.669486+0000 front 2024-08-31T06:24:23.669862+0000 (oldest deadline 2024-08-31T06:24:45.969414+0000) 2024-08-31T06:25:32.222 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:31 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:31.959+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.670468+0000 front 2024-08-31T06:24:30.670483+0000 (oldest deadline 2024-08-31T06:24:53.570276+0000) 2024-08-31T06:25:32.223 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:31 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:31.959+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:37.071304+0000 front 2024-08-31T06:24:37.071497+0000 (oldest deadline 2024-08-31T06:25:02.971434+0000) 2024-08-31T06:25:32.223 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:31 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8[41903]: 2024-08-31T06:25:31.959+0000 7f687c830640 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.173143+0000 front 2024-08-31T06:24:48.173187+0000 (oldest deadline 2024-08-31T06:25:13.472902+0000) 2024-08-31T06:25:32.723 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:32 smithi184 podman[60523]: 2024-08-31 06:25:32.46516102 +0000 UTC m=+5.621393871 container died ddd6988e75003f707b7ef3d7af953db89199046e3e3b29a16708ba4c29514632 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.1, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, CEPH_POINT_RELEASE=) 2024-08-31T06:25:32.724 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:32 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:32.227+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:32.724 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:32 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:32.227+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:32.724 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:32 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:32.227+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:32.724 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:32 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:32.227+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:58.403155+0000 front 2024-08-31T06:24:58.402997+0000 (oldest deadline 2024-08-31T06:25:19.502922+0000) 2024-08-31T06:25:32.725 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:32 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:32.227+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:03.603637+0000 front 2024-08-31T06:25:03.603696+0000 (oldest deadline 2024-08-31T06:25:29.503703+0000) 2024-08-31T06:25:33.092 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:32 smithi184 podman[60523]: 2024-08-31 06:25:32.831029792 +0000 UTC m=+5.987262644 container cleanup ddd6988e75003f707b7ef3d7af953db89199046e3e3b29a16708ba4c29514632 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.1, ceph=True) 2024-08-31T06:25:33.092 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:32 smithi184 bash[60523]: ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8 2024-08-31T06:25:33.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:32 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:32.784+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:33.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:32 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:32.784+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:33.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:32 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:32.784+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.499981+0000 front 2024-08-31T06:24:39.500178+0000 (oldest deadline 2024-08-31T06:25:04.799749+0000) 2024-08-31T06:25:33.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:32 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:32.784+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:44.800369+0000 front 2024-08-31T06:24:44.800594+0000 (oldest deadline 2024-08-31T06:25:15.400910+0000) 2024-08-31T06:25:33.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:32 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:32.784+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:55.401302+0000 front 2024-08-31T06:24:55.401342+0000 (oldest deadline 2024-08-31T06:25:20.701389+0000) 2024-08-31T06:25:33.123 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:32 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:32.784+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:06.002531+0000 front 2024-08-31T06:25:06.002753+0000 (oldest deadline 2024-08-31T06:25:27.702422+0000) 2024-08-31T06:25:33.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:32 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:32.675+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:25:33.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:32 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:32.675+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:25:33.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:32 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:32.675+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:35.901359+0000 front 2024-08-31T06:24:35.901304+0000 (oldest deadline 2024-08-31T06:25:02.302188+0000) 2024-08-31T06:25:33.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:32 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:32.675+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.203189+0000 front 2024-08-31T06:24:48.203315+0000 (oldest deadline 2024-08-31T06:25:11.003818+0000) 2024-08-31T06:25:33.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:32 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:32.675+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:56.805532+0000 front 2024-08-31T06:24:56.805595+0000 (oldest deadline 2024-08-31T06:25:20.905505+0000) 2024-08-31T06:25:33.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:32 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:32.675+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:05.308486+0000 front 2024-08-31T06:25:05.308432+0000 (oldest deadline 2024-08-31T06:25:28.208398+0000) 2024-08-31T06:25:33.446 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:33 smithi184 podman[60538]: 2024-08-31 06:25:33.090858255 +0000 UTC m=+0.626976140 container remove ddd6988e75003f707b7ef3d7af953db89199046e3e3b29a16708ba4c29514632 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611) 2024-08-31T06:25:33.446 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:33 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:33.192+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:33.446 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:33 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:33.192+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:33.447 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:33 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:33.192+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:33.447 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:33 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:33.192+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:58.403155+0000 front 2024-08-31T06:24:58.402997+0000 (oldest deadline 2024-08-31T06:25:19.502922+0000) 2024-08-31T06:25:33.447 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:33 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:33.192+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:03.603637+0000 front 2024-08-31T06:25:03.603696+0000 (oldest deadline 2024-08-31T06:25:29.503703+0000) 2024-08-31T06:25:33.699 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:33 smithi184 podman[60626]: 2024-08-31 06:25:33.347479853 +0000 UTC m=+0.016036930 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T06:25:33.699 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:33 smithi184 podman[60626]: 2024-08-31 06:25:33.458560756 +0000 UTC m=+0.127117827 container create 1b22755252242d5ddc02ac91f58f0390077b559a89750cf77cb0c4463ff00a26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8-deactivate, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.1, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-31T06:25:33.699 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:33 smithi184 podman[60626]: 2024-08-31 06:25:33.694517511 +0000 UTC m=+0.363074578 container init 1b22755252242d5ddc02ac91f58f0390077b559a89750cf77cb0c4463ff00a26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.1, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-08-31T06:25:33.699 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:33 smithi184 podman[60626]: 2024-08-31 06:25:33.697699497 +0000 UTC m=+0.366256567 container start 1b22755252242d5ddc02ac91f58f0390077b559a89750cf77cb0c4463ff00a26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-31T06:25:33.972 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:33 smithi184 podman[60626]: 2024-08-31 06:25:33.794780277 +0000 UTC m=+0.463337345 container attach 1b22755252242d5ddc02ac91f58f0390077b559a89750cf77cb0c4463ff00a26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.1, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-08-31T06:25:33.972 INFO:journalctl@ceph.osd.8.smithi184.stdout:Aug 31 06:25:33 smithi184 podman[60626]: 2024-08-31 06:25:33.889424973 +0000 UTC m=+0.557982051 container died 1b22755252242d5ddc02ac91f58f0390077b559a89750cf77cb0c4463ff00a26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-8-deactivate, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.1, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-31T06:25:34.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:33 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:33.777+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:34.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:33 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:33.777+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:34.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:33 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:33.777+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.499981+0000 front 2024-08-31T06:24:39.500178+0000 (oldest deadline 2024-08-31T06:25:04.799749+0000) 2024-08-31T06:25:34.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:33 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:33.777+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:44.800369+0000 front 2024-08-31T06:24:44.800594+0000 (oldest deadline 2024-08-31T06:25:15.400910+0000) 2024-08-31T06:25:34.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:33 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:33.777+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:55.401302+0000 front 2024-08-31T06:24:55.401342+0000 (oldest deadline 2024-08-31T06:25:20.701389+0000) 2024-08-31T06:25:34.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:33 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:33.777+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:06.002531+0000 front 2024-08-31T06:25:06.002753+0000 (oldest deadline 2024-08-31T06:25:27.702422+0000) 2024-08-31T06:25:34.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:33 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:33.714+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:25:34.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:33 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:33.714+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:25:34.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:33 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:33.714+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:35.901359+0000 front 2024-08-31T06:24:35.901304+0000 (oldest deadline 2024-08-31T06:25:02.302188+0000) 2024-08-31T06:25:34.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:33 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:33.714+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.203189+0000 front 2024-08-31T06:24:48.203315+0000 (oldest deadline 2024-08-31T06:25:11.003818+0000) 2024-08-31T06:25:34.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:33 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:33.714+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:56.805532+0000 front 2024-08-31T06:24:56.805595+0000 (oldest deadline 2024-08-31T06:25:20.905505+0000) 2024-08-31T06:25:34.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:33 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:33.714+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:05.308486+0000 front 2024-08-31T06:25:05.308432+0000 (oldest deadline 2024-08-31T06:25:28.208398+0000) 2024-08-31T06:25:34.472 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:34 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:34.226+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:34.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:34 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:34.226+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:34.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:34 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:34.226+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:34.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:34 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:34.226+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:58.403155+0000 front 2024-08-31T06:24:58.402997+0000 (oldest deadline 2024-08-31T06:25:19.502922+0000) 2024-08-31T06:25:34.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:34 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:34.226+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:03.603637+0000 front 2024-08-31T06:25:03.603696+0000 (oldest deadline 2024-08-31T06:25:29.503703+0000) 2024-08-31T06:25:34.542 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@osd.8.service' 2024-08-31T06:25:34.581 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T06:25:34.581 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-08-31T06:25:34.581 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-08-31T06:25:34.581 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@osd.9 2024-08-31T06:25:34.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:34 smithi016 systemd[1]: Stopping Ceph osd.9 for 673e4aa0-675d-11ef-bcd4-c7b262605968... 2024-08-31T06:25:34.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:34 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:34.726+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:25:34.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:34 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:34.726+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:25:34.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:34 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:34.726+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:35.901359+0000 front 2024-08-31T06:24:35.901304+0000 (oldest deadline 2024-08-31T06:25:02.302188+0000) 2024-08-31T06:25:34.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:34 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:34.726+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.203189+0000 front 2024-08-31T06:24:48.203315+0000 (oldest deadline 2024-08-31T06:25:11.003818+0000) 2024-08-31T06:25:34.913 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:34 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:34.726+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:56.805532+0000 front 2024-08-31T06:24:56.805595+0000 (oldest deadline 2024-08-31T06:25:20.905505+0000) 2024-08-31T06:25:34.914 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:34 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:34.726+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:05.308486+0000 front 2024-08-31T06:25:05.308432+0000 (oldest deadline 2024-08-31T06:25:28.208398+0000) 2024-08-31T06:25:35.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:34 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:34.767+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:35.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:34 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:34.767+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:35.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:34 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:34.767+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.499981+0000 front 2024-08-31T06:24:39.500178+0000 (oldest deadline 2024-08-31T06:25:04.799749+0000) 2024-08-31T06:25:35.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:34 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:34.767+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:44.800369+0000 front 2024-08-31T06:24:44.800594+0000 (oldest deadline 2024-08-31T06:25:15.400910+0000) 2024-08-31T06:25:35.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:34 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:34.767+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:55.401302+0000 front 2024-08-31T06:24:55.401342+0000 (oldest deadline 2024-08-31T06:25:20.701389+0000) 2024-08-31T06:25:35.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:34 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:34.767+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:06.002531+0000 front 2024-08-31T06:25:06.002753+0000 (oldest deadline 2024-08-31T06:25:27.702422+0000) 2024-08-31T06:25:35.412 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:35 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:35.025+0000 7f63f7200640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T06:25:35.413 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:35 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:35.025+0000 7f63f7200640 -1 osd.9 106 *** Got signal Terminated *** 2024-08-31T06:25:35.413 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:35 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:35.025+0000 7f63f7200640 -1 osd.9 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-31T06:25:35.472 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:35 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:35.179+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:35.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:35 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:35.179+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:35.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:35 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:35.179+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:35.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:35 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:35.179+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:58.403155+0000 front 2024-08-31T06:24:58.402997+0000 (oldest deadline 2024-08-31T06:25:19.502922+0000) 2024-08-31T06:25:35.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:35 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:35.179+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:03.603637+0000 front 2024-08-31T06:25:03.603696+0000 (oldest deadline 2024-08-31T06:25:29.503703+0000) 2024-08-31T06:25:36.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:35 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:35.803+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:36.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:35 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:35.803+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:36.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:35 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:35.803+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.499981+0000 front 2024-08-31T06:24:39.500178+0000 (oldest deadline 2024-08-31T06:25:04.799749+0000) 2024-08-31T06:25:36.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:35 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:35.803+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:44.800369+0000 front 2024-08-31T06:24:44.800594+0000 (oldest deadline 2024-08-31T06:25:15.400910+0000) 2024-08-31T06:25:36.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:35 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:35.803+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:55.401302+0000 front 2024-08-31T06:24:55.401342+0000 (oldest deadline 2024-08-31T06:25:20.701389+0000) 2024-08-31T06:25:36.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:35 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:35.803+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:06.002531+0000 front 2024-08-31T06:25:06.002753+0000 (oldest deadline 2024-08-31T06:25:27.702422+0000) 2024-08-31T06:25:36.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:35 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:35.769+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:25:36.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:35 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:35.769+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:25:36.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:35 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:35.769+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:35.901359+0000 front 2024-08-31T06:24:35.901304+0000 (oldest deadline 2024-08-31T06:25:02.302188+0000) 2024-08-31T06:25:36.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:35 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:35.769+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.203189+0000 front 2024-08-31T06:24:48.203315+0000 (oldest deadline 2024-08-31T06:25:11.003818+0000) 2024-08-31T06:25:36.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:35 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:35.769+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:56.805532+0000 front 2024-08-31T06:24:56.805595+0000 (oldest deadline 2024-08-31T06:25:20.905505+0000) 2024-08-31T06:25:36.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:35 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:35.769+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:05.308486+0000 front 2024-08-31T06:25:05.308432+0000 (oldest deadline 2024-08-31T06:25:28.208398+0000) 2024-08-31T06:25:36.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:36 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:36.175+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:36.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:36 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:36.175+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:36.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:36 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:36.175+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:36.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:36 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:36.175+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:58.403155+0000 front 2024-08-31T06:24:58.402997+0000 (oldest deadline 2024-08-31T06:25:19.502922+0000) 2024-08-31T06:25:36.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:36 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:36.175+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:03.603637+0000 front 2024-08-31T06:25:03.603696+0000 (oldest deadline 2024-08-31T06:25:29.503703+0000) 2024-08-31T06:25:37.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:36 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:36.795+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:37.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:36 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:36.795+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:37.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:36 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:36.795+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.499981+0000 front 2024-08-31T06:24:39.500178+0000 (oldest deadline 2024-08-31T06:25:04.799749+0000) 2024-08-31T06:25:37.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:36 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:36.795+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:44.800369+0000 front 2024-08-31T06:24:44.800594+0000 (oldest deadline 2024-08-31T06:25:15.400910+0000) 2024-08-31T06:25:37.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:36 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:36.795+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:55.401302+0000 front 2024-08-31T06:24:55.401342+0000 (oldest deadline 2024-08-31T06:25:20.701389+0000) 2024-08-31T06:25:37.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:36 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:36.795+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:06.002531+0000 front 2024-08-31T06:25:06.002753+0000 (oldest deadline 2024-08-31T06:25:27.702422+0000) 2024-08-31T06:25:37.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:36 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:36.801+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:25:37.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:36 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:36.801+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:25:37.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:36 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:36.801+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:35.901359+0000 front 2024-08-31T06:24:35.901304+0000 (oldest deadline 2024-08-31T06:25:02.302188+0000) 2024-08-31T06:25:37.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:36 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:36.801+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.203189+0000 front 2024-08-31T06:24:48.203315+0000 (oldest deadline 2024-08-31T06:25:11.003818+0000) 2024-08-31T06:25:37.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:36 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:36.801+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:56.805532+0000 front 2024-08-31T06:24:56.805595+0000 (oldest deadline 2024-08-31T06:25:20.905505+0000) 2024-08-31T06:25:37.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:36 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:36.801+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:05.308486+0000 front 2024-08-31T06:25:05.308432+0000 (oldest deadline 2024-08-31T06:25:28.208398+0000) 2024-08-31T06:25:37.472 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:37 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:37.164+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:37.472 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:37 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:37.164+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:37.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:37 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:37.164+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:37.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:37 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:37.164+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:58.403155+0000 front 2024-08-31T06:24:58.402997+0000 (oldest deadline 2024-08-31T06:25:19.502922+0000) 2024-08-31T06:25:37.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:37 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:37.164+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:03.603637+0000 front 2024-08-31T06:25:03.603696+0000 (oldest deadline 2024-08-31T06:25:29.503703+0000) 2024-08-31T06:25:38.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:37 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:37.824+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:38.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:37 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:37.824+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:38.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:37 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:37.824+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.499981+0000 front 2024-08-31T06:24:39.500178+0000 (oldest deadline 2024-08-31T06:25:04.799749+0000) 2024-08-31T06:25:38.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:37 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:37.824+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:44.800369+0000 front 2024-08-31T06:24:44.800594+0000 (oldest deadline 2024-08-31T06:25:15.400910+0000) 2024-08-31T06:25:38.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:37 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:37.824+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:55.401302+0000 front 2024-08-31T06:24:55.401342+0000 (oldest deadline 2024-08-31T06:25:20.701389+0000) 2024-08-31T06:25:38.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:37 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:37.824+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:06.002531+0000 front 2024-08-31T06:25:06.002753+0000 (oldest deadline 2024-08-31T06:25:27.702422+0000) 2024-08-31T06:25:38.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:37 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:37.824+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-08-31T06:25:14.604933+0000 front 2024-08-31T06:25:14.604805+0000 (oldest deadline 2024-08-31T06:25:36.904729+0000) 2024-08-31T06:25:38.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:37 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:37.831+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:25:38.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:37 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:37.831+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:25:38.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:37 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:37.831+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:35.901359+0000 front 2024-08-31T06:24:35.901304+0000 (oldest deadline 2024-08-31T06:25:02.302188+0000) 2024-08-31T06:25:38.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:37 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:37.831+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.203189+0000 front 2024-08-31T06:24:48.203315+0000 (oldest deadline 2024-08-31T06:25:11.003818+0000) 2024-08-31T06:25:38.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:37 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:37.831+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:56.805532+0000 front 2024-08-31T06:24:56.805595+0000 (oldest deadline 2024-08-31T06:25:20.905505+0000) 2024-08-31T06:25:38.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:37 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:37.831+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:05.308486+0000 front 2024-08-31T06:25:05.308432+0000 (oldest deadline 2024-08-31T06:25:28.208398+0000) 2024-08-31T06:25:38.472 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:38 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:38.151+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:38.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:38 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:38.151+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:38.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:38 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:38.151+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:38.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:38 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:38.151+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:58.403155+0000 front 2024-08-31T06:24:58.402997+0000 (oldest deadline 2024-08-31T06:25:19.502922+0000) 2024-08-31T06:25:38.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:38 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:38.151+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:03.603637+0000 front 2024-08-31T06:25:03.603696+0000 (oldest deadline 2024-08-31T06:25:29.503703+0000) 2024-08-31T06:25:38.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:38 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:38.151+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-08-31T06:25:14.005741+0000 front 2024-08-31T06:25:14.005896+0000 (oldest deadline 2024-08-31T06:25:38.105488+0000) 2024-08-31T06:25:39.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:38 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:38.813+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:39.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:38 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:38.813+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:39.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:38 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:38.813+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.499981+0000 front 2024-08-31T06:24:39.500178+0000 (oldest deadline 2024-08-31T06:25:04.799749+0000) 2024-08-31T06:25:39.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:38 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:38.813+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:44.800369+0000 front 2024-08-31T06:24:44.800594+0000 (oldest deadline 2024-08-31T06:25:15.400910+0000) 2024-08-31T06:25:39.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:38 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:38.813+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:55.401302+0000 front 2024-08-31T06:24:55.401342+0000 (oldest deadline 2024-08-31T06:25:20.701389+0000) 2024-08-31T06:25:39.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:38 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:38.813+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:06.002531+0000 front 2024-08-31T06:25:06.002753+0000 (oldest deadline 2024-08-31T06:25:27.702422+0000) 2024-08-31T06:25:39.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:38 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:38.813+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-08-31T06:25:14.604933+0000 front 2024-08-31T06:25:14.604805+0000 (oldest deadline 2024-08-31T06:25:36.904729+0000) 2024-08-31T06:25:39.162 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:38 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:38.873+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:25:39.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:38 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:38.873+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:25:39.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:38 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:38.873+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:35.901359+0000 front 2024-08-31T06:24:35.901304+0000 (oldest deadline 2024-08-31T06:25:02.302188+0000) 2024-08-31T06:25:39.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:38 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:38.873+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.203189+0000 front 2024-08-31T06:24:48.203315+0000 (oldest deadline 2024-08-31T06:25:11.003818+0000) 2024-08-31T06:25:39.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:38 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:38.873+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:56.805532+0000 front 2024-08-31T06:24:56.805595+0000 (oldest deadline 2024-08-31T06:25:20.905505+0000) 2024-08-31T06:25:39.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:38 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:38.873+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:05.308486+0000 front 2024-08-31T06:25:05.308432+0000 (oldest deadline 2024-08-31T06:25:28.208398+0000) 2024-08-31T06:25:39.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:38 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:38.873+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-08-31T06:25:13.509150+0000 front 2024-08-31T06:25:13.509193+0000 (oldest deadline 2024-08-31T06:25:38.109957+0000) 2024-08-31T06:25:39.472 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:39 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:39.182+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:39.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:39 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:39.182+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:39.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:39 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:39.182+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:39.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:39 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:39.182+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:58.403155+0000 front 2024-08-31T06:24:58.402997+0000 (oldest deadline 2024-08-31T06:25:19.502922+0000) 2024-08-31T06:25:39.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:39 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:39.182+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:03.603637+0000 front 2024-08-31T06:25:03.603696+0000 (oldest deadline 2024-08-31T06:25:29.503703+0000) 2024-08-31T06:25:39.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:39 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:39.182+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-08-31T06:25:14.005741+0000 front 2024-08-31T06:25:14.005896+0000 (oldest deadline 2024-08-31T06:25:38.105488+0000) 2024-08-31T06:25:40.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:39 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:39.791+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:40.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:39 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:39.791+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:40.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:39 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:39.791+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.499981+0000 front 2024-08-31T06:24:39.500178+0000 (oldest deadline 2024-08-31T06:25:04.799749+0000) 2024-08-31T06:25:40.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:39 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:39.791+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:44.800369+0000 front 2024-08-31T06:24:44.800594+0000 (oldest deadline 2024-08-31T06:25:15.400910+0000) 2024-08-31T06:25:40.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:39 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:39.791+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:55.401302+0000 front 2024-08-31T06:24:55.401342+0000 (oldest deadline 2024-08-31T06:25:20.701389+0000) 2024-08-31T06:25:40.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:39 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:39.791+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:06.002531+0000 front 2024-08-31T06:25:06.002753+0000 (oldest deadline 2024-08-31T06:25:27.702422+0000) 2024-08-31T06:25:40.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:39 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:39.791+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-08-31T06:25:14.604933+0000 front 2024-08-31T06:25:14.604805+0000 (oldest deadline 2024-08-31T06:25:36.904729+0000) 2024-08-31T06:25:40.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:39 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:39.880+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:22.498973+0000 front 2024-08-31T06:24:22.499054+0000 (oldest deadline 2024-08-31T06:24:44.799226+0000) 2024-08-31T06:25:40.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:39 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:39.880+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:30.700789+0000 front 2024-08-31T06:24:30.700877+0000 (oldest deadline 2024-08-31T06:24:53.600180+0000) 2024-08-31T06:25:40.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:39 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:39.880+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:35.901359+0000 front 2024-08-31T06:24:35.901304+0000 (oldest deadline 2024-08-31T06:25:02.302188+0000) 2024-08-31T06:25:40.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:39 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:39.880+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:48.203189+0000 front 2024-08-31T06:24:48.203315+0000 (oldest deadline 2024-08-31T06:25:11.003818+0000) 2024-08-31T06:25:40.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:39 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:39.880+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:56.805532+0000 front 2024-08-31T06:24:56.805595+0000 (oldest deadline 2024-08-31T06:25:20.905505+0000) 2024-08-31T06:25:40.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:39 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:39.880+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:05.308486+0000 front 2024-08-31T06:25:05.308432+0000 (oldest deadline 2024-08-31T06:25:28.208398+0000) 2024-08-31T06:25:40.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:39 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9[55510]: 2024-08-31T06:25:39.880+0000 7f63f2c00640 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-08-31T06:25:13.509150+0000 front 2024-08-31T06:25:13.509193+0000 (oldest deadline 2024-08-31T06:25:38.109957+0000) 2024-08-31T06:25:40.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:40 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:40.219+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:40.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:40 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:40.219+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:40.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:40 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:40.219+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:40.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:40 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:40.219+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:58.403155+0000 front 2024-08-31T06:24:58.402997+0000 (oldest deadline 2024-08-31T06:25:19.502922+0000) 2024-08-31T06:25:40.476 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:40 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:40.219+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:03.603637+0000 front 2024-08-31T06:25:03.603696+0000 (oldest deadline 2024-08-31T06:25:29.503703+0000) 2024-08-31T06:25:40.476 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:40 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:40.219+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-08-31T06:25:14.005741+0000 front 2024-08-31T06:25:14.005896+0000 (oldest deadline 2024-08-31T06:25:38.105488+0000) 2024-08-31T06:25:41.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:40 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:40.777+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:41.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:40 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:40.777+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:41.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:40 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:40.777+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.499981+0000 front 2024-08-31T06:24:39.500178+0000 (oldest deadline 2024-08-31T06:25:04.799749+0000) 2024-08-31T06:25:41.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:40 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:40.777+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:44.800369+0000 front 2024-08-31T06:24:44.800594+0000 (oldest deadline 2024-08-31T06:25:15.400910+0000) 2024-08-31T06:25:41.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:40 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:40.777+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:55.401302+0000 front 2024-08-31T06:24:55.401342+0000 (oldest deadline 2024-08-31T06:25:20.701389+0000) 2024-08-31T06:25:41.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:40 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:40.777+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:06.002531+0000 front 2024-08-31T06:25:06.002753+0000 (oldest deadline 2024-08-31T06:25:27.702422+0000) 2024-08-31T06:25:41.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:40 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:40.777+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-08-31T06:25:14.604933+0000 front 2024-08-31T06:25:14.604805+0000 (oldest deadline 2024-08-31T06:25:36.904729+0000) 2024-08-31T06:25:41.163 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:40 smithi016 podman[108125]: 2024-08-31 06:25:40.788798995 +0000 UTC m=+5.914734290 container died 2806ee4dcd913e0387b7158e650d0f9eccd4a6678ff05f92f1dbb510528610ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.build-date=20240731, io.buildah.version=1.37.1, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-31T06:25:41.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:41 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:41.173+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:41.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:41 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:41.173+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:41.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:41 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:41.173+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:41.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:41 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:41.173+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:58.403155+0000 front 2024-08-31T06:24:58.402997+0000 (oldest deadline 2024-08-31T06:25:19.502922+0000) 2024-08-31T06:25:41.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:41 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:41.173+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:03.603637+0000 front 2024-08-31T06:25:03.603696+0000 (oldest deadline 2024-08-31T06:25:29.503703+0000) 2024-08-31T06:25:41.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:41 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:41.173+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-08-31T06:25:14.005741+0000 front 2024-08-31T06:25:14.005896+0000 (oldest deadline 2024-08-31T06:25:38.105488+0000) 2024-08-31T06:25:41.617 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:41 smithi016 podman[108125]: 2024-08-31 06:25:41.372471799 +0000 UTC m=+6.498407082 container cleanup 2806ee4dcd913e0387b7158e650d0f9eccd4a6678ff05f92f1dbb510528610ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.1, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-08-31T06:25:41.617 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:41 smithi016 bash[108125]: ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9 2024-08-31T06:25:41.912 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:41 smithi016 podman[108142]: 2024-08-31 06:25:41.615398473 +0000 UTC m=+0.816140354 container remove 2806ee4dcd913e0387b7158e650d0f9eccd4a6678ff05f92f1dbb510528610ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.1, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-08-31T06:25:42.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:41 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:41.733+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:42.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:41 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:41.733+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:42.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:41 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:41.733+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.499981+0000 front 2024-08-31T06:24:39.500178+0000 (oldest deadline 2024-08-31T06:25:04.799749+0000) 2024-08-31T06:25:42.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:41 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:41.733+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:44.800369+0000 front 2024-08-31T06:24:44.800594+0000 (oldest deadline 2024-08-31T06:25:15.400910+0000) 2024-08-31T06:25:42.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:41 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:41.733+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:55.401302+0000 front 2024-08-31T06:24:55.401342+0000 (oldest deadline 2024-08-31T06:25:20.701389+0000) 2024-08-31T06:25:42.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:41 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:41.733+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:06.002531+0000 front 2024-08-31T06:25:06.002753+0000 (oldest deadline 2024-08-31T06:25:27.702422+0000) 2024-08-31T06:25:42.123 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:41 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:41.733+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-08-31T06:25:14.604933+0000 front 2024-08-31T06:25:14.604805+0000 (oldest deadline 2024-08-31T06:25:36.904729+0000) 2024-08-31T06:25:42.321 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:42 smithi016 podman[108225]: 2024-08-31 06:25:41.954627775 +0000 UTC m=+0.027823082 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T06:25:42.322 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:42 smithi016 podman[108225]: 2024-08-31 06:25:42.201326375 +0000 UTC m=+0.274521665 container create 57e513709c1eabaa77ab54f3322faea0bdb6cebc22308d6f9e6b9c78c2cf1194 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-08-31T06:25:42.472 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:42 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:42.184+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:42.472 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:42 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:42.184+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:42.472 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:42 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:42.184+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:42.472 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:42 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:42.184+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:58.403155+0000 front 2024-08-31T06:24:58.402997+0000 (oldest deadline 2024-08-31T06:25:19.502922+0000) 2024-08-31T06:25:42.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:42 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:42.184+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:03.603637+0000 front 2024-08-31T06:25:03.603696+0000 (oldest deadline 2024-08-31T06:25:29.503703+0000) 2024-08-31T06:25:42.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:42 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:42.184+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-08-31T06:25:14.005741+0000 front 2024-08-31T06:25:14.005896+0000 (oldest deadline 2024-08-31T06:25:38.105488+0000) 2024-08-31T06:25:42.663 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:42 smithi016 podman[108225]: 2024-08-31 06:25:42.419214119 +0000 UTC m=+0.492409410 container init 57e513709c1eabaa77ab54f3322faea0bdb6cebc22308d6f9e6b9c78c2cf1194 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9-deactivate, io.buildah.version=1.37.1, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True) 2024-08-31T06:25:42.663 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:42 smithi016 podman[108225]: 2024-08-31 06:25:42.425886671 +0000 UTC m=+0.499081971 container start 57e513709c1eabaa77ab54f3322faea0bdb6cebc22308d6f9e6b9c78c2cf1194 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_BRANCH=HEAD, io.buildah.version=1.37.1, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-08-31T06:25:42.663 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:42 smithi016 podman[108225]: 2024-08-31 06:25:42.493817371 +0000 UTC m=+0.567012676 container attach 57e513709c1eabaa77ab54f3322faea0bdb6cebc22308d6f9e6b9c78c2cf1194 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , io.buildah.version=1.37.1, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-31T06:25:43.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Aug 31 06:25:42 smithi016 podman[108248]: 2024-08-31 06:25:42.732576033 +0000 UTC m=+0.026638124 container died 57e513709c1eabaa77ab54f3322faea0bdb6cebc22308d6f9e6b9c78c2cf1194 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.1, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-08-31T06:25:43.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:42 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:42.700+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:43.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:42 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:42.700+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:43.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:42 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:42.700+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.499981+0000 front 2024-08-31T06:24:39.500178+0000 (oldest deadline 2024-08-31T06:25:04.799749+0000) 2024-08-31T06:25:43.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:42 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:42.700+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:44.800369+0000 front 2024-08-31T06:24:44.800594+0000 (oldest deadline 2024-08-31T06:25:15.400910+0000) 2024-08-31T06:25:43.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:42 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:42.700+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:55.401302+0000 front 2024-08-31T06:24:55.401342+0000 (oldest deadline 2024-08-31T06:25:20.701389+0000) 2024-08-31T06:25:43.123 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:42 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:42.700+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:06.002531+0000 front 2024-08-31T06:25:06.002753+0000 (oldest deadline 2024-08-31T06:25:27.702422+0000) 2024-08-31T06:25:43.123 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:42 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:42.700+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-08-31T06:25:14.604933+0000 front 2024-08-31T06:25:14.604805+0000 (oldest deadline 2024-08-31T06:25:36.904729+0000) 2024-08-31T06:25:43.377 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@osd.9.service' 2024-08-31T06:25:43.450 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T06:25:43.451 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-08-31T06:25:43.451 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-08-31T06:25:43.451 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@osd.10 2024-08-31T06:25:43.472 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:43 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:43.172+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:43.472 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:43 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:43.172+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:43.472 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:43 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:43.172+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:43.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:43 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:43.172+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:58.403155+0000 front 2024-08-31T06:24:58.402997+0000 (oldest deadline 2024-08-31T06:25:19.502922+0000) 2024-08-31T06:25:43.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:43 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:43.172+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:03.603637+0000 front 2024-08-31T06:25:03.603696+0000 (oldest deadline 2024-08-31T06:25:29.503703+0000) 2024-08-31T06:25:43.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:43 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:43.172+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-08-31T06:25:14.005741+0000 front 2024-08-31T06:25:14.005896+0000 (oldest deadline 2024-08-31T06:25:38.105488+0000) 2024-08-31T06:25:43.761 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:43 smithi155 systemd[1]: Stopping Ceph osd.10 for 673e4aa0-675d-11ef-bcd4-c7b262605968... 2024-08-31T06:25:43.761 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:43 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:43.720+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:43.762 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:43 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:43.720+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:43.762 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:43 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:43.720+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.499981+0000 front 2024-08-31T06:24:39.500178+0000 (oldest deadline 2024-08-31T06:25:04.799749+0000) 2024-08-31T06:25:43.762 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:43 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:43.720+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:44.800369+0000 front 2024-08-31T06:24:44.800594+0000 (oldest deadline 2024-08-31T06:25:15.400910+0000) 2024-08-31T06:25:43.762 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:43 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:43.720+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:55.401302+0000 front 2024-08-31T06:24:55.401342+0000 (oldest deadline 2024-08-31T06:25:20.701389+0000) 2024-08-31T06:25:43.762 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:43 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:43.720+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:06.002531+0000 front 2024-08-31T06:25:06.002753+0000 (oldest deadline 2024-08-31T06:25:27.702422+0000) 2024-08-31T06:25:43.762 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:43 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:43.720+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-08-31T06:25:14.604933+0000 front 2024-08-31T06:25:14.604805+0000 (oldest deadline 2024-08-31T06:25:36.904729+0000) 2024-08-31T06:25:43.762 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:43 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:43.758+0000 7fc6708d1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T06:25:43.762 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:43 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:43.758+0000 7fc6708d1640 -1 osd.10 106 *** Got signal Terminated *** 2024-08-31T06:25:43.762 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:43 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:43.758+0000 7fc6708d1640 -1 osd.10 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-31T06:25:44.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:44 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:44.134+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:44.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:44 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:44.134+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:44.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:44 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:44.134+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:44.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:44 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:44.134+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:58.403155+0000 front 2024-08-31T06:24:58.402997+0000 (oldest deadline 2024-08-31T06:25:19.502922+0000) 2024-08-31T06:25:44.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:44 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:44.134+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:03.603637+0000 front 2024-08-31T06:25:03.603696+0000 (oldest deadline 2024-08-31T06:25:29.503703+0000) 2024-08-31T06:25:44.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:44 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:44.134+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-08-31T06:25:14.005741+0000 front 2024-08-31T06:25:14.005896+0000 (oldest deadline 2024-08-31T06:25:38.105488+0000) 2024-08-31T06:25:45.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:44 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:44.749+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:45.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:44 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:44.749+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:45.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:44 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:44.749+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.499981+0000 front 2024-08-31T06:24:39.500178+0000 (oldest deadline 2024-08-31T06:25:04.799749+0000) 2024-08-31T06:25:45.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:44 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:44.749+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:44.800369+0000 front 2024-08-31T06:24:44.800594+0000 (oldest deadline 2024-08-31T06:25:15.400910+0000) 2024-08-31T06:25:45.123 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:44 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:44.749+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:55.401302+0000 front 2024-08-31T06:24:55.401342+0000 (oldest deadline 2024-08-31T06:25:20.701389+0000) 2024-08-31T06:25:45.123 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:44 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:44.749+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:06.002531+0000 front 2024-08-31T06:25:06.002753+0000 (oldest deadline 2024-08-31T06:25:27.702422+0000) 2024-08-31T06:25:45.123 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:44 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:44.749+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-08-31T06:25:14.604933+0000 front 2024-08-31T06:25:14.604805+0000 (oldest deadline 2024-08-31T06:25:36.904729+0000) 2024-08-31T06:25:45.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:45 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:45.105+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:45.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:45 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:45.105+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:45.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:45 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:45.105+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:45.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:45 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:45.105+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:58.403155+0000 front 2024-08-31T06:24:58.402997+0000 (oldest deadline 2024-08-31T06:25:19.502922+0000) 2024-08-31T06:25:45.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:45 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:45.105+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:03.603637+0000 front 2024-08-31T06:25:03.603696+0000 (oldest deadline 2024-08-31T06:25:29.503703+0000) 2024-08-31T06:25:45.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:45 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:45.105+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-08-31T06:25:14.005741+0000 front 2024-08-31T06:25:14.005896+0000 (oldest deadline 2024-08-31T06:25:38.105488+0000) 2024-08-31T06:25:46.023 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:45 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:45.756+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:46.023 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:45 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:45.756+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:46.023 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:45 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:45.756+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.499981+0000 front 2024-08-31T06:24:39.500178+0000 (oldest deadline 2024-08-31T06:25:04.799749+0000) 2024-08-31T06:25:46.023 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:45 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:45.756+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:44.800369+0000 front 2024-08-31T06:24:44.800594+0000 (oldest deadline 2024-08-31T06:25:15.400910+0000) 2024-08-31T06:25:46.023 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:45 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:45.756+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:55.401302+0000 front 2024-08-31T06:24:55.401342+0000 (oldest deadline 2024-08-31T06:25:20.701389+0000) 2024-08-31T06:25:46.023 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:45 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:45.756+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:06.002531+0000 front 2024-08-31T06:25:06.002753+0000 (oldest deadline 2024-08-31T06:25:27.702422+0000) 2024-08-31T06:25:46.023 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:45 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:45.756+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-08-31T06:25:14.604933+0000 front 2024-08-31T06:25:14.604805+0000 (oldest deadline 2024-08-31T06:25:36.904729+0000) 2024-08-31T06:25:46.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:46 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:46.145+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:46.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:46 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:46.145+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:46.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:46 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:46.145+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:46.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:46 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:46.145+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:58.403155+0000 front 2024-08-31T06:24:58.402997+0000 (oldest deadline 2024-08-31T06:25:19.502922+0000) 2024-08-31T06:25:46.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:46 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:46.145+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:03.603637+0000 front 2024-08-31T06:25:03.603696+0000 (oldest deadline 2024-08-31T06:25:29.503703+0000) 2024-08-31T06:25:46.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:46 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:46.145+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-08-31T06:25:14.005741+0000 front 2024-08-31T06:25:14.005896+0000 (oldest deadline 2024-08-31T06:25:38.105488+0000) 2024-08-31T06:25:46.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:46 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:46.145+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2024-08-31T06:25:23.406818+0000 front 2024-08-31T06:25:23.406891+0000 (oldest deadline 2024-08-31T06:25:45.706654+0000) 2024-08-31T06:25:47.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:46 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:46.751+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:47.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:46 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:46.751+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:47.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:46 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:46.751+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.499981+0000 front 2024-08-31T06:24:39.500178+0000 (oldest deadline 2024-08-31T06:25:04.799749+0000) 2024-08-31T06:25:47.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:46 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:46.751+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:44.800369+0000 front 2024-08-31T06:24:44.800594+0000 (oldest deadline 2024-08-31T06:25:15.400910+0000) 2024-08-31T06:25:47.123 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:46 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:46.751+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:55.401302+0000 front 2024-08-31T06:24:55.401342+0000 (oldest deadline 2024-08-31T06:25:20.701389+0000) 2024-08-31T06:25:47.123 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:46 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:46.751+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:06.002531+0000 front 2024-08-31T06:25:06.002753+0000 (oldest deadline 2024-08-31T06:25:27.702422+0000) 2024-08-31T06:25:47.123 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:46 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:46.751+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-08-31T06:25:14.604933+0000 front 2024-08-31T06:25:14.604805+0000 (oldest deadline 2024-08-31T06:25:36.904729+0000) 2024-08-31T06:25:47.375 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:47 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:47.121+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:47.375 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:47 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:47.121+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:47.375 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:47 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:47.121+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:47.375 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:47 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:47.121+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:58.403155+0000 front 2024-08-31T06:24:58.402997+0000 (oldest deadline 2024-08-31T06:25:19.502922+0000) 2024-08-31T06:25:47.375 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:47 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:47.121+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:03.603637+0000 front 2024-08-31T06:25:03.603696+0000 (oldest deadline 2024-08-31T06:25:29.503703+0000) 2024-08-31T06:25:47.375 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:47 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:47.121+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-08-31T06:25:14.005741+0000 front 2024-08-31T06:25:14.005896+0000 (oldest deadline 2024-08-31T06:25:38.105488+0000) 2024-08-31T06:25:47.375 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:47 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:47.121+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2024-08-31T06:25:23.406818+0000 front 2024-08-31T06:25:23.406891+0000 (oldest deadline 2024-08-31T06:25:45.706654+0000) 2024-08-31T06:25:48.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:47 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:47.754+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:48.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:47 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:47.754+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:48.123 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:47 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:47.754+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.499981+0000 front 2024-08-31T06:24:39.500178+0000 (oldest deadline 2024-08-31T06:25:04.799749+0000) 2024-08-31T06:25:48.123 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:47 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:47.754+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:44.800369+0000 front 2024-08-31T06:24:44.800594+0000 (oldest deadline 2024-08-31T06:25:15.400910+0000) 2024-08-31T06:25:48.123 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:47 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:47.754+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:55.401302+0000 front 2024-08-31T06:24:55.401342+0000 (oldest deadline 2024-08-31T06:25:20.701389+0000) 2024-08-31T06:25:48.123 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:47 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:47.754+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:06.002531+0000 front 2024-08-31T06:25:06.002753+0000 (oldest deadline 2024-08-31T06:25:27.702422+0000) 2024-08-31T06:25:48.123 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:47 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:47.754+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-08-31T06:25:14.604933+0000 front 2024-08-31T06:25:14.604805+0000 (oldest deadline 2024-08-31T06:25:36.904729+0000) 2024-08-31T06:25:48.472 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:48 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:48.128+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:48.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:48 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:48.128+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:48.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:48 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:48.128+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:48.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:48 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:48.128+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:58.403155+0000 front 2024-08-31T06:24:58.402997+0000 (oldest deadline 2024-08-31T06:25:19.502922+0000) 2024-08-31T06:25:48.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:48 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:48.128+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:03.603637+0000 front 2024-08-31T06:25:03.603696+0000 (oldest deadline 2024-08-31T06:25:29.503703+0000) 2024-08-31T06:25:48.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:48 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:48.128+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-08-31T06:25:14.005741+0000 front 2024-08-31T06:25:14.005896+0000 (oldest deadline 2024-08-31T06:25:38.105488+0000) 2024-08-31T06:25:48.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:48 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:48.128+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2024-08-31T06:25:23.406818+0000 front 2024-08-31T06:25:23.406891+0000 (oldest deadline 2024-08-31T06:25:45.706654+0000) 2024-08-31T06:25:49.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:48 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:48.743+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.296734+0000 front 2024-08-31T06:24:23.296748+0000 (oldest deadline 2024-08-31T06:24:46.196620+0000) 2024-08-31T06:25:49.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:48 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:48.743+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:32.498650+0000 front 2024-08-31T06:24:32.498626+0000 (oldest deadline 2024-08-31T06:24:54.198791+0000) 2024-08-31T06:25:49.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:48 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:48.743+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.499981+0000 front 2024-08-31T06:24:39.500178+0000 (oldest deadline 2024-08-31T06:25:04.799749+0000) 2024-08-31T06:25:49.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:48 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:48.743+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-08-31T06:24:44.800369+0000 front 2024-08-31T06:24:44.800594+0000 (oldest deadline 2024-08-31T06:25:15.400910+0000) 2024-08-31T06:25:49.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:48 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:48.743+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:55.401302+0000 front 2024-08-31T06:24:55.401342+0000 (oldest deadline 2024-08-31T06:25:20.701389+0000) 2024-08-31T06:25:49.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:48 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:48.743+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:06.002531+0000 front 2024-08-31T06:25:06.002753+0000 (oldest deadline 2024-08-31T06:25:27.702422+0000) 2024-08-31T06:25:49.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:48 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10[47842]: 2024-08-31T06:25:48.743+0000 7fc66cebc640 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-08-31T06:25:14.604933+0000 front 2024-08-31T06:25:14.604805+0000 (oldest deadline 2024-08-31T06:25:36.904729+0000) 2024-08-31T06:25:49.472 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:49 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:49.118+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:49.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:49 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:49.118+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:49.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:49 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:49.118+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:49.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:49 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:49.118+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:58.403155+0000 front 2024-08-31T06:24:58.402997+0000 (oldest deadline 2024-08-31T06:25:19.502922+0000) 2024-08-31T06:25:49.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:49 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:49.118+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:03.603637+0000 front 2024-08-31T06:25:03.603696+0000 (oldest deadline 2024-08-31T06:25:29.503703+0000) 2024-08-31T06:25:49.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:49 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:49.118+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-08-31T06:25:14.005741+0000 front 2024-08-31T06:25:14.005896+0000 (oldest deadline 2024-08-31T06:25:38.105488+0000) 2024-08-31T06:25:49.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:49 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:49.118+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2024-08-31T06:25:23.406818+0000 front 2024-08-31T06:25:23.406891+0000 (oldest deadline 2024-08-31T06:25:45.706654+0000) 2024-08-31T06:25:49.694 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:49 smithi155 podman[59578]: 2024-08-31 06:25:49.426337264 +0000 UTC m=+5.774908535 container died ab393bcfbeca60232ec35cabba1659738fa154b98342b82c1a4d57e69e0e2e06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-31T06:25:50.070 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:49 smithi155 podman[59578]: 2024-08-31 06:25:49.792099393 +0000 UTC m=+6.140670664 container cleanup ab393bcfbeca60232ec35cabba1659738fa154b98342b82c1a4d57e69e0e2e06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-08-31T06:25:50.071 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:49 smithi155 bash[59578]: ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10 2024-08-31T06:25:50.372 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:50 smithi155 podman[59605]: 2024-08-31 06:25:50.069187851 +0000 UTC m=+0.639796446 container remove ab393bcfbeca60232ec35cabba1659738fa154b98342b82c1a4d57e69e0e2e06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.1) 2024-08-31T06:25:50.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:50 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:50.077+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:50.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:50 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:50.077+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:50.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:50 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:50.077+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:50.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:50 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:50.077+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:58.403155+0000 front 2024-08-31T06:24:58.402997+0000 (oldest deadline 2024-08-31T06:25:19.502922+0000) 2024-08-31T06:25:50.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:50 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:50.077+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:03.603637+0000 front 2024-08-31T06:25:03.603696+0000 (oldest deadline 2024-08-31T06:25:29.503703+0000) 2024-08-31T06:25:50.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:50 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:50.077+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-08-31T06:25:14.005741+0000 front 2024-08-31T06:25:14.005896+0000 (oldest deadline 2024-08-31T06:25:38.105488+0000) 2024-08-31T06:25:50.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:50 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:50.077+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2024-08-31T06:25:23.406818+0000 front 2024-08-31T06:25:23.406891+0000 (oldest deadline 2024-08-31T06:25:45.706654+0000) 2024-08-31T06:25:50.698 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:50 smithi155 podman[59697]: 2024-08-31 06:25:50.377216399 +0000 UTC m=+0.111061945 container create 7366519fa2374ca88c1a41371ad077dbb6b96f6dca20d76fd6485a4ba248953a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.1, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-08-31T06:25:50.698 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:50 smithi155 podman[59697]: 2024-08-31 06:25:50.282017272 +0000 UTC m=+0.015862818 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T06:25:50.698 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:50 smithi155 podman[59697]: 2024-08-31 06:25:50.613341515 +0000 UTC m=+0.347187061 container init 7366519fa2374ca88c1a41371ad077dbb6b96f6dca20d76fd6485a4ba248953a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.1, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-08-31T06:25:50.698 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:50 smithi155 podman[59697]: 2024-08-31 06:25:50.616679238 +0000 UTC m=+0.350524784 container start 7366519fa2374ca88c1a41371ad077dbb6b96f6dca20d76fd6485a4ba248953a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.1, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-08-31T06:25:50.698 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:50 smithi155 podman[59697]: 2024-08-31 06:25:50.696410827 +0000 UTC m=+0.430256383 container attach 7366519fa2374ca88c1a41371ad077dbb6b96f6dca20d76fd6485a4ba248953a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.1) 2024-08-31T06:25:51.122 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 31 06:25:50 smithi155 podman[59697]: 2024-08-31 06:25:50.806732908 +0000 UTC m=+0.540578458 container died 7366519fa2374ca88c1a41371ad077dbb6b96f6dca20d76fd6485a4ba248953a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.1, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-31T06:25:51.443 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@osd.10.service' 2024-08-31T06:25:51.472 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:51 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:51.097+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:51.472 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:51 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:51.097+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:51.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:51 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:51.097+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:51.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:51 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:51.097+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:58.403155+0000 front 2024-08-31T06:24:58.402997+0000 (oldest deadline 2024-08-31T06:25:19.502922+0000) 2024-08-31T06:25:51.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:51 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:51.097+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:03.603637+0000 front 2024-08-31T06:25:03.603696+0000 (oldest deadline 2024-08-31T06:25:29.503703+0000) 2024-08-31T06:25:51.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:51 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:51.097+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-08-31T06:25:14.005741+0000 front 2024-08-31T06:25:14.005896+0000 (oldest deadline 2024-08-31T06:25:38.105488+0000) 2024-08-31T06:25:51.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:51 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:51.097+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2024-08-31T06:25:23.406818+0000 front 2024-08-31T06:25:23.406891+0000 (oldest deadline 2024-08-31T06:25:45.706654+0000) 2024-08-31T06:25:51.482 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T06:25:51.483 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-08-31T06:25:51.483 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-08-31T06:25:51.483 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@osd.11 2024-08-31T06:25:51.802 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:51 smithi184 systemd[1]: Stopping Ceph osd.11 for 673e4aa0-675d-11ef-bcd4-c7b262605968... 2024-08-31T06:25:52.074 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:51 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:51.800+0000 7faa93a39640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T06:25:52.074 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:51 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:51.800+0000 7faa93a39640 -1 osd.11 106 *** Got signal Terminated *** 2024-08-31T06:25:52.074 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:51 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:51.800+0000 7faa93a39640 -1 osd.11 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-31T06:25:52.074 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:52 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:52.072+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:52.074 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:52 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:52.072+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:52.074 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:52 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:52.072+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:52.074 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:52 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:52.072+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:58.403155+0000 front 2024-08-31T06:24:58.402997+0000 (oldest deadline 2024-08-31T06:25:19.502922+0000) 2024-08-31T06:25:52.074 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:52 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:52.072+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:03.603637+0000 front 2024-08-31T06:25:03.603696+0000 (oldest deadline 2024-08-31T06:25:29.503703+0000) 2024-08-31T06:25:52.075 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:52 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:52.072+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-08-31T06:25:14.005741+0000 front 2024-08-31T06:25:14.005896+0000 (oldest deadline 2024-08-31T06:25:38.105488+0000) 2024-08-31T06:25:52.075 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:52 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:52.072+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2024-08-31T06:25:23.406818+0000 front 2024-08-31T06:25:23.406891+0000 (oldest deadline 2024-08-31T06:25:45.706654+0000) 2024-08-31T06:25:53.472 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:53 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:53.075+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:53.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:53 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:53.075+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:53.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:53 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:53.075+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:53.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:53 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:53.075+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:58.403155+0000 front 2024-08-31T06:24:58.402997+0000 (oldest deadline 2024-08-31T06:25:19.502922+0000) 2024-08-31T06:25:53.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:53 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:53.075+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:03.603637+0000 front 2024-08-31T06:25:03.603696+0000 (oldest deadline 2024-08-31T06:25:29.503703+0000) 2024-08-31T06:25:53.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:53 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:53.075+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-08-31T06:25:14.005741+0000 front 2024-08-31T06:25:14.005896+0000 (oldest deadline 2024-08-31T06:25:38.105488+0000) 2024-08-31T06:25:53.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:53 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:53.075+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2024-08-31T06:25:23.406818+0000 front 2024-08-31T06:25:23.406891+0000 (oldest deadline 2024-08-31T06:25:45.706654+0000) 2024-08-31T06:25:54.472 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:54 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:54.077+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:54.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:54 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:54.077+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:54.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:54 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:54.077+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:54.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:54 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:54.077+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:58.403155+0000 front 2024-08-31T06:24:58.402997+0000 (oldest deadline 2024-08-31T06:25:19.502922+0000) 2024-08-31T06:25:54.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:54 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:54.077+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:03.603637+0000 front 2024-08-31T06:25:03.603696+0000 (oldest deadline 2024-08-31T06:25:29.503703+0000) 2024-08-31T06:25:54.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:54 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:54.077+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-08-31T06:25:14.005741+0000 front 2024-08-31T06:25:14.005896+0000 (oldest deadline 2024-08-31T06:25:38.105488+0000) 2024-08-31T06:25:54.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:54 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:54.077+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2024-08-31T06:25:23.406818+0000 front 2024-08-31T06:25:23.406891+0000 (oldest deadline 2024-08-31T06:25:45.706654+0000) 2024-08-31T06:25:54.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:54 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:54.077+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6820 osd.8 since back 2024-08-31T06:25:30.307901+0000 front 2024-08-31T06:25:30.307956+0000 (oldest deadline 2024-08-31T06:25:53.207863+0000) 2024-08-31T06:25:55.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:55 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:55.046+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:55.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:55 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:55.046+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:55.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:55 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:55.046+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:55.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:55 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:55.046+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:58.403155+0000 front 2024-08-31T06:24:58.402997+0000 (oldest deadline 2024-08-31T06:25:19.502922+0000) 2024-08-31T06:25:55.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:55 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:55.046+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:03.603637+0000 front 2024-08-31T06:25:03.603696+0000 (oldest deadline 2024-08-31T06:25:29.503703+0000) 2024-08-31T06:25:55.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:55 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:55.046+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-08-31T06:25:14.005741+0000 front 2024-08-31T06:25:14.005896+0000 (oldest deadline 2024-08-31T06:25:38.105488+0000) 2024-08-31T06:25:55.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:55 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:55.046+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2024-08-31T06:25:23.406818+0000 front 2024-08-31T06:25:23.406891+0000 (oldest deadline 2024-08-31T06:25:45.706654+0000) 2024-08-31T06:25:55.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:55 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:55.046+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6820 osd.8 since back 2024-08-31T06:25:30.307901+0000 front 2024-08-31T06:25:30.307956+0000 (oldest deadline 2024-08-31T06:25:53.207863+0000) 2024-08-31T06:25:56.472 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:56 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:56.057+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-08-31T06:24:23.497742+0000 front 2024-08-31T06:24:23.497580+0000 (oldest deadline 2024-08-31T06:24:48.197454+0000) 2024-08-31T06:25:56.472 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:56 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:56.057+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-31T06:24:31.098545+0000 front 2024-08-31T06:24:31.098611+0000 (oldest deadline 2024-08-31T06:24:54.598491+0000) 2024-08-31T06:25:56.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:56 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:56.057+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-08-31T06:24:39.299409+0000 front 2024-08-31T06:24:39.299510+0000 (oldest deadline 2024-08-31T06:25:06.300180+0000) 2024-08-31T06:25:56.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:56 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:56.057+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-31T06:24:58.403155+0000 front 2024-08-31T06:24:58.402997+0000 (oldest deadline 2024-08-31T06:25:19.502922+0000) 2024-08-31T06:25:56.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:56 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:56.057+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-08-31T06:25:03.603637+0000 front 2024-08-31T06:25:03.603696+0000 (oldest deadline 2024-08-31T06:25:29.503703+0000) 2024-08-31T06:25:56.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:56 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:56.057+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-08-31T06:25:14.005741+0000 front 2024-08-31T06:25:14.005896+0000 (oldest deadline 2024-08-31T06:25:38.105488+0000) 2024-08-31T06:25:56.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:56 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:56.057+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2024-08-31T06:25:23.406818+0000 front 2024-08-31T06:25:23.406891+0000 (oldest deadline 2024-08-31T06:25:45.706654+0000) 2024-08-31T06:25:56.473 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:56 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11[48850]: 2024-08-31T06:25:56.057+0000 7faa90024640 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.184:6820 osd.8 since back 2024-08-31T06:25:30.307901+0000 front 2024-08-31T06:25:30.307956+0000 (oldest deadline 2024-08-31T06:25:53.207863+0000) 2024-08-31T06:25:57.577 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:57 smithi184 podman[60754]: 2024-08-31 06:25:57.318052137 +0000 UTC m=+5.640293197 container died be3c3a4cfb89a195ce206128e946fd1dad1dc9f47f1c59d40051d34ec6244945 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.1, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-08-31T06:25:57.836 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:57 smithi184 podman[60754]: 2024-08-31 06:25:57.658666226 +0000 UTC m=+5.980907277 container cleanup be3c3a4cfb89a195ce206128e946fd1dad1dc9f47f1c59d40051d34ec6244945 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.1) 2024-08-31T06:25:57.836 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:57 smithi184 bash[60754]: ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11 2024-08-31T06:25:58.115 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:57 smithi184 podman[60771]: 2024-08-31 06:25:57.834997966 +0000 UTC m=+0.511846187 container remove be3c3a4cfb89a195ce206128e946fd1dad1dc9f47f1c59d40051d34ec6244945 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_BRANCH=HEAD, io.buildah.version=1.37.1, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-08-31T06:25:58.398 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:58 smithi184 podman[60858]: 2024-08-31 06:25:58.166435316 +0000 UTC m=+0.016017612 image pull 79f4cde5e36911a5dca995eeb98098c2c8aae81dd52a088ef2c867bc3363bad0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61 2024-08-31T06:25:58.399 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:58 smithi184 podman[60858]: 2024-08-31 06:25:58.28587351 +0000 UTC m=+0.135455800 container create 8eac6dd444fcdf6da4ebd6bc1df0313714bc193db16bfa765cbe55386b729c0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-08-31T06:25:58.673 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:58 smithi184 podman[60858]: 2024-08-31 06:25:58.505233441 +0000 UTC m=+0.354815727 container init 8eac6dd444fcdf6da4ebd6bc1df0313714bc193db16bfa765cbe55386b729c0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-08-31T06:25:58.673 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:58 smithi184 podman[60858]: 2024-08-31 06:25:58.50866138 +0000 UTC m=+0.358243670 container start 8eac6dd444fcdf6da4ebd6bc1df0313714bc193db16bfa765cbe55386b729c0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.1, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-31T06:25:58.673 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:58 smithi184 podman[60858]: 2024-08-31 06:25:58.580052285 +0000 UTC m=+0.429634574 container attach 8eac6dd444fcdf6da4ebd6bc1df0313714bc193db16bfa765cbe55386b729c0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.1, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-08-31T06:25:58.973 INFO:journalctl@ceph.osd.11.smithi184.stdout:Aug 31 06:25:58 smithi184 podman[60858]: 2024-08-31 06:25:58.70565841 +0000 UTC m=+0.555240713 container died 8eac6dd444fcdf6da4ebd6bc1df0313714bc193db16bfa765cbe55386b729c0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-osd-11-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.1, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-31T06:25:59.300 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@osd.11.service' 2024-08-31T06:25:59.339 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T06:25:59.339 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-08-31T06:25:59.340 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-08-31T06:25:59.340 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mds.a 2024-08-31T06:25:59.663 INFO:journalctl@ceph.mds.a.smithi016.stdout:Aug 31 06:25:59 smithi016 systemd[1]: Stopping Ceph mds.a for 673e4aa0-675d-11ef-bcd4-c7b262605968... 2024-08-31T06:25:59.994 INFO:journalctl@ceph.mds.a.smithi016.stdout:Aug 31 06:25:59 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-a[62398]: 2024-08-31T06:25:59.776+0000 7fa934000640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T06:25:59.994 INFO:journalctl@ceph.mds.a.smithi016.stdout:Aug 31 06:25:59 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-a[62398]: 2024-08-31T06:25:59.776+0000 7fa934000640 -1 mds.a *** got signal Terminated *** 2024-08-31T06:26:03.151 INFO:journalctl@ceph.mds.a.smithi016.stdout:Aug 31 06:26:02 smithi016 podman[108327]: 2024-08-31 06:26:02.872595916 +0000 UTC m=+3.239283260 container died d86cbad71303eb583dd1238165a42d1125b8a795203b96a1d7d40135ecf041dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-a, CEPH_POINT_RELEASE=, io.buildah.version=1.37.1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-08-31T06:26:03.413 INFO:journalctl@ceph.mds.a.smithi016.stdout:Aug 31 06:26:03 smithi016 podman[108327]: 2024-08-31 06:26:03.348676005 +0000 UTC m=+3.715363383 container cleanup d86cbad71303eb583dd1238165a42d1125b8a795203b96a1d7d40135ecf041dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, CEPH_POINT_RELEASE=, io.buildah.version=1.37.1, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-08-31T06:26:03.413 INFO:journalctl@ceph.mds.a.smithi016.stdout:Aug 31 06:26:03 smithi016 bash[108327]: ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-a 2024-08-31T06:26:03.903 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mds.a.service' 2024-08-31T06:26:03.952 INFO:journalctl@ceph.mds.a.smithi016.stdout:Aug 31 06:26:03 smithi016 podman[108351]: 2024-08-31 06:26:03.700184132 +0000 UTC m=+0.818031323 container remove d86cbad71303eb583dd1238165a42d1125b8a795203b96a1d7d40135ecf041dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.schema-version=1.0, io.buildah.version=1.37.1, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-08-31T06:26:03.952 INFO:journalctl@ceph.mds.a.smithi016.stdout:Aug 31 06:26:03 smithi016 systemd[1]: ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mds.a.service: Deactivated successfully. 2024-08-31T06:26:03.952 INFO:journalctl@ceph.mds.a.smithi016.stdout:Aug 31 06:26:03 smithi016 systemd[1]: Stopped Ceph mds.a for 673e4aa0-675d-11ef-bcd4-c7b262605968. 2024-08-31T06:26:03.952 INFO:journalctl@ceph.mds.a.smithi016.stdout:Aug 31 06:26:03 smithi016 systemd[1]: ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mds.a.service: Consumed 1.799s CPU time. 2024-08-31T06:26:03.976 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T06:26:03.977 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-08-31T06:26:03.977 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-08-31T06:26:03.977 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mds.d 2024-08-31T06:26:04.383 INFO:journalctl@ceph.mds.d.smithi016.stdout:Aug 31 06:26:04 smithi016 systemd[1]: Stopping Ceph mds.d for 673e4aa0-675d-11ef-bcd4-c7b262605968... 2024-08-31T06:26:04.662 INFO:journalctl@ceph.mds.d.smithi016.stdout:Aug 31 06:26:04 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-d[61209]: 2024-08-31T06:26:04.380+0000 7f89af400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T06:26:04.663 INFO:journalctl@ceph.mds.d.smithi016.stdout:Aug 31 06:26:04 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-d[61209]: 2024-08-31T06:26:04.380+0000 7f89af400640 -1 mds.d *** got signal Terminated *** 2024-08-31T06:26:08.663 INFO:journalctl@ceph.mds.d.smithi016.stdout:Aug 31 06:26:08 smithi016 podman[108457]: 2024-08-31 06:26:08.402276341 +0000 UTC m=+4.137247780 container died 08bfaf11110193eb744bb503c40d510ca1ee1967991958fa6e775e1ba947aef1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-d, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, CEPH_POINT_RELEASE=, io.buildah.version=1.37.1, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-31T06:26:09.163 INFO:journalctl@ceph.mds.d.smithi016.stdout:Aug 31 06:26:08 smithi016 podman[108457]: 2024-08-31 06:26:08.886285776 +0000 UTC m=+4.621257208 container cleanup 08bfaf11110193eb744bb503c40d510ca1ee1967991958fa6e775e1ba947aef1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-d, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.1, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611) 2024-08-31T06:26:09.163 INFO:journalctl@ceph.mds.d.smithi016.stdout:Aug 31 06:26:08 smithi016 bash[108457]: ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-d 2024-08-31T06:26:09.333 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mds.d.service' 2024-08-31T06:26:09.403 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T06:26:09.403 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-08-31T06:26:09.404 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-08-31T06:26:09.404 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mds.g 2024-08-31T06:26:09.804 INFO:journalctl@ceph.mds.g.smithi016.stdout:Aug 31 06:26:09 smithi016 systemd[1]: Stopping Ceph mds.g for 673e4aa0-675d-11ef-bcd4-c7b262605968... 2024-08-31T06:26:10.162 INFO:journalctl@ceph.mds.g.smithi016.stdout:Aug 31 06:26:09 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-g[62058]: 2024-08-31T06:26:09.801+0000 7f517c000640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T06:26:10.162 INFO:journalctl@ceph.mds.g.smithi016.stdout:Aug 31 06:26:09 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-g[62058]: 2024-08-31T06:26:09.801+0000 7f517c000640 -1 mds.g *** got signal Terminated *** 2024-08-31T06:26:11.618 INFO:journalctl@ceph.mds.g.smithi016.stdout:Aug 31 06:26:11 smithi016 podman[108577]: 2024-08-31 06:26:11.353172107 +0000 UTC m=+1.671894299 container died 52f052d8802b1224d354c9a1c6eb580fbff30ec58fc1dd8f7f5419d38d30af45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-g, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-08-31T06:26:11.871 INFO:journalctl@ceph.mds.g.smithi016.stdout:Aug 31 06:26:11 smithi016 podman[108577]: 2024-08-31 06:26:11.849218739 +0000 UTC m=+2.167940937 container cleanup 52f052d8802b1224d354c9a1c6eb580fbff30ec58fc1dd8f7f5419d38d30af45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-g, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1, maintainer=Guillaume Abrioux ) 2024-08-31T06:26:11.871 INFO:journalctl@ceph.mds.g.smithi016.stdout:Aug 31 06:26:11 smithi016 bash[108577]: ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-g 2024-08-31T06:26:12.345 INFO:journalctl@ceph.mds.g.smithi016.stdout:Aug 31 06:26:12 smithi016 podman[108588]: 2024-08-31 06:26:12.092360445 +0000 UTC m=+0.725265566 container remove 52f052d8802b1224d354c9a1c6eb580fbff30ec58fc1dd8f7f5419d38d30af45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-g, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.1) 2024-08-31T06:26:12.354 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mds.g.service' 2024-08-31T06:26:12.421 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T06:26:12.422 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-08-31T06:26:12.422 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-08-31T06:26:12.422 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mds.j 2024-08-31T06:26:12.663 INFO:journalctl@ceph.mds.j.smithi016.stdout:Aug 31 06:26:12 smithi016 systemd[1]: Stopping Ceph mds.j for 673e4aa0-675d-11ef-bcd4-c7b262605968... 2024-08-31T06:26:13.163 INFO:journalctl@ceph.mds.j.smithi016.stdout:Aug 31 06:26:12 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-j[60821]: 2024-08-31T06:26:12.805+0000 7f1c7f800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T06:26:13.163 INFO:journalctl@ceph.mds.j.smithi016.stdout:Aug 31 06:26:12 smithi016 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-j[60821]: 2024-08-31T06:26:12.805+0000 7f1c7f800640 -1 mds.j *** got signal Terminated *** 2024-08-31T06:26:17.089 INFO:journalctl@ceph.mds.j.smithi016.stdout:Aug 31 06:26:16 smithi016 podman[108698]: 2024-08-31 06:26:16.811703144 +0000 UTC m=+4.116015224 container died 66f8fc11e80cc8d679f8855935b9d4f2044b5380b713cd6946b0d10a57f72b42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-j, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-31T06:26:17.413 INFO:journalctl@ceph.mds.j.smithi016.stdout:Aug 31 06:26:17 smithi016 podman[108698]: 2024-08-31 06:26:17.286831097 +0000 UTC m=+4.591143174 container cleanup 66f8fc11e80cc8d679f8855935b9d4f2044b5380b713cd6946b0d10a57f72b42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-j, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.1, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_CLEAN=True) 2024-08-31T06:26:17.413 INFO:journalctl@ceph.mds.j.smithi016.stdout:Aug 31 06:26:17 smithi016 bash[108698]: ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-j 2024-08-31T06:26:17.724 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mds.j.service' 2024-08-31T06:26:17.792 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T06:26:17.793 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-08-31T06:26:17.793 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-08-31T06:26:17.793 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mds.b 2024-08-31T06:26:18.122 INFO:journalctl@ceph.mds.b.smithi155.stdout:Aug 31 06:26:17 smithi155 systemd[1]: Stopping Ceph mds.b for 673e4aa0-675d-11ef-bcd4-c7b262605968... 2024-08-31T06:26:18.391 INFO:journalctl@ceph.mds.b.smithi155.stdout:Aug 31 06:26:18 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-b[53353]: 2024-08-31T06:26:18.149+0000 7f67e2e4d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T06:26:18.391 INFO:journalctl@ceph.mds.b.smithi155.stdout:Aug 31 06:26:18 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-b[53353]: 2024-08-31T06:26:18.149+0000 7f67e2e4d640 -1 mds.b *** got signal Terminated *** 2024-08-31T06:26:19.544 INFO:journalctl@ceph.mds.b.smithi155.stdout:Aug 31 06:26:19 smithi155 podman[59799]: 2024-08-31 06:26:19.257841553 +0000 UTC m=+1.275250937 container died 0830ee5e87b02a28f53abf43e230caf1bad2b553efa1a1c51cfe85ee746b147e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-b, io.buildah.version=1.37.1, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-31T06:26:19.872 INFO:journalctl@ceph.mds.b.smithi155.stdout:Aug 31 06:26:19 smithi155 podman[59799]: 2024-08-31 06:26:19.641626709 +0000 UTC m=+1.659036082 container cleanup 0830ee5e87b02a28f53abf43e230caf1bad2b553efa1a1c51cfe85ee746b147e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, io.buildah.version=1.37.1, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-08-31T06:26:19.872 INFO:journalctl@ceph.mds.b.smithi155.stdout:Aug 31 06:26:19 smithi155 bash[59799]: ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-b 2024-08-31T06:26:20.128 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mds.b.service' 2024-08-31T06:26:20.164 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T06:26:20.165 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-08-31T06:26:20.165 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-08-31T06:26:20.165 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mds.e 2024-08-31T06:26:20.490 INFO:journalctl@ceph.mds.e.smithi155.stdout:Aug 31 06:26:20 smithi155 systemd[1]: Stopping Ceph mds.e for 673e4aa0-675d-11ef-bcd4-c7b262605968... 2024-08-31T06:26:20.872 INFO:journalctl@ceph.mds.e.smithi155.stdout:Aug 31 06:26:20 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-e[52736]: 2024-08-31T06:26:20.487+0000 7ff3d35c3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T06:26:20.872 INFO:journalctl@ceph.mds.e.smithi155.stdout:Aug 31 06:26:20 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-e[52736]: 2024-08-31T06:26:20.487+0000 7ff3d35c3640 -1 mds.e *** got signal Terminated *** 2024-08-31T06:26:24.056 INFO:journalctl@ceph.mds.e.smithi155.stdout:Aug 31 06:26:23 smithi155 podman[59933]: 2024-08-31 06:26:23.754467792 +0000 UTC m=+3.367413873 container died d7b37624caef48e6ec57b8a2c0ed5fdff2df36d6de8fe01c39e7d1a4a045e8dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-e, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.1, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-08-31T06:26:24.372 INFO:journalctl@ceph.mds.e.smithi155.stdout:Aug 31 06:26:24 smithi155 podman[59933]: 2024-08-31 06:26:24.15347225 +0000 UTC m=+3.766418318 container cleanup d7b37624caef48e6ec57b8a2c0ed5fdff2df36d6de8fe01c39e7d1a4a045e8dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-08-31T06:26:24.372 INFO:journalctl@ceph.mds.e.smithi155.stdout:Aug 31 06:26:24 smithi155 bash[59933]: ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-e 2024-08-31T06:26:24.630 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mds.e.service' 2024-08-31T06:26:24.664 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T06:26:24.664 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-08-31T06:26:24.665 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-08-31T06:26:24.665 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mds.h 2024-08-31T06:26:25.122 INFO:journalctl@ceph.mds.h.smithi155.stdout:Aug 31 06:26:24 smithi155 systemd[1]: Stopping Ceph mds.h for 673e4aa0-675d-11ef-bcd4-c7b262605968... 2024-08-31T06:26:25.122 INFO:journalctl@ceph.mds.h.smithi155.stdout:Aug 31 06:26:24 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-h[53037]: 2024-08-31T06:26:24.989+0000 7fcf716b4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T06:26:25.122 INFO:journalctl@ceph.mds.h.smithi155.stdout:Aug 31 06:26:24 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-h[53037]: 2024-08-31T06:26:24.989+0000 7fcf716b4640 -1 mds.h *** got signal Terminated *** 2024-08-31T06:26:27.609 INFO:journalctl@ceph.mds.h.smithi155.stdout:Aug 31 06:26:27 smithi155 podman[60053]: 2024-08-31 06:26:27.255052323 +0000 UTC m=+2.368340659 container died f74a94b68e1dfafe125401d06b7721b14505d3c3a6564eda2415ad977cd2ddc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-h, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.1, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-08-31T06:26:27.872 INFO:journalctl@ceph.mds.h.smithi155.stdout:Aug 31 06:26:27 smithi155 podman[60053]: 2024-08-31 06:26:27.707399233 +0000 UTC m=+2.820687569 container cleanup f74a94b68e1dfafe125401d06b7721b14505d3c3a6564eda2415ad977cd2ddc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-h, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.1) 2024-08-31T06:26:27.872 INFO:journalctl@ceph.mds.h.smithi155.stdout:Aug 31 06:26:27 smithi155 bash[60053]: ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-h 2024-08-31T06:26:28.138 INFO:journalctl@ceph.mds.h.smithi155.stdout:Aug 31 06:26:27 smithi155 podman[60065]: 2024-08-31 06:26:27.883254307 +0000 UTC m=+0.619767558 container remove f74a94b68e1dfafe125401d06b7721b14505d3c3a6564eda2415ad977cd2ddc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-h, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-08-31T06:26:28.142 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mds.h.service' 2024-08-31T06:26:28.174 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T06:26:28.174 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-08-31T06:26:28.175 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-08-31T06:26:28.175 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mds.k 2024-08-31T06:26:28.495 INFO:journalctl@ceph.mds.k.smithi155.stdout:Aug 31 06:26:28 smithi155 systemd[1]: Stopping Ceph mds.k for 673e4aa0-675d-11ef-bcd4-c7b262605968... 2024-08-31T06:26:28.854 INFO:journalctl@ceph.mds.k.smithi155.stdout:Aug 31 06:26:28 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-k[52436]: 2024-08-31T06:26:28.493+0000 7fca2d350640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T06:26:28.854 INFO:journalctl@ceph.mds.k.smithi155.stdout:Aug 31 06:26:28 smithi155 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-k[52436]: 2024-08-31T06:26:28.493+0000 7fca2d350640 -1 mds.k *** got signal Terminated *** 2024-08-31T06:26:32.787 INFO:journalctl@ceph.mds.k.smithi155.stdout:Aug 31 06:26:32 smithi155 podman[60174]: 2024-08-31 06:26:32.508536895 +0000 UTC m=+4.114776518 container died e3c24c5dbb468e1b40ec87a9492dd7c429f41ae029d0e5cf4b3c28e6f89d8e87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-k, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.1, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2) 2024-08-31T06:26:33.122 INFO:journalctl@ceph.mds.k.smithi155.stdout:Aug 31 06:26:32 smithi155 podman[60174]: 2024-08-31 06:26:32.851901152 +0000 UTC m=+4.458140807 container cleanup e3c24c5dbb468e1b40ec87a9492dd7c429f41ae029d0e5cf4b3c28e6f89d8e87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-k, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.1) 2024-08-31T06:26:33.122 INFO:journalctl@ceph.mds.k.smithi155.stdout:Aug 31 06:26:32 smithi155 bash[60174]: ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-k 2024-08-31T06:26:33.122 INFO:journalctl@ceph.mds.k.smithi155.stdout:Aug 31 06:26:32 smithi155 podman[60187]: 2024-08-31 06:26:32.994287357 +0000 UTC m=+0.483650197 container remove e3c24c5dbb468e1b40ec87a9492dd7c429f41ae029d0e5cf4b3c28e6f89d8e87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-k, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.1, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.license=GPLv2) 2024-08-31T06:26:33.211 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mds.k.service' 2024-08-31T06:26:33.243 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T06:26:33.244 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-08-31T06:26:33.244 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-08-31T06:26:33.244 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mds.c 2024-08-31T06:26:33.722 INFO:journalctl@ceph.mds.c.smithi184.stdout:Aug 31 06:26:33 smithi184 systemd[1]: Stopping Ceph mds.c for 673e4aa0-675d-11ef-bcd4-c7b262605968... 2024-08-31T06:26:34.222 INFO:journalctl@ceph.mds.c.smithi184.stdout:Aug 31 06:26:33 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-c[54092]: 2024-08-31T06:26:33.740+0000 7f11f121a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T06:26:34.222 INFO:journalctl@ceph.mds.c.smithi184.stdout:Aug 31 06:26:33 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-c[54092]: 2024-08-31T06:26:33.740+0000 7f11f121a640 -1 mds.c *** got signal Terminated *** 2024-08-31T06:26:35.722 INFO:journalctl@ceph.mds.c.smithi184.stdout:Aug 31 06:26:35 smithi184 podman[60959]: 2024-08-31 06:26:35.260832533 +0000 UTC m=+1.826962686 container died 8f0a89efbce5693100751c23421504dec810209bbd1429a80031c3b55d8b2570 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-c, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.1, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-08-31T06:26:36.201 INFO:journalctl@ceph.mds.c.smithi184.stdout:Aug 31 06:26:35 smithi184 podman[60959]: 2024-08-31 06:26:35.957155142 +0000 UTC m=+2.523285294 container cleanup 8f0a89efbce5693100751c23421504dec810209bbd1429a80031c3b55d8b2570 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-c, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.1, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True) 2024-08-31T06:26:36.202 INFO:journalctl@ceph.mds.c.smithi184.stdout:Aug 31 06:26:35 smithi184 bash[60959]: ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-c 2024-08-31T06:26:36.334 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mds.c.service' 2024-08-31T06:26:36.369 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T06:26:36.370 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-08-31T06:26:36.370 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-08-31T06:26:36.370 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mds.f 2024-08-31T06:26:36.472 INFO:journalctl@ceph.mds.f.smithi184.stdout:Aug 31 06:26:36 smithi184 systemd[1]: Stopping Ceph mds.f for 673e4aa0-675d-11ef-bcd4-c7b262605968... 2024-08-31T06:26:36.972 INFO:journalctl@ceph.mds.f.smithi184.stdout:Aug 31 06:26:36 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-f[53791]: 2024-08-31T06:26:36.686+0000 7fb0ca0a1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T06:26:36.972 INFO:journalctl@ceph.mds.f.smithi184.stdout:Aug 31 06:26:36 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-f[53791]: 2024-08-31T06:26:36.686+0000 7fb0ca0a1640 -1 mds.f *** got signal Terminated *** 2024-08-31T06:26:39.587 INFO:journalctl@ceph.mds.f.smithi184.stdout:Aug 31 06:26:39 smithi184 podman[61095]: 2024-08-31 06:26:39.262540026 +0000 UTC m=+2.675474041 container died d162f2b072e8c11a0e1251a7719a190e6eb0b06913870801d0a46fc0cc196b18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-f, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.1, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611) 2024-08-31T06:26:39.588 INFO:journalctl@ceph.mds.f.smithi184.stdout:Aug 31 06:26:39 smithi184 podman[61095]: 2024-08-31 06:26:39.586011056 +0000 UTC m=+2.998945070 container cleanup d162f2b072e8c11a0e1251a7719a190e6eb0b06913870801d0a46fc0cc196b18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-f, io.buildah.version=1.37.1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731) 2024-08-31T06:26:39.972 INFO:journalctl@ceph.mds.f.smithi184.stdout:Aug 31 06:26:39 smithi184 bash[61095]: ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-f 2024-08-31T06:26:39.972 INFO:journalctl@ceph.mds.f.smithi184.stdout:Aug 31 06:26:39 smithi184 podman[61107]: 2024-08-31 06:26:39.787347383 +0000 UTC m=+0.521646067 container remove d162f2b072e8c11a0e1251a7719a190e6eb0b06913870801d0a46fc0cc196b18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.1, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-31T06:26:40.020 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mds.f.service' 2024-08-31T06:26:40.053 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T06:26:40.053 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-08-31T06:26:40.054 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-08-31T06:26:40.054 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mds.i 2024-08-31T06:26:40.374 INFO:journalctl@ceph.mds.i.smithi184.stdout:Aug 31 06:26:40 smithi184 systemd[1]: Stopping Ceph mds.i for 673e4aa0-675d-11ef-bcd4-c7b262605968... 2024-08-31T06:26:40.722 INFO:journalctl@ceph.mds.i.smithi184.stdout:Aug 31 06:26:40 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-i[54398]: 2024-08-31T06:26:40.372+0000 7f1c0a345640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T06:26:40.722 INFO:journalctl@ceph.mds.i.smithi184.stdout:Aug 31 06:26:40 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-i[54398]: 2024-08-31T06:26:40.372+0000 7f1c0a345640 -1 mds.i *** got signal Terminated *** 2024-08-31T06:26:43.626 INFO:journalctl@ceph.mds.i.smithi184.stdout:Aug 31 06:26:43 smithi184 podman[61216]: 2024-08-31 06:26:43.258990264 +0000 UTC m=+2.990864416 container died bbb227bfc2fad76696427cb902cf9082995e793f404e274caa6ea4f89795c22d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-i, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.1, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True) 2024-08-31T06:26:43.909 INFO:journalctl@ceph.mds.i.smithi184.stdout:Aug 31 06:26:43 smithi184 podman[61216]: 2024-08-31 06:26:43.706520732 +0000 UTC m=+3.438394884 container cleanup bbb227bfc2fad76696427cb902cf9082995e793f404e274caa6ea4f89795c22d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.1, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-31T06:26:43.909 INFO:journalctl@ceph.mds.i.smithi184.stdout:Aug 31 06:26:43 smithi184 bash[61216]: ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-i 2024-08-31T06:26:44.130 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mds.i.service' 2024-08-31T06:26:44.162 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T06:26:44.163 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-08-31T06:26:44.163 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-08-31T06:26:44.163 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mds.l 2024-08-31T06:26:44.472 INFO:journalctl@ceph.mds.l.smithi184.stdout:Aug 31 06:26:44 smithi184 systemd[1]: Stopping Ceph mds.l for 673e4aa0-675d-11ef-bcd4-c7b262605968... 2024-08-31T06:26:44.972 INFO:journalctl@ceph.mds.l.smithi184.stdout:Aug 31 06:26:44 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-l[53489]: 2024-08-31T06:26:44.476+0000 7fddbc384640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-31T06:26:44.972 INFO:journalctl@ceph.mds.l.smithi184.stdout:Aug 31 06:26:44 smithi184 ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-l[53489]: 2024-08-31T06:26:44.476+0000 7fddbc384640 -1 mds.l *** got signal Terminated *** 2024-08-31T06:26:46.021 INFO:journalctl@ceph.mds.l.smithi184.stdout:Aug 31 06:26:45 smithi184 podman[61338]: 2024-08-31 06:26:45.736943573 +0000 UTC m=+1.354972315 container died 2eff53ca9a134cc00ac9f4c8a59ad4abe94631e4705031ccbb871707b9a76dfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-l, org.label-schema.vendor=CentOS, io.buildah.version=1.37.1, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0) 2024-08-31T06:26:46.421 INFO:journalctl@ceph.mds.l.smithi184.stdout:Aug 31 06:26:46 smithi184 podman[61338]: 2024-08-31 06:26:46.24359449 +0000 UTC m=+1.861623233 container cleanup 2eff53ca9a134cc00ac9f4c8a59ad4abe94631e4705031ccbb871707b9a76dfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e18f78946a0f2b5a73edfdd8c722a31841f880199baf0b88caf17994ffbc1a61, name=ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-l, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.1, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-20240828.085847-main-debug-d19e611, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-08-31T06:26:46.421 INFO:journalctl@ceph.mds.l.smithi184.stdout:Aug 31 06:26:46 smithi184 bash[61338]: ceph-673e4aa0-675d-11ef-bcd4-c7b262605968-mds-l 2024-08-31T06:26:46.545 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-673e4aa0-675d-11ef-bcd4-c7b262605968@mds.l.service' 2024-08-31T06:26:46.577 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-31T06:26:46.578 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-08-31T06:26:46.578 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 --force --keep-logs 2024-08-31T06:26:46.861 INFO:teuthology.orchestra.run.smithi016.stdout:Deleting cluster with fsid: 673e4aa0-675d-11ef-bcd4-c7b262605968 2024-08-31T06:26:50.959 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 --force --keep-logs 2024-08-31T06:26:51.161 INFO:teuthology.orchestra.run.smithi155.stdout:Deleting cluster with fsid: 673e4aa0-675d-11ef-bcd4-c7b262605968 2024-08-31T06:26:53.647 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 --force --keep-logs 2024-08-31T06:26:53.850 INFO:teuthology.orchestra.run.smithi184.stdout:Deleting cluster with fsid: 673e4aa0-675d-11ef-bcd4-c7b262605968 2024-08-31T06:26:56.294 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-31T06:26:56.359 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-31T06:26:56.393 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-31T06:26:56.420 INFO:tasks.cephadm:Archiving crash dumps... 2024-08-31T06:26:56.422 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/crash to /home/teuthworker/archive/mchangir-2024-08-30_08:30:37-fs-wip-mchangir-testing-20240828.085847-main-debug-distro-default-smithi/7881377/remote/smithi016/crash 2024-08-31T06:26:56.422 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/crash -- . 2024-08-31T06:26:56.488 INFO:teuthology.orchestra.run.smithi016.stderr:tar: /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/crash: Cannot open: No such file or directory 2024-08-31T06:26:56.488 INFO:teuthology.orchestra.run.smithi016.stderr:tar: Error is not recoverable: exiting now 2024-08-31T06:26:56.491 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/crash to /home/teuthworker/archive/mchangir-2024-08-30_08:30:37-fs-wip-mchangir-testing-20240828.085847-main-debug-distro-default-smithi/7881377/remote/smithi155/crash 2024-08-31T06:26:56.492 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/crash -- . 2024-08-31T06:26:56.607 INFO:teuthology.orchestra.run.smithi155.stderr:tar: /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/crash: Cannot open: No such file or directory 2024-08-31T06:26:56.607 INFO:teuthology.orchestra.run.smithi155.stderr:tar: Error is not recoverable: exiting now 2024-08-31T06:26:56.609 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/crash to /home/teuthworker/archive/mchangir-2024-08-30_08:30:37-fs-wip-mchangir-testing-20240828.085847-main-debug-distro-default-smithi/7881377/remote/smithi184/crash 2024-08-31T06:26:56.610 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/crash -- . 2024-08-31T06:26:56.651 INFO:teuthology.orchestra.run.smithi184.stderr:tar: /var/lib/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/crash: Cannot open: No such file or directory 2024-08-31T06:26:56.651 INFO:teuthology.orchestra.run.smithi184.stderr:tar: Error is not recoverable: exiting now 2024-08-31T06:26:56.652 INFO:tasks.cephadm:Checking cluster log for badness... 2024-08-31T06:26:56.653 DEBUG:teuthology.orchestra.run.smithi016:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-08-31T06:26:56.731 INFO:teuthology.orchestra.run.smithi016.stdout:2024-08-31T06:10:54.098841+0000 mds.b (mds.0) 29 : cluster [WRN] Scrub error on inode 0x10000000019 (/volumes/qa/sv_1/ba5423bc-5c83-4d64-ae1c-ad8850110ce4/postgres/data/pg_stat_tmp) see mds.b log and `damage ls` output for details 2024-08-31T06:26:56.732 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-08-31T06:26:56.733 DEBUG:teuthology.orchestra.run.smithi016:> sudo egrep '\[SEC\]' /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-08-31T06:26:56.817 DEBUG:teuthology.orchestra.run.smithi016:> sudo egrep '\[ERR\]' /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-08-31T06:26:56.901 DEBUG:teuthology.orchestra.run.smithi016:> sudo egrep '\[WRN\]' /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-08-31T06:26:56.981 INFO:teuthology.orchestra.run.smithi016.stdout:2024-08-31T06:10:54.098841+0000 mds.b (mds.0) 29 : cluster [WRN] Scrub error on inode 0x10000000019 (/volumes/qa/sv_1/ba5423bc-5c83-4d64-ae1c-ad8850110ce4/postgres/data/pg_stat_tmp) see mds.b log and `damage ls` output for details 2024-08-31T06:26:56.984 INFO:tasks.cephadm:Compressing logs... 2024-08-31T06:26:56.985 DEBUG:teuthology.orchestra.run.smithi016:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-31T06:26:56.989 DEBUG:teuthology.orchestra.run.smithi155:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-31T06:26:56.992 DEBUG:teuthology.orchestra.run.smithi184:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-31T06:26:57.017 INFO:teuthology.orchestra.run.smithi184.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-volume.log 2024-08-31T06:26:57.017 INFO:teuthology.orchestra.run.smithi184.stderr:: No such file or directory 2024-08-31T06:26:57.017 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph.audit.log 2024-08-31T06:26:57.018 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mds.i.log 2024-08-31T06:26:57.018 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mon.c.log 2024-08-31T06:26:57.020 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph.cephadm.log 2024-08-31T06:26:57.020 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-osd.11.log 2024-08-31T06:26:57.020 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mon.c.log: /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-osd.2.log 2024-08-31T06:26:57.020 INFO:teuthology.orchestra.run.smithi184.stderr: 81.4% -- replaced with /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph.cephadm.log.gz 2024-08-31T06:26:57.020 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-osd.5.log 2024-08-31T06:26:57.021 INFO:teuthology.orchestra.run.smithi155.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-osd.10.log 2024-08-31T06:26:57.021 INFO:teuthology.orchestra.run.smithi155.stderr:: No such file or directory 2024-08-31T06:26:57.021 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-osd.1.log 2024-08-31T06:26:57.021 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-osd.7.log 2024-08-31T06:26:57.021 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-osd.10.log: /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-volume.log 2024-08-31T06:26:57.022 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mds.b.log 2024-08-31T06:26:57.022 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mds.k.log 2024-08-31T06:26:57.022 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-volume.log: /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-osd.4.log 2024-08-31T06:26:57.022 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph.audit.log 2024-08-31T06:26:57.023 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph.log 2024-08-31T06:26:57.023 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mds.l.log 2024-08-31T06:26:57.023 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph.log: 90.1% -- replaced with /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph.audit.log.gz 2024-08-31T06:26:57.023 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-osd.8.log 2024-08-31T06:26:57.024 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mds.l.log: 86.8% -- replaced with /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph.log.gz 2024-08-31T06:26:57.025 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mgr.z.log 2024-08-31T06:26:57.025 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-osd.4.log: /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph.audit.log: 90.1% -- replaced with /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph.audit.log.gz 2024-08-31T06:26:57.025 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph.cephadm.log 2024-08-31T06:26:57.026 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mgr.y.log 2024-08-31T06:26:57.026 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph.cephadm.log: 81.4% -- replaced with /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph.cephadm.log.gz 2024-08-31T06:26:57.027 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph.log 2024-08-31T06:26:57.029 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-osd.8.log: 90.8% -- replaced with /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-volume.log.gz 2024-08-31T06:26:57.030 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mgr.y.log: 94.9% -- replaced with /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mds.k.log.gz 2024-08-31T06:26:57.031 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mds.h.log 2024-08-31T06:26:57.033 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mds.c.log 2024-08-31T06:26:57.033 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mgr.z.log: 94.9% -- replaced with /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mds.l.log.gz 2024-08-31T06:26:57.033 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mds.f.log 2024-08-31T06:26:57.034 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph.log: 91.0% -- replaced with /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-volume.log.gz 2024-08-31T06:26:57.035 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mon.b.log 2024-08-31T06:26:57.035 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mds.h.log: 86.8% -- replaced with /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph.log.gz 2024-08-31T06:26:57.037 INFO:teuthology.orchestra.run.smithi155.stderr: 92.9% -- replaced with /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mgr.y.log.gz 2024-08-31T06:26:57.037 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mds.e.log 2024-08-31T06:26:57.038 INFO:teuthology.orchestra.run.smithi016.stderr:find: ‘/var/log/rbd-target-api’gzip: No such file or directory -5 2024-08-31T06:26:57.039 INFO:teuthology.orchestra.run.smithi016.stderr: --verbose -- /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-osd.0.log 2024-08-31T06:26:57.039 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mds.c.log: 93.0% -- replaced with /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mgr.z.log.gz 2024-08-31T06:26:57.039 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-osd.9.log 2024-08-31T06:26:57.040 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-volume.log 2024-08-31T06:26:57.040 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-08-31T06:26:57.041 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph.audit.log 2024-08-31T06:26:57.042 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mds.a.log 2024-08-31T06:26:57.042 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-08-31T06:26:57.043 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph.cephadm.log 2024-08-31T06:26:57.044 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mon.a.log 2024-08-31T06:26:57.046 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mds.g.log 2024-08-31T06:26:57.046 INFO:teuthology.orchestra.run.smithi016.stderr: 84.2% -- replaced with /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph.cephadm.log.gz 2024-08-31T06:26:57.046 INFO:teuthology.orchestra.run.smithi016.stderr: 90.0% -- replaced with /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph.audit.log.gz 2024-08-31T06:26:57.047 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-osd.6.log 2024-08-31T06:26:57.048 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph.log 2024-08-31T06:26:57.052 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-08-31T06:26:57.052 INFO:teuthology.orchestra.run.smithi155.stderr: 94.9% -- replaced with /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mds.e.log.gz 2024-08-31T06:26:57.052 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mds.g.log: /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-osd.6.log: /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mds.d.log 2024-08-31T06:26:57.054 INFO:teuthology.orchestra.run.smithi016.stderr: 94.9% -- replaced with /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mds.a.log.gz 2024-08-31T06:26:57.055 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mgr.x.log 2024-08-31T06:26:57.055 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-08-31T06:26:57.056 INFO:teuthology.orchestra.run.smithi016.stderr: 90.8% -- replaced with /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-volume.log.gz 2024-08-31T06:26:57.056 INFO:teuthology.orchestra.run.smithi016.stderr: 86.9% -- replaced with /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph.log.gz 2024-08-31T06:26:57.057 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mds.j.log 2024-08-31T06:26:57.059 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-osd.3.log 2024-08-31T06:26:57.064 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mds.j.log: /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-08-31T06:26:57.066 INFO:teuthology.orchestra.run.smithi016.stderr: 94.8% -- replaced with /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mds.d.log.gz 2024-08-31T06:26:57.079 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/cephadm.log: 94.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-08-31T06:26:57.084 INFO:teuthology.orchestra.run.smithi016.stderr: 94.9% -- replaced with /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mds.j.log.gz 2024-08-31T06:26:57.323 INFO:teuthology.orchestra.run.smithi184.stderr: 93.3% -- replaced with /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mds.f.log.gz 2024-08-31T06:26:57.380 INFO:teuthology.orchestra.run.smithi184.stderr: 93.5% -- replaced with /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mds.c.log.gz 2024-08-31T06:26:57.557 INFO:teuthology.orchestra.run.smithi016.stderr: 90.3% -- replaced with /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mgr.x.log.gz 2024-08-31T06:26:57.644 INFO:teuthology.orchestra.run.smithi155.stderr: 93.7% -- replaced with /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mds.h.log.gz 2024-08-31T06:26:57.710 INFO:teuthology.orchestra.run.smithi184.stderr: 91.5% -- replaced with /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mon.c.log.gz 2024-08-31T06:26:57.802 INFO:teuthology.orchestra.run.smithi155.stderr: 91.4% -- replaced with /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mon.b.log.gz 2024-08-31T06:26:58.819 INFO:teuthology.orchestra.run.smithi016.stderr: 90.3% -- replaced with /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mon.a.log.gz 2024-08-31T06:27:03.266 INFO:teuthology.orchestra.run.smithi184.stderr: 94.9% -- replaced with /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mds.i.log.gz 2024-08-31T06:27:03.641 INFO:teuthology.orchestra.run.smithi016.stderr: 94.5% -- replaced with /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mds.g.log.gz 2024-08-31T06:27:05.484 INFO:teuthology.orchestra.run.smithi155.stderr: 94.3% -- replaced with /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-mds.b.log.gz 2024-08-31T06:27:24.915 INFO:teuthology.orchestra.run.smithi016.stderr: 91.6% -- replaced with /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-osd.9.log.gz 2024-08-31T06:27:31.757 INFO:teuthology.orchestra.run.smithi184.stderr: 91.5% -- replaced with /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-osd.8.log.gz 2024-08-31T06:27:36.282 INFO:teuthology.orchestra.run.smithi155.stderr: 91.8% -- replaced with /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-osd.4.log.gz 2024-08-31T06:27:38.419 INFO:teuthology.orchestra.run.smithi184.stderr: 91.3% -- replaced with /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-osd.11.log.gz 2024-08-31T06:27:39.211 INFO:teuthology.orchestra.run.smithi016.stderr: 91.4% -- replaced with /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-osd.6.log.gz 2024-08-31T06:27:41.429 INFO:teuthology.orchestra.run.smithi155.stderr: 91.5% -- replaced with /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-osd.10.log.gz 2024-08-31T06:27:42.753 INFO:teuthology.orchestra.run.smithi184.stderr: 91.6% -- replaced with /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-osd.5.log.gz 2024-08-31T06:27:43.434 INFO:teuthology.orchestra.run.smithi184.stderr: 91.5% -- replaced with /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-osd.2.log.gz 2024-08-31T06:27:43.436 INFO:teuthology.orchestra.run.smithi184.stderr: 2024-08-31T06:27:43.436 INFO:teuthology.orchestra.run.smithi184.stderr:real 0m46.429s 2024-08-31T06:27:43.436 INFO:teuthology.orchestra.run.smithi184.stderr:user 2m50.231s 2024-08-31T06:27:43.436 INFO:teuthology.orchestra.run.smithi184.stderr:sys 0m5.287s 2024-08-31T06:27:47.039 INFO:teuthology.orchestra.run.smithi155.stderr: 91.5% -- replaced with /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-osd.1.log.gz 2024-08-31T06:27:52.833 INFO:teuthology.orchestra.run.smithi016.stderr: 91.4% -- replaced with /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-osd.3.log.gz 2024-08-31T06:27:54.143 INFO:teuthology.orchestra.run.smithi155.stderr: 91.4% -- replaced with /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-osd.7.log.gz 2024-08-31T06:27:54.145 INFO:teuthology.orchestra.run.smithi155.stderr: 2024-08-31T06:27:54.145 INFO:teuthology.orchestra.run.smithi155.stderr:real 0m57.139s 2024-08-31T06:27:54.146 INFO:teuthology.orchestra.run.smithi155.stderr:user 3m13.939s 2024-08-31T06:27:54.146 INFO:teuthology.orchestra.run.smithi155.stderr:sys 0m6.057s 2024-08-31T06:27:57.479 INFO:teuthology.orchestra.run.smithi016.stderr: 91.3% -- replaced with /var/log/ceph/673e4aa0-675d-11ef-bcd4-c7b262605968/ceph-osd.0.log.gz 2024-08-31T06:27:57.482 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-08-31T06:27:57.482 INFO:teuthology.orchestra.run.smithi016.stderr:real 1m0.465s 2024-08-31T06:27:57.482 INFO:teuthology.orchestra.run.smithi016.stderr:user 2m59.996s 2024-08-31T06:27:57.482 INFO:teuthology.orchestra.run.smithi016.stderr:sys 0m14.063s 2024-08-31T06:27:57.484 INFO:tasks.cephadm:Archiving logs... 2024-08-31T06:27:57.484 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-08-30_08:30:37-fs-wip-mchangir-testing-20240828.085847-main-debug-distro-default-smithi/7881377/remote/smithi016/log 2024-08-31T06:27:57.485 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-31T06:28:26.422 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-08-30_08:30:37-fs-wip-mchangir-testing-20240828.085847-main-debug-distro-default-smithi/7881377/remote/smithi155/log 2024-08-31T06:28:26.423 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-31T06:28:58.662 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-08-30_08:30:37-fs-wip-mchangir-testing-20240828.085847-main-debug-distro-default-smithi/7881377/remote/smithi184/log 2024-08-31T06:28:58.663 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-31T06:29:27.192 INFO:tasks.cephadm:Removing cluster... 2024-08-31T06:29:27.193 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 --force 2024-08-31T06:29:27.477 INFO:teuthology.orchestra.run.smithi016.stdout:Deleting cluster with fsid: 673e4aa0-675d-11ef-bcd4-c7b262605968 2024-08-31T06:29:28.660 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 --force 2024-08-31T06:29:28.865 INFO:teuthology.orchestra.run.smithi155.stdout:Deleting cluster with fsid: 673e4aa0-675d-11ef-bcd4-c7b262605968 2024-08-31T06:29:29.498 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 673e4aa0-675d-11ef-bcd4-c7b262605968 --force 2024-08-31T06:29:29.702 INFO:teuthology.orchestra.run.smithi184.stdout:Deleting cluster with fsid: 673e4aa0-675d-11ef-bcd4-c7b262605968 2024-08-31T06:29:30.215 INFO:tasks.cephadm:Removing cephadm ... 2024-08-31T06:29:30.215 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf /home/ubuntu/cephtest/cephadm 2024-08-31T06:29:30.252 DEBUG:teuthology.orchestra.run.smithi155:> rm -rf /home/ubuntu/cephtest/cephadm 2024-08-31T06:29:30.268 DEBUG:teuthology.orchestra.run.smithi184:> rm -rf /home/ubuntu/cephtest/cephadm 2024-08-31T06:29:30.284 INFO:tasks.cephadm:Teardown complete 2024-08-31T06:29:30.284 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-08-31T06:29:30.296 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-08-31T06:29:30.296 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-31T06:29:30.300 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-31T06:29:30.310 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-31T06:29:30.422 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-08-31T06:29:30.432 INFO:teuthology.task.clock:Checking final clock skew... 2024-08-31T06:29:30.432 DEBUG:teuthology.orchestra.run.smithi016:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-31T06:29:30.436 DEBUG:teuthology.orchestra.run.smithi155:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-31T06:29:30.439 DEBUG:teuthology.orchestra.run.smithi184:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-31T06:29:30.451 INFO:teuthology.orchestra.run.smithi155.stderr:bash: line 1: ntpq: command not found 2024-08-31T06:29:30.453 INFO:teuthology.orchestra.run.smithi184.stderr:bash: line 1: ntpq: command not found 2024-08-31T06:29:30.456 INFO:teuthology.orchestra.run.smithi155.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-31T06:29:30.456 INFO:teuthology.orchestra.run.smithi155.stdout:=============================================================================== 2024-08-31T06:29:30.456 INFO:teuthology.orchestra.run.smithi155.stdout:^* hv01.front.sepia.ceph.com 3 6 377 48 -382us[ -365us] +/- 36ms 2024-08-31T06:29:30.456 INFO:teuthology.orchestra.run.smithi155.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 45 +128us[ +128us] +/- 73ms 2024-08-31T06:29:30.456 INFO:teuthology.orchestra.run.smithi155.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 45 +872us[ +872us] +/- 89ms 2024-08-31T06:29:30.456 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-08-31T06:29:30.457 INFO:teuthology.orchestra.run.smithi184.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-31T06:29:30.457 INFO:teuthology.orchestra.run.smithi184.stdout:=============================================================================== 2024-08-31T06:29:30.457 INFO:teuthology.orchestra.run.smithi184.stdout:^* hv01.front.sepia.ceph.com 3 6 377 46 -404us[ -392us] +/- 36ms 2024-08-31T06:29:30.457 INFO:teuthology.orchestra.run.smithi184.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 46 +96us[ +108us] +/- 73ms 2024-08-31T06:29:30.458 INFO:teuthology.orchestra.run.smithi184.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 46 +851us[ +864us] +/- 89ms 2024-08-31T06:29:30.458 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-08-31T06:29:30.466 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: ntpq: command not found 2024-08-31T06:29:30.473 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-31T06:29:30.473 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2024-08-31T06:29:30.473 INFO:teuthology.orchestra.run.smithi016.stdout:^* hv01.front.sepia.ceph.com 3 6 377 44 -427us[ -431us] +/- 36ms 2024-08-31T06:29:30.473 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 111 +148us[ +50us] +/- 72ms 2024-08-31T06:29:30.473 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 45 +817us[ +814us] +/- 89ms 2024-08-31T06:29:30.473 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-08-31T06:29:30.474 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-08-31T06:29:30.485 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-08-31T06:29:30.486 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-08-31T06:29:30.539 DEBUG:teuthology.orchestra.run.smithi016:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-08-31T06:29:30.662 DEBUG:teuthology.orchestra.run.smithi155:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-08-31T06:29:30.718 DEBUG:teuthology.orchestra.run.smithi184:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-08-31T06:29:30.772 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-08-31T06:29:30.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-31T06:29:30.833 DEBUG:teuthology.orchestra.run.smithi155:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-08-31T06:29:30.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-31T06:29:30.860 DEBUG:teuthology.orchestra.run.smithi184:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-08-31T06:29:30.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-31T06:29:30.887 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-08-31T06:29:30.901 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-08-31T06:29:30.910 INFO:teuthology.task.internal:Duration was 2378.985225 seconds 2024-08-31T06:29:30.910 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-08-31T06:29:30.921 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-08-31T06:29:30.921 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-31T06:29:30.925 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-31T06:29:30.927 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-31T06:29:30.963 INFO:teuthology.orchestra.run.smithi155.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-31T06:29:30.966 INFO:teuthology.orchestra.run.smithi184.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-31T06:29:31.009 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-31T06:29:31.485 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-08-31T06:29:31.485 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi016.front.sepia.ceph.com 2024-08-31T06:29:31.485 DEBUG:teuthology.orchestra.run.smithi016:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-31T06:29:31.535 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi155.front.sepia.ceph.com 2024-08-31T06:29:31.536 DEBUG:teuthology.orchestra.run.smithi155:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-31T06:29:31.565 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi184.front.sepia.ceph.com 2024-08-31T06:29:31.565 DEBUG:teuthology.orchestra.run.smithi184:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-31T06:29:31.594 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-08-31T06:29:31.594 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-31T06:29:31.598 DEBUG:teuthology.orchestra.run.smithi155:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-31T06:29:31.609 DEBUG:teuthology.orchestra.run.smithi184:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-31T06:29:31.662 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-08-31T06:29:31.662 DEBUG:teuthology.orchestra.run.smithi016:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-31T06:29:31.666 DEBUG:teuthology.orchestra.run.smithi155:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-31T06:29:31.678 DEBUG:teuthology.orchestra.run.smithi184:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-31T06:29:33.124 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-08-31T06:29:33.136 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-08-31T06:29:33.136 DEBUG:teuthology.orchestra.run.smithi016:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-31T06:29:33.193 DEBUG:teuthology.orchestra.run.smithi155:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-31T06:29:33.223 DEBUG:teuthology.orchestra.run.smithi184:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-31T06:29:33.254 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-08-31T06:29:33.271 DEBUG:teuthology.orchestra.run.smithi016:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-31T06:29:33.276 DEBUG:teuthology.orchestra.run.smithi155:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-31T06:29:33.280 DEBUG:teuthology.orchestra.run.smithi184:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-31T06:29:33.303 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern = core 2024-08-31T06:29:33.322 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern = core 2024-08-31T06:29:33.343 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = core 2024-08-31T06:29:33.373 DEBUG:teuthology.orchestra.run.smithi016:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-31T06:29:33.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-31T06:29:33.407 DEBUG:teuthology.orchestra.run.smithi155:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-31T06:29:33.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-31T06:29:33.423 DEBUG:teuthology.orchestra.run.smithi184:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-31T06:29:33.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-31T06:29:33.441 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-08-31T06:29:33.454 INFO:teuthology.task.internal:Transferring archived files... 2024-08-31T06:29:33.456 DEBUG:teuthology.misc:Transferring archived files from smithi016:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-08-30_08:30:37-fs-wip-mchangir-testing-20240828.085847-main-debug-distro-default-smithi/7881377/remote/smithi016 2024-08-31T06:29:33.456 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-31T06:29:33.553 DEBUG:teuthology.misc:Transferring archived files from smithi155:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-08-30_08:30:37-fs-wip-mchangir-testing-20240828.085847-main-debug-distro-default-smithi/7881377/remote/smithi155 2024-08-31T06:29:33.553 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-31T06:29:33.589 DEBUG:teuthology.misc:Transferring archived files from smithi184:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-08-30_08:30:37-fs-wip-mchangir-testing-20240828.085847-main-debug-distro-default-smithi/7881377/remote/smithi184 2024-08-31T06:29:33.590 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-31T06:29:33.628 INFO:teuthology.task.internal:Removing archive directory... 2024-08-31T06:29:33.628 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-31T06:29:33.633 DEBUG:teuthology.orchestra.run.smithi155:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-31T06:29:33.635 DEBUG:teuthology.orchestra.run.smithi184:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-31T06:29:33.681 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-08-31T06:29:33.694 INFO:teuthology.task.internal:Not uploading archives. 2024-08-31T06:29:33.695 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-08-31T06:29:33.708 INFO:teuthology.task.internal:Tidying up after the test... 2024-08-31T06:29:33.708 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-31T06:29:33.712 DEBUG:teuthology.orchestra.run.smithi155:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-31T06:29:33.715 DEBUG:teuthology.orchestra.run.smithi184:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-31T06:29:33.729 INFO:teuthology.orchestra.run.smithi155.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 31 06:29 /home/ubuntu/cephtest 2024-08-31T06:29:33.739 INFO:teuthology.orchestra.run.smithi184.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 31 06:29 /home/ubuntu/cephtest 2024-08-31T06:29:33.743 INFO:teuthology.orchestra.run.smithi016.stdout: 393625 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 31 06:29 /home/ubuntu/cephtest 2024-08-31T06:29:33.745 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-08-31T06:29:33.757 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-08-31T06:29:33.785 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 8ad8b8b160987a14eb199bb67cc00e23a08d43f6 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/postgres}} duration: 2378.9852249622345 failure_reason: '"2024-08-31T06:10:54.098841+0000 mds.b (mds.0) 29 : cluster [WRN] Scrub error on inode 0x10000000019 (/volumes/qa/sv_1/ba5423bc-5c83-4d64-ae1c-ad8850110ce4/postgres/data/pg_stat_tmp) see mds.b log and `damage ls` output for details" in cluster log' flavor: default owner: scheduled_mchangir@teuthology success: false 2024-08-31T06:29:33.785 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-08-31T06:29:33.871 INFO:teuthology.run:FAIL